2024-03-29T08:01:01.207 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-29T08:01:01.207 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628113 --verbose --owner scheduled_vshankar@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} --name vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi -- /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628113/orig.config.yaml 2024-03-29T08:01:01.252 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-29T08:01:01.344 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628113 branch: wip-vshankar-testing-20240326.105515-reef description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} email: vshankar@redhat.com first_in_suite: false job_id: '7628113' kernel: flavor: default kdb: true sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-vshankar-testing-20240326.105515-reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240326.105515-reef sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 owner: scheduled_vshankar@teuthology priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7327 sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 sleep_before_teardown: 0 subset: 990/1024 suite: fs suite_branch: wip-vshankar-testing-20240326.105515-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyO35H5HPJGFuhHNFtjtIjlv5ApnHunOmuTYxW+bNF+3/oX9wx9k2O2ciaYaKvoLd7IlwtsfjApdTCDY5sDlsM= smithi073.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIfgdX6RoDeC9CUYoUy1+4Ab439iXukfjJUOSMT3kXSQs1Ev1THOAgjRZ7iAskt1D6wDonDlIfV1FXULpPxnirY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-28_11:21:59 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-29T08:01:01.344 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa; will attempt to use it 2024-03-29T08:01:01.345 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa/tasks 2024-03-29T08:01:01.345 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-29T08:01:01.347 INFO:teuthology.task.internal:Checking packages... 2024-03-29T08:01:01.372 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'd0e392d9aee635f2b65768c460c7a151ead7a5e2' 2024-03-29T08:01:01.372 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-29T08:01:01.372 INFO:teuthology.packaging:ref: None 2024-03-29T08:01:01.373 INFO:teuthology.packaging:tag: None 2024-03-29T08:01:01.373 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240326.105515-reef 2024-03-29T08:01:01.373 INFO:teuthology.packaging:sha1: d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:01:01.373 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-vshankar-testing-20240326.105515-reef 2024-03-29T08:01:01.518 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1015-gd0e392d9-1jammy 2024-03-29T08:01:01.520 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-29T08:01:01.526 INFO:teuthology.task.internal:no buildpackages task found 2024-03-29T08:01:01.526 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-29T08:01:01.533 INFO:teuthology.task.internal:Saving configuration 2024-03-29T08:01:01.550 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-29T08:01:01.556 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T08:01:01.578 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628113', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 07:55:22.108927', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHrnJX0DFASJjAJabqmk8k0OBmRKF6Euff1zw2DBtDNV'} 2024-03-29T08:01:01.595 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628113', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 07:55:22.109895', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFwJuVJBYZxfFVvREkX1DaHv9eYBmfqgjgh/uS8HP6ZX'} 2024-03-29T08:01:01.595 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-29T08:01:01.601 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-03-29T08:01:01.601 INFO:teuthology.task.internal:roles: ubuntu@smithi073.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-03-29T08:01:01.601 INFO:teuthology.run_tasks:Running task console_log... 2024-03-29T08:01:01.670 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc1a391d3a0>, signals=[15]) 2024-03-29T08:01:01.670 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-29T08:01:01.677 INFO:teuthology.task.internal:Opening connections... 2024-03-29T08:01:01.678 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:01:01.680 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:01:01.764 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:01:01.765 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:01:01.844 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-29T08:01:01.851 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-03-29T08:01:01.856 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-03-29T08:01:01.857 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-03-29T08:01:01.903 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-29T08:01:01.903 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="Ubuntu" 2024-03-29T08:01:01.903 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="22.04" 2024-03-29T08:01:01.903 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-29T08:01:01.903 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_CODENAME=jammy 2024-03-29T08:01:01.903 INFO:teuthology.orchestra.run.smithi046.stdout:ID=ubuntu 2024-03-29T08:01:01.903 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE=debian 2024-03-29T08:01:01.903 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-29T08:01:01.903 INFO:teuthology.orchestra.run.smithi046.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-29T08:01:01.903 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-29T08:01:01.903 INFO:teuthology.orchestra.run.smithi046.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-29T08:01:01.903 INFO:teuthology.orchestra.run.smithi046.stdout:UBUNTU_CODENAME=jammy 2024-03-29T08:01:01.904 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-03-29T08:01:01.922 DEBUG:teuthology.orchestra.run.smithi073:> uname -m 2024-03-29T08:01:01.927 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64 2024-03-29T08:01:01.927 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release 2024-03-29T08:01:01.973 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-29T08:01:01.973 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="Ubuntu" 2024-03-29T08:01:01.973 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="22.04" 2024-03-29T08:01:01.973 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-29T08:01:01.973 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_CODENAME=jammy 2024-03-29T08:01:01.973 INFO:teuthology.orchestra.run.smithi073.stdout:ID=ubuntu 2024-03-29T08:01:01.973 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE=debian 2024-03-29T08:01:01.973 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-29T08:01:01.973 INFO:teuthology.orchestra.run.smithi073.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-29T08:01:01.973 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-29T08:01:01.973 INFO:teuthology.orchestra.run.smithi073.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-29T08:01:01.974 INFO:teuthology.orchestra.run.smithi073.stdout:UBUNTU_CODENAME=jammy 2024-03-29T08:01:01.974 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2024-03-29T08:01:01.997 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-29T08:01:02.006 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-29T08:01:02.011 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-29T08:01:02.012 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-03-29T08:01:02.014 DEBUG:teuthology.orchestra.run.smithi073:> test '!' -e /home/ubuntu/cephtest 2024-03-29T08:01:02.020 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-29T08:01:02.026 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-29T08:01:02.026 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-03-29T08:01:02.060 DEBUG:teuthology.orchestra.run.smithi073:> test -z $(ls -A /var/lib/ceph) 2024-03-29T08:01:02.080 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-29T08:01:02.131 INFO:teuthology.run_tasks:Running task kernel... 2024-03-29T08:01:02.146 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'} 2024-03-29T08:01:02.147 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, timeout 300 2024-03-29T08:01:02.147 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-29T08:01:02.147 DEBUG:teuthology.orchestra.run.smithi073:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-29T08:01:02.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:01:02.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:01:02.153 DEBUG:teuthology.task.kernel:sha1 for mon.b is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T08:01:02.153 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-29T08:01:02.153 DEBUG:teuthology.orchestra.run.smithi073:> uname -r 2024-03-29T08:01:02.153 DEBUG:teuthology.task.kernel:sha1 for mon.a is 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T08:01:02.154 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "1df0c7e8a2354c42d738e5454ba83ea7332deac2"... 2024-03-29T08:01:02.154 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-03-29T08:01:02.199 INFO:teuthology.orchestra.run.smithi046.stdout:5.15.0-56-generic 2024-03-29T08:01:02.199 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T08:01:02.199 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-29T08:01:02.200 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T08:01:02.203 INFO:teuthology.orchestra.run.smithi073.stdout:5.15.0-56-generic 2024-03-29T08:01:02.204 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T08:01:02.204 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-03-29T08:01:02.204 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T08:01:02.356 INFO:teuthology.task.kernel:Installing linux-firmware on mon.b... 2024-03-29T08:01:02.356 DEBUG:teuthology.orchestra.run.smithi073:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-03-29T08:01:02.375 INFO:teuthology.orchestra.run.smithi073.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-03-29T08:01:02.421 INFO:teuthology.task.kernel:Installing linux-firmware on mon.a... 2024-03-29T08:01:02.421 DEBUG:teuthology.orchestra.run.smithi046:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-03-29T08:01:02.438 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-03-29T08:01:02.548 INFO:teuthology.orchestra.run.smithi073.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-03-29T08:01:02.548 INFO:teuthology.orchestra.run.smithi073.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-03-29T08:01:02.548 INFO:teuthology.orchestra.run.smithi073.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-03-29T08:01:02.548 INFO:teuthology.orchestra.run.smithi073.stderr:hint: 2024-03-29T08:01:02.548 INFO:teuthology.orchestra.run.smithi073.stderr:hint: git config --global init.defaultBranch 2024-03-29T08:01:02.548 INFO:teuthology.orchestra.run.smithi073.stderr:hint: 2024-03-29T08:01:02.548 INFO:teuthology.orchestra.run.smithi073.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-03-29T08:01:02.549 INFO:teuthology.orchestra.run.smithi073.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-03-29T08:01:02.549 INFO:teuthology.orchestra.run.smithi073.stderr:hint: 2024-03-29T08:01:02.549 INFO:teuthology.orchestra.run.smithi073.stderr:hint: git branch -m 2024-03-29T08:01:02.550 INFO:teuthology.orchestra.run.smithi073.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-03-29T08:01:02.552 DEBUG:teuthology.orchestra.run.smithi073:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-03-29T08:01:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-03-29T08:01:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-03-29T08:01:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-03-29T08:01:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:hint: 2024-03-29T08:01:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:hint: git config --global init.defaultBranch 2024-03-29T08:01:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:hint: 2024-03-29T08:01:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-03-29T08:01:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-03-29T08:01:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:hint: 2024-03-29T08:01:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:hint: git branch -m 2024-03-29T08:01:02.570 INFO:teuthology.orchestra.run.smithi046.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-03-29T08:01:02.572 DEBUG:teuthology.orchestra.run.smithi046:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-03-29T08:01:02.615 DEBUG:teuthology.orchestra.run.smithi073:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-03-29T08:01:02.638 DEBUG:teuthology.orchestra.run.smithi046:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-03-29T08:01:02.668 DEBUG:teuthology.orchestra.run.smithi073:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-03-29T08:01:02.696 DEBUG:teuthology.orchestra.run.smithi046:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-03-29T08:03:08.068 INFO:teuthology.orchestra.run.smithi046.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-03-29T08:03:08.069 INFO:teuthology.orchestra.run.smithi046.stderr: * [new branch] main -> origin/main 2024-03-29T08:03:08.104 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190312 -> 20190312 2024-03-29T08:03:08.104 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190416 -> 20190416 2024-03-29T08:03:08.104 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190514 -> 20190514 2024-03-29T08:03:08.104 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190618 -> 20190618 2024-03-29T08:03:08.104 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190717 -> 20190717 2024-03-29T08:03:08.104 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190815 -> 20190815 2024-03-29T08:03:08.104 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20190923 -> 20190923 2024-03-29T08:03:08.104 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20191022 -> 20191022 2024-03-29T08:03:08.104 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20191215 -> 20191215 2024-03-29T08:03:08.104 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200122 -> 20200122 2024-03-29T08:03:08.104 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200316 -> 20200316 2024-03-29T08:03:08.105 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200421 -> 20200421 2024-03-29T08:03:08.108 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200519 -> 20200519 2024-03-29T08:03:08.108 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200619 -> 20200619 2024-03-29T08:03:08.108 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200721 -> 20200721 2024-03-29T08:03:08.108 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200817 -> 20200817 2024-03-29T08:03:08.108 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20200918 -> 20200918 2024-03-29T08:03:08.108 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20201022 -> 20201022 2024-03-29T08:03:08.108 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20201118 -> 20201118 2024-03-29T08:03:08.109 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20201218 -> 20201218 2024-03-29T08:03:08.109 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210208 -> 20210208 2024-03-29T08:03:08.109 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210315 -> 20210315 2024-03-29T08:03:08.109 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210511 -> 20210511 2024-03-29T08:03:08.109 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210716 -> 20210716 2024-03-29T08:03:08.109 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210818 -> 20210818 2024-03-29T08:03:08.109 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20210919 -> 20210919 2024-03-29T08:03:08.109 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20211027 -> 20211027 2024-03-29T08:03:08.109 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20211216 -> 20211216 2024-03-29T08:03:08.109 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220209 -> 20220209 2024-03-29T08:03:08.109 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220310 -> 20220310 2024-03-29T08:03:08.109 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220411 -> 20220411 2024-03-29T08:03:08.110 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220509 -> 20220509 2024-03-29T08:03:08.110 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220610 -> 20220610 2024-03-29T08:03:08.110 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220708 -> 20220708 2024-03-29T08:03:08.110 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220815 -> 20220815 2024-03-29T08:03:08.110 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20220913 -> 20220913 2024-03-29T08:03:08.110 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20221012 -> 20221012 2024-03-29T08:03:08.110 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20221109 -> 20221109 2024-03-29T08:03:08.110 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20221214 -> 20221214 2024-03-29T08:03:08.110 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230117 -> 20230117 2024-03-29T08:03:08.110 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230210 -> 20230210 2024-03-29T08:03:08.112 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230310 -> 20230310 2024-03-29T08:03:08.112 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230404 -> 20230404 2024-03-29T08:03:08.112 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230515 -> 20230515 2024-03-29T08:03:08.112 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230625 -> 20230625 2024-03-29T08:03:08.112 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230804 -> 20230804 2024-03-29T08:03:08.112 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20230919 -> 20230919 2024-03-29T08:03:08.112 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20231030 -> 20231030 2024-03-29T08:03:08.112 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20231111 -> 20231111 2024-03-29T08:03:08.113 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20231211 -> 20231211 2024-03-29T08:03:08.113 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20240115 -> 20240115 2024-03-29T08:03:08.113 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20240220 -> 20240220 2024-03-29T08:03:08.113 INFO:teuthology.orchestra.run.smithi046.stderr: * [new tag] 20240312 -> 20240312 2024-03-29T08:03:12.547 INFO:teuthology.orchestra.run.smithi046.stderr:Updating files: 48% (1754/3598) Updating files: 49% (1764/3598) Updating files: 50% (1799/3598) Updating files: 51% (1835/3598) Updating files: 52% (1871/3598) Updating files: 53% (1907/3598) Updating files: 54% (1943/3598) Updating files: 55% (1979/3598) Updating files: 56% (2015/3598) Updating files: 57% (2051/3598) Updating files: 58% (2087/3598) Updating files: 59% (2123/3598) Updating files: 60% (2159/3598) Updating files: 61% (2195/3598) Updating files: 61% (2224/3598) Updating files: 62% (2231/3598) Updating files: 63% (2267/3598) Updating files: 64% (2303/3598) Updating files: 65% (2339/3598) Updating files: 66% (2375/3598) Updating files: 67% (2411/3598) Updating files: 68% (2447/3598) Updating files: 69% (2483/3598) Updating files: 70% (2519/3598) Updating files: 71% (2555/3598) Updating files: 71% (2580/3598) Updating files: 72% (2591/3598) Updating files: 73% (2627/3598) Updating files: 74% (2663/3598) Updating files: 75% (2699/3598) Updating files: 76% (2735/3598) Updating files: 77% (2771/3598) Updating files: 78% (2807/3598) Updating files: 79% (2843/3598) Updating files: 80% (2879/3598) Updating files: 81% (2915/3598) Updating files: 82% (2951/3598) Updating files: 83% (2987/3598) Updating files: 83% (3008/3598) Updating files: 84% (3023/3598) Updating files: 85% (3059/3598) Updating files: 86% (3095/3598) Updating files: 87% (3131/3598) Updating files: 88% (3167/3598) Updating files: 89% (3203/3598) Updating files: 90% (3239/3598) Updating files: 91% (3275/3598) Updating files: 92% (3311/3598) Updating files: 93% (3347/3598) Updating files: 94% (3383/3598) Updating files: 95% (3419/3598) Updating files: 96% (3455/3598) Updating files: 97% (3491/3598) Updating files: 98% (3527/3598) Updating files: 99% (3563/3598) Updating files: 100% (3598/3598) Updating files: 100% (3598/3598), done. 2024-03-29T08:03:12.553 INFO:teuthology.orchestra.run.smithi046.stdout:HEAD is now at e068ccf0 Merge branch 'robot/pr-0-1711100280' into 'main' 2024-03-29T08:03:12.575 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-29T08:03:12.575 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T08:03:12.726 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g1df0c7e8a235/ 2024-03-29T08:03:12.727 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /tmp/linux-image.deb && echo linux-image-6.8.0-g1df0c7e8a235_6.8.0-g1df0c7e8a235-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g1df0c7e8a235/ --input-file=- 2024-03-29T08:03:12.729 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi046.front.sepia.ceph.com to complete... 2024-03-29T08:03:14.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29 08:03:14 URL:https://4.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g1df0c7e8a235/linux-image-6.8.0-g1df0c7e8a235_6.8.0-g1df0c7e8a235-1_amd64.deb [39278344/39278344] -> "/tmp/linux-image.deb" [1] 2024-03-29T08:03:14.406 INFO:teuthology.orchestra.run.smithi046.stderr:FINISHED --2024-03-29 08:03:14-- 2024-03-29T08:03:14.406 INFO:teuthology.orchestra.run.smithi046.stderr:Total wall clock time: 1.7s 2024-03-29T08:03:14.406 INFO:teuthology.orchestra.run.smithi046.stderr:Downloaded: 1 files, 37M in 1.5s (25.8 MB/s) 2024-03-29T08:03:14.410 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.a... 2024-03-29T08:03:14.410 DEBUG:teuthology.orchestra.run.smithi046:> sudo dpkg -i /tmp/linux-image.deb 2024-03-29T08:03:14.656 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-image-6.8.0-g1df0c7e8a235. 2024-03-29T08:03:16.637 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-03-29T08:03:16.642 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-03-29T08:03:16.876 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-image-6.8.0-g1df0c7e8a235 (6.8.0-g1df0c7e8a235-1) ... 2024-03-29T08:03:17.087 INFO:teuthology.orchestra.run.smithi073.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-03-29T08:03:17.088 INFO:teuthology.orchestra.run.smithi073.stderr: * [new branch] main -> origin/main 2024-03-29T08:03:17.113 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20190312 -> 20190312 2024-03-29T08:03:17.113 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20190416 -> 20190416 2024-03-29T08:03:17.114 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20190514 -> 20190514 2024-03-29T08:03:17.114 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20190618 -> 20190618 2024-03-29T08:03:17.114 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20190717 -> 20190717 2024-03-29T08:03:17.114 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20190815 -> 20190815 2024-03-29T08:03:17.114 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20190923 -> 20190923 2024-03-29T08:03:17.114 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20191022 -> 20191022 2024-03-29T08:03:17.116 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20191215 -> 20191215 2024-03-29T08:03:17.116 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20200122 -> 20200122 2024-03-29T08:03:17.116 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20200316 -> 20200316 2024-03-29T08:03:17.116 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20200421 -> 20200421 2024-03-29T08:03:17.116 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20200519 -> 20200519 2024-03-29T08:03:17.116 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20200619 -> 20200619 2024-03-29T08:03:17.116 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20200721 -> 20200721 2024-03-29T08:03:17.117 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20200817 -> 20200817 2024-03-29T08:03:17.117 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20200918 -> 20200918 2024-03-29T08:03:17.117 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20201022 -> 20201022 2024-03-29T08:03:17.117 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20201118 -> 20201118 2024-03-29T08:03:17.117 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20201218 -> 20201218 2024-03-29T08:03:17.117 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20210208 -> 20210208 2024-03-29T08:03:17.117 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20210315 -> 20210315 2024-03-29T08:03:17.117 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20210511 -> 20210511 2024-03-29T08:03:17.117 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20210716 -> 20210716 2024-03-29T08:03:17.117 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20210818 -> 20210818 2024-03-29T08:03:17.117 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20210919 -> 20210919 2024-03-29T08:03:17.117 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20211027 -> 20211027 2024-03-29T08:03:17.120 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20211216 -> 20211216 2024-03-29T08:03:17.120 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20220209 -> 20220209 2024-03-29T08:03:17.120 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20220310 -> 20220310 2024-03-29T08:03:17.120 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20220411 -> 20220411 2024-03-29T08:03:17.120 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20220509 -> 20220509 2024-03-29T08:03:17.120 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20220610 -> 20220610 2024-03-29T08:03:17.120 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20220708 -> 20220708 2024-03-29T08:03:17.120 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20220815 -> 20220815 2024-03-29T08:03:17.120 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20220913 -> 20220913 2024-03-29T08:03:17.121 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20221012 -> 20221012 2024-03-29T08:03:17.121 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20221109 -> 20221109 2024-03-29T08:03:17.121 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20221214 -> 20221214 2024-03-29T08:03:17.121 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20230117 -> 20230117 2024-03-29T08:03:17.121 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20230210 -> 20230210 2024-03-29T08:03:17.121 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20230310 -> 20230310 2024-03-29T08:03:17.121 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20230404 -> 20230404 2024-03-29T08:03:17.121 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20230515 -> 20230515 2024-03-29T08:03:17.121 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20230625 -> 20230625 2024-03-29T08:03:17.121 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20230804 -> 20230804 2024-03-29T08:03:17.121 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20230919 -> 20230919 2024-03-29T08:03:17.121 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20231030 -> 20231030 2024-03-29T08:03:17.121 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20231111 -> 20231111 2024-03-29T08:03:17.121 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20231211 -> 20231211 2024-03-29T08:03:17.121 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20240115 -> 20240115 2024-03-29T08:03:17.122 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20240220 -> 20240220 2024-03-29T08:03:17.122 INFO:teuthology.orchestra.run.smithi073.stderr: * [new tag] 20240312 -> 20240312 2024-03-29T08:03:20.205 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-image-6.8.0-g1df0c7e8a235 (6.8.0-g1df0c7e8a235-1) ... 2024-03-29T08:03:20.341 INFO:teuthology.orchestra.run.smithi046.stderr:update-initramfs: Generating /boot/initrd.img-6.8.0-g1df0c7e8a235 2024-03-29T08:03:21.314 INFO:teuthology.orchestra.run.smithi073.stderr:Updating files: 51% (1850/3598) Updating files: 52% (1871/3598) Updating files: 53% (1907/3598) Updating files: 54% (1943/3598) Updating files: 55% (1979/3598) Updating files: 56% (2015/3598) Updating files: 57% (2051/3598) Updating files: 58% (2087/3598) Updating files: 59% (2123/3598) Updating files: 60% (2159/3598) Updating files: 61% (2195/3598) Updating files: 62% (2231/3598) Updating files: 63% (2267/3598) Updating files: 64% (2303/3598) Updating files: 64% (2327/3598) Updating files: 65% (2339/3598) Updating files: 66% (2375/3598) Updating files: 67% (2411/3598) Updating files: 68% (2447/3598) Updating files: 69% (2483/3598) Updating files: 70% (2519/3598) Updating files: 71% (2555/3598) Updating files: 72% (2591/3598) Updating files: 73% (2627/3598) Updating files: 74% (2663/3598) Updating files: 75% (2699/3598) Updating files: 76% (2735/3598) Updating files: 77% (2771/3598) Updating files: 77% (2785/3598) Updating files: 78% (2807/3598) Updating files: 79% (2843/3598) Updating files: 80% (2879/3598) Updating files: 81% (2915/3598) Updating files: 82% (2951/3598) Updating files: 83% (2987/3598) Updating files: 84% (3023/3598) Updating files: 85% (3059/3598) Updating files: 86% (3095/3598) Updating files: 87% (3131/3598) Updating files: 88% (3167/3598) Updating files: 89% (3203/3598) Updating files: 89% (3206/3598) Updating files: 90% (3239/3598) Updating files: 91% (3275/3598) Updating files: 92% (3311/3598) Updating files: 93% (3347/3598) Updating files: 94% (3383/3598) Updating files: 95% (3419/3598) Updating files: 96% (3455/3598) Updating files: 97% (3491/3598) Updating files: 98% (3527/3598) Updating files: 99% (3563/3598) Updating files: 100% (3598/3598) Updating files: 100% (3598/3598), done. 2024-03-29T08:03:21.319 INFO:teuthology.orchestra.run.smithi073.stdout:HEAD is now at e068ccf0 Merge branch 'robot/pr-0-1711100280' into 'main' 2024-03-29T08:03:21.340 INFO:teuthology.task.kernel:Downloading kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-29T08:03:21.340 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1df0c7e8a2354c42d738e5454ba83ea7332deac2 2024-03-29T08:03:21.486 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g1df0c7e8a235/ 2024-03-29T08:03:21.486 DEBUG:teuthology.orchestra.run.smithi073:> rm -f /tmp/linux-image.deb && echo linux-image-6.8.0-g1df0c7e8a235_6.8.0-g1df0c7e8a235-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g1df0c7e8a235/ --input-file=- 2024-03-29T08:03:21.489 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi073.front.sepia.ceph.com to complete... 2024-03-29T08:03:23.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29 08:03:23 URL:https://4.chacra.ceph.com/r/kernel/testing/1df0c7e8a2354c42d738e5454ba83ea7332deac2/ubuntu/jammy/flavors/default/pool/main/l/linux-6.8.0-g1df0c7e8a235/linux-image-6.8.0-g1df0c7e8a235_6.8.0-g1df0c7e8a235-1_amd64.deb [39278344/39278344] -> "/tmp/linux-image.deb" [1] 2024-03-29T08:03:23.462 INFO:teuthology.orchestra.run.smithi073.stderr:FINISHED --2024-03-29 08:03:23-- 2024-03-29T08:03:23.462 INFO:teuthology.orchestra.run.smithi073.stderr:Total wall clock time: 2.0s 2024-03-29T08:03:23.462 INFO:teuthology.orchestra.run.smithi073.stderr:Downloaded: 1 files, 37M in 1.8s (21.3 MB/s) 2024-03-29T08:03:23.466 INFO:teuthology.task.kernel:Installing kernel 1df0c7e8a2354c42d738e5454ba83ea7332deac2 on mon.b... 2024-03-29T08:03:23.467 DEBUG:teuthology.orchestra.run.smithi073:> sudo dpkg -i /tmp/linux-image.deb 2024-03-29T08:03:23.710 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package linux-image-6.8.0-g1df0c7e8a235. 2024-03-29T08:03:25.769 INFO:teuthology.orchestra.run.smithi073.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-03-29T08:03:25.773 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-03-29T08:03:25.914 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking linux-image-6.8.0-g1df0c7e8a235 (6.8.0-g1df0c7e8a235-1) ... 2024-03-29T08:03:29.779 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up linux-image-6.8.0-g1df0c7e8a235 (6.8.0-g1df0c7e8a235-1) ... 2024-03-29T08:03:29.885 INFO:teuthology.orchestra.run.smithi073.stderr:update-initramfs: Generating /boot/initrd.img-6.8.0-g1df0c7e8a235 2024-03-29T08:03:42.490 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub' 2024-03-29T08:03:42.507 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-29T08:03:42.539 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-03-29T08:03:42.864 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-6.8.0-g1df0c7e8a235 2024-03-29T08:03:42.885 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-6.8.0-g1df0c7e8a235 2024-03-29T08:03:43.276 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-29T08:03:43.288 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-29T08:03:43.354 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-29T08:03:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-29T08:03:43.517 INFO:teuthology.orchestra.run.smithi046.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-29T08:03:43.517 INFO:teuthology.orchestra.run.smithi046.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-29T08:03:43.517 INFO:teuthology.orchestra.run.smithi046.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-29T08:03:43.555 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-03-29T08:03:43.722 DEBUG:teuthology.orchestra.run.smithi046:> dpkg-deb -c /tmp/linux-image.deb 2024-03-29T08:03:43.730 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./ 2024-03-29T08:03:43.730 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./boot/ 2024-03-29T08:03:43.730 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 4474334 2024-03-25 03:29 ./boot/System.map-6.8.0-g1df0c7e8a235 2024-03-29T08:03:43.743 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 170899 2024-03-25 03:29 ./boot/config-6.8.0-g1df0c7e8a235 2024-03-29T08:03:43.744 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11551744 2024-03-25 03:29 ./boot/vmlinuz-6.8.0-g1df0c7e8a235 2024-03-29T08:03:43.764 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/ 2024-03-29T08:03:43.764 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/ 2024-03-29T08:03:43.764 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/postinst.d/ 2024-03-29T08:03:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/postrm.d/ 2024-03-29T08:03:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/preinst.d/ 2024-03-29T08:03:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/prerm.d/ 2024-03-29T08:03:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/ 2024-03-29T08:03:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/ 2024-03-29T08:03:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/ 2024-03-29T08:03:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/ 2024-03-29T08:03:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/ 2024-03-29T08:03:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/ 2024-03-29T08:03:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ 2024-03-29T08:03:43.765 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 490441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/aesni-intel.ko 2024-03-29T08:03:43.767 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-29T08:03:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-29T08:03:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-29T08:03:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 93073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-29T08:03:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 106689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-29T08:03:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 152137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-29T08:03:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-29T08:03:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-29T08:03:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 83849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-29T08:03:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-29T08:03:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-29T08:03:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-29T08:03:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-29T08:03:43.768 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-29T08:03:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-29T08:03:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-29T08:03:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-29T08:03:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-29T08:03:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-29T08:03:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/ 2024-03-29T08:03:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/ 2024-03-29T08:03:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/ 2024-03-29T08:03:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-29T08:03:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpuid.ko 2024-03-29T08:03:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/msr.ko 2024-03-29T08:03:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/ 2024-03-29T08:03:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 354225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-29T08:03:43.769 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 886129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-29T08:03:43.770 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2294793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-29T08:03:43.786 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ 2024-03-29T08:03:43.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/af_alg.ko 2024-03-29T08:03:43.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/algif_hash.ko 2024-03-29T08:03:43.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/algif_skcipher.ko 2024-03-29T08:03:43.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-29T08:03:43.786 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-29T08:03:43.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-29T08:03:43.787 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/asymmetric_keys/ 2024-03-29T08:03:43.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-03-29T08:03:43.787 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/ 2024-03-29T08:03:43.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-29T08:03:43.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-29T08:03:43.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-29T08:03:43.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-29T08:03:43.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-29T08:03:43.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-29T08:03:43.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-29T08:03:43.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-29T08:03:43.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-29T08:03:43.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-29T08:03:43.787 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-29T08:03:43.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-29T08:03:43.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-29T08:03:43.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-29T08:03:43.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-29T08:03:43.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-29T08:03:43.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-29T08:03:43.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-29T08:03:43.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cryptd.ko 2024-03-29T08:03:43.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-29T08:03:43.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_simd.ko 2024-03-29T08:03:43.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-29T08:03:43.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-29T08:03:43.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-29T08:03:43.788 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-29T08:03:43.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-29T08:03:43.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-29T08:03:43.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-29T08:03:43.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-29T08:03:43.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lrw.ko 2024-03-29T08:03:43.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-29T08:03:43.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-29T08:03:43.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-29T08:03:43.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-29T08:03:43.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-29T08:03:43.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-29T08:03:43.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-29T08:03:43.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-29T08:03:43.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-29T08:03:43.789 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 143025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-29T08:03:43.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-29T08:03:43.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-29T08:03:43.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-29T08:03:43.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-29T08:03:43.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-29T08:03:43.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-29T08:03:43.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-29T08:03:43.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xts.ko 2024-03-29T08:03:43.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-29T08:03:43.790 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ 2024-03-29T08:03:43.790 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ 2024-03-29T08:03:43.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_extlog.ko 2024-03-29T08:03:43.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-29T08:03:43.790 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-29T08:03:43.790 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/apei/ 2024-03-29T08:03:43.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/apei/einj.ko 2024-03-29T08:03:43.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-29T08:03:43.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-29T08:03:43.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-29T08:03:43.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-29T08:03:43.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 104417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-29T08:03:43.791 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ 2024-03-29T08:03:43.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-29T08:03:43.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci.ko 2024-03-29T08:03:43.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-29T08:03:43.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 88305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci.ko 2024-03-29T08:03:43.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-29T08:03:43.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-29T08:03:43.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-29T08:03:43.791 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-29T08:03:43.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-29T08:03:43.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-29T08:03:43.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-29T08:03:43.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-29T08:03:43.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-29T08:03:43.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-29T08:03:43.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-29T08:03:43.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-29T08:03:43.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-29T08:03:43.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-29T08:03:43.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-29T08:03:43.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-29T08:03:43.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-29T08:03:43.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-29T08:03:43.792 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_legacy.ko 2024-03-29T08:03:43.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-29T08:03:43.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-29T08:03:43.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-29T08:03:43.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-29T08:03:43.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-29T08:03:43.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-29T08:03:43.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-29T08:03:43.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-29T08:03:43.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-29T08:03:43.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-29T08:03:43.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-29T08:03:43.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-29T08:03:43.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-29T08:03:43.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_radisys.ko 2024-03-29T08:03:43.793 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-29T08:03:43.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rz1000.ko 2024-03-29T08:03:43.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-29T08:03:43.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-29T08:03:43.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-29T08:03:43.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-29T08:03:43.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-29T08:03:43.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-29T08:03:43.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-29T08:03:43.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-29T08:03:43.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 83249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-29T08:03:43.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-29T08:03:43.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-29T08:03:43.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-29T08:03:43.794 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-29T08:03:43.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-29T08:03:43.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-29T08:03:43.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-29T08:03:43.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-29T08:03:43.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-29T08:03:43.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-29T08:03:43.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-29T08:03:43.795 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ 2024-03-29T08:03:43.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-29T08:03:43.795 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/ 2024-03-29T08:03:43.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 124385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-29T08:03:43.795 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/ 2024-03-29T08:03:43.795 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/ 2024-03-29T08:03:43.795 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 113801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-29T08:03:43.795 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/ 2024-03-29T08:03:43.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 964745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-29T08:03:43.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 169697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-29T08:03:43.796 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/ 2024-03-29T08:03:43.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 94209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-29T08:03:43.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 116329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-29T08:03:43.796 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/ 2024-03-29T08:03:43.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 110321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-29T08:03:43.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 98185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-29T08:03:43.796 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 263633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-29T08:03:43.839 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/ 2024-03-29T08:03:43.839 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 98105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-29T08:03:43.840 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/ 2024-03-29T08:03:43.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-29T08:03:43.840 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ 2024-03-29T08:03:43.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/applicom.ko 2024-03-29T08:03:43.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-29T08:03:43.840 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/ 2024-03-29T08:03:43.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-29T08:03:43.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-29T08:03:43.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-29T08:03:43.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-29T08:03:43.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-29T08:03:43.840 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ 2024-03-29T08:03:43.840 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-29T08:03:43.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 148513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-29T08:03:43.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-29T08:03:43.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 139305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-29T08:03:43.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-29T08:03:43.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-29T08:03:43.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-29T08:03:43.841 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/ 2024-03-29T08:03:43.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 84001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-29T08:03:43.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/nvram.ko 2024-03-29T08:03:43.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-29T08:03:43.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-29T08:03:43.841 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/ 2024-03-29T08:03:43.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-29T08:03:43.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-03-29T08:03:43.841 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-03-29T08:03:43.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-03-29T08:03:43.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-29T08:03:43.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-29T08:03:43.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-03-29T08:03:43.842 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/xillybus/ 2024-03-29T08:03:43.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/xillybus/xillybus_class.ko 2024-03-29T08:03:43.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/xillybus/xillybus_core.ko 2024-03-29T08:03:43.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-03-29T08:03:43.842 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/ 2024-03-29T08:03:43.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/clk-palmas.ko 2024-03-29T08:03:43.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/clk-si5351.ko 2024-03-29T08:03:43.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/clk-twl6040.ko 2024-03-29T08:03:43.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/clk-wm831x.ko 2024-03-29T08:03:43.842 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/ 2024-03-29T08:03:43.842 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-29T08:03:43.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-29T08:03:43.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-29T08:03:43.843 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ 2024-03-29T08:03:43.843 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ 2024-03-29T08:03:43.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-29T08:03:43.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 246777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-29T08:03:43.843 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/ 2024-03-29T08:03:43.843 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/ 2024-03-29T08:03:43.843 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/ 2024-03-29T08:03:43.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 570217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-29T08:03:43.843 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-03-29T08:03:43.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-29T08:03:43.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-29T08:03:43.843 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-29T08:03:43.844 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/ 2024-03-29T08:03:43.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-29T08:03:43.844 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/ 2024-03-29T08:03:43.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-29T08:03:43.844 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ 2024-03-29T08:03:43.844 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/ 2024-03-29T08:03:43.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-29T08:03:43.844 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ 2024-03-29T08:03:43.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 122793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-29T08:03:43.844 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ 2024-03-29T08:03:43.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-29T08:03:43.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-29T08:03:43.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-29T08:03:43.844 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-29T08:03:43.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-29T08:03:43.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-29T08:03:43.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-29T08:03:43.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-29T08:03:43.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-29T08:03:43.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-29T08:03:43.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-29T08:03:43.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-29T08:03:43.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-29T08:03:43.845 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/ 2024-03-29T08:03:43.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-gpio.ko 2024-03-29T08:03:43.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-max14577.ko 2024-03-29T08:03:43.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-max77693.ko 2024-03-29T08:03:43.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-max8997.ko 2024-03-29T08:03:43.845 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-palmas.ko 2024-03-29T08:03:43.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-rt8973a.ko 2024-03-29T08:03:43.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-sm5502.ko 2024-03-29T08:03:43.846 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/ 2024-03-29T08:03:43.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/dmi-sysfs.ko 2024-03-29T08:03:43.846 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/efi/ 2024-03-29T08:03:43.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/efi/efi-pstore.ko 2024-03-29T08:03:43.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-29T08:03:43.846 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/ 2024-03-29T08:03:43.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-adp5520.ko 2024-03-29T08:03:43.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-amd8111.ko 2024-03-29T08:03:43.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-arizona.ko 2024-03-29T08:03:43.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-da9052.ko 2024-03-29T08:03:43.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-da9055.ko 2024-03-29T08:03:43.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-dln2.ko 2024-03-29T08:03:43.846 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 77745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-f7188x.ko 2024-03-29T08:03:43.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-generic.ko 2024-03-29T08:03:43.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-29T08:03:43.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-03-29T08:03:43.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-kempld.ko 2024-03-29T08:03:43.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-lp3943.ko 2024-03-29T08:03:43.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-max7300.ko 2024-03-29T08:03:43.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-max730x.ko 2024-03-29T08:03:43.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-max732x.ko 2024-03-29T08:03:43.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-03-29T08:03:43.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-pca953x.ko 2024-03-29T08:03:43.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-pcf857x.ko 2024-03-29T08:03:43.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-rdc321x.ko 2024-03-29T08:03:43.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-sch.ko 2024-03-29T08:03:43.847 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-sch311x.ko 2024-03-29T08:03:43.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-tps65912.ko 2024-03-29T08:03:43.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-twl4030.ko 2024-03-29T08:03:43.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-twl6040.ko 2024-03-29T08:03:43.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-29T08:03:43.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-vx855.ko 2024-03-29T08:03:43.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-wm831x.ko 2024-03-29T08:03:43.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-wm8350.ko 2024-03-29T08:03:43.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-wm8994.ko 2024-03-29T08:03:43.848 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/ 2024-03-29T08:03:43.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-a4tech.ko 2024-03-29T08:03:43.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-apple.ko 2024-03-29T08:03:43.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-29T08:03:43.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-29T08:03:43.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-29T08:03:43.848 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-belkin.ko 2024-03-29T08:03:43.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-cherry.ko 2024-03-29T08:03:43.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-chicony.ko 2024-03-29T08:03:43.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-cp2112.ko 2024-03-29T08:03:43.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-cypress.ko 2024-03-29T08:03:43.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-29T08:03:43.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-29T08:03:43.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-29T08:03:43.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-29T08:03:43.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ezkey.ko 2024-03-29T08:03:43.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-29T08:03:43.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-generic.ko 2024-03-29T08:03:43.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-29T08:03:43.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-29T08:03:43.849 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-29T08:03:43.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-29T08:03:43.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-29T08:03:43.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-29T08:03:43.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-29T08:03:43.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kensington.ko 2024-03-29T08:03:43.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-29T08:03:43.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-29T08:03:43.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-29T08:03:43.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-29T08:03:43.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-29T08:03:43.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lg-g15.ko 2024-03-29T08:03:43.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-29T08:03:43.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 101537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-29T08:03:43.850 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech.ko 2024-03-29T08:03:43.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-magicmouse.ko 2024-03-29T08:03:43.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-microsoft.ko 2024-03-29T08:03:43.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-monterey.ko 2024-03-29T08:03:43.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-29T08:03:43.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ntrig.ko 2024-03-29T08:03:43.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-29T08:03:43.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-29T08:03:43.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-29T08:03:43.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-29T08:03:43.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-29T08:03:43.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-29T08:03:43.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-29T08:03:43.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-29T08:03:43.851 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-29T08:03:43.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-29T08:03:43.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-29T08:03:43.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-29T08:03:43.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-29T08:03:43.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-29T08:03:43.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-29T08:03:43.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-29T08:03:43.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-29T08:03:43.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-29T08:03:43.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-29T08:03:43.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-29T08:03:43.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-29T08:03:43.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-29T08:03:43.852 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-29T08:03:43.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-29T08:03:43.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-29T08:03:43.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-29T08:03:43.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-29T08:03:43.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-29T08:03:43.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-29T08:03:43.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-29T08:03:43.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-29T08:03:43.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-29T08:03:43.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-29T08:03:43.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-29T08:03:43.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-29T08:03:43.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 143377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-29T08:03:43.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-29T08:03:43.853 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-29T08:03:43.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-29T08:03:43.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 269209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid.ko 2024-03-29T08:03:43.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-29T08:03:43.854 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/usbhid/ 2024-03-29T08:03:43.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 125625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/usbhid/usbhid.ko 2024-03-29T08:03:43.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/usbhid/usbkbd.ko 2024-03-29T08:03:43.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/usbhid/usbmouse.ko 2024-03-29T08:03:43.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 218865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-29T08:03:43.854 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/ 2024-03-29T08:03:43.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-29T08:03:43.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-29T08:03:43.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 304913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-29T08:03:43.854 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ 2024-03-29T08:03:43.854 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-29T08:03:43.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-29T08:03:43.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-29T08:03:43.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-29T08:03:43.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-29T08:03:43.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-29T08:03:43.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-29T08:03:43.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-29T08:03:43.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-29T08:03:43.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-29T08:03:43.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-29T08:03:43.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-29T08:03:43.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-29T08:03:43.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-29T08:03:43.855 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-29T08:03:43.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-29T08:03:43.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-29T08:03:43.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-29T08:03:43.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-29T08:03:43.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-29T08:03:43.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-29T08:03:43.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-29T08:03:43.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-29T08:03:43.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-29T08:03:43.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-29T08:03:43.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/da9052-hwmon.ko 2024-03-29T08:03:43.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/da9055-hwmon.ko 2024-03-29T08:03:43.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-29T08:03:43.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 99505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-29T08:03:43.856 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-29T08:03:43.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-29T08:03:43.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-29T08:03:43.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc2103.ko 2024-03-29T08:03:43.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-29T08:03:43.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-29T08:03:43.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 94593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-29T08:03:43.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-29T08:03:43.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-29T08:03:43.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-29T08:03:43.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-29T08:03:43.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-29T08:03:43.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-29T08:03:43.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-29T08:03:43.857 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hih6130.ko 2024-03-29T08:03:43.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-29T08:03:43.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-29T08:03:43.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-29T08:03:43.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-29T08:03:43.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-29T08:03:43.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-29T08:03:43.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-29T08:03:43.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 140129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-29T08:03:43.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/jc42.ko 2024-03-29T08:03:43.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-29T08:03:43.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-29T08:03:43.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-29T08:03:43.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-29T08:03:43.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-29T08:03:43.858 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-29T08:03:43.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-29T08:03:43.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-29T08:03:43.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-29T08:03:43.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-29T08:03:43.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-29T08:03:43.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-29T08:03:43.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-29T08:03:43.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-29T08:03:43.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-29T08:03:43.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-29T08:03:43.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-29T08:03:43.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-29T08:03:43.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-29T08:03:43.859 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-29T08:03:43.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-29T08:03:43.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-29T08:03:43.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-29T08:03:43.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-29T08:03:43.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-29T08:03:43.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-29T08:03:43.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-29T08:03:43.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-29T08:03:43.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-29T08:03:43.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-29T08:03:43.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-29T08:03:43.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-29T08:03:43.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mc13783-adc.ko 2024-03-29T08:03:43.860 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-29T08:03:43.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-03-29T08:03:43.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-29T08:03:43.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 119841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-29T08:03:43.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-29T08:03:43.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-29T08:03:43.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 82713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-29T08:03:43.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-29T08:03:43.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-29T08:03:43.861 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ 2024-03-29T08:03:43.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-29T08:03:43.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-29T08:03:43.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-29T08:03:43.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-29T08:03:43.861 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-29T08:03:43.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-29T08:03:43.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-29T08:03:43.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 82473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-29T08:03:43.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-29T08:03:43.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-29T08:03:43.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-29T08:03:43.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-29T08:03:43.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-29T08:03:43.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-29T08:03:43.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-29T08:03:43.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-29T08:03:43.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-29T08:03:43.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-29T08:03:43.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-29T08:03:43.862 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-29T08:03:43.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-29T08:03:43.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-29T08:03:43.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-29T08:03:43.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-29T08:03:43.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-29T08:03:43.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-29T08:03:43.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-29T08:03:43.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-29T08:03:43.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-29T08:03:43.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-29T08:03:43.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-29T08:03:43.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-29T08:03:43.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-29T08:03:43.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 79833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-29T08:03:43.863 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-29T08:03:43.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-29T08:03:43.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-29T08:03:43.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 101377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-29T08:03:43.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 119409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-29T08:03:43.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-29T08:03:43.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-29T08:03:43.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-03-29T08:03:43.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-03-29T08:03:43.864 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/ 2024-03-29T08:03:43.864 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/ 2024-03-29T08:03:43.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-29T08:03:43.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-29T08:03:43.864 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/ 2024-03-29T08:03:43.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-03-29T08:03:43.864 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-03-29T08:03:43.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-03-29T08:03:43.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-29T08:03:43.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-29T08:03:43.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-29T08:03:43.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-03-29T08:03:43.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-29T08:03:43.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-03-29T08:03:43.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-29T08:03:43.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-29T08:03:43.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-29T08:03:43.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-29T08:03:43.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-03-29T08:03:43.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-03-29T08:03:43.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-29T08:03:43.865 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-29T08:03:43.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-29T08:03:43.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-03-29T08:03:43.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-29T08:03:43.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-29T08:03:43.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-03-29T08:03:43.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-29T08:03:43.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-29T08:03:43.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-29T08:03:43.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-03-29T08:03:43.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-29T08:03:43.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-29T08:03:43.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-03-29T08:03:43.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-03-29T08:03:43.866 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-29T08:03:43.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-03-29T08:03:43.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-29T08:03:43.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-29T08:03:43.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-29T08:03:43.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-29T08:03:43.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-03-29T08:03:43.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-29T08:03:43.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-29T08:03:43.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-29T08:03:43.867 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/muxes/ 2024-03-29T08:03:43.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-03-29T08:03:43.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-03-29T08:03:43.867 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-03-29T08:03:43.867 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ 2024-03-29T08:03:43.868 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ 2024-03-29T08:03:43.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 273513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-29T08:03:43.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 883777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-29T08:03:43.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-29T08:03:43.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 324233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-29T08:03:43.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-29T08:03:43.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 282329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-29T08:03:43.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-29T08:03:43.868 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/ 2024-03-29T08:03:43.868 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/ 2024-03-29T08:03:43.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 493129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-29T08:03:43.868 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/ 2024-03-29T08:03:43.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 420529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-29T08:03:43.868 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/ 2024-03-29T08:03:43.868 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 741609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-29T08:03:43.869 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ 2024-03-29T08:03:43.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 296785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-29T08:03:43.869 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/ocrdma/ 2024-03-29T08:03:43.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 153705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-03-29T08:03:43.869 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/usnic/ 2024-03-29T08:03:43.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 169089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-03-29T08:03:43.869 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ 2024-03-29T08:03:43.869 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ 2024-03-29T08:03:43.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 266529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-29T08:03:43.869 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ 2024-03-29T08:03:43.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 100257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-29T08:03:43.869 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ 2024-03-29T08:03:43.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 111329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-29T08:03:43.869 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ 2024-03-29T08:03:43.869 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 128657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-29T08:03:43.870 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ 2024-03-29T08:03:43.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 129793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-29T08:03:43.870 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/ 2024-03-29T08:03:43.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/evbug.ko 2024-03-29T08:03:43.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/ff-memless.ko 2024-03-29T08:03:43.870 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/ 2024-03-29T08:03:43.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-29T08:03:43.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-29T08:03:43.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-29T08:03:43.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-29T08:03:43.870 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/ 2024-03-29T08:03:43.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-03-29T08:03:43.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-03-29T08:03:43.870 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-03-29T08:03:43.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-03-29T08:03:43.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-29T08:03:43.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-03-29T08:03:43.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/lkkbd.ko 2024-03-29T08:03:43.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/lm8323.ko 2024-03-29T08:03:43.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/lm8333.ko 2024-03-29T08:03:43.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-03-29T08:03:43.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-03-29T08:03:43.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-03-29T08:03:43.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-03-29T08:03:43.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/newtonkbd.ko 2024-03-29T08:03:43.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-03-29T08:03:43.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/qt1070.ko 2024-03-29T08:03:43.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/qt2160.ko 2024-03-29T08:03:43.871 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-03-29T08:03:43.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/stowaway.ko 2024-03-29T08:03:43.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/sunkbd.ko 2024-03-29T08:03:43.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-03-29T08:03:43.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-03-29T08:03:43.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-03-29T08:03:43.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/xtkbd.ko 2024-03-29T08:03:43.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/matrix-keymap.ko 2024-03-29T08:03:43.872 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/ 2024-03-29T08:03:43.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-29T08:03:43.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-29T08:03:43.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 123177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-29T08:03:43.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-29T08:03:43.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/gpio_mouse.ko 2024-03-29T08:03:43.872 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 335857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/psmouse.ko 2024-03-29T08:03:43.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-29T08:03:43.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-29T08:03:43.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-29T08:03:43.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-29T08:03:43.873 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/ 2024-03-29T08:03:43.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 109553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-29T08:03:43.873 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/ 2024-03-29T08:03:43.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-29T08:03:43.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-29T08:03:43.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/ct82c710.ko 2024-03-29T08:03:43.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-29T08:03:43.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/parkbd.ko 2024-03-29T08:03:43.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/pcips2.ko 2024-03-29T08:03:43.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/ps2mult.ko 2024-03-29T08:03:43.873 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-29T08:03:43.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serport.ko 2024-03-29T08:03:43.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-29T08:03:43.874 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/ 2024-03-29T08:03:43.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-88pm860x.ko 2024-03-29T08:03:43.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-adp5520.ko 2024-03-29T08:03:43.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-bd2802.ko 2024-03-29T08:03:43.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-29T08:03:43.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-da903x.ko 2024-03-29T08:03:43.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-da9052.ko 2024-03-29T08:03:43.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-gpio.ko 2024-03-29T08:03:43.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-29T08:03:43.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3533.ko 2024-03-29T08:03:43.874 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm355x.ko 2024-03-29T08:03:43.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3642.ko 2024-03-29T08:03:43.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-29T08:03:43.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp8788.ko 2024-03-29T08:03:43.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-max8997.ko 2024-03-29T08:03:43.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-mc13783.ko 2024-03-29T08:03:43.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-menf21bmc.ko 2024-03-29T08:03:43.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-pca9532.ko 2024-03-29T08:03:43.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-pca955x.ko 2024-03-29T08:03:43.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-pca963x.ko 2024-03-29T08:03:43.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-29T08:03:43.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-tca6507.ko 2024-03-29T08:03:43.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-wm831x-status.ko 2024-03-29T08:03:43.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-wm8350.ko 2024-03-29T08:03:43.875 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ 2024-03-29T08:03:43.875 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-29T08:03:43.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-29T08:03:43.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-29T08:03:43.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-29T08:03:43.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-29T08:03:43.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-29T08:03:43.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-29T08:03:43.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-29T08:03:43.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-29T08:03:43.876 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/ 2024-03-29T08:03:43.876 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/ 2024-03-29T08:03:43.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 647033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-29T08:03:43.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-29T08:03:43.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 93313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bufio.ko 2024-03-29T08:03:43.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-29T08:03:43.876 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 155801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-29T08:03:43.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 107929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-29T08:03:43.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-29T08:03:43.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-era.ko 2024-03-29T08:03:43.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-29T08:03:43.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-29T08:03:43.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log.ko 2024-03-29T08:03:43.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-mirror.ko 2024-03-29T08:03:43.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 77865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-29T08:03:43.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-29T08:03:43.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-29T08:03:43.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-region-hash.ko 2024-03-29T08:03:43.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-29T08:03:43.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-29T08:03:43.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 109433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-snapshot.ko 2024-03-29T08:03:43.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-29T08:03:43.877 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 181761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-29T08:03:43.878 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-29T08:03:43.878 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-zero.ko 2024-03-29T08:03:43.878 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/ 2024-03-29T08:03:43.878 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 199825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-29T08:03:43.878 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-29T08:03:43.878 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 100833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-29T08:03:43.903 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 126361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-29T08:03:43.903 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 368481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-29T08:03:43.903 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/ 2024-03-29T08:03:43.904 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/ 2024-03-29T08:03:43.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 178041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-29T08:03:43.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-29T08:03:43.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-29T08:03:43.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-29T08:03:43.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 123225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-29T08:03:43.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-29T08:03:43.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-29T08:03:43.904 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/ 2024-03-29T08:03:43.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/88pm800.ko 2024-03-29T08:03:43.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/88pm805.ko 2024-03-29T08:03:43.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/88pm80x.ko 2024-03-29T08:03:43.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/arizona-i2c.ko 2024-03-29T08:03:43.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 123625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/arizona.ko 2024-03-29T08:03:43.904 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/bcm590xx.ko 2024-03-29T08:03:43.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/cros_ec_dev.ko 2024-03-29T08:03:43.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/dln2.ko 2024-03-29T08:03:43.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/janz-cmodio.ko 2024-03-29T08:03:43.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/kempld-core.ko 2024-03-29T08:03:43.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lm3533-core.ko 2024-03-29T08:03:43.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-03-29T08:03:43.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lp3943.ko 2024-03-29T08:03:43.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-29T08:03:43.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-29T08:03:43.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/max8907.ko 2024-03-29T08:03:43.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mc13xxx-core.ko 2024-03-29T08:03:43.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-03-29T08:03:43.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/menf21bmc.ko 2024-03-29T08:03:43.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/pcf50633-adc.ko 2024-03-29T08:03:43.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/pcf50633-gpio.ko 2024-03-29T08:03:43.905 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/pcf50633.ko 2024-03-29T08:03:43.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-03-29T08:03:43.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/retu-mfd.ko 2024-03-29T08:03:43.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/si476x-core.ko 2024-03-29T08:03:43.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-29T08:03:43.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/tps65010.ko 2024-03-29T08:03:43.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/tps6507x.ko 2024-03-29T08:03:43.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-29T08:03:43.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-29T08:03:43.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-29T08:03:43.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wm8994.ko 2024-03-29T08:03:43.906 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ 2024-03-29T08:03:43.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-03-29T08:03:43.906 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ad525x_dpot.ko 2024-03-29T08:03:43.906 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/ 2024-03-29T08:03:43.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-29T08:03:43.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-29T08:03:43.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-29T08:03:43.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-29T08:03:43.907 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/c2port/ 2024-03-29T08:03:43.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-03-29T08:03:43.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/c2port/core.ko 2024-03-29T08:03:43.907 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/ 2024-03-29T08:03:43.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-29T08:03:43.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ds1682.ko 2024-03-29T08:03:43.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/dummy-irq.ko 2024-03-29T08:03:43.907 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/echo/ 2024-03-29T08:03:43.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/echo/echo.ko 2024-03-29T08:03:43.907 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/ 2024-03-29T08:03:43.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-29T08:03:43.907 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-29T08:03:43.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-29T08:03:43.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-29T08:03:43.908 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/genwqe/ 2024-03-29T08:03:43.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 152041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-03-29T08:03:43.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hmc6352.ko 2024-03-29T08:03:43.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-29T08:03:43.908 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ibmasm/ 2024-03-29T08:03:43.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-03-29T08:03:43.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ics932s401.ko 2024-03-29T08:03:43.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-29T08:03:43.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-29T08:03:43.908 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/ 2024-03-29T08:03:43.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-29T08:03:43.908 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-29T08:03:43.908 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/ 2024-03-29T08:03:43.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 126137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-29T08:03:43.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-29T08:03:43.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 313545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-29T08:03:43.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/phantom.ko 2024-03-29T08:03:43.909 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ti-st/ 2024-03-29T08:03:43.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ti-st/st_drv.ko 2024-03-29T08:03:43.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-29T08:03:43.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-29T08:03:43.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-29T08:03:43.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-29T08:03:43.909 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/ 2024-03-29T08:03:43.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 183513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-29T08:03:43.909 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ 2024-03-29T08:03:43.909 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/ 2024-03-29T08:03:43.909 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 491561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-29T08:03:43.910 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/ 2024-03-29T08:03:43.910 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/ 2024-03-29T08:03:43.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 105929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-29T08:03:43.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-29T08:03:43.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-29T08:03:43.910 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/ 2024-03-29T08:03:43.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 458905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-29T08:03:43.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-29T08:03:43.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-29T08:03:43.910 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ 2024-03-29T08:03:43.910 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/ 2024-03-29T08:03:43.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-29T08:03:43.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-29T08:03:43.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 130657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-29T08:03:43.910 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-29T08:03:43.911 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ 2024-03-29T08:03:43.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-29T08:03:43.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-29T08:03:43.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-29T08:03:43.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 101609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-29T08:03:43.911 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/ 2024-03-29T08:03:43.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-29T08:03:43.911 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/ 2024-03-29T08:03:43.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-29T08:03:43.911 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/ 2024-03-29T08:03:43.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-29T08:03:43.911 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/ 2024-03-29T08:03:43.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-29T08:03:43.911 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/ 2024-03-29T08:03:43.911 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-29T08:03:43.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-29T08:03:43.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 87449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-29T08:03:43.912 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/ 2024-03-29T08:03:43.912 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/ 2024-03-29T08:03:43.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-29T08:03:43.912 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-03-29T08:03:43.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-29T08:03:43.912 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-03-29T08:03:43.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-29T08:03:43.912 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/ 2024-03-29T08:03:43.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-29T08:03:43.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-29T08:03:43.912 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/ 2024-03-29T08:03:43.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-29T08:03:43.912 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 176689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-29T08:03:43.913 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-03-29T08:03:43.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1434449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-29T08:03:43.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 128361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-29T08:03:43.913 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/ 2024-03-29T08:03:43.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 131265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-29T08:03:43.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 314321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-29T08:03:43.913 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/ 2024-03-29T08:03:43.913 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/ 2024-03-29T08:03:43.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 323681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-29T08:03:43.913 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/ 2024-03-29T08:03:43.913 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-03-29T08:03:43.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 193537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-29T08:03:43.913 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-03-29T08:03:43.913 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 347161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-29T08:03:43.913 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-03-29T08:03:43.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1045321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-29T08:03:43.914 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-03-29T08:03:43.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 166129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-29T08:03:43.914 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-03-29T08:03:43.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-29T08:03:43.914 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/ 2024-03-29T08:03:43.914 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/ 2024-03-29T08:03:43.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 204185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-29T08:03:43.914 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/ 2024-03-29T08:03:43.914 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/ 2024-03-29T08:03:43.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-29T08:03:43.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-29T08:03:43.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 141153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-29T08:03:43.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-29T08:03:43.914 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-29T08:03:43.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-29T08:03:43.915 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/ 2024-03-29T08:03:43.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-29T08:03:43.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-29T08:03:43.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-29T08:03:43.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ec_bhf.ko 2024-03-29T08:03:43.915 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/ 2024-03-29T08:03:43.915 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/ 2024-03-29T08:03:43.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 281193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-29T08:03:43.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-29T08:03:43.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-29T08:03:43.915 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fujitsu/ 2024-03-29T08:03:43.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-03-29T08:03:43.915 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ 2024-03-29T08:03:43.915 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 90297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-29T08:03:43.915 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/ 2024-03-29T08:03:43.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 295673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-29T08:03:43.916 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/ 2024-03-29T08:03:43.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 584297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-29T08:03:43.916 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/ 2024-03-29T08:03:43.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 245977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-29T08:03:43.916 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/ 2024-03-29T08:03:43.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 944449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-29T08:03:43.916 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/ 2024-03-29T08:03:43.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 288481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-29T08:03:43.916 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/ 2024-03-29T08:03:43.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 514337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-29T08:03:43.916 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/ 2024-03-29T08:03:43.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 93465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-29T08:03:43.916 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-03-29T08:03:43.916 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 735129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-29T08:03:43.917 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-03-29T08:03:43.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 151649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-29T08:03:43.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 86985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-29T08:03:43.917 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/ 2024-03-29T08:03:43.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-29T08:03:43.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-29T08:03:43.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 122601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-29T08:03:43.917 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/ 2024-03-29T08:03:43.917 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-03-29T08:03:43.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 727377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-29T08:03:43.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 278825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-29T08:03:43.917 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-03-29T08:03:43.917 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-03-29T08:03:43.917 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1122065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-29T08:03:43.917 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ 2024-03-29T08:03:43.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-03-29T08:03:43.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-03-29T08:03:43.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-03-29T08:03:43.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 95569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-29T08:03:43.918 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/ 2024-03-29T08:03:43.918 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-03-29T08:03:43.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-29T08:03:43.918 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ 2024-03-29T08:03:43.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-29T08:03:43.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-29T08:03:43.918 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/ 2024-03-29T08:03:43.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 151193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-29T08:03:43.918 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/ 2024-03-29T08:03:43.918 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 136793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-29T08:03:43.919 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/ 2024-03-29T08:03:43.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-29T08:03:43.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-29T08:03:43.919 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/ 2024-03-29T08:03:43.919 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-03-29T08:03:43.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 227841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-29T08:03:43.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 87009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-29T08:03:43.919 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-03-29T08:03:43.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 671985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-29T08:03:43.919 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/ 2024-03-29T08:03:43.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-29T08:03:43.919 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/ 2024-03-29T08:03:43.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-29T08:03:43.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 83801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-29T08:03:43.919 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-29T08:03:43.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 194057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-29T08:03:43.920 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/samsung/ 2024-03-29T08:03:43.920 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-03-29T08:03:43.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 104001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-03-29T08:03:43.920 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/ 2024-03-29T08:03:43.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1066817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-29T08:03:43.920 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/ 2024-03-29T08:03:43.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-29T08:03:43.920 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/ 2024-03-29T08:03:43.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-29T08:03:43.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 63193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-29T08:03:43.920 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/ 2024-03-29T08:03:43.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-29T08:03:43.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-29T08:03:43.920 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-29T08:03:43.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-29T08:03:43.921 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/ 2024-03-29T08:03:43.921 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-03-29T08:03:43.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-03-29T08:03:43.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-29T08:03:43.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-03-29T08:03:43.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 495641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-29T08:03:43.921 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/ 2024-03-29T08:03:43.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 103177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-29T08:03:43.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 166041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-29T08:03:43.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-29T08:03:43.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-29T08:03:43.921 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/ 2024-03-29T08:03:43.921 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-29T08:03:43.921 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/ 2024-03-29T08:03:43.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 92337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-29T08:03:43.922 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/ 2024-03-29T08:03:43.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 79929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-29T08:03:43.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 83713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-29T08:03:43.922 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/ 2024-03-29T08:03:43.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-29T08:03:43.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-29T08:03:43.922 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/ 2024-03-29T08:03:43.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-29T08:03:43.922 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/fddi/ 2024-03-29T08:03:43.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/fddi/defxx.ko 2024-03-29T08:03:43.922 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/fddi/skfp/ 2024-03-29T08:03:43.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 217025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/fddi/skfp/skfp.ko 2024-03-29T08:03:43.922 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-29T08:03:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/ 2024-03-29T08:03:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 237241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-29T08:03:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/ 2024-03-29T08:03:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-29T08:03:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-29T08:03:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ 2024-03-29T08:03:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-29T08:03:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-29T08:03:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-29T08:03:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/ 2024-03-29T08:03:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-29T08:03:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-29T08:03:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-gpio.ko 2024-03-29T08:03:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-29T08:03:43.923 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-29T08:03:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-29T08:03:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-29T08:03:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-29T08:03:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/ 2024-03-29T08:03:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-29T08:03:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-29T08:03:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ 2024-03-29T08:03:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-29T08:03:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-29T08:03:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-29T08:03:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-29T08:03:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-29T08:03:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-29T08:03:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-29T08:03:43.924 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-29T08:03:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-29T08:03:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-29T08:03:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-29T08:03:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 76561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-29T08:03:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-29T08:03:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 99001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-29T08:03:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-29T08:03:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-29T08:03:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-29T08:03:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-29T08:03:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-29T08:03:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/plip/ 2024-03-29T08:03:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/plip/plip.ko 2024-03-29T08:03:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ 2024-03-29T08:03:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-29T08:03:43.925 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-29T08:03:43.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-29T08:03:43.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-29T08:03:43.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-29T08:03:43.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-29T08:03:43.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-29T08:03:43.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-29T08:03:43.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sb1000.ko 2024-03-29T08:03:43.926 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/ 2024-03-29T08:03:43.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-29T08:03:43.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-29T08:03:43.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-29T08:03:43.926 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/ 2024-03-29T08:03:43.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 107225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-29T08:03:43.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-29T08:03:43.926 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-29T08:03:43.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-29T08:03:43.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-29T08:03:43.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-29T08:03:43.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 86817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-29T08:03:43.927 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/ 2024-03-29T08:03:43.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 144345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-29T08:03:43.927 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/ 2024-03-29T08:03:43.927 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 307169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-29T08:03:43.944 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/ 2024-03-29T08:03:43.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 143993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-29T08:03:43.944 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ 2024-03-29T08:03:43.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-29T08:03:43.944 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/ 2024-03-29T08:03:43.944 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/ 2024-03-29T08:03:43.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 267721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-29T08:03:43.944 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 103353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-29T08:03:43.944 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/ 2024-03-29T08:03:43.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 131297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-29T08:03:43.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-29T08:03:43.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-29T08:03:43.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-29T08:03:43.945 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/ 2024-03-29T08:03:43.945 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/ 2024-03-29T08:03:43.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-29T08:03:43.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-03-29T08:03:43.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-03-29T08:03:43.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pci-stub.ko 2024-03-29T08:03:43.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-29T08:03:43.945 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/ 2024-03-29T08:03:43.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-29T08:03:43.945 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 146129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pcmcia.ko 2024-03-29T08:03:43.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pcmcia_core.ko 2024-03-29T08:03:43.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-03-29T08:03:43.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-29T08:03:43.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 112785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-29T08:03:43.946 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/ 2024-03-29T08:03:43.946 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/broadcom/ 2024-03-29T08:03:43.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-03-29T08:03:43.946 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/samsung/ 2024-03-29T08:03:43.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-03-29T08:03:43.946 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/ 2024-03-29T08:03:43.946 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/ 2024-03-29T08:03:43.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-29T08:03:43.946 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/ 2024-03-29T08:03:43.946 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/ 2024-03-29T08:03:43.946 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-03-29T08:03:43.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec.ko 2024-03-29T08:03:43.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-03-29T08:03:43.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-03-29T08:03:43.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-03-29T08:03:43.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-03-29T08:03:43.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-03-29T08:03:43.947 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/ 2024-03-29T08:03:43.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-29T08:03:43.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-29T08:03:43.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-29T08:03:43.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-29T08:03:43.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-29T08:03:43.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 131937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-29T08:03:43.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-29T08:03:43.947 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-29T08:03:43.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-29T08:03:43.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-29T08:03:43.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-29T08:03:43.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-29T08:03:43.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/ibm_rtl.ko 2024-03-29T08:03:43.948 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/ 2024-03-29T08:03:43.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-29T08:03:43.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-03-29T08:03:43.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-29T08:03:43.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-29T08:03:43.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-29T08:03:43.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-29T08:03:43.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-29T08:03:43.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-29T08:03:43.948 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-29T08:03:43.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-29T08:03:43.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-29T08:03:43.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-29T08:03:43.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-29T08:03:43.949 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/ 2024-03-29T08:03:43.949 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ 2024-03-29T08:03:43.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/88pm860x_battery.ko 2024-03-29T08:03:43.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/88pm860x_charger.ko 2024-03-29T08:03:43.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/bq2415x_charger.ko 2024-03-29T08:03:43.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/bq24190_charger.ko 2024-03-29T08:03:43.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/bq24735-charger.ko 2024-03-29T08:03:43.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-03-29T08:03:43.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/da9030_battery.ko 2024-03-29T08:03:43.949 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/da9052-battery.ko 2024-03-29T08:03:43.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ds2760_battery.ko 2024-03-29T08:03:43.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ds2780_battery.ko 2024-03-29T08:03:43.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ds2781_battery.ko 2024-03-29T08:03:43.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ds2782_battery.ko 2024-03-29T08:03:43.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/gpio-charger.ko 2024-03-29T08:03:43.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/isp1704_charger.ko 2024-03-29T08:03:43.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/lp8727_charger.ko 2024-03-29T08:03:43.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/max14577_charger.ko 2024-03-29T08:03:43.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/max17042_battery.ko 2024-03-29T08:03:43.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/max8903_charger.ko 2024-03-29T08:03:43.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/max8925_power.ko 2024-03-29T08:03:43.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/pcf50633-charger.ko 2024-03-29T08:03:43.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/sbs-battery.ko 2024-03-29T08:03:43.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/test_power.ko 2024-03-29T08:03:43.950 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/tps65090-charger.ko 2024-03-29T08:03:43.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/wm831x_backup.ko 2024-03-29T08:03:43.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/wm831x_power.ko 2024-03-29T08:03:43.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/wm8350_power.ko 2024-03-29T08:03:43.951 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/ 2024-03-29T08:03:43.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-29T08:03:43.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-29T08:03:43.951 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/ 2024-03-29T08:03:43.951 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/ 2024-03-29T08:03:43.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-29T08:03:43.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-29T08:03:43.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-29T08:03:43.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-29T08:03:43.951 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ 2024-03-29T08:03:43.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-29T08:03:43.951 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-29T08:03:43.952 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/ 2024-03-29T08:03:43.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-88pm80x.ko 2024-03-29T08:03:43.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-88pm860x.ko 2024-03-29T08:03:43.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-29T08:03:43.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-da9052.ko 2024-03-29T08:03:43.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-da9055.ko 2024-03-29T08:03:43.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-da9063.ko 2024-03-29T08:03:43.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-29T08:03:43.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-29T08:03:43.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-29T08:03:43.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-29T08:03:43.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-29T08:03:43.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-29T08:03:43.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-29T08:03:43.952 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-29T08:03:43.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-29T08:03:43.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-29T08:03:43.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-29T08:03:43.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-29T08:03:43.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-29T08:03:43.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-lp8788.ko 2024-03-29T08:03:43.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-29T08:03:43.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-29T08:03:43.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-29T08:03:43.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t86.ko 2024-03-29T08:03:43.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-29T08:03:43.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max8907.ko 2024-03-29T08:03:43.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max8925.ko 2024-03-29T08:03:43.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max8997.ko 2024-03-29T08:03:43.953 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max8998.ko 2024-03-29T08:03:43.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-03-29T08:03:43.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-29T08:03:43.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-palmas.ko 2024-03-29T08:03:43.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-29T08:03:43.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf50633.ko 2024-03-29T08:03:43.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-29T08:03:43.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-29T08:03:43.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-29T08:03:43.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-29T08:03:43.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rc5t583.ko 2024-03-29T08:03:43.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-29T08:03:43.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-29T08:03:43.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-29T08:03:43.954 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-29T08:03:43.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-29T08:03:43.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-s35390a.ko 2024-03-29T08:03:43.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-29T08:03:43.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-tps6586x.ko 2024-03-29T08:03:43.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-tps65910.ko 2024-03-29T08:03:43.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-wm831x.ko 2024-03-29T08:03:43.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-wm8350.ko 2024-03-29T08:03:43.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-29T08:03:43.955 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ 2024-03-29T08:03:43.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 77521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-29T08:03:43.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-29T08:03:43.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-29T08:03:43.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 135249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-29T08:03:43.955 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-29T08:03:43.955 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/ 2024-03-29T08:03:43.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 271385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-29T08:03:43.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 100529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-29T08:03:43.956 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/ 2024-03-29T08:03:43.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 356833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-29T08:03:43.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 295889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-29T08:03:43.956 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic94xx/ 2024-03-29T08:03:43.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 189089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-03-29T08:03:43.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-29T08:03:43.956 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/ 2024-03-29T08:03:43.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-29T08:03:43.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-29T08:03:43.956 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/ 2024-03-29T08:03:43.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 246913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-29T08:03:43.956 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/ 2024-03-29T08:03:43.956 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1171857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-29T08:03:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/ 2024-03-29T08:03:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 238369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-29T08:03:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/ 2024-03-29T08:03:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 115305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-29T08:03:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-29T08:03:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/ 2024-03-29T08:03:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 308905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-29T08:03:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/ 2024-03-29T08:03:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-03-29T08:03:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 76033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-29T08:03:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-03-29T08:03:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 101417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-29T08:03:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 146745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-29T08:03:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 75969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-29T08:03:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-29T08:03:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/ 2024-03-29T08:03:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 212329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-29T08:03:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-29T08:03:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/ 2024-03-29T08:03:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-29T08:03:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 157713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-29T08:03:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fdomain.ko 2024-03-29T08:03:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/ 2024-03-29T08:03:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 294881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-29T08:03:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 201721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-29T08:03:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-29T08:03:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-29T08:03:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/imm.ko 2024-03-29T08:03:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-29T08:03:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 265937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-29T08:03:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 101873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-29T08:03:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/ 2024-03-29T08:03:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 317129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-29T08:03:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-29T08:03:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-29T08:03:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/ 2024-03-29T08:03:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 315273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-29T08:03:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 145489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-29T08:03:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-29T08:03:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/ 2024-03-29T08:03:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 216345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-29T08:03:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/ 2024-03-29T08:03:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2193953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-29T08:03:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/ 2024-03-29T08:03:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-29T08:03:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-29T08:03:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 310665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-29T08:03:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-29T08:03:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/ 2024-03-29T08:03:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 659585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-29T08:03:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/ 2024-03-29T08:03:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 119465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-29T08:03:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-29T08:03:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/ 2024-03-29T08:03:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-03-29T08:03:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-03-29T08:03:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-03-29T08:03:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-03-29T08:03:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/ 2024-03-29T08:03:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 366305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-29T08:03:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 124593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-29T08:03:43.961 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ppa.ko 2024-03-29T08:03:43.961 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-29T08:03:43.961 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/ 2024-03-29T08:03:43.961 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1876657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-29T08:03:43.991 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-29T08:03:43.992 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/ 2024-03-29T08:03:43.992 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 558897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-29T08:03:43.992 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qlogicfas408.ko 2024-03-29T08:03:43.992 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-29T08:03:43.992 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 261073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-29T08:03:43.992 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 173921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-29T08:03:43.992 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 280105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-29T08:03:43.992 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-29T08:03:43.992 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-29T08:03:43.992 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-29T08:03:43.992 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-29T08:03:43.992 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 115641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-29T08:03:43.992 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-29T08:03:43.993 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/ 2024-03-29T08:03:43.993 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 164865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-29T08:03:43.993 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-29T08:03:43.993 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-29T08:03:43.993 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-29T08:03:43.993 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/xen-scsifront.ko 2024-03-29T08:03:43.993 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ 2024-03-29T08:03:43.993 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 129345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-29T08:03:43.993 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/ 2024-03-29T08:03:43.993 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/ 2024-03-29T08:03:43.993 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 736665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-29T08:03:43.993 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/ 2024-03-29T08:03:43.993 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-29T08:03:43.993 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-29T08:03:43.993 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-29T08:03:43.994 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 931433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-29T08:03:43.994 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-29T08:03:43.994 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 122481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-29T08:03:43.994 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/ 2024-03-29T08:03:43.994 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 75497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-29T08:03:43.994 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/ 2024-03-29T08:03:43.994 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/ 2024-03-29T08:03:43.994 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/ 2024-03-29T08:03:43.994 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-29T08:03:43.994 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-29T08:03:43.994 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-29T08:03:43.994 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-29T08:03:43.994 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-29T08:03:43.994 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-29T08:03:43.995 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-29T08:03:43.995 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-29T08:03:43.995 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-29T08:03:43.995 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-29T08:03:43.995 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-29T08:03:43.995 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-29T08:03:43.995 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-29T08:03:43.995 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-29T08:03:43.995 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-29T08:03:43.995 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-03-29T08:03:43.995 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-29T08:03:43.995 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-29T08:03:43.995 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-29T08:03:43.995 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ 2024-03-29T08:03:43.995 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ 2024-03-29T08:03:43.996 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-29T08:03:43.996 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/moxa.ko 2024-03-29T08:03:43.996 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/mxser.ko 2024-03-29T08:03:43.996 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-29T08:03:43.996 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-29T08:03:43.996 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-29T08:03:43.996 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/ 2024-03-29T08:03:43.996 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/ 2024-03-29T08:03:43.996 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-03-29T08:03:43.996 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-29T08:03:43.996 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-03-29T08:03:43.996 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/altera_uart.ko 2024-03-29T08:03:43.996 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-29T08:03:43.996 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-03-29T08:03:43.997 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/ 2024-03-29T08:03:43.997 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 101969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-29T08:03:43.997 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/rp2.ko 2024-03-29T08:03:43.997 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/sc16is7xx.ko 2024-03-29T08:03:43.997 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/ 2024-03-29T08:03:43.997 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ 2024-03-29T08:03:43.997 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 545377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-29T08:03:43.997 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ 2024-03-29T08:03:43.997 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-29T08:03:43.997 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-03-29T08:03:43.997 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/ 2024-03-29T08:03:43.997 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-29T08:03:43.997 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-29T08:03:43.997 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-29T08:03:43.998 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_dmem_genirq.ko 2024-03-29T08:03:43.998 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_mf624.ko 2024-03-29T08:03:43.998 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_netx.ko 2024-03-29T08:03:43.998 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-29T08:03:43.998 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-03-29T08:03:43.998 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-29T08:03:43.998 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/ 2024-03-29T08:03:43.998 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/c67x00/ 2024-03-29T08:03:43.998 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/c67x00/c67x00.ko 2024-03-29T08:03:43.998 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ 2024-03-29T08:03:43.998 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 145753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-03-29T08:03:43.998 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-03-29T08:03:43.998 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-03-29T08:03:43.998 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-03-29T08:03:43.998 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-03-29T08:03:43.999 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/ 2024-03-29T08:03:43.999 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 87001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-29T08:03:43.999 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-29T08:03:43.999 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-29T08:03:43.999 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-29T08:03:43.999 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/common/ 2024-03-29T08:03:43.999 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/common/ulpi.ko 2024-03-29T08:03:43.999 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc2/ 2024-03-29T08:03:43.999 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-03-29T08:03:43.999 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc3/ 2024-03-29T08:03:43.999 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-03-29T08:03:43.999 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-03-29T08:03:43.999 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 452785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc3/dwc3.ko 2024-03-29T08:03:43.999 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/ 2024-03-29T08:03:43.999 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/ 2024-03-29T08:03:44.000 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/u_ether.ko 2024-03-29T08:03:44.000 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/u_serial.ko 2024-03-29T08:03:44.000 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-03-29T08:03:44.000 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-03-29T08:03:44.000 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-03-29T08:03:44.000 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-03-29T08:03:44.000 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 92401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-03-29T08:03:44.000 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 50937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-03-29T08:03:44.000 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 122241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-03-29T08:03:44.000 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-03-29T08:03:44.000 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-03-29T08:03:44.000 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-03-29T08:03:44.000 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-03-29T08:03:44.000 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 85505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-03-29T08:03:44.000 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-03-29T08:03:44.001 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-03-29T08:03:44.001 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-03-29T08:03:44.001 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/ 2024-03-29T08:03:44.001 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-03-29T08:03:44.001 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-03-29T08:03:44.001 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-03-29T08:03:44.001 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-03-29T08:03:44.001 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-03-29T08:03:44.001 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-03-29T08:03:44.001 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-03-29T08:03:44.001 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-03-29T08:03:44.001 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-03-29T08:03:44.001 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-03-29T08:03:44.001 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-03-29T08:03:44.001 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-03-29T08:03:44.002 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-03-29T08:03:44.002 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-03-29T08:03:44.002 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 161193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/libcomposite.ko 2024-03-29T08:03:44.002 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/ 2024-03-29T08:03:44.002 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-03-29T08:03:44.002 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/bdc/ 2024-03-29T08:03:44.002 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 157593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-03-29T08:03:44.002 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-03-29T08:03:44.002 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-03-29T08:03:44.002 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 69609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-03-29T08:03:44.002 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-03-29T08:03:44.002 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 75961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/net2272.ko 2024-03-29T08:03:44.002 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/net2280.ko 2024-03-29T08:03:44.002 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 73009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-03-29T08:03:44.003 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-03-29T08:03:44.003 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 65993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-03-29T08:03:44.003 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-03-29T08:03:44.003 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 156321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-03-29T08:03:44.003 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/ 2024-03-29T08:03:44.003 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/bcma-hcd.ko 2024-03-29T08:03:44.003 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/isp116x-hcd.ko 2024-03-29T08:03:44.003 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 87249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-03-29T08:03:44.003 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-03-29T08:03:44.003 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-29T08:03:44.003 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811_cs.ko 2024-03-29T08:03:44.003 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/ssb-hcd.ko 2024-03-29T08:03:44.003 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-03-29T08:03:44.003 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/ 2024-03-29T08:03:44.003 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-29T08:03:44.004 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-29T08:03:44.004 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ 2024-03-29T08:03:44.004 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-29T08:03:44.004 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-29T08:03:44.004 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-03-29T08:03:44.004 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/cytherm.ko 2024-03-29T08:03:44.004 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ehset.ko 2024-03-29T08:03:44.004 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-29T08:03:44.004 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-29T08:03:44.004 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-29T08:03:44.004 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-29T08:03:44.004 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-29T08:03:44.004 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-29T08:03:44.004 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-29T08:03:44.004 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-29T08:03:44.005 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/lvstest.ko 2024-03-29T08:03:44.005 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/ 2024-03-29T08:03:44.005 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-29T08:03:44.005 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-29T08:03:44.005 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-29T08:03:44.005 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-29T08:03:44.005 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-29T08:03:44.005 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbtest.ko 2024-03-29T08:03:44.005 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-29T08:03:44.005 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-29T08:03:44.005 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/mon/ 2024-03-29T08:03:44.005 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/mon/usbmon.ko 2024-03-29T08:03:44.005 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/musb/ 2024-03-29T08:03:44.005 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 301593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/musb/musb_hdrc.ko 2024-03-29T08:03:44.005 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/ 2024-03-29T08:03:44.006 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-29T08:03:44.006 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-03-29T08:03:44.006 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-isp1301.ko 2024-03-29T08:03:44.006 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-tahvo.ko 2024-03-29T08:03:44.006 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ 2024-03-29T08:03:44.006 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-29T08:03:44.006 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-29T08:03:44.006 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-29T08:03:44.006 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-29T08:03:44.006 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-29T08:03:44.006 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-29T08:03:44.006 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-29T08:03:44.006 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-29T08:03:44.006 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-29T08:03:44.007 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/f81232.ko 2024-03-29T08:03:44.007 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 175097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-29T08:03:44.007 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-29T08:03:44.007 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 136977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-29T08:03:44.007 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 124737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-29T08:03:44.007 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-29T08:03:44.007 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-29T08:03:44.007 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-29T08:03:44.007 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-29T08:03:44.007 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 107425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-29T08:03:44.007 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-29T08:03:44.007 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-29T08:03:44.007 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-29T08:03:44.007 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-29T08:03:44.007 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/metro-usb.ko 2024-03-29T08:03:44.008 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-29T08:03:44.008 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 78121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-29T08:03:44.008 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mxuport.ko 2024-03-29T08:03:44.008 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-29T08:03:44.008 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-29T08:03:44.008 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-29T08:03:44.008 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 191817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-29T08:03:44.008 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-29T08:03:44.008 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-29T08:03:44.008 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-29T08:03:44.008 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-29T08:03:44.008 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-29T08:03:44.008 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-29T08:03:44.008 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-29T08:03:44.008 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-29T08:03:44.009 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-29T08:03:44.009 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-29T08:03:44.009 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-29T08:03:44.009 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-29T08:03:44.009 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-29T08:03:44.009 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-29T08:03:44.009 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 125513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usbserial.ko 2024-03-29T08:03:44.009 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-29T08:03:44.009 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-29T08:03:44.009 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/wishbone-serial.ko 2024-03-29T08:03:44.009 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-29T08:03:44.009 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ 2024-03-29T08:03:44.009 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 52753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-29T08:03:44.009 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-29T08:03:44.009 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-29T08:03:44.010 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-29T08:03:44.010 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-29T08:03:44.010 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-29T08:03:44.010 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-29T08:03:44.010 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-29T08:03:44.010 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-29T08:03:44.010 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-29T08:03:44.010 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-29T08:03:44.010 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-29T08:03:44.010 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-29T08:03:44.010 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-29T08:03:44.010 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 180281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-29T08:03:44.010 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/ 2024-03-29T08:03:44.010 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-29T08:03:44.010 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-29T08:03:44.011 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 103977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-29T08:03:44.011 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/ 2024-03-29T08:03:44.011 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/ 2024-03-29T08:03:44.011 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 162841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-29T08:03:44.011 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-29T08:03:44.011 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 103649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-29T08:03:44.011 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 76433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-29T08:03:44.011 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/ 2024-03-29T08:03:44.011 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 113657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-29T08:03:44.011 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-29T08:03:44.011 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 57993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-29T08:03:44.011 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-29T08:03:44.011 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/ 2024-03-29T08:03:44.011 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/ 2024-03-29T08:03:44.011 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-03-29T08:03:44.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/aat2870_bl.ko 2024-03-29T08:03:44.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/adp5520_bl.ko 2024-03-29T08:03:44.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/adp8860_bl.ko 2024-03-29T08:03:44.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/adp8870_bl.ko 2024-03-29T08:03:44.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-29T08:03:44.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/as3711_bl.ko 2024-03-29T08:03:44.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/bd6107.ko 2024-03-29T08:03:44.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/da903x_bl.ko 2024-03-29T08:03:44.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/da9052_bl.ko 2024-03-29T08:03:44.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/gpio_backlight.ko 2024-03-29T08:03:44.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/kb3886_bl.ko 2024-03-29T08:03:44.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-29T08:03:44.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lm3533_bl.ko 2024-03-29T08:03:44.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lm3639_bl.ko 2024-03-29T08:03:44.012 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lv5207lp.ko 2024-03-29T08:03:44.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/max8925_bl.ko 2024-03-29T08:03:44.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/pandora_bl.ko 2024-03-29T08:03:44.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-03-29T08:03:44.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-29T08:03:44.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/wm831x_bl.ko 2024-03-29T08:03:44.013 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/ 2024-03-29T08:03:44.013 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ 2024-03-29T08:03:44.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-29T08:03:44.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-29T08:03:44.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/matrox_w1.ko 2024-03-29T08:03:44.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/w1-gpio.ko 2024-03-29T08:03:44.013 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/ 2024-03-29T08:03:44.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-29T08:03:44.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-29T08:03:44.013 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-29T08:03:44.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-29T08:03:44.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-29T08:03:44.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-29T08:03:44.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-29T08:03:44.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-29T08:03:44.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-29T08:03:44.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-29T08:03:44.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-29T08:03:44.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 103025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-29T08:03:44.014 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ 2024-03-29T08:03:44.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/acquirewdt.ko 2024-03-29T08:03:44.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/advantechwdt.ko 2024-03-29T08:03:44.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-29T08:03:44.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-29T08:03:44.014 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/cpu5wdt.ko 2024-03-29T08:03:44.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/da9052_wdt.ko 2024-03-29T08:03:44.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/da9055_wdt.ko 2024-03-29T08:03:44.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/da9063_wdt.ko 2024-03-29T08:03:44.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/dw_wdt.ko 2024-03-29T08:03:44.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/eurotechwdt.ko 2024-03-29T08:03:44.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-29T08:03:44.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-29T08:03:44.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-29T08:03:44.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-29T08:03:44.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-29T08:03:44.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-29T08:03:44.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-29T08:03:44.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-29T08:03:44.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-29T08:03:44.015 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-29T08:03:44.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/kempld_wdt.ko 2024-03-29T08:03:44.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-29T08:03:44.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/mena21_wdt.ko 2024-03-29T08:03:44.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-03-29T08:03:44.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-29T08:03:44.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-03-29T08:03:44.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pc87413_wdt.ko 2024-03-29T08:03:44.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-29T08:03:44.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-29T08:03:44.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/retu_wdt.ko 2024-03-29T08:03:44.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-03-29T08:03:44.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-03-29T08:03:44.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-29T08:03:44.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sc1200wdt.ko 2024-03-29T08:03:44.016 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-29T08:03:44.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-03-29T08:03:44.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-29T08:03:44.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-29T08:03:44.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/twl4030_wdt.ko 2024-03-29T08:03:44.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-29T08:03:44.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-29T08:03:44.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-29T08:03:44.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-29T08:03:44.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wafer5823wdt.ko 2024-03-29T08:03:44.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-29T08:03:44.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wm831x_wdt.ko 2024-03-29T08:03:44.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wm8350_wdt.ko 2024-03-29T08:03:44.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-29T08:03:44.017 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/ 2024-03-29T08:03:44.017 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-29T08:03:44.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-29T08:03:44.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-29T08:03:44.018 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/ 2024-03-29T08:03:44.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 176337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-29T08:03:44.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-29T08:03:44.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 71145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-scsiback.ko 2024-03-29T08:03:44.018 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/ 2024-03-29T08:03:44.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-29T08:03:44.018 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ 2024-03-29T08:03:44.018 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/ 2024-03-29T08:03:44.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 158217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-29T08:03:44.018 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/adfs/ 2024-03-29T08:03:44.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/adfs/adfs.ko 2024-03-29T08:03:44.018 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/ 2024-03-29T08:03:44.018 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 163369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-29T08:03:44.019 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/afs/ 2024-03-29T08:03:44.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1019049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/afs/kafs.ko 2024-03-29T08:03:44.019 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/autofs/ 2024-03-29T08:03:44.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 103185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/autofs/autofs4.ko 2024-03-29T08:03:44.019 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/ 2024-03-29T08:03:44.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 64585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-29T08:03:44.019 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/bfs/ 2024-03-29T08:03:44.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 41001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/bfs/bfs.ko 2024-03-29T08:03:44.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-29T08:03:44.019 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/ 2024-03-29T08:03:44.019 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 3487249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-29T08:03:44.027 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/ 2024-03-29T08:03:44.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 305841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-29T08:03:44.027 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ 2024-03-29T08:03:44.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1335577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-29T08:03:44.027 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/ 2024-03-29T08:03:44.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 97713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-29T08:03:44.027 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/configfs/ 2024-03-29T08:03:44.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/configfs/configfs.ko 2024-03-29T08:03:44.027 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/ 2024-03-29T08:03:44.027 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-29T08:03:44.028 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/ 2024-03-29T08:03:44.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 644161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-29T08:03:44.028 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/efs/ 2024-03-29T08:03:44.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/efs/efs.ko 2024-03-29T08:03:44.028 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/ 2024-03-29T08:03:44.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1527561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-29T08:03:44.028 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/ 2024-03-29T08:03:44.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-29T08:03:44.028 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/freevxfs/ 2024-03-29T08:03:44.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/freevxfs/freevxfs.ko 2024-03-29T08:03:44.028 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/ 2024-03-29T08:03:44.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-29T08:03:44.028 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/ 2024-03-29T08:03:44.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1231233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-29T08:03:44.028 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/ 2024-03-29T08:03:44.028 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 141625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-29T08:03:44.029 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/ 2024-03-29T08:03:44.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 223177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-29T08:03:44.029 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hpfs/ 2024-03-29T08:03:44.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 193481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hpfs/hpfs.ko 2024-03-29T08:03:44.029 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/ 2024-03-29T08:03:44.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-29T08:03:44.029 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/ 2024-03-29T08:03:44.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 420329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-29T08:03:44.029 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/ 2024-03-29T08:03:44.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 281617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-29T08:03:44.029 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/ 2024-03-29T08:03:44.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 88897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-29T08:03:44.029 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/ 2024-03-29T08:03:44.029 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 490409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-29T08:03:44.029 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/ 2024-03-29T08:03:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/ 2024-03-29T08:03:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 81433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-29T08:03:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/ 2024-03-29T08:03:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-29T08:03:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/ 2024-03-29T08:03:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 117361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-29T08:03:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1058273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-29T08:03:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv2.ko 2024-03-29T08:03:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 120169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-29T08:03:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2041705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-29T08:03:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/ 2024-03-29T08:03:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-29T08:03:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-29T08:03:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/ 2024-03-29T08:03:44.030 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1511641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-29T08:03:44.031 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/ 2024-03-29T08:03:44.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 582409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-29T08:03:44.031 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/ 2024-03-29T08:03:44.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-29T08:03:44.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-29T08:03:44.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-29T08:03:44.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-29T08:03:44.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-29T08:03:44.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-29T08:03:44.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-29T08:03:44.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-29T08:03:44.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-29T08:03:44.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-29T08:03:44.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-29T08:03:44.031 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ascii.ko 2024-03-29T08:03:44.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-29T08:03:44.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-29T08:03:44.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-29T08:03:44.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-29T08:03:44.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-29T08:03:44.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-29T08:03:44.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-29T08:03:44.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-29T08:03:44.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-29T08:03:44.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-29T08:03:44.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-29T08:03:44.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-29T08:03:44.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-29T08:03:44.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-29T08:03:44.032 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-29T08:03:44.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-29T08:03:44.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-29T08:03:44.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-29T08:03:44.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 94033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-29T08:03:44.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 142569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-29T08:03:44.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 165641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-29T08:03:44.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 118017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-29T08:03:44.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-29T08:03:44.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-29T08:03:44.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-29T08:03:44.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-29T08:03:44.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-29T08:03:44.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-29T08:03:44.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-29T08:03:44.033 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-29T08:03:44.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-29T08:03:44.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-29T08:03:44.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-29T08:03:44.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-29T08:03:44.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-29T08:03:44.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-29T08:03:44.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-29T08:03:44.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-29T08:03:44.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-29T08:03:44.034 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ntfs/ 2024-03-29T08:03:44.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 220457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ntfs/ntfs.ko 2024-03-29T08:03:44.034 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ 2024-03-29T08:03:44.034 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ 2024-03-29T08:03:44.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 244633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-29T08:03:44.034 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ 2024-03-29T08:03:44.034 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 440633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-29T08:03:44.035 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ 2024-03-29T08:03:44.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-29T08:03:44.035 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2590681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-29T08:03:44.036 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-29T08:03:44.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-29T08:03:44.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-29T08:03:44.037 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/omfs/ 2024-03-29T08:03:44.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/omfs/omfs.ko 2024-03-29T08:03:44.037 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/ 2024-03-29T08:03:44.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 360897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-29T08:03:44.037 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ 2024-03-29T08:03:44.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 56929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-29T08:03:44.037 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/qnx4/ 2024-03-29T08:03:44.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/qnx4/qnx4.ko 2024-03-29T08:03:44.037 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/qnx6/ 2024-03-29T08:03:44.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 46601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/qnx6/qnx6.ko 2024-03-29T08:03:44.037 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/quota/ 2024-03-29T08:03:44.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/quota/quota_tree.ko 2024-03-29T08:03:44.037 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/quota/quota_v1.ko 2024-03-29T08:03:44.038 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/quota/quota_v2.ko 2024-03-29T08:03:44.038 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/ 2024-03-29T08:03:44.038 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 532161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-29T08:03:44.038 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/ 2024-03-29T08:03:44.038 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-29T08:03:44.038 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/ 2024-03-29T08:03:44.038 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/ 2024-03-29T08:03:44.038 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 2712385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-29T08:03:44.039 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/ 2024-03-29T08:03:44.039 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-29T08:03:44.039 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-29T08:03:44.039 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/ 2024-03-29T08:03:44.039 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 134705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-29T08:03:44.039 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/ 2024-03-29T08:03:44.039 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 91281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-29T08:03:44.040 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/ 2024-03-29T08:03:44.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 248329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-29T08:03:44.040 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ 2024-03-29T08:03:44.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 162737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-29T08:03:44.040 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/ 2024-03-29T08:03:44.040 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 4035417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-29T08:03:44.048 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/ 2024-03-29T08:03:44.048 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/locking/ 2024-03-29T08:03:44.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 80481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/locking/locktorture.ko 2024-03-29T08:03:44.049 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/time/ 2024-03-29T08:03:44.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/time/test_udelay.ko 2024-03-29T08:03:44.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 60257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-29T08:03:44.049 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ 2024-03-29T08:03:44.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-29T08:03:44.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc7.ko 2024-03-29T08:03:44.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-29T08:03:44.049 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/ 2024-03-29T08:03:44.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-29T08:03:44.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-29T08:03:44.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/interval_tree_test.ko 2024-03-29T08:03:44.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-29T08:03:44.049 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-29T08:03:44.049 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/ 2024-03-29T08:03:44.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-29T08:03:44.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-29T08:03:44.050 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/ 2024-03-29T08:03:44.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-29T08:03:44.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/memory-notifier-error-inject.ko 2024-03-29T08:03:44.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/notifier-error-inject.ko 2024-03-29T08:03:44.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/percpu_test.ko 2024-03-29T08:03:44.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/pm-notifier-error-inject.ko 2024-03-29T08:03:44.050 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/ 2024-03-29T08:03:44.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 143257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-29T08:03:44.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/rbtree_test.ko 2024-03-29T08:03:44.050 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/ 2024-03-29T08:03:44.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-29T08:03:44.050 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test-kstrtox.ko 2024-03-29T08:03:44.051 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test-string_helpers.ko 2024-03-29T08:03:44.051 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 4863217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test_bpf.ko 2024-03-29T08:03:44.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test_firmware.ko 2024-03-29T08:03:44.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test_module.ko 2024-03-29T08:03:44.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test_user_copy.ko 2024-03-29T08:03:44.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-29T08:03:44.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-29T08:03:44.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-29T08:03:44.054 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/xz/ 2024-03-29T08:03:44.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/xz/xz_dec_test.ko 2024-03-29T08:03:44.054 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/ 2024-03-29T08:03:44.054 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-29T08:03:44.054 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ 2024-03-29T08:03:44.054 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/ 2024-03-29T08:03:44.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-29T08:03:44.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-29T08:03:44.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-29T08:03:44.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-29T08:03:44.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-29T08:03:44.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-29T08:03:44.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 6897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-29T08:03:44.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-29T08:03:44.055 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/ 2024-03-29T08:03:44.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-29T08:03:44.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-29T08:03:44.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-29T08:03:44.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-29T08:03:44.055 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-29T08:03:44.055 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/ 2024-03-29T08:03:44.056 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 82865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-29T08:03:44.056 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/ 2024-03-29T08:03:44.056 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 198809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-29T08:03:44.056 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-29T08:03:44.056 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-29T08:03:44.056 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-29T08:03:44.056 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/ 2024-03-29T08:03:44.056 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 667353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-29T08:03:44.058 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/ 2024-03-29T08:03:44.058 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-29T08:03:44.058 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 790545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-29T08:03:44.058 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ 2024-03-29T08:03:44.058 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-29T08:03:44.058 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-29T08:03:44.058 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-29T08:03:44.058 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-29T08:03:44.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-29T08:03:44.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-29T08:03:44.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-29T08:03:44.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-29T08:03:44.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-29T08:03:44.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-29T08:03:44.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-29T08:03:44.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-29T08:03:44.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-29T08:03:44.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-29T08:03:44.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-29T08:03:44.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-29T08:03:44.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-29T08:03:44.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-29T08:03:44.059 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-29T08:03:44.060 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-29T08:03:44.060 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-29T08:03:44.060 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-29T08:03:44.060 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-29T08:03:44.060 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/ 2024-03-29T08:03:44.060 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1046689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-29T08:03:44.087 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/ 2024-03-29T08:03:44.087 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 125697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-29T08:03:44.087 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/ 2024-03-29T08:03:44.087 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 194441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-29T08:03:44.087 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-29T08:03:44.087 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 58737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-29T08:03:44.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-29T08:03:44.088 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/ 2024-03-29T08:03:44.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 390033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-29T08:03:44.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-29T08:03:44.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-29T08:03:44.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-29T08:03:44.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-29T08:03:44.088 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/hsr/ 2024-03-29T08:03:44.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/hsr/hsr.ko 2024-03-29T08:03:44.088 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ 2024-03-29T08:03:44.088 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ 2024-03-29T08:03:44.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-29T08:03:44.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 284665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-29T08:03:44.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 47873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-29T08:03:44.088 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ 2024-03-29T08:03:44.088 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-29T08:03:44.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-29T08:03:44.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-29T08:03:44.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-29T08:03:44.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 51209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-29T08:03:44.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-29T08:03:44.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 61505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-29T08:03:44.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-29T08:03:44.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-29T08:03:44.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-29T08:03:44.089 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ 2024-03-29T08:03:44.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-29T08:03:44.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-29T08:03:44.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-29T08:03:44.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ip_tables.ko 2024-03-29T08:03:44.089 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-29T08:03:44.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-03-29T08:03:44.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-29T08:03:44.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-29T08:03:44.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-29T08:03:44.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-03-29T08:03:44.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-29T08:03:44.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-29T08:03:44.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-29T08:03:44.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-29T08:03:44.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-29T08:03:44.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-29T08:03:44.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-29T08:03:44.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-29T08:03:44.090 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-29T08:03:44.091 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-03-29T08:03:44.091 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-29T08:03:44.091 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-29T08:03:44.091 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-29T08:03:44.091 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-29T08:03:44.091 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-29T08:03:44.091 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-29T08:03:44.091 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-29T08:03:44.091 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-29T08:03:44.091 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-29T08:03:44.091 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-29T08:03:44.091 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-29T08:03:44.091 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-29T08:03:44.091 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-29T08:03:44.091 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-29T08:03:44.091 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-29T08:03:44.092 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-29T08:03:44.092 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-29T08:03:44.092 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-29T08:03:44.092 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-29T08:03:44.092 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-29T08:03:44.092 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 45041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-29T08:03:44.092 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-29T08:03:44.092 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ 2024-03-29T08:03:44.092 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-29T08:03:44.092 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-29T08:03:44.092 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-29T08:03:44.092 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 79921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_gre.ko 2024-03-29T08:03:44.092 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 94161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-29T08:03:44.092 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-29T08:03:44.092 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 54265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-29T08:03:44.093 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-29T08:03:44.093 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/mip6.ko 2024-03-29T08:03:44.093 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ 2024-03-29T08:03:44.093 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-29T08:03:44.093 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-03-29T08:03:44.093 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-29T08:03:44.093 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-29T08:03:44.093 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-29T08:03:44.093 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-29T08:03:44.093 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-29T08:03:44.093 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-29T08:03:44.093 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-29T08:03:44.093 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-29T08:03:44.093 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-29T08:03:44.093 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-29T08:03:44.094 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-29T08:03:44.094 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-29T08:03:44.094 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-29T08:03:44.094 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-29T08:03:44.094 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-29T08:03:44.094 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-29T08:03:44.094 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-29T08:03:44.094 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-29T08:03:44.094 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-29T08:03:44.094 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-29T08:03:44.094 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-29T08:03:44.094 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-29T08:03:44.094 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-29T08:03:44.094 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 74873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-29T08:03:44.094 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-29T08:03:44.095 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-29T08:03:44.095 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/ 2024-03-29T08:03:44.095 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 84393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-29T08:03:44.095 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/ 2024-03-29T08:03:44.095 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 125201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-29T08:03:44.095 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-29T08:03:44.095 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 25809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-29T08:03:44.095 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-29T08:03:44.095 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-29T08:03:44.095 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-29T08:03:44.095 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 53225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-29T08:03:44.095 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/lapb/ 2024-03-29T08:03:44.095 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/lapb/lapb.ko 2024-03-29T08:03:44.095 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/ 2024-03-29T08:03:44.095 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-29T08:03:44.095 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 268449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc2.ko 2024-03-29T08:03:44.096 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/ 2024-03-29T08:03:44.096 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 309361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-29T08:03:44.096 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/ 2024-03-29T08:03:44.096 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-29T08:03:44.096 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ 2024-03-29T08:03:44.096 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ 2024-03-29T08:03:44.096 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 119689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-29T08:03:44.096 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-29T08:03:44.096 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-29T08:03:44.096 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-29T08:03:44.096 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 99985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-29T08:03:44.096 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 100273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-29T08:03:44.096 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-29T08:03:44.096 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 102337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-29T08:03:44.096 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 111665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-29T08:03:44.097 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 59185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-29T08:03:44.097 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-29T08:03:44.097 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 112137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-29T08:03:44.097 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 112401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-29T08:03:44.097 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 110329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-29T08:03:44.097 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 113281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-29T08:03:44.097 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-29T08:03:44.097 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ 2024-03-29T08:03:44.097 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 412433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-29T08:03:44.097 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-29T08:03:44.097 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-29T08:03:44.097 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-29T08:03:44.097 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-29T08:03:44.097 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-29T08:03:44.097 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-29T08:03:44.098 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-29T08:03:44.098 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-29T08:03:44.098 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-29T08:03:44.098 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-29T08:03:44.098 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-29T08:03:44.098 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-29T08:03:44.098 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-29T08:03:44.098 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-29T08:03:44.098 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 411665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-29T08:03:44.098 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-29T08:03:44.098 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-29T08:03:44.098 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-29T08:03:44.098 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 118145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-29T08:03:44.098 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-29T08:03:44.098 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-29T08:03:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 108305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-29T08:03:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-29T08:03:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-29T08:03:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-29T08:03:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-29T08:03:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-29T08:03:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-29T08:03:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 92217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-29T08:03:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-29T08:03:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-29T08:03:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 96489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-29T08:03:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-29T08:03:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-29T08:03:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-29T08:03:44.099 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-29T08:03:44.100 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-29T08:03:44.100 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 44129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-29T08:03:44.100 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 623337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-29T08:03:44.100 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 42505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-29T08:03:44.100 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-29T08:03:44.100 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-03-29T08:03:44.100 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-29T08:03:44.100 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 22929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-29T08:03:44.100 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-29T08:03:44.100 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-29T08:03:44.100 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-29T08:03:44.100 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 40801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-29T08:03:44.100 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-29T08:03:44.100 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-29T08:03:44.100 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-29T08:03:44.101 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-29T08:03:44.101 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-29T08:03:44.101 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-29T08:03:44.101 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-29T08:03:44.101 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-29T08:03:44.101 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-29T08:03:44.101 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_osf.ko 2024-03-29T08:03:44.101 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-29T08:03:44.101 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-29T08:03:44.101 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-29T08:03:44.101 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-29T08:03:44.101 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-29T08:03:44.101 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_synproxy.ko 2024-03-29T08:03:44.101 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tproxy.ko 2024-03-29T08:03:44.101 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 89225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/x_tables.ko 2024-03-29T08:03:44.102 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-29T08:03:44.102 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-29T08:03:44.102 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-29T08:03:44.102 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-29T08:03:44.102 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 18073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-29T08:03:44.102 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-29T08:03:44.102 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-29T08:03:44.102 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-29T08:03:44.102 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 38769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-29T08:03:44.102 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-29T08:03:44.102 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-29T08:03:44.102 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-29T08:03:44.102 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-29T08:03:44.102 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-29T08:03:44.102 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-29T08:03:44.103 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-29T08:03:44.103 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-29T08:03:44.103 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-29T08:03:44.103 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-29T08:03:44.103 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-29T08:03:44.103 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-29T08:03:44.103 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-29T08:03:44.103 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-29T08:03:44.103 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-29T08:03:44.103 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-29T08:03:44.103 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-29T08:03:44.103 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-29T08:03:44.103 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-29T08:03:44.103 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-29T08:03:44.103 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-29T08:03:44.104 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-29T08:03:44.104 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-29T08:03:44.104 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-29T08:03:44.104 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-29T08:03:44.104 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-29T08:03:44.104 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-29T08:03:44.104 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-29T08:03:44.104 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-29T08:03:44.104 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-29T08:03:44.104 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-29T08:03:44.104 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-29T08:03:44.104 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-29T08:03:44.104 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-29T08:03:44.104 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-29T08:03:44.104 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-29T08:03:44.104 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-29T08:03:44.105 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-29T08:03:44.105 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-29T08:03:44.105 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-29T08:03:44.105 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-29T08:03:44.105 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-29T08:03:44.105 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-29T08:03:44.105 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-29T08:03:44.105 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-29T08:03:44.105 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-29T08:03:44.105 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-29T08:03:44.105 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-29T08:03:44.105 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-29T08:03:44.105 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-29T08:03:44.105 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-29T08:03:44.105 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-29T08:03:44.106 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-29T08:03:44.106 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-29T08:03:44.106 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 29993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-29T08:03:44.106 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 16809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-29T08:03:44.106 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-29T08:03:44.106 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-29T08:03:44.106 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-29T08:03:44.106 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 8449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-29T08:03:44.106 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpudp.ko 2024-03-29T08:03:44.106 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-29T08:03:44.106 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-29T08:03:44.106 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/ 2024-03-29T08:03:44.106 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 14433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-29T08:03:44.106 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/ 2024-03-29T08:03:44.107 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 12329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-29T08:03:44.107 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/ 2024-03-29T08:03:44.107 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 384137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-29T08:03:44.107 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-29T08:03:44.107 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-29T08:03:44.107 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-29T08:03:44.107 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/ 2024-03-29T08:03:44.107 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-29T08:03:44.107 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/phonet/ 2024-03-29T08:03:44.107 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 112521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/phonet/phonet.ko 2024-03-29T08:03:44.107 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/phonet/pn_pep.ko 2024-03-29T08:03:44.107 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/ 2024-03-29T08:03:44.107 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 253881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-29T08:03:44.107 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 165049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-29T08:03:44.108 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 68065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-29T08:03:44.108 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rxrpc/ 2024-03-29T08:03:44.108 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 828145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rxrpc/rxrpc.ko 2024-03-29T08:03:44.108 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/ 2024-03-29T08:03:44.108 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-29T08:03:44.108 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-29T08:03:44.108 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 37081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-29T08:03:44.108 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-29T08:03:44.108 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 43345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-29T08:03:44.108 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-29T08:03:44.108 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-29T08:03:44.108 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-29T08:03:44.108 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-29T08:03:44.108 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-29T08:03:44.108 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-29T08:03:44.108 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 19177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_cgroup.ko 2024-03-29T08:03:44.109 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-29T08:03:44.109 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-29T08:03:44.109 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 39513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-29T08:03:44.109 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 62649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-29T08:03:44.109 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-29T08:03:44.109 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 13313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-29T08:03:44.109 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 33417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-29T08:03:44.109 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-29T08:03:44.109 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-29T08:03:44.109 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-29T08:03:44.109 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-29T08:03:44.109 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-29T08:03:44.109 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 34585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-29T08:03:44.109 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-29T08:03:44.109 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq_codel.ko 2024-03-29T08:03:44.110 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 35449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-29T08:03:44.110 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 49841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-29T08:03:44.110 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 21969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-29T08:03:44.110 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 66441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-29T08:03:44.110 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-29T08:03:44.110 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 30217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-29T08:03:44.110 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-29T08:03:44.110 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-29T08:03:44.110 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 36145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-29T08:03:44.110 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 27401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-29T08:03:44.110 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-29T08:03:44.110 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 24769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-29T08:03:44.110 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 48529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-29T08:03:44.110 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 32809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-29T08:03:44.110 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 26785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-29T08:03:44.111 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 31313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-29T08:03:44.111 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-29T08:03:44.111 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 28265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-29T08:03:44.111 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/ 2024-03-29T08:03:44.111 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 932785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-29T08:03:44.115 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-29T08:03:44.115 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/ 2024-03-29T08:03:44.115 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/ 2024-03-29T08:03:44.115 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 336969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-29T08:03:44.115 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 55009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-29T08:03:44.115 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 1513769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-29T08:03:44.115 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/ 2024-03-29T08:03:44.115 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 704697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-29T08:03:44.115 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/ 2024-03-29T08:03:44.116 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 10393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-29T08:03:44.116 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 867121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-29T08:03:44.116 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/ 2024-03-29T08:03:44.116 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-29T08:03:44.116 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/ 2024-03-29T08:03:44.116 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 87737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-29T08:03:44.116 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 67937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-29T08:03:44.116 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 98089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-29T08:03:44.116 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 11073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-29T08:03:44.116 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 17809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-29T08:03:44.116 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/x25/ 2024-03-29T08:03:44.116 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 131209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/x25/x25.ko 2024-03-29T08:03:44.116 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/ 2024-03-29T08:03:44.116 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_algo.ko 2024-03-29T08:03:44.116 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 20545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-29T08:03:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 105209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_user.ko 2024-03-29T08:03:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/ 2024-03-29T08:03:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/ 2024-03-29T08:03:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 15433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-29T08:03:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 642383 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-29T08:03:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 625850 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-29T08:03:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 7677 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-29T08:03:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 23498 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-29T08:03:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 9916 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-29T08:03:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 70732 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-29T08:03:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 147214 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-29T08:03:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 233213 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-29T08:03:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 210 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-29T08:03:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 72614 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-29T08:03:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 879 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-29T08:03:44.117 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 257955 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-29T08:03:44.118 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 312782 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-29T08:03:44.118 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./usr/ 2024-03-29T08:03:44.118 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./usr/share/ 2024-03-29T08:03:44.118 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./usr/share/doc/ 2024-03-29T08:03:44.118 INFO:teuthology.orchestra.run.smithi046.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./usr/share/doc/linux-image-6.8.0-g1df0c7e8a235/ 2024-03-29T08:03:44.118 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 169 2024-03-25 03:29 ./usr/share/doc/linux-image-6.8.0-g1df0c7e8a235/changelog.Debian.gz 2024-03-29T08:03:44.118 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r-- root/root 691 2024-03-25 03:29 ./usr/share/doc/linux-image-6.8.0-g1df0c7e8a235/copyright 2024-03-29T08:03:44.123 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-29T08:03:44.123 INFO:teuthology.task.kernel:searching for kernel 6.8.0-g1df0c7e8a235 2024-03-29T08:03:44.123 DEBUG:teuthology.orchestra.run.smithi046:> egrep '(submenu|menuentry.*6.8.0-g1df0c7e8a235).*{' /boot/grub/grub.cfg 2024-03-29T08:03:44.164 INFO:teuthology.orchestra.run.smithi046.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-29T08:03:44.164 INFO:teuthology.orchestra.run.smithi046.stdout: menuentry 'Ubuntu, with Linux 6.8.0-g1df0c7e8a235' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.8.0-g1df0c7e8a235-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-29T08:03:44.164 INFO:teuthology.orchestra.run.smithi046.stdout: menuentry 'Ubuntu, with Linux 6.8.0-g1df0c7e8a235 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.8.0-g1df0c7e8a235-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-29T08:03:44.165 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-03-29T08:03:44.165 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.8.0-g1df0c7e8a235 2024-03-29T08:03:44.165 DEBUG:teuthology.orchestra.run.smithi046:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-03-29T08:03:44.208 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi046.front.sepia.ceph.com to complete... 2024-03-29T08:03:44.706 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub' 2024-03-29T08:03:44.707 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-29T08:03:44.718 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-03-29T08:03:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-6.8.0-g1df0c7e8a235 2024-03-29T08:03:44.972 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-6.8.0-g1df0c7e8a235 2024-03-29T08:03:45.323 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-29T08:03:45.329 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-29T08:03:45.374 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-29T08:03:45.379 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-29T08:03:45.521 INFO:teuthology.orchestra.run.smithi046.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-29T08:03:45.521 INFO:teuthology.orchestra.run.smithi046.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-29T08:03:45.521 INFO:teuthology.orchestra.run.smithi046.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-29T08:03:45.539 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-03-29T08:03:51.624 INFO:teuthology.orchestra.run.smithi073.stderr:Sourcing file `/etc/default/grub' 2024-03-29T08:03:51.638 INFO:teuthology.orchestra.run.smithi073.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-29T08:03:51.707 INFO:teuthology.orchestra.run.smithi073.stderr:Generating grub configuration file ... 2024-03-29T08:03:52.089 INFO:teuthology.orchestra.run.smithi073.stderr:Found linux image: /boot/vmlinuz-6.8.0-g1df0c7e8a235 2024-03-29T08:03:52.108 INFO:teuthology.orchestra.run.smithi073.stderr:Found initrd image: /boot/initrd.img-6.8.0-g1df0c7e8a235 2024-03-29T08:03:52.571 INFO:teuthology.orchestra.run.smithi073.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-29T08:03:52.578 INFO:teuthology.orchestra.run.smithi073.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-29T08:03:52.641 INFO:teuthology.orchestra.run.smithi073.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-29T08:03:52.647 INFO:teuthology.orchestra.run.smithi073.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-29T08:03:52.781 INFO:teuthology.orchestra.run.smithi073.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-29T08:03:52.781 INFO:teuthology.orchestra.run.smithi073.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-29T08:03:52.781 INFO:teuthology.orchestra.run.smithi073.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-29T08:03:52.817 INFO:teuthology.orchestra.run.smithi073.stderr:done 2024-03-29T08:03:53.005 DEBUG:teuthology.orchestra.run.smithi073:> dpkg-deb -c /tmp/linux-image.deb 2024-03-29T08:03:53.013 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./ 2024-03-29T08:03:53.013 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./boot/ 2024-03-29T08:03:53.013 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 4474334 2024-03-25 03:29 ./boot/System.map-6.8.0-g1df0c7e8a235 2024-03-29T08:03:53.025 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 170899 2024-03-25 03:29 ./boot/config-6.8.0-g1df0c7e8a235 2024-03-29T08:03:53.025 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11551744 2024-03-25 03:29 ./boot/vmlinuz-6.8.0-g1df0c7e8a235 2024-03-29T08:03:53.045 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/ 2024-03-29T08:03:53.045 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/ 2024-03-29T08:03:53.045 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/postinst.d/ 2024-03-29T08:03:53.045 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/postrm.d/ 2024-03-29T08:03:53.045 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/preinst.d/ 2024-03-29T08:03:53.045 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./etc/kernel/prerm.d/ 2024-03-29T08:03:53.045 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/ 2024-03-29T08:03:53.046 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/ 2024-03-29T08:03:53.046 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/ 2024-03-29T08:03:53.046 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/ 2024-03-29T08:03:53.046 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/ 2024-03-29T08:03:53.046 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/ 2024-03-29T08:03:53.046 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ 2024-03-29T08:03:53.046 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 490441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/aesni-intel.ko 2024-03-29T08:03:53.048 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-03-29T08:03:53.049 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-03-29T08:03:53.049 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-03-29T08:03:53.049 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 93073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-03-29T08:03:53.049 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 106689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-03-29T08:03:53.049 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 152137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-03-29T08:03:53.049 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-03-29T08:03:53.049 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-03-29T08:03:53.049 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 83849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-03-29T08:03:53.049 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-03-29T08:03:53.049 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 47177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-03-29T08:03:53.049 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 47169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-avx2.ko 2024-03-29T08:03:53.049 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 51689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-03-29T08:03:53.049 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-03-29T08:03:53.050 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 42529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-03-29T08:03:53.050 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 58481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-03-29T08:03:53.050 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 50385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-03-29T08:03:53.050 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-03-29T08:03:53.050 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-03-29T08:03:53.050 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/ 2024-03-29T08:03:53.050 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/ 2024-03-29T08:03:53.050 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/ 2024-03-29T08:03:53.050 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 41425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-03-29T08:03:53.050 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/cpuid.ko 2024-03-29T08:03:53.050 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kernel/msr.ko 2024-03-29T08:03:53.050 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/ 2024-03-29T08:03:53.050 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 354225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-amd.ko 2024-03-29T08:03:53.050 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 886129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm-intel.ko 2024-03-29T08:03:53.051 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 2294793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/arch/x86/kvm/kvm.ko 2024-03-29T08:03:53.063 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ 2024-03-29T08:03:53.063 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 53833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/af_alg.ko 2024-03-29T08:03:53.063 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/algif_hash.ko 2024-03-29T08:03:53.063 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/algif_skcipher.ko 2024-03-29T08:03:53.063 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ansi_cprng.ko 2024-03-29T08:03:53.063 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/anubis.ko 2024-03-29T08:03:53.063 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/arc4.ko 2024-03-29T08:03:53.064 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/asymmetric_keys/ 2024-03-29T08:03:53.064 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-03-29T08:03:53.064 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/ 2024-03-29T08:03:53.064 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_memcpy.ko 2024-03-29T08:03:53.064 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_pq.ko 2024-03-29T08:03:53.064 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_raid6_recov.ko 2024-03-29T08:03:53.064 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_tx.ko 2024-03-29T08:03:53.064 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/async_xor.ko 2024-03-29T08:03:53.064 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/async_tx/raid6test.ko 2024-03-29T08:03:53.064 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authenc.ko 2024-03-29T08:03:53.064 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/authencesn.ko 2024-03-29T08:03:53.064 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blake2b_generic.ko 2024-03-29T08:03:53.064 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_common.ko 2024-03-29T08:03:53.064 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/blowfish_generic.ko 2024-03-29T08:03:53.064 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 38705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/camellia_generic.ko 2024-03-29T08:03:53.065 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast5_generic.ko 2024-03-29T08:03:53.065 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast6_generic.ko 2024-03-29T08:03:53.065 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cast_common.ko 2024-03-29T08:03:53.065 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 35177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/ccm.ko 2024-03-29T08:03:53.065 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cmac.ko 2024-03-29T08:03:53.065 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crc32_generic.ko 2024-03-29T08:03:53.065 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 47673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cryptd.ko 2024-03-29T08:03:53.065 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_engine.ko 2024-03-29T08:03:53.065 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_simd.ko 2024-03-29T08:03:53.065 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/crypto_user.ko 2024-03-29T08:03:53.065 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/cts.ko 2024-03-29T08:03:53.065 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/des_generic.ko 2024-03-29T08:03:53.065 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/dh_generic.ko 2024-03-29T08:03:53.065 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/echainiv.ko 2024-03-29T08:03:53.066 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/essiv.ko 2024-03-29T08:03:53.066 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/fcrypt.ko 2024-03-29T08:03:53.066 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/khazad.ko 2024-03-29T08:03:53.066 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lrw.ko 2024-03-29T08:03:53.066 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4.ko 2024-03-29T08:03:53.066 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/lz4hc.ko 2024-03-29T08:03:53.066 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/md4.ko 2024-03-29T08:03:53.066 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/michael_mic.ko 2024-03-29T08:03:53.066 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcbc.ko 2024-03-29T08:03:53.066 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/pcrypt.ko 2024-03-29T08:03:53.066 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/rmd160.ko 2024-03-29T08:03:53.066 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/seed.ko 2024-03-29T08:03:53.066 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/serpent_generic.ko 2024-03-29T08:03:53.066 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 143025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tcrypt.ko 2024-03-29T08:03:53.066 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/tea.ko 2024-03-29T08:03:53.067 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_common.ko 2024-03-29T08:03:53.067 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/twofish_generic.ko 2024-03-29T08:03:53.067 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/vmac.ko 2024-03-29T08:03:53.067 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/wp512.ko 2024-03-29T08:03:53.067 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xcbc.ko 2024-03-29T08:03:53.067 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xor.ko 2024-03-29T08:03:53.067 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xts.ko 2024-03-29T08:03:53.067 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/crypto/xxhash_generic.ko 2024-03-29T08:03:53.067 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ 2024-03-29T08:03:53.067 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ 2024-03-29T08:03:53.067 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_extlog.ko 2024-03-29T08:03:53.067 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_ipmi.ko 2024-03-29T08:03:53.067 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/acpi_pad.ko 2024-03-29T08:03:53.067 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/apei/ 2024-03-29T08:03:53.067 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/apei/einj.ko 2024-03-29T08:03:53.068 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/ec_sys.ko 2024-03-29T08:03:53.068 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/platform_profile.ko 2024-03-29T08:03:53.068 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbs.ko 2024-03-29T08:03:53.068 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/sbshc.ko 2024-03-29T08:03:53.068 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 104417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/acpi/video.ko 2024-03-29T08:03:53.068 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ 2024-03-29T08:03:53.068 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/acard-ahci.ko 2024-03-29T08:03:53.068 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 89929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci.ko 2024-03-29T08:03:53.068 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/ahci_platform.ko 2024-03-29T08:03:53.068 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 88305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci.ko 2024-03-29T08:03:53.068 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/libahci_platform.ko 2024-03-29T08:03:53.068 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_acpi.ko 2024-03-29T08:03:53.068 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ali.ko 2024-03-29T08:03:53.068 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_amd.ko 2024-03-29T08:03:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_artop.ko 2024-03-29T08:03:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atiixp.ko 2024-03-29T08:03:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_atp867x.ko 2024-03-29T08:03:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd640.ko 2024-03-29T08:03:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cmd64x.ko 2024-03-29T08:03:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_cypress.ko 2024-03-29T08:03:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_efar.ko 2024-03-29T08:03:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt366.ko 2024-03-29T08:03:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt37x.ko 2024-03-29T08:03:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x2n.ko 2024-03-29T08:03:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_hpt3x3.ko 2024-03-29T08:03:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it8213.ko 2024-03-29T08:03:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 35169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_it821x.ko 2024-03-29T08:03:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_jmicron.ko 2024-03-29T08:03:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 52169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_legacy.ko 2024-03-29T08:03:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_marvell.ko 2024-03-29T08:03:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_mpiix.ko 2024-03-29T08:03:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_netcell.ko 2024-03-29T08:03:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ninja32.ko 2024-03-29T08:03:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87410.ko 2024-03-29T08:03:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_ns87415.ko 2024-03-29T08:03:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_oldpiix.ko 2024-03-29T08:03:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_opti.ko 2024-03-29T08:03:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_optidma.ko 2024-03-29T08:03:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 32481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pcmcia.ko 2024-03-29T08:03:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 37689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc2027x.ko 2024-03-29T08:03:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-03-29T08:03:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_piccolo.ko 2024-03-29T08:03:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_radisys.ko 2024-03-29T08:03:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rdc.ko 2024-03-29T08:03:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_rz1000.ko 2024-03-29T08:03:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sch.ko 2024-03-29T08:03:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_serverworks.ko 2024-03-29T08:03:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sil680.ko 2024-03-29T08:03:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_sl82c105.ko 2024-03-29T08:03:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_triflex.ko 2024-03-29T08:03:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pata_via.ko 2024-03-29T08:03:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/pdc_adma.ko 2024-03-29T08:03:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_inic162x.ko 2024-03-29T08:03:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 83249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_mv.ko 2024-03-29T08:03:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 70297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_nv.ko 2024-03-29T08:03:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 37273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_promise.ko 2024-03-29T08:03:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_qstor.ko 2024-03-29T08:03:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil.ko 2024-03-29T08:03:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 38169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sil24.ko 2024-03-29T08:03:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sis.ko 2024-03-29T08:03:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_svw.ko 2024-03-29T08:03:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 44697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_sx4.ko 2024-03-29T08:03:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_uli.ko 2024-03-29T08:03:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_via.ko 2024-03-29T08:03:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ata/sata_vsc.ko 2024-03-29T08:03:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ 2024-03-29T08:03:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/auxdisplay/ks0108.ko 2024-03-29T08:03:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/ 2024-03-29T08:03:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 124385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/bcma/bcma.ko 2024-03-29T08:03:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/ 2024-03-29T08:03:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/ 2024-03-29T08:03:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 113801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/aoe/aoe.ko 2024-03-29T08:03:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/ 2024-03-29T08:03:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 964745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/drbd/drbd.ko 2024-03-29T08:03:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 169697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/floppy.ko 2024-03-29T08:03:53.073 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/ 2024-03-29T08:03:53.073 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 94209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-03-29T08:03:53.073 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 116329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/nbd.ko 2024-03-29T08:03:53.122 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/ 2024-03-29T08:03:53.122 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 110321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/null_blk/null_blk.ko 2024-03-29T08:03:53.122 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 98185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/pktcdvd.ko 2024-03-29T08:03:53.122 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 263633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/rbd.ko 2024-03-29T08:03:53.122 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/ 2024-03-29T08:03:53.122 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 98105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-03-29T08:03:53.122 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/ 2024-03-29T08:03:53.123 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 58761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/block/zram/zram.ko 2024-03-29T08:03:53.123 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ 2024-03-29T08:03:53.123 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/applicom.ko 2024-03-29T08:03:53.123 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hangcheck-timer.ko 2024-03-29T08:03:53.123 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/ 2024-03-29T08:03:53.123 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/amd-rng.ko 2024-03-29T08:03:53.123 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/intel-rng.ko 2024-03-29T08:03:53.123 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-03-29T08:03:53.123 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/via-rng.ko 2024-03-29T08:03:53.123 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/hw_random/virtio-rng.ko 2024-03-29T08:03:53.123 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ 2024-03-29T08:03:53.123 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-03-29T08:03:53.123 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 148513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-03-29T08:03:53.123 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-03-29T08:03:53.123 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 139305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_si.ko 2024-03-29T08:03:53.124 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 70865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-03-29T08:03:53.124 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 53409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-03-29T08:03:53.124 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/lp.ko 2024-03-29T08:03:53.124 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/ 2024-03-29T08:03:53.124 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 84001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/mwave/mwave.ko 2024-03-29T08:03:53.124 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/nvram.ko 2024-03-29T08:03:53.124 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 32073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/ppdev.ko 2024-03-29T08:03:53.124 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 51017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tlclk.ko 2024-03-29T08:03:53.124 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/ 2024-03-29T08:03:53.124 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_atmel.ko 2024-03-29T08:03:53.124 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-03-29T08:03:53.124 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-03-29T08:03:53.124 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 32761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-03-29T08:03:53.124 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_infineon.ko 2024-03-29T08:03:53.125 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/tpm_nsc.ko 2024-03-29T08:03:53.125 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-03-29T08:03:53.125 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/xillybus/ 2024-03-29T08:03:53.125 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/xillybus/xillybus_class.ko 2024-03-29T08:03:53.125 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 50841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/xillybus/xillybus_core.ko 2024-03-29T08:03:53.125 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-03-29T08:03:53.125 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/ 2024-03-29T08:03:53.125 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/clk-palmas.ko 2024-03-29T08:03:53.125 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 47721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/clk-si5351.ko 2024-03-29T08:03:53.125 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/clk-twl6040.ko 2024-03-29T08:03:53.125 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/clk/clk-wm831x.ko 2024-03-29T08:03:53.125 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/ 2024-03-29T08:03:53.125 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-03-29T08:03:53.125 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/p4-clockmod.ko 2024-03-29T08:03:53.125 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/cpufreq/speedstep-lib.ko 2024-03-29T08:03:53.126 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ 2024-03-29T08:03:53.126 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ 2024-03-29T08:03:53.126 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 70425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-03-29T08:03:53.126 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 246777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/ccp/ccp.ko 2024-03-29T08:03:53.126 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/ 2024-03-29T08:03:53.126 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/ 2024-03-29T08:03:53.126 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/ 2024-03-29T08:03:53.126 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 570217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-03-29T08:03:53.126 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-03-29T08:03:53.126 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-03-29T08:03:53.126 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-aes.ko 2024-03-29T08:03:53.126 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/padlock-sha.ko 2024-03-29T08:03:53.126 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/ 2024-03-29T08:03:53.126 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 70409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-03-29T08:03:53.127 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/ 2024-03-29T08:03:53.127 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dca/dca.ko 2024-03-29T08:03:53.127 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ 2024-03-29T08:03:53.127 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/ 2024-03-29T08:03:53.127 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/dw/dw_dmac.ko 2024-03-29T08:03:53.127 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ 2024-03-29T08:03:53.127 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 122793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/dma/ioat/ioatdma.ko 2024-03-29T08:03:53.127 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ 2024-03-29T08:03:53.127 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 80297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/amd64_edac.ko 2024-03-29T08:03:53.127 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 35665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/e752x_edac.ko 2024-03-29T08:03:53.127 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 41633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/edac_mce_amd.ko 2024-03-29T08:03:53.127 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3000_edac.ko 2024-03-29T08:03:53.127 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i3200_edac.ko 2024-03-29T08:03:53.127 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5100_edac.ko 2024-03-29T08:03:53.127 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i5400_edac.ko 2024-03-29T08:03:53.128 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7300_edac.ko 2024-03-29T08:03:53.128 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 51937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i7core_edac.ko 2024-03-29T08:03:53.128 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/i82975x_edac.ko 2024-03-29T08:03:53.128 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/ie31200_edac.ko 2024-03-29T08:03:53.128 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 56729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/sb_edac.ko 2024-03-29T08:03:53.128 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/edac/x38_edac.ko 2024-03-29T08:03:53.128 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/ 2024-03-29T08:03:53.128 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-gpio.ko 2024-03-29T08:03:53.128 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-max14577.ko 2024-03-29T08:03:53.128 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-max77693.ko 2024-03-29T08:03:53.128 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-max8997.ko 2024-03-29T08:03:53.128 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-palmas.ko 2024-03-29T08:03:53.128 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-rt8973a.ko 2024-03-29T08:03:53.128 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/extcon/extcon-sm5502.ko 2024-03-29T08:03:53.128 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/ 2024-03-29T08:03:53.129 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/dmi-sysfs.ko 2024-03-29T08:03:53.129 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/efi/ 2024-03-29T08:03:53.129 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/efi/efi-pstore.ko 2024-03-29T08:03:53.129 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/firmware/iscsi_ibft.ko 2024-03-29T08:03:53.129 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/ 2024-03-29T08:03:53.129 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-adp5520.ko 2024-03-29T08:03:53.129 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-amd8111.ko 2024-03-29T08:03:53.129 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-arizona.ko 2024-03-29T08:03:53.129 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-da9052.ko 2024-03-29T08:03:53.129 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-da9055.ko 2024-03-29T08:03:53.129 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-dln2.ko 2024-03-29T08:03:53.129 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 77745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-f7188x.ko 2024-03-29T08:03:53.129 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-generic.ko 2024-03-29T08:03:53.129 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ich.ko 2024-03-29T08:03:53.129 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-03-29T08:03:53.130 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-kempld.ko 2024-03-29T08:03:53.130 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-lp3943.ko 2024-03-29T08:03:53.130 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-max7300.ko 2024-03-29T08:03:53.130 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-max730x.ko 2024-03-29T08:03:53.130 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-max732x.ko 2024-03-29T08:03:53.130 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-03-29T08:03:53.130 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 47441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-pca953x.ko 2024-03-29T08:03:53.130 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-pcf857x.ko 2024-03-29T08:03:53.130 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-rdc321x.ko 2024-03-29T08:03:53.130 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-sch.ko 2024-03-29T08:03:53.130 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-sch311x.ko 2024-03-29T08:03:53.130 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-tps65912.ko 2024-03-29T08:03:53.130 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-twl4030.ko 2024-03-29T08:03:53.130 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-twl6040.ko 2024-03-29T08:03:53.130 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-viperboard.ko 2024-03-29T08:03:53.130 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-vx855.ko 2024-03-29T08:03:53.131 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-wm831x.ko 2024-03-29T08:03:53.131 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-wm8350.ko 2024-03-29T08:03:53.131 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/gpio/gpio-wm8994.ko 2024-03-29T08:03:53.131 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/ 2024-03-29T08:03:53.131 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-a4tech.ko 2024-03-29T08:03:53.131 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-apple.ko 2024-03-29T08:03:53.131 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-appleir.ko 2024-03-29T08:03:53.131 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-aureal.ko 2024-03-29T08:03:53.131 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-axff.ko 2024-03-29T08:03:53.131 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-belkin.ko 2024-03-29T08:03:53.131 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-cherry.ko 2024-03-29T08:03:53.131 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-chicony.ko 2024-03-29T08:03:53.131 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 60657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-cp2112.ko 2024-03-29T08:03:53.131 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-cypress.ko 2024-03-29T08:03:53.132 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-dr.ko 2024-03-29T08:03:53.132 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elecom.ko 2024-03-29T08:03:53.132 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-elo.ko 2024-03-29T08:03:53.132 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-emsff.ko 2024-03-29T08:03:53.132 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ezkey.ko 2024-03-29T08:03:53.132 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gaff.ko 2024-03-29T08:03:53.132 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-generic.ko 2024-03-29T08:03:53.132 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gt683r.ko 2024-03-29T08:03:53.132 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-gyration.ko 2024-03-29T08:03:53.132 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-kbd.ko 2024-03-29T08:03:53.132 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtek-mouse.ko 2024-03-29T08:03:53.132 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-holtekff.ko 2024-03-29T08:03:53.132 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-hyperv.ko 2024-03-29T08:03:53.132 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-icade.ko 2024-03-29T08:03:53.132 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kensington.ko 2024-03-29T08:03:53.132 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-keytouch.ko 2024-03-29T08:03:53.133 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-kye.ko 2024-03-29T08:03:53.133 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lcpower.ko 2024-03-29T08:03:53.133 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-led.ko 2024-03-29T08:03:53.133 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 42841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lenovo.ko 2024-03-29T08:03:53.133 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-lg-g15.ko 2024-03-29T08:03:53.133 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 57969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-dj.ko 2024-03-29T08:03:53.133 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 101537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-03-29T08:03:53.133 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 80897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-logitech.ko 2024-03-29T08:03:53.133 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-magicmouse.ko 2024-03-29T08:03:53.133 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-microsoft.ko 2024-03-29T08:03:53.133 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-monterey.ko 2024-03-29T08:03:53.133 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 50273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-multitouch.ko 2024-03-29T08:03:53.133 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 32489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ntrig.ko 2024-03-29T08:03:53.133 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-ortek.ko 2024-03-29T08:03:53.133 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-penmount.ko 2024-03-29T08:03:53.134 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-petalynx.ko 2024-03-29T08:03:53.134 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 72841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-picolcd.ko 2024-03-29T08:03:53.134 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-pl.ko 2024-03-29T08:03:53.134 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-plantronics.ko 2024-03-29T08:03:53.134 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-primax.ko 2024-03-29T08:03:53.134 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 35521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-rmi.ko 2024-03-29T08:03:53.134 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-arvo.ko 2024-03-29T08:03:53.134 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-common.ko 2024-03-29T08:03:53.134 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-isku.ko 2024-03-29T08:03:53.134 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kone.ko 2024-03-29T08:03:53.134 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-03-29T08:03:53.134 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-konepure.ko 2024-03-29T08:03:53.134 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-03-29T08:03:53.134 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-lua.ko 2024-03-29T08:03:53.134 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-pyra.ko 2024-03-29T08:03:53.135 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-ryos.ko 2024-03-29T08:03:53.135 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat-savu.ko 2024-03-29T08:03:53.135 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-roccat.ko 2024-03-29T08:03:53.135 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-saitek.ko 2024-03-29T08:03:53.135 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-samsung.ko 2024-03-29T08:03:53.135 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sensor-hub.ko 2024-03-29T08:03:53.135 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sjoy.ko 2024-03-29T08:03:53.135 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 51153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sony.ko 2024-03-29T08:03:53.135 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-speedlink.ko 2024-03-29T08:03:53.135 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-steelseries.ko 2024-03-29T08:03:53.135 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-sunplus.ko 2024-03-29T08:03:53.135 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-thrustmaster.ko 2024-03-29T08:03:53.135 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tivo.ko 2024-03-29T08:03:53.135 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-tmff.ko 2024-03-29T08:03:53.135 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-topseed.ko 2024-03-29T08:03:53.135 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-twinhan.ko 2024-03-29T08:03:53.136 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 70977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-uclogic.ko 2024-03-29T08:03:53.136 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-waltop.ko 2024-03-29T08:03:53.136 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 143377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-wiimote.ko 2024-03-29T08:03:53.136 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-xinmo.ko 2024-03-29T08:03:53.136 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zpff.ko 2024-03-29T08:03:53.136 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid-zydacron.ko 2024-03-29T08:03:53.136 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 269209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/hid.ko 2024-03-29T08:03:53.136 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 32225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/uhid.ko 2024-03-29T08:03:53.136 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/usbhid/ 2024-03-29T08:03:53.136 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 125625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/usbhid/usbhid.ko 2024-03-29T08:03:53.136 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/usbhid/usbkbd.ko 2024-03-29T08:03:53.136 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/usbhid/usbmouse.ko 2024-03-29T08:03:53.136 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 218865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hid/wacom.ko 2024-03-29T08:03:53.136 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/ 2024-03-29T08:03:53.137 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 64297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_balloon.ko 2024-03-29T08:03:53.137 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 90089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_utils.ko 2024-03-29T08:03:53.137 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 304913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hv/hv_vmbus.ko 2024-03-29T08:03:53.137 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ 2024-03-29T08:03:53.137 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 66793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru.ko 2024-03-29T08:03:53.137 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 61809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/abituguru3.ko 2024-03-29T08:03:53.137 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/acpi_power_meter.ko 2024-03-29T08:03:53.137 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7414.ko 2024-03-29T08:03:53.137 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ad7418.ko 2024-03-29T08:03:53.137 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adc128d818.ko 2024-03-29T08:03:53.137 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1025.ko 2024-03-29T08:03:53.137 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 97881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1026.ko 2024-03-29T08:03:53.137 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1029.ko 2024-03-29T08:03:53.137 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 45457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm1031.ko 2024-03-29T08:03:53.137 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adm9240.ko 2024-03-29T08:03:53.138 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ads7828.ko 2024-03-29T08:03:53.138 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7410.ko 2024-03-29T08:03:53.138 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7411.ko 2024-03-29T08:03:53.138 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 74313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7462.ko 2024-03-29T08:03:53.138 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7470.ko 2024-03-29T08:03:53.138 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 69305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7475.ko 2024-03-29T08:03:53.138 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/adt7x10.ko 2024-03-29T08:03:53.138 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 38977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/amc6821.ko 2024-03-29T08:03:53.138 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 55105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/applesmc.ko 2024-03-29T08:03:53.138 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 49817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asb100.ko 2024-03-29T08:03:53.138 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 58521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asc7621.ko 2024-03-29T08:03:53.138 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 40721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/asus_atk0110.ko 2024-03-29T08:03:53.138 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/atxp1.ko 2024-03-29T08:03:53.138 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/coretemp.ko 2024-03-29T08:03:53.139 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/da9052-hwmon.ko 2024-03-29T08:03:53.139 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/da9055-hwmon.ko 2024-03-29T08:03:53.139 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 49233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-03-29T08:03:53.139 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 99505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/dme1737.ko 2024-03-29T08:03:53.139 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds1621.ko 2024-03-29T08:03:53.139 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ds620.ko 2024-03-29T08:03:53.139 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc1403.ko 2024-03-29T08:03:53.139 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc2103.ko 2024-03-29T08:03:53.139 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/emc6w201.ko 2024-03-29T08:03:53.139 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 66889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71805f.ko 2024-03-29T08:03:53.139 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 94593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f71882fg.ko 2024-03-29T08:03:53.139 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 41305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/f75375s.ko 2024-03-29T08:03:53.139 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fam15h_power.ko 2024-03-29T08:03:53.139 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 55353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/fschmd.ko 2024-03-29T08:03:53.139 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g760a.ko 2024-03-29T08:03:53.140 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/g762.ko 2024-03-29T08:03:53.140 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 46705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl518sm.ko 2024-03-29T08:03:53.140 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 45433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/gl520sm.ko 2024-03-29T08:03:53.140 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hih6130.ko 2024-03-29T08:03:53.140 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/hwmon-vid.ko 2024-03-29T08:03:53.140 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5500_temp.ko 2024-03-29T08:03:53.140 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/i5k_amb.ko 2024-03-29T08:03:53.140 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmaem.ko 2024-03-29T08:03:53.140 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ibmpex.ko 2024-03-29T08:03:53.140 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina209.ko 2024-03-29T08:03:53.140 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ina2xx.ko 2024-03-29T08:03:53.140 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 140129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/it87.ko 2024-03-29T08:03:53.140 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/jc42.ko 2024-03-29T08:03:53.140 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k10temp.ko 2024-03-29T08:03:53.140 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/k8temp.ko 2024-03-29T08:03:53.140 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lineage-pem.ko 2024-03-29T08:03:53.141 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 48505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm63.ko 2024-03-29T08:03:53.141 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm73.ko 2024-03-29T08:03:53.141 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 38841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm75.ko 2024-03-29T08:03:53.141 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm77.ko 2024-03-29T08:03:53.141 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm78.ko 2024-03-29T08:03:53.141 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm80.ko 2024-03-29T08:03:53.141 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm83.ko 2024-03-29T08:03:53.141 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 72737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm85.ko 2024-03-29T08:03:53.141 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 46057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm87.ko 2024-03-29T08:03:53.141 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 37817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm90.ko 2024-03-29T08:03:53.141 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm92.ko 2024-03-29T08:03:53.141 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 108745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm93.ko 2024-03-29T08:03:53.141 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95234.ko 2024-03-29T08:03:53.141 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95241.ko 2024-03-29T08:03:53.142 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/lm95245.ko 2024-03-29T08:03:53.142 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc2945.ko 2024-03-29T08:03:53.142 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4151.ko 2024-03-29T08:03:53.142 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4215.ko 2024-03-29T08:03:53.142 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4222.ko 2024-03-29T08:03:53.142 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4245.ko 2024-03-29T08:03:53.142 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4260.ko 2024-03-29T08:03:53.142 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/ltc4261.ko 2024-03-29T08:03:53.142 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max16065.ko 2024-03-29T08:03:53.142 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1619.ko 2024-03-29T08:03:53.142 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max1668.ko 2024-03-29T08:03:53.142 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max197.ko 2024-03-29T08:03:53.142 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6639.ko 2024-03-29T08:03:53.142 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6650.ko 2024-03-29T08:03:53.142 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 35681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/max6697.ko 2024-03-29T08:03:53.143 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mc13783-adc.ko 2024-03-29T08:03:53.143 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/mcp3021.ko 2024-03-29T08:03:53.143 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-03-29T08:03:53.143 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 47785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6683.ko 2024-03-29T08:03:53.143 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 119841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775-core.ko 2024-03-29T08:03:53.143 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 61433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct6775.ko 2024-03-29T08:03:53.143 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 59641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/nct7802.ko 2024-03-29T08:03:53.143 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 82713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87360.ko 2024-03-29T08:03:53.143 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 54337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pc87427.ko 2024-03-29T08:03:53.143 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pcf8591.ko 2024-03-29T08:03:53.143 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ 2024-03-29T08:03:53.143 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-03-29T08:03:53.143 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-03-29T08:03:53.143 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-03-29T08:03:53.144 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max16064.ko 2024-03-29T08:03:53.144 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max34440.ko 2024-03-29T08:03:53.144 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/max8688.ko 2024-03-29T08:03:53.144 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-03-29T08:03:53.144 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 82473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-03-29T08:03:53.144 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-03-29T08:03:53.144 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-03-29T08:03:53.144 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-03-29T08:03:53.144 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-03-29T08:03:53.144 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/powr1220.ko 2024-03-29T08:03:53.144 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5627.ko 2024-03-29T08:03:53.144 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch5636.ko 2024-03-29T08:03:53.144 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sch56xx-common.ko 2024-03-29T08:03:53.144 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 32041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht15.ko 2024-03-29T08:03:53.144 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sht21.ko 2024-03-29T08:03:53.144 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/shtc1.ko 2024-03-29T08:03:53.145 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 40481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/sis5595.ko 2024-03-29T08:03:53.145 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47b397.ko 2024-03-29T08:03:53.145 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 40353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m1.ko 2024-03-29T08:03:53.145 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 37281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/smsc47m192.ko 2024-03-29T08:03:53.145 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/thmc50.ko 2024-03-29T08:03:53.145 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp102.ko 2024-03-29T08:03:53.145 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp103.ko 2024-03-29T08:03:53.145 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp401.ko 2024-03-29T08:03:53.145 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/tmp421.ko 2024-03-29T08:03:53.145 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via-cputemp.ko 2024-03-29T08:03:53.145 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 43441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/via686a.ko 2024-03-29T08:03:53.145 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 52361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt1211.ko 2024-03-29T08:03:53.145 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 53201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/vt8231.ko 2024-03-29T08:03:53.145 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 58257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627ehf.ko 2024-03-29T08:03:53.145 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 79833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83627hf.ko 2024-03-29T08:03:53.146 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 71249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83781d.ko 2024-03-29T08:03:53.146 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 63129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83791d.ko 2024-03-29T08:03:53.146 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 71225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83792d.ko 2024-03-29T08:03:53.146 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 101377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83793.ko 2024-03-29T08:03:53.146 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 119409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83795.ko 2024-03-29T08:03:53.146 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l785ts.ko 2024-03-29T08:03:53.146 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 35521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/w83l786ng.ko 2024-03-29T08:03:53.146 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-03-29T08:03:53.146 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-03-29T08:03:53.146 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/ 2024-03-29T08:03:53.146 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/ 2024-03-29T08:03:53.146 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-03-29T08:03:53.146 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-03-29T08:03:53.146 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/ 2024-03-29T08:03:53.147 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-03-29T08:03:53.147 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 32473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-03-29T08:03:53.147 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-03-29T08:03:53.147 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-03-29T08:03:53.147 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-03-29T08:03:53.147 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-03-29T08:03:53.147 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-03-29T08:03:53.147 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 6825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-03-29T08:03:53.147 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-03-29T08:03:53.147 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 50289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-03-29T08:03:53.147 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-03-29T08:03:53.147 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-03-29T08:03:53.147 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-03-29T08:03:53.147 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-03-29T08:03:53.147 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-03-29T08:03:53.148 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 67681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-i801.ko 2024-03-29T08:03:53.148 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-isch.ko 2024-03-29T08:03:53.148 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 47793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-03-29T08:03:53.148 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-03-29T08:03:53.148 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-03-29T08:03:53.148 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-03-29T08:03:53.148 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-03-29T08:03:53.148 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-parport.ko 2024-03-29T08:03:53.148 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-03-29T08:03:53.148 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 50969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-03-29T08:03:53.148 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-03-29T08:03:53.148 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-03-29T08:03:53.148 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-03-29T08:03:53.148 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-03-29T08:03:53.149 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-03-29T08:03:53.149 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-03-29T08:03:53.149 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-03-29T08:03:53.149 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-03-29T08:03:53.149 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-via.ko 2024-03-29T08:03:53.149 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-03-29T08:03:53.149 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-03-29T08:03:53.149 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 47561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-03-29T08:03:53.149 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-mux.ko 2024-03-29T08:03:53.149 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-smbus.ko 2024-03-29T08:03:53.149 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/i2c-stub.ko 2024-03-29T08:03:53.149 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/muxes/ 2024-03-29T08:03:53.149 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-03-29T08:03:53.149 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-03-29T08:03:53.149 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-03-29T08:03:53.150 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ 2024-03-29T08:03:53.150 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ 2024-03-29T08:03:53.150 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 273513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_cm.ko 2024-03-29T08:03:53.150 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 883777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_core.ko 2024-03-29T08:03:53.150 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 69177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_umad.ko 2024-03-29T08:03:53.150 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 324233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-03-29T08:03:53.150 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 97825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/iw_cm.ko 2024-03-29T08:03:53.150 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 282329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_cm.ko 2024-03-29T08:03:53.150 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 66049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-03-29T08:03:53.150 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/ 2024-03-29T08:03:53.150 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/ 2024-03-29T08:03:53.150 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 493129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-03-29T08:03:53.150 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/ 2024-03-29T08:03:53.150 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 420529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-03-29T08:03:53.150 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/ 2024-03-29T08:03:53.151 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 741609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-03-29T08:03:53.151 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ 2024-03-29T08:03:53.151 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 296785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-03-29T08:03:53.151 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/ocrdma/ 2024-03-29T08:03:53.151 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 153705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-03-29T08:03:53.151 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/usnic/ 2024-03-29T08:03:53.151 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 169089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-03-29T08:03:53.151 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ 2024-03-29T08:03:53.151 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ 2024-03-29T08:03:53.151 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 266529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-03-29T08:03:53.151 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ 2024-03-29T08:03:53.151 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 100257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-03-29T08:03:53.151 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ 2024-03-29T08:03:53.151 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 111329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-03-29T08:03:53.152 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ 2024-03-29T08:03:53.152 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 128657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-03-29T08:03:53.152 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ 2024-03-29T08:03:53.152 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 129793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-03-29T08:03:53.152 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/ 2024-03-29T08:03:53.152 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/evbug.ko 2024-03-29T08:03:53.152 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/ff-memless.ko 2024-03-29T08:03:53.152 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/ 2024-03-29T08:03:53.152 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-03-29T08:03:53.152 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/fm801-gp.ko 2024-03-29T08:03:53.152 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 41449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/gameport/gameport.ko 2024-03-29T08:03:53.152 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 41705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/joydev.ko 2024-03-29T08:03:53.152 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/ 2024-03-29T08:03:53.152 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-03-29T08:03:53.152 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-03-29T08:03:53.153 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-03-29T08:03:53.153 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-03-29T08:03:53.153 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 37273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys.ko 2024-03-29T08:03:53.153 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-03-29T08:03:53.153 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/lkkbd.ko 2024-03-29T08:03:53.153 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/lm8323.ko 2024-03-29T08:03:53.153 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/lm8333.ko 2024-03-29T08:03:53.153 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-03-29T08:03:53.153 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-03-29T08:03:53.153 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-03-29T08:03:53.153 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-03-29T08:03:53.153 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/newtonkbd.ko 2024-03-29T08:03:53.153 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-03-29T08:03:53.153 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/qt1070.ko 2024-03-29T08:03:53.153 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/qt2160.ko 2024-03-29T08:03:53.153 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-03-29T08:03:53.154 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/stowaway.ko 2024-03-29T08:03:53.154 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/sunkbd.ko 2024-03-29T08:03:53.154 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-03-29T08:03:53.154 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-03-29T08:03:53.154 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-03-29T08:03:53.154 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/keyboard/xtkbd.ko 2024-03-29T08:03:53.154 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/matrix-keymap.ko 2024-03-29T08:03:53.154 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/ 2024-03-29T08:03:53.154 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/appletouch.ko 2024-03-29T08:03:53.154 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/bcm5974.ko 2024-03-29T08:03:53.154 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 123177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/cyapatp.ko 2024-03-29T08:03:53.154 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 89377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/elan_i2c.ko 2024-03-29T08:03:53.154 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/gpio_mouse.ko 2024-03-29T08:03:53.154 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 335857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/psmouse.ko 2024-03-29T08:03:53.154 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/sermouse.ko 2024-03-29T08:03:53.155 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-03-29T08:03:53.155 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/synaptics_usb.ko 2024-03-29T08:03:53.155 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/mouse/vsxxxaa.ko 2024-03-29T08:03:53.155 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/ 2024-03-29T08:03:53.155 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 109553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/rmi4/rmi_core.ko 2024-03-29T08:03:53.155 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/ 2024-03-29T08:03:53.155 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/altera_ps2.ko 2024-03-29T08:03:53.155 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/arc_ps2.ko 2024-03-29T08:03:53.155 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/ct82c710.ko 2024-03-29T08:03:53.155 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-03-29T08:03:53.155 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/parkbd.ko 2024-03-29T08:03:53.155 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/pcips2.ko 2024-03-29T08:03:53.155 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/ps2mult.ko 2024-03-29T08:03:53.155 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serio_raw.ko 2024-03-29T08:03:53.156 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/serio/serport.ko 2024-03-29T08:03:53.156 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/input/sparse-keymap.ko 2024-03-29T08:03:53.156 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/ 2024-03-29T08:03:53.156 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-88pm860x.ko 2024-03-29T08:03:53.156 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-adp5520.ko 2024-03-29T08:03:53.156 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 59649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-bd2802.ko 2024-03-29T08:03:53.156 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-blinkm.ko 2024-03-29T08:03:53.156 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-da903x.ko 2024-03-29T08:03:53.156 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-da9052.ko 2024-03-29T08:03:53.156 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-gpio.ko 2024-03-29T08:03:53.156 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3530.ko 2024-03-29T08:03:53.156 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3533.ko 2024-03-29T08:03:53.156 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm355x.ko 2024-03-29T08:03:53.156 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lm3642.ko 2024-03-29T08:03:53.156 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp3944.ko 2024-03-29T08:03:53.157 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-lp8788.ko 2024-03-29T08:03:53.157 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-max8997.ko 2024-03-29T08:03:53.157 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-mc13783.ko 2024-03-29T08:03:53.157 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-menf21bmc.ko 2024-03-29T08:03:53.157 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-pca9532.ko 2024-03-29T08:03:53.157 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-pca955x.ko 2024-03-29T08:03:53.157 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-pca963x.ko 2024-03-29T08:03:53.157 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-ss4200.ko 2024-03-29T08:03:53.157 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-tca6507.ko 2024-03-29T08:03:53.157 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-wm831x-status.ko 2024-03-29T08:03:53.157 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/leds-wm8350.ko 2024-03-29T08:03:53.157 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ 2024-03-29T08:03:53.157 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-03-29T08:03:53.157 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-03-29T08:03:53.157 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-03-29T08:03:53.158 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-03-29T08:03:53.158 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-03-29T08:03:53.158 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-03-29T08:03:53.158 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-03-29T08:03:53.158 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-03-29T08:03:53.158 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-03-29T08:03:53.158 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/ 2024-03-29T08:03:53.158 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/ 2024-03-29T08:03:53.158 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 647033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/bcache/bcache.ko 2024-03-29T08:03:53.158 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 40473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bio-prison.ko 2024-03-29T08:03:53.158 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 93313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-bufio.ko 2024-03-29T08:03:53.158 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 47433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache-smq.ko 2024-03-29T08:03:53.158 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 155801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-cache.ko 2024-03-29T08:03:53.159 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 107929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-crypt.ko 2024-03-29T08:03:53.159 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-delay.ko 2024-03-29T08:03:53.159 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 53761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-era.ko 2024-03-29T08:03:53.159 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-flakey.ko 2024-03-29T08:03:53.159 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 41425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log-userspace.ko 2024-03-29T08:03:53.159 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 32641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-log.ko 2024-03-29T08:03:53.159 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 49441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-mirror.ko 2024-03-29T08:03:53.159 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 77865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-multipath.ko 2024-03-29T08:03:53.159 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-queue-length.ko 2024-03-29T08:03:53.159 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 70977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-raid.ko 2024-03-29T08:03:53.159 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-region-hash.ko 2024-03-29T08:03:53.159 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-round-robin.ko 2024-03-29T08:03:53.159 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-service-time.ko 2024-03-29T08:03:53.159 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 109433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-snapshot.ko 2024-03-29T08:03:53.159 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-switch.ko 2024-03-29T08:03:53.159 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 181761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-thin-pool.ko 2024-03-29T08:03:53.160 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 53729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-verity.ko 2024-03-29T08:03:53.160 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/dm-zero.ko 2024-03-29T08:03:53.160 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/ 2024-03-29T08:03:53.160 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 199825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-03-29T08:03:53.160 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 37313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid0.ko 2024-03-29T08:03:53.160 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 100833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid1.ko 2024-03-29T08:03:53.160 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 126361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid10.ko 2024-03-29T08:03:53.160 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 368481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/md/raid456.ko 2024-03-29T08:03:53.160 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/ 2024-03-29T08:03:53.160 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/ 2024-03-29T08:03:53.160 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 178041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptbase.ko 2024-03-29T08:03:53.160 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 63233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptctl.ko 2024-03-29T08:03:53.160 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 46241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptfc.ko 2024-03-29T08:03:53.160 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 37113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptlan.ko 2024-03-29T08:03:53.161 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 123225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptsas.ko 2024-03-29T08:03:53.161 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 80913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptscsih.ko 2024-03-29T08:03:53.161 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 52857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/message/fusion/mptspi.ko 2024-03-29T08:03:53.161 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/ 2024-03-29T08:03:53.161 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/88pm800.ko 2024-03-29T08:03:53.161 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/88pm805.ko 2024-03-29T08:03:53.161 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/88pm80x.ko 2024-03-29T08:03:53.161 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/arizona-i2c.ko 2024-03-29T08:03:53.161 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 123625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/arizona.ko 2024-03-29T08:03:53.161 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/bcm590xx.ko 2024-03-29T08:03:53.161 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/cros_ec_dev.ko 2024-03-29T08:03:53.161 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/dln2.ko 2024-03-29T08:03:53.161 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/janz-cmodio.ko 2024-03-29T08:03:53.161 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 53481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/kempld-core.ko 2024-03-29T08:03:53.161 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lm3533-core.ko 2024-03-29T08:03:53.161 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-03-29T08:03:53.162 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lp3943.ko 2024-03-29T08:03:53.162 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 56961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_ich.ko 2024-03-29T08:03:53.162 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/lpc_sch.ko 2024-03-29T08:03:53.162 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/max8907.ko 2024-03-29T08:03:53.162 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mc13xxx-core.ko 2024-03-29T08:03:53.162 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-03-29T08:03:53.162 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/menf21bmc.ko 2024-03-29T08:03:53.162 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/pcf50633-adc.ko 2024-03-29T08:03:53.162 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/pcf50633-gpio.ko 2024-03-29T08:03:53.162 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/pcf50633.ko 2024-03-29T08:03:53.162 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-03-29T08:03:53.162 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/retu-mfd.ko 2024-03-29T08:03:53.162 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 80329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/si476x-core.ko 2024-03-29T08:03:53.162 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 48361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/sm501.ko 2024-03-29T08:03:53.163 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 51201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/tps65010.ko 2024-03-29T08:03:53.163 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/tps6507x.ko 2024-03-29T08:03:53.163 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/viperboard.ko 2024-03-29T08:03:53.163 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/vx855.ko 2024-03-29T08:03:53.163 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wl1273-core.ko 2024-03-29T08:03:53.163 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 44249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/mfd/wm8994.ko 2024-03-29T08:03:53.163 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ 2024-03-29T08:03:53.163 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-03-29T08:03:53.163 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 48385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ad525x_dpot.ko 2024-03-29T08:03:53.163 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/ 2024-03-29T08:03:53.163 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 46489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-03-29T08:03:53.163 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds9802als.ko 2024-03-29T08:03:53.163 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/apds990x.ko 2024-03-29T08:03:53.163 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 45489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/bh1770glc.ko 2024-03-29T08:03:53.163 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/c2port/ 2024-03-29T08:03:53.164 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-03-29T08:03:53.164 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 35489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/c2port/core.ko 2024-03-29T08:03:53.164 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/ 2024-03-29T08:03:53.164 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/cb710/cb710.ko 2024-03-29T08:03:53.164 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ds1682.ko 2024-03-29T08:03:53.164 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/dummy-irq.ko 2024-03-29T08:03:53.164 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/echo/ 2024-03-29T08:03:53.164 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/echo/echo.ko 2024-03-29T08:03:53.164 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/ 2024-03-29T08:03:53.164 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/at24.ko 2024-03-29T08:03:53.164 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-03-29T08:03:53.164 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/eeprom/max6875.ko 2024-03-29T08:03:53.164 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 32905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/enclosure.ko 2024-03-29T08:03:53.164 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/genwqe/ 2024-03-29T08:03:53.164 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 152041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-03-29T08:03:53.165 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hmc6352.ko 2024-03-29T08:03:53.165 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/hpilo.ko 2024-03-29T08:03:53.165 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ibmasm/ 2024-03-29T08:03:53.165 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 72505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-03-29T08:03:53.165 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ics932s401.ko 2024-03-29T08:03:53.165 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29003.ko 2024-03-29T08:03:53.165 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/isl29020.ko 2024-03-29T08:03:53.165 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/ 2024-03-29T08:03:53.165 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 42409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-03-29T08:03:53.165 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-03-29T08:03:53.165 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/ 2024-03-29T08:03:53.165 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 126137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-me.ko 2024-03-29T08:03:53.165 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 56361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei-txe.ko 2024-03-29T08:03:53.165 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 313545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/mei/mei.ko 2024-03-29T08:03:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/phantom.ko 2024-03-29T08:03:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ti-st/ 2024-03-29T08:03:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 74289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/ti-st/st_drv.ko 2024-03-29T08:03:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_7xx1.ko 2024-03-29T08:03:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tifm_core.ko 2024-03-29T08:03:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/tsl2550.ko 2024-03-29T08:03:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 48337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_balloon.ko 2024-03-29T08:03:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/ 2024-03-29T08:03:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 183513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-03-29T08:03:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ 2024-03-29T08:03:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/ 2024-03-29T08:03:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 491561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/bonding/bonding.ko 2024-03-29T08:03:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/ 2024-03-29T08:03:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/ 2024-03-29T08:03:53.166 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 105929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/b53/b53_common.ko 2024-03-29T08:03:53.167 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 40209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/bcm-sf2.ko 2024-03-29T08:03:53.167 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6060.ko 2024-03-29T08:03:53.167 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/ 2024-03-29T08:03:53.167 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 458905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-03-29T08:03:53.167 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/dummy.ko 2024-03-29T08:03:53.167 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/eql.ko 2024-03-29T08:03:53.167 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ 2024-03-29T08:03:53.167 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/ 2024-03-29T08:03:53.167 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 45953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-03-29T08:03:53.167 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 42185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-03-29T08:03:53.167 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 130657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-03-29T08:03:53.167 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 68777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-03-29T08:03:53.167 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ 2024-03-29T08:03:53.167 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 38041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/8390.ko 2024-03-29T08:03:53.167 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 64609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-03-29T08:03:53.168 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-03-29T08:03:53.168 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 101609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-03-29T08:03:53.168 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/ 2024-03-29T08:03:53.168 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 54617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-03-29T08:03:53.168 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/ 2024-03-29T08:03:53.168 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 69161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/agere/et131x.ko 2024-03-29T08:03:53.168 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/ 2024-03-29T08:03:53.168 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 59985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-03-29T08:03:53.168 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/ 2024-03-29T08:03:53.168 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 41385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-03-29T08:03:53.168 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/ 2024-03-29T08:03:53.168 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 54609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-03-29T08:03:53.168 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 49185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-03-29T08:03:53.168 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 87449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-03-29T08:03:53.168 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/ 2024-03-29T08:03:53.169 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/ 2024-03-29T08:03:53.169 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 97185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-03-29T08:03:53.169 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-03-29T08:03:53.169 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 102481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-03-29T08:03:53.169 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-03-29T08:03:53.169 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 90185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-03-29T08:03:53.169 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/ 2024-03-29T08:03:53.169 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 81313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-03-29T08:03:53.169 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 69537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-03-29T08:03:53.169 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/ 2024-03-29T08:03:53.169 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 78377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-03-29T08:03:53.169 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 176689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-03-29T08:03:53.169 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-03-29T08:03:53.169 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 1434449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-03-29T08:03:53.234 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 128361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-03-29T08:03:53.234 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/ 2024-03-29T08:03:53.234 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 131265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-03-29T08:03:53.234 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 314321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-03-29T08:03:53.234 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/ 2024-03-29T08:03:53.234 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/ 2024-03-29T08:03:53.234 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 323681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-03-29T08:03:53.234 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/ 2024-03-29T08:03:53.234 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-03-29T08:03:53.234 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 193537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-03-29T08:03:53.234 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-03-29T08:03:53.235 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 347161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-03-29T08:03:53.235 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-03-29T08:03:53.235 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 1045321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-03-29T08:03:53.235 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-03-29T08:03:53.235 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 166129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-03-29T08:03:53.235 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-03-29T08:03:53.235 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-03-29T08:03:53.235 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/ 2024-03-29T08:03:53.235 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/ 2024-03-29T08:03:53.235 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 204185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-03-29T08:03:53.235 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/ 2024-03-29T08:03:53.235 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/ 2024-03-29T08:03:53.235 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 69425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-03-29T08:03:53.235 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 61417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-03-29T08:03:53.235 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 141153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-03-29T08:03:53.236 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 51361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-03-29T08:03:53.236 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 62825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-03-29T08:03:53.236 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-03-29T08:03:53.236 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/ 2024-03-29T08:03:53.236 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 60665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-03-29T08:03:53.236 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 62825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-03-29T08:03:53.236 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 45481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/dnet.ko 2024-03-29T08:03:53.236 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ec_bhf.ko 2024-03-29T08:03:53.236 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/ 2024-03-29T08:03:53.236 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/ 2024-03-29T08:03:53.236 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 281193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-03-29T08:03:53.236 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 49449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ethoc.ko 2024-03-29T08:03:53.236 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 51777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fealnx.ko 2024-03-29T08:03:53.236 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fujitsu/ 2024-03-29T08:03:53.237 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 45545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-03-29T08:03:53.237 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ 2024-03-29T08:03:53.237 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 90297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e100.ko 2024-03-29T08:03:53.237 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/ 2024-03-29T08:03:53.237 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 295673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-03-29T08:03:53.237 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/ 2024-03-29T08:03:53.237 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 584297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-03-29T08:03:53.237 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/ 2024-03-29T08:03:53.237 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 245977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-03-29T08:03:53.237 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/ 2024-03-29T08:03:53.237 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 944449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-03-29T08:03:53.237 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/ 2024-03-29T08:03:53.237 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 288481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-03-29T08:03:53.237 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/ 2024-03-29T08:03:53.237 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 514337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-03-29T08:03:53.238 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/ 2024-03-29T08:03:53.238 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 93465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-03-29T08:03:53.238 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-03-29T08:03:53.238 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 735129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-03-29T08:03:53.238 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-03-29T08:03:53.238 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 151649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-03-29T08:03:53.238 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 86985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/jme.ko 2024-03-29T08:03:53.238 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/ 2024-03-29T08:03:53.238 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-03-29T08:03:53.238 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 102433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/skge.ko 2024-03-29T08:03:53.238 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 122601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-03-29T08:03:53.238 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/ 2024-03-29T08:03:53.238 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-03-29T08:03:53.238 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 727377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-03-29T08:03:53.238 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 278825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-03-29T08:03:53.239 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-03-29T08:03:53.239 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-03-29T08:03:53.239 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 1122065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-03-29T08:03:53.239 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ 2024-03-29T08:03:53.239 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 52977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-03-29T08:03:53.239 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 40873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-03-29T08:03:53.239 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-03-29T08:03:53.239 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 95569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-03-29T08:03:53.239 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/ 2024-03-29T08:03:53.239 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-03-29T08:03:53.239 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 108401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-03-29T08:03:53.239 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ 2024-03-29T08:03:53.239 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 73609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-03-29T08:03:53.239 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 47793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-03-29T08:03:53.239 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/ 2024-03-29T08:03:53.240 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 151193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-03-29T08:03:53.240 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/ 2024-03-29T08:03:53.240 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 136793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-03-29T08:03:53.240 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/ 2024-03-29T08:03:53.240 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 50425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-03-29T08:03:53.240 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 42977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-03-29T08:03:53.240 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/ 2024-03-29T08:03:53.240 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-03-29T08:03:53.240 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 227841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-03-29T08:03:53.240 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 87009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-03-29T08:03:53.240 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-03-29T08:03:53.240 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 671985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-03-29T08:03:53.240 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/ 2024-03-29T08:03:53.240 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 42929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-03-29T08:03:53.241 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/ 2024-03-29T08:03:53.241 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 67273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-03-29T08:03:53.241 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 83801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-03-29T08:03:53.241 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/atp.ko 2024-03-29T08:03:53.241 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 194057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-03-29T08:03:53.241 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/samsung/ 2024-03-29T08:03:53.241 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-03-29T08:03:53.241 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 104001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-03-29T08:03:53.241 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/ 2024-03-29T08:03:53.241 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 1066817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-03-29T08:03:53.241 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/ 2024-03-29T08:03:53.241 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 47001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-03-29T08:03:53.241 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/ 2024-03-29T08:03:53.241 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 54721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis190.ko 2024-03-29T08:03:53.241 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 63193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sis/sis900.ko 2024-03-29T08:03:53.242 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/ 2024-03-29T08:03:53.242 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 61417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-03-29T08:03:53.242 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 67289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-03-29T08:03:53.242 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 73249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-03-29T08:03:53.242 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 68041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-03-29T08:03:53.242 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/ 2024-03-29T08:03:53.242 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-03-29T08:03:53.242 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-03-29T08:03:53.242 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-03-29T08:03:53.242 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-03-29T08:03:53.242 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 495641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-03-29T08:03:53.242 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/ 2024-03-29T08:03:53.242 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 103177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/cassini.ko 2024-03-29T08:03:53.242 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 166041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/niu.ko 2024-03-29T08:03:53.242 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 66265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sungem.ko 2024-03-29T08:03:53.243 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 78713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-03-29T08:03:53.243 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/ 2024-03-29T08:03:53.243 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 50009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-03-29T08:03:53.243 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/ 2024-03-29T08:03:53.243 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 92337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/ti/tlan.ko 2024-03-29T08:03:53.243 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/ 2024-03-29T08:03:53.243 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 79929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-03-29T08:03:53.243 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 83713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-03-29T08:03:53.243 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/ 2024-03-29T08:03:53.243 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 46305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-03-29T08:03:53.243 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-03-29T08:03:53.243 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/ 2024-03-29T08:03:53.243 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 59977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-03-29T08:03:53.243 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/fddi/ 2024-03-29T08:03:53.243 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 41177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/fddi/defxx.ko 2024-03-29T08:03:53.244 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/fddi/skfp/ 2024-03-29T08:03:53.244 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 217025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/fddi/skfp/skfp.ko 2024-03-29T08:03:53.244 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 74985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/geneve.ko 2024-03-29T08:03:53.244 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/ 2024-03-29T08:03:53.244 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 237241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-03-29T08:03:53.244 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/ 2024-03-29T08:03:53.244 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ieee802154/fakelb.ko 2024-03-29T08:03:53.244 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ifb.ko 2024-03-29T08:03:53.244 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ 2024-03-29T08:03:53.244 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 85841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ipvlan/ipvlan.ko 2024-03-29T08:03:53.244 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 62889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvlan.ko 2024-03-29T08:03:53.244 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/macvtap.ko 2024-03-29T08:03:53.244 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/ 2024-03-29T08:03:53.244 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-03-29T08:03:53.244 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-03-29T08:03:53.244 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-gpio.ko 2024-03-29T08:03:53.245 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio/mdio-regmap.ko 2024-03-29T08:03:53.245 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mdio.ko 2024-03-29T08:03:53.245 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/mii.ko 2024-03-29T08:03:53.245 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 45049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/netconsole.ko 2024-03-29T08:03:53.245 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/nlmon.ko 2024-03-29T08:03:53.245 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/ 2024-03-29T08:03:53.245 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs-lynx.ko 2024-03-29T08:03:53.245 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-03-29T08:03:53.245 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ 2024-03-29T08:03:53.245 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/amd.ko 2024-03-29T08:03:53.245 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 49929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-03-29T08:03:53.245 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 44625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm7xxx.ko 2024-03-29T08:03:53.245 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/bcm87xx.ko 2024-03-29T08:03:53.245 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 44817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/broadcom.ko 2024-03-29T08:03:53.245 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/cicada.ko 2024-03-29T08:03:53.245 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/davicom.ko 2024-03-29T08:03:53.246 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/et1011c.ko 2024-03-29T08:03:53.246 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/icplus.ko 2024-03-29T08:03:53.246 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/lxt.ko 2024-03-29T08:03:53.246 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 81953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/marvell.ko 2024-03-29T08:03:53.246 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 76561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/micrel.ko 2024-03-29T08:03:53.246 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/national.ko 2024-03-29T08:03:53.246 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 99001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/phylink.ko 2024-03-29T08:03:53.246 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/qsemi.ko 2024-03-29T08:03:53.246 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 49969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/realtek.ko 2024-03-29T08:03:53.246 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/smsc.ko 2024-03-29T08:03:53.246 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/ste10Xp.ko 2024-03-29T08:03:53.246 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/phy/vitesse.ko 2024-03-29T08:03:53.246 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/plip/ 2024-03-29T08:03:53.246 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 43761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/plip/plip.ko 2024-03-29T08:03:53.247 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ 2024-03-29T08:03:53.247 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/bsd_comp.ko 2024-03-29T08:03:53.247 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_async.ko 2024-03-29T08:03:53.247 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_deflate.ko 2024-03-29T08:03:53.247 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_mppe.ko 2024-03-29T08:03:53.247 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/ppp_synctty.ko 2024-03-29T08:03:53.247 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 42665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppoe.ko 2024-03-29T08:03:53.247 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pppox.ko 2024-03-29T08:03:53.247 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 32665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/ppp/pptp.ko 2024-03-29T08:03:53.247 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sb1000.ko 2024-03-29T08:03:53.247 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/ 2024-03-29T08:03:53.247 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 42385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/slip/slip.ko 2024-03-29T08:03:53.247 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/sungem_phy.ko 2024-03-29T08:03:53.247 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 64553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/tap.ko 2024-03-29T08:03:53.247 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/ 2024-03-29T08:03:53.248 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 107225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team.ko 2024-03-29T08:03:53.248 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_activebackup.ko 2024-03-29T08:03:53.248 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_broadcast.ko 2024-03-29T08:03:53.248 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-03-29T08:03:53.248 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_random.ko 2024-03-29T08:03:53.248 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-03-29T08:03:53.248 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 86817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/veth.ko 2024-03-29T08:03:53.248 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/ 2024-03-29T08:03:53.248 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 144345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-03-29T08:03:53.248 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/ 2024-03-29T08:03:53.248 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 307169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/vxlan/vxlan.ko 2024-03-29T08:03:53.248 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/ 2024-03-29T08:03:53.248 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 143993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/net/xen-netback/xen-netback.ko 2024-03-29T08:03:53.248 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ 2024-03-29T08:03:53.248 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ntb/ntb.ko 2024-03-29T08:03:53.249 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/ 2024-03-29T08:03:53.249 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/ 2024-03-29T08:03:53.249 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 267721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme-core.ko 2024-03-29T08:03:53.249 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 103353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/nvme/host/nvme.ko 2024-03-29T08:03:53.249 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/ 2024-03-29T08:03:53.249 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 131297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport.ko 2024-03-29T08:03:53.249 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_cs.ko 2024-03-29T08:03:53.249 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 85209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_pc.ko 2024-03-29T08:03:53.249 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/parport/parport_serial.ko 2024-03-29T08:03:53.249 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/ 2024-03-29T08:03:53.249 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/ 2024-03-29T08:03:53.249 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-03-29T08:03:53.249 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-03-29T08:03:53.249 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-03-29T08:03:53.249 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/pci-stub.ko 2024-03-29T08:03:53.250 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 46361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pci/xen-pcifront.ko 2024-03-29T08:03:53.250 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/ 2024-03-29T08:03:53.250 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/i82092.ko 2024-03-29T08:03:53.250 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 146129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pcmcia.ko 2024-03-29T08:03:53.250 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 62065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pcmcia_core.ko 2024-03-29T08:03:53.250 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-03-29T08:03:53.250 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/pd6729.ko 2024-03-29T08:03:53.250 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 112785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pcmcia/yenta_socket.ko 2024-03-29T08:03:53.250 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/ 2024-03-29T08:03:53.250 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/broadcom/ 2024-03-29T08:03:53.250 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-03-29T08:03:53.250 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/samsung/ 2024-03-29T08:03:53.250 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-03-29T08:03:53.250 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/ 2024-03-29T08:03:53.250 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/ 2024-03-29T08:03:53.251 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 69025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-03-29T08:03:53.251 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/ 2024-03-29T08:03:53.251 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/ 2024-03-29T08:03:53.251 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 51217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-03-29T08:03:53.251 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec.ko 2024-03-29T08:03:53.251 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-03-29T08:03:53.251 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-03-29T08:03:53.251 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-03-29T08:03:53.251 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-03-29T08:03:53.251 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-03-29T08:03:53.251 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/ 2024-03-29T08:03:53.251 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 73593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acer-wmi.ko 2024-03-29T08:03:53.251 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/acerhdf.ko 2024-03-29T08:03:53.251 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/apple-gmux.ko 2024-03-29T08:03:53.251 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 59561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-laptop.ko 2024-03-29T08:03:53.252 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-03-29T08:03:53.252 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 131937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/asus-wmi.ko 2024-03-29T08:03:53.252 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/classmate-laptop.ko 2024-03-29T08:03:53.252 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 49017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-03-29T08:03:53.252 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-03-29T08:03:53.252 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 40161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-03-29T08:03:53.252 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-03-29T08:03:53.252 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/hdaps.ko 2024-03-29T08:03:53.252 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/ibm_rtl.ko 2024-03-29T08:03:53.252 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/ 2024-03-29T08:03:53.252 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-03-29T08:03:53.252 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-03-29T08:03:53.252 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 44529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/intel_ips.ko 2024-03-29T08:03:53.252 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/msi-wmi.ko 2024-03-29T08:03:53.253 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/mxm-wmi.ko 2024-03-29T08:03:53.253 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-03-29T08:03:53.253 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 42225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-laptop.ko 2024-03-29T08:03:53.253 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/samsung-q10.ko 2024-03-29T08:03:53.253 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/topstar-laptop.ko 2024-03-29T08:03:53.253 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-03-29T08:03:53.253 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/toshiba_haps.ko 2024-03-29T08:03:53.253 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi-bmof.ko 2024-03-29T08:03:53.253 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 48473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/platform/x86/wmi.ko 2024-03-29T08:03:53.253 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/ 2024-03-29T08:03:53.253 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ 2024-03-29T08:03:53.253 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/88pm860x_battery.ko 2024-03-29T08:03:53.253 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/88pm860x_charger.ko 2024-03-29T08:03:53.253 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 54953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/bq2415x_charger.ko 2024-03-29T08:03:53.253 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 59497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/bq24190_charger.ko 2024-03-29T08:03:53.254 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/bq24735-charger.ko 2024-03-29T08:03:53.254 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-03-29T08:03:53.254 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/da9030_battery.ko 2024-03-29T08:03:53.254 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/da9052-battery.ko 2024-03-29T08:03:53.254 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ds2760_battery.ko 2024-03-29T08:03:53.254 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ds2780_battery.ko 2024-03-29T08:03:53.254 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ds2781_battery.ko 2024-03-29T08:03:53.254 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/ds2782_battery.ko 2024-03-29T08:03:53.254 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/gpio-charger.ko 2024-03-29T08:03:53.254 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/isp1704_charger.ko 2024-03-29T08:03:53.254 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/lp8727_charger.ko 2024-03-29T08:03:53.254 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/max14577_charger.ko 2024-03-29T08:03:53.254 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/max17042_battery.ko 2024-03-29T08:03:53.254 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/max8903_charger.ko 2024-03-29T08:03:53.254 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/max8925_power.ko 2024-03-29T08:03:53.255 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/pcf50633-charger.ko 2024-03-29T08:03:53.255 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 37833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/sbs-battery.ko 2024-03-29T08:03:53.255 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/test_power.ko 2024-03-29T08:03:53.255 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/tps65090-charger.ko 2024-03-29T08:03:53.255 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/wm831x_backup.ko 2024-03-29T08:03:53.255 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/wm831x_power.ko 2024-03-29T08:03:53.255 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/power/supply/wm8350_power.ko 2024-03-29T08:03:53.255 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/ 2024-03-29T08:03:53.255 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 58097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_common.ko 2024-03-29T08:03:53.255 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/powercap/intel_rapl_msr.ko 2024-03-29T08:03:53.255 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/ 2024-03-29T08:03:53.255 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/ 2024-03-29T08:03:53.255 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-gpio.ko 2024-03-29T08:03:53.255 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps-ldisc.ko 2024-03-29T08:03:53.255 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/clients/pps_parport.ko 2024-03-29T08:03:53.256 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 41785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/pps/pps_core.ko 2024-03-29T08:03:53.256 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ 2024-03-29T08:03:53.256 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 70849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp.ko 2024-03-29T08:03:53.256 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ptp/ptp_kvm.ko 2024-03-29T08:03:53.256 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/ 2024-03-29T08:03:53.256 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-88pm80x.ko 2024-03-29T08:03:53.256 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-88pm860x.ko 2024-03-29T08:03:53.256 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-bq32k.ko 2024-03-29T08:03:53.256 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-da9052.ko 2024-03-29T08:03:53.256 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-da9055.ko 2024-03-29T08:03:53.256 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-da9063.ko 2024-03-29T08:03:53.256 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1286.ko 2024-03-29T08:03:53.256 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 68473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1307.ko 2024-03-29T08:03:53.256 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1374.ko 2024-03-29T08:03:53.257 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1511.ko 2024-03-29T08:03:53.257 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1553.ko 2024-03-29T08:03:53.257 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1672.ko 2024-03-29T08:03:53.257 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds1742.ko 2024-03-29T08:03:53.257 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds2404.ko 2024-03-29T08:03:53.257 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-ds3232.ko 2024-03-29T08:03:53.257 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-em3027.ko 2024-03-29T08:03:53.257 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-fm3130.ko 2024-03-29T08:03:53.257 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl12022.ko 2024-03-29T08:03:53.257 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 35865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-isl1208.ko 2024-03-29T08:03:53.257 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-lp8788.ko 2024-03-29T08:03:53.257 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 37993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m41t80.ko 2024-03-29T08:03:53.257 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t35.ko 2024-03-29T08:03:53.257 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t59.ko 2024-03-29T08:03:53.257 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-m48t86.ko 2024-03-29T08:03:53.257 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max6900.ko 2024-03-29T08:03:53.258 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max8907.ko 2024-03-29T08:03:53.258 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max8925.ko 2024-03-29T08:03:53.258 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max8997.ko 2024-03-29T08:03:53.258 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-max8998.ko 2024-03-29T08:03:53.258 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-03-29T08:03:53.258 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-msm6242.ko 2024-03-29T08:03:53.258 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-palmas.ko 2024-03-29T08:03:53.258 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 46529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf2127.ko 2024-03-29T08:03:53.258 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf50633.ko 2024-03-29T08:03:53.258 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf85063.ko 2024-03-29T08:03:53.258 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8523.ko 2024-03-29T08:03:53.258 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8563.ko 2024-03-29T08:03:53.258 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-pcf8583.ko 2024-03-29T08:03:53.258 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rc5t583.ko 2024-03-29T08:03:53.258 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rp5c01.ko 2024-03-29T08:03:53.259 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rs5c372.ko 2024-03-29T08:03:53.259 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-03-29T08:03:53.259 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8025.ko 2024-03-29T08:03:53.259 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-rx8581.ko 2024-03-29T08:03:53.259 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-s35390a.ko 2024-03-29T08:03:53.259 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-03-29T08:03:53.259 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-tps6586x.ko 2024-03-29T08:03:53.259 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-tps65910.ko 2024-03-29T08:03:53.259 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-wm831x.ko 2024-03-29T08:03:53.259 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-wm8350.ko 2024-03-29T08:03:53.259 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/rtc/rtc-x1205.ko 2024-03-29T08:03:53.259 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ 2024-03-29T08:03:53.259 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 77521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-9xxx.ko 2024-03-29T08:03:53.259 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 57713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-sas.ko 2024-03-29T08:03:53.259 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 55009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/3w-xxxx.ko 2024-03-29T08:03:53.260 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 135249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/BusLogic.ko 2024-03-29T08:03:53.260 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/a100u2w.ko 2024-03-29T08:03:53.260 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/ 2024-03-29T08:03:53.260 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 271385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aacraid/aacraid.ko 2024-03-29T08:03:53.260 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 100529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/advansys.ko 2024-03-29T08:03:53.260 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/ 2024-03-29T08:03:53.260 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 356833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-03-29T08:03:53.260 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 295889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-03-29T08:03:53.260 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic94xx/ 2024-03-29T08:03:53.260 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 189089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-03-29T08:03:53.260 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/am53c974.ko 2024-03-29T08:03:53.260 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/ 2024-03-29T08:03:53.260 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 108193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-03-29T08:03:53.260 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 38089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/atp870u.ko 2024-03-29T08:03:53.260 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/ 2024-03-29T08:03:53.261 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 246913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-03-29T08:03:53.261 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/ 2024-03-29T08:03:53.261 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 1171857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bfa/bfa.ko 2024-03-29T08:03:53.261 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/ 2024-03-29T08:03:53.261 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 238369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-03-29T08:03:53.261 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/ 2024-03-29T08:03:53.261 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 115305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-03-29T08:03:53.261 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 40777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ch.ko 2024-03-29T08:03:53.261 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/ 2024-03-29T08:03:53.261 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 308905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/csiostor/csiostor.ko 2024-03-29T08:03:53.261 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/ 2024-03-29T08:03:53.261 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-03-29T08:03:53.261 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 76033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-03-29T08:03:53.261 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-03-29T08:03:53.261 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 101417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-03-29T08:03:53.262 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 146745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-03-29T08:03:53.262 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 75969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dc395x.ko 2024-03-29T08:03:53.262 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 38201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/dmx3191d.ko 2024-03-29T08:03:53.262 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/ 2024-03-29T08:03:53.262 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 212329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esas2r/esas2r.ko 2024-03-29T08:03:53.262 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 60497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/esp_scsi.ko 2024-03-29T08:03:53.262 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/ 2024-03-29T08:03:53.262 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 78217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/fcoe.ko 2024-03-29T08:03:53.262 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 157713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-03-29T08:03:53.262 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fdomain.ko 2024-03-29T08:03:53.262 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/ 2024-03-29T08:03:53.262 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 294881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/fnic/fnic.ko 2024-03-29T08:03:53.262 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 201721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hpsa.ko 2024-03-29T08:03:53.262 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 51113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hptiop.ko 2024-03-29T08:03:53.262 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 48937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/hv_storvsc.ko 2024-03-29T08:03:53.263 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/imm.ko 2024-03-29T08:03:53.263 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 43969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/initio.ko 2024-03-29T08:03:53.263 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 265937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ipr.ko 2024-03-29T08:03:53.263 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 101873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ips.ko 2024-03-29T08:03:53.263 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/ 2024-03-29T08:03:53.263 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 317129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/isci/isci.ko 2024-03-29T08:03:53.263 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-03-29T08:03:53.263 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 52857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/iscsi_tcp.ko 2024-03-29T08:03:53.263 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/ 2024-03-29T08:03:53.263 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 315273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libfc/libfc.ko 2024-03-29T08:03:53.263 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 145489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi.ko 2024-03-29T08:03:53.263 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 53801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libiscsi_tcp.ko 2024-03-29T08:03:53.264 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/ 2024-03-29T08:03:53.264 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 216345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/libsas/libsas.ko 2024-03-29T08:03:53.264 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/ 2024-03-29T08:03:53.264 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 2193953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/lpfc/lpfc.ko 2024-03-29T08:03:53.264 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/ 2024-03-29T08:03:53.264 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 73465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-03-29T08:03:53.264 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-03-29T08:03:53.264 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 310665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-03-29T08:03:53.264 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 85513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/megaraid.ko 2024-03-29T08:03:53.264 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/ 2024-03-29T08:03:53.264 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 659585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-03-29T08:03:53.264 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/ 2024-03-29T08:03:53.264 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 119465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvsas/mvsas.ko 2024-03-29T08:03:53.264 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 78033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/mvumi.ko 2024-03-29T08:03:53.265 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/ 2024-03-29T08:03:53.265 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 81545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-03-29T08:03:53.265 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-03-29T08:03:53.265 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-03-29T08:03:53.265 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-03-29T08:03:53.265 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/ 2024-03-29T08:03:53.265 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 366305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-03-29T08:03:53.265 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 124593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/pmcraid.ko 2024-03-29T08:03:53.265 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ppa.ko 2024-03-29T08:03:53.265 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 55329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla1280.ko 2024-03-29T08:03:53.265 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/ 2024-03-29T08:03:53.265 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 1876657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-03-29T08:03:53.265 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 85185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-03-29T08:03:53.265 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/ 2024-03-29T08:03:53.266 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 558897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-03-29T08:03:53.266 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/qlogicfas408.ko 2024-03-29T08:03:53.266 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/raid_class.ko 2024-03-29T08:03:53.266 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 261073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_debug.ko 2024-03-29T08:03:53.266 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 173921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_fc.ko 2024-03-29T08:03:53.266 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 280105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-03-29T08:03:53.266 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 97617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_sas.ko 2024-03-29T08:03:53.266 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 73857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_spi.ko 2024-03-29T08:03:53.266 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 44881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/scsi_transport_srp.ko 2024-03-29T08:03:53.266 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/ses.ko 2024-03-29T08:03:53.266 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 115641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/st.ko 2024-03-29T08:03:53.266 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 46729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/stex.ko 2024-03-29T08:03:53.266 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/ 2024-03-29T08:03:53.266 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 164865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-03-29T08:03:53.266 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 44849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/virtio_scsi.ko 2024-03-29T08:03:53.267 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 50225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/vmw_pvscsi.ko 2024-03-29T08:03:53.267 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/wd719x.ko 2024-03-29T08:03:53.267 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 49921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/scsi/xen-scsifront.ko 2024-03-29T08:03:53.267 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ 2024-03-29T08:03:53.267 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 129345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ssb/ssb.ko 2024-03-29T08:03:53.267 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/ 2024-03-29T08:03:53.267 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/ 2024-03-29T08:03:53.267 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 736665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-03-29T08:03:53.267 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/ 2024-03-29T08:03:53.267 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 49873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/loopback/tcm_loop.ko 2024-03-29T08:03:53.267 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_file.ko 2024-03-29T08:03:53.267 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 42801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_iblock.ko 2024-03-29T08:03:53.267 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 931433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_mod.ko 2024-03-29T08:03:53.267 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 44145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_pscsi.ko 2024-03-29T08:03:53.268 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 122481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/target_core_user.ko 2024-03-29T08:03:53.268 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/ 2024-03-29T08:03:53.268 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 75497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-03-29T08:03:53.268 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/ 2024-03-29T08:03:53.268 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/ 2024-03-29T08:03:53.268 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/ 2024-03-29T08:03:53.268 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-03-29T08:03:53.268 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-03-29T08:03:53.268 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-03-29T08:03:53.268 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-03-29T08:03:53.268 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-03-29T08:03:53.268 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-03-29T08:03:53.268 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-03-29T08:03:53.268 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-03-29T08:03:53.268 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-03-29T08:03:53.269 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-03-29T08:03:53.269 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-03-29T08:03:53.269 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-03-29T08:03:53.269 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-03-29T08:03:53.269 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-03-29T08:03:53.269 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-03-29T08:03:53.269 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-03-29T08:03:53.269 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-03-29T08:03:53.269 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-03-29T08:03:53.269 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-03-29T08:03:53.269 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ 2024-03-29T08:03:53.269 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ 2024-03-29T08:03:53.269 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 78329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-03-29T08:03:53.269 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 57769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/moxa.ko 2024-03-29T08:03:53.269 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 50857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/mxser.ko 2024-03-29T08:03:53.270 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 108481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_gsm.ko 2024-03-29T08:03:53.270 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/n_hdlc.ko 2024-03-29T08:03:53.270 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 67833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/nozomi.ko 2024-03-29T08:03:53.270 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/ 2024-03-29T08:03:53.270 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/ 2024-03-29T08:03:53.270 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-03-29T08:03:53.270 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 59393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-03-29T08:03:53.270 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-03-29T08:03:53.270 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/altera_uart.ko 2024-03-29T08:03:53.270 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/arc_uart.ko 2024-03-29T08:03:53.270 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 72545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-03-29T08:03:53.270 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/ 2024-03-29T08:03:53.270 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 101969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/jsm/jsm.ko 2024-03-29T08:03:53.270 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/rp2.ko 2024-03-29T08:03:53.270 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 53169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/tty/serial/sc16is7xx.ko 2024-03-29T08:03:53.271 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/ 2024-03-29T08:03:53.271 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ 2024-03-29T08:03:53.271 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 545377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/core/ufshcd-core.ko 2024-03-29T08:03:53.271 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ 2024-03-29T08:03:53.271 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-03-29T08:03:53.271 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-03-29T08:03:53.271 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/ 2024-03-29T08:03:53.271 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 44465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio.ko 2024-03-29T08:03:53.271 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_aec.ko 2024-03-29T08:03:53.271 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_cif.ko 2024-03-29T08:03:53.271 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_dmem_genirq.ko 2024-03-29T08:03:53.271 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_mf624.ko 2024-03-29T08:03:53.271 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_netx.ko 2024-03-29T08:03:53.271 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pci_generic.ko 2024-03-29T08:03:53.272 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-03-29T08:03:53.272 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/uio/uio_sercos3.ko 2024-03-29T08:03:53.272 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/ 2024-03-29T08:03:53.272 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/c67x00/ 2024-03-29T08:03:53.272 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 73201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/c67x00/c67x00.ko 2024-03-29T08:03:53.272 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ 2024-03-29T08:03:53.272 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 145753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-03-29T08:03:53.272 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-03-29T08:03:53.272 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-03-29T08:03:53.272 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-03-29T08:03:53.272 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-03-29T08:03:53.272 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/ 2024-03-29T08:03:53.272 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 87001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-acm.ko 2024-03-29T08:03:53.272 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 51249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/cdc-wdm.ko 2024-03-29T08:03:53.272 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 47209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usblp.ko 2024-03-29T08:03:53.272 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 89761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/class/usbtmc.ko 2024-03-29T08:03:53.273 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/common/ 2024-03-29T08:03:53.273 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/common/ulpi.ko 2024-03-29T08:03:53.273 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc2/ 2024-03-29T08:03:53.273 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-03-29T08:03:53.273 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc3/ 2024-03-29T08:03:53.273 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-03-29T08:03:53.273 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-03-29T08:03:53.273 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 452785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/dwc3/dwc3.ko 2024-03-29T08:03:53.273 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/ 2024-03-29T08:03:53.273 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/ 2024-03-29T08:03:53.273 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 60321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/u_ether.ko 2024-03-29T08:03:53.273 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 51785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/u_serial.ko 2024-03-29T08:03:53.273 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-03-29T08:03:53.273 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 42345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-03-29T08:03:53.273 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-03-29T08:03:53.274 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-03-29T08:03:53.274 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 92401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-03-29T08:03:53.274 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 50937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-03-29T08:03:53.274 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 122241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-03-29T08:03:53.274 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 59857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-03-29T08:03:53.274 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-03-29T08:03:53.274 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-03-29T08:03:53.274 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 57625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-03-29T08:03:53.274 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 85505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-03-29T08:03:53.278 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-03-29T08:03:53.278 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 58009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-03-29T08:03:53.278 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 67345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-03-29T08:03:53.278 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/ 2024-03-29T08:03:53.278 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-03-29T08:03:53.278 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-03-29T08:03:53.278 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-03-29T08:03:53.278 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-03-29T08:03:53.278 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-03-29T08:03:53.279 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-03-29T08:03:53.279 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-03-29T08:03:53.279 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-03-29T08:03:53.279 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-03-29T08:03:53.279 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-03-29T08:03:53.279 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-03-29T08:03:53.279 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-03-29T08:03:53.279 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 62361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-03-29T08:03:53.279 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-03-29T08:03:53.279 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 161193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/libcomposite.ko 2024-03-29T08:03:53.279 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/ 2024-03-29T08:03:53.279 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-03-29T08:03:53.279 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/bdc/ 2024-03-29T08:03:53.279 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 157593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-03-29T08:03:53.280 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 35993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-03-29T08:03:53.280 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 52921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-03-29T08:03:53.280 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 69609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-03-29T08:03:53.280 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 55945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-03-29T08:03:53.280 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 75961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/net2272.ko 2024-03-29T08:03:53.280 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 74977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/net2280.ko 2024-03-29T08:03:53.280 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 73009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-03-29T08:03:53.280 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 65873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-03-29T08:03:53.280 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 65993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-03-29T08:03:53.280 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 81401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-03-29T08:03:53.280 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 156321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-03-29T08:03:53.280 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/ 2024-03-29T08:03:53.280 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/bcma-hcd.ko 2024-03-29T08:03:53.280 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 80009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/isp116x-hcd.ko 2024-03-29T08:03:53.280 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 87249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-03-29T08:03:53.281 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 78769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-03-29T08:03:53.281 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 51385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811-hcd.ko 2024-03-29T08:03:53.281 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/sl811_cs.ko 2024-03-29T08:03:53.281 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/ssb-hcd.ko 2024-03-29T08:03:53.281 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-03-29T08:03:53.281 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/ 2024-03-29T08:03:53.281 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/mdc800.ko 2024-03-29T08:03:53.281 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/image/microtek.ko 2024-03-29T08:03:53.281 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ 2024-03-29T08:03:53.281 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 41625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/adutux.ko 2024-03-29T08:03:53.281 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/appledisplay.ko 2024-03-29T08:03:53.281 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-03-29T08:03:53.281 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/cytherm.ko 2024-03-29T08:03:53.281 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ehset.ko 2024-03-29T08:03:53.282 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi26.ko 2024-03-29T08:03:53.282 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/emi62.ko 2024-03-29T08:03:53.282 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ezusb.ko 2024-03-29T08:03:53.282 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/idmouse.ko 2024-03-29T08:03:53.282 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/iowarrior.ko 2024-03-29T08:03:53.282 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/isight_firmware.ko 2024-03-29T08:03:53.282 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/ldusb.ko 2024-03-29T08:03:53.282 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/legousbtower.ko 2024-03-29T08:03:53.282 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/lvstest.ko 2024-03-29T08:03:53.282 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/ 2024-03-29T08:03:53.282 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 64737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-03-29T08:03:53.282 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/trancevibrator.ko 2024-03-29T08:03:53.282 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usb3503.ko 2024-03-29T08:03:53.282 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usblcd.ko 2024-03-29T08:03:53.283 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbsevseg.ko 2024-03-29T08:03:53.283 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 70713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/usbtest.ko 2024-03-29T08:03:53.283 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 41809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/uss720.ko 2024-03-29T08:03:53.283 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/misc/yurex.ko 2024-03-29T08:03:53.283 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/mon/ 2024-03-29T08:03:53.283 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 72801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/mon/usbmon.ko 2024-03-29T08:03:53.283 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/musb/ 2024-03-29T08:03:53.283 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 301593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/musb/musb_hdrc.ko 2024-03-29T08:03:53.283 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/ 2024-03-29T08:03:53.283 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-generic.ko 2024-03-29T08:03:53.283 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-03-29T08:03:53.283 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-isp1301.ko 2024-03-29T08:03:53.283 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/phy/phy-tahvo.ko 2024-03-29T08:03:53.283 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ 2024-03-29T08:03:53.283 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/aircable.ko 2024-03-29T08:03:53.283 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ark3116.ko 2024-03-29T08:03:53.284 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/belkin_sa.ko 2024-03-29T08:03:53.284 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 38009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ch341.ko 2024-03-29T08:03:53.284 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 70273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cp210x.ko 2024-03-29T08:03:53.284 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cyberjack.ko 2024-03-29T08:03:53.284 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 58113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/cypress_m8.ko 2024-03-29T08:03:53.284 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 49713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/digi_acceleport.ko 2024-03-29T08:03:53.284 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/empeg.ko 2024-03-29T08:03:53.284 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 44177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/f81232.ko 2024-03-29T08:03:53.284 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 175097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ftdi_sio.ko 2024-03-29T08:03:53.284 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 53009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/garmin_gps.ko 2024-03-29T08:03:53.284 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 136977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_edgeport.ko 2024-03-29T08:03:53.284 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 124737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/io_ti.ko 2024-03-29T08:03:53.284 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 67673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipaq.ko 2024-03-29T08:03:53.284 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ipw.ko 2024-03-29T08:03:53.285 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ir-usb.ko 2024-03-29T08:03:53.285 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 58129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-03-29T08:03:53.285 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 107425 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan.ko 2024-03-29T08:03:53.285 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 35513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/keyspan_pda.ko 2024-03-29T08:03:53.285 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kl5kusb105.ko 2024-03-29T08:03:53.285 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/kobil_sct.ko 2024-03-29T08:03:53.285 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 32857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mct_u232.ko 2024-03-29T08:03:53.285 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/metro-usb.ko 2024-03-29T08:03:53.285 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 66353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7720.ko 2024-03-29T08:03:53.285 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 78121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mos7840.ko 2024-03-29T08:03:53.285 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 56241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/mxuport.ko 2024-03-29T08:03:53.285 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/navman.ko 2024-03-29T08:03:53.285 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/omninet.ko 2024-03-29T08:03:53.285 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/opticon.ko 2024-03-29T08:03:53.285 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 191817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/option.ko 2024-03-29T08:03:53.286 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 38977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/oti6858.ko 2024-03-29T08:03:53.286 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 54257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/pl2303.ko 2024-03-29T08:03:53.286 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcaux.ko 2024-03-29T08:03:53.286 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 38017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/qcserial.ko 2024-03-29T08:03:53.286 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/quatech2.ko 2024-03-29T08:03:53.286 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/safe_serial.ko 2024-03-29T08:03:53.286 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 51529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/sierra.ko 2024-03-29T08:03:53.286 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/spcp8x5.ko 2024-03-29T08:03:53.286 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ssu100.ko 2024-03-29T08:03:53.286 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/symbolserial.ko 2024-03-29T08:03:53.286 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 61385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-03-29T08:03:53.286 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-03-29T08:03:53.286 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_debug.ko 2024-03-29T08:03:53.286 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usb_wwan.ko 2024-03-29T08:03:53.287 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 125513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/usbserial.ko 2024-03-29T08:03:53.287 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/visor.ko 2024-03-29T08:03:53.287 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 37305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/whiteheat.ko 2024-03-29T08:03:53.287 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/wishbone-serial.ko 2024-03-29T08:03:53.287 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/serial/xsens_mt.ko 2024-03-29T08:03:53.287 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ 2024-03-29T08:03:53.287 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 52753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/uas.ko 2024-03-29T08:03:53.287 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-alauda.ko 2024-03-29T08:03:53.287 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-cypress.ko 2024-03-29T08:03:53.287 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-datafab.ko 2024-03-29T08:03:53.287 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-03-29T08:03:53.287 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-freecom.ko 2024-03-29T08:03:53.287 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-isd200.ko 2024-03-29T08:03:53.287 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-03-29T08:03:53.287 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-karma.ko 2024-03-29T08:03:53.288 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-onetouch.ko 2024-03-29T08:03:53.288 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-realtek.ko 2024-03-29T08:03:53.288 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr09.ko 2024-03-29T08:03:53.288 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-sddr55.ko 2024-03-29T08:03:53.288 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 35241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/ums-usbat.ko 2024-03-29T08:03:53.288 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 180281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/storage/usb-storage.ko 2024-03-29T08:03:53.288 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/ 2024-03-29T08:03:53.288 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 67385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-core.ko 2024-03-29T08:03:53.288 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 71001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/usbip-host.ko 2024-03-29T08:03:53.288 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 103977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-03-29T08:03:53.288 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/ 2024-03-29T08:03:53.288 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/ 2024-03-29T08:03:53.288 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 162841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-03-29T08:03:53.288 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/pci/vfio-pci.ko 2024-03-29T08:03:53.289 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 103649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio.ko 2024-03-29T08:03:53.289 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 76433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-03-29T08:03:53.289 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/ 2024-03-29T08:03:53.289 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 113657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost.ko 2024-03-29T08:03:53.289 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_iotlb.ko 2024-03-29T08:03:53.289 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 57993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_net.ko 2024-03-29T08:03:53.289 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 89049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/vhost/vhost_scsi.ko 2024-03-29T08:03:53.289 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/ 2024-03-29T08:03:53.289 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/ 2024-03-29T08:03:53.289 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-03-29T08:03:53.289 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/aat2870_bl.ko 2024-03-29T08:03:53.289 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/adp5520_bl.ko 2024-03-29T08:03:53.289 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 32505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/adp8860_bl.ko 2024-03-29T08:03:53.289 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/adp8870_bl.ko 2024-03-29T08:03:53.289 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/apple_bl.ko 2024-03-29T08:03:53.290 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/as3711_bl.ko 2024-03-29T08:03:53.290 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/bd6107.ko 2024-03-29T08:03:53.290 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/da903x_bl.ko 2024-03-29T08:03:53.290 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/da9052_bl.ko 2024-03-29T08:03:53.290 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/gpio_backlight.ko 2024-03-29T08:03:53.290 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/kb3886_bl.ko 2024-03-29T08:03:53.290 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lcd.ko 2024-03-29T08:03:53.290 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lm3533_bl.ko 2024-03-29T08:03:53.290 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lm3639_bl.ko 2024-03-29T08:03:53.290 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/lv5207lp.ko 2024-03-29T08:03:53.290 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/max8925_bl.ko 2024-03-29T08:03:53.290 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/pandora_bl.ko 2024-03-29T08:03:53.290 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-03-29T08:03:53.290 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/platform_lcd.ko 2024-03-29T08:03:53.290 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/video/backlight/wm831x_bl.ko 2024-03-29T08:03:53.291 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/ 2024-03-29T08:03:53.291 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ 2024-03-29T08:03:53.291 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2482.ko 2024-03-29T08:03:53.291 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/ds2490.ko 2024-03-29T08:03:53.291 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/matrox_w1.ko 2024-03-29T08:03:53.291 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/masters/w1-gpio.ko 2024-03-29T08:03:53.291 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/ 2024-03-29T08:03:53.291 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-03-29T08:03:53.291 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-03-29T08:03:53.291 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-03-29T08:03:53.291 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-03-29T08:03:53.291 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-03-29T08:03:53.291 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-03-29T08:03:53.291 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-03-29T08:03:53.292 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-03-29T08:03:53.292 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-03-29T08:03:53.292 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_smem.ko 2024-03-29T08:03:53.292 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 56329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/slaves/w1_therm.ko 2024-03-29T08:03:53.292 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 103025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/w1/wire.ko 2024-03-29T08:03:53.292 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ 2024-03-29T08:03:53.292 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/acquirewdt.ko 2024-03-29T08:03:53.292 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/advantechwdt.ko 2024-03-29T08:03:53.292 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim1535_wdt.ko 2024-03-29T08:03:53.292 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/alim7101_wdt.ko 2024-03-29T08:03:53.292 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/cpu5wdt.ko 2024-03-29T08:03:53.292 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/da9052_wdt.ko 2024-03-29T08:03:53.292 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/da9055_wdt.ko 2024-03-29T08:03:53.292 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/da9063_wdt.ko 2024-03-29T08:03:53.292 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/dw_wdt.ko 2024-03-29T08:03:53.293 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/eurotechwdt.ko 2024-03-29T08:03:53.293 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/f71808e_wdt.ko 2024-03-29T08:03:53.293 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/hpwdt.ko 2024-03-29T08:03:53.293 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/i6300esb.ko 2024-03-29T08:03:53.293 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-03-29T08:03:53.293 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/iTCO_wdt.ko 2024-03-29T08:03:53.293 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ib700wdt.ko 2024-03-29T08:03:53.293 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ibmasr.ko 2024-03-29T08:03:53.293 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-03-29T08:03:53.293 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it8712f_wdt.ko 2024-03-29T08:03:53.293 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/it87_wdt.ko 2024-03-29T08:03:53.293 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/kempld_wdt.ko 2024-03-29T08:03:53.293 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/machzwd.ko 2024-03-29T08:03:53.293 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/mena21_wdt.ko 2024-03-29T08:03:53.293 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-03-29T08:03:53.294 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/nv_tco.ko 2024-03-29T08:03:53.294 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-03-29T08:03:53.294 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pc87413_wdt.ko 2024-03-29T08:03:53.294 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_pci.ko 2024-03-29T08:03:53.294 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/pcwd_usb.ko 2024-03-29T08:03:53.294 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/retu_wdt.ko 2024-03-29T08:03:53.294 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-03-29T08:03:53.294 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-03-29T08:03:53.294 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-03-29T08:03:53.294 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sc1200wdt.ko 2024-03-29T08:03:53.294 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sch311x_wdt.ko 2024-03-29T08:03:53.294 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21473 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-03-29T08:03:53.294 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/softdog.ko 2024-03-29T08:03:53.294 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/sp5100_tco.ko 2024-03-29T08:03:53.294 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/twl4030_wdt.ko 2024-03-29T08:03:53.295 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/via_wdt.ko 2024-03-29T08:03:53.295 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-03-29T08:03:53.295 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83877f_wdt.ko 2024-03-29T08:03:53.295 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/w83977f_wdt.ko 2024-03-29T08:03:53.295 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wafer5823wdt.ko 2024-03-29T08:03:53.295 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wdt_pci.ko 2024-03-29T08:03:53.295 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wm831x_wdt.ko 2024-03-29T08:03:53.295 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/wm8350_wdt.ko 2024-03-29T08:03:53.295 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/watchdog/xen_wdt.ko 2024-03-29T08:03:53.295 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/ 2024-03-29T08:03:53.295 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-evtchn.ko 2024-03-29T08:03:53.295 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntalloc.ko 2024-03-29T08:03:53.295 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 47761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-gntdev.ko 2024-03-29T08:03:53.295 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/ 2024-03-29T08:03:53.296 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 176337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-03-29T08:03:53.296 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 40081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-privcmd.ko 2024-03-29T08:03:53.296 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 71145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xen-scsiback.ko 2024-03-29T08:03:53.296 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/ 2024-03-29T08:03:53.296 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/drivers/xen/xenfs/xenfs.ko 2024-03-29T08:03:53.296 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ 2024-03-29T08:03:53.296 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/ 2024-03-29T08:03:53.296 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 158217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/9p/9p.ko 2024-03-29T08:03:53.296 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/adfs/ 2024-03-29T08:03:53.296 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 61681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/adfs/adfs.ko 2024-03-29T08:03:53.296 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/ 2024-03-29T08:03:53.296 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 163369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/affs/affs.ko 2024-03-29T08:03:53.296 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/afs/ 2024-03-29T08:03:53.296 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 1019049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/afs/kafs.ko 2024-03-29T08:03:53.296 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/autofs/ 2024-03-29T08:03:53.297 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 103185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/autofs/autofs4.ko 2024-03-29T08:03:53.297 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/ 2024-03-29T08:03:53.297 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 64585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/befs/befs.ko 2024-03-29T08:03:53.297 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/bfs/ 2024-03-29T08:03:53.297 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 41001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/bfs/bfs.ko 2024-03-29T08:03:53.297 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 38641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/binfmt_misc.ko 2024-03-29T08:03:53.297 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/ 2024-03-29T08:03:53.297 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 3487249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/btrfs/btrfs.ko 2024-03-29T08:03:53.297 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/ 2024-03-29T08:03:53.297 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 305841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cachefiles/cachefiles.ko 2024-03-29T08:03:53.297 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ 2024-03-29T08:03:53.297 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 1335577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ceph/ceph.ko 2024-03-29T08:03:53.343 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/ 2024-03-29T08:03:53.344 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 97713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/coda/coda.ko 2024-03-29T08:03:53.344 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/configfs/ 2024-03-29T08:03:53.344 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 108545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/configfs/configfs.ko 2024-03-29T08:03:53.344 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/ 2024-03-29T08:03:53.344 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/cramfs/cramfs.ko 2024-03-29T08:03:53.344 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/ 2024-03-29T08:03:53.344 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 644161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/dlm/dlm.ko 2024-03-29T08:03:53.344 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/efs/ 2024-03-29T08:03:53.344 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/efs/efs.ko 2024-03-29T08:03:53.344 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/ 2024-03-29T08:03:53.344 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 1527561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/f2fs/f2fs.ko 2024-03-29T08:03:53.344 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/ 2024-03-29T08:03:53.344 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fat/msdos.ko 2024-03-29T08:03:53.344 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/freevxfs/ 2024-03-29T08:03:53.345 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 42513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/freevxfs/freevxfs.ko 2024-03-29T08:03:53.345 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/ 2024-03-29T08:03:53.345 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/fuse/cuse.ko 2024-03-29T08:03:53.345 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/ 2024-03-29T08:03:53.345 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 1231233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/gfs2/gfs2.ko 2024-03-29T08:03:53.345 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/ 2024-03-29T08:03:53.345 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 141625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfs/hfs.ko 2024-03-29T08:03:53.345 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/ 2024-03-29T08:03:53.345 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 223177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hfsplus/hfsplus.ko 2024-03-29T08:03:53.345 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hpfs/ 2024-03-29T08:03:53.345 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 193481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/hpfs/hpfs.ko 2024-03-29T08:03:53.345 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/ 2024-03-29T08:03:53.345 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 74689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/isofs/isofs.ko 2024-03-29T08:03:53.345 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/ 2024-03-29T08:03:53.346 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 420329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/jfs/jfs.ko 2024-03-29T08:03:53.346 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/ 2024-03-29T08:03:53.346 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 281617 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/lockd/lockd.ko 2024-03-29T08:03:53.346 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/ 2024-03-29T08:03:53.346 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 88897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/minix/minix.ko 2024-03-29T08:03:53.346 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/ 2024-03-29T08:03:53.346 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 490409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/netfs/netfs.ko 2024-03-29T08:03:53.346 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/ 2024-03-29T08:03:53.346 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/ 2024-03-29T08:03:53.346 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 81433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-03-29T08:03:53.346 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/ 2024-03-29T08:03:53.346 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 59337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-03-29T08:03:53.346 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/ 2024-03-29T08:03:53.346 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 117361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-03-29T08:03:53.346 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 1058273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfs.ko 2024-03-29T08:03:53.347 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 66345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv2.ko 2024-03-29T08:03:53.347 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 120169 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv3.ko 2024-03-29T08:03:53.347 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 2041705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs/nfsv4.ko 2024-03-29T08:03:53.347 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/ 2024-03-29T08:03:53.347 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/grace.ko 2024-03-29T08:03:53.347 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfs_common/nfs_acl.ko 2024-03-29T08:03:53.347 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/ 2024-03-29T08:03:53.347 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 1511641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nfsd/nfsd.ko 2024-03-29T08:03:53.347 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/ 2024-03-29T08:03:53.347 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 582409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nilfs2/nilfs2.ko 2024-03-29T08:03:53.347 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/ 2024-03-29T08:03:53.347 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-celtic.ko 2024-03-29T08:03:53.347 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-centeuro.ko 2024-03-29T08:03:53.347 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-croatian.ko 2024-03-29T08:03:53.347 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-cyrillic.ko 2024-03-29T08:03:53.348 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-gaelic.ko 2024-03-29T08:03:53.348 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-greek.ko 2024-03-29T08:03:53.348 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-iceland.ko 2024-03-29T08:03:53.348 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-inuit.ko 2024-03-29T08:03:53.348 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-roman.ko 2024-03-29T08:03:53.348 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-romanian.ko 2024-03-29T08:03:53.348 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/mac-turkish.ko 2024-03-29T08:03:53.348 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ascii.ko 2024-03-29T08:03:53.348 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1250.ko 2024-03-29T08:03:53.348 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1251.ko 2024-03-29T08:03:53.348 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp1255.ko 2024-03-29T08:03:53.348 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp737.ko 2024-03-29T08:03:53.348 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp775.ko 2024-03-29T08:03:53.348 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp850.ko 2024-03-29T08:03:53.348 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp852.ko 2024-03-29T08:03:53.349 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp855.ko 2024-03-29T08:03:53.349 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp857.ko 2024-03-29T08:03:53.349 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp860.ko 2024-03-29T08:03:53.349 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp861.ko 2024-03-29T08:03:53.349 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp862.ko 2024-03-29T08:03:53.349 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp863.ko 2024-03-29T08:03:53.349 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp864.ko 2024-03-29T08:03:53.349 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp865.ko 2024-03-29T08:03:53.349 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp866.ko 2024-03-29T08:03:53.349 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp869.ko 2024-03-29T08:03:53.349 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp874.ko 2024-03-29T08:03:53.349 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 94033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp932.ko 2024-03-29T08:03:53.349 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 142569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp936.ko 2024-03-29T08:03:53.349 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 165641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp949.ko 2024-03-29T08:03:53.350 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 118017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_cp950.ko 2024-03-29T08:03:53.350 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_euc-jp.ko 2024-03-29T08:03:53.350 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-1.ko 2024-03-29T08:03:53.350 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-13.ko 2024-03-29T08:03:53.350 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-14.ko 2024-03-29T08:03:53.350 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-15.ko 2024-03-29T08:03:53.350 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-2.ko 2024-03-29T08:03:53.350 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-3.ko 2024-03-29T08:03:53.350 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-4.ko 2024-03-29T08:03:53.350 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-5.ko 2024-03-29T08:03:53.350 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-6.ko 2024-03-29T08:03:53.350 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-7.ko 2024-03-29T08:03:53.350 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_iso8859-9.ko 2024-03-29T08:03:53.350 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-r.ko 2024-03-29T08:03:53.351 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-ru.ko 2024-03-29T08:03:53.351 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_koi8-u.ko 2024-03-29T08:03:53.351 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_ucs2_utils.ko 2024-03-29T08:03:53.351 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/nls/nls_utf8.ko 2024-03-29T08:03:53.351 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ntfs/ 2024-03-29T08:03:53.351 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 220457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ntfs/ntfs.ko 2024-03-29T08:03:53.351 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ 2024-03-29T08:03:53.351 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ 2024-03-29T08:03:53.351 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 244633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-03-29T08:03:53.351 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ 2024-03-29T08:03:53.351 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 440633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-03-29T08:03:53.351 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ 2024-03-29T08:03:53.351 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 54161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-03-29T08:03:53.351 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 2590681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2.ko 2024-03-29T08:03:53.351 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-03-29T08:03:53.352 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-03-29T08:03:53.352 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 35729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-03-29T08:03:53.352 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/omfs/ 2024-03-29T08:03:53.352 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 53681 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/omfs/omfs.ko 2024-03-29T08:03:53.352 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/ 2024-03-29T08:03:53.352 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 360897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/overlayfs/overlay.ko 2024-03-29T08:03:53.352 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ 2024-03-29T08:03:53.352 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 56929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/pstore/ramoops.ko 2024-03-29T08:03:53.352 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/qnx4/ 2024-03-29T08:03:53.352 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/qnx4/qnx4.ko 2024-03-29T08:03:53.352 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/qnx6/ 2024-03-29T08:03:53.352 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 46601 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/qnx6/qnx6.ko 2024-03-29T08:03:53.352 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/quota/ 2024-03-29T08:03:53.352 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/quota/quota_tree.ko 2024-03-29T08:03:53.352 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/quota/quota_v1.ko 2024-03-29T08:03:53.353 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/quota/quota_v2.ko 2024-03-29T08:03:53.353 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/ 2024-03-29T08:03:53.353 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 532161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/reiserfs/reiserfs.ko 2024-03-29T08:03:53.353 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/ 2024-03-29T08:03:53.353 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/romfs/romfs.ko 2024-03-29T08:03:53.353 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/ 2024-03-29T08:03:53.353 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/ 2024-03-29T08:03:53.353 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 2712385 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/client/cifs.ko 2024-03-29T08:03:53.353 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/ 2024-03-29T08:03:53.353 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_arc4.ko 2024-03-29T08:03:53.353 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/smb/common/cifs_md4.ko 2024-03-29T08:03:53.353 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/ 2024-03-29T08:03:53.353 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 134705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/squashfs/squashfs.ko 2024-03-29T08:03:53.353 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/ 2024-03-29T08:03:53.353 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 91281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/sysv/sysv.ko 2024-03-29T08:03:53.353 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/ 2024-03-29T08:03:53.354 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 248329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/udf/udf.ko 2024-03-29T08:03:53.354 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ 2024-03-29T08:03:53.354 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 162737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/ufs/ufs.ko 2024-03-29T08:03:53.354 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/ 2024-03-29T08:03:53.354 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 4035417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/fs/xfs/xfs.ko 2024-03-29T08:03:53.354 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/ 2024-03-29T08:03:53.354 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/locking/ 2024-03-29T08:03:53.354 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 80481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/locking/locktorture.ko 2024-03-29T08:03:53.354 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/time/ 2024-03-29T08:03:53.354 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/time/test_udelay.ko 2024-03-29T08:03:53.354 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 60257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/kernel/torture.ko 2024-03-29T08:03:53.354 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ 2024-03-29T08:03:53.354 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 6729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc-itu-t.ko 2024-03-29T08:03:53.354 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 6433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc7.ko 2024-03-29T08:03:53.355 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 6929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crc8.ko 2024-03-29T08:03:53.355 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/ 2024-03-29T08:03:53.355 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libarc4.ko 2024-03-29T08:03:53.355 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/crypto/libdes.ko 2024-03-29T08:03:53.355 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/interval_tree_test.ko 2024-03-29T08:03:53.355 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/libcrc32c.ko 2024-03-29T08:03:53.355 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lru_cache.ko 2024-03-29T08:03:53.355 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/ 2024-03-29T08:03:53.355 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4_compress.ko 2024-03-29T08:03:53.355 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/lz4/lz4hc_compress.ko 2024-03-29T08:03:53.355 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/ 2024-03-29T08:03:53.355 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 6521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/math/cordic.ko 2024-03-29T08:03:53.355 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/memory-notifier-error-inject.ko 2024-03-29T08:03:53.355 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/notifier-error-inject.ko 2024-03-29T08:03:53.355 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/percpu_test.ko 2024-03-29T08:03:53.356 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/pm-notifier-error-inject.ko 2024-03-29T08:03:53.356 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/ 2024-03-29T08:03:53.356 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 143257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/raid6/raid6_pq.ko 2024-03-29T08:03:53.356 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/rbtree_test.ko 2024-03-29T08:03:53.356 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/ 2024-03-29T08:03:53.356 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/reed_solomon/reed_solomon.ko 2024-03-29T08:03:53.356 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 38737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test-kstrtox.ko 2024-03-29T08:03:53.356 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test-string_helpers.ko 2024-03-29T08:03:53.356 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 4863217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test_bpf.ko 2024-03-29T08:03:53.356 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 48985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test_firmware.ko 2024-03-29T08:03:53.356 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 6273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test_module.ko 2024-03-29T08:03:53.356 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/test_user_copy.ko 2024-03-29T08:03:53.356 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_bm.ko 2024-03-29T08:03:53.356 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_fsm.ko 2024-03-29T08:03:53.356 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/ts_kmp.ko 2024-03-29T08:03:53.357 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/xz/ 2024-03-29T08:03:53.357 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/lib/xz/xz_dec_test.ko 2024-03-29T08:03:53.357 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/ 2024-03-29T08:03:53.357 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/mm/hwpoison-inject.ko 2024-03-29T08:03:53.357 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ 2024-03-29T08:03:53.357 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/ 2024-03-29T08:03:53.357 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 55753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/6lowpan.ko 2024-03-29T08:03:53.357 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 6953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_dest.ko 2024-03-29T08:03:53.357 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 6961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_fragment.ko 2024-03-29T08:03:53.357 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 6953 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_hop.ko 2024-03-29T08:03:53.357 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 6889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_ipv6.ko 2024-03-29T08:03:53.357 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 6961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_mobility.ko 2024-03-29T08:03:53.357 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 6897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_routing.ko 2024-03-29T08:03:53.357 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/6lowpan/nhc_udp.ko 2024-03-29T08:03:53.358 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/ 2024-03-29T08:03:53.358 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/garp.ko 2024-03-29T08:03:53.358 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/mrp.ko 2024-03-29T08:03:53.358 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/p8022.ko 2024-03-29T08:03:53.358 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/psnap.ko 2024-03-29T08:03:53.358 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/802/stp.ko 2024-03-29T08:03:53.358 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/ 2024-03-29T08:03:53.358 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 82865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/8021q/8021q.ko 2024-03-29T08:03:53.358 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/ 2024-03-29T08:03:53.358 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 198809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet.ko 2024-03-29T08:03:53.358 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_fd.ko 2024-03-29T08:03:53.358 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_rdma.ko 2024-03-29T08:03:53.358 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 32241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/9p/9pnet_virtio.ko 2024-03-29T08:03:53.358 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/ 2024-03-29T08:03:53.358 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 667353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/batman-adv/batman-adv.ko 2024-03-29T08:03:53.359 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/ 2024-03-29T08:03:53.359 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 67753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/br_netfilter.ko 2024-03-29T08:03:53.359 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 790545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/bridge.ko 2024-03-29T08:03:53.359 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ 2024-03-29T08:03:53.359 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-03-29T08:03:53.359 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_among.ko 2024-03-29T08:03:53.359 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arp.ko 2024-03-29T08:03:53.359 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-03-29T08:03:53.359 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-03-29T08:03:53.359 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip.ko 2024-03-29T08:03:53.359 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-03-29T08:03:53.359 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_limit.ko 2024-03-29T08:03:53.359 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_log.ko 2024-03-29T08:03:53.359 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark.ko 2024-03-29T08:03:53.360 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-03-29T08:03:53.360 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-03-29T08:03:53.360 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-03-29T08:03:53.360 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-03-29T08:03:53.360 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_snat.ko 2024-03-29T08:03:53.360 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_stp.ko 2024-03-29T08:03:53.360 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-03-29T08:03:53.360 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-03-29T08:03:53.360 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-03-29T08:03:53.360 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-03-29T08:03:53.360 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 55937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/ebtables.ko 2024-03-29T08:03:53.360 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-03-29T08:03:53.360 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-03-29T08:03:53.360 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/ 2024-03-29T08:03:53.360 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 1046689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ceph/libceph.ko 2024-03-29T08:03:53.360 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/ 2024-03-29T08:03:53.361 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 125697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/core/pktgen.ko 2024-03-29T08:03:53.361 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/ 2024-03-29T08:03:53.361 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 194441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp.ko 2024-03-29T08:03:53.361 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_diag.ko 2024-03-29T08:03:53.361 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 58737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv4.ko 2024-03-29T08:03:53.361 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 61393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dccp/dccp_ipv6.ko 2024-03-29T08:03:53.361 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/ 2024-03-29T08:03:53.361 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 390033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/dsa_core.ko 2024-03-29T08:03:53.361 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_brcm.ko 2024-03-29T08:03:53.361 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12593 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_dsa.ko 2024-03-29T08:03:53.361 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_none.ko 2024-03-29T08:03:53.361 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/dsa/tag_trailer.ko 2024-03-29T08:03:53.361 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/hsr/ 2024-03-29T08:03:53.361 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 102033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/hsr/hsr.ko 2024-03-29T08:03:53.361 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ 2024-03-29T08:03:53.361 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ 2024-03-29T08:03:53.362 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 45481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-03-29T08:03:53.362 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 284665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154.ko 2024-03-29T08:03:53.362 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 47873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ieee802154/ieee802154_socket.ko 2024-03-29T08:03:53.362 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ 2024-03-29T08:03:53.362 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ah4.ko 2024-03-29T08:03:53.362 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/esp4.ko 2024-03-29T08:03:53.362 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 48233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/fou.ko 2024-03-29T08:03:53.362 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/gre.ko 2024-03-29T08:03:53.362 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 51209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/inet_diag.ko 2024-03-29T08:03:53.362 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 61129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_gre.ko 2024-03-29T08:03:53.362 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 61505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_tunnel.ko 2024-03-29T08:03:53.362 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ip_vti.ko 2024-03-29T08:03:53.362 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipcomp.ko 2024-03-29T08:03:53.362 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/ipip.ko 2024-03-29T08:03:53.362 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ 2024-03-29T08:03:53.363 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 38001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arp_tables.ko 2024-03-29T08:03:53.363 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-03-29T08:03:53.363 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-03-29T08:03:53.363 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 42049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ip_tables.ko 2024-03-29T08:03:53.363 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-03-29T08:03:53.363 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9577 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-03-29T08:03:53.363 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-03-29T08:03:53.363 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-03-29T08:03:53.363 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14777 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-03-29T08:03:53.363 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-03-29T08:03:53.363 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-03-29T08:03:53.363 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-03-29T08:03:53.363 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-03-29T08:03:53.363 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/iptable_security.ko 2024-03-29T08:03:53.363 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-03-29T08:03:53.364 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-03-29T08:03:53.364 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-03-29T08:03:53.364 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-03-29T08:03:53.364 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-03-29T08:03:53.364 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-03-29T08:03:53.364 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-03-29T08:03:53.364 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-03-29T08:03:53.364 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-03-29T08:03:53.364 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-03-29T08:03:53.364 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-03-29T08:03:53.364 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_bic.ko 2024-03-29T08:03:53.364 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_dctcp.ko 2024-03-29T08:03:53.364 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15961 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_diag.ko 2024-03-29T08:03:53.364 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_highspeed.ko 2024-03-29T08:03:53.365 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_htcp.ko 2024-03-29T08:03:53.365 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_hybla.ko 2024-03-29T08:03:53.365 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_illinois.ko 2024-03-29T08:03:53.365 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_lp.ko 2024-03-29T08:03:53.365 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_scalable.ko 2024-03-29T08:03:53.365 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_vegas.ko 2024-03-29T08:03:53.365 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_veno.ko 2024-03-29T08:03:53.365 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10849 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_westwood.ko 2024-03-29T08:03:53.365 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tcp_yeah.ko 2024-03-29T08:03:53.365 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/tunnel4.ko 2024-03-29T08:03:53.365 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_diag.ko 2024-03-29T08:03:53.365 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 45041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/udp_tunnel.ko 2024-03-29T08:03:53.365 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv4/xfrm4_tunnel.ko 2024-03-29T08:03:53.365 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ 2024-03-29T08:03:53.365 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ah6.ko 2024-03-29T08:03:53.365 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/esp6.ko 2024-03-29T08:03:53.366 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/fou6.ko 2024-03-29T08:03:53.366 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 79921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_gre.ko 2024-03-29T08:03:53.366 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 94161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_tunnel.ko 2024-03-29T08:03:53.366 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-03-29T08:03:53.366 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 54265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ip6_vti.ko 2024-03-29T08:03:53.366 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/ipcomp6.ko 2024-03-29T08:03:53.366 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/mip6.ko 2024-03-29T08:03:53.366 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ 2024-03-29T08:03:53.366 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 42545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-03-29T08:03:53.366 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-03-29T08:03:53.366 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-03-29T08:03:53.366 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14033 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-03-29T08:03:53.366 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-03-29T08:03:53.366 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-03-29T08:03:53.367 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16105 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-03-29T08:03:53.367 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-03-29T08:03:53.367 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-03-29T08:03:53.367 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-03-29T08:03:53.367 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-03-29T08:03:53.367 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-03-29T08:03:53.367 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-03-29T08:03:53.367 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-03-29T08:03:53.367 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15017 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-03-29T08:03:53.367 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-03-29T08:03:53.367 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-03-29T08:03:53.367 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-03-29T08:03:53.367 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-03-29T08:03:53.367 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-03-29T08:03:53.367 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-03-29T08:03:53.368 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-03-29T08:03:53.368 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-03-29T08:03:53.368 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-03-29T08:03:53.368 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8905 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-03-29T08:03:53.368 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 74873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/sit.ko 2024-03-29T08:03:53.368 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/tunnel6.ko 2024-03-29T08:03:53.368 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/ipv6/xfrm6_tunnel.ko 2024-03-29T08:03:53.368 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/ 2024-03-29T08:03:53.368 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 84393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/key/af_key.ko 2024-03-29T08:03:53.368 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/ 2024-03-29T08:03:53.368 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 125201 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_core.ko 2024-03-29T08:03:53.368 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_debugfs.ko 2024-03-29T08:03:53.368 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 25809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_eth.ko 2024-03-29T08:03:53.368 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip.ko 2024-03-29T08:03:53.369 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 37369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ip6.ko 2024-03-29T08:03:53.369 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_netlink.ko 2024-03-29T08:03:53.369 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 53225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/l2tp/l2tp_ppp.ko 2024-03-29T08:03:53.369 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/lapb/ 2024-03-29T08:03:53.369 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 48097 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/lapb/lapb.ko 2024-03-29T08:03:53.369 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/ 2024-03-29T08:03:53.369 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc.ko 2024-03-29T08:03:53.369 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 268449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/llc/llc2.ko 2024-03-29T08:03:53.369 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/ 2024-03-29T08:03:53.369 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 309361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mac802154/mac802154.ko 2024-03-29T08:03:53.369 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/ 2024-03-29T08:03:53.369 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/mpls/mpls_gso.ko 2024-03-29T08:03:53.369 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ 2024-03-29T08:03:53.369 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ 2024-03-29T08:03:53.369 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 119689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set.ko 2024-03-29T08:03:53.369 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28609 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-03-29T08:03:53.370 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-03-29T08:03:53.370 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-03-29T08:03:53.370 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 99985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-03-29T08:03:53.370 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 100273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-03-29T08:03:53.370 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 102545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-03-29T08:03:53.370 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 102337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-03-29T08:03:53.370 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 111665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-03-29T08:03:53.370 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 59185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-03-29T08:03:53.370 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 108337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-03-29T08:03:53.370 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 112137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-03-29T08:03:53.370 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 112401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-03-29T08:03:53.370 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 110329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-03-29T08:03:53.370 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 113281 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-03-29T08:03:53.370 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-03-29T08:03:53.370 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ 2024-03-29T08:03:53.370 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 412433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs.ko 2024-03-29T08:03:53.371 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11985 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-03-29T08:03:53.371 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-03-29T08:03:53.371 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-03-29T08:03:53.371 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-03-29T08:03:53.371 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-03-29T08:03:53.371 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-03-29T08:03:53.371 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-03-29T08:03:53.371 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10633 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-03-29T08:03:53.371 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9257 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-03-29T08:03:53.371 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-03-29T08:03:53.371 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13537 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-03-29T08:03:53.371 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7713 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-03-29T08:03:53.371 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-03-29T08:03:53.371 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 49161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conncount.ko 2024-03-29T08:03:53.371 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 411665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack.ko 2024-03-29T08:03:53.371 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15873 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-03-29T08:03:53.372 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-03-29T08:03:53.372 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-03-29T08:03:53.372 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 118145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_h323.ko 2024-03-29T08:03:53.372 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_irc.ko 2024-03-29T08:03:53.372 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-03-29T08:03:53.372 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 108305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-03-29T08:03:53.372 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-03-29T08:03:53.372 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17289 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sane.ko 2024-03-29T08:03:53.372 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 66529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_sip.ko 2024-03-29T08:03:53.372 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-03-29T08:03:53.372 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-03-29T08:03:53.372 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_dup_netdev.ko 2024-03-29T08:03:53.372 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 92217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table.ko 2024-03-29T08:03:53.372 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_flow_table_inet.ko 2024-03-29T08:03:53.372 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 38073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_log_syslog.ko 2024-03-29T08:03:53.373 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 96489 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat.ko 2024-03-29T08:03:53.373 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_amanda.ko 2024-03-29T08:03:53.373 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_ftp.ko 2024-03-29T08:03:53.373 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11753 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_irc.ko 2024-03-29T08:03:53.373 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_sip.ko 2024-03-29T08:03:53.373 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_nat_tftp.ko 2024-03-29T08:03:53.373 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 44129 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_synproxy_core.ko 2024-03-29T08:03:53.373 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 623337 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nf_tables.ko 2024-03-29T08:03:53.373 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 42505 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink.ko 2024-03-29T08:03:53.373 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30153 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_acct.ko 2024-03-29T08:03:53.373 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-03-29T08:03:53.373 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_log.ko 2024-03-29T08:03:53.373 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 22929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_osf.ko 2024-03-29T08:03:53.373 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 49641 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nfnetlink_queue.ko 2024-03-29T08:03:53.373 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_chain_nat.ko 2024-03-29T08:03:53.373 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_compat.ko 2024-03-29T08:03:53.374 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 40801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_ct.ko 2024-03-29T08:03:53.374 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9497 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_dup_netdev.ko 2024-03-29T08:03:53.374 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fib.ko 2024-03-29T08:03:53.374 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_fwd_netdev.ko 2024-03-29T08:03:53.374 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_hash.ko 2024-03-29T08:03:53.374 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20025 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_limit.ko 2024-03-29T08:03:53.374 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_log.ko 2024-03-29T08:03:53.374 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_masq.ko 2024-03-29T08:03:53.374 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14321 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_nat.ko 2024-03-29T08:03:53.374 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_numgen.ko 2024-03-29T08:03:53.374 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11745 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_osf.ko 2024-03-29T08:03:53.374 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_queue.ko 2024-03-29T08:03:53.374 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_redir.ko 2024-03-29T08:03:53.374 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject.ko 2024-03-29T08:03:53.374 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9977 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_reject_inet.ko 2024-03-29T08:03:53.375 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12833 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_socket.ko 2024-03-29T08:03:53.375 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_synproxy.ko 2024-03-29T08:03:53.375 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/nft_tproxy.ko 2024-03-29T08:03:53.375 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 89225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/x_tables.ko 2024-03-29T08:03:53.375 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_AUDIT.ko 2024-03-29T08:03:53.375 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CHECKSUM.ko 2024-03-29T08:03:53.375 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7625 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CLASSIFY.ko 2024-03-29T08:03:53.375 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-03-29T08:03:53.375 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 18073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_CT.ko 2024-03-29T08:03:53.375 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_DSCP.ko 2024-03-29T08:03:53.375 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HL.ko 2024-03-29T08:03:53.375 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13273 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_HMARK.ko 2024-03-29T08:03:53.375 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 38769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_IDLETIMER.ko 2024-03-29T08:03:53.375 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13793 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LED.ko 2024-03-29T08:03:53.375 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_LOG.ko 2024-03-29T08:03:53.376 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_MASQUERADE.ko 2024-03-29T08:03:53.376 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11553 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NETMAP.ko 2024-03-29T08:03:53.376 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFLOG.ko 2024-03-29T08:03:53.376 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_NFQUEUE.ko 2024-03-29T08:03:53.376 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_RATEEST.ko 2024-03-29T08:03:53.376 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_REDIRECT.ko 2024-03-29T08:03:53.376 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_SECMARK.ko 2024-03-29T08:03:53.376 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPMSS.ko 2024-03-29T08:03:53.376 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-03-29T08:03:53.376 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TEE.ko 2024-03-29T08:03:53.376 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TPROXY.ko 2024-03-29T08:03:53.376 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8225 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_TRACE.ko 2024-03-29T08:03:53.376 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_addrtype.ko 2024-03-29T08:03:53.376 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_bpf.ko 2024-03-29T08:03:53.376 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cgroup.ko 2024-03-29T08:03:53.376 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cluster.ko 2024-03-29T08:03:53.377 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_comment.ko 2024-03-29T08:03:53.377 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connbytes.ko 2024-03-29T08:03:53.377 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11249 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlabel.ko 2024-03-29T08:03:53.377 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connlimit.ko 2024-03-29T08:03:53.377 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_connmark.ko 2024-03-29T08:03:53.377 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12297 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_conntrack.ko 2024-03-29T08:03:53.377 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_cpu.ko 2024-03-29T08:03:53.377 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dccp.ko 2024-03-29T08:03:53.377 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7761 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_devgroup.ko 2024-03-29T08:03:53.377 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_dscp.ko 2024-03-29T08:03:53.377 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10561 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ecn.ko 2024-03-29T08:03:53.377 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_esp.ko 2024-03-29T08:03:53.377 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 43193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hashlimit.ko 2024-03-29T08:03:53.377 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_helper.ko 2024-03-29T08:03:53.377 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_hl.ko 2024-03-29T08:03:53.378 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11817 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipcomp.ko 2024-03-29T08:03:53.378 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_iprange.ko 2024-03-29T08:03:53.378 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11689 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_ipvs.ko 2024-03-29T08:03:53.378 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15945 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_l2tp.ko 2024-03-29T08:03:53.378 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7897 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_length.ko 2024-03-29T08:03:53.378 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10113 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_limit.ko 2024-03-29T08:03:53.378 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mac.ko 2024-03-29T08:03:53.378 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_mark.ko 2024-03-29T08:03:53.378 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11721 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_multiport.ko 2024-03-29T08:03:53.378 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13729 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nat.ko 2024-03-29T08:03:53.378 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10001 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_nfacct.ko 2024-03-29T08:03:53.378 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_osf.ko 2024-03-29T08:03:53.378 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_owner.ko 2024-03-29T08:03:53.378 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_physdev.ko 2024-03-29T08:03:53.378 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_pkttype.ko 2024-03-29T08:03:53.379 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11889 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_policy.ko 2024-03-29T08:03:53.379 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9377 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_quota.ko 2024-03-29T08:03:53.379 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_rateest.ko 2024-03-29T08:03:53.379 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8161 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_realm.ko 2024-03-29T08:03:53.379 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36353 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_recent.ko 2024-03-29T08:03:53.379 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13233 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_sctp.ko 2024-03-29T08:03:53.379 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 29993 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_set.ko 2024-03-29T08:03:53.379 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 16809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_socket.ko 2024-03-29T08:03:53.379 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9409 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_state.ko 2024-03-29T08:03:53.379 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_statistic.ko 2024-03-29T08:03:53.379 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8705 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_string.ko 2024-03-29T08:03:53.379 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 8449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpmss.ko 2024-03-29T08:03:53.379 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15673 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_tcpudp.ko 2024-03-29T08:03:53.379 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_time.ko 2024-03-29T08:03:53.379 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9305 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netfilter/xt_u32.ko 2024-03-29T08:03:53.380 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/ 2024-03-29T08:03:53.380 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 14433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/netlink/netlink_diag.ko 2024-03-29T08:03:53.380 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/ 2024-03-29T08:03:53.380 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 12329 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/nsh/nsh.ko 2024-03-29T08:03:53.380 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/ 2024-03-29T08:03:53.380 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 384137 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/openvswitch.ko 2024-03-29T08:03:53.380 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10361 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-geneve.ko 2024-03-29T08:03:53.380 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-gre.ko 2024-03-29T08:03:53.380 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11825 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/openvswitch/vport-vxlan.ko 2024-03-29T08:03:53.380 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/ 2024-03-29T08:03:53.380 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13465 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/packet/af_packet_diag.ko 2024-03-29T08:03:53.380 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/phonet/ 2024-03-29T08:03:53.380 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 112521 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/phonet/phonet.ko 2024-03-29T08:03:53.380 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 55041 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/phonet/pn_pep.ko 2024-03-29T08:03:53.380 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/ 2024-03-29T08:03:53.380 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 253881 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds.ko 2024-03-29T08:03:53.380 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 165049 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_rdma.ko 2024-03-29T08:03:53.381 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 68065 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rds/rds_tcp.ko 2024-03-29T08:03:53.381 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rxrpc/ 2024-03-29T08:03:53.381 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 828145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/rxrpc/rxrpc.ko 2024-03-29T08:03:53.381 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/ 2024-03-29T08:03:53.381 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 32145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_csum.ko 2024-03-29T08:03:53.381 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23481 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_gact.ko 2024-03-29T08:03:53.381 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 37081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_mirred.ko 2024-03-29T08:03:53.381 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23369 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_nat.ko 2024-03-29T08:03:53.381 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 43345 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_pedit.ko 2024-03-29T08:03:53.381 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_police.ko 2024-03-29T08:03:53.381 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20185 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_simple.ko 2024-03-29T08:03:53.381 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 35193 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_skbedit.ko 2024-03-29T08:03:53.381 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36569 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/act_vlan.ko 2024-03-29T08:03:53.381 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_basic.ko 2024-03-29T08:03:53.381 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33865 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_bpf.ko 2024-03-29T08:03:53.381 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 19177 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_cgroup.ko 2024-03-29T08:03:53.382 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_flow.ko 2024-03-29T08:03:53.382 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_fw.ko 2024-03-29T08:03:53.382 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 39513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_route.ko 2024-03-29T08:03:53.382 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 62649 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/cls_u32.ko 2024-03-29T08:03:53.382 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7457 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_cmp.ko 2024-03-29T08:03:53.382 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 13313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_ipset.ko 2024-03-29T08:03:53.382 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 33417 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_meta.ko 2024-03-29T08:03:53.382 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7913 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_nbyte.ko 2024-03-29T08:03:53.382 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10921 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_text.ko 2024-03-29T08:03:53.382 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/em_u32.ko 2024-03-29T08:03:53.382 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21665 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_choke.ko 2024-03-29T08:03:53.382 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20857 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_codel.ko 2024-03-29T08:03:53.382 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 34585 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_drr.ko 2024-03-29T08:03:53.382 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 35785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq.ko 2024-03-29T08:03:53.383 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31513 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_fq_codel.ko 2024-03-29T08:03:53.383 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 35449 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_gred.ko 2024-03-29T08:03:53.383 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 49841 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hfsc.ko 2024-03-29T08:03:53.383 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 21969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_hhf.ko 2024-03-29T08:03:53.383 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 66441 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_htb.ko 2024-03-29T08:03:53.383 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_ingress.ko 2024-03-29T08:03:53.383 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 30217 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio.ko 2024-03-29T08:03:53.383 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10801 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_mqprio_lib.ko 2024-03-29T08:03:53.383 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26081 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_multiq.ko 2024-03-29T08:03:53.383 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 36145 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_netem.ko 2024-03-29T08:03:53.383 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 27401 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_pie.ko 2024-03-29T08:03:53.383 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10657 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_plug.ko 2024-03-29T08:03:53.383 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 24769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_prio.ko 2024-03-29T08:03:53.383 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 48529 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_qfq.ko 2024-03-29T08:03:53.383 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 32809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_red.ko 2024-03-29T08:03:53.383 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 26785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfb.ko 2024-03-29T08:03:53.384 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 31313 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_sfq.ko 2024-03-29T08:03:53.384 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_tbf.ko 2024-03-29T08:03:53.384 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 28265 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sched/sch_teql.ko 2024-03-29T08:03:53.384 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/ 2024-03-29T08:03:53.384 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 932785 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp.ko 2024-03-29T08:03:53.384 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20241 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sctp/sctp_diag.ko 2024-03-29T08:03:53.384 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/ 2024-03-29T08:03:53.384 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/ 2024-03-29T08:03:53.384 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 336969 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-03-29T08:03:53.384 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 55009 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-03-29T08:03:53.384 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 1513769 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/sunrpc.ko 2024-03-29T08:03:53.384 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/ 2024-03-29T08:03:53.384 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 704697 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-03-29T08:03:53.384 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/ 2024-03-29T08:03:53.384 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 10393 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/diag.ko 2024-03-29T08:03:53.385 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 867121 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/tipc/tipc.ko 2024-03-29T08:03:53.385 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/ 2024-03-29T08:03:53.385 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15929 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/unix/unix_diag.ko 2024-03-29T08:03:53.385 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/ 2024-03-29T08:03:53.385 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 87737 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-03-29T08:03:53.385 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 67937 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-03-29T08:03:53.385 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 98089 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock.ko 2024-03-29T08:03:53.385 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 11073 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_diag.ko 2024-03-29T08:03:53.385 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 17809 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/vmw_vsock/vsock_loopback.ko 2024-03-29T08:03:53.385 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/x25/ 2024-03-29T08:03:53.385 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 131209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/x25/x25.ko 2024-03-29T08:03:53.385 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/ 2024-03-29T08:03:53.385 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23057 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_algo.ko 2024-03-29T08:03:53.385 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 20545 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_ipcomp.ko 2024-03-29T08:03:53.385 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 105209 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/net/xfrm/xfrm_user.ko 2024-03-29T08:03:53.385 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/ 2024-03-29T08:03:53.386 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/ 2024-03-29T08:03:53.386 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 15433 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/kernel/virt/lib/irqbypass.ko 2024-03-29T08:03:53.386 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 642383 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.alias 2024-03-29T08:03:53.386 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 625850 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.alias.bin 2024-03-29T08:03:53.386 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 7677 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.builtin 2024-03-29T08:03:53.386 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 23498 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.alias.bin 2024-03-29T08:03:53.386 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 9916 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.bin 2024-03-29T08:03:53.386 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 70732 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.builtin.modinfo 2024-03-29T08:03:53.386 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 147214 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.dep 2024-03-29T08:03:53.386 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 233213 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.dep.bin 2024-03-29T08:03:53.386 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 210 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.devname 2024-03-29T08:03:53.386 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 72614 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.order 2024-03-29T08:03:53.386 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 879 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.softdep 2024-03-29T08:03:53.386 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 257955 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.symbols 2024-03-29T08:03:53.386 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 312782 2024-03-25 03:29 ./lib/modules/6.8.0-g1df0c7e8a235/modules.symbols.bin 2024-03-29T08:03:53.387 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./usr/ 2024-03-29T08:03:53.387 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./usr/share/ 2024-03-29T08:03:53.387 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./usr/share/doc/ 2024-03-29T08:03:53.387 INFO:teuthology.orchestra.run.smithi073.stdout:drwxr-xr-x root/root 0 2024-03-25 03:29 ./usr/share/doc/linux-image-6.8.0-g1df0c7e8a235/ 2024-03-29T08:03:53.387 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 169 2024-03-25 03:29 ./usr/share/doc/linux-image-6.8.0-g1df0c7e8a235/changelog.Debian.gz 2024-03-29T08:03:53.387 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r-- root/root 691 2024-03-25 03:29 ./usr/share/doc/linux-image-6.8.0-g1df0c7e8a235/copyright 2024-03-29T08:03:53.391 DEBUG:teuthology.task.kernel:get_image_version: 6.8.0-g1df0c7e8a235 2024-03-29T08:03:53.391 INFO:teuthology.task.kernel:searching for kernel 6.8.0-g1df0c7e8a235 2024-03-29T08:03:53.392 DEBUG:teuthology.orchestra.run.smithi073:> egrep '(submenu|menuentry.*6.8.0-g1df0c7e8a235).*{' /boot/grub/grub.cfg 2024-03-29T08:03:53.398 INFO:teuthology.orchestra.run.smithi073.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-29T08:03:53.399 INFO:teuthology.orchestra.run.smithi073.stdout: menuentry 'Ubuntu, with Linux 6.8.0-g1df0c7e8a235' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.8.0-g1df0c7e8a235-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-29T08:03:53.399 INFO:teuthology.orchestra.run.smithi073.stdout: menuentry 'Ubuntu, with Linux 6.8.0-g1df0c7e8a235 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.8.0-g1df0c7e8a235-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-03-29T08:03:53.399 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-03-29T08:03:53.399 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.8.0-g1df0c7e8a235 2024-03-29T08:03:53.399 DEBUG:teuthology.orchestra.run.smithi073:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-03-29T08:03:53.444 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi073.front.sepia.ceph.com to complete... 2024-03-29T08:03:53.964 INFO:teuthology.orchestra.run.smithi073.stderr:Sourcing file `/etc/default/grub' 2024-03-29T08:03:53.965 INFO:teuthology.orchestra.run.smithi073.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-29T08:03:54.007 INFO:teuthology.orchestra.run.smithi073.stderr:Generating grub configuration file ... 2024-03-29T08:03:54.246 INFO:teuthology.orchestra.run.smithi073.stderr:Found linux image: /boot/vmlinuz-6.8.0-g1df0c7e8a235 2024-03-29T08:03:54.253 INFO:teuthology.orchestra.run.smithi073.stderr:Found initrd image: /boot/initrd.img-6.8.0-g1df0c7e8a235 2024-03-29T08:03:54.545 INFO:teuthology.orchestra.run.smithi073.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-29T08:03:54.550 INFO:teuthology.orchestra.run.smithi073.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-29T08:03:54.594 INFO:teuthology.orchestra.run.smithi073.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-29T08:03:54.600 INFO:teuthology.orchestra.run.smithi073.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-29T08:03:54.748 INFO:teuthology.orchestra.run.smithi073.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-29T08:03:54.748 INFO:teuthology.orchestra.run.smithi073.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-29T08:03:54.748 INFO:teuthology.orchestra.run.smithi073.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-29T08:03:54.774 INFO:teuthology.orchestra.run.smithi073.stderr:done 2024-03-29T08:04:16.650 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-29T08:04:16.651 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-03-29T08:04:16.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:04:25.910 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-29T08:04:25.911 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-03-29T08:04:25.911 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:04:34.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-03-29T08:04:43.984 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-03-29T08:04:43.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:04:44.446 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-03-29T08:04:47.038 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-03-29T08:04:53.450 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-03-29T08:04:53.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:04:56.510 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-03-29T08:04:59.042 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-03-29T08:04:59.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:05:08.518 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-03-29T08:05:08.519 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:05:40.226 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-03-29T08:05:40.810 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-03-29T08:05:40.810 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.8.0-g1df0c7e8a235"... 2024-03-29T08:05:40.810 DEBUG:teuthology.orchestra.run.smithi073:> uname -r 2024-03-29T08:05:40.858 INFO:teuthology.orchestra.run.smithi073.stdout:6.8.0-g1df0c7e8a235 2024-03-29T08:05:40.858 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-29T08:05:40.858 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-29T08:05:40.858 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-29T08:05:41.859 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-29T08:05:41.859 DEBUG:teuthology.orchestra.run.smithi073:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-29T08:05:41.938 INFO:teuthology.orchestra.run.smithi073.stdout:ttyS1 2024-03-29T08:05:41.974 DEBUG:teuthology.parallel:result is None 2024-03-29T08:05:59.045 DEBUG:teuthology.orchestra.remote:timed out 2024-03-29T08:06:14.046 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-03-29T08:06:14.047 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:06:14.295 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-03-29T08:06:15.010 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-03-29T08:06:15.010 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.8.0-g1df0c7e8a235"... 2024-03-29T08:06:15.010 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-03-29T08:06:15.060 INFO:teuthology.orchestra.run.smithi046.stdout:6.8.0-g1df0c7e8a235 2024-03-29T08:06:15.060 DEBUG:teuthology.task.kernel:current kernel version is 6.8.0-g1df0c7e8a235 vs 6.8.0-g1df0c7e8a235 2024-03-29T08:06:15.060 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-29T08:06:15.061 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-29T08:06:16.061 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-29T08:06:16.062 DEBUG:teuthology.orchestra.run.smithi046:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-29T08:06:16.141 INFO:teuthology.orchestra.run.smithi046.stdout:ttyS1 2024-03-29T08:06:16.191 DEBUG:teuthology.parallel:result is None 2024-03-29T08:06:16.192 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-29T08:06:16.199 INFO:teuthology.task.internal:Creating test directory... 2024-03-29T08:06:16.200 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-29T08:06:16.203 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-29T08:06:16.212 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-29T08:06:16.224 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-29T08:06:16.233 INFO:teuthology.task.internal:Creating archive directory... 2024-03-29T08:06:16.233 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-29T08:06:16.254 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-29T08:06:16.278 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-29T08:06:16.284 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-29T08:06:16.285 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-29T08:06:16.315 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-29T08:06:16.340 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T08:06:16.348 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T08:06:16.354 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T08:06:16.362 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-29T08:06:16.365 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-29T08:06:16.372 INFO:teuthology.task.internal:Configuring sudo... 2024-03-29T08:06:16.372 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-29T08:06:16.407 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-29T08:06:16.430 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-29T08:06:16.440 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-29T08:06:16.440 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-29T08:06:16.475 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-29T08:06:16.483 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T08:06:16.532 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T08:06:16.584 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:06:16.585 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-29T08:06:16.654 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-29T08:06:16.663 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-29T08:06:16.715 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:06:16.715 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-29T08:06:16.787 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-03-29T08:06:16.790 DEBUG:teuthology.orchestra.run.smithi073:> sudo service rsyslog restart 2024-03-29T08:06:16.924 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-29T08:06:16.931 INFO:teuthology.task.internal:Starting timer... 2024-03-29T08:06:16.932 INFO:teuthology.run_tasks:Running task pcp... 2024-03-29T08:06:16.977 INFO:teuthology.run_tasks:Running task selinux... 2024-03-29T08:06:16.992 DEBUG:teuthology.task.selinux:Excluding smithi046: OS 'ubuntu' does not support SELinux 2024-03-29T08:06:16.992 DEBUG:teuthology.task.selinux:Excluding smithi073: OS 'ubuntu' does not support SELinux 2024-03-29T08:06:16.992 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-29T08:06:16.992 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-03-29T08:06:16.992 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-29T08:06:16.992 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-29T08:06:17.003 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-29T08:06:17.020 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-29T08:06:17.021 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi073.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-29T08:13:59.420 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi073.front.sepia.ceph.com')] 2024-03-29T08:13:59.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-03-29T08:13:59.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:13:59.507 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-03-29T08:13:59.621 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-03-29T08:13:59.621 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-03-29T08:13:59.622 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-29T08:13:59.704 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-03-29T08:13:59.829 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-03-29T08:13:59.829 INFO:teuthology.run_tasks:Running task clock... 2024-03-29T08:13:59.842 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-29T08:13:59.842 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-29T08:13:59.842 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T08:13:59.846 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-29T08:13:59.846 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T08:13:59.883 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-29T08:13:59.884 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: Command line: ntpd -gq 2024-03-29T08:13:59.884 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: ---------------------------------------------------- 2024-03-29T08:13:59.884 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: ntp-4 is maintained by Network Time Foundation, 2024-03-29T08:13:59.884 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-29T08:13:59.884 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: corporation. Support and training for ntp-4 are 2024-03-29T08:13:59.884 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: available at https://www.nwtime.org/support 2024-03-29T08:13:59.884 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: ---------------------------------------------------- 2024-03-29T08:13:59.885 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: proto: precision = 0.042 usec (-24) 2024-03-29T08:13:59.885 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: basedate set to 2022-02-04 2024-03-29T08:13:59.885 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: gps base set to 2022-02-06 (week 2196) 2024-03-29T08:13:59.885 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-29T08:13:59.885 INFO:teuthology.orchestra.run.smithi046.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-29T08:13:59.885 INFO:teuthology.orchestra.run.smithi046.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-29T08:13:59.886 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: restrict ::: KOD does nothing without LIMITED. 2024-03-29T08:13:59.887 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: Listen and drop on 0 v6wildcard [::]:123 2024-03-29T08:13:59.887 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-29T08:13:59.887 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: Listen normally on 2 lo 127.0.0.1:123 2024-03-29T08:13:59.887 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: Listen normally on 3 ens1f0 172.21.15.46:123 2024-03-29T08:13:59.887 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: Listen normally on 4 lo [::1]:123 2024-03-29T08:13:59.887 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15f8%4]:123 2024-03-29T08:13:59.887 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:13:59 ntpd[17061]: Listening on routing socket on fd #22 for interface updates 2024-03-29T08:13:59.910 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-29T08:13:59.910 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: Command line: ntpd -gq 2024-03-29T08:13:59.911 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: ---------------------------------------------------- 2024-03-29T08:13:59.911 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: ntp-4 is maintained by Network Time Foundation, 2024-03-29T08:13:59.911 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-29T08:13:59.911 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: corporation. Support and training for ntp-4 are 2024-03-29T08:13:59.911 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: available at https://www.nwtime.org/support 2024-03-29T08:13:59.911 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: ---------------------------------------------------- 2024-03-29T08:13:59.911 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: proto: precision = 0.042 usec (-24) 2024-03-29T08:13:59.912 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: basedate set to 2022-02-04 2024-03-29T08:13:59.912 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: gps base set to 2022-02-06 (week 2196) 2024-03-29T08:13:59.912 INFO:teuthology.orchestra.run.smithi073.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-29T08:13:59.912 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-29T08:13:59.913 INFO:teuthology.orchestra.run.smithi073.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-29T08:13:59.913 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: restrict ::: KOD does nothing without LIMITED. 2024-03-29T08:13:59.914 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: Listen and drop on 0 v6wildcard [::]:123 2024-03-29T08:13:59.914 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-29T08:13:59.914 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: Listen normally on 2 lo 127.0.0.1:123 2024-03-29T08:13:59.914 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: Listen normally on 3 enp3s0f1 172.21.15.73:123 2024-03-29T08:13:59.914 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: Listen normally on 4 lo [::1]:123 2024-03-29T08:13:59.914 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce7d%5]:123 2024-03-29T08:13:59.914 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:13:59 ntpd[17176]: Listening on routing socket on fd #22 for interface updates 2024-03-29T08:14:00.884 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:00 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:00 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:00 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:00 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:00 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:00 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:00 ntpd[17061]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:00.885 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:00 ntpd[17061]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:00.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:00 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:00.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:00 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:00.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:00 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:00.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:00 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:00.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:00 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:00.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:00 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:00.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:00 ntpd[17176]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:00.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:00 ntpd[17176]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:00.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:00 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:00.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:00 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:00.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:00 ntpd[17176]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:00.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:00 ntpd[17176]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:02.885 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:02.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:02 ntpd[17061]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:02.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:02.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:02.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:02.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:02.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:02.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:02.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:02.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:02.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:02.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:02.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:02.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:02.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:02.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:02.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:02.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:02.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:02.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:02.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:02.914 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:02 ntpd[17176]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:04.887 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:04 ntpd[17061]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:04.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:04.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:04.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:04.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:04.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:04.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:04.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:04.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:04.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:04.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:04.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:04.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:04.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:04.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:04.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:04.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:04.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:04.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:04 ntpd[17176]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:06.885 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:06.885 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:06.886 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:06.887 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:06.887 INFO:teuthology.orchestra.run.smithi046.stdout:29 Mar 08:14:06 ntpd[17061]: ntpd: time slew +0.000090 s 2024-03-29T08:14:06.887 INFO:teuthology.orchestra.run.smithi046.stdout:ntpd: time slew +0.000090s 2024-03-29T08:14:06.888 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-29T08:14:06.888 INFO:teuthology.orchestra.run.smithi046.stderr:29 Mar 08:14:06 ntpd[17061]: can't open /var/log/ntpstats/loopstats.20240329: Permission denied 2024-03-29T08:14:06.911 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:06 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:06.911 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:06 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:06.911 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:06 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:06.911 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:06 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:06.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:06 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:06.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:06 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:06.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:06 ntpd[17176]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-29T08:14:06.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:06 ntpd[17176]: can't open /var/log/ntpstats/rawstats.20240329: Permission denied 2024-03-29T08:14:06.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:06 ntpd[17176]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-29T08:14:06.912 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:06 ntpd[17176]: can't open /var/log/ntpstats/peerstats.20240329: Permission denied 2024-03-29T08:14:06.912 INFO:teuthology.orchestra.run.smithi073.stdout:29 Mar 08:14:06 ntpd[17176]: ntpd: time slew -0.000722 s 2024-03-29T08:14:06.912 INFO:teuthology.orchestra.run.smithi073.stdout:ntpd: time slew -0.000722s 2024-03-29T08:14:06.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:06 ntpd[17176]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-29T08:14:06.913 INFO:teuthology.orchestra.run.smithi073.stderr:29 Mar 08:14:06 ntpd[17176]: can't open /var/log/ntpstats/loopstats.20240329: Permission denied 2024-03-29T08:14:06.969 INFO:teuthology.orchestra.run.smithi046.stdout: remote refid st t when poll reach delay offset jitter 2024-03-29T08:14:06.969 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================== 2024-03-29T08:14:06.969 INFO:teuthology.orchestra.run.smithi046.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T08:14:06.970 INFO:teuthology.orchestra.run.smithi046.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T08:14:06.970 INFO:teuthology.orchestra.run.smithi046.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T08:14:06.970 INFO:teuthology.orchestra.run.smithi046.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T08:14:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: remote refid st t when poll reach delay offset jitter 2024-03-29T08:14:07.029 INFO:teuthology.orchestra.run.smithi073.stdout:============================================================================== 2024-03-29T08:14:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T08:14:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T08:14:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T08:14:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-29T08:14:07.030 INFO:teuthology.run_tasks:Running task install... 2024-03-29T08:14:07.042 DEBUG:teuthology.task.install:project ceph 2024-03-29T08:14:07.042 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2'}} 2024-03-29T08:14:07.043 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2'} 2024-03-29T08:14:07.043 INFO:teuthology.task.install:Using flavor: default 2024-03-29T08:14:07.051 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-29T08:14:07.051 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-03-29T08:14:07.052 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-key list | grep Ceph 2024-03-29T08:14:07.052 DEBUG:teuthology.orchestra.run.smithi073:> sudo apt-key list | grep Ceph 2024-03-29T08:14:07.158 INFO:teuthology.orchestra.run.smithi046.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-29T08:14:07.171 INFO:teuthology.orchestra.run.smithi073.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-29T08:14:07.210 INFO:teuthology.orchestra.run.smithi046.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-29T08:14:07.211 INFO:teuthology.orchestra.run.smithi046.stdout:uid [ unknown] Ceph.com (release key) 2024-03-29T08:14:07.212 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-29T08:14:07.212 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-29T08:14:07.213 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:14:07.284 INFO:teuthology.orchestra.run.smithi073.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-29T08:14:07.284 INFO:teuthology.orchestra.run.smithi073.stdout:uid [ unknown] Ceph.com (release key) 2024-03-29T08:14:07.285 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-29T08:14:07.285 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-29T08:14:07.285 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:14:07.434 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default/ 2024-03-29T08:14:07.434 INFO:teuthology.task.install.deb:Package version is 18.2.2-1015-gd0e392d9-1jammy 2024-03-29T08:14:07.438 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default/ 2024-03-29T08:14:07.438 INFO:teuthology.task.install.deb:Package version is 18.2.2-1015-gd0e392d9-1jammy 2024-03-29T08:14:07.774 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:14:07.774 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-29T08:14:07.779 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:14:07.779 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-29T08:14:07.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get update 2024-03-29T08:14:07.802 DEBUG:teuthology.orchestra.run.smithi073:> sudo apt-get update 2024-03-29T08:14:08.005 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-29T08:14:08.006 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-29T08:14:08.015 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-29T08:14:08.035 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-29T08:14:08.125 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-29T08:14:08.129 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-29T08:14:08.196 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-29T08:14:08.278 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-29T08:14:08.279 INFO:teuthology.orchestra.run.smithi073.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy InRelease 2024-03-29T08:14:08.281 INFO:teuthology.orchestra.run.smithi046.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy InRelease 2024-03-29T08:14:08.361 INFO:teuthology.orchestra.run.smithi073.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-29T08:14:08.363 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-29T08:14:08.444 INFO:teuthology.orchestra.run.smithi073.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-29T08:14:08.446 INFO:teuthology.orchestra.run.smithi046.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-29T08:14:08.527 INFO:teuthology.orchestra.run.smithi073.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2024-03-29T08:14:08.531 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,407 B] 2024-03-29T08:14:08.625 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2024-03-29T08:14:08.703 INFO:teuthology.orchestra.run.smithi073.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,407 B] 2024-03-29T08:14:08.811 INFO:teuthology.orchestra.run.smithi073.stdout:Fetched 28.5 kB in 1s (36.4 kB/s) 2024-03-29T08:14:08.822 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 28.5 kB in 1s (36.3 kB/s) 2024-03-29T08:14:09.769 INFO:teuthology.orchestra.run.smithi073.stdout:Reading package lists... 2024-03-29T08:14:09.784 DEBUG:teuthology.orchestra.run.smithi073:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.2-1015-gd0e392d9-1jammy cephadm=18.2.2-1015-gd0e392d9-1jammy ceph-mds=18.2.2-1015-gd0e392d9-1jammy ceph-mgr=18.2.2-1015-gd0e392d9-1jammy ceph-common=18.2.2-1015-gd0e392d9-1jammy ceph-fuse=18.2.2-1015-gd0e392d9-1jammy ceph-test=18.2.2-1015-gd0e392d9-1jammy ceph-volume=18.2.2-1015-gd0e392d9-1jammy radosgw=18.2.2-1015-gd0e392d9-1jammy python3-rados=18.2.2-1015-gd0e392d9-1jammy python3-rgw=18.2.2-1015-gd0e392d9-1jammy python3-cephfs=18.2.2-1015-gd0e392d9-1jammy python3-rbd=18.2.2-1015-gd0e392d9-1jammy libcephfs2=18.2.2-1015-gd0e392d9-1jammy libcephfs-dev=18.2.2-1015-gd0e392d9-1jammy librados2=18.2.2-1015-gd0e392d9-1jammy librbd1=18.2.2-1015-gd0e392d9-1jammy rbd-fuse=18.2.2-1015-gd0e392d9-1jammy python3-cephfs=18.2.2-1015-gd0e392d9-1jammy cephfs-shell=18.2.2-1015-gd0e392d9-1jammy cephfs-top=18.2.2-1015-gd0e392d9-1jammy cephfs-mirror=18.2.2-1015-gd0e392d9-1jammy 2024-03-29T08:14:09.820 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-03-29T08:14:09.835 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.2-1015-gd0e392d9-1jammy cephadm=18.2.2-1015-gd0e392d9-1jammy ceph-mds=18.2.2-1015-gd0e392d9-1jammy ceph-mgr=18.2.2-1015-gd0e392d9-1jammy ceph-common=18.2.2-1015-gd0e392d9-1jammy ceph-fuse=18.2.2-1015-gd0e392d9-1jammy ceph-test=18.2.2-1015-gd0e392d9-1jammy ceph-volume=18.2.2-1015-gd0e392d9-1jammy radosgw=18.2.2-1015-gd0e392d9-1jammy python3-rados=18.2.2-1015-gd0e392d9-1jammy python3-rgw=18.2.2-1015-gd0e392d9-1jammy python3-cephfs=18.2.2-1015-gd0e392d9-1jammy python3-rbd=18.2.2-1015-gd0e392d9-1jammy libcephfs2=18.2.2-1015-gd0e392d9-1jammy libcephfs-dev=18.2.2-1015-gd0e392d9-1jammy librados2=18.2.2-1015-gd0e392d9-1jammy librbd1=18.2.2-1015-gd0e392d9-1jammy rbd-fuse=18.2.2-1015-gd0e392d9-1jammy python3-cephfs=18.2.2-1015-gd0e392d9-1jammy cephfs-shell=18.2.2-1015-gd0e392d9-1jammy cephfs-top=18.2.2-1015-gd0e392d9-1jammy cephfs-mirror=18.2.2-1015-gd0e392d9-1jammy 2024-03-29T08:14:09.858 INFO:teuthology.orchestra.run.smithi073.stdout:Reading package lists... 2024-03-29T08:14:09.914 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-03-29T08:14:10.037 INFO:teuthology.orchestra.run.smithi073.stdout:Building dependency tree... 2024-03-29T08:14:10.038 INFO:teuthology.orchestra.run.smithi073.stdout:Reading state information... 2024-03-29T08:14:10.086 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2024-03-29T08:14:10.087 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2024-03-29T08:14:10.218 INFO:teuthology.orchestra.run.smithi073.stdout:The following packages were automatically installed and are no longer required: 2024-03-29T08:14:10.218 INFO:teuthology.orchestra.run.smithi073.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-29T08:14:10.218 INFO:teuthology.orchestra.run.smithi073.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-29T08:14:10.219 INFO:teuthology.orchestra.run.smithi073.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-29T08:14:10.219 INFO:teuthology.orchestra.run.smithi073.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-29T08:14:10.219 INFO:teuthology.orchestra.run.smithi073.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-29T08:14:10.220 INFO:teuthology.orchestra.run.smithi073.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-29T08:14:10.220 INFO:teuthology.orchestra.run.smithi073.stdout:The following additional packages will be installed: 2024-03-29T08:14:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-29T08:14:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-29T08:14:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-03-29T08:14:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-29T08:14:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-03-29T08:14:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-29T08:14:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-29T08:14:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-29T08:14:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-29T08:14:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-29T08:14:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-29T08:14:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-29T08:14:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-29T08:14:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-29T08:14:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-29T08:14:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-29T08:14:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-29T08:14:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-29T08:14:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-29T08:14:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-29T08:14:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-29T08:14:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-29T08:14:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: qttranslations5-l10n socat xmlstarlet 2024-03-29T08:14:10.223 INFO:teuthology.orchestra.run.smithi073.stdout:Suggested packages: 2024-03-29T08:14:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-03-29T08:14:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-03-29T08:14:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-03-29T08:14:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-03-29T08:14:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-03-29T08:14:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-03-29T08:14:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: python3-watchdog 2024-03-29T08:14:10.224 INFO:teuthology.orchestra.run.smithi073.stdout:Recommended packages: 2024-03-29T08:14:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: btrfs-tools 2024-03-29T08:14:10.265 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2024-03-29T08:14:10.265 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-29T08:14:10.265 INFO:teuthology.orchestra.run.smithi046.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-29T08:14:10.265 INFO:teuthology.orchestra.run.smithi046.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-29T08:14:10.265 INFO:teuthology.orchestra.run.smithi046.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-29T08:14:10.265 INFO:teuthology.orchestra.run.smithi046.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-29T08:14:10.265 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-29T08:14:10.267 INFO:teuthology.orchestra.run.smithi046.stdout:The following additional packages will be installed: 2024-03-29T08:14:10.267 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-29T08:14:10.267 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-29T08:14:10.267 INFO:teuthology.orchestra.run.smithi046.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-03-29T08:14:10.267 INFO:teuthology.orchestra.run.smithi046.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-29T08:14:10.267 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-03-29T08:14:10.267 INFO:teuthology.orchestra.run.smithi046.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-29T08:14:10.267 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-29T08:14:10.267 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-29T08:14:10.267 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-29T08:14:10.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-29T08:14:10.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-29T08:14:10.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-29T08:14:10.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-29T08:14:10.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-29T08:14:10.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-29T08:14:10.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-29T08:14:10.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-29T08:14:10.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-29T08:14:10.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-29T08:14:10.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-29T08:14:10.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-29T08:14:10.268 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-29T08:14:10.268 INFO:teuthology.orchestra.run.smithi046.stdout: qttranslations5-l10n socat xmlstarlet 2024-03-29T08:14:10.269 INFO:teuthology.orchestra.run.smithi046.stdout:Suggested packages: 2024-03-29T08:14:10.269 INFO:teuthology.orchestra.run.smithi046.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-03-29T08:14:10.269 INFO:teuthology.orchestra.run.smithi046.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-03-29T08:14:10.269 INFO:teuthology.orchestra.run.smithi046.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-03-29T08:14:10.269 INFO:teuthology.orchestra.run.smithi046.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-03-29T08:14:10.269 INFO:teuthology.orchestra.run.smithi046.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-03-29T08:14:10.269 INFO:teuthology.orchestra.run.smithi046.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-03-29T08:14:10.269 INFO:teuthology.orchestra.run.smithi046.stdout: python3-watchdog 2024-03-29T08:14:10.269 INFO:teuthology.orchestra.run.smithi046.stdout:Recommended packages: 2024-03-29T08:14:10.269 INFO:teuthology.orchestra.run.smithi046.stdout: btrfs-tools 2024-03-29T08:14:10.368 INFO:teuthology.orchestra.run.smithi073.stdout:The following NEW packages will be installed: 2024-03-29T08:14:10.368 INFO:teuthology.orchestra.run.smithi073.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-29T08:14:10.368 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-29T08:14:10.368 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-29T08:14:10.368 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-29T08:14:10.369 INFO:teuthology.orchestra.run.smithi073.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-03-29T08:14:10.369 INFO:teuthology.orchestra.run.smithi073.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-29T08:14:10.369 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-03-29T08:14:10.370 INFO:teuthology.orchestra.run.smithi073.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-03-29T08:14:10.370 INFO:teuthology.orchestra.run.smithi073.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-29T08:14:10.370 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-29T08:14:10.370 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-29T08:14:10.370 INFO:teuthology.orchestra.run.smithi073.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-29T08:14:10.370 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-29T08:14:10.371 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-29T08:14:10.371 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-29T08:14:10.371 INFO:teuthology.orchestra.run.smithi073.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-29T08:14:10.371 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-29T08:14:10.371 INFO:teuthology.orchestra.run.smithi073.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-29T08:14:10.371 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-29T08:14:10.371 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-29T08:14:10.371 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-29T08:14:10.371 INFO:teuthology.orchestra.run.smithi073.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-29T08:14:10.371 INFO:teuthology.orchestra.run.smithi073.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-29T08:14:10.371 INFO:teuthology.orchestra.run.smithi073.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-29T08:14:10.371 INFO:teuthology.orchestra.run.smithi073.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-29T08:14:10.371 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-29T08:14:10.371 INFO:teuthology.orchestra.run.smithi073.stdout: socat xmlstarlet 2024-03-29T08:14:10.422 INFO:teuthology.orchestra.run.smithi046.stdout:The following NEW packages will be installed: 2024-03-29T08:14:10.422 INFO:teuthology.orchestra.run.smithi046.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-29T08:14:10.422 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-29T08:14:10.422 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-29T08:14:10.422 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-29T08:14:10.422 INFO:teuthology.orchestra.run.smithi046.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-03-29T08:14:10.422 INFO:teuthology.orchestra.run.smithi046.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-29T08:14:10.423 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-03-29T08:14:10.424 INFO:teuthology.orchestra.run.smithi046.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-03-29T08:14:10.424 INFO:teuthology.orchestra.run.smithi046.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-29T08:14:10.424 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-29T08:14:10.424 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-29T08:14:10.424 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-29T08:14:10.424 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-29T08:14:10.424 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-29T08:14:10.424 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-29T08:14:10.424 INFO:teuthology.orchestra.run.smithi046.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-29T08:14:10.424 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-29T08:14:10.425 INFO:teuthology.orchestra.run.smithi046.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-29T08:14:10.425 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-29T08:14:10.425 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-29T08:14:10.425 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-29T08:14:10.425 INFO:teuthology.orchestra.run.smithi046.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-29T08:14:10.425 INFO:teuthology.orchestra.run.smithi046.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-29T08:14:10.425 INFO:teuthology.orchestra.run.smithi046.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-29T08:14:10.425 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-29T08:14:10.425 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-29T08:14:10.425 INFO:teuthology.orchestra.run.smithi046.stdout: socat xmlstarlet 2024-03-29T08:14:10.440 INFO:teuthology.orchestra.run.smithi073.stdout:0 upgraded, 115 newly installed, 0 to remove and 301 not upgraded. 2024-03-29T08:14:10.440 INFO:teuthology.orchestra.run.smithi073.stdout:Need to get 162 MB of archives. 2024-03-29T08:14:10.440 INFO:teuthology.orchestra.run.smithi073.stdout:After this operation, 664 MB of additional disk space will be used. 2024-03-29T08:14:10.441 INFO:teuthology.orchestra.run.smithi073.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-29T08:14:10.461 INFO:teuthology.orchestra.run.smithi073.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-29T08:14:10.472 INFO:teuthology.orchestra.run.smithi073.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-29T08:14:10.520 INFO:teuthology.orchestra.run.smithi073.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-03-29T08:14:10.530 INFO:teuthology.orchestra.run.smithi073.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-29T08:14:10.549 INFO:teuthology.orchestra.run.smithi073.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-29T08:14:10.551 INFO:teuthology.orchestra.run.smithi073.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-29T08:14:10.563 INFO:teuthology.orchestra.run.smithi073.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-29T08:14:10.606 INFO:teuthology.orchestra.run.smithi073.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-29T08:14:10.608 INFO:teuthology.orchestra.run.smithi073.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-29T08:14:10.616 INFO:teuthology.orchestra.run.smithi073.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-29T08:14:10.619 INFO:teuthology.orchestra.run.smithi073.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-29T08:14:10.620 INFO:teuthology.orchestra.run.smithi073.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-29T08:14:10.620 INFO:teuthology.orchestra.run.smithi046.stdout:0 upgraded, 115 newly installed, 0 to remove and 301 not upgraded. 2024-03-29T08:14:10.621 INFO:teuthology.orchestra.run.smithi046.stdout:Need to get 162 MB of archives. 2024-03-29T08:14:10.621 INFO:teuthology.orchestra.run.smithi046.stdout:After this operation, 664 MB of additional disk space will be used. 2024-03-29T08:14:10.621 INFO:teuthology.orchestra.run.smithi046.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-29T08:14:10.626 INFO:teuthology.orchestra.run.smithi073.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-29T08:14:10.627 INFO:teuthology.orchestra.run.smithi073.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-29T08:14:10.627 INFO:teuthology.orchestra.run.smithi073.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-29T08:14:10.628 INFO:teuthology.orchestra.run.smithi073.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-29T08:14:10.628 INFO:teuthology.orchestra.run.smithi073.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-29T08:14:10.647 INFO:teuthology.orchestra.run.smithi073.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-29T08:14:10.648 INFO:teuthology.orchestra.run.smithi073.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-29T08:14:10.648 INFO:teuthology.orchestra.run.smithi073.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-29T08:14:10.649 INFO:teuthology.orchestra.run.smithi073.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-29T08:14:10.649 INFO:teuthology.orchestra.run.smithi073.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-29T08:14:10.667 INFO:teuthology.orchestra.run.smithi073.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-29T08:14:10.668 INFO:teuthology.orchestra.run.smithi073.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-29T08:14:10.669 INFO:teuthology.orchestra.run.smithi073.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-29T08:14:10.669 INFO:teuthology.orchestra.run.smithi073.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-29T08:14:10.670 INFO:teuthology.orchestra.run.smithi073.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-29T08:14:10.687 INFO:teuthology.orchestra.run.smithi073.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-29T08:14:10.687 INFO:teuthology.orchestra.run.smithi073.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-29T08:14:10.688 INFO:teuthology.orchestra.run.smithi073.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-29T08:14:10.689 INFO:teuthology.orchestra.run.smithi073.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-29T08:14:10.690 INFO:teuthology.orchestra.run.smithi073.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-29T08:14:10.702 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-29T08:14:10.706 INFO:teuthology.orchestra.run.smithi073.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-29T08:14:10.707 INFO:teuthology.orchestra.run.smithi073.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-29T08:14:10.707 INFO:teuthology.orchestra.run.smithi073.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-29T08:14:10.708 INFO:teuthology.orchestra.run.smithi073.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-29T08:14:10.709 INFO:teuthology.orchestra.run.smithi073.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-29T08:14:10.731 INFO:teuthology.orchestra.run.smithi073.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-29T08:14:10.731 INFO:teuthology.orchestra.run.smithi073.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-29T08:14:10.731 INFO:teuthology.orchestra.run.smithi073.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-29T08:14:10.731 INFO:teuthology.orchestra.run.smithi073.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-29T08:14:10.731 INFO:teuthology.orchestra.run.smithi073.stdout:Get:43 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.2-1015-gd0e392d9-1jammy [3,419 kB] 2024-03-29T08:14:10.731 INFO:teuthology.orchestra.run.smithi073.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-29T08:14:10.746 INFO:teuthology.orchestra.run.smithi073.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-29T08:14:10.748 INFO:teuthology.orchestra.run.smithi073.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-29T08:14:10.748 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-29T08:14:10.798 INFO:teuthology.orchestra.run.smithi073.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-29T08:14:10.799 INFO:teuthology.orchestra.run.smithi073.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-29T08:14:10.800 INFO:teuthology.orchestra.run.smithi073.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-29T08:14:10.801 INFO:teuthology.orchestra.run.smithi073.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-29T08:14:10.801 INFO:teuthology.orchestra.run.smithi073.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-29T08:14:10.803 INFO:teuthology.orchestra.run.smithi073.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-29T08:14:10.803 INFO:teuthology.orchestra.run.smithi073.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-29T08:14:10.804 INFO:teuthology.orchestra.run.smithi073.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-29T08:14:10.805 INFO:teuthology.orchestra.run.smithi073.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-29T08:14:10.822 INFO:teuthology.orchestra.run.smithi073.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-29T08:14:10.823 INFO:teuthology.orchestra.run.smithi073.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-29T08:14:10.844 INFO:teuthology.orchestra.run.smithi073.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-29T08:14:10.846 INFO:teuthology.orchestra.run.smithi073.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-29T08:14:10.847 INFO:teuthology.orchestra.run.smithi073.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-29T08:14:10.865 INFO:teuthology.orchestra.run.smithi073.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-29T08:14:10.865 INFO:teuthology.orchestra.run.smithi073.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-29T08:14:10.866 INFO:teuthology.orchestra.run.smithi073.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-29T08:14:10.866 INFO:teuthology.orchestra.run.smithi073.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-29T08:14:10.867 INFO:teuthology.orchestra.run.smithi073.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-29T08:14:10.868 INFO:teuthology.orchestra.run.smithi073.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-29T08:14:10.887 INFO:teuthology.orchestra.run.smithi073.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-29T08:14:10.887 INFO:teuthology.orchestra.run.smithi073.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-29T08:14:10.888 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.2-1015-gd0e392d9-1jammy [3,419 kB] 2024-03-29T08:14:10.892 INFO:teuthology.orchestra.run.smithi073.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-29T08:14:10.894 INFO:teuthology.orchestra.run.smithi073.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-29T08:14:10.895 INFO:teuthology.orchestra.run.smithi073.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-29T08:14:10.896 INFO:teuthology.orchestra.run.smithi073.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-29T08:14:10.907 INFO:teuthology.orchestra.run.smithi073.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-29T08:14:10.910 INFO:teuthology.orchestra.run.smithi073.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-29T08:14:10.910 INFO:teuthology.orchestra.run.smithi073.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-29T08:14:10.915 INFO:teuthology.orchestra.run.smithi073.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-29T08:14:10.927 INFO:teuthology.orchestra.run.smithi073.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-29T08:14:10.928 INFO:teuthology.orchestra.run.smithi073.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-29T08:14:10.929 INFO:teuthology.orchestra.run.smithi073.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-29T08:14:10.937 INFO:teuthology.orchestra.run.smithi073.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-29T08:14:10.937 INFO:teuthology.orchestra.run.smithi073.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-29T08:14:10.938 INFO:teuthology.orchestra.run.smithi073.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-29T08:14:10.954 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-03-29T08:14:10.998 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-29T08:14:11.074 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-29T08:14:11.085 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-29T08:14:11.132 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-29T08:14:11.299 INFO:teuthology.orchestra.run.smithi046.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-29T08:14:11.310 INFO:teuthology.orchestra.run.smithi046.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-29T08:14:11.337 INFO:teuthology.orchestra.run.smithi046.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-29T08:14:11.509 INFO:teuthology.orchestra.run.smithi046.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-29T08:14:11.716 INFO:teuthology.orchestra.run.smithi046.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-29T08:14:12.115 INFO:teuthology.orchestra.run.smithi046.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-29T08:14:12.122 INFO:teuthology.orchestra.run.smithi046.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-29T08:14:12.123 INFO:teuthology.orchestra.run.smithi046.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-29T08:14:12.134 INFO:teuthology.orchestra.run.smithi046.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-29T08:14:12.135 INFO:teuthology.orchestra.run.smithi046.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-29T08:14:12.139 INFO:teuthology.orchestra.run.smithi046.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-29T08:14:12.140 INFO:teuthology.orchestra.run.smithi046.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-29T08:14:12.141 INFO:teuthology.orchestra.run.smithi046.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-29T08:14:12.141 INFO:teuthology.orchestra.run.smithi046.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-29T08:14:12.308 INFO:teuthology.orchestra.run.smithi046.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-29T08:14:12.512 INFO:teuthology.orchestra.run.smithi073.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.2-1015-gd0e392d9-1jammy [3,105 kB] 2024-03-29T08:14:12.753 INFO:teuthology.orchestra.run.smithi046.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-29T08:14:12.764 INFO:teuthology.orchestra.run.smithi046.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-29T08:14:12.770 INFO:teuthology.orchestra.run.smithi046.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-29T08:14:12.773 INFO:teuthology.orchestra.run.smithi046.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-29T08:14:12.792 INFO:teuthology.orchestra.run.smithi046.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-29T08:14:12.795 INFO:teuthology.orchestra.run.smithi046.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-29T08:14:12.798 INFO:teuthology.orchestra.run.smithi046.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-29T08:14:12.840 INFO:teuthology.orchestra.run.smithi046.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-29T08:14:12.853 INFO:teuthology.orchestra.run.smithi046.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-29T08:14:12.858 INFO:teuthology.orchestra.run.smithi046.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-29T08:14:13.113 INFO:teuthology.orchestra.run.smithi046.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-29T08:14:13.195 INFO:teuthology.orchestra.run.smithi046.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-29T08:14:13.288 INFO:teuthology.orchestra.run.smithi046.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-29T08:14:13.319 INFO:teuthology.orchestra.run.smithi046.stdout:Get:38 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.2-1015-gd0e392d9-1jammy [3,105 kB] 2024-03-29T08:14:13.365 INFO:teuthology.orchestra.run.smithi046.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-29T08:14:13.473 INFO:teuthology.orchestra.run.smithi046.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-29T08:14:13.562 INFO:teuthology.orchestra.run.smithi046.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-29T08:14:13.581 INFO:teuthology.orchestra.run.smithi046.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-29T08:14:13.611 INFO:teuthology.orchestra.run.smithi046.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-29T08:14:13.621 INFO:teuthology.orchestra.run.smithi073.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.2-1015-gd0e392d9-1jammy [674 kB] 2024-03-29T08:14:13.623 INFO:teuthology.orchestra.run.smithi046.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-29T08:14:13.662 INFO:teuthology.orchestra.run.smithi046.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-29T08:14:13.669 INFO:teuthology.orchestra.run.smithi046.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-29T08:14:13.863 INFO:teuthology.orchestra.run.smithi046.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-29T08:14:13.865 INFO:teuthology.orchestra.run.smithi073.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.2-1015-gd0e392d9-1jammy [356 kB] 2024-03-29T08:14:13.957 INFO:teuthology.orchestra.run.smithi073.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.2-1015-gd0e392d9-1jammy [32.8 kB] 2024-03-29T08:14:14.009 INFO:teuthology.orchestra.run.smithi073.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.2-1015-gd0e392d9-1jammy [180 kB] 2024-03-29T08:14:14.064 INFO:teuthology.orchestra.run.smithi073.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.2-1015-gd0e392d9-1jammy [63.7 kB] 2024-03-29T08:14:14.084 INFO:teuthology.orchestra.run.smithi073.stdout:Get:89 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.2-1015-gd0e392d9-1jammy [330 kB] 2024-03-29T08:14:14.201 INFO:teuthology.orchestra.run.smithi073.stdout:Get:90 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.2-1015-gd0e392d9-1jammy [5,892 kB] 2024-03-29T08:14:14.701 INFO:teuthology.orchestra.run.smithi046.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-29T08:14:14.702 INFO:teuthology.orchestra.run.smithi046.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-29T08:14:14.703 INFO:teuthology.orchestra.run.smithi046.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-29T08:14:14.704 INFO:teuthology.orchestra.run.smithi046.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-29T08:14:14.705 INFO:teuthology.orchestra.run.smithi046.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-29T08:14:14.706 INFO:teuthology.orchestra.run.smithi046.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-29T08:14:14.706 INFO:teuthology.orchestra.run.smithi046.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-29T08:14:14.707 INFO:teuthology.orchestra.run.smithi046.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-29T08:14:14.709 INFO:teuthology.orchestra.run.smithi046.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-29T08:14:14.782 INFO:teuthology.orchestra.run.smithi046.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-29T08:14:14.948 INFO:teuthology.orchestra.run.smithi046.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-29T08:14:14.982 INFO:teuthology.orchestra.run.smithi046.stdout:Get:59 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.2-1015-gd0e392d9-1jammy [674 kB] 2024-03-29T08:14:15.407 INFO:teuthology.orchestra.run.smithi046.stdout:Get:60 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.2-1015-gd0e392d9-1jammy [356 kB] 2024-03-29T08:14:15.586 INFO:teuthology.orchestra.run.smithi046.stdout:Get:61 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.2-1015-gd0e392d9-1jammy [32.8 kB] 2024-03-29T08:14:15.588 INFO:teuthology.orchestra.run.smithi046.stdout:Get:62 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.2-1015-gd0e392d9-1jammy [180 kB] 2024-03-29T08:14:15.678 INFO:teuthology.orchestra.run.smithi046.stdout:Get:63 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.2-1015-gd0e392d9-1jammy [63.7 kB] 2024-03-29T08:14:15.682 INFO:teuthology.orchestra.run.smithi046.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-29T08:14:15.688 INFO:teuthology.orchestra.run.smithi046.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-29T08:14:15.689 INFO:teuthology.orchestra.run.smithi046.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-29T08:14:15.741 INFO:teuthology.orchestra.run.smithi046.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-29T08:14:15.742 INFO:teuthology.orchestra.run.smithi046.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-29T08:14:15.759 INFO:teuthology.orchestra.run.smithi046.stdout:Get:69 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.2-1015-gd0e392d9-1jammy [330 kB] 2024-03-29T08:14:15.764 INFO:teuthology.orchestra.run.smithi046.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-29T08:14:15.765 INFO:teuthology.orchestra.run.smithi046.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-29T08:14:15.766 INFO:teuthology.orchestra.run.smithi046.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-29T08:14:15.768 INFO:teuthology.orchestra.run.smithi046.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-29T08:14:15.768 INFO:teuthology.orchestra.run.smithi046.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-29T08:14:15.933 INFO:teuthology.orchestra.run.smithi046.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-29T08:14:15.937 INFO:teuthology.orchestra.run.smithi046.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.2-1015-gd0e392d9-1jammy [5,892 kB] 2024-03-29T08:14:16.236 INFO:teuthology.orchestra.run.smithi073.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.2-1015-gd0e392d9-1jammy [112 kB] 2024-03-29T08:14:16.302 INFO:teuthology.orchestra.run.smithi073.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.2-1015-gd0e392d9-1jammy [441 kB] 2024-03-29T08:14:16.448 INFO:teuthology.orchestra.run.smithi046.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-29T08:14:16.452 INFO:teuthology.orchestra.run.smithi073.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.2-1015-gd0e392d9-1jammy [22.0 MB] 2024-03-29T08:14:16.477 INFO:teuthology.orchestra.run.smithi046.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-29T08:14:16.500 INFO:teuthology.orchestra.run.smithi046.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-29T08:14:16.508 INFO:teuthology.orchestra.run.smithi046.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-29T08:14:16.586 INFO:teuthology.orchestra.run.smithi046.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-29T08:14:16.619 INFO:teuthology.orchestra.run.smithi046.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-29T08:14:16.621 INFO:teuthology.orchestra.run.smithi046.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-29T08:14:16.656 INFO:teuthology.orchestra.run.smithi046.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-29T08:14:16.680 INFO:teuthology.orchestra.run.smithi046.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-29T08:14:16.683 INFO:teuthology.orchestra.run.smithi046.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-29T08:14:16.858 INFO:teuthology.orchestra.run.smithi046.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-29T08:14:17.464 INFO:teuthology.orchestra.run.smithi046.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-29T08:14:17.466 INFO:teuthology.orchestra.run.smithi046.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-29T08:14:17.470 INFO:teuthology.orchestra.run.smithi046.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-29T08:14:19.259 INFO:teuthology.orchestra.run.smithi046.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.2-1015-gd0e392d9-1jammy [112 kB] 2024-03-29T08:14:19.336 INFO:teuthology.orchestra.run.smithi046.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.2-1015-gd0e392d9-1jammy [441 kB] 2024-03-29T08:14:19.519 INFO:teuthology.orchestra.run.smithi046.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.2-1015-gd0e392d9-1jammy [22.0 MB] 2024-03-29T08:14:22.302 INFO:teuthology.orchestra.run.smithi073.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.2-1015-gd0e392d9-1jammy [5,001 kB] 2024-03-29T08:14:23.163 INFO:teuthology.orchestra.run.smithi073.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.2-1015-gd0e392d9-1jammy [241 kB] 2024-03-29T08:14:23.203 INFO:teuthology.orchestra.run.smithi073.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.2-1015-gd0e392d9-1jammy [130 kB] 2024-03-29T08:14:23.223 INFO:teuthology.orchestra.run.smithi073.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.2-1015-gd0e392d9-1jammy [1,485 kB] 2024-03-29T08:14:23.439 INFO:teuthology.orchestra.run.smithi073.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.2-1015-gd0e392d9-1jammy [6,078 kB] 2024-03-29T08:14:24.297 INFO:teuthology.orchestra.run.smithi073.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.2-1015-gd0e392d9-1jammy [22.3 MB] 2024-03-29T08:14:26.280 INFO:teuthology.orchestra.run.smithi073.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.2-1015-gd0e392d9-1jammy [14.1 kB] 2024-03-29T08:14:26.281 INFO:teuthology.orchestra.run.smithi073.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.2-1015-gd0e392d9-1jammy [826 kB] 2024-03-29T08:14:26.305 INFO:teuthology.orchestra.run.smithi073.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.2-1015-gd0e392d9-1jammy [2,175 kB] 2024-03-29T08:14:26.389 INFO:teuthology.orchestra.run.smithi073.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.2-1015-gd0e392d9-1jammy [238 kB] 2024-03-29T08:14:26.398 INFO:teuthology.orchestra.run.smithi073.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.2-1015-gd0e392d9-1jammy [144 kB] 2024-03-29T08:14:26.403 INFO:teuthology.orchestra.run.smithi073.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.2-1015-gd0e392d9-1jammy [1,931 kB] 2024-03-29T08:14:26.481 INFO:teuthology.orchestra.run.smithi073.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.2-1015-gd0e392d9-1jammy [8,625 kB] 2024-03-29T08:14:26.819 INFO:teuthology.orchestra.run.smithi073.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.2-1015-gd0e392d9-1jammy [14.3 kB] 2024-03-29T08:14:26.819 INFO:teuthology.orchestra.run.smithi073.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.2-1015-gd0e392d9-1jammy [42.7 MB] 2024-03-29T08:14:28.535 INFO:teuthology.orchestra.run.smithi073.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.2-1015-gd0e392d9-1jammy [121 kB] 2024-03-29T08:14:28.536 INFO:teuthology.orchestra.run.smithi073.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.2.2-1015-gd0e392d9-1jammy [231 kB] 2024-03-29T08:14:28.538 INFO:teuthology.orchestra.run.smithi073.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.2.2-1015-gd0e392d9-1jammy [34.1 kB] 2024-03-29T08:14:28.538 INFO:teuthology.orchestra.run.smithi073.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.2.2-1015-gd0e392d9-1jammy [23.5 kB] 2024-03-29T08:14:28.539 INFO:teuthology.orchestra.run.smithi073.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.2-1015-gd0e392d9-1jammy [38.3 kB] 2024-03-29T08:14:28.540 INFO:teuthology.orchestra.run.smithi073.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.2-1015-gd0e392d9-1jammy [11.6 MB] 2024-03-29T08:14:28.999 INFO:teuthology.orchestra.run.smithi073.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.2-1015-gd0e392d9-1jammy [93.1 kB] 2024-03-29T08:14:29.566 INFO:teuthology.orchestra.run.smithi073.stdout:Fetched 162 MB in 19s (8,722 kB/s) 2024-03-29T08:14:29.718 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-29T08:14:29.782 INFO:teuthology.orchestra.run.smithi073.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132923 files and directories currently installed.) 2024-03-29T08:14:29.786 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-29T08:14:29.825 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-29T08:14:30.136 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-29T08:14:30.149 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-29T08:14:30.186 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-29T08:14:30.506 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-cmd2. 2024-03-29T08:14:30.519 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-29T08:14:30.564 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-29T08:14:30.925 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libfmt8:amd64. 2024-03-29T08:14:30.939 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-03-29T08:14:31.026 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-03-29T08:14:31.354 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-29T08:14:31.368 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-29T08:14:31.437 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-29T08:14:31.749 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-29T08:14:31.764 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-29T08:14:31.775 INFO:teuthology.orchestra.run.smithi046.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.2-1015-gd0e392d9-1jammy [5,001 kB] 2024-03-29T08:14:31.799 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-29T08:14:32.144 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-29T08:14:32.159 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-29T08:14:32.202 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-29T08:14:32.639 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-29T08:14:32.654 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-29T08:14:32.715 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T08:14:32.835 INFO:teuthology.orchestra.run.smithi046.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.2-1015-gd0e392d9-1jammy [241 kB] 2024-03-29T08:14:32.889 INFO:teuthology.orchestra.run.smithi046.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.2-1015-gd0e392d9-1jammy [130 kB] 2024-03-29T08:14:32.907 INFO:teuthology.orchestra.run.smithi046.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.2-1015-gd0e392d9-1jammy [1,485 kB] 2024-03-29T08:14:33.126 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-29T08:14:33.141 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-29T08:14:33.176 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T08:14:33.188 INFO:teuthology.orchestra.run.smithi046.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.2-1015-gd0e392d9-1jammy [6,078 kB] 2024-03-29T08:14:33.487 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-29T08:14:33.502 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-29T08:14:33.545 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T08:14:33.916 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-29T08:14:33.930 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-29T08:14:33.974 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-29T08:14:34.127 INFO:teuthology.orchestra.run.smithi046.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.2-1015-gd0e392d9-1jammy [22.3 MB] 2024-03-29T08:14:34.277 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package librados2. 2024-03-29T08:14:34.291 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../011-librados2_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:34.327 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking librados2 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:34.879 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package librbd1. 2024-03-29T08:14:34.894 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../012-librbd1_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:34.938 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking librbd1 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:35.382 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libcephfs2. 2024-03-29T08:14:35.396 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../013-libcephfs2_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:35.441 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libcephfs2 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:35.719 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-rados. 2024-03-29T08:14:35.732 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../014-python3-rados_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:35.777 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-rados (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:36.088 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-29T08:14:36.102 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:14:36.138 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-ceph-argparse (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:36.348 INFO:teuthology.orchestra.run.smithi046.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.2-1015-gd0e392d9-1jammy [14.1 kB] 2024-03-29T08:14:36.348 INFO:teuthology.orchestra.run.smithi046.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.2-1015-gd0e392d9-1jammy [826 kB] 2024-03-29T08:14:36.382 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-cephfs. 2024-03-29T08:14:36.397 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../016-python3-cephfs_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:36.409 INFO:teuthology.orchestra.run.smithi046.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.2-1015-gd0e392d9-1jammy [2,175 kB] 2024-03-29T08:14:36.440 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-cephfs (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:36.557 INFO:teuthology.orchestra.run.smithi046.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.2-1015-gd0e392d9-1jammy [238 kB] 2024-03-29T08:14:36.576 INFO:teuthology.orchestra.run.smithi046.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.2-1015-gd0e392d9-1jammy [144 kB] 2024-03-29T08:14:36.581 INFO:teuthology.orchestra.run.smithi046.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.2-1015-gd0e392d9-1jammy [1,931 kB] 2024-03-29T08:14:36.705 INFO:teuthology.orchestra.run.smithi046.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.2-1015-gd0e392d9-1jammy [8,625 kB] 2024-03-29T08:14:36.785 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-29T08:14:36.800 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../017-python3-ceph-common_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:14:36.843 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-ceph-common (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:37.138 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-prettytable. 2024-03-29T08:14:37.153 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-03-29T08:14:37.196 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-29T08:14:37.351 INFO:teuthology.orchestra.run.smithi046.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.2-1015-gd0e392d9-1jammy [14.3 kB] 2024-03-29T08:14:37.351 INFO:teuthology.orchestra.run.smithi046.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.2-1015-gd0e392d9-1jammy [42.7 MB] 2024-03-29T08:14:37.447 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-rbd. 2024-03-29T08:14:37.461 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../019-python3-rbd_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:37.499 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-rbd (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:37.827 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-29T08:14:37.842 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-29T08:14:37.885 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-29T08:14:38.238 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package librgw2. 2024-03-29T08:14:38.253 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../021-librgw2_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:38.296 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking librgw2 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:38.724 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-rgw. 2024-03-29T08:14:38.738 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../022-python3-rgw_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:38.774 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-rgw (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:39.127 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-29T08:14:39.142 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-29T08:14:39.185 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-29T08:14:39.496 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libradosstriper1. 2024-03-29T08:14:39.511 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../024-libradosstriper1_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:39.555 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libradosstriper1 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:40.000 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-common. 2024-03-29T08:14:40.015 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../025-ceph-common_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:40.050 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-common (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:40.099 INFO:teuthology.orchestra.run.smithi046.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.2-1015-gd0e392d9-1jammy [121 kB] 2024-03-29T08:14:40.100 INFO:teuthology.orchestra.run.smithi046.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.2.2-1015-gd0e392d9-1jammy [231 kB] 2024-03-29T08:14:40.102 INFO:teuthology.orchestra.run.smithi046.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.2.2-1015-gd0e392d9-1jammy [34.1 kB] 2024-03-29T08:14:40.103 INFO:teuthology.orchestra.run.smithi046.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.2.2-1015-gd0e392d9-1jammy [23.5 kB] 2024-03-29T08:14:40.103 INFO:teuthology.orchestra.run.smithi046.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.2-1015-gd0e392d9-1jammy [38.3 kB] 2024-03-29T08:14:40.104 INFO:teuthology.orchestra.run.smithi046.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.2-1015-gd0e392d9-1jammy [11.6 MB] 2024-03-29T08:14:40.811 INFO:teuthology.orchestra.run.smithi046.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240326.105515-reef/d0e392d9aee635f2b65768c460c7a151ead7a5e2/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.2-1015-gd0e392d9-1jammy [93.1 kB] 2024-03-29T08:14:41.147 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-base. 2024-03-29T08:14:41.162 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../026-ceph-base_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:41.215 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-base (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:41.397 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 162 MB in 30s (5,346 kB/s) 2024-03-29T08:14:41.536 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-29T08:14:41.604 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132923 files and directories currently installed.) 2024-03-29T08:14:41.607 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-29T08:14:41.652 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-29T08:14:41.794 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-29T08:14:41.807 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-29T08:14:41.852 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-29T08:14:41.904 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-29T08:14:41.918 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-29T08:14:41.954 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-29T08:14:42.155 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-cheroot. 2024-03-29T08:14:42.169 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-29T08:14:42.213 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-29T08:14:42.259 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cmd2. 2024-03-29T08:14:42.274 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-29T08:14:42.342 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-29T08:14:42.566 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-29T08:14:42.580 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-29T08:14:42.625 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-29T08:14:42.670 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libfmt8:amd64. 2024-03-29T08:14:42.686 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-03-29T08:14:42.771 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-03-29T08:14:43.530 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-29T08:14:43.546 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-29T08:14:43.586 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-29T08:14:43.824 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-29T08:14:43.841 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-29T08:14:43.883 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-29T08:14:43.892 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-29T08:14:43.904 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-29T08:14:43.947 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-29T08:14:44.219 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-29T08:14:44.235 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-29T08:14:44.258 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-tempora. 2024-03-29T08:14:44.273 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2024-03-29T08:14:44.277 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-29T08:14:44.317 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-29T08:14:44.562 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-portend. 2024-03-29T08:14:44.577 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2024-03-29T08:14:44.611 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-29T08:14:44.613 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-29T08:14:44.629 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-29T08:14:44.672 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-29T08:14:45.059 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-29T08:14:45.074 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-29T08:14:45.115 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-29T08:14:45.130 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-29T08:14:45.142 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T08:14:45.173 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-29T08:14:45.476 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-29T08:14:45.491 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-29T08:14:45.534 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-29T08:14:45.545 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-29T08:14:45.561 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-29T08:14:45.604 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T08:14:45.905 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-natsort. 2024-03-29T08:14:45.921 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2024-03-29T08:14:45.948 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-29T08:14:45.963 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-29T08:14:45.964 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-29T08:14:45.998 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T08:14:46.266 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-logutils. 2024-03-29T08:14:46.281 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2024-03-29T08:14:46.324 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-29T08:14:46.652 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-29T08:14:46.661 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-29T08:14:46.667 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-29T08:14:46.675 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-29T08:14:46.710 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-29T08:14:46.710 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-29T08:14:46.972 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-mako. 2024-03-29T08:14:46.987 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-29T08:14:47.021 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librados2. 2024-03-29T08:14:47.030 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-29T08:14:47.036 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../011-librados2_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:47.079 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librados2 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:47.392 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-29T08:14:47.407 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-29T08:14:47.450 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-29T08:14:47.499 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librbd1. 2024-03-29T08:14:47.514 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../012-librbd1_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:47.557 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librbd1 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:47.753 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-29T08:14:47.768 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-29T08:14:47.811 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-29T08:14:48.002 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libcephfs2. 2024-03-29T08:14:48.017 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../013-libcephfs2_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:48.060 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libcephfs2 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:48.106 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-webob. 2024-03-29T08:14:48.120 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-29T08:14:48.156 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-29T08:14:48.296 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rados. 2024-03-29T08:14:48.311 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../014-python3-rados_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:48.345 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rados (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:48.476 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-waitress. 2024-03-29T08:14:48.490 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-29T08:14:48.528 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-29T08:14:48.657 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-29T08:14:48.672 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:14:48.715 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-ceph-argparse (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:48.862 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-tempita. 2024-03-29T08:14:48.877 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-29T08:14:48.920 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-29T08:14:48.976 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cephfs. 2024-03-29T08:14:48.991 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../016-python3-cephfs_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:49.035 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cephfs (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:49.206 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-paste. 2024-03-29T08:14:49.221 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-29T08:14:49.264 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-29T08:14:49.379 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-29T08:14:49.394 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../017-python3-ceph-common_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:14:49.437 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-ceph-common (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:49.610 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-29T08:14:49.624 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-29T08:14:49.660 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-29T08:14:49.740 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-prettytable. 2024-03-29T08:14:49.755 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-03-29T08:14:49.790 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-29T08:14:49.937 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-29T08:14:49.952 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-29T08:14:49.996 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-29T08:14:50.051 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rbd. 2024-03-29T08:14:50.065 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../019-python3-rbd_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:50.109 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rbd (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:50.332 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-webtest. 2024-03-29T08:14:50.346 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2024-03-29T08:14:50.391 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-29T08:14:50.462 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-29T08:14:50.477 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-29T08:14:50.520 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-29T08:14:50.694 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pecan. 2024-03-29T08:14:50.707 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-29T08:14:50.752 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-29T08:14:50.873 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librgw2. 2024-03-29T08:14:50.888 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../021-librgw2_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:50.931 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librgw2 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:51.090 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-certifi. 2024-03-29T08:14:51.103 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2024-03-29T08:14:51.140 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-29T08:14:51.367 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rgw. 2024-03-29T08:14:51.382 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../022-python3-rgw_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:51.417 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rgw (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:51.452 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-idna. 2024-03-29T08:14:51.465 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2024-03-29T08:14:51.510 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-29T08:14:51.745 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-29T08:14:51.760 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-29T08:14:51.803 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-29T08:14:51.846 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-urllib3. 2024-03-29T08:14:51.860 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-29T08:14:51.904 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-29T08:14:52.106 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libradosstriper1. 2024-03-29T08:14:52.122 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../024-libradosstriper1_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:52.181 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libradosstriper1 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:52.216 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-requests. 2024-03-29T08:14:52.230 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-29T08:14:52.274 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-29T08:14:52.535 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-common. 2024-03-29T08:14:52.550 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../025-ceph-common_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:52.585 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-common (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:52.611 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-29T08:14:52.625 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-29T08:14:52.660 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-29T08:14:52.947 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-29T08:14:52.962 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:14:53.005 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr-modules-core (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:53.419 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-29T08:14:53.429 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:53.469 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libsqlite3-mod-ceph (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:53.707 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-base. 2024-03-29T08:14:53.723 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../026-ceph-base_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:53.775 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-base (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:53.780 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-openssl. 2024-03-29T08:14:53.791 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2024-03-29T08:14:53.830 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-29T08:14:54.208 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr. 2024-03-29T08:14:54.219 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../058-ceph-mgr_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:54.258 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:54.311 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-29T08:14:54.326 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-29T08:14:54.369 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-29T08:14:54.614 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cheroot. 2024-03-29T08:14:54.619 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mon. 2024-03-29T08:14:54.628 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-29T08:14:54.632 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../059-ceph-mon_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:54.664 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-29T08:14:54.669 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mon (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:54.981 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-29T08:14:54.995 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-29T08:14:55.042 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-29T08:14:55.354 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-29T08:14:55.369 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-29T08:14:55.381 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-osd. 2024-03-29T08:14:55.392 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../060-ceph-osd_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:55.411 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-29T08:14:55.431 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-osd (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:55.714 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-29T08:14:55.730 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-29T08:14:55.772 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-29T08:14:56.034 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-tempora. 2024-03-29T08:14:56.049 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2024-03-29T08:14:56.083 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-29T08:14:56.202 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph. 2024-03-29T08:14:56.212 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../061-ceph_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:56.251 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:56.403 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-portend. 2024-03-29T08:14:56.419 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2024-03-29T08:14:56.461 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-29T08:14:56.580 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-fuse. 2024-03-29T08:14:56.590 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../062-ceph-fuse_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:56.629 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-fuse (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:56.739 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-29T08:14:56.758 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-29T08:14:56.923 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-29T08:14:56.974 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mds. 2024-03-29T08:14:56.985 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../063-ceph-mds_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:57.024 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mds (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:57.201 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-29T08:14:57.216 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-29T08:14:57.259 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-29T08:14:57.395 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package cephadm. 2024-03-29T08:14:57.405 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../064-cephadm_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:14:57.444 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking cephadm (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:14:57.612 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-natsort. 2024-03-29T08:14:57.627 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2024-03-29T08:14:57.670 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-29T08:14:57.755 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-29T08:14:57.766 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-29T08:14:57.830 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-29T08:14:58.099 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-logutils. 2024-03-29T08:14:58.114 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2024-03-29T08:14:58.158 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-29T08:14:58.165 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-29T08:14:58.169 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-29T08:14:58.200 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-29T08:14:58.510 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-29T08:14:58.526 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-29T08:14:58.568 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-29T08:14:58.913 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-mako. 2024-03-29T08:14:58.929 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-29T08:14:58.979 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-29T08:14:59.231 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-babel. 2024-03-29T08:14:59.240 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-29T08:14:59.297 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-29T08:14:59.350 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-29T08:14:59.365 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-29T08:14:59.408 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-29T08:14:59.642 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-jinja2. 2024-03-29T08:14:59.654 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-29T08:14:59.661 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-29T08:14:59.677 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-29T08:14:59.692 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-29T08:14:59.711 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-29T08:15:00.014 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webob. 2024-03-29T08:15:00.029 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-29T08:15:00.072 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-29T08:15:00.104 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-29T08:15:00.115 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:15:00.154 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr-cephadm (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:00.425 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-waitress. 2024-03-29T08:15:00.440 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-29T08:15:00.482 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-29T08:15:00.485 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-29T08:15:00.493 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-29T08:15:00.532 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-29T08:15:00.820 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-tempita. 2024-03-29T08:15:00.835 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-29T08:15:00.852 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pluggy. 2024-03-29T08:15:00.862 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../071-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-29T08:15:00.878 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-29T08:15:00.902 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-29T08:15:01.139 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-paste. 2024-03-29T08:15:01.155 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-29T08:15:01.180 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-py. 2024-03-29T08:15:01.189 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-29T08:15:01.191 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../072-python3-py_1.10.0-1_all.deb ... 2024-03-29T08:15:01.230 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-29T08:15:01.492 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-29T08:15:01.507 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-29T08:15:01.542 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-29T08:15:01.567 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-29T08:15:01.578 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../073-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-29T08:15:01.617 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-29T08:15:01.828 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-29T08:15:01.844 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-29T08:15:01.920 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-toml. 2024-03-29T08:15:01.931 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../074-python3-toml_0.10.2-1_all.deb ... 2024-03-29T08:15:01.970 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-29T08:15:02.045 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-29T08:15:02.316 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pytest. 2024-03-29T08:15:02.326 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../075-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-29T08:15:02.348 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webtest. 2024-03-29T08:15:02.363 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2024-03-29T08:15:02.364 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-29T08:15:02.406 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-29T08:15:02.702 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-29T08:15:02.713 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-03-29T08:15:02.717 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pecan. 2024-03-29T08:15:02.733 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-29T08:15:02.752 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-29T08:15:02.784 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-29T08:15:03.088 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-routes. 2024-03-29T08:15:03.100 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-29T08:15:03.138 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-29T08:15:03.139 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-certifi. 2024-03-29T08:15:03.154 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2024-03-29T08:15:03.189 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-29T08:15:03.458 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-29T08:15:03.472 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:15:03.516 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-idna. 2024-03-29T08:15:03.525 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr-dashboard (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:03.532 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2024-03-29T08:15:03.575 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-29T08:15:03.894 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-urllib3. 2024-03-29T08:15:03.919 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-29T08:15:03.937 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-29T08:15:03.949 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-29T08:15:03.987 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-29T08:15:04.078 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-29T08:15:04.398 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-requests. 2024-03-29T08:15:04.408 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-29T08:15:04.447 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-29T08:15:04.599 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-joblib. 2024-03-29T08:15:04.610 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-29T08:15:04.648 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-29T08:15:04.759 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-29T08:15:04.772 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-29T08:15:04.808 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-29T08:15:05.019 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-29T08:15:05.111 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-29T08:15:05.111 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-29T08:15:05.120 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-29T08:15:05.133 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:15:05.170 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-modules-core (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:05.388 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-sklearn. 2024-03-29T08:15:05.402 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-29T08:15:05.446 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-29T08:15:05.533 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-29T08:15:05.547 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:05.591 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libsqlite3-mod-ceph (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:05.903 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-openssl. 2024-03-29T08:15:05.914 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2024-03-29T08:15:05.952 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-29T08:15:06.331 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr. 2024-03-29T08:15:06.342 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../058-ceph-mgr_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:06.367 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-29T08:15:06.378 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:15:06.380 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:06.417 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:06.709 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mon. 2024-03-29T08:15:06.720 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../059-ceph-mon_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:06.750 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mon (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:07.320 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-osd. 2024-03-29T08:15:07.332 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../060-ceph-osd_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:07.370 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-osd (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:07.446 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-cachetools. 2024-03-29T08:15:07.458 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-03-29T08:15:07.503 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-29T08:15:07.865 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-29T08:15:07.877 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-29T08:15:07.914 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-29T08:15:08.141 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph. 2024-03-29T08:15:08.152 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../061-ceph_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:08.190 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:08.260 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-29T08:15:08.272 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-29T08:15:08.309 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-29T08:15:08.460 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-fuse. 2024-03-29T08:15:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../062-ceph-fuse_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:08.501 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-fuse (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:08.638 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-rsa. 2024-03-29T08:15:08.650 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-03-29T08:15:08.679 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-29T08:15:08.863 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mds. 2024-03-29T08:15:08.875 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../063-ceph-mds_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:08.913 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mds (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:08.966 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-google-auth. 2024-03-29T08:15:08.979 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-03-29T08:15:09.016 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-29T08:15:09.316 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephadm. 2024-03-29T08:15:09.328 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../064-cephadm_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:09.361 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-29T08:15:09.366 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephadm (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:09.373 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-29T08:15:09.411 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-29T08:15:09.644 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-29T08:15:09.656 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-29T08:15:09.694 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-29T08:15:09.722 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-websocket. 2024-03-29T08:15:09.734 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-03-29T08:15:09.772 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-29T08:15:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-29T08:15:10.042 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-29T08:15:10.080 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-29T08:15:10.108 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-29T08:15:10.121 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-29T08:15:10.158 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-29T08:15:10.855 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-29T08:15:10.867 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:15:10.905 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr-k8sevents (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:11.118 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-babel. 2024-03-29T08:15:11.129 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-29T08:15:11.158 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-29T08:15:11.168 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-29T08:15:11.169 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-29T08:15:11.199 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-29T08:15:11.463 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jinja2. 2024-03-29T08:15:11.473 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-29T08:15:11.513 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-29T08:15:11.561 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-29T08:15:11.575 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-29T08:15:11.627 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-29T08:15:11.833 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-29T08:15:11.844 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:15:11.874 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-cephadm (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:11.888 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package jq. 2024-03-29T08:15:11.901 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-29T08:15:11.938 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-29T08:15:12.175 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package socat. 2024-03-29T08:15:12.188 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-29T08:15:12.258 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-29T08:15:12.412 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-29T08:15:12.423 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-29T08:15:12.453 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-29T08:15:12.503 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package xmlstarlet. 2024-03-29T08:15:12.516 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-29T08:15:12.561 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-29T08:15:12.731 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pluggy. 2024-03-29T08:15:12.742 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../071-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-29T08:15:12.780 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-29T08:15:12.907 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-test. 2024-03-29T08:15:12.919 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../098-ceph-test_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:12.956 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-test (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:13.092 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-py. 2024-03-29T08:15:13.104 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../072-python3-py_1.10.0-1_all.deb ... 2024-03-29T08:15:13.142 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-29T08:15:13.496 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-29T08:15:13.510 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../073-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-29T08:15:13.545 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-29T08:15:13.823 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-toml. 2024-03-29T08:15:13.835 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../074-python3-toml_0.10.2-1_all.deb ... 2024-03-29T08:15:13.864 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-29T08:15:14.134 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pytest. 2024-03-29T08:15:14.151 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../075-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-29T08:15:14.201 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-29T08:15:14.452 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-volume. 2024-03-29T08:15:14.466 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../099-ceph-volume_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:15:14.513 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-29T08:15:14.520 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-03-29T08:15:14.535 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-volume (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:14.554 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-29T08:15:14.866 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-routes. 2024-03-29T08:15:14.873 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-29T08:15:14.881 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-29T08:15:14.895 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../100-cephfs-mirror_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:14.916 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-29T08:15:14.931 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking cephfs-mirror (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:15.226 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-colorama. 2024-03-29T08:15:15.239 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../101-python3-colorama_0.4.4-1_all.deb ... 2024-03-29T08:15:15.261 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-29T08:15:15.268 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:15:15.309 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-29T08:15:15.319 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-dashboard (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:15.571 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package cephfs-shell. 2024-03-29T08:15:15.584 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../102-cephfs-shell_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:15:15.629 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking cephfs-shell (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:15.756 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-29T08:15:15.763 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-29T08:15:15.814 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-29T08:15:15.890 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package cephfs-top. 2024-03-29T08:15:15.903 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../103-cephfs-top_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:15:15.948 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking cephfs-top (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:16.401 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-29T08:15:16.409 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-joblib. 2024-03-29T08:15:16.414 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../104-libcephfs-dev_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:16.417 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-29T08:15:16.450 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libcephfs-dev (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:16.459 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-29T08:15:16.837 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package nvme-cli. 2024-03-29T08:15:16.846 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-29T08:15:16.851 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../105-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-29T08:15:16.854 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-29T08:15:16.895 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-29T08:15:16.913 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-29T08:15:17.174 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-sklearn. 2024-03-29T08:15:17.181 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-29T08:15:17.215 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-29T08:15:17.216 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-29T08:15:17.229 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../106-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-29T08:15:17.266 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-29T08:15:17.703 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pastescript. 2024-03-29T08:15:17.716 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-03-29T08:15:17.769 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-29T08:15:18.026 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-29T08:15:18.032 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:15:18.056 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-psutil. 2024-03-29T08:15:18.069 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-29T08:15:18.076 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:18.106 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-29T08:15:18.467 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pygments. 2024-03-29T08:15:18.481 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../109-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-29T08:15:18.525 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-29T08:15:18.921 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-29T08:15:18.933 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-29T08:15:19.004 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-29T08:15:19.039 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cachetools. 2024-03-29T08:15:19.047 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-03-29T08:15:19.089 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-29T08:15:19.332 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-simplejson. 2024-03-29T08:15:19.345 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-29T08:15:19.390 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-29T08:15:19.392 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-29T08:15:19.400 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-29T08:15:19.442 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-29T08:15:19.659 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-29T08:15:19.673 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-29T08:15:19.751 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-29T08:15:19.787 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-29T08:15:19.794 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-29T08:15:19.837 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-29T08:15:20.132 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rsa. 2024-03-29T08:15:20.139 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-03-29T08:15:20.181 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-29T08:15:20.421 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package radosgw. 2024-03-29T08:15:20.434 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../113-radosgw_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:20.463 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking radosgw (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:20.510 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-google-auth. 2024-03-29T08:15:20.517 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-03-29T08:15:20.559 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-29T08:15:20.879 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-29T08:15:20.890 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-29T08:15:20.929 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-29T08:15:21.024 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package rbd-fuse. 2024-03-29T08:15:21.037 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../114-rbd-fuse_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:21.074 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking rbd-fuse (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:21.241 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-websocket. 2024-03-29T08:15:21.248 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-03-29T08:15:21.290 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-29T08:15:21.389 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-29T08:15:21.619 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-29T08:15:21.626 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-29T08:15:21.654 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-29T08:15:21.660 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-29T08:15:21.780 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-29T08:15:22.035 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-29T08:15:22.299 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-29T08:15:22.305 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:15:22.342 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-k8sevents (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:22.436 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-29T08:15:22.843 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-29T08:15:22.853 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-29T08:15:22.892 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-29T08:15:23.058 INFO:teuthology.orchestra.run.smithi073.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-29T08:15:23.101 INFO:teuthology.orchestra.run.smithi073.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-03-29T08:15:23.232 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up cephadm (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:23.237 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-29T08:15:23.246 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-29T08:15:23.312 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-29T08:15:23.582 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package jq. 2024-03-29T08:15:23.590 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-29T08:15:23.632 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-29T08:15:23.776 INFO:teuthology.orchestra.run.smithi073.stdout:Adding system user cephadm....done 2024-03-29T08:15:23.842 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package socat. 2024-03-29T08:15:23.851 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-29T08:15:23.884 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-29T08:15:23.930 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-29T08:15:24.187 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package xmlstarlet. 2024-03-29T08:15:24.196 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-29T08:15:24.206 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-29T08:15:24.237 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-29T08:15:24.557 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-29T08:15:24.566 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-test. 2024-03-29T08:15:24.574 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../098-ceph-test_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:24.608 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-test (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:24.683 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-29T08:15:24.942 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-29T08:15:25.227 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-29T08:15:25.362 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-29T08:15:25.671 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-29T08:15:26.107 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-29T08:15:26.329 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-volume. 2024-03-29T08:15:26.337 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../099-ceph-volume_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:15:26.366 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-29T08:15:26.378 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-volume (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:26.643 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-29T08:15:26.691 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-29T08:15:26.700 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../100-cephfs-mirror_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:26.740 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-mirror (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:26.919 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-29T08:15:27.069 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-colorama. 2024-03-29T08:15:27.078 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../101-python3-colorama_0.4.4-1_all.deb ... 2024-03-29T08:15:27.118 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-29T08:15:27.187 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-29T08:15:27.379 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-shell. 2024-03-29T08:15:27.388 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../102-cephfs-shell_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:15:27.429 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-shell (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:27.479 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-29T08:15:27.707 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-top. 2024-03-29T08:15:27.716 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../103-cephfs-top_18.2.2-1015-gd0e392d9-1jammy_all.deb ... 2024-03-29T08:15:27.738 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-ceph-argparse (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:27.757 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-top (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:27.968 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-29T08:15:27.977 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../104-libcephfs-dev_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:28.007 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-29T08:15:28.018 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libcephfs-dev (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:28.157 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-29T08:15:28.396 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package nvme-cli. 2024-03-29T08:15:28.404 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../105-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-29T08:15:28.417 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-29T08:15:28.445 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-29T08:15:28.685 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-29T08:15:28.784 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-29T08:15:28.792 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../106-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-29T08:15:28.834 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-29T08:15:28.835 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-29T08:15:29.187 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-29T08:15:29.296 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-29T08:15:29.328 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pastescript. 2024-03-29T08:15:29.340 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-03-29T08:15:29.378 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-29T08:15:29.547 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-29T08:15:29.698 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-psutil. 2024-03-29T08:15:29.708 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-29T08:15:29.748 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-29T08:15:29.865 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-29T08:15:30.118 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pygments. 2024-03-29T08:15:30.127 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../109-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-29T08:15:30.167 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-29T08:15:30.183 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-29T08:15:30.655 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-29T08:15:30.663 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-29T08:15:30.696 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-29T08:15:30.868 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-29T08:15:30.991 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-simplejson. 2024-03-29T08:15:31.003 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-29T08:15:31.040 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-29T08:15:31.135 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-29T08:15:31.253 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-29T08:15:31.309 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-29T08:15:31.323 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-29T08:15:31.368 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-29T08:15:31.379 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-29T08:15:31.930 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-29T08:15:32.121 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package radosgw. 2024-03-29T08:15:32.132 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../113-radosgw_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:32.171 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking radosgw (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:32.332 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-29T08:15:32.616 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-29T08:15:32.916 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package rbd-fuse. 2024-03-29T08:15:32.927 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../114-rbd-fuse_18.2.2-1015-gd0e392d9-1jammy_amd64.deb ... 2024-03-29T08:15:32.965 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking rbd-fuse (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:33.009 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-29T08:15:33.283 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-29T08:15:33.294 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-29T08:15:33.562 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-29T08:15:33.645 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-29T08:15:33.688 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-29T08:15:33.958 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-29T08:15:33.980 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-29T08:15:34.248 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-29T08:15:34.373 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-ceph-common (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:34.374 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-29T08:15:34.683 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-idna (3.3-1) ... 2024-03-29T08:15:35.026 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-29T08:15:35.030 INFO:teuthology.orchestra.run.smithi046.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-29T08:15:35.062 INFO:teuthology.orchestra.run.smithi046.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-03-29T08:15:35.107 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephadm (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:35.634 INFO:teuthology.orchestra.run.smithi046.stdout:Adding system user cephadm....done 2024-03-29T08:15:35.788 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-29T08:15:36.098 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-29T08:15:36.240 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T08:15:36.399 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-29T08:15:36.457 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-29T08:15:36.583 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-29T08:15:36.667 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-29T08:15:36.784 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-29T08:15:36.851 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-29T08:15:36.902 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-29T08:15:37.143 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-29T08:15:37.187 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-29T08:15:37.269 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-29T08:15:37.446 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-29T08:15:37.588 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-29T08:15:37.706 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T08:15:37.857 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-03-29T08:15:37.973 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-29T08:15:37.982 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-29T08:15:38.224 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-29T08:15:38.309 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-29T08:15:38.500 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-29T08:15:38.560 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-29T08:15:38.793 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-29T08:15:38.861 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-29T08:15:39.102 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-29T08:15:39.146 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-29T08:15:39.379 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-29T08:15:39.405 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-29T08:15:39.646 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-ceph-argparse (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:39.689 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-29T08:15:39.815 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-29T08:15:39.922 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-29T08:15:40.040 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-29T08:15:40.108 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-29T08:15:40.267 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-29T08:15:40.316 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-29T08:15:40.552 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-29T08:15:40.575 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-29T08:15:40.702 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-29T08:15:40.870 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-29T08:15:41.077 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-29T08:15:41.171 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-29T08:15:41.203 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-29T08:15:41.488 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-29T08:15:41.567 INFO:teuthology.orchestra.run.smithi073.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-29T08:15:41.755 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-29T08:15:41.807 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-29T08:15:41.933 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-29T08:15:42.057 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-29T08:15:42.334 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-29T08:15:42.635 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-29T08:15:42.759 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-29T08:15:42.903 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T08:15:43.004 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-29T08:15:43.326 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-29T08:15:43.330 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-29T08:15:43.452 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-29T08:15:43.578 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-29T08:15:43.639 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-29T08:15:43.924 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-29T08:15:43.938 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-29T08:15:44.050 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-29T08:15:44.326 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-29T08:15:44.339 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-29T08:15:44.590 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-29T08:15:44.627 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-29T08:15:44.895 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-29T08:15:44.984 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-29T08:15:45.263 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-29T08:15:45.293 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-29T08:15:45.565 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-29T08:15:45.619 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-29T08:15:45.916 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-29T08:15:45.987 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-29T08:15:46.034 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up librados2 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:46.185 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up librgw2 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:46.255 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-29T08:15:46.373 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-ceph-common (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:46.380 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libsqlite3-mod-ceph (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:46.506 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libcephfs2 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:46.632 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libradosstriper1 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:46.665 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-idna (3.3-1) ... 2024-03-29T08:15:46.758 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-29T08:15:47.017 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-29T08:15:47.060 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up librbd1 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:47.169 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr-modules-core (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:47.295 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-fuse (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:47.505 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-29T08:15:47.506 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-29T08:15:48.147 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libcephfs-dev (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:48.273 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-rados (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:48.288 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T08:15:48.398 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-29T08:15:48.432 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up cephfs-top (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:48.558 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-29T08:15:48.657 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-29T08:15:48.808 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-29T08:15:48.934 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-29T08:15:49.244 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-29T08:15:49.520 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-29T08:15:49.779 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T08:15:49.854 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-rbd (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:49.897 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-03-29T08:15:50.014 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up rbd-fuse (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:50.015 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-29T08:15:50.139 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-rgw (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:50.265 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-cephfs (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:50.333 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-29T08:15:50.391 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up cephfs-shell (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:50.509 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-common (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:50.601 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-29T08:15:50.877 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-29T08:15:51.079 INFO:teuthology.orchestra.run.smithi073.stdout:Adding group ceph....done 2024-03-29T08:15:51.145 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-29T08:15:51.412 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-29T08:15:51.480 INFO:teuthology.orchestra.run.smithi073.stdout:Adding system user ceph....done 2024-03-29T08:15:51.649 INFO:teuthology.orchestra.run.smithi073.stdout:Setting system user ceph properties....done 2024-03-29T08:15:51.668 INFO:teuthology.orchestra.run.smithi073.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-29T08:15:51.688 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-29T08:15:51.798 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-29T08:15:51.807 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-29T08:15:52.082 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-29T08:15:52.200 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-29T08:15:52.204 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-29T08:15:52.501 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-29T08:15:52.819 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-29T08:15:52.977 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up cephfs-mirror (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:53.086 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-29T08:15:53.220 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-29T08:15:53.220 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-29T08:15:53.479 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-29T08:15:53.849 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-29T08:15:53.870 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-test (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:53.974 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-29T08:15:53.996 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up radosgw (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:54.409 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-29T08:15:54.546 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-29T08:15:54.546 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-29T08:15:54.685 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-29T08:15:54.952 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-29T08:15:55.079 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-29T08:15:55.206 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-base (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:55.438 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-29T08:15:55.570 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-29T08:15:55.739 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-29T08:15:56.040 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-29T08:15:56.149 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-29T08:15:56.208 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mds (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:56.429 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-29T08:15:56.429 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-29T08:15:56.434 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-29T08:15:56.718 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-29T08:15:56.994 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-29T08:15:57.102 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:57.307 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-29T08:15:57.307 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-29T08:15:57.395 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-29T08:15:57.697 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-29T08:15:57.971 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-osd (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:58.049 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-29T08:15:58.175 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librados2 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:58.283 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-29T08:15:58.283 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-29T08:15:58.301 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librgw2 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:58.512 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libsqlite3-mod-ceph (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:58.663 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libcephfs2 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:58.773 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libradosstriper1 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:58.899 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-29T08:15:58.940 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr-k8sevents (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:59.091 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr-diskprediction-local (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:59.192 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librbd1 (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:59.233 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mon (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:59.318 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-modules-core (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:59.442 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-29T08:15:59.442 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-29T08:15:59.452 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-fuse (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:15:59.674 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-29T08:15:59.674 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-29T08:16:00.093 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr-cephadm (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:00.219 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:00.320 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr-dashboard (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:00.395 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libcephfs-dev (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:00.454 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-volume (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:00.522 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rados (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:00.639 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-top (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:00.740 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-29T08:16:00.832 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-29T08:16:01.924 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-29T08:16:02.061 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rbd (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:02.187 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up rbd-fuse (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:02.313 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rgw (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:02.431 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cephfs (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:02.532 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-shell (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:02.558 INFO:teuthology.orchestra.run.smithi073.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-29T08:16:02.563 DEBUG:teuthology.orchestra.run.smithi073:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-29T08:16:02.633 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-common (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:02.636 INFO:teuthology.orchestra.run.smithi073.stdout:Reading package lists... 2024-03-29T08:16:02.808 INFO:teuthology.orchestra.run.smithi073.stdout:Building dependency tree... 2024-03-29T08:16:02.808 INFO:teuthology.orchestra.run.smithi073.stdout:Reading state information... 2024-03-29T08:16:02.976 INFO:teuthology.orchestra.run.smithi073.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-29T08:16:02.977 INFO:teuthology.orchestra.run.smithi073.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-29T08:16:02.977 INFO:teuthology.orchestra.run.smithi073.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-29T08:16:02.977 INFO:teuthology.orchestra.run.smithi073.stdout:The following packages were automatically installed and are no longer required: 2024-03-29T08:16:02.977 INFO:teuthology.orchestra.run.smithi073.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-29T08:16:02.977 INFO:teuthology.orchestra.run.smithi073.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-29T08:16:02.977 INFO:teuthology.orchestra.run.smithi073.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-29T08:16:02.977 INFO:teuthology.orchestra.run.smithi073.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-29T08:16:02.978 INFO:teuthology.orchestra.run.smithi073.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-29T08:16:02.978 INFO:teuthology.orchestra.run.smithi073.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-29T08:16:02.978 INFO:teuthology.orchestra.run.smithi073.stdout:The following additional packages will be installed: 2024-03-29T08:16:02.978 INFO:teuthology.orchestra.run.smithi073.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-29T08:16:02.978 INFO:teuthology.orchestra.run.smithi073.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-29T08:16:02.979 INFO:teuthology.orchestra.run.smithi073.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-03-29T08:16:02.979 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-29T08:16:02.979 INFO:teuthology.orchestra.run.smithi073.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-29T08:16:02.980 INFO:teuthology.orchestra.run.smithi073.stdout:Suggested packages: 2024-03-29T08:16:02.980 INFO:teuthology.orchestra.run.smithi073.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-29T08:16:02.980 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-03-29T08:16:02.980 INFO:teuthology.orchestra.run.smithi073.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-03-29T08:16:03.033 INFO:teuthology.orchestra.run.smithi073.stdout:The following NEW packages will be installed: 2024-03-29T08:16:03.033 INFO:teuthology.orchestra.run.smithi073.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-29T08:16:03.033 INFO:teuthology.orchestra.run.smithi073.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-29T08:16:03.034 INFO:teuthology.orchestra.run.smithi073.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-29T08:16:03.034 INFO:teuthology.orchestra.run.smithi073.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-29T08:16:03.034 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-29T08:16:03.034 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-29T08:16:03.034 INFO:teuthology.orchestra.run.smithi073.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-29T08:16:03.034 INFO:teuthology.orchestra.run.smithi073.stdout:The following packages will be upgraded: 2024-03-29T08:16:03.035 INFO:teuthology.orchestra.run.smithi073.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-03-29T08:16:03.097 INFO:teuthology.orchestra.run.smithi073.stdout:4 upgraded, 39 newly installed, 0 to remove and 297 not upgraded. 2024-03-29T08:16:03.097 INFO:teuthology.orchestra.run.smithi073.stdout:Need to get 55.7 MB of archives. 2024-03-29T08:16:03.098 INFO:teuthology.orchestra.run.smithi073.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-29T08:16:03.098 INFO:teuthology.orchestra.run.smithi073.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.3 [1,063 kB] 2024-03-29T08:16:03.244 INFO:teuthology.orchestra.run.smithi046.stdout:Adding group ceph....done 2024-03-29T08:16:03.269 INFO:teuthology.orchestra.run.smithi073.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-29T08:16:03.274 INFO:teuthology.orchestra.run.smithi073.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-29T08:16:03.311 INFO:teuthology.orchestra.run.smithi073.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.3 [33.1 kB] 2024-03-29T08:16:03.312 INFO:teuthology.orchestra.run.smithi073.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.3 [24.4 kB] 2024-03-29T08:16:03.313 INFO:teuthology.orchestra.run.smithi073.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-29T08:16:03.326 INFO:teuthology.orchestra.run.smithi073.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-29T08:16:03.327 INFO:teuthology.orchestra.run.smithi073.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-29T08:16:03.329 INFO:teuthology.orchestra.run.smithi073.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-29T08:16:03.330 INFO:teuthology.orchestra.run.smithi073.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-29T08:16:03.331 INFO:teuthology.orchestra.run.smithi073.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-29T08:16:03.332 INFO:teuthology.orchestra.run.smithi073.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-29T08:16:03.350 INFO:teuthology.orchestra.run.smithi073.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-29T08:16:03.351 INFO:teuthology.orchestra.run.smithi073.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-29T08:16:03.351 INFO:teuthology.orchestra.run.smithi073.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-29T08:16:03.371 INFO:teuthology.orchestra.run.smithi073.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-29T08:16:03.595 INFO:teuthology.orchestra.run.smithi046.stdout:Adding system user ceph....done 2024-03-29T08:16:03.701 INFO:teuthology.orchestra.run.smithi073.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-29T08:16:03.703 INFO:teuthology.orchestra.run.smithi073.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-29T08:16:03.704 INFO:teuthology.orchestra.run.smithi073.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-29T08:16:03.707 INFO:teuthology.orchestra.run.smithi073.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-29T08:16:03.708 INFO:teuthology.orchestra.run.smithi073.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-29T08:16:03.712 INFO:teuthology.orchestra.run.smithi073.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-29T08:16:03.714 INFO:teuthology.orchestra.run.smithi073.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-29T08:16:03.722 INFO:teuthology.orchestra.run.smithi073.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-29T08:16:03.723 INFO:teuthology.orchestra.run.smithi073.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-29T08:16:03.745 INFO:teuthology.orchestra.run.smithi073.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-29T08:16:03.748 INFO:teuthology.orchestra.run.smithi073.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-29T08:16:03.756 INFO:teuthology.orchestra.run.smithi046.stdout:Setting system user ceph properties....done 2024-03-29T08:16:03.760 INFO:teuthology.orchestra.run.smithi073.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-29T08:16:03.777 INFO:teuthology.orchestra.run.smithi046.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-29T08:16:03.787 INFO:teuthology.orchestra.run.smithi073.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-29T08:16:03.791 INFO:teuthology.orchestra.run.smithi073.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-29T08:16:03.791 INFO:teuthology.orchestra.run.smithi073.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-29T08:16:03.792 INFO:teuthology.orchestra.run.smithi073.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-29T08:16:03.792 INFO:teuthology.orchestra.run.smithi073.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-29T08:16:03.806 INFO:teuthology.orchestra.run.smithi073.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-29T08:16:03.811 INFO:teuthology.orchestra.run.smithi073.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-29T08:16:03.930 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-29T08:16:04.009 INFO:teuthology.orchestra.run.smithi073.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-29T08:16:04.009 INFO:teuthology.orchestra.run.smithi073.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-29T08:16:04.009 INFO:teuthology.orchestra.run.smithi073.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-29T08:16:04.009 INFO:teuthology.orchestra.run.smithi073.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-29T08:16:04.010 INFO:teuthology.orchestra.run.smithi073.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-29T08:16:04.010 INFO:teuthology.orchestra.run.smithi073.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-29T08:16:04.023 INFO:teuthology.orchestra.run.smithi073.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-29T08:16:04.024 INFO:teuthology.orchestra.run.smithi073.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-29T08:16:04.338 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-29T08:16:04.399 INFO:teuthology.orchestra.run.smithi073.stdout:Preconfiguring packages ... 2024-03-29T08:16:04.540 INFO:teuthology.orchestra.run.smithi073.stdout:Fetched 55.7 MB in 1s (56.8 MB/s) 2024-03-29T08:16:04.769 INFO:teuthology.orchestra.run.smithi073.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139840 files and directories currently installed.) 2024-03-29T08:16:04.773 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-29T08:16:04.923 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking util-linux (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-29T08:16:05.034 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-mirror (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:05.249 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-29T08:16:05.250 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-29T08:16:05.555 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up util-linux (2.37.2-4ubuntu3.3) ... 2024-03-29T08:16:05.919 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-test (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:06.045 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up radosgw (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:06.606 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-29T08:16:06.606 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-29T08:16:06.687 INFO:teuthology.orchestra.run.smithi073.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-03-29T08:16:06.953 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package flex. 2024-03-29T08:16:07.021 INFO:teuthology.orchestra.run.smithi073.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139840 files and directories currently installed.) 2024-03-29T08:16:07.024 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-29T08:16:07.076 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-29T08:16:07.263 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-base (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:07.606 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-29T08:16:08.114 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-29T08:16:08.292 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-29T08:16:08.315 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mds (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-29T08:16:08.553 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-29T08:16:08.690 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-29T08:16:09.210 INFO:teuthology.orchestra.run.smithi073.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139913 files and directories currently installed.) 2024-03-29T08:16:09.214 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-29T08:16:09.254 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:09.341 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-29T08:16:09.466 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-29T08:16:09.466 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-29T08:16:09.754 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-29T08:16:09.872 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-29T08:16:10.114 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-osd (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:10.170 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.3) ... 2024-03-29T08:16:10.521 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-29T08:16:10.521 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-29T08:16:10.531 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package bison. 2024-03-29T08:16:10.606 INFO:teuthology.orchestra.run.smithi073.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139913 files and directories currently installed.) 2024-03-29T08:16:10.610 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-29T08:16:10.659 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-29T08:16:11.017 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-29T08:16:11.032 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-29T08:16:11.067 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-29T08:16:11.190 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-k8sevents (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:11.333 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-diskprediction-local (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:11.420 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package dump. 2024-03-29T08:16:11.436 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-29T08:16:11.459 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mon (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:11.478 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking dump (0.4b47-3) ... 2024-03-29T08:16:11.658 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-29T08:16:11.659 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-29T08:16:11.757 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package indent. 2024-03-29T08:16:11.773 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-29T08:16:11.815 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-29T08:16:12.093 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-29T08:16:12.109 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-29T08:16:12.143 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-29T08:16:12.285 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-cephadm (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:12.428 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:12.464 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-29T08:16:12.481 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-29T08:16:12.522 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-29T08:16:12.553 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-dashboard (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:12.713 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-volume (18.2.2-1015-gd0e392d9-1jammy) ... 2024-03-29T08:16:12.800 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-29T08:16:12.815 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-29T08:16:12.858 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-29T08:16:13.076 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-29T08:16:13.212 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-29T08:16:13.228 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-29T08:16:13.262 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-29T08:16:13.515 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-29T08:16:13.530 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-29T08:16:13.573 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-29T08:16:13.826 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libjson-perl. 2024-03-29T08:16:13.841 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-29T08:16:13.884 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-29T08:16:14.229 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-29T08:16:14.244 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-29T08:16:14.260 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-29T08:16:14.286 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-29T08:16:14.781 INFO:teuthology.orchestra.run.smithi046.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-29T08:16:14.785 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-29T08:16:14.861 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2024-03-29T08:16:15.038 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2024-03-29T08:16:15.038 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2024-03-29T08:16:15.214 INFO:teuthology.orchestra.run.smithi046.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-29T08:16:15.214 INFO:teuthology.orchestra.run.smithi046.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-29T08:16:15.214 INFO:teuthology.orchestra.run.smithi046.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-29T08:16:15.214 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2024-03-29T08:16:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-29T08:16:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-29T08:16:15.214 INFO:teuthology.orchestra.run.smithi046.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-29T08:16:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-29T08:16:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-29T08:16:15.215 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-29T08:16:15.215 INFO:teuthology.orchestra.run.smithi046.stdout:The following additional packages will be installed: 2024-03-29T08:16:15.215 INFO:teuthology.orchestra.run.smithi046.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-29T08:16:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-29T08:16:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-03-29T08:16:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-29T08:16:15.216 INFO:teuthology.orchestra.run.smithi046.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-29T08:16:15.217 INFO:teuthology.orchestra.run.smithi046.stdout:Suggested packages: 2024-03-29T08:16:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-29T08:16:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-03-29T08:16:15.217 INFO:teuthology.orchestra.run.smithi046.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-03-29T08:16:15.271 INFO:teuthology.orchestra.run.smithi046.stdout:The following NEW packages will be installed: 2024-03-29T08:16:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-29T08:16:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-29T08:16:15.272 INFO:teuthology.orchestra.run.smithi046.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-29T08:16:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-29T08:16:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-29T08:16:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-29T08:16:15.273 INFO:teuthology.orchestra.run.smithi046.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-29T08:16:15.274 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages will be upgraded: 2024-03-29T08:16:15.274 INFO:teuthology.orchestra.run.smithi046.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-03-29T08:16:15.489 INFO:teuthology.orchestra.run.smithi046.stdout:4 upgraded, 39 newly installed, 0 to remove and 297 not upgraded. 2024-03-29T08:16:15.489 INFO:teuthology.orchestra.run.smithi046.stdout:Need to get 55.7 MB of archives. 2024-03-29T08:16:15.489 INFO:teuthology.orchestra.run.smithi046.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-29T08:16:15.489 INFO:teuthology.orchestra.run.smithi046.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.3 [1,063 kB] 2024-03-29T08:16:15.665 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-29T08:16:15.680 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-29T08:16:15.723 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-29T08:16:15.984 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-29T08:16:16.000 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-29T08:16:16.042 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-29T08:16:16.134 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-29T08:16:16.150 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-29T08:16:16.276 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.3 [33.1 kB] 2024-03-29T08:16:16.277 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.3 [24.4 kB] 2024-03-29T08:16:16.278 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-29T08:16:16.307 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-29T08:16:16.309 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-29T08:16:16.312 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-29T08:16:16.313 INFO:teuthology.orchestra.run.smithi046.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-29T08:16:16.314 INFO:teuthology.orchestra.run.smithi046.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-29T08:16:16.395 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-29T08:16:16.410 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-29T08:16:16.453 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-29T08:16:16.483 INFO:teuthology.orchestra.run.smithi046.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-29T08:16:16.740 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-29T08:16:16.755 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-29T08:16:16.762 INFO:teuthology.orchestra.run.smithi046.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-29T08:16:16.774 INFO:teuthology.orchestra.run.smithi046.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-29T08:16:16.778 INFO:teuthology.orchestra.run.smithi046.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-29T08:16:16.798 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-29T08:16:16.871 INFO:teuthology.orchestra.run.smithi046.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-29T08:16:17.159 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-29T08:16:17.174 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-29T08:16:17.217 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-29T08:16:17.545 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-29T08:16:17.561 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-29T08:16:17.603 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T08:16:17.843 INFO:teuthology.orchestra.run.smithi046.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-29T08:16:17.870 INFO:teuthology.orchestra.run.smithi046.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-29T08:16:17.870 INFO:teuthology.orchestra.run.smithi046.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-29T08:16:17.873 INFO:teuthology.orchestra.run.smithi046.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-29T08:16:17.873 INFO:teuthology.orchestra.run.smithi046.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-29T08:16:17.878 INFO:teuthology.orchestra.run.smithi046.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-29T08:16:17.965 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-29T08:16:17.980 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-29T08:16:18.023 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-29T08:16:18.043 INFO:teuthology.orchestra.run.smithi046.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-29T08:16:18.393 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-29T08:16:18.409 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-29T08:16:18.451 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-29T08:16:18.649 INFO:teuthology.orchestra.run.smithi046.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-29T08:16:18.649 INFO:teuthology.orchestra.run.smithi046.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-29T08:16:18.662 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-29T08:16:18.678 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-29T08:16:18.712 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-29T08:16:18.829 INFO:teuthology.orchestra.run.smithi046.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-29T08:16:18.830 INFO:teuthology.orchestra.run.smithi046.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-29T08:16:18.868 INFO:teuthology.orchestra.run.smithi046.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-29T08:16:18.930 INFO:teuthology.orchestra.run.smithi046.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-29T08:16:18.935 INFO:teuthology.orchestra.run.smithi046.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-29T08:16:18.936 INFO:teuthology.orchestra.run.smithi046.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-29T08:16:18.937 INFO:teuthology.orchestra.run.smithi046.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-29T08:16:18.937 INFO:teuthology.orchestra.run.smithi046.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-29T08:16:19.144 INFO:teuthology.orchestra.run.smithi046.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-29T08:16:19.199 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-29T08:16:19.214 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-29T08:16:19.257 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-29T08:16:19.559 INFO:teuthology.orchestra.run.smithi046.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-29T08:16:19.610 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package modemmanager. 2024-03-29T08:16:19.626 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-29T08:16:19.669 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-29T08:16:20.306 INFO:teuthology.orchestra.run.smithi046.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-29T08:16:20.306 INFO:teuthology.orchestra.run.smithi046.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-29T08:16:20.307 INFO:teuthology.orchestra.run.smithi046.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-29T08:16:20.307 INFO:teuthology.orchestra.run.smithi046.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-29T08:16:20.307 INFO:teuthology.orchestra.run.smithi046.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-29T08:16:20.307 INFO:teuthology.orchestra.run.smithi046.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-29T08:16:20.317 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package network-manager. 2024-03-29T08:16:20.333 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-29T08:16:20.342 INFO:teuthology.orchestra.run.smithi046.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-29T08:16:20.343 INFO:teuthology.orchestra.run.smithi046.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-29T08:16:20.384 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-29T08:16:20.739 INFO:teuthology.orchestra.run.smithi046.stdout:Preconfiguring packages ... 2024-03-29T08:16:21.018 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 55.7 MB in 5s (11.0 MB/s) 2024-03-29T08:16:21.099 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ppp. 2024-03-29T08:16:21.115 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-29T08:16:21.338 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139840 files and directories currently installed.) 2024-03-29T08:16:21.342 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-29T08:16:21.463 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-29T08:16:21.484 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking util-linux (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-29T08:16:21.796 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package pptp-linux. 2024-03-29T08:16:21.810 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-29T08:16:21.854 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-29T08:16:22.123 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up util-linux (2.37.2-4ubuntu3.3) ... 2024-03-29T08:16:22.182 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-29T08:16:22.196 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-29T08:16:22.249 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-29T08:16:22.618 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-29T08:16:22.633 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-29T08:16:22.685 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking postgresql-client-common (238) ... 2024-03-29T08:16:23.089 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-29T08:16:23.104 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-29T08:16:23.232 INFO:teuthology.orchestra.run.smithi046.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-03-29T08:16:23.326 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T08:16:23.495 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package flex. 2024-03-29T08:16:23.569 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139840 files and directories currently installed.) 2024-03-29T08:16:23.572 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-29T08:16:23.619 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-29T08:16:23.995 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package postgresql-common. 2024-03-29T08:16:24.010 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-29T08:16:24.071 INFO:teuthology.orchestra.run.smithi073.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-29T08:16:24.102 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking postgresql-common (238) ... 2024-03-29T08:16:24.130 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-29T08:16:24.275 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-29T08:16:24.602 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package postgresql-14. 2024-03-29T08:16:24.617 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-29T08:16:24.657 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-29T08:16:24.663 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T08:16:25.214 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139913 files and directories currently installed.) 2024-03-29T08:16:25.218 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-29T08:16:25.365 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-29T08:16:25.713 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package postgresql. 2024-03-29T08:16:25.728 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-29T08:16:25.771 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking postgresql (14+238) ... 2024-03-29T08:16:25.820 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.3_amd64.deb ... 2024-03-29T08:16:25.955 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.3) over (2.37.2-4ubuntu3) ... 2024-03-29T08:16:26.007 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package postgresql-client. 2024-03-29T08:16:26.022 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-29T08:16:26.091 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking postgresql-client (14+238) ... 2024-03-29T08:16:26.309 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-29T08:16:26.324 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-29T08:16:26.354 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.3) ... 2024-03-29T08:16:26.368 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-29T08:16:26.687 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package tcl8.6. 2024-03-29T08:16:26.697 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package bison. 2024-03-29T08:16:26.701 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-29T08:16:26.745 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-29T08:16:26.776 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139913 files and directories currently installed.) 2024-03-29T08:16:26.780 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-29T08:16:26.824 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-29T08:16:27.031 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package tcl. 2024-03-29T08:16:27.047 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-29T08:16:27.094 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-29T08:16:27.250 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-29T08:16:27.267 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-29T08:16:27.309 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-29T08:16:27.459 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package wpasupplicant. 2024-03-29T08:16:27.475 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-29T08:16:27.542 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-29T08:16:27.703 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package dump. 2024-03-29T08:16:27.720 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-29T08:16:27.761 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking dump (0.4b47-3) ... 2024-03-29T08:16:27.888 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-29T08:16:27.904 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-29T08:16:27.946 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-29T08:16:28.040 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package indent. 2024-03-29T08:16:28.056 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-29T08:16:28.090 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-29T08:16:28.333 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-29T08:16:28.348 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-29T08:16:28.359 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-29T08:16:28.375 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-29T08:16:28.391 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-29T08:16:28.417 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-29T08:16:28.658 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up postgresql-client-common (238) ... 2024-03-29T08:16:28.746 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-29T08:16:28.762 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-29T08:16:28.804 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-29T08:16:28.855 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-29T08:16:28.990 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-29T08:16:29.074 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-29T08:16:29.090 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-29T08:16:29.132 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-29T08:16:29.149 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T08:16:29.275 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-29T08:16:29.419 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-29T08:16:29.435 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-29T08:16:29.469 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-29T08:16:29.501 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-29T08:16:29.627 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T08:16:29.754 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-29T08:16:29.770 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-29T08:16:29.812 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-29T08:16:30.049 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjson-perl. 2024-03-29T08:16:30.065 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-29T08:16:30.107 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-29T08:16:30.410 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-29T08:16:30.427 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-29T08:16:30.468 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-29T08:16:30.561 INFO:teuthology.orchestra.run.smithi073.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-29T08:16:30.666 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-29T08:16:31.043 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-29T08:16:31.043 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-29T08:16:31.777 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.3) ... 2024-03-29T08:16:31.855 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-29T08:16:31.870 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-29T08:16:31.903 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-29T08:16:31.912 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-29T08:16:32.062 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-29T08:16:32.188 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-29T08:16:32.190 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-29T08:16:32.206 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-29T08:16:32.248 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-29T08:16:32.314 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-29T08:16:32.424 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-29T08:16:32.533 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-29T08:16:32.535 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-29T08:16:32.551 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-29T08:16:32.593 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-29T08:16:32.642 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-29T08:16:32.963 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-29T08:16:32.978 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-29T08:16:33.021 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-29T08:16:33.348 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-29T08:16:33.348 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-29T08:16:33.364 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-29T08:16:33.407 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-29T08:16:33.701 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-29T08:16:33.717 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-29T08:16:33.759 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T08:16:33.857 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up dump (0.4b47-3) ... 2024-03-29T08:16:33.952 INFO:teuthology.orchestra.run.smithi073.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-29T08:16:34.000 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-29T08:16:34.112 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-29T08:16:34.118 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-29T08:16:34.129 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-29T08:16:34.170 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-29T08:16:34.244 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-29T08:16:34.370 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-29T08:16:34.498 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-29T08:16:34.514 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-29T08:16:34.556 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-29T08:16:34.562 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-29T08:16:34.652 INFO:teuthology.orchestra.run.smithi073.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-29T08:16:34.722 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-29T08:16:34.776 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-29T08:16:34.792 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-29T08:16:34.825 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-29T08:16:34.848 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-29T08:16:35.008 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-29T08:16:35.117 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-29T08:16:35.235 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-29T08:16:35.346 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-29T08:16:35.362 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-29T08:16:35.395 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up postgresql-client (14+238) ... 2024-03-29T08:16:35.405 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-29T08:16:35.496 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-29T08:16:35.757 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package modemmanager. 2024-03-29T08:16:35.773 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-29T08:16:35.816 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-29T08:16:36.076 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:16:36.076 INFO:teuthology.orchestra.run.smithi073.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-29T08:16:36.076 INFO:teuthology.orchestra.run.smithi073.stdout:which means they are currently configured by ifupdown: 2024-03-29T08:16:36.076 INFO:teuthology.orchestra.run.smithi073.stdout:- enp3s0f1 2024-03-29T08:16:36.076 INFO:teuthology.orchestra.run.smithi073.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-29T08:16:36.076 INFO:teuthology.orchestra.run.smithi073.stdout:remove their configuration from /etc/network/interfaces. 2024-03-29T08:16:36.076 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:16:36.210 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-29T08:16:36.414 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package network-manager. 2024-03-29T08:16:36.430 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-29T08:16:36.480 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-29T08:16:36.621 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-29T08:16:37.052 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-29T08:16:37.093 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ppp. 2024-03-29T08:16:37.110 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-29T08:16:37.482 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-29T08:16:37.765 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-29T08:16:37.817 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package pptp-linux. 2024-03-29T08:16:37.832 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-29T08:16:37.884 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-29T08:16:37.908 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-29T08:16:38.034 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-29T08:16:38.103 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-29T08:16:38.118 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-29T08:16:38.151 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up postgresql-common (238) ... 2024-03-29T08:16:38.153 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-29T08:16:38.439 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-29T08:16:38.454 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-29T08:16:38.531 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-client-common (238) ... 2024-03-29T08:16:38.835 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-29T08:16:38.851 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-29T08:16:38.893 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T08:16:39.389 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-common. 2024-03-29T08:16:39.397 INFO:teuthology.orchestra.run.smithi073.stdout:Adding user postgres to group ssl-cert 2024-03-29T08:16:39.404 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-29T08:16:39.457 INFO:teuthology.orchestra.run.smithi046.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-29T08:16:39.496 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-common (238) ... 2024-03-29T08:16:39.642 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:16:39.642 INFO:teuthology.orchestra.run.smithi073.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-29T08:16:39.902 INFO:teuthology.orchestra.run.smithi073.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-29T08:16:39.902 INFO:teuthology.orchestra.run.smithi073.stdout:Removing obsolete dictionary files: 2024-03-29T08:16:40.304 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-14. 2024-03-29T08:16:40.321 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-29T08:16:40.365 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T08:16:40.616 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-29T08:16:41.080 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T08:16:41.418 INFO:teuthology.orchestra.run.smithi073.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-29T08:16:41.443 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-29T08:16:41.489 INFO:teuthology.orchestra.run.smithi073.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-29T08:16:41.489 INFO:teuthology.orchestra.run.smithi073.stdout:This user must also own the server process. 2024-03-29T08:16:41.489 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:16:41.489 INFO:teuthology.orchestra.run.smithi073.stdout:The database cluster will be initialized with locale "en_US". 2024-03-29T08:16:41.490 INFO:teuthology.orchestra.run.smithi073.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-29T08:16:41.490 INFO:teuthology.orchestra.run.smithi073.stdout:The default text search configuration will be set to "english". 2024-03-29T08:16:41.490 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:16:41.490 INFO:teuthology.orchestra.run.smithi073.stdout:Data page checksums are disabled. 2024-03-29T08:16:41.490 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:16:41.490 INFO:teuthology.orchestra.run.smithi073.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-29T08:16:41.490 INFO:teuthology.orchestra.run.smithi073.stdout:creating subdirectories ... ok 2024-03-29T08:16:41.491 INFO:teuthology.orchestra.run.smithi073.stdout:selecting dynamic shared memory implementation ... posix 2024-03-29T08:16:41.514 INFO:teuthology.orchestra.run.smithi073.stdout:selecting default max_connections ... 100 2024-03-29T08:16:41.532 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql. 2024-03-29T08:16:41.547 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-29T08:16:41.548 INFO:teuthology.orchestra.run.smithi073.stdout:selecting default shared_buffers ... 128MB 2024-03-29T08:16:41.562 INFO:teuthology.orchestra.run.smithi073.stdout:selecting default time zone ... Etc/UTC 2024-03-29T08:16:41.562 INFO:teuthology.orchestra.run.smithi073.stdout:creating configuration files ... ok 2024-03-29T08:16:41.590 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql (14+238) ... 2024-03-29T08:16:41.725 INFO:teuthology.orchestra.run.smithi073.stdout:running bootstrap script ... ok 2024-03-29T08:16:41.817 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-client. 2024-03-29T08:16:41.832 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-29T08:16:41.875 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-client (14+238) ... 2024-03-29T08:16:42.153 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-29T08:16:42.168 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-29T08:16:42.211 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-29T08:16:42.347 INFO:teuthology.orchestra.run.smithi073.stdout:performing post-bootstrap initialization ... ok 2024-03-29T08:16:42.522 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package tcl8.6. 2024-03-29T08:16:42.538 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-29T08:16:42.571 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-29T08:16:42.841 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package tcl. 2024-03-29T08:16:42.857 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-29T08:16:42.903 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-29T08:16:42.920 INFO:teuthology.orchestra.run.smithi073.stdout:syncing data to disk ... ok 2024-03-29T08:16:43.260 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package wpasupplicant. 2024-03-29T08:16:43.276 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-29T08:16:43.318 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-29T08:16:43.453 INFO:teuthology.orchestra.run.smithi073.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-29T08:16:43.706 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-29T08:16:43.722 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-29T08:16:43.764 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-29T08:16:44.084 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-29T08:16:44.100 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-29T08:16:44.141 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-29T08:16:44.443 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-client-common (238) ... 2024-03-29T08:16:44.656 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-29T08:16:44.882 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-29T08:16:45.275 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T08:16:45.392 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-29T08:16:45.502 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-29T08:16:45.636 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T08:16:46.488 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-29T08:16:46.603 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-29T08:16:46.614 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-29T08:16:46.700 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-29T08:16:46.731 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up postgresql (14+238) ... 2024-03-29T08:16:46.841 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-29T08:16:46.967 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-29T08:16:47.060 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-29T08:16:47.061 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-29T08:16:47.226 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-29T08:16:47.227 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-29T08:16:47.778 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.3) ... 2024-03-29T08:16:47.896 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-29T08:16:47.978 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-29T08:16:48.021 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-29T08:16:48.147 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-29T08:16:48.162 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-29T08:16:48.273 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-29T08:16:48.481 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-29T08:16:48.482 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-29T08:16:48.608 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-29T08:16:48.666 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-29T08:16:48.734 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-29T08:16:49.340 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-29T08:16:49.465 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up dump (0.4b47-3) ... 2024-03-29T08:16:49.560 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-29T08:16:49.617 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-29T08:16:49.743 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-29T08:16:49.877 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-29T08:16:50.003 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-29T08:16:50.044 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-29T08:16:50.112 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-29T08:16:50.170 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-29T08:16:50.202 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-29T08:16:50.272 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-29T08:16:50.398 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-29T08:16:50.574 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-29T08:16:50.701 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-29T08:16:50.826 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-29T08:16:50.831 INFO:teuthology.orchestra.run.smithi073.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-29T08:16:50.839 DEBUG:teuthology.parallel:result is None 2024-03-29T08:16:50.978 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-client (14+238) ... 2024-03-29T08:16:51.079 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-29T08:16:51.689 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:16:51.689 INFO:teuthology.orchestra.run.smithi046.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-29T08:16:51.689 INFO:teuthology.orchestra.run.smithi046.stdout:which means they are currently configured by ifupdown: 2024-03-29T08:16:51.689 INFO:teuthology.orchestra.run.smithi046.stdout:- ens1f0 2024-03-29T08:16:51.689 INFO:teuthology.orchestra.run.smithi046.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-29T08:16:51.689 INFO:teuthology.orchestra.run.smithi046.stdout:remove their configuration from /etc/network/interfaces. 2024-03-29T08:16:51.689 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:16:51.828 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-29T08:16:52.230 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-29T08:16:52.652 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-29T08:16:53.373 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-29T08:16:53.533 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-29T08:16:53.642 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-29T08:16:53.745 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-common (238) ... 2024-03-29T08:16:54.762 INFO:teuthology.orchestra.run.smithi046.stdout:Adding user postgres to group ssl-cert 2024-03-29T08:16:55.030 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:16:55.031 INFO:teuthology.orchestra.run.smithi046.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-29T08:16:55.303 INFO:teuthology.orchestra.run.smithi046.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-29T08:16:55.303 INFO:teuthology.orchestra.run.smithi046.stdout:Removing obsolete dictionary files: 2024-03-29T08:16:56.050 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-29T08:16:56.547 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-29T08:16:57.007 INFO:teuthology.orchestra.run.smithi046.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-29T08:16:57.021 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-29T08:16:57.077 INFO:teuthology.orchestra.run.smithi046.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-29T08:16:57.077 INFO:teuthology.orchestra.run.smithi046.stdout:This user must also own the server process. 2024-03-29T08:16:57.077 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:16:57.077 INFO:teuthology.orchestra.run.smithi046.stdout:The database cluster will be initialized with locale "en_US". 2024-03-29T08:16:57.077 INFO:teuthology.orchestra.run.smithi046.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-29T08:16:57.077 INFO:teuthology.orchestra.run.smithi046.stdout:The default text search configuration will be set to "english". 2024-03-29T08:16:57.077 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:16:57.078 INFO:teuthology.orchestra.run.smithi046.stdout:Data page checksums are disabled. 2024-03-29T08:16:57.078 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:16:57.078 INFO:teuthology.orchestra.run.smithi046.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-29T08:16:57.078 INFO:teuthology.orchestra.run.smithi046.stdout:creating subdirectories ... ok 2024-03-29T08:16:57.078 INFO:teuthology.orchestra.run.smithi046.stdout:selecting dynamic shared memory implementation ... posix 2024-03-29T08:16:57.104 INFO:teuthology.orchestra.run.smithi046.stdout:selecting default max_connections ... 100 2024-03-29T08:16:57.138 INFO:teuthology.orchestra.run.smithi046.stdout:selecting default shared_buffers ... 128MB 2024-03-29T08:16:57.153 INFO:teuthology.orchestra.run.smithi046.stdout:selecting default time zone ... Etc/UTC 2024-03-29T08:16:57.153 INFO:teuthology.orchestra.run.smithi046.stdout:creating configuration files ... ok 2024-03-29T08:16:57.325 INFO:teuthology.orchestra.run.smithi046.stdout:running bootstrap script ... ok 2024-03-29T08:16:57.940 INFO:teuthology.orchestra.run.smithi046.stdout:performing post-bootstrap initialization ... ok 2024-03-29T08:16:58.543 INFO:teuthology.orchestra.run.smithi046.stdout:syncing data to disk ... ok 2024-03-29T08:16:59.058 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-29T08:17:02.146 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-29T08:17:02.272 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-29T08:17:02.397 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql (14+238) ... 2024-03-29T08:17:02.523 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-29T08:17:02.674 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-29T08:17:02.942 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-29T08:17:02.942 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-29T08:17:03.693 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-29T08:17:03.894 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-29T08:17:04.213 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-29T08:17:04.389 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-29T08:17:05.834 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-29T08:17:05.934 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-29T08:17:06.650 INFO:teuthology.orchestra.run.smithi046.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-29T08:17:06.657 DEBUG:teuthology.parallel:result is None 2024-03-29T08:17:06.657 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:17:06.799 DEBUG:teuthology.orchestra.run.smithi046:> dpkg-query -W -f '${Version}' ceph 2024-03-29T08:17:06.821 INFO:teuthology.orchestra.run.smithi046.stdout:18.2.2-1015-gd0e392d9-1jammy 2024-03-29T08:17:06.821 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1015-gd0e392d9-1jammy 2024-03-29T08:17:06.822 INFO:teuthology.task.install:The correct ceph version 18.2.2-1015-gd0e392d9-1jammy is installed. 2024-03-29T08:17:06.824 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=d0e392d9aee635f2b65768c460c7a151ead7a5e2 2024-03-29T08:17:07.039 DEBUG:teuthology.orchestra.run.smithi073:> dpkg-query -W -f '${Version}' ceph 2024-03-29T08:17:07.063 INFO:teuthology.orchestra.run.smithi073.stdout:18.2.2-1015-gd0e392d9-1jammy 2024-03-29T08:17:07.063 INFO:teuthology.packaging:The installed version of ceph is 18.2.2-1015-gd0e392d9-1jammy 2024-03-29T08:17:07.063 INFO:teuthology.task.install:The correct ceph version 18.2.2-1015-gd0e392d9-1jammy is installed. 2024-03-29T08:17:07.065 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-29T08:17:07.065 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:07.065 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-29T08:17:07.086 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:07.086 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-29T08:17:07.127 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-29T08:17:07.128 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:07.128 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-03-29T08:17:07.147 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-29T08:17:07.211 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:07.211 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/daemon-helper 2024-03-29T08:17:07.232 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-29T08:17:07.298 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-29T08:17:07.298 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:07.298 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-29T08:17:07.318 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-29T08:17:07.383 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:07.383 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-29T08:17:07.404 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-29T08:17:07.470 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-29T08:17:07.471 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:07.471 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-03-29T08:17:07.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-29T08:17:07.551 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:07.551 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/stdin-killer 2024-03-29T08:17:07.571 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-29T08:17:07.637 INFO:teuthology.run_tasks:Running task ceph... 2024-03-29T08:17:07.707 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-29T08:17:07.707 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /var/log/ceph 2024-03-29T08:17:07.710 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 777 /var/log/ceph 2024-03-29T08:17:07.730 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-29T08:17:07.730 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-29T08:17:07.774 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-29T08:17:07.795 INFO:tasks.ceph:Creating extra log directories... 2024-03-29T08:17:07.795 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-29T08:17:07.838 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-29T08:17:07.869 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-29T08:17:07.869 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-29T08:17:07.869 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-29T08:17:07.869 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:07.869 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-29T08:17:07.908 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-29T08:17:07.999 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:08.000 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-29T08:17:08.008 DEBUG:teuthology.orchestra.run.smithi073:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-29T08:17:08.104 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-29T08:17:08.105 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-29T08:17:08.105 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628113', 'branch': 'wip-vshankar-testing-20240326.105515-reef', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7628113', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20240326.105515-reef'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2'}}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20240326.105515-reef', 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 120, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7327, 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2', 'sleep_before_teardown': 0, 'subset': '990/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20240326.105515-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_d0e392d9aee635f2b65768c460c7a151ead7a5e2/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi046.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyO35H5HPJGFuhHNFtjtIjlv5ApnHunOmuTYxW+bNF+3/oX9wx9k2O2ciaYaKvoLd7IlwtsfjApdTCDY5sDlsM=', 'smithi073.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIfgdX6RoDeC9CUYoUy1+4Ab439iXukfjJUOSMT3kXSQs1Ev1THOAgjRZ7iAskt1D6wDonDlIfV1FXULpPxnirY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '1df0c7e8a2354c42d738e5454ba83ea7332deac2'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': 'd0e392d9aee635f2b65768c460c7a151ead7a5e2', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-28_11:21:59', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-29T08:17:08.105 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-29T08:17:08.109 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-29T08:17:08.118 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/run/ceph 2024-03-29T08:17:08.159 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m0777 -- /var/run/ceph 2024-03-29T08:17:08.186 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:08.186 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-03-29T08:17:08.228 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-29T08:17:08.228 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-03-29T08:17:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-29T08:17:08.280 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T08:17:08.280 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 890 Links: 1 2024-03-29T08:17:08.280 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T08:17:08.280 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-03-29 08:10:28.469131406 +0000 2024-03-29T08:17:08.280 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-03-29 08:10:28.197133836 +0000 2024-03-29T08:17:08.281 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-03-29 08:10:28.197133836 +0000 2024-03-29T08:17:08.281 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-03-29 08:10:28.197133836 +0000 2024-03-29T08:17:08.281 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-29T08:17:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-03-29T08:17:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-03-29T08:17:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000249043 s, 2.1 MB/s 2024-03-29T08:17:08.343 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-29T08:17:08.397 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-03-29T08:17:08.448 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-29T08:17:08.449 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T08:17:08.449 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 909 Links: 1 2024-03-29T08:17:08.449 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T08:17:08.449 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-03-29 08:10:28.973126904 +0000 2024-03-29T08:17:08.449 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-03-29 08:10:28.713129226 +0000 2024-03-29T08:17:08.449 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-03-29 08:10:28.713129226 +0000 2024-03-29T08:17:08.449 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-03-29 08:10:28.713129226 +0000 2024-03-29T08:17:08.449 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-29T08:17:08.510 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-03-29T08:17:08.510 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-03-29T08:17:08.511 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000248249 s, 2.1 MB/s 2024-03-29T08:17:08.512 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-29T08:17:08.570 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-03-29T08:17:08.620 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-29T08:17:08.620 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T08:17:08.620 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 924 Links: 1 2024-03-29T08:17:08.621 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T08:17:08.621 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-03-29 08:10:29.477122402 +0000 2024-03-29T08:17:08.621 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-03-29 08:10:29.205124832 +0000 2024-03-29T08:17:08.621 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-03-29 08:10:29.205124832 +0000 2024-03-29T08:17:08.621 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-03-29 08:10:29.205124832 +0000 2024-03-29T08:17:08.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-29T08:17:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-03-29T08:17:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-03-29T08:17:08.687 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000254223 s, 2.0 MB/s 2024-03-29T08:17:08.689 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-29T08:17:08.746 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-03-29T08:17:08.796 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-29T08:17:08.797 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T08:17:08.797 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 943 Links: 1 2024-03-29T08:17:08.797 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T08:17:08.797 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-03-29 08:10:29.993117793 +0000 2024-03-29T08:17:08.797 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-03-29 08:10:29.725120187 +0000 2024-03-29T08:17:08.797 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-03-29 08:10:29.725120187 +0000 2024-03-29T08:17:08.797 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-03-29 08:10:29.725120187 +0000 2024-03-29T08:17:08.797 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-29T08:17:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-03-29T08:17:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-03-29T08:17:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000262158 s, 2.0 MB/s 2024-03-29T08:17:08.865 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-29T08:17:08.918 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T08:17:08.919 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:08.919 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/scratch_devs of=/dev/stdout 2024-03-29T08:17:08.928 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-29T08:17:08.928 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_1 2024-03-29T08:17:08.978 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-29T08:17:08.978 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T08:17:08.978 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 884 Links: 1 2024-03-29T08:17:08.978 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T08:17:08.978 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-03-29 08:10:26.010217629 +0000 2024-03-29T08:17:08.979 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-03-29 08:10:25.754219916 +0000 2024-03-29T08:17:08.979 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-03-29 08:10:25.754219916 +0000 2024-03-29T08:17:08.979 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-03-29 08:10:25.754219916 +0000 2024-03-29T08:17:08.979 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-29T08:17:09.040 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-03-29T08:17:09.041 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-03-29T08:17:09.041 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000401706 s, 1.3 MB/s 2024-03-29T08:17:09.043 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-29T08:17:09.100 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_2 2024-03-29T08:17:09.150 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-29T08:17:09.150 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T08:17:09.150 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 901 Links: 1 2024-03-29T08:17:09.150 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T08:17:09.151 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-03-29 08:10:26.526213020 +0000 2024-03-29T08:17:09.151 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-03-29 08:10:26.266215342 +0000 2024-03-29T08:17:09.151 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-03-29 08:10:26.266215342 +0000 2024-03-29T08:17:09.151 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-03-29 08:10:26.266215342 +0000 2024-03-29T08:17:09.151 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-29T08:17:09.212 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-03-29T08:17:09.212 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-03-29T08:17:09.212 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000383823 s, 1.3 MB/s 2024-03-29T08:17:09.215 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-29T08:17:09.268 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_3 2024-03-29T08:17:09.318 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-29T08:17:09.319 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T08:17:09.319 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-03-29T08:17:09.319 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T08:17:09.319 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-03-29 08:10:27.042208410 +0000 2024-03-29T08:17:09.319 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-03-29 08:10:26.798210591 +0000 2024-03-29T08:17:09.319 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-03-29 08:10:26.798210591 +0000 2024-03-29T08:17:09.319 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-03-29 08:10:26.798210591 +0000 2024-03-29T08:17:09.319 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-29T08:17:09.382 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-03-29T08:17:09.382 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-03-29T08:17:09.382 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000380512 s, 1.3 MB/s 2024-03-29T08:17:09.383 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-29T08:17:09.437 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_4 2024-03-29T08:17:09.487 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-29T08:17:09.487 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-29T08:17:09.487 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 936 Links: 1 2024-03-29T08:17:09.487 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-29T08:17:09.487 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-03-29 08:10:27.566203730 +0000 2024-03-29T08:17:09.487 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-03-29 08:10:27.306206054 +0000 2024-03-29T08:17:09.487 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-03-29 08:10:27.306206054 +0000 2024-03-29T08:17:09.487 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-03-29 08:10:27.306206054 +0000 2024-03-29T08:17:09.488 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-29T08:17:09.548 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-03-29T08:17:09.548 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-03-29T08:17:09.548 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000670248 s, 764 kB/s 2024-03-29T08:17:09.550 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-29T08:17:09.604 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T08:17:09.604 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi073.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-29T08:17:09.604 INFO:tasks.ceph:Generating config... 2024-03-29T08:17:09.605 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-29T08:17:09.605 INFO:tasks.ceph:[client] debug client = 20 2024-03-29T08:17:09.605 INFO:tasks.ceph:[client] debug ms = 1 2024-03-29T08:17:09.606 INFO:tasks.ceph:[client] pid file = 2024-03-29T08:17:09.606 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-29T08:17:09.606 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-29T08:17:09.606 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-29T08:17:09.606 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-29T08:17:09.606 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-29T08:17:09.606 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-29T08:17:09.606 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-29T08:17:09.606 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-29T08:17:09.606 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-29T08:17:09.606 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-29T08:17:09.606 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-29T08:17:09.606 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-29T08:17:09.606 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-29T08:17:09.607 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-29T08:17:09.607 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-29T08:17:09.607 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-29T08:17:09.607 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-29T08:17:09.607 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-29T08:17:09.607 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-29T08:17:09.607 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-29T08:17:09.607 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-29T08:17:09.607 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-29T08:17:09.607 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-29T08:17:09.607 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-29T08:17:09.607 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-29T08:17:09.607 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-29T08:17:09.607 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-29T08:17:09.607 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-29T08:17:09.607 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-29T08:17:09.608 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-29T08:17:09.608 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-29T08:17:09.608 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-29T08:17:09.608 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-29T08:17:09.608 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-29T08:17:09.608 INFO:tasks.ceph:Setting up mon.a... 2024-03-29T08:17:09.608 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-29T08:17:09.655 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.keyring 2024-03-29T08:17:09.670 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-29T08:17:09.721 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T08:17:09.741 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.46'), ('mon.b', '172.21.15.73'), ('mon.c', '[v2:172.21.15.73:3301,v1:172.21.15.73:6790]')] 2024-03-29T08:17:09.742 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.46,172.21.15.73,[v2:172.21.15.73:3301,v1:172.21.15.73:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-29T08:17:09.743 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:09.744 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-29T08:17:09.792 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.46 --add b 172.21.15.73 --addv c '[v2:172.21.15.73:3301,v1:172.21.15.73:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-29T08:17:09.883 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-29T08:17:09.883 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: generated fsid e9b7960f-ec1f-4787-95c3-c806094d37f0 2024-03-29T08:17:09.884 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = pacific 2024-03-29T08:17:09.884 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 0 2024-03-29T08:17:09.884 INFO:teuthology.orchestra.run.smithi046.stdout:fsid e9b7960f-ec1f-4787-95c3-c806094d37f0 2024-03-29T08:17:09.884 INFO:teuthology.orchestra.run.smithi046.stdout:last_changed 2024-03-29T08:17:09.884532+0000 2024-03-29T08:17:09.884 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-03-29T08:17:09.884532+0000 2024-03-29T08:17:09.884 INFO:teuthology.orchestra.run.smithi046.stdout:min_mon_release 16 (pacific) 2024-03-29T08:17:09.884 INFO:teuthology.orchestra.run.smithi046.stdout:election_strategy: 1 2024-03-29T08:17:09.884 INFO:teuthology.orchestra.run.smithi046.stdout:0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-03-29T08:17:09.884 INFO:teuthology.orchestra.run.smithi046.stdout:1: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.b 2024-03-29T08:17:09.884 INFO:teuthology.orchestra.run.smithi046.stdout:2: [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] mon.c 2024-03-29T08:17:09.884 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-29T08:17:09.887 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-29T08:17:09.936 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e9b7960f-ec1f-4787-95c3-c806094d37f0... 2024-03-29T08:17:09.936 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T08:17:09.983 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T08:17:10.023 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-03-29T08:17:10.024 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-03-29T08:17:10.024 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T08:17:10.024 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-03-29T08:17:10.024 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.024 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-03-29T08:17:10.024 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.024 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-03-29T08:17:10.024 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.024 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-03-29T08:17:10.024 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-03-29T08:17:10.024 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.024 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-03-29T08:17:10.025 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-03-29T08:17:10.025 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.025 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T08:17:10.025 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-03-29T08:17:10.025 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.025 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T08:17:10.025 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T08:17:10.025 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.025 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-03-29T08:17:10.025 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.025 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-03-29T08:17:10.025 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-03-29T08:17:10.025 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-03-29T08:17:10.025 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-03-29T08:17:10.026 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-03-29T08:17:10.026 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-03-29T08:17:10.026 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-03-29T08:17:10.026 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T08:17:10.026 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T08:17:10.026 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-03-29T08:17:10.026 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.026 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T08:17:10.026 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.026 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-03-29T08:17:10.026 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.026 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-03-29T08:17:10.026 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.026 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-03-29T08:17:10.027 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-03-29T08:17:10.027 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-03-29T08:17:10.027 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.73,[v2:172.21.15.73:3301,v1:172.21.15.73:6790]" 2024-03-29T08:17:10.027 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = e9b7960f-ec1f-4787-95c3-c806094d37f0 2024-03-29T08:17:10.027 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.027 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-03-29T08:17:10.027 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-03-29T08:17:10.027 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.027 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-03-29T08:17:10.027 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-03-29T08:17:10.027 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-03-29T08:17:10.027 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.027 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-03-29T08:17:10.027 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-03-29T08:17:10.027 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.027 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-03-29T08:17:10.028 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-03-29T08:17:10.028 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-03-29T08:17:10.028 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.028 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-03-29T08:17:10.028 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-03-29T08:17:10.028 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.028 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-03-29T08:17:10.028 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.028 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-03-29T08:17:10.028 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.028 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-03-29T08:17:10.028 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.028 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-03-29T08:17:10.028 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-03-29T08:17:10.028 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.028 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-03-29T08:17:10.029 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-03-29T08:17:10.029 INFO:teuthology.orchestra.run.smithi046.stdout: bdev async discard = True 2024-03-29T08:17:10.029 INFO:teuthology.orchestra.run.smithi046.stdout: bdev enable discard = True 2024-03-29T08:17:10.029 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore allocator = bitmap 2024-03-29T08:17:10.029 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-03-29T08:17:10.029 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-03-29T08:17:10.029 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 1/20 2024-03-29T08:17:10.029 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 1/20 2024-03-29T08:17:10.029 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:17:10.029 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-03-29T08:17:10.029 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 4/10 2024-03-29T08:17:10.029 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T08:17:10.029 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-03-29T08:17:10.029 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-03-29T08:17:10.029 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-03-29T08:17:10.029 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-03-29T08:17:10.030 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-03-29T08:17:10.030 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-03-29T08:17:10.030 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:17:10.030 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-03-29T08:17:10.030 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-03-29T08:17:10.030 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-03-29T08:17:10.030 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:17:10.030 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:17:10.030 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-03-29T08:17:10.030 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-03-29T08:17:10.030 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:17:10.030 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-03-29T08:17:10.030 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-03-29T08:17:10.030 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-03-29T08:17:10.031 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-03-29T08:17:10.031 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-03-29T08:17:10.031 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:17:10.031 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-03-29T08:17:10.031 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-03-29T08:17:10.031 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:17:10.031 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.031 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T08:17:10.031 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T08:17:10.031 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T08:17:10.031 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.031 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T08:17:10.031 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T08:17:10.031 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T08:17:10.031 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.031 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-03-29T08:17:10.031 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-03-29T08:17:10.032 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.032 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-03-29T08:17:10.032 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2024-03-29T08:17:10.032 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:10.032 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-03-29T08:17:10.032 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-03-29T08:17:10.032 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-03-29T08:17:10.032 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-03-29T08:17:10.032 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T08:17:10.032 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T08:17:10.032 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2024-03-29T08:17:10.032 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-03-29T08:17:10.032 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:17:10.032 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2024-03-29T08:17:10.032 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-03-29T08:17:10.032 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-03-29T08:17:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-03-29T08:17:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-03-29T08:17:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-03-29T08:17:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-03-29T08:17:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-03-29T08:17:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-03-29T08:17:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-03-29T08:17:10.033 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2024-03-29T08:17:10.033 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2024-03-29T08:17:10.033 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2024-03-29T08:17:10.033 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:17:10.033 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2024-03-29T08:17:10.033 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = True 2024-03-29T08:17:10.033 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2024-03-29T08:17:10.034 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = True 2024-03-29T08:17:10.034 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-03-29T08:17:10.034 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-03-29T08:17:10.034 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-03-29T08:17:10.034 INFO:teuthology.orchestra.run.smithi073.stdout:[global] 2024-03-29T08:17:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: chdir = "" 2024-03-29T08:17:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T08:17:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: auth supported = cephx 2024-03-29T08:17:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: filestore xattr use omap = true 2024-03-29T08:17:10.034 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: mon clock drift allowed = 1.000 2024-03-29T08:17:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: osd crush chooseleaf type = 0 2024-03-29T08:17:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: auth debug = true 2024-03-29T08:17:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on old message = true 2024-03-29T08:17:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on bug = true 2024-03-29T08:17:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T08:17:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: mon pg warn max object skew = 0 2024-03-29T08:17:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T08:17:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T08:17:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.035 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default size = 2 2024-03-29T08:17:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow primary affinity = true 2024-03-29T08:17:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow pg remap = true 2024-03-29T08:17:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on legacy crush tunables = false 2024-03-29T08:17:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on crush straw calc version zero = false 2024-03-29T08:17:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on no sortbitwise = false 2024-03-29T08:17:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on osd down out interval zero = false 2024-03-29T08:17:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on too few osds = false 2024-03-29T08:17:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T08:17:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T08:17:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: mon_allow_pool_size_one = true 2024-03-29T08:17:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T08:17:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: osd default data pool replay window = 5 2024-03-29T08:17:10.036 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: mon allow pool delete = true 2024-03-29T08:17:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: mon cluster log file level = debug 2024-03-29T08:17:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: debug asserts on shutdown = true 2024-03-29T08:17:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: mon health detail to clog = false 2024-03-29T08:17:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: mon host = "172.21.15.46,172.21.15.73,[v2:172.21.15.73:3301,v1:172.21.15.73:6790]" 2024-03-29T08:17:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: fsid = e9b7960f-ec1f-4787-95c3-c806094d37f0 2024-03-29T08:17:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.037 INFO:teuthology.orchestra.run.smithi073.stdout:[osd] 2024-03-29T08:17:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: osd journal size = 100 2024-03-29T08:17:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub load threshold = 5.0 2024-03-29T08:17:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub max interval = 600 2024-03-29T08:17:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: osd mclock profile = high_recovery_ops 2024-03-29T08:17:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: osd recover clone overlap = true 2024-03-29T08:17:10.037 INFO:teuthology.orchestra.run.smithi073.stdout: osd recovery max chunk = 1048576 2024-03-29T08:17:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug shutdown = true 2024-03-29T08:17:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug op order = true 2024-03-29T08:17:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify stray on activate = true 2024-03-29T08:17:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: osd open classes on start = true 2024-03-29T08:17:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug pg log writeout = true 2024-03-29T08:17:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: osd deep scrub update digest min age = 30 2024-03-29T08:17:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: osd map max advance = 10 2024-03-29T08:17:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: journal zero on create = true 2024-03-29T08:17:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: filestore ondisk finisher threads = 3 2024-03-29T08:17:10.038 INFO:teuthology.orchestra.run.smithi073.stdout: filestore apply finisher threads = 3 2024-03-29T08:17:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: bdev debug aio = true 2024-03-29T08:17:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug misdirected ops = true 2024-03-29T08:17:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: bdev async discard = True 2024-03-29T08:17:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: bdev enable discard = True 2024-03-29T08:17:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore allocator = bitmap 2024-03-29T08:17:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore block size = 96636764160 2024-03-29T08:17:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore fsck on mount = True 2024-03-29T08:17:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluefs = 1/20 2024-03-29T08:17:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluestore = 1/20 2024-03-29T08:17:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:17:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: debug osd = 20 2024-03-29T08:17:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: debug rocksdb = 4/10 2024-03-29T08:17:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T08:17:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd full ratio = 0.9 2024-03-29T08:17:10.039 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd nearfull ratio = 0.8 2024-03-29T08:17:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: osd failsafe full ratio = 0.95 2024-03-29T08:17:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: osd objectstore = bluestore 2024-03-29T08:17:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: osd op complaint time = 180 2024-03-29T08:17:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.040 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr] 2024-03-29T08:17:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:17:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: debug mgr = 20 2024-03-29T08:17:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-03-29T08:17:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-03-29T08:17:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:17:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:17:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: mgr/telemetry/nag = false 2024-03-29T08:17:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.040 INFO:teuthology.orchestra.run.smithi073.stdout:[mon] 2024-03-29T08:17:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:17:10.040 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-03-29T08:17:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: debug paxos = 20 2024-03-29T08:17:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-03-29T08:17:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: mon data avail warn = 5 2024-03-29T08:17:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: mon mgr mkfs grace = 240 2024-03-29T08:17:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:17:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd reporter subtree level = osd 2024-03-29T08:17:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd prime pg temp = true 2024-03-29T08:17:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:17:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T08:17:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T08:17:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T08:17:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T08:17:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T08:17:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T08:17:10.041 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: # 1m isn't quite enough 2024-03-29T08:17:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: mon_down_mkfs_grace = 2m 2024-03-29T08:17:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_filestore_osds = false 2024-03-29T08:17:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: mon op complaint time = 120 2024-03-29T08:17:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:17:10.042 INFO:teuthology.orchestra.run.smithi073.stdout:[client] 2024-03-29T08:17:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: rgw cache enabled = true 2024-03-29T08:17:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable ops log = true 2024-03-29T08:17:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable usage log = true 2024-03-29T08:17:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T08:17:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T08:17:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: client mount timeout = 600 2024-03-29T08:17:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: debug client = 20 2024-03-29T08:17:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:17:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = "" 2024-03-29T08:17:10.042 INFO:teuthology.orchestra.run.smithi073.stdout: rados mon op timeout = 900 2024-03-29T08:17:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: rados osd op timeout = 900 2024-03-29T08:17:10.043 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.a] 2024-03-29T08:17:10.043 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.b] 2024-03-29T08:17:10.043 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.c] 2024-03-29T08:17:10.043 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.a] 2024-03-29T08:17:10.043 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.c] 2024-03-29T08:17:10.043 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.b] 2024-03-29T08:17:10.043 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.d] 2024-03-29T08:17:10.043 INFO:teuthology.orchestra.run.smithi073.stdout:[mds] 2024-03-29T08:17:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: debug mds = 20 2024-03-29T08:17:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: debug mds balancer = 20 2024-03-29T08:17:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:17:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: mds debug frag = True 2024-03-29T08:17:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: mds debug scatterstat = True 2024-03-29T08:17:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: mds op complaint time = 180 2024-03-29T08:17:10.043 INFO:teuthology.orchestra.run.smithi073.stdout: mds verify scatter = True 2024-03-29T08:17:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: osd op complaint time = 180 2024-03-29T08:17:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: rados mon op timeout = 900 2024-03-29T08:17:10.044 INFO:teuthology.orchestra.run.smithi073.stdout: rados osd op timeout = 900 2024-03-29T08:17:10.045 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-29T08:17:10.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-29T08:17:10.145 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-29T08:17:10.145 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:10.145 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-29T08:17:10.154 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:10.154 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-29T08:17:10.204 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:17:10.204 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:10.204 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-29T08:17:10.204 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T08:17:10.280 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:10.280 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-29T08:17:10.331 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:17:10.332 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:10.332 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-29T08:17:10.332 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-29T08:17:10.367 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:10.367 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-29T08:17:10.418 INFO:tasks.ceph:Setting up mon nodes... 2024-03-29T08:17:10.418 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-29T08:17:10.419 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-29T08:17:10.475 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-29T08:17:10.480 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-29T08:17:10.539 INFO:teuthology.orchestra.run.smithi073.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-29T08:17:10.560 INFO:tasks.ceph:Setting up mds nodes... 2024-03-29T08:17:10.560 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-29T08:17:10.616 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-29T08:17:10.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-29T08:17:10.641 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-03-29T08:17:10.738 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-03-29T08:17:10.742 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-03-29T08:17:10.762 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-29T08:17:10.820 INFO:teuthology.orchestra.run.smithi073.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-29T08:17:10.825 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-29T08:17:10.846 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-03-29T08:17:10.944 INFO:teuthology.orchestra.run.smithi073.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-03-29T08:17:10.948 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-03-29T08:17:10.968 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-29T08:17:10.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-29T08:17:11.010 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-29T08:17:11.028 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-03-29T08:17:11.110 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-03-29T08:17:11.127 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-29T08:17:11.168 INFO:teuthology.orchestra.run.smithi073.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-03-29T08:17:11.185 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-03-29T08:17:11.270 INFO:teuthology.orchestra.run.smithi073.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-03-29T08:17:11.287 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-29T08:17:11.287 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi073.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-29T08:17:11.287 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-29T08:17:11.308 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T08:17:11.308 INFO:tasks.ceph:role: osd.0 2024-03-29T08:17:11.308 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:17:11.308 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-29T08:17:11.375 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T08:17:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T08:17:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T08:17:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T08:17:11.375 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T08:17:11.375 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-03-29T08:17:11.375 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T08:17:11.375 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T08:17:11.376 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T08:17:11.376 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T08:17:11.399 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-03-29T08:17:11.402 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-03-29T08:17:11.402 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-29T08:17:11.474 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-29T08:17:11.531 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-03-29T08:17:11.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:17:11.532 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-29T08:17:11.599 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T08:17:11.599 INFO:tasks.ceph:role: osd.1 2024-03-29T08:17:11.599 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:17:11.599 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-29T08:17:11.659 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T08:17:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T08:17:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T08:17:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T08:17:11.659 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T08:17:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-03-29T08:17:11.659 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T08:17:11.659 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T08:17:11.659 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T08:17:11.659 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T08:17:11.683 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-03-29T08:17:11.686 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-03-29T08:17:11.686 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-29T08:17:11.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-29T08:17:11.811 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-03-29T08:17:11.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:17:11.812 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-29T08:17:11.879 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T08:17:11.879 INFO:tasks.ceph:role: osd.2 2024-03-29T08:17:11.879 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:17:11.879 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-29T08:17:11.939 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T08:17:11.939 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T08:17:11.939 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T08:17:11.939 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T08:17:11.939 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T08:17:11.939 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-03-29T08:17:11.939 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T08:17:11.939 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T08:17:11.939 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T08:17:11.939 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T08:17:11.964 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-03-29T08:17:11.966 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-03-29T08:17:11.966 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-29T08:17:12.036 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-29T08:17:12.096 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-03-29T08:17:12.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:17:12.097 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-29T08:17:12.159 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-29T08:17:12.159 INFO:tasks.ceph:role: osd.3 2024-03-29T08:17:12.159 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:17:12.159 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-29T08:17:12.223 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T08:17:12.223 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T08:17:12.223 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T08:17:12.223 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T08:17:12.223 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T08:17:12.223 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-03-29T08:17:12.223 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T08:17:12.224 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T08:17:12.224 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T08:17:12.224 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T08:17:12.250 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-03-29T08:17:12.252 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-03-29T08:17:12.252 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-29T08:17:12.320 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-29T08:17:12.379 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2024-03-29T08:17:12.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:17:12.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T08:17:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:12.509+0000 7f26a4e2f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-29T08:17:12.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:12.509+0000 7f26a4e2f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-29T08:17:12.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:12.509+0000 7f26a4e2f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T08:17:12.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:12.509+0000 7f26a4e2f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T08:17:12.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:12.509+0000 7f26a4e2f640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-29T08:17:12.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:12.513+0000 7f26a4e2f640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-29T08:17:15.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-29T08:17:15.519 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T08:17:15.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:15.616+0000 7f6536f55640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-29T08:17:15.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:15.616+0000 7f6536f55640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-29T08:17:15.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:15.616+0000 7f6536f55640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T08:17:15.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:15.616+0000 7f6536f55640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T08:17:15.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:15.616+0000 7f6536f55640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-29T08:17:15.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:15.620+0000 7f6536f55640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-29T08:17:18.600 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-29T08:17:18.622 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T08:17:18.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:18.716+0000 7f1c00b4a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-29T08:17:18.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:18.720+0000 7f1c00b4a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-29T08:17:18.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:18.720+0000 7f1c00b4a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T08:17:18.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:18.720+0000 7f1c00b4a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T08:17:18.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:18.720+0000 7f1c00b4a640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-29T08:17:18.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:18.720+0000 7f1c00b4a640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-29T08:17:21.682 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-29T08:17:21.703 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T08:17:21.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:21.796+0000 7f97ba62b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-29T08:17:21.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:21.796+0000 7f97ba62b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-29T08:17:21.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:21.796+0000 7f97ba62b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T08:17:21.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:21.796+0000 7f97ba62b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T08:17:21.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:21.796+0000 7f97ba62b640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-29T08:17:21.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:21.800+0000 7f97ba62b640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-29T08:17:24.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-29T08:17:24.790 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-29T08:17:24.812 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T08:17:24.812 INFO:tasks.ceph:role: osd.4 2024-03-29T08:17:24.812 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:17:24.812 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-29T08:17:24.882 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T08:17:24.882 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T08:17:24.882 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T08:17:24.882 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T08:17:24.882 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T08:17:24.883 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-03-29T08:17:24.883 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T08:17:24.883 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T08:17:24.883 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T08:17:24.883 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T08:17:25.342 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-03-29T08:17:25.346 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-03-29T08:17:25.346 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-29T08:17:25.452 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-29T08:17:25.469 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: /sbin/restorecon: command not found 2024-03-29T08:17:25.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:17:25.470 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-29T08:17:25.533 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T08:17:25.533 INFO:tasks.ceph:role: osd.5 2024-03-29T08:17:25.533 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:17:25.533 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-29T08:17:25.601 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T08:17:25.601 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T08:17:25.601 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T08:17:25.601 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T08:17:25.602 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T08:17:25.602 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-03-29T08:17:25.602 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T08:17:25.602 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T08:17:25.602 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T08:17:25.602 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T08:17:26.097 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-03-29T08:17:26.100 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-03-29T08:17:26.100 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-29T08:17:26.262 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-29T08:17:26.280 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: /sbin/restorecon: command not found 2024-03-29T08:17:26.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:17:26.281 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-29T08:17:26.341 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T08:17:26.341 INFO:tasks.ceph:role: osd.6 2024-03-29T08:17:26.341 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:17:26.342 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-29T08:17:26.414 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T08:17:26.414 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T08:17:26.414 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T08:17:26.414 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T08:17:26.414 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T08:17:26.414 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-03-29T08:17:26.415 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T08:17:26.415 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T08:17:26.415 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T08:17:26.415 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T08:17:26.911 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-03-29T08:17:26.914 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-03-29T08:17:26.914 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-29T08:17:27.030 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-29T08:17:27.047 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: /sbin/restorecon: command not found 2024-03-29T08:17:27.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:17:27.049 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-29T08:17:27.114 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-29T08:17:27.114 INFO:tasks.ceph:role: osd.7 2024-03-29T08:17:27.114 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:17:27.114 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-29T08:17:27.185 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-29T08:17:27.185 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-29T08:17:27.186 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-29T08:17:27.186 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-29T08:17:27.186 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-29T08:17:27.186 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-03-29T08:17:27.186 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-29T08:17:27.186 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-29T08:17:27.186 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-29T08:17:27.186 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-29T08:17:27.634 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-03-29T08:17:27.638 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-03-29T08:17:27.638 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-29T08:17:27.755 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-29T08:17:27.773 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: /sbin/restorecon: command not found 2024-03-29T08:17:27.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:17:27.774 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T08:17:27.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:27.901+0000 7f8dcb3c6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-29T08:17:27.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:27.905+0000 7f8dcb3c6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-29T08:17:27.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:27.905+0000 7f8dcb3c6640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T08:17:27.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:27.905+0000 7f8dcb3c6640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T08:17:27.905 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:27.905+0000 7f8dcb3c6640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-29T08:17:27.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:27.909+0000 7f8dcb3c6640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-29T08:17:30.896 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-29T08:17:30.917 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T08:17:31.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:31.013+0000 7f4453ba7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-29T08:17:31.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:31.013+0000 7f4453ba7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-29T08:17:31.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:31.013+0000 7f4453ba7640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T08:17:31.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:31.013+0000 7f4453ba7640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T08:17:31.012 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:31.013+0000 7f4453ba7640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-29T08:17:31.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:31.013+0000 7f4453ba7640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-29T08:17:33.985 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-29T08:17:34.006 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T08:17:34.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:34.096+0000 7efe9c42b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-29T08:17:34.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:34.096+0000 7efe9c42b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-29T08:17:34.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:34.096+0000 7efe9c42b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T08:17:34.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:34.096+0000 7efe9c42b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T08:17:34.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:34.096+0000 7efe9c42b640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-29T08:17:34.097 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:34.096+0000 7efe9c42b640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-29T08:17:37.076 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-29T08:17:37.096 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-29T08:17:37.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:37.188+0000 7f0496e2b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-29T08:17:37.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:37.188+0000 7f0496e2b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-29T08:17:37.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:37.188+0000 7f0496e2b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T08:17:37.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:37.188+0000 7f0496e2b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T08:17:37.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:37.188+0000 7f0496e2b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-29T08:17:37.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:37.188+0000 7f0496e2b640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-29T08:17:38.106 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:17:38.110 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:17:40.173 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-29T08:17:40.192 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-29T08:17:40.193 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:40.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-29T08:17:40.215 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:40.215 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-29T08:17:40.281 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:40.281 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-03-29T08:17:40.348 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:40.349 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-29T08:17:40.416 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:40.416 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-29T08:17:40.484 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:40.485 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-29T08:17:40.548 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:40.548 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-29T08:17:40.612 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:40.612 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-29T08:17:40.633 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:40.633 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-29T08:17:40.693 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:40.694 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-03-29T08:17:40.753 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:40.753 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-29T08:17:40.813 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:40.813 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-29T08:17:40.873 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:40.873 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-29T08:17:40.933 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:40.933 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-29T08:17:40.993 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:40.993 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-29T08:17:41.002 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:41.002 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-03-29T08:17:41.052 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:41.052 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-03-29T08:17:41.060 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:41.060 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-03-29T08:17:41.114 INFO:tasks.ceph:Adding keys to all mons... 2024-03-29T08:17:41.114 DEBUG:teuthology.orchestra.run.smithi046:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-29T08:17:41.117 DEBUG:teuthology.orchestra.run.smithi073:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-29T08:17:41.158 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.y] 2024-03-29T08:17:41.158 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAGeQZm72tiHBAA2N7qQnR5Ehu/7nP2WpklGg== 2024-03-29T08:17:41.158 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-03-29T08:17:41.158 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAGeQZmlnXNJBAA7lI2lCBd0PeZudPsQyMOUQ== 2024-03-29T08:17:41.158 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-03-29T08:17:41.158 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAGeQZmkCUVLBAAxcAOWGBDUCpEHeg/1OeQzA== 2024-03-29T08:17:41.158 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.0] 2024-03-29T08:17:41.159 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAIeQZmTHV8HhAAODJ+RP1Bx6k0D1153XgI0Q== 2024-03-29T08:17:41.159 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.1] 2024-03-29T08:17:41.159 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQALeQZmZRD0JBAAjU6W/21CeqOMdIP5969iNA== 2024-03-29T08:17:41.159 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.2] 2024-03-29T08:17:41.159 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAOeQZmSSP+KhAA26V8CByUJNfkVcG37AdlZQ== 2024-03-29T08:17:41.159 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.3] 2024-03-29T08:17:41.159 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAReQZmb/e8LxAAV4yC1xlLwh8OAqOSPmcyyA== 2024-03-29T08:17:41.159 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.x] 2024-03-29T08:17:41.159 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAGeQZmyrJXIBAAv535Rz99UI3V7JvvzQ8oSw== 2024-03-29T08:17:41.159 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-03-29T08:17:41.159 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAGeQZmbPkcMRAADnwJE17uILjJo14jghJuzg== 2024-03-29T08:17:41.159 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-03-29T08:17:41.159 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAGeQZm7L11OBAAZ3fpXtEIYG1tGm9ZFswZ5Q== 2024-03-29T08:17:41.160 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.4] 2024-03-29T08:17:41.160 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAXeQZmrLnzNRAAFPYQRs9sBNNKeyvcoI9rSg== 2024-03-29T08:17:41.160 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.5] 2024-03-29T08:17:41.160 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAbeQZmjTPbABAAi4qTU5CBy9LtSSRf7cmfwg== 2024-03-29T08:17:41.160 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.6] 2024-03-29T08:17:41.160 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAeeQZmo13eBRAAWY2VVBakLze9oA4ynp1rNA== 2024-03-29T08:17:41.160 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.7] 2024-03-29T08:17:41.160 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAheQZmghpZCxAAdy5xI7qPwZb8LN8Rn6Ndaw== 2024-03-29T08:17:41.160 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-03-29T08:17:41.160 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAHeQZmyk22ABAA8Pn3OaIvxMQSLrKIM8FP4w== 2024-03-29T08:17:41.160 INFO:teuthology.orchestra.run.smithi046.stdout:[client.2] 2024-03-29T08:17:41.160 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAHeQZmhsqgBhAApqnO5qMG8VP27j9TfbZ8uw== 2024-03-29T08:17:41.160 INFO:teuthology.orchestra.run.smithi046.stdout:[client.1] 2024-03-29T08:17:41.160 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAHeQZmG/E8ChAAJTcJ2j5xn0oUsmlV2trWdw== 2024-03-29T08:17:41.161 INFO:teuthology.orchestra.run.smithi046.stdout:[client.3] 2024-03-29T08:17:41.161 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAHeQZmatlQEBAAAGKuGnAh4YXSk91TAzomXA== 2024-03-29T08:17:41.171 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr.y] 2024-03-29T08:17:41.171 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAGeQZm72tiHBAA2N7qQnR5Ehu/7nP2WpklGg== 2024-03-29T08:17:41.171 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.a] 2024-03-29T08:17:41.172 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAGeQZmlnXNJBAA7lI2lCBd0PeZudPsQyMOUQ== 2024-03-29T08:17:41.172 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.c] 2024-03-29T08:17:41.172 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAGeQZmkCUVLBAAxcAOWGBDUCpEHeg/1OeQzA== 2024-03-29T08:17:41.172 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.0] 2024-03-29T08:17:41.172 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAIeQZmTHV8HhAAODJ+RP1Bx6k0D1153XgI0Q== 2024-03-29T08:17:41.172 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.1] 2024-03-29T08:17:41.172 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQALeQZmZRD0JBAAjU6W/21CeqOMdIP5969iNA== 2024-03-29T08:17:41.172 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.2] 2024-03-29T08:17:41.172 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAOeQZmSSP+KhAA26V8CByUJNfkVcG37AdlZQ== 2024-03-29T08:17:41.172 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.3] 2024-03-29T08:17:41.172 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAReQZmb/e8LxAAV4yC1xlLwh8OAqOSPmcyyA== 2024-03-29T08:17:41.172 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr.x] 2024-03-29T08:17:41.172 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAGeQZmyrJXIBAAv535Rz99UI3V7JvvzQ8oSw== 2024-03-29T08:17:41.172 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.b] 2024-03-29T08:17:41.173 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAGeQZmbPkcMRAADnwJE17uILjJo14jghJuzg== 2024-03-29T08:17:41.173 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.d] 2024-03-29T08:17:41.173 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAGeQZm7L11OBAAZ3fpXtEIYG1tGm9ZFswZ5Q== 2024-03-29T08:17:41.173 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.4] 2024-03-29T08:17:41.173 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAXeQZmrLnzNRAAFPYQRs9sBNNKeyvcoI9rSg== 2024-03-29T08:17:41.173 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.5] 2024-03-29T08:17:41.173 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAbeQZmjTPbABAAi4qTU5CBy9LtSSRf7cmfwg== 2024-03-29T08:17:41.173 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.6] 2024-03-29T08:17:41.173 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAeeQZmo13eBRAAWY2VVBakLze9oA4ynp1rNA== 2024-03-29T08:17:41.173 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.7] 2024-03-29T08:17:41.173 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAheQZmghpZCxAAdy5xI7qPwZb8LN8Rn6Ndaw== 2024-03-29T08:17:41.173 INFO:teuthology.orchestra.run.smithi073.stdout:[client.0] 2024-03-29T08:17:41.173 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAHeQZmyk22ABAA8Pn3OaIvxMQSLrKIM8FP4w== 2024-03-29T08:17:41.173 INFO:teuthology.orchestra.run.smithi073.stdout:[client.2] 2024-03-29T08:17:41.173 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAHeQZmhsqgBhAApqnO5qMG8VP27j9TfbZ8uw== 2024-03-29T08:17:41.174 INFO:teuthology.orchestra.run.smithi073.stdout:[client.1] 2024-03-29T08:17:41.174 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAHeQZmG/E8ChAAJTcJ2j5xn0oUsmlV2trWdw== 2024-03-29T08:17:41.174 INFO:teuthology.orchestra.run.smithi073.stdout:[client.3] 2024-03-29T08:17:41.174 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAHeQZmatlQEBAAAGKuGnAh4YXSk91TAzomXA== 2024-03-29T08:17:41.175 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T08:17:41.206 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T08:17:41.271 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T08:17:41.274 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T08:17:41.331 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T08:17:41.334 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T08:17:41.389 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T08:17:41.393 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T08:17:41.445 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T08:17:41.448 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T08:17:41.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T08:17:41.503 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T08:17:41.556 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T08:17:41.559 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T08:17:41.611 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T08:17:41.614 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-29T08:17:41.667 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T08:17:41.670 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T08:17:41.726 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T08:17:41.730 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-29T08:17:41.785 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T08:17:41.789 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T08:17:41.841 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T08:17:41.844 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T08:17:41.897 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T08:17:41.900 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T08:17:41.954 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T08:17:41.957 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-29T08:17:42.010 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T08:17:42.013 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T08:17:42.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T08:17:42.071 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T08:17:42.127 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T08:17:42.130 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T08:17:42.186 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T08:17:42.189 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-29T08:17:42.245 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-29T08:17:42.246 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-29T08:17:42.266 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T08:17:42.374 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-29T08:17:42.394 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-29T08:17:42.414 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T08:17:42.521 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-29T08:17:42.540 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-29T08:17:42.601 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-29T08:17:42.706 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-29T08:17:42.725 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-29T08:17:42.728 DEBUG:teuthology.orchestra.run.smithi073:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-29T08:17:42.774 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-29T08:17:42.774 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-29T08:17:42.774 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-29T08:17:42.778 INFO:tasks.ceph.mon.a:Started 2024-03-29T08:17:42.778 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-29T08:17:42.778 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-29T08:17:42.821 INFO:tasks.ceph.mon.b:Started 2024-03-29T08:17:42.821 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-29T08:17:42.821 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-29T08:17:42.824 INFO:tasks.ceph.mon.c:Started 2024-03-29T08:17:42.824 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-29T08:17:42.824 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-29T08:17:42.824 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-29T08:17:42.827 INFO:tasks.ceph.mgr.y:Started 2024-03-29T08:17:42.827 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-29T08:17:42.828 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-29T08:17:42.830 INFO:tasks.ceph.mgr.x:Started 2024-03-29T08:17:42.831 INFO:tasks.ceph:Setting crush tunables to default 2024-03-29T08:17:42.831 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd crush tunables default 2024-03-29T08:17:42.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.969+0000 7f415a800640 1 Processor -- start 2024-03-29T08:17:42.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.969+0000 7f415a800640 1 -- start start 2024-03-29T08:17:42.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.973+0000 7f415a800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 0x7f41540b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:42.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.973+0000 7f415a800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 0x7f415415e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:42.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.973+0000 7f415a800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4154099950 con 0x7f41540b3ac0 2024-03-29T08:17:42.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.973+0000 7f415a800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4154099c30 con 0x7f415415de20 2024-03-29T08:17:42.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.973+0000 7f415a800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41540cb590 con 0x7f4154099450 2024-03-29T08:17:42.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.973+0000 7f414be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f41540b3ac0 0x7f41540cbd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39260/0 (socket says 172.21.15.46:39260) 2024-03-29T08:17:42.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.973+0000 7f414be00640 1 -- 172.21.15.46:0/3647629790 learned_addr learned my addr 172.21.15.46:0/3647629790 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:17:42.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.973+0000 7f4153e00640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 0x7f415415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:42.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.973+0000 7f4153e00640 1 -- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 msgr2=0x7f415415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-29T08:17:42.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.973+0000 7f4153e00640 1 -- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 msgr2=0x7f415415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T08:17:42.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.973+0000 7f4153e00640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 0x7f415415e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T08:17:42.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.973+0000 7f4153e00640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 0x7f415415e740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T08:17:42.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.973+0000 7f4153400640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 0x7f41540b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:42.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.973+0000 7f4153400640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 msgr2=0x7f41540b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T08:17:42.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.973+0000 7f4153400640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 msgr2=0x7f41540b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T08:17:42.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.973+0000 7f4153400640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 0x7f41540b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T08:17:42.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:42.973+0000 7f4153400640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 0x7f41540b3580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T08:17:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.173+0000 7f4153e00640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 0x7f415415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.173+0000 7f4153400640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 0x7f41540b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.173+0000 7f4153e00640 1 -- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 msgr2=0x7f415415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T08:17:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.173+0000 7f4153e00640 1 -- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 msgr2=0x7f415415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T08:17:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.173+0000 7f4153e00640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 0x7f415415e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T08:17:43.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.173+0000 7f4153e00640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 0x7f415415e740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T08:17:43.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.173+0000 7f4153400640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 msgr2=0x7f41540b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-29T08:17:43.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.173+0000 7f4153400640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 msgr2=0x7f41540b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T08:17:43.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.173+0000 7f4153400640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 0x7f41540b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T08:17:43.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.173+0000 7f4153400640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 0x7f41540b3580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T08:17:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.573+0000 7f4153e00640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 0x7f415415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.573+0000 7f4153400640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 0x7f41540b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:43.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.577+0000 7f4153e00640 1 -- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 msgr2=0x7f415415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T08:17:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.577+0000 7f4153e00640 1 -- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 msgr2=0x7f415415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T08:17:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.577+0000 7f4153e00640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 0x7f415415e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T08:17:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.577+0000 7f4153e00640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 0x7f415415e740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-29T08:17:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.577+0000 7f4153400640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 msgr2=0x7f41540b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-29T08:17:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.577+0000 7f4153400640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 msgr2=0x7f41540b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T08:17:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.577+0000 7f4153400640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 0x7f41540b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T08:17:43.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:43.577+0000 7f4153400640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 0x7f41540b3580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-29T08:17:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:44.377+0000 7f4153400640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 0x7f41540b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:44.377+0000 7f4153e00640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 0x7f415415e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:44.377+0000 7f4153e00640 1 -- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 msgr2=0x7f415415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-29T08:17:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:44.377+0000 7f4153e00640 1 -- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 msgr2=0x7f415415e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T08:17:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:44.377+0000 7f4153400640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 msgr2=0x7f41540b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-29T08:17:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:44.377+0000 7f4153e00640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 0x7f415415e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T08:17:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:44.377+0000 7f4153400640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 msgr2=0x7f41540b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T08:17:44.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:44.377+0000 7f4153400640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 0x7f41540b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T08:17:44.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:44.377+0000 7f4153400640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 0x7f41540b3580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-29T08:17:44.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:44.377+0000 7f4153e00640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 0x7f415415e740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-29T08:17:45.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:45.973+0000 7f4151600640 1 -- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 msgr2=0x7f415415e740 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T08:17:45.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:45.973+0000 7f4151600640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4154099450 0x7f415415e740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:45.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:45.973+0000 7f4151600640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 msgr2=0x7f41540b3580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T08:17:45.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:45.973+0000 7f4151600640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.73:3300/0 conn(0x7f415415de20 0x7f41540b3580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:45.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:45.973+0000 7f4151600640 1 -- 172.21.15.46:0/3647629790 >> v1:172.21.15.46:6789/0 conn(0x7f41540b3ac0 legacy=0x7f41540cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:45.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:45.973+0000 7f4151600640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 0x7f4130009820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:45.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:45.973+0000 7f4151600640 1 -- 172.21.15.46:0/3647629790 --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41540cb870 con 0x7f41300036f0 2024-03-29T08:17:45.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:45.973+0000 7f4151600640 1 -- 172.21.15.46:0/3647629790 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4154099c30 con 0x7f4130007360 2024-03-29T08:17:45.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:45.973+0000 7f414be00640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 0x7f4130009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:45.973+0000 7f414be00640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 msgr2=0x7f4130009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-29T08:17:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:45.973+0000 7f414be00640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 msgr2=0x7f4130009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T08:17:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:45.973+0000 7f414be00640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 0x7f4130009820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T08:17:45.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:45.973+0000 7f414be00640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 0x7f4130009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-29T08:17:46.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:46.173+0000 7f414be00640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 0x7f4130009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:46.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:46.173+0000 7f414be00640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 msgr2=0x7f4130009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-29T08:17:46.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:46.173+0000 7f414be00640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 msgr2=0x7f4130009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T08:17:46.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:46.173+0000 7f414be00640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 0x7f4130009820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T08:17:46.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:46.173+0000 7f414be00640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 0x7f4130009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-29T08:17:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:46.573+0000 7f414be00640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 0x7f4130009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:46.573+0000 7f414be00640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 msgr2=0x7f4130009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-29T08:17:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:46.573+0000 7f414be00640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 msgr2=0x7f4130009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T08:17:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:46.573+0000 7f414be00640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 0x7f4130009820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T08:17:46.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:46.573+0000 7f414be00640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 0x7f4130009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-29T08:17:47.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:47.377+0000 7f414be00640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 0x7f4130009820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:47.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:47.377+0000 7f414be00640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 msgr2=0x7f4130009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-29T08:17:47.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:47.377+0000 7f414be00640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 msgr2=0x7f4130009820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-29T08:17:47.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:47.377+0000 7f414be00640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 0x7f4130009820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-29T08:17:47.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:47.377+0000 7f414be00640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 0x7f4130009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-29T08:17:48.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.129+0000 7f4152a00640 1 -- 172.21.15.46:0/3647629790 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498954849 0 0) 0x7f41540cb870 con 0x7f41300036f0 2024-03-29T08:17:48.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.129+0000 7f4152a00640 1 -- 172.21.15.46:0/3647629790 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4138003530 con 0x7f41300036f0 2024-03-29T08:17:48.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4152a00640 1 -- 172.21.15.46:0/3647629790 <== mon.1 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2025010322 0 0) 0x7f4138003530 con 0x7f41300036f0 2024-03-29T08:17:48.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4152a00640 1 -- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 msgr2=0x7f4130009820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T08:17:48.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4152a00640 1 --2- 172.21.15.46:0/3647629790 >> v2:172.21.15.46:3300/0 conn(0x7f4130007360 0x7f4130009820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:48.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4152a00640 1 -- 172.21.15.46:0/3647629790 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413000af20 con 0x7f41300036f0 2024-03-29T08:17:48.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4152a00640 1 -- 172.21.15.46:0/3647629790 <== mon.1 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f414400e510 con 0x7f41300036f0 2024-03-29T08:17:48.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4152a00640 1 -- 172.21.15.46:0/3647629790 >> v1:172.21.15.73:6789/0 conn(0x7f41300036f0 legacy=0x7f4130005b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:48.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4153e00640 1 -- 172.21.15.46:0/3647629790 reap_dead start 2024-03-29T08:17:48.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4152a00640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41380039d0 0x7f4138003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4152a00640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41380047d0 0x7f4138004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4152a00640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4138005390 0x7f413800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4152a00640 1 -- 172.21.15.46:0/3647629790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4154099c30 con 0x7f41380047d0 2024-03-29T08:17:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4152a00640 1 -- 172.21.15.46:0/3647629790 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413800c250 con 0x7f4138005390 2024-03-29T08:17:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4152a00640 1 -- 172.21.15.46:0/3647629790 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413800c410 con 0x7f41380039d0 2024-03-29T08:17:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4153400640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41380039d0 0x7f4138003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4153e00640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4138005390 0x7f413800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:48.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f414be00640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41380047d0 0x7f4138004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:48.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4153400640 1 -- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4138005390 msgr2=0x7f413800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:17:48.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4153400640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4138005390 0x7f413800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:48.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4153400640 1 -- 172.21.15.46:0/3647629790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41380047d0 msgr2=0x7f4138004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:17:48.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4153400640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41380047d0 0x7f4138004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:48.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.137+0000 7f4153400640 1 -- 172.21.15.46:0/3647629790 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4138003530 con 0x7f41380039d0 2024-03-29T08:17:48.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.141+0000 7f4153400640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41380039d0 0x7f4138003dd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f414400ef90 tx=0x7f414401d830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28cc1ab43caa812d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:48.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.141+0000 7f4152a00640 1 -- 172.21.15.46:0/3647629790 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41440a76f0 con 0x7f41380039d0 2024-03-29T08:17:48.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.141+0000 7f4152a00640 1 -- 172.21.15.46:0/3647629790 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414400c040 con 0x7f41380039d0 2024-03-29T08:17:48.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.141+0000 7f4152a00640 1 -- 172.21.15.46:0/3647629790 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41440a7ab0 con 0x7f41380039d0 2024-03-29T08:17:48.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.141+0000 7f415a800640 1 -- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41380039d0 msgr2=0x7f4138003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:48.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.141+0000 7f415a800640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41380039d0 0x7f4138003dd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f414400ef90 tx=0x7f414401d830 comp rx=0 tx=0).stop 2024-03-29T08:17:48.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.141+0000 7f415a800640 1 -- 172.21.15.46:0/3647629790 shutdown_connections 2024-03-29T08:17:48.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.141+0000 7f415a800640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4138005390 0x7f413800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:48.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.141+0000 7f415a800640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41380047d0 0x7f4138004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:48.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.141+0000 7f415a800640 1 --2- 172.21.15.46:0/3647629790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41380039d0 0x7f4138003dd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:48.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.141+0000 7f415a800640 1 -- 172.21.15.46:0/3647629790 >> 172.21.15.46:0/3647629790 conn(0x7f41540b2220 msgr2=0x7f41540b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:48.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.141+0000 7f415a800640 1 -- 172.21.15.46:0/3647629790 shutdown_connections 2024-03-29T08:17:48.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.141+0000 7f415a800640 1 -- 172.21.15.46:0/3647629790 wait complete. 2024-03-29T08:17:48.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f415a800640 1 Processor -- start 2024-03-29T08:17:48.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f415a800640 1 -- start start 2024-03-29T08:17:48.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f415a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41380039d0 0x7f4154156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:48.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f415a800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41380047d0 0x7f41540c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:48.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f415a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4138005390 0x7f4154152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:48.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f415a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415406b600 con 0x7f4138005390 2024-03-29T08:17:48.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f415a800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41540c85c0 con 0x7f41380039d0 2024-03-29T08:17:48.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f415a800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41541f2020 con 0x7f41380047d0 2024-03-29T08:17:48.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f4153e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4138005390 0x7f4154152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:48.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f4153400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41380039d0 0x7f4154156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:48.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f4153e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4138005390 0x7f4154152560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38354/0 (socket says 172.21.15.46:38354) 2024-03-29T08:17:48.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f4153e00640 1 -- 172.21.15.46:0/407081314 learned_addr learned my addr 172.21.15.46:0/407081314 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:17:48.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f414be00640 1 --2- 172.21.15.46:0/407081314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41380047d0 0x7f41540c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:48.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f4153e00640 1 -- 172.21.15.46:0/407081314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41380047d0 msgr2=0x7f41540c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:48.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f4153e00640 1 --2- 172.21.15.46:0/407081314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41380047d0 0x7f41540c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:48.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f4153e00640 1 -- 172.21.15.46:0/407081314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41380039d0 msgr2=0x7f4154156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:48.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f4153e00640 1 --2- 172.21.15.46:0/407081314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41380039d0 0x7f4154156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:48.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f4153e00640 1 -- 172.21.15.46:0/407081314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4154152aa0 con 0x7f4138005390 2024-03-29T08:17:48.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f4153e00640 1 --2- 172.21.15.46:0/407081314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4138005390 0x7f4154152560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f414c009610 tx=0x7f414c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:48.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f414b400640 1 -- 172.21.15.46:0/407081314 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c015d60 con 0x7f4138005390 2024-03-29T08:17:48.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f414b400640 1 -- 172.21.15.46:0/407081314 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414c00f040 con 0x7f4138005390 2024-03-29T08:17:48.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f414b400640 1 -- 172.21.15.46:0/407081314 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c015ee0 con 0x7f4138005390 2024-03-29T08:17:48.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.145+0000 7f415a800640 1 -- 172.21.15.46:0/407081314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41540c87d0 con 0x7f4138005390 2024-03-29T08:17:48.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.149+0000 7f415a800640 1 -- 172.21.15.46:0/407081314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f415406b220 con 0x7f4138005390 2024-03-29T08:17:48.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.149+0000 7f414b400640 1 -- 172.21.15.46:0/407081314 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f414c0209a0 con 0x7f4138005390 2024-03-29T08:17:48.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.149+0000 7f414b400640 1 -- 172.21.15.46:0/407081314 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f414c017b30 con 0x7f4138005390 2024-03-29T08:17:48.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.149+0000 7f415a800640 1 -- 172.21.15.46:0/407081314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f415404e120 con 0x7f4138005390 2024-03-29T08:17:48.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.153+0000 7f414b400640 1 -- 172.21.15.46:0/407081314 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f414c025060 con 0x7f4138005390 2024-03-29T08:17:48.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.213+0000 7f415a800640 1 -- 172.21.15.46:0/407081314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f415406f570 con 0x7f4138005390 2024-03-29T08:17:48.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.217+0000 7f414b400640 1 -- 172.21.15.46:0/407081314 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f414c02d020 con 0x7f4138005390 2024-03-29T08:17:48.219 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2024-03-29T08:17:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.217+0000 7f415a800640 1 -- 172.21.15.46:0/407081314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4138005390 msgr2=0x7f4154152560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.217+0000 7f415a800640 1 --2- 172.21.15.46:0/407081314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4138005390 0x7f4154152560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f414c009610 tx=0x7f414c009f10 comp rx=0 tx=0).stop 2024-03-29T08:17:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.221+0000 7f415a800640 1 -- 172.21.15.46:0/407081314 shutdown_connections 2024-03-29T08:17:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.221+0000 7f415a800640 1 --2- 172.21.15.46:0/407081314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4138005390 0x7f4154152560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.221+0000 7f415a800640 1 --2- 172.21.15.46:0/407081314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41380047d0 0x7f41540c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.221+0000 7f415a800640 1 --2- 172.21.15.46:0/407081314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41380039d0 0x7f4154156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.221+0000 7f415a800640 1 -- 172.21.15.46:0/407081314 >> 172.21.15.46:0/407081314 conn(0x7f41540b2220 msgr2=0x7f415415d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.221+0000 7f415a800640 1 -- 172.21.15.46:0/407081314 shutdown_connections 2024-03-29T08:17:48.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:48.221+0000 7f415a800640 1 -- 172.21.15.46:0/407081314 wait complete. 2024-03-29T08:17:48.240 INFO:tasks.ceph:check_enable_crimson: False 2024-03-29T08:17:48.240 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-29T08:17:48.240 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:48.240 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-29T08:17:48.260 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:48.260 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-29T08:17:48.326 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:48.326 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-29T08:17:48.386 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:17:48.386 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-29T08:17:48.446 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:48.446 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-29T08:17:48.469 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:48.469 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-29T08:17:48.532 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:48.532 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-29T08:17:48.599 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:17:48.599 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-29T08:17:48.663 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd new 240e3cf2-f51c-4b8d-bb05-9f765ac1b07b 0 2024-03-29T08:17:48.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.847+0000 7f00d4400640 1 Processor -- start 2024-03-29T08:17:48.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.847+0000 7f00d4400640 1 -- start start 2024-03-29T08:17:48.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.847+0000 7f00d4400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f00cc15c950 0x7f00cc06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:48.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.847+0000 7f00d4400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00cc0cfb20 con 0x7f00cc0cea30 2024-03-29T08:17:48.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.847+0000 7f00d4400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00cc0cef10 con 0x7f00cc160db0 2024-03-29T08:17:48.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.847+0000 7f00d4400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc0cf1f0 con 0x7f00cc15c950 2024-03-29T08:17:48.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.847+0000 7f00d2400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f00cc160db0 0x7f00cc0cad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51204/0 (socket says 172.21.15.73:51204) 2024-03-29T08:17:48.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.847+0000 7f00d2400640 1 -- 172.21.15.73:0/3808982009 learned_addr learned my addr 172.21.15.73:0/3808982009 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-03-29T08:17:48.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.847+0000 7f00d1a00640 1 --2- 172.21.15.73:0/3808982009 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f00cc15c950 0x7f00cc06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:48.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.847+0000 7f00cbe00640 1 -- 172.21.15.73:0/3808982009 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 246989781 0 0) 0x7f00cc0cef10 con 0x7f00cc160db0 2024-03-29T08:17:48.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.847+0000 7f00cbe00640 1 -- 172.21.15.73:0/3808982009 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00b4003530 con 0x7f00cc160db0 2024-03-29T08:17:48.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.847+0000 7f00d1a00640 1 -- 172.21.15.73:0/3808982009 >> v1:172.21.15.73:6789/0 conn(0x7f00cc160db0 legacy=0x7f00cc0cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:48.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.847+0000 7f00d1a00640 1 -- 172.21.15.73:0/3808982009 >> v1:172.21.15.46:6789/0 conn(0x7f00cc0cea30 legacy=0x7f00cc1606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:48.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.847+0000 7f00cbe00640 1 -- 172.21.15.73:0/3808982009 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066021213 0 0) 0x7f00cc0cfb20 con 0x7f00cc0cea30 2024-03-29T08:17:48.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.847+0000 7f00d1a00640 1 -- 172.21.15.73:0/3808982009 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc0cf4d0 con 0x7f00cc15c950 2024-03-29T08:17:48.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.851+0000 7f00d1a00640 1 --2- 172.21.15.73:0/3808982009 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f00cc15c950 0x7f00cc06bbe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009dc0 tx=0x7f00c001f170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f39b2d23d2f8446c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:48.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.851+0000 7f00cbe00640 1 -- 172.21.15.73:0/3808982009 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c00af600 con 0x7f00cc15c950 2024-03-29T08:17:48.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.851+0000 7f00cbe00640 1 -- 172.21.15.73:0/3808982009 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c00ab050 con 0x7f00cc15c950 2024-03-29T08:17:48.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.851+0000 7f00cbe00640 1 -- 172.21.15.73:0/3808982009 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c00af980 con 0x7f00cc15c950 2024-03-29T08:17:48.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.851+0000 7f00d4400640 1 -- 172.21.15.73:0/3808982009 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f00cc15c950 msgr2=0x7f00cc06bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:48.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.851+0000 7f00d4400640 1 --2- 172.21.15.73:0/3808982009 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f00cc15c950 0x7f00cc06bbe0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f00c0009dc0 tx=0x7f00c001f170 comp rx=0 tx=0).stop 2024-03-29T08:17:48.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.851+0000 7f00d4400640 1 -- 172.21.15.73:0/3808982009 shutdown_connections 2024-03-29T08:17:48.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.851+0000 7f00d4400640 1 --2- 172.21.15.73:0/3808982009 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f00cc15c950 0x7f00cc06bbe0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:48.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.851+0000 7f00d4400640 1 -- 172.21.15.73:0/3808982009 >> 172.21.15.73:0/3808982009 conn(0x7f00cc0b21b0 msgr2=0x7f00cc0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:48.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.851+0000 7f00d4400640 1 -- 172.21.15.73:0/3808982009 shutdown_connections 2024-03-29T08:17:48.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.851+0000 7f00d4400640 1 -- 172.21.15.73:0/3808982009 wait complete. 2024-03-29T08:17:48.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.851+0000 7f00d4400640 1 Processor -- start 2024-03-29T08:17:48.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.851+0000 7f00d4400640 1 -- start start 2024-03-29T08:17:48.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d4400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f00cc0cea30 0x7f00cc09d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:48.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d4400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00cc15c950 0x7f00cc1591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:48.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d4400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00cc160db0 0x7f00cc06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:48.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d4400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc09dc10 con 0x7f00cc160db0 2024-03-29T08:17:48.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d4400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc09def0 con 0x7f00cc15c950 2024-03-29T08:17:48.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d4400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc09e1d0 con 0x7f00cc0cea30 2024-03-29T08:17:48.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d1a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f00cc0cea30 0x7f00cc09d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:48.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d1000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00cc15c950 0x7f00cc1591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:48.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d1a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f00cc0cea30 0x7f00cc09d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.73:35598/0 (socket says 172.21.15.73:35598) 2024-03-29T08:17:48.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d1a00640 1 -- 172.21.15.73:0/3627706690 learned_addr learned my addr 172.21.15.73:0/3627706690 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-03-29T08:17:48.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d2400640 1 --2- 172.21.15.73:0/3627706690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00cc160db0 0x7f00cc06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:48.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d1a00640 1 -- 172.21.15.73:0/3627706690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00cc15c950 msgr2=0x7f00cc1591b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:48.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d1a00640 1 --2- 172.21.15.73:0/3627706690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00cc15c950 0x7f00cc1591b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:48.859 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d1a00640 1 -- 172.21.15.73:0/3627706690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00cc160db0 msgr2=0x7f00cc06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:48.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d1a00640 1 --2- 172.21.15.73:0/3627706690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00cc160db0 0x7f00cc06f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:48.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d1a00640 1 -- 172.21.15.73:0/3627706690 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c0009a70 con 0x7f00cc0cea30 2024-03-29T08:17:48.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d1000640 1 --2- 172.21.15.73:0/3627706690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00cc15c950 0x7f00cc1591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:17:48.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d1a00640 1 --2- 172.21.15.73:0/3627706690 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f00cc0cea30 0x7f00cc09d530 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f00c00b2040 tx=0x7f00c001bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:48.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00c9600640 1 -- 172.21.15.73:0/3627706690 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0009450 con 0x7f00cc0cea30 2024-03-29T08:17:48.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00c9600640 1 -- 172.21.15.73:0/3627706690 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c00039a0 con 0x7f00cc0cea30 2024-03-29T08:17:48.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d4400640 1 -- 172.21.15.73:0/3627706690 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00cc09a4c0 con 0x7f00cc0cea30 2024-03-29T08:17:48.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00c9600640 1 -- 172.21.15.73:0/3627706690 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c0003d30 con 0x7f00cc0cea30 2024-03-29T08:17:48.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d4400640 1 -- 172.21.15.73:0/3627706690 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00cc1596f0 con 0x7f00cc0cea30 2024-03-29T08:17:48.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00c9600640 1 -- 172.21.15.73:0/3627706690 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f00c00b6050 con 0x7f00cc0cea30 2024-03-29T08:17:48.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00c9600640 1 -- 172.21.15.73:0/3627706690 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f00c00c69f0 con 0x7f00cc0cea30 2024-03-29T08:17:48.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.855+0000 7f00d4400640 1 -- 172.21.15.73:0/3627706690 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00cc09e690 con 0x7f00cc0cea30 2024-03-29T08:17:48.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.871+0000 7f00c9600640 1 -- 172.21.15.73:0/3627706690 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f00c00b4070 con 0x7f00cc0cea30 2024-03-29T08:17:48.938 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.935+0000 7f00d4400640 1 -- 172.21.15.73:0/3627706690 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "osd new", "uuid": "240e3cf2-f51c-4b8d-bb05-9f765ac1b07b", "id": 0} v 0) v1 -- 0x7f00cc06c600 con 0x7f00cc0cea30 2024-03-29T08:17:48.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.943+0000 7f00c9600640 1 -- 172.21.15.73:0/3627706690 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "240e3cf2-f51c-4b8d-bb05-9f765ac1b07b", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f00c00c6bb0 con 0x7f00cc0cea30 2024-03-29T08:17:48.946 INFO:teuthology.orchestra.run.smithi073.stdout:0 2024-03-29T08:17:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.943+0000 7f00d4400640 1 -- 172.21.15.73:0/3627706690 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f00cc0cea30 msgr2=0x7f00cc09d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.943+0000 7f00d4400640 1 --2- 172.21.15.73:0/3627706690 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f00cc0cea30 0x7f00cc09d530 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f00c00b2040 tx=0x7f00c001bf10 comp rx=0 tx=0).stop 2024-03-29T08:17:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.943+0000 7f00d4400640 1 -- 172.21.15.73:0/3627706690 shutdown_connections 2024-03-29T08:17:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.943+0000 7f00d4400640 1 --2- 172.21.15.73:0/3627706690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00cc15c950 0x7f00cc1591b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.943+0000 7f00d4400640 1 --2- 172.21.15.73:0/3627706690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00cc160db0 0x7f00cc06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.943+0000 7f00d4400640 1 --2- 172.21.15.73:0/3627706690 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f00cc0cea30 0x7f00cc09d530 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.943+0000 7f00d4400640 1 -- 172.21.15.73:0/3627706690 >> 172.21.15.73:0/3627706690 conn(0x7f00cc0b21b0 msgr2=0x7f00cc098a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.943+0000 7f00d4400640 1 -- 172.21.15.73:0/3627706690 shutdown_connections 2024-03-29T08:17:48.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:48.943+0000 7f00d4400640 1 -- 172.21.15.73:0/3627706690 wait complete. 2024-03-29T08:17:48.967 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd new caa4904f-5189-4846-a8c7-c769e3db5a87 1 2024-03-29T08:17:49.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.119+0000 7fb642000640 1 Processor -- start 2024-03-29T08:17:49.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.119+0000 7fb642000640 1 -- start start 2024-03-29T08:17:49.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.119+0000 7fb642000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb63c15a790 0x7fb63c0cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.119+0000 7fb642000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c0cc660 con 0x7fb63c0cbbe0 2024-03-29T08:17:49.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.119+0000 7fb642000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c0cc0a0 con 0x7fb63c0cf340 2024-03-29T08:17:49.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.119+0000 7fb642000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63c0cc380 con 0x7fb63c15a790 2024-03-29T08:17:49.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.119+0000 7fb63be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb63c0cf340 0x7fb63c0b0b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51218/0 (socket says 172.21.15.73:51218) 2024-03-29T08:17:49.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.119+0000 7fb63be00640 1 -- 172.21.15.73:0/3022260036 learned_addr learned my addr 172.21.15.73:0/3022260036 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-03-29T08:17:49.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.119+0000 7fb63b400640 1 --2- 172.21.15.73:0/3022260036 >> v2:172.21.15.46:3300/0 conn(0x7fb63c15a790 0x7fb63c0cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.119+0000 7fb63aa00640 1 -- 172.21.15.73:0/3022260036 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405667863 0 0) 0x7fb63c0cc0a0 con 0x7fb63c0cf340 2024-03-29T08:17:49.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.119+0000 7fb63aa00640 1 -- 172.21.15.73:0/3022260036 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb620003530 con 0x7fb63c0cf340 2024-03-29T08:17:49.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.119+0000 7fb63aa00640 1 -- 172.21.15.73:0/3022260036 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723589731 0 0) 0x7fb63c0cc660 con 0x7fb63c0cbbe0 2024-03-29T08:17:49.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.119+0000 7fb63aa00640 1 -- 172.21.15.73:0/3022260036 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb63c0cc0a0 con 0x7fb63c0cbbe0 2024-03-29T08:17:49.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.119+0000 7fb63aa00640 1 -- 172.21.15.73:0/3022260036 <== mon.1 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1888069880 0 0) 0x7fb620003530 con 0x7fb63c0cf340 2024-03-29T08:17:49.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.119+0000 7fb63b400640 1 -- 172.21.15.73:0/3022260036 >> v1:172.21.15.73:6789/0 conn(0x7fb63c0cf340 legacy=0x7fb63c0b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.119+0000 7fb63b400640 1 -- 172.21.15.73:0/3022260036 >> v1:172.21.15.46:6789/0 conn(0x7fb63c0cbbe0 legacy=0x7fb63c0d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.119+0000 7fb63b400640 1 -- 172.21.15.73:0/3022260036 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c0b0380 con 0x7fb63c15a790 2024-03-29T08:17:49.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.123+0000 7fb63b400640 1 --2- 172.21.15.73:0/3022260036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb63c15a790 0x7fb63c0cee00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009740 tx=0x7fb62c01c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a911f2f30f4188 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:49.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.123+0000 7fb63aa00640 1 -- 172.21.15.73:0/3022260036 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634001a00 con 0x7fb63c15a790 2024-03-29T08:17:49.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.123+0000 7fb63aa00640 1 -- 172.21.15.73:0/3022260036 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62c0a7050 con 0x7fb63c15a790 2024-03-29T08:17:49.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.123+0000 7fb63aa00640 1 -- 172.21.15.73:0/3022260036 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62c0ab620 con 0x7fb63c15a790 2024-03-29T08:17:49.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.123+0000 7fb642000640 1 -- 172.21.15.73:0/3022260036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb63c15a790 msgr2=0x7fb63c0cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.123+0000 7fb642000640 1 --2- 172.21.15.73:0/3022260036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb63c15a790 0x7fb63c0cee00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb62c009740 tx=0x7fb62c01c210 comp rx=0 tx=0).stop 2024-03-29T08:17:49.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.123+0000 7fb642000640 1 -- 172.21.15.73:0/3022260036 shutdown_connections 2024-03-29T08:17:49.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.123+0000 7fb642000640 1 --2- 172.21.15.73:0/3022260036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb63c15a790 0x7fb63c0cee00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.123+0000 7fb642000640 1 -- 172.21.15.73:0/3022260036 >> 172.21.15.73:0/3022260036 conn(0x7fb63c099a10 msgr2=0x7fb63c089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:49.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.123+0000 7fb642000640 1 -- 172.21.15.73:0/3022260036 shutdown_connections 2024-03-29T08:17:49.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.123+0000 7fb642000640 1 -- 172.21.15.73:0/3022260036 wait complete. 2024-03-29T08:17:49.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb642000640 1 Processor -- start 2024-03-29T08:17:49.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb642000640 1 -- start start 2024-03-29T08:17:49.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb642000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb63c0cbbe0 0x7fb63c14f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb642000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63c0cf340 0x7fb63c151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb642000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb63c15a790 0x7fb63c152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb642000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c0c82a0 con 0x7fb63c0cbbe0 2024-03-29T08:17:49.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb642000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c151810 con 0x7fb63c0cf340 2024-03-29T08:17:49.129 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb642000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c0c7940 con 0x7fb63c15a790 2024-03-29T08:17:49.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb633e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63c0cf340 0x7fb63c151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb633e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63c0cf340 0x7fb63c151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39798/0 (socket says 172.21.15.73:39798) 2024-03-29T08:17:49.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb63b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb63c0cbbe0 0x7fb63c14f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb633e00640 1 -- 172.21.15.73:0/2655431030 learned_addr learned my addr 172.21.15.73:0/2655431030 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-03-29T08:17:49.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb633e00640 1 -- 172.21.15.73:0/2655431030 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb63c15a790 msgr2=0x7fb63c152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb633e00640 1 --2- 172.21.15.73:0/2655431030 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb63c15a790 0x7fb63c152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb633e00640 1 -- 172.21.15.73:0/2655431030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb63c0cbbe0 msgr2=0x7fb63c14f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb633e00640 1 --2- 172.21.15.73:0/2655431030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb63c0cbbe0 0x7fb63c14f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb633e00640 1 -- 172.21.15.73:0/2655431030 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62c009420 con 0x7fb63c0cf340 2024-03-29T08:17:49.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb633e00640 1 --2- 172.21.15.73:0/2655431030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63c0cf340 0x7fb63c151c10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb62400c4c0 tx=0x7fb62400c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:49.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb633400640 1 -- 172.21.15.73:0/2655431030 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6240052d0 con 0x7fb63c0cf340 2024-03-29T08:17:49.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb633400640 1 -- 172.21.15.73:0/2655431030 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb624005430 con 0x7fb63c0cf340 2024-03-29T08:17:49.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb633400640 1 -- 172.21.15.73:0/2655431030 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624012870 con 0x7fb63c0cf340 2024-03-29T08:17:49.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb642000640 1 -- 172.21.15.73:0/2655431030 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63c0c7be0 con 0x7fb63c0cf340 2024-03-29T08:17:49.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb642000640 1 -- 172.21.15.73:0/2655431030 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63c14f820 con 0x7fb63c0cf340 2024-03-29T08:17:49.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb633400640 1 -- 172.21.15.73:0/2655431030 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb624012cd0 con 0x7fb63c0cf340 2024-03-29T08:17:49.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.127+0000 7fb633400640 1 -- 172.21.15.73:0/2655431030 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fb62401b440 con 0x7fb63c0cf340 2024-03-29T08:17:49.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.131+0000 7fb642000640 1 -- 172.21.15.73:0/2655431030 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63c0c83e0 con 0x7fb63c0cf340 2024-03-29T08:17:49.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.131+0000 7fb633400640 1 -- 172.21.15.73:0/2655431030 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fb63c0c83e0 con 0x7fb63c0cf340 2024-03-29T08:17:49.171 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:49.171+0000 7f9d37c4b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-29T08:17:49.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.199+0000 7fb642000640 1 -- 172.21.15.73:0/2655431030 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd new", "uuid": "caa4904f-5189-4846-a8c7-c769e3db5a87", "id": 1} v 0) v1 -- 0x7fb63c09e7d0 con 0x7fb63c0cf340 2024-03-29T08:17:49.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.203+0000 7fb633400640 1 -- 172.21.15.73:0/2655431030 <== mon.1 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "caa4904f-5189-4846-a8c7-c769e3db5a87", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb624003780 con 0x7fb63c0cf340 2024-03-29T08:17:49.205 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-03-29T08:17:49.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.203+0000 7fb642000640 1 -- 172.21.15.73:0/2655431030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63c0cf340 msgr2=0x7fb63c151c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.203+0000 7fb642000640 1 --2- 172.21.15.73:0/2655431030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63c0cf340 0x7fb63c151c10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb62400c4c0 tx=0x7fb62400c980 comp rx=0 tx=0).stop 2024-03-29T08:17:49.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.203+0000 7fb642000640 1 -- 172.21.15.73:0/2655431030 shutdown_connections 2024-03-29T08:17:49.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.203+0000 7fb642000640 1 --2- 172.21.15.73:0/2655431030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb63c0cf340 0x7fb63c151c10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.203+0000 7fb642000640 1 --2- 172.21.15.73:0/2655431030 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb63c15a790 0x7fb63c152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.203+0000 7fb642000640 1 --2- 172.21.15.73:0/2655431030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb63c0cbbe0 0x7fb63c14f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.203+0000 7fb642000640 1 -- 172.21.15.73:0/2655431030 >> 172.21.15.73:0/2655431030 conn(0x7fb63c099a10 msgr2=0x7fb63c152af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:49.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.203+0000 7fb642000640 1 -- 172.21.15.73:0/2655431030 shutdown_connections 2024-03-29T08:17:49.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.203+0000 7fb642000640 1 -- 172.21.15.73:0/2655431030 wait complete. 2024-03-29T08:17:49.224 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd new 4455b5b9-82b8-44b2-a73e-511adaf55986 2 2024-03-29T08:17:49.265 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:49.265+0000 7fc9d5e9b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-29T08:17:49.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.355+0000 7fa93cc00640 1 Processor -- start 2024-03-29T08:17:49.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.355+0000 7fa93cc00640 1 -- start start 2024-03-29T08:17:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.359+0000 7fa93cc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa93815c9c0 0x7fa9380afde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.359+0000 7fa93cc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9380ce8b0 0x7fa9380ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.359+0000 7fa93cc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9380af950 con 0x7fa938069d90 2024-03-29T08:17:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.359+0000 7fa93cc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9380cf390 con 0x7fa93815c9c0 2024-03-29T08:17:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.359+0000 7fa93cc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9380af020 con 0x7fa9380ce8b0 2024-03-29T08:17:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.359+0000 7fa92fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9380ce8b0 0x7fa9380ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.359+0000 7fa936800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa938069d90 0x7fa9380ac310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51232/0 (socket says 172.21.15.73:51232) 2024-03-29T08:17:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.359+0000 7fa936800640 1 -- 172.21.15.73:0/829986999 learned_addr learned my addr 172.21.15.73:0/829986999 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-03-29T08:17:49.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.359+0000 7fa935e00640 1 --2- 172.21.15.73:0/829986999 >> v2:172.21.15.46:3300/0 conn(0x7fa93815c9c0 0x7fa9380afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.359+0000 7fa92fe00640 1 -- 172.21.15.73:0/829986999 >> v2:172.21.15.46:3300/0 conn(0x7fa93815c9c0 msgr2=0x7fa9380afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:17:49.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.359+0000 7fa92fe00640 1 --2- 172.21.15.73:0/829986999 >> v2:172.21.15.46:3300/0 conn(0x7fa93815c9c0 0x7fa9380afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.359+0000 7fa92fe00640 1 -- 172.21.15.73:0/829986999 >> v1:172.21.15.73:6789/0 conn(0x7fa938069d90 legacy=0x7fa9380ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.359+0000 7fa92fe00640 1 -- 172.21.15.73:0/829986999 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9380af300 con 0x7fa9380ce8b0 2024-03-29T08:17:49.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa92fe00640 1 --2- 172.21.15.73:0/829986999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9380ce8b0 0x7fa9380ced10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa9200097c0 tx=0x7fa92001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a72f45f5849dbaa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:49.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa935400640 1 -- 172.21.15.73:0/829986999 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9200ac630 con 0x7fa9380ce8b0 2024-03-29T08:17:49.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa935400640 1 -- 172.21.15.73:0/829986999 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9200ac790 con 0x7fa9380ce8b0 2024-03-29T08:17:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa935400640 1 -- 172.21.15.73:0/829986999 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9200acb20 con 0x7fa9380ce8b0 2024-03-29T08:17:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa93cc00640 1 -- 172.21.15.73:0/829986999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9380ce8b0 msgr2=0x7fa9380ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa93cc00640 1 --2- 172.21.15.73:0/829986999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9380ce8b0 0x7fa9380ced10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa9200097c0 tx=0x7fa92001c470 comp rx=0 tx=0).stop 2024-03-29T08:17:49.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa93cc00640 1 -- 172.21.15.73:0/829986999 shutdown_connections 2024-03-29T08:17:49.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa93cc00640 1 --2- 172.21.15.73:0/829986999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9380ce8b0 0x7fa9380ced10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa93cc00640 1 --2- 172.21.15.73:0/829986999 >> v2:172.21.15.46:3300/0 conn(0x7fa93815c9c0 0x7fa9380afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa93cc00640 1 -- 172.21.15.73:0/829986999 >> 172.21.15.73:0/829986999 conn(0x7fa938098a40 msgr2=0x7fa9380cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:49.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa93cc00640 1 -- 172.21.15.73:0/829986999 shutdown_connections 2024-03-29T08:17:49.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa93cc00640 1 -- 172.21.15.73:0/829986999 wait complete. 2024-03-29T08:17:49.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa93cc00640 1 Processor -- start 2024-03-29T08:17:49.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa93cc00640 1 -- start start 2024-03-29T08:17:49.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa93cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938069d90 0x7fa93815ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa93cc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9380ce8b0 0x7fa93806df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa93cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa93815c9c0 0x7fa9381524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa93cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93815b580 con 0x7fa938069d90 2024-03-29T08:17:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa93cc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381f9ae0 con 0x7fa93815c9c0 2024-03-29T08:17:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.363+0000 7fa93cc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9381f8d80 con 0x7fa9380ce8b0 2024-03-29T08:17:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa92fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9380ce8b0 0x7fa93806df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa92fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9380ce8b0 0x7fa93806df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.73:35636/0 (socket says 172.21.15.73:35636) 2024-03-29T08:17:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa92fe00640 1 -- 172.21.15.73:0/2761391418 learned_addr learned my addr 172.21.15.73:0/2761391418 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-03-29T08:17:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa936800640 1 --2- 172.21.15.73:0/2761391418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa93815c9c0 0x7fa9381524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa935e00640 1 --2- 172.21.15.73:0/2761391418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938069d90 0x7fa93815ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa92fe00640 1 -- 172.21.15.73:0/2761391418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa93815c9c0 msgr2=0x7fa9381524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa92fe00640 1 --2- 172.21.15.73:0/2761391418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa93815c9c0 0x7fa9381524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa92fe00640 1 -- 172.21.15.73:0/2761391418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938069d90 msgr2=0x7fa93815ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa92fe00640 1 --2- 172.21.15.73:0/2761391418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938069d90 0x7fa93815ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa92fe00640 1 -- 172.21.15.73:0/2761391418 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa920009420 con 0x7fa9380ce8b0 2024-03-29T08:17:49.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa935e00640 1 --2- 172.21.15.73:0/2761391418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938069d90 0x7fa93815ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:17:49.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa92fe00640 1 --2- 172.21.15.73:0/2761391418 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9380ce8b0 0x7fa93806df10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa9200097c0 tx=0x7fa920019210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:49.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa92e000640 1 -- 172.21.15.73:0/2761391418 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9200b5610 con 0x7fa9380ce8b0 2024-03-29T08:17:49.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa92e000640 1 -- 172.21.15.73:0/2761391418 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9200b5770 con 0x7fa9380ce8b0 2024-03-29T08:17:49.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa92e000640 1 -- 172.21.15.73:0/2761391418 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9200b5b20 con 0x7fa9380ce8b0 2024-03-29T08:17:49.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa93cc00640 1 -- 172.21.15.73:0/2761391418 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9381f9020 con 0x7fa9380ce8b0 2024-03-29T08:17:49.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa93cc00640 1 -- 172.21.15.73:0/2761391418 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9381f95e0 con 0x7fa9380ce8b0 2024-03-29T08:17:49.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa92e000640 1 -- 172.21.15.73:0/2761391418 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa9200b3040 con 0x7fa9380ce8b0 2024-03-29T08:17:49.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa92e000640 1 -- 172.21.15.73:0/2761391418 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fa9200ab3d0 con 0x7fa9380ce8b0 2024-03-29T08:17:49.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.367+0000 7fa93cc00640 1 -- 172.21.15.73:0/2761391418 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9380c4290 con 0x7fa9380ce8b0 2024-03-29T08:17:49.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.371+0000 7fa92e000640 1 -- 172.21.15.73:0/2761391418 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fa9200c3060 con 0x7fa9380ce8b0 2024-03-29T08:17:49.436 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.435+0000 7fa93cc00640 1 -- 172.21.15.73:0/2761391418 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4455b5b9-82b8-44b2-a73e-511adaf55986", "id": 2} v 0) v1 -- 0x7fa93809d690 con 0x7fa9380ce8b0 2024-03-29T08:17:49.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.439+0000 7fa92e000640 1 -- 172.21.15.73:0/2761391418 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4455b5b9-82b8-44b2-a73e-511adaf55986", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa9200b9070 con 0x7fa9380ce8b0 2024-03-29T08:17:49.441 INFO:teuthology.orchestra.run.smithi073.stdout:2 2024-03-29T08:17:49.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.439+0000 7fa93cc00640 1 -- 172.21.15.73:0/2761391418 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9380ce8b0 msgr2=0x7fa93806df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.439+0000 7fa93cc00640 1 --2- 172.21.15.73:0/2761391418 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9380ce8b0 0x7fa93806df10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa9200097c0 tx=0x7fa920019210 comp rx=0 tx=0).stop 2024-03-29T08:17:49.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.439+0000 7fa93cc00640 1 -- 172.21.15.73:0/2761391418 shutdown_connections 2024-03-29T08:17:49.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.439+0000 7fa93cc00640 1 --2- 172.21.15.73:0/2761391418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa93815c9c0 0x7fa9381524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.439+0000 7fa93cc00640 1 --2- 172.21.15.73:0/2761391418 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9380ce8b0 0x7fa93806df10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.439+0000 7fa93cc00640 1 --2- 172.21.15.73:0/2761391418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938069d90 0x7fa93815ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.439+0000 7fa93cc00640 1 -- 172.21.15.73:0/2761391418 >> 172.21.15.73:0/2761391418 conn(0x7fa938098a40 msgr2=0x7fa9380ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:49.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.439+0000 7fa93cc00640 1 -- 172.21.15.73:0/2761391418 shutdown_connections 2024-03-29T08:17:49.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.439+0000 7fa93cc00640 1 -- 172.21.15.73:0/2761391418 wait complete. 2024-03-29T08:17:49.459 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd new eec611d5-4630-40c0-90cd-ab9dc2e96a94 3 2024-03-29T08:17:49.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.591+0000 7fdf83800640 1 Processor -- start 2024-03-29T08:17:49.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.591+0000 7fdf83800640 1 -- start start 2024-03-29T08:17:49.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.591+0000 7fdf83800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf7c15a790 0x7fdf7c0cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.591+0000 7fdf83800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf7c0cf340 0x7fdf7c0b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.591+0000 7fdf83800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7c0cc660 con 0x7fdf7c0cbbe0 2024-03-29T08:17:49.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.591+0000 7fdf83800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7c0cc0a0 con 0x7fdf7c15a790 2024-03-29T08:17:49.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.591+0000 7fdf83800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c0cc380 con 0x7fdf7c0cf340 2024-03-29T08:17:49.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.591+0000 7fdf81800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf7c0cf340 0x7fdf7c0b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.595+0000 7fdf81800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf7c0cf340 0x7fdf7c0b0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.73:35642/0 (socket says 172.21.15.73:35642) 2024-03-29T08:17:49.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.595+0000 7fdf81800640 1 -- 172.21.15.73:0/328344261 learned_addr learned my addr 172.21.15.73:0/328344261 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-03-29T08:17:49.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.595+0000 7fdf80e00640 1 --2- 172.21.15.73:0/328344261 >> v2:172.21.15.46:3300/0 conn(0x7fdf7c15a790 0x7fdf7c0cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.595+0000 7fdf81800640 1 -- 172.21.15.73:0/328344261 >> v2:172.21.15.46:3300/0 conn(0x7fdf7c15a790 msgr2=0x7fdf7c0cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:17:49.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.595+0000 7fdf81800640 1 --2- 172.21.15.73:0/328344261 >> v2:172.21.15.46:3300/0 conn(0x7fdf7c15a790 0x7fdf7c0cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.595+0000 7fdf81800640 1 -- 172.21.15.73:0/328344261 >> v1:172.21.15.46:6789/0 conn(0x7fdf7c0cbbe0 legacy=0x7fdf7c0d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.595+0000 7fdf81800640 1 -- 172.21.15.73:0/328344261 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c0b0380 con 0x7fdf7c0cf340 2024-03-29T08:17:49.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.595+0000 7fdf81800640 1 --2- 172.21.15.73:0/328344261 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf7c0cf340 0x7fdf7c0b0b80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009e50 tx=0x7fdf7401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60b0c20e6666abe server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:49.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.595+0000 7fdf7b400640 1 -- 172.21.15.73:0/328344261 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf740a57c0 con 0x7fdf7c0cf340 2024-03-29T08:17:49.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.595+0000 7fdf7b400640 1 -- 172.21.15.73:0/328344261 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf740a5920 con 0x7fdf7c0cf340 2024-03-29T08:17:49.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.595+0000 7fdf7b400640 1 -- 172.21.15.73:0/328344261 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf740a5cb0 con 0x7fdf7c0cf340 2024-03-29T08:17:49.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf83800640 1 -- 172.21.15.73:0/328344261 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf7c0cf340 msgr2=0x7fdf7c0b0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf83800640 1 --2- 172.21.15.73:0/328344261 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf7c0cf340 0x7fdf7c0b0b80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009e50 tx=0x7fdf7401c8e0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf83800640 1 -- 172.21.15.73:0/328344261 shutdown_connections 2024-03-29T08:17:49.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf83800640 1 --2- 172.21.15.73:0/328344261 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf7c0cf340 0x7fdf7c0b0b80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf83800640 1 --2- 172.21.15.73:0/328344261 >> v2:172.21.15.46:3300/0 conn(0x7fdf7c15a790 0x7fdf7c0cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf83800640 1 -- 172.21.15.73:0/328344261 >> 172.21.15.73:0/328344261 conn(0x7fdf7c099a10 msgr2=0x7fdf7c089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:49.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf83800640 1 -- 172.21.15.73:0/328344261 shutdown_connections 2024-03-29T08:17:49.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf83800640 1 -- 172.21.15.73:0/328344261 wait complete. 2024-03-29T08:17:49.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf83800640 1 Processor -- start 2024-03-29T08:17:49.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf83800640 1 -- start start 2024-03-29T08:17:49.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf83800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf7c0cbbe0 0x7fdf7c0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf83800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf7c0cf340 0x7fdf7c159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf83800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf7c15a790 0x7fdf7c0b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf83800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c0b4d50 con 0x7fdf7c0cf340 2024-03-29T08:17:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf83800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c0cd760 con 0x7fdf7c0cbbe0 2024-03-29T08:17:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf83800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c0c9f30 con 0x7fdf7c15a790 2024-03-29T08:17:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf80e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf7c0cbbe0 0x7fdf7c0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf81800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf7c15a790 0x7fdf7c0b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf80e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf7c0cbbe0 0x7fdf7c0cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39822/0 (socket says 172.21.15.73:39822) 2024-03-29T08:17:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf80e00640 1 -- 172.21.15.73:0/3462998715 learned_addr learned my addr 172.21.15.73:0/3462998715 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-03-29T08:17:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf7be00640 1 --2- 172.21.15.73:0/3462998715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf7c0cf340 0x7fdf7c159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf80e00640 1 -- 172.21.15.73:0/3462998715 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf7c15a790 msgr2=0x7fdf7c0b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf80e00640 1 --2- 172.21.15.73:0/3462998715 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf7c15a790 0x7fdf7c0b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf80e00640 1 -- 172.21.15.73:0/3462998715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf7c0cf340 msgr2=0x7fdf7c159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf80e00640 1 --2- 172.21.15.73:0/3462998715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf7c0cf340 0x7fdf7c159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf80e00640 1 -- 172.21.15.73:0/3462998715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf74009b30 con 0x7fdf7c0cbbe0 2024-03-29T08:17:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.599+0000 7fdf80e00640 1 --2- 172.21.15.73:0/3462998715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf7c0cbbe0 0x7fdf7c0cd220 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000e870 tx=0x7fdf7000ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.603+0000 7fdf78c00640 1 -- 172.21.15.73:0/3462998715 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70009510 con 0x7fdf7c0cbbe0 2024-03-29T08:17:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.603+0000 7fdf78c00640 1 -- 172.21.15.73:0/3462998715 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7000eec0 con 0x7fdf7c0cbbe0 2024-03-29T08:17:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.603+0000 7fdf78c00640 1 -- 172.21.15.73:0/3462998715 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7000cb40 con 0x7fdf7c0cbbe0 2024-03-29T08:17:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.603+0000 7fdf83800640 1 -- 172.21.15.73:0/3462998715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7c0ca210 con 0x7fdf7c0cbbe0 2024-03-29T08:17:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.603+0000 7fdf83800640 1 -- 172.21.15.73:0/3462998715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7c0b4970 con 0x7fdf7c0cbbe0 2024-03-29T08:17:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.603+0000 7fdf78c00640 1 -- 172.21.15.73:0/3462998715 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fdf700059a0 con 0x7fdf7c0cbbe0 2024-03-29T08:17:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.603+0000 7fdf78c00640 1 -- 172.21.15.73:0/3462998715 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fdf700067f0 con 0x7fdf7c0cbbe0 2024-03-29T08:17:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.603+0000 7fdf83800640 1 -- 172.21.15.73:0/3462998715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf7c09e7d0 con 0x7fdf7c0cbbe0 2024-03-29T08:17:49.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.603+0000 7fdf78c00640 1 -- 172.21.15.73:0/3462998715 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fdf70021060 con 0x7fdf7c0cbbe0 2024-03-29T08:17:49.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.671+0000 7fdf83800640 1 -- 172.21.15.73:0/3462998715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd new", "uuid": "eec611d5-4630-40c0-90cd-ab9dc2e96a94", "id": 3} v 0) v1 -- 0x7fdf7c08ad40 con 0x7fdf7c0cbbe0 2024-03-29T08:17:49.676 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.675+0000 7fdf78c00640 1 -- 172.21.15.73:0/3462998715 <== mon.1 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "eec611d5-4630-40c0-90cd-ab9dc2e96a94", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdf70029020 con 0x7fdf7c0cbbe0 2024-03-29T08:17:49.676 INFO:teuthology.orchestra.run.smithi073.stdout:3 2024-03-29T08:17:49.677 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.675+0000 7fdf83800640 1 -- 172.21.15.73:0/3462998715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf7c0cbbe0 msgr2=0x7fdf7c0cd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.675+0000 7fdf83800640 1 --2- 172.21.15.73:0/3462998715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf7c0cbbe0 0x7fdf7c0cd220 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000e870 tx=0x7fdf7000ec70 comp rx=0 tx=0).stop 2024-03-29T08:17:49.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.675+0000 7fdf83800640 1 -- 172.21.15.73:0/3462998715 shutdown_connections 2024-03-29T08:17:49.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.675+0000 7fdf83800640 1 --2- 172.21.15.73:0/3462998715 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf7c15a790 0x7fdf7c0b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.675+0000 7fdf83800640 1 --2- 172.21.15.73:0/3462998715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf7c0cf340 0x7fdf7c159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.675+0000 7fdf83800640 1 --2- 172.21.15.73:0/3462998715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf7c0cbbe0 0x7fdf7c0cd220 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.675+0000 7fdf83800640 1 -- 172.21.15.73:0/3462998715 >> 172.21.15.73:0/3462998715 conn(0x7fdf7c099a10 msgr2=0x7fdf7c0d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:49.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.675+0000 7fdf83800640 1 -- 172.21.15.73:0/3462998715 shutdown_connections 2024-03-29T08:17:49.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.675+0000 7fdf83800640 1 -- 172.21.15.73:0/3462998715 wait complete. 2024-03-29T08:17:49.685 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:49.683+0000 7f9d37c4b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-29T08:17:49.695 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd new 55692b87-e81a-48db-bb4a-37b2b44062e6 4 2024-03-29T08:17:49.788 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:49.789+0000 7fc9d5e9b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-29T08:17:49.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.827+0000 7ff5abc00640 1 Processor -- start 2024-03-29T08:17:49.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.827+0000 7ff5abc00640 1 -- start start 2024-03-29T08:17:49.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.827+0000 7ff5abc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5a40b0c30 0x7ff5a415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.827+0000 7ff5abc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff5a40b1d20 0x7ff5a40cb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.827+0000 7ff5abc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a40cb7a0 con 0x7ff5a415aa70 2024-03-29T08:17:49.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.827+0000 7ff5abc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a415f390 con 0x7ff5a40b1d20 2024-03-29T08:17:49.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.827+0000 7ff5abc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a415b190 con 0x7ff5a40b0c30 2024-03-29T08:17:49.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.827+0000 7ff59be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5a40b0c30 0x7ff5a415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.827+0000 7ff5a9c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff5a40b1d20 0x7ff5a40cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.827+0000 7ff59be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5a40b0c30 0x7ff5a415ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.73:35658/0 (socket says 172.21.15.73:35658) 2024-03-29T08:17:49.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.827+0000 7ff59be00640 1 -- 172.21.15.73:0/3081471038 learned_addr learned my addr 172.21.15.73:0/3081471038 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-03-29T08:17:49.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.831+0000 7ff59be00640 1 -- 172.21.15.73:0/3081471038 >> v2:172.21.15.73:3300/0 conn(0x7ff5a40b1d20 msgr2=0x7ff5a40cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:17:49.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.831+0000 7ff59be00640 1 --2- 172.21.15.73:0/3081471038 >> v2:172.21.15.73:3300/0 conn(0x7ff5a40b1d20 0x7ff5a40cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.831+0000 7ff59be00640 1 -- 172.21.15.73:0/3081471038 >> v1:172.21.15.73:6789/0 conn(0x7ff5a415aa70 legacy=0x7ff5a40b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.831+0000 7ff59be00640 1 -- 172.21.15.73:0/3081471038 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a415b470 con 0x7ff5a40b0c30 2024-03-29T08:17:49.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.831+0000 7ff5a3e00640 1 -- 172.21.15.73:0/3081471038 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945573147 0 0) 0x7ff5a40cb7a0 con 0x7ff5a415aa70 2024-03-29T08:17:49.832 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.831+0000 7ff5a9c00640 1 --2- 172.21.15.73:0/3081471038 >> v2:172.21.15.73:3300/0 conn(0x7ff5a40b1d20 0x7ff5a40cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:17:49.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.831+0000 7ff59be00640 1 --2- 172.21.15.73:0/3081471038 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5a40b0c30 0x7ff5a415ee50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009870 tx=0x7ff58c01c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29335bd130d52af9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:49.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.831+0000 7ff5a3e00640 1 -- 172.21.15.73:0/3081471038 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0a95f0 con 0x7ff5a40b0c30 2024-03-29T08:17:49.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.831+0000 7ff5a3e00640 1 -- 172.21.15.73:0/3081471038 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c0a9750 con 0x7ff5a40b0c30 2024-03-29T08:17:49.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.831+0000 7ff5a3e00640 1 -- 172.21.15.73:0/3081471038 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0a9ae0 con 0x7ff5a40b0c30 2024-03-29T08:17:49.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.831+0000 7ff5abc00640 1 -- 172.21.15.73:0/3081471038 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5a40b0c30 msgr2=0x7ff5a415ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.831+0000 7ff5abc00640 1 --2- 172.21.15.73:0/3081471038 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5a40b0c30 0x7ff5a415ee50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009870 tx=0x7ff58c01c530 comp rx=0 tx=0).stop 2024-03-29T08:17:49.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.831+0000 7ff5abc00640 1 -- 172.21.15.73:0/3081471038 shutdown_connections 2024-03-29T08:17:49.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff5abc00640 1 --2- 172.21.15.73:0/3081471038 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5a40b0c30 0x7ff5a415ee50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff5abc00640 1 --2- 172.21.15.73:0/3081471038 >> v2:172.21.15.73:3300/0 conn(0x7ff5a40b1d20 0x7ff5a40cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff5abc00640 1 -- 172.21.15.73:0/3081471038 >> 172.21.15.73:0/3081471038 conn(0x7ff5a4098a40 msgr2=0x7ff5a40b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:49.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff5abc00640 1 -- 172.21.15.73:0/3081471038 shutdown_connections 2024-03-29T08:17:49.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff5abc00640 1 -- 172.21.15.73:0/3081471038 wait complete. 2024-03-29T08:17:49.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff5abc00640 1 Processor -- start 2024-03-29T08:17:49.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff5abc00640 1 -- start start 2024-03-29T08:17:49.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff5abc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a40b0c30 0x7ff5a406e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff5abc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5a40b1d20 0x7ff5a40c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff5abc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5a415aa70 0x7ff5a40c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:49.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff5abc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a40b34c0 con 0x7ff5a415aa70 2024-03-29T08:17:49.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff5abc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a40b2940 con 0x7ff5a40b0c30 2024-03-29T08:17:49.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff5abc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a40b2c20 con 0x7ff5a40b1d20 2024-03-29T08:17:49.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff59be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5a40b1d20 0x7ff5a40c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff5a9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a40b0c30 0x7ff5a406e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff59be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5a40b1d20 0x7ff5a40c7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.73:35674/0 (socket says 172.21.15.73:35674) 2024-03-29T08:17:49.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff59be00640 1 -- 172.21.15.73:0/458307861 learned_addr learned my addr 172.21.15.73:0/458307861 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-03-29T08:17:49.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff5a9c00640 1 --2- 172.21.15.73:0/458307861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5a415aa70 0x7ff5a40c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:49.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff59be00640 1 -- 172.21.15.73:0/458307861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a40b0c30 msgr2=0x7ff5a406e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff59be00640 1 --2- 172.21.15.73:0/458307861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a40b0c30 0x7ff5a406e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff59be00640 1 -- 172.21.15.73:0/458307861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5a415aa70 msgr2=0x7ff5a40c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff59be00640 1 --2- 172.21.15.73:0/458307861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5a415aa70 0x7ff5a40c6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff59be00640 1 -- 172.21.15.73:0/458307861 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c009a70 con 0x7ff5a40b1d20 2024-03-29T08:17:49.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff59be00640 1 --2- 172.21.15.73:0/458307861 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5a40b1d20 0x7ff5a40c7da0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0ab040 tx=0x7ff58c01ad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:49.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff5a1600640 1 -- 172.21.15.73:0/458307861 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0b1800 con 0x7ff5a40b1d20 2024-03-29T08:17:49.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.835+0000 7ff5a1600640 1 -- 172.21.15.73:0/458307861 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c0b1980 con 0x7ff5a40b1d20 2024-03-29T08:17:49.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.839+0000 7ff5a1600640 1 -- 172.21.15.73:0/458307861 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c0b1d30 con 0x7ff5a40b1d20 2024-03-29T08:17:49.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.839+0000 7ff5abc00640 1 -- 172.21.15.73:0/458307861 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58c0094d0 con 0x7ff5a40b1d20 2024-03-29T08:17:49.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.839+0000 7ff5abc00640 1 -- 172.21.15.73:0/458307861 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a40b31e0 con 0x7ff5a40b1d20 2024-03-29T08:17:49.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.839+0000 7ff5a1600640 1 -- 172.21.15.73:0/458307861 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff58c0bc450 con 0x7ff5a40b1d20 2024-03-29T08:17:49.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.839+0000 7ff5a1600640 1 -- 172.21.15.73:0/458307861 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7ff58c0a9570 con 0x7ff5a40b1d20 2024-03-29T08:17:49.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.839+0000 7ff5abc00640 1 -- 172.21.15.73:0/458307861 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a40c33e0 con 0x7ff5a40b1d20 2024-03-29T08:17:49.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.839+0000 7ff5a1600640 1 -- 172.21.15.73:0/458307861 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7ff5a40c33e0 con 0x7ff5a40b1d20 2024-03-29T08:17:49.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.907+0000 7ff5abc00640 1 -- 172.21.15.73:0/458307861 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "osd new", "uuid": "55692b87-e81a-48db-bb4a-37b2b44062e6", "id": 4} v 0) v1 -- 0x7ff5a409d980 con 0x7ff5a40b1d20 2024-03-29T08:17:49.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.911+0000 7ff5a1600640 1 -- 172.21.15.73:0/458307861 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "55692b87-e81a-48db-bb4a-37b2b44062e6", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff58c0c1020 con 0x7ff5a40b1d20 2024-03-29T08:17:49.913 INFO:teuthology.orchestra.run.smithi073.stdout:4 2024-03-29T08:17:49.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.911+0000 7ff5abc00640 1 -- 172.21.15.73:0/458307861 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5a40b1d20 msgr2=0x7ff5a40c7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:49.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.911+0000 7ff5abc00640 1 --2- 172.21.15.73:0/458307861 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5a40b1d20 0x7ff5a40c7da0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0ab040 tx=0x7ff58c01ad70 comp rx=0 tx=0).stop 2024-03-29T08:17:49.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.911+0000 7ff5abc00640 1 -- 172.21.15.73:0/458307861 shutdown_connections 2024-03-29T08:17:49.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.911+0000 7ff5abc00640 1 --2- 172.21.15.73:0/458307861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5a415aa70 0x7ff5a40c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.911+0000 7ff5abc00640 1 --2- 172.21.15.73:0/458307861 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5a40b1d20 0x7ff5a40c7da0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.911+0000 7ff5abc00640 1 --2- 172.21.15.73:0/458307861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a40b0c30 0x7ff5a406e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:49.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.911+0000 7ff5abc00640 1 -- 172.21.15.73:0/458307861 >> 172.21.15.73:0/458307861 conn(0x7ff5a4098a40 msgr2=0x7ff5a40b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.911+0000 7ff5abc00640 1 -- 172.21.15.73:0/458307861 shutdown_connections 2024-03-29T08:17:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:49.911+0000 7ff5abc00640 1 -- 172.21.15.73:0/458307861 wait complete. 2024-03-29T08:17:49.916 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:49.915+0000 7f9d37c4b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-29T08:17:49.932 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd new 812dd47d-6a75-4797-be49-c324af4ff32b 5 2024-03-29T08:17:50.031 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:50.029+0000 7fc9d5e9b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-29T08:17:50.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.063+0000 7fa701200640 1 Processor -- start 2024-03-29T08:17:50.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.063+0000 7fa701200640 1 -- start start 2024-03-29T08:17:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.067+0000 7fa701200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6fc15a790 0x7fa6fc0cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.067+0000 7fa701200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc0cc660 con 0x7fa6fc0cf340 2024-03-29T08:17:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.067+0000 7fa701200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc0cc0a0 con 0x7fa6fc0cbbe0 2024-03-29T08:17:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.067+0000 7fa701200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0cc380 con 0x7fa6fc15a790 2024-03-29T08:17:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.067+0000 7fa6f9e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6fc15a790 0x7fa6fc0cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.067+0000 7fa6f9400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa6fc0cbbe0 0x7fa6fc0d0650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51248/0 (socket says 172.21.15.73:51248) 2024-03-29T08:17:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.067+0000 7fa6f9e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6fc15a790 0x7fa6fc0cee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.73:35690/0 (socket says 172.21.15.73:35690) 2024-03-29T08:17:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.067+0000 7fa6f9400640 1 -- 172.21.15.73:0/856969162 learned_addr learned my addr 172.21.15.73:0/856969162 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-03-29T08:17:50.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.067+0000 7fa6f9e00640 1 -- 172.21.15.73:0/856969162 >> v1:172.21.15.73:6789/0 conn(0x7fa6fc0cbbe0 legacy=0x7fa6fc0d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.067+0000 7fa6f9e00640 1 -- 172.21.15.73:0/856969162 >> v1:172.21.15.46:6789/0 conn(0x7fa6fc0cf340 legacy=0x7fa6fc0b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.067+0000 7fa6f9e00640 1 -- 172.21.15.73:0/856969162 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc0b0380 con 0x7fa6fc15a790 2024-03-29T08:17:50.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa6f9e00640 1 --2- 172.21.15.73:0/856969162 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6fc15a790 0x7fa6fc0cee00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0097f0 tx=0x7fa6ec01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=268938e65bc8cf4f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:50.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa6f8a00640 1 -- 172.21.15.73:0/856969162 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0ab640 con 0x7fa6fc15a790 2024-03-29T08:17:50.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa6f8a00640 1 -- 172.21.15.73:0/856969162 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec0a7070 con 0x7fa6fc15a790 2024-03-29T08:17:50.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa6f8a00640 1 -- 172.21.15.73:0/856969162 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0ab9c0 con 0x7fa6fc15a790 2024-03-29T08:17:50.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa701200640 1 -- 172.21.15.73:0/856969162 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6fc15a790 msgr2=0x7fa6fc0cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa701200640 1 --2- 172.21.15.73:0/856969162 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6fc15a790 0x7fa6fc0cee00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0097f0 tx=0x7fa6ec01c420 comp rx=0 tx=0).stop 2024-03-29T08:17:50.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa701200640 1 -- 172.21.15.73:0/856969162 shutdown_connections 2024-03-29T08:17:50.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa701200640 1 --2- 172.21.15.73:0/856969162 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6fc15a790 0x7fa6fc0cee00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa701200640 1 -- 172.21.15.73:0/856969162 >> 172.21.15.73:0/856969162 conn(0x7fa6fc099a10 msgr2=0x7fa6fc089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:50.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa701200640 1 -- 172.21.15.73:0/856969162 shutdown_connections 2024-03-29T08:17:50.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa701200640 1 -- 172.21.15.73:0/856969162 wait complete. 2024-03-29T08:17:50.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa701200640 1 Processor -- start 2024-03-29T08:17:50.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa701200640 1 -- start start 2024-03-29T08:17:50.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa701200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6fc0cbbe0 0x7fa6fc151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa701200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6fc0cf340 0x7fa6fc155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa701200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6fc15a790 0x7fa6fc145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa701200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc150ea0 con 0x7fa6fc15a790 2024-03-29T08:17:50.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa701200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc1550c0 con 0x7fa6fc0cf340 2024-03-29T08:17:50.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa701200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc150850 con 0x7fa6fc0cbbe0 2024-03-29T08:17:50.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa6f9e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6fc0cbbe0 0x7fa6fc151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.071+0000 7fa6f9400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6fc0cf340 0x7fa6fc155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.075+0000 7fa6f9e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6fc0cbbe0 0x7fa6fc151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.73:35692/0 (socket says 172.21.15.73:35692) 2024-03-29T08:17:50.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.075+0000 7fa6f9e00640 1 -- 172.21.15.73:0/269986150 learned_addr learned my addr 172.21.15.73:0/269986150 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-03-29T08:17:50.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.075+0000 7fa6fa800640 1 --2- 172.21.15.73:0/269986150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6fc15a790 0x7fa6fc145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.075+0000 7fa6f9e00640 1 -- 172.21.15.73:0/269986150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6fc0cf340 msgr2=0x7fa6fc155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.075+0000 7fa6f9e00640 1 --2- 172.21.15.73:0/269986150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6fc0cf340 0x7fa6fc155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.075+0000 7fa6f9e00640 1 -- 172.21.15.73:0/269986150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6fc15a790 msgr2=0x7fa6fc145410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.075+0000 7fa6f9e00640 1 --2- 172.21.15.73:0/269986150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6fc15a790 0x7fa6fc145410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.075+0000 7fa6f9e00640 1 -- 172.21.15.73:0/269986150 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec009420 con 0x7fa6fc0cbbe0 2024-03-29T08:17:50.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.075+0000 7fa6f9e00640 1 --2- 172.21.15.73:0/269986150 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6fc0cbbe0 0x7fa6fc151330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec01a050 tx=0x7fa6ec0191c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:50.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.075+0000 7fa6f2000640 1 -- 172.21.15.73:0/269986150 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0abbf0 con 0x7fa6fc0cbbe0 2024-03-29T08:17:50.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.075+0000 7fa6f2000640 1 -- 172.21.15.73:0/269986150 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec0abd90 con 0x7fa6fc0cbbe0 2024-03-29T08:17:50.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.075+0000 7fa701200640 1 -- 172.21.15.73:0/269986150 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6fc150b30 con 0x7fa6fc0cbbe0 2024-03-29T08:17:50.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.075+0000 7fa6f2000640 1 -- 172.21.15.73:0/269986150 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0bf690 con 0x7fa6fc0cbbe0 2024-03-29T08:17:50.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.075+0000 7fa701200640 1 -- 172.21.15.73:0/269986150 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6fc145950 con 0x7fa6fc0cbbe0 2024-03-29T08:17:50.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.075+0000 7fa6f2000640 1 -- 172.21.15.73:0/269986150 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa6ec0bfaf0 con 0x7fa6fc0cbbe0 2024-03-29T08:17:50.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.075+0000 7fa6f2000640 1 -- 172.21.15.73:0/269986150 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fa6ec0b4970 con 0x7fa6fc0cbbe0 2024-03-29T08:17:50.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.075+0000 7fa701200640 1 -- 172.21.15.73:0/269986150 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6fc09e7d0 con 0x7fa6fc0cbbe0 2024-03-29T08:17:50.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.079+0000 7fa6f2000640 1 -- 172.21.15.73:0/269986150 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7fa6ec0c83f0 con 0x7fa6fc0cbbe0 2024-03-29T08:17:50.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.147+0000 7fa701200640 1 -- 172.21.15.73:0/269986150 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "osd new", "uuid": "812dd47d-6a75-4797-be49-c324af4ff32b", "id": 5} v 0) v1 -- 0x7fa6fc150990 con 0x7fa6fc0cbbe0 2024-03-29T08:17:50.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.151+0000 7fa6f2000640 1 -- 172.21.15.73:0/269986150 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "812dd47d-6a75-4797-be49-c324af4ff32b", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa6fc150990 con 0x7fa6fc0cbbe0 2024-03-29T08:17:50.152 INFO:teuthology.orchestra.run.smithi073.stdout:5 2024-03-29T08:17:50.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.151+0000 7fa701200640 1 -- 172.21.15.73:0/269986150 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6fc0cbbe0 msgr2=0x7fa6fc151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.151+0000 7fa701200640 1 --2- 172.21.15.73:0/269986150 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6fc0cbbe0 0x7fa6fc151330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec01a050 tx=0x7fa6ec0191c0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.151+0000 7fa701200640 1 -- 172.21.15.73:0/269986150 shutdown_connections 2024-03-29T08:17:50.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.151+0000 7fa701200640 1 --2- 172.21.15.73:0/269986150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6fc0cf340 0x7fa6fc155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.151+0000 7fa701200640 1 --2- 172.21.15.73:0/269986150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6fc15a790 0x7fa6fc145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.151+0000 7fa701200640 1 --2- 172.21.15.73:0/269986150 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6fc0cbbe0 0x7fa6fc151330 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.151+0000 7fa701200640 1 -- 172.21.15.73:0/269986150 >> 172.21.15.73:0/269986150 conn(0x7fa6fc099a10 msgr2=0x7fa6fc0af030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:50.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.151+0000 7fa701200640 1 -- 172.21.15.73:0/269986150 shutdown_connections 2024-03-29T08:17:50.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.151+0000 7fa701200640 1 -- 172.21.15.73:0/269986150 wait complete. 2024-03-29T08:17:50.178 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd new 649cf36a-83c0-489b-9ac7-986efc89f190 6 2024-03-29T08:17:50.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.311+0000 7f2e3dc00640 1 Processor -- start 2024-03-29T08:17:50.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.311+0000 7f2e3dc00640 1 -- start start 2024-03-29T08:17:50.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.311+0000 7f2e3dc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2e3806e840 0x7f2e3806ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.311+0000 7f2e3dc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e381649a0 0x7f2e38166d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.311+0000 7f2e3dc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e3806f290 con 0x7f2e38069d90 2024-03-29T08:17:50.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.311+0000 7f2e3dc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e38160680 con 0x7f2e3806e840 2024-03-29T08:17:50.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.311+0000 7f2e3dc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e38160960 con 0x7f2e381649a0 2024-03-29T08:17:50.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.311+0000 7f2e2fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2e3806e840 0x7f2e3806ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.311+0000 7f2e37400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e381649a0 0x7f2e38166d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.311+0000 7f2e2fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2e3806e840 0x7f2e3806ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39866/0 (socket says 172.21.15.73:39866) 2024-03-29T08:17:50.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.311+0000 7f2e2fe00640 1 -- 172.21.15.73:0/3902605198 learned_addr learned my addr 172.21.15.73:0/3902605198 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-03-29T08:17:50.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.311+0000 7f2e2fe00640 1 -- 172.21.15.73:0/3902605198 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e381649a0 msgr2=0x7f2e38166d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:17:50.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.311+0000 7f2e2fe00640 1 --2- 172.21.15.73:0/3902605198 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e381649a0 0x7f2e38166d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.311+0000 7f2e2fe00640 1 -- 172.21.15.73:0/3902605198 >> v1:172.21.15.46:6789/0 conn(0x7f2e38069d90 legacy=0x7f2e3806a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.311+0000 7f2e2fe00640 1 -- 172.21.15.73:0/3902605198 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e38160c40 con 0x7f2e3806e840 2024-03-29T08:17:50.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.315+0000 7f2e2fe00640 1 --2- 172.21.15.73:0/3902605198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e3806e840 0x7f2e3806ecb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009870 tx=0x7f2e2001ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc9727a45cfe7848 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:50.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.315+0000 7f2e36000640 1 -- 172.21.15.73:0/3902605198 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e200a8a50 con 0x7f2e3806e840 2024-03-29T08:17:50.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.315+0000 7f2e36000640 1 -- 172.21.15.73:0/3902605198 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e200a8bb0 con 0x7f2e3806e840 2024-03-29T08:17:50.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.315+0000 7f2e36000640 1 -- 172.21.15.73:0/3902605198 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e200b0440 con 0x7f2e3806e840 2024-03-29T08:17:50.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.315+0000 7f2e3dc00640 1 -- 172.21.15.73:0/3902605198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e3806e840 msgr2=0x7f2e3806ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.315+0000 7f2e3dc00640 1 --2- 172.21.15.73:0/3902605198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e3806e840 0x7f2e3806ecb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009870 tx=0x7f2e2001ed50 comp rx=0 tx=0).stop 2024-03-29T08:17:50.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.315+0000 7f2e3dc00640 1 -- 172.21.15.73:0/3902605198 shutdown_connections 2024-03-29T08:17:50.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.315+0000 7f2e3dc00640 1 --2- 172.21.15.73:0/3902605198 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e381649a0 0x7f2e38166d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.315+0000 7f2e3dc00640 1 --2- 172.21.15.73:0/3902605198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e3806e840 0x7f2e3806ecb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.315+0000 7f2e3dc00640 1 -- 172.21.15.73:0/3902605198 >> 172.21.15.73:0/3902605198 conn(0x7f2e38098a40 msgr2=0x7f2e380cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:50.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.315+0000 7f2e3dc00640 1 -- 172.21.15.73:0/3902605198 shutdown_connections 2024-03-29T08:17:50.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.315+0000 7f2e3dc00640 1 -- 172.21.15.73:0/3902605198 wait complete. 2024-03-29T08:17:50.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e3dc00640 1 Processor -- start 2024-03-29T08:17:50.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e3dc00640 1 -- start start 2024-03-29T08:17:50.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e3dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e38069d90 0x7f2e380c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e3dc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e3806e840 0x7f2e38149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e3dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e381649a0 0x7f2e3814a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e3dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e380c83e0 con 0x7f2e381649a0 2024-03-29T08:17:50.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e3dc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e380c7760 con 0x7f2e38069d90 2024-03-29T08:17:50.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e3dc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e380c7a40 con 0x7f2e3806e840 2024-03-29T08:17:50.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e2fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e3806e840 0x7f2e38149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e36a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e38069d90 0x7f2e380c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e2fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e3806e840 0x7f2e38149bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.73:35704/0 (socket says 172.21.15.73:35704) 2024-03-29T08:17:50.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e2fe00640 1 -- 172.21.15.73:0/1504788211 learned_addr learned my addr 172.21.15.73:0/1504788211 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-03-29T08:17:50.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e37400640 1 --2- 172.21.15.73:0/1504788211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e381649a0 0x7f2e3814a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e2fe00640 1 -- 172.21.15.73:0/1504788211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e38069d90 msgr2=0x7f2e380c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e2fe00640 1 --2- 172.21.15.73:0/1504788211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e38069d90 0x7f2e380c6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e2fe00640 1 -- 172.21.15.73:0/1504788211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e381649a0 msgr2=0x7f2e3814a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e2fe00640 1 --2- 172.21.15.73:0/1504788211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e381649a0 0x7f2e3814a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e2fe00640 1 -- 172.21.15.73:0/1504788211 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e200094d0 con 0x7f2e3806e840 2024-03-29T08:17:50.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e37400640 1 --2- 172.21.15.73:0/1504788211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e381649a0 0x7f2e3814a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:17:50.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e2fe00640 1 --2- 172.21.15.73:0/1504788211 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e3806e840 0x7f2e38149bd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e200b3040 tx=0x7f2e2001baf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:50.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e2ea00640 1 -- 172.21.15.73:0/1504788211 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e200b0dd0 con 0x7f2e3806e840 2024-03-29T08:17:50.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e2ea00640 1 -- 172.21.15.73:0/1504788211 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e200bb460 con 0x7f2e3806e840 2024-03-29T08:17:50.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e2ea00640 1 -- 172.21.15.73:0/1504788211 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e200bb810 con 0x7f2e3806e840 2024-03-29T08:17:50.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e3dc00640 1 -- 172.21.15.73:0/1504788211 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e380c7d20 con 0x7f2e3806e840 2024-03-29T08:17:50.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e3dc00640 1 -- 172.21.15.73:0/1504788211 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3815b3d0 con 0x7f2e3806e840 2024-03-29T08:17:50.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e2ea00640 1 -- 172.21.15.73:0/1504788211 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2e200bbc70 con 0x7f2e3806e840 2024-03-29T08:17:50.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.319+0000 7f2e2ea00640 1 -- 172.21.15.73:0/1504788211 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f2e200b89e0 con 0x7f2e3806e840 2024-03-29T08:17:50.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.323+0000 7f2e3dc00640 1 -- 172.21.15.73:0/1504788211 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e380c8520 con 0x7f2e3806e840 2024-03-29T08:17:50.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.323+0000 7f2e2ea00640 1 -- 172.21.15.73:0/1504788211 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f2e380c8520 con 0x7f2e3806e840 2024-03-29T08:17:50.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.391+0000 7f2e3dc00640 1 -- 172.21.15.73:0/1504788211 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "osd new", "uuid": "649cf36a-83c0-489b-9ac7-986efc89f190", "id": 6} v 0) v1 -- 0x7f2e380c7b80 con 0x7f2e3806e840 2024-03-29T08:17:50.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.399+0000 7f2e2ea00640 1 -- 172.21.15.73:0/1504788211 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "649cf36a-83c0-489b-9ac7-986efc89f190", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2e380c7b80 con 0x7f2e3806e840 2024-03-29T08:17:50.400 INFO:teuthology.orchestra.run.smithi073.stdout:6 2024-03-29T08:17:50.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.399+0000 7f2e3dc00640 1 -- 172.21.15.73:0/1504788211 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e3806e840 msgr2=0x7f2e38149bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.399+0000 7f2e3dc00640 1 --2- 172.21.15.73:0/1504788211 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e3806e840 0x7f2e38149bd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2e200b3040 tx=0x7f2e2001baf0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.399+0000 7f2e3dc00640 1 -- 172.21.15.73:0/1504788211 shutdown_connections 2024-03-29T08:17:50.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.399+0000 7f2e3dc00640 1 --2- 172.21.15.73:0/1504788211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e381649a0 0x7f2e3814a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.399+0000 7f2e3dc00640 1 --2- 172.21.15.73:0/1504788211 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e3806e840 0x7f2e38149bd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.399+0000 7f2e3dc00640 1 --2- 172.21.15.73:0/1504788211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e38069d90 0x7f2e380c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.399+0000 7f2e3dc00640 1 -- 172.21.15.73:0/1504788211 >> 172.21.15.73:0/1504788211 conn(0x7f2e38098a40 msgr2=0x7f2e380ac380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.399+0000 7f2e3dc00640 1 -- 172.21.15.73:0/1504788211 shutdown_connections 2024-03-29T08:17:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.399+0000 7f2e3dc00640 1 -- 172.21.15.73:0/1504788211 wait complete. 2024-03-29T08:17:50.418 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd new d4175336-2aac-4a9d-8de1-e54d0040cce7 7 2024-03-29T08:17:50.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.551+0000 7f5904c00640 1 Processor -- start 2024-03-29T08:17:50.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.551+0000 7f5904c00640 1 -- start start 2024-03-29T08:17:50.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.555+0000 7f5904c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59000b1cb0 0x7f59000cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.555+0000 7f5904c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590015cf40 con 0x7f59000d0dc0 2024-03-29T08:17:50.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.555+0000 7f5904c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59000b0250 con 0x7f590015cb60 2024-03-29T08:17:50.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.555+0000 7f5904c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59000b0810 con 0x7f59000b1cb0 2024-03-29T08:17:50.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.555+0000 7f58fe800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59000b1cb0 0x7f59000cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.555+0000 7f58fde00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f590015cb60 0x7f59000d06b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51258/0 (socket says 172.21.15.73:51258) 2024-03-29T08:17:50.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.555+0000 7f58fe800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59000b1cb0 0x7f59000cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.73:35720/0 (socket says 172.21.15.73:35720) 2024-03-29T08:17:50.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.555+0000 7f58fde00640 1 -- 172.21.15.73:0/423683122 learned_addr learned my addr 172.21.15.73:0/423683122 (peer_addr_for_me v1:172.21.15.73:0/0) 2024-03-29T08:17:50.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.555+0000 7f58fe800640 1 -- 172.21.15.73:0/423683122 >> v1:172.21.15.73:6789/0 conn(0x7f590015cb60 legacy=0x7f59000d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.555+0000 7f58fe800640 1 -- 172.21.15.73:0/423683122 >> v1:172.21.15.46:6789/0 conn(0x7f59000d0dc0 legacy=0x7f59000cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.555+0000 7f58fe800640 1 -- 172.21.15.73:0/423683122 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59000d0010 con 0x7f59000b1cb0 2024-03-29T08:17:50.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.555+0000 7f58fca00640 1 -- 172.21.15.73:0/423683122 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153488592 0 0) 0x7f59000b0250 con 0x7f590015cb60 2024-03-29T08:17:50.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.555+0000 7f58fe800640 1 --2- 172.21.15.73:0/423683122 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59000b1cb0 0x7f59000cf9f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009dc0 tx=0x7f58f801c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7049965a1d373f70 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:50.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.555+0000 7f58fca00640 1 -- 172.21.15.73:0/423683122 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f80a5a20 con 0x7f59000b1cb0 2024-03-29T08:17:50.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f58fca00640 1 -- 172.21.15.73:0/423683122 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f80a5b80 con 0x7f59000b1cb0 2024-03-29T08:17:50.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f58fca00640 1 -- 172.21.15.73:0/423683122 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f80ad440 con 0x7f59000b1cb0 2024-03-29T08:17:50.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f5904c00640 1 -- 172.21.15.73:0/423683122 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59000b1cb0 msgr2=0x7f59000cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f5904c00640 1 --2- 172.21.15.73:0/423683122 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59000b1cb0 0x7f59000cf9f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009dc0 tx=0x7f58f801c960 comp rx=0 tx=0).stop 2024-03-29T08:17:50.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f5904c00640 1 -- 172.21.15.73:0/423683122 shutdown_connections 2024-03-29T08:17:50.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f5904c00640 1 --2- 172.21.15.73:0/423683122 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59000b1cb0 0x7f59000cf9f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f5904c00640 1 -- 172.21.15.73:0/423683122 >> 172.21.15.73:0/423683122 conn(0x7f5900099a10 msgr2=0x7f5900089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:50.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f5904c00640 1 -- 172.21.15.73:0/423683122 shutdown_connections 2024-03-29T08:17:50.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f5904c00640 1 -- 172.21.15.73:0/423683122 wait complete. 2024-03-29T08:17:50.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f5904c00640 1 Processor -- start 2024-03-29T08:17:50.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f5904c00640 1 -- start start 2024-03-29T08:17:50.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f5904c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59000b1cb0 0x7f5900153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f5904c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59000d0dc0 0x7f5900147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f5904c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f590015cb60 0x7f59000ce860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f5904c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59001498d0 con 0x7f59000d0dc0 2024-03-29T08:17:50.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f5904c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5900149bb0 con 0x7f590015cb60 2024-03-29T08:17:50.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f5904c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5900149e90 con 0x7f59000b1cb0 2024-03-29T08:17:50.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f58fde00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59000b1cb0 0x7f5900153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f58fde00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59000b1cb0 0x7f5900153770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.73:35736/0 (socket says 172.21.15.73:35736) 2024-03-29T08:17:50.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f58fde00640 1 -- 172.21.15.73:0/754527725 learned_addr learned my addr 172.21.15.73:0/754527725 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-03-29T08:17:50.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f58fe800640 1 --2- 172.21.15.73:0/754527725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f590015cb60 0x7f59000ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.559+0000 7f58fd400640 1 --2- 172.21.15.73:0/754527725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59000d0dc0 0x7f5900147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.563+0000 7f58fe800640 1 -- 172.21.15.73:0/754527725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59000b1cb0 msgr2=0x7f5900153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.563+0000 7f58fe800640 1 --2- 172.21.15.73:0/754527725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59000b1cb0 0x7f5900153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.563+0000 7f58fe800640 1 -- 172.21.15.73:0/754527725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59000d0dc0 msgr2=0x7f5900147260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.563+0000 7f58fe800640 1 --2- 172.21.15.73:0/754527725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59000d0dc0 0x7f5900147260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.563+0000 7f58fe800640 1 -- 172.21.15.73:0/754527725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f8009a70 con 0x7f590015cb60 2024-03-29T08:17:50.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.563+0000 7f58fde00640 1 --2- 172.21.15.73:0/754527725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59000b1cb0 0x7f5900153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:17:50.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.563+0000 7f58fe800640 1 --2- 172.21.15.73:0/754527725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f590015cb60 0x7f59000ce860 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f58f80ac000 tx=0x7f58f8019700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:50.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.563+0000 7f58f6000640 1 -- 172.21.15.73:0/754527725 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f80ade00 con 0x7f590015cb60 2024-03-29T08:17:50.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.563+0000 7f58f6000640 1 -- 172.21.15.73:0/754527725 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f80b8420 con 0x7f590015cb60 2024-03-29T08:17:50.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.563+0000 7f58f6000640 1 -- 172.21.15.73:0/754527725 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f80b8790 con 0x7f590015cb60 2024-03-29T08:17:50.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.563+0000 7f5904c00640 1 -- 172.21.15.73:0/754527725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590014a170 con 0x7f590015cb60 2024-03-29T08:17:50.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.563+0000 7f5904c00640 1 -- 172.21.15.73:0/754527725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590014a950 con 0x7f590015cb60 2024-03-29T08:17:50.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.563+0000 7f58f6000640 1 -- 172.21.15.73:0/754527725 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f58f80b8bf0 con 0x7f590015cb60 2024-03-29T08:17:50.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.563+0000 7f58f6000640 1 -- 172.21.15.73:0/754527725 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f58f80c0c20 con 0x7f590015cb60 2024-03-29T08:17:50.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.563+0000 7f5904c00640 1 -- 172.21.15.73:0/754527725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590009e7d0 con 0x7f590015cb60 2024-03-29T08:17:50.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.567+0000 7f58f6000640 1 -- 172.21.15.73:0/754527725 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f58f80bc070 con 0x7f590015cb60 2024-03-29T08:17:50.606 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:50.603+0000 7f9d37c4b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-29T08:17:50.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.631+0000 7f5904c00640 1 -- 172.21.15.73:0/754527725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d4175336-2aac-4a9d-8de1-e54d0040cce7", "id": 7} v 0) v1 -- 0x7f59000b2e80 con 0x7f590015cb60 2024-03-29T08:17:50.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.639+0000 7f58f6000640 1 -- 172.21.15.73:0/754527725 <== mon.1 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d4175336-2aac-4a9d-8de1-e54d0040cce7", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f58f80b2070 con 0x7f590015cb60 2024-03-29T08:17:50.641 INFO:teuthology.orchestra.run.smithi073.stdout:7 2024-03-29T08:17:50.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.639+0000 7f5904c00640 1 -- 172.21.15.73:0/754527725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f590015cb60 msgr2=0x7f59000ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.639+0000 7f5904c00640 1 --2- 172.21.15.73:0/754527725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f590015cb60 0x7f59000ce860 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f58f80ac000 tx=0x7f58f8019700 comp rx=0 tx=0).stop 2024-03-29T08:17:50.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.639+0000 7f5904c00640 1 -- 172.21.15.73:0/754527725 shutdown_connections 2024-03-29T08:17:50.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.639+0000 7f5904c00640 1 --2- 172.21.15.73:0/754527725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f590015cb60 0x7f59000ce860 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.639+0000 7f5904c00640 1 --2- 172.21.15.73:0/754527725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59000d0dc0 0x7f5900147260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.639+0000 7f5904c00640 1 --2- 172.21.15.73:0/754527725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59000b1cb0 0x7f5900153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.639+0000 7f5904c00640 1 -- 172.21.15.73:0/754527725 >> 172.21.15.73:0/754527725 conn(0x7f5900099a10 msgr2=0x7f59000cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:50.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.639+0000 7f5904c00640 1 -- 172.21.15.73:0/754527725 shutdown_connections 2024-03-29T08:17:50.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-03-29T08:17:50.639+0000 7f5904c00640 1 -- 172.21.15.73:0/754527725 wait complete. 2024-03-29T08:17:50.660 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-29T08:17:50.660 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-29T08:17:50.663 INFO:tasks.ceph.osd.0:Started 2024-03-29T08:17:50.663 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-29T08:17:50.663 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-29T08:17:50.666 INFO:tasks.ceph.osd.1:Started 2024-03-29T08:17:50.667 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-29T08:17:50.667 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-29T08:17:50.670 INFO:tasks.ceph.osd.2:Started 2024-03-29T08:17:50.670 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-29T08:17:50.670 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-29T08:17:50.671 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:50.667+0000 7f9d37c4b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-29T08:17:50.674 INFO:tasks.ceph.osd.3:Started 2024-03-29T08:17:50.674 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-29T08:17:50.675 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-29T08:17:50.676 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:50.677+0000 7fc9d5e9b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-29T08:17:50.677 INFO:tasks.ceph.osd.4:Started 2024-03-29T08:17:50.677 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-29T08:17:50.678 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-29T08:17:50.680 INFO:tasks.ceph.osd.5:Started 2024-03-29T08:17:50.680 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-29T08:17:50.680 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-29T08:17:50.683 INFO:tasks.ceph.osd.6:Started 2024-03-29T08:17:50.683 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-29T08:17:50.684 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-29T08:17:50.688 INFO:tasks.ceph.osd.7:Started 2024-03-29T08:17:50.688 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:17:50.742 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:50.739+0000 7f9d37c4b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-29T08:17:50.760 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:50.761+0000 7fc9d5e9b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-29T08:17:50.831 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:50.829+0000 7fc9d5e9b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-29T08:17:50.853 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:50.851+0000 7f9d37c4b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-29T08:17:50.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.869+0000 7faf3f600640 1 Processor -- start 2024-03-29T08:17:50.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.869+0000 7faf3f600640 1 -- start start 2024-03-29T08:17:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.869+0000 7faf3f600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faf3815ad20 0x7faf380cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.869+0000 7faf3f600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faf380b0540 0x7faf380cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.869+0000 7faf3f600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf380b0ab0 con 0x7faf3815a940 2024-03-29T08:17:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.869+0000 7faf3f600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf380b0d90 con 0x7faf380b0540 2024-03-29T08:17:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.869+0000 7faf3f600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf380b1070 con 0x7faf3815ad20 2024-03-29T08:17:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.869+0000 7faf3d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faf380b0540 0x7faf380cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.869+0000 7faf2fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faf3815ad20 0x7faf380cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.869+0000 7faf3d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faf380b0540 0x7faf380cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38396/0 (socket says 172.21.15.46:38396) 2024-03-29T08:17:50.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.869+0000 7faf3d600640 1 -- 172.21.15.46:0/2809322507 learned_addr learned my addr 172.21.15.46:0/2809322507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:17:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.869+0000 7faf3d600640 1 -- 172.21.15.46:0/2809322507 >> v2:172.21.15.73:3300/0 conn(0x7faf3815ad20 msgr2=0x7faf380cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:17:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.869+0000 7faf3d600640 1 --2- 172.21.15.46:0/2809322507 >> v2:172.21.15.73:3300/0 conn(0x7faf3815ad20 0x7faf380cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.869+0000 7faf3d600640 1 -- 172.21.15.46:0/2809322507 >> v1:172.21.15.73:6789/0 conn(0x7faf3815a940 legacy=0x7faf380afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.869+0000 7faf3d600640 1 -- 172.21.15.46:0/2809322507 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380b1350 con 0x7faf380b0540 2024-03-29T08:17:50.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.877+0000 7faf3d600640 1 --2- 172.21.15.46:0/2809322507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf380b0540 0x7faf380cf1f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faf380c41d0 tx=0x7faf3001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e5a8b37bb9425 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:50.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.877+0000 7faf37e00640 1 -- 172.21.15.46:0/2809322507 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf300a5810 con 0x7faf380b0540 2024-03-29T08:17:50.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.877+0000 7faf37e00640 1 -- 172.21.15.46:0/2809322507 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf300a5970 con 0x7faf380b0540 2024-03-29T08:17:50.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.877+0000 7faf37e00640 1 -- 172.21.15.46:0/2809322507 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf300a5d00 con 0x7faf380b0540 2024-03-29T08:17:50.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.877+0000 7faf3f600640 1 -- 172.21.15.46:0/2809322507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf380b0540 msgr2=0x7faf380cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.877+0000 7faf3f600640 1 --2- 172.21.15.46:0/2809322507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf380b0540 0x7faf380cf1f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faf380c41d0 tx=0x7faf3001c930 comp rx=0 tx=0).stop 2024-03-29T08:17:50.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.877+0000 7faf3f600640 1 -- 172.21.15.46:0/2809322507 shutdown_connections 2024-03-29T08:17:50.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.877+0000 7faf3f600640 1 --2- 172.21.15.46:0/2809322507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf380b0540 0x7faf380cf1f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.877+0000 7faf3f600640 1 --2- 172.21.15.46:0/2809322507 >> v2:172.21.15.73:3300/0 conn(0x7faf3815ad20 0x7faf380cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.877+0000 7faf3f600640 1 -- 172.21.15.46:0/2809322507 >> 172.21.15.46:0/2809322507 conn(0x7faf38089fd0 msgr2=0x7faf3808a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:50.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.877+0000 7faf3f600640 1 -- 172.21.15.46:0/2809322507 shutdown_connections 2024-03-29T08:17:50.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.877+0000 7faf3f600640 1 -- 172.21.15.46:0/2809322507 wait complete. 2024-03-29T08:17:50.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.877+0000 7faf3f600640 1 Processor -- start 2024-03-29T08:17:50.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.877+0000 7faf3f600640 1 -- start start 2024-03-29T08:17:50.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf3f600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf380b0540 0x7faf38158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf3f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3815a940 0x7faf381599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf3f600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf3815ad20 0x7faf380b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:50.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf3f600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3815c0f0 con 0x7faf3815a940 2024-03-29T08:17:50.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf3f600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3815bdc0 con 0x7faf3815ad20 2024-03-29T08:17:50.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf3f600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf380c81b0 con 0x7faf380b0540 2024-03-29T08:17:50.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3815a940 0x7faf381599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf3cc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf380b0540 0x7faf38158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3815a940 0x7faf381599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38400/0 (socket says 172.21.15.46:38400) 2024-03-29T08:17:50.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf2fe00640 1 -- 172.21.15.46:0/1636932805 learned_addr learned my addr 172.21.15.46:0/1636932805 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:17:50.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf3d600640 1 --2- 172.21.15.46:0/1636932805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf3815ad20 0x7faf380b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:50.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf2fe00640 1 -- 172.21.15.46:0/1636932805 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf380b0540 msgr2=0x7faf38158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf2fe00640 1 --2- 172.21.15.46:0/1636932805 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf380b0540 0x7faf38158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf2fe00640 1 -- 172.21.15.46:0/1636932805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf3815ad20 msgr2=0x7faf380b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf2fe00640 1 --2- 172.21.15.46:0/1636932805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf3815ad20 0x7faf380b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf2fe00640 1 -- 172.21.15.46:0/1636932805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf30009b30 con 0x7faf3815a940 2024-03-29T08:17:50.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf2fe00640 1 --2- 172.21.15.46:0/1636932805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3815a940 0x7faf381599e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faf2000e870 tx=0x7faf2000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:50.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf35600640 1 -- 172.21.15.46:0/1636932805 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20009510 con 0x7faf3815a940 2024-03-29T08:17:50.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf35600640 1 -- 172.21.15.46:0/1636932805 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2000eec0 con 0x7faf3815a940 2024-03-29T08:17:50.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf35600640 1 -- 172.21.15.46:0/1636932805 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2000cb40 con 0x7faf3815a940 2024-03-29T08:17:50.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf3f600640 1 -- 172.21.15.46:0/1636932805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf380c8490 con 0x7faf3815a940 2024-03-29T08:17:50.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf3f600640 1 -- 172.21.15.46:0/1636932805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf38158e80 con 0x7faf3815a940 2024-03-29T08:17:50.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf35600640 1 -- 172.21.15.46:0/1636932805 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7faf200059a0 con 0x7faf3815a940 2024-03-29T08:17:50.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf35600640 1 -- 172.21.15.46:0/1636932805 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7faf20006840 con 0x7faf3815a940 2024-03-29T08:17:50.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.881+0000 7faf3f600640 1 -- 172.21.15.46:0/1636932805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3815c260 con 0x7faf3815a940 2024-03-29T08:17:50.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.885+0000 7faf35600640 1 -- 172.21.15.46:0/1636932805 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7faf20021060 con 0x7faf3815a940 2024-03-29T08:17:50.914 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:50.911+0000 7f9d37c4b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-29T08:17:50.929 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:50.929+0000 7fc9d5e9b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-29T08:17:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.941+0000 7faf3f600640 1 -- 172.21.15.46:0/1636932805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf38066c90 con 0x7faf3815a940 2024-03-29T08:17:50.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.945+0000 7faf35600640 1 -- 172.21.15.46:0/1636932805 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7faf20017070 con 0x7faf3815a940 2024-03-29T08:17:50.944 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:17:50.944 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":10,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:17:50.635668+0000","last_up_change":"0.000000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:17:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.945+0000 7faf3f600640 1 -- 172.21.15.46:0/1636932805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3815a940 msgr2=0x7faf381599e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.945+0000 7faf3f600640 1 --2- 172.21.15.46:0/1636932805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3815a940 0x7faf381599e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faf2000e870 tx=0x7faf2000ec70 comp rx=0 tx=0).stop 2024-03-29T08:17:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.945+0000 7faf3f600640 1 -- 172.21.15.46:0/1636932805 shutdown_connections 2024-03-29T08:17:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.945+0000 7faf3f600640 1 --2- 172.21.15.46:0/1636932805 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf3815ad20 0x7faf380b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.945+0000 7faf3f600640 1 --2- 172.21.15.46:0/1636932805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3815a940 0x7faf381599e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.945+0000 7faf3f600640 1 --2- 172.21.15.46:0/1636932805 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf380b0540 0x7faf38158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.945+0000 7faf3f600640 1 -- 172.21.15.46:0/1636932805 >> 172.21.15.46:0/1636932805 conn(0x7faf38089fd0 msgr2=0x7faf38164c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.945+0000 7faf3f600640 1 -- 172.21.15.46:0/1636932805 shutdown_connections 2024-03-29T08:17:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:17:50.945+0000 7faf3f600640 1 -- 172.21.15.46:0/1636932805 wait complete. 2024-03-29T08:17:50.966 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-29T08:17:50.966 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-29T08:17:50.994 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:50.993+0000 7fc9d5e9b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-29T08:17:51.001 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:50.999+0000 7f9d37c4b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-29T08:17:51.060 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:51.059+0000 7f9d37c4b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-29T08:17:51.087 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:51.085+0000 7fc9d5e9b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-29T08:17:51.150 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:51.149+0000 7fc9d5e9b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-29T08:17:51.228 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:51.227+0000 7f9d37c4b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-29T08:17:51.295 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:51.295+0000 7f9d37c4b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-29T08:17:51.308 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-03-29T08:17:51.307+0000 7f724b62c640 -1 Falling back to public interface 2024-03-29T08:17:51.316 INFO:tasks.ceph.osd.5.smithi073.stderr:2024-03-29T08:17:51.315+0000 7fd32b62b640 -1 Falling back to public interface 2024-03-29T08:17:51.325 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:51.325+0000 7fc9d5e9b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-29T08:17:51.337 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-03-29T08:17:51.337+0000 7feea03fd640 -1 Falling back to public interface 2024-03-29T08:17:51.345 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-03-29T08:17:51.345+0000 7f6340fab640 -1 Falling back to public interface 2024-03-29T08:17:51.345 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-03-29T08:17:51.345+0000 7fb762d43640 -1 Falling back to public interface 2024-03-29T08:17:51.364 INFO:tasks.ceph.osd.7.smithi073.stderr:2024-03-29T08:17:51.363+0000 7fa0facd4640 -1 Falling back to public interface 2024-03-29T08:17:51.365 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-03-29T08:17:51.365+0000 7faa22b82640 -1 Falling back to public interface 2024-03-29T08:17:51.376 INFO:tasks.ceph.osd.6.smithi073.stderr:2024-03-29T08:17:51.375+0000 7f4007a2b640 -1 Falling back to public interface 2024-03-29T08:17:51.401 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:51.401+0000 7fc9d5e9b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-29T08:17:51.772 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T08:17:51.905 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.905+0000 7f0f61200640 1 Processor -- start 2024-03-29T08:17:51.905 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.905+0000 7f0f61200640 1 -- start start 2024-03-29T08:17:51.906 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.905+0000 7f0f61200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0f5c06bae0 0x7f0f5c0cdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:51.906 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.905+0000 7f0f61200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5c15cd80 0x7f0f5c0b0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:51.907 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.905+0000 7f0f61200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5c0cad90 con 0x7f0f5c06bec0 2024-03-29T08:17:51.907 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.905+0000 7f0f61200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5c0b0ab0 con 0x7f0f5c06bae0 2024-03-29T08:17:51.907 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.905+0000 7f0f61200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0b0cc0 con 0x7f0f5c15cd80 2024-03-29T08:17:51.907 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.905+0000 7f0f5a800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5c15cd80 0x7f0f5c0b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:51.907 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.905+0000 7f0f59e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0f5c06bae0 0x7f0f5c0cdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:51.908 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.905+0000 7f0f53e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0f5c06bec0 0x7f0f5c15c670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:41608/0 (socket says 172.21.15.46:41608) 2024-03-29T08:17:51.908 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.905+0000 7f0f5a800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5c15cd80 0x7f0f5c0b0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:49768/0 (socket says 172.21.15.46:49768) 2024-03-29T08:17:51.908 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.905+0000 7f0f53e00640 1 -- 172.21.15.46:0/2817790880 learned_addr learned my addr 172.21.15.46:0/2817790880 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:17:51.908 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.905+0000 7f0f59e00640 1 -- 172.21.15.46:0/2817790880 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5c15cd80 msgr2=0x7f0f5c0b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:17:51.908 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.905+0000 7f0f59e00640 1 --2- 172.21.15.46:0/2817790880 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5c15cd80 0x7f0f5c0b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:51.908 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.905+0000 7f0f59e00640 1 -- 172.21.15.46:0/2817790880 >> v1:172.21.15.73:6789/0 conn(0x7f0f5c06bec0 legacy=0x7f0f5c15c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:51.908 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.905+0000 7f0f59e00640 1 -- 172.21.15.46:0/2817790880 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c0b0fa0 con 0x7f0f5c06bae0 2024-03-29T08:17:51.908 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.905+0000 7f0f59400640 1 -- 172.21.15.46:0/2817790880 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008432002 0 0) 0x7f0f5c0cad90 con 0x7f0f5c06bec0 2024-03-29T08:17:51.910 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.909+0000 7f0f59e00640 1 --2- 172.21.15.46:0/2817790880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f5c06bae0 0x7f0f5c0cdb60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0097c0 tx=0x7f0f4c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f80d631c0d99caab server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:51.911 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.909+0000 7f0f59400640 1 -- 172.21.15.46:0/2817790880 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0ad640 con 0x7f0f5c06bae0 2024-03-29T08:17:51.911 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.909+0000 7f0f59400640 1 -- 172.21.15.46:0/2817790880 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4c0ad7a0 con 0x7f0f5c06bae0 2024-03-29T08:17:51.911 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.909+0000 7f0f59400640 1 -- 172.21.15.46:0/2817790880 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0adb30 con 0x7f0f5c06bae0 2024-03-29T08:17:51.911 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.909+0000 7f0f61200640 1 -- 172.21.15.46:0/2817790880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f5c06bae0 msgr2=0x7f0f5c0cdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:51.911 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.909+0000 7f0f61200640 1 --2- 172.21.15.46:0/2817790880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f5c06bae0 0x7f0f5c0cdb60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0097c0 tx=0x7f0f4c01c4a0 comp rx=0 tx=0).stop 2024-03-29T08:17:51.911 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.909+0000 7f0f61200640 1 -- 172.21.15.46:0/2817790880 shutdown_connections 2024-03-29T08:17:51.911 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.909+0000 7f0f61200640 1 --2- 172.21.15.46:0/2817790880 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5c15cd80 0x7f0f5c0b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:51.912 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.909+0000 7f0f61200640 1 --2- 172.21.15.46:0/2817790880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f5c06bae0 0x7f0f5c0cdb60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:51.912 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.909+0000 7f0f61200640 1 -- 172.21.15.46:0/2817790880 >> 172.21.15.46:0/2817790880 conn(0x7f0f5c099af0 msgr2=0x7f0f5c089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:51.912 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.909+0000 7f0f61200640 1 -- 172.21.15.46:0/2817790880 shutdown_connections 2024-03-29T08:17:51.912 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.909+0000 7f0f61200640 1 -- 172.21.15.46:0/2817790880 wait complete. 2024-03-29T08:17:51.913 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f61200640 1 Processor -- start 2024-03-29T08:17:51.913 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f61200640 1 -- start start 2024-03-29T08:17:51.914 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f61200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5c06bae0 0x7f0f5c15b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:51.914 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f61200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f5c06bec0 0x7f0f5c154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:51.914 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f61200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5c15cd80 0x7f0f5c0ad350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:51.914 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f61200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c15b970 con 0x7f0f5c15cd80 2024-03-29T08:17:51.914 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f61200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c159ea0 con 0x7f0f5c06bec0 2024-03-29T08:17:51.914 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f61200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c15e4e0 con 0x7f0f5c06bae0 2024-03-29T08:17:51.916 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f59e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5c06bae0 0x7f0f5c15b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:51.916 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f59e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5c06bae0 0x7f0f5c15b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:49770/0 (socket says 172.21.15.46:49770) 2024-03-29T08:17:51.916 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f59e00640 1 -- 172.21.15.46:0/3111634375 learned_addr learned my addr 172.21.15.46:0/3111634375 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:17:51.916 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f5a800640 1 --2- 172.21.15.46:0/3111634375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5c15cd80 0x7f0f5c0ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:51.916 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f53e00640 1 --2- 172.21.15.46:0/3111634375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f5c06bec0 0x7f0f5c154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:51.916 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f59e00640 1 -- 172.21.15.46:0/3111634375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f5c06bec0 msgr2=0x7f0f5c154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:51.916 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f59e00640 1 --2- 172.21.15.46:0/3111634375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f5c06bec0 0x7f0f5c154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:51.916 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f59e00640 1 -- 172.21.15.46:0/3111634375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5c15cd80 msgr2=0x7f0f5c0ad350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:51.916 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f59e00640 1 --2- 172.21.15.46:0/3111634375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5c15cd80 0x7f0f5c0ad350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:51.916 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f59e00640 1 -- 172.21.15.46:0/3111634375 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c009420 con 0x7f0f5c06bae0 2024-03-29T08:17:51.916 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f59e00640 1 --2- 172.21.15.46:0/3111634375 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5c06bae0 0x7f0f5c15b430 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0b0040 tx=0x7f0f4c019240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:51.917 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f52000640 1 -- 172.21.15.46:0/3111634375 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0b6610 con 0x7f0f5c06bae0 2024-03-29T08:17:51.917 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f52000640 1 -- 172.21.15.46:0/3111634375 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4c0a6040 con 0x7f0f5c06bae0 2024-03-29T08:17:51.917 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f52000640 1 -- 172.21.15.46:0/3111634375 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0b6990 con 0x7f0f5c06bae0 2024-03-29T08:17:51.917 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.913+0000 7f0f61200640 1 -- 172.21.15.46:0/3111634375 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c15e7c0 con 0x7f0f5c06bae0 2024-03-29T08:17:51.917 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.917+0000 7f0f61200640 1 -- 172.21.15.46:0/3111634375 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c15ed80 con 0x7f0f5c06bae0 2024-03-29T08:17:51.917 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.917+0000 7f0f52000640 1 -- 172.21.15.46:0/3111634375 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0f4c0b4040 con 0x7f0f5c06bae0 2024-03-29T08:17:51.917 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.917+0000 7f0f52000640 1 -- 172.21.15.46:0/3111634375 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f0f4c0b6df0 con 0x7f0f5c06bae0 2024-03-29T08:17:51.917 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.917+0000 7f0f61200640 1 -- 172.21.15.46:0/3111634375 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5c15ef70 con 0x7f0f5c06bae0 2024-03-29T08:17:51.919 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.917+0000 7f0f52000640 1 -- 172.21.15.46:0/3111634375 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72508 (secure 0 0 0) 0x7f0f4c0b2070 con 0x7f0f5c06bae0 2024-03-29T08:17:51.977 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.977+0000 7f0f61200640 1 -- 172.21.15.46:0/3111634375 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f5c15e620 con 0x7f0f5c06bae0 2024-03-29T08:17:51.978 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.977+0000 7f0f52000640 1 -- 172.21.15.46:0/3111634375 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f0f5c15e620 con 0x7f0f5c06bae0 2024-03-29T08:17:51.978 INFO:teuthology.misc.health.smithi046.stdout: 2024-03-29T08:17:51.978 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":10,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:17:50.635668+0000","last_up_change":"0.000000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:17:51.979 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.977+0000 7f0f61200640 1 -- 172.21.15.46:0/3111634375 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5c06bae0 msgr2=0x7f0f5c15b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:51.979 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.977+0000 7f0f61200640 1 --2- 172.21.15.46:0/3111634375 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5c06bae0 0x7f0f5c15b430 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0b0040 tx=0x7f0f4c019240 comp rx=0 tx=0).stop 2024-03-29T08:17:51.979 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.977+0000 7f0f61200640 1 -- 172.21.15.46:0/3111634375 shutdown_connections 2024-03-29T08:17:51.980 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.977+0000 7f0f61200640 1 --2- 172.21.15.46:0/3111634375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f5c06bec0 0x7f0f5c154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:51.980 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.977+0000 7f0f61200640 1 --2- 172.21.15.46:0/3111634375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5c15cd80 0x7f0f5c0ad350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:51.980 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.977+0000 7f0f61200640 1 --2- 172.21.15.46:0/3111634375 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5c06bae0 0x7f0f5c15b430 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:51.980 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.977+0000 7f0f61200640 1 -- 172.21.15.46:0/3111634375 >> 172.21.15.46:0/3111634375 conn(0x7f0f5c099af0 msgr2=0x7f0f5c0aec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:51.980 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.977+0000 7f0f61200640 1 -- 172.21.15.46:0/3111634375 shutdown_connections 2024-03-29T08:17:51.980 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:51.977+0000 7f0f61200640 1 -- 172.21.15.46:0/3111634375 wait complete. 2024-03-29T08:17:51.984 INFO:tasks.ceph.mgr.x.smithi073.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-29T08:17:51.984 INFO:tasks.ceph.mgr.x.smithi073.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-29T08:17:51.984 INFO:tasks.ceph.mgr.x.smithi073.stderr: from numpy import show_config as show_numpy_config 2024-03-29T08:17:51.995 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-29T08:17:52.029 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:52.027+0000 7f9d37c4b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-29T08:17:52.032 INFO:tasks.ceph.mgr.y.smithi046.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-29T08:17:52.032 INFO:tasks.ceph.mgr.y.smithi046.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-29T08:17:52.032 INFO:tasks.ceph.mgr.y.smithi046.stderr: from numpy import show_config as show_numpy_config 2024-03-29T08:17:52.091 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:52.089+0000 7fc9d5e9b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-29T08:17:52.176 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:52.175+0000 7f9d37c4b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-29T08:17:52.246 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:52.245+0000 7fc9d5e9b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-29T08:17:52.345 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:52.343+0000 7f9d37c4b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-29T08:17:52.417 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:52.415+0000 7f9d37c4b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-29T08:17:52.445 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:52.445+0000 7fc9d5e9b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-29T08:17:52.522 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:52.521+0000 7fc9d5e9b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-29T08:17:52.565 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:52.563+0000 7f9d37c4b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-29T08:17:52.679 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:52.677+0000 7fc9d5e9b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-29T08:17:53.252 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:53.251+0000 7f9d37c4b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-29T08:17:53.338 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:53.335+0000 7f9d37c4b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-29T08:17:53.402 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:53.401+0000 7fc9d5e9b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-29T08:17:53.493 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:53.493+0000 7fc9d5e9b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-29T08:17:53.538 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:53.535+0000 7f9d37c4b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-29T08:17:53.623 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:53.619+0000 7f9d37c4b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-29T08:17:53.704 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:53.701+0000 7fc9d5e9b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-29T08:17:53.792 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:53.793+0000 7fc9d5e9b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-29T08:17:53.951 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:53.947+0000 7f9d37c4b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-29T08:17:54.024 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:54.023+0000 7f9d37c4b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-29T08:17:54.143 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:54.141+0000 7fc9d5e9b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-29T08:17:54.220 INFO:tasks.ceph.mgr.y.smithi046.stderr:2024-03-29T08:17:54.217+0000 7fc9d5e9b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-29T08:17:55.003 INFO:tasks.ceph.osd.5.smithi073.stderr:2024-03-29T08:17:54.999+0000 7fd32b62b640 -1 osd.5 0 log_to_monitors true 2024-03-29T08:17:55.044 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:55.043+0000 7f9cdd400640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-29T08:17:55.045 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-03-29T08:17:55.043+0000 7f9cdd400640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-29T08:17:55.099 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-03-29T08:17:55.097+0000 7faa22b82640 -1 osd.1 0 log_to_monitors true 2024-03-29T08:17:55.147 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-03-29T08:17:55.143+0000 7f724b62c640 -1 osd.4 0 log_to_monitors true 2024-03-29T08:17:55.241 INFO:tasks.ceph.osd.6.smithi073.stderr:2024-03-29T08:17:55.239+0000 7f4007a2b640 -1 osd.6 0 log_to_monitors true 2024-03-29T08:17:55.295 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-03-29T08:17:55.293+0000 7f6340fab640 -1 osd.3 0 log_to_monitors true 2024-03-29T08:17:55.327 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-03-29T08:17:55.325+0000 7feea03fd640 -1 osd.0 0 log_to_monitors true 2024-03-29T08:17:55.342 INFO:tasks.ceph.osd.7.smithi073.stderr:2024-03-29T08:17:55.339+0000 7fa0facd4640 -1 osd.7 0 log_to_monitors true 2024-03-29T08:17:55.378 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-03-29T08:17:55.377+0000 7fb762d43640 -1 osd.2 0 log_to_monitors true 2024-03-29T08:17:57.112 INFO:tasks.ceph.osd.5.smithi073.stderr:2024-03-29T08:17:57.111+0000 7fd326e00640 -1 osd.5 0 waiting for initial osdmap 2024-03-29T08:17:57.269 INFO:tasks.ceph.osd.5.smithi073.stderr:2024-03-29T08:17:57.267+0000 7fd31ba00640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T08:17:58.714 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-03-29T08:17:58.713+0000 7fee9a800640 -1 osd.0 0 waiting for initial osdmap 2024-03-29T08:17:58.739 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-03-29T08:17:58.737+0000 7faa1d000640 -1 osd.1 0 waiting for initial osdmap 2024-03-29T08:17:58.745 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-03-29T08:17:58.745+0000 7fee91200640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T08:17:58.769 INFO:tasks.ceph.osd.1.smithi046.stderr:2024-03-29T08:17:58.769+0000 7faa13a00640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T08:17:58.802 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T08:17:58.811 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-03-29T08:17:58.813+0000 7fb75f000640 -1 osd.2 0 waiting for initial osdmap 2024-03-29T08:17:58.857 INFO:tasks.ceph.osd.2.smithi046.stderr:2024-03-29T08:17:58.857+0000 7fb753c00640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T08:17:58.862 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-03-29T08:17:58.861+0000 7f633b400640 -1 osd.3 0 waiting for initial osdmap 2024-03-29T08:17:58.897 INFO:tasks.ceph.osd.3.smithi046.stderr:2024-03-29T08:17:58.897+0000 7f6331e00640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T08:17:58.899 INFO:tasks.ceph.osd.7.smithi073.stderr:2024-03-29T08:17:58.895+0000 7fa0f5200640 -1 osd.7 0 waiting for initial osdmap 2024-03-29T08:17:58.906 INFO:tasks.ceph.osd.6.smithi073.stderr:2024-03-29T08:17:58.903+0000 7f4001400640 -1 osd.6 0 waiting for initial osdmap 2024-03-29T08:17:58.925 INFO:tasks.ceph.osd.7.smithi073.stderr:2024-03-29T08:17:58.923+0000 7fa0ebc00640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T08:17:58.936 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.937+0000 7f9652e00640 1 Processor -- start 2024-03-29T08:17:58.936 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.937+0000 7f9652e00640 1 -- start start 2024-03-29T08:17:58.938 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.937+0000 7f9652e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f964c0cfa00 0x7f964c099780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:58.938 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.937+0000 7f9652e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964c0cf570 con 0x7f964c099150 2024-03-29T08:17:58.938 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.937+0000 7f9652e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f964c06b920 con 0x7f964c15cb40 2024-03-29T08:17:58.938 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.937+0000 7f9652e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964c06bc00 con 0x7f964c0cfa00 2024-03-29T08:17:58.938 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.937+0000 7f9651000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f964c099150 0x7f964c0cacb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36262/0 (socket says 172.21.15.46:36262) 2024-03-29T08:17:58.938 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.937+0000 7f9651000640 1 -- 172.21.15.46:0/2525508072 learned_addr learned my addr 172.21.15.46:0/2525508072 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:17:58.939 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.937+0000 7f9652400640 1 --2- 172.21.15.46:0/2525508072 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f964c0cfa00 0x7f964c099780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:58.939 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.937+0000 7f964be00640 1 -- 172.21.15.46:0/2525508072 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4209601816 0 0) 0x7f964c0cf570 con 0x7f964c099150 2024-03-29T08:17:58.939 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.937+0000 7f964be00640 1 -- 172.21.15.46:0/2525508072 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9634003530 con 0x7f964c099150 2024-03-29T08:17:58.939 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.937+0000 7f964be00640 1 -- 172.21.15.46:0/2525508072 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1668622138 0 0) 0x7f964c06b920 con 0x7f964c15cb40 2024-03-29T08:17:58.939 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.937+0000 7f964be00640 1 -- 172.21.15.46:0/2525508072 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f964c0cf570 con 0x7f964c15cb40 2024-03-29T08:17:58.939 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.937+0000 7f9652400640 1 -- 172.21.15.46:0/2525508072 >> v1:172.21.15.73:6789/0 conn(0x7f964c15cb40 legacy=0x7f964c098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:58.939 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.937+0000 7f9652400640 1 -- 172.21.15.46:0/2525508072 >> v1:172.21.15.46:6789/0 conn(0x7f964c099150 legacy=0x7f964c0cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:58.939 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.937+0000 7f9652400640 1 -- 172.21.15.46:0/2525508072 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964c099e10 con 0x7f964c0cfa00 2024-03-29T08:17:58.941 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.941+0000 7f9652400640 1 --2- 172.21.15.46:0/2525508072 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f964c0cfa00 0x7f964c099780 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9644009dc0 tx=0x7f964401c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a463d042986ef089 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:58.941 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.941+0000 7f964be00640 1 -- 172.21.15.46:0/2525508072 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f964c06c0f0 con 0x7f964c0cfa00 2024-03-29T08:17:58.941 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.941+0000 7f964be00640 1 -- 172.21.15.46:0/2525508072 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96440a59e0 con 0x7f964c0cfa00 2024-03-29T08:17:58.941 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.941+0000 7f964be00640 1 -- 172.21.15.46:0/2525508072 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96440a5d50 con 0x7f964c0cfa00 2024-03-29T08:17:58.942 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.941+0000 7f9652e00640 1 -- 172.21.15.46:0/2525508072 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f964c0cfa00 msgr2=0x7f964c099780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:58.942 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.941+0000 7f9652e00640 1 --2- 172.21.15.46:0/2525508072 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f964c0cfa00 0x7f964c099780 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9644009dc0 tx=0x7f964401c900 comp rx=0 tx=0).stop 2024-03-29T08:17:58.942 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.941+0000 7f9652e00640 1 -- 172.21.15.46:0/2525508072 shutdown_connections 2024-03-29T08:17:58.942 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.941+0000 7f9652e00640 1 --2- 172.21.15.46:0/2525508072 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f964c0cfa00 0x7f964c099780 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:58.942 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.941+0000 7f9652e00640 1 -- 172.21.15.46:0/2525508072 >> 172.21.15.46:0/2525508072 conn(0x7f964c0b2150 msgr2=0x7f964c0b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:58.942 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.941+0000 7f9652e00640 1 -- 172.21.15.46:0/2525508072 shutdown_connections 2024-03-29T08:17:58.942 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.941+0000 7f9652e00640 1 -- 172.21.15.46:0/2525508072 wait complete. 2024-03-29T08:17:58.943 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.941+0000 7f9652e00640 1 Processor -- start 2024-03-29T08:17:58.943 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9652e00640 1 -- start start 2024-03-29T08:17:58.945 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9652e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c099150 0x7f964c1405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:58.946 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9652e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f964c0cfa00 0x7f964c0b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:58.946 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9652e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f964c15cb40 0x7f964c0b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:58.946 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9652e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c0b4da0 con 0x7f964c15cb40 2024-03-29T08:17:58.946 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9652e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c0b41f0 con 0x7f964c099150 2024-03-29T08:17:58.946 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9652e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964c0b4400 con 0x7f964c0cfa00 2024-03-29T08:17:58.946 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9651a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c099150 0x7f964c1405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:58.946 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9652400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f964c15cb40 0x7f964c0b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:58.946 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9651a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c099150 0x7f964c1405f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:53336/0 (socket says 172.21.15.46:53336) 2024-03-29T08:17:58.946 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9651a00640 1 -- 172.21.15.46:0/2540261009 learned_addr learned my addr 172.21.15.46:0/2540261009 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:17:58.946 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9651000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f964c0cfa00 0x7f964c0b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:58.946 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9652400640 1 -- 172.21.15.46:0/2540261009 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f964c0cfa00 msgr2=0x7f964c0b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:58.946 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9652400640 1 --2- 172.21.15.46:0/2540261009 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f964c0cfa00 0x7f964c0b51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:58.946 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9652400640 1 -- 172.21.15.46:0/2540261009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c099150 msgr2=0x7f964c1405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:58.947 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9652400640 1 --2- 172.21.15.46:0/2540261009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c099150 0x7f964c1405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:58.947 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9652400640 1 -- 172.21.15.46:0/2540261009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9644009a70 con 0x7f964c15cb40 2024-03-29T08:17:58.947 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9651a00640 1 --2- 172.21.15.46:0/2540261009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c099150 0x7f964c1405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:17:58.947 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9652400640 1 --2- 172.21.15.46:0/2540261009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f964c15cb40 0x7f964c0b5700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f96440ac000 tx=0x7f96440196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:58.947 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-03-29T08:17:58.943+0000 7f7245200640 -1 osd.4 0 waiting for initial osdmap 2024-03-29T08:17:58.948 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9649600640 1 -- 172.21.15.46:0/2540261009 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644003db0 con 0x7f964c15cb40 2024-03-29T08:17:58.948 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9649600640 1 -- 172.21.15.46:0/2540261009 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96440a5860 con 0x7f964c15cb40 2024-03-29T08:17:58.948 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9649600640 1 -- 172.21.15.46:0/2540261009 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644004280 con 0x7f964c15cb40 2024-03-29T08:17:58.948 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9652e00640 1 -- 172.21.15.46:0/2540261009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964c0b46e0 con 0x7f964c15cb40 2024-03-29T08:17:58.948 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.945+0000 7f9652e00640 1 -- 172.21.15.46:0/2540261009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964c157ea0 con 0x7f964c15cb40 2024-03-29T08:17:58.948 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.949+0000 7f9652e00640 1 -- 172.21.15.46:0/2540261009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964c04e040 con 0x7f964c15cb40 2024-03-29T08:17:58.949 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.949+0000 7f9649600640 1 -- 172.21.15.46:0/2540261009 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) v1 ==== 86028+0+0 (secure 0 0 0) 0x7f96440b4070 con 0x7f964c15cb40 2024-03-29T08:17:58.949 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.949+0000 7f9649600640 1 --2- 172.21.15.46:0/2540261009 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f962c066600 0x7f962c068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:17:58.949 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.949+0000 7f9649600640 1 -- 172.21.15.46:0/2540261009 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2511+0+0 (secure 0 0 0) 0x7f96440efcb0 con 0x7f964c15cb40 2024-03-29T08:17:58.950 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.949+0000 7f9651a00640 1 --2- 172.21.15.46:0/2540261009 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f962c066600 0x7f962c068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:17:58.950 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.949+0000 7f9651a00640 1 --2- 172.21.15.46:0/2540261009 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f962c066600 0x7f962c068ac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9640000fc0 tx=0x7f9640009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:17:58.952 INFO:tasks.ceph.osd.6.smithi073.stderr:2024-03-29T08:17:58.951+0000 7f3ff7e00640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T08:17:58.970 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:58.969+0000 7f9649600640 1 -- 172.21.15.46:0/2540261009 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f964414e980 con 0x7f964c15cb40 2024-03-29T08:17:58.984 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-03-29T08:17:58.983+0000 7f723bc00640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-29T08:17:59.133 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:59.133+0000 7f9652e00640 1 -- 172.21.15.46:0/2540261009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f964c0b3fd0 con 0x7f964c15cb40 2024-03-29T08:17:59.134 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:59.133+0000 7f9649600640 1 -- 172.21.15.46:0/2540261009 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+7494 (secure 0 0 0) 0x7f96440b2090 con 0x7f964c15cb40 2024-03-29T08:17:59.134 INFO:teuthology.misc.health.smithi046.stdout: 2024-03-29T08:17:59.134 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":14,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:17:58.196021+0000","last_up_change":"2024-03-29T08:17:58.196021+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:17:59.136 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:59.137+0000 7f9652e00640 1 -- 172.21.15.46:0/2540261009 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f962c066600 msgr2=0x7f962c068ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:59.136 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:59.137+0000 7f9652e00640 1 --2- 172.21.15.46:0/2540261009 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f962c066600 0x7f962c068ac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9640000fc0 tx=0x7f9640009040 comp rx=0 tx=0).stop 2024-03-29T08:17:59.136 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:59.137+0000 7f9652e00640 1 -- 172.21.15.46:0/2540261009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f964c15cb40 msgr2=0x7f964c0b5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:17:59.137 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:59.137+0000 7f9652e00640 1 --2- 172.21.15.46:0/2540261009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f964c15cb40 0x7f964c0b5700 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f96440ac000 tx=0x7f96440196a0 comp rx=0 tx=0).stop 2024-03-29T08:17:59.137 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:59.137+0000 7f9652e00640 1 -- 172.21.15.46:0/2540261009 shutdown_connections 2024-03-29T08:17:59.137 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:59.137+0000 7f9652e00640 1 --2- 172.21.15.46:0/2540261009 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f962c066600 0x7f962c068ac0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:59.137 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:59.137+0000 7f9652e00640 1 --2- 172.21.15.46:0/2540261009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f964c15cb40 0x7f964c0b5700 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:59.137 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:59.137+0000 7f9652e00640 1 --2- 172.21.15.46:0/2540261009 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f964c0cfa00 0x7f964c0b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:59.137 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:59.137+0000 7f9652e00640 1 --2- 172.21.15.46:0/2540261009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f964c099150 0x7f964c1405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:17:59.137 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:59.137+0000 7f9652e00640 1 -- 172.21.15.46:0/2540261009 >> 172.21.15.46:0/2540261009 conn(0x7f964c0b2150 msgr2=0x7f964c0b0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:17:59.137 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:59.137+0000 7f9652e00640 1 -- 172.21.15.46:0/2540261009 shutdown_connections 2024-03-29T08:17:59.137 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:17:59.137+0000 7f9652e00640 1 -- 172.21.15.46:0/2540261009 wait complete. 2024-03-29T08:17:59.153 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-03-29T08:18:05.960 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:06.102 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.101+0000 7fc80e200640 1 Processor -- start 2024-03-29T08:18:06.102 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.101+0000 7fc80e200640 1 -- start start 2024-03-29T08:18:06.104 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.101+0000 7fc80e200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc8080cb8a0 0x7fc808159e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.104 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.101+0000 7fc80e200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc80806d120 0x7fc80815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.104 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.101+0000 7fc80e200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc80815a8f0 con 0x7fc808157140 2024-03-29T08:18:06.104 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.101+0000 7fc80e200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc80815aba0 con 0x7fc80806d120 2024-03-29T08:18:06.104 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.101+0000 7fc80e200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc80806c360 con 0x7fc8080cb8a0 2024-03-29T08:18:06.105 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.105+0000 7fc807e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc80806d120 0x7fc80815a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.105 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.105+0000 7fc807e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc80806d120 0x7fc80815a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43162/0 (socket says 172.21.15.46:43162) 2024-03-29T08:18:06.105 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.105+0000 7fc807e00640 1 -- 172.21.15.46:0/4177574391 learned_addr learned my addr 172.21.15.46:0/4177574391 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:06.105 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.105+0000 7fc7fea00640 1 --2- 172.21.15.46:0/4177574391 >> v2:172.21.15.73:3300/0 conn(0x7fc8080cb8a0 0x7fc808159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.105 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.105+0000 7fc807400640 1 --1- 172.21.15.46:0/4177574391 >> v1:172.21.15.73:6789/0 conn(0x7fc808157140 0x7fc8080cb190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:59816/0 (socket says 172.21.15.46:59816) 2024-03-29T08:18:06.105 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.105+0000 7fc807e00640 1 -- 172.21.15.46:0/4177574391 >> v2:172.21.15.73:3300/0 conn(0x7fc8080cb8a0 msgr2=0x7fc808159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:06.105 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.105+0000 7fc807e00640 1 --2- 172.21.15.46:0/4177574391 >> v2:172.21.15.73:3300/0 conn(0x7fc8080cb8a0 0x7fc808159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.105 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.105+0000 7fc807e00640 1 -- 172.21.15.46:0/4177574391 >> v1:172.21.15.73:6789/0 conn(0x7fc808157140 legacy=0x7fc8080cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.105 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.105+0000 7fc807e00640 1 -- 172.21.15.46:0/4177574391 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80806c640 con 0x7fc80806d120 2024-03-29T08:18:06.107 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.105+0000 7fc807e00640 1 --2- 172.21.15.46:0/4177574391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80806d120 0x7fc80815a3b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc8080cc430 tx=0x7fc80001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e64eab0cd7ad67 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:06.107 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.105+0000 7fc806a00640 1 -- 172.21.15.46:0/4177574391 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8000ab660 con 0x7fc80806d120 2024-03-29T08:18:06.107 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc806a00640 1 -- 172.21.15.46:0/4177574391 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8000ab7c0 con 0x7fc80806d120 2024-03-29T08:18:06.108 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc806a00640 1 -- 172.21.15.46:0/4177574391 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8000abb50 con 0x7fc80806d120 2024-03-29T08:18:06.108 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc80e200640 1 -- 172.21.15.46:0/4177574391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80806d120 msgr2=0x7fc80815a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.108 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc80e200640 1 --2- 172.21.15.46:0/4177574391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80806d120 0x7fc80815a3b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc8080cc430 tx=0x7fc80001c460 comp rx=0 tx=0).stop 2024-03-29T08:18:06.108 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc80e200640 1 -- 172.21.15.46:0/4177574391 shutdown_connections 2024-03-29T08:18:06.109 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc80e200640 1 --2- 172.21.15.46:0/4177574391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80806d120 0x7fc80815a3b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.109 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc80e200640 1 --2- 172.21.15.46:0/4177574391 >> v2:172.21.15.73:3300/0 conn(0x7fc8080cb8a0 0x7fc808159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.109 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc80e200640 1 -- 172.21.15.46:0/4177574391 >> 172.21.15.46:0/4177574391 conn(0x7fc808069d60 msgr2=0x7fc80806a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:06.109 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc80e200640 1 -- 172.21.15.46:0/4177574391 shutdown_connections 2024-03-29T08:18:06.109 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc80e200640 1 -- 172.21.15.46:0/4177574391 wait complete. 2024-03-29T08:18:06.110 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc80e200640 1 Processor -- start 2024-03-29T08:18:06.110 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc80e200640 1 -- start start 2024-03-29T08:18:06.112 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc80e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80806d120 0x7fc80814f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.112 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc80e200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8080cb8a0 0x7fc80814e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.112 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc80e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc808157140 0x7fc80815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.112 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc80e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80814f240 con 0x7fc80806d120 2024-03-29T08:18:06.112 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc80e200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8081f3960 con 0x7fc808157140 2024-03-29T08:18:06.112 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc80e200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80814e8a0 con 0x7fc8080cb8a0 2024-03-29T08:18:06.113 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc807400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80806d120 0x7fc80814f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.113 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc807400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80806d120 0x7fc80814f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43164/0 (socket says 172.21.15.46:43164) 2024-03-29T08:18:06.113 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc807400640 1 -- 172.21.15.46:0/3469452167 learned_addr learned my addr 172.21.15.46:0/3469452167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:06.113 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc807e00640 1 --2- 172.21.15.46:0/3469452167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc808157140 0x7fc80815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.113 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.109+0000 7fc7fea00640 1 --2- 172.21.15.46:0/3469452167 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8080cb8a0 0x7fc80814e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.113 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.113+0000 7fc807400640 1 -- 172.21.15.46:0/3469452167 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8080cb8a0 msgr2=0x7fc80814e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.113 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.113+0000 7fc807400640 1 --2- 172.21.15.46:0/3469452167 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8080cb8a0 0x7fc80814e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.113 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.113+0000 7fc807400640 1 -- 172.21.15.46:0/3469452167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc808157140 msgr2=0x7fc80815e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.113 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.113+0000 7fc807400640 1 --2- 172.21.15.46:0/3469452167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc808157140 0x7fc80815e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.113 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.113+0000 7fc807400640 1 -- 172.21.15.46:0/3469452167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8000094d0 con 0x7fc80806d120 2024-03-29T08:18:06.113 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.113+0000 7fc807400640 1 --2- 172.21.15.46:0/3469452167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80806d120 0x7fc80814f660 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800c780 tx=0x7fc7f800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:06.113 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.113+0000 7fc7ffe00640 1 -- 172.21.15.46:0/3469452167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f800ce90 con 0x7fc80806d120 2024-03-29T08:18:06.113 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.113+0000 7fc7ffe00640 1 -- 172.21.15.46:0/3469452167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f800a640 con 0x7fc80806d120 2024-03-29T08:18:06.114 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.113+0000 7fc7ffe00640 1 -- 172.21.15.46:0/3469452167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f800a9b0 con 0x7fc80806d120 2024-03-29T08:18:06.114 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.113+0000 7fc80e200640 1 -- 172.21.15.46:0/3469452167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80814eb80 con 0x7fc80806d120 2024-03-29T08:18:06.114 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.113+0000 7fc80e200640 1 -- 172.21.15.46:0/3469452167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80814e5a0 con 0x7fc80806d120 2024-03-29T08:18:06.114 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.113+0000 7fc80e200640 1 -- 172.21.15.46:0/3469452167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc808071c70 con 0x7fc80806d120 2024-03-29T08:18:06.114 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.113+0000 7fc7ffe00640 1 -- 172.21.15.46:0/3469452167 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc7f800ab10 con 0x7fc80806d120 2024-03-29T08:18:06.115 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.113+0000 7fc7ffe00640 1 --2- 172.21.15.46:0/3469452167 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc7e4066690 0x7fc7e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.115 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.113+0000 7fc7ffe00640 1 -- 172.21.15.46:0/3469452167 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc7f8083640 con 0x7fc80806d120 2024-03-29T08:18:06.115 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.113+0000 7fc7fea00640 1 --2- 172.21.15.46:0/3469452167 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc7e4066690 0x7fc7e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.116 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.117+0000 7fc7fea00640 1 --2- 172.21.15.46:0/3469452167 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc7e4066690 0x7fc7e4068b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc808156720 tx=0x7fc7f0009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:06.120 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.121+0000 7fc7ffe00640 1 -- 172.21.15.46:0/3469452167 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc7f8050d10 con 0x7fc80806d120 2024-03-29T08:18:06.284 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.285+0000 7fc80e200640 1 -- 172.21.15.46:0/3469452167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc80804e120 con 0x7fc80806d120 2024-03-29T08:18:06.284 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.285+0000 7fc7ffe00640 1 -- 172.21.15.46:0/3469452167 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12334 (secure 0 0 0) 0x7fc7f8017090 con 0x7fc80806d120 2024-03-29T08:18:06.285 INFO:teuthology.misc.health.smithi046.stdout: 2024-03-29T08:18:06.285 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":18,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:02.761059+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:06.287 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.285+0000 7fc80e200640 1 -- 172.21.15.46:0/3469452167 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc7e4066690 msgr2=0x7fc7e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.287 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.285+0000 7fc80e200640 1 --2- 172.21.15.46:0/3469452167 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc7e4066690 0x7fc7e4068b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc808156720 tx=0x7fc7f0009190 comp rx=0 tx=0).stop 2024-03-29T08:18:06.287 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.285+0000 7fc80e200640 1 -- 172.21.15.46:0/3469452167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80806d120 msgr2=0x7fc80814f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.287 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.285+0000 7fc80e200640 1 --2- 172.21.15.46:0/3469452167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80806d120 0x7fc80814f660 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800c780 tx=0x7fc7f800cc40 comp rx=0 tx=0).stop 2024-03-29T08:18:06.287 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.285+0000 7fc80e200640 1 -- 172.21.15.46:0/3469452167 shutdown_connections 2024-03-29T08:18:06.287 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.285+0000 7fc80e200640 1 --2- 172.21.15.46:0/3469452167 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc7e4066690 0x7fc7e4068b50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.287 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.285+0000 7fc80e200640 1 --2- 172.21.15.46:0/3469452167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc808157140 0x7fc80815e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.287 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.285+0000 7fc80e200640 1 --2- 172.21.15.46:0/3469452167 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8080cb8a0 0x7fc80814e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.287 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.285+0000 7fc80e200640 1 --2- 172.21.15.46:0/3469452167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc80806d120 0x7fc80814f660 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.288 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.285+0000 7fc80e200640 1 -- 172.21.15.46:0/3469452167 >> 172.21.15.46:0/3469452167 conn(0x7fc808069d60 msgr2=0x7fc8080ccde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:06.288 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.285+0000 7fc80e200640 1 -- 172.21.15.46:0/3469452167 shutdown_connections 2024-03-29T08:18:06.288 INFO:teuthology.misc.health.smithi046.stderr:2024-03-29T08:18:06.285+0000 7fc80e200640 1 -- 172.21.15.46:0/3469452167 wait complete. 2024-03-29T08:18:06.303 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-29T08:18:06.304 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-29T08:18:06.304 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T08:18:06.304 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T08:18:06.307 INFO:tasks.ceph.mds.a:Started 2024-03-29T08:18:06.307 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T08:18:06.307 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T08:18:06.311 INFO:tasks.ceph.mds.c:Started 2024-03-29T08:18:06.311 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T08:18:06.311 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T08:18:06.315 INFO:tasks.ceph.mds.b:Started 2024-03-29T08:18:06.315 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T08:18:06.316 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T08:18:06.318 INFO:tasks.ceph.mds.d:Started 2024-03-29T08:18:06.319 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-29T08:18:06.319 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:06.401 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-03-29T08:18:06.402 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-03-29T08:18:06.426 INFO:tasks.ceph.mds.d.smithi073.stdout:starting mds.d at 2024-03-29T08:18:06.427 INFO:tasks.ceph.mds.b.smithi073.stdout:starting mds.b at 2024-03-29T08:18:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.469+0000 7ff91b200640 1 Processor -- start 2024-03-29T08:18:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.469+0000 7ff91b200640 1 -- start start 2024-03-29T08:18:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.469+0000 7ff91b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff91c0b2b30 0x7ff91c099150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.469+0000 7ff91b200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff91c099690 0x7ff91c0b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.469+0000 7ff91b200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff91c0cc2e0 con 0x7ff91c15a890 2024-03-29T08:18:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.469+0000 7ff91b200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91c0cce20 con 0x7ff91c0b2b30 2024-03-29T08:18:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.469+0000 7ff91b200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91c0cc580 con 0x7ff91c099690 2024-03-29T08:18:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.469+0000 7ff913e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff91c0b2b30 0x7ff91c099150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.469+0000 7ff913e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff91c0b2b30 0x7ff91c099150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37462/0 (socket says 172.21.15.46:37462) 2024-03-29T08:18:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.469+0000 7ff919e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff91c15a890 0x7ff91c098a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:50722/0 (socket says 172.21.15.46:50722) 2024-03-29T08:18:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.469+0000 7ff913e00640 1 -- 172.21.15.46:0/2635426798 learned_addr learned my addr 172.21.15.46:0/2635426798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.469+0000 7ff91a800640 1 --2- 172.21.15.46:0/2635426798 >> v2:172.21.15.73:3300/0 conn(0x7ff91c099690 0x7ff91c0b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.469+0000 7ff913e00640 1 -- 172.21.15.46:0/2635426798 >> v2:172.21.15.73:3300/0 conn(0x7ff91c099690 msgr2=0x7ff91c0b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.469+0000 7ff913e00640 1 --2- 172.21.15.46:0/2635426798 >> v2:172.21.15.73:3300/0 conn(0x7ff91c099690 0x7ff91c0b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.469+0000 7ff913e00640 1 -- 172.21.15.46:0/2635426798 >> v1:172.21.15.73:6789/0 conn(0x7ff91c15a890 legacy=0x7ff91c098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.469+0000 7ff913e00640 1 -- 172.21.15.46:0/2635426798 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c0cc860 con 0x7ff91c0b2b30 2024-03-29T08:18:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.473+0000 7ff913e00640 1 --2- 172.21.15.46:0/2635426798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c0b2b30 0x7ff91c099150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff91c06dc70 tx=0x7ff90401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77f49ef8da3a6bb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.473+0000 7ff919400640 1 -- 172.21.15.46:0/2635426798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9040ab600 con 0x7ff91c0b2b30 2024-03-29T08:18:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.473+0000 7ff919400640 1 -- 172.21.15.46:0/2635426798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9040a7050 con 0x7ff91c0b2b30 2024-03-29T08:18:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.473+0000 7ff919400640 1 -- 172.21.15.46:0/2635426798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9040ab980 con 0x7ff91c0b2b30 2024-03-29T08:18:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.473+0000 7ff91b200640 1 -- 172.21.15.46:0/2635426798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c0b2b30 msgr2=0x7ff91c099150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.473+0000 7ff91b200640 1 --2- 172.21.15.46:0/2635426798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c0b2b30 0x7ff91c099150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff91c06dc70 tx=0x7ff90401c460 comp rx=0 tx=0).stop 2024-03-29T08:18:06.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.473+0000 7ff91b200640 1 -- 172.21.15.46:0/2635426798 shutdown_connections 2024-03-29T08:18:06.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.473+0000 7ff91b200640 1 --2- 172.21.15.46:0/2635426798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c0b2b30 0x7ff91c099150 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.473+0000 7ff91b200640 1 --2- 172.21.15.46:0/2635426798 >> v2:172.21.15.73:3300/0 conn(0x7ff91c099690 0x7ff91c0b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.473+0000 7ff91b200640 1 -- 172.21.15.46:0/2635426798 >> 172.21.15.46:0/2635426798 conn(0x7ff91c0b2010 msgr2=0x7ff91c0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:06.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.473+0000 7ff91b200640 1 -- 172.21.15.46:0/2635426798 shutdown_connections 2024-03-29T08:18:06.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.473+0000 7ff91b200640 1 -- 172.21.15.46:0/2635426798 wait complete. 2024-03-29T08:18:06.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.473+0000 7ff91b200640 1 Processor -- start 2024-03-29T08:18:06.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.473+0000 7ff91b200640 1 -- start start 2024-03-29T08:18:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff91c099690 0x7ff91c0c5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91b200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff91c0b2b30 0x7ff91c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c15a890 0x7ff91c06ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c162840 con 0x7ff91c15a890 2024-03-29T08:18:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c0c5b40 con 0x7ff91c099690 2024-03-29T08:18:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91b200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c0c5e20 con 0x7ff91c0b2b30 2024-03-29T08:18:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c15a890 0x7ff91c06ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c15a890 0x7ff91c06ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37474/0 (socket says 172.21.15.46:37474) 2024-03-29T08:18:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91a800640 1 -- 172.21.15.46:0/3344488426 learned_addr learned my addr 172.21.15.46:0/3344488426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff919e00640 1 --2- 172.21.15.46:0/3344488426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff91c099690 0x7ff91c0c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff913e00640 1 --2- 172.21.15.46:0/3344488426 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff91c0b2b30 0x7ff91c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91a800640 1 -- 172.21.15.46:0/3344488426 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff91c0b2b30 msgr2=0x7ff91c06e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91a800640 1 --2- 172.21.15.46:0/3344488426 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff91c0b2b30 0x7ff91c06e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91a800640 1 -- 172.21.15.46:0/3344488426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff91c099690 msgr2=0x7ff91c0c5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91a800640 1 --2- 172.21.15.46:0/3344488426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff91c099690 0x7ff91c0c5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91a800640 1 -- 172.21.15.46:0/3344488426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9040094d0 con 0x7ff91c15a890 2024-03-29T08:18:06.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91a800640 1 --2- 172.21.15.46:0/3344488426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c15a890 0x7ff91c06ecb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff91400efc0 tx=0x7ff91400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff912000640 1 -- 172.21.15.46:0/3344488426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff914009ba0 con 0x7ff91c15a890 2024-03-29T08:18:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff912000640 1 -- 172.21.15.46:0/3344488426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91400f040 con 0x7ff91c15a890 2024-03-29T08:18:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91b200640 1 -- 172.21.15.46:0/3344488426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91c0c6100 con 0x7ff91c15a890 2024-03-29T08:18:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff912000640 1 -- 172.21.15.46:0/3344488426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9140146c0 con 0x7ff91c15a890 2024-03-29T08:18:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91b200640 1 -- 172.21.15.46:0/3344488426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91c0c5750 con 0x7ff91c15a890 2024-03-29T08:18:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff91b200640 1 -- 172.21.15.46:0/3344488426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91c04e040 con 0x7ff91c15a890 2024-03-29T08:18:06.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.477+0000 7ff912000640 1 -- 172.21.15.46:0/3344488426 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff914014820 con 0x7ff91c15a890 2024-03-29T08:18:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.481+0000 7ff912000640 1 --2- 172.21.15.46:0/3344488426 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff8f8066690 0x7ff8f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.481+0000 7ff912000640 1 -- 172.21.15.46:0/3344488426 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff9140847c0 con 0x7ff91c15a890 2024-03-29T08:18:06.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.481+0000 7ff919e00640 1 --2- 172.21.15.46:0/3344488426 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff8f8066690 0x7ff8f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.481+0000 7ff919e00640 1 --2- 172.21.15.46:0/3344488426 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff8f8066690 0x7ff8f8068b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff90c000f70 tx=0x7ff90c007e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:06.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.485+0000 7ff912000640 1 -- 172.21.15.46:0/3344488426 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff914051f20 con 0x7ff91c15a890 2024-03-29T08:18:06.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.645+0000 7ff91b200640 1 -- 172.21.15.46:0/3344488426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff91c0ce9e0 con 0x7ff91c15a890 2024-03-29T08:18:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.649+0000 7ff912000640 1 -- 172.21.15.46:0/3344488426 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12334 (secure 0 0 0) 0x7ff914055540 con 0x7ff91c15a890 2024-03-29T08:18:06.648 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:06.649 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:02.761059+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:06.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.649+0000 7ff91b200640 1 -- 172.21.15.46:0/3344488426 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff8f8066690 msgr2=0x7ff8f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.649+0000 7ff91b200640 1 --2- 172.21.15.46:0/3344488426 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff8f8066690 0x7ff8f8068b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff90c000f70 tx=0x7ff90c007e50 comp rx=0 tx=0).stop 2024-03-29T08:18:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.649+0000 7ff91b200640 1 -- 172.21.15.46:0/3344488426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c15a890 msgr2=0x7ff91c06ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.649+0000 7ff91b200640 1 --2- 172.21.15.46:0/3344488426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c15a890 0x7ff91c06ecb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff91400efc0 tx=0x7ff91400c490 comp rx=0 tx=0).stop 2024-03-29T08:18:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.649+0000 7ff91b200640 1 -- 172.21.15.46:0/3344488426 shutdown_connections 2024-03-29T08:18:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.649+0000 7ff91b200640 1 --2- 172.21.15.46:0/3344488426 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff8f8066690 0x7ff8f8068b50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.649+0000 7ff91b200640 1 --2- 172.21.15.46:0/3344488426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff91c15a890 0x7ff91c06ecb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.649+0000 7ff91b200640 1 --2- 172.21.15.46:0/3344488426 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff91c0b2b30 0x7ff91c06e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.649+0000 7ff91b200640 1 --2- 172.21.15.46:0/3344488426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff91c099690 0x7ff91c0c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.649+0000 7ff91b200640 1 -- 172.21.15.46:0/3344488426 >> 172.21.15.46:0/3344488426 conn(0x7ff91c0b2010 msgr2=0x7ff91c157ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.649+0000 7ff91b200640 1 -- 172.21.15.46:0/3344488426 shutdown_connections 2024-03-29T08:18:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.649+0000 7ff91b200640 1 -- 172.21.15.46:0/3344488426 wait complete. 2024-03-29T08:18:06.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:18:06.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:18:06.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.812+0000 7faefba00640 1 Processor -- start 2024-03-29T08:18:06.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.812+0000 7faefba00640 1 -- start start 2024-03-29T08:18:06.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.812+0000 7faefba00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faef415a790 0x7faef40cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.812+0000 7faefba00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faef40cf340 0x7faef40b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.812+0000 7faefba00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef40cc660 con 0x7faef40cbbe0 2024-03-29T08:18:06.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.812+0000 7faefba00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef40cc0a0 con 0x7faef40cf340 2024-03-29T08:18:06.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.812+0000 7faefba00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef40cc380 con 0x7faef415a790 2024-03-29T08:18:06.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.812+0000 7faef9a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faef40cf340 0x7faef40b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.812+0000 7faef9a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faef40cf340 0x7faef40b0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37480/0 (socket says 172.21.15.46:37480) 2024-03-29T08:18:06.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.812+0000 7faef9a00640 1 -- 172.21.15.46:0/1595609212 learned_addr learned my addr 172.21.15.46:0/1595609212 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:06.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.812+0000 7faef9000640 1 --2- 172.21.15.46:0/1595609212 >> v2:172.21.15.73:3300/0 conn(0x7faef415a790 0x7faef40cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.812+0000 7faef9a00640 1 -- 172.21.15.46:0/1595609212 >> v2:172.21.15.73:3300/0 conn(0x7faef415a790 msgr2=0x7faef40cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:06.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.812+0000 7faef9a00640 1 --2- 172.21.15.46:0/1595609212 >> v2:172.21.15.73:3300/0 conn(0x7faef415a790 0x7faef40cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.812+0000 7faef9a00640 1 -- 172.21.15.46:0/1595609212 >> v1:172.21.15.73:6789/0 conn(0x7faef40cbbe0 legacy=0x7faef40d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.812+0000 7faef9a00640 1 -- 172.21.15.46:0/1595609212 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef40b0380 con 0x7faef40cf340 2024-03-29T08:18:06.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.816+0000 7faef9a00640 1 --2- 172.21.15.46:0/1595609212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef40cf340 0x7faef40b0b80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faef40b2200 tx=0x7faeec01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4287f0ed4b963659 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.816+0000 7faef3e00640 1 -- 172.21.15.46:0/1595609212 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec0a5780 con 0x7faef40cf340 2024-03-29T08:18:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.816+0000 7faef3e00640 1 -- 172.21.15.46:0/1595609212 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeec0a58e0 con 0x7faef40cf340 2024-03-29T08:18:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.816+0000 7faef3e00640 1 -- 172.21.15.46:0/1595609212 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec0a5c70 con 0x7faef40cf340 2024-03-29T08:18:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.816+0000 7faefba00640 1 -- 172.21.15.46:0/1595609212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef40cf340 msgr2=0x7faef40b0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.816+0000 7faefba00640 1 --2- 172.21.15.46:0/1595609212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef40cf340 0x7faef40b0b80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faef40b2200 tx=0x7faeec01c8a0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.816+0000 7faefba00640 1 -- 172.21.15.46:0/1595609212 shutdown_connections 2024-03-29T08:18:06.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.816+0000 7faefba00640 1 --2- 172.21.15.46:0/1595609212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef40cf340 0x7faef40b0b80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.816+0000 7faefba00640 1 --2- 172.21.15.46:0/1595609212 >> v2:172.21.15.73:3300/0 conn(0x7faef415a790 0x7faef40cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.816+0000 7faefba00640 1 -- 172.21.15.46:0/1595609212 >> 172.21.15.46:0/1595609212 conn(0x7faef4099a10 msgr2=0x7faef4089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:06.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.816+0000 7faefba00640 1 -- 172.21.15.46:0/1595609212 shutdown_connections 2024-03-29T08:18:06.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.816+0000 7faefba00640 1 -- 172.21.15.46:0/1595609212 wait complete. 2024-03-29T08:18:06.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faefba00640 1 Processor -- start 2024-03-29T08:18:06.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faefba00640 1 -- start start 2024-03-29T08:18:06.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faefba00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef40cbbe0 0x7faef414e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faefba00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faef40cf340 0x7faef4155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faefba00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef415a790 0x7faef41508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faefba00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef41567d0 con 0x7faef415a790 2024-03-29T08:18:06.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faefba00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef4155bc0 con 0x7faef40cbbe0 2024-03-29T08:18:06.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faefba00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef4155ea0 con 0x7faef40cf340 2024-03-29T08:18:06.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faef9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef415a790 0x7faef41508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faef9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef40cbbe0 0x7faef414e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faeebe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faef40cf340 0x7faef4155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faef9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef40cbbe0 0x7faef414e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:45852/0 (socket says 172.21.15.46:45852) 2024-03-29T08:18:06.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faeebe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faef40cf340 0x7faef4155170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:34214/0 (socket says 172.21.15.46:34214) 2024-03-29T08:18:06.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faef9000640 1 -- 172.21.15.46:0/2522369565 learned_addr learned my addr 172.21.15.46:0/2522369565 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faef9a00640 1 -- 172.21.15.46:0/2522369565 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faef40cf340 msgr2=0x7faef4155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faef9a00640 1 --2- 172.21.15.46:0/2522369565 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faef40cf340 0x7faef4155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faef9a00640 1 -- 172.21.15.46:0/2522369565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef40cbbe0 msgr2=0x7faef414e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faef9a00640 1 --2- 172.21.15.46:0/2522369565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef40cbbe0 0x7faef414e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faef9a00640 1 -- 172.21.15.46:0/2522369565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec009b30 con 0x7faef415a790 2024-03-29T08:18:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faeebe00640 1 --2- 172.21.15.46:0/2522369565 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faef40cf340 0x7faef4155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faef9a00640 1 --2- 172.21.15.46:0/2522369565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef415a790 0x7faef41508d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faeec0ab000 tx=0x7faeec019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faef1600640 1 -- 172.21.15.46:0/2522369565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec0b8670 con 0x7faef415a790 2024-03-29T08:18:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faef1600640 1 -- 172.21.15.46:0/2522369565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeec0b87d0 con 0x7faef415a790 2024-03-29T08:18:06.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faef1600640 1 -- 172.21.15.46:0/2522369565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec0b8b80 con 0x7faef415a790 2024-03-29T08:18:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faefba00640 1 -- 172.21.15.46:0/2522369565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef4156180 con 0x7faef415a790 2024-03-29T08:18:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.820+0000 7faefba00640 1 -- 172.21.15.46:0/2522369565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef4156c60 con 0x7faef415a790 2024-03-29T08:18:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.824+0000 7faefba00640 1 -- 172.21.15.46:0/2522369565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef4156940 con 0x7faef415a790 2024-03-29T08:18:06.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.824+0000 7faef1600640 1 -- 172.21.15.46:0/2522369565 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7faeec0b8ce0 con 0x7faef415a790 2024-03-29T08:18:06.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.824+0000 7faef1600640 1 --2- 172.21.15.46:0/2522369565 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faed00666c0 0x7faed0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:06.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.824+0000 7faef1600640 1 -- 172.21.15.46:0/2522369565 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7faeec11efc0 con 0x7faef415a790 2024-03-29T08:18:06.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.824+0000 7faef9000640 1 --2- 172.21.15.46:0/2522369565 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faed00666c0 0x7faed0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:06.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.824+0000 7faef9000640 1 --2- 172.21.15.46:0/2522369565 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faed00666c0 0x7faed0068b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faef40c73b0 tx=0x7faee4009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:06.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.828+0000 7faef1600640 1 -- 172.21.15.46:0/2522369565 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7faeec0ec7b0 con 0x7faef415a790 2024-03-29T08:18:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.988+0000 7faefba00640 1 -- 172.21.15.46:0/2522369565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faef409e7d0 con 0x7faef415a790 2024-03-29T08:18:06.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.988+0000 7faef1600640 1 -- 172.21.15.46:0/2522369565 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7faeec0efdd0 con 0x7faef415a790 2024-03-29T08:18:06.992 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:18:06.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.992+0000 7faefba00640 1 -- 172.21.15.46:0/2522369565 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faed00666c0 msgr2=0x7faed0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.992+0000 7faefba00640 1 --2- 172.21.15.46:0/2522369565 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faed00666c0 0x7faed0068b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faef40c73b0 tx=0x7faee4009190 comp rx=0 tx=0).stop 2024-03-29T08:18:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.992+0000 7faefba00640 1 -- 172.21.15.46:0/2522369565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef415a790 msgr2=0x7faef41508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.992+0000 7faefba00640 1 --2- 172.21.15.46:0/2522369565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef415a790 0x7faef41508d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7faeec0ab000 tx=0x7faeec019640 comp rx=0 tx=0).stop 2024-03-29T08:18:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.992+0000 7faefba00640 1 -- 172.21.15.46:0/2522369565 shutdown_connections 2024-03-29T08:18:06.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.992+0000 7faefba00640 1 --2- 172.21.15.46:0/2522369565 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faed00666c0 0x7faed0068b80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.992+0000 7faefba00640 1 --2- 172.21.15.46:0/2522369565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faef415a790 0x7faef41508d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.992+0000 7faefba00640 1 --2- 172.21.15.46:0/2522369565 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faef40cf340 0x7faef4155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.992+0000 7faefba00640 1 --2- 172.21.15.46:0/2522369565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faef40cbbe0 0x7faef414e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:06.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.992+0000 7faefba00640 1 -- 172.21.15.46:0/2522369565 >> 172.21.15.46:0/2522369565 conn(0x7faef4099a10 msgr2=0x7faef4154540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:06.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.992+0000 7faefba00640 1 -- 172.21.15.46:0/2522369565 shutdown_connections 2024-03-29T08:18:06.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:06.992+0000 7faefba00640 1 -- 172.21.15.46:0/2522369565 wait complete. 2024-03-29T08:18:07.012 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:18:07.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad13e00640 1 Processor -- start 2024-03-29T08:18:07.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad13e00640 1 -- start start 2024-03-29T08:18:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad13e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad0c15a790 0x7fad0c0cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad13e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fad0c15ab70 0x7fad0c098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad13e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad0c0cc430 0x7fad0c098f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad13e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0c0cf700 con 0x7fad0c15a790 2024-03-29T08:18:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad13e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0c0ce660 con 0x7fad0c15ab70 2024-03-29T08:18:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad13e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0ce940 con 0x7fad0c0cc430 2024-03-29T08:18:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad11400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad0c15a790 0x7fad0c0cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad11400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad0c15a790 0x7fad0c0cbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37504/0 (socket says 172.21.15.46:37504) 2024-03-29T08:18:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad11400640 1 -- 172.21.15.46:0/1002744314 learned_addr learned my addr 172.21.15.46:0/1002744314 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:07.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad10a00640 1 --2- 172.21.15.46:0/1002744314 >> v2:172.21.15.73:3300/0 conn(0x7fad0c15ab70 0x7fad0c098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:07.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad11e00640 1 --2- 172.21.15.46:0/1002744314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad0c0cc430 0x7fad0c098f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:07.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad11400640 1 -- 172.21.15.46:0/1002744314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad0c0cc430 msgr2=0x7fad0c098f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:07.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad11400640 1 --2- 172.21.15.46:0/1002744314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad0c0cc430 0x7fad0c098f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:07.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad11400640 1 -- 172.21.15.46:0/1002744314 >> v2:172.21.15.73:3300/0 conn(0x7fad0c15ab70 msgr2=0x7fad0c098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:07.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad11400640 1 --2- 172.21.15.46:0/1002744314 >> v2:172.21.15.73:3300/0 conn(0x7fad0c15ab70 0x7fad0c098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:07.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad11400640 1 -- 172.21.15.46:0/1002744314 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0c0cec20 con 0x7fad0c15a790 2024-03-29T08:18:07.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.152+0000 7fad10a00640 1 --2- 172.21.15.46:0/1002744314 >> v2:172.21.15.73:3300/0 conn(0x7fad0c15ab70 0x7fad0c098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:07.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.156+0000 7fad11400640 1 --2- 172.21.15.46:0/1002744314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c15a790 0x7fad0c0cbef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fad0c071a70 tx=0x7fad0001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5a5f5c6eeea74c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.156+0000 7fad0be00640 1 -- 172.21.15.46:0/1002744314 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad000aa610 con 0x7fad0c15a790 2024-03-29T08:18:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.156+0000 7fad0be00640 1 -- 172.21.15.46:0/1002744314 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad000a6040 con 0x7fad0c15a790 2024-03-29T08:18:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.156+0000 7fad0be00640 1 -- 172.21.15.46:0/1002744314 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad000aa990 con 0x7fad0c15a790 2024-03-29T08:18:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.156+0000 7fad13e00640 1 -- 172.21.15.46:0/1002744314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c15a790 msgr2=0x7fad0c0cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.156+0000 7fad13e00640 1 --2- 172.21.15.46:0/1002744314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c15a790 0x7fad0c0cbef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fad0c071a70 tx=0x7fad0001c4f0 comp rx=0 tx=0).stop 2024-03-29T08:18:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.156+0000 7fad13e00640 1 -- 172.21.15.46:0/1002744314 shutdown_connections 2024-03-29T08:18:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.156+0000 7fad13e00640 1 --2- 172.21.15.46:0/1002744314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad0c0cc430 0x7fad0c098f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.156+0000 7fad13e00640 1 --2- 172.21.15.46:0/1002744314 >> v2:172.21.15.73:3300/0 conn(0x7fad0c15ab70 0x7fad0c098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.156+0000 7fad13e00640 1 --2- 172.21.15.46:0/1002744314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c15a790 0x7fad0c0cbef0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.156+0000 7fad13e00640 1 -- 172.21.15.46:0/1002744314 >> 172.21.15.46:0/1002744314 conn(0x7fad0c0b21a0 msgr2=0x7fad0c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:07.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.156+0000 7fad13e00640 1 -- 172.21.15.46:0/1002744314 shutdown_connections 2024-03-29T08:18:07.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.156+0000 7fad13e00640 1 -- 172.21.15.46:0/1002744314 wait complete. 2024-03-29T08:18:07.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad13e00640 1 Processor -- start 2024-03-29T08:18:07.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad13e00640 1 -- start start 2024-03-29T08:18:07.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad13e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c0cc430 0x7fad0c14de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:07.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad13e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad0c15a790 0x7fad0c15d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:07.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad13e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad0c15ab70 0x7fad0c0b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad13e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0c5490 con 0x7fad0c0cc430 2024-03-29T08:18:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad13e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c14e3c0 con 0x7fad0c15a790 2024-03-29T08:18:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad13e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c0c56e0 con 0x7fad0c15ab70 2024-03-29T08:18:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad11400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c0cc430 0x7fad0c14de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad11400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c0cc430 0x7fad0c14de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37508/0 (socket says 172.21.15.46:37508) 2024-03-29T08:18:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad11400640 1 -- 172.21.15.46:0/4038843487 learned_addr learned my addr 172.21.15.46:0/4038843487 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad11e00640 1 --2- 172.21.15.46:0/4038843487 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad0c15ab70 0x7fad0c0b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad10a00640 1 --2- 172.21.15.46:0/4038843487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad0c15a790 0x7fad0c15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad11400640 1 -- 172.21.15.46:0/4038843487 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad0c15ab70 msgr2=0x7fad0c0b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad11400640 1 --2- 172.21.15.46:0/4038843487 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad0c15ab70 0x7fad0c0b4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad11400640 1 -- 172.21.15.46:0/4038843487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad0c15a790 msgr2=0x7fad0c15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad11400640 1 --2- 172.21.15.46:0/4038843487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad0c15a790 0x7fad0c15d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad11400640 1 -- 172.21.15.46:0/4038843487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad000094d0 con 0x7fad0c0cc430 2024-03-29T08:18:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad11400640 1 --2- 172.21.15.46:0/4038843487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c0cc430 0x7fad0c14de80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fad000ac040 tx=0x7fad0001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad09600640 1 -- 172.21.15.46:0/4038843487 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad000bd5f0 con 0x7fad0c0cc430 2024-03-29T08:18:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad09600640 1 -- 172.21.15.46:0/4038843487 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad000bd750 con 0x7fad0c0cc430 2024-03-29T08:18:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad09600640 1 -- 172.21.15.46:0/4038843487 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad000bdae0 con 0x7fad0c0cc430 2024-03-29T08:18:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad13e00640 1 -- 172.21.15.46:0/4038843487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0c0c59c0 con 0x7fad0c0cc430 2024-03-29T08:18:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.160+0000 7fad13e00640 1 -- 172.21.15.46:0/4038843487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c0c5f80 con 0x7fad0c0cc430 2024-03-29T08:18:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.164+0000 7fad13e00640 1 -- 172.21.15.46:0/4038843487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0c066c90 con 0x7fad0c0cc430 2024-03-29T08:18:07.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.164+0000 7fad09600640 1 -- 172.21.15.46:0/4038843487 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fad000b33f0 con 0x7fad0c0cc430 2024-03-29T08:18:07.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.164+0000 7fad09600640 1 --2- 172.21.15.46:0/4038843487 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7facec0666c0 0x7facec068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:07.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.164+0000 7fad09600640 1 -- 172.21.15.46:0/4038843487 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fad0011e460 con 0x7fad0c0cc430 2024-03-29T08:18:07.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.164+0000 7fad10a00640 1 --2- 172.21.15.46:0/4038843487 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7facec0666c0 0x7facec068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:07.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.164+0000 7fad10a00640 1 --2- 172.21.15.46:0/4038843487 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7facec0666c0 0x7facec068b80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fad0c06fc20 tx=0x7facf8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:07.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.168+0000 7fad09600640 1 -- 172.21.15.46:0/4038843487 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fad000ebbc0 con 0x7fad0c0cc430 2024-03-29T08:18:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.360+0000 7fad13e00640 1 -- 172.21.15.46:0/4038843487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad0c0b4750 con 0x7fad0c0cc430 2024-03-29T08:18:07.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.364+0000 7fad09600640 1 -- 172.21.15.46:0/4038843487 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1253 (secure 0 0 0) 0x7fad000c3500 con 0x7fad0c0cc430 2024-03-29T08:18:07.364 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:07.365 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/339038392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":339038392},{"type":"v1","addr":"172.21.15.46:6834","nonce":339038392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-03-29T08:18:07.365 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 2 2024-03-29T08:18:07.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.364+0000 7fad13e00640 1 -- 172.21.15.46:0/4038843487 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7facec0666c0 msgr2=0x7facec068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:07.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.364+0000 7fad13e00640 1 --2- 172.21.15.46:0/4038843487 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7facec0666c0 0x7facec068b80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fad0c06fc20 tx=0x7facf8008040 comp rx=0 tx=0).stop 2024-03-29T08:18:07.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.364+0000 7fad13e00640 1 -- 172.21.15.46:0/4038843487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c0cc430 msgr2=0x7fad0c14de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:07.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.364+0000 7fad13e00640 1 --2- 172.21.15.46:0/4038843487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c0cc430 0x7fad0c14de80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fad000ac040 tx=0x7fad0001ad30 comp rx=0 tx=0).stop 2024-03-29T08:18:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.364+0000 7fad13e00640 1 -- 172.21.15.46:0/4038843487 shutdown_connections 2024-03-29T08:18:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.364+0000 7fad13e00640 1 --2- 172.21.15.46:0/4038843487 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7facec0666c0 0x7facec068b80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.364+0000 7fad13e00640 1 --2- 172.21.15.46:0/4038843487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad0c15a790 0x7fad0c15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.364+0000 7fad13e00640 1 --2- 172.21.15.46:0/4038843487 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad0c15ab70 0x7fad0c0b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.364+0000 7fad13e00640 1 --2- 172.21.15.46:0/4038843487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad0c0cc430 0x7fad0c14de80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.364+0000 7fad13e00640 1 -- 172.21.15.46:0/4038843487 >> 172.21.15.46:0/4038843487 conn(0x7fad0c0b21a0 msgr2=0x7fad0c151720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.364+0000 7fad13e00640 1 -- 172.21.15.46:0/4038843487 shutdown_connections 2024-03-29T08:18:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:07.364+0000 7fad13e00640 1 -- 172.21.15.46:0/4038843487 wait complete. 2024-03-29T08:18:08.190 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:18:08.194 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:18:09.386 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:18:09.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.552+0000 7f5979800640 1 Processor -- start 2024-03-29T08:18:09.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.552+0000 7f5979800640 1 -- start start 2024-03-29T08:18:09.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.552+0000 7f5979800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5974144210 0x7f59740cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:09.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.552+0000 7f5979800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f59740ceb20 0x7f597406c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:09.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.552+0000 7f5979800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59740cb510 con 0x7f597415ada0 2024-03-29T08:18:09.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.552+0000 7f5979800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5974070580 con 0x7f5974144210 2024-03-29T08:18:09.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.552+0000 7f5979800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5974070860 con 0x7f59740ceb20 2024-03-29T08:18:09.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.552+0000 7f5972a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5974144210 0x7f59740cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.552+0000 7f5972a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5974144210 0x7f59740cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37512/0 (socket says 172.21.15.46:37512) 2024-03-29T08:18:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.552+0000 7f5972a00640 1 -- 172.21.15.46:0/1686905705 learned_addr learned my addr 172.21.15.46:0/1686905705 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.552+0000 7f5973400640 1 --2- 172.21.15.46:0/1686905705 >> v2:172.21.15.73:3300/0 conn(0x7f59740ceb20 0x7f597406c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.552+0000 7f5972a00640 1 -- 172.21.15.46:0/1686905705 >> v2:172.21.15.73:3300/0 conn(0x7f59740ceb20 msgr2=0x7f597406c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.552+0000 7f5972a00640 1 --2- 172.21.15.46:0/1686905705 >> v2:172.21.15.73:3300/0 conn(0x7f59740ceb20 0x7f597406c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.552+0000 7f5972a00640 1 -- 172.21.15.46:0/1686905705 >> v1:172.21.15.46:6789/0 conn(0x7f597415ada0 legacy=0x7f59740ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.552+0000 7f5972000640 1 -- 172.21.15.46:0/1686905705 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1917918823 0 0) 0x7f59740cb510 con 0x7f597415ada0 2024-03-29T08:18:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.552+0000 7f5972a00640 1 -- 172.21.15.46:0/1686905705 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5974070b40 con 0x7f5974144210 2024-03-29T08:18:09.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.556+0000 7f5972a00640 1 --2- 172.21.15.46:0/1686905705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5974144210 0x7f59740cb8b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f59740cf1a0 tx=0x7f596401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c27db9f0903664 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:09.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.556+0000 7f5972000640 1 -- 172.21.15.46:0/1686905705 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59640a7070 con 0x7f5974144210 2024-03-29T08:18:09.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.556+0000 7f5972000640 1 -- 172.21.15.46:0/1686905705 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59640ab430 con 0x7f5974144210 2024-03-29T08:18:09.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.556+0000 7f5972000640 1 -- 172.21.15.46:0/1686905705 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59640ab7c0 con 0x7f5974144210 2024-03-29T08:18:09.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.556+0000 7f5979800640 1 -- 172.21.15.46:0/1686905705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5974144210 msgr2=0x7f59740cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:09.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.556+0000 7f5979800640 1 --2- 172.21.15.46:0/1686905705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5974144210 0x7f59740cb8b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f59740cf1a0 tx=0x7f596401c520 comp rx=0 tx=0).stop 2024-03-29T08:18:09.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.556+0000 7f5979800640 1 -- 172.21.15.46:0/1686905705 shutdown_connections 2024-03-29T08:18:09.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.556+0000 7f5979800640 1 --2- 172.21.15.46:0/1686905705 >> v2:172.21.15.73:3300/0 conn(0x7f59740ceb20 0x7f597406c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:09.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.556+0000 7f5979800640 1 --2- 172.21.15.46:0/1686905705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5974144210 0x7f59740cb8b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:09.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.556+0000 7f5979800640 1 -- 172.21.15.46:0/1686905705 >> 172.21.15.46:0/1686905705 conn(0x7f5974066110 msgr2=0x7f5974069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:09.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.556+0000 7f5979800640 1 -- 172.21.15.46:0/1686905705 shutdown_connections 2024-03-29T08:18:09.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.556+0000 7f5979800640 1 -- 172.21.15.46:0/1686905705 wait complete. 2024-03-29T08:18:09.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f5979800640 1 Processor -- start 2024-03-29T08:18:09.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f5979800640 1 -- start start 2024-03-29T08:18:09.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f5979800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59740ceb20 0x7f5974159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:09.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f5979800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5974144210 0x7f597413d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:09.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f5979800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f597415ada0 0x7f597413de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:09.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f5979800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597413ff50 con 0x7f5974144210 2024-03-29T08:18:09.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f5979800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597415c020 con 0x7f597415ada0 2024-03-29T08:18:09.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f5979800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597415b780 con 0x7f59740ceb20 2024-03-29T08:18:09.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f596a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5974144210 0x7f597413d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:09.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f596a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5974144210 0x7f597413d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37516/0 (socket says 172.21.15.46:37516) 2024-03-29T08:18:09.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f596a000640 1 -- 172.21.15.46:0/966211788 learned_addr learned my addr 172.21.15.46:0/966211788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:09.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f5973400640 1 --2- 172.21.15.46:0/966211788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f597415ada0 0x7f597413de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:09.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f5972a00640 1 --2- 172.21.15.46:0/966211788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59740ceb20 0x7f5974159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:09.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f596a000640 1 -- 172.21.15.46:0/966211788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59740ceb20 msgr2=0x7f5974159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:09.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f596a000640 1 --2- 172.21.15.46:0/966211788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59740ceb20 0x7f5974159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:09.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f596a000640 1 -- 172.21.15.46:0/966211788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f597415ada0 msgr2=0x7f597413de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:09.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f596a000640 1 --2- 172.21.15.46:0/966211788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f597415ada0 0x7f597413de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:09.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f596a000640 1 -- 172.21.15.46:0/966211788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59640094d0 con 0x7f5974144210 2024-03-29T08:18:09.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f5972a00640 1 --2- 172.21.15.46:0/966211788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59740ceb20 0x7f5974159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:09.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f596a000640 1 --2- 172.21.15.46:0/966211788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5974144210 0x7f597413d8d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f595c001ec0 tx=0x7f595c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:09.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f596b400640 1 -- 172.21.15.46:0/966211788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c0054e0 con 0x7f5974144210 2024-03-29T08:18:09.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f596b400640 1 -- 172.21.15.46:0/966211788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595c003600 con 0x7f5974144210 2024-03-29T08:18:09.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f596b400640 1 -- 172.21.15.46:0/966211788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c0039b0 con 0x7f5974144210 2024-03-29T08:18:09.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f5979800640 1 -- 172.21.15.46:0/966211788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f597415ba60 con 0x7f5974144210 2024-03-29T08:18:09.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.560+0000 7f5979800640 1 -- 172.21.15.46:0/966211788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f597415a430 con 0x7f5974144210 2024-03-29T08:18:09.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.564+0000 7f5979800640 1 -- 172.21.15.46:0/966211788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59740ce280 con 0x7f5974144210 2024-03-29T08:18:09.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.564+0000 7f596b400640 1 -- 172.21.15.46:0/966211788 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f595c003b10 con 0x7f5974144210 2024-03-29T08:18:09.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.564+0000 7f596b400640 1 --2- 172.21.15.46:0/966211788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5950066690 0x7f5950068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:09.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.564+0000 7f596b400640 1 -- 172.21.15.46:0/966211788 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f595c0834f0 con 0x7f5974144210 2024-03-29T08:18:09.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.564+0000 7f5972a00640 1 --2- 172.21.15.46:0/966211788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5950066690 0x7f5950068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:09.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.564+0000 7f5972a00640 1 --2- 172.21.15.46:0/966211788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5950066690 0x7f5950068b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f59640af000 tx=0x7f5964008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:09.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.568+0000 7f596b400640 1 -- 172.21.15.46:0/966211788 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f595c050b20 con 0x7f5974144210 2024-03-29T08:18:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.760+0000 7f5979800640 1 -- 172.21.15.46:0/966211788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f597413fdb0 con 0x7f5974144210 2024-03-29T08:18:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.760+0000 7f596b400640 1 -- 172.21.15.46:0/966211788 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3518 (secure 0 0 0) 0x7f597413fdb0 con 0x7f5974144210 2024-03-29T08:18:09.763 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:09.763 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/339038392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":339038392},{"type":"v1","addr":"172.21.15.46:6834","nonce":339038392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3573036715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3573036715},{"type":"v1","addr":"172.21.15.73:6835","nonce":3573036715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/746744819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":746744819},{"type":"v1","addr":"172.21.15.46:6835","nonce":746744819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2780269530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2780269530},{"type":"v1","addr":"172.21.15.73:6837","nonce":2780269530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-03-29T08:18:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 3 2024-03-29T08:18:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.764+0000 7f5979800640 1 -- 172.21.15.46:0/966211788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5950066690 msgr2=0x7f5950068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.764+0000 7f5979800640 1 --2- 172.21.15.46:0/966211788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5950066690 0x7f5950068b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f59640af000 tx=0x7f5964008040 comp rx=0 tx=0).stop 2024-03-29T08:18:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.764+0000 7f5979800640 1 -- 172.21.15.46:0/966211788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5974144210 msgr2=0x7f597413d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.764+0000 7f5979800640 1 --2- 172.21.15.46:0/966211788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5974144210 0x7f597413d8d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f595c001ec0 tx=0x7f595c00c8e0 comp rx=0 tx=0).stop 2024-03-29T08:18:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.764+0000 7f5979800640 1 -- 172.21.15.46:0/966211788 shutdown_connections 2024-03-29T08:18:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.764+0000 7f5979800640 1 --2- 172.21.15.46:0/966211788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5950066690 0x7f5950068b50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.764+0000 7f5979800640 1 --2- 172.21.15.46:0/966211788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f597415ada0 0x7f597413de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.764+0000 7f5979800640 1 --2- 172.21.15.46:0/966211788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5974144210 0x7f597413d8d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.764+0000 7f5979800640 1 --2- 172.21.15.46:0/966211788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59740ceb20 0x7f5974159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.764+0000 7f5979800640 1 -- 172.21.15.46:0/966211788 >> 172.21.15.46:0/966211788 conn(0x7f5974066110 msgr2=0x7f59740aeaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.764+0000 7f5979800640 1 -- 172.21.15.46:0/966211788 shutdown_connections 2024-03-29T08:18:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.764+0000 7f5979800640 1 -- 172.21.15.46:0/966211788 wait complete. 2024-03-29T08:18:09.783 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:09.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.920+0000 7f654ca00640 1 Processor -- start 2024-03-29T08:18:09.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.920+0000 7f654ca00640 1 -- start start 2024-03-29T08:18:09.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.924+0000 7f654ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f654815a940 0x7f65480afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:09.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.924+0000 7f654ca00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f654815ad20 0x7f65480cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:09.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.924+0000 7f654ca00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65480b00d0 con 0x7f65480b0540 2024-03-29T08:18:09.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.924+0000 7f654ca00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65480b0ab0 con 0x7f654815a940 2024-03-29T08:18:09.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.924+0000 7f654ca00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65480b0d90 con 0x7f654815ad20 2024-03-29T08:18:09.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.924+0000 7f6545e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f654815a940 0x7f65480afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:09.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.924+0000 7f6546800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f65480b0540 0x7f65480cd070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41376/0 (socket says 172.21.15.46:41376) 2024-03-29T08:18:09.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.924+0000 7f6546800640 1 -- 172.21.15.46:0/2061299727 learned_addr learned my addr 172.21.15.46:0/2061299727 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:09.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.924+0000 7f653fe00640 1 --2- 172.21.15.46:0/2061299727 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f654815ad20 0x7f65480cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:09.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.924+0000 7f6545e00640 1 -- 172.21.15.46:0/2061299727 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f654815ad20 msgr2=0x7f65480cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:09.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.924+0000 7f6545e00640 1 --2- 172.21.15.46:0/2061299727 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f654815ad20 0x7f65480cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:09.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.924+0000 7f6545e00640 1 -- 172.21.15.46:0/2061299727 >> v1:172.21.15.46:6789/0 conn(0x7f65480b0540 legacy=0x7f65480cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:09.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.924+0000 7f6545e00640 1 -- 172.21.15.46:0/2061299727 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65480b1070 con 0x7f654815a940 2024-03-29T08:18:09.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.924+0000 7f6545400640 1 -- 172.21.15.46:0/2061299727 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881580129 0 0) 0x7f65480b00d0 con 0x7f65480b0540 2024-03-29T08:18:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.928+0000 7f6545e00640 1 --2- 172.21.15.46:0/2061299727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f654815a940 0x7f65480afb90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f65380097c0 tx=0x7f653801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce39ea893fe2034 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.928+0000 7f6545400640 1 -- 172.21.15.46:0/2061299727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65380ab670 con 0x7f654815a940 2024-03-29T08:18:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.928+0000 7f6545400640 1 -- 172.21.15.46:0/2061299727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65380ab7d0 con 0x7f654815a940 2024-03-29T08:18:09.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.928+0000 7f6545400640 1 -- 172.21.15.46:0/2061299727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65380abb60 con 0x7f654815a940 2024-03-29T08:18:09.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.928+0000 7f654ca00640 1 -- 172.21.15.46:0/2061299727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f654815a940 msgr2=0x7f65480afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:09.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.928+0000 7f654ca00640 1 --2- 172.21.15.46:0/2061299727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f654815a940 0x7f65480afb90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f65380097c0 tx=0x7f653801c470 comp rx=0 tx=0).stop 2024-03-29T08:18:09.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.928+0000 7f654ca00640 1 -- 172.21.15.46:0/2061299727 shutdown_connections 2024-03-29T08:18:09.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.928+0000 7f654ca00640 1 --2- 172.21.15.46:0/2061299727 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f654815ad20 0x7f65480cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:09.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.928+0000 7f654ca00640 1 --2- 172.21.15.46:0/2061299727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f654815a940 0x7f65480afb90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:09.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.928+0000 7f654ca00640 1 -- 172.21.15.46:0/2061299727 >> 172.21.15.46:0/2061299727 conn(0x7f6548089fd0 msgr2=0x7f654808a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.928+0000 7f654ca00640 1 -- 172.21.15.46:0/2061299727 shutdown_connections 2024-03-29T08:18:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.928+0000 7f654ca00640 1 -- 172.21.15.46:0/2061299727 wait complete. 2024-03-29T08:18:09.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.928+0000 7f654ca00640 1 Processor -- start 2024-03-29T08:18:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.928+0000 7f654ca00640 1 -- start start 2024-03-29T08:18:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f654ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65480b0540 0x7f6548159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f654ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f654815a940 0x7f6548152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f654ca00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f654815ad20 0x7f6548157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f654ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65481534e0 con 0x7f65480b0540 2024-03-29T08:18:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f654ca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6548158ec0 con 0x7f654815a940 2024-03-29T08:18:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f654ca00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65481582b0 con 0x7f654815ad20 2024-03-29T08:18:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f6545e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65480b0540 0x7f6548159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f6545e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65480b0540 0x7f6548159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37534/0 (socket says 172.21.15.46:37534) 2024-03-29T08:18:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f6545e00640 1 -- 172.21.15.46:0/1775404407 learned_addr learned my addr 172.21.15.46:0/1775404407 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f653fe00640 1 --2- 172.21.15.46:0/1775404407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f654815a940 0x7f6548152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f6546800640 1 --2- 172.21.15.46:0/1775404407 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f654815ad20 0x7f6548157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f6545e00640 1 -- 172.21.15.46:0/1775404407 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f654815ad20 msgr2=0x7f6548157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f6545e00640 1 --2- 172.21.15.46:0/1775404407 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f654815ad20 0x7f6548157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f6545e00640 1 -- 172.21.15.46:0/1775404407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f654815a940 msgr2=0x7f6548152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f6545e00640 1 --2- 172.21.15.46:0/1775404407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f654815a940 0x7f6548152fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f6545e00640 1 -- 172.21.15.46:0/1775404407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6538009420 con 0x7f65480b0540 2024-03-29T08:18:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f6545e00640 1 --2- 172.21.15.46:0/1775404407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65480b0540 0x7f6548159350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f653801a0a0 tx=0x7f653801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:09.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f653e000640 1 -- 172.21.15.46:0/1775404407 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65380b5810 con 0x7f65480b0540 2024-03-29T08:18:09.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f653e000640 1 -- 172.21.15.46:0/1775404407 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65380b5990 con 0x7f65480b0540 2024-03-29T08:18:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f654ca00640 1 -- 172.21.15.46:0/1775404407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6548158590 con 0x7f65480b0540 2024-03-29T08:18:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f653e000640 1 -- 172.21.15.46:0/1775404407 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65380b5d40 con 0x7f65480b0540 2024-03-29T08:18:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f654ca00640 1 -- 172.21.15.46:0/1775404407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6548158b50 con 0x7f65480b0540 2024-03-29T08:18:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f654ca00640 1 -- 172.21.15.46:0/1775404407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6548066c90 con 0x7f65480b0540 2024-03-29T08:18:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.932+0000 7f653e000640 1 -- 172.21.15.46:0/1775404407 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f65380c83f0 con 0x7f65480b0540 2024-03-29T08:18:09.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.936+0000 7f653e000640 1 --2- 172.21.15.46:0/1775404407 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6524066690 0x7f6524068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:09.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.936+0000 7f653e000640 1 -- 172.21.15.46:0/1775404407 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f653811ece0 con 0x7f65480b0540 2024-03-29T08:18:09.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.936+0000 7f653fe00640 1 --2- 172.21.15.46:0/1775404407 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6524066690 0x7f6524068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:09.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.936+0000 7f653fe00640 1 --2- 172.21.15.46:0/1775404407 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6524066690 0x7f6524068b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f65481547a0 tx=0x7f6530008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:09.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:09.940+0000 7f653e000640 1 -- 172.21.15.46:0/1775404407 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f65380ec420 con 0x7f65480b0540 2024-03-29T08:18:10.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.100+0000 7f654ca00640 1 -- 172.21.15.46:0/1775404407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f654806b590 con 0x7f65480b0540 2024-03-29T08:18:10.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.104+0000 7f653e000640 1 -- 172.21.15.46:0/1775404407 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12334 (secure 0 0 0) 0x7f65380efa40 con 0x7f65480b0540 2024-03-29T08:18:10.104 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:10.104 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:02.761059+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:10.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.104+0000 7f654ca00640 1 -- 172.21.15.46:0/1775404407 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6524066690 msgr2=0x7f6524068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.104+0000 7f654ca00640 1 --2- 172.21.15.46:0/1775404407 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6524066690 0x7f6524068b50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f65481547a0 tx=0x7f6530008040 comp rx=0 tx=0).stop 2024-03-29T08:18:10.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.104+0000 7f654ca00640 1 -- 172.21.15.46:0/1775404407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65480b0540 msgr2=0x7f6548159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.104+0000 7f654ca00640 1 --2- 172.21.15.46:0/1775404407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65480b0540 0x7f6548159350 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f653801a0a0 tx=0x7f653801acb0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.104+0000 7f654ca00640 1 -- 172.21.15.46:0/1775404407 shutdown_connections 2024-03-29T08:18:10.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.104+0000 7f654ca00640 1 --2- 172.21.15.46:0/1775404407 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6524066690 0x7f6524068b50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.104+0000 7f654ca00640 1 --2- 172.21.15.46:0/1775404407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f654815a940 0x7f6548152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.104+0000 7f654ca00640 1 --2- 172.21.15.46:0/1775404407 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f654815ad20 0x7f6548157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.104+0000 7f654ca00640 1 --2- 172.21.15.46:0/1775404407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65480b0540 0x7f6548159350 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.104+0000 7f654ca00640 1 -- 172.21.15.46:0/1775404407 >> 172.21.15.46:0/1775404407 conn(0x7f6548089fd0 msgr2=0x7f6548164ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:10.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.104+0000 7f654ca00640 1 -- 172.21.15.46:0/1775404407 shutdown_connections 2024-03-29T08:18:10.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.104+0000 7f654ca00640 1 -- 172.21.15.46:0/1775404407 wait complete. 2024-03-29T08:18:10.126 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:18:10.126 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:18:10.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.268+0000 7f0535800640 1 Processor -- start 2024-03-29T08:18:10.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.268+0000 7f0535800640 1 -- start start 2024-03-29T08:18:10.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.268+0000 7f0535800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f05300cd190 0x7f053006b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:10.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.268+0000 7f0535800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05300af710 0x7f053006b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:10.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.268+0000 7f0535800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f053015d040 con 0x7f05300af1c0 2024-03-29T08:18:10.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.268+0000 7f0535800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f053006c550 con 0x7f05300cd190 2024-03-29T08:18:10.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.268+0000 7f0535800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05300caba0 con 0x7f05300af710 2024-03-29T08:18:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.268+0000 7f052fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f05300cd190 0x7f053006b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.268+0000 7f0527400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05300af710 0x7f053006b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.268+0000 7f052fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f05300cd190 0x7f053006b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:45924/0 (socket says 172.21.15.46:45924) 2024-03-29T08:18:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.268+0000 7f052fe00640 1 -- 172.21.15.46:0/3064750418 learned_addr learned my addr 172.21.15.46:0/3064750418 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.268+0000 7f052fe00640 1 -- 172.21.15.46:0/3064750418 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05300af710 msgr2=0x7f053006b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.268+0000 7f052fe00640 1 --2- 172.21.15.46:0/3064750418 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05300af710 0x7f053006b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.268+0000 7f052fe00640 1 -- 172.21.15.46:0/3064750418 >> v1:172.21.15.73:6789/0 conn(0x7f05300af1c0 legacy=0x7f05300caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.268+0000 7f052fe00640 1 -- 172.21.15.46:0/3064750418 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05300cf800 con 0x7f05300cd190 2024-03-29T08:18:10.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.268+0000 7f0527400640 1 --2- 172.21.15.46:0/3064750418 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05300af710 0x7f053006b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:10.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.272+0000 7f052fe00640 1 --2- 172.21.15.46:0/3064750418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05300cd190 0x7f053006b240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0520009870 tx=0x7f052001ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42054c1d4b7c21b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.272+0000 7f052f400640 1 -- 172.21.15.46:0/3064750418 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05200a8a50 con 0x7f05300cd190 2024-03-29T08:18:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.272+0000 7f052f400640 1 -- 172.21.15.46:0/3064750418 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05200a8bb0 con 0x7f05300cd190 2024-03-29T08:18:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.272+0000 7f052f400640 1 -- 172.21.15.46:0/3064750418 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05200b0440 con 0x7f05300cd190 2024-03-29T08:18:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.272+0000 7f0535800640 1 -- 172.21.15.46:0/3064750418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05300cd190 msgr2=0x7f053006b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.272+0000 7f0535800640 1 --2- 172.21.15.46:0/3064750418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05300cd190 0x7f053006b240 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0520009870 tx=0x7f052001ed50 comp rx=0 tx=0).stop 2024-03-29T08:18:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.272+0000 7f0535800640 1 -- 172.21.15.46:0/3064750418 shutdown_connections 2024-03-29T08:18:10.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.272+0000 7f0535800640 1 --2- 172.21.15.46:0/3064750418 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05300af710 0x7f053006b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.272+0000 7f0535800640 1 --2- 172.21.15.46:0/3064750418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05300cd190 0x7f053006b240 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.272+0000 7f0535800640 1 -- 172.21.15.46:0/3064750418 >> 172.21.15.46:0/3064750418 conn(0x7f05300d0550 msgr2=0x7f05300d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:10.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.272+0000 7f0535800640 1 -- 172.21.15.46:0/3064750418 shutdown_connections 2024-03-29T08:18:10.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.272+0000 7f0535800640 1 -- 172.21.15.46:0/3064750418 wait complete. 2024-03-29T08:18:10.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0535800640 1 Processor -- start 2024-03-29T08:18:10.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0535800640 1 -- start start 2024-03-29T08:18:10.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0535800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05300af1c0 0x7f05300cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0535800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05300af710 0x7f0530155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0535800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05300cd190 0x7f0530156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0535800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05300c82a0 con 0x7f05300af710 2024-03-29T08:18:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0535800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0530156670 con 0x7f05300af1c0 2024-03-29T08:18:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0535800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05300c7620 con 0x7f05300cd190 2024-03-29T08:18:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0527400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05300af710 0x7f0530155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0527400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05300af710 0x7f0530155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37540/0 (socket says 172.21.15.46:37540) 2024-03-29T08:18:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0527400640 1 -- 172.21.15.46:0/4188812038 learned_addr learned my addr 172.21.15.46:0/4188812038 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f052fe00640 1 --2- 172.21.15.46:0/4188812038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05300af1c0 0x7f05300cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0534e00640 1 --2- 172.21.15.46:0/4188812038 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05300cd190 0x7f0530156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0527400640 1 -- 172.21.15.46:0/4188812038 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05300cd190 msgr2=0x7f0530156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0527400640 1 --2- 172.21.15.46:0/4188812038 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05300cd190 0x7f0530156130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0527400640 1 -- 172.21.15.46:0/4188812038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05300af1c0 msgr2=0x7f05300cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0527400640 1 --2- 172.21.15.46:0/4188812038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05300af1c0 0x7f05300cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0527400640 1 -- 172.21.15.46:0/4188812038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0518009420 con 0x7f05300af710 2024-03-29T08:18:10.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0527400640 1 --2- 172.21.15.46:0/4188812038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05300af710 0x7f0530155bf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f051800ec10 tx=0x7f051800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:10.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f052cc00640 1 -- 172.21.15.46:0/4188812038 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051800cbd0 con 0x7f05300af710 2024-03-29T08:18:10.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f052cc00640 1 -- 172.21.15.46:0/4188812038 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051800cd30 con 0x7f05300af710 2024-03-29T08:18:10.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0535800640 1 -- 172.21.15.46:0/4188812038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05200094d0 con 0x7f05300af710 2024-03-29T08:18:10.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f052cc00640 1 -- 172.21.15.46:0/4188812038 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0518013690 con 0x7f05300af710 2024-03-29T08:18:10.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.276+0000 7f0535800640 1 -- 172.21.15.46:0/4188812038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05300c7be0 con 0x7f05300af710 2024-03-29T08:18:10.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.280+0000 7f0535800640 1 -- 172.21.15.46:0/4188812038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05300c8510 con 0x7f05300af710 2024-03-29T08:18:10.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.280+0000 7f052cc00640 1 -- 172.21.15.46:0/4188812038 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f05180137f0 con 0x7f05300af710 2024-03-29T08:18:10.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.280+0000 7f052cc00640 1 --2- 172.21.15.46:0/4188812038 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f050c066690 0x7f050c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:10.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.280+0000 7f052cc00640 1 -- 172.21.15.46:0/4188812038 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f05180840c0 con 0x7f05300af710 2024-03-29T08:18:10.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.280+0000 7f052fe00640 1 --2- 172.21.15.46:0/4188812038 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f050c066690 0x7f050c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:10.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.280+0000 7f052fe00640 1 --2- 172.21.15.46:0/4188812038 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f050c066690 0x7f050c068b50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f05200b2000 tx=0x7f0520009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:10.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.284+0000 7f052cc00640 1 -- 172.21.15.46:0/4188812038 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f05180517a0 con 0x7f05300af710 2024-03-29T08:18:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.448+0000 7f0535800640 1 -- 172.21.15.46:0/4188812038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f053004e040 con 0x7f05300af710 2024-03-29T08:18:10.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.448+0000 7f052cc00640 1 -- 172.21.15.46:0/4188812038 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f051800cea0 con 0x7f05300af710 2024-03-29T08:18:10.449 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:18:10.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.448+0000 7f0535800640 1 -- 172.21.15.46:0/4188812038 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f050c066690 msgr2=0x7f050c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.448+0000 7f0535800640 1 --2- 172.21.15.46:0/4188812038 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f050c066690 0x7f050c068b50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f05200b2000 tx=0x7f0520009040 comp rx=0 tx=0).stop 2024-03-29T08:18:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.448+0000 7f0535800640 1 -- 172.21.15.46:0/4188812038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05300af710 msgr2=0x7f0530155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.448+0000 7f0535800640 1 --2- 172.21.15.46:0/4188812038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05300af710 0x7f0530155bf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f051800ec10 tx=0x7f051800c520 comp rx=0 tx=0).stop 2024-03-29T08:18:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.448+0000 7f0535800640 1 -- 172.21.15.46:0/4188812038 shutdown_connections 2024-03-29T08:18:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.448+0000 7f0535800640 1 --2- 172.21.15.46:0/4188812038 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f050c066690 0x7f050c068b50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.448+0000 7f0535800640 1 --2- 172.21.15.46:0/4188812038 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05300cd190 0x7f0530156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.448+0000 7f0535800640 1 --2- 172.21.15.46:0/4188812038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05300af710 0x7f0530155bf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.448+0000 7f0535800640 1 --2- 172.21.15.46:0/4188812038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05300af1c0 0x7f05300cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.448+0000 7f0535800640 1 -- 172.21.15.46:0/4188812038 >> 172.21.15.46:0/4188812038 conn(0x7f05300d0550 msgr2=0x7f053006e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.448+0000 7f0535800640 1 -- 172.21.15.46:0/4188812038 shutdown_connections 2024-03-29T08:18:10.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.448+0000 7f0535800640 1 -- 172.21.15.46:0/4188812038 wait complete. 2024-03-29T08:18:10.469 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T08:18:10.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.608+0000 7f0d70000640 1 Processor -- start 2024-03-29T08:18:10.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.608+0000 7f0d70000640 1 -- start start 2024-03-29T08:18:10.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.608+0000 7f0d70000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0d6815ad20 0x7f0d680cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:10.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.608+0000 7f0d70000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d680b0540 0x7f0d680cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:10.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.608+0000 7f0d70000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d680b0ab0 con 0x7f0d6815a940 2024-03-29T08:18:10.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.608+0000 7f0d70000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d680b0d90 con 0x7f0d6815ad20 2024-03-29T08:18:10.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.608+0000 7f0d70000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680b1070 con 0x7f0d680b0540 2024-03-29T08:18:10.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.608+0000 7f0d6e000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d680b0540 0x7f0d680cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:10.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.608+0000 7f0d6cc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0d6815ad20 0x7f0d680cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:10.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.608+0000 7f0d6e000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d680b0540 0x7f0d680cf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:34292/0 (socket says 172.21.15.46:34292) 2024-03-29T08:18:10.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.608+0000 7f0d6e000640 1 -- 172.21.15.46:0/4060689081 learned_addr learned my addr 172.21.15.46:0/4060689081 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:10.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.608+0000 7f0d6e000640 1 -- 172.21.15.46:0/4060689081 >> v2:172.21.15.73:3300/0 conn(0x7f0d6815ad20 msgr2=0x7f0d680cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:10.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.608+0000 7f0d6e000640 1 --2- 172.21.15.46:0/4060689081 >> v2:172.21.15.73:3300/0 conn(0x7f0d6815ad20 0x7f0d680cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.608+0000 7f0d6e000640 1 -- 172.21.15.46:0/4060689081 >> v1:172.21.15.73:6789/0 conn(0x7f0d6815a940 legacy=0x7f0d680afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.608+0000 7f0d6e000640 1 -- 172.21.15.46:0/4060689081 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d680b1350 con 0x7f0d680b0540 2024-03-29T08:18:10.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.612+0000 7f0d6e000640 1 --2- 172.21.15.46:0/4060689081 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d680b0540 0x7f0d680cf1f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009e50 tx=0x7f0d6001ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c8c050ee7108245 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:10.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.612+0000 7f0d67e00640 1 -- 172.21.15.46:0/4060689081 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d600a58f0 con 0x7f0d680b0540 2024-03-29T08:18:10.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.612+0000 7f0d67e00640 1 -- 172.21.15.46:0/4060689081 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d600a5a50 con 0x7f0d680b0540 2024-03-29T08:18:10.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.612+0000 7f0d67e00640 1 -- 172.21.15.46:0/4060689081 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d600a5de0 con 0x7f0d680b0540 2024-03-29T08:18:10.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.612+0000 7f0d70000640 1 -- 172.21.15.46:0/4060689081 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d680b0540 msgr2=0x7f0d680cf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.612+0000 7f0d70000640 1 --2- 172.21.15.46:0/4060689081 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d680b0540 0x7f0d680cf1f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009e50 tx=0x7f0d6001ca00 comp rx=0 tx=0).stop 2024-03-29T08:18:10.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.612+0000 7f0d70000640 1 -- 172.21.15.46:0/4060689081 shutdown_connections 2024-03-29T08:18:10.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.612+0000 7f0d70000640 1 --2- 172.21.15.46:0/4060689081 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d680b0540 0x7f0d680cf1f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.612+0000 7f0d70000640 1 --2- 172.21.15.46:0/4060689081 >> v2:172.21.15.73:3300/0 conn(0x7f0d6815ad20 0x7f0d680cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.612+0000 7f0d70000640 1 -- 172.21.15.46:0/4060689081 >> 172.21.15.46:0/4060689081 conn(0x7f0d68089fd0 msgr2=0x7f0d6808a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:10.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.612+0000 7f0d70000640 1 -- 172.21.15.46:0/4060689081 shutdown_connections 2024-03-29T08:18:10.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.612+0000 7f0d70000640 1 -- 172.21.15.46:0/4060689081 wait complete. 2024-03-29T08:18:10.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d70000640 1 Processor -- start 2024-03-29T08:18:10.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d70000640 1 -- start start 2024-03-29T08:18:10.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d70000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d680b0540 0x7f0d681532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d70000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d6815a940 0x7f0d680c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d70000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d6815ad20 0x7f0d6806f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d70000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d68153830 con 0x7f0d6815a940 2024-03-29T08:18:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d70000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680c8690 con 0x7f0d6815ad20 2024-03-29T08:18:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d70000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d680c8970 con 0x7f0d680b0540 2024-03-29T08:18:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d6cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d6815a940 0x7f0d680c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d6e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d6815ad20 0x7f0d6806f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d6d600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d680b0540 0x7f0d681532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d6cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d6815a940 0x7f0d680c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37542/0 (socket says 172.21.15.46:37542) 2024-03-29T08:18:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d6cc00640 1 -- 172.21.15.46:0/2921516413 learned_addr learned my addr 172.21.15.46:0/2921516413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d6cc00640 1 -- 172.21.15.46:0/2921516413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d680b0540 msgr2=0x7f0d681532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d6cc00640 1 --2- 172.21.15.46:0/2921516413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d680b0540 0x7f0d681532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d6cc00640 1 -- 172.21.15.46:0/2921516413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d6815ad20 msgr2=0x7f0d6806f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d6cc00640 1 --2- 172.21.15.46:0/2921516413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d6815ad20 0x7f0d6806f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d6cc00640 1 -- 172.21.15.46:0/2921516413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d60009b30 con 0x7f0d6815a940 2024-03-29T08:18:10.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d6cc00640 1 --2- 172.21.15.46:0/2921516413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d6815a940 0x7f0d680c7cb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400e870 tx=0x7f0d5400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:10.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d65600640 1 -- 172.21.15.46:0/2921516413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d54009510 con 0x7f0d6815a940 2024-03-29T08:18:10.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d65600640 1 -- 172.21.15.46:0/2921516413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5400eec0 con 0x7f0d6815a940 2024-03-29T08:18:10.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d65600640 1 -- 172.21.15.46:0/2921516413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5400cb40 con 0x7f0d6815a940 2024-03-29T08:18:10.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d70000640 1 -- 172.21.15.46:0/2921516413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d680c8c50 con 0x7f0d6815a940 2024-03-29T08:18:10.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.616+0000 7f0d70000640 1 -- 172.21.15.46:0/2921516413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d680c9210 con 0x7f0d6815a940 2024-03-29T08:18:10.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.620+0000 7f0d70000640 1 -- 172.21.15.46:0/2921516413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d68066c90 con 0x7f0d6815a940 2024-03-29T08:18:10.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.620+0000 7f0d65600640 1 -- 172.21.15.46:0/2921516413 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0d5400cca0 con 0x7f0d6815a940 2024-03-29T08:18:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.620+0000 7f0d65600640 1 --2- 172.21.15.46:0/2921516413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0d480666c0 0x7f0d48068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.620+0000 7f0d65600640 1 -- 172.21.15.46:0/2921516413 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0d54083f60 con 0x7f0d6815a940 2024-03-29T08:18:10.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.620+0000 7f0d6d600640 1 --2- 172.21.15.46:0/2921516413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0d480666c0 0x7f0d48068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:10.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.620+0000 7f0d6d600640 1 --2- 172.21.15.46:0/2921516413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0d480666c0 0x7f0d48068b80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c007fd0 tx=0x7f0d5c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:10.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.624+0000 7f0d65600640 1 -- 172.21.15.46:0/2921516413 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0d54051630 con 0x7f0d6815a940 2024-03-29T08:18:10.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.784+0000 7f0d70000640 1 -- 172.21.15.46:0/2921516413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0d6815df30 con 0x7f0d6815a940 2024-03-29T08:18:10.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.784+0000 7f0d65600640 1 -- 172.21.15.46:0/2921516413 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0d54054c50 con 0x7f0d6815a940 2024-03-29T08:18:10.788 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:10.788 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-03-29T08:18:10.788 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:18:10.788 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-03-29T08:18:10.788 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-03-29T08:18:10.788 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:18:10.788 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-03-29T08:18:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.788+0000 7f0d70000640 1 -- 172.21.15.46:0/2921516413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0d480666c0 msgr2=0x7f0d48068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.788+0000 7f0d70000640 1 --2- 172.21.15.46:0/2921516413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0d480666c0 0x7f0d48068b80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c007fd0 tx=0x7f0d5c009040 comp rx=0 tx=0).stop 2024-03-29T08:18:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.788+0000 7f0d70000640 1 -- 172.21.15.46:0/2921516413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d6815a940 msgr2=0x7f0d680c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.788+0000 7f0d70000640 1 --2- 172.21.15.46:0/2921516413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d6815a940 0x7f0d680c7cb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400e870 tx=0x7f0d5400ec70 comp rx=0 tx=0).stop 2024-03-29T08:18:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.788+0000 7f0d70000640 1 -- 172.21.15.46:0/2921516413 shutdown_connections 2024-03-29T08:18:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.788+0000 7f0d70000640 1 --2- 172.21.15.46:0/2921516413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0d480666c0 0x7f0d48068b80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.788+0000 7f0d70000640 1 --2- 172.21.15.46:0/2921516413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d6815ad20 0x7f0d6806f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.788+0000 7f0d70000640 1 --2- 172.21.15.46:0/2921516413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d6815a940 0x7f0d680c7cb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.788+0000 7f0d70000640 1 --2- 172.21.15.46:0/2921516413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d680b0540 0x7f0d681532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.788+0000 7f0d70000640 1 -- 172.21.15.46:0/2921516413 >> 172.21.15.46:0/2921516413 conn(0x7f0d68089fd0 msgr2=0x7f0d68159bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.788+0000 7f0d70000640 1 -- 172.21.15.46:0/2921516413 shutdown_connections 2024-03-29T08:18:10.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.788+0000 7f0d70000640 1 -- 172.21.15.46:0/2921516413 wait complete. 2024-03-29T08:18:10.808 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T08:18:10.808 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T08:18:10.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.952+0000 7f47bd800640 1 Processor -- start 2024-03-29T08:18:10.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.952+0000 7f47bd800640 1 -- start start 2024-03-29T08:18:10.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.952+0000 7f47bd800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f47b806db40 0x7f47b806df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:10.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.952+0000 7f47bd800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b806e4b0 con 0x7f47b80cec80 2024-03-29T08:18:10.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.952+0000 7f47bd800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47b806cd80 con 0x7f47b8140f40 2024-03-29T08:18:10.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.952+0000 7f47bd800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b80cbff0 con 0x7f47b806db40 2024-03-29T08:18:10.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.952+0000 7f47ae000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f47b80cec80 0x7f47b8141770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41382/0 (socket says 172.21.15.46:41382) 2024-03-29T08:18:10.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.952+0000 7f47ae000640 1 -- 172.21.15.46:0/3000800513 learned_addr learned my addr 172.21.15.46:0/3000800513 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:10.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.952+0000 7f47b7400640 1 --2- 172.21.15.46:0/3000800513 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f47b806db40 0x7f47b806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:10.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.952+0000 7f47b6000640 1 -- 172.21.15.46:0/3000800513 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246006316 0 0) 0x7f47b806e4b0 con 0x7f47b80cec80 2024-03-29T08:18:10.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.952+0000 7f47b6000640 1 -- 172.21.15.46:0/3000800513 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f479c003530 con 0x7f47b80cec80 2024-03-29T08:18:10.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.952+0000 7f47b6000640 1 -- 172.21.15.46:0/3000800513 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202922712 0 0) 0x7f47b806cd80 con 0x7f47b8140f40 2024-03-29T08:18:10.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.952+0000 7f47b6000640 1 -- 172.21.15.46:0/3000800513 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47b806e4b0 con 0x7f47b8140f40 2024-03-29T08:18:10.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.952+0000 7f47b7400640 1 -- 172.21.15.46:0/3000800513 >> v1:172.21.15.73:6789/0 conn(0x7f47b8140f40 legacy=0x7f47b80cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.952+0000 7f47b7400640 1 -- 172.21.15.46:0/3000800513 >> v1:172.21.15.46:6789/0 conn(0x7f47b80cec80 legacy=0x7f47b8141770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.952+0000 7f47b7400640 1 -- 172.21.15.46:0/3000800513 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b806d060 con 0x7f47b806db40 2024-03-29T08:18:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.956+0000 7f47b7400640 1 --2- 172.21.15.46:0/3000800513 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f47b806db40 0x7f47b806df70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f47b00097f0 tx=0x7f47b001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f262629268fbd1b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.956+0000 7f47b6000640 1 -- 172.21.15.46:0/3000800513 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b00aa610 con 0x7f47b806db40 2024-03-29T08:18:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.956+0000 7f47b6000640 1 -- 172.21.15.46:0/3000800513 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47b00a6040 con 0x7f47b806db40 2024-03-29T08:18:10.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.956+0000 7f47b6000640 1 -- 172.21.15.46:0/3000800513 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47b00aa990 con 0x7f47b806db40 2024-03-29T08:18:10.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.956+0000 7f47bd800640 1 -- 172.21.15.46:0/3000800513 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f47b806db40 msgr2=0x7f47b806df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.956+0000 7f47bd800640 1 --2- 172.21.15.46:0/3000800513 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f47b806db40 0x7f47b806df70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f47b00097f0 tx=0x7f47b001c4b0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.956+0000 7f47bd800640 1 -- 172.21.15.46:0/3000800513 shutdown_connections 2024-03-29T08:18:10.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.956+0000 7f47bd800640 1 --2- 172.21.15.46:0/3000800513 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f47b806db40 0x7f47b806df70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.956+0000 7f47bd800640 1 -- 172.21.15.46:0/3000800513 >> 172.21.15.46:0/3000800513 conn(0x7f47b80b42d0 msgr2=0x7f47b8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:10.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.956+0000 7f47bd800640 1 -- 172.21.15.46:0/3000800513 shutdown_connections 2024-03-29T08:18:10.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.956+0000 7f47bd800640 1 -- 172.21.15.46:0/3000800513 wait complete. 2024-03-29T08:18:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47bd800640 1 Processor -- start 2024-03-29T08:18:10.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47bd800640 1 -- start start 2024-03-29T08:18:10.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47bd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b806db40 0x7f47b815d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:10.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47bd800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f47b80cec80 0x7f47b813ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:10.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47bd800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47b8140f40 0x7f47b80aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:10.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47bd800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b80aef70 con 0x7f47b806db40 2024-03-29T08:18:10.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47bd800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b815dc30 con 0x7f47b8140f40 2024-03-29T08:18:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47bd800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47b81592c0 con 0x7f47b80cec80 2024-03-29T08:18:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47b6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b806db40 0x7f47b815d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47b6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b806db40 0x7f47b815d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37558/0 (socket says 172.21.15.46:37558) 2024-03-29T08:18:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47b6a00640 1 -- 172.21.15.46:0/4203002065 learned_addr learned my addr 172.21.15.46:0/4203002065 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47ae000640 1 --2- 172.21.15.46:0/4203002065 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f47b80cec80 0x7f47b813ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47b7400640 1 --2- 172.21.15.46:0/4203002065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47b8140f40 0x7f47b80aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47b6a00640 1 -- 172.21.15.46:0/4203002065 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f47b80cec80 msgr2=0x7f47b813ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47b6a00640 1 --2- 172.21.15.46:0/4203002065 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f47b80cec80 0x7f47b813ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47b6a00640 1 -- 172.21.15.46:0/4203002065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47b8140f40 msgr2=0x7f47b80aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47b6a00640 1 --2- 172.21.15.46:0/4203002065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47b8140f40 0x7f47b80aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47b6a00640 1 -- 172.21.15.46:0/4203002065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b0009420 con 0x7f47b806db40 2024-03-29T08:18:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47ae000640 1 --2- 172.21.15.46:0/4203002065 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f47b80cec80 0x7f47b813ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:10.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47b6a00640 1 --2- 172.21.15.46:0/4203002065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b806db40 0x7f47b815d6f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f47a800c770 tx=0x7f47a800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:10.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47af400640 1 -- 172.21.15.46:0/4203002065 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8003b80 con 0x7f47b806db40 2024-03-29T08:18:10.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47af400640 1 -- 172.21.15.46:0/4203002065 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a8003ce0 con 0x7f47b806db40 2024-03-29T08:18:10.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47af400640 1 -- 172.21.15.46:0/4203002065 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a800aa90 con 0x7f47b806db40 2024-03-29T08:18:10.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47bd800640 1 -- 172.21.15.46:0/4203002065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b81595a0 con 0x7f47b806db40 2024-03-29T08:18:10.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.960+0000 7f47bd800640 1 -- 172.21.15.46:0/4203002065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b8159b60 con 0x7f47b806db40 2024-03-29T08:18:10.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.964+0000 7f47bd800640 1 -- 172.21.15.46:0/4203002065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b8066c90 con 0x7f47b806db40 2024-03-29T08:18:10.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.964+0000 7f47af400640 1 -- 172.21.15.46:0/4203002065 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f47a800abf0 con 0x7f47b806db40 2024-03-29T08:18:10.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.964+0000 7f47af400640 1 --2- 172.21.15.46:0/4203002065 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4794066690 0x7f4794068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:10.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.964+0000 7f47af400640 1 -- 172.21.15.46:0/4203002065 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f47a8083d10 con 0x7f47b806db40 2024-03-29T08:18:10.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.964+0000 7f47ae000640 1 --2- 172.21.15.46:0/4203002065 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4794066690 0x7f4794068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:10.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.964+0000 7f47ae000640 1 --2- 172.21.15.46:0/4203002065 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4794066690 0x7f4794068b50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47b815a280 tx=0x7f47a0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:10.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:10.968+0000 7f47af400640 1 -- 172.21.15.46:0/4203002065 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f47a8089020 con 0x7f47b806db40 2024-03-29T08:18:11.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:11.128+0000 7f47bd800640 1 -- 172.21.15.46:0/4203002065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f47b80ad6b0 con 0x7f47b806db40 2024-03-29T08:18:11.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:11.896+0000 7f47af400640 1 -- 172.21.15.46:0/4203002065 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f47a8051330 con 0x7f47b806db40 2024-03-29T08:18:11.899 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-03-29T08:18:11.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:11.900+0000 7f47bd800640 1 -- 172.21.15.46:0/4203002065 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4794066690 msgr2=0x7f4794068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:11.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:11.900+0000 7f47bd800640 1 --2- 172.21.15.46:0/4203002065 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4794066690 0x7f4794068b50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f47b815a280 tx=0x7f47a0009040 comp rx=0 tx=0).stop 2024-03-29T08:18:11.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:11.900+0000 7f47bd800640 1 -- 172.21.15.46:0/4203002065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b806db40 msgr2=0x7f47b815d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:11.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:11.900+0000 7f47bd800640 1 --2- 172.21.15.46:0/4203002065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b806db40 0x7f47b815d6f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f47a800c770 tx=0x7f47a800cc30 comp rx=0 tx=0).stop 2024-03-29T08:18:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:11.904+0000 7f47bd800640 1 -- 172.21.15.46:0/4203002065 shutdown_connections 2024-03-29T08:18:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:11.904+0000 7f47bd800640 1 --2- 172.21.15.46:0/4203002065 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4794066690 0x7f4794068b50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:11.904+0000 7f47bd800640 1 --2- 172.21.15.46:0/4203002065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47b8140f40 0x7f47b80aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:11.904+0000 7f47bd800640 1 --2- 172.21.15.46:0/4203002065 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f47b80cec80 0x7f47b813ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:11.904+0000 7f47bd800640 1 --2- 172.21.15.46:0/4203002065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47b806db40 0x7f47b815d6f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:11.904+0000 7f47bd800640 1 -- 172.21.15.46:0/4203002065 >> 172.21.15.46:0/4203002065 conn(0x7f47b80b42d0 msgr2=0x7f47b8141650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:11.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:11.904+0000 7f47bd800640 1 -- 172.21.15.46:0/4203002065 shutdown_connections 2024-03-29T08:18:11.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:11.904+0000 7f47bd800640 1 -- 172.21.15.46:0/4203002065 wait complete. 2024-03-29T08:18:11.931 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T08:18:12.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.080+0000 7fa650c00640 1 Processor -- start 2024-03-29T08:18:12.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.080+0000 7fa650c00640 1 -- start start 2024-03-29T08:18:12.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.084+0000 7fa650c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa64c15c9c0 0x7fa64c0ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:12.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.084+0000 7fa650c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa64c06be50 0x7fa64c098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.084+0000 7fa650c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa64c098f80 0x7fa64c1605e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.084+0000 7fa650c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64c0cf030 con 0x7fa64c098f80 2024-03-29T08:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.084+0000 7fa650c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64c0993f0 con 0x7fa64c15c9c0 2024-03-29T08:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.084+0000 7fa650c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0cfe50 con 0x7fa64c06be50 2024-03-29T08:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.084+0000 7fa64a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa64c098f80 0x7fa64c1605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.084+0000 7fa64a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa64c098f80 0x7fa64c1605e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37572/0 (socket says 172.21.15.46:37572) 2024-03-29T08:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.084+0000 7fa64a800640 1 -- 172.21.15.46:0/3716040141 learned_addr learned my addr 172.21.15.46:0/3716040141 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.084+0000 7fa64a800640 1 -- 172.21.15.46:0/3716040141 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa64c06be50 msgr2=0x7fa64c098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.084+0000 7fa64a800640 1 --2- 172.21.15.46:0/3716040141 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa64c06be50 0x7fa64c098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.084+0000 7fa64a800640 1 -- 172.21.15.46:0/3716040141 >> v2:172.21.15.73:3300/0 conn(0x7fa64c15c9c0 msgr2=0x7fa64c0ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.084+0000 7fa64a800640 1 --2- 172.21.15.46:0/3716040141 >> v2:172.21.15.73:3300/0 conn(0x7fa64c15c9c0 0x7fa64c0ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:12.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.084+0000 7fa64a800640 1 -- 172.21.15.46:0/3716040141 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c0cf240 con 0x7fa64c098f80 2024-03-29T08:18:12.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.084+0000 7fa64a800640 1 --2- 172.21.15.46:0/3716040141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa64c098f80 0x7fa64c1605e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa64c0ce270 tx=0x7fa64401c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4def6a558a9fd3c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:12.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa649400640 1 -- 172.21.15.46:0/3716040141 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6440a64b0 con 0x7fa64c098f80 2024-03-29T08:18:12.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa649400640 1 -- 172.21.15.46:0/3716040141 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6440a6610 con 0x7fa64c098f80 2024-03-29T08:18:12.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa649400640 1 -- 172.21.15.46:0/3716040141 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6440a69a0 con 0x7fa64c098f80 2024-03-29T08:18:12.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa650c00640 1 -- 172.21.15.46:0/3716040141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa64c098f80 msgr2=0x7fa64c1605e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:12.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa650c00640 1 --2- 172.21.15.46:0/3716040141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa64c098f80 0x7fa64c1605e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa64c0ce270 tx=0x7fa64401c730 comp rx=0 tx=0).stop 2024-03-29T08:18:12.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa650c00640 1 -- 172.21.15.46:0/3716040141 shutdown_connections 2024-03-29T08:18:12.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa650c00640 1 --2- 172.21.15.46:0/3716040141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa64c098f80 0x7fa64c1605e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa650c00640 1 --2- 172.21.15.46:0/3716040141 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa64c06be50 0x7fa64c098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa650c00640 1 --2- 172.21.15.46:0/3716040141 >> v2:172.21.15.73:3300/0 conn(0x7fa64c15c9c0 0x7fa64c0ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa650c00640 1 -- 172.21.15.46:0/3716040141 >> 172.21.15.46:0/3716040141 conn(0x7fa64c0b2220 msgr2=0x7fa64c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa650c00640 1 -- 172.21.15.46:0/3716040141 shutdown_connections 2024-03-29T08:18:12.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa650c00640 1 -- 172.21.15.46:0/3716040141 wait complete. 2024-03-29T08:18:12.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa650c00640 1 Processor -- start 2024-03-29T08:18:12.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa650c00640 1 -- start start 2024-03-29T08:18:12.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa650c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa64c06be50 0x7fa64c14abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:12.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa650c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa64c098f80 0x7fa64c09b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:12.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa650c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa64c15c9c0 0x7fa64c09c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:12.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa650c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0ad730 con 0x7fa64c098f80 2024-03-29T08:18:12.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa650c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c14b130 con 0x7fa64c15c9c0 2024-03-29T08:18:12.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa650c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c09a5c0 con 0x7fa64c06be50 2024-03-29T08:18:12.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa643e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa64c098f80 0x7fa64c09b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:12.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa643e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa64c098f80 0x7fa64c09b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37588/0 (socket says 172.21.15.46:37588) 2024-03-29T08:18:12.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa643e00640 1 -- 172.21.15.46:0/3261536015 learned_addr learned my addr 172.21.15.46:0/3261536015 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:12.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa64a800640 1 --2- 172.21.15.46:0/3261536015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa64c15c9c0 0x7fa64c09c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:12.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.088+0000 7fa649e00640 1 --2- 172.21.15.46:0/3261536015 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa64c06be50 0x7fa64c14abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:12.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.092+0000 7fa643e00640 1 -- 172.21.15.46:0/3261536015 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa64c06be50 msgr2=0x7fa64c14abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:12.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.092+0000 7fa643e00640 1 --2- 172.21.15.46:0/3261536015 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa64c06be50 0x7fa64c14abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:12.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.092+0000 7fa643e00640 1 -- 172.21.15.46:0/3261536015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa64c15c9c0 msgr2=0x7fa64c09c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:12.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.092+0000 7fa643e00640 1 --2- 172.21.15.46:0/3261536015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa64c15c9c0 0x7fa64c09c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:12.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.092+0000 7fa643e00640 1 -- 172.21.15.46:0/3261536015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa644009b30 con 0x7fa64c098f80 2024-03-29T08:18:12.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.092+0000 7fa643e00640 1 --2- 172.21.15.46:0/3261536015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa64c098f80 0x7fa64c09b660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa63400b580 tx=0x7fa63400ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:12.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.092+0000 7fa642000640 1 -- 172.21.15.46:0/3261536015 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634010870 con 0x7fa64c098f80 2024-03-29T08:18:12.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.092+0000 7fa642000640 1 -- 172.21.15.46:0/3261536015 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6340109f0 con 0x7fa64c098f80 2024-03-29T08:18:12.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.092+0000 7fa642000640 1 -- 172.21.15.46:0/3261536015 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634010da0 con 0x7fa64c098f80 2024-03-29T08:18:12.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.092+0000 7fa650c00640 1 -- 172.21.15.46:0/3261536015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64c09a8a0 con 0x7fa64c098f80 2024-03-29T08:18:12.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.092+0000 7fa650c00640 1 -- 172.21.15.46:0/3261536015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64c09ae60 con 0x7fa64c098f80 2024-03-29T08:18:12.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.092+0000 7fa650c00640 1 -- 172.21.15.46:0/3261536015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64c06c270 con 0x7fa64c098f80 2024-03-29T08:18:12.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.092+0000 7fa642000640 1 -- 172.21.15.46:0/3261536015 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa63400f3c0 con 0x7fa64c098f80 2024-03-29T08:18:12.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.096+0000 7fa642000640 1 --2- 172.21.15.46:0/3261536015 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa628066690 0x7fa628068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:12.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.096+0000 7fa642000640 1 -- 172.21.15.46:0/3261536015 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fa634083f40 con 0x7fa64c098f80 2024-03-29T08:18:12.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.096+0000 7fa649e00640 1 --2- 172.21.15.46:0/3261536015 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa628066690 0x7fa628068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:12.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.096+0000 7fa649e00640 1 --2- 172.21.15.46:0/3261536015 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa628066690 0x7fa628068b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0098f0 tx=0x7fa63c006f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.100+0000 7fa642000640 1 -- 172.21.15.46:0/3261536015 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa64c06c270 con 0x7fa64c098f80 2024-03-29T08:18:12.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.260+0000 7fa650c00640 1 -- 172.21.15.46:0/3261536015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa64c06c270 con 0x7fa64c098f80 2024-03-29T08:18:12.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.904+0000 7fa642000640 1 -- 172.21.15.46:0/3261536015 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa64c06c270 con 0x7fa64c098f80 2024-03-29T08:18:12.905 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-03-29T08:18:12.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.908+0000 7fa650c00640 1 -- 172.21.15.46:0/3261536015 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa628066690 msgr2=0x7fa628068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:12.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.908+0000 7fa650c00640 1 --2- 172.21.15.46:0/3261536015 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa628066690 0x7fa628068b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0098f0 tx=0x7fa63c006f90 comp rx=0 tx=0).stop 2024-03-29T08:18:12.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.908+0000 7fa650c00640 1 -- 172.21.15.46:0/3261536015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa64c098f80 msgr2=0x7fa64c09b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:12.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.908+0000 7fa650c00640 1 --2- 172.21.15.46:0/3261536015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa64c098f80 0x7fa64c09b660 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa63400b580 tx=0x7fa63400ba40 comp rx=0 tx=0).stop 2024-03-29T08:18:12.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.908+0000 7fa650c00640 1 -- 172.21.15.46:0/3261536015 shutdown_connections 2024-03-29T08:18:12.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.908+0000 7fa650c00640 1 --2- 172.21.15.46:0/3261536015 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa628066690 0x7fa628068b50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:12.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.908+0000 7fa650c00640 1 --2- 172.21.15.46:0/3261536015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa64c15c9c0 0x7fa64c09c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:12.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.908+0000 7fa650c00640 1 --2- 172.21.15.46:0/3261536015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa64c098f80 0x7fa64c09b660 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa63400b580 tx=0x7fa63400ba40 comp rx=0 tx=0).stop 2024-03-29T08:18:12.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.908+0000 7fa650c00640 1 --2- 172.21.15.46:0/3261536015 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa64c06be50 0x7fa64c14abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:12.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.908+0000 7fa650c00640 1 -- 172.21.15.46:0/3261536015 >> 172.21.15.46:0/3261536015 conn(0x7fa64c0b2220 msgr2=0x7fa64c15b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:12.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.908+0000 7fa650c00640 1 -- 172.21.15.46:0/3261536015 shutdown_connections 2024-03-29T08:18:12.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:12.908+0000 7fa650c00640 1 -- 172.21.15.46:0/3261536015 wait complete. 2024-03-29T08:18:12.953 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T08:18:13.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc51a00640 1 Processor -- start 2024-03-29T08:18:13.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc51a00640 1 -- start start 2024-03-29T08:18:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc51a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efc4c143eb0 0x7efc4c0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc51a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efc4c15efe0 0x7efc4c0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc51a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc4c0b1630 0x7efc4c0b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc51a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4c0b2360 con 0x7efc4c0b1630 2024-03-29T08:18:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc51a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4c15e500 con 0x7efc4c15efe0 2024-03-29T08:18:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc51a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c15e7e0 con 0x7efc4c143eb0 2024-03-29T08:18:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc4b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc4c0b1630 0x7efc4c0b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc4b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc4c0b1630 0x7efc4c0b1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37592/0 (socket says 172.21.15.46:37592) 2024-03-29T08:18:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc4b400640 1 -- 172.21.15.46:0/3963244218 learned_addr learned my addr 172.21.15.46:0/3963244218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc4aa00640 1 --2- 172.21.15.46:0/3963244218 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efc4c143eb0 0x7efc4c0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc42000640 1 --2- 172.21.15.46:0/3963244218 >> v2:172.21.15.73:3300/0 conn(0x7efc4c15efe0 0x7efc4c0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc4b400640 1 -- 172.21.15.46:0/3963244218 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efc4c143eb0 msgr2=0x7efc4c0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc4b400640 1 --2- 172.21.15.46:0/3963244218 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efc4c143eb0 0x7efc4c0cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc4b400640 1 -- 172.21.15.46:0/3963244218 >> v2:172.21.15.73:3300/0 conn(0x7efc4c15efe0 msgr2=0x7efc4c0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc4b400640 1 --2- 172.21.15.46:0/3963244218 >> v2:172.21.15.73:3300/0 conn(0x7efc4c15efe0 0x7efc4c0cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc4b400640 1 -- 172.21.15.46:0/3963244218 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c15eac0 con 0x7efc4c0b1630 2024-03-29T08:18:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc42000640 1 --2- 172.21.15.46:0/3963244218 >> v2:172.21.15.73:3300/0 conn(0x7efc4c15efe0 0x7efc4c0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.096+0000 7efc4aa00640 1 --2- 172.21.15.46:0/3963244218 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efc4c143eb0 0x7efc4c0cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.100+0000 7efc4b400640 1 --2- 172.21.15.46:0/3963244218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c0b1630 0x7efc4c0b1a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efc4c06e4b0 tx=0x7efc4401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1507e4409e15535e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.100+0000 7efc4a000640 1 -- 172.21.15.46:0/3963244218 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc440ab620 con 0x7efc4c0b1630 2024-03-29T08:18:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.100+0000 7efc4a000640 1 -- 172.21.15.46:0/3963244218 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc440ab780 con 0x7efc4c0b1630 2024-03-29T08:18:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.100+0000 7efc4a000640 1 -- 172.21.15.46:0/3963244218 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc440abb10 con 0x7efc4c0b1630 2024-03-29T08:18:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.100+0000 7efc51a00640 1 -- 172.21.15.46:0/3963244218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c0b1630 msgr2=0x7efc4c0b1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.100+0000 7efc51a00640 1 --2- 172.21.15.46:0/3963244218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c0b1630 0x7efc4c0b1a80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efc4c06e4b0 tx=0x7efc4401c520 comp rx=0 tx=0).stop 2024-03-29T08:18:13.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.100+0000 7efc51a00640 1 -- 172.21.15.46:0/3963244218 shutdown_connections 2024-03-29T08:18:13.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.100+0000 7efc51a00640 1 --2- 172.21.15.46:0/3963244218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c0b1630 0x7efc4c0b1a80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.100+0000 7efc51a00640 1 --2- 172.21.15.46:0/3963244218 >> v2:172.21.15.73:3300/0 conn(0x7efc4c15efe0 0x7efc4c0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.100+0000 7efc51a00640 1 --2- 172.21.15.46:0/3963244218 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efc4c143eb0 0x7efc4c0cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.100+0000 7efc51a00640 1 -- 172.21.15.46:0/3963244218 >> 172.21.15.46:0/3963244218 conn(0x7efc4c06a390 msgr2=0x7efc4c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:13.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.100+0000 7efc51a00640 1 -- 172.21.15.46:0/3963244218 shutdown_connections 2024-03-29T08:18:13.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.100+0000 7efc51a00640 1 -- 172.21.15.46:0/3963244218 wait complete. 2024-03-29T08:18:13.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc51a00640 1 Processor -- start 2024-03-29T08:18:13.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc51a00640 1 -- start start 2024-03-29T08:18:13.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc51a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efc4c0b1630 0x7efc4c13fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc51a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c143eb0 0x7efc4c13bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc51a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc4c15efe0 0x7efc4c13ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc51a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c0cdda0 con 0x7efc4c143eb0 2024-03-29T08:18:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc51a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c13e020 con 0x7efc4c15efe0 2024-03-29T08:18:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc51a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c0cdff0 con 0x7efc4c0b1630 2024-03-29T08:18:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc42000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c143eb0 0x7efc4c13bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc4aa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efc4c0b1630 0x7efc4c13fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc42000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c143eb0 0x7efc4c13bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37606/0 (socket says 172.21.15.46:37606) 2024-03-29T08:18:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc42000640 1 -- 172.21.15.46:0/1321370822 learned_addr learned my addr 172.21.15.46:0/1321370822 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc4b400640 1 --2- 172.21.15.46:0/1321370822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc4c15efe0 0x7efc4c13ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc42000640 1 -- 172.21.15.46:0/1321370822 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efc4c0b1630 msgr2=0x7efc4c13fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc42000640 1 --2- 172.21.15.46:0/1321370822 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efc4c0b1630 0x7efc4c13fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc42000640 1 -- 172.21.15.46:0/1321370822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc4c15efe0 msgr2=0x7efc4c13ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:13.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc42000640 1 --2- 172.21.15.46:0/1321370822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc4c15efe0 0x7efc4c13ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc42000640 1 -- 172.21.15.46:0/1321370822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc440094d0 con 0x7efc4c143eb0 2024-03-29T08:18:13.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc42000640 1 --2- 172.21.15.46:0/1321370822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c143eb0 0x7efc4c13bd10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efc3400c820 tx=0x7efc3400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:13.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc43400640 1 -- 172.21.15.46:0/1321370822 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc34007a30 con 0x7efc4c143eb0 2024-03-29T08:18:13.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc43400640 1 -- 172.21.15.46:0/1321370822 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc34007b90 con 0x7efc4c143eb0 2024-03-29T08:18:13.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc43400640 1 -- 172.21.15.46:0/1321370822 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3400ce10 con 0x7efc4c143eb0 2024-03-29T08:18:13.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc51a00640 1 -- 172.21.15.46:0/1321370822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4c13e290 con 0x7efc4c143eb0 2024-03-29T08:18:13.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.104+0000 7efc51a00640 1 -- 172.21.15.46:0/1321370822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4c0ce2d0 con 0x7efc4c143eb0 2024-03-29T08:18:13.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.108+0000 7efc51a00640 1 -- 172.21.15.46:0/1321370822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4c071a70 con 0x7efc4c143eb0 2024-03-29T08:18:13.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.108+0000 7efc43400640 1 -- 172.21.15.46:0/1321370822 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efc340113f0 con 0x7efc4c143eb0 2024-03-29T08:18:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.108+0000 7efc43400640 1 --2- 172.21.15.46:0/1321370822 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efc28066690 0x7efc28068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.108+0000 7efc43400640 1 -- 172.21.15.46:0/1321370822 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7efc340841e0 con 0x7efc4c143eb0 2024-03-29T08:18:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.108+0000 7efc4aa00640 1 --2- 172.21.15.46:0/1321370822 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efc28066690 0x7efc28068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.112+0000 7efc4aa00640 1 --2- 172.21.15.46:0/1321370822 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efc28066690 0x7efc28068b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efc4c06e150 tx=0x7efc3c007810 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:13.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.116+0000 7efc43400640 1 -- 172.21.15.46:0/1321370822 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efc340515c0 con 0x7efc4c143eb0 2024-03-29T08:18:13.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.308+0000 7efc51a00640 1 -- 172.21.15.46:0/1321370822 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7efc4c13e740 con 0x7efc4c143eb0 2024-03-29T08:18:13.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.316+0000 7efc43400640 1 -- 172.21.15.46:0/1321370822 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7efc34054be0 con 0x7efc4c143eb0 2024-03-29T08:18:13.316 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 2 and data pool 3 2024-03-29T08:18:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.320+0000 7efc51a00640 1 -- 172.21.15.46:0/1321370822 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efc28066690 msgr2=0x7efc28068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.320+0000 7efc51a00640 1 --2- 172.21.15.46:0/1321370822 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efc28066690 0x7efc28068b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efc4c06e150 tx=0x7efc3c007810 comp rx=0 tx=0).stop 2024-03-29T08:18:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.320+0000 7efc51a00640 1 -- 172.21.15.46:0/1321370822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c143eb0 msgr2=0x7efc4c13bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.320+0000 7efc51a00640 1 --2- 172.21.15.46:0/1321370822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c143eb0 0x7efc4c13bd10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efc3400c820 tx=0x7efc3400cce0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.320+0000 7efc51a00640 1 -- 172.21.15.46:0/1321370822 shutdown_connections 2024-03-29T08:18:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.320+0000 7efc51a00640 1 --2- 172.21.15.46:0/1321370822 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efc28066690 0x7efc28068b50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.320+0000 7efc51a00640 1 --2- 172.21.15.46:0/1321370822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc4c15efe0 0x7efc4c13ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.320+0000 7efc51a00640 1 --2- 172.21.15.46:0/1321370822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc4c143eb0 0x7efc4c13bd10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.320+0000 7efc51a00640 1 --2- 172.21.15.46:0/1321370822 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efc4c0b1630 0x7efc4c13fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.320+0000 7efc51a00640 1 -- 172.21.15.46:0/1321370822 >> 172.21.15.46:0/1321370822 conn(0x7efc4c06a390 msgr2=0x7efc4c15a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:13.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.324+0000 7efc51a00640 1 -- 172.21.15.46:0/1321370822 shutdown_connections 2024-03-29T08:18:13.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.324+0000 7efc51a00640 1 -- 172.21.15.46:0/1321370822 wait complete. 2024-03-29T08:18:13.361 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:13.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.500+0000 7faaddc00640 1 Processor -- start 2024-03-29T08:18:13.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.500+0000 7faaddc00640 1 -- start start 2024-03-29T08:18:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.500+0000 7faaddc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faad815cd90 0x7faad806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.500+0000 7faaddc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad806d130 con 0x7faad815d940 2024-03-29T08:18:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.500+0000 7faaddc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad815b8f0 con 0x7faad81593d0 2024-03-29T08:18:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.500+0000 7faaddc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad815ad50 con 0x7faad815cd90 2024-03-29T08:18:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.500+0000 7faad7400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7faad815d940 0x7faad81597b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41394/0 (socket says 172.21.15.46:41394) 2024-03-29T08:18:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.500+0000 7faad7400640 1 -- 172.21.15.46:0/2315514183 learned_addr learned my addr 172.21.15.46:0/2315514183 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:13.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.500+0000 7faace000640 1 --2- 172.21.15.46:0/2315514183 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faad815cd90 0x7faad806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.500+0000 7faad6000640 1 -- 172.21.15.46:0/2315514183 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546361099 0 0) 0x7faad806d130 con 0x7faad815d940 2024-03-29T08:18:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.500+0000 7faad6000640 1 -- 172.21.15.46:0/2315514183 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faabc003530 con 0x7faad815d940 2024-03-29T08:18:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.504+0000 7faad6000640 1 -- 172.21.15.46:0/2315514183 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3896695728 0 0) 0x7faad815b8f0 con 0x7faad81593d0 2024-03-29T08:18:13.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.504+0000 7faad6000640 1 -- 172.21.15.46:0/2315514183 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faad806d130 con 0x7faad81593d0 2024-03-29T08:18:13.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.504+0000 7faace000640 1 -- 172.21.15.46:0/2315514183 >> v1:172.21.15.73:6789/0 conn(0x7faad81593d0 legacy=0x7faad815d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:13.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.504+0000 7faace000640 1 -- 172.21.15.46:0/2315514183 >> v1:172.21.15.46:6789/0 conn(0x7faad815d940 legacy=0x7faad81597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:13.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.504+0000 7faace000640 1 -- 172.21.15.46:0/2315514183 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad815afc0 con 0x7faad815cd90 2024-03-29T08:18:13.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.504+0000 7faace000640 1 --2- 172.21.15.46:0/2315514183 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faad815cd90 0x7faad806cbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faac00097f0 tx=0x7faac001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8859a16b4784c155 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:13.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.504+0000 7faad6000640 1 -- 172.21.15.46:0/2315514183 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac00aa640 con 0x7faad815cd90 2024-03-29T08:18:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.504+0000 7faad6000640 1 -- 172.21.15.46:0/2315514183 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac00aa7a0 con 0x7faad815cd90 2024-03-29T08:18:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.504+0000 7faad6000640 1 -- 172.21.15.46:0/2315514183 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac00aab30 con 0x7faad815cd90 2024-03-29T08:18:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.504+0000 7faaddc00640 1 -- 172.21.15.46:0/2315514183 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faad815cd90 msgr2=0x7faad806cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.504+0000 7faaddc00640 1 --2- 172.21.15.46:0/2315514183 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faad815cd90 0x7faad806cbf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faac00097f0 tx=0x7faac001c4e0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.504+0000 7faaddc00640 1 -- 172.21.15.46:0/2315514183 shutdown_connections 2024-03-29T08:18:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.504+0000 7faaddc00640 1 --2- 172.21.15.46:0/2315514183 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faad815cd90 0x7faad806cbf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.504+0000 7faaddc00640 1 -- 172.21.15.46:0/2315514183 >> 172.21.15.46:0/2315514183 conn(0x7faad8066110 msgr2=0x7faad8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.504+0000 7faaddc00640 1 -- 172.21.15.46:0/2315514183 shutdown_connections 2024-03-29T08:18:13.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.504+0000 7faaddc00640 1 -- 172.21.15.46:0/2315514183 wait complete. 2024-03-29T08:18:13.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faaddc00640 1 Processor -- start 2024-03-29T08:18:13.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faaddc00640 1 -- start start 2024-03-29T08:18:13.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faaddc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faad81593d0 0x7faad8151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faaddc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faad815cd90 0x7faad8151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faaddc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faad815d940 0x7faad814ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faaddc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad8156060 con 0x7faad815d940 2024-03-29T08:18:13.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faaddc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad8156340 con 0x7faad81593d0 2024-03-29T08:18:13.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faaddc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad8156620 con 0x7faad815cd90 2024-03-29T08:18:13.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faad7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faad815d940 0x7faad814ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faad7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faad815d940 0x7faad814ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37618/0 (socket says 172.21.15.46:37618) 2024-03-29T08:18:13.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faad7400640 1 -- 172.21.15.46:0/3777965331 learned_addr learned my addr 172.21.15.46:0/3777965331 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:13.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faad6a00640 1 --2- 172.21.15.46:0/3777965331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faad81593d0 0x7faad8151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faace000640 1 --2- 172.21.15.46:0/3777965331 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faad815cd90 0x7faad8151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faad7400640 1 -- 172.21.15.46:0/3777965331 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faad815cd90 msgr2=0x7faad8151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:13.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faad7400640 1 --2- 172.21.15.46:0/3777965331 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faad815cd90 0x7faad8151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faad7400640 1 -- 172.21.15.46:0/3777965331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faad81593d0 msgr2=0x7faad8151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:13.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faad7400640 1 --2- 172.21.15.46:0/3777965331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faad81593d0 0x7faad8151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faad7400640 1 -- 172.21.15.46:0/3777965331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac0009420 con 0x7faad815d940 2024-03-29T08:18:13.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faad7400640 1 --2- 172.21.15.46:0/3777965331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faad815d940 0x7faad814ab80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faad000caa0 tx=0x7faad000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:13.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faacf400640 1 -- 172.21.15.46:0/3777965331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad000ac00 con 0x7faad815d940 2024-03-29T08:18:13.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faacf400640 1 -- 172.21.15.46:0/3777965331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad000ad80 con 0x7faad815d940 2024-03-29T08:18:13.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faacf400640 1 -- 172.21.15.46:0/3777965331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad0012640 con 0x7faad815d940 2024-03-29T08:18:13.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faaddc00640 1 -- 172.21.15.46:0/3777965331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad814f170 con 0x7faad815d940 2024-03-29T08:18:13.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.508+0000 7faaddc00640 1 -- 172.21.15.46:0/3777965331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad814bae0 con 0x7faad815d940 2024-03-29T08:18:13.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.512+0000 7faaddc00640 1 -- 172.21.15.46:0/3777965331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad814f3b0 con 0x7faad815d940 2024-03-29T08:18:13.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.512+0000 7faacf400640 1 -- 172.21.15.46:0/3777965331 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7faad0007070 con 0x7faad815d940 2024-03-29T08:18:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.512+0000 7faacf400640 1 --2- 172.21.15.46:0/3777965331 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faab4066690 0x7faab4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.512+0000 7faacf400640 1 -- 172.21.15.46:0/3777965331 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faad0084080 con 0x7faad815d940 2024-03-29T08:18:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.512+0000 7faad6a00640 1 --2- 172.21.15.46:0/3777965331 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faab4066690 0x7faab4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.512+0000 7faad6a00640 1 --2- 172.21.15.46:0/3777965331 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faab4066690 0x7faab4068b50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faac80011f0 tx=0x7faac8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:13.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.516+0000 7faacf400640 1 -- 172.21.15.46:0/3777965331 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7faad00b5980 con 0x7faad815d940 2024-03-29T08:18:13.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.680+0000 7faaddc00640 1 -- 172.21.15.46:0/3777965331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faad8066c90 con 0x7faad815d940 2024-03-29T08:18:13.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.684+0000 7faacf400640 1 -- 172.21.15.46:0/3777965331 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16006 (secure 0 0 0) 0x7faad0017090 con 0x7faad815d940 2024-03-29T08:18:13.684 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:13.684 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:13.311115+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:13.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.684+0000 7faaddc00640 1 -- 172.21.15.46:0/3777965331 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faab4066690 msgr2=0x7faab4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.684+0000 7faaddc00640 1 --2- 172.21.15.46:0/3777965331 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faab4066690 0x7faab4068b50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faac80011f0 tx=0x7faac8009040 comp rx=0 tx=0).stop 2024-03-29T08:18:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.684+0000 7faaddc00640 1 -- 172.21.15.46:0/3777965331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faad815d940 msgr2=0x7faad814ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.684+0000 7faaddc00640 1 --2- 172.21.15.46:0/3777965331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faad815d940 0x7faad814ab80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faad000caa0 tx=0x7faad000cf60 comp rx=0 tx=0).stop 2024-03-29T08:18:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.684+0000 7faaddc00640 1 -- 172.21.15.46:0/3777965331 shutdown_connections 2024-03-29T08:18:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.684+0000 7faaddc00640 1 --2- 172.21.15.46:0/3777965331 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faab4066690 0x7faab4068b50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.684+0000 7faaddc00640 1 --2- 172.21.15.46:0/3777965331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faad815d940 0x7faad814ab80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.684+0000 7faaddc00640 1 --2- 172.21.15.46:0/3777965331 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faad815cd90 0x7faad8151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.684+0000 7faaddc00640 1 --2- 172.21.15.46:0/3777965331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faad81593d0 0x7faad8151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.684+0000 7faaddc00640 1 -- 172.21.15.46:0/3777965331 >> 172.21.15.46:0/3777965331 conn(0x7faad8066110 msgr2=0x7faad80af830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.684+0000 7faaddc00640 1 -- 172.21.15.46:0/3777965331 shutdown_connections 2024-03-29T08:18:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.684+0000 7faaddc00640 1 -- 172.21.15.46:0/3777965331 wait complete. 2024-03-29T08:18:13.706 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:13.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.844+0000 7fed05800640 1 Processor -- start 2024-03-29T08:18:13.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.844+0000 7fed05800640 1 -- start start 2024-03-29T08:18:13.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.844+0000 7fed05800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fed0015a940 0x7fed000afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.844+0000 7fed05800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fed000cf3c0 0x7fed000cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.844+0000 7fed05800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed000b00d0 con 0x7fed0015ad20 2024-03-29T08:18:13.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.844+0000 7fed05800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed000b0ab0 con 0x7fed000cf3c0 2024-03-29T08:18:13.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.844+0000 7fed05800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed000b0d90 con 0x7fed0015a940 2024-03-29T08:18:13.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.844+0000 7fecf6000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fed0015ad20 0x7fed000cecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41406/0 (socket says 172.21.15.46:41406) 2024-03-29T08:18:13.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.844+0000 7fecf6000640 1 -- 172.21.15.46:0/2534150263 learned_addr learned my addr 172.21.15.46:0/2534150263 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:13.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.844+0000 7fecff400640 1 --2- 172.21.15.46:0/2534150263 >> v2:172.21.15.73:3300/0 conn(0x7fed000cf3c0 0x7fed000cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.844+0000 7fecfea00640 1 --2- 172.21.15.46:0/2534150263 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fed0015a940 0x7fed000afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.844+0000 7fecfe000640 1 -- 172.21.15.46:0/2534150263 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814723035 0 0) 0x7fed000b00d0 con 0x7fed0015ad20 2024-03-29T08:18:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.844+0000 7fecfe000640 1 -- 172.21.15.46:0/2534150263 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fece4003530 con 0x7fed0015ad20 2024-03-29T08:18:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.844+0000 7fecff400640 1 -- 172.21.15.46:0/2534150263 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fed0015a940 msgr2=0x7fed000afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.844+0000 7fecff400640 1 --2- 172.21.15.46:0/2534150263 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fed0015a940 0x7fed000afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.844+0000 7fecff400640 1 -- 172.21.15.46:0/2534150263 >> v1:172.21.15.46:6789/0 conn(0x7fed0015ad20 legacy=0x7fed000cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:13.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.844+0000 7fecff400640 1 -- 172.21.15.46:0/2534150263 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed000b1070 con 0x7fed000cf3c0 2024-03-29T08:18:13.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.848+0000 7fecff400640 1 --2- 172.21.15.46:0/2534150263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed000cf3c0 0x7fed000cb630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009d90 tx=0x7fecf801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc96e9a3fd31e82a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.848+0000 7fecfe000640 1 -- 172.21.15.46:0/2534150263 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf80a5970 con 0x7fed000cf3c0 2024-03-29T08:18:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.848+0000 7fecfe000640 1 -- 172.21.15.46:0/2534150263 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf80a5ad0 con 0x7fed000cf3c0 2024-03-29T08:18:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.848+0000 7fecfe000640 1 -- 172.21.15.46:0/2534150263 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf80a5e40 con 0x7fed000cf3c0 2024-03-29T08:18:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.848+0000 7fed05800640 1 -- 172.21.15.46:0/2534150263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed000cf3c0 msgr2=0x7fed000cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.848+0000 7fed05800640 1 --2- 172.21.15.46:0/2534150263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed000cf3c0 0x7fed000cb630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fecf8009d90 tx=0x7fecf801c8b0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.848+0000 7fed05800640 1 -- 172.21.15.46:0/2534150263 shutdown_connections 2024-03-29T08:18:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.848+0000 7fed05800640 1 --2- 172.21.15.46:0/2534150263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed000cf3c0 0x7fed000cb630 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.848+0000 7fed05800640 1 --2- 172.21.15.46:0/2534150263 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fed0015a940 0x7fed000afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.848+0000 7fed05800640 1 -- 172.21.15.46:0/2534150263 >> 172.21.15.46:0/2534150263 conn(0x7fed00089fd0 msgr2=0x7fed0008a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:13.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.848+0000 7fed05800640 1 -- 172.21.15.46:0/2534150263 shutdown_connections 2024-03-29T08:18:13.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.848+0000 7fed05800640 1 -- 172.21.15.46:0/2534150263 wait complete. 2024-03-29T08:18:13.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fed05800640 1 Processor -- start 2024-03-29T08:18:13.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fed05800640 1 -- start start 2024-03-29T08:18:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fed05800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed000cf3c0 0x7fed00153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fed05800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fed0015a940 0x7fed000ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fed05800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0015ad20 0x7fed00156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fed05800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed00155da0 con 0x7fed0015ad20 2024-03-29T08:18:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fed05800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed00156080 con 0x7fed000cf3c0 2024-03-29T08:18:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fed05800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed00156360 con 0x7fed0015a940 2024-03-29T08:18:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fecff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0015ad20 0x7fed00156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fecff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0015ad20 0x7fed00156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37628/0 (socket says 172.21.15.46:37628) 2024-03-29T08:18:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fecff400640 1 -- 172.21.15.46:0/1498787305 learned_addr learned my addr 172.21.15.46:0/1498787305 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fecf6000640 1 --2- 172.21.15.46:0/1498787305 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fed0015a940 0x7fed000ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fecfea00640 1 --2- 172.21.15.46:0/1498787305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed000cf3c0 0x7fed00153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fecff400640 1 -- 172.21.15.46:0/1498787305 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fed0015a940 msgr2=0x7fed000ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fecff400640 1 --2- 172.21.15.46:0/1498787305 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fed0015a940 0x7fed000ae700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fecff400640 1 -- 172.21.15.46:0/1498787305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed000cf3c0 msgr2=0x7fed00153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:13.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fecff400640 1 --2- 172.21.15.46:0/1498787305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed000cf3c0 0x7fed00153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fecff400640 1 -- 172.21.15.46:0/1498787305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf8009a70 con 0x7fed0015ad20 2024-03-29T08:18:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fecff400640 1 --2- 172.21.15.46:0/1498787305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0015ad20 0x7fed00156900 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fecf801a4e0 tx=0x7fecf8019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fecf7400640 1 -- 172.21.15.46:0/1498787305 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf80aed30 con 0x7fed0015ad20 2024-03-29T08:18:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fecf7400640 1 -- 172.21.15.46:0/1498787305 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf80aee90 con 0x7fed0015ad20 2024-03-29T08:18:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fecf7400640 1 -- 172.21.15.46:0/1498787305 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf80b7610 con 0x7fed0015ad20 2024-03-29T08:18:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.852+0000 7fed05800640 1 -- 172.21.15.46:0/1498787305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed00156640 con 0x7fed0015ad20 2024-03-29T08:18:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.856+0000 7fed05800640 1 -- 172.21.15.46:0/1498787305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed00154260 con 0x7fed0015ad20 2024-03-29T08:18:13.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.856+0000 7fed05800640 1 -- 172.21.15.46:0/1498787305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed00066c90 con 0x7fed0015ad20 2024-03-29T08:18:13.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.856+0000 7fecf7400640 1 -- 172.21.15.46:0/1498787305 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fecf80b4050 con 0x7fed0015ad20 2024-03-29T08:18:13.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.856+0000 7fecf7400640 1 --2- 172.21.15.46:0/1498787305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fecdc0666e0 0x7fecdc068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:13.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.856+0000 7fecf7400640 1 -- 172.21.15.46:0/1498787305 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fecf811f2b0 con 0x7fed0015ad20 2024-03-29T08:18:13.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.856+0000 7fecfea00640 1 --2- 172.21.15.46:0/1498787305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fecdc0666e0 0x7fecdc068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:13.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.856+0000 7fecfea00640 1 --2- 172.21.15.46:0/1498787305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fecdc0666e0 0x7fecdc068ba0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fecf0009af0 tx=0x7fecf0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:13.860+0000 7fecf7400640 1 -- 172.21.15.46:0/1498787305 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fecf80ec8b0 con 0x7fed0015ad20 2024-03-29T08:18:14.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.024+0000 7fed05800640 1 -- 172.21.15.46:0/1498787305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed0015bca0 con 0x7fed0015ad20 2024-03-29T08:18:14.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.024+0000 7fecf7400640 1 -- 172.21.15.46:0/1498787305 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16006 (secure 0 0 0) 0x7fecf8009450 con 0x7fed0015ad20 2024-03-29T08:18:14.025 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:14.026 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:13.311115+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.024+0000 7fed05800640 1 -- 172.21.15.46:0/1498787305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fecdc0666e0 msgr2=0x7fecdc068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.024+0000 7fed05800640 1 --2- 172.21.15.46:0/1498787305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fecdc0666e0 0x7fecdc068ba0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fecf0009af0 tx=0x7fecf0008040 comp rx=0 tx=0).stop 2024-03-29T08:18:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.024+0000 7fed05800640 1 -- 172.21.15.46:0/1498787305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0015ad20 msgr2=0x7fed00156900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.024+0000 7fed05800640 1 --2- 172.21.15.46:0/1498787305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0015ad20 0x7fed00156900 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fecf801a4e0 tx=0x7fecf8019650 comp rx=0 tx=0).stop 2024-03-29T08:18:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.024+0000 7fed05800640 1 -- 172.21.15.46:0/1498787305 shutdown_connections 2024-03-29T08:18:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.024+0000 7fed05800640 1 --2- 172.21.15.46:0/1498787305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fecdc0666e0 0x7fecdc068ba0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.024+0000 7fed05800640 1 --2- 172.21.15.46:0/1498787305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed0015ad20 0x7fed00156900 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.024+0000 7fed05800640 1 --2- 172.21.15.46:0/1498787305 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fed0015a940 0x7fed000ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.024+0000 7fed05800640 1 --2- 172.21.15.46:0/1498787305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed000cf3c0 0x7fed00153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.024+0000 7fed05800640 1 -- 172.21.15.46:0/1498787305 >> 172.21.15.46:0/1498787305 conn(0x7fed00089fd0 msgr2=0x7fed0006aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.024+0000 7fed05800640 1 -- 172.21.15.46:0/1498787305 shutdown_connections 2024-03-29T08:18:14.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.024+0000 7fed05800640 1 -- 172.21.15.46:0/1498787305 wait complete. 2024-03-29T08:18:14.046 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T08:18:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.188+0000 7f82eb600640 1 Processor -- start 2024-03-29T08:18:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.188+0000 7f82eb600640 1 -- start start 2024-03-29T08:18:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.188+0000 7f82eb600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f82e4099150 0x7f82e4167970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.188+0000 7f82eb600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e40cfe50 con 0x7f82e415c9c0 2024-03-29T08:18:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.188+0000 7f82eb600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e40cef80 con 0x7f82e40ceaf0 2024-03-29T08:18:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.188+0000 7f82eb600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e40cf240 con 0x7f82e4099150 2024-03-29T08:18:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.192+0000 7f82e8c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f82e415c9c0 0x7f82e406be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41420/0 (socket says 172.21.15.46:41420) 2024-03-29T08:18:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.192+0000 7f82e8c00640 1 -- 172.21.15.46:0/781772956 learned_addr learned my addr 172.21.15.46:0/781772956 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.192+0000 7f82e9600640 1 --2- 172.21.15.46:0/781772956 >> v2:172.21.15.73:3300/0 conn(0x7f82e4099150 0x7f82e4167970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.192+0000 7f82e3400640 1 -- 172.21.15.46:0/781772956 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394631982 0 0) 0x7f82e40cfe50 con 0x7f82e415c9c0 2024-03-29T08:18:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.192+0000 7f82e3400640 1 -- 172.21.15.46:0/781772956 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82cc003530 con 0x7f82e415c9c0 2024-03-29T08:18:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.192+0000 7f82e3400640 1 -- 172.21.15.46:0/781772956 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3435847133 0 0) 0x7f82e40cef80 con 0x7f82e40ceaf0 2024-03-29T08:18:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.192+0000 7f82e3400640 1 -- 172.21.15.46:0/781772956 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82e40cfe50 con 0x7f82e40ceaf0 2024-03-29T08:18:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.192+0000 7f82e9600640 1 -- 172.21.15.46:0/781772956 >> v1:172.21.15.73:6789/0 conn(0x7f82e40ceaf0 legacy=0x7f82e4098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.192+0000 7f82e9600640 1 -- 172.21.15.46:0/781772956 >> v1:172.21.15.46:6789/0 conn(0x7f82e415c9c0 legacy=0x7f82e406be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:14.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.192+0000 7f82e9600640 1 -- 172.21.15.46:0/781772956 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e40cf520 con 0x7f82e4099150 2024-03-29T08:18:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.192+0000 7f82e9600640 1 --2- 172.21.15.46:0/781772956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82e4099150 0x7f82e4167970 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f82dc009d90 tx=0x7f82dc01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=386b700cc965c7ee server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82e3400640 1 -- 172.21.15.46:0/781772956 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0a5920 con 0x7f82e4099150 2024-03-29T08:18:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82e3400640 1 -- 172.21.15.46:0/781772956 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc0a5a80 con 0x7f82e4099150 2024-03-29T08:18:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82e3400640 1 -- 172.21.15.46:0/781772956 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc0a5e10 con 0x7f82e4099150 2024-03-29T08:18:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82eb600640 1 -- 172.21.15.46:0/781772956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82e4099150 msgr2=0x7f82e4167970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82eb600640 1 --2- 172.21.15.46:0/781772956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82e4099150 0x7f82e4167970 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f82dc009d90 tx=0x7f82dc01c970 comp rx=0 tx=0).stop 2024-03-29T08:18:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82eb600640 1 -- 172.21.15.46:0/781772956 shutdown_connections 2024-03-29T08:18:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82eb600640 1 --2- 172.21.15.46:0/781772956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82e4099150 0x7f82e4167970 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82eb600640 1 -- 172.21.15.46:0/781772956 >> 172.21.15.46:0/781772956 conn(0x7f82e40b2220 msgr2=0x7f82e40b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82eb600640 1 -- 172.21.15.46:0/781772956 shutdown_connections 2024-03-29T08:18:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82eb600640 1 -- 172.21.15.46:0/781772956 wait complete. 2024-03-29T08:18:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82eb600640 1 Processor -- start 2024-03-29T08:18:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82eb600640 1 -- start start 2024-03-29T08:18:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82eb600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82e4099150 0x7f82e406f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82eb600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82e40ceaf0 0x7f82e4153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82eb600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f82e415c9c0 0x7f82e4152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82eb600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40c4600 con 0x7f82e40ceaf0 2024-03-29T08:18:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82eb600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40c48e0 con 0x7f82e4099150 2024-03-29T08:18:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82eb600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e40c4bc0 con 0x7f82e415c9c0 2024-03-29T08:18:14.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82e40ceaf0 0x7f82e4153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82e40ceaf0 0x7f82e4153eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37630/0 (socket says 172.21.15.46:37630) 2024-03-29T08:18:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82e3e00640 1 -- 172.21.15.46:0/75006394 learned_addr learned my addr 172.21.15.46:0/75006394 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82e9600640 1 --2- 172.21.15.46:0/75006394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f82e415c9c0 0x7f82e4152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:14.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82e8c00640 1 --2- 172.21.15.46:0/75006394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82e4099150 0x7f82e406f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82e3e00640 1 -- 172.21.15.46:0/75006394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f82e415c9c0 msgr2=0x7f82e4152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82e3e00640 1 --2- 172.21.15.46:0/75006394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f82e415c9c0 0x7f82e4152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82e3e00640 1 -- 172.21.15.46:0/75006394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82e4099150 msgr2=0x7f82e406f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.196+0000 7f82e3e00640 1 --2- 172.21.15.46:0/75006394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82e4099150 0x7f82e406f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.200+0000 7f82e3e00640 1 -- 172.21.15.46:0/75006394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc009a70 con 0x7f82e40ceaf0 2024-03-29T08:18:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.200+0000 7f82e3e00640 1 --2- 172.21.15.46:0/75006394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82e40ceaf0 0x7f82e4153eb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f82d000c490 tx=0x7f82d000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.200+0000 7f82e0c00640 1 -- 172.21.15.46:0/75006394 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d00052d0 con 0x7f82e40ceaf0 2024-03-29T08:18:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.200+0000 7f82e0c00640 1 -- 172.21.15.46:0/75006394 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d0005430 con 0x7f82e40ceaf0 2024-03-29T08:18:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.200+0000 7f82e0c00640 1 -- 172.21.15.46:0/75006394 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d0012870 con 0x7f82e40ceaf0 2024-03-29T08:18:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.200+0000 7f82eb600640 1 -- 172.21.15.46:0/75006394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e40c4ea0 con 0x7f82e40ceaf0 2024-03-29T08:18:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.200+0000 7f82eb600640 1 -- 172.21.15.46:0/75006394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e40c4260 con 0x7f82e40ceaf0 2024-03-29T08:18:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.200+0000 7f82eb600640 1 -- 172.21.15.46:0/75006394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e40b40b0 con 0x7f82e40ceaf0 2024-03-29T08:18:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.200+0000 7f82e0c00640 1 -- 172.21.15.46:0/75006394 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f82d00129d0 con 0x7f82e40ceaf0 2024-03-29T08:18:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.200+0000 7f82e0c00640 1 --2- 172.21.15.46:0/75006394 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f82c4066690 0x7f82c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.200+0000 7f82e0c00640 1 -- 172.21.15.46:0/75006394 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82d0083c90 con 0x7f82e40ceaf0 2024-03-29T08:18:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.200+0000 7f82e8c00640 1 --2- 172.21.15.46:0/75006394 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f82c4066690 0x7f82c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:14.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.204+0000 7f82e8c00640 1 --2- 172.21.15.46:0/75006394 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f82c4066690 0x7f82c4068b50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f82d80011f0 tx=0x7f82d8007d40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:14.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.204+0000 7f82e0c00640 1 -- 172.21.15.46:0/75006394 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f82d0051010 con 0x7f82e40ceaf0 2024-03-29T08:18:14.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:14.404+0000 7f82eb600640 1 -- 172.21.15.46:0/75006394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f82e409e700 con 0x7f82e40ceaf0 2024-03-29T08:18:15.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.320+0000 7f82e0c00640 1 -- 172.21.15.46:0/75006394 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f82d0054630 con 0x7f82e40ceaf0 2024-03-29T08:18:15.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.324+0000 7f82eb600640 1 -- 172.21.15.46:0/75006394 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f82c4066690 msgr2=0x7f82c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:15.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.324+0000 7f82eb600640 1 --2- 172.21.15.46:0/75006394 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f82c4066690 0x7f82c4068b50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f82d80011f0 tx=0x7f82d8007d40 comp rx=0 tx=0).stop 2024-03-29T08:18:15.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.324+0000 7f82eb600640 1 -- 172.21.15.46:0/75006394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82e40ceaf0 msgr2=0x7f82e4153eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:15.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.324+0000 7f82eb600640 1 --2- 172.21.15.46:0/75006394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82e40ceaf0 0x7f82e4153eb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f82d000c490 tx=0x7f82d000c950 comp rx=0 tx=0).stop 2024-03-29T08:18:15.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.324+0000 7f82eb600640 1 -- 172.21.15.46:0/75006394 shutdown_connections 2024-03-29T08:18:15.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.324+0000 7f82eb600640 1 --2- 172.21.15.46:0/75006394 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f82c4066690 0x7f82c4068b50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:15.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.324+0000 7f82eb600640 1 --2- 172.21.15.46:0/75006394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f82e415c9c0 0x7f82e4152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:15.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.324+0000 7f82eb600640 1 --2- 172.21.15.46:0/75006394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82e40ceaf0 0x7f82e4153eb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:15.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.324+0000 7f82eb600640 1 --2- 172.21.15.46:0/75006394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82e4099150 0x7f82e406f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:15.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.324+0000 7f82eb600640 1 -- 172.21.15.46:0/75006394 >> 172.21.15.46:0/75006394 conn(0x7f82e40b2220 msgr2=0x7f82e415d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:15.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.324+0000 7f82eb600640 1 -- 172.21.15.46:0/75006394 shutdown_connections 2024-03-29T08:18:15.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.324+0000 7f82eb600640 1 -- 172.21.15.46:0/75006394 wait complete. 2024-03-29T08:18:15.347 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-29T08:18:15.347 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T08:18:15.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf43c00640 1 Processor -- start 2024-03-29T08:18:15.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf43c00640 1 -- start start 2024-03-29T08:18:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf43c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf3c15abd0 0x7faf3c0cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf43c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faf3c098a40 0x7faf3c098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf43c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3c0cbd00 con 0x7faf3c15a7f0 2024-03-29T08:18:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf43c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3c0b4500 con 0x7faf3c098a40 2024-03-29T08:18:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf43c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0cf970 con 0x7faf3c15abd0 2024-03-29T08:18:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf41200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7faf3c15a7f0 0x7faf3c0cb490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41436/0 (socket says 172.21.15.46:41436) 2024-03-29T08:18:15.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf41200640 1 -- 172.21.15.46:0/1348755983 learned_addr learned my addr 172.21.15.46:0/1348755983 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3be00640 1 --2- 172.21.15.46:0/1348755983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf3c15abd0 0x7faf3c0cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:15.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3b400640 1 -- 172.21.15.46:0/1348755983 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442893771 0 0) 0x7faf3c0cbd00 con 0x7faf3c15a7f0 2024-03-29T08:18:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf41c00640 1 --2- 172.21.15.46:0/1348755983 >> v2:172.21.15.73:3300/0 conn(0x7faf3c098a40 0x7faf3c098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3b400640 1 -- 172.21.15.46:0/1348755983 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf24003530 con 0x7faf3c15a7f0 2024-03-29T08:18:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3b400640 1 -- 172.21.15.46:0/1348755983 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1186003567 0 0) 0x7faf24003530 con 0x7faf3c15a7f0 2024-03-29T08:18:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3b400640 1 -- 172.21.15.46:0/1348755983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf3c15abd0 msgr2=0x7faf3c0cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3b400640 1 --2- 172.21.15.46:0/1348755983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf3c15abd0 0x7faf3c0cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3b400640 1 -- 172.21.15.46:0/1348755983 >> v2:172.21.15.73:3300/0 conn(0x7faf3c098a40 msgr2=0x7faf3c098ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3b400640 1 --2- 172.21.15.46:0/1348755983 >> v2:172.21.15.73:3300/0 conn(0x7faf3c098a40 0x7faf3c098ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3b400640 1 -- 172.21.15.46:0/1348755983 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0cbfe0 con 0x7faf3c15a7f0 2024-03-29T08:18:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3b400640 1 -- 172.21.15.46:0/1348755983 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7faf30001680 con 0x7faf3c15a7f0 2024-03-29T08:18:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3b400640 1 -- 172.21.15.46:0/1348755983 >> v1:172.21.15.46:6789/0 conn(0x7faf3c15a7f0 legacy=0x7faf3c0cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3b400640 1 --2- 172.21.15.46:0/1348755983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf24003980 0x7faf24003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3b400640 1 --2- 172.21.15.46:0/1348755983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf240047d0 0x7faf24004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:15.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3b400640 1 --2- 172.21.15.46:0/1348755983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf3c15abd0 0x7faf24005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3b400640 1 -- 172.21.15.46:0/1348755983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0b4500 con 0x7faf240047d0 2024-03-29T08:18:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3b400640 1 -- 172.21.15.46:0/1348755983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0cf970 con 0x7faf24003980 2024-03-29T08:18:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3b400640 1 -- 172.21.15.46:0/1348755983 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2400b990 con 0x7faf3c15abd0 2024-03-29T08:18:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3be00640 1 --2- 172.21.15.46:0/1348755983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf240047d0 0x7faf24004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf41200640 1 --2- 172.21.15.46:0/1348755983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf24003980 0x7faf24003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf41c00640 1 --2- 172.21.15.46:0/1348755983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf3c15abd0 0x7faf24005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3be00640 1 -- 172.21.15.46:0/1348755983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf3c15abd0 msgr2=0x7faf24005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3be00640 1 --2- 172.21.15.46:0/1348755983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf3c15abd0 0x7faf24005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3be00640 1 -- 172.21.15.46:0/1348755983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf24003980 msgr2=0x7faf24003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3be00640 1 --2- 172.21.15.46:0/1348755983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf24003980 0x7faf24003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.496+0000 7faf3be00640 1 -- 172.21.15.46:0/1348755983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf24003530 con 0x7faf240047d0 2024-03-29T08:18:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.500+0000 7faf3be00640 1 --2- 172.21.15.46:0/1348755983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf240047d0 0x7faf24004c50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faf28009ae0 tx=0x7faf2801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5c3da85039cb39 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.500+0000 7faf3b400640 1 -- 172.21.15.46:0/1348755983 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf280a5580 con 0x7faf240047d0 2024-03-29T08:18:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.500+0000 7faf3b400640 1 -- 172.21.15.46:0/1348755983 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf280a56e0 con 0x7faf240047d0 2024-03-29T08:18:15.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.500+0000 7faf3b400640 1 -- 172.21.15.46:0/1348755983 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf280a5a90 con 0x7faf240047d0 2024-03-29T08:18:15.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.500+0000 7faf43c00640 1 -- 172.21.15.46:0/1348755983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf240047d0 msgr2=0x7faf24004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:15.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.500+0000 7faf43c00640 1 --2- 172.21.15.46:0/1348755983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf240047d0 0x7faf24004c50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faf28009ae0 tx=0x7faf2801ca70 comp rx=0 tx=0).stop 2024-03-29T08:18:15.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.500+0000 7faf41c00640 1 -- 172.21.15.46:0/1348755983 reap_dead start 2024-03-29T08:18:15.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.500+0000 7faf43c00640 1 -- 172.21.15.46:0/1348755983 shutdown_connections 2024-03-29T08:18:15.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.500+0000 7faf43c00640 1 -- 172.21.15.46:0/1348755983 >> 172.21.15.46:0/1348755983 conn(0x7faf3c0b2190 msgr2=0x7faf3c0b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:15.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.500+0000 7faf43c00640 1 -- 172.21.15.46:0/1348755983 shutdown_connections 2024-03-29T08:18:15.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.500+0000 7faf43c00640 1 -- 172.21.15.46:0/1348755983 wait complete. 2024-03-29T08:18:15.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf43c00640 1 Processor -- start 2024-03-29T08:18:15.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf43c00640 1 -- start start 2024-03-29T08:18:15.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf43c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf240047d0 0x7faf3c0c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:15.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf43c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf3c0b4a20 0x7faf3c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:15.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf43c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3c09d760 0x7faf3c09dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:15.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf43c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0b53e0 con 0x7faf3c09d760 2024-03-29T08:18:15.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf43c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0ca610 con 0x7faf3c0b4a20 2024-03-29T08:18:15.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf43c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3c0c9a00 con 0x7faf240047d0 2024-03-29T08:18:15.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf41c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3c09d760 0x7faf3c09dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:15.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf41c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3c09d760 0x7faf3c09dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37646/0 (socket says 172.21.15.46:37646) 2024-03-29T08:18:15.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf41c00640 1 -- 172.21.15.46:0/2881039238 learned_addr learned my addr 172.21.15.46:0/2881039238 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf41200640 1 --2- 172.21.15.46:0/2881039238 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf240047d0 0x7faf3c0c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf3be00640 1 --2- 172.21.15.46:0/2881039238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf3c0b4a20 0x7faf3c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf41c00640 1 -- 172.21.15.46:0/2881039238 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf240047d0 msgr2=0x7faf3c0c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf41c00640 1 --2- 172.21.15.46:0/2881039238 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf240047d0 0x7faf3c0c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf41c00640 1 -- 172.21.15.46:0/2881039238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf3c0b4a20 msgr2=0x7faf3c0b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf41c00640 1 --2- 172.21.15.46:0/2881039238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf3c0b4a20 0x7faf3c0b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf41c00640 1 -- 172.21.15.46:0/2881039238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0c9ce0 con 0x7faf3c09d760 2024-03-29T08:18:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf3be00640 1 --2- 172.21.15.46:0/2881039238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf3c0b4a20 0x7faf3c0b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf41c00640 1 --2- 172.21.15.46:0/2881039238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3c09d760 0x7faf3c09dbe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0b2fd0 tx=0x7faf3400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf38c00640 1 -- 172.21.15.46:0/2881039238 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3400e040 con 0x7faf3c09d760 2024-03-29T08:18:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf38c00640 1 -- 172.21.15.46:0/2881039238 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3400f040 con 0x7faf3c09d760 2024-03-29T08:18:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf38c00640 1 -- 172.21.15.46:0/2881039238 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf34015670 con 0x7faf3c09d760 2024-03-29T08:18:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf43c00640 1 -- 172.21.15.46:0/2881039238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf3c0c9fc0 con 0x7faf3c09d760 2024-03-29T08:18:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.504+0000 7faf43c00640 1 -- 172.21.15.46:0/2881039238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf3c09e120 con 0x7faf3c09d760 2024-03-29T08:18:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.508+0000 7faf43c00640 1 -- 172.21.15.46:0/2881039238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3c09e690 con 0x7faf3c09d760 2024-03-29T08:18:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.508+0000 7faf38c00640 1 -- 172.21.15.46:0/2881039238 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7faf340157d0 con 0x7faf3c09d760 2024-03-29T08:18:15.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.508+0000 7faf38c00640 1 --2- 172.21.15.46:0/2881039238 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf1c066690 0x7faf1c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:15.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.508+0000 7faf38c00640 1 -- 172.21.15.46:0/2881039238 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf34084a80 con 0x7faf3c09d760 2024-03-29T08:18:15.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.508+0000 7faf41200640 1 --2- 172.21.15.46:0/2881039238 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf1c066690 0x7faf1c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.508+0000 7faf41200640 1 --2- 172.21.15.46:0/2881039238 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf1c066690 0x7faf1c068b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf30001060 tx=0x7faf30009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.512+0000 7faf38c00640 1 -- 172.21.15.46:0/2881039238 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7faf34051d50 con 0x7faf3c09d760 2024-03-29T08:18:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:15.708+0000 7faf43c00640 1 -- 172.21.15.46:0/2881039238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7faf3c0ca750 con 0x7faf3c09d760 2024-03-29T08:18:16.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.320+0000 7faf38c00640 1 -- 172.21.15.46:0/2881039238 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7faf34055370 con 0x7faf3c09d760 2024-03-29T08:18:16.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.324+0000 7faf43c00640 1 -- 172.21.15.46:0/2881039238 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf1c066690 msgr2=0x7faf1c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:16.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.324+0000 7faf43c00640 1 --2- 172.21.15.46:0/2881039238 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf1c066690 0x7faf1c068b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf30001060 tx=0x7faf30009040 comp rx=0 tx=0).stop 2024-03-29T08:18:16.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.324+0000 7faf43c00640 1 -- 172.21.15.46:0/2881039238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3c09d760 msgr2=0x7faf3c09dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:16.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.324+0000 7faf43c00640 1 --2- 172.21.15.46:0/2881039238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3c09d760 0x7faf3c09dbe0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0b2fd0 tx=0x7faf3400f580 comp rx=0 tx=0).stop 2024-03-29T08:18:16.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.324+0000 7faf43c00640 1 -- 172.21.15.46:0/2881039238 shutdown_connections 2024-03-29T08:18:16.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.324+0000 7faf43c00640 1 --2- 172.21.15.46:0/2881039238 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf1c066690 0x7faf1c068b50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.324+0000 7faf43c00640 1 --2- 172.21.15.46:0/2881039238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf3c0b4a20 0x7faf3c0b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.324+0000 7faf43c00640 1 --2- 172.21.15.46:0/2881039238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf3c09d760 0x7faf3c09dbe0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.324+0000 7faf43c00640 1 --2- 172.21.15.46:0/2881039238 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf240047d0 0x7faf3c0c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.324+0000 7faf43c00640 1 -- 172.21.15.46:0/2881039238 >> 172.21.15.46:0/2881039238 conn(0x7faf3c0b2190 msgr2=0x7faf3c06edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:16.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.324+0000 7faf43c00640 1 -- 172.21.15.46:0/2881039238 shutdown_connections 2024-03-29T08:18:16.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.324+0000 7faf43c00640 1 -- 172.21.15.46:0/2881039238 wait complete. 2024-03-29T08:18:16.349 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T08:18:16.349 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:18:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.492+0000 7f8364a00640 1 Processor -- start 2024-03-29T08:18:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.492+0000 7f8364a00640 1 -- start start 2024-03-29T08:18:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.492+0000 7f8364a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f83601593d0 0x7f836006cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.492+0000 7f8364a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f836015d230 0x7f836015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.492+0000 7f8364a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f836015dbd0 0x7f836015bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.492+0000 7f8364a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f836006d130 con 0x7f836015d230 2024-03-29T08:18:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.492+0000 7f8364a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f836015b8f0 con 0x7f83601593d0 2024-03-29T08:18:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.492+0000 7f8364a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836015ad50 con 0x7f836015dbd0 2024-03-29T08:18:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.492+0000 7f8355400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f836015d230 0x7f836015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.492+0000 7f8355400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f836015d230 0x7f836015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59680/0 (socket says 172.21.15.46:59680) 2024-03-29T08:18:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.492+0000 7f8355400640 1 -- 172.21.15.46:0/2586428082 learned_addr learned my addr 172.21.15.46:0/2586428082 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.496+0000 7f835de00640 1 --2- 172.21.15.46:0/2586428082 >> v2:172.21.15.73:3300/0 conn(0x7f83601593d0 0x7f836006cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.496+0000 7f835e800640 1 --2- 172.21.15.46:0/2586428082 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f836015dbd0 0x7f836015bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.496+0000 7f8355400640 1 -- 172.21.15.46:0/2586428082 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f836015dbd0 msgr2=0x7f836015bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:16.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.496+0000 7f8355400640 1 --2- 172.21.15.46:0/2586428082 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f836015dbd0 0x7f836015bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.496+0000 7f8355400640 1 -- 172.21.15.46:0/2586428082 >> v2:172.21.15.73:3300/0 conn(0x7f83601593d0 msgr2=0x7f836006cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:16.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.496+0000 7f8355400640 1 --2- 172.21.15.46:0/2586428082 >> v2:172.21.15.73:3300/0 conn(0x7f83601593d0 0x7f836006cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.496+0000 7f8355400640 1 -- 172.21.15.46:0/2586428082 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836015e030 con 0x7f836015d230 2024-03-29T08:18:16.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.496+0000 7f8355400640 1 --2- 172.21.15.46:0/2586428082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836015d230 0x7f836015d690 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f836006e460 tx=0x7f834801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90168228ab1eb88c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:16.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.496+0000 7f835d400640 1 -- 172.21.15.46:0/2586428082 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83480a5870 con 0x7f836015d230 2024-03-29T08:18:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.496+0000 7f835d400640 1 -- 172.21.15.46:0/2586428082 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83480a59d0 con 0x7f836015d230 2024-03-29T08:18:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.496+0000 7f835d400640 1 -- 172.21.15.46:0/2586428082 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83480a5d60 con 0x7f836015d230 2024-03-29T08:18:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.496+0000 7f8364a00640 1 -- 172.21.15.46:0/2586428082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836015d230 msgr2=0x7f836015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.496+0000 7f8364a00640 1 --2- 172.21.15.46:0/2586428082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836015d230 0x7f836015d690 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f836006e460 tx=0x7f834801c9f0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8364a00640 1 -- 172.21.15.46:0/2586428082 shutdown_connections 2024-03-29T08:18:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8364a00640 1 --2- 172.21.15.46:0/2586428082 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f836015dbd0 0x7f836015bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8364a00640 1 --2- 172.21.15.46:0/2586428082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836015d230 0x7f836015d690 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8364a00640 1 --2- 172.21.15.46:0/2586428082 >> v2:172.21.15.73:3300/0 conn(0x7f83601593d0 0x7f836006cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8364a00640 1 -- 172.21.15.46:0/2586428082 >> 172.21.15.46:0/2586428082 conn(0x7f8360066110 msgr2=0x7f8360069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8364a00640 1 -- 172.21.15.46:0/2586428082 shutdown_connections 2024-03-29T08:18:16.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8364a00640 1 -- 172.21.15.46:0/2586428082 wait complete. 2024-03-29T08:18:16.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8364a00640 1 Processor -- start 2024-03-29T08:18:16.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8364a00640 1 -- start start 2024-03-29T08:18:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8364a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f83601593d0 0x7f83601f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8364a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836015d230 0x7f8360156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8364a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f836015dbd0 0x7f83601f5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8364a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83601f5bf0 con 0x7f836015d230 2024-03-29T08:18:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8364a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8360154000 con 0x7f836015dbd0 2024-03-29T08:18:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8364a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8360155cf0 con 0x7f83601593d0 2024-03-29T08:18:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8355400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836015d230 0x7f8360156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8355400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836015d230 0x7f8360156d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59686/0 (socket says 172.21.15.46:59686) 2024-03-29T08:18:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8355400640 1 -- 172.21.15.46:0/3699429660 learned_addr learned my addr 172.21.15.46:0/3699429660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f835e800640 1 --2- 172.21.15.46:0/3699429660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f836015dbd0 0x7f83601f5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f835de00640 1 --2- 172.21.15.46:0/3699429660 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f83601593d0 0x7f83601f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8355400640 1 -- 172.21.15.46:0/3699429660 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f83601593d0 msgr2=0x7f83601f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:16.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8355400640 1 --2- 172.21.15.46:0/3699429660 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f83601593d0 0x7f83601f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8355400640 1 -- 172.21.15.46:0/3699429660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f836015dbd0 msgr2=0x7f83601f5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:16.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8355400640 1 --2- 172.21.15.46:0/3699429660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f836015dbd0 0x7f83601f5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8355400640 1 -- 172.21.15.46:0/3699429660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8348009b30 con 0x7f836015d230 2024-03-29T08:18:16.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f835e800640 1 --2- 172.21.15.46:0/3699429660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f836015dbd0 0x7f83601f5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:16.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8355400640 1 --2- 172.21.15.46:0/3699429660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836015d230 0x7f8360156d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f834801b0a0 tx=0x7f8348019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:16.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8356a00640 1 -- 172.21.15.46:0/3699429660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83480b8880 con 0x7f836015d230 2024-03-29T08:18:16.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.500+0000 7f8356a00640 1 -- 172.21.15.46:0/3699429660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83480b8a20 con 0x7f836015d230 2024-03-29T08:18:16.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.504+0000 7f8364a00640 1 -- 172.21.15.46:0/3699429660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8360155fd0 con 0x7f836015d230 2024-03-29T08:18:16.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.504+0000 7f8356a00640 1 -- 172.21.15.46:0/3699429660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83480b8dd0 con 0x7f836015d230 2024-03-29T08:18:16.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.504+0000 7f8364a00640 1 -- 172.21.15.46:0/3699429660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8360156590 con 0x7f836015d230 2024-03-29T08:18:16.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.504+0000 7f8364a00640 1 -- 172.21.15.46:0/3699429660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8360066c90 con 0x7f836015d230 2024-03-29T08:18:16.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.504+0000 7f8356a00640 1 -- 172.21.15.46:0/3699429660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f83480c3460 con 0x7f836015d230 2024-03-29T08:18:16.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.504+0000 7f8356a00640 1 --2- 172.21.15.46:0/3699429660 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f833c066690 0x7f833c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:16.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.504+0000 7f8356a00640 1 -- 172.21.15.46:0/3699429660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f834811f780 con 0x7f836015d230 2024-03-29T08:18:16.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.504+0000 7f835de00640 1 --2- 172.21.15.46:0/3699429660 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f833c066690 0x7f833c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:16.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.504+0000 7f835de00640 1 --2- 172.21.15.46:0/3699429660 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f833c066690 0x7f833c068b50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f835000b990 tx=0x7f8350008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:16.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.508+0000 7f8356a00640 1 -- 172.21.15.46:0/3699429660 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f83480ecb00 con 0x7f836015d230 2024-03-29T08:18:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.704+0000 7f8364a00640 1 -- 172.21.15.46:0/3699429660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8360154450 con 0x7f836015d230 2024-03-29T08:18:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.704+0000 7f8356a00640 1 -- 172.21.15.46:0/3699429660 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4722 (secure 0 0 0) 0x7f83480f0120 con 0x7f836015d230 2024-03-29T08:18:16.707 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:16.707 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/339038392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":339038392},{"type":"v1","addr":"172.21.15.46:6834","nonce":339038392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3573036715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3573036715},{"type":"v1","addr":"172.21.15.73:6835","nonce":3573036715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/746744819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":746744819},{"type":"v1","addr":"172.21.15.46:6835","nonce":746744819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:18:13.311457+0000","modified":"2024-03-29T08:18:16.321300+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2780269530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2780269530},{"type":"v1","addr":"172.21.15.73:6837","nonce":2780269530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-29T08:18:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-03-29T08:18:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.708+0000 7f8364a00640 1 -- 172.21.15.46:0/3699429660 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f833c066690 msgr2=0x7f833c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.708+0000 7f8364a00640 1 --2- 172.21.15.46:0/3699429660 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f833c066690 0x7f833c068b50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f835000b990 tx=0x7f8350008040 comp rx=0 tx=0).stop 2024-03-29T08:18:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.708+0000 7f8364a00640 1 -- 172.21.15.46:0/3699429660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836015d230 msgr2=0x7f8360156d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.708+0000 7f8364a00640 1 --2- 172.21.15.46:0/3699429660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836015d230 0x7f8360156d90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f834801b0a0 tx=0x7f8348019790 comp rx=0 tx=0).stop 2024-03-29T08:18:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.708+0000 7f8364a00640 1 -- 172.21.15.46:0/3699429660 shutdown_connections 2024-03-29T08:18:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.708+0000 7f8364a00640 1 --2- 172.21.15.46:0/3699429660 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f833c066690 0x7f833c068b50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.708+0000 7f8364a00640 1 --2- 172.21.15.46:0/3699429660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f836015dbd0 0x7f83601f5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.708+0000 7f8364a00640 1 --2- 172.21.15.46:0/3699429660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836015d230 0x7f8360156d90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.708+0000 7f8364a00640 1 --2- 172.21.15.46:0/3699429660 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f83601593d0 0x7f83601f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.708+0000 7f8364a00640 1 -- 172.21.15.46:0/3699429660 >> 172.21.15.46:0/3699429660 conn(0x7f8360066110 msgr2=0x7f83601597b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.708+0000 7f8364a00640 1 -- 172.21.15.46:0/3699429660 shutdown_connections 2024-03-29T08:18:16.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.708+0000 7f8364a00640 1 -- 172.21.15.46:0/3699429660 wait complete. 2024-03-29T08:18:16.728 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:16.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.864+0000 7f1910600640 1 Processor -- start 2024-03-29T08:18:16.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.864+0000 7f1910600640 1 -- start start 2024-03-29T08:18:16.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.868+0000 7f1910600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815cb60 0x7f190806bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:16.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.868+0000 7f1910600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f19080cae00 0x7f19080b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.868+0000 7f1910600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19080b13b0 con 0x7f19080d06f0 2024-03-29T08:18:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.868+0000 7f1910600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f190806c2f0 con 0x7f19080cae00 2024-03-29T08:18:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.868+0000 7f1910600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19080b1610 con 0x7f190815cb60 2024-03-29T08:18:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.868+0000 7f190e600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f19080d06f0 0x7f19080af390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60838/0 (socket says 172.21.15.46:60838) 2024-03-29T08:18:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.868+0000 7f190e600640 1 -- 172.21.15.46:0/2779034075 learned_addr learned my addr 172.21.15.46:0/2779034075 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.868+0000 7f190d200640 1 --2- 172.21.15.46:0/2779034075 >> v2:172.21.15.73:3300/0 conn(0x7f19080cae00 0x7f19080b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:16.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.868+0000 7f190dc00640 1 --2- 172.21.15.46:0/2779034075 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815cb60 0x7f190806bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.868+0000 7f1907e00640 1 -- 172.21.15.46:0/2779034075 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769532208 0 0) 0x7f19080b13b0 con 0x7f19080d06f0 2024-03-29T08:18:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.868+0000 7f1907e00640 1 -- 172.21.15.46:0/2779034075 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18f0003530 con 0x7f19080d06f0 2024-03-29T08:18:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.868+0000 7f190d200640 1 -- 172.21.15.46:0/2779034075 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815cb60 msgr2=0x7f190806bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.868+0000 7f190d200640 1 --2- 172.21.15.46:0/2779034075 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815cb60 0x7f190806bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.868+0000 7f190d200640 1 -- 172.21.15.46:0/2779034075 >> v1:172.21.15.46:6789/0 conn(0x7f19080d06f0 legacy=0x7f19080af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.868+0000 7f190d200640 1 -- 172.21.15.46:0/2779034075 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19080b18f0 con 0x7f19080cae00 2024-03-29T08:18:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.868+0000 7f190dc00640 1 --2- 172.21.15.46:0/2779034075 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815cb60 0x7f190806bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.872+0000 7f190d200640 1 --2- 172.21.15.46:0/2779034075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19080cae00 0x7f19080b0e70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f18f40097c0 tx=0x7f18f401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89ac4484f2de3635 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.872+0000 7f1907e00640 1 -- 172.21.15.46:0/2779034075 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f40a7070 con 0x7f19080cae00 2024-03-29T08:18:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.872+0000 7f1907e00640 1 -- 172.21.15.46:0/2779034075 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f40ab430 con 0x7f19080cae00 2024-03-29T08:18:16.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.872+0000 7f1907e00640 1 -- 172.21.15.46:0/2779034075 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f40ab7c0 con 0x7f19080cae00 2024-03-29T08:18:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.872+0000 7f1910600640 1 -- 172.21.15.46:0/2779034075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19080cae00 msgr2=0x7f19080b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.872+0000 7f1910600640 1 --2- 172.21.15.46:0/2779034075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19080cae00 0x7f19080b0e70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f18f40097c0 tx=0x7f18f401c470 comp rx=0 tx=0).stop 2024-03-29T08:18:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.872+0000 7f1910600640 1 -- 172.21.15.46:0/2779034075 shutdown_connections 2024-03-29T08:18:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.872+0000 7f1910600640 1 --2- 172.21.15.46:0/2779034075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19080cae00 0x7f19080b0e70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.872+0000 7f1910600640 1 --2- 172.21.15.46:0/2779034075 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815cb60 0x7f190806bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.872+0000 7f1910600640 1 -- 172.21.15.46:0/2779034075 >> 172.21.15.46:0/2779034075 conn(0x7f1908089fd0 msgr2=0x7f190808a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.872+0000 7f1910600640 1 -- 172.21.15.46:0/2779034075 shutdown_connections 2024-03-29T08:18:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.872+0000 7f1910600640 1 -- 172.21.15.46:0/2779034075 wait complete. 2024-03-29T08:18:16.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.872+0000 7f1910600640 1 Processor -- start 2024-03-29T08:18:16.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.872+0000 7f1910600640 1 -- start start 2024-03-29T08:18:16.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f1910600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f19080cae00 0x7f19080aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f1910600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19080d06f0 0x7f1908151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f1910600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f190815cb60 0x7f190814f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f1910600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1908151bc0 con 0x7f190815cb60 2024-03-29T08:18:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f1910600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190815e310 con 0x7f19080d06f0 2024-03-29T08:18:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f1910600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190815e5f0 con 0x7f19080cae00 2024-03-29T08:18:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f190e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f190815cb60 0x7f190814f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f190d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19080d06f0 0x7f1908151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f190e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f190815cb60 0x7f190814f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59690/0 (socket says 172.21.15.46:59690) 2024-03-29T08:18:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f190e600640 1 -- 172.21.15.46:0/4191015078 learned_addr learned my addr 172.21.15.46:0/4191015078 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f190dc00640 1 --2- 172.21.15.46:0/4191015078 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f19080cae00 0x7f19080aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f190d200640 1 -- 172.21.15.46:0/4191015078 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f19080cae00 msgr2=0x7f19080aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f190d200640 1 --2- 172.21.15.46:0/4191015078 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f19080cae00 0x7f19080aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f190d200640 1 -- 172.21.15.46:0/4191015078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f190815cb60 msgr2=0x7f190814f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f190d200640 1 --2- 172.21.15.46:0/4191015078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f190815cb60 0x7f190814f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f190d200640 1 -- 172.21.15.46:0/4191015078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc009420 con 0x7f19080d06f0 2024-03-29T08:18:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f190e600640 1 --2- 172.21.15.46:0/4191015078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f190815cb60 0x7f190814f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:16.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f190d200640 1 --2- 172.21.15.46:0/4191015078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19080d06f0 0x7f1908151680 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f18f40ac000 tx=0x7f18f4019210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f1905600640 1 -- 172.21.15.46:0/4191015078 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f40b45f0 con 0x7f19080d06f0 2024-03-29T08:18:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f1905600640 1 -- 172.21.15.46:0/4191015078 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f40b4750 con 0x7f19080d06f0 2024-03-29T08:18:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f1905600640 1 -- 172.21.15.46:0/4191015078 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f40b4b00 con 0x7f19080d06f0 2024-03-29T08:18:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f1910600640 1 -- 172.21.15.46:0/4191015078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f4009420 con 0x7f19080d06f0 2024-03-29T08:18:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f1910600640 1 -- 172.21.15.46:0/4191015078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190815ebb0 con 0x7f19080d06f0 2024-03-29T08:18:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f1910600640 1 -- 172.21.15.46:0/4191015078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1908066c90 con 0x7f19080d06f0 2024-03-29T08:18:16.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.876+0000 7f1905600640 1 -- 172.21.15.46:0/4191015078 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f18f40b4c60 con 0x7f19080d06f0 2024-03-29T08:18:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.880+0000 7f1905600640 1 --2- 172.21.15.46:0/4191015078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18e80666c0 0x7f18e8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.880+0000 7f1905600640 1 -- 172.21.15.46:0/4191015078 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18f411d780 con 0x7f19080d06f0 2024-03-29T08:18:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.880+0000 7f190dc00640 1 --2- 172.21.15.46:0/4191015078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18e80666c0 0x7f18e8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.880+0000 7f190dc00640 1 --2- 172.21.15.46:0/4191015078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18e80666c0 0x7f18e8068b80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009c50 tx=0x7f18fc009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:16.884+0000 7f1905600640 1 -- 172.21.15.46:0/4191015078 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f18f40eab00 con 0x7f19080d06f0 2024-03-29T08:18:17.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.048+0000 7f1910600640 1 -- 172.21.15.46:0/4191015078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19080cffe0 con 0x7f19080d06f0 2024-03-29T08:18:17.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.048+0000 7f1905600640 1 -- 172.21.15.46:0/4191015078 <== mon.1 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f18f40ee120 con 0x7f19080d06f0 2024-03-29T08:18:17.050 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:17.050 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:14.314124+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.048+0000 7f1910600640 1 -- 172.21.15.46:0/4191015078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18e80666c0 msgr2=0x7f18e8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.048+0000 7f1910600640 1 --2- 172.21.15.46:0/4191015078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18e80666c0 0x7f18e8068b80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009c50 tx=0x7f18fc009190 comp rx=0 tx=0).stop 2024-03-29T08:18:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.048+0000 7f1910600640 1 -- 172.21.15.46:0/4191015078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19080d06f0 msgr2=0x7f1908151680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.048+0000 7f1910600640 1 --2- 172.21.15.46:0/4191015078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19080d06f0 0x7f1908151680 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f18f40ac000 tx=0x7f18f4019210 comp rx=0 tx=0).stop 2024-03-29T08:18:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.048+0000 7f1910600640 1 -- 172.21.15.46:0/4191015078 shutdown_connections 2024-03-29T08:18:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.048+0000 7f1910600640 1 --2- 172.21.15.46:0/4191015078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18e80666c0 0x7f18e8068b80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.052+0000 7f1910600640 1 --2- 172.21.15.46:0/4191015078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19080d06f0 0x7f1908151680 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.052+0000 7f1910600640 1 --2- 172.21.15.46:0/4191015078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f190815cb60 0x7f190814f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.052+0000 7f1910600640 1 --2- 172.21.15.46:0/4191015078 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f19080cae00 0x7f19080aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.052+0000 7f1910600640 1 -- 172.21.15.46:0/4191015078 >> 172.21.15.46:0/4191015078 conn(0x7f1908089fd0 msgr2=0x7f19081586d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.052+0000 7f1910600640 1 -- 172.21.15.46:0/4191015078 shutdown_connections 2024-03-29T08:18:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.052+0000 7f1910600640 1 -- 172.21.15.46:0/4191015078 wait complete. 2024-03-29T08:18:17.071 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T08:18:17.072 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:18:17.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.212+0000 7f0e72400640 1 Processor -- start 2024-03-29T08:18:17.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.212+0000 7f0e72400640 1 -- start start 2024-03-29T08:18:17.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.212+0000 7f0e72400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e6c15cd90 0x7f0e6c06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:17.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.212+0000 7f0e72400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e6c15d940 0x7f0e6c1597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:17.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.212+0000 7f0e72400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6c06d130 con 0x7f0e6c1593d0 2024-03-29T08:18:17.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.212+0000 7f0e72400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e6c15b8f0 con 0x7f0e6c15cd90 2024-03-29T08:18:17.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.212+0000 7f0e72400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c15ad50 con 0x7f0e6c15d940 2024-03-29T08:18:17.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.212+0000 7f0e6be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e6c15d940 0x7f0e6c1597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:17.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.212+0000 7f0e6be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e6c15d940 0x7f0e6c1597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:57382/0 (socket says 172.21.15.46:57382) 2024-03-29T08:18:17.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.212+0000 7f0e6be00640 1 -- 172.21.15.46:0/4132437227 learned_addr learned my addr 172.21.15.46:0/4132437227 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:17.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.212+0000 7f0e62a00640 1 --2- 172.21.15.46:0/4132437227 >> v2:172.21.15.73:3300/0 conn(0x7f0e6c15cd90 0x7f0e6c06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:17.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.212+0000 7f0e6be00640 1 -- 172.21.15.46:0/4132437227 >> v2:172.21.15.73:3300/0 conn(0x7f0e6c15cd90 msgr2=0x7f0e6c06cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:17.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.212+0000 7f0e6be00640 1 --2- 172.21.15.46:0/4132437227 >> v2:172.21.15.73:3300/0 conn(0x7f0e6c15cd90 0x7f0e6c06cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.212+0000 7f0e6be00640 1 -- 172.21.15.46:0/4132437227 >> v1:172.21.15.73:6789/0 conn(0x7f0e6c1593d0 legacy=0x7f0e6c15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:17.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.212+0000 7f0e6be00640 1 -- 172.21.15.46:0/4132437227 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c15afc0 con 0x7f0e6c15d940 2024-03-29T08:18:17.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.216+0000 7f0e6be00640 1 --2- 172.21.15.46:0/4132437227 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e6c15d940 0x7f0e6c1597b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009870 tx=0x7f0e6401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6e6863455537812 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:17.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.216+0000 7f0e6aa00640 1 -- 172.21.15.46:0/4132437227 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e640a9620 con 0x7f0e6c15d940 2024-03-29T08:18:17.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.216+0000 7f0e6aa00640 1 -- 172.21.15.46:0/4132437227 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e640a9780 con 0x7f0e6c15d940 2024-03-29T08:18:17.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.216+0000 7f0e6aa00640 1 -- 172.21.15.46:0/4132437227 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e640a9b10 con 0x7f0e6c15d940 2024-03-29T08:18:17.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.216+0000 7f0e72400640 1 -- 172.21.15.46:0/4132437227 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e6c15d940 msgr2=0x7f0e6c1597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:17.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.216+0000 7f0e72400640 1 --2- 172.21.15.46:0/4132437227 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e6c15d940 0x7f0e6c1597b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0e64009870 tx=0x7f0e6401c560 comp rx=0 tx=0).stop 2024-03-29T08:18:17.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.216+0000 7f0e72400640 1 -- 172.21.15.46:0/4132437227 shutdown_connections 2024-03-29T08:18:17.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.216+0000 7f0e72400640 1 --2- 172.21.15.46:0/4132437227 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e6c15d940 0x7f0e6c1597b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.216+0000 7f0e72400640 1 --2- 172.21.15.46:0/4132437227 >> v2:172.21.15.73:3300/0 conn(0x7f0e6c15cd90 0x7f0e6c06cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.216+0000 7f0e72400640 1 -- 172.21.15.46:0/4132437227 >> 172.21.15.46:0/4132437227 conn(0x7f0e6c066110 msgr2=0x7f0e6c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:17.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.216+0000 7f0e72400640 1 -- 172.21.15.46:0/4132437227 shutdown_connections 2024-03-29T08:18:17.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.216+0000 7f0e72400640 1 -- 172.21.15.46:0/4132437227 wait complete. 2024-03-29T08:18:17.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e72400640 1 Processor -- start 2024-03-29T08:18:17.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e72400640 1 -- start start 2024-03-29T08:18:17.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e72400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e6c1593d0 0x7f0e6c149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:17.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e72400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6c15cd90 0x7f0e6c148b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:17.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e72400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e6c15d940 0x7f0e6c150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:17.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e72400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c156d00 con 0x7f0e6c15cd90 2024-03-29T08:18:17.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e72400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c0cdbf0 con 0x7f0e6c15d940 2024-03-29T08:18:17.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e72400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6c0cd910 con 0x7f0e6c1593d0 2024-03-29T08:18:17.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e62a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6c15cd90 0x7f0e6c148b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:17.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e6b400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e6c1593d0 0x7f0e6c149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:17.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e62a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6c15cd90 0x7f0e6c148b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59704/0 (socket says 172.21.15.46:59704) 2024-03-29T08:18:17.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e62a00640 1 -- 172.21.15.46:0/2893853669 learned_addr learned my addr 172.21.15.46:0/2893853669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:17.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e6be00640 1 --2- 172.21.15.46:0/2893853669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e6c15d940 0x7f0e6c150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:17.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e62a00640 1 -- 172.21.15.46:0/2893853669 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e6c1593d0 msgr2=0x7f0e6c149320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:17.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e62a00640 1 --2- 172.21.15.46:0/2893853669 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e6c1593d0 0x7f0e6c149320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e62a00640 1 -- 172.21.15.46:0/2893853669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e6c15d940 msgr2=0x7f0e6c150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:17.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e62a00640 1 --2- 172.21.15.46:0/2893853669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e6c15d940 0x7f0e6c150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e62a00640 1 -- 172.21.15.46:0/2893853669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e640094d0 con 0x7f0e6c15cd90 2024-03-29T08:18:17.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e6b400640 1 --2- 172.21.15.46:0/2893853669 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e6c1593d0 0x7f0e6c149320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:17.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e62a00640 1 --2- 172.21.15.46:0/2893853669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6c15cd90 0x7f0e6c148b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400e870 tx=0x7f0e5400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:17.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e63e00640 1 -- 172.21.15.46:0/2893853669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e54009510 con 0x7f0e6c15cd90 2024-03-29T08:18:17.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e63e00640 1 -- 172.21.15.46:0/2893853669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5400eec0 con 0x7f0e6c15cd90 2024-03-29T08:18:17.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e72400640 1 -- 172.21.15.46:0/2893853669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6c156f30 con 0x7f0e6c15cd90 2024-03-29T08:18:17.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e63e00640 1 -- 172.21.15.46:0/2893853669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5400cb40 con 0x7f0e6c15cd90 2024-03-29T08:18:17.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e72400640 1 -- 172.21.15.46:0/2893853669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6c1574f0 con 0x7f0e6c15cd90 2024-03-29T08:18:17.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.220+0000 7f0e72400640 1 -- 172.21.15.46:0/2893853669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6c066c90 con 0x7f0e6c15cd90 2024-03-29T08:18:17.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.224+0000 7f0e63e00640 1 -- 172.21.15.46:0/2893853669 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0e5400cca0 con 0x7f0e6c15cd90 2024-03-29T08:18:17.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.224+0000 7f0e63e00640 1 --2- 172.21.15.46:0/2893853669 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e48066690 0x7f0e48068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:17.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.224+0000 7f0e63e00640 1 -- 172.21.15.46:0/2893853669 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e540841f0 con 0x7f0e6c15cd90 2024-03-29T08:18:17.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.224+0000 7f0e6b400640 1 --2- 172.21.15.46:0/2893853669 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e48066690 0x7f0e48068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:17.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.224+0000 7f0e6b400640 1 --2- 172.21.15.46:0/2893853669 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e48066690 0x7f0e48068b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0012f0 tx=0x7f0e5c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:17.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.228+0000 7f0e63e00640 1 -- 172.21.15.46:0/2893853669 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0e54051600 con 0x7f0e6c15cd90 2024-03-29T08:18:17.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.388+0000 7f0e72400640 1 -- 172.21.15.46:0/2893853669 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e6c1490c0 con 0x7f0e48066690 2024-03-29T08:18:17.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.400+0000 7f0e63e00640 1 -- 172.21.15.46:0/2893853669 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264619 (secure 0 0 0) 0x7f0e6c1490c0 con 0x7f0e48066690 2024-03-29T08:18:17.405 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:17.406 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:18:17.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.404+0000 7f0e72400640 1 -- 172.21.15.46:0/2893853669 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e48066690 msgr2=0x7f0e48068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:17.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.404+0000 7f0e72400640 1 --2- 172.21.15.46:0/2893853669 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e48066690 0x7f0e48068b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0012f0 tx=0x7f0e5c009040 comp rx=0 tx=0).stop 2024-03-29T08:18:17.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.404+0000 7f0e72400640 1 -- 172.21.15.46:0/2893853669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6c15cd90 msgr2=0x7f0e6c148b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:17.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.404+0000 7f0e72400640 1 --2- 172.21.15.46:0/2893853669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6c15cd90 0x7f0e6c148b80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400e870 tx=0x7f0e5400ec70 comp rx=0 tx=0).stop 2024-03-29T08:18:17.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.404+0000 7f0e72400640 1 -- 172.21.15.46:0/2893853669 shutdown_connections 2024-03-29T08:18:17.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.404+0000 7f0e72400640 1 --2- 172.21.15.46:0/2893853669 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e48066690 0x7f0e48068b50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.404+0000 7f0e72400640 1 --2- 172.21.15.46:0/2893853669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e6c15d940 0x7f0e6c150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.404+0000 7f0e72400640 1 --2- 172.21.15.46:0/2893853669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e6c15cd90 0x7f0e6c148b80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.404+0000 7f0e72400640 1 --2- 172.21.15.46:0/2893853669 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e6c1593d0 0x7f0e6c149320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.404+0000 7f0e72400640 1 -- 172.21.15.46:0/2893853669 >> 172.21.15.46:0/2893853669 conn(0x7f0e6c066110 msgr2=0x7f0e6c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:17.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.404+0000 7f0e72400640 1 -- 172.21.15.46:0/2893853669 shutdown_connections 2024-03-29T08:18:17.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.404+0000 7f0e72400640 1 -- 172.21.15.46:0/2893853669 wait complete. 2024-03-29T08:18:17.419 INFO:tasks.ceph.mds.b.smithi073.stderr:2024-03-29T08:18:17.414+0000 7fc949600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:18:17.423 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-29T08:18:16.042327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235428,"kb_used_data":2860,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739292,"statfs":{"total":773094113280,"available":772853035008,"internally_reserved":0,"allocated":2928640,"data_stored":1750713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.143117"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.348671+0000","last_change":"2024-03-29T08:18:13.347398+0000","last_active":"2024-03-29T08:18:13.348671+0000","last_peered":"2024-03-29T08:18:13.348671+0000","last_clean":"2024-03-29T08:18:13.348671+0000","last_became_active":"2024-03-29T08:18:13.346857+0000","last_became_peered":"2024-03-29T08:18:13.346857+0000","last_unstale":"2024-03-29T08:18:13.348671+0000","last_undegraded":"2024-03-29T08:18:13.348671+0000","last_fullsized":"2024-03-29T08:18:13.348671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:03.590543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.361542+0000","last_change":"2024-03-29T08:18:13.358700+0000","last_active":"2024-03-29T08:18:13.361542+0000","last_peered":"2024-03-29T08:18:13.361542+0000","last_clean":"2024-03-29T08:18:13.361542+0000","last_became_active":"2024-03-29T08:18:13.357646+0000","last_became_peered":"2024-03-29T08:18:13.357646+0000","last_unstale":"2024-03-29T08:18:13.361542+0000","last_undegraded":"2024-03-29T08:18:13.361542+0000","last_fullsized":"2024-03-29T08:18:13.361542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:20.587134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.355371+0000","last_change":"2024-03-29T08:18:13.352326+0000","last_active":"2024-03-29T08:18:13.355371+0000","last_peered":"2024-03-29T08:18:13.355371+0000","last_clean":"2024-03-29T08:18:13.355371+0000","last_became_active":"2024-03-29T08:18:13.351954+0000","last_became_peered":"2024-03-29T08:18:13.351954+0000","last_unstale":"2024-03-29T08:18:13.355371+0000","last_undegraded":"2024-03-29T08:18:13.355371+0000","last_fullsized":"2024-03-29T08:18:13.355371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:07:42.013921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089100+0000","last_change":"2024-03-29T08:18:13.336578+0000","last_active":"2024-03-29T08:18:15.089100+0000","last_peered":"2024-03-29T08:18:15.089100+0000","last_clean":"2024-03-29T08:18:15.089100+0000","last_became_active":"2024-03-29T08:18:13.336344+0000","last_became_peered":"2024-03-29T08:18:13.336344+0000","last_unstale":"2024-03-29T08:18:15.089100+0000","last_undegraded":"2024-03-29T08:18:15.089100+0000","last_fullsized":"2024-03-29T08:18:15.089100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:30.625311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.350069+0000","last_change":"2024-03-29T08:18:13.348320+0000","last_active":"2024-03-29T08:18:13.350069+0000","last_peered":"2024-03-29T08:18:13.350069+0000","last_clean":"2024-03-29T08:18:13.350069+0000","last_became_active":"2024-03-29T08:18:13.345863+0000","last_became_peered":"2024-03-29T08:18:13.345863+0000","last_unstale":"2024-03-29T08:18:13.350069+0000","last_undegraded":"2024-03-29T08:18:13.350069+0000","last_fullsized":"2024-03-29T08:18:13.350069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:26:51.855915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.088872+0000","last_change":"2024-03-29T08:18:13.345374+0000","last_active":"2024-03-29T08:18:15.088872+0000","last_peered":"2024-03-29T08:18:15.088872+0000","last_clean":"2024-03-29T08:18:15.088872+0000","last_became_active":"2024-03-29T08:18:13.345054+0000","last_became_peered":"2024-03-29T08:18:13.345054+0000","last_unstale":"2024-03-29T08:18:15.088872+0000","last_undegraded":"2024-03-29T08:18:15.088872+0000","last_fullsized":"2024-03-29T08:18:15.088872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:32.324564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.349140+0000","last_change":"2024-03-29T08:18:13.346637+0000","last_active":"2024-03-29T08:18:13.349140+0000","last_peered":"2024-03-29T08:18:13.349140+0000","last_clean":"2024-03-29T08:18:13.349140+0000","last_became_active":"2024-03-29T08:18:13.346335+0000","last_became_peered":"2024-03-29T08:18:13.346335+0000","last_unstale":"2024-03-29T08:18:13.349140+0000","last_undegraded":"2024-03-29T08:18:13.349140+0000","last_fullsized":"2024-03-29T08:18:13.349140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:23.267817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.353926+0000","last_change":"2024-03-29T08:18:13.348979+0000","last_active":"2024-03-29T08:18:13.353926+0000","last_peered":"2024-03-29T08:18:13.353926+0000","last_clean":"2024-03-29T08:18:13.353926+0000","last_became_active":"2024-03-29T08:18:13.348682+0000","last_became_peered":"2024-03-29T08:18:13.348682+0000","last_unstale":"2024-03-29T08:18:13.353926+0000","last_undegraded":"2024-03-29T08:18:13.353926+0000","last_fullsized":"2024-03-29T08:18:13.353926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:53:21.565448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321443+0000","last_change":"2024-03-29T08:18:13.339556+0000","last_active":"2024-03-29T08:18:14.321443+0000","last_peered":"2024-03-29T08:18:14.321443+0000","last_clean":"2024-03-29T08:18:14.321443+0000","last_became_active":"2024-03-29T08:18:13.339318+0000","last_became_peered":"2024-03-29T08:18:13.339318+0000","last_unstale":"2024-03-29T08:18:14.321443+0000","last_undegraded":"2024-03-29T08:18:14.321443+0000","last_fullsized":"2024-03-29T08:18:14.321443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:45.475256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.343185+0000","last_change":"2024-03-29T08:18:13.337238+0000","last_active":"2024-03-29T08:18:13.343185+0000","last_peered":"2024-03-29T08:18:13.343185+0000","last_clean":"2024-03-29T08:18:13.343185+0000","last_became_active":"2024-03-29T08:18:13.336549+0000","last_became_peered":"2024-03-29T08:18:13.336549+0000","last_unstale":"2024-03-29T08:18:13.343185+0000","last_undegraded":"2024-03-29T08:18:13.343185+0000","last_fullsized":"2024-03-29T08:18:13.343185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:32:48.313814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.348043+0000","last_change":"2024-03-29T08:18:13.346353+0000","last_active":"2024-03-29T08:18:13.348043+0000","last_peered":"2024-03-29T08:18:13.348043+0000","last_clean":"2024-03-29T08:18:13.348043+0000","last_became_active":"2024-03-29T08:18:13.345780+0000","last_became_peered":"2024-03-29T08:18:13.345780+0000","last_unstale":"2024-03-29T08:18:13.348043+0000","last_undegraded":"2024-03-29T08:18:13.348043+0000","last_fullsized":"2024-03-29T08:18:13.348043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:45:55.900052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.339429+0000","last_change":"2024-03-29T08:18:13.331322+0000","last_active":"2024-03-29T08:18:13.339429+0000","last_peered":"2024-03-29T08:18:13.339429+0000","last_clean":"2024-03-29T08:18:13.339429+0000","last_became_active":"2024-03-29T08:18:13.331047+0000","last_became_peered":"2024-03-29T08:18:13.331047+0000","last_unstale":"2024-03-29T08:18:13.339429+0000","last_undegraded":"2024-03-29T08:18:13.339429+0000","last_fullsized":"2024-03-29T08:18:13.339429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:40.888705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.364236+0000","last_change":"2024-03-29T08:18:13.357625+0000","last_active":"2024-03-29T08:18:13.364236+0000","last_peered":"2024-03-29T08:18:13.364236+0000","last_clean":"2024-03-29T08:18:13.364236+0000","last_became_active":"2024-03-29T08:18:13.356826+0000","last_became_peered":"2024-03-29T08:18:13.356826+0000","last_unstale":"2024-03-29T08:18:13.364236+0000","last_undegraded":"2024-03-29T08:18:13.364236+0000","last_fullsized":"2024-03-29T08:18:13.364236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:30.625311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.364053+0000","last_change":"2024-03-29T08:18:13.356033+0000","last_active":"2024-03-29T08:18:13.364053+0000","last_peered":"2024-03-29T08:18:13.364053+0000","last_clean":"2024-03-29T08:18:13.364053+0000","last_became_active":"2024-03-29T08:18:13.355720+0000","last_became_peered":"2024-03-29T08:18:13.355720+0000","last_unstale":"2024-03-29T08:18:13.364053+0000","last_undegraded":"2024-03-29T08:18:13.364053+0000","last_fullsized":"2024-03-29T08:18:13.364053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:56.798095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.353969+0000","last_change":"2024-03-29T08:18:13.349080+0000","last_active":"2024-03-29T08:18:13.353969+0000","last_peered":"2024-03-29T08:18:13.353969+0000","last_clean":"2024-03-29T08:18:13.353969+0000","last_became_active":"2024-03-29T08:18:13.348721+0000","last_became_peered":"2024-03-29T08:18:13.348721+0000","last_unstale":"2024-03-29T08:18:13.353969+0000","last_undegraded":"2024-03-29T08:18:13.353969+0000","last_fullsized":"2024-03-29T08:18:13.353969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:54.605777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.354577+0000","last_change":"2024-03-29T08:18:13.349631+0000","last_active":"2024-03-29T08:18:13.354577+0000","last_peered":"2024-03-29T08:18:13.354577+0000","last_clean":"2024-03-29T08:18:13.354577+0000","last_became_active":"2024-03-29T08:18:13.349377+0000","last_became_peered":"2024-03-29T08:18:13.349377+0000","last_unstale":"2024-03-29T08:18:13.354577+0000","last_undegraded":"2024-03-29T08:18:13.354577+0000","last_fullsized":"2024-03-29T08:18:13.354577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:44:22.135683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.354546+0000","last_change":"2024-03-29T08:18:13.342594+0000","last_active":"2024-03-29T08:18:13.354546+0000","last_peered":"2024-03-29T08:18:13.354546+0000","last_clean":"2024-03-29T08:18:13.354546+0000","last_became_active":"2024-03-29T08:18:13.341973+0000","last_became_peered":"2024-03-29T08:18:13.341973+0000","last_unstale":"2024-03-29T08:18:13.354546+0000","last_undegraded":"2024-03-29T08:18:13.354546+0000","last_fullsized":"2024-03-29T08:18:13.354546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:31:39.118083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323494+0000","last_change":"2024-03-29T08:18:13.349000+0000","last_active":"2024-03-29T08:18:14.323494+0000","last_peered":"2024-03-29T08:18:14.323494+0000","last_clean":"2024-03-29T08:18:14.323494+0000","last_became_active":"2024-03-29T08:18:13.348586+0000","last_became_peered":"2024-03-29T08:18:13.348586+0000","last_unstale":"2024-03-29T08:18:14.323494+0000","last_undegraded":"2024-03-29T08:18:14.323494+0000","last_fullsized":"2024-03-29T08:18:14.323494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:55:41.020683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.354616+0000","last_change":"2024-03-29T08:18:13.342652+0000","last_active":"2024-03-29T08:18:13.354616+0000","last_peered":"2024-03-29T08:18:13.354616+0000","last_clean":"2024-03-29T08:18:13.354616+0000","last_became_active":"2024-03-29T08:18:13.342339+0000","last_became_peered":"2024-03-29T08:18:13.342339+0000","last_unstale":"2024-03-29T08:18:13.354616+0000","last_undegraded":"2024-03-29T08:18:13.354616+0000","last_fullsized":"2024-03-29T08:18:13.354616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:46:35.075911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.332696+0000","last_change":"2024-03-29T08:18:13.325928+0000","last_active":"2024-03-29T08:18:13.332696+0000","last_peered":"2024-03-29T08:18:13.332696+0000","last_clean":"2024-03-29T08:18:13.332696+0000","last_became_active":"2024-03-29T08:18:13.325659+0000","last_became_peered":"2024-03-29T08:18:13.325659+0000","last_unstale":"2024-03-29T08:18:13.332696+0000","last_undegraded":"2024-03-29T08:18:13.332696+0000","last_fullsized":"2024-03-29T08:18:13.332696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:58.411155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324285+0000","last_change":"2024-03-29T08:18:13.339244+0000","last_active":"2024-03-29T08:18:14.324285+0000","last_peered":"2024-03-29T08:18:14.324285+0000","last_clean":"2024-03-29T08:18:14.324285+0000","last_became_active":"2024-03-29T08:18:13.338984+0000","last_became_peered":"2024-03-29T08:18:13.338984+0000","last_unstale":"2024-03-29T08:18:14.324285+0000","last_undegraded":"2024-03-29T08:18:14.324285+0000","last_fullsized":"2024-03-29T08:18:14.324285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:29:09.699201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321134+0000","last_change":"2024-03-29T08:18:13.330207+0000","last_active":"2024-03-29T08:18:14.321134+0000","last_peered":"2024-03-29T08:18:14.321134+0000","last_clean":"2024-03-29T08:18:14.321134+0000","last_became_active":"2024-03-29T08:18:13.330029+0000","last_became_peered":"2024-03-29T08:18:13.330029+0000","last_unstale":"2024-03-29T08:18:14.321134+0000","last_undegraded":"2024-03-29T08:18:14.321134+0000","last_fullsized":"2024-03-29T08:18:14.321134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:32.324564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.353573+0000","last_change":"2024-03-29T08:18:13.347985+0000","last_active":"2024-03-29T08:18:13.353573+0000","last_peered":"2024-03-29T08:18:13.353573+0000","last_clean":"2024-03-29T08:18:13.353573+0000","last_became_active":"2024-03-29T08:18:13.347561+0000","last_became_peered":"2024-03-29T08:18:13.347561+0000","last_unstale":"2024-03-29T08:18:13.353573+0000","last_undegraded":"2024-03-29T08:18:13.353573+0000","last_fullsized":"2024-03-29T08:18:13.353573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:17.775287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.340909+0000","last_change":"2024-03-29T08:18:13.338212+0000","last_active":"2024-03-29T08:18:13.340909+0000","last_peered":"2024-03-29T08:18:13.340909+0000","last_clean":"2024-03-29T08:18:13.340909+0000","last_became_active":"2024-03-29T08:18:13.337908+0000","last_became_peered":"2024-03-29T08:18:13.337908+0000","last_unstale":"2024-03-29T08:18:13.340909+0000","last_undegraded":"2024-03-29T08:18:13.340909+0000","last_fullsized":"2024-03-29T08:18:13.340909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:40.888705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.346404+0000","last_change":"2024-03-29T08:18:13.341534+0000","last_active":"2024-03-29T08:18:13.346404+0000","last_peered":"2024-03-29T08:18:13.346404+0000","last_clean":"2024-03-29T08:18:13.346404+0000","last_became_active":"2024-03-29T08:18:13.341097+0000","last_became_peered":"2024-03-29T08:18:13.341097+0000","last_unstale":"2024-03-29T08:18:13.346404+0000","last_undegraded":"2024-03-29T08:18:13.346404+0000","last_fullsized":"2024-03-29T08:18:13.346404+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:05:16.874804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089257+0000","last_change":"2024-03-29T08:18:13.342465+0000","last_active":"2024-03-29T08:18:15.089257+0000","last_peered":"2024-03-29T08:18:15.089257+0000","last_clean":"2024-03-29T08:18:15.089257+0000","last_became_active":"2024-03-29T08:18:13.342000+0000","last_became_peered":"2024-03-29T08:18:13.342000+0000","last_unstale":"2024-03-29T08:18:15.089257+0000","last_undegraded":"2024-03-29T08:18:15.089257+0000","last_fullsized":"2024-03-29T08:18:15.089257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:05.352866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.364081+0000","last_change":"2024-03-29T08:18:13.356380+0000","last_active":"2024-03-29T08:18:13.364081+0000","last_peered":"2024-03-29T08:18:13.364081+0000","last_clean":"2024-03-29T08:18:13.364081+0000","last_became_active":"2024-03-29T08:18:13.356016+0000","last_became_peered":"2024-03-29T08:18:13.356016+0000","last_unstale":"2024-03-29T08:18:13.364081+0000","last_undegraded":"2024-03-29T08:18:13.364081+0000","last_fullsized":"2024-03-29T08:18:13.364081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:05.352866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.364192+0000","last_change":"2024-03-29T08:18:13.357406+0000","last_active":"2024-03-29T08:18:13.364192+0000","last_peered":"2024-03-29T08:18:13.364192+0000","last_clean":"2024-03-29T08:18:13.364192+0000","last_became_active":"2024-03-29T08:18:13.356875+0000","last_became_peered":"2024-03-29T08:18:13.356875+0000","last_unstale":"2024-03-29T08:18:13.364192+0000","last_undegraded":"2024-03-29T08:18:13.364192+0000","last_fullsized":"2024-03-29T08:18:13.364192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:52:27.461296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.345107+0000","last_change":"2024-03-29T08:18:13.338239+0000","last_active":"2024-03-29T08:18:13.345107+0000","last_peered":"2024-03-29T08:18:13.345107+0000","last_clean":"2024-03-29T08:18:13.345107+0000","last_became_active":"2024-03-29T08:18:13.337901+0000","last_became_peered":"2024-03-29T08:18:13.337901+0000","last_unstale":"2024-03-29T08:18:13.345107+0000","last_undegraded":"2024-03-29T08:18:13.345107+0000","last_fullsized":"2024-03-29T08:18:13.345107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:41:40.150525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.340980+0000","last_change":"2024-03-29T08:18:13.338530+0000","last_active":"2024-03-29T08:18:13.340980+0000","last_peered":"2024-03-29T08:18:13.340980+0000","last_clean":"2024-03-29T08:18:13.340980+0000","last_became_active":"2024-03-29T08:18:13.338188+0000","last_became_peered":"2024-03-29T08:18:13.338188+0000","last_unstale":"2024-03-29T08:18:13.340980+0000","last_undegraded":"2024-03-29T08:18:13.340980+0000","last_fullsized":"2024-03-29T08:18:13.340980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:03.590543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323230+0000","last_change":"2024-03-29T08:18:13.344948+0000","last_active":"2024-03-29T08:18:14.323230+0000","last_peered":"2024-03-29T08:18:14.323230+0000","last_clean":"2024-03-29T08:18:14.323230+0000","last_became_active":"2024-03-29T08:18:13.341442+0000","last_became_peered":"2024-03-29T08:18:13.341442+0000","last_unstale":"2024-03-29T08:18:14.323230+0000","last_undegraded":"2024-03-29T08:18:14.323230+0000","last_fullsized":"2024-03-29T08:18:14.323230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:53:57.611306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.088799+0000","last_change":"2024-03-29T08:18:13.356080+0000","last_active":"2024-03-29T08:18:15.088799+0000","last_peered":"2024-03-29T08:18:15.088799+0000","last_clean":"2024-03-29T08:18:15.088799+0000","last_became_active":"2024-03-29T08:18:13.355885+0000","last_became_peered":"2024-03-29T08:18:13.355885+0000","last_unstale":"2024-03-29T08:18:15.088799+0000","last_undegraded":"2024-03-29T08:18:15.088799+0000","last_fullsized":"2024-03-29T08:18:15.088799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:57.716575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321203+0000","last_change":"2024-03-29T08:18:13.343561+0000","last_active":"2024-03-29T08:18:14.321203+0000","last_peered":"2024-03-29T08:18:14.321203+0000","last_clean":"2024-03-29T08:18:14.321203+0000","last_became_active":"2024-03-29T08:18:13.341129+0000","last_became_peered":"2024-03-29T08:18:13.341129+0000","last_unstale":"2024-03-29T08:18:14.321203+0000","last_undegraded":"2024-03-29T08:18:14.321203+0000","last_fullsized":"2024-03-29T08:18:14.321203+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:30.625311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.357944+0000","last_change":"2024-03-29T08:18:12.928717+0000","last_active":"2024-03-29T08:18:13.357944+0000","last_peered":"2024-03-29T08:18:13.357944+0000","last_clean":"2024-03-29T08:18:13.357944+0000","last_became_active":"2024-03-29T08:18:12.928427+0000","last_became_peered":"2024-03-29T08:18:12.928427+0000","last_unstale":"2024-03-29T08:18:13.357944+0000","last_undegraded":"2024-03-29T08:18:13.357944+0000","last_fullsized":"2024-03-29T08:18:13.357944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:28:47.778848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324418+0000","last_change":"2024-03-29T08:18:13.346417+0000","last_active":"2024-03-29T08:18:14.324418+0000","last_peered":"2024-03-29T08:18:14.324418+0000","last_clean":"2024-03-29T08:18:14.324418+0000","last_became_active":"2024-03-29T08:18:13.344718+0000","last_became_peered":"2024-03-29T08:18:13.344718+0000","last_unstale":"2024-03-29T08:18:14.324418+0000","last_undegraded":"2024-03-29T08:18:14.324418+0000","last_fullsized":"2024-03-29T08:18:14.324418+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:24:34.643975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.351566+0000","last_change":"2024-03-29T08:18:12.930498+0000","last_active":"2024-03-29T08:18:13.351566+0000","last_peered":"2024-03-29T08:18:13.351566+0000","last_clean":"2024-03-29T08:18:13.351566+0000","last_became_active":"2024-03-29T08:18:12.930232+0000","last_became_peered":"2024-03-29T08:18:12.930232+0000","last_unstale":"2024-03-29T08:18:13.351566+0000","last_undegraded":"2024-03-29T08:18:13.351566+0000","last_fullsized":"2024-03-29T08:18:13.351566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:04.346131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.339504+0000","last_change":"2024-03-29T08:18:12.927991+0000","last_active":"2024-03-29T08:18:13.339504+0000","last_peered":"2024-03-29T08:18:13.339504+0000","last_clean":"2024-03-29T08:18:13.339504+0000","last_became_active":"2024-03-29T08:18:12.927288+0000","last_became_peered":"2024-03-29T08:18:12.927288+0000","last_unstale":"2024-03-29T08:18:13.339504+0000","last_undegraded":"2024-03-29T08:18:13.339504+0000","last_fullsized":"2024-03-29T08:18:13.339504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:30:10.121158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.361521+0000","last_change":"2024-03-29T08:18:13.358303+0000","last_active":"2024-03-29T08:18:13.361521+0000","last_peered":"2024-03-29T08:18:13.361521+0000","last_clean":"2024-03-29T08:18:13.361521+0000","last_became_active":"2024-03-29T08:18:13.358041+0000","last_became_peered":"2024-03-29T08:18:13.358041+0000","last_unstale":"2024-03-29T08:18:13.361521+0000","last_undegraded":"2024-03-29T08:18:13.361521+0000","last_fullsized":"2024-03-29T08:18:13.361521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:41:41.207988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.324995+0000","last_change":"2024-03-29T08:18:12.927896+0000","last_active":"2024-03-29T08:18:13.324995+0000","last_peered":"2024-03-29T08:18:13.324995+0000","last_clean":"2024-03-29T08:18:13.324995+0000","last_became_active":"2024-03-29T08:18:12.927399+0000","last_became_peered":"2024-03-29T08:18:12.927399+0000","last_unstale":"2024-03-29T08:18:13.324995+0000","last_undegraded":"2024-03-29T08:18:13.324995+0000","last_fullsized":"2024-03-29T08:18:13.324995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:56.709840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324362+0000","last_change":"2024-03-29T08:18:13.338175+0000","last_active":"2024-03-29T08:18:14.324362+0000","last_peered":"2024-03-29T08:18:14.324362+0000","last_clean":"2024-03-29T08:18:14.324362+0000","last_became_active":"2024-03-29T08:18:13.337775+0000","last_became_peered":"2024-03-29T08:18:13.337775+0000","last_unstale":"2024-03-29T08:18:14.324362+0000","last_undegraded":"2024-03-29T08:18:14.324362+0000","last_fullsized":"2024-03-29T08:18:14.324362+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:45:52.608559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321469+0000","last_change":"2024-03-29T08:18:12.940829+0000","last_active":"2024-03-29T08:18:14.321469+0000","last_peered":"2024-03-29T08:18:14.321469+0000","last_clean":"2024-03-29T08:18:14.321469+0000","last_became_active":"2024-03-29T08:18:12.929330+0000","last_became_peered":"2024-03-29T08:18:12.929330+0000","last_unstale":"2024-03-29T08:18:14.321469+0000","last_undegraded":"2024-03-29T08:18:14.321469+0000","last_fullsized":"2024-03-29T08:18:14.321469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:55:09.380677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089346+0000","last_change":"2024-03-29T08:18:13.342303+0000","last_active":"2024-03-29T08:18:15.089346+0000","last_peered":"2024-03-29T08:18:15.089346+0000","last_clean":"2024-03-29T08:18:15.089346+0000","last_became_active":"2024-03-29T08:18:13.341642+0000","last_became_peered":"2024-03-29T08:18:13.341642+0000","last_unstale":"2024-03-29T08:18:15.089346+0000","last_undegraded":"2024-03-29T08:18:15.089346+0000","last_fullsized":"2024-03-29T08:18:15.089346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:45.475256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.316882+0000","last_change":"2024-03-29T08:18:12.930755+0000","last_active":"2024-03-29T08:18:13.316882+0000","last_peered":"2024-03-29T08:18:13.316882+0000","last_clean":"2024-03-29T08:18:13.316882+0000","last_became_active":"2024-03-29T08:18:12.929647+0000","last_became_peered":"2024-03-29T08:18:12.929647+0000","last_unstale":"2024-03-29T08:18:13.316882+0000","last_undegraded":"2024-03-29T08:18:13.316882+0000","last_fullsized":"2024-03-29T08:18:13.316882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:16:38.127431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.346719+0000","last_change":"2024-03-29T08:18:13.344231+0000","last_active":"2024-03-29T08:18:13.346719+0000","last_peered":"2024-03-29T08:18:13.346719+0000","last_clean":"2024-03-29T08:18:13.346719+0000","last_became_active":"2024-03-29T08:18:13.343788+0000","last_became_peered":"2024-03-29T08:18:13.343788+0000","last_unstale":"2024-03-29T08:18:13.346719+0000","last_undegraded":"2024-03-29T08:18:13.346719+0000","last_fullsized":"2024-03-29T08:18:13.346719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:15:15.589437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323478+0000","last_change":"2024-03-29T08:18:12.925532+0000","last_active":"2024-03-29T08:18:14.323478+0000","last_peered":"2024-03-29T08:18:14.323478+0000","last_clean":"2024-03-29T08:18:14.323478+0000","last_became_active":"2024-03-29T08:18:12.925228+0000","last_became_peered":"2024-03-29T08:18:12.925228+0000","last_unstale":"2024-03-29T08:18:14.323478+0000","last_undegraded":"2024-03-29T08:18:14.323478+0000","last_fullsized":"2024-03-29T08:18:14.323478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:30:10.121158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.348719+0000","last_change":"2024-03-29T08:18:13.339687+0000","last_active":"2024-03-29T08:18:13.348719+0000","last_peered":"2024-03-29T08:18:13.348719+0000","last_clean":"2024-03-29T08:18:13.348719+0000","last_became_active":"2024-03-29T08:18:13.339445+0000","last_became_peered":"2024-03-29T08:18:13.339445+0000","last_unstale":"2024-03-29T08:18:13.348719+0000","last_undegraded":"2024-03-29T08:18:13.348719+0000","last_fullsized":"2024-03-29T08:18:13.348719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:45:55.900052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089195+0000","last_change":"2024-03-29T08:18:12.927677+0000","last_active":"2024-03-29T08:18:15.089195+0000","last_peered":"2024-03-29T08:18:15.089195+0000","last_clean":"2024-03-29T08:18:15.089195+0000","last_became_active":"2024-03-29T08:18:12.927416+0000","last_became_peered":"2024-03-29T08:18:12.927416+0000","last_unstale":"2024-03-29T08:18:15.089195+0000","last_undegraded":"2024-03-29T08:18:15.089195+0000","last_fullsized":"2024-03-29T08:18:15.089195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:47.035098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.358272+0000","last_change":"2024-03-29T08:18:13.357432+0000","last_active":"2024-03-29T08:18:13.358272+0000","last_peered":"2024-03-29T08:18:13.358272+0000","last_clean":"2024-03-29T08:18:13.358272+0000","last_became_active":"2024-03-29T08:18:13.357221+0000","last_became_peered":"2024-03-29T08:18:13.357221+0000","last_unstale":"2024-03-29T08:18:13.358272+0000","last_undegraded":"2024-03-29T08:18:13.358272+0000","last_fullsized":"2024-03-29T08:18:13.358272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:49.457858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323462+0000","last_change":"2024-03-29T08:18:12.930512+0000","last_active":"2024-03-29T08:18:14.323462+0000","last_peered":"2024-03-29T08:18:14.323462+0000","last_clean":"2024-03-29T08:18:14.323462+0000","last_became_active":"2024-03-29T08:18:12.927576+0000","last_became_peered":"2024-03-29T08:18:12.927576+0000","last_unstale":"2024-03-29T08:18:14.323462+0000","last_undegraded":"2024-03-29T08:18:14.323462+0000","last_fullsized":"2024-03-29T08:18:14.323462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:58.083723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.361037+0000","last_change":"2024-03-29T08:18:13.344540+0000","last_active":"2024-03-29T08:18:13.361037+0000","last_peered":"2024-03-29T08:18:13.361037+0000","last_clean":"2024-03-29T08:18:13.361037+0000","last_became_active":"2024-03-29T08:18:13.343172+0000","last_became_peered":"2024-03-29T08:18:13.343172+0000","last_unstale":"2024-03-29T08:18:13.361037+0000","last_undegraded":"2024-03-29T08:18:13.361037+0000","last_fullsized":"2024-03-29T08:18:13.361037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:15:47.486329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.332921+0000","last_change":"2024-03-29T08:18:12.929722+0000","last_active":"2024-03-29T08:18:13.332921+0000","last_peered":"2024-03-29T08:18:13.332921+0000","last_clean":"2024-03-29T08:18:13.332921+0000","last_became_active":"2024-03-29T08:18:12.929401+0000","last_became_peered":"2024-03-29T08:18:12.929401+0000","last_unstale":"2024-03-29T08:18:13.332921+0000","last_undegraded":"2024-03-29T08:18:13.332921+0000","last_fullsized":"2024-03-29T08:18:13.332921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:44.468521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.350830+0000","last_change":"2024-03-29T08:18:13.331364+0000","last_active":"2024-03-29T08:18:13.350830+0000","last_peered":"2024-03-29T08:18:13.350830+0000","last_clean":"2024-03-29T08:18:13.350830+0000","last_became_active":"2024-03-29T08:18:13.331097+0000","last_became_peered":"2024-03-29T08:18:13.331097+0000","last_unstale":"2024-03-29T08:18:13.350830+0000","last_undegraded":"2024-03-29T08:18:13.350830+0000","last_fullsized":"2024-03-29T08:18:13.350830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:22:49.782486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.315979+0000","last_change":"2024-03-29T08:18:12.935254+0000","last_active":"2024-03-29T08:18:13.315979+0000","last_peered":"2024-03-29T08:18:13.315979+0000","last_clean":"2024-03-29T08:18:13.315979+0000","last_became_active":"2024-03-29T08:18:12.934862+0000","last_became_peered":"2024-03-29T08:18:12.934862+0000","last_unstale":"2024-03-29T08:18:13.315979+0000","last_undegraded":"2024-03-29T08:18:13.315979+0000","last_fullsized":"2024-03-29T08:18:13.315979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:24.943478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321225+0000","last_change":"2024-03-29T08:18:13.338357+0000","last_active":"2024-03-29T08:18:14.321225+0000","last_peered":"2024-03-29T08:18:14.321225+0000","last_clean":"2024-03-29T08:18:14.321225+0000","last_became_active":"2024-03-29T08:18:13.338109+0000","last_became_peered":"2024-03-29T08:18:13.338109+0000","last_unstale":"2024-03-29T08:18:14.321225+0000","last_undegraded":"2024-03-29T08:18:14.321225+0000","last_fullsized":"2024-03-29T08:18:14.321225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:55:41.020683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324252+0000","last_change":"2024-03-29T08:18:12.914431+0000","last_active":"2024-03-29T08:18:14.324252+0000","last_peered":"2024-03-29T08:18:14.324252+0000","last_clean":"2024-03-29T08:18:14.324252+0000","last_became_active":"2024-03-29T08:18:12.913939+0000","last_became_peered":"2024-03-29T08:18:12.913939+0000","last_unstale":"2024-03-29T08:18:14.324252+0000","last_undegraded":"2024-03-29T08:18:14.324252+0000","last_fullsized":"2024-03-29T08:18:14.324252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:16:38.127431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.355309+0000","last_change":"2024-03-29T08:18:13.351406+0000","last_active":"2024-03-29T08:18:13.355309+0000","last_peered":"2024-03-29T08:18:13.355309+0000","last_clean":"2024-03-29T08:18:13.355309+0000","last_became_active":"2024-03-29T08:18:13.351143+0000","last_became_peered":"2024-03-29T08:18:13.351143+0000","last_unstale":"2024-03-29T08:18:13.355309+0000","last_undegraded":"2024-03-29T08:18:13.355309+0000","last_fullsized":"2024-03-29T08:18:13.355309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:33.632435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.346645+0000","last_change":"2024-03-29T08:18:12.936118+0000","last_active":"2024-03-29T08:18:13.346645+0000","last_peered":"2024-03-29T08:18:13.346645+0000","last_clean":"2024-03-29T08:18:13.346645+0000","last_became_active":"2024-03-29T08:18:12.935861+0000","last_became_peered":"2024-03-29T08:18:12.935861+0000","last_unstale":"2024-03-29T08:18:13.346645+0000","last_undegraded":"2024-03-29T08:18:13.346645+0000","last_fullsized":"2024-03-29T08:18:13.346645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:44.468521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089321+0000","last_change":"2024-03-29T08:18:13.341154+0000","last_active":"2024-03-29T08:18:15.089321+0000","last_peered":"2024-03-29T08:18:15.089321+0000","last_clean":"2024-03-29T08:18:15.089321+0000","last_became_active":"2024-03-29T08:18:13.335461+0000","last_became_peered":"2024-03-29T08:18:13.335461+0000","last_unstale":"2024-03-29T08:18:15.089321+0000","last_undegraded":"2024-03-29T08:18:15.089321+0000","last_fullsized":"2024-03-29T08:18:15.089321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:55:41.020683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324529+0000","last_change":"2024-03-29T08:18:12.928629+0000","last_active":"2024-03-29T08:18:14.324529+0000","last_peered":"2024-03-29T08:18:14.324529+0000","last_clean":"2024-03-29T08:18:14.324529+0000","last_became_active":"2024-03-29T08:18:12.928405+0000","last_became_peered":"2024-03-29T08:18:12.928405+0000","last_unstale":"2024-03-29T08:18:14.324529+0000","last_undegraded":"2024-03-29T08:18:14.324529+0000","last_fullsized":"2024-03-29T08:18:14.324529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:06.406132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.328112+0000","last_change":"2024-03-29T08:18:13.325885+0000","last_active":"2024-03-29T08:18:13.328112+0000","last_peered":"2024-03-29T08:18:13.328112+0000","last_clean":"2024-03-29T08:18:13.328112+0000","last_became_active":"2024-03-29T08:18:13.325621+0000","last_became_peered":"2024-03-29T08:18:13.325621+0000","last_unstale":"2024-03-29T08:18:13.328112+0000","last_undegraded":"2024-03-29T08:18:13.328112+0000","last_fullsized":"2024-03-29T08:18:13.328112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:27:31.782313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.340223+0000","last_change":"2024-03-29T08:18:12.932348+0000","last_active":"2024-03-29T08:18:13.340223+0000","last_peered":"2024-03-29T08:18:13.340223+0000","last_clean":"2024-03-29T08:18:13.340223+0000","last_became_active":"2024-03-29T08:18:12.931959+0000","last_became_peered":"2024-03-29T08:18:12.931959+0000","last_unstale":"2024-03-29T08:18:13.340223+0000","last_undegraded":"2024-03-29T08:18:13.340223+0000","last_fullsized":"2024-03-29T08:18:13.340223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:04:56.043379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.340975+0000","last_change":"2024-03-29T08:18:13.339774+0000","last_active":"2024-03-29T08:18:13.340975+0000","last_peered":"2024-03-29T08:18:13.340975+0000","last_clean":"2024-03-29T08:18:13.340975+0000","last_became_active":"2024-03-29T08:18:13.339154+0000","last_became_peered":"2024-03-29T08:18:13.339154+0000","last_unstale":"2024-03-29T08:18:13.340975+0000","last_undegraded":"2024-03-29T08:18:13.340975+0000","last_fullsized":"2024-03-29T08:18:13.340975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:10.712191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.341794+0000","last_change":"2024-03-29T08:18:12.924839+0000","last_active":"2024-03-29T08:18:13.341794+0000","last_peered":"2024-03-29T08:18:13.341794+0000","last_clean":"2024-03-29T08:18:13.341794+0000","last_became_active":"2024-03-29T08:18:12.924535+0000","last_became_peered":"2024-03-29T08:18:12.924535+0000","last_unstale":"2024-03-29T08:18:13.341794+0000","last_undegraded":"2024-03-29T08:18:13.341794+0000","last_fullsized":"2024-03-29T08:18:13.341794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:58.083723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.331983+0000","last_change":"2024-03-29T08:18:13.328934+0000","last_active":"2024-03-29T08:18:13.331983+0000","last_peered":"2024-03-29T08:18:13.331983+0000","last_clean":"2024-03-29T08:18:13.331983+0000","last_became_active":"2024-03-29T08:18:13.328410+0000","last_became_peered":"2024-03-29T08:18:13.328410+0000","last_unstale":"2024-03-29T08:18:13.331983+0000","last_undegraded":"2024-03-29T08:18:13.331983+0000","last_fullsized":"2024-03-29T08:18:13.331983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.345206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.320590+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:18:13.320590+0000","last_peered":"2024-03-29T08:18:13.320590+0000","last_clean":"2024-03-29T08:18:13.320590+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:18:13.320590+0000","last_undegraded":"2024-03-29T08:18:13.320590+0000","last_fullsized":"2024-03-29T08:18:13.320590+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.335826+0000","last_change":"2024-03-29T08:18:12.931858+0000","last_active":"2024-03-29T08:18:13.335826+0000","last_peered":"2024-03-29T08:18:13.335826+0000","last_clean":"2024-03-29T08:18:13.335826+0000","last_became_active":"2024-03-29T08:18:12.931277+0000","last_became_peered":"2024-03-29T08:18:12.931277+0000","last_unstale":"2024-03-29T08:18:13.335826+0000","last_undegraded":"2024-03-29T08:18:13.335826+0000","last_fullsized":"2024-03-29T08:18:13.335826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:29.618576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324316+0000","last_change":"2024-03-29T08:18:13.344361+0000","last_active":"2024-03-29T08:18:14.324316+0000","last_peered":"2024-03-29T08:18:14.324316+0000","last_clean":"2024-03-29T08:18:14.324316+0000","last_became_active":"2024-03-29T08:18:13.344050+0000","last_became_peered":"2024-03-29T08:18:13.344050+0000","last_unstale":"2024-03-29T08:18:14.324316+0000","last_undegraded":"2024-03-29T08:18:14.324316+0000","last_fullsized":"2024-03-29T08:18:14.324316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:42:28.266165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.325088+0000","last_change":"2024-03-29T08:18:12.938041+0000","last_active":"2024-03-29T08:18:13.325088+0000","last_peered":"2024-03-29T08:18:13.325088+0000","last_clean":"2024-03-29T08:18:13.325088+0000","last_became_active":"2024-03-29T08:18:12.937061+0000","last_became_peered":"2024-03-29T08:18:12.937061+0000","last_unstale":"2024-03-29T08:18:13.325088+0000","last_undegraded":"2024-03-29T08:18:13.325088+0000","last_fullsized":"2024-03-29T08:18:13.325088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:55:40.013948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.364028+0000","last_change":"2024-03-29T08:18:13.355981+0000","last_active":"2024-03-29T08:18:13.364028+0000","last_peered":"2024-03-29T08:18:13.364028+0000","last_clean":"2024-03-29T08:18:13.364028+0000","last_became_active":"2024-03-29T08:18:13.355691+0000","last_became_peered":"2024-03-29T08:18:13.355691+0000","last_unstale":"2024-03-29T08:18:13.364028+0000","last_undegraded":"2024-03-29T08:18:13.364028+0000","last_fullsized":"2024-03-29T08:18:13.364028+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:18:34.795188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323242+0000","last_change":"2024-03-29T08:18:12.931240+0000","last_active":"2024-03-29T08:18:14.323242+0000","last_peered":"2024-03-29T08:18:14.323242+0000","last_clean":"2024-03-29T08:18:14.323242+0000","last_became_active":"2024-03-29T08:18:12.930854+0000","last_became_peered":"2024-03-29T08:18:12.930854+0000","last_unstale":"2024-03-29T08:18:14.323242+0000","last_undegraded":"2024-03-29T08:18:14.323242+0000","last_fullsized":"2024-03-29T08:18:14.323242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:55:09.380677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.346454+0000","last_change":"2024-03-29T08:18:13.343562+0000","last_active":"2024-03-29T08:18:13.346454+0000","last_peered":"2024-03-29T08:18:13.346454+0000","last_clean":"2024-03-29T08:18:13.346454+0000","last_became_active":"2024-03-29T08:18:13.342138+0000","last_became_peered":"2024-03-29T08:18:13.342138+0000","last_unstale":"2024-03-29T08:18:13.346454+0000","last_undegraded":"2024-03-29T08:18:13.346454+0000","last_fullsized":"2024-03-29T08:18:13.346454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:24:46.843649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.333089+0000","last_change":"2024-03-29T08:18:12.924976+0000","last_active":"2024-03-29T08:18:13.333089+0000","last_peered":"2024-03-29T08:18:13.333089+0000","last_clean":"2024-03-29T08:18:13.333089+0000","last_became_active":"2024-03-29T08:18:12.924677+0000","last_became_peered":"2024-03-29T08:18:12.924677+0000","last_unstale":"2024-03-29T08:18:13.333089+0000","last_undegraded":"2024-03-29T08:18:13.333089+0000","last_fullsized":"2024-03-29T08:18:13.333089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:18:33.788453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.340787+0000","last_change":"2024-03-29T08:18:13.335351+0000","last_active":"2024-03-29T08:18:13.340787+0000","last_peered":"2024-03-29T08:18:13.340787+0000","last_clean":"2024-03-29T08:18:13.340787+0000","last_became_active":"2024-03-29T08:18:13.334286+0000","last_became_peered":"2024-03-29T08:18:13.334286+0000","last_unstale":"2024-03-29T08:18:13.340787+0000","last_undegraded":"2024-03-29T08:18:13.340787+0000","last_fullsized":"2024-03-29T08:18:13.340787+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:47:40.547759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.335038+0000","last_change":"2024-03-29T08:18:12.930589+0000","last_active":"2024-03-29T08:18:13.335038+0000","last_peered":"2024-03-29T08:18:13.335038+0000","last_clean":"2024-03-29T08:18:13.335038+0000","last_became_active":"2024-03-29T08:18:12.930322+0000","last_became_peered":"2024-03-29T08:18:12.930322+0000","last_unstale":"2024-03-29T08:18:13.335038+0000","last_undegraded":"2024-03-29T08:18:13.335038+0000","last_fullsized":"2024-03-29T08:18:13.335038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:38:37.939078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089112+0000","last_change":"2024-03-29T08:18:13.357420+0000","last_active":"2024-03-29T08:18:15.089112+0000","last_peered":"2024-03-29T08:18:15.089112+0000","last_clean":"2024-03-29T08:18:15.089112+0000","last_became_active":"2024-03-29T08:18:13.357130+0000","last_became_peered":"2024-03-29T08:18:13.357130+0000","last_unstale":"2024-03-29T08:18:15.089112+0000","last_undegraded":"2024-03-29T08:18:15.089112+0000","last_fullsized":"2024-03-29T08:18:15.089112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:07.412867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.339216+0000","last_change":"2024-03-29T08:18:12.938320+0000","last_active":"2024-03-29T08:18:13.339216+0000","last_peered":"2024-03-29T08:18:13.339216+0000","last_clean":"2024-03-29T08:18:13.339216+0000","last_became_active":"2024-03-29T08:18:12.937654+0000","last_became_peered":"2024-03-29T08:18:12.937654+0000","last_unstale":"2024-03-29T08:18:13.339216+0000","last_undegraded":"2024-03-29T08:18:13.339216+0000","last_fullsized":"2024-03-29T08:18:13.339216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:06.406132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.364215+0000","last_change":"2024-03-29T08:18:13.357591+0000","last_active":"2024-03-29T08:18:13.364215+0000","last_peered":"2024-03-29T08:18:13.364215+0000","last_clean":"2024-03-29T08:18:13.364215+0000","last_became_active":"2024-03-29T08:18:13.357160+0000","last_became_peered":"2024-03-29T08:18:13.357160+0000","last_unstale":"2024-03-29T08:18:13.364215+0000","last_undegraded":"2024-03-29T08:18:13.364215+0000","last_fullsized":"2024-03-29T08:18:13.364215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:45:52.608559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324569+0000","last_change":"2024-03-29T08:18:12.926261+0000","last_active":"2024-03-29T08:18:14.324569+0000","last_peered":"2024-03-29T08:18:14.324569+0000","last_clean":"2024-03-29T08:18:14.324569+0000","last_became_active":"2024-03-29T08:18:12.925938+0000","last_became_peered":"2024-03-29T08:18:12.925938+0000","last_unstale":"2024-03-29T08:18:14.324569+0000","last_undegraded":"2024-03-29T08:18:14.324569+0000","last_fullsized":"2024-03-29T08:18:14.324569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:56.709840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324644+0000","last_change":"2024-03-29T08:18:13.357731+0000","last_active":"2024-03-29T08:18:14.324644+0000","last_peered":"2024-03-29T08:18:14.324644+0000","last_clean":"2024-03-29T08:18:14.324644+0000","last_became_active":"2024-03-29T08:18:13.357437+0000","last_became_peered":"2024-03-29T08:18:13.357437+0000","last_unstale":"2024-03-29T08:18:14.324644+0000","last_undegraded":"2024-03-29T08:18:14.324644+0000","last_fullsized":"2024-03-29T08:18:14.324644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:10.712191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.355358+0000","last_change":"2024-03-29T08:18:12.942493+0000","last_active":"2024-03-29T08:18:13.355358+0000","last_peered":"2024-03-29T08:18:13.355358+0000","last_clean":"2024-03-29T08:18:13.355358+0000","last_became_active":"2024-03-29T08:18:12.942116+0000","last_became_peered":"2024-03-29T08:18:12.942116+0000","last_unstale":"2024-03-29T08:18:13.355358+0000","last_undegraded":"2024-03-29T08:18:13.355358+0000","last_fullsized":"2024-03-29T08:18:13.355358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:55.791360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324502+0000","last_change":"2024-03-29T08:18:13.343380+0000","last_active":"2024-03-29T08:18:14.324502+0000","last_peered":"2024-03-29T08:18:14.324502+0000","last_clean":"2024-03-29T08:18:14.324502+0000","last_became_active":"2024-03-29T08:18:13.343121+0000","last_became_peered":"2024-03-29T08:18:13.343121+0000","last_unstale":"2024-03-29T08:18:14.324502+0000","last_undegraded":"2024-03-29T08:18:14.324502+0000","last_fullsized":"2024-03-29T08:18:14.324502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:52:27.461296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.315882+0000","last_change":"2024-03-29T08:18:12.935438+0000","last_active":"2024-03-29T08:18:13.315882+0000","last_peered":"2024-03-29T08:18:13.315882+0000","last_clean":"2024-03-29T08:18:13.315882+0000","last_became_active":"2024-03-29T08:18:12.935170+0000","last_became_peered":"2024-03-29T08:18:12.935170+0000","last_unstale":"2024-03-29T08:18:13.315882+0000","last_undegraded":"2024-03-29T08:18:13.315882+0000","last_fullsized":"2024-03-29T08:18:13.315882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:53:56.604571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.351291+0000","last_change":"2024-03-29T08:18:13.341595+0000","last_active":"2024-03-29T08:18:13.351291+0000","last_peered":"2024-03-29T08:18:13.351291+0000","last_clean":"2024-03-29T08:18:13.351291+0000","last_became_active":"2024-03-29T08:18:13.341387+0000","last_became_peered":"2024-03-29T08:18:13.341387+0000","last_unstale":"2024-03-29T08:18:13.351291+0000","last_undegraded":"2024-03-29T08:18:13.351291+0000","last_fullsized":"2024-03-29T08:18:13.351291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:37:35.410925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324393+0000","last_change":"2024-03-29T08:18:12.922897+0000","last_active":"2024-03-29T08:18:14.324393+0000","last_peered":"2024-03-29T08:18:14.324393+0000","last_clean":"2024-03-29T08:18:14.324393+0000","last_became_active":"2024-03-29T08:18:12.922583+0000","last_became_peered":"2024-03-29T08:18:12.922583+0000","last_unstale":"2024-03-29T08:18:14.324393+0000","last_undegraded":"2024-03-29T08:18:14.324393+0000","last_fullsized":"2024-03-29T08:18:14.324393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:24.943478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324257+0000","last_change":"2024-03-29T08:18:13.344357+0000","last_active":"2024-03-29T08:18:14.324257+0000","last_peered":"2024-03-29T08:18:14.324257+0000","last_clean":"2024-03-29T08:18:14.324257+0000","last_became_active":"2024-03-29T08:18:13.343725+0000","last_became_peered":"2024-03-29T08:18:13.343725+0000","last_unstale":"2024-03-29T08:18:14.324257+0000","last_undegraded":"2024-03-29T08:18:14.324257+0000","last_fullsized":"2024-03-29T08:18:14.324257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:56.798095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.339735+0000","last_change":"2024-03-29T08:18:12.930747+0000","last_active":"2024-03-29T08:18:13.339735+0000","last_peered":"2024-03-29T08:18:13.339735+0000","last_clean":"2024-03-29T08:18:13.339735+0000","last_became_active":"2024-03-29T08:18:12.930527+0000","last_became_peered":"2024-03-29T08:18:12.930527+0000","last_unstale":"2024-03-29T08:18:13.339735+0000","last_undegraded":"2024-03-29T08:18:13.339735+0000","last_fullsized":"2024-03-29T08:18:13.339735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:55:09.380677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323459+0000","last_change":"2024-03-29T08:18:13.359008+0000","last_active":"2024-03-29T08:18:14.323459+0000","last_peered":"2024-03-29T08:18:14.323459+0000","last_clean":"2024-03-29T08:18:14.323459+0000","last_became_active":"2024-03-29T08:18:13.358733+0000","last_became_peered":"2024-03-29T08:18:13.358733+0000","last_unstale":"2024-03-29T08:18:14.323459+0000","last_undegraded":"2024-03-29T08:18:14.323459+0000","last_fullsized":"2024-03-29T08:18:14.323459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:45.475256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.353985+0000","last_change":"2024-03-29T08:18:12.932074+0000","last_active":"2024-03-29T08:18:13.353985+0000","last_peered":"2024-03-29T08:18:13.353985+0000","last_clean":"2024-03-29T08:18:13.353985+0000","last_became_active":"2024-03-29T08:18:12.931603+0000","last_became_peered":"2024-03-29T08:18:12.931603+0000","last_unstale":"2024-03-29T08:18:13.353985+0000","last_undegraded":"2024-03-29T08:18:13.353985+0000","last_fullsized":"2024-03-29T08:18:13.353985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:31.317829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.338388+0000","last_change":"2024-03-29T08:18:13.331615+0000","last_active":"2024-03-29T08:18:13.338388+0000","last_peered":"2024-03-29T08:18:13.338388+0000","last_clean":"2024-03-29T08:18:13.338388+0000","last_became_active":"2024-03-29T08:18:13.331399+0000","last_became_peered":"2024-03-29T08:18:13.331399+0000","last_unstale":"2024-03-29T08:18:13.338388+0000","last_undegraded":"2024-03-29T08:18:13.338388+0000","last_fullsized":"2024-03-29T08:18:13.338388+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:49.457858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.342842+0000","last_change":"2024-03-29T08:18:13.339347+0000","last_active":"2024-03-29T08:18:13.342842+0000","last_peered":"2024-03-29T08:18:13.342842+0000","last_clean":"2024-03-29T08:18:13.342842+0000","last_became_active":"2024-03-29T08:18:13.338931+0000","last_became_peered":"2024-03-29T08:18:13.338931+0000","last_unstale":"2024-03-29T08:18:13.342842+0000","last_undegraded":"2024-03-29T08:18:13.342842+0000","last_fullsized":"2024-03-29T08:18:13.342842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:06.953763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.344654+0000","last_change":"2024-03-29T08:18:12.930282+0000","last_active":"2024-03-29T08:18:13.344654+0000","last_peered":"2024-03-29T08:18:13.344654+0000","last_clean":"2024-03-29T08:18:13.344654+0000","last_became_active":"2024-03-29T08:18:12.929883+0000","last_became_peered":"2024-03-29T08:18:12.929883+0000","last_unstale":"2024-03-29T08:18:13.344654+0000","last_undegraded":"2024-03-29T08:18:13.344654+0000","last_fullsized":"2024-03-29T08:18:13.344654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:30:10.121158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321265+0000","last_change":"2024-03-29T08:18:13.335469+0000","last_active":"2024-03-29T08:18:14.321265+0000","last_peered":"2024-03-29T08:18:14.321265+0000","last_clean":"2024-03-29T08:18:14.321265+0000","last_became_active":"2024-03-29T08:18:13.335195+0000","last_became_peered":"2024-03-29T08:18:13.335195+0000","last_unstale":"2024-03-29T08:18:14.321265+0000","last_undegraded":"2024-03-29T08:18:14.321265+0000","last_fullsized":"2024-03-29T08:18:14.321265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:53:57.611306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.338461+0000","last_change":"2024-03-29T08:18:12.922889+0000","last_active":"2024-03-29T08:18:13.338461+0000","last_peered":"2024-03-29T08:18:13.338461+0000","last_clean":"2024-03-29T08:18:13.338461+0000","last_became_active":"2024-03-29T08:18:12.921734+0000","last_became_peered":"2024-03-29T08:18:12.921734+0000","last_unstale":"2024-03-29T08:18:13.338461+0000","last_undegraded":"2024-03-29T08:18:13.338461+0000","last_fullsized":"2024-03-29T08:18:13.338461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:06.406132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.346355+0000","last_change":"2024-03-29T08:18:13.341340+0000","last_active":"2024-03-29T08:18:13.346355+0000","last_peered":"2024-03-29T08:18:13.346355+0000","last_clean":"2024-03-29T08:18:13.346355+0000","last_became_active":"2024-03-29T08:18:13.341062+0000","last_became_peered":"2024-03-29T08:18:13.341062+0000","last_unstale":"2024-03-29T08:18:13.346355+0000","last_undegraded":"2024-03-29T08:18:13.346355+0000","last_fullsized":"2024-03-29T08:18:13.346355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:54:33.899398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.348466+0000","last_change":"2024-03-29T08:18:12.928434+0000","last_active":"2024-03-29T08:18:13.348466+0000","last_peered":"2024-03-29T08:18:13.348466+0000","last_clean":"2024-03-29T08:18:13.348466+0000","last_became_active":"2024-03-29T08:18:12.928109+0000","last_became_peered":"2024-03-29T08:18:12.928109+0000","last_unstale":"2024-03-29T08:18:13.348466+0000","last_undegraded":"2024-03-29T08:18:13.348466+0000","last_fullsized":"2024-03-29T08:18:13.348466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:25:41.738901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.345342+0000","last_change":"2024-03-29T08:18:13.337191+0000","last_active":"2024-03-29T08:18:13.345342+0000","last_peered":"2024-03-29T08:18:13.345342+0000","last_clean":"2024-03-29T08:18:13.345342+0000","last_became_active":"2024-03-29T08:18:13.336967+0000","last_became_peered":"2024-03-29T08:18:13.336967+0000","last_unstale":"2024-03-29T08:18:13.345342+0000","last_undegraded":"2024-03-29T08:18:13.345342+0000","last_fullsized":"2024-03-29T08:18:13.345342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:40.987756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324681+0000","last_change":"2024-03-29T08:18:12.937840+0000","last_active":"2024-03-29T08:18:14.324681+0000","last_peered":"2024-03-29T08:18:14.324681+0000","last_clean":"2024-03-29T08:18:14.324681+0000","last_became_active":"2024-03-29T08:18:12.937403+0000","last_became_peered":"2024-03-29T08:18:12.937403+0000","last_unstale":"2024-03-29T08:18:14.324681+0000","last_undegraded":"2024-03-29T08:18:14.324681+0000","last_fullsized":"2024-03-29T08:18:14.324681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:58.083723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":70064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":69940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]}]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":78476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31320,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340520,"statfs":{"total":96636764160,"available":96604692480,"internally_reserved":0,"allocated":212992,"data_stored":70522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":221184,"data_stored":71504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":819200,"data_stored":666678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":217088,"data_stored":69996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":653533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:18:17.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:18:17.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.604+0000 7f8e9f200640 1 Processor -- start 2024-03-29T08:18:17.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.604+0000 7f8e9f200640 1 -- start start 2024-03-29T08:18:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e9f200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e980ce490 0x7f8e9815ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e9f200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e9815b370 con 0x7f8e98099790 2024-03-29T08:18:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e9f200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e9815b650 con 0x7f8e98098a40 2024-03-29T08:18:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e9f200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9815b930 con 0x7f8e980ce490 2024-03-29T08:18:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e9d200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8e98099790 0x7f8e980cb230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60852/0 (socket says 172.21.15.46:60852) 2024-03-29T08:18:17.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e9d200640 1 -- 172.21.15.46:0/2693910146 learned_addr learned my addr 172.21.15.46:0/2693910146 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:17.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e8f400640 1 --2- 172.21.15.46:0/2693910146 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e980ce490 0x7f8e9815ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:17.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e97400640 1 -- 172.21.15.46:0/2693910146 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1503941749 0 0) 0x7f8e9815b370 con 0x7f8e98099790 2024-03-29T08:18:17.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e97400640 1 -- 172.21.15.46:0/2693910146 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e7c003530 con 0x7f8e98099790 2024-03-29T08:18:17.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e97400640 1 -- 172.21.15.46:0/2693910146 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4095745199 0 0) 0x7f8e7c003530 con 0x7f8e98099790 2024-03-29T08:18:17.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e97400640 1 -- 172.21.15.46:0/2693910146 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e980ce490 msgr2=0x7f8e9815ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:17.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e97400640 1 --2- 172.21.15.46:0/2693910146 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e980ce490 0x7f8e9815ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e97400640 1 -- 172.21.15.46:0/2693910146 >> v1:172.21.15.73:6789/0 conn(0x7f8e98098a40 legacy=0x7f8e98098e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:17.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e97400640 1 -- 172.21.15.46:0/2693910146 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9815bc10 con 0x7f8e98099790 2024-03-29T08:18:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e97400640 1 -- 172.21.15.46:0/2693910146 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f8e90001a00 con 0x7f8e98099790 2024-03-29T08:18:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e97400640 1 -- 172.21.15.46:0/2693910146 >> v1:172.21.15.46:6789/0 conn(0x7f8e98099790 legacy=0x7f8e980cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e97400640 1 --2- 172.21.15.46:0/2693910146 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e980ce490 0x7f8e7c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e97400640 1 --2- 172.21.15.46:0/2693910146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e7c003f80 0x7f8e7c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e97400640 1 --2- 172.21.15.46:0/2693910146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e7c004d10 0x7f8e7c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e97400640 1 -- 172.21.15.46:0/2693910146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9815b930 con 0x7f8e7c004d10 2024-03-29T08:18:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e97400640 1 -- 172.21.15.46:0/2693910146 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c00bad0 con 0x7f8e7c003f80 2024-03-29T08:18:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e97400640 1 -- 172.21.15.46:0/2693910146 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c00bc90 con 0x7f8e980ce490 2024-03-29T08:18:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e9d200640 1 --2- 172.21.15.46:0/2693910146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e7c004d10 0x7f8e7c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e97e00640 1 --2- 172.21.15.46:0/2693910146 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e980ce490 0x7f8e7c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e8f400640 1 --2- 172.21.15.46:0/2693910146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e7c003f80 0x7f8e7c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e9d200640 1 -- 172.21.15.46:0/2693910146 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e980ce490 msgr2=0x7f8e7c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e9d200640 1 --2- 172.21.15.46:0/2693910146 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e980ce490 0x7f8e7c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e9d200640 1 -- 172.21.15.46:0/2693910146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e7c003f80 msgr2=0x7f8e7c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e9d200640 1 --2- 172.21.15.46:0/2693910146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e7c003f80 0x7f8e7c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.608+0000 7f8e9d200640 1 -- 172.21.15.46:0/2693910146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c003530 con 0x7f8e7c004d10 2024-03-29T08:18:17.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.612+0000 7f8e9d200640 1 --2- 172.21.15.46:0/2693910146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e7c004d10 0x7f8e7c005190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000a760 tx=0x7f8e9001d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2401497590f704b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:17.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.612+0000 7f8e97400640 1 -- 172.21.15.46:0/2693910146 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e900ad630 con 0x7f8e7c004d10 2024-03-29T08:18:17.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.612+0000 7f8e97400640 1 -- 172.21.15.46:0/2693910146 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e900a8070 con 0x7f8e7c004d10 2024-03-29T08:18:17.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.612+0000 7f8e97400640 1 -- 172.21.15.46:0/2693910146 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e900ada50 con 0x7f8e7c004d10 2024-03-29T08:18:17.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.612+0000 7f8e9f200640 1 -- 172.21.15.46:0/2693910146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e7c004d10 msgr2=0x7f8e7c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:17.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.612+0000 7f8e9f200640 1 --2- 172.21.15.46:0/2693910146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e7c004d10 0x7f8e7c005190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8e9000a760 tx=0x7f8e9001d280 comp rx=0 tx=0).stop 2024-03-29T08:18:17.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.612+0000 7f8e9d200640 1 -- 172.21.15.46:0/2693910146 reap_dead start 2024-03-29T08:18:17.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.612+0000 7f8e9f200640 1 -- 172.21.15.46:0/2693910146 shutdown_connections 2024-03-29T08:18:17.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.612+0000 7f8e9f200640 1 -- 172.21.15.46:0/2693910146 >> 172.21.15.46:0/2693910146 conn(0x7f8e980b21a0 msgr2=0x7f8e980b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:17.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.612+0000 7f8e9f200640 1 -- 172.21.15.46:0/2693910146 shutdown_connections 2024-03-29T08:18:17.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.612+0000 7f8e9f200640 1 -- 172.21.15.46:0/2693910146 wait complete. 2024-03-29T08:18:17.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.612+0000 7f8e9f200640 1 Processor -- start 2024-03-29T08:18:17.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.612+0000 7f8e9f200640 1 -- start start 2024-03-29T08:18:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e9f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e7c004d10 0x7f8e98158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e9f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e980c3080 0x7f8e980ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:17.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e9f200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e980c80b0 0x7f8e980c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e9f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e980c8a70 con 0x7f8e980c3080 2024-03-29T08:18:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e9f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9806ba30 con 0x7f8e7c004d10 2024-03-29T08:18:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e9f200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9806bd10 con 0x7f8e980c80b0 2024-03-29T08:18:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e8f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e980c3080 0x7f8e980ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e8f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e980c3080 0x7f8e980ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59726/0 (socket says 172.21.15.46:59726) 2024-03-29T08:18:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e8f400640 1 -- 172.21.15.46:0/2693641340 learned_addr learned my addr 172.21.15.46:0/2693641340 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e97e00640 1 --2- 172.21.15.46:0/2693641340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e7c004d10 0x7f8e98158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e9d200640 1 --2- 172.21.15.46:0/2693641340 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e980c80b0 0x7f8e980c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e8f400640 1 -- 172.21.15.46:0/2693641340 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e980c80b0 msgr2=0x7f8e980c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e8f400640 1 --2- 172.21.15.46:0/2693641340 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e980c80b0 0x7f8e980c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e8f400640 1 -- 172.21.15.46:0/2693641340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e7c004d10 msgr2=0x7f8e98158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e8f400640 1 --2- 172.21.15.46:0/2693641340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e7c004d10 0x7f8e98158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e8f400640 1 -- 172.21.15.46:0/2693641340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9806bff0 con 0x7f8e980c3080 2024-03-29T08:18:17.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e8f400640 1 --2- 172.21.15.46:0/2693641340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e980c3080 0x7f8e980ca0d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8e800119d0 tx=0x7f8e80011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:17.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e94c00640 1 -- 172.21.15.46:0/2693641340 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8000faf0 con 0x7f8e980c3080 2024-03-29T08:18:17.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e94c00640 1 -- 172.21.15.46:0/2693641340 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8000fc50 con 0x7f8e980c3080 2024-03-29T08:18:17.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e9f200640 1 -- 172.21.15.46:0/2693641340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e9806c2d0 con 0x7f8e980c3080 2024-03-29T08:18:17.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e94c00640 1 -- 172.21.15.46:0/2693641340 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e80014440 con 0x7f8e980c3080 2024-03-29T08:18:17.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e9f200640 1 -- 172.21.15.46:0/2693641340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e98159390 con 0x7f8e980c3080 2024-03-29T08:18:17.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e9f200640 1 -- 172.21.15.46:0/2693641340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e98066c90 con 0x7f8e980c3080 2024-03-29T08:18:17.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.616+0000 7f8e94c00640 1 -- 172.21.15.46:0/2693641340 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8e800145a0 con 0x7f8e980c3080 2024-03-29T08:18:17.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.620+0000 7f8e94c00640 1 --2- 172.21.15.46:0/2693641340 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8e74066690 0x7f8e74068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:17.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.620+0000 7f8e94c00640 1 -- 172.21.15.46:0/2693641340 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e80084a80 con 0x7f8e980c3080 2024-03-29T08:18:17.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.620+0000 7f8e97e00640 1 --2- 172.21.15.46:0/2693641340 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8e74066690 0x7f8e74068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:17.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.620+0000 7f8e97e00640 1 --2- 172.21.15.46:0/2693641340 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8e74066690 0x7f8e74068b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8e88007fd0 tx=0x7f8e88009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:17.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.624+0000 7f8e94c00640 1 -- 172.21.15.46:0/2693641340 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8e80051e10 con 0x7f8e980c3080 2024-03-29T08:18:17.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.780+0000 7f8e9f200640 1 -- 172.21.15.46:0/2693641340 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e980ccaa0 con 0x7f8e74066690 2024-03-29T08:18:17.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.788+0000 7f8e94c00640 1 -- 172.21.15.46:0/2693641340 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264619 (secure 0 0 0) 0x7f8e980ccaa0 con 0x7f8e74066690 2024-03-29T08:18:17.793 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:17.794 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:18:17.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.792+0000 7f8e9f200640 1 -- 172.21.15.46:0/2693641340 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8e74066690 msgr2=0x7f8e74068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:17.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.792+0000 7f8e9f200640 1 --2- 172.21.15.46:0/2693641340 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8e74066690 0x7f8e74068b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8e88007fd0 tx=0x7f8e88009040 comp rx=0 tx=0).stop 2024-03-29T08:18:17.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.792+0000 7f8e9f200640 1 -- 172.21.15.46:0/2693641340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e980c3080 msgr2=0x7f8e980ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:17.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.792+0000 7f8e9f200640 1 --2- 172.21.15.46:0/2693641340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e980c3080 0x7f8e980ca0d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8e800119d0 tx=0x7f8e80011e90 comp rx=0 tx=0).stop 2024-03-29T08:18:17.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.792+0000 7f8e9f200640 1 -- 172.21.15.46:0/2693641340 shutdown_connections 2024-03-29T08:18:17.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.792+0000 7f8e9f200640 1 --2- 172.21.15.46:0/2693641340 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8e74066690 0x7f8e74068b50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.792+0000 7f8e9f200640 1 --2- 172.21.15.46:0/2693641340 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e980c80b0 0x7f8e980c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.792+0000 7f8e9f200640 1 --2- 172.21.15.46:0/2693641340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e980c3080 0x7f8e980ca0d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.792+0000 7f8e9f200640 1 --2- 172.21.15.46:0/2693641340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e7c004d10 0x7f8e98158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:17.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.792+0000 7f8e9f200640 1 -- 172.21.15.46:0/2693641340 >> 172.21.15.46:0/2693641340 conn(0x7f8e980b21a0 msgr2=0x7f8e9815c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:17.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.792+0000 7f8e9f200640 1 -- 172.21.15.46:0/2693641340 shutdown_connections 2024-03-29T08:18:17.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.792+0000 7f8e9f200640 1 -- 172.21.15.46:0/2693641340 wait complete. 2024-03-29T08:18:17.812 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-29T08:18:16.042327+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":65,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235428,"kb_used_data":2860,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739292,"statfs":{"total":773094113280,"available":772853035008,"internally_reserved":0,"allocated":2928640,"data_stored":1750713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.143117"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.348671+0000","last_change":"2024-03-29T08:18:13.347398+0000","last_active":"2024-03-29T08:18:13.348671+0000","last_peered":"2024-03-29T08:18:13.348671+0000","last_clean":"2024-03-29T08:18:13.348671+0000","last_became_active":"2024-03-29T08:18:13.346857+0000","last_became_peered":"2024-03-29T08:18:13.346857+0000","last_unstale":"2024-03-29T08:18:13.348671+0000","last_undegraded":"2024-03-29T08:18:13.348671+0000","last_fullsized":"2024-03-29T08:18:13.348671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:03.590543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.361542+0000","last_change":"2024-03-29T08:18:13.358700+0000","last_active":"2024-03-29T08:18:13.361542+0000","last_peered":"2024-03-29T08:18:13.361542+0000","last_clean":"2024-03-29T08:18:13.361542+0000","last_became_active":"2024-03-29T08:18:13.357646+0000","last_became_peered":"2024-03-29T08:18:13.357646+0000","last_unstale":"2024-03-29T08:18:13.361542+0000","last_undegraded":"2024-03-29T08:18:13.361542+0000","last_fullsized":"2024-03-29T08:18:13.361542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:20.587134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.355371+0000","last_change":"2024-03-29T08:18:13.352326+0000","last_active":"2024-03-29T08:18:13.355371+0000","last_peered":"2024-03-29T08:18:13.355371+0000","last_clean":"2024-03-29T08:18:13.355371+0000","last_became_active":"2024-03-29T08:18:13.351954+0000","last_became_peered":"2024-03-29T08:18:13.351954+0000","last_unstale":"2024-03-29T08:18:13.355371+0000","last_undegraded":"2024-03-29T08:18:13.355371+0000","last_fullsized":"2024-03-29T08:18:13.355371+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:07:42.013921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089100+0000","last_change":"2024-03-29T08:18:13.336578+0000","last_active":"2024-03-29T08:18:15.089100+0000","last_peered":"2024-03-29T08:18:15.089100+0000","last_clean":"2024-03-29T08:18:15.089100+0000","last_became_active":"2024-03-29T08:18:13.336344+0000","last_became_peered":"2024-03-29T08:18:13.336344+0000","last_unstale":"2024-03-29T08:18:15.089100+0000","last_undegraded":"2024-03-29T08:18:15.089100+0000","last_fullsized":"2024-03-29T08:18:15.089100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:30.625311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.350069+0000","last_change":"2024-03-29T08:18:13.348320+0000","last_active":"2024-03-29T08:18:13.350069+0000","last_peered":"2024-03-29T08:18:13.350069+0000","last_clean":"2024-03-29T08:18:13.350069+0000","last_became_active":"2024-03-29T08:18:13.345863+0000","last_became_peered":"2024-03-29T08:18:13.345863+0000","last_unstale":"2024-03-29T08:18:13.350069+0000","last_undegraded":"2024-03-29T08:18:13.350069+0000","last_fullsized":"2024-03-29T08:18:13.350069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:26:51.855915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.088872+0000","last_change":"2024-03-29T08:18:13.345374+0000","last_active":"2024-03-29T08:18:15.088872+0000","last_peered":"2024-03-29T08:18:15.088872+0000","last_clean":"2024-03-29T08:18:15.088872+0000","last_became_active":"2024-03-29T08:18:13.345054+0000","last_became_peered":"2024-03-29T08:18:13.345054+0000","last_unstale":"2024-03-29T08:18:15.088872+0000","last_undegraded":"2024-03-29T08:18:15.088872+0000","last_fullsized":"2024-03-29T08:18:15.088872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:32.324564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.349140+0000","last_change":"2024-03-29T08:18:13.346637+0000","last_active":"2024-03-29T08:18:13.349140+0000","last_peered":"2024-03-29T08:18:13.349140+0000","last_clean":"2024-03-29T08:18:13.349140+0000","last_became_active":"2024-03-29T08:18:13.346335+0000","last_became_peered":"2024-03-29T08:18:13.346335+0000","last_unstale":"2024-03-29T08:18:13.349140+0000","last_undegraded":"2024-03-29T08:18:13.349140+0000","last_fullsized":"2024-03-29T08:18:13.349140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:23.267817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.353926+0000","last_change":"2024-03-29T08:18:13.348979+0000","last_active":"2024-03-29T08:18:13.353926+0000","last_peered":"2024-03-29T08:18:13.353926+0000","last_clean":"2024-03-29T08:18:13.353926+0000","last_became_active":"2024-03-29T08:18:13.348682+0000","last_became_peered":"2024-03-29T08:18:13.348682+0000","last_unstale":"2024-03-29T08:18:13.353926+0000","last_undegraded":"2024-03-29T08:18:13.353926+0000","last_fullsized":"2024-03-29T08:18:13.353926+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:53:21.565448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321443+0000","last_change":"2024-03-29T08:18:13.339556+0000","last_active":"2024-03-29T08:18:14.321443+0000","last_peered":"2024-03-29T08:18:14.321443+0000","last_clean":"2024-03-29T08:18:14.321443+0000","last_became_active":"2024-03-29T08:18:13.339318+0000","last_became_peered":"2024-03-29T08:18:13.339318+0000","last_unstale":"2024-03-29T08:18:14.321443+0000","last_undegraded":"2024-03-29T08:18:14.321443+0000","last_fullsized":"2024-03-29T08:18:14.321443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:45.475256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.343185+0000","last_change":"2024-03-29T08:18:13.337238+0000","last_active":"2024-03-29T08:18:13.343185+0000","last_peered":"2024-03-29T08:18:13.343185+0000","last_clean":"2024-03-29T08:18:13.343185+0000","last_became_active":"2024-03-29T08:18:13.336549+0000","last_became_peered":"2024-03-29T08:18:13.336549+0000","last_unstale":"2024-03-29T08:18:13.343185+0000","last_undegraded":"2024-03-29T08:18:13.343185+0000","last_fullsized":"2024-03-29T08:18:13.343185+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:32:48.313814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.348043+0000","last_change":"2024-03-29T08:18:13.346353+0000","last_active":"2024-03-29T08:18:13.348043+0000","last_peered":"2024-03-29T08:18:13.348043+0000","last_clean":"2024-03-29T08:18:13.348043+0000","last_became_active":"2024-03-29T08:18:13.345780+0000","last_became_peered":"2024-03-29T08:18:13.345780+0000","last_unstale":"2024-03-29T08:18:13.348043+0000","last_undegraded":"2024-03-29T08:18:13.348043+0000","last_fullsized":"2024-03-29T08:18:13.348043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:45:55.900052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.339429+0000","last_change":"2024-03-29T08:18:13.331322+0000","last_active":"2024-03-29T08:18:13.339429+0000","last_peered":"2024-03-29T08:18:13.339429+0000","last_clean":"2024-03-29T08:18:13.339429+0000","last_became_active":"2024-03-29T08:18:13.331047+0000","last_became_peered":"2024-03-29T08:18:13.331047+0000","last_unstale":"2024-03-29T08:18:13.339429+0000","last_undegraded":"2024-03-29T08:18:13.339429+0000","last_fullsized":"2024-03-29T08:18:13.339429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:40.888705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.364236+0000","last_change":"2024-03-29T08:18:13.357625+0000","last_active":"2024-03-29T08:18:13.364236+0000","last_peered":"2024-03-29T08:18:13.364236+0000","last_clean":"2024-03-29T08:18:13.364236+0000","last_became_active":"2024-03-29T08:18:13.356826+0000","last_became_peered":"2024-03-29T08:18:13.356826+0000","last_unstale":"2024-03-29T08:18:13.364236+0000","last_undegraded":"2024-03-29T08:18:13.364236+0000","last_fullsized":"2024-03-29T08:18:13.364236+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:30.625311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.364053+0000","last_change":"2024-03-29T08:18:13.356033+0000","last_active":"2024-03-29T08:18:13.364053+0000","last_peered":"2024-03-29T08:18:13.364053+0000","last_clean":"2024-03-29T08:18:13.364053+0000","last_became_active":"2024-03-29T08:18:13.355720+0000","last_became_peered":"2024-03-29T08:18:13.355720+0000","last_unstale":"2024-03-29T08:18:13.364053+0000","last_undegraded":"2024-03-29T08:18:13.364053+0000","last_fullsized":"2024-03-29T08:18:13.364053+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:56.798095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.353969+0000","last_change":"2024-03-29T08:18:13.349080+0000","last_active":"2024-03-29T08:18:13.353969+0000","last_peered":"2024-03-29T08:18:13.353969+0000","last_clean":"2024-03-29T08:18:13.353969+0000","last_became_active":"2024-03-29T08:18:13.348721+0000","last_became_peered":"2024-03-29T08:18:13.348721+0000","last_unstale":"2024-03-29T08:18:13.353969+0000","last_undegraded":"2024-03-29T08:18:13.353969+0000","last_fullsized":"2024-03-29T08:18:13.353969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:54.605777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.354577+0000","last_change":"2024-03-29T08:18:13.349631+0000","last_active":"2024-03-29T08:18:13.354577+0000","last_peered":"2024-03-29T08:18:13.354577+0000","last_clean":"2024-03-29T08:18:13.354577+0000","last_became_active":"2024-03-29T08:18:13.349377+0000","last_became_peered":"2024-03-29T08:18:13.349377+0000","last_unstale":"2024-03-29T08:18:13.354577+0000","last_undegraded":"2024-03-29T08:18:13.354577+0000","last_fullsized":"2024-03-29T08:18:13.354577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:44:22.135683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.354546+0000","last_change":"2024-03-29T08:18:13.342594+0000","last_active":"2024-03-29T08:18:13.354546+0000","last_peered":"2024-03-29T08:18:13.354546+0000","last_clean":"2024-03-29T08:18:13.354546+0000","last_became_active":"2024-03-29T08:18:13.341973+0000","last_became_peered":"2024-03-29T08:18:13.341973+0000","last_unstale":"2024-03-29T08:18:13.354546+0000","last_undegraded":"2024-03-29T08:18:13.354546+0000","last_fullsized":"2024-03-29T08:18:13.354546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:31:39.118083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323494+0000","last_change":"2024-03-29T08:18:13.349000+0000","last_active":"2024-03-29T08:18:14.323494+0000","last_peered":"2024-03-29T08:18:14.323494+0000","last_clean":"2024-03-29T08:18:14.323494+0000","last_became_active":"2024-03-29T08:18:13.348586+0000","last_became_peered":"2024-03-29T08:18:13.348586+0000","last_unstale":"2024-03-29T08:18:14.323494+0000","last_undegraded":"2024-03-29T08:18:14.323494+0000","last_fullsized":"2024-03-29T08:18:14.323494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:55:41.020683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.354616+0000","last_change":"2024-03-29T08:18:13.342652+0000","last_active":"2024-03-29T08:18:13.354616+0000","last_peered":"2024-03-29T08:18:13.354616+0000","last_clean":"2024-03-29T08:18:13.354616+0000","last_became_active":"2024-03-29T08:18:13.342339+0000","last_became_peered":"2024-03-29T08:18:13.342339+0000","last_unstale":"2024-03-29T08:18:13.354616+0000","last_undegraded":"2024-03-29T08:18:13.354616+0000","last_fullsized":"2024-03-29T08:18:13.354616+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:46:35.075911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.332696+0000","last_change":"2024-03-29T08:18:13.325928+0000","last_active":"2024-03-29T08:18:13.332696+0000","last_peered":"2024-03-29T08:18:13.332696+0000","last_clean":"2024-03-29T08:18:13.332696+0000","last_became_active":"2024-03-29T08:18:13.325659+0000","last_became_peered":"2024-03-29T08:18:13.325659+0000","last_unstale":"2024-03-29T08:18:13.332696+0000","last_undegraded":"2024-03-29T08:18:13.332696+0000","last_fullsized":"2024-03-29T08:18:13.332696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:58.411155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324285+0000","last_change":"2024-03-29T08:18:13.339244+0000","last_active":"2024-03-29T08:18:14.324285+0000","last_peered":"2024-03-29T08:18:14.324285+0000","last_clean":"2024-03-29T08:18:14.324285+0000","last_became_active":"2024-03-29T08:18:13.338984+0000","last_became_peered":"2024-03-29T08:18:13.338984+0000","last_unstale":"2024-03-29T08:18:14.324285+0000","last_undegraded":"2024-03-29T08:18:14.324285+0000","last_fullsized":"2024-03-29T08:18:14.324285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:29:09.699201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321134+0000","last_change":"2024-03-29T08:18:13.330207+0000","last_active":"2024-03-29T08:18:14.321134+0000","last_peered":"2024-03-29T08:18:14.321134+0000","last_clean":"2024-03-29T08:18:14.321134+0000","last_became_active":"2024-03-29T08:18:13.330029+0000","last_became_peered":"2024-03-29T08:18:13.330029+0000","last_unstale":"2024-03-29T08:18:14.321134+0000","last_undegraded":"2024-03-29T08:18:14.321134+0000","last_fullsized":"2024-03-29T08:18:14.321134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:32.324564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.353573+0000","last_change":"2024-03-29T08:18:13.347985+0000","last_active":"2024-03-29T08:18:13.353573+0000","last_peered":"2024-03-29T08:18:13.353573+0000","last_clean":"2024-03-29T08:18:13.353573+0000","last_became_active":"2024-03-29T08:18:13.347561+0000","last_became_peered":"2024-03-29T08:18:13.347561+0000","last_unstale":"2024-03-29T08:18:13.353573+0000","last_undegraded":"2024-03-29T08:18:13.353573+0000","last_fullsized":"2024-03-29T08:18:13.353573+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:17.775287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.340909+0000","last_change":"2024-03-29T08:18:13.338212+0000","last_active":"2024-03-29T08:18:13.340909+0000","last_peered":"2024-03-29T08:18:13.340909+0000","last_clean":"2024-03-29T08:18:13.340909+0000","last_became_active":"2024-03-29T08:18:13.337908+0000","last_became_peered":"2024-03-29T08:18:13.337908+0000","last_unstale":"2024-03-29T08:18:13.340909+0000","last_undegraded":"2024-03-29T08:18:13.340909+0000","last_fullsized":"2024-03-29T08:18:13.340909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:40.888705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.346404+0000","last_change":"2024-03-29T08:18:13.341534+0000","last_active":"2024-03-29T08:18:13.346404+0000","last_peered":"2024-03-29T08:18:13.346404+0000","last_clean":"2024-03-29T08:18:13.346404+0000","last_became_active":"2024-03-29T08:18:13.341097+0000","last_became_peered":"2024-03-29T08:18:13.341097+0000","last_unstale":"2024-03-29T08:18:13.346404+0000","last_undegraded":"2024-03-29T08:18:13.346404+0000","last_fullsized":"2024-03-29T08:18:13.346404+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:05:16.874804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089257+0000","last_change":"2024-03-29T08:18:13.342465+0000","last_active":"2024-03-29T08:18:15.089257+0000","last_peered":"2024-03-29T08:18:15.089257+0000","last_clean":"2024-03-29T08:18:15.089257+0000","last_became_active":"2024-03-29T08:18:13.342000+0000","last_became_peered":"2024-03-29T08:18:13.342000+0000","last_unstale":"2024-03-29T08:18:15.089257+0000","last_undegraded":"2024-03-29T08:18:15.089257+0000","last_fullsized":"2024-03-29T08:18:15.089257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:05.352866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.364081+0000","last_change":"2024-03-29T08:18:13.356380+0000","last_active":"2024-03-29T08:18:13.364081+0000","last_peered":"2024-03-29T08:18:13.364081+0000","last_clean":"2024-03-29T08:18:13.364081+0000","last_became_active":"2024-03-29T08:18:13.356016+0000","last_became_peered":"2024-03-29T08:18:13.356016+0000","last_unstale":"2024-03-29T08:18:13.364081+0000","last_undegraded":"2024-03-29T08:18:13.364081+0000","last_fullsized":"2024-03-29T08:18:13.364081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:05.352866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.364192+0000","last_change":"2024-03-29T08:18:13.357406+0000","last_active":"2024-03-29T08:18:13.364192+0000","last_peered":"2024-03-29T08:18:13.364192+0000","last_clean":"2024-03-29T08:18:13.364192+0000","last_became_active":"2024-03-29T08:18:13.356875+0000","last_became_peered":"2024-03-29T08:18:13.356875+0000","last_unstale":"2024-03-29T08:18:13.364192+0000","last_undegraded":"2024-03-29T08:18:13.364192+0000","last_fullsized":"2024-03-29T08:18:13.364192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:52:27.461296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.345107+0000","last_change":"2024-03-29T08:18:13.338239+0000","last_active":"2024-03-29T08:18:13.345107+0000","last_peered":"2024-03-29T08:18:13.345107+0000","last_clean":"2024-03-29T08:18:13.345107+0000","last_became_active":"2024-03-29T08:18:13.337901+0000","last_became_peered":"2024-03-29T08:18:13.337901+0000","last_unstale":"2024-03-29T08:18:13.345107+0000","last_undegraded":"2024-03-29T08:18:13.345107+0000","last_fullsized":"2024-03-29T08:18:13.345107+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:41:40.150525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.340980+0000","last_change":"2024-03-29T08:18:13.338530+0000","last_active":"2024-03-29T08:18:13.340980+0000","last_peered":"2024-03-29T08:18:13.340980+0000","last_clean":"2024-03-29T08:18:13.340980+0000","last_became_active":"2024-03-29T08:18:13.338188+0000","last_became_peered":"2024-03-29T08:18:13.338188+0000","last_unstale":"2024-03-29T08:18:13.340980+0000","last_undegraded":"2024-03-29T08:18:13.340980+0000","last_fullsized":"2024-03-29T08:18:13.340980+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:03.590543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323230+0000","last_change":"2024-03-29T08:18:13.344948+0000","last_active":"2024-03-29T08:18:14.323230+0000","last_peered":"2024-03-29T08:18:14.323230+0000","last_clean":"2024-03-29T08:18:14.323230+0000","last_became_active":"2024-03-29T08:18:13.341442+0000","last_became_peered":"2024-03-29T08:18:13.341442+0000","last_unstale":"2024-03-29T08:18:14.323230+0000","last_undegraded":"2024-03-29T08:18:14.323230+0000","last_fullsized":"2024-03-29T08:18:14.323230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:53:57.611306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.088799+0000","last_change":"2024-03-29T08:18:13.356080+0000","last_active":"2024-03-29T08:18:15.088799+0000","last_peered":"2024-03-29T08:18:15.088799+0000","last_clean":"2024-03-29T08:18:15.088799+0000","last_became_active":"2024-03-29T08:18:13.355885+0000","last_became_peered":"2024-03-29T08:18:13.355885+0000","last_unstale":"2024-03-29T08:18:15.088799+0000","last_undegraded":"2024-03-29T08:18:15.088799+0000","last_fullsized":"2024-03-29T08:18:15.088799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:57.716575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321203+0000","last_change":"2024-03-29T08:18:13.343561+0000","last_active":"2024-03-29T08:18:14.321203+0000","last_peered":"2024-03-29T08:18:14.321203+0000","last_clean":"2024-03-29T08:18:14.321203+0000","last_became_active":"2024-03-29T08:18:13.341129+0000","last_became_peered":"2024-03-29T08:18:13.341129+0000","last_unstale":"2024-03-29T08:18:14.321203+0000","last_undegraded":"2024-03-29T08:18:14.321203+0000","last_fullsized":"2024-03-29T08:18:14.321203+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:30.625311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.357944+0000","last_change":"2024-03-29T08:18:12.928717+0000","last_active":"2024-03-29T08:18:13.357944+0000","last_peered":"2024-03-29T08:18:13.357944+0000","last_clean":"2024-03-29T08:18:13.357944+0000","last_became_active":"2024-03-29T08:18:12.928427+0000","last_became_peered":"2024-03-29T08:18:12.928427+0000","last_unstale":"2024-03-29T08:18:13.357944+0000","last_undegraded":"2024-03-29T08:18:13.357944+0000","last_fullsized":"2024-03-29T08:18:13.357944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:28:47.778848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324418+0000","last_change":"2024-03-29T08:18:13.346417+0000","last_active":"2024-03-29T08:18:14.324418+0000","last_peered":"2024-03-29T08:18:14.324418+0000","last_clean":"2024-03-29T08:18:14.324418+0000","last_became_active":"2024-03-29T08:18:13.344718+0000","last_became_peered":"2024-03-29T08:18:13.344718+0000","last_unstale":"2024-03-29T08:18:14.324418+0000","last_undegraded":"2024-03-29T08:18:14.324418+0000","last_fullsized":"2024-03-29T08:18:14.324418+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:24:34.643975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.351566+0000","last_change":"2024-03-29T08:18:12.930498+0000","last_active":"2024-03-29T08:18:13.351566+0000","last_peered":"2024-03-29T08:18:13.351566+0000","last_clean":"2024-03-29T08:18:13.351566+0000","last_became_active":"2024-03-29T08:18:12.930232+0000","last_became_peered":"2024-03-29T08:18:12.930232+0000","last_unstale":"2024-03-29T08:18:13.351566+0000","last_undegraded":"2024-03-29T08:18:13.351566+0000","last_fullsized":"2024-03-29T08:18:13.351566+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:04.346131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.339504+0000","last_change":"2024-03-29T08:18:12.927991+0000","last_active":"2024-03-29T08:18:13.339504+0000","last_peered":"2024-03-29T08:18:13.339504+0000","last_clean":"2024-03-29T08:18:13.339504+0000","last_became_active":"2024-03-29T08:18:12.927288+0000","last_became_peered":"2024-03-29T08:18:12.927288+0000","last_unstale":"2024-03-29T08:18:13.339504+0000","last_undegraded":"2024-03-29T08:18:13.339504+0000","last_fullsized":"2024-03-29T08:18:13.339504+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:30:10.121158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.361521+0000","last_change":"2024-03-29T08:18:13.358303+0000","last_active":"2024-03-29T08:18:13.361521+0000","last_peered":"2024-03-29T08:18:13.361521+0000","last_clean":"2024-03-29T08:18:13.361521+0000","last_became_active":"2024-03-29T08:18:13.358041+0000","last_became_peered":"2024-03-29T08:18:13.358041+0000","last_unstale":"2024-03-29T08:18:13.361521+0000","last_undegraded":"2024-03-29T08:18:13.361521+0000","last_fullsized":"2024-03-29T08:18:13.361521+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:41:41.207988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.324995+0000","last_change":"2024-03-29T08:18:12.927896+0000","last_active":"2024-03-29T08:18:13.324995+0000","last_peered":"2024-03-29T08:18:13.324995+0000","last_clean":"2024-03-29T08:18:13.324995+0000","last_became_active":"2024-03-29T08:18:12.927399+0000","last_became_peered":"2024-03-29T08:18:12.927399+0000","last_unstale":"2024-03-29T08:18:13.324995+0000","last_undegraded":"2024-03-29T08:18:13.324995+0000","last_fullsized":"2024-03-29T08:18:13.324995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:56.709840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324362+0000","last_change":"2024-03-29T08:18:13.338175+0000","last_active":"2024-03-29T08:18:14.324362+0000","last_peered":"2024-03-29T08:18:14.324362+0000","last_clean":"2024-03-29T08:18:14.324362+0000","last_became_active":"2024-03-29T08:18:13.337775+0000","last_became_peered":"2024-03-29T08:18:13.337775+0000","last_unstale":"2024-03-29T08:18:14.324362+0000","last_undegraded":"2024-03-29T08:18:14.324362+0000","last_fullsized":"2024-03-29T08:18:14.324362+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:45:52.608559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321469+0000","last_change":"2024-03-29T08:18:12.940829+0000","last_active":"2024-03-29T08:18:14.321469+0000","last_peered":"2024-03-29T08:18:14.321469+0000","last_clean":"2024-03-29T08:18:14.321469+0000","last_became_active":"2024-03-29T08:18:12.929330+0000","last_became_peered":"2024-03-29T08:18:12.929330+0000","last_unstale":"2024-03-29T08:18:14.321469+0000","last_undegraded":"2024-03-29T08:18:14.321469+0000","last_fullsized":"2024-03-29T08:18:14.321469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:55:09.380677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089346+0000","last_change":"2024-03-29T08:18:13.342303+0000","last_active":"2024-03-29T08:18:15.089346+0000","last_peered":"2024-03-29T08:18:15.089346+0000","last_clean":"2024-03-29T08:18:15.089346+0000","last_became_active":"2024-03-29T08:18:13.341642+0000","last_became_peered":"2024-03-29T08:18:13.341642+0000","last_unstale":"2024-03-29T08:18:15.089346+0000","last_undegraded":"2024-03-29T08:18:15.089346+0000","last_fullsized":"2024-03-29T08:18:15.089346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:45.475256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.316882+0000","last_change":"2024-03-29T08:18:12.930755+0000","last_active":"2024-03-29T08:18:13.316882+0000","last_peered":"2024-03-29T08:18:13.316882+0000","last_clean":"2024-03-29T08:18:13.316882+0000","last_became_active":"2024-03-29T08:18:12.929647+0000","last_became_peered":"2024-03-29T08:18:12.929647+0000","last_unstale":"2024-03-29T08:18:13.316882+0000","last_undegraded":"2024-03-29T08:18:13.316882+0000","last_fullsized":"2024-03-29T08:18:13.316882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:16:38.127431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.346719+0000","last_change":"2024-03-29T08:18:13.344231+0000","last_active":"2024-03-29T08:18:13.346719+0000","last_peered":"2024-03-29T08:18:13.346719+0000","last_clean":"2024-03-29T08:18:13.346719+0000","last_became_active":"2024-03-29T08:18:13.343788+0000","last_became_peered":"2024-03-29T08:18:13.343788+0000","last_unstale":"2024-03-29T08:18:13.346719+0000","last_undegraded":"2024-03-29T08:18:13.346719+0000","last_fullsized":"2024-03-29T08:18:13.346719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:15:15.589437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323478+0000","last_change":"2024-03-29T08:18:12.925532+0000","last_active":"2024-03-29T08:18:14.323478+0000","last_peered":"2024-03-29T08:18:14.323478+0000","last_clean":"2024-03-29T08:18:14.323478+0000","last_became_active":"2024-03-29T08:18:12.925228+0000","last_became_peered":"2024-03-29T08:18:12.925228+0000","last_unstale":"2024-03-29T08:18:14.323478+0000","last_undegraded":"2024-03-29T08:18:14.323478+0000","last_fullsized":"2024-03-29T08:18:14.323478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:30:10.121158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.348719+0000","last_change":"2024-03-29T08:18:13.339687+0000","last_active":"2024-03-29T08:18:13.348719+0000","last_peered":"2024-03-29T08:18:13.348719+0000","last_clean":"2024-03-29T08:18:13.348719+0000","last_became_active":"2024-03-29T08:18:13.339445+0000","last_became_peered":"2024-03-29T08:18:13.339445+0000","last_unstale":"2024-03-29T08:18:13.348719+0000","last_undegraded":"2024-03-29T08:18:13.348719+0000","last_fullsized":"2024-03-29T08:18:13.348719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:45:55.900052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089195+0000","last_change":"2024-03-29T08:18:12.927677+0000","last_active":"2024-03-29T08:18:15.089195+0000","last_peered":"2024-03-29T08:18:15.089195+0000","last_clean":"2024-03-29T08:18:15.089195+0000","last_became_active":"2024-03-29T08:18:12.927416+0000","last_became_peered":"2024-03-29T08:18:12.927416+0000","last_unstale":"2024-03-29T08:18:15.089195+0000","last_undegraded":"2024-03-29T08:18:15.089195+0000","last_fullsized":"2024-03-29T08:18:15.089195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:47.035098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.358272+0000","last_change":"2024-03-29T08:18:13.357432+0000","last_active":"2024-03-29T08:18:13.358272+0000","last_peered":"2024-03-29T08:18:13.358272+0000","last_clean":"2024-03-29T08:18:13.358272+0000","last_became_active":"2024-03-29T08:18:13.357221+0000","last_became_peered":"2024-03-29T08:18:13.357221+0000","last_unstale":"2024-03-29T08:18:13.358272+0000","last_undegraded":"2024-03-29T08:18:13.358272+0000","last_fullsized":"2024-03-29T08:18:13.358272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:49.457858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323462+0000","last_change":"2024-03-29T08:18:12.930512+0000","last_active":"2024-03-29T08:18:14.323462+0000","last_peered":"2024-03-29T08:18:14.323462+0000","last_clean":"2024-03-29T08:18:14.323462+0000","last_became_active":"2024-03-29T08:18:12.927576+0000","last_became_peered":"2024-03-29T08:18:12.927576+0000","last_unstale":"2024-03-29T08:18:14.323462+0000","last_undegraded":"2024-03-29T08:18:14.323462+0000","last_fullsized":"2024-03-29T08:18:14.323462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:58.083723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.361037+0000","last_change":"2024-03-29T08:18:13.344540+0000","last_active":"2024-03-29T08:18:13.361037+0000","last_peered":"2024-03-29T08:18:13.361037+0000","last_clean":"2024-03-29T08:18:13.361037+0000","last_became_active":"2024-03-29T08:18:13.343172+0000","last_became_peered":"2024-03-29T08:18:13.343172+0000","last_unstale":"2024-03-29T08:18:13.361037+0000","last_undegraded":"2024-03-29T08:18:13.361037+0000","last_fullsized":"2024-03-29T08:18:13.361037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:15:47.486329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.332921+0000","last_change":"2024-03-29T08:18:12.929722+0000","last_active":"2024-03-29T08:18:13.332921+0000","last_peered":"2024-03-29T08:18:13.332921+0000","last_clean":"2024-03-29T08:18:13.332921+0000","last_became_active":"2024-03-29T08:18:12.929401+0000","last_became_peered":"2024-03-29T08:18:12.929401+0000","last_unstale":"2024-03-29T08:18:13.332921+0000","last_undegraded":"2024-03-29T08:18:13.332921+0000","last_fullsized":"2024-03-29T08:18:13.332921+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:44.468521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.350830+0000","last_change":"2024-03-29T08:18:13.331364+0000","last_active":"2024-03-29T08:18:13.350830+0000","last_peered":"2024-03-29T08:18:13.350830+0000","last_clean":"2024-03-29T08:18:13.350830+0000","last_became_active":"2024-03-29T08:18:13.331097+0000","last_became_peered":"2024-03-29T08:18:13.331097+0000","last_unstale":"2024-03-29T08:18:13.350830+0000","last_undegraded":"2024-03-29T08:18:13.350830+0000","last_fullsized":"2024-03-29T08:18:13.350830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:22:49.782486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.315979+0000","last_change":"2024-03-29T08:18:12.935254+0000","last_active":"2024-03-29T08:18:13.315979+0000","last_peered":"2024-03-29T08:18:13.315979+0000","last_clean":"2024-03-29T08:18:13.315979+0000","last_became_active":"2024-03-29T08:18:12.934862+0000","last_became_peered":"2024-03-29T08:18:12.934862+0000","last_unstale":"2024-03-29T08:18:13.315979+0000","last_undegraded":"2024-03-29T08:18:13.315979+0000","last_fullsized":"2024-03-29T08:18:13.315979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:24.943478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321225+0000","last_change":"2024-03-29T08:18:13.338357+0000","last_active":"2024-03-29T08:18:14.321225+0000","last_peered":"2024-03-29T08:18:14.321225+0000","last_clean":"2024-03-29T08:18:14.321225+0000","last_became_active":"2024-03-29T08:18:13.338109+0000","last_became_peered":"2024-03-29T08:18:13.338109+0000","last_unstale":"2024-03-29T08:18:14.321225+0000","last_undegraded":"2024-03-29T08:18:14.321225+0000","last_fullsized":"2024-03-29T08:18:14.321225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:55:41.020683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324252+0000","last_change":"2024-03-29T08:18:12.914431+0000","last_active":"2024-03-29T08:18:14.324252+0000","last_peered":"2024-03-29T08:18:14.324252+0000","last_clean":"2024-03-29T08:18:14.324252+0000","last_became_active":"2024-03-29T08:18:12.913939+0000","last_became_peered":"2024-03-29T08:18:12.913939+0000","last_unstale":"2024-03-29T08:18:14.324252+0000","last_undegraded":"2024-03-29T08:18:14.324252+0000","last_fullsized":"2024-03-29T08:18:14.324252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:16:38.127431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.355309+0000","last_change":"2024-03-29T08:18:13.351406+0000","last_active":"2024-03-29T08:18:13.355309+0000","last_peered":"2024-03-29T08:18:13.355309+0000","last_clean":"2024-03-29T08:18:13.355309+0000","last_became_active":"2024-03-29T08:18:13.351143+0000","last_became_peered":"2024-03-29T08:18:13.351143+0000","last_unstale":"2024-03-29T08:18:13.355309+0000","last_undegraded":"2024-03-29T08:18:13.355309+0000","last_fullsized":"2024-03-29T08:18:13.355309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:33.632435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.346645+0000","last_change":"2024-03-29T08:18:12.936118+0000","last_active":"2024-03-29T08:18:13.346645+0000","last_peered":"2024-03-29T08:18:13.346645+0000","last_clean":"2024-03-29T08:18:13.346645+0000","last_became_active":"2024-03-29T08:18:12.935861+0000","last_became_peered":"2024-03-29T08:18:12.935861+0000","last_unstale":"2024-03-29T08:18:13.346645+0000","last_undegraded":"2024-03-29T08:18:13.346645+0000","last_fullsized":"2024-03-29T08:18:13.346645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:44.468521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089321+0000","last_change":"2024-03-29T08:18:13.341154+0000","last_active":"2024-03-29T08:18:15.089321+0000","last_peered":"2024-03-29T08:18:15.089321+0000","last_clean":"2024-03-29T08:18:15.089321+0000","last_became_active":"2024-03-29T08:18:13.335461+0000","last_became_peered":"2024-03-29T08:18:13.335461+0000","last_unstale":"2024-03-29T08:18:15.089321+0000","last_undegraded":"2024-03-29T08:18:15.089321+0000","last_fullsized":"2024-03-29T08:18:15.089321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:55:41.020683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324529+0000","last_change":"2024-03-29T08:18:12.928629+0000","last_active":"2024-03-29T08:18:14.324529+0000","last_peered":"2024-03-29T08:18:14.324529+0000","last_clean":"2024-03-29T08:18:14.324529+0000","last_became_active":"2024-03-29T08:18:12.928405+0000","last_became_peered":"2024-03-29T08:18:12.928405+0000","last_unstale":"2024-03-29T08:18:14.324529+0000","last_undegraded":"2024-03-29T08:18:14.324529+0000","last_fullsized":"2024-03-29T08:18:14.324529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:06.406132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.328112+0000","last_change":"2024-03-29T08:18:13.325885+0000","last_active":"2024-03-29T08:18:13.328112+0000","last_peered":"2024-03-29T08:18:13.328112+0000","last_clean":"2024-03-29T08:18:13.328112+0000","last_became_active":"2024-03-29T08:18:13.325621+0000","last_became_peered":"2024-03-29T08:18:13.325621+0000","last_unstale":"2024-03-29T08:18:13.328112+0000","last_undegraded":"2024-03-29T08:18:13.328112+0000","last_fullsized":"2024-03-29T08:18:13.328112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:27:31.782313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.340223+0000","last_change":"2024-03-29T08:18:12.932348+0000","last_active":"2024-03-29T08:18:13.340223+0000","last_peered":"2024-03-29T08:18:13.340223+0000","last_clean":"2024-03-29T08:18:13.340223+0000","last_became_active":"2024-03-29T08:18:12.931959+0000","last_became_peered":"2024-03-29T08:18:12.931959+0000","last_unstale":"2024-03-29T08:18:13.340223+0000","last_undegraded":"2024-03-29T08:18:13.340223+0000","last_fullsized":"2024-03-29T08:18:13.340223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:04:56.043379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.340975+0000","last_change":"2024-03-29T08:18:13.339774+0000","last_active":"2024-03-29T08:18:13.340975+0000","last_peered":"2024-03-29T08:18:13.340975+0000","last_clean":"2024-03-29T08:18:13.340975+0000","last_became_active":"2024-03-29T08:18:13.339154+0000","last_became_peered":"2024-03-29T08:18:13.339154+0000","last_unstale":"2024-03-29T08:18:13.340975+0000","last_undegraded":"2024-03-29T08:18:13.340975+0000","last_fullsized":"2024-03-29T08:18:13.340975+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:10.712191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.341794+0000","last_change":"2024-03-29T08:18:12.924839+0000","last_active":"2024-03-29T08:18:13.341794+0000","last_peered":"2024-03-29T08:18:13.341794+0000","last_clean":"2024-03-29T08:18:13.341794+0000","last_became_active":"2024-03-29T08:18:12.924535+0000","last_became_peered":"2024-03-29T08:18:12.924535+0000","last_unstale":"2024-03-29T08:18:13.341794+0000","last_undegraded":"2024-03-29T08:18:13.341794+0000","last_fullsized":"2024-03-29T08:18:13.341794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:58.083723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.331983+0000","last_change":"2024-03-29T08:18:13.328934+0000","last_active":"2024-03-29T08:18:13.331983+0000","last_peered":"2024-03-29T08:18:13.331983+0000","last_clean":"2024-03-29T08:18:13.331983+0000","last_became_active":"2024-03-29T08:18:13.328410+0000","last_became_peered":"2024-03-29T08:18:13.328410+0000","last_unstale":"2024-03-29T08:18:13.331983+0000","last_undegraded":"2024-03-29T08:18:13.331983+0000","last_fullsized":"2024-03-29T08:18:13.331983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.345206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.320590+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:18:13.320590+0000","last_peered":"2024-03-29T08:18:13.320590+0000","last_clean":"2024-03-29T08:18:13.320590+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:18:13.320590+0000","last_undegraded":"2024-03-29T08:18:13.320590+0000","last_fullsized":"2024-03-29T08:18:13.320590+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.335826+0000","last_change":"2024-03-29T08:18:12.931858+0000","last_active":"2024-03-29T08:18:13.335826+0000","last_peered":"2024-03-29T08:18:13.335826+0000","last_clean":"2024-03-29T08:18:13.335826+0000","last_became_active":"2024-03-29T08:18:12.931277+0000","last_became_peered":"2024-03-29T08:18:12.931277+0000","last_unstale":"2024-03-29T08:18:13.335826+0000","last_undegraded":"2024-03-29T08:18:13.335826+0000","last_fullsized":"2024-03-29T08:18:13.335826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:29.618576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324316+0000","last_change":"2024-03-29T08:18:13.344361+0000","last_active":"2024-03-29T08:18:14.324316+0000","last_peered":"2024-03-29T08:18:14.324316+0000","last_clean":"2024-03-29T08:18:14.324316+0000","last_became_active":"2024-03-29T08:18:13.344050+0000","last_became_peered":"2024-03-29T08:18:13.344050+0000","last_unstale":"2024-03-29T08:18:14.324316+0000","last_undegraded":"2024-03-29T08:18:14.324316+0000","last_fullsized":"2024-03-29T08:18:14.324316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:42:28.266165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.325088+0000","last_change":"2024-03-29T08:18:12.938041+0000","last_active":"2024-03-29T08:18:13.325088+0000","last_peered":"2024-03-29T08:18:13.325088+0000","last_clean":"2024-03-29T08:18:13.325088+0000","last_became_active":"2024-03-29T08:18:12.937061+0000","last_became_peered":"2024-03-29T08:18:12.937061+0000","last_unstale":"2024-03-29T08:18:13.325088+0000","last_undegraded":"2024-03-29T08:18:13.325088+0000","last_fullsized":"2024-03-29T08:18:13.325088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:55:40.013948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.364028+0000","last_change":"2024-03-29T08:18:13.355981+0000","last_active":"2024-03-29T08:18:13.364028+0000","last_peered":"2024-03-29T08:18:13.364028+0000","last_clean":"2024-03-29T08:18:13.364028+0000","last_became_active":"2024-03-29T08:18:13.355691+0000","last_became_peered":"2024-03-29T08:18:13.355691+0000","last_unstale":"2024-03-29T08:18:13.364028+0000","last_undegraded":"2024-03-29T08:18:13.364028+0000","last_fullsized":"2024-03-29T08:18:13.364028+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:18:34.795188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323242+0000","last_change":"2024-03-29T08:18:12.931240+0000","last_active":"2024-03-29T08:18:14.323242+0000","last_peered":"2024-03-29T08:18:14.323242+0000","last_clean":"2024-03-29T08:18:14.323242+0000","last_became_active":"2024-03-29T08:18:12.930854+0000","last_became_peered":"2024-03-29T08:18:12.930854+0000","last_unstale":"2024-03-29T08:18:14.323242+0000","last_undegraded":"2024-03-29T08:18:14.323242+0000","last_fullsized":"2024-03-29T08:18:14.323242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:55:09.380677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.346454+0000","last_change":"2024-03-29T08:18:13.343562+0000","last_active":"2024-03-29T08:18:13.346454+0000","last_peered":"2024-03-29T08:18:13.346454+0000","last_clean":"2024-03-29T08:18:13.346454+0000","last_became_active":"2024-03-29T08:18:13.342138+0000","last_became_peered":"2024-03-29T08:18:13.342138+0000","last_unstale":"2024-03-29T08:18:13.346454+0000","last_undegraded":"2024-03-29T08:18:13.346454+0000","last_fullsized":"2024-03-29T08:18:13.346454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:24:46.843649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.333089+0000","last_change":"2024-03-29T08:18:12.924976+0000","last_active":"2024-03-29T08:18:13.333089+0000","last_peered":"2024-03-29T08:18:13.333089+0000","last_clean":"2024-03-29T08:18:13.333089+0000","last_became_active":"2024-03-29T08:18:12.924677+0000","last_became_peered":"2024-03-29T08:18:12.924677+0000","last_unstale":"2024-03-29T08:18:13.333089+0000","last_undegraded":"2024-03-29T08:18:13.333089+0000","last_fullsized":"2024-03-29T08:18:13.333089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:18:33.788453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.340787+0000","last_change":"2024-03-29T08:18:13.335351+0000","last_active":"2024-03-29T08:18:13.340787+0000","last_peered":"2024-03-29T08:18:13.340787+0000","last_clean":"2024-03-29T08:18:13.340787+0000","last_became_active":"2024-03-29T08:18:13.334286+0000","last_became_peered":"2024-03-29T08:18:13.334286+0000","last_unstale":"2024-03-29T08:18:13.340787+0000","last_undegraded":"2024-03-29T08:18:13.340787+0000","last_fullsized":"2024-03-29T08:18:13.340787+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:47:40.547759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.335038+0000","last_change":"2024-03-29T08:18:12.930589+0000","last_active":"2024-03-29T08:18:13.335038+0000","last_peered":"2024-03-29T08:18:13.335038+0000","last_clean":"2024-03-29T08:18:13.335038+0000","last_became_active":"2024-03-29T08:18:12.930322+0000","last_became_peered":"2024-03-29T08:18:12.930322+0000","last_unstale":"2024-03-29T08:18:13.335038+0000","last_undegraded":"2024-03-29T08:18:13.335038+0000","last_fullsized":"2024-03-29T08:18:13.335038+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:38:37.939078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089112+0000","last_change":"2024-03-29T08:18:13.357420+0000","last_active":"2024-03-29T08:18:15.089112+0000","last_peered":"2024-03-29T08:18:15.089112+0000","last_clean":"2024-03-29T08:18:15.089112+0000","last_became_active":"2024-03-29T08:18:13.357130+0000","last_became_peered":"2024-03-29T08:18:13.357130+0000","last_unstale":"2024-03-29T08:18:15.089112+0000","last_undegraded":"2024-03-29T08:18:15.089112+0000","last_fullsized":"2024-03-29T08:18:15.089112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:07.412867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.339216+0000","last_change":"2024-03-29T08:18:12.938320+0000","last_active":"2024-03-29T08:18:13.339216+0000","last_peered":"2024-03-29T08:18:13.339216+0000","last_clean":"2024-03-29T08:18:13.339216+0000","last_became_active":"2024-03-29T08:18:12.937654+0000","last_became_peered":"2024-03-29T08:18:12.937654+0000","last_unstale":"2024-03-29T08:18:13.339216+0000","last_undegraded":"2024-03-29T08:18:13.339216+0000","last_fullsized":"2024-03-29T08:18:13.339216+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:06.406132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.364215+0000","last_change":"2024-03-29T08:18:13.357591+0000","last_active":"2024-03-29T08:18:13.364215+0000","last_peered":"2024-03-29T08:18:13.364215+0000","last_clean":"2024-03-29T08:18:13.364215+0000","last_became_active":"2024-03-29T08:18:13.357160+0000","last_became_peered":"2024-03-29T08:18:13.357160+0000","last_unstale":"2024-03-29T08:18:13.364215+0000","last_undegraded":"2024-03-29T08:18:13.364215+0000","last_fullsized":"2024-03-29T08:18:13.364215+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:45:52.608559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324569+0000","last_change":"2024-03-29T08:18:12.926261+0000","last_active":"2024-03-29T08:18:14.324569+0000","last_peered":"2024-03-29T08:18:14.324569+0000","last_clean":"2024-03-29T08:18:14.324569+0000","last_became_active":"2024-03-29T08:18:12.925938+0000","last_became_peered":"2024-03-29T08:18:12.925938+0000","last_unstale":"2024-03-29T08:18:14.324569+0000","last_undegraded":"2024-03-29T08:18:14.324569+0000","last_fullsized":"2024-03-29T08:18:14.324569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:56.709840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324644+0000","last_change":"2024-03-29T08:18:13.357731+0000","last_active":"2024-03-29T08:18:14.324644+0000","last_peered":"2024-03-29T08:18:14.324644+0000","last_clean":"2024-03-29T08:18:14.324644+0000","last_became_active":"2024-03-29T08:18:13.357437+0000","last_became_peered":"2024-03-29T08:18:13.357437+0000","last_unstale":"2024-03-29T08:18:14.324644+0000","last_undegraded":"2024-03-29T08:18:14.324644+0000","last_fullsized":"2024-03-29T08:18:14.324644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:10.712191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.355358+0000","last_change":"2024-03-29T08:18:12.942493+0000","last_active":"2024-03-29T08:18:13.355358+0000","last_peered":"2024-03-29T08:18:13.355358+0000","last_clean":"2024-03-29T08:18:13.355358+0000","last_became_active":"2024-03-29T08:18:12.942116+0000","last_became_peered":"2024-03-29T08:18:12.942116+0000","last_unstale":"2024-03-29T08:18:13.355358+0000","last_undegraded":"2024-03-29T08:18:13.355358+0000","last_fullsized":"2024-03-29T08:18:13.355358+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:55.791360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324502+0000","last_change":"2024-03-29T08:18:13.343380+0000","last_active":"2024-03-29T08:18:14.324502+0000","last_peered":"2024-03-29T08:18:14.324502+0000","last_clean":"2024-03-29T08:18:14.324502+0000","last_became_active":"2024-03-29T08:18:13.343121+0000","last_became_peered":"2024-03-29T08:18:13.343121+0000","last_unstale":"2024-03-29T08:18:14.324502+0000","last_undegraded":"2024-03-29T08:18:14.324502+0000","last_fullsized":"2024-03-29T08:18:14.324502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:52:27.461296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.315882+0000","last_change":"2024-03-29T08:18:12.935438+0000","last_active":"2024-03-29T08:18:13.315882+0000","last_peered":"2024-03-29T08:18:13.315882+0000","last_clean":"2024-03-29T08:18:13.315882+0000","last_became_active":"2024-03-29T08:18:12.935170+0000","last_became_peered":"2024-03-29T08:18:12.935170+0000","last_unstale":"2024-03-29T08:18:13.315882+0000","last_undegraded":"2024-03-29T08:18:13.315882+0000","last_fullsized":"2024-03-29T08:18:13.315882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:53:56.604571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.351291+0000","last_change":"2024-03-29T08:18:13.341595+0000","last_active":"2024-03-29T08:18:13.351291+0000","last_peered":"2024-03-29T08:18:13.351291+0000","last_clean":"2024-03-29T08:18:13.351291+0000","last_became_active":"2024-03-29T08:18:13.341387+0000","last_became_peered":"2024-03-29T08:18:13.341387+0000","last_unstale":"2024-03-29T08:18:13.351291+0000","last_undegraded":"2024-03-29T08:18:13.351291+0000","last_fullsized":"2024-03-29T08:18:13.351291+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:37:35.410925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324393+0000","last_change":"2024-03-29T08:18:12.922897+0000","last_active":"2024-03-29T08:18:14.324393+0000","last_peered":"2024-03-29T08:18:14.324393+0000","last_clean":"2024-03-29T08:18:14.324393+0000","last_became_active":"2024-03-29T08:18:12.922583+0000","last_became_peered":"2024-03-29T08:18:12.922583+0000","last_unstale":"2024-03-29T08:18:14.324393+0000","last_undegraded":"2024-03-29T08:18:14.324393+0000","last_fullsized":"2024-03-29T08:18:14.324393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:24.943478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324257+0000","last_change":"2024-03-29T08:18:13.344357+0000","last_active":"2024-03-29T08:18:14.324257+0000","last_peered":"2024-03-29T08:18:14.324257+0000","last_clean":"2024-03-29T08:18:14.324257+0000","last_became_active":"2024-03-29T08:18:13.343725+0000","last_became_peered":"2024-03-29T08:18:13.343725+0000","last_unstale":"2024-03-29T08:18:14.324257+0000","last_undegraded":"2024-03-29T08:18:14.324257+0000","last_fullsized":"2024-03-29T08:18:14.324257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:56.798095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.339735+0000","last_change":"2024-03-29T08:18:12.930747+0000","last_active":"2024-03-29T08:18:13.339735+0000","last_peered":"2024-03-29T08:18:13.339735+0000","last_clean":"2024-03-29T08:18:13.339735+0000","last_became_active":"2024-03-29T08:18:12.930527+0000","last_became_peered":"2024-03-29T08:18:12.930527+0000","last_unstale":"2024-03-29T08:18:13.339735+0000","last_undegraded":"2024-03-29T08:18:13.339735+0000","last_fullsized":"2024-03-29T08:18:13.339735+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:55:09.380677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323459+0000","last_change":"2024-03-29T08:18:13.359008+0000","last_active":"2024-03-29T08:18:14.323459+0000","last_peered":"2024-03-29T08:18:14.323459+0000","last_clean":"2024-03-29T08:18:14.323459+0000","last_became_active":"2024-03-29T08:18:13.358733+0000","last_became_peered":"2024-03-29T08:18:13.358733+0000","last_unstale":"2024-03-29T08:18:14.323459+0000","last_undegraded":"2024-03-29T08:18:14.323459+0000","last_fullsized":"2024-03-29T08:18:14.323459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:45.475256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.353985+0000","last_change":"2024-03-29T08:18:12.932074+0000","last_active":"2024-03-29T08:18:13.353985+0000","last_peered":"2024-03-29T08:18:13.353985+0000","last_clean":"2024-03-29T08:18:13.353985+0000","last_became_active":"2024-03-29T08:18:12.931603+0000","last_became_peered":"2024-03-29T08:18:12.931603+0000","last_unstale":"2024-03-29T08:18:13.353985+0000","last_undegraded":"2024-03-29T08:18:13.353985+0000","last_fullsized":"2024-03-29T08:18:13.353985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:31.317829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.338388+0000","last_change":"2024-03-29T08:18:13.331615+0000","last_active":"2024-03-29T08:18:13.338388+0000","last_peered":"2024-03-29T08:18:13.338388+0000","last_clean":"2024-03-29T08:18:13.338388+0000","last_became_active":"2024-03-29T08:18:13.331399+0000","last_became_peered":"2024-03-29T08:18:13.331399+0000","last_unstale":"2024-03-29T08:18:13.338388+0000","last_undegraded":"2024-03-29T08:18:13.338388+0000","last_fullsized":"2024-03-29T08:18:13.338388+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:49.457858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.342842+0000","last_change":"2024-03-29T08:18:13.339347+0000","last_active":"2024-03-29T08:18:13.342842+0000","last_peered":"2024-03-29T08:18:13.342842+0000","last_clean":"2024-03-29T08:18:13.342842+0000","last_became_active":"2024-03-29T08:18:13.338931+0000","last_became_peered":"2024-03-29T08:18:13.338931+0000","last_unstale":"2024-03-29T08:18:13.342842+0000","last_undegraded":"2024-03-29T08:18:13.342842+0000","last_fullsized":"2024-03-29T08:18:13.342842+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:06.953763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.344654+0000","last_change":"2024-03-29T08:18:12.930282+0000","last_active":"2024-03-29T08:18:13.344654+0000","last_peered":"2024-03-29T08:18:13.344654+0000","last_clean":"2024-03-29T08:18:13.344654+0000","last_became_active":"2024-03-29T08:18:12.929883+0000","last_became_peered":"2024-03-29T08:18:12.929883+0000","last_unstale":"2024-03-29T08:18:13.344654+0000","last_undegraded":"2024-03-29T08:18:13.344654+0000","last_fullsized":"2024-03-29T08:18:13.344654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:30:10.121158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321265+0000","last_change":"2024-03-29T08:18:13.335469+0000","last_active":"2024-03-29T08:18:14.321265+0000","last_peered":"2024-03-29T08:18:14.321265+0000","last_clean":"2024-03-29T08:18:14.321265+0000","last_became_active":"2024-03-29T08:18:13.335195+0000","last_became_peered":"2024-03-29T08:18:13.335195+0000","last_unstale":"2024-03-29T08:18:14.321265+0000","last_undegraded":"2024-03-29T08:18:14.321265+0000","last_fullsized":"2024-03-29T08:18:14.321265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:53:57.611306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.338461+0000","last_change":"2024-03-29T08:18:12.922889+0000","last_active":"2024-03-29T08:18:13.338461+0000","last_peered":"2024-03-29T08:18:13.338461+0000","last_clean":"2024-03-29T08:18:13.338461+0000","last_became_active":"2024-03-29T08:18:12.921734+0000","last_became_peered":"2024-03-29T08:18:12.921734+0000","last_unstale":"2024-03-29T08:18:13.338461+0000","last_undegraded":"2024-03-29T08:18:13.338461+0000","last_fullsized":"2024-03-29T08:18:13.338461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:06.406132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.346355+0000","last_change":"2024-03-29T08:18:13.341340+0000","last_active":"2024-03-29T08:18:13.346355+0000","last_peered":"2024-03-29T08:18:13.346355+0000","last_clean":"2024-03-29T08:18:13.346355+0000","last_became_active":"2024-03-29T08:18:13.341062+0000","last_became_peered":"2024-03-29T08:18:13.341062+0000","last_unstale":"2024-03-29T08:18:13.346355+0000","last_undegraded":"2024-03-29T08:18:13.346355+0000","last_fullsized":"2024-03-29T08:18:13.346355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:54:33.899398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.348466+0000","last_change":"2024-03-29T08:18:12.928434+0000","last_active":"2024-03-29T08:18:13.348466+0000","last_peered":"2024-03-29T08:18:13.348466+0000","last_clean":"2024-03-29T08:18:13.348466+0000","last_became_active":"2024-03-29T08:18:12.928109+0000","last_became_peered":"2024-03-29T08:18:12.928109+0000","last_unstale":"2024-03-29T08:18:13.348466+0000","last_undegraded":"2024-03-29T08:18:13.348466+0000","last_fullsized":"2024-03-29T08:18:13.348466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:25:41.738901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.345342+0000","last_change":"2024-03-29T08:18:13.337191+0000","last_active":"2024-03-29T08:18:13.345342+0000","last_peered":"2024-03-29T08:18:13.345342+0000","last_clean":"2024-03-29T08:18:13.345342+0000","last_became_active":"2024-03-29T08:18:13.336967+0000","last_became_peered":"2024-03-29T08:18:13.336967+0000","last_unstale":"2024-03-29T08:18:13.345342+0000","last_undegraded":"2024-03-29T08:18:13.345342+0000","last_fullsized":"2024-03-29T08:18:13.345342+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:40.987756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324681+0000","last_change":"2024-03-29T08:18:12.937840+0000","last_active":"2024-03-29T08:18:14.324681+0000","last_peered":"2024-03-29T08:18:14.324681+0000","last_clean":"2024-03-29T08:18:14.324681+0000","last_became_active":"2024-03-29T08:18:12.937403+0000","last_became_peered":"2024-03-29T08:18:12.937403+0000","last_unstale":"2024-03-29T08:18:14.324681+0000","last_undegraded":"2024-03-29T08:18:14.324681+0000","last_fullsized":"2024-03-29T08:18:14.324681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:58.083723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":217088,"data_stored":70064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":69940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79900000000000004}]}]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":78476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31320,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340520,"statfs":{"total":96636764160,"available":96604692480,"internally_reserved":0,"allocated":212992,"data_stored":70522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]}]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":221184,"data_stored":71504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":819200,"data_stored":666678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31332,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":217088,"data_stored":69996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":653533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:18:17.816 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T08:18:17.817 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-29T08:18:17.817 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-29T08:18:17.817 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-29T08:18:17.817 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-29T08:18:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.996+0000 7f4e95a00640 1 Processor -- start 2024-03-29T08:18:17.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:17.996+0000 7f4e95a00640 1 -- start start 2024-03-29T08:18:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.000+0000 7f4e95a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e901569a0 0x7f4e9006c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.000+0000 7f4e95a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e900ce910 0x7f4e900701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.000+0000 7f4e95a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9015d7c0 con 0x7f4e900ce480 2024-03-29T08:18:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.000+0000 7f4e95a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e900b21d0 con 0x7f4e901569a0 2024-03-29T08:18:18.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.000+0000 7f4e95a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9006e020 con 0x7f4e900ce910 2024-03-29T08:18:18.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.000+0000 7f4e8ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e901569a0 0x7f4e9006c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:18.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.000+0000 7f4e8ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e901569a0 0x7f4e9006c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59736/0 (socket says 172.21.15.46:59736) 2024-03-29T08:18:18.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.000+0000 7f4e8ea00640 1 -- 172.21.15.46:0/1619256326 learned_addr learned my addr 172.21.15.46:0/1619256326 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:18.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.000+0000 7f4e8f400640 1 --2- 172.21.15.46:0/1619256326 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e900ce910 0x7f4e900701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:18.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.000+0000 7f4e8ea00640 1 -- 172.21.15.46:0/1619256326 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e900ce910 msgr2=0x7f4e900701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:18.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.000+0000 7f4e8ea00640 1 --2- 172.21.15.46:0/1619256326 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e900ce910 0x7f4e900701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.000+0000 7f4e8ea00640 1 -- 172.21.15.46:0/1619256326 >> v1:172.21.15.73:6789/0 conn(0x7f4e900ce480 legacy=0x7f4e9006d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:18.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.000+0000 7f4e8ea00640 1 -- 172.21.15.46:0/1619256326 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9015cc40 con 0x7f4e901569a0 2024-03-29T08:18:18.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e8ea00640 1 --2- 172.21.15.46:0/1619256326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e901569a0 0x7f4e9006c1e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4e800098a0 tx=0x7f4e8001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c839ed7e05497e4b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:18.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e8e000640 1 -- 172.21.15.46:0/1619256326 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e800ab600 con 0x7f4e901569a0 2024-03-29T08:18:18.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e8e000640 1 -- 172.21.15.46:0/1619256326 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e800a7050 con 0x7f4e901569a0 2024-03-29T08:18:18.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e8e000640 1 -- 172.21.15.46:0/1619256326 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e800ab980 con 0x7f4e901569a0 2024-03-29T08:18:18.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e95a00640 1 -- 172.21.15.46:0/1619256326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e901569a0 msgr2=0x7f4e9006c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e95a00640 1 --2- 172.21.15.46:0/1619256326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e901569a0 0x7f4e9006c1e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4e800098a0 tx=0x7f4e8001c500 comp rx=0 tx=0).stop 2024-03-29T08:18:18.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e95a00640 1 -- 172.21.15.46:0/1619256326 shutdown_connections 2024-03-29T08:18:18.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e95a00640 1 --2- 172.21.15.46:0/1619256326 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e900ce910 0x7f4e900701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e95a00640 1 --2- 172.21.15.46:0/1619256326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e901569a0 0x7f4e9006c1e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e95a00640 1 -- 172.21.15.46:0/1619256326 >> 172.21.15.46:0/1619256326 conn(0x7f4e90066110 msgr2=0x7f4e90069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:18.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e95a00640 1 -- 172.21.15.46:0/1619256326 shutdown_connections 2024-03-29T08:18:18.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e95a00640 1 -- 172.21.15.46:0/1619256326 wait complete. 2024-03-29T08:18:18.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e95a00640 1 Processor -- start 2024-03-29T08:18:18.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e95a00640 1 -- start start 2024-03-29T08:18:18.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e95a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e900ce480 0x7f4e900cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:18.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e95a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e900ce910 0x7f4e9015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:18.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e95a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e901569a0 0x7f4e900ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:18.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e95a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9015b340 con 0x7f4e900ce480 2024-03-29T08:18:18.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e95a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e90152350 con 0x7f4e901569a0 2024-03-29T08:18:18.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.004+0000 7f4e95a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e90152630 con 0x7f4e900ce910 2024-03-29T08:18:18.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e8ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e900ce480 0x7f4e900cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:18.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e8ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e900ce480 0x7f4e900cbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59750/0 (socket says 172.21.15.46:59750) 2024-03-29T08:18:18.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e8ea00640 1 -- 172.21.15.46:0/3509715226 learned_addr learned my addr 172.21.15.46:0/3509715226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:18.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e86000640 1 --2- 172.21.15.46:0/3509715226 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e900ce910 0x7f4e9015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:18.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e8f400640 1 --2- 172.21.15.46:0/3509715226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e901569a0 0x7f4e900ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:18.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e8ea00640 1 -- 172.21.15.46:0/3509715226 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e900ce910 msgr2=0x7f4e9015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e8ea00640 1 --2- 172.21.15.46:0/3509715226 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e900ce910 0x7f4e9015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e8ea00640 1 -- 172.21.15.46:0/3509715226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e901569a0 msgr2=0x7f4e900ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e8ea00640 1 --2- 172.21.15.46:0/3509715226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e901569a0 0x7f4e900ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e8ea00640 1 -- 172.21.15.46:0/3509715226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e800094d0 con 0x7f4e900ce480 2024-03-29T08:18:18.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e8ea00640 1 --2- 172.21.15.46:0/3509715226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e900ce480 0x7f4e900cbff0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4e800ae040 tx=0x7f4e800192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:18.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e87400640 1 -- 172.21.15.46:0/3509715226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e800b4420 con 0x7f4e900ce480 2024-03-29T08:18:18.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e87400640 1 -- 172.21.15.46:0/3509715226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e800b45a0 con 0x7f4e900ce480 2024-03-29T08:18:18.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e87400640 1 -- 172.21.15.46:0/3509715226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e800b4950 con 0x7f4e900ce480 2024-03-29T08:18:18.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e95a00640 1 -- 172.21.15.46:0/3509715226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e90152910 con 0x7f4e900ce480 2024-03-29T08:18:18.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e95a00640 1 -- 172.21.15.46:0/3509715226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9015afe0 con 0x7f4e900ce480 2024-03-29T08:18:18.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e95a00640 1 -- 172.21.15.46:0/3509715226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e90066c90 con 0x7f4e900ce480 2024-03-29T08:18:18.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e87400640 1 -- 172.21.15.46:0/3509715226 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4e800b2070 con 0x7f4e900ce480 2024-03-29T08:18:18.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.008+0000 7f4e87400640 1 --2- 172.21.15.46:0/3509715226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4e6c066690 0x7f4e6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:18.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.012+0000 7f4e87400640 1 -- 172.21.15.46:0/3509715226 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e8011d290 con 0x7f4e900ce480 2024-03-29T08:18:18.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.012+0000 7f4e86000640 1 --2- 172.21.15.46:0/3509715226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4e6c066690 0x7f4e6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:18.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.012+0000 7f4e86000640 1 --2- 172.21.15.46:0/3509715226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4e6c066690 0x7f4e6c068b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4e90147690 tx=0x7f4e78003dd0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:18.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.016+0000 7f4e87400640 1 -- 172.21.15.46:0/3509715226 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4e800ea6a0 con 0x7f4e900ce480 2024-03-29T08:18:18.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.216+0000 7f4e95a00640 1 -- 172.21.15.46:0/3509715226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f4e9015c4f0 con 0x7f4e900ce480 2024-03-29T08:18:18.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.216+0000 7f4e87400640 1 -- 172.21.15.46:0/3509715226 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164874 (secure 0 0 0) 0x7f4e800edcc0 con 0x7f4e900ce480 2024-03-29T08:18:18.219 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:18.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.220+0000 7f4e95a00640 1 -- 172.21.15.46:0/3509715226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4e6c066690 msgr2=0x7f4e6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.220+0000 7f4e95a00640 1 --2- 172.21.15.46:0/3509715226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4e6c066690 0x7f4e6c068b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4e90147690 tx=0x7f4e78003dd0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.220+0000 7f4e95a00640 1 -- 172.21.15.46:0/3509715226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e900ce480 msgr2=0x7f4e900cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.220+0000 7f4e95a00640 1 --2- 172.21.15.46:0/3509715226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e900ce480 0x7f4e900cbff0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4e800ae040 tx=0x7f4e800192a0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.220+0000 7f4e95a00640 1 -- 172.21.15.46:0/3509715226 shutdown_connections 2024-03-29T08:18:18.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.220+0000 7f4e95a00640 1 --2- 172.21.15.46:0/3509715226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4e6c066690 0x7f4e6c068b50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.220+0000 7f4e95a00640 1 --2- 172.21.15.46:0/3509715226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e901569a0 0x7f4e900ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.220+0000 7f4e95a00640 1 --2- 172.21.15.46:0/3509715226 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e900ce910 0x7f4e9015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.220+0000 7f4e95a00640 1 --2- 172.21.15.46:0/3509715226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e900ce480 0x7f4e900cbff0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.220+0000 7f4e95a00640 1 -- 172.21.15.46:0/3509715226 >> 172.21.15.46:0/3509715226 conn(0x7f4e90066110 msgr2=0x7f4e90156f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:18.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.220+0000 7f4e95a00640 1 -- 172.21.15.46:0/3509715226 shutdown_connections 2024-03-29T08:18:18.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.220+0000 7f4e95a00640 1 -- 172.21.15.46:0/3509715226 wait complete. 2024-03-29T08:18:18.241 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":5,"active_gid":4114,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":27375},{"type":"v1","addr":"172.21.15.73:6833","nonce":27375}]},"active_addr":"172.21.15.73:6833/27375","active_change":"2024-03-29T08:17:54.028108+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4119,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":1980200777}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":3329675828}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":342209607}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":1426910974}]}]} 2024-03-29T08:18:18.245 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-29T08:18:18.245 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-29T08:18:18.246 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:18.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.384+0000 7f9073e00640 1 Processor -- start 2024-03-29T08:18:18.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.384+0000 7f9073e00640 1 -- start start 2024-03-29T08:18:18.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.384+0000 7f9073e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f906c0ccd10 0x7f906c098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:18.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.388+0000 7f9073e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906c0d0030 con 0x7f906c15cdb0 2024-03-29T08:18:18.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.388+0000 7f9073e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906c0cf420 con 0x7f906c15c9d0 2024-03-29T08:18:18.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.388+0000 7f9073e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f906c0cf700 con 0x7f906c0ccd10 2024-03-29T08:18:18.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.388+0000 7f9071e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f906c15cdb0 0x7f906c166a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60862/0 (socket says 172.21.15.46:60862) 2024-03-29T08:18:18.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.388+0000 7f9071e00640 1 -- 172.21.15.46:0/770016228 learned_addr learned my addr 172.21.15.46:0/770016228 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.388+0000 7f9070a00640 1 --2- 172.21.15.46:0/770016228 >> v2:172.21.15.73:3300/0 conn(0x7f906c0ccd10 0x7f906c098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.388+0000 7f906be00640 1 -- 172.21.15.46:0/770016228 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2387938396 0 0) 0x7f906c0d0030 con 0x7f906c15cdb0 2024-03-29T08:18:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.388+0000 7f906be00640 1 -- 172.21.15.46:0/770016228 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9054003530 con 0x7f906c15cdb0 2024-03-29T08:18:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.388+0000 7f9070a00640 1 -- 172.21.15.46:0/770016228 >> v1:172.21.15.73:6789/0 conn(0x7f906c15c9d0 legacy=0x7f906c0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.388+0000 7f9070a00640 1 -- 172.21.15.46:0/770016228 >> v1:172.21.15.46:6789/0 conn(0x7f906c15cdb0 legacy=0x7f906c166a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.388+0000 7f9070a00640 1 -- 172.21.15.46:0/770016228 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c0cf9e0 con 0x7f906c0ccd10 2024-03-29T08:18:18.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.388+0000 7f9070a00640 1 --2- 172.21.15.46:0/770016228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f906c0ccd10 0x7f906c098a40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f90580097c0 tx=0x7f905801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbfff1dfd0c55f60 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:18.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f906be00640 1 -- 172.21.15.46:0/770016228 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90580ab650 con 0x7f906c0ccd10 2024-03-29T08:18:18.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f906be00640 1 -- 172.21.15.46:0/770016228 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90580ab7b0 con 0x7f906c0ccd10 2024-03-29T08:18:18.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f906be00640 1 -- 172.21.15.46:0/770016228 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90580abb20 con 0x7f906c0ccd10 2024-03-29T08:18:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9073e00640 1 -- 172.21.15.46:0/770016228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f906c0ccd10 msgr2=0x7f906c098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9073e00640 1 --2- 172.21.15.46:0/770016228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f906c0ccd10 0x7f906c098a40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f90580097c0 tx=0x7f905801c470 comp rx=0 tx=0).stop 2024-03-29T08:18:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9073e00640 1 -- 172.21.15.46:0/770016228 shutdown_connections 2024-03-29T08:18:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9073e00640 1 --2- 172.21.15.46:0/770016228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f906c0ccd10 0x7f906c098a40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9073e00640 1 -- 172.21.15.46:0/770016228 >> 172.21.15.46:0/770016228 conn(0x7f906c0b21a0 msgr2=0x7f906c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9073e00640 1 -- 172.21.15.46:0/770016228 shutdown_connections 2024-03-29T08:18:18.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9073e00640 1 -- 172.21.15.46:0/770016228 wait complete. 2024-03-29T08:18:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9073e00640 1 Processor -- start 2024-03-29T08:18:18.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9073e00640 1 -- start start 2024-03-29T08:18:18.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9073e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f906c0ccd10 0x7f906c0c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:18.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9073e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f906c15c9d0 0x7f906c154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:18.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9073e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f906c15cdb0 0x7f906c15a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:18.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9073e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c0c8a20 con 0x7f906c15c9d0 2024-03-29T08:18:18.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9073e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c0c7e10 con 0x7f906c0ccd10 2024-03-29T08:18:18.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9073e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906c0c80f0 con 0x7f906c15cdb0 2024-03-29T08:18:18.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9070a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f906c15c9d0 0x7f906c154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:18.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9070a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f906c15c9d0 0x7f906c154030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59766/0 (socket says 172.21.15.46:59766) 2024-03-29T08:18:18.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9070a00640 1 -- 172.21.15.46:0/3215407825 learned_addr learned my addr 172.21.15.46:0/3215407825 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.392+0000 7f9071400640 1 --2- 172.21.15.46:0/3215407825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f906c0ccd10 0x7f906c0c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.396+0000 7f9071e00640 1 --2- 172.21.15.46:0/3215407825 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f906c15cdb0 0x7f906c15a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.396+0000 7f9070a00640 1 -- 172.21.15.46:0/3215407825 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f906c15cdb0 msgr2=0x7f906c15a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.396+0000 7f9070a00640 1 --2- 172.21.15.46:0/3215407825 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f906c15cdb0 0x7f906c15a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.396+0000 7f9070a00640 1 -- 172.21.15.46:0/3215407825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f906c0ccd10 msgr2=0x7f906c0c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.396+0000 7f9070a00640 1 --2- 172.21.15.46:0/3215407825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f906c0ccd10 0x7f906c0c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.396+0000 7f9070a00640 1 -- 172.21.15.46:0/3215407825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9058009420 con 0x7f906c15c9d0 2024-03-29T08:18:18.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.396+0000 7f9070a00640 1 --2- 172.21.15.46:0/3215407825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f906c15c9d0 0x7f906c154030 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f905801a0a0 tx=0x7f905801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.396+0000 7f9069600640 1 -- 172.21.15.46:0/3215407825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9058004220 con 0x7f906c15c9d0 2024-03-29T08:18:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.396+0000 7f9069600640 1 -- 172.21.15.46:0/3215407825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90580b9070 con 0x7f906c15c9d0 2024-03-29T08:18:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.396+0000 7f9069600640 1 -- 172.21.15.46:0/3215407825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9058003620 con 0x7f906c15c9d0 2024-03-29T08:18:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.396+0000 7f9073e00640 1 -- 172.21.15.46:0/3215407825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906c0c83d0 con 0x7f906c15c9d0 2024-03-29T08:18:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.396+0000 7f9073e00640 1 -- 172.21.15.46:0/3215407825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906c0ca600 con 0x7f906c15c9d0 2024-03-29T08:18:18.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.396+0000 7f9073e00640 1 -- 172.21.15.46:0/3215407825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906c0c8b90 con 0x7f906c15c9d0 2024-03-29T08:18:18.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.396+0000 7f9069600640 1 -- 172.21.15.46:0/3215407825 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9058003780 con 0x7f906c15c9d0 2024-03-29T08:18:18.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.396+0000 7f9069600640 1 --2- 172.21.15.46:0/3215407825 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f904c066690 0x7f904c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:18.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.396+0000 7f9069600640 1 -- 172.21.15.46:0/3215407825 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90580bb070 con 0x7f906c15c9d0 2024-03-29T08:18:18.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.400+0000 7f9071400640 1 --2- 172.21.15.46:0/3215407825 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f904c066690 0x7f904c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:18.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.400+0000 7f9071400640 1 --2- 172.21.15.46:0/3215407825 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f904c066690 0x7f904c068b50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9060007fd0 tx=0x7f9060009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:18.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.400+0000 7f9069600640 1 -- 172.21.15.46:0/3215407825 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f90580ec050 con 0x7f906c15c9d0 2024-03-29T08:18:18.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.564+0000 7f9073e00640 1 -- 172.21.15.46:0/3215407825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f906c0c8230 con 0x7f906c15c9d0 2024-03-29T08:18:18.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.564+0000 7f9069600640 1 -- 172.21.15.46:0/3215407825 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f906c0c8230 con 0x7f906c15c9d0 2024-03-29T08:18:18.567 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:18.567 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:14.314124+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:18.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.568+0000 7f9073e00640 1 -- 172.21.15.46:0/3215407825 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f904c066690 msgr2=0x7f904c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.568+0000 7f9073e00640 1 --2- 172.21.15.46:0/3215407825 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f904c066690 0x7f904c068b50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9060007fd0 tx=0x7f9060009040 comp rx=0 tx=0).stop 2024-03-29T08:18:18.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.568+0000 7f9073e00640 1 -- 172.21.15.46:0/3215407825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f906c15c9d0 msgr2=0x7f906c154030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.568+0000 7f9073e00640 1 --2- 172.21.15.46:0/3215407825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f906c15c9d0 0x7f906c154030 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f905801a0a0 tx=0x7f905801acb0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.568+0000 7f9073e00640 1 -- 172.21.15.46:0/3215407825 shutdown_connections 2024-03-29T08:18:18.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.568+0000 7f9073e00640 1 --2- 172.21.15.46:0/3215407825 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f904c066690 0x7f904c068b50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.568+0000 7f9073e00640 1 --2- 172.21.15.46:0/3215407825 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f906c15cdb0 0x7f906c15a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.568+0000 7f9073e00640 1 --2- 172.21.15.46:0/3215407825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f906c15c9d0 0x7f906c154030 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.568+0000 7f9073e00640 1 --2- 172.21.15.46:0/3215407825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f906c0ccd10 0x7f906c0c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.568+0000 7f9073e00640 1 -- 172.21.15.46:0/3215407825 >> 172.21.15.46:0/3215407825 conn(0x7f906c0b21a0 msgr2=0x7f906c0cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:18.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.568+0000 7f9073e00640 1 -- 172.21.15.46:0/3215407825 shutdown_connections 2024-03-29T08:18:18.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.568+0000 7f9073e00640 1 -- 172.21.15.46:0/3215407825 wait complete. 2024-03-29T08:18:18.588 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-29T08:18:18.588 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.728+0000 7f2790e00640 1 Processor -- start 2024-03-29T08:18:18.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.728+0000 7f2790e00640 1 -- start start 2024-03-29T08:18:18.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.728+0000 7f2790e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f278c0b01c0 0x7f278c0cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:18.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.728+0000 7f2790e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f278c0d0150 0x7f278c0aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:18.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.728+0000 7f2790e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f278c0b0ed0 con 0x7f278c0af420 2024-03-29T08:18:18.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.728+0000 7f2790e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f278c0cff90 con 0x7f278c0d0150 2024-03-29T08:18:18.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.728+0000 7f2790e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f278c0cbd60 con 0x7f278c0b01c0 2024-03-29T08:18:18.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.728+0000 7f2783e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f278c0d0150 0x7f278c0aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:18.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.728+0000 7f2783e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f278c0d0150 0x7f278c0aeee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59778/0 (socket says 172.21.15.46:59778) 2024-03-29T08:18:18.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.728+0000 7f2783e00640 1 -- 172.21.15.46:0/3536908667 learned_addr learned my addr 172.21.15.46:0/3536908667 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:18.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.728+0000 7f2789e00640 1 --2- 172.21.15.46:0/3536908667 >> v2:172.21.15.73:3300/0 conn(0x7f278c0b01c0 0x7f278c0cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:18.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.728+0000 7f2783e00640 1 -- 172.21.15.46:0/3536908667 >> v2:172.21.15.73:3300/0 conn(0x7f278c0b01c0 msgr2=0x7f278c0cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:18.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.728+0000 7f2783e00640 1 --2- 172.21.15.46:0/3536908667 >> v2:172.21.15.73:3300/0 conn(0x7f278c0b01c0 0x7f278c0cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.728+0000 7f2783e00640 1 -- 172.21.15.46:0/3536908667 >> v1:172.21.15.46:6789/0 conn(0x7f278c0af420 legacy=0x7f278c0cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.728+0000 7f2783e00640 1 -- 172.21.15.46:0/3536908667 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c0ca5c0 con 0x7f278c0d0150 2024-03-29T08:18:18.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.728+0000 7f2789400640 1 -- 172.21.15.46:0/3536908667 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438532342 0 0) 0x7f27840014a0 con 0x7f278c0af420 2024-03-29T08:18:18.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.732+0000 7f2783e00640 1 --2- 172.21.15.46:0/3536908667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c0d0150 0x7f278c0aeee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f278c06fcc0 tx=0x7f277401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9ca131a1c49306 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.732+0000 7f2789400640 1 -- 172.21.15.46:0/3536908667 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27740ab610 con 0x7f278c0d0150 2024-03-29T08:18:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.732+0000 7f2789400640 1 -- 172.21.15.46:0/3536908667 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27740ab770 con 0x7f278c0d0150 2024-03-29T08:18:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.732+0000 7f2789400640 1 -- 172.21.15.46:0/3536908667 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27740abb00 con 0x7f278c0d0150 2024-03-29T08:18:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.732+0000 7f2790e00640 1 -- 172.21.15.46:0/3536908667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c0d0150 msgr2=0x7f278c0aeee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.732+0000 7f2790e00640 1 --2- 172.21.15.46:0/3536908667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c0d0150 0x7f278c0aeee0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f278c06fcc0 tx=0x7f277401c520 comp rx=0 tx=0).stop 2024-03-29T08:18:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.732+0000 7f2790e00640 1 -- 172.21.15.46:0/3536908667 shutdown_connections 2024-03-29T08:18:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.732+0000 7f2790e00640 1 --2- 172.21.15.46:0/3536908667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c0d0150 0x7f278c0aeee0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.732+0000 7f2790e00640 1 --2- 172.21.15.46:0/3536908667 >> v2:172.21.15.73:3300/0 conn(0x7f278c0b01c0 0x7f278c0cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.732+0000 7f2790e00640 1 -- 172.21.15.46:0/3536908667 >> 172.21.15.46:0/3536908667 conn(0x7f278c098a40 msgr2=0x7f278c0cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.732+0000 7f2790e00640 1 -- 172.21.15.46:0/3536908667 shutdown_connections 2024-03-29T08:18:18.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.732+0000 7f2790e00640 1 -- 172.21.15.46:0/3536908667 wait complete. 2024-03-29T08:18:18.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2790e00640 1 Processor -- start 2024-03-29T08:18:18.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2790e00640 1 -- start start 2024-03-29T08:18:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2790e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c0af420 0x7f278c1630b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2790e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f278c0b01c0 0x7f278c157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2790e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f278c0d0150 0x7f278c0c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:18.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2790e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0ace50 con 0x7f278c0af420 2024-03-29T08:18:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2790e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0c61f0 con 0x7f278c0d0150 2024-03-29T08:18:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2790e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1635f0 con 0x7f278c0b01c0 2024-03-29T08:18:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2789e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c0af420 0x7f278c1630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2789e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c0af420 0x7f278c1630b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59784/0 (socket says 172.21.15.46:59784) 2024-03-29T08:18:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2789e00640 1 -- 172.21.15.46:0/2988717078 learned_addr learned my addr 172.21.15.46:0/2988717078 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f278a800640 1 --2- 172.21.15.46:0/2988717078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f278c0d0150 0x7f278c0c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2783e00640 1 --2- 172.21.15.46:0/2988717078 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f278c0b01c0 0x7f278c157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2789e00640 1 -- 172.21.15.46:0/2988717078 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f278c0b01c0 msgr2=0x7f278c157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2789e00640 1 --2- 172.21.15.46:0/2988717078 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f278c0b01c0 0x7f278c157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2789e00640 1 -- 172.21.15.46:0/2988717078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f278c0d0150 msgr2=0x7f278c0c3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2789e00640 1 --2- 172.21.15.46:0/2988717078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f278c0d0150 0x7f278c0c3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2789e00640 1 -- 172.21.15.46:0/2988717078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27740094d0 con 0x7f278c0af420 2024-03-29T08:18:18.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2789e00640 1 --2- 172.21.15.46:0/2988717078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c0af420 0x7f278c1630b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f277c00e930 tx=0x7f277c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2782000640 1 -- 172.21.15.46:0/2988717078 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c009690 con 0x7f278c0af420 2024-03-29T08:18:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2782000640 1 -- 172.21.15.46:0/2988717078 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c00ca00 con 0x7f278c0af420 2024-03-29T08:18:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2782000640 1 -- 172.21.15.46:0/2988717078 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c00cd70 con 0x7f278c0af420 2024-03-29T08:18:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2790e00640 1 -- 172.21.15.46:0/2988717078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278c163820 con 0x7f278c0af420 2024-03-29T08:18:18.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.736+0000 7f2790e00640 1 -- 172.21.15.46:0/2988717078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278c0ac4b0 con 0x7f278c0af420 2024-03-29T08:18:18.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.740+0000 7f2790e00640 1 -- 172.21.15.46:0/2988717078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278c0ad0c0 con 0x7f278c0af420 2024-03-29T08:18:18.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.740+0000 7f2782000640 1 -- 172.21.15.46:0/2988717078 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f277c013470 con 0x7f278c0af420 2024-03-29T08:18:18.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.740+0000 7f2782000640 1 --2- 172.21.15.46:0/2988717078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27680666c0 0x7f2768068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:18.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.740+0000 7f2782000640 1 -- 172.21.15.46:0/2988717078 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f277c0842e0 con 0x7f278c0af420 2024-03-29T08:18:18.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.740+0000 7f2783e00640 1 --2- 172.21.15.46:0/2988717078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27680666c0 0x7f2768068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:18.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.740+0000 7f2783e00640 1 --2- 172.21.15.46:0/2988717078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27680666c0 0x7f2768068b80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27740af000 tx=0x7f27740b6040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:18.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.744+0000 7f2782000640 1 -- 172.21.15.46:0/2988717078 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f277c051690 con 0x7f278c0af420 2024-03-29T08:18:18.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.908+0000 7f2790e00640 1 -- 172.21.15.46:0/2988717078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f278c163820 con 0x7f278c0af420 2024-03-29T08:18:18.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.908+0000 7f2782000640 1 -- 172.21.15.46:0/2988717078 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f278c163820 con 0x7f278c0af420 2024-03-29T08:18:18.911 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:18.911 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:14.314124+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.912+0000 7f2790e00640 1 -- 172.21.15.46:0/2988717078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27680666c0 msgr2=0x7f2768068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.912+0000 7f2790e00640 1 --2- 172.21.15.46:0/2988717078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27680666c0 0x7f2768068b80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27740af000 tx=0x7f27740b6040 comp rx=0 tx=0).stop 2024-03-29T08:18:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.912+0000 7f2790e00640 1 -- 172.21.15.46:0/2988717078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c0af420 msgr2=0x7f278c1630b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.912+0000 7f2790e00640 1 --2- 172.21.15.46:0/2988717078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c0af420 0x7f278c1630b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f277c00e930 tx=0x7f277c00edf0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.912+0000 7f2790e00640 1 -- 172.21.15.46:0/2988717078 shutdown_connections 2024-03-29T08:18:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.912+0000 7f2790e00640 1 --2- 172.21.15.46:0/2988717078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27680666c0 0x7f2768068b80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.912+0000 7f2790e00640 1 --2- 172.21.15.46:0/2988717078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f278c0d0150 0x7f278c0c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.912+0000 7f2790e00640 1 --2- 172.21.15.46:0/2988717078 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f278c0b01c0 0x7f278c157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.912+0000 7f2790e00640 1 --2- 172.21.15.46:0/2988717078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c0af420 0x7f278c1630b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:18.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.912+0000 7f2790e00640 1 -- 172.21.15.46:0/2988717078 >> 172.21.15.46:0/2988717078 conn(0x7f278c098a40 msgr2=0x7f278c0cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:18.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.912+0000 7f2790e00640 1 -- 172.21.15.46:0/2988717078 shutdown_connections 2024-03-29T08:18:18.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:18.912+0000 7f2790e00640 1 -- 172.21.15.46:0/2988717078 wait complete. 2024-03-29T08:18:18.932 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-29T08:18:18.932 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-29T08:18:18.933 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-29T08:18:18.933 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-29T08:18:18.933 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-29T08:18:18.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-29T08:18:18.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-29T08:18:18.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-29T08:18:19.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.172+0000 7fc68ae00640 1 Processor -- start 2024-03-29T08:18:19.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.172+0000 7fc68ae00640 1 -- start start 2024-03-29T08:18:19.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.176+0000 7fc68ae00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc67c0e30a0 0x7fc67c005ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.176+0000 7fc68ae00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc67c0544d0 0x7fc67c04f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.176+0000 7fc68ae00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc67c037b90 con 0x7fc67c04fa70 2024-03-29T08:18:19.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.176+0000 7fc68ae00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc67c04f810 con 0x7fc67c0e30a0 2024-03-29T08:18:19.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.176+0000 7fc68ae00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c054290 con 0x7fc67c0544d0 2024-03-29T08:18:19.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.176+0000 7fc688e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc67c04fa70 0x7fc67c052a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:43086/0 (socket says 172.21.15.46:43086) 2024-03-29T08:18:19.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.176+0000 7fc688e00640 1 -- 172.21.15.46:0/3558733148 learned_addr learned my addr 172.21.15.46:0/3558733148 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.180+0000 7fc683400640 1 --2- 172.21.15.46:0/3558733148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc67c0544d0 0x7fc67c04f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.180+0000 7fc683e00640 1 --2- 172.21.15.46:0/3558733148 >> v2:172.21.15.73:3300/0 conn(0x7fc67c0e30a0 0x7fc67c005ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.180+0000 7fc682a00640 1 -- 172.21.15.46:0/3558733148 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369650086 0 0) 0x7fc67c037b90 con 0x7fc67c04fa70 2024-03-29T08:18:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.180+0000 7fc682a00640 1 -- 172.21.15.46:0/3558733148 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc66c003530 con 0x7fc67c04fa70 2024-03-29T08:18:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.180+0000 7fc683400640 1 -- 172.21.15.46:0/3558733148 >> v2:172.21.15.73:3300/0 conn(0x7fc67c0e30a0 msgr2=0x7fc67c005ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.180+0000 7fc683400640 1 --2- 172.21.15.46:0/3558733148 >> v2:172.21.15.73:3300/0 conn(0x7fc67c0e30a0 0x7fc67c005ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.180+0000 7fc683400640 1 -- 172.21.15.46:0/3558733148 >> v1:172.21.15.73:6789/0 conn(0x7fc67c04fa70 legacy=0x7fc67c052a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.180+0000 7fc683400640 1 -- 172.21.15.46:0/3558733148 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67c0e7820 con 0x7fc67c0544d0 2024-03-29T08:18:19.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.180+0000 7f4ce1a00640 1 Processor -- start 2024-03-29T08:18:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.180+0000 7f4ce1a00640 1 -- start start 2024-03-29T08:18:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f759f400640 1 Processor -- start 2024-03-29T08:18:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f4ce1a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4cdc0b2770 0x7f4cdc0ab240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f4ce1a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cdc0c1b00 0x7f4cdc06cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f4ce1a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cdc06cb20 con 0x7f4cdc160fe0 2024-03-29T08:18:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f4ce1a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cdc06c5a0 con 0x7f4cdc0b2770 2024-03-29T08:18:19.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f4ce1a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc1613c0 con 0x7f4cdc0c1b00 2024-03-29T08:18:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f4cdaa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4cdc160fe0 0x7f4cdc0c13f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60878/0 (socket says 172.21.15.46:60878) 2024-03-29T08:18:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f4cdaa00640 1 -- 172.21.15.46:0/295914880 learned_addr learned my addr 172.21.15.46:0/295914880 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:19.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f4cda000640 1 --2- 172.21.15.46:0/295914880 >> v2:172.21.15.73:3300/0 conn(0x7f4cdc0b2770 0x7f4cdc0ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f4cd9600640 1 -- 172.21.15.46:0/295914880 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390452499 0 0) 0x7f4cdc06cb20 con 0x7f4cdc160fe0 2024-03-29T08:18:19.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f4cd9600640 1 -- 172.21.15.46:0/295914880 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cc0003530 con 0x7f4cdc160fe0 2024-03-29T08:18:19.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f4cdb400640 1 --2- 172.21.15.46:0/295914880 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cdc0c1b00 0x7f4cdc06cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f4cd9600640 1 -- 172.21.15.46:0/295914880 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3284914727 0 0) 0x7f4cc0003530 con 0x7f4cdc160fe0 2024-03-29T08:18:19.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7fc683400640 1 --2- 172.21.15.46:0/3558733148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc67c0544d0 0x7fc67c04f2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc6780097c0 tx=0x7fc67801ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce7033ae96e4a02f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f4cda000640 1 -- 172.21.15.46:0/295914880 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cdc0c1b00 msgr2=0x7f4cdc06cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f4cda000640 1 --2- 172.21.15.46:0/295914880 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cdc0c1b00 0x7f4cdc06cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f4cda000640 1 -- 172.21.15.46:0/295914880 >> v1:172.21.15.46:6789/0 conn(0x7f4cdc160fe0 legacy=0x7f4cdc0c13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f4cda000640 1 -- 172.21.15.46:0/295914880 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cdc0ad700 con 0x7f4cdc0b2770 2024-03-29T08:18:19.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f759f400640 1 -- start start 2024-03-29T08:18:19.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f759f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7598158990 0x7f75980b1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f759f400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75980b1be0 con 0x7f75980cea30 2024-03-29T08:18:19.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f759f400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7598158d70 con 0x7f7598071fc0 2024-03-29T08:18:19.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f759f400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75980b24f0 con 0x7f7598158990 2024-03-29T08:18:19.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f7b55a00640 1 Processor -- start 2024-03-29T08:18:19.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f759d400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7598071fc0 0x7f75981624f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:43090/0 (socket says 172.21.15.46:43090) 2024-03-29T08:18:19.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f759d400640 1 -- 172.21.15.46:0/2323198884 learned_addr learned my addr 172.21.15.46:0/2323198884 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:19.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f7597e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f75980cea30 0x7f75980718b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60884/0 (socket says 172.21.15.46:60884) 2024-03-29T08:18:19.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f759ca00640 1 --2- 172.21.15.46:0/2323198884 >> v2:172.21.15.73:3300/0 conn(0x7f7598158990 0x7f75980b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597400640 1 -- 172.21.15.46:0/2323198884 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978770247 0 0) 0x7f7598158d70 con 0x7f7598071fc0 2024-03-29T08:18:19.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.184+0000 7f7b55a00640 1 -- start start 2024-03-29T08:18:19.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597400640 1 -- 172.21.15.46:0/2323198884 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f757c003530 con 0x7f7598071fc0 2024-03-29T08:18:19.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597400640 1 -- 172.21.15.46:0/2323198884 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289837137 0 0) 0x7f75980b1be0 con 0x7f75980cea30 2024-03-29T08:18:19.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597400640 1 -- 172.21.15.46:0/2323198884 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7598158d70 con 0x7f75980cea30 2024-03-29T08:18:19.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7fc682a00640 1 -- 172.21.15.46:0/3558733148 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6780a8990 con 0x7fc67c0544d0 2024-03-29T08:18:19.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7fc682a00640 1 -- 172.21.15.46:0/3558733148 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6780a8af0 con 0x7fc67c0544d0 2024-03-29T08:18:19.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7fc682a00640 1 -- 172.21.15.46:0/3558733148 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6780a8e80 con 0x7fc67c0544d0 2024-03-29T08:18:19.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7fc68ae00640 1 -- 172.21.15.46:0/3558733148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc67c0544d0 msgr2=0x7fc67c04f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7fc68ae00640 1 --2- 172.21.15.46:0/3558733148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc67c0544d0 0x7fc67c04f2d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc6780097c0 tx=0x7fc67801ec90 comp rx=0 tx=0).stop 2024-03-29T08:18:19.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597400640 1 -- 172.21.15.46:0/2323198884 <== mon.1 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1183193574 0 0) 0x7f757c003530 con 0x7f7598071fc0 2024-03-29T08:18:19.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597400640 1 -- 172.21.15.46:0/2323198884 >> v2:172.21.15.73:3300/0 conn(0x7f7598158990 msgr2=0x7f75980b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597400640 1 --2- 172.21.15.46:0/2323198884 >> v2:172.21.15.73:3300/0 conn(0x7f7598158990 0x7f75980b1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597400640 1 -- 172.21.15.46:0/2323198884 >> v1:172.21.15.46:6789/0 conn(0x7f75980cea30 legacy=0x7f75980718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597400640 1 -- 172.21.15.46:0/2323198884 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75980b1860 con 0x7f7598071fc0 2024-03-29T08:18:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7b55a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b500b3050 0x7f7b500b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7b55a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b50156a40 con 0x7f7b500702a0 2024-03-29T08:18:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7b55a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b5015c6c0 con 0x7f7b50066100 2024-03-29T08:18:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7b55a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5015c9a0 con 0x7f7b500b3050 2024-03-29T08:18:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7b4e000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b500b3050 0x7f7b500b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7b4e000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b500b3050 0x7f7b500b34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:57486/0 (socket says 172.21.15.46:57486) 2024-03-29T08:18:19.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7b4e000640 1 -- 172.21.15.46:0/2157060375 learned_addr learned my addr 172.21.15.46:0/2157060375 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7ff296a00640 1 Processor -- start 2024-03-29T08:18:19.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7ff296a00640 1 -- start start 2024-03-29T08:18:19.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597400640 1 -- 172.21.15.46:0/2323198884 <== mon.1 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f7588000ee0 con 0x7f7598071fc0 2024-03-29T08:18:19.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597400640 1 -- 172.21.15.46:0/2323198884 >> v1:172.21.15.73:6789/0 conn(0x7f7598071fc0 legacy=0x7f75981624f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597400640 1 --2- 172.21.15.46:0/2323198884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f757c003b60 0x7f757c003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597400640 1 --2- 172.21.15.46:0/2323198884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f757c0047d0 0x7f757c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597400640 1 --2- 172.21.15.46:0/2323198884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f757c005190 0x7f757c010020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597400640 1 -- 172.21.15.46:0/2323198884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75980b24f0 con 0x7f757c003b60 2024-03-29T08:18:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597400640 1 -- 172.21.15.46:0/2323198884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c007bf0 con 0x7f757c005190 2024-03-29T08:18:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597400640 1 -- 172.21.15.46:0/2323198884 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c007db0 con 0x7f757c0047d0 2024-03-29T08:18:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597e00640 1 --2- 172.21.15.46:0/2323198884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f757c0047d0 0x7f757c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f759d400640 1 --2- 172.21.15.46:0/2323198884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f757c005190 0x7f757c010020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f759ca00640 1 --2- 172.21.15.46:0/2323198884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f757c003b60 0x7f757c003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7b4e000640 1 -- 172.21.15.46:0/2157060375 >> v1:172.21.15.73:6789/0 conn(0x7f7b50066100 legacy=0x7f7b50156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7b4e000640 1 -- 172.21.15.46:0/2157060375 >> v1:172.21.15.46:6789/0 conn(0x7f7b500702a0 legacy=0x7f7b50159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7b4e000640 1 -- 172.21.15.46:0/2157060375 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5015cc80 con 0x7f7b500b3050 2024-03-29T08:18:19.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7fc68ae00640 1 -- 172.21.15.46:0/3558733148 shutdown_connections 2024-03-29T08:18:19.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7fc68ae00640 1 --2- 172.21.15.46:0/3558733148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc67c0544d0 0x7fc67c04f2d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7fc68ae00640 1 --2- 172.21.15.46:0/3558733148 >> v2:172.21.15.73:3300/0 conn(0x7fc67c0e30a0 0x7fc67c005ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7fc68ae00640 1 -- 172.21.15.46:0/3558733148 >> 172.21.15.46:0/3558733148 conn(0x7fc67c009f90 msgr2=0x7fc67c00a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597e00640 1 -- 172.21.15.46:0/2323198884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f757c005190 msgr2=0x7f757c010020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597e00640 1 --2- 172.21.15.46:0/2323198884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f757c005190 0x7f757c010020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597e00640 1 -- 172.21.15.46:0/2323198884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f757c003b60 msgr2=0x7f757c003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597e00640 1 --2- 172.21.15.46:0/2323198884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f757c003b60 0x7f757c003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f7597e00640 1 -- 172.21.15.46:0/2323198884 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75980b1860 con 0x7f757c0047d0 2024-03-29T08:18:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.188+0000 7f4cda000640 1 --2- 172.21.15.46:0/295914880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cdc0b2770 0x7f4cdc0ab240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80097c0 tx=0x7f4cc801c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7421032c76590e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7f759d400640 1 --2- 172.21.15.46:0/2323198884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f757c005190 0x7f757c010020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7f759ca00640 1 --2- 172.21.15.46:0/2323198884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f757c003b60 0x7f757c003f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:19.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7f759d400640 1 -- 172.21.15.46:0/2323198884 reap_dead start 2024-03-29T08:18:19.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7f4cd9600640 1 -- 172.21.15.46:0/295914880 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4001a10 con 0x7f4cdc0b2770 2024-03-29T08:18:19.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7f4cd9600640 1 -- 172.21.15.46:0/295914880 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc80a7050 con 0x7f4cdc0b2770 2024-03-29T08:18:19.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7f4cd9600640 1 -- 172.21.15.46:0/295914880 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4001a10 con 0x7f4cdc0b2770 2024-03-29T08:18:19.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7ff296a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2900702a0 0x7ff290070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7ff296a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff290070c40 con 0x7ff290159030 2024-03-29T08:18:19.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7ff296a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29015c6c0 con 0x7ff290156590 2024-03-29T08:18:19.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7ff296a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29015c9a0 con 0x7ff2900702a0 2024-03-29T08:18:19.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7fc68ae00640 1 -- 172.21.15.46:0/3558733148 shutdown_connections 2024-03-29T08:18:19.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7ff28f400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2900702a0 0x7ff290070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7f3372000640 1 Processor -- start 2024-03-29T08:18:19.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7ff294a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff290159030 0x7ff2901594c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60894/0 (socket says 172.21.15.46:60894) 2024-03-29T08:18:19.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7ff28f400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2900702a0 0x7ff290070700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:57504/0 (socket says 172.21.15.46:57504) 2024-03-29T08:18:19.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7ff28f400640 1 -- 172.21.15.46:0/979628231 learned_addr learned my addr 172.21.15.46:0/979628231 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7f3372000640 1 -- start start 2024-03-29T08:18:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7fc68ae00640 1 -- 172.21.15.46:0/3558733148 wait complete. 2024-03-29T08:18:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7ff28f400640 1 -- 172.21.15.46:0/979628231 >> v1:172.21.15.73:6789/0 conn(0x7ff290156590 legacy=0x7ff2900b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7ff28f400640 1 -- 172.21.15.46:0/979628231 >> v1:172.21.15.46:6789/0 conn(0x7ff290159030 legacy=0x7ff2901594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7ff28f400640 1 -- 172.21.15.46:0/979628231 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29015cc80 con 0x7ff2900702a0 2024-03-29T08:18:19.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7f4ce1a00640 1 -- 172.21.15.46:0/295914880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cdc0b2770 msgr2=0x7f4cdc0ab240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7f4ce1a00640 1 --2- 172.21.15.46:0/295914880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cdc0b2770 0x7f4cdc0ab240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80097c0 tx=0x7f4cc801c330 comp rx=0 tx=0).stop 2024-03-29T08:18:19.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.192+0000 7f4ce1a00640 1 -- 172.21.15.46:0/295914880 shutdown_connections 2024-03-29T08:18:19.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f4ce1a00640 1 --2- 172.21.15.46:0/295914880 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cdc0c1b00 0x7f4cdc06cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f4ce1a00640 1 --2- 172.21.15.46:0/295914880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cdc0b2770 0x7f4cdc0ab240 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f4ce1a00640 1 -- 172.21.15.46:0/295914880 >> 172.21.15.46:0/295914880 conn(0x7f4cdc0affd0 msgr2=0x7f4cdc0b03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f7b4e000640 1 --2- 172.21.15.46:0/2157060375 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b500b3050 0x7f7b500b34b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7b400098a0 tx=0x7f7b4001c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ca418751ff54ca0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f4ce1a00640 1 -- 172.21.15.46:0/295914880 shutdown_connections 2024-03-29T08:18:19.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f7597e00640 1 --2- 172.21.15.46:0/2323198884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f757c0047d0 0x7f757c004c50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7590002200 tx=0x7f759001ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ed9e63f2bcfd0a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f4ce1a00640 1 -- 172.21.15.46:0/295914880 wait complete. 2024-03-29T08:18:19.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f3372000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3364034130 0x7f33640e3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7fc4e1e00640 1 Processor -- start 2024-03-29T08:18:19.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7fc68ae00640 1 Processor -- start 2024-03-29T08:18:19.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f7597400640 1 -- 172.21.15.46:0/2323198884 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75900a5870 con 0x7f757c0047d0 2024-03-29T08:18:19.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f7597400640 1 -- 172.21.15.46:0/2323198884 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75900a59f0 con 0x7f757c0047d0 2024-03-29T08:18:19.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f7597400640 1 -- 172.21.15.46:0/2323198884 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75900a5da0 con 0x7f757c0047d0 2024-03-29T08:18:19.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f7b4d600640 1 -- 172.21.15.46:0/2157060375 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b400a7070 con 0x7f7b500b3050 2024-03-29T08:18:19.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f7b4d600640 1 -- 172.21.15.46:0/2157060375 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b400ab430 con 0x7f7b500b3050 2024-03-29T08:18:19.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f7b4d600640 1 -- 172.21.15.46:0/2157060375 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b400ab7c0 con 0x7f7b500b3050 2024-03-29T08:18:19.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f3372000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f33640e36a0 0x7f3364006920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f3372000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3364006e60 0x7f33640dfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f3372000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33640072c0 con 0x7f33640e36a0 2024-03-29T08:18:19.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f3370c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3364034130 0x7f33640e3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f3370c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3364034130 0x7f33640e3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:57520/0 (socket says 172.21.15.46:57520) 2024-03-29T08:18:19.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f3370c00640 1 -- 172.21.15.46:0/2222096081 learned_addr learned my addr 172.21.15.46:0/2222096081 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f30daa00640 1 Processor -- start 2024-03-29T08:18:19.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f30daa00640 1 -- start start 2024-03-29T08:18:19.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7fc4e1e00640 1 -- start start 2024-03-29T08:18:19.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f3372000640 1 -- 172.21.15.46:0/2222096081 --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3364052e70 con 0x7f3364006e60 2024-03-29T08:18:19.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f3373600640 1 --2- 172.21.15.46:0/2222096081 >> v2:172.21.15.73:3300/0 conn(0x7f3364006e60 0x7f33640dfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f3372000640 1 -- 172.21.15.46:0/2222096081 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3364052840 con 0x7f3364034130 2024-03-29T08:18:19.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f759f400640 1 -- 172.21.15.46:0/2323198884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f757c0047d0 msgr2=0x7f757c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f759f400640 1 --2- 172.21.15.46:0/2323198884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f757c0047d0 0x7f757c004c50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7590002200 tx=0x7f759001ca90 comp rx=0 tx=0).stop 2024-03-29T08:18:19.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f7b55a00640 1 -- 172.21.15.46:0/2157060375 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b500b3050 msgr2=0x7f7b500b34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f7b55a00640 1 --2- 172.21.15.46:0/2157060375 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b500b3050 0x7f7b500b34b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7b400098a0 tx=0x7f7b4001c4d0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f336be00640 1 --2- 172.21.15.46:0/2222096081 >> v2:172.21.15.46:3300/0 conn(0x7f33640e36a0 0x7f3364006920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7fc68ae00640 1 -- start start 2024-03-29T08:18:19.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f7b55a00640 1 -- 172.21.15.46:0/2157060375 shutdown_connections 2024-03-29T08:18:19.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f759f400640 1 -- 172.21.15.46:0/2323198884 shutdown_connections 2024-03-29T08:18:19.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f7b55a00640 1 --2- 172.21.15.46:0/2157060375 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b500b3050 0x7f7b500b34b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f759f400640 1 --2- 172.21.15.46:0/2323198884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f757c0047d0 0x7f757c004c50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f7b55a00640 1 -- 172.21.15.46:0/2157060375 >> 172.21.15.46:0/2157060375 conn(0x7f7b50065c50 msgr2=0x7f7b50069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f759f400640 1 -- 172.21.15.46:0/2323198884 >> 172.21.15.46:0/2323198884 conn(0x7f75980b3730 msgr2=0x7f75980b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f4ce1a00640 1 Processor -- start 2024-03-29T08:18:19.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f7b55a00640 1 -- 172.21.15.46:0/2157060375 shutdown_connections 2024-03-29T08:18:19.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7ff28f400640 1 --2- 172.21.15.46:0/979628231 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2900702a0 0x7ff290070700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff2800088a0 tx=0x7ff28001ea20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cac4384c858acc17 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f3370c00640 1 -- 172.21.15.46:0/2222096081 >> v2:172.21.15.73:3300/0 conn(0x7f3364006e60 msgr2=0x7f33640dfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f3370c00640 1 --2- 172.21.15.46:0/2222096081 >> v2:172.21.15.73:3300/0 conn(0x7f3364006e60 0x7f33640dfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f3370c00640 1 -- 172.21.15.46:0/2222096081 >> v2:172.21.15.46:3300/0 conn(0x7f33640e36a0 msgr2=0x7f3364006920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f3370c00640 1 --2- 172.21.15.46:0/2222096081 >> v2:172.21.15.46:3300/0 conn(0x7f33640e36a0 0x7f3364006920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f3370c00640 1 -- 172.21.15.46:0/2222096081 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33640e3c20 con 0x7f3364034130 2024-03-29T08:18:19.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7fc4e1e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc4dc156590 0x7fc4dc0702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7fc4e1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc4dc0707e0 0x7fc4dc1637a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f7b55a00640 1 -- 172.21.15.46:0/2157060375 wait complete. 2024-03-29T08:18:19.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f4ce1a00640 1 -- start start 2024-03-29T08:18:19.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7ff28ea00640 1 -- 172.21.15.46:0/979628231 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2800a7b10 con 0x7ff2900702a0 2024-03-29T08:18:19.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7ff28ea00640 1 -- 172.21.15.46:0/979628231 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2800ab070 con 0x7ff2900702a0 2024-03-29T08:18:19.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7ff28ea00640 1 -- 172.21.15.46:0/979628231 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2800a7e90 con 0x7ff2900702a0 2024-03-29T08:18:19.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f30daa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f30d41434f0 0x7f30d40702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f30daa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30d40707e0 0x7f30d40af5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f30daa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d4070c40 con 0x7f30d40afae0 2024-03-29T08:18:19.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f30daa00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d40adfb0 con 0x7f30d41434f0 2024-03-29T08:18:19.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7f30daa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40aeb30 con 0x7f30d40707e0 2024-03-29T08:18:19.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7fc4e1e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4dc15c6c0 con 0x7fc4dc066100 2024-03-29T08:18:19.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7fc4e1e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4dc070c50 con 0x7fc4dc0707e0 2024-03-29T08:18:19.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7fc4e1e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4dc15c9a0 con 0x7fc4dc156590 2024-03-29T08:18:19.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7fc4da000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc4dc156590 0x7fc4dc0702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7fc68ae00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc67c04fa70 0x7fc67c0ddf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7fc68ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc67c0544d0 0x7fc67c048fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7fc68ae00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc67c0e30a0 0x7fc67c0e59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7fc68ae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c04a6e0 con 0x7fc67c0544d0 2024-03-29T08:18:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7fc68ae00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c049a60 con 0x7fc67c0e30a0 2024-03-29T08:18:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7fc68ae00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c049d40 con 0x7fc67c04fa70 2024-03-29T08:18:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc683e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc67c04fa70 0x7fc67c0ddf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc683e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc67c04fa70 0x7fc67c0ddf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:57522/0 (socket says 172.21.15.46:57522) 2024-03-29T08:18:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc683e00640 1 -- 172.21.15.46:0/3412408046 learned_addr learned my addr 172.21.15.46:0/3412408046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f30d3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f30d41434f0 0x7f30d40702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f30d3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f30d41434f0 0x7f30d40702a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51754/0 (socket says 172.21.15.46:51754) 2024-03-29T08:18:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f30d3e00640 1 -- 172.21.15.46:0/2560449806 learned_addr learned my addr 172.21.15.46:0/2560449806 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.196+0000 7fc4da000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc4dc156590 0x7fc4dc0702a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51748/0 (socket says 172.21.15.46:51748) 2024-03-29T08:18:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4db400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc4dc0707e0 0x7fc4dc1637a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4da000640 1 -- 172.21.15.46:0/882567033 learned_addr learned my addr 172.21.15.46:0/882567033 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f30cb400640 1 --2- 172.21.15.46:0/2560449806 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30d40707e0 0x7f30d40af5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc683400640 1 --2- 172.21.15.46:0/3412408046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc67c0544d0 0x7fc67c048fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4ce1a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cdc0b2770 0x7f4cdc0c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4ce1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cdc0c1b00 0x7f4cdc071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4ce1a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cdc160fe0 0x7f4cdc1538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4ce1a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc0c5060 con 0x7f4cdc0c1b00 2024-03-29T08:18:19.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4ce1a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc0c44e0 con 0x7f4cdc160fe0 2024-03-29T08:18:19.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4ce1a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cdc0c47c0 con 0x7f4cdc0b2770 2024-03-29T08:18:19.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4cdb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cdc160fe0 0x7f4cdc1538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4cdb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cdc160fe0 0x7f4cdc1538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51776/0 (socket says 172.21.15.46:51776) 2024-03-29T08:18:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4cdb400640 1 -- 172.21.15.46:0/3053368297 learned_addr learned my addr 172.21.15.46:0/3053368297 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4cdaa00640 1 --2- 172.21.15.46:0/3053368297 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cdc0b2770 0x7f4cdc0c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7ff296a00640 1 -- 172.21.15.46:0/979628231 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2900702a0 msgr2=0x7ff290070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7ff296a00640 1 --2- 172.21.15.46:0/979628231 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2900702a0 0x7ff290070700 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff2800088a0 tx=0x7ff28001ea20 comp rx=0 tx=0).stop 2024-03-29T08:18:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7ff296a00640 1 -- 172.21.15.46:0/979628231 shutdown_connections 2024-03-29T08:18:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7ff296a00640 1 --2- 172.21.15.46:0/979628231 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2900702a0 0x7ff290070700 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7ff296a00640 1 -- 172.21.15.46:0/979628231 >> 172.21.15.46:0/979628231 conn(0x7ff290065c50 msgr2=0x7ff290069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f30cb400640 1 -- 172.21.15.46:0/2560449806 >> v2:172.21.15.73:3300/0 conn(0x7f30d41434f0 msgr2=0x7f30d40702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f30cb400640 1 --2- 172.21.15.46:0/2560449806 >> v2:172.21.15.73:3300/0 conn(0x7f30d41434f0 0x7f30d40702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f30cb400640 1 -- 172.21.15.46:0/2560449806 >> v1:172.21.15.46:6789/0 conn(0x7f30d40afae0 legacy=0x7f30d415e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f30cb400640 1 -- 172.21.15.46:0/2560449806 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d406d3a0 con 0x7f30d40707e0 2024-03-29T08:18:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4d9600640 1 -- 172.21.15.46:0/882567033 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280736780 0 0) 0x7fc4dc15c6c0 con 0x7fc4dc066100 2024-03-29T08:18:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4d9600640 1 -- 172.21.15.46:0/882567033 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4c0003530 con 0x7fc4dc066100 2024-03-29T08:18:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4d9600640 1 -- 172.21.15.46:0/882567033 <== mon.1 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 705726091 0 0) 0x7fc4c0003530 con 0x7fc4dc066100 2024-03-29T08:18:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4d9600640 1 -- 172.21.15.46:0/882567033 >> v2:172.21.15.73:3300/0 conn(0x7fc4dc156590 msgr2=0x7fc4dc0702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4d9600640 1 --2- 172.21.15.46:0/882567033 >> v2:172.21.15.73:3300/0 conn(0x7fc4dc156590 0x7fc4dc0702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4d9600640 1 -- 172.21.15.46:0/882567033 >> v2:172.21.15.46:3300/0 conn(0x7fc4dc0707e0 msgr2=0x7fc4dc1637a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4d9600640 1 --2- 172.21.15.46:0/882567033 >> v2:172.21.15.46:3300/0 conn(0x7fc4dc0707e0 0x7fc4dc1637a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4d9600640 1 -- 172.21.15.46:0/882567033 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc15cc80 con 0x7fc4dc066100 2024-03-29T08:18:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4d9600640 1 -- 172.21.15.46:0/882567033 <== mon.1 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fc4c4001680 con 0x7fc4dc066100 2024-03-29T08:18:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4da000640 1 --2- 172.21.15.46:0/882567033 >> v2:172.21.15.73:3300/0 conn(0x7fc4dc156590 0x7fc4dc0702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f30d3e00640 1 --2- 172.21.15.46:0/2560449806 >> v2:172.21.15.73:3300/0 conn(0x7f30d41434f0 0x7f30d40702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4d9600640 1 -- 172.21.15.46:0/882567033 >> v1:172.21.15.73:6789/0 conn(0x7fc4dc066100 legacy=0x7fc4dc0b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4cdb400640 1 -- 172.21.15.46:0/3053368297 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cdc0b2770 msgr2=0x7f4cdc0c3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4cdb400640 1 --2- 172.21.15.46:0/3053368297 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cdc0b2770 0x7f4cdc0c3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4cdb400640 1 -- 172.21.15.46:0/3053368297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cdc0c1b00 msgr2=0x7f4cdc071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4cdb400640 1 --2- 172.21.15.46:0/3053368297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cdc0c1b00 0x7f4cdc071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4cdb400640 1 -- 172.21.15.46:0/3053368297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc8009420 con 0x7f4cdc160fe0 2024-03-29T08:18:19.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f7b55a00640 1 Processor -- start 2024-03-29T08:18:19.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4cdb400640 1 --2- 172.21.15.46:0/3053368297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cdc160fe0 0x7f4cdc1538a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00e7c0 tx=0x7f4ccc00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc683e00640 1 -- 172.21.15.46:0/3412408046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc67c0e30a0 msgr2=0x7fc67c0e59d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-29T08:18:19.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc683e00640 1 --2- 172.21.15.46:0/3412408046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc67c0e30a0 0x7fc67c0e59d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc683e00640 1 -- 172.21.15.46:0/3412408046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc67c0544d0 msgr2=0x7fc67c048fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc683e00640 1 --2- 172.21.15.46:0/3412408046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc67c0544d0 0x7fc67c048fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc683e00640 1 -- 172.21.15.46:0/3412408046 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc678009420 con 0x7fc67c04fa70 2024-03-29T08:18:19.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f7b55a00640 1 -- start start 2024-03-29T08:18:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4d9600640 1 --2- 172.21.15.46:0/882567033 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4c0003980 0x7fc4c0003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4d9600640 1 --2- 172.21.15.46:0/882567033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4c00047d0 0x7fc4c0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4d9600640 1 --2- 172.21.15.46:0/882567033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c0005190 0x7fc4c000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4d9600640 1 -- 172.21.15.46:0/882567033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc070c50 con 0x7fc4c00047d0 2024-03-29T08:18:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4d9600640 1 -- 172.21.15.46:0/882567033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc15c9a0 con 0x7fc4c0005190 2024-03-29T08:18:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4d9600640 1 -- 172.21.15.46:0/882567033 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c000c000 con 0x7fc4c0003980 2024-03-29T08:18:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4db400640 1 --2- 172.21.15.46:0/882567033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c0005190 0x7fc4c000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4daa00640 1 --2- 172.21.15.46:0/882567033 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4c0003980 0x7fc4c0003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4cd2a00640 1 -- 172.21.15.46:0/3053368297 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc0094d0 con 0x7f4cdc160fe0 2024-03-29T08:18:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4cd2a00640 1 -- 172.21.15.46:0/3053368297 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ccc00ee70 con 0x7f4cdc160fe0 2024-03-29T08:18:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4cd2a00640 1 -- 172.21.15.46:0/3053368297 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc00cb60 con 0x7f4cdc160fe0 2024-03-29T08:18:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4ce1a00640 1 -- 172.21.15.46:0/3053368297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cdc0c4aa0 con 0x7f4cdc160fe0 2024-03-29T08:18:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc683e00640 1 --2- 172.21.15.46:0/3412408046 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc67c04fa70 0x7fc67c0ddf10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc67400d5f0 tx=0x7fc67400dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f759f400640 1 -- 172.21.15.46:0/2323198884 shutdown_connections 2024-03-29T08:18:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f4ce1a00640 1 -- 172.21.15.46:0/3053368297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cdc153de0 con 0x7f4cdc160fe0 2024-03-29T08:18:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7f7b55a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b50066100 0x7f7b500ad440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b55a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b500702a0 0x7f7b501545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b55a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b500b3050 0x7f7b500a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b55a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b500aaf60 con 0x7f7b50066100 2024-03-29T08:18:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b55a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b50155740 con 0x7f7b500702a0 2024-03-29T08:18:19.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b55a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b50154ea0 con 0x7f7b500b3050 2024-03-29T08:18:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7ff296a00640 1 -- 172.21.15.46:0/979628231 shutdown_connections 2024-03-29T08:18:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7ff296a00640 1 -- 172.21.15.46:0/979628231 wait complete. 2024-03-29T08:18:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b4ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b50066100 0x7f7b500ad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b4e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b500702a0 0x7f7b501545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b4e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b500702a0 0x7f7b501545a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51806/0 (socket says 172.21.15.46:51806) 2024-03-29T08:18:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b4e000640 1 -- 172.21.15.46:0/2961805659 learned_addr learned my addr 172.21.15.46:0/2961805659 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7fc673e00640 1 -- 172.21.15.46:0/3412408046 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6740125f0 con 0x7fc67c04fa70 2024-03-29T08:18:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7ff296a00640 1 Processor -- start 2024-03-29T08:18:19.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b4f400640 1 --2- 172.21.15.46:0/2961805659 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b500b3050 0x7f7b500a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7fc68ae00640 1 -- 172.21.15.46:0/3412408046 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc67c04a020 con 0x7fc67c04fa70 2024-03-29T08:18:19.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7fc68ae00640 1 -- 172.21.15.46:0/3412408046 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc67c0de450 con 0x7fc67c04fa70 2024-03-29T08:18:19.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7ff296a00640 1 -- start start 2024-03-29T08:18:19.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f759f400640 1 -- 172.21.15.46:0/2323198884 wait complete. 2024-03-29T08:18:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7fc673e00640 1 -- 172.21.15.46:0/3412408046 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc674012750 con 0x7fc67c04fa70 2024-03-29T08:18:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f3370c00640 1 --2- 172.21.15.46:0/2222096081 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3364034130 0x7f33640e3160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f336404bca0 tx=0x7f336001efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53b9d977ce02a12b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b4f400640 1 -- 172.21.15.46:0/2961805659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b500702a0 msgr2=0x7f7b501545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b4f400640 1 --2- 172.21.15.46:0/2961805659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b500702a0 0x7f7b501545a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b4f400640 1 -- 172.21.15.46:0/2961805659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b50066100 msgr2=0x7f7b500ad440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b4f400640 1 --2- 172.21.15.46:0/2961805659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b50066100 0x7f7b500ad440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b4f400640 1 -- 172.21.15.46:0/2961805659 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b400094d0 con 0x7f7b500b3050 2024-03-29T08:18:19.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b4ea00640 1 --2- 172.21.15.46:0/2961805659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b50066100 0x7f7b500ad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:19.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7fc673e00640 1 -- 172.21.15.46:0/3412408046 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6740125f0 con 0x7fc67c04fa70 2024-03-29T08:18:19.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b4f400640 1 --2- 172.21.15.46:0/2961805659 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b500b3050 0x7f7b500a8ca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800c7b0 tx=0x7f7b4800cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f336b400640 1 -- 172.21.15.46:0/2222096081 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33600a8d80 con 0x7f3364034130 2024-03-29T08:18:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f336b400640 1 -- 172.21.15.46:0/2222096081 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3360004030 con 0x7f3364034130 2024-03-29T08:18:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f336b400640 1 -- 172.21.15.46:0/2222096081 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3360004420 con 0x7f3364034130 2024-03-29T08:18:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7fc68ae00640 1 -- 172.21.15.46:0/3412408046 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc660000fd0 con 0x7fc67c04fa70 2024-03-29T08:18:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4db400640 1 -- 172.21.15.46:0/882567033 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4c0003980 msgr2=0x7fc4c0003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4db400640 1 --2- 172.21.15.46:0/882567033 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4c0003980 0x7fc4c0003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4db400640 1 -- 172.21.15.46:0/882567033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4c00047d0 msgr2=0x7fc4c0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4db400640 1 --2- 172.21.15.46:0/882567033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4c00047d0 0x7fc4c0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.200+0000 7fc4db400640 1 -- 172.21.15.46:0/882567033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c0003530 con 0x7fc4c0005190 2024-03-29T08:18:19.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b46a00640 1 -- 172.21.15.46:0/2961805659 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48003d90 con 0x7f7b500b3050 2024-03-29T08:18:19.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b46a00640 1 -- 172.21.15.46:0/2961805659 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4800ce60 con 0x7f7b500b3050 2024-03-29T08:18:19.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b46a00640 1 -- 172.21.15.46:0/2961805659 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b480052b0 con 0x7f7b500b3050 2024-03-29T08:18:19.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b55a00640 1 -- 172.21.15.46:0/2961805659 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b50155180 con 0x7f7b500b3050 2024-03-29T08:18:19.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f759f400640 1 Processor -- start 2024-03-29T08:18:19.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f30cb400640 1 --2- 172.21.15.46:0/2560449806 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30d40707e0 0x7f30d40af5a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0097c0 tx=0x7f30bc01c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fee9ac0142a0206 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f759f400640 1 -- start start 2024-03-29T08:18:19.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f4ce1a00640 1 -- 172.21.15.46:0/3053368297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4cdc06dd00 con 0x7f4cdc160fe0 2024-03-29T08:18:19.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f7b55a00640 1 -- 172.21.15.46:0/2961805659 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b500ab190 con 0x7f7b500b3050 2024-03-29T08:18:19.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7ff296a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2900702a0 0x7ff2902058d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7ff296a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff290156590 0x7ff2901575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f3372000640 1 -- 172.21.15.46:0/2222096081 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3364034130 msgr2=0x7f33640e3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f3372000640 1 --2- 172.21.15.46:0/2222096081 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3364034130 0x7f33640e3160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f336404bca0 tx=0x7f336001efb0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f3372000640 1 -- 172.21.15.46:0/2222096081 shutdown_connections 2024-03-29T08:18:19.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f3372000640 1 --2- 172.21.15.46:0/2222096081 >> v2:172.21.15.73:3300/0 conn(0x7f3364006e60 0x7f33640dfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f3372000640 1 --2- 172.21.15.46:0/2222096081 >> v2:172.21.15.46:3300/0 conn(0x7f33640e36a0 0x7f3364006920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f3372000640 1 --2- 172.21.15.46:0/2222096081 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3364034130 0x7f33640e3160 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7f3372000640 1 -- 172.21.15.46:0/2222096081 >> 172.21.15.46:0/2222096081 conn(0x7f3364054c70 msgr2=0x7f3364055080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.204+0000 7ff296a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff290159030 0x7ff290154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7ff28f400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff290156590 0x7ff2901575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f7b55a00640 1 -- 172.21.15.46:0/2961805659 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7b5006dcb0 con 0x7f7b500b3050 2024-03-29T08:18:19.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f3372000640 1 -- 172.21.15.46:0/2222096081 shutdown_connections 2024-03-29T08:18:19.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7ff28f400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff290156590 0x7ff2901575d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:57560/0 (socket says 172.21.15.46:57560) 2024-03-29T08:18:19.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7ff28f400640 1 -- 172.21.15.46:0/2390635371 learned_addr learned my addr 172.21.15.46:0/2390635371 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7ff28fe00640 1 --2- 172.21.15.46:0/2390635371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2900702a0 0x7ff2902058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7ff294a00640 1 --2- 172.21.15.46:0/2390635371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff290159030 0x7ff290154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f3372000640 1 -- 172.21.15.46:0/2222096081 wait complete. 2024-03-29T08:18:19.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7ff296a00640 1 -- 172.21.15.46:0/2390635371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff290205e10 con 0x7ff2900702a0 2024-03-29T08:18:19.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7ff296a00640 1 -- 172.21.15.46:0/2390635371 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff290158ec0 con 0x7ff290159030 2024-03-29T08:18:19.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f759f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f757c0047d0 0x7f75980ab540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7ff296a00640 1 -- 172.21.15.46:0/2390635371 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff290157e20 con 0x7ff290156590 2024-03-29T08:18:19.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f759f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7598156160 0x7f75981565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f759ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f757c0047d0 0x7f75980ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f759f400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75980974f0 0x7f7598097970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f7597e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7598156160 0x7f75981565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f7597e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7598156160 0x7f75981565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59848/0 (socket says 172.21.15.46:59848) 2024-03-29T08:18:19.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7fc673e00640 1 -- 172.21.15.46:0/3412408046 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc6740128c0 con 0x7fc67c04fa70 2024-03-29T08:18:19.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f4cd2a00640 1 -- 172.21.15.46:0/3053368297 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4ccc00ccc0 con 0x7f4cdc160fe0 2024-03-29T08:18:19.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f7597e00640 1 -- 172.21.15.46:0/4152979536 learned_addr learned my addr 172.21.15.46:0/4152979536 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f759f400640 1 -- 172.21.15.46:0/4152979536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75981506a0 con 0x7f7598156160 2024-03-29T08:18:19.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f759f400640 1 -- 172.21.15.46:0/4152979536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759814fab0 con 0x7f757c0047d0 2024-03-29T08:18:19.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f759f400640 1 -- 172.21.15.46:0/4152979536 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7598156b20 con 0x7f75980974f0 2024-03-29T08:18:19.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f7597e00640 1 -- 172.21.15.46:0/4152979536 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75980974f0 msgr2=0x7f7598097970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f7597e00640 1 --2- 172.21.15.46:0/4152979536 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75980974f0 0x7f7598097970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f7597e00640 1 -- 172.21.15.46:0/4152979536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f757c0047d0 msgr2=0x7f75980ab540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f7597e00640 1 --2- 172.21.15.46:0/4152979536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f757c0047d0 0x7f75980ab540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f7597e00640 1 -- 172.21.15.46:0/4152979536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7598097eb0 con 0x7f7598156160 2024-03-29T08:18:19.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f759ca00640 1 --2- 172.21.15.46:0/4152979536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f757c0047d0 0x7f75980ab540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:19.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f7597e00640 1 --2- 172.21.15.46:0/4152979536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7598156160 0x7f75981565e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7590000f90 tx=0x7f757c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f4cd2a00640 1 --2- 172.21.15.46:0/3053368297 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4cbc0666e0 0x7f4cbc068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f4cd2a00640 1 -- 172.21.15.46:0/3053368297 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ccc084090 con 0x7f4cdc160fe0 2024-03-29T08:18:19.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f7b46a00640 1 -- 172.21.15.46:0/2961805659 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7b48006b20 con 0x7f7b500b3050 2024-03-29T08:18:19.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f30d3400640 1 -- 172.21.15.46:0/2560449806 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc0ab600 con 0x7f30d40707e0 2024-03-29T08:18:19.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7ff28f400640 1 -- 172.21.15.46:0/2390635371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff290159030 msgr2=0x7ff290154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7ff28f400640 1 --2- 172.21.15.46:0/2390635371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff290159030 0x7ff290154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7ff28f400640 1 -- 172.21.15.46:0/2390635371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2900702a0 msgr2=0x7ff2902058d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7ff28f400640 1 --2- 172.21.15.46:0/2390635371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2900702a0 0x7ff2902058d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7ff28f400640 1 -- 172.21.15.46:0/2390635371 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff280008550 con 0x7ff290156590 2024-03-29T08:18:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7ff28f400640 1 --2- 172.21.15.46:0/2390635371 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff290156590 0x7ff2901575d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff280025030 tx=0x7ff28001d260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7ff287e00640 1 -- 172.21.15.46:0/2390635371 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2800a8df0 con 0x7ff290156590 2024-03-29T08:18:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7ff287e00640 1 -- 172.21.15.46:0/2390635371 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28000b150 con 0x7ff290156590 2024-03-29T08:18:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7ff287e00640 1 -- 172.21.15.46:0/2390635371 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28000b500 con 0x7ff290156590 2024-03-29T08:18:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f4cd2a00640 1 --2- 172.21.15.46:0/3053368297 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x7f4cbc06e880 0x7f4cbc070ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f4cd2a00640 1 -- 172.21.15.46:0/3053368297 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4cbc071350 con 0x7f4cbc06e880 2024-03-29T08:18:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7fc673e00640 1 --2- 172.21.15.46:0/3412408046 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc6680666f0 0x7fc668068bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7fc673e00640 1 -- 172.21.15.46:0/3412408046 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc674084d10 con 0x7fc67c04fa70 2024-03-29T08:18:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f7594c00640 1 -- 172.21.15.46:0/4152979536 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75900b5420 con 0x7f7598156160 2024-03-29T08:18:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f7594c00640 1 -- 172.21.15.46:0/4152979536 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75900b5580 con 0x7f7598156160 2024-03-29T08:18:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7ff296a00640 1 -- 172.21.15.46:0/2390635371 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff290158100 con 0x7ff290156590 2024-03-29T08:18:19.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f7594c00640 1 -- 172.21.15.46:0/4152979536 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75900b5930 con 0x7f7598156160 2024-03-29T08:18:19.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f3372000640 1 Processor -- start 2024-03-29T08:18:19.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f759f400640 1 -- 172.21.15.46:0/4152979536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7598150ac0 con 0x7f7598156160 2024-03-29T08:18:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.208+0000 7f7b46a00640 1 --2- 172.21.15.46:0/2961805659 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b30066690 0x7f7b30068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f7b46a00640 1 -- 172.21.15.46:0/2961805659 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b48083440 con 0x7f7b500b3050 2024-03-29T08:18:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7ff296a00640 1 -- 172.21.15.46:0/2390635371 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2901586c0 con 0x7ff290156590 2024-03-29T08:18:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f7b46a00640 1 --2- 172.21.15.46:0/2961805659 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x7f7b3006e830 0x7f7b30070c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f759f400640 1 -- 172.21.15.46:0/4152979536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759814ffe0 con 0x7f7598156160 2024-03-29T08:18:19.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f30d3400640 1 -- 172.21.15.46:0/2560449806 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30bc0a7050 con 0x7f30d40707e0 2024-03-29T08:18:19.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f30d3400640 1 -- 172.21.15.46:0/2560449806 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30bc0ab980 con 0x7f30d40707e0 2024-03-29T08:18:19.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7fc673e00640 1 --2- 172.21.15.46:0/3412408046 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x7fc66806e890 0x7fc668070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7fc683400640 1 --2- 172.21.15.46:0/3412408046 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc6680666f0 0x7fc668068bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f4cdaa00640 1 --2- 172.21.15.46:0/3053368297 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4cbc0666e0 0x7f4cbc068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f7b4ea00640 1 --2- 172.21.15.46:0/2961805659 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b30066690 0x7f7b30068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7ff296a00640 1 -- 172.21.15.46:0/2390635371 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff2900b3760 con 0x7ff290156590 2024-03-29T08:18:19.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f3372000640 1 -- start start 2024-03-29T08:18:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f30daa00640 1 -- 172.21.15.46:0/2560449806 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30d40707e0 msgr2=0x7f30d40af5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f30daa00640 1 --2- 172.21.15.46:0/2560449806 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30d40707e0 0x7f30d40af5a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f30bc0097c0 tx=0x7f30bc01c460 comp rx=0 tx=0).stop 2024-03-29T08:18:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7fc683400640 1 --2- 172.21.15.46:0/3412408046 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc6680666f0 0x7fc668068bb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc6780b2000 tx=0x7fc6780aa070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f4cda000640 1 --2- 172.21.15.46:0/3053368297 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x7f4cbc06e880 0x7f4cbc070ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f4cd2a00640 1 -- 172.21.15.46:0/3053368297 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ccc051590 con 0x7f4cdc160fe0 2024-03-29T08:18:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f4cdaa00640 1 --2- 172.21.15.46:0/3053368297 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4cbc0666e0 0x7f4cbc068ba0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40013d0 tx=0x7f4cd4007e30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f7b46a00640 1 -- 172.21.15.46:0/2961805659 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b30071300 con 0x7f7b3006e830 2024-03-29T08:18:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f7b46a00640 1 -- 172.21.15.46:0/2961805659 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b480859c0 con 0x7f7b500b3050 2024-03-29T08:18:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7fc4db400640 1 -- 172.21.15.46:0/882567033 reap_dead start 2024-03-29T08:18:19.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f759f400640 1 -- 172.21.15.46:0/4152979536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f759804e040 con 0x7f7598156160 2024-03-29T08:18:19.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f4cda000640 1 --2- 172.21.15.46:0/3053368297 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x7f4cbc06e880 0x7f4cbc070ca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f7b4e000640 1 --2- 172.21.15.46:0/2961805659 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x7f7b3006e830 0x7f7b30070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f7b4ea00640 1 --2- 172.21.15.46:0/2961805659 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b30066690 0x7f7b30068b50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c001760 tx=0x7f7b3c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7fc4db400640 1 --2- 172.21.15.46:0/882567033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c0005190 0x7fc4c000b990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400be20 tx=0x7fc4d4020670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=657a8158859b5d6b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7fc4d9600640 1 -- 172.21.15.46:0/882567033 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4004220 con 0x7fc4c0005190 2024-03-29T08:18:19.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f7b4e000640 1 --2- 172.21.15.46:0/2961805659 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x7f7b3006e830 0x7f7b30070c50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7ff287e00640 1 -- 172.21.15.46:0/2390635371 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff28000b660 con 0x7ff290156590 2024-03-29T08:18:19.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7fc688e00640 1 --2- 172.21.15.46:0/3412408046 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x7fc66806e890 0x7fc668070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7fc688e00640 1 --2- 172.21.15.46:0/3412408046 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x7fc66806e890 0x7fc668070cb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7fc673e00640 1 -- 172.21.15.46:0/3412408046 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc668071360 con 0x7fc66806e890 2024-03-29T08:18:19.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f4cd2a00640 1 -- 172.21.15.46:0/3053368297 <== osd.2 v2:172.21.15.46:6809/27428 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f4cbc071350 con 0x7f4cbc06e880 2024-03-29T08:18:19.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f3372000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3364006e60 0x7f33640dd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7f3372000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3364034130 0x7f3364053590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f3372000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33640e36a0 0x7f3364053ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f3372000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33640346b0 con 0x7f33640e36a0 2024-03-29T08:18:19.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7fc4d9600640 1 -- 172.21.15.46:0/882567033 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d40043c0 con 0x7fc4c0005190 2024-03-29T08:18:19.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.212+0000 7fc4d9600640 1 -- 172.21.15.46:0/882567033 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4003860 con 0x7fc4c0005190 2024-03-29T08:18:19.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7fc4e1e00640 1 -- 172.21.15.46:0/882567033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c0005190 msgr2=0x7fc4c000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7fc4e1e00640 1 --2- 172.21.15.46:0/882567033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c0005190 0x7fc4c000b990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400be20 tx=0x7fc4d4020670 comp rx=0 tx=0).stop 2024-03-29T08:18:19.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7fc673e00640 1 -- 172.21.15.46:0/3412408046 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc674085160 con 0x7fc67c04fa70 2024-03-29T08:18:19.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f3372000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33640374f0 con 0x7f3364034130 2024-03-29T08:18:19.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f3372000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3364034950 con 0x7f3364006e60 2024-03-29T08:18:19.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f3370c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3364006e60 0x7f33640dd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f30daa00640 1 -- 172.21.15.46:0/2560449806 shutdown_connections 2024-03-29T08:18:19.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f30daa00640 1 --2- 172.21.15.46:0/2560449806 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30d40707e0 0x7f30d40af5a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f30daa00640 1 --2- 172.21.15.46:0/2560449806 >> v2:172.21.15.73:3300/0 conn(0x7f30d41434f0 0x7f30d40702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f30daa00640 1 -- 172.21.15.46:0/2560449806 >> 172.21.15.46:0/2560449806 conn(0x7f30d4065c50 msgr2=0x7f30d4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7fc4e1e00640 1 -- 172.21.15.46:0/882567033 shutdown_connections 2024-03-29T08:18:19.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7fc4e1e00640 1 --2- 172.21.15.46:0/882567033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4c0005190 0x7fc4c000b990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7fc4e1e00640 1 -- 172.21.15.46:0/882567033 >> 172.21.15.46:0/882567033 conn(0x7fc4dc065c50 msgr2=0x7fc4dc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f336be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3364034130 0x7f3364053590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f336be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3364034130 0x7f3364053590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51836/0 (socket says 172.21.15.46:51836) 2024-03-29T08:18:19.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f336be00640 1 -- 172.21.15.46:0/2073416274 learned_addr learned my addr 172.21.15.46:0/2073416274 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f30daa00640 1 -- 172.21.15.46:0/2560449806 shutdown_connections 2024-03-29T08:18:19.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f3373600640 1 --2- 172.21.15.46:0/2073416274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33640e36a0 0x7f3364053ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f30daa00640 1 -- 172.21.15.46:0/2560449806 wait complete. 2024-03-29T08:18:19.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7fc673e00640 1 -- 172.21.15.46:0/3412408046 <== osd.0 v2:172.21.15.46:6800/27424 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fc668071360 con 0x7fc66806e890 2024-03-29T08:18:19.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7fc4e1e00640 1 -- 172.21.15.46:0/882567033 shutdown_connections 2024-03-29T08:18:19.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7fc4e1e00640 1 -- 172.21.15.46:0/882567033 wait complete. 2024-03-29T08:18:19.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f7594c00640 1 -- 172.21.15.46:0/4152979536 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f75900b5a90 con 0x7f7598156160 2024-03-29T08:18:19.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f7594c00640 1 --2- 172.21.15.46:0/4152979536 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7578066710 0x7f7578068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f7594c00640 1 -- 172.21.15.46:0/4152979536 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f759011df40 con 0x7f7598156160 2024-03-29T08:18:19.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f759ca00640 1 --2- 172.21.15.46:0/4152979536 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7578066710 0x7f7578068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f7594c00640 1 --2- 172.21.15.46:0/4152979536 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x7f757806e930 0x7f7578070d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f7594c00640 1 -- 172.21.15.46:0/4152979536 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7578071400 con 0x7f757806e930 2024-03-29T08:18:19.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f7594c00640 1 -- 172.21.15.46:0/4152979536 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75900eedc0 con 0x7f7598156160 2024-03-29T08:18:19.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f759d400640 1 --2- 172.21.15.46:0/4152979536 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x7f757806e930 0x7f7578070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f759d400640 1 --2- 172.21.15.46:0/4152979536 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x7f757806e930 0x7f7578070d50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f7b46a00640 1 -- 172.21.15.46:0/2961805659 <== osd.3 v2:172.21.15.46:6808/27426 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f7b30071300 con 0x7f7b3006e830 2024-03-29T08:18:19.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7ff287e00640 1 --2- 172.21.15.46:0/2390635371 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff270066690 0x7ff270068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7ff287e00640 1 -- 172.21.15.46:0/2390635371 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff28011e010 con 0x7ff290156590 2024-03-29T08:18:19.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7ff287e00640 1 --2- 172.21.15.46:0/2390635371 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x7ff27006e8b0 0x7ff270070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7ff287e00640 1 -- 172.21.15.46:0/2390635371 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff270071380 con 0x7ff27006e8b0 2024-03-29T08:18:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7ff287e00640 1 -- 172.21.15.46:0/2390635371 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2801205d0 con 0x7ff290156590 2024-03-29T08:18:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7ff294a00640 1 --2- 172.21.15.46:0/2390635371 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x7ff27006e8b0 0x7ff270070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7ff28fe00640 1 --2- 172.21.15.46:0/2390635371 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff270066690 0x7ff270068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7ff294a00640 1 --2- 172.21.15.46:0/2390635371 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x7ff27006e8b0 0x7ff270070cd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f336be00640 1 -- 172.21.15.46:0/2073416274 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3364006e60 msgr2=0x7f33640dd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f336be00640 1 --2- 172.21.15.46:0/2073416274 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3364006e60 0x7f33640dd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f336be00640 1 -- 172.21.15.46:0/2073416274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33640e36a0 msgr2=0x7f3364053ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f336be00640 1 --2- 172.21.15.46:0/2073416274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33640e36a0 0x7f3364053ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f336be00640 1 -- 172.21.15.46:0/2073416274 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f336000a580 con 0x7f3364034130 2024-03-29T08:18:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f336be00640 1 --2- 172.21.15.46:0/2073416274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3364034130 0x7f3364053590 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3358009f70 tx=0x7f335800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f3368c00640 1 -- 172.21.15.46:0/2073416274 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358009510 con 0x7f3364034130 2024-03-29T08:18:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f3368c00640 1 -- 172.21.15.46:0/2073416274 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3358017c60 con 0x7f3364034130 2024-03-29T08:18:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.216+0000 7f3368c00640 1 -- 172.21.15.46:0/2073416274 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358009510 con 0x7f3364034130 2024-03-29T08:18:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30daa00640 1 Processor -- start 2024-03-29T08:18:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30daa00640 1 -- start start 2024-03-29T08:18:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d40707e0 0x7f30d413d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30daa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30d40afae0 0x7f30d4162820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30daa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30d41434f0 0x7f30d4160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30daa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d413d8d0 con 0x7f30d40707e0 2024-03-29T08:18:19.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30daa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d4161780 con 0x7f30d41434f0 2024-03-29T08:18:19.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30daa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d4161a60 con 0x7f30d40afae0 2024-03-29T08:18:19.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7fc4e1e00640 1 Processor -- start 2024-03-29T08:18:19.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7fc4e1e00640 1 -- start start 2024-03-29T08:18:19.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30d8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30d41434f0 0x7f30d4160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30d8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30d41434f0 0x7f30d4160e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51844/0 (socket says 172.21.15.46:51844) 2024-03-29T08:18:19.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30d8a00640 1 -- 172.21.15.46:0/676847721 learned_addr learned my addr 172.21.15.46:0/676847721 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7fc4e1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4c0005190 0x7fc4dc159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7fc4e1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4dc156b70 0x7fc4dc145b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7fc4e1e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4dc1460a0 0x7fc4dc0cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7fc4e1e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc1572e0 con 0x7fc4c0005190 2024-03-29T08:18:19.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7fc4e1e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc1575c0 con 0x7fc4dc156b70 2024-03-29T08:18:19.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7fc4e1e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4dc1578a0 con 0x7fc4dc1460a0 2024-03-29T08:18:19.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f759ca00640 1 --2- 172.21.15.46:0/4152979536 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7578066710 0x7f7578068bd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f759804f350 tx=0x7f7584008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7ff28fe00640 1 --2- 172.21.15.46:0/2390635371 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff270066690 0x7ff270068b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff27c004e30 tx=0x7ff27c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f3372000640 1 -- 172.21.15.46:0/2073416274 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3364034c30 con 0x7f3364034130 2024-03-29T08:18:19.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f3372000640 1 -- 172.21.15.46:0/2073416274 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3364035430 con 0x7f3364034130 2024-03-29T08:18:19.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f3372000640 1 -- 172.21.15.46:0/2073416274 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f336404ff20 con 0x7f3364034130 2024-03-29T08:18:19.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30cb400640 1 --2- 172.21.15.46:0/676847721 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30d40afae0 0x7f30d4162820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f7594c00640 1 -- 172.21.15.46:0/4152979536 <== osd.1 v2:172.21.15.46:6824/27427 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f7578071400 con 0x7f757806e930 2024-03-29T08:18:19.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7ff287e00640 1 -- 172.21.15.46:0/2390635371 <== osd.4 v2:172.21.15.73:6800/27689 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7ff270071380 con 0x7ff27006e8b0 2024-03-29T08:18:19.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30d3e00640 1 --2- 172.21.15.46:0/676847721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d40707e0 0x7f30d413d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30d8a00640 1 -- 172.21.15.46:0/676847721 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30d40afae0 msgr2=0x7f30d4162820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30d8a00640 1 --2- 172.21.15.46:0/676847721 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30d40afae0 0x7f30d4162820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30d8a00640 1 -- 172.21.15.46:0/676847721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d40707e0 msgr2=0x7f30d413d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30d8a00640 1 --2- 172.21.15.46:0/676847721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30d40707e0 0x7f30d413d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7f30d8a00640 1 -- 172.21.15.46:0/676847721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c40094d0 con 0x7f30d41434f0 2024-03-29T08:18:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7fc4db400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4dc1460a0 0x7fc4dc0cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7fc4db400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4dc1460a0 0x7fc4dc0cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:57586/0 (socket says 172.21.15.46:57586) 2024-03-29T08:18:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7fc4db400640 1 -- 172.21.15.46:0/1730574108 learned_addr learned my addr 172.21.15.46:0/1730574108 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.220+0000 7fc4da000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4dc156b70 0x7fc4dc145b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7f30d8a00640 1 --2- 172.21.15.46:0/676847721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30d41434f0 0x7f30d4160e30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00c780 tx=0x7f30cc00cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7fc4db400640 1 -- 172.21.15.46:0/1730574108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4dc156b70 msgr2=0x7fc4dc145b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7fc4db400640 1 --2- 172.21.15.46:0/1730574108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4dc156b70 0x7fc4dc145b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7fc4db400640 1 -- 172.21.15.46:0/1730574108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4c0005190 msgr2=0x7fc4dc159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7fc4db400640 1 --2- 172.21.15.46:0/1730574108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4c0005190 0x7fc4dc159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7fc4db400640 1 -- 172.21.15.46:0/1730574108 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc157b80 con 0x7fc4dc1460a0 2024-03-29T08:18:19.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7f30d0c00640 1 -- 172.21.15.46:0/676847721 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc00a9f0 con 0x7f30d41434f0 2024-03-29T08:18:19.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7f30d0c00640 1 -- 172.21.15.46:0/676847721 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc00ce90 con 0x7f30d41434f0 2024-03-29T08:18:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7f30d0c00640 1 -- 172.21.15.46:0/676847721 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc00add0 con 0x7f30d41434f0 2024-03-29T08:18:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7f30daa00640 1 -- 172.21.15.46:0/676847721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30bc009420 con 0x7f30d41434f0 2024-03-29T08:18:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7f30daa00640 1 -- 172.21.15.46:0/676847721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d40b00e0 con 0x7f30d41434f0 2024-03-29T08:18:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7f30daa00640 1 -- 172.21.15.46:0/676847721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f30d40aff60 con 0x7f30d41434f0 2024-03-29T08:18:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7fc4db400640 1 --2- 172.21.15.46:0/1730574108 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4dc1460a0 0x7fc4dc0cd600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4020f10 tx=0x7fc4d401d410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7fc4d3400640 1 -- 172.21.15.46:0/1730574108 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d400b070 con 0x7fc4dc1460a0 2024-03-29T08:18:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7fc4d3400640 1 -- 172.21.15.46:0/1730574108 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d40ad670 con 0x7fc4dc1460a0 2024-03-29T08:18:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7fc4d3400640 1 -- 172.21.15.46:0/1730574108 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d40ada20 con 0x7fc4dc1460a0 2024-03-29T08:18:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7fc4e1e00640 1 -- 172.21.15.46:0/1730574108 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4dc157e60 con 0x7fc4dc1460a0 2024-03-29T08:18:19.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7fc4e1e00640 1 -- 172.21.15.46:0/1730574108 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4dc0cdb40 con 0x7fc4dc1460a0 2024-03-29T08:18:19.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7f3368c00640 1 -- 172.21.15.46:0/2073416274 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f335800c6d0 con 0x7f3364034130 2024-03-29T08:18:19.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7fc4e1e00640 1 -- 172.21.15.46:0/1730574108 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc4dc15d480 con 0x7fc4dc1460a0 2024-03-29T08:18:19.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.224+0000 7f30d0c00640 1 -- 172.21.15.46:0/676847721 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f30cc012420 con 0x7f30d41434f0 2024-03-29T08:18:19.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f3368c00640 1 --2- 172.21.15.46:0/2073416274 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f334c066720 0x7f334c068be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f3368c00640 1 -- 172.21.15.46:0/2073416274 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f335808bdc0 con 0x7f3364034130 2024-03-29T08:18:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f3368c00640 1 --2- 172.21.15.46:0/2073416274 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x7f334c06e940 0x7f334c070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f3368c00640 1 -- 172.21.15.46:0/2073416274 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f334c071410 con 0x7f334c06e940 2024-03-29T08:18:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f3368c00640 1 -- 172.21.15.46:0/2073416274 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f335808c0c0 con 0x7f3364034130 2024-03-29T08:18:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f30d0c00640 1 --2- 172.21.15.46:0/676847721 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f30b00666e0 0x7f30b0068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f30d0c00640 1 -- 172.21.15.46:0/676847721 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30cc0843a0 con 0x7f30d41434f0 2024-03-29T08:18:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f30d0c00640 1 --2- 172.21.15.46:0/676847721 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x7f30b006e900 0x7f30b0070d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f30d0c00640 1 -- 172.21.15.46:0/676847721 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f30b00713d0 con 0x7f30b006e900 2024-03-29T08:18:19.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f30d0c00640 1 -- 172.21.15.46:0/676847721 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30cc086980 con 0x7f30d41434f0 2024-03-29T08:18:19.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f3370c00640 1 --2- 172.21.15.46:0/2073416274 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f334c066720 0x7f334c068be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f30d3e00640 1 --2- 172.21.15.46:0/676847721 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f30b00666e0 0x7f30b0068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7fc4d3400640 1 -- 172.21.15.46:0/1730574108 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc4d40adb80 con 0x7fc4dc1460a0 2024-03-29T08:18:19.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f3373600640 1 --2- 172.21.15.46:0/2073416274 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x7f334c06e940 0x7f334c070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f30d3e00640 1 --2- 172.21.15.46:0/676847721 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f30b00666e0 0x7f30b0068ba0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009bd0 tx=0x7f30c4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f3373600640 1 --2- 172.21.15.46:0/2073416274 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x7f334c06e940 0x7f334c070d60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f3370c00640 1 --2- 172.21.15.46:0/2073416274 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f334c066720 0x7f334c068be0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f33600b1000 tx=0x7f33600aa070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7fc4d3400640 1 --2- 172.21.15.46:0/1730574108 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc4b8066710 0x7fc4b8068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7fc4d3400640 1 -- 172.21.15.46:0/1730574108 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4d40eefd0 con 0x7fc4dc1460a0 2024-03-29T08:18:19.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7fc4daa00640 1 --2- 172.21.15.46:0/1730574108 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc4b8066710 0x7fc4b8068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7fc4d3400640 1 --2- 172.21.15.46:0/1730574108 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x7fc4b806f540 0x7fc4b80719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7fc4d3400640 1 -- 172.21.15.46:0/1730574108 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4b80720b0 con 0x7fc4b806f540 2024-03-29T08:18:19.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f30cb400640 1 --2- 172.21.15.46:0/676847721 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x7f30b006e900 0x7f30b0070d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.228+0000 7f3368c00640 1 -- 172.21.15.46:0/2073416274 <== osd.5 v2:172.21.15.73:6808/27690 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f334c071410 con 0x7f334c06e940 2024-03-29T08:18:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.232+0000 7fc4da000640 1 --2- 172.21.15.46:0/1730574108 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x7fc4b806f540 0x7fc4b80719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.232+0000 7fc4d3400640 1 -- 172.21.15.46:0/1730574108 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4d40ee6a0 con 0x7fc4dc1460a0 2024-03-29T08:18:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.232+0000 7fc4daa00640 1 --2- 172.21.15.46:0/1730574108 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc4b8066710 0x7fc4b8068bd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4003710 tx=0x7fc4c40036a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.232+0000 7fc4da000640 1 --2- 172.21.15.46:0/1730574108 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x7fc4b806f540 0x7fc4b80719e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.232+0000 7f30cb400640 1 --2- 172.21.15.46:0/676847721 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x7f30b006e900 0x7f30b0070d20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.232+0000 7fc4d3400640 1 -- 172.21.15.46:0/1730574108 <== osd.7 v2:172.21.15.73:6816/27700 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fc4b80720b0 con 0x7fc4b806f540 2024-03-29T08:18:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.232+0000 7f30d0c00640 1 -- 172.21.15.46:0/676847721 <== osd.6 v2:172.21.15.73:6824/27691 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f30b00713d0 con 0x7f30b006e900 2024-03-29T08:18:19.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.244+0000 7f7b55a00640 1 -- 172.21.15.46:0/2961805659 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7b50071a90 con 0x7f7b3006e830 2024-03-29T08:18:19.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.244+0000 7f7b46a00640 1 -- 172.21.15.46:0/2961805659 <== osd.3 v2:172.21.15.46:6808/27426 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7b50071a90 con 0x7f7b3006e830 2024-03-29T08:18:19.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.244+0000 7f4ce1a00640 1 -- 172.21.15.46:0/3053368297 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4cdc04e040 con 0x7f4cbc06e880 2024-03-29T08:18:19.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.244+0000 7fc68ae00640 1 -- 172.21.15.46:0/3412408046 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc660002cc0 con 0x7fc66806e890 2024-03-29T08:18:19.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7ff296a00640 1 -- 172.21.15.46:0/2390635371 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff290159fe0 con 0x7ff27006e8b0 2024-03-29T08:18:19.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f7b55a00640 1 -- 172.21.15.46:0/2961805659 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x7f7b3006e830 msgr2=0x7f7b30070c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f7b55a00640 1 --2- 172.21.15.46:0/2961805659 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x7f7b3006e830 0x7f7b30070c50 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7ff287e00640 1 -- 172.21.15.46:0/2390635371 <== osd.4 v2:172.21.15.73:6800/27689 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff288007170 con 0x7ff27006e8b0 2024-03-29T08:18:19.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f4cd2a00640 1 -- 172.21.15.46:0/3053368297 <== osd.2 v2:172.21.15.46:6809/27428 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4cdc04e040 con 0x7f4cbc06e880 2024-03-29T08:18:19.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f7b55a00640 1 -- 172.21.15.46:0/2961805659 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b30066690 msgr2=0x7f7b30068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f7b55a00640 1 --2- 172.21.15.46:0/2961805659 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b30066690 0x7f7b30068b50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c001760 tx=0x7f7b3c009040 comp rx=0 tx=0).stop 2024-03-29T08:18:19.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f7b55a00640 1 -- 172.21.15.46:0/2961805659 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b500b3050 msgr2=0x7f7b500a8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f7b55a00640 1 --2- 172.21.15.46:0/2961805659 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b500b3050 0x7f7b500a8ca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800c7b0 tx=0x7f7b4800cc70 comp rx=0 tx=0).stop 2024-03-29T08:18:19.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f7b4f400640 1 -- 172.21.15.46:0/2961805659 reap_dead start 2024-03-29T08:18:19.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f4ce1a00640 1 -- 172.21.15.46:0/3053368297 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x7f4cbc06e880 msgr2=0x7f4cbc070ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f4ce1a00640 1 --2- 172.21.15.46:0/3053368297 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x7f4cbc06e880 0x7f4cbc070ca0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f7b55a00640 1 -- 172.21.15.46:0/2961805659 shutdown_connections 2024-03-29T08:18:19.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f7b55a00640 1 -- 172.21.15.46:0/2961805659 >> 172.21.15.46:0/2961805659 conn(0x7f7b50065c50 msgr2=0x7f7b50069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7ff296a00640 1 -- 172.21.15.46:0/2390635371 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x7ff27006e8b0 msgr2=0x7ff270070cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7ff296a00640 1 --2- 172.21.15.46:0/2390635371 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x7ff27006e8b0 0x7ff270070cd0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f4ce1a00640 1 -- 172.21.15.46:0/3053368297 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4cbc0666e0 msgr2=0x7f4cbc068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f4ce1a00640 1 --2- 172.21.15.46:0/3053368297 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4cbc0666e0 0x7f4cbc068ba0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40013d0 tx=0x7f4cd4007e30 comp rx=0 tx=0).stop 2024-03-29T08:18:19.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f7b55a00640 1 -- 172.21.15.46:0/2961805659 shutdown_connections 2024-03-29T08:18:19.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f4ce1a00640 1 -- 172.21.15.46:0/3053368297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cdc160fe0 msgr2=0x7f4cdc1538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f4ce1a00640 1 --2- 172.21.15.46:0/3053368297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cdc160fe0 0x7f4cdc1538a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc00e7c0 tx=0x7f4ccc00ec80 comp rx=0 tx=0).stop 2024-03-29T08:18:19.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f4cdb400640 1 -- 172.21.15.46:0/3053368297 reap_dead start 2024-03-29T08:18:19.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f4ce1a00640 1 -- 172.21.15.46:0/3053368297 shutdown_connections 2024-03-29T08:18:19.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f4ce1a00640 1 -- 172.21.15.46:0/3053368297 >> 172.21.15.46:0/3053368297 conn(0x7f4cdc0affd0 msgr2=0x7f4cdc0accc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f4ce1a00640 1 -- 172.21.15.46:0/3053368297 shutdown_connections 2024-03-29T08:18:19.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f7b55a00640 1 -- 172.21.15.46:0/2961805659 wait complete. 2024-03-29T08:18:19.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7ff296a00640 1 -- 172.21.15.46:0/2390635371 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff270066690 msgr2=0x7ff270068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7ff296a00640 1 --2- 172.21.15.46:0/2390635371 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff270066690 0x7ff270068b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff27c004e30 tx=0x7ff27c009040 comp rx=0 tx=0).stop 2024-03-29T08:18:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7ff296a00640 1 -- 172.21.15.46:0/2390635371 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff290156590 msgr2=0x7ff2901575d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7ff296a00640 1 --2- 172.21.15.46:0/2390635371 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff290156590 0x7ff2901575d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff280025030 tx=0x7ff28001d260 comp rx=0 tx=0).stop 2024-03-29T08:18:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7ff294a00640 1 -- 172.21.15.46:0/2390635371 reap_dead start 2024-03-29T08:18:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7ff296a00640 1 -- 172.21.15.46:0/2390635371 shutdown_connections 2024-03-29T08:18:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7ff296a00640 1 -- 172.21.15.46:0/2390635371 >> 172.21.15.46:0/2390635371 conn(0x7ff290065c50 msgr2=0x7ff290069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7f4ce1a00640 1 -- 172.21.15.46:0/3053368297 wait complete. 2024-03-29T08:18:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7ff296a00640 1 -- 172.21.15.46:0/2390635371 shutdown_connections 2024-03-29T08:18:19.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.248+0000 7ff296a00640 1 -- 172.21.15.46:0/2390635371 wait complete. 2024-03-29T08:18:19.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.252+0000 7fc673e00640 1 -- 172.21.15.46:0/3412408046 <== osd.0 v2:172.21.15.46:6800/27424 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc660002cc0 con 0x7fc66806e890 2024-03-29T08:18:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.252+0000 7fc68ae00640 1 -- 172.21.15.46:0/3412408046 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x7fc66806e890 msgr2=0x7fc668070cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.252+0000 7fc68ae00640 1 --2- 172.21.15.46:0/3412408046 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x7fc66806e890 0x7fc668070cb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.252+0000 7fc68ae00640 1 -- 172.21.15.46:0/3412408046 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc6680666f0 msgr2=0x7fc668068bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.252+0000 7fc68ae00640 1 --2- 172.21.15.46:0/3412408046 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc6680666f0 0x7fc668068bb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc6780b2000 tx=0x7fc6780aa070 comp rx=0 tx=0).stop 2024-03-29T08:18:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.252+0000 7fc68ae00640 1 -- 172.21.15.46:0/3412408046 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc67c04fa70 msgr2=0x7fc67c0ddf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.252+0000 7fc68ae00640 1 --2- 172.21.15.46:0/3412408046 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc67c04fa70 0x7fc67c0ddf10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc67400d5f0 tx=0x7fc67400dab0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.252+0000 7fc688e00640 1 -- 172.21.15.46:0/3412408046 reap_dead start 2024-03-29T08:18:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.252+0000 7fc68ae00640 1 -- 172.21.15.46:0/3412408046 shutdown_connections 2024-03-29T08:18:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.252+0000 7fc68ae00640 1 -- 172.21.15.46:0/3412408046 >> 172.21.15.46:0/3412408046 conn(0x7fc67c009f90 msgr2=0x7fc67c0e2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.252+0000 7fc68ae00640 1 -- 172.21.15.46:0/3412408046 shutdown_connections 2024-03-29T08:18:19.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.252+0000 7fc68ae00640 1 -- 172.21.15.46:0/3412408046 wait complete. 2024-03-29T08:18:19.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.252+0000 7f759f400640 1 -- 172.21.15.46:0/4152979536 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f75981507e0 con 0x7f757806e930 2024-03-29T08:18:19.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.252+0000 7f7594c00640 1 -- 172.21.15.46:0/4152979536 <== osd.1 v2:172.21.15.46:6824/27427 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f75981507e0 con 0x7f757806e930 2024-03-29T08:18:19.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.256+0000 7f759f400640 1 -- 172.21.15.46:0/4152979536 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x7f757806e930 msgr2=0x7f7578070d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.256+0000 7f759f400640 1 --2- 172.21.15.46:0/4152979536 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x7f757806e930 0x7f7578070d50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f759f400640 1 -- 172.21.15.46:0/4152979536 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7578066710 msgr2=0x7f7578068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f759f400640 1 --2- 172.21.15.46:0/4152979536 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7578066710 0x7f7578068bd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f759804f350 tx=0x7f7584008040 comp rx=0 tx=0).stop 2024-03-29T08:18:19.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f759f400640 1 -- 172.21.15.46:0/4152979536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7598156160 msgr2=0x7f75981565e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f759f400640 1 --2- 172.21.15.46:0/4152979536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7598156160 0x7f75981565e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7590000f90 tx=0x7f757c005800 comp rx=0 tx=0).stop 2024-03-29T08:18:19.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f759d400640 1 -- 172.21.15.46:0/4152979536 reap_dead start 2024-03-29T08:18:19.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.256+0000 7f30daa00640 1 -- 172.21.15.46:0/676847721 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f30d4161bd0 con 0x7f30b006e900 2024-03-29T08:18:19.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f30d0c00640 1 -- 172.21.15.46:0/676847721 <== osd.6 v2:172.21.15.73:6824/27691 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f30d4161bd0 con 0x7f30b006e900 2024-03-29T08:18:19.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f30c9600640 1 -- 172.21.15.46:0/676847721 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x7f30b006e900 msgr2=0x7f30b0070d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f30c9600640 1 --2- 172.21.15.46:0/676847721 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x7f30b006e900 0x7f30b0070d20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f30c9600640 1 -- 172.21.15.46:0/676847721 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f30b00666e0 msgr2=0x7f30b0068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f30c9600640 1 --2- 172.21.15.46:0/676847721 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f30b00666e0 0x7f30b0068ba0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009bd0 tx=0x7f30c4008040 comp rx=0 tx=0).stop 2024-03-29T08:18:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f30c9600640 1 -- 172.21.15.46:0/676847721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30d41434f0 msgr2=0x7f30d4160e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f30c9600640 1 --2- 172.21.15.46:0/676847721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30d41434f0 0x7f30d4160e30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00c780 tx=0x7f30cc00cc40 comp rx=0 tx=0).stop 2024-03-29T08:18:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f30d8a00640 1 -- 172.21.15.46:0/676847721 reap_dead start 2024-03-29T08:18:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f30c9600640 1 -- 172.21.15.46:0/676847721 shutdown_connections 2024-03-29T08:18:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f30c9600640 1 -- 172.21.15.46:0/676847721 >> 172.21.15.46:0/676847721 conn(0x7f30d4065c50 msgr2=0x7f30d4140680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f30c9600640 1 -- 172.21.15.46:0/676847721 shutdown_connections 2024-03-29T08:18:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f30c9600640 1 -- 172.21.15.46:0/676847721 wait complete. 2024-03-29T08:18:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7fc4e1e00640 1 -- 172.21.15.46:0/1730574108 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc4dc1569f0 con 0x7fc4b806f540 2024-03-29T08:18:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7fc4d3400640 1 -- 172.21.15.46:0/1730574108 <== osd.7 v2:172.21.15.73:6816/27700 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc4cc009c90 con 0x7fc4b806f540 2024-03-29T08:18:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f759f400640 1 -- 172.21.15.46:0/4152979536 shutdown_connections 2024-03-29T08:18:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f759f400640 1 -- 172.21.15.46:0/4152979536 >> 172.21.15.46:0/4152979536 conn(0x7f75980b3730 msgr2=0x7f75980ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f759f400640 1 -- 172.21.15.46:0/4152979536 shutdown_connections 2024-03-29T08:18:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7f759f400640 1 -- 172.21.15.46:0/4152979536 wait complete. 2024-03-29T08:18:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7fc4e1e00640 1 -- 172.21.15.46:0/1730574108 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x7fc4b806f540 msgr2=0x7fc4b80719e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7fc4e1e00640 1 --2- 172.21.15.46:0/1730574108 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x7fc4b806f540 0x7fc4b80719e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7fc4e1e00640 1 -- 172.21.15.46:0/1730574108 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc4b8066710 msgr2=0x7fc4b8068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7fc4e1e00640 1 --2- 172.21.15.46:0/1730574108 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc4b8066710 0x7fc4b8068bd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4003710 tx=0x7fc4c40036a0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7fc4e1e00640 1 -- 172.21.15.46:0/1730574108 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4dc1460a0 msgr2=0x7fc4dc0cd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7fc4e1e00640 1 --2- 172.21.15.46:0/1730574108 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4dc1460a0 0x7fc4dc0cd600 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4020f10 tx=0x7fc4d401d410 comp rx=0 tx=0).stop 2024-03-29T08:18:19.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7fc4db400640 1 -- 172.21.15.46:0/1730574108 reap_dead start 2024-03-29T08:18:19.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7fc4e1e00640 1 -- 172.21.15.46:0/1730574108 shutdown_connections 2024-03-29T08:18:19.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7fc4e1e00640 1 -- 172.21.15.46:0/1730574108 >> 172.21.15.46:0/1730574108 conn(0x7fc4dc065c50 msgr2=0x7fc4dc0ad820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7fc4e1e00640 1 -- 172.21.15.46:0/1730574108 shutdown_connections 2024-03-29T08:18:19.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.260+0000 7fc4e1e00640 1 -- 172.21.15.46:0/1730574108 wait complete. 2024-03-29T08:18:19.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.264+0000 7f3372000640 1 -- 172.21.15.46:0/2073416274 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3364052b00 con 0x7f334c06e940 2024-03-29T08:18:19.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.264+0000 7f3368c00640 1 -- 172.21.15.46:0/2073416274 <== osd.5 v2:172.21.15.73:6808/27690 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f336c072350 con 0x7f334c06e940 2024-03-29T08:18:19.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.264+0000 7f3372000640 1 -- 172.21.15.46:0/2073416274 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x7f334c06e940 msgr2=0x7f334c070d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.268+0000 7f3372000640 1 --2- 172.21.15.46:0/2073416274 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x7f334c06e940 0x7f334c070d60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.272+0000 7f3372000640 1 -- 172.21.15.46:0/2073416274 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f334c066720 msgr2=0x7f334c068be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.272+0000 7f3372000640 1 --2- 172.21.15.46:0/2073416274 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f334c066720 0x7f334c068be0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f33600b1000 tx=0x7f33600aa070 comp rx=0 tx=0).stop 2024-03-29T08:18:19.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.272+0000 7f3372000640 1 -- 172.21.15.46:0/2073416274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3364034130 msgr2=0x7f3364053590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.272+0000 7f3372000640 1 --2- 172.21.15.46:0/2073416274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3364034130 0x7f3364053590 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3358009f70 tx=0x7f335800ecd0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.272+0000 7f3373600640 1 -- 172.21.15.46:0/2073416274 reap_dead start 2024-03-29T08:18:19.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.272+0000 7f3372000640 1 -- 172.21.15.46:0/2073416274 shutdown_connections 2024-03-29T08:18:19.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.272+0000 7f3372000640 1 -- 172.21.15.46:0/2073416274 >> 172.21.15.46:0/2073416274 conn(0x7f3364054c70 msgr2=0x7f33640de2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.276+0000 7f3372000640 1 -- 172.21.15.46:0/2073416274 shutdown_connections 2024-03-29T08:18:19.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.276+0000 7f3372000640 1 -- 172.21.15.46:0/2073416274 wait complete. 2024-03-29T08:18:19.293 INFO:teuthology.orchestra.run.smithi046.stdout:64424509448 2024-03-29T08:18:19.294 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-29T08:18:19.295 INFO:teuthology.orchestra.run.smithi046.stdout:64424509447 2024-03-29T08:18:19.295 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-29T08:18:19.296 INFO:teuthology.orchestra.run.smithi046.stdout:64424509445 2024-03-29T08:18:19.296 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-29T08:18:19.300 INFO:teuthology.orchestra.run.smithi046.stdout:64424509447 2024-03-29T08:18:19.300 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-29T08:18:19.301 INFO:teuthology.orchestra.run.smithi046.stdout:64424509446 2024-03-29T08:18:19.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-29T08:18:19.303 INFO:teuthology.orchestra.run.smithi046.stdout:64424509446 2024-03-29T08:18:19.303 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-29T08:18:19.306 INFO:teuthology.orchestra.run.smithi046.stdout:64424509446 2024-03-29T08:18:19.307 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-29T08:18:19.320 INFO:teuthology.orchestra.run.smithi046.stdout:60129542152 2024-03-29T08:18:19.321 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-29T08:18:19.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.532+0000 7f7eff600640 1 Processor -- start 2024-03-29T08:18:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.532+0000 7f7eff600640 1 -- start start 2024-03-29T08:18:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.532+0000 7f7eff600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7ef0047bf0 0x7f7ef00eb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.532+0000 7f7eff600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ef0037b90 con 0x7f7ef0005ec0 2024-03-29T08:18:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.532+0000 7f7eff600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ef004f060 con 0x7f7ef0033e10 2024-03-29T08:18:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.532+0000 7f7eff600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef00501e0 con 0x7f7ef0047bf0 2024-03-29T08:18:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.532+0000 7f7efd600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7ef0005ec0 0x7f7ef0006310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60918/0 (socket says 172.21.15.46:60918) 2024-03-29T08:18:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.532+0000 7f7efd600640 1 -- 172.21.15.46:0/4219022109 learned_addr learned my addr 172.21.15.46:0/4219022109 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.532+0000 7f7ef7e00640 1 --2- 172.21.15.46:0/4219022109 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7ef0047bf0 0x7f7ef00eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.532+0000 7f7ef7400640 1 -- 172.21.15.46:0/4219022109 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893067861 0 0) 0x7f7ef004f060 con 0x7f7ef0033e10 2024-03-29T08:18:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.532+0000 7f7ef7400640 1 -- 172.21.15.46:0/4219022109 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ee0003530 con 0x7f7ef0033e10 2024-03-29T08:18:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.532+0000 7f7ef7e00640 1 -- 172.21.15.46:0/4219022109 >> v1:172.21.15.73:6789/0 conn(0x7f7ef0033e10 legacy=0x7f7ef00eb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.532+0000 7f7ef7e00640 1 -- 172.21.15.46:0/4219022109 >> v1:172.21.15.46:6789/0 conn(0x7f7ef0005ec0 legacy=0x7f7ef0006310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.532+0000 7f7ef7e00640 1 -- 172.21.15.46:0/4219022109 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef004fd50 con 0x7f7ef0047bf0 2024-03-29T08:18:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.536+0000 7f9432400640 1 Processor -- start 2024-03-29T08:18:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.536+0000 7f9432400640 1 -- start start 2024-03-29T08:18:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f9432400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f942c160d10 0x7f942c0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f9432400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f942c0caa30 0x7f942c0b0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f9432400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f942c0b0ba0 0x7f942c06d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f9432400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c0b1010 con 0x7f942c160d10 2024-03-29T08:18:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f7ef7e00640 1 --2- 172.21.15.46:0/4219022109 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7ef0047bf0 0x7f7ef00eb860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0097f0 tx=0x7f7eec01ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c48c180480dc3d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f9432400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c06aa00 con 0x7f942c0caa30 2024-03-29T08:18:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f9432400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0ccb80 con 0x7f942c0b0ba0 2024-03-29T08:18:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f942aa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f942c0caa30 0x7f942c0b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f942aa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f942c0caa30 0x7f942c0b0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51864/0 (socket says 172.21.15.46:51864) 2024-03-29T08:18:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f7ef7400640 1 -- 172.21.15.46:0/4219022109 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec0a8990 con 0x7f7ef0047bf0 2024-03-29T08:18:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f942aa00640 1 -- 172.21.15.46:0/2575400915 learned_addr learned my addr 172.21.15.46:0/2575400915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f942b400640 1 --2- 172.21.15.46:0/2575400915 >> v2:172.21.15.46:3300/0 conn(0x7f942c160d10 0x7f942c0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f7ef7400640 1 -- 172.21.15.46:0/4219022109 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eec0a8af0 con 0x7f7ef0047bf0 2024-03-29T08:18:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f7ef7400640 1 -- 172.21.15.46:0/4219022109 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec0a8e60 con 0x7f7ef0047bf0 2024-03-29T08:18:19.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f942b400640 1 -- 172.21.15.46:0/2575400915 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f942c0b0ba0 msgr2=0x7f942c06d090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T08:18:19.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f942b400640 1 --2- 172.21.15.46:0/2575400915 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f942c0b0ba0 0x7f942c06d090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f942b400640 1 -- 172.21.15.46:0/2575400915 >> v2:172.21.15.73:3300/0 conn(0x7f942c0caa30 msgr2=0x7f942c0b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f942b400640 1 --2- 172.21.15.46:0/2575400915 >> v2:172.21.15.73:3300/0 conn(0x7f942c0caa30 0x7f942c0b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.540+0000 7f942b400640 1 -- 172.21.15.46:0/2575400915 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0af870 con 0x7f942c160d10 2024-03-29T08:18:19.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.544+0000 7f7eff600640 1 -- 172.21.15.46:0/4219022109 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7ef0047bf0 msgr2=0x7f7ef00eb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.544+0000 7f7eff600640 1 --2- 172.21.15.46:0/4219022109 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7ef0047bf0 0x7f7ef00eb860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0097f0 tx=0x7f7eec01ec90 comp rx=0 tx=0).stop 2024-03-29T08:18:19.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.544+0000 7f7eff600640 1 -- 172.21.15.46:0/4219022109 shutdown_connections 2024-03-29T08:18:19.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.544+0000 7f7eff600640 1 --2- 172.21.15.46:0/4219022109 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7ef0047bf0 0x7f7ef00eb860 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.544+0000 7f7eff600640 1 -- 172.21.15.46:0/4219022109 >> 172.21.15.46:0/4219022109 conn(0x7f7ef0009f90 msgr2=0x7f7ef000a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.544+0000 7f7eff600640 1 -- 172.21.15.46:0/4219022109 shutdown_connections 2024-03-29T08:18:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.544+0000 7f7eff600640 1 -- 172.21.15.46:0/4219022109 wait complete. 2024-03-29T08:18:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.544+0000 7fe0d3200640 1 Processor -- start 2024-03-29T08:18:19.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.544+0000 7fe0d3200640 1 -- start start 2024-03-29T08:18:19.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.544+0000 7fc3e2e00640 1 Processor -- start 2024-03-29T08:18:19.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.544+0000 7fc3e2e00640 1 -- start start 2024-03-29T08:18:19.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7f942b400640 1 --2- 172.21.15.46:0/2575400915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f942c160d10 0x7f942c0cc640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f942c071700 tx=0x7f942401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb45cce5da0ac05d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fe0d3200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe0d4160e00 0x7fe0d406cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fe0d3200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0d406d9f0 0x7fe0d40c1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fe0d3200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0d40c17c0 0x7fe0d4069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fe0d3200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d40adc90 con 0x7fe0d406d9f0 2024-03-29T08:18:19.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fe0d3200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d40b2340 con 0x7fe0d4160e00 2024-03-29T08:18:19.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fe0d3200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d40c1c30 con 0x7fe0d40c17c0 2024-03-29T08:18:19.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7f7eff600640 1 Processor -- start 2024-03-29T08:18:19.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7f942a000640 1 -- 172.21.15.46:0/2575400915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94240a59f0 con 0x7f942c160d10 2024-03-29T08:18:19.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7f942a000640 1 -- 172.21.15.46:0/2575400915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94240a5b50 con 0x7f942c160d10 2024-03-29T08:18:19.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7f942a000640 1 -- 172.21.15.46:0/2575400915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94240ad440 con 0x7f942c160d10 2024-03-29T08:18:19.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7f7eff600640 1 -- start start 2024-03-29T08:18:19.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fe0d2800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0d40c17c0 0x7fe0d4069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fe0d1e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe0d4160e00 0x7fe0d406cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fc3e2e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc3d4053060 0x7fc3d4053ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fe0d2800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0d40c17c0 0x7fe0d4069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:57618/0 (socket says 172.21.15.46:57618) 2024-03-29T08:18:19.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fe0d2800640 1 -- 172.21.15.46:0/1501702764 learned_addr learned my addr 172.21.15.46:0/1501702764 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fc3e2e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc3d404fa50 0x7fc3d404fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fe0d2800640 1 -- 172.21.15.46:0/1501702764 >> v2:172.21.15.73:3300/0 conn(0x7fe0d4160e00 msgr2=0x7fe0d406cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fe0d2800640 1 --2- 172.21.15.46:0/1501702764 >> v2:172.21.15.73:3300/0 conn(0x7fe0d4160e00 0x7fe0d406cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fe0d2800640 1 -- 172.21.15.46:0/1501702764 >> v2:172.21.15.46:3300/0 conn(0x7fe0d406d9f0 msgr2=0x7fe0d40c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fe0d2800640 1 --2- 172.21.15.46:0/1501702764 >> v2:172.21.15.46:3300/0 conn(0x7fe0d406d9f0 0x7fe0d40c1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fe0d2800640 1 -- 172.21.15.46:0/1501702764 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d406df50 con 0x7fe0d40c17c0 2024-03-29T08:18:19.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fc3e2e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d4005ec0 con 0x7fc3d40df260 2024-03-29T08:18:19.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fc3e2e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d4050420 con 0x7fc3d4053060 2024-03-29T08:18:19.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fc3e2e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d4006160 con 0x7fc3d404fa50 2024-03-29T08:18:19.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7f4583800640 1 Processor -- start 2024-03-29T08:18:19.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fc3e1000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc3d4053060 0x7fc3d4053ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fc3e1000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc3d4053060 0x7fc3d4053ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59896/0 (socket says 172.21.15.46:59896) 2024-03-29T08:18:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fc3e1000640 1 -- 172.21.15.46:0/2754473703 learned_addr learned my addr 172.21.15.46:0/2754473703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7f4583800640 1 -- start start 2024-03-29T08:18:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7f9432400640 1 -- 172.21.15.46:0/2575400915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f942c160d10 msgr2=0x7f942c0cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fc3e1000640 1 -- 172.21.15.46:0/2754473703 >> v2:172.21.15.73:3300/0 conn(0x7fc3d404fa50 msgr2=0x7fc3d404fee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T08:18:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fc3e1000640 1 --2- 172.21.15.46:0/2754473703 >> v2:172.21.15.73:3300/0 conn(0x7fc3d404fa50 0x7fc3d404fee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fc3e1000640 1 -- 172.21.15.46:0/2754473703 >> v1:172.21.15.46:6789/0 conn(0x7fc3d40df260 legacy=0x7fc3d4007200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7fc3e1000640 1 -- 172.21.15.46:0/2754473703 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d4006440 con 0x7fc3d4053060 2024-03-29T08:18:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7f9432400640 1 --2- 172.21.15.46:0/2575400915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f942c160d10 0x7f942c0cc640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f942c071700 tx=0x7f942401c950 comp rx=0 tx=0).stop 2024-03-29T08:18:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f9432400640 1 -- 172.21.15.46:0/2575400915 shutdown_connections 2024-03-29T08:18:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f9432400640 1 --2- 172.21.15.46:0/2575400915 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f942c0b0ba0 0x7f942c06d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f9432400640 1 --2- 172.21.15.46:0/2575400915 >> v2:172.21.15.73:3300/0 conn(0x7f942c0caa30 0x7f942c0b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f9432400640 1 --2- 172.21.15.46:0/2575400915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f942c160d10 0x7f942c0cc640 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f9432400640 1 -- 172.21.15.46:0/2575400915 >> 172.21.15.46:0/2575400915 conn(0x7f942c066080 msgr2=0x7f942c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.548+0000 7f7eff600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0005ec0 0x7f7ef017c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f9432400640 1 -- 172.21.15.46:0/2575400915 shutdown_connections 2024-03-29T08:18:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f4583800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f457c141010 0x7f457c06db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f4583800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f457c15e4f0 0x7f457c06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f4583800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f457c1413f0 con 0x7f457c0702a0 2024-03-29T08:18:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f457be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f457c15e4f0 0x7f457c06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f4583800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f457c071380 con 0x7f457c141010 2024-03-29T08:18:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f4583800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f457c06e0c0 con 0x7f457c15e4f0 2024-03-29T08:18:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f457be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f457c15e4f0 0x7f457c06c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51902/0 (socket says 172.21.15.46:51902) 2024-03-29T08:18:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f457be00640 1 -- 172.21.15.46:0/368880417 learned_addr learned my addr 172.21.15.46:0/368880417 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7eff600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7ef0033e10 0x7f7ef00e8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7eff600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ef0047bf0 0x7f7ef00e4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7eff600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef00d5880 con 0x7f7ef0047bf0 2024-03-29T08:18:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7eff600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef00ea4c0 con 0x7f7ef0005ec0 2024-03-29T08:18:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7eff600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef00e98b0 con 0x7f7ef0033e10 2024-03-29T08:18:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7efcc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0005ec0 0x7f7ef017c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7efcc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0005ec0 0x7f7ef017c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51898/0 (socket says 172.21.15.46:51898) 2024-03-29T08:18:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7efcc00640 1 -- 172.21.15.46:0/3666302307 learned_addr learned my addr 172.21.15.46:0/3666302307 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7ef7e00640 1 --2- 172.21.15.46:0/3666302307 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7ef0033e10 0x7f7ef00e8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f9432400640 1 -- 172.21.15.46:0/2575400915 wait complete. 2024-03-29T08:18:19.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7efcc00640 1 -- 172.21.15.46:0/3666302307 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7ef0033e10 msgr2=0x7f7ef00e8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7efcc00640 1 --2- 172.21.15.46:0/3666302307 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7ef0033e10 0x7f7ef00e8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7efcc00640 1 -- 172.21.15.46:0/3666302307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ef0047bf0 msgr2=0x7f7ef00e4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7efcc00640 1 --2- 172.21.15.46:0/3666302307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ef0047bf0 0x7f7ef00e4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7efcc00640 1 -- 172.21.15.46:0/3666302307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec009420 con 0x7f7ef0005ec0 2024-03-29T08:18:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7efcc00640 1 --2- 172.21.15.46:0/3666302307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0005ec0 0x7f7ef017c870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7ee400c490 tx=0x7f7ee400c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f457be00640 1 -- 172.21.15.46:0/368880417 >> v2:172.21.15.46:3300/0 conn(0x7f457c141010 msgr2=0x7f457c06db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f457be00640 1 --2- 172.21.15.46:0/368880417 >> v2:172.21.15.46:3300/0 conn(0x7f457c141010 0x7f457c06db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f457be00640 1 -- 172.21.15.46:0/368880417 >> v1:172.21.15.73:6789/0 conn(0x7f457c0702a0 legacy=0x7f457c0ae530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T08:18:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f457be00640 1 -- 172.21.15.46:0/368880417 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c06c8d0 con 0x7f457c15e4f0 2024-03-29T08:18:19.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7fe0d2800640 1 --2- 172.21.15.46:0/1501702764 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0d40c17c0 0x7fe0d4069d90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe0d406e4f0 tx=0x7fe0cc01c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=514a1c996c102ada server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7ef4c00640 1 -- 172.21.15.46:0/3666302307 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4012830 con 0x7f7ef0005ec0 2024-03-29T08:18:19.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7ef4c00640 1 -- 172.21.15.46:0/3666302307 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee4012990 con 0x7f7ef0005ec0 2024-03-29T08:18:19.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7ef4c00640 1 -- 172.21.15.46:0/3666302307 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4012d80 con 0x7f7ef0005ec0 2024-03-29T08:18:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7eff600640 1 -- 172.21.15.46:0/3666302307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef00e9b90 con 0x7f7ef0005ec0 2024-03-29T08:18:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7f7eff600640 1 -- 172.21.15.46:0/3666302307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef00ea150 con 0x7f7ef0005ec0 2024-03-29T08:18:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.552+0000 7fe0d0a00640 1 -- 172.21.15.46:0/1501702764 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc0a8420 con 0x7fe0d40c17c0 2024-03-29T08:18:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fe0d0a00640 1 -- 172.21.15.46:0/1501702764 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0cc0a8580 con 0x7fe0d40c17c0 2024-03-29T08:18:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fe0d0a00640 1 -- 172.21.15.46:0/1501702764 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc0a8910 con 0x7fe0d40c17c0 2024-03-29T08:18:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7f9432400640 1 Processor -- start 2024-03-29T08:18:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fdf13400640 1 Processor -- start 2024-03-29T08:18:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7f9432400640 1 -- start start 2024-03-29T08:18:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fc3e1000640 1 --2- 172.21.15.46:0/2754473703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d4053060 0x7fc3d4053ee0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400d4b0 tx=0x7fc3c801e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7acb54998b12c712 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fdf13400640 1 -- start start 2024-03-29T08:18:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fe0d3200640 1 -- 172.21.15.46:0/1501702764 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0d40c17c0 msgr2=0x7fe0d4069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fe0d3200640 1 --2- 172.21.15.46:0/1501702764 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0d40c17c0 0x7fe0d4069d90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe0d406e4f0 tx=0x7fe0cc01c730 comp rx=0 tx=0).stop 2024-03-29T08:18:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fe0d3200640 1 -- 172.21.15.46:0/1501702764 shutdown_connections 2024-03-29T08:18:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fe0d3200640 1 --2- 172.21.15.46:0/1501702764 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0d40c17c0 0x7fe0d4069d90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fe0d3200640 1 --2- 172.21.15.46:0/1501702764 >> v2:172.21.15.46:3300/0 conn(0x7fe0d406d9f0 0x7fe0d40c1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fe0d3200640 1 --2- 172.21.15.46:0/1501702764 >> v2:172.21.15.73:3300/0 conn(0x7fe0d4160e00 0x7fe0d406cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fe0d3200640 1 -- 172.21.15.46:0/1501702764 >> 172.21.15.46:0/1501702764 conn(0x7fe0d40b0030 msgr2=0x7fe0d40b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fc3dbe00640 1 -- 172.21.15.46:0/2754473703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80a79b0 con 0x7fc3d4053060 2024-03-29T08:18:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fc3dbe00640 1 -- 172.21.15.46:0/2754473703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c80ab070 con 0x7fc3d4053060 2024-03-29T08:18:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fc3dbe00640 1 -- 172.21.15.46:0/2754473703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80a7d30 con 0x7fc3d4053060 2024-03-29T08:18:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fe0d3200640 1 -- 172.21.15.46:0/1501702764 shutdown_connections 2024-03-29T08:18:19.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7f7eff600640 1 -- 172.21.15.46:0/3666302307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef00052a0 con 0x7f7ef0005ec0 2024-03-29T08:18:19.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fe0d3200640 1 -- 172.21.15.46:0/1501702764 wait complete. 2024-03-29T08:18:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fc3e2e00640 1 -- 172.21.15.46:0/2754473703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d4053060 msgr2=0x7fc3d4053ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fc3e2e00640 1 --2- 172.21.15.46:0/2754473703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d4053060 0x7fc3d4053ee0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400d4b0 tx=0x7fc3c801e9f0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fc3e2e00640 1 -- 172.21.15.46:0/2754473703 shutdown_connections 2024-03-29T08:18:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fc3e2e00640 1 --2- 172.21.15.46:0/2754473703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d4053060 0x7fc3d4053ee0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fc3e2e00640 1 --2- 172.21.15.46:0/2754473703 >> v2:172.21.15.73:3300/0 conn(0x7fc3d404fa50 0x7fc3d404fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fc3e2e00640 1 -- 172.21.15.46:0/2754473703 >> 172.21.15.46:0/2754473703 conn(0x7fc3d4009f90 msgr2=0x7fc3d400a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fc3e2e00640 1 -- 172.21.15.46:0/2754473703 shutdown_connections 2024-03-29T08:18:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7fc3e2e00640 1 -- 172.21.15.46:0/2754473703 wait complete. 2024-03-29T08:18:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7f9432400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f942c0b0ba0 0x7f942c1616a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7f9432400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0caa30 0x7f942c15ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7f457be00640 1 --2- 172.21.15.46:0/368880417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457c15e4f0 0x7f457c06c390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f457c0b3aa0 tx=0x7f456801c4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d089ddd6546249e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.556+0000 7f7ef4c00640 1 -- 172.21.15.46:0/3666302307 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7ee401a3b0 con 0x7f7ef0005ec0 2024-03-29T08:18:19.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f9432400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f942c160d10 0x7f942c201b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f9432400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c15f240 con 0x7f942c160d10 2024-03-29T08:18:19.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f9432400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c15f5d0 con 0x7f942c0caa30 2024-03-29T08:18:19.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f9432400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c15f8b0 con 0x7f942c0b0ba0 2024-03-29T08:18:19.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fdf13400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf0c142ab0 0x7fdf0c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fdf13400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdf0c06d9e0 0x7fdf0c15d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fdf13400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0c06de40 con 0x7fdf0c15d940 2024-03-29T08:18:19.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fdf13400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf0c070c70 con 0x7fdf0c06d9e0 2024-03-29T08:18:19.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fdf13400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c070340 con 0x7fdf0c142ab0 2024-03-29T08:18:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f942aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0caa30 0x7f942c15ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f457b400640 1 -- 172.21.15.46:0/368880417 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680ac630 con 0x7f457c15e4f0 2024-03-29T08:18:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f457b400640 1 -- 172.21.15.46:0/368880417 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45680ac790 con 0x7f457c15e4f0 2024-03-29T08:18:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f457b400640 1 -- 172.21.15.46:0/368880417 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680acb20 con 0x7f457c15e4f0 2024-03-29T08:18:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fe0d3200640 1 Processor -- start 2024-03-29T08:18:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f7ef4c00640 1 --2- 172.21.15.46:0/3666302307 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ed80666e0 0x7f7ed8068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f942aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0caa30 0x7f942c15ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51914/0 (socket says 172.21.15.46:51914) 2024-03-29T08:18:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f942aa00640 1 -- 172.21.15.46:0/677099644 learned_addr learned my addr 172.21.15.46:0/677099644 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f942b400640 1 --2- 172.21.15.46:0/677099644 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f942c0b0ba0 0x7f942c1616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f942aa00640 1 -- 172.21.15.46:0/677099644 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f942c0b0ba0 msgr2=0x7f942c1616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fdf0be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdf0c06d9e0 0x7fdf0c15d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fdf0be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdf0c06d9e0 0x7fdf0c15d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51920/0 (socket says 172.21.15.46:51920) 2024-03-29T08:18:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fdf0be00640 1 -- 172.21.15.46:0/3205677633 learned_addr learned my addr 172.21.15.46:0/3205677633 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f7ef4c00640 1 -- 172.21.15.46:0/3666302307 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ee4083c10 con 0x7f7ef0005ec0 2024-03-29T08:18:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fc3e2e00640 1 Processor -- start 2024-03-29T08:18:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fdf10a00640 1 --2- 172.21.15.46:0/3205677633 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf0c142ab0 0x7fdf0c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f942aa00640 1 --2- 172.21.15.46:0/677099644 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f942c0b0ba0 0x7f942c1616a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f942aa00640 1 -- 172.21.15.46:0/677099644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f942c160d10 msgr2=0x7f942c201b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f942aa00640 1 --2- 172.21.15.46:0/677099644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f942c160d10 0x7f942c201b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f942aa00640 1 -- 172.21.15.46:0/677099644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9424009a70 con 0x7f942c0caa30 2024-03-29T08:18:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fc3e2e00640 1 -- start start 2024-03-29T08:18:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f942b400640 1 --2- 172.21.15.46:0/677099644 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f942c0b0ba0 0x7f942c1616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fdf0be00640 1 -- 172.21.15.46:0/3205677633 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf0c142ab0 msgr2=0x7fdf0c06d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fdf0be00640 1 --2- 172.21.15.46:0/3205677633 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf0c142ab0 0x7fdf0c06d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fdf0be00640 1 -- 172.21.15.46:0/3205677633 >> v1:172.21.15.46:6789/0 conn(0x7fdf0c15d940 legacy=0x7fdf0c0ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f942aa00640 1 --2- 172.21.15.46:0/677099644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0caa30 0x7f942c15ed00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f941c00e810 tx=0x7f941c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f7ef7e00640 1 --2- 172.21.15.46:0/3666302307 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ed80666e0 0x7f7ed8068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fdf0be00640 1 -- 172.21.15.46:0/3205677633 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c070620 con 0x7fdf0c06d9e0 2024-03-29T08:18:19.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fe0d3200640 1 -- start start 2024-03-29T08:18:19.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f4583800640 1 -- 172.21.15.46:0/368880417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457c15e4f0 msgr2=0x7f457c06c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fe0d3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d406d9f0 0x7fe0d4163ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f4583800640 1 --2- 172.21.15.46:0/368880417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457c15e4f0 0x7f457c06c390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f457c0b3aa0 tx=0x7f456801c4e0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f4583800640 1 -- 172.21.15.46:0/368880417 shutdown_connections 2024-03-29T08:18:19.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f4583800640 1 --2- 172.21.15.46:0/368880417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457c15e4f0 0x7f457c06c390 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f4583800640 1 --2- 172.21.15.46:0/368880417 >> v2:172.21.15.46:3300/0 conn(0x7f457c141010 0x7f457c06db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f4583800640 1 -- 172.21.15.46:0/368880417 >> 172.21.15.46:0/368880417 conn(0x7f457c066080 msgr2=0x7f457c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fe0d3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0d40c17c0 0x7fe0d4159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fe0d3200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0d4160e00 0x7fe0d415bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fe0d3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d41626f0 con 0x7fe0d406d9f0 2024-03-29T08:18:19.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fe0d3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d4158900 con 0x7fe0d40c17c0 2024-03-29T08:18:19.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fe0d3200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d4157c80 con 0x7fe0d4160e00 2024-03-29T08:18:19.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fe0d1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d406d9f0 0x7fe0d4163ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fe0d1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0d40c17c0 0x7fe0d4159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fe0d1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d406d9f0 0x7fe0d4163ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59948/0 (socket says 172.21.15.46:59948) 2024-03-29T08:18:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7fe0d1e00640 1 -- 172.21.15.46:0/1986345609 learned_addr learned my addr 172.21.15.46:0/1986345609 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.560+0000 7f7ef7e00640 1 --2- 172.21.15.46:0/3666302307 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ed80666e0 0x7f7ed8068ba0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0b2000 tx=0x7f7eec0aa070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fe0d2800640 1 --2- 172.21.15.46:0/1986345609 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0d4160e00 0x7fe0d415bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fe0d1400640 1 -- 172.21.15.46:0/1986345609 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0d4160e00 msgr2=0x7fe0d415bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fe0d1400640 1 --2- 172.21.15.46:0/1986345609 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0d4160e00 0x7fe0d415bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fe0d1400640 1 -- 172.21.15.46:0/1986345609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d406d9f0 msgr2=0x7fe0d4163ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fe0d1400640 1 --2- 172.21.15.46:0/1986345609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d406d9f0 0x7fe0d4163ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fe0d1400640 1 -- 172.21.15.46:0/1986345609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0cc009b30 con 0x7fe0d40c17c0 2024-03-29T08:18:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e2e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d404fa50 0x7fc3d40e0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fe0d1400640 1 --2- 172.21.15.46:0/1986345609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0d40c17c0 0x7fe0d4159d60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000b3f0 tx=0x7fe0c000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e2e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc3d4053060 0x7fc3d404d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e2e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3d40df260 0x7fc3d40d44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e2e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d404c620 con 0x7fc3d404fa50 2024-03-29T08:18:19.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e2e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d404c900 con 0x7fc3d40df260 2024-03-29T08:18:19.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e2e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d404cbe0 con 0x7fc3d4053060 2024-03-29T08:18:19.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d404fa50 0x7fc3d40e0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7f4583800640 1 -- 172.21.15.46:0/368880417 shutdown_connections 2024-03-29T08:18:19.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e1a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d404fa50 0x7fc3d40e0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59962/0 (socket says 172.21.15.46:59962) 2024-03-29T08:18:19.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e1a00640 1 -- 172.21.15.46:0/1216002520 learned_addr learned my addr 172.21.15.46:0/1216002520 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e1000640 1 --2- 172.21.15.46:0/1216002520 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc3d4053060 0x7fc3d404d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7f4583800640 1 -- 172.21.15.46:0/368880417 wait complete. 2024-03-29T08:18:19.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e2400640 1 --2- 172.21.15.46:0/1216002520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3d40df260 0x7fc3d40d44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e1a00640 1 -- 172.21.15.46:0/1216002520 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc3d4053060 msgr2=0x7fc3d404d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e1a00640 1 --2- 172.21.15.46:0/1216002520 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc3d4053060 0x7fc3d404d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e1a00640 1 -- 172.21.15.46:0/1216002520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3d40df260 msgr2=0x7fc3d40d44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e1a00640 1 --2- 172.21.15.46:0/1216002520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3d40df260 0x7fc3d40d44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e1a00640 1 -- 172.21.15.46:0/1216002520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c8008550 con 0x7fc3d404fa50 2024-03-29T08:18:19.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7f9423400640 1 -- 172.21.15.46:0/677099644 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c00c8d0 con 0x7f942c0caa30 2024-03-29T08:18:19.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7f9423400640 1 -- 172.21.15.46:0/677099644 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941c00ca30 con 0x7f942c0caa30 2024-03-29T08:18:19.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7f9423400640 1 -- 172.21.15.46:0/677099644 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c00cde0 con 0x7f942c0caa30 2024-03-29T08:18:19.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e1a00640 1 --2- 172.21.15.46:0/1216002520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d404fa50 0x7fc3d40e0e90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000c490 tx=0x7fc3d0001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3d9600640 1 -- 172.21.15.46:0/1216002520 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0003600 con 0x7fc3d404fa50 2024-03-29T08:18:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e2e00640 1 -- 172.21.15.46:0/1216002520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d404cec0 con 0x7fc3d404fa50 2024-03-29T08:18:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3e2e00640 1 -- 172.21.15.46:0/1216002520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d40d5280 con 0x7fc3d404fa50 2024-03-29T08:18:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7f9432400640 1 -- 172.21.15.46:0/677099644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c161270 con 0x7f942c0caa30 2024-03-29T08:18:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7f9432400640 1 -- 172.21.15.46:0/677099644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c163c80 con 0x7f942c0caa30 2024-03-29T08:18:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fe0ca000640 1 -- 172.21.15.46:0/1986345609 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0011830 con 0x7fe0d40c17c0 2024-03-29T08:18:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fe0ca000640 1 -- 172.21.15.46:0/1986345609 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c0011990 con 0x7fe0d40c17c0 2024-03-29T08:18:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fe0ca000640 1 -- 172.21.15.46:0/1986345609 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0011d40 con 0x7fe0d40c17c0 2024-03-29T08:18:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fe0d3200640 1 -- 172.21.15.46:0/1986345609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d4157f60 con 0x7fe0d40c17c0 2024-03-29T08:18:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fe0d3200640 1 -- 172.21.15.46:0/1986345609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d4158520 con 0x7fe0d40c17c0 2024-03-29T08:18:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f9420c00640 1 -- 172.21.15.46:0/677099644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942c04e040 con 0x7f942c0caa30 2024-03-29T08:18:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7fdf0be00640 1 --2- 172.21.15.46:0/3205677633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf0c06d9e0 0x7fdf0c15d400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009e50 tx=0x7fdf0401ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e3d3c7353a35707 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3d9600640 1 -- 172.21.15.46:0/1216002520 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d0003760 con 0x7fc3d404fa50 2024-03-29T08:18:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.564+0000 7fc3d9600640 1 -- 172.21.15.46:0/1216002520 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0003600 con 0x7fc3d404fa50 2024-03-29T08:18:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f4583800640 1 Processor -- start 2024-03-29T08:18:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7fc3cea00640 1 -- 172.21.15.46:0/1216002520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d40052a0 con 0x7fc3d404fa50 2024-03-29T08:18:19.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f4583800640 1 -- start start 2024-03-29T08:18:19.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7fe0d3200640 1 -- 172.21.15.46:0/1986345609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d404e040 con 0x7fe0d40c17c0 2024-03-29T08:18:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f9423400640 1 -- 172.21.15.46:0/677099644 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f941c013410 con 0x7f942c0caa30 2024-03-29T08:18:19.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f7ef4c00640 1 -- 172.21.15.46:0/3666302307 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7ee40510f0 con 0x7f7ef0005ec0 2024-03-29T08:18:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7fdf0b400640 1 -- 172.21.15.46:0/3205677633 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf040a5610 con 0x7fdf0c06d9e0 2024-03-29T08:18:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7fdf0b400640 1 -- 172.21.15.46:0/3205677633 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf040a5770 con 0x7fdf0c06d9e0 2024-03-29T08:18:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7fdf0b400640 1 -- 172.21.15.46:0/3205677633 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf040a5b00 con 0x7fdf0c06d9e0 2024-03-29T08:18:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f9423400640 1 --2- 172.21.15.46:0/677099644 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f940c0666e0 0x7f940c068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f9423400640 1 -- 172.21.15.46:0/677099644 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f941c083420 con 0x7f942c0caa30 2024-03-29T08:18:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7f942b400640 1 --2- 172.21.15.46:0/677099644 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f940c0666e0 0x7f940c068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7fdf13400640 1 -- 172.21.15.46:0/3205677633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf0c06d9e0 msgr2=0x7fdf0c15d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7fdf13400640 1 --2- 172.21.15.46:0/3205677633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf0c06d9e0 0x7fdf0c15d400 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009e50 tx=0x7fdf0401ca30 comp rx=0 tx=0).stop 2024-03-29T08:18:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7fdf13400640 1 -- 172.21.15.46:0/3205677633 shutdown_connections 2024-03-29T08:18:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7fdf13400640 1 --2- 172.21.15.46:0/3205677633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf0c06d9e0 0x7fdf0c15d400 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7fdf13400640 1 --2- 172.21.15.46:0/3205677633 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf0c142ab0 0x7fdf0c06d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7fdf13400640 1 -- 172.21.15.46:0/3205677633 >> 172.21.15.46:0/3205677633 conn(0x7fdf0c066080 msgr2=0x7fdf0c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7f942b400640 1 --2- 172.21.15.46:0/677099644 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f940c0666e0 0x7f940c068ba0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f94240af000 tx=0x7f94240a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7fe0ca000640 1 -- 172.21.15.46:0/1986345609 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe0c001a400 con 0x7fe0d40c17c0 2024-03-29T08:18:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7fe0ca000640 1 --2- 172.21.15.46:0/1986345609 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe0b4066760 0x7fe0b4068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7fe0ca000640 1 -- 172.21.15.46:0/1986345609 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0c0084290 con 0x7fe0d40c17c0 2024-03-29T08:18:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f4583800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457c0702a0 0x7f457c159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f4583800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f457c141010 0x7f457c1415b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f4583800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457c15e4f0 0x7f457c13cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f4583800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457c13e690 con 0x7f457c0702a0 2024-03-29T08:18:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f4583800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457c159bb0 con 0x7f457c15e4f0 2024-03-29T08:18:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f4583800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f457c13da80 con 0x7f457c141010 2024-03-29T08:18:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f4580e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457c0702a0 0x7f457c159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f4580e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457c0702a0 0x7f457c159670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59968/0 (socket says 172.21.15.46:59968) 2024-03-29T08:18:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f457be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f457c141010 0x7f457c1415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f4580e00640 1 -- 172.21.15.46:0/1252236139 learned_addr learned my addr 172.21.15.46:0/1252236139 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f4581800640 1 --2- 172.21.15.46:0/1252236139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457c15e4f0 0x7f457c13cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f457be00640 1 -- 172.21.15.46:0/1252236139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457c15e4f0 msgr2=0x7f457c13cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f457be00640 1 --2- 172.21.15.46:0/1252236139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457c15e4f0 0x7f457c13cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f457be00640 1 -- 172.21.15.46:0/1252236139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457c0702a0 msgr2=0x7f457c159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f457be00640 1 --2- 172.21.15.46:0/1252236139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457c0702a0 0x7f457c159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.568+0000 7f457be00640 1 -- 172.21.15.46:0/1252236139 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45680094d0 con 0x7f457c141010 2024-03-29T08:18:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7f457be00640 1 --2- 172.21.15.46:0/1252236139 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f457c141010 0x7f457c1415b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f45680ac000 tx=0x7f4568019280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7f4578c00640 1 -- 172.21.15.46:0/1252236139 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680b5830 con 0x7f457c141010 2024-03-29T08:18:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7f4578c00640 1 -- 172.21.15.46:0/1252236139 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45680b5990 con 0x7f457c141010 2024-03-29T08:18:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7f4578c00640 1 -- 172.21.15.46:0/1252236139 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680b5d00 con 0x7f457c141010 2024-03-29T08:18:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7f4583800640 1 -- 172.21.15.46:0/1252236139 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457c13dd60 con 0x7f457c141010 2024-03-29T08:18:19.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7f4583800640 1 -- 172.21.15.46:0/1252236139 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f457c13e320 con 0x7f457c141010 2024-03-29T08:18:19.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7fdf13400640 1 -- 172.21.15.46:0/3205677633 shutdown_connections 2024-03-29T08:18:19.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7f9423400640 1 -- 172.21.15.46:0/677099644 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f941c04b980 con 0x7f942c0caa30 2024-03-29T08:18:19.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7fe0d1e00640 1 --2- 172.21.15.46:0/1986345609 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe0b4066760 0x7fe0b4068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7fc3d9600640 1 -- 172.21.15.46:0/1216002520 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc3d0003930 con 0x7fc3d404fa50 2024-03-29T08:18:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.576+0000 7fc3d9600640 1 --2- 172.21.15.46:0/1216002520 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc3bc0666f0 0x7fc3bc068bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7f4583800640 1 -- 172.21.15.46:0/1252236139 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f457c0cbe10 con 0x7f457c141010 2024-03-29T08:18:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.572+0000 7f4578c00640 1 -- 172.21.15.46:0/1252236139 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f45680b3070 con 0x7f457c141010 2024-03-29T08:18:19.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.576+0000 7fc3d9600640 1 -- 172.21.15.46:0/1216002520 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3d0083830 con 0x7fc3d404fa50 2024-03-29T08:18:19.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.576+0000 7fdf13400640 1 -- 172.21.15.46:0/3205677633 wait complete. 2024-03-29T08:18:19.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.576+0000 7fe0d1e00640 1 --2- 172.21.15.46:0/1986345609 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe0b4066760 0x7fe0b4068c20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe0d415a690 tx=0x7fe0c4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.576+0000 7f4578c00640 1 --2- 172.21.15.46:0/1252236139 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f455c066790 0x7f455c068c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.576+0000 7f4580e00640 1 --2- 172.21.15.46:0/1252236139 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f455c066790 0x7f455c068c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.576+0000 7f4578c00640 1 -- 172.21.15.46:0/1252236139 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f456811e900 con 0x7f457c141010 2024-03-29T08:18:19.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.576+0000 7fc3e1000640 1 --2- 172.21.15.46:0/1216002520 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc3bc0666f0 0x7fc3bc068bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.576+0000 7f4580e00640 1 --2- 172.21.15.46:0/1252236139 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f455c066790 0x7f455c068c50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4570009a50 tx=0x7f4570009210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.576+0000 7fc3e1000640 1 --2- 172.21.15.46:0/1216002520 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc3bc0666f0 0x7fc3bc068bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80b2000 tx=0x7fc3c800a040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.576+0000 7fc3d9600640 1 -- 172.21.15.46:0/1216002520 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc3d0050c30 con 0x7fc3d404fa50 2024-03-29T08:18:19.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.576+0000 7f62da800640 1 Processor -- start 2024-03-29T08:18:19.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7f62da800640 1 -- start start 2024-03-29T08:18:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7fdf13400640 1 Processor -- start 2024-03-29T08:18:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7fe0ca000640 1 -- 172.21.15.46:0/1986345609 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe0c0051670 con 0x7fe0d40c17c0 2024-03-29T08:18:19.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7fdf13400640 1 -- start start 2024-03-29T08:18:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7f62da800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f62d415aab0 0x7f62d40ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7f62da800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f62d40cb120 0x7f62d40cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7f62da800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d40ae120 con 0x7f62d40ad450 2024-03-29T08:18:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7f62da800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d40d1500 con 0x7f62d415aab0 2024-03-29T08:18:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7f62da800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d40b01b0 con 0x7f62d40cb120 2024-03-29T08:18:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7fdf13400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf0c06d9e0 0x7fdf0c158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7fdf13400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf0c142ab0 0x7fdf0c160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7fdf13400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf0c15d940 0x7fdf0c161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7fdf13400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1641f0 con 0x7fdf0c142ab0 2024-03-29T08:18:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7fdf13400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c13cdd0 con 0x7fdf0c15d940 2024-03-29T08:18:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7fdf13400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c13d0b0 con 0x7fdf0c06d9e0 2024-03-29T08:18:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7f4578c00640 1 -- 172.21.15.46:0/1252236139 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f45680ebbc0 con 0x7f457c141010 2024-03-29T08:18:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7fdf11400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf0c15d940 0x7fdf0c161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7fdf11400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf0c15d940 0x7fdf0c161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51960/0 (socket says 172.21.15.46:51960) 2024-03-29T08:18:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7fdf11400640 1 -- 172.21.15.46:0/339091899 learned_addr learned my addr 172.21.15.46:0/339091899 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7fdf0be00640 1 --2- 172.21.15.46:0/339091899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf0c142ab0 0x7fdf0c160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7f62d3e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f62d40ad450 0x7f62d40ad900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:43132/0 (socket says 172.21.15.46:43132) 2024-03-29T08:18:19.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7f62d3e00640 1 -- 172.21.15.46:0/2035927166 learned_addr learned my addr 172.21.15.46:0/2035927166 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:19.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7f62d2a00640 1 --2- 172.21.15.46:0/2035927166 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f62d40cb120 0x7f62d40cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7fdf10a00640 1 --2- 172.21.15.46:0/339091899 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf0c06d9e0 0x7fdf0c158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7fdf0be00640 1 -- 172.21.15.46:0/339091899 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf0c06d9e0 msgr2=0x7fdf0c158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.580+0000 7f62d3400640 1 --2- 172.21.15.46:0/2035927166 >> v2:172.21.15.46:3300/0 conn(0x7f62d415aab0 0x7f62d40ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7f62d2a00640 1 -- 172.21.15.46:0/2035927166 >> v2:172.21.15.46:3300/0 conn(0x7f62d415aab0 msgr2=0x7f62d40ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7fdf0be00640 1 --2- 172.21.15.46:0/339091899 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf0c06d9e0 0x7fdf0c158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7fdf0be00640 1 -- 172.21.15.46:0/339091899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf0c15d940 msgr2=0x7fdf0c161090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7fdf0be00640 1 --2- 172.21.15.46:0/339091899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf0c15d940 0x7fdf0c161090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7fdf0be00640 1 -- 172.21.15.46:0/339091899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf04009b30 con 0x7fdf0c142ab0 2024-03-29T08:18:19.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7f62d2000640 1 -- 172.21.15.46:0/2035927166 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201213959 0 0) 0x7f62d40ae120 con 0x7f62d40ad450 2024-03-29T08:18:19.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7f62d2a00640 1 --2- 172.21.15.46:0/2035927166 >> v2:172.21.15.46:3300/0 conn(0x7f62d415aab0 0x7f62d40ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7f62d2a00640 1 -- 172.21.15.46:0/2035927166 >> v1:172.21.15.73:6789/0 conn(0x7f62d40ad450 legacy=0x7f62d40ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7f62d2a00640 1 -- 172.21.15.46:0/2035927166 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d40ae3e0 con 0x7f62d40cb120 2024-03-29T08:18:19.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7fdf0be00640 1 --2- 172.21.15.46:0/339091899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf0c142ab0 0x7fdf0c160b50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdf040ab000 tx=0x7fdf040197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7f62d3400640 1 --2- 172.21.15.46:0/2035927166 >> v2:172.21.15.46:3300/0 conn(0x7f62d415aab0 0x7f62d40ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7fdf08c00640 1 -- 172.21.15.46:0/339091899 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf040a5610 con 0x7fdf0c142ab0 2024-03-29T08:18:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7fdf08c00640 1 -- 172.21.15.46:0/339091899 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf040b8670 con 0x7fdf0c142ab0 2024-03-29T08:18:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7fdf08c00640 1 -- 172.21.15.46:0/339091899 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf040b8a20 con 0x7fdf0c142ab0 2024-03-29T08:18:19.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7fdf13400640 1 -- 172.21.15.46:0/339091899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c13d390 con 0x7fdf0c142ab0 2024-03-29T08:18:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.584+0000 7fdf13400640 1 -- 172.21.15.46:0/339091899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c159210 con 0x7fdf0c142ab0 2024-03-29T08:18:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.588+0000 7fdf13400640 1 -- 172.21.15.46:0/339091899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee4005260 con 0x7fdf0c142ab0 2024-03-29T08:18:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.588+0000 7fdf08c00640 1 -- 172.21.15.46:0/339091899 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdf040b8b80 con 0x7fdf0c142ab0 2024-03-29T08:18:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.588+0000 7f62d2a00640 1 --2- 172.21.15.46:0/2035927166 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f62d40cb120 0x7f62d40cf0d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f62c00097c0 tx=0x7f62c001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91d911b63cd1e367 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.588+0000 7f62d2000640 1 -- 172.21.15.46:0/2035927166 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c00ac410 con 0x7f62d40cb120 2024-03-29T08:18:19.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.588+0000 7fdf08c00640 1 --2- 172.21.15.46:0/339091899 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeec066760 0x7fdeec068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.588+0000 7f62d2000640 1 -- 172.21.15.46:0/2035927166 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c00ac570 con 0x7f62d40cb120 2024-03-29T08:18:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.588+0000 7f62d2000640 1 -- 172.21.15.46:0/2035927166 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c00ac410 con 0x7f62d40cb120 2024-03-29T08:18:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.588+0000 7fdf08c00640 1 -- 172.21.15.46:0/339091899 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf0411e830 con 0x7fdf0c142ab0 2024-03-29T08:18:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.588+0000 7fdf10a00640 1 --2- 172.21.15.46:0/339091899 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeec066760 0x7fdeec068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.592+0000 7f62da800640 1 -- 172.21.15.46:0/2035927166 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f62d40cb120 msgr2=0x7f62d40cf0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.592+0000 7f62da800640 1 --2- 172.21.15.46:0/2035927166 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f62d40cb120 0x7f62d40cf0d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f62c00097c0 tx=0x7f62c001c4c0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.592+0000 7fdf10a00640 1 --2- 172.21.15.46:0/339091899 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeec066760 0x7fdeec068c20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009af0 tx=0x7fdefc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.592+0000 7f62da800640 1 -- 172.21.15.46:0/2035927166 shutdown_connections 2024-03-29T08:18:19.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.592+0000 7f62da800640 1 --2- 172.21.15.46:0/2035927166 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f62d40cb120 0x7f62d40cf0d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.592+0000 7f62da800640 1 --2- 172.21.15.46:0/2035927166 >> v2:172.21.15.46:3300/0 conn(0x7f62d415aab0 0x7f62d40ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.592+0000 7f62da800640 1 -- 172.21.15.46:0/2035927166 >> 172.21.15.46:0/2035927166 conn(0x7f62d4099a10 msgr2=0x7f62d4089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.592+0000 7f62da800640 1 -- 172.21.15.46:0/2035927166 shutdown_connections 2024-03-29T08:18:19.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.592+0000 7f62da800640 1 -- 172.21.15.46:0/2035927166 wait complete. 2024-03-29T08:18:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.596+0000 7f62da800640 1 Processor -- start 2024-03-29T08:18:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.600+0000 7f62da800640 1 -- start start 2024-03-29T08:18:19.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.600+0000 7fdf08c00640 1 -- 172.21.15.46:0/339091899 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdf040ebbc0 con 0x7fdf0c142ab0 2024-03-29T08:18:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.600+0000 7f62da800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f62d40ad450 0x7f62d4142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.600+0000 7f62da800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d40cb120 0x7f62d41432f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.600+0000 7f62da800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d415aab0 0x7f62d415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.600+0000 7f62d3400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f62d40ad450 0x7f62d4142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.600+0000 7f62d3400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f62d40ad450 0x7f62d4142db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:57716/0 (socket says 172.21.15.46:57716) 2024-03-29T08:18:19.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.600+0000 7f62d2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d40cb120 0x7f62d41432f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.600+0000 7f62d2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d40cb120 0x7f62d41432f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51970/0 (socket says 172.21.15.46:51970) 2024-03-29T08:18:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.600+0000 7f62d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d415aab0 0x7f62d415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.600+0000 7f62d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d415aab0 0x7f62d415bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59986/0 (socket says 172.21.15.46:59986) 2024-03-29T08:18:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.604+0000 7f62d3400640 1 -- 172.21.15.46:0/1401397134 learned_addr learned my addr 172.21.15.46:0/1401397134 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.604+0000 7f62da800640 1 -- 172.21.15.46:0/1401397134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d40cd6d0 con 0x7f62d415aab0 2024-03-29T08:18:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.604+0000 7f62da800640 1 -- 172.21.15.46:0/1401397134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d40b5da0 con 0x7f62d40cb120 2024-03-29T08:18:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.604+0000 7f62da800640 1 -- 172.21.15.46:0/1401397134 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d4143cd0 con 0x7f62d40ad450 2024-03-29T08:18:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.604+0000 7f62d3400640 1 -- 172.21.15.46:0/1401397134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d40cb120 msgr2=0x7f62d41432f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.604+0000 7f62d3400640 1 --2- 172.21.15.46:0/1401397134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d40cb120 0x7f62d41432f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.604+0000 7f62d3400640 1 -- 172.21.15.46:0/1401397134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d415aab0 msgr2=0x7f62d415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.604+0000 7f62d3400640 1 --2- 172.21.15.46:0/1401397134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d415aab0 0x7f62d415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.604+0000 7f62d3400640 1 -- 172.21.15.46:0/1401397134 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c0009420 con 0x7f62d40ad450 2024-03-29T08:18:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.608+0000 7f62d3400640 1 --2- 172.21.15.46:0/1401397134 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f62d40ad450 0x7f62d4142db0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f62c400c7f0 tx=0x7f62c400ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.608+0000 7f62cb400640 1 -- 172.21.15.46:0/1401397134 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4007820 con 0x7f62d40ad450 2024-03-29T08:18:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.608+0000 7f62da800640 1 -- 172.21.15.46:0/1401397134 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d40b5b00 con 0x7f62d40ad450 2024-03-29T08:18:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.608+0000 7f62da800640 1 -- 172.21.15.46:0/1401397134 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d415dce0 con 0x7f62d40ad450 2024-03-29T08:18:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.608+0000 7f62cb400640 1 -- 172.21.15.46:0/1401397134 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c4007980 con 0x7f62d40ad450 2024-03-29T08:18:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.608+0000 7f62cb400640 1 -- 172.21.15.46:0/1401397134 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c4007820 con 0x7f62d40ad450 2024-03-29T08:18:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.612+0000 7f62da800640 1 -- 172.21.15.46:0/1401397134 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ac005260 con 0x7f62d40ad450 2024-03-29T08:18:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.612+0000 7f62cb400640 1 -- 172.21.15.46:0/1401397134 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f62c4007af0 con 0x7f62d40ad450 2024-03-29T08:18:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.612+0000 7f62cb400640 1 --2- 172.21.15.46:0/1401397134 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f62b40666f0 0x7f62b4068bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.612+0000 7f62cb400640 1 -- 172.21.15.46:0/1401397134 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62c4083dd0 con 0x7f62d40ad450 2024-03-29T08:18:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.616+0000 7f62d2a00640 1 --2- 172.21.15.46:0/1401397134 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f62b40666f0 0x7f62b4068bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.616+0000 7f62cb400640 1 -- 172.21.15.46:0/1401397134 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f62c40511e0 con 0x7f62d40ad450 2024-03-29T08:18:19.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.620+0000 7f62d2a00640 1 --2- 172.21.15.46:0/1401397134 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f62b40666f0 0x7f62b4068bb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f62c00ae000 tx=0x7f62c00a6040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.620+0000 7f44c0c00640 1 Processor -- start 2024-03-29T08:18:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.620+0000 7f44c0c00640 1 -- start start 2024-03-29T08:18:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.624+0000 7f44c0c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f44bc0709b0 0x7f44bc0b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.624+0000 7f44c0c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44bc0b39f0 con 0x7f44bc0add10 2024-03-29T08:18:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.624+0000 7f44c0c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44bc0b3710 con 0x7f44bc165590 2024-03-29T08:18:19.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.624+0000 7f44c0c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44bc164ab0 con 0x7f44bc0709b0 2024-03-29T08:18:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.624+0000 7f44b9e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f44bc0add10 0x7f44bc0702a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60934/0 (socket says 172.21.15.46:60934) 2024-03-29T08:18:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.624+0000 7f44b9e00640 1 -- 172.21.15.46:0/2945098835 learned_addr learned my addr 172.21.15.46:0/2945098835 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.624+0000 7f44b9400640 1 --2- 172.21.15.46:0/2945098835 >> v2:172.21.15.46:3300/0 conn(0x7f44bc0709b0 0x7f44bc0b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.624+0000 7f44b8a00640 1 -- 172.21.15.46:0/2945098835 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 274018734 0 0) 0x7f44bc0b3710 con 0x7f44bc165590 2024-03-29T08:18:19.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.624+0000 7f44b8a00640 1 -- 172.21.15.46:0/2945098835 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44a0003530 con 0x7f44bc165590 2024-03-29T08:18:19.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b8a00640 1 -- 172.21.15.46:0/2945098835 <== mon.1 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1379999395 0 0) 0x7f44a0003530 con 0x7f44bc165590 2024-03-29T08:18:19.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b8a00640 1 -- 172.21.15.46:0/2945098835 >> v2:172.21.15.46:3300/0 conn(0x7f44bc0709b0 msgr2=0x7f44bc0b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b8a00640 1 --2- 172.21.15.46:0/2945098835 >> v2:172.21.15.46:3300/0 conn(0x7f44bc0709b0 0x7f44bc0b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b8a00640 1 -- 172.21.15.46:0/2945098835 >> v1:172.21.15.46:6789/0 conn(0x7f44bc0add10 legacy=0x7f44bc0702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b8a00640 1 -- 172.21.15.46:0/2945098835 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc164d90 con 0x7f44bc165590 2024-03-29T08:18:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b8a00640 1 -- 172.21.15.46:0/2945098835 <== mon.1 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f44b4001a00 con 0x7f44bc165590 2024-03-29T08:18:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b8a00640 1 -- 172.21.15.46:0/2945098835 >> v1:172.21.15.73:6789/0 conn(0x7f44bc165590 legacy=0x7f44bc0b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b8a00640 1 --2- 172.21.15.46:0/2945098835 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44a00039d0 0x7f44a0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b8a00640 1 --2- 172.21.15.46:0/2945098835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a00047d0 0x7f44a0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b8a00640 1 --2- 172.21.15.46:0/2945098835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44a0005190 0x7f44a000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b8a00640 1 -- 172.21.15.46:0/2945098835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc164ab0 con 0x7f44a0005190 2024-03-29T08:18:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b8a00640 1 -- 172.21.15.46:0/2945098835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a000c090 con 0x7f44a00047d0 2024-03-29T08:18:19.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b8a00640 1 -- 172.21.15.46:0/2945098835 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a000c250 con 0x7f44a00039d0 2024-03-29T08:18:19.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b9e00640 1 --2- 172.21.15.46:0/2945098835 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44a00039d0 0x7f44a0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b9400640 1 --2- 172.21.15.46:0/2945098835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a00047d0 0x7f44a0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b9400640 1 -- 172.21.15.46:0/2945098835 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44a00039d0 msgr2=0x7f44a0003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b9400640 1 --2- 172.21.15.46:0/2945098835 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44a00039d0 0x7f44a0003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b9400640 1 -- 172.21.15.46:0/2945098835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44a0005190 msgr2=0x7f44a000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:19.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b9400640 1 --2- 172.21.15.46:0/2945098835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44a0005190 0x7f44a000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b9400640 1 -- 172.21.15.46:0/2945098835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a0003530 con 0x7f44a00047d0 2024-03-29T08:18:19.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44ba800640 1 -- 172.21.15.46:0/2945098835 reap_dead start 2024-03-29T08:18:19.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.628+0000 7f44b9e00640 1 --2- 172.21.15.46:0/2945098835 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44a00039d0 0x7f44a0003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:19.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.632+0000 7f44b9400640 1 --2- 172.21.15.46:0/2945098835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a00047d0 0x7f44a0004c50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009ae0 tx=0x7f44a801ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6696f016e47e9b7a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.636+0000 7f44b8a00640 1 -- 172.21.15.46:0/2945098835 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a80a5660 con 0x7f44a00047d0 2024-03-29T08:18:19.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.636+0000 7f44b8a00640 1 -- 172.21.15.46:0/2945098835 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a80a57c0 con 0x7f44a00047d0 2024-03-29T08:18:19.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.636+0000 7f44c0c00640 1 -- 172.21.15.46:0/2945098835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a00047d0 msgr2=0x7f44a0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.636+0000 7f44c0c00640 1 --2- 172.21.15.46:0/2945098835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a00047d0 0x7f44a0004c50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009ae0 tx=0x7f44a801ca70 comp rx=0 tx=0).stop 2024-03-29T08:18:19.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.636+0000 7f44c0c00640 1 -- 172.21.15.46:0/2945098835 shutdown_connections 2024-03-29T08:18:19.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.636+0000 7f44c0c00640 1 --2- 172.21.15.46:0/2945098835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a00047d0 0x7f44a0004c50 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009ae0 tx=0x7f44a801ca70 comp rx=0 tx=0).stop 2024-03-29T08:18:19.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.636+0000 7f44c0c00640 1 -- 172.21.15.46:0/2945098835 >> 172.21.15.46:0/2945098835 conn(0x7f44bc066080 msgr2=0x7f44bc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.636+0000 7f44c0c00640 1 -- 172.21.15.46:0/2945098835 shutdown_connections 2024-03-29T08:18:19.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.636+0000 7f44c0c00640 1 -- 172.21.15.46:0/2945098835 wait complete. 2024-03-29T08:18:19.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.640+0000 7f44c0c00640 1 Processor -- start 2024-03-29T08:18:19.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.640+0000 7f44c0c00640 1 -- start start 2024-03-29T08:18:19.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.640+0000 7f44c0c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44bc14f760 0x7f44bc0a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44c0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44bc15cc00 0x7f44bc0a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44c0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44bc15e100 0x7f44bc15e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44b9e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44bc14f760 0x7f44bc0a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44b9e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44bc14f760 0x7f44bc0a8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:57728/0 (socket says 172.21.15.46:57728) 2024-03-29T08:18:19.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44b9e00640 1 -- 172.21.15.46:0/1299617068 learned_addr learned my addr 172.21.15.46:0/1299617068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44b9400640 1 --2- 172.21.15.46:0/1299617068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44bc15cc00 0x7f44bc0a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44c0c00640 1 -- 172.21.15.46:0/1299617068 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc15ead0 con 0x7f44bc15e100 2024-03-29T08:18:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44c0c00640 1 -- 172.21.15.46:0/1299617068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc0a7df0 con 0x7f44bc15cc00 2024-03-29T08:18:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44c0c00640 1 -- 172.21.15.46:0/1299617068 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc0a80d0 con 0x7f44bc14f760 2024-03-29T08:18:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44b9e00640 1 -- 172.21.15.46:0/1299617068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44bc15cc00 msgr2=0x7f44bc0a74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44b9e00640 1 --2- 172.21.15.46:0/1299617068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44bc15cc00 0x7f44bc0a74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44b9e00640 1 -- 172.21.15.46:0/1299617068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44bc15e100 msgr2=0x7f44bc15e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44b9e00640 1 --2- 172.21.15.46:0/1299617068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44bc15e100 0x7f44bc15e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44b9e00640 1 -- 172.21.15.46:0/1299617068 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc0a83b0 con 0x7f44bc14f760 2024-03-29T08:18:19.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44b9e00640 1 --2- 172.21.15.46:0/1299617068 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44bc14f760 0x7f44bc0a8e90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f44ac001870 tx=0x7f44ac001d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44b2000640 1 -- 172.21.15.46:0/1299617068 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac001310 con 0x7f44bc14f760 2024-03-29T08:18:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44b2000640 1 -- 172.21.15.46:0/1299617068 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ac007740 con 0x7f44bc14f760 2024-03-29T08:18:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44b2000640 1 -- 172.21.15.46:0/1299617068 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac00fbc0 con 0x7f44bc14f760 2024-03-29T08:18:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44c0c00640 1 -- 172.21.15.46:0/1299617068 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44bc0a8690 con 0x7f44bc14f760 2024-03-29T08:18:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.644+0000 7f44c0c00640 1 -- 172.21.15.46:0/1299617068 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44bc14fb40 con 0x7f44bc14f760 2024-03-29T08:18:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.648+0000 7f44b2000640 1 -- 172.21.15.46:0/1299617068 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f44ac00fd20 con 0x7f44bc14f760 2024-03-29T08:18:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.648+0000 7f44c0c00640 1 -- 172.21.15.46:0/1299617068 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44bc070280 con 0x7f44bc14f760 2024-03-29T08:18:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.648+0000 7f44b2000640 1 --2- 172.21.15.46:0/1299617068 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f449c066680 0x7f449c068b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.648+0000 7f44b2000640 1 -- 172.21.15.46:0/1299617068 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44ac083fd0 con 0x7f44bc14f760 2024-03-29T08:18:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.652+0000 7f44b9400640 1 --2- 172.21.15.46:0/1299617068 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f449c066680 0x7f449c068b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:19.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.652+0000 7f44b9400640 1 --2- 172.21.15.46:0/1299617068 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f449c066680 0x7f449c068b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f44a80af000 tx=0x7f44a8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:19.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.660+0000 7f44b2000640 1 -- 172.21.15.46:0/1299617068 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f44bc070280 con 0x7f44bc14f760 2024-03-29T08:18:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.912+0000 7fc3cea00640 1 -- 172.21.15.46:0/1216002520 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc3d404d370 con 0x7fc3d404fa50 2024-03-29T08:18:19.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.912+0000 7f7eff600640 1 -- 172.21.15.46:0/3666302307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7ef0052800 con 0x7f7ef0005ec0 2024-03-29T08:18:19.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.912+0000 7fc3d9600640 1 -- 172.21.15.46:0/1216002520 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc3d0017020 con 0x7fc3d404fa50 2024-03-29T08:18:19.916 INFO:teuthology.orchestra.run.smithi046.stdout:64424509446 2024-03-29T08:18:19.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.916+0000 7f7ef4c00640 1 -- 172.21.15.46:0/3666302307 <== mon.1 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7ee4054710 con 0x7f7ef0005ec0 2024-03-29T08:18:19.917 INFO:teuthology.orchestra.run.smithi046.stdout:64424509447 2024-03-29T08:18:19.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.916+0000 7fc3cea00640 1 -- 172.21.15.46:0/1216002520 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc3bc0666f0 msgr2=0x7fc3bc068bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.916+0000 7fc3cea00640 1 --2- 172.21.15.46:0/1216002520 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc3bc0666f0 0x7fc3bc068bb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80b2000 tx=0x7fc3c800a040 comp rx=0 tx=0).stop 2024-03-29T08:18:19.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.916+0000 7fc3cea00640 1 -- 172.21.15.46:0/1216002520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d404fa50 msgr2=0x7fc3d40e0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.916+0000 7fc3cea00640 1 --2- 172.21.15.46:0/1216002520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d404fa50 0x7fc3d40e0e90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000c490 tx=0x7fc3d0001e80 comp rx=0 tx=0).stop 2024-03-29T08:18:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.916+0000 7fc3cea00640 1 -- 172.21.15.46:0/1216002520 shutdown_connections 2024-03-29T08:18:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.916+0000 7fc3cea00640 1 --2- 172.21.15.46:0/1216002520 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc3bc0666f0 0x7fc3bc068bb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.916+0000 7fc3cea00640 1 --2- 172.21.15.46:0/1216002520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3d40df260 0x7fc3d40d44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.916+0000 7fc3cea00640 1 --2- 172.21.15.46:0/1216002520 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc3d4053060 0x7fc3d404d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.916+0000 7fc3cea00640 1 --2- 172.21.15.46:0/1216002520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3d404fa50 0x7fc3d40e0e90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.916+0000 7fc3cea00640 1 -- 172.21.15.46:0/1216002520 >> 172.21.15.46:0/1216002520 conn(0x7fc3d4009f90 msgr2=0x7fc3d40dc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.920+0000 7fe0d3200640 1 -- 172.21.15.46:0/1986345609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fe0d41580f0 con 0x7fe0d40c17c0 2024-03-29T08:18:19.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.920+0000 7f7eff600640 1 -- 172.21.15.46:0/3666302307 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ed80666e0 msgr2=0x7f7ed8068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.920+0000 7f7eff600640 1 --2- 172.21.15.46:0/3666302307 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ed80666e0 0x7f7ed8068ba0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7eec0b2000 tx=0x7f7eec0aa070 comp rx=0 tx=0).stop 2024-03-29T08:18:19.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.920+0000 7fc3cea00640 1 -- 172.21.15.46:0/1216002520 shutdown_connections 2024-03-29T08:18:19.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.920+0000 7fc3cea00640 1 -- 172.21.15.46:0/1216002520 wait complete. 2024-03-29T08:18:19.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.920+0000 7f7eff600640 1 -- 172.21.15.46:0/3666302307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0005ec0 msgr2=0x7f7ef017c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.920+0000 7fe0ca000640 1 -- 172.21.15.46:0/1986345609 <== mon.1 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe0c0054c90 con 0x7fe0d40c17c0 2024-03-29T08:18:19.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.920+0000 7f7eff600640 1 --2- 172.21.15.46:0/3666302307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0005ec0 0x7f7ef017c870 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7ee400c490 tx=0x7f7ee400c950 comp rx=0 tx=0).stop 2024-03-29T08:18:19.925 INFO:teuthology.orchestra.run.smithi046.stdout:64424509444 2024-03-29T08:18:19.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.920+0000 7f7eff600640 1 -- 172.21.15.46:0/3666302307 shutdown_connections 2024-03-29T08:18:19.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.920+0000 7f7eff600640 1 --2- 172.21.15.46:0/3666302307 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ed80666e0 0x7f7ed8068ba0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.920+0000 7f7eff600640 1 --2- 172.21.15.46:0/3666302307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ef0047bf0 0x7f7ef00e4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.920+0000 7f7eff600640 1 --2- 172.21.15.46:0/3666302307 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7ef0033e10 0x7f7ef00e8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.920+0000 7f7eff600640 1 --2- 172.21.15.46:0/3666302307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0005ec0 0x7f7ef017c870 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.920+0000 7f7eff600640 1 -- 172.21.15.46:0/3666302307 >> 172.21.15.46:0/3666302307 conn(0x7f7ef0009f90 msgr2=0x7f7ef0039b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.920+0000 7f7eff600640 1 -- 172.21.15.46:0/3666302307 shutdown_connections 2024-03-29T08:18:19.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.920+0000 7f7eff600640 1 -- 172.21.15.46:0/3666302307 wait complete. 2024-03-29T08:18:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.924+0000 7f4583800640 1 -- 172.21.15.46:0/1252236139 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f457c071a90 con 0x7f457c141010 2024-03-29T08:18:19.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.924+0000 7f9420c00640 1 -- 172.21.15.46:0/677099644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f942c154040 con 0x7f942c0caa30 2024-03-29T08:18:19.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.924+0000 7fe0d3200640 1 -- 172.21.15.46:0/1986345609 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe0b4066760 msgr2=0x7fe0b4068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.924+0000 7fe0d3200640 1 --2- 172.21.15.46:0/1986345609 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe0b4066760 0x7fe0b4068c20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe0d415a690 tx=0x7fe0c4008040 comp rx=0 tx=0).stop 2024-03-29T08:18:19.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.924+0000 7fe0d3200640 1 -- 172.21.15.46:0/1986345609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0d40c17c0 msgr2=0x7fe0d4159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.924+0000 7fe0d3200640 1 --2- 172.21.15.46:0/1986345609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0d40c17c0 0x7fe0d4159d60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000b3f0 tx=0x7fe0c000b8b0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.924+0000 7fe0d3200640 1 -- 172.21.15.46:0/1986345609 shutdown_connections 2024-03-29T08:18:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.924+0000 7fe0d3200640 1 --2- 172.21.15.46:0/1986345609 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe0b4066760 0x7fe0b4068c20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.924+0000 7fe0d3200640 1 --2- 172.21.15.46:0/1986345609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0d40c17c0 0x7fe0d4159d60 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe0c000b3f0 tx=0x7fe0c000b8b0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.924+0000 7fe0d3200640 1 --2- 172.21.15.46:0/1986345609 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0d4160e00 0x7fe0d415bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.924+0000 7fe0d3200640 1 --2- 172.21.15.46:0/1986345609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0d406d9f0 0x7fe0d4163ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.924+0000 7fe0d3200640 1 -- 172.21.15.46:0/1986345609 >> 172.21.15.46:0/1986345609 conn(0x7fe0d40b0030 msgr2=0x7fe0d415d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.924+0000 7f4578c00640 1 -- 172.21.15.46:0/1252236139 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f45680a8090 con 0x7f457c141010 2024-03-29T08:18:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.924+0000 7f9423400640 1 -- 172.21.15.46:0/677099644 <== mon.1 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f941c050780 con 0x7f942c0caa30 2024-03-29T08:18:19.932 INFO:teuthology.orchestra.run.smithi046.stdout:64424509445 2024-03-29T08:18:19.932 INFO:teuthology.orchestra.run.smithi046.stdout:64424509446 2024-03-29T08:18:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.924+0000 7fe0d3200640 1 -- 172.21.15.46:0/1986345609 shutdown_connections 2024-03-29T08:18:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7fe0d3200640 1 -- 172.21.15.46:0/1986345609 wait complete. 2024-03-29T08:18:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f456e000640 1 -- 172.21.15.46:0/1252236139 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f455c066790 msgr2=0x7f455c068c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f456e000640 1 --2- 172.21.15.46:0/1252236139 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f455c066790 0x7f455c068c50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4570009a50 tx=0x7f4570009210 comp rx=0 tx=0).stop 2024-03-29T08:18:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f9420c00640 1 -- 172.21.15.46:0/677099644 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f940c0666e0 msgr2=0x7f940c068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f9420c00640 1 --2- 172.21.15.46:0/677099644 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f940c0666e0 0x7f940c068ba0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f94240af000 tx=0x7f94240a7070 comp rx=0 tx=0).stop 2024-03-29T08:18:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f9420c00640 1 -- 172.21.15.46:0/677099644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0caa30 msgr2=0x7f942c15ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f9420c00640 1 --2- 172.21.15.46:0/677099644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0caa30 0x7f942c15ed00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f941c00e810 tx=0x7f941c00ecd0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f456e000640 1 -- 172.21.15.46:0/1252236139 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f457c141010 msgr2=0x7f457c1415b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f456e000640 1 --2- 172.21.15.46:0/1252236139 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f457c141010 0x7f457c1415b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f45680ac000 tx=0x7f4568019280 comp rx=0 tx=0).stop 2024-03-29T08:18:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f9420c00640 1 -- 172.21.15.46:0/677099644 shutdown_connections 2024-03-29T08:18:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f9420c00640 1 --2- 172.21.15.46:0/677099644 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f940c0666e0 0x7f940c068ba0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f456e000640 1 -- 172.21.15.46:0/1252236139 shutdown_connections 2024-03-29T08:18:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f456e000640 1 --2- 172.21.15.46:0/1252236139 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f455c066790 0x7f455c068c50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f456e000640 1 --2- 172.21.15.46:0/1252236139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f457c15e4f0 0x7f457c13cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f456e000640 1 --2- 172.21.15.46:0/1252236139 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f457c141010 0x7f457c1415b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f456e000640 1 --2- 172.21.15.46:0/1252236139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457c0702a0 0x7f457c159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f456e000640 1 -- 172.21.15.46:0/1252236139 >> 172.21.15.46:0/1252236139 conn(0x7f457c066080 msgr2=0x7f457c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f9420c00640 1 --2- 172.21.15.46:0/677099644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f942c0caa30 0x7f942c15ed00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f456e000640 1 -- 172.21.15.46:0/1252236139 shutdown_connections 2024-03-29T08:18:19.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f456e000640 1 -- 172.21.15.46:0/1252236139 wait complete. 2024-03-29T08:18:19.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f9420c00640 1 --2- 172.21.15.46:0/677099644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f942c160d10 0x7f942c201b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f9420c00640 1 --2- 172.21.15.46:0/677099644 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f942c0b0ba0 0x7f942c1616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f9420c00640 1 -- 172.21.15.46:0/677099644 >> 172.21.15.46:0/677099644 conn(0x7f942c066080 msgr2=0x7f942c0cdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f9420c00640 1 -- 172.21.15.46:0/677099644 shutdown_connections 2024-03-29T08:18:19.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.928+0000 7f9420c00640 1 -- 172.21.15.46:0/677099644 wait complete. 2024-03-29T08:18:19.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.948+0000 7fdf13400640 1 -- 172.21.15.46:0/339091899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fdee4005510 con 0x7fdf0c142ab0 2024-03-29T08:18:19.953 INFO:teuthology.orchestra.run.smithi046.stdout:64424509445 2024-03-29T08:18:19.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.948+0000 7fdf08c00640 1 -- 172.21.15.46:0/339091899 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdf040ef1e0 con 0x7fdf0c142ab0 2024-03-29T08:18:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.956+0000 7fdf13400640 1 -- 172.21.15.46:0/339091899 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeec066760 msgr2=0x7fdeec068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.956+0000 7fdf13400640 1 --2- 172.21.15.46:0/339091899 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeec066760 0x7fdeec068c20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009af0 tx=0x7fdefc008040 comp rx=0 tx=0).stop 2024-03-29T08:18:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.956+0000 7fdf13400640 1 -- 172.21.15.46:0/339091899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf0c142ab0 msgr2=0x7fdf0c160b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.956+0000 7fdf13400640 1 --2- 172.21.15.46:0/339091899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf0c142ab0 0x7fdf0c160b50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdf040ab000 tx=0x7fdf040197d0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.956+0000 7fdf13400640 1 -- 172.21.15.46:0/339091899 shutdown_connections 2024-03-29T08:18:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.956+0000 7fdf13400640 1 --2- 172.21.15.46:0/339091899 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeec066760 0x7fdeec068c20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.956+0000 7fdf13400640 1 --2- 172.21.15.46:0/339091899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf0c15d940 0x7fdf0c161090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.956+0000 7fdf13400640 1 --2- 172.21.15.46:0/339091899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf0c142ab0 0x7fdf0c160b50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.956+0000 7fdf13400640 1 --2- 172.21.15.46:0/339091899 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf0c06d9e0 0x7fdf0c158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.956+0000 7fdf13400640 1 -- 172.21.15.46:0/339091899 >> 172.21.15.46:0/339091899 conn(0x7fdf0c066080 msgr2=0x7fdf0c159d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.956+0000 7fdf13400640 1 -- 172.21.15.46:0/339091899 shutdown_connections 2024-03-29T08:18:19.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.956+0000 7fdf13400640 1 -- 172.21.15.46:0/339091899 wait complete. 2024-03-29T08:18:19.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.1 2024-03-29T08:18:19.965 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.3 2024-03-29T08:18:19.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.2 2024-03-29T08:18:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.968+0000 7f62da800640 1 -- 172.21.15.46:0/1401397134 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f62ac005510 con 0x7f62d40ad450 2024-03-29T08:18:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.968+0000 7f62cb400640 1 -- 172.21.15.46:0/1401397134 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f62c4054800 con 0x7f62d40ad450 2024-03-29T08:18:19.969 INFO:teuthology.orchestra.run.smithi046.stdout:64424509445 2024-03-29T08:18:19.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.4 2024-03-29T08:18:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.972+0000 7f62da800640 1 -- 172.21.15.46:0/1401397134 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f62b40666f0 msgr2=0x7f62b4068bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.972+0000 7f62da800640 1 --2- 172.21.15.46:0/1401397134 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f62b40666f0 0x7f62b4068bb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f62c00ae000 tx=0x7f62c00a6040 comp rx=0 tx=0).stop 2024-03-29T08:18:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.972+0000 7f62da800640 1 -- 172.21.15.46:0/1401397134 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f62d40ad450 msgr2=0x7f62d4142db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.972+0000 7f62da800640 1 --2- 172.21.15.46:0/1401397134 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f62d40ad450 0x7f62d4142db0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f62c400c7f0 tx=0x7f62c400ccb0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.972+0000 7f62da800640 1 -- 172.21.15.46:0/1401397134 shutdown_connections 2024-03-29T08:18:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.972+0000 7f62da800640 1 --2- 172.21.15.46:0/1401397134 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f62b40666f0 0x7f62b4068bb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.972+0000 7f62da800640 1 --2- 172.21.15.46:0/1401397134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d40cb120 0x7f62d41432f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.972+0000 7f62da800640 1 --2- 172.21.15.46:0/1401397134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f62d415aab0 0x7f62d415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.972+0000 7f62da800640 1 --2- 172.21.15.46:0/1401397134 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f62d40ad450 0x7f62d4142db0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.972+0000 7f62da800640 1 -- 172.21.15.46:0/1401397134 >> 172.21.15.46:0/1401397134 conn(0x7f62d4099a10 msgr2=0x7f62d40d0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.972+0000 7f62da800640 1 -- 172.21.15.46:0/1401397134 shutdown_connections 2024-03-29T08:18:19.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.972+0000 7f62da800640 1 -- 172.21.15.46:0/1401397134 wait complete. 2024-03-29T08:18:19.974 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.0 2024-03-29T08:18:19.984 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.7 2024-03-29T08:18:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.984+0000 7f44c0c00640 1 -- 172.21.15.46:0/1299617068 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f44bc070280 con 0x7f44bc14f760 2024-03-29T08:18:19.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.984+0000 7f44b2000640 1 -- 172.21.15.46:0/1299617068 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f44bc070280 con 0x7f44bc14f760 2024-03-29T08:18:19.986 INFO:teuthology.orchestra.run.smithi046.stdout:60129542150 2024-03-29T08:18:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.988+0000 7f44c0c00640 1 -- 172.21.15.46:0/1299617068 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f449c066680 msgr2=0x7f449c068b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.988+0000 7f44c0c00640 1 --2- 172.21.15.46:0/1299617068 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f449c066680 0x7f449c068b40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f44a80af000 tx=0x7f44a8008040 comp rx=0 tx=0).stop 2024-03-29T08:18:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.988+0000 7f44c0c00640 1 -- 172.21.15.46:0/1299617068 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44bc14f760 msgr2=0x7f44bc0a8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.988+0000 7f44c0c00640 1 --2- 172.21.15.46:0/1299617068 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44bc14f760 0x7f44bc0a8e90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f44ac001870 tx=0x7f44ac001d30 comp rx=0 tx=0).stop 2024-03-29T08:18:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.988+0000 7f44c0c00640 1 -- 172.21.15.46:0/1299617068 shutdown_connections 2024-03-29T08:18:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.988+0000 7f44c0c00640 1 --2- 172.21.15.46:0/1299617068 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f449c066680 0x7f449c068b40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.988+0000 7f44c0c00640 1 --2- 172.21.15.46:0/1299617068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44bc15cc00 0x7f44bc0a74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.988+0000 7f44c0c00640 1 --2- 172.21.15.46:0/1299617068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44bc15e100 0x7f44bc15e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.988+0000 7f44c0c00640 1 --2- 172.21.15.46:0/1299617068 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44bc14f760 0x7f44bc0a8e90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:19.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.988+0000 7f44c0c00640 1 -- 172.21.15.46:0/1299617068 >> 172.21.15.46:0/1299617068 conn(0x7f44bc066080 msgr2=0x7f44bc1637c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:19.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.988+0000 7f44c0c00640 1 -- 172.21.15.46:0/1299617068 shutdown_connections 2024-03-29T08:18:19.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:19.988+0000 7f44c0c00640 1 -- 172.21.15.46:0/1299617068 wait complete. 2024-03-29T08:18:19.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.6 2024-03-29T08:18:20.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542150 for osd.5 2024-03-29T08:18:20.964 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-29T08:18:20.965 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-29T08:18:20.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-29T08:18:20.972 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-29T08:18:20.975 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-29T08:18:20.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-29T08:18:20.995 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-29T08:18:21.009 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-29T08:18:21.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.188+0000 7f7a0be00640 1 Processor -- start 2024-03-29T08:18:21.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.188+0000 7f31aaa00640 1 Processor -- start 2024-03-29T08:18:21.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.188+0000 7f31aaa00640 1 -- start start 2024-03-29T08:18:21.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.188+0000 7efe2fe00640 1 Processor -- start 2024-03-29T08:18:21.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.188+0000 7efe2fe00640 1 -- start start 2024-03-29T08:18:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.188+0000 7f7a0be00640 1 -- start start 2024-03-29T08:18:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.188+0000 7f7a0be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7a0c0957d0 0x7f7a0c097550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.188+0000 7f7a0be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7a0c0ada90 0x7f7a0c097a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.188+0000 7f7a0be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7a0c0cbe80 0x7f7a0c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.188+0000 7f7a0be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a0c0cc850 con 0x7f7a0c0cbe80 2024-03-29T08:18:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.188+0000 7f7a0be00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a0c0ab140 con 0x7f7a0c0ada90 2024-03-29T08:18:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.188+0000 7f7a0be00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c0ab420 con 0x7f7a0c0957d0 2024-03-29T08:18:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f31aaa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31a40cb4a0 0x7f31a4167850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7efe2fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efe300cb4a0 0x7efe30167850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7efe2fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe300cb8d0 con 0x7efe300cc590 2024-03-29T08:18:21.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7efe2fe00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe30144750 con 0x7efe3015c9c0 2024-03-29T08:18:21.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7efe2fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe300cbb00 con 0x7efe300cb4a0 2024-03-29T08:18:21.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f31a9600640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f31a415c9c0 0x7f31a40cbe80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:43156/0 (socket says 172.21.15.46:43156) 2024-03-29T08:18:21.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f31aaa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a4144750 con 0x7f31a40cc590 2024-03-29T08:18:21.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f31aaa00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a40cbb00 con 0x7f31a415c9c0 2024-03-29T08:18:21.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f31aaa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a40cb910 con 0x7f31a40cb4a0 2024-03-29T08:18:21.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f31a8c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f31a40cc590 0x7f31a40cad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60948/0 (socket says 172.21.15.46:60948) 2024-03-29T08:18:21.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f31a8c00640 1 -- 172.21.15.46:0/394312980 learned_addr learned my addr 172.21.15.46:0/394312980 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f31aa000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31a40cb4a0 0x7f31a4167850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7efe2ea00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7efe3015c9c0 0x7efe300cbe80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:43162/0 (socket says 172.21.15.46:43162) 2024-03-29T08:18:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7efe2ea00640 1 -- 172.21.15.46:0/270547712 learned_addr learned my addr 172.21.15.46:0/270547712 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7efe2f400640 1 --2- 172.21.15.46:0/270547712 >> v2:172.21.15.46:3300/0 conn(0x7efe300cb4a0 0x7efe30167850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f7a0b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7a0c0cbe80 0x7f7a0c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f31aa000640 1 -- 172.21.15.46:0/394312980 >> v1:172.21.15.73:6789/0 conn(0x7f31a415c9c0 legacy=0x7f31a40cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f31aa000640 1 -- 172.21.15.46:0/394312980 >> v1:172.21.15.46:6789/0 conn(0x7f31a40cc590 legacy=0x7f31a40cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f7a0b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7a0c0cbe80 0x7f7a0c0cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60020/0 (socket says 172.21.15.46:60020) 2024-03-29T08:18:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f7a0b400640 1 -- 172.21.15.46:0/1630825867 learned_addr learned my addr 172.21.15.46:0/1630825867 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:21.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7efe2f400640 1 -- 172.21.15.46:0/270547712 >> v1:172.21.15.73:6789/0 conn(0x7efe3015c9c0 legacy=0x7efe300cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7efe2f400640 1 -- 172.21.15.46:0/270547712 >> v1:172.21.15.46:6789/0 conn(0x7efe300cc590 legacy=0x7efe300cad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:21.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f31aa000640 1 -- 172.21.15.46:0/394312980 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a40c8900 con 0x7f31a40cb4a0 2024-03-29T08:18:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f7a0a000640 1 --2- 172.21.15.46:0/1630825867 >> v2:172.21.15.73:3300/0 conn(0x7f7a0c0ada90 0x7f7a0c097a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f7a0aa00640 1 --2- 172.21.15.46:0/1630825867 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7a0c0957d0 0x7f7a0c097550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f7a0aa00640 1 -- 172.21.15.46:0/1630825867 >> v2:172.21.15.73:3300/0 conn(0x7f7a0c0ada90 msgr2=0x7f7a0c097a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f7a0aa00640 1 --2- 172.21.15.46:0/1630825867 >> v2:172.21.15.73:3300/0 conn(0x7f7a0c0ada90 0x7f7a0c097a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f7a0aa00640 1 -- 172.21.15.46:0/1630825867 >> v2:172.21.15.46:3300/0 conn(0x7f7a0c0cbe80 msgr2=0x7f7a0c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f7a0aa00640 1 --2- 172.21.15.46:0/1630825867 >> v2:172.21.15.46:3300/0 conn(0x7f7a0c0cbe80 0x7f7a0c0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f7a0aa00640 1 -- 172.21.15.46:0/1630825867 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c0ab700 con 0x7f7a0c0957d0 2024-03-29T08:18:21.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7f7a0a000640 1 --2- 172.21.15.46:0/1630825867 >> v2:172.21.15.73:3300/0 conn(0x7f7a0c0ada90 0x7f7a0c097a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.192+0000 7efe2f400640 1 -- 172.21.15.46:0/270547712 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe300c8900 con 0x7efe300cb4a0 2024-03-29T08:18:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.196+0000 7f31aa000640 1 --2- 172.21.15.46:0/394312980 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31a40cb4a0 0x7f31a4167850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f319c009dc0 tx=0x7f319c01c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b70f5760c428ae01 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.196+0000 7efe2f400640 1 --2- 172.21.15.46:0/270547712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe300cb4a0 0x7efe30167850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efe1c0097c0 tx=0x7efe1c01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b95521462422d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7efe2d600640 1 -- 172.21.15.46:0/270547712 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c0a8990 con 0x7efe300cb4a0 2024-03-29T08:18:21.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f31a3e00640 1 -- 172.21.15.46:0/394312980 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c0a5740 con 0x7f31a40cb4a0 2024-03-29T08:18:21.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f31a3e00640 1 -- 172.21.15.46:0/394312980 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319c0a58a0 con 0x7f31a40cb4a0 2024-03-29T08:18:21.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f31a3e00640 1 -- 172.21.15.46:0/394312980 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c0a5c30 con 0x7f31a40cb4a0 2024-03-29T08:18:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7efe2d600640 1 -- 172.21.15.46:0/270547712 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1c0a8af0 con 0x7efe300cb4a0 2024-03-29T08:18:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7efe2d600640 1 -- 172.21.15.46:0/270547712 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c0a8e80 con 0x7efe300cb4a0 2024-03-29T08:18:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f7a0aa00640 1 --2- 172.21.15.46:0/1630825867 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7a0c0957d0 0x7f7a0c097550 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0724f0 tx=0x7f79fc01d480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a07ac066b3f2c2ee server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f31aaa00640 1 -- 172.21.15.46:0/394312980 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31a40cb4a0 msgr2=0x7f31a4167850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f31aaa00640 1 --2- 172.21.15.46:0/394312980 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31a40cb4a0 0x7f31a4167850 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f319c009dc0 tx=0x7f319c01c760 comp rx=0 tx=0).stop 2024-03-29T08:18:21.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f7a09600640 1 -- 172.21.15.46:0/1630825867 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc008be0 con 0x7f7a0c0957d0 2024-03-29T08:18:21.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f7a09600640 1 -- 172.21.15.46:0/1630825867 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79fc008d40 con 0x7f7a0c0957d0 2024-03-29T08:18:21.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f7a09600640 1 -- 172.21.15.46:0/1630825867 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79fc00a2f0 con 0x7f7a0c0957d0 2024-03-29T08:18:21.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f31aaa00640 1 -- 172.21.15.46:0/394312980 shutdown_connections 2024-03-29T08:18:21.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f31aaa00640 1 --2- 172.21.15.46:0/394312980 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31a40cb4a0 0x7f31a4167850 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f31aaa00640 1 -- 172.21.15.46:0/394312980 >> 172.21.15.46:0/394312980 conn(0x7f31a40afe60 msgr2=0x7f31a40b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:21.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f31aaa00640 1 -- 172.21.15.46:0/394312980 shutdown_connections 2024-03-29T08:18:21.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f31aaa00640 1 -- 172.21.15.46:0/394312980 wait complete. 2024-03-29T08:18:21.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7efe2fe00640 1 -- 172.21.15.46:0/270547712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe300cb4a0 msgr2=0x7efe30167850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7efe2fe00640 1 --2- 172.21.15.46:0/270547712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe300cb4a0 0x7efe30167850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efe1c0097c0 tx=0x7efe1c01ec90 comp rx=0 tx=0).stop 2024-03-29T08:18:21.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f7a0be00640 1 -- 172.21.15.46:0/1630825867 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7a0c0957d0 msgr2=0x7f7a0c097550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f7a0be00640 1 --2- 172.21.15.46:0/1630825867 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7a0c0957d0 0x7f7a0c097550 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c0724f0 tx=0x7f79fc01d480 comp rx=0 tx=0).stop 2024-03-29T08:18:21.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f7a0be00640 1 -- 172.21.15.46:0/1630825867 shutdown_connections 2024-03-29T08:18:21.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f7a0be00640 1 --2- 172.21.15.46:0/1630825867 >> v2:172.21.15.46:3300/0 conn(0x7f7a0c0cbe80 0x7f7a0c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f7a0be00640 1 --2- 172.21.15.46:0/1630825867 >> v2:172.21.15.73:3300/0 conn(0x7f7a0c0ada90 0x7f7a0c097a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f7a0be00640 1 --2- 172.21.15.46:0/1630825867 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7a0c0957d0 0x7f7a0c097550 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7f7a0be00640 1 -- 172.21.15.46:0/1630825867 >> 172.21.15.46:0/1630825867 conn(0x7f7a0c0afe60 msgr2=0x7f7a0c0b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:21.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7efe2fe00640 1 -- 172.21.15.46:0/270547712 shutdown_connections 2024-03-29T08:18:21.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.204+0000 7f7a0be00640 1 -- 172.21.15.46:0/1630825867 shutdown_connections 2024-03-29T08:18:21.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7efe2fe00640 1 --2- 172.21.15.46:0/270547712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe300cb4a0 0x7efe30167850 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.200+0000 7efe2fe00640 1 -- 172.21.15.46:0/270547712 >> 172.21.15.46:0/270547712 conn(0x7efe300afe60 msgr2=0x7efe300b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:21.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.204+0000 7efe2fe00640 1 -- 172.21.15.46:0/270547712 shutdown_connections 2024-03-29T08:18:21.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.204+0000 7f7a0be00640 1 -- 172.21.15.46:0/1630825867 wait complete. 2024-03-29T08:18:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.204+0000 7efe2fe00640 1 -- 172.21.15.46:0/270547712 wait complete. 2024-03-29T08:18:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.204+0000 7f31aaa00640 1 Processor -- start 2024-03-29T08:18:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.204+0000 7f31aaa00640 1 -- start start 2024-03-29T08:18:21.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.204+0000 7efe2fe00640 1 Processor -- start 2024-03-29T08:18:21.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.204+0000 7efe2fe00640 1 -- start start 2024-03-29T08:18:21.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31aaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31a40cb4a0 0x7f31a40b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31aaa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31a40cc590 0x7f31a415cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31aaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a415c9c0 0x7f31a40c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31aaa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a415d4c0 con 0x7f31a40cb4a0 2024-03-29T08:18:21.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31aaa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a415ecc0 con 0x7f31a415c9c0 2024-03-29T08:18:21.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31aaa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a4147370 con 0x7f31a40cc590 2024-03-29T08:18:21.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31a9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31a40cb4a0 0x7f31a40b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31a9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31a40cb4a0 0x7f31a40b1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60034/0 (socket says 172.21.15.46:60034) 2024-03-29T08:18:21.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31a9600640 1 -- 172.21.15.46:0/3790205046 learned_addr learned my addr 172.21.15.46:0/3790205046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:21.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31a8c00640 1 --2- 172.21.15.46:0/3790205046 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31a40cc590 0x7f31a415cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31aa000640 1 --2- 172.21.15.46:0/3790205046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a415c9c0 0x7f31a40c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f7a0be00640 1 Processor -- start 2024-03-29T08:18:21.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31a9600640 1 -- 172.21.15.46:0/3790205046 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31a40cc590 msgr2=0x7f31a415cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31a9600640 1 --2- 172.21.15.46:0/3790205046 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31a40cc590 0x7f31a415cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31a9600640 1 -- 172.21.15.46:0/3790205046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a415c9c0 msgr2=0x7f31a40c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31a9600640 1 --2- 172.21.15.46:0/3790205046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a415c9c0 0x7f31a40c44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7efe2fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe300cb4a0 0x7efe30097a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7efe2fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe300cc590 0x7efe300c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7efe2fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3015c9c0 0x7efe300c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31a9600640 1 -- 172.21.15.46:0/3790205046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c009a70 con 0x7f31a40cb4a0 2024-03-29T08:18:21.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7efe2fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe300b3440 con 0x7efe300cc590 2024-03-29T08:18:21.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7efe2fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe300b2b80 con 0x7efe3015c9c0 2024-03-29T08:18:21.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7efe2fe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe300b2df0 con 0x7efe300cb4a0 2024-03-29T08:18:21.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7efe2e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe300cc590 0x7efe300c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7efe2e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe300cc590 0x7efe300c75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60046/0 (socket says 172.21.15.46:60046) 2024-03-29T08:18:21.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7efe2e000640 1 -- 172.21.15.46:0/3546959790 learned_addr learned my addr 172.21.15.46:0/3546959790 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:21.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7efe2ea00640 1 --2- 172.21.15.46:0/3546959790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe300cb4a0 0x7efe30097a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31a9600640 1 --2- 172.21.15.46:0/3790205046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31a40cb4a0 0x7f31a40b1fa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f319000c780 tx=0x7f319000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7efe2f400640 1 --2- 172.21.15.46:0/3546959790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3015c9c0 0x7efe300c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31a1600640 1 -- 172.21.15.46:0/3790205046 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31900072d0 con 0x7f31a40cb4a0 2024-03-29T08:18:21.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31a1600640 1 -- 172.21.15.46:0/3790205046 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3190007430 con 0x7f31a40cb4a0 2024-03-29T08:18:21.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.208+0000 7f31aaa00640 1 -- 172.21.15.46:0/3790205046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a4147610 con 0x7f31a40cb4a0 2024-03-29T08:18:21.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7f7a0be00640 1 -- start start 2024-03-29T08:18:21.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7efe2e000640 1 -- 172.21.15.46:0/3546959790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe300cb4a0 msgr2=0x7efe30097a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7efe2e000640 1 --2- 172.21.15.46:0/3546959790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe300cb4a0 0x7efe30097a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7efe2e000640 1 -- 172.21.15.46:0/3546959790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3015c9c0 msgr2=0x7efe300c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7efe2e000640 1 --2- 172.21.15.46:0/3546959790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3015c9c0 0x7efe300c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7efe2e000640 1 -- 172.21.15.46:0/3546959790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c009420 con 0x7efe300cc590 2024-03-29T08:18:21.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7efe2e000640 1 --2- 172.21.15.46:0/3546959790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe300cc590 0x7efe300c75f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efe2000c500 tx=0x7efe2000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7f31a1600640 1 -- 172.21.15.46:0/3790205046 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31900047f0 con 0x7f31a40cb4a0 2024-03-29T08:18:21.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7efe26a00640 1 -- 172.21.15.46:0/3546959790 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe20005220 con 0x7efe300cc590 2024-03-29T08:18:21.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7efe26a00640 1 -- 172.21.15.46:0/3546959790 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe20005380 con 0x7efe300cc590 2024-03-29T08:18:21.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7efe26a00640 1 -- 172.21.15.46:0/3546959790 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe20007270 con 0x7efe300cc590 2024-03-29T08:18:21.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7f31aaa00640 1 -- 172.21.15.46:0/3790205046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a415e8e0 con 0x7f31a40cb4a0 2024-03-29T08:18:21.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7f7a0be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a0c0957d0 0x7f7a0c0c4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7f7a0be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7a0c0ada90 0x7f7a0c1545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7f7a0be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a0c0cbe80 0x7f7a0c150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7efe2fe00640 1 -- 172.21.15.46:0/3546959790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe300b30d0 con 0x7efe300cc590 2024-03-29T08:18:21.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7efe2fe00640 1 -- 172.21.15.46:0/3546959790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe300ce630 con 0x7efe300cc590 2024-03-29T08:18:21.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7efe2fe00640 1 -- 172.21.15.46:0/3546959790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3004e040 con 0x7efe300cc590 2024-03-29T08:18:21.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7f7a0be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c157ac0 con 0x7f7a0c0957d0 2024-03-29T08:18:21.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7f7a0be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c097af0 con 0x7f7a0c0cbe80 2024-03-29T08:18:21.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.212+0000 7f7a0be00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c097810 con 0x7f7a0c0ada90 2024-03-29T08:18:21.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f7a0b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a0c0cbe80 0x7f7a0c150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f7a0b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a0c0cbe80 0x7f7a0c150510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52020/0 (socket says 172.21.15.46:52020) 2024-03-29T08:18:21.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f7a0b400640 1 -- 172.21.15.46:0/2458648606 learned_addr learned my addr 172.21.15.46:0/2458648606 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:21.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f7a0a000640 1 --2- 172.21.15.46:0/2458648606 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7a0c0ada90 0x7f7a0c1545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f7a0aa00640 1 --2- 172.21.15.46:0/2458648606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a0c0957d0 0x7f7a0c0c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f7a0a000640 1 -- 172.21.15.46:0/2458648606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a0c0cbe80 msgr2=0x7f7a0c150510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f7a0a000640 1 --2- 172.21.15.46:0/2458648606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a0c0cbe80 0x7f7a0c150510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f7a0a000640 1 -- 172.21.15.46:0/2458648606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a0c0957d0 msgr2=0x7f7a0c0c4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f7a0a000640 1 --2- 172.21.15.46:0/2458648606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a0c0957d0 0x7f7a0c0c4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f7a0a000640 1 -- 172.21.15.46:0/2458648606 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f8009420 con 0x7f7a0c0ada90 2024-03-29T08:18:21.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f7a0aa00640 1 --2- 172.21.15.46:0/2458648606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a0c0957d0 0x7f7a0c0c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:21.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7efe26a00640 1 -- 172.21.15.46:0/3546959790 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efe200073d0 con 0x7efe300cc590 2024-03-29T08:18:21.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7efe26a00640 1 --2- 172.21.15.46:0/3546959790 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe10066690 0x7efe10068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7efe26a00640 1 -- 172.21.15.46:0/3546959790 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe20083a70 con 0x7efe300cc590 2024-03-29T08:18:21.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f31a1600640 1 -- 172.21.15.46:0/3790205046 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3190004950 con 0x7f31a40cb4a0 2024-03-29T08:18:21.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f3196a00640 1 -- 172.21.15.46:0/3790205046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a4145a10 con 0x7f31a40cb4a0 2024-03-29T08:18:21.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f31a1600640 1 --2- 172.21.15.46:0/3790205046 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f31840666c0 0x7f3184068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f31a1600640 1 -- 172.21.15.46:0/3790205046 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31900834f0 con 0x7f31a40cb4a0 2024-03-29T08:18:21.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7efe2ea00640 1 --2- 172.21.15.46:0/3546959790 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe10066690 0x7efe10068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f7a0a000640 1 --2- 172.21.15.46:0/2458648606 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7a0c0ada90 0x7f7a0c1545f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f79f80093f0 tx=0x7f79f800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f31a8c00640 1 --2- 172.21.15.46:0/3790205046 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f31840666c0 0x7f3184068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f7a02a00640 1 -- 172.21.15.46:0/2458648606 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8004030 con 0x7f7a0c0ada90 2024-03-29T08:18:21.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f7a02a00640 1 -- 172.21.15.46:0/2458648606 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f8004190 con 0x7f7a0c0ada90 2024-03-29T08:18:21.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.216+0000 7f7a02a00640 1 -- 172.21.15.46:0/2458648606 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f800c940 con 0x7f7a0c0ada90 2024-03-29T08:18:21.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.220+0000 7f7a0be00640 1 -- 172.21.15.46:0/2458648606 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79fc00ae50 con 0x7f7a0c0ada90 2024-03-29T08:18:21.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.220+0000 7f7a0be00640 1 -- 172.21.15.46:0/2458648606 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0c1577e0 con 0x7f7a0c0ada90 2024-03-29T08:18:21.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.220+0000 7f31a8c00640 1 --2- 172.21.15.46:0/3790205046 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f31840666c0 0x7f3184068b80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f31a406c860 tx=0x7f319800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.220+0000 7efe2ea00640 1 --2- 172.21.15.46:0/3546959790 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe10066690 0x7efe10068b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efe28005c10 tx=0x7efe28009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.224+0000 7f7a0be00640 1 -- 172.21.15.46:0/2458648606 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0c1642c0 con 0x7f7a0c0ada90 2024-03-29T08:18:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.224+0000 7f7a02a00640 1 -- 172.21.15.46:0/2458648606 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f79f800caa0 con 0x7f7a0c0ada90 2024-03-29T08:18:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.224+0000 7f7a02a00640 1 --2- 172.21.15.46:0/2458648606 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f79ec066690 0x7f79ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.224+0000 7f7a02a00640 1 -- 172.21.15.46:0/2458648606 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79f8084140 con 0x7f7a0c0ada90 2024-03-29T08:18:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.224+0000 7f31a1600640 1 -- 172.21.15.46:0/3790205046 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3190083e60 con 0x7f31a40cb4a0 2024-03-29T08:18:21.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.224+0000 7f7a0aa00640 1 --2- 172.21.15.46:0/2458648606 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f79ec066690 0x7f79ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.228+0000 7f7a0aa00640 1 --2- 172.21.15.46:0/2458648606 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f79ec066690 0x7f79ec068b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0af000 tx=0x7f79fc0a8000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.232+0000 7efe26a00640 1 -- 172.21.15.46:0/3546959790 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efe20050df0 con 0x7efe300cc590 2024-03-29T08:18:21.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.232+0000 7f7a02a00640 1 -- 172.21.15.46:0/2458648606 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7a0c1642c0 con 0x7f7a0c0ada90 2024-03-29T08:18:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.268+0000 7f56b8800640 1 Processor -- start 2024-03-29T08:18:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f56b8800640 1 -- start start 2024-03-29T08:18:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f5d8ca00640 1 Processor -- start 2024-03-29T08:18:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f5d8ca00640 1 -- start start 2024-03-29T08:18:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f5d8ca00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5d88160d30 0x7f5d8806c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f56b8800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56b0066100 0x7f56b0156330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f56b8800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56b00702a0 0x7f56b0070730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f56b5e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56b0066100 0x7f56b0156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f56b5e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56b0066100 0x7f56b0156330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:57778/0 (socket says 172.21.15.46:57778) 2024-03-29T08:18:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f5d8ca00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d88071b70 0x7f5d8816a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f5d8ca00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8806c860 con 0x7f5d880b11d0 2024-03-29T08:18:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f5d8ca00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8806e210 con 0x7f5d88160d30 2024-03-29T08:18:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f5d8ca00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d880ac150 con 0x7f5d88071b70 2024-03-29T08:18:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f5d85400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5d880b11d0 0x7f5d88071460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:43172/0 (socket says 172.21.15.46:43172) 2024-03-29T08:18:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f5d85400640 1 -- 172.21.15.46:0/448318952 learned_addr learned my addr 172.21.15.46:0/448318952 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f56b5e00640 1 -- 172.21.15.46:0/4001167445 learned_addr learned my addr 172.21.15.46:0/4001167445 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f56b8800640 1 -- 172.21.15.46:0/4001167445 --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b015c6c0 con 0x7f56b0156870 2024-03-29T08:18:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f56b8800640 1 -- 172.21.15.46:0/4001167445 --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b0070c70 con 0x7f56b00702a0 2024-03-29T08:18:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f56b8800640 1 -- 172.21.15.46:0/4001167445 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b015a040 con 0x7f56b0066100 2024-03-29T08:18:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f5d85e00640 1 --2- 172.21.15.46:0/448318952 >> v2:172.21.15.73:3300/0 conn(0x7f5d88160d30 0x7f5d8806c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.272+0000 7f5d86800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d88071b70 0x7f5d8816a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.276+0000 7f56b5e00640 1 -- 172.21.15.46:0/4001167445 >> v2:172.21.15.46:3300/0 conn(0x7f56b00702a0 msgr2=0x7f56b0070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.276+0000 7f56b5e00640 1 --2- 172.21.15.46:0/4001167445 >> v2:172.21.15.46:3300/0 conn(0x7f56b00702a0 0x7f56b0070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.276+0000 7f56b5e00640 1 -- 172.21.15.46:0/4001167445 >> v1:172.21.15.46:6789/0 conn(0x7f56b0156870 legacy=0x7f56b00b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.276+0000 7f56b5e00640 1 -- 172.21.15.46:0/4001167445 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b015c9a0 con 0x7f56b0066100 2024-03-29T08:18:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.276+0000 7f5d85e00640 1 -- 172.21.15.46:0/448318952 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d88071b70 msgr2=0x7f5d8816a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.276+0000 7f5d85e00640 1 --2- 172.21.15.46:0/448318952 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d88071b70 0x7f5d8816a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.276+0000 7f5d85e00640 1 -- 172.21.15.46:0/448318952 >> v1:172.21.15.73:6789/0 conn(0x7f5d880b11d0 legacy=0x7f5d88071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.276+0000 7f5d84a00640 1 -- 172.21.15.46:0/448318952 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2471203914 0 0) 0x7f5d8806c860 con 0x7f5d880b11d0 2024-03-29T08:18:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.276+0000 7f5d85e00640 1 -- 172.21.15.46:0/448318952 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8806ca90 con 0x7f5d88160d30 2024-03-29T08:18:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7fc594e00640 1 Processor -- start 2024-03-29T08:18:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7fc594e00640 1 -- start start 2024-03-29T08:18:21.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7fb52fc00640 1 Processor -- start 2024-03-29T08:18:21.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7f5d85e00640 1 --2- 172.21.15.46:0/448318952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d88160d30 0x7f5d8806c320 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5d740097c0 tx=0x7f5d7401c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50422b080f26e13c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7fb52fc00640 1 -- start start 2024-03-29T08:18:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7f56b5e00640 1 --2- 172.21.15.46:0/4001167445 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56b0066100 0x7f56b0156330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f56a40098a0 tx=0x7f56a401c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d47981939887a90 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7fc594e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc588047c20 0x7fc5880eb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7f2f9f000640 1 Processor -- start 2024-03-29T08:18:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7fc594e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc588037bc0 con 0x7fc588033e40 2024-03-29T08:18:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7fc594e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc58804f090 con 0x7fc588005ef0 2024-03-29T08:18:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7fc594e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc588050210 con 0x7fc588047c20 2024-03-29T08:18:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7f5d84a00640 1 -- 172.21.15.46:0/448318952 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d740ac640 con 0x7f5d88160d30 2024-03-29T08:18:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7f5d84a00640 1 -- 172.21.15.46:0/448318952 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d740ac7a0 con 0x7f5d88160d30 2024-03-29T08:18:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7fc58e800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc588005ef0 0x7fc588006340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:43184/0 (socket says 172.21.15.46:43184) 2024-03-29T08:18:21.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7fc58e800640 1 -- 172.21.15.46:0/2463049544 learned_addr learned my addr 172.21.15.46:0/2463049544 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7f5d84a00640 1 -- 172.21.15.46:0/448318952 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d740ac640 con 0x7f5d88160d30 2024-03-29T08:18:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7f56b4a00640 1 -- 172.21.15.46:0/4001167445 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a40ac600 con 0x7f56b0066100 2024-03-29T08:18:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7f56b4a00640 1 -- 172.21.15.46:0/4001167445 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a40a8050 con 0x7f56b0066100 2024-03-29T08:18:21.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7f56b4a00640 1 -- 172.21.15.46:0/4001167445 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a40ac980 con 0x7f56b0066100 2024-03-29T08:18:21.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7fc58ca00640 1 -- 172.21.15.46:0/2463049544 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076559939 0 0) 0x7fc588037bc0 con 0x7fc588033e40 2024-03-29T08:18:21.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58ca00640 1 -- 172.21.15.46:0/2463049544 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc574003530 con 0x7fc588033e40 2024-03-29T08:18:21.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7f5d8ca00640 1 -- 172.21.15.46:0/448318952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d88160d30 msgr2=0x7f5d8806c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7f5d8ca00640 1 --2- 172.21.15.46:0/448318952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d88160d30 0x7f5d8806c320 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5d740097c0 tx=0x7f5d7401c3e0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f56b8800640 1 -- 172.21.15.46:0/4001167445 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56b0066100 msgr2=0x7f56b0156330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f56b8800640 1 --2- 172.21.15.46:0/4001167445 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56b0066100 0x7f56b0156330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f56a40098a0 tx=0x7f56a401c4f0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58d400640 1 --2- 172.21.15.46:0/2463049544 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc588047c20 0x7fc5880eb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.280+0000 7f2f9f000640 1 -- start start 2024-03-29T08:18:21.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fb52fc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5280b5990 0x7fb5280b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f56b8800640 1 -- 172.21.15.46:0/4001167445 shutdown_connections 2024-03-29T08:18:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f56b8800640 1 --2- 172.21.15.46:0/4001167445 >> v2:172.21.15.46:3300/0 conn(0x7f56b00702a0 0x7f56b0070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f56b8800640 1 --2- 172.21.15.46:0/4001167445 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56b0066100 0x7f56b0156330 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f56b8800640 1 -- 172.21.15.46:0/4001167445 >> 172.21.15.46:0/4001167445 conn(0x7f56b0065c50 msgr2=0x7f56b0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f56b8800640 1 -- 172.21.15.46:0/4001167445 shutdown_connections 2024-03-29T08:18:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58ca00640 1 -- 172.21.15.46:0/2463049544 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3188199512 0 0) 0x7fc574003530 con 0x7fc588033e40 2024-03-29T08:18:21.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58ca00640 1 -- 172.21.15.46:0/2463049544 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc588047c20 msgr2=0x7fc5880eb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58ca00640 1 --2- 172.21.15.46:0/2463049544 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc588047c20 0x7fc5880eb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58ca00640 1 -- 172.21.15.46:0/2463049544 >> v1:172.21.15.73:6789/0 conn(0x7fc588005ef0 legacy=0x7fc588006340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58ca00640 1 -- 172.21.15.46:0/2463049544 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58804fd80 con 0x7fc588033e40 2024-03-29T08:18:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58ca00640 1 -- 172.21.15.46:0/2463049544 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fc59004e560 con 0x7fc588033e40 2024-03-29T08:18:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58ca00640 1 -- 172.21.15.46:0/2463049544 >> v1:172.21.15.46:6789/0 conn(0x7fc588033e40 legacy=0x7fc5880eb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58ca00640 1 --2- 172.21.15.46:0/2463049544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5740039b0 0x7fc574003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58ca00640 1 --2- 172.21.15.46:0/2463049544 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc588047c20 0x7fc5740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58ca00640 1 --2- 172.21.15.46:0/2463049544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc574004d10 0x7fc574005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58ca00640 1 -- 172.21.15.46:0/2463049544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc588050210 con 0x7fc574004d10 2024-03-29T08:18:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58ca00640 1 -- 172.21.15.46:0/2463049544 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57400ba50 con 0x7fc5740039b0 2024-03-29T08:18:21.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58ca00640 1 -- 172.21.15.46:0/2463049544 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57400bc10 con 0x7fc588047c20 2024-03-29T08:18:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fb52fc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5280b88e0 con 0x7fb52815cb00 2024-03-29T08:18:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fb52fc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5280b60e0 con 0x7fb5280cd5f0 2024-03-29T08:18:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fb52fc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280b63c0 con 0x7fb5280b5990 2024-03-29T08:18:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fb527e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5280b5990 0x7fb5280b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fb527e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5280b5990 0x7fb5280b83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:57814/0 (socket says 172.21.15.46:57814) 2024-03-29T08:18:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fb527e00640 1 -- 172.21.15.46:0/196867301 learned_addr learned my addr 172.21.15.46:0/196867301 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:21.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58d400640 1 --2- 172.21.15.46:0/2463049544 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc588047c20 0x7fc5740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f5d8ca00640 1 -- 172.21.15.46:0/448318952 shutdown_connections 2024-03-29T08:18:21.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f5d8ca00640 1 --2- 172.21.15.46:0/448318952 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d88071b70 0x7f5d8816a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f5d8ca00640 1 --2- 172.21.15.46:0/448318952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d88160d30 0x7f5d8806c320 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f5d8ca00640 1 -- 172.21.15.46:0/448318952 >> 172.21.15.46:0/448318952 conn(0x7f5d880b3730 msgr2=0x7f5d880b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:21.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fb527e00640 1 -- 172.21.15.46:0/196867301 >> v1:172.21.15.73:6789/0 conn(0x7fb5280cd5f0 legacy=0x7fb5280cda80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T08:18:21.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fb527e00640 1 -- 172.21.15.46:0/196867301 >> v1:172.21.15.46:6789/0 conn(0x7fb52815cb00 legacy=0x7fb5280d0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fb527e00640 1 -- 172.21.15.46:0/196867301 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5280b66a0 con 0x7fb5280b5990 2024-03-29T08:18:21.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f2f9f000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2f9004f020 0x7f2f90053640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f2f9f000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f9004f4b0 0x7f2f90053b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f2f9f000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f900531b0 con 0x7f2f900eb180 2024-03-29T08:18:21.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f2f9f000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f900502d0 con 0x7f2f9004f4b0 2024-03-29T08:18:21.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f2f9f000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f90052c30 con 0x7f2f9004f020 2024-03-29T08:18:21.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f2f97400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2f9004f020 0x7f2f90053640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f2f97400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2f9004f020 0x7f2f90053640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52034/0 (socket says 172.21.15.46:52034) 2024-03-29T08:18:21.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f2f97400640 1 -- 172.21.15.46:0/893861675 learned_addr learned my addr 172.21.15.46:0/893861675 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:21.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f5d8ca00640 1 -- 172.21.15.46:0/448318952 shutdown_connections 2024-03-29T08:18:21.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58d400640 1 -- 172.21.15.46:0/2463049544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5740039b0 msgr2=0x7fc574003dd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-29T08:18:21.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58d400640 1 --2- 172.21.15.46:0/2463049544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5740039b0 0x7fc574003dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58d400640 1 -- 172.21.15.46:0/2463049544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc574004d10 msgr2=0x7fc574005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:21.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58d400640 1 --2- 172.21.15.46:0/2463049544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc574004d10 0x7fc574005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7fc58d400640 1 -- 172.21.15.46:0/2463049544 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc574003530 con 0x7fc588047c20 2024-03-29T08:18:21.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f56b8800640 1 -- 172.21.15.46:0/4001167445 wait complete. 2024-03-29T08:18:21.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f56b8800640 1 Processor -- start 2024-03-29T08:18:21.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f5d8ca00640 1 -- 172.21.15.46:0/448318952 wait complete. 2024-03-29T08:18:21.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f2f97400640 1 -- 172.21.15.46:0/893861675 >> v2:172.21.15.46:3300/0 conn(0x7f2f9004f4b0 msgr2=0x7f2f90053b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:21.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f2f97400640 1 --2- 172.21.15.46:0/893861675 >> v2:172.21.15.46:3300/0 conn(0x7f2f9004f4b0 0x7f2f90053b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f2f97400640 1 -- 172.21.15.46:0/893861675 >> v1:172.21.15.46:6789/0 conn(0x7f2f900eb180 legacy=0x7f2f900eb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:21.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.284+0000 7f2f97400640 1 -- 172.21.15.46:0/893861675 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9004fa30 con 0x7f2f9004f020 2024-03-29T08:18:21.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f56b8800640 1 -- start start 2024-03-29T08:18:21.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f5d8ca00640 1 Processor -- start 2024-03-29T08:18:21.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f5d8ca00640 1 -- start start 2024-03-29T08:18:21.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f56b8800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56b0066100 0x7f56b0149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f56b8800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56b00702a0 0x7f56b014bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f56b5e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56b0066100 0x7f56b0149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f56b5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56b00702a0 0x7f56b014bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f56b5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56b00702a0 0x7f56b014bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:52042/0 (socket says 172.21.15.46:52042) 2024-03-29T08:18:21.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f56b5400640 1 -- 172.21.15.46:0/1439282489 learned_addr learned my addr 172.21.15.46:0/1439282489 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:21.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f56b8800640 1 --2- 172.21.15.46:0/1439282489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56b0156870 0x7f56b015a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7fb527e00640 1 --2- 172.21.15.46:0/196867301 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5280b5990 0x7fb5280b83a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb5100098a0 tx=0x7fb51001ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e342a7de4d2edb6b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f56b8800640 1 -- 172.21.15.46:0/1439282489 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b00ae080 con 0x7f56b0156870 2024-03-29T08:18:21.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f56b8800640 1 -- 172.21.15.46:0/1439282489 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0149b40 con 0x7f56b00702a0 2024-03-29T08:18:21.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f56b8800640 1 -- 172.21.15.46:0/1439282489 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b00adda0 con 0x7f56b0066100 2024-03-29T08:18:21.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7fc58d400640 1 --2- 172.21.15.46:0/2463049544 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc588047c20 0x7fc5740047d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009ae0 tx=0x7fc57c01cab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd653eec8e7b4d91 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f5d8ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d88071b70 0x7f5d88156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7fc58ca00640 1 -- 172.21.15.46:0/2463049544 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c0a56a0 con 0x7fc588047c20 2024-03-29T08:18:21.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7fc58ca00640 1 -- 172.21.15.46:0/2463049544 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc57c0a5800 con 0x7fc588047c20 2024-03-29T08:18:21.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7fc58ca00640 1 -- 172.21.15.46:0/2463049544 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc57c0a5bf0 con 0x7fc588047c20 2024-03-29T08:18:21.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f56b5400640 1 -- 172.21.15.46:0/1439282489 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56b0066100 msgr2=0x7f56b0149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f56b5400640 1 --2- 172.21.15.46:0/1439282489 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56b0066100 0x7f56b0149fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f56b5400640 1 -- 172.21.15.46:0/1439282489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56b0156870 msgr2=0x7f56b015a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f56b6800640 1 --2- 172.21.15.46:0/1439282489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56b0156870 0x7f56b015a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f5d8ca00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d880b11d0 0x7f5d88150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f5d8ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d88160d30 0x7f5d88158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f5d8ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d88156940 con 0x7f5d88071b70 2024-03-29T08:18:21.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f5d8ca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d88150a90 con 0x7f5d88160d30 2024-03-29T08:18:21.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f5d8ca00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d88155080 con 0x7f5d880b11d0 2024-03-29T08:18:21.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.288+0000 7f2f97400640 1 --2- 172.21.15.46:0/893861675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9004f020 0x7f2f90053640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2f900524c0 tx=0x7f2f8401ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56aeac0c708980a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f5d86800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d88160d30 0x7f5d88158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f5d85400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d880b11d0 0x7f5d88150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7fb527400640 1 -- 172.21.15.46:0/196867301 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5100a8a30 con 0x7fb5280b5990 2024-03-29T08:18:21.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7fb527400640 1 -- 172.21.15.46:0/196867301 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5100a8b90 con 0x7fb5280b5990 2024-03-29T08:18:21.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7fb527400640 1 -- 172.21.15.46:0/196867301 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5100b0440 con 0x7fb5280b5990 2024-03-29T08:18:21.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f56b5400640 1 --2- 172.21.15.46:0/1439282489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56b0156870 0x7f56b015a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f5d85400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d880b11d0 0x7f5d88150550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:57832/0 (socket says 172.21.15.46:57832) 2024-03-29T08:18:21.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f5d85400640 1 -- 172.21.15.46:0/846265380 learned_addr learned my addr 172.21.15.46:0/846265380 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:21.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7fc594e00640 1 -- 172.21.15.46:0/2463049544 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc588047c20 msgr2=0x7fc5740047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7fc594e00640 1 --2- 172.21.15.46:0/2463049544 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc588047c20 0x7fc5740047d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009ae0 tx=0x7fc57c01cab0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7fc58e800640 1 -- 172.21.15.46:0/2463049544 reap_dead start 2024-03-29T08:18:21.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f5d85e00640 1 --2- 172.21.15.46:0/846265380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d88071b70 0x7f5d88156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f56b5400640 1 -- 172.21.15.46:0/1439282489 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a40094d0 con 0x7f56b00702a0 2024-03-29T08:18:21.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f2f96a00640 1 -- 172.21.15.46:0/893861675 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f840a89f0 con 0x7f2f9004f020 2024-03-29T08:18:21.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f56b5e00640 1 --2- 172.21.15.46:0/1439282489 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56b0066100 0x7f56b0149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:21.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f56b6800640 1 --2- 172.21.15.46:0/1439282489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56b0156870 0x7f56b015a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:21.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f5d85400640 1 -- 172.21.15.46:0/846265380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d88160d30 msgr2=0x7f5d88158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f5d85400640 1 --2- 172.21.15.46:0/846265380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d88160d30 0x7f5d88158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f5d85400640 1 -- 172.21.15.46:0/846265380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d88071b70 msgr2=0x7f5d88156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f5d85400640 1 --2- 172.21.15.46:0/846265380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d88071b70 0x7f5d88156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f5d85400640 1 -- 172.21.15.46:0/846265380 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d74009420 con 0x7f5d880b11d0 2024-03-29T08:18:21.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7fb52fc00640 1 -- 172.21.15.46:0/196867301 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5280b5990 msgr2=0x7fb5280b83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7fb52fc00640 1 --2- 172.21.15.46:0/196867301 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5280b5990 0x7fb5280b83a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb5100098a0 tx=0x7fb51001ed30 comp rx=0 tx=0).stop 2024-03-29T08:18:21.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7fc594e00640 1 -- 172.21.15.46:0/2463049544 shutdown_connections 2024-03-29T08:18:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7fb52fc00640 1 -- 172.21.15.46:0/196867301 shutdown_connections 2024-03-29T08:18:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7fb52fc00640 1 --2- 172.21.15.46:0/196867301 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5280b5990 0x7fb5280b83a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7fb52fc00640 1 -- 172.21.15.46:0/196867301 >> 172.21.15.46:0/196867301 conn(0x7fb5280718c0 msgr2=0x7fb528071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f2f96a00640 1 -- 172.21.15.46:0/893861675 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f840a8b50 con 0x7f2f9004f020 2024-03-29T08:18:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f2f96a00640 1 -- 172.21.15.46:0/893861675 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f840b0440 con 0x7f2f9004f020 2024-03-29T08:18:21.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7fc594e00640 1 -- 172.21.15.46:0/2463049544 >> 172.21.15.46:0/2463049544 conn(0x7fc588009fc0 msgr2=0x7fc58800a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:21.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f5d85400640 1 --2- 172.21.15.46:0/846265380 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d880b11d0 0x7f5d88150550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800c4d0 tx=0x7f5d7800c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7fc594e00640 1 -- 172.21.15.46:0/2463049544 shutdown_connections 2024-03-29T08:18:21.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7fc594e00640 1 -- 172.21.15.46:0/2463049544 wait complete. 2024-03-29T08:18:21.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f56b5400640 1 --2- 172.21.15.46:0/1439282489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56b00702a0 0x7f56b014bf50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f56a000b410 tx=0x7f56a000b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f5d7e000640 1 -- 172.21.15.46:0/846265380 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d780074e0 con 0x7f5d880b11d0 2024-03-29T08:18:21.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f5d7e000640 1 -- 172.21.15.46:0/846265380 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d78007640 con 0x7f5d880b11d0 2024-03-29T08:18:21.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f56aa000640 1 -- 172.21.15.46:0/1439282489 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0004020 con 0x7f56b00702a0 2024-03-29T08:18:21.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f56aa000640 1 -- 172.21.15.46:0/1439282489 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a0004180 con 0x7f56b00702a0 2024-03-29T08:18:21.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f56b8800640 1 -- 172.21.15.46:0/1439282489 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b01494f0 con 0x7f56b00702a0 2024-03-29T08:18:21.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f56b8800640 1 -- 172.21.15.46:0/1439282489 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b014aee0 con 0x7f56b00702a0 2024-03-29T08:18:21.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f2f9f000640 1 -- 172.21.15.46:0/893861675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9004f020 msgr2=0x7f2f90053640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f2f9f000640 1 --2- 172.21.15.46:0/893861675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9004f020 0x7f2f90053640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2f900524c0 tx=0x7f2f8401ecf0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f2f9f000640 1 -- 172.21.15.46:0/893861675 shutdown_connections 2024-03-29T08:18:21.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f2f9f000640 1 --2- 172.21.15.46:0/893861675 >> v2:172.21.15.46:3300/0 conn(0x7f2f9004f4b0 0x7f2f90053b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f2f9f000640 1 --2- 172.21.15.46:0/893861675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f9004f020 0x7f2f90053640 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f2f9f000640 1 -- 172.21.15.46:0/893861675 >> 172.21.15.46:0/893861675 conn(0x7f2f90009f50 msgr2=0x7f2f9000a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f5d7e000640 1 -- 172.21.15.46:0/846265380 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d78004a30 con 0x7f5d880b11d0 2024-03-29T08:18:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7fb52fc00640 1 -- 172.21.15.46:0/196867301 shutdown_connections 2024-03-29T08:18:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f5d8ca00640 1 -- 172.21.15.46:0/846265380 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d88155360 con 0x7f5d880b11d0 2024-03-29T08:18:21.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f5d8ca00640 1 -- 172.21.15.46:0/846265380 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d88155920 con 0x7f5d880b11d0 2024-03-29T08:18:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7fb52fc00640 1 -- 172.21.15.46:0/196867301 wait complete. 2024-03-29T08:18:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f2f9f000640 1 -- 172.21.15.46:0/893861675 shutdown_connections 2024-03-29T08:18:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f56aa000640 1 -- 172.21.15.46:0/1439282489 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0011630 con 0x7f56b00702a0 2024-03-29T08:18:21.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.292+0000 7f2f9f000640 1 -- 172.21.15.46:0/893861675 wait complete. 2024-03-29T08:18:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.296+0000 7fc594e00640 1 Processor -- start 2024-03-29T08:18:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.296+0000 7fc594e00640 1 -- start start 2024-03-29T08:18:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.296+0000 7fb52fc00640 1 Processor -- start 2024-03-29T08:18:21.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.296+0000 7fb52fc00640 1 -- start start 2024-03-29T08:18:21.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.296+0000 7f5d7e000640 1 -- 172.21.15.46:0/846265380 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5d78004b90 con 0x7f5d880b11d0 2024-03-29T08:18:21.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.296+0000 7f56aa000640 1 -- 172.21.15.46:0/1439282489 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f56a0011790 con 0x7f56b00702a0 2024-03-29T08:18:21.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.296+0000 7f2f9f000640 1 Processor -- start 2024-03-29T08:18:21.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.296+0000 7f2f9f000640 1 -- start start 2024-03-29T08:18:21.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f9f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f9004f020 0x7f2f9004a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f9f000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f9004f4b0 0x7f2f900e32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fc594e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc588047c20 0x7fc5880e9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f56b8800640 1 -- 172.21.15.46:0/1439282489 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568c005260 con 0x7f56b00702a0 2024-03-29T08:18:21.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f9f000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f900eb180 0x7f2f900e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fc594e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5880e28c0 0x7fc5880e9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f5d73400640 1 -- 172.21.15.46:0/846265380 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d8804e040 con 0x7f5d880b11d0 2024-03-29T08:18:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fc594e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5880ea3f0 0x7fc5880491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fc58de00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc588047c20 0x7fc5880e9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f97400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f9004f4b0 0x7f2f900e32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f97400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f9004f4b0 0x7f2f900e32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:57844/0 (socket says 172.21.15.46:57844) 2024-03-29T08:18:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fc58de00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc588047c20 0x7fc5880e9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:57840/0 (socket says 172.21.15.46:57840) 2024-03-29T08:18:21.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fc58d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5880e28c0 0x7fc5880e9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f97e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f9004f020 0x7f2f9004a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f97e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f9004f020 0x7f2f9004a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60096/0 (socket says 172.21.15.46:60096) 2024-03-29T08:18:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f56aa000640 1 --2- 172.21.15.46:0/1439282489 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5694066760 0x7f5694068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f9f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f90049210 con 0x7f2f9004f020 2024-03-29T08:18:21.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f9f000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f900e3810 con 0x7f2f900eb180 2024-03-29T08:18:21.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f9f000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f90037160 con 0x7f2f9004f4b0 2024-03-29T08:18:21.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f97400640 1 -- 172.21.15.46:0/3584904214 learned_addr learned my addr 172.21.15.46:0/3584904214 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:21.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f56b5e00640 1 --2- 172.21.15.46:0/1439282489 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5694066760 0x7f5694068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f5d7e000640 1 --2- 172.21.15.46:0/846265380 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d68066690 0x7f5d68068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fc58de00640 1 -- 172.21.15.46:0/2168171868 learned_addr learned my addr 172.21.15.46:0/2168171868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:21.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fc594e00640 1 -- 172.21.15.46:0/2168171868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5880e8e20 con 0x7fc5880ea3f0 2024-03-29T08:18:21.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fc594e00640 1 -- 172.21.15.46:0/2168171868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc588185c90 con 0x7fc5880e28c0 2024-03-29T08:18:21.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fc594e00640 1 -- 172.21.15.46:0/2168171868 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5880e81a0 con 0x7fc588047c20 2024-03-29T08:18:21.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fb52fc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5280b5990 0x7fb5281f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fb52fc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5280cd5f0 0x7fb5280c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f56aa000640 1 -- 172.21.15.46:0/1439282489 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56a0083d50 con 0x7f56b00702a0 2024-03-29T08:18:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f5d7e000640 1 -- 172.21.15.46:0/846265380 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d780832b0 con 0x7f5d880b11d0 2024-03-29T08:18:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fb527e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5280cd5f0 0x7fb5280c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fb527e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5280cd5f0 0x7fb5280c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:57860/0 (socket says 172.21.15.46:57860) 2024-03-29T08:18:21.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fb527e00640 1 -- 172.21.15.46:0/212711305 learned_addr learned my addr 172.21.15.46:0/212711305 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:21.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f56b5e00640 1 --2- 172.21.15.46:0/1439282489 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5694066760 0x7f5694068c20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f56a40ae000 tx=0x7f56a40a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f5d85e00640 1 --2- 172.21.15.46:0/846265380 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d68066690 0x7f5d68068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fc58de00640 1 -- 172.21.15.46:0/2168171868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5880e28c0 msgr2=0x7fc5880e9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fc58de00640 1 --2- 172.21.15.46:0/2168171868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5880e28c0 0x7fc5880e9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fc58de00640 1 -- 172.21.15.46:0/2168171868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5880ea3f0 msgr2=0x7fc5880491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fc58de00640 1 --2- 172.21.15.46:0/2168171868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5880ea3f0 0x7fc5880491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fc58de00640 1 -- 172.21.15.46:0/2168171868 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5880e8480 con 0x7fc588047c20 2024-03-29T08:18:21.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f9d000640 1 --2- 172.21.15.46:0/3584904214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f900eb180 0x7f2f900e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f97400640 1 -- 172.21.15.46:0/3584904214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f900eb180 msgr2=0x7f2f900e5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f97400640 1 --2- 172.21.15.46:0/3584904214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f900eb180 0x7f2f900e5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f97400640 1 -- 172.21.15.46:0/3584904214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f9004f020 msgr2=0x7f2f9004a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f97400640 1 --2- 172.21.15.46:0/3584904214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f9004f020 0x7f2f9004a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f97400640 1 -- 172.21.15.46:0/3584904214 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f840094d0 con 0x7f2f9004f4b0 2024-03-29T08:18:21.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f9d000640 1 --2- 172.21.15.46:0/3584904214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f900eb180 0x7f2f900e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fb52d200640 1 --2- 172.21.15.46:0/212711305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5280b5990 0x7fb5281f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fc58de00640 1 --2- 172.21.15.46:0/2168171868 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc588047c20 0x7fc5880e9eb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc590069ac0 tx=0x7fc5900704b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f97e00640 1 --2- 172.21.15.46:0/3584904214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f9004f020 0x7f2f9004a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:21.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7f2f97400640 1 --2- 172.21.15.46:0/3584904214 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f9004f4b0 0x7f2f900e32d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2f8401b900 tx=0x7f2f8401ba90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fb52fc00640 1 --2- 172.21.15.46:0/212711305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52815cb00 0x7fb5280c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fb52fc00640 1 -- 172.21.15.46:0/212711305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280c8c80 con 0x7fb52815cb00 2024-03-29T08:18:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fb52fc00640 1 -- 172.21.15.46:0/212711305 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280ca5c0 con 0x7fb5280b5990 2024-03-29T08:18:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fb52fc00640 1 -- 172.21.15.46:0/212711305 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5280cae60 con 0x7fb5280cd5f0 2024-03-29T08:18:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.300+0000 7fb52dc00640 1 --2- 172.21.15.46:0/212711305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52815cb00 0x7fb5280c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7fb527e00640 1 -- 172.21.15.46:0/212711305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5280b5990 msgr2=0x7fb5281f1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7fb527e00640 1 --2- 172.21.15.46:0/212711305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5280b5990 0x7fb5281f1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7fb527e00640 1 -- 172.21.15.46:0/212711305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52815cb00 msgr2=0x7fb5280c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7fb527e00640 1 --2- 172.21.15.46:0/212711305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52815cb00 0x7fb5280c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7fb527e00640 1 -- 172.21.15.46:0/212711305 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5100094d0 con 0x7fb5280cd5f0 2024-03-29T08:18:21.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7f5d85e00640 1 --2- 172.21.15.46:0/846265380 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d68066690 0x7f5d68068b50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5d740ae000 tx=0x7f5d740a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7fb527e00640 1 --2- 172.21.15.46:0/212711305 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5280cd5f0 0x7fb5280c99f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb51001c960 tx=0x7fb51001bad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7fc586000640 1 -- 172.21.15.46:0/2168171868 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5900723c0 con 0x7fc588047c20 2024-03-29T08:18:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7f2f8be00640 1 -- 172.21.15.46:0/3584904214 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f840bf070 con 0x7f2f9004f4b0 2024-03-29T08:18:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7f2f8be00640 1 -- 172.21.15.46:0/3584904214 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f840b0c40 con 0x7f2f9004f4b0 2024-03-29T08:18:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7f2f8be00640 1 -- 172.21.15.46:0/3584904214 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f840ba440 con 0x7f2f9004f4b0 2024-03-29T08:18:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7f5d7e000640 1 -- 172.21.15.46:0/846265380 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5d78050630 con 0x7f5d880b11d0 2024-03-29T08:18:21.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7fc586000640 1 -- 172.21.15.46:0/2168171868 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc590070b30 con 0x7fc588047c20 2024-03-29T08:18:21.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7f2f9f000640 1 -- 172.21.15.46:0/3584904214 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f900494b0 con 0x7f2f9004f4b0 2024-03-29T08:18:21.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7f56aa000640 1 -- 172.21.15.46:0/1439282489 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f56a0051130 con 0x7f56b00702a0 2024-03-29T08:18:21.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7f2f9f000640 1 -- 172.21.15.46:0/3584904214 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f90049a70 con 0x7f2f9004f4b0 2024-03-29T08:18:21.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.308+0000 7fc586000640 1 -- 172.21.15.46:0/2168171868 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc590092480 con 0x7fc588047c20 2024-03-29T08:18:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.308+0000 7fc594e00640 1 -- 172.21.15.46:0/2168171868 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5880e8760 con 0x7fc588047c20 2024-03-29T08:18:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7fb524c00640 1 -- 172.21.15.46:0/212711305 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5100b0e60 con 0x7fb5280cd5f0 2024-03-29T08:18:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.308+0000 7fc594e00640 1 -- 172.21.15.46:0/2168171868 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc58804c860 con 0x7fc588047c20 2024-03-29T08:18:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.304+0000 7fb524c00640 1 -- 172.21.15.46:0/212711305 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5100ba440 con 0x7fb5280cd5f0 2024-03-29T08:18:21.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.308+0000 7fb524c00640 1 -- 172.21.15.46:0/212711305 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5100ba7f0 con 0x7fb5280cd5f0 2024-03-29T08:18:21.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.308+0000 7fb52fc00640 1 -- 172.21.15.46:0/212711305 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5280ca0f0 con 0x7fb5280cd5f0 2024-03-29T08:18:21.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.308+0000 7fb52fc00640 1 -- 172.21.15.46:0/212711305 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5280caaf0 con 0x7fb5280cd5f0 2024-03-29T08:18:21.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.308+0000 7f2f9f000640 1 -- 172.21.15.46:0/3584904214 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f90005260 con 0x7f2f9004f4b0 2024-03-29T08:18:21.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.308+0000 7fc594e00640 1 -- 172.21.15.46:0/2168171868 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5880e9090 con 0x7fc588047c20 2024-03-29T08:18:21.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.308+0000 7fc586000640 1 -- 172.21.15.46:0/2168171868 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc5900925e0 con 0x7fc588047c20 2024-03-29T08:18:21.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.308+0000 7f2f8be00640 1 -- 172.21.15.46:0/3584904214 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2f840ba5c0 con 0x7f2f9004f4b0 2024-03-29T08:18:21.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.312+0000 7fb51e000640 1 -- 172.21.15.46:0/212711305 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f8005260 con 0x7fb5280cd5f0 2024-03-29T08:18:21.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.312+0000 7fc586000640 1 --2- 172.21.15.46:0/2168171868 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc5700666c0 0x7fc570068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.312+0000 7f2f8be00640 1 --2- 172.21.15.46:0/3584904214 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f78066690 0x7f2f78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.312+0000 7fc586000640 1 -- 172.21.15.46:0/2168171868 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc59008d070 con 0x7fc588047c20 2024-03-29T08:18:21.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.312+0000 7f2f8be00640 1 -- 172.21.15.46:0/3584904214 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f84121a80 con 0x7f2f9004f4b0 2024-03-29T08:18:21.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.312+0000 7f2f97e00640 1 --2- 172.21.15.46:0/3584904214 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f78066690 0x7f2f78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.312+0000 7fc58d400640 1 --2- 172.21.15.46:0/2168171868 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc5700666c0 0x7fc570068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.312+0000 7f2f97e00640 1 --2- 172.21.15.46:0/3584904214 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f78066690 0x7f2f78068b50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009cb0 tx=0x7f2f8c009060 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.312+0000 7fb524c00640 1 -- 172.21.15.46:0/212711305 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb5100ba950 con 0x7fb5280cd5f0 2024-03-29T08:18:21.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.312+0000 7fc58d400640 1 --2- 172.21.15.46:0/2168171868 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc5700666c0 0x7fc570068b80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0af000 tx=0x7fc57c0046d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.312+0000 7fb524c00640 1 --2- 172.21.15.46:0/212711305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb504066760 0x7fb504068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.312+0000 7fb524c00640 1 -- 172.21.15.46:0/212711305 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb510121f40 con 0x7fb5280cd5f0 2024-03-29T08:18:21.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.312+0000 7fb52d200640 1 --2- 172.21.15.46:0/212711305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb504066760 0x7fb504068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.316+0000 7fc586000640 1 -- 172.21.15.46:0/2168171868 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc5900caa70 con 0x7fc588047c20 2024-03-29T08:18:21.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.316+0000 7fb52d200640 1 --2- 172.21.15.46:0/212711305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb504066760 0x7fb504068c20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb528152f90 tx=0x7fb518016040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.320+0000 7f2f8be00640 1 -- 172.21.15.46:0/3584904214 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2f840eee80 con 0x7f2f9004f4b0 2024-03-29T08:18:21.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.324+0000 7fb524c00640 1 -- 172.21.15.46:0/212711305 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb5100ef3d0 con 0x7fb5280cd5f0 2024-03-29T08:18:21.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.572+0000 7f3196a00640 1 -- 172.21.15.46:0/3790205046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f31a404e040 con 0x7f31a40cb4a0 2024-03-29T08:18:21.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.572+0000 7f31a1600640 1 -- 172.21.15.46:0/3790205046 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3190018020 con 0x7f31a40cb4a0 2024-03-29T08:18:21.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.572+0000 7efe2fe00640 1 -- 172.21.15.46:0/3546959790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efe300b3260 con 0x7efe300cc590 2024-03-29T08:18:21.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.572+0000 7efe26a00640 1 -- 172.21.15.46:0/3546959790 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efe20050df0 con 0x7efe300cc590 2024-03-29T08:18:21.575 INFO:teuthology.orchestra.run.smithi046.stdout:64424509447 2024-03-29T08:18:21.575 INFO:teuthology.orchestra.run.smithi046.stdout:64424509447 2024-03-29T08:18:21.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7f3196a00640 1 -- 172.21.15.46:0/3790205046 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f31840666c0 msgr2=0x7f3184068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7f3196a00640 1 --2- 172.21.15.46:0/3790205046 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f31840666c0 0x7f3184068b80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f31a406c860 tx=0x7f319800b040 comp rx=0 tx=0).stop 2024-03-29T08:18:21.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7f3196a00640 1 -- 172.21.15.46:0/3790205046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31a40cb4a0 msgr2=0x7f31a40b1fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7f3196a00640 1 --2- 172.21.15.46:0/3790205046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31a40cb4a0 0x7f31a40b1fa0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f319000c780 tx=0x7f319000cc40 comp rx=0 tx=0).stop 2024-03-29T08:18:21.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7efe2fe00640 1 -- 172.21.15.46:0/3546959790 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe10066690 msgr2=0x7efe10068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7efe2fe00640 1 --2- 172.21.15.46:0/3546959790 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe10066690 0x7efe10068b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efe28005c10 tx=0x7efe28009040 comp rx=0 tx=0).stop 2024-03-29T08:18:21.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7efe2fe00640 1 -- 172.21.15.46:0/3546959790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe300cc590 msgr2=0x7efe300c75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7efe2fe00640 1 --2- 172.21.15.46:0/3546959790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe300cc590 0x7efe300c75f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efe2000c500 tx=0x7efe2000c9c0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7efe2fe00640 1 -- 172.21.15.46:0/3546959790 shutdown_connections 2024-03-29T08:18:21.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7efe2fe00640 1 --2- 172.21.15.46:0/3546959790 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe10066690 0x7efe10068b50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7efe2fe00640 1 --2- 172.21.15.46:0/3546959790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe3015c9c0 0x7efe300c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7efe2fe00640 1 --2- 172.21.15.46:0/3546959790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe300cc590 0x7efe300c75f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7efe2fe00640 1 --2- 172.21.15.46:0/3546959790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe300cb4a0 0x7efe30097a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7efe2fe00640 1 -- 172.21.15.46:0/3546959790 >> 172.21.15.46:0/3546959790 conn(0x7efe300afe60 msgr2=0x7efe300ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:21.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7f3196a00640 1 -- 172.21.15.46:0/3790205046 shutdown_connections 2024-03-29T08:18:21.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7f3196a00640 1 --2- 172.21.15.46:0/3790205046 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f31840666c0 0x7f3184068b80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7f3196a00640 1 --2- 172.21.15.46:0/3790205046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a415c9c0 0x7f31a40c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7f3196a00640 1 --2- 172.21.15.46:0/3790205046 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31a40cc590 0x7f31a415cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7f3196a00640 1 --2- 172.21.15.46:0/3790205046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31a40cb4a0 0x7f31a40b1fa0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7f3196a00640 1 -- 172.21.15.46:0/3790205046 >> 172.21.15.46:0/3790205046 conn(0x7f31a40afe60 msgr2=0x7f31a4145200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:21.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.580+0000 7f3196a00640 1 -- 172.21.15.46:0/3790205046 shutdown_connections 2024-03-29T08:18:21.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.580+0000 7f3196a00640 1 -- 172.21.15.46:0/3790205046 wait complete. 2024-03-29T08:18:21.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7efe2fe00640 1 -- 172.21.15.46:0/3546959790 shutdown_connections 2024-03-29T08:18:21.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.576+0000 7efe2fe00640 1 -- 172.21.15.46:0/3546959790 wait complete. 2024-03-29T08:18:21.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.584+0000 7f7a0be00640 1 -- 172.21.15.46:0/2458648606 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7a0c1642c0 con 0x7f7a0c0ada90 2024-03-29T08:18:21.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.584+0000 7f7a02a00640 1 -- 172.21.15.46:0/2458648606 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7a0c1642c0 con 0x7f7a0c0ada90 2024-03-29T08:18:21.587 INFO:teuthology.orchestra.run.smithi046.stdout:64424509448 2024-03-29T08:18:21.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.596+0000 7f7a0be00640 1 -- 172.21.15.46:0/2458648606 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f79ec066690 msgr2=0x7f79ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.596+0000 7f7a0be00640 1 --2- 172.21.15.46:0/2458648606 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f79ec066690 0x7f79ec068b50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f79fc0af000 tx=0x7f79fc0a8000 comp rx=0 tx=0).stop 2024-03-29T08:18:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.596+0000 7f7a0be00640 1 -- 172.21.15.46:0/2458648606 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7a0c0ada90 msgr2=0x7f7a0c1545f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.596+0000 7f7a0be00640 1 --2- 172.21.15.46:0/2458648606 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7a0c0ada90 0x7f7a0c1545f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f79f80093f0 tx=0x7f79f800ed60 comp rx=0 tx=0).stop 2024-03-29T08:18:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.596+0000 7f7a0be00640 1 -- 172.21.15.46:0/2458648606 shutdown_connections 2024-03-29T08:18:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.596+0000 7f7a0be00640 1 --2- 172.21.15.46:0/2458648606 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f79ec066690 0x7f79ec068b50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.596+0000 7f7a0be00640 1 --2- 172.21.15.46:0/2458648606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a0c0cbe80 0x7f7a0c150510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.596+0000 7f7a0be00640 1 --2- 172.21.15.46:0/2458648606 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7a0c0ada90 0x7f7a0c1545f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.596+0000 7f7a0be00640 1 --2- 172.21.15.46:0/2458648606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a0c0957d0 0x7f7a0c0c4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.596+0000 7f7a0be00640 1 -- 172.21.15.46:0/2458648606 >> 172.21.15.46:0/2458648606 conn(0x7f7a0c0afe60 msgr2=0x7f7a0c0ac4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.596+0000 7f7a0be00640 1 -- 172.21.15.46:0/2458648606 shutdown_connections 2024-03-29T08:18:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.596+0000 7f7a0be00640 1 -- 172.21.15.46:0/2458648606 wait complete. 2024-03-29T08:18:21.615 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.2 2024-03-29T08:18:21.615 DEBUG:teuthology.parallel:result is None 2024-03-29T08:18:21.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.1 2024-03-29T08:18:21.617 DEBUG:teuthology.parallel:result is None 2024-03-29T08:18:21.625 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.3 2024-03-29T08:18:21.625 DEBUG:teuthology.parallel:result is None 2024-03-29T08:18:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.636+0000 7f56b8800640 1 -- 172.21.15.46:0/1439282489 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f568c005510 con 0x7f56b00702a0 2024-03-29T08:18:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.636+0000 7f56aa000640 1 -- 172.21.15.46:0/1439282489 <== mon.1 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f56a0054750 con 0x7f56b00702a0 2024-03-29T08:18:21.637 INFO:teuthology.orchestra.run.smithi046.stdout:64424509445 2024-03-29T08:18:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.636+0000 7f56b8800640 1 -- 172.21.15.46:0/1439282489 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5694066760 msgr2=0x7f5694068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.636+0000 7f56b8800640 1 --2- 172.21.15.46:0/1439282489 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5694066760 0x7f5694068c20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f56a40ae000 tx=0x7f56a40a7000 comp rx=0 tx=0).stop 2024-03-29T08:18:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.636+0000 7f56b8800640 1 -- 172.21.15.46:0/1439282489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56b00702a0 msgr2=0x7f56b014bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.636+0000 7f56b8800640 1 --2- 172.21.15.46:0/1439282489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56b00702a0 0x7f56b014bf50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f56a000b410 tx=0x7f56a000b8d0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.636+0000 7f56b8800640 1 -- 172.21.15.46:0/1439282489 shutdown_connections 2024-03-29T08:18:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.636+0000 7f56b8800640 1 --2- 172.21.15.46:0/1439282489 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5694066760 0x7f5694068c20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.636+0000 7f56b8800640 1 --2- 172.21.15.46:0/1439282489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56b00702a0 0x7f56b014bf50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.636+0000 7f56b8800640 1 --2- 172.21.15.46:0/1439282489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56b0156870 0x7f56b015a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.636+0000 7f56b8800640 1 --2- 172.21.15.46:0/1439282489 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56b0066100 0x7f56b0149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.636+0000 7f56b8800640 1 -- 172.21.15.46:0/1439282489 >> 172.21.15.46:0/1439282489 conn(0x7f56b0065c50 msgr2=0x7f56b01556c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.636+0000 7f56b8800640 1 -- 172.21.15.46:0/1439282489 shutdown_connections 2024-03-29T08:18:21.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.636+0000 7f56b8800640 1 -- 172.21.15.46:0/1439282489 wait complete. 2024-03-29T08:18:21.659 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.4 2024-03-29T08:18:21.659 DEBUG:teuthology.parallel:result is None 2024-03-29T08:18:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.660+0000 7f5d73400640 1 -- 172.21.15.46:0/846265380 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5d88159f30 con 0x7f5d880b11d0 2024-03-29T08:18:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.660+0000 7f5d7e000640 1 -- 172.21.15.46:0/846265380 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5d78053c50 con 0x7f5d880b11d0 2024-03-29T08:18:21.661 INFO:teuthology.orchestra.run.smithi046.stdout:64424509446 2024-03-29T08:18:21.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7f2f9f000640 1 -- 172.21.15.46:0/3584904214 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2f90005260 con 0x7f2f9004f4b0 2024-03-29T08:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7f5d73400640 1 -- 172.21.15.46:0/846265380 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d68066690 msgr2=0x7f5d68068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7f5d73400640 1 --2- 172.21.15.46:0/846265380 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d68066690 0x7f5d68068b50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5d740ae000 tx=0x7f5d740a7000 comp rx=0 tx=0).stop 2024-03-29T08:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7f5d73400640 1 -- 172.21.15.46:0/846265380 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d880b11d0 msgr2=0x7f5d88150550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7f5d73400640 1 --2- 172.21.15.46:0/846265380 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d880b11d0 0x7f5d88150550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d7800c4d0 tx=0x7f5d7800c990 comp rx=0 tx=0).stop 2024-03-29T08:18:21.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7fc594e00640 1 -- 172.21.15.46:0/2168171868 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc5880052d0 con 0x7fc588047c20 2024-03-29T08:18:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7f5d73400640 1 -- 172.21.15.46:0/846265380 shutdown_connections 2024-03-29T08:18:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7f5d73400640 1 --2- 172.21.15.46:0/846265380 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d68066690 0x7f5d68068b50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7f5d73400640 1 --2- 172.21.15.46:0/846265380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d88160d30 0x7f5d88158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7f5d73400640 1 --2- 172.21.15.46:0/846265380 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d880b11d0 0x7f5d88150550 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7f5d73400640 1 --2- 172.21.15.46:0/846265380 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d88071b70 0x7f5d88156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7f5d73400640 1 -- 172.21.15.46:0/846265380 >> 172.21.15.46:0/846265380 conn(0x7f5d880b3730 msgr2=0x7f5d880b16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7f5d73400640 1 -- 172.21.15.46:0/846265380 shutdown_connections 2024-03-29T08:18:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7f5d73400640 1 -- 172.21.15.46:0/846265380 wait complete. 2024-03-29T08:18:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7f2f8be00640 1 -- 172.21.15.46:0/3584904214 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2f840f24a0 con 0x7f2f9004f4b0 2024-03-29T08:18:21.667 INFO:teuthology.orchestra.run.smithi046.stdout:64424509446 2024-03-29T08:18:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7fc586000640 1 -- 172.21.15.46:0/2168171868 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc5900ce090 con 0x7fc588047c20 2024-03-29T08:18:21.668 INFO:teuthology.orchestra.run.smithi046.stdout:64424509446 2024-03-29T08:18:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7fb51e000640 1 -- 172.21.15.46:0/212711305 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb4f8005510 con 0x7fb5280cd5f0 2024-03-29T08:18:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.664+0000 7fb524c00640 1 -- 172.21.15.46:0/212711305 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb5100f29f0 con 0x7fb5280cd5f0 2024-03-29T08:18:21.668 INFO:teuthology.orchestra.run.smithi046.stdout:60129542152 2024-03-29T08:18:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7f2f9f000640 1 -- 172.21.15.46:0/3584904214 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f78066690 msgr2=0x7f2f78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7f2f9f000640 1 --2- 172.21.15.46:0/3584904214 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f78066690 0x7f2f78068b50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009cb0 tx=0x7f2f8c009060 comp rx=0 tx=0).stop 2024-03-29T08:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7f2f9f000640 1 -- 172.21.15.46:0/3584904214 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f9004f4b0 msgr2=0x7f2f900e32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7f2f9f000640 1 --2- 172.21.15.46:0/3584904214 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f9004f4b0 0x7f2f900e32d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2f8401b900 tx=0x7f2f8401ba90 comp rx=0 tx=0).stop 2024-03-29T08:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7f2f9f000640 1 -- 172.21.15.46:0/3584904214 shutdown_connections 2024-03-29T08:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7f2f9f000640 1 --2- 172.21.15.46:0/3584904214 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f78066690 0x7f2f78068b50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7f2f9f000640 1 --2- 172.21.15.46:0/3584904214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f900eb180 0x7f2f900e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7f2f9f000640 1 --2- 172.21.15.46:0/3584904214 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f9004f4b0 0x7f2f900e32d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7f2f9f000640 1 --2- 172.21.15.46:0/3584904214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f9004f020 0x7f2f9004a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7f2f9f000640 1 -- 172.21.15.46:0/3584904214 >> 172.21.15.46:0/3584904214 conn(0x7f2f90009f50 msgr2=0x7f2f9000a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7f2f9f000640 1 -- 172.21.15.46:0/3584904214 shutdown_connections 2024-03-29T08:18:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7f2f9f000640 1 -- 172.21.15.46:0/3584904214 wait complete. 2024-03-29T08:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fc594e00640 1 -- 172.21.15.46:0/2168171868 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc5700666c0 msgr2=0x7fc570068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fc594e00640 1 --2- 172.21.15.46:0/2168171868 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc5700666c0 0x7fc570068b80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0af000 tx=0x7fc57c0046d0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fc594e00640 1 -- 172.21.15.46:0/2168171868 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc588047c20 msgr2=0x7fc5880e9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fc594e00640 1 --2- 172.21.15.46:0/2168171868 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc588047c20 0x7fc5880e9eb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc590069ac0 tx=0x7fc5900704b0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fc594e00640 1 -- 172.21.15.46:0/2168171868 shutdown_connections 2024-03-29T08:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fc594e00640 1 --2- 172.21.15.46:0/2168171868 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc5700666c0 0x7fc570068b80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fc594e00640 1 --2- 172.21.15.46:0/2168171868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5880e28c0 0x7fc5880e9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fc594e00640 1 --2- 172.21.15.46:0/2168171868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5880ea3f0 0x7fc5880491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fc594e00640 1 --2- 172.21.15.46:0/2168171868 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc588047c20 0x7fc5880e9eb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fc594e00640 1 -- 172.21.15.46:0/2168171868 >> 172.21.15.46:0/2168171868 conn(0x7fc588009fc0 msgr2=0x7fc588052f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fc594e00640 1 -- 172.21.15.46:0/2168171868 shutdown_connections 2024-03-29T08:18:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fc594e00640 1 -- 172.21.15.46:0/2168171868 wait complete. 2024-03-29T08:18:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fb51e000640 1 -- 172.21.15.46:0/212711305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb504066760 msgr2=0x7fb504068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fb51e000640 1 --2- 172.21.15.46:0/212711305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb504066760 0x7fb504068c20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb528152f90 tx=0x7fb518016040 comp rx=0 tx=0).stop 2024-03-29T08:18:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fb51e000640 1 -- 172.21.15.46:0/212711305 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5280cd5f0 msgr2=0x7fb5280c99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fb51e000640 1 --2- 172.21.15.46:0/212711305 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5280cd5f0 0x7fb5280c99f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb51001c960 tx=0x7fb51001bad0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fb51e000640 1 -- 172.21.15.46:0/212711305 shutdown_connections 2024-03-29T08:18:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fb51e000640 1 --2- 172.21.15.46:0/212711305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb504066760 0x7fb504068c20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fb51e000640 1 --2- 172.21.15.46:0/212711305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb52815cb00 0x7fb5280c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fb51e000640 1 --2- 172.21.15.46:0/212711305 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5280cd5f0 0x7fb5280c99f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fb51e000640 1 --2- 172.21.15.46:0/212711305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5280b5990 0x7fb5281f1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fb51e000640 1 -- 172.21.15.46:0/212711305 >> 172.21.15.46:0/212711305 conn(0x7fb5280718c0 msgr2=0x7fb528151f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:21.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fb51e000640 1 -- 172.21.15.46:0/212711305 shutdown_connections 2024-03-29T08:18:21.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.668+0000 7fb51e000640 1 -- 172.21.15.46:0/212711305 wait complete. 2024-03-29T08:18:21.693 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.0 2024-03-29T08:18:21.693 DEBUG:teuthology.parallel:result is None 2024-03-29T08:18:21.698 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.5 2024-03-29T08:18:21.699 DEBUG:teuthology.parallel:result is None 2024-03-29T08:18:21.699 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.6 2024-03-29T08:18:21.699 DEBUG:teuthology.parallel:result is None 2024-03-29T08:18:21.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.7 2024-03-29T08:18:21.700 DEBUG:teuthology.parallel:result is None 2024-03-29T08:18:21.700 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-29T08:18:21.700 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:18:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.880+0000 7f87ab000640 1 Processor -- start 2024-03-29T08:18:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.880+0000 7f87ab000640 1 -- start start 2024-03-29T08:18:21.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87ab000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87a406a4a0 0x7f87a406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87ab000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a4161020 con 0x7f87a415feb0 2024-03-29T08:18:21.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87ab000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a4161300 con 0x7f87a41605f0 2024-03-29T08:18:21.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87ab000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a4070b70 con 0x7f87a406a4a0 2024-03-29T08:18:21.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f87a415feb0 0x7f87a4071370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32776/0 (socket says 172.21.15.46:32776) 2024-03-29T08:18:21.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3e00640 1 -- 172.21.15.46:0/2702056252 learned_addr learned my addr 172.21.15.46:0/2702056252 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a9000640 1 --2- 172.21.15.46:0/2702056252 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87a406a4a0 0x7f87a406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3400640 1 -- 172.21.15.46:0/2702056252 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816769478 0 0) 0x7f87a4161020 con 0x7f87a415feb0 2024-03-29T08:18:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3400640 1 -- 172.21.15.46:0/2702056252 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8788003530 con 0x7f87a415feb0 2024-03-29T08:18:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3400640 1 -- 172.21.15.46:0/2702056252 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4239824469 0 0) 0x7f87a4161300 con 0x7f87a41605f0 2024-03-29T08:18:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3400640 1 -- 172.21.15.46:0/2702056252 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87a4161020 con 0x7f87a41605f0 2024-03-29T08:18:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3400640 1 -- 172.21.15.46:0/2702056252 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 506847973 0 0) 0x7f8788003530 con 0x7f87a415feb0 2024-03-29T08:18:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3400640 1 -- 172.21.15.46:0/2702056252 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87a406a4a0 msgr2=0x7f87a406d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3400640 1 --2- 172.21.15.46:0/2702056252 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87a406a4a0 0x7f87a406d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3400640 1 -- 172.21.15.46:0/2702056252 >> v1:172.21.15.73:6789/0 conn(0x7f87a41605f0 legacy=0x7f87a4069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3400640 1 -- 172.21.15.46:0/2702056252 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a4070e50 con 0x7f87a415feb0 2024-03-29T08:18:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3400640 1 -- 172.21.15.46:0/2702056252 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f8794001680 con 0x7f87a415feb0 2024-03-29T08:18:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3400640 1 -- 172.21.15.46:0/2702056252 >> v1:172.21.15.46:6789/0 conn(0x7f87a415feb0 legacy=0x7f87a4071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3400640 1 --2- 172.21.15.46:0/2702056252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8788003b40 0x7f8788003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3400640 1 --2- 172.21.15.46:0/2702056252 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87a406a4a0 0x7f87880047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3400640 1 --2- 172.21.15.46:0/2702056252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8788004d10 0x7f8788005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3400640 1 -- 172.21.15.46:0/2702056252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a4070b70 con 0x7f8788004d10 2024-03-29T08:18:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3400640 1 -- 172.21.15.46:0/2702056252 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878800ba40 con 0x7f8788003b40 2024-03-29T08:18:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3400640 1 -- 172.21.15.46:0/2702056252 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f878800bc00 con 0x7f87a406a4a0 2024-03-29T08:18:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a9000640 1 --2- 172.21.15.46:0/2702056252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8788004d10 0x7f8788005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f879b400640 1 --2- 172.21.15.46:0/2702056252 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87a406a4a0 0x7f87880047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a3e00640 1 --2- 172.21.15.46:0/2702056252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8788003b40 0x7f8788003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a9000640 1 -- 172.21.15.46:0/2702056252 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87a406a4a0 msgr2=0x7f87880047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a9000640 1 --2- 172.21.15.46:0/2702056252 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87a406a4a0 0x7f87880047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a9000640 1 -- 172.21.15.46:0/2702056252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8788003b40 msgr2=0x7f8788003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a9000640 1 --2- 172.21.15.46:0/2702056252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8788003b40 0x7f8788003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.884+0000 7f87a9000640 1 -- 172.21.15.46:0/2702056252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8788003530 con 0x7f8788004d10 2024-03-29T08:18:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.888+0000 7f87a9000640 1 --2- 172.21.15.46:0/2702056252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8788004d10 0x7f8788005190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f879c009fc0 tx=0x7f879c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a62081c6ab0215 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.888+0000 7f87a3400640 1 -- 172.21.15.46:0/2702056252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8794007ea0 con 0x7f8788004d10 2024-03-29T08:18:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.888+0000 7f87a3400640 1 -- 172.21.15.46:0/2702056252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879c00f040 con 0x7f8788004d10 2024-03-29T08:18:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.888+0000 7f87a3400640 1 -- 172.21.15.46:0/2702056252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c0a5dc0 con 0x7f8788004d10 2024-03-29T08:18:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.888+0000 7f87ab000640 1 -- 172.21.15.46:0/2702056252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8788004d10 msgr2=0x7f8788005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.888+0000 7f87ab000640 1 --2- 172.21.15.46:0/2702056252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8788004d10 0x7f8788005190 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f879c009fc0 tx=0x7f879c01cc80 comp rx=0 tx=0).stop 2024-03-29T08:18:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.888+0000 7f87a9000640 1 -- 172.21.15.46:0/2702056252 reap_dead start 2024-03-29T08:18:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.888+0000 7f87ab000640 1 -- 172.21.15.46:0/2702056252 shutdown_connections 2024-03-29T08:18:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.888+0000 7f87ab000640 1 -- 172.21.15.46:0/2702056252 >> 172.21.15.46:0/2702056252 conn(0x7f87a40ced50 msgr2=0x7f87a40cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.888+0000 7f87ab000640 1 -- 172.21.15.46:0/2702056252 shutdown_connections 2024-03-29T08:18:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.888+0000 7f87ab000640 1 -- 172.21.15.46:0/2702056252 wait complete. 2024-03-29T08:18:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.888+0000 7f87ab000640 1 Processor -- start 2024-03-29T08:18:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87ab000640 1 -- start start 2024-03-29T08:18:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87ab000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8788004d10 0x7f87a40afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87ab000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a41f2430 0x7f87a415e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87ab000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87a415e600 0x7f87a4154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87ab000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40b1e10 con 0x7f87a415e600 2024-03-29T08:18:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87ab000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a41f2930 con 0x7f87a41f2430 2024-03-29T08:18:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87ab000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a41f2c10 con 0x7f8788004d10 2024-03-29T08:18:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87a9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87a415e600 0x7f87a4154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87a9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87a415e600 0x7f87a4154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60128/0 (socket says 172.21.15.46:60128) 2024-03-29T08:18:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87a9000640 1 -- 172.21.15.46:0/1175713083 learned_addr learned my addr 172.21.15.46:0/1175713083 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87a3e00640 1 --2- 172.21.15.46:0/1175713083 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8788004d10 0x7f87a40afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f879b400640 1 --2- 172.21.15.46:0/1175713083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a41f2430 0x7f87a415e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87a9000640 1 -- 172.21.15.46:0/1175713083 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8788004d10 msgr2=0x7f87a40afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87a9000640 1 --2- 172.21.15.46:0/1175713083 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8788004d10 0x7f87a40afe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87a9000640 1 -- 172.21.15.46:0/1175713083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a41f2430 msgr2=0x7f87a415e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87a9000640 1 --2- 172.21.15.46:0/1175713083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a41f2430 0x7f87a415e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87a9000640 1 -- 172.21.15.46:0/1175713083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a41f2ef0 con 0x7f87a415e600 2024-03-29T08:18:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87a3e00640 1 --2- 172.21.15.46:0/1175713083 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8788004d10 0x7f87a40afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:21.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87a9000640 1 --2- 172.21.15.46:0/1175713083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87a415e600 0x7f87a4154770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f879c002240 tx=0x7f879c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87a0c00640 1 -- 172.21.15.46:0/1175713083 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c0039a0 con 0x7f87a415e600 2024-03-29T08:18:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87a0c00640 1 -- 172.21.15.46:0/1175713083 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879c003b00 con 0x7f87a415e600 2024-03-29T08:18:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87a0c00640 1 -- 172.21.15.46:0/1175713083 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c003e70 con 0x7f87a415e600 2024-03-29T08:18:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87ab000640 1 -- 172.21.15.46:0/1175713083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a41f31d0 con 0x7f87a415e600 2024-03-29T08:18:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87ab000640 1 -- 172.21.15.46:0/1175713083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a40b0550 con 0x7f87a415e600 2024-03-29T08:18:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.892+0000 7f87ab000640 1 -- 172.21.15.46:0/1175713083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a40b1f80 con 0x7f87a415e600 2024-03-29T08:18:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.896+0000 7f87a0c00640 1 -- 172.21.15.46:0/1175713083 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f879c0b4040 con 0x7f87a415e600 2024-03-29T08:18:21.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.896+0000 7f87a0c00640 1 --2- 172.21.15.46:0/1175713083 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8780066690 0x7f8780068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:21.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.896+0000 7f87a0c00640 1 -- 172.21.15.46:0/1175713083 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f879c11da40 con 0x7f87a415e600 2024-03-29T08:18:21.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.896+0000 7f87a3e00640 1 --2- 172.21.15.46:0/1175713083 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8780066690 0x7f8780068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:21.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.896+0000 7f87a3e00640 1 --2- 172.21.15.46:0/1175713083 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8780066690 0x7f8780068b50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8794000c10 tx=0x7f8794009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:21.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:21.900+0000 7f87a0c00640 1 -- 172.21.15.46:0/1175713083 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f879c0eaed0 con 0x7f87a415e600 2024-03-29T08:18:22.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.060+0000 7f87ab000640 1 -- 172.21.15.46:0/1175713083 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87a4160a80 con 0x7f8780066690 2024-03-29T08:18:22.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.068+0000 7f87a0c00640 1 -- 172.21.15.46:0/1175713083 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264552 (secure 0 0 0) 0x7f8794005730 con 0x7f8780066690 2024-03-29T08:18:22.071 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:22.073 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:18:22.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.068+0000 7f87ab000640 1 -- 172.21.15.46:0/1175713083 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8780066690 msgr2=0x7f8780068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:22.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.068+0000 7f87ab000640 1 --2- 172.21.15.46:0/1175713083 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8780066690 0x7f8780068b50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8794000c10 tx=0x7f8794009040 comp rx=0 tx=0).stop 2024-03-29T08:18:22.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.068+0000 7f87ab000640 1 -- 172.21.15.46:0/1175713083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87a415e600 msgr2=0x7f87a4154770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:22.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.068+0000 7f87ab000640 1 --2- 172.21.15.46:0/1175713083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87a415e600 0x7f87a4154770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f879c002240 tx=0x7f879c009750 comp rx=0 tx=0).stop 2024-03-29T08:18:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.068+0000 7f87ab000640 1 -- 172.21.15.46:0/1175713083 shutdown_connections 2024-03-29T08:18:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.068+0000 7f87ab000640 1 --2- 172.21.15.46:0/1175713083 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8780066690 0x7f8780068b50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.068+0000 7f87ab000640 1 --2- 172.21.15.46:0/1175713083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87a41f2430 0x7f87a415e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.068+0000 7f87ab000640 1 --2- 172.21.15.46:0/1175713083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87a415e600 0x7f87a4154770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.068+0000 7f87ab000640 1 --2- 172.21.15.46:0/1175713083 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8788004d10 0x7f87a40afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.068+0000 7f87ab000640 1 -- 172.21.15.46:0/1175713083 >> 172.21.15.46:0/1175713083 conn(0x7f87a40ced50 msgr2=0x7f87a4072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.068+0000 7f87ab000640 1 -- 172.21.15.46:0/1175713083 shutdown_connections 2024-03-29T08:18:22.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.068+0000 7f87ab000640 1 -- 172.21.15.46:0/1175713083 wait complete. 2024-03-29T08:18:22.090 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-03-29T08:18:22.043829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235528,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739192,"statfs":{"total":773094113280,"available":772852932608,"internally_reserved":0,"allocated":3022848,"data_stored":1795380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.724786"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102211+0000","last_change":"2024-03-29T08:18:13.347398+0000","last_active":"2024-03-29T08:18:15.102211+0000","last_peered":"2024-03-29T08:18:15.102211+0000","last_clean":"2024-03-29T08:18:15.102211+0000","last_became_active":"2024-03-29T08:18:13.346857+0000","last_became_peered":"2024-03-29T08:18:13.346857+0000","last_unstale":"2024-03-29T08:18:15.102211+0000","last_undegraded":"2024-03-29T08:18:15.102211+0000","last_fullsized":"2024-03-29T08:18:15.102211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:03.590543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097264+0000","last_change":"2024-03-29T08:18:13.358700+0000","last_active":"2024-03-29T08:18:14.097264+0000","last_peered":"2024-03-29T08:18:14.097264+0000","last_clean":"2024-03-29T08:18:14.097264+0000","last_became_active":"2024-03-29T08:18:13.357646+0000","last_became_peered":"2024-03-29T08:18:13.357646+0000","last_unstale":"2024-03-29T08:18:14.097264+0000","last_undegraded":"2024-03-29T08:18:14.097264+0000","last_fullsized":"2024-03-29T08:18:14.097264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:20.587134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097240+0000","last_change":"2024-03-29T08:18:13.352326+0000","last_active":"2024-03-29T08:18:14.097240+0000","last_peered":"2024-03-29T08:18:14.097240+0000","last_clean":"2024-03-29T08:18:14.097240+0000","last_became_active":"2024-03-29T08:18:13.351954+0000","last_became_peered":"2024-03-29T08:18:13.351954+0000","last_unstale":"2024-03-29T08:18:14.097240+0000","last_undegraded":"2024-03-29T08:18:14.097240+0000","last_fullsized":"2024-03-29T08:18:14.097240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:07:42.013921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089100+0000","last_change":"2024-03-29T08:18:13.336578+0000","last_active":"2024-03-29T08:18:15.089100+0000","last_peered":"2024-03-29T08:18:15.089100+0000","last_clean":"2024-03-29T08:18:15.089100+0000","last_became_active":"2024-03-29T08:18:13.336344+0000","last_became_peered":"2024-03-29T08:18:13.336344+0000","last_unstale":"2024-03-29T08:18:15.089100+0000","last_undegraded":"2024-03-29T08:18:15.089100+0000","last_fullsized":"2024-03-29T08:18:15.089100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:30.625311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102083+0000","last_change":"2024-03-29T08:18:13.348320+0000","last_active":"2024-03-29T08:18:15.102083+0000","last_peered":"2024-03-29T08:18:15.102083+0000","last_clean":"2024-03-29T08:18:15.102083+0000","last_became_active":"2024-03-29T08:18:13.345863+0000","last_became_peered":"2024-03-29T08:18:13.345863+0000","last_unstale":"2024-03-29T08:18:15.102083+0000","last_undegraded":"2024-03-29T08:18:15.102083+0000","last_fullsized":"2024-03-29T08:18:15.102083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:26:51.855915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.088872+0000","last_change":"2024-03-29T08:18:13.345374+0000","last_active":"2024-03-29T08:18:15.088872+0000","last_peered":"2024-03-29T08:18:15.088872+0000","last_clean":"2024-03-29T08:18:15.088872+0000","last_became_active":"2024-03-29T08:18:13.345054+0000","last_became_peered":"2024-03-29T08:18:13.345054+0000","last_unstale":"2024-03-29T08:18:15.088872+0000","last_undegraded":"2024-03-29T08:18:15.088872+0000","last_fullsized":"2024-03-29T08:18:15.088872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:32.324564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102055+0000","last_change":"2024-03-29T08:18:13.346637+0000","last_active":"2024-03-29T08:18:15.102055+0000","last_peered":"2024-03-29T08:18:15.102055+0000","last_clean":"2024-03-29T08:18:15.102055+0000","last_became_active":"2024-03-29T08:18:13.346335+0000","last_became_peered":"2024-03-29T08:18:13.346335+0000","last_unstale":"2024-03-29T08:18:15.102055+0000","last_undegraded":"2024-03-29T08:18:15.102055+0000","last_fullsized":"2024-03-29T08:18:15.102055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:23.267817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097122+0000","last_change":"2024-03-29T08:18:13.348979+0000","last_active":"2024-03-29T08:18:14.097122+0000","last_peered":"2024-03-29T08:18:14.097122+0000","last_clean":"2024-03-29T08:18:14.097122+0000","last_became_active":"2024-03-29T08:18:13.348682+0000","last_became_peered":"2024-03-29T08:18:13.348682+0000","last_unstale":"2024-03-29T08:18:14.097122+0000","last_undegraded":"2024-03-29T08:18:14.097122+0000","last_fullsized":"2024-03-29T08:18:14.097122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:53:21.565448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321443+0000","last_change":"2024-03-29T08:18:13.339556+0000","last_active":"2024-03-29T08:18:14.321443+0000","last_peered":"2024-03-29T08:18:14.321443+0000","last_clean":"2024-03-29T08:18:14.321443+0000","last_became_active":"2024-03-29T08:18:13.339318+0000","last_became_peered":"2024-03-29T08:18:13.339318+0000","last_unstale":"2024-03-29T08:18:14.321443+0000","last_undegraded":"2024-03-29T08:18:14.321443+0000","last_fullsized":"2024-03-29T08:18:14.321443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:45.475256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102190+0000","last_change":"2024-03-29T08:18:13.337238+0000","last_active":"2024-03-29T08:18:15.102190+0000","last_peered":"2024-03-29T08:18:15.102190+0000","last_clean":"2024-03-29T08:18:15.102190+0000","last_became_active":"2024-03-29T08:18:13.336549+0000","last_became_peered":"2024-03-29T08:18:13.336549+0000","last_unstale":"2024-03-29T08:18:15.102190+0000","last_undegraded":"2024-03-29T08:18:15.102190+0000","last_fullsized":"2024-03-29T08:18:15.102190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:32:48.313814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102161+0000","last_change":"2024-03-29T08:18:13.346353+0000","last_active":"2024-03-29T08:18:15.102161+0000","last_peered":"2024-03-29T08:18:15.102161+0000","last_clean":"2024-03-29T08:18:15.102161+0000","last_became_active":"2024-03-29T08:18:13.345780+0000","last_became_peered":"2024-03-29T08:18:13.345780+0000","last_unstale":"2024-03-29T08:18:15.102161+0000","last_undegraded":"2024-03-29T08:18:15.102161+0000","last_fullsized":"2024-03-29T08:18:15.102161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:45:55.900052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102103+0000","last_change":"2024-03-29T08:18:13.331322+0000","last_active":"2024-03-29T08:18:15.102103+0000","last_peered":"2024-03-29T08:18:15.102103+0000","last_clean":"2024-03-29T08:18:15.102103+0000","last_became_active":"2024-03-29T08:18:13.331047+0000","last_became_peered":"2024-03-29T08:18:13.331047+0000","last_unstale":"2024-03-29T08:18:15.102103+0000","last_undegraded":"2024-03-29T08:18:15.102103+0000","last_fullsized":"2024-03-29T08:18:15.102103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:40.888705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102826+0000","last_change":"2024-03-29T08:18:13.357625+0000","last_active":"2024-03-29T08:18:15.102826+0000","last_peered":"2024-03-29T08:18:15.102826+0000","last_clean":"2024-03-29T08:18:15.102826+0000","last_became_active":"2024-03-29T08:18:13.356826+0000","last_became_peered":"2024-03-29T08:18:13.356826+0000","last_unstale":"2024-03-29T08:18:15.102826+0000","last_undegraded":"2024-03-29T08:18:15.102826+0000","last_fullsized":"2024-03-29T08:18:15.102826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:30.625311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102767+0000","last_change":"2024-03-29T08:18:13.356033+0000","last_active":"2024-03-29T08:18:15.102767+0000","last_peered":"2024-03-29T08:18:15.102767+0000","last_clean":"2024-03-29T08:18:15.102767+0000","last_became_active":"2024-03-29T08:18:13.355720+0000","last_became_peered":"2024-03-29T08:18:13.355720+0000","last_unstale":"2024-03-29T08:18:15.102767+0000","last_undegraded":"2024-03-29T08:18:15.102767+0000","last_fullsized":"2024-03-29T08:18:15.102767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:56.798095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097166+0000","last_change":"2024-03-29T08:18:13.349080+0000","last_active":"2024-03-29T08:18:14.097166+0000","last_peered":"2024-03-29T08:18:14.097166+0000","last_clean":"2024-03-29T08:18:14.097166+0000","last_became_active":"2024-03-29T08:18:13.348721+0000","last_became_peered":"2024-03-29T08:18:13.348721+0000","last_unstale":"2024-03-29T08:18:14.097166+0000","last_undegraded":"2024-03-29T08:18:14.097166+0000","last_fullsized":"2024-03-29T08:18:14.097166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:54.605777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106254+0000","last_change":"2024-03-29T08:18:13.349631+0000","last_active":"2024-03-29T08:18:15.106254+0000","last_peered":"2024-03-29T08:18:15.106254+0000","last_clean":"2024-03-29T08:18:15.106254+0000","last_became_active":"2024-03-29T08:18:13.349377+0000","last_became_peered":"2024-03-29T08:18:13.349377+0000","last_unstale":"2024-03-29T08:18:15.106254+0000","last_undegraded":"2024-03-29T08:18:15.106254+0000","last_fullsized":"2024-03-29T08:18:15.106254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:44:22.135683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106572+0000","last_change":"2024-03-29T08:18:13.342594+0000","last_active":"2024-03-29T08:18:15.106572+0000","last_peered":"2024-03-29T08:18:15.106572+0000","last_clean":"2024-03-29T08:18:15.106572+0000","last_became_active":"2024-03-29T08:18:13.341973+0000","last_became_peered":"2024-03-29T08:18:13.341973+0000","last_unstale":"2024-03-29T08:18:15.106572+0000","last_undegraded":"2024-03-29T08:18:15.106572+0000","last_fullsized":"2024-03-29T08:18:15.106572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:31:39.118083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323494+0000","last_change":"2024-03-29T08:18:13.349000+0000","last_active":"2024-03-29T08:18:14.323494+0000","last_peered":"2024-03-29T08:18:14.323494+0000","last_clean":"2024-03-29T08:18:14.323494+0000","last_became_active":"2024-03-29T08:18:13.348586+0000","last_became_peered":"2024-03-29T08:18:13.348586+0000","last_unstale":"2024-03-29T08:18:14.323494+0000","last_undegraded":"2024-03-29T08:18:14.323494+0000","last_fullsized":"2024-03-29T08:18:14.323494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:55:41.020683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106529+0000","last_change":"2024-03-29T08:18:13.342652+0000","last_active":"2024-03-29T08:18:15.106529+0000","last_peered":"2024-03-29T08:18:15.106529+0000","last_clean":"2024-03-29T08:18:15.106529+0000","last_became_active":"2024-03-29T08:18:13.342339+0000","last_became_peered":"2024-03-29T08:18:13.342339+0000","last_unstale":"2024-03-29T08:18:15.106529+0000","last_undegraded":"2024-03-29T08:18:15.106529+0000","last_fullsized":"2024-03-29T08:18:15.106529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:46:35.075911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102113+0000","last_change":"2024-03-29T08:18:13.325928+0000","last_active":"2024-03-29T08:18:15.102113+0000","last_peered":"2024-03-29T08:18:15.102113+0000","last_clean":"2024-03-29T08:18:15.102113+0000","last_became_active":"2024-03-29T08:18:13.325659+0000","last_became_peered":"2024-03-29T08:18:13.325659+0000","last_unstale":"2024-03-29T08:18:15.102113+0000","last_undegraded":"2024-03-29T08:18:15.102113+0000","last_fullsized":"2024-03-29T08:18:15.102113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:58.411155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324285+0000","last_change":"2024-03-29T08:18:13.339244+0000","last_active":"2024-03-29T08:18:14.324285+0000","last_peered":"2024-03-29T08:18:14.324285+0000","last_clean":"2024-03-29T08:18:14.324285+0000","last_became_active":"2024-03-29T08:18:13.338984+0000","last_became_peered":"2024-03-29T08:18:13.338984+0000","last_unstale":"2024-03-29T08:18:14.324285+0000","last_undegraded":"2024-03-29T08:18:14.324285+0000","last_fullsized":"2024-03-29T08:18:14.324285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:29:09.699201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321134+0000","last_change":"2024-03-29T08:18:13.330207+0000","last_active":"2024-03-29T08:18:14.321134+0000","last_peered":"2024-03-29T08:18:14.321134+0000","last_clean":"2024-03-29T08:18:14.321134+0000","last_became_active":"2024-03-29T08:18:13.330029+0000","last_became_peered":"2024-03-29T08:18:13.330029+0000","last_unstale":"2024-03-29T08:18:14.321134+0000","last_undegraded":"2024-03-29T08:18:14.321134+0000","last_fullsized":"2024-03-29T08:18:14.321134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:32.324564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097178+0000","last_change":"2024-03-29T08:18:13.347985+0000","last_active":"2024-03-29T08:18:14.097178+0000","last_peered":"2024-03-29T08:18:14.097178+0000","last_clean":"2024-03-29T08:18:14.097178+0000","last_became_active":"2024-03-29T08:18:13.347561+0000","last_became_peered":"2024-03-29T08:18:13.347561+0000","last_unstale":"2024-03-29T08:18:14.097178+0000","last_undegraded":"2024-03-29T08:18:14.097178+0000","last_fullsized":"2024-03-29T08:18:14.097178+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:17.775287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106271+0000","last_change":"2024-03-29T08:18:13.338212+0000","last_active":"2024-03-29T08:18:15.106271+0000","last_peered":"2024-03-29T08:18:15.106271+0000","last_clean":"2024-03-29T08:18:15.106271+0000","last_became_active":"2024-03-29T08:18:13.337908+0000","last_became_peered":"2024-03-29T08:18:13.337908+0000","last_unstale":"2024-03-29T08:18:15.106271+0000","last_undegraded":"2024-03-29T08:18:15.106271+0000","last_fullsized":"2024-03-29T08:18:15.106271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:40.888705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097296+0000","last_change":"2024-03-29T08:18:13.341534+0000","last_active":"2024-03-29T08:18:14.097296+0000","last_peered":"2024-03-29T08:18:14.097296+0000","last_clean":"2024-03-29T08:18:14.097296+0000","last_became_active":"2024-03-29T08:18:13.341097+0000","last_became_peered":"2024-03-29T08:18:13.341097+0000","last_unstale":"2024-03-29T08:18:14.097296+0000","last_undegraded":"2024-03-29T08:18:14.097296+0000","last_fullsized":"2024-03-29T08:18:14.097296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:05:16.874804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089257+0000","last_change":"2024-03-29T08:18:13.342465+0000","last_active":"2024-03-29T08:18:15.089257+0000","last_peered":"2024-03-29T08:18:15.089257+0000","last_clean":"2024-03-29T08:18:15.089257+0000","last_became_active":"2024-03-29T08:18:13.342000+0000","last_became_peered":"2024-03-29T08:18:13.342000+0000","last_unstale":"2024-03-29T08:18:15.089257+0000","last_undegraded":"2024-03-29T08:18:15.089257+0000","last_fullsized":"2024-03-29T08:18:15.089257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:05.352866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102918+0000","last_change":"2024-03-29T08:18:13.356380+0000","last_active":"2024-03-29T08:18:15.102918+0000","last_peered":"2024-03-29T08:18:15.102918+0000","last_clean":"2024-03-29T08:18:15.102918+0000","last_became_active":"2024-03-29T08:18:13.356016+0000","last_became_peered":"2024-03-29T08:18:13.356016+0000","last_unstale":"2024-03-29T08:18:15.102918+0000","last_undegraded":"2024-03-29T08:18:15.102918+0000","last_fullsized":"2024-03-29T08:18:15.102918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:05.352866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102841+0000","last_change":"2024-03-29T08:18:13.357406+0000","last_active":"2024-03-29T08:18:15.102841+0000","last_peered":"2024-03-29T08:18:15.102841+0000","last_clean":"2024-03-29T08:18:15.102841+0000","last_became_active":"2024-03-29T08:18:13.356875+0000","last_became_peered":"2024-03-29T08:18:13.356875+0000","last_unstale":"2024-03-29T08:18:15.102841+0000","last_undegraded":"2024-03-29T08:18:15.102841+0000","last_fullsized":"2024-03-29T08:18:15.102841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:52:27.461296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097189+0000","last_change":"2024-03-29T08:18:13.338239+0000","last_active":"2024-03-29T08:18:14.097189+0000","last_peered":"2024-03-29T08:18:14.097189+0000","last_clean":"2024-03-29T08:18:14.097189+0000","last_became_active":"2024-03-29T08:18:13.337901+0000","last_became_peered":"2024-03-29T08:18:13.337901+0000","last_unstale":"2024-03-29T08:18:14.097189+0000","last_undegraded":"2024-03-29T08:18:14.097189+0000","last_fullsized":"2024-03-29T08:18:14.097189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:41:40.150525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106380+0000","last_change":"2024-03-29T08:18:13.338530+0000","last_active":"2024-03-29T08:18:15.106380+0000","last_peered":"2024-03-29T08:18:15.106380+0000","last_clean":"2024-03-29T08:18:15.106380+0000","last_became_active":"2024-03-29T08:18:13.338188+0000","last_became_peered":"2024-03-29T08:18:13.338188+0000","last_unstale":"2024-03-29T08:18:15.106380+0000","last_undegraded":"2024-03-29T08:18:15.106380+0000","last_fullsized":"2024-03-29T08:18:15.106380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:03.590543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323230+0000","last_change":"2024-03-29T08:18:13.344948+0000","last_active":"2024-03-29T08:18:14.323230+0000","last_peered":"2024-03-29T08:18:14.323230+0000","last_clean":"2024-03-29T08:18:14.323230+0000","last_became_active":"2024-03-29T08:18:13.341442+0000","last_became_peered":"2024-03-29T08:18:13.341442+0000","last_unstale":"2024-03-29T08:18:14.323230+0000","last_undegraded":"2024-03-29T08:18:14.323230+0000","last_fullsized":"2024-03-29T08:18:14.323230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:53:57.611306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.088799+0000","last_change":"2024-03-29T08:18:13.356080+0000","last_active":"2024-03-29T08:18:15.088799+0000","last_peered":"2024-03-29T08:18:15.088799+0000","last_clean":"2024-03-29T08:18:15.088799+0000","last_became_active":"2024-03-29T08:18:13.355885+0000","last_became_peered":"2024-03-29T08:18:13.355885+0000","last_unstale":"2024-03-29T08:18:15.088799+0000","last_undegraded":"2024-03-29T08:18:15.088799+0000","last_fullsized":"2024-03-29T08:18:15.088799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:57.716575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321203+0000","last_change":"2024-03-29T08:18:13.343561+0000","last_active":"2024-03-29T08:18:14.321203+0000","last_peered":"2024-03-29T08:18:14.321203+0000","last_clean":"2024-03-29T08:18:14.321203+0000","last_became_active":"2024-03-29T08:18:13.341129+0000","last_became_peered":"2024-03-29T08:18:13.341129+0000","last_unstale":"2024-03-29T08:18:14.321203+0000","last_undegraded":"2024-03-29T08:18:14.321203+0000","last_fullsized":"2024-03-29T08:18:14.321203+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:30.625311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097200+0000","last_change":"2024-03-29T08:18:12.928717+0000","last_active":"2024-03-29T08:18:14.097200+0000","last_peered":"2024-03-29T08:18:14.097200+0000","last_clean":"2024-03-29T08:18:14.097200+0000","last_became_active":"2024-03-29T08:18:12.928427+0000","last_became_peered":"2024-03-29T08:18:12.928427+0000","last_unstale":"2024-03-29T08:18:14.097200+0000","last_undegraded":"2024-03-29T08:18:14.097200+0000","last_fullsized":"2024-03-29T08:18:14.097200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:28:47.778848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324418+0000","last_change":"2024-03-29T08:18:13.346417+0000","last_active":"2024-03-29T08:18:14.324418+0000","last_peered":"2024-03-29T08:18:14.324418+0000","last_clean":"2024-03-29T08:18:14.324418+0000","last_became_active":"2024-03-29T08:18:13.344718+0000","last_became_peered":"2024-03-29T08:18:13.344718+0000","last_unstale":"2024-03-29T08:18:14.324418+0000","last_undegraded":"2024-03-29T08:18:14.324418+0000","last_fullsized":"2024-03-29T08:18:14.324418+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:24:34.643975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106404+0000","last_change":"2024-03-29T08:18:12.930498+0000","last_active":"2024-03-29T08:18:15.106404+0000","last_peered":"2024-03-29T08:18:15.106404+0000","last_clean":"2024-03-29T08:18:15.106404+0000","last_became_active":"2024-03-29T08:18:12.930232+0000","last_became_peered":"2024-03-29T08:18:12.930232+0000","last_unstale":"2024-03-29T08:18:15.106404+0000","last_undegraded":"2024-03-29T08:18:15.106404+0000","last_fullsized":"2024-03-29T08:18:15.106404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:04.346131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102747+0000","last_change":"2024-03-29T08:18:12.927991+0000","last_active":"2024-03-29T08:18:15.102747+0000","last_peered":"2024-03-29T08:18:15.102747+0000","last_clean":"2024-03-29T08:18:15.102747+0000","last_became_active":"2024-03-29T08:18:12.927288+0000","last_became_peered":"2024-03-29T08:18:12.927288+0000","last_unstale":"2024-03-29T08:18:15.102747+0000","last_undegraded":"2024-03-29T08:18:15.102747+0000","last_fullsized":"2024-03-29T08:18:15.102747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:30:10.121158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097153+0000","last_change":"2024-03-29T08:18:13.358303+0000","last_active":"2024-03-29T08:18:14.097153+0000","last_peered":"2024-03-29T08:18:14.097153+0000","last_clean":"2024-03-29T08:18:14.097153+0000","last_became_active":"2024-03-29T08:18:13.358041+0000","last_became_peered":"2024-03-29T08:18:13.358041+0000","last_unstale":"2024-03-29T08:18:14.097153+0000","last_undegraded":"2024-03-29T08:18:14.097153+0000","last_fullsized":"2024-03-29T08:18:14.097153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:41:41.207988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106281+0000","last_change":"2024-03-29T08:18:12.927896+0000","last_active":"2024-03-29T08:18:15.106281+0000","last_peered":"2024-03-29T08:18:15.106281+0000","last_clean":"2024-03-29T08:18:15.106281+0000","last_became_active":"2024-03-29T08:18:12.927399+0000","last_became_peered":"2024-03-29T08:18:12.927399+0000","last_unstale":"2024-03-29T08:18:15.106281+0000","last_undegraded":"2024-03-29T08:18:15.106281+0000","last_fullsized":"2024-03-29T08:18:15.106281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:56.709840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324362+0000","last_change":"2024-03-29T08:18:13.338175+0000","last_active":"2024-03-29T08:18:14.324362+0000","last_peered":"2024-03-29T08:18:14.324362+0000","last_clean":"2024-03-29T08:18:14.324362+0000","last_became_active":"2024-03-29T08:18:13.337775+0000","last_became_peered":"2024-03-29T08:18:13.337775+0000","last_unstale":"2024-03-29T08:18:14.324362+0000","last_undegraded":"2024-03-29T08:18:14.324362+0000","last_fullsized":"2024-03-29T08:18:14.324362+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:45:52.608559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321469+0000","last_change":"2024-03-29T08:18:12.940829+0000","last_active":"2024-03-29T08:18:14.321469+0000","last_peered":"2024-03-29T08:18:14.321469+0000","last_clean":"2024-03-29T08:18:14.321469+0000","last_became_active":"2024-03-29T08:18:12.929330+0000","last_became_peered":"2024-03-29T08:18:12.929330+0000","last_unstale":"2024-03-29T08:18:14.321469+0000","last_undegraded":"2024-03-29T08:18:14.321469+0000","last_fullsized":"2024-03-29T08:18:14.321469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:55:09.380677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089346+0000","last_change":"2024-03-29T08:18:13.342303+0000","last_active":"2024-03-29T08:18:15.089346+0000","last_peered":"2024-03-29T08:18:15.089346+0000","last_clean":"2024-03-29T08:18:15.089346+0000","last_became_active":"2024-03-29T08:18:13.341642+0000","last_became_peered":"2024-03-29T08:18:13.341642+0000","last_unstale":"2024-03-29T08:18:15.089346+0000","last_undegraded":"2024-03-29T08:18:15.089346+0000","last_fullsized":"2024-03-29T08:18:15.089346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:45.475256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102200+0000","last_change":"2024-03-29T08:18:12.930755+0000","last_active":"2024-03-29T08:18:15.102200+0000","last_peered":"2024-03-29T08:18:15.102200+0000","last_clean":"2024-03-29T08:18:15.102200+0000","last_became_active":"2024-03-29T08:18:12.929647+0000","last_became_peered":"2024-03-29T08:18:12.929647+0000","last_unstale":"2024-03-29T08:18:15.102200+0000","last_undegraded":"2024-03-29T08:18:15.102200+0000","last_fullsized":"2024-03-29T08:18:15.102200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:16:38.127431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097317+0000","last_change":"2024-03-29T08:18:13.344231+0000","last_active":"2024-03-29T08:18:14.097317+0000","last_peered":"2024-03-29T08:18:14.097317+0000","last_clean":"2024-03-29T08:18:14.097317+0000","last_became_active":"2024-03-29T08:18:13.343788+0000","last_became_peered":"2024-03-29T08:18:13.343788+0000","last_unstale":"2024-03-29T08:18:14.097317+0000","last_undegraded":"2024-03-29T08:18:14.097317+0000","last_fullsized":"2024-03-29T08:18:14.097317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:15:15.589437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323478+0000","last_change":"2024-03-29T08:18:12.925532+0000","last_active":"2024-03-29T08:18:14.323478+0000","last_peered":"2024-03-29T08:18:14.323478+0000","last_clean":"2024-03-29T08:18:14.323478+0000","last_became_active":"2024-03-29T08:18:12.925228+0000","last_became_peered":"2024-03-29T08:18:12.925228+0000","last_unstale":"2024-03-29T08:18:14.323478+0000","last_undegraded":"2024-03-29T08:18:14.323478+0000","last_fullsized":"2024-03-29T08:18:14.323478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:30:10.121158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106489+0000","last_change":"2024-03-29T08:18:13.339687+0000","last_active":"2024-03-29T08:18:15.106489+0000","last_peered":"2024-03-29T08:18:15.106489+0000","last_clean":"2024-03-29T08:18:15.106489+0000","last_became_active":"2024-03-29T08:18:13.339445+0000","last_became_peered":"2024-03-29T08:18:13.339445+0000","last_unstale":"2024-03-29T08:18:15.106489+0000","last_undegraded":"2024-03-29T08:18:15.106489+0000","last_fullsized":"2024-03-29T08:18:15.106489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:45:55.900052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089195+0000","last_change":"2024-03-29T08:18:12.927677+0000","last_active":"2024-03-29T08:18:15.089195+0000","last_peered":"2024-03-29T08:18:15.089195+0000","last_clean":"2024-03-29T08:18:15.089195+0000","last_became_active":"2024-03-29T08:18:12.927416+0000","last_became_peered":"2024-03-29T08:18:12.927416+0000","last_unstale":"2024-03-29T08:18:15.089195+0000","last_undegraded":"2024-03-29T08:18:15.089195+0000","last_fullsized":"2024-03-29T08:18:15.089195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:47.035098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102170+0000","last_change":"2024-03-29T08:18:13.357432+0000","last_active":"2024-03-29T08:18:15.102170+0000","last_peered":"2024-03-29T08:18:15.102170+0000","last_clean":"2024-03-29T08:18:15.102170+0000","last_became_active":"2024-03-29T08:18:13.357221+0000","last_became_peered":"2024-03-29T08:18:13.357221+0000","last_unstale":"2024-03-29T08:18:15.102170+0000","last_undegraded":"2024-03-29T08:18:15.102170+0000","last_fullsized":"2024-03-29T08:18:15.102170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:49.457858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323462+0000","last_change":"2024-03-29T08:18:12.930512+0000","last_active":"2024-03-29T08:18:14.323462+0000","last_peered":"2024-03-29T08:18:14.323462+0000","last_clean":"2024-03-29T08:18:14.323462+0000","last_became_active":"2024-03-29T08:18:12.927576+0000","last_became_peered":"2024-03-29T08:18:12.927576+0000","last_unstale":"2024-03-29T08:18:14.323462+0000","last_undegraded":"2024-03-29T08:18:14.323462+0000","last_fullsized":"2024-03-29T08:18:14.323462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:58.083723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102785+0000","last_change":"2024-03-29T08:18:13.344540+0000","last_active":"2024-03-29T08:18:15.102785+0000","last_peered":"2024-03-29T08:18:15.102785+0000","last_clean":"2024-03-29T08:18:15.102785+0000","last_became_active":"2024-03-29T08:18:13.343172+0000","last_became_peered":"2024-03-29T08:18:13.343172+0000","last_unstale":"2024-03-29T08:18:15.102785+0000","last_undegraded":"2024-03-29T08:18:15.102785+0000","last_fullsized":"2024-03-29T08:18:15.102785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:15:47.486329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106413+0000","last_change":"2024-03-29T08:18:12.929722+0000","last_active":"2024-03-29T08:18:15.106413+0000","last_peered":"2024-03-29T08:18:15.106413+0000","last_clean":"2024-03-29T08:18:15.106413+0000","last_became_active":"2024-03-29T08:18:12.929401+0000","last_became_peered":"2024-03-29T08:18:12.929401+0000","last_unstale":"2024-03-29T08:18:15.106413+0000","last_undegraded":"2024-03-29T08:18:15.106413+0000","last_fullsized":"2024-03-29T08:18:15.106413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:44.468521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106446+0000","last_change":"2024-03-29T08:18:13.331364+0000","last_active":"2024-03-29T08:18:15.106446+0000","last_peered":"2024-03-29T08:18:15.106446+0000","last_clean":"2024-03-29T08:18:15.106446+0000","last_became_active":"2024-03-29T08:18:13.331097+0000","last_became_peered":"2024-03-29T08:18:13.331097+0000","last_unstale":"2024-03-29T08:18:15.106446+0000","last_undegraded":"2024-03-29T08:18:15.106446+0000","last_fullsized":"2024-03-29T08:18:15.106446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:22:49.782486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102069+0000","last_change":"2024-03-29T08:18:12.935254+0000","last_active":"2024-03-29T08:18:15.102069+0000","last_peered":"2024-03-29T08:18:15.102069+0000","last_clean":"2024-03-29T08:18:15.102069+0000","last_became_active":"2024-03-29T08:18:12.934862+0000","last_became_peered":"2024-03-29T08:18:12.934862+0000","last_unstale":"2024-03-29T08:18:15.102069+0000","last_undegraded":"2024-03-29T08:18:15.102069+0000","last_fullsized":"2024-03-29T08:18:15.102069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:24.943478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321225+0000","last_change":"2024-03-29T08:18:13.338357+0000","last_active":"2024-03-29T08:18:14.321225+0000","last_peered":"2024-03-29T08:18:14.321225+0000","last_clean":"2024-03-29T08:18:14.321225+0000","last_became_active":"2024-03-29T08:18:13.338109+0000","last_became_peered":"2024-03-29T08:18:13.338109+0000","last_unstale":"2024-03-29T08:18:14.321225+0000","last_undegraded":"2024-03-29T08:18:14.321225+0000","last_fullsized":"2024-03-29T08:18:14.321225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:55:41.020683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324252+0000","last_change":"2024-03-29T08:18:12.914431+0000","last_active":"2024-03-29T08:18:14.324252+0000","last_peered":"2024-03-29T08:18:14.324252+0000","last_clean":"2024-03-29T08:18:14.324252+0000","last_became_active":"2024-03-29T08:18:12.913939+0000","last_became_peered":"2024-03-29T08:18:12.913939+0000","last_unstale":"2024-03-29T08:18:14.324252+0000","last_undegraded":"2024-03-29T08:18:14.324252+0000","last_fullsized":"2024-03-29T08:18:14.324252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:16:38.127431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106300+0000","last_change":"2024-03-29T08:18:13.351406+0000","last_active":"2024-03-29T08:18:15.106300+0000","last_peered":"2024-03-29T08:18:15.106300+0000","last_clean":"2024-03-29T08:18:15.106300+0000","last_became_active":"2024-03-29T08:18:13.351143+0000","last_became_peered":"2024-03-29T08:18:13.351143+0000","last_unstale":"2024-03-29T08:18:15.106300+0000","last_undegraded":"2024-03-29T08:18:15.106300+0000","last_fullsized":"2024-03-29T08:18:15.106300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:33.632435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102249+0000","last_change":"2024-03-29T08:18:12.936118+0000","last_active":"2024-03-29T08:18:15.102249+0000","last_peered":"2024-03-29T08:18:15.102249+0000","last_clean":"2024-03-29T08:18:15.102249+0000","last_became_active":"2024-03-29T08:18:12.935861+0000","last_became_peered":"2024-03-29T08:18:12.935861+0000","last_unstale":"2024-03-29T08:18:15.102249+0000","last_undegraded":"2024-03-29T08:18:15.102249+0000","last_fullsized":"2024-03-29T08:18:15.102249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:44.468521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089321+0000","last_change":"2024-03-29T08:18:13.341154+0000","last_active":"2024-03-29T08:18:15.089321+0000","last_peered":"2024-03-29T08:18:15.089321+0000","last_clean":"2024-03-29T08:18:15.089321+0000","last_became_active":"2024-03-29T08:18:13.335461+0000","last_became_peered":"2024-03-29T08:18:13.335461+0000","last_unstale":"2024-03-29T08:18:15.089321+0000","last_undegraded":"2024-03-29T08:18:15.089321+0000","last_fullsized":"2024-03-29T08:18:15.089321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:55:41.020683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324529+0000","last_change":"2024-03-29T08:18:12.928629+0000","last_active":"2024-03-29T08:18:14.324529+0000","last_peered":"2024-03-29T08:18:14.324529+0000","last_clean":"2024-03-29T08:18:14.324529+0000","last_became_active":"2024-03-29T08:18:12.928405+0000","last_became_peered":"2024-03-29T08:18:12.928405+0000","last_unstale":"2024-03-29T08:18:14.324529+0000","last_undegraded":"2024-03-29T08:18:14.324529+0000","last_fullsized":"2024-03-29T08:18:14.324529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:06.406132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102237+0000","last_change":"2024-03-29T08:18:13.325885+0000","last_active":"2024-03-29T08:18:15.102237+0000","last_peered":"2024-03-29T08:18:15.102237+0000","last_clean":"2024-03-29T08:18:15.102237+0000","last_became_active":"2024-03-29T08:18:13.325621+0000","last_became_peered":"2024-03-29T08:18:13.325621+0000","last_unstale":"2024-03-29T08:18:15.102237+0000","last_undegraded":"2024-03-29T08:18:15.102237+0000","last_fullsized":"2024-03-29T08:18:15.102237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:27:31.782313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097253+0000","last_change":"2024-03-29T08:18:12.932348+0000","last_active":"2024-03-29T08:18:14.097253+0000","last_peered":"2024-03-29T08:18:14.097253+0000","last_clean":"2024-03-29T08:18:14.097253+0000","last_became_active":"2024-03-29T08:18:12.931959+0000","last_became_peered":"2024-03-29T08:18:12.931959+0000","last_unstale":"2024-03-29T08:18:14.097253+0000","last_undegraded":"2024-03-29T08:18:14.097253+0000","last_fullsized":"2024-03-29T08:18:14.097253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:04:56.043379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102138+0000","last_change":"2024-03-29T08:18:13.339774+0000","last_active":"2024-03-29T08:18:15.102138+0000","last_peered":"2024-03-29T08:18:15.102138+0000","last_clean":"2024-03-29T08:18:15.102138+0000","last_became_active":"2024-03-29T08:18:13.339154+0000","last_became_peered":"2024-03-29T08:18:13.339154+0000","last_unstale":"2024-03-29T08:18:15.102138+0000","last_undegraded":"2024-03-29T08:18:15.102138+0000","last_fullsized":"2024-03-29T08:18:15.102138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:10.712191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102878+0000","last_change":"2024-03-29T08:18:12.924839+0000","last_active":"2024-03-29T08:18:15.102878+0000","last_peered":"2024-03-29T08:18:15.102878+0000","last_clean":"2024-03-29T08:18:15.102878+0000","last_became_active":"2024-03-29T08:18:12.924535+0000","last_became_peered":"2024-03-29T08:18:12.924535+0000","last_unstale":"2024-03-29T08:18:15.102878+0000","last_undegraded":"2024-03-29T08:18:15.102878+0000","last_fullsized":"2024-03-29T08:18:15.102878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:58.083723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106508+0000","last_change":"2024-03-29T08:18:13.328934+0000","last_active":"2024-03-29T08:18:15.106508+0000","last_peered":"2024-03-29T08:18:15.106508+0000","last_clean":"2024-03-29T08:18:15.106508+0000","last_became_active":"2024-03-29T08:18:13.328410+0000","last_became_peered":"2024-03-29T08:18:13.328410+0000","last_unstale":"2024-03-29T08:18:15.106508+0000","last_undegraded":"2024-03-29T08:18:15.106508+0000","last_fullsized":"2024-03-29T08:18:15.106508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.345206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.320590+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:18:13.320590+0000","last_peered":"2024-03-29T08:18:13.320590+0000","last_clean":"2024-03-29T08:18:13.320590+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:18:13.320590+0000","last_undegraded":"2024-03-29T08:18:13.320590+0000","last_fullsized":"2024-03-29T08:18:13.320590+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106466+0000","last_change":"2024-03-29T08:18:12.931858+0000","last_active":"2024-03-29T08:18:15.106466+0000","last_peered":"2024-03-29T08:18:15.106466+0000","last_clean":"2024-03-29T08:18:15.106466+0000","last_became_active":"2024-03-29T08:18:12.931277+0000","last_became_peered":"2024-03-29T08:18:12.931277+0000","last_unstale":"2024-03-29T08:18:15.106466+0000","last_undegraded":"2024-03-29T08:18:15.106466+0000","last_fullsized":"2024-03-29T08:18:15.106466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:29.618576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324316+0000","last_change":"2024-03-29T08:18:13.344361+0000","last_active":"2024-03-29T08:18:14.324316+0000","last_peered":"2024-03-29T08:18:14.324316+0000","last_clean":"2024-03-29T08:18:14.324316+0000","last_became_active":"2024-03-29T08:18:13.344050+0000","last_became_peered":"2024-03-29T08:18:13.344050+0000","last_unstale":"2024-03-29T08:18:14.324316+0000","last_undegraded":"2024-03-29T08:18:14.324316+0000","last_fullsized":"2024-03-29T08:18:14.324316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:42:28.266165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106290+0000","last_change":"2024-03-29T08:18:12.938041+0000","last_active":"2024-03-29T08:18:15.106290+0000","last_peered":"2024-03-29T08:18:15.106290+0000","last_clean":"2024-03-29T08:18:15.106290+0000","last_became_active":"2024-03-29T08:18:12.937061+0000","last_became_peered":"2024-03-29T08:18:12.937061+0000","last_unstale":"2024-03-29T08:18:15.106290+0000","last_undegraded":"2024-03-29T08:18:15.106290+0000","last_fullsized":"2024-03-29T08:18:15.106290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:55:40.013948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102719+0000","last_change":"2024-03-29T08:18:13.355981+0000","last_active":"2024-03-29T08:18:15.102719+0000","last_peered":"2024-03-29T08:18:15.102719+0000","last_clean":"2024-03-29T08:18:15.102719+0000","last_became_active":"2024-03-29T08:18:13.355691+0000","last_became_peered":"2024-03-29T08:18:13.355691+0000","last_unstale":"2024-03-29T08:18:15.102719+0000","last_undegraded":"2024-03-29T08:18:15.102719+0000","last_fullsized":"2024-03-29T08:18:15.102719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:18:34.795188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323242+0000","last_change":"2024-03-29T08:18:12.931240+0000","last_active":"2024-03-29T08:18:14.323242+0000","last_peered":"2024-03-29T08:18:14.323242+0000","last_clean":"2024-03-29T08:18:14.323242+0000","last_became_active":"2024-03-29T08:18:12.930854+0000","last_became_peered":"2024-03-29T08:18:12.930854+0000","last_unstale":"2024-03-29T08:18:14.323242+0000","last_undegraded":"2024-03-29T08:18:14.323242+0000","last_fullsized":"2024-03-29T08:18:14.323242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:55:09.380677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097142+0000","last_change":"2024-03-29T08:18:13.343562+0000","last_active":"2024-03-29T08:18:14.097142+0000","last_peered":"2024-03-29T08:18:14.097142+0000","last_clean":"2024-03-29T08:18:14.097142+0000","last_became_active":"2024-03-29T08:18:13.342138+0000","last_became_peered":"2024-03-29T08:18:13.342138+0000","last_unstale":"2024-03-29T08:18:14.097142+0000","last_undegraded":"2024-03-29T08:18:14.097142+0000","last_fullsized":"2024-03-29T08:18:14.097142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:24:46.843649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106422+0000","last_change":"2024-03-29T08:18:12.924976+0000","last_active":"2024-03-29T08:18:15.106422+0000","last_peered":"2024-03-29T08:18:15.106422+0000","last_clean":"2024-03-29T08:18:15.106422+0000","last_became_active":"2024-03-29T08:18:12.924677+0000","last_became_peered":"2024-03-29T08:18:12.924677+0000","last_unstale":"2024-03-29T08:18:15.106422+0000","last_undegraded":"2024-03-29T08:18:15.106422+0000","last_fullsized":"2024-03-29T08:18:15.106422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:18:33.788453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102094+0000","last_change":"2024-03-29T08:18:13.335351+0000","last_active":"2024-03-29T08:18:15.102094+0000","last_peered":"2024-03-29T08:18:15.102094+0000","last_clean":"2024-03-29T08:18:15.102094+0000","last_became_active":"2024-03-29T08:18:13.334286+0000","last_became_peered":"2024-03-29T08:18:13.334286+0000","last_unstale":"2024-03-29T08:18:15.102094+0000","last_undegraded":"2024-03-29T08:18:15.102094+0000","last_fullsized":"2024-03-29T08:18:15.102094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:47:40.547759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097210+0000","last_change":"2024-03-29T08:18:12.930589+0000","last_active":"2024-03-29T08:18:14.097210+0000","last_peered":"2024-03-29T08:18:14.097210+0000","last_clean":"2024-03-29T08:18:14.097210+0000","last_became_active":"2024-03-29T08:18:12.930322+0000","last_became_peered":"2024-03-29T08:18:12.930322+0000","last_unstale":"2024-03-29T08:18:14.097210+0000","last_undegraded":"2024-03-29T08:18:14.097210+0000","last_fullsized":"2024-03-29T08:18:14.097210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:38:37.939078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089112+0000","last_change":"2024-03-29T08:18:13.357420+0000","last_active":"2024-03-29T08:18:15.089112+0000","last_peered":"2024-03-29T08:18:15.089112+0000","last_clean":"2024-03-29T08:18:15.089112+0000","last_became_active":"2024-03-29T08:18:13.357130+0000","last_became_peered":"2024-03-29T08:18:13.357130+0000","last_unstale":"2024-03-29T08:18:15.089112+0000","last_undegraded":"2024-03-29T08:18:15.089112+0000","last_fullsized":"2024-03-29T08:18:15.089112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:07.412867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106538+0000","last_change":"2024-03-29T08:18:12.938320+0000","last_active":"2024-03-29T08:18:15.106538+0000","last_peered":"2024-03-29T08:18:15.106538+0000","last_clean":"2024-03-29T08:18:15.106538+0000","last_became_active":"2024-03-29T08:18:12.937654+0000","last_became_peered":"2024-03-29T08:18:12.937654+0000","last_unstale":"2024-03-29T08:18:15.106538+0000","last_undegraded":"2024-03-29T08:18:15.106538+0000","last_fullsized":"2024-03-29T08:18:15.106538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:06.406132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102857+0000","last_change":"2024-03-29T08:18:13.357591+0000","last_active":"2024-03-29T08:18:15.102857+0000","last_peered":"2024-03-29T08:18:15.102857+0000","last_clean":"2024-03-29T08:18:15.102857+0000","last_became_active":"2024-03-29T08:18:13.357160+0000","last_became_peered":"2024-03-29T08:18:13.357160+0000","last_unstale":"2024-03-29T08:18:15.102857+0000","last_undegraded":"2024-03-29T08:18:15.102857+0000","last_fullsized":"2024-03-29T08:18:15.102857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:45:52.608559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.103103+0000","last_change":"2024-03-29T08:18:12.926261+0000","last_active":"2024-03-29T08:18:15.103103+0000","last_peered":"2024-03-29T08:18:15.103103+0000","last_clean":"2024-03-29T08:18:15.103103+0000","last_became_active":"2024-03-29T08:18:12.925938+0000","last_became_peered":"2024-03-29T08:18:12.925938+0000","last_unstale":"2024-03-29T08:18:15.103103+0000","last_undegraded":"2024-03-29T08:18:15.103103+0000","last_fullsized":"2024-03-29T08:18:15.103103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:56.709840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324644+0000","last_change":"2024-03-29T08:18:13.357731+0000","last_active":"2024-03-29T08:18:14.324644+0000","last_peered":"2024-03-29T08:18:14.324644+0000","last_clean":"2024-03-29T08:18:14.324644+0000","last_became_active":"2024-03-29T08:18:13.357437+0000","last_became_peered":"2024-03-29T08:18:13.357437+0000","last_unstale":"2024-03-29T08:18:14.324644+0000","last_undegraded":"2024-03-29T08:18:14.324644+0000","last_fullsized":"2024-03-29T08:18:14.324644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:10.712191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106582+0000","last_change":"2024-03-29T08:18:12.942493+0000","last_active":"2024-03-29T08:18:15.106582+0000","last_peered":"2024-03-29T08:18:15.106582+0000","last_clean":"2024-03-29T08:18:15.106582+0000","last_became_active":"2024-03-29T08:18:12.942116+0000","last_became_peered":"2024-03-29T08:18:12.942116+0000","last_unstale":"2024-03-29T08:18:15.106582+0000","last_undegraded":"2024-03-29T08:18:15.106582+0000","last_fullsized":"2024-03-29T08:18:15.106582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:55.791360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324502+0000","last_change":"2024-03-29T08:18:13.343380+0000","last_active":"2024-03-29T08:18:14.324502+0000","last_peered":"2024-03-29T08:18:14.324502+0000","last_clean":"2024-03-29T08:18:14.324502+0000","last_became_active":"2024-03-29T08:18:13.343121+0000","last_became_peered":"2024-03-29T08:18:13.343121+0000","last_unstale":"2024-03-29T08:18:14.324502+0000","last_undegraded":"2024-03-29T08:18:14.324502+0000","last_fullsized":"2024-03-29T08:18:14.324502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:52:27.461296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102035+0000","last_change":"2024-03-29T08:18:12.935438+0000","last_active":"2024-03-29T08:18:15.102035+0000","last_peered":"2024-03-29T08:18:15.102035+0000","last_clean":"2024-03-29T08:18:15.102035+0000","last_became_active":"2024-03-29T08:18:12.935170+0000","last_became_peered":"2024-03-29T08:18:12.935170+0000","last_unstale":"2024-03-29T08:18:15.102035+0000","last_undegraded":"2024-03-29T08:18:15.102035+0000","last_fullsized":"2024-03-29T08:18:15.102035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:53:56.604571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106309+0000","last_change":"2024-03-29T08:18:13.341595+0000","last_active":"2024-03-29T08:18:15.106309+0000","last_peered":"2024-03-29T08:18:15.106309+0000","last_clean":"2024-03-29T08:18:15.106309+0000","last_became_active":"2024-03-29T08:18:13.341387+0000","last_became_peered":"2024-03-29T08:18:13.341387+0000","last_unstale":"2024-03-29T08:18:15.106309+0000","last_undegraded":"2024-03-29T08:18:15.106309+0000","last_fullsized":"2024-03-29T08:18:15.106309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:37:35.410925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324393+0000","last_change":"2024-03-29T08:18:12.922897+0000","last_active":"2024-03-29T08:18:14.324393+0000","last_peered":"2024-03-29T08:18:14.324393+0000","last_clean":"2024-03-29T08:18:14.324393+0000","last_became_active":"2024-03-29T08:18:12.922583+0000","last_became_peered":"2024-03-29T08:18:12.922583+0000","last_unstale":"2024-03-29T08:18:14.324393+0000","last_undegraded":"2024-03-29T08:18:14.324393+0000","last_fullsized":"2024-03-29T08:18:14.324393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:24.943478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324257+0000","last_change":"2024-03-29T08:18:13.344357+0000","last_active":"2024-03-29T08:18:14.324257+0000","last_peered":"2024-03-29T08:18:14.324257+0000","last_clean":"2024-03-29T08:18:14.324257+0000","last_became_active":"2024-03-29T08:18:13.343725+0000","last_became_peered":"2024-03-29T08:18:13.343725+0000","last_unstale":"2024-03-29T08:18:14.324257+0000","last_undegraded":"2024-03-29T08:18:14.324257+0000","last_fullsized":"2024-03-29T08:18:14.324257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:56.798095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102810+0000","last_change":"2024-03-29T08:18:12.930747+0000","last_active":"2024-03-29T08:18:15.102810+0000","last_peered":"2024-03-29T08:18:15.102810+0000","last_clean":"2024-03-29T08:18:15.102810+0000","last_became_active":"2024-03-29T08:18:12.930527+0000","last_became_peered":"2024-03-29T08:18:12.930527+0000","last_unstale":"2024-03-29T08:18:15.102810+0000","last_undegraded":"2024-03-29T08:18:15.102810+0000","last_fullsized":"2024-03-29T08:18:15.102810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:55:09.380677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323459+0000","last_change":"2024-03-29T08:18:13.359008+0000","last_active":"2024-03-29T08:18:14.323459+0000","last_peered":"2024-03-29T08:18:14.323459+0000","last_clean":"2024-03-29T08:18:14.323459+0000","last_became_active":"2024-03-29T08:18:13.358733+0000","last_became_peered":"2024-03-29T08:18:13.358733+0000","last_unstale":"2024-03-29T08:18:14.323459+0000","last_undegraded":"2024-03-29T08:18:14.323459+0000","last_fullsized":"2024-03-29T08:18:14.323459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:45.475256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106456+0000","last_change":"2024-03-29T08:18:12.932074+0000","last_active":"2024-03-29T08:18:15.106456+0000","last_peered":"2024-03-29T08:18:15.106456+0000","last_clean":"2024-03-29T08:18:15.106456+0000","last_became_active":"2024-03-29T08:18:12.931603+0000","last_became_peered":"2024-03-29T08:18:12.931603+0000","last_unstale":"2024-03-29T08:18:15.106456+0000","last_undegraded":"2024-03-29T08:18:15.106456+0000","last_fullsized":"2024-03-29T08:18:15.106456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:31.317829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106394+0000","last_change":"2024-03-29T08:18:13.331615+0000","last_active":"2024-03-29T08:18:15.106394+0000","last_peered":"2024-03-29T08:18:15.106394+0000","last_clean":"2024-03-29T08:18:15.106394+0000","last_became_active":"2024-03-29T08:18:13.331399+0000","last_became_peered":"2024-03-29T08:18:13.331399+0000","last_unstale":"2024-03-29T08:18:15.106394+0000","last_undegraded":"2024-03-29T08:18:15.106394+0000","last_fullsized":"2024-03-29T08:18:15.106394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:49.457858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106499+0000","last_change":"2024-03-29T08:18:13.339347+0000","last_active":"2024-03-29T08:18:15.106499+0000","last_peered":"2024-03-29T08:18:15.106499+0000","last_clean":"2024-03-29T08:18:15.106499+0000","last_became_active":"2024-03-29T08:18:13.338931+0000","last_became_peered":"2024-03-29T08:18:13.338931+0000","last_unstale":"2024-03-29T08:18:15.106499+0000","last_undegraded":"2024-03-29T08:18:15.106499+0000","last_fullsized":"2024-03-29T08:18:15.106499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:06.953763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102148+0000","last_change":"2024-03-29T08:18:12.930282+0000","last_active":"2024-03-29T08:18:15.102148+0000","last_peered":"2024-03-29T08:18:15.102148+0000","last_clean":"2024-03-29T08:18:15.102148+0000","last_became_active":"2024-03-29T08:18:12.929883+0000","last_became_peered":"2024-03-29T08:18:12.929883+0000","last_unstale":"2024-03-29T08:18:15.102148+0000","last_undegraded":"2024-03-29T08:18:15.102148+0000","last_fullsized":"2024-03-29T08:18:15.102148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:30:10.121158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321265+0000","last_change":"2024-03-29T08:18:13.335469+0000","last_active":"2024-03-29T08:18:14.321265+0000","last_peered":"2024-03-29T08:18:14.321265+0000","last_clean":"2024-03-29T08:18:14.321265+0000","last_became_active":"2024-03-29T08:18:13.335195+0000","last_became_peered":"2024-03-29T08:18:13.335195+0000","last_unstale":"2024-03-29T08:18:14.321265+0000","last_undegraded":"2024-03-29T08:18:14.321265+0000","last_fullsized":"2024-03-29T08:18:14.321265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:53:57.611306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102179+0000","last_change":"2024-03-29T08:18:12.922889+0000","last_active":"2024-03-29T08:18:15.102179+0000","last_peered":"2024-03-29T08:18:15.102179+0000","last_clean":"2024-03-29T08:18:15.102179+0000","last_became_active":"2024-03-29T08:18:12.921734+0000","last_became_peered":"2024-03-29T08:18:12.921734+0000","last_unstale":"2024-03-29T08:18:15.102179+0000","last_undegraded":"2024-03-29T08:18:15.102179+0000","last_fullsized":"2024-03-29T08:18:15.102179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:06.406132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097307+0000","last_change":"2024-03-29T08:18:13.341340+0000","last_active":"2024-03-29T08:18:14.097307+0000","last_peered":"2024-03-29T08:18:14.097307+0000","last_clean":"2024-03-29T08:18:14.097307+0000","last_became_active":"2024-03-29T08:18:13.341062+0000","last_became_peered":"2024-03-29T08:18:13.341062+0000","last_unstale":"2024-03-29T08:18:14.097307+0000","last_undegraded":"2024-03-29T08:18:14.097307+0000","last_fullsized":"2024-03-29T08:18:14.097307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:54:33.899398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097285+0000","last_change":"2024-03-29T08:18:12.928434+0000","last_active":"2024-03-29T08:18:14.097285+0000","last_peered":"2024-03-29T08:18:14.097285+0000","last_clean":"2024-03-29T08:18:14.097285+0000","last_became_active":"2024-03-29T08:18:12.928109+0000","last_became_peered":"2024-03-29T08:18:12.928109+0000","last_unstale":"2024-03-29T08:18:14.097285+0000","last_undegraded":"2024-03-29T08:18:14.097285+0000","last_fullsized":"2024-03-29T08:18:14.097285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:25:41.738901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097275+0000","last_change":"2024-03-29T08:18:13.337191+0000","last_active":"2024-03-29T08:18:14.097275+0000","last_peered":"2024-03-29T08:18:14.097275+0000","last_clean":"2024-03-29T08:18:14.097275+0000","last_became_active":"2024-03-29T08:18:13.336967+0000","last_became_peered":"2024-03-29T08:18:13.336967+0000","last_unstale":"2024-03-29T08:18:14.097275+0000","last_undegraded":"2024-03-29T08:18:14.097275+0000","last_fullsized":"2024-03-29T08:18:14.097275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:40.987756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324681+0000","last_change":"2024-03-29T08:18:12.937840+0000","last_active":"2024-03-29T08:18:14.324681+0000","last_peered":"2024-03-29T08:18:14.324681+0000","last_clean":"2024-03-29T08:18:14.324681+0000","last_became_active":"2024-03-29T08:18:12.937403+0000","last_became_peered":"2024-03-29T08:18:12.937403+0000","last_unstale":"2024-03-29T08:18:14.324681+0000","last_undegraded":"2024-03-29T08:18:14.324681+0000","last_fullsized":"2024-03-29T08:18:14.324681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:58.083723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":76364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]}]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":76240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]}]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":78476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1709999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.524}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.363}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3029999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":76822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]}]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":233472,"data_stored":77804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":819200,"data_stored":666678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":229376,"data_stored":76296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":823296,"data_stored":666700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:18:22.095 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:18:22.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.268+0000 7f7e66c00640 1 Processor -- start 2024-03-29T08:18:22.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.268+0000 7f7e66c00640 1 -- start start 2024-03-29T08:18:22.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.272+0000 7f7e66c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7e6015ada0 0x7f7e600ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.272+0000 7f7e66c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7e600ce950 0x7f7e6006c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.272+0000 7f7e66c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e600cb510 con 0x7f7e60144210 2024-03-29T08:18:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.272+0000 7f7e66c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e600cede0 con 0x7f7e600ce950 2024-03-29T08:18:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.272+0000 7f7e66c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e60070580 con 0x7f7e6015ada0 2024-03-29T08:18:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.272+0000 7f7e64c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7e600ce950 0x7f7e6006c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.272+0000 7f7e64c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7e600ce950 0x7f7e6006c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60132/0 (socket says 172.21.15.46:60132) 2024-03-29T08:18:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.272+0000 7f7e64c00640 1 -- 172.21.15.46:0/3725217746 learned_addr learned my addr 172.21.15.46:0/3725217746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.272+0000 7f7e57e00640 1 --2- 172.21.15.46:0/3725217746 >> v2:172.21.15.73:3300/0 conn(0x7f7e6015ada0 0x7f7e600ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.272+0000 7f7e64c00640 1 -- 172.21.15.46:0/3725217746 >> v2:172.21.15.73:3300/0 conn(0x7f7e6015ada0 msgr2=0x7f7e600ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.272+0000 7f7e64c00640 1 --2- 172.21.15.46:0/3725217746 >> v2:172.21.15.73:3300/0 conn(0x7f7e6015ada0 0x7f7e600ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.272+0000 7f7e64c00640 1 -- 172.21.15.46:0/3725217746 >> v1:172.21.15.73:6789/0 conn(0x7f7e60144210 legacy=0x7f7e600cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.272+0000 7f7e64c00640 1 -- 172.21.15.46:0/3725217746 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e60070860 con 0x7f7e600ce950 2024-03-29T08:18:22.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.272+0000 7f7e57e00640 1 --2- 172.21.15.46:0/3725217746 >> v2:172.21.15.73:3300/0 conn(0x7f7e6015ada0 0x7f7e600ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:22.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.276+0000 7f7e64c00640 1 --2- 172.21.15.46:0/3725217746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e600ce950 0x7f7e6006c950 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7e6006e410 tx=0x7f7e5801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb0f0174004f198 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:22.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.276+0000 7f7e5f400640 1 -- 172.21.15.46:0/3725217746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580a5760 con 0x7f7e600ce950 2024-03-29T08:18:22.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.276+0000 7f7e5f400640 1 -- 172.21.15.46:0/3725217746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e580a58c0 con 0x7f7e600ce950 2024-03-29T08:18:22.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.276+0000 7f7e5f400640 1 -- 172.21.15.46:0/3725217746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580a5c50 con 0x7f7e600ce950 2024-03-29T08:18:22.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.276+0000 7f7e66c00640 1 -- 172.21.15.46:0/3725217746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e600ce950 msgr2=0x7f7e6006c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:22.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.276+0000 7f7e66c00640 1 --2- 172.21.15.46:0/3725217746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e600ce950 0x7f7e6006c950 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7e6006e410 tx=0x7f7e5801c9b0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.276+0000 7f7e66c00640 1 -- 172.21.15.46:0/3725217746 shutdown_connections 2024-03-29T08:18:22.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.276+0000 7f7e66c00640 1 --2- 172.21.15.46:0/3725217746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e600ce950 0x7f7e6006c950 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.276+0000 7f7e66c00640 1 --2- 172.21.15.46:0/3725217746 >> v2:172.21.15.73:3300/0 conn(0x7f7e6015ada0 0x7f7e600ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.276+0000 7f7e66c00640 1 -- 172.21.15.46:0/3725217746 >> 172.21.15.46:0/3725217746 conn(0x7f7e60066110 msgr2=0x7f7e60069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:22.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.276+0000 7f7e66c00640 1 -- 172.21.15.46:0/3725217746 shutdown_connections 2024-03-29T08:18:22.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.276+0000 7f7e66c00640 1 -- 172.21.15.46:0/3725217746 wait complete. 2024-03-29T08:18:22.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.276+0000 7f7e66c00640 1 Processor -- start 2024-03-29T08:18:22.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.276+0000 7f7e66c00640 1 -- start start 2024-03-29T08:18:22.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e66c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e600ce950 0x7f7e600b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:22.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e66c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7e60144210 0x7f7e60142500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:22.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e66c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e6015ada0 0x7f7e601f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:22.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e66c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e60142a40 con 0x7f7e600ce950 2024-03-29T08:18:22.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e66c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e601565e0 con 0x7f7e6015ada0 2024-03-29T08:18:22.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e66c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e601f4f60 con 0x7f7e60144210 2024-03-29T08:18:22.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e5fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e600ce950 0x7f7e600b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:22.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e5fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e600ce950 0x7f7e600b2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60148/0 (socket says 172.21.15.46:60148) 2024-03-29T08:18:22.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e5fe00640 1 -- 172.21.15.46:0/409233428 learned_addr learned my addr 172.21.15.46:0/409233428 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:22.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e57e00640 1 --2- 172.21.15.46:0/409233428 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7e60144210 0x7f7e60142500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:22.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e64c00640 1 --2- 172.21.15.46:0/409233428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e6015ada0 0x7f7e601f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:22.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e5fe00640 1 -- 172.21.15.46:0/409233428 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7e60144210 msgr2=0x7f7e60142500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:22.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e5fe00640 1 --2- 172.21.15.46:0/409233428 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7e60144210 0x7f7e60142500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e5fe00640 1 -- 172.21.15.46:0/409233428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e6015ada0 msgr2=0x7f7e601f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:22.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e5fe00640 1 --2- 172.21.15.46:0/409233428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e6015ada0 0x7f7e601f4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e5fe00640 1 -- 172.21.15.46:0/409233428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e58009b30 con 0x7f7e600ce950 2024-03-29T08:18:22.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e5fe00640 1 --2- 172.21.15.46:0/409233428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e600ce950 0x7f7e600b2d20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000c450 tx=0x7f7e5000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:22.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e5cc00640 1 -- 172.21.15.46:0/409233428 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e50012830 con 0x7f7e600ce950 2024-03-29T08:18:22.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e5cc00640 1 -- 172.21.15.46:0/409233428 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e500129b0 con 0x7f7e600ce950 2024-03-29T08:18:22.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e66c00640 1 -- 172.21.15.46:0/409233428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e600b3420 con 0x7f7e600ce950 2024-03-29T08:18:22.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e5cc00640 1 -- 172.21.15.46:0/409233428 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e50012d60 con 0x7f7e600ce950 2024-03-29T08:18:22.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e66c00640 1 -- 172.21.15.46:0/409233428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e60156b10 con 0x7f7e600ce950 2024-03-29T08:18:22.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e66c00640 1 -- 172.21.15.46:0/409233428 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e600ce280 con 0x7f7e600ce950 2024-03-29T08:18:22.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.280+0000 7f7e5cc00640 1 -- 172.21.15.46:0/409233428 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7e5001c460 con 0x7f7e600ce950 2024-03-29T08:18:22.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.284+0000 7f7e5cc00640 1 --2- 172.21.15.46:0/409233428 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7e3c066690 0x7f7e3c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:22.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.284+0000 7f7e5cc00640 1 -- 172.21.15.46:0/409233428 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e50083fa0 con 0x7f7e600ce950 2024-03-29T08:18:22.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.284+0000 7f7e57e00640 1 --2- 172.21.15.46:0/409233428 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7e3c066690 0x7f7e3c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:22.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.284+0000 7f7e57e00640 1 --2- 172.21.15.46:0/409233428 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7e3c066690 0x7f7e3c068b50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7e48006fd0 tx=0x7f7e48008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:22.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.288+0000 7f7e5cc00640 1 -- 172.21.15.46:0/409233428 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7e50051300 con 0x7f7e600ce950 2024-03-29T08:18:22.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.444+0000 7f7e66c00640 1 -- 172.21.15.46:0/409233428 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7e6006d6a0 con 0x7f7e3c066690 2024-03-29T08:18:22.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.452+0000 7f7e5cc00640 1 -- 172.21.15.46:0/409233428 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264552 (secure 0 0 0) 0x7f7e4800c050 con 0x7f7e3c066690 2024-03-29T08:18:22.455 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:22.457 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:18:22.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.456+0000 7f7e66c00640 1 -- 172.21.15.46:0/409233428 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7e3c066690 msgr2=0x7f7e3c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:22.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.456+0000 7f7e66c00640 1 --2- 172.21.15.46:0/409233428 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7e3c066690 0x7f7e3c068b50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7e48006fd0 tx=0x7f7e48008040 comp rx=0 tx=0).stop 2024-03-29T08:18:22.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.456+0000 7f7e66c00640 1 -- 172.21.15.46:0/409233428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e600ce950 msgr2=0x7f7e600b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:22.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.456+0000 7f7e66c00640 1 --2- 172.21.15.46:0/409233428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e600ce950 0x7f7e600b2d20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000c450 tx=0x7f7e5000c910 comp rx=0 tx=0).stop 2024-03-29T08:18:22.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.456+0000 7f7e66c00640 1 -- 172.21.15.46:0/409233428 shutdown_connections 2024-03-29T08:18:22.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.456+0000 7f7e66c00640 1 --2- 172.21.15.46:0/409233428 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7e3c066690 0x7f7e3c068b50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.456+0000 7f7e66c00640 1 --2- 172.21.15.46:0/409233428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e6015ada0 0x7f7e601f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.456+0000 7f7e66c00640 1 --2- 172.21.15.46:0/409233428 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7e60144210 0x7f7e60142500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.456+0000 7f7e66c00640 1 --2- 172.21.15.46:0/409233428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e600ce950 0x7f7e600b2d20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.456+0000 7f7e66c00640 1 -- 172.21.15.46:0/409233428 >> 172.21.15.46:0/409233428 conn(0x7f7e60066110 msgr2=0x7f7e60069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:22.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.456+0000 7f7e66c00640 1 -- 172.21.15.46:0/409233428 shutdown_connections 2024-03-29T08:18:22.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.456+0000 7f7e66c00640 1 -- 172.21.15.46:0/409233428 wait complete. 2024-03-29T08:18:22.475 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-03-29T08:18:22.043829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235528,"kb_used_data":2952,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754739192,"statfs":{"total":773094113280,"available":772852932608,"internally_reserved":0,"allocated":3022848,"data_stored":1795380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.724786"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102211+0000","last_change":"2024-03-29T08:18:13.347398+0000","last_active":"2024-03-29T08:18:15.102211+0000","last_peered":"2024-03-29T08:18:15.102211+0000","last_clean":"2024-03-29T08:18:15.102211+0000","last_became_active":"2024-03-29T08:18:13.346857+0000","last_became_peered":"2024-03-29T08:18:13.346857+0000","last_unstale":"2024-03-29T08:18:15.102211+0000","last_undegraded":"2024-03-29T08:18:15.102211+0000","last_fullsized":"2024-03-29T08:18:15.102211+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:03.590543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097264+0000","last_change":"2024-03-29T08:18:13.358700+0000","last_active":"2024-03-29T08:18:14.097264+0000","last_peered":"2024-03-29T08:18:14.097264+0000","last_clean":"2024-03-29T08:18:14.097264+0000","last_became_active":"2024-03-29T08:18:13.357646+0000","last_became_peered":"2024-03-29T08:18:13.357646+0000","last_unstale":"2024-03-29T08:18:14.097264+0000","last_undegraded":"2024-03-29T08:18:14.097264+0000","last_fullsized":"2024-03-29T08:18:14.097264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:01:20.587134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097240+0000","last_change":"2024-03-29T08:18:13.352326+0000","last_active":"2024-03-29T08:18:14.097240+0000","last_peered":"2024-03-29T08:18:14.097240+0000","last_clean":"2024-03-29T08:18:14.097240+0000","last_became_active":"2024-03-29T08:18:13.351954+0000","last_became_peered":"2024-03-29T08:18:13.351954+0000","last_unstale":"2024-03-29T08:18:14.097240+0000","last_undegraded":"2024-03-29T08:18:14.097240+0000","last_fullsized":"2024-03-29T08:18:14.097240+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:07:42.013921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089100+0000","last_change":"2024-03-29T08:18:13.336578+0000","last_active":"2024-03-29T08:18:15.089100+0000","last_peered":"2024-03-29T08:18:15.089100+0000","last_clean":"2024-03-29T08:18:15.089100+0000","last_became_active":"2024-03-29T08:18:13.336344+0000","last_became_peered":"2024-03-29T08:18:13.336344+0000","last_unstale":"2024-03-29T08:18:15.089100+0000","last_undegraded":"2024-03-29T08:18:15.089100+0000","last_fullsized":"2024-03-29T08:18:15.089100+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:30.625311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102083+0000","last_change":"2024-03-29T08:18:13.348320+0000","last_active":"2024-03-29T08:18:15.102083+0000","last_peered":"2024-03-29T08:18:15.102083+0000","last_clean":"2024-03-29T08:18:15.102083+0000","last_became_active":"2024-03-29T08:18:13.345863+0000","last_became_peered":"2024-03-29T08:18:13.345863+0000","last_unstale":"2024-03-29T08:18:15.102083+0000","last_undegraded":"2024-03-29T08:18:15.102083+0000","last_fullsized":"2024-03-29T08:18:15.102083+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:26:51.855915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.088872+0000","last_change":"2024-03-29T08:18:13.345374+0000","last_active":"2024-03-29T08:18:15.088872+0000","last_peered":"2024-03-29T08:18:15.088872+0000","last_clean":"2024-03-29T08:18:15.088872+0000","last_became_active":"2024-03-29T08:18:13.345054+0000","last_became_peered":"2024-03-29T08:18:13.345054+0000","last_unstale":"2024-03-29T08:18:15.088872+0000","last_undegraded":"2024-03-29T08:18:15.088872+0000","last_fullsized":"2024-03-29T08:18:15.088872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:32.324564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102055+0000","last_change":"2024-03-29T08:18:13.346637+0000","last_active":"2024-03-29T08:18:15.102055+0000","last_peered":"2024-03-29T08:18:15.102055+0000","last_clean":"2024-03-29T08:18:15.102055+0000","last_became_active":"2024-03-29T08:18:13.346335+0000","last_became_peered":"2024-03-29T08:18:13.346335+0000","last_unstale":"2024-03-29T08:18:15.102055+0000","last_undegraded":"2024-03-29T08:18:15.102055+0000","last_fullsized":"2024-03-29T08:18:15.102055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:23.267817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097122+0000","last_change":"2024-03-29T08:18:13.348979+0000","last_active":"2024-03-29T08:18:14.097122+0000","last_peered":"2024-03-29T08:18:14.097122+0000","last_clean":"2024-03-29T08:18:14.097122+0000","last_became_active":"2024-03-29T08:18:13.348682+0000","last_became_peered":"2024-03-29T08:18:13.348682+0000","last_unstale":"2024-03-29T08:18:14.097122+0000","last_undegraded":"2024-03-29T08:18:14.097122+0000","last_fullsized":"2024-03-29T08:18:14.097122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:53:21.565448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321443+0000","last_change":"2024-03-29T08:18:13.339556+0000","last_active":"2024-03-29T08:18:14.321443+0000","last_peered":"2024-03-29T08:18:14.321443+0000","last_clean":"2024-03-29T08:18:14.321443+0000","last_became_active":"2024-03-29T08:18:13.339318+0000","last_became_peered":"2024-03-29T08:18:13.339318+0000","last_unstale":"2024-03-29T08:18:14.321443+0000","last_undegraded":"2024-03-29T08:18:14.321443+0000","last_fullsized":"2024-03-29T08:18:14.321443+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:45.475256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102190+0000","last_change":"2024-03-29T08:18:13.337238+0000","last_active":"2024-03-29T08:18:15.102190+0000","last_peered":"2024-03-29T08:18:15.102190+0000","last_clean":"2024-03-29T08:18:15.102190+0000","last_became_active":"2024-03-29T08:18:13.336549+0000","last_became_peered":"2024-03-29T08:18:13.336549+0000","last_unstale":"2024-03-29T08:18:15.102190+0000","last_undegraded":"2024-03-29T08:18:15.102190+0000","last_fullsized":"2024-03-29T08:18:15.102190+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:32:48.313814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102161+0000","last_change":"2024-03-29T08:18:13.346353+0000","last_active":"2024-03-29T08:18:15.102161+0000","last_peered":"2024-03-29T08:18:15.102161+0000","last_clean":"2024-03-29T08:18:15.102161+0000","last_became_active":"2024-03-29T08:18:13.345780+0000","last_became_peered":"2024-03-29T08:18:13.345780+0000","last_unstale":"2024-03-29T08:18:15.102161+0000","last_undegraded":"2024-03-29T08:18:15.102161+0000","last_fullsized":"2024-03-29T08:18:15.102161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:45:55.900052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102103+0000","last_change":"2024-03-29T08:18:13.331322+0000","last_active":"2024-03-29T08:18:15.102103+0000","last_peered":"2024-03-29T08:18:15.102103+0000","last_clean":"2024-03-29T08:18:15.102103+0000","last_became_active":"2024-03-29T08:18:13.331047+0000","last_became_peered":"2024-03-29T08:18:13.331047+0000","last_unstale":"2024-03-29T08:18:15.102103+0000","last_undegraded":"2024-03-29T08:18:15.102103+0000","last_fullsized":"2024-03-29T08:18:15.102103+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:40.888705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102826+0000","last_change":"2024-03-29T08:18:13.357625+0000","last_active":"2024-03-29T08:18:15.102826+0000","last_peered":"2024-03-29T08:18:15.102826+0000","last_clean":"2024-03-29T08:18:15.102826+0000","last_became_active":"2024-03-29T08:18:13.356826+0000","last_became_peered":"2024-03-29T08:18:13.356826+0000","last_unstale":"2024-03-29T08:18:15.102826+0000","last_undegraded":"2024-03-29T08:18:15.102826+0000","last_fullsized":"2024-03-29T08:18:15.102826+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:30.625311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102767+0000","last_change":"2024-03-29T08:18:13.356033+0000","last_active":"2024-03-29T08:18:15.102767+0000","last_peered":"2024-03-29T08:18:15.102767+0000","last_clean":"2024-03-29T08:18:15.102767+0000","last_became_active":"2024-03-29T08:18:13.355720+0000","last_became_peered":"2024-03-29T08:18:13.355720+0000","last_unstale":"2024-03-29T08:18:15.102767+0000","last_undegraded":"2024-03-29T08:18:15.102767+0000","last_fullsized":"2024-03-29T08:18:15.102767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:56.798095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097166+0000","last_change":"2024-03-29T08:18:13.349080+0000","last_active":"2024-03-29T08:18:14.097166+0000","last_peered":"2024-03-29T08:18:14.097166+0000","last_clean":"2024-03-29T08:18:14.097166+0000","last_became_active":"2024-03-29T08:18:13.348721+0000","last_became_peered":"2024-03-29T08:18:13.348721+0000","last_unstale":"2024-03-29T08:18:14.097166+0000","last_undegraded":"2024-03-29T08:18:14.097166+0000","last_fullsized":"2024-03-29T08:18:14.097166+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:28:54.605777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106254+0000","last_change":"2024-03-29T08:18:13.349631+0000","last_active":"2024-03-29T08:18:15.106254+0000","last_peered":"2024-03-29T08:18:15.106254+0000","last_clean":"2024-03-29T08:18:15.106254+0000","last_became_active":"2024-03-29T08:18:13.349377+0000","last_became_peered":"2024-03-29T08:18:13.349377+0000","last_unstale":"2024-03-29T08:18:15.106254+0000","last_undegraded":"2024-03-29T08:18:15.106254+0000","last_fullsized":"2024-03-29T08:18:15.106254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:44:22.135683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106572+0000","last_change":"2024-03-29T08:18:13.342594+0000","last_active":"2024-03-29T08:18:15.106572+0000","last_peered":"2024-03-29T08:18:15.106572+0000","last_clean":"2024-03-29T08:18:15.106572+0000","last_became_active":"2024-03-29T08:18:13.341973+0000","last_became_peered":"2024-03-29T08:18:13.341973+0000","last_unstale":"2024-03-29T08:18:15.106572+0000","last_undegraded":"2024-03-29T08:18:15.106572+0000","last_fullsized":"2024-03-29T08:18:15.106572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:31:39.118083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323494+0000","last_change":"2024-03-29T08:18:13.349000+0000","last_active":"2024-03-29T08:18:14.323494+0000","last_peered":"2024-03-29T08:18:14.323494+0000","last_clean":"2024-03-29T08:18:14.323494+0000","last_became_active":"2024-03-29T08:18:13.348586+0000","last_became_peered":"2024-03-29T08:18:13.348586+0000","last_unstale":"2024-03-29T08:18:14.323494+0000","last_undegraded":"2024-03-29T08:18:14.323494+0000","last_fullsized":"2024-03-29T08:18:14.323494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:55:41.020683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106529+0000","last_change":"2024-03-29T08:18:13.342652+0000","last_active":"2024-03-29T08:18:15.106529+0000","last_peered":"2024-03-29T08:18:15.106529+0000","last_clean":"2024-03-29T08:18:15.106529+0000","last_became_active":"2024-03-29T08:18:13.342339+0000","last_became_peered":"2024-03-29T08:18:13.342339+0000","last_unstale":"2024-03-29T08:18:15.106529+0000","last_undegraded":"2024-03-29T08:18:15.106529+0000","last_fullsized":"2024-03-29T08:18:15.106529+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:46:35.075911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102113+0000","last_change":"2024-03-29T08:18:13.325928+0000","last_active":"2024-03-29T08:18:15.102113+0000","last_peered":"2024-03-29T08:18:15.102113+0000","last_clean":"2024-03-29T08:18:15.102113+0000","last_became_active":"2024-03-29T08:18:13.325659+0000","last_became_peered":"2024-03-29T08:18:13.325659+0000","last_unstale":"2024-03-29T08:18:15.102113+0000","last_undegraded":"2024-03-29T08:18:15.102113+0000","last_fullsized":"2024-03-29T08:18:15.102113+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:58.411155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324285+0000","last_change":"2024-03-29T08:18:13.339244+0000","last_active":"2024-03-29T08:18:14.324285+0000","last_peered":"2024-03-29T08:18:14.324285+0000","last_clean":"2024-03-29T08:18:14.324285+0000","last_became_active":"2024-03-29T08:18:13.338984+0000","last_became_peered":"2024-03-29T08:18:13.338984+0000","last_unstale":"2024-03-29T08:18:14.324285+0000","last_undegraded":"2024-03-29T08:18:14.324285+0000","last_fullsized":"2024-03-29T08:18:14.324285+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:29:09.699201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321134+0000","last_change":"2024-03-29T08:18:13.330207+0000","last_active":"2024-03-29T08:18:14.321134+0000","last_peered":"2024-03-29T08:18:14.321134+0000","last_clean":"2024-03-29T08:18:14.321134+0000","last_became_active":"2024-03-29T08:18:13.330029+0000","last_became_peered":"2024-03-29T08:18:13.330029+0000","last_unstale":"2024-03-29T08:18:14.321134+0000","last_undegraded":"2024-03-29T08:18:14.321134+0000","last_fullsized":"2024-03-29T08:18:14.321134+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:32.324564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097178+0000","last_change":"2024-03-29T08:18:13.347985+0000","last_active":"2024-03-29T08:18:14.097178+0000","last_peered":"2024-03-29T08:18:14.097178+0000","last_clean":"2024-03-29T08:18:14.097178+0000","last_became_active":"2024-03-29T08:18:13.347561+0000","last_became_peered":"2024-03-29T08:18:13.347561+0000","last_unstale":"2024-03-29T08:18:14.097178+0000","last_undegraded":"2024-03-29T08:18:14.097178+0000","last_fullsized":"2024-03-29T08:18:14.097178+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:17.775287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106271+0000","last_change":"2024-03-29T08:18:13.338212+0000","last_active":"2024-03-29T08:18:15.106271+0000","last_peered":"2024-03-29T08:18:15.106271+0000","last_clean":"2024-03-29T08:18:15.106271+0000","last_became_active":"2024-03-29T08:18:13.337908+0000","last_became_peered":"2024-03-29T08:18:13.337908+0000","last_unstale":"2024-03-29T08:18:15.106271+0000","last_undegraded":"2024-03-29T08:18:15.106271+0000","last_fullsized":"2024-03-29T08:18:15.106271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:40.888705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097296+0000","last_change":"2024-03-29T08:18:13.341534+0000","last_active":"2024-03-29T08:18:14.097296+0000","last_peered":"2024-03-29T08:18:14.097296+0000","last_clean":"2024-03-29T08:18:14.097296+0000","last_became_active":"2024-03-29T08:18:13.341097+0000","last_became_peered":"2024-03-29T08:18:13.341097+0000","last_unstale":"2024-03-29T08:18:14.097296+0000","last_undegraded":"2024-03-29T08:18:14.097296+0000","last_fullsized":"2024-03-29T08:18:14.097296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:05:16.874804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089257+0000","last_change":"2024-03-29T08:18:13.342465+0000","last_active":"2024-03-29T08:18:15.089257+0000","last_peered":"2024-03-29T08:18:15.089257+0000","last_clean":"2024-03-29T08:18:15.089257+0000","last_became_active":"2024-03-29T08:18:13.342000+0000","last_became_peered":"2024-03-29T08:18:13.342000+0000","last_unstale":"2024-03-29T08:18:15.089257+0000","last_undegraded":"2024-03-29T08:18:15.089257+0000","last_fullsized":"2024-03-29T08:18:15.089257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:05.352866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102918+0000","last_change":"2024-03-29T08:18:13.356380+0000","last_active":"2024-03-29T08:18:15.102918+0000","last_peered":"2024-03-29T08:18:15.102918+0000","last_clean":"2024-03-29T08:18:15.102918+0000","last_became_active":"2024-03-29T08:18:13.356016+0000","last_became_peered":"2024-03-29T08:18:13.356016+0000","last_unstale":"2024-03-29T08:18:15.102918+0000","last_undegraded":"2024-03-29T08:18:15.102918+0000","last_fullsized":"2024-03-29T08:18:15.102918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:05.352866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102841+0000","last_change":"2024-03-29T08:18:13.357406+0000","last_active":"2024-03-29T08:18:15.102841+0000","last_peered":"2024-03-29T08:18:15.102841+0000","last_clean":"2024-03-29T08:18:15.102841+0000","last_became_active":"2024-03-29T08:18:13.356875+0000","last_became_peered":"2024-03-29T08:18:13.356875+0000","last_unstale":"2024-03-29T08:18:15.102841+0000","last_undegraded":"2024-03-29T08:18:15.102841+0000","last_fullsized":"2024-03-29T08:18:15.102841+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:52:27.461296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097189+0000","last_change":"2024-03-29T08:18:13.338239+0000","last_active":"2024-03-29T08:18:14.097189+0000","last_peered":"2024-03-29T08:18:14.097189+0000","last_clean":"2024-03-29T08:18:14.097189+0000","last_became_active":"2024-03-29T08:18:13.337901+0000","last_became_peered":"2024-03-29T08:18:13.337901+0000","last_unstale":"2024-03-29T08:18:14.097189+0000","last_undegraded":"2024-03-29T08:18:14.097189+0000","last_fullsized":"2024-03-29T08:18:14.097189+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:41:40.150525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106380+0000","last_change":"2024-03-29T08:18:13.338530+0000","last_active":"2024-03-29T08:18:15.106380+0000","last_peered":"2024-03-29T08:18:15.106380+0000","last_clean":"2024-03-29T08:18:15.106380+0000","last_became_active":"2024-03-29T08:18:13.338188+0000","last_became_peered":"2024-03-29T08:18:13.338188+0000","last_unstale":"2024-03-29T08:18:15.106380+0000","last_undegraded":"2024-03-29T08:18:15.106380+0000","last_fullsized":"2024-03-29T08:18:15.106380+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:03.590543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323230+0000","last_change":"2024-03-29T08:18:13.344948+0000","last_active":"2024-03-29T08:18:14.323230+0000","last_peered":"2024-03-29T08:18:14.323230+0000","last_clean":"2024-03-29T08:18:14.323230+0000","last_became_active":"2024-03-29T08:18:13.341442+0000","last_became_peered":"2024-03-29T08:18:13.341442+0000","last_unstale":"2024-03-29T08:18:14.323230+0000","last_undegraded":"2024-03-29T08:18:14.323230+0000","last_fullsized":"2024-03-29T08:18:14.323230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:53:57.611306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.088799+0000","last_change":"2024-03-29T08:18:13.356080+0000","last_active":"2024-03-29T08:18:15.088799+0000","last_peered":"2024-03-29T08:18:15.088799+0000","last_clean":"2024-03-29T08:18:15.088799+0000","last_became_active":"2024-03-29T08:18:13.355885+0000","last_became_peered":"2024-03-29T08:18:13.355885+0000","last_unstale":"2024-03-29T08:18:15.088799+0000","last_undegraded":"2024-03-29T08:18:15.088799+0000","last_fullsized":"2024-03-29T08:18:15.088799+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:57.716575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321203+0000","last_change":"2024-03-29T08:18:13.343561+0000","last_active":"2024-03-29T08:18:14.321203+0000","last_peered":"2024-03-29T08:18:14.321203+0000","last_clean":"2024-03-29T08:18:14.321203+0000","last_became_active":"2024-03-29T08:18:13.341129+0000","last_became_peered":"2024-03-29T08:18:13.341129+0000","last_unstale":"2024-03-29T08:18:14.321203+0000","last_undegraded":"2024-03-29T08:18:14.321203+0000","last_fullsized":"2024-03-29T08:18:14.321203+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:30.625311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097200+0000","last_change":"2024-03-29T08:18:12.928717+0000","last_active":"2024-03-29T08:18:14.097200+0000","last_peered":"2024-03-29T08:18:14.097200+0000","last_clean":"2024-03-29T08:18:14.097200+0000","last_became_active":"2024-03-29T08:18:12.928427+0000","last_became_peered":"2024-03-29T08:18:12.928427+0000","last_unstale":"2024-03-29T08:18:14.097200+0000","last_undegraded":"2024-03-29T08:18:14.097200+0000","last_fullsized":"2024-03-29T08:18:14.097200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:28:47.778848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324418+0000","last_change":"2024-03-29T08:18:13.346417+0000","last_active":"2024-03-29T08:18:14.324418+0000","last_peered":"2024-03-29T08:18:14.324418+0000","last_clean":"2024-03-29T08:18:14.324418+0000","last_became_active":"2024-03-29T08:18:13.344718+0000","last_became_peered":"2024-03-29T08:18:13.344718+0000","last_unstale":"2024-03-29T08:18:14.324418+0000","last_undegraded":"2024-03-29T08:18:14.324418+0000","last_fullsized":"2024-03-29T08:18:14.324418+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:24:34.643975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106404+0000","last_change":"2024-03-29T08:18:12.930498+0000","last_active":"2024-03-29T08:18:15.106404+0000","last_peered":"2024-03-29T08:18:15.106404+0000","last_clean":"2024-03-29T08:18:15.106404+0000","last_became_active":"2024-03-29T08:18:12.930232+0000","last_became_peered":"2024-03-29T08:18:12.930232+0000","last_unstale":"2024-03-29T08:18:15.106404+0000","last_undegraded":"2024-03-29T08:18:15.106404+0000","last_fullsized":"2024-03-29T08:18:15.106404+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:04.346131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102747+0000","last_change":"2024-03-29T08:18:12.927991+0000","last_active":"2024-03-29T08:18:15.102747+0000","last_peered":"2024-03-29T08:18:15.102747+0000","last_clean":"2024-03-29T08:18:15.102747+0000","last_became_active":"2024-03-29T08:18:12.927288+0000","last_became_peered":"2024-03-29T08:18:12.927288+0000","last_unstale":"2024-03-29T08:18:15.102747+0000","last_undegraded":"2024-03-29T08:18:15.102747+0000","last_fullsized":"2024-03-29T08:18:15.102747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:30:10.121158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097153+0000","last_change":"2024-03-29T08:18:13.358303+0000","last_active":"2024-03-29T08:18:14.097153+0000","last_peered":"2024-03-29T08:18:14.097153+0000","last_clean":"2024-03-29T08:18:14.097153+0000","last_became_active":"2024-03-29T08:18:13.358041+0000","last_became_peered":"2024-03-29T08:18:13.358041+0000","last_unstale":"2024-03-29T08:18:14.097153+0000","last_undegraded":"2024-03-29T08:18:14.097153+0000","last_fullsized":"2024-03-29T08:18:14.097153+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:41:41.207988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106281+0000","last_change":"2024-03-29T08:18:12.927896+0000","last_active":"2024-03-29T08:18:15.106281+0000","last_peered":"2024-03-29T08:18:15.106281+0000","last_clean":"2024-03-29T08:18:15.106281+0000","last_became_active":"2024-03-29T08:18:12.927399+0000","last_became_peered":"2024-03-29T08:18:12.927399+0000","last_unstale":"2024-03-29T08:18:15.106281+0000","last_undegraded":"2024-03-29T08:18:15.106281+0000","last_fullsized":"2024-03-29T08:18:15.106281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:56.709840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324362+0000","last_change":"2024-03-29T08:18:13.338175+0000","last_active":"2024-03-29T08:18:14.324362+0000","last_peered":"2024-03-29T08:18:14.324362+0000","last_clean":"2024-03-29T08:18:14.324362+0000","last_became_active":"2024-03-29T08:18:13.337775+0000","last_became_peered":"2024-03-29T08:18:13.337775+0000","last_unstale":"2024-03-29T08:18:14.324362+0000","last_undegraded":"2024-03-29T08:18:14.324362+0000","last_fullsized":"2024-03-29T08:18:14.324362+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:45:52.608559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321469+0000","last_change":"2024-03-29T08:18:12.940829+0000","last_active":"2024-03-29T08:18:14.321469+0000","last_peered":"2024-03-29T08:18:14.321469+0000","last_clean":"2024-03-29T08:18:14.321469+0000","last_became_active":"2024-03-29T08:18:12.929330+0000","last_became_peered":"2024-03-29T08:18:12.929330+0000","last_unstale":"2024-03-29T08:18:14.321469+0000","last_undegraded":"2024-03-29T08:18:14.321469+0000","last_fullsized":"2024-03-29T08:18:14.321469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:55:09.380677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089346+0000","last_change":"2024-03-29T08:18:13.342303+0000","last_active":"2024-03-29T08:18:15.089346+0000","last_peered":"2024-03-29T08:18:15.089346+0000","last_clean":"2024-03-29T08:18:15.089346+0000","last_became_active":"2024-03-29T08:18:13.341642+0000","last_became_peered":"2024-03-29T08:18:13.341642+0000","last_unstale":"2024-03-29T08:18:15.089346+0000","last_undegraded":"2024-03-29T08:18:15.089346+0000","last_fullsized":"2024-03-29T08:18:15.089346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:45.475256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102200+0000","last_change":"2024-03-29T08:18:12.930755+0000","last_active":"2024-03-29T08:18:15.102200+0000","last_peered":"2024-03-29T08:18:15.102200+0000","last_clean":"2024-03-29T08:18:15.102200+0000","last_became_active":"2024-03-29T08:18:12.929647+0000","last_became_peered":"2024-03-29T08:18:12.929647+0000","last_unstale":"2024-03-29T08:18:15.102200+0000","last_undegraded":"2024-03-29T08:18:15.102200+0000","last_fullsized":"2024-03-29T08:18:15.102200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:16:38.127431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097317+0000","last_change":"2024-03-29T08:18:13.344231+0000","last_active":"2024-03-29T08:18:14.097317+0000","last_peered":"2024-03-29T08:18:14.097317+0000","last_clean":"2024-03-29T08:18:14.097317+0000","last_became_active":"2024-03-29T08:18:13.343788+0000","last_became_peered":"2024-03-29T08:18:13.343788+0000","last_unstale":"2024-03-29T08:18:14.097317+0000","last_undegraded":"2024-03-29T08:18:14.097317+0000","last_fullsized":"2024-03-29T08:18:14.097317+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:15:15.589437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323478+0000","last_change":"2024-03-29T08:18:12.925532+0000","last_active":"2024-03-29T08:18:14.323478+0000","last_peered":"2024-03-29T08:18:14.323478+0000","last_clean":"2024-03-29T08:18:14.323478+0000","last_became_active":"2024-03-29T08:18:12.925228+0000","last_became_peered":"2024-03-29T08:18:12.925228+0000","last_unstale":"2024-03-29T08:18:14.323478+0000","last_undegraded":"2024-03-29T08:18:14.323478+0000","last_fullsized":"2024-03-29T08:18:14.323478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:30:10.121158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106489+0000","last_change":"2024-03-29T08:18:13.339687+0000","last_active":"2024-03-29T08:18:15.106489+0000","last_peered":"2024-03-29T08:18:15.106489+0000","last_clean":"2024-03-29T08:18:15.106489+0000","last_became_active":"2024-03-29T08:18:13.339445+0000","last_became_peered":"2024-03-29T08:18:13.339445+0000","last_unstale":"2024-03-29T08:18:15.106489+0000","last_undegraded":"2024-03-29T08:18:15.106489+0000","last_fullsized":"2024-03-29T08:18:15.106489+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:45:55.900052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089195+0000","last_change":"2024-03-29T08:18:12.927677+0000","last_active":"2024-03-29T08:18:15.089195+0000","last_peered":"2024-03-29T08:18:15.089195+0000","last_clean":"2024-03-29T08:18:15.089195+0000","last_became_active":"2024-03-29T08:18:12.927416+0000","last_became_peered":"2024-03-29T08:18:12.927416+0000","last_unstale":"2024-03-29T08:18:15.089195+0000","last_undegraded":"2024-03-29T08:18:15.089195+0000","last_fullsized":"2024-03-29T08:18:15.089195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:47.035098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102170+0000","last_change":"2024-03-29T08:18:13.357432+0000","last_active":"2024-03-29T08:18:15.102170+0000","last_peered":"2024-03-29T08:18:15.102170+0000","last_clean":"2024-03-29T08:18:15.102170+0000","last_became_active":"2024-03-29T08:18:13.357221+0000","last_became_peered":"2024-03-29T08:18:13.357221+0000","last_unstale":"2024-03-29T08:18:15.102170+0000","last_undegraded":"2024-03-29T08:18:15.102170+0000","last_fullsized":"2024-03-29T08:18:15.102170+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:49.457858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323462+0000","last_change":"2024-03-29T08:18:12.930512+0000","last_active":"2024-03-29T08:18:14.323462+0000","last_peered":"2024-03-29T08:18:14.323462+0000","last_clean":"2024-03-29T08:18:14.323462+0000","last_became_active":"2024-03-29T08:18:12.927576+0000","last_became_peered":"2024-03-29T08:18:12.927576+0000","last_unstale":"2024-03-29T08:18:14.323462+0000","last_undegraded":"2024-03-29T08:18:14.323462+0000","last_fullsized":"2024-03-29T08:18:14.323462+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:58.083723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102785+0000","last_change":"2024-03-29T08:18:13.344540+0000","last_active":"2024-03-29T08:18:15.102785+0000","last_peered":"2024-03-29T08:18:15.102785+0000","last_clean":"2024-03-29T08:18:15.102785+0000","last_became_active":"2024-03-29T08:18:13.343172+0000","last_became_peered":"2024-03-29T08:18:13.343172+0000","last_unstale":"2024-03-29T08:18:15.102785+0000","last_undegraded":"2024-03-29T08:18:15.102785+0000","last_fullsized":"2024-03-29T08:18:15.102785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:15:47.486329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106413+0000","last_change":"2024-03-29T08:18:12.929722+0000","last_active":"2024-03-29T08:18:15.106413+0000","last_peered":"2024-03-29T08:18:15.106413+0000","last_clean":"2024-03-29T08:18:15.106413+0000","last_became_active":"2024-03-29T08:18:12.929401+0000","last_became_peered":"2024-03-29T08:18:12.929401+0000","last_unstale":"2024-03-29T08:18:15.106413+0000","last_undegraded":"2024-03-29T08:18:15.106413+0000","last_fullsized":"2024-03-29T08:18:15.106413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:44.468521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106446+0000","last_change":"2024-03-29T08:18:13.331364+0000","last_active":"2024-03-29T08:18:15.106446+0000","last_peered":"2024-03-29T08:18:15.106446+0000","last_clean":"2024-03-29T08:18:15.106446+0000","last_became_active":"2024-03-29T08:18:13.331097+0000","last_became_peered":"2024-03-29T08:18:13.331097+0000","last_unstale":"2024-03-29T08:18:15.106446+0000","last_undegraded":"2024-03-29T08:18:15.106446+0000","last_fullsized":"2024-03-29T08:18:15.106446+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:22:49.782486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102069+0000","last_change":"2024-03-29T08:18:12.935254+0000","last_active":"2024-03-29T08:18:15.102069+0000","last_peered":"2024-03-29T08:18:15.102069+0000","last_clean":"2024-03-29T08:18:15.102069+0000","last_became_active":"2024-03-29T08:18:12.934862+0000","last_became_peered":"2024-03-29T08:18:12.934862+0000","last_unstale":"2024-03-29T08:18:15.102069+0000","last_undegraded":"2024-03-29T08:18:15.102069+0000","last_fullsized":"2024-03-29T08:18:15.102069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:24.943478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321225+0000","last_change":"2024-03-29T08:18:13.338357+0000","last_active":"2024-03-29T08:18:14.321225+0000","last_peered":"2024-03-29T08:18:14.321225+0000","last_clean":"2024-03-29T08:18:14.321225+0000","last_became_active":"2024-03-29T08:18:13.338109+0000","last_became_peered":"2024-03-29T08:18:13.338109+0000","last_unstale":"2024-03-29T08:18:14.321225+0000","last_undegraded":"2024-03-29T08:18:14.321225+0000","last_fullsized":"2024-03-29T08:18:14.321225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:55:41.020683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324252+0000","last_change":"2024-03-29T08:18:12.914431+0000","last_active":"2024-03-29T08:18:14.324252+0000","last_peered":"2024-03-29T08:18:14.324252+0000","last_clean":"2024-03-29T08:18:14.324252+0000","last_became_active":"2024-03-29T08:18:12.913939+0000","last_became_peered":"2024-03-29T08:18:12.913939+0000","last_unstale":"2024-03-29T08:18:14.324252+0000","last_undegraded":"2024-03-29T08:18:14.324252+0000","last_fullsized":"2024-03-29T08:18:14.324252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:16:38.127431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106300+0000","last_change":"2024-03-29T08:18:13.351406+0000","last_active":"2024-03-29T08:18:15.106300+0000","last_peered":"2024-03-29T08:18:15.106300+0000","last_clean":"2024-03-29T08:18:15.106300+0000","last_became_active":"2024-03-29T08:18:13.351143+0000","last_became_peered":"2024-03-29T08:18:13.351143+0000","last_unstale":"2024-03-29T08:18:15.106300+0000","last_undegraded":"2024-03-29T08:18:15.106300+0000","last_fullsized":"2024-03-29T08:18:15.106300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:33.632435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102249+0000","last_change":"2024-03-29T08:18:12.936118+0000","last_active":"2024-03-29T08:18:15.102249+0000","last_peered":"2024-03-29T08:18:15.102249+0000","last_clean":"2024-03-29T08:18:15.102249+0000","last_became_active":"2024-03-29T08:18:12.935861+0000","last_became_peered":"2024-03-29T08:18:12.935861+0000","last_unstale":"2024-03-29T08:18:15.102249+0000","last_undegraded":"2024-03-29T08:18:15.102249+0000","last_fullsized":"2024-03-29T08:18:15.102249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:44.468521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089321+0000","last_change":"2024-03-29T08:18:13.341154+0000","last_active":"2024-03-29T08:18:15.089321+0000","last_peered":"2024-03-29T08:18:15.089321+0000","last_clean":"2024-03-29T08:18:15.089321+0000","last_became_active":"2024-03-29T08:18:13.335461+0000","last_became_peered":"2024-03-29T08:18:13.335461+0000","last_unstale":"2024-03-29T08:18:15.089321+0000","last_undegraded":"2024-03-29T08:18:15.089321+0000","last_fullsized":"2024-03-29T08:18:15.089321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:55:41.020683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324529+0000","last_change":"2024-03-29T08:18:12.928629+0000","last_active":"2024-03-29T08:18:14.324529+0000","last_peered":"2024-03-29T08:18:14.324529+0000","last_clean":"2024-03-29T08:18:14.324529+0000","last_became_active":"2024-03-29T08:18:12.928405+0000","last_became_peered":"2024-03-29T08:18:12.928405+0000","last_unstale":"2024-03-29T08:18:14.324529+0000","last_undegraded":"2024-03-29T08:18:14.324529+0000","last_fullsized":"2024-03-29T08:18:14.324529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:06.406132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102237+0000","last_change":"2024-03-29T08:18:13.325885+0000","last_active":"2024-03-29T08:18:15.102237+0000","last_peered":"2024-03-29T08:18:15.102237+0000","last_clean":"2024-03-29T08:18:15.102237+0000","last_became_active":"2024-03-29T08:18:13.325621+0000","last_became_peered":"2024-03-29T08:18:13.325621+0000","last_unstale":"2024-03-29T08:18:15.102237+0000","last_undegraded":"2024-03-29T08:18:15.102237+0000","last_fullsized":"2024-03-29T08:18:15.102237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:27:31.782313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097253+0000","last_change":"2024-03-29T08:18:12.932348+0000","last_active":"2024-03-29T08:18:14.097253+0000","last_peered":"2024-03-29T08:18:14.097253+0000","last_clean":"2024-03-29T08:18:14.097253+0000","last_became_active":"2024-03-29T08:18:12.931959+0000","last_became_peered":"2024-03-29T08:18:12.931959+0000","last_unstale":"2024-03-29T08:18:14.097253+0000","last_undegraded":"2024-03-29T08:18:14.097253+0000","last_fullsized":"2024-03-29T08:18:14.097253+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:04:56.043379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102138+0000","last_change":"2024-03-29T08:18:13.339774+0000","last_active":"2024-03-29T08:18:15.102138+0000","last_peered":"2024-03-29T08:18:15.102138+0000","last_clean":"2024-03-29T08:18:15.102138+0000","last_became_active":"2024-03-29T08:18:13.339154+0000","last_became_peered":"2024-03-29T08:18:13.339154+0000","last_unstale":"2024-03-29T08:18:15.102138+0000","last_undegraded":"2024-03-29T08:18:15.102138+0000","last_fullsized":"2024-03-29T08:18:15.102138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:10.712191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102878+0000","last_change":"2024-03-29T08:18:12.924839+0000","last_active":"2024-03-29T08:18:15.102878+0000","last_peered":"2024-03-29T08:18:15.102878+0000","last_clean":"2024-03-29T08:18:15.102878+0000","last_became_active":"2024-03-29T08:18:12.924535+0000","last_became_peered":"2024-03-29T08:18:12.924535+0000","last_unstale":"2024-03-29T08:18:15.102878+0000","last_undegraded":"2024-03-29T08:18:15.102878+0000","last_fullsized":"2024-03-29T08:18:15.102878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:58.083723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106508+0000","last_change":"2024-03-29T08:18:13.328934+0000","last_active":"2024-03-29T08:18:15.106508+0000","last_peered":"2024-03-29T08:18:15.106508+0000","last_clean":"2024-03-29T08:18:15.106508+0000","last_became_active":"2024-03-29T08:18:13.328410+0000","last_became_peered":"2024-03-29T08:18:13.328410+0000","last_unstale":"2024-03-29T08:18:15.106508+0000","last_undegraded":"2024-03-29T08:18:15.106508+0000","last_fullsized":"2024-03-29T08:18:15.106508+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.345206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":164,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:13.320590+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:18:13.320590+0000","last_peered":"2024-03-29T08:18:13.320590+0000","last_clean":"2024-03-29T08:18:13.320590+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:18:13.320590+0000","last_undegraded":"2024-03-29T08:18:13.320590+0000","last_fullsized":"2024-03-29T08:18:13.320590+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106466+0000","last_change":"2024-03-29T08:18:12.931858+0000","last_active":"2024-03-29T08:18:15.106466+0000","last_peered":"2024-03-29T08:18:15.106466+0000","last_clean":"2024-03-29T08:18:15.106466+0000","last_became_active":"2024-03-29T08:18:12.931277+0000","last_became_peered":"2024-03-29T08:18:12.931277+0000","last_unstale":"2024-03-29T08:18:15.106466+0000","last_undegraded":"2024-03-29T08:18:15.106466+0000","last_fullsized":"2024-03-29T08:18:15.106466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:19:29.618576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324316+0000","last_change":"2024-03-29T08:18:13.344361+0000","last_active":"2024-03-29T08:18:14.324316+0000","last_peered":"2024-03-29T08:18:14.324316+0000","last_clean":"2024-03-29T08:18:14.324316+0000","last_became_active":"2024-03-29T08:18:13.344050+0000","last_became_peered":"2024-03-29T08:18:13.344050+0000","last_unstale":"2024-03-29T08:18:14.324316+0000","last_undegraded":"2024-03-29T08:18:14.324316+0000","last_fullsized":"2024-03-29T08:18:14.324316+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:42:28.266165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106290+0000","last_change":"2024-03-29T08:18:12.938041+0000","last_active":"2024-03-29T08:18:15.106290+0000","last_peered":"2024-03-29T08:18:15.106290+0000","last_clean":"2024-03-29T08:18:15.106290+0000","last_became_active":"2024-03-29T08:18:12.937061+0000","last_became_peered":"2024-03-29T08:18:12.937061+0000","last_unstale":"2024-03-29T08:18:15.106290+0000","last_undegraded":"2024-03-29T08:18:15.106290+0000","last_fullsized":"2024-03-29T08:18:15.106290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:55:40.013948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102719+0000","last_change":"2024-03-29T08:18:13.355981+0000","last_active":"2024-03-29T08:18:15.102719+0000","last_peered":"2024-03-29T08:18:15.102719+0000","last_clean":"2024-03-29T08:18:15.102719+0000","last_became_active":"2024-03-29T08:18:13.355691+0000","last_became_peered":"2024-03-29T08:18:13.355691+0000","last_unstale":"2024-03-29T08:18:15.102719+0000","last_undegraded":"2024-03-29T08:18:15.102719+0000","last_fullsized":"2024-03-29T08:18:15.102719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:18:34.795188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323242+0000","last_change":"2024-03-29T08:18:12.931240+0000","last_active":"2024-03-29T08:18:14.323242+0000","last_peered":"2024-03-29T08:18:14.323242+0000","last_clean":"2024-03-29T08:18:14.323242+0000","last_became_active":"2024-03-29T08:18:12.930854+0000","last_became_peered":"2024-03-29T08:18:12.930854+0000","last_unstale":"2024-03-29T08:18:14.323242+0000","last_undegraded":"2024-03-29T08:18:14.323242+0000","last_fullsized":"2024-03-29T08:18:14.323242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:55:09.380677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097142+0000","last_change":"2024-03-29T08:18:13.343562+0000","last_active":"2024-03-29T08:18:14.097142+0000","last_peered":"2024-03-29T08:18:14.097142+0000","last_clean":"2024-03-29T08:18:14.097142+0000","last_became_active":"2024-03-29T08:18:13.342138+0000","last_became_peered":"2024-03-29T08:18:13.342138+0000","last_unstale":"2024-03-29T08:18:14.097142+0000","last_undegraded":"2024-03-29T08:18:14.097142+0000","last_fullsized":"2024-03-29T08:18:14.097142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:24:46.843649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106422+0000","last_change":"2024-03-29T08:18:12.924976+0000","last_active":"2024-03-29T08:18:15.106422+0000","last_peered":"2024-03-29T08:18:15.106422+0000","last_clean":"2024-03-29T08:18:15.106422+0000","last_became_active":"2024-03-29T08:18:12.924677+0000","last_became_peered":"2024-03-29T08:18:12.924677+0000","last_unstale":"2024-03-29T08:18:15.106422+0000","last_undegraded":"2024-03-29T08:18:15.106422+0000","last_fullsized":"2024-03-29T08:18:15.106422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:18:33.788453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102094+0000","last_change":"2024-03-29T08:18:13.335351+0000","last_active":"2024-03-29T08:18:15.102094+0000","last_peered":"2024-03-29T08:18:15.102094+0000","last_clean":"2024-03-29T08:18:15.102094+0000","last_became_active":"2024-03-29T08:18:13.334286+0000","last_became_peered":"2024-03-29T08:18:13.334286+0000","last_unstale":"2024-03-29T08:18:15.102094+0000","last_undegraded":"2024-03-29T08:18:15.102094+0000","last_fullsized":"2024-03-29T08:18:15.102094+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:47:40.547759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097210+0000","last_change":"2024-03-29T08:18:12.930589+0000","last_active":"2024-03-29T08:18:14.097210+0000","last_peered":"2024-03-29T08:18:14.097210+0000","last_clean":"2024-03-29T08:18:14.097210+0000","last_became_active":"2024-03-29T08:18:12.930322+0000","last_became_peered":"2024-03-29T08:18:12.930322+0000","last_unstale":"2024-03-29T08:18:14.097210+0000","last_undegraded":"2024-03-29T08:18:14.097210+0000","last_fullsized":"2024-03-29T08:18:14.097210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:38:37.939078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.089112+0000","last_change":"2024-03-29T08:18:13.357420+0000","last_active":"2024-03-29T08:18:15.089112+0000","last_peered":"2024-03-29T08:18:15.089112+0000","last_clean":"2024-03-29T08:18:15.089112+0000","last_became_active":"2024-03-29T08:18:13.357130+0000","last_became_peered":"2024-03-29T08:18:13.357130+0000","last_unstale":"2024-03-29T08:18:15.089112+0000","last_undegraded":"2024-03-29T08:18:15.089112+0000","last_fullsized":"2024-03-29T08:18:15.089112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:07.412867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106538+0000","last_change":"2024-03-29T08:18:12.938320+0000","last_active":"2024-03-29T08:18:15.106538+0000","last_peered":"2024-03-29T08:18:15.106538+0000","last_clean":"2024-03-29T08:18:15.106538+0000","last_became_active":"2024-03-29T08:18:12.937654+0000","last_became_peered":"2024-03-29T08:18:12.937654+0000","last_unstale":"2024-03-29T08:18:15.106538+0000","last_undegraded":"2024-03-29T08:18:15.106538+0000","last_fullsized":"2024-03-29T08:18:15.106538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:06.406132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102857+0000","last_change":"2024-03-29T08:18:13.357591+0000","last_active":"2024-03-29T08:18:15.102857+0000","last_peered":"2024-03-29T08:18:15.102857+0000","last_clean":"2024-03-29T08:18:15.102857+0000","last_became_active":"2024-03-29T08:18:13.357160+0000","last_became_peered":"2024-03-29T08:18:13.357160+0000","last_unstale":"2024-03-29T08:18:15.102857+0000","last_undegraded":"2024-03-29T08:18:15.102857+0000","last_fullsized":"2024-03-29T08:18:15.102857+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:45:52.608559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.103103+0000","last_change":"2024-03-29T08:18:12.926261+0000","last_active":"2024-03-29T08:18:15.103103+0000","last_peered":"2024-03-29T08:18:15.103103+0000","last_clean":"2024-03-29T08:18:15.103103+0000","last_became_active":"2024-03-29T08:18:12.925938+0000","last_became_peered":"2024-03-29T08:18:12.925938+0000","last_unstale":"2024-03-29T08:18:15.103103+0000","last_undegraded":"2024-03-29T08:18:15.103103+0000","last_fullsized":"2024-03-29T08:18:15.103103+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:56.709840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324644+0000","last_change":"2024-03-29T08:18:13.357731+0000","last_active":"2024-03-29T08:18:14.324644+0000","last_peered":"2024-03-29T08:18:14.324644+0000","last_clean":"2024-03-29T08:18:14.324644+0000","last_became_active":"2024-03-29T08:18:13.357437+0000","last_became_peered":"2024-03-29T08:18:13.357437+0000","last_unstale":"2024-03-29T08:18:14.324644+0000","last_undegraded":"2024-03-29T08:18:14.324644+0000","last_fullsized":"2024-03-29T08:18:14.324644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:10.712191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106582+0000","last_change":"2024-03-29T08:18:12.942493+0000","last_active":"2024-03-29T08:18:15.106582+0000","last_peered":"2024-03-29T08:18:15.106582+0000","last_clean":"2024-03-29T08:18:15.106582+0000","last_became_active":"2024-03-29T08:18:12.942116+0000","last_became_peered":"2024-03-29T08:18:12.942116+0000","last_unstale":"2024-03-29T08:18:15.106582+0000","last_undegraded":"2024-03-29T08:18:15.106582+0000","last_fullsized":"2024-03-29T08:18:15.106582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:55.791360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324502+0000","last_change":"2024-03-29T08:18:13.343380+0000","last_active":"2024-03-29T08:18:14.324502+0000","last_peered":"2024-03-29T08:18:14.324502+0000","last_clean":"2024-03-29T08:18:14.324502+0000","last_became_active":"2024-03-29T08:18:13.343121+0000","last_became_peered":"2024-03-29T08:18:13.343121+0000","last_unstale":"2024-03-29T08:18:14.324502+0000","last_undegraded":"2024-03-29T08:18:14.324502+0000","last_fullsized":"2024-03-29T08:18:14.324502+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:52:27.461296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102035+0000","last_change":"2024-03-29T08:18:12.935438+0000","last_active":"2024-03-29T08:18:15.102035+0000","last_peered":"2024-03-29T08:18:15.102035+0000","last_clean":"2024-03-29T08:18:15.102035+0000","last_became_active":"2024-03-29T08:18:12.935170+0000","last_became_peered":"2024-03-29T08:18:12.935170+0000","last_unstale":"2024-03-29T08:18:15.102035+0000","last_undegraded":"2024-03-29T08:18:15.102035+0000","last_fullsized":"2024-03-29T08:18:15.102035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:53:56.604571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106309+0000","last_change":"2024-03-29T08:18:13.341595+0000","last_active":"2024-03-29T08:18:15.106309+0000","last_peered":"2024-03-29T08:18:15.106309+0000","last_clean":"2024-03-29T08:18:15.106309+0000","last_became_active":"2024-03-29T08:18:13.341387+0000","last_became_peered":"2024-03-29T08:18:13.341387+0000","last_unstale":"2024-03-29T08:18:15.106309+0000","last_undegraded":"2024-03-29T08:18:15.106309+0000","last_fullsized":"2024-03-29T08:18:15.106309+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:37:35.410925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324393+0000","last_change":"2024-03-29T08:18:12.922897+0000","last_active":"2024-03-29T08:18:14.324393+0000","last_peered":"2024-03-29T08:18:14.324393+0000","last_clean":"2024-03-29T08:18:14.324393+0000","last_became_active":"2024-03-29T08:18:12.922583+0000","last_became_peered":"2024-03-29T08:18:12.922583+0000","last_unstale":"2024-03-29T08:18:14.324393+0000","last_undegraded":"2024-03-29T08:18:14.324393+0000","last_fullsized":"2024-03-29T08:18:14.324393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:24.943478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324257+0000","last_change":"2024-03-29T08:18:13.344357+0000","last_active":"2024-03-29T08:18:14.324257+0000","last_peered":"2024-03-29T08:18:14.324257+0000","last_clean":"2024-03-29T08:18:14.324257+0000","last_became_active":"2024-03-29T08:18:13.343725+0000","last_became_peered":"2024-03-29T08:18:13.343725+0000","last_unstale":"2024-03-29T08:18:14.324257+0000","last_undegraded":"2024-03-29T08:18:14.324257+0000","last_fullsized":"2024-03-29T08:18:14.324257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:31:56.798095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102810+0000","last_change":"2024-03-29T08:18:12.930747+0000","last_active":"2024-03-29T08:18:15.102810+0000","last_peered":"2024-03-29T08:18:15.102810+0000","last_clean":"2024-03-29T08:18:15.102810+0000","last_became_active":"2024-03-29T08:18:12.930527+0000","last_became_peered":"2024-03-29T08:18:12.930527+0000","last_unstale":"2024-03-29T08:18:15.102810+0000","last_undegraded":"2024-03-29T08:18:15.102810+0000","last_fullsized":"2024-03-29T08:18:15.102810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:55:09.380677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.323459+0000","last_change":"2024-03-29T08:18:13.359008+0000","last_active":"2024-03-29T08:18:14.323459+0000","last_peered":"2024-03-29T08:18:14.323459+0000","last_clean":"2024-03-29T08:18:14.323459+0000","last_became_active":"2024-03-29T08:18:13.358733+0000","last_became_peered":"2024-03-29T08:18:13.358733+0000","last_unstale":"2024-03-29T08:18:14.323459+0000","last_undegraded":"2024-03-29T08:18:14.323459+0000","last_fullsized":"2024-03-29T08:18:14.323459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:45.475256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106456+0000","last_change":"2024-03-29T08:18:12.932074+0000","last_active":"2024-03-29T08:18:15.106456+0000","last_peered":"2024-03-29T08:18:15.106456+0000","last_clean":"2024-03-29T08:18:15.106456+0000","last_became_active":"2024-03-29T08:18:12.931603+0000","last_became_peered":"2024-03-29T08:18:12.931603+0000","last_unstale":"2024-03-29T08:18:15.106456+0000","last_undegraded":"2024-03-29T08:18:15.106456+0000","last_fullsized":"2024-03-29T08:18:15.106456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:05:31.317829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106394+0000","last_change":"2024-03-29T08:18:13.331615+0000","last_active":"2024-03-29T08:18:15.106394+0000","last_peered":"2024-03-29T08:18:15.106394+0000","last_clean":"2024-03-29T08:18:15.106394+0000","last_became_active":"2024-03-29T08:18:13.331399+0000","last_became_peered":"2024-03-29T08:18:13.331399+0000","last_unstale":"2024-03-29T08:18:15.106394+0000","last_undegraded":"2024-03-29T08:18:15.106394+0000","last_fullsized":"2024-03-29T08:18:15.106394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:49.457858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.106499+0000","last_change":"2024-03-29T08:18:13.339347+0000","last_active":"2024-03-29T08:18:15.106499+0000","last_peered":"2024-03-29T08:18:15.106499+0000","last_clean":"2024-03-29T08:18:15.106499+0000","last_became_active":"2024-03-29T08:18:13.338931+0000","last_became_peered":"2024-03-29T08:18:13.338931+0000","last_unstale":"2024-03-29T08:18:15.106499+0000","last_undegraded":"2024-03-29T08:18:15.106499+0000","last_fullsized":"2024-03-29T08:18:15.106499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:06.953763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102148+0000","last_change":"2024-03-29T08:18:12.930282+0000","last_active":"2024-03-29T08:18:15.102148+0000","last_peered":"2024-03-29T08:18:15.102148+0000","last_clean":"2024-03-29T08:18:15.102148+0000","last_became_active":"2024-03-29T08:18:12.929883+0000","last_became_peered":"2024-03-29T08:18:12.929883+0000","last_unstale":"2024-03-29T08:18:15.102148+0000","last_undegraded":"2024-03-29T08:18:15.102148+0000","last_fullsized":"2024-03-29T08:18:15.102148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:30:10.121158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.321265+0000","last_change":"2024-03-29T08:18:13.335469+0000","last_active":"2024-03-29T08:18:14.321265+0000","last_peered":"2024-03-29T08:18:14.321265+0000","last_clean":"2024-03-29T08:18:14.321265+0000","last_became_active":"2024-03-29T08:18:13.335195+0000","last_became_peered":"2024-03-29T08:18:13.335195+0000","last_unstale":"2024-03-29T08:18:14.321265+0000","last_undegraded":"2024-03-29T08:18:14.321265+0000","last_fullsized":"2024-03-29T08:18:14.321265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:53:57.611306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:15.102179+0000","last_change":"2024-03-29T08:18:12.922889+0000","last_active":"2024-03-29T08:18:15.102179+0000","last_peered":"2024-03-29T08:18:15.102179+0000","last_clean":"2024-03-29T08:18:15.102179+0000","last_became_active":"2024-03-29T08:18:12.921734+0000","last_became_peered":"2024-03-29T08:18:12.921734+0000","last_unstale":"2024-03-29T08:18:15.102179+0000","last_undegraded":"2024-03-29T08:18:15.102179+0000","last_fullsized":"2024-03-29T08:18:15.102179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:06.406132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097307+0000","last_change":"2024-03-29T08:18:13.341340+0000","last_active":"2024-03-29T08:18:14.097307+0000","last_peered":"2024-03-29T08:18:14.097307+0000","last_clean":"2024-03-29T08:18:14.097307+0000","last_became_active":"2024-03-29T08:18:13.341062+0000","last_became_peered":"2024-03-29T08:18:13.341062+0000","last_unstale":"2024-03-29T08:18:14.097307+0000","last_undegraded":"2024-03-29T08:18:14.097307+0000","last_fullsized":"2024-03-29T08:18:14.097307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:54:33.899398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097285+0000","last_change":"2024-03-29T08:18:12.928434+0000","last_active":"2024-03-29T08:18:14.097285+0000","last_peered":"2024-03-29T08:18:14.097285+0000","last_clean":"2024-03-29T08:18:14.097285+0000","last_became_active":"2024-03-29T08:18:12.928109+0000","last_became_peered":"2024-03-29T08:18:12.928109+0000","last_unstale":"2024-03-29T08:18:14.097285+0000","last_undegraded":"2024-03-29T08:18:14.097285+0000","last_fullsized":"2024-03-29T08:18:14.097285+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:25:41.738901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.097275+0000","last_change":"2024-03-29T08:18:13.337191+0000","last_active":"2024-03-29T08:18:14.097275+0000","last_peered":"2024-03-29T08:18:14.097275+0000","last_clean":"2024-03-29T08:18:14.097275+0000","last_became_active":"2024-03-29T08:18:13.336967+0000","last_became_peered":"2024-03-29T08:18:13.336967+0000","last_unstale":"2024-03-29T08:18:14.097275+0000","last_undegraded":"2024-03-29T08:18:14.097275+0000","last_fullsized":"2024-03-29T08:18:14.097275+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:12.898515+0000","last_clean_scrub_stamp":"2024-03-29T08:18:12.898515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:40.987756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-03-29T08:18:14.324681+0000","last_change":"2024-03-29T08:18:12.937840+0000","last_active":"2024-03-29T08:18:14.324681+0000","last_peered":"2024-03-29T08:18:14.324681+0000","last_clean":"2024-03-29T08:18:14.324681+0000","last_became_active":"2024-03-29T08:18:12.937403+0000","last_became_peered":"2024-03-29T08:18:12.937403+0000","last_unstale":"2024-03-29T08:18:14.324681+0000","last_undegraded":"2024-03-29T08:18:14.324681+0000","last_fullsized":"2024-03-29T08:18:14.324681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:11.891780+0000","last_clean_scrub_stamp":"2024-03-29T08:18:11.891780+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:41:58.083723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":76364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]}]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":76240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]}]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":78476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1709999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.524}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.363}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3029999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":76822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.01}]}]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31352,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":233472,"data_stored":77804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82199999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]}]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31920,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339920,"statfs":{"total":96636764160,"available":96604078080,"internally_reserved":0,"allocated":819200,"data_stored":666678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.026}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]}]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":229376,"data_stored":76296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":823296,"data_stored":666700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:18:22.479 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-29T08:18:22.479 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-29T08:18:22.479 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-29T08:18:22.479 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-29T08:18:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.656+0000 7f6a66a00640 1 Processor -- start 2024-03-29T08:18:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.656+0000 7f6a66a00640 1 -- start start 2024-03-29T08:18:22.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.656+0000 7f6a66a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6a600d0660 0x7f6a600d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:22.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.656+0000 7f6a66a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6015cfb0 con 0x7f6a6015dc00 2024-03-29T08:18:22.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.656+0000 7f6a66a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a600d0f80 con 0x7f6a6006b8b0 2024-03-29T08:18:22.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.656+0000 7f6a66a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6006c100 con 0x7f6a600d0660 2024-03-29T08:18:22.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.656+0000 7f6a64a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6a6015dc00 0x7f6a600cf7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32782/0 (socket says 172.21.15.46:32782) 2024-03-29T08:18:22.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.656+0000 7f6a64a00640 1 -- 172.21.15.46:0/1689495472 learned_addr learned my addr 172.21.15.46:0/1689495472 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:22.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.656+0000 7f6a5fe00640 1 --2- 172.21.15.46:0/1689495472 >> v2:172.21.15.73:3300/0 conn(0x7f6a600d0660 0x7f6a600d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:22.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.656+0000 7f6a5ea00640 1 -- 172.21.15.46:0/1689495472 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741890080 0 0) 0x7f6a6015cfb0 con 0x7f6a6015dc00 2024-03-29T08:18:22.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.656+0000 7f6a5ea00640 1 -- 172.21.15.46:0/1689495472 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a44003530 con 0x7f6a6015dc00 2024-03-29T08:18:22.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.656+0000 7f6a5ea00640 1 -- 172.21.15.46:0/1689495472 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167816346 0 0) 0x7f6a600d0f80 con 0x7f6a6006b8b0 2024-03-29T08:18:22.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.656+0000 7f6a5ea00640 1 -- 172.21.15.46:0/1689495472 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a6015cfb0 con 0x7f6a6006b8b0 2024-03-29T08:18:22.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.656+0000 7f6a5fe00640 1 -- 172.21.15.46:0/1689495472 >> v1:172.21.15.73:6789/0 conn(0x7f6a6006b8b0 legacy=0x7f6a600cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:22.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.656+0000 7f6a5fe00640 1 -- 172.21.15.46:0/1689495472 >> v1:172.21.15.46:6789/0 conn(0x7f6a6015dc00 legacy=0x7f6a600cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:22.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.656+0000 7f6a5fe00640 1 -- 172.21.15.46:0/1689495472 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6006be20 con 0x7f6a600d0660 2024-03-29T08:18:22.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.660+0000 7f6a5fe00640 1 --2- 172.21.15.46:0/1689495472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a600d0660 0x7f6a600d0a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6a500097c0 tx=0x7f6a5001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e55c312a79e83db server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:22.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.660+0000 7f6a5ea00640 1 -- 172.21.15.46:0/1689495472 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a500ab670 con 0x7f6a600d0660 2024-03-29T08:18:22.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.660+0000 7f6a5ea00640 1 -- 172.21.15.46:0/1689495472 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a500ab7d0 con 0x7f6a600d0660 2024-03-29T08:18:22.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.660+0000 7f6a5ea00640 1 -- 172.21.15.46:0/1689495472 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a500abb40 con 0x7f6a600d0660 2024-03-29T08:18:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.660+0000 7f6a66a00640 1 -- 172.21.15.46:0/1689495472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a600d0660 msgr2=0x7f6a600d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.660+0000 7f6a66a00640 1 --2- 172.21.15.46:0/1689495472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a600d0660 0x7f6a600d0a40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6a500097c0 tx=0x7f6a5001c470 comp rx=0 tx=0).stop 2024-03-29T08:18:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.660+0000 7f6a66a00640 1 -- 172.21.15.46:0/1689495472 shutdown_connections 2024-03-29T08:18:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.660+0000 7f6a66a00640 1 --2- 172.21.15.46:0/1689495472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a600d0660 0x7f6a600d0a40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.660+0000 7f6a66a00640 1 -- 172.21.15.46:0/1689495472 >> 172.21.15.46:0/1689495472 conn(0x7f6a60089fd0 msgr2=0x7f6a6008a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:22.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.660+0000 7f6a66a00640 1 -- 172.21.15.46:0/1689495472 shutdown_connections 2024-03-29T08:18:22.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.660+0000 7f6a66a00640 1 -- 172.21.15.46:0/1689495472 wait complete. 2024-03-29T08:18:22.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.660+0000 7f6a66a00640 1 Processor -- start 2024-03-29T08:18:22.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a66a00640 1 -- start start 2024-03-29T08:18:22.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a66a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6a6006b8b0 0x7f6a600b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:22.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a66a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a600d0660 0x7f6a6006f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:22.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a66a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6015dc00 0x7f6a60155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:22.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a66a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a600b5860 con 0x7f6a600d0660 2024-03-29T08:18:22.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a66a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a600c8060 con 0x7f6a6015dc00 2024-03-29T08:18:22.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a66a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a600b5ec0 con 0x7f6a6006b8b0 2024-03-29T08:18:22.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a5f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a600d0660 0x7f6a6006f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:22.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a5f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a600d0660 0x7f6a6006f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60162/0 (socket says 172.21.15.46:60162) 2024-03-29T08:18:22.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a5f400640 1 -- 172.21.15.46:0/2005702074 learned_addr learned my addr 172.21.15.46:0/2005702074 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:22.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a64a00640 1 --2- 172.21.15.46:0/2005702074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6015dc00 0x7f6a60155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:22.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a5fe00640 1 --2- 172.21.15.46:0/2005702074 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6a6006b8b0 0x7f6a600b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:22.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a5f400640 1 -- 172.21.15.46:0/2005702074 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6a6006b8b0 msgr2=0x7f6a600b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:22.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a5f400640 1 --2- 172.21.15.46:0/2005702074 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6a6006b8b0 0x7f6a600b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a5f400640 1 -- 172.21.15.46:0/2005702074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6015dc00 msgr2=0x7f6a60155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:22.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a5f400640 1 --2- 172.21.15.46:0/2005702074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6015dc00 0x7f6a60155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a5f400640 1 -- 172.21.15.46:0/2005702074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a50009420 con 0x7f6a600d0660 2024-03-29T08:18:22.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a5f400640 1 --2- 172.21.15.46:0/2005702074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a600d0660 0x7f6a6006f150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00c500 tx=0x7f6a4c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:22.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a57e00640 1 -- 172.21.15.46:0/2005702074 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c0052d0 con 0x7f6a600d0660 2024-03-29T08:18:22.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a57e00640 1 -- 172.21.15.46:0/2005702074 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c005470 con 0x7f6a600d0660 2024-03-29T08:18:22.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a57e00640 1 -- 172.21.15.46:0/2005702074 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c003840 con 0x7f6a600d0660 2024-03-29T08:18:22.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a66a00640 1 -- 172.21.15.46:0/2005702074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a600b3750 con 0x7f6a600d0660 2024-03-29T08:18:22.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a66a00640 1 -- 172.21.15.46:0/2005702074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a600b3d10 con 0x7f6a600d0660 2024-03-29T08:18:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.664+0000 7f6a66a00640 1 -- 172.21.15.46:0/2005702074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a600c81f0 con 0x7f6a600d0660 2024-03-29T08:18:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.668+0000 7f6a57e00640 1 -- 172.21.15.46:0/2005702074 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6a4c0039a0 con 0x7f6a600d0660 2024-03-29T08:18:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.668+0000 7f6a57e00640 1 --2- 172.21.15.46:0/2005702074 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a40066690 0x7f6a40068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.668+0000 7f6a57e00640 1 -- 172.21.15.46:0/2005702074 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a4c0836a0 con 0x7f6a600d0660 2024-03-29T08:18:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.668+0000 7f6a5fe00640 1 --2- 172.21.15.46:0/2005702074 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a40066690 0x7f6a40068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.668+0000 7f6a5fe00640 1 --2- 172.21.15.46:0/2005702074 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a40066690 0x7f6a40068b50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6a500af000 tx=0x7f6a500a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:22.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.672+0000 7f6a57e00640 1 -- 172.21.15.46:0/2005702074 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6a4c050a30 con 0x7f6a600d0660 2024-03-29T08:18:22.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.872+0000 7f6a66a00640 1 -- 172.21.15.46:0/2005702074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6a60066c90 con 0x7f6a600d0660 2024-03-29T08:18:22.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.872+0000 7f6a57e00640 1 -- 172.21.15.46:0/2005702074 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f6a4c054050 con 0x7f6a600d0660 2024-03-29T08:18:22.872 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:22.872 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-29T08:18:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.872+0000 7f6a66a00640 1 -- 172.21.15.46:0/2005702074 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a40066690 msgr2=0x7f6a40068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.872+0000 7f6a66a00640 1 --2- 172.21.15.46:0/2005702074 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a40066690 0x7f6a40068b50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6a500af000 tx=0x7f6a500a7000 comp rx=0 tx=0).stop 2024-03-29T08:18:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.872+0000 7f6a66a00640 1 -- 172.21.15.46:0/2005702074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a600d0660 msgr2=0x7f6a6006f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.872+0000 7f6a66a00640 1 --2- 172.21.15.46:0/2005702074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a600d0660 0x7f6a6006f150 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00c500 tx=0x7f6a4c00c9c0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.872+0000 7f6a66a00640 1 -- 172.21.15.46:0/2005702074 shutdown_connections 2024-03-29T08:18:22.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.872+0000 7f6a66a00640 1 --2- 172.21.15.46:0/2005702074 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a40066690 0x7f6a40068b50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.872+0000 7f6a66a00640 1 --2- 172.21.15.46:0/2005702074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6015dc00 0x7f6a60155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.872+0000 7f6a66a00640 1 --2- 172.21.15.46:0/2005702074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a600d0660 0x7f6a6006f150 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.872+0000 7f6a66a00640 1 --2- 172.21.15.46:0/2005702074 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6a6006b8b0 0x7f6a600b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:22.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.872+0000 7f6a66a00640 1 -- 172.21.15.46:0/2005702074 >> 172.21.15.46:0/2005702074 conn(0x7f6a60089fd0 msgr2=0x7f6a60159d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:22.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.872+0000 7f6a66a00640 1 -- 172.21.15.46:0/2005702074 shutdown_connections 2024-03-29T08:18:22.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:22.872+0000 7f6a66a00640 1 -- 172.21.15.46:0/2005702074 wait complete. 2024-03-29T08:18:22.893 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-29T08:18:22.894 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:23.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.032+0000 7fd215200640 1 Processor -- start 2024-03-29T08:18:23.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.032+0000 7fd215200640 1 -- start start 2024-03-29T08:18:23.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.032+0000 7fd215200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd21015ad70 0x7fd2100cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.032+0000 7fd215200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21015b370 con 0x7fd2100ce490 2024-03-29T08:18:23.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.032+0000 7fd215200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21015b650 con 0x7fd210098a40 2024-03-29T08:18:23.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.032+0000 7fd215200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21015b930 con 0x7fd21015ad70 2024-03-29T08:18:23.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd207e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd2100ce490 0x7fd210099790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32792/0 (socket says 172.21.15.46:32792) 2024-03-29T08:18:23.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd207e00640 1 -- 172.21.15.46:0/1284775739 learned_addr learned my addr 172.21.15.46:0/1284775739 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:23.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20e800640 1 --2- 172.21.15.46:0/1284775739 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd21015ad70 0x7fd2100cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20d400640 1 -- 172.21.15.46:0/1284775739 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082212896 0 0) 0x7fd21015b370 con 0x7fd2100ce490 2024-03-29T08:18:23.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20d400640 1 -- 172.21.15.46:0/1284775739 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1f4003530 con 0x7fd2100ce490 2024-03-29T08:18:23.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20d400640 1 -- 172.21.15.46:0/1284775739 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508107983 0 0) 0x7fd21015b650 con 0x7fd210098a40 2024-03-29T08:18:23.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20d400640 1 -- 172.21.15.46:0/1284775739 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd21015b370 con 0x7fd210098a40 2024-03-29T08:18:23.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20d400640 1 -- 172.21.15.46:0/1284775739 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 82422939 0 0) 0x7fd1f4003530 con 0x7fd2100ce490 2024-03-29T08:18:23.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20d400640 1 -- 172.21.15.46:0/1284775739 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd21015ad70 msgr2=0x7fd2100cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:23.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20d400640 1 --2- 172.21.15.46:0/1284775739 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd21015ad70 0x7fd2100cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20d400640 1 -- 172.21.15.46:0/1284775739 >> v1:172.21.15.73:6789/0 conn(0x7fd210098a40 legacy=0x7fd210098e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20d400640 1 -- 172.21.15.46:0/1284775739 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21015bc10 con 0x7fd2100ce490 2024-03-29T08:18:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20d400640 1 -- 172.21.15.46:0/1284775739 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fd1f8001680 con 0x7fd2100ce490 2024-03-29T08:18:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20e800640 1 --2- 172.21.15.46:0/1284775739 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd21015ad70 0x7fd2100cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20d400640 1 -- 172.21.15.46:0/1284775739 >> v1:172.21.15.46:6789/0 conn(0x7fd2100ce490 legacy=0x7fd210099790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20d400640 1 --2- 172.21.15.46:0/1284775739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f4003b40 0x7fd1f4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20d400640 1 --2- 172.21.15.46:0/1284775739 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd21015ad70 0x7fd1f40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20d400640 1 --2- 172.21.15.46:0/1284775739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1f4004d10 0x7fd1f4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20d400640 1 -- 172.21.15.46:0/1284775739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21015b930 con 0x7fd1f4003b40 2024-03-29T08:18:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20d400640 1 -- 172.21.15.46:0/1284775739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f400ba40 con 0x7fd1f4004d10 2024-03-29T08:18:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20d400640 1 -- 172.21.15.46:0/1284775739 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f400bc00 con 0x7fd21015ad70 2024-03-29T08:18:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20de00640 1 --2- 172.21.15.46:0/1284775739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f4003b40 0x7fd1f4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd207e00640 1 --2- 172.21.15.46:0/1284775739 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd21015ad70 0x7fd1f40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20e800640 1 --2- 172.21.15.46:0/1284775739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1f4004d10 0x7fd1f4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20de00640 1 -- 172.21.15.46:0/1284775739 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd21015ad70 msgr2=0x7fd1f40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:23.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20de00640 1 --2- 172.21.15.46:0/1284775739 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd21015ad70 0x7fd1f40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20de00640 1 -- 172.21.15.46:0/1284775739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1f4004d10 msgr2=0x7fd1f4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:23.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20de00640 1 --2- 172.21.15.46:0/1284775739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd1f4004d10 0x7fd1f4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd20de00640 1 -- 172.21.15.46:0/1284775739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd208009a70 con 0x7fd1f4003b40 2024-03-29T08:18:23.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.036+0000 7fd207e00640 1 --2- 172.21.15.46:0/1284775739 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd21015ad70 0x7fd1f40047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.040+0000 7fd20de00640 1 --2- 172.21.15.46:0/1284775739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f4003b40 0x7fd1f4003f40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd2000018d0 tx=0x7fd20001c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dc09a4fa8a5f87c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.040+0000 7fd20d400640 1 -- 172.21.15.46:0/1284775739 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2000050d0 con 0x7fd1f4003b40 2024-03-29T08:18:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.040+0000 7fd20d400640 1 -- 172.21.15.46:0/1284775739 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd200005250 con 0x7fd1f4003b40 2024-03-29T08:18:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.040+0000 7fd20d400640 1 -- 172.21.15.46:0/1284775739 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd200003600 con 0x7fd1f4003b40 2024-03-29T08:18:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.040+0000 7fd215200640 1 -- 172.21.15.46:0/1284775739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f4003b40 msgr2=0x7fd1f4003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.040+0000 7fd215200640 1 --2- 172.21.15.46:0/1284775739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f4003b40 0x7fd1f4003f40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd2000018d0 tx=0x7fd20001c580 comp rx=0 tx=0).stop 2024-03-29T08:18:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.040+0000 7fd20e800640 1 -- 172.21.15.46:0/1284775739 reap_dead start 2024-03-29T08:18:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.040+0000 7fd215200640 1 -- 172.21.15.46:0/1284775739 shutdown_connections 2024-03-29T08:18:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.040+0000 7fd215200640 1 -- 172.21.15.46:0/1284775739 >> 172.21.15.46:0/1284775739 conn(0x7fd2100b21a0 msgr2=0x7fd2100b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.040+0000 7fd215200640 1 -- 172.21.15.46:0/1284775739 shutdown_connections 2024-03-29T08:18:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.040+0000 7fd215200640 1 -- 172.21.15.46:0/1284775739 wait complete. 2024-03-29T08:18:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.040+0000 7fd215200640 1 Processor -- start 2024-03-29T08:18:23.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.040+0000 7fd215200640 1 -- start start 2024-03-29T08:18:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd215200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f4003b40 0x7fd210151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd215200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd2101542d0 0x7fd2100cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd215200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd21015c520 0x7fd2100cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd215200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd210156cc0 con 0x7fd1f4003b40 2024-03-29T08:18:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd215200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd210156fa0 con 0x7fd21015c520 2024-03-29T08:18:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd215200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd210157280 con 0x7fd2101542d0 2024-03-29T08:18:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd20de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f4003b40 0x7fd210151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd20de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f4003b40 0x7fd210151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60190/0 (socket says 172.21.15.46:60190) 2024-03-29T08:18:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd20de00640 1 -- 172.21.15.46:0/3563542305 learned_addr learned my addr 172.21.15.46:0/3563542305 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd207e00640 1 --2- 172.21.15.46:0/3563542305 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd2101542d0 0x7fd2100cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd20e800640 1 --2- 172.21.15.46:0/3563542305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd21015c520 0x7fd2100cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd20de00640 1 -- 172.21.15.46:0/3563542305 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd2101542d0 msgr2=0x7fd2100cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd20de00640 1 --2- 172.21.15.46:0/3563542305 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd2101542d0 0x7fd2100cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd20de00640 1 -- 172.21.15.46:0/3563542305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd21015c520 msgr2=0x7fd2100cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd20de00640 1 --2- 172.21.15.46:0/3563542305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd21015c520 0x7fd2100cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd20de00640 1 -- 172.21.15.46:0/3563542305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd210157560 con 0x7fd1f4003b40 2024-03-29T08:18:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd207e00640 1 --2- 172.21.15.46:0/3563542305 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd2101542d0 0x7fd2100cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd20e800640 1 --2- 172.21.15.46:0/3563542305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd21015c520 0x7fd2100cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd20de00640 1 --2- 172.21.15.46:0/3563542305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f4003b40 0x7fd210151ac0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd200005040 tx=0x7fd1f4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd206000640 1 -- 172.21.15.46:0/3563542305 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd200005480 con 0x7fd1f4003b40 2024-03-29T08:18:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd206000640 1 -- 172.21.15.46:0/3563542305 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd200003810 con 0x7fd1f4003b40 2024-03-29T08:18:23.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd206000640 1 -- 172.21.15.46:0/3563542305 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd200003b80 con 0x7fd1f4003b40 2024-03-29T08:18:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd215200640 1 -- 172.21.15.46:0/3563542305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd210152000 con 0x7fd1f4003b40 2024-03-29T08:18:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd215200640 1 -- 172.21.15.46:0/3563542305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd210152310 con 0x7fd1f4003b40 2024-03-29T08:18:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd215200640 1 -- 172.21.15.46:0/3563542305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2100c5a50 con 0x7fd1f4003b40 2024-03-29T08:18:23.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.044+0000 7fd206000640 1 -- 172.21.15.46:0/3563542305 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd200003ce0 con 0x7fd1f4003b40 2024-03-29T08:18:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.048+0000 7fd206000640 1 --2- 172.21.15.46:0/3563542305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd1ec066690 0x7fd1ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.048+0000 7fd206000640 1 -- 172.21.15.46:0/3563542305 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd20011dae0 con 0x7fd1f4003b40 2024-03-29T08:18:23.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.048+0000 7fd207e00640 1 --2- 172.21.15.46:0/3563542305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd1ec066690 0x7fd1ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.048+0000 7fd207e00640 1 --2- 172.21.15.46:0/3563542305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd1ec066690 0x7fd1ec068b50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8001210 tx=0x7fd1f8001120 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:23.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.052+0000 7fd206000640 1 -- 172.21.15.46:0/3563542305 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd2000ee7b0 con 0x7fd1f4003b40 2024-03-29T08:18:23.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.212+0000 7fd215200640 1 -- 172.21.15.46:0/3563542305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd210157750 con 0x7fd1f4003b40 2024-03-29T08:18:23.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.212+0000 7fd206000640 1 -- 172.21.15.46:0/3563542305 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15993 (secure 0 0 0) 0x7fd2000ee340 con 0x7fd1f4003b40 2024-03-29T08:18:23.215 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:23.215 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:14.314124+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:23.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.216+0000 7fd215200640 1 -- 172.21.15.46:0/3563542305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd1ec066690 msgr2=0x7fd1ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.216+0000 7fd215200640 1 --2- 172.21.15.46:0/3563542305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd1ec066690 0x7fd1ec068b50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8001210 tx=0x7fd1f8001120 comp rx=0 tx=0).stop 2024-03-29T08:18:23.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.216+0000 7fd215200640 1 -- 172.21.15.46:0/3563542305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f4003b40 msgr2=0x7fd210151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.216+0000 7fd215200640 1 --2- 172.21.15.46:0/3563542305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f4003b40 0x7fd210151ac0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd200005040 tx=0x7fd1f4005800 comp rx=0 tx=0).stop 2024-03-29T08:18:23.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.216+0000 7fd215200640 1 -- 172.21.15.46:0/3563542305 shutdown_connections 2024-03-29T08:18:23.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.216+0000 7fd215200640 1 --2- 172.21.15.46:0/3563542305 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd1ec066690 0x7fd1ec068b50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.216+0000 7fd215200640 1 --2- 172.21.15.46:0/3563542305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd21015c520 0x7fd2100cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.216+0000 7fd215200640 1 --2- 172.21.15.46:0/3563542305 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd2101542d0 0x7fd2100cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.216+0000 7fd215200640 1 --2- 172.21.15.46:0/3563542305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd1f4003b40 0x7fd210151ac0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.216+0000 7fd215200640 1 -- 172.21.15.46:0/3563542305 >> 172.21.15.46:0/3563542305 conn(0x7fd2100b21a0 msgr2=0x7fd210099790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:23.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.216+0000 7fd215200640 1 -- 172.21.15.46:0/3563542305 shutdown_connections 2024-03-29T08:18:23.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.216+0000 7fd215200640 1 -- 172.21.15.46:0/3563542305 wait complete. 2024-03-29T08:18:23.236 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:18:11.177842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:18:12.308252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:18:23.237 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:18:23.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.376+0000 7f3453000640 1 Processor -- start 2024-03-29T08:18:23.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.380+0000 7f3453000640 1 -- start start 2024-03-29T08:18:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.380+0000 7f3453000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f344c142a50 0x7f344c0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.380+0000 7f3453000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f344c15d260 con 0x7f344c15df90 2024-03-29T08:18:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.380+0000 7f3453000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f344c0aefd0 con 0x7f344c0ae990 2024-03-29T08:18:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.380+0000 7f3453000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f344c15d4b0 con 0x7f344c142a50 2024-03-29T08:18:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.380+0000 7f344be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f344c142a50 0x7f344c0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.380+0000 7f3443400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f344c15df90 0x7f344c0ae280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32808/0 (socket says 172.21.15.46:32808) 2024-03-29T08:18:23.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.380+0000 7f3443400640 1 -- 172.21.15.46:0/1628570596 learned_addr learned my addr 172.21.15.46:0/1628570596 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.380+0000 7f344be00640 1 -- 172.21.15.46:0/1628570596 >> v1:172.21.15.73:6789/0 conn(0x7f344c0ae990 legacy=0x7f344c0af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.380+0000 7f344be00640 1 -- 172.21.15.46:0/1628570596 >> v1:172.21.15.46:6789/0 conn(0x7f344c15df90 legacy=0x7f344c0ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.380+0000 7f344be00640 1 -- 172.21.15.46:0/1628570596 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c15d790 con 0x7f344c142a50 2024-03-29T08:18:23.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.380+0000 7f344b400640 1 -- 172.21.15.46:0/1628570596 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344203923 0 0) 0x7f344c15d260 con 0x7f344c15df90 2024-03-29T08:18:23.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.384+0000 7f344be00640 1 --2- 172.21.15.46:0/1628570596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f344c142a50 0x7f344c0cb790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f343c0097c0 tx=0x7f343c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf983c96491ebbf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:23.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.384+0000 7f344b400640 1 -- 172.21.15.46:0/1628570596 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c0a4ea0 con 0x7f344c142a50 2024-03-29T08:18:23.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.384+0000 7f344b400640 1 -- 172.21.15.46:0/1628570596 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343c0a7050 con 0x7f344c142a50 2024-03-29T08:18:23.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.384+0000 7f344b400640 1 -- 172.21.15.46:0/1628570596 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c0ab620 con 0x7f344c142a50 2024-03-29T08:18:23.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.384+0000 7f3453000640 1 -- 172.21.15.46:0/1628570596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f344c142a50 msgr2=0x7f344c0cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.384+0000 7f3453000640 1 --2- 172.21.15.46:0/1628570596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f344c142a50 0x7f344c0cb790 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f343c0097c0 tx=0x7f343c01c410 comp rx=0 tx=0).stop 2024-03-29T08:18:23.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.384+0000 7f3453000640 1 -- 172.21.15.46:0/1628570596 shutdown_connections 2024-03-29T08:18:23.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.384+0000 7f3453000640 1 --2- 172.21.15.46:0/1628570596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f344c142a50 0x7f344c0cb790 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.384+0000 7f3453000640 1 -- 172.21.15.46:0/1628570596 >> 172.21.15.46:0/1628570596 conn(0x7f344c06a390 msgr2=0x7f344c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:23.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.384+0000 7f3453000640 1 -- 172.21.15.46:0/1628570596 shutdown_connections 2024-03-29T08:18:23.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.384+0000 7f3453000640 1 -- 172.21.15.46:0/1628570596 wait complete. 2024-03-29T08:18:23.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.384+0000 7f3453000640 1 Processor -- start 2024-03-29T08:18:23.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.384+0000 7f3453000640 1 -- start start 2024-03-29T08:18:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f3453000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f344c0ae990 0x7f344c1580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f3453000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f344c142a50 0x7f344c13fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f3453000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f344c15df90 0x7f344c168690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f3453000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c13a620 con 0x7f344c0ae990 2024-03-29T08:18:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f3453000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c139a10 con 0x7f344c142a50 2024-03-29T08:18:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f3453000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344c139cf0 con 0x7f344c15df90 2024-03-29T08:18:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f344be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f344c0ae990 0x7f344c1580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f344be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f344c0ae990 0x7f344c1580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60204/0 (socket says 172.21.15.46:60204) 2024-03-29T08:18:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f344be00640 1 -- 172.21.15.46:0/803814642 learned_addr learned my addr 172.21.15.46:0/803814642 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f3443400640 1 --2- 172.21.15.46:0/803814642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f344c142a50 0x7f344c13fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f3451000640 1 --2- 172.21.15.46:0/803814642 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f344c15df90 0x7f344c168690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f344be00640 1 -- 172.21.15.46:0/803814642 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f344c15df90 msgr2=0x7f344c168690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f344be00640 1 --2- 172.21.15.46:0/803814642 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f344c15df90 0x7f344c168690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f344be00640 1 -- 172.21.15.46:0/803814642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f344c142a50 msgr2=0x7f344c13fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f344be00640 1 --2- 172.21.15.46:0/803814642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f344c142a50 0x7f344c13fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f344be00640 1 -- 172.21.15.46:0/803814642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c009420 con 0x7f344c0ae990 2024-03-29T08:18:23.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f344be00640 1 --2- 172.21.15.46:0/803814642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f344c0ae990 0x7f344c1580a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f343c01a040 tx=0x7f343c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f3448c00640 1 -- 172.21.15.46:0/803814642 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c004230 con 0x7f344c0ae990 2024-03-29T08:18:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f3448c00640 1 -- 172.21.15.46:0/803814642 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343c004390 con 0x7f344c0ae990 2024-03-29T08:18:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f3453000640 1 -- 172.21.15.46:0/803814642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344c139fd0 con 0x7f344c0ae990 2024-03-29T08:18:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f3448c00640 1 -- 172.21.15.46:0/803814642 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c003820 con 0x7f344c0ae990 2024-03-29T08:18:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f3453000640 1 -- 172.21.15.46:0/803814642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344c13ff40 con 0x7f344c0ae990 2024-03-29T08:18:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f3453000640 1 -- 172.21.15.46:0/803814642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344c071a70 con 0x7f344c0ae990 2024-03-29T08:18:23.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.388+0000 7f3448c00640 1 -- 172.21.15.46:0/803814642 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f343c0b2040 con 0x7f344c0ae990 2024-03-29T08:18:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.392+0000 7f3448c00640 1 --2- 172.21.15.46:0/803814642 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3428066690 0x7f3428068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.392+0000 7f3448c00640 1 -- 172.21.15.46:0/803814642 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f343c11e230 con 0x7f344c0ae990 2024-03-29T08:18:23.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.392+0000 7f3443400640 1 --2- 172.21.15.46:0/803814642 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3428066690 0x7f3428068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.392+0000 7f3443400640 1 --2- 172.21.15.46:0/803814642 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3428066690 0x7f3428068b50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f344c13cfb0 tx=0x7f3434007240 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:23.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.396+0000 7f3448c00640 1 -- 172.21.15.46:0/803814642 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f343c0eb640 con 0x7f344c0ae990 2024-03-29T08:18:23.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.556+0000 7f3453000640 1 -- 172.21.15.46:0/803814642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f344c04e120 con 0x7f344c0ae990 2024-03-29T08:18:23.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.556+0000 7f3448c00640 1 -- 172.21.15.46:0/803814642 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f343c0eec60 con 0x7f344c0ae990 2024-03-29T08:18:23.557 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:18:23.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.556+0000 7f3453000640 1 -- 172.21.15.46:0/803814642 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3428066690 msgr2=0x7f3428068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.556+0000 7f3453000640 1 --2- 172.21.15.46:0/803814642 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3428066690 0x7f3428068b50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f344c13cfb0 tx=0x7f3434007240 comp rx=0 tx=0).stop 2024-03-29T08:18:23.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.556+0000 7f3453000640 1 -- 172.21.15.46:0/803814642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f344c0ae990 msgr2=0x7f344c1580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.556+0000 7f3453000640 1 --2- 172.21.15.46:0/803814642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f344c0ae990 0x7f344c1580a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f343c01a040 tx=0x7f343c0191b0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.556+0000 7f3453000640 1 -- 172.21.15.46:0/803814642 shutdown_connections 2024-03-29T08:18:23.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.556+0000 7f3453000640 1 --2- 172.21.15.46:0/803814642 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3428066690 0x7f3428068b50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.556+0000 7f3453000640 1 --2- 172.21.15.46:0/803814642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f344c142a50 0x7f344c13fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.556+0000 7f3453000640 1 --2- 172.21.15.46:0/803814642 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f344c15df90 0x7f344c168690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.556+0000 7f3453000640 1 --2- 172.21.15.46:0/803814642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f344c0ae990 0x7f344c1580a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.556+0000 7f3453000640 1 -- 172.21.15.46:0/803814642 >> 172.21.15.46:0/803814642 conn(0x7f344c06a390 msgr2=0x7f344c158690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:23.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.556+0000 7f3453000640 1 -- 172.21.15.46:0/803814642 shutdown_connections 2024-03-29T08:18:23.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.556+0000 7f3453000640 1 -- 172.21.15.46:0/803814642 wait complete. 2024-03-29T08:18:23.577 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:18:23.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.716+0000 7f22e8a00640 1 Processor -- start 2024-03-29T08:18:23.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.720+0000 7f22e8a00640 1 -- start start 2024-03-29T08:18:23.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.720+0000 7f22e8a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22e4142a50 0x7f22e40cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.720+0000 7f22e8a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e415d260 con 0x7f22e40ae990 2024-03-29T08:18:23.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.720+0000 7f22e8a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e40aefd0 con 0x7f22e415df90 2024-03-29T08:18:23.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.720+0000 7f22e8a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e415d4b0 con 0x7f22e4142a50 2024-03-29T08:18:23.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.720+0000 7f22e2800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f22e40ae990 0x7f22e40af530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32812/0 (socket says 172.21.15.46:32812) 2024-03-29T08:18:23.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.720+0000 7f22e2800640 1 -- 172.21.15.46:0/3304385392 learned_addr learned my addr 172.21.15.46:0/3304385392 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:23.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.720+0000 7f22e1e00640 1 --2- 172.21.15.46:0/3304385392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22e4142a50 0x7f22e40cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.720+0000 7f22e1400640 1 -- 172.21.15.46:0/3304385392 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1098295208 0 0) 0x7f22e415d260 con 0x7f22e40ae990 2024-03-29T08:18:23.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.720+0000 7f22e1400640 1 -- 172.21.15.46:0/3304385392 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22c8003530 con 0x7f22e40ae990 2024-03-29T08:18:23.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.720+0000 7f22e1400640 1 -- 172.21.15.46:0/3304385392 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750291749 0 0) 0x7f22e40aefd0 con 0x7f22e415df90 2024-03-29T08:18:23.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.720+0000 7f22e1400640 1 -- 172.21.15.46:0/3304385392 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22e415d260 con 0x7f22e415df90 2024-03-29T08:18:23.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.720+0000 7f22e1e00640 1 -- 172.21.15.46:0/3304385392 >> v1:172.21.15.73:6789/0 conn(0x7f22e415df90 legacy=0x7f22e40ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.720+0000 7f22e1e00640 1 -- 172.21.15.46:0/3304385392 >> v1:172.21.15.46:6789/0 conn(0x7f22e40ae990 legacy=0x7f22e40af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.720+0000 7f22e1e00640 1 -- 172.21.15.46:0/3304385392 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e415d790 con 0x7f22e4142a50 2024-03-29T08:18:23.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.724+0000 7f22e1e00640 1 --2- 172.21.15.46:0/3304385392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22e4142a50 0x7f22e40cb790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f22d40097f0 tx=0x7f22d401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=125e0fb9f13d0e42 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:23.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.724+0000 7f22e1400640 1 -- 172.21.15.46:0/3304385392 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40ab400 con 0x7f22e4142a50 2024-03-29T08:18:23.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.724+0000 7f22e1400640 1 -- 172.21.15.46:0/3304385392 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d40a7050 con 0x7f22e4142a50 2024-03-29T08:18:23.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.724+0000 7f22e1400640 1 -- 172.21.15.46:0/3304385392 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40ab780 con 0x7f22e4142a50 2024-03-29T08:18:23.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.724+0000 7f22e8a00640 1 -- 172.21.15.46:0/3304385392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22e4142a50 msgr2=0x7f22e40cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.724+0000 7f22e8a00640 1 --2- 172.21.15.46:0/3304385392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22e4142a50 0x7f22e40cb790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f22d40097f0 tx=0x7f22d401c450 comp rx=0 tx=0).stop 2024-03-29T08:18:23.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.724+0000 7f22e8a00640 1 -- 172.21.15.46:0/3304385392 shutdown_connections 2024-03-29T08:18:23.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.724+0000 7f22e8a00640 1 --2- 172.21.15.46:0/3304385392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22e4142a50 0x7f22e40cb790 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.724+0000 7f22e8a00640 1 -- 172.21.15.46:0/3304385392 >> 172.21.15.46:0/3304385392 conn(0x7f22e406a390 msgr2=0x7f22e40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:23.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.724+0000 7f22e8a00640 1 -- 172.21.15.46:0/3304385392 shutdown_connections 2024-03-29T08:18:23.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.724+0000 7f22e8a00640 1 -- 172.21.15.46:0/3304385392 wait complete. 2024-03-29T08:18:23.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.724+0000 7f22e8a00640 1 Processor -- start 2024-03-29T08:18:23.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.724+0000 7f22e8a00640 1 -- start start 2024-03-29T08:18:23.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e8a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22e40ae990 0x7f22e4158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22e4142a50 0x7f22e40ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22e415df90 0x7f22e413fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e8a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e413c590 con 0x7f22e415df90 2024-03-29T08:18:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e8a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40abd50 con 0x7f22e4142a50 2024-03-29T08:18:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e8a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e413bfd0 con 0x7f22e40ae990 2024-03-29T08:18:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22e415df90 0x7f22e413fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22e415df90 0x7f22e413fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60214/0 (socket says 172.21.15.46:60214) 2024-03-29T08:18:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e2800640 1 -- 172.21.15.46:0/3242483221 learned_addr learned my addr 172.21.15.46:0/3242483221 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e1e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22e40ae990 0x7f22e4158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22d9400640 1 --2- 172.21.15.46:0/3242483221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22e4142a50 0x7f22e40ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e2800640 1 -- 172.21.15.46:0/3242483221 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22e40ae990 msgr2=0x7f22e4158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e2800640 1 --2- 172.21.15.46:0/3242483221 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22e40ae990 0x7f22e4158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e2800640 1 -- 172.21.15.46:0/3242483221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22e4142a50 msgr2=0x7f22e40ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e2800640 1 --2- 172.21.15.46:0/3242483221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22e4142a50 0x7f22e40ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e2800640 1 -- 172.21.15.46:0/3242483221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d4009420 con 0x7f22e415df90 2024-03-29T08:18:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e2800640 1 --2- 172.21.15.46:0/3242483221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22e415df90 0x7f22e413fd70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00cb70 tx=0x7f22dc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22daa00640 1 -- 172.21.15.46:0/3242483221 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc00ace0 con 0x7f22e415df90 2024-03-29T08:18:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22daa00640 1 -- 172.21.15.46:0/3242483221 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc00ae60 con 0x7f22e415df90 2024-03-29T08:18:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22daa00640 1 -- 172.21.15.46:0/3242483221 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc012640 con 0x7f22e415df90 2024-03-29T08:18:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e8a00640 1 -- 172.21.15.46:0/3242483221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e413c2b0 con 0x7f22e415df90 2024-03-29T08:18:23.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e8a00640 1 -- 172.21.15.46:0/3242483221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e40ca550 con 0x7f22e415df90 2024-03-29T08:18:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.728+0000 7f22e8a00640 1 -- 172.21.15.46:0/3242483221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e40733a0 con 0x7f22e415df90 2024-03-29T08:18:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.732+0000 7f22daa00640 1 -- 172.21.15.46:0/3242483221 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f22dc007070 con 0x7f22e415df90 2024-03-29T08:18:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.732+0000 7f22daa00640 1 --2- 172.21.15.46:0/3242483221 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22c0066690 0x7f22c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.732+0000 7f22daa00640 1 -- 172.21.15.46:0/3242483221 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22dc084140 con 0x7f22e415df90 2024-03-29T08:18:23.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.732+0000 7f22e1e00640 1 --2- 172.21.15.46:0/3242483221 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22c0066690 0x7f22c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:23.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.732+0000 7f22e1e00640 1 --2- 172.21.15.46:0/3242483221 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22c0066690 0x7f22c0068b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f22d40ad000 tx=0x7f22d4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:23.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.736+0000 7f22daa00640 1 -- 172.21.15.46:0/3242483221 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f22dc0514d0 con 0x7f22e415df90 2024-03-29T08:18:23.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.896+0000 7f22e8a00640 1 -- 172.21.15.46:0/3242483221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f22e4071a70 con 0x7f22e415df90 2024-03-29T08:18:23.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.896+0000 7f22daa00640 1 -- 172.21.15.46:0/3242483221 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f22dc054af0 con 0x7f22e415df90 2024-03-29T08:18:23.896 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:18:23.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.896+0000 7f22e8a00640 1 -- 172.21.15.46:0/3242483221 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22c0066690 msgr2=0x7f22c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.896+0000 7f22e8a00640 1 --2- 172.21.15.46:0/3242483221 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22c0066690 0x7f22c0068b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f22d40ad000 tx=0x7f22d4008040 comp rx=0 tx=0).stop 2024-03-29T08:18:23.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.896+0000 7f22e8a00640 1 -- 172.21.15.46:0/3242483221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22e415df90 msgr2=0x7f22e413fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:23.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.896+0000 7f22e8a00640 1 --2- 172.21.15.46:0/3242483221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22e415df90 0x7f22e413fd70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00cb70 tx=0x7f22dc00a430 comp rx=0 tx=0).stop 2024-03-29T08:18:23.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.896+0000 7f22e8a00640 1 -- 172.21.15.46:0/3242483221 shutdown_connections 2024-03-29T08:18:23.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.896+0000 7f22e8a00640 1 --2- 172.21.15.46:0/3242483221 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22c0066690 0x7f22c0068b50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.896+0000 7f22e8a00640 1 --2- 172.21.15.46:0/3242483221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22e4142a50 0x7f22e40ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.896+0000 7f22e8a00640 1 --2- 172.21.15.46:0/3242483221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22e415df90 0x7f22e413fd70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.896+0000 7f22e8a00640 1 --2- 172.21.15.46:0/3242483221 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22e40ae990 0x7f22e4158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:23.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.896+0000 7f22e8a00640 1 -- 172.21.15.46:0/3242483221 >> 172.21.15.46:0/3242483221 conn(0x7f22e406a390 msgr2=0x7f22e415bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:23.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.896+0000 7f22e8a00640 1 -- 172.21.15.46:0/3242483221 shutdown_connections 2024-03-29T08:18:23.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:23.896+0000 7f22e8a00640 1 -- 172.21.15.46:0/3242483221 wait complete. 2024-03-29T08:18:23.917 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:18:24.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.056+0000 7fca8ac00640 1 Processor -- start 2024-03-29T08:18:24.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.056+0000 7fca8ac00640 1 -- start start 2024-03-29T08:18:24.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.060+0000 7fca8ac00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fca8406b4b0 0x7fca840b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.060+0000 7fca8ac00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca840b3f20 0x7fca840b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.060+0000 7fca8ac00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca84157490 con 0x7fca8415dbb0 2024-03-29T08:18:24.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.060+0000 7fca8ac00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca840b27b0 con 0x7fca8406b4b0 2024-03-29T08:18:24.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.060+0000 7fca8ac00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca840b2c90 con 0x7fca840b3f20 2024-03-29T08:18:24.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.060+0000 7fca88c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca840b3f20 0x7fca840b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.060+0000 7fca7b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fca8406b4b0 0x7fca840b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.060+0000 7fca88c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca840b3f20 0x7fca840b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:57968/0 (socket says 172.21.15.46:57968) 2024-03-29T08:18:24.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.060+0000 7fca88c00640 1 -- 172.21.15.46:0/257397175 learned_addr learned my addr 172.21.15.46:0/257397175 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:24.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.060+0000 7fca7b400640 1 -- 172.21.15.46:0/257397175 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca840b3f20 msgr2=0x7fca840b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:24.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.060+0000 7fca7b400640 1 --2- 172.21.15.46:0/257397175 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca840b3f20 0x7fca840b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.060+0000 7fca7b400640 1 -- 172.21.15.46:0/257397175 >> v1:172.21.15.73:6789/0 conn(0x7fca8415dbb0 legacy=0x7fca8406ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.060+0000 7fca7b400640 1 -- 172.21.15.46:0/257397175 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca840b29b0 con 0x7fca8406b4b0 2024-03-29T08:18:24.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.064+0000 7fca7b400640 1 --2- 172.21.15.46:0/257397175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca8406b4b0 0x7fca840b1d30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0097c0 tx=0x7fca6c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3335a22c3f7e5e31 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:24.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.064+0000 7fca83400640 1 -- 172.21.15.46:0/257397175 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c0ab610 con 0x7fca8406b4b0 2024-03-29T08:18:24.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.064+0000 7fca83400640 1 -- 172.21.15.46:0/257397175 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6c0ab770 con 0x7fca8406b4b0 2024-03-29T08:18:24.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.064+0000 7fca83400640 1 -- 172.21.15.46:0/257397175 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c0abb00 con 0x7fca8406b4b0 2024-03-29T08:18:24.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.064+0000 7fca8ac00640 1 -- 172.21.15.46:0/257397175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca8406b4b0 msgr2=0x7fca840b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.064+0000 7fca8ac00640 1 --2- 172.21.15.46:0/257397175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca8406b4b0 0x7fca840b1d30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fca6c0097c0 tx=0x7fca6c01c410 comp rx=0 tx=0).stop 2024-03-29T08:18:24.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.064+0000 7fca8ac00640 1 -- 172.21.15.46:0/257397175 shutdown_connections 2024-03-29T08:18:24.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.064+0000 7fca8ac00640 1 --2- 172.21.15.46:0/257397175 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca840b3f20 0x7fca840b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.064+0000 7fca8ac00640 1 --2- 172.21.15.46:0/257397175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca8406b4b0 0x7fca840b1d30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.064+0000 7fca8ac00640 1 -- 172.21.15.46:0/257397175 >> 172.21.15.46:0/257397175 conn(0x7fca84070ad0 msgr2=0x7fca84070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:24.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.064+0000 7fca8ac00640 1 -- 172.21.15.46:0/257397175 shutdown_connections 2024-03-29T08:18:24.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.064+0000 7fca8ac00640 1 -- 172.21.15.46:0/257397175 wait complete. 2024-03-29T08:18:24.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.064+0000 7fca8ac00640 1 Processor -- start 2024-03-29T08:18:24.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.064+0000 7fca8ac00640 1 -- start start 2024-03-29T08:18:24.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca8ac00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca8406b4b0 0x7fca8414f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca8ac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca840b3f20 0x7fca84150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca8ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8415dbb0 0x7fca8414e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca8ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca84154b40 con 0x7fca8415dbb0 2024-03-29T08:18:24.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca8ac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca84154e20 con 0x7fca840b3f20 2024-03-29T08:18:24.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca8ac00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca84155100 con 0x7fca8406b4b0 2024-03-29T08:18:24.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca88c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8415dbb0 0x7fca8414e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca88c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8415dbb0 0x7fca8414e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60224/0 (socket says 172.21.15.46:60224) 2024-03-29T08:18:24.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca88c00640 1 -- 172.21.15.46:0/3799954803 learned_addr learned my addr 172.21.15.46:0/3799954803 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:24.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca83e00640 1 --2- 172.21.15.46:0/3799954803 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca8406b4b0 0x7fca8414f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca7b400640 1 --2- 172.21.15.46:0/3799954803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca840b3f20 0x7fca84150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca88c00640 1 -- 172.21.15.46:0/3799954803 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca8406b4b0 msgr2=0x7fca8414f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca88c00640 1 --2- 172.21.15.46:0/3799954803 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca8406b4b0 0x7fca8414f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca88c00640 1 -- 172.21.15.46:0/3799954803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca840b3f20 msgr2=0x7fca84150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca88c00640 1 --2- 172.21.15.46:0/3799954803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca840b3f20 0x7fca84150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca88c00640 1 -- 172.21.15.46:0/3799954803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c009420 con 0x7fca8415dbb0 2024-03-29T08:18:24.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca83e00640 1 --2- 172.21.15.46:0/3799954803 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca8406b4b0 0x7fca8414f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca88c00640 1 --2- 172.21.15.46:0/3799954803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8415dbb0 0x7fca8414e920 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00efc0 tx=0x7fca7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca80c00640 1 -- 172.21.15.46:0/3799954803 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c00cd20 con 0x7fca8415dbb0 2024-03-29T08:18:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca80c00640 1 -- 172.21.15.46:0/3799954803 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7c00f040 con 0x7fca8415dbb0 2024-03-29T08:18:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca8ac00640 1 -- 172.21.15.46:0/3799954803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca841553e0 con 0x7fca8415dbb0 2024-03-29T08:18:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca80c00640 1 -- 172.21.15.46:0/3799954803 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7c003be0 con 0x7fca8415dbb0 2024-03-29T08:18:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca8ac00640 1 -- 172.21.15.46:0/3799954803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca84155be0 con 0x7fca8415dbb0 2024-03-29T08:18:24.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca8ac00640 1 -- 172.21.15.46:0/3799954803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca8409d100 con 0x7fca8415dbb0 2024-03-29T08:18:24.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.068+0000 7fca80c00640 1 -- 172.21.15.46:0/3799954803 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fca7c003d40 con 0x7fca8415dbb0 2024-03-29T08:18:24.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.072+0000 7fca80c00640 1 --2- 172.21.15.46:0/3799954803 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca600666c0 0x7fca60068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.072+0000 7fca80c00640 1 -- 172.21.15.46:0/3799954803 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca7c0846d0 con 0x7fca8415dbb0 2024-03-29T08:18:24.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.072+0000 7fca83e00640 1 --2- 172.21.15.46:0/3799954803 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca600666c0 0x7fca60068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.072+0000 7fca83e00640 1 --2- 172.21.15.46:0/3799954803 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca600666c0 0x7fca60068b80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fca74007fd0 tx=0x7fca74009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:24.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.076+0000 7fca80c00640 1 -- 172.21.15.46:0/3799954803 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fca7c051a50 con 0x7fca8415dbb0 2024-03-29T08:18:24.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.236+0000 7fca8ac00640 1 -- 172.21.15.46:0/3799954803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fca84155890 con 0x7fca8415dbb0 2024-03-29T08:18:24.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.236+0000 7fca80c00640 1 -- 172.21.15.46:0/3799954803 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fca7c055070 con 0x7fca8415dbb0 2024-03-29T08:18:24.236 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:18:24.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.236+0000 7fca8ac00640 1 -- 172.21.15.46:0/3799954803 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca600666c0 msgr2=0x7fca60068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.236+0000 7fca8ac00640 1 --2- 172.21.15.46:0/3799954803 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca600666c0 0x7fca60068b80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fca74007fd0 tx=0x7fca74009040 comp rx=0 tx=0).stop 2024-03-29T08:18:24.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.236+0000 7fca8ac00640 1 -- 172.21.15.46:0/3799954803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8415dbb0 msgr2=0x7fca8414e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.236+0000 7fca8ac00640 1 --2- 172.21.15.46:0/3799954803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8415dbb0 0x7fca8414e920 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00efc0 tx=0x7fca7c00c490 comp rx=0 tx=0).stop 2024-03-29T08:18:24.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.236+0000 7fca8ac00640 1 -- 172.21.15.46:0/3799954803 shutdown_connections 2024-03-29T08:18:24.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.236+0000 7fca8ac00640 1 --2- 172.21.15.46:0/3799954803 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca600666c0 0x7fca60068b80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.236+0000 7fca8ac00640 1 --2- 172.21.15.46:0/3799954803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca840b3f20 0x7fca84150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.236+0000 7fca8ac00640 1 --2- 172.21.15.46:0/3799954803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8415dbb0 0x7fca8414e920 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.236+0000 7fca8ac00640 1 --2- 172.21.15.46:0/3799954803 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca8406b4b0 0x7fca8414f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.236+0000 7fca8ac00640 1 -- 172.21.15.46:0/3799954803 >> 172.21.15.46:0/3799954803 conn(0x7fca84070ad0 msgr2=0x7fca8415c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:24.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.236+0000 7fca8ac00640 1 -- 172.21.15.46:0/3799954803 shutdown_connections 2024-03-29T08:18:24.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.236+0000 7fca8ac00640 1 -- 172.21.15.46:0/3799954803 wait complete. 2024-03-29T08:18:24.258 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:18:24.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.396+0000 7f858f400640 1 Processor -- start 2024-03-29T08:18:24.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.396+0000 7f858f400640 1 -- start start 2024-03-29T08:18:24.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f858f400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8588098a40 0x7f858815ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f858f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f85880ce490 0x7f8588098e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f858f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8588099790 0x7f858815e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f858f400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8588099360 con 0x7f8588099790 2024-03-29T08:18:24.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f858f400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858815b370 con 0x7f85880ce490 2024-03-29T08:18:24.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f858f400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858815b650 con 0x7f8588098a40 2024-03-29T08:18:24.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f858d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8588099790 0x7f858815e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f858d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8588099790 0x7f858815e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60236/0 (socket says 172.21.15.46:60236) 2024-03-29T08:18:24.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f858d400640 1 -- 172.21.15.46:0/3383249569 learned_addr learned my addr 172.21.15.46:0/3383249569 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:24.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f858ca00640 1 --2- 172.21.15.46:0/3383249569 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8588098a40 0x7f858815ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f857fe00640 1 --2- 172.21.15.46:0/3383249569 >> v2:172.21.15.73:3300/0 conn(0x7f85880ce490 0x7f8588098e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f858d400640 1 -- 172.21.15.46:0/3383249569 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8588098a40 msgr2=0x7f858815ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:24.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f858d400640 1 --2- 172.21.15.46:0/3383249569 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8588098a40 0x7f858815ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f858d400640 1 -- 172.21.15.46:0/3383249569 >> v2:172.21.15.73:3300/0 conn(0x7f85880ce490 msgr2=0x7f8588098e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:24.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f858d400640 1 --2- 172.21.15.46:0/3383249569 >> v2:172.21.15.73:3300/0 conn(0x7f85880ce490 0x7f8588098e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f858d400640 1 -- 172.21.15.46:0/3383249569 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858815b930 con 0x7f8588099790 2024-03-29T08:18:24.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f858ca00640 1 --2- 172.21.15.46:0/3383249569 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8588098a40 0x7f858815ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:24.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f857fe00640 1 --2- 172.21.15.46:0/3383249569 >> v2:172.21.15.73:3300/0 conn(0x7f85880ce490 0x7f8588098e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:24.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.400+0000 7f858d400640 1 --2- 172.21.15.46:0/3383249569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8588099790 0x7f858815e590 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8588073700 tx=0x7f858001c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77de6ae376ff1376 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:24.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f8587e00640 1 -- 172.21.15.46:0/3383249569 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85800a64a0 con 0x7f8588099790 2024-03-29T08:18:24.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f8587e00640 1 -- 172.21.15.46:0/3383249569 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85800a6600 con 0x7f8588099790 2024-03-29T08:18:24.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f8587e00640 1 -- 172.21.15.46:0/3383249569 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85800a6990 con 0x7f8588099790 2024-03-29T08:18:24.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858f400640 1 -- 172.21.15.46:0/3383249569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8588099790 msgr2=0x7f858815e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858f400640 1 --2- 172.21.15.46:0/3383249569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8588099790 0x7f858815e590 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8588073700 tx=0x7f858001c700 comp rx=0 tx=0).stop 2024-03-29T08:18:24.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858f400640 1 -- 172.21.15.46:0/3383249569 shutdown_connections 2024-03-29T08:18:24.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858f400640 1 --2- 172.21.15.46:0/3383249569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8588099790 0x7f858815e590 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858f400640 1 --2- 172.21.15.46:0/3383249569 >> v2:172.21.15.73:3300/0 conn(0x7f85880ce490 0x7f8588098e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858f400640 1 --2- 172.21.15.46:0/3383249569 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8588098a40 0x7f858815ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858f400640 1 -- 172.21.15.46:0/3383249569 >> 172.21.15.46:0/3383249569 conn(0x7f85880b21a0 msgr2=0x7f85880b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858f400640 1 -- 172.21.15.46:0/3383249569 shutdown_connections 2024-03-29T08:18:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858f400640 1 -- 172.21.15.46:0/3383249569 wait complete. 2024-03-29T08:18:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858f400640 1 Processor -- start 2024-03-29T08:18:24.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858f400640 1 -- start start 2024-03-29T08:18:24.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858f400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8588098a40 0x7f8588159240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8588099790 0x7f85881f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85880ce490 0x7f85881f7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858f400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85881579a0 con 0x7f8588099790 2024-03-29T08:18:24.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858f400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85881573e0 con 0x7f85880ce490 2024-03-29T08:18:24.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858f400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85881576c0 con 0x7f8588098a40 2024-03-29T08:18:24.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f857fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8588099790 0x7f85881f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f857fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8588099790 0x7f85881f6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60246/0 (socket says 172.21.15.46:60246) 2024-03-29T08:18:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f857fe00640 1 -- 172.21.15.46:0/1348686336 learned_addr learned my addr 172.21.15.46:0/1348686336 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858ca00640 1 --2- 172.21.15.46:0/1348686336 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8588098a40 0x7f8588159240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.404+0000 7f858d400640 1 --2- 172.21.15.46:0/1348686336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85880ce490 0x7f85881f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.408+0000 7f857fe00640 1 -- 172.21.15.46:0/1348686336 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8588098a40 msgr2=0x7f8588159240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.408+0000 7f857fe00640 1 --2- 172.21.15.46:0/1348686336 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8588098a40 0x7f8588159240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.408+0000 7f857fe00640 1 -- 172.21.15.46:0/1348686336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85880ce490 msgr2=0x7f85881f7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.408+0000 7f857fe00640 1 --2- 172.21.15.46:0/1348686336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85880ce490 0x7f85881f7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.408+0000 7f857fe00640 1 -- 172.21.15.46:0/1348686336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8580009b30 con 0x7f8588099790 2024-03-29T08:18:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.408+0000 7f858d400640 1 --2- 172.21.15.46:0/1348686336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85880ce490 0x7f85881f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.408+0000 7f857fe00640 1 --2- 172.21.15.46:0/1348686336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8588099790 0x7f85881f6ae0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f857000d800 tx=0x7f857000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.408+0000 7f8585600640 1 -- 172.21.15.46:0/1348686336 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85700096c0 con 0x7f8588099790 2024-03-29T08:18:24.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.408+0000 7f8585600640 1 -- 172.21.15.46:0/1348686336 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8570009820 con 0x7f8588099790 2024-03-29T08:18:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.408+0000 7f8585600640 1 -- 172.21.15.46:0/1348686336 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8570009bd0 con 0x7f8588099790 2024-03-29T08:18:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.408+0000 7f858f400640 1 -- 172.21.15.46:0/1348686336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85880c4380 con 0x7f8588099790 2024-03-29T08:18:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.408+0000 7f858f400640 1 -- 172.21.15.46:0/1348686336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8588159780 con 0x7f8588099790 2024-03-29T08:18:24.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.408+0000 7f858f400640 1 -- 172.21.15.46:0/1348686336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8588066c90 con 0x7f8588099790 2024-03-29T08:18:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.408+0000 7f8585600640 1 -- 172.21.15.46:0/1348686336 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8570009d30 con 0x7f8588099790 2024-03-29T08:18:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.408+0000 7f8585600640 1 --2- 172.21.15.46:0/1348686336 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8564066690 0x7f8564068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.408+0000 7f8585600640 1 -- 172.21.15.46:0/1348686336 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85700847d0 con 0x7f8588099790 2024-03-29T08:18:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.408+0000 7f858ca00640 1 --2- 172.21.15.46:0/1348686336 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8564066690 0x7f8564068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.412+0000 7f858ca00640 1 --2- 172.21.15.46:0/1348686336 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8564066690 0x7f8564068b50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f85880cc1e0 tx=0x7f8578008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:24.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.412+0000 7f8585600640 1 -- 172.21.15.46:0/1348686336 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8570051b30 con 0x7f8588099790 2024-03-29T08:18:24.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.608+0000 7f858f400640 1 -- 172.21.15.46:0/1348686336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f858815e8b0 con 0x7f8588099790 2024-03-29T08:18:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.608+0000 7f8585600640 1 -- 172.21.15.46:0/1348686336 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4722 (secure 0 0 0) 0x7f8570055150 con 0x7f8588099790 2024-03-29T08:18:24.611 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:24.611 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/339038392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":339038392},{"type":"v1","addr":"172.21.15.46:6834","nonce":339038392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3573036715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3573036715},{"type":"v1","addr":"172.21.15.73:6835","nonce":3573036715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/746744819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":746744819},{"type":"v1","addr":"172.21.15.46:6835","nonce":746744819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:18:13.311457+0000","modified":"2024-03-29T08:18:16.321300+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2780269530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2780269530},{"type":"v1","addr":"172.21.15.73:6837","nonce":2780269530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-29T08:18:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-03-29T08:18:24.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.612+0000 7f858f400640 1 -- 172.21.15.46:0/1348686336 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8564066690 msgr2=0x7f8564068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.612+0000 7f858f400640 1 --2- 172.21.15.46:0/1348686336 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8564066690 0x7f8564068b50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f85880cc1e0 tx=0x7f8578008040 comp rx=0 tx=0).stop 2024-03-29T08:18:24.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.612+0000 7f858f400640 1 -- 172.21.15.46:0/1348686336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8588099790 msgr2=0x7f85881f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.612+0000 7f858f400640 1 --2- 172.21.15.46:0/1348686336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8588099790 0x7f85881f6ae0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f857000d800 tx=0x7f857000dcc0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.612+0000 7f858f400640 1 -- 172.21.15.46:0/1348686336 shutdown_connections 2024-03-29T08:18:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.612+0000 7f858f400640 1 --2- 172.21.15.46:0/1348686336 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8564066690 0x7f8564068b50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.612+0000 7f858f400640 1 --2- 172.21.15.46:0/1348686336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85880ce490 0x7f85881f7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.612+0000 7f858f400640 1 --2- 172.21.15.46:0/1348686336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8588099790 0x7f85881f6ae0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.612+0000 7f858f400640 1 --2- 172.21.15.46:0/1348686336 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8588098a40 0x7f8588159240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.612+0000 7f858f400640 1 -- 172.21.15.46:0/1348686336 >> 172.21.15.46:0/1348686336 conn(0x7f85880b21a0 msgr2=0x7f85880cf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.612+0000 7f858f400640 1 -- 172.21.15.46:0/1348686336 shutdown_connections 2024-03-29T08:18:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.612+0000 7f858f400640 1 -- 172.21.15.46:0/1348686336 wait complete. 2024-03-29T08:18:24.633 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:24.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.772+0000 7f2eb5200640 1 Processor -- start 2024-03-29T08:18:24.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.772+0000 7f2eb5200640 1 -- start start 2024-03-29T08:18:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.772+0000 7f2eb5200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2eb015a7c0 0x7f2eb006cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.772+0000 7f2eb5200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb006d130 con 0x7f2eb015cd90 2024-03-29T08:18:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.772+0000 7f2eb5200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb015b8f0 con 0x7f2eb01593d0 2024-03-29T08:18:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.772+0000 7f2eb5200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb015ad50 con 0x7f2eb015a7c0 2024-03-29T08:18:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.772+0000 7f2ead400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2eb015cd90 0x7f2eb015d940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32818/0 (socket says 172.21.15.46:32818) 2024-03-29T08:18:24.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.772+0000 7f2ead400640 1 -- 172.21.15.46:0/2368174715 learned_addr learned my addr 172.21.15.46:0/2368174715 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:24.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.772+0000 7f2eae800640 1 --2- 172.21.15.46:0/2368174715 >> v2:172.21.15.73:3300/0 conn(0x7f2eb015a7c0 0x7f2eb006cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.772+0000 7f2eaca00640 1 -- 172.21.15.46:0/2368174715 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868033670 0 0) 0x7f2eb006d130 con 0x7f2eb015cd90 2024-03-29T08:18:24.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.772+0000 7f2eaca00640 1 -- 172.21.15.46:0/2368174715 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e94005740 con 0x7f2eb015cd90 2024-03-29T08:18:24.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.772+0000 7f2eaca00640 1 -- 172.21.15.46:0/2368174715 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291326252 0 0) 0x7f2eb015b8f0 con 0x7f2eb01593d0 2024-03-29T08:18:24.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.772+0000 7f2eaca00640 1 -- 172.21.15.46:0/2368174715 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2eb006d130 con 0x7f2eb01593d0 2024-03-29T08:18:24.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.772+0000 7f2eaca00640 1 -- 172.21.15.46:0/2368174715 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 242766524 0 0) 0x7f2e94005740 con 0x7f2eb015cd90 2024-03-29T08:18:24.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.772+0000 7f2eae800640 1 -- 172.21.15.46:0/2368174715 >> v1:172.21.15.73:6789/0 conn(0x7f2eb01593d0 legacy=0x7f2eb015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.772+0000 7f2eae800640 1 -- 172.21.15.46:0/2368174715 >> v1:172.21.15.46:6789/0 conn(0x7f2eb015cd90 legacy=0x7f2eb015d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.772+0000 7f2eae800640 1 -- 172.21.15.46:0/2368174715 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb015e050 con 0x7f2eb015a7c0 2024-03-29T08:18:24.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.776+0000 7f2eae800640 1 --2- 172.21.15.46:0/2368174715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2eb015a7c0 0x7f2eb006cbf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009d90 tx=0x7f2ea801c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22c2190b5eec8521 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:24.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.776+0000 7f2eaca00640 1 -- 172.21.15.46:0/2368174715 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c001680 con 0x7f2eb015a7c0 2024-03-29T08:18:24.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.776+0000 7f2eaca00640 1 -- 172.21.15.46:0/2368174715 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea80a55d0 con 0x7f2eb015a7c0 2024-03-29T08:18:24.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.776+0000 7f2eaca00640 1 -- 172.21.15.46:0/2368174715 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea80a5960 con 0x7f2eb015a7c0 2024-03-29T08:18:24.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.776+0000 7f2eb5200640 1 -- 172.21.15.46:0/2368174715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2eb015a7c0 msgr2=0x7f2eb006cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.776+0000 7f2eb5200640 1 --2- 172.21.15.46:0/2368174715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2eb015a7c0 0x7f2eb006cbf0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009d90 tx=0x7f2ea801c890 comp rx=0 tx=0).stop 2024-03-29T08:18:24.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.776+0000 7f2eb5200640 1 -- 172.21.15.46:0/2368174715 shutdown_connections 2024-03-29T08:18:24.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.776+0000 7f2eb5200640 1 --2- 172.21.15.46:0/2368174715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2eb015a7c0 0x7f2eb006cbf0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.776+0000 7f2eb5200640 1 -- 172.21.15.46:0/2368174715 >> 172.21.15.46:0/2368174715 conn(0x7f2eb0066110 msgr2=0x7f2eb0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:24.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.776+0000 7f2eb5200640 1 -- 172.21.15.46:0/2368174715 shutdown_connections 2024-03-29T08:18:24.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.776+0000 7f2eb5200640 1 -- 172.21.15.46:0/2368174715 wait complete. 2024-03-29T08:18:24.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eb5200640 1 Processor -- start 2024-03-29T08:18:24.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eb5200640 1 -- start start 2024-03-29T08:18:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eb5200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2eb01593d0 0x7f2eb0146330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eb5200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2eb015a7c0 0x7f2eb014fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eb5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb015cd90 0x7f2eb00b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eb5200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb0146870 con 0x7f2eb015cd90 2024-03-29T08:18:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eb5200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb01f8790 con 0x7f2eb015a7c0 2024-03-29T08:18:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eb5200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb01f8a70 con 0x7f2eb01593d0 2024-03-29T08:18:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eae800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb015cd90 0x7f2eb00b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eae800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb015cd90 0x7f2eb00b1620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60252/0 (socket says 172.21.15.46:60252) 2024-03-29T08:18:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eae800640 1 -- 172.21.15.46:0/4216007494 learned_addr learned my addr 172.21.15.46:0/4216007494 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eade00640 1 --2- 172.21.15.46:0/4216007494 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2eb01593d0 0x7f2eb0146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2ead400640 1 --2- 172.21.15.46:0/4216007494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2eb015a7c0 0x7f2eb014fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eae800640 1 -- 172.21.15.46:0/4216007494 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2eb01593d0 msgr2=0x7f2eb0146330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eae800640 1 --2- 172.21.15.46:0/4216007494 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2eb01593d0 0x7f2eb0146330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eae800640 1 -- 172.21.15.46:0/4216007494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2eb015a7c0 msgr2=0x7f2eb014fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eae800640 1 --2- 172.21.15.46:0/4216007494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2eb015a7c0 0x7f2eb014fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eae800640 1 -- 172.21.15.46:0/4216007494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea8009a70 con 0x7f2eb015cd90 2024-03-29T08:18:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eae800640 1 --2- 172.21.15.46:0/4216007494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb015cd90 0x7f2eb00b1620 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80b0040 tx=0x7f2ea8019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2ea6000640 1 -- 172.21.15.46:0/4216007494 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8003ba0 con 0x7f2eb015cd90 2024-03-29T08:18:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2ea6000640 1 -- 172.21.15.46:0/4216007494 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea8003d00 con 0x7f2eb015cd90 2024-03-29T08:18:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2ea6000640 1 -- 172.21.15.46:0/4216007494 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8004260 con 0x7f2eb015cd90 2024-03-29T08:18:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eb5200640 1 -- 172.21.15.46:0/4216007494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb01f8d50 con 0x7f2eb015cd90 2024-03-29T08:18:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eb5200640 1 -- 172.21.15.46:0/4216007494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb014d560 con 0x7f2eb015cd90 2024-03-29T08:18:24.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.780+0000 7f2eb5200640 1 -- 172.21.15.46:0/4216007494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb01f9550 con 0x7f2eb015cd90 2024-03-29T08:18:24.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.784+0000 7f2ea6000640 1 -- 172.21.15.46:0/4216007494 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2ea80043c0 con 0x7f2eb015cd90 2024-03-29T08:18:24.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.784+0000 7f2ea6000640 1 --2- 172.21.15.46:0/4216007494 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e90066690 0x7f2e90068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:24.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.784+0000 7f2ea6000640 1 -- 172.21.15.46:0/4216007494 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ea811ee40 con 0x7f2eb015cd90 2024-03-29T08:18:24.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.784+0000 7f2eade00640 1 --2- 172.21.15.46:0/4216007494 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e90066690 0x7f2e90068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:24.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.784+0000 7f2eade00640 1 --2- 172.21.15.46:0/4216007494 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e90066690 0x7f2e90068b50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00011f0 tx=0x7f2ea0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:24.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.788+0000 7f2ea6000640 1 -- 172.21.15.46:0/4216007494 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2eb01f9550 con 0x7f2eb015cd90 2024-03-29T08:18:24.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.948+0000 7f2eb5200640 1 -- 172.21.15.46:0/4216007494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2eb01f9550 con 0x7f2eb015cd90 2024-03-29T08:18:24.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.952+0000 7f2ea6000640 1 -- 172.21.15.46:0/4216007494 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f2eb01f9550 con 0x7f2eb015cd90 2024-03-29T08:18:24.952 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:24.952 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:14.314124+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:24.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.952+0000 7f2eb5200640 1 -- 172.21.15.46:0/4216007494 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e90066690 msgr2=0x7f2e90068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.952+0000 7f2eb5200640 1 --2- 172.21.15.46:0/4216007494 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e90066690 0x7f2e90068b50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2ea00011f0 tx=0x7f2ea0009040 comp rx=0 tx=0).stop 2024-03-29T08:18:24.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.952+0000 7f2eb5200640 1 -- 172.21.15.46:0/4216007494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb015cd90 msgr2=0x7f2eb00b1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:24.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.952+0000 7f2eb5200640 1 --2- 172.21.15.46:0/4216007494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb015cd90 0x7f2eb00b1620 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80b0040 tx=0x7f2ea8019630 comp rx=0 tx=0).stop 2024-03-29T08:18:24.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.952+0000 7f2eb5200640 1 -- 172.21.15.46:0/4216007494 shutdown_connections 2024-03-29T08:18:24.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.952+0000 7f2eb5200640 1 --2- 172.21.15.46:0/4216007494 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e90066690 0x7f2e90068b50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.952+0000 7f2eb5200640 1 --2- 172.21.15.46:0/4216007494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2eb015a7c0 0x7f2eb014fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.952+0000 7f2eb5200640 1 --2- 172.21.15.46:0/4216007494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb015cd90 0x7f2eb00b1620 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.952+0000 7f2eb5200640 1 --2- 172.21.15.46:0/4216007494 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2eb01593d0 0x7f2eb0146330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:24.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.952+0000 7f2eb5200640 1 -- 172.21.15.46:0/4216007494 >> 172.21.15.46:0/4216007494 conn(0x7f2eb0066110 msgr2=0x7f2eb015d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:24.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.952+0000 7f2eb5200640 1 -- 172.21.15.46:0/4216007494 shutdown_connections 2024-03-29T08:18:24.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:24.952+0000 7f2eb5200640 1 -- 172.21.15.46:0/4216007494 wait complete. 2024-03-29T08:18:24.973 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:18:11.177842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:18:12.308252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:18:24.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:18:25.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.120+0000 7f8345400640 1 Processor -- start 2024-03-29T08:18:25.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.120+0000 7f8345400640 1 -- start start 2024-03-29T08:18:25.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.120+0000 7f8345400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8340142a50 0x7f83400cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.120+0000 7f8345400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f834015df90 0x7f83400cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.120+0000 7f8345400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f834015d260 con 0x7f83400ae280 2024-03-29T08:18:25.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.120+0000 7f8345400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83400aefd0 con 0x7f834015df90 2024-03-29T08:18:25.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.120+0000 7f8345400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f834015d4b0 con 0x7f8340142a50 2024-03-29T08:18:25.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.120+0000 7f8337e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f834015df90 0x7f83400cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.120+0000 7f833f200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f83400ae280 0x7f83400ae710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32820/0 (socket says 172.21.15.46:32820) 2024-03-29T08:18:25.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.120+0000 7f833f200640 1 -- 172.21.15.46:0/737187764 learned_addr learned my addr 172.21.15.46:0/737187764 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:25.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.120+0000 7f833e800640 1 --2- 172.21.15.46:0/737187764 >> v2:172.21.15.73:3300/0 conn(0x7f8340142a50 0x7f83400cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.120+0000 7f8337e00640 1 -- 172.21.15.46:0/737187764 >> v2:172.21.15.73:3300/0 conn(0x7f8340142a50 msgr2=0x7f83400cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.120+0000 7f8337e00640 1 --2- 172.21.15.46:0/737187764 >> v2:172.21.15.73:3300/0 conn(0x7f8340142a50 0x7f83400cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.120+0000 7f8337e00640 1 -- 172.21.15.46:0/737187764 >> v1:172.21.15.46:6789/0 conn(0x7f83400ae280 legacy=0x7f83400ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.120+0000 7f8337e00640 1 -- 172.21.15.46:0/737187764 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834015d790 con 0x7f834015df90 2024-03-29T08:18:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.120+0000 7f833e800640 1 --2- 172.21.15.46:0/737187764 >> v2:172.21.15.73:3300/0 conn(0x7f8340142a50 0x7f83400cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.120+0000 7f833de00640 1 -- 172.21.15.46:0/737187764 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323210472 0 0) 0x7f834015d260 con 0x7f83400ae280 2024-03-29T08:18:25.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.124+0000 7f8337e00640 1 --2- 172.21.15.46:0/737187764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f834015df90 0x7f83400cbcd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f834006d8f0 tx=0x7f832801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee44dc599cad44e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:25.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.124+0000 7f833de00640 1 -- 172.21.15.46:0/737187764 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83280ab600 con 0x7f834015df90 2024-03-29T08:18:25.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.124+0000 7f833de00640 1 -- 172.21.15.46:0/737187764 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83280a7050 con 0x7f834015df90 2024-03-29T08:18:25.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.124+0000 7f833de00640 1 -- 172.21.15.46:0/737187764 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83280ab980 con 0x7f834015df90 2024-03-29T08:18:25.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.124+0000 7f8345400640 1 -- 172.21.15.46:0/737187764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f834015df90 msgr2=0x7f83400cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.124+0000 7f8345400640 1 --2- 172.21.15.46:0/737187764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f834015df90 0x7f83400cbcd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f834006d8f0 tx=0x7f832801c3f0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.124+0000 7f8345400640 1 -- 172.21.15.46:0/737187764 shutdown_connections 2024-03-29T08:18:25.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.124+0000 7f8345400640 1 --2- 172.21.15.46:0/737187764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f834015df90 0x7f83400cbcd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.124+0000 7f8345400640 1 --2- 172.21.15.46:0/737187764 >> v2:172.21.15.73:3300/0 conn(0x7f8340142a50 0x7f83400cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.124+0000 7f8345400640 1 -- 172.21.15.46:0/737187764 >> 172.21.15.46:0/737187764 conn(0x7f834006a390 msgr2=0x7f83400cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:25.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.124+0000 7f8345400640 1 -- 172.21.15.46:0/737187764 shutdown_connections 2024-03-29T08:18:25.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.124+0000 7f8345400640 1 -- 172.21.15.46:0/737187764 wait complete. 2024-03-29T08:18:25.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.124+0000 7f8345400640 1 Processor -- start 2024-03-29T08:18:25.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.124+0000 7f8345400640 1 -- start start 2024-03-29T08:18:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8345400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83400ae280 0x7f83400cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8345400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8340142a50 0x7f834013bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8345400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f834015df90 0x7f834013c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8345400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83401657d0 con 0x7f8340142a50 2024-03-29T08:18:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8345400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8340166350 con 0x7f83400ae280 2024-03-29T08:18:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8345400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8340165a20 con 0x7f834015df90 2024-03-29T08:18:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8337e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8340142a50 0x7f834013bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8337e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8340142a50 0x7f834013bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60270/0 (socket says 172.21.15.46:60270) 2024-03-29T08:18:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8337e00640 1 -- 172.21.15.46:0/814761527 learned_addr learned my addr 172.21.15.46:0/814761527 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f833e800640 1 --2- 172.21.15.46:0/814761527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83400ae280 0x7f83400cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f833f200640 1 --2- 172.21.15.46:0/814761527 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f834015df90 0x7f834013c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8337e00640 1 -- 172.21.15.46:0/814761527 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f834015df90 msgr2=0x7f834013c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8337e00640 1 --2- 172.21.15.46:0/814761527 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f834015df90 0x7f834013c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8337e00640 1 -- 172.21.15.46:0/814761527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83400ae280 msgr2=0x7f83400cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8337e00640 1 --2- 172.21.15.46:0/814761527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83400ae280 0x7f83400cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8337e00640 1 -- 172.21.15.46:0/814761527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8328009420 con 0x7f8340142a50 2024-03-29T08:18:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f833e800640 1 --2- 172.21.15.46:0/814761527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83400ae280 0x7f83400cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8337e00640 1 --2- 172.21.15.46:0/814761527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8340142a50 0x7f834013bc40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f832801a020 tx=0x7f8328019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8336a00640 1 -- 172.21.15.46:0/814761527 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83280b4840 con 0x7f8340142a50 2024-03-29T08:18:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8336a00640 1 -- 172.21.15.46:0/814761527 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83280b49c0 con 0x7f8340142a50 2024-03-29T08:18:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8336a00640 1 -- 172.21.15.46:0/814761527 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83280b4d70 con 0x7f8340142a50 2024-03-29T08:18:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8345400640 1 -- 172.21.15.46:0/814761527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8340165d00 con 0x7f8340142a50 2024-03-29T08:18:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8345400640 1 -- 172.21.15.46:0/814761527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83400ce040 con 0x7f8340142a50 2024-03-29T08:18:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8345400640 1 -- 172.21.15.46:0/814761527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83401664e0 con 0x7f8340142a50 2024-03-29T08:18:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.128+0000 7f8336a00640 1 -- 172.21.15.46:0/814761527 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f83280b2040 con 0x7f8340142a50 2024-03-29T08:18:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.132+0000 7f8336a00640 1 --2- 172.21.15.46:0/814761527 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f831c066690 0x7f831c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.132+0000 7f8336a00640 1 -- 172.21.15.46:0/814761527 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f832811e500 con 0x7f8340142a50 2024-03-29T08:18:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.132+0000 7f833e800640 1 --2- 172.21.15.46:0/814761527 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f831c066690 0x7f831c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.132+0000 7f833e800640 1 --2- 172.21.15.46:0/814761527 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f831c066690 0x7f831c068b50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f83400ca880 tx=0x7f8330008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.136+0000 7f8336a00640 1 -- 172.21.15.46:0/814761527 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f83280eb860 con 0x7f8340142a50 2024-03-29T08:18:25.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.296+0000 7f8345400640 1 -- 172.21.15.46:0/814761527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83400ce1d0 con 0x7f8340142a50 2024-03-29T08:18:25.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.296+0000 7f8336a00640 1 -- 172.21.15.46:0/814761527 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83280eee80 con 0x7f8340142a50 2024-03-29T08:18:25.297 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:18:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.296+0000 7f8345400640 1 -- 172.21.15.46:0/814761527 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f831c066690 msgr2=0x7f831c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.296+0000 7f8345400640 1 --2- 172.21.15.46:0/814761527 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f831c066690 0x7f831c068b50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f83400ca880 tx=0x7f8330008040 comp rx=0 tx=0).stop 2024-03-29T08:18:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.296+0000 7f8345400640 1 -- 172.21.15.46:0/814761527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8340142a50 msgr2=0x7f834013bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.296+0000 7f8345400640 1 --2- 172.21.15.46:0/814761527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8340142a50 0x7f834013bc40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f832801a020 tx=0x7f8328019190 comp rx=0 tx=0).stop 2024-03-29T08:18:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.296+0000 7f8345400640 1 -- 172.21.15.46:0/814761527 shutdown_connections 2024-03-29T08:18:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.296+0000 7f8345400640 1 --2- 172.21.15.46:0/814761527 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f831c066690 0x7f831c068b50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.296+0000 7f8345400640 1 --2- 172.21.15.46:0/814761527 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f834015df90 0x7f834013c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.296+0000 7f8345400640 1 --2- 172.21.15.46:0/814761527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8340142a50 0x7f834013bc40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.296+0000 7f8345400640 1 --2- 172.21.15.46:0/814761527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83400ae280 0x7f83400cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.296+0000 7f8345400640 1 -- 172.21.15.46:0/814761527 >> 172.21.15.46:0/814761527 conn(0x7f834006a390 msgr2=0x7f83400ae6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.296+0000 7f8345400640 1 -- 172.21.15.46:0/814761527 shutdown_connections 2024-03-29T08:18:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.296+0000 7f8345400640 1 -- 172.21.15.46:0/814761527 wait complete. 2024-03-29T08:18:25.318 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:18:25.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.460+0000 7eff2d600640 1 Processor -- start 2024-03-29T08:18:25.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.460+0000 7eff2d600640 1 -- start start 2024-03-29T08:18:25.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.460+0000 7eff2d600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7eff2815cb60 0x7eff280d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.460+0000 7eff2d600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7eff280cf9f0 0x7eff280b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.460+0000 7eff2d600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff2815cf40 con 0x7eff280d0bf0 2024-03-29T08:18:25.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.460+0000 7eff2d600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff280b0250 con 0x7eff280cf9f0 2024-03-29T08:18:25.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.460+0000 7eff2d600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff280b0810 con 0x7eff2815cb60 2024-03-29T08:18:25.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.460+0000 7eff27400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7eff280cf9f0 0x7eff280b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.460+0000 7eff26a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7eff2815cb60 0x7eff280d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.460+0000 7eff26a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7eff2815cb60 0x7eff280d06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:58016/0 (socket says 172.21.15.46:58016) 2024-03-29T08:18:25.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.460+0000 7eff26a00640 1 -- 172.21.15.46:0/2799107211 learned_addr learned my addr 172.21.15.46:0/2799107211 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:25.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.460+0000 7eff26a00640 1 -- 172.21.15.46:0/2799107211 >> v2:172.21.15.73:3300/0 conn(0x7eff280cf9f0 msgr2=0x7eff280b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.460+0000 7eff26a00640 1 --2- 172.21.15.46:0/2799107211 >> v2:172.21.15.73:3300/0 conn(0x7eff280cf9f0 0x7eff280b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.460+0000 7eff26a00640 1 -- 172.21.15.46:0/2799107211 >> v1:172.21.15.73:6789/0 conn(0x7eff280d0bf0 legacy=0x7eff280cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.460+0000 7eff26a00640 1 -- 172.21.15.46:0/2799107211 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff280d1050 con 0x7eff2815cb60 2024-03-29T08:18:25.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.464+0000 7eff26a00640 1 --2- 172.21.15.46:0/2799107211 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7eff2815cb60 0x7eff280d06b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7eff18009e50 tx=0x7eff1801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7839bd9bccb966a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.464+0000 7eff26000640 1 -- 172.21.15.46:0/2799107211 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff180a5900 con 0x7eff2815cb60 2024-03-29T08:18:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.464+0000 7eff26000640 1 -- 172.21.15.46:0/2799107211 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff180a5a60 con 0x7eff2815cb60 2024-03-29T08:18:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.464+0000 7eff26000640 1 -- 172.21.15.46:0/2799107211 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff180a5df0 con 0x7eff2815cb60 2024-03-29T08:18:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.464+0000 7eff2d600640 1 -- 172.21.15.46:0/2799107211 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7eff2815cb60 msgr2=0x7eff280d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.464+0000 7eff2d600640 1 --2- 172.21.15.46:0/2799107211 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7eff2815cb60 0x7eff280d06b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7eff18009e50 tx=0x7eff1801ca30 comp rx=0 tx=0).stop 2024-03-29T08:18:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.464+0000 7eff2d600640 1 -- 172.21.15.46:0/2799107211 shutdown_connections 2024-03-29T08:18:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.464+0000 7eff2d600640 1 --2- 172.21.15.46:0/2799107211 >> v2:172.21.15.73:3300/0 conn(0x7eff280cf9f0 0x7eff280b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.464+0000 7eff2d600640 1 --2- 172.21.15.46:0/2799107211 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7eff2815cb60 0x7eff280d06b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.464+0000 7eff2d600640 1 -- 172.21.15.46:0/2799107211 >> 172.21.15.46:0/2799107211 conn(0x7eff28099a10 msgr2=0x7eff28089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:25.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.464+0000 7eff2d600640 1 -- 172.21.15.46:0/2799107211 shutdown_connections 2024-03-29T08:18:25.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.464+0000 7eff2d600640 1 -- 172.21.15.46:0/2799107211 wait complete. 2024-03-29T08:18:25.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff2d600640 1 Processor -- start 2024-03-29T08:18:25.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff2d600640 1 -- start start 2024-03-29T08:18:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff2d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff280cf9f0 0x7eff280b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff2d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff280d0bf0 0x7eff281556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff2d600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7eff2815cb60 0x7eff28155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff2d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2814bd40 con 0x7eff280d0bf0 2024-03-29T08:18:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff2d600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2806ef60 con 0x7eff280cf9f0 2024-03-29T08:18:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff2d600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2806f1b0 con 0x7eff2815cb60 2024-03-29T08:18:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff1e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff280d0bf0 0x7eff281556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff1e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff280d0bf0 0x7eff281556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60280/0 (socket says 172.21.15.46:60280) 2024-03-29T08:18:25.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff1e000640 1 -- 172.21.15.46:0/1486093100 learned_addr learned my addr 172.21.15.46:0/1486093100 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff26a00640 1 --2- 172.21.15.46:0/1486093100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff280cf9f0 0x7eff280b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff27400640 1 --2- 172.21.15.46:0/1486093100 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7eff2815cb60 0x7eff28155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff1e000640 1 -- 172.21.15.46:0/1486093100 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7eff2815cb60 msgr2=0x7eff28155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff1e000640 1 --2- 172.21.15.46:0/1486093100 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7eff2815cb60 0x7eff28155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff1e000640 1 -- 172.21.15.46:0/1486093100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff280cf9f0 msgr2=0x7eff280b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff1e000640 1 --2- 172.21.15.46:0/1486093100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff280cf9f0 0x7eff280b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff1e000640 1 -- 172.21.15.46:0/1486093100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff18009b30 con 0x7eff280d0bf0 2024-03-29T08:18:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff27400640 1 --2- 172.21.15.46:0/1486093100 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7eff2815cb60 0x7eff28155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff1e000640 1 --2- 172.21.15.46:0/1486093100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff280d0bf0 0x7eff281556e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7eff1000c420 tx=0x7eff1000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff1f400640 1 -- 172.21.15.46:0/1486093100 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff100052d0 con 0x7eff280d0bf0 2024-03-29T08:18:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff1f400640 1 -- 172.21.15.46:0/1486093100 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff10005430 con 0x7eff280d0bf0 2024-03-29T08:18:25.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff1f400640 1 -- 172.21.15.46:0/1486093100 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff10007260 con 0x7eff280d0bf0 2024-03-29T08:18:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff2d600640 1 -- 172.21.15.46:0/1486093100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2806f490 con 0x7eff280d0bf0 2024-03-29T08:18:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.468+0000 7eff2d600640 1 -- 172.21.15.46:0/1486093100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff28159e30 con 0x7eff280d0bf0 2024-03-29T08:18:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.472+0000 7eff2d600640 1 -- 172.21.15.46:0/1486093100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2809e7d0 con 0x7eff280d0bf0 2024-03-29T08:18:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.472+0000 7eff1f400640 1 -- 172.21.15.46:0/1486093100 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7eff100073c0 con 0x7eff280d0bf0 2024-03-29T08:18:25.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.472+0000 7eff1f400640 1 --2- 172.21.15.46:0/1486093100 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7eff04066690 0x7eff04068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.472+0000 7eff1f400640 1 -- 172.21.15.46:0/1486093100 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff10083600 con 0x7eff280d0bf0 2024-03-29T08:18:25.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.472+0000 7eff26a00640 1 --2- 172.21.15.46:0/1486093100 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7eff04066690 0x7eff04068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.472+0000 7eff26a00640 1 --2- 172.21.15.46:0/1486093100 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7eff04066690 0x7eff04068b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff180ae000 tx=0x7eff180a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:25.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.476+0000 7eff1f400640 1 -- 172.21.15.46:0/1486093100 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7eff100508d0 con 0x7eff280d0bf0 2024-03-29T08:18:25.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.640+0000 7eff2d600640 1 -- 172.21.15.46:0/1486093100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff2804e120 con 0x7eff280d0bf0 2024-03-29T08:18:25.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.640+0000 7eff1f400640 1 -- 172.21.15.46:0/1486093100 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7eff10053ef0 con 0x7eff280d0bf0 2024-03-29T08:18:25.642 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:18:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.644+0000 7eff2d600640 1 -- 172.21.15.46:0/1486093100 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7eff04066690 msgr2=0x7eff04068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.644+0000 7eff2d600640 1 --2- 172.21.15.46:0/1486093100 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7eff04066690 0x7eff04068b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff180ae000 tx=0x7eff180a7070 comp rx=0 tx=0).stop 2024-03-29T08:18:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.644+0000 7eff2d600640 1 -- 172.21.15.46:0/1486093100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff280d0bf0 msgr2=0x7eff281556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.644+0000 7eff2d600640 1 --2- 172.21.15.46:0/1486093100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff280d0bf0 0x7eff281556e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7eff1000c420 tx=0x7eff1000c8e0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.644+0000 7eff2d600640 1 -- 172.21.15.46:0/1486093100 shutdown_connections 2024-03-29T08:18:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.644+0000 7eff2d600640 1 --2- 172.21.15.46:0/1486093100 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7eff04066690 0x7eff04068b50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.644+0000 7eff2d600640 1 --2- 172.21.15.46:0/1486093100 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7eff2815cb60 0x7eff28155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.644+0000 7eff2d600640 1 --2- 172.21.15.46:0/1486093100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff280d0bf0 0x7eff281556e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.644+0000 7eff2d600640 1 --2- 172.21.15.46:0/1486093100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff280cf9f0 0x7eff280b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.644+0000 7eff2d600640 1 -- 172.21.15.46:0/1486093100 >> 172.21.15.46:0/1486093100 conn(0x7eff28099a10 msgr2=0x7eff2806e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:25.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.644+0000 7eff2d600640 1 -- 172.21.15.46:0/1486093100 shutdown_connections 2024-03-29T08:18:25.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.644+0000 7eff2d600640 1 -- 172.21.15.46:0/1486093100 wait complete. 2024-03-29T08:18:25.664 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:18:25.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.804+0000 7fc73dc00640 1 Processor -- start 2024-03-29T08:18:25.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.804+0000 7fc73dc00640 1 -- start start 2024-03-29T08:18:25.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.804+0000 7fc73dc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc73815df90 0x7fc7380ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.804+0000 7fc73dc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc7380cbea0 0x7fc7380ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.804+0000 7fc73dc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc73815d260 con 0x7fc738142a50 2024-03-29T08:18:25.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.804+0000 7fc73dc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7380aed00 con 0x7fc73815df90 2024-03-29T08:18:25.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.804+0000 7fc73dc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7380aefd0 con 0x7fc7380cbea0 2024-03-29T08:18:25.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.804+0000 7fc72e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc73815df90 0x7fc7380ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.808+0000 7fc72e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc73815df90 0x7fc7380ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60290/0 (socket says 172.21.15.46:60290) 2024-03-29T08:18:25.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.808+0000 7fc72e000640 1 -- 172.21.15.46:0/2576635333 learned_addr learned my addr 172.21.15.46:0/2576635333 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:25.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.808+0000 7fc736a00640 1 --1- 172.21.15.46:0/2576635333 >> v1:172.21.15.46:6789/0 conn(0x7fc738142a50 0x7fc7380cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32826/0 (socket says 172.21.15.46:32826) 2024-03-29T08:18:25.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.808+0000 7fc737400640 1 --2- 172.21.15.46:0/2576635333 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc7380cbea0 0x7fc7380ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.808+0000 7fc72e000640 1 -- 172.21.15.46:0/2576635333 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc7380cbea0 msgr2=0x7fc7380ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:25.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.808+0000 7fc72e000640 1 --2- 172.21.15.46:0/2576635333 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc7380cbea0 0x7fc7380ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.808+0000 7fc72e000640 1 -- 172.21.15.46:0/2576635333 >> v1:172.21.15.46:6789/0 conn(0x7fc738142a50 legacy=0x7fc7380cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.808+0000 7fc72e000640 1 -- 172.21.15.46:0/2576635333 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73815d4b0 con 0x7fc73815df90 2024-03-29T08:18:25.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.808+0000 7fc72e000640 1 --2- 172.21.15.46:0/2576635333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73815df90 0x7fc7380ae280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc720009e50 tx=0x7fc72001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334cf4c9b582d850 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.808+0000 7fc736000640 1 -- 172.21.15.46:0/2576635333 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7200a5640 con 0x7fc73815df90 2024-03-29T08:18:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.808+0000 7fc736000640 1 -- 172.21.15.46:0/2576635333 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7200a57a0 con 0x7fc73815df90 2024-03-29T08:18:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.808+0000 7fc736000640 1 -- 172.21.15.46:0/2576635333 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7200a5b30 con 0x7fc73815df90 2024-03-29T08:18:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc73dc00640 1 -- 172.21.15.46:0/2576635333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73815df90 msgr2=0x7fc7380ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc73dc00640 1 --2- 172.21.15.46:0/2576635333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73815df90 0x7fc7380ae280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc720009e50 tx=0x7fc72001c9a0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc73dc00640 1 -- 172.21.15.46:0/2576635333 shutdown_connections 2024-03-29T08:18:25.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc73dc00640 1 --2- 172.21.15.46:0/2576635333 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc7380cbea0 0x7fc7380ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc73dc00640 1 --2- 172.21.15.46:0/2576635333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73815df90 0x7fc7380ae280 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc73dc00640 1 -- 172.21.15.46:0/2576635333 >> 172.21.15.46:0/2576635333 conn(0x7fc73806a390 msgr2=0x7fc7380cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:25.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc73dc00640 1 -- 172.21.15.46:0/2576635333 shutdown_connections 2024-03-29T08:18:25.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc73dc00640 1 -- 172.21.15.46:0/2576635333 wait complete. 2024-03-29T08:18:25.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc73dc00640 1 Processor -- start 2024-03-29T08:18:25.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc73dc00640 1 -- start start 2024-03-29T08:18:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc73dc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7380cbea0 0x7fc738144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc73dc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc738142a50 0x7fc73806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc73dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73815df90 0x7fc7380a9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc73dc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc738143ba0 con 0x7fc73815df90 2024-03-29T08:18:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc73dc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc738147770 con 0x7fc7380cbea0 2024-03-29T08:18:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc73dc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc738147490 con 0x7fc738142a50 2024-03-29T08:18:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc737400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73815df90 0x7fc7380a9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc737400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73815df90 0x7fc7380a9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60300/0 (socket says 172.21.15.46:60300) 2024-03-29T08:18:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc737400640 1 -- 172.21.15.46:0/2835870464 learned_addr learned my addr 172.21.15.46:0/2835870464 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:25.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc736a00640 1 --2- 172.21.15.46:0/2835870464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7380cbea0 0x7fc738144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc72e000640 1 --2- 172.21.15.46:0/2835870464 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc738142a50 0x7fc73806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc737400640 1 -- 172.21.15.46:0/2835870464 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc738142a50 msgr2=0x7fc73806c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc737400640 1 --2- 172.21.15.46:0/2835870464 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc738142a50 0x7fc73806c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc737400640 1 -- 172.21.15.46:0/2835870464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7380cbea0 msgr2=0x7fc738144030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc737400640 1 --2- 172.21.15.46:0/2835870464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7380cbea0 0x7fc738144030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc737400640 1 -- 172.21.15.46:0/2835870464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc720009b30 con 0x7fc73815df90 2024-03-29T08:18:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc736a00640 1 --2- 172.21.15.46:0/2835870464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7380cbea0 0x7fc738144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc72e000640 1 --2- 172.21.15.46:0/2835870464 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc738142a50 0x7fc73806c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.812+0000 7fc737400640 1 --2- 172.21.15.46:0/2835870464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73815df90 0x7fc7380a9e70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc73000e7f0 tx=0x7fc73000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.816+0000 7fc72f400640 1 -- 172.21.15.46:0/2835870464 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7300094d0 con 0x7fc73815df90 2024-03-29T08:18:25.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.816+0000 7fc72f400640 1 -- 172.21.15.46:0/2835870464 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73000eea0 con 0x7fc73815df90 2024-03-29T08:18:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.816+0000 7fc72f400640 1 -- 172.21.15.46:0/2835870464 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73000cb60 con 0x7fc73815df90 2024-03-29T08:18:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.816+0000 7fc73dc00640 1 -- 172.21.15.46:0/2835870464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc738143550 con 0x7fc73815df90 2024-03-29T08:18:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.816+0000 7fc73dc00640 1 -- 172.21.15.46:0/2835870464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73806cb10 con 0x7fc73815df90 2024-03-29T08:18:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.816+0000 7fc73dc00640 1 -- 172.21.15.46:0/2835870464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc738143d50 con 0x7fc73815df90 2024-03-29T08:18:25.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.816+0000 7fc72f400640 1 -- 172.21.15.46:0/2835870464 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc73000ccc0 con 0x7fc73815df90 2024-03-29T08:18:25.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.816+0000 7fc72f400640 1 --2- 172.21.15.46:0/2835870464 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc714066690 0x7fc714068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:25.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.816+0000 7fc72f400640 1 -- 172.21.15.46:0/2835870464 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc730084670 con 0x7fc73815df90 2024-03-29T08:18:25.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.816+0000 7fc736a00640 1 --2- 172.21.15.46:0/2835870464 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc714066690 0x7fc714068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:25.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.816+0000 7fc736a00640 1 --2- 172.21.15.46:0/2835870464 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc714066690 0x7fc714068b50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc728000f70 tx=0x7fc728007d50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.820+0000 7fc72f400640 1 -- 172.21.15.46:0/2835870464 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc738143d50 con 0x7fc73815df90 2024-03-29T08:18:25.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.980+0000 7fc73dc00640 1 -- 172.21.15.46:0/2835870464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc738143d50 con 0x7fc73815df90 2024-03-29T08:18:25.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.980+0000 7fc72f400640 1 -- 172.21.15.46:0/2835870464 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc738143d50 con 0x7fc73815df90 2024-03-29T08:18:25.983 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:18:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.984+0000 7fc73dc00640 1 -- 172.21.15.46:0/2835870464 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc714066690 msgr2=0x7fc714068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.984+0000 7fc73dc00640 1 --2- 172.21.15.46:0/2835870464 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc714066690 0x7fc714068b50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc728000f70 tx=0x7fc728007d50 comp rx=0 tx=0).stop 2024-03-29T08:18:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.984+0000 7fc73dc00640 1 -- 172.21.15.46:0/2835870464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73815df90 msgr2=0x7fc7380a9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.984+0000 7fc73dc00640 1 --2- 172.21.15.46:0/2835870464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73815df90 0x7fc7380a9e70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc73000e7f0 tx=0x7fc73000ecb0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.984+0000 7fc73dc00640 1 -- 172.21.15.46:0/2835870464 shutdown_connections 2024-03-29T08:18:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.984+0000 7fc73dc00640 1 --2- 172.21.15.46:0/2835870464 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc714066690 0x7fc714068b50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.984+0000 7fc73dc00640 1 --2- 172.21.15.46:0/2835870464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc73815df90 0x7fc7380a9e70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.984+0000 7fc73dc00640 1 --2- 172.21.15.46:0/2835870464 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc738142a50 0x7fc73806c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.984+0000 7fc73dc00640 1 --2- 172.21.15.46:0/2835870464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7380cbea0 0x7fc738144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.984+0000 7fc73dc00640 1 -- 172.21.15.46:0/2835870464 >> 172.21.15.46:0/2835870464 conn(0x7fc73806a390 msgr2=0x7fc73813f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.984+0000 7fc73dc00640 1 -- 172.21.15.46:0/2835870464 shutdown_connections 2024-03-29T08:18:25.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:25.984+0000 7fc73dc00640 1 -- 172.21.15.46:0/2835870464 wait complete. 2024-03-29T08:18:26.004 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:18:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.144+0000 7f0cc8a00640 1 Processor -- start 2024-03-29T08:18:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.144+0000 7f0cc8a00640 1 -- start start 2024-03-29T08:18:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.144+0000 7f0cc8a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cc41569a0 0x7f0cc406c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.144+0000 7f0cc8a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0cc40708d0 0x7f0cc41596c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.144+0000 7f0cc8a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc415d7c0 con 0x7f0cc406d910 2024-03-29T08:18:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.144+0000 7f0cc8a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc40b21d0 con 0x7f0cc41569a0 2024-03-29T08:18:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.144+0000 7f0cc8a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc415cc40 con 0x7f0cc40708d0 2024-03-29T08:18:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.144+0000 7f0cc1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cc41569a0 0x7f0cc406c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.144+0000 7f0cc1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cc41569a0 0x7f0cc406c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60316/0 (socket says 172.21.15.46:60316) 2024-03-29T08:18:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.144+0000 7f0cc1e00640 1 -- 172.21.15.46:0/4091171872 learned_addr learned my addr 172.21.15.46:0/4091171872 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.144+0000 7f0cbbe00640 1 --1- 172.21.15.46:0/4091171872 >> v1:172.21.15.46:6789/0 conn(0x7f0cc406d910 0x7f0cc40701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:32834/0 (socket says 172.21.15.46:32834) 2024-03-29T08:18:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.144+0000 7f0cc2800640 1 --2- 172.21.15.46:0/4091171872 >> v2:172.21.15.73:3300/0 conn(0x7f0cc40708d0 0x7f0cc41596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:26.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.148+0000 7f0cc1e00640 1 -- 172.21.15.46:0/4091171872 >> v2:172.21.15.73:3300/0 conn(0x7f0cc40708d0 msgr2=0x7f0cc41596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:26.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.148+0000 7f0cc1e00640 1 --2- 172.21.15.46:0/4091171872 >> v2:172.21.15.73:3300/0 conn(0x7f0cc40708d0 0x7f0cc41596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.148+0000 7f0cc1e00640 1 -- 172.21.15.46:0/4091171872 >> v1:172.21.15.46:6789/0 conn(0x7f0cc406d910 legacy=0x7f0cc40701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.148+0000 7f0cc1400640 1 -- 172.21.15.46:0/4091171872 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2565345737 0 0) 0x7f0cc415d7c0 con 0x7f0cc406d910 2024-03-29T08:18:26.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.148+0000 7f0cc1e00640 1 -- 172.21.15.46:0/4091171872 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc415cf20 con 0x7f0cc41569a0 2024-03-29T08:18:26.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.148+0000 7f0cc1e00640 1 --2- 172.21.15.46:0/4091171872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc41569a0 0x7f0cc406c1e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009870 tx=0x7f0cb401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ac5cd341567c54 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:26.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.148+0000 7f0cc1400640 1 -- 172.21.15.46:0/4091171872 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb40ab600 con 0x7f0cc41569a0 2024-03-29T08:18:26.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.148+0000 7f0cc1400640 1 -- 172.21.15.46:0/4091171872 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb40a7050 con 0x7f0cc41569a0 2024-03-29T08:18:26.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.148+0000 7f0cc1400640 1 -- 172.21.15.46:0/4091171872 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb40ab980 con 0x7f0cc41569a0 2024-03-29T08:18:26.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.148+0000 7f0cc8a00640 1 -- 172.21.15.46:0/4091171872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc41569a0 msgr2=0x7f0cc406c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.148+0000 7f0cc8a00640 1 --2- 172.21.15.46:0/4091171872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc41569a0 0x7f0cc406c1e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4009870 tx=0x7f0cb401c520 comp rx=0 tx=0).stop 2024-03-29T08:18:26.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.148+0000 7f0cc8a00640 1 -- 172.21.15.46:0/4091171872 shutdown_connections 2024-03-29T08:18:26.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.148+0000 7f0cc8a00640 1 --2- 172.21.15.46:0/4091171872 >> v2:172.21.15.73:3300/0 conn(0x7f0cc40708d0 0x7f0cc41596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.148+0000 7f0cc8a00640 1 --2- 172.21.15.46:0/4091171872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc41569a0 0x7f0cc406c1e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.148+0000 7f0cc8a00640 1 -- 172.21.15.46:0/4091171872 >> 172.21.15.46:0/4091171872 conn(0x7f0cc4066110 msgr2=0x7f0cc4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:26.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.148+0000 7f0cc8a00640 1 -- 172.21.15.46:0/4091171872 shutdown_connections 2024-03-29T08:18:26.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.148+0000 7f0cc8a00640 1 -- 172.21.15.46:0/4091171872 wait complete. 2024-03-29T08:18:26.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc8a00640 1 Processor -- start 2024-03-29T08:18:26.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc8a00640 1 -- start start 2024-03-29T08:18:26.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc406d910 0x7f0cc4155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:26.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc8a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cc40708d0 0x7f0cc4145000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:26.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cc41569a0 0x7f0cc40cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:26.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc8a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4155220 con 0x7f0cc406d910 2024-03-29T08:18:26.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc8a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4153450 con 0x7f0cc41569a0 2024-03-29T08:18:26.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc8a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4154300 con 0x7f0cc40708d0 2024-03-29T08:18:26.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc406d910 0x7f0cc4155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:26.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc406d910 0x7f0cc4155640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60332/0 (socket says 172.21.15.46:60332) 2024-03-29T08:18:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc1e00640 1 -- 172.21.15.46:0/1020905030 learned_addr learned my addr 172.21.15.46:0/1020905030 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc2800640 1 --2- 172.21.15.46:0/1020905030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cc41569a0 0x7f0cc40cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cbbe00640 1 --2- 172.21.15.46:0/1020905030 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cc40708d0 0x7f0cc4145000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc1e00640 1 -- 172.21.15.46:0/1020905030 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cc40708d0 msgr2=0x7f0cc4145000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc1e00640 1 --2- 172.21.15.46:0/1020905030 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cc40708d0 0x7f0cc4145000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc1e00640 1 -- 172.21.15.46:0/1020905030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cc41569a0 msgr2=0x7f0cc40cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc1e00640 1 --2- 172.21.15.46:0/1020905030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cc41569a0 0x7f0cc40cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc1e00640 1 -- 172.21.15.46:0/1020905030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb40094d0 con 0x7f0cc406d910 2024-03-29T08:18:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc2800640 1 --2- 172.21.15.46:0/1020905030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cc41569a0 0x7f0cc40cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc1e00640 1 --2- 172.21.15.46:0/1020905030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc406d910 0x7f0cc4155640 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40ae040 tx=0x7f0cb40192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cba000640 1 -- 172.21.15.46:0/1020905030 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb40b4620 con 0x7f0cc406d910 2024-03-29T08:18:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cba000640 1 -- 172.21.15.46:0/1020905030 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb40b47a0 con 0x7f0cc406d910 2024-03-29T08:18:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cba000640 1 -- 172.21.15.46:0/1020905030 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb40b4b50 con 0x7f0cc406d910 2024-03-29T08:18:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc8a00640 1 -- 172.21.15.46:0/1020905030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc4145540 con 0x7f0cc406d910 2024-03-29T08:18:26.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.152+0000 7f0cc8a00640 1 -- 172.21.15.46:0/1020905030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc4154880 con 0x7f0cc406d910 2024-03-29T08:18:26.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.156+0000 7f0cc8a00640 1 -- 172.21.15.46:0/1020905030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc4066c90 con 0x7f0cc406d910 2024-03-29T08:18:26.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.156+0000 7f0cba000640 1 -- 172.21.15.46:0/1020905030 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0cb40b2070 con 0x7f0cc406d910 2024-03-29T08:18:26.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.156+0000 7f0cba000640 1 --2- 172.21.15.46:0/1020905030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ca0066690 0x7f0ca0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:26.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.156+0000 7f0cba000640 1 -- 172.21.15.46:0/1020905030 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0cb411d3a0 con 0x7f0cc406d910 2024-03-29T08:18:26.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.156+0000 7f0cbbe00640 1 --2- 172.21.15.46:0/1020905030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ca0066690 0x7f0ca0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:26.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.156+0000 7f0cbbe00640 1 --2- 172.21.15.46:0/1020905030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ca0066690 0x7f0ca0068b50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40ce330 tx=0x7f0cac007b80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:26.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.160+0000 7f0cba000640 1 -- 172.21.15.46:0/1020905030 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0cb40ea670 con 0x7f0cc406d910 2024-03-29T08:18:26.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.356+0000 7f0cc8a00640 1 -- 172.21.15.46:0/1020905030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cc40afdd0 con 0x7f0cc406d910 2024-03-29T08:18:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.356+0000 7f0cba000640 1 -- 172.21.15.46:0/1020905030 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4722 (secure 0 0 0) 0x7f0cb40edc90 con 0x7f0cc406d910 2024-03-29T08:18:26.357 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:26.357 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/339038392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":339038392},{"type":"v1","addr":"172.21.15.46:6834","nonce":339038392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3573036715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3573036715},{"type":"v1","addr":"172.21.15.73:6835","nonce":3573036715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/746744819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":746744819},{"type":"v1","addr":"172.21.15.46:6835","nonce":746744819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:18:13.311457+0000","modified":"2024-03-29T08:18:16.321300+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2780269530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2780269530},{"type":"v1","addr":"172.21.15.73:6837","nonce":2780269530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-29T08:18:26.358 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-03-29T08:18:26.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.356+0000 7f0cc8a00640 1 -- 172.21.15.46:0/1020905030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ca0066690 msgr2=0x7f0ca0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.356+0000 7f0cc8a00640 1 --2- 172.21.15.46:0/1020905030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ca0066690 0x7f0ca0068b50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0cc40ce330 tx=0x7f0cac007b80 comp rx=0 tx=0).stop 2024-03-29T08:18:26.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.356+0000 7f0cc8a00640 1 -- 172.21.15.46:0/1020905030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc406d910 msgr2=0x7f0cc4155640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.356+0000 7f0cc8a00640 1 --2- 172.21.15.46:0/1020905030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc406d910 0x7f0cc4155640 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0cb40ae040 tx=0x7f0cb40192c0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.356+0000 7f0cc8a00640 1 -- 172.21.15.46:0/1020905030 shutdown_connections 2024-03-29T08:18:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.356+0000 7f0cc8a00640 1 --2- 172.21.15.46:0/1020905030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ca0066690 0x7f0ca0068b50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.356+0000 7f0cc8a00640 1 --2- 172.21.15.46:0/1020905030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cc41569a0 0x7f0cc40cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.356+0000 7f0cc8a00640 1 --2- 172.21.15.46:0/1020905030 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cc40708d0 0x7f0cc4145000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.356+0000 7f0cc8a00640 1 --2- 172.21.15.46:0/1020905030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc406d910 0x7f0cc4155640 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.356+0000 7f0cc8a00640 1 -- 172.21.15.46:0/1020905030 >> 172.21.15.46:0/1020905030 conn(0x7f0cc4066110 msgr2=0x7f0cc40af0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.356+0000 7f0cc8a00640 1 -- 172.21.15.46:0/1020905030 shutdown_connections 2024-03-29T08:18:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.356+0000 7f0cc8a00640 1 -- 172.21.15.46:0/1020905030 wait complete. 2024-03-29T08:18:26.378 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:26.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.516+0000 7fcea9c00640 1 Processor -- start 2024-03-29T08:18:26.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.516+0000 7fcea9c00640 1 -- start start 2024-03-29T08:18:26.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.516+0000 7fcea9c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcea40ceb20 0x7fcea41656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:26.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.516+0000 7fcea9c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea40cb510 con 0x7fcea4144210 2024-03-29T08:18:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.516+0000 7fcea9c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcea4070580 con 0x7fcea415ada0 2024-03-29T08:18:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.516+0000 7fcea9c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcea406d960 con 0x7fcea40ceb20 2024-03-29T08:18:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.516+0000 7fcea3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcea40ceb20 0x7fcea41656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.520+0000 7fcea3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcea40ceb20 0x7fcea41656d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51188/0 (socket says 172.21.15.46:51188) 2024-03-29T08:18:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.520+0000 7fcea3400640 1 -- 172.21.15.46:0/3312472407 learned_addr learned my addr 172.21.15.46:0/3312472407 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.520+0000 7fcea2a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcea4144210 0x7fcea40cb8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45200/0 (socket says 172.21.15.46:45200) 2024-03-29T08:18:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.520+0000 7fcea3400640 1 -- 172.21.15.46:0/3312472407 >> v1:172.21.15.73:6789/0 conn(0x7fcea415ada0 legacy=0x7fcea40ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.520+0000 7fcea3400640 1 -- 172.21.15.46:0/3312472407 >> v1:172.21.15.46:6789/0 conn(0x7fcea4144210 legacy=0x7fcea40cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.520+0000 7fcea3400640 1 -- 172.21.15.46:0/3312472407 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea4070860 con 0x7fcea40ceb20 2024-03-29T08:18:26.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.520+0000 7fcea2000640 1 -- 172.21.15.46:0/3312472407 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3421668414 0 0) 0x7fcea40cb510 con 0x7fcea4144210 2024-03-29T08:18:26.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.520+0000 7fcea3400640 1 --2- 172.21.15.46:0/3312472407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcea40ceb20 0x7fcea41656d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009e50 tx=0x7fce9c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350ad7d361945758 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:26.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.520+0000 7fcea2000640 1 -- 172.21.15.46:0/3312472407 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0a57f0 con 0x7fcea40ceb20 2024-03-29T08:18:26.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.520+0000 7fcea2000640 1 -- 172.21.15.46:0/3312472407 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9c0a5950 con 0x7fcea40ceb20 2024-03-29T08:18:26.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.520+0000 7fcea2000640 1 -- 172.21.15.46:0/3312472407 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0a5ce0 con 0x7fcea40ceb20 2024-03-29T08:18:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea9c00640 1 -- 172.21.15.46:0/3312472407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcea40ceb20 msgr2=0x7fcea41656d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea9c00640 1 --2- 172.21.15.46:0/3312472407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcea40ceb20 0x7fcea41656d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009e50 tx=0x7fce9c01ca30 comp rx=0 tx=0).stop 2024-03-29T08:18:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea9c00640 1 -- 172.21.15.46:0/3312472407 shutdown_connections 2024-03-29T08:18:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea9c00640 1 --2- 172.21.15.46:0/3312472407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcea40ceb20 0x7fcea41656d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea9c00640 1 -- 172.21.15.46:0/3312472407 >> 172.21.15.46:0/3312472407 conn(0x7fcea4066110 msgr2=0x7fcea4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:26.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea9c00640 1 -- 172.21.15.46:0/3312472407 shutdown_connections 2024-03-29T08:18:26.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea9c00640 1 -- 172.21.15.46:0/3312472407 wait complete. 2024-03-29T08:18:26.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea9c00640 1 Processor -- start 2024-03-29T08:18:26.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea9c00640 1 -- start start 2024-03-29T08:18:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea9c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcea40ceb20 0x7fcea413c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea9c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcea4144210 0x7fcea40af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcea415ada0 0x7fcea40aead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea9c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40ae6b0 con 0x7fcea415ada0 2024-03-29T08:18:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea9c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcea40afe60 con 0x7fcea4144210 2024-03-29T08:18:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea9c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcea415b180 con 0x7fcea40ceb20 2024-03-29T08:18:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcea415ada0 0x7fcea40aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcea415ada0 0x7fcea40aead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51196/0 (socket says 172.21.15.46:51196) 2024-03-29T08:18:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea3400640 1 -- 172.21.15.46:0/3907905510 learned_addr learned my addr 172.21.15.46:0/3907905510 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:26.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea2a00640 1 --2- 172.21.15.46:0/3907905510 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcea40ceb20 0x7fcea413c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fce9be00640 1 --2- 172.21.15.46:0/3907905510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcea4144210 0x7fcea40af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea3400640 1 -- 172.21.15.46:0/3907905510 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcea40ceb20 msgr2=0x7fcea413c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea3400640 1 --2- 172.21.15.46:0/3907905510 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcea40ceb20 0x7fcea413c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea3400640 1 -- 172.21.15.46:0/3907905510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcea4144210 msgr2=0x7fcea40af920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea3400640 1 --2- 172.21.15.46:0/3907905510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcea4144210 0x7fcea40af920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea3400640 1 -- 172.21.15.46:0/3907905510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9c009b30 con 0x7fcea415ada0 2024-03-29T08:18:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fcea3400640 1 --2- 172.21.15.46:0/3907905510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcea415ada0 0x7fcea40aead0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0af040 tx=0x7fce9c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.524+0000 7fce9aa00640 1 -- 172.21.15.46:0/3907905510 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c009210 con 0x7fcea415ada0 2024-03-29T08:18:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.528+0000 7fce9aa00640 1 -- 172.21.15.46:0/3907905510 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9c0b8680 con 0x7fcea415ada0 2024-03-29T08:18:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.528+0000 7fce9aa00640 1 -- 172.21.15.46:0/3907905510 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9c0b89f0 con 0x7fcea415ada0 2024-03-29T08:18:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.528+0000 7fcea9c00640 1 -- 172.21.15.46:0/3907905510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcea415b460 con 0x7fcea415ada0 2024-03-29T08:18:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.528+0000 7fcea9c00640 1 -- 172.21.15.46:0/3907905510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcea40ae2d0 con 0x7fcea415ada0 2024-03-29T08:18:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.528+0000 7fcea9c00640 1 -- 172.21.15.46:0/3907905510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea40ce280 con 0x7fcea415ada0 2024-03-29T08:18:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.528+0000 7fce9aa00640 1 -- 172.21.15.46:0/3907905510 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fce9c0b8b50 con 0x7fcea415ada0 2024-03-29T08:18:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.528+0000 7fce9aa00640 1 --2- 172.21.15.46:0/3907905510 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce80066690 0x7fce80068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.528+0000 7fce9aa00640 1 -- 172.21.15.46:0/3907905510 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce9c11f320 con 0x7fcea415ada0 2024-03-29T08:18:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.528+0000 7fcea2a00640 1 --2- 172.21.15.46:0/3907905510 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce80066690 0x7fce80068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.532+0000 7fcea2a00640 1 --2- 172.21.15.46:0/3907905510 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce80066690 0x7fce80068b50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fce94007a90 tx=0x7fce940079e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:26.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.532+0000 7fce9aa00640 1 -- 172.21.15.46:0/3907905510 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fce9c0ec7a0 con 0x7fcea415ada0 2024-03-29T08:18:26.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.696+0000 7fcea9c00640 1 -- 172.21.15.46:0/3907905510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcea40ae7f0 con 0x7fcea415ada0 2024-03-29T08:18:26.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.696+0000 7fce9aa00640 1 -- 172.21.15.46:0/3907905510 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15993 (secure 0 0 0) 0x7fce9c0efdc0 con 0x7fcea415ada0 2024-03-29T08:18:26.697 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:26.697 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:14.314124+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:26.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.696+0000 7fcea9c00640 1 -- 172.21.15.46:0/3907905510 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce80066690 msgr2=0x7fce80068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.696+0000 7fcea9c00640 1 --2- 172.21.15.46:0/3907905510 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce80066690 0x7fce80068b50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fce94007a90 tx=0x7fce940079e0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.696+0000 7fcea9c00640 1 -- 172.21.15.46:0/3907905510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcea415ada0 msgr2=0x7fcea40aead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.696+0000 7fcea9c00640 1 --2- 172.21.15.46:0/3907905510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcea415ada0 0x7fcea40aead0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0af040 tx=0x7fce9c0197d0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.696+0000 7fcea9c00640 1 -- 172.21.15.46:0/3907905510 shutdown_connections 2024-03-29T08:18:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.696+0000 7fcea9c00640 1 --2- 172.21.15.46:0/3907905510 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce80066690 0x7fce80068b50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.696+0000 7fcea9c00640 1 --2- 172.21.15.46:0/3907905510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcea4144210 0x7fcea40af920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.696+0000 7fcea9c00640 1 --2- 172.21.15.46:0/3907905510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcea415ada0 0x7fcea40aead0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.696+0000 7fcea9c00640 1 --2- 172.21.15.46:0/3907905510 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcea40ceb20 0x7fcea413c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.696+0000 7fcea9c00640 1 -- 172.21.15.46:0/3907905510 >> 172.21.15.46:0/3907905510 conn(0x7fcea4066110 msgr2=0x7fcea40cb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.696+0000 7fcea9c00640 1 -- 172.21.15.46:0/3907905510 shutdown_connections 2024-03-29T08:18:26.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.696+0000 7fcea9c00640 1 -- 172.21.15.46:0/3907905510 wait complete. 2024-03-29T08:18:26.720 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:18:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.860+0000 7f49eca00640 1 Processor -- start 2024-03-29T08:18:26.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.860+0000 7f49eca00640 1 -- start start 2024-03-29T08:18:26.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.860+0000 7f49eca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49e815a940 0x7f49e80afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.860+0000 7f49eca00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e80b00d0 con 0x7f49e80cf3c0 2024-03-29T08:18:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.860+0000 7f49eca00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e80b0ab0 con 0x7f49e815ad20 2024-03-29T08:18:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.860+0000 7f49eca00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e80b0d90 con 0x7f49e815a940 2024-03-29T08:18:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.860+0000 7f49e5e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49e815a940 0x7f49e80afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.860+0000 7f49e5e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f49e815a940 0x7f49e80afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51202/0 (socket says 172.21.15.46:51202) 2024-03-29T08:18:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.860+0000 7f49e5e00640 1 -- 172.21.15.46:0/2521520559 learned_addr learned my addr 172.21.15.46:0/2521520559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.860+0000 7f49e6800640 1 --1- 172.21.15.46:0/2521520559 >> v1:172.21.15.46:6789/0 conn(0x7f49e80cf3c0 0x7f49e80cb630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45206/0 (socket says 172.21.15.46:45206) 2024-03-29T08:18:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.860+0000 7f49e5e00640 1 -- 172.21.15.46:0/2521520559 >> v1:172.21.15.73:6789/0 conn(0x7f49e815ad20 legacy=0x7f49e80cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.860+0000 7f49e5e00640 1 -- 172.21.15.46:0/2521520559 >> v1:172.21.15.46:6789/0 conn(0x7f49e80cf3c0 legacy=0x7f49e80cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.860+0000 7f49e5e00640 1 -- 172.21.15.46:0/2521520559 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e80b1070 con 0x7f49e815a940 2024-03-29T08:18:26.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.860+0000 7f49e5400640 1 -- 172.21.15.46:0/2521520559 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892988321 0 0) 0x7f49e80b00d0 con 0x7f49e80cf3c0 2024-03-29T08:18:26.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.864+0000 7f49e5e00640 1 --2- 172.21.15.46:0/2521520559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e815a940 0x7f49e80afb90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009870 tx=0x7f49d801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2059fd1766b7ceb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:26.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.864+0000 7f49e5400640 1 -- 172.21.15.46:0/2521520559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d80a6040 con 0x7f49e815a940 2024-03-29T08:18:26.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.864+0000 7f49e5400640 1 -- 172.21.15.46:0/2521520559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d80aa600 con 0x7f49e815a940 2024-03-29T08:18:26.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.864+0000 7f49e5400640 1 -- 172.21.15.46:0/2521520559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d80aa990 con 0x7f49e815a940 2024-03-29T08:18:26.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.864+0000 7f49eca00640 1 -- 172.21.15.46:0/2521520559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e815a940 msgr2=0x7f49e80afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.864+0000 7f49eca00640 1 --2- 172.21.15.46:0/2521520559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e815a940 0x7f49e80afb90 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f49d8009870 tx=0x7f49d801c4a0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.864+0000 7f49eca00640 1 -- 172.21.15.46:0/2521520559 shutdown_connections 2024-03-29T08:18:26.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.864+0000 7f49eca00640 1 --2- 172.21.15.46:0/2521520559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e815a940 0x7f49e80afb90 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.864+0000 7f49eca00640 1 -- 172.21.15.46:0/2521520559 >> 172.21.15.46:0/2521520559 conn(0x7f49e8089fd0 msgr2=0x7f49e808a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:26.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.864+0000 7f49eca00640 1 -- 172.21.15.46:0/2521520559 shutdown_connections 2024-03-29T08:18:26.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.864+0000 7f49eca00640 1 -- 172.21.15.46:0/2521520559 wait complete. 2024-03-29T08:18:26.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49eca00640 1 Processor -- start 2024-03-29T08:18:26.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49eca00640 1 -- start start 2024-03-29T08:18:26.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49eca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49e80cf3c0 0x7f49e80c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:26.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49eca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e815a940 0x7f49e80b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:26.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49eca00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f49e815ad20 0x7f49e8153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:26.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49eca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e80c6c30 con 0x7f49e815a940 2024-03-29T08:18:26.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49eca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e81502d0 con 0x7f49e80cf3c0 2024-03-29T08:18:26.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49eca00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e8156740 con 0x7f49e815ad20 2024-03-29T08:18:26.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49dfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e815a940 0x7f49e80b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:26.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49dfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e815a940 0x7f49e80b5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51216/0 (socket says 172.21.15.46:51216) 2024-03-29T08:18:26.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49dfe00640 1 -- 172.21.15.46:0/1822230229 learned_addr learned my addr 172.21.15.46:0/1822230229 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:26.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49e5e00640 1 --2- 172.21.15.46:0/1822230229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49e80cf3c0 0x7f49e80c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:26.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49e6800640 1 --2- 172.21.15.46:0/1822230229 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f49e815ad20 0x7f49e8153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:26.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49dfe00640 1 -- 172.21.15.46:0/1822230229 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f49e815ad20 msgr2=0x7f49e8153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49dfe00640 1 --2- 172.21.15.46:0/1822230229 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f49e815ad20 0x7f49e8153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49dfe00640 1 -- 172.21.15.46:0/1822230229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49e80cf3c0 msgr2=0x7f49e80c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:26.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49dfe00640 1 --2- 172.21.15.46:0/1822230229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49e80cf3c0 0x7f49e80c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:26.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49dfe00640 1 -- 172.21.15.46:0/1822230229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d80094d0 con 0x7f49e815a940 2024-03-29T08:18:26.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49e6800640 1 --2- 172.21.15.46:0/1822230229 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f49e815ad20 0x7f49e8153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:26.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49dfe00640 1 --2- 172.21.15.46:0/1822230229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e815a940 0x7f49e80b5ec0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f49d000c3f0 tx=0x7f49d000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:26.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49de000640 1 -- 172.21.15.46:0/1822230229 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d00052d0 con 0x7f49e815a940 2024-03-29T08:18:26.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49de000640 1 -- 172.21.15.46:0/1822230229 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d0005430 con 0x7f49e815a940 2024-03-29T08:18:26.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49de000640 1 -- 172.21.15.46:0/1822230229 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0003840 con 0x7f49e815a940 2024-03-29T08:18:26.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49eca00640 1 -- 172.21.15.46:0/1822230229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e80c9740 con 0x7f49e815a940 2024-03-29T08:18:26.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49eca00640 1 -- 172.21.15.46:0/1822230229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e80c9c50 con 0x7f49e815a940 2024-03-29T08:18:26.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.868+0000 7f49eca00640 1 -- 172.21.15.46:0/1822230229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e8150410 con 0x7f49e815a940 2024-03-29T08:18:26.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.872+0000 7f49de000640 1 -- 172.21.15.46:0/1822230229 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f49d00039a0 con 0x7f49e815a940 2024-03-29T08:18:26.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.872+0000 7f49de000640 1 --2- 172.21.15.46:0/1822230229 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f49c4066690 0x7f49c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:26.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.872+0000 7f49de000640 1 -- 172.21.15.46:0/1822230229 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49d0083640 con 0x7f49e815a940 2024-03-29T08:18:26.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.872+0000 7f49e5e00640 1 --2- 172.21.15.46:0/1822230229 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f49c4066690 0x7f49c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:26.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.872+0000 7f49e5e00640 1 --2- 172.21.15.46:0/1822230229 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f49c4066690 0x7f49c4068b50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f49d80ae000 tx=0x7f49d80b5040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:26.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:26.876+0000 7f49de000640 1 -- 172.21.15.46:0/1822230229 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f49e8150410 con 0x7f49e815a940 2024-03-29T08:18:27.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.072+0000 7f49eca00640 1 -- 172.21.15.46:0/1822230229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49e8150410 con 0x7f49e815a940 2024-03-29T08:18:27.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.072+0000 7f49de000640 1 -- 172.21.15.46:0/1822230229 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4722 (secure 0 0 0) 0x7f49e8150410 con 0x7f49e815a940 2024-03-29T08:18:27.074 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:27.074 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/339038392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":339038392},{"type":"v1","addr":"172.21.15.46:6834","nonce":339038392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3573036715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3573036715},{"type":"v1","addr":"172.21.15.73:6835","nonce":3573036715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/746744819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":746744819},{"type":"v1","addr":"172.21.15.46:6835","nonce":746744819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:18:13.311457+0000","modified":"2024-03-29T08:18:16.321300+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2780269530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2780269530},{"type":"v1","addr":"172.21.15.73:6837","nonce":2780269530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-29T08:18:27.074 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-03-29T08:18:27.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.072+0000 7f49eca00640 1 -- 172.21.15.46:0/1822230229 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f49c4066690 msgr2=0x7f49c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:27.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.072+0000 7f49eca00640 1 --2- 172.21.15.46:0/1822230229 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f49c4066690 0x7f49c4068b50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f49d80ae000 tx=0x7f49d80b5040 comp rx=0 tx=0).stop 2024-03-29T08:18:27.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.072+0000 7f49eca00640 1 -- 172.21.15.46:0/1822230229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e815a940 msgr2=0x7f49e80b5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:27.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.072+0000 7f49eca00640 1 --2- 172.21.15.46:0/1822230229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e815a940 0x7f49e80b5ec0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f49d000c3f0 tx=0x7f49d000c8b0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.072+0000 7f49eca00640 1 -- 172.21.15.46:0/1822230229 shutdown_connections 2024-03-29T08:18:27.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.072+0000 7f49eca00640 1 --2- 172.21.15.46:0/1822230229 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f49c4066690 0x7f49c4068b50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.072+0000 7f49eca00640 1 --2- 172.21.15.46:0/1822230229 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f49e815ad20 0x7f49e8153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.072+0000 7f49eca00640 1 --2- 172.21.15.46:0/1822230229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49e815a940 0x7f49e80b5ec0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.072+0000 7f49eca00640 1 --2- 172.21.15.46:0/1822230229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49e80cf3c0 0x7f49e80c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.072+0000 7f49eca00640 1 -- 172.21.15.46:0/1822230229 >> 172.21.15.46:0/1822230229 conn(0x7f49e8089fd0 msgr2=0x7f49e80aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:27.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.072+0000 7f49eca00640 1 -- 172.21.15.46:0/1822230229 shutdown_connections 2024-03-29T08:18:27.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.072+0000 7f49eca00640 1 -- 172.21.15.46:0/1822230229 wait complete. 2024-03-29T08:18:27.497 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:18:13.311457+0000', 'modified': '2024-03-29T08:18:16.321300+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4291}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4291': {'gid': 4291, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6837/2780269530', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 2780269530}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 2780269530}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:18:27.497 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T08:18:27.498 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-29T08:18:27.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.644+0000 7fca94c00640 1 Processor -- start 2024-03-29T08:18:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.644+0000 7fca94c00640 1 -- start start 2024-03-29T08:18:27.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.644+0000 7fca94c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fca900af530 0x7fca900af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.644+0000 7fca94c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca900702a0 0x7fca90070730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.644+0000 7fca94c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9006c390 con 0x7fca90158300 2024-03-29T08:18:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.644+0000 7fca94c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca90070c70 con 0x7fca900af530 2024-03-29T08:18:27.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.644+0000 7fca94c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca900afff0 con 0x7fca900702a0 2024-03-29T08:18:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.644+0000 7fca85400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fca900af530 0x7fca900af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.644+0000 7fca85400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fca900af530 0x7fca900af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:44390/0 (socket says 172.21.15.46:44390) 2024-03-29T08:18:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.644+0000 7fca85400640 1 -- 172.21.15.46:0/987543637 learned_addr learned my addr 172.21.15.46:0/987543637 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.644+0000 7fca8de00640 1 --1- 172.21.15.46:0/987543637 >> v1:172.21.15.73:6789/0 conn(0x7fca90158300 0x7fca9006d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:39506/0 (socket says 172.21.15.46:39506) 2024-03-29T08:18:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.644+0000 7fca8e800640 1 --2- 172.21.15.46:0/987543637 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca900702a0 0x7fca90070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.644+0000 7fca8e800640 1 -- 172.21.15.46:0/987543637 >> v2:172.21.15.73:3300/0 conn(0x7fca900af530 msgr2=0x7fca900af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.644+0000 7fca8e800640 1 --2- 172.21.15.46:0/987543637 >> v2:172.21.15.73:3300/0 conn(0x7fca900af530 0x7fca900af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.644+0000 7fca8e800640 1 -- 172.21.15.46:0/987543637 >> v1:172.21.15.73:6789/0 conn(0x7fca90158300 legacy=0x7fca9006d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.644+0000 7fca8e800640 1 -- 172.21.15.46:0/987543637 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9006c5e0 con 0x7fca900702a0 2024-03-29T08:18:27.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.644+0000 7fca85400640 1 --2- 172.21.15.46:0/987543637 >> v2:172.21.15.73:3300/0 conn(0x7fca900af530 0x7fca900af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:27.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.648+0000 7fca8e800640 1 --2- 172.21.15.46:0/987543637 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca900702a0 0x7fca90070730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fca880097c0 tx=0x7fca8801c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eeec386af12ad724 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:27.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.648+0000 7fca8d400640 1 -- 172.21.15.46:0/987543637 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca88004230 con 0x7fca900702a0 2024-03-29T08:18:27.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.648+0000 7fca8d400640 1 -- 172.21.15.46:0/987543637 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca880a7050 con 0x7fca900702a0 2024-03-29T08:18:27.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.648+0000 7fca8d400640 1 -- 172.21.15.46:0/987543637 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca88003620 con 0x7fca900702a0 2024-03-29T08:18:27.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.648+0000 7fca94c00640 1 -- 172.21.15.46:0/987543637 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca900702a0 msgr2=0x7fca90070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:27.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.648+0000 7fca94c00640 1 --2- 172.21.15.46:0/987543637 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca900702a0 0x7fca90070730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fca880097c0 tx=0x7fca8801c350 comp rx=0 tx=0).stop 2024-03-29T08:18:27.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.648+0000 7fca94c00640 1 -- 172.21.15.46:0/987543637 shutdown_connections 2024-03-29T08:18:27.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.648+0000 7fca94c00640 1 --2- 172.21.15.46:0/987543637 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca900702a0 0x7fca90070730 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.648+0000 7fca94c00640 1 --2- 172.21.15.46:0/987543637 >> v2:172.21.15.73:3300/0 conn(0x7fca900af530 0x7fca900af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.648+0000 7fca94c00640 1 -- 172.21.15.46:0/987543637 >> 172.21.15.46:0/987543637 conn(0x7fca90066080 msgr2=0x7fca90069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:27.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.648+0000 7fca94c00640 1 -- 172.21.15.46:0/987543637 shutdown_connections 2024-03-29T08:18:27.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.648+0000 7fca94c00640 1 -- 172.21.15.46:0/987543637 wait complete. 2024-03-29T08:18:27.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca94c00640 1 Processor -- start 2024-03-29T08:18:27.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca94c00640 1 -- start start 2024-03-29T08:18:27.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca94c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca900702a0 0x7fca90154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca94c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca900af530 0x7fca90164e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca94c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca90158300 0x7fca900c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca94c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca90161f60 con 0x7fca900af530 2024-03-29T08:18:27.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca94c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca90162240 con 0x7fca90158300 2024-03-29T08:18:27.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca94c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca90162520 con 0x7fca900702a0 2024-03-29T08:18:27.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca85400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca900af530 0x7fca90164e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca85400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca900af530 0x7fca90164e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51222/0 (socket says 172.21.15.46:51222) 2024-03-29T08:18:27.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca85400640 1 -- 172.21.15.46:0/1086260148 learned_addr learned my addr 172.21.15.46:0/1086260148 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:27.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca8e800640 1 --2- 172.21.15.46:0/1086260148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca90158300 0x7fca900c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca8de00640 1 --2- 172.21.15.46:0/1086260148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca900702a0 0x7fca90154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca85400640 1 -- 172.21.15.46:0/1086260148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca900702a0 msgr2=0x7fca90154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:27.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca85400640 1 --2- 172.21.15.46:0/1086260148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca900702a0 0x7fca90154ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca85400640 1 -- 172.21.15.46:0/1086260148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca90158300 msgr2=0x7fca900c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:27.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca85400640 1 --2- 172.21.15.46:0/1086260148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca90158300 0x7fca900c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca85400640 1 -- 172.21.15.46:0/1086260148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca780094d0 con 0x7fca900af530 2024-03-29T08:18:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca85400640 1 --2- 172.21.15.46:0/1086260148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca900af530 0x7fca90164e60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fca7800e9d0 tx=0x7fca7800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca86a00640 1 -- 172.21.15.46:0/1086260148 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7800ca60 con 0x7fca900af530 2024-03-29T08:18:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca86a00640 1 -- 172.21.15.46:0/1086260148 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7800cbc0 con 0x7fca900af530 2024-03-29T08:18:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca86a00640 1 -- 172.21.15.46:0/1086260148 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78013470 con 0x7fca900af530 2024-03-29T08:18:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca94c00640 1 -- 172.21.15.46:0/1086260148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca88009420 con 0x7fca900af530 2024-03-29T08:18:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.652+0000 7fca94c00640 1 -- 172.21.15.46:0/1086260148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca90162ae0 con 0x7fca900af530 2024-03-29T08:18:27.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.656+0000 7fca86a00640 1 -- 172.21.15.46:0/1086260148 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fca780135d0 con 0x7fca900af530 2024-03-29T08:18:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.656+0000 7fca86a00640 1 --2- 172.21.15.46:0/1086260148 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca6c066690 0x7fca6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.656+0000 7fca86a00640 1 -- 172.21.15.46:0/1086260148 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca780844d0 con 0x7fca900af530 2024-03-29T08:18:27.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.656+0000 7fca8de00640 1 --2- 172.21.15.46:0/1086260148 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca6c066690 0x7fca6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.656+0000 7fca9812f000 1 Processor -- start 2024-03-29T08:18:27.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.656+0000 7fca9812f000 1 -- start start 2024-03-29T08:18:27.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.656+0000 7fca8de00640 1 --2- 172.21.15.46:0/1086260148 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca6c066690 0x7fca6c068b50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fca800012f0 tx=0x7fca80001660 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:27.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.656+0000 7fca9812f000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e90db0 0x55e500e91190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.656+0000 7fca9812f000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e916d0 0x55e500e46f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.656+0000 7fca9812f000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e474c0 0x55e500e478a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.656+0000 7fca9812f000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e500d0f5a0 con 0x55e500e90db0 2024-03-29T08:18:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.656+0000 7fca9812f000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e500c9caf0 con 0x55e500e474c0 2024-03-29T08:18:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.656+0000 7fca9812f000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e500c9a0c0 con 0x55e500e916d0 2024-03-29T08:18:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca8de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e90db0 0x55e500e91190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca85400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e916d0 0x55e500e46f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca8e800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e474c0 0x55e500e478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca85400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e916d0 0x55e500e46f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:38980/0 (socket says 172.21.15.46:38980) 2024-03-29T08:18:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca85400640 1 -- 172.21.15.46:0/854186773 learned_addr learned my addr 172.21.15.46:0/854186773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca8de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e90db0 0x55e500e91190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51224/0 (socket says 172.21.15.46:51224) 2024-03-29T08:18:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca85400640 1 -- 172.21.15.46:0/854186773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e474c0 msgr2=0x55e500e478a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca85400640 1 --2- 172.21.15.46:0/854186773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e474c0 0x55e500e478a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca85400640 1 -- 172.21.15.46:0/854186773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e90db0 msgr2=0x55e500e91190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:27.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca85400640 1 --2- 172.21.15.46:0/854186773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e90db0 0x55e500e91190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca85400640 1 -- 172.21.15.46:0/854186773 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e500991890 con 0x55e500e916d0 2024-03-29T08:18:27.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca8de00640 1 --2- 172.21.15.46:0/854186773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e90db0 0x55e500e91190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:27.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca85400640 1 --2- 172.21.15.46:0/854186773 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e916d0 0x55e500e46f80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fca780828a0 tx=0x7fca78054f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5815d914a467e006 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:27.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca7ea00640 1 -- 172.21.15.46:0/854186773 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7805d850 con 0x55e500e916d0 2024-03-29T08:18:27.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca7ea00640 1 -- 172.21.15.46:0/854186773 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca78053650 con 0x55e500e916d0 2024-03-29T08:18:27.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca7ea00640 1 -- 172.21.15.46:0/854186773 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78051c20 con 0x55e500e916d0 2024-03-29T08:18:27.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 1 -- 172.21.15.46:0/854186773 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e916d0 msgr2=0x55e500e46f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:27.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 1 --2- 172.21.15.46:0/854186773 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e916d0 0x55e500e46f80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fca780828a0 tx=0x7fca78054f30 comp rx=0 tx=0).stop 2024-03-29T08:18:27.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 1 -- 172.21.15.46:0/854186773 shutdown_connections 2024-03-29T08:18:27.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 1 --2- 172.21.15.46:0/854186773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e474c0 0x55e500e478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 1 --2- 172.21.15.46:0/854186773 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e916d0 0x55e500e46f80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 1 --2- 172.21.15.46:0/854186773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e90db0 0x55e500e91190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 1 -- 172.21.15.46:0/854186773 >> 172.21.15.46:0/854186773 conn(0x55e500e4d830 msgr2=0x55e500e45860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:27.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 1 -- 172.21.15.46:0/854186773 shutdown_connections 2024-03-29T08:18:27.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 1 -- 172.21.15.46:0/854186773 wait complete. 2024-03-29T08:18:27.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 1 -- start start 2024-03-29T08:18:27.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 1 Processor -- start 2024-03-29T08:18:27.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500e905a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e90db0 0x55e500e91ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e916d0 0x55e500e91ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e500c959b0 con 0x55e500e474c0 2024-03-29T08:18:27.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e500c34c00 con 0x55e500e90db0 2024-03-29T08:18:27.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e500c53070 con 0x55e500e916d0 2024-03-29T08:18:27.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca9812f000 10 client.0 _ll_register_callbacks cb 0x55e500e330e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:18:27.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca8de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500e905a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca8e800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e916d0 0x55e500e91ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca8de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500e905a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51238/0 (socket says 172.21.15.46:51238) 2024-03-29T08:18:27.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca8de00640 1 -- 172.21.15.46:0/3397239818 learned_addr learned my addr 172.21.15.46:0/3397239818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:27.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.660+0000 7fca85400640 1 --2- 172.21.15.46:0/3397239818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e90db0 0x55e500e91ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca8de00640 1 -- 172.21.15.46:0/3397239818 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e916d0 msgr2=0x55e500e91ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:27.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca8de00640 1 --2- 172.21.15.46:0/3397239818 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e916d0 0x55e500e91ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca8de00640 1 -- 172.21.15.46:0/3397239818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e90db0 msgr2=0x55e500e91ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:27.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca8de00640 1 --2- 172.21.15.46:0/3397239818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e90db0 0x55e500e91ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca8de00640 1 -- 172.21.15.46:0/3397239818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9006c390 con 0x55e500e474c0 2024-03-29T08:18:27.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca8de00640 1 --2- 172.21.15.46:0/3397239818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500e905a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fca800139a0 tx=0x7fca80001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae926491dfca8876 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:27.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca73400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:18:27.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca73400640 1 -- 172.21.15.46:0/3397239818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca800128d0 con 0x55e500e474c0 2024-03-29T08:18:27.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca73400640 1 -- 172.21.15.46:0/3397239818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca8001b070 con 0x55e500e474c0 2024-03-29T08:18:27.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 1 -- 172.21.15.46:0/3397239818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e500c525f0 con 0x55e500e474c0 2024-03-29T08:18:27.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca73400640 1 -- 172.21.15.46:0/3397239818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca80012a30 con 0x55e500e474c0 2024-03-29T08:18:27.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 10 client.4608 fetch_fsmap learned FSMap version 8 2024-03-29T08:18:27.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 1 -- 172.21.15.46:0/3397239818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x7fca90162ae0 con 0x55e500e474c0 2024-03-29T08:18:27.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca73400640 1 -- 172.21.15.46:0/3397239818 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e500c525f0 con 0x55e500e474c0 2024-03-29T08:18:27.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca73400640 1 -- 172.21.15.46:0/3397239818 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fca80012070 con 0x55e500e474c0 2024-03-29T08:18:27.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 10 client.4608 fetch_fsmap finished waiting for FSMap version 8 2024-03-29T08:18:27.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 10 client.4608 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-29T08:18:27.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 20 client.4608 populate_metadata read hostname 'smithi046' 2024-03-29T08:18:27.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 1 --2- 172.21.15.46:0/3397239818 >> [v2:172.21.15.73:6836/2780269530,v1:172.21.15.73:6837/2780269530] conn(0x55e500e92a00 0x55e500e92de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 4 client.4608 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:18:27.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 1 -- 172.21.15.46:0/3397239818 --> [v2:172.21.15.73:6836/2780269530,v1:172.21.15.73:6837/2780269530] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e500c92790 con 0x55e500e92a00 2024-03-29T08:18:27.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca85400640 1 --2- 172.21.15.46:0/3397239818 >> [v2:172.21.15.73:6836/2780269530,v1:172.21.15.73:6837/2780269530] conn(0x55e500e92a00 0x55e500e92de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca85400640 1 --2- 172.21.15.46:0/3397239818 >> [v2:172.21.15.73:6836/2780269530,v1:172.21.15.73:6837/2780269530] conn(0x55e500e92a00 0x55e500e92de0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9c2af0391d8d15e server_cookie=f4f99c3c0b1daac3 in_seq=0 out_seq=0 2024-03-29T08:18:27.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca73400640 10 client.4608 ms_handle_connect on v2:172.21.15.73:6836/2780269530 2024-03-29T08:18:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca73400640 1 -- 172.21.15.46:0/3397239818 <== mds.0 v2:172.21.15.73:6836/2780269530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e500c92790 con 0x55e500e92a00 2024-03-29T08:18:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca73400640 10 client.4608 handle_command_reply: tid=0 2024-03-29T08:18:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 1 client.4608 shutdown 2024-03-29T08:18:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 1 -- 172.21.15.46:0/3397239818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 msgr2=0x55e500e905a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 1 --2- 172.21.15.46:0/3397239818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500e905a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fca800139a0 tx=0x7fca80001260 comp rx=0 tx=0).stop 2024-03-29T08:18:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 1 -- 172.21.15.46:0/3397239818 shutdown_connections 2024-03-29T08:18:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 1 --2- 172.21.15.46:0/3397239818 >> [v2:172.21.15.73:6836/2780269530,v1:172.21.15.73:6837/2780269530] conn(0x55e500e92a00 0x55e500e92de0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 1 --2- 172.21.15.46:0/3397239818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e90db0 0x55e500e91ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 1 --2- 172.21.15.46:0/3397239818 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e916d0 0x55e500e91ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 1 --2- 172.21.15.46:0/3397239818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500e905a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca73400640 0 client.4608 ms_handle_reset on v2:172.21.15.73:6836/2780269530 2024-03-29T08:18:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 1 -- 172.21.15.46:0/3397239818 >> 172.21.15.46:0/3397239818 conn(0x55e500e4d830 msgr2=0x55e500e4bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 1 -- 172.21.15.46:0/3397239818 shutdown_connections 2024-03-29T08:18:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.664+0000 7fca9812f000 1 -- 172.21.15.46:0/3397239818 wait complete. 2024-03-29T08:18:27.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.668+0000 7fca9812f000 20 client.4608 trim_cache size 0 max 16384 2024-03-29T08:18:27.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.680+0000 7fca9812f000 1 Processor -- start 2024-03-29T08:18:27.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.680+0000 7fca9812f000 1 -- start start 2024-03-29T08:18:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.680+0000 7fca9812f000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500e45fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e90db0 0x55e500e46520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e916d0 0x55e500ea6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e500952940 con 0x55e500e474c0 2024-03-29T08:18:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e500c32c10 con 0x55e500e90db0 2024-03-29T08:18:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e500d32400 con 0x55e500e916d0 2024-03-29T08:18:27.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca8de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500e45fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca8de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500e45fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51240/0 (socket says 172.21.15.46:51240) 2024-03-29T08:18:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca8de00640 1 -- 172.21.15.46:0/4266771002 learned_addr learned my addr 172.21.15.46:0/4266771002 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca85400640 1 --2- 172.21.15.46:0/4266771002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e90db0 0x55e500e46520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca8e800640 1 --2- 172.21.15.46:0/4266771002 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e916d0 0x55e500ea6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca8de00640 1 -- 172.21.15.46:0/4266771002 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e916d0 msgr2=0x55e500ea6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca8de00640 1 --2- 172.21.15.46:0/4266771002 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e916d0 0x55e500ea6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca8de00640 1 -- 172.21.15.46:0/4266771002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e90db0 msgr2=0x55e500e46520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca8de00640 1 --2- 172.21.15.46:0/4266771002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e90db0 0x55e500e46520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca8de00640 1 -- 172.21.15.46:0/4266771002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e500c5aab0 con 0x55e500e474c0 2024-03-29T08:18:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca8de00640 1 --2- 172.21.15.46:0/4266771002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500e45fe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fca80007430 tx=0x7fca80021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c3b06e2dda45e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca7f400640 1 -- 172.21.15.46:0/4266771002 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80012e30 con 0x55e500e474c0 2024-03-29T08:18:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca7f400640 1 -- 172.21.15.46:0/4266771002 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8000b070 con 0x55e500e474c0 2024-03-29T08:18:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca7f400640 1 -- 172.21.15.46:0/4266771002 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80012920 con 0x55e500e474c0 2024-03-29T08:18:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 -- 172.21.15.46:0/4266771002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 msgr2=0x55e500e45fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 --2- 172.21.15.46:0/4266771002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500e45fe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fca80007430 tx=0x7fca80021000 comp rx=0 tx=0).stop 2024-03-29T08:18:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 -- 172.21.15.46:0/4266771002 shutdown_connections 2024-03-29T08:18:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 --2- 172.21.15.46:0/4266771002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e90db0 0x55e500e46520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 --2- 172.21.15.46:0/4266771002 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e916d0 0x55e500ea6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 --2- 172.21.15.46:0/4266771002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500e45fe0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 -- 172.21.15.46:0/4266771002 >> 172.21.15.46:0/4266771002 conn(0x55e500e4d830 msgr2=0x55e500e8f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 -- 172.21.15.46:0/4266771002 shutdown_connections 2024-03-29T08:18:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 -- 172.21.15.46:0/4266771002 wait complete. 2024-03-29T08:18:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 -- start start 2024-03-29T08:18:27.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 Processor -- start 2024-03-29T08:18:27.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500ea9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e90db0 0x55e500eaf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e916d0 0x55e500eafa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e500c931c0 con 0x55e500e474c0 2024-03-29T08:18:27.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e500c5cc60 con 0x55e500e916d0 2024-03-29T08:18:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e500c999b0 con 0x55e500e90db0 2024-03-29T08:18:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca9812f000 10 client.0 _ll_register_callbacks cb 0x55e500c5a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:18:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca8de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500ea9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca8de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500ea9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51252/0 (socket says 172.21.15.46:51252) 2024-03-29T08:18:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca8de00640 1 -- 172.21.15.46:0/3551491682 learned_addr learned my addr 172.21.15.46:0/3551491682 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca85400640 1 --2- 172.21.15.46:0/3551491682 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e90db0 0x55e500eaf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.684+0000 7fca8e800640 1 --2- 172.21.15.46:0/3551491682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e916d0 0x55e500eafa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca8de00640 1 -- 172.21.15.46:0/3551491682 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e90db0 msgr2=0x55e500eaf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca8de00640 1 --2- 172.21.15.46:0/3551491682 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e90db0 0x55e500eaf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca8de00640 1 -- 172.21.15.46:0/3551491682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e916d0 msgr2=0x55e500eafa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca8de00640 1 --2- 172.21.15.46:0/3551491682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e916d0 0x55e500eafa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca8de00640 1 -- 172.21.15.46:0/3551491682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca90162ae0 con 0x55e500e474c0 2024-03-29T08:18:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca8e800640 1 --2- 172.21.15.46:0/3551491682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e916d0 0x55e500eafa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca8de00640 1 --2- 172.21.15.46:0/3551491682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500ea9c80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fca80007550 tx=0x7fca8000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58379bfdf518af1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:27.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca73400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:18:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca73400640 1 -- 172.21.15.46:0/3551491682 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80012e30 con 0x55e500e474c0 2024-03-29T08:18:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca73400640 1 -- 172.21.15.46:0/3551491682 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca8002b650 con 0x55e500e474c0 2024-03-29T08:18:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 1 -- 172.21.15.46:0/3551491682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e500e4ccc0 con 0x55e500e474c0 2024-03-29T08:18:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca73400640 1 -- 172.21.15.46:0/3551491682 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca8002b810 con 0x55e500e474c0 2024-03-29T08:18:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 10 client.4614 fetch_fsmap learned FSMap version 8 2024-03-29T08:18:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 1 -- 172.21.15.46:0/3551491682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55e500c40c00 con 0x55e500e474c0 2024-03-29T08:18:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca73400640 1 -- 172.21.15.46:0/3551491682 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e500e4ccc0 con 0x55e500e474c0 2024-03-29T08:18:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca73400640 1 -- 172.21.15.46:0/3551491682 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fca8002b970 con 0x55e500e474c0 2024-03-29T08:18:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 10 client.4614 fetch_fsmap finished waiting for FSMap version 8 2024-03-29T08:18:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 10 client.4614 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-29T08:18:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 20 client.4614 populate_metadata read hostname 'smithi046' 2024-03-29T08:18:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 1 --2- 172.21.15.46:0/3551491682 >> [v2:172.21.15.73:6836/2780269530,v1:172.21.15.73:6837/2780269530] conn(0x55e500eaff70 0x55e500ef3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 4 client.4614 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T08:18:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 1 -- 172.21.15.46:0/3551491682 --> [v2:172.21.15.73:6836/2780269530,v1:172.21.15.73:6837/2780269530] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e500c65460 con 0x55e500eaff70 2024-03-29T08:18:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca85400640 1 --2- 172.21.15.46:0/3551491682 >> [v2:172.21.15.73:6836/2780269530,v1:172.21.15.73:6837/2780269530] conn(0x55e500eaff70 0x55e500ef3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:27.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca73400640 10 client.4614 ms_handle_connect on v2:172.21.15.73:6836/2780269530 2024-03-29T08:18:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca85400640 1 --2- 172.21.15.46:0/3551491682 >> [v2:172.21.15.73:6836/2780269530,v1:172.21.15.73:6837/2780269530] conn(0x55e500eaff70 0x55e500ef3d10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e242f116a8bbee3a server_cookie=187eb26369e8d60a in_seq=0 out_seq=0 2024-03-29T08:18:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca73400640 1 -- 172.21.15.46:0/3551491682 <== mds.0 v2:172.21.15.73:6836/2780269530 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55e500c65460 con 0x55e500eaff70 2024-03-29T08:18:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca73400640 10 client.4614 handle_command_reply: tid=0 2024-03-29T08:18:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 1 client.4614 shutdown 2024-03-29T08:18:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 1 -- 172.21.15.46:0/3551491682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 msgr2=0x55e500ea9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 1 --2- 172.21.15.46:0/3551491682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500ea9c80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fca80007550 tx=0x7fca8000a000 comp rx=0 tx=0).stop 2024-03-29T08:18:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 1 -- 172.21.15.46:0/3551491682 shutdown_connections 2024-03-29T08:18:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 1 --2- 172.21.15.46:0/3551491682 >> [v2:172.21.15.73:6836/2780269530,v1:172.21.15.73:6837/2780269530] conn(0x55e500eaff70 0x55e500ef3d10 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 1 --2- 172.21.15.46:0/3551491682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e500e916d0 0x55e500eafa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 1 --2- 172.21.15.46:0/3551491682 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e500e90db0 0x55e500eaf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca73400640 0 client.4614 ms_handle_reset on v2:172.21.15.73:6836/2780269530 2024-03-29T08:18:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 1 --2- 172.21.15.46:0/3551491682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e500e474c0 0x55e500ea9c80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 1 -- 172.21.15.46:0/3551491682 >> 172.21.15.46:0/3551491682 conn(0x55e500e4d830 msgr2=0x55e500e44fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:27.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 1 -- 172.21.15.46:0/3551491682 shutdown_connections 2024-03-29T08:18:27.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 1 -- 172.21.15.46:0/3551491682 wait complete. 2024-03-29T08:18:27.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca9812f000 20 client.4614 trim_cache size 0 max 16384 2024-03-29T08:18:27.695 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:18:27.695 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "e9b7960f-ec1f-4787-95c3-c806094d37f0", 2024-03-29T08:18:27.695 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-03-29T08:18:27.695 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4291, 2024-03-29T08:18:27.695 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-29T08:18:27.695 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-29T08:18:27.695 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-29T08:18:27.695 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 14.375486581000001, 2024-03-29T08:18:27.695 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 8, 2024-03-29T08:18:27.696 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 22, 2024-03-29T08:18:27.696 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 22, 2024-03-29T08:18:27.696 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 21.258485311000001 2024-03-29T08:18:27.696 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:18:27.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca94c00640 1 -- 172.21.15.46:0/1086260148 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca6c066690 msgr2=0x7fca6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:27.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca94c00640 1 --2- 172.21.15.46:0/1086260148 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca6c066690 0x7fca6c068b50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fca800012f0 tx=0x7fca80001660 comp rx=0 tx=0).stop 2024-03-29T08:18:27.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca94c00640 1 -- 172.21.15.46:0/1086260148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca900af530 msgr2=0x7fca90164e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:27.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca94c00640 1 --2- 172.21.15.46:0/1086260148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca900af530 0x7fca90164e60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fca7800e9d0 tx=0x7fca7800ee90 comp rx=0 tx=0).stop 2024-03-29T08:18:27.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca94c00640 1 -- 172.21.15.46:0/1086260148 shutdown_connections 2024-03-29T08:18:27.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca94c00640 1 --2- 172.21.15.46:0/1086260148 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca6c066690 0x7fca6c068b50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca94c00640 1 --2- 172.21.15.46:0/1086260148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca90158300 0x7fca900c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca94c00640 1 --2- 172.21.15.46:0/1086260148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca900af530 0x7fca90164e60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca94c00640 1 --2- 172.21.15.46:0/1086260148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca900702a0 0x7fca90154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:27.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca94c00640 1 -- 172.21.15.46:0/1086260148 >> 172.21.15.46:0/1086260148 conn(0x7fca90066080 msgr2=0x7fca90069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:27.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.688+0000 7fca94c00640 1 -- 172.21.15.46:0/1086260148 shutdown_connections 2024-03-29T08:18:27.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:27.692+0000 7fca94c00640 1 -- 172.21.15.46:0/1086260148 wait complete. 2024-03-29T08:18:27.709 INFO:teuthology.run_tasks:Running task sequential... 2024-03-29T08:18:27.719 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-03-29T08:18:27.719 INFO:teuthology.task.sequential:In sequential, running task print... 2024-03-29T08:18:27.720 INFO:teuthology.task.print:Enabling mgr modules 2024-03-29T08:18:27.720 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-29T08:18:27.745 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-29T08:18:27.745 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-03-29T08:18:27.745 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-29T08:18:27.745 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T08:18:27.746 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:18:27.746 INFO:tasks.ceph_fuse:client.2 config is {} 2024-03-29T08:18:27.746 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T08:18:27.746 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-29T08:18:27.746 INFO:tasks.ceph_fuse:client.1 config is {} 2024-03-29T08:18:27.746 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T08:18:27.746 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T08:18:27.746 INFO:tasks.ceph_fuse:client.3 config is {} 2024-03-29T08:18:27.746 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-29T08:18:27.746 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-29T08:18:27.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:27.747 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-03-29T08:18:27.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:27.758 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link delete ceph-brx 2024-03-29T08:18:27.824 INFO:teuthology.orchestra.run.smithi073.stderr:Cannot find device "ceph-brx" 2024-03-29T08:18:27.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:18:27.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:27.827 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:18:27.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:27.837 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-03-29T08:18:27.903 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2024-03-29T08:18:27.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:18:27.905 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-29T08:18:27.905 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:18:28.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.084+0000 7fc00b800640 1 Processor -- start 2024-03-29T08:18:28.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.084+0000 7fc00b800640 1 -- start start 2024-03-29T08:18:28.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.088+0000 7fc00b800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0040ceb20 0x7fc0041656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:28.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.088+0000 7fc00b800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0040cb510 con 0x7fc00415ada0 2024-03-29T08:18:28.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.088+0000 7fc00b800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc004070580 con 0x7fc004144210 2024-03-29T08:18:28.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.088+0000 7fc00b800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00406d960 con 0x7fc0040ceb20 2024-03-29T08:18:28.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.088+0000 7fc009800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0040ceb20 0x7fc0041656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:28.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.088+0000 7fc003e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc00415ada0 0x7fc0040ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45222/0 (socket says 172.21.15.46:45222) 2024-03-29T08:18:28.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.088+0000 7fc003e00640 1 -- 172.21.15.46:0/30896429 learned_addr learned my addr 172.21.15.46:0/30896429 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:28.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.088+0000 7fc009800640 1 -- 172.21.15.46:0/30896429 >> v1:172.21.15.73:6789/0 conn(0x7fc004144210 legacy=0x7fc0040cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:28.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.088+0000 7fc009800640 1 -- 172.21.15.46:0/30896429 >> v1:172.21.15.46:6789/0 conn(0x7fc00415ada0 legacy=0x7fc0040ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:28.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.088+0000 7fc009800640 1 -- 172.21.15.46:0/30896429 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc004070860 con 0x7fc0040ceb20 2024-03-29T08:18:28.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc009800640 1 --2- 172.21.15.46:0/30896429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040ceb20 0x7fc0041656d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009d90 tx=0x7fbffc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0aaf0d66ab4557 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:28.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc003400640 1 -- 172.21.15.46:0/30896429 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0a5920 con 0x7fc0040ceb20 2024-03-29T08:18:28.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc003400640 1 -- 172.21.15.46:0/30896429 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc0a5a80 con 0x7fc0040ceb20 2024-03-29T08:18:28.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc003400640 1 -- 172.21.15.46:0/30896429 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0a5e10 con 0x7fc0040ceb20 2024-03-29T08:18:28.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc00b800640 1 -- 172.21.15.46:0/30896429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040ceb20 msgr2=0x7fc0041656d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:28.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc00b800640 1 --2- 172.21.15.46:0/30896429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040ceb20 0x7fc0041656d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009d90 tx=0x7fbffc01c970 comp rx=0 tx=0).stop 2024-03-29T08:18:28.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc00b800640 1 -- 172.21.15.46:0/30896429 shutdown_connections 2024-03-29T08:18:28.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc00b800640 1 --2- 172.21.15.46:0/30896429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040ceb20 0x7fc0041656d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc00b800640 1 -- 172.21.15.46:0/30896429 >> 172.21.15.46:0/30896429 conn(0x7fc004066110 msgr2=0x7fc004069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:28.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc00b800640 1 -- 172.21.15.46:0/30896429 shutdown_connections 2024-03-29T08:18:28.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc00b800640 1 -- 172.21.15.46:0/30896429 wait complete. 2024-03-29T08:18:28.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc00b800640 1 Processor -- start 2024-03-29T08:18:28.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc00b800640 1 -- start start 2024-03-29T08:18:28.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc00b800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0040ceb20 0x7fc004162bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:28.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc00b800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc004144210 0x7fc004140370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:28.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc00b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415ada0 0x7fc004160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:28.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc00b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00415f7f0 con 0x7fc00415ada0 2024-03-29T08:18:28.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc00b800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00415fad0 con 0x7fc0040ceb20 2024-03-29T08:18:28.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc00b800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00415fdb0 con 0x7fc004144210 2024-03-29T08:18:28.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc009800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415ada0 0x7fc004160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:28.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc009800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415ada0 0x7fc004160890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51266/0 (socket says 172.21.15.46:51266) 2024-03-29T08:18:28.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.092+0000 7fc009800640 1 -- 172.21.15.46:0/786734458 learned_addr learned my addr 172.21.15.46:0/786734458 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:28.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.096+0000 7fc008e00640 1 --2- 172.21.15.46:0/786734458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0040ceb20 0x7fc004162bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.096+0000 7fc003e00640 1 --2- 172.21.15.46:0/786734458 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc004144210 0x7fc004140370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.096+0000 7fc009800640 1 -- 172.21.15.46:0/786734458 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc004144210 msgr2=0x7fc004140370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.096+0000 7fc009800640 1 --2- 172.21.15.46:0/786734458 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc004144210 0x7fc004140370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.096+0000 7fc009800640 1 -- 172.21.15.46:0/786734458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0040ceb20 msgr2=0x7fc004162bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.096+0000 7fc009800640 1 --2- 172.21.15.46:0/786734458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0040ceb20 0x7fc004162bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.096+0000 7fc009800640 1 -- 172.21.15.46:0/786734458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc009a70 con 0x7fc00415ada0 2024-03-29T08:18:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.096+0000 7fc008e00640 1 --2- 172.21.15.46:0/786734458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0040ceb20 0x7fc004162bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.096+0000 7fc003e00640 1 --2- 172.21.15.46:0/786734458 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc004144210 0x7fc004140370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.096+0000 7fc009800640 1 --2- 172.21.15.46:0/786734458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415ada0 0x7fc004160890 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009d90 tx=0x7fbffc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.096+0000 7fc000c00640 1 -- 172.21.15.46:0/786734458 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0aedb0 con 0x7fc00415ada0 2024-03-29T08:18:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.096+0000 7fc000c00640 1 -- 172.21.15.46:0/786734458 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc0b7410 con 0x7fc00415ada0 2024-03-29T08:18:28.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.096+0000 7fc000c00640 1 -- 172.21.15.46:0/786734458 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0b77c0 con 0x7fc00415ada0 2024-03-29T08:18:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.096+0000 7fc00b800640 1 -- 172.21.15.46:0/786734458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc004160090 con 0x7fc00415ada0 2024-03-29T08:18:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.096+0000 7fc00b800640 1 -- 172.21.15.46:0/786734458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0041408b0 con 0x7fc00415ada0 2024-03-29T08:18:28.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.096+0000 7fc00b800640 1 -- 172.21.15.46:0/786734458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0040ce280 con 0x7fc00415ada0 2024-03-29T08:18:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.096+0000 7fc000c00640 1 -- 172.21.15.46:0/786734458 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbffc0b4050 con 0x7fc00415ada0 2024-03-29T08:18:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.100+0000 7fc000c00640 1 --2- 172.21.15.46:0/786734458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbfe4066690 0x7fbfe4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.100+0000 7fc000c00640 1 -- 172.21.15.46:0/786734458 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbffc11f570 con 0x7fc00415ada0 2024-03-29T08:18:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.100+0000 7fc008e00640 1 --2- 172.21.15.46:0/786734458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbfe4066690 0x7fbfe4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:28.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.100+0000 7fc008e00640 1 --2- 172.21.15.46:0/786734458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbfe4066690 0x7fbfe4068b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbff8000f70 tx=0x7fbff8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:28.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.104+0000 7fc000c00640 1 -- 172.21.15.46:0/786734458 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbffc0ec840 con 0x7fc00415ada0 2024-03-29T08:18:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.304+0000 7fc00b800640 1 -- 172.21.15.46:0/786734458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc004067410 con 0x7fc00415ada0 2024-03-29T08:18:28.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.304+0000 7fc000c00640 1 -- 172.21.15.46:0/786734458 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbffc0b2060 con 0x7fc00415ada0 2024-03-29T08:18:28.304 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:18:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.304+0000 7fc00b800640 1 -- 172.21.15.46:0/786734458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbfe4066690 msgr2=0x7fbfe4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.304+0000 7fc00b800640 1 --2- 172.21.15.46:0/786734458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbfe4066690 0x7fbfe4068b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbff8000f70 tx=0x7fbff8009040 comp rx=0 tx=0).stop 2024-03-29T08:18:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.304+0000 7fc00b800640 1 -- 172.21.15.46:0/786734458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415ada0 msgr2=0x7fc004160890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:28.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.304+0000 7fc00b800640 1 --2- 172.21.15.46:0/786734458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415ada0 0x7fc004160890 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009d90 tx=0x7fbffc019710 comp rx=0 tx=0).stop 2024-03-29T08:18:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.304+0000 7fc00b800640 1 -- 172.21.15.46:0/786734458 shutdown_connections 2024-03-29T08:18:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.304+0000 7fc00b800640 1 --2- 172.21.15.46:0/786734458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbfe4066690 0x7fbfe4068b50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.304+0000 7fc00b800640 1 --2- 172.21.15.46:0/786734458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc00415ada0 0x7fc004160890 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.304+0000 7fc00b800640 1 --2- 172.21.15.46:0/786734458 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc004144210 0x7fc004140370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.304+0000 7fc00b800640 1 --2- 172.21.15.46:0/786734458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0040ceb20 0x7fc004162bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.304+0000 7fc00b800640 1 -- 172.21.15.46:0/786734458 >> 172.21.15.46:0/786734458 conn(0x7fc004066110 msgr2=0x7fc0040ae530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.304+0000 7fc00b800640 1 -- 172.21.15.46:0/786734458 shutdown_connections 2024-03-29T08:18:28.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:28.304+0000 7fc00b800640 1 -- 172.21.15.46:0/786734458 wait complete. 2024-03-29T08:18:28.324 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:18:28.324 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:18:28.325 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-29T08:18:28.325 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T08:18:28.325 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:18:28.325 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:18:28.325 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:18:28.325 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:18:28.325 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:18:28.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:28.325 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-29T08:18:28.335 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:18:28.335 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:18:28.335 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:18:28.335 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:28.336 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:18:28.336 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:28.336 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:18:28.336 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:28.336 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-29T08:18:28.336 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:18:28.336 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:28.336 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-29T08:18:28.336 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:18:28.336 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:28.336 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-29T08:18:28.336 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T08:18:28.336 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42429sec preferred_lft 42429sec 2024-03-29T08:18:28.337 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-29T08:18:28.337 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:28.337 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:18:28.337 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:28.337 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-29T08:18:28.337 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:18:28.337 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:d9:58:09:4c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:28.337 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:18:28.337 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:28.337 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T08:18:28.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:28.338 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:18:28.338 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:18:28.338 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-03-29T08:18:28.338 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-03-29T08:18:28.338 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-03-29T08:18:28.338 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T08:18:28.338 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:18:28.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:28.451 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:18:28.470 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:18:28.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:28.472 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-29T08:18:28.541 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-29T08:18:28.542 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:18:28.542 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T08:18:28.542 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:18:28.542 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T08:18:28.542 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:18:28.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:28.542 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:18:28.543 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:18:28.543 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:18:28.543 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:18:28.543 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:18:28.543 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:18:28.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:28.632 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:18:28.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:28.643 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-03-29T08:18:28.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:28.696 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:18:28.696 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:18:28.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:18:28.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-29T08:18:28.696 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:18:28.779 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-29T08:18:28.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:28.779 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:18:28.779 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:18:28.779 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-29T08:18:28.779 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T08:18:28.779 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-29T08:18:28.779 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-29T08:18:28.779 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-29T08:18:28.779 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:18:28.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:28.944 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:18:28.944 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:18:28.944 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-03-29T08:18:28.944 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T08:18:28.944 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:18:28.983 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T08:18:28.984 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:18:28.984 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T08:18:29.035 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:18:29.036 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:18:29.036 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:18:29.087 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:18:29.152 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:18:29.152 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:18:29.214 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:18:29.215 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:18:29.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:29.216 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:18:29.268 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:18:29.268 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T08:18:29.314 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:18:29.336 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:18:29.336 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:18:29.398 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:18:29.399 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:18:29.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:29.400 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:18:29.413 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-03-29T08:18:29.412+0000 7f579c411580 -1 init, newargv = 0x55dbddab5fc0 newargc=15 2024-03-29T08:18:29.413 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[33375]: starting ceph client 2024-03-29T08:18:29.423 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[33375]: starting fuse 2024-03-29T08:18:30.523 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:18:30.548 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:18:30.548 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:18:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:18:30.617 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:18:30.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:30.618 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:18:30.668 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:18:30.668 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-29T08:18:30.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:18:30.669 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-03-29T08:18:30.669 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-03-29T08:18:30.669 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:18:30.669 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-03-29T08:18:30.669 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:18:30.669 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-03-29T08:18:30.669 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T08:18:30.669 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-03-29T08:18:30.670 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:18:30.670 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:18:30.670 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-03-29T08:18:30.670 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-03-29T08:18:30.670 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-03-29T08:18:30.670 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-03-29T08:18:30.670 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:18:30.670 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-03-29T08:18:30.670 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:18:30.670 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:18:30.670 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:18:30.670 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-03-29T08:18:30.670 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-03-29T08:18:30.671 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-03-29T08:18:30.671 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:18:30.671 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:18:30.671 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-03-29T08:18:30.671 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:18:30.801 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.33375.asok 2024-03-29T08:18:30.984 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.33375.asok 2024-03-29T08:18:30.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:30.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.33375.asok status 2024-03-29T08:18:31.170 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:18:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-03-29T08:18:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:18:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:18:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-03-29T08:18:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-03-29T08:18:31.170 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T08:18:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "33375", 2024-03-29T08:18:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-03-29T08:18:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:18:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-03-29T08:18:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-03-29T08:18:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4629, 2024-03-29T08:18:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-03-29T08:18:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-03-29T08:18:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-03-29T08:18:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 4629 2024-03-29T08:18:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:18:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:18:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:18:31.171 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:18:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 237289534 2024-03-29T08:18:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:18:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:18:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:18:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:18:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:18:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 237289534 2024-03-29T08:18:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:18:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.4629 192.168.0.1:0/237289534", 2024-03-29T08:18:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/237289534", 2024-03-29T08:18:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-03-29T08:18:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 8, 2024-03-29T08:18:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 22, 2024-03-29T08:18:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:18:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-03-29T08:18:31.172 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-03-29T08:18:31.173 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:18:31.186 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:18:31.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.324+0000 7fd42a800640 1 Processor -- start 2024-03-29T08:18:31.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.324+0000 7fd42a800640 1 -- start start 2024-03-29T08:18:31.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.328+0000 7fd42a800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd42415cd90 0x7fd42406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:31.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.328+0000 7fd42a800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd42415d940 0x7fd4241597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:31.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.328+0000 7fd42a800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd42406d130 con 0x7fd4241593d0 2024-03-29T08:18:31.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.328+0000 7fd42a800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd42415b8f0 con 0x7fd42415d940 2024-03-29T08:18:31.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.328+0000 7fd42a800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42415ad50 con 0x7fd42415cd90 2024-03-29T08:18:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.328+0000 7fd423e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd42415d940 0x7fd4241597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.328+0000 7fd41be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd42415cd90 0x7fd42406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.328+0000 7fd423e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd42415d940 0x7fd4241597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:44456/0 (socket says 172.21.15.46:44456) 2024-03-29T08:18:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.328+0000 7fd423e00640 1 -- 172.21.15.46:0/2918635689 learned_addr learned my addr 172.21.15.46:0/2918635689 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.328+0000 7fd41be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd42415cd90 0x7fd42406cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:39038/0 (socket says 172.21.15.46:39038) 2024-03-29T08:18:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.328+0000 7fd423e00640 1 -- 172.21.15.46:0/2918635689 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd42415cd90 msgr2=0x7fd42406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.328+0000 7fd423e00640 1 --2- 172.21.15.46:0/2918635689 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd42415cd90 0x7fd42406cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.328+0000 7fd423e00640 1 -- 172.21.15.46:0/2918635689 >> v1:172.21.15.73:6789/0 conn(0x7fd4241593d0 legacy=0x7fd42415d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.328+0000 7fd423e00640 1 -- 172.21.15.46:0/2918635689 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42415afc0 con 0x7fd42415d940 2024-03-29T08:18:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.328+0000 7fd422a00640 1 -- 172.21.15.46:0/2918635689 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3324282182 0 0) 0x7fd42406d130 con 0x7fd4241593d0 2024-03-29T08:18:31.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.328+0000 7fd41be00640 1 --2- 172.21.15.46:0/2918635689 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd42415cd90 0x7fd42406cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:31.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd423e00640 1 --2- 172.21.15.46:0/2918635689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd42415d940 0x7fd4241597b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009e50 tx=0x7fd41c01ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f4e905b6096475e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:31.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd422a00640 1 -- 172.21.15.46:0/2918635689 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c0a5900 con 0x7fd42415d940 2024-03-29T08:18:31.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd422a00640 1 -- 172.21.15.46:0/2918635689 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41c0a5a60 con 0x7fd42415d940 2024-03-29T08:18:31.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd422a00640 1 -- 172.21.15.46:0/2918635689 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c0a5df0 con 0x7fd42415d940 2024-03-29T08:18:31.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd42a800640 1 -- 172.21.15.46:0/2918635689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd42415d940 msgr2=0x7fd4241597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:31.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd42a800640 1 --2- 172.21.15.46:0/2918635689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd42415d940 0x7fd4241597b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd41c009e50 tx=0x7fd41c01ca30 comp rx=0 tx=0).stop 2024-03-29T08:18:31.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd42a800640 1 -- 172.21.15.46:0/2918635689 shutdown_connections 2024-03-29T08:18:31.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd42a800640 1 --2- 172.21.15.46:0/2918635689 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd42415cd90 0x7fd42406cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:31.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd42a800640 1 --2- 172.21.15.46:0/2918635689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd42415d940 0x7fd4241597b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:31.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd42a800640 1 -- 172.21.15.46:0/2918635689 >> 172.21.15.46:0/2918635689 conn(0x7fd424066110 msgr2=0x7fd424069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:31.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd42a800640 1 -- 172.21.15.46:0/2918635689 shutdown_connections 2024-03-29T08:18:31.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd42a800640 1 -- 172.21.15.46:0/2918635689 wait complete. 2024-03-29T08:18:31.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd42a800640 1 Processor -- start 2024-03-29T08:18:31.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd42a800640 1 -- start start 2024-03-29T08:18:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd42a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4241593d0 0x7fd4241f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd42a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd42415cd90 0x7fd424157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd42a800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd42415d940 0x7fd4241535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd42a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd424153150 con 0x7fd42415cd90 2024-03-29T08:18:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd42a800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd424152540 con 0x7fd4241593d0 2024-03-29T08:18:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd42a800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd424152820 con 0x7fd42415d940 2024-03-29T08:18:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd41be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd42415cd90 0x7fd424157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:31.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.332+0000 7fd41be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd42415cd90 0x7fd424157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51276/0 (socket says 172.21.15.46:51276) 2024-03-29T08:18:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.336+0000 7fd41be00640 1 -- 172.21.15.46:0/3105851879 learned_addr learned my addr 172.21.15.46:0/3105851879 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.336+0000 7fd423e00640 1 --2- 172.21.15.46:0/3105851879 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd42415d940 0x7fd4241535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.336+0000 7fd423400640 1 --2- 172.21.15.46:0/3105851879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4241593d0 0x7fd4241f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.336+0000 7fd41be00640 1 -- 172.21.15.46:0/3105851879 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd42415d940 msgr2=0x7fd4241535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.336+0000 7fd41be00640 1 --2- 172.21.15.46:0/3105851879 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd42415d940 0x7fd4241535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.336+0000 7fd41be00640 1 -- 172.21.15.46:0/3105851879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4241593d0 msgr2=0x7fd4241f5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.336+0000 7fd41be00640 1 --2- 172.21.15.46:0/3105851879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4241593d0 0x7fd4241f5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.336+0000 7fd41be00640 1 -- 172.21.15.46:0/3105851879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c009420 con 0x7fd42415cd90 2024-03-29T08:18:31.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.336+0000 7fd41be00640 1 --2- 172.21.15.46:0/3105851879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd42415cd90 0x7fd424157f30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00e980 tx=0x7fd40c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:31.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.336+0000 7fd41b400640 1 -- 172.21.15.46:0/3105851879 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c00ca00 con 0x7fd42415cd90 2024-03-29T08:18:31.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.336+0000 7fd41b400640 1 -- 172.21.15.46:0/3105851879 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40c00cb60 con 0x7fd42415cd90 2024-03-29T08:18:31.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.336+0000 7fd41b400640 1 -- 172.21.15.46:0/3105851879 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c013470 con 0x7fd42415cd90 2024-03-29T08:18:31.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.336+0000 7fd42a800640 1 -- 172.21.15.46:0/3105851879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41c009b30 con 0x7fd42415cd90 2024-03-29T08:18:31.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.336+0000 7fd42a800640 1 -- 172.21.15.46:0/3105851879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd424152de0 con 0x7fd42415cd90 2024-03-29T08:18:31.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.336+0000 7fd42a800640 1 -- 172.21.15.46:0/3105851879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd424066c90 con 0x7fd42415cd90 2024-03-29T08:18:31.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.336+0000 7fd41b400640 1 -- 172.21.15.46:0/3105851879 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd40c0135d0 con 0x7fd42415cd90 2024-03-29T08:18:31.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.340+0000 7fd41b400640 1 --2- 172.21.15.46:0/3105851879 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd400066690 0x7fd400068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:31.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.340+0000 7fd41b400640 1 -- 172.21.15.46:0/3105851879 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd40c084830 con 0x7fd42415cd90 2024-03-29T08:18:31.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.340+0000 7fd423400640 1 --2- 172.21.15.46:0/3105851879 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd400066690 0x7fd400068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:31.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.340+0000 7fd423400640 1 --2- 172.21.15.46:0/3105851879 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd400066690 0x7fd400068b50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd414000f70 tx=0x7fd414009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:31.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.344+0000 7fd41b400640 1 -- 172.21.15.46:0/3105851879 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd40c051b00 con 0x7fd42415cd90 2024-03-29T08:18:31.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.536+0000 7fd42a800640 1 -- 172.21.15.46:0/3105851879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd424153290 con 0x7fd42415cd90 2024-03-29T08:18:31.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.540+0000 7fd41b400640 1 -- 172.21.15.46:0/3105851879 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd40c055120 con 0x7fd42415cd90 2024-03-29T08:18:31.540 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:18:31.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.540+0000 7fd42a800640 1 -- 172.21.15.46:0/3105851879 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd400066690 msgr2=0x7fd400068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:31.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.540+0000 7fd42a800640 1 --2- 172.21.15.46:0/3105851879 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd400066690 0x7fd400068b50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd414000f70 tx=0x7fd414009040 comp rx=0 tx=0).stop 2024-03-29T08:18:31.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.540+0000 7fd42a800640 1 -- 172.21.15.46:0/3105851879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd42415cd90 msgr2=0x7fd424157f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:31.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.540+0000 7fd42a800640 1 --2- 172.21.15.46:0/3105851879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd42415cd90 0x7fd424157f30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00e980 tx=0x7fd40c00ee40 comp rx=0 tx=0).stop 2024-03-29T08:18:31.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.540+0000 7fd42a800640 1 -- 172.21.15.46:0/3105851879 shutdown_connections 2024-03-29T08:18:31.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.540+0000 7fd42a800640 1 --2- 172.21.15.46:0/3105851879 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd400066690 0x7fd400068b50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:31.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.540+0000 7fd42a800640 1 --2- 172.21.15.46:0/3105851879 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd42415d940 0x7fd4241535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:31.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.540+0000 7fd42a800640 1 --2- 172.21.15.46:0/3105851879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd42415cd90 0x7fd424157f30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:31.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.540+0000 7fd42a800640 1 --2- 172.21.15.46:0/3105851879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4241593d0 0x7fd4241f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:31.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.540+0000 7fd42a800640 1 -- 172.21.15.46:0/3105851879 >> 172.21.15.46:0/3105851879 conn(0x7fd424066110 msgr2=0x7fd424163140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:31.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.540+0000 7fd42a800640 1 -- 172.21.15.46:0/3105851879 shutdown_connections 2024-03-29T08:18:31.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:31.540+0000 7fd42a800640 1 -- 172.21.15.46:0/3105851879 wait complete. 2024-03-29T08:18:31.561 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:18:31.561 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:18:31.561 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-29T08:18:31.561 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-03-29T08:18:31.561 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-03-29T08:18:31.561 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:18:31.561 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:18:31.561 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:18:31.561 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-03-29T08:18:31.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:31.562 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-29T08:18:31.570 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:18:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:18:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:18:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:18:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:31.571 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:18:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-29T08:18:31.571 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:18:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-29T08:18:31.571 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:18:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:31.571 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-29T08:18:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T08:18:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42426sec preferred_lft 42426sec 2024-03-29T08:18:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-29T08:18:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:31.572 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:18:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-29T08:18:31.572 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:18:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:d9:58:09:4c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:18:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:31.572 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:18:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T08:18:31.572 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T08:18:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:31.573 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:18:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:18:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T08:18:31.573 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:31.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:31.574 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:18:31.633 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:18:31.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:31.635 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-29T08:18:31.692 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-29T08:18:31.693 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:18:31.693 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T08:18:31.693 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:18:31.693 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T08:18:31.693 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:18:31.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:31.693 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:18:31.693 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:18:31.693 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:18:31.693 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:18:31.693 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:18:31.694 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:18:31.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:31.785 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:18:31.794 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T08:18:31.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:31.795 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-03-29T08:18:31.843 INFO:teuthology.orchestra.run.smithi046.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-03-29T08:18:31.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:31.844 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:18:31.844 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:18:31.844 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T08:18:31.844 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-03-29T08:18:31.844 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:18:31.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:31.926 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-29T08:18:31.988 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:18:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:18:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:18:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:31.988 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:18:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:31.989 INFO:teuthology.orchestra.run.smithi046.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:18:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether ba:d6:dc:cb:b0:f5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T08:18:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T08:18:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::b8d6:dcff:fecb:b0f5/64 scope link 2024-03-29T08:18:31.989 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:31.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:31.990 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-03-29T08:18:32.052 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:18:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:18:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:18:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:18:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:32.052 INFO:teuthology.orchestra.run.smithi046.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:18:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether ba:d6:dc:cb:b0:f5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T08:18:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T08:18:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:32.052 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::b8d6:dcff:fecb:b0f5/64 scope link 2024-03-29T08:18:32.053 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:32.054 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-03-29T08:18:32.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:32.054 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:18:32.054 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:18:32.054 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-03-29T08:18:32.054 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-29T08:18:32.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-03-29T08:18:32.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-03-29T08:18:32.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-03-29T08:18:32.055 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:18:32.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:32.190 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:18:32.190 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:18:32.190 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.1 up 2024-03-29T08:18:32.190 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.1 master ceph-brx 2024-03-29T08:18:32.191 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:18:32.229 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-03-29T08:18:32.229 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:18:32.229 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-03-29T08:18:32.284 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-03-29T08:18:32.284 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:18:32.284 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-03-29T08:18:32.335 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:18:32.398 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:18:32.398 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:18:32.457 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:18:32.459 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:18:32.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:32.460 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:18:32.515 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:18:32.516 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2024-03-29T08:18:32.516 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-03-29T08:18:32.561 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:18:32.582 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:18:32.583 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:18:32.646 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:18:32.648 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:18:32.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:32.649 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:18:32.659 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi046.stderr:2024-03-29T08:18:32.656+0000 7fe5d88d1580 -1 init, newargv = 0x55f65e680ee0 newargc=15 2024-03-29T08:18:32.659 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi046.stderr:ceph-fuse[33514]: starting ceph client 2024-03-29T08:18:32.667 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:18:32.667 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi046.stderr:ceph-fuse[33514]: starting fuse 2024-03-29T08:18:33.770 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:18:33.794 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:18:33.794 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:18:33.859 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:18:33.861 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:18:33.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:33.862 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:18:33.912 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:18:33.912 INFO:teuthology.orchestra.run.smithi046.stdout:44 2024-03-29T08:18:33.912 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2024-03-29T08:18:33.913 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:18:33.913 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-03-29T08:18:33.913 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-03-29T08:18:33.913 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:18:33.913 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-03-29T08:18:33.913 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:18:33.913 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-03-29T08:18:33.913 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-03-29T08:18:33.913 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-03-29T08:18:33.913 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-03-29T08:18:33.913 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:18:33.913 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-03-29T08:18:33.913 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-03-29T08:18:33.913 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-03-29T08:18:33.913 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-03-29T08:18:33.914 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:18:33.914 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-03-29T08:18:33.914 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:18:33.914 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:18:33.914 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:18:33.914 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-03-29T08:18:33.914 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-03-29T08:18:33.914 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-03-29T08:18:33.914 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:18:33.914 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:18:33.915 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.2")) 2024-03-29T08:18:33.915 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:18:34.044 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.2.33514.asok 2024-03-29T08:18:34.229 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.33514.asok 2024-03-29T08:18:34.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:34.229 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.33514.asok status 2024-03-29T08:18:34.414 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:18:34.414 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-03-29T08:18:34.415 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:18:34.415 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:18:34.415 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "2", 2024-03-29T08:18:34.415 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-03-29T08:18:34.415 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-03-29T08:18:34.415 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "33514", 2024-03-29T08:18:34.415 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-03-29T08:18:34.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:18:34.415 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-03-29T08:18:34.415 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-03-29T08:18:34.415 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4638, 2024-03-29T08:18:34.415 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-03-29T08:18:34.415 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-03-29T08:18:34.415 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-03-29T08:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 4638 2024-03-29T08:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.2:0", 2024-03-29T08:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 103370198 2024-03-29T08:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.2:0", 2024-03-29T08:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 103370198 2024-03-29T08:18:34.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:18:34.417 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.4638 192.168.0.2:0/103370198", 2024-03-29T08:18:34.417 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.2:0/103370198", 2024-03-29T08:18:34.417 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-03-29T08:18:34.417 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 8, 2024-03-29T08:18:34.417 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 22, 2024-03-29T08:18:34.417 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:18:34.417 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-03-29T08:18:34.417 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-03-29T08:18:34.417 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:18:34.430 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:18:34.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.568+0000 7f1bb0c00640 1 Processor -- start 2024-03-29T08:18:34.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.568+0000 7f1bb0c00640 1 -- start start 2024-03-29T08:18:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.568+0000 7f1bb0c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1bac144210 0x7f1bac0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.568+0000 7f1bb0c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1bac0ceb20 0x7f1bac06c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.568+0000 7f1bb0c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bac0cb510 con 0x7f1bac15ada0 2024-03-29T08:18:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.568+0000 7f1bb0c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bac070580 con 0x7f1bac144210 2024-03-29T08:18:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.568+0000 7f1bb0c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bac070860 con 0x7f1bac0ceb20 2024-03-29T08:18:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.568+0000 7f1ba9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1bac144210 0x7f1bac0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.568+0000 7f1ba9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1bac144210 0x7f1bac0cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51284/0 (socket says 172.21.15.46:51284) 2024-03-29T08:18:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.568+0000 7f1ba9e00640 1 -- 172.21.15.46:0/166227894 learned_addr learned my addr 172.21.15.46:0/166227894 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.572+0000 7f1baa800640 1 --2- 172.21.15.46:0/166227894 >> v2:172.21.15.73:3300/0 conn(0x7f1bac0ceb20 0x7f1bac06c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.572+0000 7f1ba9e00640 1 -- 172.21.15.46:0/166227894 >> v2:172.21.15.73:3300/0 conn(0x7f1bac0ceb20 msgr2=0x7f1bac06c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.572+0000 7f1ba9e00640 1 --2- 172.21.15.46:0/166227894 >> v2:172.21.15.73:3300/0 conn(0x7f1bac0ceb20 0x7f1bac06c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.572+0000 7f1ba9e00640 1 -- 172.21.15.46:0/166227894 >> v1:172.21.15.46:6789/0 conn(0x7f1bac15ada0 legacy=0x7f1bac0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.572+0000 7f1ba9e00640 1 -- 172.21.15.46:0/166227894 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bac070b40 con 0x7f1bac144210 2024-03-29T08:18:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.572+0000 7f1ba9400640 1 -- 172.21.15.46:0/166227894 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459537877 0 0) 0x7f1bac0cb510 con 0x7f1bac15ada0 2024-03-29T08:18:34.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.572+0000 7f1ba9e00640 1 --2- 172.21.15.46:0/166227894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac144210 0x7f1bac0cb8b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0cf1a0 tx=0x7f1b9c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c111286b981d27 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:34.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.572+0000 7f1ba9400640 1 -- 172.21.15.46:0/166227894 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0a7070 con 0x7f1bac144210 2024-03-29T08:18:34.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.572+0000 7f1ba9400640 1 -- 172.21.15.46:0/166227894 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9c0ab430 con 0x7f1bac144210 2024-03-29T08:18:34.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.572+0000 7f1ba9400640 1 -- 172.21.15.46:0/166227894 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c0ab7c0 con 0x7f1bac144210 2024-03-29T08:18:34.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.572+0000 7f1bb0c00640 1 -- 172.21.15.46:0/166227894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac144210 msgr2=0x7f1bac0cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:34.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.572+0000 7f1bb0c00640 1 --2- 172.21.15.46:0/166227894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac144210 0x7f1bac0cb8b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1bac0cf1a0 tx=0x7f1b9c01c520 comp rx=0 tx=0).stop 2024-03-29T08:18:34.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.572+0000 7f1bb0c00640 1 -- 172.21.15.46:0/166227894 shutdown_connections 2024-03-29T08:18:34.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.572+0000 7f1bb0c00640 1 --2- 172.21.15.46:0/166227894 >> v2:172.21.15.73:3300/0 conn(0x7f1bac0ceb20 0x7f1bac06c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1bb0c00640 1 --2- 172.21.15.46:0/166227894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac144210 0x7f1bac0cb8b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1bb0c00640 1 -- 172.21.15.46:0/166227894 >> 172.21.15.46:0/166227894 conn(0x7f1bac066110 msgr2=0x7f1bac069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1bb0c00640 1 -- 172.21.15.46:0/166227894 shutdown_connections 2024-03-29T08:18:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1bb0c00640 1 -- 172.21.15.46:0/166227894 wait complete. 2024-03-29T08:18:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1bb0c00640 1 Processor -- start 2024-03-29T08:18:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1bb0c00640 1 -- start start 2024-03-29T08:18:34.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1bb0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bac0ceb20 0x7f1bac1367a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1bb0c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1bac144210 0x7f1bac13fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1bb0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac15ada0 0x7f1bac155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1bb0c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac152a70 con 0x7f1bac15ada0 2024-03-29T08:18:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1bb0c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac140540 con 0x7f1bac0ceb20 2024-03-29T08:18:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1bb0c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bac140820 con 0x7f1bac144210 2024-03-29T08:18:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1baa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac15ada0 0x7f1bac155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1baa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac15ada0 0x7f1bac155460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51300/0 (socket says 172.21.15.46:51300) 2024-03-29T08:18:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1baa800640 1 -- 172.21.15.46:0/3967730756 learned_addr learned my addr 172.21.15.46:0/3967730756 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1ba1400640 1 --2- 172.21.15.46:0/3967730756 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1bac144210 0x7f1bac13fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:34.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1ba9e00640 1 --2- 172.21.15.46:0/3967730756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bac0ceb20 0x7f1bac1367a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1baa800640 1 -- 172.21.15.46:0/3967730756 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1bac144210 msgr2=0x7f1bac13fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1baa800640 1 --2- 172.21.15.46:0/3967730756 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1bac144210 0x7f1bac13fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1baa800640 1 -- 172.21.15.46:0/3967730756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bac0ceb20 msgr2=0x7f1bac1367a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1baa800640 1 --2- 172.21.15.46:0/3967730756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bac0ceb20 0x7f1bac1367a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:34.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1baa800640 1 -- 172.21.15.46:0/3967730756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c0094d0 con 0x7f1bac15ada0 2024-03-29T08:18:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.576+0000 7f1baa800640 1 --2- 172.21.15.46:0/3967730756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac15ada0 0x7f1bac155460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4007e30 tx=0x7f1ba400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.580+0000 7f1ba2a00640 1 -- 172.21.15.46:0/3967730756 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4014620 con 0x7f1bac15ada0 2024-03-29T08:18:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.580+0000 7f1ba2a00640 1 -- 172.21.15.46:0/3967730756 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba400ee90 con 0x7f1bac15ada0 2024-03-29T08:18:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.580+0000 7f1bb0c00640 1 -- 172.21.15.46:0/3967730756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bac140b00 con 0x7f1bac15ada0 2024-03-29T08:18:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.580+0000 7f1ba2a00640 1 -- 172.21.15.46:0/3967730756 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba40149c0 con 0x7f1bac15ada0 2024-03-29T08:18:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.580+0000 7f1bb0c00640 1 -- 172.21.15.46:0/3967730756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bac136ce0 con 0x7f1bac15ada0 2024-03-29T08:18:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.580+0000 7f1bb0c00640 1 -- 172.21.15.46:0/3967730756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bac0ce280 con 0x7f1bac15ada0 2024-03-29T08:18:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.580+0000 7f1ba2a00640 1 -- 172.21.15.46:0/3967730756 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1ba4014b20 con 0x7f1bac15ada0 2024-03-29T08:18:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.580+0000 7f1ba2a00640 1 --2- 172.21.15.46:0/3967730756 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b88066690 0x7f1b88068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.580+0000 7f1ba2a00640 1 -- 172.21.15.46:0/3967730756 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ba4084a10 con 0x7f1bac15ada0 2024-03-29T08:18:34.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.580+0000 7f1ba9e00640 1 --2- 172.21.15.46:0/3967730756 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b88066690 0x7f1b88068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:34.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.584+0000 7f1ba9e00640 1 --2- 172.21.15.46:0/3967730756 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b88066690 0x7f1b88068b50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0af000 tx=0x7f1b9c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:34.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.584+0000 7f1ba2a00640 1 -- 172.21.15.46:0/3967730756 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1ba4051e20 con 0x7f1bac15ada0 2024-03-29T08:18:34.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.784+0000 7f1bb0c00640 1 -- 172.21.15.46:0/3967730756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1bac066c90 con 0x7f1bac15ada0 2024-03-29T08:18:34.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.784+0000 7f1ba2a00640 1 -- 172.21.15.46:0/3967730756 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1ba4055440 con 0x7f1bac15ada0 2024-03-29T08:18:34.785 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:18:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.784+0000 7f1bb0c00640 1 -- 172.21.15.46:0/3967730756 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b88066690 msgr2=0x7f1b88068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.784+0000 7f1bb0c00640 1 --2- 172.21.15.46:0/3967730756 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b88066690 0x7f1b88068b50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0af000 tx=0x7f1b9c008040 comp rx=0 tx=0).stop 2024-03-29T08:18:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.784+0000 7f1bb0c00640 1 -- 172.21.15.46:0/3967730756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac15ada0 msgr2=0x7f1bac155460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:34.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.784+0000 7f1bb0c00640 1 --2- 172.21.15.46:0/3967730756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac15ada0 0x7f1bac155460 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4007e30 tx=0x7f1ba400e4b0 comp rx=0 tx=0).stop 2024-03-29T08:18:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.784+0000 7f1bb0c00640 1 -- 172.21.15.46:0/3967730756 shutdown_connections 2024-03-29T08:18:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.784+0000 7f1bb0c00640 1 --2- 172.21.15.46:0/3967730756 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b88066690 0x7f1b88068b50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.784+0000 7f1bb0c00640 1 --2- 172.21.15.46:0/3967730756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1bac15ada0 0x7f1bac155460 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.784+0000 7f1bb0c00640 1 --2- 172.21.15.46:0/3967730756 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1bac144210 0x7f1bac13fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.784+0000 7f1bb0c00640 1 --2- 172.21.15.46:0/3967730756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1bac0ceb20 0x7f1bac1367a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.784+0000 7f1bb0c00640 1 -- 172.21.15.46:0/3967730756 >> 172.21.15.46:0/3967730756 conn(0x7f1bac066110 msgr2=0x7f1bac13d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.784+0000 7f1bb0c00640 1 -- 172.21.15.46:0/3967730756 shutdown_connections 2024-03-29T08:18:34.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:34.784+0000 7f1bb0c00640 1 -- 172.21.15.46:0/3967730756 wait complete. 2024-03-29T08:18:34.806 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:18:34.806 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:18:34.807 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-03-29T08:18:34.807 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-03-29T08:18:34.807 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-03-29T08:18:34.807 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:18:34.807 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:18:34.807 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:18:34.807 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-03-29T08:18:34.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:34.807 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-03-29T08:18:34.817 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:18:34.817 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:18:34.817 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:18:34.817 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:34.817 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-03-29T08:18:34.817 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:34.817 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:18:34.817 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:34.818 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-03-29T08:18:34.818 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:18:34.818 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:34.818 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-03-29T08:18:34.818 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:18:34.818 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:34.818 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:18:34.818 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:34.818 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-29T08:18:34.818 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42419sec preferred_lft 42419sec 2024-03-29T08:18:34.818 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-03-29T08:18:34.818 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:34.818 INFO:teuthology.orchestra.run.smithi073.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:18:34.819 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 02:42:b4:ff:49:f0 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:34.819 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:18:34.819 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:34.819 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-29T08:18:34.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:34.820 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-03-29T08:18:34.820 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-03-29T08:18:34.820 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add name ceph-brx type bridge 2024-03-29T08:18:34.820 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip addr flush dev ceph-brx 2024-03-29T08:18:34.820 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set ceph-brx up 2024-03-29T08:18:34.820 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-29T08:18:34.820 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-03-29T08:18:34.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:34.935 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:18:34.954 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-03-29T08:18:34.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:34.957 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-03-29T08:18:35.037 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-03-29T08:18:35.037 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:18:35.037 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-29T08:18:35.038 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T08:18:35.038 INFO:teuthology.orchestra.run.smithi073.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-29T08:18:35.038 INFO:teuthology.orchestra.run.smithi073.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:18:35.038 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T08:18:35.038 INFO:teuthology.orchestra.run.smithi073.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-29T08:18:35.038 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-29T08:18:35.038 INFO:teuthology.orchestra.run.smithi073.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-29T08:18:35.038 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:18:35.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:35.038 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-03-29T08:18:35.039 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-03-29T08:18:35.039 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T08:18:35.039 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T08:18:35.039 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T08:18:35.039 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-03-29T08:18:35.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:35.134 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-03-29T08:18:35.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:35.144 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list-id 2024-03-29T08:18:35.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:35.200 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-03-29T08:18:35.200 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-03-29T08:18:35.200 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T08:18:35.200 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-03-29T08:18:35.200 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-03-29T08:18:35.285 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-03-29T08:18:35.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:35.285 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-03-29T08:18:35.285 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-03-29T08:18:35.285 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-03-29T08:18:35.285 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-29T08:18:35.285 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-03-29T08:18:35.285 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-03-29T08:18:35.285 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-03-29T08:18:35.285 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-03-29T08:18:35.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:35.451 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-03-29T08:18:35.451 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-03-29T08:18:35.451 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-03-29T08:18:35.451 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set dev brx.0 master ceph-brx 2024-03-29T08:18:35.452 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-03-29T08:18:35.492 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-03-29T08:18:35.492 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:18:35.492 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-03-29T08:18:35.543 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-03-29T08:18:35.543 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:18:35.543 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-03-29T08:18:35.594 DEBUG:teuthology.orchestra.run.smithi073:> sudo modprobe fuse 2024-03-29T08:18:35.655 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:18:35.656 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:18:35.717 INFO:teuthology.orchestra.run.smithi073.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:18:35.719 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:18:35.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:35.719 DEBUG:teuthology.orchestra.run.smithi073:> ls /sys/fs/fuse/connections 2024-03-29T08:18:35.771 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:18:35.771 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-03-29T08:18:35.814 DEBUG:teuthology.orchestra.run.smithi073:> sudo modprobe fuse 2024-03-29T08:18:35.834 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:18:35.834 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:18:35.893 INFO:teuthology.orchestra.run.smithi073.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:18:35.895 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:18:35.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:35.896 DEBUG:teuthology.orchestra.run.smithi073:> ls /sys/fs/fuse/connections 2024-03-29T08:18:35.907 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi073.stderr:2024-03-29T08:18:35.902+0000 7fdfefd61580 -1 init, newargv = 0x562d4688ffa0 newargc=15 2024-03-29T08:18:35.907 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi073.stderr:ceph-fuse[32976]: starting ceph client 2024-03-29T08:18:35.915 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi073.stderr:ceph-fuse[32976]: starting fuse 2024-03-29T08:18:35.916 INFO:teuthology.orchestra.run.smithi073.stdout:43 2024-03-29T08:18:35.916 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-29T08:18:35.916 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:18:35.916 DEBUG:teuthology.orchestra.run.smithi073:> import glob 2024-03-29T08:18:35.916 DEBUG:teuthology.orchestra.run.smithi073:> import re 2024-03-29T08:18:35.916 DEBUG:teuthology.orchestra.run.smithi073:> import os 2024-03-29T08:18:35.916 DEBUG:teuthology.orchestra.run.smithi073:> import subprocess 2024-03-29T08:18:35.916 DEBUG:teuthology.orchestra.run.smithi073:> 2024-03-29T08:18:35.917 DEBUG:teuthology.orchestra.run.smithi073:> def _find_admin_socket(client_name): 2024-03-29T08:18:35.917 DEBUG:teuthology.orchestra.run.smithi073:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-03-29T08:18:35.917 DEBUG:teuthology.orchestra.run.smithi073:> files = glob.glob(asok_path) 2024-03-29T08:18:35.917 DEBUG:teuthology.orchestra.run.smithi073:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-03-29T08:18:35.917 DEBUG:teuthology.orchestra.run.smithi073:> 2024-03-29T08:18:35.917 DEBUG:teuthology.orchestra.run.smithi073:> # Given a non-glob path, it better be there 2024-03-29T08:18:35.917 DEBUG:teuthology.orchestra.run.smithi073:> if "*" not in asok_path: 2024-03-29T08:18:35.917 DEBUG:teuthology.orchestra.run.smithi073:> assert(len(files) == 1) 2024-03-29T08:18:35.917 DEBUG:teuthology.orchestra.run.smithi073:> return files[0] 2024-03-29T08:18:35.917 DEBUG:teuthology.orchestra.run.smithi073:> 2024-03-29T08:18:35.917 DEBUG:teuthology.orchestra.run.smithi073:> for f in files: 2024-03-29T08:18:35.917 DEBUG:teuthology.orchestra.run.smithi073:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:18:35.917 DEBUG:teuthology.orchestra.run.smithi073:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:18:35.917 DEBUG:teuthology.orchestra.run.smithi073:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:18:35.917 DEBUG:teuthology.orchestra.run.smithi073:> contents = proc_f.read() 2024-03-29T08:18:35.918 DEBUG:teuthology.orchestra.run.smithi073:> if mountpoint in contents: 2024-03-29T08:18:35.918 DEBUG:teuthology.orchestra.run.smithi073:> return f 2024-03-29T08:18:35.918 DEBUG:teuthology.orchestra.run.smithi073:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:18:35.918 DEBUG:teuthology.orchestra.run.smithi073:> 2024-03-29T08:18:35.918 DEBUG:teuthology.orchestra.run.smithi073:> print(_find_admin_socket("client.1")) 2024-03-29T08:18:35.918 DEBUG:teuthology.orchestra.run.smithi073:> ' 2024-03-29T08:18:36.036 INFO:teuthology.orchestra.run.smithi073.stdout:/var/run/ceph/ceph-client.1.32976.asok 2024-03-29T08:18:36.221 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.32976.asok 2024-03-29T08:18:36.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:36.221 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.32976.asok status 2024-03-29T08:18:36.395 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-03-29T08:18:36.395 INFO:teuthology.orchestra.run.smithi073.stdout: "metadata": { 2024-03-29T08:18:36.395 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:18:36.395 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:18:36.395 INFO:teuthology.orchestra.run.smithi073.stdout: "entity_id": "1", 2024-03-29T08:18:36.396 INFO:teuthology.orchestra.run.smithi073.stdout: "hostname": "smithi073", 2024-03-29T08:18:36.396 INFO:teuthology.orchestra.run.smithi073.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-03-29T08:18:36.396 INFO:teuthology.orchestra.run.smithi073.stdout: "pid": "32976", 2024-03-29T08:18:36.396 INFO:teuthology.orchestra.run.smithi073.stdout: "root": "/" 2024-03-29T08:18:36.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-29T08:18:36.396 INFO:teuthology.orchestra.run.smithi073.stdout: "dentry_count": 0, 2024-03-29T08:18:36.396 INFO:teuthology.orchestra.run.smithi073.stdout: "dentry_pinned_count": 0, 2024-03-29T08:18:36.396 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 4714, 2024-03-29T08:18:36.396 INFO:teuthology.orchestra.run.smithi073.stdout: "inst": { 2024-03-29T08:18:36.396 INFO:teuthology.orchestra.run.smithi073.stdout: "name": { 2024-03-29T08:18:36.396 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "client", 2024-03-29T08:18:36.396 INFO:teuthology.orchestra.run.smithi073.stdout: "num": 4714 2024-03-29T08:18:36.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-29T08:18:36.396 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": { 2024-03-29T08:18:36.397 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-03-29T08:18:36.397 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:18:36.397 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 792340345 2024-03-29T08:18:36.397 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-29T08:18:36.397 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-29T08:18:36.397 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": { 2024-03-29T08:18:36.397 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-03-29T08:18:36.397 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:18:36.397 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 792340345 2024-03-29T08:18:36.397 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-29T08:18:36.397 INFO:teuthology.orchestra.run.smithi073.stdout: "inst_str": "client.4714 192.168.0.1:0/792340345", 2024-03-29T08:18:36.397 INFO:teuthology.orchestra.run.smithi073.stdout: "addr_str": "192.168.0.1:0/792340345", 2024-03-29T08:18:36.397 INFO:teuthology.orchestra.run.smithi073.stdout: "inode_count": 1, 2024-03-29T08:18:36.397 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_epoch": 8, 2024-03-29T08:18:36.397 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_epoch": 22, 2024-03-29T08:18:36.398 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:18:36.398 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklisted": false, 2024-03-29T08:18:36.398 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs" 2024-03-29T08:18:36.398 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-03-29T08:18:36.409 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:18:36.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.560+0000 7fc136600640 1 Processor -- start 2024-03-29T08:18:36.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.560+0000 7fc136600640 1 -- start start 2024-03-29T08:18:36.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.560+0000 7fc136600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc13015cd90 0x7fc13006cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:36.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.560+0000 7fc136600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc13015d940 0x7fc1301597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:36.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.560+0000 7fc136600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc13006d130 con 0x7fc1301593d0 2024-03-29T08:18:36.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.560+0000 7fc136600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc13015b8f0 con 0x7fc13015cd90 2024-03-29T08:18:36.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.560+0000 7fc136600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13015ad50 con 0x7fc13015d940 2024-03-29T08:18:36.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.560+0000 7fc126a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc13015cd90 0x7fc13006cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:36.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.560+0000 7fc126a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc13015cd90 0x7fc13006cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43196/0 (socket says 172.21.15.46:43196) 2024-03-29T08:18:36.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.560+0000 7fc126a00640 1 -- 172.21.15.46:0/80730860 learned_addr learned my addr 172.21.15.46:0/80730860 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:36.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.564+0000 7fc12fe00640 1 --2- 172.21.15.46:0/80730860 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc13015d940 0x7fc1301597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:36.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.564+0000 7fc126a00640 1 -- 172.21.15.46:0/80730860 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc13015d940 msgr2=0x7fc1301597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:36.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.564+0000 7fc126a00640 1 --2- 172.21.15.46:0/80730860 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc13015d940 0x7fc1301597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:36.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.564+0000 7fc126a00640 1 -- 172.21.15.46:0/80730860 >> v1:172.21.15.73:6789/0 conn(0x7fc1301593d0 legacy=0x7fc13015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:36.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.564+0000 7fc126a00640 1 -- 172.21.15.46:0/80730860 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13015afc0 con 0x7fc13015cd90 2024-03-29T08:18:36.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.564+0000 7fc12fe00640 1 --2- 172.21.15.46:0/80730860 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc13015d940 0x7fc1301597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:36.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.564+0000 7fc126a00640 1 --2- 172.21.15.46:0/80730860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc13015cd90 0x7fc13006cbf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc11800bb10 tx=0x7fc11801ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d6c385b4228247 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:36.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.564+0000 7fc12ea00640 1 -- 172.21.15.46:0/80730860 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1180a7c80 con 0x7fc13015cd90 2024-03-29T08:18:36.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.564+0000 7fc12ea00640 1 -- 172.21.15.46:0/80730860 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1180a7de0 con 0x7fc13015cd90 2024-03-29T08:18:36.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.564+0000 7fc12ea00640 1 -- 172.21.15.46:0/80730860 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1180af660 con 0x7fc13015cd90 2024-03-29T08:18:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.564+0000 7fc136600640 1 -- 172.21.15.46:0/80730860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc13015cd90 msgr2=0x7fc13006cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.564+0000 7fc136600640 1 --2- 172.21.15.46:0/80730860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc13015cd90 0x7fc13006cbf0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc11800bb10 tx=0x7fc11801ebf0 comp rx=0 tx=0).stop 2024-03-29T08:18:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc136600640 1 -- 172.21.15.46:0/80730860 shutdown_connections 2024-03-29T08:18:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc136600640 1 --2- 172.21.15.46:0/80730860 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc13015d940 0x7fc1301597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc136600640 1 --2- 172.21.15.46:0/80730860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc13015cd90 0x7fc13006cbf0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc136600640 1 -- 172.21.15.46:0/80730860 >> 172.21.15.46:0/80730860 conn(0x7fc130066110 msgr2=0x7fc130069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc136600640 1 -- 172.21.15.46:0/80730860 shutdown_connections 2024-03-29T08:18:36.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc136600640 1 -- 172.21.15.46:0/80730860 wait complete. 2024-03-29T08:18:36.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc136600640 1 Processor -- start 2024-03-29T08:18:36.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc136600640 1 -- start start 2024-03-29T08:18:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc136600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1301593d0 0x7fc130143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc136600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc13015cd90 0x7fc130155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc136600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc13015d940 0x7fc130156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc136600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1300cd430 con 0x7fc13015cd90 2024-03-29T08:18:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc136600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1300cc8b0 con 0x7fc1301593d0 2024-03-29T08:18:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc136600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1300ccb90 con 0x7fc13015d940 2024-03-29T08:18:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc126a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc13015cd90 0x7fc130155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc126a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc13015cd90 0x7fc130155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43204/0 (socket says 172.21.15.46:43204) 2024-03-29T08:18:36.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc126a00640 1 -- 172.21.15.46:0/1271895322 learned_addr learned my addr 172.21.15.46:0/1271895322 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:36.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc12f400640 1 --2- 172.21.15.46:0/1271895322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1301593d0 0x7fc130143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:36.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc12fe00640 1 --2- 172.21.15.46:0/1271895322 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc13015d940 0x7fc130156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:36.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc126a00640 1 -- 172.21.15.46:0/1271895322 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc13015d940 msgr2=0x7fc130156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:36.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc126a00640 1 --2- 172.21.15.46:0/1271895322 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc13015d940 0x7fc130156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:36.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc126a00640 1 -- 172.21.15.46:0/1271895322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1301593d0 msgr2=0x7fc130143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:36.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc126a00640 1 --2- 172.21.15.46:0/1271895322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1301593d0 0x7fc130143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:36.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc126a00640 1 -- 172.21.15.46:0/1271895322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11800b770 con 0x7fc13015cd90 2024-03-29T08:18:36.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.568+0000 7fc126a00640 1 --2- 172.21.15.46:0/1271895322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc13015cd90 0x7fc130155a70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc11800b2c0 tx=0x7fc11801d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:36.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.572+0000 7fc127e00640 1 -- 172.21.15.46:0/1271895322 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1180afe30 con 0x7fc13015cd90 2024-03-29T08:18:36.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.572+0000 7fc127e00640 1 -- 172.21.15.46:0/1271895322 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1180ba480 con 0x7fc13015cd90 2024-03-29T08:18:36.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.572+0000 7fc127e00640 1 -- 172.21.15.46:0/1271895322 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1180ba830 con 0x7fc13015cd90 2024-03-29T08:18:36.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.572+0000 7fc136600640 1 -- 172.21.15.46:0/1271895322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1300cce70 con 0x7fc13015cd90 2024-03-29T08:18:36.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.572+0000 7fc136600640 1 -- 172.21.15.46:0/1271895322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1301442e0 con 0x7fc13015cd90 2024-03-29T08:18:36.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.572+0000 7fc136600640 1 -- 172.21.15.46:0/1271895322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc130066c90 con 0x7fc13015cd90 2024-03-29T08:18:36.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.572+0000 7fc127e00640 1 -- 172.21.15.46:0/1271895322 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc1180ba990 con 0x7fc13015cd90 2024-03-29T08:18:36.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.572+0000 7fc127e00640 1 --2- 172.21.15.46:0/1271895322 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc10c0666c0 0x7fc10c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:36.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.572+0000 7fc127e00640 1 -- 172.21.15.46:0/1271895322 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc11811f670 con 0x7fc13015cd90 2024-03-29T08:18:36.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.572+0000 7fc12f400640 1 --2- 172.21.15.46:0/1271895322 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc10c0666c0 0x7fc10c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:36.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.572+0000 7fc12f400640 1 --2- 172.21.15.46:0/1271895322 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc10c0666c0 0x7fc10c068b80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc1200098e0 tx=0x7fc120009060 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:36.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.576+0000 7fc127e00640 1 -- 172.21.15.46:0/1271895322 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc1180ecb10 con 0x7fc13015cd90 2024-03-29T08:18:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.772+0000 7fc136600640 1 -- 172.21.15.46:0/1271895322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc130066c90 con 0x7fc13015cd90 2024-03-29T08:18:36.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.772+0000 7fc127e00640 1 -- 172.21.15.46:0/1271895322 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc1180b4070 con 0x7fc13015cd90 2024-03-29T08:18:36.775 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:18:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.776+0000 7fc136600640 1 -- 172.21.15.46:0/1271895322 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc10c0666c0 msgr2=0x7fc10c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.776+0000 7fc136600640 1 --2- 172.21.15.46:0/1271895322 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc10c0666c0 0x7fc10c068b80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc1200098e0 tx=0x7fc120009060 comp rx=0 tx=0).stop 2024-03-29T08:18:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.776+0000 7fc136600640 1 -- 172.21.15.46:0/1271895322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc13015cd90 msgr2=0x7fc130155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.776+0000 7fc136600640 1 --2- 172.21.15.46:0/1271895322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc13015cd90 0x7fc130155a70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc11800b2c0 tx=0x7fc11801d430 comp rx=0 tx=0).stop 2024-03-29T08:18:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.776+0000 7fc136600640 1 -- 172.21.15.46:0/1271895322 shutdown_connections 2024-03-29T08:18:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.776+0000 7fc136600640 1 --2- 172.21.15.46:0/1271895322 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc10c0666c0 0x7fc10c068b80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.776+0000 7fc136600640 1 --2- 172.21.15.46:0/1271895322 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc13015d940 0x7fc130156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.776+0000 7fc136600640 1 --2- 172.21.15.46:0/1271895322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc13015cd90 0x7fc130155a70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.776+0000 7fc136600640 1 --2- 172.21.15.46:0/1271895322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1301593d0 0x7fc130143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.776+0000 7fc136600640 1 -- 172.21.15.46:0/1271895322 >> 172.21.15.46:0/1271895322 conn(0x7fc130066110 msgr2=0x7fc13006c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.776+0000 7fc136600640 1 -- 172.21.15.46:0/1271895322 shutdown_connections 2024-03-29T08:18:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:36.776+0000 7fc136600640 1 -- 172.21.15.46:0/1271895322 wait complete. 2024-03-29T08:18:36.796 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:18:36.796 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:18:36.796 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-03-29T08:18:36.796 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-03-29T08:18:36.796 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-03-29T08:18:36.796 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:18:36.796 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:18:36.797 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:18:36.797 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-03-29T08:18:36.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:36.797 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-03-29T08:18:36.806 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:18:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:18:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:18:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-03-29T08:18:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:36.807 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:18:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-03-29T08:18:36.807 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:18:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-03-29T08:18:36.807 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:18:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:36.807 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:18:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-29T08:18:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42417sec preferred_lft 42417sec 2024-03-29T08:18:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-03-29T08:18:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:36.808 INFO:teuthology.orchestra.run.smithi073.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:18:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 02:42:b4:ff:49:f0 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:18:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:36.808 INFO:teuthology.orchestra.run.smithi073.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:18:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T08:18:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T08:18:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T08:18:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:36.809 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:18:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T08:18:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2024-03-29T08:18:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:36.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:36.809 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:18:36.865 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-03-29T08:18:36.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:36.868 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-03-29T08:18:36.923 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-03-29T08:18:36.923 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:18:36.923 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-29T08:18:36.923 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-29T08:18:36.923 INFO:teuthology.orchestra.run.smithi073.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-29T08:18:36.923 INFO:teuthology.orchestra.run.smithi073.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:18:36.924 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-29T08:18:36.924 INFO:teuthology.orchestra.run.smithi073.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-29T08:18:36.924 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-29T08:18:36.924 INFO:teuthology.orchestra.run.smithi073.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-29T08:18:36.924 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:18:36.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:36.924 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-03-29T08:18:36.924 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-03-29T08:18:36.924 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-29T08:18:36.924 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-29T08:18:36.925 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-29T08:18:36.925 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-03-29T08:18:37.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:37.016 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-03-29T08:18:37.067 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-29T08:18:37.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:37.068 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list-id 2024-03-29T08:18:37.119 INFO:teuthology.orchestra.run.smithi073.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-03-29T08:18:37.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:37.120 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-03-29T08:18:37.120 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-03-29T08:18:37.121 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T08:18:37.121 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-03-29T08:18:37.121 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-03-29T08:18:37.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:37.205 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-29T08:18:37.267 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:18:37.267 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:18:37.267 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:18:37.267 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:37.267 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-03-29T08:18:37.267 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:37.267 INFO:teuthology.orchestra.run.smithi073.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:18:37.267 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ea:3e:05:97:53:b4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T08:18:37.267 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T08:18:37.267 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:37.267 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::e83e:5ff:fe97:53b4/64 scope link 2024-03-29T08:18:37.267 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:37.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:37.269 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-03-29T08:18:37.330 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:18:37.331 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:18:37.331 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:18:37.331 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:37.331 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-03-29T08:18:37.331 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:37.331 INFO:teuthology.orchestra.run.smithi073.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:18:37.331 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether ea:3e:05:97:53:b4 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-03-29T08:18:37.331 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-03-29T08:18:37.331 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:37.331 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::e83e:5ff:fe97:53b4/64 scope link 2024-03-29T08:18:37.331 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:18:37.332 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-03-29T08:18:37.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:37.333 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-03-29T08:18:37.333 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-03-29T08:18:37.333 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-03-29T08:18:37.333 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-03-29T08:18:37.333 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-03-29T08:18:37.333 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-03-29T08:18:37.333 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-03-29T08:18:37.333 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-03-29T08:18:37.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:37.474 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2024-03-29T08:18:37.474 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-03-29T08:18:37.474 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.1 up 2024-03-29T08:18:37.474 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set dev brx.1 master ceph-brx 2024-03-29T08:18:37.475 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-03-29T08:18:37.511 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-03-29T08:18:37.511 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:18:37.511 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-03-29T08:18:37.563 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-03-29T08:18:37.563 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:18:37.564 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-03-29T08:18:37.614 DEBUG:teuthology.orchestra.run.smithi073:> sudo modprobe fuse 2024-03-29T08:18:37.675 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:18:37.676 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:18:37.737 INFO:teuthology.orchestra.run.smithi073.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:18:37.738 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:18:37.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:37.739 DEBUG:teuthology.orchestra.run.smithi073:> ls /sys/fs/fuse/connections 2024-03-29T08:18:37.787 INFO:teuthology.orchestra.run.smithi073.stdout:43 2024-03-29T08:18:37.787 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2024-03-29T08:18:37.787 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-03-29T08:18:37.834 DEBUG:teuthology.orchestra.run.smithi073:> sudo modprobe fuse 2024-03-29T08:18:37.854 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:18:37.854 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:18:37.913 INFO:teuthology.orchestra.run.smithi073.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:18:37.915 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:18:37.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:37.915 DEBUG:teuthology.orchestra.run.smithi073:> ls /sys/fs/fuse/connections 2024-03-29T08:18:37.928 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi073.stderr:2024-03-29T08:18:37.925+0000 7f1a68848580 -1 init, newargv = 0x5591a5608fa0 newargc=15 2024-03-29T08:18:37.928 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi073.stderr:ceph-fuse[33084]: starting ceph client 2024-03-29T08:18:37.935 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi073.stderr:ceph-fuse[33084]: starting fuse 2024-03-29T08:18:37.936 INFO:teuthology.orchestra.run.smithi073.stdout:43 2024-03-29T08:18:37.936 INFO:teuthology.orchestra.run.smithi073.stdout:44 2024-03-29T08:18:37.937 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2024-03-29T08:18:37.937 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:18:37.937 DEBUG:teuthology.orchestra.run.smithi073:> import glob 2024-03-29T08:18:37.937 DEBUG:teuthology.orchestra.run.smithi073:> import re 2024-03-29T08:18:37.937 DEBUG:teuthology.orchestra.run.smithi073:> import os 2024-03-29T08:18:37.937 DEBUG:teuthology.orchestra.run.smithi073:> import subprocess 2024-03-29T08:18:37.937 DEBUG:teuthology.orchestra.run.smithi073:> 2024-03-29T08:18:37.937 DEBUG:teuthology.orchestra.run.smithi073:> def _find_admin_socket(client_name): 2024-03-29T08:18:37.937 DEBUG:teuthology.orchestra.run.smithi073:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-03-29T08:18:37.938 DEBUG:teuthology.orchestra.run.smithi073:> files = glob.glob(asok_path) 2024-03-29T08:18:37.938 DEBUG:teuthology.orchestra.run.smithi073:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-03-29T08:18:37.938 DEBUG:teuthology.orchestra.run.smithi073:> 2024-03-29T08:18:37.938 DEBUG:teuthology.orchestra.run.smithi073:> # Given a non-glob path, it better be there 2024-03-29T08:18:37.938 DEBUG:teuthology.orchestra.run.smithi073:> if "*" not in asok_path: 2024-03-29T08:18:37.938 DEBUG:teuthology.orchestra.run.smithi073:> assert(len(files) == 1) 2024-03-29T08:18:37.938 DEBUG:teuthology.orchestra.run.smithi073:> return files[0] 2024-03-29T08:18:37.938 DEBUG:teuthology.orchestra.run.smithi073:> 2024-03-29T08:18:37.938 DEBUG:teuthology.orchestra.run.smithi073:> for f in files: 2024-03-29T08:18:37.938 DEBUG:teuthology.orchestra.run.smithi073:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:18:37.938 DEBUG:teuthology.orchestra.run.smithi073:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:18:37.939 DEBUG:teuthology.orchestra.run.smithi073:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:18:37.939 DEBUG:teuthology.orchestra.run.smithi073:> contents = proc_f.read() 2024-03-29T08:18:37.939 DEBUG:teuthology.orchestra.run.smithi073:> if mountpoint in contents: 2024-03-29T08:18:37.939 DEBUG:teuthology.orchestra.run.smithi073:> return f 2024-03-29T08:18:37.939 DEBUG:teuthology.orchestra.run.smithi073:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:18:37.939 DEBUG:teuthology.orchestra.run.smithi073:> 2024-03-29T08:18:37.939 DEBUG:teuthology.orchestra.run.smithi073:> print(_find_admin_socket("client.3")) 2024-03-29T08:18:37.939 DEBUG:teuthology.orchestra.run.smithi073:> ' 2024-03-29T08:18:38.061 INFO:teuthology.orchestra.run.smithi073.stdout:/var/run/ceph/ceph-client.3.33084.asok 2024-03-29T08:18:38.245 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33084.asok 2024-03-29T08:18:38.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:38.246 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33084.asok status 2024-03-29T08:18:38.246 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:18:38.250 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:18:38.416 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-03-29T08:18:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: "metadata": { 2024-03-29T08:18:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:18:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:18:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: "entity_id": "3", 2024-03-29T08:18:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: "hostname": "smithi073", 2024-03-29T08:18:38.416 INFO:teuthology.orchestra.run.smithi073.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-03-29T08:18:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "pid": "33084", 2024-03-29T08:18:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "root": "/" 2024-03-29T08:18:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-29T08:18:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "dentry_count": 0, 2024-03-29T08:18:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "dentry_pinned_count": 0, 2024-03-29T08:18:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 4723, 2024-03-29T08:18:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "inst": { 2024-03-29T08:18:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "name": { 2024-03-29T08:18:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "client", 2024-03-29T08:18:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "num": 4723 2024-03-29T08:18:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-29T08:18:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": { 2024-03-29T08:18:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-03-29T08:18:38.417 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "192.168.0.2:0", 2024-03-29T08:18:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3954894846 2024-03-29T08:18:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-03-29T08:18:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-29T08:18:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": { 2024-03-29T08:18:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-03-29T08:18:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "192.168.0.2:0", 2024-03-29T08:18:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3954894846 2024-03-29T08:18:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-03-29T08:18:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "inst_str": "client.4723 192.168.0.2:0/3954894846", 2024-03-29T08:18:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "addr_str": "192.168.0.2:0/3954894846", 2024-03-29T08:18:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "inode_count": 1, 2024-03-29T08:18:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_epoch": 8, 2024-03-29T08:18:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_epoch": 22, 2024-03-29T08:18:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:18:38.418 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklisted": false, 2024-03-29T08:18:38.419 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs" 2024-03-29T08:18:38.419 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-03-29T08:18:38.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:38.431 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-03-29T08:18:38.431 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:18:38.442 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-03-29T08:18:38.442 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T08:18:38.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:38.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:18:38.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:38.508 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-03-29T08:18:38.508 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.2 2024-03-29T08:18:38.561 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-03-29T08:18:38.561 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-03-29T08:18:38.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:38.561 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-03-29T08:18:38.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:38.628 DEBUG:teuthology.orchestra.run.smithi073:> stat --file-system '--printf=%T 2024-03-29T08:18:38.628 DEBUG:teuthology.orchestra.run.smithi073:> ' -- /home/ubuntu/cephtest/mnt.1 2024-03-29T08:18:38.637 INFO:teuthology.orchestra.run.smithi073.stdout:fuseblk 2024-03-29T08:18:38.637 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-03-29T08:18:38.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:38.637 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-03-29T08:18:38.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:38.699 DEBUG:teuthology.orchestra.run.smithi073:> stat --file-system '--printf=%T 2024-03-29T08:18:38.699 DEBUG:teuthology.orchestra.run.smithi073:> ' -- /home/ubuntu/cephtest/mnt.3 2024-03-29T08:18:38.756 INFO:teuthology.orchestra.run.smithi073.stdout:fuseblk 2024-03-29T08:18:38.756 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-03-29T08:18:38.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:38.756 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-03-29T08:18:38.818 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-29T08:18:38.858 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:39.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.008+0000 7f146ae00640 1 Processor -- start 2024-03-29T08:18:39.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.008+0000 7f146ae00640 1 -- start start 2024-03-29T08:18:39.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.008+0000 7f146ae00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1464144210 0x7f14640cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.012+0000 7f146ae00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14640ceb20 0x7f146406c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.012+0000 7f146ae00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14640cb510 con 0x7f146415ada0 2024-03-29T08:18:39.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.012+0000 7f146ae00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1464070580 con 0x7f14640ceb20 2024-03-29T08:18:39.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.012+0000 7f146ae00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1464070860 con 0x7f1464144210 2024-03-29T08:18:39.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.012+0000 7f1468e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14640ceb20 0x7f146406c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.012+0000 7f1468e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14640ceb20 0x7f146406c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43206/0 (socket says 172.21.15.46:43206) 2024-03-29T08:18:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.012+0000 7f1468e00640 1 -- 172.21.15.46:0/572883402 learned_addr learned my addr 172.21.15.46:0/572883402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.012+0000 7f1463e00640 1 --2- 172.21.15.46:0/572883402 >> v2:172.21.15.73:3300/0 conn(0x7f1464144210 0x7f14640cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.012+0000 7f1468e00640 1 -- 172.21.15.46:0/572883402 >> v2:172.21.15.73:3300/0 conn(0x7f1464144210 msgr2=0x7f14640cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.012+0000 7f1468e00640 1 --2- 172.21.15.46:0/572883402 >> v2:172.21.15.73:3300/0 conn(0x7f1464144210 0x7f14640cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.012+0000 7f1468e00640 1 -- 172.21.15.46:0/572883402 >> v1:172.21.15.73:6789/0 conn(0x7f146415ada0 legacy=0x7f14640ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:39.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.012+0000 7f1468e00640 1 -- 172.21.15.46:0/572883402 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1464070b40 con 0x7f14640ceb20 2024-03-29T08:18:39.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.012+0000 7f1468e00640 1 --2- 172.21.15.46:0/572883402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14640ceb20 0x7f146406c950 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1464071360 tx=0x7f145c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c9e3993dd5b8fc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:39.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.012+0000 7f1463400640 1 -- 172.21.15.46:0/572883402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c0a5780 con 0x7f14640ceb20 2024-03-29T08:18:39.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.012+0000 7f1463400640 1 -- 172.21.15.46:0/572883402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c0a58e0 con 0x7f14640ceb20 2024-03-29T08:18:39.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.012+0000 7f1463400640 1 -- 172.21.15.46:0/572883402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c0a5c70 con 0x7f14640ceb20 2024-03-29T08:18:39.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f146ae00640 1 -- 172.21.15.46:0/572883402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14640ceb20 msgr2=0x7f146406c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:39.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f146ae00640 1 --2- 172.21.15.46:0/572883402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14640ceb20 0x7f146406c950 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1464071360 tx=0x7f145c01c9b0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f146ae00640 1 -- 172.21.15.46:0/572883402 shutdown_connections 2024-03-29T08:18:39.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f146ae00640 1 --2- 172.21.15.46:0/572883402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14640ceb20 0x7f146406c950 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f146ae00640 1 --2- 172.21.15.46:0/572883402 >> v2:172.21.15.73:3300/0 conn(0x7f1464144210 0x7f14640cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f146ae00640 1 -- 172.21.15.46:0/572883402 >> 172.21.15.46:0/572883402 conn(0x7f1464066110 msgr2=0x7f1464069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:39.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f146ae00640 1 -- 172.21.15.46:0/572883402 shutdown_connections 2024-03-29T08:18:39.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f146ae00640 1 -- 172.21.15.46:0/572883402 wait complete. 2024-03-29T08:18:39.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f146ae00640 1 Processor -- start 2024-03-29T08:18:39.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f146ae00640 1 -- start start 2024-03-29T08:18:39.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f146ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14640ceb20 0x7f1464164fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f146ae00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1464144210 0x7f146415fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f146ae00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f146415ada0 0x7f1464142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f146ae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146415dc30 con 0x7f14640ceb20 2024-03-29T08:18:39.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f146ae00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146415de80 con 0x7f1464144210 2024-03-29T08:18:39.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f146ae00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146415e160 con 0x7f146415ada0 2024-03-29T08:18:39.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f1463e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14640ceb20 0x7f1464164fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:39.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f1463e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14640ceb20 0x7f1464164fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43218/0 (socket says 172.21.15.46:43218) 2024-03-29T08:18:39.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f1463e00640 1 -- 172.21.15.46:0/2655875067 learned_addr learned my addr 172.21.15.46:0/2655875067 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:39.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f145be00640 1 --2- 172.21.15.46:0/2655875067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1464144210 0x7f146415fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:39.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f1468e00640 1 --2- 172.21.15.46:0/2655875067 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f146415ada0 0x7f1464142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:39.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f1463e00640 1 -- 172.21.15.46:0/2655875067 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f146415ada0 msgr2=0x7f1464142da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:39.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f1463e00640 1 --2- 172.21.15.46:0/2655875067 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f146415ada0 0x7f1464142da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f1463e00640 1 -- 172.21.15.46:0/2655875067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1464144210 msgr2=0x7f146415fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:39.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f1463e00640 1 --2- 172.21.15.46:0/2655875067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1464144210 0x7f146415fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.016+0000 7f1463e00640 1 -- 172.21.15.46:0/2655875067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c009b30 con 0x7f14640ceb20 2024-03-29T08:18:39.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.020+0000 7f1463e00640 1 --2- 172.21.15.46:0/2655875067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14640ceb20 0x7f1464164fa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f145400d800 tx=0x7f145400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:39.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.020+0000 7f1460c00640 1 -- 172.21.15.46:0/2655875067 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145400b6c0 con 0x7f14640ceb20 2024-03-29T08:18:39.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.020+0000 7f1460c00640 1 -- 172.21.15.46:0/2655875067 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145400b820 con 0x7f14640ceb20 2024-03-29T08:18:39.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.020+0000 7f146ae00640 1 -- 172.21.15.46:0/2655875067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146415e440 con 0x7f14640ceb20 2024-03-29T08:18:39.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.020+0000 7f1460c00640 1 -- 172.21.15.46:0/2655875067 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145400bb90 con 0x7f14640ceb20 2024-03-29T08:18:39.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.020+0000 7f146ae00640 1 -- 172.21.15.46:0/2655875067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146415ea00 con 0x7f14640ceb20 2024-03-29T08:18:39.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.020+0000 7f146ae00640 1 -- 172.21.15.46:0/2655875067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14640ce280 con 0x7f14640ceb20 2024-03-29T08:18:39.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.020+0000 7f1460c00640 1 -- 172.21.15.46:0/2655875067 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f145400bcf0 con 0x7f14640ceb20 2024-03-29T08:18:39.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.020+0000 7f1460c00640 1 --2- 172.21.15.46:0/2655875067 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1440066690 0x7f1440068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.020+0000 7f1460c00640 1 -- 172.21.15.46:0/2655875067 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1454084520 con 0x7f14640ceb20 2024-03-29T08:18:39.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.020+0000 7f145be00640 1 --2- 172.21.15.46:0/2655875067 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1440066690 0x7f1440068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:39.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.024+0000 7f145be00640 1 --2- 172.21.15.46:0/2655875067 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1440066690 0x7f1440068b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1464142030 tx=0x7f144c009060 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:39.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.024+0000 7f1460c00640 1 -- 172.21.15.46:0/2655875067 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f14540519d0 con 0x7f14640ceb20 2024-03-29T08:18:39.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.188+0000 7f146ae00640 1 -- 172.21.15.46:0/2655875067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1464066c90 con 0x7f14640ceb20 2024-03-29T08:18:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.188+0000 7f1460c00640 1 -- 172.21.15.46:0/2655875067 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f1454054ff0 con 0x7f14640ceb20 2024-03-29T08:18:39.191 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:39.191 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:14.314124+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.192+0000 7f146ae00640 1 -- 172.21.15.46:0/2655875067 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1440066690 msgr2=0x7f1440068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.192+0000 7f146ae00640 1 --2- 172.21.15.46:0/2655875067 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1440066690 0x7f1440068b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1464142030 tx=0x7f144c009060 comp rx=0 tx=0).stop 2024-03-29T08:18:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.192+0000 7f146ae00640 1 -- 172.21.15.46:0/2655875067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14640ceb20 msgr2=0x7f1464164fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.192+0000 7f146ae00640 1 --2- 172.21.15.46:0/2655875067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14640ceb20 0x7f1464164fa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f145400d800 tx=0x7f145400dcc0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.192+0000 7f146ae00640 1 -- 172.21.15.46:0/2655875067 shutdown_connections 2024-03-29T08:18:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.192+0000 7f146ae00640 1 --2- 172.21.15.46:0/2655875067 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1440066690 0x7f1440068b50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.192+0000 7f146ae00640 1 --2- 172.21.15.46:0/2655875067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1464144210 0x7f146415fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.192+0000 7f146ae00640 1 --2- 172.21.15.46:0/2655875067 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f146415ada0 0x7f1464142da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.192+0000 7f146ae00640 1 --2- 172.21.15.46:0/2655875067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14640ceb20 0x7f1464164fa0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.192+0000 7f146ae00640 1 -- 172.21.15.46:0/2655875067 >> 172.21.15.46:0/2655875067 conn(0x7f1464066110 msgr2=0x7f14640ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.192+0000 7f146ae00640 1 -- 172.21.15.46:0/2655875067 shutdown_connections 2024-03-29T08:18:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.192+0000 7f146ae00640 1 -- 172.21.15.46:0/2655875067 wait complete. 2024-03-29T08:18:39.212 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:18:11.177842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:18:12.308252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:18:39.213 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:18:39.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.352+0000 7efd2bc00640 1 Processor -- start 2024-03-29T08:18:39.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.356+0000 7efd2bc00640 1 -- start start 2024-03-29T08:18:39.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.356+0000 7efd2bc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efd24142a50 0x7efd240cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.356+0000 7efd2bc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efd2415df90 0x7efd240cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.356+0000 7efd2bc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd2415d260 con 0x7efd240ae280 2024-03-29T08:18:39.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.356+0000 7efd2bc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd240aefd0 con 0x7efd24142a50 2024-03-29T08:18:39.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.356+0000 7efd2bc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd2415d4b0 con 0x7efd2415df90 2024-03-29T08:18:39.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.356+0000 7efd29200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efd24142a50 0x7efd240cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:39.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.356+0000 7efd29200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efd24142a50 0x7efd240cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43230/0 (socket says 172.21.15.46:43230) 2024-03-29T08:18:39.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.356+0000 7efd29200640 1 -- 172.21.15.46:0/4243148068 learned_addr learned my addr 172.21.15.46:0/4243148068 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:39.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.356+0000 7efd29200640 1 -- 172.21.15.46:0/4243148068 >> v2:172.21.15.73:3300/0 conn(0x7efd2415df90 msgr2=0x7efd240cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:39.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.356+0000 7efd29200640 1 --2- 172.21.15.46:0/4243148068 >> v2:172.21.15.73:3300/0 conn(0x7efd2415df90 0x7efd240cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.356+0000 7efd29200640 1 -- 172.21.15.46:0/4243148068 >> v1:172.21.15.73:6789/0 conn(0x7efd240ae280 legacy=0x7efd240ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:39.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.356+0000 7efd29200640 1 -- 172.21.15.46:0/4243148068 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd2415d790 con 0x7efd24142a50 2024-03-29T08:18:39.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.360+0000 7efd29200640 1 --2- 172.21.15.46:0/4243148068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd24142a50 0x7efd240cb790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efd2406e480 tx=0x7efd1401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6918a0a4fb5492be server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:39.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.360+0000 7efd23e00640 1 -- 172.21.15.46:0/4243148068 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd140ab600 con 0x7efd24142a50 2024-03-29T08:18:39.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.360+0000 7efd23e00640 1 -- 172.21.15.46:0/4243148068 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd140a7050 con 0x7efd24142a50 2024-03-29T08:18:39.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.360+0000 7efd23e00640 1 -- 172.21.15.46:0/4243148068 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd140ab980 con 0x7efd24142a50 2024-03-29T08:18:39.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.360+0000 7efd2bc00640 1 -- 172.21.15.46:0/4243148068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd24142a50 msgr2=0x7efd240cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:39.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.360+0000 7efd2bc00640 1 --2- 172.21.15.46:0/4243148068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd24142a50 0x7efd240cb790 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efd2406e480 tx=0x7efd1401c450 comp rx=0 tx=0).stop 2024-03-29T08:18:39.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.360+0000 7efd2bc00640 1 -- 172.21.15.46:0/4243148068 shutdown_connections 2024-03-29T08:18:39.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.360+0000 7efd2bc00640 1 --2- 172.21.15.46:0/4243148068 >> v2:172.21.15.73:3300/0 conn(0x7efd2415df90 0x7efd240cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.360+0000 7efd2bc00640 1 --2- 172.21.15.46:0/4243148068 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd24142a50 0x7efd240cb790 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.360+0000 7efd2bc00640 1 -- 172.21.15.46:0/4243148068 >> 172.21.15.46:0/4243148068 conn(0x7efd2406a390 msgr2=0x7efd240cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:39.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.360+0000 7efd2bc00640 1 -- 172.21.15.46:0/4243148068 shutdown_connections 2024-03-29T08:18:39.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.360+0000 7efd2bc00640 1 -- 172.21.15.46:0/4243148068 wait complete. 2024-03-29T08:18:39.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.360+0000 7efd2bc00640 1 Processor -- start 2024-03-29T08:18:39.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.360+0000 7efd2bc00640 1 -- start start 2024-03-29T08:18:39.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd2bc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efd240ae280 0x7efd24158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd2bc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd24142a50 0x7efd24163ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd2bc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd2415df90 0x7efd241f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd2bc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd241f5560 con 0x7efd24142a50 2024-03-29T08:18:39.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd2bc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd241f5ad0 con 0x7efd2415df90 2024-03-29T08:18:39.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd2bc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd24164400 con 0x7efd240ae280 2024-03-29T08:18:39.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd1be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd24142a50 0x7efd24163ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:39.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd1be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd24142a50 0x7efd24163ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43244/0 (socket says 172.21.15.46:43244) 2024-03-29T08:18:39.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd1be00640 1 -- 172.21.15.46:0/274093194 learned_addr learned my addr 172.21.15.46:0/274093194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:39.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd29200640 1 --2- 172.21.15.46:0/274093194 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efd240ae280 0x7efd24158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:39.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd29c00640 1 --2- 172.21.15.46:0/274093194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd2415df90 0x7efd241f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:39.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd1be00640 1 -- 172.21.15.46:0/274093194 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efd240ae280 msgr2=0x7efd24158aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:39.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd1be00640 1 --2- 172.21.15.46:0/274093194 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efd240ae280 0x7efd24158aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd1be00640 1 -- 172.21.15.46:0/274093194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd2415df90 msgr2=0x7efd241f5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:39.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd1be00640 1 --2- 172.21.15.46:0/274093194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd2415df90 0x7efd241f5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd1be00640 1 -- 172.21.15.46:0/274093194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd140094d0 con 0x7efd24142a50 2024-03-29T08:18:39.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd29c00640 1 --2- 172.21.15.46:0/274093194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd2415df90 0x7efd241f5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:39.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd1be00640 1 --2- 172.21.15.46:0/274093194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd24142a50 0x7efd24163ec0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00b410 tx=0x7efd0c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:39.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd21600640 1 -- 172.21.15.46:0/274093194 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c011830 con 0x7efd24142a50 2024-03-29T08:18:39.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd21600640 1 -- 172.21.15.46:0/274093194 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0c011990 con 0x7efd24142a50 2024-03-29T08:18:39.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd21600640 1 -- 172.21.15.46:0/274093194 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0c011d40 con 0x7efd24142a50 2024-03-29T08:18:39.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd2bc00640 1 -- 172.21.15.46:0/274093194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd241635d0 con 0x7efd24142a50 2024-03-29T08:18:39.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd2bc00640 1 -- 172.21.15.46:0/274093194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd241f5760 con 0x7efd24142a50 2024-03-29T08:18:39.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd2bc00640 1 -- 172.21.15.46:0/274093194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd240733a0 con 0x7efd24142a50 2024-03-29T08:18:39.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.364+0000 7efd21600640 1 -- 172.21.15.46:0/274093194 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efd0c01a400 con 0x7efd24142a50 2024-03-29T08:18:39.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.368+0000 7efd21600640 1 --2- 172.21.15.46:0/274093194 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efd000666c0 0x7efd00068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.368+0000 7efd21600640 1 -- 172.21.15.46:0/274093194 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efd0c0841e0 con 0x7efd24142a50 2024-03-29T08:18:39.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.368+0000 7efd29200640 1 --2- 172.21.15.46:0/274093194 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efd000666c0 0x7efd00068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:39.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.368+0000 7efd29200640 1 --2- 172.21.15.46:0/274093194 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efd000666c0 0x7efd00068b80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efd140ad000 tx=0x7efd140a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:39.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.372+0000 7efd21600640 1 -- 172.21.15.46:0/274093194 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efd0c0515e0 con 0x7efd24142a50 2024-03-29T08:18:39.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.532+0000 7efd2bc00640 1 -- 172.21.15.46:0/274093194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd24071a70 con 0x7efd24142a50 2024-03-29T08:18:39.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.532+0000 7efd21600640 1 -- 172.21.15.46:0/274093194 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd0c054c00 con 0x7efd24142a50 2024-03-29T08:18:39.533 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:18:39.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.532+0000 7efd2bc00640 1 -- 172.21.15.46:0/274093194 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efd000666c0 msgr2=0x7efd00068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:39.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.532+0000 7efd2bc00640 1 --2- 172.21.15.46:0/274093194 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efd000666c0 0x7efd00068b80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efd140ad000 tx=0x7efd140a6000 comp rx=0 tx=0).stop 2024-03-29T08:18:39.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.532+0000 7efd2bc00640 1 -- 172.21.15.46:0/274093194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd24142a50 msgr2=0x7efd24163ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:39.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.532+0000 7efd2bc00640 1 --2- 172.21.15.46:0/274093194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd24142a50 0x7efd24163ec0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efd0c00b410 tx=0x7efd0c00b8d0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.532+0000 7efd2bc00640 1 -- 172.21.15.46:0/274093194 shutdown_connections 2024-03-29T08:18:39.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.532+0000 7efd2bc00640 1 --2- 172.21.15.46:0/274093194 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efd000666c0 0x7efd00068b80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.532+0000 7efd2bc00640 1 --2- 172.21.15.46:0/274093194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd2415df90 0x7efd241f5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.532+0000 7efd2bc00640 1 --2- 172.21.15.46:0/274093194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd24142a50 0x7efd24163ec0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.532+0000 7efd2bc00640 1 --2- 172.21.15.46:0/274093194 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efd240ae280 0x7efd24158aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.532+0000 7efd2bc00640 1 -- 172.21.15.46:0/274093194 >> 172.21.15.46:0/274093194 conn(0x7efd2406a390 msgr2=0x7efd240ca8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:39.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.532+0000 7efd2bc00640 1 -- 172.21.15.46:0/274093194 shutdown_connections 2024-03-29T08:18:39.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.532+0000 7efd2bc00640 1 -- 172.21.15.46:0/274093194 wait complete. 2024-03-29T08:18:39.554 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:18:39.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.696+0000 7f13b9e00640 1 Processor -- start 2024-03-29T08:18:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.696+0000 7f13b9e00640 1 -- start start 2024-03-29T08:18:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.696+0000 7f13b9e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f13b415cad0 0x7f13b406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.696+0000 7f13b9e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f13b406eff0 0x7f13b4069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.696+0000 7f13b9e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13b406f450 con 0x7f13b406a2d0 2024-03-29T08:18:39.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.696+0000 7f13b9e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b415dd00 con 0x7f13b406eff0 2024-03-29T08:18:39.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.696+0000 7f13b9e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b406a740 con 0x7f13b415cad0 2024-03-29T08:18:39.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.696+0000 7f13aa000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f13b406eff0 0x7f13b4069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:39.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.696+0000 7f13b2a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f13b415cad0 0x7f13b406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:39.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.696+0000 7f13aa000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f13b406eff0 0x7f13b4069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:59784/0 (socket says 172.21.15.46:59784) 2024-03-29T08:18:39.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.696+0000 7f13b3400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f13b406a2d0 0x7f13b41604d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:60292/0 (socket says 172.21.15.46:60292) 2024-03-29T08:18:39.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.696+0000 7f13b2a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f13b415cad0 0x7f13b406eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:46070/0 (socket says 172.21.15.46:46070) 2024-03-29T08:18:39.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.696+0000 7f13aa000640 1 -- 172.21.15.46:0/620678181 learned_addr learned my addr 172.21.15.46:0/620678181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:39.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.696+0000 7f13b2a00640 1 -- 172.21.15.46:0/620678181 >> v2:172.21.15.73:3300/0 conn(0x7f13b406eff0 msgr2=0x7f13b4069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:39.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.696+0000 7f13b2a00640 1 --2- 172.21.15.46:0/620678181 >> v2:172.21.15.73:3300/0 conn(0x7f13b406eff0 0x7f13b4069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.700+0000 7f13b2a00640 1 -- 172.21.15.46:0/620678181 >> v1:172.21.15.73:6789/0 conn(0x7f13b406a2d0 legacy=0x7f13b41604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:39.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.700+0000 7f13b2a00640 1 -- 172.21.15.46:0/620678181 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b415d080 con 0x7f13b415cad0 2024-03-29T08:18:39.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.700+0000 7f13b2a00640 1 --2- 172.21.15.46:0/620678181 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f13b415cad0 0x7f13b406eab0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009d90 tx=0x7f13a401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1465701dcdb2193f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:39.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.700+0000 7f13b2000640 1 -- 172.21.15.46:0/620678181 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a40a58b0 con 0x7f13b415cad0 2024-03-29T08:18:39.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.700+0000 7f13b2000640 1 -- 172.21.15.46:0/620678181 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a40a5a10 con 0x7f13b415cad0 2024-03-29T08:18:39.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.700+0000 7f13b2000640 1 -- 172.21.15.46:0/620678181 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a40a5da0 con 0x7f13b415cad0 2024-03-29T08:18:39.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.700+0000 7f13b9e00640 1 -- 172.21.15.46:0/620678181 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f13b415cad0 msgr2=0x7f13b406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:39.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.700+0000 7f13b9e00640 1 --2- 172.21.15.46:0/620678181 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f13b415cad0 0x7f13b406eab0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009d90 tx=0x7f13a401c8e0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.700+0000 7f13b9e00640 1 -- 172.21.15.46:0/620678181 shutdown_connections 2024-03-29T08:18:39.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.700+0000 7f13b9e00640 1 --2- 172.21.15.46:0/620678181 >> v2:172.21.15.73:3300/0 conn(0x7f13b406eff0 0x7f13b4069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.700+0000 7f13b9e00640 1 --2- 172.21.15.46:0/620678181 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f13b415cad0 0x7f13b406eab0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.700+0000 7f13b9e00640 1 -- 172.21.15.46:0/620678181 >> 172.21.15.46:0/620678181 conn(0x7f13b4098a40 msgr2=0x7f13b40b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:39.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.700+0000 7f13b9e00640 1 -- 172.21.15.46:0/620678181 shutdown_connections 2024-03-29T08:18:39.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.700+0000 7f13b9e00640 1 -- 172.21.15.46:0/620678181 wait complete. 2024-03-29T08:18:39.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13b9e00640 1 Processor -- start 2024-03-29T08:18:39.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13b9e00640 1 -- start start 2024-03-29T08:18:39.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13b9e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f13b406a2d0 0x7f13b4150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b406eff0 0x7f13b40b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13b9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13b415cad0 0x7f13b40aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13b9e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b4154dd0 con 0x7f13b406eff0 2024-03-29T08:18:39.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13b9e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b41550b0 con 0x7f13b415cad0 2024-03-29T08:18:39.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13b9e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b4155390 con 0x7f13b406a2d0 2024-03-29T08:18:39.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13aa000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b406eff0 0x7f13b40b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:39.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13aa000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b406eff0 0x7f13b40b0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43258/0 (socket says 172.21.15.46:43258) 2024-03-29T08:18:39.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13aa000640 1 -- 172.21.15.46:0/238505909 learned_addr learned my addr 172.21.15.46:0/238505909 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:39.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13b3400640 1 --2- 172.21.15.46:0/238505909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13b415cad0 0x7f13b40aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:39.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13b2a00640 1 --2- 172.21.15.46:0/238505909 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f13b406a2d0 0x7f13b4150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13aa000640 1 -- 172.21.15.46:0/238505909 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f13b406a2d0 msgr2=0x7f13b4150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13aa000640 1 --2- 172.21.15.46:0/238505909 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f13b406a2d0 0x7f13b4150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13aa000640 1 -- 172.21.15.46:0/238505909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13b415cad0 msgr2=0x7f13b40aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13aa000640 1 --2- 172.21.15.46:0/238505909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13b415cad0 0x7f13b40aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13aa000640 1 -- 172.21.15.46:0/238505909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a4009a70 con 0x7f13b406eff0 2024-03-29T08:18:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.704+0000 7f13aa000640 1 --2- 172.21.15.46:0/238505909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b406eff0 0x7f13b40b0d60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f139c00e890 tx=0x7f139c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.708+0000 7f13abe00640 1 -- 172.21.15.46:0/238505909 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c009620 con 0x7f13b406eff0 2024-03-29T08:18:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.708+0000 7f13abe00640 1 -- 172.21.15.46:0/238505909 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139c00c940 con 0x7f13b406eff0 2024-03-29T08:18:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.708+0000 7f13abe00640 1 -- 172.21.15.46:0/238505909 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c00ccf0 con 0x7f13b406eff0 2024-03-29T08:18:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.708+0000 7f13b9e00640 1 -- 172.21.15.46:0/238505909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b4155670 con 0x7f13b406eff0 2024-03-29T08:18:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.708+0000 7f13b9e00640 1 -- 172.21.15.46:0/238505909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b4151450 con 0x7f13b406eff0 2024-03-29T08:18:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.708+0000 7f13b9e00640 1 -- 172.21.15.46:0/238505909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b40cd8f0 con 0x7f13b406eff0 2024-03-29T08:18:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.708+0000 7f13abe00640 1 -- 172.21.15.46:0/238505909 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f139c013470 con 0x7f13b406eff0 2024-03-29T08:18:39.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.708+0000 7f13abe00640 1 --2- 172.21.15.46:0/238505909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f13900666c0 0x7f1390068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:39.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.708+0000 7f13abe00640 1 -- 172.21.15.46:0/238505909 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f139c084350 con 0x7f13b406eff0 2024-03-29T08:18:39.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.708+0000 7f13b2a00640 1 --2- 172.21.15.46:0/238505909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f13900666c0 0x7f1390068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:39.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.712+0000 7f13b2a00640 1 --2- 172.21.15.46:0/238505909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f13900666c0 0x7f1390068b80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13a40af000 tx=0x7f13a40a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:39.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.712+0000 7f13abe00640 1 -- 172.21.15.46:0/238505909 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f139c051650 con 0x7f13b406eff0 2024-03-29T08:18:39.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.872+0000 7f13b9e00640 1 -- 172.21.15.46:0/238505909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13b409d700 con 0x7f13b406eff0 2024-03-29T08:18:39.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.872+0000 7f13abe00640 1 -- 172.21.15.46:0/238505909 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f139c054c70 con 0x7f13b406eff0 2024-03-29T08:18:39.874 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:18:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.876+0000 7f13b9e00640 1 -- 172.21.15.46:0/238505909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f13900666c0 msgr2=0x7f1390068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.876+0000 7f13b9e00640 1 --2- 172.21.15.46:0/238505909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f13900666c0 0x7f1390068b80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f13a40af000 tx=0x7f13a40a7070 comp rx=0 tx=0).stop 2024-03-29T08:18:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.876+0000 7f13b9e00640 1 -- 172.21.15.46:0/238505909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b406eff0 msgr2=0x7f13b40b0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.876+0000 7f13b9e00640 1 --2- 172.21.15.46:0/238505909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b406eff0 0x7f13b40b0d60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f139c00e890 tx=0x7f139c00ed50 comp rx=0 tx=0).stop 2024-03-29T08:18:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.876+0000 7f13b9e00640 1 -- 172.21.15.46:0/238505909 shutdown_connections 2024-03-29T08:18:39.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.876+0000 7f13b9e00640 1 --2- 172.21.15.46:0/238505909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f13900666c0 0x7f1390068b80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.876+0000 7f13b9e00640 1 --2- 172.21.15.46:0/238505909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13b415cad0 0x7f13b40aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.876+0000 7f13b9e00640 1 --2- 172.21.15.46:0/238505909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13b406eff0 0x7f13b40b0d60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.876+0000 7f13b9e00640 1 --2- 172.21.15.46:0/238505909 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f13b406a2d0 0x7f13b4150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.876+0000 7f13b9e00640 1 -- 172.21.15.46:0/238505909 >> 172.21.15.46:0/238505909 conn(0x7f13b4098a40 msgr2=0x7f13b4157080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.876+0000 7f13b9e00640 1 -- 172.21.15.46:0/238505909 shutdown_connections 2024-03-29T08:18:39.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:39.876+0000 7f13b9e00640 1 -- 172.21.15.46:0/238505909 wait complete. 2024-03-29T08:18:39.895 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:18:40.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.036+0000 7f1c40400640 1 Processor -- start 2024-03-29T08:18:40.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.036+0000 7f1c40400640 1 -- start start 2024-03-29T08:18:40.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.040+0000 7f1c40400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c3815a7f0 0x7f1c380cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.040+0000 7f1c40400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1c3815abd0 0x7f1c380cb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.040+0000 7f1c40400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c380cbd00 con 0x7f1c38098a40 2024-03-29T08:18:40.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.040+0000 7f1c40400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c380b4500 con 0x7f1c3815abd0 2024-03-29T08:18:40.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.040+0000 7f1c40400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c380cf970 con 0x7f1c3815a7f0 2024-03-29T08:18:40.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.040+0000 7f1c3da00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c3815a7f0 0x7f1c380cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.040+0000 7f1c3d000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1c3815abd0 0x7f1c380cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.040+0000 7f1c3d000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1c3815abd0 0x7f1c380cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:59798/0 (socket says 172.21.15.46:59798) 2024-03-29T08:18:40.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.040+0000 7f1c3d000640 1 -- 172.21.15.46:0/3971248815 learned_addr learned my addr 172.21.15.46:0/3971248815 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:40.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.040+0000 7f1c3d000640 1 -- 172.21.15.46:0/3971248815 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c3815a7f0 msgr2=0x7f1c380cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:40.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.040+0000 7f1c3d000640 1 --2- 172.21.15.46:0/3971248815 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c3815a7f0 0x7f1c380cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.040+0000 7f1c3d000640 1 -- 172.21.15.46:0/3971248815 >> v1:172.21.15.73:6789/0 conn(0x7f1c38098a40 legacy=0x7f1c3815e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.040+0000 7f1c3d000640 1 -- 172.21.15.46:0/3971248815 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c380cbfe0 con 0x7f1c3815abd0 2024-03-29T08:18:40.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.040+0000 7f1c37e00640 1 -- 172.21.15.46:0/3971248815 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2294307858 0 0) 0x7f1c380cbd00 con 0x7f1c38098a40 2024-03-29T08:18:40.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.044+0000 7f1c3d000640 1 --2- 172.21.15.46:0/3971248815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c3815abd0 0x7f1c380cb490 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009870 tx=0x7f1c2401c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6d2de9b40e0b42c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:40.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.044+0000 7f1c37e00640 1 -- 172.21.15.46:0/3971248815 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c240a95f0 con 0x7f1c3815abd0 2024-03-29T08:18:40.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.044+0000 7f1c37e00640 1 -- 172.21.15.46:0/3971248815 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c240a9750 con 0x7f1c3815abd0 2024-03-29T08:18:40.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.044+0000 7f1c37e00640 1 -- 172.21.15.46:0/3971248815 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c240a9ae0 con 0x7f1c3815abd0 2024-03-29T08:18:40.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.044+0000 7f1c40400640 1 -- 172.21.15.46:0/3971248815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c3815abd0 msgr2=0x7f1c380cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.044+0000 7f1c40400640 1 --2- 172.21.15.46:0/3971248815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c3815abd0 0x7f1c380cb490 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1c24009870 tx=0x7f1c2401c530 comp rx=0 tx=0).stop 2024-03-29T08:18:40.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.044+0000 7f1c40400640 1 -- 172.21.15.46:0/3971248815 shutdown_connections 2024-03-29T08:18:40.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.044+0000 7f1c40400640 1 --2- 172.21.15.46:0/3971248815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c3815abd0 0x7f1c380cb490 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.044+0000 7f1c40400640 1 --2- 172.21.15.46:0/3971248815 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c3815a7f0 0x7f1c380cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.044+0000 7f1c40400640 1 -- 172.21.15.46:0/3971248815 >> 172.21.15.46:0/3971248815 conn(0x7f1c380b2190 msgr2=0x7f1c380b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:40.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.044+0000 7f1c40400640 1 -- 172.21.15.46:0/3971248815 shutdown_connections 2024-03-29T08:18:40.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.044+0000 7f1c40400640 1 -- 172.21.15.46:0/3971248815 wait complete. 2024-03-29T08:18:40.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.044+0000 7f1c40400640 1 Processor -- start 2024-03-29T08:18:40.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.044+0000 7f1c40400640 1 -- start start 2024-03-29T08:18:40.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c40400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c38098a40 0x7f1c3814d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c40400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c3815a7f0 0x7f1c3814d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c40400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c3815abd0 0x7f1c380b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c40400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3814d740 con 0x7f1c3815abd0 2024-03-29T08:18:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c40400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3814ea70 con 0x7f1c3815a7f0 2024-03-29T08:18:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c40400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3814e040 con 0x7f1c38098a40 2024-03-29T08:18:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c3da00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c38098a40 0x7f1c3814d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c3e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c3815abd0 0x7f1c380b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c3da00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c38098a40 0x7f1c3814d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:46112/0 (socket says 172.21.15.46:46112) 2024-03-29T08:18:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c3da00640 1 -- 172.21.15.46:0/135421101 learned_addr learned my addr 172.21.15.46:0/135421101 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c3d000640 1 --2- 172.21.15.46:0/135421101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c3815a7f0 0x7f1c3814d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c3e400640 1 -- 172.21.15.46:0/135421101 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c38098a40 msgr2=0x7f1c3814d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c3e400640 1 --2- 172.21.15.46:0/135421101 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c38098a40 0x7f1c3814d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c3e400640 1 -- 172.21.15.46:0/135421101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c3815a7f0 msgr2=0x7f1c3814d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c3e400640 1 --2- 172.21.15.46:0/135421101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c3815a7f0 0x7f1c3814d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c3e400640 1 -- 172.21.15.46:0/135421101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c240094d0 con 0x7f1c3815abd0 2024-03-29T08:18:40.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c3da00640 1 --2- 172.21.15.46:0/135421101 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c38098a40 0x7f1c3814d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c3d000640 1 --2- 172.21.15.46:0/135421101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c3815a7f0 0x7f1c3814d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c3e400640 1 --2- 172.21.15.46:0/135421101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c3815abd0 0x7f1c380b4bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000c700 tx=0x7f1c3000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c35600640 1 -- 172.21.15.46:0/135421101 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c30003db0 con 0x7f1c3815abd0 2024-03-29T08:18:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c35600640 1 -- 172.21.15.46:0/135421101 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3000ce10 con 0x7f1c3815abd0 2024-03-29T08:18:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c40400640 1 -- 172.21.15.46:0/135421101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c3814e320 con 0x7f1c3815abd0 2024-03-29T08:18:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c35600640 1 -- 172.21.15.46:0/135421101 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c300052b0 con 0x7f1c3815abd0 2024-03-29T08:18:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c40400640 1 -- 172.21.15.46:0/135421101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3814cce0 con 0x7f1c3815abd0 2024-03-29T08:18:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c40400640 1 -- 172.21.15.46:0/135421101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3814e560 con 0x7f1c3815abd0 2024-03-29T08:18:40.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.048+0000 7f1c35600640 1 -- 172.21.15.46:0/135421101 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1c30005410 con 0x7f1c3815abd0 2024-03-29T08:18:40.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.052+0000 7f1c35600640 1 --2- 172.21.15.46:0/135421101 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c18066690 0x7f1c18068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.052+0000 7f1c35600640 1 -- 172.21.15.46:0/135421101 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c300839b0 con 0x7f1c3815abd0 2024-03-29T08:18:40.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.052+0000 7f1c3da00640 1 --2- 172.21.15.46:0/135421101 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c18066690 0x7f1c18068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.052+0000 7f1c3da00640 1 --2- 172.21.15.46:0/135421101 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c18066690 0x7f1c18068b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c007f00 tx=0x7f1c2c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:40.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.056+0000 7f1c35600640 1 -- 172.21.15.46:0/135421101 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1c30050d30 con 0x7f1c3815abd0 2024-03-29T08:18:40.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.220+0000 7f1c40400640 1 -- 172.21.15.46:0/135421101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1c3814ced0 con 0x7f1c3815abd0 2024-03-29T08:18:40.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.220+0000 7f1c35600640 1 -- 172.21.15.46:0/135421101 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1c30017070 con 0x7f1c3815abd0 2024-03-29T08:18:40.221 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:18:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.220+0000 7f1c40400640 1 -- 172.21.15.46:0/135421101 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c18066690 msgr2=0x7f1c18068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.220+0000 7f1c40400640 1 --2- 172.21.15.46:0/135421101 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c18066690 0x7f1c18068b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c007f00 tx=0x7f1c2c008040 comp rx=0 tx=0).stop 2024-03-29T08:18:40.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.220+0000 7f1c40400640 1 -- 172.21.15.46:0/135421101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c3815abd0 msgr2=0x7f1c380b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.220+0000 7f1c40400640 1 --2- 172.21.15.46:0/135421101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c3815abd0 0x7f1c380b4bf0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000c700 tx=0x7f1c3000cbc0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.220+0000 7f1c40400640 1 -- 172.21.15.46:0/135421101 shutdown_connections 2024-03-29T08:18:40.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.220+0000 7f1c40400640 1 --2- 172.21.15.46:0/135421101 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c18066690 0x7f1c18068b50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.220+0000 7f1c40400640 1 --2- 172.21.15.46:0/135421101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c3815a7f0 0x7f1c3814d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.220+0000 7f1c40400640 1 --2- 172.21.15.46:0/135421101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c3815abd0 0x7f1c380b4bf0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.220+0000 7f1c40400640 1 --2- 172.21.15.46:0/135421101 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c38098a40 0x7f1c3814d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.220+0000 7f1c40400640 1 -- 172.21.15.46:0/135421101 >> 172.21.15.46:0/135421101 conn(0x7f1c380b2190 msgr2=0x7f1c380cee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:40.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.220+0000 7f1c40400640 1 -- 172.21.15.46:0/135421101 shutdown_connections 2024-03-29T08:18:40.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.220+0000 7f1c40400640 1 -- 172.21.15.46:0/135421101 wait complete. 2024-03-29T08:18:40.242 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:40.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.380+0000 7f864a000640 1 Processor -- start 2024-03-29T08:18:40.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.380+0000 7f864a000640 1 -- start start 2024-03-29T08:18:40.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.380+0000 7f864a000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86441593d0 0x7f864406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.380+0000 7f864a000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f864415cd90 0x7f86441597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.380+0000 7f864a000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f864406d130 con 0x7f864415d230 2024-03-29T08:18:40.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.380+0000 7f864a000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864415b8f0 con 0x7f86441593d0 2024-03-29T08:18:40.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.380+0000 7f864a000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864415ad50 con 0x7f864415cd90 2024-03-29T08:18:40.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.380+0000 7f8643400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86441593d0 0x7f864406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.380+0000 7f8643400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86441593d0 0x7f864406cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43278/0 (socket says 172.21.15.46:43278) 2024-03-29T08:18:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.380+0000 7f8643400640 1 -- 172.21.15.46:0/1320945276 learned_addr learned my addr 172.21.15.46:0/1320945276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.380+0000 7f8643e00640 1 --2- 172.21.15.46:0/1320945276 >> v2:172.21.15.73:3300/0 conn(0x7f864415cd90 0x7f86441597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.380+0000 7f8643400640 1 -- 172.21.15.46:0/1320945276 >> v2:172.21.15.73:3300/0 conn(0x7f864415cd90 msgr2=0x7f86441597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.380+0000 7f8643400640 1 --2- 172.21.15.46:0/1320945276 >> v2:172.21.15.73:3300/0 conn(0x7f864415cd90 0x7f86441597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.380+0000 7f8643400640 1 -- 172.21.15.46:0/1320945276 >> v1:172.21.15.73:6789/0 conn(0x7f864415d230 legacy=0x7f864415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:40.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.380+0000 7f8643400640 1 -- 172.21.15.46:0/1320945276 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864415afc0 con 0x7f86441593d0 2024-03-29T08:18:40.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.384+0000 7f8643400640 1 --2- 172.21.15.46:0/1320945276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86441593d0 0x7f864406cbf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f864406d360 tx=0x7f863401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c930294d4d11b61d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:40.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.384+0000 7f8642a00640 1 -- 172.21.15.46:0/1320945276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86340aa640 con 0x7f86441593d0 2024-03-29T08:18:40.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.384+0000 7f8642a00640 1 -- 172.21.15.46:0/1320945276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86340aa7a0 con 0x7f86441593d0 2024-03-29T08:18:40.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.384+0000 7f8642a00640 1 -- 172.21.15.46:0/1320945276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86340aab30 con 0x7f86441593d0 2024-03-29T08:18:40.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.384+0000 7f864a000640 1 -- 172.21.15.46:0/1320945276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86441593d0 msgr2=0x7f864406cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.384+0000 7f864a000640 1 --2- 172.21.15.46:0/1320945276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86441593d0 0x7f864406cbf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f864406d360 tx=0x7f863401c4e0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.384+0000 7f864a000640 1 -- 172.21.15.46:0/1320945276 shutdown_connections 2024-03-29T08:18:40.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.384+0000 7f864a000640 1 --2- 172.21.15.46:0/1320945276 >> v2:172.21.15.73:3300/0 conn(0x7f864415cd90 0x7f86441597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.384+0000 7f864a000640 1 --2- 172.21.15.46:0/1320945276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86441593d0 0x7f864406cbf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.384+0000 7f864a000640 1 -- 172.21.15.46:0/1320945276 >> 172.21.15.46:0/1320945276 conn(0x7f8644066110 msgr2=0x7f8644069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:40.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.384+0000 7f864a000640 1 -- 172.21.15.46:0/1320945276 shutdown_connections 2024-03-29T08:18:40.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.384+0000 7f864a000640 1 -- 172.21.15.46:0/1320945276 wait complete. 2024-03-29T08:18:40.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f864a000640 1 Processor -- start 2024-03-29T08:18:40.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f864a000640 1 -- start start 2024-03-29T08:18:40.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f864a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86441593d0 0x7f864414a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f864a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f864415cd90 0x7f86440701c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f864a000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f864415d230 0x7f8644070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f864a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86440cd0f0 con 0x7f86441593d0 2024-03-29T08:18:40.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f864a000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86440cc4e0 con 0x7f864415cd90 2024-03-29T08:18:40.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f864a000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86440cc7c0 con 0x7f864415d230 2024-03-29T08:18:40.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f8643400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86441593d0 0x7f864414a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f8643400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86441593d0 0x7f864414a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43282/0 (socket says 172.21.15.46:43282) 2024-03-29T08:18:40.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f8643400640 1 -- 172.21.15.46:0/1159442024 learned_addr learned my addr 172.21.15.46:0/1159442024 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:40.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f8643e00640 1 --2- 172.21.15.46:0/1159442024 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f864415d230 0x7f8644070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f863be00640 1 --2- 172.21.15.46:0/1159442024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f864415cd90 0x7f86440701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f8643400640 1 -- 172.21.15.46:0/1159442024 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f864415d230 msgr2=0x7f8644070700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f8643400640 1 --2- 172.21.15.46:0/1159442024 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f864415d230 0x7f8644070700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f8643400640 1 -- 172.21.15.46:0/1159442024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f864415cd90 msgr2=0x7f86440701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f8643400640 1 --2- 172.21.15.46:0/1159442024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f864415cd90 0x7f86440701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f8643400640 1 -- 172.21.15.46:0/1159442024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86340094d0 con 0x7f86441593d0 2024-03-29T08:18:40.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f8643e00640 1 --2- 172.21.15.46:0/1159442024 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f864415d230 0x7f8644070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:40.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f863be00640 1 --2- 172.21.15.46:0/1159442024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f864415cd90 0x7f86440701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:40.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f8643400640 1 --2- 172.21.15.46:0/1159442024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86441593d0 0x7f864414a170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f863401cd80 tx=0x7f863401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:40.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f863b400640 1 -- 172.21.15.46:0/1159442024 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86340bd5f0 con 0x7f86441593d0 2024-03-29T08:18:40.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f863b400640 1 -- 172.21.15.46:0/1159442024 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86340bd750 con 0x7f86441593d0 2024-03-29T08:18:40.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f863b400640 1 -- 172.21.15.46:0/1159442024 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86340bdac0 con 0x7f86441593d0 2024-03-29T08:18:40.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f864a000640 1 -- 172.21.15.46:0/1159442024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86440ccaa0 con 0x7f86441593d0 2024-03-29T08:18:40.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.388+0000 7f864a000640 1 -- 172.21.15.46:0/1159442024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864414a6b0 con 0x7f86441593d0 2024-03-29T08:18:40.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.392+0000 7f864a000640 1 -- 172.21.15.46:0/1159442024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86441642a0 con 0x7f86441593d0 2024-03-29T08:18:40.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.392+0000 7f863b400640 1 -- 172.21.15.46:0/1159442024 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f86340a8050 con 0x7f86441593d0 2024-03-29T08:18:40.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.392+0000 7f863b400640 1 --2- 172.21.15.46:0/1159442024 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f86200666c0 0x7f8620068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.392+0000 7f863b400640 1 -- 172.21.15.46:0/1159442024 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f863411e1c0 con 0x7f86441593d0 2024-03-29T08:18:40.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.392+0000 7f863be00640 1 --2- 172.21.15.46:0/1159442024 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f86200666c0 0x7f8620068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.392+0000 7f863be00640 1 --2- 172.21.15.46:0/1159442024 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f86200666c0 0x7f8620068b80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f86440711a0 tx=0x7f862c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.396+0000 7f863b400640 1 -- 172.21.15.46:0/1159442024 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f86340eb700 con 0x7f86441593d0 2024-03-29T08:18:40.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.560+0000 7f864a000640 1 -- 172.21.15.46:0/1159442024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86440cc900 con 0x7f86441593d0 2024-03-29T08:18:40.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.560+0000 7f863b400640 1 -- 172.21.15.46:0/1159442024 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f86440cc900 con 0x7f86441593d0 2024-03-29T08:18:40.563 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:40.563 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:14.314124+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.564+0000 7f864a000640 1 -- 172.21.15.46:0/1159442024 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f86200666c0 msgr2=0x7f8620068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.564+0000 7f864a000640 1 --2- 172.21.15.46:0/1159442024 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f86200666c0 0x7f8620068b80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f86440711a0 tx=0x7f862c009040 comp rx=0 tx=0).stop 2024-03-29T08:18:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.564+0000 7f864a000640 1 -- 172.21.15.46:0/1159442024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86441593d0 msgr2=0x7f864414a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.564+0000 7f864a000640 1 --2- 172.21.15.46:0/1159442024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86441593d0 0x7f864414a170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f863401cd80 tx=0x7f863401ad20 comp rx=0 tx=0).stop 2024-03-29T08:18:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.564+0000 7f864a000640 1 -- 172.21.15.46:0/1159442024 shutdown_connections 2024-03-29T08:18:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.564+0000 7f864a000640 1 --2- 172.21.15.46:0/1159442024 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f86200666c0 0x7f8620068b80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.564+0000 7f864a000640 1 --2- 172.21.15.46:0/1159442024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f864415cd90 0x7f86440701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.564+0000 7f864a000640 1 --2- 172.21.15.46:0/1159442024 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f864415d230 0x7f8644070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.564+0000 7f864a000640 1 --2- 172.21.15.46:0/1159442024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86441593d0 0x7f864414a170 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.564+0000 7f864a000640 1 -- 172.21.15.46:0/1159442024 >> 172.21.15.46:0/1159442024 conn(0x7f8644066110 msgr2=0x7f864415c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.564+0000 7f864a000640 1 -- 172.21.15.46:0/1159442024 shutdown_connections 2024-03-29T08:18:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.564+0000 7f864a000640 1 -- 172.21.15.46:0/1159442024 wait complete. 2024-03-29T08:18:40.585 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:18:11.177842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:18:12.308252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:18:40.585 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:18:40.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f3282a00640 1 Processor -- start 2024-03-29T08:18:40.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f3282a00640 1 -- start start 2024-03-29T08:18:40.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f3282a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f327c15dfb0 0x7f327c0b1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f3282a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f327c0cb3f0 con 0x7f327c098a40 2024-03-29T08:18:40.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f3282a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f327c0cb620 con 0x7f327c0ced10 2024-03-29T08:18:40.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f3282a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327c0b1e00 con 0x7f327c15dfb0 2024-03-29T08:18:40.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f3280a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f327c098a40 0x7f327c0cc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54018/0 (socket says 172.21.15.46:54018) 2024-03-29T08:18:40.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f3280a00640 1 -- 172.21.15.46:0/4126382307 learned_addr learned my addr 172.21.15.46:0/4126382307 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:40.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327be00640 1 --2- 172.21.15.46:0/4126382307 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f327c15dfb0 0x7f327c0b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327b400640 1 -- 172.21.15.46:0/4126382307 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748616617 0 0) 0x7f327c0cb3f0 con 0x7f327c098a40 2024-03-29T08:18:40.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327b400640 1 -- 172.21.15.46:0/4126382307 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3260003530 con 0x7f327c098a40 2024-03-29T08:18:40.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327b400640 1 -- 172.21.15.46:0/4126382307 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996295519 0 0) 0x7f327c0cb620 con 0x7f327c0ced10 2024-03-29T08:18:40.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327b400640 1 -- 172.21.15.46:0/4126382307 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f327c0cb3f0 con 0x7f327c0ced10 2024-03-29T08:18:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327b400640 1 -- 172.21.15.46:0/4126382307 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2246570557 0 0) 0x7f3260003530 con 0x7f327c098a40 2024-03-29T08:18:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327b400640 1 -- 172.21.15.46:0/4126382307 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f327c15dfb0 msgr2=0x7f327c0b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327b400640 1 --2- 172.21.15.46:0/4126382307 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f327c15dfb0 0x7f327c0b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327b400640 1 -- 172.21.15.46:0/4126382307 >> v1:172.21.15.73:6789/0 conn(0x7f327c0ced10 legacy=0x7f327c0b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327b400640 1 -- 172.21.15.46:0/4126382307 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c0cf4f0 con 0x7f327c098a40 2024-03-29T08:18:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327b400640 1 -- 172.21.15.46:0/4126382307 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f3274001a00 con 0x7f327c098a40 2024-03-29T08:18:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327b400640 1 -- 172.21.15.46:0/4126382307 >> v1:172.21.15.46:6789/0 conn(0x7f327c098a40 legacy=0x7f327c0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327b400640 1 --2- 172.21.15.46:0/4126382307 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f327c15dfb0 0x7f3260003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327b400640 1 --2- 172.21.15.46:0/4126382307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32600040f0 0x7f32600047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327b400640 1 --2- 172.21.15.46:0/4126382307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3260004d10 0x7f3260005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327b400640 1 -- 172.21.15.46:0/4126382307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c0b1e00 con 0x7f3260004d10 2024-03-29T08:18:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327b400640 1 -- 172.21.15.46:0/4126382307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326000bad0 con 0x7f32600040f0 2024-03-29T08:18:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.728+0000 7f327b400640 1 -- 172.21.15.46:0/4126382307 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326000bc90 con 0x7f327c15dfb0 2024-03-29T08:18:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.732+0000 7f327be00640 1 --2- 172.21.15.46:0/4126382307 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f327c15dfb0 0x7f3260003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.732+0000 7f3280a00640 1 --2- 172.21.15.46:0/4126382307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3260004d10 0x7f3260005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.732+0000 7f3273400640 1 --2- 172.21.15.46:0/4126382307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32600040f0 0x7f32600047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.732+0000 7f327be00640 1 -- 172.21.15.46:0/4126382307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32600040f0 msgr2=0x7f32600047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.732+0000 7f327be00640 1 --2- 172.21.15.46:0/4126382307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32600040f0 0x7f32600047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.732+0000 7f327be00640 1 -- 172.21.15.46:0/4126382307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3260004d10 msgr2=0x7f3260005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.732+0000 7f327be00640 1 --2- 172.21.15.46:0/4126382307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3260004d10 0x7f3260005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.732+0000 7f327be00640 1 -- 172.21.15.46:0/4126382307 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3260003530 con 0x7f327c15dfb0 2024-03-29T08:18:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.732+0000 7f3280a00640 1 --2- 172.21.15.46:0/4126382307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3260004d10 0x7f3260005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:40.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.732+0000 7f327be00640 1 --2- 172.21.15.46:0/4126382307 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f327c15dfb0 0x7f3260003b40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f326c009ae0 tx=0x7f326c01c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=530e362847d68f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:40.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.732+0000 7f327b400640 1 -- 172.21.15.46:0/4126382307 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c0a64e0 con 0x7f327c15dfb0 2024-03-29T08:18:40.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.732+0000 7f327b400640 1 -- 172.21.15.46:0/4126382307 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326c0a6640 con 0x7f327c15dfb0 2024-03-29T08:18:40.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.732+0000 7f327b400640 1 -- 172.21.15.46:0/4126382307 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c0a69f0 con 0x7f327c15dfb0 2024-03-29T08:18:40.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3282a00640 1 -- 172.21.15.46:0/4126382307 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f327c15dfb0 msgr2=0x7f3260003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3282a00640 1 --2- 172.21.15.46:0/4126382307 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f327c15dfb0 0x7f3260003b40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f326c009ae0 tx=0x7f326c01c9f0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3280a00640 1 -- 172.21.15.46:0/4126382307 reap_dead start 2024-03-29T08:18:40.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3282a00640 1 -- 172.21.15.46:0/4126382307 shutdown_connections 2024-03-29T08:18:40.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3282a00640 1 -- 172.21.15.46:0/4126382307 >> 172.21.15.46:0/4126382307 conn(0x7f327c0b2220 msgr2=0x7f327c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:40.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3282a00640 1 -- 172.21.15.46:0/4126382307 shutdown_connections 2024-03-29T08:18:40.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3282a00640 1 -- 172.21.15.46:0/4126382307 wait complete. 2024-03-29T08:18:40.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3282a00640 1 Processor -- start 2024-03-29T08:18:40.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3282a00640 1 -- start start 2024-03-29T08:18:40.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3282a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f327c15dfb0 0x7f327c15d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3282a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f327c1587d0 0x7f327c157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3282a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f327c0b4fc0 0x7f327c0b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3282a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c0b5980 con 0x7f327c0b4fc0 2024-03-29T08:18:40.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3282a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c15a790 con 0x7f327c15dfb0 2024-03-29T08:18:40.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3282a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327c15a1d0 con 0x7f327c1587d0 2024-03-29T08:18:40.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3280a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f327c0b4fc0 0x7f327c0b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3280a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f327c0b4fc0 0x7f327c0b5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43294/0 (socket says 172.21.15.46:43294) 2024-03-29T08:18:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3280a00640 1 -- 172.21.15.46:0/2891380430 learned_addr learned my addr 172.21.15.46:0/2891380430 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f327be00640 1 --2- 172.21.15.46:0/2891380430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f327c15dfb0 0x7f327c15d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3273400640 1 --2- 172.21.15.46:0/2891380430 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f327c1587d0 0x7f327c157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3280a00640 1 -- 172.21.15.46:0/2891380430 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f327c1587d0 msgr2=0x7f327c157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3280a00640 1 --2- 172.21.15.46:0/2891380430 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f327c1587d0 0x7f327c157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3280a00640 1 -- 172.21.15.46:0/2891380430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f327c15dfb0 msgr2=0x7f327c15d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3280a00640 1 --2- 172.21.15.46:0/2891380430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f327c15dfb0 0x7f327c15d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.736+0000 7f3280a00640 1 -- 172.21.15.46:0/2891380430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c15a4b0 con 0x7f327c0b4fc0 2024-03-29T08:18:40.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.740+0000 7f3280a00640 1 --2- 172.21.15.46:0/2891380430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f327c0b4fc0 0x7f327c0b5440 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3274004c00 tx=0x7f3274012920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.740+0000 7f3278c00640 1 -- 172.21.15.46:0/2891380430 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327401b070 con 0x7f327c0b4fc0 2024-03-29T08:18:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.740+0000 7f3278c00640 1 -- 172.21.15.46:0/2891380430 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3274016440 con 0x7f327c0b4fc0 2024-03-29T08:18:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.740+0000 7f3282a00640 1 -- 172.21.15.46:0/2891380430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327c15abf0 con 0x7f327c0b4fc0 2024-03-29T08:18:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.740+0000 7f3278c00640 1 -- 172.21.15.46:0/2891380430 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32740167b0 con 0x7f327c0b4fc0 2024-03-29T08:18:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.740+0000 7f3282a00640 1 -- 172.21.15.46:0/2891380430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327c15d790 con 0x7f327c0b4fc0 2024-03-29T08:18:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.740+0000 7f3282a00640 1 -- 172.21.15.46:0/2891380430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f327c09e700 con 0x7f327c0b4fc0 2024-03-29T08:18:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.740+0000 7f3278c00640 1 -- 172.21.15.46:0/2891380430 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3274016910 con 0x7f327c0b4fc0 2024-03-29T08:18:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.740+0000 7f3278c00640 1 --2- 172.21.15.46:0/2891380430 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3258066690 0x7f3258068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.740+0000 7f3278c00640 1 -- 172.21.15.46:0/2891380430 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3274087e10 con 0x7f327c0b4fc0 2024-03-29T08:18:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.740+0000 7f327be00640 1 --2- 172.21.15.46:0/2891380430 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3258066690 0x7f3258068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:40.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.744+0000 7f327be00640 1 --2- 172.21.15.46:0/2891380430 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3258066690 0x7f3258068b50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f326c0af000 tx=0x7f326c0046d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:40.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.744+0000 7f3278c00640 1 -- 172.21.15.46:0/2891380430 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f327408b030 con 0x7f327c0b4fc0 2024-03-29T08:18:40.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.912+0000 7f3282a00640 1 -- 172.21.15.46:0/2891380430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f327c04e120 con 0x7f327c0b4fc0 2024-03-29T08:18:40.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.912+0000 7f3278c00640 1 -- 172.21.15.46:0/2891380430 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3274055220 con 0x7f327c0b4fc0 2024-03-29T08:18:40.915 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:18:40.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.916+0000 7f3282a00640 1 -- 172.21.15.46:0/2891380430 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3258066690 msgr2=0x7f3258068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.916+0000 7f3282a00640 1 --2- 172.21.15.46:0/2891380430 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3258066690 0x7f3258068b50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f326c0af000 tx=0x7f326c0046d0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.916+0000 7f3282a00640 1 -- 172.21.15.46:0/2891380430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f327c0b4fc0 msgr2=0x7f327c0b5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:40.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.916+0000 7f3282a00640 1 --2- 172.21.15.46:0/2891380430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f327c0b4fc0 0x7f327c0b5440 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3274004c00 tx=0x7f3274012920 comp rx=0 tx=0).stop 2024-03-29T08:18:40.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.916+0000 7f3282a00640 1 -- 172.21.15.46:0/2891380430 shutdown_connections 2024-03-29T08:18:40.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.916+0000 7f3282a00640 1 --2- 172.21.15.46:0/2891380430 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3258066690 0x7f3258068b50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.916+0000 7f3282a00640 1 --2- 172.21.15.46:0/2891380430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f327c0b4fc0 0x7f327c0b5440 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.916+0000 7f3282a00640 1 --2- 172.21.15.46:0/2891380430 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f327c1587d0 0x7f327c157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.916+0000 7f3282a00640 1 --2- 172.21.15.46:0/2891380430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f327c15dfb0 0x7f327c15d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:40.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.916+0000 7f3282a00640 1 -- 172.21.15.46:0/2891380430 >> 172.21.15.46:0/2891380430 conn(0x7f327c0b2220 msgr2=0x7f327c06fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:40.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.916+0000 7f3282a00640 1 -- 172.21.15.46:0/2891380430 shutdown_connections 2024-03-29T08:18:40.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:40.916+0000 7f3282a00640 1 -- 172.21.15.46:0/2891380430 wait complete. 2024-03-29T08:18:40.936 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:18:41.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.080+0000 7f7c8b200640 1 Processor -- start 2024-03-29T08:18:41.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.080+0000 7f7c8b200640 1 -- start start 2024-03-29T08:18:41.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.084+0000 7f7c8b200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c840cc000 0x7f7c8406b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.084+0000 7f7c8b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7c840af710 0x7f7c84167280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.084+0000 7f7c8b200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c8415d040 con 0x7f7c840cd190 2024-03-29T08:18:41.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.084+0000 7f7c8b200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c8406c550 con 0x7f7c840af710 2024-03-29T08:18:41.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.084+0000 7f7c8b200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840caba0 con 0x7f7c840cc000 2024-03-29T08:18:41.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.084+0000 7f7c8a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7c840af710 0x7f7c84167280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:41.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.084+0000 7f7c8a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7c840af710 0x7f7c84167280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43304/0 (socket says 172.21.15.46:43304) 2024-03-29T08:18:41.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.084+0000 7f7c8a800640 1 -- 172.21.15.46:0/2501849803 learned_addr learned my addr 172.21.15.46:0/2501849803 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:41.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.084+0000 7f7c89e00640 1 --1- 172.21.15.46:0/2501849803 >> v1:172.21.15.46:6789/0 conn(0x7f7c840cd190 0x7f7c8406b240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54024/0 (socket says 172.21.15.46:54024) 2024-03-29T08:18:41.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.084+0000 7f7c89400640 1 --2- 172.21.15.46:0/2501849803 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c840cc000 0x7f7c8406b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:41.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.084+0000 7f7c8a800640 1 -- 172.21.15.46:0/2501849803 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c840cc000 msgr2=0x7f7c8406b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:41.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.084+0000 7f7c8a800640 1 --2- 172.21.15.46:0/2501849803 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c840cc000 0x7f7c8406b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.084+0000 7f7c8a800640 1 -- 172.21.15.46:0/2501849803 >> v1:172.21.15.46:6789/0 conn(0x7f7c840cd190 legacy=0x7f7c8406b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.084+0000 7f7c8a800640 1 -- 172.21.15.46:0/2501849803 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c840cf800 con 0x7f7c840af710 2024-03-29T08:18:41.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.084+0000 7f7c8a800640 1 --2- 172.21.15.46:0/2501849803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c840af710 0x7f7c84167280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7c80009e50 tx=0x7f7c8001f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fa9d4c38f00639 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:41.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c88a00640 1 -- 172.21.15.46:0/2501849803 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c800af400 con 0x7f7c840af710 2024-03-29T08:18:41.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c88a00640 1 -- 172.21.15.46:0/2501849803 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c800ab050 con 0x7f7c840af710 2024-03-29T08:18:41.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c88a00640 1 -- 172.21.15.46:0/2501849803 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c800af780 con 0x7f7c840af710 2024-03-29T08:18:41.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c8b200640 1 -- 172.21.15.46:0/2501849803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c840af710 msgr2=0x7f7c84167280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c8b200640 1 --2- 172.21.15.46:0/2501849803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c840af710 0x7f7c84167280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7c80009e50 tx=0x7f7c8001f210 comp rx=0 tx=0).stop 2024-03-29T08:18:41.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c8b200640 1 -- 172.21.15.46:0/2501849803 shutdown_connections 2024-03-29T08:18:41.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c8b200640 1 --2- 172.21.15.46:0/2501849803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c840af710 0x7f7c84167280 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c8b200640 1 --2- 172.21.15.46:0/2501849803 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c840cc000 0x7f7c8406b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c8b200640 1 -- 172.21.15.46:0/2501849803 >> 172.21.15.46:0/2501849803 conn(0x7f7c840d0550 msgr2=0x7f7c840d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:41.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c8b200640 1 -- 172.21.15.46:0/2501849803 shutdown_connections 2024-03-29T08:18:41.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c8b200640 1 -- 172.21.15.46:0/2501849803 wait complete. 2024-03-29T08:18:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c8b200640 1 Processor -- start 2024-03-29T08:18:41.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c8b200640 1 -- start start 2024-03-29T08:18:41.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c8b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c840af710 0x7f7c840c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c8b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c840cc000 0x7f7c840c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c8b200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c840cd190 0x7f7c841590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c8b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8409a4c0 con 0x7f7c840af710 2024-03-29T08:18:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c8b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840c7300 con 0x7f7c840cc000 2024-03-29T08:18:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c8b200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c840c4e00 con 0x7f7c840cd190 2024-03-29T08:18:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c89e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c840af710 0x7f7c840c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c89e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c840af710 0x7f7c840c5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43316/0 (socket says 172.21.15.46:43316) 2024-03-29T08:18:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.088+0000 7f7c89e00640 1 -- 172.21.15.46:0/1367555009 learned_addr learned my addr 172.21.15.46:0/1367555009 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c89400640 1 --2- 172.21.15.46:0/1367555009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c840cc000 0x7f7c840c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c8a800640 1 --2- 172.21.15.46:0/1367555009 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c840cd190 0x7f7c841590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c89e00640 1 -- 172.21.15.46:0/1367555009 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c840cd190 msgr2=0x7f7c841590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c89e00640 1 --2- 172.21.15.46:0/1367555009 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c840cd190 0x7f7c841590b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c89e00640 1 -- 172.21.15.46:0/1367555009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c840cc000 msgr2=0x7f7c840c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c89e00640 1 --2- 172.21.15.46:0/1367555009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c840cc000 0x7f7c840c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c89e00640 1 -- 172.21.15.46:0/1367555009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c80009b30 con 0x7f7c840af710 2024-03-29T08:18:41.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c89400640 1 --2- 172.21.15.46:0/1367555009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c840cc000 0x7f7c840c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c8a800640 1 --2- 172.21.15.46:0/1367555009 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c840cd190 0x7f7c841590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c89e00640 1 --2- 172.21.15.46:0/1367555009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c840af710 0x7f7c840c5bc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800c490 tx=0x7f7c7800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c7e000640 1 -- 172.21.15.46:0/1367555009 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78007480 con 0x7f7c840af710 2024-03-29T08:18:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c7e000640 1 -- 172.21.15.46:0/1367555009 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c780075e0 con 0x7f7c840af710 2024-03-29T08:18:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c7e000640 1 -- 172.21.15.46:0/1367555009 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78004a10 con 0x7f7c840af710 2024-03-29T08:18:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c8b200640 1 -- 172.21.15.46:0/1367555009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c840c50e0 con 0x7f7c840af710 2024-03-29T08:18:41.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c8b200640 1 -- 172.21.15.46:0/1367555009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c840c56a0 con 0x7f7c840af710 2024-03-29T08:18:41.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c8b200640 1 -- 172.21.15.46:0/1367555009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c840c58c0 con 0x7f7c840af710 2024-03-29T08:18:41.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c7e000640 1 -- 172.21.15.46:0/1367555009 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7c78004b70 con 0x7f7c840af710 2024-03-29T08:18:41.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c7e000640 1 --2- 172.21.15.46:0/1367555009 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c680666c0 0x7f7c68068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c7e000640 1 -- 172.21.15.46:0/1367555009 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c78083230 con 0x7f7c840af710 2024-03-29T08:18:41.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.092+0000 7f7c89400640 1 --2- 172.21.15.46:0/1367555009 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c680666c0 0x7f7c68068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:41.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.096+0000 7f7c89400640 1 --2- 172.21.15.46:0/1367555009 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c680666c0 0x7f7c68068b80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7c840cc630 tx=0x7f7c74008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:41.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.096+0000 7f7c7e000640 1 -- 172.21.15.46:0/1367555009 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7c78050590 con 0x7f7c840af710 2024-03-29T08:18:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.256+0000 7f7c8b200640 1 -- 172.21.15.46:0/1367555009 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7c8404e040 con 0x7f7c840af710 2024-03-29T08:18:41.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.256+0000 7f7c7e000640 1 -- 172.21.15.46:0/1367555009 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7c78053bb0 con 0x7f7c840af710 2024-03-29T08:18:41.259 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:18:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.260+0000 7f7c8b200640 1 -- 172.21.15.46:0/1367555009 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c680666c0 msgr2=0x7f7c68068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.260+0000 7f7c8b200640 1 --2- 172.21.15.46:0/1367555009 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c680666c0 0x7f7c68068b80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7c840cc630 tx=0x7f7c74008040 comp rx=0 tx=0).stop 2024-03-29T08:18:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.260+0000 7f7c8b200640 1 -- 172.21.15.46:0/1367555009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c840af710 msgr2=0x7f7c840c5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.260+0000 7f7c8b200640 1 --2- 172.21.15.46:0/1367555009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c840af710 0x7f7c840c5bc0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800c490 tx=0x7f7c7800c950 comp rx=0 tx=0).stop 2024-03-29T08:18:41.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.260+0000 7f7c8b200640 1 -- 172.21.15.46:0/1367555009 shutdown_connections 2024-03-29T08:18:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.260+0000 7f7c8b200640 1 --2- 172.21.15.46:0/1367555009 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c680666c0 0x7f7c68068b80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.260+0000 7f7c8b200640 1 --2- 172.21.15.46:0/1367555009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c840cc000 0x7f7c840c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.260+0000 7f7c8b200640 1 --2- 172.21.15.46:0/1367555009 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c840cd190 0x7f7c841590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.260+0000 7f7c8b200640 1 --2- 172.21.15.46:0/1367555009 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c840af710 0x7f7c840c5bc0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.260+0000 7f7c8b200640 1 -- 172.21.15.46:0/1367555009 >> 172.21.15.46:0/1367555009 conn(0x7f7c840d0550 msgr2=0x7f7c8406b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.260+0000 7f7c8b200640 1 -- 172.21.15.46:0/1367555009 shutdown_connections 2024-03-29T08:18:41.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.260+0000 7f7c8b200640 1 -- 172.21.15.46:0/1367555009 wait complete. 2024-03-29T08:18:41.280 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:18:41.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.420+0000 7f1924e00640 1 Processor -- start 2024-03-29T08:18:41.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.420+0000 7f1924e00640 1 -- start start 2024-03-29T08:18:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.424+0000 7f1924e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1920142a50 0x7f19200cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.424+0000 7f1924e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f192015df90 0x7f19200cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.424+0000 7f1924e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f192015d260 con 0x7f19200ae280 2024-03-29T08:18:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.424+0000 7f1924e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19200aefd0 con 0x7f1920142a50 2024-03-29T08:18:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.424+0000 7f1924e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192015d4b0 con 0x7f192015df90 2024-03-29T08:18:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.424+0000 7f191de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1920142a50 0x7f19200cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.424+0000 7f191e800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f19200ae280 0x7f19200ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54034/0 (socket says 172.21.15.46:54034) 2024-03-29T08:18:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.424+0000 7f191e800640 1 -- 172.21.15.46:0/4140954237 learned_addr learned my addr 172.21.15.46:0/4140954237 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.424+0000 7f191de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1920142a50 0x7f19200cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43332/0 (socket says 172.21.15.46:43332) 2024-03-29T08:18:41.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.424+0000 7f1915400640 1 --2- 172.21.15.46:0/4140954237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f192015df90 0x7f19200cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:41.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.424+0000 7f191de00640 1 -- 172.21.15.46:0/4140954237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f192015df90 msgr2=0x7f19200cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:41.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.424+0000 7f191de00640 1 --2- 172.21.15.46:0/4140954237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f192015df90 0x7f19200cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.424+0000 7f191de00640 1 -- 172.21.15.46:0/4140954237 >> v1:172.21.15.46:6789/0 conn(0x7f19200ae280 legacy=0x7f19200ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.424+0000 7f191de00640 1 -- 172.21.15.46:0/4140954237 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192015d790 con 0x7f1920142a50 2024-03-29T08:18:41.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.424+0000 7f191d400640 1 -- 172.21.15.46:0/4140954237 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5264548 0 0) 0x7f192015d260 con 0x7f19200ae280 2024-03-29T08:18:41.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f191de00640 1 --2- 172.21.15.46:0/4140954237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1920142a50 0x7f19200cb790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f19100097c0 tx=0x7f191001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7254178616aed9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:41.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f191d400640 1 -- 172.21.15.46:0/4140954237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19100a4ea0 con 0x7f1920142a50 2024-03-29T08:18:41.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f191d400640 1 -- 172.21.15.46:0/4140954237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19100a7050 con 0x7f1920142a50 2024-03-29T08:18:41.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f191d400640 1 -- 172.21.15.46:0/4140954237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19100ab820 con 0x7f1920142a50 2024-03-29T08:18:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f1924e00640 1 -- 172.21.15.46:0/4140954237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1920142a50 msgr2=0x7f19200cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f1924e00640 1 --2- 172.21.15.46:0/4140954237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1920142a50 0x7f19200cb790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f19100097c0 tx=0x7f191001c410 comp rx=0 tx=0).stop 2024-03-29T08:18:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f1924e00640 1 -- 172.21.15.46:0/4140954237 shutdown_connections 2024-03-29T08:18:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f1924e00640 1 --2- 172.21.15.46:0/4140954237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f192015df90 0x7f19200cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f1924e00640 1 --2- 172.21.15.46:0/4140954237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1920142a50 0x7f19200cb790 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f1924e00640 1 -- 172.21.15.46:0/4140954237 >> 172.21.15.46:0/4140954237 conn(0x7f192006a390 msgr2=0x7f19200cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f1924e00640 1 -- 172.21.15.46:0/4140954237 shutdown_connections 2024-03-29T08:18:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f1924e00640 1 -- 172.21.15.46:0/4140954237 wait complete. 2024-03-29T08:18:41.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f1924e00640 1 Processor -- start 2024-03-29T08:18:41.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f1924e00640 1 -- start start 2024-03-29T08:18:41.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f1924e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19200ae280 0x7f1920139000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f1924e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1920142a50 0x7f192013bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f1924e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f192015df90 0x7f192013c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f1924e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192013b0b0 con 0x7f19200ae280 2024-03-29T08:18:41.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f1924e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192013a4a0 con 0x7f192015df90 2024-03-29T08:18:41.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.428+0000 7f1924e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192013a780 con 0x7f1920142a50 2024-03-29T08:18:41.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f191de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19200ae280 0x7f1920139000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:41.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f191de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19200ae280 0x7f1920139000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43346/0 (socket says 172.21.15.46:43346) 2024-03-29T08:18:41.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f191de00640 1 -- 172.21.15.46:0/807473433 learned_addr learned my addr 172.21.15.46:0/807473433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:41.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f191de00640 1 -- 172.21.15.46:0/807473433 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1920142a50 msgr2=0x7f192013bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f191de00640 1 --2- 172.21.15.46:0/807473433 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1920142a50 0x7f192013bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f191de00640 1 -- 172.21.15.46:0/807473433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f192015df90 msgr2=0x7f192013c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f191e800640 1 --2- 172.21.15.46:0/807473433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f192015df90 0x7f192013c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:41.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f191de00640 1 --2- 172.21.15.46:0/807473433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f192015df90 0x7f192013c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f191de00640 1 -- 172.21.15.46:0/807473433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1910009420 con 0x7f19200ae280 2024-03-29T08:18:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f191e800640 1 --2- 172.21.15.46:0/807473433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f192015df90 0x7f192013c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f191de00640 1 --2- 172.21.15.46:0/807473433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19200ae280 0x7f1920139000 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f19100ae040 tx=0x7f191001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f1916a00640 1 -- 172.21.15.46:0/807473433 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19100b7860 con 0x7f19200ae280 2024-03-29T08:18:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f1916a00640 1 -- 172.21.15.46:0/807473433 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19100b79e0 con 0x7f19200ae280 2024-03-29T08:18:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f1916a00640 1 -- 172.21.15.46:0/807473433 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19100b7d90 con 0x7f19200ae280 2024-03-29T08:18:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f1924e00640 1 -- 172.21.15.46:0/807473433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f192013aa60 con 0x7f19200ae280 2024-03-29T08:18:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f1924e00640 1 -- 172.21.15.46:0/807473433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1920139540 con 0x7f19200ae280 2024-03-29T08:18:41.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f1924e00640 1 -- 172.21.15.46:0/807473433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f192006ad60 con 0x7f19200ae280 2024-03-29T08:18:41.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.432+0000 7f1916a00640 1 -- 172.21.15.46:0/807473433 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f19100b5400 con 0x7f19200ae280 2024-03-29T08:18:41.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.436+0000 7f1916a00640 1 --2- 172.21.15.46:0/807473433 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18fc066690 0x7f18fc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.436+0000 7f1916a00640 1 -- 172.21.15.46:0/807473433 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f191011eb40 con 0x7f19200ae280 2024-03-29T08:18:41.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.436+0000 7f1915400640 1 --2- 172.21.15.46:0/807473433 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18fc066690 0x7f18fc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:41.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.436+0000 7f1915400640 1 --2- 172.21.15.46:0/807473433 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18fc066690 0x7f18fc068b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1920159220 tx=0x7f1908008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:41.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.440+0000 7f1916a00640 1 -- 172.21.15.46:0/807473433 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f19100ebeb0 con 0x7f19200ae280 2024-03-29T08:18:41.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.600+0000 7f1924e00640 1 -- 172.21.15.46:0/807473433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1920071a70 con 0x7f19200ae280 2024-03-29T08:18:41.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.600+0000 7f1916a00640 1 -- 172.21.15.46:0/807473433 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f19100b00a0 con 0x7f19200ae280 2024-03-29T08:18:41.601 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:18:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.600+0000 7f1924e00640 1 -- 172.21.15.46:0/807473433 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18fc066690 msgr2=0x7f18fc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.600+0000 7f1924e00640 1 --2- 172.21.15.46:0/807473433 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18fc066690 0x7f18fc068b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1920159220 tx=0x7f1908008040 comp rx=0 tx=0).stop 2024-03-29T08:18:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.600+0000 7f1924e00640 1 -- 172.21.15.46:0/807473433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19200ae280 msgr2=0x7f1920139000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.600+0000 7f1924e00640 1 --2- 172.21.15.46:0/807473433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19200ae280 0x7f1920139000 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f19100ae040 tx=0x7f191001ac50 comp rx=0 tx=0).stop 2024-03-29T08:18:41.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.600+0000 7f1924e00640 1 -- 172.21.15.46:0/807473433 shutdown_connections 2024-03-29T08:18:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.600+0000 7f1924e00640 1 --2- 172.21.15.46:0/807473433 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18fc066690 0x7f18fc068b50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.600+0000 7f1924e00640 1 --2- 172.21.15.46:0/807473433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f192015df90 0x7f192013c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.600+0000 7f1924e00640 1 --2- 172.21.15.46:0/807473433 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1920142a50 0x7f192013bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.600+0000 7f1924e00640 1 --2- 172.21.15.46:0/807473433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19200ae280 0x7f1920139000 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.600+0000 7f1924e00640 1 -- 172.21.15.46:0/807473433 >> 172.21.15.46:0/807473433 conn(0x7f192006a390 msgr2=0x7f19200cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.600+0000 7f1924e00640 1 -- 172.21.15.46:0/807473433 shutdown_connections 2024-03-29T08:18:41.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.600+0000 7f1924e00640 1 -- 172.21.15.46:0/807473433 wait complete. 2024-03-29T08:18:41.623 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:41.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.760+0000 7f5d57a00640 1 Processor -- start 2024-03-29T08:18:41.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.760+0000 7f5d57a00640 1 -- start start 2024-03-29T08:18:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.760+0000 7f5d57a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d500ae810 0x7f5d500caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.760+0000 7f5d57a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d500cb780 0x7f5d500aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.760+0000 7f5d57a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d500cbc10 con 0x7f5d500ce8a0 2024-03-29T08:18:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.760+0000 7f5d57a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d500c9bd0 con 0x7f5d500ae810 2024-03-29T08:18:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.760+0000 7f5d57a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d500c9eb0 con 0x7f5d500cb780 2024-03-29T08:18:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.760+0000 7f5d4fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d500ae810 0x7f5d500caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.760+0000 7f5d4fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d500ae810 0x7f5d500caf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43348/0 (socket says 172.21.15.46:43348) 2024-03-29T08:18:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.760+0000 7f5d4fe00640 1 -- 172.21.15.46:0/1357978985 learned_addr learned my addr 172.21.15.46:0/1357978985 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.760+0000 7f5d55a00640 1 --2- 172.21.15.46:0/1357978985 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d500cb780 0x7f5d500aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.764+0000 7f5d4fe00640 1 -- 172.21.15.46:0/1357978985 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d500cb780 msgr2=0x7f5d500aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.764+0000 7f5d4fe00640 1 --2- 172.21.15.46:0/1357978985 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d500cb780 0x7f5d500aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.764+0000 7f5d4fe00640 1 -- 172.21.15.46:0/1357978985 >> v1:172.21.15.73:6789/0 conn(0x7f5d500ce8a0 legacy=0x7f5d500cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.764+0000 7f5d4fe00640 1 -- 172.21.15.46:0/1357978985 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d500ca190 con 0x7f5d500ae810 2024-03-29T08:18:41.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.764+0000 7f5d4fe00640 1 --2- 172.21.15.46:0/1357978985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d500ae810 0x7f5d500caf50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009870 tx=0x7f5d3c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5cdb8a03316d5c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:41.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.764+0000 7f5d4f400640 1 -- 172.21.15.46:0/1357978985 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d500afa20 con 0x7f5d500ae810 2024-03-29T08:18:41.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.764+0000 7f5d4f400640 1 -- 172.21.15.46:0/1357978985 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c0a95f0 con 0x7f5d500ae810 2024-03-29T08:18:41.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.764+0000 7f5d4f400640 1 -- 172.21.15.46:0/1357978985 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c0a9980 con 0x7f5d500ae810 2024-03-29T08:18:41.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.764+0000 7f5d57a00640 1 -- 172.21.15.46:0/1357978985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d500ae810 msgr2=0x7f5d500caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.764+0000 7f5d57a00640 1 --2- 172.21.15.46:0/1357978985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d500ae810 0x7f5d500caf50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009870 tx=0x7f5d3c01c530 comp rx=0 tx=0).stop 2024-03-29T08:18:41.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d57a00640 1 -- 172.21.15.46:0/1357978985 shutdown_connections 2024-03-29T08:18:41.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d57a00640 1 --2- 172.21.15.46:0/1357978985 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d500cb780 0x7f5d500aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d57a00640 1 --2- 172.21.15.46:0/1357978985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d500ae810 0x7f5d500caf50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d57a00640 1 -- 172.21.15.46:0/1357978985 >> 172.21.15.46:0/1357978985 conn(0x7f5d50098a40 msgr2=0x7f5d500cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:41.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d57a00640 1 -- 172.21.15.46:0/1357978985 shutdown_connections 2024-03-29T08:18:41.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d57a00640 1 -- 172.21.15.46:0/1357978985 wait complete. 2024-03-29T08:18:41.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d57a00640 1 Processor -- start 2024-03-29T08:18:41.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d57a00640 1 -- start start 2024-03-29T08:18:41.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d57a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d500ae810 0x7f5d5006e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d57a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d500cb780 0x7f5d5014ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d57a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d500ce8a0 0x7f5d5006d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d57a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d50150510 con 0x7f5d500ce8a0 2024-03-29T08:18:41.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d57a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d501f7fa0 con 0x7f5d500ae810 2024-03-29T08:18:41.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d57a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d501f8280 con 0x7f5d500cb780 2024-03-29T08:18:41.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d55a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d500ce8a0 0x7f5d5006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:41.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d55a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d500ce8a0 0x7f5d5006d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43356/0 (socket says 172.21.15.46:43356) 2024-03-29T08:18:41.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d55a00640 1 -- 172.21.15.46:0/2112691311 learned_addr learned my addr 172.21.15.46:0/2112691311 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:41.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d4fe00640 1 --2- 172.21.15.46:0/2112691311 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d500cb780 0x7f5d5014ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:41.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d55000640 1 --2- 172.21.15.46:0/2112691311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d500ae810 0x7f5d5006e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:41.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d55a00640 1 -- 172.21.15.46:0/2112691311 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d500cb780 msgr2=0x7f5d5014ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d55a00640 1 --2- 172.21.15.46:0/2112691311 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d500cb780 0x7f5d5014ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d55a00640 1 -- 172.21.15.46:0/2112691311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d500ae810 msgr2=0x7f5d5006e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d55a00640 1 --2- 172.21.15.46:0/2112691311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d500ae810 0x7f5d5006e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d55a00640 1 -- 172.21.15.46:0/2112691311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c0094d0 con 0x7f5d500ce8a0 2024-03-29T08:18:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.768+0000 7f5d55a00640 1 --2- 172.21.15.46:0/2112691311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d500ce8a0 0x7f5d5006d320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800efc0 tx=0x7f5d4800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.772+0000 7f5d4cc00640 1 -- 172.21.15.46:0/2112691311 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48014480 con 0x7f5d500ce8a0 2024-03-29T08:18:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.772+0000 7f5d4cc00640 1 -- 172.21.15.46:0/2112691311 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4800f040 con 0x7f5d500ce8a0 2024-03-29T08:18:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.772+0000 7f5d4cc00640 1 -- 172.21.15.46:0/2112691311 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d48014820 con 0x7f5d500ce8a0 2024-03-29T08:18:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.772+0000 7f5d57a00640 1 -- 172.21.15.46:0/2112691311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d501f8560 con 0x7f5d500ce8a0 2024-03-29T08:18:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.772+0000 7f5d57a00640 1 -- 172.21.15.46:0/2112691311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d501f8b20 con 0x7f5d500ce8a0 2024-03-29T08:18:41.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.772+0000 7f5d57a00640 1 -- 172.21.15.46:0/2112691311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d50066c90 con 0x7f5d500ce8a0 2024-03-29T08:18:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.772+0000 7f5d4cc00640 1 -- 172.21.15.46:0/2112691311 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5d48014980 con 0x7f5d500ce8a0 2024-03-29T08:18:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.772+0000 7f5d4cc00640 1 --2- 172.21.15.46:0/2112691311 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d30066690 0x7f5d30068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.772+0000 7f5d4cc00640 1 -- 172.21.15.46:0/2112691311 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d48084c60 con 0x7f5d500ce8a0 2024-03-29T08:18:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.772+0000 7f5d55000640 1 --2- 172.21.15.46:0/2112691311 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d30066690 0x7f5d30068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.776+0000 7f5d55000640 1 --2- 172.21.15.46:0/2112691311 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d30066690 0x7f5d30068b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d500b5610 tx=0x7f5d44009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:41.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.776+0000 7f5d4cc00640 1 -- 172.21.15.46:0/2112691311 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5d48051fe0 con 0x7f5d500ce8a0 2024-03-29T08:18:41.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.936+0000 7f5d57a00640 1 -- 172.21.15.46:0/2112691311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d5006c240 con 0x7f5d500ce8a0 2024-03-29T08:18:41.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.940+0000 7f5d4cc00640 1 -- 172.21.15.46:0/2112691311 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f5d48018090 con 0x7f5d500ce8a0 2024-03-29T08:18:41.941 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:41.941 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:14.314124+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.940+0000 7f5d57a00640 1 -- 172.21.15.46:0/2112691311 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d30066690 msgr2=0x7f5d30068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.940+0000 7f5d57a00640 1 --2- 172.21.15.46:0/2112691311 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d30066690 0x7f5d30068b50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5d500b5610 tx=0x7f5d44009040 comp rx=0 tx=0).stop 2024-03-29T08:18:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.940+0000 7f5d57a00640 1 -- 172.21.15.46:0/2112691311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d500ce8a0 msgr2=0x7f5d5006d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.940+0000 7f5d57a00640 1 --2- 172.21.15.46:0/2112691311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d500ce8a0 0x7f5d5006d320 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5d4800efc0 tx=0x7f5d4800c490 comp rx=0 tx=0).stop 2024-03-29T08:18:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.940+0000 7f5d57a00640 1 -- 172.21.15.46:0/2112691311 shutdown_connections 2024-03-29T08:18:41.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.940+0000 7f5d57a00640 1 --2- 172.21.15.46:0/2112691311 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d30066690 0x7f5d30068b50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.940+0000 7f5d57a00640 1 --2- 172.21.15.46:0/2112691311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d500ce8a0 0x7f5d5006d320 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.940+0000 7f5d57a00640 1 --2- 172.21.15.46:0/2112691311 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d500cb780 0x7f5d5014ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.940+0000 7f5d57a00640 1 --2- 172.21.15.46:0/2112691311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d500ae810 0x7f5d5006e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.940+0000 7f5d57a00640 1 -- 172.21.15.46:0/2112691311 >> 172.21.15.46:0/2112691311 conn(0x7f5d50098a40 msgr2=0x7f5d50156ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.940+0000 7f5d57a00640 1 -- 172.21.15.46:0/2112691311 shutdown_connections 2024-03-29T08:18:41.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:41.940+0000 7f5d57a00640 1 -- 172.21.15.46:0/2112691311 wait complete. 2024-03-29T08:18:41.963 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:18:11.177842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:18:12.308252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:18:41.963 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:18:42.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.108+0000 7f3b1f600640 1 Processor -- start 2024-03-29T08:18:42.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.108+0000 7f3b1f600640 1 -- start start 2024-03-29T08:18:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.108+0000 7f3b1f600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3b1815c950 0x7f3b1806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.108+0000 7f3b1f600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3b180cea30 0x7f3b181606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.108+0000 7f3b1f600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b180cfb20 con 0x7f3b1815cd30 2024-03-29T08:18:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.108+0000 7f3b1f600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b180cef10 con 0x7f3b180cea30 2024-03-29T08:18:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.108+0000 7f3b1f600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b180cf1f0 con 0x7f3b1815c950 2024-03-29T08:18:42.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.108+0000 7f3b0fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3b180cea30 0x7f3b181606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.108+0000 7f3b0fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3b180cea30 0x7f3b181606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43370/0 (socket says 172.21.15.46:43370) 2024-03-29T08:18:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.108+0000 7f3b0fe00640 1 -- 172.21.15.46:0/2903946048 learned_addr learned my addr 172.21.15.46:0/2903946048 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.108+0000 7f3b1cc00640 1 --2- 172.21.15.46:0/2903946048 >> v2:172.21.15.73:3300/0 conn(0x7f3b1815c950 0x7f3b1806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.108+0000 7f3b0fe00640 1 -- 172.21.15.46:0/2903946048 >> v2:172.21.15.73:3300/0 conn(0x7f3b1815c950 msgr2=0x7f3b1806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.108+0000 7f3b0fe00640 1 --2- 172.21.15.46:0/2903946048 >> v2:172.21.15.73:3300/0 conn(0x7f3b1815c950 0x7f3b1806bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.108+0000 7f3b0fe00640 1 -- 172.21.15.46:0/2903946048 >> v1:172.21.15.73:6789/0 conn(0x7f3b1815cd30 legacy=0x7f3b180cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:42.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.108+0000 7f3b0fe00640 1 -- 172.21.15.46:0/2903946048 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b180cf4d0 con 0x7f3b180cea30 2024-03-29T08:18:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.112+0000 7f3b0fe00640 1 --2- 172.21.15.46:0/2903946048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b180cea30 0x7f3b181606a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3b18071840 tx=0x7f3b0001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9057a21591483a51 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.112+0000 7f3b17e00640 1 -- 172.21.15.46:0/2903946048 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b000a8a10 con 0x7f3b180cea30 2024-03-29T08:18:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.112+0000 7f3b17e00640 1 -- 172.21.15.46:0/2903946048 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b000a8b70 con 0x7f3b180cea30 2024-03-29T08:18:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.112+0000 7f3b17e00640 1 -- 172.21.15.46:0/2903946048 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b000b0440 con 0x7f3b180cea30 2024-03-29T08:18:42.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.112+0000 7f3b1f600640 1 -- 172.21.15.46:0/2903946048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b180cea30 msgr2=0x7f3b181606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.112+0000 7f3b1f600640 1 --2- 172.21.15.46:0/2903946048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b180cea30 0x7f3b181606a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3b18071840 tx=0x7f3b0001ed10 comp rx=0 tx=0).stop 2024-03-29T08:18:42.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.112+0000 7f3b1f600640 1 -- 172.21.15.46:0/2903946048 shutdown_connections 2024-03-29T08:18:42.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.112+0000 7f3b1f600640 1 --2- 172.21.15.46:0/2903946048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b180cea30 0x7f3b181606a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.112+0000 7f3b1f600640 1 --2- 172.21.15.46:0/2903946048 >> v2:172.21.15.73:3300/0 conn(0x7f3b1815c950 0x7f3b1806bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.112+0000 7f3b1f600640 1 -- 172.21.15.46:0/2903946048 >> 172.21.15.46:0/2903946048 conn(0x7f3b180b21b0 msgr2=0x7f3b180b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:42.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.112+0000 7f3b1f600640 1 -- 172.21.15.46:0/2903946048 shutdown_connections 2024-03-29T08:18:42.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.112+0000 7f3b1f600640 1 -- 172.21.15.46:0/2903946048 wait complete. 2024-03-29T08:18:42.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.112+0000 7f3b1f600640 1 Processor -- start 2024-03-29T08:18:42.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b1f600640 1 -- start start 2024-03-29T08:18:42.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b1f600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3b180cea30 0x7f3b1809de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b1f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b1815c950 0x7f3b1809ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b1f600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b1815cd30 0x7f3b1814bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b1f600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1809b270 con 0x7f3b1815c950 2024-03-29T08:18:42.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b1f600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1809b510 con 0x7f3b1815cd30 2024-03-29T08:18:42.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b1f600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b180c5670 con 0x7f3b180cea30 2024-03-29T08:18:42.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b0fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b1815c950 0x7f3b1809ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b0fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b1815c950 0x7f3b1809ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43380/0 (socket says 172.21.15.46:43380) 2024-03-29T08:18:42.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b1cc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3b180cea30 0x7f3b1809de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b0fe00640 1 -- 172.21.15.46:0/4270582283 learned_addr learned my addr 172.21.15.46:0/4270582283 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:42.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b1d600640 1 --2- 172.21.15.46:0/4270582283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b1815cd30 0x7f3b1814bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b0fe00640 1 -- 172.21.15.46:0/4270582283 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3b180cea30 msgr2=0x7f3b1809de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b0fe00640 1 --2- 172.21.15.46:0/4270582283 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3b180cea30 0x7f3b1809de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b0fe00640 1 -- 172.21.15.46:0/4270582283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b1815cd30 msgr2=0x7f3b1814bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b0fe00640 1 --2- 172.21.15.46:0/4270582283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b1815cd30 0x7f3b1814bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b0fe00640 1 -- 172.21.15.46:0/4270582283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b000094d0 con 0x7f3b1815c950 2024-03-29T08:18:42.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b1d600640 1 --2- 172.21.15.46:0/4270582283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b1815cd30 0x7f3b1814bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:42.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b0fe00640 1 --2- 172.21.15.46:0/4270582283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b1815c950 0x7f3b1809ad30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3b0001b920 tx=0x7f3b0001bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:42.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b15600640 1 -- 172.21.15.46:0/4270582283 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b000ba600 con 0x7f3b1815c950 2024-03-29T08:18:42.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b15600640 1 -- 172.21.15.46:0/4270582283 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b000ba760 con 0x7f3b1815c950 2024-03-29T08:18:42.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b15600640 1 -- 172.21.15.46:0/4270582283 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b000bab10 con 0x7f3b1815c950 2024-03-29T08:18:42.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b1f600640 1 -- 172.21.15.46:0/4270582283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b180c5950 con 0x7f3b1815c950 2024-03-29T08:18:42.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b1f600640 1 -- 172.21.15.46:0/4270582283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1809b7f0 con 0x7f3b1815c950 2024-03-29T08:18:42.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.116+0000 7f3b1f600640 1 -- 172.21.15.46:0/4270582283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1809e690 con 0x7f3b1815c950 2024-03-29T08:18:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.120+0000 7f3b15600640 1 -- 172.21.15.46:0/4270582283 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3b000bac70 con 0x7f3b1815c950 2024-03-29T08:18:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.120+0000 7f3b15600640 1 --2- 172.21.15.46:0/4270582283 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3af4066690 0x7f3af4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.120+0000 7f3b15600640 1 -- 172.21.15.46:0/4270582283 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b00122740 con 0x7f3b1815c950 2024-03-29T08:18:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.120+0000 7f3b1cc00640 1 --2- 172.21.15.46:0/4270582283 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3af4066690 0x7f3af4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.120+0000 7f3b1cc00640 1 --2- 172.21.15.46:0/4270582283 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3af4066690 0x7f3af4068b50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3b08007a60 tx=0x7f3b08008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.124+0000 7f3b15600640 1 -- 172.21.15.46:0/4270582283 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3b000efa10 con 0x7f3b1815c950 2024-03-29T08:18:42.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.284+0000 7f3b1f600640 1 -- 172.21.15.46:0/4270582283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3b180c5e00 con 0x7f3b1815c950 2024-03-29T08:18:42.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.284+0000 7f3b15600640 1 -- 172.21.15.46:0/4270582283 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3b000f3030 con 0x7f3b1815c950 2024-03-29T08:18:42.286 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:18:42.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.284+0000 7f3b1f600640 1 -- 172.21.15.46:0/4270582283 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3af4066690 msgr2=0x7f3af4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.284+0000 7f3b1f600640 1 --2- 172.21.15.46:0/4270582283 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3af4066690 0x7f3af4068b50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3b08007a60 tx=0x7f3b08008040 comp rx=0 tx=0).stop 2024-03-29T08:18:42.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.284+0000 7f3b1f600640 1 -- 172.21.15.46:0/4270582283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b1815c950 msgr2=0x7f3b1809ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.284+0000 7f3b1f600640 1 --2- 172.21.15.46:0/4270582283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b1815c950 0x7f3b1809ad30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3b0001b920 tx=0x7f3b0001bab0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.284+0000 7f3b1f600640 1 -- 172.21.15.46:0/4270582283 shutdown_connections 2024-03-29T08:18:42.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.284+0000 7f3b1f600640 1 --2- 172.21.15.46:0/4270582283 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3af4066690 0x7f3af4068b50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.284+0000 7f3b1f600640 1 --2- 172.21.15.46:0/4270582283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b1815cd30 0x7f3b1814bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.284+0000 7f3b1f600640 1 --2- 172.21.15.46:0/4270582283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b1815c950 0x7f3b1809ad30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.284+0000 7f3b1f600640 1 --2- 172.21.15.46:0/4270582283 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3b180cea30 0x7f3b1809de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.284+0000 7f3b1f600640 1 -- 172.21.15.46:0/4270582283 >> 172.21.15.46:0/4270582283 conn(0x7f3b180b21b0 msgr2=0x7f3b18098a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:42.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.284+0000 7f3b1f600640 1 -- 172.21.15.46:0/4270582283 shutdown_connections 2024-03-29T08:18:42.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.284+0000 7f3b1f600640 1 -- 172.21.15.46:0/4270582283 wait complete. 2024-03-29T08:18:42.307 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:18:42.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc2a800640 1 Processor -- start 2024-03-29T08:18:42.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc2a800640 1 -- start start 2024-03-29T08:18:42.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc2a800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc2415c950 0x7fbc2406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc2a800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbc240cea30 0x7fbc241606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc2a800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc240cfb20 con 0x7fbc2415cd30 2024-03-29T08:18:42.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc2a800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc240cef10 con 0x7fbc240cea30 2024-03-29T08:18:42.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc2a800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240cf1f0 con 0x7fbc2415c950 2024-03-29T08:18:42.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc23e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbc2415cd30 0x7fbc240cad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54044/0 (socket says 172.21.15.46:54044) 2024-03-29T08:18:42.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc23e00640 1 -- 172.21.15.46:0/66738725 learned_addr learned my addr 172.21.15.46:0/66738725 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:42.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc1be00640 1 --2- 172.21.15.46:0/66738725 >> v2:172.21.15.73:3300/0 conn(0x7fbc240cea30 0x7fbc241606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc23400640 1 --2- 172.21.15.46:0/66738725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc2415c950 0x7fbc2406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc22a00640 1 -- 172.21.15.46:0/66738725 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665171082 0 0) 0x7fbc240cfb20 con 0x7fbc2415cd30 2024-03-29T08:18:42.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc22a00640 1 -- 172.21.15.46:0/66738725 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc08003530 con 0x7fbc2415cd30 2024-03-29T08:18:42.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc22a00640 1 -- 172.21.15.46:0/66738725 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 792917891 0 0) 0x7fbc08003530 con 0x7fbc2415cd30 2024-03-29T08:18:42.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc22a00640 1 -- 172.21.15.46:0/66738725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc2415c950 msgr2=0x7fbc2406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:42.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc22a00640 1 --2- 172.21.15.46:0/66738725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc2415c950 0x7fbc2406bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc22a00640 1 -- 172.21.15.46:0/66738725 >> v2:172.21.15.73:3300/0 conn(0x7fbc240cea30 msgr2=0x7fbc241606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:42.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc22a00640 1 --2- 172.21.15.46:0/66738725 >> v2:172.21.15.73:3300/0 conn(0x7fbc240cea30 0x7fbc241606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc22a00640 1 -- 172.21.15.46:0/66738725 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc240cf4d0 con 0x7fbc2415cd30 2024-03-29T08:18:42.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc22a00640 1 -- 172.21.15.46:0/66738725 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fbc1c001a00 con 0x7fbc2415cd30 2024-03-29T08:18:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc22a00640 1 -- 172.21.15.46:0/66738725 >> v1:172.21.15.46:6789/0 conn(0x7fbc2415cd30 legacy=0x7fbc240cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc22a00640 1 --2- 172.21.15.46:0/66738725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc08003980 0x7fbc08003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc22a00640 1 --2- 172.21.15.46:0/66738725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc2415c950 0x7fbc080047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc22a00640 1 --2- 172.21.15.46:0/66738725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc08004d10 0x7fbc08005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc22a00640 1 -- 172.21.15.46:0/66738725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240cef10 con 0x7fbc08004d10 2024-03-29T08:18:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc22a00640 1 -- 172.21.15.46:0/66738725 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240cf1f0 con 0x7fbc08003980 2024-03-29T08:18:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc22a00640 1 -- 172.21.15.46:0/66738725 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0800b9c0 con 0x7fbc2415c950 2024-03-29T08:18:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc23e00640 1 --2- 172.21.15.46:0/66738725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc08004d10 0x7fbc08005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc23400640 1 --2- 172.21.15.46:0/66738725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc08003980 0x7fbc08003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc1be00640 1 --2- 172.21.15.46:0/66738725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc2415c950 0x7fbc080047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc23e00640 1 -- 172.21.15.46:0/66738725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc2415c950 msgr2=0x7fbc080047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc23e00640 1 --2- 172.21.15.46:0/66738725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc2415c950 0x7fbc080047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc23e00640 1 -- 172.21.15.46:0/66738725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc08003980 msgr2=0x7fbc08003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc23e00640 1 --2- 172.21.15.46:0/66738725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc08003980 0x7fbc08003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.452+0000 7fbc23e00640 1 -- 172.21.15.46:0/66738725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc08003530 con 0x7fbc08004d10 2024-03-29T08:18:42.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.456+0000 7fbc23e00640 1 --2- 172.21.15.46:0/66738725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc08004d10 0x7fbc08005190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00a790 tx=0x7fbc1c01d2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2cb62564233f90 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.456+0000 7fbc22a00640 1 -- 172.21.15.46:0/66738725 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c0a5ea0 con 0x7fbc08004d10 2024-03-29T08:18:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.456+0000 7fbc22a00640 1 -- 172.21.15.46:0/66738725 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1c0a8070 con 0x7fbc08004d10 2024-03-29T08:18:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.456+0000 7fbc22a00640 1 -- 172.21.15.46:0/66738725 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c0ad890 con 0x7fbc08004d10 2024-03-29T08:18:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.456+0000 7fbc2a800640 1 -- 172.21.15.46:0/66738725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc08004d10 msgr2=0x7fbc08005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.456+0000 7fbc2a800640 1 --2- 172.21.15.46:0/66738725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc08004d10 0x7fbc08005190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00a790 tx=0x7fbc1c01d2c0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.456+0000 7fbc23e00640 1 -- 172.21.15.46:0/66738725 reap_dead start 2024-03-29T08:18:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.456+0000 7fbc2a800640 1 -- 172.21.15.46:0/66738725 shutdown_connections 2024-03-29T08:18:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.456+0000 7fbc2a800640 1 -- 172.21.15.46:0/66738725 >> 172.21.15.46:0/66738725 conn(0x7fbc240b21b0 msgr2=0x7fbc240b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.456+0000 7fbc2a800640 1 -- 172.21.15.46:0/66738725 shutdown_connections 2024-03-29T08:18:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.456+0000 7fbc2a800640 1 -- 172.21.15.46:0/66738725 wait complete. 2024-03-29T08:18:42.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc2a800640 1 Processor -- start 2024-03-29T08:18:42.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc2a800640 1 -- start start 2024-03-29T08:18:42.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc2a800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc08004d10 0x7fbc24147c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc2a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc24156a60 0x7fbc24156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc2a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc24144320 0x7fbc241447a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc2a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc24144ce0 con 0x7fbc24156a60 2024-03-29T08:18:42.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc2a800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc24148170 con 0x7fbc24144320 2024-03-29T08:18:42.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc2a800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc241581e0 con 0x7fbc08004d10 2024-03-29T08:18:42.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc1be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc24156a60 0x7fbc24156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc1be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc24156a60 0x7fbc24156ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43388/0 (socket says 172.21.15.46:43388) 2024-03-29T08:18:42.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc1be00640 1 -- 172.21.15.46:0/586002732 learned_addr learned my addr 172.21.15.46:0/586002732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:42.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc23e00640 1 --2- 172.21.15.46:0/586002732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc24144320 0x7fbc241447a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc23400640 1 --2- 172.21.15.46:0/586002732 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc08004d10 0x7fbc24147c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc1be00640 1 -- 172.21.15.46:0/586002732 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc08004d10 msgr2=0x7fbc24147c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc1be00640 1 --2- 172.21.15.46:0/586002732 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc08004d10 0x7fbc24147c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc1be00640 1 -- 172.21.15.46:0/586002732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc24144320 msgr2=0x7fbc241447a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc1be00640 1 --2- 172.21.15.46:0/586002732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc24144320 0x7fbc241447a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc1be00640 1 -- 172.21.15.46:0/586002732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc241575d0 con 0x7fbc24156a60 2024-03-29T08:18:42.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc1be00640 1 --2- 172.21.15.46:0/586002732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc24156a60 0x7fbc24156ee0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c011630 tx=0x7fbc0c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:42.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc1b400640 1 -- 172.21.15.46:0/586002732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c009510 con 0x7fbc24156a60 2024-03-29T08:18:42.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc1b400640 1 -- 172.21.15.46:0/586002732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c00f8c0 con 0x7fbc24156a60 2024-03-29T08:18:42.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc1b400640 1 -- 172.21.15.46:0/586002732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c00fc30 con 0x7fbc24156a60 2024-03-29T08:18:42.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc2a800640 1 -- 172.21.15.46:0/586002732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc241578b0 con 0x7fbc24156a60 2024-03-29T08:18:42.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.460+0000 7fbc2a800640 1 -- 172.21.15.46:0/586002732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc24157e70 con 0x7fbc24156a60 2024-03-29T08:18:42.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.464+0000 7fbc2a800640 1 -- 172.21.15.46:0/586002732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2409e690 con 0x7fbc24156a60 2024-03-29T08:18:42.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.464+0000 7fbc1b400640 1 -- 172.21.15.46:0/586002732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbc0c00fd90 con 0x7fbc24156a60 2024-03-29T08:18:42.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.464+0000 7fbc1b400640 1 --2- 172.21.15.46:0/586002732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc00066690 0x7fbc00068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.464+0000 7fbc1b400640 1 -- 172.21.15.46:0/586002732 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc0c084530 con 0x7fbc24156a60 2024-03-29T08:18:42.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.464+0000 7fbc23400640 1 --2- 172.21.15.46:0/586002732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc00066690 0x7fbc00068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.464+0000 7fbc23400640 1 --2- 172.21.15.46:0/586002732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc00066690 0x7fbc00068b50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009aa0 tx=0x7fbc14008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:42.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.468+0000 7fbc1b400640 1 -- 172.21.15.46:0/586002732 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbc0c051930 con 0x7fbc24156a60 2024-03-29T08:18:42.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.632+0000 7fbc2a800640 1 -- 172.21.15.46:0/586002732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbc241557d0 con 0x7fbc24156a60 2024-03-29T08:18:42.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.632+0000 7fbc1b400640 1 -- 172.21.15.46:0/586002732 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbc0c054f50 con 0x7fbc24156a60 2024-03-29T08:18:42.632 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:18:42.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.632+0000 7fbc2a800640 1 -- 172.21.15.46:0/586002732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc00066690 msgr2=0x7fbc00068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.632+0000 7fbc2a800640 1 --2- 172.21.15.46:0/586002732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc00066690 0x7fbc00068b50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009aa0 tx=0x7fbc14008040 comp rx=0 tx=0).stop 2024-03-29T08:18:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.632+0000 7fbc2a800640 1 -- 172.21.15.46:0/586002732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc24156a60 msgr2=0x7fbc24156ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.632+0000 7fbc2a800640 1 --2- 172.21.15.46:0/586002732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc24156a60 0x7fbc24156ee0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c011630 tx=0x7fbc0c011af0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.632+0000 7fbc2a800640 1 -- 172.21.15.46:0/586002732 shutdown_connections 2024-03-29T08:18:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.632+0000 7fbc2a800640 1 --2- 172.21.15.46:0/586002732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc00066690 0x7fbc00068b50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.632+0000 7fbc2a800640 1 --2- 172.21.15.46:0/586002732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc24144320 0x7fbc241447a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.632+0000 7fbc2a800640 1 --2- 172.21.15.46:0/586002732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc24156a60 0x7fbc24156ee0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.632+0000 7fbc2a800640 1 --2- 172.21.15.46:0/586002732 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc08004d10 0x7fbc24147c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.632+0000 7fbc2a800640 1 -- 172.21.15.46:0/586002732 >> 172.21.15.46:0/586002732 conn(0x7fbc240b21b0 msgr2=0x7fbc240b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.632+0000 7fbc2a800640 1 -- 172.21.15.46:0/586002732 shutdown_connections 2024-03-29T08:18:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.632+0000 7fbc2a800640 1 -- 172.21.15.46:0/586002732 wait complete. 2024-03-29T08:18:42.654 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:18:42.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.796+0000 7f4096a00640 1 Processor -- start 2024-03-29T08:18:42.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.796+0000 7f4096a00640 1 -- start start 2024-03-29T08:18:42.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.796+0000 7f4096a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f409006aff0 0x7f40900d3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.796+0000 7f4096a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40900cd860 con 0x7f40900ce620 2024-03-29T08:18:42.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.796+0000 7f4096a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40900cdb40 con 0x7f409006b7f0 2024-03-29T08:18:42.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.796+0000 7f4096a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409006b480 con 0x7f409006aff0 2024-03-29T08:18:42.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.800+0000 7f4094c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f40900ce620 0x7f40900d0620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54050/0 (socket says 172.21.15.46:54050) 2024-03-29T08:18:42.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.800+0000 7f4094c00640 1 -- 172.21.15.46:0/1990815112 learned_addr learned my addr 172.21.15.46:0/1990815112 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:42.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.800+0000 7f4096000640 1 --2- 172.21.15.46:0/1990815112 >> v2:172.21.15.73:3300/0 conn(0x7f409006aff0 0x7f40900d3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.800+0000 7f408fe00640 1 -- 172.21.15.46:0/1990815112 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339673508 0 0) 0x7f40900cd860 con 0x7f40900ce620 2024-03-29T08:18:42.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.800+0000 7f408fe00640 1 -- 172.21.15.46:0/1990815112 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4078003530 con 0x7f40900ce620 2024-03-29T08:18:42.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.800+0000 7f408fe00640 1 -- 172.21.15.46:0/1990815112 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1978021790 0 0) 0x7f40900cdb40 con 0x7f409006b7f0 2024-03-29T08:18:42.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.800+0000 7f4096000640 1 -- 172.21.15.46:0/1990815112 >> v1:172.21.15.73:6789/0 conn(0x7f409006b7f0 legacy=0x7f409006bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.800+0000 7f4096000640 1 -- 172.21.15.46:0/1990815112 >> v1:172.21.15.46:6789/0 conn(0x7f40900ce620 legacy=0x7f40900d0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.800+0000 7f4096000640 1 -- 172.21.15.46:0/1990815112 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40900cde20 con 0x7f409006aff0 2024-03-29T08:18:42.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.800+0000 7f4096000640 1 --2- 172.21.15.46:0/1990815112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f409006aff0 0x7f40900d3040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4088009d90 tx=0x7f408801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3896045f9f9b2118 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:42.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.800+0000 7f408fe00640 1 -- 172.21.15.46:0/1990815112 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40880a5830 con 0x7f409006aff0 2024-03-29T08:18:42.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.800+0000 7f408fe00640 1 -- 172.21.15.46:0/1990815112 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40880a5990 con 0x7f409006aff0 2024-03-29T08:18:42.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.800+0000 7f408fe00640 1 -- 172.21.15.46:0/1990815112 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40880a5d20 con 0x7f409006aff0 2024-03-29T08:18:42.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4096a00640 1 -- 172.21.15.46:0/1990815112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f409006aff0 msgr2=0x7f40900d3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4096a00640 1 --2- 172.21.15.46:0/1990815112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f409006aff0 0x7f40900d3040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4088009d90 tx=0x7f408801c970 comp rx=0 tx=0).stop 2024-03-29T08:18:42.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4096a00640 1 -- 172.21.15.46:0/1990815112 shutdown_connections 2024-03-29T08:18:42.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4096a00640 1 --2- 172.21.15.46:0/1990815112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f409006aff0 0x7f40900d3040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4096a00640 1 -- 172.21.15.46:0/1990815112 >> 172.21.15.46:0/1990815112 conn(0x7f4090089fd0 msgr2=0x7f40900d2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:42.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4096a00640 1 -- 172.21.15.46:0/1990815112 shutdown_connections 2024-03-29T08:18:42.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4096a00640 1 -- 172.21.15.46:0/1990815112 wait complete. 2024-03-29T08:18:42.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4096a00640 1 Processor -- start 2024-03-29T08:18:42.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4096a00640 1 -- start start 2024-03-29T08:18:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4096a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f409006aff0 0x7f4090157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4096a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409006b7f0 0x7f40901591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4096a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40900ce620 0x7f409015fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4096a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409015ff70 con 0x7f409006b7f0 2024-03-29T08:18:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4096a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409015e150 con 0x7f40900ce620 2024-03-29T08:18:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4096a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409015e3f0 con 0x7f409006aff0 2024-03-29T08:18:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4094c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409006b7f0 0x7f40901591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4094c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409006b7f0 0x7f40901591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43402/0 (socket says 172.21.15.46:43402) 2024-03-29T08:18:42.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4094c00640 1 -- 172.21.15.46:0/1716744237 learned_addr learned my addr 172.21.15.46:0/1716744237 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:42.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4095600640 1 --2- 172.21.15.46:0/1716744237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f409006aff0 0x7f4090157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4096000640 1 --2- 172.21.15.46:0/1716744237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40900ce620 0x7f409015fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4094c00640 1 -- 172.21.15.46:0/1716744237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f409006aff0 msgr2=0x7f4090157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4094c00640 1 --2- 172.21.15.46:0/1716744237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f409006aff0 0x7f4090157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4094c00640 1 -- 172.21.15.46:0/1716744237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40900ce620 msgr2=0x7f409015fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4094c00640 1 --2- 172.21.15.46:0/1716744237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40900ce620 0x7f409015fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.804+0000 7f4094c00640 1 -- 172.21.15.46:0/1716744237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4088009a70 con 0x7f409006b7f0 2024-03-29T08:18:42.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.808+0000 7f4094c00640 1 --2- 172.21.15.46:0/1716744237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409006b7f0 0x7f40901591c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f407c00c700 tx=0x7f407c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:42.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.808+0000 7f408d600640 1 -- 172.21.15.46:0/1716744237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c007270 con 0x7f409006b7f0 2024-03-29T08:18:42.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.808+0000 7f408d600640 1 -- 172.21.15.46:0/1716744237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c0073d0 con 0x7f409006b7f0 2024-03-29T08:18:42.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.808+0000 7f408d600640 1 -- 172.21.15.46:0/1716744237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c001d50 con 0x7f409006b7f0 2024-03-29T08:18:42.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.808+0000 7f4096a00640 1 -- 172.21.15.46:0/1716744237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409015e6d0 con 0x7f409006b7f0 2024-03-29T08:18:42.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.808+0000 7f4096a00640 1 -- 172.21.15.46:0/1716744237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409015ec90 con 0x7f409006b7f0 2024-03-29T08:18:42.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.808+0000 7f4096a00640 1 -- 172.21.15.46:0/1716744237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409004e040 con 0x7f409006b7f0 2024-03-29T08:18:42.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.808+0000 7f408d600640 1 -- 172.21.15.46:0/1716744237 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f407c0057b0 con 0x7f409006b7f0 2024-03-29T08:18:42.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.808+0000 7f408d600640 1 --2- 172.21.15.46:0/1716744237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4070066690 0x7f4070068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:42.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.808+0000 7f408d600640 1 -- 172.21.15.46:0/1716744237 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f407c083720 con 0x7f409006b7f0 2024-03-29T08:18:42.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.812+0000 7f4095600640 1 --2- 172.21.15.46:0/1716744237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4070066690 0x7f4070068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:42.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.812+0000 7f4095600640 1 --2- 172.21.15.46:0/1716744237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4070066690 0x7f4070068b50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4084007fd0 tx=0x7f4084009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:42.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.812+0000 7f408d600640 1 -- 172.21.15.46:0/1716744237 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f407c050aa0 con 0x7f409006b7f0 2024-03-29T08:18:42.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.976+0000 7f4096a00640 1 -- 172.21.15.46:0/1716744237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f409006b5d0 con 0x7f409006b7f0 2024-03-29T08:18:42.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.976+0000 7f408d600640 1 -- 172.21.15.46:0/1716744237 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f407c0170a0 con 0x7f409006b7f0 2024-03-29T08:18:42.977 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:18:42.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.976+0000 7f4096a00640 1 -- 172.21.15.46:0/1716744237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4070066690 msgr2=0x7f4070068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.976+0000 7f4096a00640 1 --2- 172.21.15.46:0/1716744237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4070066690 0x7f4070068b50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4084007fd0 tx=0x7f4084009040 comp rx=0 tx=0).stop 2024-03-29T08:18:42.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.976+0000 7f4096a00640 1 -- 172.21.15.46:0/1716744237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409006b7f0 msgr2=0x7f40901591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:42.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.976+0000 7f4096a00640 1 --2- 172.21.15.46:0/1716744237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409006b7f0 0x7f40901591c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f407c00c700 tx=0x7f407c00cbc0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.976+0000 7f4096a00640 1 -- 172.21.15.46:0/1716744237 shutdown_connections 2024-03-29T08:18:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.976+0000 7f4096a00640 1 --2- 172.21.15.46:0/1716744237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4070066690 0x7f4070068b50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.976+0000 7f4096a00640 1 --2- 172.21.15.46:0/1716744237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40900ce620 0x7f409015fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.976+0000 7f4096a00640 1 --2- 172.21.15.46:0/1716744237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409006b7f0 0x7f40901591c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.976+0000 7f4096a00640 1 --2- 172.21.15.46:0/1716744237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f409006aff0 0x7f4090157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.976+0000 7f4096a00640 1 -- 172.21.15.46:0/1716744237 >> 172.21.15.46:0/1716744237 conn(0x7f4090089fd0 msgr2=0x7f4090160340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.976+0000 7f4096a00640 1 -- 172.21.15.46:0/1716744237 shutdown_connections 2024-03-29T08:18:42.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:42.976+0000 7f4096a00640 1 -- 172.21.15.46:0/1716744237 wait complete. 2024-03-29T08:18:42.999 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:43.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.136+0000 7f7947400640 1 Processor -- start 2024-03-29T08:18:43.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.136+0000 7f7947400640 1 -- start start 2024-03-29T08:18:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.140+0000 7f7947400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f794015cd90 0x7f794006cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.140+0000 7f7947400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f794015d940 0x7f79401597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.140+0000 7f7947400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f794006d130 con 0x7f79401593d0 2024-03-29T08:18:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.140+0000 7f7947400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794015b8f0 con 0x7f794015d940 2024-03-29T08:18:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.140+0000 7f7947400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f794015ad50 con 0x7f794015cd90 2024-03-29T08:18:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.140+0000 7f7944a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f79401593d0 0x7f794015d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54056/0 (socket says 172.21.15.46:54056) 2024-03-29T08:18:43.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.140+0000 7f7944a00640 1 -- 172.21.15.46:0/399673817 learned_addr learned my addr 172.21.15.46:0/399673817 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.140+0000 7f7945400640 1 --2- 172.21.15.46:0/399673817 >> v2:172.21.15.46:3300/0 conn(0x7f794015d940 0x7f79401597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.140+0000 7f793fe00640 1 --2- 172.21.15.46:0/399673817 >> v2:172.21.15.73:3300/0 conn(0x7f794015cd90 0x7f794006cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.140+0000 7f7945400640 1 -- 172.21.15.46:0/399673817 >> v2:172.21.15.73:3300/0 conn(0x7f794015cd90 msgr2=0x7f794006cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.140+0000 7f7945400640 1 --2- 172.21.15.46:0/399673817 >> v2:172.21.15.73:3300/0 conn(0x7f794015cd90 0x7f794006cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.140+0000 7f7945400640 1 -- 172.21.15.46:0/399673817 >> v1:172.21.15.46:6789/0 conn(0x7f79401593d0 legacy=0x7f794015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.140+0000 7f7945400640 1 -- 172.21.15.46:0/399673817 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794015afc0 con 0x7f794015d940 2024-03-29T08:18:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.140+0000 7f793f400640 1 -- 172.21.15.46:0/399673817 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61058899 0 0) 0x7f794006d130 con 0x7f79401593d0 2024-03-29T08:18:43.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f7945400640 1 --2- 172.21.15.46:0/399673817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f794015d940 0x7f79401597b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f79400cf180 tx=0x7f793801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f435959e03b33a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:43.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f793f400640 1 -- 172.21.15.46:0/399673817 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79380a59f0 con 0x7f794015d940 2024-03-29T08:18:43.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f793f400640 1 -- 172.21.15.46:0/399673817 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79380a5b50 con 0x7f794015d940 2024-03-29T08:18:43.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f793f400640 1 -- 172.21.15.46:0/399673817 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79380ad440 con 0x7f794015d940 2024-03-29T08:18:43.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f7947400640 1 -- 172.21.15.46:0/399673817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f794015d940 msgr2=0x7f79401597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:43.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f7947400640 1 --2- 172.21.15.46:0/399673817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f794015d940 0x7f79401597b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f79400cf180 tx=0x7f793801c930 comp rx=0 tx=0).stop 2024-03-29T08:18:43.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f7947400640 1 -- 172.21.15.46:0/399673817 shutdown_connections 2024-03-29T08:18:43.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f7947400640 1 --2- 172.21.15.46:0/399673817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f794015d940 0x7f79401597b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f7947400640 1 --2- 172.21.15.46:0/399673817 >> v2:172.21.15.73:3300/0 conn(0x7f794015cd90 0x7f794006cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f7947400640 1 -- 172.21.15.46:0/399673817 >> 172.21.15.46:0/399673817 conn(0x7f7940066110 msgr2=0x7f7940069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:43.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f7947400640 1 -- 172.21.15.46:0/399673817 shutdown_connections 2024-03-29T08:18:43.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f7947400640 1 -- 172.21.15.46:0/399673817 wait complete. 2024-03-29T08:18:43.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f7947400640 1 Processor -- start 2024-03-29T08:18:43.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f7947400640 1 -- start start 2024-03-29T08:18:43.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f7947400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f79401593d0 0x7f7940158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f7947400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f794015cd90 0x7f79401f8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f7947400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f794015d940 0x7f79400cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f7947400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79400cd1b0 con 0x7f794015d940 2024-03-29T08:18:43.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f7947400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79401f90f0 con 0x7f794015cd90 2024-03-29T08:18:43.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.144+0000 7f7947400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79401f93d0 con 0x7f79401593d0 2024-03-29T08:18:43.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f7945400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f794015d940 0x7f79400cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:43.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f7945400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f794015d940 0x7f79400cd640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43416/0 (socket says 172.21.15.46:43416) 2024-03-29T08:18:43.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f7945400640 1 -- 172.21.15.46:0/1479034782 learned_addr learned my addr 172.21.15.46:0/1479034782 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:43.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f793fe00640 1 --2- 172.21.15.46:0/1479034782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f794015cd90 0x7f79401f8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:43.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f7944a00640 1 --2- 172.21.15.46:0/1479034782 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f79401593d0 0x7f7940158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:43.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f7945400640 1 -- 172.21.15.46:0/1479034782 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f79401593d0 msgr2=0x7f7940158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:43.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f7945400640 1 --2- 172.21.15.46:0/1479034782 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f79401593d0 0x7f7940158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f7945400640 1 -- 172.21.15.46:0/1479034782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f794015cd90 msgr2=0x7f79401f8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:43.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f7945400640 1 --2- 172.21.15.46:0/1479034782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f794015cd90 0x7f79401f8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f7945400640 1 -- 172.21.15.46:0/1479034782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7938009a70 con 0x7f794015d940 2024-03-29T08:18:43.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f7944a00640 1 --2- 172.21.15.46:0/1479034782 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f79401593d0 0x7f7940158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:43.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f7945400640 1 --2- 172.21.15.46:0/1479034782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f794015d940 0x7f79400cd640 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7938019540 tx=0x7f79380196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:43.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f793cc00640 1 -- 172.21.15.46:0/1479034782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79380ade00 con 0x7f794015d940 2024-03-29T08:18:43.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f793cc00640 1 -- 172.21.15.46:0/1479034782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7938009450 con 0x7f794015d940 2024-03-29T08:18:43.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f793cc00640 1 -- 172.21.15.46:0/1479034782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79380b7650 con 0x7f794015d940 2024-03-29T08:18:43.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f7947400640 1 -- 172.21.15.46:0/1479034782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79401f96b0 con 0x7f794015d940 2024-03-29T08:18:43.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f7947400640 1 -- 172.21.15.46:0/1479034782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79401f8d90 con 0x7f794015d940 2024-03-29T08:18:43.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f7947400640 1 -- 172.21.15.46:0/1479034782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79401509e0 con 0x7f794015d940 2024-03-29T08:18:43.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f793cc00640 1 -- 172.21.15.46:0/1479034782 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f79380b4050 con 0x7f794015d940 2024-03-29T08:18:43.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f793cc00640 1 --2- 172.21.15.46:0/1479034782 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7920066690 0x7f7920068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.148+0000 7f793cc00640 1 -- 172.21.15.46:0/1479034782 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f793811f240 con 0x7f794015d940 2024-03-29T08:18:43.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.152+0000 7f7944a00640 1 --2- 172.21.15.46:0/1479034782 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7920066690 0x7f7920068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:43.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.152+0000 7f7944a00640 1 --2- 172.21.15.46:0/1479034782 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7920066690 0x7f7920068b50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7934007fd0 tx=0x7f7934007c20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:43.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.156+0000 7f793cc00640 1 -- 172.21.15.46:0/1479034782 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f79380ec5b0 con 0x7f794015d940 2024-03-29T08:18:43.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.316+0000 7f7947400640 1 -- 172.21.15.46:0/1479034782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79400cd360 con 0x7f794015d940 2024-03-29T08:18:43.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.316+0000 7f793cc00640 1 -- 172.21.15.46:0/1479034782 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f79400cd360 con 0x7f794015d940 2024-03-29T08:18:43.319 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:43.319 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:14.314124+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:43.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.320+0000 7f7947400640 1 -- 172.21.15.46:0/1479034782 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7920066690 msgr2=0x7f7920068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:43.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.320+0000 7f7947400640 1 --2- 172.21.15.46:0/1479034782 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7920066690 0x7f7920068b50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7934007fd0 tx=0x7f7934007c20 comp rx=0 tx=0).stop 2024-03-29T08:18:43.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.320+0000 7f7947400640 1 -- 172.21.15.46:0/1479034782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f794015d940 msgr2=0x7f79400cd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:43.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.320+0000 7f7947400640 1 --2- 172.21.15.46:0/1479034782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f794015d940 0x7f79400cd640 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7938019540 tx=0x7f79380196d0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.320+0000 7f7947400640 1 -- 172.21.15.46:0/1479034782 shutdown_connections 2024-03-29T08:18:43.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.320+0000 7f7947400640 1 --2- 172.21.15.46:0/1479034782 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7920066690 0x7f7920068b50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.320+0000 7f7947400640 1 --2- 172.21.15.46:0/1479034782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f794015cd90 0x7f79401f8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.320+0000 7f7947400640 1 --2- 172.21.15.46:0/1479034782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f794015d940 0x7f79400cd640 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.320+0000 7f7947400640 1 --2- 172.21.15.46:0/1479034782 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f79401593d0 0x7f7940158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.320+0000 7f7947400640 1 -- 172.21.15.46:0/1479034782 >> 172.21.15.46:0/1479034782 conn(0x7f7940066110 msgr2=0x7f79400701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.320+0000 7f7947400640 1 -- 172.21.15.46:0/1479034782 shutdown_connections 2024-03-29T08:18:43.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.320+0000 7f7947400640 1 -- 172.21.15.46:0/1479034782 wait complete. 2024-03-29T08:18:43.339 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:18:11.177842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:18:12.308252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:18:43.340 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:18:43.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.480+0000 7fe2bfe00640 1 Processor -- start 2024-03-29T08:18:43.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.480+0000 7fe2bfe00640 1 -- start start 2024-03-29T08:18:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.484+0000 7fe2bfe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe2c015a9f0 0x7fe2c00ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.484+0000 7fe2bfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe2c00ccf60 0x7fe2c00cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.484+0000 7fe2bfe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe2c006eef0 0x7fe2c006f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.484+0000 7fe2bfe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c00cd8e0 con 0x7fe2c00ccf60 2024-03-29T08:18:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.484+0000 7fe2bfe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c00cf590 con 0x7fe2c006eef0 2024-03-29T08:18:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.484+0000 7fe2bfe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c006e130 con 0x7fe2c015a9f0 2024-03-29T08:18:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.484+0000 7fe2b6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe2c00ccf60 0x7fe2c00cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.484+0000 7fe2b6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe2c00ccf60 0x7fe2c00cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43418/0 (socket says 172.21.15.46:43418) 2024-03-29T08:18:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.484+0000 7fe2b6000640 1 -- 172.21.15.46:0/835388839 learned_addr learned my addr 172.21.15.46:0/835388839 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.484+0000 7fe2bea00640 1 --2- 172.21.15.46:0/835388839 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe2c015a9f0 0x7fe2c00ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.484+0000 7fe2bf400640 1 --2- 172.21.15.46:0/835388839 >> v2:172.21.15.73:3300/0 conn(0x7fe2c006eef0 0x7fe2c006f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.484+0000 7fe2b6000640 1 -- 172.21.15.46:0/835388839 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe2c015a9f0 msgr2=0x7fe2c00ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:43.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.484+0000 7fe2b6000640 1 --2- 172.21.15.46:0/835388839 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe2c015a9f0 0x7fe2c00ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.484+0000 7fe2b6000640 1 -- 172.21.15.46:0/835388839 >> v2:172.21.15.73:3300/0 conn(0x7fe2c006eef0 msgr2=0x7fe2c006f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:43.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.484+0000 7fe2b6000640 1 --2- 172.21.15.46:0/835388839 >> v2:172.21.15.73:3300/0 conn(0x7fe2c006eef0 0x7fe2c006f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.484+0000 7fe2b6000640 1 -- 172.21.15.46:0/835388839 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c006e410 con 0x7fe2c00ccf60 2024-03-29T08:18:43.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2b6000640 1 --2- 172.21.15.46:0/835388839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2c00ccf60 0x7fe2c00cd3a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0071c60 tx=0x7fe2a801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb45144fbb88c50 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:43.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2be000640 1 -- 172.21.15.46:0/835388839 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80a89d0 con 0x7fe2c00ccf60 2024-03-29T08:18:43.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2be000640 1 -- 172.21.15.46:0/835388839 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a80a8b30 con 0x7fe2c00ccf60 2024-03-29T08:18:43.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2be000640 1 -- 172.21.15.46:0/835388839 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80b0440 con 0x7fe2c00ccf60 2024-03-29T08:18:43.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2bfe00640 1 -- 172.21.15.46:0/835388839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2c00ccf60 msgr2=0x7fe2c00cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:43.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2bfe00640 1 --2- 172.21.15.46:0/835388839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2c00ccf60 0x7fe2c00cd3a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0071c60 tx=0x7fe2a801ed10 comp rx=0 tx=0).stop 2024-03-29T08:18:43.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2bfe00640 1 -- 172.21.15.46:0/835388839 shutdown_connections 2024-03-29T08:18:43.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2bfe00640 1 --2- 172.21.15.46:0/835388839 >> v2:172.21.15.73:3300/0 conn(0x7fe2c006eef0 0x7fe2c006f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2bfe00640 1 --2- 172.21.15.46:0/835388839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2c00ccf60 0x7fe2c00cd3a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2bfe00640 1 --2- 172.21.15.46:0/835388839 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe2c015a9f0 0x7fe2c00ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2bfe00640 1 -- 172.21.15.46:0/835388839 >> 172.21.15.46:0/835388839 conn(0x7fe2c00d0550 msgr2=0x7fe2c00d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:43.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2bfe00640 1 -- 172.21.15.46:0/835388839 shutdown_connections 2024-03-29T08:18:43.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2bfe00640 1 -- 172.21.15.46:0/835388839 wait complete. 2024-03-29T08:18:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2bfe00640 1 Processor -- start 2024-03-29T08:18:43.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2bfe00640 1 -- start start 2024-03-29T08:18:43.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2bfe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2c006eef0 0x7fe2c00c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2bfe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe2c00ccf60 0x7fe2c0152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2bfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2c015a9f0 0x7fe2c0145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2bfe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c00c6f30 con 0x7fe2c015a9f0 2024-03-29T08:18:43.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2bfe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c00c6c70 con 0x7fe2c006eef0 2024-03-29T08:18:43.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.488+0000 7fe2bfe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c00b5ac0 con 0x7fe2c00ccf60 2024-03-29T08:18:43.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2c015a9f0 0x7fe2c0145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:43.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2c015a9f0 0x7fe2c0145630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43422/0 (socket says 172.21.15.46:43422) 2024-03-29T08:18:43.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2bf400640 1 -- 172.21.15.46:0/204278136 learned_addr learned my addr 172.21.15.46:0/204278136 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:43.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2b6000640 1 --2- 172.21.15.46:0/204278136 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe2c00ccf60 0x7fe2c0152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:43.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2bea00640 1 --2- 172.21.15.46:0/204278136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2c006eef0 0x7fe2c00c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:43.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2bf400640 1 -- 172.21.15.46:0/204278136 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe2c00ccf60 msgr2=0x7fe2c0152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:43.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2bf400640 1 --2- 172.21.15.46:0/204278136 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe2c00ccf60 0x7fe2c0152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2bf400640 1 -- 172.21.15.46:0/204278136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2c006eef0 msgr2=0x7fe2c00c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:43.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2bf400640 1 --2- 172.21.15.46:0/204278136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2c006eef0 0x7fe2c00c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2bf400640 1 -- 172.21.15.46:0/204278136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a80094d0 con 0x7fe2c015a9f0 2024-03-29T08:18:43.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2bf400640 1 --2- 172.21.15.46:0/204278136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2c015a9f0 0x7fe2c0145630 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe2b800eed0 tx=0x7fe2b800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:43.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2b7400640 1 -- 172.21.15.46:0/204278136 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8010040 con 0x7fe2c015a9f0 2024-03-29T08:18:43.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2b7400640 1 -- 172.21.15.46:0/204278136 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b800bd20 con 0x7fe2c015a9f0 2024-03-29T08:18:43.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2bfe00640 1 -- 172.21.15.46:0/204278136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c00b5da0 con 0x7fe2c015a9f0 2024-03-29T08:18:43.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2b7400640 1 -- 172.21.15.46:0/204278136 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8014640 con 0x7fe2c015a9f0 2024-03-29T08:18:43.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2bfe00640 1 -- 172.21.15.46:0/204278136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c015cb80 con 0x7fe2c015a9f0 2024-03-29T08:18:43.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2bfe00640 1 -- 172.21.15.46:0/204278136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c015cd70 con 0x7fe2c015a9f0 2024-03-29T08:18:43.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.492+0000 7fe2b7400640 1 -- 172.21.15.46:0/204278136 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe2b80147a0 con 0x7fe2c015a9f0 2024-03-29T08:18:43.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.496+0000 7fe2b7400640 1 --2- 172.21.15.46:0/204278136 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe29c066690 0x7fe29c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.496+0000 7fe2b7400640 1 -- 172.21.15.46:0/204278136 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2b8084600 con 0x7fe2c015a9f0 2024-03-29T08:18:43.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.496+0000 7fe2bea00640 1 --2- 172.21.15.46:0/204278136 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe29c066690 0x7fe29c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:43.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.496+0000 7fe2bea00640 1 --2- 172.21.15.46:0/204278136 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe29c066690 0x7fe29c068b50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009b60 tx=0x7fe2b0009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:43.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.500+0000 7fe2b7400640 1 -- 172.21.15.46:0/204278136 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe2b8051ab0 con 0x7fe2c015a9f0 2024-03-29T08:18:43.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.660+0000 7fe2bfe00640 1 -- 172.21.15.46:0/204278136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2c004e040 con 0x7fe2c015a9f0 2024-03-29T08:18:43.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.660+0000 7fe2b7400640 1 -- 172.21.15.46:0/204278136 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2b80550d0 con 0x7fe2c015a9f0 2024-03-29T08:18:43.661 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:18:43.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.660+0000 7fe2bfe00640 1 -- 172.21.15.46:0/204278136 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe29c066690 msgr2=0x7fe29c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:43.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.660+0000 7fe2bfe00640 1 --2- 172.21.15.46:0/204278136 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe29c066690 0x7fe29c068b50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009b60 tx=0x7fe2b0009190 comp rx=0 tx=0).stop 2024-03-29T08:18:43.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.660+0000 7fe2bfe00640 1 -- 172.21.15.46:0/204278136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2c015a9f0 msgr2=0x7fe2c0145630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:43.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.660+0000 7fe2bfe00640 1 --2- 172.21.15.46:0/204278136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2c015a9f0 0x7fe2c0145630 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe2b800eed0 tx=0x7fe2b800b480 comp rx=0 tx=0).stop 2024-03-29T08:18:43.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.660+0000 7fe2bfe00640 1 -- 172.21.15.46:0/204278136 shutdown_connections 2024-03-29T08:18:43.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.660+0000 7fe2bfe00640 1 --2- 172.21.15.46:0/204278136 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe29c066690 0x7fe29c068b50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.660+0000 7fe2bfe00640 1 --2- 172.21.15.46:0/204278136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2c015a9f0 0x7fe2c0145630 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.660+0000 7fe2bfe00640 1 --2- 172.21.15.46:0/204278136 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe2c00ccf60 0x7fe2c0152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.660+0000 7fe2bfe00640 1 --2- 172.21.15.46:0/204278136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe2c006eef0 0x7fe2c00c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.660+0000 7fe2bfe00640 1 -- 172.21.15.46:0/204278136 >> 172.21.15.46:0/204278136 conn(0x7fe2c00d0550 msgr2=0x7fe2c006f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:43.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.660+0000 7fe2bfe00640 1 -- 172.21.15.46:0/204278136 shutdown_connections 2024-03-29T08:18:43.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.660+0000 7fe2bfe00640 1 -- 172.21.15.46:0/204278136 wait complete. 2024-03-29T08:18:43.681 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:18:43.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.824+0000 7f0ad4800640 1 Processor -- start 2024-03-29T08:18:43.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.824+0000 7f0ad4800640 1 -- start start 2024-03-29T08:18:43.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.824+0000 7f0ad4800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0acc142a50 0x7f0acc0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.824+0000 7f0ad4800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0acc15d260 con 0x7f0acc0ae990 2024-03-29T08:18:43.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.824+0000 7f0ad4800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0acc0aefd0 con 0x7f0acc15df90 2024-03-29T08:18:43.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.824+0000 7f0ad4800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc15d4b0 con 0x7f0acc142a50 2024-03-29T08:18:43.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.824+0000 7f0ad2800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0acc0ae990 0x7f0acc0af530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54066/0 (socket says 172.21.15.46:54066) 2024-03-29T08:18:43.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.824+0000 7f0ad2800640 1 -- 172.21.15.46:0/399043034 learned_addr learned my addr 172.21.15.46:0/399043034 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:43.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.824+0000 7f0ad1e00640 1 --2- 172.21.15.46:0/399043034 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0acc142a50 0x7f0acc0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:43.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.824+0000 7f0ad0a00640 1 -- 172.21.15.46:0/399043034 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388029027 0 0) 0x7f0acc15d260 con 0x7f0acc0ae990 2024-03-29T08:18:43.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.824+0000 7f0ad0a00640 1 -- 172.21.15.46:0/399043034 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ab4003530 con 0x7f0acc0ae990 2024-03-29T08:18:43.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.824+0000 7f0ad0a00640 1 -- 172.21.15.46:0/399043034 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635105641 0 0) 0x7f0acc0aefd0 con 0x7f0acc15df90 2024-03-29T08:18:43.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.824+0000 7f0ad0a00640 1 -- 172.21.15.46:0/399043034 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0acc15d260 con 0x7f0acc15df90 2024-03-29T08:18:43.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.824+0000 7f0ad1e00640 1 -- 172.21.15.46:0/399043034 >> v1:172.21.15.73:6789/0 conn(0x7f0acc15df90 legacy=0x7f0acc0ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:43.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.824+0000 7f0ad1e00640 1 -- 172.21.15.46:0/399043034 >> v1:172.21.15.46:6789/0 conn(0x7f0acc0ae990 legacy=0x7f0acc0af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:43.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.824+0000 7f0ad1e00640 1 -- 172.21.15.46:0/399043034 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0acc15d790 con 0x7f0acc142a50 2024-03-29T08:18:43.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.828+0000 7f0ad1e00640 1 --2- 172.21.15.46:0/399043034 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0acc142a50 0x7f0acc0cb790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0ac00097f0 tx=0x7f0ac001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=753d1e932a51cffe server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:43.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.828+0000 7f0ad0a00640 1 -- 172.21.15.46:0/399043034 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00ab400 con 0x7f0acc142a50 2024-03-29T08:18:43.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.828+0000 7f0ad0a00640 1 -- 172.21.15.46:0/399043034 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac00a7050 con 0x7f0acc142a50 2024-03-29T08:18:43.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.828+0000 7f0ad0a00640 1 -- 172.21.15.46:0/399043034 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00ab780 con 0x7f0acc142a50 2024-03-29T08:18:43.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.828+0000 7f0ad4800640 1 -- 172.21.15.46:0/399043034 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0acc142a50 msgr2=0x7f0acc0cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:43.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.828+0000 7f0ad4800640 1 --2- 172.21.15.46:0/399043034 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0acc142a50 0x7f0acc0cb790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0ac00097f0 tx=0x7f0ac001c450 comp rx=0 tx=0).stop 2024-03-29T08:18:43.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.828+0000 7f0ad4800640 1 -- 172.21.15.46:0/399043034 shutdown_connections 2024-03-29T08:18:43.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.828+0000 7f0ad4800640 1 --2- 172.21.15.46:0/399043034 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0acc142a50 0x7f0acc0cb790 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.828+0000 7f0ad4800640 1 -- 172.21.15.46:0/399043034 >> 172.21.15.46:0/399043034 conn(0x7f0acc06a390 msgr2=0x7f0acc0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:43.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.828+0000 7f0ad4800640 1 -- 172.21.15.46:0/399043034 shutdown_connections 2024-03-29T08:18:43.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.828+0000 7f0ad4800640 1 -- 172.21.15.46:0/399043034 wait complete. 2024-03-29T08:18:43.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad4800640 1 Processor -- start 2024-03-29T08:18:43.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad4800640 1 -- start start 2024-03-29T08:18:43.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad4800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0acc0ae990 0x7f0acc0cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad4800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0acc142a50 0x7f0acc138300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad4800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0acc15df90 0x7f0acc138840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad4800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc158f50 con 0x7f0acc0ae990 2024-03-29T08:18:43.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad4800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc159c70 con 0x7f0acc142a50 2024-03-29T08:18:43.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad4800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0acc159f50 con 0x7f0acc15df90 2024-03-29T08:18:43.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0acc0ae990 0x7f0acc0cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:43.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0acc0ae990 0x7f0acc0cd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43438/0 (socket says 172.21.15.46:43438) 2024-03-29T08:18:43.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad1e00640 1 -- 172.21.15.46:0/1544471049 learned_addr learned my addr 172.21.15.46:0/1544471049 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:43.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad1e00640 1 -- 172.21.15.46:0/1544471049 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0acc15df90 msgr2=0x7f0acc138840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:43.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad1e00640 1 --2- 172.21.15.46:0/1544471049 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0acc15df90 0x7f0acc138840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad1e00640 1 -- 172.21.15.46:0/1544471049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0acc142a50 msgr2=0x7f0acc138300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:43.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad1e00640 1 --2- 172.21.15.46:0/1544471049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0acc142a50 0x7f0acc138300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:43.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad1e00640 1 -- 172.21.15.46:0/1544471049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac0009420 con 0x7f0acc0ae990 2024-03-29T08:18:43.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad1e00640 1 --2- 172.21.15.46:0/1544471049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0acc0ae990 0x7f0acc0cd860 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0ac00097f0 tx=0x7f0ac00191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:43.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ac6000640 1 -- 172.21.15.46:0/1544471049 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00b4610 con 0x7f0acc0ae990 2024-03-29T08:18:43.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ac6000640 1 -- 172.21.15.46:0/1544471049 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac00b4770 con 0x7f0acc0ae990 2024-03-29T08:18:43.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ac6000640 1 -- 172.21.15.46:0/1544471049 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00b4b20 con 0x7f0acc0ae990 2024-03-29T08:18:43.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad4800640 1 -- 172.21.15.46:0/1544471049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0acc15a230 con 0x7f0acc0ae990 2024-03-29T08:18:43.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad4800640 1 -- 172.21.15.46:0/1544471049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0acc0cdda0 con 0x7f0acc0ae990 2024-03-29T08:18:43.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.832+0000 7f0ad4800640 1 -- 172.21.15.46:0/1544471049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0acc0cdf90 con 0x7f0acc0ae990 2024-03-29T08:18:43.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.836+0000 7f0ac6000640 1 -- 172.21.15.46:0/1544471049 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0ac00b2040 con 0x7f0acc0ae990 2024-03-29T08:18:43.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.836+0000 7f0ac6000640 1 --2- 172.21.15.46:0/1544471049 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ab0066690 0x7f0ab0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:43.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.836+0000 7f0ac6000640 1 -- 172.21.15.46:0/1544471049 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ac011e2b0 con 0x7f0acc0ae990 2024-03-29T08:18:43.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.836+0000 7f0ad1400640 1 --2- 172.21.15.46:0/1544471049 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ab0066690 0x7f0ab0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:43.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.836+0000 7f0ad1400640 1 --2- 172.21.15.46:0/1544471049 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ab0066690 0x7f0ab0068b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0abc007fd0 tx=0x7f0abc007960 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:43.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:43.840+0000 7f0ac6000640 1 -- 172.21.15.46:0/1544471049 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0ac00eb610 con 0x7f0acc0ae990 2024-03-29T08:18:44.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.000+0000 7f0ad4800640 1 -- 172.21.15.46:0/1544471049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0acc071a70 con 0x7f0acc0ae990 2024-03-29T08:18:44.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.000+0000 7f0ac6000640 1 -- 172.21.15.46:0/1544471049 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0ac00eec30 con 0x7f0acc0ae990 2024-03-29T08:18:44.003 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:18:44.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.004+0000 7f0ad4800640 1 -- 172.21.15.46:0/1544471049 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ab0066690 msgr2=0x7f0ab0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:44.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.004+0000 7f0ad4800640 1 --2- 172.21.15.46:0/1544471049 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ab0066690 0x7f0ab0068b50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0abc007fd0 tx=0x7f0abc007960 comp rx=0 tx=0).stop 2024-03-29T08:18:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.004+0000 7f0ad4800640 1 -- 172.21.15.46:0/1544471049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0acc0ae990 msgr2=0x7f0acc0cd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.004+0000 7f0ad4800640 1 --2- 172.21.15.46:0/1544471049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0acc0ae990 0x7f0acc0cd860 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0ac00097f0 tx=0x7f0ac00191f0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.004+0000 7f0ad4800640 1 -- 172.21.15.46:0/1544471049 shutdown_connections 2024-03-29T08:18:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.004+0000 7f0ad4800640 1 --2- 172.21.15.46:0/1544471049 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ab0066690 0x7f0ab0068b50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.004+0000 7f0ad4800640 1 --2- 172.21.15.46:0/1544471049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0acc142a50 0x7f0acc138300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.004+0000 7f0ad4800640 1 --2- 172.21.15.46:0/1544471049 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0acc15df90 0x7f0acc138840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.004+0000 7f0ad4800640 1 --2- 172.21.15.46:0/1544471049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0acc0ae990 0x7f0acc0cd860 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.004+0000 7f0ad4800640 1 -- 172.21.15.46:0/1544471049 >> 172.21.15.46:0/1544471049 conn(0x7f0acc06a390 msgr2=0x7f0acc0b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.004+0000 7f0ad4800640 1 -- 172.21.15.46:0/1544471049 shutdown_connections 2024-03-29T08:18:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.004+0000 7f0ad4800640 1 -- 172.21.15.46:0/1544471049 wait complete. 2024-03-29T08:18:44.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:18:44.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35db800640 1 Processor -- start 2024-03-29T08:18:44.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35db800640 1 -- start start 2024-03-29T08:18:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35db800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f35d415c950 0x7f35d406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35db800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f35d40cea30 0x7f35d41606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35db800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f35d415cd30 0x7f35d4160be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35db800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d40cfb20 con 0x7f35d415c950 2024-03-29T08:18:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35db800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d40cef10 con 0x7f35d415cd30 2024-03-29T08:18:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35db800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40cf1f0 con 0x7f35d40cea30 2024-03-29T08:18:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35d8e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f35d415c950 0x7f35d406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35d8e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f35d415c950 0x7f35d406bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43448/0 (socket says 172.21.15.46:43448) 2024-03-29T08:18:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35d8e00640 1 -- 172.21.15.46:0/2481439064 learned_addr learned my addr 172.21.15.46:0/2481439064 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35d9800640 1 --2- 172.21.15.46:0/2481439064 >> v2:172.21.15.73:3300/0 conn(0x7f35d415cd30 0x7f35d4160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35cbe00640 1 --2- 172.21.15.46:0/2481439064 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f35d40cea30 0x7f35d41606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:44.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35d8e00640 1 -- 172.21.15.46:0/2481439064 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f35d40cea30 msgr2=0x7f35d41606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35d8e00640 1 --2- 172.21.15.46:0/2481439064 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f35d40cea30 0x7f35d41606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35d8e00640 1 -- 172.21.15.46:0/2481439064 >> v2:172.21.15.73:3300/0 conn(0x7f35d415cd30 msgr2=0x7f35d4160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35d8e00640 1 --2- 172.21.15.46:0/2481439064 >> v2:172.21.15.73:3300/0 conn(0x7f35d415cd30 0x7f35d4160be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.168+0000 7f35d8e00640 1 -- 172.21.15.46:0/2481439064 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d40cf4d0 con 0x7f35d415c950 2024-03-29T08:18:44.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.172+0000 7f35d8e00640 1 --2- 172.21.15.46:0/2481439064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d415c950 0x7f35d406bbe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f35d4071700 tx=0x7f35c401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb8fadfc9d9e863 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.172+0000 7f35d3e00640 1 -- 172.21.15.46:0/2481439064 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c40a89d0 con 0x7f35d415c950 2024-03-29T08:18:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.172+0000 7f35d3e00640 1 -- 172.21.15.46:0/2481439064 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c40a8b30 con 0x7f35d415c950 2024-03-29T08:18:44.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.172+0000 7f35d3e00640 1 -- 172.21.15.46:0/2481439064 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c40b0440 con 0x7f35d415c950 2024-03-29T08:18:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.172+0000 7f35db800640 1 -- 172.21.15.46:0/2481439064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d415c950 msgr2=0x7f35d406bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.172+0000 7f35db800640 1 --2- 172.21.15.46:0/2481439064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d415c950 0x7f35d406bbe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f35d4071700 tx=0x7f35c401ed10 comp rx=0 tx=0).stop 2024-03-29T08:18:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.172+0000 7f35db800640 1 -- 172.21.15.46:0/2481439064 shutdown_connections 2024-03-29T08:18:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.172+0000 7f35db800640 1 --2- 172.21.15.46:0/2481439064 >> v2:172.21.15.73:3300/0 conn(0x7f35d415cd30 0x7f35d4160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.172+0000 7f35db800640 1 --2- 172.21.15.46:0/2481439064 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f35d40cea30 0x7f35d41606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.172+0000 7f35db800640 1 --2- 172.21.15.46:0/2481439064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d415c950 0x7f35d406bbe0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.172+0000 7f35db800640 1 -- 172.21.15.46:0/2481439064 >> 172.21.15.46:0/2481439064 conn(0x7f35d40b21b0 msgr2=0x7f35d40b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.172+0000 7f35db800640 1 -- 172.21.15.46:0/2481439064 shutdown_connections 2024-03-29T08:18:44.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.172+0000 7f35db800640 1 -- 172.21.15.46:0/2481439064 wait complete. 2024-03-29T08:18:44.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35db800640 1 Processor -- start 2024-03-29T08:18:44.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35db800640 1 -- start start 2024-03-29T08:18:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35db800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f35d40cea30 0x7f35d4157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35db800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d415c950 0x7f35d409b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35db800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35d415cd30 0x7f35d40b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35db800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40b39d0 con 0x7f35d415c950 2024-03-29T08:18:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35db800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d4157cb0 con 0x7f35d415cd30 2024-03-29T08:18:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35db800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d409d510 con 0x7f35d40cea30 2024-03-29T08:18:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35cbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d415c950 0x7f35d409b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35cbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d415c950 0x7f35d409b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43456/0 (socket says 172.21.15.46:43456) 2024-03-29T08:18:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35cbe00640 1 -- 172.21.15.46:0/2023810958 learned_addr learned my addr 172.21.15.46:0/2023810958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35d8e00640 1 --2- 172.21.15.46:0/2023810958 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f35d40cea30 0x7f35d4157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:44.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35d9800640 1 --2- 172.21.15.46:0/2023810958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35d415cd30 0x7f35d40b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35cbe00640 1 -- 172.21.15.46:0/2023810958 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f35d40cea30 msgr2=0x7f35d4157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35cbe00640 1 --2- 172.21.15.46:0/2023810958 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f35d40cea30 0x7f35d4157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35cbe00640 1 -- 172.21.15.46:0/2023810958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35d415cd30 msgr2=0x7f35d40b50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35cbe00640 1 --2- 172.21.15.46:0/2023810958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35d415cd30 0x7f35d40b50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35cbe00640 1 -- 172.21.15.46:0/2023810958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c40094d0 con 0x7f35d415c950 2024-03-29T08:18:44.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35cbe00640 1 --2- 172.21.15.46:0/2023810958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d415c950 0x7f35d409b0e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00d800 tx=0x7f35bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:44.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35d1600640 1 -- 172.21.15.46:0/2023810958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00b6c0 con 0x7f35d415c950 2024-03-29T08:18:44.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35d1600640 1 -- 172.21.15.46:0/2023810958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35bc00b820 con 0x7f35d415c950 2024-03-29T08:18:44.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35db800640 1 -- 172.21.15.46:0/2023810958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d40b5b10 con 0x7f35d415c950 2024-03-29T08:18:44.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35d1600640 1 -- 172.21.15.46:0/2023810958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00bb90 con 0x7f35d415c950 2024-03-29T08:18:44.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35db800640 1 -- 172.21.15.46:0/2023810958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d40b35f0 con 0x7f35d415c950 2024-03-29T08:18:44.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.176+0000 7f35db800640 1 -- 172.21.15.46:0/2023810958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d409e690 con 0x7f35d415c950 2024-03-29T08:18:44.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.180+0000 7f35d1600640 1 -- 172.21.15.46:0/2023810958 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f35bc00bcf0 con 0x7f35d415c950 2024-03-29T08:18:44.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.180+0000 7f35d1600640 1 --2- 172.21.15.46:0/2023810958 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35b0066690 0x7f35b0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:44.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.180+0000 7f35d1600640 1 -- 172.21.15.46:0/2023810958 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35bc085160 con 0x7f35d415c950 2024-03-29T08:18:44.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.180+0000 7f35d8e00640 1 --2- 172.21.15.46:0/2023810958 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35b0066690 0x7f35b0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:44.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.180+0000 7f35d8e00640 1 --2- 172.21.15.46:0/2023810958 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35b0066690 0x7f35b0068b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f35c40b2000 tx=0x7f35c4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:44.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.184+0000 7f35d1600640 1 -- 172.21.15.46:0/2023810958 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f35bc0524c0 con 0x7f35d415c950 2024-03-29T08:18:44.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.344+0000 7f35db800640 1 -- 172.21.15.46:0/2023810958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f35d406c600 con 0x7f35d415c950 2024-03-29T08:18:44.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.344+0000 7f35d1600640 1 -- 172.21.15.46:0/2023810958 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f35bc055ae0 con 0x7f35d415c950 2024-03-29T08:18:44.346 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:18:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.344+0000 7f35db800640 1 -- 172.21.15.46:0/2023810958 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35b0066690 msgr2=0x7f35b0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.344+0000 7f35db800640 1 --2- 172.21.15.46:0/2023810958 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35b0066690 0x7f35b0068b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f35c40b2000 tx=0x7f35c4009040 comp rx=0 tx=0).stop 2024-03-29T08:18:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.344+0000 7f35db800640 1 -- 172.21.15.46:0/2023810958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d415c950 msgr2=0x7f35d409b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.344+0000 7f35db800640 1 --2- 172.21.15.46:0/2023810958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d415c950 0x7f35d409b0e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00d800 tx=0x7f35bc00dcc0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.344+0000 7f35db800640 1 -- 172.21.15.46:0/2023810958 shutdown_connections 2024-03-29T08:18:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.344+0000 7f35db800640 1 --2- 172.21.15.46:0/2023810958 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35b0066690 0x7f35b0068b50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.344+0000 7f35db800640 1 --2- 172.21.15.46:0/2023810958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35d415cd30 0x7f35d40b50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.344+0000 7f35db800640 1 --2- 172.21.15.46:0/2023810958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35d415c950 0x7f35d409b0e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.344+0000 7f35db800640 1 --2- 172.21.15.46:0/2023810958 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f35d40cea30 0x7f35d4157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.344+0000 7f35db800640 1 -- 172.21.15.46:0/2023810958 >> 172.21.15.46:0/2023810958 conn(0x7f35d40b21b0 msgr2=0x7f35d415b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.348+0000 7f35db800640 1 -- 172.21.15.46:0/2023810958 shutdown_connections 2024-03-29T08:18:44.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.348+0000 7f35db800640 1 -- 172.21.15.46:0/2023810958 wait complete. 2024-03-29T08:18:44.376 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-29T08:18:44.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-03-29T08:18:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.516+0000 7f26cfa00640 1 Processor -- start 2024-03-29T08:18:44.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.516+0000 7f26cfa00640 1 -- start start 2024-03-29T08:18:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.516+0000 7f26cfa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f26c815ada0 0x7f26c80ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.516+0000 7f26cfa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c80cb510 con 0x7f26c8144210 2024-03-29T08:18:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.516+0000 7f26cfa00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c8070580 con 0x7f26c80ce950 2024-03-29T08:18:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.516+0000 7f26cfa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c80cede0 con 0x7f26c815ada0 2024-03-29T08:18:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.516+0000 7f26cd000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f26c8144210 0x7f26c80cb8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54078/0 (socket says 172.21.15.46:54078) 2024-03-29T08:18:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.516+0000 7f26cd000640 1 -- 172.21.15.46:0/1079995873 learned_addr learned my addr 172.21.15.46:0/1079995873 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:44.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.516+0000 7f26bfe00640 1 --2- 172.21.15.46:0/1079995873 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f26c815ada0 0x7f26c80ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.516+0000 7f26c7e00640 1 -- 172.21.15.46:0/1079995873 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992628525 0 0) 0x7f26c80cb510 con 0x7f26c8144210 2024-03-29T08:18:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.516+0000 7f26c7e00640 1 -- 172.21.15.46:0/1079995873 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26ac003530 con 0x7f26c8144210 2024-03-29T08:18:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.516+0000 7f26c7e00640 1 -- 172.21.15.46:0/1079995873 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317683214 0 0) 0x7f26c8070580 con 0x7f26c80ce950 2024-03-29T08:18:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.516+0000 7f26c7e00640 1 -- 172.21.15.46:0/1079995873 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26c80cb510 con 0x7f26c80ce950 2024-03-29T08:18:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26c7e00640 1 -- 172.21.15.46:0/1079995873 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2427529982 0 0) 0x7f26ac003530 con 0x7f26c8144210 2024-03-29T08:18:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26c7e00640 1 -- 172.21.15.46:0/1079995873 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f26c815ada0 msgr2=0x7f26c80ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26c7e00640 1 --2- 172.21.15.46:0/1079995873 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f26c815ada0 0x7f26c80ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26c7e00640 1 -- 172.21.15.46:0/1079995873 >> v1:172.21.15.73:6789/0 conn(0x7f26c80ce950 legacy=0x7f26c806c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:44.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26c7e00640 1 -- 172.21.15.46:0/1079995873 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c8070860 con 0x7f26c8144210 2024-03-29T08:18:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26c7e00640 1 -- 172.21.15.46:0/1079995873 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f26b8001680 con 0x7f26c8144210 2024-03-29T08:18:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26bfe00640 1 --2- 172.21.15.46:0/1079995873 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f26c815ada0 0x7f26c80ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26c7e00640 1 -- 172.21.15.46:0/1079995873 >> v1:172.21.15.46:6789/0 conn(0x7f26c8144210 legacy=0x7f26c80cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26c7e00640 1 --2- 172.21.15.46:0/1079995873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26ac003b40 0x7f26ac003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26c7e00640 1 --2- 172.21.15.46:0/1079995873 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f26c815ada0 0x7f26ac0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26c7e00640 1 --2- 172.21.15.46:0/1079995873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac004d10 0x7f26ac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26c7e00640 1 -- 172.21.15.46:0/1079995873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c80cede0 con 0x7f26ac004d10 2024-03-29T08:18:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26c7e00640 1 -- 172.21.15.46:0/1079995873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac00ba40 con 0x7f26ac003b40 2024-03-29T08:18:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26c7e00640 1 -- 172.21.15.46:0/1079995873 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26ac00bb80 con 0x7f26c815ada0 2024-03-29T08:18:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26cda00640 1 --2- 172.21.15.46:0/1079995873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac004d10 0x7f26ac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26cd000640 1 --2- 172.21.15.46:0/1079995873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26ac003b40 0x7f26ac003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26bfe00640 1 --2- 172.21.15.46:0/1079995873 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f26c815ada0 0x7f26ac0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26cda00640 1 -- 172.21.15.46:0/1079995873 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f26c815ada0 msgr2=0x7f26ac0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:44.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26cda00640 1 --2- 172.21.15.46:0/1079995873 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f26c815ada0 0x7f26ac0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26cda00640 1 -- 172.21.15.46:0/1079995873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26ac003b40 msgr2=0x7f26ac003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:44.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26cda00640 1 --2- 172.21.15.46:0/1079995873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26ac003b40 0x7f26ac003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26cda00640 1 -- 172.21.15.46:0/1079995873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b0009420 con 0x7f26ac004d10 2024-03-29T08:18:44.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26cda00640 1 --2- 172.21.15.46:0/1079995873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac004d10 0x7f26ac005190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f26c0002000 tx=0x7f26c001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=578962ad1b366d65 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:44.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26c7e00640 1 -- 172.21.15.46:0/1079995873 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8007ea0 con 0x7f26ac004d10 2024-03-29T08:18:44.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26c7e00640 1 -- 172.21.15.46:0/1079995873 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c0003ba0 con 0x7f26ac004d10 2024-03-29T08:18:44.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.520+0000 7f26c7e00640 1 -- 172.21.15.46:0/1079995873 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c00050d0 con 0x7f26ac004d10 2024-03-29T08:18:44.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cfa00640 1 -- 172.21.15.46:0/1079995873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac004d10 msgr2=0x7f26ac005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:44.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cfa00640 1 --2- 172.21.15.46:0/1079995873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26ac004d10 0x7f26ac005190 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f26c0002000 tx=0x7f26c001c900 comp rx=0 tx=0).stop 2024-03-29T08:18:44.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cda00640 1 -- 172.21.15.46:0/1079995873 reap_dead start 2024-03-29T08:18:44.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cfa00640 1 -- 172.21.15.46:0/1079995873 shutdown_connections 2024-03-29T08:18:44.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cfa00640 1 -- 172.21.15.46:0/1079995873 >> 172.21.15.46:0/1079995873 conn(0x7f26c8066110 msgr2=0x7f26c8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:44.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cfa00640 1 -- 172.21.15.46:0/1079995873 shutdown_connections 2024-03-29T08:18:44.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cfa00640 1 -- 172.21.15.46:0/1079995873 wait complete. 2024-03-29T08:18:44.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cfa00640 1 Processor -- start 2024-03-29T08:18:44.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cfa00640 1 -- start start 2024-03-29T08:18:44.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cfa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26ac004d10 0x7f26c815b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:44.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cfa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f26c8134e00 0x7f26c80b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:44.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cfa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8159c00 0x7f26c815a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:44.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cfa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c815c6d0 con 0x7f26c8159c00 2024-03-29T08:18:44.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cfa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c815a5c0 con 0x7f26ac004d10 2024-03-29T08:18:44.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cfa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c815ba50 con 0x7f26c8134e00 2024-03-29T08:18:44.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8159c00 0x7f26c815a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:44.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8159c00 0x7f26c815a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43476/0 (socket says 172.21.15.46:43476) 2024-03-29T08:18:44.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cda00640 1 -- 172.21.15.46:0/2311251925 learned_addr learned my addr 172.21.15.46:0/2311251925 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cd000640 1 --2- 172.21.15.46:0/2311251925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26ac004d10 0x7f26c815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26bfe00640 1 --2- 172.21.15.46:0/2311251925 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f26c8134e00 0x7f26c80b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cda00640 1 -- 172.21.15.46:0/2311251925 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f26c8134e00 msgr2=0x7f26c80b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cda00640 1 --2- 172.21.15.46:0/2311251925 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f26c8134e00 0x7f26c80b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cda00640 1 -- 172.21.15.46:0/2311251925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26ac004d10 msgr2=0x7f26c815b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cda00640 1 --2- 172.21.15.46:0/2311251925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26ac004d10 0x7f26c815b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.524+0000 7f26cda00640 1 -- 172.21.15.46:0/2311251925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c815bd30 con 0x7f26c8159c00 2024-03-29T08:18:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.528+0000 7f26cda00640 1 --2- 172.21.15.46:0/2311251925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8159c00 0x7f26c815a080 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f26c0005040 tx=0x7f26ac005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.528+0000 7f26c5600640 1 -- 172.21.15.46:0/2311251925 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c00b2040 con 0x7f26c8159c00 2024-03-29T08:18:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.528+0000 7f26c5600640 1 -- 172.21.15.46:0/2311251925 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c00050d0 con 0x7f26c8159c00 2024-03-29T08:18:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.528+0000 7f26c5600640 1 -- 172.21.15.46:0/2311251925 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c0005480 con 0x7f26c8159c00 2024-03-29T08:18:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.528+0000 7f26cfa00640 1 -- 172.21.15.46:0/2311251925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c815c010 con 0x7f26c8159c00 2024-03-29T08:18:44.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.528+0000 7f26cfa00640 1 -- 172.21.15.46:0/2311251925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c815b740 con 0x7f26c8159c00 2024-03-29T08:18:44.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.528+0000 7f26cfa00640 1 -- 172.21.15.46:0/2311251925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c80ce280 con 0x7f26c8159c00 2024-03-29T08:18:44.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.528+0000 7f26c5600640 1 -- 172.21.15.46:0/2311251925 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f26c00055e0 con 0x7f26c8159c00 2024-03-29T08:18:44.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.528+0000 7f26c5600640 1 --2- 172.21.15.46:0/2311251925 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f26a40666c0 0x7f26a4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:44.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.528+0000 7f26c5600640 1 -- 172.21.15.46:0/2311251925 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26c00aa070 con 0x7f26c8159c00 2024-03-29T08:18:44.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.528+0000 7f26cd000640 1 --2- 172.21.15.46:0/2311251925 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f26a40666c0 0x7f26a4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:44.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.532+0000 7f26cd000640 1 --2- 172.21.15.46:0/2311251925 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f26a40666c0 0x7f26a4068b80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f26b8001c30 tx=0x7f26b8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:44.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.532+0000 7f26c5600640 1 -- 172.21.15.46:0/2311251925 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f26c00ead70 con 0x7f26c8159c00 2024-03-29T08:18:44.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.752+0000 7f26cfa00640 1 -- 172.21.15.46:0/2311251925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f26c815c810 con 0x7f26c8159c00 2024-03-29T08:18:44.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.984+0000 7f26c5600640 1 -- 172.21.15.46:0/2311251925 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v51) v1 ==== 145+0+0 (secure 0 0 0) 0x7f26c815c810 con 0x7f26c8159c00 2024-03-29T08:18:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.988+0000 7f26cfa00640 1 -- 172.21.15.46:0/2311251925 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f26a40666c0 msgr2=0x7f26a4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.988+0000 7f26cfa00640 1 --2- 172.21.15.46:0/2311251925 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f26a40666c0 0x7f26a4068b80 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f26b8001c30 tx=0x7f26b8009040 comp rx=0 tx=0).stop 2024-03-29T08:18:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.988+0000 7f26cfa00640 1 -- 172.21.15.46:0/2311251925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8159c00 msgr2=0x7f26c815a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.988+0000 7f26cfa00640 1 --2- 172.21.15.46:0/2311251925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8159c00 0x7f26c815a080 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f26c0005040 tx=0x7f26ac005800 comp rx=0 tx=0).stop 2024-03-29T08:18:44.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.988+0000 7f26cfa00640 1 -- 172.21.15.46:0/2311251925 shutdown_connections 2024-03-29T08:18:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.988+0000 7f26cfa00640 1 --2- 172.21.15.46:0/2311251925 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f26a40666c0 0x7f26a4068b80 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.988+0000 7f26cfa00640 1 --2- 172.21.15.46:0/2311251925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26c8159c00 0x7f26c815a080 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.988+0000 7f26cfa00640 1 --2- 172.21.15.46:0/2311251925 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f26c8134e00 0x7f26c80b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.988+0000 7f26cfa00640 1 --2- 172.21.15.46:0/2311251925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26ac004d10 0x7f26c815b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.988+0000 7f26cfa00640 1 -- 172.21.15.46:0/2311251925 >> 172.21.15.46:0/2311251925 conn(0x7f26c8066110 msgr2=0x7f26c80b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.988+0000 7f26cfa00640 1 -- 172.21.15.46:0/2311251925 shutdown_connections 2024-03-29T08:18:44.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:44.988+0000 7f26cfa00640 1 -- 172.21.15.46:0/2311251925 wait complete. 2024-03-29T08:18:45.008 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-29T08:18:45.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.152+0000 7f25a3400640 1 Processor -- start 2024-03-29T08:18:45.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.152+0000 7f25a3400640 1 -- start start 2024-03-29T08:18:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.152+0000 7f25a3400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f259c15c9c0 0x7f259c0ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.152+0000 7f25a3400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f259c06be50 0x7f259c098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.152+0000 7f25a3400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f259c0cfe50 con 0x7f259c098f80 2024-03-29T08:18:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.152+0000 7f25a3400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f259c0cf030 con 0x7f259c15c9c0 2024-03-29T08:18:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.152+0000 7f25a3400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0993f0 con 0x7f259c06be50 2024-03-29T08:18:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.152+0000 7f25a1400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f259c098f80 0x7f259c1605e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54086/0 (socket says 172.21.15.46:54086) 2024-03-29T08:18:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.152+0000 7f25a1400640 1 -- 172.21.15.46:0/2417730247 learned_addr learned my addr 172.21.15.46:0/2417730247 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.152+0000 7f25a0a00640 1 --2- 172.21.15.46:0/2417730247 >> v2:172.21.15.73:3300/0 conn(0x7f259c15c9c0 0x7f259c0ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.152+0000 7f2593e00640 1 --2- 172.21.15.46:0/2417730247 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f259c06be50 0x7f259c098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.152+0000 7f259be00640 1 -- 172.21.15.46:0/2417730247 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636624963 0 0) 0x7f259c0cfe50 con 0x7f259c098f80 2024-03-29T08:18:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.152+0000 7f259be00640 1 -- 172.21.15.46:0/2417730247 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2580003530 con 0x7f259c098f80 2024-03-29T08:18:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.152+0000 7f25a0a00640 1 -- 172.21.15.46:0/2417730247 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f259c06be50 msgr2=0x7f259c098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.152+0000 7f25a0a00640 1 --2- 172.21.15.46:0/2417730247 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f259c06be50 0x7f259c098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.152+0000 7f25a0a00640 1 -- 172.21.15.46:0/2417730247 >> v1:172.21.15.46:6789/0 conn(0x7f259c098f80 legacy=0x7f259c1605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.152+0000 7f25a0a00640 1 -- 172.21.15.46:0/2417730247 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259c0cf240 con 0x7f259c15c9c0 2024-03-29T08:18:45.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.156+0000 7f25a0a00640 1 --2- 172.21.15.46:0/2417730247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f259c15c9c0 0x7f259c0ceaf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f258c0097c0 tx=0x7f258c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4a595d8f38e2237 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:45.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.156+0000 7f259be00640 1 -- 172.21.15.46:0/2417730247 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0ab600 con 0x7f259c15c9c0 2024-03-29T08:18:45.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.156+0000 7f259be00640 1 -- 172.21.15.46:0/2417730247 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c0a7050 con 0x7f259c15c9c0 2024-03-29T08:18:45.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.156+0000 7f259be00640 1 -- 172.21.15.46:0/2417730247 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0ab960 con 0x7f259c15c9c0 2024-03-29T08:18:45.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.156+0000 7f25a3400640 1 -- 172.21.15.46:0/2417730247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f259c15c9c0 msgr2=0x7f259c0ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:45.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.156+0000 7f25a3400640 1 --2- 172.21.15.46:0/2417730247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f259c15c9c0 0x7f259c0ceaf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f258c0097c0 tx=0x7f258c01c4a0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.156+0000 7f25a3400640 1 -- 172.21.15.46:0/2417730247 shutdown_connections 2024-03-29T08:18:45.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.156+0000 7f25a3400640 1 --2- 172.21.15.46:0/2417730247 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f259c06be50 0x7f259c098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.156+0000 7f25a3400640 1 --2- 172.21.15.46:0/2417730247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f259c15c9c0 0x7f259c0ceaf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.156+0000 7f25a3400640 1 -- 172.21.15.46:0/2417730247 >> 172.21.15.46:0/2417730247 conn(0x7f259c0b2220 msgr2=0x7f259c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.156+0000 7f25a3400640 1 -- 172.21.15.46:0/2417730247 shutdown_connections 2024-03-29T08:18:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.156+0000 7f25a3400640 1 -- 172.21.15.46:0/2417730247 wait complete. 2024-03-29T08:18:45.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f25a3400640 1 Processor -- start 2024-03-29T08:18:45.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f25a3400640 1 -- start start 2024-03-29T08:18:45.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f25a3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f259c06be50 0x7f259c06ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:45.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f25a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f259c098f80 0x7f259c159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:45.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f25a3400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f259c15c9c0 0x7f259c0cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f25a3400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0c8e10 con 0x7f259c098f80 2024-03-29T08:18:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f25a3400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c06f510 con 0x7f259c06be50 2024-03-29T08:18:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f25a3400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259c06f7f0 con 0x7f259c15c9c0 2024-03-29T08:18:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f2593e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f259c098f80 0x7f259c159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f2593e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f259c098f80 0x7f259c159490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43484/0 (socket says 172.21.15.46:43484) 2024-03-29T08:18:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f2593e00640 1 -- 172.21.15.46:0/3718325711 learned_addr learned my addr 172.21.15.46:0/3718325711 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f25a1400640 1 --2- 172.21.15.46:0/3718325711 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f259c15c9c0 0x7f259c0cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:45.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f25a0a00640 1 --2- 172.21.15.46:0/3718325711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f259c06be50 0x7f259c06ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:45.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f2593e00640 1 -- 172.21.15.46:0/3718325711 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f259c15c9c0 msgr2=0x7f259c0cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:45.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f2593e00640 1 --2- 172.21.15.46:0/3718325711 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f259c15c9c0 0x7f259c0cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f2593e00640 1 -- 172.21.15.46:0/3718325711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f259c06be50 msgr2=0x7f259c06ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:45.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f2593e00640 1 --2- 172.21.15.46:0/3718325711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f259c06be50 0x7f259c06ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f2593e00640 1 -- 172.21.15.46:0/3718325711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c009420 con 0x7f259c098f80 2024-03-29T08:18:45.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f25a1400640 1 --2- 172.21.15.46:0/3718325711 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f259c15c9c0 0x7f259c0cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:45.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f25a0a00640 1 --2- 172.21.15.46:0/3718325711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f259c06be50 0x7f259c06ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:45.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f2593e00640 1 --2- 172.21.15.46:0/3718325711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f259c098f80 0x7f259c159490 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f258400e870 tx=0x7f258400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:45.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f2599600640 1 -- 172.21.15.46:0/3718325711 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2584009510 con 0x7f259c098f80 2024-03-29T08:18:45.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f2599600640 1 -- 172.21.15.46:0/3718325711 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258400eec0 con 0x7f259c098f80 2024-03-29T08:18:45.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f2599600640 1 -- 172.21.15.46:0/3718325711 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258400cb40 con 0x7f259c098f80 2024-03-29T08:18:45.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f25a3400640 1 -- 172.21.15.46:0/3718325711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259c06fad0 con 0x7f259c098f80 2024-03-29T08:18:45.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f25a3400640 1 -- 172.21.15.46:0/3718325711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c1599d0 con 0x7f259c098f80 2024-03-29T08:18:45.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.160+0000 7f25a3400640 1 -- 172.21.15.46:0/3718325711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259c0c8be0 con 0x7f259c098f80 2024-03-29T08:18:45.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.164+0000 7f2599600640 1 -- 172.21.15.46:0/3718325711 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f258400ccc0 con 0x7f259c098f80 2024-03-29T08:18:45.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.164+0000 7f2599600640 1 --2- 172.21.15.46:0/3718325711 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2578066690 0x7f2578068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:45.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.164+0000 7f2599600640 1 -- 172.21.15.46:0/3718325711 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2584084350 con 0x7f259c098f80 2024-03-29T08:18:45.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.164+0000 7f25a0a00640 1 --2- 172.21.15.46:0/3718325711 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2578066690 0x7f2578068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.164+0000 7f25a0a00640 1 --2- 172.21.15.46:0/3718325711 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2578066690 0x7f2578068b50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f258c0ad000 tx=0x7f258c0a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.168+0000 7f2599600640 1 -- 172.21.15.46:0/3718325711 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f259c0c8be0 con 0x7f259c098f80 2024-03-29T08:18:45.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.328+0000 7f25a3400640 1 -- 172.21.15.46:0/3718325711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f259c0c8be0 con 0x7f259c098f80 2024-03-29T08:18:45.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.332+0000 7f2599600640 1 -- 172.21.15.46:0/3718325711 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f259c0c8be0 con 0x7f259c098f80 2024-03-29T08:18:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.336+0000 7f25a3400640 1 -- 172.21.15.46:0/3718325711 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2578066690 msgr2=0x7f2578068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.336+0000 7f25a3400640 1 --2- 172.21.15.46:0/3718325711 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2578066690 0x7f2578068b50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f258c0ad000 tx=0x7f258c0a6000 comp rx=0 tx=0).stop 2024-03-29T08:18:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.336+0000 7f25a3400640 1 -- 172.21.15.46:0/3718325711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f259c098f80 msgr2=0x7f259c159490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.336+0000 7f25a3400640 1 --2- 172.21.15.46:0/3718325711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f259c098f80 0x7f259c159490 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f258400e870 tx=0x7f258400ec70 comp rx=0 tx=0).stop 2024-03-29T08:18:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.336+0000 7f25a3400640 1 -- 172.21.15.46:0/3718325711 shutdown_connections 2024-03-29T08:18:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.336+0000 7f25a3400640 1 --2- 172.21.15.46:0/3718325711 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2578066690 0x7f2578068b50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.336+0000 7f25a3400640 1 --2- 172.21.15.46:0/3718325711 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f259c15c9c0 0x7f259c0cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.336+0000 7f25a3400640 1 --2- 172.21.15.46:0/3718325711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f259c098f80 0x7f259c159490 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.336+0000 7f25a3400640 1 --2- 172.21.15.46:0/3718325711 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f259c06be50 0x7f259c06ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.336+0000 7f25a3400640 1 -- 172.21.15.46:0/3718325711 >> 172.21.15.46:0/3718325711 conn(0x7f259c0b2220 msgr2=0x7f259c0cd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.336+0000 7f25a3400640 1 -- 172.21.15.46:0/3718325711 shutdown_connections 2024-03-29T08:18:45.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.336+0000 7f25a3400640 1 -- 172.21.15.46:0/3718325711 wait complete. 2024-03-29T08:18:45.356 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T08:18:45.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:18:45.430 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:18:45.491 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T08:18:45.491 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:18:45.491 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:18:45.500 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:18:45.500 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:18:45.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:45.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.732+0000 7fd279600640 1 Processor -- start 2024-03-29T08:18:45.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.736+0000 7fd279600640 1 -- start start 2024-03-29T08:18:45.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.736+0000 7fd279600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd27415ada0 0x7fd2740ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:45.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.736+0000 7fd279600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd2740ce950 0x7fd27406c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:45.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.736+0000 7fd279600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2740cb510 con 0x7fd274144210 2024-03-29T08:18:45.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.736+0000 7fd279600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2740cede0 con 0x7fd27415ada0 2024-03-29T08:18:45.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.736+0000 7fd279600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd274070580 con 0x7fd2740ce950 2024-03-29T08:18:45.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.736+0000 7fd26a000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd27415ada0 0x7fd2740ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:45.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.736+0000 7fd26a000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd27415ada0 0x7fd2740ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43500/0 (socket says 172.21.15.46:43500) 2024-03-29T08:18:45.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.736+0000 7fd26a000640 1 -- 172.21.15.46:0/531011336 learned_addr learned my addr 172.21.15.46:0/531011336 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:45.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.736+0000 7fd273400640 1 --2- 172.21.15.46:0/531011336 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd2740ce950 0x7fd27406c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:45.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.736+0000 7fd26a000640 1 -- 172.21.15.46:0/531011336 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd2740ce950 msgr2=0x7fd27406c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:45.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.736+0000 7fd26a000640 1 --2- 172.21.15.46:0/531011336 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd2740ce950 0x7fd27406c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.736+0000 7fd26a000640 1 -- 172.21.15.46:0/531011336 >> v1:172.21.15.73:6789/0 conn(0x7fd274144210 legacy=0x7fd2740cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:45.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.736+0000 7fd26a000640 1 -- 172.21.15.46:0/531011336 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd274070860 con 0x7fd27415ada0 2024-03-29T08:18:45.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.740+0000 7fd26a000640 1 --2- 172.21.15.46:0/531011336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27415ada0 0x7fd2740ce410 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009e50 tx=0x7fd25c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12c490be0310465 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:45.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.740+0000 7fd272000640 1 -- 172.21.15.46:0/531011336 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c0a57f0 con 0x7fd27415ada0 2024-03-29T08:18:45.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.740+0000 7fd272000640 1 -- 172.21.15.46:0/531011336 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25c0a5950 con 0x7fd27415ada0 2024-03-29T08:18:45.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.740+0000 7fd272000640 1 -- 172.21.15.46:0/531011336 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c0a5ce0 con 0x7fd27415ada0 2024-03-29T08:18:45.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.740+0000 7fd279600640 1 -- 172.21.15.46:0/531011336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27415ada0 msgr2=0x7fd2740ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:45.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.740+0000 7fd279600640 1 --2- 172.21.15.46:0/531011336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27415ada0 0x7fd2740ce410 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009e50 tx=0x7fd25c01ca30 comp rx=0 tx=0).stop 2024-03-29T08:18:45.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.740+0000 7fd279600640 1 -- 172.21.15.46:0/531011336 shutdown_connections 2024-03-29T08:18:45.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.740+0000 7fd279600640 1 --2- 172.21.15.46:0/531011336 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd2740ce950 0x7fd27406c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.740+0000 7fd279600640 1 --2- 172.21.15.46:0/531011336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd27415ada0 0x7fd2740ce410 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.740+0000 7fd279600640 1 -- 172.21.15.46:0/531011336 >> 172.21.15.46:0/531011336 conn(0x7fd274066110 msgr2=0x7fd274069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:45.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.740+0000 7fd279600640 1 -- 172.21.15.46:0/531011336 shutdown_connections 2024-03-29T08:18:45.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.740+0000 7fd279600640 1 -- 172.21.15.46:0/531011336 wait complete. 2024-03-29T08:18:45.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.740+0000 7fd279600640 1 Processor -- start 2024-03-29T08:18:45.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.740+0000 7fd279600640 1 -- start start 2024-03-29T08:18:45.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd279600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2740ce950 0x7fd2740b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:45.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd279600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd274144210 0x7fd2741641d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:45.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd279600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd27415ada0 0x7fd2741426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:45.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd279600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2740b34c0 con 0x7fd2740ce950 2024-03-29T08:18:45.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd279600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2740b2560 con 0x7fd274144210 2024-03-29T08:18:45.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd279600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2740b2840 con 0x7fd27415ada0 2024-03-29T08:18:45.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd272a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2740ce950 0x7fd2740b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:45.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd272a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2740ce950 0x7fd2740b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43508/0 (socket says 172.21.15.46:43508) 2024-03-29T08:18:45.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd272a00640 1 -- 172.21.15.46:0/1727920875 learned_addr learned my addr 172.21.15.46:0/1727920875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:45.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd26a000640 1 --2- 172.21.15.46:0/1727920875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd274144210 0x7fd2741641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:45.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd273400640 1 --2- 172.21.15.46:0/1727920875 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd27415ada0 0x7fd2741426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:45.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd272a00640 1 -- 172.21.15.46:0/1727920875 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd27415ada0 msgr2=0x7fd2741426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:45.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd272a00640 1 --2- 172.21.15.46:0/1727920875 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd27415ada0 0x7fd2741426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd272a00640 1 -- 172.21.15.46:0/1727920875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd274144210 msgr2=0x7fd2741641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:45.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd272a00640 1 --2- 172.21.15.46:0/1727920875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd274144210 0x7fd2741641d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd272a00640 1 -- 172.21.15.46:0/1727920875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c009b30 con 0x7fd2740ce950 2024-03-29T08:18:45.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd272a00640 1 --2- 172.21.15.46:0/1727920875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2740ce950 0x7fd2740b1e80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd26400c500 tx=0x7fd26400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:45.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd26b400640 1 -- 172.21.15.46:0/1727920875 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264005220 con 0x7fd2740ce950 2024-03-29T08:18:45.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd26b400640 1 -- 172.21.15.46:0/1727920875 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd264005380 con 0x7fd2740ce950 2024-03-29T08:18:45.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd279600640 1 -- 172.21.15.46:0/1727920875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2740b2b20 con 0x7fd2740ce950 2024-03-29T08:18:45.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd26b400640 1 -- 172.21.15.46:0/1727920875 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264007270 con 0x7fd2740ce950 2024-03-29T08:18:45.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd279600640 1 -- 172.21.15.46:0/1727920875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2740b30e0 con 0x7fd2740ce950 2024-03-29T08:18:45.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd279600640 1 -- 172.21.15.46:0/1727920875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2740ce280 con 0x7fd2740ce950 2024-03-29T08:18:45.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.744+0000 7fd26b400640 1 -- 172.21.15.46:0/1727920875 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd2640073d0 con 0x7fd2740ce950 2024-03-29T08:18:45.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.748+0000 7fd26b400640 1 --2- 172.21.15.46:0/1727920875 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd250066690 0x7fd250068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:45.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.748+0000 7fd26b400640 1 -- 172.21.15.46:0/1727920875 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd2640839e0 con 0x7fd2740ce950 2024-03-29T08:18:45.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.748+0000 7fd26a000640 1 --2- 172.21.15.46:0/1727920875 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd250066690 0x7fd250068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:45.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.748+0000 7fd26a000640 1 --2- 172.21.15.46:0/1727920875 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd250066690 0x7fd250068b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0ae000 tx=0x7fd25c0a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:45.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.752+0000 7fd26b400640 1 -- 172.21.15.46:0/1727920875 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd264050d70 con 0x7fd2740ce950 2024-03-29T08:18:45.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.912+0000 7fd279600640 1 -- 172.21.15.46:0/1727920875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2740b32d0 con 0x7fd2740ce950 2024-03-29T08:18:45.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.912+0000 7fd26b400640 1 -- 172.21.15.46:0/1727920875 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15993 (secure 0 0 0) 0x7fd264054390 con 0x7fd2740ce950 2024-03-29T08:18:45.915 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:45.915 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:14.314124+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:45.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.916+0000 7fd279600640 1 -- 172.21.15.46:0/1727920875 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd250066690 msgr2=0x7fd250068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:45.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.916+0000 7fd279600640 1 --2- 172.21.15.46:0/1727920875 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd250066690 0x7fd250068b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0ae000 tx=0x7fd25c0a7070 comp rx=0 tx=0).stop 2024-03-29T08:18:45.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.916+0000 7fd279600640 1 -- 172.21.15.46:0/1727920875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2740ce950 msgr2=0x7fd2740b1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:45.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.916+0000 7fd279600640 1 --2- 172.21.15.46:0/1727920875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2740ce950 0x7fd2740b1e80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd26400c500 tx=0x7fd26400c9c0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.916+0000 7fd279600640 1 -- 172.21.15.46:0/1727920875 shutdown_connections 2024-03-29T08:18:45.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.916+0000 7fd279600640 1 --2- 172.21.15.46:0/1727920875 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd250066690 0x7fd250068b50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.916+0000 7fd279600640 1 --2- 172.21.15.46:0/1727920875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd274144210 0x7fd2741641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.916+0000 7fd279600640 1 --2- 172.21.15.46:0/1727920875 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd27415ada0 0x7fd2741426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.916+0000 7fd279600640 1 --2- 172.21.15.46:0/1727920875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd2740ce950 0x7fd2740b1e80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:45.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.916+0000 7fd279600640 1 -- 172.21.15.46:0/1727920875 >> 172.21.15.46:0/1727920875 conn(0x7fd274066110 msgr2=0x7fd2740cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:45.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.916+0000 7fd279600640 1 -- 172.21.15.46:0/1727920875 shutdown_connections 2024-03-29T08:18:45.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:45.916+0000 7fd279600640 1 -- 172.21.15.46:0/1727920875 wait complete. 2024-03-29T08:18:45.936 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:18:11.177842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:18:12.308252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:18:45.937 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:18:46.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.080+0000 7fb6fc400640 1 Processor -- start 2024-03-29T08:18:46.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.080+0000 7fb6fc400640 1 -- start start 2024-03-29T08:18:46.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.080+0000 7fb6fc400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6f415a790 0x7fb6f40cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.080+0000 7fb6fc400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6f40cbbe0 0x7fb6f40d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.080+0000 7fb6fc400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f40cc660 con 0x7fb6f40cf340 2024-03-29T08:18:46.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.080+0000 7fb6fc400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f40cc0a0 con 0x7fb6f415a790 2024-03-29T08:18:46.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.080+0000 7fb6fc400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f40cc380 con 0x7fb6f40cbbe0 2024-03-29T08:18:46.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.080+0000 7fb6f9a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6f415a790 0x7fb6f40cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.080+0000 7fb6fa400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb6f40cf340 0x7fb6f40ccf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54088/0 (socket says 172.21.15.46:54088) 2024-03-29T08:18:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.080+0000 7fb6f9a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6f415a790 0x7fb6f40cee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43510/0 (socket says 172.21.15.46:43510) 2024-03-29T08:18:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.080+0000 7fb6fa400640 1 -- 172.21.15.46:0/2592310962 learned_addr learned my addr 172.21.15.46:0/2592310962 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.080+0000 7fb6f9000640 1 --2- 172.21.15.46:0/2592310962 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6f40cbbe0 0x7fb6f40d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.080+0000 7fb6f9a00640 1 -- 172.21.15.46:0/2592310962 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6f40cbbe0 msgr2=0x7fb6f40d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:46.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.080+0000 7fb6f9a00640 1 --2- 172.21.15.46:0/2592310962 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6f40cbbe0 0x7fb6f40d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.080+0000 7fb6f9a00640 1 -- 172.21.15.46:0/2592310962 >> v1:172.21.15.46:6789/0 conn(0x7fb6f40cf340 legacy=0x7fb6f40ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.080+0000 7fb6f9a00640 1 -- 172.21.15.46:0/2592310962 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f40b0380 con 0x7fb6f415a790 2024-03-29T08:18:46.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.080+0000 7fb6f3e00640 1 -- 172.21.15.46:0/2592310962 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432845908 0 0) 0x7fb6f40cc660 con 0x7fb6f40cf340 2024-03-29T08:18:46.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.084+0000 7fb6f9a00640 1 --2- 172.21.15.46:0/2592310962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415a790 0x7fb6f40cee00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80097c0 tx=0x7fb6e801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c458962e96788ed5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.084+0000 7fb6f3e00640 1 -- 172.21.15.46:0/2592310962 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e80ab600 con 0x7fb6f415a790 2024-03-29T08:18:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.084+0000 7fb6f3e00640 1 -- 172.21.15.46:0/2592310962 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e80ab760 con 0x7fb6f415a790 2024-03-29T08:18:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.084+0000 7fb6f3e00640 1 -- 172.21.15.46:0/2592310962 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e80abaf0 con 0x7fb6f415a790 2024-03-29T08:18:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.084+0000 7fb6fc400640 1 -- 172.21.15.46:0/2592310962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415a790 msgr2=0x7fb6f40cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.084+0000 7fb6fc400640 1 --2- 172.21.15.46:0/2592310962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415a790 0x7fb6f40cee00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80097c0 tx=0x7fb6e801c3e0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.084+0000 7fb6fc400640 1 -- 172.21.15.46:0/2592310962 shutdown_connections 2024-03-29T08:18:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.084+0000 7fb6fc400640 1 --2- 172.21.15.46:0/2592310962 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6f40cbbe0 0x7fb6f40d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.084+0000 7fb6fc400640 1 --2- 172.21.15.46:0/2592310962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415a790 0x7fb6f40cee00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.084+0000 7fb6fc400640 1 -- 172.21.15.46:0/2592310962 >> 172.21.15.46:0/2592310962 conn(0x7fb6f4099a10 msgr2=0x7fb6f4089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.084+0000 7fb6fc400640 1 -- 172.21.15.46:0/2592310962 shutdown_connections 2024-03-29T08:18:46.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.084+0000 7fb6fc400640 1 -- 172.21.15.46:0/2592310962 wait complete. 2024-03-29T08:18:46.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fc400640 1 Processor -- start 2024-03-29T08:18:46.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fc400640 1 -- start start 2024-03-29T08:18:46.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fc400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6f40cbbe0 0x7fb6f4152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fc400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6f40cf340 0x7fb6f4158860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fc400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415a790 0x7fb6f41591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fc400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f4158da0 con 0x7fb6f415a790 2024-03-29T08:18:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fc400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f415a070 con 0x7fb6f40cf340 2024-03-29T08:18:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fc400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f41520e0 con 0x7fb6f40cbbe0 2024-03-29T08:18:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fa400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415a790 0x7fb6f41591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fa400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415a790 0x7fb6f41591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43526/0 (socket says 172.21.15.46:43526) 2024-03-29T08:18:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fa400640 1 -- 172.21.15.46:0/3591641303 learned_addr learned my addr 172.21.15.46:0/3591641303 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6f9a00640 1 --2- 172.21.15.46:0/3591641303 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6f40cbbe0 0x7fb6f4152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6f9000640 1 --2- 172.21.15.46:0/3591641303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6f40cf340 0x7fb6f4158860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fa400640 1 -- 172.21.15.46:0/3591641303 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6f40cbbe0 msgr2=0x7fb6f4152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fa400640 1 --2- 172.21.15.46:0/3591641303 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6f40cbbe0 0x7fb6f4152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fa400640 1 -- 172.21.15.46:0/3591641303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6f40cf340 msgr2=0x7fb6f4158860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fa400640 1 --2- 172.21.15.46:0/3591641303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6f40cf340 0x7fb6f4158860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fa400640 1 -- 172.21.15.46:0/3591641303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e8009420 con 0x7fb6f415a790 2024-03-29T08:18:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6f9000640 1 --2- 172.21.15.46:0/3591641303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6f40cf340 0x7fb6f4158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:46.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fa400640 1 --2- 172.21.15.46:0/3591641303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415a790 0x7fb6f41591f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00c7a0 tx=0x7fb6ec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:46.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6f1600640 1 -- 172.21.15.46:0/3591641303 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec00a640 con 0x7fb6f415a790 2024-03-29T08:18:46.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6f1600640 1 -- 172.21.15.46:0/3591641303 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ec00a7a0 con 0x7fb6f415a790 2024-03-29T08:18:46.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6f1600640 1 -- 172.21.15.46:0/3591641303 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec00abd0 con 0x7fb6f415a790 2024-03-29T08:18:46.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fc400640 1 -- 172.21.15.46:0/3591641303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f41523c0 con 0x7fb6f415a790 2024-03-29T08:18:46.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.088+0000 7fb6fc400640 1 -- 172.21.15.46:0/3591641303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f415a350 con 0x7fb6f415a790 2024-03-29T08:18:46.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.092+0000 7fb6fc400640 1 -- 172.21.15.46:0/3591641303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f409e7d0 con 0x7fb6f415a790 2024-03-29T08:18:46.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.092+0000 7fb6f1600640 1 -- 172.21.15.46:0/3591641303 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb6ec00ad30 con 0x7fb6f415a790 2024-03-29T08:18:46.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.092+0000 7fb6f1600640 1 --2- 172.21.15.46:0/3591641303 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6d40666c0 0x7fb6d4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.092+0000 7fb6f1600640 1 -- 172.21.15.46:0/3591641303 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6ec0835f0 con 0x7fb6f415a790 2024-03-29T08:18:46.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.092+0000 7fb6f9a00640 1 --2- 172.21.15.46:0/3591641303 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6d40666c0 0x7fb6d4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.092+0000 7fb6f9a00640 1 --2- 172.21.15.46:0/3591641303 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6d40666c0 0x7fb6d4068b80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80af000 tx=0x7fb6e80a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:46.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.096+0000 7fb6f1600640 1 -- 172.21.15.46:0/3591641303 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb6ec050950 con 0x7fb6f415a790 2024-03-29T08:18:46.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.260+0000 7fb6fc400640 1 -- 172.21.15.46:0/3591641303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6f4158fd0 con 0x7fb6f415a790 2024-03-29T08:18:46.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.264+0000 7fb6f1600640 1 -- 172.21.15.46:0/3591641303 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb6ec018020 con 0x7fb6f415a790 2024-03-29T08:18:46.264 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:18:46.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.264+0000 7fb6fc400640 1 -- 172.21.15.46:0/3591641303 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6d40666c0 msgr2=0x7fb6d4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.264+0000 7fb6fc400640 1 --2- 172.21.15.46:0/3591641303 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6d40666c0 0x7fb6d4068b80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb6e80af000 tx=0x7fb6e80a7000 comp rx=0 tx=0).stop 2024-03-29T08:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.264+0000 7fb6fc400640 1 -- 172.21.15.46:0/3591641303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415a790 msgr2=0x7fb6f41591f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.264+0000 7fb6fc400640 1 --2- 172.21.15.46:0/3591641303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415a790 0x7fb6f41591f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00c7a0 tx=0x7fb6ec00cc60 comp rx=0 tx=0).stop 2024-03-29T08:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.264+0000 7fb6fc400640 1 -- 172.21.15.46:0/3591641303 shutdown_connections 2024-03-29T08:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.264+0000 7fb6fc400640 1 --2- 172.21.15.46:0/3591641303 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6d40666c0 0x7fb6d4068b80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.264+0000 7fb6fc400640 1 --2- 172.21.15.46:0/3591641303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6f40cf340 0x7fb6f4158860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.264+0000 7fb6fc400640 1 --2- 172.21.15.46:0/3591641303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f415a790 0x7fb6f41591f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.264+0000 7fb6fc400640 1 --2- 172.21.15.46:0/3591641303 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6f40cbbe0 0x7fb6f4152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.264+0000 7fb6fc400640 1 -- 172.21.15.46:0/3591641303 >> 172.21.15.46:0/3591641303 conn(0x7fb6f4099a10 msgr2=0x7fb6f40b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.264+0000 7fb6fc400640 1 -- 172.21.15.46:0/3591641303 shutdown_connections 2024-03-29T08:18:46.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.264+0000 7fb6fc400640 1 -- 172.21.15.46:0/3591641303 wait complete. 2024-03-29T08:18:46.285 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:18:46.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.424+0000 7fc1aaa00640 1 Processor -- start 2024-03-29T08:18:46.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.424+0000 7fc1aaa00640 1 -- start start 2024-03-29T08:18:46.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.428+0000 7fc1aaa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc1a40cbea0 0x7fc1a40b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.428+0000 7fc1aaa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1a415efe0 0x7fc1a40b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.428+0000 7fc1aaa00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a40b20b0 con 0x7fc1a4143eb0 2024-03-29T08:18:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.428+0000 7fc1aaa00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1a40b2360 con 0x7fc1a40cbea0 2024-03-29T08:18:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.428+0000 7fc1aaa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a415e500 con 0x7fc1a415efe0 2024-03-29T08:18:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.428+0000 7fc19b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc1a40cbea0 0x7fc1a40b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.428+0000 7fc19b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc1a40cbea0 0x7fc1a40b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43532/0 (socket says 172.21.15.46:43532) 2024-03-29T08:18:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.428+0000 7fc19b400640 1 -- 172.21.15.46:0/3332665396 learned_addr learned my addr 172.21.15.46:0/3332665396 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.428+0000 7fc1a8a00640 1 --2- 172.21.15.46:0/3332665396 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1a415efe0 0x7fc1a40b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.428+0000 7fc19b400640 1 -- 172.21.15.46:0/3332665396 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1a415efe0 msgr2=0x7fc1a40b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.428+0000 7fc19b400640 1 --2- 172.21.15.46:0/3332665396 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1a415efe0 0x7fc1a40b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.428+0000 7fc19b400640 1 -- 172.21.15.46:0/3332665396 >> v1:172.21.15.73:6789/0 conn(0x7fc1a4143eb0 legacy=0x7fc1a40cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:46.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.428+0000 7fc19b400640 1 -- 172.21.15.46:0/3332665396 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a415e7e0 con 0x7fc1a40cbea0 2024-03-29T08:18:46.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc19b400640 1 --2- 172.21.15.46:0/3332665396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a40cbea0 0x7fc1a40b1630 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009e50 tx=0x7fc18c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fd66210e85c838 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:46.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1a3400640 1 -- 172.21.15.46:0/3332665396 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c0a5870 con 0x7fc1a40cbea0 2024-03-29T08:18:46.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1a3400640 1 -- 172.21.15.46:0/3332665396 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c0a59d0 con 0x7fc1a40cbea0 2024-03-29T08:18:46.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1a3400640 1 -- 172.21.15.46:0/3332665396 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c0a5d60 con 0x7fc1a40cbea0 2024-03-29T08:18:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1aaa00640 1 -- 172.21.15.46:0/3332665396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a40cbea0 msgr2=0x7fc1a40b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1aaa00640 1 --2- 172.21.15.46:0/3332665396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a40cbea0 0x7fc1a40b1630 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009e50 tx=0x7fc18c01ca00 comp rx=0 tx=0).stop 2024-03-29T08:18:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1aaa00640 1 -- 172.21.15.46:0/3332665396 shutdown_connections 2024-03-29T08:18:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1aaa00640 1 --2- 172.21.15.46:0/3332665396 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1a415efe0 0x7fc1a40b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1aaa00640 1 --2- 172.21.15.46:0/3332665396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a40cbea0 0x7fc1a40b1630 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1aaa00640 1 -- 172.21.15.46:0/3332665396 >> 172.21.15.46:0/3332665396 conn(0x7fc1a406a390 msgr2=0x7fc1a40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1aaa00640 1 -- 172.21.15.46:0/3332665396 shutdown_connections 2024-03-29T08:18:46.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1aaa00640 1 -- 172.21.15.46:0/3332665396 wait complete. 2024-03-29T08:18:46.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1aaa00640 1 Processor -- start 2024-03-29T08:18:46.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1aaa00640 1 -- start start 2024-03-29T08:18:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1aaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a40cbea0 0x7fc1a4158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1aaa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1a4143eb0 0x7fc1a40cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1aaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a415efe0 0x7fc1a40cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1aaa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a4158750 con 0x7fc1a415efe0 2024-03-29T08:18:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1aaa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a415d5f0 con 0x7fc1a40cbea0 2024-03-29T08:18:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1aaa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a415d8d0 con 0x7fc1a4143eb0 2024-03-29T08:18:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.432+0000 7fc1a8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a415efe0 0x7fc1a40cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1a8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a415efe0 0x7fc1a40cdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43544/0 (socket says 172.21.15.46:43544) 2024-03-29T08:18:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1a8a00640 1 -- 172.21.15.46:0/1721438691 learned_addr learned my addr 172.21.15.46:0/1721438691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc19b400640 1 --2- 172.21.15.46:0/1721438691 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1a4143eb0 0x7fc1a40cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1a3e00640 1 --2- 172.21.15.46:0/1721438691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a40cbea0 0x7fc1a4158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1a8a00640 1 -- 172.21.15.46:0/1721438691 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1a4143eb0 msgr2=0x7fc1a40cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1a8a00640 1 --2- 172.21.15.46:0/1721438691 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1a4143eb0 0x7fc1a40cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1a8a00640 1 -- 172.21.15.46:0/1721438691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a40cbea0 msgr2=0x7fc1a4158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:46.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1a8a00640 1 --2- 172.21.15.46:0/1721438691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a40cbea0 0x7fc1a4158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1a8a00640 1 -- 172.21.15.46:0/1721438691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c009b30 con 0x7fc1a415efe0 2024-03-29T08:18:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1a8a00640 1 --2- 172.21.15.46:0/1721438691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a415efe0 0x7fc1a40cdcb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00e880 tx=0x7fc19c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1a0c00640 1 -- 172.21.15.46:0/1721438691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c009650 con 0x7fc1a415efe0 2024-03-29T08:18:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1a0c00640 1 -- 172.21.15.46:0/1721438691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c00c940 con 0x7fc1a415efe0 2024-03-29T08:18:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1a0c00640 1 -- 172.21.15.46:0/1721438691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00ccd0 con 0x7fc1a415efe0 2024-03-29T08:18:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1aaa00640 1 -- 172.21.15.46:0/1721438691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a415dbb0 con 0x7fc1a415efe0 2024-03-29T08:18:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1aaa00640 1 -- 172.21.15.46:0/1721438691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a415c8c0 con 0x7fc1a415efe0 2024-03-29T08:18:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1aaa00640 1 -- 172.21.15.46:0/1721438691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a4071a70 con 0x7fc1a415efe0 2024-03-29T08:18:46.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1a0c00640 1 -- 172.21.15.46:0/1721438691 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc19c013470 con 0x7fc1a415efe0 2024-03-29T08:18:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1a0c00640 1 --2- 172.21.15.46:0/1721438691 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc180066690 0x7fc180068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.436+0000 7fc1a0c00640 1 -- 172.21.15.46:0/1721438691 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc19c084380 con 0x7fc1a415efe0 2024-03-29T08:18:46.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.440+0000 7fc1a3e00640 1 --2- 172.21.15.46:0/1721438691 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc180066690 0x7fc180068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.440+0000 7fc1a3e00640 1 --2- 172.21.15.46:0/1721438691 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc180066690 0x7fc180068b50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc194007fd0 tx=0x7fc194009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:46.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.440+0000 7fc1a0c00640 1 -- 172.21.15.46:0/1721438691 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc19c051790 con 0x7fc1a415efe0 2024-03-29T08:18:46.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.604+0000 7fc1aaa00640 1 -- 172.21.15.46:0/1721438691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc1a40cadd0 con 0x7fc1a415efe0 2024-03-29T08:18:46.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.604+0000 7fc1a0c00640 1 -- 172.21.15.46:0/1721438691 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc19c054db0 con 0x7fc1a415efe0 2024-03-29T08:18:46.605 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:18:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.604+0000 7fc1aaa00640 1 -- 172.21.15.46:0/1721438691 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc180066690 msgr2=0x7fc180068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.604+0000 7fc1aaa00640 1 --2- 172.21.15.46:0/1721438691 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc180066690 0x7fc180068b50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc194007fd0 tx=0x7fc194009040 comp rx=0 tx=0).stop 2024-03-29T08:18:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.604+0000 7fc1aaa00640 1 -- 172.21.15.46:0/1721438691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a415efe0 msgr2=0x7fc1a40cdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.604+0000 7fc1aaa00640 1 --2- 172.21.15.46:0/1721438691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a415efe0 0x7fc1a40cdcb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00e880 tx=0x7fc19c00ed40 comp rx=0 tx=0).stop 2024-03-29T08:18:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.604+0000 7fc1aaa00640 1 -- 172.21.15.46:0/1721438691 shutdown_connections 2024-03-29T08:18:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.604+0000 7fc1aaa00640 1 --2- 172.21.15.46:0/1721438691 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc180066690 0x7fc180068b50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.604+0000 7fc1aaa00640 1 --2- 172.21.15.46:0/1721438691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1a415efe0 0x7fc1a40cdcb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.604+0000 7fc1aaa00640 1 --2- 172.21.15.46:0/1721438691 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1a4143eb0 0x7fc1a40cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.604+0000 7fc1aaa00640 1 --2- 172.21.15.46:0/1721438691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1a40cbea0 0x7fc1a4158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.604+0000 7fc1aaa00640 1 -- 172.21.15.46:0/1721438691 >> 172.21.15.46:0/1721438691 conn(0x7fc1a406a390 msgr2=0x7fc1a40cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.604+0000 7fc1aaa00640 1 -- 172.21.15.46:0/1721438691 shutdown_connections 2024-03-29T08:18:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.604+0000 7fc1aaa00640 1 -- 172.21.15.46:0/1721438691 wait complete. 2024-03-29T08:18:46.628 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:18:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.768+0000 7fcbc1800640 1 Processor -- start 2024-03-29T08:18:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.768+0000 7fcbc1800640 1 -- start start 2024-03-29T08:18:46.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.768+0000 7fcbc1800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcbbc143eb0 0x7fcbbc0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.768+0000 7fcbc1800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcbbc15efe0 0x7fcbbc0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.768+0000 7fcbc1800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcbbc0b1630 0x7fcbbc0b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.768+0000 7fcbc1800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbbc0b2360 con 0x7fcbbc0b1630 2024-03-29T08:18:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.772+0000 7fcbc1800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbbc15e500 con 0x7fcbbc15efe0 2024-03-29T08:18:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.772+0000 7fcbc1800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc15e7e0 con 0x7fcbbc143eb0 2024-03-29T08:18:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.772+0000 7fcbbb400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcbbc0b1630 0x7fcbbc0b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.772+0000 7fcbbb400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcbbc0b1630 0x7fcbbc0b1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50186/0 (socket says 172.21.15.46:50186) 2024-03-29T08:18:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.772+0000 7fcbbb400640 1 -- 172.21.15.46:0/3091474139 learned_addr learned my addr 172.21.15.46:0/3091474139 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.772+0000 7fcbb2000640 1 --2- 172.21.15.46:0/3091474139 >> v2:172.21.15.73:3300/0 conn(0x7fcbbc15efe0 0x7fcbbc0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.772+0000 7fcbbaa00640 1 --2- 172.21.15.46:0/3091474139 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcbbc143eb0 0x7fcbbc0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.772+0000 7fcbbb400640 1 -- 172.21.15.46:0/3091474139 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcbbc143eb0 msgr2=0x7fcbbc0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.772+0000 7fcbbb400640 1 --2- 172.21.15.46:0/3091474139 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcbbc143eb0 0x7fcbbc0cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.772+0000 7fcbbb400640 1 -- 172.21.15.46:0/3091474139 >> v2:172.21.15.73:3300/0 conn(0x7fcbbc15efe0 msgr2=0x7fcbbc0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.772+0000 7fcbbb400640 1 --2- 172.21.15.46:0/3091474139 >> v2:172.21.15.73:3300/0 conn(0x7fcbbc15efe0 0x7fcbbc0cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.772+0000 7fcbbb400640 1 -- 172.21.15.46:0/3091474139 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc15eac0 con 0x7fcbbc0b1630 2024-03-29T08:18:46.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.772+0000 7fcbbb400640 1 --2- 172.21.15.46:0/3091474139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbbc0b1630 0x7fcbbc0b1a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc06e4b0 tx=0x7fcbb401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a445575cf7ddbc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.772+0000 7fcbba000640 1 -- 172.21.15.46:0/3091474139 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40ab620 con 0x7fcbbc0b1630 2024-03-29T08:18:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.772+0000 7fcbba000640 1 -- 172.21.15.46:0/3091474139 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb40ab780 con 0x7fcbbc0b1630 2024-03-29T08:18:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.772+0000 7fcbba000640 1 -- 172.21.15.46:0/3091474139 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40abb10 con 0x7fcbbc0b1630 2024-03-29T08:18:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbc1800640 1 -- 172.21.15.46:0/3091474139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbbc0b1630 msgr2=0x7fcbbc0b1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:46.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbc1800640 1 --2- 172.21.15.46:0/3091474139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbbc0b1630 0x7fcbbc0b1a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcbbc06e4b0 tx=0x7fcbb401c520 comp rx=0 tx=0).stop 2024-03-29T08:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbc1800640 1 -- 172.21.15.46:0/3091474139 shutdown_connections 2024-03-29T08:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbc1800640 1 --2- 172.21.15.46:0/3091474139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbbc0b1630 0x7fcbbc0b1a80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbc1800640 1 --2- 172.21.15.46:0/3091474139 >> v2:172.21.15.73:3300/0 conn(0x7fcbbc15efe0 0x7fcbbc0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbc1800640 1 --2- 172.21.15.46:0/3091474139 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcbbc143eb0 0x7fcbbc0cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbc1800640 1 -- 172.21.15.46:0/3091474139 >> 172.21.15.46:0/3091474139 conn(0x7fcbbc06a390 msgr2=0x7fcbbc0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbc1800640 1 -- 172.21.15.46:0/3091474139 shutdown_connections 2024-03-29T08:18:46.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbc1800640 1 -- 172.21.15.46:0/3091474139 wait complete. 2024-03-29T08:18:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbc1800640 1 Processor -- start 2024-03-29T08:18:46.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbc1800640 1 -- start start 2024-03-29T08:18:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbc1800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcbbc0b1630 0x7fcbbc0ae310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbc1800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbbc143eb0 0x7fcbbc0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbc1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbbc15efe0 0x7fcbbc1575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbc1800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc0adf50 con 0x7fcbbc15efe0 2024-03-29T08:18:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbc1800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc0ae850 con 0x7fcbbc143eb0 2024-03-29T08:18:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbc1800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc13e010 con 0x7fcbbc0b1630 2024-03-29T08:18:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbbb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbbc15efe0 0x7fcbbc1575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbbb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbbc15efe0 0x7fcbbc1575f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50198/0 (socket says 172.21.15.46:50198) 2024-03-29T08:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbbb400640 1 -- 172.21.15.46:0/3414267007 learned_addr learned my addr 172.21.15.46:0/3414267007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbbaa00640 1 --2- 172.21.15.46:0/3414267007 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcbbc0b1630 0x7fcbbc0ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbb2000640 1 --2- 172.21.15.46:0/3414267007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbbc143eb0 0x7fcbbc0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbbb400640 1 -- 172.21.15.46:0/3414267007 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcbbc0b1630 msgr2=0x7fcbbc0ae310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbbb400640 1 --2- 172.21.15.46:0/3414267007 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcbbc0b1630 0x7fcbbc0ae310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbbb400640 1 -- 172.21.15.46:0/3414267007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbbc143eb0 msgr2=0x7fcbbc0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbbb400640 1 --2- 172.21.15.46:0/3414267007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbbc143eb0 0x7fcbbc0cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.776+0000 7fcbbb400640 1 -- 172.21.15.46:0/3414267007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb40094d0 con 0x7fcbbc15efe0 2024-03-29T08:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.780+0000 7fcbbb400640 1 --2- 172.21.15.46:0/3414267007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbbc15efe0 0x7fcbbc1575f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcbb401abd0 tx=0x7fcbb401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.780+0000 7fcbb3400640 1 -- 172.21.15.46:0/3414267007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40abe80 con 0x7fcbbc15efe0 2024-03-29T08:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.780+0000 7fcbb3400640 1 -- 172.21.15.46:0/3414267007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb40be3f0 con 0x7fcbbc15efe0 2024-03-29T08:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.780+0000 7fcbc1800640 1 -- 172.21.15.46:0/3414267007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbbc0afc70 con 0x7fcbbc15efe0 2024-03-29T08:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.780+0000 7fcbb3400640 1 -- 172.21.15.46:0/3414267007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40be760 con 0x7fcbbc15efe0 2024-03-29T08:18:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.780+0000 7fcbc1800640 1 -- 172.21.15.46:0/3414267007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbbc0aff10 con 0x7fcbbc15efe0 2024-03-29T08:18:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.780+0000 7fcbc1800640 1 -- 172.21.15.46:0/3414267007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbbc071a70 con 0x7fcbbc15efe0 2024-03-29T08:18:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.780+0000 7fcbb3400640 1 -- 172.21.15.46:0/3414267007 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcbb40b43f0 con 0x7fcbbc15efe0 2024-03-29T08:18:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.780+0000 7fcbb3400640 1 --2- 172.21.15.46:0/3414267007 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcb980666c0 0x7fcb98068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.780+0000 7fcbb3400640 1 -- 172.21.15.46:0/3414267007 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbb411e790 con 0x7fcbbc15efe0 2024-03-29T08:18:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.780+0000 7fcbbaa00640 1 --2- 172.21.15.46:0/3414267007 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcb980666c0 0x7fcb98068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.784+0000 7fcbbaa00640 1 --2- 172.21.15.46:0/3414267007 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcb980666c0 0x7fcb98068b80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcbac003b90 tx=0x7fcbac0098d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:46.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.784+0000 7fcbb3400640 1 -- 172.21.15.46:0/3414267007 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcbb40ebb10 con 0x7fcbbc15efe0 2024-03-29T08:18:46.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.948+0000 7fcbc1800640 1 -- 172.21.15.46:0/3414267007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcbbc13e150 con 0x7fcbbc15efe0 2024-03-29T08:18:46.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.948+0000 7fcbb3400640 1 -- 172.21.15.46:0/3414267007 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcbb40ef130 con 0x7fcbbc15efe0 2024-03-29T08:18:46.950 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:18:46.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.952+0000 7fcbc1800640 1 -- 172.21.15.46:0/3414267007 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcb980666c0 msgr2=0x7fcb98068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:46.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.952+0000 7fcbc1800640 1 --2- 172.21.15.46:0/3414267007 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcb980666c0 0x7fcb98068b80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcbac003b90 tx=0x7fcbac0098d0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.952+0000 7fcbc1800640 1 -- 172.21.15.46:0/3414267007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbbc15efe0 msgr2=0x7fcbbc1575f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:46.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.952+0000 7fcbc1800640 1 --2- 172.21.15.46:0/3414267007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbbc15efe0 0x7fcbbc1575f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcbb401abd0 tx=0x7fcbb401ad60 comp rx=0 tx=0).stop 2024-03-29T08:18:46.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.952+0000 7fcbc1800640 1 -- 172.21.15.46:0/3414267007 shutdown_connections 2024-03-29T08:18:46.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.952+0000 7fcbc1800640 1 --2- 172.21.15.46:0/3414267007 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcb980666c0 0x7fcb98068b80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.952+0000 7fcbc1800640 1 --2- 172.21.15.46:0/3414267007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbbc143eb0 0x7fcbbc0cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.952+0000 7fcbc1800640 1 --2- 172.21.15.46:0/3414267007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbbc15efe0 0x7fcbbc1575f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.952+0000 7fcbc1800640 1 --2- 172.21.15.46:0/3414267007 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcbbc0b1630 0x7fcbbc0ae310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.952+0000 7fcbc1800640 1 -- 172.21.15.46:0/3414267007 >> 172.21.15.46:0/3414267007 conn(0x7fcbbc06a390 msgr2=0x7fcbbc15a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.952+0000 7fcbc1800640 1 -- 172.21.15.46:0/3414267007 shutdown_connections 2024-03-29T08:18:46.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:46.952+0000 7fcbc1800640 1 -- 172.21.15.46:0/3414267007 wait complete. 2024-03-29T08:18:46.971 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:18:47.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.112+0000 7f4301000640 1 Processor -- start 2024-03-29T08:18:47.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.112+0000 7f4301000640 1 -- start start 2024-03-29T08:18:47.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.112+0000 7f4301000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f42fc142ab0 0x7f42fc06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.112+0000 7f4301000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f42fc15db10 0x7f42fc0b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.112+0000 7f4301000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42fc06de40 con 0x7f42fc06d9e0 2024-03-29T08:18:47.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.112+0000 7f4301000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42fc070c70 con 0x7f42fc15db10 2024-03-29T08:18:47.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.112+0000 7f4301000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42fc070340 con 0x7f42fc142ab0 2024-03-29T08:18:47.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.112+0000 7f42f3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f42fc06d9e0 0x7f42fc15d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38546/0 (socket says 172.21.15.46:38546) 2024-03-29T08:18:47.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.112+0000 7f42f3e00640 1 -- 172.21.15.46:0/3125684728 learned_addr learned my addr 172.21.15.46:0/3125684728 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:47.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.112+0000 7f42fa800640 1 --2- 172.21.15.46:0/3125684728 >> v2:172.21.15.46:3300/0 conn(0x7f42fc15db10 0x7f42fc0b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.112+0000 7f42f9e00640 1 --2- 172.21.15.46:0/3125684728 >> v2:172.21.15.73:3300/0 conn(0x7f42fc142ab0 0x7f42fc06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.112+0000 7f42fa800640 1 -- 172.21.15.46:0/3125684728 >> v2:172.21.15.73:3300/0 conn(0x7f42fc142ab0 msgr2=0x7f42fc06d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:47.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.112+0000 7f42fa800640 1 --2- 172.21.15.46:0/3125684728 >> v2:172.21.15.73:3300/0 conn(0x7f42fc142ab0 0x7f42fc06d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.112+0000 7f42fa800640 1 -- 172.21.15.46:0/3125684728 >> v1:172.21.15.46:6789/0 conn(0x7f42fc06d9e0 legacy=0x7f42fc15d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:47.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.112+0000 7f42f9400640 1 -- 172.21.15.46:0/3125684728 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3803807577 0 0) 0x7f42fc06de40 con 0x7f42fc06d9e0 2024-03-29T08:18:47.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.112+0000 7f42fa800640 1 -- 172.21.15.46:0/3125684728 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc070620 con 0x7f42fc15db10 2024-03-29T08:18:47.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.116+0000 7f42fa800640 1 --2- 172.21.15.46:0/3125684728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42fc15db10 0x7f42fc0b1e60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f42fc06e120 tx=0x7f42f401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2054cf92bee6ec5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:47.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.116+0000 7f42f9400640 1 -- 172.21.15.46:0/3125684728 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f40ab670 con 0x7f42fc15db10 2024-03-29T08:18:47.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.116+0000 7f42f9400640 1 -- 172.21.15.46:0/3125684728 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f40ab7d0 con 0x7f42fc15db10 2024-03-29T08:18:47.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.116+0000 7f42f9400640 1 -- 172.21.15.46:0/3125684728 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f40abb60 con 0x7f42fc15db10 2024-03-29T08:18:47.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.116+0000 7f4301000640 1 -- 172.21.15.46:0/3125684728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42fc15db10 msgr2=0x7f42fc0b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:47.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.116+0000 7f4301000640 1 --2- 172.21.15.46:0/3125684728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42fc15db10 0x7f42fc0b1e60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f42fc06e120 tx=0x7f42f401c930 comp rx=0 tx=0).stop 2024-03-29T08:18:47.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.116+0000 7f4301000640 1 -- 172.21.15.46:0/3125684728 shutdown_connections 2024-03-29T08:18:47.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.116+0000 7f4301000640 1 --2- 172.21.15.46:0/3125684728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42fc15db10 0x7f42fc0b1e60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.116+0000 7f4301000640 1 --2- 172.21.15.46:0/3125684728 >> v2:172.21.15.73:3300/0 conn(0x7f42fc142ab0 0x7f42fc06d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.116+0000 7f4301000640 1 -- 172.21.15.46:0/3125684728 >> 172.21.15.46:0/3125684728 conn(0x7f42fc066080 msgr2=0x7f42fc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:47.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.116+0000 7f4301000640 1 -- 172.21.15.46:0/3125684728 shutdown_connections 2024-03-29T08:18:47.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.116+0000 7f4301000640 1 -- 172.21.15.46:0/3125684728 wait complete. 2024-03-29T08:18:47.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f4301000640 1 Processor -- start 2024-03-29T08:18:47.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f4301000640 1 -- start start 2024-03-29T08:18:47.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f4301000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f42fc06d9e0 0x7f42fc15bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f4301000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42fc142ab0 0x7f42fc13e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f4301000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42fc15db10 0x7f42fc157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f4301000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc15c160 con 0x7f42fc15db10 2024-03-29T08:18:47.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f4301000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc13fe90 con 0x7f42fc142ab0 2024-03-29T08:18:47.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f4301000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc13aad0 con 0x7f42fc06d9e0 2024-03-29T08:18:47.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f42fa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42fc15db10 0x7f42fc157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f42fa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42fc15db10 0x7f42fc157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50218/0 (socket says 172.21.15.46:50218) 2024-03-29T08:18:47.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f42fa800640 1 -- 172.21.15.46:0/3535735112 learned_addr learned my addr 172.21.15.46:0/3535735112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:47.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f42f3e00640 1 --2- 172.21.15.46:0/3535735112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42fc142ab0 0x7f42fc13e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f42f9e00640 1 --2- 172.21.15.46:0/3535735112 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f42fc06d9e0 0x7f42fc15bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f42fa800640 1 -- 172.21.15.46:0/3535735112 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f42fc06d9e0 msgr2=0x7f42fc15bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:47.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f42fa800640 1 --2- 172.21.15.46:0/3535735112 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f42fc06d9e0 0x7f42fc15bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f42fa800640 1 -- 172.21.15.46:0/3535735112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42fc142ab0 msgr2=0x7f42fc13e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:47.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f42fa800640 1 --2- 172.21.15.46:0/3535735112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42fc142ab0 0x7f42fc13e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f42fa800640 1 -- 172.21.15.46:0/3535735112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f4009a70 con 0x7f42fc15db10 2024-03-29T08:18:47.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f42fa800640 1 --2- 172.21.15.46:0/3535735112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42fc15db10 0x7f42fc157f40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f42f4019540 tx=0x7f42f40196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:47.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f42f2000640 1 -- 172.21.15.46:0/3535735112 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f40abd50 con 0x7f42fc15db10 2024-03-29T08:18:47.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f42f2000640 1 -- 172.21.15.46:0/3535735112 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f40c0450 con 0x7f42fc15db10 2024-03-29T08:18:47.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f42f2000640 1 -- 172.21.15.46:0/3535735112 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f40c0800 con 0x7f42fc15db10 2024-03-29T08:18:47.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f4301000640 1 -- 172.21.15.46:0/3535735112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42fc13f840 con 0x7f42fc15db10 2024-03-29T08:18:47.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.120+0000 7f4301000640 1 -- 172.21.15.46:0/3535735112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42fc158f60 con 0x7f42fc15db10 2024-03-29T08:18:47.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.124+0000 7f4301000640 1 -- 172.21.15.46:0/3535735112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42fc15b940 con 0x7f42fc15db10 2024-03-29T08:18:47.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.124+0000 7f42f2000640 1 -- 172.21.15.46:0/3535735112 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f42f40c0960 con 0x7f42fc15db10 2024-03-29T08:18:47.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.124+0000 7f42f2000640 1 --2- 172.21.15.46:0/3535735112 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f42d8066710 0x7f42d8068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.124+0000 7f42f2000640 1 -- 172.21.15.46:0/3535735112 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42f411efc0 con 0x7f42fc15db10 2024-03-29T08:18:47.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.124+0000 7f42f9e00640 1 --2- 172.21.15.46:0/3535735112 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f42d8066710 0x7f42d8068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.124+0000 7f42f9e00640 1 --2- 172.21.15.46:0/3535735112 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f42d8066710 0x7f42d8068bd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f42ec007a60 tx=0x7f42ec008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:47.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.128+0000 7f42f2000640 1 -- 172.21.15.46:0/3535735112 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f42fc15b940 con 0x7f42fc15db10 2024-03-29T08:18:47.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.300+0000 7f4301000640 1 -- 172.21.15.46:0/3535735112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f42fc15b940 con 0x7f42fc15db10 2024-03-29T08:18:47.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.300+0000 7f42f2000640 1 -- 172.21.15.46:0/3535735112 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f42fc15b940 con 0x7f42fc15db10 2024-03-29T08:18:47.302 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:18:47.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.300+0000 7f4301000640 1 -- 172.21.15.46:0/3535735112 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f42d8066710 msgr2=0x7f42d8068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:47.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.300+0000 7f4301000640 1 --2- 172.21.15.46:0/3535735112 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f42d8066710 0x7f42d8068bd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f42ec007a60 tx=0x7f42ec008040 comp rx=0 tx=0).stop 2024-03-29T08:18:47.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.300+0000 7f4301000640 1 -- 172.21.15.46:0/3535735112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42fc15db10 msgr2=0x7f42fc157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:47.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.300+0000 7f4301000640 1 --2- 172.21.15.46:0/3535735112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42fc15db10 0x7f42fc157f40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f42f4019540 tx=0x7f42f40196d0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.300+0000 7f4301000640 1 -- 172.21.15.46:0/3535735112 shutdown_connections 2024-03-29T08:18:47.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.300+0000 7f4301000640 1 --2- 172.21.15.46:0/3535735112 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f42d8066710 0x7f42d8068bd0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.300+0000 7f4301000640 1 --2- 172.21.15.46:0/3535735112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42fc142ab0 0x7f42fc13e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.300+0000 7f4301000640 1 --2- 172.21.15.46:0/3535735112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42fc15db10 0x7f42fc157f40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.300+0000 7f4301000640 1 --2- 172.21.15.46:0/3535735112 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f42fc06d9e0 0x7f42fc15bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.300+0000 7f4301000640 1 -- 172.21.15.46:0/3535735112 >> 172.21.15.46:0/3535735112 conn(0x7f42fc066080 msgr2=0x7f42fc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:47.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.300+0000 7f4301000640 1 -- 172.21.15.46:0/3535735112 shutdown_connections 2024-03-29T08:18:47.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.300+0000 7f4301000640 1 -- 172.21.15.46:0/3535735112 wait complete. 2024-03-29T08:18:47.322 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:18:47.322 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:18:47.331 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.512+0000 7fd346600640 1 Processor -- start 2024-03-29T08:18:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.512+0000 7fd346600640 1 -- start start 2024-03-29T08:18:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.516+0000 7fd346600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd340099150 0x7fd3400ce580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.516+0000 7fd346600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3400cb390 con 0x7fd34015acf0 2024-03-29T08:18:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.516+0000 7fd346600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3400cb630 con 0x7fd34015a910 2024-03-29T08:18:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.516+0000 7fd346600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400cbfd0 con 0x7fd340099150 2024-03-29T08:18:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.516+0000 7fd33fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd34015acf0 0x7fd3400b0100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38552/0 (socket says 172.21.15.46:38552) 2024-03-29T08:18:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.516+0000 7fd33fe00640 1 -- 172.21.15.46:0/2299663280 learned_addr learned my addr 172.21.15.46:0/2299663280 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.516+0000 7fd33ea00640 1 --2- 172.21.15.46:0/2299663280 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd340099150 0x7fd3400ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.516+0000 7fd33e000640 1 -- 172.21.15.46:0/2299663280 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546007340 0 0) 0x7fd3400cb390 con 0x7fd34015acf0 2024-03-29T08:18:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.516+0000 7fd33e000640 1 -- 172.21.15.46:0/2299663280 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd324003530 con 0x7fd34015acf0 2024-03-29T08:18:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.516+0000 7fd33e000640 1 -- 172.21.15.46:0/2299663280 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1850342391 0 0) 0x7fd3400cb630 con 0x7fd34015a910 2024-03-29T08:18:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.516+0000 7fd33ea00640 1 -- 172.21.15.46:0/2299663280 >> v1:172.21.15.73:6789/0 conn(0x7fd34015a910 legacy=0x7fd340098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.516+0000 7fd33ea00640 1 -- 172.21.15.46:0/2299663280 >> v1:172.21.15.46:6789/0 conn(0x7fd34015acf0 legacy=0x7fd3400b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.516+0000 7fd33ea00640 1 -- 172.21.15.46:0/2299663280 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3400cb910 con 0x7fd340099150 2024-03-29T08:18:47.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.520+0000 7fd33ea00640 1 --2- 172.21.15.46:0/2299663280 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd340099150 0x7fd3400ce580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0097f0 tx=0x7fd32c01ceb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d68a45c2c8e509 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.520+0000 7fd33e000640 1 -- 172.21.15.46:0/2299663280 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c0ad660 con 0x7fd340099150 2024-03-29T08:18:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.520+0000 7fd33e000640 1 -- 172.21.15.46:0/2299663280 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32c0ad7c0 con 0x7fd340099150 2024-03-29T08:18:47.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.520+0000 7fd33e000640 1 -- 172.21.15.46:0/2299663280 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c0adb30 con 0x7fd340099150 2024-03-29T08:18:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.520+0000 7fd346600640 1 -- 172.21.15.46:0/2299663280 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd340099150 msgr2=0x7fd3400ce580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.520+0000 7fd346600640 1 --2- 172.21.15.46:0/2299663280 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd340099150 0x7fd3400ce580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0097f0 tx=0x7fd32c01ceb0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.520+0000 7fd346600640 1 -- 172.21.15.46:0/2299663280 shutdown_connections 2024-03-29T08:18:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.520+0000 7fd346600640 1 --2- 172.21.15.46:0/2299663280 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd340099150 0x7fd3400ce580 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.520+0000 7fd346600640 1 -- 172.21.15.46:0/2299663280 >> 172.21.15.46:0/2299663280 conn(0x7fd3400b2290 msgr2=0x7fd3400b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.520+0000 7fd346600640 1 -- 172.21.15.46:0/2299663280 shutdown_connections 2024-03-29T08:18:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.520+0000 7fd346600640 1 -- 172.21.15.46:0/2299663280 wait complete. 2024-03-29T08:18:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.520+0000 7fd346600640 1 Processor -- start 2024-03-29T08:18:47.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.520+0000 7fd346600640 1 -- start start 2024-03-29T08:18:47.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd346600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd340099150 0x7fd34015c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd346600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34015a910 0x7fd340159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd346600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd34015acf0 0x7fd340152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd346600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34015ddd0 con 0x7fd34015a910 2024-03-29T08:18:47.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd346600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400ae4f0 con 0x7fd340099150 2024-03-29T08:18:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd346600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400ae7d0 con 0x7fd34015acf0 2024-03-29T08:18:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd33ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34015a910 0x7fd340159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd33ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34015a910 0x7fd340159800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50232/0 (socket says 172.21.15.46:50232) 2024-03-29T08:18:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd33ea00640 1 -- 172.21.15.46:0/2342593237 learned_addr learned my addr 172.21.15.46:0/2342593237 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd33f400640 1 --2- 172.21.15.46:0/2342593237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd340099150 0x7fd34015c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd33fe00640 1 --2- 172.21.15.46:0/2342593237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd34015acf0 0x7fd340152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd33ea00640 1 -- 172.21.15.46:0/2342593237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd34015acf0 msgr2=0x7fd340152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd33ea00640 1 --2- 172.21.15.46:0/2342593237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd34015acf0 0x7fd340152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd33ea00640 1 -- 172.21.15.46:0/2342593237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd340099150 msgr2=0x7fd34015c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd33ea00640 1 --2- 172.21.15.46:0/2342593237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd340099150 0x7fd34015c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd33ea00640 1 -- 172.21.15.46:0/2342593237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c009420 con 0x7fd34015a910 2024-03-29T08:18:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd33ea00640 1 --2- 172.21.15.46:0/2342593237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34015a910 0x7fd340159800 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd32c000c10 tx=0x7fd32c0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd337400640 1 -- 172.21.15.46:0/2342593237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c0b6600 con 0x7fd34015a910 2024-03-29T08:18:47.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd337400640 1 -- 172.21.15.46:0/2342593237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32c0b6760 con 0x7fd34015a910 2024-03-29T08:18:47.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd337400640 1 -- 172.21.15.46:0/2342593237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c0b6af0 con 0x7fd34015a910 2024-03-29T08:18:47.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd346600640 1 -- 172.21.15.46:0/2342593237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3400aeab0 con 0x7fd34015a910 2024-03-29T08:18:47.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd346600640 1 -- 172.21.15.46:0/2342593237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3400af070 con 0x7fd34015a910 2024-03-29T08:18:47.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd346600640 1 -- 172.21.15.46:0/2342593237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3400af260 con 0x7fd34015a910 2024-03-29T08:18:47.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.524+0000 7fd337400640 1 -- 172.21.15.46:0/2342593237 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd32c0b4040 con 0x7fd34015a910 2024-03-29T08:18:47.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.528+0000 7fd337400640 1 --2- 172.21.15.46:0/2342593237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3200666c0 0x7fd320068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.528+0000 7fd337400640 1 -- 172.21.15.46:0/2342593237 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd32c11e310 con 0x7fd34015a910 2024-03-29T08:18:47.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.528+0000 7fd33f400640 1 --2- 172.21.15.46:0/2342593237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3200666c0 0x7fd320068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.528+0000 7fd33f400640 1 --2- 172.21.15.46:0/2342593237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3200666c0 0x7fd320068b80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd3300012f0 tx=0x7fd330009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.532+0000 7fd337400640 1 -- 172.21.15.46:0/2342593237 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd32c0eb720 con 0x7fd34015a910 2024-03-29T08:18:47.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.692+0000 7fd346600640 1 -- 172.21.15.46:0/2342593237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3400673a0 con 0x7fd34015a910 2024-03-29T08:18:47.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.692+0000 7fd337400640 1 -- 172.21.15.46:0/2342593237 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15993 (secure 0 0 0) 0x7fd32c0eed40 con 0x7fd34015a910 2024-03-29T08:18:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:47.694 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:14.314124+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:47.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.696+0000 7fd346600640 1 -- 172.21.15.46:0/2342593237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3200666c0 msgr2=0x7fd320068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:47.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.696+0000 7fd346600640 1 --2- 172.21.15.46:0/2342593237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3200666c0 0x7fd320068b80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd3300012f0 tx=0x7fd330009040 comp rx=0 tx=0).stop 2024-03-29T08:18:47.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.696+0000 7fd346600640 1 -- 172.21.15.46:0/2342593237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34015a910 msgr2=0x7fd340159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:47.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.696+0000 7fd346600640 1 --2- 172.21.15.46:0/2342593237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34015a910 0x7fd340159800 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd32c000c10 tx=0x7fd32c0191f0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.696+0000 7fd346600640 1 -- 172.21.15.46:0/2342593237 shutdown_connections 2024-03-29T08:18:47.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.696+0000 7fd346600640 1 --2- 172.21.15.46:0/2342593237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3200666c0 0x7fd320068b80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.696+0000 7fd346600640 1 --2- 172.21.15.46:0/2342593237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd34015acf0 0x7fd340152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.696+0000 7fd346600640 1 --2- 172.21.15.46:0/2342593237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34015a910 0x7fd340159800 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.696+0000 7fd346600640 1 --2- 172.21.15.46:0/2342593237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd340099150 0x7fd34015c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.696+0000 7fd346600640 1 -- 172.21.15.46:0/2342593237 >> 172.21.15.46:0/2342593237 conn(0x7fd3400b2290 msgr2=0x7fd34006fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:47.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.696+0000 7fd346600640 1 -- 172.21.15.46:0/2342593237 shutdown_connections 2024-03-29T08:18:47.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.696+0000 7fd346600640 1 -- 172.21.15.46:0/2342593237 wait complete. 2024-03-29T08:18:47.717 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:18:11.177842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:18:12.308252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:18:47.717 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:18:47.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2bec00640 1 Processor -- start 2024-03-29T08:18:47.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2bec00640 1 -- start start 2024-03-29T08:18:47.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2bec00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2b80ceaf0 0x7fc2b8098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2bec00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2b8098f80 0x7fc2b80cad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2bec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b80cfe50 con 0x7fc2b815c9c0 2024-03-29T08:18:47.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2bec00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b80993f0 con 0x7fc2b8098f80 2024-03-29T08:18:47.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2bec00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b80cf240 con 0x7fc2b80ceaf0 2024-03-29T08:18:47.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc2b815c9c0 0x7fc2b806be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38564/0 (socket says 172.21.15.46:38564) 2024-03-29T08:18:47.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7e00640 1 -- 172.21.15.46:0/783609132 learned_addr learned my addr 172.21.15.46:0/783609132 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:47.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2bcc00640 1 --2- 172.21.15.46:0/783609132 >> v2:172.21.15.73:3300/0 conn(0x7fc2b8098f80 0x7fc2b80cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2af400640 1 --2- 172.21.15.46:0/783609132 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2b80ceaf0 0x7fc2b8098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7400640 1 -- 172.21.15.46:0/783609132 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574975174 0 0) 0x7fc2b80cfe50 con 0x7fc2b815c9c0 2024-03-29T08:18:47.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7400640 1 -- 172.21.15.46:0/783609132 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc29c003530 con 0x7fc2b815c9c0 2024-03-29T08:18:47.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7400640 1 -- 172.21.15.46:0/783609132 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1042197397 0 0) 0x7fc29c003530 con 0x7fc2b815c9c0 2024-03-29T08:18:47.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7400640 1 -- 172.21.15.46:0/783609132 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2b80ceaf0 msgr2=0x7fc2b8098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:47.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7400640 1 --2- 172.21.15.46:0/783609132 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2b80ceaf0 0x7fc2b8098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7400640 1 -- 172.21.15.46:0/783609132 >> v2:172.21.15.73:3300/0 conn(0x7fc2b8098f80 msgr2=0x7fc2b80cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:47.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7400640 1 --2- 172.21.15.46:0/783609132 >> v2:172.21.15.73:3300/0 conn(0x7fc2b8098f80 0x7fc2b80cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7400640 1 -- 172.21.15.46:0/783609132 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b80cf520 con 0x7fc2b815c9c0 2024-03-29T08:18:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7400640 1 -- 172.21.15.46:0/783609132 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fc2a8001680 con 0x7fc2b815c9c0 2024-03-29T08:18:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2bcc00640 1 --2- 172.21.15.46:0/783609132 >> v2:172.21.15.73:3300/0 conn(0x7fc2b8098f80 0x7fc2b80cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7400640 1 -- 172.21.15.46:0/783609132 >> v1:172.21.15.46:6789/0 conn(0x7fc2b815c9c0 legacy=0x7fc2b806be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7400640 1 --2- 172.21.15.46:0/783609132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc29c003980 0x7fc29c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7400640 1 --2- 172.21.15.46:0/783609132 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2b80ceaf0 0x7fc29c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7400640 1 --2- 172.21.15.46:0/783609132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc29c004d10 0x7fc29c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7400640 1 -- 172.21.15.46:0/783609132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b80993f0 con 0x7fc29c003980 2024-03-29T08:18:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7400640 1 -- 172.21.15.46:0/783609132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b80cf240 con 0x7fc29c004d10 2024-03-29T08:18:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7400640 1 -- 172.21.15.46:0/783609132 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc29c00b990 con 0x7fc2b80ceaf0 2024-03-29T08:18:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2b7e00640 1 --2- 172.21.15.46:0/783609132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc29c003980 0x7fc29c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2af400640 1 --2- 172.21.15.46:0/783609132 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2b80ceaf0 0x7fc29c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.860+0000 7fc2bcc00640 1 --2- 172.21.15.46:0/783609132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc29c004d10 0x7fc29c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.864+0000 7fc2b7e00640 1 -- 172.21.15.46:0/783609132 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2b80ceaf0 msgr2=0x7fc29c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:47.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.864+0000 7fc2b7e00640 1 --2- 172.21.15.46:0/783609132 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2b80ceaf0 0x7fc29c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.864+0000 7fc2b7e00640 1 -- 172.21.15.46:0/783609132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc29c004d10 msgr2=0x7fc29c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:47.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.864+0000 7fc2b7e00640 1 --2- 172.21.15.46:0/783609132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc29c004d10 0x7fc29c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.864+0000 7fc2b7e00640 1 -- 172.21.15.46:0/783609132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c003530 con 0x7fc29c003980 2024-03-29T08:18:47.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.864+0000 7fc2b7e00640 1 --2- 172.21.15.46:0/783609132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc29c003980 0x7fc29c003da0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc2a800c240 tx=0x7fc2a801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93efa52ee4cf2fe0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:47.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.864+0000 7fc2b7400640 1 -- 172.21.15.46:0/783609132 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a80058e0 con 0x7fc29c003980 2024-03-29T08:18:47.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.864+0000 7fc2b7400640 1 -- 172.21.15.46:0/783609132 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a8005a40 con 0x7fc29c003980 2024-03-29T08:18:47.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.864+0000 7fc2b7400640 1 -- 172.21.15.46:0/783609132 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8005e10 con 0x7fc29c003980 2024-03-29T08:18:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.864+0000 7fc2bec00640 1 -- 172.21.15.46:0/783609132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc29c003980 msgr2=0x7fc29c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.864+0000 7fc2bec00640 1 --2- 172.21.15.46:0/783609132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc29c003980 0x7fc29c003da0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc2a800c240 tx=0x7fc2a801cc80 comp rx=0 tx=0).stop 2024-03-29T08:18:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.864+0000 7fc2bcc00640 1 -- 172.21.15.46:0/783609132 reap_dead start 2024-03-29T08:18:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.864+0000 7fc2bec00640 1 -- 172.21.15.46:0/783609132 shutdown_connections 2024-03-29T08:18:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.864+0000 7fc2bec00640 1 -- 172.21.15.46:0/783609132 >> 172.21.15.46:0/783609132 conn(0x7fc2b80b2220 msgr2=0x7fc2b80b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.864+0000 7fc2bec00640 1 -- 172.21.15.46:0/783609132 shutdown_connections 2024-03-29T08:18:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.864+0000 7fc2bec00640 1 -- 172.21.15.46:0/783609132 wait complete. 2024-03-29T08:18:47.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2bec00640 1 Processor -- start 2024-03-29T08:18:47.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2bec00640 1 -- start start 2024-03-29T08:18:47.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2bec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc29c003980 0x7fc2b8161b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2bec00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2b81604e0 0x7fc2b81f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2bec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2b8155f30 0x7fc2b80c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2bec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b8157400 con 0x7fc2b8155f30 2024-03-29T08:18:47.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2bec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b81f5f90 con 0x7fc29c003980 2024-03-29T08:18:47.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2bec00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b8159400 con 0x7fc2b81604e0 2024-03-29T08:18:47.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2bcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2b8155f30 0x7fc2b80c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2bcc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2b8155f30 0x7fc2b80c9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50242/0 (socket says 172.21.15.46:50242) 2024-03-29T08:18:47.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2bcc00640 1 -- 172.21.15.46:0/3992993347 learned_addr learned my addr 172.21.15.46:0/3992993347 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:47.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2af400640 1 --2- 172.21.15.46:0/3992993347 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2b81604e0 0x7fc2b81f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2b7e00640 1 --2- 172.21.15.46:0/3992993347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc29c003980 0x7fc2b8161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2bcc00640 1 -- 172.21.15.46:0/3992993347 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2b81604e0 msgr2=0x7fc2b81f5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:47.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2bcc00640 1 --2- 172.21.15.46:0/3992993347 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2b81604e0 0x7fc2b81f5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2bcc00640 1 -- 172.21.15.46:0/3992993347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc29c003980 msgr2=0x7fc2b8161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:47.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2bcc00640 1 --2- 172.21.15.46:0/3992993347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc29c003980 0x7fc2b8161b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:47.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2bcc00640 1 -- 172.21.15.46:0/3992993347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b81567c0 con 0x7fc2b8155f30 2024-03-29T08:18:47.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2af400640 1 --2- 172.21.15.46:0/3992993347 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2b81604e0 0x7fc2b81f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:47.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2bcc00640 1 --2- 172.21.15.46:0/3992993347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2b8155f30 0x7fc2b80c9a60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8071800 tx=0x7fc2b000f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.868+0000 7fc2b4c00640 1 -- 172.21.15.46:0/3992993347 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b000e040 con 0x7fc2b8155f30 2024-03-29T08:18:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.872+0000 7fc2b4c00640 1 -- 172.21.15.46:0/3992993347 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b000f040 con 0x7fc2b8155f30 2024-03-29T08:18:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.872+0000 7fc2bec00640 1 -- 172.21.15.46:0/3992993347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b8160940 con 0x7fc2b8155f30 2024-03-29T08:18:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.872+0000 7fc2b4c00640 1 -- 172.21.15.46:0/3992993347 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b0003be0 con 0x7fc2b8155f30 2024-03-29T08:18:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.872+0000 7fc2bec00640 1 -- 172.21.15.46:0/3992993347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b8156d40 con 0x7fc2b8155f30 2024-03-29T08:18:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.872+0000 7fc2bec00640 1 -- 172.21.15.46:0/3992993347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b8157540 con 0x7fc2b8155f30 2024-03-29T08:18:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.872+0000 7fc2b4c00640 1 -- 172.21.15.46:0/3992993347 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc2b0015430 con 0x7fc2b8155f30 2024-03-29T08:18:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.872+0000 7fc2b4c00640 1 --2- 172.21.15.46:0/3992993347 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc294066690 0x7fc294068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:47.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.872+0000 7fc2b4c00640 1 -- 172.21.15.46:0/3992993347 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2b0084840 con 0x7fc2b8155f30 2024-03-29T08:18:47.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.872+0000 7fc2b7e00640 1 --2- 172.21.15.46:0/3992993347 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc294066690 0x7fc294068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:47.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.876+0000 7fc2b7e00640 1 --2- 172.21.15.46:0/3992993347 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc294066690 0x7fc294068b50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc2a80aa000 tx=0x7fc2a8001a80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:47.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:47.876+0000 7fc2b4c00640 1 -- 172.21.15.46:0/3992993347 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc2b0051cf0 con 0x7fc2b8155f30 2024-03-29T08:18:48.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.036+0000 7fc2bec00640 1 -- 172.21.15.46:0/3992993347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc2b809e700 con 0x7fc2b8155f30 2024-03-29T08:18:48.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.040+0000 7fc2b4c00640 1 -- 172.21.15.46:0/3992993347 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc2b0055310 con 0x7fc2b8155f30 2024-03-29T08:18:48.040 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:18:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.040+0000 7fc2bec00640 1 -- 172.21.15.46:0/3992993347 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc294066690 msgr2=0x7fc294068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.040+0000 7fc2bec00640 1 --2- 172.21.15.46:0/3992993347 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc294066690 0x7fc294068b50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc2a80aa000 tx=0x7fc2a8001a80 comp rx=0 tx=0).stop 2024-03-29T08:18:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.040+0000 7fc2bec00640 1 -- 172.21.15.46:0/3992993347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2b8155f30 msgr2=0x7fc2b80c9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.040+0000 7fc2bec00640 1 --2- 172.21.15.46:0/3992993347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2b8155f30 0x7fc2b80c9a60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8071800 tx=0x7fc2b000f580 comp rx=0 tx=0).stop 2024-03-29T08:18:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.040+0000 7fc2bec00640 1 -- 172.21.15.46:0/3992993347 shutdown_connections 2024-03-29T08:18:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.040+0000 7fc2bec00640 1 --2- 172.21.15.46:0/3992993347 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc294066690 0x7fc294068b50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.040+0000 7fc2bec00640 1 --2- 172.21.15.46:0/3992993347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2b8155f30 0x7fc2b80c9a60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.040+0000 7fc2bec00640 1 --2- 172.21.15.46:0/3992993347 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2b81604e0 0x7fc2b81f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.040+0000 7fc2bec00640 1 --2- 172.21.15.46:0/3992993347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc29c003980 0x7fc2b8161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.040+0000 7fc2bec00640 1 -- 172.21.15.46:0/3992993347 >> 172.21.15.46:0/3992993347 conn(0x7fc2b80b2220 msgr2=0x7fc2b815d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.040+0000 7fc2bec00640 1 -- 172.21.15.46:0/3992993347 shutdown_connections 2024-03-29T08:18:48.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.040+0000 7fc2bec00640 1 -- 172.21.15.46:0/3992993347 wait complete. 2024-03-29T08:18:48.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:18:48.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.204+0000 7f7f17600640 1 Processor -- start 2024-03-29T08:18:48.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.204+0000 7f7f17600640 1 -- start start 2024-03-29T08:18:48.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.204+0000 7f7f17600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f100cbbe0 0x7f7f100d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.204+0000 7f7f17600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7f1015ab70 0x7f7f100d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.204+0000 7f7f17600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f100cc660 con 0x7f7f1015a790 2024-03-29T08:18:48.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.204+0000 7f7f17600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f100cc0a0 con 0x7f7f1015ab70 2024-03-29T08:18:48.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.204+0000 7f7f17600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100cc380 con 0x7f7f100cbbe0 2024-03-29T08:18:48.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.204+0000 7f7f07e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f100cbbe0 0x7f7f100d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.204+0000 7f7f15600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7f1015ab70 0x7f7f100d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.204+0000 7f7f07e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f100cbbe0 0x7f7f100d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:39992/0 (socket says 172.21.15.46:39992) 2024-03-29T08:18:48.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.204+0000 7f7f07e00640 1 -- 172.21.15.46:0/2510031899 learned_addr learned my addr 172.21.15.46:0/2510031899 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:48.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.204+0000 7f7f07e00640 1 -- 172.21.15.46:0/2510031899 >> v2:172.21.15.73:3300/0 conn(0x7f7f1015ab70 msgr2=0x7f7f100d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:48.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.204+0000 7f7f07e00640 1 --2- 172.21.15.46:0/2510031899 >> v2:172.21.15.73:3300/0 conn(0x7f7f1015ab70 0x7f7f100d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.204+0000 7f7f07e00640 1 -- 172.21.15.46:0/2510031899 >> v1:172.21.15.73:6789/0 conn(0x7f7f1015a790 legacy=0x7f7f100cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.204+0000 7f7f07e00640 1 -- 172.21.15.46:0/2510031899 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f100b0380 con 0x7f7f100cbbe0 2024-03-29T08:18:48.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.204+0000 7f7f15600640 1 --2- 172.21.15.46:0/2510031899 >> v2:172.21.15.73:3300/0 conn(0x7f7f1015ab70 0x7f7f100d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:48.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.208+0000 7f7f07e00640 1 --2- 172.21.15.46:0/2510031899 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f100cbbe0 0x7f7f100d0650 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8009870 tx=0x7f7ef801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8528cc0c5e7e7d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:48.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.208+0000 7f7f0fe00640 1 -- 172.21.15.46:0/2510031899 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef80ab660 con 0x7f7f100cbbe0 2024-03-29T08:18:48.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.208+0000 7f7f0fe00640 1 -- 172.21.15.46:0/2510031899 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef80ab7c0 con 0x7f7f100cbbe0 2024-03-29T08:18:48.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.208+0000 7f7f0fe00640 1 -- 172.21.15.46:0/2510031899 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef80abb50 con 0x7f7f100cbbe0 2024-03-29T08:18:48.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f17600640 1 -- 172.21.15.46:0/2510031899 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f100cbbe0 msgr2=0x7f7f100d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f17600640 1 --2- 172.21.15.46:0/2510031899 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f100cbbe0 0x7f7f100d0650 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8009870 tx=0x7f7ef801c4a0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f17600640 1 -- 172.21.15.46:0/2510031899 shutdown_connections 2024-03-29T08:18:48.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f17600640 1 --2- 172.21.15.46:0/2510031899 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f100cbbe0 0x7f7f100d0650 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f17600640 1 --2- 172.21.15.46:0/2510031899 >> v2:172.21.15.73:3300/0 conn(0x7f7f1015ab70 0x7f7f100d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f17600640 1 -- 172.21.15.46:0/2510031899 >> 172.21.15.46:0/2510031899 conn(0x7f7f10099a10 msgr2=0x7f7f10089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:48.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f17600640 1 -- 172.21.15.46:0/2510031899 shutdown_connections 2024-03-29T08:18:48.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f17600640 1 -- 172.21.15.46:0/2510031899 wait complete. 2024-03-29T08:18:48.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f17600640 1 Processor -- start 2024-03-29T08:18:48.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f17600640 1 -- start start 2024-03-29T08:18:48.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f17600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f100cbbe0 0x7f7f101589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f17600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f1015a790 0x7f7f10153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f17600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f1015ab70 0x7f7f10159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f17600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f10159ec0 con 0x7f7f100cbbe0 2024-03-29T08:18:48.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f17600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f10153880 con 0x7f7f1015ab70 2024-03-29T08:18:48.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f17600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f10157f30 con 0x7f7f1015a790 2024-03-29T08:18:48.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f14c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f100cbbe0 0x7f7f101589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f14c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f100cbbe0 0x7f7f101589d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50254/0 (socket says 172.21.15.46:50254) 2024-03-29T08:18:48.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f14c00640 1 -- 172.21.15.46:0/542229862 learned_addr learned my addr 172.21.15.46:0/542229862 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:48.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f07e00640 1 --2- 172.21.15.46:0/542229862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f1015a790 0x7f7f10153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f15600640 1 --2- 172.21.15.46:0/542229862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f1015ab70 0x7f7f10159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f14c00640 1 -- 172.21.15.46:0/542229862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f1015a790 msgr2=0x7f7f10153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f14c00640 1 --2- 172.21.15.46:0/542229862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f1015a790 0x7f7f10153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f14c00640 1 -- 172.21.15.46:0/542229862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f1015ab70 msgr2=0x7f7f10159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f14c00640 1 --2- 172.21.15.46:0/542229862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f1015ab70 0x7f7f10159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.212+0000 7f7f14c00640 1 -- 172.21.15.46:0/542229862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f08009a70 con 0x7f7f100cbbe0 2024-03-29T08:18:48.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.216+0000 7f7f14c00640 1 --2- 172.21.15.46:0/542229862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f100cbbe0 0x7f7f101589d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000c500 tx=0x7f7f0000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:48.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.216+0000 7f7f0d600640 1 -- 172.21.15.46:0/542229862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f000052d0 con 0x7f7f100cbbe0 2024-03-29T08:18:48.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.216+0000 7f7f0d600640 1 -- 172.21.15.46:0/542229862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f00005430 con 0x7f7f100cbbe0 2024-03-29T08:18:48.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.216+0000 7f7f17600640 1 -- 172.21.15.46:0/542229862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef80094d0 con 0x7f7f100cbbe0 2024-03-29T08:18:48.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.216+0000 7f7f0d600640 1 -- 172.21.15.46:0/542229862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f00003840 con 0x7f7f100cbbe0 2024-03-29T08:18:48.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.216+0000 7f7f17600640 1 -- 172.21.15.46:0/542229862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f101581d0 con 0x7f7f100cbbe0 2024-03-29T08:18:48.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.216+0000 7f7f17600640 1 -- 172.21.15.46:0/542229862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1009e7d0 con 0x7f7f100cbbe0 2024-03-29T08:18:48.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.216+0000 7f7f0d600640 1 -- 172.21.15.46:0/542229862 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7f000039a0 con 0x7f7f100cbbe0 2024-03-29T08:18:48.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.216+0000 7f7f0d600640 1 --2- 172.21.15.46:0/542229862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7eec0666e0 0x7f7eec068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.216+0000 7f7f0d600640 1 -- 172.21.15.46:0/542229862 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f00083610 con 0x7f7f100cbbe0 2024-03-29T08:18:48.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.216+0000 7f7f07e00640 1 --2- 172.21.15.46:0/542229862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7eec0666e0 0x7f7eec068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.220+0000 7f7f07e00640 1 --2- 172.21.15.46:0/542229862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7eec0666e0 0x7f7eec068ba0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ef80af000 tx=0x7f7ef80b6040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:48.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.220+0000 7f7f0d600640 1 -- 172.21.15.46:0/542229862 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7f00050aa0 con 0x7f7f100cbbe0 2024-03-29T08:18:48.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.384+0000 7f7f17600640 1 -- 172.21.15.46:0/542229862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7f100d13e0 con 0x7f7f100cbbe0 2024-03-29T08:18:48.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.384+0000 7f7f0d600640 1 -- 172.21.15.46:0/542229862 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7f000540c0 con 0x7f7f100cbbe0 2024-03-29T08:18:48.386 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:18:48.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.388+0000 7f7f17600640 1 -- 172.21.15.46:0/542229862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7eec0666e0 msgr2=0x7f7eec068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.388+0000 7f7f17600640 1 --2- 172.21.15.46:0/542229862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7eec0666e0 0x7f7eec068ba0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ef80af000 tx=0x7f7ef80b6040 comp rx=0 tx=0).stop 2024-03-29T08:18:48.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.388+0000 7f7f17600640 1 -- 172.21.15.46:0/542229862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f100cbbe0 msgr2=0x7f7f101589d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.388+0000 7f7f17600640 1 --2- 172.21.15.46:0/542229862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f100cbbe0 0x7f7f101589d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000c500 tx=0x7f7f0000c900 comp rx=0 tx=0).stop 2024-03-29T08:18:48.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.388+0000 7f7f17600640 1 -- 172.21.15.46:0/542229862 shutdown_connections 2024-03-29T08:18:48.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.388+0000 7f7f17600640 1 --2- 172.21.15.46:0/542229862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7eec0666e0 0x7f7eec068ba0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.388+0000 7f7f17600640 1 --2- 172.21.15.46:0/542229862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f1015ab70 0x7f7f10159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.388+0000 7f7f17600640 1 --2- 172.21.15.46:0/542229862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f1015a790 0x7f7f10153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.388+0000 7f7f17600640 1 --2- 172.21.15.46:0/542229862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f100cbbe0 0x7f7f101589d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.388+0000 7f7f17600640 1 -- 172.21.15.46:0/542229862 >> 172.21.15.46:0/542229862 conn(0x7f7f10099a10 msgr2=0x7f7f1006e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:48.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.388+0000 7f7f17600640 1 -- 172.21.15.46:0/542229862 shutdown_connections 2024-03-29T08:18:48.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.388+0000 7f7f17600640 1 -- 172.21.15.46:0/542229862 wait complete. 2024-03-29T08:18:48.406 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:18:48.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb6a3200640 1 Processor -- start 2024-03-29T08:18:48.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb6a3200640 1 -- start start 2024-03-29T08:18:48.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb6a3200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb69c070280 0x7fb69c0706e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb6a3200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb69c0b1e80 0x7fb69c0b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb6a3200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69c0ade50 con 0x7fb69c142a50 2024-03-29T08:18:48.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb6a3200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69c0b2830 con 0x7fb69c0b1e80 2024-03-29T08:18:48.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb6a3200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c142e30 con 0x7fb69c070280 2024-03-29T08:18:48.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb6a1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb69c0b1e80 0x7fb69c0b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb69be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb69c142a50 0x7fb69c0ae8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38566/0 (socket says 172.21.15.46:38566) 2024-03-29T08:18:48.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb6a1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb69c0b1e80 0x7fb69c0b22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50270/0 (socket says 172.21.15.46:50270) 2024-03-29T08:18:48.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb69be00640 1 -- 172.21.15.46:0/3238768893 learned_addr learned my addr 172.21.15.46:0/3238768893 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:48.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb693400640 1 --2- 172.21.15.46:0/3238768893 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb69c070280 0x7fb69c0706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb6a1200640 1 -- 172.21.15.46:0/3238768893 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb69c070280 msgr2=0x7fb69c0706e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:48.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb6a1200640 1 --2- 172.21.15.46:0/3238768893 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb69c070280 0x7fb69c0706e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb6a1200640 1 -- 172.21.15.46:0/3238768893 >> v1:172.21.15.46:6789/0 conn(0x7fb69c142a50 legacy=0x7fb69c0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb6a1200640 1 -- 172.21.15.46:0/3238768893 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c1432f0 con 0x7fb69c0b1e80 2024-03-29T08:18:48.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb69b400640 1 -- 172.21.15.46:0/3238768893 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612722557 0 0) 0x7fb69c0ade50 con 0x7fb69c142a50 2024-03-29T08:18:48.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.548+0000 7fb693400640 1 --2- 172.21.15.46:0/3238768893 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb69c070280 0x7fb69c0706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:48.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.552+0000 7fb6a1200640 1 --2- 172.21.15.46:0/3238768893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb69c0b1e80 0x7fb69c0b22f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb694009d90 tx=0x7fb69401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e9f2d684e6a8c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:48.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.552+0000 7fb69b400640 1 -- 172.21.15.46:0/3238768893 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6940a5920 con 0x7fb69c0b1e80 2024-03-29T08:18:48.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.552+0000 7fb69b400640 1 -- 172.21.15.46:0/3238768893 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6940a5a80 con 0x7fb69c0b1e80 2024-03-29T08:18:48.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.552+0000 7fb69b400640 1 -- 172.21.15.46:0/3238768893 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6940a5e10 con 0x7fb69c0b1e80 2024-03-29T08:18:48.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.552+0000 7fb6a3200640 1 -- 172.21.15.46:0/3238768893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb69c0b1e80 msgr2=0x7fb69c0b22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.552+0000 7fb6a3200640 1 --2- 172.21.15.46:0/3238768893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb69c0b1e80 0x7fb69c0b22f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb694009d90 tx=0x7fb69401c970 comp rx=0 tx=0).stop 2024-03-29T08:18:48.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.552+0000 7fb6a3200640 1 -- 172.21.15.46:0/3238768893 shutdown_connections 2024-03-29T08:18:48.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.552+0000 7fb6a3200640 1 --2- 172.21.15.46:0/3238768893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb69c0b1e80 0x7fb69c0b22f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.552+0000 7fb6a3200640 1 --2- 172.21.15.46:0/3238768893 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb69c070280 0x7fb69c0706e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.552+0000 7fb6a3200640 1 -- 172.21.15.46:0/3238768893 >> 172.21.15.46:0/3238768893 conn(0x7fb69c066080 msgr2=0x7fb69c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:48.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.552+0000 7fb6a3200640 1 -- 172.21.15.46:0/3238768893 shutdown_connections 2024-03-29T08:18:48.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.552+0000 7fb6a3200640 1 -- 172.21.15.46:0/3238768893 wait complete. 2024-03-29T08:18:48.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a3200640 1 Processor -- start 2024-03-29T08:18:48.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a3200640 1 -- start start 2024-03-29T08:18:48.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a3200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb69c070280 0x7fb69c158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb69c0b1e80 0x7fb69c141e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb69c142a50 0x7fb69c1397c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c13b250 con 0x7fb69c142a50 2024-03-29T08:18:48.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c13b9c0 con 0x7fb69c0b1e80 2024-03-29T08:18:48.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a3200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c158b20 con 0x7fb69c070280 2024-03-29T08:18:48.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb69c142a50 0x7fb69c1397c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb69c142a50 0x7fb69c1397c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50274/0 (socket says 172.21.15.46:50274) 2024-03-29T08:18:48.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a1200640 1 -- 172.21.15.46:0/2956836226 learned_addr learned my addr 172.21.15.46:0/2956836226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:48.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb693400640 1 --2- 172.21.15.46:0/2956836226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb69c0b1e80 0x7fb69c141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb69be00640 1 --2- 172.21.15.46:0/2956836226 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb69c070280 0x7fb69c158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a1200640 1 -- 172.21.15.46:0/2956836226 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb69c070280 msgr2=0x7fb69c158350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a1200640 1 --2- 172.21.15.46:0/2956836226 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb69c070280 0x7fb69c158350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a1200640 1 -- 172.21.15.46:0/2956836226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb69c0b1e80 msgr2=0x7fb69c141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a1200640 1 --2- 172.21.15.46:0/2956836226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb69c0b1e80 0x7fb69c141e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a1200640 1 -- 172.21.15.46:0/2956836226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb694009a70 con 0x7fb69c142a50 2024-03-29T08:18:48.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a1200640 1 --2- 172.21.15.46:0/2956836226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb69c142a50 0x7fb69c1397c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb694009d90 tx=0x7fb694019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:48.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb698c00640 1 -- 172.21.15.46:0/2956836226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6940bf070 con 0x7fb69c142a50 2024-03-29T08:18:48.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb698c00640 1 -- 172.21.15.46:0/2956836226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb694009450 con 0x7fb69c142a50 2024-03-29T08:18:48.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb698c00640 1 -- 172.21.15.46:0/2956836226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6940a5920 con 0x7fb69c142a50 2024-03-29T08:18:48.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a3200640 1 -- 172.21.15.46:0/2956836226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69c158e00 con 0x7fb69c142a50 2024-03-29T08:18:48.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a3200640 1 -- 172.21.15.46:0/2956836226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69c13b4a0 con 0x7fb69c142a50 2024-03-29T08:18:48.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.556+0000 7fb6a3200640 1 -- 172.21.15.46:0/2956836226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69c0ae360 con 0x7fb69c142a50 2024-03-29T08:18:48.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.560+0000 7fb698c00640 1 -- 172.21.15.46:0/2956836226 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb6940b4050 con 0x7fb69c142a50 2024-03-29T08:18:48.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.560+0000 7fb698c00640 1 --2- 172.21.15.46:0/2956836226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb678066690 0x7fb678068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.560+0000 7fb698c00640 1 -- 172.21.15.46:0/2956836226 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb69411ee00 con 0x7fb69c142a50 2024-03-29T08:18:48.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.560+0000 7fb69be00640 1 --2- 172.21.15.46:0/2956836226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb678066690 0x7fb678068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.560+0000 7fb69be00640 1 --2- 172.21.15.46:0/2956836226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb678066690 0x7fb678068b50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb68c000f70 tx=0x7fb68c007df0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:48.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.564+0000 7fb698c00640 1 -- 172.21.15.46:0/2956836226 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb69c0ae360 con 0x7fb69c142a50 2024-03-29T08:18:48.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.720+0000 7fb6a3200640 1 -- 172.21.15.46:0/2956836226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb69c0ae360 con 0x7fb69c142a50 2024-03-29T08:18:48.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.724+0000 7fb698c00640 1 -- 172.21.15.46:0/2956836226 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb69c0ae360 con 0x7fb69c142a50 2024-03-29T08:18:48.724 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:18:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.724+0000 7fb6a3200640 1 -- 172.21.15.46:0/2956836226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb678066690 msgr2=0x7fb678068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.724+0000 7fb6a3200640 1 --2- 172.21.15.46:0/2956836226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb678066690 0x7fb678068b50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb68c000f70 tx=0x7fb68c007df0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.724+0000 7fb6a3200640 1 -- 172.21.15.46:0/2956836226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb69c142a50 msgr2=0x7fb69c1397c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.724+0000 7fb6a3200640 1 --2- 172.21.15.46:0/2956836226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb69c142a50 0x7fb69c1397c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb694009d90 tx=0x7fb694019710 comp rx=0 tx=0).stop 2024-03-29T08:18:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.724+0000 7fb6a3200640 1 -- 172.21.15.46:0/2956836226 shutdown_connections 2024-03-29T08:18:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.724+0000 7fb6a3200640 1 --2- 172.21.15.46:0/2956836226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb678066690 0x7fb678068b50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.724+0000 7fb6a3200640 1 --2- 172.21.15.46:0/2956836226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb69c0b1e80 0x7fb69c141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.724+0000 7fb6a3200640 1 --2- 172.21.15.46:0/2956836226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb69c142a50 0x7fb69c1397c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.724+0000 7fb6a3200640 1 --2- 172.21.15.46:0/2956836226 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb69c070280 0x7fb69c158350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.724+0000 7fb6a3200640 1 -- 172.21.15.46:0/2956836226 >> 172.21.15.46:0/2956836226 conn(0x7fb69c066080 msgr2=0x7fb69c0b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.724+0000 7fb6a3200640 1 -- 172.21.15.46:0/2956836226 shutdown_connections 2024-03-29T08:18:48.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.724+0000 7fb6a3200640 1 -- 172.21.15.46:0/2956836226 wait complete. 2024-03-29T08:18:48.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:18:48.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.884+0000 7f4c79c00640 1 Processor -- start 2024-03-29T08:18:48.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.884+0000 7f4c79c00640 1 -- start start 2024-03-29T08:18:48.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.884+0000 7f4c79c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7415e080 0x7f4c740ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.884+0000 7f4c79c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c740cb500 con 0x7f4c74099950 2024-03-29T08:18:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.884+0000 7f4c79c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c740cb7e0 con 0x7f4c74099150 2024-03-29T08:18:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.884+0000 7f4c79c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740cbac0 con 0x7f4c7415e080 2024-03-29T08:18:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.884+0000 7f4c6b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4c74099950 0x7f4c74098a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38580/0 (socket says 172.21.15.46:38580) 2024-03-29T08:18:48.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.884+0000 7f4c6b400640 1 -- 172.21.15.46:0/251305871 learned_addr learned my addr 172.21.15.46:0/251305871 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:48.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73e00640 1 --2- 172.21.15.46:0/251305871 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7415e080 0x7f4c740ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73400640 1 -- 172.21.15.46:0/251305871 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126797590 0 0) 0x7f4c740cb500 con 0x7f4c74099950 2024-03-29T08:18:48.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73400640 1 -- 172.21.15.46:0/251305871 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c58003530 con 0x7f4c74099950 2024-03-29T08:18:48.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73400640 1 -- 172.21.15.46:0/251305871 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 643114366 0 0) 0x7f4c58003530 con 0x7f4c74099950 2024-03-29T08:18:48.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73400640 1 -- 172.21.15.46:0/251305871 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7415e080 msgr2=0x7f4c740ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:48.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73400640 1 --2- 172.21.15.46:0/251305871 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7415e080 0x7f4c740ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73400640 1 -- 172.21.15.46:0/251305871 >> v1:172.21.15.73:6789/0 conn(0x7f4c74099150 legacy=0x7f4c7415c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73400640 1 -- 172.21.15.46:0/251305871 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c740cbda0 con 0x7f4c74099950 2024-03-29T08:18:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73400640 1 -- 172.21.15.46:0/251305871 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f4c5c001680 con 0x7f4c74099950 2024-03-29T08:18:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73400640 1 -- 172.21.15.46:0/251305871 >> v1:172.21.15.46:6789/0 conn(0x7f4c74099950 legacy=0x7f4c74098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73400640 1 --2- 172.21.15.46:0/251305871 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7415e080 0x7f4c580039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73400640 1 --2- 172.21.15.46:0/251305871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c58003ef0 0x7f4c580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73400640 1 --2- 172.21.15.46:0/251305871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c58004d10 0x7f4c58005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73400640 1 -- 172.21.15.46:0/251305871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740cbac0 con 0x7f4c58004d10 2024-03-29T08:18:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73400640 1 -- 172.21.15.46:0/251305871 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5800b9f0 con 0x7f4c58003ef0 2024-03-29T08:18:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73400640 1 -- 172.21.15.46:0/251305871 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5800bbb0 con 0x7f4c7415e080 2024-03-29T08:18:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73e00640 1 --2- 172.21.15.46:0/251305871 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7415e080 0x7f4c580039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c79200640 1 --2- 172.21.15.46:0/251305871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c58004d10 0x7f4c58005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c6b400640 1 --2- 172.21.15.46:0/251305871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c58003ef0 0x7f4c580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73e00640 1 -- 172.21.15.46:0/251305871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c58003ef0 msgr2=0x7f4c580047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73e00640 1 --2- 172.21.15.46:0/251305871 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c58003ef0 0x7f4c580047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73e00640 1 -- 172.21.15.46:0/251305871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c58004d10 msgr2=0x7f4c58005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73e00640 1 --2- 172.21.15.46:0/251305871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c58004d10 0x7f4c58005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c73e00640 1 -- 172.21.15.46:0/251305871 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c58003530 con 0x7f4c7415e080 2024-03-29T08:18:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.888+0000 7f4c79200640 1 --2- 172.21.15.46:0/251305871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c58004d10 0x7f4c58005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.892+0000 7f4c73e00640 1 --2- 172.21.15.46:0/251305871 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7415e080 0x7f4c580039b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009ae0 tx=0x7f4c6401c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8f88262c291769b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.892+0000 7f4c73400640 1 -- 172.21.15.46:0/251305871 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c007ea0 con 0x7f4c7415e080 2024-03-29T08:18:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.892+0000 7f4c73400640 1 -- 172.21.15.46:0/251305871 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c640a55e0 con 0x7f4c7415e080 2024-03-29T08:18:48.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.892+0000 7f4c73400640 1 -- 172.21.15.46:0/251305871 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c640a5990 con 0x7f4c7415e080 2024-03-29T08:18:48.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.892+0000 7f4c79c00640 1 -- 172.21.15.46:0/251305871 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7415e080 msgr2=0x7f4c580039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.892+0000 7f4c79c00640 1 --2- 172.21.15.46:0/251305871 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7415e080 0x7f4c580039b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009ae0 tx=0x7f4c6401c9f0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.892+0000 7f4c79200640 1 -- 172.21.15.46:0/251305871 reap_dead start 2024-03-29T08:18:48.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.892+0000 7f4c79c00640 1 -- 172.21.15.46:0/251305871 shutdown_connections 2024-03-29T08:18:48.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.892+0000 7f4c79c00640 1 -- 172.21.15.46:0/251305871 >> 172.21.15.46:0/251305871 conn(0x7f4c740b1fe0 msgr2=0x7f4c740b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:48.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.892+0000 7f4c79c00640 1 -- 172.21.15.46:0/251305871 shutdown_connections 2024-03-29T08:18:48.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.892+0000 7f4c79c00640 1 -- 172.21.15.46:0/251305871 wait complete. 2024-03-29T08:18:48.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.892+0000 7f4c79c00640 1 Processor -- start 2024-03-29T08:18:48.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.892+0000 7f4c79c00640 1 -- start start 2024-03-29T08:18:48.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.892+0000 7f4c79c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c7415e080 0x7f4c740c4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.892+0000 7f4c79c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c74158ec0 0x7f4c7409dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.892+0000 7f4c79c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c740b5110 0x7f4c740b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c79c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740c7010 con 0x7f4c74158ec0 2024-03-29T08:18:48.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c79c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740b5ad0 con 0x7f4c7415e080 2024-03-29T08:18:48.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c79c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c740c41e0 con 0x7f4c740b5110 2024-03-29T08:18:48.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c6b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c74158ec0 0x7f4c7409dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c6b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c74158ec0 0x7f4c7409dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50284/0 (socket says 172.21.15.46:50284) 2024-03-29T08:18:48.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c6b400640 1 -- 172.21.15.46:0/3829384249 learned_addr learned my addr 172.21.15.46:0/3829384249 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:48.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c73e00640 1 --2- 172.21.15.46:0/3829384249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c7415e080 0x7f4c740c4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c79200640 1 --2- 172.21.15.46:0/3829384249 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c740b5110 0x7f4c740b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c6b400640 1 -- 172.21.15.46:0/3829384249 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c740b5110 msgr2=0x7f4c740b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c6b400640 1 --2- 172.21.15.46:0/3829384249 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c740b5110 0x7f4c740b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c6b400640 1 -- 172.21.15.46:0/3829384249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c7415e080 msgr2=0x7f4c740c4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c6b400640 1 --2- 172.21.15.46:0/3829384249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c7415e080 0x7f4c740c4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c6b400640 1 -- 172.21.15.46:0/3829384249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c74159340 con 0x7f4c74158ec0 2024-03-29T08:18:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c73e00640 1 --2- 172.21.15.46:0/3829384249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c7415e080 0x7f4c740c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c6b400640 1 --2- 172.21.15.46:0/3829384249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c74158ec0 0x7f4c7409dbe0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c001060 tx=0x7f4c5c012860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c70c00640 1 -- 172.21.15.46:0/3829384249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c014750 con 0x7f4c74158ec0 2024-03-29T08:18:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c70c00640 1 -- 172.21.15.46:0/3829384249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5c0148d0 con 0x7f4c74158ec0 2024-03-29T08:18:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c70c00640 1 -- 172.21.15.46:0/3829384249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c014c80 con 0x7f4c74158ec0 2024-03-29T08:18:48.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c79c00640 1 -- 172.21.15.46:0/3829384249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c740c3c20 con 0x7f4c74158ec0 2024-03-29T08:18:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c79c00640 1 -- 172.21.15.46:0/3829384249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c740c6c30 con 0x7f4c74158ec0 2024-03-29T08:18:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c79c00640 1 -- 172.21.15.46:0/3829384249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c740c7150 con 0x7f4c74158ec0 2024-03-29T08:18:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.896+0000 7f4c70c00640 1 -- 172.21.15.46:0/3829384249 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4c5c0153e0 con 0x7f4c74158ec0 2024-03-29T08:18:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.900+0000 7f4c70c00640 1 --2- 172.21.15.46:0/3829384249 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4c500666c0 0x7f4c50068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.900+0000 7f4c70c00640 1 -- 172.21.15.46:0/3829384249 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c5c0868e0 con 0x7f4c74158ec0 2024-03-29T08:18:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.900+0000 7f4c73e00640 1 --2- 172.21.15.46:0/3829384249 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4c500666c0 0x7f4c50068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:48.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.900+0000 7f4c73e00640 1 --2- 172.21.15.46:0/3829384249 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4c500666c0 0x7f4c50068b80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4c640af000 tx=0x7f4c640046d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:48.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:48.904+0000 7f4c70c00640 1 -- 172.21.15.46:0/3829384249 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4c740c7150 con 0x7f4c74158ec0 2024-03-29T08:18:49.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.076+0000 7f4c79c00640 1 -- 172.21.15.46:0/3829384249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4c740c7150 con 0x7f4c74158ec0 2024-03-29T08:18:49.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.076+0000 7f4c70c00640 1 -- 172.21.15.46:0/3829384249 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4c740c7150 con 0x7f4c74158ec0 2024-03-29T08:18:49.076 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:18:49.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.076+0000 7f4c79c00640 1 -- 172.21.15.46:0/3829384249 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4c500666c0 msgr2=0x7f4c50068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:49.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.076+0000 7f4c79c00640 1 --2- 172.21.15.46:0/3829384249 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4c500666c0 0x7f4c50068b80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4c640af000 tx=0x7f4c640046d0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.076+0000 7f4c79c00640 1 -- 172.21.15.46:0/3829384249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c74158ec0 msgr2=0x7f4c7409dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:49.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.076+0000 7f4c79c00640 1 --2- 172.21.15.46:0/3829384249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c74158ec0 0x7f4c7409dbe0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c001060 tx=0x7f4c5c012860 comp rx=0 tx=0).stop 2024-03-29T08:18:49.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.076+0000 7f4c79c00640 1 -- 172.21.15.46:0/3829384249 shutdown_connections 2024-03-29T08:18:49.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.076+0000 7f4c79c00640 1 --2- 172.21.15.46:0/3829384249 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4c500666c0 0x7f4c50068b80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.076+0000 7f4c79c00640 1 --2- 172.21.15.46:0/3829384249 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c740b5110 0x7f4c740b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.076+0000 7f4c79c00640 1 --2- 172.21.15.46:0/3829384249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c74158ec0 0x7f4c7409dbe0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.076+0000 7f4c79c00640 1 --2- 172.21.15.46:0/3829384249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c7415e080 0x7f4c740c4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.076+0000 7f4c79c00640 1 -- 172.21.15.46:0/3829384249 >> 172.21.15.46:0/3829384249 conn(0x7f4c740b1fe0 msgr2=0x7f4c7415e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:49.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.076+0000 7f4c79c00640 1 -- 172.21.15.46:0/3829384249 shutdown_connections 2024-03-29T08:18:49.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.076+0000 7f4c79c00640 1 -- 172.21.15.46:0/3829384249 wait complete. 2024-03-29T08:18:49.098 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-03-29T08:18:49.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:49.098 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T08:18:49.127 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:18:49.128 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:18:49.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:49.128 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:18:49.187 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:18:49.187 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:18:49.198 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:18:49.198 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:18:49.248 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:49.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.388+0000 7f0623200640 1 Processor -- start 2024-03-29T08:18:49.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.388+0000 7f0623200640 1 -- start start 2024-03-29T08:18:49.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.388+0000 7f0623200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0624159530 0x7f062406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:49.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.388+0000 7f0623200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0624070a70 0x7f0624159e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:49.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.388+0000 7f0623200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f062415d4b0 con 0x7f062406ddd0 2024-03-29T08:18:49.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.388+0000 7f0623200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f062415d790 con 0x7f0624159530 2024-03-29T08:18:49.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.388+0000 7f0623200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f062415da70 con 0x7f0624070a70 2024-03-29T08:18:49.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.388+0000 7f0621e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0624159530 0x7f062406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:49.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.388+0000 7f0621400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f062406ddd0 0x7f0624070360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38582/0 (socket says 172.21.15.46:38582) 2024-03-29T08:18:49.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.388+0000 7f0621400640 1 -- 172.21.15.46:0/4243037528 learned_addr learned my addr 172.21.15.46:0/4243037528 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.388+0000 7f0621e00640 1 -- 172.21.15.46:0/4243037528 >> v2:172.21.15.73:3300/0 conn(0x7f0624070a70 msgr2=0x7f0624159e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-29T08:18:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.388+0000 7f0621e00640 1 --2- 172.21.15.46:0/4243037528 >> v2:172.21.15.73:3300/0 conn(0x7f0624070a70 0x7f0624159e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.388+0000 7f0621e00640 1 -- 172.21.15.46:0/4243037528 >> v1:172.21.15.46:6789/0 conn(0x7f062406ddd0 legacy=0x7f0624070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.388+0000 7f0621e00640 1 -- 172.21.15.46:0/4243037528 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062415dd50 con 0x7f0624159530 2024-03-29T08:18:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.392+0000 7f0621e00640 1 --2- 172.21.15.46:0/4243037528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0624159530 0x7f062406d890 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f06240b0090 tx=0x7f061401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9eef044e4077e72 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.392+0000 7f0620a00640 1 -- 172.21.15.46:0/4243037528 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06140aa620 con 0x7f0624159530 2024-03-29T08:18:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.392+0000 7f0620a00640 1 -- 172.21.15.46:0/4243037528 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06140aa780 con 0x7f0624159530 2024-03-29T08:18:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.392+0000 7f0620a00640 1 -- 172.21.15.46:0/4243037528 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06140aab10 con 0x7f0624159530 2024-03-29T08:18:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.392+0000 7f0623200640 1 -- 172.21.15.46:0/4243037528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0624159530 msgr2=0x7f062406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.392+0000 7f0623200640 1 --2- 172.21.15.46:0/4243037528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0624159530 0x7f062406d890 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f06240b0090 tx=0x7f061401c4a0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.392+0000 7f0623200640 1 -- 172.21.15.46:0/4243037528 shutdown_connections 2024-03-29T08:18:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.392+0000 7f0623200640 1 --2- 172.21.15.46:0/4243037528 >> v2:172.21.15.73:3300/0 conn(0x7f0624070a70 0x7f0624159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.392+0000 7f0623200640 1 --2- 172.21.15.46:0/4243037528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0624159530 0x7f062406d890 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.392+0000 7f0623200640 1 -- 172.21.15.46:0/4243037528 >> 172.21.15.46:0/4243037528 conn(0x7f0624065e80 msgr2=0x7f0624069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.392+0000 7f0623200640 1 -- 172.21.15.46:0/4243037528 shutdown_connections 2024-03-29T08:18:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.392+0000 7f0623200640 1 -- 172.21.15.46:0/4243037528 wait complete. 2024-03-29T08:18:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.392+0000 7f0623200640 1 Processor -- start 2024-03-29T08:18:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.392+0000 7f0623200640 1 -- start start 2024-03-29T08:18:49.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0623200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f062406ddd0 0x7f062415b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0623200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0624070a70 0x7f06241f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0623200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0624159530 0x7f06241f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0623200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062415b970 con 0x7f0624070a70 2024-03-29T08:18:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0623200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062415bc50 con 0x7f062406ddd0 2024-03-29T08:18:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0623200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062415bf30 con 0x7f0624159530 2024-03-29T08:18:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0621400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0624070a70 0x7f06241f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0621400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0624070a70 0x7f06241f5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50296/0 (socket says 172.21.15.46:50296) 2024-03-29T08:18:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0621400640 1 -- 172.21.15.46:0/875391721 learned_addr learned my addr 172.21.15.46:0/875391721 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0621e00640 1 --2- 172.21.15.46:0/875391721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f062406ddd0 0x7f062415b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0622800640 1 --2- 172.21.15.46:0/875391721 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0624159530 0x7f06241f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0621400640 1 -- 172.21.15.46:0/875391721 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0624159530 msgr2=0x7f06241f4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0621400640 1 --2- 172.21.15.46:0/875391721 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0624159530 0x7f06241f4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0621400640 1 -- 172.21.15.46:0/875391721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f062406ddd0 msgr2=0x7f062415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:49.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0621400640 1 --2- 172.21.15.46:0/875391721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f062406ddd0 0x7f062415b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0621400640 1 -- 172.21.15.46:0/875391721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0614009420 con 0x7f0624070a70 2024-03-29T08:18:49.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0621e00640 1 --2- 172.21.15.46:0/875391721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f062406ddd0 0x7f062415b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:49.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0622800640 1 --2- 172.21.15.46:0/875391721 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0624159530 0x7f06241f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:49.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0621400640 1 --2- 172.21.15.46:0/875391721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0624070a70 0x7f06241f5a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f061000c500 tx=0x7f061000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:49.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f061a000640 1 -- 172.21.15.46:0/875391721 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0610005cf0 con 0x7f0624070a70 2024-03-29T08:18:49.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f061a000640 1 -- 172.21.15.46:0/875391721 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0610005e50 con 0x7f0624070a70 2024-03-29T08:18:49.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f061a000640 1 -- 172.21.15.46:0/875391721 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06100072f0 con 0x7f0624070a70 2024-03-29T08:18:49.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0623200640 1 -- 172.21.15.46:0/875391721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062415c210 con 0x7f0624070a70 2024-03-29T08:18:49.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0623200640 1 -- 172.21.15.46:0/875391721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06241f4d70 con 0x7f0624070a70 2024-03-29T08:18:49.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f0623200640 1 -- 172.21.15.46:0/875391721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062413e9d0 con 0x7f0624070a70 2024-03-29T08:18:49.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.396+0000 7f061a000640 1 -- 172.21.15.46:0/875391721 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0610007450 con 0x7f0624070a70 2024-03-29T08:18:49.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.400+0000 7f061a000640 1 --2- 172.21.15.46:0/875391721 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f06040666c0 0x7f0604068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:49.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.400+0000 7f061a000640 1 -- 172.21.15.46:0/875391721 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06100835f0 con 0x7f0624070a70 2024-03-29T08:18:49.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.400+0000 7f0621e00640 1 --2- 172.21.15.46:0/875391721 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f06040666c0 0x7f0604068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:49.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.400+0000 7f0621e00640 1 --2- 172.21.15.46:0/875391721 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f06040666c0 0x7f0604068b80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f06140ae000 tx=0x7f0614008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:49.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.404+0000 7f061a000640 1 -- 172.21.15.46:0/875391721 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f062413e9d0 con 0x7f0624070a70 2024-03-29T08:18:49.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.568+0000 7f0623200640 1 -- 172.21.15.46:0/875391721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f062413e9d0 con 0x7f0624070a70 2024-03-29T08:18:49.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.568+0000 7f061a000640 1 -- 172.21.15.46:0/875391721 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f062413e9d0 con 0x7f0624070a70 2024-03-29T08:18:49.570 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:49.570 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:14.314124+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:49.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.568+0000 7f0623200640 1 -- 172.21.15.46:0/875391721 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f06040666c0 msgr2=0x7f0604068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:49.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.568+0000 7f0623200640 1 --2- 172.21.15.46:0/875391721 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f06040666c0 0x7f0604068b80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f06140ae000 tx=0x7f0614008040 comp rx=0 tx=0).stop 2024-03-29T08:18:49.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.572+0000 7f0623200640 1 -- 172.21.15.46:0/875391721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0624070a70 msgr2=0x7f06241f5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:49.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.572+0000 7f0623200640 1 --2- 172.21.15.46:0/875391721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0624070a70 0x7f06241f5a70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f061000c500 tx=0x7f061000c9c0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.572+0000 7f0623200640 1 -- 172.21.15.46:0/875391721 shutdown_connections 2024-03-29T08:18:49.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.572+0000 7f0623200640 1 --2- 172.21.15.46:0/875391721 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f06040666c0 0x7f0604068b80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.572+0000 7f0623200640 1 --2- 172.21.15.46:0/875391721 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0624159530 0x7f06241f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.572+0000 7f0623200640 1 --2- 172.21.15.46:0/875391721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0624070a70 0x7f06241f5a70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.572+0000 7f0623200640 1 --2- 172.21.15.46:0/875391721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f062406ddd0 0x7f062415b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.572+0000 7f0623200640 1 -- 172.21.15.46:0/875391721 >> 172.21.15.46:0/875391721 conn(0x7f0624065e80 msgr2=0x7f0624158870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:49.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.572+0000 7f0623200640 1 -- 172.21.15.46:0/875391721 shutdown_connections 2024-03-29T08:18:49.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.572+0000 7f0623200640 1 -- 172.21.15.46:0/875391721 wait complete. 2024-03-29T08:18:49.592 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:18:11.177842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:18:12.308252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:18:49.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:18:49.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.732+0000 7f9eb4600640 1 Processor -- start 2024-03-29T08:18:49.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.732+0000 7f9eb4600640 1 -- start start 2024-03-29T08:18:49.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.736+0000 7f9eb4600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9eac15c950 0x7f9eac06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:49.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.736+0000 7f9eb4600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9eac0cea30 0x7f9eac1606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:49.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.736+0000 7f9eb4600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eac0cfb20 con 0x7f9eac15cd30 2024-03-29T08:18:49.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.736+0000 7f9eb4600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eac0cef10 con 0x7f9eac15c950 2024-03-29T08:18:49.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.736+0000 7f9eb4600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eac0cf1f0 con 0x7f9eac0cea30 2024-03-29T08:18:49.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.736+0000 7f9eb1c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9eac15c950 0x7f9eac06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:49.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.736+0000 7f9eb2600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9eac15cd30 0x7f9eac0cad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38586/0 (socket says 172.21.15.46:38586) 2024-03-29T08:18:49.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.736+0000 7f9eb2600640 1 -- 172.21.15.46:0/1883290633 learned_addr learned my addr 172.21.15.46:0/1883290633 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:49.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.736+0000 7f9eb1200640 1 --2- 172.21.15.46:0/1883290633 >> v2:172.21.15.73:3300/0 conn(0x7f9eac0cea30 0x7f9eac1606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:49.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.736+0000 7f9eb1c00640 1 -- 172.21.15.46:0/1883290633 >> v2:172.21.15.73:3300/0 conn(0x7f9eac0cea30 msgr2=0x7f9eac1606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:49.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.736+0000 7f9eb1c00640 1 --2- 172.21.15.46:0/1883290633 >> v2:172.21.15.73:3300/0 conn(0x7f9eac0cea30 0x7f9eac1606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.736+0000 7f9eb1c00640 1 -- 172.21.15.46:0/1883290633 >> v1:172.21.15.46:6789/0 conn(0x7f9eac15cd30 legacy=0x7f9eac0cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:49.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.736+0000 7f9eabe00640 1 -- 172.21.15.46:0/1883290633 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 677066083 0 0) 0x7f9eac0cfb20 con 0x7f9eac15cd30 2024-03-29T08:18:49.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.736+0000 7f9eb1c00640 1 -- 172.21.15.46:0/1883290633 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac0cf4d0 con 0x7f9eac15c950 2024-03-29T08:18:49.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.740+0000 7f9eb1c00640 1 --2- 172.21.15.46:0/1883290633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eac15c950 0x7f9eac06bbe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0c41c0 tx=0x7f9ea001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72151516c3e49aea server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:49.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.740+0000 7f9eabe00640 1 -- 172.21.15.46:0/1883290633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00a8970 con 0x7f9eac15c950 2024-03-29T08:18:49.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.740+0000 7f9eabe00640 1 -- 172.21.15.46:0/1883290633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea00a8ad0 con 0x7f9eac15c950 2024-03-29T08:18:49.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.740+0000 7f9eabe00640 1 -- 172.21.15.46:0/1883290633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00a8e60 con 0x7f9eac15c950 2024-03-29T08:18:49.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.740+0000 7f9eb4600640 1 -- 172.21.15.46:0/1883290633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eac15c950 msgr2=0x7f9eac06bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:49.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.740+0000 7f9eb4600640 1 --2- 172.21.15.46:0/1883290633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eac15c950 0x7f9eac06bbe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0c41c0 tx=0x7f9ea001ec70 comp rx=0 tx=0).stop 2024-03-29T08:18:49.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.740+0000 7f9eb4600640 1 -- 172.21.15.46:0/1883290633 shutdown_connections 2024-03-29T08:18:49.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.740+0000 7f9eb4600640 1 --2- 172.21.15.46:0/1883290633 >> v2:172.21.15.73:3300/0 conn(0x7f9eac0cea30 0x7f9eac1606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.740+0000 7f9eb4600640 1 --2- 172.21.15.46:0/1883290633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eac15c950 0x7f9eac06bbe0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.740+0000 7f9eb4600640 1 -- 172.21.15.46:0/1883290633 >> 172.21.15.46:0/1883290633 conn(0x7f9eac0b21b0 msgr2=0x7f9eac0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:49.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.740+0000 7f9eb4600640 1 -- 172.21.15.46:0/1883290633 shutdown_connections 2024-03-29T08:18:49.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.740+0000 7f9eb4600640 1 -- 172.21.15.46:0/1883290633 wait complete. 2024-03-29T08:18:49.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.740+0000 7f9eb4600640 1 Processor -- start 2024-03-29T08:18:49.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.740+0000 7f9eb4600640 1 -- start start 2024-03-29T08:18:49.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb4600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac0cea30 0x7f9eac1586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:49.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb4600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9eac15c950 0x7f9eac156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:49.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb4600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eac15cd30 0x7f9eac1e6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:49.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb4600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac157510 con 0x7f9eac15cd30 2024-03-29T08:18:49.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb4600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac158cb0 con 0x7f9eac0cea30 2024-03-29T08:18:49.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb4600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac158f90 con 0x7f9eac15c950 2024-03-29T08:18:49.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eac15cd30 0x7f9eac1e6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:49.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eac15cd30 0x7f9eac1e6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50318/0 (socket says 172.21.15.46:50318) 2024-03-29T08:18:49.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb2600640 1 -- 172.21.15.46:0/431157258 learned_addr learned my addr 172.21.15.46:0/431157258 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:49.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb1200640 1 --2- 172.21.15.46:0/431157258 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9eac15c950 0x7f9eac156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:49.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb1c00640 1 --2- 172.21.15.46:0/431157258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac0cea30 0x7f9eac1586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:49.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb2600640 1 -- 172.21.15.46:0/431157258 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9eac15c950 msgr2=0x7f9eac156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:49.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb2600640 1 --2- 172.21.15.46:0/431157258 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9eac15c950 0x7f9eac156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb2600640 1 -- 172.21.15.46:0/431157258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac0cea30 msgr2=0x7f9eac1586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:49.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb2600640 1 --2- 172.21.15.46:0/431157258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac0cea30 0x7f9eac1586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb2600640 1 -- 172.21.15.46:0/431157258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea0009420 con 0x7f9eac15cd30 2024-03-29T08:18:49.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb2600640 1 --2- 172.21.15.46:0/431157258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eac15cd30 0x7f9eac1e6850 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400c780 tx=0x7f9ea400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:49.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9ea9600640 1 -- 172.21.15.46:0/431157258 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea400ce90 con 0x7f9eac15cd30 2024-03-29T08:18:49.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9ea9600640 1 -- 172.21.15.46:0/431157258 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea400a640 con 0x7f9eac15cd30 2024-03-29T08:18:49.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9ea9600640 1 -- 172.21.15.46:0/431157258 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea400a9d0 con 0x7f9eac15cd30 2024-03-29T08:18:49.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb4600640 1 -- 172.21.15.46:0/431157258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eac159270 con 0x7f9eac15cd30 2024-03-29T08:18:49.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb4600640 1 -- 172.21.15.46:0/431157258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eac159830 con 0x7f9eac15cd30 2024-03-29T08:18:49.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9eb4600640 1 -- 172.21.15.46:0/431157258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eac159a50 con 0x7f9eac15cd30 2024-03-29T08:18:49.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.744+0000 7f9ea9600640 1 -- 172.21.15.46:0/431157258 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9ea400ab30 con 0x7f9eac15cd30 2024-03-29T08:18:49.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.748+0000 7f9ea9600640 1 --2- 172.21.15.46:0/431157258 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9e8c066690 0x7f9e8c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:49.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.748+0000 7f9ea9600640 1 -- 172.21.15.46:0/431157258 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ea40835f0 con 0x7f9eac15cd30 2024-03-29T08:18:49.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.748+0000 7f9eb1c00640 1 --2- 172.21.15.46:0/431157258 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9e8c066690 0x7f9e8c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:49.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.748+0000 7f9eb1c00640 1 --2- 172.21.15.46:0/431157258 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9e8c066690 0x7f9e8c068b50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00b2000 tx=0x7f9ea0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:49.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.752+0000 7f9ea9600640 1 -- 172.21.15.46:0/431157258 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9ea4050a10 con 0x7f9eac15cd30 2024-03-29T08:18:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.908+0000 7f9eb4600640 1 -- 172.21.15.46:0/431157258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9eac09e690 con 0x7f9eac15cd30 2024-03-29T08:18:49.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.908+0000 7f9ea9600640 1 -- 172.21.15.46:0/431157258 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ea40180a0 con 0x7f9eac15cd30 2024-03-29T08:18:49.911 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:18:49.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.912+0000 7f9eb4600640 1 -- 172.21.15.46:0/431157258 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9e8c066690 msgr2=0x7f9e8c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:49.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.912+0000 7f9eb4600640 1 --2- 172.21.15.46:0/431157258 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9e8c066690 0x7f9e8c068b50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00b2000 tx=0x7f9ea0008040 comp rx=0 tx=0).stop 2024-03-29T08:18:49.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.912+0000 7f9eb4600640 1 -- 172.21.15.46:0/431157258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eac15cd30 msgr2=0x7f9eac1e6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:49.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.912+0000 7f9eb4600640 1 --2- 172.21.15.46:0/431157258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eac15cd30 0x7f9eac1e6850 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9ea400c780 tx=0x7f9ea400cc40 comp rx=0 tx=0).stop 2024-03-29T08:18:49.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.912+0000 7f9eb4600640 1 -- 172.21.15.46:0/431157258 shutdown_connections 2024-03-29T08:18:49.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.912+0000 7f9eb4600640 1 --2- 172.21.15.46:0/431157258 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9e8c066690 0x7f9e8c068b50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.912+0000 7f9eb4600640 1 --2- 172.21.15.46:0/431157258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9eac15cd30 0x7f9eac1e6850 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.912+0000 7f9eb4600640 1 --2- 172.21.15.46:0/431157258 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9eac15c950 0x7f9eac156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.912+0000 7f9eb4600640 1 --2- 172.21.15.46:0/431157258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9eac0cea30 0x7f9eac1586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:49.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.912+0000 7f9eb4600640 1 -- 172.21.15.46:0/431157258 >> 172.21.15.46:0/431157258 conn(0x7f9eac0b21b0 msgr2=0x7f9eac15d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:49.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.912+0000 7f9eb4600640 1 -- 172.21.15.46:0/431157258 shutdown_connections 2024-03-29T08:18:49.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:49.912+0000 7f9eb4600640 1 -- 172.21.15.46:0/431157258 wait complete. 2024-03-29T08:18:49.932 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:18:50.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.076+0000 7f215a600640 1 Processor -- start 2024-03-29T08:18:50.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.076+0000 7f215a600640 1 -- start start 2024-03-29T08:18:50.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.076+0000 7f215a600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f215415cbf0 0x7f215406a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:50.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.076+0000 7f215a600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f21540c43d0 0x7f215406a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:50.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.076+0000 7f215a600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2154069ec0 con 0x7f215415c810 2024-03-29T08:18:50.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.076+0000 7f215a600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f215406d700 con 0x7f21540c43d0 2024-03-29T08:18:50.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.076+0000 7f215a600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f215406d9e0 con 0x7f215415cbf0 2024-03-29T08:18:50.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.076+0000 7f2159c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f21540c43d0 0x7f215406a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:50.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.076+0000 7f2159c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f21540c43d0 0x7f215406a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50332/0 (socket says 172.21.15.46:50332) 2024-03-29T08:18:50.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.076+0000 7f2159c00640 1 -- 172.21.15.46:0/3840110795 learned_addr learned my addr 172.21.15.46:0/3840110795 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:50.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.076+0000 7f2153e00640 1 --2- 172.21.15.46:0/3840110795 >> v2:172.21.15.73:3300/0 conn(0x7f215415cbf0 0x7f215406a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:50.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.076+0000 7f2159c00640 1 -- 172.21.15.46:0/3840110795 >> v2:172.21.15.73:3300/0 conn(0x7f215415cbf0 msgr2=0x7f215406a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:50.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.076+0000 7f2159c00640 1 --2- 172.21.15.46:0/3840110795 >> v2:172.21.15.73:3300/0 conn(0x7f215415cbf0 0x7f215406a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.076+0000 7f2159c00640 1 -- 172.21.15.46:0/3840110795 >> v1:172.21.15.73:6789/0 conn(0x7f215415c810 legacy=0x7f21540c3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:50.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.076+0000 7f2159c00640 1 -- 172.21.15.46:0/3840110795 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215406dcc0 con 0x7f21540c43d0 2024-03-29T08:18:50.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.080+0000 7f2159c00640 1 --2- 172.21.15.46:0/3840110795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21540c43d0 0x7f215406a890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f21540737a0 tx=0x7f214c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e757cb46cc7d12a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:50.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.080+0000 7f2153400640 1 -- 172.21.15.46:0/3840110795 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c0a5790 con 0x7f21540c43d0 2024-03-29T08:18:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.080+0000 7f2153400640 1 -- 172.21.15.46:0/3840110795 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c0a58f0 con 0x7f21540c43d0 2024-03-29T08:18:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.080+0000 7f2153400640 1 -- 172.21.15.46:0/3840110795 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c0a5c80 con 0x7f21540c43d0 2024-03-29T08:18:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.080+0000 7f215a600640 1 -- 172.21.15.46:0/3840110795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21540c43d0 msgr2=0x7f215406a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.080+0000 7f215a600640 1 --2- 172.21.15.46:0/3840110795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21540c43d0 0x7f215406a890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f21540737a0 tx=0x7f214c01c9c0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.080+0000 7f215a600640 1 -- 172.21.15.46:0/3840110795 shutdown_connections 2024-03-29T08:18:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.080+0000 7f215a600640 1 --2- 172.21.15.46:0/3840110795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21540c43d0 0x7f215406a890 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.080+0000 7f215a600640 1 --2- 172.21.15.46:0/3840110795 >> v2:172.21.15.73:3300/0 conn(0x7f215415cbf0 0x7f215406a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.080+0000 7f215a600640 1 -- 172.21.15.46:0/3840110795 >> 172.21.15.46:0/3840110795 conn(0x7f21540cb730 msgr2=0x7f21540cbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.080+0000 7f215a600640 1 -- 172.21.15.46:0/3840110795 shutdown_connections 2024-03-29T08:18:50.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.080+0000 7f215a600640 1 -- 172.21.15.46:0/3840110795 wait complete. 2024-03-29T08:18:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.080+0000 7f215a600640 1 Processor -- start 2024-03-29T08:18:50.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.080+0000 7f215a600640 1 -- start start 2024-03-29T08:18:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f215a600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f21540c43d0 0x7f215415ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f215a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f215415c810 0x7f215413c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f215a600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f215415cbf0 0x7f2154154860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f215a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2154150f70 con 0x7f215415c810 2024-03-29T08:18:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f215a600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215415f480 con 0x7f215415cbf0 2024-03-29T08:18:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f215a600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215415f760 con 0x7f21540c43d0 2024-03-29T08:18:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f2153e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f215415c810 0x7f215413c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f2153e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f215415c810 0x7f215413c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50334/0 (socket says 172.21.15.46:50334) 2024-03-29T08:18:50.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f2153e00640 1 -- 172.21.15.46:0/496848135 learned_addr learned my addr 172.21.15.46:0/496848135 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f2159c00640 1 --2- 172.21.15.46:0/496848135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f215415cbf0 0x7f2154154860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f2159200640 1 --2- 172.21.15.46:0/496848135 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f21540c43d0 0x7f215415ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f2159c00640 1 -- 172.21.15.46:0/496848135 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f21540c43d0 msgr2=0x7f215415ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f2159c00640 1 --2- 172.21.15.46:0/496848135 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f21540c43d0 0x7f215415ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f2159c00640 1 -- 172.21.15.46:0/496848135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f215415c810 msgr2=0x7f215413c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f2159c00640 1 --2- 172.21.15.46:0/496848135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f215415c810 0x7f215413c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f2159c00640 1 -- 172.21.15.46:0/496848135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c009b30 con 0x7f215415cbf0 2024-03-29T08:18:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f2153e00640 1 --2- 172.21.15.46:0/496848135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f215415c810 0x7f215413c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:50.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f2159c00640 1 --2- 172.21.15.46:0/496848135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f215415cbf0 0x7f2154154860 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f214c0ab000 tx=0x7f214c019760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f2150c00640 1 -- 172.21.15.46:0/496848135 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c0b8670 con 0x7f215415cbf0 2024-03-29T08:18:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f2150c00640 1 -- 172.21.15.46:0/496848135 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c0b87d0 con 0x7f215415cbf0 2024-03-29T08:18:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f215a600640 1 -- 172.21.15.46:0/496848135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215415fa40 con 0x7f215415cbf0 2024-03-29T08:18:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f2150c00640 1 -- 172.21.15.46:0/496848135 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c0b8b60 con 0x7f215415cbf0 2024-03-29T08:18:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f215a600640 1 -- 172.21.15.46:0/496848135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f215413cb50 con 0x7f215415cbf0 2024-03-29T08:18:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.084+0000 7f215a600640 1 -- 172.21.15.46:0/496848135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21540671b0 con 0x7f215415cbf0 2024-03-29T08:18:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.088+0000 7f2150c00640 1 -- 172.21.15.46:0/496848135 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f214c0b8cc0 con 0x7f215415cbf0 2024-03-29T08:18:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.088+0000 7f2150c00640 1 --2- 172.21.15.46:0/496848135 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2134066690 0x7f2134068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.088+0000 7f2150c00640 1 -- 172.21.15.46:0/496848135 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f214c11f670 con 0x7f215415cbf0 2024-03-29T08:18:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.088+0000 7f2159200640 1 --2- 172.21.15.46:0/496848135 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2134066690 0x7f2134068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:50.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.088+0000 7f2159200640 1 --2- 172.21.15.46:0/496848135 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2134066690 0x7f2134068b50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2148007fd0 tx=0x7f2148009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:50.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.092+0000 7f2150c00640 1 -- 172.21.15.46:0/496848135 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f214c0ecca0 con 0x7f215415cbf0 2024-03-29T08:18:50.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.252+0000 7f215a600640 1 -- 172.21.15.46:0/496848135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f21541510e0 con 0x7f215415cbf0 2024-03-29T08:18:50.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.252+0000 7f2150c00640 1 -- 172.21.15.46:0/496848135 <== mon.1 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f21541510e0 con 0x7f215415cbf0 2024-03-29T08:18:50.254 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:18:50.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.256+0000 7f215a600640 1 -- 172.21.15.46:0/496848135 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2134066690 msgr2=0x7f2134068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:50.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.256+0000 7f215a600640 1 --2- 172.21.15.46:0/496848135 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2134066690 0x7f2134068b50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2148007fd0 tx=0x7f2148009040 comp rx=0 tx=0).stop 2024-03-29T08:18:50.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.256+0000 7f215a600640 1 -- 172.21.15.46:0/496848135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f215415cbf0 msgr2=0x7f2154154860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:50.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.256+0000 7f215a600640 1 --2- 172.21.15.46:0/496848135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f215415cbf0 0x7f2154154860 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f214c0ab000 tx=0x7f214c019760 comp rx=0 tx=0).stop 2024-03-29T08:18:50.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.256+0000 7f215a600640 1 -- 172.21.15.46:0/496848135 shutdown_connections 2024-03-29T08:18:50.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.256+0000 7f215a600640 1 --2- 172.21.15.46:0/496848135 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2134066690 0x7f2134068b50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.256+0000 7f215a600640 1 --2- 172.21.15.46:0/496848135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f215415cbf0 0x7f2154154860 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.256+0000 7f215a600640 1 --2- 172.21.15.46:0/496848135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f215415c810 0x7f215413c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.256+0000 7f215a600640 1 --2- 172.21.15.46:0/496848135 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f21540c43d0 0x7f215415ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.256+0000 7f215a600640 1 -- 172.21.15.46:0/496848135 >> 172.21.15.46:0/496848135 conn(0x7f21540cb730 msgr2=0x7f21540ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:50.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.256+0000 7f215a600640 1 -- 172.21.15.46:0/496848135 shutdown_connections 2024-03-29T08:18:50.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.256+0000 7f215a600640 1 -- 172.21.15.46:0/496848135 wait complete. 2024-03-29T08:18:50.276 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:18:50.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.540+0000 7fabff200640 1 Processor -- start 2024-03-29T08:18:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.540+0000 7fabff200640 1 -- start start 2024-03-29T08:18:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.544+0000 7fabff200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabf815a9f0 0x7fabf80ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.544+0000 7fabff200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fabf80ccf60 0x7fabf80cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.544+0000 7fabff200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf80cd8e0 con 0x7fabf806eef0 2024-03-29T08:18:50.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.544+0000 7fabff200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf80cf590 con 0x7fabf80ccf60 2024-03-29T08:18:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.544+0000 7fabff200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf806e130 con 0x7fabf815a9f0 2024-03-29T08:18:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.544+0000 7fabfd400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fabf80ccf60 0x7fabf80cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.544+0000 7fabfde00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabf815a9f0 0x7fabf80ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.544+0000 7fabfe800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fabf806eef0 0x7fabf806f340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:41860/0 (socket says 172.21.15.46:41860) 2024-03-29T08:18:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.544+0000 7fabfd400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fabf80ccf60 0x7fabf80cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47266/0 (socket says 172.21.15.46:47266) 2024-03-29T08:18:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.544+0000 7fabfe800640 1 -- 172.21.15.46:0/1618678496 learned_addr learned my addr 172.21.15.46:0/1618678496 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.544+0000 7fabfca00640 1 -- 172.21.15.46:0/1618678496 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 117622057 0 0) 0x7fabf80cd8e0 con 0x7fabf806eef0 2024-03-29T08:18:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.544+0000 7fabfca00640 1 -- 172.21.15.46:0/1618678496 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabe0003530 con 0x7fabf806eef0 2024-03-29T08:18:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.544+0000 7fabfde00640 1 -- 172.21.15.46:0/1618678496 >> v2:172.21.15.73:3300/0 conn(0x7fabf80ccf60 msgr2=0x7fabf80cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.544+0000 7fabfde00640 1 --2- 172.21.15.46:0/1618678496 >> v2:172.21.15.73:3300/0 conn(0x7fabf80ccf60 0x7fabf80cd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.544+0000 7fabfde00640 1 -- 172.21.15.46:0/1618678496 >> v1:172.21.15.73:6789/0 conn(0x7fabf806eef0 legacy=0x7fabf806f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.544+0000 7fabfde00640 1 -- 172.21.15.46:0/1618678496 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf806e410 con 0x7fabf815a9f0 2024-03-29T08:18:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.544+0000 7fabfd400640 1 --2- 172.21.15.46:0/1618678496 >> v2:172.21.15.73:3300/0 conn(0x7fabf80ccf60 0x7fabf80cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.548+0000 7fabfde00640 1 --2- 172.21.15.46:0/1618678496 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabf815a9f0 0x7fabf80ced40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fabec0097c0 tx=0x7fabec01ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e0d87535c2aa22e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.548+0000 7fabfca00640 1 -- 172.21.15.46:0/1618678496 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec0a8990 con 0x7fabf815a9f0 2024-03-29T08:18:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.548+0000 7fabfca00640 1 -- 172.21.15.46:0/1618678496 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabec0a8af0 con 0x7fabf815a9f0 2024-03-29T08:18:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.548+0000 7fabfca00640 1 -- 172.21.15.46:0/1618678496 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec0a8e80 con 0x7fabf815a9f0 2024-03-29T08:18:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.548+0000 7fabff200640 1 -- 172.21.15.46:0/1618678496 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabf815a9f0 msgr2=0x7fabf80ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.548+0000 7fabff200640 1 --2- 172.21.15.46:0/1618678496 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabf815a9f0 0x7fabf80ced40 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fabec0097c0 tx=0x7fabec01ec90 comp rx=0 tx=0).stop 2024-03-29T08:18:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.548+0000 7fabff200640 1 -- 172.21.15.46:0/1618678496 shutdown_connections 2024-03-29T08:18:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.548+0000 7fabff200640 1 --2- 172.21.15.46:0/1618678496 >> v2:172.21.15.73:3300/0 conn(0x7fabf80ccf60 0x7fabf80cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.548+0000 7fabff200640 1 --2- 172.21.15.46:0/1618678496 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabf815a9f0 0x7fabf80ced40 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.548+0000 7fabff200640 1 -- 172.21.15.46:0/1618678496 >> 172.21.15.46:0/1618678496 conn(0x7fabf80d0550 msgr2=0x7fabf80d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.548+0000 7fabff200640 1 -- 172.21.15.46:0/1618678496 shutdown_connections 2024-03-29T08:18:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.548+0000 7fabff200640 1 -- 172.21.15.46:0/1618678496 wait complete. 2024-03-29T08:18:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.548+0000 7fabff200640 1 Processor -- start 2024-03-29T08:18:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.548+0000 7fabff200640 1 -- start start 2024-03-29T08:18:50.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabff200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabf806eef0 0x7fabf80b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:50.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabff200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf80ccf60 0x7fabf80b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:50.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabff200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabf815a9f0 0x7fabf80c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:50.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabff200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80c5070 con 0x7fabf815a9f0 2024-03-29T08:18:50.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabff200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80b61d0 con 0x7fabf80ccf60 2024-03-29T08:18:50.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabff200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80c52c0 con 0x7fabf806eef0 2024-03-29T08:18:50.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabfd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf80ccf60 0x7fabf80b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:50.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabfe800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabf815a9f0 0x7fabf80c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:50.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabfde00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabf806eef0 0x7fabf80b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:50.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabfd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf80ccf60 0x7fabf80b5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47268/0 (socket says 172.21.15.46:47268) 2024-03-29T08:18:50.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabfd400640 1 -- 172.21.15.46:0/2075800365 learned_addr learned my addr 172.21.15.46:0/2075800365 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabfd400640 1 -- 172.21.15.46:0/2075800365 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabf806eef0 msgr2=0x7fabf80b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabfd400640 1 --2- 172.21.15.46:0/2075800365 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabf806eef0 0x7fabf80b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabfd400640 1 -- 172.21.15.46:0/2075800365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabf815a9f0 msgr2=0x7fabf80c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabfd400640 1 --2- 172.21.15.46:0/2075800365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabf815a9f0 0x7fabf80c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabfd400640 1 -- 172.21.15.46:0/2075800365 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe8009420 con 0x7fabf80ccf60 2024-03-29T08:18:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabfd400640 1 --2- 172.21.15.46:0/2075800365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf80ccf60 0x7fabf80b5490 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fabe800ebe0 tx=0x7fabe800c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabf2000640 1 -- 172.21.15.46:0/2075800365 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe800eee0 con 0x7fabf80ccf60 2024-03-29T08:18:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabf2000640 1 -- 172.21.15.46:0/2075800365 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe800cc30 con 0x7fabf80ccf60 2024-03-29T08:18:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabff200640 1 -- 172.21.15.46:0/2075800365 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabec009420 con 0x7fabf80ccf60 2024-03-29T08:18:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabf2000640 1 -- 172.21.15.46:0/2075800365 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8013470 con 0x7fabf80ccf60 2024-03-29T08:18:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabff200640 1 -- 172.21.15.46:0/2075800365 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf80c5b60 con 0x7fabf80ccf60 2024-03-29T08:18:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabff200640 1 -- 172.21.15.46:0/2075800365 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf80c56b0 con 0x7fabf80ccf60 2024-03-29T08:18:50.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.552+0000 7fabf2000640 1 -- 172.21.15.46:0/2075800365 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fabe80135d0 con 0x7fabf80ccf60 2024-03-29T08:18:50.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.556+0000 7fabf2000640 1 --2- 172.21.15.46:0/2075800365 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fabdc066690 0x7fabdc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:50.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.556+0000 7fabf2000640 1 -- 172.21.15.46:0/2075800365 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabe80844d0 con 0x7fabf80ccf60 2024-03-29T08:18:50.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.556+0000 7fabfde00640 1 --2- 172.21.15.46:0/2075800365 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fabdc066690 0x7fabdc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:50.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.556+0000 7fabfde00640 1 --2- 172.21.15.46:0/2075800365 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fabdc066690 0x7fabdc068b50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fabec0b2000 tx=0x7fabec008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:50.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.560+0000 7fabf2000640 1 -- 172.21.15.46:0/2075800365 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fabe80517c0 con 0x7fabf80ccf60 2024-03-29T08:18:50.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.716+0000 7fabff200640 1 -- 172.21.15.46:0/2075800365 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fabf804e040 con 0x7fabf80ccf60 2024-03-29T08:18:50.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.716+0000 7fabf2000640 1 -- 172.21.15.46:0/2075800365 <== mon.1 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fabe8054de0 con 0x7fabf80ccf60 2024-03-29T08:18:50.719 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:18:50.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.720+0000 7fabff200640 1 -- 172.21.15.46:0/2075800365 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fabdc066690 msgr2=0x7fabdc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:50.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.720+0000 7fabff200640 1 --2- 172.21.15.46:0/2075800365 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fabdc066690 0x7fabdc068b50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fabec0b2000 tx=0x7fabec008040 comp rx=0 tx=0).stop 2024-03-29T08:18:50.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.720+0000 7fabff200640 1 -- 172.21.15.46:0/2075800365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf80ccf60 msgr2=0x7fabf80b5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:50.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.720+0000 7fabff200640 1 --2- 172.21.15.46:0/2075800365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf80ccf60 0x7fabf80b5490 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fabe800ebe0 tx=0x7fabe800c520 comp rx=0 tx=0).stop 2024-03-29T08:18:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.720+0000 7fabff200640 1 -- 172.21.15.46:0/2075800365 shutdown_connections 2024-03-29T08:18:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.720+0000 7fabff200640 1 --2- 172.21.15.46:0/2075800365 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fabdc066690 0x7fabdc068b50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.720+0000 7fabff200640 1 --2- 172.21.15.46:0/2075800365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabf80ccf60 0x7fabf80b5490 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.720+0000 7fabff200640 1 --2- 172.21.15.46:0/2075800365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabf815a9f0 0x7fabf80c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.720+0000 7fabff200640 1 --2- 172.21.15.46:0/2075800365 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabf806eef0 0x7fabf80b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.720+0000 7fabff200640 1 -- 172.21.15.46:0/2075800365 >> 172.21.15.46:0/2075800365 conn(0x7fabf80d0550 msgr2=0x7fabf806f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.720+0000 7fabff200640 1 -- 172.21.15.46:0/2075800365 shutdown_connections 2024-03-29T08:18:50.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.720+0000 7fabff200640 1 -- 172.21.15.46:0/2075800365 wait complete. 2024-03-29T08:18:50.742 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:18:50.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.884+0000 7fa94fc00640 1 Processor -- start 2024-03-29T08:18:50.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.884+0000 7fa94fc00640 1 -- start start 2024-03-29T08:18:50.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.884+0000 7fa94fc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa94815db10 0x7fa948164410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:50.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.884+0000 7fa94fc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa948070c70 con 0x7fa94806dbb0 2024-03-29T08:18:50.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.884+0000 7fa94fc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa948070340 con 0x7fa948142ab0 2024-03-29T08:18:50.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.884+0000 7fa94fc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94815df40 con 0x7fa94815db10 2024-03-29T08:18:50.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.884+0000 7fa93fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa94806dbb0 0x7fa94815d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38594/0 (socket says 172.21.15.46:38594) 2024-03-29T08:18:50.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.884+0000 7fa93fe00640 1 -- 172.21.15.46:0/1653147592 learned_addr learned my addr 172.21.15.46:0/1653147592 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:50.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.884+0000 7fa94dc00640 1 --2- 172.21.15.46:0/1653147592 >> v2:172.21.15.73:3300/0 conn(0x7fa94815db10 0x7fa948164410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:50.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.884+0000 7fa947e00640 1 -- 172.21.15.46:0/1653147592 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57069630 0 0) 0x7fa948070c70 con 0x7fa94806dbb0 2024-03-29T08:18:50.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.884+0000 7fa947e00640 1 -- 172.21.15.46:0/1653147592 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa92c003530 con 0x7fa94806dbb0 2024-03-29T08:18:50.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.884+0000 7fa947e00640 1 -- 172.21.15.46:0/1653147592 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020521995 0 0) 0x7fa948070340 con 0x7fa948142ab0 2024-03-29T08:18:50.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.884+0000 7fa947e00640 1 -- 172.21.15.46:0/1653147592 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa948070c70 con 0x7fa948142ab0 2024-03-29T08:18:50.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.884+0000 7fa94dc00640 1 -- 172.21.15.46:0/1653147592 >> v1:172.21.15.73:6789/0 conn(0x7fa948142ab0 legacy=0x7fa94806d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:50.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.884+0000 7fa94dc00640 1 -- 172.21.15.46:0/1653147592 >> v1:172.21.15.46:6789/0 conn(0x7fa94806dbb0 legacy=0x7fa94815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:50.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.884+0000 7fa94dc00640 1 -- 172.21.15.46:0/1653147592 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa948070620 con 0x7fa94815db10 2024-03-29T08:18:50.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.888+0000 7fa94dc00640 1 --2- 172.21.15.46:0/1653147592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa94815db10 0x7fa948164410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa940009d90 tx=0x7fa94001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faaa44ac165ff29f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:50.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.888+0000 7fa947e00640 1 -- 172.21.15.46:0/1653147592 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9400a5830 con 0x7fa94815db10 2024-03-29T08:18:50.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.888+0000 7fa947e00640 1 -- 172.21.15.46:0/1653147592 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9400a5990 con 0x7fa94815db10 2024-03-29T08:18:50.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.888+0000 7fa947e00640 1 -- 172.21.15.46:0/1653147592 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9400a5d20 con 0x7fa94815db10 2024-03-29T08:18:50.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.888+0000 7fa94fc00640 1 -- 172.21.15.46:0/1653147592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa94815db10 msgr2=0x7fa948164410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:50.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.888+0000 7fa94fc00640 1 --2- 172.21.15.46:0/1653147592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa94815db10 0x7fa948164410 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa940009d90 tx=0x7fa94001c970 comp rx=0 tx=0).stop 2024-03-29T08:18:50.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.888+0000 7fa94fc00640 1 -- 172.21.15.46:0/1653147592 shutdown_connections 2024-03-29T08:18:50.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.888+0000 7fa94fc00640 1 --2- 172.21.15.46:0/1653147592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa94815db10 0x7fa948164410 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.888+0000 7fa94fc00640 1 -- 172.21.15.46:0/1653147592 >> 172.21.15.46:0/1653147592 conn(0x7fa948066080 msgr2=0x7fa948069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:50.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.888+0000 7fa94fc00640 1 -- 172.21.15.46:0/1653147592 shutdown_connections 2024-03-29T08:18:50.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.888+0000 7fa94fc00640 1 -- 172.21.15.46:0/1653147592 wait complete. 2024-03-29T08:18:50.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94fc00640 1 Processor -- start 2024-03-29T08:18:50.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94fc00640 1 -- start start 2024-03-29T08:18:50.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94fc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa94806dbb0 0x7fa9480cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:50.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94fc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa948142ab0 0x7fa9481f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:50.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94fc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94815db10 0x7fa94815cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:50.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94fc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94815aeb0 con 0x7fa94815db10 2024-03-29T08:18:50.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94fc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9481f85f0 con 0x7fa94806dbb0 2024-03-29T08:18:50.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94fc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa948163100 con 0x7fa948142ab0 2024-03-29T08:18:50.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94815db10 0x7fa94815cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:50.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94815db10 0x7fa94815cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50344/0 (socket says 172.21.15.46:50344) 2024-03-29T08:18:50.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94dc00640 1 -- 172.21.15.46:0/1240053250 learned_addr learned my addr 172.21.15.46:0/1240053250 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:50.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94d200640 1 --2- 172.21.15.46:0/1240053250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa94806dbb0 0x7fa9480cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:50.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa93fe00640 1 --2- 172.21.15.46:0/1240053250 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa948142ab0 0x7fa9481f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:50.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94dc00640 1 -- 172.21.15.46:0/1240053250 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa948142ab0 msgr2=0x7fa9481f80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:50.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94dc00640 1 --2- 172.21.15.46:0/1240053250 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa948142ab0 0x7fa9481f80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94dc00640 1 -- 172.21.15.46:0/1240053250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa94806dbb0 msgr2=0x7fa9480cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:50.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94dc00640 1 --2- 172.21.15.46:0/1240053250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa94806dbb0 0x7fa9480cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:50.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94dc00640 1 -- 172.21.15.46:0/1240053250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa940009a70 con 0x7fa94815db10 2024-03-29T08:18:50.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94d200640 1 --2- 172.21.15.46:0/1240053250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa94806dbb0 0x7fa9480cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:50.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94dc00640 1 --2- 172.21.15.46:0/1240053250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94815db10 0x7fa94815cf70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa9400b0040 tx=0x7fa940019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa945600640 1 -- 172.21.15.46:0/1240053250 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9400b9650 con 0x7fa94815db10 2024-03-29T08:18:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa945600640 1 -- 172.21.15.46:0/1240053250 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9400b97b0 con 0x7fa94815db10 2024-03-29T08:18:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa945600640 1 -- 172.21.15.46:0/1240053250 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9400b9b20 con 0x7fa94815db10 2024-03-29T08:18:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94fc00640 1 -- 172.21.15.46:0/1240053250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94815a2a0 con 0x7fa94815db10 2024-03-29T08:18:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.892+0000 7fa94fc00640 1 -- 172.21.15.46:0/1240053250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94815a860 con 0x7fa94815db10 2024-03-29T08:18:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.896+0000 7fa94fc00640 1 -- 172.21.15.46:0/1240053250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9480671b0 con 0x7fa94815db10 2024-03-29T08:18:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.896+0000 7fa945600640 1 -- 172.21.15.46:0/1240053250 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa9400b9c80 con 0x7fa94815db10 2024-03-29T08:18:50.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.896+0000 7fa945600640 1 --2- 172.21.15.46:0/1240053250 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa924066690 0x7fa924068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:50.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.896+0000 7fa945600640 1 -- 172.21.15.46:0/1240053250 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa94011f380 con 0x7fa94815db10 2024-03-29T08:18:50.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.896+0000 7fa94d200640 1 --2- 172.21.15.46:0/1240053250 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa924066690 0x7fa924068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:50.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.896+0000 7fa94d200640 1 --2- 172.21.15.46:0/1240053250 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa924066690 0x7fa924068b50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa9380011f0 tx=0x7fa938009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:50.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:50.900+0000 7fa945600640 1 -- 172.21.15.46:0/1240053250 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa9400ec770 con 0x7fa94815db10 2024-03-29T08:18:51.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.072+0000 7fa94fc00640 1 -- 172.21.15.46:0/1240053250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa948071a90 con 0x7fa94815db10 2024-03-29T08:18:51.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.072+0000 7fa945600640 1 -- 172.21.15.46:0/1240053250 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa9400efd90 con 0x7fa94815db10 2024-03-29T08:18:51.075 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:18:51.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.076+0000 7fa94fc00640 1 -- 172.21.15.46:0/1240053250 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa924066690 msgr2=0x7fa924068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.076+0000 7fa94fc00640 1 --2- 172.21.15.46:0/1240053250 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa924066690 0x7fa924068b50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa9380011f0 tx=0x7fa938009040 comp rx=0 tx=0).stop 2024-03-29T08:18:51.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.076+0000 7fa94fc00640 1 -- 172.21.15.46:0/1240053250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94815db10 msgr2=0x7fa94815cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.076+0000 7fa94fc00640 1 --2- 172.21.15.46:0/1240053250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94815db10 0x7fa94815cf70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa9400b0040 tx=0x7fa940019710 comp rx=0 tx=0).stop 2024-03-29T08:18:51.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.076+0000 7fa94fc00640 1 -- 172.21.15.46:0/1240053250 shutdown_connections 2024-03-29T08:18:51.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.076+0000 7fa94fc00640 1 --2- 172.21.15.46:0/1240053250 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa924066690 0x7fa924068b50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.076+0000 7fa94fc00640 1 --2- 172.21.15.46:0/1240053250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa94815db10 0x7fa94815cf70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.076+0000 7fa94fc00640 1 --2- 172.21.15.46:0/1240053250 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa948142ab0 0x7fa9481f80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.076+0000 7fa94fc00640 1 --2- 172.21.15.46:0/1240053250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa94806dbb0 0x7fa9480cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.076+0000 7fa94fc00640 1 -- 172.21.15.46:0/1240053250 >> 172.21.15.46:0/1240053250 conn(0x7fa948066080 msgr2=0x7fa9480b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:51.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.076+0000 7fa94fc00640 1 -- 172.21.15.46:0/1240053250 shutdown_connections 2024-03-29T08:18:51.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.076+0000 7fa94fc00640 1 -- 172.21.15.46:0/1240053250 wait complete. 2024-03-29T08:18:51.097 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:18:51.097 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:18:51.106 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:51.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.244+0000 7f96c6a00640 1 Processor -- start 2024-03-29T08:18:51.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.244+0000 7f96c6a00640 1 -- start start 2024-03-29T08:18:51.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.244+0000 7f96c6a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96c01593d0 0x7f96c006cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:51.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.244+0000 7f96c6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96c015d230 0x7f96c015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:51.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.244+0000 7f96c6a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f96c015dbd0 0x7f96c015bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:51.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.244+0000 7f96c6a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c006d130 con 0x7f96c015d230 2024-03-29T08:18:51.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.244+0000 7f96c6a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96c015b8f0 con 0x7f96c015dbd0 2024-03-29T08:18:51.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.244+0000 7f96c6a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c015ad50 con 0x7f96c01593d0 2024-03-29T08:18:51.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.244+0000 7f96b7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96c015d230 0x7f96c015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:51.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.248+0000 7f96b7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96c015d230 0x7f96c015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50360/0 (socket says 172.21.15.46:50360) 2024-03-29T08:18:51.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.248+0000 7f96b7400640 1 -- 172.21.15.46:0/1485620099 learned_addr learned my addr 172.21.15.46:0/1485620099 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:51.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.248+0000 7f96bfe00640 1 --2- 172.21.15.46:0/1485620099 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96c01593d0 0x7f96c006cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:51.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.248+0000 7f96c4a00640 1 --2- 172.21.15.46:0/1485620099 >> v2:172.21.15.73:3300/0 conn(0x7f96c015dbd0 0x7f96c015bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:51.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.248+0000 7f96b7400640 1 -- 172.21.15.46:0/1485620099 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96c01593d0 msgr2=0x7f96c006cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:51.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.248+0000 7f96b7400640 1 --2- 172.21.15.46:0/1485620099 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96c01593d0 0x7f96c006cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.248+0000 7f96b7400640 1 -- 172.21.15.46:0/1485620099 >> v2:172.21.15.73:3300/0 conn(0x7f96c015dbd0 msgr2=0x7f96c015bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:51.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.248+0000 7f96b7400640 1 --2- 172.21.15.46:0/1485620099 >> v2:172.21.15.73:3300/0 conn(0x7f96c015dbd0 0x7f96c015bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.248+0000 7f96b7400640 1 -- 172.21.15.46:0/1485620099 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c015e060 con 0x7f96c015d230 2024-03-29T08:18:51.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.248+0000 7f96c4a00640 1 --2- 172.21.15.46:0/1485620099 >> v2:172.21.15.73:3300/0 conn(0x7f96c015dbd0 0x7f96c015bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:51.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.248+0000 7f96b7400640 1 --2- 172.21.15.46:0/1485620099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96c015d230 0x7f96c015d690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f96c015aef0 tx=0x7f96a801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3cb641d0d28dcc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:51.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.248+0000 7f96bf400640 1 -- 172.21.15.46:0/1485620099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a80aa620 con 0x7f96c015d230 2024-03-29T08:18:51.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.248+0000 7f96bf400640 1 -- 172.21.15.46:0/1485620099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a80aa780 con 0x7f96c015d230 2024-03-29T08:18:51.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.248+0000 7f96bf400640 1 -- 172.21.15.46:0/1485620099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a80aab10 con 0x7f96c015d230 2024-03-29T08:18:51.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c6a00640 1 -- 172.21.15.46:0/1485620099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96c015d230 msgr2=0x7f96c015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c6a00640 1 --2- 172.21.15.46:0/1485620099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96c015d230 0x7f96c015d690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f96c015aef0 tx=0x7f96a801c500 comp rx=0 tx=0).stop 2024-03-29T08:18:51.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c6a00640 1 -- 172.21.15.46:0/1485620099 shutdown_connections 2024-03-29T08:18:51.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c6a00640 1 --2- 172.21.15.46:0/1485620099 >> v2:172.21.15.73:3300/0 conn(0x7f96c015dbd0 0x7f96c015bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c6a00640 1 --2- 172.21.15.46:0/1485620099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96c015d230 0x7f96c015d690 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c6a00640 1 --2- 172.21.15.46:0/1485620099 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96c01593d0 0x7f96c006cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c6a00640 1 -- 172.21.15.46:0/1485620099 >> 172.21.15.46:0/1485620099 conn(0x7f96c0066110 msgr2=0x7f96c0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:51.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c6a00640 1 -- 172.21.15.46:0/1485620099 shutdown_connections 2024-03-29T08:18:51.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c6a00640 1 -- 172.21.15.46:0/1485620099 wait complete. 2024-03-29T08:18:51.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c6a00640 1 Processor -- start 2024-03-29T08:18:51.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c6a00640 1 -- start start 2024-03-29T08:18:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c6a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96c01593d0 0x7f96c0150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96c015d230 0x7f96c00cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96c015dbd0 0x7f96c00b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c6a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c00cc820 con 0x7f96c015d230 2024-03-29T08:18:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c6a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c0164660 con 0x7f96c015dbd0 2024-03-29T08:18:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c6a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96c014ec30 con 0x7f96c01593d0 2024-03-29T08:18:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96b7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96c015d230 0x7f96c00cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:51.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96b7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96c015d230 0x7f96c00cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50368/0 (socket says 172.21.15.46:50368) 2024-03-29T08:18:51.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96b7400640 1 -- 172.21.15.46:0/114034652 learned_addr learned my addr 172.21.15.46:0/114034652 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:51.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96bfe00640 1 --2- 172.21.15.46:0/114034652 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96c01593d0 0x7f96c0150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:51.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c4a00640 1 --2- 172.21.15.46:0/114034652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96c015dbd0 0x7f96c00b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:51.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96b7400640 1 -- 172.21.15.46:0/114034652 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96c01593d0 msgr2=0x7f96c0150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96b7400640 1 --2- 172.21.15.46:0/114034652 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96c01593d0 0x7f96c0150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96b7400640 1 -- 172.21.15.46:0/114034652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96c015dbd0 msgr2=0x7f96c00b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96b7400640 1 --2- 172.21.15.46:0/114034652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96c015dbd0 0x7f96c00b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96b7400640 1 -- 172.21.15.46:0/114034652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a80094d0 con 0x7f96c015d230 2024-03-29T08:18:51.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96c4a00640 1 --2- 172.21.15.46:0/114034652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96c015dbd0 0x7f96c00b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:51.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.252+0000 7f96b7400640 1 --2- 172.21.15.46:0/114034652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96c015d230 0x7f96c00cc2e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f96a801abb0 tx=0x7f96a801ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:51.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.256+0000 7f96bcc00640 1 -- 172.21.15.46:0/114034652 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a80aae40 con 0x7f96c015d230 2024-03-29T08:18:51.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.256+0000 7f96bcc00640 1 -- 172.21.15.46:0/114034652 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a80bd3f0 con 0x7f96c015d230 2024-03-29T08:18:51.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.256+0000 7f96bcc00640 1 -- 172.21.15.46:0/114034652 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a80bd7a0 con 0x7f96c015d230 2024-03-29T08:18:51.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.256+0000 7f96c6a00640 1 -- 172.21.15.46:0/114034652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c014e950 con 0x7f96c015d230 2024-03-29T08:18:51.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.256+0000 7f96c6a00640 1 -- 172.21.15.46:0/114034652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c0164900 con 0x7f96c015d230 2024-03-29T08:18:51.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.256+0000 7f96c6a00640 1 -- 172.21.15.46:0/114034652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c0066c90 con 0x7f96c015d230 2024-03-29T08:18:51.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.256+0000 7f96bcc00640 1 -- 172.21.15.46:0/114034652 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f96a80b33f0 con 0x7f96c015d230 2024-03-29T08:18:51.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.256+0000 7f96bcc00640 1 --2- 172.21.15.46:0/114034652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f969c066690 0x7f969c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:51.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.256+0000 7f96bcc00640 1 -- 172.21.15.46:0/114034652 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96a811e650 con 0x7f96c015d230 2024-03-29T08:18:51.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.256+0000 7f96bfe00640 1 --2- 172.21.15.46:0/114034652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f969c066690 0x7f969c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:51.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.256+0000 7f96bfe00640 1 --2- 172.21.15.46:0/114034652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f969c066690 0x7f969c068b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009a40 tx=0x7f96b0009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:51.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.260+0000 7f96bcc00640 1 -- 172.21.15.46:0/114034652 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f96a80ebaf0 con 0x7f96c015d230 2024-03-29T08:18:51.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.428+0000 7f96c6a00640 1 -- 172.21.15.46:0/114034652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96c006d3d0 con 0x7f96c015d230 2024-03-29T08:18:51.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.432+0000 7f96bcc00640 1 -- 172.21.15.46:0/114034652 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f96a80ef110 con 0x7f96c015d230 2024-03-29T08:18:51.432 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:51.432 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:14.314124+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:51.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.432+0000 7f96c6a00640 1 -- 172.21.15.46:0/114034652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f969c066690 msgr2=0x7f969c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.432+0000 7f96c6a00640 1 --2- 172.21.15.46:0/114034652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f969c066690 0x7f969c068b50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009a40 tx=0x7f96b0009190 comp rx=0 tx=0).stop 2024-03-29T08:18:51.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.432+0000 7f96c6a00640 1 -- 172.21.15.46:0/114034652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96c015d230 msgr2=0x7f96c00cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.432+0000 7f96c6a00640 1 --2- 172.21.15.46:0/114034652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96c015d230 0x7f96c00cc2e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f96a801abb0 tx=0x7f96a801ad40 comp rx=0 tx=0).stop 2024-03-29T08:18:51.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.432+0000 7f96c6a00640 1 -- 172.21.15.46:0/114034652 shutdown_connections 2024-03-29T08:18:51.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.432+0000 7f96c6a00640 1 --2- 172.21.15.46:0/114034652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f969c066690 0x7f969c068b50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.432+0000 7f96c6a00640 1 --2- 172.21.15.46:0/114034652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96c015dbd0 0x7f96c00b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.432+0000 7f96c6a00640 1 --2- 172.21.15.46:0/114034652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96c015d230 0x7f96c00cc2e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.432+0000 7f96c6a00640 1 --2- 172.21.15.46:0/114034652 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96c01593d0 0x7f96c0150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.432+0000 7f96c6a00640 1 -- 172.21.15.46:0/114034652 >> 172.21.15.46:0/114034652 conn(0x7f96c0066110 msgr2=0x7f96c00ade10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:51.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.432+0000 7f96c6a00640 1 -- 172.21.15.46:0/114034652 shutdown_connections 2024-03-29T08:18:51.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.432+0000 7f96c6a00640 1 -- 172.21.15.46:0/114034652 wait complete. 2024-03-29T08:18:51.453 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:18:11.177842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:18:12.308252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:18:51.453 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:18:51.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.596+0000 7f0d20600640 1 Processor -- start 2024-03-29T08:18:51.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.596+0000 7f0d20600640 1 -- start start 2024-03-29T08:18:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.596+0000 7f0d20600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d1815c950 0x7f0d1806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.596+0000 7f0d20600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d180cfb20 con 0x7f0d18160db0 2024-03-29T08:18:51.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.596+0000 7f0d20600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d180cef10 con 0x7f0d180cea30 2024-03-29T08:18:51.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.596+0000 7f0d20600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d180cf1f0 con 0x7f0d1815c950 2024-03-29T08:18:51.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.596+0000 7f0d1e600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0d18160db0 0x7f0d180cad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38606/0 (socket says 172.21.15.46:38606) 2024-03-29T08:18:51.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.596+0000 7f0d1e600640 1 -- 172.21.15.46:0/90090339 learned_addr learned my addr 172.21.15.46:0/90090339 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:51.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.596+0000 7f0d1dc00640 1 --2- 172.21.15.46:0/90090339 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d1815c950 0x7f0d1806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:51.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.596+0000 7f0d17e00640 1 -- 172.21.15.46:0/90090339 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1881762203 0 0) 0x7f0d180cfb20 con 0x7f0d18160db0 2024-03-29T08:18:51.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.596+0000 7f0d17e00640 1 -- 172.21.15.46:0/90090339 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d00003530 con 0x7f0d18160db0 2024-03-29T08:18:51.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.596+0000 7f0d1dc00640 1 -- 172.21.15.46:0/90090339 >> v1:172.21.15.73:6789/0 conn(0x7f0d180cea30 legacy=0x7f0d181606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.596+0000 7f0d17e00640 1 -- 172.21.15.46:0/90090339 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 735715875 0 0) 0x7f0d180cef10 con 0x7f0d180cea30 2024-03-29T08:18:51.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.596+0000 7f0d1dc00640 1 -- 172.21.15.46:0/90090339 >> v1:172.21.15.46:6789/0 conn(0x7f0d18160db0 legacy=0x7f0d180cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.596+0000 7f0d1dc00640 1 -- 172.21.15.46:0/90090339 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d180cf4d0 con 0x7f0d1815c950 2024-03-29T08:18:51.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.600+0000 7f0d1dc00640 1 --2- 172.21.15.46:0/90090339 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d1815c950 0x7f0d1806bbe0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0097f0 tx=0x7f0d0c01ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c39a3367f2e46d31 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:51.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.600+0000 7f0d17e00640 1 -- 172.21.15.46:0/90090339 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0a89d0 con 0x7f0d1815c950 2024-03-29T08:18:51.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.600+0000 7f0d17e00640 1 -- 172.21.15.46:0/90090339 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c0a8b30 con 0x7f0d1815c950 2024-03-29T08:18:51.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.600+0000 7f0d17e00640 1 -- 172.21.15.46:0/90090339 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0b0440 con 0x7f0d1815c950 2024-03-29T08:18:51.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.600+0000 7f0d20600640 1 -- 172.21.15.46:0/90090339 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d1815c950 msgr2=0x7f0d1806bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.600+0000 7f0d20600640 1 --2- 172.21.15.46:0/90090339 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d1815c950 0x7f0d1806bbe0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0097f0 tx=0x7f0d0c01ecd0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.600+0000 7f0d20600640 1 -- 172.21.15.46:0/90090339 shutdown_connections 2024-03-29T08:18:51.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.600+0000 7f0d20600640 1 --2- 172.21.15.46:0/90090339 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d1815c950 0x7f0d1806bbe0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.600+0000 7f0d20600640 1 -- 172.21.15.46:0/90090339 >> 172.21.15.46:0/90090339 conn(0x7f0d180b21b0 msgr2=0x7f0d180b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:51.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.600+0000 7f0d20600640 1 -- 172.21.15.46:0/90090339 shutdown_connections 2024-03-29T08:18:51.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.600+0000 7f0d20600640 1 -- 172.21.15.46:0/90090339 wait complete. 2024-03-29T08:18:51.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d20600640 1 Processor -- start 2024-03-29T08:18:51.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d20600640 1 -- start start 2024-03-29T08:18:51.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d20600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d180cea30 0x7f0d18157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:51.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d20600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d1815c950 0x7f0d181e7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:51.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d20600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d18160db0 0x7f0d180b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:51.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d20600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d181e8ad0 con 0x7f0d1815c950 2024-03-29T08:18:51.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d20600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d181e7ec0 con 0x7f0d18160db0 2024-03-29T08:18:51.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d20600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d181e81a0 con 0x7f0d180cea30 2024-03-29T08:18:51.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d1d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d1815c950 0x7f0d181e7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:51.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d1dc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d180cea30 0x7f0d18157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d1d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d1815c950 0x7f0d181e7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50384/0 (socket says 172.21.15.46:50384) 2024-03-29T08:18:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d1d200640 1 -- 172.21.15.46:0/103681063 learned_addr learned my addr 172.21.15.46:0/103681063 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d1e600640 1 --2- 172.21.15.46:0/103681063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d18160db0 0x7f0d180b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d1d200640 1 -- 172.21.15.46:0/103681063 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d180cea30 msgr2=0x7f0d18157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d1d200640 1 --2- 172.21.15.46:0/103681063 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d180cea30 0x7f0d18157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d1d200640 1 -- 172.21.15.46:0/103681063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d18160db0 msgr2=0x7f0d180b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d1d200640 1 --2- 172.21.15.46:0/103681063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d18160db0 0x7f0d180b4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d1d200640 1 -- 172.21.15.46:0/103681063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c009420 con 0x7f0d1815c950 2024-03-29T08:18:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d1dc00640 1 --2- 172.21.15.46:0/103681063 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d180cea30 0x7f0d18157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d1d200640 1 --2- 172.21.15.46:0/103681063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d1815c950 0x7f0d181e7440 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400c4d0 tx=0x7f0d0400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d15600640 1 -- 172.21.15.46:0/103681063 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04005480 con 0x7f0d1815c950 2024-03-29T08:18:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d15600640 1 -- 172.21.15.46:0/103681063 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d04003600 con 0x7f0d1815c950 2024-03-29T08:18:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d15600640 1 -- 172.21.15.46:0/103681063 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d040039b0 con 0x7f0d1815c950 2024-03-29T08:18:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d20600640 1 -- 172.21.15.46:0/103681063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d181e8480 con 0x7f0d1815c950 2024-03-29T08:18:51.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d20600640 1 -- 172.21.15.46:0/103681063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d18158030 con 0x7f0d1815c950 2024-03-29T08:18:51.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.604+0000 7f0d20600640 1 -- 172.21.15.46:0/103681063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d181e8c40 con 0x7f0d1815c950 2024-03-29T08:18:51.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.608+0000 7f0d15600640 1 -- 172.21.15.46:0/103681063 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0d04003b10 con 0x7f0d1815c950 2024-03-29T08:18:51.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.608+0000 7f0d15600640 1 --2- 172.21.15.46:0/103681063 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0cf8066690 0x7f0cf8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:51.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.608+0000 7f0d15600640 1 -- 172.21.15.46:0/103681063 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d04083710 con 0x7f0d1815c950 2024-03-29T08:18:51.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.608+0000 7f0d1dc00640 1 --2- 172.21.15.46:0/103681063 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0cf8066690 0x7f0cf8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:51.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.608+0000 7f0d1dc00640 1 --2- 172.21.15.46:0/103681063 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0cf8066690 0x7f0cf8068b50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0b2000 tx=0x7f0d0c0aa070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:51.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.612+0000 7f0d15600640 1 -- 172.21.15.46:0/103681063 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0d04050b60 con 0x7f0d1815c950 2024-03-29T08:18:51.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.772+0000 7f0d20600640 1 -- 172.21.15.46:0/103681063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d1809e690 con 0x7f0d1815c950 2024-03-29T08:18:51.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.772+0000 7f0d15600640 1 -- 172.21.15.46:0/103681063 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d04054180 con 0x7f0d1815c950 2024-03-29T08:18:51.773 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:18:51.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.772+0000 7f0d20600640 1 -- 172.21.15.46:0/103681063 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0cf8066690 msgr2=0x7f0cf8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.772+0000 7f0d20600640 1 --2- 172.21.15.46:0/103681063 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0cf8066690 0x7f0cf8068b50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0b2000 tx=0x7f0d0c0aa070 comp rx=0 tx=0).stop 2024-03-29T08:18:51.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.772+0000 7f0d20600640 1 -- 172.21.15.46:0/103681063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d1815c950 msgr2=0x7f0d181e7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.772+0000 7f0d20600640 1 --2- 172.21.15.46:0/103681063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d1815c950 0x7f0d181e7440 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400c4d0 tx=0x7f0d0400c990 comp rx=0 tx=0).stop 2024-03-29T08:18:51.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.772+0000 7f0d20600640 1 -- 172.21.15.46:0/103681063 shutdown_connections 2024-03-29T08:18:51.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.772+0000 7f0d20600640 1 --2- 172.21.15.46:0/103681063 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0cf8066690 0x7f0cf8068b50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.772+0000 7f0d20600640 1 --2- 172.21.15.46:0/103681063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d18160db0 0x7f0d180b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.772+0000 7f0d20600640 1 --2- 172.21.15.46:0/103681063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d1815c950 0x7f0d181e7440 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.772+0000 7f0d20600640 1 --2- 172.21.15.46:0/103681063 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d180cea30 0x7f0d18157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.772+0000 7f0d20600640 1 -- 172.21.15.46:0/103681063 >> 172.21.15.46:0/103681063 conn(0x7f0d180b21b0 msgr2=0x7f0d1806f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:51.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.772+0000 7f0d20600640 1 -- 172.21.15.46:0/103681063 shutdown_connections 2024-03-29T08:18:51.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.772+0000 7f0d20600640 1 -- 172.21.15.46:0/103681063 wait complete. 2024-03-29T08:18:51.794 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:18:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.936+0000 7f51d3400640 1 Processor -- start 2024-03-29T08:18:51.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.936+0000 7f51d3400640 1 -- start start 2024-03-29T08:18:51.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.936+0000 7f51d3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f51cc0b1570 0x7f51cc0b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:51.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.936+0000 7f51d3400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51cc0cb3f0 con 0x7f51cc098a40 2024-03-29T08:18:51.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.936+0000 7f51d3400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51cc0cb620 con 0x7f51cc15dfb0 2024-03-29T08:18:51.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.936+0000 7f51d3400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51cc0b1e00 con 0x7f51cc0b1570 2024-03-29T08:18:51.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.936+0000 7f51c3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f51cc0b1570 0x7f51cc0b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:51.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.936+0000 7f51d1400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f51cc098a40 0x7f51cc0cc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38616/0 (socket says 172.21.15.46:38616) 2024-03-29T08:18:51.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.936+0000 7f51d1400640 1 -- 172.21.15.46:0/1536712666 learned_addr learned my addr 172.21.15.46:0/1536712666 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.936+0000 7f51cbe00640 1 -- 172.21.15.46:0/1536712666 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1077541323 0 0) 0x7f51cc0cb3f0 con 0x7f51cc098a40 2024-03-29T08:18:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.936+0000 7f51cbe00640 1 -- 172.21.15.46:0/1536712666 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51b0003530 con 0x7f51cc098a40 2024-03-29T08:18:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.940+0000 7f51c3e00640 1 -- 172.21.15.46:0/1536712666 >> v1:172.21.15.73:6789/0 conn(0x7f51cc15dfb0 legacy=0x7f51cc0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.940+0000 7f51cbe00640 1 -- 172.21.15.46:0/1536712666 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 566827315 0 0) 0x7f51cc0cb620 con 0x7f51cc15dfb0 2024-03-29T08:18:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.940+0000 7f51c3e00640 1 -- 172.21.15.46:0/1536712666 >> v1:172.21.15.46:6789/0 conn(0x7f51cc098a40 legacy=0x7f51cc0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.940+0000 7f51c3e00640 1 -- 172.21.15.46:0/1536712666 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51cc0cf4f0 con 0x7f51cc0b1570 2024-03-29T08:18:51.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.940+0000 7f51c3e00640 1 --2- 172.21.15.46:0/1536712666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51cc0b1570 0x7f51cc0b3580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f51b40097c0 tx=0x7f51b401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4d5841808765f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:51.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.940+0000 7f51cbe00640 1 -- 172.21.15.46:0/1536712666 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b40ad680 con 0x7f51cc0b1570 2024-03-29T08:18:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.940+0000 7f51cbe00640 1 -- 172.21.15.46:0/1536712666 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b40ad7e0 con 0x7f51cc0b1570 2024-03-29T08:18:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.940+0000 7f51cbe00640 1 -- 172.21.15.46:0/1536712666 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b40adb70 con 0x7f51cc0b1570 2024-03-29T08:18:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.940+0000 7f51d3400640 1 -- 172.21.15.46:0/1536712666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51cc0b1570 msgr2=0x7f51cc0b3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.940+0000 7f51d3400640 1 --2- 172.21.15.46:0/1536712666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51cc0b1570 0x7f51cc0b3580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f51b40097c0 tx=0x7f51b401c470 comp rx=0 tx=0).stop 2024-03-29T08:18:51.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.940+0000 7f51d3400640 1 -- 172.21.15.46:0/1536712666 shutdown_connections 2024-03-29T08:18:51.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.940+0000 7f51d3400640 1 --2- 172.21.15.46:0/1536712666 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51cc0b1570 0x7f51cc0b3580 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.940+0000 7f51d3400640 1 -- 172.21.15.46:0/1536712666 >> 172.21.15.46:0/1536712666 conn(0x7f51cc0b2220 msgr2=0x7f51cc0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:51.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.940+0000 7f51d3400640 1 -- 172.21.15.46:0/1536712666 shutdown_connections 2024-03-29T08:18:51.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.940+0000 7f51d3400640 1 -- 172.21.15.46:0/1536712666 wait complete. 2024-03-29T08:18:51.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d3400640 1 Processor -- start 2024-03-29T08:18:51.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d3400640 1 -- start start 2024-03-29T08:18:51.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d3400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51cc098a40 0x7f51cc154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:51.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d3400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51cc0b1570 0x7f51cc0c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51cc15dfb0 0x7f51cc06f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d3400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc0c8b80 con 0x7f51cc15dfb0 2024-03-29T08:18:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d3400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc0c7f00 con 0x7f51cc098a40 2024-03-29T08:18:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d3400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51cc0c81e0 con 0x7f51cc0b1570 2024-03-29T08:18:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51cc15dfb0 0x7f51cc06f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51cc15dfb0 0x7f51cc06f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50404/0 (socket says 172.21.15.46:50404) 2024-03-29T08:18:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d1400640 1 -- 172.21.15.46:0/1675120250 learned_addr learned my addr 172.21.15.46:0/1675120250 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d0a00640 1 --2- 172.21.15.46:0/1675120250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51cc098a40 0x7f51cc154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51c3e00640 1 --2- 172.21.15.46:0/1675120250 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51cc0b1570 0x7f51cc0c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d1400640 1 -- 172.21.15.46:0/1675120250 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51cc0b1570 msgr2=0x7f51cc0c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d1400640 1 --2- 172.21.15.46:0/1675120250 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51cc0b1570 0x7f51cc0c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d1400640 1 -- 172.21.15.46:0/1675120250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51cc098a40 msgr2=0x7f51cc154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d1400640 1 --2- 172.21.15.46:0/1675120250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51cc098a40 0x7f51cc154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d1400640 1 -- 172.21.15.46:0/1675120250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b4009420 con 0x7f51cc15dfb0 2024-03-29T08:18:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51c3e00640 1 --2- 172.21.15.46:0/1675120250 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51cc0b1570 0x7f51cc0c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d1400640 1 --2- 172.21.15.46:0/1675120250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51cc15dfb0 0x7f51cc06f750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f51c400cb40 tx=0x7f51c400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51c9600640 1 -- 172.21.15.46:0/1675120250 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c400aca0 con 0x7f51cc15dfb0 2024-03-29T08:18:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51c9600640 1 -- 172.21.15.46:0/1675120250 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c400ae00 con 0x7f51cc15dfb0 2024-03-29T08:18:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51c9600640 1 -- 172.21.15.46:0/1675120250 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4012620 con 0x7f51cc15dfb0 2024-03-29T08:18:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d3400640 1 -- 172.21.15.46:0/1675120250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51cc0993e0 con 0x7f51cc15dfb0 2024-03-29T08:18:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.944+0000 7f51d3400640 1 -- 172.21.15.46:0/1675120250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51cc0c87a0 con 0x7f51cc15dfb0 2024-03-29T08:18:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.948+0000 7f51d3400640 1 -- 172.21.15.46:0/1675120250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51cc09e700 con 0x7f51cc15dfb0 2024-03-29T08:18:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.948+0000 7f51c9600640 1 -- 172.21.15.46:0/1675120250 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f51c4007070 con 0x7f51cc15dfb0 2024-03-29T08:18:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.948+0000 7f51c9600640 1 --2- 172.21.15.46:0/1675120250 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f51a80666c0 0x7f51a8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.948+0000 7f51c9600640 1 -- 172.21.15.46:0/1675120250 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51c4084200 con 0x7f51cc15dfb0 2024-03-29T08:18:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.948+0000 7f51d0a00640 1 --2- 172.21.15.46:0/1675120250 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f51a80666c0 0x7f51a8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.948+0000 7f51d0a00640 1 --2- 172.21.15.46:0/1675120250 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f51a80666c0 0x7f51a8068b80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f51bc000f70 tx=0x7f51bc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:51.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:51.952+0000 7f51c9600640 1 -- 172.21.15.46:0/1675120250 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f51c40514f0 con 0x7f51cc15dfb0 2024-03-29T08:18:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.116+0000 7f51d3400640 1 -- 172.21.15.46:0/1675120250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51cc04e120 con 0x7f51cc15dfb0 2024-03-29T08:18:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.116+0000 7f51c9600640 1 -- 172.21.15.46:0/1675120250 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f51c4054b10 con 0x7f51cc15dfb0 2024-03-29T08:18:52.118 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:18:52.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.120+0000 7f51d3400640 1 -- 172.21.15.46:0/1675120250 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f51a80666c0 msgr2=0x7f51a8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:52.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.120+0000 7f51d3400640 1 --2- 172.21.15.46:0/1675120250 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f51a80666c0 0x7f51a8068b80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f51bc000f70 tx=0x7f51bc009040 comp rx=0 tx=0).stop 2024-03-29T08:18:52.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.120+0000 7f51d3400640 1 -- 172.21.15.46:0/1675120250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51cc15dfb0 msgr2=0x7f51cc06f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.120+0000 7f51d3400640 1 --2- 172.21.15.46:0/1675120250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51cc15dfb0 0x7f51cc06f750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f51c400cb40 tx=0x7f51c400a060 comp rx=0 tx=0).stop 2024-03-29T08:18:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.120+0000 7f51d3400640 1 -- 172.21.15.46:0/1675120250 shutdown_connections 2024-03-29T08:18:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.120+0000 7f51d3400640 1 --2- 172.21.15.46:0/1675120250 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f51a80666c0 0x7f51a8068b80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.120+0000 7f51d3400640 1 --2- 172.21.15.46:0/1675120250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51cc15dfb0 0x7f51cc06f750 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.120+0000 7f51d3400640 1 --2- 172.21.15.46:0/1675120250 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51cc0b1570 0x7f51cc0c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.120+0000 7f51d3400640 1 --2- 172.21.15.46:0/1675120250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51cc098a40 0x7f51cc154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.120+0000 7f51d3400640 1 -- 172.21.15.46:0/1675120250 >> 172.21.15.46:0/1675120250 conn(0x7f51cc0b2220 msgr2=0x7f51cc0b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.120+0000 7f51d3400640 1 -- 172.21.15.46:0/1675120250 shutdown_connections 2024-03-29T08:18:52.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.120+0000 7f51d3400640 1 -- 172.21.15.46:0/1675120250 wait complete. 2024-03-29T08:18:52.139 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:18:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac30c00640 1 Processor -- start 2024-03-29T08:18:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac30c00640 1 -- start start 2024-03-29T08:18:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac30c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fac2c143eb0 0x7fac2c0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac30c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fac2c15efe0 0x7fac2c0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac30c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac2c0b2360 con 0x7fac2c0b1630 2024-03-29T08:18:52.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac30c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac2c15e500 con 0x7fac2c15efe0 2024-03-29T08:18:52.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac30c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c15e7e0 con 0x7fac2c143eb0 2024-03-29T08:18:52.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac21400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fac2c15efe0 0x7fac2c0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:52.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac29e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fac2c143eb0 0x7fac2c0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:52.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac21400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fac2c15efe0 0x7fac2c0cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47306/0 (socket says 172.21.15.46:47306) 2024-03-29T08:18:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac29e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fac2c143eb0 0x7fac2c0cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:40152/0 (socket says 172.21.15.46:40152) 2024-03-29T08:18:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac21400640 1 -- 172.21.15.46:0/3612239976 learned_addr learned my addr 172.21.15.46:0/3612239976 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac2a800640 1 --1- 172.21.15.46:0/3612239976 >> v1:172.21.15.73:6789/0 conn(0x7fac2c0b1630 0x7fac2c0b1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:41906/0 (socket says 172.21.15.46:41906) 2024-03-29T08:18:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac29e00640 1 -- 172.21.15.46:0/3612239976 >> v2:172.21.15.73:3300/0 conn(0x7fac2c15efe0 msgr2=0x7fac2c0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac29e00640 1 --2- 172.21.15.46:0/3612239976 >> v2:172.21.15.73:3300/0 conn(0x7fac2c15efe0 0x7fac2c0cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac29e00640 1 -- 172.21.15.46:0/3612239976 >> v1:172.21.15.73:6789/0 conn(0x7fac2c0b1630 legacy=0x7fac2c0b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac29e00640 1 -- 172.21.15.46:0/3612239976 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2c15eac0 con 0x7fac2c143eb0 2024-03-29T08:18:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.280+0000 7fac21400640 1 --2- 172.21.15.46:0/3612239976 >> v2:172.21.15.73:3300/0 conn(0x7fac2c15efe0 0x7fac2c0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:52.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.284+0000 7fac29e00640 1 --2- 172.21.15.46:0/3612239976 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fac2c143eb0 0x7fac2c0cb790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009d90 tx=0x7fac1c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23349649161b7383 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:52.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.284+0000 7fac29400640 1 -- 172.21.15.46:0/3612239976 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0a5830 con 0x7fac2c143eb0 2024-03-29T08:18:52.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.284+0000 7fac29400640 1 -- 172.21.15.46:0/3612239976 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c0a5990 con 0x7fac2c143eb0 2024-03-29T08:18:52.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.284+0000 7fac29400640 1 -- 172.21.15.46:0/3612239976 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0a5d20 con 0x7fac2c143eb0 2024-03-29T08:18:52.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.284+0000 7fac30c00640 1 -- 172.21.15.46:0/3612239976 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fac2c143eb0 msgr2=0x7fac2c0cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.284+0000 7fac30c00640 1 --2- 172.21.15.46:0/3612239976 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fac2c143eb0 0x7fac2c0cb790 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009d90 tx=0x7fac1c01c970 comp rx=0 tx=0).stop 2024-03-29T08:18:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.284+0000 7fac30c00640 1 -- 172.21.15.46:0/3612239976 shutdown_connections 2024-03-29T08:18:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.284+0000 7fac30c00640 1 --2- 172.21.15.46:0/3612239976 >> v2:172.21.15.73:3300/0 conn(0x7fac2c15efe0 0x7fac2c0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.284+0000 7fac30c00640 1 --2- 172.21.15.46:0/3612239976 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fac2c143eb0 0x7fac2c0cb790 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.284+0000 7fac30c00640 1 -- 172.21.15.46:0/3612239976 >> 172.21.15.46:0/3612239976 conn(0x7fac2c06a390 msgr2=0x7fac2c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.284+0000 7fac30c00640 1 -- 172.21.15.46:0/3612239976 shutdown_connections 2024-03-29T08:18:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.284+0000 7fac30c00640 1 -- 172.21.15.46:0/3612239976 wait complete. 2024-03-29T08:18:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac30c00640 1 Processor -- start 2024-03-29T08:18:52.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac30c00640 1 -- start start 2024-03-29T08:18:52.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac30c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac2c0b1630 0x7fac2c163b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:52.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac30c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac2c143eb0 0x7fac2c159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:52.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac30c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fac2c15efe0 0x7fac2c15a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:52.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac30c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c1f93f0 con 0x7fac2c143eb0 2024-03-29T08:18:52.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac30c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c0ae2c0 con 0x7fac2c0b1630 2024-03-29T08:18:52.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac30c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c0ae5a0 con 0x7fac2c15efe0 2024-03-29T08:18:52.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac21400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac2c143eb0 0x7fac2c159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac21400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac2c143eb0 0x7fac2c159e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50406/0 (socket says 172.21.15.46:50406) 2024-03-29T08:18:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac21400640 1 -- 172.21.15.46:0/1812465955 learned_addr learned my addr 172.21.15.46:0/1812465955 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac29e00640 1 --2- 172.21.15.46:0/1812465955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac2c0b1630 0x7fac2c163b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac2a800640 1 --2- 172.21.15.46:0/1812465955 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fac2c15efe0 0x7fac2c15a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac21400640 1 -- 172.21.15.46:0/1812465955 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fac2c15efe0 msgr2=0x7fac2c15a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac21400640 1 --2- 172.21.15.46:0/1812465955 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fac2c15efe0 0x7fac2c15a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac21400640 1 -- 172.21.15.46:0/1812465955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac2c0b1630 msgr2=0x7fac2c163b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac21400640 1 --2- 172.21.15.46:0/1812465955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac2c0b1630 0x7fac2c163b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac21400640 1 -- 172.21.15.46:0/1812465955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac140094d0 con 0x7fac2c143eb0 2024-03-29T08:18:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac21400640 1 --2- 172.21.15.46:0/1812465955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac2c143eb0 0x7fac2c159e90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fac1400e9d0 tx=0x7fac1400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac22a00640 1 -- 172.21.15.46:0/1812465955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1400ca60 con 0x7fac2c143eb0 2024-03-29T08:18:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac22a00640 1 -- 172.21.15.46:0/1812465955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1400cbc0 con 0x7fac2c143eb0 2024-03-29T08:18:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac22a00640 1 -- 172.21.15.46:0/1812465955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14013470 con 0x7fac2c143eb0 2024-03-29T08:18:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac30c00640 1 -- 172.21.15.46:0/1812465955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1c009a70 con 0x7fac2c143eb0 2024-03-29T08:18:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.288+0000 7fac30c00640 1 -- 172.21.15.46:0/1812465955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2c0aeb60 con 0x7fac2c143eb0 2024-03-29T08:18:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.292+0000 7fac30c00640 1 -- 172.21.15.46:0/1812465955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2c071a70 con 0x7fac2c143eb0 2024-03-29T08:18:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.292+0000 7fac22a00640 1 -- 172.21.15.46:0/1812465955 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fac140135d0 con 0x7fac2c143eb0 2024-03-29T08:18:52.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.292+0000 7fac22a00640 1 --2- 172.21.15.46:0/1812465955 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fac080666c0 0x7fac08068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:52.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.292+0000 7fac22a00640 1 -- 172.21.15.46:0/1812465955 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac14084490 con 0x7fac2c143eb0 2024-03-29T08:18:52.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.292+0000 7fac29e00640 1 --2- 172.21.15.46:0/1812465955 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fac080666c0 0x7fac08068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:52.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.292+0000 7fac29e00640 1 --2- 172.21.15.46:0/1812465955 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fac080666c0 0x7fac08068b80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0af000 tx=0x7fac1c0a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:52.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.296+0000 7fac22a00640 1 -- 172.21.15.46:0/1812465955 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fac14051810 con 0x7fac2c143eb0 2024-03-29T08:18:52.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.456+0000 7fac30c00640 1 -- 172.21.15.46:0/1812465955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fac2c0cadd0 con 0x7fac2c143eb0 2024-03-29T08:18:52.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.456+0000 7fac22a00640 1 -- 172.21.15.46:0/1812465955 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fac1405b030 con 0x7fac2c143eb0 2024-03-29T08:18:52.457 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:18:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.456+0000 7fac30c00640 1 -- 172.21.15.46:0/1812465955 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fac080666c0 msgr2=0x7fac08068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.456+0000 7fac30c00640 1 --2- 172.21.15.46:0/1812465955 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fac080666c0 0x7fac08068b80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0af000 tx=0x7fac1c0a7070 comp rx=0 tx=0).stop 2024-03-29T08:18:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.456+0000 7fac30c00640 1 -- 172.21.15.46:0/1812465955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac2c143eb0 msgr2=0x7fac2c159e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.456+0000 7fac30c00640 1 --2- 172.21.15.46:0/1812465955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac2c143eb0 0x7fac2c159e90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fac1400e9d0 tx=0x7fac1400ee90 comp rx=0 tx=0).stop 2024-03-29T08:18:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.456+0000 7fac30c00640 1 -- 172.21.15.46:0/1812465955 shutdown_connections 2024-03-29T08:18:52.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.456+0000 7fac30c00640 1 --2- 172.21.15.46:0/1812465955 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fac080666c0 0x7fac08068b80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.456+0000 7fac30c00640 1 --2- 172.21.15.46:0/1812465955 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fac2c15efe0 0x7fac2c15a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.456+0000 7fac30c00640 1 --2- 172.21.15.46:0/1812465955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac2c143eb0 0x7fac2c159e90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.456+0000 7fac30c00640 1 --2- 172.21.15.46:0/1812465955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac2c0b1630 0x7fac2c163b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.456+0000 7fac30c00640 1 -- 172.21.15.46:0/1812465955 >> 172.21.15.46:0/1812465955 conn(0x7fac2c06a390 msgr2=0x7fac2c140a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.456+0000 7fac30c00640 1 -- 172.21.15.46:0/1812465955 shutdown_connections 2024-03-29T08:18:52.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.456+0000 7fac30c00640 1 -- 172.21.15.46:0/1812465955 wait complete. 2024-03-29T08:18:52.478 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:18:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.616+0000 7fdbf9a00640 1 Processor -- start 2024-03-29T08:18:52.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.616+0000 7fdbf9a00640 1 -- start start 2024-03-29T08:18:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.616+0000 7fdbf9a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbf4158300 0x7fdbf406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.616+0000 7fdbf9a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdbf40af530 0x7fdbf40af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.616+0000 7fdbf9a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbf406c390 con 0x7fdbf40702a0 2024-03-29T08:18:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.616+0000 7fdbf9a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf406d600 con 0x7fdbf40af530 2024-03-29T08:18:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.616+0000 7fdbf9a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf40afff0 con 0x7fdbf4158300 2024-03-29T08:18:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.620+0000 7fdbea000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdbf40af530 0x7fdbf40af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.620+0000 7fdbea000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdbf40af530 0x7fdbf40af990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50410/0 (socket says 172.21.15.46:50410) 2024-03-29T08:18:52.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.620+0000 7fdbea000640 1 -- 172.21.15.46:0/2572916124 learned_addr learned my addr 172.21.15.46:0/2572916124 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.620+0000 7fdbf2a00640 1 --2- 172.21.15.46:0/2572916124 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbf4158300 0x7fdbf406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.620+0000 7fdbea000640 1 -- 172.21.15.46:0/2572916124 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbf4158300 msgr2=0x7fdbf406d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.620+0000 7fdbea000640 1 --2- 172.21.15.46:0/2572916124 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbf4158300 0x7fdbf406d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.620+0000 7fdbea000640 1 -- 172.21.15.46:0/2572916124 >> v1:172.21.15.46:6789/0 conn(0x7fdbf40702a0 legacy=0x7fdbf415e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:52.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.620+0000 7fdbea000640 1 -- 172.21.15.46:0/2572916124 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf406c5e0 con 0x7fdbf40af530 2024-03-29T08:18:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.620+0000 7fdbea000640 1 --2- 172.21.15.46:0/2572916124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf40af530 0x7fdbf40af990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009870 tx=0x7fdbdc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a729738c081ec320 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.620+0000 7fdbf2000640 1 -- 172.21.15.46:0/2572916124 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0ab610 con 0x7fdbf40af530 2024-03-29T08:18:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.620+0000 7fdbf2000640 1 -- 172.21.15.46:0/2572916124 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbdc0ab770 con 0x7fdbf40af530 2024-03-29T08:18:52.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.620+0000 7fdbf2000640 1 -- 172.21.15.46:0/2572916124 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0abb00 con 0x7fdbf40af530 2024-03-29T08:18:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf9a00640 1 -- 172.21.15.46:0/2572916124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf40af530 msgr2=0x7fdbf40af990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf9a00640 1 --2- 172.21.15.46:0/2572916124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf40af530 0x7fdbf40af990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc009870 tx=0x7fdbdc01c410 comp rx=0 tx=0).stop 2024-03-29T08:18:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf9a00640 1 -- 172.21.15.46:0/2572916124 shutdown_connections 2024-03-29T08:18:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf9a00640 1 --2- 172.21.15.46:0/2572916124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf40af530 0x7fdbf40af990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf9a00640 1 --2- 172.21.15.46:0/2572916124 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbf4158300 0x7fdbf406d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf9a00640 1 -- 172.21.15.46:0/2572916124 >> 172.21.15.46:0/2572916124 conn(0x7fdbf4066080 msgr2=0x7fdbf4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:52.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf9a00640 1 -- 172.21.15.46:0/2572916124 shutdown_connections 2024-03-29T08:18:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf9a00640 1 -- 172.21.15.46:0/2572916124 wait complete. 2024-03-29T08:18:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf9a00640 1 Processor -- start 2024-03-29T08:18:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf9a00640 1 -- start start 2024-03-29T08:18:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf9a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbf40702a0 0x7fdbf4142820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf9a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbf4158300 0x7fdbf4145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf40a8800 0x7fdbf40cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf9a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf40adf50 con 0x7fdbf40a8800 2024-03-29T08:18:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf9a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf40ad3b0 con 0x7fdbf40702a0 2024-03-29T08:18:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf9a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf40ad620 con 0x7fdbf4158300 2024-03-29T08:18:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf40a8800 0x7fdbf40cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf40a8800 0x7fdbf40cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50416/0 (socket says 172.21.15.46:50416) 2024-03-29T08:18:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf3400640 1 -- 172.21.15.46:0/3364202379 learned_addr learned my addr 172.21.15.46:0/3364202379 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbea000640 1 --2- 172.21.15.46:0/3364202379 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbf4158300 0x7fdbf4145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf2a00640 1 --2- 172.21.15.46:0/3364202379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbf40702a0 0x7fdbf4142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf3400640 1 -- 172.21.15.46:0/3364202379 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbf4158300 msgr2=0x7fdbf4145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf3400640 1 --2- 172.21.15.46:0/3364202379 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbf4158300 0x7fdbf4145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf3400640 1 -- 172.21.15.46:0/3364202379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbf40702a0 msgr2=0x7fdbf4142820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf3400640 1 --2- 172.21.15.46:0/3364202379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbf40702a0 0x7fdbf4142820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf3400640 1 -- 172.21.15.46:0/3364202379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc0094d0 con 0x7fdbf40a8800 2024-03-29T08:18:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.624+0000 7fdbf3400640 1 --2- 172.21.15.46:0/3364202379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf40a8800 0x7fdbf40cd4d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c760 tx=0x7fdbec00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.628+0000 7fdbeb400640 1 -- 172.21.15.46:0/3364202379 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec003b80 con 0x7fdbf40a8800 2024-03-29T08:18:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.628+0000 7fdbeb400640 1 -- 172.21.15.46:0/3364202379 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbec003ce0 con 0x7fdbf40a8800 2024-03-29T08:18:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.628+0000 7fdbeb400640 1 -- 172.21.15.46:0/3364202379 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbec00aa30 con 0x7fdbf40a8800 2024-03-29T08:18:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.628+0000 7fdbf9a00640 1 -- 172.21.15.46:0/3364202379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf4070c40 con 0x7fdbf40a8800 2024-03-29T08:18:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.628+0000 7fdbf9a00640 1 -- 172.21.15.46:0/3364202379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf40adbe0 con 0x7fdbf40a8800 2024-03-29T08:18:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.628+0000 7fdbf9a00640 1 -- 172.21.15.46:0/3364202379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf40ae100 con 0x7fdbf40a8800 2024-03-29T08:18:52.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.628+0000 7fdbeb400640 1 -- 172.21.15.46:0/3364202379 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdbec00ab90 con 0x7fdbf40a8800 2024-03-29T08:18:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.628+0000 7fdbeb400640 1 --2- 172.21.15.46:0/3364202379 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdbd00666c0 0x7fdbd0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.628+0000 7fdbeb400640 1 -- 172.21.15.46:0/3364202379 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbec083e40 con 0x7fdbf40a8800 2024-03-29T08:18:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.628+0000 7fdbf2a00640 1 --2- 172.21.15.46:0/3364202379 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdbd00666c0 0x7fdbd0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.628+0000 7fdbf2a00640 1 --2- 172.21.15.46:0/3364202379 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdbd00666c0 0x7fdbd0068b80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4007a60 tx=0x7fdbe4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:52.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.632+0000 7fdbeb400640 1 -- 172.21.15.46:0/3364202379 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdbf40ae100 con 0x7fdbf40a8800 2024-03-29T08:18:52.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.804+0000 7fdbf9a00640 1 -- 172.21.15.46:0/3364202379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdbf40ae100 con 0x7fdbf40a8800 2024-03-29T08:18:52.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.804+0000 7fdbeb400640 1 -- 172.21.15.46:0/3364202379 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdbf40ae100 con 0x7fdbf40a8800 2024-03-29T08:18:52.807 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:18:52.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.808+0000 7fdbf9a00640 1 -- 172.21.15.46:0/3364202379 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdbd00666c0 msgr2=0x7fdbd0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:52.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.808+0000 7fdbf9a00640 1 --2- 172.21.15.46:0/3364202379 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdbd00666c0 0x7fdbd0068b80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4007a60 tx=0x7fdbe4008040 comp rx=0 tx=0).stop 2024-03-29T08:18:52.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.808+0000 7fdbf9a00640 1 -- 172.21.15.46:0/3364202379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf40a8800 msgr2=0x7fdbf40cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:52.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.808+0000 7fdbf9a00640 1 --2- 172.21.15.46:0/3364202379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf40a8800 0x7fdbf40cd4d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00c760 tx=0x7fdbec00cc20 comp rx=0 tx=0).stop 2024-03-29T08:18:52.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.808+0000 7fdbf9a00640 1 -- 172.21.15.46:0/3364202379 shutdown_connections 2024-03-29T08:18:52.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.808+0000 7fdbf9a00640 1 --2- 172.21.15.46:0/3364202379 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdbd00666c0 0x7fdbd0068b80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.808+0000 7fdbf9a00640 1 --2- 172.21.15.46:0/3364202379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf40a8800 0x7fdbf40cd4d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.808+0000 7fdbf9a00640 1 --2- 172.21.15.46:0/3364202379 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbf4158300 0x7fdbf4145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.808+0000 7fdbf9a00640 1 --2- 172.21.15.46:0/3364202379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbf40702a0 0x7fdbf4142820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:52.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.808+0000 7fdbf9a00640 1 -- 172.21.15.46:0/3364202379 >> 172.21.15.46:0/3364202379 conn(0x7fdbf4066080 msgr2=0x7fdbf4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:52.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.808+0000 7fdbf9a00640 1 -- 172.21.15.46:0/3364202379 shutdown_connections 2024-03-29T08:18:52.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:52.808+0000 7fdbf9a00640 1 -- 172.21.15.46:0/3364202379 wait complete. 2024-03-29T08:18:52.828 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi073.front.sepia.ceph.com... 2024-03-29T08:18:52.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:52.828 DEBUG:teuthology.orchestra.run.smithi073:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-03-29T08:18:52.861 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:18:52.861 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:18:52.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:52.861 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T08:18:52.924 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:18:52.924 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:18:52.934 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:18:52.934 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:18:52.984 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.168+0000 7f8bb1800640 1 Processor -- start 2024-03-29T08:18:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.168+0000 7f8bb1800640 1 -- start start 2024-03-29T08:18:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.168+0000 7f8bb1800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8bac15cb00 0x7f8bac098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.168+0000 7f8bb1800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bac0cb5d0 con 0x7f8bac099810 2024-03-29T08:18:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.168+0000 7f8bb1800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bac0caca0 con 0x7f8bac0b2b30 2024-03-29T08:18:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.168+0000 7f8bb1800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac099d10 con 0x7f8bac15cb00 2024-03-29T08:18:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.168+0000 7f8ba3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8bac099810 0x7f8bac06b600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38630/0 (socket says 172.21.15.46:38630) 2024-03-29T08:18:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.168+0000 7f8ba3400640 1 -- 172.21.15.46:0/895732294 learned_addr learned my addr 172.21.15.46:0/895732294 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.168+0000 7f8babe00640 1 --2- 172.21.15.46:0/895732294 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8bac15cb00 0x7f8bac098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:53.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.168+0000 7f8bab400640 1 -- 172.21.15.46:0/895732294 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725722566 0 0) 0x7f8bac0cb5d0 con 0x7f8bac099810 2024-03-29T08:18:53.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.168+0000 7f8bab400640 1 -- 172.21.15.46:0/895732294 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b90003530 con 0x7f8bac099810 2024-03-29T08:18:53.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.168+0000 7f8bab400640 1 -- 172.21.15.46:0/895732294 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 508198881 0 0) 0x7f8bac0caca0 con 0x7f8bac0b2b30 2024-03-29T08:18:53.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.168+0000 7f8bab400640 1 -- 172.21.15.46:0/895732294 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bac0cb5d0 con 0x7f8bac0b2b30 2024-03-29T08:18:53.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.168+0000 7f8babe00640 1 -- 172.21.15.46:0/895732294 >> v1:172.21.15.73:6789/0 conn(0x7f8bac0b2b30 legacy=0x7f8bac06bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:53.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.168+0000 7f8bab400640 1 -- 172.21.15.46:0/895732294 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3439206878 0 0) 0x7f8b90003530 con 0x7f8bac099810 2024-03-29T08:18:53.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.168+0000 7f8babe00640 1 -- 172.21.15.46:0/895732294 >> v1:172.21.15.46:6789/0 conn(0x7f8bac099810 legacy=0x7f8bac06b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:53.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.168+0000 7f8babe00640 1 -- 172.21.15.46:0/895732294 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac0caf80 con 0x7f8bac15cb00 2024-03-29T08:18:53.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.172+0000 7f8babe00640 1 --2- 172.21.15.46:0/895732294 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8bac15cb00 0x7f8bac098a40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0097f0 tx=0x7f8b9c01c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e29e18ea687c0ee server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:53.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.172+0000 7f8bab400640 1 -- 172.21.15.46:0/895732294 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b94001a00 con 0x7f8bac15cb00 2024-03-29T08:18:53.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.172+0000 7f8bab400640 1 -- 172.21.15.46:0/895732294 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b9c0a7050 con 0x7f8bac15cb00 2024-03-29T08:18:53.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.172+0000 7f8bab400640 1 -- 172.21.15.46:0/895732294 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c0ab600 con 0x7f8bac15cb00 2024-03-29T08:18:53.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.172+0000 7f8bb1800640 1 -- 172.21.15.46:0/895732294 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8bac15cb00 msgr2=0x7f8bac098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:53.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.172+0000 7f8bb1800640 1 --2- 172.21.15.46:0/895732294 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8bac15cb00 0x7f8bac098a40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0097f0 tx=0x7f8b9c01c370 comp rx=0 tx=0).stop 2024-03-29T08:18:53.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.172+0000 7f8bb1800640 1 -- 172.21.15.46:0/895732294 shutdown_connections 2024-03-29T08:18:53.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.172+0000 7f8bb1800640 1 --2- 172.21.15.46:0/895732294 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8bac15cb00 0x7f8bac098a40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.172+0000 7f8bb1800640 1 -- 172.21.15.46:0/895732294 >> 172.21.15.46:0/895732294 conn(0x7f8bac0b2010 msgr2=0x7f8bac0b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:53.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.172+0000 7f8bb1800640 1 -- 172.21.15.46:0/895732294 shutdown_connections 2024-03-29T08:18:53.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.172+0000 7f8bb1800640 1 -- 172.21.15.46:0/895732294 wait complete. 2024-03-29T08:18:53.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb1800640 1 Processor -- start 2024-03-29T08:18:53.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb1800640 1 -- start start 2024-03-29T08:18:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb1800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bac099810 0x7f8bac15bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb1800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8bac0b2b30 0x7f8bac0c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bac15cb00 0x7f8bac0b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb1800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac0c68f0 con 0x7f8bac15cb00 2024-03-29T08:18:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb1800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac14ddd0 con 0x7f8bac099810 2024-03-29T08:18:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb1800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac0c8a90 con 0x7f8bac0b2b30 2024-03-29T08:18:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bac15cb00 0x7f8bac0b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8babe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bac099810 0x7f8bac15bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bac15cb00 0x7f8bac0b3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50422/0 (socket says 172.21.15.46:50422) 2024-03-29T08:18:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb0e00640 1 -- 172.21.15.46:0/2356123689 learned_addr learned my addr 172.21.15.46:0/2356123689 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:53.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8babe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bac099810 0x7f8bac15bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47334/0 (socket says 172.21.15.46:47334) 2024-03-29T08:18:53.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8ba3400640 1 --2- 172.21.15.46:0/2356123689 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8bac0b2b30 0x7f8bac0c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:53.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb0e00640 1 -- 172.21.15.46:0/2356123689 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8bac0b2b30 msgr2=0x7f8bac0c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:53.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb0e00640 1 --2- 172.21.15.46:0/2356123689 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8bac0b2b30 0x7f8bac0c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb0e00640 1 -- 172.21.15.46:0/2356123689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bac099810 msgr2=0x7f8bac15bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:53.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb0e00640 1 --2- 172.21.15.46:0/2356123689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bac099810 0x7f8bac15bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb0e00640 1 -- 172.21.15.46:0/2356123689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9c009420 con 0x7f8bac15cb00 2024-03-29T08:18:53.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb0e00640 1 --2- 172.21.15.46:0/2356123689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bac15cb00 0x7f8bac0b3120 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400c440 tx=0x7f8ba400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:53.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8ba8c00640 1 -- 172.21.15.46:0/2356123689 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba40052d0 con 0x7f8bac15cb00 2024-03-29T08:18:53.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8ba8c00640 1 -- 172.21.15.46:0/2356123689 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba4005430 con 0x7f8bac15cb00 2024-03-29T08:18:53.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8ba8c00640 1 -- 172.21.15.46:0/2356123689 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4012850 con 0x7f8bac15cb00 2024-03-29T08:18:53.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb1800640 1 -- 172.21.15.46:0/2356123689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bac0c8d70 con 0x7f8bac15cb00 2024-03-29T08:18:53.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.176+0000 7f8bb1800640 1 -- 172.21.15.46:0/2356123689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bac0c9330 con 0x7f8bac15cb00 2024-03-29T08:18:53.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.180+0000 7f8bb1800640 1 -- 172.21.15.46:0/2356123689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bac04e040 con 0x7f8bac15cb00 2024-03-29T08:18:53.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.180+0000 7f8ba8c00640 1 -- 172.21.15.46:0/2356123689 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8ba40129b0 con 0x7f8bac15cb00 2024-03-29T08:18:53.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.180+0000 7f8ba8c00640 1 --2- 172.21.15.46:0/2356123689 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8b88066690 0x7f8b88068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:53.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.180+0000 7f8ba8c00640 1 -- 172.21.15.46:0/2356123689 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ba4083b90 con 0x7f8bac15cb00 2024-03-29T08:18:53.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.180+0000 7f8babe00640 1 --2- 172.21.15.46:0/2356123689 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8b88066690 0x7f8b88068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:53.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.180+0000 7f8babe00640 1 --2- 172.21.15.46:0/2356123689 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8b88066690 0x7f8b88068b50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0ad000 tx=0x7f8b9c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:53.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.184+0000 7f8ba8c00640 1 -- 172.21.15.46:0/2356123689 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8ba4050fa0 con 0x7f8bac15cb00 2024-03-29T08:18:53.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.348+0000 7f8bb1800640 1 -- 172.21.15.46:0/2356123689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bac06bf70 con 0x7f8bac15cb00 2024-03-29T08:18:53.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.352+0000 7f8ba8c00640 1 -- 172.21.15.46:0/2356123689 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f8ba40545c0 con 0x7f8bac15cb00 2024-03-29T08:18:53.352 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:53.352 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":22,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:14.314124+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:53.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.352+0000 7f8bb1800640 1 -- 172.21.15.46:0/2356123689 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8b88066690 msgr2=0x7f8b88068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:53.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.352+0000 7f8bb1800640 1 --2- 172.21.15.46:0/2356123689 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8b88066690 0x7f8b88068b50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c0ad000 tx=0x7f8b9c008040 comp rx=0 tx=0).stop 2024-03-29T08:18:53.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.352+0000 7f8bb1800640 1 -- 172.21.15.46:0/2356123689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bac15cb00 msgr2=0x7f8bac0b3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:53.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.352+0000 7f8bb1800640 1 --2- 172.21.15.46:0/2356123689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bac15cb00 0x7f8bac0b3120 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400c440 tx=0x7f8ba400c900 comp rx=0 tx=0).stop 2024-03-29T08:18:53.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.352+0000 7f8bb1800640 1 -- 172.21.15.46:0/2356123689 shutdown_connections 2024-03-29T08:18:53.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.352+0000 7f8bb1800640 1 --2- 172.21.15.46:0/2356123689 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8b88066690 0x7f8b88068b50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.352+0000 7f8bb1800640 1 --2- 172.21.15.46:0/2356123689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8bac15cb00 0x7f8bac0b3120 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.352+0000 7f8bb1800640 1 --2- 172.21.15.46:0/2356123689 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8bac0b2b30 0x7f8bac0c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.352+0000 7f8bb1800640 1 --2- 172.21.15.46:0/2356123689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8bac099810 0x7f8bac15bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.352+0000 7f8bb1800640 1 -- 172.21.15.46:0/2356123689 >> 172.21.15.46:0/2356123689 conn(0x7f8bac0b2010 msgr2=0x7f8bac0ced50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:53.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.352+0000 7f8bb1800640 1 -- 172.21.15.46:0/2356123689 shutdown_connections 2024-03-29T08:18:53.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.352+0000 7f8bb1800640 1 -- 172.21.15.46:0/2356123689 wait complete. 2024-03-29T08:18:53.374 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:18:11.177842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:18:12.308252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:18:53.374 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:18:53.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.516+0000 7f4cf4c00640 1 Processor -- start 2024-03-29T08:18:53.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.516+0000 7f4cf4c00640 1 -- start start 2024-03-29T08:18:53.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.516+0000 7f4cf4c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cf00cbea0 0x7f4cf00b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:53.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.516+0000 7f4cf4c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4cf015efe0 0x7f4cf00b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.516+0000 7f4cf4c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf00b20b0 con 0x7f4cf0143eb0 2024-03-29T08:18:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.516+0000 7f4cf4c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf00b2360 con 0x7f4cf015efe0 2024-03-29T08:18:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.516+0000 7f4cf4c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf015e500 con 0x7f4cf00cbea0 2024-03-29T08:18:53.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.516+0000 7f4ce5400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cf00cbea0 0x7f4cf00b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.516+0000 7f4cee800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4cf015efe0 0x7f4cf00b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.516+0000 7f4ce5400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cf00cbea0 0x7f4cf00b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:40218/0 (socket says 172.21.15.46:40218) 2024-03-29T08:18:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.516+0000 7f4ce5400640 1 -- 172.21.15.46:0/3180120170 learned_addr learned my addr 172.21.15.46:0/3180120170 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.520+0000 7f4ce5400640 1 -- 172.21.15.46:0/3180120170 >> v2:172.21.15.73:3300/0 conn(0x7f4cf015efe0 msgr2=0x7f4cf00b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.520+0000 7f4ce5400640 1 --2- 172.21.15.46:0/3180120170 >> v2:172.21.15.73:3300/0 conn(0x7f4cf015efe0 0x7f4cf00b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.520+0000 7f4ce5400640 1 -- 172.21.15.46:0/3180120170 >> v1:172.21.15.73:6789/0 conn(0x7f4cf0143eb0 legacy=0x7f4cf00cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.520+0000 7f4ce5400640 1 -- 172.21.15.46:0/3180120170 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf015e7e0 con 0x7f4cf00cbea0 2024-03-29T08:18:53.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.520+0000 7f4cee800640 1 --2- 172.21.15.46:0/3180120170 >> v2:172.21.15.73:3300/0 conn(0x7f4cf015efe0 0x7f4cf00b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.520+0000 7f4ce5400640 1 --2- 172.21.15.46:0/3180120170 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cf00cbea0 0x7f4cf00b1630 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8009870 tx=0x7f4cd801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4aabcaa97e88f4b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.520+0000 7f4ced400640 1 -- 172.21.15.46:0/3180120170 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd80aa430 con 0x7f4cf00cbea0 2024-03-29T08:18:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.520+0000 7f4ced400640 1 -- 172.21.15.46:0/3180120170 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cd80aa590 con 0x7f4cf00cbea0 2024-03-29T08:18:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.520+0000 7f4ced400640 1 -- 172.21.15.46:0/3180120170 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd80aa920 con 0x7f4cf00cbea0 2024-03-29T08:18:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.520+0000 7f4cf4c00640 1 -- 172.21.15.46:0/3180120170 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cf00cbea0 msgr2=0x7f4cf00b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.520+0000 7f4cf4c00640 1 --2- 172.21.15.46:0/3180120170 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cf00cbea0 0x7f4cf00b1630 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4cd8009870 tx=0x7f4cd801c530 comp rx=0 tx=0).stop 2024-03-29T08:18:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.520+0000 7f4cf4c00640 1 -- 172.21.15.46:0/3180120170 shutdown_connections 2024-03-29T08:18:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.520+0000 7f4cf4c00640 1 --2- 172.21.15.46:0/3180120170 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cf00cbea0 0x7f4cf00b1630 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.520+0000 7f4cf4c00640 1 --2- 172.21.15.46:0/3180120170 >> v2:172.21.15.73:3300/0 conn(0x7f4cf015efe0 0x7f4cf00b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.520+0000 7f4cf4c00640 1 -- 172.21.15.46:0/3180120170 >> 172.21.15.46:0/3180120170 conn(0x7f4cf006a390 msgr2=0x7f4cf00cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.520+0000 7f4cf4c00640 1 -- 172.21.15.46:0/3180120170 shutdown_connections 2024-03-29T08:18:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cf4c00640 1 -- 172.21.15.46:0/3180120170 wait complete. 2024-03-29T08:18:53.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cf4c00640 1 Processor -- start 2024-03-29T08:18:53.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cf4c00640 1 -- start start 2024-03-29T08:18:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cf4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cf00cbea0 0x7f4cf013a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cf4c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cf0143eb0 0x7f4cf01595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cf4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cf015efe0 0x7f4cf015cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cf4c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf013ac80 con 0x7f4cf00cbea0 2024-03-29T08:18:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cf4c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf013def0 con 0x7f4cf015efe0 2024-03-29T08:18:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cf4c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf0139980 con 0x7f4cf0143eb0 2024-03-29T08:18:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cf00cbea0 0x7f4cf013a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cf00cbea0 0x7f4cf013a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50434/0 (socket says 172.21.15.46:50434) 2024-03-29T08:18:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cede00640 1 -- 172.21.15.46:0/869640413 learned_addr learned my addr 172.21.15.46:0/869640413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:53.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cee800640 1 --2- 172.21.15.46:0/869640413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cf015efe0 0x7f4cf015cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4ce5400640 1 --2- 172.21.15.46:0/869640413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cf0143eb0 0x7f4cf01595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cede00640 1 -- 172.21.15.46:0/869640413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cf0143eb0 msgr2=0x7f4cf01595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cede00640 1 --2- 172.21.15.46:0/869640413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cf0143eb0 0x7f4cf01595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cede00640 1 -- 172.21.15.46:0/869640413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cf015efe0 msgr2=0x7f4cf015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cede00640 1 --2- 172.21.15.46:0/869640413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cf015efe0 0x7f4cf015cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cede00640 1 -- 172.21.15.46:0/869640413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce8009a70 con 0x7f4cf00cbea0 2024-03-29T08:18:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cede00640 1 --2- 172.21.15.46:0/869640413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cf00cbea0 0x7f4cf013a740 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000c3c0 tx=0x7f4ce000c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4ce6a00640 1 -- 172.21.15.46:0/869640413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00052d0 con 0x7f4cf00cbea0 2024-03-29T08:18:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4ce6a00640 1 -- 172.21.15.46:0/869640413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce0005430 con 0x7f4cf00cbea0 2024-03-29T08:18:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4cf4c00640 1 -- 172.21.15.46:0/869640413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd80094d0 con 0x7f4cf00cbea0 2024-03-29T08:18:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.524+0000 7f4ce6a00640 1 -- 172.21.15.46:0/869640413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0003840 con 0x7f4cf00cbea0 2024-03-29T08:18:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.528+0000 7f4cf4c00640 1 -- 172.21.15.46:0/869640413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf0139f40 con 0x7f4cf00cbea0 2024-03-29T08:18:53.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.528+0000 7f4cf4c00640 1 -- 172.21.15.46:0/869640413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf0071a70 con 0x7f4cf00cbea0 2024-03-29T08:18:53.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.528+0000 7f4ce6a00640 1 -- 172.21.15.46:0/869640413 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4ce00039a0 con 0x7f4cf00cbea0 2024-03-29T08:18:53.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.528+0000 7f4ce6a00640 1 --2- 172.21.15.46:0/869640413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ccc066690 0x7f4ccc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:53.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.528+0000 7f4ce6a00640 1 -- 172.21.15.46:0/869640413 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ce0083720 con 0x7f4cf00cbea0 2024-03-29T08:18:53.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.528+0000 7f4ce5400640 1 --2- 172.21.15.46:0/869640413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ccc066690 0x7f4ccc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.528+0000 7f4ce5400640 1 --2- 172.21.15.46:0/869640413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ccc066690 0x7f4ccc068b50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80ae000 tx=0x7f4cd80b5040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.532+0000 7f4ce6a00640 1 -- 172.21.15.46:0/869640413 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4ce0050aa0 con 0x7f4cf00cbea0 2024-03-29T08:18:53.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.692+0000 7f4cf4c00640 1 -- 172.21.15.46:0/869640413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4cf004e120 con 0x7f4cf00cbea0 2024-03-29T08:18:53.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.692+0000 7f4ce6a00640 1 -- 172.21.15.46:0/869640413 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4ce00540c0 con 0x7f4cf00cbea0 2024-03-29T08:18:53.695 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:18:53.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.696+0000 7f4cf4c00640 1 -- 172.21.15.46:0/869640413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ccc066690 msgr2=0x7f4ccc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:53.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.696+0000 7f4cf4c00640 1 --2- 172.21.15.46:0/869640413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ccc066690 0x7f4ccc068b50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80ae000 tx=0x7f4cd80b5040 comp rx=0 tx=0).stop 2024-03-29T08:18:53.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.696+0000 7f4cf4c00640 1 -- 172.21.15.46:0/869640413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cf00cbea0 msgr2=0x7f4cf013a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:53.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.696+0000 7f4cf4c00640 1 --2- 172.21.15.46:0/869640413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cf00cbea0 0x7f4cf013a740 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000c3c0 tx=0x7f4ce000c7c0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.696+0000 7f4cf4c00640 1 -- 172.21.15.46:0/869640413 shutdown_connections 2024-03-29T08:18:53.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.696+0000 7f4cf4c00640 1 --2- 172.21.15.46:0/869640413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ccc066690 0x7f4ccc068b50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.696+0000 7f4cf4c00640 1 --2- 172.21.15.46:0/869640413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cf015efe0 0x7f4cf015cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.696+0000 7f4cf4c00640 1 --2- 172.21.15.46:0/869640413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4cf0143eb0 0x7f4cf01595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.696+0000 7f4cf4c00640 1 --2- 172.21.15.46:0/869640413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cf00cbea0 0x7f4cf013a740 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.696+0000 7f4cf4c00640 1 -- 172.21.15.46:0/869640413 >> 172.21.15.46:0/869640413 conn(0x7f4cf006a390 msgr2=0x7f4cf00cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:53.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.696+0000 7f4cf4c00640 1 -- 172.21.15.46:0/869640413 shutdown_connections 2024-03-29T08:18:53.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.696+0000 7f4cf4c00640 1 -- 172.21.15.46:0/869640413 wait complete. 2024-03-29T08:18:53.717 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:18:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.856+0000 7f196fe00640 1 Processor -- start 2024-03-29T08:18:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.856+0000 7f196fe00640 1 -- start start 2024-03-29T08:18:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.860+0000 7f196fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f19680cbea0 0x7f19680b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.860+0000 7f196fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f196815efe0 0x7f19680b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.860+0000 7f196fe00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19680b20b0 con 0x7f1968143eb0 2024-03-29T08:18:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.860+0000 7f196fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19680b2360 con 0x7f196815efe0 2024-03-29T08:18:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.860+0000 7f196fe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196815e500 con 0x7f19680cbea0 2024-03-29T08:18:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.860+0000 7f196de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f196815efe0 0x7f19680b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.860+0000 7f196de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f196815efe0 0x7f19680b1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50442/0 (socket says 172.21.15.46:50442) 2024-03-29T08:18:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.860+0000 7f196de00640 1 -- 172.21.15.46:0/837799122 learned_addr learned my addr 172.21.15.46:0/837799122 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.860+0000 7f196ca00640 1 --2- 172.21.15.46:0/837799122 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f19680cbea0 0x7f19680b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.860+0000 7f196de00640 1 -- 172.21.15.46:0/837799122 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f19680cbea0 msgr2=0x7f19680b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.860+0000 7f196de00640 1 --2- 172.21.15.46:0/837799122 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f19680cbea0 0x7f19680b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.860+0000 7f196de00640 1 -- 172.21.15.46:0/837799122 >> v1:172.21.15.73:6789/0 conn(0x7f1968143eb0 legacy=0x7f19680cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.860+0000 7f196de00640 1 -- 172.21.15.46:0/837799122 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196815e7e0 con 0x7f196815efe0 2024-03-29T08:18:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.864+0000 7f196de00640 1 --2- 172.21.15.46:0/837799122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196815efe0 0x7f19680b1b70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1960009e50 tx=0x7f196001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e995a7989a093d51 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.864+0000 7f1967e00640 1 -- 172.21.15.46:0/837799122 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19600a5870 con 0x7f196815efe0 2024-03-29T08:18:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.864+0000 7f1967e00640 1 -- 172.21.15.46:0/837799122 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19600a59d0 con 0x7f196815efe0 2024-03-29T08:18:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.864+0000 7f1967e00640 1 -- 172.21.15.46:0/837799122 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19600a5d60 con 0x7f196815efe0 2024-03-29T08:18:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.864+0000 7f196fe00640 1 -- 172.21.15.46:0/837799122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196815efe0 msgr2=0x7f19680b1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.864+0000 7f196fe00640 1 --2- 172.21.15.46:0/837799122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196815efe0 0x7f19680b1b70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1960009e50 tx=0x7f196001ca00 comp rx=0 tx=0).stop 2024-03-29T08:18:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.864+0000 7f196fe00640 1 -- 172.21.15.46:0/837799122 shutdown_connections 2024-03-29T08:18:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.864+0000 7f196fe00640 1 --2- 172.21.15.46:0/837799122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f196815efe0 0x7f19680b1b70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.864+0000 7f196fe00640 1 --2- 172.21.15.46:0/837799122 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f19680cbea0 0x7f19680b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.864+0000 7f196fe00640 1 -- 172.21.15.46:0/837799122 >> 172.21.15.46:0/837799122 conn(0x7f196806a390 msgr2=0x7f19680cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:53.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.864+0000 7f196fe00640 1 -- 172.21.15.46:0/837799122 shutdown_connections 2024-03-29T08:18:53.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.864+0000 7f196fe00640 1 -- 172.21.15.46:0/837799122 wait complete. 2024-03-29T08:18:53.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.864+0000 7f196fe00640 1 Processor -- start 2024-03-29T08:18:53.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.864+0000 7f196fe00640 1 -- start start 2024-03-29T08:18:53.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.864+0000 7f196fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19680cbea0 0x7f196813a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:53.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.864+0000 7f196fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1968143eb0 0x7f196813e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:53.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f196815efe0 0x7f1968140700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:53.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196813ecc0 con 0x7f19680cbea0 2024-03-29T08:18:53.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196813ef00 con 0x7f1968143eb0 2024-03-29T08:18:53.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196fe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196813f940 con 0x7f196815efe0 2024-03-29T08:18:53.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19680cbea0 0x7f196813a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19680cbea0 0x7f196813a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50446/0 (socket says 172.21.15.46:50446) 2024-03-29T08:18:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196d400640 1 -- 172.21.15.46:0/1627622157 learned_addr learned my addr 172.21.15.46:0/1627622157 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196ca00640 1 --2- 172.21.15.46:0/1627622157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1968143eb0 0x7f196813e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196de00640 1 --2- 172.21.15.46:0/1627622157 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f196815efe0 0x7f1968140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196d400640 1 -- 172.21.15.46:0/1627622157 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f196815efe0 msgr2=0x7f1968140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196d400640 1 --2- 172.21.15.46:0/1627622157 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f196815efe0 0x7f1968140700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196d400640 1 -- 172.21.15.46:0/1627622157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1968143eb0 msgr2=0x7f196813e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196d400640 1 --2- 172.21.15.46:0/1627622157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1968143eb0 0x7f196813e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196d400640 1 -- 172.21.15.46:0/1627622157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1960009b30 con 0x7f19680cbea0 2024-03-29T08:18:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196d400640 1 --2- 172.21.15.46:0/1627622157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19680cbea0 0x7f196813a7f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c500 tx=0x7f195c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f1965600640 1 -- 172.21.15.46:0/1627622157 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c005220 con 0x7f19680cbea0 2024-03-29T08:18:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f1965600640 1 -- 172.21.15.46:0/1627622157 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c005380 con 0x7f19680cbea0 2024-03-29T08:18:53.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f1965600640 1 -- 172.21.15.46:0/1627622157 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c007270 con 0x7f19680cbea0 2024-03-29T08:18:53.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196fe00640 1 -- 172.21.15.46:0/1627622157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196813fc20 con 0x7f19680cbea0 2024-03-29T08:18:53.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196fe00640 1 -- 172.21.15.46:0/1627622157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19681401e0 con 0x7f19680cbea0 2024-03-29T08:18:53.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f196fe00640 1 -- 172.21.15.46:0/1627622157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1968071a70 con 0x7f19680cbea0 2024-03-29T08:18:53.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.868+0000 7f1965600640 1 -- 172.21.15.46:0/1627622157 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f195c0073d0 con 0x7f19680cbea0 2024-03-29T08:18:53.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.872+0000 7f1965600640 1 --2- 172.21.15.46:0/1627622157 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f19480666c0 0x7f1948068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:53.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.872+0000 7f1965600640 1 -- 172.21.15.46:0/1627622157 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f195c083980 con 0x7f19680cbea0 2024-03-29T08:18:53.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.872+0000 7f196ca00640 1 --2- 172.21.15.46:0/1627622157 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f19480666c0 0x7f1948068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:53.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.872+0000 7f196ca00640 1 --2- 172.21.15.46:0/1627622157 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f19480666c0 0x7f1948068b80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1968159bf0 tx=0x7f1954009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:53.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:53.876+0000 7f1965600640 1 -- 172.21.15.46:0/1627622157 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f195c050d10 con 0x7f19680cbea0 2024-03-29T08:18:54.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.036+0000 7f196fe00640 1 -- 172.21.15.46:0/1627622157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f19680cadd0 con 0x7f19680cbea0 2024-03-29T08:18:54.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.036+0000 7f1965600640 1 -- 172.21.15.46:0/1627622157 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f195c054330 con 0x7f19680cbea0 2024-03-29T08:18:54.036 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:18:54.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.036+0000 7f196fe00640 1 -- 172.21.15.46:0/1627622157 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f19480666c0 msgr2=0x7f1948068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:54.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.036+0000 7f196fe00640 1 --2- 172.21.15.46:0/1627622157 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f19480666c0 0x7f1948068b80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1968159bf0 tx=0x7f1954009190 comp rx=0 tx=0).stop 2024-03-29T08:18:54.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.036+0000 7f196fe00640 1 -- 172.21.15.46:0/1627622157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19680cbea0 msgr2=0x7f196813a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:54.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.036+0000 7f196fe00640 1 --2- 172.21.15.46:0/1627622157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19680cbea0 0x7f196813a7f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c500 tx=0x7f195c00c9c0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.036+0000 7f196fe00640 1 -- 172.21.15.46:0/1627622157 shutdown_connections 2024-03-29T08:18:54.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.036+0000 7f196fe00640 1 --2- 172.21.15.46:0/1627622157 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f19480666c0 0x7f1948068b80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.036+0000 7f196fe00640 1 --2- 172.21.15.46:0/1627622157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1968143eb0 0x7f196813e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.036+0000 7f196fe00640 1 --2- 172.21.15.46:0/1627622157 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f196815efe0 0x7f1968140700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.036+0000 7f196fe00640 1 --2- 172.21.15.46:0/1627622157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f19680cbea0 0x7f196813a7f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.036+0000 7f196fe00640 1 -- 172.21.15.46:0/1627622157 >> 172.21.15.46:0/1627622157 conn(0x7f196806a390 msgr2=0x7f19680cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:54.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.036+0000 7f196fe00640 1 -- 172.21.15.46:0/1627622157 shutdown_connections 2024-03-29T08:18:54.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.036+0000 7f196fe00640 1 -- 172.21.15.46:0/1627622157 wait complete. 2024-03-29T08:18:54.058 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:18:54.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.200+0000 7f1cb1400640 1 Processor -- start 2024-03-29T08:18:54.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.200+0000 7f1cb1400640 1 -- start start 2024-03-29T08:18:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.200+0000 7f1cb1400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1cac15df90 0x7f1cac0ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.200+0000 7f1cb1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1cac0cbea0 0x7f1cac0ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.200+0000 7f1cb1400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cac15d260 con 0x7f1cac142a50 2024-03-29T08:18:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.200+0000 7f1cb1400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cac0aed00 con 0x7f1cac0cbea0 2024-03-29T08:18:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.200+0000 7f1cb1400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac0aefd0 con 0x7f1cac15df90 2024-03-29T08:18:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.200+0000 7f1cab200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1cac0cbea0 0x7f1cac0ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.200+0000 7f1cab200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1cac0cbea0 0x7f1cac0ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50460/0 (socket says 172.21.15.46:50460) 2024-03-29T08:18:54.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.200+0000 7f1cab200640 1 -- 172.21.15.46:0/3850792001 learned_addr learned my addr 172.21.15.46:0/3850792001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:54.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.204+0000 7f1ca1e00640 1 --2- 172.21.15.46:0/3850792001 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1cac15df90 0x7f1cac0ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.204+0000 7f1cab200640 1 -- 172.21.15.46:0/3850792001 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1cac15df90 msgr2=0x7f1cac0ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:54.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.204+0000 7f1cab200640 1 --2- 172.21.15.46:0/3850792001 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1cac15df90 0x7f1cac0ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.204+0000 7f1cab200640 1 -- 172.21.15.46:0/3850792001 >> v1:172.21.15.73:6789/0 conn(0x7f1cac142a50 legacy=0x7f1cac0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:54.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.204+0000 7f1cab200640 1 -- 172.21.15.46:0/3850792001 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac15d4b0 con 0x7f1cac0cbea0 2024-03-29T08:18:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.204+0000 7f1cab200640 1 --2- 172.21.15.46:0/3850792001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cac0cbea0 0x7f1cac0ae7c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009e50 tx=0x7f1ca401eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbbd4f62c46745a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.204+0000 7f1ca9e00640 1 -- 172.21.15.46:0/3850792001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40a77f0 con 0x7f1cac0cbea0 2024-03-29T08:18:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.204+0000 7f1ca9e00640 1 -- 172.21.15.46:0/3850792001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca40a7950 con 0x7f1cac0cbea0 2024-03-29T08:18:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.204+0000 7f1ca9e00640 1 -- 172.21.15.46:0/3850792001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40a7ce0 con 0x7f1cac0cbea0 2024-03-29T08:18:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.204+0000 7f1cb1400640 1 -- 172.21.15.46:0/3850792001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cac0cbea0 msgr2=0x7f1cac0ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.204+0000 7f1cb1400640 1 --2- 172.21.15.46:0/3850792001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cac0cbea0 0x7f1cac0ae7c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4009e50 tx=0x7f1ca401eb70 comp rx=0 tx=0).stop 2024-03-29T08:18:54.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.204+0000 7f1cb1400640 1 -- 172.21.15.46:0/3850792001 shutdown_connections 2024-03-29T08:18:54.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.204+0000 7f1cb1400640 1 --2- 172.21.15.46:0/3850792001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cac0cbea0 0x7f1cac0ae7c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.204+0000 7f1cb1400640 1 --2- 172.21.15.46:0/3850792001 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1cac15df90 0x7f1cac0ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.204+0000 7f1cb1400640 1 -- 172.21.15.46:0/3850792001 >> 172.21.15.46:0/3850792001 conn(0x7f1cac06a390 msgr2=0x7f1cac0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:54.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1cb1400640 1 -- 172.21.15.46:0/3850792001 shutdown_connections 2024-03-29T08:18:54.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1cb1400640 1 -- 172.21.15.46:0/3850792001 wait complete. 2024-03-29T08:18:54.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1cb1400640 1 Processor -- start 2024-03-29T08:18:54.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1cb1400640 1 -- start start 2024-03-29T08:18:54.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1cb1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cac0cbea0 0x7f1cac0ac870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1cb1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cac142a50 0x7f1cac0afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1cb1400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1cac15df90 0x7f1cac0b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1cb1400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac13f130 con 0x7f1cac142a50 2024-03-29T08:18:54.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1cb1400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac13ab10 con 0x7f1cac0cbea0 2024-03-29T08:18:54.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1cb1400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac13adb0 con 0x7f1cac15df90 2024-03-29T08:18:54.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1ca1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cac142a50 0x7f1cac0afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1ca1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cac142a50 0x7f1cac0afbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50470/0 (socket says 172.21.15.46:50470) 2024-03-29T08:18:54.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1ca1e00640 1 -- 172.21.15.46:0/713876097 learned_addr learned my addr 172.21.15.46:0/713876097 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:54.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1cab200640 1 --2- 172.21.15.46:0/713876097 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1cac15df90 0x7f1cac0b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1caa800640 1 --2- 172.21.15.46:0/713876097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cac0cbea0 0x7f1cac0ac870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1ca1e00640 1 -- 172.21.15.46:0/713876097 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1cac15df90 msgr2=0x7f1cac0b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:54.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1ca1e00640 1 --2- 172.21.15.46:0/713876097 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1cac15df90 0x7f1cac0b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1ca1e00640 1 -- 172.21.15.46:0/713876097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cac0cbea0 msgr2=0x7f1cac0ac870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:54.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1ca1e00640 1 --2- 172.21.15.46:0/713876097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cac0cbea0 0x7f1cac0ac870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1ca1e00640 1 -- 172.21.15.46:0/713876097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca4009b30 con 0x7f1cac142a50 2024-03-29T08:18:54.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1ca1e00640 1 --2- 172.21.15.46:0/713876097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cac142a50 0x7f1cac0afbd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400d800 tx=0x7f1c9400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1ca3400640 1 -- 172.21.15.46:0/713876097 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400b6c0 con 0x7f1cac142a50 2024-03-29T08:18:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1ca3400640 1 -- 172.21.15.46:0/713876097 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9400b820 con 0x7f1cac142a50 2024-03-29T08:18:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1ca3400640 1 -- 172.21.15.46:0/713876097 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400bb90 con 0x7f1cac142a50 2024-03-29T08:18:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.208+0000 7f1cb1400640 1 -- 172.21.15.46:0/713876097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cac13b090 con 0x7f1cac142a50 2024-03-29T08:18:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.212+0000 7f1cb1400640 1 -- 172.21.15.46:0/713876097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cac13b650 con 0x7f1cac142a50 2024-03-29T08:18:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.212+0000 7f1cb1400640 1 -- 172.21.15.46:0/713876097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cac13b810 con 0x7f1cac142a50 2024-03-29T08:18:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.212+0000 7f1ca3400640 1 -- 172.21.15.46:0/713876097 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1c9400bcf0 con 0x7f1cac142a50 2024-03-29T08:18:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.212+0000 7f1ca3400640 1 --2- 172.21.15.46:0/713876097 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c88066690 0x7f1c88068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.212+0000 7f1ca3400640 1 -- 172.21.15.46:0/713876097 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c94084650 con 0x7f1cac142a50 2024-03-29T08:18:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.212+0000 7f1caa800640 1 --2- 172.21.15.46:0/713876097 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c88066690 0x7f1c88068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.216+0000 7f1caa800640 1 --2- 172.21.15.46:0/713876097 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c88066690 0x7f1c88068b50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0671a0 tx=0x7f1c9c007e40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:54.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.216+0000 7f1ca3400640 1 -- 172.21.15.46:0/713876097 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1c940519d0 con 0x7f1cac142a50 2024-03-29T08:18:54.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.380+0000 7f1cb1400640 1 -- 172.21.15.46:0/713876097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1cac071a70 con 0x7f1cac142a50 2024-03-29T08:18:54.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.380+0000 7f1ca3400640 1 -- 172.21.15.46:0/713876097 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1c94054ff0 con 0x7f1cac142a50 2024-03-29T08:18:54.381 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:18:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.380+0000 7f1cb1400640 1 -- 172.21.15.46:0/713876097 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c88066690 msgr2=0x7f1c88068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.380+0000 7f1cb1400640 1 --2- 172.21.15.46:0/713876097 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c88066690 0x7f1c88068b50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0671a0 tx=0x7f1c9c007e40 comp rx=0 tx=0).stop 2024-03-29T08:18:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.380+0000 7f1cb1400640 1 -- 172.21.15.46:0/713876097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cac142a50 msgr2=0x7f1cac0afbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.380+0000 7f1cb1400640 1 --2- 172.21.15.46:0/713876097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cac142a50 0x7f1cac0afbd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400d800 tx=0x7f1c9400dcc0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.380+0000 7f1cb1400640 1 -- 172.21.15.46:0/713876097 shutdown_connections 2024-03-29T08:18:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.380+0000 7f1cb1400640 1 --2- 172.21.15.46:0/713876097 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c88066690 0x7f1c88068b50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.380+0000 7f1cb1400640 1 --2- 172.21.15.46:0/713876097 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1cac15df90 0x7f1cac0b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.380+0000 7f1cb1400640 1 --2- 172.21.15.46:0/713876097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cac142a50 0x7f1cac0afbd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.380+0000 7f1cb1400640 1 --2- 172.21.15.46:0/713876097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1cac0cbea0 0x7f1cac0ac870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.380+0000 7f1cb1400640 1 -- 172.21.15.46:0/713876097 >> 172.21.15.46:0/713876097 conn(0x7f1cac06a390 msgr2=0x7f1cac0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.380+0000 7f1cb1400640 1 -- 172.21.15.46:0/713876097 shutdown_connections 2024-03-29T08:18:54.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.380+0000 7f1cb1400640 1 -- 172.21.15.46:0/713876097 wait complete. 2024-03-29T08:18:54.403 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:18:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.544+0000 7fda0f200640 1 Processor -- start 2024-03-29T08:18:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.544+0000 7fda0f200640 1 -- start start 2024-03-29T08:18:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.544+0000 7fda0f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda08159970 0x7fda080cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.544+0000 7fda0f200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda080ce2d0 0x7fda080c3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.544+0000 7fda0f200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda080c34c0 con 0x7fda080cbf60 2024-03-29T08:18:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.544+0000 7fda0f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda080cc460 con 0x7fda08159970 2024-03-29T08:18:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.544+0000 7fda0f200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda080cc740 con 0x7fda080ce2d0 2024-03-29T08:18:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.544+0000 7fda07e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda08159970 0x7fda080cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.544+0000 7fda07e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda08159970 0x7fda080cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50480/0 (socket says 172.21.15.46:50480) 2024-03-29T08:18:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.544+0000 7fda07e00640 1 -- 172.21.15.46:0/2335094035 learned_addr learned my addr 172.21.15.46:0/2335094035 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.544+0000 7fda0d200640 1 --2- 172.21.15.46:0/2335094035 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda080ce2d0 0x7fda080c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.544+0000 7fda07e00640 1 -- 172.21.15.46:0/2335094035 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda080ce2d0 msgr2=0x7fda080c3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.544+0000 7fda07e00640 1 --2- 172.21.15.46:0/2335094035 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda080ce2d0 0x7fda080c3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.544+0000 7fda07e00640 1 -- 172.21.15.46:0/2335094035 >> v1:172.21.15.73:6789/0 conn(0x7fda080cbf60 legacy=0x7fda080cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.544+0000 7fda07e00640 1 -- 172.21.15.46:0/2335094035 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda080cca20 con 0x7fda08159970 2024-03-29T08:18:54.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.548+0000 7fda07e00640 1 --2- 172.21.15.46:0/2335094035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda08159970 0x7fda080cd320 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009870 tx=0x7fd9f801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f741bbdb9c256fb5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.548+0000 7fda07400640 1 -- 172.21.15.46:0/2335094035 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80aa430 con 0x7fda08159970 2024-03-29T08:18:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.548+0000 7fda07400640 1 -- 172.21.15.46:0/2335094035 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f80aa590 con 0x7fda08159970 2024-03-29T08:18:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.548+0000 7fda07400640 1 -- 172.21.15.46:0/2335094035 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f80aa920 con 0x7fda08159970 2024-03-29T08:18:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.548+0000 7fda0f200640 1 -- 172.21.15.46:0/2335094035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda08159970 msgr2=0x7fda080cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.548+0000 7fda0f200640 1 --2- 172.21.15.46:0/2335094035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda08159970 0x7fda080cd320 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009870 tx=0x7fd9f801c530 comp rx=0 tx=0).stop 2024-03-29T08:18:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.548+0000 7fda0f200640 1 -- 172.21.15.46:0/2335094035 shutdown_connections 2024-03-29T08:18:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.548+0000 7fda0f200640 1 --2- 172.21.15.46:0/2335094035 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda080ce2d0 0x7fda080c3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.548+0000 7fda0f200640 1 --2- 172.21.15.46:0/2335094035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda08159970 0x7fda080cd320 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.548+0000 7fda0f200640 1 -- 172.21.15.46:0/2335094035 >> 172.21.15.46:0/2335094035 conn(0x7fda08098a40 msgr2=0x7fda080cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.548+0000 7fda0f200640 1 -- 172.21.15.46:0/2335094035 shutdown_connections 2024-03-29T08:18:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.548+0000 7fda0f200640 1 -- 172.21.15.46:0/2335094035 wait complete. 2024-03-29T08:18:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fda0f200640 1 Processor -- start 2024-03-29T08:18:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fda0f200640 1 -- start start 2024-03-29T08:18:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fda0f200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda080cbf60 0x7fda08156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fda0f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda080ce2d0 0x7fda0814d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fda0f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda08159970 0x7fda0815c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fda0f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0815ce70 con 0x7fda080ce2d0 2024-03-29T08:18:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fda0f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda08157490 con 0x7fda08159970 2024-03-29T08:18:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fda0f200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda08152f40 con 0x7fda080cbf60 2024-03-29T08:18:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fd9ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda080ce2d0 0x7fda0814d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fd9ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda080ce2d0 0x7fda0814d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50490/0 (socket says 172.21.15.46:50490) 2024-03-29T08:18:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fd9ffe00640 1 -- 172.21.15.46:0/3017341732 learned_addr learned my addr 172.21.15.46:0/3017341732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fda0d200640 1 --2- 172.21.15.46:0/3017341732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda08159970 0x7fda0815c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fda07e00640 1 --2- 172.21.15.46:0/3017341732 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda080cbf60 0x7fda08156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fd9ffe00640 1 -- 172.21.15.46:0/3017341732 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda080cbf60 msgr2=0x7fda08156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fd9ffe00640 1 --2- 172.21.15.46:0/3017341732 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda080cbf60 0x7fda08156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fd9ffe00640 1 -- 172.21.15.46:0/3017341732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda08159970 msgr2=0x7fda0815c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fd9ffe00640 1 --2- 172.21.15.46:0/3017341732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda08159970 0x7fda0815c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fd9ffe00640 1 -- 172.21.15.46:0/3017341732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f80094d0 con 0x7fda080ce2d0 2024-03-29T08:18:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fda0d200640 1 --2- 172.21.15.46:0/3017341732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda08159970 0x7fda0815c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:54.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fd9ffe00640 1 --2- 172.21.15.46:0/3017341732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda080ce2d0 0x7fda0814d670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000c340 tx=0x7fd9f000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:54.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fda04c00640 1 -- 172.21.15.46:0/3017341732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f00052d0 con 0x7fda080ce2d0 2024-03-29T08:18:54.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fda04c00640 1 -- 172.21.15.46:0/3017341732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f0005430 con 0x7fda080ce2d0 2024-03-29T08:18:54.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fda04c00640 1 -- 172.21.15.46:0/3017341732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0007260 con 0x7fda080ce2d0 2024-03-29T08:18:54.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fda0f200640 1 -- 172.21.15.46:0/3017341732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda08153220 con 0x7fda080ce2d0 2024-03-29T08:18:54.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.552+0000 7fda0f200640 1 -- 172.21.15.46:0/3017341732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0814f1c0 con 0x7fda080ce2d0 2024-03-29T08:18:54.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.556+0000 7fda0f200640 1 -- 172.21.15.46:0/3017341732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda08099080 con 0x7fda080ce2d0 2024-03-29T08:18:54.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.556+0000 7fda04c00640 1 -- 172.21.15.46:0/3017341732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd9f00073c0 con 0x7fda080ce2d0 2024-03-29T08:18:54.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.556+0000 7fda04c00640 1 --2- 172.21.15.46:0/3017341732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd9e4066690 0x7fd9e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.556+0000 7fda04c00640 1 -- 172.21.15.46:0/3017341732 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9f0083480 con 0x7fda080ce2d0 2024-03-29T08:18:54.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.556+0000 7fda07e00640 1 --2- 172.21.15.46:0/3017341732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd9e4066690 0x7fd9e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.556+0000 7fda07e00640 1 --2- 172.21.15.46:0/3017341732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd9e4066690 0x7fd9e4068b50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80ae000 tx=0x7fd9f80b5040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:54.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.560+0000 7fda04c00640 1 -- 172.21.15.46:0/3017341732 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd9f0050890 con 0x7fda080ce2d0 2024-03-29T08:18:54.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.732+0000 7fda0f200640 1 -- 172.21.15.46:0/3017341732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fda0814f020 con 0x7fda080ce2d0 2024-03-29T08:18:54.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.732+0000 7fda04c00640 1 -- 172.21.15.46:0/3017341732 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fda0814f020 con 0x7fda080ce2d0 2024-03-29T08:18:54.733 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:18:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.732+0000 7fda0f200640 1 -- 172.21.15.46:0/3017341732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd9e4066690 msgr2=0x7fd9e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.732+0000 7fda0f200640 1 --2- 172.21.15.46:0/3017341732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd9e4066690 0x7fd9e4068b50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd9f80ae000 tx=0x7fd9f80b5040 comp rx=0 tx=0).stop 2024-03-29T08:18:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.732+0000 7fda0f200640 1 -- 172.21.15.46:0/3017341732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda080ce2d0 msgr2=0x7fda0814d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.732+0000 7fda0f200640 1 --2- 172.21.15.46:0/3017341732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda080ce2d0 0x7fda0814d670 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd9f000c340 tx=0x7fd9f000c800 comp rx=0 tx=0).stop 2024-03-29T08:18:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.732+0000 7fda0f200640 1 -- 172.21.15.46:0/3017341732 shutdown_connections 2024-03-29T08:18:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.732+0000 7fda0f200640 1 --2- 172.21.15.46:0/3017341732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd9e4066690 0x7fd9e4068b50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.732+0000 7fda0f200640 1 --2- 172.21.15.46:0/3017341732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda08159970 0x7fda0815c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.732+0000 7fda0f200640 1 --2- 172.21.15.46:0/3017341732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda080ce2d0 0x7fda0814d670 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.732+0000 7fda0f200640 1 --2- 172.21.15.46:0/3017341732 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda080cbf60 0x7fda08156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.732+0000 7fda0f200640 1 -- 172.21.15.46:0/3017341732 >> 172.21.15.46:0/3017341732 conn(0x7fda08098a40 msgr2=0x7fda081551f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:54.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.732+0000 7fda0f200640 1 -- 172.21.15.46:0/3017341732 shutdown_connections 2024-03-29T08:18:54.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.732+0000 7fda0f200640 1 -- 172.21.15.46:0/3017341732 wait complete. 2024-03-29T08:18:54.754 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:18:54.754 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:18:54.763 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.948+0000 7ffadf200640 1 Processor -- start 2024-03-29T08:18:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.948+0000 7ffadf200640 1 -- start start 2024-03-29T08:18:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.948+0000 7ffadf200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffad80d0660 0x7ffad80d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.948+0000 7ffadf200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffad806b8b0 0x7ffad80cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.948+0000 7ffadf200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad815cfb0 con 0x7ffad815dc00 2024-03-29T08:18:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.948+0000 7ffadf200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad806c100 con 0x7ffad806b8b0 2024-03-29T08:18:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.948+0000 7ffadf200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad80d0f80 con 0x7ffad80d0660 2024-03-29T08:18:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.952+0000 7ffacf400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffad806b8b0 0x7ffad80cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.952+0000 7ffacf400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffad806b8b0 0x7ffad80cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50494/0 (socket says 172.21.15.46:50494) 2024-03-29T08:18:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.952+0000 7ffacf400640 1 -- 172.21.15.46:0/560356434 learned_addr learned my addr 172.21.15.46:0/560356434 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.952+0000 7ffad7e00640 1 --2- 172.21.15.46:0/560356434 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffad80d0660 0x7ffad80d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.952+0000 7ffacf400640 1 -- 172.21.15.46:0/560356434 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffad80d0660 msgr2=0x7ffad80d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.952+0000 7ffacf400640 1 --2- 172.21.15.46:0/560356434 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffad80d0660 0x7ffad80d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.952+0000 7ffacf400640 1 -- 172.21.15.46:0/560356434 >> v1:172.21.15.73:6789/0 conn(0x7ffad815dc00 legacy=0x7ffad80cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.952+0000 7ffacf400640 1 -- 172.21.15.46:0/560356434 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad806be20 con 0x7ffad806b8b0 2024-03-29T08:18:54.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.952+0000 7ffacf400640 1 --2- 172.21.15.46:0/560356434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad806b8b0 0x7ffad80cad20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009870 tx=0x7ffac001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb70bf4a407ded07 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.952+0000 7ffad7400640 1 -- 172.21.15.46:0/560356434 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac00a95f0 con 0x7ffad806b8b0 2024-03-29T08:18:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.952+0000 7ffad7400640 1 -- 172.21.15.46:0/560356434 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac00a9750 con 0x7ffad806b8b0 2024-03-29T08:18:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.952+0000 7ffad7400640 1 -- 172.21.15.46:0/560356434 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac00a9ae0 con 0x7ffad806b8b0 2024-03-29T08:18:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffadf200640 1 -- 172.21.15.46:0/560356434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad806b8b0 msgr2=0x7ffad80cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffadf200640 1 --2- 172.21.15.46:0/560356434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad806b8b0 0x7ffad80cad20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009870 tx=0x7ffac001c530 comp rx=0 tx=0).stop 2024-03-29T08:18:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffadf200640 1 -- 172.21.15.46:0/560356434 shutdown_connections 2024-03-29T08:18:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffadf200640 1 --2- 172.21.15.46:0/560356434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad806b8b0 0x7ffad80cad20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffadf200640 1 --2- 172.21.15.46:0/560356434 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffad80d0660 0x7ffad80d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffadf200640 1 -- 172.21.15.46:0/560356434 >> 172.21.15.46:0/560356434 conn(0x7ffad8089fd0 msgr2=0x7ffad808a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffadf200640 1 -- 172.21.15.46:0/560356434 shutdown_connections 2024-03-29T08:18:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffadf200640 1 -- 172.21.15.46:0/560356434 wait complete. 2024-03-29T08:18:54.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffadf200640 1 Processor -- start 2024-03-29T08:18:54.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffadf200640 1 -- start start 2024-03-29T08:18:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffadf200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffad806b8b0 0x7ffad8152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffadf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad80d0660 0x7ffad80c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffadf200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffad815dc00 0x7ffad80b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffadf200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad80b5080 con 0x7ffad80d0660 2024-03-29T08:18:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffadf200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad80cb070 con 0x7ffad815dc00 2024-03-29T08:18:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffadf200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad80ca310 con 0x7ffad806b8b0 2024-03-29T08:18:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffacf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad80d0660 0x7ffad80c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffacf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad80d0660 0x7ffad80c9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50506/0 (socket says 172.21.15.46:50506) 2024-03-29T08:18:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffacf400640 1 -- 172.21.15.46:0/2316282879 learned_addr learned my addr 172.21.15.46:0/2316282879 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffad7e00640 1 --2- 172.21.15.46:0/2316282879 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffad806b8b0 0x7ffad8152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffadd200640 1 --2- 172.21.15.46:0/2316282879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffad815dc00 0x7ffad80b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffacf400640 1 -- 172.21.15.46:0/2316282879 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffad806b8b0 msgr2=0x7ffad8152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffacf400640 1 --2- 172.21.15.46:0/2316282879 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffad806b8b0 0x7ffad8152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffacf400640 1 -- 172.21.15.46:0/2316282879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffad815dc00 msgr2=0x7ffad80b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffacf400640 1 --2- 172.21.15.46:0/2316282879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffad815dc00 0x7ffad80b5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffacf400640 1 -- 172.21.15.46:0/2316282879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac00094d0 con 0x7ffad80d0660 2024-03-29T08:18:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.956+0000 7ffacf400640 1 --2- 172.21.15.46:0/2316282879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad80d0660 0x7ffad80c9910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ffac001a160 tx=0x7ffac00192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.960+0000 7ffad4c00640 1 -- 172.21.15.46:0/2316282879 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac00b9620 con 0x7ffad80d0660 2024-03-29T08:18:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.960+0000 7ffad4c00640 1 -- 172.21.15.46:0/2316282879 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac00b97a0 con 0x7ffad80d0660 2024-03-29T08:18:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.960+0000 7ffadf200640 1 -- 172.21.15.46:0/2316282879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad80b4d50 con 0x7ffad80d0660 2024-03-29T08:18:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.960+0000 7ffad4c00640 1 -- 172.21.15.46:0/2316282879 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac00b9b50 con 0x7ffad80d0660 2024-03-29T08:18:54.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.960+0000 7ffadf200640 1 -- 172.21.15.46:0/2316282879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad80cad20 con 0x7ffad80d0660 2024-03-29T08:18:54.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.960+0000 7ffadf200640 1 -- 172.21.15.46:0/2316282879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad8066c90 con 0x7ffad80d0660 2024-03-29T08:18:54.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.960+0000 7ffad4c00640 1 -- 172.21.15.46:0/2316282879 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ffac00b13f0 con 0x7ffad80d0660 2024-03-29T08:18:54.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.960+0000 7ffad4c00640 1 --2- 172.21.15.46:0/2316282879 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffab40666c0 0x7ffab4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:54.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.960+0000 7ffad4c00640 1 -- 172.21.15.46:0/2316282879 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffac011e930 con 0x7ffad80d0660 2024-03-29T08:18:54.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.960+0000 7ffad7e00640 1 --2- 172.21.15.46:0/2316282879 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffab40666c0 0x7ffab4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:54.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.964+0000 7ffad7e00640 1 --2- 172.21.15.46:0/2316282879 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffab40666c0 0x7ffab4068b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffac80113e0 tx=0x7ffac800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:54.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:54.964+0000 7ffad4c00640 1 -- 172.21.15.46:0/2316282879 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ffac00ebcb0 con 0x7ffad80d0660 2024-03-29T08:18:55.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.132+0000 7ffadf200640 1 -- 172.21.15.46:0/2316282879 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffad80cc570 con 0x7ffad80d0660 2024-03-29T08:18:55.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.132+0000 7ffad4c00640 1 -- 172.21.15.46:0/2316282879 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16042 (secure 0 0 0) 0x7ffac00ef2d0 con 0x7ffad80d0660 2024-03-29T08:18:55.135 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:55.135 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":23,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:55.014530+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:55.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.136+0000 7ffadf200640 1 -- 172.21.15.46:0/2316282879 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffab40666c0 msgr2=0x7ffab4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.136+0000 7ffadf200640 1 --2- 172.21.15.46:0/2316282879 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffab40666c0 0x7ffab4068b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffac80113e0 tx=0x7ffac800b040 comp rx=0 tx=0).stop 2024-03-29T08:18:55.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.136+0000 7ffadf200640 1 -- 172.21.15.46:0/2316282879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad80d0660 msgr2=0x7ffad80c9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.136+0000 7ffadf200640 1 --2- 172.21.15.46:0/2316282879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad80d0660 0x7ffad80c9910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ffac001a160 tx=0x7ffac00192d0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.136+0000 7ffadf200640 1 -- 172.21.15.46:0/2316282879 shutdown_connections 2024-03-29T08:18:55.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.136+0000 7ffadf200640 1 --2- 172.21.15.46:0/2316282879 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffab40666c0 0x7ffab4068b80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.136+0000 7ffadf200640 1 --2- 172.21.15.46:0/2316282879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffad815dc00 0x7ffad80b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.136+0000 7ffadf200640 1 --2- 172.21.15.46:0/2316282879 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffad80d0660 0x7ffad80c9910 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.136+0000 7ffadf200640 1 --2- 172.21.15.46:0/2316282879 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffad806b8b0 0x7ffad8152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.136+0000 7ffadf200640 1 -- 172.21.15.46:0/2316282879 >> 172.21.15.46:0/2316282879 conn(0x7ffad8089fd0 msgr2=0x7ffad80cb500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:55.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.136+0000 7ffadf200640 1 -- 172.21.15.46:0/2316282879 shutdown_connections 2024-03-29T08:18:55.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.136+0000 7ffadf200640 1 -- 172.21.15.46:0/2316282879 wait complete. 2024-03-29T08:18:55.157 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:18:11.177842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:18:12.308252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:18:55.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:18:55.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.300+0000 7f399d800640 1 Processor -- start 2024-03-29T08:18:55.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.300+0000 7f399d800640 1 -- start start 2024-03-29T08:18:55.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.300+0000 7f399d800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f399815a7f0 0x7f39980cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.300+0000 7f399d800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3998098a40 0x7f3998098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.300+0000 7f399d800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39980cbd00 con 0x7f399815abd0 2024-03-29T08:18:55.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.300+0000 7f399d800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39980b4500 con 0x7f3998098a40 2024-03-29T08:18:55.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.300+0000 7f399d800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39980cf970 con 0x7f399815a7f0 2024-03-29T08:18:55.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.300+0000 7f3996a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f399815a7f0 0x7f39980cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.300+0000 7f3997400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3998098a40 0x7f3998098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.300+0000 7f3996a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f399815a7f0 0x7f39980cf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:40320/0 (socket says 172.21.15.46:40320) 2024-03-29T08:18:55.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.300+0000 7f3997400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3998098a40 0x7f3998098ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47374/0 (socket says 172.21.15.46:47374) 2024-03-29T08:18:55.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.300+0000 7f3996a00640 1 -- 172.21.15.46:0/2310996215 learned_addr learned my addr 172.21.15.46:0/2310996215 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:55.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.300+0000 7f3997400640 1 -- 172.21.15.46:0/2310996215 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f399815a7f0 msgr2=0x7f39980cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:55.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.300+0000 7f3997400640 1 --2- 172.21.15.46:0/2310996215 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f399815a7f0 0x7f39980cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.300+0000 7f3997400640 1 -- 172.21.15.46:0/2310996215 >> v1:172.21.15.73:6789/0 conn(0x7f399815abd0 legacy=0x7f39980cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.300+0000 7f3997400640 1 -- 172.21.15.46:0/2310996215 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39980cbfe0 con 0x7f3998098a40 2024-03-29T08:18:55.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.304+0000 7f3997400640 1 --2- 172.21.15.46:0/2310996215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3998098a40 0x7f3998098ed0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f39900097c0 tx=0x7f399001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99204d404ae0bec2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:55.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.304+0000 7f3996000640 1 -- 172.21.15.46:0/2310996215 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39900ad640 con 0x7f3998098a40 2024-03-29T08:18:55.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.304+0000 7f3996000640 1 -- 172.21.15.46:0/2310996215 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39900ad7a0 con 0x7f3998098a40 2024-03-29T08:18:55.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.304+0000 7f3996000640 1 -- 172.21.15.46:0/2310996215 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39900adb30 con 0x7f3998098a40 2024-03-29T08:18:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.304+0000 7f399d800640 1 -- 172.21.15.46:0/2310996215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3998098a40 msgr2=0x7f3998098ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.304+0000 7f399d800640 1 --2- 172.21.15.46:0/2310996215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3998098a40 0x7f3998098ed0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f39900097c0 tx=0x7f399001c4a0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.304+0000 7f399d800640 1 -- 172.21.15.46:0/2310996215 shutdown_connections 2024-03-29T08:18:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.304+0000 7f399d800640 1 --2- 172.21.15.46:0/2310996215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3998098a40 0x7f3998098ed0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.304+0000 7f399d800640 1 --2- 172.21.15.46:0/2310996215 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f399815a7f0 0x7f39980cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.304+0000 7f399d800640 1 -- 172.21.15.46:0/2310996215 >> 172.21.15.46:0/2310996215 conn(0x7f39980b2190 msgr2=0x7f39980b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.304+0000 7f399d800640 1 -- 172.21.15.46:0/2310996215 shutdown_connections 2024-03-29T08:18:55.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.304+0000 7f399d800640 1 -- 172.21.15.46:0/2310996215 wait complete. 2024-03-29T08:18:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f399d800640 1 Processor -- start 2024-03-29T08:18:55.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f399d800640 1 -- start start 2024-03-29T08:18:55.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f399d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3998098a40 0x7f399814c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f399d800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f399815a7f0 0x7f3998153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f399d800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f399815abd0 0x7f399814b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f399d800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399806aea0 con 0x7f3998098a40 2024-03-29T08:18:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f399d800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399806b0f0 con 0x7f399815abd0 2024-03-29T08:18:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f399d800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399806b3d0 con 0x7f399815a7f0 2024-03-29T08:18:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f3996a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3998098a40 0x7f399814c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f3996a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3998098a40 0x7f399814c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50508/0 (socket says 172.21.15.46:50508) 2024-03-29T08:18:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f3996a00640 1 -- 172.21.15.46:0/2823766485 learned_addr learned my addr 172.21.15.46:0/2823766485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:55.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f3997400640 1 --2- 172.21.15.46:0/2823766485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f399815abd0 0x7f399814b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f398e000640 1 --2- 172.21.15.46:0/2823766485 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f399815a7f0 0x7f3998153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f3996a00640 1 -- 172.21.15.46:0/2823766485 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f399815a7f0 msgr2=0x7f3998153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f3996a00640 1 --2- 172.21.15.46:0/2823766485 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f399815a7f0 0x7f3998153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f3996a00640 1 -- 172.21.15.46:0/2823766485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f399815abd0 msgr2=0x7f399814b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f3996a00640 1 --2- 172.21.15.46:0/2823766485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f399815abd0 0x7f399814b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f3996a00640 1 -- 172.21.15.46:0/2823766485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3990009420 con 0x7f3998098a40 2024-03-29T08:18:55.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f3996a00640 1 --2- 172.21.15.46:0/2823766485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3998098a40 0x7f399814c660 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f398800e890 tx=0x7f398800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:55.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f398f400640 1 -- 172.21.15.46:0/2823766485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3988009620 con 0x7f3998098a40 2024-03-29T08:18:55.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f398f400640 1 -- 172.21.15.46:0/2823766485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398800c940 con 0x7f3998098a40 2024-03-29T08:18:55.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f398f400640 1 -- 172.21.15.46:0/2823766485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398800ccf0 con 0x7f3998098a40 2024-03-29T08:18:55.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f399d800640 1 -- 172.21.15.46:0/2823766485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399806b6b0 con 0x7f3998098a40 2024-03-29T08:18:55.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.308+0000 7f399d800640 1 -- 172.21.15.46:0/2823766485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399806bc70 con 0x7f3998098a40 2024-03-29T08:18:55.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.312+0000 7f399d800640 1 -- 172.21.15.46:0/2823766485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399809e690 con 0x7f3998098a40 2024-03-29T08:18:55.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.312+0000 7f398f400640 1 -- 172.21.15.46:0/2823766485 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3988013470 con 0x7f3998098a40 2024-03-29T08:18:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.312+0000 7f398f400640 1 --2- 172.21.15.46:0/2823766485 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3974066690 0x7f3974068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.312+0000 7f398f400640 1 -- 172.21.15.46:0/2823766485 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f3988084390 con 0x7f3998098a40 2024-03-29T08:18:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.312+0000 7f398e000640 1 --2- 172.21.15.46:0/2823766485 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3974066690 0x7f3974068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.312+0000 7f398e000640 1 --2- 172.21.15.46:0/2823766485 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3974066690 0x7f3974068b50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3998157900 tx=0x7f3980009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:55.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.316+0000 7f398f400640 1 -- 172.21.15.46:0/2823766485 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f39880516f0 con 0x7f3998098a40 2024-03-29T08:18:55.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.480+0000 7f399d800640 1 -- 172.21.15.46:0/2823766485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3998157bb0 con 0x7f3998098a40 2024-03-29T08:18:55.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.480+0000 7f398f400640 1 -- 172.21.15.46:0/2823766485 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3988054d10 con 0x7f3998098a40 2024-03-29T08:18:55.481 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:18:55.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.480+0000 7f399d800640 1 -- 172.21.15.46:0/2823766485 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3974066690 msgr2=0x7f3974068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.480+0000 7f399d800640 1 --2- 172.21.15.46:0/2823766485 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3974066690 0x7f3974068b50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3998157900 tx=0x7f3980009040 comp rx=0 tx=0).stop 2024-03-29T08:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.480+0000 7f399d800640 1 -- 172.21.15.46:0/2823766485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3998098a40 msgr2=0x7f399814c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.480+0000 7f399d800640 1 --2- 172.21.15.46:0/2823766485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3998098a40 0x7f399814c660 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f398800e890 tx=0x7f398800ed50 comp rx=0 tx=0).stop 2024-03-29T08:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.480+0000 7f399d800640 1 -- 172.21.15.46:0/2823766485 shutdown_connections 2024-03-29T08:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.480+0000 7f399d800640 1 --2- 172.21.15.46:0/2823766485 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3974066690 0x7f3974068b50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.480+0000 7f399d800640 1 --2- 172.21.15.46:0/2823766485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f399815abd0 0x7f399814b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.480+0000 7f399d800640 1 --2- 172.21.15.46:0/2823766485 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f399815a7f0 0x7f3998153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.480+0000 7f399d800640 1 --2- 172.21.15.46:0/2823766485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3998098a40 0x7f399814c660 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.480+0000 7f399d800640 1 -- 172.21.15.46:0/2823766485 >> 172.21.15.46:0/2823766485 conn(0x7f39980b2190 msgr2=0x7f39980cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.480+0000 7f399d800640 1 -- 172.21.15.46:0/2823766485 shutdown_connections 2024-03-29T08:18:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.480+0000 7f399d800640 1 -- 172.21.15.46:0/2823766485 wait complete. 2024-03-29T08:18:55.503 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:18:55.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.644+0000 7f9820a00640 1 Processor -- start 2024-03-29T08:18:55.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.644+0000 7f9820a00640 1 -- start start 2024-03-29T08:18:55.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.644+0000 7f9820a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f981c0d0dc0 0x7f981c0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.644+0000 7f9820a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f981c0cf820 0x7f981c0b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.644+0000 7f9820a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981c15cf40 con 0x7f981c15cb60 2024-03-29T08:18:55.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.644+0000 7f9820a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f981c0b0250 con 0x7f981c0cf820 2024-03-29T08:18:55.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.644+0000 7f9820a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0b0810 con 0x7f981c0d0dc0 2024-03-29T08:18:55.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.644+0000 7f981a800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f981c0cf820 0x7f981c0b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.644+0000 7f981a800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f981c0cf820 0x7f981c0b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47392/0 (socket says 172.21.15.46:47392) 2024-03-29T08:18:55.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.644+0000 7f981a800640 1 -- 172.21.15.46:0/1576886278 learned_addr learned my addr 172.21.15.46:0/1576886278 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:55.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.644+0000 7f9811400640 1 --2- 172.21.15.46:0/1576886278 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f981c0d0dc0 0x7f981c0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.644+0000 7f981a800640 1 -- 172.21.15.46:0/1576886278 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f981c0d0dc0 msgr2=0x7f981c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:55.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.644+0000 7f981a800640 1 --2- 172.21.15.46:0/1576886278 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f981c0d0dc0 0x7f981c0cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.644+0000 7f981a800640 1 -- 172.21.15.46:0/1576886278 >> v1:172.21.15.73:6789/0 conn(0x7f981c15cb60 legacy=0x7f981c0d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.644+0000 7f981a800640 1 -- 172.21.15.46:0/1576886278 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c0d0010 con 0x7f981c0cf820 2024-03-29T08:18:55.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.648+0000 7f981a800640 1 --2- 172.21.15.46:0/1576886278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981c0cf820 0x7f981c0b0ca0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9814009870 tx=0x7f981401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7bc22b9e22a016c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.648+0000 7f9819400640 1 -- 172.21.15.46:0/1576886278 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98140ab600 con 0x7f981c0cf820 2024-03-29T08:18:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.648+0000 7f9819400640 1 -- 172.21.15.46:0/1576886278 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98140a7050 con 0x7f981c0cf820 2024-03-29T08:18:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.648+0000 7f9819400640 1 -- 172.21.15.46:0/1576886278 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98140ab980 con 0x7f981c0cf820 2024-03-29T08:18:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.648+0000 7f9820a00640 1 -- 172.21.15.46:0/1576886278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981c0cf820 msgr2=0x7f981c0b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.648+0000 7f9820a00640 1 --2- 172.21.15.46:0/1576886278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981c0cf820 0x7f981c0b0ca0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9814009870 tx=0x7f981401c3c0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.648+0000 7f9820a00640 1 -- 172.21.15.46:0/1576886278 shutdown_connections 2024-03-29T08:18:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.648+0000 7f9820a00640 1 --2- 172.21.15.46:0/1576886278 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f981c0d0dc0 0x7f981c0cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.648+0000 7f9820a00640 1 --2- 172.21.15.46:0/1576886278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981c0cf820 0x7f981c0b0ca0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.648+0000 7f9820a00640 1 -- 172.21.15.46:0/1576886278 >> 172.21.15.46:0/1576886278 conn(0x7f981c099a10 msgr2=0x7f981c089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.648+0000 7f9820a00640 1 -- 172.21.15.46:0/1576886278 shutdown_connections 2024-03-29T08:18:55.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.648+0000 7f9820a00640 1 -- 172.21.15.46:0/1576886278 wait complete. 2024-03-29T08:18:55.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9820a00640 1 Processor -- start 2024-03-29T08:18:55.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9820a00640 1 -- start start 2024-03-29T08:18:55.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9820a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981c0cf820 0x7f981c149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9820a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c0d0dc0 0x7f981c15bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9820a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f981c15cb60 0x7f981c06e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9820a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c14a4c0 con 0x7f981c0d0dc0 2024-03-29T08:18:55.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9820a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c15a7f0 con 0x7f981c0cf820 2024-03-29T08:18:55.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9820a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c15b140 con 0x7f981c15cb60 2024-03-29T08:18:55.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9811400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c0d0dc0 0x7f981c15bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9811400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c0d0dc0 0x7f981c15bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50520/0 (socket says 172.21.15.46:50520) 2024-03-29T08:18:55.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9811400640 1 -- 172.21.15.46:0/2904834212 learned_addr learned my addr 172.21.15.46:0/2904834212 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:55.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f981a800640 1 --2- 172.21.15.46:0/2904834212 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f981c15cb60 0x7f981c06e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9819e00640 1 --2- 172.21.15.46:0/2904834212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981c0cf820 0x7f981c149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9811400640 1 -- 172.21.15.46:0/2904834212 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f981c15cb60 msgr2=0x7f981c06e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9811400640 1 --2- 172.21.15.46:0/2904834212 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f981c15cb60 0x7f981c06e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9811400640 1 -- 172.21.15.46:0/2904834212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981c0cf820 msgr2=0x7f981c149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9811400640 1 --2- 172.21.15.46:0/2904834212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981c0cf820 0x7f981c149f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9811400640 1 -- 172.21.15.46:0/2904834212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98140094d0 con 0x7f981c0d0dc0 2024-03-29T08:18:55.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9819e00640 1 --2- 172.21.15.46:0/2904834212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981c0cf820 0x7f981c149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:55.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9811400640 1 --2- 172.21.15.46:0/2904834212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c0d0dc0 0x7f981c15bbe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f980400efc0 tx=0x7f980400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:55.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9812a00640 1 -- 172.21.15.46:0/2904834212 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804014480 con 0x7f981c0d0dc0 2024-03-29T08:18:55.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9812a00640 1 -- 172.21.15.46:0/2904834212 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980400f040 con 0x7f981c0d0dc0 2024-03-29T08:18:55.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9820a00640 1 -- 172.21.15.46:0/2904834212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c15aa40 con 0x7f981c0d0dc0 2024-03-29T08:18:55.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9812a00640 1 -- 172.21.15.46:0/2904834212 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804014800 con 0x7f981c0d0dc0 2024-03-29T08:18:55.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.652+0000 7f9820a00640 1 -- 172.21.15.46:0/2904834212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c15b3e0 con 0x7f981c0d0dc0 2024-03-29T08:18:55.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.656+0000 7f9820a00640 1 -- 172.21.15.46:0/2904834212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981c09e7d0 con 0x7f981c0d0dc0 2024-03-29T08:18:55.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.656+0000 7f9812a00640 1 -- 172.21.15.46:0/2904834212 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9804014960 con 0x7f981c0d0dc0 2024-03-29T08:18:55.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.656+0000 7f9812a00640 1 --2- 172.21.15.46:0/2904834212 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97f8066690 0x7f97f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.656+0000 7f9812a00640 1 -- 172.21.15.46:0/2904834212 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9804084dd0 con 0x7f981c0d0dc0 2024-03-29T08:18:55.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.656+0000 7f9819e00640 1 --2- 172.21.15.46:0/2904834212 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97f8066690 0x7f97f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.656+0000 7f9819e00640 1 --2- 172.21.15.46:0/2904834212 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97f8066690 0x7f97f8068b50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f980c000f70 tx=0x7f980c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:55.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.660+0000 7f9812a00640 1 -- 172.21.15.46:0/2904834212 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9804052080 con 0x7f981c0d0dc0 2024-03-29T08:18:55.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.820+0000 7f9820a00640 1 -- 172.21.15.46:0/2904834212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f981c04e120 con 0x7f981c0d0dc0 2024-03-29T08:18:55.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.820+0000 7f9812a00640 1 -- 172.21.15.46:0/2904834212 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f98040556a0 con 0x7f981c0d0dc0 2024-03-29T08:18:55.822 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:18:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.824+0000 7f9820a00640 1 -- 172.21.15.46:0/2904834212 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97f8066690 msgr2=0x7f97f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.824+0000 7f9820a00640 1 --2- 172.21.15.46:0/2904834212 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97f8066690 0x7f97f8068b50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f980c000f70 tx=0x7f980c009040 comp rx=0 tx=0).stop 2024-03-29T08:18:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.824+0000 7f9820a00640 1 -- 172.21.15.46:0/2904834212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c0d0dc0 msgr2=0x7f981c15bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.824+0000 7f9820a00640 1 --2- 172.21.15.46:0/2904834212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c0d0dc0 0x7f981c15bbe0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f980400efc0 tx=0x7f980400c490 comp rx=0 tx=0).stop 2024-03-29T08:18:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.824+0000 7f9820a00640 1 -- 172.21.15.46:0/2904834212 shutdown_connections 2024-03-29T08:18:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.824+0000 7f9820a00640 1 --2- 172.21.15.46:0/2904834212 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97f8066690 0x7f97f8068b50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.824+0000 7f9820a00640 1 --2- 172.21.15.46:0/2904834212 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f981c15cb60 0x7f981c06e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.824+0000 7f9820a00640 1 --2- 172.21.15.46:0/2904834212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c0d0dc0 0x7f981c15bbe0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.824+0000 7f9820a00640 1 --2- 172.21.15.46:0/2904834212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f981c0cf820 0x7f981c149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.824+0000 7f9820a00640 1 -- 172.21.15.46:0/2904834212 >> 172.21.15.46:0/2904834212 conn(0x7f981c099a10 msgr2=0x7f981c0af710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.824+0000 7f9820a00640 1 -- 172.21.15.46:0/2904834212 shutdown_connections 2024-03-29T08:18:55.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.824+0000 7f9820a00640 1 -- 172.21.15.46:0/2904834212 wait complete. 2024-03-29T08:18:55.843 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:18:55.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.984+0000 7f81c6200640 1 Processor -- start 2024-03-29T08:18:55.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.984+0000 7f81c6200640 1 -- start start 2024-03-29T08:18:55.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.984+0000 7f81c6200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f81c0160100 0x7f81c00cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.984+0000 7f81c6200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f81c006bac0 0x7f81c00b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.984+0000 7f81c6200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c00cb230 con 0x7f81c0098a40 2024-03-29T08:18:55.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.984+0000 7f81c6200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81c006c030 con 0x7f81c0160100 2024-03-29T08:18:55.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.984+0000 7f81c6200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c006c310 con 0x7f81c006bac0 2024-03-29T08:18:55.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.984+0000 7f81bf400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f81c0160100 0x7f81c00cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.988+0000 7f81bf400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f81c0160100 0x7f81c00cacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50522/0 (socket says 172.21.15.46:50522) 2024-03-29T08:18:55.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.988+0000 7f81bf400640 1 -- 172.21.15.46:0/3641327774 learned_addr learned my addr 172.21.15.46:0/3641327774 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.988+0000 7f81b7e00640 1 --2- 172.21.15.46:0/3641327774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f81c006bac0 0x7f81c00b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.988+0000 7f81bf400640 1 -- 172.21.15.46:0/3641327774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f81c006bac0 msgr2=0x7f81c00b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.988+0000 7f81bf400640 1 --2- 172.21.15.46:0/3641327774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f81c006bac0 0x7f81c00b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.988+0000 7f81bf400640 1 -- 172.21.15.46:0/3641327774 >> v1:172.21.15.73:6789/0 conn(0x7f81c0098a40 legacy=0x7f81c00cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.988+0000 7f81bf400640 1 -- 172.21.15.46:0/3641327774 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c00cee60 con 0x7f81c0160100 2024-03-29T08:18:55.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.988+0000 7f81b7e00640 1 --2- 172.21.15.46:0/3641327774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f81c006bac0 0x7f81c00b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:55.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.988+0000 7f81bf400640 1 --2- 172.21.15.46:0/3641327774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c0160100 0x7f81c00cacf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009870 tx=0x7f81b001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905302be32686481 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:55.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.988+0000 7f81bea00640 1 -- 172.21.15.46:0/3641327774 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b00ab650 con 0x7f81c0160100 2024-03-29T08:18:55.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.988+0000 7f81bea00640 1 -- 172.21.15.46:0/3641327774 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b00ab7b0 con 0x7f81c0160100 2024-03-29T08:18:55.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.988+0000 7f81bea00640 1 -- 172.21.15.46:0/3641327774 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b00abb40 con 0x7f81c0160100 2024-03-29T08:18:55.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81c6200640 1 -- 172.21.15.46:0/3641327774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c0160100 msgr2=0x7f81c00cacf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81c6200640 1 --2- 172.21.15.46:0/3641327774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c0160100 0x7f81c00cacf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f81b0009870 tx=0x7f81b001c560 comp rx=0 tx=0).stop 2024-03-29T08:18:55.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81c6200640 1 -- 172.21.15.46:0/3641327774 shutdown_connections 2024-03-29T08:18:55.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81c6200640 1 --2- 172.21.15.46:0/3641327774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f81c006bac0 0x7f81c00b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81c6200640 1 --2- 172.21.15.46:0/3641327774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c0160100 0x7f81c00cacf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81c6200640 1 -- 172.21.15.46:0/3641327774 >> 172.21.15.46:0/3641327774 conn(0x7f81c00b2190 msgr2=0x7f81c00b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:55.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81c6200640 1 -- 172.21.15.46:0/3641327774 shutdown_connections 2024-03-29T08:18:55.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81c6200640 1 -- 172.21.15.46:0/3641327774 wait complete. 2024-03-29T08:18:55.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81c6200640 1 Processor -- start 2024-03-29T08:18:55.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81c6200640 1 -- start start 2024-03-29T08:18:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81c6200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f81c006bac0 0x7f81c00b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81c6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c0098a40 0x7f81c009a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81c6200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81c0160100 0x7f81c009d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81c6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c015ecb0 con 0x7f81c0098a40 2024-03-29T08:18:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81c6200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c0154a70 con 0x7f81c0160100 2024-03-29T08:18:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81c6200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c015ef00 con 0x7f81c006bac0 2024-03-29T08:18:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c0098a40 0x7f81c009a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c0098a40 0x7f81c009a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50538/0 (socket says 172.21.15.46:50538) 2024-03-29T08:18:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81b7e00640 1 -- 172.21.15.46:0/4012727487 learned_addr learned my addr 172.21.15.46:0/4012727487 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81bf400640 1 --2- 172.21.15.46:0/4012727487 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f81c006bac0 0x7f81c00b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81bfe00640 1 --2- 172.21.15.46:0/4012727487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81c0160100 0x7f81c009d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81b7e00640 1 -- 172.21.15.46:0/4012727487 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f81c006bac0 msgr2=0x7f81c00b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81b7e00640 1 --2- 172.21.15.46:0/4012727487 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f81c006bac0 0x7f81c00b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81b7e00640 1 -- 172.21.15.46:0/4012727487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81c0160100 msgr2=0x7f81c009d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81b7e00640 1 --2- 172.21.15.46:0/4012727487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81c0160100 0x7f81c009d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.992+0000 7f81b7e00640 1 -- 172.21.15.46:0/4012727487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b00094d0 con 0x7f81c0098a40 2024-03-29T08:18:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.996+0000 7f81b7e00640 1 --2- 172.21.15.46:0/4012727487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c0098a40 0x7f81c009a680 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f81a800d800 tx=0x7f81a800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.996+0000 7f81b7400640 1 -- 172.21.15.46:0/4012727487 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a800b6c0 con 0x7f81c0098a40 2024-03-29T08:18:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.996+0000 7f81b7400640 1 -- 172.21.15.46:0/4012727487 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a800b820 con 0x7f81c0098a40 2024-03-29T08:18:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.996+0000 7f81b7400640 1 -- 172.21.15.46:0/4012727487 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a800bbd0 con 0x7f81c0098a40 2024-03-29T08:18:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.996+0000 7f81c6200640 1 -- 172.21.15.46:0/4012727487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c00993e0 con 0x7f81c0098a40 2024-03-29T08:18:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.996+0000 7f81c6200640 1 -- 172.21.15.46:0/4012727487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c00c7380 con 0x7f81c0098a40 2024-03-29T08:18:55.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.996+0000 7f81c6200640 1 -- 172.21.15.46:0/4012727487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c015f980 con 0x7f81c0098a40 2024-03-29T08:18:55.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.996+0000 7f81b7400640 1 -- 172.21.15.46:0/4012727487 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f81a800bd30 con 0x7f81c0098a40 2024-03-29T08:18:55.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.996+0000 7f81b7400640 1 --2- 172.21.15.46:0/4012727487 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f819c066690 0x7f819c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:55.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.996+0000 7f81b7400640 1 -- 172.21.15.46:0/4012727487 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f81a8084620 con 0x7f81c0098a40 2024-03-29T08:18:55.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:55.996+0000 7f81bf400640 1 --2- 172.21.15.46:0/4012727487 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f819c066690 0x7f819c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:55.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.000+0000 7f81bf400640 1 --2- 172.21.15.46:0/4012727487 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f819c066690 0x7f819c068b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f81b00af000 tx=0x7f81b00b6040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:56.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.000+0000 7f81b7400640 1 -- 172.21.15.46:0/4012727487 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f81a8051960 con 0x7f81c0098a40 2024-03-29T08:18:56.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.164+0000 7f81c6200640 1 -- 172.21.15.46:0/4012727487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f81c009e690 con 0x7f81c0098a40 2024-03-29T08:18:56.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.164+0000 7f81b7400640 1 -- 172.21.15.46:0/4012727487 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f81a8054f80 con 0x7f81c0098a40 2024-03-29T08:18:56.165 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:18:56.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.164+0000 7f81c6200640 1 -- 172.21.15.46:0/4012727487 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f819c066690 msgr2=0x7f819c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:56.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.164+0000 7f81c6200640 1 --2- 172.21.15.46:0/4012727487 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f819c066690 0x7f819c068b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f81b00af000 tx=0x7f81b00b6040 comp rx=0 tx=0).stop 2024-03-29T08:18:56.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.164+0000 7f81c6200640 1 -- 172.21.15.46:0/4012727487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c0098a40 msgr2=0x7f81c009a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:56.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.164+0000 7f81c6200640 1 --2- 172.21.15.46:0/4012727487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c0098a40 0x7f81c009a680 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f81a800d800 tx=0x7f81a800dcc0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.164+0000 7f81c6200640 1 -- 172.21.15.46:0/4012727487 shutdown_connections 2024-03-29T08:18:56.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.164+0000 7f81c6200640 1 --2- 172.21.15.46:0/4012727487 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f819c066690 0x7f819c068b50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.164+0000 7f81c6200640 1 --2- 172.21.15.46:0/4012727487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81c0160100 0x7f81c009d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.164+0000 7f81c6200640 1 --2- 172.21.15.46:0/4012727487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81c0098a40 0x7f81c009a680 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.164+0000 7f81c6200640 1 --2- 172.21.15.46:0/4012727487 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f81c006bac0 0x7f81c00b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.164+0000 7f81c6200640 1 -- 172.21.15.46:0/4012727487 >> 172.21.15.46:0/4012727487 conn(0x7f81c00b2190 msgr2=0x7f81c01591b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:56.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.164+0000 7f81c6200640 1 -- 172.21.15.46:0/4012727487 shutdown_connections 2024-03-29T08:18:56.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.164+0000 7f81c6200640 1 -- 172.21.15.46:0/4012727487 wait complete. 2024-03-29T08:18:56.186 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:18:56.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.328+0000 7fe740a00640 1 Processor -- start 2024-03-29T08:18:56.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.328+0000 7fe740a00640 1 -- start start 2024-03-29T08:18:56.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.328+0000 7fe740a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe73c098a40 0x7fe73c098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:56.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.328+0000 7fe740a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73c0ce290 con 0x7fe73c0b3510 2024-03-29T08:18:56.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.328+0000 7fe740a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73c0ce570 con 0x7fe73c15a9c0 2024-03-29T08:18:56.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.328+0000 7fe740a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73c099410 con 0x7fe73c098a40 2024-03-29T08:18:56.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.328+0000 7fe739400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe73c098a40 0x7fe73c098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:56.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.328+0000 7fe73a800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe73c0b3510 0x7fe73c0b3940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38636/0 (socket says 172.21.15.46:38636) 2024-03-29T08:18:56.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.328+0000 7fe73a800640 1 -- 172.21.15.46:0/2673259454 learned_addr learned my addr 172.21.15.46:0/2673259454 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:56.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.328+0000 7fe738a00640 1 -- 172.21.15.46:0/2673259454 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440407240 0 0) 0x7fe73c0ce290 con 0x7fe73c0b3510 2024-03-29T08:18:56.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.328+0000 7fe738a00640 1 -- 172.21.15.46:0/2673259454 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe720003530 con 0x7fe73c0b3510 2024-03-29T08:18:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.328+0000 7fe739400640 1 -- 172.21.15.46:0/2673259454 >> v1:172.21.15.73:6789/0 conn(0x7fe73c15a9c0 legacy=0x7fe73c0cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.328+0000 7fe739400640 1 -- 172.21.15.46:0/2673259454 >> v1:172.21.15.46:6789/0 conn(0x7fe73c0b3510 legacy=0x7fe73c0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.328+0000 7fe739400640 1 -- 172.21.15.46:0/2673259454 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73c0ce850 con 0x7fe73c098a40 2024-03-29T08:18:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.332+0000 7fe739400640 1 --2- 172.21.15.46:0/2673259454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73c098a40 0x7fe73c098ed0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe7280097c0 tx=0x7fe72801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90aabb06af10c350 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.332+0000 7fe738a00640 1 -- 172.21.15.46:0/2673259454 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7280ac640 con 0x7fe73c098a40 2024-03-29T08:18:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.332+0000 7fe738a00640 1 -- 172.21.15.46:0/2673259454 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7280ac7a0 con 0x7fe73c098a40 2024-03-29T08:18:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.332+0000 7fe738a00640 1 -- 172.21.15.46:0/2673259454 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7280acb30 con 0x7fe73c098a40 2024-03-29T08:18:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.332+0000 7fe740a00640 1 -- 172.21.15.46:0/2673259454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73c098a40 msgr2=0x7fe73c098ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.332+0000 7fe740a00640 1 --2- 172.21.15.46:0/2673259454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73c098a40 0x7fe73c098ed0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe7280097c0 tx=0x7fe72801c3e0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.332+0000 7fe740a00640 1 -- 172.21.15.46:0/2673259454 shutdown_connections 2024-03-29T08:18:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.332+0000 7fe740a00640 1 --2- 172.21.15.46:0/2673259454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73c098a40 0x7fe73c098ed0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.332+0000 7fe740a00640 1 -- 172.21.15.46:0/2673259454 >> 172.21.15.46:0/2673259454 conn(0x7fe73c0b21b0 msgr2=0x7fe73c0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.332+0000 7fe740a00640 1 -- 172.21.15.46:0/2673259454 shutdown_connections 2024-03-29T08:18:56.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.332+0000 7fe740a00640 1 -- 172.21.15.46:0/2673259454 wait complete. 2024-03-29T08:18:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.332+0000 7fe740a00640 1 Processor -- start 2024-03-29T08:18:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.332+0000 7fe740a00640 1 -- start start 2024-03-29T08:18:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe740a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe73c098a40 0x7fe73c154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe740a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73c0b3510 0x7fe73c153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe740a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe73c15a9c0 0x7fe73c157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe740a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0c7220 con 0x7fe73c0b3510 2024-03-29T08:18:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe740a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0c8760 con 0x7fe73c098a40 2024-03-29T08:18:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe740a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0c8c60 con 0x7fe73c15a9c0 2024-03-29T08:18:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe739400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73c0b3510 0x7fe73c153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:56.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe739400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73c0b3510 0x7fe73c153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50564/0 (socket says 172.21.15.46:50564) 2024-03-29T08:18:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe739400640 1 -- 172.21.15.46:0/1683138932 learned_addr learned my addr 172.21.15.46:0/1683138932 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe739e00640 1 --2- 172.21.15.46:0/1683138932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe73c098a40 0x7fe73c154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe73a800640 1 --2- 172.21.15.46:0/1683138932 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe73c15a9c0 0x7fe73c157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe739400640 1 -- 172.21.15.46:0/1683138932 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe73c15a9c0 msgr2=0x7fe73c157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe739400640 1 --2- 172.21.15.46:0/1683138932 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe73c15a9c0 0x7fe73c157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe739400640 1 -- 172.21.15.46:0/1683138932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe73c098a40 msgr2=0x7fe73c154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe739400640 1 --2- 172.21.15.46:0/1683138932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe73c098a40 0x7fe73c154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe739400640 1 -- 172.21.15.46:0/1683138932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe728009420 con 0x7fe73c0b3510 2024-03-29T08:18:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe739e00640 1 --2- 172.21.15.46:0/1683138932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe73c098a40 0x7fe73c154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:18:56.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe739400640 1 --2- 172.21.15.46:0/1683138932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73c0b3510 0x7fe73c153ad0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe7280af040 tx=0x7fe728019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:56.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe732000640 1 -- 172.21.15.46:0/1683138932 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7280b5610 con 0x7fe73c0b3510 2024-03-29T08:18:56.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe732000640 1 -- 172.21.15.46:0/1683138932 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7280b5770 con 0x7fe73c0b3510 2024-03-29T08:18:56.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe740a00640 1 -- 172.21.15.46:0/1683138932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73c0cdb30 con 0x7fe73c0b3510 2024-03-29T08:18:56.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe732000640 1 -- 172.21.15.46:0/1683138932 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7280b5b00 con 0x7fe73c0b3510 2024-03-29T08:18:56.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe740a00640 1 -- 172.21.15.46:0/1683138932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73c0cd050 con 0x7fe73c0b3510 2024-03-29T08:18:56.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.336+0000 7fe740a00640 1 -- 172.21.15.46:0/1683138932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73c0d0040 con 0x7fe73c0b3510 2024-03-29T08:18:56.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.340+0000 7fe732000640 1 -- 172.21.15.46:0/1683138932 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe7280b3040 con 0x7fe73c0b3510 2024-03-29T08:18:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.340+0000 7fe732000640 1 --2- 172.21.15.46:0/1683138932 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe71c066690 0x7fe71c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.340+0000 7fe732000640 1 -- 172.21.15.46:0/1683138932 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe72811e5a0 con 0x7fe73c0b3510 2024-03-29T08:18:56.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.340+0000 7fe739e00640 1 --2- 172.21.15.46:0/1683138932 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe71c066690 0x7fe71c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:56.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.340+0000 7fe739e00640 1 --2- 172.21.15.46:0/1683138932 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe71c066690 0x7fe71c068b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0b66b0 tx=0x7fe72c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:56.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.344+0000 7fe732000640 1 -- 172.21.15.46:0/1683138932 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe7280eb850 con 0x7fe73c0b3510 2024-03-29T08:18:56.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.520+0000 7fe740a00640 1 -- 172.21.15.46:0/1683138932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe73c09e690 con 0x7fe73c0b3510 2024-03-29T08:18:56.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.520+0000 7fe732000640 1 -- 172.21.15.46:0/1683138932 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe7280eee70 con 0x7fe73c0b3510 2024-03-29T08:18:56.521 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:18:56.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.520+0000 7fe740a00640 1 -- 172.21.15.46:0/1683138932 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe71c066690 msgr2=0x7fe71c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:56.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.520+0000 7fe740a00640 1 --2- 172.21.15.46:0/1683138932 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe71c066690 0x7fe71c068b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe73c0b66b0 tx=0x7fe72c009040 comp rx=0 tx=0).stop 2024-03-29T08:18:56.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.520+0000 7fe740a00640 1 -- 172.21.15.46:0/1683138932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73c0b3510 msgr2=0x7fe73c153ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:56.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.520+0000 7fe740a00640 1 --2- 172.21.15.46:0/1683138932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73c0b3510 0x7fe73c153ad0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe7280af040 tx=0x7fe728019180 comp rx=0 tx=0).stop 2024-03-29T08:18:56.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.520+0000 7fe740a00640 1 -- 172.21.15.46:0/1683138932 shutdown_connections 2024-03-29T08:18:56.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.520+0000 7fe740a00640 1 --2- 172.21.15.46:0/1683138932 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe71c066690 0x7fe71c068b50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.520+0000 7fe740a00640 1 --2- 172.21.15.46:0/1683138932 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe73c15a9c0 0x7fe73c157fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.520+0000 7fe740a00640 1 --2- 172.21.15.46:0/1683138932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe73c0b3510 0x7fe73c153ad0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.520+0000 7fe740a00640 1 --2- 172.21.15.46:0/1683138932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe73c098a40 0x7fe73c154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.520+0000 7fe740a00640 1 -- 172.21.15.46:0/1683138932 >> 172.21.15.46:0/1683138932 conn(0x7fe73c0b21b0 msgr2=0x7fe73c159ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.520+0000 7fe740a00640 1 -- 172.21.15.46:0/1683138932 shutdown_connections 2024-03-29T08:18:56.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.520+0000 7fe740a00640 1 -- 172.21.15.46:0/1683138932 wait complete. 2024-03-29T08:18:56.542 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-03-29T08:18:56.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:56.543 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-03-29T08:18:56.563 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:18:56.563 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:18:56.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:18:56.564 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-29T08:18:56.620 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:18:56.620 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:18:56.630 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:18:56.630 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:18:56.682 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:56.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.820+0000 7fb6df800640 1 Processor -- start 2024-03-29T08:18:56.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.820+0000 7fb6df800640 1 -- start start 2024-03-29T08:18:56.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.820+0000 7fb6df800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb6d81593d0 0x7fb6d806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:56.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.820+0000 7fb6df800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d815cd90 0x7fb6d81597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:56.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.820+0000 7fb6df800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d806d130 con 0x7fb6d815d230 2024-03-29T08:18:56.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.820+0000 7fb6df800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d815b8f0 con 0x7fb6d81593d0 2024-03-29T08:18:56.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.820+0000 7fb6df800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d815ad50 con 0x7fb6d815cd90 2024-03-29T08:18:56.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.824+0000 7fb6dd800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d815cd90 0x7fb6d81597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:56.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.824+0000 7fb6dce00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb6d81593d0 0x7fb6d806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:56.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.824+0000 7fb6cfe00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb6d815d230 0x7fb6d815d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:60992/0 (socket says 172.21.15.46:60992) 2024-03-29T08:18:56.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.824+0000 7fb6cfe00640 1 -- 172.21.15.46:0/4130761985 learned_addr learned my addr 172.21.15.46:0/4130761985 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:56.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.824+0000 7fb6d7e00640 1 -- 172.21.15.46:0/4130761985 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066971979 0 0) 0x7fb6d806d130 con 0x7fb6d815d230 2024-03-29T08:18:56.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.824+0000 7fb6d7e00640 1 -- 172.21.15.46:0/4130761985 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6bc003530 con 0x7fb6d815d230 2024-03-29T08:18:56.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.824+0000 7fb6dce00640 1 -- 172.21.15.46:0/4130761985 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d815cd90 msgr2=0x7fb6d81597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:56.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.824+0000 7fb6dce00640 1 --2- 172.21.15.46:0/4130761985 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d815cd90 0x7fb6d81597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.824+0000 7fb6dce00640 1 -- 172.21.15.46:0/4130761985 >> v1:172.21.15.73:6789/0 conn(0x7fb6d815d230 legacy=0x7fb6d815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:56.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.824+0000 7fb6dce00640 1 -- 172.21.15.46:0/4130761985 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d815afc0 con 0x7fb6d81593d0 2024-03-29T08:18:56.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.824+0000 7fb6dce00640 1 --2- 172.21.15.46:0/4130761985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d81593d0 0x7fb6d806cbf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80097c0 tx=0x7fb6c801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7d5ede99c980bc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:56.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.824+0000 7fb6d7e00640 1 -- 172.21.15.46:0/4130761985 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c80a6040 con 0x7fb6d81593d0 2024-03-29T08:18:56.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.824+0000 7fb6d7e00640 1 -- 172.21.15.46:0/4130761985 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c80aa600 con 0x7fb6d81593d0 2024-03-29T08:18:56.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.824+0000 7fb6d7e00640 1 -- 172.21.15.46:0/4130761985 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c80aa990 con 0x7fb6d81593d0 2024-03-29T08:18:56.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6df800640 1 -- 172.21.15.46:0/4130761985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d81593d0 msgr2=0x7fb6d806cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:56.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6df800640 1 --2- 172.21.15.46:0/4130761985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d81593d0 0x7fb6d806cbf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80097c0 tx=0x7fb6c801c4a0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6df800640 1 -- 172.21.15.46:0/4130761985 shutdown_connections 2024-03-29T08:18:56.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6df800640 1 --2- 172.21.15.46:0/4130761985 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d815cd90 0x7fb6d81597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6df800640 1 --2- 172.21.15.46:0/4130761985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d81593d0 0x7fb6d806cbf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6df800640 1 -- 172.21.15.46:0/4130761985 >> 172.21.15.46:0/4130761985 conn(0x7fb6d8066110 msgr2=0x7fb6d8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:56.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6df800640 1 -- 172.21.15.46:0/4130761985 shutdown_connections 2024-03-29T08:18:56.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6df800640 1 -- 172.21.15.46:0/4130761985 wait complete. 2024-03-29T08:18:56.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6df800640 1 Processor -- start 2024-03-29T08:18:56.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6df800640 1 -- start start 2024-03-29T08:18:56.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6df800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d81593d0 0x7fb6d80a8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:56.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6df800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d815cd90 0x7fb6d81566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:56.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6df800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d815d230 0x7fb6d8162ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:56.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6df800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d8156c10 con 0x7fb6d815cd90 2024-03-29T08:18:56.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6df800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d81635e0 con 0x7fb6d815d230 2024-03-29T08:18:56.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6df800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d80a7540 con 0x7fb6d81593d0 2024-03-29T08:18:56.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d815cd90 0x7fb6d81566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:56.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6cfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d815cd90 0x7fb6d81566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55124/0 (socket says 172.21.15.46:55124) 2024-03-29T08:18:56.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6cfe00640 1 -- 172.21.15.46:0/1212251788 learned_addr learned my addr 172.21.15.46:0/1212251788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:56.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6dd800640 1 --2- 172.21.15.46:0/1212251788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d815d230 0x7fb6d8162ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:56.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6dce00640 1 --2- 172.21.15.46:0/1212251788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d81593d0 0x7fb6d80a8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:56.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6cfe00640 1 -- 172.21.15.46:0/1212251788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d81593d0 msgr2=0x7fb6d80a8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:56.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6cfe00640 1 --2- 172.21.15.46:0/1212251788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d81593d0 0x7fb6d80a8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6cfe00640 1 -- 172.21.15.46:0/1212251788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d815d230 msgr2=0x7fb6d8162ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:56.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6cfe00640 1 --2- 172.21.15.46:0/1212251788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d815d230 0x7fb6d8162ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:56.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6cfe00640 1 -- 172.21.15.46:0/1212251788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c8009420 con 0x7fb6d815cd90 2024-03-29T08:18:56.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6dce00640 1 --2- 172.21.15.46:0/1212251788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d81593d0 0x7fb6d80a8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:56.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.828+0000 7fb6dd800640 1 --2- 172.21.15.46:0/1212251788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d815d230 0x7fb6d8162ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:56.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.832+0000 7fb6cfe00640 1 --2- 172.21.15.46:0/1212251788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d815cd90 0x7fb6d81566d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000c450 tx=0x7fb6c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:56.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.832+0000 7fb6d5600640 1 -- 172.21.15.46:0/1212251788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0005cf0 con 0x7fb6d815cd90 2024-03-29T08:18:56.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.832+0000 7fb6d5600640 1 -- 172.21.15.46:0/1212251788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c0005e50 con 0x7fb6d815cd90 2024-03-29T08:18:56.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.832+0000 7fb6d5600640 1 -- 172.21.15.46:0/1212251788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0007290 con 0x7fb6d815cd90 2024-03-29T08:18:56.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.832+0000 7fb6df800640 1 -- 172.21.15.46:0/1212251788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d80a7820 con 0x7fb6d815cd90 2024-03-29T08:18:56.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.832+0000 7fb6df800640 1 -- 172.21.15.46:0/1212251788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d81638c0 con 0x7fb6d815cd90 2024-03-29T08:18:56.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.832+0000 7fb6df800640 1 -- 172.21.15.46:0/1212251788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d8066c90 con 0x7fb6d815cd90 2024-03-29T08:18:56.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.832+0000 7fb6d5600640 1 -- 172.21.15.46:0/1212251788 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb6c00073f0 con 0x7fb6d815cd90 2024-03-29T08:18:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.832+0000 7fb6d5600640 1 --2- 172.21.15.46:0/1212251788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6b40666c0 0x7fb6b4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.832+0000 7fb6d5600640 1 -- 172.21.15.46:0/1212251788 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb6c00836d0 con 0x7fb6d815cd90 2024-03-29T08:18:56.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.832+0000 7fb6dce00640 1 --2- 172.21.15.46:0/1212251788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6b40666c0 0x7fb6b4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:56.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.836+0000 7fb6dce00640 1 --2- 172.21.15.46:0/1212251788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6b40666c0 0x7fb6b4068b80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80ae000 tx=0x7fb6c8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:56.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:56.836+0000 7fb6d5600640 1 -- 172.21.15.46:0/1212251788 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb6c0050ab0 con 0x7fb6d815cd90 2024-03-29T08:18:57.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.000+0000 7fb6df800640 1 -- 172.21.15.46:0/1212251788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6d80a7cd0 con 0x7fb6d815cd90 2024-03-29T08:18:57.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.000+0000 7fb6d5600640 1 -- 172.21.15.46:0/1212251788 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16042 (secure 0 0 0) 0x7fb6c0017090 con 0x7fb6d815cd90 2024-03-29T08:18:57.003 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:57.004 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:56.021693+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.004+0000 7fb6df800640 1 -- 172.21.15.46:0/1212251788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6b40666c0 msgr2=0x7fb6b4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.004+0000 7fb6df800640 1 --2- 172.21.15.46:0/1212251788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6b40666c0 0x7fb6b4068b80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb6c80ae000 tx=0x7fb6c8008040 comp rx=0 tx=0).stop 2024-03-29T08:18:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.004+0000 7fb6df800640 1 -- 172.21.15.46:0/1212251788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d815cd90 msgr2=0x7fb6d81566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.004+0000 7fb6df800640 1 --2- 172.21.15.46:0/1212251788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d815cd90 0x7fb6d81566d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000c450 tx=0x7fb6c000c910 comp rx=0 tx=0).stop 2024-03-29T08:18:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.004+0000 7fb6df800640 1 -- 172.21.15.46:0/1212251788 shutdown_connections 2024-03-29T08:18:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.004+0000 7fb6df800640 1 --2- 172.21.15.46:0/1212251788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6b40666c0 0x7fb6b4068b80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.004+0000 7fb6df800640 1 --2- 172.21.15.46:0/1212251788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d815d230 0x7fb6d8162ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.004+0000 7fb6df800640 1 --2- 172.21.15.46:0/1212251788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d815cd90 0x7fb6d81566d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.004+0000 7fb6df800640 1 --2- 172.21.15.46:0/1212251788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d81593d0 0x7fb6d80a8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.004+0000 7fb6df800640 1 -- 172.21.15.46:0/1212251788 >> 172.21.15.46:0/1212251788 conn(0x7fb6d8066110 msgr2=0x7fb6d80cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:57.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.004+0000 7fb6df800640 1 -- 172.21.15.46:0/1212251788 shutdown_connections 2024-03-29T08:18:57.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.004+0000 7fb6df800640 1 -- 172.21.15.46:0/1212251788 wait complete. 2024-03-29T08:18:57.024 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:18:11.177842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:18:12.308252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:18:57.025 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:18:57.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f1312600640 1 Processor -- start 2024-03-29T08:18:57.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f1312600640 1 -- start start 2024-03-29T08:18:57.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f1312600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f130c0cea30 0x7f130c1606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f1312600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f130c15cd30 0x7f130c160be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f1312600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130c0cfb20 con 0x7f130c15c950 2024-03-29T08:18:57.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f1312600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f130c0cef10 con 0x7f130c15cd30 2024-03-29T08:18:57.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f1312600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130c0cf1f0 con 0x7f130c0cea30 2024-03-29T08:18:57.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f130b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f130c15c950 0x7f130c06bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54690/0 (socket says 172.21.15.46:54690) 2024-03-29T08:18:57.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f130b400640 1 -- 172.21.15.46:0/416888596 learned_addr learned my addr 172.21.15.46:0/416888596 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:57.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f130be00640 1 --2- 172.21.15.46:0/416888596 >> v2:172.21.15.73:3300/0 conn(0x7f130c15cd30 0x7f130c160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f1303e00640 1 --2- 172.21.15.46:0/416888596 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f130c0cea30 0x7f130c1606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f130aa00640 1 -- 172.21.15.46:0/416888596 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3352366445 0 0) 0x7f130c0cfb20 con 0x7f130c15c950 2024-03-29T08:18:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f130aa00640 1 -- 172.21.15.46:0/416888596 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12f0003530 con 0x7f130c15c950 2024-03-29T08:18:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f130be00640 1 -- 172.21.15.46:0/416888596 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f130c0cea30 msgr2=0x7f130c1606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f130be00640 1 --2- 172.21.15.46:0/416888596 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f130c0cea30 0x7f130c1606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f130be00640 1 -- 172.21.15.46:0/416888596 >> v1:172.21.15.46:6789/0 conn(0x7f130c15c950 legacy=0x7f130c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f130be00640 1 -- 172.21.15.46:0/416888596 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c0cf4d0 con 0x7f130c15cd30 2024-03-29T08:18:57.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.168+0000 7f1303e00640 1 --2- 172.21.15.46:0/416888596 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f130c0cea30 0x7f130c1606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.172+0000 7f130be00640 1 --2- 172.21.15.46:0/416888596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f130c15cd30 0x7f130c160be0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1304009d90 tx=0x7f130401f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ffd64462990db69 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.172+0000 7f130aa00640 1 -- 172.21.15.46:0/416888596 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13040af600 con 0x7f130c15cd30 2024-03-29T08:18:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.172+0000 7f130aa00640 1 -- 172.21.15.46:0/416888596 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13040ab050 con 0x7f130c15cd30 2024-03-29T08:18:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.172+0000 7f130aa00640 1 -- 172.21.15.46:0/416888596 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13040af960 con 0x7f130c15cd30 2024-03-29T08:18:57.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.172+0000 7f1312600640 1 -- 172.21.15.46:0/416888596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f130c15cd30 msgr2=0x7f130c160be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.172+0000 7f1312600640 1 --2- 172.21.15.46:0/416888596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f130c15cd30 0x7f130c160be0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1304009d90 tx=0x7f130401f150 comp rx=0 tx=0).stop 2024-03-29T08:18:57.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.172+0000 7f1312600640 1 -- 172.21.15.46:0/416888596 shutdown_connections 2024-03-29T08:18:57.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.172+0000 7f1312600640 1 --2- 172.21.15.46:0/416888596 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f130c0cea30 0x7f130c1606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.172+0000 7f1312600640 1 --2- 172.21.15.46:0/416888596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f130c15cd30 0x7f130c160be0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.172+0000 7f1312600640 1 -- 172.21.15.46:0/416888596 >> 172.21.15.46:0/416888596 conn(0x7f130c0b21b0 msgr2=0x7f130c0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:57.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.172+0000 7f1312600640 1 -- 172.21.15.46:0/416888596 shutdown_connections 2024-03-29T08:18:57.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.172+0000 7f1312600640 1 -- 172.21.15.46:0/416888596 wait complete. 2024-03-29T08:18:57.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f1312600640 1 Processor -- start 2024-03-29T08:18:57.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f1312600640 1 -- start start 2024-03-29T08:18:57.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f1312600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f130c0cea30 0x7f130c1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f1312600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f130c15c950 0x7f130c15f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f1312600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f130c15cd30 0x7f130c0b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f1312600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c159c20 con 0x7f130c0cea30 2024-03-29T08:18:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f1312600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130c09c720 con 0x7f130c15c950 2024-03-29T08:18:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f1312600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f130c09ca00 con 0x7f130c15cd30 2024-03-29T08:18:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f130b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f130c0cea30 0x7f130c1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f130b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f130c0cea30 0x7f130c1596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55132/0 (socket says 172.21.15.46:55132) 2024-03-29T08:18:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f130b400640 1 -- 172.21.15.46:0/2982434732 learned_addr learned my addr 172.21.15.46:0/2982434732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f1303e00640 1 --2- 172.21.15.46:0/2982434732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f130c15c950 0x7f130c15f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f130be00640 1 --2- 172.21.15.46:0/2982434732 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f130c15cd30 0x7f130c0b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f130b400640 1 -- 172.21.15.46:0/2982434732 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f130c15cd30 msgr2=0x7f130c0b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f130b400640 1 --2- 172.21.15.46:0/2982434732 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f130c15cd30 0x7f130c0b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f130b400640 1 -- 172.21.15.46:0/2982434732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f130c15c950 msgr2=0x7f130c15f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f130b400640 1 --2- 172.21.15.46:0/2982434732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f130c15c950 0x7f130c15f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f130b400640 1 -- 172.21.15.46:0/2982434732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f4009420 con 0x7f130c0cea30 2024-03-29T08:18:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f130b400640 1 --2- 172.21.15.46:0/2982434732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f130c0cea30 0x7f130c1596e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f12fc00c450 tx=0x7f12fc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f1303400640 1 -- 172.21.15.46:0/2982434732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc005cf0 con 0x7f130c0cea30 2024-03-29T08:18:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f1303400640 1 -- 172.21.15.46:0/2982434732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12fc005e50 con 0x7f130c0cea30 2024-03-29T08:18:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f1303400640 1 -- 172.21.15.46:0/2982434732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12fc007290 con 0x7f130c0cea30 2024-03-29T08:18:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f1312600640 1 -- 172.21.15.46:0/2982434732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1304009a70 con 0x7f130c0cea30 2024-03-29T08:18:57.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.176+0000 7f1312600640 1 -- 172.21.15.46:0/2982434732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f130c06c470 con 0x7f130c0cea30 2024-03-29T08:18:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.180+0000 7f1312600640 1 -- 172.21.15.46:0/2982434732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130c06c270 con 0x7f130c0cea30 2024-03-29T08:18:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.180+0000 7f1303400640 1 -- 172.21.15.46:0/2982434732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f12fc0073f0 con 0x7f130c0cea30 2024-03-29T08:18:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.180+0000 7f1303400640 1 --2- 172.21.15.46:0/2982434732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12e8066690 0x7f12e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.180+0000 7f1303400640 1 -- 172.21.15.46:0/2982434732 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f12fc083840 con 0x7f130c0cea30 2024-03-29T08:18:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.180+0000 7f1303e00640 1 --2- 172.21.15.46:0/2982434732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12e8066690 0x7f12e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.180+0000 7f1303e00640 1 --2- 172.21.15.46:0/2982434732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12e8066690 0x7f12e8068b50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f130c15b840 tx=0x7f12f4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:57.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.184+0000 7f1303400640 1 -- 172.21.15.46:0/2982434732 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f12fc050af0 con 0x7f130c0cea30 2024-03-29T08:18:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.348+0000 7f1312600640 1 -- 172.21.15.46:0/2982434732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f130c09e690 con 0x7f130c0cea30 2024-03-29T08:18:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.348+0000 7f1303400640 1 -- 172.21.15.46:0/2982434732 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f12fc017020 con 0x7f130c0cea30 2024-03-29T08:18:57.348 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:18:57.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.348+0000 7f1312600640 1 -- 172.21.15.46:0/2982434732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12e8066690 msgr2=0x7f12e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.348+0000 7f1312600640 1 --2- 172.21.15.46:0/2982434732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12e8066690 0x7f12e8068b50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f130c15b840 tx=0x7f12f4008040 comp rx=0 tx=0).stop 2024-03-29T08:18:57.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.348+0000 7f1312600640 1 -- 172.21.15.46:0/2982434732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f130c0cea30 msgr2=0x7f130c1596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.348+0000 7f1312600640 1 --2- 172.21.15.46:0/2982434732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f130c0cea30 0x7f130c1596e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f12fc00c450 tx=0x7f12fc00c910 comp rx=0 tx=0).stop 2024-03-29T08:18:57.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.348+0000 7f1312600640 1 -- 172.21.15.46:0/2982434732 shutdown_connections 2024-03-29T08:18:57.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.348+0000 7f1312600640 1 --2- 172.21.15.46:0/2982434732 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12e8066690 0x7f12e8068b50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.348+0000 7f1312600640 1 --2- 172.21.15.46:0/2982434732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f130c15c950 0x7f130c15f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.348+0000 7f1312600640 1 --2- 172.21.15.46:0/2982434732 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f130c15cd30 0x7f130c0b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.348+0000 7f1312600640 1 --2- 172.21.15.46:0/2982434732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f130c0cea30 0x7f130c1596e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.348+0000 7f1312600640 1 -- 172.21.15.46:0/2982434732 >> 172.21.15.46:0/2982434732 conn(0x7f130c0b21b0 msgr2=0x7f130c15ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.348+0000 7f1312600640 1 -- 172.21.15.46:0/2982434732 shutdown_connections 2024-03-29T08:18:57.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.348+0000 7f1312600640 1 -- 172.21.15.46:0/2982434732 wait complete. 2024-03-29T08:18:57.369 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:18:57.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.512+0000 7f230be00640 1 Processor -- start 2024-03-29T08:18:57.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.512+0000 7f230be00640 1 -- start start 2024-03-29T08:18:57.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.512+0000 7f230be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f230c15a9f0 0x7f230c0ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.512+0000 7f230be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f230c0cf590 con 0x7f230c0cc390 2024-03-29T08:18:57.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.512+0000 7f230be00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f230c06e130 con 0x7f230c0ccf60 2024-03-29T08:18:57.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.512+0000 7f230be00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230c06e410 con 0x7f230c15a9f0 2024-03-29T08:18:57.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.512+0000 7f230b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f230c0cc390 0x7f230c06f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54704/0 (socket says 172.21.15.46:54704) 2024-03-29T08:18:57.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.512+0000 7f230b400640 1 -- 172.21.15.46:0/3573631997 learned_addr learned my addr 172.21.15.46:0/3573631997 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.512+0000 7f230aa00640 1 --2- 172.21.15.46:0/3573631997 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f230c15a9f0 0x7f230c0ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.512+0000 7f230a000640 1 -- 172.21.15.46:0/3573631997 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4264109337 0 0) 0x7f230c0cf590 con 0x7f230c0cc390 2024-03-29T08:18:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.512+0000 7f230a000640 1 -- 172.21.15.46:0/3573631997 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22f0003530 con 0x7f230c0cc390 2024-03-29T08:18:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.512+0000 7f230a000640 1 -- 172.21.15.46:0/3573631997 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547246588 0 0) 0x7f230c06e130 con 0x7f230c0ccf60 2024-03-29T08:18:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.512+0000 7f230a000640 1 -- 172.21.15.46:0/3573631997 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f230c0cf590 con 0x7f230c0ccf60 2024-03-29T08:18:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.512+0000 7f230aa00640 1 -- 172.21.15.46:0/3573631997 >> v1:172.21.15.73:6789/0 conn(0x7f230c0ccf60 legacy=0x7f230c06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.512+0000 7f230aa00640 1 -- 172.21.15.46:0/3573631997 >> v1:172.21.15.46:6789/0 conn(0x7f230c0cc390 legacy=0x7f230c06f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.512+0000 7f230aa00640 1 -- 172.21.15.46:0/3573631997 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230c06e6f0 con 0x7f230c15a9f0 2024-03-29T08:18:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.516+0000 7f230aa00640 1 --2- 172.21.15.46:0/3573631997 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f230c15a9f0 0x7f230c0ced40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0097f0 tx=0x7f22fc01ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=358d3ec141f1e2aa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.516+0000 7f230a000640 1 -- 172.21.15.46:0/3573631997 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc0a89d0 con 0x7f230c15a9f0 2024-03-29T08:18:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.516+0000 7f230a000640 1 -- 172.21.15.46:0/3573631997 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22fc0a8b30 con 0x7f230c15a9f0 2024-03-29T08:18:57.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.516+0000 7f230a000640 1 -- 172.21.15.46:0/3573631997 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc0b0440 con 0x7f230c15a9f0 2024-03-29T08:18:57.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.516+0000 7f230be00640 1 -- 172.21.15.46:0/3573631997 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f230c15a9f0 msgr2=0x7f230c0ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.516+0000 7f230be00640 1 --2- 172.21.15.46:0/3573631997 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f230c15a9f0 0x7f230c0ced40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0097f0 tx=0x7f22fc01ecd0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.516+0000 7f230be00640 1 -- 172.21.15.46:0/3573631997 shutdown_connections 2024-03-29T08:18:57.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.516+0000 7f230be00640 1 --2- 172.21.15.46:0/3573631997 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f230c15a9f0 0x7f230c0ced40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.516+0000 7f230be00640 1 -- 172.21.15.46:0/3573631997 >> 172.21.15.46:0/3573631997 conn(0x7f230c0d0550 msgr2=0x7f230c0d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:57.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.516+0000 7f230be00640 1 -- 172.21.15.46:0/3573631997 shutdown_connections 2024-03-29T08:18:57.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.516+0000 7f230be00640 1 -- 172.21.15.46:0/3573631997 wait complete. 2024-03-29T08:18:57.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230be00640 1 Processor -- start 2024-03-29T08:18:57.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230be00640 1 -- start start 2024-03-29T08:18:57.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f230c0cc390 0x7f230c15c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f230c0ccf60 0x7f230c14ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230c15a9f0 0x7f230c0b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c14ef90 con 0x7f230c15a9f0 2024-03-29T08:18:57.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c0c5b00 con 0x7f230c0ccf60 2024-03-29T08:18:57.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230be00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230c0c5de0 con 0x7f230c0cc390 2024-03-29T08:18:57.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230c15a9f0 0x7f230c0b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230c15a9f0 0x7f230c0b5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55144/0 (socket says 172.21.15.46:55144) 2024-03-29T08:18:57.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230b400640 1 -- 172.21.15.46:0/2647523183 learned_addr learned my addr 172.21.15.46:0/2647523183 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:57.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230aa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f230c0cc390 0x7f230c15c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f2303e00640 1 --2- 172.21.15.46:0/2647523183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f230c0ccf60 0x7f230c14ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230b400640 1 -- 172.21.15.46:0/2647523183 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f230c0cc390 msgr2=0x7f230c15c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230b400640 1 --2- 172.21.15.46:0/2647523183 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f230c0cc390 0x7f230c15c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230b400640 1 -- 172.21.15.46:0/2647523183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f230c0ccf60 msgr2=0x7f230c14ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230b400640 1 --2- 172.21.15.46:0/2647523183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f230c0ccf60 0x7f230c14ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230b400640 1 -- 172.21.15.46:0/2647523183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22fc009420 con 0x7f230c15a9f0 2024-03-29T08:18:57.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230b400640 1 --2- 172.21.15.46:0/2647523183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230c15a9f0 0x7f230c0b5320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f230400caa0 tx=0x7f230400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:57.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f2302a00640 1 -- 172.21.15.46:0/2647523183 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230400ac00 con 0x7f230c15a9f0 2024-03-29T08:18:57.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f2302a00640 1 -- 172.21.15.46:0/2647523183 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f230400ad80 con 0x7f230c15a9f0 2024-03-29T08:18:57.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f2302a00640 1 -- 172.21.15.46:0/2647523183 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2304012640 con 0x7f230c15a9f0 2024-03-29T08:18:57.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230be00640 1 -- 172.21.15.46:0/2647523183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230c0c60c0 con 0x7f230c15a9f0 2024-03-29T08:18:57.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230be00640 1 -- 172.21.15.46:0/2647523183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230c0c6680 con 0x7f230c15a9f0 2024-03-29T08:18:57.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.520+0000 7f230be00640 1 -- 172.21.15.46:0/2647523183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230c0c68a0 con 0x7f230c15a9f0 2024-03-29T08:18:57.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.524+0000 7f2302a00640 1 -- 172.21.15.46:0/2647523183 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2304007070 con 0x7f230c15a9f0 2024-03-29T08:18:57.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.524+0000 7f2302a00640 1 --2- 172.21.15.46:0/2647523183 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22e80666e0 0x7f22e8068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.524+0000 7f2302a00640 1 -- 172.21.15.46:0/2647523183 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f23040840f0 con 0x7f230c15a9f0 2024-03-29T08:18:57.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.524+0000 7f230aa00640 1 --2- 172.21.15.46:0/2647523183 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22e80666e0 0x7f22e8068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.524+0000 7f230aa00640 1 --2- 172.21.15.46:0/2647523183 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22e80666e0 0x7f22e8068ba0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0b2000 tx=0x7f22fc0aa070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:57.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.528+0000 7f2302a00640 1 -- 172.21.15.46:0/2647523183 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f23040514d0 con 0x7f230c15a9f0 2024-03-29T08:18:57.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.688+0000 7f230be00640 1 -- 172.21.15.46:0/2647523183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f230c04e040 con 0x7f230c15a9f0 2024-03-29T08:18:57.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.692+0000 7f2302a00640 1 -- 172.21.15.46:0/2647523183 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2304054af0 con 0x7f230c15a9f0 2024-03-29T08:18:57.692 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:18:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.692+0000 7f230be00640 1 -- 172.21.15.46:0/2647523183 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22e80666e0 msgr2=0x7f22e8068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.692+0000 7f230be00640 1 --2- 172.21.15.46:0/2647523183 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22e80666e0 0x7f22e8068ba0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0b2000 tx=0x7f22fc0aa070 comp rx=0 tx=0).stop 2024-03-29T08:18:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.692+0000 7f230be00640 1 -- 172.21.15.46:0/2647523183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230c15a9f0 msgr2=0x7f230c0b5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.692+0000 7f230be00640 1 --2- 172.21.15.46:0/2647523183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230c15a9f0 0x7f230c0b5320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f230400caa0 tx=0x7f230400cf60 comp rx=0 tx=0).stop 2024-03-29T08:18:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.692+0000 7f230be00640 1 -- 172.21.15.46:0/2647523183 shutdown_connections 2024-03-29T08:18:57.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.692+0000 7f230be00640 1 --2- 172.21.15.46:0/2647523183 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22e80666e0 0x7f22e8068ba0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.692+0000 7f230be00640 1 --2- 172.21.15.46:0/2647523183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f230c0ccf60 0x7f230c14ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.692+0000 7f230be00640 1 --2- 172.21.15.46:0/2647523183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f230c15a9f0 0x7f230c0b5320 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.692+0000 7f230be00640 1 --2- 172.21.15.46:0/2647523183 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f230c0cc390 0x7f230c15c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.692+0000 7f230be00640 1 -- 172.21.15.46:0/2647523183 >> 172.21.15.46:0/2647523183 conn(0x7f230c0d0550 msgr2=0x7f230c0ad810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:57.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.692+0000 7f230be00640 1 -- 172.21.15.46:0/2647523183 shutdown_connections 2024-03-29T08:18:57.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.692+0000 7f230be00640 1 -- 172.21.15.46:0/2647523183 wait complete. 2024-03-29T08:18:57.713 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:18:57.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.852+0000 7efec6400640 1 Processor -- start 2024-03-29T08:18:57.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.852+0000 7efec6400640 1 -- start start 2024-03-29T08:18:57.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efec6400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efec015df90 0x7efec00ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efec6400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efec00cbea0 0x7efec00ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efec6400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec015d260 con 0x7efec0142a50 2024-03-29T08:18:57.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efec6400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec00aed00 con 0x7efec00cbea0 2024-03-29T08:18:57.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efec6400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec00aefd0 con 0x7efec015df90 2024-03-29T08:18:57.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebf400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efec0142a50 0x7efec00cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54708/0 (socket says 172.21.15.46:54708) 2024-03-29T08:18:57.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebf400640 1 -- 172.21.15.46:0/1779465963 learned_addr learned my addr 172.21.15.46:0/1779465963 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efeb6a00640 1 --2- 172.21.15.46:0/1779465963 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efec015df90 0x7efec00ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebfe00640 1 --2- 172.21.15.46:0/1779465963 >> v2:172.21.15.73:3300/0 conn(0x7efec00cbea0 0x7efec00ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebea00640 1 -- 172.21.15.46:0/1779465963 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100541954 0 0) 0x7efec015d260 con 0x7efec0142a50 2024-03-29T08:18:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebea00640 1 -- 172.21.15.46:0/1779465963 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efea4003530 con 0x7efec0142a50 2024-03-29T08:18:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebea00640 1 -- 172.21.15.46:0/1779465963 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1231811490 0 0) 0x7efea4003530 con 0x7efec0142a50 2024-03-29T08:18:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebea00640 1 -- 172.21.15.46:0/1779465963 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efec015df90 msgr2=0x7efec00ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebea00640 1 --2- 172.21.15.46:0/1779465963 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efec015df90 0x7efec00ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebea00640 1 -- 172.21.15.46:0/1779465963 >> v2:172.21.15.73:3300/0 conn(0x7efec00cbea0 msgr2=0x7efec00ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebea00640 1 --2- 172.21.15.46:0/1779465963 >> v2:172.21.15.73:3300/0 conn(0x7efec00cbea0 0x7efec00ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebea00640 1 -- 172.21.15.46:0/1779465963 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec015d4b0 con 0x7efec0142a50 2024-03-29T08:18:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebea00640 1 -- 172.21.15.46:0/1779465963 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7efeb0001680 con 0x7efec0142a50 2024-03-29T08:18:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebfe00640 1 --2- 172.21.15.46:0/1779465963 >> v2:172.21.15.73:3300/0 conn(0x7efec00cbea0 0x7efec00ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebea00640 1 -- 172.21.15.46:0/1779465963 >> v1:172.21.15.46:6789/0 conn(0x7efec0142a50 legacy=0x7efec00cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebea00640 1 --2- 172.21.15.46:0/1779465963 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efec015df90 0x7efea4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebea00640 1 --2- 172.21.15.46:0/1779465963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efea4003f00 0x7efea40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebea00640 1 --2- 172.21.15.46:0/1779465963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea4004d10 0x7efea4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebea00640 1 -- 172.21.15.46:0/1779465963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec00aed00 con 0x7efea4004d10 2024-03-29T08:18:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebea00640 1 -- 172.21.15.46:0/1779465963 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec00aefd0 con 0x7efea4003f00 2024-03-29T08:18:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebea00640 1 -- 172.21.15.46:0/1779465963 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea400dbf0 con 0x7efec015df90 2024-03-29T08:18:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebfe00640 1 --2- 172.21.15.46:0/1779465963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea4004d10 0x7efea4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebf400640 1 --2- 172.21.15.46:0/1779465963 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efec015df90 0x7efea4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efeb6a00640 1 --2- 172.21.15.46:0/1779465963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efea4003f00 0x7efea40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebfe00640 1 -- 172.21.15.46:0/1779465963 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efec015df90 msgr2=0x7efea4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:57.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebfe00640 1 --2- 172.21.15.46:0/1779465963 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efec015df90 0x7efea4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebfe00640 1 -- 172.21.15.46:0/1779465963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efea4003f00 msgr2=0x7efea40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:57.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebfe00640 1 --2- 172.21.15.46:0/1779465963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efea4003f00 0x7efea40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.856+0000 7efebfe00640 1 -- 172.21.15.46:0/1779465963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea4003530 con 0x7efea4004d10 2024-03-29T08:18:57.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.860+0000 7efebfe00640 1 --2- 172.21.15.46:0/1779465963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea4004d10 0x7efea4005190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efeb800e350 tx=0x7efeb801ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed15893e8d7a7a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:57.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.860+0000 7efebea00640 1 -- 172.21.15.46:0/1779465963 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb80a5e50 con 0x7efea4004d10 2024-03-29T08:18:57.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.860+0000 7efebea00640 1 -- 172.21.15.46:0/1779465963 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb800f040 con 0x7efea4004d10 2024-03-29T08:18:57.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.860+0000 7efebea00640 1 -- 172.21.15.46:0/1779465963 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb8003ba0 con 0x7efea4004d10 2024-03-29T08:18:57.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.860+0000 7efec6400640 1 -- 172.21.15.46:0/1779465963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea4004d10 msgr2=0x7efea4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.860+0000 7efec6400640 1 --2- 172.21.15.46:0/1779465963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea4004d10 0x7efea4005190 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efeb800e350 tx=0x7efeb801ce40 comp rx=0 tx=0).stop 2024-03-29T08:18:57.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.860+0000 7efebfe00640 1 -- 172.21.15.46:0/1779465963 reap_dead start 2024-03-29T08:18:57.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.860+0000 7efec6400640 1 -- 172.21.15.46:0/1779465963 shutdown_connections 2024-03-29T08:18:57.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.860+0000 7efec6400640 1 -- 172.21.15.46:0/1779465963 >> 172.21.15.46:0/1779465963 conn(0x7efec006a390 msgr2=0x7efec00cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:57.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.860+0000 7efec6400640 1 -- 172.21.15.46:0/1779465963 shutdown_connections 2024-03-29T08:18:57.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.860+0000 7efec6400640 1 -- 172.21.15.46:0/1779465963 wait complete. 2024-03-29T08:18:57.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efec6400640 1 Processor -- start 2024-03-29T08:18:57.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efec6400640 1 -- start start 2024-03-29T08:18:57.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efec6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea4004d10 0x7efec013a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efec6400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efec0158e90 0x7efec015beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efec6400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efec01f24a0 0x7efec01f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efec6400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec013e3f0 con 0x7efea4004d10 2024-03-29T08:18:57.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efec6400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec013ace0 con 0x7efec0158e90 2024-03-29T08:18:57.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efec6400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec015c3f0 con 0x7efec01f24a0 2024-03-29T08:18:57.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efebf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea4004d10 0x7efec013a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efebf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea4004d10 0x7efec013a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55160/0 (socket says 172.21.15.46:55160) 2024-03-29T08:18:57.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efebf400640 1 -- 172.21.15.46:0/3713505109 learned_addr learned my addr 172.21.15.46:0/3713505109 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:57.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efeb6a00640 1 --2- 172.21.15.46:0/3713505109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efec0158e90 0x7efec015beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efebfe00640 1 --2- 172.21.15.46:0/3713505109 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efec01f24a0 0x7efec01f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efebf400640 1 -- 172.21.15.46:0/3713505109 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efec01f24a0 msgr2=0x7efec01f2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efebf400640 1 --2- 172.21.15.46:0/3713505109 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efec01f24a0 0x7efec01f2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efebf400640 1 -- 172.21.15.46:0/3713505109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efec0158e90 msgr2=0x7efec015beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:57.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efebf400640 1 --2- 172.21.15.46:0/3713505109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efec0158e90 0x7efec015beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:57.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efebf400640 1 -- 172.21.15.46:0/3713505109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec013e600 con 0x7efea4004d10 2024-03-29T08:18:57.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efebf400640 1 --2- 172.21.15.46:0/3713505109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea4004d10 0x7efec013a7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efeb0001ab0 tx=0x7efeb0012c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:57.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efeb7e00640 1 -- 172.21.15.46:0/3713505109 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0010990 con 0x7efea4004d10 2024-03-29T08:18:57.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efeb7e00640 1 -- 172.21.15.46:0/3713505109 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb0010af0 con 0x7efea4004d10 2024-03-29T08:18:57.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efeb7e00640 1 -- 172.21.15.46:0/3713505109 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb0010ea0 con 0x7efea4004d10 2024-03-29T08:18:57.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efec6400640 1 -- 172.21.15.46:0/3713505109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec013e8e0 con 0x7efea4004d10 2024-03-29T08:18:57.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efec6400640 1 -- 172.21.15.46:0/3713505109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec013eea0 con 0x7efea4004d10 2024-03-29T08:18:57.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.864+0000 7efec6400640 1 -- 172.21.15.46:0/3713505109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec0071a70 con 0x7efea4004d10 2024-03-29T08:18:57.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.868+0000 7efeb7e00640 1 -- 172.21.15.46:0/3713505109 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efeb0016480 con 0x7efea4004d10 2024-03-29T08:18:57.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.868+0000 7efeb7e00640 1 --2- 172.21.15.46:0/3713505109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe9c0666c0 0x7efe9c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:57.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.868+0000 7efeb7e00640 1 -- 172.21.15.46:0/3713505109 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7efeb00875e0 con 0x7efea4004d10 2024-03-29T08:18:57.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.868+0000 7efeb6a00640 1 --2- 172.21.15.46:0/3713505109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe9c0666c0 0x7efe9c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:57.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.868+0000 7efeb6a00640 1 --2- 172.21.15.46:0/3713505109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe9c0666c0 0x7efe9c068b80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efec015b430 tx=0x7efea8009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:57.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:57.872+0000 7efeb7e00640 1 -- 172.21.15.46:0/3713505109 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efeb0054890 con 0x7efea4004d10 2024-03-29T08:18:58.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.036+0000 7efec6400640 1 -- 172.21.15.46:0/3713505109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efec004e120 con 0x7efea4004d10 2024-03-29T08:18:58.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.036+0000 7efeb7e00640 1 -- 172.21.15.46:0/3713505109 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7efeb0057eb0 con 0x7efea4004d10 2024-03-29T08:18:58.036 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:18:58.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.036+0000 7efec6400640 1 -- 172.21.15.46:0/3713505109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe9c0666c0 msgr2=0x7efe9c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.036+0000 7efec6400640 1 --2- 172.21.15.46:0/3713505109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe9c0666c0 0x7efe9c068b80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efec015b430 tx=0x7efea8009190 comp rx=0 tx=0).stop 2024-03-29T08:18:58.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.036+0000 7efec6400640 1 -- 172.21.15.46:0/3713505109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea4004d10 msgr2=0x7efec013a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.036+0000 7efec6400640 1 --2- 172.21.15.46:0/3713505109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea4004d10 0x7efec013a7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7efeb0001ab0 tx=0x7efeb0012c50 comp rx=0 tx=0).stop 2024-03-29T08:18:58.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.036+0000 7efec6400640 1 -- 172.21.15.46:0/3713505109 shutdown_connections 2024-03-29T08:18:58.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.036+0000 7efec6400640 1 --2- 172.21.15.46:0/3713505109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe9c0666c0 0x7efe9c068b80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.036+0000 7efec6400640 1 --2- 172.21.15.46:0/3713505109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efec0158e90 0x7efec015beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.036+0000 7efec6400640 1 --2- 172.21.15.46:0/3713505109 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efec01f24a0 0x7efec01f2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.036+0000 7efec6400640 1 --2- 172.21.15.46:0/3713505109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efea4004d10 0x7efec013a7a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.036+0000 7efec6400640 1 -- 172.21.15.46:0/3713505109 >> 172.21.15.46:0/3713505109 conn(0x7efec006a390 msgr2=0x7efec00cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:58.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.036+0000 7efec6400640 1 -- 172.21.15.46:0/3713505109 shutdown_connections 2024-03-29T08:18:58.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.036+0000 7efec6400640 1 -- 172.21.15.46:0/3713505109 wait complete. 2024-03-29T08:18:58.057 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:18:58.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.196+0000 7f419ee00640 1 Processor -- start 2024-03-29T08:18:58.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.196+0000 7f419ee00640 1 -- start start 2024-03-29T08:18:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.196+0000 7f419ee00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f419815db10 0x7f4198164410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.196+0000 7f419ee00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4198070c70 con 0x7f4198142ab0 2024-03-29T08:18:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.196+0000 7f419ee00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4198070340 con 0x7f419806dbb0 2024-03-29T08:18:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.196+0000 7f419ee00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4198070620 con 0x7f419815db10 2024-03-29T08:18:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.196+0000 7f4197e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4198142ab0 0x7f419806d4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54716/0 (socket says 172.21.15.46:54716) 2024-03-29T08:18:58.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.196+0000 7f4197e00640 1 -- 172.21.15.46:0/518893801 learned_addr learned my addr 172.21.15.46:0/518893801 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:58.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.196+0000 7f419ce00640 1 --2- 172.21.15.46:0/518893801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f419815db10 0x7f4198164410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:58.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.200+0000 7f4197400640 1 -- 172.21.15.46:0/518893801 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823098125 0 0) 0x7f4198070c70 con 0x7f4198142ab0 2024-03-29T08:18:58.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.200+0000 7f4197400640 1 -- 172.21.15.46:0/518893801 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f417c003530 con 0x7f4198142ab0 2024-03-29T08:18:58.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.200+0000 7f4197400640 1 -- 172.21.15.46:0/518893801 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1938817534 0 0) 0x7f4198070340 con 0x7f419806dbb0 2024-03-29T08:18:58.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.200+0000 7f4197400640 1 -- 172.21.15.46:0/518893801 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4198070c70 con 0x7f419806dbb0 2024-03-29T08:18:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.200+0000 7f419ce00640 1 -- 172.21.15.46:0/518893801 >> v1:172.21.15.73:6789/0 conn(0x7f419806dbb0 legacy=0x7f419815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.200+0000 7f419ce00640 1 -- 172.21.15.46:0/518893801 >> v1:172.21.15.46:6789/0 conn(0x7f4198142ab0 legacy=0x7f419806d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.200+0000 7f4197400640 1 -- 172.21.15.46:0/518893801 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 425895502 0 0) 0x7f417c003530 con 0x7f4198142ab0 2024-03-29T08:18:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.200+0000 7f419ce00640 1 -- 172.21.15.46:0/518893801 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4198070900 con 0x7f419815db10 2024-03-29T08:18:58.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.200+0000 7f419ce00640 1 --2- 172.21.15.46:0/518893801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f419815db10 0x7f4198164410 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4190009dc0 tx=0x7f419001c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=514447f2ee8f7f33 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.200+0000 7f4197400640 1 -- 172.21.15.46:0/518893801 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4188001680 con 0x7f419815db10 2024-03-29T08:18:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.200+0000 7f4197400640 1 -- 172.21.15.46:0/518893801 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41900a8450 con 0x7f419815db10 2024-03-29T08:18:58.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.200+0000 7f4197400640 1 -- 172.21.15.46:0/518893801 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41900a87e0 con 0x7f419815db10 2024-03-29T08:18:58.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ee00640 1 -- 172.21.15.46:0/518893801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f419815db10 msgr2=0x7f4198164410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ee00640 1 --2- 172.21.15.46:0/518893801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f419815db10 0x7f4198164410 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4190009dc0 tx=0x7f419001c8b0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ee00640 1 -- 172.21.15.46:0/518893801 shutdown_connections 2024-03-29T08:18:58.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ee00640 1 --2- 172.21.15.46:0/518893801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f419815db10 0x7f4198164410 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ee00640 1 -- 172.21.15.46:0/518893801 >> 172.21.15.46:0/518893801 conn(0x7f4198066080 msgr2=0x7f4198069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:58.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ee00640 1 -- 172.21.15.46:0/518893801 shutdown_connections 2024-03-29T08:18:58.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ee00640 1 -- 172.21.15.46:0/518893801 wait complete. 2024-03-29T08:18:58.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ee00640 1 Processor -- start 2024-03-29T08:18:58.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ee00640 1 -- start start 2024-03-29T08:18:58.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ee00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f419806dbb0 0x7f4198138c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:58.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4198142ab0 0x7f419815c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:58.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f419815db10 0x7f4198163c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:58.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4198139350 con 0x7f419815db10 2024-03-29T08:18:58.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419815acc0 con 0x7f4198142ab0 2024-03-29T08:18:58.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ee00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419815afa0 con 0x7f419806dbb0 2024-03-29T08:18:58.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f419815db10 0x7f4198163c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:58.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f419815db10 0x7f4198163c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55162/0 (socket says 172.21.15.46:55162) 2024-03-29T08:18:58.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ce00640 1 -- 172.21.15.46:0/2530594328 learned_addr learned my addr 172.21.15.46:0/2530594328 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f4197e00640 1 --2- 172.21.15.46:0/2530594328 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f419806dbb0 0x7f4198138c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f418f400640 1 --2- 172.21.15.46:0/2530594328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4198142ab0 0x7f419815c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ce00640 1 -- 172.21.15.46:0/2530594328 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f419806dbb0 msgr2=0x7f4198138c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ce00640 1 --2- 172.21.15.46:0/2530594328 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f419806dbb0 0x7f4198138c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ce00640 1 -- 172.21.15.46:0/2530594328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4198142ab0 msgr2=0x7f419815c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ce00640 1 --2- 172.21.15.46:0/2530594328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4198142ab0 0x7f419815c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ce00640 1 -- 172.21.15.46:0/2530594328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4190009a70 con 0x7f419815db10 2024-03-29T08:18:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f4197e00640 1 --2- 172.21.15.46:0/2530594328 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f419806dbb0 0x7f4198138c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f418f400640 1 --2- 172.21.15.46:0/2530594328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4198142ab0 0x7f419815c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.204+0000 7f419ce00640 1 --2- 172.21.15.46:0/2530594328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f419815db10 0x7f4198163c80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f419001cd90 tx=0x7f4190019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.208+0000 7f4194c00640 1 -- 172.21.15.46:0/2530594328 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41900b6070 con 0x7f419815db10 2024-03-29T08:18:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.208+0000 7f4194c00640 1 -- 172.21.15.46:0/2530594328 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4190009450 con 0x7f419815db10 2024-03-29T08:18:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.208+0000 7f4194c00640 1 -- 172.21.15.46:0/2530594328 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41900ba610 con 0x7f419815db10 2024-03-29T08:18:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.208+0000 7f419ee00640 1 -- 172.21.15.46:0/2530594328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419815b280 con 0x7f419815db10 2024-03-29T08:18:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.208+0000 7f419ee00640 1 -- 172.21.15.46:0/2530594328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41981641c0 con 0x7f419815db10 2024-03-29T08:18:58.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.208+0000 7f419ee00640 1 -- 172.21.15.46:0/2530594328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4198071a90 con 0x7f419815db10 2024-03-29T08:18:58.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.208+0000 7f4194c00640 1 -- 172.21.15.46:0/2530594328 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f41900b0400 con 0x7f419815db10 2024-03-29T08:18:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.208+0000 7f4194c00640 1 --2- 172.21.15.46:0/2530594328 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4174066690 0x7f4174068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.208+0000 7f4194c00640 1 -- 172.21.15.46:0/2530594328 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f419011e670 con 0x7f419815db10 2024-03-29T08:18:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.208+0000 7f4197e00640 1 --2- 172.21.15.46:0/2530594328 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4174066690 0x7f4174068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.212+0000 7f4197e00640 1 --2- 172.21.15.46:0/2530594328 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4174066690 0x7f4174068b50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4188000fc0 tx=0x7f4188009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:58.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.212+0000 7f4194c00640 1 -- 172.21.15.46:0/2530594328 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f41900ef570 con 0x7f419815db10 2024-03-29T08:18:58.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.384+0000 7f419ee00640 1 -- 172.21.15.46:0/2530594328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f419804e120 con 0x7f419815db10 2024-03-29T08:18:58.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.384+0000 7f4194c00640 1 -- 172.21.15.46:0/2530594328 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f41900a8e60 con 0x7f419815db10 2024-03-29T08:18:58.387 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:18:58.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.388+0000 7f419ee00640 1 -- 172.21.15.46:0/2530594328 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4174066690 msgr2=0x7f4174068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.388+0000 7f419ee00640 1 --2- 172.21.15.46:0/2530594328 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4174066690 0x7f4174068b50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4188000fc0 tx=0x7f4188009040 comp rx=0 tx=0).stop 2024-03-29T08:18:58.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.388+0000 7f419ee00640 1 -- 172.21.15.46:0/2530594328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f419815db10 msgr2=0x7f4198163c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.388+0000 7f419ee00640 1 --2- 172.21.15.46:0/2530594328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f419815db10 0x7f4198163c80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f419001cd90 tx=0x7f4190019650 comp rx=0 tx=0).stop 2024-03-29T08:18:58.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.388+0000 7f419ee00640 1 -- 172.21.15.46:0/2530594328 shutdown_connections 2024-03-29T08:18:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.388+0000 7f419ee00640 1 --2- 172.21.15.46:0/2530594328 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4174066690 0x7f4174068b50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.388+0000 7f419ee00640 1 --2- 172.21.15.46:0/2530594328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4198142ab0 0x7f419815c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.388+0000 7f419ee00640 1 --2- 172.21.15.46:0/2530594328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f419815db10 0x7f4198163c80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.388+0000 7f419ee00640 1 --2- 172.21.15.46:0/2530594328 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f419806dbb0 0x7f4198138c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.388+0000 7f419ee00640 1 -- 172.21.15.46:0/2530594328 >> 172.21.15.46:0/2530594328 conn(0x7f4198066080 msgr2=0x7f4198139fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.388+0000 7f419ee00640 1 -- 172.21.15.46:0/2530594328 shutdown_connections 2024-03-29T08:18:58.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.388+0000 7f419ee00640 1 -- 172.21.15.46:0/2530594328 wait complete. 2024-03-29T08:18:58.410 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:18:58.410 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:18:58.420 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:18:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.556+0000 7f53dae00640 1 Processor -- start 2024-03-29T08:18:58.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.560+0000 7f53dae00640 1 -- start start 2024-03-29T08:18:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.560+0000 7f53dae00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53d415a640 0x7f53d40ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.560+0000 7f53dae00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f53d415eac0 0x7f53d40cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.560+0000 7f53dae00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d40cee30 con 0x7f53d40cf2c0 2024-03-29T08:18:58.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.560+0000 7f53dae00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d415ef10 con 0x7f53d415a640 2024-03-29T08:18:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.560+0000 7f53dae00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d415f120 con 0x7f53d415eac0 2024-03-29T08:18:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.560+0000 7f53d9a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53d415a640 0x7f53d40ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.560+0000 7f53d9a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53d415a640 0x7f53d40ccf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55176/0 (socket says 172.21.15.46:55176) 2024-03-29T08:18:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.560+0000 7f53d9000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f53d40cf2c0 0x7f53d415e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54728/0 (socket says 172.21.15.46:54728) 2024-03-29T08:18:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.560+0000 7f53d9a00640 1 -- 172.21.15.46:0/3211973137 learned_addr learned my addr 172.21.15.46:0/3211973137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.560+0000 7f53da400640 1 --2- 172.21.15.46:0/3211973137 >> v2:172.21.15.73:3300/0 conn(0x7f53d415eac0 0x7f53d40cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.560+0000 7f53d9a00640 1 -- 172.21.15.46:0/3211973137 >> v2:172.21.15.73:3300/0 conn(0x7f53d415eac0 msgr2=0x7f53d40cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.560+0000 7f53d9a00640 1 --2- 172.21.15.46:0/3211973137 >> v2:172.21.15.73:3300/0 conn(0x7f53d415eac0 0x7f53d40cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.560+0000 7f53d9a00640 1 -- 172.21.15.46:0/3211973137 >> v1:172.21.15.46:6789/0 conn(0x7f53d40cf2c0 legacy=0x7f53d415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.560+0000 7f53d9a00640 1 -- 172.21.15.46:0/3211973137 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d40ce500 con 0x7f53d415a640 2024-03-29T08:18:58.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.560+0000 7f53da400640 1 --2- 172.21.15.46:0/3211973137 >> v2:172.21.15.73:3300/0 conn(0x7f53d415eac0 0x7f53d40cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:18:58.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.564+0000 7f53d9a00640 1 --2- 172.21.15.46:0/3211973137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d415a640 0x7f53d40ccf50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f53d40c75c0 tx=0x7f53c801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7a00213357fdbb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:58.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.564+0000 7f53d3e00640 1 -- 172.21.15.46:0/3211973137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c80ab600 con 0x7f53d415a640 2024-03-29T08:18:58.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.564+0000 7f53d3e00640 1 -- 172.21.15.46:0/3211973137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c80ab760 con 0x7f53d415a640 2024-03-29T08:18:58.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.564+0000 7f53d3e00640 1 -- 172.21.15.46:0/3211973137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c80abaf0 con 0x7f53d415a640 2024-03-29T08:18:58.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.564+0000 7f53dae00640 1 -- 172.21.15.46:0/3211973137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d415a640 msgr2=0x7f53d40ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.564+0000 7f53dae00640 1 --2- 172.21.15.46:0/3211973137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d415a640 0x7f53d40ccf50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f53d40c75c0 tx=0x7f53c801c4f0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.564+0000 7f53dae00640 1 -- 172.21.15.46:0/3211973137 shutdown_connections 2024-03-29T08:18:58.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.564+0000 7f53dae00640 1 --2- 172.21.15.46:0/3211973137 >> v2:172.21.15.73:3300/0 conn(0x7f53d415eac0 0x7f53d40cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.564+0000 7f53dae00640 1 --2- 172.21.15.46:0/3211973137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d415a640 0x7f53d40ccf50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.564+0000 7f53dae00640 1 -- 172.21.15.46:0/3211973137 >> 172.21.15.46:0/3211973137 conn(0x7f53d40d0560 msgr2=0x7f53d40d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:58.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.564+0000 7f53dae00640 1 -- 172.21.15.46:0/3211973137 shutdown_connections 2024-03-29T08:18:58.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.564+0000 7f53dae00640 1 -- 172.21.15.46:0/3211973137 wait complete. 2024-03-29T08:18:58.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.564+0000 7f53dae00640 1 Processor -- start 2024-03-29T08:18:58.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.564+0000 7f53dae00640 1 -- start start 2024-03-29T08:18:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53dae00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d40cf2c0 0x7f53d4158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53dae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d415a640 0x7f53d40ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53dae00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53d415eac0 0x7f53d40b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53dae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d40ae0b0 con 0x7f53d415a640 2024-03-29T08:18:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53dae00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d40ae390 con 0x7f53d40cf2c0 2024-03-29T08:18:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53dae00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d40ae670 con 0x7f53d415eac0 2024-03-29T08:18:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53d9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d415a640 0x7f53d40ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53d9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d415a640 0x7f53d40ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55184/0 (socket says 172.21.15.46:55184) 2024-03-29T08:18:58.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53d9000640 1 -- 172.21.15.46:0/90742593 learned_addr learned my addr 172.21.15.46:0/90742593 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53d9a00640 1 --2- 172.21.15.46:0/90742593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d40cf2c0 0x7f53d4158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53da400640 1 --2- 172.21.15.46:0/90742593 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53d415eac0 0x7f53d40b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53d9000640 1 -- 172.21.15.46:0/90742593 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53d415eac0 msgr2=0x7f53d40b0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53d9000640 1 --2- 172.21.15.46:0/90742593 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53d415eac0 0x7f53d40b0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53d9000640 1 -- 172.21.15.46:0/90742593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d40cf2c0 msgr2=0x7f53d4158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53d9000640 1 --2- 172.21.15.46:0/90742593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d40cf2c0 0x7f53d4158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53d9000640 1 -- 172.21.15.46:0/90742593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c80094d0 con 0x7f53d415a640 2024-03-29T08:18:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53d9000640 1 --2- 172.21.15.46:0/90742593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d415a640 0x7f53d40ad860 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f53c000c490 tx=0x7f53c000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53d1600640 1 -- 172.21.15.46:0/90742593 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c0007480 con 0x7f53d415a640 2024-03-29T08:18:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53d1600640 1 -- 172.21.15.46:0/90742593 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c00075e0 con 0x7f53d415a640 2024-03-29T08:18:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53d1600640 1 -- 172.21.15.46:0/90742593 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c0004a10 con 0x7f53d415a640 2024-03-29T08:18:58.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53dae00640 1 -- 172.21.15.46:0/90742593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d40ae950 con 0x7f53d415a640 2024-03-29T08:18:58.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53dae00640 1 -- 172.21.15.46:0/90742593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d40b1310 con 0x7f53d415a640 2024-03-29T08:18:58.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53dae00640 1 -- 172.21.15.46:0/90742593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d404e040 con 0x7f53d415a640 2024-03-29T08:18:58.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.568+0000 7f53d1600640 1 -- 172.21.15.46:0/90742593 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f53c0004b70 con 0x7f53d415a640 2024-03-29T08:18:58.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.572+0000 7f53d1600640 1 --2- 172.21.15.46:0/90742593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53b4066690 0x7f53b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:58.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.572+0000 7f53d1600640 1 -- 172.21.15.46:0/90742593 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f53c0083280 con 0x7f53d415a640 2024-03-29T08:18:58.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.572+0000 7f53d9a00640 1 --2- 172.21.15.46:0/90742593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53b4066690 0x7f53b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:58.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.572+0000 7f53d9a00640 1 --2- 172.21.15.46:0/90742593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53b4066690 0x7f53b4068b50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f53c80af000 tx=0x7f53c80b6040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:58.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.576+0000 7f53d1600640 1 -- 172.21.15.46:0/90742593 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f53c0050710 con 0x7f53d415a640 2024-03-29T08:18:58.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.736+0000 7f53dae00640 1 -- 172.21.15.46:0/90742593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53d40b1590 con 0x7f53d415a640 2024-03-29T08:18:58.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.736+0000 7f53d1600640 1 -- 172.21.15.46:0/90742593 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16042 (secure 0 0 0) 0x7f53c0053d30 con 0x7f53d415a640 2024-03-29T08:18:58.739 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:18:58.739 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:56.021693+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:18:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.740+0000 7f53dae00640 1 -- 172.21.15.46:0/90742593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53b4066690 msgr2=0x7f53b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.740+0000 7f53dae00640 1 --2- 172.21.15.46:0/90742593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53b4066690 0x7f53b4068b50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f53c80af000 tx=0x7f53c80b6040 comp rx=0 tx=0).stop 2024-03-29T08:18:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.740+0000 7f53dae00640 1 -- 172.21.15.46:0/90742593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d415a640 msgr2=0x7f53d40ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.740+0000 7f53dae00640 1 --2- 172.21.15.46:0/90742593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d415a640 0x7f53d40ad860 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f53c000c490 tx=0x7f53c000c950 comp rx=0 tx=0).stop 2024-03-29T08:18:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.740+0000 7f53dae00640 1 -- 172.21.15.46:0/90742593 shutdown_connections 2024-03-29T08:18:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.740+0000 7f53dae00640 1 --2- 172.21.15.46:0/90742593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53b4066690 0x7f53b4068b50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.740+0000 7f53dae00640 1 --2- 172.21.15.46:0/90742593 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53d415eac0 0x7f53d40b0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.740+0000 7f53dae00640 1 --2- 172.21.15.46:0/90742593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53d415a640 0x7f53d40ad860 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.740+0000 7f53dae00640 1 --2- 172.21.15.46:0/90742593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d40cf2c0 0x7f53d4158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.740+0000 7f53dae00640 1 -- 172.21.15.46:0/90742593 >> 172.21.15.46:0/90742593 conn(0x7f53d40d0560 msgr2=0x7f53d415e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.740+0000 7f53dae00640 1 -- 172.21.15.46:0/90742593 shutdown_connections 2024-03-29T08:18:58.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.740+0000 7f53dae00640 1 -- 172.21.15.46:0/90742593 wait complete. 2024-03-29T08:18:58.759 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:18:11.177842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:18:12.308252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:18:58.760 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:18:58.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.900+0000 7f149d600640 1 Processor -- start 2024-03-29T08:18:58.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.900+0000 7f149d600640 1 -- start start 2024-03-29T08:18:58.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.904+0000 7f149d600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f149815c9d0 0x7f14980cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:58.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.904+0000 7f149d600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14980cc250 con 0x7f14980cee30 2024-03-29T08:18:58.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.904+0000 7f149d600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14980ccc20 con 0x7f14980b0440 2024-03-29T08:18:58.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.904+0000 7f149d600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f149815cdb0 con 0x7f149815c9d0 2024-03-29T08:18:58.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.904+0000 7f149cc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f14980cee30 0x7f14980c9a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54732/0 (socket says 172.21.15.46:54732) 2024-03-29T08:18:58.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.904+0000 7f149cc00640 1 -- 172.21.15.46:0/4055255233 learned_addr learned my addr 172.21.15.46:0/4055255233 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:18:58.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.904+0000 7f1497e00640 1 --2- 172.21.15.46:0/4055255233 >> v2:172.21.15.73:3300/0 conn(0x7f149815c9d0 0x7f14980cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:58.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.904+0000 7f1497400640 1 -- 172.21.15.46:0/4055255233 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307673779 0 0) 0x7f14980cc250 con 0x7f14980cee30 2024-03-29T08:18:58.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.904+0000 7f1497400640 1 -- 172.21.15.46:0/4055255233 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f147c003530 con 0x7f14980cee30 2024-03-29T08:18:58.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.904+0000 7f1497400640 1 -- 172.21.15.46:0/4055255233 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3504172905 0 0) 0x7f14980ccc20 con 0x7f14980b0440 2024-03-29T08:18:58.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.904+0000 7f1497400640 1 -- 172.21.15.46:0/4055255233 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14980cc250 con 0x7f14980b0440 2024-03-29T08:18:58.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.904+0000 7f1497e00640 1 -- 172.21.15.46:0/4055255233 >> v1:172.21.15.73:6789/0 conn(0x7f14980b0440 legacy=0x7f14980ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.904+0000 7f1497e00640 1 -- 172.21.15.46:0/4055255233 >> v1:172.21.15.46:6789/0 conn(0x7f14980cee30 legacy=0x7f14980c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.904+0000 7f1497e00640 1 -- 172.21.15.46:0/4055255233 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14980cb640 con 0x7f149815c9d0 2024-03-29T08:18:58.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f1497e00640 1 --2- 172.21.15.46:0/4055255233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f149815c9d0 0x7f14980cc6e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f14880097c0 tx=0x7f148801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c74065f18ccb692e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:58.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f1497400640 1 -- 172.21.15.46:0/4055255233 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14880ad680 con 0x7f149815c9d0 2024-03-29T08:18:58.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f1497400640 1 -- 172.21.15.46:0/4055255233 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14880ad7e0 con 0x7f149815c9d0 2024-03-29T08:18:58.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f1497400640 1 -- 172.21.15.46:0/4055255233 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14880adb50 con 0x7f149815c9d0 2024-03-29T08:18:58.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f149d600640 1 -- 172.21.15.46:0/4055255233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f149815c9d0 msgr2=0x7f14980cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f149d600640 1 --2- 172.21.15.46:0/4055255233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f149815c9d0 0x7f14980cc6e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f14880097c0 tx=0x7f148801c470 comp rx=0 tx=0).stop 2024-03-29T08:18:58.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f149d600640 1 -- 172.21.15.46:0/4055255233 shutdown_connections 2024-03-29T08:18:58.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f149d600640 1 --2- 172.21.15.46:0/4055255233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f149815c9d0 0x7f14980cc6e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f149d600640 1 -- 172.21.15.46:0/4055255233 >> 172.21.15.46:0/4055255233 conn(0x7f1498098a40 msgr2=0x7f14980cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:58.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f149d600640 1 -- 172.21.15.46:0/4055255233 shutdown_connections 2024-03-29T08:18:58.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f149d600640 1 -- 172.21.15.46:0/4055255233 wait complete. 2024-03-29T08:18:58.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f149d600640 1 Processor -- start 2024-03-29T08:18:58.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f149d600640 1 -- start start 2024-03-29T08:18:58.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f149d600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f14980b0440 0x7f1498153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:58.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f149d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14980cee30 0x7f149814b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:58.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f149d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f149815c9d0 0x7f1498147530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:58.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f149d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149814bf30 con 0x7f149815c9d0 2024-03-29T08:18:58.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f149d600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149814cae0 con 0x7f14980cee30 2024-03-29T08:18:58.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.908+0000 7f149d600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14981534b0 con 0x7f14980b0440 2024-03-29T08:18:58.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f149cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f149815c9d0 0x7f1498147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:58.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f149cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f149815c9d0 0x7f1498147530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55200/0 (socket says 172.21.15.46:55200) 2024-03-29T08:18:58.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f149cc00640 1 -- 172.21.15.46:0/1818479874 learned_addr learned my addr 172.21.15.46:0/1818479874 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:58.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f148f400640 1 --2- 172.21.15.46:0/1818479874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14980cee30 0x7f149814b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:58.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f1497e00640 1 --2- 172.21.15.46:0/1818479874 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f14980b0440 0x7f1498153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:58.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f149cc00640 1 -- 172.21.15.46:0/1818479874 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f14980b0440 msgr2=0x7f1498153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f149cc00640 1 --2- 172.21.15.46:0/1818479874 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f14980b0440 0x7f1498153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f149cc00640 1 -- 172.21.15.46:0/1818479874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14980cee30 msgr2=0x7f149814b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:58.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f149cc00640 1 --2- 172.21.15.46:0/1818479874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14980cee30 0x7f149814b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:58.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f149cc00640 1 -- 172.21.15.46:0/1818479874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1488009420 con 0x7f149815c9d0 2024-03-29T08:18:58.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f149cc00640 1 --2- 172.21.15.46:0/1818479874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f149815c9d0 0x7f1498147530 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f149000c480 tx=0x7f149000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:58.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f1494c00640 1 -- 172.21.15.46:0/1818479874 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1490005cf0 con 0x7f149815c9d0 2024-03-29T08:18:58.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f1494c00640 1 -- 172.21.15.46:0/1818479874 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1490005e50 con 0x7f149815c9d0 2024-03-29T08:18:58.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f149d600640 1 -- 172.21.15.46:0/1818479874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f149814a6a0 con 0x7f149815c9d0 2024-03-29T08:18:58.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f1494c00640 1 -- 172.21.15.46:0/1818479874 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1490007270 con 0x7f149815c9d0 2024-03-29T08:18:58.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f149d600640 1 -- 172.21.15.46:0/1818479874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149814c800 con 0x7f149815c9d0 2024-03-29T08:18:58.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f149d600640 1 -- 172.21.15.46:0/1818479874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1498152d00 con 0x7f149815c9d0 2024-03-29T08:18:58.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f1494c00640 1 -- 172.21.15.46:0/1818479874 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f14900073d0 con 0x7f149815c9d0 2024-03-29T08:18:58.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.912+0000 7f1494c00640 1 --2- 172.21.15.46:0/1818479874 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1474066690 0x7f1474068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:58.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.916+0000 7f1494c00640 1 -- 172.21.15.46:0/1818479874 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1490083660 con 0x7f149815c9d0 2024-03-29T08:18:58.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.916+0000 7f1497e00640 1 --2- 172.21.15.46:0/1818479874 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1474066690 0x7f1474068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:58.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.916+0000 7f1497e00640 1 --2- 172.21.15.46:0/1818479874 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1474066690 0x7f1474068b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f14880af000 tx=0x7f1488008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:58.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:58.916+0000 7f1494c00640 1 -- 172.21.15.46:0/1818479874 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1498152d00 con 0x7f149815c9d0 2024-03-29T08:18:59.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.080+0000 7f149d600640 1 -- 172.21.15.46:0/1818479874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1498152d00 con 0x7f149815c9d0 2024-03-29T08:18:59.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.080+0000 7f1494c00640 1 -- 172.21.15.46:0/1818479874 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1498152d00 con 0x7f149815c9d0 2024-03-29T08:18:59.082 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:18:59.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.084+0000 7f149d600640 1 -- 172.21.15.46:0/1818479874 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1474066690 msgr2=0x7f1474068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:59.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.084+0000 7f149d600640 1 --2- 172.21.15.46:0/1818479874 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1474066690 0x7f1474068b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f14880af000 tx=0x7f1488008040 comp rx=0 tx=0).stop 2024-03-29T08:18:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.084+0000 7f149d600640 1 -- 172.21.15.46:0/1818479874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f149815c9d0 msgr2=0x7f1498147530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.084+0000 7f149d600640 1 --2- 172.21.15.46:0/1818479874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f149815c9d0 0x7f1498147530 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f149000c480 tx=0x7f149000c940 comp rx=0 tx=0).stop 2024-03-29T08:18:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.084+0000 7f149d600640 1 -- 172.21.15.46:0/1818479874 shutdown_connections 2024-03-29T08:18:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.084+0000 7f149d600640 1 --2- 172.21.15.46:0/1818479874 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1474066690 0x7f1474068b50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.084+0000 7f149d600640 1 --2- 172.21.15.46:0/1818479874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14980cee30 0x7f149814b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.084+0000 7f149d600640 1 --2- 172.21.15.46:0/1818479874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f149815c9d0 0x7f1498147530 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.084+0000 7f149d600640 1 --2- 172.21.15.46:0/1818479874 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f14980b0440 0x7f1498153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.084+0000 7f149d600640 1 -- 172.21.15.46:0/1818479874 >> 172.21.15.46:0/1818479874 conn(0x7f1498098a40 msgr2=0x7f14980b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.084+0000 7f149d600640 1 -- 172.21.15.46:0/1818479874 shutdown_connections 2024-03-29T08:18:59.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.084+0000 7f149d600640 1 -- 172.21.15.46:0/1818479874 wait complete. 2024-03-29T08:18:59.103 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:18:59.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.244+0000 7f8062a00640 1 Processor -- start 2024-03-29T08:18:59.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.244+0000 7f8062a00640 1 -- start start 2024-03-29T08:18:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.244+0000 7f8062a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f805c0cbea0 0x7f805c0b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.244+0000 7f8062a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f805c15efe0 0x7f805c0b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.244+0000 7f8062a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f805c0b20b0 con 0x7f805c143eb0 2024-03-29T08:18:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.244+0000 7f8062a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f805c0b2360 con 0x7f805c15efe0 2024-03-29T08:18:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.244+0000 7f8062a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805c15e500 con 0x7f805c0cbea0 2024-03-29T08:18:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.248+0000 7f8060a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f805c15efe0 0x7f805c0b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.248+0000 7f8060a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f805c15efe0 0x7f805c0b1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55204/0 (socket says 172.21.15.46:55204) 2024-03-29T08:18:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.248+0000 7f8060a00640 1 -- 172.21.15.46:0/2497689410 learned_addr learned my addr 172.21.15.46:0/2497689410 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.248+0000 7f8053400640 1 --2- 172.21.15.46:0/2497689410 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f805c0cbea0 0x7f805c0b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.248+0000 7f8060a00640 1 -- 172.21.15.46:0/2497689410 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f805c0cbea0 msgr2=0x7f805c0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.248+0000 7f8060a00640 1 --2- 172.21.15.46:0/2497689410 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f805c0cbea0 0x7f805c0b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.248+0000 7f8060a00640 1 -- 172.21.15.46:0/2497689410 >> v1:172.21.15.73:6789/0 conn(0x7f805c143eb0 legacy=0x7f805c0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.248+0000 7f8060a00640 1 -- 172.21.15.46:0/2497689410 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805c15e7e0 con 0x7f805c15efe0 2024-03-29T08:18:59.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.248+0000 7f8060a00640 1 --2- 172.21.15.46:0/2497689410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c15efe0 0x7f805c0b1b70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8054009e50 tx=0x7f805401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb05ca5fe6617e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:59.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.248+0000 7f805b400640 1 -- 172.21.15.46:0/2497689410 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80540a5870 con 0x7f805c15efe0 2024-03-29T08:18:59.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.248+0000 7f805b400640 1 -- 172.21.15.46:0/2497689410 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80540a59d0 con 0x7f805c15efe0 2024-03-29T08:18:59.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.248+0000 7f805b400640 1 -- 172.21.15.46:0/2497689410 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80540a5d60 con 0x7f805c15efe0 2024-03-29T08:18:59.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8062a00640 1 -- 172.21.15.46:0/2497689410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c15efe0 msgr2=0x7f805c0b1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:59.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8062a00640 1 --2- 172.21.15.46:0/2497689410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c15efe0 0x7f805c0b1b70 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8054009e50 tx=0x7f805401ca00 comp rx=0 tx=0).stop 2024-03-29T08:18:59.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8062a00640 1 -- 172.21.15.46:0/2497689410 shutdown_connections 2024-03-29T08:18:59.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8062a00640 1 --2- 172.21.15.46:0/2497689410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c15efe0 0x7f805c0b1b70 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8062a00640 1 --2- 172.21.15.46:0/2497689410 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f805c0cbea0 0x7f805c0b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8062a00640 1 -- 172.21.15.46:0/2497689410 >> 172.21.15.46:0/2497689410 conn(0x7f805c06a390 msgr2=0x7f805c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:59.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8062a00640 1 -- 172.21.15.46:0/2497689410 shutdown_connections 2024-03-29T08:18:59.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8062a00640 1 -- 172.21.15.46:0/2497689410 wait complete. 2024-03-29T08:18:59.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8062a00640 1 Processor -- start 2024-03-29T08:18:59.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8062a00640 1 -- start start 2024-03-29T08:18:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8062a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f805c0cbea0 0x7f805c158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8062a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f805c143eb0 0x7f805c13c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8062a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c15efe0 0x7f805c0ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8062a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805c13e270 con 0x7f805c15efe0 2024-03-29T08:18:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8062a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805c13e4c0 con 0x7f805c0cbea0 2024-03-29T08:18:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8062a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805c13e7a0 con 0x7f805c143eb0 2024-03-29T08:18:59.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8060a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c15efe0 0x7f805c0ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8060a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c15efe0 0x7f805c0ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55210/0 (socket says 172.21.15.46:55210) 2024-03-29T08:18:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8060a00640 1 -- 172.21.15.46:0/4026449819 learned_addr learned my addr 172.21.15.46:0/4026449819 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8053400640 1 --2- 172.21.15.46:0/4026449819 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f805c143eb0 0x7f805c13c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f805be00640 1 --2- 172.21.15.46:0/4026449819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f805c0cbea0 0x7f805c158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8060a00640 1 -- 172.21.15.46:0/4026449819 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f805c143eb0 msgr2=0x7f805c13c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8060a00640 1 --2- 172.21.15.46:0/4026449819 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f805c143eb0 0x7f805c13c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8060a00640 1 -- 172.21.15.46:0/4026449819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f805c0cbea0 msgr2=0x7f805c158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8060a00640 1 --2- 172.21.15.46:0/4026449819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f805c0cbea0 0x7f805c158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.252+0000 7f8060a00640 1 -- 172.21.15.46:0/4026449819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8054009b30 con 0x7f805c15efe0 2024-03-29T08:18:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.256+0000 7f8060a00640 1 --2- 172.21.15.46:0/4026449819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c15efe0 0x7f805c0ccf00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f80540af040 tx=0x7f80540197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.256+0000 7f8058c00640 1 -- 172.21.15.46:0/4026449819 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80540b8670 con 0x7f805c15efe0 2024-03-29T08:18:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.256+0000 7f8058c00640 1 -- 172.21.15.46:0/4026449819 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80540b8810 con 0x7f805c15efe0 2024-03-29T08:18:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.256+0000 7f8058c00640 1 -- 172.21.15.46:0/4026449819 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80540b8bc0 con 0x7f805c15efe0 2024-03-29T08:18:59.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.256+0000 7f8062a00640 1 -- 172.21.15.46:0/4026449819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f805c13ea80 con 0x7f805c15efe0 2024-03-29T08:18:59.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.256+0000 7f8062a00640 1 -- 172.21.15.46:0/4026449819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f805c13cd00 con 0x7f805c15efe0 2024-03-29T08:18:59.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.256+0000 7f8062a00640 1 -- 172.21.15.46:0/4026449819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f805c13ee60 con 0x7f805c15efe0 2024-03-29T08:18:59.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.256+0000 7f8058c00640 1 -- 172.21.15.46:0/4026449819 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f80540b8d20 con 0x7f805c15efe0 2024-03-29T08:18:59.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.256+0000 7f8058c00640 1 --2- 172.21.15.46:0/4026449819 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8038066690 0x7f8038068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.256+0000 7f8058c00640 1 -- 172.21.15.46:0/4026449819 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f805411f7b0 con 0x7f805c15efe0 2024-03-29T08:18:59.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.256+0000 7f805be00640 1 --2- 172.21.15.46:0/4026449819 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8038066690 0x7f8038068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.260+0000 7f805be00640 1 --2- 172.21.15.46:0/4026449819 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8038066690 0x7f8038068b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f804c000f70 tx=0x7f804c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:59.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.260+0000 7f8058c00640 1 -- 172.21.15.46:0/4026449819 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f80540ecc20 con 0x7f805c15efe0 2024-03-29T08:18:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.420+0000 7f8062a00640 1 -- 172.21.15.46:0/4026449819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f805c1385b0 con 0x7f805c15efe0 2024-03-29T08:18:59.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.420+0000 7f8058c00640 1 -- 172.21.15.46:0/4026449819 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f805c1385b0 con 0x7f805c15efe0 2024-03-29T08:18:59.423 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:18:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.424+0000 7f8062a00640 1 -- 172.21.15.46:0/4026449819 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8038066690 msgr2=0x7f8038068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.424+0000 7f8062a00640 1 --2- 172.21.15.46:0/4026449819 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8038066690 0x7f8038068b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f804c000f70 tx=0x7f804c009040 comp rx=0 tx=0).stop 2024-03-29T08:18:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.424+0000 7f8062a00640 1 -- 172.21.15.46:0/4026449819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c15efe0 msgr2=0x7f805c0ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.424+0000 7f8062a00640 1 --2- 172.21.15.46:0/4026449819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c15efe0 0x7f805c0ccf00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f80540af040 tx=0x7f80540197a0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.424+0000 7f8062a00640 1 -- 172.21.15.46:0/4026449819 shutdown_connections 2024-03-29T08:18:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.424+0000 7f8062a00640 1 --2- 172.21.15.46:0/4026449819 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8038066690 0x7f8038068b50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.424+0000 7f8062a00640 1 --2- 172.21.15.46:0/4026449819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f805c15efe0 0x7f805c0ccf00 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.424+0000 7f8062a00640 1 --2- 172.21.15.46:0/4026449819 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f805c143eb0 0x7f805c13c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.424+0000 7f8062a00640 1 --2- 172.21.15.46:0/4026449819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f805c0cbea0 0x7f805c158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.424+0000 7f8062a00640 1 -- 172.21.15.46:0/4026449819 >> 172.21.15.46:0/4026449819 conn(0x7f805c06a390 msgr2=0x7f805c0cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.424+0000 7f8062a00640 1 -- 172.21.15.46:0/4026449819 shutdown_connections 2024-03-29T08:18:59.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.424+0000 7f8062a00640 1 -- 172.21.15.46:0/4026449819 wait complete. 2024-03-29T08:18:59.443 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:18:59.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.584+0000 7f12cec00640 1 Processor -- start 2024-03-29T08:18:59.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.584+0000 7f12cec00640 1 -- start start 2024-03-29T08:18:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12cec00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12c8143eb0 0x7f12c80cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12cec00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f12c815efe0 0x7f12c80cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12cec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c80b2360 con 0x7f12c80b1630 2024-03-29T08:18:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12cec00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c815e500 con 0x7f12c8143eb0 2024-03-29T08:18:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12cec00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c815e7e0 con 0x7f12c815efe0 2024-03-29T08:18:59.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12c7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12c8143eb0 0x7f12c80cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12c7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12c8143eb0 0x7f12c80cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55226/0 (socket says 172.21.15.46:55226) 2024-03-29T08:18:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12c7e00640 1 -- 172.21.15.46:0/3238092985 learned_addr learned my addr 172.21.15.46:0/3238092985 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12ccc00640 1 --1- 172.21.15.46:0/3238092985 >> v1:172.21.15.46:6789/0 conn(0x7f12c80b1630 0x7f12c80b1a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54740/0 (socket says 172.21.15.46:54740) 2024-03-29T08:18:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12bf400640 1 --2- 172.21.15.46:0/3238092985 >> v2:172.21.15.73:3300/0 conn(0x7f12c815efe0 0x7f12c80cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12c7e00640 1 -- 172.21.15.46:0/3238092985 >> v2:172.21.15.73:3300/0 conn(0x7f12c815efe0 msgr2=0x7f12c80cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12c7e00640 1 --2- 172.21.15.46:0/3238092985 >> v2:172.21.15.73:3300/0 conn(0x7f12c815efe0 0x7f12c80cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12c7e00640 1 -- 172.21.15.46:0/3238092985 >> v1:172.21.15.46:6789/0 conn(0x7f12c80b1630 legacy=0x7f12c80b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12c7e00640 1 -- 172.21.15.46:0/3238092985 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c815eac0 con 0x7f12c8143eb0 2024-03-29T08:18:59.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12c7400640 1 -- 172.21.15.46:0/3238092985 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3047100170 0 0) 0x7f12c80b2360 con 0x7f12c80b1630 2024-03-29T08:18:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12c7e00640 1 --2- 172.21.15.46:0/3238092985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c8143eb0 0x7f12c80cb790 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f12c8158930 tx=0x7f12b801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a472dd01504e1b29 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12c7400640 1 -- 172.21.15.46:0/3238092985 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b80a57d0 con 0x7f12c8143eb0 2024-03-29T08:18:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12c7400640 1 -- 172.21.15.46:0/3238092985 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b80a5930 con 0x7f12c8143eb0 2024-03-29T08:18:59.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.588+0000 7f12c7400640 1 -- 172.21.15.46:0/3238092985 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b80a5cc0 con 0x7f12c8143eb0 2024-03-29T08:18:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12cec00640 1 -- 172.21.15.46:0/3238092985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c8143eb0 msgr2=0x7f12c80cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12cec00640 1 --2- 172.21.15.46:0/3238092985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c8143eb0 0x7f12c80cb790 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f12c8158930 tx=0x7f12b801c9f0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12cec00640 1 -- 172.21.15.46:0/3238092985 shutdown_connections 2024-03-29T08:18:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12cec00640 1 --2- 172.21.15.46:0/3238092985 >> v2:172.21.15.73:3300/0 conn(0x7f12c815efe0 0x7f12c80cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12cec00640 1 --2- 172.21.15.46:0/3238092985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c8143eb0 0x7f12c80cb790 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12cec00640 1 -- 172.21.15.46:0/3238092985 >> 172.21.15.46:0/3238092985 conn(0x7f12c806a390 msgr2=0x7f12c80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12cec00640 1 -- 172.21.15.46:0/3238092985 shutdown_connections 2024-03-29T08:18:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12cec00640 1 -- 172.21.15.46:0/3238092985 wait complete. 2024-03-29T08:18:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12cec00640 1 Processor -- start 2024-03-29T08:18:59.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12cec00640 1 -- start start 2024-03-29T08:18:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12cec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c80b1630 0x7f12c8159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12cec00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12c8143eb0 0x7f12c8139d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12cec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c815efe0 0x7f12c813ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12cec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c815f450 con 0x7f12c815efe0 2024-03-29T08:18:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12cec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c8144b80 con 0x7f12c80b1630 2024-03-29T08:18:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12cec00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c815f6a0 con 0x7f12c8143eb0 2024-03-29T08:18:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12ccc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c815efe0 0x7f12c813ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12ccc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c815efe0 0x7f12c813ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55234/0 (socket says 172.21.15.46:55234) 2024-03-29T08:18:59.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12ccc00640 1 -- 172.21.15.46:0/4164324360 learned_addr learned my addr 172.21.15.46:0/4164324360 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12bf400640 1 --2- 172.21.15.46:0/4164324360 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12c8143eb0 0x7f12c8139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12c7e00640 1 --2- 172.21.15.46:0/4164324360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c80b1630 0x7f12c8159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12ccc00640 1 -- 172.21.15.46:0/4164324360 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12c8143eb0 msgr2=0x7f12c8139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12ccc00640 1 --2- 172.21.15.46:0/4164324360 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12c8143eb0 0x7f12c8139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12ccc00640 1 -- 172.21.15.46:0/4164324360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c80b1630 msgr2=0x7f12c8159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12ccc00640 1 --2- 172.21.15.46:0/4164324360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c80b1630 0x7f12c8159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.592+0000 7f12ccc00640 1 -- 172.21.15.46:0/4164324360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b8009b30 con 0x7f12c815efe0 2024-03-29T08:18:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.596+0000 7f12ccc00640 1 --2- 172.21.15.46:0/4164324360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c815efe0 0x7f12c813ca20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f12c000c3f0 tx=0x7f12c000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.596+0000 7f12c4c00640 1 -- 172.21.15.46:0/4164324360 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c00074e0 con 0x7f12c815efe0 2024-03-29T08:18:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.596+0000 7f12c4c00640 1 -- 172.21.15.46:0/4164324360 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c0007640 con 0x7f12c815efe0 2024-03-29T08:18:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.596+0000 7f12cec00640 1 -- 172.21.15.46:0/4164324360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c815f980 con 0x7f12c815efe0 2024-03-29T08:18:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.596+0000 7f12c4c00640 1 -- 172.21.15.46:0/4164324360 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0004a30 con 0x7f12c815efe0 2024-03-29T08:18:59.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.596+0000 7f12cec00640 1 -- 172.21.15.46:0/4164324360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c815fc60 con 0x7f12c815efe0 2024-03-29T08:18:59.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.596+0000 7f12cec00640 1 -- 172.21.15.46:0/4164324360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c8071a70 con 0x7f12c815efe0 2024-03-29T08:18:59.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.596+0000 7f12c4c00640 1 -- 172.21.15.46:0/4164324360 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f12c0004b90 con 0x7f12c815efe0 2024-03-29T08:18:59.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.596+0000 7f12c4c00640 1 --2- 172.21.15.46:0/4164324360 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12a4066690 0x7f12a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.596+0000 7f12c4c00640 1 -- 172.21.15.46:0/4164324360 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f12c0083ed0 con 0x7f12c815efe0 2024-03-29T08:18:59.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.600+0000 7f12c7e00640 1 --2- 172.21.15.46:0/4164324360 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12a4066690 0x7f12a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.600+0000 7f12c7e00640 1 --2- 172.21.15.46:0/4164324360 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12a4066690 0x7f12a4068b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f12b80ae000 tx=0x7f12b80a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:59.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.600+0000 7f12c4c00640 1 -- 172.21.15.46:0/4164324360 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f12c0054d40 con 0x7f12c815efe0 2024-03-29T08:18:59.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.764+0000 7f12cec00640 1 -- 172.21.15.46:0/4164324360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f12c8144cc0 con 0x7f12c815efe0 2024-03-29T08:18:59.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.764+0000 7f12c4c00640 1 -- 172.21.15.46:0/4164324360 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f12c00548d0 con 0x7f12c815efe0 2024-03-29T08:18:59.767 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:18:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.768+0000 7f12cec00640 1 -- 172.21.15.46:0/4164324360 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12a4066690 msgr2=0x7f12a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.768+0000 7f12cec00640 1 --2- 172.21.15.46:0/4164324360 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12a4066690 0x7f12a4068b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f12b80ae000 tx=0x7f12b80a7070 comp rx=0 tx=0).stop 2024-03-29T08:18:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.768+0000 7f12cec00640 1 -- 172.21.15.46:0/4164324360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c815efe0 msgr2=0x7f12c813ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.768+0000 7f12cec00640 1 --2- 172.21.15.46:0/4164324360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c815efe0 0x7f12c813ca20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f12c000c3f0 tx=0x7f12c000c8b0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.768+0000 7f12cec00640 1 -- 172.21.15.46:0/4164324360 shutdown_connections 2024-03-29T08:18:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.768+0000 7f12cec00640 1 --2- 172.21.15.46:0/4164324360 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12a4066690 0x7f12a4068b50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.768+0000 7f12cec00640 1 --2- 172.21.15.46:0/4164324360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12c815efe0 0x7f12c813ca20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.768+0000 7f12cec00640 1 --2- 172.21.15.46:0/4164324360 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12c8143eb0 0x7f12c8139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.768+0000 7f12cec00640 1 --2- 172.21.15.46:0/4164324360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c80b1630 0x7f12c8159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.768+0000 7f12cec00640 1 -- 172.21.15.46:0/4164324360 >> 172.21.15.46:0/4164324360 conn(0x7f12c806a390 msgr2=0x7f12c815c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.768+0000 7f12cec00640 1 -- 172.21.15.46:0/4164324360 shutdown_connections 2024-03-29T08:18:59.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.768+0000 7f12cec00640 1 -- 172.21.15.46:0/4164324360 wait complete. 2024-03-29T08:18:59.789 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:18:59.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.928+0000 7f2711000640 1 Processor -- start 2024-03-29T08:18:59.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.928+0000 7f2711000640 1 -- start start 2024-03-29T08:18:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.928+0000 7f2711000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f270c142ab0 0x7f270c06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.928+0000 7f2711000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f270c06d9e0 0x7f270c15d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.928+0000 7f2711000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f270c15d940 0x7f270c0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.928+0000 7f2711000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f270c06de40 con 0x7f270c142ab0 2024-03-29T08:18:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.928+0000 7f2711000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f270c070c70 con 0x7f270c15d940 2024-03-29T08:18:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.928+0000 7f2711000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270c070340 con 0x7f270c06d9e0 2024-03-29T08:18:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.928+0000 7f2709e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f270c142ab0 0x7f270c06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.928+0000 7f2701400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f270c06d9e0 0x7f270c15d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.928+0000 7f270a800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f270c15d940 0x7f270c0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.928+0000 7f2701400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f270c06d9e0 0x7f270c15d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:51448/0 (socket says 172.21.15.46:51448) 2024-03-29T08:18:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.928+0000 7f2701400640 1 -- 172.21.15.46:0/1648494035 learned_addr learned my addr 172.21.15.46:0/1648494035 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:59.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.928+0000 7f2709e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f270c142ab0 0x7f270c06d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55236/0 (socket says 172.21.15.46:55236) 2024-03-29T08:18:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.932+0000 7f2701400640 1 -- 172.21.15.46:0/1648494035 >> v2:172.21.15.73:3300/0 conn(0x7f270c15d940 msgr2=0x7f270c0ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.932+0000 7f2701400640 1 --2- 172.21.15.46:0/1648494035 >> v2:172.21.15.73:3300/0 conn(0x7f270c15d940 0x7f270c0ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.932+0000 7f2701400640 1 -- 172.21.15.46:0/1648494035 >> v2:172.21.15.46:3300/0 conn(0x7f270c142ab0 msgr2=0x7f270c06d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:18:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.932+0000 7f2701400640 1 --2- 172.21.15.46:0/1648494035 >> v2:172.21.15.46:3300/0 conn(0x7f270c142ab0 0x7f270c06d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.932+0000 7f2701400640 1 -- 172.21.15.46:0/1648494035 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270c070620 con 0x7f270c06d9e0 2024-03-29T08:18:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.932+0000 7f270a800640 1 --2- 172.21.15.46:0/1648494035 >> v2:172.21.15.73:3300/0 conn(0x7f270c15d940 0x7f270c0ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.932+0000 7f2709e00640 1 --2- 172.21.15.46:0/1648494035 >> v2:172.21.15.46:3300/0 conn(0x7f270c142ab0 0x7f270c06d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:18:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.932+0000 7f2701400640 1 --2- 172.21.15.46:0/1648494035 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f270c06d9e0 0x7f270c15d400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f270c070be0 tx=0x7f26f401e470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=540b14bef6d31904 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:59.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.932+0000 7f2709400640 1 -- 172.21.15.46:0/1648494035 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f40ad600 con 0x7f270c06d9e0 2024-03-29T08:18:59.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.932+0000 7f2709400640 1 -- 172.21.15.46:0/1648494035 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f40a9050 con 0x7f270c06d9e0 2024-03-29T08:18:59.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.932+0000 7f2709400640 1 -- 172.21.15.46:0/1648494035 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f40ad980 con 0x7f270c06d9e0 2024-03-29T08:18:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2711000640 1 -- 172.21.15.46:0/1648494035 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f270c06d9e0 msgr2=0x7f270c15d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2711000640 1 --2- 172.21.15.46:0/1648494035 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f270c06d9e0 0x7f270c15d400 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f270c070be0 tx=0x7f26f401e470 comp rx=0 tx=0).stop 2024-03-29T08:18:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2711000640 1 -- 172.21.15.46:0/1648494035 shutdown_connections 2024-03-29T08:18:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2711000640 1 --2- 172.21.15.46:0/1648494035 >> v2:172.21.15.73:3300/0 conn(0x7f270c15d940 0x7f270c0ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2711000640 1 --2- 172.21.15.46:0/1648494035 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f270c06d9e0 0x7f270c15d400 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2711000640 1 --2- 172.21.15.46:0/1648494035 >> v2:172.21.15.46:3300/0 conn(0x7f270c142ab0 0x7f270c06d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2711000640 1 -- 172.21.15.46:0/1648494035 >> 172.21.15.46:0/1648494035 conn(0x7f270c066080 msgr2=0x7f270c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:18:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2711000640 1 -- 172.21.15.46:0/1648494035 shutdown_connections 2024-03-29T08:18:59.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2711000640 1 -- 172.21.15.46:0/1648494035 wait complete. 2024-03-29T08:18:59.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2711000640 1 Processor -- start 2024-03-29T08:18:59.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2711000640 1 -- start start 2024-03-29T08:18:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2711000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f270c06d9e0 0x7f270c13e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2711000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f270c142ab0 0x7f270c0a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2711000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f270c15d940 0x7f270c1f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2711000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c13ed80 con 0x7f270c15d940 2024-03-29T08:18:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2711000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270c13e3b0 con 0x7f270c142ab0 2024-03-29T08:18:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2711000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f270c13d7a0 con 0x7f270c06d9e0 2024-03-29T08:18:59.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f270a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f270c15d940 0x7f270c1f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f270a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f270c15d940 0x7f270c1f8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55250/0 (socket says 172.21.15.46:55250) 2024-03-29T08:18:59.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f270a800640 1 -- 172.21.15.46:0/4121307703 learned_addr learned my addr 172.21.15.46:0/4121307703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:18:59.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2709e00640 1 --2- 172.21.15.46:0/4121307703 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f270c06d9e0 0x7f270c13e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f2701400640 1 --2- 172.21.15.46:0/4121307703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f270c142ab0 0x7f270c0a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f270a800640 1 -- 172.21.15.46:0/4121307703 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f270c06d9e0 msgr2=0x7f270c13e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:59.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f270a800640 1 --2- 172.21.15.46:0/4121307703 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f270c06d9e0 0x7f270c13e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f270a800640 1 -- 172.21.15.46:0/4121307703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f270c142ab0 msgr2=0x7f270c0a74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:18:59.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f270a800640 1 --2- 172.21.15.46:0/4121307703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f270c142ab0 0x7f270c0a74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:18:59.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.936+0000 7f270a800640 1 -- 172.21.15.46:0/4121307703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26fc009420 con 0x7f270c15d940 2024-03-29T08:18:59.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.940+0000 7f270a800640 1 --2- 172.21.15.46:0/4121307703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f270c15d940 0x7f270c1f8d90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f270400efc0 tx=0x7f270400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:59.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.940+0000 7f2702a00640 1 -- 172.21.15.46:0/4121307703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2704009cd0 con 0x7f270c15d940 2024-03-29T08:18:59.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.940+0000 7f2702a00640 1 -- 172.21.15.46:0/4121307703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270400f040 con 0x7f270c15d940 2024-03-29T08:18:59.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.940+0000 7f2702a00640 1 -- 172.21.15.46:0/4121307703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27040039a0 con 0x7f270c15d940 2024-03-29T08:18:59.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.940+0000 7f2711000640 1 -- 172.21.15.46:0/4121307703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2704009a70 con 0x7f270c15d940 2024-03-29T08:18:59.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.940+0000 7f2711000640 1 -- 172.21.15.46:0/4121307703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f40094d0 con 0x7f270c15d940 2024-03-29T08:18:59.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.940+0000 7f2711000640 1 -- 172.21.15.46:0/4121307703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f270c13e540 con 0x7f270c15d940 2024-03-29T08:18:59.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.940+0000 7f2702a00640 1 -- 172.21.15.46:0/4121307703 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2704014480 con 0x7f270c15d940 2024-03-29T08:18:59.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.940+0000 7f2702a00640 1 --2- 172.21.15.46:0/4121307703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f26e8066690 0x7f26e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:18:59.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.940+0000 7f2702a00640 1 -- 172.21.15.46:0/4121307703 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2704084cf0 con 0x7f270c15d940 2024-03-29T08:18:59.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.940+0000 7f2709e00640 1 --2- 172.21.15.46:0/4121307703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f26e8066690 0x7f26e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:18:59.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.944+0000 7f2709e00640 1 --2- 172.21.15.46:0/4121307703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f26e8066690 0x7f26e8068b50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009cf0 tx=0x7f26fc009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:18:59.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:18:59.944+0000 7f2702a00640 1 -- 172.21.15.46:0/4121307703 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2704052050 con 0x7f270c15d940 2024-03-29T08:19:00.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.120+0000 7f2711000640 1 -- 172.21.15.46:0/4121307703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f270c071a90 con 0x7f270c15d940 2024-03-29T08:19:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.120+0000 7f2702a00640 1 -- 172.21.15.46:0/4121307703 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2704018080 con 0x7f270c15d940 2024-03-29T08:19:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:19:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.120+0000 7f2711000640 1 -- 172.21.15.46:0/4121307703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f26e8066690 msgr2=0x7f26e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.120+0000 7f2711000640 1 --2- 172.21.15.46:0/4121307703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f26e8066690 0x7f26e8068b50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f26fc009cf0 tx=0x7f26fc009190 comp rx=0 tx=0).stop 2024-03-29T08:19:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.120+0000 7f2711000640 1 -- 172.21.15.46:0/4121307703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f270c15d940 msgr2=0x7f270c1f8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.120+0000 7f2711000640 1 --2- 172.21.15.46:0/4121307703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f270c15d940 0x7f270c1f8d90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f270400efc0 tx=0x7f270400c490 comp rx=0 tx=0).stop 2024-03-29T08:19:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.120+0000 7f2711000640 1 -- 172.21.15.46:0/4121307703 shutdown_connections 2024-03-29T08:19:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.120+0000 7f2711000640 1 --2- 172.21.15.46:0/4121307703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f26e8066690 0x7f26e8068b50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.120+0000 7f2711000640 1 --2- 172.21.15.46:0/4121307703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f270c142ab0 0x7f270c0a74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.120+0000 7f2711000640 1 --2- 172.21.15.46:0/4121307703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f270c15d940 0x7f270c1f8d90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.120+0000 7f2711000640 1 --2- 172.21.15.46:0/4121307703 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f270c06d9e0 0x7f270c13e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.120+0000 7f2711000640 1 -- 172.21.15.46:0/4121307703 >> 172.21.15.46:0/4121307703 conn(0x7f270c066080 msgr2=0x7f270c0cd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.120+0000 7f2711000640 1 -- 172.21.15.46:0/4121307703 shutdown_connections 2024-03-29T08:19:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.120+0000 7f2711000640 1 -- 172.21.15.46:0/4121307703 wait complete. 2024-03-29T08:19:00.142 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi073.front.sepia.ceph.com... 2024-03-29T08:19:00.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:00.142 DEBUG:teuthology.orchestra.run.smithi073:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-03-29T08:19:00.164 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:19:00.165 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:19:00.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:00.165 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-29T08:19:00.217 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-29T08:19:00.217 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:19:00.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.356+0000 7f23d1e00640 1 Processor -- start 2024-03-29T08:19:00.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.356+0000 7f23d1e00640 1 -- start start 2024-03-29T08:19:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.356+0000 7f23d1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f23cc15ccb0 0x7f23cc06bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.356+0000 7f23d1e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23cc06c000 0x7f23cc0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.356+0000 7f23d1e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23cc0d0ff0 con 0x7f23cc0d0ba0 2024-03-29T08:19:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.356+0000 7f23d1e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23cc0b14c0 con 0x7f23cc15ccb0 2024-03-29T08:19:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.356+0000 7f23d1e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc15d090 con 0x7f23cc06c000 2024-03-29T08:19:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.356+0000 7f23caa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f23cc15ccb0 0x7f23cc06bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.356+0000 7f23caa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f23cc15ccb0 0x7f23cc06bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55264/0 (socket says 172.21.15.46:55264) 2024-03-29T08:19:00.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.356+0000 7f23caa00640 1 -- 172.21.15.46:0/1204645367 learned_addr learned my addr 172.21.15.46:0/1204645367 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:00.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.356+0000 7f23c2000640 1 --2- 172.21.15.46:0/1204645367 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23cc06c000 0x7f23cc0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:00.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.356+0000 7f23caa00640 1 -- 172.21.15.46:0/1204645367 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23cc06c000 msgr2=0x7f23cc0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:00.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.356+0000 7f23caa00640 1 --2- 172.21.15.46:0/1204645367 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23cc06c000 0x7f23cc0d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.356+0000 7f23caa00640 1 -- 172.21.15.46:0/1204645367 >> v1:172.21.15.73:6789/0 conn(0x7f23cc0d0ba0 legacy=0x7f23cc0ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:00.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.356+0000 7f23caa00640 1 -- 172.21.15.46:0/1204645367 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc0b08b0 con 0x7f23cc15ccb0 2024-03-29T08:19:00.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.360+0000 7f23caa00640 1 --2- 172.21.15.46:0/1204645367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23cc15ccb0 0x7f23cc06bac0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009870 tx=0x7f23bc01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3304d42537f53af9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:00.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.360+0000 7f23ca000640 1 -- 172.21.15.46:0/1204645367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0a8a50 con 0x7f23cc15ccb0 2024-03-29T08:19:00.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.360+0000 7f23ca000640 1 -- 172.21.15.46:0/1204645367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc0a8bb0 con 0x7f23cc15ccb0 2024-03-29T08:19:00.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.360+0000 7f23ca000640 1 -- 172.21.15.46:0/1204645367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0b0440 con 0x7f23cc15ccb0 2024-03-29T08:19:00.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.360+0000 7f23d1e00640 1 -- 172.21.15.46:0/1204645367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23cc15ccb0 msgr2=0x7f23cc06bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:00.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.360+0000 7f23d1e00640 1 --2- 172.21.15.46:0/1204645367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23cc15ccb0 0x7f23cc06bac0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009870 tx=0x7f23bc01ed50 comp rx=0 tx=0).stop 2024-03-29T08:19:00.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.360+0000 7f23d1e00640 1 -- 172.21.15.46:0/1204645367 shutdown_connections 2024-03-29T08:19:00.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.360+0000 7f23d1e00640 1 --2- 172.21.15.46:0/1204645367 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23cc06c000 0x7f23cc0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.360+0000 7f23d1e00640 1 --2- 172.21.15.46:0/1204645367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23cc15ccb0 0x7f23cc06bac0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.360+0000 7f23d1e00640 1 -- 172.21.15.46:0/1204645367 >> 172.21.15.46:0/1204645367 conn(0x7f23cc089fd0 msgr2=0x7f23cc08a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:00.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.360+0000 7f23d1e00640 1 -- 172.21.15.46:0/1204645367 shutdown_connections 2024-03-29T08:19:00.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.360+0000 7f23d1e00640 1 -- 172.21.15.46:0/1204645367 wait complete. 2024-03-29T08:19:00.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23d1e00640 1 Processor -- start 2024-03-29T08:19:00.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23d1e00640 1 -- start start 2024-03-29T08:19:00.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23d1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23cc06c000 0x7f23cc1596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:00.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23d1e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23cc0d0ba0 0x7f23cc15f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:00.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23d1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23cc15ccb0 0x7f23cc06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23d1e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc1f2dd0 con 0x7f23cc06c000 2024-03-29T08:19:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23d1e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc1f1f00 con 0x7f23cc15ccb0 2024-03-29T08:19:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23d1e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc1f21c0 con 0x7f23cc0d0ba0 2024-03-29T08:19:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23caa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23cc06c000 0x7f23cc1596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23caa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23cc06c000 0x7f23cc1596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55280/0 (socket says 172.21.15.46:55280) 2024-03-29T08:19:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23caa00640 1 -- 172.21.15.46:0/2376697098 learned_addr learned my addr 172.21.15.46:0/2376697098 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23cb400640 1 --2- 172.21.15.46:0/2376697098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23cc15ccb0 0x7f23cc06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23c2000640 1 --2- 172.21.15.46:0/2376697098 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23cc0d0ba0 0x7f23cc15f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23caa00640 1 -- 172.21.15.46:0/2376697098 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23cc0d0ba0 msgr2=0x7f23cc15f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23caa00640 1 --2- 172.21.15.46:0/2376697098 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23cc0d0ba0 0x7f23cc15f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23caa00640 1 -- 172.21.15.46:0/2376697098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23cc15ccb0 msgr2=0x7f23cc06f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23caa00640 1 --2- 172.21.15.46:0/2376697098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23cc15ccb0 0x7f23cc06f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23caa00640 1 -- 172.21.15.46:0/2376697098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc0094d0 con 0x7f23cc06c000 2024-03-29T08:19:00.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23caa00640 1 --2- 172.21.15.46:0/2376697098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23cc06c000 0x7f23cc1596f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0b3040 tx=0x7f23bc01baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:00.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23c3e00640 1 -- 172.21.15.46:0/2376697098 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0be070 con 0x7f23cc06c000 2024-03-29T08:19:00.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23c3e00640 1 -- 172.21.15.46:0/2376697098 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc0ba400 con 0x7f23cc06c000 2024-03-29T08:19:00.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23d1e00640 1 -- 172.21.15.46:0/2376697098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23cc1f24a0 con 0x7f23cc06c000 2024-03-29T08:19:00.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23c3e00640 1 -- 172.21.15.46:0/2376697098 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc0ba7d0 con 0x7f23cc06c000 2024-03-29T08:19:00.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23d1e00640 1 -- 172.21.15.46:0/2376697098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc1f2a60 con 0x7f23cc06c000 2024-03-29T08:19:00.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.364+0000 7f23d1e00640 1 -- 172.21.15.46:0/2376697098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23cc066c90 con 0x7f23cc06c000 2024-03-29T08:19:00.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.368+0000 7f23c3e00640 1 -- 172.21.15.46:0/2376697098 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f23bc0c3410 con 0x7f23cc06c000 2024-03-29T08:19:00.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.368+0000 7f23c3e00640 1 --2- 172.21.15.46:0/2376697098 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23a8066690 0x7f23a8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:00.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.368+0000 7f23c3e00640 1 -- 172.21.15.46:0/2376697098 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f23bc121c80 con 0x7f23cc06c000 2024-03-29T08:19:00.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.368+0000 7f23c2000640 1 --2- 172.21.15.46:0/2376697098 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23a8066690 0x7f23a8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:00.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.368+0000 7f23c2000640 1 --2- 172.21.15.46:0/2376697098 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23a8066690 0x7f23a8068b50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f23cc08a780 tx=0x7f23b4009830 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:00.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.372+0000 7f23c3e00640 1 -- 172.21.15.46:0/2376697098 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f23bc0ef080 con 0x7f23cc06c000 2024-03-29T08:19:00.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.568+0000 7f23d1e00640 1 -- 172.21.15.46:0/2376697098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23cc0b2760 con 0x7f23cc06c000 2024-03-29T08:19:00.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.568+0000 7f23c3e00640 1 -- 172.21.15.46:0/2376697098 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4722 (secure 0 0 0) 0x7f23bc0f26a0 con 0x7f23cc06c000 2024-03-29T08:19:00.570 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:00.570 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/339038392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":339038392},{"type":"v1","addr":"172.21.15.46:6834","nonce":339038392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3573036715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3573036715},{"type":"v1","addr":"172.21.15.73:6835","nonce":3573036715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/746744819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":746744819},{"type":"v1","addr":"172.21.15.46:6835","nonce":746744819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:18:13.311457+0000","modified":"2024-03-29T08:18:16.321300+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2780269530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2780269530},{"type":"v1","addr":"172.21.15.73:6837","nonce":2780269530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-29T08:19:00.570 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-03-29T08:19:00.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.568+0000 7f23d1e00640 1 -- 172.21.15.46:0/2376697098 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23a8066690 msgr2=0x7f23a8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:00.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.568+0000 7f23d1e00640 1 --2- 172.21.15.46:0/2376697098 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23a8066690 0x7f23a8068b50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f23cc08a780 tx=0x7f23b4009830 comp rx=0 tx=0).stop 2024-03-29T08:19:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.568+0000 7f23d1e00640 1 -- 172.21.15.46:0/2376697098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23cc06c000 msgr2=0x7f23cc1596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.568+0000 7f23d1e00640 1 --2- 172.21.15.46:0/2376697098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23cc06c000 0x7f23cc1596f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0b3040 tx=0x7f23bc01baf0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.568+0000 7f23d1e00640 1 -- 172.21.15.46:0/2376697098 shutdown_connections 2024-03-29T08:19:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.568+0000 7f23d1e00640 1 --2- 172.21.15.46:0/2376697098 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23a8066690 0x7f23a8068b50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.568+0000 7f23d1e00640 1 --2- 172.21.15.46:0/2376697098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23cc15ccb0 0x7f23cc06f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.568+0000 7f23d1e00640 1 --2- 172.21.15.46:0/2376697098 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23cc0d0ba0 0x7f23cc15f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.568+0000 7f23d1e00640 1 --2- 172.21.15.46:0/2376697098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23cc06c000 0x7f23cc1596f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.568+0000 7f23d1e00640 1 -- 172.21.15.46:0/2376697098 >> 172.21.15.46:0/2376697098 conn(0x7f23cc089fd0 msgr2=0x7f23cc0ad900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.568+0000 7f23d1e00640 1 -- 172.21.15.46:0/2376697098 shutdown_connections 2024-03-29T08:19:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.568+0000 7f23d1e00640 1 -- 172.21.15.46:0/2376697098 wait complete. 2024-03-29T08:19:00.591 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:19:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.732+0000 7f5d66a00640 1 Processor -- start 2024-03-29T08:19:00.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.732+0000 7f5d66a00640 1 -- start start 2024-03-29T08:19:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.732+0000 7f5d66a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d6015a890 0x7f5d60098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.732+0000 7f5d66a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5d60099690 0x7f5d600b0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.732+0000 7f5d66a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d600cc2e0 con 0x7f5d600b2b30 2024-03-29T08:19:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.732+0000 7f5d66a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d600cce20 con 0x7f5d6015a890 2024-03-29T08:19:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.732+0000 7f5d66a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d600cc580 con 0x7f5d60099690 2024-03-29T08:19:00.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.732+0000 7f5d65600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d6015a890 0x7f5d60098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:00.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.732+0000 7f5d65600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5d6015a890 0x7f5d60098a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55290/0 (socket says 172.21.15.46:55290) 2024-03-29T08:19:00.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.732+0000 7f5d65600640 1 -- 172.21.15.46:0/809839075 learned_addr learned my addr 172.21.15.46:0/809839075 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:00.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.732+0000 7f5d66000640 1 --2- 172.21.15.46:0/809839075 >> v2:172.21.15.73:3300/0 conn(0x7f5d60099690 0x7f5d600b0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:00.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.732+0000 7f5d65600640 1 -- 172.21.15.46:0/809839075 >> v2:172.21.15.73:3300/0 conn(0x7f5d60099690 msgr2=0x7f5d600b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:00.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.732+0000 7f5d65600640 1 --2- 172.21.15.46:0/809839075 >> v2:172.21.15.73:3300/0 conn(0x7f5d60099690 0x7f5d600b0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.732+0000 7f5d65600640 1 -- 172.21.15.46:0/809839075 >> v1:172.21.15.73:6789/0 conn(0x7f5d600b2b30 legacy=0x7f5d60098f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:00.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.732+0000 7f5d65600640 1 -- 172.21.15.46:0/809839075 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d600cc860 con 0x7f5d6015a890 2024-03-29T08:19:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.736+0000 7f5d65600640 1 --2- 172.21.15.46:0/809839075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d6015a890 0x7f5d60098a40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5d6006dc70 tx=0x7f5d5401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc166ff0f9c8107f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.736+0000 7f5d5fe00640 1 -- 172.21.15.46:0/809839075 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d540ab600 con 0x7f5d6015a890 2024-03-29T08:19:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.736+0000 7f5d5fe00640 1 -- 172.21.15.46:0/809839075 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d540a7050 con 0x7f5d6015a890 2024-03-29T08:19:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.736+0000 7f5d5fe00640 1 -- 172.21.15.46:0/809839075 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d540ab980 con 0x7f5d6015a890 2024-03-29T08:19:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.736+0000 7f5d66a00640 1 -- 172.21.15.46:0/809839075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d6015a890 msgr2=0x7f5d60098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.736+0000 7f5d66a00640 1 --2- 172.21.15.46:0/809839075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d6015a890 0x7f5d60098a40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5d6006dc70 tx=0x7f5d5401c420 comp rx=0 tx=0).stop 2024-03-29T08:19:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.736+0000 7f5d66a00640 1 -- 172.21.15.46:0/809839075 shutdown_connections 2024-03-29T08:19:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.736+0000 7f5d66a00640 1 --2- 172.21.15.46:0/809839075 >> v2:172.21.15.73:3300/0 conn(0x7f5d60099690 0x7f5d600b0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.736+0000 7f5d66a00640 1 --2- 172.21.15.46:0/809839075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d6015a890 0x7f5d60098a40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.736+0000 7f5d66a00640 1 -- 172.21.15.46:0/809839075 >> 172.21.15.46:0/809839075 conn(0x7f5d600b2010 msgr2=0x7f5d600b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.736+0000 7f5d66a00640 1 -- 172.21.15.46:0/809839075 shutdown_connections 2024-03-29T08:19:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.736+0000 7f5d66a00640 1 -- 172.21.15.46:0/809839075 wait complete. 2024-03-29T08:19:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.736+0000 7f5d66a00640 1 Processor -- start 2024-03-29T08:19:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.736+0000 7f5d66a00640 1 -- start start 2024-03-29T08:19:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d60099690 0x7f5d600b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d600b2b30 0x7f5d600c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d6015a890 0x7f5d600c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d600b3c30 con 0x7f5d6015a890 2024-03-29T08:19:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d60159aa0 con 0x7f5d60099690 2024-03-29T08:19:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d60159d80 con 0x7f5d600b2b30 2024-03-29T08:19:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d6015a890 0x7f5d600c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d6015a890 0x7f5d600c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55300/0 (socket says 172.21.15.46:55300) 2024-03-29T08:19:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66000640 1 -- 172.21.15.46:0/4085083903 learned_addr learned my addr 172.21.15.46:0/4085083903 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d64c00640 1 --2- 172.21.15.46:0/4085083903 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d600b2b30 0x7f5d600c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d65600640 1 --2- 172.21.15.46:0/4085083903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d60099690 0x7f5d600b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:00.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66000640 1 -- 172.21.15.46:0/4085083903 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d600b2b30 msgr2=0x7f5d600c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:00.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66000640 1 --2- 172.21.15.46:0/4085083903 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d600b2b30 0x7f5d600c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66000640 1 -- 172.21.15.46:0/4085083903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d60099690 msgr2=0x7f5d600b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:00.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66000640 1 --2- 172.21.15.46:0/4085083903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d60099690 0x7f5d600b40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66000640 1 -- 172.21.15.46:0/4085083903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540094d0 con 0x7f5d6015a890 2024-03-29T08:19:00.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66000640 1 --2- 172.21.15.46:0/4085083903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d6015a890 0x7f5d600c7b30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5d58007e60 tx=0x7f5d5800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:00.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d5d600640 1 -- 172.21.15.46:0/4085083903 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58014420 con 0x7f5d6015a890 2024-03-29T08:19:00.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d5d600640 1 -- 172.21.15.46:0/4085083903 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d580145a0 con 0x7f5d6015a890 2024-03-29T08:19:00.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d5d600640 1 -- 172.21.15.46:0/4085083903 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58014950 con 0x7f5d6015a890 2024-03-29T08:19:00.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66a00640 1 -- 172.21.15.46:0/4085083903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d601581b0 con 0x7f5d6015a890 2024-03-29T08:19:00.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66a00640 1 -- 172.21.15.46:0/4085083903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d60152a90 con 0x7f5d6015a890 2024-03-29T08:19:00.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.740+0000 7f5d66a00640 1 -- 172.21.15.46:0/4085083903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d600b3da0 con 0x7f5d6015a890 2024-03-29T08:19:00.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.744+0000 7f5d5d600640 1 -- 172.21.15.46:0/4085083903 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5d58014ab0 con 0x7f5d6015a890 2024-03-29T08:19:00.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.744+0000 7f5d5d600640 1 --2- 172.21.15.46:0/4085083903 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d40066690 0x7f5d40068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:00.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.744+0000 7f5d5d600640 1 -- 172.21.15.46:0/4085083903 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5d58084b70 con 0x7f5d6015a890 2024-03-29T08:19:00.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.744+0000 7f5d65600640 1 --2- 172.21.15.46:0/4085083903 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d40066690 0x7f5d40068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.744+0000 7f5d65600640 1 --2- 172.21.15.46:0/4085083903 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d40066690 0x7f5d40068b50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5d540ad000 tx=0x7f5d540a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:00.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.748+0000 7f5d5d600640 1 -- 172.21.15.46:0/4085083903 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5d58051d90 con 0x7f5d6015a890 2024-03-29T08:19:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.908+0000 7f5d66a00640 1 -- 172.21.15.46:0/4085083903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d601583a0 con 0x7f5d6015a890 2024-03-29T08:19:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.908+0000 7f5d5d600640 1 -- 172.21.15.46:0/4085083903 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16042 (secure 0 0 0) 0x7f5d601583a0 con 0x7f5d6015a890 2024-03-29T08:19:00.910 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:00.910 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:56.021693+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:19:00.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.908+0000 7f5d66a00640 1 -- 172.21.15.46:0/4085083903 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d40066690 msgr2=0x7f5d40068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.908+0000 7f5d66a00640 1 --2- 172.21.15.46:0/4085083903 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d40066690 0x7f5d40068b50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5d540ad000 tx=0x7f5d540a6000 comp rx=0 tx=0).stop 2024-03-29T08:19:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.908+0000 7f5d66a00640 1 -- 172.21.15.46:0/4085083903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d6015a890 msgr2=0x7f5d600c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.908+0000 7f5d66a00640 1 --2- 172.21.15.46:0/4085083903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d6015a890 0x7f5d600c7b30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5d58007e60 tx=0x7f5d5800e4b0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.908+0000 7f5d66a00640 1 -- 172.21.15.46:0/4085083903 shutdown_connections 2024-03-29T08:19:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.908+0000 7f5d66a00640 1 --2- 172.21.15.46:0/4085083903 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d40066690 0x7f5d40068b50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.908+0000 7f5d66a00640 1 --2- 172.21.15.46:0/4085083903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d6015a890 0x7f5d600c7b30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.908+0000 7f5d66a00640 1 --2- 172.21.15.46:0/4085083903 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d600b2b30 0x7f5d600c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.908+0000 7f5d66a00640 1 --2- 172.21.15.46:0/4085083903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d60099690 0x7f5d600b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.908+0000 7f5d66a00640 1 -- 172.21.15.46:0/4085083903 >> 172.21.15.46:0/4085083903 conn(0x7f5d600b2010 msgr2=0x7f5d600ad3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.908+0000 7f5d66a00640 1 -- 172.21.15.46:0/4085083903 shutdown_connections 2024-03-29T08:19:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:00.908+0000 7f5d66a00640 1 -- 172.21.15.46:0/4085083903 wait complete. 2024-03-29T08:19:00.932 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:18:11.177842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:18:12.308252+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:19:00.933 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:19:01.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.076+0000 7f28eec00640 1 Processor -- start 2024-03-29T08:19:01.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.076+0000 7f28eec00640 1 -- start start 2024-03-29T08:19:01.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.076+0000 7f28eec00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f28e8143eb0 0x7f28e80cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.076+0000 7f28eec00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f28e815efe0 0x7f28e80cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.076+0000 7f28eec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e80b2360 con 0x7f28e80b1630 2024-03-29T08:19:01.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.076+0000 7f28eec00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e815e500 con 0x7f28e815efe0 2024-03-29T08:19:01.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.076+0000 7f28eec00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e815e7e0 con 0x7f28e8143eb0 2024-03-29T08:19:01.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.076+0000 7f28ecc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f28e80b1630 0x7f28e80b1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54752/0 (socket says 172.21.15.46:54752) 2024-03-29T08:19:01.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.076+0000 7f28ecc00640 1 -- 172.21.15.46:0/2861599398 learned_addr learned my addr 172.21.15.46:0/2861599398 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:01.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.076+0000 7f28e7e00640 1 --2- 172.21.15.46:0/2861599398 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f28e8143eb0 0x7f28e80cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.080+0000 7f28dfe00640 1 --2- 172.21.15.46:0/2861599398 >> v2:172.21.15.73:3300/0 conn(0x7f28e815efe0 0x7f28e80cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.080+0000 7f28e7400640 1 -- 172.21.15.46:0/2861599398 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386039606 0 0) 0x7f28e80b2360 con 0x7f28e80b1630 2024-03-29T08:19:01.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.080+0000 7f28e7400640 1 -- 172.21.15.46:0/2861599398 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28cc003530 con 0x7f28e80b1630 2024-03-29T08:19:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.080+0000 7f28e7e00640 1 -- 172.21.15.46:0/2861599398 >> v2:172.21.15.73:3300/0 conn(0x7f28e815efe0 msgr2=0x7f28e80cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.080+0000 7f28e7e00640 1 --2- 172.21.15.46:0/2861599398 >> v2:172.21.15.73:3300/0 conn(0x7f28e815efe0 0x7f28e80cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.080+0000 7f28e7e00640 1 -- 172.21.15.46:0/2861599398 >> v1:172.21.15.46:6789/0 conn(0x7f28e80b1630 legacy=0x7f28e80b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.080+0000 7f28e7e00640 1 -- 172.21.15.46:0/2861599398 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e815eac0 con 0x7f28e8143eb0 2024-03-29T08:19:01.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.080+0000 7f28dfe00640 1 --2- 172.21.15.46:0/2861599398 >> v2:172.21.15.73:3300/0 conn(0x7f28e815efe0 0x7f28e80cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:01.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.080+0000 7f28e7e00640 1 --2- 172.21.15.46:0/2861599398 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f28e8143eb0 0x7f28e80cb790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f28d80097c0 tx=0x7f28d801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9afd5b96731ce787 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.080+0000 7f28e7400640 1 -- 172.21.15.46:0/2861599398 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d80ab600 con 0x7f28e8143eb0 2024-03-29T08:19:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.080+0000 7f28e7400640 1 -- 172.21.15.46:0/2861599398 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d80a7050 con 0x7f28e8143eb0 2024-03-29T08:19:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.080+0000 7f28e7400640 1 -- 172.21.15.46:0/2861599398 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d80ab960 con 0x7f28e8143eb0 2024-03-29T08:19:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28eec00640 1 -- 172.21.15.46:0/2861599398 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f28e8143eb0 msgr2=0x7f28e80cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28eec00640 1 --2- 172.21.15.46:0/2861599398 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f28e8143eb0 0x7f28e80cb790 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f28d80097c0 tx=0x7f28d801c4a0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28eec00640 1 -- 172.21.15.46:0/2861599398 shutdown_connections 2024-03-29T08:19:01.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28eec00640 1 --2- 172.21.15.46:0/2861599398 >> v2:172.21.15.73:3300/0 conn(0x7f28e815efe0 0x7f28e80cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28eec00640 1 --2- 172.21.15.46:0/2861599398 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f28e8143eb0 0x7f28e80cb790 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28eec00640 1 -- 172.21.15.46:0/2861599398 >> 172.21.15.46:0/2861599398 conn(0x7f28e806a390 msgr2=0x7f28e80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28eec00640 1 -- 172.21.15.46:0/2861599398 shutdown_connections 2024-03-29T08:19:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28eec00640 1 -- 172.21.15.46:0/2861599398 wait complete. 2024-03-29T08:19:01.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28eec00640 1 Processor -- start 2024-03-29T08:19:01.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28eec00640 1 -- start start 2024-03-29T08:19:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28eec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e80b1630 0x7f28e812e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28eec00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f28e8143eb0 0x7f28e80cd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28eec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28e815efe0 0x7f28e81f8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28eec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e812e500 con 0x7f28e80b1630 2024-03-29T08:19:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28eec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e8132170 con 0x7f28e815efe0 2024-03-29T08:19:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28eec00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e81f9440 con 0x7f28e8143eb0 2024-03-29T08:19:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e80b1630 0x7f28e812e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e80b1630 0x7f28e812e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55314/0 (socket says 172.21.15.46:55314) 2024-03-29T08:19:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28e7e00640 1 -- 172.21.15.46:0/3452493311 learned_addr learned my addr 172.21.15.46:0/3452493311 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28ecc00640 1 --2- 172.21.15.46:0/3452493311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28e815efe0 0x7f28e81f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28dfe00640 1 --2- 172.21.15.46:0/3452493311 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f28e8143eb0 0x7f28e80cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28e7e00640 1 -- 172.21.15.46:0/3452493311 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f28e8143eb0 msgr2=0x7f28e80cd820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28e7e00640 1 --2- 172.21.15.46:0/3452493311 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f28e8143eb0 0x7f28e80cd820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28e7e00640 1 -- 172.21.15.46:0/3452493311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28e815efe0 msgr2=0x7f28e81f8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28e7e00640 1 --2- 172.21.15.46:0/3452493311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28e815efe0 0x7f28e81f8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.084+0000 7f28e7e00640 1 -- 172.21.15.46:0/3452493311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d0009420 con 0x7f28e80b1630 2024-03-29T08:19:01.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.088+0000 7f28e7e00640 1 --2- 172.21.15.46:0/3452493311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e80b1630 0x7f28e812e990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f28d8000c10 tx=0x7f28d8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:01.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.088+0000 7f28e4c00640 1 -- 172.21.15.46:0/3452493311 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d80b9070 con 0x7f28e80b1630 2024-03-29T08:19:01.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.088+0000 7f28e4c00640 1 -- 172.21.15.46:0/3452493311 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d8004500 con 0x7f28e80b1630 2024-03-29T08:19:01.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.088+0000 7f28e4c00640 1 -- 172.21.15.46:0/3452493311 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d8003de0 con 0x7f28e80b1630 2024-03-29T08:19:01.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.088+0000 7f28eec00640 1 -- 172.21.15.46:0/3452493311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d8009420 con 0x7f28e80b1630 2024-03-29T08:19:01.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.088+0000 7f28eec00640 1 -- 172.21.15.46:0/3452493311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e812deb0 con 0x7f28e80b1630 2024-03-29T08:19:01.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.088+0000 7f28eec00640 1 -- 172.21.15.46:0/3452493311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e8071a70 con 0x7f28e80b1630 2024-03-29T08:19:01.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.088+0000 7f28e4c00640 1 -- 172.21.15.46:0/3452493311 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f28d80b2040 con 0x7f28e80b1630 2024-03-29T08:19:01.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.088+0000 7f28e4c00640 1 --2- 172.21.15.46:0/3452493311 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f28c4066690 0x7f28c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.088+0000 7f28e4c00640 1 -- 172.21.15.46:0/3452493311 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f28d811dba0 con 0x7f28e80b1630 2024-03-29T08:19:01.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.088+0000 7f28dfe00640 1 --2- 172.21.15.46:0/3452493311 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f28c4066690 0x7f28c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.092+0000 7f28dfe00640 1 --2- 172.21.15.46:0/3452493311 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f28c4066690 0x7f28c4068b50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f28e8143610 tx=0x7f28d0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:01.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.092+0000 7f28e4c00640 1 -- 172.21.15.46:0/3452493311 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f28d80eae50 con 0x7f28e80b1630 2024-03-29T08:19:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.260+0000 7f28eec00640 1 -- 172.21.15.46:0/3452493311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28e815df60 con 0x7f28e80b1630 2024-03-29T08:19:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.260+0000 7f28e4c00640 1 -- 172.21.15.46:0/3452493311 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f28d80b0020 con 0x7f28e80b1630 2024-03-29T08:19:01.261 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:19:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.260+0000 7f28eec00640 1 -- 172.21.15.46:0/3452493311 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f28c4066690 msgr2=0x7f28c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.260+0000 7f28eec00640 1 --2- 172.21.15.46:0/3452493311 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f28c4066690 0x7f28c4068b50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f28e8143610 tx=0x7f28d0008040 comp rx=0 tx=0).stop 2024-03-29T08:19:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.260+0000 7f28eec00640 1 -- 172.21.15.46:0/3452493311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e80b1630 msgr2=0x7f28e812e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.260+0000 7f28eec00640 1 --2- 172.21.15.46:0/3452493311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e80b1630 0x7f28e812e990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f28d8000c10 tx=0x7f28d8019240 comp rx=0 tx=0).stop 2024-03-29T08:19:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.260+0000 7f28eec00640 1 -- 172.21.15.46:0/3452493311 shutdown_connections 2024-03-29T08:19:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.260+0000 7f28eec00640 1 --2- 172.21.15.46:0/3452493311 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f28c4066690 0x7f28c4068b50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.260+0000 7f28eec00640 1 --2- 172.21.15.46:0/3452493311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f28e815efe0 0x7f28e81f8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.260+0000 7f28eec00640 1 --2- 172.21.15.46:0/3452493311 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f28e8143eb0 0x7f28e80cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.260+0000 7f28eec00640 1 --2- 172.21.15.46:0/3452493311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28e80b1630 0x7f28e812e990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.260+0000 7f28eec00640 1 -- 172.21.15.46:0/3452493311 >> 172.21.15.46:0/3452493311 conn(0x7f28e806a390 msgr2=0x7f28e80b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:01.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.260+0000 7f28eec00640 1 -- 172.21.15.46:0/3452493311 shutdown_connections 2024-03-29T08:19:01.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.260+0000 7f28eec00640 1 -- 172.21.15.46:0/3452493311 wait complete. 2024-03-29T08:19:01.281 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:19:01.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.424+0000 7fcec5000640 1 Processor -- start 2024-03-29T08:19:01.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.424+0000 7fcec5000640 1 -- start start 2024-03-29T08:19:01.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.428+0000 7fcec5000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcec00ceaf0 0x7fcec0098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.428+0000 7fcec5000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec00cfe50 con 0x7fcec015c9c0 2024-03-29T08:19:01.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.428+0000 7fcec5000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec00993a0 con 0x7fcec0098f80 2024-03-29T08:19:01.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.428+0000 7fcec5000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec00cef80 con 0x7fcec00ceaf0 2024-03-29T08:19:01.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.428+0000 7fcebde00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcec015c9c0 0x7fcec006be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54762/0 (socket says 172.21.15.46:54762) 2024-03-29T08:19:01.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.428+0000 7fcebde00640 1 -- 172.21.15.46:0/2330185420 learned_addr learned my addr 172.21.15.46:0/2330185420 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:01.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.428+0000 7fceb5400640 1 --2- 172.21.15.46:0/2330185420 >> v2:172.21.15.46:3300/0 conn(0x7fcec00ceaf0 0x7fcec0098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.428+0000 7fcebd400640 1 -- 172.21.15.46:0/2330185420 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198398805 0 0) 0x7fcec00cfe50 con 0x7fcec015c9c0 2024-03-29T08:19:01.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.428+0000 7fcebd400640 1 -- 172.21.15.46:0/2330185420 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcea4003530 con 0x7fcec015c9c0 2024-03-29T08:19:01.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.428+0000 7fceb5400640 1 -- 172.21.15.46:0/2330185420 >> v1:172.21.15.73:6789/0 conn(0x7fcec0098f80 legacy=0x7fcec00cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.428+0000 7fceb5400640 1 -- 172.21.15.46:0/2330185420 >> v1:172.21.15.46:6789/0 conn(0x7fcec015c9c0 legacy=0x7fcec006be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.428+0000 7fceb5400640 1 -- 172.21.15.46:0/2330185420 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec00cf240 con 0x7fcec00ceaf0 2024-03-29T08:19:01.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.432+0000 7fceb5400640 1 --2- 172.21.15.46:0/2330185420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec00ceaf0 0x7fcec0098a40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcea80097c0 tx=0x7fcea801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a75cba9cd46f510 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:01.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.432+0000 7fcebd400640 1 -- 172.21.15.46:0/2330185420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea80a8050 con 0x7fcec00ceaf0 2024-03-29T08:19:01.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.432+0000 7fcebd400640 1 -- 172.21.15.46:0/2330185420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea80ac5f0 con 0x7fcec00ceaf0 2024-03-29T08:19:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.432+0000 7fcebd400640 1 -- 172.21.15.46:0/2330185420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea80ac980 con 0x7fcec00ceaf0 2024-03-29T08:19:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.432+0000 7fcec5000640 1 -- 172.21.15.46:0/2330185420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec00ceaf0 msgr2=0x7fcec0098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.432+0000 7fcec5000640 1 --2- 172.21.15.46:0/2330185420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec00ceaf0 0x7fcec0098a40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcea80097c0 tx=0x7fcea801c4a0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.432+0000 7fcec5000640 1 -- 172.21.15.46:0/2330185420 shutdown_connections 2024-03-29T08:19:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.432+0000 7fcec5000640 1 --2- 172.21.15.46:0/2330185420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec00ceaf0 0x7fcec0098a40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.432+0000 7fcec5000640 1 -- 172.21.15.46:0/2330185420 >> 172.21.15.46:0/2330185420 conn(0x7fcec00b2220 msgr2=0x7fcec00b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.432+0000 7fcec5000640 1 -- 172.21.15.46:0/2330185420 shutdown_connections 2024-03-29T08:19:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.432+0000 7fcec5000640 1 -- 172.21.15.46:0/2330185420 wait complete. 2024-03-29T08:19:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.432+0000 7fcec5000640 1 Processor -- start 2024-03-29T08:19:01.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.432+0000 7fcec5000640 1 -- start start 2024-03-29T08:19:01.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fcec5000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcec0098f80 0x7fcec00b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fcec5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec00ceaf0 0x7fcec00b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fcec5000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcec015c9c0 0x7fcec0153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fcec5000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00b33f0 con 0x7fcec00ceaf0 2024-03-29T08:19:01.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fcec5000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00b36d0 con 0x7fcec0098f80 2024-03-29T08:19:01.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fcec5000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec00b39b0 con 0x7fcec015c9c0 2024-03-29T08:19:01.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fceb5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec00ceaf0 0x7fcec00b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fceb5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec00ceaf0 0x7fcec00b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55328/0 (socket says 172.21.15.46:55328) 2024-03-29T08:19:01.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fceb5400640 1 -- 172.21.15.46:0/3836378298 learned_addr learned my addr 172.21.15.46:0/3836378298 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:01.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fcebe800640 1 --2- 172.21.15.46:0/3836378298 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcec015c9c0 0x7fcec0153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fcebde00640 1 --2- 172.21.15.46:0/3836378298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcec0098f80 0x7fcec00b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fceb5400640 1 -- 172.21.15.46:0/3836378298 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcec015c9c0 msgr2=0x7fcec0153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fceb5400640 1 --2- 172.21.15.46:0/3836378298 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcec015c9c0 0x7fcec0153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fceb5400640 1 -- 172.21.15.46:0/3836378298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcec0098f80 msgr2=0x7fcec00b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fceb5400640 1 --2- 172.21.15.46:0/3836378298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcec0098f80 0x7fcec00b5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fceb5400640 1 -- 172.21.15.46:0/3836378298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea8009420 con 0x7fcec00ceaf0 2024-03-29T08:19:01.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fceb5400640 1 --2- 172.21.15.46:0/3836378298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec00ceaf0 0x7fcec00b4ff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcea80af040 tx=0x7fcea8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:01.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fceb6a00640 1 -- 172.21.15.46:0/3836378298 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8006e20 con 0x7fcec00ceaf0 2024-03-29T08:19:01.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fceb6a00640 1 -- 172.21.15.46:0/3836378298 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea80b5620 con 0x7fcec00ceaf0 2024-03-29T08:19:01.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fceb6a00640 1 -- 172.21.15.46:0/3836378298 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea80b59f0 con 0x7fcec00ceaf0 2024-03-29T08:19:01.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fcec5000640 1 -- 172.21.15.46:0/3836378298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec00b3c90 con 0x7fcec00ceaf0 2024-03-29T08:19:01.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fcec5000640 1 -- 172.21.15.46:0/3836378298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec0154120 con 0x7fcec00ceaf0 2024-03-29T08:19:01.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fcec5000640 1 -- 172.21.15.46:0/3836378298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec009e700 con 0x7fcec00ceaf0 2024-03-29T08:19:01.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.436+0000 7fceb6a00640 1 -- 172.21.15.46:0/3836378298 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcea80b3040 con 0x7fcec00ceaf0 2024-03-29T08:19:01.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.440+0000 7fceb6a00640 1 --2- 172.21.15.46:0/3836378298 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce9c066690 0x7fce9c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.440+0000 7fceb6a00640 1 -- 172.21.15.46:0/3836378298 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fcea811e260 con 0x7fcec00ceaf0 2024-03-29T08:19:01.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.440+0000 7fcebde00640 1 --2- 172.21.15.46:0/3836378298 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce9c066690 0x7fce9c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.440+0000 7fcebde00640 1 --2- 172.21.15.46:0/3836378298 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce9c066690 0x7fce9c068b50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fceb00011f0 tx=0x7fceb0007d40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:01.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.444+0000 7fceb6a00640 1 -- 172.21.15.46:0/3836378298 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcea80eb650 con 0x7fcec00ceaf0 2024-03-29T08:19:01.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.604+0000 7fcec5000640 1 -- 172.21.15.46:0/3836378298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcec00b3530 con 0x7fcec00ceaf0 2024-03-29T08:19:01.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.604+0000 7fceb6a00640 1 -- 172.21.15.46:0/3836378298 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcec00b3530 con 0x7fcec00ceaf0 2024-03-29T08:19:01.604 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:19:01.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.604+0000 7fcec5000640 1 -- 172.21.15.46:0/3836378298 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce9c066690 msgr2=0x7fce9c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.604+0000 7fcec5000640 1 --2- 172.21.15.46:0/3836378298 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce9c066690 0x7fce9c068b50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fceb00011f0 tx=0x7fceb0007d40 comp rx=0 tx=0).stop 2024-03-29T08:19:01.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.604+0000 7fcec5000640 1 -- 172.21.15.46:0/3836378298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec00ceaf0 msgr2=0x7fcec00b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.604+0000 7fcec5000640 1 --2- 172.21.15.46:0/3836378298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec00ceaf0 0x7fcec00b4ff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcea80af040 tx=0x7fcea8019240 comp rx=0 tx=0).stop 2024-03-29T08:19:01.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.604+0000 7fcec5000640 1 -- 172.21.15.46:0/3836378298 shutdown_connections 2024-03-29T08:19:01.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.604+0000 7fcec5000640 1 --2- 172.21.15.46:0/3836378298 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce9c066690 0x7fce9c068b50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.604+0000 7fcec5000640 1 --2- 172.21.15.46:0/3836378298 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcec015c9c0 0x7fcec0153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.604+0000 7fcec5000640 1 --2- 172.21.15.46:0/3836378298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcec00ceaf0 0x7fcec00b4ff0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.604+0000 7fcec5000640 1 --2- 172.21.15.46:0/3836378298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcec0098f80 0x7fcec00b5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.604+0000 7fcec5000640 1 -- 172.21.15.46:0/3836378298 >> 172.21.15.46:0/3836378298 conn(0x7fcec00b2220 msgr2=0x7fcec006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:01.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.604+0000 7fcec5000640 1 -- 172.21.15.46:0/3836378298 shutdown_connections 2024-03-29T08:19:01.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.604+0000 7fcec5000640 1 -- 172.21.15.46:0/3836378298 wait complete. 2024-03-29T08:19:01.625 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:19:01.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.768+0000 7f3703000640 1 Processor -- start 2024-03-29T08:19:01.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.768+0000 7f3703000640 1 -- start start 2024-03-29T08:19:01.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.768+0000 7f3703000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36fc15dbb0 0x7f36fc06ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.768+0000 7f3703000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36fc157490 con 0x7f36fc0b2440 2024-03-29T08:19:01.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.768+0000 7f3703000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36fc0b2c90 con 0x7f36fc0b3f20 2024-03-29T08:19:01.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.768+0000 7f3703000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc0b29b0 con 0x7f36fc15dbb0 2024-03-29T08:19:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.768+0000 7f3701000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f36fc0b2440 0x7f36fc0ac450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54778/0 (socket says 172.21.15.46:54778) 2024-03-29T08:19:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.768+0000 7f3701000640 1 -- 172.21.15.46:0/3955227184 learned_addr learned my addr 172.21.15.46:0/3955227184 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.768+0000 7f36fbe00640 1 --2- 172.21.15.46:0/3955227184 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36fc15dbb0 0x7f36fc06ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.768+0000 7f36faa00640 1 -- 172.21.15.46:0/3955227184 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2678167560 0 0) 0x7f36fc157490 con 0x7f36fc0b2440 2024-03-29T08:19:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.768+0000 7f36faa00640 1 -- 172.21.15.46:0/3955227184 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36e0003530 con 0x7f36fc0b2440 2024-03-29T08:19:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.768+0000 7f36faa00640 1 -- 172.21.15.46:0/3955227184 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406452105 0 0) 0x7f36fc0b2c90 con 0x7f36fc0b3f20 2024-03-29T08:19:01.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.768+0000 7f36faa00640 1 -- 172.21.15.46:0/3955227184 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36fc157490 con 0x7f36fc0b3f20 2024-03-29T08:19:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.768+0000 7f36fbe00640 1 -- 172.21.15.46:0/3955227184 >> v1:172.21.15.73:6789/0 conn(0x7f36fc0b3f20 legacy=0x7f36fc0b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.768+0000 7f36fbe00640 1 -- 172.21.15.46:0/3955227184 >> v1:172.21.15.46:6789/0 conn(0x7f36fc0b2440 legacy=0x7f36fc0ac450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.768+0000 7f36fbe00640 1 -- 172.21.15.46:0/3955227184 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc0b3440 con 0x7f36fc15dbb0 2024-03-29T08:19:01.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.772+0000 7f36fbe00640 1 --2- 172.21.15.46:0/3955227184 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36fc15dbb0 0x7f36fc06ada0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0097f0 tx=0x7f36ec01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a530fb71d0dce33 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:01.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.772+0000 7f36faa00640 1 -- 172.21.15.46:0/3955227184 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec0ad660 con 0x7f36fc15dbb0 2024-03-29T08:19:01.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.772+0000 7f36faa00640 1 -- 172.21.15.46:0/3955227184 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36ec0ad7c0 con 0x7f36fc15dbb0 2024-03-29T08:19:01.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.772+0000 7f36faa00640 1 -- 172.21.15.46:0/3955227184 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec0adb30 con 0x7f36fc15dbb0 2024-03-29T08:19:01.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.772+0000 7f3703000640 1 -- 172.21.15.46:0/3955227184 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36fc15dbb0 msgr2=0x7f36fc06ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.772+0000 7f3703000640 1 --2- 172.21.15.46:0/3955227184 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36fc15dbb0 0x7f36fc06ada0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0097f0 tx=0x7f36ec01c450 comp rx=0 tx=0).stop 2024-03-29T08:19:01.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.772+0000 7f3703000640 1 -- 172.21.15.46:0/3955227184 shutdown_connections 2024-03-29T08:19:01.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.772+0000 7f3703000640 1 --2- 172.21.15.46:0/3955227184 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36fc15dbb0 0x7f36fc06ada0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.772+0000 7f3703000640 1 -- 172.21.15.46:0/3955227184 >> 172.21.15.46:0/3955227184 conn(0x7f36fc070ad0 msgr2=0x7f36fc070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:01.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.772+0000 7f3703000640 1 -- 172.21.15.46:0/3955227184 shutdown_connections 2024-03-29T08:19:01.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.772+0000 7f3703000640 1 -- 172.21.15.46:0/3955227184 wait complete. 2024-03-29T08:19:01.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f3703000640 1 Processor -- start 2024-03-29T08:19:01.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f3703000640 1 -- start start 2024-03-29T08:19:01.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f3703000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36fc0b2440 0x7f36fc1534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f3703000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36fc0b3f20 0x7f36fc152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f3703000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36fc15dbb0 0x7f36fc14aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f3703000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc152640 con 0x7f36fc0b3f20 2024-03-29T08:19:01.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f3703000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc14e0d0 con 0x7f36fc15dbb0 2024-03-29T08:19:01.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f3703000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc14afb0 con 0x7f36fc0b2440 2024-03-29T08:19:01.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f36fb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36fc0b3f20 0x7f36fc152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f36fb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36fc0b3f20 0x7f36fc152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55332/0 (socket says 172.21.15.46:55332) 2024-03-29T08:19:01.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f36fb400640 1 -- 172.21.15.46:0/3551100658 learned_addr learned my addr 172.21.15.46:0/3551100658 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:01.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f36fbe00640 1 --2- 172.21.15.46:0/3551100658 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36fc0b2440 0x7f36fc1534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f3701000640 1 --2- 172.21.15.46:0/3551100658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36fc15dbb0 0x7f36fc14aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f36fb400640 1 -- 172.21.15.46:0/3551100658 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36fc0b2440 msgr2=0x7f36fc1534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f36fb400640 1 --2- 172.21.15.46:0/3551100658 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36fc0b2440 0x7f36fc1534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f36fb400640 1 -- 172.21.15.46:0/3551100658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36fc15dbb0 msgr2=0x7f36fc14aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f36fb400640 1 --2- 172.21.15.46:0/3551100658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36fc15dbb0 0x7f36fc14aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f36fb400640 1 -- 172.21.15.46:0/3551100658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ec009420 con 0x7f36fc0b3f20 2024-03-29T08:19:01.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f36fb400640 1 --2- 172.21.15.46:0/3551100658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36fc0b3f20 0x7f36fc152100 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f36e800c530 tx=0x7f36e800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:01.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f36f3e00640 1 -- 172.21.15.46:0/3551100658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e80052d0 con 0x7f36fc0b3f20 2024-03-29T08:19:01.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f36f3e00640 1 -- 172.21.15.46:0/3551100658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e8005450 con 0x7f36fc0b3f20 2024-03-29T08:19:01.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f36f3e00640 1 -- 172.21.15.46:0/3551100658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8012870 con 0x7f36fc0b3f20 2024-03-29T08:19:01.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f3703000640 1 -- 172.21.15.46:0/3551100658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36fc1567e0 con 0x7f36fc0b3f20 2024-03-29T08:19:01.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f3703000640 1 -- 172.21.15.46:0/3551100658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36fc156da0 con 0x7f36fc0b3f20 2024-03-29T08:19:01.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.776+0000 7f3703000640 1 -- 172.21.15.46:0/3551100658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36fc14e260 con 0x7f36fc0b3f20 2024-03-29T08:19:01.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.780+0000 7f36f3e00640 1 -- 172.21.15.46:0/3551100658 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f36e80129d0 con 0x7f36fc0b3f20 2024-03-29T08:19:01.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.780+0000 7f36f3e00640 1 --2- 172.21.15.46:0/3551100658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36dc0666c0 0x7f36dc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:01.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.780+0000 7f36f3e00640 1 -- 172.21.15.46:0/3551100658 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f36e8083c20 con 0x7f36fc0b3f20 2024-03-29T08:19:01.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.780+0000 7f36fbe00640 1 --2- 172.21.15.46:0/3551100658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36dc0666c0 0x7f36dc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:01.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.780+0000 7f36fbe00640 1 --2- 172.21.15.46:0/3551100658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36dc0666c0 0x7f36dc068b80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0af000 tx=0x7f36ec0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:01.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.784+0000 7f36f3e00640 1 -- 172.21.15.46:0/3551100658 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f36e8050f90 con 0x7f36fc0b3f20 2024-03-29T08:19:01.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.940+0000 7f3703000640 1 -- 172.21.15.46:0/3551100658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f36fc09d100 con 0x7f36fc0b3f20 2024-03-29T08:19:01.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.940+0000 7f36f3e00640 1 -- 172.21.15.46:0/3551100658 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f36e80545b0 con 0x7f36fc0b3f20 2024-03-29T08:19:01.943 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:19:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.944+0000 7f3703000640 1 -- 172.21.15.46:0/3551100658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36dc0666c0 msgr2=0x7f36dc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.944+0000 7f3703000640 1 --2- 172.21.15.46:0/3551100658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36dc0666c0 0x7f36dc068b80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f36ec0af000 tx=0x7f36ec0a7000 comp rx=0 tx=0).stop 2024-03-29T08:19:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.944+0000 7f3703000640 1 -- 172.21.15.46:0/3551100658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36fc0b3f20 msgr2=0x7f36fc152100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.944+0000 7f3703000640 1 --2- 172.21.15.46:0/3551100658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36fc0b3f20 0x7f36fc152100 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f36e800c530 tx=0x7f36e800c9f0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.944+0000 7f3703000640 1 -- 172.21.15.46:0/3551100658 shutdown_connections 2024-03-29T08:19:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.944+0000 7f3703000640 1 --2- 172.21.15.46:0/3551100658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36dc0666c0 0x7f36dc068b80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.944+0000 7f3703000640 1 --2- 172.21.15.46:0/3551100658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36fc15dbb0 0x7f36fc14aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.944+0000 7f3703000640 1 --2- 172.21.15.46:0/3551100658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36fc0b3f20 0x7f36fc152100 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.944+0000 7f3703000640 1 --2- 172.21.15.46:0/3551100658 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36fc0b2440 0x7f36fc1534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:01.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.944+0000 7f3703000640 1 -- 172.21.15.46:0/3551100658 >> 172.21.15.46:0/3551100658 conn(0x7f36fc070ad0 msgr2=0x7f36fc06ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:01.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.944+0000 7f3703000640 1 -- 172.21.15.46:0/3551100658 shutdown_connections 2024-03-29T08:19:01.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:01.944+0000 7f3703000640 1 -- 172.21.15.46:0/3551100658 wait complete. 2024-03-29T08:19:01.964 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:19:02.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84ba200640 1 Processor -- start 2024-03-29T08:19:02.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84ba200640 1 -- start start 2024-03-29T08:19:02.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84ba200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84b415ad20 0x7f84b40cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84ba200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f84b40b0540 0x7f84b40cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84ba200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b40b0ab0 con 0x7f84b415a940 2024-03-29T08:19:02.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84ba200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b40b0d90 con 0x7f84b40b0540 2024-03-29T08:19:02.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84ba200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b40b1070 con 0x7f84b415ad20 2024-03-29T08:19:02.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f84b415a940 0x7f84b40afb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54782/0 (socket says 172.21.15.46:54782) 2024-03-29T08:19:02.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b3400640 1 -- 172.21.15.46:0/835794884 learned_addr learned my addr 172.21.15.46:0/835794884 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:02.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b3e00640 1 --2- 172.21.15.46:0/835794884 >> v2:172.21.15.73:3300/0 conn(0x7f84b40b0540 0x7f84b40cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84abe00640 1 --2- 172.21.15.46:0/835794884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84b415ad20 0x7f84b40cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b2a00640 1 -- 172.21.15.46:0/835794884 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385765035 0 0) 0x7f84b40b0ab0 con 0x7f84b415a940 2024-03-29T08:19:02.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b2a00640 1 -- 172.21.15.46:0/835794884 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8498003530 con 0x7f84b415a940 2024-03-29T08:19:02.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b2a00640 1 -- 172.21.15.46:0/835794884 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3463859202 0 0) 0x7f8498003530 con 0x7f84b415a940 2024-03-29T08:19:02.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b2a00640 1 -- 172.21.15.46:0/835794884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84b415ad20 msgr2=0x7f84b40cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:02.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b2a00640 1 --2- 172.21.15.46:0/835794884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84b415ad20 0x7f84b40cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b2a00640 1 -- 172.21.15.46:0/835794884 >> v2:172.21.15.73:3300/0 conn(0x7f84b40b0540 msgr2=0x7f84b40cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:02.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b2a00640 1 --2- 172.21.15.46:0/835794884 >> v2:172.21.15.73:3300/0 conn(0x7f84b40b0540 0x7f84b40cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b2a00640 1 -- 172.21.15.46:0/835794884 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b40b1350 con 0x7f84b415a940 2024-03-29T08:19:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b2a00640 1 -- 172.21.15.46:0/835794884 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f84a4001680 con 0x7f84b415a940 2024-03-29T08:19:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b2a00640 1 -- 172.21.15.46:0/835794884 >> v1:172.21.15.46:6789/0 conn(0x7f84b415a940 legacy=0x7f84b40afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b2a00640 1 --2- 172.21.15.46:0/835794884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8498003980 0x7f8498003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b2a00640 1 --2- 172.21.15.46:0/835794884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84980047d0 0x7f8498004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b2a00640 1 --2- 172.21.15.46:0/835794884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84b415ad20 0x7f8498005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b2a00640 1 -- 172.21.15.46:0/835794884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b40b0d90 con 0x7f84980047d0 2024-03-29T08:19:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b2a00640 1 -- 172.21.15.46:0/835794884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b40b1070 con 0x7f8498003980 2024-03-29T08:19:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b2a00640 1 -- 172.21.15.46:0/835794884 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849800b990 con 0x7f84b415ad20 2024-03-29T08:19:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84abe00640 1 --2- 172.21.15.46:0/835794884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84980047d0 0x7f8498004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b3400640 1 --2- 172.21.15.46:0/835794884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8498003980 0x7f8498003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b3e00640 1 --2- 172.21.15.46:0/835794884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84b415ad20 0x7f8498005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84abe00640 1 -- 172.21.15.46:0/835794884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84b415ad20 msgr2=0x7f8498005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:02.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84abe00640 1 --2- 172.21.15.46:0/835794884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84b415ad20 0x7f8498005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84abe00640 1 -- 172.21.15.46:0/835794884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8498003980 msgr2=0x7f8498003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:02.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84abe00640 1 --2- 172.21.15.46:0/835794884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8498003980 0x7f8498003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84abe00640 1 -- 172.21.15.46:0/835794884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8498003530 con 0x7f84980047d0 2024-03-29T08:19:02.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.104+0000 7f84b3e00640 1 --2- 172.21.15.46:0/835794884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84b415ad20 0x7f8498005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.108+0000 7f84abe00640 1 --2- 172.21.15.46:0/835794884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84980047d0 0x7f8498004c50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f849c009ab0 tx=0x7f849c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef594e51e30e970 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.108+0000 7f84b2a00640 1 -- 172.21.15.46:0/835794884 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c0a5560 con 0x7f84980047d0 2024-03-29T08:19:02.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.108+0000 7f84b2a00640 1 -- 172.21.15.46:0/835794884 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f849c0a56c0 con 0x7f84980047d0 2024-03-29T08:19:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.108+0000 7f84b2a00640 1 -- 172.21.15.46:0/835794884 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f849c0a5a70 con 0x7f84980047d0 2024-03-29T08:19:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.108+0000 7f84ba200640 1 -- 172.21.15.46:0/835794884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84980047d0 msgr2=0x7f8498004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.108+0000 7f84ba200640 1 --2- 172.21.15.46:0/835794884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84980047d0 0x7f8498004c50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f849c009ab0 tx=0x7f849c01c9f0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.108+0000 7f84b3e00640 1 -- 172.21.15.46:0/835794884 reap_dead start 2024-03-29T08:19:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.108+0000 7f84ba200640 1 -- 172.21.15.46:0/835794884 shutdown_connections 2024-03-29T08:19:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.108+0000 7f84ba200640 1 -- 172.21.15.46:0/835794884 >> 172.21.15.46:0/835794884 conn(0x7f84b4089fd0 msgr2=0x7f84b408a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.108+0000 7f84ba200640 1 -- 172.21.15.46:0/835794884 shutdown_connections 2024-03-29T08:19:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.108+0000 7f84ba200640 1 -- 172.21.15.46:0/835794884 wait complete. 2024-03-29T08:19:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84ba200640 1 Processor -- start 2024-03-29T08:19:02.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84ba200640 1 -- start start 2024-03-29T08:19:02.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84ba200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84980047d0 0x7f84b414fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84ba200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84b40ad300 0x7f84b40ad780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84ba200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84b40b4340 0x7f84b40b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84ba200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b4150220 con 0x7f84980047d0 2024-03-29T08:19:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84ba200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b40adcc0 con 0x7f84b40b4340 2024-03-29T08:19:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84ba200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b40b4d00 con 0x7f84b40ad300 2024-03-29T08:19:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84980047d0 0x7f84b414fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84980047d0 0x7f84b414fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55340/0 (socket says 172.21.15.46:55340) 2024-03-29T08:19:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84b3400640 1 -- 172.21.15.46:0/2003952529 learned_addr learned my addr 172.21.15.46:0/2003952529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84abe00640 1 --2- 172.21.15.46:0/2003952529 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84b40ad300 0x7f84b40ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84b3e00640 1 --2- 172.21.15.46:0/2003952529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84b40b4340 0x7f84b40b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84b3400640 1 -- 172.21.15.46:0/2003952529 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84b40ad300 msgr2=0x7f84b40ad780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84b3400640 1 --2- 172.21.15.46:0/2003952529 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84b40ad300 0x7f84b40ad780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84b3400640 1 -- 172.21.15.46:0/2003952529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84b40b4340 msgr2=0x7f84b40b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84b3400640 1 --2- 172.21.15.46:0/2003952529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84b40b4340 0x7f84b40b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84b3400640 1 -- 172.21.15.46:0/2003952529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b40b4f90 con 0x7f84980047d0 2024-03-29T08:19:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84b3e00640 1 --2- 172.21.15.46:0/2003952529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84b40b4340 0x7f84b40b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84b3400640 1 --2- 172.21.15.46:0/2003952529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84980047d0 0x7f84b414fce0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f84a4004880 tx=0x7f84a4012980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84ab400640 1 -- 172.21.15.46:0/2003952529 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4014810 con 0x7f84980047d0 2024-03-29T08:19:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84ab400640 1 -- 172.21.15.46:0/2003952529 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a4014990 con 0x7f84980047d0 2024-03-29T08:19:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84ab400640 1 -- 172.21.15.46:0/2003952529 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4014d40 con 0x7f84980047d0 2024-03-29T08:19:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84ba200640 1 -- 172.21.15.46:0/2003952529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b40b5270 con 0x7f84980047d0 2024-03-29T08:19:02.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.112+0000 7f84ba200640 1 -- 172.21.15.46:0/2003952529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b406b220 con 0x7f84980047d0 2024-03-29T08:19:02.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.116+0000 7f84ba200640 1 -- 172.21.15.46:0/2003952529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b4066c90 con 0x7f84980047d0 2024-03-29T08:19:02.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.116+0000 7f84ab400640 1 -- 172.21.15.46:0/2003952529 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f84a40153e0 con 0x7f84980047d0 2024-03-29T08:19:02.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.116+0000 7f84ab400640 1 --2- 172.21.15.46:0/2003952529 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8490066690 0x7f8490068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.116+0000 7f84ab400640 1 -- 172.21.15.46:0/2003952529 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f84a4086ca0 con 0x7f84980047d0 2024-03-29T08:19:02.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.116+0000 7f84abe00640 1 --2- 172.21.15.46:0/2003952529 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8490066690 0x7f8490068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.116+0000 7f84abe00640 1 --2- 172.21.15.46:0/2003952529 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8490066690 0x7f8490068b50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f849c0af000 tx=0x7f849c0046d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:02.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.120+0000 7f84ab400640 1 -- 172.21.15.46:0/2003952529 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f84a4054000 con 0x7f84980047d0 2024-03-29T08:19:02.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.316+0000 7f84ba200640 1 -- 172.21.15.46:0/2003952529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84b40b6550 con 0x7f84980047d0 2024-03-29T08:19:02.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.320+0000 7f84ab400640 1 -- 172.21.15.46:0/2003952529 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4722 (secure 0 0 0) 0x7f84a401c070 con 0x7f84980047d0 2024-03-29T08:19:02.320 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:02.320 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/339038392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":339038392},{"type":"v1","addr":"172.21.15.46:6834","nonce":339038392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3573036715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3573036715},{"type":"v1","addr":"172.21.15.73:6835","nonce":3573036715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/746744819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":746744819},{"type":"v1","addr":"172.21.15.46:6835","nonce":746744819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:18:13.311457+0000","modified":"2024-03-29T08:18:16.321300+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2780269530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2780269530},{"type":"v1","addr":"172.21.15.73:6837","nonce":2780269530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-29T08:19:02.321 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-03-29T08:19:02.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.320+0000 7f84ba200640 1 -- 172.21.15.46:0/2003952529 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8490066690 msgr2=0x7f8490068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.320+0000 7f84ba200640 1 --2- 172.21.15.46:0/2003952529 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8490066690 0x7f8490068b50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f849c0af000 tx=0x7f849c0046d0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.320+0000 7f84ba200640 1 -- 172.21.15.46:0/2003952529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84980047d0 msgr2=0x7f84b414fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.320+0000 7f84ba200640 1 --2- 172.21.15.46:0/2003952529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84980047d0 0x7f84b414fce0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f84a4004880 tx=0x7f84a4012980 comp rx=0 tx=0).stop 2024-03-29T08:19:02.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.320+0000 7f84ba200640 1 -- 172.21.15.46:0/2003952529 shutdown_connections 2024-03-29T08:19:02.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.320+0000 7f84ba200640 1 --2- 172.21.15.46:0/2003952529 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8490066690 0x7f8490068b50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.320+0000 7f84ba200640 1 --2- 172.21.15.46:0/2003952529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84b40b4340 0x7f84b40b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.320+0000 7f84ba200640 1 --2- 172.21.15.46:0/2003952529 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84b40ad300 0x7f84b40ad780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.320+0000 7f84ba200640 1 --2- 172.21.15.46:0/2003952529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84980047d0 0x7f84b414fce0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.320+0000 7f84ba200640 1 -- 172.21.15.46:0/2003952529 >> 172.21.15.46:0/2003952529 conn(0x7f84b4089fd0 msgr2=0x7f84b40afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:02.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.320+0000 7f84ba200640 1 -- 172.21.15.46:0/2003952529 shutdown_connections 2024-03-29T08:19:02.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.320+0000 7f84ba200640 1 -- 172.21.15.46:0/2003952529 wait complete. 2024-03-29T08:19:02.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:19:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.480+0000 7f0ede000640 1 Processor -- start 2024-03-29T08:19:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.480+0000 7f0ede000640 1 -- start start 2024-03-29T08:19:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.480+0000 7f0ede000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0ed80b0910 0x7f0ed80cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.480+0000 7f0ede000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0ed80d0030 0x7f0ed80cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.480+0000 7f0ede000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed80b0ee0 con 0x7f0ed815a900 2024-03-29T08:19:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.480+0000 7f0ede000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed815ace0 con 0x7f0ed80d0030 2024-03-29T08:19:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.480+0000 7f0ede000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed80afcf0 con 0x7f0ed80b0910 2024-03-29T08:19:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.480+0000 7f0ed7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0ed80d0030 0x7f0ed80cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.480+0000 7f0ed7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0ed80d0030 0x7f0ed80cb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55342/0 (socket says 172.21.15.46:55342) 2024-03-29T08:19:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.480+0000 7f0ed7e00640 1 -- 172.21.15.46:0/941069254 learned_addr learned my addr 172.21.15.46:0/941069254 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.480+0000 7f0ecea00640 1 --2- 172.21.15.46:0/941069254 >> v2:172.21.15.73:3300/0 conn(0x7f0ed80b0910 0x7f0ed80cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.480+0000 7f0ed7e00640 1 -- 172.21.15.46:0/941069254 >> v2:172.21.15.73:3300/0 conn(0x7f0ed80b0910 msgr2=0x7f0ed80cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.480+0000 7f0ed7e00640 1 --2- 172.21.15.46:0/941069254 >> v2:172.21.15.73:3300/0 conn(0x7f0ed80b0910 0x7f0ed80cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.480+0000 7f0ed7e00640 1 -- 172.21.15.46:0/941069254 >> v1:172.21.15.73:6789/0 conn(0x7f0ed815a900 legacy=0x7f0ed80b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.480+0000 7f0ed7e00640 1 -- 172.21.15.46:0/941069254 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed80aee00 con 0x7f0ed80d0030 2024-03-29T08:19:02.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.484+0000 7f0ed7e00640 1 --2- 172.21.15.46:0/941069254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed80d0030 0x7f0ed80cb8c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8070c90 tx=0x7f0ed001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5071088298089655 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:02.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.484+0000 7f0ed6a00640 1 -- 172.21.15.46:0/941069254 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed00aa610 con 0x7f0ed80d0030 2024-03-29T08:19:02.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.484+0000 7f0ed6a00640 1 -- 172.21.15.46:0/941069254 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed00a6040 con 0x7f0ed80d0030 2024-03-29T08:19:02.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.484+0000 7f0ed6a00640 1 -- 172.21.15.46:0/941069254 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed00aa990 con 0x7f0ed80d0030 2024-03-29T08:19:02.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.484+0000 7f0ede000640 1 -- 172.21.15.46:0/941069254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed80d0030 msgr2=0x7f0ed80cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.484+0000 7f0ede000640 1 --2- 172.21.15.46:0/941069254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed80d0030 0x7f0ed80cb8c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0ed8070c90 tx=0x7f0ed001c4b0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.484+0000 7f0ede000640 1 -- 172.21.15.46:0/941069254 shutdown_connections 2024-03-29T08:19:02.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.484+0000 7f0ede000640 1 --2- 172.21.15.46:0/941069254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed80d0030 0x7f0ed80cb8c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.484+0000 7f0ede000640 1 --2- 172.21.15.46:0/941069254 >> v2:172.21.15.73:3300/0 conn(0x7f0ed80b0910 0x7f0ed80cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.484+0000 7f0ede000640 1 -- 172.21.15.46:0/941069254 >> 172.21.15.46:0/941069254 conn(0x7f0ed8098a40 msgr2=0x7f0ed80cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:02.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.484+0000 7f0ede000640 1 -- 172.21.15.46:0/941069254 shutdown_connections 2024-03-29T08:19:02.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.484+0000 7f0ede000640 1 -- 172.21.15.46:0/941069254 wait complete. 2024-03-29T08:19:02.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ede000640 1 Processor -- start 2024-03-29T08:19:02.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ede000640 1 -- start start 2024-03-29T08:19:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ede000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ed80b0910 0x7f0ed80ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ede000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed80d0030 0x7f0ed8151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ede000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ed815a900 0x7f0ed8154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ede000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed80adde0 con 0x7f0ed80d0030 2024-03-29T08:19:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ede000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed8152d70 con 0x7f0ed80b0910 2024-03-29T08:19:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ede000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed8153050 con 0x7f0ed815a900 2024-03-29T08:19:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ecea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed80d0030 0x7f0ed8151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ecea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed80d0030 0x7f0ed8151900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55354/0 (socket says 172.21.15.46:55354) 2024-03-29T08:19:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ecea00640 1 -- 172.21.15.46:0/2883807750 learned_addr learned my addr 172.21.15.46:0/2883807750 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ed7400640 1 --2- 172.21.15.46:0/2883807750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ed80b0910 0x7f0ed80ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ed7e00640 1 --2- 172.21.15.46:0/2883807750 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ed815a900 0x7f0ed8154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ecea00640 1 -- 172.21.15.46:0/2883807750 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ed815a900 msgr2=0x7f0ed8154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ecea00640 1 --2- 172.21.15.46:0/2883807750 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ed815a900 0x7f0ed8154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ecea00640 1 -- 172.21.15.46:0/2883807750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ed80b0910 msgr2=0x7f0ed80ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ecea00640 1 --2- 172.21.15.46:0/2883807750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ed80b0910 0x7f0ed80ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ecea00640 1 -- 172.21.15.46:0/2883807750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed00094d0 con 0x7f0ed80d0030 2024-03-29T08:19:02.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ecea00640 1 --2- 172.21.15.46:0/2883807750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed80d0030 0x7f0ed8151900 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000d5f0 tx=0x7f0ec000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:02.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ecfe00640 1 -- 172.21.15.46:0/2883807750 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec00125f0 con 0x7f0ed80d0030 2024-03-29T08:19:02.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ecfe00640 1 -- 172.21.15.46:0/2883807750 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec0012750 con 0x7f0ed80d0030 2024-03-29T08:19:02.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ecfe00640 1 -- 172.21.15.46:0/2883807750 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0012ae0 con 0x7f0ed80d0030 2024-03-29T08:19:02.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ede000640 1 -- 172.21.15.46:0/2883807750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed8153330 con 0x7f0ed80d0030 2024-03-29T08:19:02.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ede000640 1 -- 172.21.15.46:0/2883807750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed81538f0 con 0x7f0ed80d0030 2024-03-29T08:19:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.488+0000 7f0ede000640 1 -- 172.21.15.46:0/2883807750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed8066c90 con 0x7f0ed80d0030 2024-03-29T08:19:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.492+0000 7f0ecfe00640 1 -- 172.21.15.46:0/2883807750 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0ec0012c40 con 0x7f0ed80d0030 2024-03-29T08:19:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.492+0000 7f0ecfe00640 1 --2- 172.21.15.46:0/2883807750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0eb40666e0 0x7f0eb4068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.492+0000 7f0ecfe00640 1 -- 172.21.15.46:0/2883807750 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f0ec0084660 con 0x7f0ed80d0030 2024-03-29T08:19:02.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.492+0000 7f0ed7400640 1 --2- 172.21.15.46:0/2883807750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0eb40666e0 0x7f0eb4068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.492+0000 7f0ed7400640 1 --2- 172.21.15.46:0/2883807750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0eb40666e0 0x7f0eb4068ba0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8005530 tx=0x7f0ec8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:02.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.496+0000 7f0ecfe00640 1 -- 172.21.15.46:0/2883807750 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0ec0051a10 con 0x7f0ed80d0030 2024-03-29T08:19:02.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.656+0000 7f0ede000640 1 -- 172.21.15.46:0/2883807750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ed8073ef0 con 0x7f0ed80d0030 2024-03-29T08:19:02.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.656+0000 7f0ecfe00640 1 -- 172.21.15.46:0/2883807750 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16042 (secure 0 0 0) 0x7f0ec0055030 con 0x7f0ed80d0030 2024-03-29T08:19:02.659 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:02.659 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:56.021693+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:19:02.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.660+0000 7f0ede000640 1 -- 172.21.15.46:0/2883807750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0eb40666e0 msgr2=0x7f0eb4068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.660+0000 7f0ede000640 1 --2- 172.21.15.46:0/2883807750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0eb40666e0 0x7f0eb4068ba0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8005530 tx=0x7f0ec8009040 comp rx=0 tx=0).stop 2024-03-29T08:19:02.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.660+0000 7f0ede000640 1 -- 172.21.15.46:0/2883807750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed80d0030 msgr2=0x7f0ed8151900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.660+0000 7f0ede000640 1 --2- 172.21.15.46:0/2883807750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed80d0030 0x7f0ed8151900 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0ec000d5f0 tx=0x7f0ec000dab0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.660+0000 7f0ede000640 1 -- 172.21.15.46:0/2883807750 shutdown_connections 2024-03-29T08:19:02.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.660+0000 7f0ede000640 1 --2- 172.21.15.46:0/2883807750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0eb40666e0 0x7f0eb4068ba0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.660+0000 7f0ede000640 1 --2- 172.21.15.46:0/2883807750 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ed815a900 0x7f0ed8154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.660+0000 7f0ede000640 1 --2- 172.21.15.46:0/2883807750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed80d0030 0x7f0ed8151900 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.660+0000 7f0ede000640 1 --2- 172.21.15.46:0/2883807750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ed80b0910 0x7f0ed80ad8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.660+0000 7f0ede000640 1 -- 172.21.15.46:0/2883807750 >> 172.21.15.46:0/2883807750 conn(0x7f0ed8098a40 msgr2=0x7f0ed806eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:02.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.660+0000 7f0ede000640 1 -- 172.21.15.46:0/2883807750 shutdown_connections 2024-03-29T08:19:02.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.660+0000 7f0ede000640 1 -- 172.21.15.46:0/2883807750 wait complete. 2024-03-29T08:19:02.680 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T08:19:02.680 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:19:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.820+0000 7f385c400640 1 Processor -- start 2024-03-29T08:19:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.820+0000 7f385c400640 1 -- start start 2024-03-29T08:19:02.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.820+0000 7f385c400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38541593d0 0x7f385406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.820+0000 7f385c400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f385415cd90 0x7f38541597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.820+0000 7f385c400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f385406d130 con 0x7f385415d230 2024-03-29T08:19:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.820+0000 7f385c400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385415b8f0 con 0x7f385415cd90 2024-03-29T08:19:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.820+0000 7f385c400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385415ad50 con 0x7f38541593d0 2024-03-29T08:19:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.820+0000 7f385a400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f385415cd90 0x7f38541597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.820+0000 7f385a400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f385415cd90 0x7f38541597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55356/0 (socket says 172.21.15.46:55356) 2024-03-29T08:19:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.820+0000 7f385a400640 1 -- 172.21.15.46:0/3645623352 learned_addr learned my addr 172.21.15.46:0/3645623352 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:02.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.820+0000 7f3859a00640 1 --2- 172.21.15.46:0/3645623352 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38541593d0 0x7f385406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.820+0000 7f385a400640 1 -- 172.21.15.46:0/3645623352 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38541593d0 msgr2=0x7f385406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.820+0000 7f385a400640 1 --2- 172.21.15.46:0/3645623352 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38541593d0 0x7f385406cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.820+0000 7f385a400640 1 -- 172.21.15.46:0/3645623352 >> v1:172.21.15.73:6789/0 conn(0x7f385415d230 legacy=0x7f385415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.820+0000 7f385a400640 1 -- 172.21.15.46:0/3645623352 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385415afc0 con 0x7f385415cd90 2024-03-29T08:19:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.824+0000 7f385a400640 1 --2- 172.21.15.46:0/3645623352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f385415cd90 0x7f38541597b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f384c009e80 tx=0x7f384c01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70476147fb6bb23a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.824+0000 7f3853e00640 1 -- 172.21.15.46:0/3645623352 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c0a58e0 con 0x7f385415cd90 2024-03-29T08:19:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.824+0000 7f3853e00640 1 -- 172.21.15.46:0/3645623352 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384c0a5a40 con 0x7f385415cd90 2024-03-29T08:19:02.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.824+0000 7f3853e00640 1 -- 172.21.15.46:0/3645623352 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384c0a5dd0 con 0x7f385415cd90 2024-03-29T08:19:02.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.824+0000 7f385c400640 1 -- 172.21.15.46:0/3645623352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f385415cd90 msgr2=0x7f38541597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.824+0000 7f385c400640 1 --2- 172.21.15.46:0/3645623352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f385415cd90 0x7f38541597b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f384c009e80 tx=0x7f384c01ca10 comp rx=0 tx=0).stop 2024-03-29T08:19:02.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.824+0000 7f385c400640 1 -- 172.21.15.46:0/3645623352 shutdown_connections 2024-03-29T08:19:02.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.824+0000 7f385c400640 1 --2- 172.21.15.46:0/3645623352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f385415cd90 0x7f38541597b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.824+0000 7f385c400640 1 --2- 172.21.15.46:0/3645623352 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38541593d0 0x7f385406cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.824+0000 7f385c400640 1 -- 172.21.15.46:0/3645623352 >> 172.21.15.46:0/3645623352 conn(0x7f3854066110 msgr2=0x7f3854069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:02.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.824+0000 7f385c400640 1 -- 172.21.15.46:0/3645623352 shutdown_connections 2024-03-29T08:19:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.824+0000 7f385c400640 1 -- 172.21.15.46:0/3645623352 wait complete. 2024-03-29T08:19:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.824+0000 7f385c400640 1 Processor -- start 2024-03-29T08:19:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.824+0000 7f385c400640 1 -- start start 2024-03-29T08:19:02.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f385c400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38541593d0 0x7f38541f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f385c400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f385415cd90 0x7f385414dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f385c400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f385415d230 0x7f3854153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f385c400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385414e1d0 con 0x7f385415cd90 2024-03-29T08:19:02.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f385c400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3854154330 con 0x7f38541593d0 2024-03-29T08:19:02.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f385c400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385414e3c0 con 0x7f385415d230 2024-03-29T08:19:02.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f3859000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f385415cd90 0x7f385414dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f3859a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38541593d0 0x7f38541f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f385a400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f385415d230 0x7f3854153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f3859000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f385415cd90 0x7f385414dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55368/0 (socket says 172.21.15.46:55368) 2024-03-29T08:19:02.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f3859000640 1 -- 172.21.15.46:0/382334750 learned_addr learned my addr 172.21.15.46:0/382334750 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:02.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f3859000640 1 -- 172.21.15.46:0/382334750 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f385415d230 msgr2=0x7f3854153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f3859000640 1 --2- 172.21.15.46:0/382334750 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f385415d230 0x7f3854153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f3859000640 1 -- 172.21.15.46:0/382334750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38541593d0 msgr2=0x7f38541f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:02.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f3859000640 1 --2- 172.21.15.46:0/382334750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38541593d0 0x7f38541f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:02.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f3859000640 1 -- 172.21.15.46:0/382334750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c009b30 con 0x7f385415cd90 2024-03-29T08:19:02.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f3859000640 1 --2- 172.21.15.46:0/382334750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f385415cd90 0x7f385414dc90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f384000c450 tx=0x7f384000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:02.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f3851600640 1 -- 172.21.15.46:0/382334750 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38400052d0 con 0x7f385415cd90 2024-03-29T08:19:02.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f3851600640 1 -- 172.21.15.46:0/382334750 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3840005430 con 0x7f385415cd90 2024-03-29T08:19:02.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f3851600640 1 -- 172.21.15.46:0/382334750 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840007260 con 0x7f385415cd90 2024-03-29T08:19:02.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f385c400640 1 -- 172.21.15.46:0/382334750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385414e6a0 con 0x7f385415cd90 2024-03-29T08:19:02.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f385c400640 1 -- 172.21.15.46:0/382334750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38540cc5c0 con 0x7f385415cd90 2024-03-29T08:19:02.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f385c400640 1 -- 172.21.15.46:0/382334750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3854158c50 con 0x7f385415cd90 2024-03-29T08:19:02.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.828+0000 7f3851600640 1 -- 172.21.15.46:0/382334750 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f38400073c0 con 0x7f385415cd90 2024-03-29T08:19:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.832+0000 7f3851600640 1 --2- 172.21.15.46:0/382334750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3834066690 0x7f3834068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.832+0000 7f3851600640 1 -- 172.21.15.46:0/382334750 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f3840083960 con 0x7f385415cd90 2024-03-29T08:19:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.832+0000 7f3859a00640 1 --2- 172.21.15.46:0/382334750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3834066690 0x7f3834068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.832+0000 7f3859a00640 1 --2- 172.21.15.46:0/382334750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3834066690 0x7f3834068b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3848009ac0 tx=0x7f3848009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:02.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:02.836+0000 7f3851600640 1 -- 172.21.15.46:0/382334750 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3840050d50 con 0x7f385415cd90 2024-03-29T08:19:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.032+0000 7f385c400640 1 -- 172.21.15.46:0/382334750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3854066c90 con 0x7f385415cd90 2024-03-29T08:19:03.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.032+0000 7f3851600640 1 -- 172.21.15.46:0/382334750 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4722 (secure 0 0 0) 0x7f3840054370 con 0x7f385415cd90 2024-03-29T08:19:03.034 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:03.035 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/339038392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":339038392},{"type":"v1","addr":"172.21.15.46:6834","nonce":339038392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3573036715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3573036715},{"type":"v1","addr":"172.21.15.73:6835","nonce":3573036715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/746744819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":746744819},{"type":"v1","addr":"172.21.15.46:6835","nonce":746744819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:18:13.311457+0000","modified":"2024-03-29T08:18:16.321300+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2780269530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2780269530},{"type":"v1","addr":"172.21.15.73:6837","nonce":2780269530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-29T08:19:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-03-29T08:19:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.036+0000 7f385c400640 1 -- 172.21.15.46:0/382334750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3834066690 msgr2=0x7f3834068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.036+0000 7f385c400640 1 --2- 172.21.15.46:0/382334750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3834066690 0x7f3834068b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3848009ac0 tx=0x7f3848009190 comp rx=0 tx=0).stop 2024-03-29T08:19:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.036+0000 7f385c400640 1 -- 172.21.15.46:0/382334750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f385415cd90 msgr2=0x7f385414dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.036+0000 7f385c400640 1 --2- 172.21.15.46:0/382334750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f385415cd90 0x7f385414dc90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f384000c450 tx=0x7f384000c910 comp rx=0 tx=0).stop 2024-03-29T08:19:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.036+0000 7f385c400640 1 -- 172.21.15.46:0/382334750 shutdown_connections 2024-03-29T08:19:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.036+0000 7f385c400640 1 --2- 172.21.15.46:0/382334750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3834066690 0x7f3834068b50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.036+0000 7f385c400640 1 --2- 172.21.15.46:0/382334750 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f385415d230 0x7f3854153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.036+0000 7f385c400640 1 --2- 172.21.15.46:0/382334750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f385415cd90 0x7f385414dc90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.036+0000 7f385c400640 1 --2- 172.21.15.46:0/382334750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38541593d0 0x7f38541f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.036+0000 7f385c400640 1 -- 172.21.15.46:0/382334750 >> 172.21.15.46:0/382334750 conn(0x7f3854066110 msgr2=0x7f38540b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:03.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.036+0000 7f385c400640 1 -- 172.21.15.46:0/382334750 shutdown_connections 2024-03-29T08:19:03.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.036+0000 7f385c400640 1 -- 172.21.15.46:0/382334750 wait complete. 2024-03-29T08:19:03.056 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:19:03.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.196+0000 7f3946400640 1 Processor -- start 2024-03-29T08:19:03.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.196+0000 7f3946400640 1 -- start start 2024-03-29T08:19:03.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.196+0000 7f3946400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f39401593d0 0x7f394006cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.196+0000 7f3946400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f394015d230 0x7f394015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.196+0000 7f3946400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394006d130 con 0x7f394015dbd0 2024-03-29T08:19:03.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.196+0000 7f3946400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394015b8f0 con 0x7f394015d230 2024-03-29T08:19:03.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.196+0000 7f3946400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394015ad50 con 0x7f39401593d0 2024-03-29T08:19:03.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.196+0000 7f3936a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f394015d230 0x7f394015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.196+0000 7f3936a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f394015d230 0x7f394015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55384/0 (socket says 172.21.15.46:55384) 2024-03-29T08:19:03.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.196+0000 7f3936a00640 1 -- 172.21.15.46:0/3069161403 learned_addr learned my addr 172.21.15.46:0/3069161403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:03.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.196+0000 7f393fe00640 1 --1- 172.21.15.46:0/3069161403 >> v1:172.21.15.46:6789/0 conn(0x7f394015dbd0 0x7f394015bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54794/0 (socket says 172.21.15.46:54794) 2024-03-29T08:19:03.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.196+0000 7f393f400640 1 --2- 172.21.15.46:0/3069161403 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f39401593d0 0x7f394006cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.196+0000 7f3936a00640 1 -- 172.21.15.46:0/3069161403 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f39401593d0 msgr2=0x7f394006cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:03.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.196+0000 7f3936a00640 1 --2- 172.21.15.46:0/3069161403 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f39401593d0 0x7f394006cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.196+0000 7f3936a00640 1 -- 172.21.15.46:0/3069161403 >> v1:172.21.15.46:6789/0 conn(0x7f394015dbd0 legacy=0x7f394015bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.196+0000 7f3936a00640 1 -- 172.21.15.46:0/3069161403 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394015e060 con 0x7f394015d230 2024-03-29T08:19:03.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.200+0000 7f3936a00640 1 --2- 172.21.15.46:0/3069161403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394015d230 0x7f394015d690 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3928009870 tx=0x7f392801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc7dc2fed72e615 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:03.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.200+0000 7f393ea00640 1 -- 172.21.15.46:0/3069161403 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39280a9620 con 0x7f394015d230 2024-03-29T08:19:03.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.200+0000 7f393ea00640 1 -- 172.21.15.46:0/3069161403 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39280a9780 con 0x7f394015d230 2024-03-29T08:19:03.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.200+0000 7f393ea00640 1 -- 172.21.15.46:0/3069161403 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39280a9b10 con 0x7f394015d230 2024-03-29T08:19:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.200+0000 7f3946400640 1 -- 172.21.15.46:0/3069161403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394015d230 msgr2=0x7f394015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.200+0000 7f3946400640 1 --2- 172.21.15.46:0/3069161403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394015d230 0x7f394015d690 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3928009870 tx=0x7f392801c560 comp rx=0 tx=0).stop 2024-03-29T08:19:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.200+0000 7f3946400640 1 -- 172.21.15.46:0/3069161403 shutdown_connections 2024-03-29T08:19:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.200+0000 7f3946400640 1 --2- 172.21.15.46:0/3069161403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394015d230 0x7f394015d690 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.200+0000 7f3946400640 1 --2- 172.21.15.46:0/3069161403 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f39401593d0 0x7f394006cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.200+0000 7f3946400640 1 -- 172.21.15.46:0/3069161403 >> 172.21.15.46:0/3069161403 conn(0x7f3940066110 msgr2=0x7f3940069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.200+0000 7f3946400640 1 -- 172.21.15.46:0/3069161403 shutdown_connections 2024-03-29T08:19:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.200+0000 7f3946400640 1 -- 172.21.15.46:0/3069161403 wait complete. 2024-03-29T08:19:03.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f3946400640 1 Processor -- start 2024-03-29T08:19:03.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f3946400640 1 -- start start 2024-03-29T08:19:03.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f3946400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f39401593d0 0x7f394014c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f3946400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f394015d230 0x7f3940152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f3946400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394015dbd0 0x7f3940152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f3946400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394014c840 con 0x7f394015dbd0 2024-03-29T08:19:03.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f3946400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3940150fa0 con 0x7f394015d230 2024-03-29T08:19:03.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f3946400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3940156bd0 con 0x7f39401593d0 2024-03-29T08:19:03.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f393fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394015dbd0 0x7f3940152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f393fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394015dbd0 0x7f3940152ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55400/0 (socket says 172.21.15.46:55400) 2024-03-29T08:19:03.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f393fe00640 1 -- 172.21.15.46:0/3206489363 learned_addr learned my addr 172.21.15.46:0/3206489363 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:03.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f393f400640 1 --2- 172.21.15.46:0/3206489363 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f39401593d0 0x7f394014c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f3936a00640 1 --2- 172.21.15.46:0/3206489363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f394015d230 0x7f3940152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f393fe00640 1 -- 172.21.15.46:0/3206489363 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f39401593d0 msgr2=0x7f394014c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f393fe00640 1 --2- 172.21.15.46:0/3206489363 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f39401593d0 0x7f394014c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f393fe00640 1 -- 172.21.15.46:0/3206489363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f394015d230 msgr2=0x7f3940152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f393fe00640 1 --2- 172.21.15.46:0/3206489363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f394015d230 0x7f3940152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f393fe00640 1 -- 172.21.15.46:0/3206489363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39280094d0 con 0x7f394015dbd0 2024-03-29T08:19:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f393fe00640 1 --2- 172.21.15.46:0/3206489363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394015dbd0 0x7f3940152ad0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f393800c7a0 tx=0x7f393800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f3937e00640 1 -- 172.21.15.46:0/3206489363 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393800a640 con 0x7f394015dbd0 2024-03-29T08:19:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f3937e00640 1 -- 172.21.15.46:0/3206489363 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393800a7a0 con 0x7f394015dbd0 2024-03-29T08:19:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f3937e00640 1 -- 172.21.15.46:0/3206489363 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393800ab10 con 0x7f394015dbd0 2024-03-29T08:19:03.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f3946400640 1 -- 172.21.15.46:0/3206489363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39401511f0 con 0x7f394015dbd0 2024-03-29T08:19:03.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.204+0000 7f3946400640 1 -- 172.21.15.46:0/3206489363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39401517b0 con 0x7f394015dbd0 2024-03-29T08:19:03.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.208+0000 7f3946400640 1 -- 172.21.15.46:0/3206489363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39400b10b0 con 0x7f394015dbd0 2024-03-29T08:19:03.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.208+0000 7f3937e00640 1 -- 172.21.15.46:0/3206489363 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f393800ac70 con 0x7f394015dbd0 2024-03-29T08:19:03.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.208+0000 7f3937e00640 1 --2- 172.21.15.46:0/3206489363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f391c0666c0 0x7f391c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.208+0000 7f3937e00640 1 -- 172.21.15.46:0/3206489363 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f3938083a30 con 0x7f394015dbd0 2024-03-29T08:19:03.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.208+0000 7f393f400640 1 --2- 172.21.15.46:0/3206489363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f391c0666c0 0x7f391c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.208+0000 7f393f400640 1 --2- 172.21.15.46:0/3206489363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f391c0666c0 0x7f391c068b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f39300079c0 tx=0x7f3930008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:03.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.212+0000 7f3937e00640 1 -- 172.21.15.46:0/3206489363 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3938050d00 con 0x7f394015dbd0 2024-03-29T08:19:03.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.412+0000 7f3946400640 1 -- 172.21.15.46:0/3206489363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3940156d40 con 0x7f394015dbd0 2024-03-29T08:19:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.412+0000 7f3937e00640 1 -- 172.21.15.46:0/3206489363 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4722 (secure 0 0 0) 0x7f3938017070 con 0x7f394015dbd0 2024-03-29T08:19:03.413 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:03.413 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/339038392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":339038392},{"type":"v1","addr":"172.21.15.46:6834","nonce":339038392}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/3573036715","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3573036715},{"type":"v1","addr":"172.21.15.73:6835","nonce":3573036715}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/746744819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":746744819},{"type":"v1","addr":"172.21.15.46:6835","nonce":746744819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:18:13.311457+0000","modified":"2024-03-29T08:18:16.321300+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4291},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4291":{"gid":4291,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2780269530","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2780269530},{"type":"v1","addr":"172.21.15.73:6837","nonce":2780269530}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-03-29T08:19:03.413 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-03-29T08:19:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.412+0000 7f3946400640 1 -- 172.21.15.46:0/3206489363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f391c0666c0 msgr2=0x7f391c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.412+0000 7f3946400640 1 --2- 172.21.15.46:0/3206489363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f391c0666c0 0x7f391c068b80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f39300079c0 tx=0x7f3930008040 comp rx=0 tx=0).stop 2024-03-29T08:19:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.412+0000 7f3946400640 1 -- 172.21.15.46:0/3206489363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394015dbd0 msgr2=0x7f3940152ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.412+0000 7f3946400640 1 --2- 172.21.15.46:0/3206489363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394015dbd0 0x7f3940152ad0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f393800c7a0 tx=0x7f393800cc60 comp rx=0 tx=0).stop 2024-03-29T08:19:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.412+0000 7f3946400640 1 -- 172.21.15.46:0/3206489363 shutdown_connections 2024-03-29T08:19:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.412+0000 7f3946400640 1 --2- 172.21.15.46:0/3206489363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f391c0666c0 0x7f391c068b80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.412+0000 7f3946400640 1 --2- 172.21.15.46:0/3206489363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f394015d230 0x7f3940152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.412+0000 7f3946400640 1 --2- 172.21.15.46:0/3206489363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f394015dbd0 0x7f3940152ad0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.412+0000 7f3946400640 1 --2- 172.21.15.46:0/3206489363 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f39401593d0 0x7f394014c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.412+0000 7f3946400640 1 -- 172.21.15.46:0/3206489363 >> 172.21.15.46:0/3206489363 conn(0x7f3940066110 msgr2=0x7f394015bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.412+0000 7f3946400640 1 -- 172.21.15.46:0/3206489363 shutdown_connections 2024-03-29T08:19:03.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.412+0000 7f3946400640 1 -- 172.21.15.46:0/3206489363 wait complete. 2024-03-29T08:19:03.435 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:19:03.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.572+0000 7f6213400640 1 Processor -- start 2024-03-29T08:19:03.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.572+0000 7f6213400640 1 -- start start 2024-03-29T08:19:03.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.576+0000 7f6213400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f620c144210 0x7f620c0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.576+0000 7f6213400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f620c15ada0 0x7f620c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.576+0000 7f6213400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f620c0cb510 con 0x7f620c0ce950 2024-03-29T08:19:03.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.576+0000 7f6213400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f620c0cede0 con 0x7f620c144210 2024-03-29T08:19:03.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.576+0000 7f6213400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620c06d960 con 0x7f620c15ada0 2024-03-29T08:19:03.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.576+0000 7f6210a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f620c144210 0x7f620c0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.576+0000 7f6210a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f620c144210 0x7f620c0cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55404/0 (socket says 172.21.15.46:55404) 2024-03-29T08:19:03.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.576+0000 7f6210a00640 1 -- 172.21.15.46:0/1650074946 learned_addr learned my addr 172.21.15.46:0/1650074946 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:03.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.576+0000 7f6203e00640 1 --2- 172.21.15.46:0/1650074946 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f620c15ada0 0x7f620c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.576+0000 7f6210a00640 1 -- 172.21.15.46:0/1650074946 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f620c15ada0 msgr2=0x7f620c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:03.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.576+0000 7f6210a00640 1 --2- 172.21.15.46:0/1650074946 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f620c15ada0 0x7f620c0ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.576+0000 7f6210a00640 1 -- 172.21.15.46:0/1650074946 >> v1:172.21.15.73:6789/0 conn(0x7f620c0ce950 legacy=0x7f620c15e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:03.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.576+0000 7f6210a00640 1 -- 172.21.15.46:0/1650074946 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c070580 con 0x7f620c144210 2024-03-29T08:19:03.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.580+0000 7f6210a00640 1 --2- 172.21.15.46:0/1650074946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c144210 0x7f620c0cb8b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009870 tx=0x7f61fc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8626715c73216f63 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:03.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.580+0000 7f620be00640 1 -- 172.21.15.46:0/1650074946 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc0ab600 con 0x7f620c144210 2024-03-29T08:19:03.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.580+0000 7f620be00640 1 -- 172.21.15.46:0/1650074946 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61fc0a7050 con 0x7f620c144210 2024-03-29T08:19:03.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.580+0000 7f620be00640 1 -- 172.21.15.46:0/1650074946 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc0ab980 con 0x7f620c144210 2024-03-29T08:19:03.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.580+0000 7f6213400640 1 -- 172.21.15.46:0/1650074946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c144210 msgr2=0x7f620c0cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.580+0000 7f6213400640 1 --2- 172.21.15.46:0/1650074946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c144210 0x7f620c0cb8b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009870 tx=0x7f61fc01c520 comp rx=0 tx=0).stop 2024-03-29T08:19:03.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.580+0000 7f6213400640 1 -- 172.21.15.46:0/1650074946 shutdown_connections 2024-03-29T08:19:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.580+0000 7f6213400640 1 --2- 172.21.15.46:0/1650074946 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f620c15ada0 0x7f620c0ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.580+0000 7f6213400640 1 --2- 172.21.15.46:0/1650074946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c144210 0x7f620c0cb8b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.580+0000 7f6213400640 1 -- 172.21.15.46:0/1650074946 >> 172.21.15.46:0/1650074946 conn(0x7f620c066110 msgr2=0x7f620c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.580+0000 7f6213400640 1 -- 172.21.15.46:0/1650074946 shutdown_connections 2024-03-29T08:19:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.580+0000 7f6213400640 1 -- 172.21.15.46:0/1650074946 wait complete. 2024-03-29T08:19:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.580+0000 7f6213400640 1 Processor -- start 2024-03-29T08:19:03.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.580+0000 7f6213400640 1 -- start start 2024-03-29T08:19:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6213400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c0ce950 0x7f620c140350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6213400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f620c144210 0x7f620c138f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6213400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f620c15ada0 0x7f620c13d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6213400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c13fec0 con 0x7f620c0ce950 2024-03-29T08:19:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6213400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620c13f2b0 con 0x7f620c144210 2024-03-29T08:19:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6213400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620c13f590 con 0x7f620c15ada0 2024-03-29T08:19:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6210a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c0ce950 0x7f620c140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6210a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c0ce950 0x7f620c140350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55406/0 (socket says 172.21.15.46:55406) 2024-03-29T08:19:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6210a00640 1 -- 172.21.15.46:0/3569609240 learned_addr learned my addr 172.21.15.46:0/3569609240 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6203e00640 1 --2- 172.21.15.46:0/3569609240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f620c144210 0x7f620c138f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6211400640 1 --2- 172.21.15.46:0/3569609240 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f620c15ada0 0x7f620c13d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6210a00640 1 -- 172.21.15.46:0/3569609240 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f620c15ada0 msgr2=0x7f620c13d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6210a00640 1 --2- 172.21.15.46:0/3569609240 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f620c15ada0 0x7f620c13d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6210a00640 1 -- 172.21.15.46:0/3569609240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f620c144210 msgr2=0x7f620c138f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6210a00640 1 --2- 172.21.15.46:0/3569609240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f620c144210 0x7f620c138f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6210a00640 1 -- 172.21.15.46:0/3569609240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc0094d0 con 0x7f620c0ce950 2024-03-29T08:19:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6203e00640 1 --2- 172.21.15.46:0/3569609240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f620c144210 0x7f620c138f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6210a00640 1 --2- 172.21.15.46:0/3569609240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c0ce950 0x7f620c140350 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0ae040 tx=0x7f61fc01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6209600640 1 -- 172.21.15.46:0/3569609240 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc0b6620 con 0x7f620c0ce950 2024-03-29T08:19:03.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6209600640 1 -- 172.21.15.46:0/3569609240 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61fc0abe90 con 0x7f620c0ce950 2024-03-29T08:19:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6209600640 1 -- 172.21.15.46:0/3569609240 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61fc0b69c0 con 0x7f620c0ce950 2024-03-29T08:19:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6213400640 1 -- 172.21.15.46:0/3569609240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f620c13f870 con 0x7f620c0ce950 2024-03-29T08:19:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6213400640 1 -- 172.21.15.46:0/3569609240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f620c139480 con 0x7f620c0ce950 2024-03-29T08:19:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6213400640 1 -- 172.21.15.46:0/3569609240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620c0ce280 con 0x7f620c0ce950 2024-03-29T08:19:03.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.584+0000 7f6209600640 1 -- 172.21.15.46:0/3569609240 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f61fc0b2040 con 0x7f620c0ce950 2024-03-29T08:19:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.588+0000 7f6209600640 1 --2- 172.21.15.46:0/3569609240 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f61e8066690 0x7f61e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.588+0000 7f6209600640 1 -- 172.21.15.46:0/3569609240 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f61fc11e570 con 0x7f620c0ce950 2024-03-29T08:19:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.588+0000 7f6203e00640 1 --2- 172.21.15.46:0/3569609240 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f61e8066690 0x7f61e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.588+0000 7f6203e00640 1 --2- 172.21.15.46:0/3569609240 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f61e8066690 0x7f61e8068b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f620c13e350 tx=0x7f61f4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:03.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.592+0000 7f6209600640 1 -- 172.21.15.46:0/3569609240 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f61fc0eb960 con 0x7f620c0ce950 2024-03-29T08:19:03.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.752+0000 7f6213400640 1 -- 172.21.15.46:0/3569609240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f620c066c90 con 0x7f620c0ce950 2024-03-29T08:19:03.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.752+0000 7f6209600640 1 -- 172.21.15.46:0/3569609240 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16042 (secure 0 0 0) 0x7f61fc0eef80 con 0x7f620c0ce950 2024-03-29T08:19:03.754 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:03.754 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:18:56.021693+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:18:11.177842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-29T08:18:12.308252+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.35","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:19:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.756+0000 7f6213400640 1 -- 172.21.15.46:0/3569609240 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f61e8066690 msgr2=0x7f61e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.756+0000 7f6213400640 1 --2- 172.21.15.46:0/3569609240 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f61e8066690 0x7f61e8068b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f620c13e350 tx=0x7f61f4008040 comp rx=0 tx=0).stop 2024-03-29T08:19:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.756+0000 7f6213400640 1 -- 172.21.15.46:0/3569609240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c0ce950 msgr2=0x7f620c140350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.756+0000 7f6213400640 1 --2- 172.21.15.46:0/3569609240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c0ce950 0x7f620c140350 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f61fc0ae040 tx=0x7f61fc01ad60 comp rx=0 tx=0).stop 2024-03-29T08:19:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.756+0000 7f6213400640 1 -- 172.21.15.46:0/3569609240 shutdown_connections 2024-03-29T08:19:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.756+0000 7f6213400640 1 --2- 172.21.15.46:0/3569609240 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f61e8066690 0x7f61e8068b50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.756+0000 7f6213400640 1 --2- 172.21.15.46:0/3569609240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f620c144210 0x7f620c138f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.756+0000 7f6213400640 1 --2- 172.21.15.46:0/3569609240 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f620c15ada0 0x7f620c13d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.756+0000 7f6213400640 1 --2- 172.21.15.46:0/3569609240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f620c0ce950 0x7f620c140350 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.756+0000 7f6213400640 1 -- 172.21.15.46:0/3569609240 >> 172.21.15.46:0/3569609240 conn(0x7f620c066110 msgr2=0x7f620c072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.756+0000 7f6213400640 1 -- 172.21.15.46:0/3569609240 shutdown_connections 2024-03-29T08:19:03.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.756+0000 7f6213400640 1 -- 172.21.15.46:0/3569609240 wait complete. 2024-03-29T08:19:03.775 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T08:19:03.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.916+0000 7f1df7200640 1 Processor -- start 2024-03-29T08:19:03.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.916+0000 7f1df7200640 1 -- start start 2024-03-29T08:19:03.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.916+0000 7f1df7200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1df815a620 0x7f1df815aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.916+0000 7f1df7200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1df815f170 0x7f1df80ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.916+0000 7f1df7200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df80cf710 con 0x7f1df80d04c0 2024-03-29T08:19:03.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.916+0000 7f1df7200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df80b16f0 con 0x7f1df815f170 2024-03-29T08:19:03.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.916+0000 7f1df7200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80ceb90 con 0x7f1df815a620 2024-03-29T08:19:03.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.916+0000 7f1defe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1df815f170 0x7f1df80ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.916+0000 7f1defe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1df815f170 0x7f1df80ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55414/0 (socket says 172.21.15.46:55414) 2024-03-29T08:19:03.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.916+0000 7f1defe00640 1 -- 172.21.15.46:0/439288512 learned_addr learned my addr 172.21.15.46:0/439288512 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:03.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.916+0000 7f1df5e00640 1 --2- 172.21.15.46:0/439288512 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1df815a620 0x7f1df815aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.916+0000 7f1defe00640 1 -- 172.21.15.46:0/439288512 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1df815a620 msgr2=0x7f1df815aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:03.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.916+0000 7f1defe00640 1 --2- 172.21.15.46:0/439288512 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1df815a620 0x7f1df815aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.916+0000 7f1defe00640 1 -- 172.21.15.46:0/439288512 >> v1:172.21.15.73:6789/0 conn(0x7f1df80d04c0 legacy=0x7f1df80ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.916+0000 7f1defe00640 1 -- 172.21.15.46:0/439288512 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df80cee70 con 0x7f1df815f170 2024-03-29T08:19:03.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.920+0000 7f1defe00640 1 --2- 172.21.15.46:0/439288512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df815f170 0x7f1df80ce340 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009870 tx=0x7f1de001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5199ea695973194c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:03.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.920+0000 7f1df5400640 1 -- 172.21.15.46:0/439288512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de00a95f0 con 0x7f1df815f170 2024-03-29T08:19:03.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.920+0000 7f1df5400640 1 -- 172.21.15.46:0/439288512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de00a9750 con 0x7f1df815f170 2024-03-29T08:19:03.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.920+0000 7f1df5400640 1 -- 172.21.15.46:0/439288512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de00a9ae0 con 0x7f1df815f170 2024-03-29T08:19:03.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.920+0000 7f1df7200640 1 -- 172.21.15.46:0/439288512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df815f170 msgr2=0x7f1df80ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.920+0000 7f1df7200640 1 --2- 172.21.15.46:0/439288512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df815f170 0x7f1df80ce340 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009870 tx=0x7f1de001c530 comp rx=0 tx=0).stop 2024-03-29T08:19:03.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.920+0000 7f1df7200640 1 -- 172.21.15.46:0/439288512 shutdown_connections 2024-03-29T08:19:03.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.920+0000 7f1df7200640 1 --2- 172.21.15.46:0/439288512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df815f170 0x7f1df80ce340 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.920+0000 7f1df7200640 1 --2- 172.21.15.46:0/439288512 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1df815a620 0x7f1df815aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.920+0000 7f1df7200640 1 -- 172.21.15.46:0/439288512 >> 172.21.15.46:0/439288512 conn(0x7f1df8089fd0 msgr2=0x7f1df808a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:03.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.920+0000 7f1df7200640 1 -- 172.21.15.46:0/439288512 shutdown_connections 2024-03-29T08:19:03.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.920+0000 7f1df7200640 1 -- 172.21.15.46:0/439288512 wait complete. 2024-03-29T08:19:03.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1df7200640 1 Processor -- start 2024-03-29T08:19:03.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1df7200640 1 -- start start 2024-03-29T08:19:03.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1df7200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1df80d04c0 0x7f1df80c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1df7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df815a620 0x7f1df80c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1df7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df815f170 0x7f1df80c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1df7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80c4e80 con 0x7f1df815a620 2024-03-29T08:19:03.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1df7200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80c5160 con 0x7f1df815f170 2024-03-29T08:19:03.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1df7200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80c5440 con 0x7f1df80d04c0 2024-03-29T08:19:03.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1defe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df815a620 0x7f1df80c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1defe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df815a620 0x7f1df80c42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55416/0 (socket says 172.21.15.46:55416) 2024-03-29T08:19:03.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1defe00640 1 -- 172.21.15.46:0/2293216638 learned_addr learned my addr 172.21.15.46:0/2293216638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:03.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1df5e00640 1 --2- 172.21.15.46:0/2293216638 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1df80d04c0 0x7f1df80c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1df6800640 1 --2- 172.21.15.46:0/2293216638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df815f170 0x7f1df80c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1defe00640 1 -- 172.21.15.46:0/2293216638 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1df80d04c0 msgr2=0x7f1df80c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1defe00640 1 --2- 172.21.15.46:0/2293216638 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1df80d04c0 0x7f1df80c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1defe00640 1 -- 172.21.15.46:0/2293216638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df815f170 msgr2=0x7f1df80c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1defe00640 1 --2- 172.21.15.46:0/2293216638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df815f170 0x7f1df80c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1defe00640 1 -- 172.21.15.46:0/2293216638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de00094d0 con 0x7f1df815a620 2024-03-29T08:19:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1df6800640 1 --2- 172.21.15.46:0/2293216638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df815f170 0x7f1df80c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:03.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1defe00640 1 --2- 172.21.15.46:0/2293216638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df815a620 0x7f1df80c42f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1de001abe0 tx=0x7f1de00192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1dee000640 1 -- 172.21.15.46:0/2293216638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de00bb640 con 0x7f1df815a620 2024-03-29T08:19:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1dee000640 1 -- 172.21.15.46:0/2293216638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de00bb7a0 con 0x7f1df815a620 2024-03-29T08:19:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1dee000640 1 -- 172.21.15.46:0/2293216638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de00bbb30 con 0x7f1df815a620 2024-03-29T08:19:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1df7200640 1 -- 172.21.15.46:0/2293216638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df806b030 con 0x7f1df815a620 2024-03-29T08:19:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1df7200640 1 -- 172.21.15.46:0/2293216638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df8151440 con 0x7f1df815a620 2024-03-29T08:19:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.924+0000 7f1df7200640 1 -- 172.21.15.46:0/2293216638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df804e040 con 0x7f1df815a620 2024-03-29T08:19:03.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.928+0000 7f1dee000640 1 -- 172.21.15.46:0/2293216638 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1de00b13f0 con 0x7f1df815a620 2024-03-29T08:19:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.928+0000 7f1dee000640 1 --2- 172.21.15.46:0/2293216638 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1dd4066690 0x7f1dd4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:03.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.928+0000 7f1dee000640 1 -- 172.21.15.46:0/2293216638 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1de011e700 con 0x7f1df815a620 2024-03-29T08:19:03.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.928+0000 7f1df5e00640 1 --2- 172.21.15.46:0/2293216638 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1dd4066690 0x7f1dd4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:03.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.928+0000 7f1df5e00640 1 --2- 172.21.15.46:0/2293216638 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1dd4066690 0x7f1dd4068b50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1de8007a60 tx=0x7f1de8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:03.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:03.932+0000 7f1dee000640 1 -- 172.21.15.46:0/2293216638 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1de00eba40 con 0x7f1df815a620 2024-03-29T08:19:04.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.128+0000 7f1df7200640 1 -- 172.21.15.46:0/2293216638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1df80c4fc0 con 0x7f1df815a620 2024-03-29T08:19:04.129 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-03-29T08:19:04.128+0000 7fc16ae00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:19:04.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.136+0000 7f1dee000640 1 -- 172.21.15.46:0/2293216638 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1df80c4fc0 con 0x7f1df815a620 2024-03-29T08:19:04.136 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T08:19:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.136+0000 7f1df7200640 1 -- 172.21.15.46:0/2293216638 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1dd4066690 msgr2=0x7f1dd4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.136+0000 7f1df7200640 1 --2- 172.21.15.46:0/2293216638 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1dd4066690 0x7f1dd4068b50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1de8007a60 tx=0x7f1de8008040 comp rx=0 tx=0).stop 2024-03-29T08:19:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.136+0000 7f1df7200640 1 -- 172.21.15.46:0/2293216638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df815a620 msgr2=0x7f1df80c42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:04.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.136+0000 7f1df7200640 1 --2- 172.21.15.46:0/2293216638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df815a620 0x7f1df80c42f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1de001abe0 tx=0x7f1de00192d0 comp rx=0 tx=0).stop 2024-03-29T08:19:04.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.136+0000 7f1df7200640 1 -- 172.21.15.46:0/2293216638 shutdown_connections 2024-03-29T08:19:04.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.136+0000 7f1df7200640 1 --2- 172.21.15.46:0/2293216638 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1dd4066690 0x7f1dd4068b50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:04.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.136+0000 7f1df7200640 1 --2- 172.21.15.46:0/2293216638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1df815f170 0x7f1df80c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:04.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.136+0000 7f1df7200640 1 --2- 172.21.15.46:0/2293216638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1df815a620 0x7f1df80c42f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:04.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.136+0000 7f1df7200640 1 --2- 172.21.15.46:0/2293216638 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1df80d04c0 0x7f1df80c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:04.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.136+0000 7f1df7200640 1 -- 172.21.15.46:0/2293216638 >> 172.21.15.46:0/2293216638 conn(0x7f1df8089fd0 msgr2=0x7f1df815df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:04.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.136+0000 7f1df7200640 1 -- 172.21.15.46:0/2293216638 shutdown_connections 2024-03-29T08:19:04.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.136+0000 7f1df7200640 1 -- 172.21.15.46:0/2293216638 wait complete. 2024-03-29T08:19:04.144 INFO:tasks.ceph.mds.b.smithi073.stderr: -2911> 2024-03-29T08:18:17.414+0000 7fc949600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:19:04.158 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T08:19:04.194 INFO:tasks.ceph.mds.b.smithi073.stdout:starting mds.b at 2024-03-29T08:19:04.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe064a00640 1 Processor -- start 2024-03-29T08:19:04.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe064a00640 1 -- start start 2024-03-29T08:19:04.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe064a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe0600d0dc0 0x7fe0600cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:04.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe064a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0600cf820 0x7fe0600b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:04.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe064a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe06015cf40 con 0x7fe06015cb60 2024-03-29T08:19:04.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe064a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0600b0250 con 0x7fe0600d0dc0 2024-03-29T08:19:04.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe064a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0600b0810 con 0x7fe0600cf820 2024-03-29T08:19:04.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05de00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe06015cb60 0x7fe0600d06b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54810/0 (socket says 172.21.15.46:54810) 2024-03-29T08:19:04.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05de00640 1 -- 172.21.15.46:0/1746745521 learned_addr learned my addr 172.21.15.46:0/1746745521 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05e800640 1 --2- 172.21.15.46:0/1746745521 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0600cf820 0x7fe0600b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe057e00640 1 --2- 172.21.15.46:0/1746745521 >> v2:172.21.15.73:3300/0 conn(0x7fe0600d0dc0 0x7fe0600cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05d400640 1 -- 172.21.15.46:0/1746745521 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1098779298 0 0) 0x7fe06015cf40 con 0x7fe06015cb60 2024-03-29T08:19:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05d400640 1 -- 172.21.15.46:0/1746745521 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe044003530 con 0x7fe06015cb60 2024-03-29T08:19:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05d400640 1 -- 172.21.15.46:0/1746745521 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 362023811 0 0) 0x7fe044003530 con 0x7fe06015cb60 2024-03-29T08:19:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05d400640 1 -- 172.21.15.46:0/1746745521 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0600cf820 msgr2=0x7fe0600b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05d400640 1 --2- 172.21.15.46:0/1746745521 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0600cf820 0x7fe0600b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05d400640 1 -- 172.21.15.46:0/1746745521 >> v2:172.21.15.73:3300/0 conn(0x7fe0600d0dc0 msgr2=0x7fe0600cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05d400640 1 --2- 172.21.15.46:0/1746745521 >> v2:172.21.15.73:3300/0 conn(0x7fe0600d0dc0 0x7fe0600cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05d400640 1 -- 172.21.15.46:0/1746745521 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0600d0010 con 0x7fe06015cb60 2024-03-29T08:19:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05d400640 1 -- 172.21.15.46:0/1746745521 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fe050001680 con 0x7fe06015cb60 2024-03-29T08:19:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05d400640 1 -- 172.21.15.46:0/1746745521 >> v1:172.21.15.46:6789/0 conn(0x7fe06015cb60 legacy=0x7fe0600d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05d400640 1 --2- 172.21.15.46:0/1746745521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe044003980 0x7fe044003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05d400640 1 --2- 172.21.15.46:0/1746745521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0440047d0 0x7fe044004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05d400640 1 --2- 172.21.15.46:0/1746745521 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0600cf820 0x7fe044005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05d400640 1 -- 172.21.15.46:0/1746745521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0600b0250 con 0x7fe044003980 2024-03-29T08:19:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05d400640 1 -- 172.21.15.46:0/1746745521 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0600b0810 con 0x7fe0440047d0 2024-03-29T08:19:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05d400640 1 -- 172.21.15.46:0/1746745521 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04400b990 con 0x7fe0600cf820 2024-03-29T08:19:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05de00640 1 --2- 172.21.15.46:0/1746745521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe044003980 0x7fe044003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe057e00640 1 --2- 172.21.15.46:0/1746745521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0440047d0 0x7fe044004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05e800640 1 --2- 172.21.15.46:0/1746745521 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0600cf820 0x7fe044005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05de00640 1 -- 172.21.15.46:0/1746745521 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0600cf820 msgr2=0x7fe044005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05de00640 1 --2- 172.21.15.46:0/1746745521 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0600cf820 0x7fe044005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05de00640 1 -- 172.21.15.46:0/1746745521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0440047d0 msgr2=0x7fe044004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05de00640 1 --2- 172.21.15.46:0/1746745521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0440047d0 0x7fe044004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.300+0000 7fe05de00640 1 -- 172.21.15.46:0/1746745521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe044003530 con 0x7fe044003980 2024-03-29T08:19:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.304+0000 7fe05de00640 1 --2- 172.21.15.46:0/1746745521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe044003980 0x7fe044003da0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe05000c240 tx=0x7fe05001cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f303a0fe9a729079 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.304+0000 7fe05d400640 1 -- 172.21.15.46:0/1746745521 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0500058e0 con 0x7fe044003980 2024-03-29T08:19:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.304+0000 7fe05d400640 1 -- 172.21.15.46:0/1746745521 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe050005a40 con 0x7fe044003980 2024-03-29T08:19:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.304+0000 7fe05d400640 1 -- 172.21.15.46:0/1746745521 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe050005dd0 con 0x7fe044003980 2024-03-29T08:19:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.304+0000 7fe064a00640 1 -- 172.21.15.46:0/1746745521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe044003980 msgr2=0x7fe044003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.304+0000 7fe064a00640 1 --2- 172.21.15.46:0/1746745521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe044003980 0x7fe044003da0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe05000c240 tx=0x7fe05001cc50 comp rx=0 tx=0).stop 2024-03-29T08:19:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.304+0000 7fe05e800640 1 -- 172.21.15.46:0/1746745521 reap_dead start 2024-03-29T08:19:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.304+0000 7fe064a00640 1 -- 172.21.15.46:0/1746745521 shutdown_connections 2024-03-29T08:19:04.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.304+0000 7fe064a00640 1 -- 172.21.15.46:0/1746745521 >> 172.21.15.46:0/1746745521 conn(0x7fe060099a10 msgr2=0x7fe060089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.304+0000 7fe064a00640 1 -- 172.21.15.46:0/1746745521 shutdown_connections 2024-03-29T08:19:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.304+0000 7fe064a00640 1 -- 172.21.15.46:0/1746745521 wait complete. 2024-03-29T08:19:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe064a00640 1 Processor -- start 2024-03-29T08:19:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe064a00640 1 -- start start 2024-03-29T08:19:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe064a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe044003980 0x7fe06014d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe064a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0601476d0 0x7fe060156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe064a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0601593d0 0x7fe060159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe064a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe060147480 con 0x7fe0601476d0 2024-03-29T08:19:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe064a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe060156da0 con 0x7fe0601593d0 2024-03-29T08:19:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe064a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe060158f40 con 0x7fe044003980 2024-03-29T08:19:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe057e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0601476d0 0x7fe060156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe057e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0601476d0 0x7fe060156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55434/0 (socket says 172.21.15.46:55434) 2024-03-29T08:19:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe057e00640 1 -- 172.21.15.46:0/2931459048 learned_addr learned my addr 172.21.15.46:0/2931459048 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:04.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe05e800640 1 --2- 172.21.15.46:0/2931459048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0601593d0 0x7fe060159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe05de00640 1 --2- 172.21.15.46:0/2931459048 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe044003980 0x7fe06014d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe057e00640 1 -- 172.21.15.46:0/2931459048 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe044003980 msgr2=0x7fe06014d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe057e00640 1 --2- 172.21.15.46:0/2931459048 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe044003980 0x7fe06014d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe057e00640 1 -- 172.21.15.46:0/2931459048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0601593d0 msgr2=0x7fe060159850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe057e00640 1 --2- 172.21.15.46:0/2931459048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0601593d0 0x7fe060159850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe057e00640 1 -- 172.21.15.46:0/2931459048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe060158610 con 0x7fe0601476d0 2024-03-29T08:19:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe05de00640 1 --2- 172.21.15.46:0/2931459048 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe044003980 0x7fe06014d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:04.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe057e00640 1 --2- 172.21.15.46:0/2931459048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0601476d0 0x7fe060156430 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe048011a00 tx=0x7fe048011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:04.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe056000640 1 -- 172.21.15.46:0/2931459048 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04800faf0 con 0x7fe0601476d0 2024-03-29T08:19:04.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe056000640 1 -- 172.21.15.46:0/2931459048 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04800fc50 con 0x7fe0601476d0 2024-03-29T08:19:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe056000640 1 -- 172.21.15.46:0/2931459048 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe048014440 con 0x7fe0601476d0 2024-03-29T08:19:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe064a00640 1 -- 172.21.15.46:0/2931459048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0601588f0 con 0x7fe0601476d0 2024-03-29T08:19:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.308+0000 7fe064a00640 1 -- 172.21.15.46:0/2931459048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe060156970 con 0x7fe0601476d0 2024-03-29T08:19:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.312+0000 7fe064a00640 1 -- 172.21.15.46:0/2931459048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe06009e7d0 con 0x7fe0601476d0 2024-03-29T08:19:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.312+0000 7fe056000640 1 -- 172.21.15.46:0/2931459048 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe0480145a0 con 0x7fe0601476d0 2024-03-29T08:19:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.312+0000 7fe056000640 1 --2- 172.21.15.46:0/2931459048 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe03c066690 0x7fe03c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.312+0000 7fe056000640 1 -- 172.21.15.46:0/2931459048 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fe048084840 con 0x7fe0601476d0 2024-03-29T08:19:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.312+0000 7fe05de00640 1 --2- 172.21.15.46:0/2931459048 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe03c066690 0x7fe03c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:04.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.312+0000 7fe05de00640 1 --2- 172.21.15.46:0/2931459048 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe03c066690 0x7fe03c068b50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe0500aa000 tx=0x7fe050001a80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:04.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.316+0000 7fe056000640 1 -- 172.21.15.46:0/2931459048 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe048051c90 con 0x7fe0601476d0 2024-03-29T08:19:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:04.512+0000 7fe064a00640 1 -- 172.21.15.46:0/2931459048 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe060159080 con 0x7fe0601476d0 2024-03-29T08:19:05.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.136+0000 7fe056000640 1 -- 172.21.15.46:0/2931459048 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe0480220a0 con 0x7fe0601476d0 2024-03-29T08:19:05.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.140+0000 7fe064a00640 1 -- 172.21.15.46:0/2931459048 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe03c066690 msgr2=0x7fe03c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.140+0000 7fe064a00640 1 --2- 172.21.15.46:0/2931459048 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe03c066690 0x7fe03c068b50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe0500aa000 tx=0x7fe050001a80 comp rx=0 tx=0).stop 2024-03-29T08:19:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.140+0000 7fe064a00640 1 -- 172.21.15.46:0/2931459048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0601476d0 msgr2=0x7fe060156430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.140+0000 7fe064a00640 1 --2- 172.21.15.46:0/2931459048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0601476d0 0x7fe060156430 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe048011a00 tx=0x7fe048011ec0 comp rx=0 tx=0).stop 2024-03-29T08:19:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.140+0000 7fe064a00640 1 -- 172.21.15.46:0/2931459048 shutdown_connections 2024-03-29T08:19:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.140+0000 7fe064a00640 1 --2- 172.21.15.46:0/2931459048 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe03c066690 0x7fe03c068b50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.140+0000 7fe064a00640 1 --2- 172.21.15.46:0/2931459048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0601593d0 0x7fe060159850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.140+0000 7fe064a00640 1 --2- 172.21.15.46:0/2931459048 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0601476d0 0x7fe060156430 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.140+0000 7fe064a00640 1 --2- 172.21.15.46:0/2931459048 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe044003980 0x7fe06014d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.140+0000 7fe064a00640 1 -- 172.21.15.46:0/2931459048 >> 172.21.15.46:0/2931459048 conn(0x7fe060099a10 msgr2=0x7fe0600af680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.140+0000 7fe064a00640 1 -- 172.21.15.46:0/2931459048 shutdown_connections 2024-03-29T08:19:05.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.140+0000 7fe064a00640 1 -- 172.21.15.46:0/2931459048 wait complete. 2024-03-29T08:19:05.162 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T08:19:05.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.308+0000 7f8f06800640 1 Processor -- start 2024-03-29T08:19:05.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.308+0000 7f8f06800640 1 -- start start 2024-03-29T08:19:05.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.308+0000 7f8f06800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8f0015d160 0x7f8f0015d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:05.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.308+0000 7f8f06800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8f00069d90 0x7f8f0006a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.308+0000 7f8f06800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0015dcd0 con 0x7f8f00146410 2024-03-29T08:19:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.308+0000 7f8f06800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0006a760 con 0x7f8f00069d90 2024-03-29T08:19:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.308+0000 7f8f06800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f00160790 con 0x7f8f0015d160 2024-03-29T08:19:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.308+0000 7f8eff400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8f00146410 0x7f8f001611a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54820/0 (socket says 172.21.15.46:54820) 2024-03-29T08:19:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.308+0000 7f8eff400640 1 -- 172.21.15.46:0/4250635955 learned_addr learned my addr 172.21.15.46:0/4250635955 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.308+0000 7f8effe00640 1 --2- 172.21.15.46:0/4250635955 >> v2:172.21.15.46:3300/0 conn(0x7f8f00069d90 0x7f8f0006a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.308+0000 7f8ef6a00640 1 --2- 172.21.15.46:0/4250635955 >> v2:172.21.15.73:3300/0 conn(0x7f8f0015d160 0x7f8f0015d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8effe00640 1 -- 172.21.15.46:0/4250635955 >> v2:172.21.15.73:3300/0 conn(0x7f8f0015d160 msgr2=0x7f8f0015d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8effe00640 1 --2- 172.21.15.46:0/4250635955 >> v2:172.21.15.73:3300/0 conn(0x7f8f0015d160 0x7f8f0015d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8effe00640 1 -- 172.21.15.46:0/4250635955 >> v1:172.21.15.46:6789/0 conn(0x7f8f00146410 legacy=0x7f8f001611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8effe00640 1 -- 172.21.15.46:0/4250635955 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0015dfb0 con 0x7f8f00069d90 2024-03-29T08:19:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8efea00640 1 -- 172.21.15.46:0/4250635955 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623408237 0 0) 0x7f8f0015dcd0 con 0x7f8f00146410 2024-03-29T08:19:05.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8ef6a00640 1 --2- 172.21.15.46:0/4250635955 >> v2:172.21.15.73:3300/0 conn(0x7f8f0015d160 0x7f8f0015d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8effe00640 1 --2- 172.21.15.46:0/4250635955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f00069d90 0x7f8f0006a220 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8f00073580 tx=0x7f8ef801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d771511f435b3ac server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8efea00640 1 -- 172.21.15.46:0/4250635955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80a5850 con 0x7f8f00069d90 2024-03-29T08:19:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8efea00640 1 -- 172.21.15.46:0/4250635955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef80a59b0 con 0x7f8f00069d90 2024-03-29T08:19:05.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8efea00640 1 -- 172.21.15.46:0/4250635955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80a5d40 con 0x7f8f00069d90 2024-03-29T08:19:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8f06800640 1 -- 172.21.15.46:0/4250635955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f00069d90 msgr2=0x7f8f0006a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8f06800640 1 --2- 172.21.15.46:0/4250635955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f00069d90 0x7f8f0006a220 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8f00073580 tx=0x7f8ef801c8a0 comp rx=0 tx=0).stop 2024-03-29T08:19:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8f06800640 1 -- 172.21.15.46:0/4250635955 shutdown_connections 2024-03-29T08:19:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8f06800640 1 --2- 172.21.15.46:0/4250635955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f00069d90 0x7f8f0006a220 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8f06800640 1 --2- 172.21.15.46:0/4250635955 >> v2:172.21.15.73:3300/0 conn(0x7f8f0015d160 0x7f8f0015d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:05.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8f06800640 1 -- 172.21.15.46:0/4250635955 >> 172.21.15.46:0/4250635955 conn(0x7f8f000b05d0 msgr2=0x7f8f000b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:05.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8f06800640 1 -- 172.21.15.46:0/4250635955 shutdown_connections 2024-03-29T08:19:05.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.312+0000 7f8f06800640 1 -- 172.21.15.46:0/4250635955 wait complete. 2024-03-29T08:19:05.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8f06800640 1 Processor -- start 2024-03-29T08:19:05.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8f06800640 1 -- start start 2024-03-29T08:19:05.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8f06800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8f00069d90 0x7f8f0015c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8f06800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f00146410 0x7f8f00161f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8f06800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f0015d160 0x7f8f000ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8f06800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f000cd010 con 0x7f8f00146410 2024-03-29T08:19:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8f06800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f000cd270 con 0x7f8f0015d160 2024-03-29T08:19:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8f06800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f000cd550 con 0x7f8f00069d90 2024-03-29T08:19:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8ef6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f00146410 0x7f8f00161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8ef6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f00146410 0x7f8f00161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55454/0 (socket says 172.21.15.46:55454) 2024-03-29T08:19:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8ef6a00640 1 -- 172.21.15.46:0/620937320 learned_addr learned my addr 172.21.15.46:0/620937320 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8eff400640 1 --2- 172.21.15.46:0/620937320 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8f00069d90 0x7f8f0015c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8effe00640 1 --2- 172.21.15.46:0/620937320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f0015d160 0x7f8f000ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8ef6a00640 1 -- 172.21.15.46:0/620937320 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8f00069d90 msgr2=0x7f8f0015c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:05.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8ef6a00640 1 --2- 172.21.15.46:0/620937320 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8f00069d90 0x7f8f0015c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:05.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8ef6a00640 1 -- 172.21.15.46:0/620937320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f0015d160 msgr2=0x7f8f000ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:05.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8ef6a00640 1 --2- 172.21.15.46:0/620937320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f0015d160 0x7f8f000ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:05.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8ef6a00640 1 -- 172.21.15.46:0/620937320 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef8009a70 con 0x7f8f00146410 2024-03-29T08:19:05.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8effe00640 1 --2- 172.21.15.46:0/620937320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f0015d160 0x7f8f000ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:05.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8ef6a00640 1 --2- 172.21.15.46:0/620937320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f00146410 0x7f8f00161f70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800c7f0 tx=0x7f8ee800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:05.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8ef7e00640 1 -- 172.21.15.46:0/620937320 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8007820 con 0x7f8f00146410 2024-03-29T08:19:05.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8ef7e00640 1 -- 172.21.15.46:0/620937320 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee8007980 con 0x7f8f00146410 2024-03-29T08:19:05.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8f06800640 1 -- 172.21.15.46:0/620937320 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f000cd830 con 0x7f8f00146410 2024-03-29T08:19:05.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8ef7e00640 1 -- 172.21.15.46:0/620937320 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8007d30 con 0x7f8f00146410 2024-03-29T08:19:05.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.316+0000 7f8f06800640 1 -- 172.21.15.46:0/620937320 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f000cddf0 con 0x7f8f00146410 2024-03-29T08:19:05.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.320+0000 7f8f06800640 1 -- 172.21.15.46:0/620937320 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f00066c90 con 0x7f8f00146410 2024-03-29T08:19:05.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.320+0000 7f8ef7e00640 1 -- 172.21.15.46:0/620937320 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8ee80113f0 con 0x7f8f00146410 2024-03-29T08:19:05.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.320+0000 7f8ef7e00640 1 --2- 172.21.15.46:0/620937320 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8edc066690 0x7f8edc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:05.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.320+0000 7f8ef7e00640 1 -- 172.21.15.46:0/620937320 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f8ee8084660 con 0x7f8f00146410 2024-03-29T08:19:05.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.320+0000 7f8eff400640 1 --2- 172.21.15.46:0/620937320 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8edc066690 0x7f8edc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:05.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.320+0000 7f8eff400640 1 --2- 172.21.15.46:0/620937320 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8edc066690 0x7f8edc068b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0000f70 tx=0x7f8ef0007b60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:05.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.324+0000 7f8ef7e00640 1 -- 172.21.15.46:0/620937320 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8ee8051850 con 0x7f8f00146410 2024-03-29T08:19:05.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:05.488+0000 7f8f06800640 1 -- 172.21.15.46:0/620937320 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8f0006c750 con 0x7f8f00146410 2024-03-29T08:19:06.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.140+0000 7f8ef7e00640 1 -- 172.21.15.46:0/620937320 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8ee801f0d0 con 0x7f8f00146410 2024-03-29T08:19:06.141 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-03-29T08:19:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.144+0000 7f8f06800640 1 -- 172.21.15.46:0/620937320 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8edc066690 msgr2=0x7f8edc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.144+0000 7f8f06800640 1 --2- 172.21.15.46:0/620937320 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8edc066690 0x7f8edc068b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0000f70 tx=0x7f8ef0007b60 comp rx=0 tx=0).stop 2024-03-29T08:19:06.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.144+0000 7f8f06800640 1 -- 172.21.15.46:0/620937320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f00146410 msgr2=0x7f8f00161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:06.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.144+0000 7f8f06800640 1 --2- 172.21.15.46:0/620937320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f00146410 0x7f8f00161f70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800c7f0 tx=0x7f8ee800ccb0 comp rx=0 tx=0).stop 2024-03-29T08:19:06.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.144+0000 7f8f06800640 1 -- 172.21.15.46:0/620937320 shutdown_connections 2024-03-29T08:19:06.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.144+0000 7f8f06800640 1 --2- 172.21.15.46:0/620937320 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8edc066690 0x7f8edc068b50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:06.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.144+0000 7f8f06800640 1 --2- 172.21.15.46:0/620937320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f0015d160 0x7f8f000ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:06.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.144+0000 7f8f06800640 1 --2- 172.21.15.46:0/620937320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f00146410 0x7f8f00161f70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:06.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.144+0000 7f8f06800640 1 --2- 172.21.15.46:0/620937320 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8f00069d90 0x7f8f0015c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:06.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.144+0000 7f8f06800640 1 -- 172.21.15.46:0/620937320 >> 172.21.15.46:0/620937320 conn(0x7f8f000b05d0 msgr2=0x7f8f000afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:06.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.144+0000 7f8f06800640 1 -- 172.21.15.46:0/620937320 shutdown_connections 2024-03-29T08:19:06.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.144+0000 7f8f06800640 1 -- 172.21.15.46:0/620937320 wait complete. 2024-03-29T08:19:06.171 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T08:19:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.316+0000 7ff192400640 1 Processor -- start 2024-03-29T08:19:06.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.316+0000 7ff192400640 1 -- start start 2024-03-29T08:19:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.320+0000 7ff192400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff18c0cadb0 0x7ff18c0b1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.320+0000 7ff192400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff18c0cb220 con 0x7ff18c1598a0 2024-03-29T08:19:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.320+0000 7ff192400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff18c0b1bc0 con 0x7ff18c15dc30 2024-03-29T08:19:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.320+0000 7ff192400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c0b19b0 con 0x7ff18c0cadb0 2024-03-29T08:19:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.320+0000 7ff18b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff18c1598a0 0x7ff18c15d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54834/0 (socket says 172.21.15.46:54834) 2024-03-29T08:19:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.320+0000 7ff18b400640 1 -- 172.21.15.46:0/3970918679 learned_addr learned my addr 172.21.15.46:0/3970918679 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:06.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.320+0000 7ff18be00640 1 --2- 172.21.15.46:0/3970918679 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff18c0cadb0 0x7ff18c0b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:06.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.320+0000 7ff18aa00640 1 -- 172.21.15.46:0/3970918679 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700586871 0 0) 0x7ff17c001120 con 0x7ff18c1598a0 2024-03-29T08:19:06.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.320+0000 7ff18aa00640 1 -- 172.21.15.46:0/3970918679 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff170005740 con 0x7ff18c1598a0 2024-03-29T08:19:06.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.320+0000 7ff18aa00640 1 -- 172.21.15.46:0/3970918679 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668076345 0 0) 0x7ff18c0b1bc0 con 0x7ff18c15dc30 2024-03-29T08:19:06.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.320+0000 7ff18aa00640 1 -- 172.21.15.46:0/3970918679 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff17c001120 con 0x7ff18c15dc30 2024-03-29T08:19:06.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.320+0000 7ff18be00640 1 -- 172.21.15.46:0/3970918679 >> v1:172.21.15.73:6789/0 conn(0x7ff18c15dc30 legacy=0x7ff18c06c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:06.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.320+0000 7ff18be00640 1 -- 172.21.15.46:0/3970918679 >> v1:172.21.15.46:6789/0 conn(0x7ff18c1598a0 legacy=0x7ff18c15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:06.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.320+0000 7ff18be00640 1 -- 172.21.15.46:0/3970918679 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18c0b1ea0 con 0x7ff18c0cadb0 2024-03-29T08:19:06.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.320+0000 7ff18be00640 1 --2- 172.21.15.46:0/3970918679 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff18c0cadb0 0x7ff18c0b1470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff184009dc0 tx=0x7ff18401c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fccb735b2edb0064 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:06.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff18aa00640 1 -- 172.21.15.46:0/3970918679 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1840a5a20 con 0x7ff18c0cadb0 2024-03-29T08:19:06.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff18aa00640 1 -- 172.21.15.46:0/3970918679 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1840a5b80 con 0x7ff18c0cadb0 2024-03-29T08:19:06.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff18aa00640 1 -- 172.21.15.46:0/3970918679 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1840ad440 con 0x7ff18c0cadb0 2024-03-29T08:19:06.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff192400640 1 -- 172.21.15.46:0/3970918679 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff18c0cadb0 msgr2=0x7ff18c0b1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:06.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff192400640 1 --2- 172.21.15.46:0/3970918679 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff18c0cadb0 0x7ff18c0b1470 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff184009dc0 tx=0x7ff18401c960 comp rx=0 tx=0).stop 2024-03-29T08:19:06.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff192400640 1 -- 172.21.15.46:0/3970918679 shutdown_connections 2024-03-29T08:19:06.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff192400640 1 --2- 172.21.15.46:0/3970918679 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff18c0cadb0 0x7ff18c0b1470 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:06.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff192400640 1 -- 172.21.15.46:0/3970918679 >> 172.21.15.46:0/3970918679 conn(0x7ff18c0b42d0 msgr2=0x7ff18c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:06.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff192400640 1 -- 172.21.15.46:0/3970918679 shutdown_connections 2024-03-29T08:19:06.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff192400640 1 -- 172.21.15.46:0/3970918679 wait complete. 2024-03-29T08:19:06.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff192400640 1 Processor -- start 2024-03-29T08:19:06.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff192400640 1 -- start start 2024-03-29T08:19:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff192400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c0cadb0 0x7ff18c1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff192400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff18c1598a0 0x7ff18c0ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff192400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff18c15dc30 0x7ff18c0ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff192400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c1f5fe0 con 0x7ff18c0cadb0 2024-03-29T08:19:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff192400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c1f56b0 con 0x7ff18c1598a0 2024-03-29T08:19:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff192400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff18c1f5990 con 0x7ff18c15dc30 2024-03-29T08:19:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff18b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c0cadb0 0x7ff18c1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff18b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c0cadb0 0x7ff18c1572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55458/0 (socket says 172.21.15.46:55458) 2024-03-29T08:19:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff18b400640 1 -- 172.21.15.46:0/3364749256 learned_addr learned my addr 172.21.15.46:0/3364749256 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:06.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff183e00640 1 --2- 172.21.15.46:0/3364749256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff18c1598a0 0x7ff18c0ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.324+0000 7ff18be00640 1 --2- 172.21.15.46:0/3364749256 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff18c15dc30 0x7ff18c0ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.328+0000 7ff18b400640 1 -- 172.21.15.46:0/3364749256 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff18c15dc30 msgr2=0x7ff18c0ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.328+0000 7ff18b400640 1 --2- 172.21.15.46:0/3364749256 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff18c15dc30 0x7ff18c0ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.328+0000 7ff18b400640 1 -- 172.21.15.46:0/3364749256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff18c1598a0 msgr2=0x7ff18c0ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.328+0000 7ff18b400640 1 --2- 172.21.15.46:0/3364749256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff18c1598a0 0x7ff18c0ace40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.328+0000 7ff18b400640 1 -- 172.21.15.46:0/3364749256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff184009a70 con 0x7ff18c0cadb0 2024-03-29T08:19:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.328+0000 7ff18b400640 1 --2- 172.21.15.46:0/3364749256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c0cadb0 0x7ff18c1572d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00c730 tx=0x7ff17c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.328+0000 7ff183400640 1 -- 172.21.15.46:0/3364749256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c005ac0 con 0x7ff18c0cadb0 2024-03-29T08:19:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.328+0000 7ff183400640 1 -- 172.21.15.46:0/3364749256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c005c20 con 0x7ff18c0cadb0 2024-03-29T08:19:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.328+0000 7ff183400640 1 -- 172.21.15.46:0/3364749256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c007070 con 0x7ff18c0cadb0 2024-03-29T08:19:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.328+0000 7ff192400640 1 -- 172.21.15.46:0/3364749256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18c1f5c70 con 0x7ff18c0cadb0 2024-03-29T08:19:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.328+0000 7ff192400640 1 -- 172.21.15.46:0/3364749256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18c1f6470 con 0x7ff18c0cadb0 2024-03-29T08:19:06.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.328+0000 7ff192400640 1 -- 172.21.15.46:0/3364749256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18c066c90 con 0x7ff18c0cadb0 2024-03-29T08:19:06.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.328+0000 7ff183400640 1 -- 172.21.15.46:0/3364749256 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff17c0073e0 con 0x7ff18c0cadb0 2024-03-29T08:19:06.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.328+0000 7ff183400640 1 --2- 172.21.15.46:0/3364749256 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff168066690 0x7ff168068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:06.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.328+0000 7ff183400640 1 -- 172.21.15.46:0/3364749256 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5386+0+0 (secure 0 0 0) 0x7ff17c0837b0 con 0x7ff18c0cadb0 2024-03-29T08:19:06.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.332+0000 7ff183e00640 1 --2- 172.21.15.46:0/3364749256 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff168066690 0x7ff168068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:06.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.332+0000 7ff183e00640 1 --2- 172.21.15.46:0/3364749256 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff168066690 0x7ff168068b50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff1740011f0 tx=0x7ff174009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:06.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.332+0000 7ff183400640 1 -- 172.21.15.46:0/3364749256 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff17c050d00 con 0x7ff18c0cadb0 2024-03-29T08:19:06.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:06.496+0000 7ff192400640 1 -- 172.21.15.46:0/3364749256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff18c0cb220 con 0x7ff18c0cadb0 2024-03-29T08:19:07.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.152+0000 7ff183400640 1 -- 172.21.15.46:0/3364749256 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff18c0cb220 con 0x7ff18c0cadb0 2024-03-29T08:19:07.154 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-03-29T08:19:07.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.160+0000 7ff192400640 1 -- 172.21.15.46:0/3364749256 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff168066690 msgr2=0x7ff168068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:07.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.160+0000 7ff192400640 1 --2- 172.21.15.46:0/3364749256 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff168066690 0x7ff168068b50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff1740011f0 tx=0x7ff174009040 comp rx=0 tx=0).stop 2024-03-29T08:19:07.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.160+0000 7ff192400640 1 -- 172.21.15.46:0/3364749256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c0cadb0 msgr2=0x7ff18c1572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:07.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.160+0000 7ff192400640 1 --2- 172.21.15.46:0/3364749256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c0cadb0 0x7ff18c1572d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00c730 tx=0x7ff17c00cbf0 comp rx=0 tx=0).stop 2024-03-29T08:19:07.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.160+0000 7ff192400640 1 -- 172.21.15.46:0/3364749256 shutdown_connections 2024-03-29T08:19:07.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.160+0000 7ff192400640 1 --2- 172.21.15.46:0/3364749256 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff168066690 0x7ff168068b50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:07.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.160+0000 7ff192400640 1 --2- 172.21.15.46:0/3364749256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff18c1598a0 0x7ff18c0ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:07.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.160+0000 7ff192400640 1 --2- 172.21.15.46:0/3364749256 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff18c15dc30 0x7ff18c0ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:07.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.160+0000 7ff192400640 1 --2- 172.21.15.46:0/3364749256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c0cadb0 0x7ff18c1572d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:07.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.160+0000 7ff192400640 1 -- 172.21.15.46:0/3364749256 >> 172.21.15.46:0/3364749256 conn(0x7ff18c0b42d0 msgr2=0x7ff18c06dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:07.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.160+0000 7ff192400640 1 -- 172.21.15.46:0/3364749256 shutdown_connections 2024-03-29T08:19:07.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.160+0000 7ff192400640 1 -- 172.21.15.46:0/3364749256 wait complete. 2024-03-29T08:19:07.191 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T08:19:07.191 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-29T08:19:07.191 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T08:19:07.191 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:19:07.191 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T08:19:07.192 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-29T08:19:07.192 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T08:19:07.192 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:19:07.192 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T08:19:07.192 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-29T08:19:07.192 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T08:19:07.192 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:19:07.193 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T08:19:07.193 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-29T08:19:07.193 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T08:19:07.193 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:19:07.204 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T08:19:07.204 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T08:19:07.205 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T08:19:07.205 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T08:19:07.207 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T08:19:07.207 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T08:19:07.208 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T08:19:07.208 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T08:19:07.251 INFO:tasks.ceph.mds.a:Started 2024-03-29T08:19:07.251 DEBUG:teuthology.parallel:result is None 2024-03-29T08:19:07.252 INFO:tasks.ceph.mds.d:Started 2024-03-29T08:19:07.252 DEBUG:teuthology.parallel:result is None 2024-03-29T08:19:07.252 INFO:tasks.ceph.mds.b:Started 2024-03-29T08:19:07.253 DEBUG:teuthology.parallel:result is None 2024-03-29T08:19:07.253 INFO:tasks.ceph.mds.c:Started 2024-03-29T08:19:07.254 DEBUG:teuthology.parallel:result is None 2024-03-29T08:19:07.254 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-29T08:19:07.347 INFO:tasks.ceph.mds.b.smithi073.stdout:starting mds.b at 2024-03-29T08:19:07.348 INFO:tasks.ceph.mds.d.smithi073.stdout:starting mds.d at 2024-03-29T08:19:07.351 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-03-29T08:19:07.351 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-03-29T08:19:07.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.412+0000 7fc07ae00640 1 Processor -- start 2024-03-29T08:19:07.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.412+0000 7fc07ae00640 1 -- start start 2024-03-29T08:19:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.416+0000 7fc07ae00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc074158300 0x7fc07406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.416+0000 7fc07ae00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc0740709b0 0x7fc0740af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.416+0000 7fc07ae00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc07406c390 con 0x7fc0740af530 2024-03-29T08:19:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.416+0000 7fc07ae00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07406d600 con 0x7fc0740709b0 2024-03-29T08:19:07.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.416+0000 7fc07ae00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0740afff0 con 0x7fc074158300 2024-03-29T08:19:07.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.416+0000 7fc06b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc0740af530 0x7fc0740702a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58410/0 (socket says 172.21.15.46:58410) 2024-03-29T08:19:07.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.416+0000 7fc06b400640 1 -- 172.21.15.46:0/1014744948 learned_addr learned my addr 172.21.15.46:0/1014744948 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:07.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.416+0000 7fc078e00640 1 --2- 172.21.15.46:0/1014744948 >> v2:172.21.15.73:3300/0 conn(0x7fc0740709b0 0x7fc0740af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.416+0000 7fc073e00640 1 --2- 172.21.15.46:0/1014744948 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc074158300 0x7fc07406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.416+0000 7fc073400640 1 -- 172.21.15.46:0/1014744948 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2757251343 0 0) 0x7fc07406c390 con 0x7fc0740af530 2024-03-29T08:19:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.416+0000 7fc073400640 1 -- 172.21.15.46:0/1014744948 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc058003530 con 0x7fc0740af530 2024-03-29T08:19:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.416+0000 7fc078e00640 1 -- 172.21.15.46:0/1014744948 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc074158300 msgr2=0x7fc07406d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.416+0000 7fc078e00640 1 --2- 172.21.15.46:0/1014744948 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc074158300 0x7fc07406d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.416+0000 7fc078e00640 1 -- 172.21.15.46:0/1014744948 >> v1:172.21.15.46:6789/0 conn(0x7fc0740af530 legacy=0x7fc0740702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.416+0000 7fc078e00640 1 -- 172.21.15.46:0/1014744948 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07406c5e0 con 0x7fc0740709b0 2024-03-29T08:19:07.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.416+0000 7fc073e00640 1 --2- 172.21.15.46:0/1014744948 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc074158300 0x7fc07406d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:07.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.420+0000 7fc078e00640 1 --2- 172.21.15.46:0/1014744948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0740709b0 0x7fc0740af990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009d90 tx=0x7fc06c01c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b88ece7b6cf3acb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.420+0000 7fc073400640 1 -- 172.21.15.46:0/1014744948 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0a58e0 con 0x7fc0740709b0 2024-03-29T08:19:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.420+0000 7fc073400640 1 -- 172.21.15.46:0/1014744948 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06c0a5a40 con 0x7fc0740709b0 2024-03-29T08:19:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.420+0000 7fc073400640 1 -- 172.21.15.46:0/1014744948 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0a5dd0 con 0x7fc0740709b0 2024-03-29T08:19:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.420+0000 7fc07ae00640 1 -- 172.21.15.46:0/1014744948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0740709b0 msgr2=0x7fc0740af990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.420+0000 7fc07ae00640 1 --2- 172.21.15.46:0/1014744948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0740709b0 0x7fc0740af990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009d90 tx=0x7fc06c01c820 comp rx=0 tx=0).stop 2024-03-29T08:19:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.420+0000 7fc07ae00640 1 -- 172.21.15.46:0/1014744948 shutdown_connections 2024-03-29T08:19:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.420+0000 7fc07ae00640 1 --2- 172.21.15.46:0/1014744948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0740709b0 0x7fc0740af990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.420+0000 7fc07ae00640 1 --2- 172.21.15.46:0/1014744948 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc074158300 0x7fc07406d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.420+0000 7fc07ae00640 1 -- 172.21.15.46:0/1014744948 >> 172.21.15.46:0/1014744948 conn(0x7fc074066080 msgr2=0x7fc074069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:07.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.420+0000 7fc07ae00640 1 -- 172.21.15.46:0/1014744948 shutdown_connections 2024-03-29T08:19:07.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.420+0000 7fc07ae00640 1 -- 172.21.15.46:0/1014744948 wait complete. 2024-03-29T08:19:07.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.420+0000 7fc07ae00640 1 Processor -- start 2024-03-29T08:19:07.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.420+0000 7fc07ae00640 1 -- start start 2024-03-29T08:19:07.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc07ae00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0740709b0 0x7fc0740ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:07.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc07ae00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0740af530 0x7fc0740cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:07.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc07ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc074158300 0x7fc0740cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:07.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc07ae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0741440c0 con 0x7fc074158300 2024-03-29T08:19:07.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc07ae00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0740ae5d0 con 0x7fc0740af530 2024-03-29T08:19:07.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc07ae00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0740a9560 con 0x7fc0740709b0 2024-03-29T08:19:07.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc078e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc074158300 0x7fc0740cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:07.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc073e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0740709b0 0x7fc0740ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:07.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc078e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc074158300 0x7fc0740cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56036/0 (socket says 172.21.15.46:56036) 2024-03-29T08:19:07.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc078e00640 1 -- 172.21.15.46:0/193456033 learned_addr learned my addr 172.21.15.46:0/193456033 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:07.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc06b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0740af530 0x7fc0740cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:07.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc073e00640 1 -- 172.21.15.46:0/193456033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0740af530 msgr2=0x7fc0740cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:07.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc073e00640 1 --2- 172.21.15.46:0/193456033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0740af530 0x7fc0740cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:07.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc073e00640 1 -- 172.21.15.46:0/193456033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc074158300 msgr2=0x7fc0740cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:07.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc073e00640 1 --2- 172.21.15.46:0/193456033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc074158300 0x7fc0740cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:07.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc073e00640 1 -- 172.21.15.46:0/193456033 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc064009420 con 0x7fc0740709b0 2024-03-29T08:19:07.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc06b400640 1 --2- 172.21.15.46:0/193456033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0740af530 0x7fc0740cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:07.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc073e00640 1 --2- 172.21.15.46:0/193456033 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0740709b0 0x7fc0740ae090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc06400e8e0 tx=0x7fc06400eda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:07.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc070c00640 1 -- 172.21.15.46:0/193456033 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06400c8e0 con 0x7fc0740709b0 2024-03-29T08:19:07.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc070c00640 1 -- 172.21.15.46:0/193456033 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06400ca40 con 0x7fc0740709b0 2024-03-29T08:19:07.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc070c00640 1 -- 172.21.15.46:0/193456033 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06400cdf0 con 0x7fc0740709b0 2024-03-29T08:19:07.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc07ae00640 1 -- 172.21.15.46:0/193456033 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06c009a70 con 0x7fc0740709b0 2024-03-29T08:19:07.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc07ae00640 1 -- 172.21.15.46:0/193456033 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0740a9b20 con 0x7fc0740709b0 2024-03-29T08:19:07.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc07ae00640 1 -- 172.21.15.46:0/193456033 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc074071a90 con 0x7fc0740709b0 2024-03-29T08:19:07.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc070c00640 1 -- 172.21.15.46:0/193456033 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc064013470 con 0x7fc0740709b0 2024-03-29T08:19:07.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.424+0000 7fc070c00640 1 --2- 172.21.15.46:0/193456033 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0500666c0 0x7fc050068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:07.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.428+0000 7fc070c00640 1 -- 172.21.15.46:0/193456033 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc064083fe0 con 0x7fc0740709b0 2024-03-29T08:19:07.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.428+0000 7fc06b400640 1 --2- 172.21.15.46:0/193456033 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0500666c0 0x7fc050068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:07.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.428+0000 7fc06b400640 1 --2- 172.21.15.46:0/193456033 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0500666c0 0x7fc050068b80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc0740cce00 tx=0x7fc05c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:07.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.432+0000 7fc070c00640 1 -- 172.21.15.46:0/193456033 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc064051750 con 0x7fc0740709b0 2024-03-29T08:19:07.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:07.600+0000 7fc07ae00640 1 -- 172.21.15.46:0/193456033 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc0740a9df0 con 0x7fc0740709b0 2024-03-29T08:19:08.344 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:19:08.348 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:19:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.384+0000 7fc070c00640 1 -- 172.21.15.46:0/193456033 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc064054d70 con 0x7fc0740709b0 2024-03-29T08:19:08.384 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-03-29T08:19:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.384+0000 7fc07ae00640 1 -- 172.21.15.46:0/193456033 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0500666c0 msgr2=0x7fc050068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.384+0000 7fc07ae00640 1 --2- 172.21.15.46:0/193456033 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0500666c0 0x7fc050068b80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc0740cce00 tx=0x7fc05c009040 comp rx=0 tx=0).stop 2024-03-29T08:19:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.384+0000 7fc07ae00640 1 -- 172.21.15.46:0/193456033 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0740709b0 msgr2=0x7fc0740ae090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.384+0000 7fc07ae00640 1 --2- 172.21.15.46:0/193456033 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0740709b0 0x7fc0740ae090 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc06400e8e0 tx=0x7fc06400eda0 comp rx=0 tx=0).stop 2024-03-29T08:19:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.384+0000 7fc07ae00640 1 -- 172.21.15.46:0/193456033 shutdown_connections 2024-03-29T08:19:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.384+0000 7fc07ae00640 1 --2- 172.21.15.46:0/193456033 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0500666c0 0x7fc050068b80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.384+0000 7fc07ae00640 1 --2- 172.21.15.46:0/193456033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0740af530 0x7fc0740cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.384+0000 7fc07ae00640 1 --2- 172.21.15.46:0/193456033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc074158300 0x7fc0740cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.384+0000 7fc07ae00640 1 --2- 172.21.15.46:0/193456033 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0740709b0 0x7fc0740ae090 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.384+0000 7fc07ae00640 1 -- 172.21.15.46:0/193456033 >> 172.21.15.46:0/193456033 conn(0x7fc074066080 msgr2=0x7fc0740702a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.384+0000 7fc07ae00640 1 -- 172.21.15.46:0/193456033 shutdown_connections 2024-03-29T08:19:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.384+0000 7fc07ae00640 1 -- 172.21.15.46:0/193456033 wait complete. 2024-03-29T08:19:08.411 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-29T08:19:08.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd582000640 1 Processor -- start 2024-03-29T08:19:08.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd582000640 1 -- start start 2024-03-29T08:19:08.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd582000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd57c0d0660 0x7fd57c0d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:08.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd582000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd57c06b8b0 0x7fd57c0cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:08.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd582000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd57c15cfb0 con 0x7fd57c15dc00 2024-03-29T08:19:08.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd582000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd57c06c100 con 0x7fd57c0d0660 2024-03-29T08:19:08.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd582000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c0d0f80 con 0x7fd57c06b8b0 2024-03-29T08:19:08.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd57be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd57c15dc00 0x7fd57c0cf7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58420/0 (socket says 172.21.15.46:58420) 2024-03-29T08:19:08.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd57be00640 1 -- 172.21.15.46:0/3482744340 learned_addr learned my addr 172.21.15.46:0/3482744340 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:08.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd57b400640 1 --2- 172.21.15.46:0/3482744340 >> v2:172.21.15.73:3300/0 conn(0x7fd57c0d0660 0x7fd57c0d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd572a00640 1 --2- 172.21.15.46:0/3482744340 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd57c06b8b0 0x7fd57c0cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd57aa00640 1 -- 172.21.15.46:0/3482744340 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984550412 0 0) 0x7fd57c15cfb0 con 0x7fd57c15dc00 2024-03-29T08:19:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd57aa00640 1 -- 172.21.15.46:0/3482744340 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd560003530 con 0x7fd57c15dc00 2024-03-29T08:19:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd57b400640 1 -- 172.21.15.46:0/3482744340 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd57c06b8b0 msgr2=0x7fd57c0cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd57b400640 1 --2- 172.21.15.46:0/3482744340 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd57c06b8b0 0x7fd57c0cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd57b400640 1 -- 172.21.15.46:0/3482744340 >> v1:172.21.15.46:6789/0 conn(0x7fd57c15dc00 legacy=0x7fd57c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd57b400640 1 -- 172.21.15.46:0/3482744340 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c06be20 con 0x7fd57c0d0660 2024-03-29T08:19:08.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.604+0000 7fd572a00640 1 --2- 172.21.15.46:0/3482744340 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd57c06b8b0 0x7fd57c0cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:08.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.608+0000 7fd57b400640 1 --2- 172.21.15.46:0/3482744340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd57c0d0660 0x7fd57c0d0a40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009d90 tx=0x7fd56c01c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9e0d78d694f16f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:08.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.608+0000 7fd57aa00640 1 -- 172.21.15.46:0/3482744340 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c0a5a00 con 0x7fd57c0d0660 2024-03-29T08:19:08.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.608+0000 7fd57aa00640 1 -- 172.21.15.46:0/3482744340 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56c0a5b60 con 0x7fd57c0d0660 2024-03-29T08:19:08.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.608+0000 7fd57aa00640 1 -- 172.21.15.46:0/3482744340 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56c0ad440 con 0x7fd57c0d0660 2024-03-29T08:19:08.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.608+0000 7fd582000640 1 -- 172.21.15.46:0/3482744340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd57c0d0660 msgr2=0x7fd57c0d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:08.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.608+0000 7fd582000640 1 --2- 172.21.15.46:0/3482744340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd57c0d0660 0x7fd57c0d0a40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009d90 tx=0x7fd56c01c940 comp rx=0 tx=0).stop 2024-03-29T08:19:08.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.608+0000 7fd582000640 1 -- 172.21.15.46:0/3482744340 shutdown_connections 2024-03-29T08:19:08.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.608+0000 7fd582000640 1 --2- 172.21.15.46:0/3482744340 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd57c06b8b0 0x7fd57c0cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:08.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.608+0000 7fd582000640 1 --2- 172.21.15.46:0/3482744340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd57c0d0660 0x7fd57c0d0a40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:08.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.608+0000 7fd582000640 1 -- 172.21.15.46:0/3482744340 >> 172.21.15.46:0/3482744340 conn(0x7fd57c089fd0 msgr2=0x7fd57c08a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:08.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.608+0000 7fd582000640 1 -- 172.21.15.46:0/3482744340 shutdown_connections 2024-03-29T08:19:08.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.608+0000 7fd582000640 1 -- 172.21.15.46:0/3482744340 wait complete. 2024-03-29T08:19:08.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd582000640 1 Processor -- start 2024-03-29T08:19:08.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd582000640 1 -- start start 2024-03-29T08:19:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd582000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd57c06b8b0 0x7fd57c152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd582000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd57c0d0660 0x7fd57c1508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd582000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd57c15dc00 0x7fd57c151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd582000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c0c58b0 con 0x7fd57c15dc00 2024-03-29T08:19:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd582000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c150e00 con 0x7fd57c0d0660 2024-03-29T08:19:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd582000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c0c5490 con 0x7fd57c06b8b0 2024-03-29T08:19:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd57be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd57c15dc00 0x7fd57c151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd57be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd57c15dc00 0x7fd57c151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56046/0 (socket says 172.21.15.46:56046) 2024-03-29T08:19:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd57be00640 1 -- 172.21.15.46:0/126579557 learned_addr learned my addr 172.21.15.46:0/126579557 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd572a00640 1 --2- 172.21.15.46:0/126579557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd57c0d0660 0x7fd57c1508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd57b400640 1 --2- 172.21.15.46:0/126579557 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd57c06b8b0 0x7fd57c152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd57be00640 1 -- 172.21.15.46:0/126579557 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd57c06b8b0 msgr2=0x7fd57c152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd57be00640 1 --2- 172.21.15.46:0/126579557 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd57c06b8b0 0x7fd57c152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd57be00640 1 -- 172.21.15.46:0/126579557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd57c0d0660 msgr2=0x7fd57c1508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd57be00640 1 --2- 172.21.15.46:0/126579557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd57c0d0660 0x7fd57c1508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd57be00640 1 -- 172.21.15.46:0/126579557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd564009420 con 0x7fd57c15dc00 2024-03-29T08:19:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd572a00640 1 --2- 172.21.15.46:0/126579557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd57c0d0660 0x7fd57c1508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd57be00640 1 --2- 172.21.15.46:0/126579557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd57c15dc00 0x7fd57c151040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd57400c450 tx=0x7fd57400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd573e00640 1 -- 172.21.15.46:0/126579557 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574005cf0 con 0x7fd57c15dc00 2024-03-29T08:19:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd573e00640 1 -- 172.21.15.46:0/126579557 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd574005e50 con 0x7fd57c15dc00 2024-03-29T08:19:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd573e00640 1 -- 172.21.15.46:0/126579557 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5740072b0 con 0x7fd57c15dc00 2024-03-29T08:19:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd582000640 1 -- 172.21.15.46:0/126579557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56c009a70 con 0x7fd57c15dc00 2024-03-29T08:19:08.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.612+0000 7fd582000640 1 -- 172.21.15.46:0/126579557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c0c70a0 con 0x7fd57c15dc00 2024-03-29T08:19:08.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.616+0000 7fd582000640 1 -- 172.21.15.46:0/126579557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57c066c90 con 0x7fd57c15dc00 2024-03-29T08:19:08.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.616+0000 7fd573e00640 1 -- 172.21.15.46:0/126579557 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd574007410 con 0x7fd57c15dc00 2024-03-29T08:19:08.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.616+0000 7fd573e00640 1 --2- 172.21.15.46:0/126579557 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd558066690 0x7fd558068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:08.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.616+0000 7fd573e00640 1 -- 172.21.15.46:0/126579557 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd574082fc0 con 0x7fd57c15dc00 2024-03-29T08:19:08.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.616+0000 7fd57b400640 1 --2- 172.21.15.46:0/126579557 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd558066690 0x7fd558068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:08.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.616+0000 7fd57b400640 1 --2- 172.21.15.46:0/126579557 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd558066690 0x7fd558068b50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0af000 tx=0x7fd56c0a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:08.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.620+0000 7fd573e00640 1 -- 172.21.15.46:0/126579557 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd574050720 con 0x7fd57c15dc00 2024-03-29T08:19:08.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.840+0000 7fd582000640 1 -- 172.21.15.46:0/126579557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd57c06e530 con 0x7fd57c15dc00 2024-03-29T08:19:08.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.840+0000 7fd573e00640 1 -- 172.21.15.46:0/126579557 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fd574017020 con 0x7fd57c15dc00 2024-03-29T08:19:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:08.841 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:19:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-03-29T08:19:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-03-29T08:19:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmlnXNJBAA7lI2lCBd0PeZudPsQyMOUQ==", 2024-03-29T08:19:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:19:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:19:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:19:08.841 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:19:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-03-29T08:19:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmbPkcMRAADnwJE17uILjJo14jghJuzg==", 2024-03-29T08:19:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:19:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:19:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:19:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:19:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.842 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-03-29T08:19:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmkCUVLBAAxcAOWGBDUCpEHeg/1OeQzA==", 2024-03-29T08:19:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:19:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:19:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:19:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:19:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-03-29T08:19:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZm7L11OBAAZ3fpXtEIYG1tGm9ZFswZ5Q==", 2024-03-29T08:19:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:19:08.843 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:19:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:19:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:19:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-03-29T08:19:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAIeQZmTHV8HhAAODJ+RP1Bx6k0D1153XgI0Q==", 2024-03-29T08:19:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:19:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:19:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:19:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.844 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-03-29T08:19:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQALeQZmZRD0JBAAjU6W/21CeqOMdIP5969iNA==", 2024-03-29T08:19:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:19:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:19:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:19:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-03-29T08:19:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAOeQZmSSP+KhAA26V8CByUJNfkVcG37AdlZQ==", 2024-03-29T08:19:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:19:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:19:08.845 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:19:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-03-29T08:19:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAReQZmb/e8LxAAV4yC1xlLwh8OAqOSPmcyyA==", 2024-03-29T08:19:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:19:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:19:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:19:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-03-29T08:19:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAXeQZmrLnzNRAAFPYQRs9sBNNKeyvcoI9rSg==", 2024-03-29T08:19:08.846 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:19:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:19:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:19:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-03-29T08:19:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAbeQZmjTPbABAAi4qTU5CBy9LtSSRf7cmfwg==", 2024-03-29T08:19:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:19:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:19:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:19:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.847 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-03-29T08:19:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAeeQZmo13eBRAAWY2VVBakLze9oA4ynp1rNA==", 2024-03-29T08:19:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:19:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:19:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:19:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-03-29T08:19:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAheQZmghpZCxAAdy5xI7qPwZb8LN8Rn6Ndaw==", 2024-03-29T08:19:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:19:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:19:08.848 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:19:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-03-29T08:19:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmyk22ABAA8Pn3OaIvxMQSLrKIM8FP4w==", 2024-03-29T08:19:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:19:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:19:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:19:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:19:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-03-29T08:19:08.849 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmG/E8ChAAJTcJ2j5xn0oUsmlV2trWdw==", 2024-03-29T08:19:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:19:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:19:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:19:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:19:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-03-29T08:19:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmhsqgBhAApqnO5qMG8VP27j9TfbZ8uw==", 2024-03-29T08:19:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:19:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:19:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:19:08.850 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:19:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-03-29T08:19:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmatlQEBAAAGKuGnAh4YXSk91TAzomXA==", 2024-03-29T08:19:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:19:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:19:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:19:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:19:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-03-29T08:19:08.851 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmp2B3CBAA4cfe/hUJeOg0SIr1KbWyUg==", 2024-03-29T08:19:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:19:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:19:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:19:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:19:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-mds", 2024-03-29T08:19:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAseQZmkgDHBhAAFJo+FL2BBzTpzDutD36Ehw==", 2024-03-29T08:19:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-mds" 2024-03-29T08:19:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-mgr", 2024-03-29T08:19:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAseQZmIQ3HBhAAKr9lWpYchzA75miCZqg9Cg==", 2024-03-29T08:19:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-29T08:19:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-osd", 2024-03-29T08:19:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAseQZmgxbHBhAAmUqKhSGhE08T1CXoPSzaPg==", 2024-03-29T08:19:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-osd" 2024-03-29T08:19:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rbd", 2024-03-29T08:19:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAseQZm+B/HBhAADCTwNqBKHB9QDKFwwSSq4g==", 2024-03-29T08:19:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-29T08:19:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-29T08:19:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAseQZmCCnHBhAA0niV8zVAyU0n+Gh6gh6c/w==", 2024-03-29T08:19:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-29T08:19:08.854 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rgw", 2024-03-29T08:19:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAseQZmDTLHBhAAmkOQtGm4dy+1Xk/wEcSiXw==", 2024-03-29T08:19:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-29T08:19:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-03-29T08:19:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmyrJXIBAAv535Rz99UI3V7JvvzQ8oSw==", 2024-03-29T08:19:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:19:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-03-29T08:19:08.855 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:19:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-03-29T08:19:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZm72tiHBAA2N7qQnR5Ehu/7nP2WpklGg==", 2024-03-29T08:19:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:19:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:19:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-03-29T08:19:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:19:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:08.856 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-29T08:19:08.856 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:19:08.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.840+0000 7fd582000640 1 -- 172.21.15.46:0/126579557 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd558066690 msgr2=0x7fd558068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.840+0000 7fd582000640 1 --2- 172.21.15.46:0/126579557 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd558066690 0x7fd558068b50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0af000 tx=0x7fd56c0a7070 comp rx=0 tx=0).stop 2024-03-29T08:19:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.840+0000 7fd582000640 1 -- 172.21.15.46:0/126579557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd57c15dc00 msgr2=0x7fd57c151040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.840+0000 7fd582000640 1 --2- 172.21.15.46:0/126579557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd57c15dc00 0x7fd57c151040 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd57400c450 tx=0x7fd57400c910 comp rx=0 tx=0).stop 2024-03-29T08:19:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.840+0000 7fd582000640 1 -- 172.21.15.46:0/126579557 shutdown_connections 2024-03-29T08:19:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.840+0000 7fd582000640 1 --2- 172.21.15.46:0/126579557 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd558066690 0x7fd558068b50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.840+0000 7fd582000640 1 --2- 172.21.15.46:0/126579557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd57c0d0660 0x7fd57c1508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.840+0000 7fd582000640 1 --2- 172.21.15.46:0/126579557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd57c15dc00 0x7fd57c151040 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.840+0000 7fd582000640 1 --2- 172.21.15.46:0/126579557 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd57c06b8b0 0x7fd57c152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.840+0000 7fd582000640 1 -- 172.21.15.46:0/126579557 >> 172.21.15.46:0/126579557 conn(0x7fd57c089fd0 msgr2=0x7fd57c0cb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.840+0000 7fd582000640 1 -- 172.21.15.46:0/126579557 shutdown_connections 2024-03-29T08:19:08.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:08.840+0000 7fd582000640 1 -- 172.21.15.46:0/126579557 wait complete. 2024-03-29T08:19:08.861 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-29T08:19:09.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.000+0000 7fdede000640 1 Processor -- start 2024-03-29T08:19:09.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.000+0000 7fdede000640 1 -- start start 2024-03-29T08:19:09.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.004+0000 7fdede000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fded8158300 0x7fded806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.004+0000 7fdede000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fded80af530 0x7fded80af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.004+0000 7fdede000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded80702a0 0x7fded8070730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.004+0000 7fdede000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded8070c70 con 0x7fded8158300 2024-03-29T08:19:09.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.004+0000 7fdede000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded806c390 con 0x7fded80af530 2024-03-29T08:19:09.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.004+0000 7fdede000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded806d600 con 0x7fded80702a0 2024-03-29T08:19:09.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.004+0000 7fded7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fded8158300 0x7fded806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.004+0000 7fded7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fded8158300 0x7fded806d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56054/0 (socket says 172.21.15.46:56054) 2024-03-29T08:19:09.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.004+0000 7fded7400640 1 -- 172.21.15.46:0/3000675164 learned_addr learned my addr 172.21.15.46:0/3000675164 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:09.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.004+0000 7fded7e00640 1 --2- 172.21.15.46:0/3000675164 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded80702a0 0x7fded8070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.004+0000 7fdecea00640 1 --2- 172.21.15.46:0/3000675164 >> v2:172.21.15.73:3300/0 conn(0x7fded80af530 0x7fded80af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.004+0000 7fded7400640 1 -- 172.21.15.46:0/3000675164 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded80702a0 msgr2=0x7fded8070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:09.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.004+0000 7fded7400640 1 --2- 172.21.15.46:0/3000675164 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded80702a0 0x7fded8070730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.004+0000 7fded7400640 1 -- 172.21.15.46:0/3000675164 >> v2:172.21.15.73:3300/0 conn(0x7fded80af530 msgr2=0x7fded80af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:09.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.004+0000 7fded7400640 1 --2- 172.21.15.46:0/3000675164 >> v2:172.21.15.73:3300/0 conn(0x7fded80af530 0x7fded80af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.004+0000 7fded7400640 1 -- 172.21.15.46:0/3000675164 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded80afff0 con 0x7fded8158300 2024-03-29T08:19:09.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fded7400640 1 --2- 172.21.15.46:0/3000675164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded8158300 0x7fded806d0c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fded80665f0 tx=0x7fdec801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505992b28ff7181d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:09.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fded6a00640 1 -- 172.21.15.46:0/3000675164 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec80a5720 con 0x7fded8158300 2024-03-29T08:19:09.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fded6a00640 1 -- 172.21.15.46:0/3000675164 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec80a5880 con 0x7fded8158300 2024-03-29T08:19:09.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fded6a00640 1 -- 172.21.15.46:0/3000675164 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec80a5c10 con 0x7fded8158300 2024-03-29T08:19:09.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fdede000640 1 -- 172.21.15.46:0/3000675164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded8158300 msgr2=0x7fded806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:09.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fdede000640 1 --2- 172.21.15.46:0/3000675164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded8158300 0x7fded806d0c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fded80665f0 tx=0x7fdec801c8a0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fdede000640 1 -- 172.21.15.46:0/3000675164 shutdown_connections 2024-03-29T08:19:09.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fdede000640 1 --2- 172.21.15.46:0/3000675164 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded80702a0 0x7fded8070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fdede000640 1 --2- 172.21.15.46:0/3000675164 >> v2:172.21.15.73:3300/0 conn(0x7fded80af530 0x7fded80af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fdede000640 1 --2- 172.21.15.46:0/3000675164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded8158300 0x7fded806d0c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fdede000640 1 -- 172.21.15.46:0/3000675164 >> 172.21.15.46:0/3000675164 conn(0x7fded8066080 msgr2=0x7fded8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:09.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fdede000640 1 -- 172.21.15.46:0/3000675164 shutdown_connections 2024-03-29T08:19:09.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fdede000640 1 -- 172.21.15.46:0/3000675164 wait complete. 2024-03-29T08:19:09.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fdede000640 1 Processor -- start 2024-03-29T08:19:09.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fdede000640 1 -- start start 2024-03-29T08:19:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fdede000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fded80702a0 0x7fded814cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fdede000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded80af530 0x7fded814d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fdede000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded8158300 0x7fded8152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fdede000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded814e130 con 0x7fded8158300 2024-03-29T08:19:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fdede000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded814e410 con 0x7fded80702a0 2024-03-29T08:19:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.008+0000 7fdede000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded814e6f0 con 0x7fded80af530 2024-03-29T08:19:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fded7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded8158300 0x7fded8152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fded7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded8158300 0x7fded8152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56064/0 (socket says 172.21.15.46:56064) 2024-03-29T08:19:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fded7e00640 1 -- 172.21.15.46:0/716371277 learned_addr learned my addr 172.21.15.46:0/716371277 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fded7400640 1 --2- 172.21.15.46:0/716371277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fded80702a0 0x7fded814cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fdecea00640 1 --2- 172.21.15.46:0/716371277 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded80af530 0x7fded814d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fded7e00640 1 -- 172.21.15.46:0/716371277 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded80af530 msgr2=0x7fded814d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fded7e00640 1 --2- 172.21.15.46:0/716371277 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded80af530 0x7fded814d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fded7e00640 1 -- 172.21.15.46:0/716371277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fded80702a0 msgr2=0x7fded814cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fded7e00640 1 --2- 172.21.15.46:0/716371277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fded80702a0 0x7fded814cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fded7e00640 1 -- 172.21.15.46:0/716371277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec8009b30 con 0x7fded8158300 2024-03-29T08:19:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fded7400640 1 --2- 172.21.15.46:0/716371277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fded80702a0 0x7fded814cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fdecea00640 1 --2- 172.21.15.46:0/716371277 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded80af530 0x7fded814d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fded7e00640 1 --2- 172.21.15.46:0/716371277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded8158300 0x7fded8152110 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fded000e930 tx=0x7fded000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fdecfe00640 1 -- 172.21.15.46:0/716371277 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded00096b0 con 0x7fded8158300 2024-03-29T08:19:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fdecfe00640 1 -- 172.21.15.46:0/716371277 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded000c9a0 con 0x7fded8158300 2024-03-29T08:19:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fdecfe00640 1 -- 172.21.15.46:0/716371277 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded000cd30 con 0x7fded8158300 2024-03-29T08:19:09.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fdede000640 1 -- 172.21.15.46:0/716371277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded814e9d0 con 0x7fded8158300 2024-03-29T08:19:09.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fdede000640 1 -- 172.21.15.46:0/716371277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded814d480 con 0x7fded8158300 2024-03-29T08:19:09.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fdede000640 1 -- 172.21.15.46:0/716371277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded8071a90 con 0x7fded8158300 2024-03-29T08:19:09.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fdecfe00640 1 -- 172.21.15.46:0/716371277 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fded0013470 con 0x7fded8158300 2024-03-29T08:19:09.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.012+0000 7fdecfe00640 1 --2- 172.21.15.46:0/716371277 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeb4066690 0x7fdeb4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.016+0000 7fdecfe00640 1 -- 172.21.15.46:0/716371277 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fded0084140 con 0x7fded8158300 2024-03-29T08:19:09.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.016+0000 7fded7400640 1 --2- 172.21.15.46:0/716371277 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeb4066690 0x7fdeb4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.016+0000 7fded7400640 1 --2- 172.21.15.46:0/716371277 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeb4066690 0x7fdeb4068b50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdec80ae000 tx=0x7fdec80a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:09.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.020+0000 7fdecfe00640 1 -- 172.21.15.46:0/716371277 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fded0051820 con 0x7fded8158300 2024-03-29T08:19:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.236+0000 7fdede000640 1 -- 172.21.15.46:0/716371277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fded804e120 con 0x7fded8158300 2024-03-29T08:19:09.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.240+0000 7fdecfe00640 1 -- 172.21.15.46:0/716371277 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fded0054e40 con 0x7fded8158300 2024-03-29T08:19:09.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.240+0000 7fdede000640 1 -- 172.21.15.46:0/716371277 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeb4066690 msgr2=0x7fdeb4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.244+0000 7fdede000640 1 --2- 172.21.15.46:0/716371277 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeb4066690 0x7fdeb4068b50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdec80ae000 tx=0x7fdec80a7070 comp rx=0 tx=0).stop 2024-03-29T08:19:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.244+0000 7fdede000640 1 -- 172.21.15.46:0/716371277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded8158300 msgr2=0x7fded8152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.244+0000 7fdede000640 1 --2- 172.21.15.46:0/716371277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded8158300 0x7fded8152110 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fded000e930 tx=0x7fded000edf0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.244+0000 7fdede000640 1 -- 172.21.15.46:0/716371277 shutdown_connections 2024-03-29T08:19:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.244+0000 7fdede000640 1 --2- 172.21.15.46:0/716371277 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeb4066690 0x7fdeb4068b50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.244+0000 7fdede000640 1 --2- 172.21.15.46:0/716371277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded8158300 0x7fded8152110 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.244+0000 7fdede000640 1 --2- 172.21.15.46:0/716371277 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded80af530 0x7fded814d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.244+0000 7fdede000640 1 --2- 172.21.15.46:0/716371277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fded80702a0 0x7fded814cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.244+0000 7fdede000640 1 -- 172.21.15.46:0/716371277 >> 172.21.15.46:0/716371277 conn(0x7fded8066080 msgr2=0x7fded81599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:09.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.244+0000 7fdede000640 1 -- 172.21.15.46:0/716371277 shutdown_connections 2024-03-29T08:19:09.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.244+0000 7fdede000640 1 -- 172.21.15.46:0/716371277 wait complete. 2024-03-29T08:19:09.262 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-29T08:19:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d1ce00640 1 Processor -- start 2024-03-29T08:19:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d1ce00640 1 -- start start 2024-03-29T08:19:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d1ce00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d180af530 0x7f4d180af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d1ce00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1806c390 con 0x7f4d180702a0 2024-03-29T08:19:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d1ce00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d180afff0 con 0x7f4d18158300 2024-03-29T08:19:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d1ce00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1806c5e0 con 0x7f4d180af530 2024-03-29T08:19:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d16800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4d180702a0 0x7f4d1815bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58432/0 (socket says 172.21.15.46:58432) 2024-03-29T08:19:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d16800640 1 -- 172.21.15.46:0/241224853 learned_addr learned my addr 172.21.15.46:0/241224853 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:09.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d0d400640 1 --2- 172.21.15.46:0/241224853 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d180af530 0x7f4d180af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15400640 1 -- 172.21.15.46:0/241224853 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 989787331 0 0) 0x7f4d1806c390 con 0x7f4d180702a0 2024-03-29T08:19:09.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15400640 1 -- 172.21.15.46:0/241224853 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cfc003530 con 0x7f4d180702a0 2024-03-29T08:19:09.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15400640 1 -- 172.21.15.46:0/241224853 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001887162 0 0) 0x7f4d180afff0 con 0x7f4d18158300 2024-03-29T08:19:09.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15400640 1 -- 172.21.15.46:0/241224853 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d1806c390 con 0x7f4d18158300 2024-03-29T08:19:09.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15400640 1 -- 172.21.15.46:0/241224853 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2322330792 0 0) 0x7f4cfc003530 con 0x7f4d180702a0 2024-03-29T08:19:09.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15400640 1 -- 172.21.15.46:0/241224853 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d180af530 msgr2=0x7f4d180af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:09.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15400640 1 --2- 172.21.15.46:0/241224853 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d180af530 0x7f4d180af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15400640 1 -- 172.21.15.46:0/241224853 >> v1:172.21.15.73:6789/0 conn(0x7f4d18158300 legacy=0x7f4d1806d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:09.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15400640 1 -- 172.21.15.46:0/241224853 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1806c8c0 con 0x7f4d180702a0 2024-03-29T08:19:09.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15400640 1 -- 172.21.15.46:0/241224853 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f4d10001680 con 0x7f4d180702a0 2024-03-29T08:19:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15400640 1 -- 172.21.15.46:0/241224853 >> v1:172.21.15.46:6789/0 conn(0x7f4d180702a0 legacy=0x7f4d1815bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15400640 1 --2- 172.21.15.46:0/241224853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cfc003b40 0x7f4cfc003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15400640 1 --2- 172.21.15.46:0/241224853 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d180af530 0x7f4cfc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15400640 1 --2- 172.21.15.46:0/241224853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cfc004d10 0x7f4cfc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15400640 1 -- 172.21.15.46:0/241224853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1806c5e0 con 0x7f4cfc004d10 2024-03-29T08:19:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15400640 1 -- 172.21.15.46:0/241224853 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc00bba0 con 0x7f4cfc003b40 2024-03-29T08:19:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15400640 1 -- 172.21.15.46:0/241224853 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc00bd60 con 0x7f4d180af530 2024-03-29T08:19:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d16800640 1 --2- 172.21.15.46:0/241224853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cfc004d10 0x7f4cfc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d15e00640 1 --2- 172.21.15.46:0/241224853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cfc003b40 0x7f4cfc003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d0d400640 1 --2- 172.21.15.46:0/241224853 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d180af530 0x7f4cfc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d16800640 1 -- 172.21.15.46:0/241224853 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d180af530 msgr2=0x7f4cfc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d16800640 1 --2- 172.21.15.46:0/241224853 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d180af530 0x7f4cfc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d16800640 1 -- 172.21.15.46:0/241224853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cfc003b40 msgr2=0x7f4cfc003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d16800640 1 --2- 172.21.15.46:0/241224853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cfc003b40 0x7f4cfc003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.404+0000 7f4d16800640 1 -- 172.21.15.46:0/241224853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc003530 con 0x7f4cfc004d10 2024-03-29T08:19:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.408+0000 7f4d16800640 1 --2- 172.21.15.46:0/241224853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cfc004d10 0x7f4cfc005190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4d10007f90 tx=0x7f4d1001cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53315e724f9d0c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.408+0000 7f4d15400640 1 -- 172.21.15.46:0/241224853 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10005ba0 con 0x7f4cfc004d10 2024-03-29T08:19:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.408+0000 7f4d15400640 1 -- 172.21.15.46:0/241224853 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d10005d00 con 0x7f4cfc004d10 2024-03-29T08:19:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.408+0000 7f4d15400640 1 -- 172.21.15.46:0/241224853 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d100a59e0 con 0x7f4cfc004d10 2024-03-29T08:19:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.408+0000 7f4d1ce00640 1 -- 172.21.15.46:0/241224853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cfc004d10 msgr2=0x7f4cfc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.408+0000 7f4d1ce00640 1 --2- 172.21.15.46:0/241224853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cfc004d10 0x7f4cfc005190 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4d10007f90 tx=0x7f4d1001cad0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.408+0000 7f4d16800640 1 -- 172.21.15.46:0/241224853 reap_dead start 2024-03-29T08:19:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.408+0000 7f4d1ce00640 1 -- 172.21.15.46:0/241224853 shutdown_connections 2024-03-29T08:19:09.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.408+0000 7f4d1ce00640 1 -- 172.21.15.46:0/241224853 >> 172.21.15.46:0/241224853 conn(0x7f4d18066080 msgr2=0x7f4d18069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.408+0000 7f4d1ce00640 1 -- 172.21.15.46:0/241224853 shutdown_connections 2024-03-29T08:19:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.408+0000 7f4d1ce00640 1 -- 172.21.15.46:0/241224853 wait complete. 2024-03-29T08:19:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d1ce00640 1 Processor -- start 2024-03-29T08:19:09.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d1ce00640 1 -- start start 2024-03-29T08:19:09.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d1ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cfc004d10 0x7f4d18147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d1ce00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d180ac180 0x7f4d180c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d1ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d181447b0 0x7f4d181471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d1ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d18145130 con 0x7f4cfc004d10 2024-03-29T08:19:09.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d1ce00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d18145cb0 con 0x7f4d181447b0 2024-03-29T08:19:09.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d1ce00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d180c3ae0 con 0x7f4d180ac180 2024-03-29T08:19:09.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d15e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cfc004d10 0x7f4d18147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d15e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cfc004d10 0x7f4d18147da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56086/0 (socket says 172.21.15.46:56086) 2024-03-29T08:19:09.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d15e00640 1 -- 172.21.15.46:0/512996471 learned_addr learned my addr 172.21.15.46:0/512996471 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:09.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d15e00640 1 -- 172.21.15.46:0/512996471 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d180ac180 msgr2=0x7f4d180c35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d15e00640 1 --2- 172.21.15.46:0/512996471 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d180ac180 0x7f4d180c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d15e00640 1 -- 172.21.15.46:0/512996471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d181447b0 msgr2=0x7f4d181471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d16800640 1 --2- 172.21.15.46:0/512996471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d181447b0 0x7f4d181471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d15e00640 1 --2- 172.21.15.46:0/512996471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d181447b0 0x7f4d181471c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d15e00640 1 -- 172.21.15.46:0/512996471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d18145380 con 0x7f4cfc004d10 2024-03-29T08:19:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d16800640 1 --2- 172.21.15.46:0/512996471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d181447b0 0x7f4d181471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d15e00640 1 --2- 172.21.15.46:0/512996471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cfc004d10 0x7f4d18147da0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800fb70 tx=0x7f4d0800d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d0ea00640 1 -- 172.21.15.46:0/512996471 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0800dbd0 con 0x7f4cfc004d10 2024-03-29T08:19:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d0ea00640 1 -- 172.21.15.46:0/512996471 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0800dd50 con 0x7f4cfc004d10 2024-03-29T08:19:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d0ea00640 1 -- 172.21.15.46:0/512996471 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d08006da0 con 0x7f4cfc004d10 2024-03-29T08:19:09.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d1ce00640 1 -- 172.21.15.46:0/512996471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d18145660 con 0x7f4cfc004d10 2024-03-29T08:19:09.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.412+0000 7f4d1ce00640 1 -- 172.21.15.46:0/512996471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d18144c30 con 0x7f4cfc004d10 2024-03-29T08:19:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.416+0000 7f4d1ce00640 1 -- 172.21.15.46:0/512996471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d18071a90 con 0x7f4cfc004d10 2024-03-29T08:19:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.416+0000 7f4d0ea00640 1 -- 172.21.15.46:0/512996471 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4d08003980 con 0x7f4cfc004d10 2024-03-29T08:19:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.416+0000 7f4d0ea00640 1 --2- 172.21.15.46:0/512996471 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4cf4066690 0x7f4cf4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.416+0000 7f4d0ea00640 1 -- 172.21.15.46:0/512996471 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4d080837f0 con 0x7f4cfc004d10 2024-03-29T08:19:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.416+0000 7f4d0d400640 1 --2- 172.21.15.46:0/512996471 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4cf4066690 0x7f4cf4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.416+0000 7f4d0d400640 1 --2- 172.21.15.46:0/512996471 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4cf4066690 0x7f4cf4068b50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4d18159b20 tx=0x7f4d000095f0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:09.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.420+0000 7f4d0ea00640 1 -- 172.21.15.46:0/512996471 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4d08050ea0 con 0x7f4cfc004d10 2024-03-29T08:19:09.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.636+0000 7f4d1ce00640 1 -- 172.21.15.46:0/512996471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f4d1806d900 con 0x7f4cfc004d10 2024-03-29T08:19:09.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.640+0000 7f4d0ea00640 1 -- 172.21.15.46:0/512996471 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4d080544c0 con 0x7f4cfc004d10 2024-03-29T08:19:09.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.640+0000 7f4d1ce00640 1 -- 172.21.15.46:0/512996471 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4cf4066690 msgr2=0x7f4cf4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:09.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.640+0000 7f4d1ce00640 1 --2- 172.21.15.46:0/512996471 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4cf4066690 0x7f4cf4068b50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4d18159b20 tx=0x7f4d000095f0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.640+0000 7f4d1ce00640 1 -- 172.21.15.46:0/512996471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cfc004d10 msgr2=0x7f4d18147da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:09.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.640+0000 7f4d1ce00640 1 --2- 172.21.15.46:0/512996471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cfc004d10 0x7f4d18147da0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4d0800fb70 tx=0x7f4d0800d520 comp rx=0 tx=0).stop 2024-03-29T08:19:09.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.640+0000 7f4d1ce00640 1 -- 172.21.15.46:0/512996471 shutdown_connections 2024-03-29T08:19:09.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.640+0000 7f4d1ce00640 1 --2- 172.21.15.46:0/512996471 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4cf4066690 0x7f4cf4068b50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.640+0000 7f4d1ce00640 1 --2- 172.21.15.46:0/512996471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d181447b0 0x7f4d181471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.640+0000 7f4d1ce00640 1 --2- 172.21.15.46:0/512996471 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d180ac180 0x7f4d180c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.640+0000 7f4d1ce00640 1 --2- 172.21.15.46:0/512996471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cfc004d10 0x7f4d18147da0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.640+0000 7f4d1ce00640 1 -- 172.21.15.46:0/512996471 >> 172.21.15.46:0/512996471 conn(0x7f4d18066080 msgr2=0x7f4d180b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:09.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.640+0000 7f4d1ce00640 1 -- 172.21.15.46:0/512996471 shutdown_connections 2024-03-29T08:19:09.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.640+0000 7f4d1ce00640 1 -- 172.21.15.46:0/512996471 wait complete. 2024-03-29T08:19:09.663 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-29T08:19:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.804+0000 7f7b78a00640 1 Processor -- start 2024-03-29T08:19:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.804+0000 7f7b78a00640 1 -- start start 2024-03-29T08:19:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.804+0000 7f7b78a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7b74142ab0 0x7f7b7406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.804+0000 7f7b78a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7b7415db10 0x7f7b740b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.804+0000 7f7b78a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7406de40 con 0x7f7b7406d9e0 2024-03-29T08:19:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.804+0000 7f7b78a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b74070c70 con 0x7f7b74142ab0 2024-03-29T08:19:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.804+0000 7f7b78a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b74070340 con 0x7f7b7415db10 2024-03-29T08:19:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.804+0000 7f7b71e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7b74142ab0 0x7f7b7406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.804+0000 7f7b69400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7b7406d9e0 0x7f7b7415d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58438/0 (socket says 172.21.15.46:58438) 2024-03-29T08:19:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.804+0000 7f7b69400640 1 -- 172.21.15.46:0/1085508209 learned_addr learned my addr 172.21.15.46:0/1085508209 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.804+0000 7f7b72800640 1 --2- 172.21.15.46:0/1085508209 >> v2:172.21.15.73:3300/0 conn(0x7f7b7415db10 0x7f7b740b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.804+0000 7f7b71e00640 1 -- 172.21.15.46:0/1085508209 >> v2:172.21.15.73:3300/0 conn(0x7f7b7415db10 msgr2=0x7f7b740b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.804+0000 7f7b71e00640 1 --2- 172.21.15.46:0/1085508209 >> v2:172.21.15.73:3300/0 conn(0x7f7b7415db10 0x7f7b740b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.804+0000 7f7b71e00640 1 -- 172.21.15.46:0/1085508209 >> v1:172.21.15.46:6789/0 conn(0x7f7b7406d9e0 legacy=0x7f7b7415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.804+0000 7f7b71e00640 1 -- 172.21.15.46:0/1085508209 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b74070620 con 0x7f7b74142ab0 2024-03-29T08:19:09.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.808+0000 7f7b71e00640 1 --2- 172.21.15.46:0/1085508209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b74142ab0 0x7f7b7406d4a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7b740b3ce0 tx=0x7f7b6401c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07a4a66376f0968 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:09.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.808+0000 7f7b71400640 1 -- 172.21.15.46:0/1085508209 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b640a5820 con 0x7f7b74142ab0 2024-03-29T08:19:09.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.808+0000 7f7b71400640 1 -- 172.21.15.46:0/1085508209 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b640a5980 con 0x7f7b74142ab0 2024-03-29T08:19:09.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.808+0000 7f7b71400640 1 -- 172.21.15.46:0/1085508209 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b640a5d10 con 0x7f7b74142ab0 2024-03-29T08:19:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.808+0000 7f7b78a00640 1 -- 172.21.15.46:0/1085508209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b74142ab0 msgr2=0x7f7b7406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.808+0000 7f7b78a00640 1 --2- 172.21.15.46:0/1085508209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b74142ab0 0x7f7b7406d4a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7b740b3ce0 tx=0x7f7b6401c7c0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.808+0000 7f7b78a00640 1 -- 172.21.15.46:0/1085508209 shutdown_connections 2024-03-29T08:19:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.808+0000 7f7b78a00640 1 --2- 172.21.15.46:0/1085508209 >> v2:172.21.15.73:3300/0 conn(0x7f7b7415db10 0x7f7b740b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.808+0000 7f7b78a00640 1 --2- 172.21.15.46:0/1085508209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b74142ab0 0x7f7b7406d4a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.808+0000 7f7b78a00640 1 -- 172.21.15.46:0/1085508209 >> 172.21.15.46:0/1085508209 conn(0x7f7b74066080 msgr2=0x7f7b74069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.808+0000 7f7b78a00640 1 -- 172.21.15.46:0/1085508209 shutdown_connections 2024-03-29T08:19:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.808+0000 7f7b78a00640 1 -- 172.21.15.46:0/1085508209 wait complete. 2024-03-29T08:19:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.808+0000 7f7b78a00640 1 Processor -- start 2024-03-29T08:19:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.808+0000 7f7b78a00640 1 -- start start 2024-03-29T08:19:09.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b78a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b7406d9e0 0x7f7b7413be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b78a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b74142ab0 0x7f7b74156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b78a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b7415db10 0x7f7b740aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b78a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b740a7d30 con 0x7f7b74142ab0 2024-03-29T08:19:09.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b78a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b740a8010 con 0x7f7b7406d9e0 2024-03-29T08:19:09.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b78a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b740a82f0 con 0x7f7b7415db10 2024-03-29T08:19:09.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b69400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b74142ab0 0x7f7b74156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b69400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b74142ab0 0x7f7b74156fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56098/0 (socket says 172.21.15.46:56098) 2024-03-29T08:19:09.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b69400640 1 -- 172.21.15.46:0/1064266758 learned_addr learned my addr 172.21.15.46:0/1064266758 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:09.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b71e00640 1 --2- 172.21.15.46:0/1064266758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b7406d9e0 0x7f7b7413be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b72800640 1 --2- 172.21.15.46:0/1064266758 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b7415db10 0x7f7b740aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b69400640 1 -- 172.21.15.46:0/1064266758 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b7415db10 msgr2=0x7f7b740aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:09.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b69400640 1 --2- 172.21.15.46:0/1064266758 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b7415db10 0x7f7b740aa3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b69400640 1 -- 172.21.15.46:0/1064266758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b7406d9e0 msgr2=0x7f7b7413be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:09.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b69400640 1 --2- 172.21.15.46:0/1064266758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b7406d9e0 0x7f7b7413be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:09.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b69400640 1 -- 172.21.15.46:0/1064266758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b64009a70 con 0x7f7b74142ab0 2024-03-29T08:19:09.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b72800640 1 --2- 172.21.15.46:0/1064266758 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b7415db10 0x7f7b740aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:09.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b69400640 1 --2- 172.21.15.46:0/1064266758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b74142ab0 0x7f7b74156fe0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009e80 tx=0x7f7b5c00c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:09.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b6aa00640 1 -- 172.21.15.46:0/1064266758 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0078f0 con 0x7f7b74142ab0 2024-03-29T08:19:09.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b6aa00640 1 -- 172.21.15.46:0/1064266758 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5c007a50 con 0x7f7b74142ab0 2024-03-29T08:19:09.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b78a00640 1 -- 172.21.15.46:0/1064266758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b740a85d0 con 0x7f7b74142ab0 2024-03-29T08:19:09.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b6aa00640 1 -- 172.21.15.46:0/1064266758 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c007e00 con 0x7f7b74142ab0 2024-03-29T08:19:09.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b78a00640 1 -- 172.21.15.46:0/1064266758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7413c370 con 0x7f7b74142ab0 2024-03-29T08:19:09.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.812+0000 7f7b78a00640 1 -- 172.21.15.46:0/1064266758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b74071a90 con 0x7f7b74142ab0 2024-03-29T08:19:09.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.816+0000 7f7b6aa00640 1 -- 172.21.15.46:0/1064266758 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7b5c009070 con 0x7f7b74142ab0 2024-03-29T08:19:09.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.816+0000 7f7b6aa00640 1 --2- 172.21.15.46:0/1064266758 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b50066690 0x7f7b50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:09.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.816+0000 7f7b6aa00640 1 -- 172.21.15.46:0/1064266758 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7b5c085d80 con 0x7f7b74142ab0 2024-03-29T08:19:09.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.816+0000 7f7b71e00640 1 --2- 172.21.15.46:0/1064266758 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b50066690 0x7f7b50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:09.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.816+0000 7f7b71e00640 1 --2- 172.21.15.46:0/1064266758 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b50066690 0x7f7b50068b50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7b640af000 tx=0x7f7b640a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:09.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:09.820+0000 7f7b6aa00640 1 -- 172.21.15.46:0/1064266758 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7b5c053450 con 0x7f7b74142ab0 2024-03-29T08:19:10.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.036+0000 7f7b78a00640 1 -- 172.21.15.46:0/1064266758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f7b7404e120 con 0x7f7b74142ab0 2024-03-29T08:19:10.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.040+0000 7f7b6aa00640 1 -- 172.21.15.46:0/1064266758 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7b5c056a70 con 0x7f7b74142ab0 2024-03-29T08:19:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.044+0000 7f7b78a00640 1 -- 172.21.15.46:0/1064266758 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b50066690 msgr2=0x7f7b50068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:10.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.044+0000 7f7b78a00640 1 --2- 172.21.15.46:0/1064266758 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b50066690 0x7f7b50068b50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7b640af000 tx=0x7f7b640a7070 comp rx=0 tx=0).stop 2024-03-29T08:19:10.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.044+0000 7f7b78a00640 1 -- 172.21.15.46:0/1064266758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b74142ab0 msgr2=0x7f7b74156fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:10.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.044+0000 7f7b78a00640 1 --2- 172.21.15.46:0/1064266758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b74142ab0 0x7f7b74156fe0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009e80 tx=0x7f7b5c00c430 comp rx=0 tx=0).stop 2024-03-29T08:19:10.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.044+0000 7f7b78a00640 1 -- 172.21.15.46:0/1064266758 shutdown_connections 2024-03-29T08:19:10.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.044+0000 7f7b78a00640 1 --2- 172.21.15.46:0/1064266758 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b50066690 0x7f7b50068b50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.044+0000 7f7b78a00640 1 --2- 172.21.15.46:0/1064266758 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b7415db10 0x7f7b740aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.044+0000 7f7b78a00640 1 --2- 172.21.15.46:0/1064266758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b74142ab0 0x7f7b74156fe0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.044+0000 7f7b78a00640 1 --2- 172.21.15.46:0/1064266758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b7406d9e0 0x7f7b7413be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.044+0000 7f7b78a00640 1 -- 172.21.15.46:0/1064266758 >> 172.21.15.46:0/1064266758 conn(0x7f7b74066080 msgr2=0x7f7b740b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:10.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.044+0000 7f7b78a00640 1 -- 172.21.15.46:0/1064266758 shutdown_connections 2024-03-29T08:19:10.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.044+0000 7f7b78a00640 1 -- 172.21.15.46:0/1064266758 wait complete. 2024-03-29T08:19:10.063 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-29T08:19:10.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.204+0000 7efd20200640 1 Processor -- start 2024-03-29T08:19:10.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.204+0000 7efd20200640 1 -- start start 2024-03-29T08:19:10.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.204+0000 7efd20200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efd18142ab0 0x7efd1806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:10.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.204+0000 7efd20200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efd1806d9e0 0x7efd1815d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:10.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.204+0000 7efd20200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1806de40 con 0x7efd1815d940 2024-03-29T08:19:10.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.204+0000 7efd20200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd18070c70 con 0x7efd1806d9e0 2024-03-29T08:19:10.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.204+0000 7efd20200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd18070340 con 0x7efd18142ab0 2024-03-29T08:19:10.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.204+0000 7efd1ce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efd1806d9e0 0x7efd1815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:10.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.208+0000 7efd1ce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efd1806d9e0 0x7efd1815d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56102/0 (socket says 172.21.15.46:56102) 2024-03-29T08:19:10.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.208+0000 7efd1ce00640 1 -- 172.21.15.46:0/4159781787 learned_addr learned my addr 172.21.15.46:0/4159781787 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:10.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.208+0000 7efd1d800640 1 --2- 172.21.15.46:0/4159781787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efd18142ab0 0x7efd1806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:10.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.208+0000 7efd1ce00640 1 -- 172.21.15.46:0/4159781787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efd18142ab0 msgr2=0x7efd1806d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:10.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.208+0000 7efd1ce00640 1 --2- 172.21.15.46:0/4159781787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efd18142ab0 0x7efd1806d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.208+0000 7efd1ce00640 1 -- 172.21.15.46:0/4159781787 >> v1:172.21.15.73:6789/0 conn(0x7efd1815d940 legacy=0x7efd180ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:10.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.208+0000 7efd1ce00640 1 -- 172.21.15.46:0/4159781787 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd18070620 con 0x7efd1806d9e0 2024-03-29T08:19:10.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.208+0000 7efd1d800640 1 --2- 172.21.15.46:0/4159781787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efd18142ab0 0x7efd1806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:10.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.208+0000 7efd1ce00640 1 --2- 172.21.15.46:0/4159781787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd1806d9e0 0x7efd1815d400 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efd04009870 tx=0x7efd0401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a3b3c72454df59 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:10.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.208+0000 7efd17e00640 1 -- 172.21.15.46:0/4159781787 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd040ab600 con 0x7efd1806d9e0 2024-03-29T08:19:10.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.208+0000 7efd17e00640 1 -- 172.21.15.46:0/4159781787 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd040a7050 con 0x7efd1806d9e0 2024-03-29T08:19:10.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.208+0000 7efd17e00640 1 -- 172.21.15.46:0/4159781787 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd040ab980 con 0x7efd1806d9e0 2024-03-29T08:19:10.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd20200640 1 -- 172.21.15.46:0/4159781787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd1806d9e0 msgr2=0x7efd1815d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:10.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd20200640 1 --2- 172.21.15.46:0/4159781787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd1806d9e0 0x7efd1815d400 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efd04009870 tx=0x7efd0401c560 comp rx=0 tx=0).stop 2024-03-29T08:19:10.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd20200640 1 -- 172.21.15.46:0/4159781787 shutdown_connections 2024-03-29T08:19:10.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd20200640 1 --2- 172.21.15.46:0/4159781787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd1806d9e0 0x7efd1815d400 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd20200640 1 --2- 172.21.15.46:0/4159781787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efd18142ab0 0x7efd1806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd20200640 1 -- 172.21.15.46:0/4159781787 >> 172.21.15.46:0/4159781787 conn(0x7efd18066080 msgr2=0x7efd18069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:10.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd20200640 1 -- 172.21.15.46:0/4159781787 shutdown_connections 2024-03-29T08:19:10.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd20200640 1 -- 172.21.15.46:0/4159781787 wait complete. 2024-03-29T08:19:10.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd20200640 1 Processor -- start 2024-03-29T08:19:10.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd20200640 1 -- start start 2024-03-29T08:19:10.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd20200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efd1806d9e0 0x7efd181f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd20200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd18142ab0 0x7efd181588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd20200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd1815d940 0x7efd180a7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd20200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1813cb50 con 0x7efd18142ab0 2024-03-29T08:19:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd20200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd18158de0 con 0x7efd1815d940 2024-03-29T08:19:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd20200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1813bed0 con 0x7efd1806d9e0 2024-03-29T08:19:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd1ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd18142ab0 0x7efd181588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd1ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd18142ab0 0x7efd181588a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56108/0 (socket says 172.21.15.46:56108) 2024-03-29T08:19:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd1ce00640 1 -- 172.21.15.46:0/2920367776 learned_addr learned my addr 172.21.15.46:0/2920367776 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd1d800640 1 --2- 172.21.15.46:0/2920367776 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efd1806d9e0 0x7efd181f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd1e200640 1 --2- 172.21.15.46:0/2920367776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd1815d940 0x7efd180a7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd1ce00640 1 -- 172.21.15.46:0/2920367776 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efd1806d9e0 msgr2=0x7efd181f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd1ce00640 1 --2- 172.21.15.46:0/2920367776 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efd1806d9e0 0x7efd181f5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd1ce00640 1 -- 172.21.15.46:0/2920367776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd1815d940 msgr2=0x7efd180a7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd1ce00640 1 --2- 172.21.15.46:0/2920367776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd1815d940 0x7efd180a7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd1ce00640 1 -- 172.21.15.46:0/2920367776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd040094d0 con 0x7efd18142ab0 2024-03-29T08:19:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd1e200640 1 --2- 172.21.15.46:0/2920367776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd1815d940 0x7efd180a7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.212+0000 7efd1ce00640 1 --2- 172.21.15.46:0/2920367776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd18142ab0 0x7efd181588a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efd040ae040 tx=0x7efd04019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.216+0000 7efd15600640 1 -- 172.21.15.46:0/2920367776 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd040b4420 con 0x7efd18142ab0 2024-03-29T08:19:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.216+0000 7efd15600640 1 -- 172.21.15.46:0/2920367776 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd040b4580 con 0x7efd18142ab0 2024-03-29T08:19:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.216+0000 7efd15600640 1 -- 172.21.15.46:0/2920367776 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd040b4930 con 0x7efd18142ab0 2024-03-29T08:19:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.216+0000 7efd20200640 1 -- 172.21.15.46:0/2920367776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1813c1b0 con 0x7efd18142ab0 2024-03-29T08:19:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.216+0000 7efd20200640 1 -- 172.21.15.46:0/2920367776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1813c770 con 0x7efd18142ab0 2024-03-29T08:19:10.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.216+0000 7efd20200640 1 -- 172.21.15.46:0/2920367776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1813cc90 con 0x7efd18142ab0 2024-03-29T08:19:10.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.216+0000 7efd15600640 1 -- 172.21.15.46:0/2920367776 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efd040b2070 con 0x7efd18142ab0 2024-03-29T08:19:10.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.216+0000 7efd15600640 1 --2- 172.21.15.46:0/2920367776 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efcf8066690 0x7efcf8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:10.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.216+0000 7efd15600640 1 -- 172.21.15.46:0/2920367776 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efd0411d070 con 0x7efd18142ab0 2024-03-29T08:19:10.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.216+0000 7efd1d800640 1 --2- 172.21.15.46:0/2920367776 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efcf8066690 0x7efcf8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:10.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.216+0000 7efd1d800640 1 --2- 172.21.15.46:0/2920367776 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efcf8066690 0x7efcf8068b50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efd1815be10 tx=0x7efd0c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:10.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.220+0000 7efd15600640 1 -- 172.21.15.46:0/2920367776 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efd1813cc90 con 0x7efd18142ab0 2024-03-29T08:19:10.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.440+0000 7efd20200640 1 -- 172.21.15.46:0/2920367776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7efd1813cc90 con 0x7efd18142ab0 2024-03-29T08:19:10.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.444+0000 7efd15600640 1 -- 172.21.15.46:0/2920367776 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7efd1813cc90 con 0x7efd18142ab0 2024-03-29T08:19:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.444+0000 7efd20200640 1 -- 172.21.15.46:0/2920367776 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efcf8066690 msgr2=0x7efcf8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.444+0000 7efd20200640 1 --2- 172.21.15.46:0/2920367776 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efcf8066690 0x7efcf8068b50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efd1815be10 tx=0x7efd0c008040 comp rx=0 tx=0).stop 2024-03-29T08:19:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.444+0000 7efd20200640 1 -- 172.21.15.46:0/2920367776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd18142ab0 msgr2=0x7efd181588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.444+0000 7efd20200640 1 --2- 172.21.15.46:0/2920367776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd18142ab0 0x7efd181588a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efd040ae040 tx=0x7efd04019300 comp rx=0 tx=0).stop 2024-03-29T08:19:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.444+0000 7efd20200640 1 -- 172.21.15.46:0/2920367776 shutdown_connections 2024-03-29T08:19:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.444+0000 7efd20200640 1 --2- 172.21.15.46:0/2920367776 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efcf8066690 0x7efcf8068b50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.444+0000 7efd20200640 1 --2- 172.21.15.46:0/2920367776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd1815d940 0x7efd180a7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.444+0000 7efd20200640 1 --2- 172.21.15.46:0/2920367776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efd18142ab0 0x7efd181588a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.444+0000 7efd20200640 1 --2- 172.21.15.46:0/2920367776 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efd1806d9e0 0x7efd181f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.444+0000 7efd20200640 1 -- 172.21.15.46:0/2920367776 >> 172.21.15.46:0/2920367776 conn(0x7efd18066080 msgr2=0x7efd180ae210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.444+0000 7efd20200640 1 -- 172.21.15.46:0/2920367776 shutdown_connections 2024-03-29T08:19:10.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.444+0000 7efd20200640 1 -- 172.21.15.46:0/2920367776 wait complete. 2024-03-29T08:19:10.467 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-29T08:19:10.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.604+0000 7f6e08e00640 1 Processor -- start 2024-03-29T08:19:10.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.608+0000 7f6e08e00640 1 -- start start 2024-03-29T08:19:10.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.608+0000 7f6e08e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6e04142ab0 0x7f6e0406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:10.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.608+0000 7f6e08e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6e0415db10 0x7f6e040b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:10.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.608+0000 7f6e08e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e0406de40 con 0x7f6e0406d9e0 2024-03-29T08:19:10.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.608+0000 7f6e08e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e04070c70 con 0x7f6e0415db10 2024-03-29T08:19:10.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.608+0000 7f6e08e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e04070340 con 0x7f6e04142ab0 2024-03-29T08:19:10.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.608+0000 7f6e02800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6e0415db10 0x7f6e040b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:10.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.608+0000 7f6e01e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6e04142ab0 0x7f6e0406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:10.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.608+0000 7f6e02800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6e0415db10 0x7f6e040b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:55014/0 (socket says 172.21.15.46:55014) 2024-03-29T08:19:10.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.608+0000 7f6e02800640 1 -- 172.21.15.46:0/371425904 learned_addr learned my addr 172.21.15.46:0/371425904 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:10.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.608+0000 7f6df9400640 1 --1- 172.21.15.46:0/371425904 >> v1:172.21.15.73:6789/0 conn(0x7f6e0406d9e0 0x7f6e0415d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:51620/0 (socket says 172.21.15.46:51620) 2024-03-29T08:19:10.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.608+0000 7f6e01e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6e04142ab0 0x7f6e0406d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:50998/0 (socket says 172.21.15.46:50998) 2024-03-29T08:19:10.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.608+0000 7f6e01e00640 1 -- 172.21.15.46:0/371425904 >> v2:172.21.15.73:3300/0 conn(0x7f6e0415db10 msgr2=0x7f6e040b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:10.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.608+0000 7f6e01e00640 1 --2- 172.21.15.46:0/371425904 >> v2:172.21.15.73:3300/0 conn(0x7f6e0415db10 0x7f6e040b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.608+0000 7f6e01e00640 1 -- 172.21.15.46:0/371425904 >> v1:172.21.15.73:6789/0 conn(0x7f6e0406d9e0 legacy=0x7f6e0415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:10.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.608+0000 7f6e01e00640 1 -- 172.21.15.46:0/371425904 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e04070620 con 0x7f6e04142ab0 2024-03-29T08:19:10.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.612+0000 7f6e01e00640 1 --2- 172.21.15.46:0/371425904 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6e04142ab0 0x7f6e0406d4a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0097c0 tx=0x7f6dec01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1583692176c8dfe7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:10.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.612+0000 7f6e01400640 1 -- 172.21.15.46:0/371425904 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec0a4e80 con 0x7f6e04142ab0 2024-03-29T08:19:10.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.612+0000 7f6e01400640 1 -- 172.21.15.46:0/371425904 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dec0ac440 con 0x7f6e04142ab0 2024-03-29T08:19:10.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.612+0000 7f6e01400640 1 -- 172.21.15.46:0/371425904 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dec0ac7b0 con 0x7f6e04142ab0 2024-03-29T08:19:10.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.612+0000 7f6e08e00640 1 -- 172.21.15.46:0/371425904 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6e04142ab0 msgr2=0x7f6e0406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:10.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.612+0000 7f6e08e00640 1 --2- 172.21.15.46:0/371425904 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6e04142ab0 0x7f6e0406d4a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0097c0 tx=0x7f6dec01c4a0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.612+0000 7f6e08e00640 1 -- 172.21.15.46:0/371425904 shutdown_connections 2024-03-29T08:19:10.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.612+0000 7f6e08e00640 1 --2- 172.21.15.46:0/371425904 >> v2:172.21.15.73:3300/0 conn(0x7f6e0415db10 0x7f6e040b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.612+0000 7f6e08e00640 1 --2- 172.21.15.46:0/371425904 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6e04142ab0 0x7f6e0406d4a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.612+0000 7f6e08e00640 1 -- 172.21.15.46:0/371425904 >> 172.21.15.46:0/371425904 conn(0x7f6e04066080 msgr2=0x7f6e04069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:10.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.612+0000 7f6e08e00640 1 -- 172.21.15.46:0/371425904 shutdown_connections 2024-03-29T08:19:10.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.612+0000 7f6e08e00640 1 -- 172.21.15.46:0/371425904 wait complete. 2024-03-29T08:19:10.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.612+0000 7f6e08e00640 1 Processor -- start 2024-03-29T08:19:10.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e08e00640 1 -- start start 2024-03-29T08:19:10.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e08e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6e0406d9e0 0x7f6e04130f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:10.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e08e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e04142ab0 0x7f6e04166e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:10.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e08e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e0415db10 0x7f6e04166180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:10.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e08e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040ab8b0 con 0x7f6e0415db10 2024-03-29T08:19:10.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e08e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e040abb90 con 0x7f6e04142ab0 2024-03-29T08:19:10.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e08e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0415c1c0 con 0x7f6e0406d9e0 2024-03-29T08:19:10.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e02800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e0415db10 0x7f6e04166180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:10.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e02800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e0415db10 0x7f6e04166180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56116/0 (socket says 172.21.15.46:56116) 2024-03-29T08:19:10.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e02800640 1 -- 172.21.15.46:0/2234740840 learned_addr learned my addr 172.21.15.46:0/2234740840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:10.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e01e00640 1 --2- 172.21.15.46:0/2234740840 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6e0406d9e0 0x7f6e04130f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:10.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6df9400640 1 --2- 172.21.15.46:0/2234740840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e04142ab0 0x7f6e04166e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:10.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e02800640 1 -- 172.21.15.46:0/2234740840 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6e0406d9e0 msgr2=0x7f6e04130f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:10.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e02800640 1 --2- 172.21.15.46:0/2234740840 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6e0406d9e0 0x7f6e04130f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e02800640 1 -- 172.21.15.46:0/2234740840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e04142ab0 msgr2=0x7f6e04166e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e02800640 1 --2- 172.21.15.46:0/2234740840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e04142ab0 0x7f6e04166e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e02800640 1 -- 172.21.15.46:0/2234740840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec009420 con 0x7f6e0415db10 2024-03-29T08:19:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6df9400640 1 --2- 172.21.15.46:0/2234740840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e04142ab0 0x7f6e04166e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e02800640 1 --2- 172.21.15.46:0/2234740840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e0415db10 0x7f6e04166180 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00efc0 tx=0x7f6dfc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6dfaa00640 1 -- 172.21.15.46:0/2234740840 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc009b60 con 0x7f6e0415db10 2024-03-29T08:19:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6dfaa00640 1 -- 172.21.15.46:0/2234740840 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc00f040 con 0x7f6e0415db10 2024-03-29T08:19:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e08e00640 1 -- 172.21.15.46:0/2234740840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0415c4a0 con 0x7f6e0415db10 2024-03-29T08:19:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6dfaa00640 1 -- 172.21.15.46:0/2234740840 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc0039a0 con 0x7f6e0415db10 2024-03-29T08:19:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e08e00640 1 -- 172.21.15.46:0/2234740840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e04131450 con 0x7f6e0415db10 2024-03-29T08:19:10.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.616+0000 7f6e08e00640 1 -- 172.21.15.46:0/2234740840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e040b05c0 con 0x7f6e0415db10 2024-03-29T08:19:10.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.620+0000 7f6dfaa00640 1 -- 172.21.15.46:0/2234740840 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6dfc003b00 con 0x7f6e0415db10 2024-03-29T08:19:10.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.620+0000 7f6dfaa00640 1 --2- 172.21.15.46:0/2234740840 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6de00666c0 0x7f6de0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:10.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.620+0000 7f6dfaa00640 1 -- 172.21.15.46:0/2234740840 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6dfc084370 con 0x7f6e0415db10 2024-03-29T08:19:10.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.620+0000 7f6e01e00640 1 --2- 172.21.15.46:0/2234740840 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6de00666c0 0x7f6de0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:10.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.620+0000 7f6e01e00640 1 --2- 172.21.15.46:0/2234740840 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6de00666c0 0x7f6de0068b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0097c0 tx=0x7f6dec0a6070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:10.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.624+0000 7f6dfaa00640 1 -- 172.21.15.46:0/2234740840 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6dfc051a20 con 0x7f6e0415db10 2024-03-29T08:19:10.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.840+0000 7f6e08e00640 1 -- 172.21.15.46:0/2234740840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f6e040abd00 con 0x7f6e0415db10 2024-03-29T08:19:10.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.844+0000 7f6dfaa00640 1 -- 172.21.15.46:0/2234740840 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6dfc055040 con 0x7f6e0415db10 2024-03-29T08:19:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.844+0000 7f6e08e00640 1 -- 172.21.15.46:0/2234740840 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6de00666c0 msgr2=0x7f6de0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.844+0000 7f6e08e00640 1 --2- 172.21.15.46:0/2234740840 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6de00666c0 0x7f6de0068b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0097c0 tx=0x7f6dec0a6070 comp rx=0 tx=0).stop 2024-03-29T08:19:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.844+0000 7f6e08e00640 1 -- 172.21.15.46:0/2234740840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e0415db10 msgr2=0x7f6e04166180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.844+0000 7f6e08e00640 1 --2- 172.21.15.46:0/2234740840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e0415db10 0x7f6e04166180 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00efc0 tx=0x7f6dfc00c490 comp rx=0 tx=0).stop 2024-03-29T08:19:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.844+0000 7f6e08e00640 1 -- 172.21.15.46:0/2234740840 shutdown_connections 2024-03-29T08:19:10.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.844+0000 7f6e08e00640 1 --2- 172.21.15.46:0/2234740840 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6de00666c0 0x7f6de0068b80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.844+0000 7f6e08e00640 1 --2- 172.21.15.46:0/2234740840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e04142ab0 0x7f6e04166e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.844+0000 7f6e08e00640 1 --2- 172.21.15.46:0/2234740840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6e0415db10 0x7f6e04166180 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.844+0000 7f6e08e00640 1 --2- 172.21.15.46:0/2234740840 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6e0406d9e0 0x7f6e04130f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:10.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.844+0000 7f6e08e00640 1 -- 172.21.15.46:0/2234740840 >> 172.21.15.46:0/2234740840 conn(0x7f6e04066080 msgr2=0x7f6e04069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:10.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.844+0000 7f6e08e00640 1 -- 172.21.15.46:0/2234740840 shutdown_connections 2024-03-29T08:19:10.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:10.844+0000 7f6e08e00640 1 -- 172.21.15.46:0/2234740840 wait complete. 2024-03-29T08:19:10.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-29T08:19:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.004+0000 7f93f0600640 1 Processor -- start 2024-03-29T08:19:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.004+0000 7f93f0600640 1 -- start start 2024-03-29T08:19:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.008+0000 7f93f0600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f93e8142ab0 0x7f93e806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.008+0000 7f93f0600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f93e815db10 0x7f93e80b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.008+0000 7f93f0600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e806de40 con 0x7f93e806d9e0 2024-03-29T08:19:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.008+0000 7f93f0600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e8070c70 con 0x7f93e8142ab0 2024-03-29T08:19:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.008+0000 7f93f0600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e8070340 con 0x7f93e815db10 2024-03-29T08:19:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.008+0000 7f93edc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f93e8142ab0 0x7f93e806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.008+0000 7f93edc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f93e8142ab0 0x7f93e806d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56124/0 (socket says 172.21.15.46:56124) 2024-03-29T08:19:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.008+0000 7f93ed200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f93e806d9e0 0x7f93e815d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58440/0 (socket says 172.21.15.46:58440) 2024-03-29T08:19:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.008+0000 7f93edc00640 1 -- 172.21.15.46:0/2176769651 learned_addr learned my addr 172.21.15.46:0/2176769651 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.008+0000 7f93ee600640 1 --2- 172.21.15.46:0/2176769651 >> v2:172.21.15.73:3300/0 conn(0x7f93e815db10 0x7f93e80b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.008+0000 7f93edc00640 1 -- 172.21.15.46:0/2176769651 >> v2:172.21.15.73:3300/0 conn(0x7f93e815db10 msgr2=0x7f93e80b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.008+0000 7f93edc00640 1 --2- 172.21.15.46:0/2176769651 >> v2:172.21.15.73:3300/0 conn(0x7f93e815db10 0x7f93e80b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.008+0000 7f93edc00640 1 -- 172.21.15.46:0/2176769651 >> v1:172.21.15.46:6789/0 conn(0x7f93e806d9e0 legacy=0x7f93e815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.008+0000 7f93edc00640 1 -- 172.21.15.46:0/2176769651 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e8070620 con 0x7f93e8142ab0 2024-03-29T08:19:11.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.008+0000 7f93edc00640 1 --2- 172.21.15.46:0/2176769651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e8142ab0 0x7f93e806d4a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f93e80b3ce0 tx=0x7f93dc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd5641b31ec246d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:11.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93e7e00640 1 -- 172.21.15.46:0/2176769651 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0ac440 con 0x7f93e8142ab0 2024-03-29T08:19:11.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93e7e00640 1 -- 172.21.15.46:0/2176769651 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93dc0ac5a0 con 0x7f93e8142ab0 2024-03-29T08:19:11.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93e7e00640 1 -- 172.21.15.46:0/2176769651 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0ac930 con 0x7f93e8142ab0 2024-03-29T08:19:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93f0600640 1 -- 172.21.15.46:0/2176769651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e8142ab0 msgr2=0x7f93e806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93f0600640 1 --2- 172.21.15.46:0/2176769651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e8142ab0 0x7f93e806d4a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f93e80b3ce0 tx=0x7f93dc01c520 comp rx=0 tx=0).stop 2024-03-29T08:19:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93f0600640 1 -- 172.21.15.46:0/2176769651 shutdown_connections 2024-03-29T08:19:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93f0600640 1 --2- 172.21.15.46:0/2176769651 >> v2:172.21.15.73:3300/0 conn(0x7f93e815db10 0x7f93e80b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93f0600640 1 --2- 172.21.15.46:0/2176769651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e8142ab0 0x7f93e806d4a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93f0600640 1 -- 172.21.15.46:0/2176769651 >> 172.21.15.46:0/2176769651 conn(0x7f93e8066080 msgr2=0x7f93e8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:11.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93f0600640 1 -- 172.21.15.46:0/2176769651 shutdown_connections 2024-03-29T08:19:11.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93f0600640 1 -- 172.21.15.46:0/2176769651 wait complete. 2024-03-29T08:19:11.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93f0600640 1 Processor -- start 2024-03-29T08:19:11.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93f0600640 1 -- start start 2024-03-29T08:19:11.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93f0600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93e806d9e0 0x7f93e8158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93f0600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f93e8142ab0 0x7f93e8161770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93f0600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e815db10 0x7f93e81632d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93f0600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e8162230 con 0x7f93e815db10 2024-03-29T08:19:11.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93f0600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e8162510 con 0x7f93e806d9e0 2024-03-29T08:19:11.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93f0600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e81627f0 con 0x7f93e8142ab0 2024-03-29T08:19:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93ee600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e815db10 0x7f93e81632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93ee600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e815db10 0x7f93e81632d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56130/0 (socket says 172.21.15.46:56130) 2024-03-29T08:19:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93ee600640 1 -- 172.21.15.46:0/4101158025 learned_addr learned my addr 172.21.15.46:0/4101158025 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93ed200640 1 --2- 172.21.15.46:0/4101158025 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f93e8142ab0 0x7f93e8161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93edc00640 1 --2- 172.21.15.46:0/4101158025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93e806d9e0 0x7f93e8158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93ee600640 1 -- 172.21.15.46:0/4101158025 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f93e8142ab0 msgr2=0x7f93e8161770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.012+0000 7f93ee600640 1 --2- 172.21.15.46:0/4101158025 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f93e8142ab0 0x7f93e8161770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.016+0000 7f93ee600640 1 -- 172.21.15.46:0/4101158025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93e806d9e0 msgr2=0x7f93e8158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.016+0000 7f93ee600640 1 --2- 172.21.15.46:0/4101158025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93e806d9e0 0x7f93e8158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.016+0000 7f93ee600640 1 -- 172.21.15.46:0/4101158025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93dc0094d0 con 0x7f93e815db10 2024-03-29T08:19:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.016+0000 7f93ee600640 1 --2- 172.21.15.46:0/4101158025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e815db10 0x7f93e81632d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f93e0007e40 tx=0x7f93e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.016+0000 7f93e5600640 1 -- 172.21.15.46:0/4101158025 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0014620 con 0x7f93e815db10 2024-03-29T08:19:11.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.016+0000 7f93e5600640 1 -- 172.21.15.46:0/4101158025 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e000ee90 con 0x7f93e815db10 2024-03-29T08:19:11.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.016+0000 7f93e5600640 1 -- 172.21.15.46:0/4101158025 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e00149e0 con 0x7f93e815db10 2024-03-29T08:19:11.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.016+0000 7f93f0600640 1 -- 172.21.15.46:0/4101158025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e8162ad0 con 0x7f93e815db10 2024-03-29T08:19:11.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.016+0000 7f93f0600640 1 -- 172.21.15.46:0/4101158025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e81593b0 con 0x7f93e815db10 2024-03-29T08:19:11.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.016+0000 7f93f0600640 1 -- 172.21.15.46:0/4101158025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e8162ff0 con 0x7f93e815db10 2024-03-29T08:19:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.016+0000 7f93e5600640 1 -- 172.21.15.46:0/4101158025 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f93e0014b40 con 0x7f93e815db10 2024-03-29T08:19:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.016+0000 7f93e5600640 1 --2- 172.21.15.46:0/4101158025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f93c8066690 0x7f93c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.016+0000 7f93e5600640 1 -- 172.21.15.46:0/4101158025 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f93e0084840 con 0x7f93e815db10 2024-03-29T08:19:11.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.016+0000 7f93edc00640 1 --2- 172.21.15.46:0/4101158025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f93c8066690 0x7f93c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.020+0000 7f93edc00640 1 --2- 172.21.15.46:0/4101158025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f93c8066690 0x7f93c8068b50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0ae000 tx=0x7f93dc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:11.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.020+0000 7f93e5600640 1 -- 172.21.15.46:0/4101158025 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f93e8162ff0 con 0x7f93e815db10 2024-03-29T08:19:11.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.240+0000 7f93f0600640 1 -- 172.21.15.46:0/4101158025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f93e8162ff0 con 0x7f93e815db10 2024-03-29T08:19:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.244+0000 7f93e5600640 1 -- 172.21.15.46:0/4101158025 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f93e8162ff0 con 0x7f93e815db10 2024-03-29T08:19:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.244+0000 7f93f0600640 1 -- 172.21.15.46:0/4101158025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f93c8066690 msgr2=0x7f93c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.244+0000 7f93f0600640 1 --2- 172.21.15.46:0/4101158025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f93c8066690 0x7f93c8068b50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f93dc0ae000 tx=0x7f93dc008040 comp rx=0 tx=0).stop 2024-03-29T08:19:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.244+0000 7f93f0600640 1 -- 172.21.15.46:0/4101158025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e815db10 msgr2=0x7f93e81632d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.244+0000 7f93f0600640 1 --2- 172.21.15.46:0/4101158025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e815db10 0x7f93e81632d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f93e0007e40 tx=0x7f93e000e4b0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.244+0000 7f93f0600640 1 -- 172.21.15.46:0/4101158025 shutdown_connections 2024-03-29T08:19:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.244+0000 7f93f0600640 1 --2- 172.21.15.46:0/4101158025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f93c8066690 0x7f93c8068b50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.244+0000 7f93f0600640 1 --2- 172.21.15.46:0/4101158025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93e815db10 0x7f93e81632d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.244+0000 7f93f0600640 1 --2- 172.21.15.46:0/4101158025 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f93e8142ab0 0x7f93e8161770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.244+0000 7f93f0600640 1 --2- 172.21.15.46:0/4101158025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93e806d9e0 0x7f93e8158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.244+0000 7f93f0600640 1 -- 172.21.15.46:0/4101158025 >> 172.21.15.46:0/4101158025 conn(0x7f93e8066080 msgr2=0x7f93e8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.244+0000 7f93f0600640 1 -- 172.21.15.46:0/4101158025 shutdown_connections 2024-03-29T08:19:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.244+0000 7f93f0600640 1 -- 172.21.15.46:0/4101158025 wait complete. 2024-03-29T08:19:11.265 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:19:11.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.404+0000 7f388a200640 1 Processor -- start 2024-03-29T08:19:11.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.404+0000 7f388a200640 1 -- start start 2024-03-29T08:19:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.404+0000 7f388a200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38841593d0 0x7f388406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.404+0000 7f388a200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f388415d230 0x7f388415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.404+0000 7f388a200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f388415dbd0 0x7f388415bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.404+0000 7f388a200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388406d130 con 0x7f388415dbd0 2024-03-29T08:19:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.404+0000 7f388a200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388415b8f0 con 0x7f388415d230 2024-03-29T08:19:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.404+0000 7f388a200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388415ad50 con 0x7f38841593d0 2024-03-29T08:19:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.408+0000 7f3883e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f388415dbd0 0x7f388415bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.408+0000 7f3883e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f388415dbd0 0x7f388415bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56144/0 (socket says 172.21.15.46:56144) 2024-03-29T08:19:11.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.408+0000 7f3883e00640 1 -- 172.21.15.46:0/1261925774 learned_addr learned my addr 172.21.15.46:0/1261925774 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:11.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.408+0000 7f3882a00640 1 --2- 172.21.15.46:0/1261925774 >> v2:172.21.15.73:3300/0 conn(0x7f388415d230 0x7f388415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.408+0000 7f3883400640 1 --2- 172.21.15.46:0/1261925774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38841593d0 0x7f388406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.408+0000 7f3883e00640 1 -- 172.21.15.46:0/1261925774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38841593d0 msgr2=0x7f388406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:11.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.408+0000 7f3883e00640 1 --2- 172.21.15.46:0/1261925774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38841593d0 0x7f388406cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.408+0000 7f3883e00640 1 -- 172.21.15.46:0/1261925774 >> v2:172.21.15.73:3300/0 conn(0x7f388415d230 msgr2=0x7f388415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:11.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.408+0000 7f3883e00640 1 --2- 172.21.15.46:0/1261925774 >> v2:172.21.15.73:3300/0 conn(0x7f388415d230 0x7f388415d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.408+0000 7f3883e00640 1 -- 172.21.15.46:0/1261925774 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388415e060 con 0x7f388415dbd0 2024-03-29T08:19:11.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.408+0000 7f3883e00640 1 --2- 172.21.15.46:0/1261925774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388415dbd0 0x7f388415bd80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3884071520 tx=0x7f387c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad530ee29aa2f3f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:11.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.408+0000 7f3882000640 1 -- 172.21.15.46:0/1261925774 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c0a5870 con 0x7f388415dbd0 2024-03-29T08:19:11.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.408+0000 7f3882000640 1 -- 172.21.15.46:0/1261925774 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387c0a59d0 con 0x7f388415dbd0 2024-03-29T08:19:11.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f3882000640 1 -- 172.21.15.46:0/1261925774 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c0a5d60 con 0x7f388415dbd0 2024-03-29T08:19:11.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f388a200640 1 -- 172.21.15.46:0/1261925774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388415dbd0 msgr2=0x7f388415bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f388a200640 1 --2- 172.21.15.46:0/1261925774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388415dbd0 0x7f388415bd80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3884071520 tx=0x7f387c01c9f0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f388a200640 1 -- 172.21.15.46:0/1261925774 shutdown_connections 2024-03-29T08:19:11.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f388a200640 1 --2- 172.21.15.46:0/1261925774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388415dbd0 0x7f388415bd80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f388a200640 1 --2- 172.21.15.46:0/1261925774 >> v2:172.21.15.73:3300/0 conn(0x7f388415d230 0x7f388415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f388a200640 1 --2- 172.21.15.46:0/1261925774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38841593d0 0x7f388406cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f388a200640 1 -- 172.21.15.46:0/1261925774 >> 172.21.15.46:0/1261925774 conn(0x7f3884066110 msgr2=0x7f3884069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:11.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f388a200640 1 -- 172.21.15.46:0/1261925774 shutdown_connections 2024-03-29T08:19:11.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f388a200640 1 -- 172.21.15.46:0/1261925774 wait complete. 2024-03-29T08:19:11.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f388a200640 1 Processor -- start 2024-03-29T08:19:11.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f388a200640 1 -- start start 2024-03-29T08:19:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f388a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38841593d0 0x7f38840ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f388a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388415d230 0x7f3884145240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f388a200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f388415dbd0 0x7f3884155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f388a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38840aec20 con 0x7f388415d230 2024-03-29T08:19:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f388a200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38840aef00 con 0x7f38841593d0 2024-03-29T08:19:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f388a200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38840af1e0 con 0x7f388415dbd0 2024-03-29T08:19:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f3882a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388415d230 0x7f3884145240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f3882a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388415d230 0x7f3884145240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56148/0 (socket says 172.21.15.46:56148) 2024-03-29T08:19:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f3882a00640 1 -- 172.21.15.46:0/3236728837 learned_addr learned my addr 172.21.15.46:0/3236728837 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f3883400640 1 --2- 172.21.15.46:0/3236728837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38841593d0 0x7f38840ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f3883e00640 1 --2- 172.21.15.46:0/3236728837 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f388415dbd0 0x7f3884155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f3882a00640 1 -- 172.21.15.46:0/3236728837 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f388415dbd0 msgr2=0x7f3884155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f3882a00640 1 --2- 172.21.15.46:0/3236728837 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f388415dbd0 0x7f3884155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f3882a00640 1 -- 172.21.15.46:0/3236728837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38841593d0 msgr2=0x7f38840ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f3882a00640 1 --2- 172.21.15.46:0/3236728837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38841593d0 0x7f38840ae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f3882a00640 1 -- 172.21.15.46:0/3236728837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c009b30 con 0x7f388415d230 2024-03-29T08:19:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f3883e00640 1 --2- 172.21.15.46:0/3236728837 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f388415dbd0 0x7f3884155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.412+0000 7f3883400640 1 --2- 172.21.15.46:0/3236728837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38841593d0 0x7f38840ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.416+0000 7f3882a00640 1 --2- 172.21.15.46:0/3236728837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388415d230 0x7f3884145240 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f387000d5f0 tx=0x7f387000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.416+0000 7f387b400640 1 -- 172.21.15.46:0/3236728837 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38700125f0 con 0x7f388415d230 2024-03-29T08:19:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.416+0000 7f387b400640 1 -- 172.21.15.46:0/3236728837 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3870012750 con 0x7f388415d230 2024-03-29T08:19:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.416+0000 7f387b400640 1 -- 172.21.15.46:0/3236728837 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3870012ae0 con 0x7f388415d230 2024-03-29T08:19:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.416+0000 7f388a200640 1 -- 172.21.15.46:0/3236728837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38840af4c0 con 0x7f388415d230 2024-03-29T08:19:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.416+0000 7f388a200640 1 -- 172.21.15.46:0/3236728837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3884145780 con 0x7f388415d230 2024-03-29T08:19:11.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.416+0000 7f388a200640 1 -- 172.21.15.46:0/3236728837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3884066c90 con 0x7f388415d230 2024-03-29T08:19:11.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.416+0000 7f387b400640 1 -- 172.21.15.46:0/3236728837 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3870012c40 con 0x7f388415d230 2024-03-29T08:19:11.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.416+0000 7f387b400640 1 --2- 172.21.15.46:0/3236728837 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3864066690 0x7f3864068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.416+0000 7f387b400640 1 -- 172.21.15.46:0/3236728837 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f38700844a0 con 0x7f388415d230 2024-03-29T08:19:11.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.416+0000 7f3883400640 1 --2- 172.21.15.46:0/3236728837 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3864066690 0x7f3864068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.420+0000 7f3883400640 1 --2- 172.21.15.46:0/3236728837 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3864066690 0x7f3864068b50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f387400ba50 tx=0x7f3874008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:11.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.420+0000 7f387b400640 1 -- 172.21.15.46:0/3236728837 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3870051a30 con 0x7f388415d230 2024-03-29T08:19:11.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.584+0000 7f388a200640 1 -- 172.21.15.46:0/3236728837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f388406d3d0 con 0x7f388415d230 2024-03-29T08:19:11.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.584+0000 7f387b400640 1 -- 172.21.15.46:0/3236728837 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+12341 (secure 0 0 0) 0x7f3870055050 con 0x7f388415d230 2024-03-29T08:19:11.585 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:11.585 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":32,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:19:08.380612+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:19:11.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.584+0000 7f388a200640 1 -- 172.21.15.46:0/3236728837 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3864066690 msgr2=0x7f3864068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.584+0000 7f388a200640 1 --2- 172.21.15.46:0/3236728837 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3864066690 0x7f3864068b50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f387400ba50 tx=0x7f3874008040 comp rx=0 tx=0).stop 2024-03-29T08:19:11.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.584+0000 7f388a200640 1 -- 172.21.15.46:0/3236728837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388415d230 msgr2=0x7f3884145240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.584+0000 7f388a200640 1 --2- 172.21.15.46:0/3236728837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388415d230 0x7f3884145240 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f387000d5f0 tx=0x7f387000dab0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.584+0000 7f388a200640 1 -- 172.21.15.46:0/3236728837 shutdown_connections 2024-03-29T08:19:11.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.584+0000 7f388a200640 1 --2- 172.21.15.46:0/3236728837 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3864066690 0x7f3864068b50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.584+0000 7f388a200640 1 --2- 172.21.15.46:0/3236728837 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f388415dbd0 0x7f3884155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.584+0000 7f388a200640 1 --2- 172.21.15.46:0/3236728837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f388415d230 0x7f3884145240 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.584+0000 7f388a200640 1 --2- 172.21.15.46:0/3236728837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38841593d0 0x7f38840ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.584+0000 7f388a200640 1 -- 172.21.15.46:0/3236728837 >> 172.21.15.46:0/3236728837 conn(0x7f3884066110 msgr2=0x7f38840cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:11.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.584+0000 7f388a200640 1 -- 172.21.15.46:0/3236728837 shutdown_connections 2024-03-29T08:19:11.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.584+0000 7f388a200640 1 -- 172.21.15.46:0/3236728837 wait complete. 2024-03-29T08:19:11.607 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:19:11.608 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:19:11.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.748+0000 7f99e1800640 1 Processor -- start 2024-03-29T08:19:11.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.748+0000 7f99e1800640 1 -- start start 2024-03-29T08:19:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99e1800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f99dc15c9c0 0x7f99dc0ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99e1800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99dc0cfe50 con 0x7f99dc099150 2024-03-29T08:19:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99e1800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99dc0cf240 con 0x7f99dc06be50 2024-03-29T08:19:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99e1800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99dc0cf030 con 0x7f99dc15c9c0 2024-03-29T08:19:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99db400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f99dc099150 0x7f99dc0cad80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58448/0 (socket says 172.21.15.46:58448) 2024-03-29T08:19:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99db400640 1 -- 172.21.15.46:0/1884435586 learned_addr learned my addr 172.21.15.46:0/1884435586 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:11.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99daa00640 1 --2- 172.21.15.46:0/1884435586 >> v2:172.21.15.73:3300/0 conn(0x7f99dc15c9c0 0x7f99dc0ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99da000640 1 -- 172.21.15.46:0/1884435586 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2840540750 0 0) 0x7f99dc0cfe50 con 0x7f99dc099150 2024-03-29T08:19:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99da000640 1 -- 172.21.15.46:0/1884435586 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99c0003530 con 0x7f99dc099150 2024-03-29T08:19:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99da000640 1 -- 172.21.15.46:0/1884435586 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 290828104 0 0) 0x7f99c0003530 con 0x7f99dc099150 2024-03-29T08:19:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99da000640 1 -- 172.21.15.46:0/1884435586 >> v2:172.21.15.73:3300/0 conn(0x7f99dc15c9c0 msgr2=0x7f99dc0ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99da000640 1 --2- 172.21.15.46:0/1884435586 >> v2:172.21.15.73:3300/0 conn(0x7f99dc15c9c0 0x7f99dc0ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99da000640 1 -- 172.21.15.46:0/1884435586 >> v1:172.21.15.73:6789/0 conn(0x7f99dc06be50 legacy=0x7f99dc098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99da000640 1 -- 172.21.15.46:0/1884435586 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99dc0cf520 con 0x7f99dc099150 2024-03-29T08:19:11.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99da000640 1 -- 172.21.15.46:0/1884435586 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f99d4001680 con 0x7f99dc099150 2024-03-29T08:19:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99daa00640 1 --2- 172.21.15.46:0/1884435586 >> v2:172.21.15.73:3300/0 conn(0x7f99dc15c9c0 0x7f99dc0ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99da000640 1 -- 172.21.15.46:0/1884435586 >> v1:172.21.15.46:6789/0 conn(0x7f99dc099150 legacy=0x7f99dc0cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99da000640 1 --2- 172.21.15.46:0/1884435586 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99c0003980 0x7f99c0003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99da000640 1 --2- 172.21.15.46:0/1884435586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99c0004380 0x7f99c0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99da000640 1 --2- 172.21.15.46:0/1884435586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c0005050 0x7f99c000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99da000640 1 -- 172.21.15.46:0/1884435586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0cf030 con 0x7f99c0005050 2024-03-29T08:19:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99da000640 1 -- 172.21.15.46:0/1884435586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c000bfb0 con 0x7f99c0004380 2024-03-29T08:19:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99da000640 1 -- 172.21.15.46:0/1884435586 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c000c170 con 0x7f99c0003980 2024-03-29T08:19:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99db400640 1 --2- 172.21.15.46:0/1884435586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c0005050 0x7f99c000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99daa00640 1 --2- 172.21.15.46:0/1884435586 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99c0003980 0x7f99c0003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99d2000640 1 --2- 172.21.15.46:0/1884435586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99c0004380 0x7f99c0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99db400640 1 -- 172.21.15.46:0/1884435586 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99c0003980 msgr2=0x7f99c0003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99db400640 1 --2- 172.21.15.46:0/1884435586 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99c0003980 0x7f99c0003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99db400640 1 -- 172.21.15.46:0/1884435586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99c0004380 msgr2=0x7f99c0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99db400640 1 --2- 172.21.15.46:0/1884435586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99c0004380 0x7f99c0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.752+0000 7f99db400640 1 -- 172.21.15.46:0/1884435586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c0003530 con 0x7f99c0005050 2024-03-29T08:19:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.756+0000 7f99db400640 1 -- 172.21.15.46:0/1884435586 reap_dead start 2024-03-29T08:19:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.756+0000 7f99db400640 1 --2- 172.21.15.46:0/1884435586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c0005050 0x7f99c000b990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f99d400c190 tx=0x7f99d401f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b33d26b61a939b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.756+0000 7f99da000640 1 -- 172.21.15.46:0/1884435586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d4005ae0 con 0x7f99c0005050 2024-03-29T08:19:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.756+0000 7f99da000640 1 -- 172.21.15.46:0/1884435586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d40ab070 con 0x7f99c0005050 2024-03-29T08:19:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.756+0000 7f99da000640 1 -- 172.21.15.46:0/1884435586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d4005ea0 con 0x7f99c0005050 2024-03-29T08:19:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.756+0000 7f99e1800640 1 -- 172.21.15.46:0/1884435586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c0005050 msgr2=0x7f99c000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.756+0000 7f99e1800640 1 --2- 172.21.15.46:0/1884435586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c0005050 0x7f99c000b990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f99d400c190 tx=0x7f99d401f330 comp rx=0 tx=0).stop 2024-03-29T08:19:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.756+0000 7f99e1800640 1 -- 172.21.15.46:0/1884435586 shutdown_connections 2024-03-29T08:19:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.756+0000 7f99e1800640 1 --2- 172.21.15.46:0/1884435586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c0005050 0x7f99c000b990 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.756+0000 7f99e1800640 1 -- 172.21.15.46:0/1884435586 >> 172.21.15.46:0/1884435586 conn(0x7f99dc0b2220 msgr2=0x7f99dc0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.756+0000 7f99e1800640 1 -- 172.21.15.46:0/1884435586 shutdown_connections 2024-03-29T08:19:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.756+0000 7f99e1800640 1 -- 172.21.15.46:0/1884435586 wait complete. 2024-03-29T08:19:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.756+0000 7f99e1800640 1 Processor -- start 2024-03-29T08:19:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.756+0000 7f99e1800640 1 -- start start 2024-03-29T08:19:11.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99e1800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c0005050 0x7f99dc1f2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99e1800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99dc1f5630 0x7f99dc09d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99e1800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99dc09df30 0x7f99dc06f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99e1800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0b0180 con 0x7f99c0005050 2024-03-29T08:19:11.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99e1800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0af570 con 0x7f99dc1f5630 2024-03-29T08:19:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99e1800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0af850 con 0x7f99dc09df30 2024-03-29T08:19:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c0005050 0x7f99dc1f2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99daa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c0005050 0x7f99dc1f2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56168/0 (socket says 172.21.15.46:56168) 2024-03-29T08:19:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99daa00640 1 -- 172.21.15.46:0/1725841787 learned_addr learned my addr 172.21.15.46:0/1725841787 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99d2000640 1 --2- 172.21.15.46:0/1725841787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99dc1f5630 0x7f99dc09d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99db400640 1 --2- 172.21.15.46:0/1725841787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99dc09df30 0x7f99dc06f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99daa00640 1 -- 172.21.15.46:0/1725841787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99dc09df30 msgr2=0x7f99dc06f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99daa00640 1 --2- 172.21.15.46:0/1725841787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99dc09df30 0x7f99dc06f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99daa00640 1 -- 172.21.15.46:0/1725841787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99dc1f5630 msgr2=0x7f99dc09d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99daa00640 1 --2- 172.21.15.46:0/1725841787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99dc1f5630 0x7f99dc09d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99daa00640 1 -- 172.21.15.46:0/1725841787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99dc0afb30 con 0x7f99c0005050 2024-03-29T08:19:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99d2000640 1 --2- 172.21.15.46:0/1725841787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99dc1f5630 0x7f99dc09d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99db400640 1 --2- 172.21.15.46:0/1725841787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99dc09df30 0x7f99dc06f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99daa00640 1 --2- 172.21.15.46:0/1725841787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c0005050 0x7f99dc1f2820 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00f430 tx=0x7f99cc00f8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99d3400640 1 -- 172.21.15.46:0/1725841787 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc00e040 con 0x7f99c0005050 2024-03-29T08:19:11.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99d3400640 1 -- 172.21.15.46:0/1725841787 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99cc00f040 con 0x7f99c0005050 2024-03-29T08:19:11.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99d3400640 1 -- 172.21.15.46:0/1725841787 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99cc003e00 con 0x7f99c0005050 2024-03-29T08:19:11.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99e1800640 1 -- 172.21.15.46:0/1725841787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99dc0afe10 con 0x7f99c0005050 2024-03-29T08:19:11.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99e1800640 1 -- 172.21.15.46:0/1725841787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99dc06fb30 con 0x7f99c0005050 2024-03-29T08:19:11.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99e1800640 1 -- 172.21.15.46:0/1725841787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99dc09e310 con 0x7f99c0005050 2024-03-29T08:19:11.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.760+0000 7f99d3400640 1 -- 172.21.15.46:0/1725841787 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f99cc004030 con 0x7f99c0005050 2024-03-29T08:19:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.764+0000 7f99d3400640 1 --2- 172.21.15.46:0/1725841787 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99b80666c0 0x7f99b8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.764+0000 7f99d3400640 1 -- 172.21.15.46:0/1725841787 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f99cc084b60 con 0x7f99c0005050 2024-03-29T08:19:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.764+0000 7f99d2000640 1 --2- 172.21.15.46:0/1725841787 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99b80666c0 0x7f99b8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.764+0000 7f99d2000640 1 --2- 172.21.15.46:0/1725841787 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99b80666c0 0x7f99b8068b80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0b17f0 tx=0x7f99c4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.768+0000 7f99d3400640 1 -- 172.21.15.46:0/1725841787 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f99cc052180 con 0x7f99c0005050 2024-03-29T08:19:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.928+0000 7f99e1800640 1 -- 172.21.15.46:0/1725841787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99dc09e700 con 0x7f99c0005050 2024-03-29T08:19:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.928+0000 7f99d3400640 1 -- 172.21.15.46:0/1725841787 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7f99cc022020 con 0x7f99c0005050 2024-03-29T08:19:11.929 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:19:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.928+0000 7f99e1800640 1 -- 172.21.15.46:0/1725841787 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99b80666c0 msgr2=0x7f99b8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.928+0000 7f99e1800640 1 --2- 172.21.15.46:0/1725841787 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99b80666c0 0x7f99b8068b80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0b17f0 tx=0x7f99c4009040 comp rx=0 tx=0).stop 2024-03-29T08:19:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.928+0000 7f99e1800640 1 -- 172.21.15.46:0/1725841787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c0005050 msgr2=0x7f99dc1f2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.928+0000 7f99e1800640 1 --2- 172.21.15.46:0/1725841787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c0005050 0x7f99dc1f2820 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f99cc00f430 tx=0x7f99cc00f8f0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.928+0000 7f99e1800640 1 -- 172.21.15.46:0/1725841787 shutdown_connections 2024-03-29T08:19:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.928+0000 7f99e1800640 1 --2- 172.21.15.46:0/1725841787 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99b80666c0 0x7f99b8068b80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.928+0000 7f99e1800640 1 --2- 172.21.15.46:0/1725841787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99dc1f5630 0x7f99dc09d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.928+0000 7f99e1800640 1 --2- 172.21.15.46:0/1725841787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99dc09df30 0x7f99dc06f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.928+0000 7f99e1800640 1 --2- 172.21.15.46:0/1725841787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99c0005050 0x7f99dc1f2820 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.928+0000 7f99e1800640 1 -- 172.21.15.46:0/1725841787 >> 172.21.15.46:0/1725841787 conn(0x7f99dc0b2220 msgr2=0x7f99dc0c6270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.928+0000 7f99e1800640 1 -- 172.21.15.46:0/1725841787 shutdown_connections 2024-03-29T08:19:11.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:11.928+0000 7f99e1800640 1 -- 172.21.15.46:0/1725841787 wait complete. 2024-03-29T08:19:11.951 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T08:19:12.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.088+0000 7f569b000640 1 Processor -- start 2024-03-29T08:19:12.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.088+0000 7f569b000640 1 -- start start 2024-03-29T08:19:12.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.088+0000 7f569b000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56940ae810 0x7f56940caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:12.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.088+0000 7f569b000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56940cbc10 con 0x7f56940cb780 2024-03-29T08:19:12.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.088+0000 7f569b000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56940c9bd0 con 0x7f56940ce8a0 2024-03-29T08:19:12.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.088+0000 7f569b000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56940c9eb0 con 0x7f56940ae810 2024-03-29T08:19:12.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.092+0000 7f568b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f56940ae810 0x7f56940caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:12.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.092+0000 7f5699000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f56940cb780 0x7f56940aecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58454/0 (socket says 172.21.15.46:58454) 2024-03-29T08:19:12.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.092+0000 7f5699000640 1 -- 172.21.15.46:0/3174923923 learned_addr learned my addr 172.21.15.46:0/3174923923 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:12.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.092+0000 7f568b400640 1 -- 172.21.15.46:0/3174923923 >> v1:172.21.15.73:6789/0 conn(0x7f56940ce8a0 legacy=0x7f56940cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:12.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.092+0000 7f568b400640 1 -- 172.21.15.46:0/3174923923 >> v1:172.21.15.46:6789/0 conn(0x7f56940cb780 legacy=0x7f56940aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:12.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.092+0000 7f568b400640 1 -- 172.21.15.46:0/3174923923 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56940ca190 con 0x7f56940ae810 2024-03-29T08:19:12.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.092+0000 7f5693400640 1 -- 172.21.15.46:0/3174923923 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850779792 0 0) 0x7f56940cbc10 con 0x7f56940cb780 2024-03-29T08:19:12.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.092+0000 7f568b400640 1 --2- 172.21.15.46:0/3174923923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56940ae810 0x7f56940caf50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f567c0097c0 tx=0x7f567c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90de5ecfab07eb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:12.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.092+0000 7f5693400640 1 -- 172.21.15.46:0/3174923923 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c0a7070 con 0x7f56940ae810 2024-03-29T08:19:12.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.092+0000 7f5693400640 1 -- 172.21.15.46:0/3174923923 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c0ab430 con 0x7f56940ae810 2024-03-29T08:19:12.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.092+0000 7f5693400640 1 -- 172.21.15.46:0/3174923923 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c0ab7c0 con 0x7f56940ae810 2024-03-29T08:19:12.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f569b000640 1 -- 172.21.15.46:0/3174923923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56940ae810 msgr2=0x7f56940caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:12.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f569b000640 1 --2- 172.21.15.46:0/3174923923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56940ae810 0x7f56940caf50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f567c0097c0 tx=0x7f567c01c430 comp rx=0 tx=0).stop 2024-03-29T08:19:12.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f569b000640 1 -- 172.21.15.46:0/3174923923 shutdown_connections 2024-03-29T08:19:12.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f569b000640 1 --2- 172.21.15.46:0/3174923923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56940ae810 0x7f56940caf50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:12.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f569b000640 1 -- 172.21.15.46:0/3174923923 >> 172.21.15.46:0/3174923923 conn(0x7f5694098a40 msgr2=0x7f56940cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:12.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f569b000640 1 -- 172.21.15.46:0/3174923923 shutdown_connections 2024-03-29T08:19:12.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f569b000640 1 -- 172.21.15.46:0/3174923923 wait complete. 2024-03-29T08:19:12.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f569b000640 1 Processor -- start 2024-03-29T08:19:12.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f569b000640 1 -- start start 2024-03-29T08:19:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f569b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56940ae810 0x7f5694155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f569b000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56940cb780 0x7f569415b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f569b000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56940ce8a0 0x7f569415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f569b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5694156260 con 0x7f56940ae810 2024-03-29T08:19:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f569b000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56941564c0 con 0x7f56940cb780 2024-03-29T08:19:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f569b000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5694157040 con 0x7f56940ce8a0 2024-03-29T08:19:12.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f5693e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56940ae810 0x7f5694155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f5693e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56940ae810 0x7f5694155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56180/0 (socket says 172.21.15.46:56180) 2024-03-29T08:19:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f5693e00640 1 -- 172.21.15.46:0/1012485552 learned_addr learned my addr 172.21.15.46:0/1012485552 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f5699000640 1 --2- 172.21.15.46:0/1012485552 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56940ce8a0 0x7f569415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f568b400640 1 --2- 172.21.15.46:0/1012485552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56940cb780 0x7f569415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f5693e00640 1 -- 172.21.15.46:0/1012485552 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56940ce8a0 msgr2=0x7f569415ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f5693e00640 1 --2- 172.21.15.46:0/1012485552 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56940ce8a0 0x7f569415ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f5693e00640 1 -- 172.21.15.46:0/1012485552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56940cb780 msgr2=0x7f569415b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f5693e00640 1 --2- 172.21.15.46:0/1012485552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56940cb780 0x7f569415b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f5693e00640 1 -- 172.21.15.46:0/1012485552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567c009420 con 0x7f56940ae810 2024-03-29T08:19:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f5699000640 1 --2- 172.21.15.46:0/1012485552 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56940ce8a0 0x7f569415ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f568b400640 1 --2- 172.21.15.46:0/1012485552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56940cb780 0x7f569415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.096+0000 7f5693e00640 1 --2- 172.21.15.46:0/1012485552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56940ae810 0x7f5694155d20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f568400b410 tx=0x7f568400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.100+0000 7f5690c00640 1 -- 172.21.15.46:0/1012485552 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684004020 con 0x7f56940ae810 2024-03-29T08:19:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.100+0000 7f5690c00640 1 -- 172.21.15.46:0/1012485552 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5684004180 con 0x7f56940ae810 2024-03-29T08:19:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.100+0000 7f5690c00640 1 -- 172.21.15.46:0/1012485552 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684011630 con 0x7f56940ae810 2024-03-29T08:19:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.100+0000 7f569b000640 1 -- 172.21.15.46:0/1012485552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5694156710 con 0x7f56940ae810 2024-03-29T08:19:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.100+0000 7f569b000640 1 -- 172.21.15.46:0/1012485552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5694156cd0 con 0x7f56940ae810 2024-03-29T08:19:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.100+0000 7f569b000640 1 -- 172.21.15.46:0/1012485552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5694066c90 con 0x7f56940ae810 2024-03-29T08:19:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.100+0000 7f5690c00640 1 -- 172.21.15.46:0/1012485552 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5684011790 con 0x7f56940ae810 2024-03-29T08:19:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.100+0000 7f5690c00640 1 --2- 172.21.15.46:0/1012485552 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5670066690 0x7f5670068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.100+0000 7f5690c00640 1 -- 172.21.15.46:0/1012485552 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5684083b20 con 0x7f56940ae810 2024-03-29T08:19:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.100+0000 7f568b400640 1 --2- 172.21.15.46:0/1012485552 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5670066690 0x7f5670068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.104+0000 7f568b400640 1 --2- 172.21.15.46:0/1012485552 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5670066690 0x7f5670068b50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f567c0af000 tx=0x7f567c0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.104+0000 7f5690c00640 1 -- 172.21.15.46:0/1012485552 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5684051270 con 0x7f56940ae810 2024-03-29T08:19:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.268+0000 7f569b000640 1 -- 172.21.15.46:0/1012485552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f56940cdb00 con 0x7f56940ae810 2024-03-29T08:19:12.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.268+0000 7f5690c00640 1 -- 172.21.15.46:0/1012485552 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5684015060 con 0x7f56940ae810 2024-03-29T08:19:12.270 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:12.270 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-03-29T08:19:12.270 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:19:12.270 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-03-29T08:19:12.270 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-03-29T08:19:12.270 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:12.270 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-03-29T08:19:12.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.268+0000 7f569b000640 1 -- 172.21.15.46:0/1012485552 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5670066690 msgr2=0x7f5670068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:12.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.268+0000 7f569b000640 1 --2- 172.21.15.46:0/1012485552 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5670066690 0x7f5670068b50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f567c0af000 tx=0x7f567c0a7000 comp rx=0 tx=0).stop 2024-03-29T08:19:12.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.268+0000 7f569b000640 1 -- 172.21.15.46:0/1012485552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56940ae810 msgr2=0x7f5694155d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:12.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.272+0000 7f569b000640 1 --2- 172.21.15.46:0/1012485552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56940ae810 0x7f5694155d20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f568400b410 tx=0x7f568400b8d0 comp rx=0 tx=0).stop 2024-03-29T08:19:12.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.272+0000 7f569b000640 1 -- 172.21.15.46:0/1012485552 shutdown_connections 2024-03-29T08:19:12.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.272+0000 7f569b000640 1 --2- 172.21.15.46:0/1012485552 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5670066690 0x7f5670068b50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:12.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.272+0000 7f569b000640 1 --2- 172.21.15.46:0/1012485552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56940cb780 0x7f569415b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:12.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.272+0000 7f569b000640 1 --2- 172.21.15.46:0/1012485552 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56940ce8a0 0x7f569415ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.272+0000 7f569b000640 1 --2- 172.21.15.46:0/1012485552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56940ae810 0x7f5694155d20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.272+0000 7f569b000640 1 -- 172.21.15.46:0/1012485552 >> 172.21.15.46:0/1012485552 conn(0x7f5694098a40 msgr2=0x7f56940cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.272+0000 7f569b000640 1 -- 172.21.15.46:0/1012485552 shutdown_connections 2024-03-29T08:19:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.272+0000 7f569b000640 1 -- 172.21.15.46:0/1012485552 wait complete. 2024-03-29T08:19:12.291 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T08:19:12.292 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T08:19:12.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c31e00640 1 Processor -- start 2024-03-29T08:19:12.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c31e00640 1 -- start start 2024-03-29T08:19:12.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c31e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c2c1598a0 0x7f7c2c0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:12.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c31e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2c0b1bc0 con 0x7f7c2c0b1470 2024-03-29T08:19:12.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c31e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2c0b1ea0 con 0x7f7c2c15dc30 2024-03-29T08:19:12.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c31e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c0b2180 con 0x7f7c2c1598a0 2024-03-29T08:19:12.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c22000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7c2c0b1470 0x7f7c2c15d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58466/0 (socket says 172.21.15.46:58466) 2024-03-29T08:19:12.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c22000640 1 -- 172.21.15.46:0/374157479 learned_addr learned my addr 172.21.15.46:0/374157479 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2aa00640 1 --2- 172.21.15.46:0/374157479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c2c1598a0 0x7f7c2c0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2a000640 1 -- 172.21.15.46:0/374157479 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732328043 0 0) 0x7f7c2c0b1bc0 con 0x7f7c2c0b1470 2024-03-29T08:19:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2a000640 1 -- 172.21.15.46:0/374157479 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c10005740 con 0x7f7c2c0b1470 2024-03-29T08:19:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2a000640 1 -- 172.21.15.46:0/374157479 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3741768319 0 0) 0x7f7c10005740 con 0x7f7c2c0b1470 2024-03-29T08:19:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2a000640 1 -- 172.21.15.46:0/374157479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c2c1598a0 msgr2=0x7f7c2c0cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2a000640 1 --2- 172.21.15.46:0/374157479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c2c1598a0 0x7f7c2c0cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2a000640 1 -- 172.21.15.46:0/374157479 >> v1:172.21.15.73:6789/0 conn(0x7f7c2c15dc30 legacy=0x7f7c2c06c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2a000640 1 -- 172.21.15.46:0/374157479 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c0b2460 con 0x7f7c2c0b1470 2024-03-29T08:19:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2a000640 1 -- 172.21.15.46:0/374157479 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f7c140016a0 con 0x7f7c2c0b1470 2024-03-29T08:19:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2a000640 1 -- 172.21.15.46:0/374157479 >> v1:172.21.15.46:6789/0 conn(0x7f7c2c0b1470 legacy=0x7f7c2c15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2a000640 1 --2- 172.21.15.46:0/374157479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c10005bc0 0x7f7c10005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2a000640 1 --2- 172.21.15.46:0/374157479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c10006570 0x7f7c10006a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2a000640 1 --2- 172.21.15.46:0/374157479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c2c1598a0 0x7f7c100072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:12.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2a000640 1 -- 172.21.15.46:0/374157479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c0b2180 con 0x7f7c10005bc0 2024-03-29T08:19:12.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2a000640 1 -- 172.21.15.46:0/374157479 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1000dba0 con 0x7f7c10006570 2024-03-29T08:19:12.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2a000640 1 -- 172.21.15.46:0/374157479 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1000dd60 con 0x7f7c2c1598a0 2024-03-29T08:19:12.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2aa00640 1 --2- 172.21.15.46:0/374157479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c10005bc0 0x7f7c10005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:12.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2b400640 1 --2- 172.21.15.46:0/374157479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c2c1598a0 0x7f7c100072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:12.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c22000640 1 --2- 172.21.15.46:0/374157479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c10006570 0x7f7c10006a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:12.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2aa00640 1 -- 172.21.15.46:0/374157479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c2c1598a0 msgr2=0x7f7c100072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:12.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2aa00640 1 --2- 172.21.15.46:0/374157479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c2c1598a0 0x7f7c100072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:12.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2aa00640 1 -- 172.21.15.46:0/374157479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c10006570 msgr2=0x7f7c10006a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:12.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2aa00640 1 --2- 172.21.15.46:0/374157479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c10006570 0x7f7c10006a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:12.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.436+0000 7f7c2aa00640 1 -- 172.21.15.46:0/374157479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c10005740 con 0x7f7c10005bc0 2024-03-29T08:19:12.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.440+0000 7f7c2aa00640 1 --2- 172.21.15.46:0/374157479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c10005bc0 0x7f7c10005fa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009ae0 tx=0x7f7c1c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a787a20ad67428 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.440+0000 7f7c2a000640 1 -- 172.21.15.46:0/374157479 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c0a5660 con 0x7f7c10005bc0 2024-03-29T08:19:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.440+0000 7f7c2a000640 1 -- 172.21.15.46:0/374157479 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1c0a57c0 con 0x7f7c10005bc0 2024-03-29T08:19:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.440+0000 7f7c2a000640 1 -- 172.21.15.46:0/374157479 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c0a5b70 con 0x7f7c10005bc0 2024-03-29T08:19:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.440+0000 7f7c31e00640 1 -- 172.21.15.46:0/374157479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c10005bc0 msgr2=0x7f7c10005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.440+0000 7f7c31e00640 1 --2- 172.21.15.46:0/374157479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c10005bc0 0x7f7c10005fa0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009ae0 tx=0x7f7c1c01ca70 comp rx=0 tx=0).stop 2024-03-29T08:19:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.440+0000 7f7c2b400640 1 -- 172.21.15.46:0/374157479 reap_dead start 2024-03-29T08:19:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.440+0000 7f7c31e00640 1 -- 172.21.15.46:0/374157479 shutdown_connections 2024-03-29T08:19:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.440+0000 7f7c31e00640 1 -- 172.21.15.46:0/374157479 >> 172.21.15.46:0/374157479 conn(0x7f7c2c0b42d0 msgr2=0x7f7c2c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.440+0000 7f7c31e00640 1 -- 172.21.15.46:0/374157479 shutdown_connections 2024-03-29T08:19:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.440+0000 7f7c31e00640 1 -- 172.21.15.46:0/374157479 wait complete. 2024-03-29T08:19:12.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c31e00640 1 Processor -- start 2024-03-29T08:19:12.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c31e00640 1 -- start start 2024-03-29T08:19:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c31e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c10005bc0 0x7f7c2c1f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c31e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c2c0ac2c0 0x7f7c2c0ac740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c31e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c2c0b0150 0x7f7c2c0b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c31e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c0b0b10 con 0x7f7c2c0b0150 2024-03-29T08:19:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c31e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c0c33e0 con 0x7f7c10005bc0 2024-03-29T08:19:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c31e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2c1f1f10 con 0x7f7c2c0ac2c0 2024-03-29T08:19:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c2c0b0150 0x7f7c2c0b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c2b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c2c0b0150 0x7f7c2c0b05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56212/0 (socket says 172.21.15.46:56212) 2024-03-29T08:19:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c2b400640 1 -- 172.21.15.46:0/4209638619 learned_addr learned my addr 172.21.15.46:0/4209638619 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c22000640 1 --2- 172.21.15.46:0/4209638619 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c2c0ac2c0 0x7f7c2c0ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:12.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c2b400640 1 -- 172.21.15.46:0/4209638619 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c2c0ac2c0 msgr2=0x7f7c2c0ac740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c2b400640 1 --2- 172.21.15.46:0/4209638619 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c2c0ac2c0 0x7f7c2c0ac740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c2aa00640 1 --2- 172.21.15.46:0/4209638619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c10005bc0 0x7f7c2c1f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c2b400640 1 -- 172.21.15.46:0/4209638619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c10005bc0 msgr2=0x7f7c2c1f19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c2b400640 1 --2- 172.21.15.46:0/4209638619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c10005bc0 0x7f7c2c1f19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c2b400640 1 -- 172.21.15.46:0/4209638619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c2c0acd80 con 0x7f7c2c0b0150 2024-03-29T08:19:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c22000640 1 --2- 172.21.15.46:0/4209638619 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c2c0ac2c0 0x7f7c2c0ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c2aa00640 1 --2- 172.21.15.46:0/4209638619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c10005bc0 0x7f7c2c1f19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c2b400640 1 --2- 172.21.15.46:0/4209638619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c2c0b0150 0x7f7c2c0b05d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0b3370 tx=0x7f7c24010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c23e00640 1 -- 172.21.15.46:0/4209638619 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c24003d90 con 0x7f7c2c0b0150 2024-03-29T08:19:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c23e00640 1 -- 172.21.15.46:0/4209638619 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2400d070 con 0x7f7c2c0b0150 2024-03-29T08:19:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c23e00640 1 -- 172.21.15.46:0/4209638619 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2400eaa0 con 0x7f7c2c0b0150 2024-03-29T08:19:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c31e00640 1 -- 172.21.15.46:0/4209638619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c2c0ad060 con 0x7f7c2c0b0150 2024-03-29T08:19:12.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.444+0000 7f7c31e00640 1 -- 172.21.15.46:0/4209638619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2c0c3630 con 0x7f7c2c0b0150 2024-03-29T08:19:12.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.448+0000 7f7c31e00640 1 -- 172.21.15.46:0/4209638619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c2c066c90 con 0x7f7c2c0b0150 2024-03-29T08:19:12.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.448+0000 7f7c23e00640 1 -- 172.21.15.46:0/4209638619 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7c2400ec00 con 0x7f7c2c0b0150 2024-03-29T08:19:12.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.448+0000 7f7c23e00640 1 --2- 172.21.15.46:0/4209638619 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c08066690 0x7f7c08068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:12.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.448+0000 7f7c23e00640 1 -- 172.21.15.46:0/4209638619 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7c24086f80 con 0x7f7c2c0b0150 2024-03-29T08:19:12.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.448+0000 7f7c2aa00640 1 --2- 172.21.15.46:0/4209638619 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c08066690 0x7f7c08068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:12.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.448+0000 7f7c2aa00640 1 --2- 172.21.15.46:0/4209638619 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c08066690 0x7f7c08068b50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009ae0 tx=0x7f7c1c0046d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:12.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.452+0000 7f7c23e00640 1 -- 172.21.15.46:0/4209638619 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7c24054630 con 0x7f7c2c0b0150 2024-03-29T08:19:12.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:12.624+0000 7f7c31e00640 1 -- 172.21.15.46:0/4209638619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7c2c0cec20 con 0x7f7c2c0b0150 2024-03-29T08:19:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.252+0000 7f7c23e00640 1 -- 172.21.15.46:0/4209638619 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7c24057c50 con 0x7f7c2c0b0150 2024-03-29T08:19:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-03-29T08:19:13.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.256+0000 7f7c31e00640 1 -- 172.21.15.46:0/4209638619 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c08066690 msgr2=0x7f7c08068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:13.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.256+0000 7f7c31e00640 1 --2- 172.21.15.46:0/4209638619 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c08066690 0x7f7c08068b50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c009ae0 tx=0x7f7c1c0046d0 comp rx=0 tx=0).stop 2024-03-29T08:19:13.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.256+0000 7f7c31e00640 1 -- 172.21.15.46:0/4209638619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c2c0b0150 msgr2=0x7f7c2c0b05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:13.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.256+0000 7f7c31e00640 1 --2- 172.21.15.46:0/4209638619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c2c0b0150 0x7f7c2c0b05d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0b3370 tx=0x7f7c24010f20 comp rx=0 tx=0).stop 2024-03-29T08:19:13.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.256+0000 7f7c31e00640 1 -- 172.21.15.46:0/4209638619 shutdown_connections 2024-03-29T08:19:13.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.256+0000 7f7c31e00640 1 --2- 172.21.15.46:0/4209638619 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c08066690 0x7f7c08068b50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:13.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.256+0000 7f7c31e00640 1 --2- 172.21.15.46:0/4209638619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c2c0b0150 0x7f7c2c0b05d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.256+0000 7f7c31e00640 1 --2- 172.21.15.46:0/4209638619 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c2c0ac2c0 0x7f7c2c0ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.256+0000 7f7c31e00640 1 --2- 172.21.15.46:0/4209638619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c10005bc0 0x7f7c2c1f19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.256+0000 7f7c31e00640 1 -- 172.21.15.46:0/4209638619 >> 172.21.15.46:0/4209638619 conn(0x7f7c2c0b42d0 msgr2=0x7f7c2c158d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.256+0000 7f7c31e00640 1 -- 172.21.15.46:0/4209638619 shutdown_connections 2024-03-29T08:19:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.260+0000 7f7c31e00640 1 -- 172.21.15.46:0/4209638619 wait complete. 2024-03-29T08:19:13.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T08:19:13.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.432+0000 7f55bbc00640 1 Processor -- start 2024-03-29T08:19:13.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.432+0000 7f55bbc00640 1 -- start start 2024-03-29T08:19:13.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.436+0000 7f55bbc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f55b406dbb0 0x7f55b415d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:13.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.436+0000 7f55bbc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f55b415d940 0x7f55b40b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:13.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.436+0000 7f55bbc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b4070c70 con 0x7f55b4142ab0 2024-03-29T08:19:13.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.436+0000 7f55bbc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b4070340 con 0x7f55b406dbb0 2024-03-29T08:19:13.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.436+0000 7f55bbc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b4070620 con 0x7f55b415d940 2024-03-29T08:19:13.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.436+0000 7f55b9c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f55b415d940 0x7f55b40b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:13.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.436+0000 7f55abe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f55b406dbb0 0x7f55b415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:13.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.436+0000 7f55b9c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f55b415d940 0x7f55b40b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:51086/0 (socket says 172.21.15.46:51086) 2024-03-29T08:19:13.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.436+0000 7f55b9c00640 1 -- 172.21.15.46:0/1215232199 learned_addr learned my addr 172.21.15.46:0/1215232199 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:13.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.436+0000 7f55abe00640 1 -- 172.21.15.46:0/1215232199 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f55b415d940 msgr2=0x7f55b40b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:13.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.436+0000 7f55b3e00640 1 -- 172.21.15.46:0/1215232199 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276771560 0 0) 0x7f55b4070c70 con 0x7f55b4142ab0 2024-03-29T08:19:13.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.436+0000 7f55abe00640 1 --2- 172.21.15.46:0/1215232199 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f55b415d940 0x7f55b40b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:13.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.436+0000 7f55abe00640 1 -- 172.21.15.46:0/1215232199 >> v1:172.21.15.73:6789/0 conn(0x7f55b4142ab0 legacy=0x7f55b406d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:13.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.436+0000 7f55abe00640 1 -- 172.21.15.46:0/1215232199 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b4070900 con 0x7f55b406dbb0 2024-03-29T08:19:13.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.436+0000 7f55b9c00640 1 --2- 172.21.15.46:0/1215232199 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f55b415d940 0x7f55b40b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:13.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.440+0000 7f55abe00640 1 --2- 172.21.15.46:0/1215232199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b406dbb0 0x7f55b415d400 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f559c0097c0 tx=0x7f559c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=345aaeba09a068c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:13.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.440+0000 7f55b3e00640 1 -- 172.21.15.46:0/1215232199 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c0ab600 con 0x7f55b406dbb0 2024-03-29T08:19:13.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.440+0000 7f55b3e00640 1 -- 172.21.15.46:0/1215232199 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559c0a7050 con 0x7f55b406dbb0 2024-03-29T08:19:13.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.440+0000 7f55b3e00640 1 -- 172.21.15.46:0/1215232199 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559c0ab980 con 0x7f55b406dbb0 2024-03-29T08:19:13.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.440+0000 7f55bbc00640 1 -- 172.21.15.46:0/1215232199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b406dbb0 msgr2=0x7f55b415d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:13.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.440+0000 7f55bbc00640 1 --2- 172.21.15.46:0/1215232199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b406dbb0 0x7f55b415d400 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f559c0097c0 tx=0x7f559c01c4a0 comp rx=0 tx=0).stop 2024-03-29T08:19:13.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.440+0000 7f55bbc00640 1 -- 172.21.15.46:0/1215232199 shutdown_connections 2024-03-29T08:19:13.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.440+0000 7f55bbc00640 1 --2- 172.21.15.46:0/1215232199 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f55b415d940 0x7f55b40b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:13.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.440+0000 7f55bbc00640 1 --2- 172.21.15.46:0/1215232199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b406dbb0 0x7f55b415d400 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:13.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.440+0000 7f55bbc00640 1 -- 172.21.15.46:0/1215232199 >> 172.21.15.46:0/1215232199 conn(0x7f55b4066080 msgr2=0x7f55b4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:13.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.440+0000 7f55bbc00640 1 -- 172.21.15.46:0/1215232199 shutdown_connections 2024-03-29T08:19:13.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.440+0000 7f55bbc00640 1 -- 172.21.15.46:0/1215232199 wait complete. 2024-03-29T08:19:13.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.440+0000 7f55bbc00640 1 Processor -- start 2024-03-29T08:19:13.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55bbc00640 1 -- start start 2024-03-29T08:19:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55bbc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55b406dbb0 0x7f55b41f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55bbc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b4142ab0 0x7f55b40c3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55bbc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f55b415d940 0x7f55b4163a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55bbc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b40a9910 con 0x7f55b406dbb0 2024-03-29T08:19:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55bbc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b41f9550 con 0x7f55b4142ab0 2024-03-29T08:19:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55bbc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b40c3ca0 con 0x7f55b415d940 2024-03-29T08:19:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55b9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55b406dbb0 0x7f55b41f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55b9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55b406dbb0 0x7f55b41f85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56216/0 (socket says 172.21.15.46:56216) 2024-03-29T08:19:13.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55b9200640 1 -- 172.21.15.46:0/317555071 learned_addr learned my addr 172.21.15.46:0/317555071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55b9c00640 1 --2- 172.21.15.46:0/317555071 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f55b415d940 0x7f55b4163a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55abe00640 1 --2- 172.21.15.46:0/317555071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b4142ab0 0x7f55b40c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55b9200640 1 -- 172.21.15.46:0/317555071 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f55b415d940 msgr2=0x7f55b4163a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55b9200640 1 --2- 172.21.15.46:0/317555071 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f55b415d940 0x7f55b4163a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55b9200640 1 -- 172.21.15.46:0/317555071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b4142ab0 msgr2=0x7f55b40c3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55b9200640 1 --2- 172.21.15.46:0/317555071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b4142ab0 0x7f55b40c3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55b9200640 1 -- 172.21.15.46:0/317555071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ac009b30 con 0x7f55b406dbb0 2024-03-29T08:19:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55b9200640 1 --2- 172.21.15.46:0/317555071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55b406dbb0 0x7f55b41f85b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f55a400c500 tx=0x7f55a400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55b1600640 1 -- 172.21.15.46:0/317555071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a4003600 con 0x7f55b406dbb0 2024-03-29T08:19:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55b1600640 1 -- 172.21.15.46:0/317555071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a4003760 con 0x7f55b406dbb0 2024-03-29T08:19:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55b1600640 1 -- 172.21.15.46:0/317555071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a4003b10 con 0x7f55b406dbb0 2024-03-29T08:19:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55bbc00640 1 -- 172.21.15.46:0/317555071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559c009420 con 0x7f55b406dbb0 2024-03-29T08:19:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55bbc00640 1 -- 172.21.15.46:0/317555071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b41f8f90 con 0x7f55b406dbb0 2024-03-29T08:19:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.444+0000 7f55bbc00640 1 -- 172.21.15.46:0/317555071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b40a9670 con 0x7f55b406dbb0 2024-03-29T08:19:13.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.448+0000 7f55b1600640 1 -- 172.21.15.46:0/317555071 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f55a4003c70 con 0x7f55b406dbb0 2024-03-29T08:19:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.448+0000 7f55b1600640 1 --2- 172.21.15.46:0/317555071 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5590066690 0x7f5590068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.448+0000 7f55b1600640 1 -- 172.21.15.46:0/317555071 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f55a4083770 con 0x7f55b406dbb0 2024-03-29T08:19:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.448+0000 7f55abe00640 1 --2- 172.21.15.46:0/317555071 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5590066690 0x7f5590068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:13.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.448+0000 7f55abe00640 1 --2- 172.21.15.46:0/317555071 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5590066690 0x7f5590068b50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f559c0ad000 tx=0x7f559c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:13.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.452+0000 7f55b1600640 1 -- 172.21.15.46:0/317555071 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f55a4050fa0 con 0x7f55b406dbb0 2024-03-29T08:19:13.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:13.612+0000 7f55bbc00640 1 -- 172.21.15.46:0/317555071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f55b40a9670 con 0x7f55b406dbb0 2024-03-29T08:19:14.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.260+0000 7f55b1600640 1 -- 172.21.15.46:0/317555071 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) v1 ==== 170+0+0 (secure 0 0 0) 0x7f55b40a9670 con 0x7f55b406dbb0 2024-03-29T08:19:14.260 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-03-29T08:19:14.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.264+0000 7f55bbc00640 1 -- 172.21.15.46:0/317555071 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5590066690 msgr2=0x7f5590068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:14.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.264+0000 7f55bbc00640 1 --2- 172.21.15.46:0/317555071 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5590066690 0x7f5590068b50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f559c0ad000 tx=0x7f559c008040 comp rx=0 tx=0).stop 2024-03-29T08:19:14.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.264+0000 7f55bbc00640 1 -- 172.21.15.46:0/317555071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55b406dbb0 msgr2=0x7f55b41f85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:14.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.264+0000 7f55bbc00640 1 --2- 172.21.15.46:0/317555071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55b406dbb0 0x7f55b41f85b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f55a400c500 tx=0x7f55a400c9c0 comp rx=0 tx=0).stop 2024-03-29T08:19:14.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.268+0000 7f55bbc00640 1 -- 172.21.15.46:0/317555071 shutdown_connections 2024-03-29T08:19:14.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.268+0000 7f55bbc00640 1 --2- 172.21.15.46:0/317555071 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5590066690 0x7f5590068b50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:14.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.268+0000 7f55bbc00640 1 --2- 172.21.15.46:0/317555071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b4142ab0 0x7f55b40c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:14.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.268+0000 7f55bbc00640 1 --2- 172.21.15.46:0/317555071 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f55b415d940 0x7f55b4163a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:14.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.268+0000 7f55bbc00640 1 --2- 172.21.15.46:0/317555071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55b406dbb0 0x7f55b41f85b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:14.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.268+0000 7f55bbc00640 1 -- 172.21.15.46:0/317555071 >> 172.21.15.46:0/317555071 conn(0x7f55b4066080 msgr2=0x7f55b40ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:14.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.268+0000 7f55bbc00640 1 -- 172.21.15.46:0/317555071 shutdown_connections 2024-03-29T08:19:14.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.268+0000 7f55bbc00640 1 -- 172.21.15.46:0/317555071 wait complete. 2024-03-29T08:19:14.301 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T08:19:14.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.444+0000 7ffb9c000640 1 Processor -- start 2024-03-29T08:19:14.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.444+0000 7ffb9c000640 1 -- start start 2024-03-29T08:19:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.448+0000 7ffb9c000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffb9415dbb0 0x7ffb9406ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.448+0000 7ffb9c000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb94157490 con 0x7ffb940b3f20 2024-03-29T08:19:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.448+0000 7ffb9c000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb940b2c90 con 0x7ffb940b2440 2024-03-29T08:19:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.448+0000 7ffb9c000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb940b29b0 con 0x7ffb9415dbb0 2024-03-29T08:19:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.448+0000 7ffb99600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffb9415dbb0 0x7ffb9406ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.448+0000 7ffb98c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffb940b3f20 0x7ffb940b1d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58480/0 (socket says 172.21.15.46:58480) 2024-03-29T08:19:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.448+0000 7ffb98c00640 1 -- 172.21.15.46:0/515757008 learned_addr learned my addr 172.21.15.46:0/515757008 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:14.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.448+0000 7ffb99600640 1 -- 172.21.15.46:0/515757008 >> v1:172.21.15.73:6789/0 conn(0x7ffb940b2440 legacy=0x7ffb940ac450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:14.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.448+0000 7ffb99600640 1 -- 172.21.15.46:0/515757008 >> v1:172.21.15.46:6789/0 conn(0x7ffb940b3f20 legacy=0x7ffb940b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:14.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.448+0000 7ffb99600640 1 -- 172.21.15.46:0/515757008 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb940b3440 con 0x7ffb9415dbb0 2024-03-29T08:19:14.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.448+0000 7ffb93e00640 1 -- 172.21.15.46:0/515757008 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1812041209 0 0) 0x7ffb94157490 con 0x7ffb940b3f20 2024-03-29T08:19:14.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.448+0000 7ffb99600640 1 --2- 172.21.15.46:0/515757008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9415dbb0 0x7ffb9406ada0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ffb880097c0 tx=0x7ffb8801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5987f0007147d76 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:14.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb93e00640 1 -- 172.21.15.46:0/515757008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb880ab600 con 0x7ffb9415dbb0 2024-03-29T08:19:14.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb93e00640 1 -- 172.21.15.46:0/515757008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb880a7050 con 0x7ffb9415dbb0 2024-03-29T08:19:14.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb93e00640 1 -- 172.21.15.46:0/515757008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb880ab980 con 0x7ffb9415dbb0 2024-03-29T08:19:14.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb9c000640 1 -- 172.21.15.46:0/515757008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9415dbb0 msgr2=0x7ffb9406ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:14.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb9c000640 1 --2- 172.21.15.46:0/515757008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9415dbb0 0x7ffb9406ada0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ffb880097c0 tx=0x7ffb8801c410 comp rx=0 tx=0).stop 2024-03-29T08:19:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb9c000640 1 -- 172.21.15.46:0/515757008 shutdown_connections 2024-03-29T08:19:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb9c000640 1 --2- 172.21.15.46:0/515757008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb9415dbb0 0x7ffb9406ada0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb9c000640 1 -- 172.21.15.46:0/515757008 >> 172.21.15.46:0/515757008 conn(0x7ffb94070ad0 msgr2=0x7ffb94070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb9c000640 1 -- 172.21.15.46:0/515757008 shutdown_connections 2024-03-29T08:19:14.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb9c000640 1 -- 172.21.15.46:0/515757008 wait complete. 2024-03-29T08:19:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb9c000640 1 Processor -- start 2024-03-29T08:19:14.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb9c000640 1 -- start start 2024-03-29T08:19:14.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb9c000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffb940b2440 0x7ffb94156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:14.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb9c000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb940b3f20 0x7ffb940cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:14.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb9c000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb9415dbb0 0x7ffb9406d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:14.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb9c000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb940cd760 con 0x7ffb940b3f20 2024-03-29T08:19:14.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb9c000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb940cc740 con 0x7ffb9415dbb0 2024-03-29T08:19:14.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb9c000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb940cd130 con 0x7ffb940b2440 2024-03-29T08:19:14.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb98c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb940b3f20 0x7ffb940cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:14.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb98c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb940b3f20 0x7ffb940cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56242/0 (socket says 172.21.15.46:56242) 2024-03-29T08:19:14.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb98c00640 1 -- 172.21.15.46:0/2012987540 learned_addr learned my addr 172.21.15.46:0/2012987540 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:14.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb9a000640 1 --2- 172.21.15.46:0/2012987540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb9415dbb0 0x7ffb9406d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:14.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.452+0000 7ffb99600640 1 --2- 172.21.15.46:0/2012987540 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffb940b2440 0x7ffb94156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:14.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.456+0000 7ffb98c00640 1 -- 172.21.15.46:0/2012987540 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffb940b2440 msgr2=0x7ffb94156b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:14.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.456+0000 7ffb98c00640 1 --2- 172.21.15.46:0/2012987540 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffb940b2440 0x7ffb94156b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:14.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.456+0000 7ffb98c00640 1 -- 172.21.15.46:0/2012987540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb9415dbb0 msgr2=0x7ffb9406d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:14.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.456+0000 7ffb98c00640 1 --2- 172.21.15.46:0/2012987540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb9415dbb0 0x7ffb9406d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:14.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.456+0000 7ffb98c00640 1 -- 172.21.15.46:0/2012987540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb88009420 con 0x7ffb940b3f20 2024-03-29T08:19:14.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.456+0000 7ffb98c00640 1 --2- 172.21.15.46:0/2012987540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb940b3f20 0x7ffb940cc200 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000c4a0 tx=0x7ffb8000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:14.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.456+0000 7ffb91600640 1 -- 172.21.15.46:0/2012987540 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb800074e0 con 0x7ffb940b3f20 2024-03-29T08:19:14.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.456+0000 7ffb91600640 1 -- 172.21.15.46:0/2012987540 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb80007640 con 0x7ffb940b3f20 2024-03-29T08:19:14.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.456+0000 7ffb91600640 1 -- 172.21.15.46:0/2012987540 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb80004a30 con 0x7ffb940b3f20 2024-03-29T08:19:14.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.456+0000 7ffb9c000640 1 -- 172.21.15.46:0/2012987540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb940cc9a0 con 0x7ffb940b3f20 2024-03-29T08:19:14.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.456+0000 7ffb9c000640 1 -- 172.21.15.46:0/2012987540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9406d9c0 con 0x7ffb940b3f20 2024-03-29T08:19:14.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.456+0000 7ffb9c000640 1 -- 172.21.15.46:0/2012987540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb940cd910 con 0x7ffb940b3f20 2024-03-29T08:19:14.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.456+0000 7ffb91600640 1 -- 172.21.15.46:0/2012987540 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ffb80004b90 con 0x7ffb940b3f20 2024-03-29T08:19:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.456+0000 7ffb91600640 1 --2- 172.21.15.46:0/2012987540 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffb74066690 0x7ffb74068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.456+0000 7ffb91600640 1 -- 172.21.15.46:0/2012987540 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5609+0+0 (secure 0 0 0) 0x7ffb800832e0 con 0x7ffb940b3f20 2024-03-29T08:19:14.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.456+0000 7ffb99600640 1 --2- 172.21.15.46:0/2012987540 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffb74066690 0x7ffb74068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:14.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.460+0000 7ffb99600640 1 --2- 172.21.15.46:0/2012987540 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffb74066690 0x7ffb74068b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffb880ad000 tx=0x7ffb880a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.464+0000 7ffb91600640 1 -- 172.21.15.46:0/2012987540 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ffb940cd910 con 0x7ffb940b3f20 2024-03-29T08:19:14.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.660+0000 7ffb9c000640 1 -- 172.21.15.46:0/2012987540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ffb9409d100 con 0x7ffb940b3f20 2024-03-29T08:19:14.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.664+0000 7ffb91600640 1 -- 172.21.15.46:0/2012987540 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7ffb800180c0 con 0x7ffb940b3f20 2024-03-29T08:19:14.666 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 4 and data pool 5 2024-03-29T08:19:14.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.668+0000 7ffb9c000640 1 -- 172.21.15.46:0/2012987540 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffb74066690 msgr2=0x7ffb74068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:14.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.668+0000 7ffb9c000640 1 --2- 172.21.15.46:0/2012987540 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffb74066690 0x7ffb74068b50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ffb880ad000 tx=0x7ffb880a6000 comp rx=0 tx=0).stop 2024-03-29T08:19:14.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.668+0000 7ffb9c000640 1 -- 172.21.15.46:0/2012987540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb940b3f20 msgr2=0x7ffb940cc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:14.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.668+0000 7ffb9c000640 1 --2- 172.21.15.46:0/2012987540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb940b3f20 0x7ffb940cc200 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000c4a0 tx=0x7ffb8000c960 comp rx=0 tx=0).stop 2024-03-29T08:19:14.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.668+0000 7ffb9c000640 1 -- 172.21.15.46:0/2012987540 shutdown_connections 2024-03-29T08:19:14.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.668+0000 7ffb9c000640 1 --2- 172.21.15.46:0/2012987540 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffb74066690 0x7ffb74068b50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:14.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.668+0000 7ffb9c000640 1 --2- 172.21.15.46:0/2012987540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb9415dbb0 0x7ffb9406d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:14.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.668+0000 7ffb9c000640 1 --2- 172.21.15.46:0/2012987540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffb940b3f20 0x7ffb940cc200 secure :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000c4a0 tx=0x7ffb8000c960 comp rx=0 tx=0).stop 2024-03-29T08:19:14.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.668+0000 7ffb9c000640 1 --2- 172.21.15.46:0/2012987540 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffb940b2440 0x7ffb94156b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:14.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.668+0000 7ffb9c000640 1 -- 172.21.15.46:0/2012987540 >> 172.21.15.46:0/2012987540 conn(0x7ffb94070ad0 msgr2=0x7ffb9406e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:14.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.672+0000 7ffb9c000640 1 -- 172.21.15.46:0/2012987540 shutdown_connections 2024-03-29T08:19:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.672+0000 7ffb9c000640 1 -- 172.21.15.46:0/2012987540 wait complete. 2024-03-29T08:19:14.709 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:19:14.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.848+0000 7ff118400640 1 Processor -- start 2024-03-29T08:19:14.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.848+0000 7ff118400640 1 -- start start 2024-03-29T08:19:14.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.848+0000 7ff118400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff1101605f0 0x7ff110069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:14.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.848+0000 7ff118400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff110161020 con 0x7ff110160a80 2024-03-29T08:19:14.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.848+0000 7ff118400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff110161300 con 0x7ff11015feb0 2024-03-29T08:19:14.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.848+0000 7ff118400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff110070b70 con 0x7ff1101605f0 2024-03-29T08:19:14.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.848+0000 7ff116400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff110160a80 0x7ff11006d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58484/0 (socket says 172.21.15.46:58484) 2024-03-29T08:19:14.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.848+0000 7ff116400640 1 -- 172.21.15.46:0/583682555 learned_addr learned my addr 172.21.15.46:0/583682555 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:14.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.848+0000 7ff115000640 1 --2- 172.21.15.46:0/583682555 >> v2:172.21.15.73:3300/0 conn(0x7ff1101605f0 0x7ff110069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:14.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.848+0000 7ff10fe00640 1 -- 172.21.15.46:0/583682555 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265374420 0 0) 0x7ff110161020 con 0x7ff110160a80 2024-03-29T08:19:14.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.848+0000 7ff10fe00640 1 -- 172.21.15.46:0/583682555 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0f8003530 con 0x7ff110160a80 2024-03-29T08:19:14.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.848+0000 7ff10fe00640 1 -- 172.21.15.46:0/583682555 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175672613 0 0) 0x7ff110161300 con 0x7ff11015feb0 2024-03-29T08:19:14.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.852+0000 7ff10fe00640 1 -- 172.21.15.46:0/583682555 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff110161020 con 0x7ff11015feb0 2024-03-29T08:19:14.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.852+0000 7ff115000640 1 -- 172.21.15.46:0/583682555 >> v1:172.21.15.73:6789/0 conn(0x7ff11015feb0 legacy=0x7ff110071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:14.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.852+0000 7ff115000640 1 -- 172.21.15.46:0/583682555 >> v1:172.21.15.46:6789/0 conn(0x7ff110160a80 legacy=0x7ff11006d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:14.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.852+0000 7ff115000640 1 -- 172.21.15.46:0/583682555 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff110070e50 con 0x7ff1101605f0 2024-03-29T08:19:14.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.852+0000 7ff115000640 1 --2- 172.21.15.46:0/583682555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1101605f0 0x7ff110069d90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0097f0 tx=0x7ff0fc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32bd1a166f03c559 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:14.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.852+0000 7ff10fe00640 1 -- 172.21.15.46:0/583682555 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc0ac410 con 0x7ff1101605f0 2024-03-29T08:19:14.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.852+0000 7ff10fe00640 1 -- 172.21.15.46:0/583682555 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0fc0ac570 con 0x7ff1101605f0 2024-03-29T08:19:14.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.852+0000 7ff10fe00640 1 -- 172.21.15.46:0/583682555 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc0ac8e0 con 0x7ff1101605f0 2024-03-29T08:19:14.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.852+0000 7ff118400640 1 -- 172.21.15.46:0/583682555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1101605f0 msgr2=0x7ff110069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:14.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.852+0000 7ff118400640 1 --2- 172.21.15.46:0/583682555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1101605f0 0x7ff110069d90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0097f0 tx=0x7ff0fc01c4a0 comp rx=0 tx=0).stop 2024-03-29T08:19:14.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.852+0000 7ff118400640 1 -- 172.21.15.46:0/583682555 shutdown_connections 2024-03-29T08:19:14.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.852+0000 7ff118400640 1 --2- 172.21.15.46:0/583682555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1101605f0 0x7ff110069d90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:14.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.852+0000 7ff118400640 1 -- 172.21.15.46:0/583682555 >> 172.21.15.46:0/583682555 conn(0x7ff1100ced50 msgr2=0x7ff1100cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:14.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.852+0000 7ff118400640 1 -- 172.21.15.46:0/583682555 shutdown_connections 2024-03-29T08:19:14.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.852+0000 7ff118400640 1 -- 172.21.15.46:0/583682555 wait complete. 2024-03-29T08:19:14.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff118400640 1 Processor -- start 2024-03-29T08:19:14.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff118400640 1 -- start start 2024-03-29T08:19:14.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff118400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff11015feb0 0x7ff110156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:14.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff118400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1101605f0 0x7ff11015cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:14.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff118400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff110160a80 0x7ff110151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:14.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff118400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff110155e00 con 0x7ff11015feb0 2024-03-29T08:19:14.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff118400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1101551f0 con 0x7ff1101605f0 2024-03-29T08:19:14.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff118400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1101554d0 con 0x7ff110160a80 2024-03-29T08:19:14.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff115a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff11015feb0 0x7ff110156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:14.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff115a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff11015feb0 0x7ff110156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56250/0 (socket says 172.21.15.46:56250) 2024-03-29T08:19:14.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff115a00640 1 -- 172.21.15.46:0/3214274655 learned_addr learned my addr 172.21.15.46:0/3214274655 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:14.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff115000640 1 --2- 172.21.15.46:0/3214274655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1101605f0 0x7ff11015cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:14.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff116400640 1 --2- 172.21.15.46:0/3214274655 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff110160a80 0x7ff110151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:14.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff115a00640 1 -- 172.21.15.46:0/3214274655 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff110160a80 msgr2=0x7ff110151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:14.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff115a00640 1 --2- 172.21.15.46:0/3214274655 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff110160a80 0x7ff110151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:14.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff115a00640 1 -- 172.21.15.46:0/3214274655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1101605f0 msgr2=0x7ff11015cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:14.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff115a00640 1 --2- 172.21.15.46:0/3214274655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1101605f0 0x7ff11015cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:14.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff115a00640 1 -- 172.21.15.46:0/3214274655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc009420 con 0x7ff11015feb0 2024-03-29T08:19:14.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff116400640 1 --2- 172.21.15.46:0/3214274655 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff110160a80 0x7ff110151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:14.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff115a00640 1 --2- 172.21.15.46:0/3214274655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff11015feb0 0x7ff110156290 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff10400c500 tx=0x7ff10400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:14.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff10d600640 1 -- 172.21.15.46:0/3214274655 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1040052d0 con 0x7ff11015feb0 2024-03-29T08:19:14.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff10d600640 1 -- 172.21.15.46:0/3214274655 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff104005450 con 0x7ff11015feb0 2024-03-29T08:19:14.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff118400640 1 -- 172.21.15.46:0/3214274655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1101557b0 con 0x7ff11015feb0 2024-03-29T08:19:14.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff10d600640 1 -- 172.21.15.46:0/3214274655 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff104012870 con 0x7ff11015feb0 2024-03-29T08:19:14.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.856+0000 7ff118400640 1 -- 172.21.15.46:0/3214274655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11015d010 con 0x7ff11015feb0 2024-03-29T08:19:14.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.860+0000 7ff118400640 1 -- 172.21.15.46:0/3214274655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff110066c90 con 0x7ff11015feb0 2024-03-29T08:19:14.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.860+0000 7ff10d600640 1 -- 172.21.15.46:0/3214274655 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff1040129d0 con 0x7ff11015feb0 2024-03-29T08:19:14.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.860+0000 7ff10d600640 1 --2- 172.21.15.46:0/3214274655 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff0f00666c0 0x7ff0f0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:14.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.860+0000 7ff10d600640 1 -- 172.21.15.46:0/3214274655 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff104083e60 con 0x7ff11015feb0 2024-03-29T08:19:14.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.860+0000 7ff115000640 1 --2- 172.21.15.46:0/3214274655 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff0f00666c0 0x7ff0f0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:14.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.860+0000 7ff115000640 1 --2- 172.21.15.46:0/3214274655 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff0f00666c0 0x7ff0f0068b80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0ae000 tx=0x7ff0fc0a6040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:14.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:14.864+0000 7ff10d600640 1 -- 172.21.15.46:0/3214274655 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff104051140 con 0x7ff11015feb0 2024-03-29T08:19:15.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.028+0000 7ff118400640 1 -- 172.21.15.46:0/3214274655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1100cb540 con 0x7ff11015feb0 2024-03-29T08:19:15.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.032+0000 7ff10d600640 1 -- 172.21.15.46:0/3214274655 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16006 (secure 0 0 0) 0x7ff104054760 con 0x7ff11015feb0 2024-03-29T08:19:15.032 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:15.032 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":35,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:19:14.662826+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:19:12.674055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-29T08:19:13.661287+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:19:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.032+0000 7ff118400640 1 -- 172.21.15.46:0/3214274655 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff0f00666c0 msgr2=0x7ff0f0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.032+0000 7ff118400640 1 --2- 172.21.15.46:0/3214274655 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff0f00666c0 0x7ff0f0068b80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0ae000 tx=0x7ff0fc0a6040 comp rx=0 tx=0).stop 2024-03-29T08:19:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.032+0000 7ff118400640 1 -- 172.21.15.46:0/3214274655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff11015feb0 msgr2=0x7ff110156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.032+0000 7ff118400640 1 --2- 172.21.15.46:0/3214274655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff11015feb0 0x7ff110156290 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff10400c500 tx=0x7ff10400c9c0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.032+0000 7ff118400640 1 -- 172.21.15.46:0/3214274655 shutdown_connections 2024-03-29T08:19:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.032+0000 7ff118400640 1 --2- 172.21.15.46:0/3214274655 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff0f00666c0 0x7ff0f0068b80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.032+0000 7ff118400640 1 --2- 172.21.15.46:0/3214274655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1101605f0 0x7ff11015cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.032+0000 7ff118400640 1 --2- 172.21.15.46:0/3214274655 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff110160a80 0x7ff110151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.032+0000 7ff118400640 1 --2- 172.21.15.46:0/3214274655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff11015feb0 0x7ff110156290 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.032+0000 7ff118400640 1 -- 172.21.15.46:0/3214274655 >> 172.21.15.46:0/3214274655 conn(0x7ff1100ced50 msgr2=0x7ff1100ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.032+0000 7ff118400640 1 -- 172.21.15.46:0/3214274655 shutdown_connections 2024-03-29T08:19:15.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.032+0000 7ff118400640 1 -- 172.21.15.46:0/3214274655 wait complete. 2024-03-29T08:19:15.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:19:15.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.192+0000 7fce12000640 1 Processor -- start 2024-03-29T08:19:15.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.192+0000 7fce12000640 1 -- start start 2024-03-29T08:19:15.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.192+0000 7fce12000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce0c15a790 0x7fce0c0cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:15.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.192+0000 7fce12000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fce0c099150 0x7fce0c0b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:15.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.192+0000 7fce12000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce0c0ce660 con 0x7fce0c15ab70 2024-03-29T08:19:15.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.192+0000 7fce12000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce0c0cf700 con 0x7fce0c099150 2024-03-29T08:19:15.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.192+0000 7fce12000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c0ce940 con 0x7fce0c15a790 2024-03-29T08:19:15.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.192+0000 7fce0be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fce0c099150 0x7fce0c0b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:15.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.192+0000 7fce0be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fce0c099150 0x7fce0c0b5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56258/0 (socket says 172.21.15.46:56258) 2024-03-29T08:19:15.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.192+0000 7fce0be00640 1 -- 172.21.15.46:0/795211125 learned_addr learned my addr 172.21.15.46:0/795211125 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:15.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.192+0000 7fce0b400640 1 --2- 172.21.15.46:0/795211125 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce0c15a790 0x7fce0c0cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:15.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.192+0000 7fce0be00640 1 -- 172.21.15.46:0/795211125 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce0c15a790 msgr2=0x7fce0c0cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:15.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.192+0000 7fce0be00640 1 --2- 172.21.15.46:0/795211125 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce0c15a790 0x7fce0c0cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.192+0000 7fce0be00640 1 -- 172.21.15.46:0/795211125 >> v1:172.21.15.73:6789/0 conn(0x7fce0c15ab70 legacy=0x7fce0c098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:15.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.192+0000 7fce0be00640 1 -- 172.21.15.46:0/795211125 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce0c0cec20 con 0x7fce0c099150 2024-03-29T08:19:15.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.196+0000 7fce0be00640 1 --2- 172.21.15.46:0/795211125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0c099150 0x7fce0c0b5860 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fce04009e50 tx=0x7fce0401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d725fac8a05284a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:15.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.196+0000 7fce0aa00640 1 -- 172.21.15.46:0/795211125 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce040a58d0 con 0x7fce0c099150 2024-03-29T08:19:15.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.196+0000 7fce0aa00640 1 -- 172.21.15.46:0/795211125 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce040a5a30 con 0x7fce0c099150 2024-03-29T08:19:15.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.196+0000 7fce0aa00640 1 -- 172.21.15.46:0/795211125 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce040a5dc0 con 0x7fce0c099150 2024-03-29T08:19:15.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.196+0000 7fce12000640 1 -- 172.21.15.46:0/795211125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0c099150 msgr2=0x7fce0c0b5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:15.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.196+0000 7fce12000640 1 --2- 172.21.15.46:0/795211125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0c099150 0x7fce0c0b5860 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fce04009e50 tx=0x7fce0401ca00 comp rx=0 tx=0).stop 2024-03-29T08:19:15.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.196+0000 7fce12000640 1 -- 172.21.15.46:0/795211125 shutdown_connections 2024-03-29T08:19:15.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.196+0000 7fce12000640 1 --2- 172.21.15.46:0/795211125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0c099150 0x7fce0c0b5860 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.196+0000 7fce12000640 1 --2- 172.21.15.46:0/795211125 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce0c15a790 0x7fce0c0cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.196+0000 7fce12000640 1 -- 172.21.15.46:0/795211125 >> 172.21.15.46:0/795211125 conn(0x7fce0c0b21a0 msgr2=0x7fce0c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:15.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.196+0000 7fce12000640 1 -- 172.21.15.46:0/795211125 shutdown_connections 2024-03-29T08:19:15.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.196+0000 7fce12000640 1 -- 172.21.15.46:0/795211125 wait complete. 2024-03-29T08:19:15.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.196+0000 7fce12000640 1 Processor -- start 2024-03-29T08:19:15.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce12000640 1 -- start start 2024-03-29T08:19:15.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce12000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0c099150 0x7fce0c15b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:15.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce12000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce0c15a790 0x7fce0c159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:15.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce12000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce0c15ab70 0x7fce0c14ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:15.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce12000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c158f00 con 0x7fce0c099150 2024-03-29T08:19:15.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce12000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c158bd0 con 0x7fce0c15ab70 2024-03-29T08:19:15.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce12000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c150e70 con 0x7fce0c15a790 2024-03-29T08:19:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce0b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0c099150 0x7fce0c15b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce0b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0c099150 0x7fce0c15b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56262/0 (socket says 172.21.15.46:56262) 2024-03-29T08:19:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce0b400640 1 -- 172.21.15.46:0/1589621800 learned_addr learned my addr 172.21.15.46:0/1589621800 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce03e00640 1 --2- 172.21.15.46:0/1589621800 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce0c15a790 0x7fce0c159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce0be00640 1 --2- 172.21.15.46:0/1589621800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce0c15ab70 0x7fce0c14ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce0b400640 1 -- 172.21.15.46:0/1589621800 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce0c15a790 msgr2=0x7fce0c159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce0b400640 1 --2- 172.21.15.46:0/1589621800 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce0c15a790 0x7fce0c159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce0b400640 1 -- 172.21.15.46:0/1589621800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce0c15ab70 msgr2=0x7fce0c14ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce0b400640 1 --2- 172.21.15.46:0/1589621800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce0c15ab70 0x7fce0c14ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce0b400640 1 -- 172.21.15.46:0/1589621800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce04009b30 con 0x7fce0c099150 2024-03-29T08:19:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce0be00640 1 --2- 172.21.15.46:0/1589621800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce0c15ab70 0x7fce0c14ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce0b400640 1 --2- 172.21.15.46:0/1589621800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0c099150 0x7fce0c15b330 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00d800 tx=0x7fcdfc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:15.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce03400640 1 -- 172.21.15.46:0/1589621800 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc00b6c0 con 0x7fce0c099150 2024-03-29T08:19:15.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce03400640 1 -- 172.21.15.46:0/1589621800 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc00b820 con 0x7fce0c099150 2024-03-29T08:19:15.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce03400640 1 -- 172.21.15.46:0/1589621800 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc00bb90 con 0x7fce0c099150 2024-03-29T08:19:15.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce12000640 1 -- 172.21.15.46:0/1589621800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce0c15b980 con 0x7fce0c099150 2024-03-29T08:19:15.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce12000640 1 -- 172.21.15.46:0/1589621800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0c15bf40 con 0x7fce0c099150 2024-03-29T08:19:15.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.200+0000 7fce12000640 1 -- 172.21.15.46:0/1589621800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0c066c90 con 0x7fce0c099150 2024-03-29T08:19:15.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.204+0000 7fce03400640 1 -- 172.21.15.46:0/1589621800 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcdfc00bcf0 con 0x7fce0c099150 2024-03-29T08:19:15.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.204+0000 7fce03400640 1 --2- 172.21.15.46:0/1589621800 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcde80666c0 0x7fcde8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:15.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.204+0000 7fce03400640 1 -- 172.21.15.46:0/1589621800 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdfc084590 con 0x7fce0c099150 2024-03-29T08:19:15.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.204+0000 7fce03e00640 1 --2- 172.21.15.46:0/1589621800 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcde80666c0 0x7fcde8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:15.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.204+0000 7fce03e00640 1 --2- 172.21.15.46:0/1589621800 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcde80666c0 0x7fcde8068b80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce0c0c3b20 tx=0x7fcdf4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:15.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.208+0000 7fce03400640 1 -- 172.21.15.46:0/1589621800 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcdfc051990 con 0x7fce0c099150 2024-03-29T08:19:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.372+0000 7fce12000640 1 -- 172.21.15.46:0/1589621800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce0c0b5590 con 0x7fce0c099150 2024-03-29T08:19:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.372+0000 7fce03400640 1 -- 172.21.15.46:0/1589621800 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16006 (secure 0 0 0) 0x7fcdfc054fb0 con 0x7fce0c099150 2024-03-29T08:19:15.375 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:15.375 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":35,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:19:14.662826+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:19:12.674055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-29T08:19:13.661287+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:19:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.376+0000 7fce12000640 1 -- 172.21.15.46:0/1589621800 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcde80666c0 msgr2=0x7fcde8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.376+0000 7fce12000640 1 --2- 172.21.15.46:0/1589621800 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcde80666c0 0x7fcde8068b80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fce0c0c3b20 tx=0x7fcdf4009040 comp rx=0 tx=0).stop 2024-03-29T08:19:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.376+0000 7fce12000640 1 -- 172.21.15.46:0/1589621800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0c099150 msgr2=0x7fce0c15b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.376+0000 7fce12000640 1 --2- 172.21.15.46:0/1589621800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0c099150 0x7fce0c15b330 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00d800 tx=0x7fcdfc00dcc0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.376+0000 7fce12000640 1 -- 172.21.15.46:0/1589621800 shutdown_connections 2024-03-29T08:19:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.376+0000 7fce12000640 1 --2- 172.21.15.46:0/1589621800 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcde80666c0 0x7fcde8068b80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.376+0000 7fce12000640 1 --2- 172.21.15.46:0/1589621800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce0c15ab70 0x7fce0c14ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.376+0000 7fce12000640 1 --2- 172.21.15.46:0/1589621800 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce0c15a790 0x7fce0c159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.376+0000 7fce12000640 1 --2- 172.21.15.46:0/1589621800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce0c099150 0x7fce0c15b330 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.376+0000 7fce12000640 1 -- 172.21.15.46:0/1589621800 >> 172.21.15.46:0/1589621800 conn(0x7fce0c0b21a0 msgr2=0x7fce0c06e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.376+0000 7fce12000640 1 -- 172.21.15.46:0/1589621800 shutdown_connections 2024-03-29T08:19:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.376+0000 7fce12000640 1 -- 172.21.15.46:0/1589621800 wait complete. 2024-03-29T08:19:15.396 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T08:19:15.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.536+0000 7f0ff9a00640 1 Processor -- start 2024-03-29T08:19:15.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.536+0000 7f0ff9a00640 1 -- start start 2024-03-29T08:19:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.540+0000 7f0ff9a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0ff415a790 0x7f0ff40cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.540+0000 7f0ff9a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ff40cbbe0 0x7f0ff40d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.540+0000 7f0ff9a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0ff40cf340 0x7f0ff40d0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.540+0000 7f0ff9a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff40cc660 con 0x7f0ff415a790 2024-03-29T08:19:15.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.540+0000 7f0ff9a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff40cc0a0 con 0x7f0ff40cf340 2024-03-29T08:19:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.540+0000 7f0ff9a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40cc380 con 0x7f0ff40cbbe0 2024-03-29T08:19:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.540+0000 7f0ff2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0ff415a790 0x7f0ff40cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.540+0000 7f0ff2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0ff415a790 0x7f0ff40cee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56278/0 (socket says 172.21.15.46:56278) 2024-03-29T08:19:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.540+0000 7f0ff2a00640 1 -- 172.21.15.46:0/16848212 learned_addr learned my addr 172.21.15.46:0/16848212 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.540+0000 7f0fea000640 1 --2- 172.21.15.46:0/16848212 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ff40cbbe0 0x7f0ff40d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.540+0000 7f0ff3400640 1 --2- 172.21.15.46:0/16848212 >> v2:172.21.15.73:3300/0 conn(0x7f0ff40cf340 0x7f0ff40d0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.540+0000 7f0ff2a00640 1 -- 172.21.15.46:0/16848212 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ff40cbbe0 msgr2=0x7f0ff40d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.540+0000 7f0ff2a00640 1 --2- 172.21.15.46:0/16848212 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ff40cbbe0 0x7f0ff40d0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.540+0000 7f0ff2a00640 1 -- 172.21.15.46:0/16848212 >> v2:172.21.15.73:3300/0 conn(0x7f0ff40cf340 msgr2=0x7f0ff40d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.540+0000 7f0ff2a00640 1 --2- 172.21.15.46:0/16848212 >> v2:172.21.15.73:3300/0 conn(0x7f0ff40cf340 0x7f0ff40d0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.540+0000 7f0ff2a00640 1 -- 172.21.15.46:0/16848212 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff40b0380 con 0x7f0ff415a790 2024-03-29T08:19:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.544+0000 7f0ff2a00640 1 --2- 172.21.15.46:0/16848212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff415a790 0x7f0ff40cee00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40b67c0 tx=0x7f0fe401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c1861c5394705a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.544+0000 7f0ff2000640 1 -- 172.21.15.46:0/16848212 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40ab640 con 0x7f0ff415a790 2024-03-29T08:19:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.544+0000 7f0ff2000640 1 -- 172.21.15.46:0/16848212 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe40a7070 con 0x7f0ff415a790 2024-03-29T08:19:15.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.544+0000 7f0ff2000640 1 -- 172.21.15.46:0/16848212 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40ab9c0 con 0x7f0ff415a790 2024-03-29T08:19:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.544+0000 7f0ff9a00640 1 -- 172.21.15.46:0/16848212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff415a790 msgr2=0x7f0ff40cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.544+0000 7f0ff9a00640 1 --2- 172.21.15.46:0/16848212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff415a790 0x7f0ff40cee00 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40b67c0 tx=0x7f0fe401c460 comp rx=0 tx=0).stop 2024-03-29T08:19:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.544+0000 7f0ff9a00640 1 -- 172.21.15.46:0/16848212 shutdown_connections 2024-03-29T08:19:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.544+0000 7f0ff9a00640 1 --2- 172.21.15.46:0/16848212 >> v2:172.21.15.73:3300/0 conn(0x7f0ff40cf340 0x7f0ff40d0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.544+0000 7f0ff9a00640 1 --2- 172.21.15.46:0/16848212 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ff40cbbe0 0x7f0ff40d0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.544+0000 7f0ff9a00640 1 --2- 172.21.15.46:0/16848212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff415a790 0x7f0ff40cee00 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.544+0000 7f0ff9a00640 1 -- 172.21.15.46:0/16848212 >> 172.21.15.46:0/16848212 conn(0x7f0ff4099a10 msgr2=0x7f0ff4089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.544+0000 7f0ff9a00640 1 -- 172.21.15.46:0/16848212 shutdown_connections 2024-03-29T08:19:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.544+0000 7f0ff9a00640 1 -- 172.21.15.46:0/16848212 wait complete. 2024-03-29T08:19:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.544+0000 7f0ff9a00640 1 Processor -- start 2024-03-29T08:19:15.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.544+0000 7f0ff9a00640 1 -- start start 2024-03-29T08:19:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.544+0000 7f0ff9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40cbbe0 0x7f0ff4158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.544+0000 7f0ff9a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ff40cf340 0x7f0ff4157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0ff9a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff415a790 0x7f0ff4153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0ff9a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff4159350 con 0x7f0ff40cbbe0 2024-03-29T08:19:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0ff9a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff4157950 con 0x7f0ff415a790 2024-03-29T08:19:15.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0ff9a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff415a0b0 con 0x7f0ff40cf340 2024-03-29T08:19:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0ff2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40cbbe0 0x7f0ff4158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0ff2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40cbbe0 0x7f0ff4158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56290/0 (socket says 172.21.15.46:56290) 2024-03-29T08:19:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0ff2a00640 1 -- 172.21.15.46:0/74951876 learned_addr learned my addr 172.21.15.46:0/74951876 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0ff3400640 1 --2- 172.21.15.46:0/74951876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff415a790 0x7f0ff4153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0fea000640 1 --2- 172.21.15.46:0/74951876 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ff40cf340 0x7f0ff4157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0ff2a00640 1 -- 172.21.15.46:0/74951876 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ff40cf340 msgr2=0x7f0ff4157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0ff2a00640 1 --2- 172.21.15.46:0/74951876 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ff40cf340 0x7f0ff4157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0ff2a00640 1 -- 172.21.15.46:0/74951876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff415a790 msgr2=0x7f0ff4153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0ff2a00640 1 --2- 172.21.15.46:0/74951876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff415a790 0x7f0ff4153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0ff2a00640 1 -- 172.21.15.46:0/74951876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe40094d0 con 0x7f0ff40cbbe0 2024-03-29T08:19:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0fea000640 1 --2- 172.21.15.46:0/74951876 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ff40cf340 0x7f0ff4157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:15.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0ff2a00640 1 --2- 172.21.15.46:0/74951876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40cbbe0 0x7f0ff4158e10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40ad040 tx=0x7f0fe401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:15.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0feb400640 1 -- 172.21.15.46:0/74951876 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40be5f0 con 0x7f0ff40cbbe0 2024-03-29T08:19:15.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0feb400640 1 -- 172.21.15.46:0/74951876 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe40be770 con 0x7f0ff40cbbe0 2024-03-29T08:19:15.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0ff9a00640 1 -- 172.21.15.46:0/74951876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff40c80d0 con 0x7f0ff40cbbe0 2024-03-29T08:19:15.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0feb400640 1 -- 172.21.15.46:0/74951876 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40beb20 con 0x7f0ff40cbbe0 2024-03-29T08:19:15.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0ff9a00640 1 -- 172.21.15.46:0/74951876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff41595e0 con 0x7f0ff40cbbe0 2024-03-29T08:19:15.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0ff9a00640 1 -- 172.21.15.46:0/74951876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff409e7d0 con 0x7f0ff40cbbe0 2024-03-29T08:19:15.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.548+0000 7f0feb400640 1 -- 172.21.15.46:0/74951876 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0fe40b43f0 con 0x7f0ff40cbbe0 2024-03-29T08:19:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.552+0000 7f0feb400640 1 --2- 172.21.15.46:0/74951876 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0fd0066690 0x7f0fd0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.552+0000 7f0feb400640 1 -- 172.21.15.46:0/74951876 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fe411f440 con 0x7f0ff40cbbe0 2024-03-29T08:19:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.552+0000 7f0fea000640 1 --2- 172.21.15.46:0/74951876 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0fd0066690 0x7f0fd0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.552+0000 7f0fea000640 1 --2- 172.21.15.46:0/74951876 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0fd0066690 0x7f0fd0068b50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4154a50 tx=0x7f0fdc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:15.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.552+0000 7f0feb400640 1 -- 172.21.15.46:0/74951876 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0fe40ec710 con 0x7f0ff40cbbe0 2024-03-29T08:19:15.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:15.752+0000 7f0ff9a00640 1 -- 172.21.15.46:0/74951876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0ff40c8260 con 0x7f0ff40cbbe0 2024-03-29T08:19:16.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.680+0000 7f0feb400640 1 -- 172.21.15.46:0/74951876 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0fe40efd30 con 0x7f0ff40cbbe0 2024-03-29T08:19:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.680+0000 7f0ff9a00640 1 -- 172.21.15.46:0/74951876 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0fd0066690 msgr2=0x7f0fd0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.680+0000 7f0ff9a00640 1 --2- 172.21.15.46:0/74951876 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0fd0066690 0x7f0fd0068b50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4154a50 tx=0x7f0fdc008040 comp rx=0 tx=0).stop 2024-03-29T08:19:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.680+0000 7f0ff9a00640 1 -- 172.21.15.46:0/74951876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40cbbe0 msgr2=0x7f0ff4158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.680+0000 7f0ff9a00640 1 --2- 172.21.15.46:0/74951876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40cbbe0 0x7f0ff4158e10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0fe40ad040 tx=0x7f0fe401aca0 comp rx=0 tx=0).stop 2024-03-29T08:19:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.680+0000 7f0ff9a00640 1 -- 172.21.15.46:0/74951876 shutdown_connections 2024-03-29T08:19:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.680+0000 7f0ff9a00640 1 --2- 172.21.15.46:0/74951876 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0fd0066690 0x7f0fd0068b50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.680+0000 7f0ff9a00640 1 --2- 172.21.15.46:0/74951876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff415a790 0x7f0ff4153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.680+0000 7f0ff9a00640 1 --2- 172.21.15.46:0/74951876 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ff40cf340 0x7f0ff4157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.680+0000 7f0ff9a00640 1 --2- 172.21.15.46:0/74951876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff40cbbe0 0x7f0ff4158e10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.680+0000 7f0ff9a00640 1 -- 172.21.15.46:0/74951876 >> 172.21.15.46:0/74951876 conn(0x7f0ff4099a10 msgr2=0x7f0ff406fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:16.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.680+0000 7f0ff9a00640 1 -- 172.21.15.46:0/74951876 shutdown_connections 2024-03-29T08:19:16.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.680+0000 7f0ff9a00640 1 -- 172.21.15.46:0/74951876 wait complete. 2024-03-29T08:19:16.706 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-29T08:19:16.706 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T08:19:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.852+0000 7faaec600640 1 Processor -- start 2024-03-29T08:19:16.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.852+0000 7faaec600640 1 -- start start 2024-03-29T08:19:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.852+0000 7faaec600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faae40b1570 0x7faae40b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.852+0000 7faaec600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faae4098a40 0x7faae4098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.852+0000 7faaec600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae40cb3f0 con 0x7faae415dfb0 2024-03-29T08:19:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.852+0000 7faaec600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae4099410 con 0x7faae4098a40 2024-03-29T08:19:16.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.852+0000 7faaec600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae40b1e00 con 0x7faae40b1570 2024-03-29T08:19:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.856+0000 7faaea600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faae4098a40 0x7faae4098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.856+0000 7faaea600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faae4098a40 0x7faae4098ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34748/0 (socket says 172.21.15.46:34748) 2024-03-29T08:19:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.856+0000 7faaea600640 1 -- 172.21.15.46:0/1011198303 learned_addr learned my addr 172.21.15.46:0/1011198303 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.856+0000 7faae9200640 1 --2- 172.21.15.46:0/1011198303 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faae40b1570 0x7faae40b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:16.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.856+0000 7faaea600640 1 -- 172.21.15.46:0/1011198303 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faae40b1570 msgr2=0x7faae40b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.856+0000 7faaea600640 1 --2- 172.21.15.46:0/1011198303 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faae40b1570 0x7faae40b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.856+0000 7faaea600640 1 -- 172.21.15.46:0/1011198303 >> v1:172.21.15.73:6789/0 conn(0x7faae415dfb0 legacy=0x7faae40ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:16.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.856+0000 7faaea600640 1 -- 172.21.15.46:0/1011198303 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae40cb620 con 0x7faae4098a40 2024-03-29T08:19:16.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.856+0000 7faaea600640 1 --2- 172.21.15.46:0/1011198303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae4098a40 0x7faae4098ed0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7faadc009e50 tx=0x7faadc01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a403d6295fb59847 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:16.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.856+0000 7faae3e00640 1 -- 172.21.15.46:0/1011198303 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc0a57e0 con 0x7faae4098a40 2024-03-29T08:19:16.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.856+0000 7faae3e00640 1 -- 172.21.15.46:0/1011198303 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faadc0a5940 con 0x7faae4098a40 2024-03-29T08:19:16.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.856+0000 7faae3e00640 1 -- 172.21.15.46:0/1011198303 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faadc0a5cd0 con 0x7faae4098a40 2024-03-29T08:19:16.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaec600640 1 -- 172.21.15.46:0/1011198303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae4098a40 msgr2=0x7faae4098ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:16.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaec600640 1 --2- 172.21.15.46:0/1011198303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae4098a40 0x7faae4098ed0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7faadc009e50 tx=0x7faadc01ca00 comp rx=0 tx=0).stop 2024-03-29T08:19:16.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaec600640 1 -- 172.21.15.46:0/1011198303 shutdown_connections 2024-03-29T08:19:16.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaec600640 1 --2- 172.21.15.46:0/1011198303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae4098a40 0x7faae4098ed0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:16.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaec600640 1 --2- 172.21.15.46:0/1011198303 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faae40b1570 0x7faae40b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:16.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaec600640 1 -- 172.21.15.46:0/1011198303 >> 172.21.15.46:0/1011198303 conn(0x7faae40b2220 msgr2=0x7faae40b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:16.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaec600640 1 -- 172.21.15.46:0/1011198303 shutdown_connections 2024-03-29T08:19:16.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaec600640 1 -- 172.21.15.46:0/1011198303 wait complete. 2024-03-29T08:19:16.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaec600640 1 Processor -- start 2024-03-29T08:19:16.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaec600640 1 -- start start 2024-03-29T08:19:16.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaec600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae4098a40 0x7faae414bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:16.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaec600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faae40b1570 0x7faae40b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:16.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaec600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faae415dfb0 0x7faae40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:16.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaec600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae40ce850 con 0x7faae4098a40 2024-03-29T08:19:16.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaec600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae414c460 con 0x7faae415dfb0 2024-03-29T08:19:16.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaec600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae415ca40 con 0x7faae40b1570 2024-03-29T08:19:16.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faae9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae4098a40 0x7faae414bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:16.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faae9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae4098a40 0x7faae414bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34752/0 (socket says 172.21.15.46:34752) 2024-03-29T08:19:16.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faae9c00640 1 -- 172.21.15.46:0/2164135025 learned_addr learned my addr 172.21.15.46:0/2164135025 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:16.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faae9200640 1 --2- 172.21.15.46:0/2164135025 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faae40b1570 0x7faae40b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:16.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaea600640 1 --2- 172.21.15.46:0/2164135025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faae415dfb0 0x7faae40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:16.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faae9c00640 1 -- 172.21.15.46:0/2164135025 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faae40b1570 msgr2=0x7faae40b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:16.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faae9c00640 1 --2- 172.21.15.46:0/2164135025 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faae40b1570 0x7faae40b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:16.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faae9c00640 1 -- 172.21.15.46:0/2164135025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faae415dfb0 msgr2=0x7faae40b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:16.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faae9c00640 1 --2- 172.21.15.46:0/2164135025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faae415dfb0 0x7faae40b4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:16.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faae9c00640 1 -- 172.21.15.46:0/2164135025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faadc009b30 con 0x7faae4098a40 2024-03-29T08:19:16.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.860+0000 7faaea600640 1 --2- 172.21.15.46:0/2164135025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faae415dfb0 0x7faae40b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:16.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.864+0000 7faae9c00640 1 --2- 172.21.15.46:0/2164135025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae4098a40 0x7faae414bf20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faad800c3c0 tx=0x7faad800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:16.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.864+0000 7faae1600640 1 -- 172.21.15.46:0/2164135025 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad80052d0 con 0x7faae4098a40 2024-03-29T08:19:16.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.864+0000 7faae1600640 1 -- 172.21.15.46:0/2164135025 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad8005430 con 0x7faae4098a40 2024-03-29T08:19:16.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.864+0000 7faaec600640 1 -- 172.21.15.46:0/2164135025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae415cd20 con 0x7faae4098a40 2024-03-29T08:19:16.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.864+0000 7faae1600640 1 -- 172.21.15.46:0/2164135025 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad8003840 con 0x7faae4098a40 2024-03-29T08:19:16.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.864+0000 7faaec600640 1 -- 172.21.15.46:0/2164135025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae40ce200 con 0x7faae4098a40 2024-03-29T08:19:16.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.864+0000 7faaec600640 1 -- 172.21.15.46:0/2164135025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae409e700 con 0x7faae4098a40 2024-03-29T08:19:16.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.864+0000 7faae1600640 1 -- 172.21.15.46:0/2164135025 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7faad80039a0 con 0x7faae4098a40 2024-03-29T08:19:16.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.864+0000 7faae1600640 1 --2- 172.21.15.46:0/2164135025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faac4066690 0x7faac4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:16.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.864+0000 7faae1600640 1 -- 172.21.15.46:0/2164135025 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faad8083620 con 0x7faae4098a40 2024-03-29T08:19:16.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.864+0000 7faae9200640 1 --2- 172.21.15.46:0/2164135025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faac4066690 0x7faac4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:16.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.868+0000 7faae9200640 1 --2- 172.21.15.46:0/2164135025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faac4066690 0x7faac4068b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faae40c3bb0 tx=0x7faad0009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:16.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:16.868+0000 7faae1600640 1 -- 172.21.15.46:0/2164135025 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7faad8050a30 con 0x7faae4098a40 2024-03-29T08:19:17.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.064+0000 7faaec600640 1 -- 172.21.15.46:0/2164135025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7faae404e120 con 0x7faae4098a40 2024-03-29T08:19:17.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.680+0000 7faae1600640 1 -- 172.21.15.46:0/2164135025 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7faad8054050 con 0x7faae4098a40 2024-03-29T08:19:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.684+0000 7faaec600640 1 -- 172.21.15.46:0/2164135025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faac4066690 msgr2=0x7faac4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.684+0000 7faaec600640 1 --2- 172.21.15.46:0/2164135025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faac4066690 0x7faac4068b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7faae40c3bb0 tx=0x7faad0009190 comp rx=0 tx=0).stop 2024-03-29T08:19:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.684+0000 7faaec600640 1 -- 172.21.15.46:0/2164135025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae4098a40 msgr2=0x7faae414bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.684+0000 7faaec600640 1 --2- 172.21.15.46:0/2164135025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae4098a40 0x7faae414bf20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7faad800c3c0 tx=0x7faad800c880 comp rx=0 tx=0).stop 2024-03-29T08:19:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.684+0000 7faaec600640 1 -- 172.21.15.46:0/2164135025 shutdown_connections 2024-03-29T08:19:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.684+0000 7faaec600640 1 --2- 172.21.15.46:0/2164135025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faac4066690 0x7faac4068b50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.684+0000 7faaec600640 1 --2- 172.21.15.46:0/2164135025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faae415dfb0 0x7faae40b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:17.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.684+0000 7faaec600640 1 --2- 172.21.15.46:0/2164135025 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faae40b1570 0x7faae40b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:17.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.684+0000 7faaec600640 1 --2- 172.21.15.46:0/2164135025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faae4098a40 0x7faae414bf20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:17.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.684+0000 7faaec600640 1 -- 172.21.15.46:0/2164135025 >> 172.21.15.46:0/2164135025 conn(0x7faae40b2220 msgr2=0x7faae4153ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:17.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.684+0000 7faaec600640 1 -- 172.21.15.46:0/2164135025 shutdown_connections 2024-03-29T08:19:17.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.684+0000 7faaec600640 1 -- 172.21.15.46:0/2164135025 wait complete. 2024-03-29T08:19:17.707 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T08:19:17.707 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:19:17.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.848+0000 7f64de200640 1 Processor -- start 2024-03-29T08:19:17.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.848+0000 7f64de200640 1 -- start start 2024-03-29T08:19:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64de200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f64d8144210 0x7f64d80cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64de200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64d80ceb20 0x7f64d806c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64de200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d80cb510 con 0x7f64d815ada0 2024-03-29T08:19:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64de200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d8070580 con 0x7f64d8144210 2024-03-29T08:19:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64de200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d8070860 con 0x7f64d80ceb20 2024-03-29T08:19:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64cfe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f64d815ada0 0x7f64d80ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56078/0 (socket says 172.21.15.46:56078) 2024-03-29T08:19:17.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64cfe00640 1 -- 172.21.15.46:0/2261587903 learned_addr learned my addr 172.21.15.46:0/2261587903 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:17.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d6a00640 1 -- 172.21.15.46:0/2261587903 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1603276407 0 0) 0x7f64d80cb510 con 0x7f64d815ada0 2024-03-29T08:19:17.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d7e00640 1 --2- 172.21.15.46:0/2261587903 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64d80ceb20 0x7f64d806c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:17.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d6a00640 1 -- 172.21.15.46:0/2261587903 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64bc003530 con 0x7f64d815ada0 2024-03-29T08:19:17.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d7400640 1 --2- 172.21.15.46:0/2261587903 >> v2:172.21.15.73:3300/0 conn(0x7f64d8144210 0x7f64d80cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:17.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d6a00640 1 -- 172.21.15.46:0/2261587903 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 44954839 0 0) 0x7f64bc003530 con 0x7f64d815ada0 2024-03-29T08:19:17.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d6a00640 1 -- 172.21.15.46:0/2261587903 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64d80ceb20 msgr2=0x7f64d806c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:17.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d6a00640 1 --2- 172.21.15.46:0/2261587903 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64d80ceb20 0x7f64d806c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:17.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d6a00640 1 -- 172.21.15.46:0/2261587903 >> v2:172.21.15.73:3300/0 conn(0x7f64d8144210 msgr2=0x7f64d80cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:17.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d6a00640 1 --2- 172.21.15.46:0/2261587903 >> v2:172.21.15.73:3300/0 conn(0x7f64d8144210 0x7f64d80cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:17.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d6a00640 1 -- 172.21.15.46:0/2261587903 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d8070b40 con 0x7f64d815ada0 2024-03-29T08:19:17.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d6a00640 1 -- 172.21.15.46:0/2261587903 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f64c0001680 con 0x7f64d815ada0 2024-03-29T08:19:17.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d6a00640 1 -- 172.21.15.46:0/2261587903 >> v1:172.21.15.46:6789/0 conn(0x7f64d815ada0 legacy=0x7f64d80ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:17.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d6a00640 1 --2- 172.21.15.46:0/2261587903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64bc003980 0x7f64bc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:17.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d6a00640 1 --2- 172.21.15.46:0/2261587903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64bc0047d0 0x7f64bc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:17.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d6a00640 1 --2- 172.21.15.46:0/2261587903 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64d80ceb20 0x7f64bc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:17.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d6a00640 1 -- 172.21.15.46:0/2261587903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d8070580 con 0x7f64bc0047d0 2024-03-29T08:19:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d6a00640 1 -- 172.21.15.46:0/2261587903 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d8070860 con 0x7f64bc003980 2024-03-29T08:19:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d6a00640 1 -- 172.21.15.46:0/2261587903 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc00b990 con 0x7f64d80ceb20 2024-03-29T08:19:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64cfe00640 1 --2- 172.21.15.46:0/2261587903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64bc0047d0 0x7f64bc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d7400640 1 --2- 172.21.15.46:0/2261587903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64bc003980 0x7f64bc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64d7e00640 1 --2- 172.21.15.46:0/2261587903 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64d80ceb20 0x7f64bc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64cfe00640 1 -- 172.21.15.46:0/2261587903 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64d80ceb20 msgr2=0x7f64bc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64cfe00640 1 --2- 172.21.15.46:0/2261587903 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64d80ceb20 0x7f64bc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64cfe00640 1 -- 172.21.15.46:0/2261587903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64bc003980 msgr2=0x7f64bc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64cfe00640 1 --2- 172.21.15.46:0/2261587903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64bc003980 0x7f64bc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:17.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.852+0000 7f64cfe00640 1 -- 172.21.15.46:0/2261587903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64bc003530 con 0x7f64bc0047d0 2024-03-29T08:19:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.856+0000 7f64cfe00640 1 --2- 172.21.15.46:0/2261587903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64bc0047d0 0x7f64bc004c50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f64c000c240 tx=0x7f64c001f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693d4cd8ca0f44d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.856+0000 7f64d6a00640 1 -- 172.21.15.46:0/2261587903 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c0005ae0 con 0x7f64bc0047d0 2024-03-29T08:19:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.856+0000 7f64d6a00640 1 -- 172.21.15.46:0/2261587903 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c00ab070 con 0x7f64bc0047d0 2024-03-29T08:19:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.856+0000 7f64d6a00640 1 -- 172.21.15.46:0/2261587903 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c0005e60 con 0x7f64bc0047d0 2024-03-29T08:19:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.856+0000 7f64de200640 1 -- 172.21.15.46:0/2261587903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64bc0047d0 msgr2=0x7f64bc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.856+0000 7f64de200640 1 --2- 172.21.15.46:0/2261587903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64bc0047d0 0x7f64bc004c50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f64c000c240 tx=0x7f64c001f3f0 comp rx=0 tx=0).stop 2024-03-29T08:19:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.856+0000 7f64d7e00640 1 -- 172.21.15.46:0/2261587903 reap_dead start 2024-03-29T08:19:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.856+0000 7f64de200640 1 -- 172.21.15.46:0/2261587903 shutdown_connections 2024-03-29T08:19:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.856+0000 7f64de200640 1 -- 172.21.15.46:0/2261587903 >> 172.21.15.46:0/2261587903 conn(0x7f64d8066110 msgr2=0x7f64d8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.856+0000 7f64de200640 1 -- 172.21.15.46:0/2261587903 shutdown_connections 2024-03-29T08:19:17.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.856+0000 7f64de200640 1 -- 172.21.15.46:0/2261587903 wait complete. 2024-03-29T08:19:17.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64de200640 1 Processor -- start 2024-03-29T08:19:17.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64de200640 1 -- start start 2024-03-29T08:19:17.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64de200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64bc0047d0 0x7f64d8159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:17.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64de200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64d80ac2c0 0x7f64d813ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:17.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64de200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64d8140680 0x7f64d813efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:17.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64de200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80b1600 con 0x7f64bc0047d0 2024-03-29T08:19:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64de200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80ac7c0 con 0x7f64d80ac2c0 2024-03-29T08:19:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64de200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80acaa0 con 0x7f64d8140680 2024-03-29T08:19:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64d7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64bc0047d0 0x7f64d8159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64d7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64bc0047d0 0x7f64d8159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34770/0 (socket says 172.21.15.46:34770) 2024-03-29T08:19:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64d7400640 1 -- 172.21.15.46:0/1781267464 learned_addr learned my addr 172.21.15.46:0/1781267464 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64d7e00640 1 --2- 172.21.15.46:0/1781267464 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64d8140680 0x7f64d813efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64cfe00640 1 --2- 172.21.15.46:0/1781267464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64d80ac2c0 0x7f64d813ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64d7400640 1 -- 172.21.15.46:0/1781267464 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64d8140680 msgr2=0x7f64d813efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64d7400640 1 --2- 172.21.15.46:0/1781267464 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64d8140680 0x7f64d813efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64d7400640 1 -- 172.21.15.46:0/1781267464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64d80ac2c0 msgr2=0x7f64d813ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64d7400640 1 --2- 172.21.15.46:0/1781267464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64d80ac2c0 0x7f64d813ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64d7400640 1 -- 172.21.15.46:0/1781267464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d80acd80 con 0x7f64bc0047d0 2024-03-29T08:19:17.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64d7400640 1 --2- 172.21.15.46:0/1781267464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64bc0047d0 0x7f64d8159bd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f64c80119d0 tx=0x7f64c8011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:17.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64cf400640 1 -- 172.21.15.46:0/1781267464 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c800fb50 con 0x7f64bc0047d0 2024-03-29T08:19:17.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64cf400640 1 -- 172.21.15.46:0/1781267464 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c800fcb0 con 0x7f64bc0047d0 2024-03-29T08:19:17.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64cf400640 1 -- 172.21.15.46:0/1781267464 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8014440 con 0x7f64bc0047d0 2024-03-29T08:19:17.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64de200640 1 -- 172.21.15.46:0/1781267464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d80ad060 con 0x7f64bc0047d0 2024-03-29T08:19:17.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64de200640 1 -- 172.21.15.46:0/1781267464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d815f070 con 0x7f64bc0047d0 2024-03-29T08:19:17.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.860+0000 7f64de200640 1 -- 172.21.15.46:0/1781267464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d80ce280 con 0x7f64bc0047d0 2024-03-29T08:19:17.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.864+0000 7f64cf400640 1 -- 172.21.15.46:0/1781267464 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f64c80145a0 con 0x7f64bc0047d0 2024-03-29T08:19:17.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.864+0000 7f64cf400640 1 --2- 172.21.15.46:0/1781267464 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f64b40666c0 0x7f64b4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:17.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.864+0000 7f64cf400640 1 -- 172.21.15.46:0/1781267464 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64c80848d0 con 0x7f64bc0047d0 2024-03-29T08:19:17.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.864+0000 7f64cfe00640 1 --2- 172.21.15.46:0/1781267464 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f64b40666c0 0x7f64b4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:17.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.864+0000 7f64cfe00640 1 --2- 172.21.15.46:0/1781267464 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f64b40666c0 0x7f64b4068b80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f64c00ae000 tx=0x7f64c00b4040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:17.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:17.868+0000 7f64cf400640 1 -- 172.21.15.46:0/1781267464 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f64c8051da0 con 0x7f64bc0047d0 2024-03-29T08:19:18.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.064+0000 7f64de200640 1 -- 172.21.15.46:0/1781267464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64d8066c90 con 0x7f64bc0047d0 2024-03-29T08:19:18.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.064+0000 7f64cf400640 1 -- 172.21.15.46:0/1781267464 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f64c8022020 con 0x7f64bc0047d0 2024-03-29T08:19:18.068 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:18.068 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/4274575126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4274575126},{"type":"v1","addr":"172.21.15.46:6834","nonce":4274575126}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3739899623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3739899623},{"type":"v1","addr":"172.21.15.73:6837","nonce":3739899623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1905427475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1905427475},{"type":"v1","addr":"172.21.15.46:6835","nonce":1905427475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:19:14.663142+0000","modified":"2024-03-29T08:19:17.680359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5152},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5152":{"gid":5152,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/1941408960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1941408960},{"type":"v1","addr":"172.21.15.73:6836","nonce":1941408960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-29T08:19:18.069 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2024-03-29T08:19:18.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.068+0000 7f64de200640 1 -- 172.21.15.46:0/1781267464 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f64b40666c0 msgr2=0x7f64b4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.068+0000 7f64de200640 1 --2- 172.21.15.46:0/1781267464 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f64b40666c0 0x7f64b4068b80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f64c00ae000 tx=0x7f64c00b4040 comp rx=0 tx=0).stop 2024-03-29T08:19:18.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.068+0000 7f64de200640 1 -- 172.21.15.46:0/1781267464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64bc0047d0 msgr2=0x7f64d8159bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.068+0000 7f64de200640 1 --2- 172.21.15.46:0/1781267464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64bc0047d0 0x7f64d8159bd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f64c80119d0 tx=0x7f64c8011e90 comp rx=0 tx=0).stop 2024-03-29T08:19:18.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.068+0000 7f64de200640 1 -- 172.21.15.46:0/1781267464 shutdown_connections 2024-03-29T08:19:18.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.068+0000 7f64de200640 1 --2- 172.21.15.46:0/1781267464 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f64b40666c0 0x7f64b4068b80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.068+0000 7f64de200640 1 --2- 172.21.15.46:0/1781267464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64d80ac2c0 0x7f64d813ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.068+0000 7f64de200640 1 --2- 172.21.15.46:0/1781267464 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64d8140680 0x7f64d813efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.068+0000 7f64de200640 1 --2- 172.21.15.46:0/1781267464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64bc0047d0 0x7f64d8159bd0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.068+0000 7f64de200640 1 -- 172.21.15.46:0/1781267464 >> 172.21.15.46:0/1781267464 conn(0x7f64d8066110 msgr2=0x7f64d80af760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:18.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.068+0000 7f64de200640 1 -- 172.21.15.46:0/1781267464 shutdown_connections 2024-03-29T08:19:18.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.068+0000 7f64de200640 1 -- 172.21.15.46:0/1781267464 wait complete. 2024-03-29T08:19:18.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:19:18.165 INFO:tasks.ceph.mds.b.smithi073.stderr:2024-03-29T08:19:18.164+0000 7fcff9000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:19:18.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fd7400640 1 Processor -- start 2024-03-29T08:19:18.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fd7400640 1 -- start start 2024-03-29T08:19:18.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fd7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6fd0153d50 0x7f6fd01611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:18.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fd7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6fd00ad3e0 0x7f6fd00ad850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:18.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fd7400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd015d870 con 0x7f6fd0153970 2024-03-29T08:19:18.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fd7400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd00add90 con 0x7f6fd00ad3e0 2024-03-29T08:19:18.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fd7400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd015db50 con 0x7f6fd0153d50 2024-03-29T08:19:18.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fd5400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6fd00ad3e0 0x7f6fd00ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:18.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fd5400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6fd00ad3e0 0x7f6fd00ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34774/0 (socket says 172.21.15.46:34774) 2024-03-29T08:19:18.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fd4a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6fd0153970 0x7f6fd015cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56086/0 (socket says 172.21.15.46:56086) 2024-03-29T08:19:18.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fd5400640 1 -- 172.21.15.46:0/1227801034 learned_addr learned my addr 172.21.15.46:0/1227801034 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:18.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fd5400640 1 -- 172.21.15.46:0/1227801034 >> v2:172.21.15.73:3300/0 conn(0x7f6fd0153d50 msgr2=0x7f6fd01611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:18.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fcfe00640 1 --2- 172.21.15.46:0/1227801034 >> v2:172.21.15.73:3300/0 conn(0x7f6fd0153d50 0x7f6fd01611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:18.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fcf400640 1 -- 172.21.15.46:0/1227801034 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428984853 0 0) 0x7f6fd015d870 con 0x7f6fd0153970 2024-03-29T08:19:18.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fd5400640 1 --2- 172.21.15.46:0/1227801034 >> v2:172.21.15.73:3300/0 conn(0x7f6fd0153d50 0x7f6fd01611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fd5400640 1 -- 172.21.15.46:0/1227801034 >> v1:172.21.15.46:6789/0 conn(0x7f6fd0153970 legacy=0x7f6fd015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fd5400640 1 -- 172.21.15.46:0/1227801034 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd015de30 con 0x7f6fd00ad3e0 2024-03-29T08:19:18.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.228+0000 7f6fcfe00640 1 --2- 172.21.15.46:0/1227801034 >> v2:172.21.15.73:3300/0 conn(0x7f6fd0153d50 0x7f6fd01611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.232+0000 7f6fd5400640 1 --2- 172.21.15.46:0/1227801034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd00ad3e0 0x7f6fd00ad850 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6fd006e480 tx=0x7f6fc801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9819f0173e037e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.232+0000 7f6fcf400640 1 -- 172.21.15.46:0/1227801034 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc80a5820 con 0x7f6fd00ad3e0 2024-03-29T08:19:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.232+0000 7f6fcf400640 1 -- 172.21.15.46:0/1227801034 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc80a5980 con 0x7f6fd00ad3e0 2024-03-29T08:19:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.232+0000 7f6fcf400640 1 -- 172.21.15.46:0/1227801034 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc80a5d10 con 0x7f6fd00ad3e0 2024-03-29T08:19:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.232+0000 7f6fd7400640 1 -- 172.21.15.46:0/1227801034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd00ad3e0 msgr2=0x7f6fd00ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.232+0000 7f6fd7400640 1 --2- 172.21.15.46:0/1227801034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd00ad3e0 0x7f6fd00ad850 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f6fd006e480 tx=0x7f6fc801c9c0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.232+0000 7f6fd7400640 1 -- 172.21.15.46:0/1227801034 shutdown_connections 2024-03-29T08:19:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.232+0000 7f6fd7400640 1 --2- 172.21.15.46:0/1227801034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd00ad3e0 0x7f6fd00ad850 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.232+0000 7f6fd7400640 1 --2- 172.21.15.46:0/1227801034 >> v2:172.21.15.73:3300/0 conn(0x7f6fd0153d50 0x7f6fd01611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.232+0000 7f6fd7400640 1 -- 172.21.15.46:0/1227801034 >> 172.21.15.46:0/1227801034 conn(0x7f6fd0066110 msgr2=0x7f6fd0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:18.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.232+0000 7f6fd7400640 1 -- 172.21.15.46:0/1227801034 shutdown_connections 2024-03-29T08:19:18.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.232+0000 7f6fd7400640 1 -- 172.21.15.46:0/1227801034 wait complete. 2024-03-29T08:19:18.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fd7400640 1 Processor -- start 2024-03-29T08:19:18.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fd7400640 1 -- start start 2024-03-29T08:19:18.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fd7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd00ad3e0 0x7f6fd0141ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fd7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0153970 0x7f6fd014fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:18.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fd7400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fd00af1f0 0x7f6fd01499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fd7400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00cd9a0 con 0x7f6fd0153970 2024-03-29T08:19:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fd7400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd01420e0 con 0x7f6fd00ad3e0 2024-03-29T08:19:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fd7400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd00caac0 con 0x7f6fd00af1f0 2024-03-29T08:19:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fcfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0153970 0x7f6fd014fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fcfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0153970 0x7f6fd014fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34780/0 (socket says 172.21.15.46:34780) 2024-03-29T08:19:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fcfe00640 1 -- 172.21.15.46:0/1283610456 learned_addr learned my addr 172.21.15.46:0/1283610456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fd4a00640 1 --2- 172.21.15.46:0/1283610456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd00ad3e0 0x7f6fd0141ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fd5400640 1 --2- 172.21.15.46:0/1283610456 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fd00af1f0 0x7f6fd01499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fcfe00640 1 -- 172.21.15.46:0/1283610456 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fd00af1f0 msgr2=0x7f6fd01499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fcfe00640 1 --2- 172.21.15.46:0/1283610456 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fd00af1f0 0x7f6fd01499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fcfe00640 1 -- 172.21.15.46:0/1283610456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd00ad3e0 msgr2=0x7f6fd0141ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fcfe00640 1 --2- 172.21.15.46:0/1283610456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd00ad3e0 0x7f6fd0141ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fcfe00640 1 -- 172.21.15.46:0/1283610456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc8009b30 con 0x7f6fd0153970 2024-03-29T08:19:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fd4a00640 1 --2- 172.21.15.46:0/1283610456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd00ad3e0 0x7f6fd0141ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:18.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fcfe00640 1 --2- 172.21.15.46:0/1283610456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0153970 0x7f6fd014fb70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00c7f0 tx=0x7f6fbc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:18.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fccc00640 1 -- 172.21.15.46:0/1283610456 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc007820 con 0x7f6fd0153970 2024-03-29T08:19:18.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fccc00640 1 -- 172.21.15.46:0/1283610456 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc007980 con 0x7f6fd0153970 2024-03-29T08:19:18.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fccc00640 1 -- 172.21.15.46:0/1283610456 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc007d30 con 0x7f6fd0153970 2024-03-29T08:19:18.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fd7400640 1 -- 172.21.15.46:0/1283610456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd00b0990 con 0x7f6fd0153970 2024-03-29T08:19:18.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.236+0000 7f6fd7400640 1 -- 172.21.15.46:0/1283610456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd00cad10 con 0x7f6fd0153970 2024-03-29T08:19:18.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.240+0000 7f6fd7400640 1 -- 172.21.15.46:0/1283610456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd0066c90 con 0x7f6fd0153970 2024-03-29T08:19:18.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.240+0000 7f6fccc00640 1 -- 172.21.15.46:0/1283610456 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6fbc0113f0 con 0x7f6fd0153970 2024-03-29T08:19:18.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.240+0000 7f6fccc00640 1 --2- 172.21.15.46:0/1283610456 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6fb0066690 0x7f6fb0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:18.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.240+0000 7f6fccc00640 1 -- 172.21.15.46:0/1283610456 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fbc0843e0 con 0x7f6fd0153970 2024-03-29T08:19:18.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.240+0000 7f6fd4a00640 1 --2- 172.21.15.46:0/1283610456 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6fb0066690 0x7f6fb0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:18.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.240+0000 7f6fd4a00640 1 --2- 172.21.15.46:0/1283610456 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6fb0066690 0x7f6fb0068b50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4007fd0 tx=0x7f6fc4007c20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:18.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.244+0000 7f6fccc00640 1 -- 172.21.15.46:0/1283610456 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6fbc0517f0 con 0x7f6fd0153970 2024-03-29T08:19:18.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.408+0000 7f6fd7400640 1 -- 172.21.15.46:0/1283610456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fd006c240 con 0x7f6fd0153970 2024-03-29T08:19:18.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.408+0000 7f6fccc00640 1 -- 172.21.15.46:0/1283610456 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f6fbc054e10 con 0x7f6fd0153970 2024-03-29T08:19:18.410 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:18.410 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:19:15.665674+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:19:12.674055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-29T08:19:13.661287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:19:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.412+0000 7f6fd7400640 1 -- 172.21.15.46:0/1283610456 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6fb0066690 msgr2=0x7f6fb0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.412+0000 7f6fd7400640 1 --2- 172.21.15.46:0/1283610456 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6fb0066690 0x7f6fb0068b50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4007fd0 tx=0x7f6fc4007c20 comp rx=0 tx=0).stop 2024-03-29T08:19:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.412+0000 7f6fd7400640 1 -- 172.21.15.46:0/1283610456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0153970 msgr2=0x7f6fd014fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.412+0000 7f6fd7400640 1 --2- 172.21.15.46:0/1283610456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0153970 0x7f6fd014fb70 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00c7f0 tx=0x7f6fbc00ccb0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.412+0000 7f6fd7400640 1 -- 172.21.15.46:0/1283610456 shutdown_connections 2024-03-29T08:19:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.412+0000 7f6fd7400640 1 --2- 172.21.15.46:0/1283610456 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6fb0066690 0x7f6fb0068b50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.412+0000 7f6fd7400640 1 --2- 172.21.15.46:0/1283610456 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fd00af1f0 0x7f6fd01499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.412+0000 7f6fd7400640 1 --2- 172.21.15.46:0/1283610456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fd0153970 0x7f6fd014fb70 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.412+0000 7f6fd7400640 1 --2- 172.21.15.46:0/1283610456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd00ad3e0 0x7f6fd0141ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.412+0000 7f6fd7400640 1 -- 172.21.15.46:0/1283610456 >> 172.21.15.46:0/1283610456 conn(0x7f6fd0066110 msgr2=0x7f6fd015fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.412+0000 7f6fd7400640 1 -- 172.21.15.46:0/1283610456 shutdown_connections 2024-03-29T08:19:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.412+0000 7f6fd7400640 1 -- 172.21.15.46:0/1283610456 wait complete. 2024-03-29T08:19:18.432 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T08:19:18.432 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:19:18.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.568+0000 7f2543800640 1 Processor -- start 2024-03-29T08:19:18.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.568+0000 7f2543800640 1 -- start start 2024-03-29T08:19:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.572+0000 7f2543800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f253c0adaf0 0x7f253c1611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.572+0000 7f2543800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253c15d870 con 0x7f253c153970 2024-03-29T08:19:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.572+0000 7f2543800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253c15db50 con 0x7f253c153d50 2024-03-29T08:19:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.572+0000 7f2543800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c15de30 con 0x7f253c0adaf0 2024-03-29T08:19:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.572+0000 7f2540e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f253c153970 0x7f253c15cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56096/0 (socket says 172.21.15.46:56096) 2024-03-29T08:19:18.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.572+0000 7f2540e00640 1 -- 172.21.15.46:0/3433488983 learned_addr learned my addr 172.21.15.46:0/3433488983 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:18.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.572+0000 7f2541800640 1 --2- 172.21.15.46:0/3433488983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f253c0adaf0 0x7f253c1611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:18.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.572+0000 7f253be00640 1 -- 172.21.15.46:0/3433488983 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139993075 0 0) 0x7f253c15d870 con 0x7f253c153970 2024-03-29T08:19:18.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.572+0000 7f253be00640 1 -- 172.21.15.46:0/3433488983 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2520003530 con 0x7f253c153970 2024-03-29T08:19:18.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.572+0000 7f253be00640 1 -- 172.21.15.46:0/3433488983 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 136939475 0 0) 0x7f253c15db50 con 0x7f253c153d50 2024-03-29T08:19:18.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.572+0000 7f253be00640 1 -- 172.21.15.46:0/3433488983 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f253c15d870 con 0x7f253c153d50 2024-03-29T08:19:18.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.572+0000 7f2541800640 1 -- 172.21.15.46:0/3433488983 >> v1:172.21.15.73:6789/0 conn(0x7f253c153d50 legacy=0x7f253c0ad3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.572+0000 7f2541800640 1 -- 172.21.15.46:0/3433488983 >> v1:172.21.15.46:6789/0 conn(0x7f253c153970 legacy=0x7f253c15cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.572+0000 7f253be00640 1 -- 172.21.15.46:0/3433488983 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 659618489 0 0) 0x7f2520003530 con 0x7f253c153970 2024-03-29T08:19:18.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.572+0000 7f2541800640 1 -- 172.21.15.46:0/3433488983 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c0abea0 con 0x7f253c0adaf0 2024-03-29T08:19:18.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.576+0000 7f2541800640 1 --2- 172.21.15.46:0/3433488983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f253c0adaf0 0x7f253c1611e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2534009dc0 tx=0x7f253401c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e022d10d9b0e0e72 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:18.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.576+0000 7f253be00640 1 -- 172.21.15.46:0/3433488983 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c001680 con 0x7f253c0adaf0 2024-03-29T08:19:18.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.576+0000 7f253be00640 1 -- 172.21.15.46:0/3433488983 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253c0b2aa0 con 0x7f253c0adaf0 2024-03-29T08:19:18.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.576+0000 7f253be00640 1 -- 172.21.15.46:0/3433488983 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25340a58b0 con 0x7f253c0adaf0 2024-03-29T08:19:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.576+0000 7f2543800640 1 -- 172.21.15.46:0/3433488983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f253c0adaf0 msgr2=0x7f253c1611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.576+0000 7f2543800640 1 --2- 172.21.15.46:0/3433488983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f253c0adaf0 0x7f253c1611e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2534009dc0 tx=0x7f253401c8b0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.576+0000 7f2543800640 1 -- 172.21.15.46:0/3433488983 shutdown_connections 2024-03-29T08:19:18.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.576+0000 7f2543800640 1 --2- 172.21.15.46:0/3433488983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f253c0adaf0 0x7f253c1611e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.576+0000 7f2543800640 1 -- 172.21.15.46:0/3433488983 >> 172.21.15.46:0/3433488983 conn(0x7f253c066110 msgr2=0x7f253c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:18.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.576+0000 7f2543800640 1 -- 172.21.15.46:0/3433488983 shutdown_connections 2024-03-29T08:19:18.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.576+0000 7f2543800640 1 -- 172.21.15.46:0/3433488983 wait complete. 2024-03-29T08:19:18.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.576+0000 7f2543800640 1 Processor -- start 2024-03-29T08:19:18.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2543800640 1 -- start start 2024-03-29T08:19:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2543800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f253c0adaf0 0x7f253c14fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2543800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f253c153970 0x7f253c14d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2543800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c15c330 0x7f253c0aefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2543800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c157000 con 0x7f253c15c330 2024-03-29T08:19:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2543800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253c1567a0 con 0x7f253c0adaf0 2024-03-29T08:19:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2543800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253c156a40 con 0x7f253c153970 2024-03-29T08:19:18.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2541800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c15c330 0x7f253c0aefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2541800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c15c330 0x7f253c0aefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34788/0 (socket says 172.21.15.46:34788) 2024-03-29T08:19:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2541800640 1 -- 172.21.15.46:0/3643761664 learned_addr learned my addr 172.21.15.46:0/3643761664 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2540e00640 1 --2- 172.21.15.46:0/3643761664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f253c0adaf0 0x7f253c14fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2533e00640 1 --2- 172.21.15.46:0/3643761664 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f253c153970 0x7f253c14d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2541800640 1 -- 172.21.15.46:0/3643761664 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f253c153970 msgr2=0x7f253c14d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2541800640 1 --2- 172.21.15.46:0/3643761664 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f253c153970 0x7f253c14d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2541800640 1 -- 172.21.15.46:0/3643761664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f253c0adaf0 msgr2=0x7f253c14fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2541800640 1 --2- 172.21.15.46:0/3643761664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f253c0adaf0 0x7f253c14fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2541800640 1 -- 172.21.15.46:0/3643761664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2534009a70 con 0x7f253c15c330 2024-03-29T08:19:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2541800640 1 --2- 172.21.15.46:0/3643761664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c15c330 0x7f253c0aefb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f25340b0040 tx=0x7f2534019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2539600640 1 -- 172.21.15.46:0/3643761664 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534003ba0 con 0x7f253c15c330 2024-03-29T08:19:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2539600640 1 -- 172.21.15.46:0/3643761664 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2534003d20 con 0x7f253c15c330 2024-03-29T08:19:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2539600640 1 -- 172.21.15.46:0/3643761664 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534004270 con 0x7f253c15c330 2024-03-29T08:19:18.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2543800640 1 -- 172.21.15.46:0/3643761664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253c156d20 con 0x7f253c15c330 2024-03-29T08:19:18.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2543800640 1 -- 172.21.15.46:0/3643761664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253c0cab90 con 0x7f253c15c330 2024-03-29T08:19:18.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.580+0000 7f2543800640 1 -- 172.21.15.46:0/3643761664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253c066c90 con 0x7f253c15c330 2024-03-29T08:19:18.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.584+0000 7f2539600640 1 -- 172.21.15.46:0/3643761664 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f25340b4050 con 0x7f253c15c330 2024-03-29T08:19:18.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.584+0000 7f2539600640 1 --2- 172.21.15.46:0/3643761664 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2518066690 0x7f2518068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:18.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.584+0000 7f2539600640 1 -- 172.21.15.46:0/3643761664 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f253411dca0 con 0x7f253c15c330 2024-03-29T08:19:18.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.584+0000 7f2540e00640 1 --2- 172.21.15.46:0/3643761664 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2518066690 0x7f2518068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:18.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.584+0000 7f2540e00640 1 --2- 172.21.15.46:0/3643761664 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2518066690 0x7f2518068b50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f252c000f70 tx=0x7f252c0016c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:18.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.588+0000 7f2539600640 1 -- 172.21.15.46:0/3643761664 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f25340eb140 con 0x7f253c15c330 2024-03-29T08:19:18.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.744+0000 7f2543800640 1 -- 172.21.15.46:0/3643761664 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f253c16a9c0 con 0x7f2518066690 2024-03-29T08:19:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.756+0000 7f2539600640 1 -- 172.21.15.46:0/3643761664 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288161 (secure 0 0 0) 0x7f252c00fa50 con 0x7f2518066690 2024-03-29T08:19:18.761 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:19:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.760+0000 7f2543800640 1 -- 172.21.15.46:0/3643761664 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2518066690 msgr2=0x7f2518068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.760+0000 7f2543800640 1 --2- 172.21.15.46:0/3643761664 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2518066690 0x7f2518068b50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f252c000f70 tx=0x7f252c0016c0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.760+0000 7f2543800640 1 -- 172.21.15.46:0/3643761664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c15c330 msgr2=0x7f253c0aefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.760+0000 7f2543800640 1 --2- 172.21.15.46:0/3643761664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c15c330 0x7f253c0aefb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f25340b0040 tx=0x7f2534019650 comp rx=0 tx=0).stop 2024-03-29T08:19:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.760+0000 7f2543800640 1 -- 172.21.15.46:0/3643761664 shutdown_connections 2024-03-29T08:19:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.760+0000 7f2543800640 1 --2- 172.21.15.46:0/3643761664 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2518066690 0x7f2518068b50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.760+0000 7f2543800640 1 --2- 172.21.15.46:0/3643761664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f253c15c330 0x7f253c0aefb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.760+0000 7f2543800640 1 --2- 172.21.15.46:0/3643761664 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f253c153970 0x7f253c14d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.760+0000 7f2543800640 1 --2- 172.21.15.46:0/3643761664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f253c0adaf0 0x7f253c14fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.760+0000 7f2543800640 1 -- 172.21.15.46:0/3643761664 >> 172.21.15.46:0/3643761664 conn(0x7f253c066110 msgr2=0x7f253c15e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.760+0000 7f2543800640 1 -- 172.21.15.46:0/3643761664 shutdown_connections 2024-03-29T08:19:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.760+0000 7f2543800640 1 -- 172.21.15.46:0/3643761664 wait complete. 2024-03-29T08:19:18.781 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-03-29T08:19:18.057099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":184,"acting":184,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236656,"kb_used_data":4100,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738064,"statfs":{"total":773094113280,"available":772851777536,"internally_reserved":0,"allocated":4198400,"data_stored":2324897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.804539"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.679129+0000","last_change":"2024-03-29T08:19:14.677048+0000","last_active":"2024-03-29T08:19:14.679129+0000","last_peered":"2024-03-29T08:19:14.679129+0000","last_clean":"2024-03-29T08:19:14.679129+0000","last_became_active":"2024-03-29T08:19:14.676802+0000","last_became_peered":"2024-03-29T08:19:14.676802+0000","last_unstale":"2024-03-29T08:19:14.679129+0000","last_undegraded":"2024-03-29T08:19:14.679129+0000","last_fullsized":"2024-03-29T08:19:14.679129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:37:16.877010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:19:14.252791+0000","last_change":"2024-03-29T08:19:14.252791+0000","last_active":"2024-03-29T08:19:14.252791+0000","last_peered":"2024-03-29T08:19:14.252791+0000","last_clean":"2024-03-29T08:19:14.252791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:14.252791+0000","last_undegraded":"2024-03-29T08:19:14.252791+0000","last_fullsized":"2024-03-29T08:19:14.252791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.681839+0000","last_change":"2024-03-29T08:19:14.677189+0000","last_active":"2024-03-29T08:19:14.681839+0000","last_peered":"2024-03-29T08:19:14.681839+0000","last_clean":"2024-03-29T08:19:14.681839+0000","last_became_active":"2024-03-29T08:19:14.676526+0000","last_became_peered":"2024-03-29T08:19:14.676526+0000","last_unstale":"2024-03-29T08:19:14.681839+0000","last_undegraded":"2024-03-29T08:19:14.681839+0000","last_fullsized":"2024-03-29T08:19:14.681839+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:20.587889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705567+0000","last_change":"2024-03-29T08:19:14.699305+0000","last_active":"2024-03-29T08:19:14.705567+0000","last_peered":"2024-03-29T08:19:14.705567+0000","last_clean":"2024-03-29T08:19:14.705567+0000","last_became_active":"2024-03-29T08:19:14.698637+0000","last_became_peered":"2024-03-29T08:19:14.698637+0000","last_unstale":"2024-03-29T08:19:14.705567+0000","last_undegraded":"2024-03-29T08:19:14.705567+0000","last_fullsized":"2024-03-29T08:19:14.705567+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:35.234827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:19:14.252791+0000","last_change":"2024-03-29T08:19:14.252791+0000","last_active":"2024-03-29T08:19:14.252791+0000","last_peered":"2024-03-29T08:19:14.252791+0000","last_clean":"2024-03-29T08:19:14.252791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:14.252791+0000","last_undegraded":"2024-03-29T08:19:14.252791+0000","last_fullsized":"2024-03-29T08:19:14.252791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694387+0000","last_change":"2024-03-29T08:19:14.691105+0000","last_active":"2024-03-29T08:19:14.694387+0000","last_peered":"2024-03-29T08:19:14.694387+0000","last_clean":"2024-03-29T08:19:14.694387+0000","last_became_active":"2024-03-29T08:19:14.690486+0000","last_became_peered":"2024-03-29T08:19:14.690486+0000","last_unstale":"2024-03-29T08:19:14.694387+0000","last_undegraded":"2024-03-29T08:19:14.694387+0000","last_fullsized":"2024-03-29T08:19:14.694387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:21.736865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.678975+0000","last_change":"2024-03-29T08:19:14.675543+0000","last_active":"2024-03-29T08:19:14.678975+0000","last_peered":"2024-03-29T08:19:14.678975+0000","last_clean":"2024-03-29T08:19:14.678975+0000","last_became_active":"2024-03-29T08:19:14.675201+0000","last_became_peered":"2024-03-29T08:19:14.675201+0000","last_unstale":"2024-03-29T08:19:14.678975+0000","last_undegraded":"2024-03-29T08:19:14.678975+0000","last_fullsized":"2024-03-29T08:19:14.678975+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:08:58.521804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687632+0000","last_change":"2024-03-29T08:19:14.681677+0000","last_active":"2024-03-29T08:19:14.687632+0000","last_peered":"2024-03-29T08:19:14.687632+0000","last_clean":"2024-03-29T08:19:14.687632+0000","last_became_active":"2024-03-29T08:19:14.680707+0000","last_became_peered":"2024-03-29T08:19:14.680707+0000","last_unstale":"2024-03-29T08:19:14.687632+0000","last_undegraded":"2024-03-29T08:19:14.687632+0000","last_fullsized":"2024-03-29T08:19:14.687632+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:04:29.029546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.704743+0000","last_change":"2024-03-29T08:19:14.703892+0000","last_active":"2024-03-29T08:19:14.704743+0000","last_peered":"2024-03-29T08:19:14.704743+0000","last_clean":"2024-03-29T08:19:14.704743+0000","last_became_active":"2024-03-29T08:19:14.701909+0000","last_became_peered":"2024-03-29T08:19:14.701909+0000","last_unstale":"2024-03-29T08:19:14.704743+0000","last_undegraded":"2024-03-29T08:19:14.704743+0000","last_fullsized":"2024-03-29T08:19:14.704743+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:57:15.587273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701010+0000","last_change":"2024-03-29T08:19:14.694312+0000","last_active":"2024-03-29T08:19:14.701010+0000","last_peered":"2024-03-29T08:19:14.701010+0000","last_clean":"2024-03-29T08:19:14.701010+0000","last_became_active":"2024-03-29T08:19:14.694027+0000","last_became_peered":"2024-03-29T08:19:14.694027+0000","last_unstale":"2024-03-29T08:19:14.701010+0000","last_undegraded":"2024-03-29T08:19:14.701010+0000","last_fullsized":"2024-03-29T08:19:14.701010+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:15:31.529776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.704807+0000","last_change":"2024-03-29T08:19:14.703541+0000","last_active":"2024-03-29T08:19:14.704807+0000","last_peered":"2024-03-29T08:19:14.704807+0000","last_clean":"2024-03-29T08:19:14.704807+0000","last_became_active":"2024-03-29T08:19:14.703317+0000","last_became_peered":"2024-03-29T08:19:14.703317+0000","last_unstale":"2024-03-29T08:19:14.704807+0000","last_undegraded":"2024-03-29T08:19:14.704807+0000","last_fullsized":"2024-03-29T08:19:14.704807+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:18:37.057951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687173+0000","last_change":"2024-03-29T08:19:14.684087+0000","last_active":"2024-03-29T08:19:14.687173+0000","last_peered":"2024-03-29T08:19:14.687173+0000","last_clean":"2024-03-29T08:19:14.687173+0000","last_became_active":"2024-03-29T08:19:14.683827+0000","last_became_peered":"2024-03-29T08:19:14.683827+0000","last_unstale":"2024-03-29T08:19:14.687173+0000","last_undegraded":"2024-03-29T08:19:14.687173+0000","last_fullsized":"2024-03-29T08:19:14.687173+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:21.736865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702127+0000","last_change":"2024-03-29T08:19:14.699133+0000","last_active":"2024-03-29T08:19:14.702127+0000","last_peered":"2024-03-29T08:19:14.702127+0000","last_clean":"2024-03-29T08:19:14.702127+0000","last_became_active":"2024-03-29T08:19:14.698579+0000","last_became_peered":"2024-03-29T08:19:14.698579+0000","last_unstale":"2024-03-29T08:19:14.702127+0000","last_undegraded":"2024-03-29T08:19:14.702127+0000","last_fullsized":"2024-03-29T08:19:14.702127+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:00:07.571380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702206+0000","last_change":"2024-03-29T08:19:14.694580+0000","last_active":"2024-03-29T08:19:14.702206+0000","last_peered":"2024-03-29T08:19:14.702206+0000","last_clean":"2024-03-29T08:19:14.702206+0000","last_became_active":"2024-03-29T08:19:14.694305+0000","last_became_peered":"2024-03-29T08:19:14.694305+0000","last_unstale":"2024-03-29T08:19:14.702206+0000","last_undegraded":"2024-03-29T08:19:14.702206+0000","last_fullsized":"2024-03-29T08:19:14.702206+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:19:13.272178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.676292+0000","last_change":"2024-03-29T08:19:14.674438+0000","last_active":"2024-03-29T08:19:14.676292+0000","last_peered":"2024-03-29T08:19:14.676292+0000","last_clean":"2024-03-29T08:19:14.676292+0000","last_became_active":"2024-03-29T08:19:14.674172+0000","last_became_peered":"2024-03-29T08:19:14.674172+0000","last_unstale":"2024-03-29T08:19:14.676292+0000","last_undegraded":"2024-03-29T08:19:14.676292+0000","last_fullsized":"2024-03-29T08:19:14.676292+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:29.147117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687494+0000","last_change":"2024-03-29T08:19:14.681624+0000","last_active":"2024-03-29T08:19:14.687494+0000","last_peered":"2024-03-29T08:19:14.687494+0000","last_clean":"2024-03-29T08:19:14.687494+0000","last_became_active":"2024-03-29T08:19:14.681290+0000","last_became_peered":"2024-03-29T08:19:14.681290+0000","last_unstale":"2024-03-29T08:19:14.687494+0000","last_undegraded":"2024-03-29T08:19:14.687494+0000","last_fullsized":"2024-03-29T08:19:14.687494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:34:59.393546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700295+0000","last_change":"2024-03-29T08:19:14.695305+0000","last_active":"2024-03-29T08:19:14.700295+0000","last_peered":"2024-03-29T08:19:14.700295+0000","last_clean":"2024-03-29T08:19:14.700295+0000","last_became_active":"2024-03-29T08:19:14.694942+0000","last_became_peered":"2024-03-29T08:19:14.694942+0000","last_unstale":"2024-03-29T08:19:14.700295+0000","last_undegraded":"2024-03-29T08:19:14.700295+0000","last_fullsized":"2024-03-29T08:19:14.700295+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:19:41.115142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694349+0000","last_change":"2024-03-29T08:19:14.690840+0000","last_active":"2024-03-29T08:19:14.694349+0000","last_peered":"2024-03-29T08:19:14.694349+0000","last_clean":"2024-03-29T08:19:14.694349+0000","last_became_active":"2024-03-29T08:19:14.689554+0000","last_became_peered":"2024-03-29T08:19:14.689554+0000","last_unstale":"2024-03-29T08:19:14.694349+0000","last_undegraded":"2024-03-29T08:19:14.694349+0000","last_fullsized":"2024-03-29T08:19:14.694349+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:49:04.176971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694388+0000","last_change":"2024-03-29T08:19:14.689362+0000","last_active":"2024-03-29T08:19:14.694388+0000","last_peered":"2024-03-29T08:19:14.694388+0000","last_clean":"2024-03-29T08:19:14.694388+0000","last_became_active":"2024-03-29T08:19:14.689050+0000","last_became_peered":"2024-03-29T08:19:14.689050+0000","last_unstale":"2024-03-29T08:19:14.694388+0000","last_undegraded":"2024-03-29T08:19:14.694388+0000","last_fullsized":"2024-03-29T08:19:14.694388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:20:38.835695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.692494+0000","last_change":"2024-03-29T08:19:14.683957+0000","last_active":"2024-03-29T08:19:14.692494+0000","last_peered":"2024-03-29T08:19:14.692494+0000","last_clean":"2024-03-29T08:19:14.692494+0000","last_became_active":"2024-03-29T08:19:14.683710+0000","last_became_peered":"2024-03-29T08:19:14.683710+0000","last_unstale":"2024-03-29T08:19:14.692494+0000","last_undegraded":"2024-03-29T08:19:14.692494+0000","last_fullsized":"2024-03-29T08:19:14.692494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:11:49.840732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.689299+0000","last_change":"2024-03-29T08:19:14.685648+0000","last_active":"2024-03-29T08:19:14.689299+0000","last_peered":"2024-03-29T08:19:14.689299+0000","last_clean":"2024-03-29T08:19:14.689299+0000","last_became_active":"2024-03-29T08:19:14.685404+0000","last_became_peered":"2024-03-29T08:19:14.685404+0000","last_unstale":"2024-03-29T08:19:14.689299+0000","last_undegraded":"2024-03-29T08:19:14.689299+0000","last_fullsized":"2024-03-29T08:19:14.689299+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:14:08.867120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701342+0000","last_change":"2024-03-29T08:19:14.698732+0000","last_active":"2024-03-29T08:19:14.701342+0000","last_peered":"2024-03-29T08:19:14.701342+0000","last_clean":"2024-03-29T08:19:14.701342+0000","last_became_active":"2024-03-29T08:19:14.698484+0000","last_became_peered":"2024-03-29T08:19:14.698484+0000","last_unstale":"2024-03-29T08:19:14.701342+0000","last_undegraded":"2024-03-29T08:19:14.701342+0000","last_fullsized":"2024-03-29T08:19:14.701342+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:08.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.688435+0000","last_change":"2024-03-29T08:19:14.685174+0000","last_active":"2024-03-29T08:19:14.688435+0000","last_peered":"2024-03-29T08:19:14.688435+0000","last_clean":"2024-03-29T08:19:14.688435+0000","last_became_active":"2024-03-29T08:19:14.684605+0000","last_became_peered":"2024-03-29T08:19:14.684605+0000","last_unstale":"2024-03-29T08:19:14.688435+0000","last_undegraded":"2024-03-29T08:19:14.688435+0000","last_fullsized":"2024-03-29T08:19:14.688435+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:34:59.393546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.699334+0000","last_change":"2024-03-29T08:19:14.698117+0000","last_active":"2024-03-29T08:19:14.699334+0000","last_peered":"2024-03-29T08:19:14.699334+0000","last_clean":"2024-03-29T08:19:14.699334+0000","last_became_active":"2024-03-29T08:19:14.697861+0000","last_became_peered":"2024-03-29T08:19:14.697861+0000","last_unstale":"2024-03-29T08:19:14.699334+0000","last_undegraded":"2024-03-29T08:19:14.699334+0000","last_fullsized":"2024-03-29T08:19:14.699334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:14:08.867120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702337+0000","last_change":"2024-03-29T08:19:14.695925+0000","last_active":"2024-03-29T08:19:14.702337+0000","last_peered":"2024-03-29T08:19:14.702337+0000","last_clean":"2024-03-29T08:19:14.702337+0000","last_became_active":"2024-03-29T08:19:14.695180+0000","last_became_peered":"2024-03-29T08:19:14.695180+0000","last_unstale":"2024-03-29T08:19:14.702337+0000","last_undegraded":"2024-03-29T08:19:14.702337+0000","last_fullsized":"2024-03-29T08:19:14.702337+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:20:02.908287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702375+0000","last_change":"2024-03-29T08:19:14.696275+0000","last_active":"2024-03-29T08:19:14.702375+0000","last_peered":"2024-03-29T08:19:14.702375+0000","last_clean":"2024-03-29T08:19:14.702375+0000","last_became_active":"2024-03-29T08:19:14.696065+0000","last_became_peered":"2024-03-29T08:19:14.696065+0000","last_unstale":"2024-03-29T08:19:14.702375+0000","last_undegraded":"2024-03-29T08:19:14.702375+0000","last_fullsized":"2024-03-29T08:19:14.702375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:00:46.960177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:19:14.252791+0000","last_change":"2024-03-29T08:19:14.252791+0000","last_active":"2024-03-29T08:19:14.252791+0000","last_peered":"2024-03-29T08:19:14.252791+0000","last_clean":"2024-03-29T08:19:14.252791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:14.252791+0000","last_undegraded":"2024-03-29T08:19:14.252791+0000","last_fullsized":"2024-03-29T08:19:14.252791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.693704+0000","last_change":"2024-03-29T08:19:14.684317+0000","last_active":"2024-03-29T08:19:14.693704+0000","last_peered":"2024-03-29T08:19:14.693704+0000","last_clean":"2024-03-29T08:19:14.693704+0000","last_became_active":"2024-03-29T08:19:14.683125+0000","last_became_peered":"2024-03-29T08:19:14.683125+0000","last_unstale":"2024-03-29T08:19:14.693704+0000","last_undegraded":"2024-03-29T08:19:14.693704+0000","last_fullsized":"2024-03-29T08:19:14.693704+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:52:54.511385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702775+0000","last_change":"2024-03-29T08:19:14.701650+0000","last_active":"2024-03-29T08:19:14.702775+0000","last_peered":"2024-03-29T08:19:14.702775+0000","last_clean":"2024-03-29T08:19:14.702775+0000","last_became_active":"2024-03-29T08:19:14.697686+0000","last_became_peered":"2024-03-29T08:19:14.697686+0000","last_unstale":"2024-03-29T08:19:14.702775+0000","last_undegraded":"2024-03-29T08:19:14.702775+0000","last_fullsized":"2024-03-29T08:19:14.702775+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:10:23.783843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687174+0000","last_change":"2024-03-29T08:19:14.684872+0000","last_active":"2024-03-29T08:19:14.687174+0000","last_peered":"2024-03-29T08:19:14.687174+0000","last_clean":"2024-03-29T08:19:14.687174+0000","last_became_active":"2024-03-29T08:19:14.684622+0000","last_became_peered":"2024-03-29T08:19:14.684622+0000","last_unstale":"2024-03-29T08:19:14.687174+0000","last_undegraded":"2024-03-29T08:19:14.687174+0000","last_fullsized":"2024-03-29T08:19:14.687174+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:00:07.571380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687689+0000","last_change":"2024-03-29T08:19:14.681150+0000","last_active":"2024-03-29T08:19:14.687689+0000","last_peered":"2024-03-29T08:19:14.687689+0000","last_clean":"2024-03-29T08:19:14.687689+0000","last_became_active":"2024-03-29T08:19:14.680888+0000","last_became_peered":"2024-03-29T08:19:14.680888+0000","last_unstale":"2024-03-29T08:19:14.687689+0000","last_undegraded":"2024-03-29T08:19:14.687689+0000","last_fullsized":"2024-03-29T08:19:14.687689+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:10:23.783843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700291+0000","last_change":"2024-03-29T08:19:14.698385+0000","last_active":"2024-03-29T08:19:14.700291+0000","last_peered":"2024-03-29T08:19:14.700291+0000","last_clean":"2024-03-29T08:19:14.700291+0000","last_became_active":"2024-03-29T08:19:14.698119+0000","last_became_peered":"2024-03-29T08:19:14.698119+0000","last_unstale":"2024-03-29T08:19:14.700291+0000","last_undegraded":"2024-03-29T08:19:14.700291+0000","last_fullsized":"2024-03-29T08:19:14.700291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:03:28.287471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.692119+0000","last_change":"2024-03-29T08:19:14.686891+0000","last_active":"2024-03-29T08:19:14.692119+0000","last_peered":"2024-03-29T08:19:14.692119+0000","last_clean":"2024-03-29T08:19:14.692119+0000","last_became_active":"2024-03-29T08:19:14.686615+0000","last_became_peered":"2024-03-29T08:19:14.686615+0000","last_unstale":"2024-03-29T08:19:14.692119+0000","last_undegraded":"2024-03-29T08:19:14.692119+0000","last_fullsized":"2024-03-29T08:19:14.692119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:10:23.783843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701442+0000","last_change":"2024-03-29T08:19:14.278073+0000","last_active":"2024-03-29T08:19:14.701442+0000","last_peered":"2024-03-29T08:19:14.701442+0000","last_clean":"2024-03-29T08:19:14.701442+0000","last_became_active":"2024-03-29T08:19:14.277778+0000","last_became_peered":"2024-03-29T08:19:14.277778+0000","last_unstale":"2024-03-29T08:19:14.701442+0000","last_undegraded":"2024-03-29T08:19:14.701442+0000","last_fullsized":"2024-03-29T08:19:14.701442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:36:08.787710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687978+0000","last_change":"2024-03-29T08:19:14.682096+0000","last_active":"2024-03-29T08:19:14.687978+0000","last_peered":"2024-03-29T08:19:14.687978+0000","last_clean":"2024-03-29T08:19:14.687978+0000","last_became_active":"2024-03-29T08:19:14.681779+0000","last_became_peered":"2024-03-29T08:19:14.681779+0000","last_unstale":"2024-03-29T08:19:14.687978+0000","last_undegraded":"2024-03-29T08:19:14.687978+0000","last_fullsized":"2024-03-29T08:19:14.687978+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:52:20.065176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.703320+0000","last_change":"2024-03-29T08:19:14.281218+0000","last_active":"2024-03-29T08:19:14.703320+0000","last_peered":"2024-03-29T08:19:14.703320+0000","last_clean":"2024-03-29T08:19:14.703320+0000","last_became_active":"2024-03-29T08:19:14.280883+0000","last_became_peered":"2024-03-29T08:19:14.280883+0000","last_unstale":"2024-03-29T08:19:14.703320+0000","last_undegraded":"2024-03-29T08:19:14.703320+0000","last_fullsized":"2024-03-29T08:19:14.703320+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:14:22.394882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701140+0000","last_change":"2024-03-29T08:19:14.698993+0000","last_active":"2024-03-29T08:19:14.701140+0000","last_peered":"2024-03-29T08:19:14.701140+0000","last_clean":"2024-03-29T08:19:14.701140+0000","last_became_active":"2024-03-29T08:19:14.698757+0000","last_became_peered":"2024-03-29T08:19:14.698757+0000","last_unstale":"2024-03-29T08:19:14.701140+0000","last_undegraded":"2024-03-29T08:19:14.701140+0000","last_fullsized":"2024-03-29T08:19:14.701140+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:34:59.393546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.692376+0000","last_change":"2024-03-29T08:19:14.283589+0000","last_active":"2024-03-29T08:19:14.692376+0000","last_peered":"2024-03-29T08:19:14.692376+0000","last_clean":"2024-03-29T08:19:14.692376+0000","last_became_active":"2024-03-29T08:19:14.282933+0000","last_became_peered":"2024-03-29T08:19:14.282933+0000","last_unstale":"2024-03-29T08:19:14.692376+0000","last_undegraded":"2024-03-29T08:19:14.692376+0000","last_fullsized":"2024-03-29T08:19:14.692376+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:16:08.725000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702696+0000","last_change":"2024-03-29T08:19:14.696813+0000","last_active":"2024-03-29T08:19:14.702696+0000","last_peered":"2024-03-29T08:19:14.702696+0000","last_clean":"2024-03-29T08:19:14.702696+0000","last_became_active":"2024-03-29T08:19:14.696519+0000","last_became_peered":"2024-03-29T08:19:14.696519+0000","last_unstale":"2024-03-29T08:19:14.702696+0000","last_undegraded":"2024-03-29T08:19:14.702696+0000","last_fullsized":"2024-03-29T08:19:14.702696+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:46:59.856286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705886+0000","last_change":"2024-03-29T08:19:14.279768+0000","last_active":"2024-03-29T08:19:14.705886+0000","last_peered":"2024-03-29T08:19:14.705886+0000","last_clean":"2024-03-29T08:19:14.705886+0000","last_became_active":"2024-03-29T08:19:14.279493+0000","last_became_peered":"2024-03-29T08:19:14.279493+0000","last_unstale":"2024-03-29T08:19:14.705886+0000","last_undegraded":"2024-03-29T08:19:14.705886+0000","last_fullsized":"2024-03-29T08:19:14.705886+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:11:12.357829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.703025+0000","last_change":"2024-03-29T08:19:14.702417+0000","last_active":"2024-03-29T08:19:14.703025+0000","last_peered":"2024-03-29T08:19:14.703025+0000","last_clean":"2024-03-29T08:19:14.703025+0000","last_became_active":"2024-03-29T08:19:14.701927+0000","last_became_peered":"2024-03-29T08:19:14.701927+0000","last_unstale":"2024-03-29T08:19:14.703025+0000","last_undegraded":"2024-03-29T08:19:14.703025+0000","last_fullsized":"2024-03-29T08:19:14.703025+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:30:19.536345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701280+0000","last_change":"2024-03-29T08:19:14.279062+0000","last_active":"2024-03-29T08:19:14.701280+0000","last_peered":"2024-03-29T08:19:14.701280+0000","last_clean":"2024-03-29T08:19:14.701280+0000","last_became_active":"2024-03-29T08:19:14.278726+0000","last_became_peered":"2024-03-29T08:19:14.278726+0000","last_unstale":"2024-03-29T08:19:14.701280+0000","last_undegraded":"2024-03-29T08:19:14.701280+0000","last_fullsized":"2024-03-29T08:19:14.701280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:35.048072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687660+0000","last_change":"2024-03-29T08:19:14.679888+0000","last_active":"2024-03-29T08:19:14.687660+0000","last_peered":"2024-03-29T08:19:14.687660+0000","last_clean":"2024-03-29T08:19:14.687660+0000","last_became_active":"2024-03-29T08:19:14.679674+0000","last_became_peered":"2024-03-29T08:19:14.679674+0000","last_unstale":"2024-03-29T08:19:14.687660+0000","last_undegraded":"2024-03-29T08:19:14.687660+0000","last_fullsized":"2024-03-29T08:19:14.687660+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:55:24.587078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701177+0000","last_change":"2024-03-29T08:19:14.280371+0000","last_active":"2024-03-29T08:19:14.701177+0000","last_peered":"2024-03-29T08:19:14.701177+0000","last_clean":"2024-03-29T08:19:14.701177+0000","last_became_active":"2024-03-29T08:19:14.280045+0000","last_became_peered":"2024-03-29T08:19:14.280045+0000","last_unstale":"2024-03-29T08:19:14.701177+0000","last_undegraded":"2024-03-29T08:19:14.701177+0000","last_fullsized":"2024-03-29T08:19:14.701177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:20.730956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.681942+0000","last_change":"2024-03-29T08:19:14.677276+0000","last_active":"2024-03-29T08:19:14.681942+0000","last_peered":"2024-03-29T08:19:14.681942+0000","last_clean":"2024-03-29T08:19:14.681942+0000","last_became_active":"2024-03-29T08:19:14.677049+0000","last_became_peered":"2024-03-29T08:19:14.677049+0000","last_unstale":"2024-03-29T08:19:14.681942+0000","last_undegraded":"2024-03-29T08:19:14.681942+0000","last_fullsized":"2024-03-29T08:19:14.681942+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:27.668682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.678961+0000","last_change":"2024-03-29T08:19:14.285270+0000","last_active":"2024-03-29T08:19:14.678961+0000","last_peered":"2024-03-29T08:19:14.678961+0000","last_clean":"2024-03-29T08:19:14.678961+0000","last_became_active":"2024-03-29T08:19:14.283882+0000","last_became_peered":"2024-03-29T08:19:14.283882+0000","last_unstale":"2024-03-29T08:19:14.678961+0000","last_undegraded":"2024-03-29T08:19:14.678961+0000","last_fullsized":"2024-03-29T08:19:14.678961+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:09:33.409328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702457+0000","last_change":"2024-03-29T08:19:14.696486+0000","last_active":"2024-03-29T08:19:14.702457+0000","last_peered":"2024-03-29T08:19:14.702457+0000","last_clean":"2024-03-29T08:19:14.702457+0000","last_became_active":"2024-03-29T08:19:14.696236+0000","last_became_peered":"2024-03-29T08:19:14.696236+0000","last_unstale":"2024-03-29T08:19:14.702457+0000","last_undegraded":"2024-03-29T08:19:14.702457+0000","last_fullsized":"2024-03-29T08:19:14.702457+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:08:58.521804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.671922+0000","last_change":"2024-03-29T08:19:14.275451+0000","last_active":"2024-03-29T08:19:14.671922+0000","last_peered":"2024-03-29T08:19:14.671922+0000","last_clean":"2024-03-29T08:19:14.671922+0000","last_became_active":"2024-03-29T08:19:14.275148+0000","last_became_peered":"2024-03-29T08:19:14.275148+0000","last_unstale":"2024-03-29T08:19:14.671922+0000","last_undegraded":"2024-03-29T08:19:14.671922+0000","last_fullsized":"2024-03-29T08:19:14.671922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:57:14.581364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694290+0000","last_change":"2024-03-29T08:19:14.690396+0000","last_active":"2024-03-29T08:19:14.694290+0000","last_peered":"2024-03-29T08:19:14.694290+0000","last_clean":"2024-03-29T08:19:14.694290+0000","last_became_active":"2024-03-29T08:19:14.690106+0000","last_became_peered":"2024-03-29T08:19:14.690106+0000","last_unstale":"2024-03-29T08:19:14.694290+0000","last_undegraded":"2024-03-29T08:19:14.694290+0000","last_fullsized":"2024-03-29T08:19:14.694290+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:54:09.222693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-29T08:19:13.260506+0000","last_change":"2024-03-29T08:19:13.260506+0000","last_active":"2024-03-29T08:19:13.246882+0000","last_peered":"2024-03-29T08:19:13.246882+0000","last_clean":"2024-03-29T08:19:13.246882+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:13.260506+0000","last_undegraded":"2024-03-29T08:19:13.260506+0000","last_fullsized":"2024-03-29T08:19:13.260506+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:58:40.235616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700952+0000","last_change":"2024-03-29T08:19:14.694063+0000","last_active":"2024-03-29T08:19:14.700952+0000","last_peered":"2024-03-29T08:19:14.700952+0000","last_clean":"2024-03-29T08:19:14.700952+0000","last_became_active":"2024-03-29T08:19:14.693817+0000","last_became_peered":"2024-03-29T08:19:14.693817+0000","last_unstale":"2024-03-29T08:19:14.700952+0000","last_undegraded":"2024-03-29T08:19:14.700952+0000","last_fullsized":"2024-03-29T08:19:14.700952+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:30:57.036017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-29T08:19:13.261393+0000","last_change":"2024-03-29T08:19:13.261393+0000","last_active":"2024-03-29T08:19:13.246882+0000","last_peered":"2024-03-29T08:19:13.246882+0000","last_clean":"2024-03-29T08:19:13.246882+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:13.261393+0000","last_undegraded":"2024-03-29T08:19:13.261393+0000","last_fullsized":"2024-03-29T08:19:13.261393+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:36:01.837144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705548+0000","last_change":"2024-03-29T08:19:14.698954+0000","last_active":"2024-03-29T08:19:14.705548+0000","last_peered":"2024-03-29T08:19:14.705548+0000","last_clean":"2024-03-29T08:19:14.705548+0000","last_became_active":"2024-03-29T08:19:14.698317+0000","last_became_peered":"2024-03-29T08:19:14.698317+0000","last_unstale":"2024-03-29T08:19:14.705548+0000","last_undegraded":"2024-03-29T08:19:14.705548+0000","last_fullsized":"2024-03-29T08:19:14.705548+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:16:39.883087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700954+0000","last_change":"2024-03-29T08:19:14.285068+0000","last_active":"2024-03-29T08:19:14.700954+0000","last_peered":"2024-03-29T08:19:14.700954+0000","last_clean":"2024-03-29T08:19:14.700954+0000","last_became_active":"2024-03-29T08:19:14.284830+0000","last_became_peered":"2024-03-29T08:19:14.284830+0000","last_unstale":"2024-03-29T08:19:14.700954+0000","last_undegraded":"2024-03-29T08:19:14.700954+0000","last_fullsized":"2024-03-29T08:19:14.700954+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:16:38.877179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705493+0000","last_change":"2024-03-29T08:19:14.704710+0000","last_active":"2024-03-29T08:19:14.705493+0000","last_peered":"2024-03-29T08:19:14.705493+0000","last_clean":"2024-03-29T08:19:14.705493+0000","last_became_active":"2024-03-29T08:19:14.704396+0000","last_became_peered":"2024-03-29T08:19:14.704396+0000","last_unstale":"2024-03-29T08:19:14.705493+0000","last_undegraded":"2024-03-29T08:19:14.705493+0000","last_fullsized":"2024-03-29T08:19:14.705493+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:24.176397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.706659+0000","last_change":"2024-03-29T08:19:14.280954+0000","last_active":"2024-03-29T08:19:14.706659+0000","last_peered":"2024-03-29T08:19:14.706659+0000","last_clean":"2024-03-29T08:19:14.706659+0000","last_became_active":"2024-03-29T08:19:14.280392+0000","last_became_peered":"2024-03-29T08:19:14.280392+0000","last_unstale":"2024-03-29T08:19:14.706659+0000","last_undegraded":"2024-03-29T08:19:14.706659+0000","last_fullsized":"2024-03-29T08:19:14.706659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:26.662774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.688513+0000","last_change":"2024-03-29T08:19:14.685902+0000","last_active":"2024-03-29T08:19:14.688513+0000","last_peered":"2024-03-29T08:19:14.688513+0000","last_clean":"2024-03-29T08:19:14.688513+0000","last_became_active":"2024-03-29T08:19:14.685705+0000","last_became_peered":"2024-03-29T08:19:14.685705+0000","last_unstale":"2024-03-29T08:19:14.688513+0000","last_undegraded":"2024-03-29T08:19:14.688513+0000","last_fullsized":"2024-03-29T08:19:14.688513+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:08.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.695832+0000","last_change":"2024-03-29T08:19:14.272012+0000","last_active":"2024-03-29T08:19:14.695832+0000","last_peered":"2024-03-29T08:19:14.695832+0000","last_clean":"2024-03-29T08:19:14.695832+0000","last_became_active":"2024-03-29T08:19:14.271736+0000","last_became_peered":"2024-03-29T08:19:14.271736+0000","last_unstale":"2024-03-29T08:19:14.695832+0000","last_undegraded":"2024-03-29T08:19:14.695832+0000","last_fullsized":"2024-03-29T08:19:14.695832+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:33.064793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.696404+0000","last_change":"2024-03-29T08:19:14.690513+0000","last_active":"2024-03-29T08:19:14.696404+0000","last_peered":"2024-03-29T08:19:14.696404+0000","last_clean":"2024-03-29T08:19:14.696404+0000","last_became_active":"2024-03-29T08:19:14.690235+0000","last_became_peered":"2024-03-29T08:19:14.690235+0000","last_unstale":"2024-03-29T08:19:14.696404+0000","last_undegraded":"2024-03-29T08:19:14.696404+0000","last_fullsized":"2024-03-29T08:19:14.696404+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:01:53.702260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.693539+0000","last_change":"2024-03-29T08:19:14.274775+0000","last_active":"2024-03-29T08:19:14.693539+0000","last_peered":"2024-03-29T08:19:14.693539+0000","last_clean":"2024-03-29T08:19:14.693539+0000","last_became_active":"2024-03-29T08:19:14.274502+0000","last_became_peered":"2024-03-29T08:19:14.274502+0000","last_unstale":"2024-03-29T08:19:14.693539+0000","last_undegraded":"2024-03-29T08:19:14.693539+0000","last_fullsized":"2024-03-29T08:19:14.693539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:20:37.829787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.681902+0000","last_change":"2024-03-29T08:19:14.675914+0000","last_active":"2024-03-29T08:19:14.681902+0000","last_peered":"2024-03-29T08:19:14.681902+0000","last_clean":"2024-03-29T08:19:14.681902+0000","last_became_active":"2024-03-29T08:19:14.675564+0000","last_became_peered":"2024-03-29T08:19:14.675564+0000","last_unstale":"2024-03-29T08:19:14.681902+0000","last_undegraded":"2024-03-29T08:19:14.681902+0000","last_fullsized":"2024-03-29T08:19:14.681902+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:36:09.793618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.682997+0000","last_change":"2024-03-29T08:19:14.281362+0000","last_active":"2024-03-29T08:19:14.682997+0000","last_peered":"2024-03-29T08:19:14.682997+0000","last_clean":"2024-03-29T08:19:14.682997+0000","last_became_active":"2024-03-29T08:19:14.281111+0000","last_became_peered":"2024-03-29T08:19:14.281111+0000","last_unstale":"2024-03-29T08:19:14.682997+0000","last_undegraded":"2024-03-29T08:19:14.682997+0000","last_fullsized":"2024-03-29T08:19:14.682997+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:26.870833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700427+0000","last_change":"2024-03-29T08:19:14.698963+0000","last_active":"2024-03-29T08:19:14.700427+0000","last_peered":"2024-03-29T08:19:14.700427+0000","last_clean":"2024-03-29T08:19:14.700427+0000","last_became_active":"2024-03-29T08:19:14.698502+0000","last_became_peered":"2024-03-29T08:19:14.698502+0000","last_unstale":"2024-03-29T08:19:14.700427+0000","last_undegraded":"2024-03-29T08:19:14.700427+0000","last_fullsized":"2024-03-29T08:19:14.700427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:21:27.491192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700579+0000","last_change":"2024-03-29T08:19:14.284917+0000","last_active":"2024-03-29T08:19:14.700579+0000","last_peered":"2024-03-29T08:19:14.700579+0000","last_clean":"2024-03-29T08:19:14.700579+0000","last_became_active":"2024-03-29T08:19:14.284667+0000","last_became_peered":"2024-03-29T08:19:14.284667+0000","last_unstale":"2024-03-29T08:19:14.700579+0000","last_undegraded":"2024-03-29T08:19:14.700579+0000","last_fullsized":"2024-03-29T08:19:14.700579+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:42.739141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.704840+0000","last_change":"2024-03-29T08:19:14.703652+0000","last_active":"2024-03-29T08:19:14.704840+0000","last_peered":"2024-03-29T08:19:14.704840+0000","last_clean":"2024-03-29T08:19:14.704840+0000","last_became_active":"2024-03-29T08:19:14.703386+0000","last_became_peered":"2024-03-29T08:19:14.703386+0000","last_unstale":"2024-03-29T08:19:14.704840+0000","last_undegraded":"2024-03-29T08:19:14.704840+0000","last_fullsized":"2024-03-29T08:19:14.704840+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:55:15.376664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702612+0000","last_change":"2024-03-29T08:19:14.287908+0000","last_active":"2024-03-29T08:19:14.702612+0000","last_peered":"2024-03-29T08:19:14.702612+0000","last_clean":"2024-03-29T08:19:14.702612+0000","last_became_active":"2024-03-29T08:19:14.287562+0000","last_became_peered":"2024-03-29T08:19:14.287562+0000","last_unstale":"2024-03-29T08:19:14.702612+0000","last_undegraded":"2024-03-29T08:19:14.702612+0000","last_fullsized":"2024-03-29T08:19:14.702612+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:02:28.529095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687362+0000","last_change":"2024-03-29T08:19:14.682493+0000","last_active":"2024-03-29T08:19:14.687362+0000","last_peered":"2024-03-29T08:19:14.687362+0000","last_clean":"2024-03-29T08:19:14.687362+0000","last_became_active":"2024-03-29T08:19:14.678342+0000","last_became_peered":"2024-03-29T08:19:14.678342+0000","last_unstale":"2024-03-29T08:19:14.687362+0000","last_undegraded":"2024-03-29T08:19:14.687362+0000","last_fullsized":"2024-03-29T08:19:14.687362+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:36.053980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.695684+0000","last_change":"2024-03-29T08:19:14.272763+0000","last_active":"2024-03-29T08:19:14.695684+0000","last_peered":"2024-03-29T08:19:14.695684+0000","last_clean":"2024-03-29T08:19:14.695684+0000","last_became_active":"2024-03-29T08:19:14.272450+0000","last_became_peered":"2024-03-29T08:19:14.272450+0000","last_unstale":"2024-03-29T08:19:14.695684+0000","last_undegraded":"2024-03-29T08:19:14.695684+0000","last_fullsized":"2024-03-29T08:19:14.695684+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:19.581981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:19:14.252791+0000","last_change":"2024-03-29T08:19:14.252791+0000","last_active":"2024-03-29T08:19:14.252791+0000","last_peered":"2024-03-29T08:19:14.252791+0000","last_clean":"2024-03-29T08:19:14.252791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:14.252791+0000","last_undegraded":"2024-03-29T08:19:14.252791+0000","last_fullsized":"2024-03-29T08:19:14.252791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701079+0000","last_change":"2024-03-29T08:19:14.287716+0000","last_active":"2024-03-29T08:19:14.701079+0000","last_peered":"2024-03-29T08:19:14.701079+0000","last_clean":"2024-03-29T08:19:14.701079+0000","last_became_active":"2024-03-29T08:19:14.287437+0000","last_became_peered":"2024-03-29T08:19:14.287437+0000","last_unstale":"2024-03-29T08:19:14.701079+0000","last_undegraded":"2024-03-29T08:19:14.701079+0000","last_fullsized":"2024-03-29T08:19:14.701079+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:08:57.515896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694406+0000","last_change":"2024-03-29T08:19:14.689798+0000","last_active":"2024-03-29T08:19:14.694406+0000","last_peered":"2024-03-29T08:19:14.694406+0000","last_clean":"2024-03-29T08:19:14.694406+0000","last_became_active":"2024-03-29T08:19:14.689479+0000","last_became_peered":"2024-03-29T08:19:14.689479+0000","last_unstale":"2024-03-29T08:19:14.694406+0000","last_undegraded":"2024-03-29T08:19:14.694406+0000","last_fullsized":"2024-03-29T08:19:14.694406+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:20:03.888899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.667785+0000","last_change":"2024-03-29T08:19:14.281935+0000","last_active":"2024-03-29T08:19:14.667785+0000","last_peered":"2024-03-29T08:19:14.667785+0000","last_clean":"2024-03-29T08:19:14.667785+0000","last_became_active":"2024-03-29T08:19:14.281670+0000","last_became_peered":"2024-03-29T08:19:14.281670+0000","last_unstale":"2024-03-29T08:19:14.667785+0000","last_undegraded":"2024-03-29T08:19:14.667785+0000","last_fullsized":"2024-03-29T08:19:14.667785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:49:03.171063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.698797+0000","last_change":"2024-03-29T08:19:14.689677+0000","last_active":"2024-03-29T08:19:14.698797+0000","last_peered":"2024-03-29T08:19:14.698797+0000","last_clean":"2024-03-29T08:19:14.698797+0000","last_became_active":"2024-03-29T08:19:14.689343+0000","last_became_peered":"2024-03-29T08:19:14.689343+0000","last_unstale":"2024-03-29T08:19:14.698797+0000","last_undegraded":"2024-03-29T08:19:14.698797+0000","last_fullsized":"2024-03-29T08:19:14.698797+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:55.247391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-29T08:19:13.260439+0000","last_change":"2024-03-29T08:19:13.260439+0000","last_active":"2024-03-29T08:19:13.246882+0000","last_peered":"2024-03-29T08:19:13.246882+0000","last_clean":"2024-03-29T08:19:13.246882+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:13.260439+0000","last_undegraded":"2024-03-29T08:19:13.260439+0000","last_fullsized":"2024-03-29T08:19:13.260439+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:27:45.403801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:19:14.252791+0000","last_change":"2024-03-29T08:19:14.252791+0000","last_active":"2024-03-29T08:19:14.252791+0000","last_peered":"2024-03-29T08:19:14.252791+0000","last_clean":"2024-03-29T08:19:14.252791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:14.252791+0000","last_undegraded":"2024-03-29T08:19:14.252791+0000","last_fullsized":"2024-03-29T08:19:14.252791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.673192+0000","last_change":"2024-03-29T08:19:14.272023+0000","last_active":"2024-03-29T08:19:14.673192+0000","last_peered":"2024-03-29T08:19:14.673192+0000","last_clean":"2024-03-29T08:19:14.673192+0000","last_became_active":"2024-03-29T08:19:14.271722+0000","last_became_peered":"2024-03-29T08:19:14.271722+0000","last_unstale":"2024-03-29T08:19:14.673192+0000","last_undegraded":"2024-03-29T08:19:14.673192+0000","last_fullsized":"2024-03-29T08:19:14.673192+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:26.662774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.699233+0000","last_change":"2024-03-29T08:19:14.694537+0000","last_active":"2024-03-29T08:19:14.699233+0000","last_peered":"2024-03-29T08:19:14.699233+0000","last_clean":"2024-03-29T08:19:14.699233+0000","last_became_active":"2024-03-29T08:19:14.694296+0000","last_became_peered":"2024-03-29T08:19:14.694296+0000","last_unstale":"2024-03-29T08:19:14.699233+0000","last_undegraded":"2024-03-29T08:19:14.699233+0000","last_fullsized":"2024-03-29T08:19:14.699233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:37:16.877010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.673618+0000","last_change":"2024-03-29T08:19:14.285590+0000","last_active":"2024-03-29T08:19:14.673618+0000","last_peered":"2024-03-29T08:19:14.673618+0000","last_clean":"2024-03-29T08:19:14.673618+0000","last_became_active":"2024-03-29T08:19:14.285211+0000","last_became_peered":"2024-03-29T08:19:14.285211+0000","last_unstale":"2024-03-29T08:19:14.673618+0000","last_undegraded":"2024-03-29T08:19:14.673618+0000","last_fullsized":"2024-03-29T08:19:14.673618+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:52:20.153715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.682877+0000","last_change":"2024-03-29T08:19:14.679560+0000","last_active":"2024-03-29T08:19:14.682877+0000","last_peered":"2024-03-29T08:19:14.682877+0000","last_clean":"2024-03-29T08:19:14.682877+0000","last_became_active":"2024-03-29T08:19:14.677544+0000","last_became_peered":"2024-03-29T08:19:14.677544+0000","last_unstale":"2024-03-29T08:19:14.682877+0000","last_undegraded":"2024-03-29T08:19:14.682877+0000","last_fullsized":"2024-03-29T08:19:14.682877+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:34.410446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.683328+0000","last_change":"2024-03-29T08:19:14.281813+0000","last_active":"2024-03-29T08:19:14.683328+0000","last_peered":"2024-03-29T08:19:14.683328+0000","last_clean":"2024-03-29T08:19:14.683328+0000","last_became_active":"2024-03-29T08:19:14.281356+0000","last_became_peered":"2024-03-29T08:19:14.281356+0000","last_unstale":"2024-03-29T08:19:14.683328+0000","last_undegraded":"2024-03-29T08:19:14.683328+0000","last_fullsized":"2024-03-29T08:19:14.683328+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:04:28.023637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.697297+0000","last_change":"2024-03-29T08:19:14.691637+0000","last_active":"2024-03-29T08:19:14.697297+0000","last_peered":"2024-03-29T08:19:14.697297+0000","last_clean":"2024-03-29T08:19:14.697297+0000","last_became_active":"2024-03-29T08:19:14.691391+0000","last_became_peered":"2024-03-29T08:19:14.691391+0000","last_unstale":"2024-03-29T08:19:14.697297+0000","last_undegraded":"2024-03-29T08:19:14.697297+0000","last_fullsized":"2024-03-29T08:19:14.697297+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:11:13.363737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.697667+0000","last_change":"2024-03-29T08:19:14.286350+0000","last_active":"2024-03-29T08:19:14.697667+0000","last_peered":"2024-03-29T08:19:14.697667+0000","last_clean":"2024-03-29T08:19:14.697667+0000","last_became_active":"2024-03-29T08:19:14.286055+0000","last_became_peered":"2024-03-29T08:19:14.286055+0000","last_unstale":"2024-03-29T08:19:14.697667+0000","last_undegraded":"2024-03-29T08:19:14.697667+0000","last_fullsized":"2024-03-29T08:19:14.697667+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:35.048072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700275+0000","last_change":"2024-03-29T08:19:14.694685+0000","last_active":"2024-03-29T08:19:14.700275+0000","last_peered":"2024-03-29T08:19:14.700275+0000","last_clean":"2024-03-29T08:19:14.700275+0000","last_became_active":"2024-03-29T08:19:14.693892+0000","last_became_peered":"2024-03-29T08:19:14.693892+0000","last_unstale":"2024-03-29T08:19:14.700275+0000","last_undegraded":"2024-03-29T08:19:14.700275+0000","last_fullsized":"2024-03-29T08:19:14.700275+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:02:39.788723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-29T08:19:13.260969+0000","last_change":"2024-03-29T08:19:13.260969+0000","last_active":"2024-03-29T08:19:13.246882+0000","last_peered":"2024-03-29T08:19:13.246882+0000","last_clean":"2024-03-29T08:19:13.246882+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:13.260969+0000","last_undegraded":"2024-03-29T08:19:13.260969+0000","last_fullsized":"2024-03-29T08:19:13.260969+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:59:27.620393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705394+0000","last_change":"2024-03-29T08:19:14.701490+0000","last_active":"2024-03-29T08:19:14.705394+0000","last_peered":"2024-03-29T08:19:14.705394+0000","last_clean":"2024-03-29T08:19:14.705394+0000","last_became_active":"2024-03-29T08:19:14.701146+0000","last_became_peered":"2024-03-29T08:19:14.701146+0000","last_unstale":"2024-03-29T08:19:14.705394+0000","last_undegraded":"2024-03-29T08:19:14.705394+0000","last_fullsized":"2024-03-29T08:19:14.705394+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:32:52.080327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.676163+0000","last_change":"2024-03-29T08:19:14.279625+0000","last_active":"2024-03-29T08:19:14.676163+0000","last_peered":"2024-03-29T08:19:14.676163+0000","last_clean":"2024-03-29T08:19:14.676163+0000","last_became_active":"2024-03-29T08:19:14.279364+0000","last_became_peered":"2024-03-29T08:19:14.279364+0000","last_unstale":"2024-03-29T08:19:14.676163+0000","last_undegraded":"2024-03-29T08:19:14.676163+0000","last_fullsized":"2024-03-29T08:19:14.676163+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:20:02.882991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.685821+0000","last_change":"2024-03-29T08:19:14.679991+0000","last_active":"2024-03-29T08:19:14.685821+0000","last_peered":"2024-03-29T08:19:14.685821+0000","last_clean":"2024-03-29T08:19:14.685821+0000","last_became_active":"2024-03-29T08:19:14.679701+0000","last_became_peered":"2024-03-29T08:19:14.679701+0000","last_unstale":"2024-03-29T08:19:14.685821+0000","last_undegraded":"2024-03-29T08:19:14.685821+0000","last_fullsized":"2024-03-29T08:19:14.685821+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:08.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.675477+0000","last_change":"2024-03-29T08:19:14.284652+0000","last_active":"2024-03-29T08:19:14.675477+0000","last_peered":"2024-03-29T08:19:14.675477+0000","last_clean":"2024-03-29T08:19:14.675477+0000","last_became_active":"2024-03-29T08:19:14.284376+0000","last_became_peered":"2024-03-29T08:19:14.284376+0000","last_unstale":"2024-03-29T08:19:14.675477+0000","last_undegraded":"2024-03-29T08:19:14.675477+0000","last_fullsized":"2024-03-29T08:19:14.675477+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:20.730956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701325+0000","last_change":"2024-03-29T08:19:14.693198+0000","last_active":"2024-03-29T08:19:14.701325+0000","last_peered":"2024-03-29T08:19:14.701325+0000","last_clean":"2024-03-29T08:19:14.701325+0000","last_became_active":"2024-03-29T08:19:14.692849+0000","last_became_peered":"2024-03-29T08:19:14.692849+0000","last_unstale":"2024-03-29T08:19:14.701325+0000","last_undegraded":"2024-03-29T08:19:14.701325+0000","last_fullsized":"2024-03-29T08:19:14.701325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:46.218186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.676556+0000","last_change":"2024-03-29T08:19:14.284484+0000","last_active":"2024-03-29T08:19:14.676556+0000","last_peered":"2024-03-29T08:19:14.676556+0000","last_clean":"2024-03-29T08:19:14.676556+0000","last_became_active":"2024-03-29T08:19:14.284145+0000","last_became_peered":"2024-03-29T08:19:14.284145+0000","last_unstale":"2024-03-29T08:19:14.676556+0000","last_undegraded":"2024-03-29T08:19:14.676556+0000","last_fullsized":"2024-03-29T08:19:14.676556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:33.404538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702578+0000","last_change":"2024-03-29T08:19:14.695365+0000","last_active":"2024-03-29T08:19:14.702578+0000","last_peered":"2024-03-29T08:19:14.702578+0000","last_clean":"2024-03-29T08:19:14.702578+0000","last_became_active":"2024-03-29T08:19:14.695111+0000","last_became_peered":"2024-03-29T08:19:14.695111+0000","last_unstale":"2024-03-29T08:19:14.702578+0000","last_undegraded":"2024-03-29T08:19:14.702578+0000","last_fullsized":"2024-03-29T08:19:14.702578+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:15.077172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.672171+0000","last_change":"2024-03-29T08:19:14.270630+0000","last_active":"2024-03-29T08:19:14.672171+0000","last_peered":"2024-03-29T08:19:14.672171+0000","last_clean":"2024-03-29T08:19:14.672171+0000","last_became_active":"2024-03-29T08:19:14.270379+0000","last_became_peered":"2024-03-29T08:19:14.270379+0000","last_unstale":"2024-03-29T08:19:14.672171+0000","last_undegraded":"2024-03-29T08:19:14.672171+0000","last_fullsized":"2024-03-29T08:19:14.672171+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:30:18.530436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700408+0000","last_change":"2024-03-29T08:19:14.698728+0000","last_active":"2024-03-29T08:19:14.700408+0000","last_peered":"2024-03-29T08:19:14.700408+0000","last_clean":"2024-03-29T08:19:14.700408+0000","last_became_active":"2024-03-29T08:19:14.698138+0000","last_became_peered":"2024-03-29T08:19:14.698138+0000","last_unstale":"2024-03-29T08:19:14.700408+0000","last_undegraded":"2024-03-29T08:19:14.700408+0000","last_fullsized":"2024-03-29T08:19:14.700408+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:07:30.640723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.683645+0000","last_change":"2024-03-29T08:19:14.281517+0000","last_active":"2024-03-29T08:19:14.683645+0000","last_peered":"2024-03-29T08:19:14.683645+0000","last_clean":"2024-03-29T08:19:14.683645+0000","last_became_active":"2024-03-29T08:19:14.281182+0000","last_became_peered":"2024-03-29T08:19:14.281182+0000","last_unstale":"2024-03-29T08:19:14.683645+0000","last_undegraded":"2024-03-29T08:19:14.683645+0000","last_fullsized":"2024-03-29T08:19:14.683645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:21:54.574176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.689204+0000","last_change":"2024-03-29T08:19:14.685120+0000","last_active":"2024-03-29T08:19:14.689204+0000","last_peered":"2024-03-29T08:19:14.689204+0000","last_clean":"2024-03-29T08:19:14.689204+0000","last_became_active":"2024-03-29T08:19:14.682875+0000","last_became_peered":"2024-03-29T08:19:14.682875+0000","last_unstale":"2024-03-29T08:19:14.689204+0000","last_undegraded":"2024-03-29T08:19:14.689204+0000","last_fullsized":"2024-03-29T08:19:14.689204+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:55:24.587078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.683597+0000","last_change":"2024-03-29T08:19:14.283788+0000","last_active":"2024-03-29T08:19:14.683597+0000","last_peered":"2024-03-29T08:19:14.683597+0000","last_clean":"2024-03-29T08:19:14.683597+0000","last_became_active":"2024-03-29T08:19:14.283586+0000","last_became_peered":"2024-03-29T08:19:14.283586+0000","last_unstale":"2024-03-29T08:19:14.683597+0000","last_undegraded":"2024-03-29T08:19:14.683597+0000","last_fullsized":"2024-03-29T08:19:14.683597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:52:19.059268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":173,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.282014+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:19:14.282014+0000","last_peered":"2024-03-29T08:19:14.282014+0000","last_clean":"2024-03-29T08:19:14.282014+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:19:14.282014+0000","last_undegraded":"2024-03-29T08:19:14.282014+0000","last_fullsized":"2024-03-29T08:19:14.282014+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":118,"acting":118,"num_store_stats":7},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":150465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68799999999999994}]},{"osd":1,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.79800000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.78300000000000003}]},{"osd":3,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.88600000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.81299999999999994}]},{"osd":5,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.873},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.90700000000000003}]}]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":143668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55800000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.371}]},{"osd":5,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58499999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.56499999999999995}]}]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":137393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.1709999999999998,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.81399999999999995}]},{"osd":1,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.363,"5min":2.363,"15min":2.363},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.93899999999999995}]},{"osd":3,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.89600000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.86799999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.91500000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.99199999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":376832,"data_stored":143668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.35899999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.34200000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.437}]},{"osd":4,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.70599999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.73199999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.66100000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":0.83399999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":376832,"data_stored":143668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.44}]},{"osd":1,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.54500000000000004}]},{"osd":4,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.502}]},{"osd":5,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.64500000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.625}]},{"osd":7,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.63600000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":987136,"data_stored":740881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":4.2539999999999996,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.1909999999999998,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.42299999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51700000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.627}]},{"osd":5,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.76800000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":364544,"data_stored":137393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":3.9359999999999999,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.497},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":8.0419999999999998,"5min":8.0419999999999998,"15min":8.0419999999999998},"last":0.51000000000000001}]},{"osd":2,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":4.907,"5min":4.907,"15min":4.907},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":7.9569999999999999,"5min":7.9569999999999999,"15min":7.9569999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.44400000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":4.4720000000000004,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.625},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":4.3920000000000003,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.58299999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.3200000000000003,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":4.4100000000000001,"5min":4.4100000000000001,"15min":4.4100000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":4.5209999999999999,"5min":4.5209999999999999,"15min":4.5209999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":4.4420000000000002,"5min":4.4420000000000002,"15min":4.4420000000000002},"last":0.63900000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":727761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.39800000000000002}]},{"osd":1,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.53000000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.80200000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.48499999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.439},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.55500000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.59099999999999997}]},{"osd":7,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.68300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:19:18.787 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:19:18.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.964+0000 7f8d29c00640 1 Processor -- start 2024-03-29T08:19:18.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.964+0000 7f8d29c00640 1 -- start start 2024-03-29T08:19:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.968+0000 7f8d29c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d2415c9d0 0x7f8d240cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.968+0000 7f8d29c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8d24099150 0x7f8d24166a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.968+0000 7f8d29c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d240d0030 con 0x7f8d240cef60 2024-03-29T08:19:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.968+0000 7f8d29c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d240cf420 con 0x7f8d24099150 2024-03-29T08:19:18.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.968+0000 7f8d29c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d240cf700 con 0x7f8d2415c9d0 2024-03-29T08:19:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.968+0000 7f8d22a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d2415c9d0 0x7f8d240cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.968+0000 7f8d23400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8d24099150 0x7f8d24166a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.968+0000 7f8d22a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d2415c9d0 0x7f8d240cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:51658/0 (socket says 172.21.15.46:51658) 2024-03-29T08:19:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.968+0000 7f8d22a00640 1 -- 172.21.15.46:0/1947449894 learned_addr learned my addr 172.21.15.46:0/1947449894 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.968+0000 7f8d1a000640 1 --1- 172.21.15.46:0/1947449894 >> v1:172.21.15.73:6789/0 conn(0x7f8d240cef60 0x7f8d24098a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:34002/0 (socket says 172.21.15.46:34002) 2024-03-29T08:19:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.968+0000 7f8d22000640 1 -- 172.21.15.46:0/1947449894 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778323851 0 0) 0x7f8d240d0030 con 0x7f8d240cef60 2024-03-29T08:19:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.968+0000 7f8d22000640 1 -- 172.21.15.46:0/1947449894 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d08003530 con 0x7f8d240cef60 2024-03-29T08:19:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.968+0000 7f8d22a00640 1 -- 172.21.15.46:0/1947449894 >> v2:172.21.15.73:3300/0 conn(0x7f8d24099150 msgr2=0x7f8d24166a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.968+0000 7f8d22a00640 1 --2- 172.21.15.46:0/1947449894 >> v2:172.21.15.73:3300/0 conn(0x7f8d24099150 0x7f8d24166a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.968+0000 7f8d22a00640 1 -- 172.21.15.46:0/1947449894 >> v1:172.21.15.73:6789/0 conn(0x7f8d240cef60 legacy=0x7f8d24098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.968+0000 7f8d22a00640 1 -- 172.21.15.46:0/1947449894 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d240cf9e0 con 0x7f8d2415c9d0 2024-03-29T08:19:18.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.968+0000 7f8d23400640 1 --2- 172.21.15.46:0/1947449894 >> v2:172.21.15.73:3300/0 conn(0x7f8d24099150 0x7f8d24166a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:18.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.972+0000 7f8d22a00640 1 --2- 172.21.15.46:0/1947449894 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d2415c9d0 0x7f8d240cea20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009870 tx=0x7f8d1401c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa7492f60cea3026 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:18.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.972+0000 7f8d22000640 1 -- 172.21.15.46:0/1947449894 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d140a9610 con 0x7f8d2415c9d0 2024-03-29T08:19:18.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.972+0000 7f8d22000640 1 -- 172.21.15.46:0/1947449894 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d140a9770 con 0x7f8d2415c9d0 2024-03-29T08:19:18.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.972+0000 7f8d22000640 1 -- 172.21.15.46:0/1947449894 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d140a9b00 con 0x7f8d2415c9d0 2024-03-29T08:19:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.972+0000 7f8d29c00640 1 -- 172.21.15.46:0/1947449894 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d2415c9d0 msgr2=0x7f8d240cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.972+0000 7f8d29c00640 1 --2- 172.21.15.46:0/1947449894 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d2415c9d0 0x7f8d240cea20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009870 tx=0x7f8d1401c530 comp rx=0 tx=0).stop 2024-03-29T08:19:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.972+0000 7f8d29c00640 1 -- 172.21.15.46:0/1947449894 shutdown_connections 2024-03-29T08:19:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.972+0000 7f8d29c00640 1 --2- 172.21.15.46:0/1947449894 >> v2:172.21.15.73:3300/0 conn(0x7f8d24099150 0x7f8d24166a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.972+0000 7f8d29c00640 1 --2- 172.21.15.46:0/1947449894 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d2415c9d0 0x7f8d240cea20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.972+0000 7f8d29c00640 1 -- 172.21.15.46:0/1947449894 >> 172.21.15.46:0/1947449894 conn(0x7f8d240b21a0 msgr2=0x7f8d240b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.972+0000 7f8d29c00640 1 -- 172.21.15.46:0/1947449894 shutdown_connections 2024-03-29T08:19:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.972+0000 7f8d29c00640 1 -- 172.21.15.46:0/1947449894 wait complete. 2024-03-29T08:19:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d29c00640 1 Processor -- start 2024-03-29T08:19:18.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d29c00640 1 -- start start 2024-03-29T08:19:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d29c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d24099150 0x7f8d2406f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d29c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d240cef60 0x7f8d2414dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d29c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d2415c9d0 0x7f8d2414f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d29c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2414f1e0 con 0x7f8d24099150 2024-03-29T08:19:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d29c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2406fab0 con 0x7f8d2415c9d0 2024-03-29T08:19:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d29c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2415b090 con 0x7f8d240cef60 2024-03-29T08:19:18.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d22a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d24099150 0x7f8d2406f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:18.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d22a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d24099150 0x7f8d2406f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34800/0 (socket says 172.21.15.46:34800) 2024-03-29T08:19:18.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d22a00640 1 -- 172.21.15.46:0/650461561 learned_addr learned my addr 172.21.15.46:0/650461561 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:18.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d23400640 1 --2- 172.21.15.46:0/650461561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d2415c9d0 0x7f8d2414f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:18.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d1a000640 1 --2- 172.21.15.46:0/650461561 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d240cef60 0x7f8d2414dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d22a00640 1 -- 172.21.15.46:0/650461561 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d240cef60 msgr2=0x7f8d2414dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d22a00640 1 --2- 172.21.15.46:0/650461561 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d240cef60 0x7f8d2414dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d22a00640 1 -- 172.21.15.46:0/650461561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d2415c9d0 msgr2=0x7f8d2414f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d22a00640 1 --2- 172.21.15.46:0/650461561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d2415c9d0 0x7f8d2414f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d22a00640 1 -- 172.21.15.46:0/650461561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c009a70 con 0x7f8d24099150 2024-03-29T08:19:18.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d23400640 1 --2- 172.21.15.46:0/650461561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d2415c9d0 0x7f8d2414f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:18.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d22a00640 1 --2- 172.21.15.46:0/650461561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d24099150 0x7f8d2406f570 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8d1401abe0 tx=0x7f8d140192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:18.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d1b400640 1 -- 172.21.15.46:0/650461561 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d140bb3f0 con 0x7f8d24099150 2024-03-29T08:19:18.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d1b400640 1 -- 172.21.15.46:0/650461561 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d140bb570 con 0x7f8d24099150 2024-03-29T08:19:18.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d1b400640 1 -- 172.21.15.46:0/650461561 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d140bb920 con 0x7f8d24099150 2024-03-29T08:19:18.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d29c00640 1 -- 172.21.15.46:0/650461561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d140094d0 con 0x7f8d24099150 2024-03-29T08:19:18.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d29c00640 1 -- 172.21.15.46:0/650461561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2415b650 con 0x7f8d24099150 2024-03-29T08:19:18.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.976+0000 7f8d29c00640 1 -- 172.21.15.46:0/650461561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d24066c90 con 0x7f8d24099150 2024-03-29T08:19:18.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.980+0000 7f8d1b400640 1 -- 172.21.15.46:0/650461561 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8d140bba80 con 0x7f8d24099150 2024-03-29T08:19:18.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.980+0000 7f8d1b400640 1 --2- 172.21.15.46:0/650461561 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8d000666c0 0x7f8d00068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:18.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.980+0000 7f8d1b400640 1 -- 172.21.15.46:0/650461561 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d1411e5d0 con 0x7f8d24099150 2024-03-29T08:19:18.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.980+0000 7f8d1a000640 1 --2- 172.21.15.46:0/650461561 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8d000666c0 0x7f8d00068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:18.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.980+0000 7f8d1a000640 1 --2- 172.21.15.46:0/650461561 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8d000666c0 0x7f8d00068b80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8d240b65f0 tx=0x7f8d0c007b60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:18.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:18.984+0000 7f8d1b400640 1 -- 172.21.15.46:0/650461561 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8d140eb8a0 con 0x7f8d24099150 2024-03-29T08:19:19.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.140+0000 7f8d29c00640 1 -- 172.21.15.46:0/650461561 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8d2406b330 con 0x7f8d000666c0 2024-03-29T08:19:19.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.148+0000 7f8d1b400640 1 -- 172.21.15.46:0/650461561 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288161 (secure 0 0 0) 0x7f8d0c007670 con 0x7f8d000666c0 2024-03-29T08:19:19.152 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:19:19.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.152+0000 7f8d29c00640 1 -- 172.21.15.46:0/650461561 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8d000666c0 msgr2=0x7f8d00068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:19.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.152+0000 7f8d29c00640 1 --2- 172.21.15.46:0/650461561 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8d000666c0 0x7f8d00068b80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8d240b65f0 tx=0x7f8d0c007b60 comp rx=0 tx=0).stop 2024-03-29T08:19:19.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.152+0000 7f8d29c00640 1 -- 172.21.15.46:0/650461561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d24099150 msgr2=0x7f8d2406f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:19.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.152+0000 7f8d29c00640 1 --2- 172.21.15.46:0/650461561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d24099150 0x7f8d2406f570 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f8d1401abe0 tx=0x7f8d140192d0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.152+0000 7f8d29c00640 1 -- 172.21.15.46:0/650461561 shutdown_connections 2024-03-29T08:19:19.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.152+0000 7f8d29c00640 1 --2- 172.21.15.46:0/650461561 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8d000666c0 0x7f8d00068b80 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.152+0000 7f8d29c00640 1 --2- 172.21.15.46:0/650461561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d2415c9d0 0x7f8d2414f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.152+0000 7f8d29c00640 1 --2- 172.21.15.46:0/650461561 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d240cef60 0x7f8d2414dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.152+0000 7f8d29c00640 1 --2- 172.21.15.46:0/650461561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d24099150 0x7f8d2406f570 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.152+0000 7f8d29c00640 1 -- 172.21.15.46:0/650461561 >> 172.21.15.46:0/650461561 conn(0x7f8d240b21a0 msgr2=0x7f8d241585f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:19.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.152+0000 7f8d29c00640 1 -- 172.21.15.46:0/650461561 shutdown_connections 2024-03-29T08:19:19.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.152+0000 7f8d29c00640 1 -- 172.21.15.46:0/650461561 wait complete. 2024-03-29T08:19:19.172 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-03-29T08:19:18.057099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":184,"acting":184,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236656,"kb_used_data":4100,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738064,"statfs":{"total":773094113280,"available":772851777536,"internally_reserved":0,"allocated":4198400,"data_stored":2324897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.804539"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.679129+0000","last_change":"2024-03-29T08:19:14.677048+0000","last_active":"2024-03-29T08:19:14.679129+0000","last_peered":"2024-03-29T08:19:14.679129+0000","last_clean":"2024-03-29T08:19:14.679129+0000","last_became_active":"2024-03-29T08:19:14.676802+0000","last_became_peered":"2024-03-29T08:19:14.676802+0000","last_unstale":"2024-03-29T08:19:14.679129+0000","last_undegraded":"2024-03-29T08:19:14.679129+0000","last_fullsized":"2024-03-29T08:19:14.679129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:37:16.877010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:19:14.252791+0000","last_change":"2024-03-29T08:19:14.252791+0000","last_active":"2024-03-29T08:19:14.252791+0000","last_peered":"2024-03-29T08:19:14.252791+0000","last_clean":"2024-03-29T08:19:14.252791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:14.252791+0000","last_undegraded":"2024-03-29T08:19:14.252791+0000","last_fullsized":"2024-03-29T08:19:14.252791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.681839+0000","last_change":"2024-03-29T08:19:14.677189+0000","last_active":"2024-03-29T08:19:14.681839+0000","last_peered":"2024-03-29T08:19:14.681839+0000","last_clean":"2024-03-29T08:19:14.681839+0000","last_became_active":"2024-03-29T08:19:14.676526+0000","last_became_peered":"2024-03-29T08:19:14.676526+0000","last_unstale":"2024-03-29T08:19:14.681839+0000","last_undegraded":"2024-03-29T08:19:14.681839+0000","last_fullsized":"2024-03-29T08:19:14.681839+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:20.587889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705567+0000","last_change":"2024-03-29T08:19:14.699305+0000","last_active":"2024-03-29T08:19:14.705567+0000","last_peered":"2024-03-29T08:19:14.705567+0000","last_clean":"2024-03-29T08:19:14.705567+0000","last_became_active":"2024-03-29T08:19:14.698637+0000","last_became_peered":"2024-03-29T08:19:14.698637+0000","last_unstale":"2024-03-29T08:19:14.705567+0000","last_undegraded":"2024-03-29T08:19:14.705567+0000","last_fullsized":"2024-03-29T08:19:14.705567+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:35.234827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:19:14.252791+0000","last_change":"2024-03-29T08:19:14.252791+0000","last_active":"2024-03-29T08:19:14.252791+0000","last_peered":"2024-03-29T08:19:14.252791+0000","last_clean":"2024-03-29T08:19:14.252791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:14.252791+0000","last_undegraded":"2024-03-29T08:19:14.252791+0000","last_fullsized":"2024-03-29T08:19:14.252791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694387+0000","last_change":"2024-03-29T08:19:14.691105+0000","last_active":"2024-03-29T08:19:14.694387+0000","last_peered":"2024-03-29T08:19:14.694387+0000","last_clean":"2024-03-29T08:19:14.694387+0000","last_became_active":"2024-03-29T08:19:14.690486+0000","last_became_peered":"2024-03-29T08:19:14.690486+0000","last_unstale":"2024-03-29T08:19:14.694387+0000","last_undegraded":"2024-03-29T08:19:14.694387+0000","last_fullsized":"2024-03-29T08:19:14.694387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:21.736865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.678975+0000","last_change":"2024-03-29T08:19:14.675543+0000","last_active":"2024-03-29T08:19:14.678975+0000","last_peered":"2024-03-29T08:19:14.678975+0000","last_clean":"2024-03-29T08:19:14.678975+0000","last_became_active":"2024-03-29T08:19:14.675201+0000","last_became_peered":"2024-03-29T08:19:14.675201+0000","last_unstale":"2024-03-29T08:19:14.678975+0000","last_undegraded":"2024-03-29T08:19:14.678975+0000","last_fullsized":"2024-03-29T08:19:14.678975+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:08:58.521804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687632+0000","last_change":"2024-03-29T08:19:14.681677+0000","last_active":"2024-03-29T08:19:14.687632+0000","last_peered":"2024-03-29T08:19:14.687632+0000","last_clean":"2024-03-29T08:19:14.687632+0000","last_became_active":"2024-03-29T08:19:14.680707+0000","last_became_peered":"2024-03-29T08:19:14.680707+0000","last_unstale":"2024-03-29T08:19:14.687632+0000","last_undegraded":"2024-03-29T08:19:14.687632+0000","last_fullsized":"2024-03-29T08:19:14.687632+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:04:29.029546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.704743+0000","last_change":"2024-03-29T08:19:14.703892+0000","last_active":"2024-03-29T08:19:14.704743+0000","last_peered":"2024-03-29T08:19:14.704743+0000","last_clean":"2024-03-29T08:19:14.704743+0000","last_became_active":"2024-03-29T08:19:14.701909+0000","last_became_peered":"2024-03-29T08:19:14.701909+0000","last_unstale":"2024-03-29T08:19:14.704743+0000","last_undegraded":"2024-03-29T08:19:14.704743+0000","last_fullsized":"2024-03-29T08:19:14.704743+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:57:15.587273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701010+0000","last_change":"2024-03-29T08:19:14.694312+0000","last_active":"2024-03-29T08:19:14.701010+0000","last_peered":"2024-03-29T08:19:14.701010+0000","last_clean":"2024-03-29T08:19:14.701010+0000","last_became_active":"2024-03-29T08:19:14.694027+0000","last_became_peered":"2024-03-29T08:19:14.694027+0000","last_unstale":"2024-03-29T08:19:14.701010+0000","last_undegraded":"2024-03-29T08:19:14.701010+0000","last_fullsized":"2024-03-29T08:19:14.701010+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:15:31.529776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.704807+0000","last_change":"2024-03-29T08:19:14.703541+0000","last_active":"2024-03-29T08:19:14.704807+0000","last_peered":"2024-03-29T08:19:14.704807+0000","last_clean":"2024-03-29T08:19:14.704807+0000","last_became_active":"2024-03-29T08:19:14.703317+0000","last_became_peered":"2024-03-29T08:19:14.703317+0000","last_unstale":"2024-03-29T08:19:14.704807+0000","last_undegraded":"2024-03-29T08:19:14.704807+0000","last_fullsized":"2024-03-29T08:19:14.704807+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:18:37.057951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687173+0000","last_change":"2024-03-29T08:19:14.684087+0000","last_active":"2024-03-29T08:19:14.687173+0000","last_peered":"2024-03-29T08:19:14.687173+0000","last_clean":"2024-03-29T08:19:14.687173+0000","last_became_active":"2024-03-29T08:19:14.683827+0000","last_became_peered":"2024-03-29T08:19:14.683827+0000","last_unstale":"2024-03-29T08:19:14.687173+0000","last_undegraded":"2024-03-29T08:19:14.687173+0000","last_fullsized":"2024-03-29T08:19:14.687173+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:21.736865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702127+0000","last_change":"2024-03-29T08:19:14.699133+0000","last_active":"2024-03-29T08:19:14.702127+0000","last_peered":"2024-03-29T08:19:14.702127+0000","last_clean":"2024-03-29T08:19:14.702127+0000","last_became_active":"2024-03-29T08:19:14.698579+0000","last_became_peered":"2024-03-29T08:19:14.698579+0000","last_unstale":"2024-03-29T08:19:14.702127+0000","last_undegraded":"2024-03-29T08:19:14.702127+0000","last_fullsized":"2024-03-29T08:19:14.702127+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:00:07.571380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702206+0000","last_change":"2024-03-29T08:19:14.694580+0000","last_active":"2024-03-29T08:19:14.702206+0000","last_peered":"2024-03-29T08:19:14.702206+0000","last_clean":"2024-03-29T08:19:14.702206+0000","last_became_active":"2024-03-29T08:19:14.694305+0000","last_became_peered":"2024-03-29T08:19:14.694305+0000","last_unstale":"2024-03-29T08:19:14.702206+0000","last_undegraded":"2024-03-29T08:19:14.702206+0000","last_fullsized":"2024-03-29T08:19:14.702206+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:19:13.272178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.676292+0000","last_change":"2024-03-29T08:19:14.674438+0000","last_active":"2024-03-29T08:19:14.676292+0000","last_peered":"2024-03-29T08:19:14.676292+0000","last_clean":"2024-03-29T08:19:14.676292+0000","last_became_active":"2024-03-29T08:19:14.674172+0000","last_became_peered":"2024-03-29T08:19:14.674172+0000","last_unstale":"2024-03-29T08:19:14.676292+0000","last_undegraded":"2024-03-29T08:19:14.676292+0000","last_fullsized":"2024-03-29T08:19:14.676292+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:29.147117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687494+0000","last_change":"2024-03-29T08:19:14.681624+0000","last_active":"2024-03-29T08:19:14.687494+0000","last_peered":"2024-03-29T08:19:14.687494+0000","last_clean":"2024-03-29T08:19:14.687494+0000","last_became_active":"2024-03-29T08:19:14.681290+0000","last_became_peered":"2024-03-29T08:19:14.681290+0000","last_unstale":"2024-03-29T08:19:14.687494+0000","last_undegraded":"2024-03-29T08:19:14.687494+0000","last_fullsized":"2024-03-29T08:19:14.687494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:34:59.393546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700295+0000","last_change":"2024-03-29T08:19:14.695305+0000","last_active":"2024-03-29T08:19:14.700295+0000","last_peered":"2024-03-29T08:19:14.700295+0000","last_clean":"2024-03-29T08:19:14.700295+0000","last_became_active":"2024-03-29T08:19:14.694942+0000","last_became_peered":"2024-03-29T08:19:14.694942+0000","last_unstale":"2024-03-29T08:19:14.700295+0000","last_undegraded":"2024-03-29T08:19:14.700295+0000","last_fullsized":"2024-03-29T08:19:14.700295+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:19:41.115142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694349+0000","last_change":"2024-03-29T08:19:14.690840+0000","last_active":"2024-03-29T08:19:14.694349+0000","last_peered":"2024-03-29T08:19:14.694349+0000","last_clean":"2024-03-29T08:19:14.694349+0000","last_became_active":"2024-03-29T08:19:14.689554+0000","last_became_peered":"2024-03-29T08:19:14.689554+0000","last_unstale":"2024-03-29T08:19:14.694349+0000","last_undegraded":"2024-03-29T08:19:14.694349+0000","last_fullsized":"2024-03-29T08:19:14.694349+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:49:04.176971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694388+0000","last_change":"2024-03-29T08:19:14.689362+0000","last_active":"2024-03-29T08:19:14.694388+0000","last_peered":"2024-03-29T08:19:14.694388+0000","last_clean":"2024-03-29T08:19:14.694388+0000","last_became_active":"2024-03-29T08:19:14.689050+0000","last_became_peered":"2024-03-29T08:19:14.689050+0000","last_unstale":"2024-03-29T08:19:14.694388+0000","last_undegraded":"2024-03-29T08:19:14.694388+0000","last_fullsized":"2024-03-29T08:19:14.694388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:20:38.835695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.692494+0000","last_change":"2024-03-29T08:19:14.683957+0000","last_active":"2024-03-29T08:19:14.692494+0000","last_peered":"2024-03-29T08:19:14.692494+0000","last_clean":"2024-03-29T08:19:14.692494+0000","last_became_active":"2024-03-29T08:19:14.683710+0000","last_became_peered":"2024-03-29T08:19:14.683710+0000","last_unstale":"2024-03-29T08:19:14.692494+0000","last_undegraded":"2024-03-29T08:19:14.692494+0000","last_fullsized":"2024-03-29T08:19:14.692494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:11:49.840732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.689299+0000","last_change":"2024-03-29T08:19:14.685648+0000","last_active":"2024-03-29T08:19:14.689299+0000","last_peered":"2024-03-29T08:19:14.689299+0000","last_clean":"2024-03-29T08:19:14.689299+0000","last_became_active":"2024-03-29T08:19:14.685404+0000","last_became_peered":"2024-03-29T08:19:14.685404+0000","last_unstale":"2024-03-29T08:19:14.689299+0000","last_undegraded":"2024-03-29T08:19:14.689299+0000","last_fullsized":"2024-03-29T08:19:14.689299+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:14:08.867120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701342+0000","last_change":"2024-03-29T08:19:14.698732+0000","last_active":"2024-03-29T08:19:14.701342+0000","last_peered":"2024-03-29T08:19:14.701342+0000","last_clean":"2024-03-29T08:19:14.701342+0000","last_became_active":"2024-03-29T08:19:14.698484+0000","last_became_peered":"2024-03-29T08:19:14.698484+0000","last_unstale":"2024-03-29T08:19:14.701342+0000","last_undegraded":"2024-03-29T08:19:14.701342+0000","last_fullsized":"2024-03-29T08:19:14.701342+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:08.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.688435+0000","last_change":"2024-03-29T08:19:14.685174+0000","last_active":"2024-03-29T08:19:14.688435+0000","last_peered":"2024-03-29T08:19:14.688435+0000","last_clean":"2024-03-29T08:19:14.688435+0000","last_became_active":"2024-03-29T08:19:14.684605+0000","last_became_peered":"2024-03-29T08:19:14.684605+0000","last_unstale":"2024-03-29T08:19:14.688435+0000","last_undegraded":"2024-03-29T08:19:14.688435+0000","last_fullsized":"2024-03-29T08:19:14.688435+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:34:59.393546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.699334+0000","last_change":"2024-03-29T08:19:14.698117+0000","last_active":"2024-03-29T08:19:14.699334+0000","last_peered":"2024-03-29T08:19:14.699334+0000","last_clean":"2024-03-29T08:19:14.699334+0000","last_became_active":"2024-03-29T08:19:14.697861+0000","last_became_peered":"2024-03-29T08:19:14.697861+0000","last_unstale":"2024-03-29T08:19:14.699334+0000","last_undegraded":"2024-03-29T08:19:14.699334+0000","last_fullsized":"2024-03-29T08:19:14.699334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:14:08.867120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702337+0000","last_change":"2024-03-29T08:19:14.695925+0000","last_active":"2024-03-29T08:19:14.702337+0000","last_peered":"2024-03-29T08:19:14.702337+0000","last_clean":"2024-03-29T08:19:14.702337+0000","last_became_active":"2024-03-29T08:19:14.695180+0000","last_became_peered":"2024-03-29T08:19:14.695180+0000","last_unstale":"2024-03-29T08:19:14.702337+0000","last_undegraded":"2024-03-29T08:19:14.702337+0000","last_fullsized":"2024-03-29T08:19:14.702337+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:20:02.908287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702375+0000","last_change":"2024-03-29T08:19:14.696275+0000","last_active":"2024-03-29T08:19:14.702375+0000","last_peered":"2024-03-29T08:19:14.702375+0000","last_clean":"2024-03-29T08:19:14.702375+0000","last_became_active":"2024-03-29T08:19:14.696065+0000","last_became_peered":"2024-03-29T08:19:14.696065+0000","last_unstale":"2024-03-29T08:19:14.702375+0000","last_undegraded":"2024-03-29T08:19:14.702375+0000","last_fullsized":"2024-03-29T08:19:14.702375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:00:46.960177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:19:14.252791+0000","last_change":"2024-03-29T08:19:14.252791+0000","last_active":"2024-03-29T08:19:14.252791+0000","last_peered":"2024-03-29T08:19:14.252791+0000","last_clean":"2024-03-29T08:19:14.252791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:14.252791+0000","last_undegraded":"2024-03-29T08:19:14.252791+0000","last_fullsized":"2024-03-29T08:19:14.252791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.693704+0000","last_change":"2024-03-29T08:19:14.684317+0000","last_active":"2024-03-29T08:19:14.693704+0000","last_peered":"2024-03-29T08:19:14.693704+0000","last_clean":"2024-03-29T08:19:14.693704+0000","last_became_active":"2024-03-29T08:19:14.683125+0000","last_became_peered":"2024-03-29T08:19:14.683125+0000","last_unstale":"2024-03-29T08:19:14.693704+0000","last_undegraded":"2024-03-29T08:19:14.693704+0000","last_fullsized":"2024-03-29T08:19:14.693704+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:52:54.511385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702775+0000","last_change":"2024-03-29T08:19:14.701650+0000","last_active":"2024-03-29T08:19:14.702775+0000","last_peered":"2024-03-29T08:19:14.702775+0000","last_clean":"2024-03-29T08:19:14.702775+0000","last_became_active":"2024-03-29T08:19:14.697686+0000","last_became_peered":"2024-03-29T08:19:14.697686+0000","last_unstale":"2024-03-29T08:19:14.702775+0000","last_undegraded":"2024-03-29T08:19:14.702775+0000","last_fullsized":"2024-03-29T08:19:14.702775+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:10:23.783843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687174+0000","last_change":"2024-03-29T08:19:14.684872+0000","last_active":"2024-03-29T08:19:14.687174+0000","last_peered":"2024-03-29T08:19:14.687174+0000","last_clean":"2024-03-29T08:19:14.687174+0000","last_became_active":"2024-03-29T08:19:14.684622+0000","last_became_peered":"2024-03-29T08:19:14.684622+0000","last_unstale":"2024-03-29T08:19:14.687174+0000","last_undegraded":"2024-03-29T08:19:14.687174+0000","last_fullsized":"2024-03-29T08:19:14.687174+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:00:07.571380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687689+0000","last_change":"2024-03-29T08:19:14.681150+0000","last_active":"2024-03-29T08:19:14.687689+0000","last_peered":"2024-03-29T08:19:14.687689+0000","last_clean":"2024-03-29T08:19:14.687689+0000","last_became_active":"2024-03-29T08:19:14.680888+0000","last_became_peered":"2024-03-29T08:19:14.680888+0000","last_unstale":"2024-03-29T08:19:14.687689+0000","last_undegraded":"2024-03-29T08:19:14.687689+0000","last_fullsized":"2024-03-29T08:19:14.687689+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:10:23.783843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700291+0000","last_change":"2024-03-29T08:19:14.698385+0000","last_active":"2024-03-29T08:19:14.700291+0000","last_peered":"2024-03-29T08:19:14.700291+0000","last_clean":"2024-03-29T08:19:14.700291+0000","last_became_active":"2024-03-29T08:19:14.698119+0000","last_became_peered":"2024-03-29T08:19:14.698119+0000","last_unstale":"2024-03-29T08:19:14.700291+0000","last_undegraded":"2024-03-29T08:19:14.700291+0000","last_fullsized":"2024-03-29T08:19:14.700291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:03:28.287471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.692119+0000","last_change":"2024-03-29T08:19:14.686891+0000","last_active":"2024-03-29T08:19:14.692119+0000","last_peered":"2024-03-29T08:19:14.692119+0000","last_clean":"2024-03-29T08:19:14.692119+0000","last_became_active":"2024-03-29T08:19:14.686615+0000","last_became_peered":"2024-03-29T08:19:14.686615+0000","last_unstale":"2024-03-29T08:19:14.692119+0000","last_undegraded":"2024-03-29T08:19:14.692119+0000","last_fullsized":"2024-03-29T08:19:14.692119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:10:23.783843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701442+0000","last_change":"2024-03-29T08:19:14.278073+0000","last_active":"2024-03-29T08:19:14.701442+0000","last_peered":"2024-03-29T08:19:14.701442+0000","last_clean":"2024-03-29T08:19:14.701442+0000","last_became_active":"2024-03-29T08:19:14.277778+0000","last_became_peered":"2024-03-29T08:19:14.277778+0000","last_unstale":"2024-03-29T08:19:14.701442+0000","last_undegraded":"2024-03-29T08:19:14.701442+0000","last_fullsized":"2024-03-29T08:19:14.701442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:36:08.787710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687978+0000","last_change":"2024-03-29T08:19:14.682096+0000","last_active":"2024-03-29T08:19:14.687978+0000","last_peered":"2024-03-29T08:19:14.687978+0000","last_clean":"2024-03-29T08:19:14.687978+0000","last_became_active":"2024-03-29T08:19:14.681779+0000","last_became_peered":"2024-03-29T08:19:14.681779+0000","last_unstale":"2024-03-29T08:19:14.687978+0000","last_undegraded":"2024-03-29T08:19:14.687978+0000","last_fullsized":"2024-03-29T08:19:14.687978+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:52:20.065176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.703320+0000","last_change":"2024-03-29T08:19:14.281218+0000","last_active":"2024-03-29T08:19:14.703320+0000","last_peered":"2024-03-29T08:19:14.703320+0000","last_clean":"2024-03-29T08:19:14.703320+0000","last_became_active":"2024-03-29T08:19:14.280883+0000","last_became_peered":"2024-03-29T08:19:14.280883+0000","last_unstale":"2024-03-29T08:19:14.703320+0000","last_undegraded":"2024-03-29T08:19:14.703320+0000","last_fullsized":"2024-03-29T08:19:14.703320+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:14:22.394882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701140+0000","last_change":"2024-03-29T08:19:14.698993+0000","last_active":"2024-03-29T08:19:14.701140+0000","last_peered":"2024-03-29T08:19:14.701140+0000","last_clean":"2024-03-29T08:19:14.701140+0000","last_became_active":"2024-03-29T08:19:14.698757+0000","last_became_peered":"2024-03-29T08:19:14.698757+0000","last_unstale":"2024-03-29T08:19:14.701140+0000","last_undegraded":"2024-03-29T08:19:14.701140+0000","last_fullsized":"2024-03-29T08:19:14.701140+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:34:59.393546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.692376+0000","last_change":"2024-03-29T08:19:14.283589+0000","last_active":"2024-03-29T08:19:14.692376+0000","last_peered":"2024-03-29T08:19:14.692376+0000","last_clean":"2024-03-29T08:19:14.692376+0000","last_became_active":"2024-03-29T08:19:14.282933+0000","last_became_peered":"2024-03-29T08:19:14.282933+0000","last_unstale":"2024-03-29T08:19:14.692376+0000","last_undegraded":"2024-03-29T08:19:14.692376+0000","last_fullsized":"2024-03-29T08:19:14.692376+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:16:08.725000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702696+0000","last_change":"2024-03-29T08:19:14.696813+0000","last_active":"2024-03-29T08:19:14.702696+0000","last_peered":"2024-03-29T08:19:14.702696+0000","last_clean":"2024-03-29T08:19:14.702696+0000","last_became_active":"2024-03-29T08:19:14.696519+0000","last_became_peered":"2024-03-29T08:19:14.696519+0000","last_unstale":"2024-03-29T08:19:14.702696+0000","last_undegraded":"2024-03-29T08:19:14.702696+0000","last_fullsized":"2024-03-29T08:19:14.702696+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:46:59.856286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705886+0000","last_change":"2024-03-29T08:19:14.279768+0000","last_active":"2024-03-29T08:19:14.705886+0000","last_peered":"2024-03-29T08:19:14.705886+0000","last_clean":"2024-03-29T08:19:14.705886+0000","last_became_active":"2024-03-29T08:19:14.279493+0000","last_became_peered":"2024-03-29T08:19:14.279493+0000","last_unstale":"2024-03-29T08:19:14.705886+0000","last_undegraded":"2024-03-29T08:19:14.705886+0000","last_fullsized":"2024-03-29T08:19:14.705886+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:11:12.357829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.703025+0000","last_change":"2024-03-29T08:19:14.702417+0000","last_active":"2024-03-29T08:19:14.703025+0000","last_peered":"2024-03-29T08:19:14.703025+0000","last_clean":"2024-03-29T08:19:14.703025+0000","last_became_active":"2024-03-29T08:19:14.701927+0000","last_became_peered":"2024-03-29T08:19:14.701927+0000","last_unstale":"2024-03-29T08:19:14.703025+0000","last_undegraded":"2024-03-29T08:19:14.703025+0000","last_fullsized":"2024-03-29T08:19:14.703025+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:30:19.536345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701280+0000","last_change":"2024-03-29T08:19:14.279062+0000","last_active":"2024-03-29T08:19:14.701280+0000","last_peered":"2024-03-29T08:19:14.701280+0000","last_clean":"2024-03-29T08:19:14.701280+0000","last_became_active":"2024-03-29T08:19:14.278726+0000","last_became_peered":"2024-03-29T08:19:14.278726+0000","last_unstale":"2024-03-29T08:19:14.701280+0000","last_undegraded":"2024-03-29T08:19:14.701280+0000","last_fullsized":"2024-03-29T08:19:14.701280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:35.048072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687660+0000","last_change":"2024-03-29T08:19:14.679888+0000","last_active":"2024-03-29T08:19:14.687660+0000","last_peered":"2024-03-29T08:19:14.687660+0000","last_clean":"2024-03-29T08:19:14.687660+0000","last_became_active":"2024-03-29T08:19:14.679674+0000","last_became_peered":"2024-03-29T08:19:14.679674+0000","last_unstale":"2024-03-29T08:19:14.687660+0000","last_undegraded":"2024-03-29T08:19:14.687660+0000","last_fullsized":"2024-03-29T08:19:14.687660+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:55:24.587078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701177+0000","last_change":"2024-03-29T08:19:14.280371+0000","last_active":"2024-03-29T08:19:14.701177+0000","last_peered":"2024-03-29T08:19:14.701177+0000","last_clean":"2024-03-29T08:19:14.701177+0000","last_became_active":"2024-03-29T08:19:14.280045+0000","last_became_peered":"2024-03-29T08:19:14.280045+0000","last_unstale":"2024-03-29T08:19:14.701177+0000","last_undegraded":"2024-03-29T08:19:14.701177+0000","last_fullsized":"2024-03-29T08:19:14.701177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:20.730956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.681942+0000","last_change":"2024-03-29T08:19:14.677276+0000","last_active":"2024-03-29T08:19:14.681942+0000","last_peered":"2024-03-29T08:19:14.681942+0000","last_clean":"2024-03-29T08:19:14.681942+0000","last_became_active":"2024-03-29T08:19:14.677049+0000","last_became_peered":"2024-03-29T08:19:14.677049+0000","last_unstale":"2024-03-29T08:19:14.681942+0000","last_undegraded":"2024-03-29T08:19:14.681942+0000","last_fullsized":"2024-03-29T08:19:14.681942+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:27.668682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.678961+0000","last_change":"2024-03-29T08:19:14.285270+0000","last_active":"2024-03-29T08:19:14.678961+0000","last_peered":"2024-03-29T08:19:14.678961+0000","last_clean":"2024-03-29T08:19:14.678961+0000","last_became_active":"2024-03-29T08:19:14.283882+0000","last_became_peered":"2024-03-29T08:19:14.283882+0000","last_unstale":"2024-03-29T08:19:14.678961+0000","last_undegraded":"2024-03-29T08:19:14.678961+0000","last_fullsized":"2024-03-29T08:19:14.678961+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:09:33.409328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702457+0000","last_change":"2024-03-29T08:19:14.696486+0000","last_active":"2024-03-29T08:19:14.702457+0000","last_peered":"2024-03-29T08:19:14.702457+0000","last_clean":"2024-03-29T08:19:14.702457+0000","last_became_active":"2024-03-29T08:19:14.696236+0000","last_became_peered":"2024-03-29T08:19:14.696236+0000","last_unstale":"2024-03-29T08:19:14.702457+0000","last_undegraded":"2024-03-29T08:19:14.702457+0000","last_fullsized":"2024-03-29T08:19:14.702457+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:08:58.521804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.671922+0000","last_change":"2024-03-29T08:19:14.275451+0000","last_active":"2024-03-29T08:19:14.671922+0000","last_peered":"2024-03-29T08:19:14.671922+0000","last_clean":"2024-03-29T08:19:14.671922+0000","last_became_active":"2024-03-29T08:19:14.275148+0000","last_became_peered":"2024-03-29T08:19:14.275148+0000","last_unstale":"2024-03-29T08:19:14.671922+0000","last_undegraded":"2024-03-29T08:19:14.671922+0000","last_fullsized":"2024-03-29T08:19:14.671922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:57:14.581364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694290+0000","last_change":"2024-03-29T08:19:14.690396+0000","last_active":"2024-03-29T08:19:14.694290+0000","last_peered":"2024-03-29T08:19:14.694290+0000","last_clean":"2024-03-29T08:19:14.694290+0000","last_became_active":"2024-03-29T08:19:14.690106+0000","last_became_peered":"2024-03-29T08:19:14.690106+0000","last_unstale":"2024-03-29T08:19:14.694290+0000","last_undegraded":"2024-03-29T08:19:14.694290+0000","last_fullsized":"2024-03-29T08:19:14.694290+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:54:09.222693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-29T08:19:13.260506+0000","last_change":"2024-03-29T08:19:13.260506+0000","last_active":"2024-03-29T08:19:13.246882+0000","last_peered":"2024-03-29T08:19:13.246882+0000","last_clean":"2024-03-29T08:19:13.246882+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:13.260506+0000","last_undegraded":"2024-03-29T08:19:13.260506+0000","last_fullsized":"2024-03-29T08:19:13.260506+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:58:40.235616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700952+0000","last_change":"2024-03-29T08:19:14.694063+0000","last_active":"2024-03-29T08:19:14.700952+0000","last_peered":"2024-03-29T08:19:14.700952+0000","last_clean":"2024-03-29T08:19:14.700952+0000","last_became_active":"2024-03-29T08:19:14.693817+0000","last_became_peered":"2024-03-29T08:19:14.693817+0000","last_unstale":"2024-03-29T08:19:14.700952+0000","last_undegraded":"2024-03-29T08:19:14.700952+0000","last_fullsized":"2024-03-29T08:19:14.700952+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:30:57.036017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-29T08:19:13.261393+0000","last_change":"2024-03-29T08:19:13.261393+0000","last_active":"2024-03-29T08:19:13.246882+0000","last_peered":"2024-03-29T08:19:13.246882+0000","last_clean":"2024-03-29T08:19:13.246882+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:13.261393+0000","last_undegraded":"2024-03-29T08:19:13.261393+0000","last_fullsized":"2024-03-29T08:19:13.261393+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:36:01.837144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705548+0000","last_change":"2024-03-29T08:19:14.698954+0000","last_active":"2024-03-29T08:19:14.705548+0000","last_peered":"2024-03-29T08:19:14.705548+0000","last_clean":"2024-03-29T08:19:14.705548+0000","last_became_active":"2024-03-29T08:19:14.698317+0000","last_became_peered":"2024-03-29T08:19:14.698317+0000","last_unstale":"2024-03-29T08:19:14.705548+0000","last_undegraded":"2024-03-29T08:19:14.705548+0000","last_fullsized":"2024-03-29T08:19:14.705548+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:16:39.883087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700954+0000","last_change":"2024-03-29T08:19:14.285068+0000","last_active":"2024-03-29T08:19:14.700954+0000","last_peered":"2024-03-29T08:19:14.700954+0000","last_clean":"2024-03-29T08:19:14.700954+0000","last_became_active":"2024-03-29T08:19:14.284830+0000","last_became_peered":"2024-03-29T08:19:14.284830+0000","last_unstale":"2024-03-29T08:19:14.700954+0000","last_undegraded":"2024-03-29T08:19:14.700954+0000","last_fullsized":"2024-03-29T08:19:14.700954+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:16:38.877179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705493+0000","last_change":"2024-03-29T08:19:14.704710+0000","last_active":"2024-03-29T08:19:14.705493+0000","last_peered":"2024-03-29T08:19:14.705493+0000","last_clean":"2024-03-29T08:19:14.705493+0000","last_became_active":"2024-03-29T08:19:14.704396+0000","last_became_peered":"2024-03-29T08:19:14.704396+0000","last_unstale":"2024-03-29T08:19:14.705493+0000","last_undegraded":"2024-03-29T08:19:14.705493+0000","last_fullsized":"2024-03-29T08:19:14.705493+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:24.176397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.706659+0000","last_change":"2024-03-29T08:19:14.280954+0000","last_active":"2024-03-29T08:19:14.706659+0000","last_peered":"2024-03-29T08:19:14.706659+0000","last_clean":"2024-03-29T08:19:14.706659+0000","last_became_active":"2024-03-29T08:19:14.280392+0000","last_became_peered":"2024-03-29T08:19:14.280392+0000","last_unstale":"2024-03-29T08:19:14.706659+0000","last_undegraded":"2024-03-29T08:19:14.706659+0000","last_fullsized":"2024-03-29T08:19:14.706659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:26.662774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.688513+0000","last_change":"2024-03-29T08:19:14.685902+0000","last_active":"2024-03-29T08:19:14.688513+0000","last_peered":"2024-03-29T08:19:14.688513+0000","last_clean":"2024-03-29T08:19:14.688513+0000","last_became_active":"2024-03-29T08:19:14.685705+0000","last_became_peered":"2024-03-29T08:19:14.685705+0000","last_unstale":"2024-03-29T08:19:14.688513+0000","last_undegraded":"2024-03-29T08:19:14.688513+0000","last_fullsized":"2024-03-29T08:19:14.688513+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:08.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.695832+0000","last_change":"2024-03-29T08:19:14.272012+0000","last_active":"2024-03-29T08:19:14.695832+0000","last_peered":"2024-03-29T08:19:14.695832+0000","last_clean":"2024-03-29T08:19:14.695832+0000","last_became_active":"2024-03-29T08:19:14.271736+0000","last_became_peered":"2024-03-29T08:19:14.271736+0000","last_unstale":"2024-03-29T08:19:14.695832+0000","last_undegraded":"2024-03-29T08:19:14.695832+0000","last_fullsized":"2024-03-29T08:19:14.695832+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:33.064793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.696404+0000","last_change":"2024-03-29T08:19:14.690513+0000","last_active":"2024-03-29T08:19:14.696404+0000","last_peered":"2024-03-29T08:19:14.696404+0000","last_clean":"2024-03-29T08:19:14.696404+0000","last_became_active":"2024-03-29T08:19:14.690235+0000","last_became_peered":"2024-03-29T08:19:14.690235+0000","last_unstale":"2024-03-29T08:19:14.696404+0000","last_undegraded":"2024-03-29T08:19:14.696404+0000","last_fullsized":"2024-03-29T08:19:14.696404+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:01:53.702260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.693539+0000","last_change":"2024-03-29T08:19:14.274775+0000","last_active":"2024-03-29T08:19:14.693539+0000","last_peered":"2024-03-29T08:19:14.693539+0000","last_clean":"2024-03-29T08:19:14.693539+0000","last_became_active":"2024-03-29T08:19:14.274502+0000","last_became_peered":"2024-03-29T08:19:14.274502+0000","last_unstale":"2024-03-29T08:19:14.693539+0000","last_undegraded":"2024-03-29T08:19:14.693539+0000","last_fullsized":"2024-03-29T08:19:14.693539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:20:37.829787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.681902+0000","last_change":"2024-03-29T08:19:14.675914+0000","last_active":"2024-03-29T08:19:14.681902+0000","last_peered":"2024-03-29T08:19:14.681902+0000","last_clean":"2024-03-29T08:19:14.681902+0000","last_became_active":"2024-03-29T08:19:14.675564+0000","last_became_peered":"2024-03-29T08:19:14.675564+0000","last_unstale":"2024-03-29T08:19:14.681902+0000","last_undegraded":"2024-03-29T08:19:14.681902+0000","last_fullsized":"2024-03-29T08:19:14.681902+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:36:09.793618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.682997+0000","last_change":"2024-03-29T08:19:14.281362+0000","last_active":"2024-03-29T08:19:14.682997+0000","last_peered":"2024-03-29T08:19:14.682997+0000","last_clean":"2024-03-29T08:19:14.682997+0000","last_became_active":"2024-03-29T08:19:14.281111+0000","last_became_peered":"2024-03-29T08:19:14.281111+0000","last_unstale":"2024-03-29T08:19:14.682997+0000","last_undegraded":"2024-03-29T08:19:14.682997+0000","last_fullsized":"2024-03-29T08:19:14.682997+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:26.870833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700427+0000","last_change":"2024-03-29T08:19:14.698963+0000","last_active":"2024-03-29T08:19:14.700427+0000","last_peered":"2024-03-29T08:19:14.700427+0000","last_clean":"2024-03-29T08:19:14.700427+0000","last_became_active":"2024-03-29T08:19:14.698502+0000","last_became_peered":"2024-03-29T08:19:14.698502+0000","last_unstale":"2024-03-29T08:19:14.700427+0000","last_undegraded":"2024-03-29T08:19:14.700427+0000","last_fullsized":"2024-03-29T08:19:14.700427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:21:27.491192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700579+0000","last_change":"2024-03-29T08:19:14.284917+0000","last_active":"2024-03-29T08:19:14.700579+0000","last_peered":"2024-03-29T08:19:14.700579+0000","last_clean":"2024-03-29T08:19:14.700579+0000","last_became_active":"2024-03-29T08:19:14.284667+0000","last_became_peered":"2024-03-29T08:19:14.284667+0000","last_unstale":"2024-03-29T08:19:14.700579+0000","last_undegraded":"2024-03-29T08:19:14.700579+0000","last_fullsized":"2024-03-29T08:19:14.700579+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:42.739141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.704840+0000","last_change":"2024-03-29T08:19:14.703652+0000","last_active":"2024-03-29T08:19:14.704840+0000","last_peered":"2024-03-29T08:19:14.704840+0000","last_clean":"2024-03-29T08:19:14.704840+0000","last_became_active":"2024-03-29T08:19:14.703386+0000","last_became_peered":"2024-03-29T08:19:14.703386+0000","last_unstale":"2024-03-29T08:19:14.704840+0000","last_undegraded":"2024-03-29T08:19:14.704840+0000","last_fullsized":"2024-03-29T08:19:14.704840+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:55:15.376664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702612+0000","last_change":"2024-03-29T08:19:14.287908+0000","last_active":"2024-03-29T08:19:14.702612+0000","last_peered":"2024-03-29T08:19:14.702612+0000","last_clean":"2024-03-29T08:19:14.702612+0000","last_became_active":"2024-03-29T08:19:14.287562+0000","last_became_peered":"2024-03-29T08:19:14.287562+0000","last_unstale":"2024-03-29T08:19:14.702612+0000","last_undegraded":"2024-03-29T08:19:14.702612+0000","last_fullsized":"2024-03-29T08:19:14.702612+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:02:28.529095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687362+0000","last_change":"2024-03-29T08:19:14.682493+0000","last_active":"2024-03-29T08:19:14.687362+0000","last_peered":"2024-03-29T08:19:14.687362+0000","last_clean":"2024-03-29T08:19:14.687362+0000","last_became_active":"2024-03-29T08:19:14.678342+0000","last_became_peered":"2024-03-29T08:19:14.678342+0000","last_unstale":"2024-03-29T08:19:14.687362+0000","last_undegraded":"2024-03-29T08:19:14.687362+0000","last_fullsized":"2024-03-29T08:19:14.687362+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:36.053980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.695684+0000","last_change":"2024-03-29T08:19:14.272763+0000","last_active":"2024-03-29T08:19:14.695684+0000","last_peered":"2024-03-29T08:19:14.695684+0000","last_clean":"2024-03-29T08:19:14.695684+0000","last_became_active":"2024-03-29T08:19:14.272450+0000","last_became_peered":"2024-03-29T08:19:14.272450+0000","last_unstale":"2024-03-29T08:19:14.695684+0000","last_undegraded":"2024-03-29T08:19:14.695684+0000","last_fullsized":"2024-03-29T08:19:14.695684+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:19.581981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:19:14.252791+0000","last_change":"2024-03-29T08:19:14.252791+0000","last_active":"2024-03-29T08:19:14.252791+0000","last_peered":"2024-03-29T08:19:14.252791+0000","last_clean":"2024-03-29T08:19:14.252791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:14.252791+0000","last_undegraded":"2024-03-29T08:19:14.252791+0000","last_fullsized":"2024-03-29T08:19:14.252791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701079+0000","last_change":"2024-03-29T08:19:14.287716+0000","last_active":"2024-03-29T08:19:14.701079+0000","last_peered":"2024-03-29T08:19:14.701079+0000","last_clean":"2024-03-29T08:19:14.701079+0000","last_became_active":"2024-03-29T08:19:14.287437+0000","last_became_peered":"2024-03-29T08:19:14.287437+0000","last_unstale":"2024-03-29T08:19:14.701079+0000","last_undegraded":"2024-03-29T08:19:14.701079+0000","last_fullsized":"2024-03-29T08:19:14.701079+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:08:57.515896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694406+0000","last_change":"2024-03-29T08:19:14.689798+0000","last_active":"2024-03-29T08:19:14.694406+0000","last_peered":"2024-03-29T08:19:14.694406+0000","last_clean":"2024-03-29T08:19:14.694406+0000","last_became_active":"2024-03-29T08:19:14.689479+0000","last_became_peered":"2024-03-29T08:19:14.689479+0000","last_unstale":"2024-03-29T08:19:14.694406+0000","last_undegraded":"2024-03-29T08:19:14.694406+0000","last_fullsized":"2024-03-29T08:19:14.694406+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:20:03.888899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.667785+0000","last_change":"2024-03-29T08:19:14.281935+0000","last_active":"2024-03-29T08:19:14.667785+0000","last_peered":"2024-03-29T08:19:14.667785+0000","last_clean":"2024-03-29T08:19:14.667785+0000","last_became_active":"2024-03-29T08:19:14.281670+0000","last_became_peered":"2024-03-29T08:19:14.281670+0000","last_unstale":"2024-03-29T08:19:14.667785+0000","last_undegraded":"2024-03-29T08:19:14.667785+0000","last_fullsized":"2024-03-29T08:19:14.667785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:49:03.171063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.698797+0000","last_change":"2024-03-29T08:19:14.689677+0000","last_active":"2024-03-29T08:19:14.698797+0000","last_peered":"2024-03-29T08:19:14.698797+0000","last_clean":"2024-03-29T08:19:14.698797+0000","last_became_active":"2024-03-29T08:19:14.689343+0000","last_became_peered":"2024-03-29T08:19:14.689343+0000","last_unstale":"2024-03-29T08:19:14.698797+0000","last_undegraded":"2024-03-29T08:19:14.698797+0000","last_fullsized":"2024-03-29T08:19:14.698797+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:55.247391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-29T08:19:13.260439+0000","last_change":"2024-03-29T08:19:13.260439+0000","last_active":"2024-03-29T08:19:13.246882+0000","last_peered":"2024-03-29T08:19:13.246882+0000","last_clean":"2024-03-29T08:19:13.246882+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:13.260439+0000","last_undegraded":"2024-03-29T08:19:13.260439+0000","last_fullsized":"2024-03-29T08:19:13.260439+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:27:45.403801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:19:14.252791+0000","last_change":"2024-03-29T08:19:14.252791+0000","last_active":"2024-03-29T08:19:14.252791+0000","last_peered":"2024-03-29T08:19:14.252791+0000","last_clean":"2024-03-29T08:19:14.252791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:14.252791+0000","last_undegraded":"2024-03-29T08:19:14.252791+0000","last_fullsized":"2024-03-29T08:19:14.252791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.673192+0000","last_change":"2024-03-29T08:19:14.272023+0000","last_active":"2024-03-29T08:19:14.673192+0000","last_peered":"2024-03-29T08:19:14.673192+0000","last_clean":"2024-03-29T08:19:14.673192+0000","last_became_active":"2024-03-29T08:19:14.271722+0000","last_became_peered":"2024-03-29T08:19:14.271722+0000","last_unstale":"2024-03-29T08:19:14.673192+0000","last_undegraded":"2024-03-29T08:19:14.673192+0000","last_fullsized":"2024-03-29T08:19:14.673192+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:26.662774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.699233+0000","last_change":"2024-03-29T08:19:14.694537+0000","last_active":"2024-03-29T08:19:14.699233+0000","last_peered":"2024-03-29T08:19:14.699233+0000","last_clean":"2024-03-29T08:19:14.699233+0000","last_became_active":"2024-03-29T08:19:14.694296+0000","last_became_peered":"2024-03-29T08:19:14.694296+0000","last_unstale":"2024-03-29T08:19:14.699233+0000","last_undegraded":"2024-03-29T08:19:14.699233+0000","last_fullsized":"2024-03-29T08:19:14.699233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:37:16.877010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.673618+0000","last_change":"2024-03-29T08:19:14.285590+0000","last_active":"2024-03-29T08:19:14.673618+0000","last_peered":"2024-03-29T08:19:14.673618+0000","last_clean":"2024-03-29T08:19:14.673618+0000","last_became_active":"2024-03-29T08:19:14.285211+0000","last_became_peered":"2024-03-29T08:19:14.285211+0000","last_unstale":"2024-03-29T08:19:14.673618+0000","last_undegraded":"2024-03-29T08:19:14.673618+0000","last_fullsized":"2024-03-29T08:19:14.673618+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:52:20.153715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.682877+0000","last_change":"2024-03-29T08:19:14.679560+0000","last_active":"2024-03-29T08:19:14.682877+0000","last_peered":"2024-03-29T08:19:14.682877+0000","last_clean":"2024-03-29T08:19:14.682877+0000","last_became_active":"2024-03-29T08:19:14.677544+0000","last_became_peered":"2024-03-29T08:19:14.677544+0000","last_unstale":"2024-03-29T08:19:14.682877+0000","last_undegraded":"2024-03-29T08:19:14.682877+0000","last_fullsized":"2024-03-29T08:19:14.682877+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:34.410446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.683328+0000","last_change":"2024-03-29T08:19:14.281813+0000","last_active":"2024-03-29T08:19:14.683328+0000","last_peered":"2024-03-29T08:19:14.683328+0000","last_clean":"2024-03-29T08:19:14.683328+0000","last_became_active":"2024-03-29T08:19:14.281356+0000","last_became_peered":"2024-03-29T08:19:14.281356+0000","last_unstale":"2024-03-29T08:19:14.683328+0000","last_undegraded":"2024-03-29T08:19:14.683328+0000","last_fullsized":"2024-03-29T08:19:14.683328+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:04:28.023637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.697297+0000","last_change":"2024-03-29T08:19:14.691637+0000","last_active":"2024-03-29T08:19:14.697297+0000","last_peered":"2024-03-29T08:19:14.697297+0000","last_clean":"2024-03-29T08:19:14.697297+0000","last_became_active":"2024-03-29T08:19:14.691391+0000","last_became_peered":"2024-03-29T08:19:14.691391+0000","last_unstale":"2024-03-29T08:19:14.697297+0000","last_undegraded":"2024-03-29T08:19:14.697297+0000","last_fullsized":"2024-03-29T08:19:14.697297+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:11:13.363737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.697667+0000","last_change":"2024-03-29T08:19:14.286350+0000","last_active":"2024-03-29T08:19:14.697667+0000","last_peered":"2024-03-29T08:19:14.697667+0000","last_clean":"2024-03-29T08:19:14.697667+0000","last_became_active":"2024-03-29T08:19:14.286055+0000","last_became_peered":"2024-03-29T08:19:14.286055+0000","last_unstale":"2024-03-29T08:19:14.697667+0000","last_undegraded":"2024-03-29T08:19:14.697667+0000","last_fullsized":"2024-03-29T08:19:14.697667+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:35.048072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700275+0000","last_change":"2024-03-29T08:19:14.694685+0000","last_active":"2024-03-29T08:19:14.700275+0000","last_peered":"2024-03-29T08:19:14.700275+0000","last_clean":"2024-03-29T08:19:14.700275+0000","last_became_active":"2024-03-29T08:19:14.693892+0000","last_became_peered":"2024-03-29T08:19:14.693892+0000","last_unstale":"2024-03-29T08:19:14.700275+0000","last_undegraded":"2024-03-29T08:19:14.700275+0000","last_fullsized":"2024-03-29T08:19:14.700275+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:02:39.788723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-29T08:19:13.260969+0000","last_change":"2024-03-29T08:19:13.260969+0000","last_active":"2024-03-29T08:19:13.246882+0000","last_peered":"2024-03-29T08:19:13.246882+0000","last_clean":"2024-03-29T08:19:13.246882+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:13.260969+0000","last_undegraded":"2024-03-29T08:19:13.260969+0000","last_fullsized":"2024-03-29T08:19:13.260969+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:59:27.620393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705394+0000","last_change":"2024-03-29T08:19:14.701490+0000","last_active":"2024-03-29T08:19:14.705394+0000","last_peered":"2024-03-29T08:19:14.705394+0000","last_clean":"2024-03-29T08:19:14.705394+0000","last_became_active":"2024-03-29T08:19:14.701146+0000","last_became_peered":"2024-03-29T08:19:14.701146+0000","last_unstale":"2024-03-29T08:19:14.705394+0000","last_undegraded":"2024-03-29T08:19:14.705394+0000","last_fullsized":"2024-03-29T08:19:14.705394+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:32:52.080327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.676163+0000","last_change":"2024-03-29T08:19:14.279625+0000","last_active":"2024-03-29T08:19:14.676163+0000","last_peered":"2024-03-29T08:19:14.676163+0000","last_clean":"2024-03-29T08:19:14.676163+0000","last_became_active":"2024-03-29T08:19:14.279364+0000","last_became_peered":"2024-03-29T08:19:14.279364+0000","last_unstale":"2024-03-29T08:19:14.676163+0000","last_undegraded":"2024-03-29T08:19:14.676163+0000","last_fullsized":"2024-03-29T08:19:14.676163+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:20:02.882991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.685821+0000","last_change":"2024-03-29T08:19:14.679991+0000","last_active":"2024-03-29T08:19:14.685821+0000","last_peered":"2024-03-29T08:19:14.685821+0000","last_clean":"2024-03-29T08:19:14.685821+0000","last_became_active":"2024-03-29T08:19:14.679701+0000","last_became_peered":"2024-03-29T08:19:14.679701+0000","last_unstale":"2024-03-29T08:19:14.685821+0000","last_undegraded":"2024-03-29T08:19:14.685821+0000","last_fullsized":"2024-03-29T08:19:14.685821+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:08.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.675477+0000","last_change":"2024-03-29T08:19:14.284652+0000","last_active":"2024-03-29T08:19:14.675477+0000","last_peered":"2024-03-29T08:19:14.675477+0000","last_clean":"2024-03-29T08:19:14.675477+0000","last_became_active":"2024-03-29T08:19:14.284376+0000","last_became_peered":"2024-03-29T08:19:14.284376+0000","last_unstale":"2024-03-29T08:19:14.675477+0000","last_undegraded":"2024-03-29T08:19:14.675477+0000","last_fullsized":"2024-03-29T08:19:14.675477+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:20.730956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701325+0000","last_change":"2024-03-29T08:19:14.693198+0000","last_active":"2024-03-29T08:19:14.701325+0000","last_peered":"2024-03-29T08:19:14.701325+0000","last_clean":"2024-03-29T08:19:14.701325+0000","last_became_active":"2024-03-29T08:19:14.692849+0000","last_became_peered":"2024-03-29T08:19:14.692849+0000","last_unstale":"2024-03-29T08:19:14.701325+0000","last_undegraded":"2024-03-29T08:19:14.701325+0000","last_fullsized":"2024-03-29T08:19:14.701325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:46.218186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.676556+0000","last_change":"2024-03-29T08:19:14.284484+0000","last_active":"2024-03-29T08:19:14.676556+0000","last_peered":"2024-03-29T08:19:14.676556+0000","last_clean":"2024-03-29T08:19:14.676556+0000","last_became_active":"2024-03-29T08:19:14.284145+0000","last_became_peered":"2024-03-29T08:19:14.284145+0000","last_unstale":"2024-03-29T08:19:14.676556+0000","last_undegraded":"2024-03-29T08:19:14.676556+0000","last_fullsized":"2024-03-29T08:19:14.676556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:33.404538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702578+0000","last_change":"2024-03-29T08:19:14.695365+0000","last_active":"2024-03-29T08:19:14.702578+0000","last_peered":"2024-03-29T08:19:14.702578+0000","last_clean":"2024-03-29T08:19:14.702578+0000","last_became_active":"2024-03-29T08:19:14.695111+0000","last_became_peered":"2024-03-29T08:19:14.695111+0000","last_unstale":"2024-03-29T08:19:14.702578+0000","last_undegraded":"2024-03-29T08:19:14.702578+0000","last_fullsized":"2024-03-29T08:19:14.702578+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:15.077172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.672171+0000","last_change":"2024-03-29T08:19:14.270630+0000","last_active":"2024-03-29T08:19:14.672171+0000","last_peered":"2024-03-29T08:19:14.672171+0000","last_clean":"2024-03-29T08:19:14.672171+0000","last_became_active":"2024-03-29T08:19:14.270379+0000","last_became_peered":"2024-03-29T08:19:14.270379+0000","last_unstale":"2024-03-29T08:19:14.672171+0000","last_undegraded":"2024-03-29T08:19:14.672171+0000","last_fullsized":"2024-03-29T08:19:14.672171+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:30:18.530436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700408+0000","last_change":"2024-03-29T08:19:14.698728+0000","last_active":"2024-03-29T08:19:14.700408+0000","last_peered":"2024-03-29T08:19:14.700408+0000","last_clean":"2024-03-29T08:19:14.700408+0000","last_became_active":"2024-03-29T08:19:14.698138+0000","last_became_peered":"2024-03-29T08:19:14.698138+0000","last_unstale":"2024-03-29T08:19:14.700408+0000","last_undegraded":"2024-03-29T08:19:14.700408+0000","last_fullsized":"2024-03-29T08:19:14.700408+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:07:30.640723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.683645+0000","last_change":"2024-03-29T08:19:14.281517+0000","last_active":"2024-03-29T08:19:14.683645+0000","last_peered":"2024-03-29T08:19:14.683645+0000","last_clean":"2024-03-29T08:19:14.683645+0000","last_became_active":"2024-03-29T08:19:14.281182+0000","last_became_peered":"2024-03-29T08:19:14.281182+0000","last_unstale":"2024-03-29T08:19:14.683645+0000","last_undegraded":"2024-03-29T08:19:14.683645+0000","last_fullsized":"2024-03-29T08:19:14.683645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:21:54.574176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.689204+0000","last_change":"2024-03-29T08:19:14.685120+0000","last_active":"2024-03-29T08:19:14.689204+0000","last_peered":"2024-03-29T08:19:14.689204+0000","last_clean":"2024-03-29T08:19:14.689204+0000","last_became_active":"2024-03-29T08:19:14.682875+0000","last_became_peered":"2024-03-29T08:19:14.682875+0000","last_unstale":"2024-03-29T08:19:14.689204+0000","last_undegraded":"2024-03-29T08:19:14.689204+0000","last_fullsized":"2024-03-29T08:19:14.689204+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:55:24.587078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.683597+0000","last_change":"2024-03-29T08:19:14.283788+0000","last_active":"2024-03-29T08:19:14.683597+0000","last_peered":"2024-03-29T08:19:14.683597+0000","last_clean":"2024-03-29T08:19:14.683597+0000","last_became_active":"2024-03-29T08:19:14.283586+0000","last_became_peered":"2024-03-29T08:19:14.283586+0000","last_unstale":"2024-03-29T08:19:14.683597+0000","last_undegraded":"2024-03-29T08:19:14.683597+0000","last_fullsized":"2024-03-29T08:19:14.683597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:52:19.059268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":173,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.282014+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:19:14.282014+0000","last_peered":"2024-03-29T08:19:14.282014+0000","last_clean":"2024-03-29T08:19:14.282014+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:19:14.282014+0000","last_undegraded":"2024-03-29T08:19:14.282014+0000","last_fullsized":"2024-03-29T08:19:14.282014+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":118,"acting":118,"num_store_stats":7},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":150465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68799999999999994}]},{"osd":1,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.79800000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.78300000000000003}]},{"osd":3,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.88600000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.81299999999999994}]},{"osd":5,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.873},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.90700000000000003}]}]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":143668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55800000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.371}]},{"osd":5,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58499999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.56499999999999995}]}]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":137393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.1709999999999998,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.81399999999999995}]},{"osd":1,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.363,"5min":2.363,"15min":2.363},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.93899999999999995}]},{"osd":3,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.89600000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.86799999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.91500000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.99199999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":376832,"data_stored":143668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.35899999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.34200000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.437}]},{"osd":4,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.70599999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.73199999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.66100000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":0.83399999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":376832,"data_stored":143668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.44}]},{"osd":1,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.54500000000000004}]},{"osd":4,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.502}]},{"osd":5,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.64500000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.625}]},{"osd":7,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.63600000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":987136,"data_stored":740881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":4.2539999999999996,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.1909999999999998,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.42299999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51700000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.627}]},{"osd":5,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.76800000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":364544,"data_stored":137393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":3.9359999999999999,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.497},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":8.0419999999999998,"5min":8.0419999999999998,"15min":8.0419999999999998},"last":0.51000000000000001}]},{"osd":2,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":4.907,"5min":4.907,"15min":4.907},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":7.9569999999999999,"5min":7.9569999999999999,"15min":7.9569999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.44400000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":4.4720000000000004,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.625},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":4.3920000000000003,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.58299999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.3200000000000003,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":4.4100000000000001,"5min":4.4100000000000001,"15min":4.4100000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":4.5209999999999999,"5min":4.5209999999999999,"15min":4.5209999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":4.4420000000000002,"5min":4.4420000000000002,"15min":4.4420000000000002},"last":0.63900000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":727761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.39800000000000002}]},{"osd":1,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.53000000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.80200000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.48499999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.439},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.55500000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.59099999999999997}]},{"osd":7,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.68300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:19:19.177 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:19:19.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.352+0000 7f6145a00640 1 Processor -- start 2024-03-29T08:19:19.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.352+0000 7f6145a00640 1 -- start start 2024-03-29T08:19:19.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.356+0000 7f6145a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6140153d50 0x7f61401611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:19.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.356+0000 7f6145a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614015d870 con 0x7f61400ad3e0 2024-03-29T08:19:19.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.356+0000 7f6145a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614015db50 con 0x7f6140153970 2024-03-29T08:19:19.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.356+0000 7f6145a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614015de30 con 0x7f6140153d50 2024-03-29T08:19:19.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.356+0000 7f6137e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6140153d50 0x7f61401611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:19.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.356+0000 7f6137e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6140153d50 0x7f61401611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34806/0 (socket says 172.21.15.46:34806) 2024-03-29T08:19:19.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.356+0000 7f6137e00640 1 -- 172.21.15.46:0/3154393500 learned_addr learned my addr 172.21.15.46:0/3154393500 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:19.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.356+0000 7f613f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f61400ad3e0 0x7f61400ac3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56110/0 (socket says 172.21.15.46:56110) 2024-03-29T08:19:19.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.356+0000 7f6137e00640 1 -- 172.21.15.46:0/3154393500 >> v1:172.21.15.73:6789/0 conn(0x7f6140153970 legacy=0x7f614015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:19.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.356+0000 7f6137e00640 1 -- 172.21.15.46:0/3154393500 >> v1:172.21.15.46:6789/0 conn(0x7f61400ad3e0 legacy=0x7f61400ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:19.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.356+0000 7f6137e00640 1 -- 172.21.15.46:0/3154393500 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61400abea0 con 0x7f6140153d50 2024-03-29T08:19:19.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.356+0000 7f613e000640 1 -- 172.21.15.46:0/3154393500 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034881059 0 0) 0x7f614015d870 con 0x7f61400ad3e0 2024-03-29T08:19:19.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.356+0000 7f6137e00640 1 --2- 172.21.15.46:0/3154393500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6140153d50 0x7f61401611e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6128009870 tx=0x7f612801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bb344841de99af server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f613e000640 1 -- 172.21.15.46:0/3154393500 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61280ab670 con 0x7f6140153d50 2024-03-29T08:19:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f613e000640 1 -- 172.21.15.46:0/3154393500 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61280ab7d0 con 0x7f6140153d50 2024-03-29T08:19:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f613e000640 1 -- 172.21.15.46:0/3154393500 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61280abb60 con 0x7f6140153d50 2024-03-29T08:19:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f6145a00640 1 -- 172.21.15.46:0/3154393500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6140153d50 msgr2=0x7f61401611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f6145a00640 1 --2- 172.21.15.46:0/3154393500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6140153d50 0x7f61401611e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6128009870 tx=0x7f612801c560 comp rx=0 tx=0).stop 2024-03-29T08:19:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f6145a00640 1 -- 172.21.15.46:0/3154393500 shutdown_connections 2024-03-29T08:19:19.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f6145a00640 1 --2- 172.21.15.46:0/3154393500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6140153d50 0x7f61401611e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f6145a00640 1 -- 172.21.15.46:0/3154393500 >> 172.21.15.46:0/3154393500 conn(0x7f6140066110 msgr2=0x7f6140069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:19.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f6145a00640 1 -- 172.21.15.46:0/3154393500 shutdown_connections 2024-03-29T08:19:19.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f6145a00640 1 -- 172.21.15.46:0/3154393500 wait complete. 2024-03-29T08:19:19.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f6145a00640 1 Processor -- start 2024-03-29T08:19:19.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f6145a00640 1 -- start start 2024-03-29T08:19:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f6145a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61400ad3e0 0x7f614014c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f6145a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6140153970 0x7f614014cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f6145a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614014d3f0 0x7f6140146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f6145a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61400cc0b0 con 0x7f614014d3f0 2024-03-29T08:19:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f6145a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61400cc390 con 0x7f6140153970 2024-03-29T08:19:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f6145a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61400cc670 con 0x7f61400ad3e0 2024-03-29T08:19:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f613f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614014d3f0 0x7f6140146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f613f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614014d3f0 0x7f6140146860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34820/0 (socket says 172.21.15.46:34820) 2024-03-29T08:19:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f613f400640 1 -- 172.21.15.46:0/2928709452 learned_addr learned my addr 172.21.15.46:0/2928709452 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f613ea00640 1 --2- 172.21.15.46:0/2928709452 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61400ad3e0 0x7f614014c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f6137e00640 1 --2- 172.21.15.46:0/2928709452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6140153970 0x7f614014cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f613f400640 1 -- 172.21.15.46:0/2928709452 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61400ad3e0 msgr2=0x7f614014c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.360+0000 7f613f400640 1 --2- 172.21.15.46:0/2928709452 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61400ad3e0 0x7f614014c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.364+0000 7f613f400640 1 -- 172.21.15.46:0/2928709452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6140153970 msgr2=0x7f614014cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.364+0000 7f613f400640 1 --2- 172.21.15.46:0/2928709452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6140153970 0x7f614014cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.364+0000 7f613f400640 1 -- 172.21.15.46:0/2928709452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61280094d0 con 0x7f614014d3f0 2024-03-29T08:19:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.364+0000 7f6137e00640 1 --2- 172.21.15.46:0/2928709452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6140153970 0x7f614014cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.364+0000 7f613ea00640 1 --2- 172.21.15.46:0/2928709452 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61400ad3e0 0x7f614014c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.364+0000 7f613f400640 1 --2- 172.21.15.46:0/2928709452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614014d3f0 0x7f6140146860 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f613800c7a0 tx=0x7f613800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.364+0000 7f6136a00640 1 -- 172.21.15.46:0/2928709452 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613800a640 con 0x7f614014d3f0 2024-03-29T08:19:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.364+0000 7f6136a00640 1 -- 172.21.15.46:0/2928709452 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f613800a7a0 con 0x7f614014d3f0 2024-03-29T08:19:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.364+0000 7f6136a00640 1 -- 172.21.15.46:0/2928709452 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f613800abd0 con 0x7f614014d3f0 2024-03-29T08:19:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.364+0000 7f6145a00640 1 -- 172.21.15.46:0/2928709452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61400cc950 con 0x7f614014d3f0 2024-03-29T08:19:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.364+0000 7f6145a00640 1 -- 172.21.15.46:0/2928709452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61400cbcd0 con 0x7f614014d3f0 2024-03-29T08:19:19.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.364+0000 7f6145a00640 1 -- 172.21.15.46:0/2928709452 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61401493b0 con 0x7f614014d3f0 2024-03-29T08:19:19.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.364+0000 7f6136a00640 1 -- 172.21.15.46:0/2928709452 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f613800ad30 con 0x7f614014d3f0 2024-03-29T08:19:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.364+0000 7f6136a00640 1 --2- 172.21.15.46:0/2928709452 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f611c066690 0x7f611c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.364+0000 7f6136a00640 1 -- 172.21.15.46:0/2928709452 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61380837d0 con 0x7f614014d3f0 2024-03-29T08:19:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.364+0000 7f613ea00640 1 --2- 172.21.15.46:0/2928709452 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f611c066690 0x7f611c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:19.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.368+0000 7f613ea00640 1 --2- 172.21.15.46:0/2928709452 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f611c066690 0x7f611c068b50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6130000f70 tx=0x7f6130009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.368+0000 7f6136a00640 1 -- 172.21.15.46:0/2928709452 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f61401493b0 con 0x7f614014d3f0 2024-03-29T08:19:19.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.528+0000 7f6145a00640 1 -- 172.21.15.46:0/2928709452 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f61401493b0 con 0x7f611c066690 2024-03-29T08:19:19.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.536+0000 7f6136a00640 1 -- 172.21.15.46:0/2928709452 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288161 (secure 0 0 0) 0x7f61401493b0 con 0x7f611c066690 2024-03-29T08:19:19.541 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:19:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.540+0000 7f6145a00640 1 -- 172.21.15.46:0/2928709452 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f611c066690 msgr2=0x7f611c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.540+0000 7f6145a00640 1 --2- 172.21.15.46:0/2928709452 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f611c066690 0x7f611c068b50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6130000f70 tx=0x7f6130009040 comp rx=0 tx=0).stop 2024-03-29T08:19:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.540+0000 7f6145a00640 1 -- 172.21.15.46:0/2928709452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614014d3f0 msgr2=0x7f6140146860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.540+0000 7f6145a00640 1 --2- 172.21.15.46:0/2928709452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614014d3f0 0x7f6140146860 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f613800c7a0 tx=0x7f613800cc60 comp rx=0 tx=0).stop 2024-03-29T08:19:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.540+0000 7f6145a00640 1 -- 172.21.15.46:0/2928709452 shutdown_connections 2024-03-29T08:19:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.540+0000 7f6145a00640 1 --2- 172.21.15.46:0/2928709452 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f611c066690 0x7f611c068b50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.540+0000 7f6145a00640 1 --2- 172.21.15.46:0/2928709452 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6140153970 0x7f614014cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.540+0000 7f6145a00640 1 --2- 172.21.15.46:0/2928709452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614014d3f0 0x7f6140146860 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.540+0000 7f6145a00640 1 --2- 172.21.15.46:0/2928709452 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61400ad3e0 0x7f614014c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.540+0000 7f6145a00640 1 -- 172.21.15.46:0/2928709452 >> 172.21.15.46:0/2928709452 conn(0x7f6140066110 msgr2=0x7f614014b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.540+0000 7f6145a00640 1 -- 172.21.15.46:0/2928709452 shutdown_connections 2024-03-29T08:19:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.540+0000 7f6145a00640 1 -- 172.21.15.46:0/2928709452 wait complete. 2024-03-29T08:19:19.561 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-03-29T08:19:18.057099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153,"ondisk_log_size":153,"up":184,"acting":184,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236656,"kb_used_data":4100,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738064,"statfs":{"total":773094113280,"available":772851777536,"internally_reserved":0,"allocated":4198400,"data_stored":2324897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.804539"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.679129+0000","last_change":"2024-03-29T08:19:14.677048+0000","last_active":"2024-03-29T08:19:14.679129+0000","last_peered":"2024-03-29T08:19:14.679129+0000","last_clean":"2024-03-29T08:19:14.679129+0000","last_became_active":"2024-03-29T08:19:14.676802+0000","last_became_peered":"2024-03-29T08:19:14.676802+0000","last_unstale":"2024-03-29T08:19:14.679129+0000","last_undegraded":"2024-03-29T08:19:14.679129+0000","last_fullsized":"2024-03-29T08:19:14.679129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:37:16.877010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:19:14.252791+0000","last_change":"2024-03-29T08:19:14.252791+0000","last_active":"2024-03-29T08:19:14.252791+0000","last_peered":"2024-03-29T08:19:14.252791+0000","last_clean":"2024-03-29T08:19:14.252791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:14.252791+0000","last_undegraded":"2024-03-29T08:19:14.252791+0000","last_fullsized":"2024-03-29T08:19:14.252791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.681839+0000","last_change":"2024-03-29T08:19:14.677189+0000","last_active":"2024-03-29T08:19:14.681839+0000","last_peered":"2024-03-29T08:19:14.681839+0000","last_clean":"2024-03-29T08:19:14.681839+0000","last_became_active":"2024-03-29T08:19:14.676526+0000","last_became_peered":"2024-03-29T08:19:14.676526+0000","last_unstale":"2024-03-29T08:19:14.681839+0000","last_undegraded":"2024-03-29T08:19:14.681839+0000","last_fullsized":"2024-03-29T08:19:14.681839+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:20.587889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705567+0000","last_change":"2024-03-29T08:19:14.699305+0000","last_active":"2024-03-29T08:19:14.705567+0000","last_peered":"2024-03-29T08:19:14.705567+0000","last_clean":"2024-03-29T08:19:14.705567+0000","last_became_active":"2024-03-29T08:19:14.698637+0000","last_became_peered":"2024-03-29T08:19:14.698637+0000","last_unstale":"2024-03-29T08:19:14.705567+0000","last_undegraded":"2024-03-29T08:19:14.705567+0000","last_fullsized":"2024-03-29T08:19:14.705567+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:35.234827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:19:14.252791+0000","last_change":"2024-03-29T08:19:14.252791+0000","last_active":"2024-03-29T08:19:14.252791+0000","last_peered":"2024-03-29T08:19:14.252791+0000","last_clean":"2024-03-29T08:19:14.252791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:14.252791+0000","last_undegraded":"2024-03-29T08:19:14.252791+0000","last_fullsized":"2024-03-29T08:19:14.252791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694387+0000","last_change":"2024-03-29T08:19:14.691105+0000","last_active":"2024-03-29T08:19:14.694387+0000","last_peered":"2024-03-29T08:19:14.694387+0000","last_clean":"2024-03-29T08:19:14.694387+0000","last_became_active":"2024-03-29T08:19:14.690486+0000","last_became_peered":"2024-03-29T08:19:14.690486+0000","last_unstale":"2024-03-29T08:19:14.694387+0000","last_undegraded":"2024-03-29T08:19:14.694387+0000","last_fullsized":"2024-03-29T08:19:14.694387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:21.736865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.678975+0000","last_change":"2024-03-29T08:19:14.675543+0000","last_active":"2024-03-29T08:19:14.678975+0000","last_peered":"2024-03-29T08:19:14.678975+0000","last_clean":"2024-03-29T08:19:14.678975+0000","last_became_active":"2024-03-29T08:19:14.675201+0000","last_became_peered":"2024-03-29T08:19:14.675201+0000","last_unstale":"2024-03-29T08:19:14.678975+0000","last_undegraded":"2024-03-29T08:19:14.678975+0000","last_fullsized":"2024-03-29T08:19:14.678975+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:08:58.521804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687632+0000","last_change":"2024-03-29T08:19:14.681677+0000","last_active":"2024-03-29T08:19:14.687632+0000","last_peered":"2024-03-29T08:19:14.687632+0000","last_clean":"2024-03-29T08:19:14.687632+0000","last_became_active":"2024-03-29T08:19:14.680707+0000","last_became_peered":"2024-03-29T08:19:14.680707+0000","last_unstale":"2024-03-29T08:19:14.687632+0000","last_undegraded":"2024-03-29T08:19:14.687632+0000","last_fullsized":"2024-03-29T08:19:14.687632+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:04:29.029546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.704743+0000","last_change":"2024-03-29T08:19:14.703892+0000","last_active":"2024-03-29T08:19:14.704743+0000","last_peered":"2024-03-29T08:19:14.704743+0000","last_clean":"2024-03-29T08:19:14.704743+0000","last_became_active":"2024-03-29T08:19:14.701909+0000","last_became_peered":"2024-03-29T08:19:14.701909+0000","last_unstale":"2024-03-29T08:19:14.704743+0000","last_undegraded":"2024-03-29T08:19:14.704743+0000","last_fullsized":"2024-03-29T08:19:14.704743+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:57:15.587273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701010+0000","last_change":"2024-03-29T08:19:14.694312+0000","last_active":"2024-03-29T08:19:14.701010+0000","last_peered":"2024-03-29T08:19:14.701010+0000","last_clean":"2024-03-29T08:19:14.701010+0000","last_became_active":"2024-03-29T08:19:14.694027+0000","last_became_peered":"2024-03-29T08:19:14.694027+0000","last_unstale":"2024-03-29T08:19:14.701010+0000","last_undegraded":"2024-03-29T08:19:14.701010+0000","last_fullsized":"2024-03-29T08:19:14.701010+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:15:31.529776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.704807+0000","last_change":"2024-03-29T08:19:14.703541+0000","last_active":"2024-03-29T08:19:14.704807+0000","last_peered":"2024-03-29T08:19:14.704807+0000","last_clean":"2024-03-29T08:19:14.704807+0000","last_became_active":"2024-03-29T08:19:14.703317+0000","last_became_peered":"2024-03-29T08:19:14.703317+0000","last_unstale":"2024-03-29T08:19:14.704807+0000","last_undegraded":"2024-03-29T08:19:14.704807+0000","last_fullsized":"2024-03-29T08:19:14.704807+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:18:37.057951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687173+0000","last_change":"2024-03-29T08:19:14.684087+0000","last_active":"2024-03-29T08:19:14.687173+0000","last_peered":"2024-03-29T08:19:14.687173+0000","last_clean":"2024-03-29T08:19:14.687173+0000","last_became_active":"2024-03-29T08:19:14.683827+0000","last_became_peered":"2024-03-29T08:19:14.683827+0000","last_unstale":"2024-03-29T08:19:14.687173+0000","last_undegraded":"2024-03-29T08:19:14.687173+0000","last_fullsized":"2024-03-29T08:19:14.687173+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:21.736865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702127+0000","last_change":"2024-03-29T08:19:14.699133+0000","last_active":"2024-03-29T08:19:14.702127+0000","last_peered":"2024-03-29T08:19:14.702127+0000","last_clean":"2024-03-29T08:19:14.702127+0000","last_became_active":"2024-03-29T08:19:14.698579+0000","last_became_peered":"2024-03-29T08:19:14.698579+0000","last_unstale":"2024-03-29T08:19:14.702127+0000","last_undegraded":"2024-03-29T08:19:14.702127+0000","last_fullsized":"2024-03-29T08:19:14.702127+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:00:07.571380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702206+0000","last_change":"2024-03-29T08:19:14.694580+0000","last_active":"2024-03-29T08:19:14.702206+0000","last_peered":"2024-03-29T08:19:14.702206+0000","last_clean":"2024-03-29T08:19:14.702206+0000","last_became_active":"2024-03-29T08:19:14.694305+0000","last_became_peered":"2024-03-29T08:19:14.694305+0000","last_unstale":"2024-03-29T08:19:14.702206+0000","last_undegraded":"2024-03-29T08:19:14.702206+0000","last_fullsized":"2024-03-29T08:19:14.702206+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:19:13.272178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.676292+0000","last_change":"2024-03-29T08:19:14.674438+0000","last_active":"2024-03-29T08:19:14.676292+0000","last_peered":"2024-03-29T08:19:14.676292+0000","last_clean":"2024-03-29T08:19:14.676292+0000","last_became_active":"2024-03-29T08:19:14.674172+0000","last_became_peered":"2024-03-29T08:19:14.674172+0000","last_unstale":"2024-03-29T08:19:14.676292+0000","last_undegraded":"2024-03-29T08:19:14.676292+0000","last_fullsized":"2024-03-29T08:19:14.676292+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:29.147117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687494+0000","last_change":"2024-03-29T08:19:14.681624+0000","last_active":"2024-03-29T08:19:14.687494+0000","last_peered":"2024-03-29T08:19:14.687494+0000","last_clean":"2024-03-29T08:19:14.687494+0000","last_became_active":"2024-03-29T08:19:14.681290+0000","last_became_peered":"2024-03-29T08:19:14.681290+0000","last_unstale":"2024-03-29T08:19:14.687494+0000","last_undegraded":"2024-03-29T08:19:14.687494+0000","last_fullsized":"2024-03-29T08:19:14.687494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:34:59.393546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700295+0000","last_change":"2024-03-29T08:19:14.695305+0000","last_active":"2024-03-29T08:19:14.700295+0000","last_peered":"2024-03-29T08:19:14.700295+0000","last_clean":"2024-03-29T08:19:14.700295+0000","last_became_active":"2024-03-29T08:19:14.694942+0000","last_became_peered":"2024-03-29T08:19:14.694942+0000","last_unstale":"2024-03-29T08:19:14.700295+0000","last_undegraded":"2024-03-29T08:19:14.700295+0000","last_fullsized":"2024-03-29T08:19:14.700295+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:19:41.115142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694349+0000","last_change":"2024-03-29T08:19:14.690840+0000","last_active":"2024-03-29T08:19:14.694349+0000","last_peered":"2024-03-29T08:19:14.694349+0000","last_clean":"2024-03-29T08:19:14.694349+0000","last_became_active":"2024-03-29T08:19:14.689554+0000","last_became_peered":"2024-03-29T08:19:14.689554+0000","last_unstale":"2024-03-29T08:19:14.694349+0000","last_undegraded":"2024-03-29T08:19:14.694349+0000","last_fullsized":"2024-03-29T08:19:14.694349+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:49:04.176971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694388+0000","last_change":"2024-03-29T08:19:14.689362+0000","last_active":"2024-03-29T08:19:14.694388+0000","last_peered":"2024-03-29T08:19:14.694388+0000","last_clean":"2024-03-29T08:19:14.694388+0000","last_became_active":"2024-03-29T08:19:14.689050+0000","last_became_peered":"2024-03-29T08:19:14.689050+0000","last_unstale":"2024-03-29T08:19:14.694388+0000","last_undegraded":"2024-03-29T08:19:14.694388+0000","last_fullsized":"2024-03-29T08:19:14.694388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:20:38.835695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.692494+0000","last_change":"2024-03-29T08:19:14.683957+0000","last_active":"2024-03-29T08:19:14.692494+0000","last_peered":"2024-03-29T08:19:14.692494+0000","last_clean":"2024-03-29T08:19:14.692494+0000","last_became_active":"2024-03-29T08:19:14.683710+0000","last_became_peered":"2024-03-29T08:19:14.683710+0000","last_unstale":"2024-03-29T08:19:14.692494+0000","last_undegraded":"2024-03-29T08:19:14.692494+0000","last_fullsized":"2024-03-29T08:19:14.692494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:11:49.840732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.689299+0000","last_change":"2024-03-29T08:19:14.685648+0000","last_active":"2024-03-29T08:19:14.689299+0000","last_peered":"2024-03-29T08:19:14.689299+0000","last_clean":"2024-03-29T08:19:14.689299+0000","last_became_active":"2024-03-29T08:19:14.685404+0000","last_became_peered":"2024-03-29T08:19:14.685404+0000","last_unstale":"2024-03-29T08:19:14.689299+0000","last_undegraded":"2024-03-29T08:19:14.689299+0000","last_fullsized":"2024-03-29T08:19:14.689299+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:14:08.867120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701342+0000","last_change":"2024-03-29T08:19:14.698732+0000","last_active":"2024-03-29T08:19:14.701342+0000","last_peered":"2024-03-29T08:19:14.701342+0000","last_clean":"2024-03-29T08:19:14.701342+0000","last_became_active":"2024-03-29T08:19:14.698484+0000","last_became_peered":"2024-03-29T08:19:14.698484+0000","last_unstale":"2024-03-29T08:19:14.701342+0000","last_undegraded":"2024-03-29T08:19:14.701342+0000","last_fullsized":"2024-03-29T08:19:14.701342+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:08.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.688435+0000","last_change":"2024-03-29T08:19:14.685174+0000","last_active":"2024-03-29T08:19:14.688435+0000","last_peered":"2024-03-29T08:19:14.688435+0000","last_clean":"2024-03-29T08:19:14.688435+0000","last_became_active":"2024-03-29T08:19:14.684605+0000","last_became_peered":"2024-03-29T08:19:14.684605+0000","last_unstale":"2024-03-29T08:19:14.688435+0000","last_undegraded":"2024-03-29T08:19:14.688435+0000","last_fullsized":"2024-03-29T08:19:14.688435+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:34:59.393546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.699334+0000","last_change":"2024-03-29T08:19:14.698117+0000","last_active":"2024-03-29T08:19:14.699334+0000","last_peered":"2024-03-29T08:19:14.699334+0000","last_clean":"2024-03-29T08:19:14.699334+0000","last_became_active":"2024-03-29T08:19:14.697861+0000","last_became_peered":"2024-03-29T08:19:14.697861+0000","last_unstale":"2024-03-29T08:19:14.699334+0000","last_undegraded":"2024-03-29T08:19:14.699334+0000","last_fullsized":"2024-03-29T08:19:14.699334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:14:08.867120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702337+0000","last_change":"2024-03-29T08:19:14.695925+0000","last_active":"2024-03-29T08:19:14.702337+0000","last_peered":"2024-03-29T08:19:14.702337+0000","last_clean":"2024-03-29T08:19:14.702337+0000","last_became_active":"2024-03-29T08:19:14.695180+0000","last_became_peered":"2024-03-29T08:19:14.695180+0000","last_unstale":"2024-03-29T08:19:14.702337+0000","last_undegraded":"2024-03-29T08:19:14.702337+0000","last_fullsized":"2024-03-29T08:19:14.702337+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:20:02.908287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702375+0000","last_change":"2024-03-29T08:19:14.696275+0000","last_active":"2024-03-29T08:19:14.702375+0000","last_peered":"2024-03-29T08:19:14.702375+0000","last_clean":"2024-03-29T08:19:14.702375+0000","last_became_active":"2024-03-29T08:19:14.696065+0000","last_became_peered":"2024-03-29T08:19:14.696065+0000","last_unstale":"2024-03-29T08:19:14.702375+0000","last_undegraded":"2024-03-29T08:19:14.702375+0000","last_fullsized":"2024-03-29T08:19:14.702375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:00:46.960177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:19:14.252791+0000","last_change":"2024-03-29T08:19:14.252791+0000","last_active":"2024-03-29T08:19:14.252791+0000","last_peered":"2024-03-29T08:19:14.252791+0000","last_clean":"2024-03-29T08:19:14.252791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:14.252791+0000","last_undegraded":"2024-03-29T08:19:14.252791+0000","last_fullsized":"2024-03-29T08:19:14.252791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.693704+0000","last_change":"2024-03-29T08:19:14.684317+0000","last_active":"2024-03-29T08:19:14.693704+0000","last_peered":"2024-03-29T08:19:14.693704+0000","last_clean":"2024-03-29T08:19:14.693704+0000","last_became_active":"2024-03-29T08:19:14.683125+0000","last_became_peered":"2024-03-29T08:19:14.683125+0000","last_unstale":"2024-03-29T08:19:14.693704+0000","last_undegraded":"2024-03-29T08:19:14.693704+0000","last_fullsized":"2024-03-29T08:19:14.693704+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:52:54.511385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702775+0000","last_change":"2024-03-29T08:19:14.701650+0000","last_active":"2024-03-29T08:19:14.702775+0000","last_peered":"2024-03-29T08:19:14.702775+0000","last_clean":"2024-03-29T08:19:14.702775+0000","last_became_active":"2024-03-29T08:19:14.697686+0000","last_became_peered":"2024-03-29T08:19:14.697686+0000","last_unstale":"2024-03-29T08:19:14.702775+0000","last_undegraded":"2024-03-29T08:19:14.702775+0000","last_fullsized":"2024-03-29T08:19:14.702775+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:10:23.783843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687174+0000","last_change":"2024-03-29T08:19:14.684872+0000","last_active":"2024-03-29T08:19:14.687174+0000","last_peered":"2024-03-29T08:19:14.687174+0000","last_clean":"2024-03-29T08:19:14.687174+0000","last_became_active":"2024-03-29T08:19:14.684622+0000","last_became_peered":"2024-03-29T08:19:14.684622+0000","last_unstale":"2024-03-29T08:19:14.687174+0000","last_undegraded":"2024-03-29T08:19:14.687174+0000","last_fullsized":"2024-03-29T08:19:14.687174+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:00:07.571380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687689+0000","last_change":"2024-03-29T08:19:14.681150+0000","last_active":"2024-03-29T08:19:14.687689+0000","last_peered":"2024-03-29T08:19:14.687689+0000","last_clean":"2024-03-29T08:19:14.687689+0000","last_became_active":"2024-03-29T08:19:14.680888+0000","last_became_peered":"2024-03-29T08:19:14.680888+0000","last_unstale":"2024-03-29T08:19:14.687689+0000","last_undegraded":"2024-03-29T08:19:14.687689+0000","last_fullsized":"2024-03-29T08:19:14.687689+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:10:23.783843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700291+0000","last_change":"2024-03-29T08:19:14.698385+0000","last_active":"2024-03-29T08:19:14.700291+0000","last_peered":"2024-03-29T08:19:14.700291+0000","last_clean":"2024-03-29T08:19:14.700291+0000","last_became_active":"2024-03-29T08:19:14.698119+0000","last_became_peered":"2024-03-29T08:19:14.698119+0000","last_unstale":"2024-03-29T08:19:14.700291+0000","last_undegraded":"2024-03-29T08:19:14.700291+0000","last_fullsized":"2024-03-29T08:19:14.700291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:03:28.287471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.692119+0000","last_change":"2024-03-29T08:19:14.686891+0000","last_active":"2024-03-29T08:19:14.692119+0000","last_peered":"2024-03-29T08:19:14.692119+0000","last_clean":"2024-03-29T08:19:14.692119+0000","last_became_active":"2024-03-29T08:19:14.686615+0000","last_became_peered":"2024-03-29T08:19:14.686615+0000","last_unstale":"2024-03-29T08:19:14.692119+0000","last_undegraded":"2024-03-29T08:19:14.692119+0000","last_fullsized":"2024-03-29T08:19:14.692119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:10:23.783843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701442+0000","last_change":"2024-03-29T08:19:14.278073+0000","last_active":"2024-03-29T08:19:14.701442+0000","last_peered":"2024-03-29T08:19:14.701442+0000","last_clean":"2024-03-29T08:19:14.701442+0000","last_became_active":"2024-03-29T08:19:14.277778+0000","last_became_peered":"2024-03-29T08:19:14.277778+0000","last_unstale":"2024-03-29T08:19:14.701442+0000","last_undegraded":"2024-03-29T08:19:14.701442+0000","last_fullsized":"2024-03-29T08:19:14.701442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:36:08.787710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687978+0000","last_change":"2024-03-29T08:19:14.682096+0000","last_active":"2024-03-29T08:19:14.687978+0000","last_peered":"2024-03-29T08:19:14.687978+0000","last_clean":"2024-03-29T08:19:14.687978+0000","last_became_active":"2024-03-29T08:19:14.681779+0000","last_became_peered":"2024-03-29T08:19:14.681779+0000","last_unstale":"2024-03-29T08:19:14.687978+0000","last_undegraded":"2024-03-29T08:19:14.687978+0000","last_fullsized":"2024-03-29T08:19:14.687978+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:52:20.065176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.703320+0000","last_change":"2024-03-29T08:19:14.281218+0000","last_active":"2024-03-29T08:19:14.703320+0000","last_peered":"2024-03-29T08:19:14.703320+0000","last_clean":"2024-03-29T08:19:14.703320+0000","last_became_active":"2024-03-29T08:19:14.280883+0000","last_became_peered":"2024-03-29T08:19:14.280883+0000","last_unstale":"2024-03-29T08:19:14.703320+0000","last_undegraded":"2024-03-29T08:19:14.703320+0000","last_fullsized":"2024-03-29T08:19:14.703320+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:14:22.394882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701140+0000","last_change":"2024-03-29T08:19:14.698993+0000","last_active":"2024-03-29T08:19:14.701140+0000","last_peered":"2024-03-29T08:19:14.701140+0000","last_clean":"2024-03-29T08:19:14.701140+0000","last_became_active":"2024-03-29T08:19:14.698757+0000","last_became_peered":"2024-03-29T08:19:14.698757+0000","last_unstale":"2024-03-29T08:19:14.701140+0000","last_undegraded":"2024-03-29T08:19:14.701140+0000","last_fullsized":"2024-03-29T08:19:14.701140+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:34:59.393546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.692376+0000","last_change":"2024-03-29T08:19:14.283589+0000","last_active":"2024-03-29T08:19:14.692376+0000","last_peered":"2024-03-29T08:19:14.692376+0000","last_clean":"2024-03-29T08:19:14.692376+0000","last_became_active":"2024-03-29T08:19:14.282933+0000","last_became_peered":"2024-03-29T08:19:14.282933+0000","last_unstale":"2024-03-29T08:19:14.692376+0000","last_undegraded":"2024-03-29T08:19:14.692376+0000","last_fullsized":"2024-03-29T08:19:14.692376+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:16:08.725000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702696+0000","last_change":"2024-03-29T08:19:14.696813+0000","last_active":"2024-03-29T08:19:14.702696+0000","last_peered":"2024-03-29T08:19:14.702696+0000","last_clean":"2024-03-29T08:19:14.702696+0000","last_became_active":"2024-03-29T08:19:14.696519+0000","last_became_peered":"2024-03-29T08:19:14.696519+0000","last_unstale":"2024-03-29T08:19:14.702696+0000","last_undegraded":"2024-03-29T08:19:14.702696+0000","last_fullsized":"2024-03-29T08:19:14.702696+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:46:59.856286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705886+0000","last_change":"2024-03-29T08:19:14.279768+0000","last_active":"2024-03-29T08:19:14.705886+0000","last_peered":"2024-03-29T08:19:14.705886+0000","last_clean":"2024-03-29T08:19:14.705886+0000","last_became_active":"2024-03-29T08:19:14.279493+0000","last_became_peered":"2024-03-29T08:19:14.279493+0000","last_unstale":"2024-03-29T08:19:14.705886+0000","last_undegraded":"2024-03-29T08:19:14.705886+0000","last_fullsized":"2024-03-29T08:19:14.705886+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:11:12.357829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.703025+0000","last_change":"2024-03-29T08:19:14.702417+0000","last_active":"2024-03-29T08:19:14.703025+0000","last_peered":"2024-03-29T08:19:14.703025+0000","last_clean":"2024-03-29T08:19:14.703025+0000","last_became_active":"2024-03-29T08:19:14.701927+0000","last_became_peered":"2024-03-29T08:19:14.701927+0000","last_unstale":"2024-03-29T08:19:14.703025+0000","last_undegraded":"2024-03-29T08:19:14.703025+0000","last_fullsized":"2024-03-29T08:19:14.703025+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:30:19.536345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701280+0000","last_change":"2024-03-29T08:19:14.279062+0000","last_active":"2024-03-29T08:19:14.701280+0000","last_peered":"2024-03-29T08:19:14.701280+0000","last_clean":"2024-03-29T08:19:14.701280+0000","last_became_active":"2024-03-29T08:19:14.278726+0000","last_became_peered":"2024-03-29T08:19:14.278726+0000","last_unstale":"2024-03-29T08:19:14.701280+0000","last_undegraded":"2024-03-29T08:19:14.701280+0000","last_fullsized":"2024-03-29T08:19:14.701280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:35.048072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687660+0000","last_change":"2024-03-29T08:19:14.679888+0000","last_active":"2024-03-29T08:19:14.687660+0000","last_peered":"2024-03-29T08:19:14.687660+0000","last_clean":"2024-03-29T08:19:14.687660+0000","last_became_active":"2024-03-29T08:19:14.679674+0000","last_became_peered":"2024-03-29T08:19:14.679674+0000","last_unstale":"2024-03-29T08:19:14.687660+0000","last_undegraded":"2024-03-29T08:19:14.687660+0000","last_fullsized":"2024-03-29T08:19:14.687660+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:55:24.587078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701177+0000","last_change":"2024-03-29T08:19:14.280371+0000","last_active":"2024-03-29T08:19:14.701177+0000","last_peered":"2024-03-29T08:19:14.701177+0000","last_clean":"2024-03-29T08:19:14.701177+0000","last_became_active":"2024-03-29T08:19:14.280045+0000","last_became_peered":"2024-03-29T08:19:14.280045+0000","last_unstale":"2024-03-29T08:19:14.701177+0000","last_undegraded":"2024-03-29T08:19:14.701177+0000","last_fullsized":"2024-03-29T08:19:14.701177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:20.730956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.681942+0000","last_change":"2024-03-29T08:19:14.677276+0000","last_active":"2024-03-29T08:19:14.681942+0000","last_peered":"2024-03-29T08:19:14.681942+0000","last_clean":"2024-03-29T08:19:14.681942+0000","last_became_active":"2024-03-29T08:19:14.677049+0000","last_became_peered":"2024-03-29T08:19:14.677049+0000","last_unstale":"2024-03-29T08:19:14.681942+0000","last_undegraded":"2024-03-29T08:19:14.681942+0000","last_fullsized":"2024-03-29T08:19:14.681942+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:27.668682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.678961+0000","last_change":"2024-03-29T08:19:14.285270+0000","last_active":"2024-03-29T08:19:14.678961+0000","last_peered":"2024-03-29T08:19:14.678961+0000","last_clean":"2024-03-29T08:19:14.678961+0000","last_became_active":"2024-03-29T08:19:14.283882+0000","last_became_peered":"2024-03-29T08:19:14.283882+0000","last_unstale":"2024-03-29T08:19:14.678961+0000","last_undegraded":"2024-03-29T08:19:14.678961+0000","last_fullsized":"2024-03-29T08:19:14.678961+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:09:33.409328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702457+0000","last_change":"2024-03-29T08:19:14.696486+0000","last_active":"2024-03-29T08:19:14.702457+0000","last_peered":"2024-03-29T08:19:14.702457+0000","last_clean":"2024-03-29T08:19:14.702457+0000","last_became_active":"2024-03-29T08:19:14.696236+0000","last_became_peered":"2024-03-29T08:19:14.696236+0000","last_unstale":"2024-03-29T08:19:14.702457+0000","last_undegraded":"2024-03-29T08:19:14.702457+0000","last_fullsized":"2024-03-29T08:19:14.702457+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:08:58.521804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.671922+0000","last_change":"2024-03-29T08:19:14.275451+0000","last_active":"2024-03-29T08:19:14.671922+0000","last_peered":"2024-03-29T08:19:14.671922+0000","last_clean":"2024-03-29T08:19:14.671922+0000","last_became_active":"2024-03-29T08:19:14.275148+0000","last_became_peered":"2024-03-29T08:19:14.275148+0000","last_unstale":"2024-03-29T08:19:14.671922+0000","last_undegraded":"2024-03-29T08:19:14.671922+0000","last_fullsized":"2024-03-29T08:19:14.671922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:57:14.581364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694290+0000","last_change":"2024-03-29T08:19:14.690396+0000","last_active":"2024-03-29T08:19:14.694290+0000","last_peered":"2024-03-29T08:19:14.694290+0000","last_clean":"2024-03-29T08:19:14.694290+0000","last_became_active":"2024-03-29T08:19:14.690106+0000","last_became_peered":"2024-03-29T08:19:14.690106+0000","last_unstale":"2024-03-29T08:19:14.694290+0000","last_undegraded":"2024-03-29T08:19:14.694290+0000","last_fullsized":"2024-03-29T08:19:14.694290+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:54:09.222693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-29T08:19:13.260506+0000","last_change":"2024-03-29T08:19:13.260506+0000","last_active":"2024-03-29T08:19:13.246882+0000","last_peered":"2024-03-29T08:19:13.246882+0000","last_clean":"2024-03-29T08:19:13.246882+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:13.260506+0000","last_undegraded":"2024-03-29T08:19:13.260506+0000","last_fullsized":"2024-03-29T08:19:13.260506+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:58:40.235616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700952+0000","last_change":"2024-03-29T08:19:14.694063+0000","last_active":"2024-03-29T08:19:14.700952+0000","last_peered":"2024-03-29T08:19:14.700952+0000","last_clean":"2024-03-29T08:19:14.700952+0000","last_became_active":"2024-03-29T08:19:14.693817+0000","last_became_peered":"2024-03-29T08:19:14.693817+0000","last_unstale":"2024-03-29T08:19:14.700952+0000","last_undegraded":"2024-03-29T08:19:14.700952+0000","last_fullsized":"2024-03-29T08:19:14.700952+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:30:57.036017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-29T08:19:13.261393+0000","last_change":"2024-03-29T08:19:13.261393+0000","last_active":"2024-03-29T08:19:13.246882+0000","last_peered":"2024-03-29T08:19:13.246882+0000","last_clean":"2024-03-29T08:19:13.246882+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:13.261393+0000","last_undegraded":"2024-03-29T08:19:13.261393+0000","last_fullsized":"2024-03-29T08:19:13.261393+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:36:01.837144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705548+0000","last_change":"2024-03-29T08:19:14.698954+0000","last_active":"2024-03-29T08:19:14.705548+0000","last_peered":"2024-03-29T08:19:14.705548+0000","last_clean":"2024-03-29T08:19:14.705548+0000","last_became_active":"2024-03-29T08:19:14.698317+0000","last_became_peered":"2024-03-29T08:19:14.698317+0000","last_unstale":"2024-03-29T08:19:14.705548+0000","last_undegraded":"2024-03-29T08:19:14.705548+0000","last_fullsized":"2024-03-29T08:19:14.705548+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:16:39.883087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700954+0000","last_change":"2024-03-29T08:19:14.285068+0000","last_active":"2024-03-29T08:19:14.700954+0000","last_peered":"2024-03-29T08:19:14.700954+0000","last_clean":"2024-03-29T08:19:14.700954+0000","last_became_active":"2024-03-29T08:19:14.284830+0000","last_became_peered":"2024-03-29T08:19:14.284830+0000","last_unstale":"2024-03-29T08:19:14.700954+0000","last_undegraded":"2024-03-29T08:19:14.700954+0000","last_fullsized":"2024-03-29T08:19:14.700954+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:16:38.877179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705493+0000","last_change":"2024-03-29T08:19:14.704710+0000","last_active":"2024-03-29T08:19:14.705493+0000","last_peered":"2024-03-29T08:19:14.705493+0000","last_clean":"2024-03-29T08:19:14.705493+0000","last_became_active":"2024-03-29T08:19:14.704396+0000","last_became_peered":"2024-03-29T08:19:14.704396+0000","last_unstale":"2024-03-29T08:19:14.705493+0000","last_undegraded":"2024-03-29T08:19:14.705493+0000","last_fullsized":"2024-03-29T08:19:14.705493+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:24.176397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.706659+0000","last_change":"2024-03-29T08:19:14.280954+0000","last_active":"2024-03-29T08:19:14.706659+0000","last_peered":"2024-03-29T08:19:14.706659+0000","last_clean":"2024-03-29T08:19:14.706659+0000","last_became_active":"2024-03-29T08:19:14.280392+0000","last_became_peered":"2024-03-29T08:19:14.280392+0000","last_unstale":"2024-03-29T08:19:14.706659+0000","last_undegraded":"2024-03-29T08:19:14.706659+0000","last_fullsized":"2024-03-29T08:19:14.706659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:26.662774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.688513+0000","last_change":"2024-03-29T08:19:14.685902+0000","last_active":"2024-03-29T08:19:14.688513+0000","last_peered":"2024-03-29T08:19:14.688513+0000","last_clean":"2024-03-29T08:19:14.688513+0000","last_became_active":"2024-03-29T08:19:14.685705+0000","last_became_peered":"2024-03-29T08:19:14.685705+0000","last_unstale":"2024-03-29T08:19:14.688513+0000","last_undegraded":"2024-03-29T08:19:14.688513+0000","last_fullsized":"2024-03-29T08:19:14.688513+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:08.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.695832+0000","last_change":"2024-03-29T08:19:14.272012+0000","last_active":"2024-03-29T08:19:14.695832+0000","last_peered":"2024-03-29T08:19:14.695832+0000","last_clean":"2024-03-29T08:19:14.695832+0000","last_became_active":"2024-03-29T08:19:14.271736+0000","last_became_peered":"2024-03-29T08:19:14.271736+0000","last_unstale":"2024-03-29T08:19:14.695832+0000","last_undegraded":"2024-03-29T08:19:14.695832+0000","last_fullsized":"2024-03-29T08:19:14.695832+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:33.064793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.696404+0000","last_change":"2024-03-29T08:19:14.690513+0000","last_active":"2024-03-29T08:19:14.696404+0000","last_peered":"2024-03-29T08:19:14.696404+0000","last_clean":"2024-03-29T08:19:14.696404+0000","last_became_active":"2024-03-29T08:19:14.690235+0000","last_became_peered":"2024-03-29T08:19:14.690235+0000","last_unstale":"2024-03-29T08:19:14.696404+0000","last_undegraded":"2024-03-29T08:19:14.696404+0000","last_fullsized":"2024-03-29T08:19:14.696404+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:01:53.702260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.693539+0000","last_change":"2024-03-29T08:19:14.274775+0000","last_active":"2024-03-29T08:19:14.693539+0000","last_peered":"2024-03-29T08:19:14.693539+0000","last_clean":"2024-03-29T08:19:14.693539+0000","last_became_active":"2024-03-29T08:19:14.274502+0000","last_became_peered":"2024-03-29T08:19:14.274502+0000","last_unstale":"2024-03-29T08:19:14.693539+0000","last_undegraded":"2024-03-29T08:19:14.693539+0000","last_fullsized":"2024-03-29T08:19:14.693539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:20:37.829787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.681902+0000","last_change":"2024-03-29T08:19:14.675914+0000","last_active":"2024-03-29T08:19:14.681902+0000","last_peered":"2024-03-29T08:19:14.681902+0000","last_clean":"2024-03-29T08:19:14.681902+0000","last_became_active":"2024-03-29T08:19:14.675564+0000","last_became_peered":"2024-03-29T08:19:14.675564+0000","last_unstale":"2024-03-29T08:19:14.681902+0000","last_undegraded":"2024-03-29T08:19:14.681902+0000","last_fullsized":"2024-03-29T08:19:14.681902+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:36:09.793618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.682997+0000","last_change":"2024-03-29T08:19:14.281362+0000","last_active":"2024-03-29T08:19:14.682997+0000","last_peered":"2024-03-29T08:19:14.682997+0000","last_clean":"2024-03-29T08:19:14.682997+0000","last_became_active":"2024-03-29T08:19:14.281111+0000","last_became_peered":"2024-03-29T08:19:14.281111+0000","last_unstale":"2024-03-29T08:19:14.682997+0000","last_undegraded":"2024-03-29T08:19:14.682997+0000","last_fullsized":"2024-03-29T08:19:14.682997+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:26.870833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700427+0000","last_change":"2024-03-29T08:19:14.698963+0000","last_active":"2024-03-29T08:19:14.700427+0000","last_peered":"2024-03-29T08:19:14.700427+0000","last_clean":"2024-03-29T08:19:14.700427+0000","last_became_active":"2024-03-29T08:19:14.698502+0000","last_became_peered":"2024-03-29T08:19:14.698502+0000","last_unstale":"2024-03-29T08:19:14.700427+0000","last_undegraded":"2024-03-29T08:19:14.700427+0000","last_fullsized":"2024-03-29T08:19:14.700427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:21:27.491192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700579+0000","last_change":"2024-03-29T08:19:14.284917+0000","last_active":"2024-03-29T08:19:14.700579+0000","last_peered":"2024-03-29T08:19:14.700579+0000","last_clean":"2024-03-29T08:19:14.700579+0000","last_became_active":"2024-03-29T08:19:14.284667+0000","last_became_peered":"2024-03-29T08:19:14.284667+0000","last_unstale":"2024-03-29T08:19:14.700579+0000","last_undegraded":"2024-03-29T08:19:14.700579+0000","last_fullsized":"2024-03-29T08:19:14.700579+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:42.739141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.704840+0000","last_change":"2024-03-29T08:19:14.703652+0000","last_active":"2024-03-29T08:19:14.704840+0000","last_peered":"2024-03-29T08:19:14.704840+0000","last_clean":"2024-03-29T08:19:14.704840+0000","last_became_active":"2024-03-29T08:19:14.703386+0000","last_became_peered":"2024-03-29T08:19:14.703386+0000","last_unstale":"2024-03-29T08:19:14.704840+0000","last_undegraded":"2024-03-29T08:19:14.704840+0000","last_fullsized":"2024-03-29T08:19:14.704840+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:55:15.376664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702612+0000","last_change":"2024-03-29T08:19:14.287908+0000","last_active":"2024-03-29T08:19:14.702612+0000","last_peered":"2024-03-29T08:19:14.702612+0000","last_clean":"2024-03-29T08:19:14.702612+0000","last_became_active":"2024-03-29T08:19:14.287562+0000","last_became_peered":"2024-03-29T08:19:14.287562+0000","last_unstale":"2024-03-29T08:19:14.702612+0000","last_undegraded":"2024-03-29T08:19:14.702612+0000","last_fullsized":"2024-03-29T08:19:14.702612+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:02:28.529095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687362+0000","last_change":"2024-03-29T08:19:14.682493+0000","last_active":"2024-03-29T08:19:14.687362+0000","last_peered":"2024-03-29T08:19:14.687362+0000","last_clean":"2024-03-29T08:19:14.687362+0000","last_became_active":"2024-03-29T08:19:14.678342+0000","last_became_peered":"2024-03-29T08:19:14.678342+0000","last_unstale":"2024-03-29T08:19:14.687362+0000","last_undegraded":"2024-03-29T08:19:14.687362+0000","last_fullsized":"2024-03-29T08:19:14.687362+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:36.053980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.695684+0000","last_change":"2024-03-29T08:19:14.272763+0000","last_active":"2024-03-29T08:19:14.695684+0000","last_peered":"2024-03-29T08:19:14.695684+0000","last_clean":"2024-03-29T08:19:14.695684+0000","last_became_active":"2024-03-29T08:19:14.272450+0000","last_became_peered":"2024-03-29T08:19:14.272450+0000","last_unstale":"2024-03-29T08:19:14.695684+0000","last_undegraded":"2024-03-29T08:19:14.695684+0000","last_fullsized":"2024-03-29T08:19:14.695684+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:19.581981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:19:14.252791+0000","last_change":"2024-03-29T08:19:14.252791+0000","last_active":"2024-03-29T08:19:14.252791+0000","last_peered":"2024-03-29T08:19:14.252791+0000","last_clean":"2024-03-29T08:19:14.252791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:14.252791+0000","last_undegraded":"2024-03-29T08:19:14.252791+0000","last_fullsized":"2024-03-29T08:19:14.252791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701079+0000","last_change":"2024-03-29T08:19:14.287716+0000","last_active":"2024-03-29T08:19:14.701079+0000","last_peered":"2024-03-29T08:19:14.701079+0000","last_clean":"2024-03-29T08:19:14.701079+0000","last_became_active":"2024-03-29T08:19:14.287437+0000","last_became_peered":"2024-03-29T08:19:14.287437+0000","last_unstale":"2024-03-29T08:19:14.701079+0000","last_undegraded":"2024-03-29T08:19:14.701079+0000","last_fullsized":"2024-03-29T08:19:14.701079+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:08:57.515896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694406+0000","last_change":"2024-03-29T08:19:14.689798+0000","last_active":"2024-03-29T08:19:14.694406+0000","last_peered":"2024-03-29T08:19:14.694406+0000","last_clean":"2024-03-29T08:19:14.694406+0000","last_became_active":"2024-03-29T08:19:14.689479+0000","last_became_peered":"2024-03-29T08:19:14.689479+0000","last_unstale":"2024-03-29T08:19:14.694406+0000","last_undegraded":"2024-03-29T08:19:14.694406+0000","last_fullsized":"2024-03-29T08:19:14.694406+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:20:03.888899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.667785+0000","last_change":"2024-03-29T08:19:14.281935+0000","last_active":"2024-03-29T08:19:14.667785+0000","last_peered":"2024-03-29T08:19:14.667785+0000","last_clean":"2024-03-29T08:19:14.667785+0000","last_became_active":"2024-03-29T08:19:14.281670+0000","last_became_peered":"2024-03-29T08:19:14.281670+0000","last_unstale":"2024-03-29T08:19:14.667785+0000","last_undegraded":"2024-03-29T08:19:14.667785+0000","last_fullsized":"2024-03-29T08:19:14.667785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:49:03.171063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.698797+0000","last_change":"2024-03-29T08:19:14.689677+0000","last_active":"2024-03-29T08:19:14.698797+0000","last_peered":"2024-03-29T08:19:14.698797+0000","last_clean":"2024-03-29T08:19:14.698797+0000","last_became_active":"2024-03-29T08:19:14.689343+0000","last_became_peered":"2024-03-29T08:19:14.689343+0000","last_unstale":"2024-03-29T08:19:14.698797+0000","last_undegraded":"2024-03-29T08:19:14.698797+0000","last_fullsized":"2024-03-29T08:19:14.698797+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:55.247391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-29T08:19:13.260439+0000","last_change":"2024-03-29T08:19:13.260439+0000","last_active":"2024-03-29T08:19:13.246882+0000","last_peered":"2024-03-29T08:19:13.246882+0000","last_clean":"2024-03-29T08:19:13.246882+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:13.260439+0000","last_undegraded":"2024-03-29T08:19:13.260439+0000","last_fullsized":"2024-03-29T08:19:13.260439+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:27:45.403801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:19:14.252791+0000","last_change":"2024-03-29T08:19:14.252791+0000","last_active":"2024-03-29T08:19:14.252791+0000","last_peered":"2024-03-29T08:19:14.252791+0000","last_clean":"2024-03-29T08:19:14.252791+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:14.252791+0000","last_undegraded":"2024-03-29T08:19:14.252791+0000","last_fullsized":"2024-03-29T08:19:14.252791+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.673192+0000","last_change":"2024-03-29T08:19:14.272023+0000","last_active":"2024-03-29T08:19:14.673192+0000","last_peered":"2024-03-29T08:19:14.673192+0000","last_clean":"2024-03-29T08:19:14.673192+0000","last_became_active":"2024-03-29T08:19:14.271722+0000","last_became_peered":"2024-03-29T08:19:14.271722+0000","last_unstale":"2024-03-29T08:19:14.673192+0000","last_undegraded":"2024-03-29T08:19:14.673192+0000","last_fullsized":"2024-03-29T08:19:14.673192+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:26.662774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.699233+0000","last_change":"2024-03-29T08:19:14.694537+0000","last_active":"2024-03-29T08:19:14.699233+0000","last_peered":"2024-03-29T08:19:14.699233+0000","last_clean":"2024-03-29T08:19:14.699233+0000","last_became_active":"2024-03-29T08:19:14.694296+0000","last_became_peered":"2024-03-29T08:19:14.694296+0000","last_unstale":"2024-03-29T08:19:14.699233+0000","last_undegraded":"2024-03-29T08:19:14.699233+0000","last_fullsized":"2024-03-29T08:19:14.699233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:37:16.877010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.673618+0000","last_change":"2024-03-29T08:19:14.285590+0000","last_active":"2024-03-29T08:19:14.673618+0000","last_peered":"2024-03-29T08:19:14.673618+0000","last_clean":"2024-03-29T08:19:14.673618+0000","last_became_active":"2024-03-29T08:19:14.285211+0000","last_became_peered":"2024-03-29T08:19:14.285211+0000","last_unstale":"2024-03-29T08:19:14.673618+0000","last_undegraded":"2024-03-29T08:19:14.673618+0000","last_fullsized":"2024-03-29T08:19:14.673618+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:52:20.153715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.682877+0000","last_change":"2024-03-29T08:19:14.679560+0000","last_active":"2024-03-29T08:19:14.682877+0000","last_peered":"2024-03-29T08:19:14.682877+0000","last_clean":"2024-03-29T08:19:14.682877+0000","last_became_active":"2024-03-29T08:19:14.677544+0000","last_became_peered":"2024-03-29T08:19:14.677544+0000","last_unstale":"2024-03-29T08:19:14.682877+0000","last_undegraded":"2024-03-29T08:19:14.682877+0000","last_fullsized":"2024-03-29T08:19:14.682877+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:34.410446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.683328+0000","last_change":"2024-03-29T08:19:14.281813+0000","last_active":"2024-03-29T08:19:14.683328+0000","last_peered":"2024-03-29T08:19:14.683328+0000","last_clean":"2024-03-29T08:19:14.683328+0000","last_became_active":"2024-03-29T08:19:14.281356+0000","last_became_peered":"2024-03-29T08:19:14.281356+0000","last_unstale":"2024-03-29T08:19:14.683328+0000","last_undegraded":"2024-03-29T08:19:14.683328+0000","last_fullsized":"2024-03-29T08:19:14.683328+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:04:28.023637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.697297+0000","last_change":"2024-03-29T08:19:14.691637+0000","last_active":"2024-03-29T08:19:14.697297+0000","last_peered":"2024-03-29T08:19:14.697297+0000","last_clean":"2024-03-29T08:19:14.697297+0000","last_became_active":"2024-03-29T08:19:14.691391+0000","last_became_peered":"2024-03-29T08:19:14.691391+0000","last_unstale":"2024-03-29T08:19:14.697297+0000","last_undegraded":"2024-03-29T08:19:14.697297+0000","last_fullsized":"2024-03-29T08:19:14.697297+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:11:13.363737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.697667+0000","last_change":"2024-03-29T08:19:14.286350+0000","last_active":"2024-03-29T08:19:14.697667+0000","last_peered":"2024-03-29T08:19:14.697667+0000","last_clean":"2024-03-29T08:19:14.697667+0000","last_became_active":"2024-03-29T08:19:14.286055+0000","last_became_peered":"2024-03-29T08:19:14.286055+0000","last_unstale":"2024-03-29T08:19:14.697667+0000","last_undegraded":"2024-03-29T08:19:14.697667+0000","last_fullsized":"2024-03-29T08:19:14.697667+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:35.048072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700275+0000","last_change":"2024-03-29T08:19:14.694685+0000","last_active":"2024-03-29T08:19:14.700275+0000","last_peered":"2024-03-29T08:19:14.700275+0000","last_clean":"2024-03-29T08:19:14.700275+0000","last_became_active":"2024-03-29T08:19:14.693892+0000","last_became_peered":"2024-03-29T08:19:14.693892+0000","last_unstale":"2024-03-29T08:19:14.700275+0000","last_undegraded":"2024-03-29T08:19:14.700275+0000","last_fullsized":"2024-03-29T08:19:14.700275+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:02:39.788723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-03-29T08:19:13.260969+0000","last_change":"2024-03-29T08:19:13.260969+0000","last_active":"2024-03-29T08:19:13.246882+0000","last_peered":"2024-03-29T08:19:13.246882+0000","last_clean":"2024-03-29T08:19:13.246882+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:19:13.260969+0000","last_undegraded":"2024-03-29T08:19:13.260969+0000","last_fullsized":"2024-03-29T08:19:13.260969+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:59:27.620393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705394+0000","last_change":"2024-03-29T08:19:14.701490+0000","last_active":"2024-03-29T08:19:14.705394+0000","last_peered":"2024-03-29T08:19:14.705394+0000","last_clean":"2024-03-29T08:19:14.705394+0000","last_became_active":"2024-03-29T08:19:14.701146+0000","last_became_peered":"2024-03-29T08:19:14.701146+0000","last_unstale":"2024-03-29T08:19:14.705394+0000","last_undegraded":"2024-03-29T08:19:14.705394+0000","last_fullsized":"2024-03-29T08:19:14.705394+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:32:52.080327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.676163+0000","last_change":"2024-03-29T08:19:14.279625+0000","last_active":"2024-03-29T08:19:14.676163+0000","last_peered":"2024-03-29T08:19:14.676163+0000","last_clean":"2024-03-29T08:19:14.676163+0000","last_became_active":"2024-03-29T08:19:14.279364+0000","last_became_peered":"2024-03-29T08:19:14.279364+0000","last_unstale":"2024-03-29T08:19:14.676163+0000","last_undegraded":"2024-03-29T08:19:14.676163+0000","last_fullsized":"2024-03-29T08:19:14.676163+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:20:02.882991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.685821+0000","last_change":"2024-03-29T08:19:14.679991+0000","last_active":"2024-03-29T08:19:14.685821+0000","last_peered":"2024-03-29T08:19:14.685821+0000","last_clean":"2024-03-29T08:19:14.685821+0000","last_became_active":"2024-03-29T08:19:14.679701+0000","last_became_peered":"2024-03-29T08:19:14.679701+0000","last_unstale":"2024-03-29T08:19:14.685821+0000","last_undegraded":"2024-03-29T08:19:14.685821+0000","last_fullsized":"2024-03-29T08:19:14.685821+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:08.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.675477+0000","last_change":"2024-03-29T08:19:14.284652+0000","last_active":"2024-03-29T08:19:14.675477+0000","last_peered":"2024-03-29T08:19:14.675477+0000","last_clean":"2024-03-29T08:19:14.675477+0000","last_became_active":"2024-03-29T08:19:14.284376+0000","last_became_peered":"2024-03-29T08:19:14.284376+0000","last_unstale":"2024-03-29T08:19:14.675477+0000","last_undegraded":"2024-03-29T08:19:14.675477+0000","last_fullsized":"2024-03-29T08:19:14.675477+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:20.730956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701325+0000","last_change":"2024-03-29T08:19:14.693198+0000","last_active":"2024-03-29T08:19:14.701325+0000","last_peered":"2024-03-29T08:19:14.701325+0000","last_clean":"2024-03-29T08:19:14.701325+0000","last_became_active":"2024-03-29T08:19:14.692849+0000","last_became_peered":"2024-03-29T08:19:14.692849+0000","last_unstale":"2024-03-29T08:19:14.701325+0000","last_undegraded":"2024-03-29T08:19:14.701325+0000","last_fullsized":"2024-03-29T08:19:14.701325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:46.218186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.676556+0000","last_change":"2024-03-29T08:19:14.284484+0000","last_active":"2024-03-29T08:19:14.676556+0000","last_peered":"2024-03-29T08:19:14.676556+0000","last_clean":"2024-03-29T08:19:14.676556+0000","last_became_active":"2024-03-29T08:19:14.284145+0000","last_became_peered":"2024-03-29T08:19:14.284145+0000","last_unstale":"2024-03-29T08:19:14.676556+0000","last_undegraded":"2024-03-29T08:19:14.676556+0000","last_fullsized":"2024-03-29T08:19:14.676556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:33.404538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702578+0000","last_change":"2024-03-29T08:19:14.695365+0000","last_active":"2024-03-29T08:19:14.702578+0000","last_peered":"2024-03-29T08:19:14.702578+0000","last_clean":"2024-03-29T08:19:14.702578+0000","last_became_active":"2024-03-29T08:19:14.695111+0000","last_became_peered":"2024-03-29T08:19:14.695111+0000","last_unstale":"2024-03-29T08:19:14.702578+0000","last_undegraded":"2024-03-29T08:19:14.702578+0000","last_fullsized":"2024-03-29T08:19:14.702578+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:15.077172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.672171+0000","last_change":"2024-03-29T08:19:14.270630+0000","last_active":"2024-03-29T08:19:14.672171+0000","last_peered":"2024-03-29T08:19:14.672171+0000","last_clean":"2024-03-29T08:19:14.672171+0000","last_became_active":"2024-03-29T08:19:14.270379+0000","last_became_peered":"2024-03-29T08:19:14.270379+0000","last_unstale":"2024-03-29T08:19:14.672171+0000","last_undegraded":"2024-03-29T08:19:14.672171+0000","last_fullsized":"2024-03-29T08:19:14.672171+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:30:18.530436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700408+0000","last_change":"2024-03-29T08:19:14.698728+0000","last_active":"2024-03-29T08:19:14.700408+0000","last_peered":"2024-03-29T08:19:14.700408+0000","last_clean":"2024-03-29T08:19:14.700408+0000","last_became_active":"2024-03-29T08:19:14.698138+0000","last_became_peered":"2024-03-29T08:19:14.698138+0000","last_unstale":"2024-03-29T08:19:14.700408+0000","last_undegraded":"2024-03-29T08:19:14.700408+0000","last_fullsized":"2024-03-29T08:19:14.700408+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:07:30.640723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.683645+0000","last_change":"2024-03-29T08:19:14.281517+0000","last_active":"2024-03-29T08:19:14.683645+0000","last_peered":"2024-03-29T08:19:14.683645+0000","last_clean":"2024-03-29T08:19:14.683645+0000","last_became_active":"2024-03-29T08:19:14.281182+0000","last_became_peered":"2024-03-29T08:19:14.281182+0000","last_unstale":"2024-03-29T08:19:14.683645+0000","last_undegraded":"2024-03-29T08:19:14.683645+0000","last_fullsized":"2024-03-29T08:19:14.683645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:21:54.574176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.689204+0000","last_change":"2024-03-29T08:19:14.685120+0000","last_active":"2024-03-29T08:19:14.689204+0000","last_peered":"2024-03-29T08:19:14.689204+0000","last_clean":"2024-03-29T08:19:14.689204+0000","last_became_active":"2024-03-29T08:19:14.682875+0000","last_became_peered":"2024-03-29T08:19:14.682875+0000","last_unstale":"2024-03-29T08:19:14.689204+0000","last_undegraded":"2024-03-29T08:19:14.689204+0000","last_fullsized":"2024-03-29T08:19:14.689204+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:55:24.587078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.683597+0000","last_change":"2024-03-29T08:19:14.283788+0000","last_active":"2024-03-29T08:19:14.683597+0000","last_peered":"2024-03-29T08:19:14.683597+0000","last_clean":"2024-03-29T08:19:14.683597+0000","last_became_active":"2024-03-29T08:19:14.283586+0000","last_became_peered":"2024-03-29T08:19:14.283586+0000","last_unstale":"2024-03-29T08:19:14.683597+0000","last_undegraded":"2024-03-29T08:19:14.683597+0000","last_fullsized":"2024-03-29T08:19:14.683597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:52:19.059268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":173,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.282014+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:19:14.282014+0000","last_peered":"2024-03-29T08:19:14.282014+0000","last_clean":"2024-03-29T08:19:14.282014+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:19:14.282014+0000","last_undegraded":"2024-03-29T08:19:14.282014+0000","last_fullsized":"2024-03-29T08:19:14.282014+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":118,"acting":118,"num_store_stats":7},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":36,"ondisk_log_size":36,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":150465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68799999999999994}]},{"osd":1,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.79800000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.78300000000000003}]},{"osd":3,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.88600000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.81299999999999994}]},{"osd":5,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.873},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.90700000000000003}]}]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":143668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55800000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.371}]},{"osd":5,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58499999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.56499999999999995}]}]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":137393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.1709999999999998,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.81399999999999995}]},{"osd":1,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.363,"5min":2.363,"15min":2.363},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.93899999999999995}]},{"osd":3,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.89600000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.86799999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.91500000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.99199999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":376832,"data_stored":143668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.35899999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.34200000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.437}]},{"osd":4,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.70599999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.73199999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.66100000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":0.83399999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":376832,"data_stored":143668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.44}]},{"osd":1,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.54500000000000004}]},{"osd":4,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.502}]},{"osd":5,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.64500000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.625}]},{"osd":7,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.63600000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":987136,"data_stored":740881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":4.2539999999999996,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.1909999999999998,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.42299999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51700000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.627}]},{"osd":5,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.76800000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":364544,"data_stored":137393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":3.9359999999999999,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.497},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":8.0419999999999998,"5min":8.0419999999999998,"15min":8.0419999999999998},"last":0.51000000000000001}]},{"osd":2,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":4.907,"5min":4.907,"15min":4.907},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":7.9569999999999999,"5min":7.9569999999999999,"15min":7.9569999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.44400000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":4.4720000000000004,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.625},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":4.3920000000000003,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.58299999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.3200000000000003,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":4.4100000000000001,"5min":4.4100000000000001,"15min":4.4100000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":4.5209999999999999,"5min":4.5209999999999999,"15min":4.5209999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":4.4420000000000002,"5min":4.4420000000000002,"15min":4.4420000000000002},"last":0.63900000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542163,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":958464,"data_stored":727761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.39800000000000002}]},{"osd":1,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.53000000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.80200000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.48499999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.439},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.55500000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.59099999999999997}]},{"osd":7,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.68300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:19:19.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-03-29T08:19:19.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T08:19:14.252791+0000', 'last_change': '2024-03-29T08:19:14.252791+0000', 'last_active': '2024-03-29T08:19:14.252791+0000', 'last_peered': '2024-03-29T08:19:14.252791+0000', 'last_clean': '2024-03-29T08:19:14.252791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:19:14.252791+0000', 'last_undegraded': '2024-03-29T08:19:14.252791+0000', 'last_fullsized': '2024-03-29T08:19:14.252791+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:19:14.252791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:19:14.252791+0000', 'last_clean_scrub_stamp': '2024-03-29T08:19:14.252791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T08:19:19.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-03-29T08:19:19.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T08:19:14.252791+0000', 'last_change': '2024-03-29T08:19:14.252791+0000', 'last_active': '2024-03-29T08:19:14.252791+0000', 'last_peered': '2024-03-29T08:19:14.252791+0000', 'last_clean': '2024-03-29T08:19:14.252791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:19:14.252791+0000', 'last_undegraded': '2024-03-29T08:19:14.252791+0000', 'last_fullsized': '2024-03-29T08:19:14.252791+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:19:14.252791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:19:14.252791+0000', 'last_clean_scrub_stamp': '2024-03-29T08:19:14.252791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T08:19:19.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-03-29T08:19:19.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T08:19:14.252791+0000', 'last_change': '2024-03-29T08:19:14.252791+0000', 'last_active': '2024-03-29T08:19:14.252791+0000', 'last_peered': '2024-03-29T08:19:14.252791+0000', 'last_clean': '2024-03-29T08:19:14.252791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:19:14.252791+0000', 'last_undegraded': '2024-03-29T08:19:14.252791+0000', 'last_fullsized': '2024-03-29T08:19:14.252791+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:19:14.252791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:19:14.252791+0000', 'last_clean_scrub_stamp': '2024-03-29T08:19:14.252791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T08:19:19.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2024-03-29T08:19:19.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:19:13.260506+0000', 'last_change': '2024-03-29T08:19:13.260506+0000', 'last_active': '2024-03-29T08:19:13.246882+0000', 'last_peered': '2024-03-29T08:19:13.246882+0000', 'last_clean': '2024-03-29T08:19:13.246882+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:19:13.260506+0000', 'last_undegraded': '2024-03-29T08:19:13.260506+0000', 'last_fullsized': '2024-03-29T08:19:13.260506+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:19:13.246882+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:19:13.246882+0000', 'last_clean_scrub_stamp': '2024-03-29T08:19:13.246882+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T13:58:40.235616+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:19:19.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2024-03-29T08:19:19.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:19:13.261393+0000', 'last_change': '2024-03-29T08:19:13.261393+0000', 'last_active': '2024-03-29T08:19:13.246882+0000', 'last_peered': '2024-03-29T08:19:13.246882+0000', 'last_clean': '2024-03-29T08:19:13.246882+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:19:13.261393+0000', 'last_undegraded': '2024-03-29T08:19:13.261393+0000', 'last_fullsized': '2024-03-29T08:19:13.261393+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:19:13.246882+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:19:13.246882+0000', 'last_clean_scrub_stamp': '2024-03-29T08:19:13.246882+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T10:36:01.837144+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:19:19.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-03-29T08:19:19.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T08:19:14.252791+0000', 'last_change': '2024-03-29T08:19:14.252791+0000', 'last_active': '2024-03-29T08:19:14.252791+0000', 'last_peered': '2024-03-29T08:19:14.252791+0000', 'last_clean': '2024-03-29T08:19:14.252791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:19:14.252791+0000', 'last_undegraded': '2024-03-29T08:19:14.252791+0000', 'last_fullsized': '2024-03-29T08:19:14.252791+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:19:14.252791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:19:14.252791+0000', 'last_clean_scrub_stamp': '2024-03-29T08:19:14.252791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T08:19:19.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2024-03-29T08:19:19.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:19:13.260439+0000', 'last_change': '2024-03-29T08:19:13.260439+0000', 'last_active': '2024-03-29T08:19:13.246882+0000', 'last_peered': '2024-03-29T08:19:13.246882+0000', 'last_clean': '2024-03-29T08:19:13.246882+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:19:13.260439+0000', 'last_undegraded': '2024-03-29T08:19:13.260439+0000', 'last_fullsized': '2024-03-29T08:19:13.260439+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:19:13.246882+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:19:13.246882+0000', 'last_clean_scrub_stamp': '2024-03-29T08:19:13.246882+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T14:27:45.403801+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:19:19.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-03-29T08:19:19.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T08:19:14.252791+0000', 'last_change': '2024-03-29T08:19:14.252791+0000', 'last_active': '2024-03-29T08:19:14.252791+0000', 'last_peered': '2024-03-29T08:19:14.252791+0000', 'last_clean': '2024-03-29T08:19:14.252791+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:19:14.252791+0000', 'last_undegraded': '2024-03-29T08:19:14.252791+0000', 'last_fullsized': '2024-03-29T08:19:14.252791+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:19:14.252791+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:19:14.252791+0000', 'last_clean_scrub_stamp': '2024-03-29T08:19:14.252791+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T08:19:19.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2024-03-29T08:19:19.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:19:13.260969+0000', 'last_change': '2024-03-29T08:19:13.260969+0000', 'last_active': '2024-03-29T08:19:13.246882+0000', 'last_peered': '2024-03-29T08:19:13.246882+0000', 'last_clean': '2024-03-29T08:19:13.246882+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:19:13.260969+0000', 'last_undegraded': '2024-03-29T08:19:13.260969+0000', 'last_fullsized': '2024-03-29T08:19:13.260969+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:19:13.246882+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:19:13.246882+0000', 'last_clean_scrub_stamp': '2024-03-29T08:19:13.246882+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T18:59:27.620393+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:19:19.569 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-29T08:19:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.740+0000 7fd628e00640 1 Processor -- start 2024-03-29T08:19:19.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.740+0000 7fd628e00640 1 -- start start 2024-03-29T08:19:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.740+0000 7fd628e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd6241569a0 0x7fd62406c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.740+0000 7fd628e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd62415d7c0 con 0x7fd6240708d0 2024-03-29T08:19:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.740+0000 7fd628e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd62415cc40 con 0x7fd62406d910 2024-03-29T08:19:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.740+0000 7fd628e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6240b21d0 con 0x7fd6241569a0 2024-03-29T08:19:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.740+0000 7fd621e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd6241569a0 0x7fd62406c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.740+0000 7fd621e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd6241569a0 0x7fd62406c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34828/0 (socket says 172.21.15.46:34828) 2024-03-29T08:19:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.740+0000 7fd622800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd6240708d0 0x7fd6241596c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56116/0 (socket says 172.21.15.46:56116) 2024-03-29T08:19:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.740+0000 7fd621e00640 1 -- 172.21.15.46:0/1916565947 learned_addr learned my addr 172.21.15.46:0/1916565947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.740+0000 7fd621e00640 1 -- 172.21.15.46:0/1916565947 >> v1:172.21.15.73:6789/0 conn(0x7fd62406d910 legacy=0x7fd6240701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.740+0000 7fd621e00640 1 -- 172.21.15.46:0/1916565947 >> v1:172.21.15.46:6789/0 conn(0x7fd6240708d0 legacy=0x7fd6241596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.740+0000 7fd621e00640 1 -- 172.21.15.46:0/1916565947 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62415cf20 con 0x7fd6241569a0 2024-03-29T08:19:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.740+0000 7fd621400640 1 -- 172.21.15.46:0/1916565947 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303667731 0 0) 0x7fd62415d7c0 con 0x7fd6240708d0 2024-03-29T08:19:19.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.744+0000 7fd621e00640 1 --2- 172.21.15.46:0/1916565947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6241569a0 0x7fd62406c1e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd6140098a0 tx=0x7fd61401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3896b0cd7809c64 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:19.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.744+0000 7fd621400640 1 -- 172.21.15.46:0/1916565947 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6140ab600 con 0x7fd6241569a0 2024-03-29T08:19:19.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.744+0000 7fd621400640 1 -- 172.21.15.46:0/1916565947 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6140a7050 con 0x7fd6241569a0 2024-03-29T08:19:19.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.744+0000 7fd621400640 1 -- 172.21.15.46:0/1916565947 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6140ab980 con 0x7fd6241569a0 2024-03-29T08:19:19.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.744+0000 7fd628e00640 1 -- 172.21.15.46:0/1916565947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6241569a0 msgr2=0x7fd62406c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:19.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.744+0000 7fd628e00640 1 --2- 172.21.15.46:0/1916565947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6241569a0 0x7fd62406c1e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd6140098a0 tx=0x7fd61401c540 comp rx=0 tx=0).stop 2024-03-29T08:19:19.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.744+0000 7fd628e00640 1 -- 172.21.15.46:0/1916565947 shutdown_connections 2024-03-29T08:19:19.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.744+0000 7fd628e00640 1 --2- 172.21.15.46:0/1916565947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6241569a0 0x7fd62406c1e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.744+0000 7fd628e00640 1 -- 172.21.15.46:0/1916565947 >> 172.21.15.46:0/1916565947 conn(0x7fd624066110 msgr2=0x7fd624069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.744+0000 7fd628e00640 1 -- 172.21.15.46:0/1916565947 shutdown_connections 2024-03-29T08:19:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.744+0000 7fd628e00640 1 -- 172.21.15.46:0/1916565947 wait complete. 2024-03-29T08:19:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd628e00640 1 Processor -- start 2024-03-29T08:19:19.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd628e00640 1 -- start start 2024-03-29T08:19:19.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd628e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd62406d910 0x7fd62415da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:19.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd628e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240708d0 0x7fd624152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:19.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd628e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd6241569a0 0x7fd62415f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:19.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd628e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62415b4a0 con 0x7fd6240708d0 2024-03-29T08:19:19.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd628e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62415b780 con 0x7fd62406d910 2024-03-29T08:19:19.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd628e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62415ba60 con 0x7fd6241569a0 2024-03-29T08:19:19.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd619400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240708d0 0x7fd624152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:19.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd619400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240708d0 0x7fd624152940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34844/0 (socket says 172.21.15.46:34844) 2024-03-29T08:19:19.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd619400640 1 -- 172.21.15.46:0/1292198297 learned_addr learned my addr 172.21.15.46:0/1292198297 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:19.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd621e00640 1 --2- 172.21.15.46:0/1292198297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd62406d910 0x7fd62415da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:19.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd622800640 1 --2- 172.21.15.46:0/1292198297 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd6241569a0 0x7fd62415f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:19.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd619400640 1 -- 172.21.15.46:0/1292198297 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd6241569a0 msgr2=0x7fd62415f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:19.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd619400640 1 --2- 172.21.15.46:0/1292198297 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd6241569a0 0x7fd62415f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd619400640 1 -- 172.21.15.46:0/1292198297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd62406d910 msgr2=0x7fd62415da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:19.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd619400640 1 --2- 172.21.15.46:0/1292198297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd62406d910 0x7fd62415da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd619400640 1 -- 172.21.15.46:0/1292198297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6140094d0 con 0x7fd6240708d0 2024-03-29T08:19:19.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd619400640 1 --2- 172.21.15.46:0/1292198297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240708d0 0x7fd624152940 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00c500 tx=0x7fd60c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:19.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd61aa00640 1 -- 172.21.15.46:0/1292198297 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c005220 con 0x7fd6240708d0 2024-03-29T08:19:19.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd61aa00640 1 -- 172.21.15.46:0/1292198297 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60c005380 con 0x7fd6240708d0 2024-03-29T08:19:19.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd61aa00640 1 -- 172.21.15.46:0/1292198297 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c007270 con 0x7fd6240708d0 2024-03-29T08:19:19.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd628e00640 1 -- 172.21.15.46:0/1292198297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6240ac2c0 con 0x7fd6240708d0 2024-03-29T08:19:19.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.748+0000 7fd628e00640 1 -- 172.21.15.46:0/1292198297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62415c2e0 con 0x7fd6240708d0 2024-03-29T08:19:19.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.752+0000 7fd628e00640 1 -- 172.21.15.46:0/1292198297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6240ac4e0 con 0x7fd6240708d0 2024-03-29T08:19:19.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.752+0000 7fd61aa00640 1 -- 172.21.15.46:0/1292198297 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd60c0073d0 con 0x7fd6240708d0 2024-03-29T08:19:19.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.752+0000 7fd61aa00640 1 --2- 172.21.15.46:0/1292198297 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd600066690 0x7fd600068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:19.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.752+0000 7fd61aa00640 1 -- 172.21.15.46:0/1292198297 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd60c083a20 con 0x7fd6240708d0 2024-03-29T08:19:19.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.752+0000 7fd621e00640 1 --2- 172.21.15.46:0/1292198297 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd600066690 0x7fd600068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:19.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.752+0000 7fd621e00640 1 --2- 172.21.15.46:0/1292198297 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd600066690 0x7fd600068b50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd624070d80 tx=0x7fd6140a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:19.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.756+0000 7fd61aa00640 1 -- 172.21.15.46:0/1292198297 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd60c050db0 con 0x7fd6240708d0 2024-03-29T08:19:19.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.976+0000 7fd628e00640 1 -- 172.21.15.46:0/1292198297 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd624066c90 con 0x7fd6240708d0 2024-03-29T08:19:19.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.976+0000 7fd61aa00640 1 -- 172.21.15.46:0/1292198297 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1334 (secure 0 0 0) 0x7fd60c0543d0 con 0x7fd6240708d0 2024-03-29T08:19:19.977 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:19.978 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":91,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":36,"num_osds":8,"num_up_osds":8,"osd_up_since":1711700279,"num_in_osds":8,"osd_in_since":1711700270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":88},{"state_name":"unknown","count":5},{"state_name":"creating+peering","count":4}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":592730,"bytes_used":242335744,"bytes_avail":772851777536,"bytes_total":773094113280,"unknown_pgs_ratio":0.051546391099691391,"inactive_pgs_ratio":0.041237112134695053,"write_bytes_sec":2983,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":5152}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-03-29T08:19:08.055085+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-29T08:19:19.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.976+0000 7fd628e00640 1 -- 172.21.15.46:0/1292198297 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd600066690 msgr2=0x7fd600068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:19.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.976+0000 7fd628e00640 1 --2- 172.21.15.46:0/1292198297 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd600066690 0x7fd600068b50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd624070d80 tx=0x7fd6140a6000 comp rx=0 tx=0).stop 2024-03-29T08:19:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.976+0000 7fd628e00640 1 -- 172.21.15.46:0/1292198297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240708d0 msgr2=0x7fd624152940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.976+0000 7fd628e00640 1 --2- 172.21.15.46:0/1292198297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240708d0 0x7fd624152940 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00c500 tx=0x7fd60c00c9c0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.976+0000 7fd628e00640 1 -- 172.21.15.46:0/1292198297 shutdown_connections 2024-03-29T08:19:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.976+0000 7fd628e00640 1 --2- 172.21.15.46:0/1292198297 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd600066690 0x7fd600068b50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.976+0000 7fd628e00640 1 --2- 172.21.15.46:0/1292198297 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd6241569a0 0x7fd62415f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.976+0000 7fd628e00640 1 --2- 172.21.15.46:0/1292198297 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6240708d0 0x7fd624152940 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.976+0000 7fd628e00640 1 --2- 172.21.15.46:0/1292198297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd62406d910 0x7fd62415da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.976+0000 7fd628e00640 1 -- 172.21.15.46:0/1292198297 >> 172.21.15.46:0/1292198297 conn(0x7fd624066110 msgr2=0x7fd624155c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:19.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.976+0000 7fd628e00640 1 -- 172.21.15.46:0/1292198297 shutdown_connections 2024-03-29T08:19:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:19.976+0000 7fd628e00640 1 -- 172.21.15.46:0/1292198297 wait complete. 2024-03-29T08:19:19.999 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-29T08:19:19.999 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:19:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.136+0000 7fa393e00640 1 Processor -- start 2024-03-29T08:19:20.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.140+0000 7fa393e00640 1 -- start start 2024-03-29T08:19:20.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.140+0000 7fa393e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa39415cb00 0x7fa394098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.140+0000 7fa393e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa394099810 0x7fa394098f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.140+0000 7fa393e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa3940b2b30 0x7fa39406b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.140+0000 7fa393e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa394099d10 con 0x7fa394099810 2024-03-29T08:19:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.140+0000 7fa393e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3940cb5d0 con 0x7fa3940b2b30 2024-03-29T08:19:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.140+0000 7fa393e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3940caca0 con 0x7fa39415cb00 2024-03-29T08:19:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.140+0000 7fa392000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa394099810 0x7fa394098f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.140+0000 7fa392000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa394099810 0x7fa394098f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34854/0 (socket says 172.21.15.46:34854) 2024-03-29T08:19:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.140+0000 7fa392000640 1 -- 172.21.15.46:0/4150778485 learned_addr learned my addr 172.21.15.46:0/4150778485 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.140+0000 7fa392a00640 1 --2- 172.21.15.46:0/4150778485 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa39415cb00 0x7fa394098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.140+0000 7fa393400640 1 --2- 172.21.15.46:0/4150778485 >> v2:172.21.15.73:3300/0 conn(0x7fa3940b2b30 0x7fa39406b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.140+0000 7fa392000640 1 -- 172.21.15.46:0/4150778485 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa39415cb00 msgr2=0x7fa394098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.140+0000 7fa392000640 1 --2- 172.21.15.46:0/4150778485 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa39415cb00 0x7fa394098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.140+0000 7fa392000640 1 -- 172.21.15.46:0/4150778485 >> v2:172.21.15.73:3300/0 conn(0x7fa3940b2b30 msgr2=0x7fa39406b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.140+0000 7fa392000640 1 --2- 172.21.15.46:0/4150778485 >> v2:172.21.15.73:3300/0 conn(0x7fa3940b2b30 0x7fa39406b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:20.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.140+0000 7fa392000640 1 -- 172.21.15.46:0/4150778485 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3940caf80 con 0x7fa394099810 2024-03-29T08:19:20.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.144+0000 7fa392000640 1 --2- 172.21.15.46:0/4150778485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa394099810 0x7fa394098f80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa39406cfe0 tx=0x7fa38001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a429dd60e066e09 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:20.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.144+0000 7fa391600640 1 -- 172.21.15.46:0/4150778485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3800ac620 con 0x7fa394099810 2024-03-29T08:19:20.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.144+0000 7fa391600640 1 -- 172.21.15.46:0/4150778485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3800ac780 con 0x7fa394099810 2024-03-29T08:19:20.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.144+0000 7fa391600640 1 -- 172.21.15.46:0/4150778485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3800acb10 con 0x7fa394099810 2024-03-29T08:19:20.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.144+0000 7fa393e00640 1 -- 172.21.15.46:0/4150778485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa394099810 msgr2=0x7fa394098f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:20.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.144+0000 7fa393e00640 1 --2- 172.21.15.46:0/4150778485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa394099810 0x7fa394098f80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa39406cfe0 tx=0x7fa38001c460 comp rx=0 tx=0).stop 2024-03-29T08:19:20.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.144+0000 7fa393e00640 1 -- 172.21.15.46:0/4150778485 shutdown_connections 2024-03-29T08:19:20.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.144+0000 7fa393e00640 1 --2- 172.21.15.46:0/4150778485 >> v2:172.21.15.73:3300/0 conn(0x7fa3940b2b30 0x7fa39406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.144+0000 7fa393e00640 1 --2- 172.21.15.46:0/4150778485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa394099810 0x7fa394098f80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.144+0000 7fa393e00640 1 --2- 172.21.15.46:0/4150778485 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa39415cb00 0x7fa394098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.144+0000 7fa393e00640 1 -- 172.21.15.46:0/4150778485 >> 172.21.15.46:0/4150778485 conn(0x7fa3940b2010 msgr2=0x7fa3940b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.144+0000 7fa393e00640 1 -- 172.21.15.46:0/4150778485 shutdown_connections 2024-03-29T08:19:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.144+0000 7fa393e00640 1 -- 172.21.15.46:0/4150778485 wait complete. 2024-03-29T08:19:20.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.144+0000 7fa393e00640 1 Processor -- start 2024-03-29T08:19:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.144+0000 7fa393e00640 1 -- start start 2024-03-29T08:19:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa393e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa394099810 0x7fa3940c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa393e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3940b2b30 0x7fa3940b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa393e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa39415cb00 0x7fa39413f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa393e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3940b53f0 con 0x7fa3940b2b30 2024-03-29T08:19:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa393e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3940b56c0 con 0x7fa39415cb00 2024-03-29T08:19:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa393e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3940b59a0 con 0x7fa394099810 2024-03-29T08:19:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa392000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3940b2b30 0x7fa3940b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa392000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3940b2b30 0x7fa3940b4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34868/0 (socket says 172.21.15.46:34868) 2024-03-29T08:19:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa392000640 1 -- 172.21.15.46:0/540781324 learned_addr learned my addr 172.21.15.46:0/540781324 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa392a00640 1 --2- 172.21.15.46:0/540781324 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa394099810 0x7fa3940c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa393400640 1 --2- 172.21.15.46:0/540781324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa39415cb00 0x7fa39413f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa392000640 1 -- 172.21.15.46:0/540781324 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa394099810 msgr2=0x7fa3940c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa392000640 1 --2- 172.21.15.46:0/540781324 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa394099810 0x7fa3940c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa392000640 1 -- 172.21.15.46:0/540781324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa39415cb00 msgr2=0x7fa39413f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa392000640 1 --2- 172.21.15.46:0/540781324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa39415cb00 0x7fa39413f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa392000640 1 -- 172.21.15.46:0/540781324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3800094d0 con 0x7fa3940b2b30 2024-03-29T08:19:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa392000640 1 --2- 172.21.15.46:0/540781324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3940b2b30 0x7fa3940b4eb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa3800af040 tx=0x7fa38001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa38aa00640 1 -- 172.21.15.46:0/540781324 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3800b7620 con 0x7fa3940b2b30 2024-03-29T08:19:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa38aa00640 1 -- 172.21.15.46:0/540781324 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3800b77a0 con 0x7fa3940b2b30 2024-03-29T08:19:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa38aa00640 1 -- 172.21.15.46:0/540781324 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3800b7b50 con 0x7fa3940b2b30 2024-03-29T08:19:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa393e00640 1 -- 172.21.15.46:0/540781324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3940b5c80 con 0x7fa3940b2b30 2024-03-29T08:19:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa393e00640 1 -- 172.21.15.46:0/540781324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3940b6240 con 0x7fa3940b2b30 2024-03-29T08:19:20.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa393e00640 1 -- 172.21.15.46:0/540781324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa39415a170 con 0x7fa3940b2b30 2024-03-29T08:19:20.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.148+0000 7fa38aa00640 1 -- 172.21.15.46:0/540781324 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa3800b3040 con 0x7fa3940b2b30 2024-03-29T08:19:20.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.152+0000 7fa38aa00640 1 --2- 172.21.15.46:0/540781324 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa374066690 0x7fa374068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:20.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.152+0000 7fa38aa00640 1 -- 172.21.15.46:0/540781324 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa38011e6b0 con 0x7fa3940b2b30 2024-03-29T08:19:20.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.152+0000 7fa392a00640 1 --2- 172.21.15.46:0/540781324 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa374066690 0x7fa374068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:20.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.152+0000 7fa392a00640 1 --2- 172.21.15.46:0/540781324 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa374066690 0x7fa374068b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa384007a60 tx=0x7fa384008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:20.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.156+0000 7fa38aa00640 1 -- 172.21.15.46:0/540781324 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa3800eba30 con 0x7fa3940b2b30 2024-03-29T08:19:20.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.312+0000 7fa393e00640 1 -- 172.21.15.46:0/540781324 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa39406ca00 con 0x7fa374066690 2024-03-29T08:19:20.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.320+0000 7fa38aa00640 1 -- 172.21.15.46:0/540781324 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289144 (secure 0 0 0) 0x7fa384010990 con 0x7fa374066690 2024-03-29T08:19:20.323 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:20.325 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:19:20.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.324+0000 7fa393e00640 1 -- 172.21.15.46:0/540781324 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa374066690 msgr2=0x7fa374068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:20.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.324+0000 7fa393e00640 1 --2- 172.21.15.46:0/540781324 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa374066690 0x7fa374068b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa384007a60 tx=0x7fa384008040 comp rx=0 tx=0).stop 2024-03-29T08:19:20.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.324+0000 7fa393e00640 1 -- 172.21.15.46:0/540781324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3940b2b30 msgr2=0x7fa3940b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:20.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.324+0000 7fa393e00640 1 --2- 172.21.15.46:0/540781324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3940b2b30 0x7fa3940b4eb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa3800af040 tx=0x7fa38001aca0 comp rx=0 tx=0).stop 2024-03-29T08:19:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.324+0000 7fa393e00640 1 -- 172.21.15.46:0/540781324 shutdown_connections 2024-03-29T08:19:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.324+0000 7fa393e00640 1 --2- 172.21.15.46:0/540781324 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa374066690 0x7fa374068b50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.324+0000 7fa393e00640 1 --2- 172.21.15.46:0/540781324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa39415cb00 0x7fa39413f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.324+0000 7fa393e00640 1 --2- 172.21.15.46:0/540781324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3940b2b30 0x7fa3940b4eb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.324+0000 7fa393e00640 1 --2- 172.21.15.46:0/540781324 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa394099810 0x7fa3940c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.324+0000 7fa393e00640 1 -- 172.21.15.46:0/540781324 >> 172.21.15.46:0/540781324 conn(0x7fa3940b2010 msgr2=0x7fa3940ad1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.324+0000 7fa393e00640 1 -- 172.21.15.46:0/540781324 shutdown_connections 2024-03-29T08:19:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:20.324+0000 7fa393e00640 1 -- 172.21.15.46:0/540781324 wait complete. 2024-03-29T08:19:20.343 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-03-29T08:19:20.057480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236696,"kb_used_data":4140,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754738024,"statfs":{"total":773094113280,"available":772851736576,"internally_reserved":0,"allocated":4239360,"data_stored":2344829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001162"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.679129+0000","last_change":"2024-03-29T08:19:14.677048+0000","last_active":"2024-03-29T08:19:14.679129+0000","last_peered":"2024-03-29T08:19:14.679129+0000","last_clean":"2024-03-29T08:19:14.679129+0000","last_became_active":"2024-03-29T08:19:14.676802+0000","last_became_peered":"2024-03-29T08:19:14.676802+0000","last_unstale":"2024-03-29T08:19:14.679129+0000","last_undegraded":"2024-03-29T08:19:14.679129+0000","last_fullsized":"2024-03-29T08:19:14.679129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:37:16.877010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-29T08:19:16.660674+0000","last_change":"2024-03-29T08:19:14.702139+0000","last_active":"2024-03-29T08:19:16.660674+0000","last_peered":"2024-03-29T08:19:16.660674+0000","last_clean":"2024-03-29T08:19:16.660674+0000","last_became_active":"2024-03-29T08:19:14.701881+0000","last_became_peered":"2024-03-29T08:19:14.701881+0000","last_unstale":"2024-03-29T08:19:16.660674+0000","last_undegraded":"2024-03-29T08:19:16.660674+0000","last_fullsized":"2024-03-29T08:19:16.660674+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:35.312841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.681839+0000","last_change":"2024-03-29T08:19:14.677189+0000","last_active":"2024-03-29T08:19:14.681839+0000","last_peered":"2024-03-29T08:19:14.681839+0000","last_clean":"2024-03-29T08:19:14.681839+0000","last_became_active":"2024-03-29T08:19:14.676526+0000","last_became_peered":"2024-03-29T08:19:14.676526+0000","last_unstale":"2024-03-29T08:19:14.681839+0000","last_undegraded":"2024-03-29T08:19:14.681839+0000","last_fullsized":"2024-03-29T08:19:14.681839+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:20.587889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705567+0000","last_change":"2024-03-29T08:19:14.699305+0000","last_active":"2024-03-29T08:19:14.705567+0000","last_peered":"2024-03-29T08:19:14.705567+0000","last_clean":"2024-03-29T08:19:14.705567+0000","last_became_active":"2024-03-29T08:19:14.698637+0000","last_became_peered":"2024-03-29T08:19:14.698637+0000","last_unstale":"2024-03-29T08:19:14.705567+0000","last_undegraded":"2024-03-29T08:19:14.705567+0000","last_fullsized":"2024-03-29T08:19:14.705567+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:35.234827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-29T08:19:16.660584+0000","last_change":"2024-03-29T08:19:14.682597+0000","last_active":"2024-03-29T08:19:16.660584+0000","last_peered":"2024-03-29T08:19:16.660584+0000","last_clean":"2024-03-29T08:19:16.660584+0000","last_became_active":"2024-03-29T08:19:14.682322+0000","last_became_peered":"2024-03-29T08:19:14.682322+0000","last_unstale":"2024-03-29T08:19:16.660584+0000","last_undegraded":"2024-03-29T08:19:16.660584+0000","last_fullsized":"2024-03-29T08:19:16.660584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:51:41.300882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694387+0000","last_change":"2024-03-29T08:19:14.691105+0000","last_active":"2024-03-29T08:19:14.694387+0000","last_peered":"2024-03-29T08:19:14.694387+0000","last_clean":"2024-03-29T08:19:14.694387+0000","last_became_active":"2024-03-29T08:19:14.690486+0000","last_became_peered":"2024-03-29T08:19:14.690486+0000","last_unstale":"2024-03-29T08:19:14.694387+0000","last_undegraded":"2024-03-29T08:19:14.694387+0000","last_fullsized":"2024-03-29T08:19:14.694387+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:21.736865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.678975+0000","last_change":"2024-03-29T08:19:14.675543+0000","last_active":"2024-03-29T08:19:14.678975+0000","last_peered":"2024-03-29T08:19:14.678975+0000","last_clean":"2024-03-29T08:19:14.678975+0000","last_became_active":"2024-03-29T08:19:14.675201+0000","last_became_peered":"2024-03-29T08:19:14.675201+0000","last_unstale":"2024-03-29T08:19:14.678975+0000","last_undegraded":"2024-03-29T08:19:14.678975+0000","last_fullsized":"2024-03-29T08:19:14.678975+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:08:58.521804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687632+0000","last_change":"2024-03-29T08:19:14.681677+0000","last_active":"2024-03-29T08:19:14.687632+0000","last_peered":"2024-03-29T08:19:14.687632+0000","last_clean":"2024-03-29T08:19:14.687632+0000","last_became_active":"2024-03-29T08:19:14.680707+0000","last_became_peered":"2024-03-29T08:19:14.680707+0000","last_unstale":"2024-03-29T08:19:14.687632+0000","last_undegraded":"2024-03-29T08:19:14.687632+0000","last_fullsized":"2024-03-29T08:19:14.687632+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:04:29.029546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.704743+0000","last_change":"2024-03-29T08:19:14.703892+0000","last_active":"2024-03-29T08:19:14.704743+0000","last_peered":"2024-03-29T08:19:14.704743+0000","last_clean":"2024-03-29T08:19:14.704743+0000","last_became_active":"2024-03-29T08:19:14.701909+0000","last_became_peered":"2024-03-29T08:19:14.701909+0000","last_unstale":"2024-03-29T08:19:14.704743+0000","last_undegraded":"2024-03-29T08:19:14.704743+0000","last_fullsized":"2024-03-29T08:19:14.704743+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:57:15.587273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701010+0000","last_change":"2024-03-29T08:19:14.694312+0000","last_active":"2024-03-29T08:19:14.701010+0000","last_peered":"2024-03-29T08:19:14.701010+0000","last_clean":"2024-03-29T08:19:14.701010+0000","last_became_active":"2024-03-29T08:19:14.694027+0000","last_became_peered":"2024-03-29T08:19:14.694027+0000","last_unstale":"2024-03-29T08:19:14.701010+0000","last_undegraded":"2024-03-29T08:19:14.701010+0000","last_fullsized":"2024-03-29T08:19:14.701010+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:15:31.529776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.704807+0000","last_change":"2024-03-29T08:19:14.703541+0000","last_active":"2024-03-29T08:19:14.704807+0000","last_peered":"2024-03-29T08:19:14.704807+0000","last_clean":"2024-03-29T08:19:14.704807+0000","last_became_active":"2024-03-29T08:19:14.703317+0000","last_became_peered":"2024-03-29T08:19:14.703317+0000","last_unstale":"2024-03-29T08:19:14.704807+0000","last_undegraded":"2024-03-29T08:19:14.704807+0000","last_fullsized":"2024-03-29T08:19:14.704807+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:18:37.057951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687173+0000","last_change":"2024-03-29T08:19:14.684087+0000","last_active":"2024-03-29T08:19:14.687173+0000","last_peered":"2024-03-29T08:19:14.687173+0000","last_clean":"2024-03-29T08:19:14.687173+0000","last_became_active":"2024-03-29T08:19:14.683827+0000","last_became_peered":"2024-03-29T08:19:14.683827+0000","last_unstale":"2024-03-29T08:19:14.687173+0000","last_undegraded":"2024-03-29T08:19:14.687173+0000","last_fullsized":"2024-03-29T08:19:14.687173+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:21.736865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702127+0000","last_change":"2024-03-29T08:19:14.699133+0000","last_active":"2024-03-29T08:19:14.702127+0000","last_peered":"2024-03-29T08:19:14.702127+0000","last_clean":"2024-03-29T08:19:14.702127+0000","last_became_active":"2024-03-29T08:19:14.698579+0000","last_became_peered":"2024-03-29T08:19:14.698579+0000","last_unstale":"2024-03-29T08:19:14.702127+0000","last_undegraded":"2024-03-29T08:19:14.702127+0000","last_fullsized":"2024-03-29T08:19:14.702127+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:00:07.571380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702206+0000","last_change":"2024-03-29T08:19:14.694580+0000","last_active":"2024-03-29T08:19:14.702206+0000","last_peered":"2024-03-29T08:19:14.702206+0000","last_clean":"2024-03-29T08:19:14.702206+0000","last_became_active":"2024-03-29T08:19:14.694305+0000","last_became_peered":"2024-03-29T08:19:14.694305+0000","last_unstale":"2024-03-29T08:19:14.702206+0000","last_undegraded":"2024-03-29T08:19:14.702206+0000","last_fullsized":"2024-03-29T08:19:14.702206+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:19:13.272178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.676292+0000","last_change":"2024-03-29T08:19:14.674438+0000","last_active":"2024-03-29T08:19:14.676292+0000","last_peered":"2024-03-29T08:19:14.676292+0000","last_clean":"2024-03-29T08:19:14.676292+0000","last_became_active":"2024-03-29T08:19:14.674172+0000","last_became_peered":"2024-03-29T08:19:14.674172+0000","last_unstale":"2024-03-29T08:19:14.676292+0000","last_undegraded":"2024-03-29T08:19:14.676292+0000","last_fullsized":"2024-03-29T08:19:14.676292+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:29.147117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687494+0000","last_change":"2024-03-29T08:19:14.681624+0000","last_active":"2024-03-29T08:19:14.687494+0000","last_peered":"2024-03-29T08:19:14.687494+0000","last_clean":"2024-03-29T08:19:14.687494+0000","last_became_active":"2024-03-29T08:19:14.681290+0000","last_became_peered":"2024-03-29T08:19:14.681290+0000","last_unstale":"2024-03-29T08:19:14.687494+0000","last_undegraded":"2024-03-29T08:19:14.687494+0000","last_fullsized":"2024-03-29T08:19:14.687494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:34:59.393546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700295+0000","last_change":"2024-03-29T08:19:14.695305+0000","last_active":"2024-03-29T08:19:14.700295+0000","last_peered":"2024-03-29T08:19:14.700295+0000","last_clean":"2024-03-29T08:19:14.700295+0000","last_became_active":"2024-03-29T08:19:14.694942+0000","last_became_peered":"2024-03-29T08:19:14.694942+0000","last_unstale":"2024-03-29T08:19:14.700295+0000","last_undegraded":"2024-03-29T08:19:14.700295+0000","last_fullsized":"2024-03-29T08:19:14.700295+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:19:41.115142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694349+0000","last_change":"2024-03-29T08:19:14.690840+0000","last_active":"2024-03-29T08:19:14.694349+0000","last_peered":"2024-03-29T08:19:14.694349+0000","last_clean":"2024-03-29T08:19:14.694349+0000","last_became_active":"2024-03-29T08:19:14.689554+0000","last_became_peered":"2024-03-29T08:19:14.689554+0000","last_unstale":"2024-03-29T08:19:14.694349+0000","last_undegraded":"2024-03-29T08:19:14.694349+0000","last_fullsized":"2024-03-29T08:19:14.694349+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:49:04.176971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694388+0000","last_change":"2024-03-29T08:19:14.689362+0000","last_active":"2024-03-29T08:19:14.694388+0000","last_peered":"2024-03-29T08:19:14.694388+0000","last_clean":"2024-03-29T08:19:14.694388+0000","last_became_active":"2024-03-29T08:19:14.689050+0000","last_became_peered":"2024-03-29T08:19:14.689050+0000","last_unstale":"2024-03-29T08:19:14.694388+0000","last_undegraded":"2024-03-29T08:19:14.694388+0000","last_fullsized":"2024-03-29T08:19:14.694388+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:20:38.835695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.692494+0000","last_change":"2024-03-29T08:19:14.683957+0000","last_active":"2024-03-29T08:19:14.692494+0000","last_peered":"2024-03-29T08:19:14.692494+0000","last_clean":"2024-03-29T08:19:14.692494+0000","last_became_active":"2024-03-29T08:19:14.683710+0000","last_became_peered":"2024-03-29T08:19:14.683710+0000","last_unstale":"2024-03-29T08:19:14.692494+0000","last_undegraded":"2024-03-29T08:19:14.692494+0000","last_fullsized":"2024-03-29T08:19:14.692494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:11:49.840732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.689299+0000","last_change":"2024-03-29T08:19:14.685648+0000","last_active":"2024-03-29T08:19:14.689299+0000","last_peered":"2024-03-29T08:19:14.689299+0000","last_clean":"2024-03-29T08:19:14.689299+0000","last_became_active":"2024-03-29T08:19:14.685404+0000","last_became_peered":"2024-03-29T08:19:14.685404+0000","last_unstale":"2024-03-29T08:19:14.689299+0000","last_undegraded":"2024-03-29T08:19:14.689299+0000","last_fullsized":"2024-03-29T08:19:14.689299+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:14:08.867120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701342+0000","last_change":"2024-03-29T08:19:14.698732+0000","last_active":"2024-03-29T08:19:14.701342+0000","last_peered":"2024-03-29T08:19:14.701342+0000","last_clean":"2024-03-29T08:19:14.701342+0000","last_became_active":"2024-03-29T08:19:14.698484+0000","last_became_peered":"2024-03-29T08:19:14.698484+0000","last_unstale":"2024-03-29T08:19:14.701342+0000","last_undegraded":"2024-03-29T08:19:14.701342+0000","last_fullsized":"2024-03-29T08:19:14.701342+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:08.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.688435+0000","last_change":"2024-03-29T08:19:14.685174+0000","last_active":"2024-03-29T08:19:14.688435+0000","last_peered":"2024-03-29T08:19:14.688435+0000","last_clean":"2024-03-29T08:19:14.688435+0000","last_became_active":"2024-03-29T08:19:14.684605+0000","last_became_peered":"2024-03-29T08:19:14.684605+0000","last_unstale":"2024-03-29T08:19:14.688435+0000","last_undegraded":"2024-03-29T08:19:14.688435+0000","last_fullsized":"2024-03-29T08:19:14.688435+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:34:59.393546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.699334+0000","last_change":"2024-03-29T08:19:14.698117+0000","last_active":"2024-03-29T08:19:14.699334+0000","last_peered":"2024-03-29T08:19:14.699334+0000","last_clean":"2024-03-29T08:19:14.699334+0000","last_became_active":"2024-03-29T08:19:14.697861+0000","last_became_peered":"2024-03-29T08:19:14.697861+0000","last_unstale":"2024-03-29T08:19:14.699334+0000","last_undegraded":"2024-03-29T08:19:14.699334+0000","last_fullsized":"2024-03-29T08:19:14.699334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:14:08.867120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702337+0000","last_change":"2024-03-29T08:19:14.695925+0000","last_active":"2024-03-29T08:19:14.702337+0000","last_peered":"2024-03-29T08:19:14.702337+0000","last_clean":"2024-03-29T08:19:14.702337+0000","last_became_active":"2024-03-29T08:19:14.695180+0000","last_became_peered":"2024-03-29T08:19:14.695180+0000","last_unstale":"2024-03-29T08:19:14.702337+0000","last_undegraded":"2024-03-29T08:19:14.702337+0000","last_fullsized":"2024-03-29T08:19:14.702337+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:20:02.908287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702375+0000","last_change":"2024-03-29T08:19:14.696275+0000","last_active":"2024-03-29T08:19:14.702375+0000","last_peered":"2024-03-29T08:19:14.702375+0000","last_clean":"2024-03-29T08:19:14.702375+0000","last_became_active":"2024-03-29T08:19:14.696065+0000","last_became_peered":"2024-03-29T08:19:14.696065+0000","last_unstale":"2024-03-29T08:19:14.702375+0000","last_undegraded":"2024-03-29T08:19:14.702375+0000","last_fullsized":"2024-03-29T08:19:14.702375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:00:46.960177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-29T08:19:16.660670+0000","last_change":"2024-03-29T08:19:14.677537+0000","last_active":"2024-03-29T08:19:16.660670+0000","last_peered":"2024-03-29T08:19:16.660670+0000","last_clean":"2024-03-29T08:19:16.660670+0000","last_became_active":"2024-03-29T08:19:14.677288+0000","last_became_peered":"2024-03-29T08:19:14.677288+0000","last_unstale":"2024-03-29T08:19:16.660670+0000","last_undegraded":"2024-03-29T08:19:16.660670+0000","last_fullsized":"2024-03-29T08:19:16.660670+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:59:18.746998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.693704+0000","last_change":"2024-03-29T08:19:14.684317+0000","last_active":"2024-03-29T08:19:14.693704+0000","last_peered":"2024-03-29T08:19:14.693704+0000","last_clean":"2024-03-29T08:19:14.693704+0000","last_became_active":"2024-03-29T08:19:14.683125+0000","last_became_peered":"2024-03-29T08:19:14.683125+0000","last_unstale":"2024-03-29T08:19:14.693704+0000","last_undegraded":"2024-03-29T08:19:14.693704+0000","last_fullsized":"2024-03-29T08:19:14.693704+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:52:54.511385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702775+0000","last_change":"2024-03-29T08:19:14.701650+0000","last_active":"2024-03-29T08:19:14.702775+0000","last_peered":"2024-03-29T08:19:14.702775+0000","last_clean":"2024-03-29T08:19:14.702775+0000","last_became_active":"2024-03-29T08:19:14.697686+0000","last_became_peered":"2024-03-29T08:19:14.697686+0000","last_unstale":"2024-03-29T08:19:14.702775+0000","last_undegraded":"2024-03-29T08:19:14.702775+0000","last_fullsized":"2024-03-29T08:19:14.702775+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:10:23.783843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687174+0000","last_change":"2024-03-29T08:19:14.684872+0000","last_active":"2024-03-29T08:19:14.687174+0000","last_peered":"2024-03-29T08:19:14.687174+0000","last_clean":"2024-03-29T08:19:14.687174+0000","last_became_active":"2024-03-29T08:19:14.684622+0000","last_became_peered":"2024-03-29T08:19:14.684622+0000","last_unstale":"2024-03-29T08:19:14.687174+0000","last_undegraded":"2024-03-29T08:19:14.687174+0000","last_fullsized":"2024-03-29T08:19:14.687174+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:00:07.571380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687689+0000","last_change":"2024-03-29T08:19:14.681150+0000","last_active":"2024-03-29T08:19:14.687689+0000","last_peered":"2024-03-29T08:19:14.687689+0000","last_clean":"2024-03-29T08:19:14.687689+0000","last_became_active":"2024-03-29T08:19:14.680888+0000","last_became_peered":"2024-03-29T08:19:14.680888+0000","last_unstale":"2024-03-29T08:19:14.687689+0000","last_undegraded":"2024-03-29T08:19:14.687689+0000","last_fullsized":"2024-03-29T08:19:14.687689+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:10:23.783843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700291+0000","last_change":"2024-03-29T08:19:14.698385+0000","last_active":"2024-03-29T08:19:14.700291+0000","last_peered":"2024-03-29T08:19:14.700291+0000","last_clean":"2024-03-29T08:19:14.700291+0000","last_became_active":"2024-03-29T08:19:14.698119+0000","last_became_peered":"2024-03-29T08:19:14.698119+0000","last_unstale":"2024-03-29T08:19:14.700291+0000","last_undegraded":"2024-03-29T08:19:14.700291+0000","last_fullsized":"2024-03-29T08:19:14.700291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:03:28.287471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.692119+0000","last_change":"2024-03-29T08:19:14.686891+0000","last_active":"2024-03-29T08:19:14.692119+0000","last_peered":"2024-03-29T08:19:14.692119+0000","last_clean":"2024-03-29T08:19:14.692119+0000","last_became_active":"2024-03-29T08:19:14.686615+0000","last_became_peered":"2024-03-29T08:19:14.686615+0000","last_unstale":"2024-03-29T08:19:14.692119+0000","last_undegraded":"2024-03-29T08:19:14.692119+0000","last_fullsized":"2024-03-29T08:19:14.692119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:10:23.783843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701442+0000","last_change":"2024-03-29T08:19:14.278073+0000","last_active":"2024-03-29T08:19:14.701442+0000","last_peered":"2024-03-29T08:19:14.701442+0000","last_clean":"2024-03-29T08:19:14.701442+0000","last_became_active":"2024-03-29T08:19:14.277778+0000","last_became_peered":"2024-03-29T08:19:14.277778+0000","last_unstale":"2024-03-29T08:19:14.701442+0000","last_undegraded":"2024-03-29T08:19:14.701442+0000","last_fullsized":"2024-03-29T08:19:14.701442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:36:08.787710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687978+0000","last_change":"2024-03-29T08:19:14.682096+0000","last_active":"2024-03-29T08:19:14.687978+0000","last_peered":"2024-03-29T08:19:14.687978+0000","last_clean":"2024-03-29T08:19:14.687978+0000","last_became_active":"2024-03-29T08:19:14.681779+0000","last_became_peered":"2024-03-29T08:19:14.681779+0000","last_unstale":"2024-03-29T08:19:14.687978+0000","last_undegraded":"2024-03-29T08:19:14.687978+0000","last_fullsized":"2024-03-29T08:19:14.687978+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:52:20.065176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.703320+0000","last_change":"2024-03-29T08:19:14.281218+0000","last_active":"2024-03-29T08:19:14.703320+0000","last_peered":"2024-03-29T08:19:14.703320+0000","last_clean":"2024-03-29T08:19:14.703320+0000","last_became_active":"2024-03-29T08:19:14.280883+0000","last_became_peered":"2024-03-29T08:19:14.280883+0000","last_unstale":"2024-03-29T08:19:14.703320+0000","last_undegraded":"2024-03-29T08:19:14.703320+0000","last_fullsized":"2024-03-29T08:19:14.703320+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:14:22.394882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701140+0000","last_change":"2024-03-29T08:19:14.698993+0000","last_active":"2024-03-29T08:19:14.701140+0000","last_peered":"2024-03-29T08:19:14.701140+0000","last_clean":"2024-03-29T08:19:14.701140+0000","last_became_active":"2024-03-29T08:19:14.698757+0000","last_became_peered":"2024-03-29T08:19:14.698757+0000","last_unstale":"2024-03-29T08:19:14.701140+0000","last_undegraded":"2024-03-29T08:19:14.701140+0000","last_fullsized":"2024-03-29T08:19:14.701140+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:34:59.393546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.692376+0000","last_change":"2024-03-29T08:19:14.283589+0000","last_active":"2024-03-29T08:19:14.692376+0000","last_peered":"2024-03-29T08:19:14.692376+0000","last_clean":"2024-03-29T08:19:14.692376+0000","last_became_active":"2024-03-29T08:19:14.282933+0000","last_became_peered":"2024-03-29T08:19:14.282933+0000","last_unstale":"2024-03-29T08:19:14.692376+0000","last_undegraded":"2024-03-29T08:19:14.692376+0000","last_fullsized":"2024-03-29T08:19:14.692376+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:16:08.725000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702696+0000","last_change":"2024-03-29T08:19:14.696813+0000","last_active":"2024-03-29T08:19:14.702696+0000","last_peered":"2024-03-29T08:19:14.702696+0000","last_clean":"2024-03-29T08:19:14.702696+0000","last_became_active":"2024-03-29T08:19:14.696519+0000","last_became_peered":"2024-03-29T08:19:14.696519+0000","last_unstale":"2024-03-29T08:19:14.702696+0000","last_undegraded":"2024-03-29T08:19:14.702696+0000","last_fullsized":"2024-03-29T08:19:14.702696+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:46:59.856286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705886+0000","last_change":"2024-03-29T08:19:14.279768+0000","last_active":"2024-03-29T08:19:14.705886+0000","last_peered":"2024-03-29T08:19:14.705886+0000","last_clean":"2024-03-29T08:19:14.705886+0000","last_became_active":"2024-03-29T08:19:14.279493+0000","last_became_peered":"2024-03-29T08:19:14.279493+0000","last_unstale":"2024-03-29T08:19:14.705886+0000","last_undegraded":"2024-03-29T08:19:14.705886+0000","last_fullsized":"2024-03-29T08:19:14.705886+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:11:12.357829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.703025+0000","last_change":"2024-03-29T08:19:14.702417+0000","last_active":"2024-03-29T08:19:14.703025+0000","last_peered":"2024-03-29T08:19:14.703025+0000","last_clean":"2024-03-29T08:19:14.703025+0000","last_became_active":"2024-03-29T08:19:14.701927+0000","last_became_peered":"2024-03-29T08:19:14.701927+0000","last_unstale":"2024-03-29T08:19:14.703025+0000","last_undegraded":"2024-03-29T08:19:14.703025+0000","last_fullsized":"2024-03-29T08:19:14.703025+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:30:19.536345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701280+0000","last_change":"2024-03-29T08:19:14.279062+0000","last_active":"2024-03-29T08:19:14.701280+0000","last_peered":"2024-03-29T08:19:14.701280+0000","last_clean":"2024-03-29T08:19:14.701280+0000","last_became_active":"2024-03-29T08:19:14.278726+0000","last_became_peered":"2024-03-29T08:19:14.278726+0000","last_unstale":"2024-03-29T08:19:14.701280+0000","last_undegraded":"2024-03-29T08:19:14.701280+0000","last_fullsized":"2024-03-29T08:19:14.701280+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:35.048072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687660+0000","last_change":"2024-03-29T08:19:14.679888+0000","last_active":"2024-03-29T08:19:14.687660+0000","last_peered":"2024-03-29T08:19:14.687660+0000","last_clean":"2024-03-29T08:19:14.687660+0000","last_became_active":"2024-03-29T08:19:14.679674+0000","last_became_peered":"2024-03-29T08:19:14.679674+0000","last_unstale":"2024-03-29T08:19:14.687660+0000","last_undegraded":"2024-03-29T08:19:14.687660+0000","last_fullsized":"2024-03-29T08:19:14.687660+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:55:24.587078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701177+0000","last_change":"2024-03-29T08:19:14.280371+0000","last_active":"2024-03-29T08:19:14.701177+0000","last_peered":"2024-03-29T08:19:14.701177+0000","last_clean":"2024-03-29T08:19:14.701177+0000","last_became_active":"2024-03-29T08:19:14.280045+0000","last_became_peered":"2024-03-29T08:19:14.280045+0000","last_unstale":"2024-03-29T08:19:14.701177+0000","last_undegraded":"2024-03-29T08:19:14.701177+0000","last_fullsized":"2024-03-29T08:19:14.701177+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:20.730956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.681942+0000","last_change":"2024-03-29T08:19:14.677276+0000","last_active":"2024-03-29T08:19:14.681942+0000","last_peered":"2024-03-29T08:19:14.681942+0000","last_clean":"2024-03-29T08:19:14.681942+0000","last_became_active":"2024-03-29T08:19:14.677049+0000","last_became_peered":"2024-03-29T08:19:14.677049+0000","last_unstale":"2024-03-29T08:19:14.681942+0000","last_undegraded":"2024-03-29T08:19:14.681942+0000","last_fullsized":"2024-03-29T08:19:14.681942+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:27.668682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.678961+0000","last_change":"2024-03-29T08:19:14.285270+0000","last_active":"2024-03-29T08:19:14.678961+0000","last_peered":"2024-03-29T08:19:14.678961+0000","last_clean":"2024-03-29T08:19:14.678961+0000","last_became_active":"2024-03-29T08:19:14.283882+0000","last_became_peered":"2024-03-29T08:19:14.283882+0000","last_unstale":"2024-03-29T08:19:14.678961+0000","last_undegraded":"2024-03-29T08:19:14.678961+0000","last_fullsized":"2024-03-29T08:19:14.678961+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:09:33.409328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702457+0000","last_change":"2024-03-29T08:19:14.696486+0000","last_active":"2024-03-29T08:19:14.702457+0000","last_peered":"2024-03-29T08:19:14.702457+0000","last_clean":"2024-03-29T08:19:14.702457+0000","last_became_active":"2024-03-29T08:19:14.696236+0000","last_became_peered":"2024-03-29T08:19:14.696236+0000","last_unstale":"2024-03-29T08:19:14.702457+0000","last_undegraded":"2024-03-29T08:19:14.702457+0000","last_fullsized":"2024-03-29T08:19:14.702457+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:08:58.521804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.671922+0000","last_change":"2024-03-29T08:19:14.275451+0000","last_active":"2024-03-29T08:19:14.671922+0000","last_peered":"2024-03-29T08:19:14.671922+0000","last_clean":"2024-03-29T08:19:14.671922+0000","last_became_active":"2024-03-29T08:19:14.275148+0000","last_became_peered":"2024-03-29T08:19:14.275148+0000","last_unstale":"2024-03-29T08:19:14.671922+0000","last_undegraded":"2024-03-29T08:19:14.671922+0000","last_fullsized":"2024-03-29T08:19:14.671922+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:57:14.581364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694290+0000","last_change":"2024-03-29T08:19:14.690396+0000","last_active":"2024-03-29T08:19:14.694290+0000","last_peered":"2024-03-29T08:19:14.694290+0000","last_clean":"2024-03-29T08:19:14.694290+0000","last_became_active":"2024-03-29T08:19:14.690106+0000","last_became_peered":"2024-03-29T08:19:14.690106+0000","last_unstale":"2024-03-29T08:19:14.694290+0000","last_undegraded":"2024-03-29T08:19:14.694290+0000","last_fullsized":"2024-03-29T08:19:14.694290+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:54:09.222693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.672936+0000","last_change":"2024-03-29T08:19:14.286636+0000","last_active":"2024-03-29T08:19:14.672936+0000","last_peered":"2024-03-29T08:19:14.672936+0000","last_clean":"2024-03-29T08:19:14.672936+0000","last_became_active":"2024-03-29T08:19:14.285309+0000","last_became_peered":"2024-03-29T08:19:14.285309+0000","last_unstale":"2024-03-29T08:19:14.672936+0000","last_undegraded":"2024-03-29T08:19:14.672936+0000","last_fullsized":"2024-03-29T08:19:14.672936+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:58:40.235616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700952+0000","last_change":"2024-03-29T08:19:14.694063+0000","last_active":"2024-03-29T08:19:14.700952+0000","last_peered":"2024-03-29T08:19:14.700952+0000","last_clean":"2024-03-29T08:19:14.700952+0000","last_became_active":"2024-03-29T08:19:14.693817+0000","last_became_peered":"2024-03-29T08:19:14.693817+0000","last_unstale":"2024-03-29T08:19:14.700952+0000","last_undegraded":"2024-03-29T08:19:14.700952+0000","last_fullsized":"2024-03-29T08:19:14.700952+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:30:57.036017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.671221+0000","last_change":"2024-03-29T08:19:14.283531+0000","last_active":"2024-03-29T08:19:14.671221+0000","last_peered":"2024-03-29T08:19:14.671221+0000","last_clean":"2024-03-29T08:19:14.671221+0000","last_became_active":"2024-03-29T08:19:14.283128+0000","last_became_peered":"2024-03-29T08:19:14.283128+0000","last_unstale":"2024-03-29T08:19:14.671221+0000","last_undegraded":"2024-03-29T08:19:14.671221+0000","last_fullsized":"2024-03-29T08:19:14.671221+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:36:01.837144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705548+0000","last_change":"2024-03-29T08:19:14.698954+0000","last_active":"2024-03-29T08:19:14.705548+0000","last_peered":"2024-03-29T08:19:14.705548+0000","last_clean":"2024-03-29T08:19:14.705548+0000","last_became_active":"2024-03-29T08:19:14.698317+0000","last_became_peered":"2024-03-29T08:19:14.698317+0000","last_unstale":"2024-03-29T08:19:14.705548+0000","last_undegraded":"2024-03-29T08:19:14.705548+0000","last_fullsized":"2024-03-29T08:19:14.705548+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:16:39.883087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700954+0000","last_change":"2024-03-29T08:19:14.285068+0000","last_active":"2024-03-29T08:19:14.700954+0000","last_peered":"2024-03-29T08:19:14.700954+0000","last_clean":"2024-03-29T08:19:14.700954+0000","last_became_active":"2024-03-29T08:19:14.284830+0000","last_became_peered":"2024-03-29T08:19:14.284830+0000","last_unstale":"2024-03-29T08:19:14.700954+0000","last_undegraded":"2024-03-29T08:19:14.700954+0000","last_fullsized":"2024-03-29T08:19:14.700954+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:16:38.877179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705493+0000","last_change":"2024-03-29T08:19:14.704710+0000","last_active":"2024-03-29T08:19:14.705493+0000","last_peered":"2024-03-29T08:19:14.705493+0000","last_clean":"2024-03-29T08:19:14.705493+0000","last_became_active":"2024-03-29T08:19:14.704396+0000","last_became_peered":"2024-03-29T08:19:14.704396+0000","last_unstale":"2024-03-29T08:19:14.705493+0000","last_undegraded":"2024-03-29T08:19:14.705493+0000","last_fullsized":"2024-03-29T08:19:14.705493+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:24.176397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.706659+0000","last_change":"2024-03-29T08:19:14.280954+0000","last_active":"2024-03-29T08:19:14.706659+0000","last_peered":"2024-03-29T08:19:14.706659+0000","last_clean":"2024-03-29T08:19:14.706659+0000","last_became_active":"2024-03-29T08:19:14.280392+0000","last_became_peered":"2024-03-29T08:19:14.280392+0000","last_unstale":"2024-03-29T08:19:14.706659+0000","last_undegraded":"2024-03-29T08:19:14.706659+0000","last_fullsized":"2024-03-29T08:19:14.706659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:26.662774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.688513+0000","last_change":"2024-03-29T08:19:14.685902+0000","last_active":"2024-03-29T08:19:14.688513+0000","last_peered":"2024-03-29T08:19:14.688513+0000","last_clean":"2024-03-29T08:19:14.688513+0000","last_became_active":"2024-03-29T08:19:14.685705+0000","last_became_peered":"2024-03-29T08:19:14.685705+0000","last_unstale":"2024-03-29T08:19:14.688513+0000","last_undegraded":"2024-03-29T08:19:14.688513+0000","last_fullsized":"2024-03-29T08:19:14.688513+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:08.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.695832+0000","last_change":"2024-03-29T08:19:14.272012+0000","last_active":"2024-03-29T08:19:14.695832+0000","last_peered":"2024-03-29T08:19:14.695832+0000","last_clean":"2024-03-29T08:19:14.695832+0000","last_became_active":"2024-03-29T08:19:14.271736+0000","last_became_peered":"2024-03-29T08:19:14.271736+0000","last_unstale":"2024-03-29T08:19:14.695832+0000","last_undegraded":"2024-03-29T08:19:14.695832+0000","last_fullsized":"2024-03-29T08:19:14.695832+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:33.064793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.696404+0000","last_change":"2024-03-29T08:19:14.690513+0000","last_active":"2024-03-29T08:19:14.696404+0000","last_peered":"2024-03-29T08:19:14.696404+0000","last_clean":"2024-03-29T08:19:14.696404+0000","last_became_active":"2024-03-29T08:19:14.690235+0000","last_became_peered":"2024-03-29T08:19:14.690235+0000","last_unstale":"2024-03-29T08:19:14.696404+0000","last_undegraded":"2024-03-29T08:19:14.696404+0000","last_fullsized":"2024-03-29T08:19:14.696404+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:01:53.702260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.693539+0000","last_change":"2024-03-29T08:19:14.274775+0000","last_active":"2024-03-29T08:19:14.693539+0000","last_peered":"2024-03-29T08:19:14.693539+0000","last_clean":"2024-03-29T08:19:14.693539+0000","last_became_active":"2024-03-29T08:19:14.274502+0000","last_became_peered":"2024-03-29T08:19:14.274502+0000","last_unstale":"2024-03-29T08:19:14.693539+0000","last_undegraded":"2024-03-29T08:19:14.693539+0000","last_fullsized":"2024-03-29T08:19:14.693539+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:20:37.829787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.681902+0000","last_change":"2024-03-29T08:19:14.675914+0000","last_active":"2024-03-29T08:19:14.681902+0000","last_peered":"2024-03-29T08:19:14.681902+0000","last_clean":"2024-03-29T08:19:14.681902+0000","last_became_active":"2024-03-29T08:19:14.675564+0000","last_became_peered":"2024-03-29T08:19:14.675564+0000","last_unstale":"2024-03-29T08:19:14.681902+0000","last_undegraded":"2024-03-29T08:19:14.681902+0000","last_fullsized":"2024-03-29T08:19:14.681902+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:36:09.793618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.682997+0000","last_change":"2024-03-29T08:19:14.281362+0000","last_active":"2024-03-29T08:19:14.682997+0000","last_peered":"2024-03-29T08:19:14.682997+0000","last_clean":"2024-03-29T08:19:14.682997+0000","last_became_active":"2024-03-29T08:19:14.281111+0000","last_became_peered":"2024-03-29T08:19:14.281111+0000","last_unstale":"2024-03-29T08:19:14.682997+0000","last_undegraded":"2024-03-29T08:19:14.682997+0000","last_fullsized":"2024-03-29T08:19:14.682997+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:26.870833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700427+0000","last_change":"2024-03-29T08:19:14.698963+0000","last_active":"2024-03-29T08:19:14.700427+0000","last_peered":"2024-03-29T08:19:14.700427+0000","last_clean":"2024-03-29T08:19:14.700427+0000","last_became_active":"2024-03-29T08:19:14.698502+0000","last_became_peered":"2024-03-29T08:19:14.698502+0000","last_unstale":"2024-03-29T08:19:14.700427+0000","last_undegraded":"2024-03-29T08:19:14.700427+0000","last_fullsized":"2024-03-29T08:19:14.700427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:21:27.491192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700579+0000","last_change":"2024-03-29T08:19:14.284917+0000","last_active":"2024-03-29T08:19:14.700579+0000","last_peered":"2024-03-29T08:19:14.700579+0000","last_clean":"2024-03-29T08:19:14.700579+0000","last_became_active":"2024-03-29T08:19:14.284667+0000","last_became_peered":"2024-03-29T08:19:14.284667+0000","last_unstale":"2024-03-29T08:19:14.700579+0000","last_undegraded":"2024-03-29T08:19:14.700579+0000","last_fullsized":"2024-03-29T08:19:14.700579+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:42.739141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.704840+0000","last_change":"2024-03-29T08:19:14.703652+0000","last_active":"2024-03-29T08:19:14.704840+0000","last_peered":"2024-03-29T08:19:14.704840+0000","last_clean":"2024-03-29T08:19:14.704840+0000","last_became_active":"2024-03-29T08:19:14.703386+0000","last_became_peered":"2024-03-29T08:19:14.703386+0000","last_unstale":"2024-03-29T08:19:14.704840+0000","last_undegraded":"2024-03-29T08:19:14.704840+0000","last_fullsized":"2024-03-29T08:19:14.704840+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:55:15.376664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702612+0000","last_change":"2024-03-29T08:19:14.287908+0000","last_active":"2024-03-29T08:19:14.702612+0000","last_peered":"2024-03-29T08:19:14.702612+0000","last_clean":"2024-03-29T08:19:14.702612+0000","last_became_active":"2024-03-29T08:19:14.287562+0000","last_became_peered":"2024-03-29T08:19:14.287562+0000","last_unstale":"2024-03-29T08:19:14.702612+0000","last_undegraded":"2024-03-29T08:19:14.702612+0000","last_fullsized":"2024-03-29T08:19:14.702612+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:02:28.529095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.687362+0000","last_change":"2024-03-29T08:19:14.682493+0000","last_active":"2024-03-29T08:19:14.687362+0000","last_peered":"2024-03-29T08:19:14.687362+0000","last_clean":"2024-03-29T08:19:14.687362+0000","last_became_active":"2024-03-29T08:19:14.678342+0000","last_became_peered":"2024-03-29T08:19:14.678342+0000","last_unstale":"2024-03-29T08:19:14.687362+0000","last_undegraded":"2024-03-29T08:19:14.687362+0000","last_fullsized":"2024-03-29T08:19:14.687362+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:36.053980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.695684+0000","last_change":"2024-03-29T08:19:14.272763+0000","last_active":"2024-03-29T08:19:14.695684+0000","last_peered":"2024-03-29T08:19:14.695684+0000","last_clean":"2024-03-29T08:19:14.695684+0000","last_became_active":"2024-03-29T08:19:14.272450+0000","last_became_peered":"2024-03-29T08:19:14.272450+0000","last_unstale":"2024-03-29T08:19:14.695684+0000","last_undegraded":"2024-03-29T08:19:14.695684+0000","last_fullsized":"2024-03-29T08:19:14.695684+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:19.581981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-29T08:19:16.660513+0000","last_change":"2024-03-29T08:19:14.676556+0000","last_active":"2024-03-29T08:19:16.660513+0000","last_peered":"2024-03-29T08:19:16.660513+0000","last_clean":"2024-03-29T08:19:16.660513+0000","last_became_active":"2024-03-29T08:19:14.676341+0000","last_became_peered":"2024-03-29T08:19:14.676341+0000","last_unstale":"2024-03-29T08:19:16.660513+0000","last_undegraded":"2024-03-29T08:19:16.660513+0000","last_fullsized":"2024-03-29T08:19:16.660513+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:05:38.988839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701079+0000","last_change":"2024-03-29T08:19:14.287716+0000","last_active":"2024-03-29T08:19:14.701079+0000","last_peered":"2024-03-29T08:19:14.701079+0000","last_clean":"2024-03-29T08:19:14.701079+0000","last_became_active":"2024-03-29T08:19:14.287437+0000","last_became_peered":"2024-03-29T08:19:14.287437+0000","last_unstale":"2024-03-29T08:19:14.701079+0000","last_undegraded":"2024-03-29T08:19:14.701079+0000","last_fullsized":"2024-03-29T08:19:14.701079+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:08:57.515896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.694406+0000","last_change":"2024-03-29T08:19:14.689798+0000","last_active":"2024-03-29T08:19:14.694406+0000","last_peered":"2024-03-29T08:19:14.694406+0000","last_clean":"2024-03-29T08:19:14.694406+0000","last_became_active":"2024-03-29T08:19:14.689479+0000","last_became_peered":"2024-03-29T08:19:14.689479+0000","last_unstale":"2024-03-29T08:19:14.694406+0000","last_undegraded":"2024-03-29T08:19:14.694406+0000","last_fullsized":"2024-03-29T08:19:14.694406+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:20:03.888899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.667785+0000","last_change":"2024-03-29T08:19:14.281935+0000","last_active":"2024-03-29T08:19:14.667785+0000","last_peered":"2024-03-29T08:19:14.667785+0000","last_clean":"2024-03-29T08:19:14.667785+0000","last_became_active":"2024-03-29T08:19:14.281670+0000","last_became_peered":"2024-03-29T08:19:14.281670+0000","last_unstale":"2024-03-29T08:19:14.667785+0000","last_undegraded":"2024-03-29T08:19:14.667785+0000","last_fullsized":"2024-03-29T08:19:14.667785+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:49:03.171063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.698797+0000","last_change":"2024-03-29T08:19:14.689677+0000","last_active":"2024-03-29T08:19:14.698797+0000","last_peered":"2024-03-29T08:19:14.698797+0000","last_clean":"2024-03-29T08:19:14.698797+0000","last_became_active":"2024-03-29T08:19:14.689343+0000","last_became_peered":"2024-03-29T08:19:14.689343+0000","last_unstale":"2024-03-29T08:19:14.698797+0000","last_undegraded":"2024-03-29T08:19:14.698797+0000","last_fullsized":"2024-03-29T08:19:14.698797+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:55.247391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702821+0000","last_change":"2024-03-29T08:19:14.279362+0000","last_active":"2024-03-29T08:19:14.702821+0000","last_peered":"2024-03-29T08:19:14.702821+0000","last_clean":"2024-03-29T08:19:14.702821+0000","last_became_active":"2024-03-29T08:19:14.278866+0000","last_became_peered":"2024-03-29T08:19:14.278866+0000","last_unstale":"2024-03-29T08:19:14.702821+0000","last_undegraded":"2024-03-29T08:19:14.702821+0000","last_fullsized":"2024-03-29T08:19:14.702821+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:27:45.403801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-29T08:19:16.660698+0000","last_change":"2024-03-29T08:19:14.679765+0000","last_active":"2024-03-29T08:19:16.660698+0000","last_peered":"2024-03-29T08:19:16.660698+0000","last_clean":"2024-03-29T08:19:16.660698+0000","last_became_active":"2024-03-29T08:19:14.679529+0000","last_became_peered":"2024-03-29T08:19:14.679529+0000","last_unstale":"2024-03-29T08:19:16.660698+0000","last_undegraded":"2024-03-29T08:19:16.660698+0000","last_fullsized":"2024-03-29T08:19:16.660698+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:18.770880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.673192+0000","last_change":"2024-03-29T08:19:14.272023+0000","last_active":"2024-03-29T08:19:14.673192+0000","last_peered":"2024-03-29T08:19:14.673192+0000","last_clean":"2024-03-29T08:19:14.673192+0000","last_became_active":"2024-03-29T08:19:14.271722+0000","last_became_peered":"2024-03-29T08:19:14.271722+0000","last_unstale":"2024-03-29T08:19:14.673192+0000","last_undegraded":"2024-03-29T08:19:14.673192+0000","last_fullsized":"2024-03-29T08:19:14.673192+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:26.662774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.699233+0000","last_change":"2024-03-29T08:19:14.694537+0000","last_active":"2024-03-29T08:19:14.699233+0000","last_peered":"2024-03-29T08:19:14.699233+0000","last_clean":"2024-03-29T08:19:14.699233+0000","last_became_active":"2024-03-29T08:19:14.694296+0000","last_became_peered":"2024-03-29T08:19:14.694296+0000","last_unstale":"2024-03-29T08:19:14.699233+0000","last_undegraded":"2024-03-29T08:19:14.699233+0000","last_fullsized":"2024-03-29T08:19:14.699233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:37:16.877010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.673618+0000","last_change":"2024-03-29T08:19:14.285590+0000","last_active":"2024-03-29T08:19:14.673618+0000","last_peered":"2024-03-29T08:19:14.673618+0000","last_clean":"2024-03-29T08:19:14.673618+0000","last_became_active":"2024-03-29T08:19:14.285211+0000","last_became_peered":"2024-03-29T08:19:14.285211+0000","last_unstale":"2024-03-29T08:19:14.673618+0000","last_undegraded":"2024-03-29T08:19:14.673618+0000","last_fullsized":"2024-03-29T08:19:14.673618+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:52:20.153715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.682877+0000","last_change":"2024-03-29T08:19:14.679560+0000","last_active":"2024-03-29T08:19:14.682877+0000","last_peered":"2024-03-29T08:19:14.682877+0000","last_clean":"2024-03-29T08:19:14.682877+0000","last_became_active":"2024-03-29T08:19:14.677544+0000","last_became_peered":"2024-03-29T08:19:14.677544+0000","last_unstale":"2024-03-29T08:19:14.682877+0000","last_undegraded":"2024-03-29T08:19:14.682877+0000","last_fullsized":"2024-03-29T08:19:14.682877+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:34.410446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.683328+0000","last_change":"2024-03-29T08:19:14.281813+0000","last_active":"2024-03-29T08:19:14.683328+0000","last_peered":"2024-03-29T08:19:14.683328+0000","last_clean":"2024-03-29T08:19:14.683328+0000","last_became_active":"2024-03-29T08:19:14.281356+0000","last_became_peered":"2024-03-29T08:19:14.281356+0000","last_unstale":"2024-03-29T08:19:14.683328+0000","last_undegraded":"2024-03-29T08:19:14.683328+0000","last_fullsized":"2024-03-29T08:19:14.683328+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:04:28.023637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.697297+0000","last_change":"2024-03-29T08:19:14.691637+0000","last_active":"2024-03-29T08:19:14.697297+0000","last_peered":"2024-03-29T08:19:14.697297+0000","last_clean":"2024-03-29T08:19:14.697297+0000","last_became_active":"2024-03-29T08:19:14.691391+0000","last_became_peered":"2024-03-29T08:19:14.691391+0000","last_unstale":"2024-03-29T08:19:14.697297+0000","last_undegraded":"2024-03-29T08:19:14.697297+0000","last_fullsized":"2024-03-29T08:19:14.697297+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:11:13.363737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.697667+0000","last_change":"2024-03-29T08:19:14.286350+0000","last_active":"2024-03-29T08:19:14.697667+0000","last_peered":"2024-03-29T08:19:14.697667+0000","last_clean":"2024-03-29T08:19:14.697667+0000","last_became_active":"2024-03-29T08:19:14.286055+0000","last_became_peered":"2024-03-29T08:19:14.286055+0000","last_unstale":"2024-03-29T08:19:14.697667+0000","last_undegraded":"2024-03-29T08:19:14.697667+0000","last_fullsized":"2024-03-29T08:19:14.697667+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:35.048072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700275+0000","last_change":"2024-03-29T08:19:14.694685+0000","last_active":"2024-03-29T08:19:14.700275+0000","last_peered":"2024-03-29T08:19:14.700275+0000","last_clean":"2024-03-29T08:19:14.700275+0000","last_became_active":"2024-03-29T08:19:14.693892+0000","last_became_peered":"2024-03-29T08:19:14.693892+0000","last_unstale":"2024-03-29T08:19:14.700275+0000","last_undegraded":"2024-03-29T08:19:14.700275+0000","last_fullsized":"2024-03-29T08:19:14.700275+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:02:39.788723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-29T08:19:16.660802+0000","last_change":"2024-03-29T08:19:14.283370+0000","last_active":"2024-03-29T08:19:16.660802+0000","last_peered":"2024-03-29T08:19:16.660802+0000","last_clean":"2024-03-29T08:19:16.660802+0000","last_became_active":"2024-03-29T08:19:14.282773+0000","last_became_peered":"2024-03-29T08:19:14.282773+0000","last_unstale":"2024-03-29T08:19:16.660802+0000","last_undegraded":"2024-03-29T08:19:16.660802+0000","last_fullsized":"2024-03-29T08:19:16.660802+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:59:27.620393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.705394+0000","last_change":"2024-03-29T08:19:14.701490+0000","last_active":"2024-03-29T08:19:14.705394+0000","last_peered":"2024-03-29T08:19:14.705394+0000","last_clean":"2024-03-29T08:19:14.705394+0000","last_became_active":"2024-03-29T08:19:14.701146+0000","last_became_peered":"2024-03-29T08:19:14.701146+0000","last_unstale":"2024-03-29T08:19:14.705394+0000","last_undegraded":"2024-03-29T08:19:14.705394+0000","last_fullsized":"2024-03-29T08:19:14.705394+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:32:52.080327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.676163+0000","last_change":"2024-03-29T08:19:14.279625+0000","last_active":"2024-03-29T08:19:14.676163+0000","last_peered":"2024-03-29T08:19:14.676163+0000","last_clean":"2024-03-29T08:19:14.676163+0000","last_became_active":"2024-03-29T08:19:14.279364+0000","last_became_peered":"2024-03-29T08:19:14.279364+0000","last_unstale":"2024-03-29T08:19:14.676163+0000","last_undegraded":"2024-03-29T08:19:14.676163+0000","last_fullsized":"2024-03-29T08:19:14.676163+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:20:02.882991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.685821+0000","last_change":"2024-03-29T08:19:14.679991+0000","last_active":"2024-03-29T08:19:14.685821+0000","last_peered":"2024-03-29T08:19:14.685821+0000","last_clean":"2024-03-29T08:19:14.685821+0000","last_became_active":"2024-03-29T08:19:14.679701+0000","last_became_peered":"2024-03-29T08:19:14.679701+0000","last_unstale":"2024-03-29T08:19:14.685821+0000","last_undegraded":"2024-03-29T08:19:14.685821+0000","last_fullsized":"2024-03-29T08:19:14.685821+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:08.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.675477+0000","last_change":"2024-03-29T08:19:14.284652+0000","last_active":"2024-03-29T08:19:14.675477+0000","last_peered":"2024-03-29T08:19:14.675477+0000","last_clean":"2024-03-29T08:19:14.675477+0000","last_became_active":"2024-03-29T08:19:14.284376+0000","last_became_peered":"2024-03-29T08:19:14.284376+0000","last_unstale":"2024-03-29T08:19:14.675477+0000","last_undegraded":"2024-03-29T08:19:14.675477+0000","last_fullsized":"2024-03-29T08:19:14.675477+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:20.730956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.701325+0000","last_change":"2024-03-29T08:19:14.693198+0000","last_active":"2024-03-29T08:19:14.701325+0000","last_peered":"2024-03-29T08:19:14.701325+0000","last_clean":"2024-03-29T08:19:14.701325+0000","last_became_active":"2024-03-29T08:19:14.692849+0000","last_became_peered":"2024-03-29T08:19:14.692849+0000","last_unstale":"2024-03-29T08:19:14.701325+0000","last_undegraded":"2024-03-29T08:19:14.701325+0000","last_fullsized":"2024-03-29T08:19:14.701325+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:46.218186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.676556+0000","last_change":"2024-03-29T08:19:14.284484+0000","last_active":"2024-03-29T08:19:14.676556+0000","last_peered":"2024-03-29T08:19:14.676556+0000","last_clean":"2024-03-29T08:19:14.676556+0000","last_became_active":"2024-03-29T08:19:14.284145+0000","last_became_peered":"2024-03-29T08:19:14.284145+0000","last_unstale":"2024-03-29T08:19:14.676556+0000","last_undegraded":"2024-03-29T08:19:14.676556+0000","last_fullsized":"2024-03-29T08:19:14.676556+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:33.404538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702578+0000","last_change":"2024-03-29T08:19:14.695365+0000","last_active":"2024-03-29T08:19:14.702578+0000","last_peered":"2024-03-29T08:19:14.702578+0000","last_clean":"2024-03-29T08:19:14.702578+0000","last_became_active":"2024-03-29T08:19:14.695111+0000","last_became_peered":"2024-03-29T08:19:14.695111+0000","last_unstale":"2024-03-29T08:19:14.702578+0000","last_undegraded":"2024-03-29T08:19:14.702578+0000","last_fullsized":"2024-03-29T08:19:14.702578+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:15.077172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.672171+0000","last_change":"2024-03-29T08:19:14.270630+0000","last_active":"2024-03-29T08:19:14.672171+0000","last_peered":"2024-03-29T08:19:14.672171+0000","last_clean":"2024-03-29T08:19:14.672171+0000","last_became_active":"2024-03-29T08:19:14.270379+0000","last_became_peered":"2024-03-29T08:19:14.270379+0000","last_unstale":"2024-03-29T08:19:14.672171+0000","last_undegraded":"2024-03-29T08:19:14.672171+0000","last_fullsized":"2024-03-29T08:19:14.672171+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:30:18.530436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.700408+0000","last_change":"2024-03-29T08:19:14.698728+0000","last_active":"2024-03-29T08:19:14.700408+0000","last_peered":"2024-03-29T08:19:14.700408+0000","last_clean":"2024-03-29T08:19:14.700408+0000","last_became_active":"2024-03-29T08:19:14.698138+0000","last_became_peered":"2024-03-29T08:19:14.698138+0000","last_unstale":"2024-03-29T08:19:14.700408+0000","last_undegraded":"2024-03-29T08:19:14.700408+0000","last_fullsized":"2024-03-29T08:19:14.700408+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:07:30.640723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.683645+0000","last_change":"2024-03-29T08:19:14.281517+0000","last_active":"2024-03-29T08:19:14.683645+0000","last_peered":"2024-03-29T08:19:14.683645+0000","last_clean":"2024-03-29T08:19:14.683645+0000","last_became_active":"2024-03-29T08:19:14.281182+0000","last_became_peered":"2024-03-29T08:19:14.281182+0000","last_unstale":"2024-03-29T08:19:14.683645+0000","last_undegraded":"2024-03-29T08:19:14.683645+0000","last_fullsized":"2024-03-29T08:19:14.683645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:21:54.574176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.689204+0000","last_change":"2024-03-29T08:19:14.685120+0000","last_active":"2024-03-29T08:19:14.689204+0000","last_peered":"2024-03-29T08:19:14.689204+0000","last_clean":"2024-03-29T08:19:14.689204+0000","last_became_active":"2024-03-29T08:19:14.682875+0000","last_became_peered":"2024-03-29T08:19:14.682875+0000","last_unstale":"2024-03-29T08:19:14.689204+0000","last_undegraded":"2024-03-29T08:19:14.689204+0000","last_fullsized":"2024-03-29T08:19:14.689204+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:55:24.587078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.683597+0000","last_change":"2024-03-29T08:19:14.283788+0000","last_active":"2024-03-29T08:19:14.683597+0000","last_peered":"2024-03-29T08:19:14.683597+0000","last_clean":"2024-03-29T08:19:14.683597+0000","last_became_active":"2024-03-29T08:19:14.283586+0000","last_became_peered":"2024-03-29T08:19:14.283586+0000","last_unstale":"2024-03-29T08:19:14.683597+0000","last_undegraded":"2024-03-29T08:19:14.683597+0000","last_fullsized":"2024-03-29T08:19:14.683597+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:52:19.059268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":173,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.282014+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:19:14.282014+0000","last_peered":"2024-03-29T08:19:14.282014+0000","last_clean":"2024-03-29T08:19:14.282014+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:19:14.282014+0000","last_undegraded":"2024-03-29T08:19:14.282014+0000","last_fullsized":"2024-03-29T08:19:14.282014+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509458,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":150465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68799999999999994}]},{"osd":1,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.79800000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.78300000000000003}]},{"osd":3,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.88600000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.81299999999999994}]},{"osd":5,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.873},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.90700000000000003}]}]},{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":376832,"data_stored":143668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55800000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.371}]},{"osd":5,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58499999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.56499999999999995}]}]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":364544,"data_stored":137393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.1709999999999998,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.81399999999999995}]},{"osd":1,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.363,"5min":2.363,"15min":2.363},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.93899999999999995}]},{"osd":3,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.89600000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.86799999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.91500000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.99199999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509460,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31484,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340356,"statfs":{"total":96636764160,"available":96604524544,"internally_reserved":0,"allocated":376832,"data_stored":143668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.35899999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.34200000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.437}]},{"osd":4,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.70599999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.73199999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.66100000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":0.83399999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509459,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31488,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340352,"statfs":{"total":96636764160,"available":96604520448,"internally_reserved":0,"allocated":376832,"data_stored":143668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.44}]},{"osd":1,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.54500000000000004}]},{"osd":4,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.502}]},{"osd":5,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.64500000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.625}]},{"osd":7,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.63600000000000001}]}]},{"osd":1,"up_from":15,"seq":64424509459,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":987136,"data_stored":740881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":4.2539999999999996,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.1909999999999998,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.42299999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51700000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.627}]},{"osd":5,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.76800000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509458,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31472,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340368,"statfs":{"total":96636764160,"available":96604536832,"internally_reserved":0,"allocated":364544,"data_stored":137393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":3.9359999999999999,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.497},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":8.0419999999999998,"5min":8.0419999999999998,"15min":8.0419999999999998},"last":0.51000000000000001}]},{"osd":2,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":4.907,"5min":4.907,"15min":4.907},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":7.9569999999999999,"5min":7.9569999999999999,"15min":7.9569999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.44400000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":4.4720000000000004,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.625},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":4.3920000000000003,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.58299999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.3200000000000003,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":4.4100000000000001,"5min":4.4100000000000001,"15min":4.4100000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":4.5209999999999999,"5min":4.5209999999999999,"15min":4.5209999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":4.4420000000000002,"5min":4.4420000000000002,"15min":4.4420000000000002},"last":0.63900000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":999424,"data_stored":747693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.503}]},{"osd":1,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63900000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.60899999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.45800000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.71899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:19:23.348 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:19:23.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.500+0000 7f3c02c00640 1 Processor -- start 2024-03-29T08:19:23.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.500+0000 7f3c02c00640 1 -- start start 2024-03-29T08:19:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.500+0000 7f3c02c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3bfc0d0660 0x7f3bfc0d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.500+0000 7f3c02c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3bfc15dc00 0x7f3bfc0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.500+0000 7f3c02c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bfc15cfb0 con 0x7f3bfc06b8b0 2024-03-29T08:19:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.500+0000 7f3c02c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bfc06c100 con 0x7f3bfc0d0660 2024-03-29T08:19:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.500+0000 7f3c02c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc0d0f80 con 0x7f3bfc15dc00 2024-03-29T08:19:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.500+0000 7f3bfbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3bfc0d0660 0x7f3bfc0d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.500+0000 7f3bfbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3bfc0d0660 0x7f3bfc0d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34878/0 (socket says 172.21.15.46:34878) 2024-03-29T08:19:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.500+0000 7f3bfbe00640 1 -- 172.21.15.46:0/1700275330 learned_addr learned my addr 172.21.15.46:0/1700275330 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.500+0000 7f3c00c00640 1 --2- 172.21.15.46:0/1700275330 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3bfc15dc00 0x7f3bfc0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.500+0000 7f3bfbe00640 1 -- 172.21.15.46:0/1700275330 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3bfc15dc00 msgr2=0x7f3bfc0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.500+0000 7f3bfbe00640 1 --2- 172.21.15.46:0/1700275330 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3bfc15dc00 0x7f3bfc0cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.500+0000 7f3bfbe00640 1 -- 172.21.15.46:0/1700275330 >> v1:172.21.15.73:6789/0 conn(0x7f3bfc06b8b0 legacy=0x7f3bfc0cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:23.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.500+0000 7f3bfbe00640 1 -- 172.21.15.46:0/1700275330 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc06be20 con 0x7f3bfc0d0660 2024-03-29T08:19:23.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.504+0000 7f3bfbe00640 1 --2- 172.21.15.46:0/1700275330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc0d0660 0x7f3bfc0d0a40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009870 tx=0x7f3bec01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783b49ba97df82fb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:23.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.504+0000 7f3bfb400640 1 -- 172.21.15.46:0/1700275330 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec0aa660 con 0x7f3bfc0d0660 2024-03-29T08:19:23.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.504+0000 7f3bfb400640 1 -- 172.21.15.46:0/1700275330 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bec0aa7c0 con 0x7f3bfc0d0660 2024-03-29T08:19:23.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.504+0000 7f3bfb400640 1 -- 172.21.15.46:0/1700275330 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bec0aab50 con 0x7f3bfc0d0660 2024-03-29T08:19:23.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.504+0000 7f3c02c00640 1 -- 172.21.15.46:0/1700275330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc0d0660 msgr2=0x7f3bfc0d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:23.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.504+0000 7f3c02c00640 1 --2- 172.21.15.46:0/1700275330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc0d0660 0x7f3bfc0d0a40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009870 tx=0x7f3bec01c530 comp rx=0 tx=0).stop 2024-03-29T08:19:23.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.504+0000 7f3c02c00640 1 -- 172.21.15.46:0/1700275330 shutdown_connections 2024-03-29T08:19:23.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.504+0000 7f3c02c00640 1 --2- 172.21.15.46:0/1700275330 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3bfc15dc00 0x7f3bfc0cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:23.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.504+0000 7f3c02c00640 1 --2- 172.21.15.46:0/1700275330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc0d0660 0x7f3bfc0d0a40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:23.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.504+0000 7f3c02c00640 1 -- 172.21.15.46:0/1700275330 >> 172.21.15.46:0/1700275330 conn(0x7f3bfc089fd0 msgr2=0x7f3bfc08a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:23.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.504+0000 7f3c02c00640 1 -- 172.21.15.46:0/1700275330 shutdown_connections 2024-03-29T08:19:23.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.504+0000 7f3c02c00640 1 -- 172.21.15.46:0/1700275330 wait complete. 2024-03-29T08:19:23.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c02c00640 1 Processor -- start 2024-03-29T08:19:23.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c02c00640 1 -- start start 2024-03-29T08:19:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c02c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3bfc06b8b0 0x7f3bfc1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c02c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bfc0d0660 0x7f3bfc0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c02c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc15dc00 0x7f3bfc155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c02c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc155440 con 0x7f3bfc15dc00 2024-03-29T08:19:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c02c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc1533e0 con 0x7f3bfc0d0660 2024-03-29T08:19:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c02c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bfc153100 con 0x7f3bfc06b8b0 2024-03-29T08:19:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c00c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc15dc00 0x7f3bfc155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c00c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc15dc00 0x7f3bfc155940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34880/0 (socket says 172.21.15.46:34880) 2024-03-29T08:19:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c00c00640 1 -- 172.21.15.46:0/452768190 learned_addr learned my addr 172.21.15.46:0/452768190 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:23.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3bfbe00640 1 --2- 172.21.15.46:0/452768190 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3bfc06b8b0 0x7f3bfc1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3bf3e00640 1 --2- 172.21.15.46:0/452768190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bfc0d0660 0x7f3bfc0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c00c00640 1 -- 172.21.15.46:0/452768190 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3bfc06b8b0 msgr2=0x7f3bfc1565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c00c00640 1 --2- 172.21.15.46:0/452768190 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3bfc06b8b0 0x7f3bfc1565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c00c00640 1 -- 172.21.15.46:0/452768190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bfc0d0660 msgr2=0x7f3bfc0b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c00c00640 1 --2- 172.21.15.46:0/452768190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bfc0d0660 0x7f3bfc0b32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c00c00640 1 -- 172.21.15.46:0/452768190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec0094d0 con 0x7f3bfc15dc00 2024-03-29T08:19:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3bf3e00640 1 --2- 172.21.15.46:0/452768190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bfc0d0660 0x7f3bfc0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c00c00640 1 --2- 172.21.15.46:0/452768190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc15dc00 0x7f3bfc155940 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4007e60 tx=0x7f3bf400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3bf8c00640 1 -- 172.21.15.46:0/452768190 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4014420 con 0x7f3bfc15dc00 2024-03-29T08:19:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3bf8c00640 1 -- 172.21.15.46:0/452768190 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf40145a0 con 0x7f3bfc15dc00 2024-03-29T08:19:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3bf8c00640 1 -- 172.21.15.46:0/452768190 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf4014950 con 0x7f3bfc15dc00 2024-03-29T08:19:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c02c00640 1 -- 172.21.15.46:0/452768190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bfc154e80 con 0x7f3bfc15dc00 2024-03-29T08:19:23.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.508+0000 7f3c02c00640 1 -- 172.21.15.46:0/452768190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bfc0b3800 con 0x7f3bfc15dc00 2024-03-29T08:19:23.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.512+0000 7f3c02c00640 1 -- 172.21.15.46:0/452768190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bfc066c90 con 0x7f3bfc15dc00 2024-03-29T08:19:23.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.512+0000 7f3bf8c00640 1 -- 172.21.15.46:0/452768190 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3bf4014ab0 con 0x7f3bfc15dc00 2024-03-29T08:19:23.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.512+0000 7f3bf8c00640 1 --2- 172.21.15.46:0/452768190 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3bd8066690 0x7f3bd8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:23.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.512+0000 7f3bf8c00640 1 -- 172.21.15.46:0/452768190 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bf40849a0 con 0x7f3bfc15dc00 2024-03-29T08:19:23.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.512+0000 7f3bfbe00640 1 --2- 172.21.15.46:0/452768190 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3bd8066690 0x7f3bd8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:23.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.512+0000 7f3bfbe00640 1 --2- 172.21.15.46:0/452768190 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3bd8066690 0x7f3bd8068b50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0ae000 tx=0x7f3bec0b5040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:23.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.516+0000 7f3bf8c00640 1 -- 172.21.15.46:0/452768190 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3bf4051d00 con 0x7f3bfc15dc00 2024-03-29T08:19:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.672+0000 7f3c02c00640 1 -- 172.21.15.46:0/452768190 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3bfc0ce9a0 con 0x7f3bd8066690 2024-03-29T08:19:23.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.680+0000 7f3bf8c00640 1 -- 172.21.15.46:0/452768190 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289236 (secure 0 0 0) 0x7f3bec0aa930 con 0x7f3bd8066690 2024-03-29T08:19:23.686 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:23.688 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:19:23.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.684+0000 7f3c02c00640 1 -- 172.21.15.46:0/452768190 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3bd8066690 msgr2=0x7f3bd8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:23.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.684+0000 7f3c02c00640 1 --2- 172.21.15.46:0/452768190 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3bd8066690 0x7f3bd8068b50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0ae000 tx=0x7f3bec0b5040 comp rx=0 tx=0).stop 2024-03-29T08:19:23.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.684+0000 7f3c02c00640 1 -- 172.21.15.46:0/452768190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc15dc00 msgr2=0x7f3bfc155940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.684+0000 7f3c02c00640 1 --2- 172.21.15.46:0/452768190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc15dc00 0x7f3bfc155940 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3bf4007e60 tx=0x7f3bf400e4b0 comp rx=0 tx=0).stop 2024-03-29T08:19:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.684+0000 7f3c02c00640 1 -- 172.21.15.46:0/452768190 shutdown_connections 2024-03-29T08:19:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.684+0000 7f3c02c00640 1 --2- 172.21.15.46:0/452768190 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3bd8066690 0x7f3bd8068b50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.684+0000 7f3c02c00640 1 --2- 172.21.15.46:0/452768190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bfc0d0660 0x7f3bfc0b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.684+0000 7f3c02c00640 1 --2- 172.21.15.46:0/452768190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bfc15dc00 0x7f3bfc155940 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.684+0000 7f3c02c00640 1 --2- 172.21.15.46:0/452768190 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3bfc06b8b0 0x7f3bfc1565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.684+0000 7f3c02c00640 1 -- 172.21.15.46:0/452768190 >> 172.21.15.46:0/452768190 conn(0x7f3bfc089fd0 msgr2=0x7f3bfc06ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.684+0000 7f3c02c00640 1 -- 172.21.15.46:0/452768190 shutdown_connections 2024-03-29T08:19:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.684+0000 7f3c02c00640 1 -- 172.21.15.46:0/452768190 wait complete. 2024-03-29T08:19:23.706 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-03-29T08:19:22.057859+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":177,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236920,"kb_used_data":4360,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754737800,"statfs":{"total":773094113280,"available":772851507200,"internally_reserved":0,"allocated":4464640,"data_stored":2439404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":238014009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.796261"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108597+0000","last_change":"2024-03-29T08:19:14.677048+0000","last_active":"2024-03-29T08:19:15.108597+0000","last_peered":"2024-03-29T08:19:15.108597+0000","last_clean":"2024-03-29T08:19:15.108597+0000","last_became_active":"2024-03-29T08:19:14.676802+0000","last_became_peered":"2024-03-29T08:19:14.676802+0000","last_unstale":"2024-03-29T08:19:15.108597+0000","last_undegraded":"2024-03-29T08:19:15.108597+0000","last_fullsized":"2024-03-29T08:19:15.108597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:37:16.877010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-29T08:19:16.660674+0000","last_change":"2024-03-29T08:19:14.702139+0000","last_active":"2024-03-29T08:19:16.660674+0000","last_peered":"2024-03-29T08:19:16.660674+0000","last_clean":"2024-03-29T08:19:16.660674+0000","last_became_active":"2024-03-29T08:19:14.701881+0000","last_became_peered":"2024-03-29T08:19:14.701881+0000","last_unstale":"2024-03-29T08:19:16.660674+0000","last_undegraded":"2024-03-29T08:19:16.660674+0000","last_fullsized":"2024-03-29T08:19:16.660674+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:35.312841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.106161+0000","last_change":"2024-03-29T08:19:14.677189+0000","last_active":"2024-03-29T08:19:15.106161+0000","last_peered":"2024-03-29T08:19:15.106161+0000","last_clean":"2024-03-29T08:19:15.106161+0000","last_became_active":"2024-03-29T08:19:14.676526+0000","last_became_peered":"2024-03-29T08:19:14.676526+0000","last_unstale":"2024-03-29T08:19:15.106161+0000","last_undegraded":"2024-03-29T08:19:15.106161+0000","last_fullsized":"2024-03-29T08:19:15.106161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:20.587889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109685+0000","last_change":"2024-03-29T08:19:14.699305+0000","last_active":"2024-03-29T08:19:15.109685+0000","last_peered":"2024-03-29T08:19:15.109685+0000","last_clean":"2024-03-29T08:19:15.109685+0000","last_became_active":"2024-03-29T08:19:14.698637+0000","last_became_peered":"2024-03-29T08:19:14.698637+0000","last_unstale":"2024-03-29T08:19:15.109685+0000","last_undegraded":"2024-03-29T08:19:15.109685+0000","last_fullsized":"2024-03-29T08:19:15.109685+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:35.234827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-29T08:19:16.660584+0000","last_change":"2024-03-29T08:19:14.682597+0000","last_active":"2024-03-29T08:19:16.660584+0000","last_peered":"2024-03-29T08:19:16.660584+0000","last_clean":"2024-03-29T08:19:16.660584+0000","last_became_active":"2024-03-29T08:19:14.682322+0000","last_became_peered":"2024-03-29T08:19:14.682322+0000","last_unstale":"2024-03-29T08:19:16.660584+0000","last_undegraded":"2024-03-29T08:19:16.660584+0000","last_fullsized":"2024-03-29T08:19:16.660584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:51:41.300882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109460+0000","last_change":"2024-03-29T08:19:14.691105+0000","last_active":"2024-03-29T08:19:15.109460+0000","last_peered":"2024-03-29T08:19:15.109460+0000","last_clean":"2024-03-29T08:19:15.109460+0000","last_became_active":"2024-03-29T08:19:14.690486+0000","last_became_peered":"2024-03-29T08:19:14.690486+0000","last_unstale":"2024-03-29T08:19:15.109460+0000","last_undegraded":"2024-03-29T08:19:15.109460+0000","last_fullsized":"2024-03-29T08:19:15.109460+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:21.736865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108455+0000","last_change":"2024-03-29T08:19:14.675543+0000","last_active":"2024-03-29T08:19:15.108455+0000","last_peered":"2024-03-29T08:19:15.108455+0000","last_clean":"2024-03-29T08:19:15.108455+0000","last_became_active":"2024-03-29T08:19:14.675201+0000","last_became_peered":"2024-03-29T08:19:14.675201+0000","last_unstale":"2024-03-29T08:19:15.108455+0000","last_undegraded":"2024-03-29T08:19:15.108455+0000","last_fullsized":"2024-03-29T08:19:15.108455+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:08:58.521804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108409+0000","last_change":"2024-03-29T08:19:14.681677+0000","last_active":"2024-03-29T08:19:15.108409+0000","last_peered":"2024-03-29T08:19:15.108409+0000","last_clean":"2024-03-29T08:19:15.108409+0000","last_became_active":"2024-03-29T08:19:14.680707+0000","last_became_peered":"2024-03-29T08:19:14.680707+0000","last_unstale":"2024-03-29T08:19:15.108409+0000","last_undegraded":"2024-03-29T08:19:15.108409+0000","last_fullsized":"2024-03-29T08:19:15.108409+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:04:29.029546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.106169+0000","last_change":"2024-03-29T08:19:14.703892+0000","last_active":"2024-03-29T08:19:15.106169+0000","last_peered":"2024-03-29T08:19:15.106169+0000","last_clean":"2024-03-29T08:19:15.106169+0000","last_became_active":"2024-03-29T08:19:14.701909+0000","last_became_peered":"2024-03-29T08:19:14.701909+0000","last_unstale":"2024-03-29T08:19:15.106169+0000","last_undegraded":"2024-03-29T08:19:15.106169+0000","last_fullsized":"2024-03-29T08:19:15.106169+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:57:15.587273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109722+0000","last_change":"2024-03-29T08:19:14.694312+0000","last_active":"2024-03-29T08:19:15.109722+0000","last_peered":"2024-03-29T08:19:15.109722+0000","last_clean":"2024-03-29T08:19:15.109722+0000","last_became_active":"2024-03-29T08:19:14.694027+0000","last_became_peered":"2024-03-29T08:19:14.694027+0000","last_unstale":"2024-03-29T08:19:15.109722+0000","last_undegraded":"2024-03-29T08:19:15.109722+0000","last_fullsized":"2024-03-29T08:19:15.109722+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:15:31.529776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.113011+0000","last_change":"2024-03-29T08:19:14.703541+0000","last_active":"2024-03-29T08:19:15.113011+0000","last_peered":"2024-03-29T08:19:15.113011+0000","last_clean":"2024-03-29T08:19:15.113011+0000","last_became_active":"2024-03-29T08:19:14.703317+0000","last_became_peered":"2024-03-29T08:19:14.703317+0000","last_unstale":"2024-03-29T08:19:15.113011+0000","last_undegraded":"2024-03-29T08:19:15.113011+0000","last_fullsized":"2024-03-29T08:19:15.113011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:18:37.057951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.106147+0000","last_change":"2024-03-29T08:19:14.684087+0000","last_active":"2024-03-29T08:19:15.106147+0000","last_peered":"2024-03-29T08:19:15.106147+0000","last_clean":"2024-03-29T08:19:15.106147+0000","last_became_active":"2024-03-29T08:19:14.683827+0000","last_became_peered":"2024-03-29T08:19:14.683827+0000","last_unstale":"2024-03-29T08:19:15.106147+0000","last_undegraded":"2024-03-29T08:19:15.106147+0000","last_fullsized":"2024-03-29T08:19:15.106147+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:21.736865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109448+0000","last_change":"2024-03-29T08:19:14.699133+0000","last_active":"2024-03-29T08:19:15.109448+0000","last_peered":"2024-03-29T08:19:15.109448+0000","last_clean":"2024-03-29T08:19:15.109448+0000","last_became_active":"2024-03-29T08:19:14.698579+0000","last_became_peered":"2024-03-29T08:19:14.698579+0000","last_unstale":"2024-03-29T08:19:15.109448+0000","last_undegraded":"2024-03-29T08:19:15.109448+0000","last_fullsized":"2024-03-29T08:19:15.109448+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:00:07.571380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109635+0000","last_change":"2024-03-29T08:19:14.694580+0000","last_active":"2024-03-29T08:19:15.109635+0000","last_peered":"2024-03-29T08:19:15.109635+0000","last_clean":"2024-03-29T08:19:15.109635+0000","last_became_active":"2024-03-29T08:19:14.694305+0000","last_became_peered":"2024-03-29T08:19:14.694305+0000","last_unstale":"2024-03-29T08:19:15.109635+0000","last_undegraded":"2024-03-29T08:19:15.109635+0000","last_fullsized":"2024-03-29T08:19:15.109635+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:19:13.272178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108382+0000","last_change":"2024-03-29T08:19:14.674438+0000","last_active":"2024-03-29T08:19:15.108382+0000","last_peered":"2024-03-29T08:19:15.108382+0000","last_clean":"2024-03-29T08:19:15.108382+0000","last_became_active":"2024-03-29T08:19:14.674172+0000","last_became_peered":"2024-03-29T08:19:14.674172+0000","last_unstale":"2024-03-29T08:19:15.108382+0000","last_undegraded":"2024-03-29T08:19:15.108382+0000","last_fullsized":"2024-03-29T08:19:15.108382+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:57:29.147117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108361+0000","last_change":"2024-03-29T08:19:14.681624+0000","last_active":"2024-03-29T08:19:15.108361+0000","last_peered":"2024-03-29T08:19:15.108361+0000","last_clean":"2024-03-29T08:19:15.108361+0000","last_became_active":"2024-03-29T08:19:14.681290+0000","last_became_peered":"2024-03-29T08:19:14.681290+0000","last_unstale":"2024-03-29T08:19:15.108361+0000","last_undegraded":"2024-03-29T08:19:15.108361+0000","last_fullsized":"2024-03-29T08:19:15.108361+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:34:59.393546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.113062+0000","last_change":"2024-03-29T08:19:14.695305+0000","last_active":"2024-03-29T08:19:15.113062+0000","last_peered":"2024-03-29T08:19:15.113062+0000","last_clean":"2024-03-29T08:19:15.113062+0000","last_became_active":"2024-03-29T08:19:14.694942+0000","last_became_peered":"2024-03-29T08:19:14.694942+0000","last_unstale":"2024-03-29T08:19:15.113062+0000","last_undegraded":"2024-03-29T08:19:15.113062+0000","last_fullsized":"2024-03-29T08:19:15.113062+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:19:41.115142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109565+0000","last_change":"2024-03-29T08:19:14.690840+0000","last_active":"2024-03-29T08:19:15.109565+0000","last_peered":"2024-03-29T08:19:15.109565+0000","last_clean":"2024-03-29T08:19:15.109565+0000","last_became_active":"2024-03-29T08:19:14.689554+0000","last_became_peered":"2024-03-29T08:19:14.689554+0000","last_unstale":"2024-03-29T08:19:15.109565+0000","last_undegraded":"2024-03-29T08:19:15.109565+0000","last_fullsized":"2024-03-29T08:19:15.109565+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:49:04.176971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109477+0000","last_change":"2024-03-29T08:19:14.689362+0000","last_active":"2024-03-29T08:19:15.109477+0000","last_peered":"2024-03-29T08:19:15.109477+0000","last_clean":"2024-03-29T08:19:15.109477+0000","last_became_active":"2024-03-29T08:19:14.689050+0000","last_became_peered":"2024-03-29T08:19:14.689050+0000","last_unstale":"2024-03-29T08:19:15.109477+0000","last_undegraded":"2024-03-29T08:19:15.109477+0000","last_fullsized":"2024-03-29T08:19:15.109477+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:20:38.835695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109480+0000","last_change":"2024-03-29T08:19:14.683957+0000","last_active":"2024-03-29T08:19:15.109480+0000","last_peered":"2024-03-29T08:19:15.109480+0000","last_clean":"2024-03-29T08:19:15.109480+0000","last_became_active":"2024-03-29T08:19:14.683710+0000","last_became_peered":"2024-03-29T08:19:14.683710+0000","last_unstale":"2024-03-29T08:19:15.109480+0000","last_undegraded":"2024-03-29T08:19:15.109480+0000","last_fullsized":"2024-03-29T08:19:15.109480+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:11:49.840732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108450+0000","last_change":"2024-03-29T08:19:14.685648+0000","last_active":"2024-03-29T08:19:15.108450+0000","last_peered":"2024-03-29T08:19:15.108450+0000","last_clean":"2024-03-29T08:19:15.108450+0000","last_became_active":"2024-03-29T08:19:14.685404+0000","last_became_peered":"2024-03-29T08:19:14.685404+0000","last_unstale":"2024-03-29T08:19:15.108450+0000","last_undegraded":"2024-03-29T08:19:15.108450+0000","last_fullsized":"2024-03-29T08:19:15.108450+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:14:08.867120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108479+0000","last_change":"2024-03-29T08:19:14.698732+0000","last_active":"2024-03-29T08:19:15.108479+0000","last_peered":"2024-03-29T08:19:15.108479+0000","last_clean":"2024-03-29T08:19:15.108479+0000","last_became_active":"2024-03-29T08:19:14.698484+0000","last_became_peered":"2024-03-29T08:19:14.698484+0000","last_unstale":"2024-03-29T08:19:15.108479+0000","last_undegraded":"2024-03-29T08:19:15.108479+0000","last_fullsized":"2024-03-29T08:19:15.108479+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:08.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109373+0000","last_change":"2024-03-29T08:19:14.685174+0000","last_active":"2024-03-29T08:19:15.109373+0000","last_peered":"2024-03-29T08:19:15.109373+0000","last_clean":"2024-03-29T08:19:15.109373+0000","last_became_active":"2024-03-29T08:19:14.684605+0000","last_became_peered":"2024-03-29T08:19:14.684605+0000","last_unstale":"2024-03-29T08:19:15.109373+0000","last_undegraded":"2024-03-29T08:19:15.109373+0000","last_fullsized":"2024-03-29T08:19:15.109373+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:34:59.393546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108423+0000","last_change":"2024-03-29T08:19:14.698117+0000","last_active":"2024-03-29T08:19:15.108423+0000","last_peered":"2024-03-29T08:19:15.108423+0000","last_clean":"2024-03-29T08:19:15.108423+0000","last_became_active":"2024-03-29T08:19:14.697861+0000","last_became_peered":"2024-03-29T08:19:14.697861+0000","last_unstale":"2024-03-29T08:19:15.108423+0000","last_undegraded":"2024-03-29T08:19:15.108423+0000","last_fullsized":"2024-03-29T08:19:15.108423+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:14:08.867120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109771+0000","last_change":"2024-03-29T08:19:14.695925+0000","last_active":"2024-03-29T08:19:15.109771+0000","last_peered":"2024-03-29T08:19:15.109771+0000","last_clean":"2024-03-29T08:19:15.109771+0000","last_became_active":"2024-03-29T08:19:14.695180+0000","last_became_peered":"2024-03-29T08:19:14.695180+0000","last_unstale":"2024-03-29T08:19:15.109771+0000","last_undegraded":"2024-03-29T08:19:15.109771+0000","last_fullsized":"2024-03-29T08:19:15.109771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:20:02.908287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109738+0000","last_change":"2024-03-29T08:19:14.696275+0000","last_active":"2024-03-29T08:19:15.109738+0000","last_peered":"2024-03-29T08:19:15.109738+0000","last_clean":"2024-03-29T08:19:15.109738+0000","last_became_active":"2024-03-29T08:19:14.696065+0000","last_became_peered":"2024-03-29T08:19:14.696065+0000","last_unstale":"2024-03-29T08:19:15.109738+0000","last_undegraded":"2024-03-29T08:19:15.109738+0000","last_fullsized":"2024-03-29T08:19:15.109738+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:00:46.960177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-29T08:19:16.660670+0000","last_change":"2024-03-29T08:19:14.677537+0000","last_active":"2024-03-29T08:19:16.660670+0000","last_peered":"2024-03-29T08:19:16.660670+0000","last_clean":"2024-03-29T08:19:16.660670+0000","last_became_active":"2024-03-29T08:19:14.677288+0000","last_became_peered":"2024-03-29T08:19:14.677288+0000","last_unstale":"2024-03-29T08:19:16.660670+0000","last_undegraded":"2024-03-29T08:19:16.660670+0000","last_fullsized":"2024-03-29T08:19:16.660670+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:59:18.746998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108537+0000","last_change":"2024-03-29T08:19:14.684317+0000","last_active":"2024-03-29T08:19:15.108537+0000","last_peered":"2024-03-29T08:19:15.108537+0000","last_clean":"2024-03-29T08:19:15.108537+0000","last_became_active":"2024-03-29T08:19:14.683125+0000","last_became_peered":"2024-03-29T08:19:14.683125+0000","last_unstale":"2024-03-29T08:19:15.108537+0000","last_undegraded":"2024-03-29T08:19:15.108537+0000","last_fullsized":"2024-03-29T08:19:15.108537+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:52:54.511385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108429+0000","last_change":"2024-03-29T08:19:14.701650+0000","last_active":"2024-03-29T08:19:15.108429+0000","last_peered":"2024-03-29T08:19:15.108429+0000","last_clean":"2024-03-29T08:19:15.108429+0000","last_became_active":"2024-03-29T08:19:14.697686+0000","last_became_peered":"2024-03-29T08:19:14.697686+0000","last_unstale":"2024-03-29T08:19:15.108429+0000","last_undegraded":"2024-03-29T08:19:15.108429+0000","last_fullsized":"2024-03-29T08:19:15.108429+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:10:23.783843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108398+0000","last_change":"2024-03-29T08:19:14.684872+0000","last_active":"2024-03-29T08:19:15.108398+0000","last_peered":"2024-03-29T08:19:15.108398+0000","last_clean":"2024-03-29T08:19:15.108398+0000","last_became_active":"2024-03-29T08:19:14.684622+0000","last_became_peered":"2024-03-29T08:19:14.684622+0000","last_unstale":"2024-03-29T08:19:15.108398+0000","last_undegraded":"2024-03-29T08:19:15.108398+0000","last_fullsized":"2024-03-29T08:19:15.108398+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:00:07.571380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108463+0000","last_change":"2024-03-29T08:19:14.681150+0000","last_active":"2024-03-29T08:19:15.108463+0000","last_peered":"2024-03-29T08:19:15.108463+0000","last_clean":"2024-03-29T08:19:15.108463+0000","last_became_active":"2024-03-29T08:19:14.680888+0000","last_became_peered":"2024-03-29T08:19:14.680888+0000","last_unstale":"2024-03-29T08:19:15.108463+0000","last_undegraded":"2024-03-29T08:19:15.108463+0000","last_fullsized":"2024-03-29T08:19:15.108463+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:10:23.783843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.112961+0000","last_change":"2024-03-29T08:19:14.698385+0000","last_active":"2024-03-29T08:19:15.112961+0000","last_peered":"2024-03-29T08:19:15.112961+0000","last_clean":"2024-03-29T08:19:15.112961+0000","last_became_active":"2024-03-29T08:19:14.698119+0000","last_became_peered":"2024-03-29T08:19:14.698119+0000","last_unstale":"2024-03-29T08:19:15.112961+0000","last_undegraded":"2024-03-29T08:19:15.112961+0000","last_fullsized":"2024-03-29T08:19:15.112961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:03:28.287471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109554+0000","last_change":"2024-03-29T08:19:14.686891+0000","last_active":"2024-03-29T08:19:15.109554+0000","last_peered":"2024-03-29T08:19:15.109554+0000","last_clean":"2024-03-29T08:19:15.109554+0000","last_became_active":"2024-03-29T08:19:14.686615+0000","last_became_peered":"2024-03-29T08:19:14.686615+0000","last_unstale":"2024-03-29T08:19:15.109554+0000","last_undegraded":"2024-03-29T08:19:15.109554+0000","last_fullsized":"2024-03-29T08:19:15.109554+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:10:23.783843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.113078+0000","last_change":"2024-03-29T08:19:14.278073+0000","last_active":"2024-03-29T08:19:15.113078+0000","last_peered":"2024-03-29T08:19:15.113078+0000","last_clean":"2024-03-29T08:19:15.113078+0000","last_became_active":"2024-03-29T08:19:14.277778+0000","last_became_peered":"2024-03-29T08:19:14.277778+0000","last_unstale":"2024-03-29T08:19:15.113078+0000","last_undegraded":"2024-03-29T08:19:15.113078+0000","last_fullsized":"2024-03-29T08:19:15.113078+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:36:08.787710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108620+0000","last_change":"2024-03-29T08:19:14.682096+0000","last_active":"2024-03-29T08:19:15.108620+0000","last_peered":"2024-03-29T08:19:15.108620+0000","last_clean":"2024-03-29T08:19:15.108620+0000","last_became_active":"2024-03-29T08:19:14.681779+0000","last_became_peered":"2024-03-29T08:19:14.681779+0000","last_unstale":"2024-03-29T08:19:15.108620+0000","last_undegraded":"2024-03-29T08:19:15.108620+0000","last_fullsized":"2024-03-29T08:19:15.108620+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:52:20.065176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109754+0000","last_change":"2024-03-29T08:19:14.281218+0000","last_active":"2024-03-29T08:19:15.109754+0000","last_peered":"2024-03-29T08:19:15.109754+0000","last_clean":"2024-03-29T08:19:15.109754+0000","last_became_active":"2024-03-29T08:19:14.280883+0000","last_became_peered":"2024-03-29T08:19:14.280883+0000","last_unstale":"2024-03-29T08:19:15.109754+0000","last_undegraded":"2024-03-29T08:19:15.109754+0000","last_fullsized":"2024-03-29T08:19:15.109754+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:14:22.394882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108578+0000","last_change":"2024-03-29T08:19:14.698993+0000","last_active":"2024-03-29T08:19:15.108578+0000","last_peered":"2024-03-29T08:19:15.108578+0000","last_clean":"2024-03-29T08:19:15.108578+0000","last_became_active":"2024-03-29T08:19:14.698757+0000","last_became_peered":"2024-03-29T08:19:14.698757+0000","last_unstale":"2024-03-29T08:19:15.108578+0000","last_undegraded":"2024-03-29T08:19:15.108578+0000","last_fullsized":"2024-03-29T08:19:15.108578+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:34:59.393546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109547+0000","last_change":"2024-03-29T08:19:14.283589+0000","last_active":"2024-03-29T08:19:15.109547+0000","last_peered":"2024-03-29T08:19:15.109547+0000","last_clean":"2024-03-29T08:19:15.109547+0000","last_became_active":"2024-03-29T08:19:14.282933+0000","last_became_peered":"2024-03-29T08:19:14.282933+0000","last_unstale":"2024-03-29T08:19:15.109547+0000","last_undegraded":"2024-03-29T08:19:15.109547+0000","last_fullsized":"2024-03-29T08:19:15.109547+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:16:08.725000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109712+0000","last_change":"2024-03-29T08:19:14.696813+0000","last_active":"2024-03-29T08:19:15.109712+0000","last_peered":"2024-03-29T08:19:15.109712+0000","last_clean":"2024-03-29T08:19:15.109712+0000","last_became_active":"2024-03-29T08:19:14.696519+0000","last_became_peered":"2024-03-29T08:19:14.696519+0000","last_unstale":"2024-03-29T08:19:15.109712+0000","last_undegraded":"2024-03-29T08:19:15.109712+0000","last_fullsized":"2024-03-29T08:19:15.109712+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:46:59.856286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109703+0000","last_change":"2024-03-29T08:19:14.279768+0000","last_active":"2024-03-29T08:19:15.109703+0000","last_peered":"2024-03-29T08:19:15.109703+0000","last_clean":"2024-03-29T08:19:15.109703+0000","last_became_active":"2024-03-29T08:19:14.279493+0000","last_became_peered":"2024-03-29T08:19:14.279493+0000","last_unstale":"2024-03-29T08:19:15.109703+0000","last_undegraded":"2024-03-29T08:19:15.109703+0000","last_fullsized":"2024-03-29T08:19:15.109703+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:11:12.357829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.106139+0000","last_change":"2024-03-29T08:19:14.702417+0000","last_active":"2024-03-29T08:19:15.106139+0000","last_peered":"2024-03-29T08:19:15.106139+0000","last_clean":"2024-03-29T08:19:15.106139+0000","last_became_active":"2024-03-29T08:19:14.701927+0000","last_became_peered":"2024-03-29T08:19:14.701927+0000","last_unstale":"2024-03-29T08:19:15.106139+0000","last_undegraded":"2024-03-29T08:19:15.106139+0000","last_fullsized":"2024-03-29T08:19:15.106139+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:30:19.536345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108518+0000","last_change":"2024-03-29T08:19:14.279062+0000","last_active":"2024-03-29T08:19:15.108518+0000","last_peered":"2024-03-29T08:19:15.108518+0000","last_clean":"2024-03-29T08:19:15.108518+0000","last_became_active":"2024-03-29T08:19:14.278726+0000","last_became_peered":"2024-03-29T08:19:14.278726+0000","last_unstale":"2024-03-29T08:19:15.108518+0000","last_undegraded":"2024-03-29T08:19:15.108518+0000","last_fullsized":"2024-03-29T08:19:15.108518+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:35.048072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108506+0000","last_change":"2024-03-29T08:19:14.679888+0000","last_active":"2024-03-29T08:19:15.108506+0000","last_peered":"2024-03-29T08:19:15.108506+0000","last_clean":"2024-03-29T08:19:15.108506+0000","last_became_active":"2024-03-29T08:19:14.679674+0000","last_became_peered":"2024-03-29T08:19:14.679674+0000","last_unstale":"2024-03-29T08:19:15.108506+0000","last_undegraded":"2024-03-29T08:19:15.108506+0000","last_fullsized":"2024-03-29T08:19:15.108506+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:55:24.587078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108488+0000","last_change":"2024-03-29T08:19:14.280371+0000","last_active":"2024-03-29T08:19:15.108488+0000","last_peered":"2024-03-29T08:19:15.108488+0000","last_clean":"2024-03-29T08:19:15.108488+0000","last_became_active":"2024-03-29T08:19:14.280045+0000","last_became_peered":"2024-03-29T08:19:14.280045+0000","last_unstale":"2024-03-29T08:19:15.108488+0000","last_undegraded":"2024-03-29T08:19:15.108488+0000","last_fullsized":"2024-03-29T08:19:15.108488+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:20.730956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.106097+0000","last_change":"2024-03-29T08:19:14.677276+0000","last_active":"2024-03-29T08:19:15.106097+0000","last_peered":"2024-03-29T08:19:15.106097+0000","last_clean":"2024-03-29T08:19:15.106097+0000","last_became_active":"2024-03-29T08:19:14.677049+0000","last_became_peered":"2024-03-29T08:19:14.677049+0000","last_unstale":"2024-03-29T08:19:15.106097+0000","last_undegraded":"2024-03-29T08:19:15.106097+0000","last_fullsized":"2024-03-29T08:19:15.106097+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:27.668682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109442+0000","last_change":"2024-03-29T08:19:14.285270+0000","last_active":"2024-03-29T08:19:15.109442+0000","last_peered":"2024-03-29T08:19:15.109442+0000","last_clean":"2024-03-29T08:19:15.109442+0000","last_became_active":"2024-03-29T08:19:14.283882+0000","last_became_peered":"2024-03-29T08:19:14.283882+0000","last_unstale":"2024-03-29T08:19:15.109442+0000","last_undegraded":"2024-03-29T08:19:15.109442+0000","last_fullsized":"2024-03-29T08:19:15.109442+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:09:33.409328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109603+0000","last_change":"2024-03-29T08:19:14.696486+0000","last_active":"2024-03-29T08:19:15.109603+0000","last_peered":"2024-03-29T08:19:15.109603+0000","last_clean":"2024-03-29T08:19:15.109603+0000","last_became_active":"2024-03-29T08:19:14.696236+0000","last_became_peered":"2024-03-29T08:19:14.696236+0000","last_unstale":"2024-03-29T08:19:15.109603+0000","last_undegraded":"2024-03-29T08:19:15.109603+0000","last_fullsized":"2024-03-29T08:19:15.109603+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:08:58.521804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108447+0000","last_change":"2024-03-29T08:19:14.275451+0000","last_active":"2024-03-29T08:19:15.108447+0000","last_peered":"2024-03-29T08:19:15.108447+0000","last_clean":"2024-03-29T08:19:15.108447+0000","last_became_active":"2024-03-29T08:19:14.275148+0000","last_became_peered":"2024-03-29T08:19:14.275148+0000","last_unstale":"2024-03-29T08:19:15.108447+0000","last_undegraded":"2024-03-29T08:19:15.108447+0000","last_fullsized":"2024-03-29T08:19:15.108447+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:57:14.581364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.113046+0000","last_change":"2024-03-29T08:19:14.690396+0000","last_active":"2024-03-29T08:19:15.113046+0000","last_peered":"2024-03-29T08:19:15.113046+0000","last_clean":"2024-03-29T08:19:15.113046+0000","last_became_active":"2024-03-29T08:19:14.690106+0000","last_became_peered":"2024-03-29T08:19:14.690106+0000","last_unstale":"2024-03-29T08:19:15.113046+0000","last_undegraded":"2024-03-29T08:19:15.113046+0000","last_fullsized":"2024-03-29T08:19:15.113046+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:54:09.222693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.672936+0000","last_change":"2024-03-29T08:19:14.286636+0000","last_active":"2024-03-29T08:19:14.672936+0000","last_peered":"2024-03-29T08:19:14.672936+0000","last_clean":"2024-03-29T08:19:14.672936+0000","last_became_active":"2024-03-29T08:19:14.285309+0000","last_became_peered":"2024-03-29T08:19:14.285309+0000","last_unstale":"2024-03-29T08:19:14.672936+0000","last_undegraded":"2024-03-29T08:19:14.672936+0000","last_fullsized":"2024-03-29T08:19:14.672936+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:58:40.235616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109787+0000","last_change":"2024-03-29T08:19:14.694063+0000","last_active":"2024-03-29T08:19:15.109787+0000","last_peered":"2024-03-29T08:19:15.109787+0000","last_clean":"2024-03-29T08:19:15.109787+0000","last_became_active":"2024-03-29T08:19:14.693817+0000","last_became_peered":"2024-03-29T08:19:14.693817+0000","last_unstale":"2024-03-29T08:19:15.109787+0000","last_undegraded":"2024-03-29T08:19:15.109787+0000","last_fullsized":"2024-03-29T08:19:15.109787+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:30:57.036017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.671221+0000","last_change":"2024-03-29T08:19:14.283531+0000","last_active":"2024-03-29T08:19:14.671221+0000","last_peered":"2024-03-29T08:19:14.671221+0000","last_clean":"2024-03-29T08:19:14.671221+0000","last_became_active":"2024-03-29T08:19:14.283128+0000","last_became_peered":"2024-03-29T08:19:14.283128+0000","last_unstale":"2024-03-29T08:19:14.671221+0000","last_undegraded":"2024-03-29T08:19:14.671221+0000","last_fullsized":"2024-03-29T08:19:14.671221+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:36:01.837144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109694+0000","last_change":"2024-03-29T08:19:14.698954+0000","last_active":"2024-03-29T08:19:15.109694+0000","last_peered":"2024-03-29T08:19:15.109694+0000","last_clean":"2024-03-29T08:19:15.109694+0000","last_became_active":"2024-03-29T08:19:14.698317+0000","last_became_peered":"2024-03-29T08:19:14.698317+0000","last_unstale":"2024-03-29T08:19:15.109694+0000","last_undegraded":"2024-03-29T08:19:15.109694+0000","last_fullsized":"2024-03-29T08:19:15.109694+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:16:39.883087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.113029+0000","last_change":"2024-03-29T08:19:14.285068+0000","last_active":"2024-03-29T08:19:15.113029+0000","last_peered":"2024-03-29T08:19:15.113029+0000","last_clean":"2024-03-29T08:19:15.113029+0000","last_became_active":"2024-03-29T08:19:14.284830+0000","last_became_peered":"2024-03-29T08:19:14.284830+0000","last_unstale":"2024-03-29T08:19:15.113029+0000","last_undegraded":"2024-03-29T08:19:15.113029+0000","last_fullsized":"2024-03-29T08:19:15.113029+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:16:38.877179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.113001+0000","last_change":"2024-03-29T08:19:14.704710+0000","last_active":"2024-03-29T08:19:15.113001+0000","last_peered":"2024-03-29T08:19:15.113001+0000","last_clean":"2024-03-29T08:19:15.113001+0000","last_became_active":"2024-03-29T08:19:14.704396+0000","last_became_peered":"2024-03-29T08:19:14.704396+0000","last_unstale":"2024-03-29T08:19:15.113001+0000","last_undegraded":"2024-03-29T08:19:15.113001+0000","last_fullsized":"2024-03-29T08:19:15.113001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:24.176397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109519+0000","last_change":"2024-03-29T08:19:14.280954+0000","last_active":"2024-03-29T08:19:15.109519+0000","last_peered":"2024-03-29T08:19:15.109519+0000","last_clean":"2024-03-29T08:19:15.109519+0000","last_became_active":"2024-03-29T08:19:14.280392+0000","last_became_peered":"2024-03-29T08:19:14.280392+0000","last_unstale":"2024-03-29T08:19:15.109519+0000","last_undegraded":"2024-03-29T08:19:15.109519+0000","last_fullsized":"2024-03-29T08:19:15.109519+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:26.662774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109389+0000","last_change":"2024-03-29T08:19:14.685902+0000","last_active":"2024-03-29T08:19:15.109389+0000","last_peered":"2024-03-29T08:19:15.109389+0000","last_clean":"2024-03-29T08:19:15.109389+0000","last_became_active":"2024-03-29T08:19:14.685705+0000","last_became_peered":"2024-03-29T08:19:14.685705+0000","last_unstale":"2024-03-29T08:19:15.109389+0000","last_undegraded":"2024-03-29T08:19:15.109389+0000","last_fullsized":"2024-03-29T08:19:15.109389+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:08.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108529+0000","last_change":"2024-03-29T08:19:14.272012+0000","last_active":"2024-03-29T08:19:15.108529+0000","last_peered":"2024-03-29T08:19:15.108529+0000","last_clean":"2024-03-29T08:19:15.108529+0000","last_became_active":"2024-03-29T08:19:14.271736+0000","last_became_peered":"2024-03-29T08:19:14.271736+0000","last_unstale":"2024-03-29T08:19:15.108529+0000","last_undegraded":"2024-03-29T08:19:15.108529+0000","last_fullsized":"2024-03-29T08:19:15.108529+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:43:33.064793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109666+0000","last_change":"2024-03-29T08:19:14.690513+0000","last_active":"2024-03-29T08:19:15.109666+0000","last_peered":"2024-03-29T08:19:15.109666+0000","last_clean":"2024-03-29T08:19:15.109666+0000","last_became_active":"2024-03-29T08:19:14.690235+0000","last_became_peered":"2024-03-29T08:19:14.690235+0000","last_unstale":"2024-03-29T08:19:15.109666+0000","last_undegraded":"2024-03-29T08:19:15.109666+0000","last_fullsized":"2024-03-29T08:19:15.109666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:01:53.702260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109651+0000","last_change":"2024-03-29T08:19:14.274775+0000","last_active":"2024-03-29T08:19:15.109651+0000","last_peered":"2024-03-29T08:19:15.109651+0000","last_clean":"2024-03-29T08:19:15.109651+0000","last_became_active":"2024-03-29T08:19:14.274502+0000","last_became_peered":"2024-03-29T08:19:14.274502+0000","last_unstale":"2024-03-29T08:19:15.109651+0000","last_undegraded":"2024-03-29T08:19:15.109651+0000","last_fullsized":"2024-03-29T08:19:15.109651+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:20:37.829787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108437+0000","last_change":"2024-03-29T08:19:14.675914+0000","last_active":"2024-03-29T08:19:15.108437+0000","last_peered":"2024-03-29T08:19:15.108437+0000","last_clean":"2024-03-29T08:19:15.108437+0000","last_became_active":"2024-03-29T08:19:14.675564+0000","last_became_peered":"2024-03-29T08:19:14.675564+0000","last_unstale":"2024-03-29T08:19:15.108437+0000","last_undegraded":"2024-03-29T08:19:15.108437+0000","last_fullsized":"2024-03-29T08:19:15.108437+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:36:09.793618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109613+0000","last_change":"2024-03-29T08:19:14.281362+0000","last_active":"2024-03-29T08:19:15.109613+0000","last_peered":"2024-03-29T08:19:15.109613+0000","last_clean":"2024-03-29T08:19:15.109613+0000","last_became_active":"2024-03-29T08:19:14.281111+0000","last_became_peered":"2024-03-29T08:19:14.281111+0000","last_unstale":"2024-03-29T08:19:15.109613+0000","last_undegraded":"2024-03-29T08:19:15.109613+0000","last_fullsized":"2024-03-29T08:19:15.109613+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:26.870833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.112985+0000","last_change":"2024-03-29T08:19:14.698963+0000","last_active":"2024-03-29T08:19:15.112985+0000","last_peered":"2024-03-29T08:19:15.112985+0000","last_clean":"2024-03-29T08:19:15.112985+0000","last_became_active":"2024-03-29T08:19:14.698502+0000","last_became_peered":"2024-03-29T08:19:14.698502+0000","last_unstale":"2024-03-29T08:19:15.112985+0000","last_undegraded":"2024-03-29T08:19:15.112985+0000","last_fullsized":"2024-03-29T08:19:15.112985+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:21:27.491192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109419+0000","last_change":"2024-03-29T08:19:14.284917+0000","last_active":"2024-03-29T08:19:15.109419+0000","last_peered":"2024-03-29T08:19:15.109419+0000","last_clean":"2024-03-29T08:19:15.109419+0000","last_became_active":"2024-03-29T08:19:14.284667+0000","last_became_peered":"2024-03-29T08:19:14.284667+0000","last_unstale":"2024-03-29T08:19:15.109419+0000","last_undegraded":"2024-03-29T08:19:15.109419+0000","last_fullsized":"2024-03-29T08:19:15.109419+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:23:42.739141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.113054+0000","last_change":"2024-03-29T08:19:14.703652+0000","last_active":"2024-03-29T08:19:15.113054+0000","last_peered":"2024-03-29T08:19:15.113054+0000","last_clean":"2024-03-29T08:19:15.113054+0000","last_became_active":"2024-03-29T08:19:14.703386+0000","last_became_peered":"2024-03-29T08:19:14.703386+0000","last_unstale":"2024-03-29T08:19:15.113054+0000","last_undegraded":"2024-03-29T08:19:15.113054+0000","last_fullsized":"2024-03-29T08:19:15.113054+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:55:15.376664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109599+0000","last_change":"2024-03-29T08:19:14.287908+0000","last_active":"2024-03-29T08:19:15.109599+0000","last_peered":"2024-03-29T08:19:15.109599+0000","last_clean":"2024-03-29T08:19:15.109599+0000","last_became_active":"2024-03-29T08:19:14.287562+0000","last_became_peered":"2024-03-29T08:19:14.287562+0000","last_unstale":"2024-03-29T08:19:15.109599+0000","last_undegraded":"2024-03-29T08:19:15.109599+0000","last_fullsized":"2024-03-29T08:19:15.109599+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:02:28.529095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.106177+0000","last_change":"2024-03-29T08:19:14.682493+0000","last_active":"2024-03-29T08:19:15.106177+0000","last_peered":"2024-03-29T08:19:15.106177+0000","last_clean":"2024-03-29T08:19:15.106177+0000","last_became_active":"2024-03-29T08:19:14.678342+0000","last_became_peered":"2024-03-29T08:19:14.678342+0000","last_unstale":"2024-03-29T08:19:15.106177+0000","last_undegraded":"2024-03-29T08:19:15.106177+0000","last_fullsized":"2024-03-29T08:19:15.106177+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:36.053980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108588+0000","last_change":"2024-03-29T08:19:14.272763+0000","last_active":"2024-03-29T08:19:15.108588+0000","last_peered":"2024-03-29T08:19:15.108588+0000","last_clean":"2024-03-29T08:19:15.108588+0000","last_became_active":"2024-03-29T08:19:14.272450+0000","last_became_peered":"2024-03-29T08:19:14.272450+0000","last_unstale":"2024-03-29T08:19:15.108588+0000","last_undegraded":"2024-03-29T08:19:15.108588+0000","last_fullsized":"2024-03-29T08:19:15.108588+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:19.581981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-29T08:19:16.660513+0000","last_change":"2024-03-29T08:19:14.676556+0000","last_active":"2024-03-29T08:19:16.660513+0000","last_peered":"2024-03-29T08:19:16.660513+0000","last_clean":"2024-03-29T08:19:16.660513+0000","last_became_active":"2024-03-29T08:19:14.676341+0000","last_became_peered":"2024-03-29T08:19:14.676341+0000","last_unstale":"2024-03-29T08:19:16.660513+0000","last_undegraded":"2024-03-29T08:19:16.660513+0000","last_fullsized":"2024-03-29T08:19:16.660513+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:05:38.988839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.113020+0000","last_change":"2024-03-29T08:19:14.287716+0000","last_active":"2024-03-29T08:19:15.113020+0000","last_peered":"2024-03-29T08:19:15.113020+0000","last_clean":"2024-03-29T08:19:15.113020+0000","last_became_active":"2024-03-29T08:19:14.287437+0000","last_became_peered":"2024-03-29T08:19:14.287437+0000","last_unstale":"2024-03-29T08:19:15.113020+0000","last_undegraded":"2024-03-29T08:19:15.113020+0000","last_fullsized":"2024-03-29T08:19:15.113020+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:08:57.515896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109511+0000","last_change":"2024-03-29T08:19:14.689798+0000","last_active":"2024-03-29T08:19:15.109511+0000","last_peered":"2024-03-29T08:19:15.109511+0000","last_clean":"2024-03-29T08:19:15.109511+0000","last_became_active":"2024-03-29T08:19:14.689479+0000","last_became_peered":"2024-03-29T08:19:14.689479+0000","last_unstale":"2024-03-29T08:19:15.109511+0000","last_undegraded":"2024-03-29T08:19:15.109511+0000","last_fullsized":"2024-03-29T08:19:15.109511+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:20:03.888899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108483+0000","last_change":"2024-03-29T08:19:14.281935+0000","last_active":"2024-03-29T08:19:15.108483+0000","last_peered":"2024-03-29T08:19:15.108483+0000","last_clean":"2024-03-29T08:19:15.108483+0000","last_became_active":"2024-03-29T08:19:14.281670+0000","last_became_peered":"2024-03-29T08:19:14.281670+0000","last_unstale":"2024-03-29T08:19:15.108483+0000","last_undegraded":"2024-03-29T08:19:15.108483+0000","last_fullsized":"2024-03-29T08:19:15.108483+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:49:03.171063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109404+0000","last_change":"2024-03-29T08:19:14.689677+0000","last_active":"2024-03-29T08:19:15.109404+0000","last_peered":"2024-03-29T08:19:15.109404+0000","last_clean":"2024-03-29T08:19:15.109404+0000","last_became_active":"2024-03-29T08:19:14.689343+0000","last_became_peered":"2024-03-29T08:19:14.689343+0000","last_unstale":"2024-03-29T08:19:15.109404+0000","last_undegraded":"2024-03-29T08:19:15.109404+0000","last_fullsized":"2024-03-29T08:19:15.109404+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:55.247391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.702821+0000","last_change":"2024-03-29T08:19:14.279362+0000","last_active":"2024-03-29T08:19:14.702821+0000","last_peered":"2024-03-29T08:19:14.702821+0000","last_clean":"2024-03-29T08:19:14.702821+0000","last_became_active":"2024-03-29T08:19:14.278866+0000","last_became_peered":"2024-03-29T08:19:14.278866+0000","last_unstale":"2024-03-29T08:19:14.702821+0000","last_undegraded":"2024-03-29T08:19:14.702821+0000","last_fullsized":"2024-03-29T08:19:14.702821+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:27:45.403801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-29T08:19:16.660698+0000","last_change":"2024-03-29T08:19:14.679765+0000","last_active":"2024-03-29T08:19:16.660698+0000","last_peered":"2024-03-29T08:19:16.660698+0000","last_clean":"2024-03-29T08:19:16.660698+0000","last_became_active":"2024-03-29T08:19:14.679529+0000","last_became_peered":"2024-03-29T08:19:14.679529+0000","last_unstale":"2024-03-29T08:19:16.660698+0000","last_undegraded":"2024-03-29T08:19:16.660698+0000","last_fullsized":"2024-03-29T08:19:16.660698+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:18.770880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108498+0000","last_change":"2024-03-29T08:19:14.272023+0000","last_active":"2024-03-29T08:19:15.108498+0000","last_peered":"2024-03-29T08:19:15.108498+0000","last_clean":"2024-03-29T08:19:15.108498+0000","last_became_active":"2024-03-29T08:19:14.271722+0000","last_became_peered":"2024-03-29T08:19:14.271722+0000","last_unstale":"2024-03-29T08:19:15.108498+0000","last_undegraded":"2024-03-29T08:19:15.108498+0000","last_fullsized":"2024-03-29T08:19:15.108498+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:51:26.662774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109352+0000","last_change":"2024-03-29T08:19:14.694537+0000","last_active":"2024-03-29T08:19:15.109352+0000","last_peered":"2024-03-29T08:19:15.109352+0000","last_clean":"2024-03-29T08:19:15.109352+0000","last_became_active":"2024-03-29T08:19:14.694296+0000","last_became_peered":"2024-03-29T08:19:14.694296+0000","last_unstale":"2024-03-29T08:19:15.109352+0000","last_undegraded":"2024-03-29T08:19:15.109352+0000","last_fullsized":"2024-03-29T08:19:15.109352+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:37:16.877010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.106131+0000","last_change":"2024-03-29T08:19:14.285590+0000","last_active":"2024-03-29T08:19:15.106131+0000","last_peered":"2024-03-29T08:19:15.106131+0000","last_clean":"2024-03-29T08:19:15.106131+0000","last_became_active":"2024-03-29T08:19:14.285211+0000","last_became_peered":"2024-03-29T08:19:14.285211+0000","last_unstale":"2024-03-29T08:19:15.106131+0000","last_undegraded":"2024-03-29T08:19:15.106131+0000","last_fullsized":"2024-03-29T08:19:15.106131+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:52:20.153715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.106122+0000","last_change":"2024-03-29T08:19:14.679560+0000","last_active":"2024-03-29T08:19:15.106122+0000","last_peered":"2024-03-29T08:19:15.106122+0000","last_clean":"2024-03-29T08:19:15.106122+0000","last_became_active":"2024-03-29T08:19:14.677544+0000","last_became_peered":"2024-03-29T08:19:14.677544+0000","last_unstale":"2024-03-29T08:19:15.106122+0000","last_undegraded":"2024-03-29T08:19:15.106122+0000","last_fullsized":"2024-03-29T08:19:15.106122+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:34.410446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.112974+0000","last_change":"2024-03-29T08:19:14.281813+0000","last_active":"2024-03-29T08:19:15.112974+0000","last_peered":"2024-03-29T08:19:15.112974+0000","last_clean":"2024-03-29T08:19:15.112974+0000","last_became_active":"2024-03-29T08:19:14.281356+0000","last_became_peered":"2024-03-29T08:19:14.281356+0000","last_unstale":"2024-03-29T08:19:15.112974+0000","last_undegraded":"2024-03-29T08:19:15.112974+0000","last_fullsized":"2024-03-29T08:19:15.112974+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:04:28.023637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109581+0000","last_change":"2024-03-29T08:19:14.691637+0000","last_active":"2024-03-29T08:19:15.109581+0000","last_peered":"2024-03-29T08:19:15.109581+0000","last_clean":"2024-03-29T08:19:15.109581+0000","last_became_active":"2024-03-29T08:19:14.691391+0000","last_became_peered":"2024-03-29T08:19:14.691391+0000","last_unstale":"2024-03-29T08:19:15.109581+0000","last_undegraded":"2024-03-29T08:19:15.109581+0000","last_fullsized":"2024-03-29T08:19:15.109581+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:11:13.363737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109571+0000","last_change":"2024-03-29T08:19:14.286350+0000","last_active":"2024-03-29T08:19:15.109571+0000","last_peered":"2024-03-29T08:19:15.109571+0000","last_clean":"2024-03-29T08:19:15.109571+0000","last_became_active":"2024-03-29T08:19:14.286055+0000","last_became_peered":"2024-03-29T08:19:14.286055+0000","last_unstale":"2024-03-29T08:19:15.109571+0000","last_undegraded":"2024-03-29T08:19:15.109571+0000","last_fullsized":"2024-03-29T08:19:15.109571+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:09:35.048072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.113070+0000","last_change":"2024-03-29T08:19:14.694685+0000","last_active":"2024-03-29T08:19:15.113070+0000","last_peered":"2024-03-29T08:19:15.113070+0000","last_clean":"2024-03-29T08:19:15.113070+0000","last_became_active":"2024-03-29T08:19:14.693892+0000","last_became_peered":"2024-03-29T08:19:14.693892+0000","last_unstale":"2024-03-29T08:19:15.113070+0000","last_undegraded":"2024-03-29T08:19:15.113070+0000","last_fullsized":"2024-03-29T08:19:15.113070+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:02:39.788723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-03-29T08:19:16.660802+0000","last_change":"2024-03-29T08:19:14.283370+0000","last_active":"2024-03-29T08:19:16.660802+0000","last_peered":"2024-03-29T08:19:16.660802+0000","last_clean":"2024-03-29T08:19:16.660802+0000","last_became_active":"2024-03-29T08:19:14.282773+0000","last_became_peered":"2024-03-29T08:19:14.282773+0000","last_unstale":"2024-03-29T08:19:16.660802+0000","last_undegraded":"2024-03-29T08:19:16.660802+0000","last_fullsized":"2024-03-29T08:19:16.660802+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:59:27.620393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109530+0000","last_change":"2024-03-29T08:19:14.701490+0000","last_active":"2024-03-29T08:19:15.109530+0000","last_peered":"2024-03-29T08:19:15.109530+0000","last_clean":"2024-03-29T08:19:15.109530+0000","last_became_active":"2024-03-29T08:19:14.701146+0000","last_became_peered":"2024-03-29T08:19:14.701146+0000","last_unstale":"2024-03-29T08:19:15.109530+0000","last_undegraded":"2024-03-29T08:19:15.109530+0000","last_fullsized":"2024-03-29T08:19:15.109530+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:32:52.080327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109538+0000","last_change":"2024-03-29T08:19:14.279625+0000","last_active":"2024-03-29T08:19:15.109538+0000","last_peered":"2024-03-29T08:19:15.109538+0000","last_clean":"2024-03-29T08:19:15.109538+0000","last_became_active":"2024-03-29T08:19:14.279364+0000","last_became_peered":"2024-03-29T08:19:14.279364+0000","last_unstale":"2024-03-29T08:19:15.109538+0000","last_undegraded":"2024-03-29T08:19:15.109538+0000","last_fullsized":"2024-03-29T08:19:15.109538+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:20:02.882991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108414+0000","last_change":"2024-03-29T08:19:14.679991+0000","last_active":"2024-03-29T08:19:15.108414+0000","last_peered":"2024-03-29T08:19:15.108414+0000","last_clean":"2024-03-29T08:19:15.108414+0000","last_became_active":"2024-03-29T08:19:14.679701+0000","last_became_peered":"2024-03-29T08:19:14.679701+0000","last_unstale":"2024-03-29T08:19:15.108414+0000","last_undegraded":"2024-03-29T08:19:15.108414+0000","last_fullsized":"2024-03-29T08:19:15.108414+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:08.098819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109464+0000","last_change":"2024-03-29T08:19:14.284652+0000","last_active":"2024-03-29T08:19:15.109464+0000","last_peered":"2024-03-29T08:19:15.109464+0000","last_clean":"2024-03-29T08:19:15.109464+0000","last_became_active":"2024-03-29T08:19:14.284376+0000","last_became_peered":"2024-03-29T08:19:14.284376+0000","last_unstale":"2024-03-29T08:19:15.109464+0000","last_undegraded":"2024-03-29T08:19:15.109464+0000","last_fullsized":"2024-03-29T08:19:15.109464+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:20.730956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109676+0000","last_change":"2024-03-29T08:19:14.693198+0000","last_active":"2024-03-29T08:19:15.109676+0000","last_peered":"2024-03-29T08:19:15.109676+0000","last_clean":"2024-03-29T08:19:15.109676+0000","last_became_active":"2024-03-29T08:19:14.692849+0000","last_became_peered":"2024-03-29T08:19:14.692849+0000","last_unstale":"2024-03-29T08:19:15.109676+0000","last_undegraded":"2024-03-29T08:19:15.109676+0000","last_fullsized":"2024-03-29T08:19:15.109676+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:36:46.218186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109431+0000","last_change":"2024-03-29T08:19:14.284484+0000","last_active":"2024-03-29T08:19:15.109431+0000","last_peered":"2024-03-29T08:19:15.109431+0000","last_clean":"2024-03-29T08:19:15.109431+0000","last_became_active":"2024-03-29T08:19:14.284145+0000","last_became_peered":"2024-03-29T08:19:14.284145+0000","last_unstale":"2024-03-29T08:19:15.109431+0000","last_undegraded":"2024-03-29T08:19:15.109431+0000","last_fullsized":"2024-03-29T08:19:15.109431+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:12:33.404538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109580+0000","last_change":"2024-03-29T08:19:14.695365+0000","last_active":"2024-03-29T08:19:15.109580+0000","last_peered":"2024-03-29T08:19:15.109580+0000","last_clean":"2024-03-29T08:19:15.109580+0000","last_became_active":"2024-03-29T08:19:14.695111+0000","last_became_peered":"2024-03-29T08:19:14.695111+0000","last_unstale":"2024-03-29T08:19:15.109580+0000","last_undegraded":"2024-03-29T08:19:15.109580+0000","last_fullsized":"2024-03-29T08:19:15.109580+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:15.077172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108471+0000","last_change":"2024-03-29T08:19:14.270630+0000","last_active":"2024-03-29T08:19:15.108471+0000","last_peered":"2024-03-29T08:19:15.108471+0000","last_clean":"2024-03-29T08:19:15.108471+0000","last_became_active":"2024-03-29T08:19:14.270379+0000","last_became_peered":"2024-03-29T08:19:14.270379+0000","last_unstale":"2024-03-29T08:19:15.108471+0000","last_undegraded":"2024-03-29T08:19:15.108471+0000","last_fullsized":"2024-03-29T08:19:15.108471+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:30:18.530436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.112993+0000","last_change":"2024-03-29T08:19:14.698728+0000","last_active":"2024-03-29T08:19:15.112993+0000","last_peered":"2024-03-29T08:19:15.112993+0000","last_clean":"2024-03-29T08:19:15.112993+0000","last_became_active":"2024-03-29T08:19:14.698138+0000","last_became_peered":"2024-03-29T08:19:14.698138+0000","last_unstale":"2024-03-29T08:19:15.112993+0000","last_undegraded":"2024-03-29T08:19:15.112993+0000","last_fullsized":"2024-03-29T08:19:15.112993+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:07:30.640723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.109594+0000","last_change":"2024-03-29T08:19:14.281517+0000","last_active":"2024-03-29T08:19:15.109594+0000","last_peered":"2024-03-29T08:19:15.109594+0000","last_clean":"2024-03-29T08:19:15.109594+0000","last_became_active":"2024-03-29T08:19:14.281182+0000","last_became_peered":"2024-03-29T08:19:14.281182+0000","last_unstale":"2024-03-29T08:19:15.109594+0000","last_undegraded":"2024-03-29T08:19:15.109594+0000","last_fullsized":"2024-03-29T08:19:15.109594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:21:54.574176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.108467+0000","last_change":"2024-03-29T08:19:14.685120+0000","last_active":"2024-03-29T08:19:15.108467+0000","last_peered":"2024-03-29T08:19:15.108467+0000","last_clean":"2024-03-29T08:19:15.108467+0000","last_became_active":"2024-03-29T08:19:14.682875+0000","last_became_peered":"2024-03-29T08:19:14.682875+0000","last_unstale":"2024-03-29T08:19:15.108467+0000","last_undegraded":"2024-03-29T08:19:15.108467+0000","last_fullsized":"2024-03-29T08:19:15.108467+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:14.252791+0000","last_clean_scrub_stamp":"2024-03-29T08:19:14.252791+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:55:24.587078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:15.113038+0000","last_change":"2024-03-29T08:19:14.283788+0000","last_active":"2024-03-29T08:19:15.113038+0000","last_peered":"2024-03-29T08:19:15.113038+0000","last_clean":"2024-03-29T08:19:15.113038+0000","last_became_active":"2024-03-29T08:19:14.283586+0000","last_became_peered":"2024-03-29T08:19:14.283586+0000","last_unstale":"2024-03-29T08:19:15.113038+0000","last_undegraded":"2024-03-29T08:19:15.113038+0000","last_fullsized":"2024-03-29T08:19:15.113038+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:19:13.246882+0000","last_clean_scrub_stamp":"2024-03-29T08:19:13.246882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:52:19.059268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":174,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-03-29T08:19:14.673750+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:19:14.673750+0000","last_peered":"2024-03-29T08:19:14.673750+0000","last_clean":"2024-03-29T08:19:14.673750+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:19:14.673750+0000","last_undegraded":"2024-03-29T08:19:14.673750+0000","last_fullsized":"2024-03-29T08:19:14.673750+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509459,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":405504,"data_stored":156765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.64400000000000002}]},{"osd":1,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.58399999999999996}]},{"osd":2,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.72499999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.66800000000000004}]},{"osd":4,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.628}]},{"osd":5,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.77600000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.745},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.71399999999999997}]}]},{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":158307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.52900000000000003}]},{"osd":1,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.55800000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.51100000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.0539999999999998,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.371}]},{"osd":5,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58499999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.56499999999999995}]}]},{"osd":4,"up_from":15,"seq":64424509458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":417792,"data_stored":158329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.1709999999999998,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":2.524,"5min":2.524,"15min":2.524},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.2679999999999998,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.69599999999999995}]},{"osd":2,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.363,"5min":2.363,"15min":2.363},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.83199999999999996}]},{"osd":3,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":2.3029999999999999,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.68000000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.82099999999999995}]},{"osd":6,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.746},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.80900000000000005}]}]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":417792,"data_stored":157519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.41399999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.434}]},{"osd":2,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.8580000000000001,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.53700000000000003}]},{"osd":4,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.57999999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.68400000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.66400000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:19:01 2024","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":2.044,"5min":2.044,"15min":2.044},"last":0.54800000000000004}]}]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":405504,"data_stored":156777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.36399999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.33700000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.42599999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.77500000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.871,"5min":0.871,"15min":0.871},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.90300000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.78100000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:18:59 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.91700000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509460,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":999424,"data_stored":747181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":4.2539999999999996,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":4.1909999999999998,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.42299999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.51700000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.627}]},{"osd":5,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.71099999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:19:02 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.76800000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509459,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31512,"kb_used_data":396,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340328,"statfs":{"total":96636764160,"available":96604495872,"internally_reserved":0,"allocated":405504,"data_stored":156833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":3.9359999999999999,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":8.0419999999999998,"5min":8.0419999999999998,"15min":8.0419999999999998},"last":0.35999999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":4.907,"5min":4.907,"15min":4.907},"last":0.43099999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":7.9569999999999999,"5min":7.9569999999999999,"15min":7.9569999999999999},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.442}]},{"osd":4,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":4.4720000000000004,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":4.3920000000000003,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.61599999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.3200000000000003,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6759999999999999,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":4.4100000000000001,"5min":4.4100000000000001,"15min":4.4100000000000001},"last":0.65500000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.0589999999999999,"15min":1.0589999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":4.5209999999999999,"5min":4.5209999999999999,"15min":4.5209999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":4.4420000000000002,"5min":4.4420000000000002,"15min":4.4420000000000002},"last":0.66500000000000004}]}]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343844,"statfs":{"total":96636764160,"available":96608096256,"internally_reserved":0,"allocated":999424,"data_stored":747693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.503}]},{"osd":1,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":2.5129999999999999,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63900000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.60899999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.45800000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.65700000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:19:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.71899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:19:23.712 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T08:19:23.713 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-29T08:19:23.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.896+0000 7f4947600640 1 Processor -- start 2024-03-29T08:19:23.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.896+0000 7f4947600640 1 -- start start 2024-03-29T08:19:23.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.900+0000 7f4947600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4940160db0 0x7f49400cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:23.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.900+0000 7f4947600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49400cfb20 con 0x7f494015c950 2024-03-29T08:19:23.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.900+0000 7f4947600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49400cef10 con 0x7f49400cea30 2024-03-29T08:19:23.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.900+0000 7f4947600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49400cf1f0 con 0x7f4940160db0 2024-03-29T08:19:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.900+0000 7f4945600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4940160db0 0x7f49400cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.900+0000 7f4945600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4940160db0 0x7f49400cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34882/0 (socket says 172.21.15.46:34882) 2024-03-29T08:19:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.900+0000 7f4945600640 1 -- 172.21.15.46:0/160531333 learned_addr learned my addr 172.21.15.46:0/160531333 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.900+0000 7f4945600640 1 -- 172.21.15.46:0/160531333 >> v1:172.21.15.73:6789/0 conn(0x7f49400cea30 legacy=0x7f49401606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.900+0000 7f4945600640 1 -- 172.21.15.46:0/160531333 >> v1:172.21.15.46:6789/0 conn(0x7f494015c950 legacy=0x7f494006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.900+0000 7f493fe00640 1 -- 172.21.15.46:0/160531333 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045403994 0 0) 0x7f49400cfb20 con 0x7f494015c950 2024-03-29T08:19:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.900+0000 7f4945600640 1 -- 172.21.15.46:0/160531333 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49400cf4d0 con 0x7f4940160db0 2024-03-29T08:19:23.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.900+0000 7f4945600640 1 --2- 172.21.15.46:0/160531333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4940160db0 0x7f49400cad10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4938009e50 tx=0x7f493801f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7f03d287ad255c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:23.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.900+0000 7f493fe00640 1 -- 172.21.15.46:0/160531333 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49380af400 con 0x7f4940160db0 2024-03-29T08:19:23.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.900+0000 7f493fe00640 1 -- 172.21.15.46:0/160531333 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49380ab050 con 0x7f4940160db0 2024-03-29T08:19:23.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.900+0000 7f493fe00640 1 -- 172.21.15.46:0/160531333 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49380af780 con 0x7f4940160db0 2024-03-29T08:19:23.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4947600640 1 -- 172.21.15.46:0/160531333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4940160db0 msgr2=0x7f49400cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:23.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4947600640 1 --2- 172.21.15.46:0/160531333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4940160db0 0x7f49400cad10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4938009e50 tx=0x7f493801f210 comp rx=0 tx=0).stop 2024-03-29T08:19:23.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4947600640 1 -- 172.21.15.46:0/160531333 shutdown_connections 2024-03-29T08:19:23.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4947600640 1 --2- 172.21.15.46:0/160531333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4940160db0 0x7f49400cad10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:23.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4947600640 1 -- 172.21.15.46:0/160531333 >> 172.21.15.46:0/160531333 conn(0x7f49400b21b0 msgr2=0x7f49400b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:23.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4947600640 1 -- 172.21.15.46:0/160531333 shutdown_connections 2024-03-29T08:19:23.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4947600640 1 -- 172.21.15.46:0/160531333 wait complete. 2024-03-29T08:19:23.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4947600640 1 Processor -- start 2024-03-29T08:19:23.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4947600640 1 -- start start 2024-03-29T08:19:23.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4947600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49400cea30 0x7f494014ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:23.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4947600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f494015c950 0x7f49400b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:23.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4947600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4940160db0 0x7f49400b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:23.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4947600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49400b65c0 con 0x7f494015c950 2024-03-29T08:19:23.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4947600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494014b0a0 con 0x7f49400cea30 2024-03-29T08:19:23.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4947600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49400b62e0 con 0x7f4940160db0 2024-03-29T08:19:23.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4937e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f494015c950 0x7f49400b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:23.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4937e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f494015c950 0x7f49400b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34888/0 (socket says 172.21.15.46:34888) 2024-03-29T08:19:23.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4937e00640 1 -- 172.21.15.46:0/2228679894 learned_addr learned my addr 172.21.15.46:0/2228679894 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:23.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4944c00640 1 --2- 172.21.15.46:0/2228679894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49400cea30 0x7f494014ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4945600640 1 --2- 172.21.15.46:0/2228679894 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4940160db0 0x7f49400b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4937e00640 1 -- 172.21.15.46:0/2228679894 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4940160db0 msgr2=0x7f49400b5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4937e00640 1 --2- 172.21.15.46:0/2228679894 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4940160db0 0x7f49400b5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4937e00640 1 -- 172.21.15.46:0/2228679894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49400cea30 msgr2=0x7f494014ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4937e00640 1 --2- 172.21.15.46:0/2228679894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49400cea30 0x7f494014ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.904+0000 7f4937e00640 1 -- 172.21.15.46:0/2228679894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4938009b30 con 0x7f494015c950 2024-03-29T08:19:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.908+0000 7f4937e00640 1 --2- 172.21.15.46:0/2228679894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f494015c950 0x7f49400b5520 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f492800c4d0 tx=0x7f492800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.908+0000 7f493d600640 1 -- 172.21.15.46:0/2228679894 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49280052d0 con 0x7f494015c950 2024-03-29T08:19:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.908+0000 7f493d600640 1 -- 172.21.15.46:0/2228679894 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4928005430 con 0x7f494015c950 2024-03-29T08:19:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.908+0000 7f493d600640 1 -- 172.21.15.46:0/2228679894 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4928012830 con 0x7f494015c950 2024-03-29T08:19:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.908+0000 7f4947600640 1 -- 172.21.15.46:0/2228679894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494006c470 con 0x7f494015c950 2024-03-29T08:19:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.908+0000 7f4947600640 1 -- 172.21.15.46:0/2228679894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49400c4640 con 0x7f494015c950 2024-03-29T08:19:23.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.908+0000 7f4947600640 1 -- 172.21.15.46:0/2228679894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49400c41c0 con 0x7f494015c950 2024-03-29T08:19:23.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.908+0000 7f493d600640 1 -- 172.21.15.46:0/2228679894 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4928012990 con 0x7f494015c950 2024-03-29T08:19:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.908+0000 7f493d600640 1 --2- 172.21.15.46:0/2228679894 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f491c066690 0x7f491c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.908+0000 7f493d600640 1 -- 172.21.15.46:0/2228679894 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4928084490 con 0x7f494015c950 2024-03-29T08:19:23.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.912+0000 7f4944c00640 1 --2- 172.21.15.46:0/2228679894 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f491c066690 0x7f491c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:23.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.912+0000 7f4944c00640 1 --2- 172.21.15.46:0/2228679894 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f491c066690 0x7f491c068b50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f49400c4150 tx=0x7f4930009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:23.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:23.912+0000 7f493d600640 1 -- 172.21.15.46:0/2228679894 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4928051920 con 0x7f494015c950 2024-03-29T08:19:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.128+0000 7f4947600640 1 -- 172.21.15.46:0/2228679894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f49400c41c0 con 0x7f494015c950 2024-03-29T08:19:24.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.132+0000 7f493d600640 1 -- 172.21.15.46:0/2228679894 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7f49400c41c0 con 0x7f494015c950 2024-03-29T08:19:24.135 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-03-29T08:19:24.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.136+0000 7f4947600640 1 -- 172.21.15.46:0/2228679894 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f491c066690 msgr2=0x7f491c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:24.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.136+0000 7f4947600640 1 --2- 172.21.15.46:0/2228679894 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f491c066690 0x7f491c068b50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f49400c4150 tx=0x7f4930009040 comp rx=0 tx=0).stop 2024-03-29T08:19:24.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.136+0000 7f4947600640 1 -- 172.21.15.46:0/2228679894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f494015c950 msgr2=0x7f49400b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:24.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.136+0000 7f4947600640 1 --2- 172.21.15.46:0/2228679894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f494015c950 0x7f49400b5520 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f492800c4d0 tx=0x7f492800c990 comp rx=0 tx=0).stop 2024-03-29T08:19:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.136+0000 7f4947600640 1 -- 172.21.15.46:0/2228679894 shutdown_connections 2024-03-29T08:19:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.136+0000 7f4947600640 1 --2- 172.21.15.46:0/2228679894 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f491c066690 0x7f491c068b50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.136+0000 7f4947600640 1 --2- 172.21.15.46:0/2228679894 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4940160db0 0x7f49400b5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.136+0000 7f4947600640 1 --2- 172.21.15.46:0/2228679894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f494015c950 0x7f49400b5520 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.136+0000 7f4947600640 1 --2- 172.21.15.46:0/2228679894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49400cea30 0x7f494014ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.136+0000 7f4947600640 1 -- 172.21.15.46:0/2228679894 >> 172.21.15.46:0/2228679894 conn(0x7f49400b21b0 msgr2=0x7f494015d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.136+0000 7f4947600640 1 -- 172.21.15.46:0/2228679894 shutdown_connections 2024-03-29T08:19:24.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.136+0000 7f4947600640 1 -- 172.21.15.46:0/2228679894 wait complete. 2024-03-29T08:19:24.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:19:24.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.296+0000 7f2845200640 1 Processor -- start 2024-03-29T08:19:24.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.296+0000 7f2845200640 1 -- start start 2024-03-29T08:19:24.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.296+0000 7f2845200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f284015a790 0x7f28400cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:24.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.296+0000 7f2845200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f284015ab70 0x7f2840098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:24.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.296+0000 7f2845200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28400ce660 con 0x7f28400cc430 2024-03-29T08:19:24.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.296+0000 7f2845200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28400cf700 con 0x7f284015ab70 2024-03-29T08:19:24.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.296+0000 7f2845200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28400ce940 con 0x7f284015a790 2024-03-29T08:19:24.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.296+0000 7f2835400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f284015ab70 0x7f2840098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:24.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.296+0000 7f2835400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f284015ab70 0x7f2840098a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34904/0 (socket says 172.21.15.46:34904) 2024-03-29T08:19:24.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.296+0000 7f283e800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f28400cc430 0x7f28400b5860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56134/0 (socket says 172.21.15.46:56134) 2024-03-29T08:19:24.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.296+0000 7f2835400640 1 -- 172.21.15.46:0/2517025226 learned_addr learned my addr 172.21.15.46:0/2517025226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:24.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.296+0000 7f283de00640 1 --2- 172.21.15.46:0/2517025226 >> v2:172.21.15.73:3300/0 conn(0x7f284015a790 0x7f28400cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:24.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.296+0000 7f2835400640 1 -- 172.21.15.46:0/2517025226 >> v2:172.21.15.73:3300/0 conn(0x7f284015a790 msgr2=0x7f28400cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:24.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.296+0000 7f2835400640 1 --2- 172.21.15.46:0/2517025226 >> v2:172.21.15.73:3300/0 conn(0x7f284015a790 0x7f28400cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:24.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.296+0000 7f2835400640 1 -- 172.21.15.46:0/2517025226 >> v1:172.21.15.46:6789/0 conn(0x7f28400cc430 legacy=0x7f28400b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:24.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.296+0000 7f2835400640 1 -- 172.21.15.46:0/2517025226 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28400cec20 con 0x7f284015ab70 2024-03-29T08:19:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.300+0000 7f2835400640 1 --2- 172.21.15.46:0/2517025226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f284015ab70 0x7f2840098a40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f28400d0610 tx=0x7f282801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae5223586bda134 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.300+0000 7f283d400640 1 -- 172.21.15.46:0/2517025226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28280a5790 con 0x7f284015ab70 2024-03-29T08:19:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.300+0000 7f283d400640 1 -- 172.21.15.46:0/2517025226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28280a58f0 con 0x7f284015ab70 2024-03-29T08:19:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.300+0000 7f283d400640 1 -- 172.21.15.46:0/2517025226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28280a5c80 con 0x7f284015ab70 2024-03-29T08:19:24.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.300+0000 7f2845200640 1 -- 172.21.15.46:0/2517025226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f284015ab70 msgr2=0x7f2840098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:24.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.300+0000 7f2845200640 1 --2- 172.21.15.46:0/2517025226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f284015ab70 0x7f2840098a40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f28400d0610 tx=0x7f282801c9c0 comp rx=0 tx=0).stop 2024-03-29T08:19:24.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.300+0000 7f2845200640 1 -- 172.21.15.46:0/2517025226 shutdown_connections 2024-03-29T08:19:24.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.300+0000 7f2845200640 1 --2- 172.21.15.46:0/2517025226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f284015ab70 0x7f2840098a40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:24.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.300+0000 7f2845200640 1 --2- 172.21.15.46:0/2517025226 >> v2:172.21.15.73:3300/0 conn(0x7f284015a790 0x7f28400cbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:24.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.300+0000 7f2845200640 1 -- 172.21.15.46:0/2517025226 >> 172.21.15.46:0/2517025226 conn(0x7f28400b21a0 msgr2=0x7f28400b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:24.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.300+0000 7f2845200640 1 -- 172.21.15.46:0/2517025226 shutdown_connections 2024-03-29T08:19:24.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.300+0000 7f2845200640 1 -- 172.21.15.46:0/2517025226 wait complete. 2024-03-29T08:19:24.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f2845200640 1 Processor -- start 2024-03-29T08:19:24.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f2845200640 1 -- start start 2024-03-29T08:19:24.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f2845200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28400cc430 0x7f2840151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:24.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f2845200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f284015a790 0x7f284006c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:24.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f2845200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f284015ab70 0x7f28400c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:24.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f2845200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28401522b0 con 0x7f28400cc430 2024-03-29T08:19:24.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f2845200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28400c8e10 con 0x7f284015ab70 2024-03-29T08:19:24.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f2845200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28400c90f0 con 0x7f284015a790 2024-03-29T08:19:24.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f283de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28400cc430 0x7f2840151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:24.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f283de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28400cc430 0x7f2840151d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34908/0 (socket says 172.21.15.46:34908) 2024-03-29T08:19:24.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f283de00640 1 -- 172.21.15.46:0/1187768444 learned_addr learned my addr 172.21.15.46:0/1187768444 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:24.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f283e800640 1 --2- 172.21.15.46:0/1187768444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f284015ab70 0x7f28400c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:24.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f2835400640 1 --2- 172.21.15.46:0/1187768444 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f284015a790 0x7f284006c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:24.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f283de00640 1 -- 172.21.15.46:0/1187768444 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f284015a790 msgr2=0x7f284006c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:24.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f283de00640 1 --2- 172.21.15.46:0/1187768444 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f284015a790 0x7f284006c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:24.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f283de00640 1 -- 172.21.15.46:0/1187768444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f284015ab70 msgr2=0x7f28400c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:24.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f283de00640 1 --2- 172.21.15.46:0/1187768444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f284015ab70 0x7f28400c9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:24.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f283de00640 1 -- 172.21.15.46:0/1187768444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2828009b30 con 0x7f28400cc430 2024-03-29T08:19:24.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f283de00640 1 --2- 172.21.15.46:0/1187768444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28400cc430 0x7f2840151d70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f283000d800 tx=0x7f283000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:24.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f2836a00640 1 -- 172.21.15.46:0/1187768444 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283000b6c0 con 0x7f28400cc430 2024-03-29T08:19:24.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f2836a00640 1 -- 172.21.15.46:0/1187768444 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283000b820 con 0x7f28400cc430 2024-03-29T08:19:24.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f2845200640 1 -- 172.21.15.46:0/1187768444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28400c93d0 con 0x7f28400cc430 2024-03-29T08:19:24.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f2836a00640 1 -- 172.21.15.46:0/1187768444 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283000bb90 con 0x7f28400cc430 2024-03-29T08:19:24.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.304+0000 7f2845200640 1 -- 172.21.15.46:0/1187768444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28400c9990 con 0x7f28400cc430 2024-03-29T08:19:24.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.308+0000 7f2845200640 1 -- 172.21.15.46:0/1187768444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28400736f0 con 0x7f28400cc430 2024-03-29T08:19:24.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.308+0000 7f2836a00640 1 -- 172.21.15.46:0/1187768444 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f283000bcf0 con 0x7f28400cc430 2024-03-29T08:19:24.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.308+0000 7f2836a00640 1 --2- 172.21.15.46:0/1187768444 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f281c066690 0x7f281c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:24.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.308+0000 7f2836a00640 1 -- 172.21.15.46:0/1187768444 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28300847c0 con 0x7f28400cc430 2024-03-29T08:19:24.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.308+0000 7f2835400640 1 --2- 172.21.15.46:0/1187768444 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f281c066690 0x7f281c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:24.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.308+0000 7f2835400640 1 --2- 172.21.15.46:0/1187768444 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f281c066690 0x7f281c068b50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f28280ae000 tx=0x7f28280a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:24.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.312+0000 7f2836a00640 1 -- 172.21.15.46:0/1187768444 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2830051a90 con 0x7f28400cc430 2024-03-29T08:19:24.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.508+0000 7f2845200640 1 -- 172.21.15.46:0/1187768444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2840066c90 con 0x7f28400cc430 2024-03-29T08:19:24.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.512+0000 7f2836a00640 1 -- 172.21.15.46:0/1187768444 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f2830011340 con 0x7f28400cc430 2024-03-29T08:19:24.512 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:24.512 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/4274575126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4274575126},{"type":"v1","addr":"172.21.15.46:6834","nonce":4274575126}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3739899623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3739899623},{"type":"v1","addr":"172.21.15.73:6837","nonce":3739899623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1905427475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1905427475},{"type":"v1","addr":"172.21.15.46:6835","nonce":1905427475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:19:14.663142+0000","modified":"2024-03-29T08:19:17.680359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5152},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5152":{"gid":5152,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/1941408960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1941408960},{"type":"v1","addr":"172.21.15.73:6836","nonce":1941408960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-29T08:19:24.512 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2024-03-29T08:19:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.512+0000 7f2845200640 1 -- 172.21.15.46:0/1187768444 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f281c066690 msgr2=0x7f281c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.512+0000 7f2845200640 1 --2- 172.21.15.46:0/1187768444 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f281c066690 0x7f281c068b50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f28280ae000 tx=0x7f28280a7070 comp rx=0 tx=0).stop 2024-03-29T08:19:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.512+0000 7f2845200640 1 -- 172.21.15.46:0/1187768444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28400cc430 msgr2=0x7f2840151d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.512+0000 7f2845200640 1 --2- 172.21.15.46:0/1187768444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28400cc430 0x7f2840151d70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f283000d800 tx=0x7f283000dcc0 comp rx=0 tx=0).stop 2024-03-29T08:19:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.512+0000 7f2845200640 1 -- 172.21.15.46:0/1187768444 shutdown_connections 2024-03-29T08:19:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.512+0000 7f2845200640 1 --2- 172.21.15.46:0/1187768444 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f281c066690 0x7f281c068b50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:24.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.512+0000 7f2845200640 1 --2- 172.21.15.46:0/1187768444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f284015ab70 0x7f28400c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.512+0000 7f2845200640 1 --2- 172.21.15.46:0/1187768444 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f284015a790 0x7f284006c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.512+0000 7f2845200640 1 --2- 172.21.15.46:0/1187768444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28400cc430 0x7f2840151d70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.512+0000 7f2845200640 1 -- 172.21.15.46:0/1187768444 >> 172.21.15.46:0/1187768444 conn(0x7f28400b21a0 msgr2=0x7f2840152c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.512+0000 7f2845200640 1 -- 172.21.15.46:0/1187768444 shutdown_connections 2024-03-29T08:19:24.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:24.512+0000 7f2845200640 1 -- 172.21.15.46:0/1187768444 wait complete. 2024-03-29T08:19:24.935 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:19:14.663142+0000', 'modified': '2024-03-29T08:19:17.680359+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5152}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5152': {'gid': 5152, 'name': 'b', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6836/1941408960', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 1941408960}, {'type': 'v1', 'addr': '172.21.15.73:6836', 'nonce': 1941408960}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:19:24.935 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T08:19:24.936 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-29T08:19:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.080+0000 7fdec9200640 1 Processor -- start 2024-03-29T08:19:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.080+0000 7fdec9200640 1 -- start start 2024-03-29T08:19:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.080+0000 7fdec9200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdec415a9c0 0x7fdec40cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec9200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdec40cc370 0x7fdec4098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec9200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdec4098f80 0x7fdec415e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec9200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec40cc7d0 con 0x7fdec4098f80 2024-03-29T08:19:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec9200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec4099410 con 0x7fdec415a9c0 2024-03-29T08:19:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec9200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec40ce290 con 0x7fdec40cc370 2024-03-29T08:19:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec2800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdec4098f80 0x7fdec415e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec2800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdec4098f80 0x7fdec415e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34916/0 (socket says 172.21.15.46:34916) 2024-03-29T08:19:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec2800640 1 -- 172.21.15.46:0/2721890269 learned_addr learned my addr 172.21.15.46:0/2721890269 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec1e00640 1 --2- 172.21.15.46:0/2721890269 >> v2:172.21.15.73:3300/0 conn(0x7fdec415a9c0 0x7fdec40cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec2800640 1 -- 172.21.15.46:0/2721890269 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdec40cc370 msgr2=0x7fdec4098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdeb9400640 1 --2- 172.21.15.46:0/2721890269 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdec40cc370 0x7fdec4098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec2800640 1 --2- 172.21.15.46:0/2721890269 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdec40cc370 0x7fdec4098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec2800640 1 -- 172.21.15.46:0/2721890269 >> v2:172.21.15.73:3300/0 conn(0x7fdec415a9c0 msgr2=0x7fdec40cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec2800640 1 --2- 172.21.15.46:0/2721890269 >> v2:172.21.15.73:3300/0 conn(0x7fdec415a9c0 0x7fdec40cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec2800640 1 -- 172.21.15.46:0/2721890269 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec40ce570 con 0x7fdec4098f80 2024-03-29T08:19:25.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec1e00640 1 --2- 172.21.15.46:0/2721890269 >> v2:172.21.15.73:3300/0 conn(0x7fdec415a9c0 0x7fdec40cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:25.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdeb9400640 1 --2- 172.21.15.46:0/2721890269 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdec40cc370 0x7fdec4098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec2800640 1 --2- 172.21.15.46:0/2721890269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec4098f80 0x7fdec415e5a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdec4073af0 tx=0x7fdebc01c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e11b8f822f2066 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec1400640 1 -- 172.21.15.46:0/2721890269 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc0ab600 con 0x7fdec4098f80 2024-03-29T08:19:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec1400640 1 -- 172.21.15.46:0/2721890269 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdebc0a7050 con 0x7fdec4098f80 2024-03-29T08:19:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.084+0000 7fdec1400640 1 -- 172.21.15.46:0/2721890269 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc0ab980 con 0x7fdec4098f80 2024-03-29T08:19:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec9200640 1 -- 172.21.15.46:0/2721890269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec4098f80 msgr2=0x7fdec415e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec9200640 1 --2- 172.21.15.46:0/2721890269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec4098f80 0x7fdec415e5a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdec4073af0 tx=0x7fdebc01c1a0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec9200640 1 -- 172.21.15.46:0/2721890269 shutdown_connections 2024-03-29T08:19:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec9200640 1 --2- 172.21.15.46:0/2721890269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec4098f80 0x7fdec415e5a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec9200640 1 --2- 172.21.15.46:0/2721890269 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdec40cc370 0x7fdec4098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec9200640 1 --2- 172.21.15.46:0/2721890269 >> v2:172.21.15.73:3300/0 conn(0x7fdec415a9c0 0x7fdec40cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec9200640 1 -- 172.21.15.46:0/2721890269 >> 172.21.15.46:0/2721890269 conn(0x7fdec40b21b0 msgr2=0x7fdec40b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec9200640 1 -- 172.21.15.46:0/2721890269 shutdown_connections 2024-03-29T08:19:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec9200640 1 -- 172.21.15.46:0/2721890269 wait complete. 2024-03-29T08:19:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec9200640 1 Processor -- start 2024-03-29T08:19:25.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec9200640 1 -- start start 2024-03-29T08:19:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec9200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdec4098f80 0x7fdec4156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40cc370 0x7fdec4156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec415a9c0 0x7fdec4156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec9200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec40c7440 con 0x7fdec415a9c0 2024-03-29T08:19:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec9200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec40c67c0 con 0x7fdec40cc370 2024-03-29T08:19:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec9200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec40c6aa0 con 0x7fdec4098f80 2024-03-29T08:19:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec415a9c0 0x7fdec4156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec415a9c0 0x7fdec4156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34924/0 (socket says 172.21.15.46:34924) 2024-03-29T08:19:25.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec2800640 1 -- 172.21.15.46:0/1618638088 learned_addr learned my addr 172.21.15.46:0/1618638088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec1e00640 1 --2- 172.21.15.46:0/1618638088 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdec4098f80 0x7fdec4156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdeb9400640 1 --2- 172.21.15.46:0/1618638088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40cc370 0x7fdec4156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec2800640 1 -- 172.21.15.46:0/1618638088 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdec4098f80 msgr2=0x7fdec4156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec2800640 1 --2- 172.21.15.46:0/1618638088 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdec4098f80 0x7fdec4156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec2800640 1 -- 172.21.15.46:0/1618638088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40cc370 msgr2=0x7fdec4156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec2800640 1 --2- 172.21.15.46:0/1618638088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40cc370 0x7fdec4156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec2800640 1 -- 172.21.15.46:0/1618638088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc0094d0 con 0x7fdec415a9c0 2024-03-29T08:19:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdec1e00640 1 --2- 172.21.15.46:0/1618638088 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdec4098f80 0x7fdec4156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.088+0000 7fdeb9400640 1 --2- 172.21.15.46:0/1618638088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40cc370 0x7fdec4156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.092+0000 7fdec2800640 1 --2- 172.21.15.46:0/1618638088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec415a9c0 0x7fdec4156560 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0ae040 tx=0x7fdebc018f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.092+0000 7fdebaa00640 1 -- 172.21.15.46:0/1618638088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc0b4630 con 0x7fdec415a9c0 2024-03-29T08:19:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.092+0000 7fdebaa00640 1 -- 172.21.15.46:0/1618638088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdebc0b4790 con 0x7fdec415a9c0 2024-03-29T08:19:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.092+0000 7fdec9200640 1 -- 172.21.15.46:0/1618638088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec40c6d80 con 0x7fdec415a9c0 2024-03-29T08:19:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.092+0000 7fdebaa00640 1 -- 172.21.15.46:0/1618638088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc0b4b40 con 0x7fdec415a9c0 2024-03-29T08:19:25.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.092+0000 7fdec9200640 1 -- 172.21.15.46:0/1618638088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec41574f0 con 0x7fdec415a9c0 2024-03-29T08:19:25.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.092+0000 7fdebaa00640 1 -- 172.21.15.46:0/1618638088 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdebc0b2040 con 0x7fdec415a9c0 2024-03-29T08:19:25.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.092+0000 7fdebaa00640 1 --2- 172.21.15.46:0/1618638088 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdea0066690 0x7fdea0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.092+0000 7fdebaa00640 1 -- 172.21.15.46:0/1618638088 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdebc11e630 con 0x7fdec415a9c0 2024-03-29T08:19:25.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.092+0000 7fdec1e00640 1 --2- 172.21.15.46:0/1618638088 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdea0066690 0x7fdea0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.092+0000 7fdecc696000 1 Processor -- start 2024-03-29T08:19:25.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.092+0000 7fdecc696000 1 -- start start 2024-03-29T08:19:25.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdec1e00640 1 --2- 172.21.15.46:0/1618638088 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdea0066690 0x7fdea0068b50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400e3d0 tx=0x7fdeb4007a10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:25.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdecc696000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba7a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdecc696000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba7a6d0 0x560b8ba2ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdecc696000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba304c0 0x560b8ba308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdecc696000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8b8f85a0 con 0x560b8ba79db0 2024-03-29T08:19:25.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdecc696000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8b885af0 con 0x560b8ba304c0 2024-03-29T08:19:25.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdecc696000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8b8830c0 con 0x560b8ba7a6d0 2024-03-29T08:19:25.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdec1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba7a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdec1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba7a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34940/0 (socket says 172.21.15.46:34940) 2024-03-29T08:19:25.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdec1e00640 1 -- 172.21.15.46:0/2842625515 learned_addr learned my addr 172.21.15.46:0/2842625515 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:25.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdec2800640 1 --2- 172.21.15.46:0/2842625515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba304c0 0x560b8ba308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdeb9400640 1 --2- 172.21.15.46:0/2842625515 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba7a6d0 0x560b8ba2ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdec1e00640 1 -- 172.21.15.46:0/2842625515 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba7a6d0 msgr2=0x560b8ba2ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:25.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdec1e00640 1 --2- 172.21.15.46:0/2842625515 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba7a6d0 0x560b8ba2ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdec1e00640 1 -- 172.21.15.46:0/2842625515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba304c0 msgr2=0x560b8ba308a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:25.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdec1e00640 1 --2- 172.21.15.46:0/2842625515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba304c0 0x560b8ba308a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdec1e00640 1 -- 172.21.15.46:0/2842625515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b8b57a890 con 0x560b8ba79db0 2024-03-29T08:19:25.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdec2800640 1 --2- 172.21.15.46:0/2842625515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba304c0 0x560b8ba308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:25.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdec1e00640 1 --2- 172.21.15.46:0/2842625515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba7a190 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400f800 tx=0x7fdeb4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e2cde282eb1522 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:25.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdeb2a00640 1 -- 172.21.15.46:0/2842625515 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb401acd0 con 0x560b8ba79db0 2024-03-29T08:19:25.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdeb2a00640 1 -- 172.21.15.46:0/2842625515 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb4013070 con 0x560b8ba79db0 2024-03-29T08:19:25.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdeb2a00640 1 -- 172.21.15.46:0/2842625515 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4025680 con 0x560b8ba79db0 2024-03-29T08:19:25.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdecc696000 1 -- 172.21.15.46:0/2842625515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 msgr2=0x560b8ba7a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:25.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdecc696000 1 --2- 172.21.15.46:0/2842625515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba7a190 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400f800 tx=0x7fdeb4011000 comp rx=0 tx=0).stop 2024-03-29T08:19:25.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdecc696000 1 -- 172.21.15.46:0/2842625515 shutdown_connections 2024-03-29T08:19:25.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdecc696000 1 --2- 172.21.15.46:0/2842625515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba304c0 0x560b8ba308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdecc696000 1 --2- 172.21.15.46:0/2842625515 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba7a6d0 0x560b8ba2ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdecc696000 1 --2- 172.21.15.46:0/2842625515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba7a190 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdecc696000 1 -- 172.21.15.46:0/2842625515 >> 172.21.15.46:0/2842625515 conn(0x560b8ba36830 msgr2=0x560b8ba2e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:25.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdecc696000 1 -- 172.21.15.46:0/2842625515 shutdown_connections 2024-03-29T08:19:25.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdecc696000 1 -- 172.21.15.46:0/2842625515 wait complete. 2024-03-29T08:19:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdecc696000 1 -- start start 2024-03-29T08:19:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.096+0000 7fdecc696000 1 Processor -- start 2024-03-29T08:19:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdecc696000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba304c0 0x560b8ba6a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdecc696000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba7aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdecc696000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba7a6d0 0x560b8ba7aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdecc696000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8b87e9b0 con 0x560b8ba79db0 2024-03-29T08:19:25.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdecc696000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8b81dc00 con 0x560b8ba304c0 2024-03-29T08:19:25.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdecc696000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8b83c070 con 0x560b8ba7a6d0 2024-03-29T08:19:25.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdecc696000 10 client.0 _ll_register_callbacks cb 0x560b8ba1c0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:19:25.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdeb9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba7aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdeb9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba7aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34948/0 (socket says 172.21.15.46:34948) 2024-03-29T08:19:25.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdeb9400640 1 -- 172.21.15.46:0/3356351056 learned_addr learned my addr 172.21.15.46:0/3356351056 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:25.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdec2800640 1 --2- 172.21.15.46:0/3356351056 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba7a6d0 0x560b8ba7aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdec1e00640 1 --2- 172.21.15.46:0/3356351056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba304c0 0x560b8ba6a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdeb9400640 1 -- 172.21.15.46:0/3356351056 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba7a6d0 msgr2=0x560b8ba7aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:25.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdeb9400640 1 --2- 172.21.15.46:0/3356351056 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba7a6d0 0x560b8ba7aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdeb9400640 1 -- 172.21.15.46:0/3356351056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba304c0 msgr2=0x560b8ba6a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:25.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdeb9400640 1 --2- 172.21.15.46:0/3356351056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba304c0 0x560b8ba6a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdeb9400640 1 -- 172.21.15.46:0/3356351056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeb400f4b0 con 0x560b8ba79db0 2024-03-29T08:19:25.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdeb9400640 1 --2- 172.21.15.46:0/3356351056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba7aab0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdeac010550 tx=0x7fdeac010a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9934595e13a178 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:25.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdea7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:19:25.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdea7400640 1 -- 172.21.15.46:0/3356351056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac014660 con 0x560b8ba79db0 2024-03-29T08:19:25.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdea7400640 1 -- 172.21.15.46:0/3356351056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdeac0147c0 con 0x560b8ba79db0 2024-03-29T08:19:25.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdecc696000 1 -- 172.21.15.46:0/3356351056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b8b83b5f0 con 0x560b8ba79db0 2024-03-29T08:19:25.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdea7400640 1 -- 172.21.15.46:0/3356351056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdeac014980 con 0x560b8ba79db0 2024-03-29T08:19:25.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdecc696000 10 client.5346 fetch_fsmap learned FSMap version 16 2024-03-29T08:19:25.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdecc696000 1 -- 172.21.15.46:0/3356351056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x560b8b843ab0 con 0x560b8ba79db0 2024-03-29T08:19:25.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdea7400640 1 -- 172.21.15.46:0/3356351056 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b8b83b5f0 con 0x560b8ba79db0 2024-03-29T08:19:25.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdea7400640 1 -- 172.21.15.46:0/3356351056 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdeac014ae0 con 0x560b8ba79db0 2024-03-29T08:19:25.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdecc696000 10 client.5346 fetch_fsmap finished waiting for FSMap version 16 2024-03-29T08:19:25.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdecc696000 10 client.5346 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-29T08:19:25.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdecc696000 20 client.5346 populate_metadata read hostname 'smithi046' 2024-03-29T08:19:25.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdecc696000 1 --2- 172.21.15.46:0/3356351056 >> [v2:172.21.15.73:6834/1941408960,v1:172.21.15.73:6836/1941408960] conn(0x560b8ba7bd60 0x560b8ba8f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdecc696000 4 client.5346 mds_command: new command op to 5152 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:19:25.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdecc696000 1 -- 172.21.15.46:0/3356351056 --> [v2:172.21.15.73:6834/1941408960,v1:172.21.15.73:6836/1941408960] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b8b87b790 con 0x560b8ba7bd60 2024-03-29T08:19:25.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdec1e00640 1 --2- 172.21.15.46:0/3356351056 >> [v2:172.21.15.73:6834/1941408960,v1:172.21.15.73:6836/1941408960] conn(0x560b8ba7bd60 0x560b8ba8f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdea7400640 10 client.5346 ms_handle_connect on v2:172.21.15.73:6834/1941408960 2024-03-29T08:19:25.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.100+0000 7fdec1e00640 1 --2- 172.21.15.46:0/3356351056 >> [v2:172.21.15.73:6834/1941408960,v1:172.21.15.73:6836/1941408960] conn(0x560b8ba7bd60 0x560b8ba8f8a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a2539275b027adc server_cookie=4a28d8815699eaf4 in_seq=0 out_seq=0 2024-03-29T08:19:25.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.104+0000 7fdea7400640 1 -- 172.21.15.46:0/3356351056 <== mds.0 v2:172.21.15.73:6834/1941408960 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560b8b87b790 con 0x560b8ba7bd60 2024-03-29T08:19:25.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.104+0000 7fdea7400640 10 client.5346 handle_command_reply: tid=0 2024-03-29T08:19:25.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.104+0000 7fdecc696000 1 client.5346 shutdown 2024-03-29T08:19:25.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.104+0000 7fdecc696000 1 -- 172.21.15.46:0/3356351056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 msgr2=0x560b8ba7aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:25.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.104+0000 7fdecc696000 1 --2- 172.21.15.46:0/3356351056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba7aab0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdeac010550 tx=0x7fdeac010a10 comp rx=0 tx=0).stop 2024-03-29T08:19:25.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.104+0000 7fdecc696000 1 -- 172.21.15.46:0/3356351056 shutdown_connections 2024-03-29T08:19:25.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.104+0000 7fdecc696000 1 --2- 172.21.15.46:0/3356351056 >> [v2:172.21.15.73:6834/1941408960,v1:172.21.15.73:6836/1941408960] conn(0x560b8ba7bd60 0x560b8ba8f8a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.104+0000 7fdecc696000 1 --2- 172.21.15.46:0/3356351056 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba7a6d0 0x560b8ba7aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.104+0000 7fdecc696000 1 --2- 172.21.15.46:0/3356351056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba7aab0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.104+0000 7fdecc696000 1 --2- 172.21.15.46:0/3356351056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba304c0 0x560b8ba6a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.104+0000 7fdecc696000 1 -- 172.21.15.46:0/3356351056 >> 172.21.15.46:0/3356351056 conn(0x560b8ba36830 msgr2=0x560b8ba2ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:25.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.104+0000 7fdea7400640 0 client.5346 ms_handle_reset on v2:172.21.15.73:6834/1941408960 2024-03-29T08:19:25.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.104+0000 7fdecc696000 1 -- 172.21.15.46:0/3356351056 shutdown_connections 2024-03-29T08:19:25.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.104+0000 7fdecc696000 1 -- 172.21.15.46:0/3356351056 wait complete. 2024-03-29T08:19:25.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.104+0000 7fdecc696000 20 client.5346 trim_cache size 0 max 16384 2024-03-29T08:19:25.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdecc696000 1 Processor -- start 2024-03-29T08:19:25.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdecc696000 1 -- start start 2024-03-29T08:19:25.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdecc696000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba304c0 0x560b8ba908b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdecc696000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba79db0 0x560b8ba90df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdecc696000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba7a6d0 0x560b8ba65700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdecc696000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8ba8b110 con 0x560b8ba304c0 2024-03-29T08:19:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdecc696000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8b53b940 con 0x560b8ba79db0 2024-03-29T08:19:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdecc696000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8b87d660 con 0x560b8ba7a6d0 2024-03-29T08:19:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdec1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba304c0 0x560b8ba908b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdec1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba304c0 0x560b8ba908b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34964/0 (socket says 172.21.15.46:34964) 2024-03-29T08:19:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdec1e00640 1 -- 172.21.15.46:0/1577198284 learned_addr learned my addr 172.21.15.46:0/1577198284 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdec2800640 1 --2- 172.21.15.46:0/1577198284 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba7a6d0 0x560b8ba65700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdeb9400640 1 --2- 172.21.15.46:0/1577198284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba79db0 0x560b8ba90df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdec1e00640 1 -- 172.21.15.46:0/1577198284 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba7a6d0 msgr2=0x560b8ba65700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdec1e00640 1 --2- 172.21.15.46:0/1577198284 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba7a6d0 0x560b8ba65700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdec1e00640 1 -- 172.21.15.46:0/1577198284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba79db0 msgr2=0x560b8ba90df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdec1e00640 1 --2- 172.21.15.46:0/1577198284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba79db0 0x560b8ba90df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdec1e00640 1 -- 172.21.15.46:0/1577198284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b8b829c00 con 0x560b8ba304c0 2024-03-29T08:19:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdec2800640 1 --2- 172.21.15.46:0/1577198284 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba7a6d0 0x560b8ba65700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:25.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdeb9400640 1 --2- 172.21.15.46:0/1577198284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba79db0 0x560b8ba90df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:25.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdec1e00640 1 --2- 172.21.15.46:0/1577198284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba304c0 0x560b8ba908b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400f800 tx=0x7fdeb4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a4b03793c7fe67 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:25.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdeb3400640 1 -- 172.21.15.46:0/1577198284 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4007d80 con 0x560b8ba304c0 2024-03-29T08:19:25.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdeb3400640 1 -- 172.21.15.46:0/1577198284 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb40319c0 con 0x560b8ba304c0 2024-03-29T08:19:25.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdeb3400640 1 -- 172.21.15.46:0/1577198284 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4031d50 con 0x560b8ba304c0 2024-03-29T08:19:25.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdecc696000 1 -- 172.21.15.46:0/1577198284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba304c0 msgr2=0x560b8ba908b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:25.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdecc696000 1 --2- 172.21.15.46:0/1577198284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba304c0 0x560b8ba908b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400f800 tx=0x7fdeb4015000 comp rx=0 tx=0).stop 2024-03-29T08:19:25.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdecc696000 1 -- 172.21.15.46:0/1577198284 shutdown_connections 2024-03-29T08:19:25.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdecc696000 1 --2- 172.21.15.46:0/1577198284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba79db0 0x560b8ba90df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdecc696000 1 --2- 172.21.15.46:0/1577198284 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba7a6d0 0x560b8ba65700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdecc696000 1 --2- 172.21.15.46:0/1577198284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba304c0 0x560b8ba908b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdecc696000 1 -- 172.21.15.46:0/1577198284 >> 172.21.15.46:0/1577198284 conn(0x560b8ba36830 msgr2=0x560b8ba793c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:25.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdecc696000 1 -- 172.21.15.46:0/1577198284 shutdown_connections 2024-03-29T08:19:25.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.120+0000 7fdecc696000 1 -- 172.21.15.46:0/1577198284 wait complete. 2024-03-29T08:19:25.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 1 -- start start 2024-03-29T08:19:25.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 1 Processor -- start 2024-03-29T08:19:25.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba304c0 0x560b8bac36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba67f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba7a6d0 0x560b8ba68460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8b87a920 con 0x560b8ba79db0 2024-03-29T08:19:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b8b83cce0 con 0x560b8ba7a6d0 2024-03-29T08:19:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b8b8c1ac0 con 0x560b8ba304c0 2024-03-29T08:19:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 10 client.0 _ll_register_callbacks cb 0x560b8b843590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:19:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdeb9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba67f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdeb9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba67f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34968/0 (socket says 172.21.15.46:34968) 2024-03-29T08:19:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdeb9400640 1 -- 172.21.15.46:0/2080480031 learned_addr learned my addr 172.21.15.46:0/2080480031 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdec2800640 1 --2- 172.21.15.46:0/2080480031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba7a6d0 0x560b8ba68460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdec1e00640 1 --2- 172.21.15.46:0/2080480031 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba304c0 0x560b8bac36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdeb9400640 1 -- 172.21.15.46:0/2080480031 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba304c0 msgr2=0x560b8bac36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdeb9400640 1 --2- 172.21.15.46:0/2080480031 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba304c0 0x560b8bac36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdeb9400640 1 -- 172.21.15.46:0/2080480031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba7a6d0 msgr2=0x560b8ba68460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdeb9400640 1 --2- 172.21.15.46:0/2080480031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba7a6d0 0x560b8ba68460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdeb9400640 1 -- 172.21.15.46:0/2080480031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b8b57a890 con 0x560b8ba79db0 2024-03-29T08:19:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdec1e00640 1 --2- 172.21.15.46:0/2080480031 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba304c0 0x560b8bac36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdeb9400640 1 --2- 172.21.15.46:0/2080480031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba67f20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdeac010220 tx=0x7fdeac0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4898f9f9c4243faf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdea7400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:19:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdea7400640 1 -- 172.21.15.46:0/2080480031 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac014a70 con 0x560b8ba79db0 2024-03-29T08:19:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdea7400640 1 -- 172.21.15.46:0/2080480031 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdeac00b040 con 0x560b8ba79db0 2024-03-29T08:19:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 1 -- 172.21.15.46:0/2080480031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b8b84e460 con 0x560b8ba79db0 2024-03-29T08:19:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdea7400640 1 -- 172.21.15.46:0/2080480031 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdeac00ed90 con 0x560b8ba79db0 2024-03-29T08:19:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 10 client.5352 fetch_fsmap learned FSMap version 16 2024-03-29T08:19:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 1 -- 172.21.15.46:0/2080480031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x560b8b57acc0 con 0x560b8ba79db0 2024-03-29T08:19:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdea7400640 1 -- 172.21.15.46:0/2080480031 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b8b84e460 con 0x560b8ba79db0 2024-03-29T08:19:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdea7400640 1 -- 172.21.15.46:0/2080480031 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdeac0059a0 con 0x560b8ba79db0 2024-03-29T08:19:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 10 client.5352 fetch_fsmap finished waiting for FSMap version 16 2024-03-29T08:19:25.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 10 client.5352 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-29T08:19:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 20 client.5352 populate_metadata read hostname 'smithi046' 2024-03-29T08:19:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 1 --2- 172.21.15.46:0/2080480031 >> [v2:172.21.15.73:6834/1941408960,v1:172.21.15.73:6836/1941408960] conn(0x560b8ba689a0 0x560b8ba68d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 4 client.5352 mds_command: new command op to 5152 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T08:19:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdecc696000 1 -- 172.21.15.46:0/2080480031 --> [v2:172.21.15.73:6834/1941408960,v1:172.21.15.73:6836/1941408960] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560b8b8479c0 con 0x560b8ba689a0 2024-03-29T08:19:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.124+0000 7fdec1e00640 1 --2- 172.21.15.46:0/2080480031 >> [v2:172.21.15.73:6834/1941408960,v1:172.21.15.73:6836/1941408960] conn(0x560b8ba689a0 0x560b8ba68d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdec1e00640 1 --2- 172.21.15.46:0/2080480031 >> [v2:172.21.15.73:6834/1941408960,v1:172.21.15.73:6836/1941408960] conn(0x560b8ba689a0 0x560b8ba68d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2367b5f34e1698db server_cookie=2bc597f6c3e1c0f3 in_seq=0 out_seq=0 2024-03-29T08:19:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdea7400640 10 client.5352 ms_handle_connect on v2:172.21.15.73:6834/1941408960 2024-03-29T08:19:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdea7400640 1 -- 172.21.15.46:0/2080480031 <== mds.0 v2:172.21.15.73:6834/1941408960 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x560b8b8479c0 con 0x560b8ba689a0 2024-03-29T08:19:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdea7400640 10 client.5352 handle_command_reply: tid=0 2024-03-29T08:19:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdecc696000 1 client.5352 shutdown 2024-03-29T08:19:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdecc696000 1 -- 172.21.15.46:0/2080480031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 msgr2=0x560b8ba67f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdecc696000 1 --2- 172.21.15.46:0/2080480031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba67f20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdeac010220 tx=0x7fdeac0046d0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdecc696000 1 -- 172.21.15.46:0/2080480031 shutdown_connections 2024-03-29T08:19:25.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdecc696000 1 --2- 172.21.15.46:0/2080480031 >> [v2:172.21.15.73:6834/1941408960,v1:172.21.15.73:6836/1941408960] conn(0x560b8ba689a0 0x560b8ba68d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdecc696000 1 --2- 172.21.15.46:0/2080480031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b8ba7a6d0 0x560b8ba68460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdecc696000 1 --2- 172.21.15.46:0/2080480031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b8ba79db0 0x560b8ba67f20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdea7400640 0 client.5352 ms_handle_reset on v2:172.21.15.73:6834/1941408960 2024-03-29T08:19:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdecc696000 1 --2- 172.21.15.46:0/2080480031 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b8ba304c0 0x560b8bac36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdecc696000 1 -- 172.21.15.46:0/2080480031 >> 172.21.15.46:0/2080480031 conn(0x560b8ba36830 msgr2=0x560b8ba8f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdecc696000 1 -- 172.21.15.46:0/2080480031 shutdown_connections 2024-03-29T08:19:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdecc696000 1 -- 172.21.15.46:0/2080480031 wait complete. 2024-03-29T08:19:25.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdecc696000 20 client.5352 trim_cache size 0 max 16384 2024-03-29T08:19:25.133 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:19:25.133 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "e9b7960f-ec1f-4787-95c3-c806094d37f0", 2024-03-29T08:19:25.134 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-03-29T08:19:25.134 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5152, 2024-03-29T08:19:25.134 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-29T08:19:25.134 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-29T08:19:25.134 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-29T08:19:25.134 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 10.460030773, 2024-03-29T08:19:25.134 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 16, 2024-03-29T08:19:25.134 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 36, 2024-03-29T08:19:25.134 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 36, 2024-03-29T08:19:25.134 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 17.777396755000002 2024-03-29T08:19:25.134 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:19:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdec9200640 1 -- 172.21.15.46:0/1618638088 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdea0066690 msgr2=0x7fdea0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdec9200640 1 --2- 172.21.15.46:0/1618638088 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdea0066690 0x7fdea0068b50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdeb400e3d0 tx=0x7fdeb4007a10 comp rx=0 tx=0).stop 2024-03-29T08:19:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdec9200640 1 -- 172.21.15.46:0/1618638088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec415a9c0 msgr2=0x7fdec4156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdec9200640 1 --2- 172.21.15.46:0/1618638088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec415a9c0 0x7fdec4156560 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0ae040 tx=0x7fdebc018f40 comp rx=0 tx=0).stop 2024-03-29T08:19:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdec9200640 1 -- 172.21.15.46:0/1618638088 shutdown_connections 2024-03-29T08:19:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdec9200640 1 --2- 172.21.15.46:0/1618638088 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdea0066690 0x7fdea0068b50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdec9200640 1 --2- 172.21.15.46:0/1618638088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec40cc370 0x7fdec4156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdec9200640 1 --2- 172.21.15.46:0/1618638088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdec415a9c0 0x7fdec4156560 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdec9200640 1 --2- 172.21.15.46:0/1618638088 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdec4098f80 0x7fdec4156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdec9200640 1 -- 172.21.15.46:0/1618638088 >> 172.21.15.46:0/1618638088 conn(0x7fdec40b21b0 msgr2=0x7fdec40b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdec9200640 1 -- 172.21.15.46:0/1618638088 shutdown_connections 2024-03-29T08:19:25.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.128+0000 7fdec9200640 1 -- 172.21.15.46:0/1618638088 wait complete. 2024-03-29T08:19:25.148 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:19:25.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.288+0000 7fb4b8600640 1 Processor -- start 2024-03-29T08:19:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.288+0000 7fb4b8600640 1 -- start start 2024-03-29T08:19:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.288+0000 7fb4b8600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4b015ad20 0x7fb4b00cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.288+0000 7fb4b8600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b00b0ab0 con 0x7fb4b015a940 2024-03-29T08:19:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.288+0000 7fb4b8600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b00b0d90 con 0x7fb4b00b0540 2024-03-29T08:19:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.288+0000 7fb4b8600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00b1070 con 0x7fb4b015ad20 2024-03-29T08:19:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.288+0000 7fb4b5c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb4b015a940 0x7fb4b00afb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56146/0 (socket says 172.21.15.46:56146) 2024-03-29T08:19:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.288+0000 7fb4b5c00640 1 -- 172.21.15.46:0/259458387 learned_addr learned my addr 172.21.15.46:0/259458387 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.288+0000 7fb4b5200640 1 --2- 172.21.15.46:0/259458387 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4b015ad20 0x7fb4b00cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.288+0000 7fb4afe00640 1 -- 172.21.15.46:0/259458387 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230363521 0 0) 0x7fb4b00b0ab0 con 0x7fb4b015a940 2024-03-29T08:19:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.288+0000 7fb4afe00640 1 -- 172.21.15.46:0/259458387 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb498003530 con 0x7fb4b015a940 2024-03-29T08:19:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.288+0000 7fb4afe00640 1 -- 172.21.15.46:0/259458387 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774839615 0 0) 0x7fb4b00b0d90 con 0x7fb4b00b0540 2024-03-29T08:19:25.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.288+0000 7fb4b5200640 1 -- 172.21.15.46:0/259458387 >> v1:172.21.15.73:6789/0 conn(0x7fb4b00b0540 legacy=0x7fb4b00cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:25.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.288+0000 7fb4b5200640 1 -- 172.21.15.46:0/259458387 >> v1:172.21.15.46:6789/0 conn(0x7fb4b015a940 legacy=0x7fb4b00afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:25.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.288+0000 7fb4b5200640 1 -- 172.21.15.46:0/259458387 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b00b1350 con 0x7fb4b015ad20 2024-03-29T08:19:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.292+0000 7fb4b5200640 1 --2- 172.21.15.46:0/259458387 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4b015ad20 0x7fb4b00cecb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb49c0097f0 tx=0x7fb49c01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=870d5f3661987dae server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:25.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.292+0000 7fb4afe00640 1 -- 172.21.15.46:0/259458387 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c0ac620 con 0x7fb4b015ad20 2024-03-29T08:19:25.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.292+0000 7fb4afe00640 1 -- 172.21.15.46:0/259458387 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49c0ac780 con 0x7fb4b015ad20 2024-03-29T08:19:25.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.292+0000 7fb4afe00640 1 -- 172.21.15.46:0/259458387 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c0acb10 con 0x7fb4b015ad20 2024-03-29T08:19:25.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.292+0000 7fb4b8600640 1 -- 172.21.15.46:0/259458387 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4b015ad20 msgr2=0x7fb4b00cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:25.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.292+0000 7fb4b8600640 1 --2- 172.21.15.46:0/259458387 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4b015ad20 0x7fb4b00cecb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb49c0097f0 tx=0x7fb49c01c4b0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.292+0000 7fb4b8600640 1 -- 172.21.15.46:0/259458387 shutdown_connections 2024-03-29T08:19:25.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.292+0000 7fb4b8600640 1 --2- 172.21.15.46:0/259458387 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4b015ad20 0x7fb4b00cecb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.292+0000 7fb4b8600640 1 -- 172.21.15.46:0/259458387 >> 172.21.15.46:0/259458387 conn(0x7fb4b0089fd0 msgr2=0x7fb4b008a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:25.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.292+0000 7fb4b8600640 1 -- 172.21.15.46:0/259458387 shutdown_connections 2024-03-29T08:19:25.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.292+0000 7fb4b8600640 1 -- 172.21.15.46:0/259458387 wait complete. 2024-03-29T08:19:25.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b8600640 1 Processor -- start 2024-03-29T08:19:25.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b8600640 1 -- start start 2024-03-29T08:19:25.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b8600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4b00b0540 0x7fb4b00c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b8600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4b015a940 0x7fb4b006edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b8600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b015ad20 0x7fb4b0165bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b8600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00c66e0 con 0x7fb4b015ad20 2024-03-29T08:19:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b8600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00c60d0 con 0x7fb4b00b0540 2024-03-29T08:19:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b8600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00b6100 con 0x7fb4b015a940 2024-03-29T08:19:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b6600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b015ad20 0x7fb4b0165bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b5200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4b015a940 0x7fb4b006edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b5c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4b00b0540 0x7fb4b00c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b5200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4b015a940 0x7fb4b006edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:51842/0 (socket says 172.21.15.46:51842) 2024-03-29T08:19:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b5c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4b00b0540 0x7fb4b00c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:38142/0 (socket says 172.21.15.46:38142) 2024-03-29T08:19:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b5200640 1 -- 172.21.15.46:0/2446914583 learned_addr learned my addr 172.21.15.46:0/2446914583 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b5c00640 1 -- 172.21.15.46:0/2446914583 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4b015a940 msgr2=0x7fb4b006edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b5c00640 1 --2- 172.21.15.46:0/2446914583 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4b015a940 0x7fb4b006edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b5c00640 1 -- 172.21.15.46:0/2446914583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b015ad20 msgr2=0x7fb4b0165bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b5c00640 1 --2- 172.21.15.46:0/2446914583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b015ad20 0x7fb4b0165bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b5c00640 1 -- 172.21.15.46:0/2446914583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c009420 con 0x7fb4b00b0540 2024-03-29T08:19:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b5200640 1 --2- 172.21.15.46:0/2446914583 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4b015a940 0x7fb4b006edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b6600640 1 --2- 172.21.15.46:0/2446914583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b015ad20 0x7fb4b0165bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b5c00640 1 --2- 172.21.15.46:0/2446914583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4b00b0540 0x7fb4b00c6b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4007fd0 tx=0x7fb4a400c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4ad600640 1 -- 172.21.15.46:0/2446914583 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4005cf0 con 0x7fb4b00b0540 2024-03-29T08:19:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4ad600640 1 -- 172.21.15.46:0/2446914583 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a4005e50 con 0x7fb4b00b0540 2024-03-29T08:19:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4ad600640 1 -- 172.21.15.46:0/2446914583 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a40072f0 con 0x7fb4b00b0540 2024-03-29T08:19:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b8600640 1 -- 172.21.15.46:0/2446914583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b006cec0 con 0x7fb4b00b0540 2024-03-29T08:19:25.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b8600640 1 -- 172.21.15.46:0/2446914583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b00b5e20 con 0x7fb4b00b0540 2024-03-29T08:19:25.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.296+0000 7fb4b8600640 1 -- 172.21.15.46:0/2446914583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b00c6950 con 0x7fb4b00b0540 2024-03-29T08:19:25.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.300+0000 7fb4ad600640 1 -- 172.21.15.46:0/2446914583 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb4a4007450 con 0x7fb4b00b0540 2024-03-29T08:19:25.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.300+0000 7fb4ad600640 1 --2- 172.21.15.46:0/2446914583 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb4900666c0 0x7fb490068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:25.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.300+0000 7fb4ad600640 1 -- 172.21.15.46:0/2446914583 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4a40836d0 con 0x7fb4b00b0540 2024-03-29T08:19:25.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.300+0000 7fb4b5200640 1 --2- 172.21.15.46:0/2446914583 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb4900666c0 0x7fb490068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:25.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.300+0000 7fb4b5200640 1 --2- 172.21.15.46:0/2446914583 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb4900666c0 0x7fb490068b80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb49c01a0e0 tx=0x7fb49c0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:25.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.304+0000 7fb4ad600640 1 -- 172.21.15.46:0/2446914583 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb4a4050a60 con 0x7fb4b00b0540 2024-03-29T08:19:25.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.504+0000 7fb4b8600640 1 -- 172.21.15.46:0/2446914583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb4b0066c90 con 0x7fb4b00b0540 2024-03-29T08:19:25.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.508+0000 7fb4ad600640 1 -- 172.21.15.46:0/2446914583 <== mon.1 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb4a4007e30 con 0x7fb4b00b0540 2024-03-29T08:19:25.508 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:19:25.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.508+0000 7fb4b8600640 1 -- 172.21.15.46:0/2446914583 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb4900666c0 msgr2=0x7fb490068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:25.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.508+0000 7fb4b8600640 1 --2- 172.21.15.46:0/2446914583 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb4900666c0 0x7fb490068b80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb49c01a0e0 tx=0x7fb49c0a7000 comp rx=0 tx=0).stop 2024-03-29T08:19:25.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.508+0000 7fb4b8600640 1 -- 172.21.15.46:0/2446914583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4b00b0540 msgr2=0x7fb4b00c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:25.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.508+0000 7fb4b8600640 1 --2- 172.21.15.46:0/2446914583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4b00b0540 0x7fb4b00c6b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4007fd0 tx=0x7fb4a400c890 comp rx=0 tx=0).stop 2024-03-29T08:19:25.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.508+0000 7fb4b8600640 1 -- 172.21.15.46:0/2446914583 shutdown_connections 2024-03-29T08:19:25.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.508+0000 7fb4b8600640 1 --2- 172.21.15.46:0/2446914583 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb4900666c0 0x7fb490068b80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.508+0000 7fb4b8600640 1 --2- 172.21.15.46:0/2446914583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4b015ad20 0x7fb4b0165bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.508+0000 7fb4b8600640 1 --2- 172.21.15.46:0/2446914583 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4b015a940 0x7fb4b006edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.508+0000 7fb4b8600640 1 --2- 172.21.15.46:0/2446914583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4b00b0540 0x7fb4b00c6b00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:25.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.508+0000 7fb4b8600640 1 -- 172.21.15.46:0/2446914583 >> 172.21.15.46:0/2446914583 conn(0x7fb4b0089fd0 msgr2=0x7fb4b00cb630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:25.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.508+0000 7fb4b8600640 1 -- 172.21.15.46:0/2446914583 shutdown_connections 2024-03-29T08:19:25.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:25.508+0000 7fb4b8600640 1 -- 172.21.15.46:0/2446914583 wait complete. 2024-03-29T08:19:25.530 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:19:25.530 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:19:25.530 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-29T08:19:25.530 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T08:19:25.530 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:19:25.530 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:19:25.530 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:19:25.530 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:19:25.530 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:19:25.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:25.531 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-29T08:19:25.540 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:19:25.540 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:19:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:19:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:19:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:19:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:19:25.541 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:19:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:19:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-29T08:19:25.541 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:19:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:19:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-29T08:19:25.541 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:19:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:19:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-29T08:19:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T08:19:25.541 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42372sec preferred_lft 42372sec 2024-03-29T08:19:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-29T08:19:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:19:25.542 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:19:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:19:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-29T08:19:25.542 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:19:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:d9:58:09:4c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:19:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:19:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:19:25.542 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:19:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T08:19:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T08:19:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:19:25.542 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T08:19:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:19:25.543 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:19:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:19:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T08:19:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:19:25.543 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:19:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T08:19:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-29T08:19:25.543 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:19:25.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:25.544 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:19:25.605 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:19:25.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:25.607 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-29T08:19:25.661 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-29T08:19:25.661 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:19:25.661 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T08:19:25.661 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:19:25.661 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T08:19:25.661 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:19:25.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:25.662 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:19:25.662 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:19:25.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:19:25.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:19:25.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:19:25.662 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:19:25.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:25.757 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:19:25.807 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T08:19:25.807 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T08:19:25.808 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:19:25.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:25.809 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-03-29T08:19:25.871 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T08:19:25.871 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:19:25.872 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T08:19:25.919 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:19:25.920 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:19:25.920 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:19:25.971 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:19:26.036 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:19:26.036 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:19:26.098 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:19:26.099 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:19:26.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:26.100 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:19:26.151 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:19:26.152 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T08:19:26.198 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:19:26.219 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:19:26.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:19:26.281 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:19:26.283 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:19:26.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:26.284 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:19:26.292 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-03-29T08:19:26.292+0000 7fa0d96f8580 -1 init, newargv = 0x559194f630e0 newargc=15 2024-03-29T08:19:26.292 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[36401]: starting ceph client 2024-03-29T08:19:26.300 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[36401]: starting fuse 2024-03-29T08:19:27.402 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:19:27.426 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:19:27.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:19:27.491 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:19:27.492 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:19:27.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:27.493 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:19:27.544 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:19:27.544 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-29T08:19:27.545 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:19:27.545 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-03-29T08:19:27.545 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-03-29T08:19:27.545 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:19:27.545 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-03-29T08:19:27.545 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:19:27.545 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-03-29T08:19:27.545 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T08:19:27.545 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-03-29T08:19:27.545 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:19:27.545 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:19:27.545 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-03-29T08:19:27.545 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-03-29T08:19:27.545 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-03-29T08:19:27.545 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-03-29T08:19:27.546 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:19:27.546 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-03-29T08:19:27.546 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:19:27.546 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:19:27.546 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:19:27.546 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-03-29T08:19:27.546 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-03-29T08:19:27.546 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-03-29T08:19:27.546 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:19:27.546 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:19:27.547 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-03-29T08:19:27.547 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:19:27.681 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.36401.asok 2024-03-29T08:19:27.864 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36401.asok 2024-03-29T08:19:27.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:27.865 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36401.asok status 2024-03-29T08:19:28.049 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:19:28.049 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-03-29T08:19:28.049 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:19:28.050 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:19:28.050 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-03-29T08:19:28.050 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-03-29T08:19:28.050 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T08:19:28.050 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "36401", 2024-03-29T08:19:28.050 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-03-29T08:19:28.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:28.050 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-03-29T08:19:28.050 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-03-29T08:19:28.050 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5364, 2024-03-29T08:19:28.050 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-03-29T08:19:28.050 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-03-29T08:19:28.050 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-03-29T08:19:28.051 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 5364 2024-03-29T08:19:28.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:28.051 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:19:28.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:19:28.051 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:19:28.051 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1636711963 2024-03-29T08:19:28.051 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:19:28.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:28.051 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:19:28.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:19:28.051 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:19:28.051 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1636711963 2024-03-29T08:19:28.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:19:28.052 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.5364 192.168.0.1:0/1636711963", 2024-03-29T08:19:28.052 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/1636711963", 2024-03-29T08:19:28.052 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-03-29T08:19:28.052 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 16, 2024-03-29T08:19:28.052 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 36, 2024-03-29T08:19:28.052 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:19:28.052 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-03-29T08:19:28.052 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-03-29T08:19:28.052 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:19:28.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:28.064 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-03-29T08:19:28.065 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:19:28.074 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-03-29T08:19:28.074 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T08:19:28.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:28.074 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:19:28.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:28.137 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-03-29T08:19:28.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:28.197 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-03-29T08:19:28.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:28.251 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-03-29T08:19:28.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:28.309 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-03-29T08:19:28.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:28.366 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-03-29T08:19:28.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:28.419 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-29T08:19:28.480 INFO:teuthology.orchestra.run.smithi046.stdout:./ 2024-03-29T08:19:28.480 INFO:teuthology.orchestra.run.smithi046.stdout:./rfile 2024-03-29T08:19:28.480 INFO:teuthology.orchestra.run.smithi046.stdout:./parentdir 2024-03-29T08:19:28.480 INFO:teuthology.orchestra.run.smithi046.stdout:./parentdir/pfile 2024-03-29T08:19:28.480 INFO:teuthology.orchestra.run.smithi046.stdout:./parentdir/childdir 2024-03-29T08:19:28.480 INFO:teuthology.orchestra.run.smithi046.stdout:./parentdir/childdir/cfile 2024-03-29T08:19:28.481 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:19:28.482 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:19:28.482 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:19:28.482 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:19:28.482 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-29T08:19:28.482 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:19:28.591 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:19:28.781 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:19:28.781 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:19:28.781 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:19:28.781 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:19:28.781 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-03-29T08:19:28.781 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:19:28.888 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627778 2024-03-29T08:19:29.077 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:19:29.077 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:19:29.077 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:19:29.077 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:19:29.077 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-03-29T08:19:29.077 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:19:29.183 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627776 2024-03-29T08:19:29.373 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:19:29.373 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:19:29.374 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:19:29.374 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:19:29.374 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-03-29T08:19:29.374 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:19:29.479 INFO:teuthology.orchestra.run.smithi046.stdout:1099511628279 2024-03-29T08:19:29.669 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:19:29.669 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:19:29.669 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:19:29.669 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:19:29.669 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-03-29T08:19:29.669 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:19:29.779 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627777 2024-03-29T08:19:29.968 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:19:29.968 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:19:29.969 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:19:29.969 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:19:29.969 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-03-29T08:19:29.969 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:19:30.077 INFO:teuthology.orchestra.run.smithi046.stdout:1099511628280 2024-03-29T08:19:30.266 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:19:30.266 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:19:30.320 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:19:30.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.504+0000 7f4341400640 1 Processor -- start 2024-03-29T08:19:30.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.504+0000 7f4341400640 1 -- start start 2024-03-29T08:19:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.504+0000 7f4341400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f433c156e80 0x7f433c15d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.504+0000 7f4341400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f433c0b1d30 0x7f433c0b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.504+0000 7f4341400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f433c066100 con 0x7f433c1604c0 2024-03-29T08:19:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.504+0000 7f4341400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f433c0b2720 con 0x7f433c156e80 2024-03-29T08:19:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.504+0000 7f4341400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f433c15de60 con 0x7f433c0b1d30 2024-03-29T08:19:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.504+0000 7f433be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f433c156e80 0x7f433c15d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.504+0000 7f433be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f433c156e80 0x7f433c15d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48084/0 (socket says 172.21.15.46:48084) 2024-03-29T08:19:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.504+0000 7f433be00640 1 -- 172.21.15.46:0/1478609953 learned_addr learned my addr 172.21.15.46:0/1478609953 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.504+0000 7f433b400640 1 --2- 172.21.15.46:0/1478609953 >> v2:172.21.15.73:3300/0 conn(0x7f433c0b1d30 0x7f433c0b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:30.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.504+0000 7f433be00640 1 -- 172.21.15.46:0/1478609953 >> v2:172.21.15.73:3300/0 conn(0x7f433c0b1d30 msgr2=0x7f433c0b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:30.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.504+0000 7f433be00640 1 --2- 172.21.15.46:0/1478609953 >> v2:172.21.15.73:3300/0 conn(0x7f433c0b1d30 0x7f433c0b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:30.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.504+0000 7f433be00640 1 -- 172.21.15.46:0/1478609953 >> v1:172.21.15.73:6789/0 conn(0x7f433c1604c0 legacy=0x7f433c1608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:30.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.504+0000 7f433be00640 1 -- 172.21.15.46:0/1478609953 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433c065e20 con 0x7f433c156e80 2024-03-29T08:19:30.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.504+0000 7f433b400640 1 --2- 172.21.15.46:0/1478609953 >> v2:172.21.15.73:3300/0 conn(0x7f433c0b1d30 0x7f433c0b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:30.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.508+0000 7f433be00640 1 --2- 172.21.15.46:0/1478609953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c156e80 0x7f433c15d920 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f433c070440 tx=0x7f432c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21a02612ac00d19 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:30.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.508+0000 7f433aa00640 1 -- 172.21.15.46:0/1478609953 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c0ab620 con 0x7f433c156e80 2024-03-29T08:19:30.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.508+0000 7f433aa00640 1 -- 172.21.15.46:0/1478609953 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432c0ab780 con 0x7f433c156e80 2024-03-29T08:19:30.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.508+0000 7f433aa00640 1 -- 172.21.15.46:0/1478609953 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432c0abb10 con 0x7f433c156e80 2024-03-29T08:19:30.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.508+0000 7f4341400640 1 -- 172.21.15.46:0/1478609953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c156e80 msgr2=0x7f433c15d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:30.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.508+0000 7f4341400640 1 --2- 172.21.15.46:0/1478609953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c156e80 0x7f433c15d920 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f433c070440 tx=0x7f432c01c420 comp rx=0 tx=0).stop 2024-03-29T08:19:30.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.508+0000 7f4341400640 1 -- 172.21.15.46:0/1478609953 shutdown_connections 2024-03-29T08:19:30.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.508+0000 7f4341400640 1 --2- 172.21.15.46:0/1478609953 >> v2:172.21.15.73:3300/0 conn(0x7f433c0b1d30 0x7f433c0b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:30.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.508+0000 7f4341400640 1 --2- 172.21.15.46:0/1478609953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c156e80 0x7f433c15d920 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:30.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.508+0000 7f4341400640 1 -- 172.21.15.46:0/1478609953 >> 172.21.15.46:0/1478609953 conn(0x7f433c0cb8a0 msgr2=0x7f433c0cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:30.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.508+0000 7f4341400640 1 -- 172.21.15.46:0/1478609953 shutdown_connections 2024-03-29T08:19:30.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.508+0000 7f4341400640 1 -- 172.21.15.46:0/1478609953 wait complete. 2024-03-29T08:19:30.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.508+0000 7f4341400640 1 Processor -- start 2024-03-29T08:19:30.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.508+0000 7f4341400640 1 -- start start 2024-03-29T08:19:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f4341400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f433c0b1d30 0x7f433c06e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f4341400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c156e80 0x7f433c0cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f4341400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f433c1604c0 0x7f433c141b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f4341400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c15b8e0 con 0x7f433c156e80 2024-03-29T08:19:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f4341400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433c06e7c0 con 0x7f433c0b1d30 2024-03-29T08:19:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f4341400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433c140a70 con 0x7f433c1604c0 2024-03-29T08:19:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f433b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c156e80 0x7f433c0cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f433b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c156e80 0x7f433c0cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48098/0 (socket says 172.21.15.46:48098) 2024-03-29T08:19:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f433b400640 1 -- 172.21.15.46:0/2590104050 learned_addr learned my addr 172.21.15.46:0/2590104050 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f433be00640 1 --2- 172.21.15.46:0/2590104050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f433c0b1d30 0x7f433c06e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f4340a00640 1 --2- 172.21.15.46:0/2590104050 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f433c1604c0 0x7f433c141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f433b400640 1 -- 172.21.15.46:0/2590104050 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f433c1604c0 msgr2=0x7f433c141b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f433b400640 1 --2- 172.21.15.46:0/2590104050 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f433c1604c0 0x7f433c141b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f433b400640 1 -- 172.21.15.46:0/2590104050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f433c0b1d30 msgr2=0x7f433c06e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f433b400640 1 --2- 172.21.15.46:0/2590104050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f433c0b1d30 0x7f433c06e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f433b400640 1 -- 172.21.15.46:0/2590104050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c0094d0 con 0x7f433c156e80 2024-03-29T08:19:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f433b400640 1 --2- 172.21.15.46:0/2590104050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c156e80 0x7f433c0cd520 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f432800c7c0 tx=0x7f432800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f4333e00640 1 -- 172.21.15.46:0/2590104050 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328007a30 con 0x7f433c156e80 2024-03-29T08:19:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f4333e00640 1 -- 172.21.15.46:0/2590104050 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4328007b90 con 0x7f433c156e80 2024-03-29T08:19:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f4341400640 1 -- 172.21.15.46:0/2590104050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433c140d50 con 0x7f433c156e80 2024-03-29T08:19:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f4333e00640 1 -- 172.21.15.46:0/2590104050 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432800ce70 con 0x7f433c156e80 2024-03-29T08:19:30.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f4341400640 1 -- 172.21.15.46:0/2590104050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433c141310 con 0x7f433c156e80 2024-03-29T08:19:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.512+0000 7f4341400640 1 -- 172.21.15.46:0/2590104050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f433c167680 con 0x7f433c156e80 2024-03-29T08:19:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.516+0000 7f4333e00640 1 -- 172.21.15.46:0/2590104050 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f43280113f0 con 0x7f433c156e80 2024-03-29T08:19:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.516+0000 7f4333e00640 1 --2- 172.21.15.46:0/2590104050 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f431c066690 0x7f431c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.516+0000 7f4333e00640 1 -- 172.21.15.46:0/2590104050 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4328084680 con 0x7f433c156e80 2024-03-29T08:19:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.516+0000 7f433be00640 1 --2- 172.21.15.46:0/2590104050 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f431c066690 0x7f431c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.516+0000 7f433be00640 1 --2- 172.21.15.46:0/2590104050 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f431c066690 0x7f431c068b50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f432c0af000 tx=0x7f432c0b6040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:30.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.520+0000 7f4333e00640 1 -- 172.21.15.46:0/2590104050 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4328051a00 con 0x7f433c156e80 2024-03-29T08:19:30.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.684+0000 7f4341400640 1 -- 172.21.15.46:0/2590104050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f433c04e040 con 0x7f433c156e80 2024-03-29T08:19:30.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.688+0000 7f4333e00640 1 -- 172.21.15.46:0/2590104050 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f4328055020 con 0x7f433c156e80 2024-03-29T08:19:30.688 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:30.688 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:19:15.665674+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:19:12.674055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-29T08:19:13.661287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:19:30.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.688+0000 7f4341400640 1 -- 172.21.15.46:0/2590104050 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f431c066690 msgr2=0x7f431c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:30.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.688+0000 7f4341400640 1 --2- 172.21.15.46:0/2590104050 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f431c066690 0x7f431c068b50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f432c0af000 tx=0x7f432c0b6040 comp rx=0 tx=0).stop 2024-03-29T08:19:30.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.688+0000 7f4341400640 1 -- 172.21.15.46:0/2590104050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c156e80 msgr2=0x7f433c0cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:30.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.688+0000 7f4341400640 1 --2- 172.21.15.46:0/2590104050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c156e80 0x7f433c0cd520 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f432800c7c0 tx=0x7f432800cc80 comp rx=0 tx=0).stop 2024-03-29T08:19:30.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.688+0000 7f4341400640 1 -- 172.21.15.46:0/2590104050 shutdown_connections 2024-03-29T08:19:30.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.688+0000 7f4341400640 1 --2- 172.21.15.46:0/2590104050 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f431c066690 0x7f431c068b50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.688+0000 7f4341400640 1 --2- 172.21.15.46:0/2590104050 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f433c1604c0 0x7f433c141b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.688+0000 7f4341400640 1 --2- 172.21.15.46:0/2590104050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f433c156e80 0x7f433c0cd520 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.688+0000 7f4341400640 1 --2- 172.21.15.46:0/2590104050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f433c0b1d30 0x7f433c06e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.688+0000 7f4341400640 1 -- 172.21.15.46:0/2590104050 >> 172.21.15.46:0/2590104050 conn(0x7f433c0cb8a0 msgr2=0x7f433c15d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.688+0000 7f4341400640 1 -- 172.21.15.46:0/2590104050 shutdown_connections 2024-03-29T08:19:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.688+0000 7f4341400640 1 -- 172.21.15.46:0/2590104050 wait complete. 2024-03-29T08:19:30.710 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:19:12.674055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:19:13.661287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:19:30.711 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:19:30.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.852+0000 7fbd3d000640 1 Processor -- start 2024-03-29T08:19:30.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.852+0000 7fbd3d000640 1 -- start start 2024-03-29T08:19:30.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.856+0000 7fbd3d000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbd380cbea0 0x7fbd380b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:30.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.856+0000 7fbd3d000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd3815efe0 0x7fbd380b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:30.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.856+0000 7fbd3d000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd380b20b0 con 0x7fbd38143eb0 2024-03-29T08:19:30.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.856+0000 7fbd3d000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd380b2360 con 0x7fbd380cbea0 2024-03-29T08:19:30.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.856+0000 7fbd3d000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3815e500 con 0x7fbd3815efe0 2024-03-29T08:19:30.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.856+0000 7fbd2d400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbd380cbea0 0x7fbd380b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:30.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.856+0000 7fbd36800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd3815efe0 0x7fbd380b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:30.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.856+0000 7fbd35e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbd38143eb0 0x7fbd380cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:59040/0 (socket says 172.21.15.46:59040) 2024-03-29T08:19:30.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.856+0000 7fbd35e00640 1 -- 172.21.15.46:0/4036148936 learned_addr learned my addr 172.21.15.46:0/4036148936 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:30.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.856+0000 7fbd2d400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbd380cbea0 0x7fbd380b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:59794/0 (socket says 172.21.15.46:59794) 2024-03-29T08:19:30.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.856+0000 7fbd2d400640 1 -- 172.21.15.46:0/4036148936 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd3815efe0 msgr2=0x7fbd380b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:30.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.856+0000 7fbd2d400640 1 --2- 172.21.15.46:0/4036148936 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd3815efe0 0x7fbd380b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:30.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.856+0000 7fbd2d400640 1 -- 172.21.15.46:0/4036148936 >> v1:172.21.15.73:6789/0 conn(0x7fbd38143eb0 legacy=0x7fbd380cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:30.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.856+0000 7fbd2d400640 1 -- 172.21.15.46:0/4036148936 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3815e7e0 con 0x7fbd380cbea0 2024-03-29T08:19:30.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.856+0000 7fbd35400640 1 -- 172.21.15.46:0/4036148936 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951641447 0 0) 0x7fbd380b20b0 con 0x7fbd38143eb0 2024-03-29T08:19:30.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.856+0000 7fbd36800640 1 --2- 172.21.15.46:0/4036148936 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd3815efe0 0x7fbd380b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:30.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.860+0000 7fbd2d400640 1 --2- 172.21.15.46:0/4036148936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd380cbea0 0x7fbd380b1630 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbd200097c0 tx=0x7fbd2001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=770263b89fbc8fb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:30.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.860+0000 7fbd35400640 1 -- 172.21.15.46:0/4036148936 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd200ab610 con 0x7fbd380cbea0 2024-03-29T08:19:30.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.860+0000 7fbd35400640 1 -- 172.21.15.46:0/4036148936 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd200ab770 con 0x7fbd380cbea0 2024-03-29T08:19:30.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.860+0000 7fbd35400640 1 -- 172.21.15.46:0/4036148936 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd200abb00 con 0x7fbd380cbea0 2024-03-29T08:19:30.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.860+0000 7fbd3d000640 1 -- 172.21.15.46:0/4036148936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd380cbea0 msgr2=0x7fbd380b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:30.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.860+0000 7fbd3d000640 1 --2- 172.21.15.46:0/4036148936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd380cbea0 0x7fbd380b1630 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbd200097c0 tx=0x7fbd2001c470 comp rx=0 tx=0).stop 2024-03-29T08:19:30.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.860+0000 7fbd3d000640 1 -- 172.21.15.46:0/4036148936 shutdown_connections 2024-03-29T08:19:30.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.860+0000 7fbd3d000640 1 --2- 172.21.15.46:0/4036148936 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd3815efe0 0x7fbd380b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:30.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.860+0000 7fbd3d000640 1 --2- 172.21.15.46:0/4036148936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd380cbea0 0x7fbd380b1630 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:30.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.860+0000 7fbd3d000640 1 -- 172.21.15.46:0/4036148936 >> 172.21.15.46:0/4036148936 conn(0x7fbd3806a390 msgr2=0x7fbd380cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:30.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.860+0000 7fbd3d000640 1 -- 172.21.15.46:0/4036148936 shutdown_connections 2024-03-29T08:19:30.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.860+0000 7fbd3d000640 1 -- 172.21.15.46:0/4036148936 wait complete. 2024-03-29T08:19:30.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.860+0000 7fbd3d000640 1 Processor -- start 2024-03-29T08:19:30.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.860+0000 7fbd3d000640 1 -- start start 2024-03-29T08:19:30.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd3d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd380cbea0 0x7fbd380cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:30.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd3d000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd38143eb0 0x7fbd381f8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:30.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd3d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd3815efe0 0x7fbd381f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:30.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd3d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd381f8df0 con 0x7fbd380cbea0 2024-03-29T08:19:30.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd3d000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd381f90a0 con 0x7fbd3815efe0 2024-03-29T08:19:30.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd3d000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3812f410 con 0x7fbd38143eb0 2024-03-29T08:19:30.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd35e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd380cbea0 0x7fbd380cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:30.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd35e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd380cbea0 0x7fbd380cc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48104/0 (socket says 172.21.15.46:48104) 2024-03-29T08:19:30.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd35e00640 1 -- 172.21.15.46:0/2761056757 learned_addr learned my addr 172.21.15.46:0/2761056757 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:30.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd36800640 1 --2- 172.21.15.46:0/2761056757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd3815efe0 0x7fbd381f88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:30.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd2d400640 1 --2- 172.21.15.46:0/2761056757 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd38143eb0 0x7fbd381f8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:30.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd35e00640 1 -- 172.21.15.46:0/2761056757 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd38143eb0 msgr2=0x7fbd381f8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:30.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd35e00640 1 --2- 172.21.15.46:0/2761056757 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd38143eb0 0x7fbd381f8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:30.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd35e00640 1 -- 172.21.15.46:0/2761056757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd3815efe0 msgr2=0x7fbd381f88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:30.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd35e00640 1 --2- 172.21.15.46:0/2761056757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd3815efe0 0x7fbd381f88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:30.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd35e00640 1 -- 172.21.15.46:0/2761056757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd30009a70 con 0x7fbd380cbea0 2024-03-29T08:19:30.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd36800640 1 --2- 172.21.15.46:0/2761056757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd3815efe0 0x7fbd381f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:30.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd35e00640 1 --2- 172.21.15.46:0/2761056757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd380cbea0 0x7fbd380cc6e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbd280011f0 tx=0x7fbd2800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:30.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd2ea00640 1 -- 172.21.15.46:0/2761056757 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd280074e0 con 0x7fbd380cbea0 2024-03-29T08:19:30.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd2ea00640 1 -- 172.21.15.46:0/2761056757 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd28007640 con 0x7fbd380cbea0 2024-03-29T08:19:30.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd3d000640 1 -- 172.21.15.46:0/2761056757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd20009420 con 0x7fbd380cbea0 2024-03-29T08:19:30.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd2ea00640 1 -- 172.21.15.46:0/2761056757 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28004a10 con 0x7fbd380cbea0 2024-03-29T08:19:30.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd3d000640 1 -- 172.21.15.46:0/2761056757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3812f9d0 con 0x7fbd380cbea0 2024-03-29T08:19:30.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd3d000640 1 -- 172.21.15.46:0/2761056757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd38071a70 con 0x7fbd380cbea0 2024-03-29T08:19:30.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.864+0000 7fbd2ea00640 1 -- 172.21.15.46:0/2761056757 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbd28004b70 con 0x7fbd380cbea0 2024-03-29T08:19:30.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.868+0000 7fbd2ea00640 1 --2- 172.21.15.46:0/2761056757 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbd14066690 0x7fbd14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:30.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.868+0000 7fbd2ea00640 1 -- 172.21.15.46:0/2761056757 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd28083690 con 0x7fbd380cbea0 2024-03-29T08:19:30.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.868+0000 7fbd2d400640 1 --2- 172.21.15.46:0/2761056757 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbd14066690 0x7fbd14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:30.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.868+0000 7fbd2d400640 1 --2- 172.21.15.46:0/2761056757 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbd14066690 0x7fbd14068b50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbd200af000 tx=0x7fbd200a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:30.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:30.872+0000 7fbd2ea00640 1 -- 172.21.15.46:0/2761056757 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbd28050a90 con 0x7fbd380cbea0 2024-03-29T08:19:31.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.036+0000 7fbd3d000640 1 -- 172.21.15.46:0/2761056757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd3804e120 con 0x7fbd380cbea0 2024-03-29T08:19:31.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.036+0000 7fbd2ea00640 1 -- 172.21.15.46:0/2761056757 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd28018020 con 0x7fbd380cbea0 2024-03-29T08:19:31.037 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:19:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.036+0000 7fbd3d000640 1 -- 172.21.15.46:0/2761056757 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbd14066690 msgr2=0x7fbd14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:31.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.036+0000 7fbd3d000640 1 --2- 172.21.15.46:0/2761056757 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbd14066690 0x7fbd14068b50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbd200af000 tx=0x7fbd200a7000 comp rx=0 tx=0).stop 2024-03-29T08:19:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.036+0000 7fbd3d000640 1 -- 172.21.15.46:0/2761056757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd380cbea0 msgr2=0x7fbd380cc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.036+0000 7fbd3d000640 1 --2- 172.21.15.46:0/2761056757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd380cbea0 0x7fbd380cc6e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fbd280011f0 tx=0x7fbd2800c9b0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.036+0000 7fbd3d000640 1 -- 172.21.15.46:0/2761056757 shutdown_connections 2024-03-29T08:19:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.036+0000 7fbd3d000640 1 --2- 172.21.15.46:0/2761056757 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbd14066690 0x7fbd14068b50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.036+0000 7fbd3d000640 1 --2- 172.21.15.46:0/2761056757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd3815efe0 0x7fbd381f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.036+0000 7fbd3d000640 1 --2- 172.21.15.46:0/2761056757 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd38143eb0 0x7fbd381f8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.036+0000 7fbd3d000640 1 --2- 172.21.15.46:0/2761056757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd380cbea0 0x7fbd380cc6e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.036+0000 7fbd3d000640 1 -- 172.21.15.46:0/2761056757 >> 172.21.15.46:0/2761056757 conn(0x7fbd3806a390 msgr2=0x7fbd38164d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.036+0000 7fbd3d000640 1 -- 172.21.15.46:0/2761056757 shutdown_connections 2024-03-29T08:19:31.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.036+0000 7fbd3d000640 1 -- 172.21.15.46:0/2761056757 wait complete. 2024-03-29T08:19:31.057 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:19:31.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.200+0000 7f6784200640 1 Processor -- start 2024-03-29T08:19:31.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.200+0000 7f6784200640 1 -- start start 2024-03-29T08:19:31.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.200+0000 7f6784200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f677c142da0 0x7f677c0adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.200+0000 7f6784200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f677c143180 0x7f677c0ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.200+0000 7f6784200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f677c0abac0 0x7f677c0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.200+0000 7f6784200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f677c0ae720 con 0x7f677c0abac0 2024-03-29T08:19:31.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.200+0000 7f6784200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f677c0afe80 con 0x7f677c142da0 2024-03-29T08:19:31.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.200+0000 7f6784200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677c06cd80 con 0x7f677c143180 2024-03-29T08:19:31.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.200+0000 7f6782200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f677c0abac0 0x7f677c0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.200+0000 7f6782200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f677c0abac0 0x7f677c0cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48110/0 (socket says 172.21.15.46:48110) 2024-03-29T08:19:31.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.200+0000 7f6782200640 1 -- 172.21.15.46:0/1512435807 learned_addr learned my addr 172.21.15.46:0/1512435807 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:31.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.200+0000 7f6780e00640 1 --2- 172.21.15.46:0/1512435807 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f677c143180 0x7f677c0ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.200+0000 7f6781800640 1 --2- 172.21.15.46:0/1512435807 >> v2:172.21.15.73:3300/0 conn(0x7f677c142da0 0x7f677c0adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f6782200640 1 -- 172.21.15.46:0/1512435807 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f677c143180 msgr2=0x7f677c0ae1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:31.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f6782200640 1 --2- 172.21.15.46:0/1512435807 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f677c143180 0x7f677c0ae1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f6782200640 1 -- 172.21.15.46:0/1512435807 >> v2:172.21.15.73:3300/0 conn(0x7f677c142da0 msgr2=0x7f677c0adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:31.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f6782200640 1 --2- 172.21.15.46:0/1512435807 >> v2:172.21.15.73:3300/0 conn(0x7f677c142da0 0x7f677c0adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f6782200640 1 -- 172.21.15.46:0/1512435807 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677c06d020 con 0x7f677c0abac0 2024-03-29T08:19:31.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f6781800640 1 --2- 172.21.15.46:0/1512435807 >> v2:172.21.15.73:3300/0 conn(0x7f677c142da0 0x7f677c0adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f6782200640 1 --2- 172.21.15.46:0/1512435807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f677c0abac0 0x7f677c0cb790 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f677c06d7f0 tx=0x7f677401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5f37944cfa2ee7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:31.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f677be00640 1 -- 172.21.15.46:0/1512435807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67740a5750 con 0x7f677c0abac0 2024-03-29T08:19:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f677be00640 1 -- 172.21.15.46:0/1512435807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67740a58b0 con 0x7f677c0abac0 2024-03-29T08:19:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f677be00640 1 -- 172.21.15.46:0/1512435807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67740a5c40 con 0x7f677c0abac0 2024-03-29T08:19:31.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f6784200640 1 -- 172.21.15.46:0/1512435807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f677c0abac0 msgr2=0x7f677c0cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f6784200640 1 --2- 172.21.15.46:0/1512435807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f677c0abac0 0x7f677c0cb790 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f677c06d7f0 tx=0x7f677401c9c0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f6784200640 1 -- 172.21.15.46:0/1512435807 shutdown_connections 2024-03-29T08:19:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f6784200640 1 --2- 172.21.15.46:0/1512435807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f677c0abac0 0x7f677c0cb790 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f6784200640 1 --2- 172.21.15.46:0/1512435807 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f677c143180 0x7f677c0ae1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f6784200640 1 --2- 172.21.15.46:0/1512435807 >> v2:172.21.15.73:3300/0 conn(0x7f677c142da0 0x7f677c0adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f6784200640 1 -- 172.21.15.46:0/1512435807 >> 172.21.15.46:0/1512435807 conn(0x7f677c06a390 msgr2=0x7f677c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f6784200640 1 -- 172.21.15.46:0/1512435807 shutdown_connections 2024-03-29T08:19:31.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.204+0000 7f6784200640 1 -- 172.21.15.46:0/1512435807 wait complete. 2024-03-29T08:19:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6784200640 1 Processor -- start 2024-03-29T08:19:31.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6784200640 1 -- start start 2024-03-29T08:19:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6784200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f677c0abac0 0x7f677c132960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6784200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f677c142da0 0x7f677c15a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6784200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f677c143180 0x7f677c141a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6784200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c15bdd0 con 0x7f677c143180 2024-03-29T08:19:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6784200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677c15c260 con 0x7f677c142da0 2024-03-29T08:19:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6784200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677c15aee0 con 0x7f677c0abac0 2024-03-29T08:19:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6782200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f677c143180 0x7f677c141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6782200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f677c143180 0x7f677c141a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48122/0 (socket says 172.21.15.46:48122) 2024-03-29T08:19:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6782200640 1 -- 172.21.15.46:0/1961640976 learned_addr learned my addr 172.21.15.46:0/1961640976 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6781800640 1 --2- 172.21.15.46:0/1961640976 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f677c0abac0 0x7f677c132960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6780e00640 1 --2- 172.21.15.46:0/1961640976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f677c142da0 0x7f677c15a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6782200640 1 -- 172.21.15.46:0/1961640976 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f677c0abac0 msgr2=0x7f677c132960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6782200640 1 --2- 172.21.15.46:0/1961640976 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f677c0abac0 0x7f677c132960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6782200640 1 -- 172.21.15.46:0/1961640976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f677c142da0 msgr2=0x7f677c15a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6782200640 1 --2- 172.21.15.46:0/1961640976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f677c142da0 0x7f677c15a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6782200640 1 -- 172.21.15.46:0/1961640976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6774009b30 con 0x7f677c143180 2024-03-29T08:19:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6782200640 1 --2- 172.21.15.46:0/1961640976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f677c143180 0x7f677c141a60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f67740af040 tx=0x7f6774019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:31.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6779600640 1 -- 172.21.15.46:0/1961640976 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67740b8670 con 0x7f677c143180 2024-03-29T08:19:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6779600640 1 -- 172.21.15.46:0/1961640976 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67740b87f0 con 0x7f677c143180 2024-03-29T08:19:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6784200640 1 -- 172.21.15.46:0/1961640976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677c15b1c0 con 0x7f677c143180 2024-03-29T08:19:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.208+0000 7f6779600640 1 -- 172.21.15.46:0/1961640976 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67740b8ba0 con 0x7f677c143180 2024-03-29T08:19:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.212+0000 7f6784200640 1 -- 172.21.15.46:0/1961640976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f677c15b780 con 0x7f677c143180 2024-03-29T08:19:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.212+0000 7f6784200640 1 -- 172.21.15.46:0/1961640976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f677c15bf40 con 0x7f677c143180 2024-03-29T08:19:31.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.212+0000 7f6779600640 1 -- 172.21.15.46:0/1961640976 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f67740b8d00 con 0x7f677c143180 2024-03-29T08:19:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.212+0000 7f6779600640 1 --2- 172.21.15.46:0/1961640976 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f675c066690 0x7f675c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.212+0000 7f6779600640 1 -- 172.21.15.46:0/1961640976 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f677411f640 con 0x7f677c143180 2024-03-29T08:19:31.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.212+0000 7f6781800640 1 --2- 172.21.15.46:0/1961640976 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f675c066690 0x7f675c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.212+0000 7f6781800640 1 --2- 172.21.15.46:0/1961640976 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f675c066690 0x7f675c068b50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6770006fd0 tx=0x7f6770008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:31.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.216+0000 7f6779600640 1 -- 172.21.15.46:0/1961640976 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f67740ecb80 con 0x7f677c143180 2024-03-29T08:19:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.376+0000 7f6784200640 1 -- 172.21.15.46:0/1961640976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f677c071a70 con 0x7f677c143180 2024-03-29T08:19:31.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.380+0000 7f6779600640 1 -- 172.21.15.46:0/1961640976 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f67740f01a0 con 0x7f677c143180 2024-03-29T08:19:31.380 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:19:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.380+0000 7f6784200640 1 -- 172.21.15.46:0/1961640976 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f675c066690 msgr2=0x7f675c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.380+0000 7f6784200640 1 --2- 172.21.15.46:0/1961640976 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f675c066690 0x7f675c068b50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6770006fd0 tx=0x7f6770008040 comp rx=0 tx=0).stop 2024-03-29T08:19:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.380+0000 7f6784200640 1 -- 172.21.15.46:0/1961640976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f677c143180 msgr2=0x7f677c141a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.380+0000 7f6784200640 1 --2- 172.21.15.46:0/1961640976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f677c143180 0x7f677c141a60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f67740af040 tx=0x7f6774019760 comp rx=0 tx=0).stop 2024-03-29T08:19:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.380+0000 7f6784200640 1 -- 172.21.15.46:0/1961640976 shutdown_connections 2024-03-29T08:19:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.380+0000 7f6784200640 1 --2- 172.21.15.46:0/1961640976 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f675c066690 0x7f675c068b50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.380+0000 7f6784200640 1 --2- 172.21.15.46:0/1961640976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f677c142da0 0x7f677c15a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.380+0000 7f6784200640 1 --2- 172.21.15.46:0/1961640976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f677c143180 0x7f677c141a60 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.380+0000 7f6784200640 1 --2- 172.21.15.46:0/1961640976 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f677c0abac0 0x7f677c132960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.380+0000 7f6784200640 1 -- 172.21.15.46:0/1961640976 >> 172.21.15.46:0/1961640976 conn(0x7f677c06a390 msgr2=0x7f677c0acb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.380+0000 7f6784200640 1 -- 172.21.15.46:0/1961640976 shutdown_connections 2024-03-29T08:19:31.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.380+0000 7f6784200640 1 -- 172.21.15.46:0/1961640976 wait complete. 2024-03-29T08:19:31.400 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:19:31.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.544+0000 7f5dccc00640 1 Processor -- start 2024-03-29T08:19:31.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.544+0000 7f5dccc00640 1 -- start start 2024-03-29T08:19:31.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.544+0000 7f5dccc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5dc815abd0 0x7f5dc80cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.544+0000 7f5dccc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5dc8098a40 0x7f5dc8098eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.544+0000 7f5dccc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc80cbd00 con 0x7f5dc815a7f0 2024-03-29T08:19:31.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.544+0000 7f5dccc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc80b4500 con 0x7f5dc815abd0 2024-03-29T08:19:31.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.544+0000 7f5dccc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc80cf970 con 0x7f5dc8098a40 2024-03-29T08:19:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.544+0000 7f5dc6800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5dc8098a40 0x7f5dc8098eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.544+0000 7f5dbfe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5dc815abd0 0x7f5dc80cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.544+0000 7f5dc6800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5dc8098a40 0x7f5dc8098eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:52312/0 (socket says 172.21.15.46:52312) 2024-03-29T08:19:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.544+0000 7f5dc6800640 1 -- 172.21.15.46:0/459128598 learned_addr learned my addr 172.21.15.46:0/459128598 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.544+0000 7f5dc6800640 1 -- 172.21.15.46:0/459128598 >> v2:172.21.15.73:3300/0 conn(0x7f5dc815abd0 msgr2=0x7f5dc80cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.544+0000 7f5dc6800640 1 --2- 172.21.15.46:0/459128598 >> v2:172.21.15.73:3300/0 conn(0x7f5dc815abd0 0x7f5dc80cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.544+0000 7f5dc6800640 1 -- 172.21.15.46:0/459128598 >> v1:172.21.15.73:6789/0 conn(0x7f5dc815a7f0 legacy=0x7f5dc80cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:31.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.544+0000 7f5dc6800640 1 -- 172.21.15.46:0/459128598 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc80cbfe0 con 0x7f5dc8098a40 2024-03-29T08:19:31.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.548+0000 7f5dc6800640 1 --2- 172.21.15.46:0/459128598 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5dc8098a40 0x7f5dc8098eb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009e50 tx=0x7f5dc001ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecb74561357e75fc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:31.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.548+0000 7f5dc5400640 1 -- 172.21.15.46:0/459128598 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc00a57f0 con 0x7f5dc8098a40 2024-03-29T08:19:31.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.548+0000 7f5dc5400640 1 -- 172.21.15.46:0/459128598 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc00a5950 con 0x7f5dc8098a40 2024-03-29T08:19:31.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.548+0000 7f5dc5400640 1 -- 172.21.15.46:0/459128598 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc00a5ce0 con 0x7f5dc8098a40 2024-03-29T08:19:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.548+0000 7f5dccc00640 1 -- 172.21.15.46:0/459128598 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5dc8098a40 msgr2=0x7f5dc8098eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.548+0000 7f5dccc00640 1 --2- 172.21.15.46:0/459128598 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5dc8098a40 0x7f5dc8098eb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5dc0009e50 tx=0x7f5dc001ca30 comp rx=0 tx=0).stop 2024-03-29T08:19:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.548+0000 7f5dccc00640 1 -- 172.21.15.46:0/459128598 shutdown_connections 2024-03-29T08:19:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.548+0000 7f5dccc00640 1 --2- 172.21.15.46:0/459128598 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5dc8098a40 0x7f5dc8098eb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.548+0000 7f5dccc00640 1 --2- 172.21.15.46:0/459128598 >> v2:172.21.15.73:3300/0 conn(0x7f5dc815abd0 0x7f5dc80cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.548+0000 7f5dccc00640 1 -- 172.21.15.46:0/459128598 >> 172.21.15.46:0/459128598 conn(0x7f5dc80b2190 msgr2=0x7f5dc80b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.548+0000 7f5dccc00640 1 -- 172.21.15.46:0/459128598 shutdown_connections 2024-03-29T08:19:31.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.548+0000 7f5dccc00640 1 -- 172.21.15.46:0/459128598 wait complete. 2024-03-29T08:19:31.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.548+0000 7f5dccc00640 1 Processor -- start 2024-03-29T08:19:31.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.548+0000 7f5dccc00640 1 -- start start 2024-03-29T08:19:31.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dccc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5dc8098a40 0x7f5dc80c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dccc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc815a7f0 0x7f5dc81f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dccc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5dc815abd0 0x7f5dc814dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dccc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc80c37e0 con 0x7f5dc815a7f0 2024-03-29T08:19:31.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dccc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc8150bc0 con 0x7f5dc815abd0 2024-03-29T08:19:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dccc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc81510f0 con 0x7f5dc8098a40 2024-03-29T08:19:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dbfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc815a7f0 0x7f5dc81f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dbfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc815a7f0 0x7f5dc81f8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48132/0 (socket says 172.21.15.46:48132) 2024-03-29T08:19:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dbfe00640 1 -- 172.21.15.46:0/1427412347 learned_addr learned my addr 172.21.15.46:0/1427412347 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dc5e00640 1 --2- 172.21.15.46:0/1427412347 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5dc8098a40 0x7f5dc80c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dc6800640 1 --2- 172.21.15.46:0/1427412347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5dc815abd0 0x7f5dc814dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dbfe00640 1 -- 172.21.15.46:0/1427412347 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5dc8098a40 msgr2=0x7f5dc80c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dbfe00640 1 --2- 172.21.15.46:0/1427412347 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5dc8098a40 0x7f5dc80c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dbfe00640 1 -- 172.21.15.46:0/1427412347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5dc815abd0 msgr2=0x7f5dc814dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dbfe00640 1 --2- 172.21.15.46:0/1427412347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5dc815abd0 0x7f5dc814dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dbfe00640 1 -- 172.21.15.46:0/1427412347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc0009b30 con 0x7f5dc815a7f0 2024-03-29T08:19:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dbfe00640 1 --2- 172.21.15.46:0/1427412347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc815a7f0 0x7f5dc81f8e30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5db000e870 tx=0x7f5db000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dbe000640 1 -- 172.21.15.46:0/1427412347 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0009510 con 0x7f5dc815a7f0 2024-03-29T08:19:31.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dbe000640 1 -- 172.21.15.46:0/1427412347 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db000eec0 con 0x7f5dc815a7f0 2024-03-29T08:19:31.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dbe000640 1 -- 172.21.15.46:0/1427412347 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db000cb40 con 0x7f5dc815a7f0 2024-03-29T08:19:31.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dccc00640 1 -- 172.21.15.46:0/1427412347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc81f9370 con 0x7f5dc815a7f0 2024-03-29T08:19:31.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dccc00640 1 -- 172.21.15.46:0/1427412347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc815d4e0 con 0x7f5dc815a7f0 2024-03-29T08:19:31.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.552+0000 7f5dccc00640 1 -- 172.21.15.46:0/1427412347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc809e690 con 0x7f5dc815a7f0 2024-03-29T08:19:31.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.556+0000 7f5dbe000640 1 -- 172.21.15.46:0/1427412347 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5db000ccc0 con 0x7f5dc815a7f0 2024-03-29T08:19:31.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.556+0000 7f5dbe000640 1 --2- 172.21.15.46:0/1427412347 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5da4066690 0x7f5da4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.556+0000 7f5dbe000640 1 -- 172.21.15.46:0/1427412347 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5db0084240 con 0x7f5dc815a7f0 2024-03-29T08:19:31.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.556+0000 7f5dc5e00640 1 --2- 172.21.15.46:0/1427412347 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5da4066690 0x7f5da4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.556+0000 7f5dc5e00640 1 --2- 172.21.15.46:0/1427412347 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5da4066690 0x7f5da4068b50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5db8007fd0 tx=0x7f5db8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:31.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.560+0000 7f5dbe000640 1 -- 172.21.15.46:0/1427412347 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5db00515c0 con 0x7f5dc815a7f0 2024-03-29T08:19:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.720+0000 7f5dccc00640 1 -- 172.21.15.46:0/1427412347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5dc804e120 con 0x7f5dc815a7f0 2024-03-29T08:19:31.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.720+0000 7f5dbe000640 1 -- 172.21.15.46:0/1427412347 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5db0054be0 con 0x7f5dc815a7f0 2024-03-29T08:19:31.720 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:19:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.720+0000 7f5dccc00640 1 -- 172.21.15.46:0/1427412347 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5da4066690 msgr2=0x7f5da4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:31.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.720+0000 7f5dccc00640 1 --2- 172.21.15.46:0/1427412347 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5da4066690 0x7f5da4068b50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5db8007fd0 tx=0x7f5db8009040 comp rx=0 tx=0).stop 2024-03-29T08:19:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.720+0000 7f5dccc00640 1 -- 172.21.15.46:0/1427412347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc815a7f0 msgr2=0x7f5dc81f8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.720+0000 7f5dccc00640 1 --2- 172.21.15.46:0/1427412347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc815a7f0 0x7f5dc81f8e30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5db000e870 tx=0x7f5db000ec70 comp rx=0 tx=0).stop 2024-03-29T08:19:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.720+0000 7f5dccc00640 1 -- 172.21.15.46:0/1427412347 shutdown_connections 2024-03-29T08:19:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.720+0000 7f5dccc00640 1 --2- 172.21.15.46:0/1427412347 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5da4066690 0x7f5da4068b50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.720+0000 7f5dccc00640 1 --2- 172.21.15.46:0/1427412347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5dc815abd0 0x7f5dc814dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.720+0000 7f5dccc00640 1 --2- 172.21.15.46:0/1427412347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5dc815a7f0 0x7f5dc81f8e30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.720+0000 7f5dccc00640 1 --2- 172.21.15.46:0/1427412347 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5dc8098a40 0x7f5dc80c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.720+0000 7f5dccc00640 1 -- 172.21.15.46:0/1427412347 >> 172.21.15.46:0/1427412347 conn(0x7f5dc80b2190 msgr2=0x7f5dc806fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.720+0000 7f5dccc00640 1 -- 172.21.15.46:0/1427412347 shutdown_connections 2024-03-29T08:19:31.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.720+0000 7f5dccc00640 1 -- 172.21.15.46:0/1427412347 wait complete. 2024-03-29T08:19:31.742 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:19:31.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.880+0000 7f5770a00640 1 Processor -- start 2024-03-29T08:19:31.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.880+0000 7f5770a00640 1 -- start start 2024-03-29T08:19:31.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.884+0000 7f5770a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f576c098a40 0x7f576c098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.884+0000 7f5770a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576c0b3510 0x7f576c0b3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.884+0000 7f5770a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f576c0ce290 con 0x7f576c15a9c0 2024-03-29T08:19:31.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.884+0000 7f5770a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576c099410 con 0x7f576c098a40 2024-03-29T08:19:31.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.884+0000 7f5770a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576c0ce570 con 0x7f576c0b3510 2024-03-29T08:19:31.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.884+0000 7f5761400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f576c098a40 0x7f576c098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.884+0000 7f5761400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f576c098a40 0x7f576c098ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48140/0 (socket says 172.21.15.46:48140) 2024-03-29T08:19:31.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.884+0000 7f5761400640 1 -- 172.21.15.46:0/792006570 learned_addr learned my addr 172.21.15.46:0/792006570 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:31.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.884+0000 7f576a800640 1 --2- 172.21.15.46:0/792006570 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576c0b3510 0x7f576c0b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.884+0000 7f5761400640 1 -- 172.21.15.46:0/792006570 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576c0b3510 msgr2=0x7f576c0b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:31.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.884+0000 7f5761400640 1 --2- 172.21.15.46:0/792006570 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576c0b3510 0x7f576c0b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.884+0000 7f5761400640 1 -- 172.21.15.46:0/792006570 >> v1:172.21.15.73:6789/0 conn(0x7f576c15a9c0 legacy=0x7f576c0cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:31.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.884+0000 7f5761400640 1 -- 172.21.15.46:0/792006570 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c0ce850 con 0x7f576c098a40 2024-03-29T08:19:31.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.884+0000 7f5761400640 1 --2- 172.21.15.46:0/792006570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f576c098a40 0x7f576c098ed0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5754009870 tx=0x7f575401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26caaaadebed0b63 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:31.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.884+0000 7f5769400640 1 -- 172.21.15.46:0/792006570 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57540ab620 con 0x7f576c098a40 2024-03-29T08:19:31.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.884+0000 7f5769400640 1 -- 172.21.15.46:0/792006570 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57540ab780 con 0x7f576c098a40 2024-03-29T08:19:31.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.884+0000 7f5769400640 1 -- 172.21.15.46:0/792006570 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57540abb10 con 0x7f576c098a40 2024-03-29T08:19:31.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5770a00640 1 -- 172.21.15.46:0/792006570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f576c098a40 msgr2=0x7f576c098ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:31.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5770a00640 1 --2- 172.21.15.46:0/792006570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f576c098a40 0x7f576c098ed0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5754009870 tx=0x7f575401c460 comp rx=0 tx=0).stop 2024-03-29T08:19:31.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5770a00640 1 -- 172.21.15.46:0/792006570 shutdown_connections 2024-03-29T08:19:31.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5770a00640 1 --2- 172.21.15.46:0/792006570 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576c0b3510 0x7f576c0b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5770a00640 1 --2- 172.21.15.46:0/792006570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f576c098a40 0x7f576c098ed0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5770a00640 1 -- 172.21.15.46:0/792006570 >> 172.21.15.46:0/792006570 conn(0x7f576c0b21b0 msgr2=0x7f576c0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:31.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5770a00640 1 -- 172.21.15.46:0/792006570 shutdown_connections 2024-03-29T08:19:31.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5770a00640 1 -- 172.21.15.46:0/792006570 wait complete. 2024-03-29T08:19:31.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5770a00640 1 Processor -- start 2024-03-29T08:19:31.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5770a00640 1 -- start start 2024-03-29T08:19:31.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5770a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f576c098a40 0x7f576c0c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5770a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576c0b3510 0x7f576c149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5770a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f576c15a9c0 0x7f576c15b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5770a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c158480 con 0x7f576c098a40 2024-03-29T08:19:31.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5770a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c158a40 con 0x7f576c15a9c0 2024-03-29T08:19:31.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5770a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576c149c10 con 0x7f576c0b3510 2024-03-29T08:19:31.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5769e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f576c098a40 0x7f576c0c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5769e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f576c098a40 0x7f576c0c7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48156/0 (socket says 172.21.15.46:48156) 2024-03-29T08:19:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5769e00640 1 -- 172.21.15.46:0/1465794928 learned_addr learned my addr 172.21.15.46:0/1465794928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5761400640 1 --2- 172.21.15.46:0/1465794928 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576c0b3510 0x7f576c149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f576a800640 1 --2- 172.21.15.46:0/1465794928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f576c15a9c0 0x7f576c15b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5769e00640 1 -- 172.21.15.46:0/1465794928 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576c0b3510 msgr2=0x7f576c149460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.888+0000 7f5769e00640 1 --2- 172.21.15.46:0/1465794928 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576c0b3510 0x7f576c149460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.892+0000 7f5769e00640 1 -- 172.21.15.46:0/1465794928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f576c15a9c0 msgr2=0x7f576c15b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.892+0000 7f5769e00640 1 --2- 172.21.15.46:0/1465794928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f576c15a9c0 0x7f576c15b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.892+0000 7f5769e00640 1 -- 172.21.15.46:0/1465794928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57540094d0 con 0x7f576c098a40 2024-03-29T08:19:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.892+0000 7f5769e00640 1 --2- 172.21.15.46:0/1465794928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f576c098a40 0x7f576c0c7790 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f575c00ba50 tx=0x7f575c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.892+0000 7f5762a00640 1 -- 172.21.15.46:0/1465794928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c00cbe0 con 0x7f576c098a40 2024-03-29T08:19:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.892+0000 7f5762a00640 1 -- 172.21.15.46:0/1465794928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c00cd80 con 0x7f576c098a40 2024-03-29T08:19:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.892+0000 7f5770a00640 1 -- 172.21.15.46:0/1465794928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576c149e60 con 0x7f576c098a40 2024-03-29T08:19:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.892+0000 7f5762a00640 1 -- 172.21.15.46:0/1465794928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c014680 con 0x7f576c098a40 2024-03-29T08:19:31.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.892+0000 7f5770a00640 1 -- 172.21.15.46:0/1465794928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576c158660 con 0x7f576c098a40 2024-03-29T08:19:31.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.892+0000 7f5770a00640 1 -- 172.21.15.46:0/1465794928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576c0d0040 con 0x7f576c098a40 2024-03-29T08:19:31.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.892+0000 7f5762a00640 1 -- 172.21.15.46:0/1465794928 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f575c0147e0 con 0x7f576c098a40 2024-03-29T08:19:31.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.892+0000 7f5762a00640 1 --2- 172.21.15.46:0/1465794928 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5748066690 0x7f5748068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:31.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.892+0000 7f5762a00640 1 -- 172.21.15.46:0/1465794928 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f575c084040 con 0x7f576c098a40 2024-03-29T08:19:31.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.896+0000 7f5761400640 1 --2- 172.21.15.46:0/1465794928 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5748066690 0x7f5748068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:31.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.896+0000 7f5761400640 1 --2- 172.21.15.46:0/1465794928 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5748066690 0x7f5748068b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f57540af000 tx=0x7f57540b6040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:31.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:31.900+0000 7f5762a00640 1 -- 172.21.15.46:0/1465794928 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f575c051440 con 0x7f576c098a40 2024-03-29T08:19:32.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.072+0000 7f5770a00640 1 -- 172.21.15.46:0/1465794928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f576c14a310 con 0x7f576c098a40 2024-03-29T08:19:32.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.072+0000 7f5762a00640 1 -- 172.21.15.46:0/1465794928 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7f575c054a60 con 0x7f576c098a40 2024-03-29T08:19:32.073 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:19:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.072+0000 7f5770a00640 1 -- 172.21.15.46:0/1465794928 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5748066690 msgr2=0x7f5748068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.072+0000 7f5770a00640 1 --2- 172.21.15.46:0/1465794928 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5748066690 0x7f5748068b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f57540af000 tx=0x7f57540b6040 comp rx=0 tx=0).stop 2024-03-29T08:19:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.072+0000 7f5770a00640 1 -- 172.21.15.46:0/1465794928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f576c098a40 msgr2=0x7f576c0c7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.072+0000 7f5770a00640 1 --2- 172.21.15.46:0/1465794928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f576c098a40 0x7f576c0c7790 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f575c00ba50 tx=0x7f575c00bf10 comp rx=0 tx=0).stop 2024-03-29T08:19:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.072+0000 7f5770a00640 1 -- 172.21.15.46:0/1465794928 shutdown_connections 2024-03-29T08:19:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.072+0000 7f5770a00640 1 --2- 172.21.15.46:0/1465794928 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5748066690 0x7f5748068b50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.072+0000 7f5770a00640 1 --2- 172.21.15.46:0/1465794928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f576c15a9c0 0x7f576c15b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.072+0000 7f5770a00640 1 --2- 172.21.15.46:0/1465794928 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576c0b3510 0x7f576c149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.072+0000 7f5770a00640 1 --2- 172.21.15.46:0/1465794928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f576c098a40 0x7f576c0c7790 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.072+0000 7f5770a00640 1 -- 172.21.15.46:0/1465794928 >> 172.21.15.46:0/1465794928 conn(0x7f576c0b21b0 msgr2=0x7f576c0cc170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:32.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.072+0000 7f5770a00640 1 -- 172.21.15.46:0/1465794928 shutdown_connections 2024-03-29T08:19:32.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.072+0000 7f5770a00640 1 -- 172.21.15.46:0/1465794928 wait complete. 2024-03-29T08:19:32.095 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:19:32.095 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:19:32.103 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:19:32.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.284+0000 7f3776a00640 1 Processor -- start 2024-03-29T08:19:32.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.284+0000 7f3776a00640 1 -- start start 2024-03-29T08:19:32.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.288+0000 7f3776a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3770144210 0x7f37700cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.288+0000 7f3776a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f37700ceb20 0x7f377006c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.288+0000 7f3776a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37700cb510 con 0x7f377015ada0 2024-03-29T08:19:32.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.288+0000 7f3776a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3770070580 con 0x7f37700ceb20 2024-03-29T08:19:32.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.288+0000 7f3776a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3770070860 con 0x7f3770144210 2024-03-29T08:19:32.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.288+0000 7f3774a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f37700ceb20 0x7f377006c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.288+0000 7f376fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3770144210 0x7f37700cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.288+0000 7f3774a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f37700ceb20 0x7f377006c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:59860/0 (socket says 172.21.15.46:59860) 2024-03-29T08:19:32.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.288+0000 7f3774a00640 1 -- 172.21.15.46:0/1143910254 learned_addr learned my addr 172.21.15.46:0/1143910254 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:32.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.288+0000 7f3767400640 1 --1- 172.21.15.46:0/1143910254 >> v1:172.21.15.73:6789/0 conn(0x7f377015ada0 0x7f37700ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:59064/0 (socket says 172.21.15.46:59064) 2024-03-29T08:19:32.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.288+0000 7f3774a00640 1 -- 172.21.15.46:0/1143910254 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3770144210 msgr2=0x7f37700cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:32.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.288+0000 7f3774a00640 1 --2- 172.21.15.46:0/1143910254 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3770144210 0x7f37700cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.288+0000 7f3774a00640 1 -- 172.21.15.46:0/1143910254 >> v1:172.21.15.73:6789/0 conn(0x7f377015ada0 legacy=0x7f37700ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:32.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.288+0000 7f3774a00640 1 -- 172.21.15.46:0/1143910254 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3770070b40 con 0x7f37700ceb20 2024-03-29T08:19:32.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.292+0000 7f3774a00640 1 --2- 172.21.15.46:0/1143910254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37700ceb20 0x7f377006c950 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3768009e50 tx=0x7f376801ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d08271b985ea327d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:32.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.292+0000 7f376f400640 1 -- 172.21.15.46:0/1143910254 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37680a5810 con 0x7f37700ceb20 2024-03-29T08:19:32.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.292+0000 7f376f400640 1 -- 172.21.15.46:0/1143910254 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37680a5970 con 0x7f37700ceb20 2024-03-29T08:19:32.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.292+0000 7f376f400640 1 -- 172.21.15.46:0/1143910254 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37680a5d00 con 0x7f37700ceb20 2024-03-29T08:19:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.292+0000 7f3776a00640 1 -- 172.21.15.46:0/1143910254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37700ceb20 msgr2=0x7f377006c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.292+0000 7f3776a00640 1 --2- 172.21.15.46:0/1143910254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37700ceb20 0x7f377006c950 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3768009e50 tx=0x7f376801ca30 comp rx=0 tx=0).stop 2024-03-29T08:19:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.292+0000 7f3776a00640 1 -- 172.21.15.46:0/1143910254 shutdown_connections 2024-03-29T08:19:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.292+0000 7f3776a00640 1 --2- 172.21.15.46:0/1143910254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37700ceb20 0x7f377006c950 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.292+0000 7f3776a00640 1 --2- 172.21.15.46:0/1143910254 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3770144210 0x7f37700cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.292+0000 7f3776a00640 1 -- 172.21.15.46:0/1143910254 >> 172.21.15.46:0/1143910254 conn(0x7f3770066110 msgr2=0x7f3770069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.292+0000 7f3776a00640 1 -- 172.21.15.46:0/1143910254 shutdown_connections 2024-03-29T08:19:32.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.292+0000 7f3776a00640 1 -- 172.21.15.46:0/1143910254 wait complete. 2024-03-29T08:19:32.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.292+0000 7f3776a00640 1 Processor -- start 2024-03-29T08:19:32.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.292+0000 7f3776a00640 1 -- start start 2024-03-29T08:19:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.292+0000 7f3776a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f37700ceb20 0x7f37701576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.292+0000 7f3776a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3770144210 0x7f3770156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.292+0000 7f3776a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f377015ada0 0x7f3770138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f3776a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3770142d70 con 0x7f3770144210 2024-03-29T08:19:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f3776a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37701f35d0 con 0x7f377015ada0 2024-03-29T08:19:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f3776a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3770156e50 con 0x7f37700ceb20 2024-03-29T08:19:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f3767400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3770144210 0x7f3770156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f3767400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3770144210 0x7f3770156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48164/0 (socket says 172.21.15.46:48164) 2024-03-29T08:19:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f3767400640 1 -- 172.21.15.46:0/457420694 learned_addr learned my addr 172.21.15.46:0/457420694 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f376fe00640 1 --2- 172.21.15.46:0/457420694 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f37700ceb20 0x7f37701576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f3774a00640 1 --2- 172.21.15.46:0/457420694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f377015ada0 0x7f3770138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f3767400640 1 -- 172.21.15.46:0/457420694 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f37700ceb20 msgr2=0x7f37701576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f3767400640 1 --2- 172.21.15.46:0/457420694 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f37700ceb20 0x7f37701576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f3767400640 1 -- 172.21.15.46:0/457420694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f377015ada0 msgr2=0x7f3770138cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f3767400640 1 --2- 172.21.15.46:0/457420694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f377015ada0 0x7f3770138cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f3767400640 1 -- 172.21.15.46:0/457420694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3768009b30 con 0x7f3770144210 2024-03-29T08:19:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f376fe00640 1 --2- 172.21.15.46:0/457420694 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f37700ceb20 0x7f37701576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f3767400640 1 --2- 172.21.15.46:0/457420694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3770144210 0x7f3770156910 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f375800c430 tx=0x7f375800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f376cc00640 1 -- 172.21.15.46:0/457420694 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37580074e0 con 0x7f3770144210 2024-03-29T08:19:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f376cc00640 1 -- 172.21.15.46:0/457420694 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3758007640 con 0x7f3770144210 2024-03-29T08:19:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f376cc00640 1 -- 172.21.15.46:0/457420694 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758004a10 con 0x7f3770144210 2024-03-29T08:19:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f3776a00640 1 -- 172.21.15.46:0/457420694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377013f6f0 con 0x7f3770144210 2024-03-29T08:19:32.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f3776a00640 1 -- 172.21.15.46:0/457420694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37701424d0 con 0x7f3770144210 2024-03-29T08:19:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f3776a00640 1 -- 172.21.15.46:0/457420694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37700ce280 con 0x7f3770144210 2024-03-29T08:19:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.296+0000 7f376cc00640 1 -- 172.21.15.46:0/457420694 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3758004b70 con 0x7f3770144210 2024-03-29T08:19:32.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.300+0000 7f376cc00640 1 --2- 172.21.15.46:0/457420694 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f374c066690 0x7f374c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.300+0000 7f376cc00640 1 -- 172.21.15.46:0/457420694 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3758083b00 con 0x7f3770144210 2024-03-29T08:19:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.300+0000 7f376fe00640 1 --2- 172.21.15.46:0/457420694 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f374c066690 0x7f374c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.300+0000 7f376fe00640 1 --2- 172.21.15.46:0/457420694 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f374c066690 0x7f374c068b50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3760009f50 tx=0x7f3760009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.304+0000 7f376cc00640 1 -- 172.21.15.46:0/457420694 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3758085e60 con 0x7f3770144210 2024-03-29T08:19:32.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.464+0000 7f3776a00640 1 -- 172.21.15.46:0/457420694 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3770066c90 con 0x7f3770144210 2024-03-29T08:19:32.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.464+0000 7f376cc00640 1 -- 172.21.15.46:0/457420694 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f3758050e80 con 0x7f3770144210 2024-03-29T08:19:32.466 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:32.466 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:19:15.665674+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:19:12.674055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-29T08:19:13.661287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:19:32.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.468+0000 7f3776a00640 1 -- 172.21.15.46:0/457420694 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f374c066690 msgr2=0x7f374c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.468+0000 7f3776a00640 1 --2- 172.21.15.46:0/457420694 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f374c066690 0x7f374c068b50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3760009f50 tx=0x7f3760009190 comp rx=0 tx=0).stop 2024-03-29T08:19:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.468+0000 7f3776a00640 1 -- 172.21.15.46:0/457420694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3770144210 msgr2=0x7f3770156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.468+0000 7f3776a00640 1 --2- 172.21.15.46:0/457420694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3770144210 0x7f3770156910 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f375800c430 tx=0x7f375800c8f0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.468+0000 7f3776a00640 1 -- 172.21.15.46:0/457420694 shutdown_connections 2024-03-29T08:19:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.468+0000 7f3776a00640 1 --2- 172.21.15.46:0/457420694 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f374c066690 0x7f374c068b50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.468+0000 7f3776a00640 1 --2- 172.21.15.46:0/457420694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f377015ada0 0x7f3770138cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.468+0000 7f3776a00640 1 --2- 172.21.15.46:0/457420694 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3770144210 0x7f3770156910 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.468+0000 7f3776a00640 1 --2- 172.21.15.46:0/457420694 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f37700ceb20 0x7f37701576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.468+0000 7f3776a00640 1 -- 172.21.15.46:0/457420694 >> 172.21.15.46:0/457420694 conn(0x7f3770066110 msgr2=0x7f3770069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.468+0000 7f3776a00640 1 -- 172.21.15.46:0/457420694 shutdown_connections 2024-03-29T08:19:32.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.468+0000 7f3776a00640 1 -- 172.21.15.46:0/457420694 wait complete. 2024-03-29T08:19:32.488 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:19:12.674055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:19:13.661287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:19:32.488 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:19:32.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.632+0000 7f4c82800640 1 Processor -- start 2024-03-29T08:19:32.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.632+0000 7f4c82800640 1 -- start start 2024-03-29T08:19:32.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.632+0000 7f4c82800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7c160100 0x7f4c7c0cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.632+0000 7f4c82800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4c7c098a40 0x7f4c7c098eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.632+0000 7f4c82800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c7c0cb230 con 0x7f4c7c06bac0 2024-03-29T08:19:32.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.632+0000 7f4c82800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7c0b1f20 con 0x7f4c7c098a40 2024-03-29T08:19:32.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.632+0000 7f4c82800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0993f0 con 0x7f4c7c160100 2024-03-29T08:19:32.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.632+0000 7f4c7be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4c7c098a40 0x7f4c7c098eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.632+0000 7f4c7be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4c7c098a40 0x7f4c7c098eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48178/0 (socket says 172.21.15.46:48178) 2024-03-29T08:19:32.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.632+0000 7f4c7be00640 1 -- 172.21.15.46:0/2989045392 learned_addr learned my addr 172.21.15.46:0/2989045392 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:32.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.632+0000 7f4c7b400640 1 --2- 172.21.15.46:0/2989045392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7c160100 0x7f4c7c0cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.632+0000 7f4c7be00640 1 -- 172.21.15.46:0/2989045392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7c160100 msgr2=0x7f4c7c0cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:32.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.632+0000 7f4c7be00640 1 --2- 172.21.15.46:0/2989045392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7c160100 0x7f4c7c0cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.632+0000 7f4c7be00640 1 -- 172.21.15.46:0/2989045392 >> v1:172.21.15.73:6789/0 conn(0x7f4c7c06bac0 legacy=0x7f4c7c0b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:32.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.632+0000 7f4c7be00640 1 -- 172.21.15.46:0/2989045392 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7c06c030 con 0x7f4c7c098a40 2024-03-29T08:19:32.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.636+0000 7f4c7be00640 1 --2- 172.21.15.46:0/2989045392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c7c098a40 0x7f4c7c098eb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009e80 tx=0x7f4c7401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fddc341d90564135 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:32.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.636+0000 7f4c7aa00640 1 -- 172.21.15.46:0/2989045392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c740a57d0 con 0x7f4c7c098a40 2024-03-29T08:19:32.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.636+0000 7f4c7aa00640 1 -- 172.21.15.46:0/2989045392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c740a5930 con 0x7f4c7c098a40 2024-03-29T08:19:32.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.636+0000 7f4c7aa00640 1 -- 172.21.15.46:0/2989045392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c740a5cc0 con 0x7f4c7c098a40 2024-03-29T08:19:32.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.636+0000 7f4c82800640 1 -- 172.21.15.46:0/2989045392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c7c098a40 msgr2=0x7f4c7c098eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:32.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.636+0000 7f4c82800640 1 --2- 172.21.15.46:0/2989045392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c7c098a40 0x7f4c7c098eb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009e80 tx=0x7f4c7401ca10 comp rx=0 tx=0).stop 2024-03-29T08:19:32.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.636+0000 7f4c82800640 1 -- 172.21.15.46:0/2989045392 shutdown_connections 2024-03-29T08:19:32.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.636+0000 7f4c82800640 1 --2- 172.21.15.46:0/2989045392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c7c098a40 0x7f4c7c098eb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.636+0000 7f4c82800640 1 --2- 172.21.15.46:0/2989045392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7c160100 0x7f4c7c0cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.636+0000 7f4c82800640 1 -- 172.21.15.46:0/2989045392 >> 172.21.15.46:0/2989045392 conn(0x7f4c7c0b2190 msgr2=0x7f4c7c0b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:32.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.636+0000 7f4c82800640 1 -- 172.21.15.46:0/2989045392 shutdown_connections 2024-03-29T08:19:32.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.636+0000 7f4c82800640 1 -- 172.21.15.46:0/2989045392 wait complete. 2024-03-29T08:19:32.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.636+0000 7f4c82800640 1 Processor -- start 2024-03-29T08:19:32.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.636+0000 7f4c82800640 1 -- start start 2024-03-29T08:19:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c82800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c7c06bac0 0x7f4c7c154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c82800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c7c098a40 0x7f4c7c15c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c82800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7c160100 0x7f4c7c0c7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c82800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0c7cb0 con 0x7f4c7c098a40 2024-03-29T08:19:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c82800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0c7ea0 con 0x7f4c7c06bac0 2024-03-29T08:19:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c82800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0c8180 con 0x7f4c7c160100 2024-03-29T08:19:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c7b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c7c06bac0 0x7f4c7c154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c73e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c7c098a40 0x7f4c7c15c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c7b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c7c06bac0 0x7f4c7c154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:59882/0 (socket says 172.21.15.46:59882) 2024-03-29T08:19:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c7b400640 1 -- 172.21.15.46:0/4126789642 learned_addr learned my addr 172.21.15.46:0/4126789642 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:32.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c7be00640 1 --2- 172.21.15.46:0/4126789642 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7c160100 0x7f4c7c0c7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c7b400640 1 -- 172.21.15.46:0/4126789642 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7c160100 msgr2=0x7f4c7c0c7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c7b400640 1 --2- 172.21.15.46:0/4126789642 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7c160100 0x7f4c7c0c7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c7b400640 1 -- 172.21.15.46:0/4126789642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c7c098a40 msgr2=0x7f4c7c15c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c7b400640 1 --2- 172.21.15.46:0/4126789642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c7c098a40 0x7f4c7c15c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c7b400640 1 -- 172.21.15.46:0/4126789642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c74009b30 con 0x7f4c7c06bac0 2024-03-29T08:19:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c7b400640 1 --2- 172.21.15.46:0/4126789642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c7c06bac0 0x7f4c7c154d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00d5f0 tx=0x7f4c6c00dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c73400640 1 -- 172.21.15.46:0/4126789642 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c0125f0 con 0x7f4c7c06bac0 2024-03-29T08:19:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c73400640 1 -- 172.21.15.46:0/4126789642 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c012750 con 0x7f4c7c06bac0 2024-03-29T08:19:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c73400640 1 -- 172.21.15.46:0/4126789642 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c012b00 con 0x7f4c7c06bac0 2024-03-29T08:19:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c82800640 1 -- 172.21.15.46:0/4126789642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c7c0c8460 con 0x7f4c7c06bac0 2024-03-29T08:19:32.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c82800640 1 -- 172.21.15.46:0/4126789642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7c0c8a20 con 0x7f4c7c06bac0 2024-03-29T08:19:32.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.640+0000 7f4c82800640 1 -- 172.21.15.46:0/4126789642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7c0c8c40 con 0x7f4c7c06bac0 2024-03-29T08:19:32.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.644+0000 7f4c73400640 1 -- 172.21.15.46:0/4126789642 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4c6c012c60 con 0x7f4c7c06bac0 2024-03-29T08:19:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.644+0000 7f4c73400640 1 --2- 172.21.15.46:0/4126789642 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4c580666c0 0x7f4c58068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.644+0000 7f4c73400640 1 -- 172.21.15.46:0/4126789642 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c6c0850b0 con 0x7f4c7c06bac0 2024-03-29T08:19:32.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.644+0000 7f4c73e00640 1 --2- 172.21.15.46:0/4126789642 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4c580666c0 0x7f4c58068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.644+0000 7f4c73e00640 1 --2- 172.21.15.46:0/4126789642 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4c580666c0 0x7f4c58068b80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0cc760 tx=0x7f4c64009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:32.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.648+0000 7f4c73400640 1 -- 172.21.15.46:0/4126789642 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4c6c0524b0 con 0x7f4c7c06bac0 2024-03-29T08:19:32.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.808+0000 7f4c82800640 1 -- 172.21.15.46:0/4126789642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4c7c09e690 con 0x7f4c7c06bac0 2024-03-29T08:19:32.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.808+0000 7f4c73400640 1 -- 172.21.15.46:0/4126789642 <== mon.1 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c6c055ad0 con 0x7f4c7c06bac0 2024-03-29T08:19:32.811 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:19:32.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.812+0000 7f4c82800640 1 -- 172.21.15.46:0/4126789642 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4c580666c0 msgr2=0x7f4c58068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:32.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.812+0000 7f4c82800640 1 --2- 172.21.15.46:0/4126789642 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4c580666c0 0x7f4c58068b80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0cc760 tx=0x7f4c64009040 comp rx=0 tx=0).stop 2024-03-29T08:19:32.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.812+0000 7f4c82800640 1 -- 172.21.15.46:0/4126789642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c7c06bac0 msgr2=0x7f4c7c154d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:32.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.812+0000 7f4c82800640 1 --2- 172.21.15.46:0/4126789642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c7c06bac0 0x7f4c7c154d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00d5f0 tx=0x7f4c6c00dab0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.812+0000 7f4c82800640 1 -- 172.21.15.46:0/4126789642 shutdown_connections 2024-03-29T08:19:32.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.812+0000 7f4c82800640 1 --2- 172.21.15.46:0/4126789642 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4c580666c0 0x7f4c58068b80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.812+0000 7f4c82800640 1 --2- 172.21.15.46:0/4126789642 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4c7c160100 0x7f4c7c0c7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.812+0000 7f4c82800640 1 --2- 172.21.15.46:0/4126789642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c7c098a40 0x7f4c7c15c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.812+0000 7f4c82800640 1 --2- 172.21.15.46:0/4126789642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c7c06bac0 0x7f4c7c154d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.812+0000 7f4c82800640 1 -- 172.21.15.46:0/4126789642 >> 172.21.15.46:0/4126789642 conn(0x7f4c7c0b2190 msgr2=0x7f4c7c099530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:32.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.812+0000 7f4c82800640 1 -- 172.21.15.46:0/4126789642 shutdown_connections 2024-03-29T08:19:32.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.812+0000 7f4c82800640 1 -- 172.21.15.46:0/4126789642 wait complete. 2024-03-29T08:19:32.832 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:19:32.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.976+0000 7f6b35600640 1 Processor -- start 2024-03-29T08:19:32.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.976+0000 7f6b35600640 1 -- start start 2024-03-29T08:19:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.976+0000 7f6b35600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6b3015cb60 0x7f6b300d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.976+0000 7f6b35600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6b300cf9f0 0x7f6b300b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.976+0000 7f6b35600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b3015cf40 con 0x7f6b300d0bf0 2024-03-29T08:19:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.976+0000 7f6b35600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b300b0250 con 0x7f6b300cf9f0 2024-03-29T08:19:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.976+0000 7f6b35600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b300b0810 con 0x7f6b3015cb60 2024-03-29T08:19:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.976+0000 7f6b2f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6b300cf9f0 0x7f6b300b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.976+0000 7f6b2f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6b300cf9f0 0x7f6b300b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48198/0 (socket says 172.21.15.46:48198) 2024-03-29T08:19:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.976+0000 7f6b2f400640 1 -- 172.21.15.46:0/4240339698 learned_addr learned my addr 172.21.15.46:0/4240339698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.976+0000 7f6b2ea00640 1 --2- 172.21.15.46:0/4240339698 >> v2:172.21.15.73:3300/0 conn(0x7f6b3015cb60 0x7f6b300d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.976+0000 7f6b2f400640 1 -- 172.21.15.46:0/4240339698 >> v2:172.21.15.73:3300/0 conn(0x7f6b3015cb60 msgr2=0x7f6b300d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.976+0000 7f6b2f400640 1 --2- 172.21.15.46:0/4240339698 >> v2:172.21.15.73:3300/0 conn(0x7f6b3015cb60 0x7f6b300d06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.976+0000 7f6b2f400640 1 -- 172.21.15.46:0/4240339698 >> v1:172.21.15.73:6789/0 conn(0x7f6b300d0bf0 legacy=0x7f6b300cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:32.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.976+0000 7f6b2f400640 1 -- 172.21.15.46:0/4240339698 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b300d1050 con 0x7f6b300cf9f0 2024-03-29T08:19:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.980+0000 7f6b2f400640 1 --2- 172.21.15.46:0/4240339698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b300cf9f0 0x7f6b300b0ca0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6b3009a020 tx=0x7f6b2801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90672456856b242b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.980+0000 7f6b2e000640 1 -- 172.21.15.46:0/4240339698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b280a5880 con 0x7f6b300cf9f0 2024-03-29T08:19:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.980+0000 7f6b2e000640 1 -- 172.21.15.46:0/4240339698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b280a59e0 con 0x7f6b300cf9f0 2024-03-29T08:19:32.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.980+0000 7f6b2e000640 1 -- 172.21.15.46:0/4240339698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b280a5d70 con 0x7f6b300cf9f0 2024-03-29T08:19:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.980+0000 7f6b35600640 1 -- 172.21.15.46:0/4240339698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b300cf9f0 msgr2=0x7f6b300b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.980+0000 7f6b35600640 1 --2- 172.21.15.46:0/4240339698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b300cf9f0 0x7f6b300b0ca0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f6b3009a020 tx=0x7f6b2801c9c0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.980+0000 7f6b35600640 1 -- 172.21.15.46:0/4240339698 shutdown_connections 2024-03-29T08:19:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.980+0000 7f6b35600640 1 --2- 172.21.15.46:0/4240339698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b300cf9f0 0x7f6b300b0ca0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.980+0000 7f6b35600640 1 --2- 172.21.15.46:0/4240339698 >> v2:172.21.15.73:3300/0 conn(0x7f6b3015cb60 0x7f6b300d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.980+0000 7f6b35600640 1 -- 172.21.15.46:0/4240339698 >> 172.21.15.46:0/4240339698 conn(0x7f6b30099a10 msgr2=0x7f6b30089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:32.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.980+0000 7f6b35600640 1 -- 172.21.15.46:0/4240339698 shutdown_connections 2024-03-29T08:19:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.980+0000 7f6b35600640 1 -- 172.21.15.46:0/4240339698 wait complete. 2024-03-29T08:19:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b35600640 1 Processor -- start 2024-03-29T08:19:32.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b35600640 1 -- start start 2024-03-29T08:19:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b35600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b300cf9f0 0x7f6b3014bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b35600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6b300d0bf0 0x7f6b30154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b35600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b3015cb60 0x7f6b3014a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b35600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b30155120 con 0x7f6b300cf9f0 2024-03-29T08:19:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b35600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b300ade60 con 0x7f6b3015cb60 2024-03-29T08:19:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b35600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b301577d0 con 0x7f6b300d0bf0 2024-03-29T08:19:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b2ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b300cf9f0 0x7f6b3014bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b2ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b300cf9f0 0x7f6b3014bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48202/0 (socket says 172.21.15.46:48202) 2024-03-29T08:19:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b2ea00640 1 -- 172.21.15.46:0/1457792044 learned_addr learned my addr 172.21.15.46:0/1457792044 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b2f400640 1 --2- 172.21.15.46:0/1457792044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b3015cb60 0x7f6b3014a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b27e00640 1 --2- 172.21.15.46:0/1457792044 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6b300d0bf0 0x7f6b30154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b2ea00640 1 -- 172.21.15.46:0/1457792044 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6b300d0bf0 msgr2=0x7f6b30154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b2ea00640 1 --2- 172.21.15.46:0/1457792044 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6b300d0bf0 0x7f6b30154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b2ea00640 1 -- 172.21.15.46:0/1457792044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b3015cb60 msgr2=0x7f6b3014a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b2ea00640 1 --2- 172.21.15.46:0/1457792044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b3015cb60 0x7f6b3014a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b2ea00640 1 -- 172.21.15.46:0/1457792044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b28009b30 con 0x7f6b300cf9f0 2024-03-29T08:19:32.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b2ea00640 1 --2- 172.21.15.46:0/1457792044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b300cf9f0 0x7f6b3014bb80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000d620 tx=0x7f6b2000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b26a00640 1 -- 172.21.15.46:0/1457792044 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000b620 con 0x7f6b300cf9f0 2024-03-29T08:19:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b26a00640 1 -- 172.21.15.46:0/1457792044 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2000b780 con 0x7f6b300cf9f0 2024-03-29T08:19:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b26a00640 1 -- 172.21.15.46:0/1457792044 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2000bb10 con 0x7f6b300cf9f0 2024-03-29T08:19:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b35600640 1 -- 172.21.15.46:0/1457792044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b300ae0b0 con 0x7f6b300cf9f0 2024-03-29T08:19:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.984+0000 7f6b35600640 1 -- 172.21.15.46:0/1457792044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b30157c60 con 0x7f6b300cf9f0 2024-03-29T08:19:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.988+0000 7f6b35600640 1 -- 172.21.15.46:0/1457792044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b300ae560 con 0x7f6b300cf9f0 2024-03-29T08:19:32.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.988+0000 7f6b26a00640 1 -- 172.21.15.46:0/1457792044 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6b2000bc70 con 0x7f6b300cf9f0 2024-03-29T08:19:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.988+0000 7f6b26a00640 1 --2- 172.21.15.46:0/1457792044 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6b0c066690 0x7f6b0c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.988+0000 7f6b26a00640 1 -- 172.21.15.46:0/1457792044 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b20084650 con 0x7f6b300cf9f0 2024-03-29T08:19:32.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.988+0000 7f6b27e00640 1 --2- 172.21.15.46:0/1457792044 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6b0c066690 0x7f6b0c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.988+0000 7f6b27e00640 1 --2- 172.21.15.46:0/1457792044 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6b0c066690 0x7f6b0c068b50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6b300cc480 tx=0x7f6b18009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:32.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:32.992+0000 7f6b26a00640 1 -- 172.21.15.46:0/1457792044 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6b20051920 con 0x7f6b300cf9f0 2024-03-29T08:19:33.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.152+0000 7f6b35600640 1 -- 172.21.15.46:0/1457792044 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6b3009e7d0 con 0x7f6b300cf9f0 2024-03-29T08:19:33.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.152+0000 7f6b26a00640 1 -- 172.21.15.46:0/1457792044 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6b20007380 con 0x7f6b300cf9f0 2024-03-29T08:19:33.154 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:19:33.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.152+0000 7f6b35600640 1 -- 172.21.15.46:0/1457792044 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6b0c066690 msgr2=0x7f6b0c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:33.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.152+0000 7f6b35600640 1 --2- 172.21.15.46:0/1457792044 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6b0c066690 0x7f6b0c068b50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6b300cc480 tx=0x7f6b18009040 comp rx=0 tx=0).stop 2024-03-29T08:19:33.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.152+0000 7f6b35600640 1 -- 172.21.15.46:0/1457792044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b300cf9f0 msgr2=0x7f6b3014bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:33.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.152+0000 7f6b35600640 1 --2- 172.21.15.46:0/1457792044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b300cf9f0 0x7f6b3014bb80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000d620 tx=0x7f6b2000dae0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.156+0000 7f6b35600640 1 -- 172.21.15.46:0/1457792044 shutdown_connections 2024-03-29T08:19:33.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.156+0000 7f6b35600640 1 --2- 172.21.15.46:0/1457792044 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6b0c066690 0x7f6b0c068b50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.156+0000 7f6b35600640 1 --2- 172.21.15.46:0/1457792044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b3015cb60 0x7f6b3014a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.156+0000 7f6b35600640 1 --2- 172.21.15.46:0/1457792044 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6b300d0bf0 0x7f6b30154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.156+0000 7f6b35600640 1 --2- 172.21.15.46:0/1457792044 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6b300cf9f0 0x7f6b3014bb80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.156+0000 7f6b35600640 1 -- 172.21.15.46:0/1457792044 >> 172.21.15.46:0/1457792044 conn(0x7f6b30099a10 msgr2=0x7f6b30155a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:33.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.156+0000 7f6b35600640 1 -- 172.21.15.46:0/1457792044 shutdown_connections 2024-03-29T08:19:33.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.156+0000 7f6b35600640 1 -- 172.21.15.46:0/1457792044 wait complete. 2024-03-29T08:19:33.175 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:19:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.316+0000 7f9caf400640 1 Processor -- start 2024-03-29T08:19:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.316+0000 7f9caf400640 1 -- start start 2024-03-29T08:19:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9caf400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca8142a50 0x7f9ca80cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9caf400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9ca815df90 0x7f9ca80cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9caf400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca815d260 con 0x7f9ca80ae280 2024-03-29T08:19:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9caf400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca80aefd0 con 0x7f9ca815df90 2024-03-29T08:19:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9caf400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca815d4b0 con 0x7f9ca8142a50 2024-03-29T08:19:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9cad400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9ca80ae280 0x7f9ca80ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49504/0 (socket says 172.21.15.46:49504) 2024-03-29T08:19:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9cad400640 1 -- 172.21.15.46:0/2069443023 learned_addr learned my addr 172.21.15.46:0/2069443023 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:33.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9caca00640 1 --2- 172.21.15.46:0/2069443023 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca8142a50 0x7f9ca80cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9c9fe00640 1 --2- 172.21.15.46:0/2069443023 >> v2:172.21.15.73:3300/0 conn(0x7f9ca815df90 0x7f9ca80cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9ca7e00640 1 -- 172.21.15.46:0/2069443023 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683408553 0 0) 0x7f9ca815d260 con 0x7f9ca80ae280 2024-03-29T08:19:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9ca7e00640 1 -- 172.21.15.46:0/2069443023 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c8c005740 con 0x7f9ca80ae280 2024-03-29T08:19:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9ca7e00640 1 -- 172.21.15.46:0/2069443023 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1099256345 0 0) 0x7f9c8c005740 con 0x7f9ca80ae280 2024-03-29T08:19:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9ca7e00640 1 -- 172.21.15.46:0/2069443023 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca8142a50 msgr2=0x7f9ca80cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9ca7e00640 1 --2- 172.21.15.46:0/2069443023 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca8142a50 0x7f9ca80cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9ca7e00640 1 -- 172.21.15.46:0/2069443023 >> v2:172.21.15.73:3300/0 conn(0x7f9ca815df90 msgr2=0x7f9ca80cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9ca7e00640 1 --2- 172.21.15.46:0/2069443023 >> v2:172.21.15.73:3300/0 conn(0x7f9ca815df90 0x7f9ca80cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9ca7e00640 1 -- 172.21.15.46:0/2069443023 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca815d790 con 0x7f9ca80ae280 2024-03-29T08:19:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9ca7e00640 1 -- 172.21.15.46:0/2069443023 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f9ca0001a00 con 0x7f9ca80ae280 2024-03-29T08:19:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9ca7e00640 1 -- 172.21.15.46:0/2069443023 >> v1:172.21.15.46:6789/0 conn(0x7f9ca80ae280 legacy=0x7f9ca80ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9ca7e00640 1 --2- 172.21.15.46:0/2069443023 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca8142a50 0x7f9c8c005b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9ca7e00640 1 --2- 172.21.15.46:0/2069443023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c8c006110 0x7f9c8c0069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9ca7e00640 1 --2- 172.21.15.46:0/2069443023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c8c006f20 0x7f9c8c0073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9ca7e00640 1 -- 172.21.15.46:0/2069443023 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca80aefd0 con 0x7f9c8c006f20 2024-03-29T08:19:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9ca7e00640 1 -- 172.21.15.46:0/2069443023 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca815d4b0 con 0x7f9c8c006110 2024-03-29T08:19:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9ca7e00640 1 -- 172.21.15.46:0/2069443023 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c00dbf0 con 0x7f9ca8142a50 2024-03-29T08:19:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9cad400640 1 --2- 172.21.15.46:0/2069443023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c8c006f20 0x7f9c8c0073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9caca00640 1 --2- 172.21.15.46:0/2069443023 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca8142a50 0x7f9c8c005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9c9fe00640 1 --2- 172.21.15.46:0/2069443023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c8c006110 0x7f9c8c0069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9caca00640 1 -- 172.21.15.46:0/2069443023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c8c006110 msgr2=0x7f9c8c0069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9caca00640 1 --2- 172.21.15.46:0/2069443023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c8c006110 0x7f9c8c0069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9caca00640 1 -- 172.21.15.46:0/2069443023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c8c006f20 msgr2=0x7f9c8c0073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:33.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9caca00640 1 --2- 172.21.15.46:0/2069443023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c8c006f20 0x7f9c8c0073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9caca00640 1 -- 172.21.15.46:0/2069443023 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8c005740 con 0x7f9ca8142a50 2024-03-29T08:19:33.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.320+0000 7f9cad400640 1 --2- 172.21.15.46:0/2069443023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c8c006f20 0x7f9c8c0073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.324+0000 7f9caca00640 1 --2- 172.21.15.46:0/2069443023 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca8142a50 0x7f9c8c005b90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009b10 tx=0x7f9c9801cab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac3d37129ce320b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.324+0000 7f9ca7e00640 1 -- 172.21.15.46:0/2069443023 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c980a56a0 con 0x7f9ca8142a50 2024-03-29T08:19:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.324+0000 7f9ca7e00640 1 -- 172.21.15.46:0/2069443023 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c980a5800 con 0x7f9ca8142a50 2024-03-29T08:19:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.324+0000 7f9ca7e00640 1 -- 172.21.15.46:0/2069443023 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c980a5bb0 con 0x7f9ca8142a50 2024-03-29T08:19:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.324+0000 7f9caf400640 1 -- 172.21.15.46:0/2069443023 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca8142a50 msgr2=0x7f9c8c005b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.324+0000 7f9caf400640 1 --2- 172.21.15.46:0/2069443023 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca8142a50 0x7f9c8c005b90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009b10 tx=0x7f9c9801cab0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.324+0000 7f9cad400640 1 -- 172.21.15.46:0/2069443023 reap_dead start 2024-03-29T08:19:33.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.324+0000 7f9caf400640 1 -- 172.21.15.46:0/2069443023 shutdown_connections 2024-03-29T08:19:33.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.324+0000 7f9caf400640 1 -- 172.21.15.46:0/2069443023 >> 172.21.15.46:0/2069443023 conn(0x7f9ca806a390 msgr2=0x7f9ca80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:33.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.324+0000 7f9caf400640 1 -- 172.21.15.46:0/2069443023 shutdown_connections 2024-03-29T08:19:33.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.324+0000 7f9caf400640 1 -- 172.21.15.46:0/2069443023 wait complete. 2024-03-29T08:19:33.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9caf400640 1 Processor -- start 2024-03-29T08:19:33.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9caf400640 1 -- start start 2024-03-29T08:19:33.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9caf400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca8142a50 0x7f9ca813b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:33.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9caf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca8139220 0x7f9ca815c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:33.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9caf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ca8139680 0x7f9ca80acf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:33.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9caf400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca815c130 con 0x7f9ca8139220 2024-03-29T08:19:33.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9caf400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca815a0d0 con 0x7f9ca8139680 2024-03-29T08:19:33.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9caf400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca815bae0 con 0x7f9ca8142a50 2024-03-29T08:19:33.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9c9fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca8139220 0x7f9ca815c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:33.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9c9fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca8139220 0x7f9ca815c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48224/0 (socket says 172.21.15.46:48224) 2024-03-29T08:19:33.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9c9fe00640 1 -- 172.21.15.46:0/1866037247 learned_addr learned my addr 172.21.15.46:0/1866037247 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:33.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9cad400640 1 --2- 172.21.15.46:0/1866037247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ca8139680 0x7f9ca80acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:33.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9caca00640 1 --2- 172.21.15.46:0/1866037247 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca8142a50 0x7f9ca813b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:33.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9c9fe00640 1 -- 172.21.15.46:0/1866037247 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca8142a50 msgr2=0x7f9ca813b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:33.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9c9fe00640 1 --2- 172.21.15.46:0/1866037247 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca8142a50 0x7f9ca813b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9c9fe00640 1 -- 172.21.15.46:0/1866037247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ca8139680 msgr2=0x7f9ca80acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:33.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9c9fe00640 1 --2- 172.21.15.46:0/1866037247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ca8139680 0x7f9ca80acf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9c9fe00640 1 -- 172.21.15.46:0/1866037247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca815bdc0 con 0x7f9ca8139220 2024-03-29T08:19:33.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9cad400640 1 --2- 172.21.15.46:0/1866037247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ca8139680 0x7f9ca80acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:33.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9c9fe00640 1 --2- 172.21.15.46:0/1866037247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca8139220 0x7f9ca815c5c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9c900119d0 tx=0x7f9c90011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:33.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9ca5600640 1 -- 172.21.15.46:0/1866037247 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9000fb50 con 0x7f9ca8139220 2024-03-29T08:19:33.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9ca5600640 1 -- 172.21.15.46:0/1866037247 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9000fcb0 con 0x7f9ca8139220 2024-03-29T08:19:33.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9caf400640 1 -- 172.21.15.46:0/1866037247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca8152270 con 0x7f9ca8139220 2024-03-29T08:19:33.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9ca5600640 1 -- 172.21.15.46:0/1866037247 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c90014440 con 0x7f9ca8139220 2024-03-29T08:19:33.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9caf400640 1 -- 172.21.15.46:0/1866037247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca815cb00 con 0x7f9ca8139220 2024-03-29T08:19:33.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.328+0000 7f9caf400640 1 -- 172.21.15.46:0/1866037247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca815c2a0 con 0x7f9ca8139220 2024-03-29T08:19:33.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.332+0000 7f9ca5600640 1 -- 172.21.15.46:0/1866037247 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9c9001d400 con 0x7f9ca8139220 2024-03-29T08:19:33.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.332+0000 7f9ca5600640 1 --2- 172.21.15.46:0/1866037247 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c84066690 0x7f9c84068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:33.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.332+0000 7f9ca5600640 1 -- 172.21.15.46:0/1866037247 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c900849d0 con 0x7f9ca8139220 2024-03-29T08:19:33.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.332+0000 7f9caca00640 1 --2- 172.21.15.46:0/1866037247 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c84066690 0x7f9c84068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:33.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.332+0000 7f9caca00640 1 --2- 172.21.15.46:0/1866037247 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c84066690 0x7f9c84068b50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9c980af000 tx=0x7f9c980046d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:33.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.336+0000 7f9ca5600640 1 -- 172.21.15.46:0/1866037247 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9c90051d60 con 0x7f9ca8139220 2024-03-29T08:19:33.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.496+0000 7f9caf400640 1 -- 172.21.15.46:0/1866037247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ca815bf50 con 0x7f9ca8139220 2024-03-29T08:19:33.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.496+0000 7f9ca5600640 1 -- 172.21.15.46:0/1866037247 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9c90055380 con 0x7f9ca8139220 2024-03-29T08:19:33.499 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:19:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.500+0000 7f9caf400640 1 -- 172.21.15.46:0/1866037247 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c84066690 msgr2=0x7f9c84068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:33.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.500+0000 7f9caf400640 1 --2- 172.21.15.46:0/1866037247 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c84066690 0x7f9c84068b50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9c980af000 tx=0x7f9c980046d0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.500+0000 7f9caf400640 1 -- 172.21.15.46:0/1866037247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca8139220 msgr2=0x7f9ca815c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.500+0000 7f9caf400640 1 --2- 172.21.15.46:0/1866037247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca8139220 0x7f9ca815c5c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9c900119d0 tx=0x7f9c90011e90 comp rx=0 tx=0).stop 2024-03-29T08:19:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.500+0000 7f9caf400640 1 -- 172.21.15.46:0/1866037247 shutdown_connections 2024-03-29T08:19:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.500+0000 7f9caf400640 1 --2- 172.21.15.46:0/1866037247 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c84066690 0x7f9c84068b50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.500+0000 7f9caf400640 1 --2- 172.21.15.46:0/1866037247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ca8139680 0x7f9ca80acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.500+0000 7f9caf400640 1 --2- 172.21.15.46:0/1866037247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca8139220 0x7f9ca815c5c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.500+0000 7f9caf400640 1 --2- 172.21.15.46:0/1866037247 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca8142a50 0x7f9ca813b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.500+0000 7f9caf400640 1 -- 172.21.15.46:0/1866037247 >> 172.21.15.46:0/1866037247 conn(0x7f9ca806a390 msgr2=0x7f9ca806c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.500+0000 7f9caf400640 1 -- 172.21.15.46:0/1866037247 shutdown_connections 2024-03-29T08:19:33.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.500+0000 7f9caf400640 1 -- 172.21.15.46:0/1866037247 wait complete. 2024-03-29T08:19:33.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:19:33.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.660+0000 7ff260600640 1 Processor -- start 2024-03-29T08:19:33.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.660+0000 7ff260600640 1 -- start start 2024-03-29T08:19:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.664+0000 7ff260600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff2580ccfc0 0x7ff2580cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.664+0000 7ff260600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2580ce040 con 0x7ff25815a5d0 2024-03-29T08:19:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.664+0000 7ff260600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2580cda80 con 0x7ff25806ae80 2024-03-29T08:19:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.664+0000 7ff260600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2580b0ce0 con 0x7ff2580ccfc0 2024-03-29T08:19:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.664+0000 7ff25d200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff2580ccfc0 0x7ff2580cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.664+0000 7ff25dc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff25815a5d0 0x7ff2580b1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:49520/0 (socket says 172.21.15.46:49520) 2024-03-29T08:19:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.664+0000 7ff25dc00640 1 -- 172.21.15.46:0/1082817991 learned_addr learned my addr 172.21.15.46:0/1082817991 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.664+0000 7ff25d200640 1 -- 172.21.15.46:0/1082817991 >> v1:172.21.15.73:6789/0 conn(0x7ff25806ae80 legacy=0x7ff2580d0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.664+0000 7ff25d200640 1 -- 172.21.15.46:0/1082817991 >> v1:172.21.15.46:6789/0 conn(0x7ff25815a5d0 legacy=0x7ff2580b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:33.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.664+0000 7ff25d200640 1 -- 172.21.15.46:0/1082817991 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2580cdd60 con 0x7ff2580ccfc0 2024-03-29T08:19:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff25d200640 1 --2- 172.21.15.46:0/1082817991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2580ccfc0 0x7ff2580cd470 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff2440097f0 tx=0x7ff24401ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1058f6c8402bda2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff257e00640 1 -- 172.21.15.46:0/1082817991 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2440ab600 con 0x7ff2580ccfc0 2024-03-29T08:19:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff257e00640 1 -- 172.21.15.46:0/1082817991 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2440ab760 con 0x7ff2580ccfc0 2024-03-29T08:19:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff257e00640 1 -- 172.21.15.46:0/1082817991 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2440abaf0 con 0x7ff2580ccfc0 2024-03-29T08:19:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff260600640 1 -- 172.21.15.46:0/1082817991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2580ccfc0 msgr2=0x7ff2580cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff260600640 1 --2- 172.21.15.46:0/1082817991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2580ccfc0 0x7ff2580cd470 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff2440097f0 tx=0x7ff24401ce40 comp rx=0 tx=0).stop 2024-03-29T08:19:33.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff260600640 1 -- 172.21.15.46:0/1082817991 shutdown_connections 2024-03-29T08:19:33.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff260600640 1 --2- 172.21.15.46:0/1082817991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2580ccfc0 0x7ff2580cd470 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff260600640 1 -- 172.21.15.46:0/1082817991 >> 172.21.15.46:0/1082817991 conn(0x7ff258099a10 msgr2=0x7ff258089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:33.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff260600640 1 -- 172.21.15.46:0/1082817991 shutdown_connections 2024-03-29T08:19:33.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff260600640 1 -- 172.21.15.46:0/1082817991 wait complete. 2024-03-29T08:19:33.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff260600640 1 Processor -- start 2024-03-29T08:19:33.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff260600640 1 -- start start 2024-03-29T08:19:33.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff260600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff25806ae80 0x7ff25814c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:33.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff260600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2580ccfc0 0x7ff2580aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:33.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff260600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff25815a5d0 0x7ff2580af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:33.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff260600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2580b6170 con 0x7ff2580ccfc0 2024-03-29T08:19:33.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff260600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25814cc80 con 0x7ff25806ae80 2024-03-29T08:19:33.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.668+0000 7ff260600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2580b54f0 con 0x7ff25815a5d0 2024-03-29T08:19:33.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff25d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2580ccfc0 0x7ff2580aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:33.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff25d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2580ccfc0 0x7ff2580aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48236/0 (socket says 172.21.15.46:48236) 2024-03-29T08:19:33.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff25d200640 1 -- 172.21.15.46:0/360102282 learned_addr learned my addr 172.21.15.46:0/360102282 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:33.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff25dc00640 1 --2- 172.21.15.46:0/360102282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff25806ae80 0x7ff25814c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:33.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff25e600640 1 --2- 172.21.15.46:0/360102282 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff25815a5d0 0x7ff2580af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:33.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff25d200640 1 -- 172.21.15.46:0/360102282 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff25815a5d0 msgr2=0x7ff2580af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:33.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff25d200640 1 --2- 172.21.15.46:0/360102282 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff25815a5d0 0x7ff2580af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff25d200640 1 -- 172.21.15.46:0/360102282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff25806ae80 msgr2=0x7ff25814c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:33.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff25d200640 1 --2- 172.21.15.46:0/360102282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff25806ae80 0x7ff25814c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff25d200640 1 -- 172.21.15.46:0/360102282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff244009420 con 0x7ff2580ccfc0 2024-03-29T08:19:33.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff25d200640 1 --2- 172.21.15.46:0/360102282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2580ccfc0 0x7ff2580aeac0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff244018ff0 tx=0x7ff24401ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff255600640 1 -- 172.21.15.46:0/360102282 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2440b5810 con 0x7ff2580ccfc0 2024-03-29T08:19:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff255600640 1 -- 172.21.15.46:0/360102282 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2440b5990 con 0x7ff2580ccfc0 2024-03-29T08:19:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff255600640 1 -- 172.21.15.46:0/360102282 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2440b5d40 con 0x7ff2580ccfc0 2024-03-29T08:19:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff260600640 1 -- 172.21.15.46:0/360102282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2580b57d0 con 0x7ff2580ccfc0 2024-03-29T08:19:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff260600640 1 -- 172.21.15.46:0/360102282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2580b5d90 con 0x7ff2580ccfc0 2024-03-29T08:19:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff260600640 1 -- 172.21.15.46:0/360102282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25809e7d0 con 0x7ff2580ccfc0 2024-03-29T08:19:33.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.672+0000 7ff255600640 1 -- 172.21.15.46:0/360102282 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff2440b4390 con 0x7ff2580ccfc0 2024-03-29T08:19:33.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.676+0000 7ff255600640 1 --2- 172.21.15.46:0/360102282 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff238066690 0x7ff238068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:33.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.676+0000 7ff255600640 1 -- 172.21.15.46:0/360102282 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff24411ec20 con 0x7ff2580ccfc0 2024-03-29T08:19:33.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.676+0000 7ff25dc00640 1 --2- 172.21.15.46:0/360102282 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff238066690 0x7ff238068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:33.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.676+0000 7ff25dc00640 1 --2- 172.21.15.46:0/360102282 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff238066690 0x7ff238068b50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff24c005890 tx=0x7ff24c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:33.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.680+0000 7ff255600640 1 -- 172.21.15.46:0/360102282 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff2440ebef0 con 0x7ff2580ccfc0 2024-03-29T08:19:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.848+0000 7ff260600640 1 -- 172.21.15.46:0/360102282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff25804e120 con 0x7ff2580ccfc0 2024-03-29T08:19:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.852+0000 7ff255600640 1 -- 172.21.15.46:0/360102282 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff2440ef510 con 0x7ff2580ccfc0 2024-03-29T08:19:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:19:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.852+0000 7ff260600640 1 -- 172.21.15.46:0/360102282 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff238066690 msgr2=0x7ff238068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.852+0000 7ff260600640 1 --2- 172.21.15.46:0/360102282 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff238066690 0x7ff238068b50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff24c005890 tx=0x7ff24c009040 comp rx=0 tx=0).stop 2024-03-29T08:19:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.852+0000 7ff260600640 1 -- 172.21.15.46:0/360102282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2580ccfc0 msgr2=0x7ff2580aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.852+0000 7ff260600640 1 --2- 172.21.15.46:0/360102282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2580ccfc0 0x7ff2580aeac0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7ff244018ff0 tx=0x7ff24401ac20 comp rx=0 tx=0).stop 2024-03-29T08:19:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.852+0000 7ff260600640 1 -- 172.21.15.46:0/360102282 shutdown_connections 2024-03-29T08:19:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.852+0000 7ff260600640 1 --2- 172.21.15.46:0/360102282 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff238066690 0x7ff238068b50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.852+0000 7ff260600640 1 --2- 172.21.15.46:0/360102282 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff25815a5d0 0x7ff2580af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.852+0000 7ff260600640 1 --2- 172.21.15.46:0/360102282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2580ccfc0 0x7ff2580aeac0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.852+0000 7ff260600640 1 --2- 172.21.15.46:0/360102282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff25806ae80 0x7ff25814c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.852+0000 7ff260600640 1 -- 172.21.15.46:0/360102282 >> 172.21.15.46:0/360102282 conn(0x7ff258099a10 msgr2=0x7ff25806b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.852+0000 7ff260600640 1 -- 172.21.15.46:0/360102282 shutdown_connections 2024-03-29T08:19:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:33.852+0000 7ff260600640 1 -- 172.21.15.46:0/360102282 wait complete. 2024-03-29T08:19:33.873 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-03-29T08:19:33.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:33.873 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T08:19:33.892 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[36401]: fuse finished with error 0 and tester_r 0 2024-03-29T08:19:33.895 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:19:38.400 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:19:38.404 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:19:39.997 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:19:39.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:39.997 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:19:40.008 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:19:40.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.200+0000 7f1fc4600640 1 Processor -- start 2024-03-29T08:19:40.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.200+0000 7f1fc4600640 1 -- start start 2024-03-29T08:19:40.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.204+0000 7f1fc4600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1fbc1593d0 0x7f1fbc06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:40.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.204+0000 7f1fc4600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fbc15d230 0x7f1fbc15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:40.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.204+0000 7f1fc4600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fbc06d130 con 0x7f1fbc15dbd0 2024-03-29T08:19:40.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.204+0000 7f1fc4600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fbc15b8f0 con 0x7f1fbc1593d0 2024-03-29T08:19:40.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.204+0000 7f1fc4600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc15ad50 con 0x7f1fbc15d230 2024-03-29T08:19:40.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.204+0000 7f1fc2600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1fbc15dbd0 0x7f1fbc15bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39982/0 (socket says 172.21.15.46:39982) 2024-03-29T08:19:40.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.204+0000 7f1fc2600640 1 -- 172.21.15.46:0/2852056500 learned_addr learned my addr 172.21.15.46:0/2852056500 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:40.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.204+0000 7f1fc1c00640 1 --2- 172.21.15.46:0/2852056500 >> v2:172.21.15.73:3300/0 conn(0x7f1fbc1593d0 0x7f1fbc06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:40.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.204+0000 7f1fc1200640 1 --2- 172.21.15.46:0/2852056500 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fbc15d230 0x7f1fbc15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:40.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.204+0000 7f1fbbe00640 1 -- 172.21.15.46:0/2852056500 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038858312 0 0) 0x7f1fbc06d130 con 0x7f1fbc15dbd0 2024-03-29T08:19:40.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.204+0000 7f1fbbe00640 1 -- 172.21.15.46:0/2852056500 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fa4003530 con 0x7f1fbc15dbd0 2024-03-29T08:19:40.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.204+0000 7f1fc1c00640 1 -- 172.21.15.46:0/2852056500 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fbc15d230 msgr2=0x7f1fbc15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:40.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.204+0000 7f1fc1c00640 1 --2- 172.21.15.46:0/2852056500 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fbc15d230 0x7f1fbc15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.204+0000 7f1fc1c00640 1 -- 172.21.15.46:0/2852056500 >> v1:172.21.15.46:6789/0 conn(0x7f1fbc15dbd0 legacy=0x7f1fbc15bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:40.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.204+0000 7f1fc1c00640 1 -- 172.21.15.46:0/2852056500 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc15e060 con 0x7f1fbc1593d0 2024-03-29T08:19:40.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.208+0000 7f1fc1c00640 1 --2- 172.21.15.46:0/2852056500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fbc1593d0 0x7f1fbc06cbf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00097c0 tx=0x7f1fb001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c45cf144d2572535 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:40.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.208+0000 7f1fbbe00640 1 -- 172.21.15.46:0/2852056500 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb00a6040 con 0x7f1fbc1593d0 2024-03-29T08:19:40.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.208+0000 7f1fbbe00640 1 -- 172.21.15.46:0/2852056500 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb00aa600 con 0x7f1fbc1593d0 2024-03-29T08:19:40.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.208+0000 7f1fbbe00640 1 -- 172.21.15.46:0/2852056500 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb00aa990 con 0x7f1fbc1593d0 2024-03-29T08:19:40.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.208+0000 7f1fc4600640 1 -- 172.21.15.46:0/2852056500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fbc1593d0 msgr2=0x7f1fbc06cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:40.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.208+0000 7f1fc4600640 1 --2- 172.21.15.46:0/2852056500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fbc1593d0 0x7f1fbc06cbf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00097c0 tx=0x7f1fb001c4a0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.208+0000 7f1fc4600640 1 -- 172.21.15.46:0/2852056500 shutdown_connections 2024-03-29T08:19:40.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.208+0000 7f1fc4600640 1 --2- 172.21.15.46:0/2852056500 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fbc15d230 0x7f1fbc15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.208+0000 7f1fc4600640 1 --2- 172.21.15.46:0/2852056500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fbc1593d0 0x7f1fbc06cbf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.208+0000 7f1fc4600640 1 -- 172.21.15.46:0/2852056500 >> 172.21.15.46:0/2852056500 conn(0x7f1fbc066110 msgr2=0x7f1fbc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:40.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.208+0000 7f1fc4600640 1 -- 172.21.15.46:0/2852056500 shutdown_connections 2024-03-29T08:19:40.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.208+0000 7f1fc4600640 1 -- 172.21.15.46:0/2852056500 wait complete. 2024-03-29T08:19:40.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.208+0000 7f1fc4600640 1 Processor -- start 2024-03-29T08:19:40.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.208+0000 7f1fc4600640 1 -- start start 2024-03-29T08:19:40.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc4600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fbc1593d0 0x7f1fbc158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:40.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc4600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fbc15d230 0x7f1fbc14d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:40.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc4600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc15dbd0 0x7f1fbc06c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:40.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc4600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc150c50 con 0x7f1fbc15dbd0 2024-03-29T08:19:40.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc4600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc1517d0 con 0x7f1fbc15d230 2024-03-29T08:19:40.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc4600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fbc150ea0 con 0x7f1fbc1593d0 2024-03-29T08:19:40.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc15dbd0 0x7f1fbc06c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:40.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc15dbd0 0x7f1fbc06c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59870/0 (socket says 172.21.15.46:59870) 2024-03-29T08:19:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc2600640 1 -- 172.21.15.46:0/717523294 learned_addr learned my addr 172.21.15.46:0/717523294 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc1c00640 1 --2- 172.21.15.46:0/717523294 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fbc1593d0 0x7f1fbc158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc1200640 1 --2- 172.21.15.46:0/717523294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fbc15d230 0x7f1fbc14d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc2600640 1 -- 172.21.15.46:0/717523294 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fbc1593d0 msgr2=0x7f1fbc158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc2600640 1 --2- 172.21.15.46:0/717523294 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fbc1593d0 0x7f1fbc158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc2600640 1 -- 172.21.15.46:0/717523294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fbc15d230 msgr2=0x7f1fbc14d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc2600640 1 --2- 172.21.15.46:0/717523294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fbc15d230 0x7f1fbc14d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc2600640 1 -- 172.21.15.46:0/717523294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb0009420 con 0x7f1fbc15dbd0 2024-03-29T08:19:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc2600640 1 --2- 172.21.15.46:0/717523294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc15dbd0 0x7f1fbc06c1e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400cb40 tx=0x7f1fb400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fb9600640 1 -- 172.21.15.46:0/717523294 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb400aca0 con 0x7f1fbc15dbd0 2024-03-29T08:19:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fb9600640 1 -- 172.21.15.46:0/717523294 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb400ae00 con 0x7f1fbc15dbd0 2024-03-29T08:19:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc4600640 1 -- 172.21.15.46:0/717523294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fbc151180 con 0x7f1fbc15dbd0 2024-03-29T08:19:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fb9600640 1 -- 172.21.15.46:0/717523294 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4012620 con 0x7f1fbc15dbd0 2024-03-29T08:19:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc4600640 1 -- 172.21.15.46:0/717523294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fbc151c60 con 0x7f1fbc15dbd0 2024-03-29T08:19:40.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fc4600640 1 -- 172.21.15.46:0/717523294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fbc151940 con 0x7f1fbc15dbd0 2024-03-29T08:19:40.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.212+0000 7f1fb9600640 1 -- 172.21.15.46:0/717523294 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1fb4007070 con 0x7f1fbc15dbd0 2024-03-29T08:19:40.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.216+0000 7f1fb9600640 1 --2- 172.21.15.46:0/717523294 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f9c0666c0 0x7f1f9c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:40.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.216+0000 7f1fb9600640 1 -- 172.21.15.46:0/717523294 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fb4084190 con 0x7f1fbc15dbd0 2024-03-29T08:19:40.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.216+0000 7f1fc1c00640 1 --2- 172.21.15.46:0/717523294 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f9c0666c0 0x7f1f9c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:40.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.216+0000 7f1fc1c00640 1 --2- 172.21.15.46:0/717523294 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f9c0666c0 0x7f1f9c068b80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00ae000 tx=0x7f1fb00b5040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:40.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.220+0000 7f1fb9600640 1 -- 172.21.15.46:0/717523294 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1fb4051520 con 0x7f1fbc15dbd0 2024-03-29T08:19:40.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.420+0000 7f1fc4600640 1 -- 172.21.15.46:0/717523294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fbc066c90 con 0x7f1fbc15dbd0 2024-03-29T08:19:40.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.420+0000 7f1fb9600640 1 -- 172.21.15.46:0/717523294 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f1fb4054b40 con 0x7f1fbc15dbd0 2024-03-29T08:19:40.423 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:40.423 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/4274575126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4274575126},{"type":"v1","addr":"172.21.15.46:6834","nonce":4274575126}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3739899623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3739899623},{"type":"v1","addr":"172.21.15.73:6837","nonce":3739899623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1905427475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1905427475},{"type":"v1","addr":"172.21.15.46:6835","nonce":1905427475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:19:14.663142+0000","modified":"2024-03-29T08:19:17.680359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5152},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5152":{"gid":5152,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/1941408960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1941408960},{"type":"v1","addr":"172.21.15.73:6836","nonce":1941408960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-29T08:19:40.423 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2024-03-29T08:19:40.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.424+0000 7f1fc4600640 1 -- 172.21.15.46:0/717523294 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f9c0666c0 msgr2=0x7f1f9c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:40.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.424+0000 7f1fc4600640 1 --2- 172.21.15.46:0/717523294 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f9c0666c0 0x7f1f9c068b80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00ae000 tx=0x7f1fb00b5040 comp rx=0 tx=0).stop 2024-03-29T08:19:40.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.424+0000 7f1fc4600640 1 -- 172.21.15.46:0/717523294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc15dbd0 msgr2=0x7f1fbc06c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:40.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.424+0000 7f1fc4600640 1 --2- 172.21.15.46:0/717523294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc15dbd0 0x7f1fbc06c1e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400cb40 tx=0x7f1fb400a060 comp rx=0 tx=0).stop 2024-03-29T08:19:40.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.424+0000 7f1fc4600640 1 -- 172.21.15.46:0/717523294 shutdown_connections 2024-03-29T08:19:40.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.424+0000 7f1fc4600640 1 --2- 172.21.15.46:0/717523294 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f9c0666c0 0x7f1f9c068b80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.424+0000 7f1fc4600640 1 --2- 172.21.15.46:0/717523294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fbc15d230 0x7f1fbc14d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.424+0000 7f1fc4600640 1 --2- 172.21.15.46:0/717523294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fbc15dbd0 0x7f1fbc06c1e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.424+0000 7f1fc4600640 1 --2- 172.21.15.46:0/717523294 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fbc1593d0 0x7f1fbc158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.424+0000 7f1fc4600640 1 -- 172.21.15.46:0/717523294 >> 172.21.15.46:0/717523294 conn(0x7f1fbc066110 msgr2=0x7f1fbc0b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:40.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.424+0000 7f1fc4600640 1 -- 172.21.15.46:0/717523294 shutdown_connections 2024-03-29T08:19:40.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.424+0000 7f1fc4600640 1 -- 172.21.15.46:0/717523294 wait complete. 2024-03-29T08:19:40.443 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-03-29T08:19:40.616 INFO:teuthology.orchestra.run.smithi073.stdout:{"message":"","return_code":0} 2024-03-29T08:19:40.616 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-29T08:19:40.616 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:19:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.756+0000 7f4207a00640 1 Processor -- start 2024-03-29T08:19:40.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.756+0000 7f4207a00640 1 -- start start 2024-03-29T08:19:40.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.756+0000 7f4207a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f420015a940 0x7f42000afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.756+0000 7f4207a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f420015ad20 0x7f42000cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.756+0000 7f4207a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42000b00d0 con 0x7f42000b0540 2024-03-29T08:19:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.756+0000 7f4207a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42000b0ab0 con 0x7f420015ad20 2024-03-29T08:19:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.756+0000 7f4207a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42000b0d90 con 0x7f420015a940 2024-03-29T08:19:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.756+0000 7f41ffe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f420015ad20 0x7f42000cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.756+0000 7f4205a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f42000b0540 0x7f42000cd070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39996/0 (socket says 172.21.15.46:39996) 2024-03-29T08:19:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.756+0000 7f4205a00640 1 -- 172.21.15.46:0/3339272609 learned_addr learned my addr 172.21.15.46:0/3339272609 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:40.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.756+0000 7f4205000640 1 --2- 172.21.15.46:0/3339272609 >> v2:172.21.15.73:3300/0 conn(0x7f420015a940 0x7f42000afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.756+0000 7f41ffe00640 1 -- 172.21.15.46:0/3339272609 >> v2:172.21.15.73:3300/0 conn(0x7f420015a940 msgr2=0x7f42000afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.756+0000 7f41ffe00640 1 --2- 172.21.15.46:0/3339272609 >> v2:172.21.15.73:3300/0 conn(0x7f420015a940 0x7f42000afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.756+0000 7f41ffe00640 1 -- 172.21.15.46:0/3339272609 >> v1:172.21.15.46:6789/0 conn(0x7f42000b0540 legacy=0x7f42000cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.756+0000 7f41ffe00640 1 -- 172.21.15.46:0/3339272609 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42000b1070 con 0x7f420015ad20 2024-03-29T08:19:40.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.756+0000 7f41ff400640 1 -- 172.21.15.46:0/3339272609 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421771270 0 0) 0x7f42000b00d0 con 0x7f42000b0540 2024-03-29T08:19:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.760+0000 7f41ffe00640 1 --2- 172.21.15.46:0/3339272609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f420015ad20 0x7f42000cecb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4200070d60 tx=0x7f41ec01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b630d76fa86587 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.760+0000 7f41ff400640 1 -- 172.21.15.46:0/3339272609 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec0ac620 con 0x7f420015ad20 2024-03-29T08:19:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.760+0000 7f41ff400640 1 -- 172.21.15.46:0/3339272609 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ec0ac780 con 0x7f420015ad20 2024-03-29T08:19:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.760+0000 7f41ff400640 1 -- 172.21.15.46:0/3339272609 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec0acb10 con 0x7f420015ad20 2024-03-29T08:19:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.760+0000 7f4207a00640 1 -- 172.21.15.46:0/3339272609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f420015ad20 msgr2=0x7f42000cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.760+0000 7f4207a00640 1 --2- 172.21.15.46:0/3339272609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f420015ad20 0x7f42000cecb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4200070d60 tx=0x7f41ec01c3c0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.760+0000 7f4207a00640 1 -- 172.21.15.46:0/3339272609 shutdown_connections 2024-03-29T08:19:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.760+0000 7f4207a00640 1 --2- 172.21.15.46:0/3339272609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f420015ad20 0x7f42000cecb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.760+0000 7f4207a00640 1 --2- 172.21.15.46:0/3339272609 >> v2:172.21.15.73:3300/0 conn(0x7f420015a940 0x7f42000afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.760+0000 7f4207a00640 1 -- 172.21.15.46:0/3339272609 >> 172.21.15.46:0/3339272609 conn(0x7f4200089fd0 msgr2=0x7f420008a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.760+0000 7f4207a00640 1 -- 172.21.15.46:0/3339272609 shutdown_connections 2024-03-29T08:19:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.760+0000 7f4207a00640 1 -- 172.21.15.46:0/3339272609 wait complete. 2024-03-29T08:19:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4207a00640 1 Processor -- start 2024-03-29T08:19:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4207a00640 1 -- start start 2024-03-29T08:19:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4207a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42000b0540 0x7f42000ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4207a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f420015a940 0x7f4200159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4207a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f420015ad20 0x7f4200159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4207a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42000c6d30 con 0x7f42000b0540 2024-03-29T08:19:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4207a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42000ca6b0 con 0x7f420015ad20 2024-03-29T08:19:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4207a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4200163f60 con 0x7f420015a940 2024-03-29T08:19:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4205000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42000b0540 0x7f42000ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4205000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42000b0540 0x7f42000ca170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59882/0 (socket says 172.21.15.46:59882) 2024-03-29T08:19:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4205000640 1 -- 172.21.15.46:0/2810703198 learned_addr learned my addr 172.21.15.46:0/2810703198 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4205a00640 1 --2- 172.21.15.46:0/2810703198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f420015ad20 0x7f4200159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f41ffe00640 1 --2- 172.21.15.46:0/2810703198 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f420015a940 0x7f4200159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4205000640 1 -- 172.21.15.46:0/2810703198 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f420015a940 msgr2=0x7f4200159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4205000640 1 --2- 172.21.15.46:0/2810703198 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f420015a940 0x7f4200159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4205000640 1 -- 172.21.15.46:0/2810703198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f420015ad20 msgr2=0x7f4200159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4205000640 1 --2- 172.21.15.46:0/2810703198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f420015ad20 0x7f4200159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4205000640 1 -- 172.21.15.46:0/2810703198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec009420 con 0x7f42000b0540 2024-03-29T08:19:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4205000640 1 --2- 172.21.15.46:0/2810703198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42000b0540 0x7f42000ca170 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f41f400e960 tx=0x7f41f400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f41fcc00640 1 -- 172.21.15.46:0/2810703198 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f4009690 con 0x7f42000b0540 2024-03-29T08:19:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f41fcc00640 1 -- 172.21.15.46:0/2810703198 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41f400ca00 con 0x7f42000b0540 2024-03-29T08:19:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f41fcc00640 1 -- 172.21.15.46:0/2810703198 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41f400cd70 con 0x7f42000b0540 2024-03-29T08:19:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4207a00640 1 -- 172.21.15.46:0/2810703198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4200164240 con 0x7f42000b0540 2024-03-29T08:19:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4207a00640 1 -- 172.21.15.46:0/2810703198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420006aea0 con 0x7f42000b0540 2024-03-29T08:19:40.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.764+0000 7f4207a00640 1 -- 172.21.15.46:0/2810703198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4200066c90 con 0x7f42000b0540 2024-03-29T08:19:40.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.768+0000 7f41fcc00640 1 -- 172.21.15.46:0/2810703198 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f41f4013470 con 0x7f42000b0540 2024-03-29T08:19:40.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.768+0000 7f41fcc00640 1 --2- 172.21.15.46:0/2810703198 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f41e0066690 0x7f41e0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:40.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.768+0000 7f41fcc00640 1 -- 172.21.15.46:0/2810703198 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41f4084410 con 0x7f42000b0540 2024-03-29T08:19:40.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.768+0000 7f41ffe00640 1 --2- 172.21.15.46:0/2810703198 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f41e0066690 0x7f41e0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:40.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.768+0000 7f41ffe00640 1 --2- 172.21.15.46:0/2810703198 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f41e0066690 0x7f41e0068b50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0ae000 tx=0x7f41ec008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:40.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.772+0000 7f41fcc00640 1 -- 172.21.15.46:0/2810703198 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f41f4051770 con 0x7f42000b0540 2024-03-29T08:19:40.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.964+0000 7f4207a00640 1 -- 172.21.15.46:0/2810703198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42000cee50 con 0x7f42000b0540 2024-03-29T08:19:40.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.964+0000 7f41fcc00640 1 -- 172.21.15.46:0/2810703198 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f41f4054d90 con 0x7f42000b0540 2024-03-29T08:19:40.967 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:40.967 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/4274575126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4274575126},{"type":"v1","addr":"172.21.15.46:6834","nonce":4274575126}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3739899623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3739899623},{"type":"v1","addr":"172.21.15.73:6837","nonce":3739899623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1905427475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1905427475},{"type":"v1","addr":"172.21.15.46:6835","nonce":1905427475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:19:14.663142+0000","modified":"2024-03-29T08:19:17.680359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5152},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5152":{"gid":5152,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/1941408960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1941408960},{"type":"v1","addr":"172.21.15.73:6836","nonce":1941408960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-29T08:19:40.967 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2024-03-29T08:19:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.968+0000 7f4207a00640 1 -- 172.21.15.46:0/2810703198 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f41e0066690 msgr2=0x7f41e0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.968+0000 7f4207a00640 1 --2- 172.21.15.46:0/2810703198 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f41e0066690 0x7f41e0068b50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0ae000 tx=0x7f41ec008040 comp rx=0 tx=0).stop 2024-03-29T08:19:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.968+0000 7f4207a00640 1 -- 172.21.15.46:0/2810703198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42000b0540 msgr2=0x7f42000ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.968+0000 7f4207a00640 1 --2- 172.21.15.46:0/2810703198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42000b0540 0x7f42000ca170 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f41f400e960 tx=0x7f41f400ee20 comp rx=0 tx=0).stop 2024-03-29T08:19:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.968+0000 7f4207a00640 1 -- 172.21.15.46:0/2810703198 shutdown_connections 2024-03-29T08:19:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.968+0000 7f4207a00640 1 --2- 172.21.15.46:0/2810703198 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f41e0066690 0x7f41e0068b50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.968+0000 7f4207a00640 1 --2- 172.21.15.46:0/2810703198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f420015ad20 0x7f4200159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.968+0000 7f4207a00640 1 --2- 172.21.15.46:0/2810703198 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f420015a940 0x7f4200159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.968+0000 7f4207a00640 1 --2- 172.21.15.46:0/2810703198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42000b0540 0x7f42000ca170 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:40.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.968+0000 7f4207a00640 1 -- 172.21.15.46:0/2810703198 >> 172.21.15.46:0/2810703198 conn(0x7f4200089fd0 msgr2=0x7f42000aee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:40.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.968+0000 7f4207a00640 1 -- 172.21.15.46:0/2810703198 shutdown_connections 2024-03-29T08:19:40.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:40.968+0000 7f4207a00640 1 -- 172.21.15.46:0/2810703198 wait complete. 2024-03-29T08:19:40.989 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json tag path /parentdir mytag 2024-03-29T08:19:41.146 INFO:teuthology.orchestra.run.smithi073.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-03-29T08:19:41.146 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-03-29T08:19:51.148 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-03-29T08:19:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb2527f6080 1 Processor -- start 2024-03-29T08:19:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb2527f6080 1 -- start start 2024-03-29T08:19:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb2527f6080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e962eb8ca0 0x55e962f62240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb2527f6080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e962f43a30 0x55e962e8cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb2527f6080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e962f43e10 0x55e962e8d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb2527f6080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e962dd2960 con 0x55e962eb8ca0 2024-03-29T08:19:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb2527f6080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e962dd2600 con 0x55e962f43e10 2024-03-29T08:19:51.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb2527f6080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e962dc9690 con 0x55e962f43a30 2024-03-29T08:19:51.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb24fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e962eb8ca0 0x55e962f62240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb24fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e962eb8ca0 0x55e962f62240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45414/0 (socket says 172.21.15.46:45414) 2024-03-29T08:19:51.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb24fa00640 1 -- 172.21.15.46:0/433925832 learned_addr learned my addr 172.21.15.46:0/433925832 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:51.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb250400640 1 --2- 172.21.15.46:0/433925832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e962f43e10 0x55e962e8d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb24f000640 1 --2- 172.21.15.46:0/433925832 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e962f43a30 0x55e962e8cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb24fa00640 1 -- 172.21.15.46:0/433925832 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e962f43a30 msgr2=0x55e962e8cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:51.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb24fa00640 1 --2- 172.21.15.46:0/433925832 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e962f43a30 0x55e962e8cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb24fa00640 1 -- 172.21.15.46:0/433925832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e962f43e10 msgr2=0x55e962e8d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:51.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb24fa00640 1 --2- 172.21.15.46:0/433925832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e962f43e10 0x55e962e8d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.200+0000 7fb24fa00640 1 -- 172.21.15.46:0/433925832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb240005df0 con 0x55e962eb8ca0 2024-03-29T08:19:51.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb24fa00640 1 --2- 172.21.15.46:0/433925832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e962eb8ca0 0x55e962f62240 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb2400af040 tx=0x7fb240003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991e440d3f94fd02 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb24dc00640 1 -- 172.21.15.46:0/433925832 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2400a49e0 con 0x55e962eb8ca0 2024-03-29T08:19:51.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb24dc00640 1 -- 172.21.15.46:0/433925832 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2400a4b40 con 0x55e962eb8ca0 2024-03-29T08:19:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb24dc00640 1 -- 172.21.15.46:0/433925832 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2400b6440 con 0x55e962eb8ca0 2024-03-29T08:19:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb2527f6080 1 -- 172.21.15.46:0/433925832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e962eb8ca0 msgr2=0x55e962f62240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb2527f6080 1 --2- 172.21.15.46:0/433925832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e962eb8ca0 0x55e962f62240 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb2400af040 tx=0x7fb240003030 comp rx=0 tx=0).stop 2024-03-29T08:19:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb2527f6080 1 -- 172.21.15.46:0/433925832 shutdown_connections 2024-03-29T08:19:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb2527f6080 1 --2- 172.21.15.46:0/433925832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e962f43e10 0x55e962e8d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb2527f6080 1 --2- 172.21.15.46:0/433925832 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e962f43a30 0x55e962e8cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb2527f6080 1 --2- 172.21.15.46:0/433925832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e962eb8ca0 0x55e962f62240 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb2527f6080 1 -- 172.21.15.46:0/433925832 >> 172.21.15.46:0/433925832 conn(0x55e962eb5910 msgr2=0x55e962e8dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb2527f6080 1 -- 172.21.15.46:0/433925832 shutdown_connections 2024-03-29T08:19:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb2527f6080 1 -- 172.21.15.46:0/433925832 wait complete. 2024-03-29T08:19:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb2527f6080 1 Processor -- start 2024-03-29T08:19:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb2527f6080 1 -- start start 2024-03-29T08:19:51.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb2527f6080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e962eb8ca0 0x55e962e8cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb2527f6080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e962f43a30 0x55e962ff9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb2527f6080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e962f43e10 0x55e962ebb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb2527f6080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e962e8db80 con 0x55e962f43a30 2024-03-29T08:19:51.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb2527f6080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e962e8d3a0 con 0x55e962eb8ca0 2024-03-29T08:19:51.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb2527f6080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e962e8d640 con 0x55e962f43e10 2024-03-29T08:19:51.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb24f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e962f43a30 0x55e962ff9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb24f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e962f43a30 0x55e962ff9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45426/0 (socket says 172.21.15.46:45426) 2024-03-29T08:19:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb24f000640 1 -- 172.21.15.46:0/1734888916 learned_addr learned my addr 172.21.15.46:0/1734888916 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb24fa00640 1 --2- 172.21.15.46:0/1734888916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e962eb8ca0 0x55e962e8cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb250400640 1 --2- 172.21.15.46:0/1734888916 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e962f43e10 0x55e962ebb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb24f000640 1 -- 172.21.15.46:0/1734888916 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e962f43e10 msgr2=0x55e962ebb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb24f000640 1 --2- 172.21.15.46:0/1734888916 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e962f43e10 0x55e962ebb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb24f000640 1 -- 172.21.15.46:0/1734888916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e962eb8ca0 msgr2=0x55e962e8cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb24f000640 1 --2- 172.21.15.46:0/1734888916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e962eb8ca0 0x55e962e8cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb24f000640 1 -- 172.21.15.46:0/1734888916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e962dfba60 con 0x55e962f43a30 2024-03-29T08:19:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.204+0000 7fb24fa00640 1 --2- 172.21.15.46:0/1734888916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e962eb8ca0 0x55e962e8cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.208+0000 7fb24f000640 1 --2- 172.21.15.46:0/1734888916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e962f43a30 0x55e962ff9430 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb23c008720 tx=0x7fb23c008be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.208+0000 7fb247e00640 1 -- 172.21.15.46:0/1734888916 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c00f040 con 0x55e962f43a30 2024-03-29T08:19:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.208+0000 7fb247e00640 1 -- 172.21.15.46:0/1734888916 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23c010ba0 con 0x55e962f43a30 2024-03-29T08:19:51.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.208+0000 7fb2527f6080 1 -- 172.21.15.46:0/1734888916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e962de18c0 con 0x55e962f43a30 2024-03-29T08:19:51.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.208+0000 7fb247e00640 1 -- 172.21.15.46:0/1734888916 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23c008e30 con 0x55e962f43a30 2024-03-29T08:19:51.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.208+0000 7fb2527f6080 1 -- 172.21.15.46:0/1734888916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e962dd4d00 con 0x55e962f43a30 2024-03-29T08:19:51.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.208+0000 7fb247e00640 1 -- 172.21.15.46:0/1734888916 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb23c015450 con 0x55e962f43a30 2024-03-29T08:19:51.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.208+0000 7fb247e00640 1 --2- 172.21.15.46:0/1734888916 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb230065070 0x7fb230067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.208+0000 7fb247e00640 1 -- 172.21.15.46:0/1734888916 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb23c083990 con 0x55e962f43a30 2024-03-29T08:19:51.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.208+0000 7fb2527f6080 1 --2- 172.21.15.46:0/1734888916 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x55e962f435f0 0x55e962f638c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.208+0000 7fb24fa00640 1 --2- 172.21.15.46:0/1734888916 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb230065070 0x7fb230067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.208+0000 7fb2527f6080 1 -- 172.21.15.46:0/1734888916 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) v8 -- 0x55e962eccc40 con 0x55e962f435f0 2024-03-29T08:19:51.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.212+0000 7fb250400640 1 --2- 172.21.15.46:0/1734888916 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x55e962f435f0 0x55e962f638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.212+0000 7fb24fa00640 1 --2- 172.21.15.46:0/1734888916 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb230065070 0x7fb230067530 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb240018460 tx=0x7fb2400a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.212+0000 7fb250400640 1 --2- 172.21.15.46:0/1734888916 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x55e962f435f0 0x55e962f638c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.212+0000 7fb250400640 1 -- 172.21.15.46:0/1734888916 <== osd.7 v2:172.21.15.73:6816/27700 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fb248002630 con 0x55e962f435f0 2024-03-29T08:19:51.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.212+0000 7fb2527f6080 1 -- 172.21.15.46:0/1734888916 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x55e962f435f0 msgr2=0x55e962f638c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.212+0000 7fb2527f6080 1 --2- 172.21.15.46:0/1734888916 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x55e962f435f0 0x55e962f638c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.212+0000 7fb2527f6080 1 -- 172.21.15.46:0/1734888916 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb230065070 msgr2=0x7fb230067530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.212+0000 7fb2527f6080 1 --2- 172.21.15.46:0/1734888916 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb230065070 0x7fb230067530 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb240018460 tx=0x7fb2400a6000 comp rx=0 tx=0).stop 2024-03-29T08:19:51.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.212+0000 7fb2527f6080 1 -- 172.21.15.46:0/1734888916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e962f43a30 msgr2=0x55e962ff9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.212+0000 7fb2527f6080 1 --2- 172.21.15.46:0/1734888916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e962f43a30 0x55e962ff9430 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb23c008720 tx=0x7fb23c008be0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.212+0000 7fb250400640 1 -- 172.21.15.46:0/1734888916 reap_dead start 2024-03-29T08:19:51.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.212+0000 7fb2527f6080 1 -- 172.21.15.46:0/1734888916 shutdown_connections 2024-03-29T08:19:51.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.212+0000 7fb2527f6080 1 -- 172.21.15.46:0/1734888916 >> 172.21.15.46:0/1734888916 conn(0x55e962eb5910 msgr2=0x55e962f62240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:51.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.212+0000 7fb2527f6080 1 -- 172.21.15.46:0/1734888916 shutdown_connections 2024-03-29T08:19:51.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.212+0000 7fb2527f6080 1 -- 172.21.15.46:0/1734888916 wait complete. 2024-03-29T08:19:51.218 INFO:teuthology.orchestra.run.smithi046.stdout:mytag 2024-03-29T08:19:51.218 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-03-29T08:19:51.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.264+0000 7fab06649080 1 Processor -- start 2024-03-29T08:19:51.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.264+0000 7fab06649080 1 -- start start 2024-03-29T08:19:51.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.264+0000 7fab06649080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cc6ac2270 0x561cc6ac1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.264+0000 7fab06649080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cc6ac4eb0 0x561cc6ac1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.264+0000 7fab06649080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561cc6adf0d0 0x561cc6a97050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.264+0000 7fab06649080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc69dd960 con 0x561cc6ac2270 2024-03-29T08:19:51.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.264+0000 7fab06649080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc69dd600 con 0x561cc6ac4eb0 2024-03-29T08:19:51.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.264+0000 7fab06649080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cc69d4690 con 0x561cc6adf0d0 2024-03-29T08:19:51.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.264+0000 7fab02c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cc6ac2270 0x561cc6ac1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab02c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cc6ac2270 0x561cc6ac1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45430/0 (socket says 172.21.15.46:45430) 2024-03-29T08:19:51.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab02c00640 1 -- 172.21.15.46:0/253712828 learned_addr learned my addr 172.21.15.46:0/253712828 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:51.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab02200640 1 --2- 172.21.15.46:0/253712828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cc6ac4eb0 0x561cc6ac1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab03600640 1 --2- 172.21.15.46:0/253712828 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561cc6adf0d0 0x561cc6a97050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab02c00640 1 -- 172.21.15.46:0/253712828 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561cc6adf0d0 msgr2=0x561cc6a97050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:51.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab02c00640 1 --2- 172.21.15.46:0/253712828 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561cc6adf0d0 0x561cc6a97050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab02c00640 1 -- 172.21.15.46:0/253712828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cc6ac4eb0 msgr2=0x561cc6ac1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:51.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab02c00640 1 --2- 172.21.15.46:0/253712828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cc6ac4eb0 0x561cc6ac1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab02c00640 1 -- 172.21.15.46:0/253712828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf0005d30 con 0x561cc6ac2270 2024-03-29T08:19:51.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab02c00640 1 --2- 172.21.15.46:0/253712828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cc6ac2270 0x561cc6ac1580 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7faaf40097d0 tx=0x7faaf4005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0239d2883cc0dc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab00e00640 1 -- 172.21.15.46:0/253712828 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf400f040 con 0x561cc6ac2270 2024-03-29T08:19:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab00e00640 1 -- 172.21.15.46:0/253712828 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf4008d40 con 0x561cc6ac2270 2024-03-29T08:19:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab00e00640 1 -- 172.21.15.46:0/253712828 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf4005e20 con 0x561cc6ac2270 2024-03-29T08:19:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab06649080 1 -- 172.21.15.46:0/253712828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cc6ac2270 msgr2=0x561cc6ac1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab06649080 1 --2- 172.21.15.46:0/253712828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cc6ac2270 0x561cc6ac1580 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7faaf40097d0 tx=0x7faaf4005a10 comp rx=0 tx=0).stop 2024-03-29T08:19:51.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab06649080 1 -- 172.21.15.46:0/253712828 shutdown_connections 2024-03-29T08:19:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab06649080 1 --2- 172.21.15.46:0/253712828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cc6ac4eb0 0x561cc6ac1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab06649080 1 --2- 172.21.15.46:0/253712828 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561cc6adf0d0 0x561cc6a97050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab06649080 1 --2- 172.21.15.46:0/253712828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cc6ac2270 0x561cc6ac1580 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab06649080 1 -- 172.21.15.46:0/253712828 >> 172.21.15.46:0/253712828 conn(0x561cc6ac0910 msgr2=0x561cc6ac64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab06649080 1 -- 172.21.15.46:0/253712828 shutdown_connections 2024-03-29T08:19:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab06649080 1 -- 172.21.15.46:0/253712828 wait complete. 2024-03-29T08:19:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab06649080 1 Processor -- start 2024-03-29T08:19:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab06649080 1 -- start start 2024-03-29T08:19:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab06649080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561cc6ac2270 0x561cc6ac6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab06649080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cc6ac4eb0 0x561cc6adba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab06649080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cc6adf0d0 0x561cc6b66930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab06649080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc6c06090 con 0x561cc6adf0d0 2024-03-29T08:19:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab06649080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cc6adc230 con 0x561cc6ac4eb0 2024-03-29T08:19:51.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab06649080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x561cc6adc4d0 con 0x561cc6ac2270 2024-03-29T08:19:51.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab02c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561cc6ac2270 0x561cc6ac6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab02c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561cc6ac2270 0x561cc6ac6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:38116/0 (socket says 172.21.15.46:38116) 2024-03-29T08:19:51.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab03600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cc6adf0d0 0x561cc6b66930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab02c00640 1 -- 172.21.15.46:0/2169757179 learned_addr learned my addr 172.21.15.46:0/2169757179 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:51.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.268+0000 7fab02200640 1 --2- 172.21.15.46:0/2169757179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cc6ac4eb0 0x561cc6adba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.272+0000 7fab02c00640 1 -- 172.21.15.46:0/2169757179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cc6ac4eb0 msgr2=0x561cc6adba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.272+0000 7fab02c00640 1 --2- 172.21.15.46:0/2169757179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cc6ac4eb0 0x561cc6adba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.272+0000 7fab02c00640 1 -- 172.21.15.46:0/2169757179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cc6adf0d0 msgr2=0x561cc6b66930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.272+0000 7fab02c00640 1 --2- 172.21.15.46:0/2169757179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cc6adf0d0 0x561cc6b66930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.272+0000 7fab02c00640 1 -- 172.21.15.46:0/2169757179 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf4009420 con 0x561cc6ac2270 2024-03-29T08:19:51.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.272+0000 7fab02200640 1 --2- 172.21.15.46:0/2169757179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561cc6ac4eb0 0x561cc6adba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.272+0000 7fab03600640 1 --2- 172.21.15.46:0/2169757179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561cc6adf0d0 0x561cc6b66930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.272+0000 7fab02c00640 1 --2- 172.21.15.46:0/2169757179 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561cc6ac2270 0x561cc6ac6660 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faaf4008780 tx=0x7faaf4003030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.272+0000 7faafb400640 1 -- 172.21.15.46:0/2169757179 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf400f070 con 0x561cc6ac2270 2024-03-29T08:19:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.272+0000 7faafb400640 1 -- 172.21.15.46:0/2169757179 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaf40056d0 con 0x561cc6ac2270 2024-03-29T08:19:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.272+0000 7faafb400640 1 -- 172.21.15.46:0/2169757179 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaf401e410 con 0x561cc6ac2270 2024-03-29T08:19:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.272+0000 7fab06649080 1 -- 172.21.15.46:0/2169757179 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561cc69ec8c0 con 0x561cc6ac2270 2024-03-29T08:19:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.272+0000 7fab06649080 1 -- 172.21.15.46:0/2169757179 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561cc69dfd00 con 0x561cc6ac2270 2024-03-29T08:19:51.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.272+0000 7faafb400640 1 -- 172.21.15.46:0/2169757179 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7faaf401b050 con 0x561cc6ac2270 2024-03-29T08:19:51.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.272+0000 7faafb400640 1 --2- 172.21.15.46:0/2169757179 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faae4065070 0x7faae4067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.272+0000 7faafb400640 1 -- 172.21.15.46:0/2169757179 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faaf4084a80 con 0x561cc6ac2270 2024-03-29T08:19:51.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab02200640 1 --2- 172.21.15.46:0/2169757179 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faae4065070 0x7faae4067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab06649080 1 --2- 172.21.15.46:0/2169757179 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x561cc6c064c0 0x561cc6c068a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab06649080 1 -- 172.21.15.46:0/2169757179 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:1 4.7 4:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) v8 -- 0x561cc6c06de0 con 0x561cc6c064c0 2024-03-29T08:19:51.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab03600640 1 --2- 172.21.15.46:0/2169757179 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x561cc6c064c0 0x561cc6c068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab02200640 1 --2- 172.21.15.46:0/2169757179 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faae4065070 0x7faae4067530 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faaf00ae000 tx=0x7faaf00a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab03600640 1 --2- 172.21.15.46:0/2169757179 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x561cc6c064c0 0x561cc6c068a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab03600640 1 -- 172.21.15.46:0/2169757179 <== osd.6 v2:172.21.15.73:6824/27691 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7faafc005020 con 0x561cc6c064c0 2024-03-29T08:19:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab06649080 1 -- 172.21.15.46:0/2169757179 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x561cc6c064c0 msgr2=0x561cc6c068a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab06649080 1 --2- 172.21.15.46:0/2169757179 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x561cc6c064c0 0x561cc6c068a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab06649080 1 -- 172.21.15.46:0/2169757179 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faae4065070 msgr2=0x7faae4067530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab06649080 1 --2- 172.21.15.46:0/2169757179 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faae4065070 0x7faae4067530 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faaf00ae000 tx=0x7faaf00a6000 comp rx=0 tx=0).stop 2024-03-29T08:19:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab06649080 1 -- 172.21.15.46:0/2169757179 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561cc6ac2270 msgr2=0x561cc6ac6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab06649080 1 --2- 172.21.15.46:0/2169757179 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561cc6ac2270 0x561cc6ac6660 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faaf4008780 tx=0x7faaf4003030 comp rx=0 tx=0).stop 2024-03-29T08:19:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab03600640 1 -- 172.21.15.46:0/2169757179 reap_dead start 2024-03-29T08:19:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab06649080 1 -- 172.21.15.46:0/2169757179 shutdown_connections 2024-03-29T08:19:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab06649080 1 -- 172.21.15.46:0/2169757179 >> 172.21.15.46:0/2169757179 conn(0x561cc6ac0910 msgr2=0x561cc6a978e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:51.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab06649080 1 -- 172.21.15.46:0/2169757179 shutdown_connections 2024-03-29T08:19:51.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.276+0000 7fab06649080 1 -- 172.21.15.46:0/2169757179 wait complete. 2024-03-29T08:19:51.282 INFO:teuthology.orchestra.run.smithi046.stdout:mytag 2024-03-29T08:19:51.282 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-03-29T08:19:51.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d67ce2080 1 Processor -- start 2024-03-29T08:19:51.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d67ce2080 1 -- start start 2024-03-29T08:19:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d67ce2080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558c933e8ca0 0x558c93490510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d67ce2080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c93473a30 0x558c93492240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d67ce2080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c93473e10 0x558c93402550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d67ce2080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c93302960 con 0x558c93473a30 2024-03-29T08:19:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d67ce2080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c93302600 con 0x558c93473e10 2024-03-29T08:19:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d67ce2080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c932f9690 con 0x558c933e8ca0 2024-03-29T08:19:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d5fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c93473a30 0x558c93492240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d5fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c93473a30 0x558c93492240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45464/0 (socket says 172.21.15.46:45464) 2024-03-29T08:19:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d5fe00640 1 -- 172.21.15.46:0/790551072 learned_addr learned my addr 172.21.15.46:0/790551072 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d65000640 1 --2- 172.21.15.46:0/790551072 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558c933e8ca0 0x558c93490510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d65a00640 1 --2- 172.21.15.46:0/790551072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c93473e10 0x558c93402550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d5fe00640 1 -- 172.21.15.46:0/790551072 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558c933e8ca0 msgr2=0x558c93490510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d5fe00640 1 --2- 172.21.15.46:0/790551072 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558c933e8ca0 0x558c93490510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d5fe00640 1 -- 172.21.15.46:0/790551072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c93473e10 msgr2=0x558c93402550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d5fe00640 1 --2- 172.21.15.46:0/790551072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c93473e10 0x558c93402550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.368+0000 7f7d5fe00640 1 -- 172.21.15.46:0/790551072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d58005df0 con 0x558c93473a30 2024-03-29T08:19:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d5fe00640 1 --2- 172.21.15.46:0/790551072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c93473a30 0x558c93492240 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009950 tx=0x7f7d50005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ca682e27d7b82 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d5ea00640 1 -- 172.21.15.46:0/790551072 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5000f040 con 0x558c93473a30 2024-03-29T08:19:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d5ea00640 1 -- 172.21.15.46:0/790551072 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d50008da0 con 0x558c93473a30 2024-03-29T08:19:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d5ea00640 1 -- 172.21.15.46:0/790551072 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50013640 con 0x558c93473a30 2024-03-29T08:19:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d67ce2080 1 -- 172.21.15.46:0/790551072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c93473a30 msgr2=0x558c93492240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d67ce2080 1 --2- 172.21.15.46:0/790551072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c93473a30 0x558c93492240 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009950 tx=0x7f7d50005a10 comp rx=0 tx=0).stop 2024-03-29T08:19:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d67ce2080 1 -- 172.21.15.46:0/790551072 shutdown_connections 2024-03-29T08:19:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d67ce2080 1 --2- 172.21.15.46:0/790551072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c93473e10 0x558c93402550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d67ce2080 1 --2- 172.21.15.46:0/790551072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c93473a30 0x558c93492240 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d67ce2080 1 --2- 172.21.15.46:0/790551072 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558c933e8ca0 0x558c93490510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d67ce2080 1 -- 172.21.15.46:0/790551072 >> 172.21.15.46:0/790551072 conn(0x558c933e5910 msgr2=0x558c933bdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d67ce2080 1 -- 172.21.15.46:0/790551072 shutdown_connections 2024-03-29T08:19:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d67ce2080 1 -- 172.21.15.46:0/790551072 wait complete. 2024-03-29T08:19:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d67ce2080 1 Processor -- start 2024-03-29T08:19:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d67ce2080 1 -- start start 2024-03-29T08:19:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d67ce2080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c933e8ca0 0x558c9352e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d67ce2080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c93473a30 0x558c9348c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d67ce2080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558c93473e10 0x558c93493560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d67ce2080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9348d310 con 0x558c93473a30 2024-03-29T08:19:51.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d67ce2080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9348d5b0 con 0x558c933e8ca0 2024-03-29T08:19:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d67ce2080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c9348d850 con 0x558c93473e10 2024-03-29T08:19:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d5fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c93473a30 0x558c9348c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d5fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c93473a30 0x558c9348c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45472/0 (socket says 172.21.15.46:45472) 2024-03-29T08:19:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d5fe00640 1 -- 172.21.15.46:0/437931113 learned_addr learned my addr 172.21.15.46:0/437931113 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d65a00640 1 --2- 172.21.15.46:0/437931113 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558c93473e10 0x558c93493560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d65000640 1 --2- 172.21.15.46:0/437931113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c933e8ca0 0x558c9352e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d5fe00640 1 -- 172.21.15.46:0/437931113 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558c93473e10 msgr2=0x558c93493560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d5fe00640 1 --2- 172.21.15.46:0/437931113 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558c93473e10 0x558c93493560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d5fe00640 1 -- 172.21.15.46:0/437931113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c933e8ca0 msgr2=0x558c9352e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d5fe00640 1 --2- 172.21.15.46:0/437931113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558c933e8ca0 0x558c9352e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d5fe00640 1 -- 172.21.15.46:0/437931113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d50009520 con 0x558c93473a30 2024-03-29T08:19:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.372+0000 7f7d5fe00640 1 --2- 172.21.15.46:0/437931113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c93473a30 0x558c9348c840 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7d50008780 tx=0x7f7d50003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.376+0000 7f7d5d600640 1 -- 172.21.15.46:0/437931113 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5000f050 con 0x558c93473a30 2024-03-29T08:19:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.376+0000 7f7d5d600640 1 -- 172.21.15.46:0/437931113 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d50013d90 con 0x558c93473a30 2024-03-29T08:19:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.376+0000 7f7d5d600640 1 -- 172.21.15.46:0/437931113 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d50025690 con 0x558c93473a30 2024-03-29T08:19:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.376+0000 7f7d67ce2080 1 -- 172.21.15.46:0/437931113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558c933118c0 con 0x558c93473a30 2024-03-29T08:19:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.376+0000 7f7d67ce2080 1 -- 172.21.15.46:0/437931113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c93304d00 con 0x558c93473a30 2024-03-29T08:19:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.376+0000 7f7d5d600640 1 -- 172.21.15.46:0/437931113 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7d50022070 con 0x558c93473a30 2024-03-29T08:19:51.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.376+0000 7f7d5d600640 1 --2- 172.21.15.46:0/437931113 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7d440650a0 0x7f7d44067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.376+0000 7f7d5d600640 1 -- 172.21.15.46:0/437931113 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d50084a10 con 0x558c93473a30 2024-03-29T08:19:51.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.376+0000 7f7d65000640 1 --2- 172.21.15.46:0/437931113 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7d440650a0 0x7f7d44067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.376+0000 7f7d67ce2080 1 --2- 172.21.15.46:0/437931113 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x558c9352e580 0x558c9352b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.376+0000 7f7d67ce2080 1 -- 172.21.15.46:0/437931113 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:1 5.36 5:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) v8 -- 0x558c9352bcd0 con 0x558c9352e580 2024-03-29T08:19:51.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.376+0000 7f7d65a00640 1 --2- 172.21.15.46:0/437931113 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x558c9352e580 0x558c9352b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.376+0000 7f7d65000640 1 --2- 172.21.15.46:0/437931113 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7d440650a0 0x7f7d44067560 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7d580ae000 tx=0x7f7d580a6070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.376+0000 7f7d65a00640 1 --2- 172.21.15.46:0/437931113 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x558c9352e580 0x558c9352b790 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.380+0000 7f7d65a00640 1 -- 172.21.15.46:0/437931113 <== osd.4 v2:172.21.15.73:6800/27689 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f7d60006020 con 0x558c9352e580 2024-03-29T08:19:51.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.380+0000 7f7d67ce2080 1 -- 172.21.15.46:0/437931113 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x558c9352e580 msgr2=0x558c9352b790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.380+0000 7f7d67ce2080 1 --2- 172.21.15.46:0/437931113 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x558c9352e580 0x558c9352b790 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.380+0000 7f7d67ce2080 1 -- 172.21.15.46:0/437931113 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7d440650a0 msgr2=0x7f7d44067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.380+0000 7f7d67ce2080 1 --2- 172.21.15.46:0/437931113 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7d440650a0 0x7f7d44067560 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7d580ae000 tx=0x7f7d580a6070 comp rx=0 tx=0).stop 2024-03-29T08:19:51.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.380+0000 7f7d67ce2080 1 -- 172.21.15.46:0/437931113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c93473a30 msgr2=0x558c9348c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.380+0000 7f7d67ce2080 1 --2- 172.21.15.46:0/437931113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558c93473a30 0x558c9348c840 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7d50008780 tx=0x7f7d50003030 comp rx=0 tx=0).stop 2024-03-29T08:19:51.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.380+0000 7f7d65a00640 1 -- 172.21.15.46:0/437931113 reap_dead start 2024-03-29T08:19:51.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.380+0000 7f7d67ce2080 1 -- 172.21.15.46:0/437931113 shutdown_connections 2024-03-29T08:19:51.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.380+0000 7f7d67ce2080 1 -- 172.21.15.46:0/437931113 >> 172.21.15.46:0/437931113 conn(0x558c933e5910 msgr2=0x558c93492240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:51.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.380+0000 7f7d67ce2080 1 -- 172.21.15.46:0/437931113 shutdown_connections 2024-03-29T08:19:51.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.380+0000 7f7d67ce2080 1 -- 172.21.15.46:0/437931113 wait complete. 2024-03-29T08:19:51.384 INFO:teuthology.orchestra.run.smithi046.stdout:mytag 2024-03-29T08:19:51.384 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-03-29T08:19:51.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.428+0000 7ff672f38080 1 Processor -- start 2024-03-29T08:19:51.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.428+0000 7ff672f38080 1 -- start start 2024-03-29T08:19:51.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558351f5f480 0x558351fe78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558351fed900 0x558351f15f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558351ff0340 0x558351fe7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558351e5d960 con 0x558351f5f480 2024-03-29T08:19:51.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558351e5d600 con 0x558351fed900 2024-03-29T08:19:51.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x558351e54690 con 0x558351ff0340 2024-03-29T08:19:51.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff670200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558351f5f480 0x558351fe78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff670200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558351f5f480 0x558351fe78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45478/0 (socket says 172.21.15.46:45478) 2024-03-29T08:19:51.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff670200640 1 -- 172.21.15.46:0/2607068640 learned_addr learned my addr 172.21.15.46:0/2607068640 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:51.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff670c00640 1 --2- 172.21.15.46:0/2607068640 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558351ff0340 0x558351fe7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff66f800640 1 --2- 172.21.15.46:0/2607068640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558351fed900 0x558351f15f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff670200640 1 -- 172.21.15.46:0/2607068640 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558351ff0340 msgr2=0x558351fe7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:51.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff670200640 1 --2- 172.21.15.46:0/2607068640 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558351ff0340 0x558351fe7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff670200640 1 -- 172.21.15.46:0/2607068640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558351fed900 msgr2=0x558351f15f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:51.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff670200640 1 --2- 172.21.15.46:0/2607068640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558351fed900 0x558351f15f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff670200640 1 -- 172.21.15.46:0/2607068640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c005d30 con 0x558351f5f480 2024-03-29T08:19:51.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff670200640 1 --2- 172.21.15.46:0/2607068640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558351f5f480 0x558351fe78a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff66000b320 tx=0x7ff66000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ccdbdcc9b9a2a0d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff66e400640 1 -- 172.21.15.46:0/2607068640 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff660014070 con 0x558351f5f480 2024-03-29T08:19:51.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff66e400640 1 -- 172.21.15.46:0/2607068640 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66000d5d0 con 0x558351f5f480 2024-03-29T08:19:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff66e400640 1 -- 172.21.15.46:0/2607068640 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66000d940 con 0x558351f5f480 2024-03-29T08:19:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 -- 172.21.15.46:0/2607068640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558351f5f480 msgr2=0x558351fe78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 --2- 172.21.15.46:0/2607068640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558351f5f480 0x558351fe78a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff66000b320 tx=0x7ff66000b7e0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 -- 172.21.15.46:0/2607068640 shutdown_connections 2024-03-29T08:19:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 --2- 172.21.15.46:0/2607068640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558351fed900 0x558351f15f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 --2- 172.21.15.46:0/2607068640 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558351ff0340 0x558351fe7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 --2- 172.21.15.46:0/2607068640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558351f5f480 0x558351fe78a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 -- 172.21.15.46:0/2607068640 >> 172.21.15.46:0/2607068640 conn(0x558351f42fa0 msgr2=0x558351f43380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 -- 172.21.15.46:0/2607068640 shutdown_connections 2024-03-29T08:19:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 -- 172.21.15.46:0/2607068640 wait complete. 2024-03-29T08:19:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 Processor -- start 2024-03-29T08:19:51.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 -- start start 2024-03-29T08:19:51.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558351f5f480 0x558351f46010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558351fed900 0x558351fe8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558351ff0340 0x558351fe5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558351fec5b0 con 0x558351fed900 2024-03-29T08:19:51.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558351fec310 con 0x558351f5f480 2024-03-29T08:19:51.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff672f38080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x558351f18a40 con 0x558351ff0340 2024-03-29T08:19:51.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.432+0000 7ff66f800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558351fed900 0x558351fe8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff66f800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558351fed900 0x558351fe8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45488/0 (socket says 172.21.15.46:45488) 2024-03-29T08:19:51.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff66f800640 1 -- 172.21.15.46:0/3054859569 learned_addr learned my addr 172.21.15.46:0/3054859569 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:51.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff670c00640 1 --2- 172.21.15.46:0/3054859569 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558351ff0340 0x558351fe5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff670200640 1 --2- 172.21.15.46:0/3054859569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558351f5f480 0x558351f46010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff66f800640 1 -- 172.21.15.46:0/3054859569 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558351ff0340 msgr2=0x558351fe5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff66f800640 1 --2- 172.21.15.46:0/3054859569 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558351ff0340 0x558351fe5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff66f800640 1 -- 172.21.15.46:0/3054859569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558351f5f480 msgr2=0x558351f46010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff66f800640 1 --2- 172.21.15.46:0/3054859569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558351f5f480 0x558351f46010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff66f800640 1 -- 172.21.15.46:0/3054859569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff660001c20 con 0x558351fed900 2024-03-29T08:19:51.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff670200640 1 --2- 172.21.15.46:0/3054859569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558351f5f480 0x558351f46010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:51.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff66f800640 1 --2- 172.21.15.46:0/3054859569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558351fed900 0x558351fe8050 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff65c0087a0 tx=0x7ff65c008c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff66d000640 1 -- 172.21.15.46:0/3054859569 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c00f040 con 0x558351fed900 2024-03-29T08:19:51.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff66d000640 1 -- 172.21.15.46:0/3054859569 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65c010ba0 con 0x558351fed900 2024-03-29T08:19:51.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff672f38080 1 -- 172.21.15.46:0/3054859569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66800a100 con 0x558351fed900 2024-03-29T08:19:51.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff66d000640 1 -- 172.21.15.46:0/3054859569 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c015450 con 0x558351fed900 2024-03-29T08:19:51.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff672f38080 1 -- 172.21.15.46:0/3054859569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558351e6c8c0 con 0x558351fed900 2024-03-29T08:19:51.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff66d000640 1 -- 172.21.15.46:0/3054859569 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff65c0155b0 con 0x558351fed900 2024-03-29T08:19:51.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff66d000640 1 --2- 172.21.15.46:0/3054859569 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff6500650a0 0x7ff650067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.436+0000 7ff66d000640 1 -- 172.21.15.46:0/3054859569 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff65c083c60 con 0x558351fed900 2024-03-29T08:19:51.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff672f38080 1 --2- 172.21.15.46:0/3054859569 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x558351fe8590 0x558352085dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff672f38080 1 -- 172.21.15.46:0/3054859569 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:1 5.19 5:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) v8 -- 0x558351febd90 con 0x558351fe8590 2024-03-29T08:19:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff670200640 1 --2- 172.21.15.46:0/3054859569 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff6500650a0 0x7ff650067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff670c00640 1 --2- 172.21.15.46:0/3054859569 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x558351fe8590 0x558352085dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff670200640 1 --2- 172.21.15.46:0/3054859569 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff6500650a0 0x7ff650067560 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff66000b820 tx=0x7ff660008000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff670c00640 1 --2- 172.21.15.46:0/3054859569 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x558351fe8590 0x558352085dd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff670c00640 1 -- 172.21.15.46:0/3054859569 <== osd.4 v2:172.21.15.73:6800/27689 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7ff6680b0020 con 0x558351fe8590 2024-03-29T08:19:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff672f38080 1 -- 172.21.15.46:0/3054859569 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x558351fe8590 msgr2=0x558352085dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff672f38080 1 --2- 172.21.15.46:0/3054859569 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x558351fe8590 0x558352085dd0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff672f38080 1 -- 172.21.15.46:0/3054859569 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff6500650a0 msgr2=0x7ff650067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff672f38080 1 --2- 172.21.15.46:0/3054859569 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff6500650a0 0x7ff650067560 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff66000b820 tx=0x7ff660008000 comp rx=0 tx=0).stop 2024-03-29T08:19:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff672f38080 1 -- 172.21.15.46:0/3054859569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558351fed900 msgr2=0x558351fe8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff672f38080 1 --2- 172.21.15.46:0/3054859569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558351fed900 0x558351fe8050 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff65c0087a0 tx=0x7ff65c008c60 comp rx=0 tx=0).stop 2024-03-29T08:19:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff670c00640 1 -- 172.21.15.46:0/3054859569 reap_dead start 2024-03-29T08:19:51.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff672f38080 1 -- 172.21.15.46:0/3054859569 shutdown_connections 2024-03-29T08:19:51.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff672f38080 1 -- 172.21.15.46:0/3054859569 >> 172.21.15.46:0/3054859569 conn(0x558351f42fa0 msgr2=0x558351f43380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:51.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff672f38080 1 -- 172.21.15.46:0/3054859569 shutdown_connections 2024-03-29T08:19:51.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.440+0000 7ff672f38080 1 -- 172.21.15.46:0/3054859569 wait complete. 2024-03-29T08:19:51.446 INFO:teuthology.orchestra.run.smithi046.stdout:mytag 2024-03-29T08:19:51.446 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-03-29T08:19:51.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf4c80080 1 Processor -- start 2024-03-29T08:19:51.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf4c80080 1 -- start start 2024-03-29T08:19:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf4c80080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e2baf20a0 0x558e2bbc98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf4c80080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e2baf2480 0x558e2bbc65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf4c80080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558e2bb25e10 0x558e2bb368c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf4c80080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e2ba39960 con 0x558e2baf20a0 2024-03-29T08:19:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf4c80080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e2ba39600 con 0x558e2baf2480 2024-03-29T08:19:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf4c80080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e2ba30690 con 0x558e2bb25e10 2024-03-29T08:19:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e2baf20a0 0x558e2bbc98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e2baf20a0 0x558e2bbc98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45510/0 (socket says 172.21.15.46:45510) 2024-03-29T08:19:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf1e00640 1 -- 172.21.15.46:0/1286097766 learned_addr learned my addr 172.21.15.46:0/1286097766 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf1400640 1 --2- 172.21.15.46:0/1286097766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e2baf2480 0x558e2bbc65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf2800640 1 --2- 172.21.15.46:0/1286097766 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558e2bb25e10 0x558e2bb368c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf1e00640 1 -- 172.21.15.46:0/1286097766 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558e2bb25e10 msgr2=0x558e2bb368c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf1e00640 1 --2- 172.21.15.46:0/1286097766 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558e2bb25e10 0x558e2bb368c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf1e00640 1 -- 172.21.15.46:0/1286097766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e2baf2480 msgr2=0x558e2bbc65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf1e00640 1 --2- 172.21.15.46:0/1286097766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e2baf2480 0x558e2bbc65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf1e00640 1 -- 172.21.15.46:0/1286097766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec00a1c0 con 0x558e2baf20a0 2024-03-29T08:19:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf1400640 1 --2- 172.21.15.46:0/1286097766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e2baf2480 0x558e2bbc65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf1e00640 1 --2- 172.21.15.46:0/1286097766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e2baf20a0 0x558e2bbc98b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4001df0 tx=0x7fcbe400b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4342bfa2995cd810 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbebe00640 1 -- 172.21.15.46:0/1286097766 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4014070 con 0x558e2baf20a0 2024-03-29T08:19:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbebe00640 1 -- 172.21.15.46:0/1286097766 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe400f410 con 0x558e2baf20a0 2024-03-29T08:19:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbebe00640 1 -- 172.21.15.46:0/1286097766 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400f7a0 con 0x558e2baf20a0 2024-03-29T08:19:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf4c80080 1 -- 172.21.15.46:0/1286097766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e2baf20a0 msgr2=0x558e2bbc98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf4c80080 1 --2- 172.21.15.46:0/1286097766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e2baf20a0 0x558e2bbc98b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4001df0 tx=0x7fcbe400b5c0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf4c80080 1 -- 172.21.15.46:0/1286097766 shutdown_connections 2024-03-29T08:19:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf4c80080 1 --2- 172.21.15.46:0/1286097766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e2baf2480 0x558e2bbc65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf4c80080 1 --2- 172.21.15.46:0/1286097766 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558e2bb25e10 0x558e2bb368c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf4c80080 1 --2- 172.21.15.46:0/1286097766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e2baf20a0 0x558e2bbc98b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf4c80080 1 -- 172.21.15.46:0/1286097766 >> 172.21.15.46:0/1286097766 conn(0x558e2baf8550 msgr2=0x558e2bb23df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.536+0000 7fcbf4c80080 1 -- 172.21.15.46:0/1286097766 shutdown_connections 2024-03-29T08:19:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf4c80080 1 -- 172.21.15.46:0/1286097766 wait complete. 2024-03-29T08:19:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf4c80080 1 Processor -- start 2024-03-29T08:19:51.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf4c80080 1 -- start start 2024-03-29T08:19:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf4c80080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e2baf20a0 0x558e2bbc5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf4c80080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e2baf2480 0x558e2bb37e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf4c80080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558e2bb25e10 0x558e2bb383c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf4c80080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e2bbc5870 con 0x558e2baf2480 2024-03-29T08:19:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf4c80080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e2bbbdc30 con 0x558e2baf20a0 2024-03-29T08:19:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf4c80080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e2bbbdea0 con 0x558e2bb25e10 2024-03-29T08:19:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e2baf2480 0x558e2bb37e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e2baf2480 0x558e2bb37e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45516/0 (socket says 172.21.15.46:45516) 2024-03-29T08:19:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf1400640 1 -- 172.21.15.46:0/3357601853 learned_addr learned my addr 172.21.15.46:0/3357601853 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf1e00640 1 --2- 172.21.15.46:0/3357601853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e2baf20a0 0x558e2bbc5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf2800640 1 --2- 172.21.15.46:0/3357601853 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558e2bb25e10 0x558e2bb383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf1400640 1 -- 172.21.15.46:0/3357601853 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558e2bb25e10 msgr2=0x558e2bb383c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf1400640 1 --2- 172.21.15.46:0/3357601853 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558e2bb25e10 0x558e2bb383c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf1400640 1 -- 172.21.15.46:0/3357601853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e2baf20a0 msgr2=0x558e2bbc5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf1400640 1 --2- 172.21.15.46:0/3357601853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558e2baf20a0 0x558e2bbc5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf1400640 1 -- 172.21.15.46:0/3357601853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe0005cc0 con 0x558e2baf2480 2024-03-29T08:19:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf1400640 1 --2- 172.21.15.46:0/3357601853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e2baf2480 0x558e2bb37e80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00087f0 tx=0x7fcbe0008820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbeaa00640 1 -- 172.21.15.46:0/3357601853 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe000f040 con 0x558e2baf2480 2024-03-29T08:19:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbeaa00640 1 -- 172.21.15.46:0/3357601853 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe0006460 con 0x558e2baf2480 2024-03-29T08:19:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf4c80080 1 -- 172.21.15.46:0/3357601853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558e2ba3b970 con 0x558e2baf2480 2024-03-29T08:19:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbeaa00640 1 -- 172.21.15.46:0/3357601853 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00067f0 con 0x558e2baf2480 2024-03-29T08:19:51.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.540+0000 7fcbf4c80080 1 -- 172.21.15.46:0/3357601853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e2ba488c0 con 0x558e2baf2480 2024-03-29T08:19:51.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbeaa00640 1 -- 172.21.15.46:0/3357601853 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcbe0006950 con 0x558e2baf2480 2024-03-29T08:19:51.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbeaa00640 1 --2- 172.21.15.46:0/3357601853 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcbd40650a0 0x7fcbd4067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbeaa00640 1 -- 172.21.15.46:0/3357601853 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbe0082dc0 con 0x558e2baf2480 2024-03-29T08:19:51.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf1e00640 1 --2- 172.21.15.46:0/3357601853 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcbd40650a0 0x7fcbd4067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf4c80080 1 --2- 172.21.15.46:0/3357601853 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x558e2bb38900 0x558e2bb3aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:51.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf4c80080 1 -- 172.21.15.46:0/3357601853 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:1 5.12 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36) v8 -- 0x558e2bbbeb20 con 0x558e2bb38900 2024-03-29T08:19:51.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf2800640 1 --2- 172.21.15.46:0/3357601853 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x558e2bb38900 0x558e2bb3aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:51.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf1e00640 1 --2- 172.21.15.46:0/3357601853 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcbd40650a0 0x7fcbd4067560 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400b600 tx=0x7fcbe4009000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf2800640 1 --2- 172.21.15.46:0/3357601853 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x558e2bb38900 0x558e2bb3aa70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:51.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf2800640 1 -- 172.21.15.46:0/3357601853 <== osd.6 v2:172.21.15.73:6824/27691 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fcbec0b5720 con 0x558e2bb38900 2024-03-29T08:19:51.548 INFO:teuthology.orchestra.run.smithi046.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-03-29T08:19:51.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf4c80080 1 -- 172.21.15.46:0/3357601853 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x558e2bb38900 msgr2=0x558e2bb3aa70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf4c80080 1 --2- 172.21.15.46:0/3357601853 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x558e2bb38900 0x558e2bb3aa70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:51.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf4c80080 1 -- 172.21.15.46:0/3357601853 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcbd40650a0 msgr2=0x7fcbd4067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf4c80080 1 --2- 172.21.15.46:0/3357601853 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcbd40650a0 0x7fcbd4067560 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcbe400b600 tx=0x7fcbe4009000 comp rx=0 tx=0).stop 2024-03-29T08:19:51.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf4c80080 1 -- 172.21.15.46:0/3357601853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e2baf2480 msgr2=0x558e2bb37e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:51.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf4c80080 1 --2- 172.21.15.46:0/3357601853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558e2baf2480 0x558e2bb37e80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00087f0 tx=0x7fcbe0008820 comp rx=0 tx=0).stop 2024-03-29T08:19:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf2800640 1 -- 172.21.15.46:0/3357601853 reap_dead start 2024-03-29T08:19:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf4c80080 1 -- 172.21.15.46:0/3357601853 shutdown_connections 2024-03-29T08:19:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf4c80080 1 -- 172.21.15.46:0/3357601853 >> 172.21.15.46:0/3357601853 conn(0x558e2baf8550 msgr2=0x558e2bbc60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf4c80080 1 -- 172.21.15.46:0/3357601853 shutdown_connections 2024-03-29T08:19:51.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:51.544+0000 7fcbf4c80080 1 -- 172.21.15.46:0/3357601853 wait complete. 2024-03-29T08:19:51.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:19:51.551 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T08:19:51.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:19:51.678 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:19:51.731 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T08:19:51.731 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:19:51.731 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:19:51.740 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:19:51.740 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:19:51.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:51.740 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:19:51.790 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-29T08:19:51.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:19:51.792 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:19:51.792 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:19:51.800 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:19:51.800 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:19:51.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:51.801 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T08:19:51.851 INFO:teuthology.orchestra.run.smithi073.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-29T08:19:51.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:19:51.852 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:19:51.852 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:19:51.861 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:19:51.861 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:19:51.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:51.861 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-29T08:19:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-29T08:19:51.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:19:51.912 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:19:51.912 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:19:51.920 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:19:51.921 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:19:51.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:19:51.921 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-29T08:19:51.971 INFO:teuthology.orchestra.run.smithi073.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-29T08:19:51.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:19:51.972 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:19:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.112+0000 7fa3cd400640 1 Processor -- start 2024-03-29T08:19:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.112+0000 7fa3cd400640 1 -- start start 2024-03-29T08:19:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.112+0000 7fa3cd400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3c81593d0 0x7fa3c806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.112+0000 7fa3cd400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa3c815cd90 0x7fa3c81597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.112+0000 7fa3cd400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c806d130 con 0x7fa3c815d230 2024-03-29T08:19:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.112+0000 7fa3cd400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c815b8f0 con 0x7fa3c81593d0 2024-03-29T08:19:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.112+0000 7fa3cd400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c815ad50 con 0x7fa3c815cd90 2024-03-29T08:19:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.112+0000 7fa3c6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3c81593d0 0x7fa3c806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.112+0000 7fa3bde00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa3c815d230 0x7fa3c815d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44842/0 (socket says 172.21.15.46:44842) 2024-03-29T08:19:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.112+0000 7fa3bde00640 1 -- 172.21.15.46:0/3601015758 learned_addr learned my addr 172.21.15.46:0/3601015758 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.112+0000 7fa3c7200640 1 --2- 172.21.15.46:0/3601015758 >> v2:172.21.15.73:3300/0 conn(0x7fa3c815cd90 0x7fa3c81597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.112+0000 7fa3c6800640 1 -- 172.21.15.46:0/3601015758 >> v2:172.21.15.73:3300/0 conn(0x7fa3c815cd90 msgr2=0x7fa3c81597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.112+0000 7fa3c6800640 1 --2- 172.21.15.46:0/3601015758 >> v2:172.21.15.73:3300/0 conn(0x7fa3c815cd90 0x7fa3c81597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.112+0000 7fa3c6800640 1 -- 172.21.15.46:0/3601015758 >> v1:172.21.15.46:6789/0 conn(0x7fa3c815d230 legacy=0x7fa3c815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.112+0000 7fa3c6800640 1 -- 172.21.15.46:0/3601015758 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c815afc0 con 0x7fa3c81593d0 2024-03-29T08:19:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.112+0000 7fa3c5e00640 1 -- 172.21.15.46:0/3601015758 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4015735155 0 0) 0x7fa3c806d130 con 0x7fa3c815d230 2024-03-29T08:19:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.116+0000 7fa3c6800640 1 --2- 172.21.15.46:0/3601015758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c81593d0 0x7fa3c806cbf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa3c806d360 tx=0x7fa3b801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3cf7cee872513e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.116+0000 7fa3c5e00640 1 -- 172.21.15.46:0/3601015758 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80ab680 con 0x7fa3c81593d0 2024-03-29T08:19:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.116+0000 7fa3c5e00640 1 -- 172.21.15.46:0/3601015758 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b80ab7e0 con 0x7fa3c81593d0 2024-03-29T08:19:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.116+0000 7fa3c5e00640 1 -- 172.21.15.46:0/3601015758 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80abb70 con 0x7fa3c81593d0 2024-03-29T08:19:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.116+0000 7fa3cd400640 1 -- 172.21.15.46:0/3601015758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c81593d0 msgr2=0x7fa3c806cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.116+0000 7fa3cd400640 1 --2- 172.21.15.46:0/3601015758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c81593d0 0x7fa3c806cbf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fa3c806d360 tx=0x7fa3b801c480 comp rx=0 tx=0).stop 2024-03-29T08:19:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.116+0000 7fa3cd400640 1 -- 172.21.15.46:0/3601015758 shutdown_connections 2024-03-29T08:19:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.116+0000 7fa3cd400640 1 --2- 172.21.15.46:0/3601015758 >> v2:172.21.15.73:3300/0 conn(0x7fa3c815cd90 0x7fa3c81597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.116+0000 7fa3cd400640 1 --2- 172.21.15.46:0/3601015758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c81593d0 0x7fa3c806cbf0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.116+0000 7fa3cd400640 1 -- 172.21.15.46:0/3601015758 >> 172.21.15.46:0/3601015758 conn(0x7fa3c8066110 msgr2=0x7fa3c8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.116+0000 7fa3cd400640 1 -- 172.21.15.46:0/3601015758 shutdown_connections 2024-03-29T08:19:52.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.116+0000 7fa3cd400640 1 -- 172.21.15.46:0/3601015758 wait complete. 2024-03-29T08:19:52.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3cd400640 1 Processor -- start 2024-03-29T08:19:52.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3cd400640 1 -- start start 2024-03-29T08:19:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3cd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3c81593d0 0x7fa3c8155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3cd400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3c815cd90 0x7fa3c8149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3cd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c815d230 0x7fa3c8152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3cd400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c8155bc0 con 0x7fa3c815d230 2024-03-29T08:19:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3cd400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c8152ae0 con 0x7fa3c81593d0 2024-03-29T08:19:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3cd400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c81548c0 con 0x7fa3c815cd90 2024-03-29T08:19:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3c7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c815d230 0x7fa3c8152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3c7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c815d230 0x7fa3c8152190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45540/0 (socket says 172.21.15.46:45540) 2024-03-29T08:19:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3c7200640 1 -- 172.21.15.46:0/3262779001 learned_addr learned my addr 172.21.15.46:0/3262779001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3c6800640 1 --2- 172.21.15.46:0/3262779001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3c81593d0 0x7fa3c8155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3bde00640 1 --2- 172.21.15.46:0/3262779001 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3c815cd90 0x7fa3c8149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:52.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3c7200640 1 -- 172.21.15.46:0/3262779001 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3c815cd90 msgr2=0x7fa3c8149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3c7200640 1 --2- 172.21.15.46:0/3262779001 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3c815cd90 0x7fa3c8149820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3c7200640 1 -- 172.21.15.46:0/3262779001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3c81593d0 msgr2=0x7fa3c8155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3c7200640 1 --2- 172.21.15.46:0/3262779001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3c81593d0 0x7fa3c8155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3c7200640 1 -- 172.21.15.46:0/3262779001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b8009420 con 0x7fa3c815d230 2024-03-29T08:19:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3c7200640 1 --2- 172.21.15.46:0/3262779001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c815d230 0x7fa3c8152190 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000efc0 tx=0x7fa3c000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3bf400640 1 -- 172.21.15.46:0/3262779001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0009b20 con 0x7fa3c815d230 2024-03-29T08:19:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3bf400640 1 -- 172.21.15.46:0/3262779001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3c000f040 con 0x7fa3c815d230 2024-03-29T08:19:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3bf400640 1 -- 172.21.15.46:0/3262779001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3c00146c0 con 0x7fa3c815d230 2024-03-29T08:19:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3cd400640 1 -- 172.21.15.46:0/3262779001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c8154ba0 con 0x7fa3c815d230 2024-03-29T08:19:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.120+0000 7fa3cd400640 1 -- 172.21.15.46:0/3262779001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c8155160 con 0x7fa3c815d230 2024-03-29T08:19:52.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.124+0000 7fa3cd400640 1 -- 172.21.15.46:0/3262779001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c8155380 con 0x7fa3c815d230 2024-03-29T08:19:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.124+0000 7fa3bf400640 1 -- 172.21.15.46:0/3262779001 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa3c0014820 con 0x7fa3c815d230 2024-03-29T08:19:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.124+0000 7fa3bf400640 1 --2- 172.21.15.46:0/3262779001 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa3a4066690 0x7fa3a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.124+0000 7fa3bf400640 1 -- 172.21.15.46:0/3262779001 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3c0084f60 con 0x7fa3c815d230 2024-03-29T08:19:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.124+0000 7fa3c6800640 1 --2- 172.21.15.46:0/3262779001 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa3a4066690 0x7fa3a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:52.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.124+0000 7fa3c6800640 1 --2- 172.21.15.46:0/3262779001 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa3a4066690 0x7fa3a4068b50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80af000 tx=0x7fa3b80a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:52.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.128+0000 7fa3bf400640 1 -- 172.21.15.46:0/3262779001 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa3c00522c0 con 0x7fa3c815d230 2024-03-29T08:19:52.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.328+0000 7fa3cd400640 1 -- 172.21.15.46:0/3262779001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3c80cb070 con 0x7fa3c815d230 2024-03-29T08:19:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.328+0000 7fa3bf400640 1 -- 172.21.15.46:0/3262779001 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fa3c0018090 con 0x7fa3c815d230 2024-03-29T08:19:52.329 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:52.329 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/4274575126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4274575126},{"type":"v1","addr":"172.21.15.46:6834","nonce":4274575126}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3739899623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3739899623},{"type":"v1","addr":"172.21.15.73:6837","nonce":3739899623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1905427475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1905427475},{"type":"v1","addr":"172.21.15.46:6835","nonce":1905427475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:19:14.663142+0000","modified":"2024-03-29T08:19:17.680359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5152},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5152":{"gid":5152,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/1941408960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1941408960},{"type":"v1","addr":"172.21.15.73:6836","nonce":1941408960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-29T08:19:52.329 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2024-03-29T08:19:52.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.328+0000 7fa3cd400640 1 -- 172.21.15.46:0/3262779001 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa3a4066690 msgr2=0x7fa3a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.328+0000 7fa3cd400640 1 --2- 172.21.15.46:0/3262779001 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa3a4066690 0x7fa3a4068b50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80af000 tx=0x7fa3b80a7000 comp rx=0 tx=0).stop 2024-03-29T08:19:52.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.328+0000 7fa3cd400640 1 -- 172.21.15.46:0/3262779001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c815d230 msgr2=0x7fa3c8152190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.328+0000 7fa3cd400640 1 --2- 172.21.15.46:0/3262779001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c815d230 0x7fa3c8152190 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa3c000efc0 tx=0x7fa3c000c490 comp rx=0 tx=0).stop 2024-03-29T08:19:52.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.328+0000 7fa3cd400640 1 -- 172.21.15.46:0/3262779001 shutdown_connections 2024-03-29T08:19:52.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.328+0000 7fa3cd400640 1 --2- 172.21.15.46:0/3262779001 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa3a4066690 0x7fa3a4068b50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.328+0000 7fa3cd400640 1 --2- 172.21.15.46:0/3262779001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3c815d230 0x7fa3c8152190 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.328+0000 7fa3cd400640 1 --2- 172.21.15.46:0/3262779001 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3c815cd90 0x7fa3c8149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.328+0000 7fa3cd400640 1 --2- 172.21.15.46:0/3262779001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3c81593d0 0x7fa3c8155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.328+0000 7fa3cd400640 1 -- 172.21.15.46:0/3262779001 >> 172.21.15.46:0/3262779001 conn(0x7fa3c8066110 msgr2=0x7fa3c8163c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:52.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.328+0000 7fa3cd400640 1 -- 172.21.15.46:0/3262779001 shutdown_connections 2024-03-29T08:19:52.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.328+0000 7fa3cd400640 1 -- 172.21.15.46:0/3262779001 wait complete. 2024-03-29T08:19:52.351 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:19:52.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.492+0000 7f4133200640 1 Processor -- start 2024-03-29T08:19:52.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.492+0000 7f4133200640 1 -- start start 2024-03-29T08:19:52.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.492+0000 7f4133200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f412c15c9d0 0x7f412c0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.492+0000 7f4133200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412c0d0030 con 0x7f412c099150 2024-03-29T08:19:52.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.492+0000 7f4133200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f412c0cf420 con 0x7f412c0cef60 2024-03-29T08:19:52.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.492+0000 7f4133200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f412c0cf700 con 0x7f412c15c9d0 2024-03-29T08:19:52.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.492+0000 7f412be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f412c15c9d0 0x7f412c0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:52.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.492+0000 7f4131200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f412c099150 0x7f412c166a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44852/0 (socket says 172.21.15.46:44852) 2024-03-29T08:19:52.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.492+0000 7f4131200640 1 -- 172.21.15.46:0/2216386824 learned_addr learned my addr 172.21.15.46:0/2216386824 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:52.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.492+0000 7f412be00640 1 -- 172.21.15.46:0/2216386824 >> v1:172.21.15.73:6789/0 conn(0x7f412c0cef60 legacy=0x7f412c098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.492+0000 7f412be00640 1 -- 172.21.15.46:0/2216386824 >> v1:172.21.15.46:6789/0 conn(0x7f412c099150 legacy=0x7f412c166a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.492+0000 7f412be00640 1 -- 172.21.15.46:0/2216386824 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c0cf9e0 con 0x7f412c15c9d0 2024-03-29T08:19:52.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.492+0000 7f412aa00640 1 -- 172.21.15.46:0/2216386824 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3448592860 0 0) 0x7f412c0d0030 con 0x7f412c099150 2024-03-29T08:19:52.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.496+0000 7f412be00640 1 --2- 172.21.15.46:0/2216386824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f412c15c9d0 0x7f412c0cea20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f411c0097c0 tx=0x7f411c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c4e34c033c4b18 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:52.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.496+0000 7f412aa00640 1 -- 172.21.15.46:0/2216386824 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c0aa5f0 con 0x7f412c15c9d0 2024-03-29T08:19:52.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.496+0000 7f412aa00640 1 -- 172.21.15.46:0/2216386824 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411c0aa750 con 0x7f412c15c9d0 2024-03-29T08:19:52.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.496+0000 7f412aa00640 1 -- 172.21.15.46:0/2216386824 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411c0aaae0 con 0x7f412c15c9d0 2024-03-29T08:19:52.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.496+0000 7f4133200640 1 -- 172.21.15.46:0/2216386824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f412c15c9d0 msgr2=0x7f412c0cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.496+0000 7f4133200640 1 --2- 172.21.15.46:0/2216386824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f412c15c9d0 0x7f412c0cea20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f411c0097c0 tx=0x7f411c01c470 comp rx=0 tx=0).stop 2024-03-29T08:19:52.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.496+0000 7f4133200640 1 -- 172.21.15.46:0/2216386824 shutdown_connections 2024-03-29T08:19:52.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.496+0000 7f4133200640 1 --2- 172.21.15.46:0/2216386824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f412c15c9d0 0x7f412c0cea20 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.496+0000 7f4133200640 1 -- 172.21.15.46:0/2216386824 >> 172.21.15.46:0/2216386824 conn(0x7f412c0b21a0 msgr2=0x7f412c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:52.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.496+0000 7f4133200640 1 -- 172.21.15.46:0/2216386824 shutdown_connections 2024-03-29T08:19:52.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.496+0000 7f4133200640 1 -- 172.21.15.46:0/2216386824 wait complete. 2024-03-29T08:19:52.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f4133200640 1 Processor -- start 2024-03-29T08:19:52.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f4133200640 1 -- start start 2024-03-29T08:19:52.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f4133200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f412c099150 0x7f412c157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f4133200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f412c0cef60 0x7f412c09a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f4133200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f412c15c9d0 0x7f412c15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f4133200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c15a310 con 0x7f412c0cef60 2024-03-29T08:19:52.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f4133200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c159740 con 0x7f412c15c9d0 2024-03-29T08:19:52.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f4133200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412c1599e0 con 0x7f412c099150 2024-03-29T08:19:52.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f412b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f412c0cef60 0x7f412c09a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:52.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f412b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f412c0cef60 0x7f412c09a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45556/0 (socket says 172.21.15.46:45556) 2024-03-29T08:19:52.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f4131200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f412c15c9d0 0x7f412c15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:52.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f412be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f412c099150 0x7f412c157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:52.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f412b400640 1 -- 172.21.15.46:0/1506015652 learned_addr learned my addr 172.21.15.46:0/1506015652 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:52.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f412b400640 1 -- 172.21.15.46:0/1506015652 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f412c099150 msgr2=0x7f412c157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f412b400640 1 --2- 172.21.15.46:0/1506015652 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f412c099150 0x7f412c157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f412b400640 1 -- 172.21.15.46:0/1506015652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f412c15c9d0 msgr2=0x7f412c15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f412b400640 1 --2- 172.21.15.46:0/1506015652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f412c15c9d0 0x7f412c15a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f412b400640 1 -- 172.21.15.46:0/1506015652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c009420 con 0x7f412c0cef60 2024-03-29T08:19:52.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f412b400640 1 --2- 172.21.15.46:0/1506015652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f412c0cef60 0x7f412c09a3b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f411800c500 tx=0x7f411800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:52.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f4123e00640 1 -- 172.21.15.46:0/1506015652 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118005220 con 0x7f412c0cef60 2024-03-29T08:19:52.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f4123e00640 1 -- 172.21.15.46:0/1506015652 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4118005380 con 0x7f412c0cef60 2024-03-29T08:19:52.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f4133200640 1 -- 172.21.15.46:0/1506015652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412c159cc0 con 0x7f412c0cef60 2024-03-29T08:19:52.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f4123e00640 1 -- 172.21.15.46:0/1506015652 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4118007270 con 0x7f412c0cef60 2024-03-29T08:19:52.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f4133200640 1 -- 172.21.15.46:0/1506015652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412c158040 con 0x7f412c0cef60 2024-03-29T08:19:52.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.500+0000 7f4133200640 1 -- 172.21.15.46:0/1506015652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f412c066c90 con 0x7f412c0cef60 2024-03-29T08:19:52.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.504+0000 7f4123e00640 1 -- 172.21.15.46:0/1506015652 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f41180073d0 con 0x7f412c0cef60 2024-03-29T08:19:52.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.504+0000 7f4123e00640 1 --2- 172.21.15.46:0/1506015652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f410c066690 0x7f410c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.504+0000 7f4123e00640 1 -- 172.21.15.46:0/1506015652 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4118083a10 con 0x7f412c0cef60 2024-03-29T08:19:52.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.504+0000 7f412be00640 1 --2- 172.21.15.46:0/1506015652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f410c066690 0x7f410c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:52.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.504+0000 7f412be00640 1 --2- 172.21.15.46:0/1506015652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f410c066690 0x7f410c068b50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f411c0ae000 tx=0x7f411c0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:52.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.508+0000 7f4123e00640 1 -- 172.21.15.46:0/1506015652 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4118050da0 con 0x7f412c0cef60 2024-03-29T08:19:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.668+0000 7f4133200640 1 -- 172.21.15.46:0/1506015652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f412c167f30 con 0x7f412c0cef60 2024-03-29T08:19:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.672+0000 7f4123e00640 1 -- 172.21.15.46:0/1506015652 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f41180543c0 con 0x7f412c0cef60 2024-03-29T08:19:52.672 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:52.672 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:19:15.665674+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:19:12.674055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-29T08:19:13.661287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:19:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.672+0000 7f4133200640 1 -- 172.21.15.46:0/1506015652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f410c066690 msgr2=0x7f410c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.672+0000 7f4133200640 1 --2- 172.21.15.46:0/1506015652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f410c066690 0x7f410c068b50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f411c0ae000 tx=0x7f411c0a7000 comp rx=0 tx=0).stop 2024-03-29T08:19:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.672+0000 7f4133200640 1 -- 172.21.15.46:0/1506015652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f412c0cef60 msgr2=0x7f412c09a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.672+0000 7f4133200640 1 --2- 172.21.15.46:0/1506015652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f412c0cef60 0x7f412c09a3b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f411800c500 tx=0x7f411800c9c0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.672+0000 7f4133200640 1 -- 172.21.15.46:0/1506015652 shutdown_connections 2024-03-29T08:19:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.672+0000 7f4133200640 1 --2- 172.21.15.46:0/1506015652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f410c066690 0x7f410c068b50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.672+0000 7f4133200640 1 --2- 172.21.15.46:0/1506015652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f412c15c9d0 0x7f412c15a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.672+0000 7f4133200640 1 --2- 172.21.15.46:0/1506015652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f412c0cef60 0x7f412c09a3b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.672+0000 7f4133200640 1 --2- 172.21.15.46:0/1506015652 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f412c099150 0x7f412c157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.672+0000 7f4133200640 1 -- 172.21.15.46:0/1506015652 >> 172.21.15.46:0/1506015652 conn(0x7f412c0b21a0 msgr2=0x7f412c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.672+0000 7f4133200640 1 -- 172.21.15.46:0/1506015652 shutdown_connections 2024-03-29T08:19:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.672+0000 7f4133200640 1 -- 172.21.15.46:0/1506015652 wait complete. 2024-03-29T08:19:52.693 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:19:12.674055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:19:13.661287+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:19:52.694 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:19:52.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99ad600640 1 Processor -- start 2024-03-29T08:19:52.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99ad600640 1 -- start start 2024-03-29T08:19:52.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99ad600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99a80d0d60 0x7f99a81654f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99ad600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a80cc660 con 0x7f99a815a790 2024-03-29T08:19:52.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99ad600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a80cc0a0 con 0x7f99a80cbbe0 2024-03-29T08:19:52.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99ad600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a80cc380 con 0x7f99a80d0d60 2024-03-29T08:19:52.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a6a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f99a815a790 0x7f99a80cee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44858/0 (socket says 172.21.15.46:44858) 2024-03-29T08:19:52.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a6a00640 1 -- 172.21.15.46:0/4240542072 learned_addr learned my addr 172.21.15.46:0/4240542072 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a7400640 1 --2- 172.21.15.46:0/4240542072 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99a80d0d60 0x7f99a81654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a6000640 1 -- 172.21.15.46:0/4240542072 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914990719 0 0) 0x7f99a80cc660 con 0x7f99a815a790 2024-03-29T08:19:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a6000640 1 -- 172.21.15.46:0/4240542072 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f998c003530 con 0x7f99a815a790 2024-03-29T08:19:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a6000640 1 -- 172.21.15.46:0/4240542072 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 155970594 0 0) 0x7f998c003530 con 0x7f99a815a790 2024-03-29T08:19:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a6000640 1 -- 172.21.15.46:0/4240542072 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99a80d0d60 msgr2=0x7f99a81654f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a6000640 1 --2- 172.21.15.46:0/4240542072 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99a80d0d60 0x7f99a81654f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a6000640 1 -- 172.21.15.46:0/4240542072 >> v1:172.21.15.73:6789/0 conn(0x7f99a80cbbe0 legacy=0x7f99a80d0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a6000640 1 -- 172.21.15.46:0/4240542072 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a80b0380 con 0x7f99a815a790 2024-03-29T08:19:52.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a6000640 1 -- 172.21.15.46:0/4240542072 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f9998001680 con 0x7f99a815a790 2024-03-29T08:19:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a7400640 1 --2- 172.21.15.46:0/4240542072 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99a80d0d60 0x7f99a81654f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:19:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a6000640 1 -- 172.21.15.46:0/4240542072 >> v1:172.21.15.46:6789/0 conn(0x7f99a815a790 legacy=0x7f99a80cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a6000640 1 --2- 172.21.15.46:0/4240542072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f998c0039b0 0x7f998c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a6000640 1 --2- 172.21.15.46:0/4240542072 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99a80d0d60 0x7f998c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a6000640 1 --2- 172.21.15.46:0/4240542072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f998c004d10 0x7f998c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a6000640 1 -- 172.21.15.46:0/4240542072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a80cc380 con 0x7f998c004d10 2024-03-29T08:19:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a6000640 1 -- 172.21.15.46:0/4240542072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c00bb70 con 0x7f998c0039b0 2024-03-29T08:19:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.836+0000 7f99a6000640 1 -- 172.21.15.46:0/4240542072 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998c00bd30 con 0x7f99a80d0d60 2024-03-29T08:19:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.840+0000 7f99a7400640 1 --2- 172.21.15.46:0/4240542072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f998c004d10 0x7f998c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.840+0000 7f99a6a00640 1 --2- 172.21.15.46:0/4240542072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f998c0039b0 0x7f998c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.840+0000 7f999fe00640 1 --2- 172.21.15.46:0/4240542072 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99a80d0d60 0x7f998c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.840+0000 7f99a7400640 1 -- 172.21.15.46:0/4240542072 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99a80d0d60 msgr2=0x7f998c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:52.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.840+0000 7f99a7400640 1 --2- 172.21.15.46:0/4240542072 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99a80d0d60 0x7f998c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.840+0000 7f99a7400640 1 -- 172.21.15.46:0/4240542072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f998c0039b0 msgr2=0x7f998c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:52.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.840+0000 7f99a7400640 1 --2- 172.21.15.46:0/4240542072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f998c0039b0 0x7f998c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.840+0000 7f99a7400640 1 -- 172.21.15.46:0/4240542072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a0009a70 con 0x7f998c004d10 2024-03-29T08:19:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.840+0000 7f99a7400640 1 --2- 172.21.15.46:0/4240542072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f998c004d10 0x7f998c005190 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e320 tx=0x7f99a001cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5541020ce910843 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.840+0000 7f99a6000640 1 -- 172.21.15.46:0/4240542072 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a00a5d40 con 0x7f998c004d10 2024-03-29T08:19:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.840+0000 7f99a6000640 1 -- 172.21.15.46:0/4240542072 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a000f040 con 0x7f998c004d10 2024-03-29T08:19:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.840+0000 7f99a6000640 1 -- 172.21.15.46:0/4240542072 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a00a5ea0 con 0x7f998c004d10 2024-03-29T08:19:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99ad600640 1 -- 172.21.15.46:0/4240542072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f998c004d10 msgr2=0x7f998c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99ad600640 1 --2- 172.21.15.46:0/4240542072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f998c004d10 0x7f998c005190 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f99a000e320 tx=0x7f99a001cd90 comp rx=0 tx=0).stop 2024-03-29T08:19:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99a7400640 1 -- 172.21.15.46:0/4240542072 reap_dead start 2024-03-29T08:19:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99ad600640 1 -- 172.21.15.46:0/4240542072 shutdown_connections 2024-03-29T08:19:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99ad600640 1 -- 172.21.15.46:0/4240542072 >> 172.21.15.46:0/4240542072 conn(0x7f99a8099a10 msgr2=0x7f99a8089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99ad600640 1 -- 172.21.15.46:0/4240542072 shutdown_connections 2024-03-29T08:19:52.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99ad600640 1 -- 172.21.15.46:0/4240542072 wait complete. 2024-03-29T08:19:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99ad600640 1 Processor -- start 2024-03-29T08:19:52.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99ad600640 1 -- start start 2024-03-29T08:19:52.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99ad600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f998c004d10 0x7f99a8154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99ad600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99a80ad5e0 0x7f99a806c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99ad600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99a806ea80 0x7f99a806ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99ad600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a806b970 con 0x7f998c004d10 2024-03-29T08:19:52.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99ad600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8154950 con 0x7f99a806ea80 2024-03-29T08:19:52.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99ad600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a806f440 con 0x7f99a80ad5e0 2024-03-29T08:19:52.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99a6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f998c004d10 0x7f99a8154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:52.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99a6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f998c004d10 0x7f99a8154410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45570/0 (socket says 172.21.15.46:45570) 2024-03-29T08:19:52.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99a6a00640 1 -- 172.21.15.46:0/560652065 learned_addr learned my addr 172.21.15.46:0/560652065 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:52.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99a6a00640 1 -- 172.21.15.46:0/560652065 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99a80ad5e0 msgr2=0x7f99a806c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99a6a00640 1 --2- 172.21.15.46:0/560652065 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99a80ad5e0 0x7f99a806c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99a6a00640 1 -- 172.21.15.46:0/560652065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99a806ea80 msgr2=0x7f99a806ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:52.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99a6a00640 1 --2- 172.21.15.46:0/560652065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99a806ea80 0x7f99a806ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:52.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99a6a00640 1 -- 172.21.15.46:0/560652065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a806bd90 con 0x7f998c004d10 2024-03-29T08:19:52.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99a6a00640 1 --2- 172.21.15.46:0/560652065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f998c004d10 0x7f99a8154410 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f99980072c0 tx=0x7f9998007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:52.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f999ea00640 1 -- 172.21.15.46:0/560652065 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998001610 con 0x7f998c004d10 2024-03-29T08:19:52.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f999ea00640 1 -- 172.21.15.46:0/560652065 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999800f940 con 0x7f998c004d10 2024-03-29T08:19:52.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f999ea00640 1 -- 172.21.15.46:0/560652065 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999800fcf0 con 0x7f998c004d10 2024-03-29T08:19:52.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.844+0000 7f99ad600640 1 -- 172.21.15.46:0/560652065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a8154b60 con 0x7f998c004d10 2024-03-29T08:19:52.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.848+0000 7f99ad600640 1 -- 172.21.15.46:0/560652065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a806afd0 con 0x7f998c004d10 2024-03-29T08:19:52.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.848+0000 7f99ad600640 1 -- 172.21.15.46:0/560652065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a806bbe0 con 0x7f998c004d10 2024-03-29T08:19:52.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.848+0000 7f999ea00640 1 -- 172.21.15.46:0/560652065 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f999801d460 con 0x7f998c004d10 2024-03-29T08:19:52.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.848+0000 7f999ea00640 1 --2- 172.21.15.46:0/560652065 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9984066690 0x7f9984068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:52.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.848+0000 7f999ea00640 1 -- 172.21.15.46:0/560652065 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9998084390 con 0x7f998c004d10 2024-03-29T08:19:52.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.848+0000 7f999fe00640 1 --2- 172.21.15.46:0/560652065 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9984066690 0x7f9984068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:52.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.852+0000 7f999fe00640 1 --2- 172.21.15.46:0/560652065 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9984066690 0x7f9984068b50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f99a80ca810 tx=0x7f9990009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:52.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:52.852+0000 7f999ea00640 1 -- 172.21.15.46:0/560652065 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f99980517a0 con 0x7f998c004d10 2024-03-29T08:19:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.020+0000 7f99ad600640 1 -- 172.21.15.46:0/560652065 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99a809e7d0 con 0x7f998c004d10 2024-03-29T08:19:53.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.020+0000 7f999ea00640 1 -- 172.21.15.46:0/560652065 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9998054dc0 con 0x7f998c004d10 2024-03-29T08:19:53.020 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:19:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.020+0000 7f99ad600640 1 -- 172.21.15.46:0/560652065 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9984066690 msgr2=0x7f9984068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.020+0000 7f99ad600640 1 --2- 172.21.15.46:0/560652065 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9984066690 0x7f9984068b50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f99a80ca810 tx=0x7f9990009040 comp rx=0 tx=0).stop 2024-03-29T08:19:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.020+0000 7f99ad600640 1 -- 172.21.15.46:0/560652065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f998c004d10 msgr2=0x7f99a8154410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.020+0000 7f99ad600640 1 --2- 172.21.15.46:0/560652065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f998c004d10 0x7f99a8154410 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f99980072c0 tx=0x7f9998007780 comp rx=0 tx=0).stop 2024-03-29T08:19:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.020+0000 7f99ad600640 1 -- 172.21.15.46:0/560652065 shutdown_connections 2024-03-29T08:19:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.020+0000 7f99ad600640 1 --2- 172.21.15.46:0/560652065 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9984066690 0x7f9984068b50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.020+0000 7f99ad600640 1 --2- 172.21.15.46:0/560652065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99a806ea80 0x7f99a806ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.020+0000 7f99ad600640 1 --2- 172.21.15.46:0/560652065 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99a80ad5e0 0x7f99a806c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.020+0000 7f99ad600640 1 --2- 172.21.15.46:0/560652065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f998c004d10 0x7f99a8154410 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.020+0000 7f99ad600640 1 -- 172.21.15.46:0/560652065 >> 172.21.15.46:0/560652065 conn(0x7f99a8099a10 msgr2=0x7f99a80b5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.020+0000 7f99ad600640 1 -- 172.21.15.46:0/560652065 shutdown_connections 2024-03-29T08:19:53.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.020+0000 7f99ad600640 1 -- 172.21.15.46:0/560652065 wait complete. 2024-03-29T08:19:53.041 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:19:53.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.184+0000 7f94a6c00640 1 Processor -- start 2024-03-29T08:19:53.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.184+0000 7f94a6c00640 1 -- start start 2024-03-29T08:19:53.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.184+0000 7f94a6c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f94a015c950 0x7f94a006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:53.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.184+0000 7f94a6c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a00cfb20 con 0x7f94a0160db0 2024-03-29T08:19:53.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.184+0000 7f94a6c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a00cef10 con 0x7f94a00cea30 2024-03-29T08:19:53.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.184+0000 7f94a6c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a00cf1f0 con 0x7f94a015c950 2024-03-29T08:19:53.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.184+0000 7f94a4c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f94a0160db0 0x7f94a00cad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44872/0 (socket says 172.21.15.46:44872) 2024-03-29T08:19:53.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.184+0000 7f94a4c00640 1 -- 172.21.15.46:0/2360352502 learned_addr learned my addr 172.21.15.46:0/2360352502 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:53.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.184+0000 7f949fe00640 1 --2- 172.21.15.46:0/2360352502 >> v2:172.21.15.73:3300/0 conn(0x7f94a015c950 0x7f94a006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:53.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.184+0000 7f949f400640 1 -- 172.21.15.46:0/2360352502 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592319979 0 0) 0x7f94a00cfb20 con 0x7f94a0160db0 2024-03-29T08:19:53.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.184+0000 7f949f400640 1 -- 172.21.15.46:0/2360352502 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9484003530 con 0x7f94a0160db0 2024-03-29T08:19:53.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.184+0000 7f949fe00640 1 -- 172.21.15.46:0/2360352502 >> v1:172.21.15.73:6789/0 conn(0x7f94a00cea30 legacy=0x7f94a01606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.184+0000 7f949fe00640 1 -- 172.21.15.46:0/2360352502 >> v1:172.21.15.46:6789/0 conn(0x7f94a0160db0 legacy=0x7f94a00cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.184+0000 7f949fe00640 1 -- 172.21.15.46:0/2360352502 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a00cf4d0 con 0x7f94a015c950 2024-03-29T08:19:53.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.188+0000 7f949fe00640 1 --2- 172.21.15.46:0/2360352502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a015c950 0x7f94a006bbe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f94900097c0 tx=0x7f949001ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8004aee36702c50 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:53.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.188+0000 7f949f400640 1 -- 172.21.15.46:0/2360352502 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94900a8990 con 0x7f94a015c950 2024-03-29T08:19:53.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.188+0000 7f949f400640 1 -- 172.21.15.46:0/2360352502 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94900a8af0 con 0x7f94a015c950 2024-03-29T08:19:53.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.188+0000 7f949f400640 1 -- 172.21.15.46:0/2360352502 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94900a8e60 con 0x7f94a015c950 2024-03-29T08:19:53.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.188+0000 7f94a6c00640 1 -- 172.21.15.46:0/2360352502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a015c950 msgr2=0x7f94a006bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.188+0000 7f94a6c00640 1 --2- 172.21.15.46:0/2360352502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a015c950 0x7f94a006bbe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f94900097c0 tx=0x7f949001ec90 comp rx=0 tx=0).stop 2024-03-29T08:19:53.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.188+0000 7f94a6c00640 1 -- 172.21.15.46:0/2360352502 shutdown_connections 2024-03-29T08:19:53.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.188+0000 7f94a6c00640 1 --2- 172.21.15.46:0/2360352502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a015c950 0x7f94a006bbe0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.188+0000 7f94a6c00640 1 -- 172.21.15.46:0/2360352502 >> 172.21.15.46:0/2360352502 conn(0x7f94a00b21b0 msgr2=0x7f94a00b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:53.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.188+0000 7f94a6c00640 1 -- 172.21.15.46:0/2360352502 shutdown_connections 2024-03-29T08:19:53.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.188+0000 7f94a6c00640 1 -- 172.21.15.46:0/2360352502 wait complete. 2024-03-29T08:19:53.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f94a6c00640 1 Processor -- start 2024-03-29T08:19:53.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f94a6c00640 1 -- start start 2024-03-29T08:19:53.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f94a6c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a00cea30 0x7f94a01557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:53.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f94a6c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a015c950 0x7f94a009c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:53.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f94a6c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f94a0160db0 0x7f94a009a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f94a6c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a009db00 con 0x7f94a00cea30 2024-03-29T08:19:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f94a6c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a009ce80 con 0x7f94a015c950 2024-03-29T08:19:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f94a6c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a009d160 con 0x7f94a0160db0 2024-03-29T08:19:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f949fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a00cea30 0x7f94a01557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f949fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a00cea30 0x7f94a01557e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45576/0 (socket says 172.21.15.46:45576) 2024-03-29T08:19:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f949fe00640 1 -- 172.21.15.46:0/930897606 learned_addr learned my addr 172.21.15.46:0/930897606 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f94a4c00640 1 --2- 172.21.15.46:0/930897606 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f94a0160db0 0x7f94a009a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f9497e00640 1 --2- 172.21.15.46:0/930897606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a015c950 0x7f94a009c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f949fe00640 1 -- 172.21.15.46:0/930897606 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f94a0160db0 msgr2=0x7f94a009a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f949fe00640 1 --2- 172.21.15.46:0/930897606 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f94a0160db0 0x7f94a009a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f949fe00640 1 -- 172.21.15.46:0/930897606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a015c950 msgr2=0x7f94a009c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f949fe00640 1 --2- 172.21.15.46:0/930897606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a015c950 0x7f94a009c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f949fe00640 1 -- 172.21.15.46:0/930897606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9490009420 con 0x7f94a00cea30 2024-03-29T08:19:53.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f949fe00640 1 --2- 172.21.15.46:0/930897606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a00cea30 0x7f94a01557e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f949001c8c0 tx=0x7f949001d4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:53.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f949cc00640 1 -- 172.21.15.46:0/930897606 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94900b1e00 con 0x7f94a00cea30 2024-03-29T08:19:53.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f949cc00640 1 -- 172.21.15.46:0/930897606 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9490006e20 con 0x7f94a00cea30 2024-03-29T08:19:53.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f94a6c00640 1 -- 172.21.15.46:0/930897606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a009d440 con 0x7f94a00cea30 2024-03-29T08:19:53.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f949cc00640 1 -- 172.21.15.46:0/930897606 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490004230 con 0x7f94a00cea30 2024-03-29T08:19:53.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f94a6c00640 1 -- 172.21.15.46:0/930897606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a0155d20 con 0x7f94a00cea30 2024-03-29T08:19:53.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.192+0000 7f94a6c00640 1 -- 172.21.15.46:0/930897606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a009e690 con 0x7f94a00cea30 2024-03-29T08:19:53.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.196+0000 7f949cc00640 1 -- 172.21.15.46:0/930897606 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9490004390 con 0x7f94a00cea30 2024-03-29T08:19:53.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.196+0000 7f949cc00640 1 --2- 172.21.15.46:0/930897606 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f947c066690 0x7f947c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:53.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.196+0000 7f949cc00640 1 -- 172.21.15.46:0/930897606 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9490121d50 con 0x7f94a00cea30 2024-03-29T08:19:53.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.196+0000 7f9497e00640 1 --2- 172.21.15.46:0/930897606 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f947c066690 0x7f947c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:53.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.196+0000 7f9497e00640 1 --2- 172.21.15.46:0/930897606 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f947c066690 0x7f947c068b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f94a00c7740 tx=0x7f9488009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:53.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.200+0000 7f949cc00640 1 -- 172.21.15.46:0/930897606 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f94900ef1d0 con 0x7f94a00cea30 2024-03-29T08:19:53.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.364+0000 7f94a6c00640 1 -- 172.21.15.46:0/930897606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f94a006c600 con 0x7f94a00cea30 2024-03-29T08:19:53.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.364+0000 7f949cc00640 1 -- 172.21.15.46:0/930897606 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f94900f27f0 con 0x7f94a00cea30 2024-03-29T08:19:53.365 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:19:53.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.364+0000 7f94a6c00640 1 -- 172.21.15.46:0/930897606 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f947c066690 msgr2=0x7f947c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.364+0000 7f94a6c00640 1 --2- 172.21.15.46:0/930897606 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f947c066690 0x7f947c068b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f94a00c7740 tx=0x7f9488009040 comp rx=0 tx=0).stop 2024-03-29T08:19:53.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.364+0000 7f94a6c00640 1 -- 172.21.15.46:0/930897606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a00cea30 msgr2=0x7f94a01557e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.364+0000 7f94a6c00640 1 --2- 172.21.15.46:0/930897606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a00cea30 0x7f94a01557e0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f949001c8c0 tx=0x7f949001d4d0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.364+0000 7f94a6c00640 1 -- 172.21.15.46:0/930897606 shutdown_connections 2024-03-29T08:19:53.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.364+0000 7f94a6c00640 1 --2- 172.21.15.46:0/930897606 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f947c066690 0x7f947c068b50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.364+0000 7f94a6c00640 1 --2- 172.21.15.46:0/930897606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a015c950 0x7f94a009c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.364+0000 7f94a6c00640 1 --2- 172.21.15.46:0/930897606 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f94a0160db0 0x7f94a009a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.364+0000 7f94a6c00640 1 --2- 172.21.15.46:0/930897606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a00cea30 0x7f94a01557e0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.364+0000 7f94a6c00640 1 -- 172.21.15.46:0/930897606 >> 172.21.15.46:0/930897606 conn(0x7f94a00b21b0 msgr2=0x7f94a0157ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:53.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.364+0000 7f94a6c00640 1 -- 172.21.15.46:0/930897606 shutdown_connections 2024-03-29T08:19:53.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.364+0000 7f94a6c00640 1 -- 172.21.15.46:0/930897606 wait complete. 2024-03-29T08:19:53.386 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:19:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.528+0000 7f7c40a00640 1 Processor -- start 2024-03-29T08:19:53.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.528+0000 7f7c40a00640 1 -- start start 2024-03-29T08:19:53.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.528+0000 7f7c40a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7c3c0cbbe0 0x7f7c3c0d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:53.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.528+0000 7f7c40a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c3c0cc660 con 0x7f7c3c15a790 2024-03-29T08:19:53.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.528+0000 7f7c40a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c3c0cc0a0 con 0x7f7c3c15ab70 2024-03-29T08:19:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.528+0000 7f7c40a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c3c0cc380 con 0x7f7c3c0cbbe0 2024-03-29T08:19:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.528+0000 7f7c31400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7c3c0cbbe0 0x7f7c3c0d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.528+0000 7f7c39e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7c3c15a790 0x7f7c3c0cee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44884/0 (socket says 172.21.15.46:44884) 2024-03-29T08:19:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.528+0000 7f7c39e00640 1 -- 172.21.15.46:0/630500041 learned_addr learned my addr 172.21.15.46:0/630500041 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.528+0000 7f7c31400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7c3c0cbbe0 0x7f7c3c0d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45592/0 (socket says 172.21.15.46:45592) 2024-03-29T08:19:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.528+0000 7f7c31400640 1 -- 172.21.15.46:0/630500041 >> v1:172.21.15.73:6789/0 conn(0x7f7c3c15ab70 legacy=0x7f7c3c0b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.528+0000 7f7c31400640 1 -- 172.21.15.46:0/630500041 >> v1:172.21.15.46:6789/0 conn(0x7f7c3c15a790 legacy=0x7f7c3c0cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.528+0000 7f7c31400640 1 -- 172.21.15.46:0/630500041 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3c0b0380 con 0x7f7c3c0cbbe0 2024-03-29T08:19:53.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.528+0000 7f7c39400640 1 -- 172.21.15.46:0/630500041 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352211181 0 0) 0x7f7c3c0cc660 con 0x7f7c3c15a790 2024-03-29T08:19:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.532+0000 7f7c31400640 1 --2- 172.21.15.46:0/630500041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c0cbbe0 0x7f7c3c0d0650 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7c240097c0 tx=0x7f7c2401c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561e5b4c6d7fe88c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.532+0000 7f7c39400640 1 -- 172.21.15.46:0/630500041 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c240ab600 con 0x7f7c3c0cbbe0 2024-03-29T08:19:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.532+0000 7f7c39400640 1 -- 172.21.15.46:0/630500041 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c240a7050 con 0x7f7c3c0cbbe0 2024-03-29T08:19:53.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.532+0000 7f7c39400640 1 -- 172.21.15.46:0/630500041 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c240ab980 con 0x7f7c3c0cbbe0 2024-03-29T08:19:53.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.532+0000 7f7c40a00640 1 -- 172.21.15.46:0/630500041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c0cbbe0 msgr2=0x7f7c3c0d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.532+0000 7f7c40a00640 1 --2- 172.21.15.46:0/630500041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c0cbbe0 0x7f7c3c0d0650 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7c240097c0 tx=0x7f7c2401c350 comp rx=0 tx=0).stop 2024-03-29T08:19:53.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.532+0000 7f7c40a00640 1 -- 172.21.15.46:0/630500041 shutdown_connections 2024-03-29T08:19:53.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.532+0000 7f7c40a00640 1 --2- 172.21.15.46:0/630500041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c0cbbe0 0x7f7c3c0d0650 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.532+0000 7f7c40a00640 1 -- 172.21.15.46:0/630500041 >> 172.21.15.46:0/630500041 conn(0x7f7c3c099a10 msgr2=0x7f7c3c089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:53.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.532+0000 7f7c40a00640 1 -- 172.21.15.46:0/630500041 shutdown_connections 2024-03-29T08:19:53.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.532+0000 7f7c40a00640 1 -- 172.21.15.46:0/630500041 wait complete. 2024-03-29T08:19:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c40a00640 1 Processor -- start 2024-03-29T08:19:53.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c40a00640 1 -- start start 2024-03-29T08:19:53.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c40a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c3c0cbbe0 0x7f7c3c0c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:53.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c40a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c3c15a790 0x7f7c3c0c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:53.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c40a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c15ab70 0x7f7c3c14cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:53.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c40a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c0c4910 con 0x7f7c3c15ab70 2024-03-29T08:19:53.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c40a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c0c5df0 con 0x7f7c3c15a790 2024-03-29T08:19:53.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c40a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c0ca480 con 0x7f7c3c0cbbe0 2024-03-29T08:19:53.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c3a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c15ab70 0x7f7c3c14cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:53.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c3a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c15ab70 0x7f7c3c14cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45598/0 (socket says 172.21.15.46:45598) 2024-03-29T08:19:53.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c3a800640 1 -- 172.21.15.46:0/2656274701 learned_addr learned my addr 172.21.15.46:0/2656274701 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:53.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c31400640 1 --2- 172.21.15.46:0/2656274701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c3c15a790 0x7f7c3c0c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:53.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c39e00640 1 --2- 172.21.15.46:0/2656274701 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c3c0cbbe0 0x7f7c3c0c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:53.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c3a800640 1 -- 172.21.15.46:0/2656274701 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c3c0cbbe0 msgr2=0x7f7c3c0c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c3a800640 1 --2- 172.21.15.46:0/2656274701 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c3c0cbbe0 0x7f7c3c0c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c3a800640 1 -- 172.21.15.46:0/2656274701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c3c15a790 msgr2=0x7f7c3c0c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c3a800640 1 --2- 172.21.15.46:0/2656274701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c3c15a790 0x7f7c3c0c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c3a800640 1 -- 172.21.15.46:0/2656274701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c24009420 con 0x7f7c3c15ab70 2024-03-29T08:19:53.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c3a800640 1 --2- 172.21.15.46:0/2656274701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c15ab70 0x7f7c3c14cd00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400c310 tx=0x7f7c3400c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:53.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c32a00640 1 -- 172.21.15.46:0/2656274701 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c340052d0 con 0x7f7c3c15ab70 2024-03-29T08:19:53.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c32a00640 1 -- 172.21.15.46:0/2656274701 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c34005430 con 0x7f7c3c15ab70 2024-03-29T08:19:53.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c32a00640 1 -- 172.21.15.46:0/2656274701 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c34003800 con 0x7f7c3c15ab70 2024-03-29T08:19:53.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c40a00640 1 -- 172.21.15.46:0/2656274701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3c0c3f70 con 0x7f7c3c15ab70 2024-03-29T08:19:53.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.536+0000 7f7c40a00640 1 -- 172.21.15.46:0/2656274701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3c0c4530 con 0x7f7c3c15ab70 2024-03-29T08:19:53.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.540+0000 7f7c40a00640 1 -- 172.21.15.46:0/2656274701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c3c09e7d0 con 0x7f7c3c15ab70 2024-03-29T08:19:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.540+0000 7f7c32a00640 1 -- 172.21.15.46:0/2656274701 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7c34003960 con 0x7f7c3c15ab70 2024-03-29T08:19:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.540+0000 7f7c32a00640 1 --2- 172.21.15.46:0/2656274701 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c18066690 0x7f7c18068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:53.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.540+0000 7f7c32a00640 1 -- 172.21.15.46:0/2656274701 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c340833f0 con 0x7f7c3c15ab70 2024-03-29T08:19:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.540+0000 7f7c39e00640 1 --2- 172.21.15.46:0/2656274701 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c18066690 0x7f7c18068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:53.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.540+0000 7f7c39e00640 1 --2- 172.21.15.46:0/2656274701 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c18066690 0x7f7c18068b50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0012f0 tx=0x7f7c2c004d50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:53.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.544+0000 7f7c32a00640 1 -- 172.21.15.46:0/2656274701 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7c34050960 con 0x7f7c3c15ab70 2024-03-29T08:19:53.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.708+0000 7f7c40a00640 1 -- 172.21.15.46:0/2656274701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7c3c0d0610 con 0x7f7c3c15ab70 2024-03-29T08:19:53.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.708+0000 7f7c32a00640 1 -- 172.21.15.46:0/2656274701 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7c34053f80 con 0x7f7c3c15ab70 2024-03-29T08:19:53.710 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:19:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.708+0000 7f7c40a00640 1 -- 172.21.15.46:0/2656274701 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c18066690 msgr2=0x7f7c18068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.708+0000 7f7c40a00640 1 --2- 172.21.15.46:0/2656274701 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c18066690 0x7f7c18068b50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c0012f0 tx=0x7f7c2c004d50 comp rx=0 tx=0).stop 2024-03-29T08:19:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.708+0000 7f7c40a00640 1 -- 172.21.15.46:0/2656274701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c15ab70 msgr2=0x7f7c3c14cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.708+0000 7f7c40a00640 1 --2- 172.21.15.46:0/2656274701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c15ab70 0x7f7c3c14cd00 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400c310 tx=0x7f7c3400c710 comp rx=0 tx=0).stop 2024-03-29T08:19:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.708+0000 7f7c40a00640 1 -- 172.21.15.46:0/2656274701 shutdown_connections 2024-03-29T08:19:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.708+0000 7f7c40a00640 1 --2- 172.21.15.46:0/2656274701 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7c18066690 0x7f7c18068b50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.708+0000 7f7c40a00640 1 --2- 172.21.15.46:0/2656274701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c3c15a790 0x7f7c3c0c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.708+0000 7f7c40a00640 1 --2- 172.21.15.46:0/2656274701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c3c15ab70 0x7f7c3c14cd00 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.708+0000 7f7c40a00640 1 --2- 172.21.15.46:0/2656274701 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7c3c0cbbe0 0x7f7c3c0c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.708+0000 7f7c40a00640 1 -- 172.21.15.46:0/2656274701 >> 172.21.15.46:0/2656274701 conn(0x7f7c3c099a10 msgr2=0x7f7c3c06ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.708+0000 7f7c40a00640 1 -- 172.21.15.46:0/2656274701 shutdown_connections 2024-03-29T08:19:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.712+0000 7f7c40a00640 1 -- 172.21.15.46:0/2656274701 wait complete. 2024-03-29T08:19:53.731 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:19:53.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.868+0000 7efe22c00640 1 Processor -- start 2024-03-29T08:19:53.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.872+0000 7efe22c00640 1 -- start start 2024-03-29T08:19:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.872+0000 7efe22c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efe1c15a890 0x7efe1c0cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.872+0000 7efe22c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe1c069d90 0x7efe1c15e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.872+0000 7efe22c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe1c0aef30 con 0x7efe1c0b02b0 2024-03-29T08:19:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.872+0000 7efe22c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe1c15bc80 con 0x7efe1c15a890 2024-03-29T08:19:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.872+0000 7efe22c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe1c0aec00 con 0x7efe1c069d90 2024-03-29T08:19:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.872+0000 7efe21800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efe1c15a890 0x7efe1c0cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.872+0000 7efe21800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efe1c15a890 0x7efe1c0cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45608/0 (socket says 172.21.15.46:45608) 2024-03-29T08:19:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.872+0000 7efe21800640 1 -- 172.21.15.46:0/1764973791 learned_addr learned my addr 172.21.15.46:0/1764973791 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.872+0000 7efe20e00640 1 --2- 172.21.15.46:0/1764973791 >> v2:172.21.15.73:3300/0 conn(0x7efe1c069d90 0x7efe1c15e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.872+0000 7efe21800640 1 -- 172.21.15.46:0/1764973791 >> v2:172.21.15.73:3300/0 conn(0x7efe1c069d90 msgr2=0x7efe1c15e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.872+0000 7efe21800640 1 --2- 172.21.15.46:0/1764973791 >> v2:172.21.15.73:3300/0 conn(0x7efe1c069d90 0x7efe1c15e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.872+0000 7efe21800640 1 -- 172.21.15.46:0/1764973791 >> v1:172.21.15.73:6789/0 conn(0x7efe1c0b02b0 legacy=0x7efe1c0af3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.872+0000 7efe21800640 1 -- 172.21.15.46:0/1764973791 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c0cb550 con 0x7efe1c15a890 2024-03-29T08:19:53.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.876+0000 7efe21800640 1 --2- 172.21.15.46:0/1764973791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe1c15a890 0x7efe1c0cb010 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efe1c09d4a0 tx=0x7efe1001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7951914d4ef06d97 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:53.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.876+0000 7efe1be00640 1 -- 172.21.15.46:0/1764973791 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe100aa650 con 0x7efe1c15a890 2024-03-29T08:19:53.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.876+0000 7efe1be00640 1 -- 172.21.15.46:0/1764973791 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe100aa7b0 con 0x7efe1c15a890 2024-03-29T08:19:53.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.876+0000 7efe1be00640 1 -- 172.21.15.46:0/1764973791 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe100aab40 con 0x7efe1c15a890 2024-03-29T08:19:53.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.876+0000 7efe22c00640 1 -- 172.21.15.46:0/1764973791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe1c15a890 msgr2=0x7efe1c0cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.876+0000 7efe22c00640 1 --2- 172.21.15.46:0/1764973791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe1c15a890 0x7efe1c0cb010 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efe1c09d4a0 tx=0x7efe1001c4f0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.876+0000 7efe22c00640 1 -- 172.21.15.46:0/1764973791 shutdown_connections 2024-03-29T08:19:53.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.876+0000 7efe22c00640 1 --2- 172.21.15.46:0/1764973791 >> v2:172.21.15.73:3300/0 conn(0x7efe1c069d90 0x7efe1c15e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.876+0000 7efe22c00640 1 --2- 172.21.15.46:0/1764973791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe1c15a890 0x7efe1c0cb010 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.876+0000 7efe22c00640 1 -- 172.21.15.46:0/1764973791 >> 172.21.15.46:0/1764973791 conn(0x7efe1c098a40 msgr2=0x7efe1c0cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:53.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.876+0000 7efe22c00640 1 -- 172.21.15.46:0/1764973791 shutdown_connections 2024-03-29T08:19:53.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.876+0000 7efe22c00640 1 -- 172.21.15.46:0/1764973791 wait complete. 2024-03-29T08:19:53.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.876+0000 7efe22c00640 1 Processor -- start 2024-03-29T08:19:53.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.876+0000 7efe22c00640 1 -- start start 2024-03-29T08:19:53.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe22c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe1c069d90 0x7efe1c1499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:53.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe22c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe1c0b02b0 0x7efe1c1638e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:53.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe22c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe1c15a890 0x7efe1c06d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:53.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe22c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c0cdda0 con 0x7efe1c069d90 2024-03-29T08:19:53.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe22c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c164ed0 con 0x7efe1c0b02b0 2024-03-29T08:19:53.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe22c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1c165360 con 0x7efe1c15a890 2024-03-29T08:19:53.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe21800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe1c069d90 0x7efe1c1499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:53.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe21800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe1c069d90 0x7efe1c1499a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45624/0 (socket says 172.21.15.46:45624) 2024-03-29T08:19:53.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe21800640 1 -- 172.21.15.46:0/1439713561 learned_addr learned my addr 172.21.15.46:0/1439713561 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe20e00640 1 --2- 172.21.15.46:0/1439713561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe1c0b02b0 0x7efe1c1638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe22200640 1 --2- 172.21.15.46:0/1439713561 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe1c15a890 0x7efe1c06d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe21800640 1 -- 172.21.15.46:0/1439713561 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe1c15a890 msgr2=0x7efe1c06d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe21800640 1 --2- 172.21.15.46:0/1439713561 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe1c15a890 0x7efe1c06d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe21800640 1 -- 172.21.15.46:0/1439713561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe1c0b02b0 msgr2=0x7efe1c1638e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe21800640 1 --2- 172.21.15.46:0/1439713561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe1c0b02b0 0x7efe1c1638e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe21800640 1 -- 172.21.15.46:0/1439713561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe100094d0 con 0x7efe1c069d90 2024-03-29T08:19:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe22200640 1 --2- 172.21.15.46:0/1439713561 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe1c15a890 0x7efe1c06d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe21800640 1 --2- 172.21.15.46:0/1439713561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe1c069d90 0x7efe1c1499a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efe100ac040 tx=0x7efe1001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe19600640 1 -- 172.21.15.46:0/1439713561 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe100b3800 con 0x7efe1c069d90 2024-03-29T08:19:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe19600640 1 -- 172.21.15.46:0/1439713561 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe100b3960 con 0x7efe1c069d90 2024-03-29T08:19:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe22c00640 1 -- 172.21.15.46:0/1439713561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1c1642c0 con 0x7efe1c069d90 2024-03-29T08:19:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe19600640 1 -- 172.21.15.46:0/1439713561 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe100b3d10 con 0x7efe1c069d90 2024-03-29T08:19:53.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe22c00640 1 -- 172.21.15.46:0/1439713561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1c164880 con 0x7efe1c069d90 2024-03-29T08:19:53.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe22c00640 1 -- 172.21.15.46:0/1439713561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe1c0c6320 con 0x7efe1c069d90 2024-03-29T08:19:53.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.880+0000 7efe19600640 1 -- 172.21.15.46:0/1439713561 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efe100b2380 con 0x7efe1c069d90 2024-03-29T08:19:53.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.884+0000 7efe19600640 1 --2- 172.21.15.46:0/1439713561 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efdfc066690 0x7efdfc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:53.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.884+0000 7efe19600640 1 -- 172.21.15.46:0/1439713561 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe1011e860 con 0x7efe1c069d90 2024-03-29T08:19:53.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.884+0000 7efe20e00640 1 --2- 172.21.15.46:0/1439713561 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efdfc066690 0x7efdfc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:53.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.884+0000 7efe20e00640 1 --2- 172.21.15.46:0/1439713561 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efdfc066690 0x7efdfc068b50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efe1c159cf0 tx=0x7efe08009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:53.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:53.888+0000 7efe19600640 1 -- 172.21.15.46:0/1439713561 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efe100ebbe0 con 0x7efe1c069d90 2024-03-29T08:19:54.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.084+0000 7efe22c00640 1 -- 172.21.15.46:0/1439713561 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe1c0d0360 con 0x7efe1c069d90 2024-03-29T08:19:54.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.084+0000 7efe19600640 1 -- 172.21.15.46:0/1439713561 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4734 (secure 0 0 0) 0x7efe100aa610 con 0x7efe1c069d90 2024-03-29T08:19:54.113 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:54.113 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/4274575126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4274575126},{"type":"v1","addr":"172.21.15.46:6834","nonce":4274575126}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3739899623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3739899623},{"type":"v1","addr":"172.21.15.73:6837","nonce":3739899623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1905427475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1905427475},{"type":"v1","addr":"172.21.15.46:6835","nonce":1905427475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:19:14.663142+0000","modified":"2024-03-29T08:19:17.680359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5152},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5152":{"gid":5152,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/1941408960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1941408960},{"type":"v1","addr":"172.21.15.73:6836","nonce":1941408960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-29T08:19:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2024-03-29T08:19:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.084+0000 7efe22c00640 1 -- 172.21.15.46:0/1439713561 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efdfc066690 msgr2=0x7efdfc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.084+0000 7efe22c00640 1 --2- 172.21.15.46:0/1439713561 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efdfc066690 0x7efdfc068b50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7efe1c159cf0 tx=0x7efe08009190 comp rx=0 tx=0).stop 2024-03-29T08:19:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.084+0000 7efe22c00640 1 -- 172.21.15.46:0/1439713561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe1c069d90 msgr2=0x7efe1c1499a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.084+0000 7efe22c00640 1 --2- 172.21.15.46:0/1439713561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe1c069d90 0x7efe1c1499a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7efe100ac040 tx=0x7efe1001ad30 comp rx=0 tx=0).stop 2024-03-29T08:19:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.084+0000 7efe22c00640 1 -- 172.21.15.46:0/1439713561 shutdown_connections 2024-03-29T08:19:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.084+0000 7efe22c00640 1 --2- 172.21.15.46:0/1439713561 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efdfc066690 0x7efdfc068b50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.084+0000 7efe22c00640 1 --2- 172.21.15.46:0/1439713561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe1c0b02b0 0x7efe1c1638e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.084+0000 7efe22c00640 1 --2- 172.21.15.46:0/1439713561 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe1c15a890 0x7efe1c06d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.084+0000 7efe22c00640 1 --2- 172.21.15.46:0/1439713561 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe1c069d90 0x7efe1c1499a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.084+0000 7efe22c00640 1 -- 172.21.15.46:0/1439713561 >> 172.21.15.46:0/1439713561 conn(0x7efe1c098a40 msgr2=0x7efe1c0af3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:54.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.084+0000 7efe22c00640 1 -- 172.21.15.46:0/1439713561 shutdown_connections 2024-03-29T08:19:54.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.084+0000 7efe22c00640 1 -- 172.21.15.46:0/1439713561 wait complete. 2024-03-29T08:19:54.116 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:19:54.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0de200640 1 Processor -- start 2024-03-29T08:19:54.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0de200640 1 -- start start 2024-03-29T08:19:54.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0de200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd0d8156e80 0x7fd0d815d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0de200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0d80b1d30 0x7fd0d80b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0de200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd0d81604c0 0x7fd0d8160930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0de200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d80b26f0 con 0x7fd0d81604c0 2024-03-29T08:19:54.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0de200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d8066100 con 0x7fd0d8156e80 2024-03-29T08:19:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0de200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d815de60 con 0x7fd0d80b1d30 2024-03-29T08:19:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0dd800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd0d81604c0 0x7fd0d8160930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0dd800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd0d81604c0 0x7fd0d8160930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45630/0 (socket says 172.21.15.46:45630) 2024-03-29T08:19:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0dd800640 1 -- 172.21.15.46:0/515864015 learned_addr learned my addr 172.21.15.46:0/515864015 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0d7e00640 1 --2- 172.21.15.46:0/515864015 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0d80b1d30 0x7fd0d80b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0dce00640 1 --2- 172.21.15.46:0/515864015 >> v2:172.21.15.73:3300/0 conn(0x7fd0d8156e80 0x7fd0d815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0dd800640 1 -- 172.21.15.46:0/515864015 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0d80b1d30 msgr2=0x7fd0d80b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0dd800640 1 --2- 172.21.15.46:0/515864015 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0d80b1d30 0x7fd0d80b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0dd800640 1 -- 172.21.15.46:0/515864015 >> v2:172.21.15.73:3300/0 conn(0x7fd0d8156e80 msgr2=0x7fd0d815d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0dd800640 1 --2- 172.21.15.46:0/515864015 >> v2:172.21.15.73:3300/0 conn(0x7fd0d8156e80 0x7fd0d815d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.256+0000 7fd0dd800640 1 -- 172.21.15.46:0/515864015 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d8065e20 con 0x7fd0d81604c0 2024-03-29T08:19:54.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.260+0000 7fd0dd800640 1 --2- 172.21.15.46:0/515864015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0d81604c0 0x7fd0d8160930 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80cc4e0 tx=0x7fd0d001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c93714235952a48 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:54.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.260+0000 7fd0d7400640 1 -- 172.21.15.46:0/515864015 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d00a57b0 con 0x7fd0d81604c0 2024-03-29T08:19:54.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.260+0000 7fd0d7400640 1 -- 172.21.15.46:0/515864015 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d00a5910 con 0x7fd0d81604c0 2024-03-29T08:19:54.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.260+0000 7fd0d7400640 1 -- 172.21.15.46:0/515864015 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d00a5ca0 con 0x7fd0d81604c0 2024-03-29T08:19:54.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.260+0000 7fd0de200640 1 -- 172.21.15.46:0/515864015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0d81604c0 msgr2=0x7fd0d8160930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:54.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.260+0000 7fd0de200640 1 --2- 172.21.15.46:0/515864015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0d81604c0 0x7fd0d8160930 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80cc4e0 tx=0x7fd0d001c930 comp rx=0 tx=0).stop 2024-03-29T08:19:54.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.260+0000 7fd0de200640 1 -- 172.21.15.46:0/515864015 shutdown_connections 2024-03-29T08:19:54.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.260+0000 7fd0de200640 1 --2- 172.21.15.46:0/515864015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0d81604c0 0x7fd0d8160930 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.260+0000 7fd0de200640 1 --2- 172.21.15.46:0/515864015 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0d80b1d30 0x7fd0d80b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.260+0000 7fd0de200640 1 --2- 172.21.15.46:0/515864015 >> v2:172.21.15.73:3300/0 conn(0x7fd0d8156e80 0x7fd0d815d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.260+0000 7fd0de200640 1 -- 172.21.15.46:0/515864015 >> 172.21.15.46:0/515864015 conn(0x7fd0d80cb8a0 msgr2=0x7fd0d80cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:54.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.260+0000 7fd0de200640 1 -- 172.21.15.46:0/515864015 shutdown_connections 2024-03-29T08:19:54.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.260+0000 7fd0de200640 1 -- 172.21.15.46:0/515864015 wait complete. 2024-03-29T08:19:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0de200640 1 Processor -- start 2024-03-29T08:19:54.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0de200640 1 -- start start 2024-03-29T08:19:54.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0de200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0d80b1d30 0x7fd0d814d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0de200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0d8156e80 0x7fd0d80ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0de200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0d81604c0 0x7fd0d814aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0de200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d814bd20 con 0x7fd0d80b1d30 2024-03-29T08:19:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0de200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d814e1c0 con 0x7fd0d8156e80 2024-03-29T08:19:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0de200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d814e4a0 con 0x7fd0d81604c0 2024-03-29T08:19:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0dce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0d80b1d30 0x7fd0d814d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0dce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0d80b1d30 0x7fd0d814d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45638/0 (socket says 172.21.15.46:45638) 2024-03-29T08:19:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0dce00640 1 -- 172.21.15.46:0/1469789226 learned_addr learned my addr 172.21.15.46:0/1469789226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0d7e00640 1 --2- 172.21.15.46:0/1469789226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0d8156e80 0x7fd0d80ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0dd800640 1 --2- 172.21.15.46:0/1469789226 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0d81604c0 0x7fd0d814aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0dce00640 1 -- 172.21.15.46:0/1469789226 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0d81604c0 msgr2=0x7fd0d814aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0dce00640 1 --2- 172.21.15.46:0/1469789226 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0d81604c0 0x7fd0d814aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0dce00640 1 -- 172.21.15.46:0/1469789226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0d8156e80 msgr2=0x7fd0d80ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0dce00640 1 --2- 172.21.15.46:0/1469789226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0d8156e80 0x7fd0d80ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0dce00640 1 -- 172.21.15.46:0/1469789226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d0009b30 con 0x7fd0d80b1d30 2024-03-29T08:19:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0dd800640 1 --2- 172.21.15.46:0/1469789226 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0d81604c0 0x7fd0d814aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:54.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0d7e00640 1 --2- 172.21.15.46:0/1469789226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0d8156e80 0x7fd0d80ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0dce00640 1 --2- 172.21.15.46:0/1469789226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0d80b1d30 0x7fd0d814d480 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00e930 tx=0x7fd0cc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0d4c00640 1 -- 172.21.15.46:0/1469789226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc009690 con 0x7fd0d80b1d30 2024-03-29T08:19:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0d4c00640 1 -- 172.21.15.46:0/1469789226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0cc00ca00 con 0x7fd0d80b1d30 2024-03-29T08:19:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0d4c00640 1 -- 172.21.15.46:0/1469789226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc00cd70 con 0x7fd0d80b1d30 2024-03-29T08:19:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0de200640 1 -- 172.21.15.46:0/1469789226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d814a510 con 0x7fd0d80b1d30 2024-03-29T08:19:54.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0de200640 1 -- 172.21.15.46:0/1469789226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d814aad0 con 0x7fd0d80b1d30 2024-03-29T08:19:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.264+0000 7fd0de200640 1 -- 172.21.15.46:0/1469789226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d804e040 con 0x7fd0d80b1d30 2024-03-29T08:19:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.268+0000 7fd0d4c00640 1 -- 172.21.15.46:0/1469789226 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd0cc013470 con 0x7fd0d80b1d30 2024-03-29T08:19:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.268+0000 7fd0d4c00640 1 --2- 172.21.15.46:0/1469789226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd0b80666e0 0x7fd0b8068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.268+0000 7fd0d4c00640 1 -- 172.21.15.46:0/1469789226 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0cc084420 con 0x7fd0d80b1d30 2024-03-29T08:19:54.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.268+0000 7fd0d7e00640 1 --2- 172.21.15.46:0/1469789226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd0b80666e0 0x7fd0b8068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.268+0000 7fd0d7e00640 1 --2- 172.21.15.46:0/1469789226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd0b80666e0 0x7fd0b8068ba0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd0d806b320 tx=0x7fd0c4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:54.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.272+0000 7fd0d4c00640 1 -- 172.21.15.46:0/1469789226 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd0cc051830 con 0x7fd0d80b1d30 2024-03-29T08:19:54.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.436+0000 7fd0de200640 1 -- 172.21.15.46:0/1469789226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0d80cc960 con 0x7fd0d80b1d30 2024-03-29T08:19:54.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.436+0000 7fd0d4c00640 1 -- 172.21.15.46:0/1469789226 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+15993 (secure 0 0 0) 0x7fd0cc054e50 con 0x7fd0d80b1d30 2024-03-29T08:19:54.439 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:54.440 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:19:15.665674+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:19:12.674055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-29T08:19:13.661287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:19:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.440+0000 7fd0de200640 1 -- 172.21.15.46:0/1469789226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd0b80666e0 msgr2=0x7fd0b8068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.440+0000 7fd0de200640 1 --2- 172.21.15.46:0/1469789226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd0b80666e0 0x7fd0b8068ba0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd0d806b320 tx=0x7fd0c4008040 comp rx=0 tx=0).stop 2024-03-29T08:19:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.440+0000 7fd0de200640 1 -- 172.21.15.46:0/1469789226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0d80b1d30 msgr2=0x7fd0d814d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.440+0000 7fd0de200640 1 --2- 172.21.15.46:0/1469789226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0d80b1d30 0x7fd0d814d480 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00e930 tx=0x7fd0cc00edf0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.440+0000 7fd0de200640 1 -- 172.21.15.46:0/1469789226 shutdown_connections 2024-03-29T08:19:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.440+0000 7fd0de200640 1 --2- 172.21.15.46:0/1469789226 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd0b80666e0 0x7fd0b8068ba0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.440+0000 7fd0de200640 1 --2- 172.21.15.46:0/1469789226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0d8156e80 0x7fd0d80ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.440+0000 7fd0de200640 1 --2- 172.21.15.46:0/1469789226 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0d81604c0 0x7fd0d814aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.440+0000 7fd0de200640 1 --2- 172.21.15.46:0/1469789226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0d80b1d30 0x7fd0d814d480 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.440+0000 7fd0de200640 1 -- 172.21.15.46:0/1469789226 >> 172.21.15.46:0/1469789226 conn(0x7fd0d80cb8a0 msgr2=0x7fd0d806eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.440+0000 7fd0de200640 1 -- 172.21.15.46:0/1469789226 shutdown_connections 2024-03-29T08:19:54.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.440+0000 7fd0de200640 1 -- 172.21.15.46:0/1469789226 wait complete. 2024-03-29T08:19:54.460 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T08:19:54.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:19:54.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.600+0000 7fa63a400640 1 Processor -- start 2024-03-29T08:19:54.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.600+0000 7fa63a400640 1 -- start start 2024-03-29T08:19:54.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.600+0000 7fa63a400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa6341593d0 0x7fa63406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.600+0000 7fa63a400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa63415cd90 0x7fa6341597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.600+0000 7fa63a400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63406d130 con 0x7fa63415d230 2024-03-29T08:19:54.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.600+0000 7fa63a400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63415b8f0 con 0x7fa6341593d0 2024-03-29T08:19:54.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.600+0000 7fa63a400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63415ad50 con 0x7fa63415cd90 2024-03-29T08:19:54.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.600+0000 7fa633400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa6341593d0 0x7fa63406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.600+0000 7fa62be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa63415d230 0x7fa63415d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44886/0 (socket says 172.21.15.46:44886) 2024-03-29T08:19:54.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.600+0000 7fa62be00640 1 -- 172.21.15.46:0/3046383535 learned_addr learned my addr 172.21.15.46:0/3046383535 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.600+0000 7fa633e00640 1 --2- 172.21.15.46:0/3046383535 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa63415cd90 0x7fa6341597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.600+0000 7fa632a00640 1 -- 172.21.15.46:0/3046383535 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1524870981 0 0) 0x7fa63406d130 con 0x7fa63415d230 2024-03-29T08:19:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.600+0000 7fa633400640 1 -- 172.21.15.46:0/3046383535 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa63415cd90 msgr2=0x7fa6341597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.600+0000 7fa633400640 1 --2- 172.21.15.46:0/3046383535 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa63415cd90 0x7fa6341597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.600+0000 7fa633400640 1 -- 172.21.15.46:0/3046383535 >> v1:172.21.15.46:6789/0 conn(0x7fa63415d230 legacy=0x7fa63415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:54.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.600+0000 7fa633400640 1 -- 172.21.15.46:0/3046383535 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63415afc0 con 0x7fa6341593d0 2024-03-29T08:19:54.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.604+0000 7fa633400640 1 --2- 172.21.15.46:0/3046383535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6341593d0 0x7fa63406cbf0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa6240097c0 tx=0x7fa62401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b420e806e89507 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:54.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.604+0000 7fa632a00640 1 -- 172.21.15.46:0/3046383535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6240a6040 con 0x7fa6341593d0 2024-03-29T08:19:54.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.604+0000 7fa632a00640 1 -- 172.21.15.46:0/3046383535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6240aa600 con 0x7fa6341593d0 2024-03-29T08:19:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.604+0000 7fa632a00640 1 -- 172.21.15.46:0/3046383535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6240aa990 con 0x7fa6341593d0 2024-03-29T08:19:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.604+0000 7fa63a400640 1 -- 172.21.15.46:0/3046383535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6341593d0 msgr2=0x7fa63406cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.604+0000 7fa63a400640 1 --2- 172.21.15.46:0/3046383535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6341593d0 0x7fa63406cbf0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa6240097c0 tx=0x7fa62401c4a0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.604+0000 7fa63a400640 1 -- 172.21.15.46:0/3046383535 shutdown_connections 2024-03-29T08:19:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.604+0000 7fa63a400640 1 --2- 172.21.15.46:0/3046383535 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa63415cd90 0x7fa6341597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.604+0000 7fa63a400640 1 --2- 172.21.15.46:0/3046383535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6341593d0 0x7fa63406cbf0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.604+0000 7fa63a400640 1 -- 172.21.15.46:0/3046383535 >> 172.21.15.46:0/3046383535 conn(0x7fa634066110 msgr2=0x7fa634069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.604+0000 7fa63a400640 1 -- 172.21.15.46:0/3046383535 shutdown_connections 2024-03-29T08:19:54.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.604+0000 7fa63a400640 1 -- 172.21.15.46:0/3046383535 wait complete. 2024-03-29T08:19:54.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.604+0000 7fa63a400640 1 Processor -- start 2024-03-29T08:19:54.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.604+0000 7fa63a400640 1 -- start start 2024-03-29T08:19:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa63a400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6341593d0 0x7fa634151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa63a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63415cd90 0x7fa6341574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa63a400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa63415d230 0x7fa63414c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa63a400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63414c6b0 con 0x7fa63415cd90 2024-03-29T08:19:54.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa63a400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa634152a50 con 0x7fa63415d230 2024-03-29T08:19:54.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa63a400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa634152d30 con 0x7fa6341593d0 2024-03-29T08:19:54.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa62be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63415cd90 0x7fa6341574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa62be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63415cd90 0x7fa6341574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45648/0 (socket says 172.21.15.46:45648) 2024-03-29T08:19:54.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa62be00640 1 -- 172.21.15.46:0/390113466 learned_addr learned my addr 172.21.15.46:0/390113466 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:54.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa633e00640 1 --2- 172.21.15.46:0/390113466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa63415d230 0x7fa63414c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa633400640 1 --2- 172.21.15.46:0/390113466 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6341593d0 0x7fa634151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa62be00640 1 -- 172.21.15.46:0/390113466 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6341593d0 msgr2=0x7fa634151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:54.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa62be00640 1 --2- 172.21.15.46:0/390113466 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6341593d0 0x7fa634151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa62be00640 1 -- 172.21.15.46:0/390113466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa63415d230 msgr2=0x7fa63414c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:54.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa62be00640 1 --2- 172.21.15.46:0/390113466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa63415d230 0x7fa63414c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa62be00640 1 -- 172.21.15.46:0/390113466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa624009420 con 0x7fa63415cd90 2024-03-29T08:19:54.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa633400640 1 --2- 172.21.15.46:0/390113466 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6341593d0 0x7fa634151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:54.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa62be00640 1 --2- 172.21.15.46:0/390113466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63415cd90 0x7fa6341574a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00c480 tx=0x7fa61c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:54.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa62b400640 1 -- 172.21.15.46:0/390113466 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0074e0 con 0x7fa63415cd90 2024-03-29T08:19:54.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa62b400640 1 -- 172.21.15.46:0/390113466 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61c007660 con 0x7fa63415cd90 2024-03-29T08:19:54.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa62b400640 1 -- 172.21.15.46:0/390113466 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61c004a30 con 0x7fa63415cd90 2024-03-29T08:19:54.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa63a400640 1 -- 172.21.15.46:0/390113466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6341569c0 con 0x7fa63415cd90 2024-03-29T08:19:54.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa63a400640 1 -- 172.21.15.46:0/390113466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa634156f80 con 0x7fa63415cd90 2024-03-29T08:19:54.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa63a400640 1 -- 172.21.15.46:0/390113466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa634066c90 con 0x7fa63415cd90 2024-03-29T08:19:54.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.608+0000 7fa62b400640 1 -- 172.21.15.46:0/390113466 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa61c004b90 con 0x7fa63415cd90 2024-03-29T08:19:54.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.612+0000 7fa62b400640 1 --2- 172.21.15.46:0/390113466 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa610066690 0x7fa610068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.612+0000 7fa62b400640 1 -- 172.21.15.46:0/390113466 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa61c083270 con 0x7fa63415cd90 2024-03-29T08:19:54.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.612+0000 7fa633400640 1 --2- 172.21.15.46:0/390113466 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa610066690 0x7fa610068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.612+0000 7fa633400640 1 --2- 172.21.15.46:0/390113466 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa610066690 0x7fa610068b50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa6240ae000 tx=0x7fa6240a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:54.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.616+0000 7fa62b400640 1 -- 172.21.15.46:0/390113466 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa61c054100 con 0x7fa63415cd90 2024-03-29T08:19:54.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.812+0000 7fa63a400640 1 -- 172.21.15.46:0/390113466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6340add70 con 0x7fa63415cd90 2024-03-29T08:19:54.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.812+0000 7fa62b400640 1 -- 172.21.15.46:0/390113466 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4734 (secure 0 0 0) 0x7fa61c053c90 con 0x7fa63415cd90 2024-03-29T08:19:54.814 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:54.814 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/4274575126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4274575126},{"type":"v1","addr":"172.21.15.46:6834","nonce":4274575126}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3739899623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3739899623},{"type":"v1","addr":"172.21.15.73:6837","nonce":3739899623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1905427475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1905427475},{"type":"v1","addr":"172.21.15.46:6835","nonce":1905427475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:19:14.663142+0000","modified":"2024-03-29T08:19:17.680359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5152},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5152":{"gid":5152,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/1941408960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1941408960},{"type":"v1","addr":"172.21.15.73:6836","nonce":1941408960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-29T08:19:54.815 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2024-03-29T08:19:54.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.816+0000 7fa63a400640 1 -- 172.21.15.46:0/390113466 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa610066690 msgr2=0x7fa610068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:54.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.816+0000 7fa63a400640 1 --2- 172.21.15.46:0/390113466 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa610066690 0x7fa610068b50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa6240ae000 tx=0x7fa6240a7000 comp rx=0 tx=0).stop 2024-03-29T08:19:54.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.816+0000 7fa63a400640 1 -- 172.21.15.46:0/390113466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63415cd90 msgr2=0x7fa6341574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:54.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.816+0000 7fa63a400640 1 --2- 172.21.15.46:0/390113466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63415cd90 0x7fa6341574a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00c480 tx=0x7fa61c00c940 comp rx=0 tx=0).stop 2024-03-29T08:19:54.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.816+0000 7fa63a400640 1 -- 172.21.15.46:0/390113466 shutdown_connections 2024-03-29T08:19:54.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.816+0000 7fa63a400640 1 --2- 172.21.15.46:0/390113466 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa610066690 0x7fa610068b50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.816+0000 7fa63a400640 1 --2- 172.21.15.46:0/390113466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa63415d230 0x7fa63414c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.816+0000 7fa63a400640 1 --2- 172.21.15.46:0/390113466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63415cd90 0x7fa6341574a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.816+0000 7fa63a400640 1 --2- 172.21.15.46:0/390113466 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa6341593d0 0x7fa634151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.816+0000 7fa63a400640 1 -- 172.21.15.46:0/390113466 >> 172.21.15.46:0/390113466 conn(0x7fa634066110 msgr2=0x7fa634069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:54.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.816+0000 7fa63a400640 1 -- 172.21.15.46:0/390113466 shutdown_connections 2024-03-29T08:19:54.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.816+0000 7fa63a400640 1 -- 172.21.15.46:0/390113466 wait complete. 2024-03-29T08:19:54.835 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:19:54.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.976+0000 7f58c4e00640 1 Processor -- start 2024-03-29T08:19:54.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.976+0000 7f58c4e00640 1 -- start start 2024-03-29T08:19:54.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.976+0000 7f58c4e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f58c0069d90 0x7f58c006a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.976+0000 7f58c4e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f58c0161170 0x7f58c00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.976+0000 7f58c4e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c0160440 con 0x7f58c00cb550 2024-03-29T08:19:54.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.976+0000 7f58c4e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c0160ce0 con 0x7f58c0161170 2024-03-29T08:19:54.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.976+0000 7f58c4e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c006a6b0 con 0x7f58c0069d90 2024-03-29T08:19:54.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.976+0000 7f58b7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f58c0161170 0x7f58c00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.976+0000 7f58bfe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f58c00cb550 0x7f58c00cbb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44900/0 (socket says 172.21.15.46:44900) 2024-03-29T08:19:54.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.976+0000 7f58bfe00640 1 -- 172.21.15.46:0/1520673030 learned_addr learned my addr 172.21.15.46:0/1520673030 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:54.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.976+0000 7f58bf400640 1 --2- 172.21.15.46:0/1520673030 >> v2:172.21.15.73:3300/0 conn(0x7f58c0069d90 0x7f58c006a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.976+0000 7f58b7e00640 1 -- 172.21.15.46:0/1520673030 >> v2:172.21.15.73:3300/0 conn(0x7f58c0069d90 msgr2=0x7f58c006a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:54.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.976+0000 7f58b7e00640 1 --2- 172.21.15.46:0/1520673030 >> v2:172.21.15.73:3300/0 conn(0x7f58c0069d90 0x7f58c006a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.976+0000 7f58b7e00640 1 -- 172.21.15.46:0/1520673030 >> v1:172.21.15.46:6789/0 conn(0x7f58c00cb550 legacy=0x7f58c00cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:54.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.976+0000 7f58b7e00640 1 -- 172.21.15.46:0/1520673030 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c00cefa0 con 0x7f58c0161170 2024-03-29T08:19:54.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.976+0000 7f58bea00640 1 -- 172.21.15.46:0/1520673030 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622358453 0 0) 0x7f58c0160440 con 0x7f58c00cb550 2024-03-29T08:19:54.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.976+0000 7f58bf400640 1 --2- 172.21.15.46:0/1520673030 >> v2:172.21.15.73:3300/0 conn(0x7f58c0069d90 0x7f58c006a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:54.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.980+0000 7f58b7e00640 1 --2- 172.21.15.46:0/1520673030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c0161170 0x7f58c00ce670 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f58c00b1fd0 tx=0x7f58a801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae57b14145103fcf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:54.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.980+0000 7f58bea00640 1 -- 172.21.15.46:0/1520673030 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a80ac620 con 0x7f58c0161170 2024-03-29T08:19:54.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.980+0000 7f58bea00640 1 -- 172.21.15.46:0/1520673030 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a80ac780 con 0x7f58c0161170 2024-03-29T08:19:54.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.980+0000 7f58bea00640 1 -- 172.21.15.46:0/1520673030 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a80acb10 con 0x7f58c0161170 2024-03-29T08:19:54.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.980+0000 7f58c4e00640 1 -- 172.21.15.46:0/1520673030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c0161170 msgr2=0x7f58c00ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:54.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.980+0000 7f58c4e00640 1 --2- 172.21.15.46:0/1520673030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c0161170 0x7f58c00ce670 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f58c00b1fd0 tx=0x7f58a801c3c0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.980+0000 7f58c4e00640 1 -- 172.21.15.46:0/1520673030 shutdown_connections 2024-03-29T08:19:54.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.980+0000 7f58c4e00640 1 --2- 172.21.15.46:0/1520673030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c0161170 0x7f58c00ce670 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.980+0000 7f58c4e00640 1 --2- 172.21.15.46:0/1520673030 >> v2:172.21.15.73:3300/0 conn(0x7f58c0069d90 0x7f58c006a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.980+0000 7f58c4e00640 1 -- 172.21.15.46:0/1520673030 >> 172.21.15.46:0/1520673030 conn(0x7f58c0098a40 msgr2=0x7f58c00cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:54.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.980+0000 7f58c4e00640 1 -- 172.21.15.46:0/1520673030 shutdown_connections 2024-03-29T08:19:54.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.980+0000 7f58c4e00640 1 -- 172.21.15.46:0/1520673030 wait complete. 2024-03-29T08:19:54.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58c4e00640 1 Processor -- start 2024-03-29T08:19:54.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58c4e00640 1 -- start start 2024-03-29T08:19:54.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58c4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c0069d90 0x7f58c00b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58c4e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58c00cb550 0x7f58c00c3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58c4e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f58c0161170 0x7f58c0148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58c4e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c0153170 con 0x7f58c0069d90 2024-03-29T08:19:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58c4e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c00b5170 con 0x7f58c00cb550 2024-03-29T08:19:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58c4e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c00b48d0 con 0x7f58c0161170 2024-03-29T08:19:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c0069d90 0x7f58c00b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c0069d90 0x7f58c00b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45658/0 (socket says 172.21.15.46:45658) 2024-03-29T08:19:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58bf400640 1 -- 172.21.15.46:0/2690400801 learned_addr learned my addr 172.21.15.46:0/2690400801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58b7e00640 1 --2- 172.21.15.46:0/2690400801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58c00cb550 0x7f58c00c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58bfe00640 1 --2- 172.21.15.46:0/2690400801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f58c0161170 0x7f58c0148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58bf400640 1 -- 172.21.15.46:0/2690400801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f58c0161170 msgr2=0x7f58c0148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58bf400640 1 --2- 172.21.15.46:0/2690400801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f58c0161170 0x7f58c0148790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58bf400640 1 -- 172.21.15.46:0/2690400801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58c00cb550 msgr2=0x7f58c00c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:54.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58bf400640 1 --2- 172.21.15.46:0/2690400801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58c00cb550 0x7f58c00c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58bf400640 1 -- 172.21.15.46:0/2690400801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a8009420 con 0x7f58c0069d90 2024-03-29T08:19:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58bf400640 1 --2- 172.21.15.46:0/2690400801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c0069d90 0x7f58c00b24d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f58b000e810 tx=0x7f58b000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58b7400640 1 -- 172.21.15.46:0/2690400801 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b000c8d0 con 0x7f58c0069d90 2024-03-29T08:19:54.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58b7400640 1 -- 172.21.15.46:0/2690400801 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b000ca30 con 0x7f58c0069d90 2024-03-29T08:19:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58b7400640 1 -- 172.21.15.46:0/2690400801 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b000cdc0 con 0x7f58c0069d90 2024-03-29T08:19:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58c4e00640 1 -- 172.21.15.46:0/2690400801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c00b4bb0 con 0x7f58c0069d90 2024-03-29T08:19:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58c4e00640 1 -- 172.21.15.46:0/2690400801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c00c3570 con 0x7f58c0069d90 2024-03-29T08:19:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.984+0000 7f58c4e00640 1 -- 172.21.15.46:0/2690400801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c004e040 con 0x7f58c0069d90 2024-03-29T08:19:54.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.988+0000 7f58b7400640 1 -- 172.21.15.46:0/2690400801 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f58b0013410 con 0x7f58c0069d90 2024-03-29T08:19:54.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.988+0000 7f58b7400640 1 --2- 172.21.15.46:0/2690400801 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f589c0666c0 0x7f589c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:54.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.988+0000 7f58b7400640 1 -- 172.21.15.46:0/2690400801 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58b0083ed0 con 0x7f58c0069d90 2024-03-29T08:19:54.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.988+0000 7f58b7e00640 1 --2- 172.21.15.46:0/2690400801 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f589c0666c0 0x7f589c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:54.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.988+0000 7f58b7e00640 1 --2- 172.21.15.46:0/2690400801 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f589c0666c0 0x7f589c068b80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f58a80ae000 tx=0x7f58a8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:54.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:54.992+0000 7f58b7400640 1 -- 172.21.15.46:0/2690400801 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f58b00512e0 con 0x7f58c0069d90 2024-03-29T08:19:55.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.188+0000 7f58c4e00640 1 -- 172.21.15.46:0/2690400801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58c01685c0 con 0x7f58c0069d90 2024-03-29T08:19:55.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.188+0000 7f58b7400640 1 -- 172.21.15.46:0/2690400801 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4734 (secure 0 0 0) 0x7f58b0009480 con 0x7f58c0069d90 2024-03-29T08:19:55.190 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:55.191 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/4274575126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4274575126},{"type":"v1","addr":"172.21.15.46:6834","nonce":4274575126}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3739899623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3739899623},{"type":"v1","addr":"172.21.15.73:6837","nonce":3739899623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1905427475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1905427475},{"type":"v1","addr":"172.21.15.46:6835","nonce":1905427475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:19:14.663142+0000","modified":"2024-03-29T08:19:17.680359+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5152},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5152":{"gid":5152,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/1941408960","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1941408960},{"type":"v1","addr":"172.21.15.73:6836","nonce":1941408960}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-03-29T08:19:55.191 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 16 2024-03-29T08:19:55.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.192+0000 7f58c4e00640 1 -- 172.21.15.46:0/2690400801 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f589c0666c0 msgr2=0x7f589c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:55.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.192+0000 7f58c4e00640 1 --2- 172.21.15.46:0/2690400801 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f589c0666c0 0x7f589c068b80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f58a80ae000 tx=0x7f58a8008040 comp rx=0 tx=0).stop 2024-03-29T08:19:55.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.192+0000 7f58c4e00640 1 -- 172.21.15.46:0/2690400801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c0069d90 msgr2=0x7f58c00b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:55.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.192+0000 7f58c4e00640 1 --2- 172.21.15.46:0/2690400801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c0069d90 0x7f58c00b24d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f58b000e810 tx=0x7f58b000ecd0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.192+0000 7f58c4e00640 1 -- 172.21.15.46:0/2690400801 shutdown_connections 2024-03-29T08:19:55.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.192+0000 7f58c4e00640 1 --2- 172.21.15.46:0/2690400801 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f589c0666c0 0x7f589c068b80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.192+0000 7f58c4e00640 1 --2- 172.21.15.46:0/2690400801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58c00cb550 0x7f58c00c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.192+0000 7f58c4e00640 1 --2- 172.21.15.46:0/2690400801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f58c0161170 0x7f58c0148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.192+0000 7f58c4e00640 1 --2- 172.21.15.46:0/2690400801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58c0069d90 0x7f58c00b24d0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.192+0000 7f58c4e00640 1 -- 172.21.15.46:0/2690400801 >> 172.21.15.46:0/2690400801 conn(0x7f58c0098a40 msgr2=0x7f58c0155400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:55.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.192+0000 7f58c4e00640 1 -- 172.21.15.46:0/2690400801 shutdown_connections 2024-03-29T08:19:55.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.192+0000 7f58c4e00640 1 -- 172.21.15.46:0/2690400801 wait complete. 2024-03-29T08:19:55.211 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:19:55.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.348+0000 7f696e000640 1 Processor -- start 2024-03-29T08:19:55.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.352+0000 7f696e000640 1 -- start start 2024-03-29T08:19:55.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.352+0000 7f696e000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f69680b2b30 0x7f6968099150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:55.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.352+0000 7f696e000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69680cc2e0 con 0x7f696815a890 2024-03-29T08:19:55.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.352+0000 7f696e000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69680cc580 con 0x7f6968099690 2024-03-29T08:19:55.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.352+0000 7f696e000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69680cce20 con 0x7f69680b2b30 2024-03-29T08:19:55.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.352+0000 7f696cc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f696815a890 0x7f6968098a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44914/0 (socket says 172.21.15.46:44914) 2024-03-29T08:19:55.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.352+0000 7f696cc00640 1 -- 172.21.15.46:0/2740834511 learned_addr learned my addr 172.21.15.46:0/2740834511 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:55.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.352+0000 7f6967e00640 1 --2- 172.21.15.46:0/2740834511 >> v2:172.21.15.73:3300/0 conn(0x7f69680b2b30 0x7f6968099150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.352+0000 7f6967400640 1 -- 172.21.15.46:0/2740834511 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978512594 0 0) 0x7f69680cc2e0 con 0x7f696815a890 2024-03-29T08:19:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.352+0000 7f6967400640 1 -- 172.21.15.46:0/2740834511 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6950003530 con 0x7f696815a890 2024-03-29T08:19:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.352+0000 7f6967e00640 1 -- 172.21.15.46:0/2740834511 >> v1:172.21.15.73:6789/0 conn(0x7f6968099690 legacy=0x7f69680b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.352+0000 7f6967400640 1 -- 172.21.15.46:0/2740834511 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 148351306 0 0) 0x7f69680cc580 con 0x7f6968099690 2024-03-29T08:19:55.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.352+0000 7f6967e00640 1 -- 172.21.15.46:0/2740834511 >> v1:172.21.15.46:6789/0 conn(0x7f696815a890 legacy=0x7f6968098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:55.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.352+0000 7f6967e00640 1 -- 172.21.15.46:0/2740834511 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69680cc860 con 0x7f69680b2b30 2024-03-29T08:19:55.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.356+0000 7f6967e00640 1 --2- 172.21.15.46:0/2740834511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69680b2b30 0x7f6968099150 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f69540097c0 tx=0x7f695401c220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d2172a409567588 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:55.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.356+0000 7f6967400640 1 -- 172.21.15.46:0/2740834511 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695c001680 con 0x7f69680b2b30 2024-03-29T08:19:55.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.356+0000 7f6967400640 1 -- 172.21.15.46:0/2740834511 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69540a7050 con 0x7f69680b2b30 2024-03-29T08:19:55.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.356+0000 7f6967400640 1 -- 172.21.15.46:0/2740834511 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69540ab620 con 0x7f69680b2b30 2024-03-29T08:19:55.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.356+0000 7f696e000640 1 -- 172.21.15.46:0/2740834511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69680b2b30 msgr2=0x7f6968099150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:55.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.356+0000 7f696e000640 1 --2- 172.21.15.46:0/2740834511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69680b2b30 0x7f6968099150 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f69540097c0 tx=0x7f695401c220 comp rx=0 tx=0).stop 2024-03-29T08:19:55.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.356+0000 7f696e000640 1 -- 172.21.15.46:0/2740834511 shutdown_connections 2024-03-29T08:19:55.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.356+0000 7f696e000640 1 --2- 172.21.15.46:0/2740834511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69680b2b30 0x7f6968099150 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.356+0000 7f696e000640 1 -- 172.21.15.46:0/2740834511 >> 172.21.15.46:0/2740834511 conn(0x7f69680b2010 msgr2=0x7f69680b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:55.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.356+0000 7f696e000640 1 -- 172.21.15.46:0/2740834511 shutdown_connections 2024-03-29T08:19:55.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.356+0000 7f696e000640 1 -- 172.21.15.46:0/2740834511 wait complete. 2024-03-29T08:19:55.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.356+0000 7f696e000640 1 Processor -- start 2024-03-29T08:19:55.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f696e000640 1 -- start start 2024-03-29T08:19:55.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f696e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6968099690 0x7f6968156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:55.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f696e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69680b2b30 0x7f6968148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:55.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f696e000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f696815a890 0x7f6968150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:55.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f696e000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6968151d50 con 0x7f69680b2b30 2024-03-29T08:19:55.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f696e000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6968154690 con 0x7f6968099690 2024-03-29T08:19:55.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f696e000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69681517d0 con 0x7f696815a890 2024-03-29T08:19:55.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f6967e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69680b2b30 0x7f6968148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:55.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f6967e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69680b2b30 0x7f6968148b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45662/0 (socket says 172.21.15.46:45662) 2024-03-29T08:19:55.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f6967e00640 1 -- 172.21.15.46:0/1026892508 learned_addr learned my addr 172.21.15.46:0/1026892508 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:55.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f696cc00640 1 --2- 172.21.15.46:0/1026892508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6968099690 0x7f6968156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:55.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f696d600640 1 --2- 172.21.15.46:0/1026892508 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f696815a890 0x7f6968150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:55.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f6967e00640 1 -- 172.21.15.46:0/1026892508 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f696815a890 msgr2=0x7f6968150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:55.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f6967e00640 1 --2- 172.21.15.46:0/1026892508 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f696815a890 0x7f6968150c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f6967e00640 1 -- 172.21.15.46:0/1026892508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6968099690 msgr2=0x7f6968156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:55.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f6967e00640 1 --2- 172.21.15.46:0/1026892508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6968099690 0x7f6968156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f6967e00640 1 -- 172.21.15.46:0/1026892508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6954009420 con 0x7f69680b2b30 2024-03-29T08:19:55.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f696cc00640 1 --2- 172.21.15.46:0/1026892508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6968099690 0x7f6968156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:55.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f6967e00640 1 --2- 172.21.15.46:0/1026892508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69680b2b30 0x7f6968148b30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6954018e30 tx=0x7f6954018fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:55.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f6964c00640 1 -- 172.21.15.46:0/1026892508 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69540b4400 con 0x7f69680b2b30 2024-03-29T08:19:55.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f6964c00640 1 -- 172.21.15.46:0/1026892508 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69540b4560 con 0x7f69680b2b30 2024-03-29T08:19:55.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f6964c00640 1 -- 172.21.15.46:0/1026892508 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69540b4910 con 0x7f69680b2b30 2024-03-29T08:19:55.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f696e000640 1 -- 172.21.15.46:0/1026892508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6968151a70 con 0x7f69680b2b30 2024-03-29T08:19:55.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f696e000640 1 -- 172.21.15.46:0/1026892508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6968149af0 con 0x7f69680b2b30 2024-03-29T08:19:55.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.360+0000 7f696e000640 1 -- 172.21.15.46:0/1026892508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6968154180 con 0x7f69680b2b30 2024-03-29T08:19:55.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.364+0000 7f6964c00640 1 -- 172.21.15.46:0/1026892508 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f69540b2040 con 0x7f69680b2b30 2024-03-29T08:19:55.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.364+0000 7f6964c00640 1 --2- 172.21.15.46:0/1026892508 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6948066690 0x7f6948068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:55.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.364+0000 7f6964c00640 1 -- 172.21.15.46:0/1026892508 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f695411e370 con 0x7f69680b2b30 2024-03-29T08:19:55.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.364+0000 7f696cc00640 1 --2- 172.21.15.46:0/1026892508 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6948066690 0x7f6948068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:55.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.364+0000 7f696cc00640 1 --2- 172.21.15.46:0/1026892508 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6948066690 0x7f6948068b50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f695c000fc0 tx=0x7f695c001920 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:55.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.368+0000 7f6964c00640 1 -- 172.21.15.46:0/1026892508 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6968154180 con 0x7f69680b2b30 2024-03-29T08:19:55.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.528+0000 7f696e000640 1 -- 172.21.15.46:0/1026892508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6968154180 con 0x7f69680b2b30 2024-03-29T08:19:55.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.528+0000 7f6964c00640 1 -- 172.21.15.46:0/1026892508 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f6968154180 con 0x7f69680b2b30 2024-03-29T08:19:55.530 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:19:55.530 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:19:15.665674+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:19:12.674055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-29T08:19:13.661287+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:19:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.528+0000 7f696e000640 1 -- 172.21.15.46:0/1026892508 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6948066690 msgr2=0x7f6948068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.528+0000 7f696e000640 1 --2- 172.21.15.46:0/1026892508 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6948066690 0x7f6948068b50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f695c000fc0 tx=0x7f695c001920 comp rx=0 tx=0).stop 2024-03-29T08:19:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.528+0000 7f696e000640 1 -- 172.21.15.46:0/1026892508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69680b2b30 msgr2=0x7f6968148b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.528+0000 7f696e000640 1 --2- 172.21.15.46:0/1026892508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69680b2b30 0x7f6968148b30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6954018e30 tx=0x7f6954018fc0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.528+0000 7f696e000640 1 -- 172.21.15.46:0/1026892508 shutdown_connections 2024-03-29T08:19:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.528+0000 7f696e000640 1 --2- 172.21.15.46:0/1026892508 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6948066690 0x7f6948068b50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.528+0000 7f696e000640 1 --2- 172.21.15.46:0/1026892508 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f696815a890 0x7f6968150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.528+0000 7f696e000640 1 --2- 172.21.15.46:0/1026892508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69680b2b30 0x7f6968148b30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.528+0000 7f696e000640 1 --2- 172.21.15.46:0/1026892508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6968099690 0x7f6968156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.528+0000 7f696e000640 1 -- 172.21.15.46:0/1026892508 >> 172.21.15.46:0/1026892508 conn(0x7f69680b2010 msgr2=0x7f69680cfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.532+0000 7f696e000640 1 -- 172.21.15.46:0/1026892508 shutdown_connections 2024-03-29T08:19:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.532+0000 7f696e000640 1 -- 172.21.15.46:0/1026892508 wait complete. 2024-03-29T08:19:55.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T08:19:55.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.692+0000 7fc1cf400640 1 Processor -- start 2024-03-29T08:19:55.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.692+0000 7fc1cf400640 1 -- start start 2024-03-29T08:19:55.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.692+0000 7fc1cf400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc1c806dbb0 0x7fc1c815d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:55.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.692+0000 7fc1cf400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c8070c70 con 0x7fc1c8142ab0 2024-03-29T08:19:55.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.692+0000 7fc1cf400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c8070340 con 0x7fc1c815d940 2024-03-29T08:19:55.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.692+0000 7fc1cf400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c8070620 con 0x7fc1c806dbb0 2024-03-29T08:19:55.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.692+0000 7fc1cca00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc1c8142ab0 0x7fc1c806d4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44920/0 (socket says 172.21.15.46:44920) 2024-03-29T08:19:55.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.692+0000 7fc1cca00640 1 -- 172.21.15.46:0/1887388955 learned_addr learned my addr 172.21.15.46:0/1887388955 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:55.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.692+0000 7fc1c7e00640 1 --2- 172.21.15.46:0/1887388955 >> v2:172.21.15.73:3300/0 conn(0x7fc1c806dbb0 0x7fc1c815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:55.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.692+0000 7fc1c7400640 1 -- 172.21.15.46:0/1887388955 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954086462 0 0) 0x7fc1c8070c70 con 0x7fc1c8142ab0 2024-03-29T08:19:55.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.692+0000 7fc1c7400640 1 -- 172.21.15.46:0/1887388955 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1b0003530 con 0x7fc1c8142ab0 2024-03-29T08:19:55.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.692+0000 7fc1c7400640 1 -- 172.21.15.46:0/1887388955 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189148787 0 0) 0x7fc1c8070340 con 0x7fc1c815d940 2024-03-29T08:19:55.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.692+0000 7fc1c7400640 1 -- 172.21.15.46:0/1887388955 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1c8070c70 con 0x7fc1c815d940 2024-03-29T08:19:55.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.692+0000 7fc1c7e00640 1 -- 172.21.15.46:0/1887388955 >> v1:172.21.15.73:6789/0 conn(0x7fc1c815d940 legacy=0x7fc1c80b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:55.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.692+0000 7fc1c7e00640 1 -- 172.21.15.46:0/1887388955 >> v1:172.21.15.46:6789/0 conn(0x7fc1c8142ab0 legacy=0x7fc1c806d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:55.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.692+0000 7fc1c7400640 1 -- 172.21.15.46:0/1887388955 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 889492019 0 0) 0x7fc1b0003530 con 0x7fc1c8142ab0 2024-03-29T08:19:55.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.692+0000 7fc1c7e00640 1 -- 172.21.15.46:0/1887388955 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c8070900 con 0x7fc1c806dbb0 2024-03-29T08:19:55.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.696+0000 7fc1c7e00640 1 --2- 172.21.15.46:0/1887388955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c806dbb0 0x7fc1c815d400 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40097c0 tx=0x7fc1b401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a4d6687b5dbb26 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:55.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.696+0000 7fc1c7400640 1 -- 172.21.15.46:0/1887388955 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc001680 con 0x7fc1c806dbb0 2024-03-29T08:19:55.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.696+0000 7fc1c7400640 1 -- 172.21.15.46:0/1887388955 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b40a7050 con 0x7fc1c806dbb0 2024-03-29T08:19:55.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.696+0000 7fc1c7400640 1 -- 172.21.15.46:0/1887388955 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40ab400 con 0x7fc1c806dbb0 2024-03-29T08:19:55.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.696+0000 7fc1cf400640 1 -- 172.21.15.46:0/1887388955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c806dbb0 msgr2=0x7fc1c815d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:55.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.696+0000 7fc1cf400640 1 --2- 172.21.15.46:0/1887388955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c806dbb0 0x7fc1c815d400 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc1b40097c0 tx=0x7fc1b401c3c0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.696+0000 7fc1cf400640 1 -- 172.21.15.46:0/1887388955 shutdown_connections 2024-03-29T08:19:55.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.696+0000 7fc1cf400640 1 --2- 172.21.15.46:0/1887388955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c806dbb0 0x7fc1c815d400 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.696+0000 7fc1cf400640 1 -- 172.21.15.46:0/1887388955 >> 172.21.15.46:0/1887388955 conn(0x7fc1c8066080 msgr2=0x7fc1c8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:55.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.696+0000 7fc1cf400640 1 -- 172.21.15.46:0/1887388955 shutdown_connections 2024-03-29T08:19:55.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.696+0000 7fc1cf400640 1 -- 172.21.15.46:0/1887388955 wait complete. 2024-03-29T08:19:55.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1cf400640 1 Processor -- start 2024-03-29T08:19:55.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1cf400640 1 -- start start 2024-03-29T08:19:55.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1cf400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1c806dbb0 0x7fc1c81396c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1cf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c8142ab0 0x7fc1c813b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1cf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c815d940 0x7fc1c813bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1cf400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c813ca70 con 0x7fc1c8142ab0 2024-03-29T08:19:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1cf400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c813c140 con 0x7fc1c815d940 2024-03-29T08:19:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1cf400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c813c420 con 0x7fc1c806dbb0 2024-03-29T08:19:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c8142ab0 0x7fc1c813b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c8142ab0 0x7fc1c813b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45678/0 (socket says 172.21.15.46:45678) 2024-03-29T08:19:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1c7e00640 1 -- 172.21.15.46:0/1968147593 learned_addr learned my addr 172.21.15.46:0/1968147593 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1cca00640 1 --2- 172.21.15.46:0/1968147593 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1c806dbb0 0x7fc1c81396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1cd400640 1 --2- 172.21.15.46:0/1968147593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c815d940 0x7fc1c813bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1c7e00640 1 -- 172.21.15.46:0/1968147593 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1c806dbb0 msgr2=0x7fc1c81396c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1c7e00640 1 --2- 172.21.15.46:0/1968147593 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1c806dbb0 0x7fc1c81396c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1c7e00640 1 -- 172.21.15.46:0/1968147593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c815d940 msgr2=0x7fc1c813bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1c7e00640 1 --2- 172.21.15.46:0/1968147593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c815d940 0x7fc1c813bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1c7e00640 1 -- 172.21.15.46:0/1968147593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b4009420 con 0x7fc1c8142ab0 2024-03-29T08:19:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1cd400640 1 --2- 172.21.15.46:0/1968147593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c815d940 0x7fc1c813bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:19:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1c7e00640 1 --2- 172.21.15.46:0/1968147593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c8142ab0 0x7fc1c813b620 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4018fd0 tx=0x7fc1b4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1c4c00640 1 -- 172.21.15.46:0/1968147593 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40b4400 con 0x7fc1c8142ab0 2024-03-29T08:19:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1c4c00640 1 -- 172.21.15.46:0/1968147593 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b40b45a0 con 0x7fc1c8142ab0 2024-03-29T08:19:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1c4c00640 1 -- 172.21.15.46:0/1968147593 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40b4950 con 0x7fc1c8142ab0 2024-03-29T08:19:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1cf400640 1 -- 172.21.15.46:0/1968147593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c813c700 con 0x7fc1c8142ab0 2024-03-29T08:19:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.700+0000 7fc1cf400640 1 -- 172.21.15.46:0/1968147593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c8158d10 con 0x7fc1c8142ab0 2024-03-29T08:19:55.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.704+0000 7fc1cf400640 1 -- 172.21.15.46:0/1968147593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c8159ec0 con 0x7fc1c8142ab0 2024-03-29T08:19:55.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.704+0000 7fc1c4c00640 1 -- 172.21.15.46:0/1968147593 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc1b40b2040 con 0x7fc1c8142ab0 2024-03-29T08:19:55.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.704+0000 7fc1c4c00640 1 --2- 172.21.15.46:0/1968147593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc1a8066690 0x7fc1a8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.704+0000 7fc1c4c00640 1 -- 172.21.15.46:0/1968147593 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1b411dc60 con 0x7fc1c8142ab0 2024-03-29T08:19:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.704+0000 7fc1cca00640 1 --2- 172.21.15.46:0/1968147593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc1a8066690 0x7fc1a8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.704+0000 7fc1cca00640 1 --2- 172.21.15.46:0/1968147593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc1a8066690 0x7fc1a8068b50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc005890 tx=0x7fc1bc0016c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:55.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.708+0000 7fc1c4c00640 1 -- 172.21.15.46:0/1968147593 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc1c8159ec0 con 0x7fc1c8142ab0 2024-03-29T08:19:55.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.904+0000 7fc1cf400640 1 -- 172.21.15.46:0/1968147593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc1c8159ec0 con 0x7fc1c8142ab0 2024-03-29T08:19:55.906 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-03-29T08:19:55.904+0000 7fc16ae00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:19:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.916+0000 7fc1c4c00640 1 -- 172.21.15.46:0/1968147593 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc1c8159ec0 con 0x7fc1c8142ab0 2024-03-29T08:19:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T08:19:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.920+0000 7fc1cf400640 1 -- 172.21.15.46:0/1968147593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc1a8066690 msgr2=0x7fc1a8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.920+0000 7fc1cf400640 1 --2- 172.21.15.46:0/1968147593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc1a8066690 0x7fc1a8068b50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc005890 tx=0x7fc1bc0016c0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.920+0000 7fc1cf400640 1 -- 172.21.15.46:0/1968147593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c8142ab0 msgr2=0x7fc1c813b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.920+0000 7fc1cf400640 1 --2- 172.21.15.46:0/1968147593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c8142ab0 0x7fc1c813b620 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4018fd0 tx=0x7fc1b4019160 comp rx=0 tx=0).stop 2024-03-29T08:19:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.920+0000 7fc1cf400640 1 -- 172.21.15.46:0/1968147593 shutdown_connections 2024-03-29T08:19:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.920+0000 7fc1cf400640 1 --2- 172.21.15.46:0/1968147593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc1a8066690 0x7fc1a8068b50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.920+0000 7fc1cf400640 1 --2- 172.21.15.46:0/1968147593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c815d940 0x7fc1c813bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.920+0000 7fc1cf400640 1 --2- 172.21.15.46:0/1968147593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1c8142ab0 0x7fc1c813b620 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.920+0000 7fc1cf400640 1 --2- 172.21.15.46:0/1968147593 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1c806dbb0 0x7fc1c81396c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.920+0000 7fc1cf400640 1 -- 172.21.15.46:0/1968147593 >> 172.21.15.46:0/1968147593 conn(0x7fc1c8066080 msgr2=0x7fc1c8164c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.920+0000 7fc1cf400640 1 -- 172.21.15.46:0/1968147593 shutdown_connections 2024-03-29T08:19:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:55.920+0000 7fc1cf400640 1 -- 172.21.15.46:0/1968147593 wait complete. 2024-03-29T08:19:55.937 INFO:tasks.ceph.mds.b.smithi073.stderr: -5940> 2024-03-29T08:19:18.164+0000 7fcff9000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:19:55.943 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T08:19:55.988 INFO:tasks.ceph.mds.b.smithi073.stdout:starting mds.b at 2024-03-29T08:19:56.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.084+0000 7f4080c00640 1 Processor -- start 2024-03-29T08:19:56.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.084+0000 7f4080c00640 1 -- start start 2024-03-29T08:19:56.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.084+0000 7f4080c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f407c142a50 0x7f407c0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:56.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.084+0000 7f4080c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f407c15df90 0x7f407c0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:56.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.084+0000 7f4080c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f407c15d260 con 0x7f407c0ae280 2024-03-29T08:19:56.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.084+0000 7f4080c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f407c0aefd0 con 0x7f407c15df90 2024-03-29T08:19:56.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.084+0000 7f4080c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407c15d4b0 con 0x7f407c142a50 2024-03-29T08:19:56.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.084+0000 7f4073e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f407c15df90 0x7f407c0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:56.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.084+0000 7f407a800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f407c0ae280 0x7f407c0ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44932/0 (socket says 172.21.15.46:44932) 2024-03-29T08:19:56.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.084+0000 7f407a800640 1 -- 172.21.15.46:0/1740936773 learned_addr learned my addr 172.21.15.46:0/1740936773 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.084+0000 7f4079e00640 1 --2- 172.21.15.46:0/1740936773 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f407c142a50 0x7f407c0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.084+0000 7f4073e00640 1 -- 172.21.15.46:0/1740936773 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f407c142a50 msgr2=0x7f407c0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.084+0000 7f4073e00640 1 --2- 172.21.15.46:0/1740936773 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f407c142a50 0x7f407c0cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.084+0000 7f4073e00640 1 -- 172.21.15.46:0/1740936773 >> v1:172.21.15.46:6789/0 conn(0x7f407c0ae280 legacy=0x7f407c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.084+0000 7f4073e00640 1 -- 172.21.15.46:0/1740936773 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c15d790 con 0x7f407c15df90 2024-03-29T08:19:56.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.084+0000 7f4079400640 1 -- 172.21.15.46:0/1740936773 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404851160 0 0) 0x7f407c15d260 con 0x7f407c0ae280 2024-03-29T08:19:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.088+0000 7f4073e00640 1 --2- 172.21.15.46:0/1740936773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f407c15df90 0x7f407c0cbcd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f406400ba60 tx=0x7f406401eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637568f1d4a7a5d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.088+0000 7f4079400640 1 -- 172.21.15.46:0/1740936773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40640a79c0 con 0x7f407c15df90 2024-03-29T08:19:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.088+0000 7f4079400640 1 -- 172.21.15.46:0/1740936773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40640a7b20 con 0x7f407c15df90 2024-03-29T08:19:56.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.088+0000 7f4079400640 1 -- 172.21.15.46:0/1740936773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40640af440 con 0x7f407c15df90 2024-03-29T08:19:56.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.088+0000 7f4080c00640 1 -- 172.21.15.46:0/1740936773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f407c15df90 msgr2=0x7f407c0cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:56.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.088+0000 7f4080c00640 1 --2- 172.21.15.46:0/1740936773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f407c15df90 0x7f407c0cbcd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f406400ba60 tx=0x7f406401eae0 comp rx=0 tx=0).stop 2024-03-29T08:19:56.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.088+0000 7f4080c00640 1 -- 172.21.15.46:0/1740936773 shutdown_connections 2024-03-29T08:19:56.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.088+0000 7f4080c00640 1 --2- 172.21.15.46:0/1740936773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f407c15df90 0x7f407c0cbcd0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:56.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.088+0000 7f4080c00640 1 --2- 172.21.15.46:0/1740936773 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f407c142a50 0x7f407c0cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:56.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.088+0000 7f4080c00640 1 -- 172.21.15.46:0/1740936773 >> 172.21.15.46:0/1740936773 conn(0x7f407c06a390 msgr2=0x7f407c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:56.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.088+0000 7f4080c00640 1 -- 172.21.15.46:0/1740936773 shutdown_connections 2024-03-29T08:19:56.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.088+0000 7f4080c00640 1 -- 172.21.15.46:0/1740936773 wait complete. 2024-03-29T08:19:56.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4080c00640 1 Processor -- start 2024-03-29T08:19:56.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4080c00640 1 -- start start 2024-03-29T08:19:56.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4080c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f407c0ae280 0x7f407c149410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:56.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4080c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f407c142a50 0x7f407c159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:56.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4080c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f407c15df90 0x7f407c163a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:56.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4080c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c148ff0 con 0x7f407c0ae280 2024-03-29T08:19:56.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4080c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f407c13a660 con 0x7f407c142a50 2024-03-29T08:19:56.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4080c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f407c15ab60 con 0x7f407c15df90 2024-03-29T08:19:56.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4079e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f407c0ae280 0x7f407c149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:56.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4079e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f407c0ae280 0x7f407c149410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45696/0 (socket says 172.21.15.46:45696) 2024-03-29T08:19:56.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4079e00640 1 -- 172.21.15.46:0/4283390582 learned_addr learned my addr 172.21.15.46:0/4283390582 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:56.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4073e00640 1 --2- 172.21.15.46:0/4283390582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f407c142a50 0x7f407c159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:56.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f407a800640 1 --2- 172.21.15.46:0/4283390582 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f407c15df90 0x7f407c163a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:56.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4079e00640 1 -- 172.21.15.46:0/4283390582 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f407c15df90 msgr2=0x7f407c163a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:56.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4079e00640 1 --2- 172.21.15.46:0/4283390582 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f407c15df90 0x7f407c163a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:56.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4079e00640 1 -- 172.21.15.46:0/4283390582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f407c142a50 msgr2=0x7f407c159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:56.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4079e00640 1 --2- 172.21.15.46:0/4283390582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f407c142a50 0x7f407c159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:56.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4079e00640 1 -- 172.21.15.46:0/4283390582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406400b6c0 con 0x7f407c0ae280 2024-03-29T08:19:56.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f407a800640 1 --2- 172.21.15.46:0/4283390582 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f407c15df90 0x7f407c163a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:56.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4073e00640 1 --2- 172.21.15.46:0/4283390582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f407c142a50 0x7f407c159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:56.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4079e00640 1 --2- 172.21.15.46:0/4283390582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f407c0ae280 0x7f407c149410 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f406c00d800 tx=0x7f406c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:56.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4072000640 1 -- 172.21.15.46:0/4283390582 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c00b6c0 con 0x7f407c0ae280 2024-03-29T08:19:56.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4072000640 1 -- 172.21.15.46:0/4283390582 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f406c00b820 con 0x7f407c0ae280 2024-03-29T08:19:56.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4072000640 1 -- 172.21.15.46:0/4283390582 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406c00bb90 con 0x7f407c0ae280 2024-03-29T08:19:56.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4080c00640 1 -- 172.21.15.46:0/4283390582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407c15a5a0 con 0x7f407c0ae280 2024-03-29T08:19:56.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.092+0000 7f4080c00640 1 -- 172.21.15.46:0/4283390582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407c148c10 con 0x7f407c0ae280 2024-03-29T08:19:56.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.096+0000 7f4080c00640 1 -- 172.21.15.46:0/4283390582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f407c071a70 con 0x7f407c0ae280 2024-03-29T08:19:56.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.096+0000 7f4072000640 1 -- 172.21.15.46:0/4283390582 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f406c00bcf0 con 0x7f407c0ae280 2024-03-29T08:19:56.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.096+0000 7f4072000640 1 --2- 172.21.15.46:0/4283390582 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4058066690 0x7f4058068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:56.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.096+0000 7f4072000640 1 -- 172.21.15.46:0/4283390582 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f406c084800 con 0x7f407c0ae280 2024-03-29T08:19:56.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.096+0000 7f4073e00640 1 --2- 172.21.15.46:0/4283390582 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4058066690 0x7f4058068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:56.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.096+0000 7f4073e00640 1 --2- 172.21.15.46:0/4283390582 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4058066690 0x7f4058068b50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f40640b1000 tx=0x7f4064008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:56.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.100+0000 7f4072000640 1 -- 172.21.15.46:0/4283390582 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f406c051a90 con 0x7f407c0ae280 2024-03-29T08:19:56.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.296+0000 7f4080c00640 1 -- 172.21.15.46:0/4283390582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f407c04e120 con 0x7f407c0ae280 2024-03-29T08:19:56.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.936+0000 7f4072000640 1 -- 172.21.15.46:0/4283390582 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f406c011340 con 0x7f407c0ae280 2024-03-29T08:19:56.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.940+0000 7f4080c00640 1 -- 172.21.15.46:0/4283390582 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4058066690 msgr2=0x7f4058068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:56.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.940+0000 7f4080c00640 1 --2- 172.21.15.46:0/4283390582 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4058066690 0x7f4058068b50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f40640b1000 tx=0x7f4064008040 comp rx=0 tx=0).stop 2024-03-29T08:19:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.940+0000 7f4080c00640 1 -- 172.21.15.46:0/4283390582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f407c0ae280 msgr2=0x7f407c149410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.940+0000 7f4080c00640 1 --2- 172.21.15.46:0/4283390582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f407c0ae280 0x7f407c149410 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f406c00d800 tx=0x7f406c00dcc0 comp rx=0 tx=0).stop 2024-03-29T08:19:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.940+0000 7f4080c00640 1 -- 172.21.15.46:0/4283390582 shutdown_connections 2024-03-29T08:19:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.940+0000 7f4080c00640 1 --2- 172.21.15.46:0/4283390582 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4058066690 0x7f4058068b50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.940+0000 7f4080c00640 1 --2- 172.21.15.46:0/4283390582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f407c142a50 0x7f407c159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.940+0000 7f4080c00640 1 --2- 172.21.15.46:0/4283390582 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f407c15df90 0x7f407c163a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.940+0000 7f4080c00640 1 --2- 172.21.15.46:0/4283390582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f407c0ae280 0x7f407c149410 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.940+0000 7f4080c00640 1 -- 172.21.15.46:0/4283390582 >> 172.21.15.46:0/4283390582 conn(0x7f407c06a390 msgr2=0x7f407c0b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.940+0000 7f4080c00640 1 -- 172.21.15.46:0/4283390582 shutdown_connections 2024-03-29T08:19:56.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:56.940+0000 7f4080c00640 1 -- 172.21.15.46:0/4283390582 wait complete. 2024-03-29T08:19:56.962 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T08:19:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.108+0000 7f3f23400640 1 Processor -- start 2024-03-29T08:19:57.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.108+0000 7f3f23400640 1 -- start start 2024-03-29T08:19:57.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.108+0000 7f3f23400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f1c15a970 0x7f3f1c0d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.108+0000 7f3f23400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3f1c0d0b00 0x7f3f1c0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.108+0000 7f3f23400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1c0cba70 con 0x7f3f1c0b0b30 2024-03-29T08:19:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.108+0000 7f3f23400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1c0d0f60 con 0x7f3f1c0d0b00 2024-03-29T08:19:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.108+0000 7f3f23400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0b1320 con 0x7f3f1c15a970 2024-03-29T08:19:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.108+0000 7f3f21400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3f1c0b0b30 0x7f3f1c0cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53534/0 (socket says 172.21.15.46:53534) 2024-03-29T08:19:57.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.108+0000 7f3f21400640 1 -- 172.21.15.46:0/245815558 learned_addr learned my addr 172.21.15.46:0/245815558 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:19:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.108+0000 7f3f20a00640 1 --2- 172.21.15.46:0/245815558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f1c15a970 0x7f3f1c0d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.108+0000 7f3f13e00640 1 --2- 172.21.15.46:0/245815558 >> v2:172.21.15.73:3300/0 conn(0x7f3f1c0d0b00 0x7f3f1c0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.108+0000 7f3f1be00640 1 -- 172.21.15.46:0/245815558 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930225727 0 0) 0x7f3f1c0cba70 con 0x7f3f1c0b0b30 2024-03-29T08:19:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.108+0000 7f3f1be00640 1 -- 172.21.15.46:0/245815558 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f00003530 con 0x7f3f1c0b0b30 2024-03-29T08:19:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f1be00640 1 -- 172.21.15.46:0/245815558 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2852139736 0 0) 0x7f3f00003530 con 0x7f3f1c0b0b30 2024-03-29T08:19:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f1be00640 1 -- 172.21.15.46:0/245815558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f1c15a970 msgr2=0x7f3f1c0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f1be00640 1 --2- 172.21.15.46:0/245815558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f1c15a970 0x7f3f1c0d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f1be00640 1 -- 172.21.15.46:0/245815558 >> v2:172.21.15.73:3300/0 conn(0x7f3f1c0d0b00 msgr2=0x7f3f1c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f1be00640 1 --2- 172.21.15.46:0/245815558 >> v2:172.21.15.73:3300/0 conn(0x7f3f1c0d0b00 0x7f3f1c0b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f1be00640 1 -- 172.21.15.46:0/245815558 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c0cb420 con 0x7f3f1c0b0b30 2024-03-29T08:19:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f1be00640 1 -- 172.21.15.46:0/245815558 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f3f14001a00 con 0x7f3f1c0b0b30 2024-03-29T08:19:57.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f1be00640 1 -- 172.21.15.46:0/245815558 >> v1:172.21.15.46:6789/0 conn(0x7f3f1c0b0b30 legacy=0x7f3f1c0cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f1be00640 1 --2- 172.21.15.46:0/245815558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f1c15a970 0x7f3f00003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f1be00640 1 --2- 172.21.15.46:0/245815558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f00003f50 0x7f3f000047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f1be00640 1 --2- 172.21.15.46:0/245815558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f00004d10 0x7f3f00005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f1be00640 1 -- 172.21.15.46:0/245815558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0d0f60 con 0x7f3f00004d10 2024-03-29T08:19:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f1be00640 1 -- 172.21.15.46:0/245815558 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0b1320 con 0x7f3f00003f50 2024-03-29T08:19:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f1be00640 1 -- 172.21.15.46:0/245815558 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0000b990 con 0x7f3f1c15a970 2024-03-29T08:19:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f21400640 1 --2- 172.21.15.46:0/245815558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f00004d10 0x7f3f00005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f20a00640 1 --2- 172.21.15.46:0/245815558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f1c15a970 0x7f3f00003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f13e00640 1 --2- 172.21.15.46:0/245815558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f00003f50 0x7f3f000047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f21400640 1 -- 172.21.15.46:0/245815558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f1c15a970 msgr2=0x7f3f00003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f21400640 1 --2- 172.21.15.46:0/245815558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f1c15a970 0x7f3f00003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f21400640 1 -- 172.21.15.46:0/245815558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f00003f50 msgr2=0x7f3f000047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:57.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f21400640 1 --2- 172.21.15.46:0/245815558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f00003f50 0x7f3f000047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:57.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f21400640 1 -- 172.21.15.46:0/245815558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f00003530 con 0x7f3f00004d10 2024-03-29T08:19:57.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.112+0000 7f3f21400640 1 --2- 172.21.15.46:0/245815558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f00004d10 0x7f3f00005190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400a7c0 tx=0x7f3f1401d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00bdb16d48d8ff4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f1be00640 1 -- 172.21.15.46:0/245815558 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f140ad630 con 0x7f3f00004d10 2024-03-29T08:19:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f1be00640 1 -- 172.21.15.46:0/245815558 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f140a8070 con 0x7f3f00004d10 2024-03-29T08:19:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f1be00640 1 -- 172.21.15.46:0/245815558 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f140ada30 con 0x7f3f00004d10 2024-03-29T08:19:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f23400640 1 -- 172.21.15.46:0/245815558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f00004d10 msgr2=0x7f3f00005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f23400640 1 --2- 172.21.15.46:0/245815558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f00004d10 0x7f3f00005190 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400a7c0 tx=0x7f3f1401d2a0 comp rx=0 tx=0).stop 2024-03-29T08:19:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f21400640 1 -- 172.21.15.46:0/245815558 reap_dead start 2024-03-29T08:19:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f23400640 1 -- 172.21.15.46:0/245815558 shutdown_connections 2024-03-29T08:19:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f23400640 1 -- 172.21.15.46:0/245815558 >> 172.21.15.46:0/245815558 conn(0x7f3f1c089fd0 msgr2=0x7f3f1c08a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:57.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f23400640 1 -- 172.21.15.46:0/245815558 shutdown_connections 2024-03-29T08:19:57.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f23400640 1 -- 172.21.15.46:0/245815558 wait complete. 2024-03-29T08:19:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f23400640 1 Processor -- start 2024-03-29T08:19:57.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f23400640 1 -- start start 2024-03-29T08:19:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f23400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f00004d10 0x7f3f1c1f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:57.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f23400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f1c0b23c0 0x7f3f1c151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f23400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f1c0b2840 0x7f3f1c151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f23400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c1f6a10 con 0x7f3f00004d10 2024-03-29T08:19:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f23400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0c9ec0 con 0x7f3f1c0b2840 2024-03-29T08:19:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f23400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0b36c0 con 0x7f3f1c0b23c0 2024-03-29T08:19:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f20a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f00004d10 0x7f3f1c1f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f20a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f00004d10 0x7f3f1c1f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54720/0 (socket says 172.21.15.46:54720) 2024-03-29T08:19:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f20a00640 1 -- 172.21.15.46:0/2892850862 learned_addr learned my addr 172.21.15.46:0/2892850862 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f13e00640 1 --2- 172.21.15.46:0/2892850862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f1c0b23c0 0x7f3f1c151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.116+0000 7f3f21400640 1 --2- 172.21.15.46:0/2892850862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f1c0b2840 0x7f3f1c151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.120+0000 7f3f20a00640 1 -- 172.21.15.46:0/2892850862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f1c0b23c0 msgr2=0x7f3f1c151760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.120+0000 7f3f20a00640 1 --2- 172.21.15.46:0/2892850862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f1c0b23c0 0x7f3f1c151760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.120+0000 7f3f20a00640 1 -- 172.21.15.46:0/2892850862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f1c0b2840 msgr2=0x7f3f1c151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.120+0000 7f3f20a00640 1 --2- 172.21.15.46:0/2892850862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f1c0b2840 0x7f3f1c151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:57.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.120+0000 7f3f20a00640 1 -- 172.21.15.46:0/2892850862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c0b39a0 con 0x7f3f00004d10 2024-03-29T08:19:57.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.120+0000 7f3f20a00640 1 --2- 172.21.15.46:0/2892850862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f00004d10 0x7f3f1c1f64d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0119d0 tx=0x7f3f0c011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:57.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.120+0000 7f3f19600640 1 -- 172.21.15.46:0/2892850862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c00faf0 con 0x7f3f00004d10 2024-03-29T08:19:57.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.120+0000 7f3f19600640 1 -- 172.21.15.46:0/2892850862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0c00fc50 con 0x7f3f00004d10 2024-03-29T08:19:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.120+0000 7f3f23400640 1 -- 172.21.15.46:0/2892850862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1c0b3c80 con 0x7f3f00004d10 2024-03-29T08:19:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.120+0000 7f3f19600640 1 -- 172.21.15.46:0/2892850862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c014440 con 0x7f3f00004d10 2024-03-29T08:19:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.120+0000 7f3f23400640 1 -- 172.21.15.46:0/2892850862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1c0b4240 con 0x7f3f00004d10 2024-03-29T08:19:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.120+0000 7f3f23400640 1 -- 172.21.15.46:0/2892850862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f1c066c90 con 0x7f3f00004d10 2024-03-29T08:19:57.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.120+0000 7f3f19600640 1 -- 172.21.15.46:0/2892850862 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3f0c0145a0 con 0x7f3f00004d10 2024-03-29T08:19:57.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.120+0000 7f3f19600640 1 --2- 172.21.15.46:0/2892850862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3ef8066690 0x7f3ef8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:57.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.120+0000 7f3f19600640 1 -- 172.21.15.46:0/2892850862 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f3f0c0849d0 con 0x7f3f00004d10 2024-03-29T08:19:57.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.120+0000 7f3f13e00640 1 --2- 172.21.15.46:0/2892850862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3ef8066690 0x7f3ef8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:57.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.124+0000 7f3f13e00640 1 --2- 172.21.15.46:0/2892850862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3ef8066690 0x7f3ef8068b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0b2da0 tx=0x7f3f04009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:57.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.124+0000 7f3f19600640 1 -- 172.21.15.46:0/2892850862 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3f0c051d20 con 0x7f3f00004d10 2024-03-29T08:19:57.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.288+0000 7f3f23400640 1 -- 172.21.15.46:0/2892850862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3f1c0cd7b0 con 0x7f3f00004d10 2024-03-29T08:19:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.940+0000 7f3f19600640 1 -- 172.21.15.46:0/2892850862 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3f0c022020 con 0x7f3f00004d10 2024-03-29T08:19:57.943 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-03-29T08:19:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.944+0000 7f3f23400640 1 -- 172.21.15.46:0/2892850862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3ef8066690 msgr2=0x7f3ef8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.944+0000 7f3f23400640 1 --2- 172.21.15.46:0/2892850862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3ef8066690 0x7f3ef8068b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c0b2da0 tx=0x7f3f04009190 comp rx=0 tx=0).stop 2024-03-29T08:19:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.944+0000 7f3f23400640 1 -- 172.21.15.46:0/2892850862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f00004d10 msgr2=0x7f3f1c1f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.944+0000 7f3f23400640 1 --2- 172.21.15.46:0/2892850862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f00004d10 0x7f3f1c1f64d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0119d0 tx=0x7f3f0c011dd0 comp rx=0 tx=0).stop 2024-03-29T08:19:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.944+0000 7f3f23400640 1 -- 172.21.15.46:0/2892850862 shutdown_connections 2024-03-29T08:19:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.944+0000 7f3f23400640 1 --2- 172.21.15.46:0/2892850862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3ef8066690 0x7f3ef8068b50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.944+0000 7f3f23400640 1 --2- 172.21.15.46:0/2892850862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f1c0b2840 0x7f3f1c151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.944+0000 7f3f23400640 1 --2- 172.21.15.46:0/2892850862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f1c0b23c0 0x7f3f1c151760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:57.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.944+0000 7f3f23400640 1 --2- 172.21.15.46:0/2892850862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f00004d10 0x7f3f1c1f64d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:57.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.944+0000 7f3f23400640 1 -- 172.21.15.46:0/2892850862 >> 172.21.15.46:0/2892850862 conn(0x7f3f1c089fd0 msgr2=0x7f3f1c08a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:57.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.944+0000 7f3f23400640 1 -- 172.21.15.46:0/2892850862 shutdown_connections 2024-03-29T08:19:57.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:57.944+0000 7f3f23400640 1 -- 172.21.15.46:0/2892850862 wait complete. 2024-03-29T08:19:57.970 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T08:19:58.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.116+0000 7f3f8ca00640 1 Processor -- start 2024-03-29T08:19:58.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.116+0000 7f3f8ca00640 1 -- start start 2024-03-29T08:19:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.120+0000 7f3f8ca00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3f88160060 0x7f3f880cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.120+0000 7f3f8ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3f880b1780 0x7f3f880cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.120+0000 7f3f8ca00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f880cfa40 0x7f3f880cfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.120+0000 7f3f8ca00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f880cea60 con 0x7f3f880b1780 2024-03-29T08:19:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.120+0000 7f3f8ca00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f8806bea0 con 0x7f3f88160060 2024-03-29T08:19:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.120+0000 7f3f8ca00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880ced00 con 0x7f3f880cfa40 2024-03-29T08:19:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.120+0000 7f3f85400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3f880b1780 0x7f3f880cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.120+0000 7f3f85400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3f880b1780 0x7f3f880cf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54734/0 (socket says 172.21.15.46:54734) 2024-03-29T08:19:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.120+0000 7f3f85400640 1 -- 172.21.15.46:0/1840268093 learned_addr learned my addr 172.21.15.46:0/1840268093 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.120+0000 7f3f86800640 1 --2- 172.21.15.46:0/1840268093 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f880cfa40 0x7f3f880cfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:58.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.120+0000 7f3f85e00640 1 --2- 172.21.15.46:0/1840268093 >> v2:172.21.15.73:3300/0 conn(0x7f3f88160060 0x7f3f880cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.120+0000 7f3f85400640 1 -- 172.21.15.46:0/1840268093 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f880cfa40 msgr2=0x7f3f880cfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.120+0000 7f3f85400640 1 --2- 172.21.15.46:0/1840268093 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f880cfa40 0x7f3f880cfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.120+0000 7f3f85400640 1 -- 172.21.15.46:0/1840268093 >> v2:172.21.15.73:3300/0 conn(0x7f3f88160060 msgr2=0x7f3f880cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.120+0000 7f3f85400640 1 --2- 172.21.15.46:0/1840268093 >> v2:172.21.15.73:3300/0 conn(0x7f3f88160060 0x7f3f880cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:58.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.120+0000 7f3f85400640 1 -- 172.21.15.46:0/1840268093 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f880cefe0 con 0x7f3f880b1780 2024-03-29T08:19:58.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.124+0000 7f3f85400640 1 --2- 172.21.15.46:0/1840268093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880b1780 0x7f3f880cf500 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3f8806e2a0 tx=0x7f3f7401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5af8c99d9db765 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:58.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.124+0000 7f3f84a00640 1 -- 172.21.15.46:0/1840268093 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f740ab600 con 0x7f3f880b1780 2024-03-29T08:19:58.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.124+0000 7f3f84a00640 1 -- 172.21.15.46:0/1840268093 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f740a7050 con 0x7f3f880b1780 2024-03-29T08:19:58.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.124+0000 7f3f84a00640 1 -- 172.21.15.46:0/1840268093 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f740ab980 con 0x7f3f880b1780 2024-03-29T08:19:58.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.124+0000 7f3f8ca00640 1 -- 172.21.15.46:0/1840268093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880b1780 msgr2=0x7f3f880cf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:58.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.124+0000 7f3f8ca00640 1 --2- 172.21.15.46:0/1840268093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880b1780 0x7f3f880cf500 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3f8806e2a0 tx=0x7f3f7401c490 comp rx=0 tx=0).stop 2024-03-29T08:19:58.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.124+0000 7f3f8ca00640 1 -- 172.21.15.46:0/1840268093 shutdown_connections 2024-03-29T08:19:58.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.124+0000 7f3f8ca00640 1 --2- 172.21.15.46:0/1840268093 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f880cfa40 0x7f3f880cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:58.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.124+0000 7f3f8ca00640 1 --2- 172.21.15.46:0/1840268093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880b1780 0x7f3f880cf500 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:58.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.124+0000 7f3f8ca00640 1 --2- 172.21.15.46:0/1840268093 >> v2:172.21.15.73:3300/0 conn(0x7f3f88160060 0x7f3f880cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:58.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.124+0000 7f3f8ca00640 1 -- 172.21.15.46:0/1840268093 >> 172.21.15.46:0/1840268093 conn(0x7f3f880b21b0 msgr2=0x7f3f880b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:58.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.124+0000 7f3f8ca00640 1 -- 172.21.15.46:0/1840268093 shutdown_connections 2024-03-29T08:19:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.124+0000 7f3f8ca00640 1 -- 172.21.15.46:0/1840268093 wait complete. 2024-03-29T08:19:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.124+0000 7f3f8ca00640 1 Processor -- start 2024-03-29T08:19:58.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.124+0000 7f3f8ca00640 1 -- start start 2024-03-29T08:19:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f8ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880b1780 0x7f3f881584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f8ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f880cfa40 0x7f3f880c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f8ca00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f88160060 0x7f3f880c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f8ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f881580d0 con 0x7f3f880b1780 2024-03-29T08:19:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f8ca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880c4c70 con 0x7f3f880cfa40 2024-03-29T08:19:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f8ca00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8815eb30 con 0x7f3f88160060 2024-03-29T08:19:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f85e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880b1780 0x7f3f881584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f85e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880b1780 0x7f3f881584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54742/0 (socket says 172.21.15.46:54742) 2024-03-29T08:19:58.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f85e00640 1 -- 172.21.15.46:0/1627856735 learned_addr learned my addr 172.21.15.46:0/1627856735 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f86800640 1 --2- 172.21.15.46:0/1627856735 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f88160060 0x7f3f880c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f85e00640 1 -- 172.21.15.46:0/1627856735 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f88160060 msgr2=0x7f3f880c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f85400640 1 --2- 172.21.15.46:0/1627856735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f880cfa40 0x7f3f880c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f85e00640 1 --2- 172.21.15.46:0/1627856735 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f88160060 0x7f3f880c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f85e00640 1 -- 172.21.15.46:0/1627856735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f880cfa40 msgr2=0x7f3f880c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f85e00640 1 --2- 172.21.15.46:0/1627856735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f880cfa40 0x7f3f880c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f85e00640 1 -- 172.21.15.46:0/1627856735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f740094d0 con 0x7f3f880b1780 2024-03-29T08:19:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f86800640 1 --2- 172.21.15.46:0/1627856735 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f88160060 0x7f3f880c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f85400640 1 --2- 172.21.15.46:0/1627856735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f880cfa40 0x7f3f880c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f85e00640 1 --2- 172.21.15.46:0/1627856735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880b1780 0x7f3f881584f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800e870 tx=0x7f3f7800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f7e000640 1 -- 172.21.15.46:0/1627856735 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78009510 con 0x7f3f880b1780 2024-03-29T08:19:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f7e000640 1 -- 172.21.15.46:0/1627856735 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7800eec0 con 0x7f3f880b1780 2024-03-29T08:19:58.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f8ca00640 1 -- 172.21.15.46:0/1627856735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f88157a10 con 0x7f3f880b1780 2024-03-29T08:19:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f7e000640 1 -- 172.21.15.46:0/1627856735 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7800cb40 con 0x7f3f880b1780 2024-03-29T08:19:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f8ca00640 1 -- 172.21.15.46:0/1627856735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f88158a30 con 0x7f3f880b1780 2024-03-29T08:19:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f8ca00640 1 -- 172.21.15.46:0/1627856735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f88066c90 con 0x7f3f880b1780 2024-03-29T08:19:58.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.128+0000 7f3f7e000640 1 -- 172.21.15.46:0/1627856735 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3f7800cca0 con 0x7f3f880b1780 2024-03-29T08:19:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.132+0000 7f3f7e000640 1 --2- 172.21.15.46:0/1627856735 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3f68066690 0x7f3f68068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.132+0000 7f3f7e000640 1 -- 172.21.15.46:0/1627856735 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f3f78084270 con 0x7f3f880b1780 2024-03-29T08:19:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.132+0000 7f3f85400640 1 --2- 172.21.15.46:0/1627856735 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3f68066690 0x7f3f68068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:58.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.132+0000 7f3f85400640 1 --2- 172.21.15.46:0/1627856735 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3f68066690 0x7f3f68068b50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3f740ad000 tx=0x7f3f74008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:58.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.136+0000 7f3f7e000640 1 -- 172.21.15.46:0/1627856735 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3f78051590 con 0x7f3f880b1780 2024-03-29T08:19:58.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.296+0000 7f3f8ca00640 1 -- 172.21.15.46:0/1627856735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3f880cb030 con 0x7f3f880b1780 2024-03-29T08:19:58.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.947+0000 7f3f7e000640 1 -- 172.21.15.46:0/1627856735 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3f78054bb0 con 0x7f3f880b1780 2024-03-29T08:19:58.950 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-03-29T08:19:58.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.951+0000 7f3f8ca00640 1 -- 172.21.15.46:0/1627856735 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3f68066690 msgr2=0x7f3f68068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:58.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.951+0000 7f3f8ca00640 1 --2- 172.21.15.46:0/1627856735 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3f68066690 0x7f3f68068b50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3f740ad000 tx=0x7f3f74008040 comp rx=0 tx=0).stop 2024-03-29T08:19:58.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.951+0000 7f3f8ca00640 1 -- 172.21.15.46:0/1627856735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880b1780 msgr2=0x7f3f881584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:58.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.951+0000 7f3f8ca00640 1 --2- 172.21.15.46:0/1627856735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880b1780 0x7f3f881584f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800e870 tx=0x7f3f7800ec70 comp rx=0 tx=0).stop 2024-03-29T08:19:58.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.955+0000 7f3f8ca00640 1 -- 172.21.15.46:0/1627856735 shutdown_connections 2024-03-29T08:19:58.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.955+0000 7f3f8ca00640 1 --2- 172.21.15.46:0/1627856735 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3f68066690 0x7f3f68068b50 secure :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f3f740ad000 tx=0x7f3f74008040 comp rx=0 tx=0).stop 2024-03-29T08:19:58.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.955+0000 7f3f8ca00640 1 --2- 172.21.15.46:0/1627856735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f880cfa40 0x7f3f880c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:58.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.955+0000 7f3f8ca00640 1 --2- 172.21.15.46:0/1627856735 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f88160060 0x7f3f880c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:58.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.955+0000 7f3f8ca00640 1 --2- 172.21.15.46:0/1627856735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f880b1780 0x7f3f881584f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:58.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.955+0000 7f3f8ca00640 1 -- 172.21.15.46:0/1627856735 >> 172.21.15.46:0/1627856735 conn(0x7f3f880b21b0 msgr2=0x7f3f8806e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:58.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.955+0000 7f3f8ca00640 1 -- 172.21.15.46:0/1627856735 shutdown_connections 2024-03-29T08:19:58.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:58.955+0000 7f3f8ca00640 1 -- 172.21.15.46:0/1627856735 wait complete. 2024-03-29T08:19:58.986 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-29T08:19:59.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.131+0000 7f1f3d200640 1 Processor -- start 2024-03-29T08:19:59.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.131+0000 7f1f3d200640 1 -- start start 2024-03-29T08:19:59.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.131+0000 7f1f3d200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f3815a7f0 0x7f1f380cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:59.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.131+0000 7f1f3d200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1f3815abd0 0x7f1f380cb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:59.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.131+0000 7f1f3d200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f380cbd00 con 0x7f1f38098a40 2024-03-29T08:19:59.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.131+0000 7f1f3d200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f380b4500 con 0x7f1f3815abd0 2024-03-29T08:19:59.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.131+0000 7f1f3d200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380cf970 con 0x7f1f3815a7f0 2024-03-29T08:19:59.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.131+0000 7f1f35e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f3815a7f0 0x7f1f380cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.131+0000 7f1f2d400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1f3815abd0 0x7f1f380cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.131+0000 7f1f35e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f3815a7f0 0x7f1f380cf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:60936/0 (socket says 172.21.15.46:60936) 2024-03-29T08:19:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.131+0000 7f1f36800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1f38098a40 0x7f1f3815e580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:37062/0 (socket says 172.21.15.46:37062) 2024-03-29T08:19:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.131+0000 7f1f35e00640 1 -- 172.21.15.46:0/1443548641 learned_addr learned my addr 172.21.15.46:0/1443548641 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.131+0000 7f1f35400640 1 -- 172.21.15.46:0/1443548641 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225975489 0 0) 0x7f1f380cbd00 con 0x7f1f38098a40 2024-03-29T08:19:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.131+0000 7f1f35e00640 1 -- 172.21.15.46:0/1443548641 >> v2:172.21.15.73:3300/0 conn(0x7f1f3815abd0 msgr2=0x7f1f380cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.131+0000 7f1f35e00640 1 --2- 172.21.15.46:0/1443548641 >> v2:172.21.15.73:3300/0 conn(0x7f1f3815abd0 0x7f1f380cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.131+0000 7f1f35e00640 1 -- 172.21.15.46:0/1443548641 >> v1:172.21.15.73:6789/0 conn(0x7f1f38098a40 legacy=0x7f1f3815e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:59.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.131+0000 7f1f35e00640 1 -- 172.21.15.46:0/1443548641 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f380cbfe0 con 0x7f1f3815a7f0 2024-03-29T08:19:59.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.135+0000 7f1f35e00640 1 --2- 172.21.15.46:0/1443548641 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f3815a7f0 0x7f1f380cf430 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009870 tx=0x7f1f2801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cc1924ffdd4f2ed server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:59.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.135+0000 7f1f35400640 1 -- 172.21.15.46:0/1443548641 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f280a95f0 con 0x7f1f3815a7f0 2024-03-29T08:19:59.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.135+0000 7f1f35400640 1 -- 172.21.15.46:0/1443548641 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f280a9750 con 0x7f1f3815a7f0 2024-03-29T08:19:59.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.135+0000 7f1f35400640 1 -- 172.21.15.46:0/1443548641 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f280a9ae0 con 0x7f1f3815a7f0 2024-03-29T08:19:59.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.135+0000 7f1f3d200640 1 -- 172.21.15.46:0/1443548641 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f3815a7f0 msgr2=0x7f1f380cf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:59.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.135+0000 7f1f3d200640 1 --2- 172.21.15.46:0/1443548641 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f3815a7f0 0x7f1f380cf430 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009870 tx=0x7f1f2801c530 comp rx=0 tx=0).stop 2024-03-29T08:19:59.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.135+0000 7f1f3d200640 1 -- 172.21.15.46:0/1443548641 shutdown_connections 2024-03-29T08:19:59.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.135+0000 7f1f3d200640 1 --2- 172.21.15.46:0/1443548641 >> v2:172.21.15.73:3300/0 conn(0x7f1f3815abd0 0x7f1f380cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.135+0000 7f1f3d200640 1 --2- 172.21.15.46:0/1443548641 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f3815a7f0 0x7f1f380cf430 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.135+0000 7f1f3d200640 1 -- 172.21.15.46:0/1443548641 >> 172.21.15.46:0/1443548641 conn(0x7f1f380b2190 msgr2=0x7f1f380b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.135+0000 7f1f3d200640 1 -- 172.21.15.46:0/1443548641 shutdown_connections 2024-03-29T08:19:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.135+0000 7f1f3d200640 1 -- 172.21.15.46:0/1443548641 wait complete. 2024-03-29T08:19:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f3d200640 1 Processor -- start 2024-03-29T08:19:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f3d200640 1 -- start start 2024-03-29T08:19:59.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f3d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f38098a40 0x7f1f380c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:59.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f3d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f3815a7f0 0x7f1f3814fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:59.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f3d200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f3815abd0 0x7f1f3806bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:59.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f3d200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380c51a0 con 0x7f1f38098a40 2024-03-29T08:19:59.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f3d200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f381f5660 con 0x7f1f3815a7f0 2024-03-29T08:19:59.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f3d200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380c4590 con 0x7f1f3815abd0 2024-03-29T08:19:59.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f35e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f38098a40 0x7f1f380c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f35e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f38098a40 0x7f1f380c3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54758/0 (socket says 172.21.15.46:54758) 2024-03-29T08:19:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f35e00640 1 -- 172.21.15.46:0/3072950597 learned_addr learned my addr 172.21.15.46:0/3072950597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f36800640 1 --2- 172.21.15.46:0/3072950597 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f3815abd0 0x7f1f3806bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f2d400640 1 --2- 172.21.15.46:0/3072950597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f3815a7f0 0x7f1f3814fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f35e00640 1 -- 172.21.15.46:0/3072950597 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f3815abd0 msgr2=0x7f1f3806bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f35e00640 1 --2- 172.21.15.46:0/3072950597 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f3815abd0 0x7f1f3806bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f35e00640 1 -- 172.21.15.46:0/3072950597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f3815a7f0 msgr2=0x7f1f3814fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f35e00640 1 --2- 172.21.15.46:0/3072950597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f3815a7f0 0x7f1f3814fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f35e00640 1 -- 172.21.15.46:0/3072950597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f280094d0 con 0x7f1f38098a40 2024-03-29T08:19:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f2d400640 1 --2- 172.21.15.46:0/3072950597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f3815a7f0 0x7f1f3814fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f36800640 1 --2- 172.21.15.46:0/3072950597 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f3815abd0 0x7f1f3806bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:19:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f35e00640 1 --2- 172.21.15.46:0/3072950597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f38098a40 0x7f1f380c3f00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1f280ab040 tx=0x7f1f2801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f2ea00640 1 -- 172.21.15.46:0/3072950597 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f280b1800 con 0x7f1f38098a40 2024-03-29T08:19:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f2ea00640 1 -- 172.21.15.46:0/3072950597 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f280b1980 con 0x7f1f38098a40 2024-03-29T08:19:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f3d200640 1 -- 172.21.15.46:0/3072950597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f380c4870 con 0x7f1f38098a40 2024-03-29T08:19:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f2ea00640 1 -- 172.21.15.46:0/3072950597 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f280b1d30 con 0x7f1f38098a40 2024-03-29T08:19:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f3d200640 1 -- 172.21.15.46:0/3072950597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f380c4e30 con 0x7f1f38098a40 2024-03-29T08:19:59.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.139+0000 7f1f3d200640 1 -- 172.21.15.46:0/3072950597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f3809e690 con 0x7f1f38098a40 2024-03-29T08:19:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.143+0000 7f1f2ea00640 1 -- 172.21.15.46:0/3072950597 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1f280bc450 con 0x7f1f38098a40 2024-03-29T08:19:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.143+0000 7f1f2ea00640 1 --2- 172.21.15.46:0/3072950597 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f14066690 0x7f1f14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.143+0000 7f1f2ea00640 1 -- 172.21.15.46:0/3072950597 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1f2811e840 con 0x7f1f38098a40 2024-03-29T08:19:59.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.143+0000 7f1f2d400640 1 --2- 172.21.15.46:0/3072950597 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f14066690 0x7f1f14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:59.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.143+0000 7f1f2d400640 1 --2- 172.21.15.46:0/3072950597 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f14066690 0x7f1f14068b50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1f380d0270 tx=0x7f1f20008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:59.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.147+0000 7f1f2ea00640 1 -- 172.21.15.46:0/3072950597 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1f280ebe10 con 0x7f1f38098a40 2024-03-29T08:19:59.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.311+0000 7f1f3d200640 1 -- 172.21.15.46:0/3072950597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1f3804e120 con 0x7f1f38098a40 2024-03-29T08:19:59.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.315+0000 7f1f2ea00640 1 -- 172.21.15.46:0/3072950597 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1f280ef430 con 0x7f1f38098a40 2024-03-29T08:19:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.315+0000 7f1f3d200640 1 -- 172.21.15.46:0/3072950597 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f14066690 msgr2=0x7f1f14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.315+0000 7f1f3d200640 1 --2- 172.21.15.46:0/3072950597 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f14066690 0x7f1f14068b50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1f380d0270 tx=0x7f1f20008040 comp rx=0 tx=0).stop 2024-03-29T08:19:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.315+0000 7f1f3d200640 1 -- 172.21.15.46:0/3072950597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f38098a40 msgr2=0x7f1f380c3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.315+0000 7f1f3d200640 1 --2- 172.21.15.46:0/3072950597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f38098a40 0x7f1f380c3f00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1f280ab040 tx=0x7f1f2801ad70 comp rx=0 tx=0).stop 2024-03-29T08:19:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.315+0000 7f1f3d200640 1 -- 172.21.15.46:0/3072950597 shutdown_connections 2024-03-29T08:19:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.315+0000 7f1f3d200640 1 --2- 172.21.15.46:0/3072950597 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f14066690 0x7f1f14068b50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.315+0000 7f1f3d200640 1 --2- 172.21.15.46:0/3072950597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f3815a7f0 0x7f1f3814fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.315+0000 7f1f3d200640 1 --2- 172.21.15.46:0/3072950597 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f3815abd0 0x7f1f3806bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.315+0000 7f1f3d200640 1 --2- 172.21.15.46:0/3072950597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f38098a40 0x7f1f380c3f00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.315+0000 7f1f3d200640 1 -- 172.21.15.46:0/3072950597 >> 172.21.15.46:0/3072950597 conn(0x7f1f380b2190 msgr2=0x7f1f3806fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.315+0000 7f1f3d200640 1 -- 172.21.15.46:0/3072950597 shutdown_connections 2024-03-29T08:19:59.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.315+0000 7f1f3d200640 1 -- 172.21.15.46:0/3072950597 wait complete. 2024-03-29T08:19:59.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-03-29T08:19:59.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad9ac00640 1 Processor -- start 2024-03-29T08:19:59.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad9ac00640 1 -- start start 2024-03-29T08:19:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad9ac00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad94144210 0x7fad940cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad9ac00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fad9415ada0 0x7fad940ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad9ac00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad940ce950 0x7fad9415e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad9ac00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad940cede0 con 0x7fad940ce950 2024-03-29T08:19:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad9ac00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad940cb510 con 0x7fad9415ada0 2024-03-29T08:19:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad9ac00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad94070580 con 0x7fad94144210 2024-03-29T08:19:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad98c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad940ce950 0x7fad9415e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad98c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad940ce950 0x7fad9415e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54766/0 (socket says 172.21.15.46:54766) 2024-03-29T08:19:59.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad98c00640 1 -- 172.21.15.46:0/3195135779 learned_addr learned my addr 172.21.15.46:0/3195135779 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad93400640 1 --2- 172.21.15.46:0/3195135779 >> v2:172.21.15.73:3300/0 conn(0x7fad9415ada0 0x7fad940ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad93e00640 1 --2- 172.21.15.46:0/3195135779 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad94144210 0x7fad940cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad98c00640 1 -- 172.21.15.46:0/3195135779 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad94144210 msgr2=0x7fad940cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad98c00640 1 --2- 172.21.15.46:0/3195135779 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad94144210 0x7fad940cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad98c00640 1 -- 172.21.15.46:0/3195135779 >> v2:172.21.15.73:3300/0 conn(0x7fad9415ada0 msgr2=0x7fad940ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:19:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad98c00640 1 --2- 172.21.15.46:0/3195135779 >> v2:172.21.15.73:3300/0 conn(0x7fad9415ada0 0x7fad940ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.487+0000 7fad98c00640 1 -- 172.21.15.46:0/3195135779 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad94070860 con 0x7fad940ce950 2024-03-29T08:19:59.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.491+0000 7fad98c00640 1 --2- 172.21.15.46:0/3195135779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad940ce950 0x7fad9415e360 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fad940b39d0 tx=0x7fad8c01c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860efc2aeb613a7c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.491+0000 7fad92a00640 1 -- 172.21.15.46:0/3195135779 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0a64b0 con 0x7fad940ce950 2024-03-29T08:19:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.491+0000 7fad92a00640 1 -- 172.21.15.46:0/3195135779 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8c0a6610 con 0x7fad940ce950 2024-03-29T08:19:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.491+0000 7fad92a00640 1 -- 172.21.15.46:0/3195135779 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0a69a0 con 0x7fad940ce950 2024-03-29T08:19:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.491+0000 7fad9ac00640 1 -- 172.21.15.46:0/3195135779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad940ce950 msgr2=0x7fad9415e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.491+0000 7fad9ac00640 1 --2- 172.21.15.46:0/3195135779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad940ce950 0x7fad9415e360 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fad940b39d0 tx=0x7fad8c01c730 comp rx=0 tx=0).stop 2024-03-29T08:19:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.491+0000 7fad9ac00640 1 -- 172.21.15.46:0/3195135779 shutdown_connections 2024-03-29T08:19:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.491+0000 7fad9ac00640 1 --2- 172.21.15.46:0/3195135779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad940ce950 0x7fad9415e360 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.491+0000 7fad9ac00640 1 --2- 172.21.15.46:0/3195135779 >> v2:172.21.15.73:3300/0 conn(0x7fad9415ada0 0x7fad940ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.491+0000 7fad9ac00640 1 --2- 172.21.15.46:0/3195135779 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad94144210 0x7fad940cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.491+0000 7fad9ac00640 1 -- 172.21.15.46:0/3195135779 >> 172.21.15.46:0/3195135779 conn(0x7fad94066110 msgr2=0x7fad94069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.491+0000 7fad9ac00640 1 -- 172.21.15.46:0/3195135779 shutdown_connections 2024-03-29T08:19:59.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.491+0000 7fad9ac00640 1 -- 172.21.15.46:0/3195135779 wait complete. 2024-03-29T08:19:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad9ac00640 1 Processor -- start 2024-03-29T08:19:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad9ac00640 1 -- start start 2024-03-29T08:19:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad9ac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad940ce950 0x7fad9406ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad9ac00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad94144210 0x7fad94158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad9ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad9415ada0 0x7fad94159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad9ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9415b510 con 0x7fad9415ada0 2024-03-29T08:19:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad9ac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9415a5e0 con 0x7fad940ce950 2024-03-29T08:19:59.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad9ac00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9415b1c0 con 0x7fad94144210 2024-03-29T08:19:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad98c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad9415ada0 0x7fad94159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad98c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad9415ada0 0x7fad94159210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54782/0 (socket says 172.21.15.46:54782) 2024-03-29T08:19:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad98c00640 1 -- 172.21.15.46:0/1151711670 learned_addr learned my addr 172.21.15.46:0/1151711670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:19:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad93e00640 1 --2- 172.21.15.46:0/1151711670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad940ce950 0x7fad9406ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad93400640 1 --2- 172.21.15.46:0/1151711670 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad94144210 0x7fad94158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad98c00640 1 -- 172.21.15.46:0/1151711670 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad94144210 msgr2=0x7fad94158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad98c00640 1 --2- 172.21.15.46:0/1151711670 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad94144210 0x7fad94158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad98c00640 1 -- 172.21.15.46:0/1151711670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad940ce950 msgr2=0x7fad9406ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad98c00640 1 --2- 172.21.15.46:0/1151711670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad940ce950 0x7fad9406ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad98c00640 1 -- 172.21.15.46:0/1151711670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c009b30 con 0x7fad9415ada0 2024-03-29T08:19:59.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad98c00640 1 --2- 172.21.15.46:0/1151711670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad9415ada0 0x7fad94159210 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fad8c01ade0 tx=0x7fad8c0194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:59.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad8be00640 1 -- 172.21.15.46:0/1151711670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0b4070 con 0x7fad9415ada0 2024-03-29T08:19:59.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad8be00640 1 -- 172.21.15.46:0/1151711670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8c0090d0 con 0x7fad9415ada0 2024-03-29T08:19:59.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad8be00640 1 -- 172.21.15.46:0/1151711670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8c0a69c0 con 0x7fad9415ada0 2024-03-29T08:19:59.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad9ac00640 1 -- 172.21.15.46:0/1151711670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad941598e0 con 0x7fad9415ada0 2024-03-29T08:19:59.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.495+0000 7fad9ac00640 1 -- 172.21.15.46:0/1151711670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad94159ea0 con 0x7fad9415ada0 2024-03-29T08:19:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.499+0000 7fad9ac00640 1 -- 172.21.15.46:0/1151711670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad940ce280 con 0x7fad9415ada0 2024-03-29T08:19:59.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.499+0000 7fad8be00640 1 -- 172.21.15.46:0/1151711670 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fad8c0b8070 con 0x7fad9415ada0 2024-03-29T08:19:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.499+0000 7fad8be00640 1 --2- 172.21.15.46:0/1151711670 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fad740666c0 0x7fad74068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:19:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.499+0000 7fad8be00640 1 -- 172.21.15.46:0/1151711670 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fad8c0aa070 con 0x7fad9415ada0 2024-03-29T08:19:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.499+0000 7fad93e00640 1 --2- 172.21.15.46:0/1151711670 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fad740666c0 0x7fad74068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:19:59.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.499+0000 7fad93e00640 1 --2- 172.21.15.46:0/1151711670 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fad740666c0 0x7fad74068b80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fad84009ae0 tx=0x7fad84009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:19:59.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.503+0000 7fad8be00640 1 -- 172.21.15.46:0/1151711670 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fad8c0eae40 con 0x7fad9415ada0 2024-03-29T08:19:59.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.723+0000 7fad9ac00640 1 -- 172.21.15.46:0/1151711670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fad9415b650 con 0x7fad9415ada0 2024-03-29T08:19:59.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.947+0000 7fad8be00640 1 -- 172.21.15.46:0/1151711670 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v115) v1 ==== 142+0+0 (secure 0 0 0) 0x7fad9415b650 con 0x7fad9415ada0 2024-03-29T08:19:59.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.951+0000 7fad9ac00640 1 -- 172.21.15.46:0/1151711670 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fad740666c0 msgr2=0x7fad74068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:59.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.951+0000 7fad9ac00640 1 --2- 172.21.15.46:0/1151711670 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fad740666c0 0x7fad74068b80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fad84009ae0 tx=0x7fad84009190 comp rx=0 tx=0).stop 2024-03-29T08:19:59.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.951+0000 7fad9ac00640 1 -- 172.21.15.46:0/1151711670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad9415ada0 msgr2=0x7fad94159210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:19:59.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.951+0000 7fad9ac00640 1 --2- 172.21.15.46:0/1151711670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad9415ada0 0x7fad94159210 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fad8c01ade0 tx=0x7fad8c0194d0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.951+0000 7fad9ac00640 1 -- 172.21.15.46:0/1151711670 shutdown_connections 2024-03-29T08:19:59.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.951+0000 7fad9ac00640 1 --2- 172.21.15.46:0/1151711670 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fad740666c0 0x7fad74068b80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.951+0000 7fad9ac00640 1 --2- 172.21.15.46:0/1151711670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad9415ada0 0x7fad94159210 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.951+0000 7fad9ac00640 1 --2- 172.21.15.46:0/1151711670 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fad94144210 0x7fad94158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.951+0000 7fad9ac00640 1 --2- 172.21.15.46:0/1151711670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad940ce950 0x7fad9406ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:19:59.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.951+0000 7fad9ac00640 1 -- 172.21.15.46:0/1151711670 >> 172.21.15.46:0/1151711670 conn(0x7fad94066110 msgr2=0x7fad9415dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:19:59.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.951+0000 7fad9ac00640 1 -- 172.21.15.46:0/1151711670 shutdown_connections 2024-03-29T08:19:59.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:19:59.951+0000 7fad9ac00640 1 -- 172.21.15.46:0/1151711670 wait complete. 2024-03-29T08:19:59.973 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-03-29T08:19:59.974 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-03-29T08:19:59.974 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-29T08:19:59.974 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-03-29T08:20:00.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f603d400640 1 Processor -- start 2024-03-29T08:20:00.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f603d400640 1 -- start start 2024-03-29T08:20:00.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f603d400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6038144210 0x7f60380cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:00.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f603d400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f603815ada0 0x7f60380ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:00.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f603d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60380ce950 0x7f603815e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f603d400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60380cede0 con 0x7f60380ce950 2024-03-29T08:20:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f603d400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60380cb510 con 0x7f6038144210 2024-03-29T08:20:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f603d400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6038070580 con 0x7f603815ada0 2024-03-29T08:20:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f6037200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60380ce950 0x7f603815e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f6037200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60380ce950 0x7f603815e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54786/0 (socket says 172.21.15.46:54786) 2024-03-29T08:20:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f6037200640 1 -- 172.21.15.46:0/2496138413 learned_addr learned my addr 172.21.15.46:0/2496138413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f6036800640 1 --2- 172.21.15.46:0/2496138413 >> v2:172.21.15.73:3300/0 conn(0x7f6038144210 0x7f60380cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f602de00640 1 --2- 172.21.15.46:0/2496138413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f603815ada0 0x7f60380ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f6037200640 1 -- 172.21.15.46:0/2496138413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f603815ada0 msgr2=0x7f60380ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f6037200640 1 --2- 172.21.15.46:0/2496138413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f603815ada0 0x7f60380ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f6037200640 1 -- 172.21.15.46:0/2496138413 >> v2:172.21.15.73:3300/0 conn(0x7f6038144210 msgr2=0x7f60380cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f6037200640 1 --2- 172.21.15.46:0/2496138413 >> v2:172.21.15.73:3300/0 conn(0x7f6038144210 0x7f60380cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:00.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.115+0000 7f6037200640 1 -- 172.21.15.46:0/2496138413 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6038070860 con 0x7f60380ce950 2024-03-29T08:20:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.119+0000 7f6037200640 1 --2- 172.21.15.46:0/2496138413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60380ce950 0x7f603815e360 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6038071360 tx=0x7f603001c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f367e1f684d0276 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.119+0000 7f6035e00640 1 -- 172.21.15.46:0/2496138413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60300a64b0 con 0x7f60380ce950 2024-03-29T08:20:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.119+0000 7f6035e00640 1 -- 172.21.15.46:0/2496138413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60300a6610 con 0x7f60380ce950 2024-03-29T08:20:00.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.119+0000 7f6035e00640 1 -- 172.21.15.46:0/2496138413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60300a69a0 con 0x7f60380ce950 2024-03-29T08:20:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.119+0000 7f603d400640 1 -- 172.21.15.46:0/2496138413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60380ce950 msgr2=0x7f603815e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.119+0000 7f603d400640 1 --2- 172.21.15.46:0/2496138413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60380ce950 0x7f603815e360 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6038071360 tx=0x7f603001c730 comp rx=0 tx=0).stop 2024-03-29T08:20:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.119+0000 7f603d400640 1 -- 172.21.15.46:0/2496138413 shutdown_connections 2024-03-29T08:20:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.119+0000 7f603d400640 1 --2- 172.21.15.46:0/2496138413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60380ce950 0x7f603815e360 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.119+0000 7f603d400640 1 --2- 172.21.15.46:0/2496138413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f603815ada0 0x7f60380ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.119+0000 7f603d400640 1 --2- 172.21.15.46:0/2496138413 >> v2:172.21.15.73:3300/0 conn(0x7f6038144210 0x7f60380cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.119+0000 7f603d400640 1 -- 172.21.15.46:0/2496138413 >> 172.21.15.46:0/2496138413 conn(0x7f6038066110 msgr2=0x7f6038069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.119+0000 7f603d400640 1 -- 172.21.15.46:0/2496138413 shutdown_connections 2024-03-29T08:20:00.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.119+0000 7f603d400640 1 -- 172.21.15.46:0/2496138413 wait complete. 2024-03-29T08:20:00.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f603d400640 1 Processor -- start 2024-03-29T08:20:00.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f603d400640 1 -- start start 2024-03-29T08:20:00.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f603d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60380ce950 0x7f603813d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f603d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6038144210 0x7f6038142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f603d400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f603815ada0 0x7f603813f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f603d400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603813d5a0 con 0x7f60380ce950 2024-03-29T08:20:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f603d400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603813fd40 con 0x7f6038144210 2024-03-29T08:20:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f603d400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603813cc00 con 0x7f603815ada0 2024-03-29T08:20:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f6036800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60380ce950 0x7f603813d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f6036800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60380ce950 0x7f603813d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54802/0 (socket says 172.21.15.46:54802) 2024-03-29T08:20:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f6036800640 1 -- 172.21.15.46:0/2715776515 learned_addr learned my addr 172.21.15.46:0/2715776515 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f602de00640 1 --2- 172.21.15.46:0/2715776515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6038144210 0x7f6038142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f6037200640 1 --2- 172.21.15.46:0/2715776515 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f603815ada0 0x7f603813f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f6036800640 1 -- 172.21.15.46:0/2715776515 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f603815ada0 msgr2=0x7f603813f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f6036800640 1 --2- 172.21.15.46:0/2715776515 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f603815ada0 0x7f603813f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:00.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f6036800640 1 -- 172.21.15.46:0/2715776515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6038144210 msgr2=0x7f6038142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f6036800640 1 --2- 172.21.15.46:0/2715776515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6038144210 0x7f6038142ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f6036800640 1 -- 172.21.15.46:0/2715776515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6030009b30 con 0x7f60380ce950 2024-03-29T08:20:00.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f6036800640 1 --2- 172.21.15.46:0/2715776515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60380ce950 0x7f603813d9c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f602800d800 tx=0x7f602800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f602f400640 1 -- 172.21.15.46:0/2715776515 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60280096a0 con 0x7f60380ce950 2024-03-29T08:20:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f602f400640 1 -- 172.21.15.46:0/2715776515 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6028009800 con 0x7f60380ce950 2024-03-29T08:20:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f603d400640 1 -- 172.21.15.46:0/2715776515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603813cee0 con 0x7f60380ce950 2024-03-29T08:20:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f602f400640 1 -- 172.21.15.46:0/2715776515 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028009b70 con 0x7f60380ce950 2024-03-29T08:20:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f603d400640 1 -- 172.21.15.46:0/2715776515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603813df00 con 0x7f60380ce950 2024-03-29T08:20:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.123+0000 7f603d400640 1 -- 172.21.15.46:0/2715776515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f603813d810 con 0x7f60380ce950 2024-03-29T08:20:00.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.127+0000 7f602f400640 1 -- 172.21.15.46:0/2715776515 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6028009cd0 con 0x7f60380ce950 2024-03-29T08:20:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.127+0000 7f602f400640 1 --2- 172.21.15.46:0/2715776515 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6014066690 0x7f6014068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.127+0000 7f602f400640 1 -- 172.21.15.46:0/2715776515 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6028084640 con 0x7f60380ce950 2024-03-29T08:20:00.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.127+0000 7f602de00640 1 --2- 172.21.15.46:0/2715776515 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6014066690 0x7f6014068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:00.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.127+0000 7f602de00640 1 --2- 172.21.15.46:0/2715776515 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6014066690 0x7f6014068b50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6038140040 tx=0x7f6020008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:00.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.131+0000 7f602f400640 1 -- 172.21.15.46:0/2715776515 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6028051c10 con 0x7f60380ce950 2024-03-29T08:20:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.351+0000 7f603d400640 1 -- 172.21.15.46:0/2715776515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f60380ce250 con 0x7f60380ce950 2024-03-29T08:20:00.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.955+0000 7f602f400640 1 -- 172.21.15.46:0/2715776515 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v116) v1 ==== 152+0+0 (secure 0 0 0) 0x7f6028055230 con 0x7f60380ce950 2024-03-29T08:20:00.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.955+0000 7f603d400640 1 -- 172.21.15.46:0/2715776515 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6014066690 msgr2=0x7f6014068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.955+0000 7f603d400640 1 --2- 172.21.15.46:0/2715776515 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6014066690 0x7f6014068b50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f6038140040 tx=0x7f6020008040 comp rx=0 tx=0).stop 2024-03-29T08:20:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.955+0000 7f603d400640 1 -- 172.21.15.46:0/2715776515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60380ce950 msgr2=0x7f603813d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.955+0000 7f603d400640 1 --2- 172.21.15.46:0/2715776515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60380ce950 0x7f603813d9c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f602800d800 tx=0x7f602800dcc0 comp rx=0 tx=0).stop 2024-03-29T08:20:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.955+0000 7f603d400640 1 -- 172.21.15.46:0/2715776515 shutdown_connections 2024-03-29T08:20:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.955+0000 7f603d400640 1 --2- 172.21.15.46:0/2715776515 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6014066690 0x7f6014068b50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.955+0000 7f603d400640 1 --2- 172.21.15.46:0/2715776515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6038144210 0x7f6038142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.955+0000 7f603d400640 1 --2- 172.21.15.46:0/2715776515 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f603815ada0 0x7f603813f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.955+0000 7f603d400640 1 --2- 172.21.15.46:0/2715776515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60380ce950 0x7f603813d9c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.955+0000 7f603d400640 1 -- 172.21.15.46:0/2715776515 >> 172.21.15.46:0/2715776515 conn(0x7f6038066110 msgr2=0x7f603806c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.955+0000 7f603d400640 1 -- 172.21.15.46:0/2715776515 shutdown_connections 2024-03-29T08:20:00.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:00.955+0000 7f603d400640 1 -- 172.21.15.46:0/2715776515 wait complete. 2024-03-29T08:20:00.978 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-29T08:20:01.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.119+0000 7f27f7200640 1 Processor -- start 2024-03-29T08:20:01.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.119+0000 7f27f7200640 1 -- start start 2024-03-29T08:20:01.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.123+0000 7f27f7200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27f80ccf60 0x7f27f80cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:01.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.123+0000 7f27f7200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f806ff00 con 0x7f27f80cb380 2024-03-29T08:20:01.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.123+0000 7f27f7200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f80cf590 con 0x7f27f815a9f0 2024-03-29T08:20:01.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.123+0000 7f27f7200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f80cd910 con 0x7f27f80ccf60 2024-03-29T08:20:01.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.123+0000 7f27f6800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f27f80cb380 0x7f27f80cb7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53538/0 (socket says 172.21.15.46:53538) 2024-03-29T08:20:01.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.123+0000 7f27f6800640 1 -- 172.21.15.46:0/1539313688 learned_addr learned my addr 172.21.15.46:0/1539313688 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:01.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.123+0000 7f27efe00640 1 --2- 172.21.15.46:0/1539313688 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27f80ccf60 0x7f27f80cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:01.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.123+0000 7f27f5400640 1 -- 172.21.15.46:0/1539313688 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1179920668 0 0) 0x7f27f806ff00 con 0x7f27f80cb380 2024-03-29T08:20:01.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.123+0000 7f27f5400640 1 -- 172.21.15.46:0/1539313688 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27dc003530 con 0x7f27f80cb380 2024-03-29T08:20:01.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.123+0000 7f27f5400640 1 -- 172.21.15.46:0/1539313688 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4006580222 0 0) 0x7f27f80cf590 con 0x7f27f815a9f0 2024-03-29T08:20:01.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.123+0000 7f27f5400640 1 -- 172.21.15.46:0/1539313688 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27f806ff00 con 0x7f27f815a9f0 2024-03-29T08:20:01.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.123+0000 7f27efe00640 1 -- 172.21.15.46:0/1539313688 >> v1:172.21.15.73:6789/0 conn(0x7f27f815a9f0 legacy=0x7f27f80ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:01.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.123+0000 7f27efe00640 1 -- 172.21.15.46:0/1539313688 >> v1:172.21.15.46:6789/0 conn(0x7f27f80cb380 legacy=0x7f27f80cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:01.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.123+0000 7f27efe00640 1 -- 172.21.15.46:0/1539313688 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f806e130 con 0x7f27f80ccf60 2024-03-29T08:20:01.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.127+0000 7f27efe00640 1 --2- 172.21.15.46:0/1539313688 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27f80ccf60 0x7f27f80cd3d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f27e00097f0 tx=0x7f27e001ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e42bdc04665705f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:01.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.127+0000 7f27f5400640 1 -- 172.21.15.46:0/1539313688 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00a89d0 con 0x7f27f80ccf60 2024-03-29T08:20:01.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.127+0000 7f27f5400640 1 -- 172.21.15.46:0/1539313688 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e00a8b30 con 0x7f27f80ccf60 2024-03-29T08:20:01.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.127+0000 7f27f5400640 1 -- 172.21.15.46:0/1539313688 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00b0440 con 0x7f27f80ccf60 2024-03-29T08:20:01.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.127+0000 7f27f7200640 1 -- 172.21.15.46:0/1539313688 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27f80ccf60 msgr2=0x7f27f80cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:01.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.127+0000 7f27f7200640 1 --2- 172.21.15.46:0/1539313688 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27f80ccf60 0x7f27f80cd3d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f27e00097f0 tx=0x7f27e001ecd0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.127+0000 7f27f7200640 1 -- 172.21.15.46:0/1539313688 shutdown_connections 2024-03-29T08:20:01.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.127+0000 7f27f7200640 1 --2- 172.21.15.46:0/1539313688 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27f80ccf60 0x7f27f80cd3d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.127+0000 7f27f7200640 1 -- 172.21.15.46:0/1539313688 >> 172.21.15.46:0/1539313688 conn(0x7f27f80d0550 msgr2=0x7f27f80d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:01.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.127+0000 7f27f7200640 1 -- 172.21.15.46:0/1539313688 shutdown_connections 2024-03-29T08:20:01.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.127+0000 7f27f7200640 1 -- 172.21.15.46:0/1539313688 wait complete. 2024-03-29T08:20:01.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.127+0000 7f27f7200640 1 Processor -- start 2024-03-29T08:20:01.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.127+0000 7f27f7200640 1 -- start start 2024-03-29T08:20:01.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.127+0000 7f27f7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f80cb380 0x7f27f80ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:01.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.127+0000 7f27f7200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27f80ccf60 0x7f27f80c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:01.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27f815a9f0 0x7f27f80b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:01.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f81efd30 con 0x7f27f80cb380 2024-03-29T08:20:01.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f7200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f80c97b0 con 0x7f27f815a9f0 2024-03-29T08:20:01.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f7200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f80c9a90 con 0x7f27f80ccf60 2024-03-29T08:20:01.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f80cb380 0x7f27f80ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:01.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f80cb380 0x7f27f80ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54808/0 (socket says 172.21.15.46:54808) 2024-03-29T08:20:01.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f5e00640 1 -- 172.21.15.46:0/4013230992 learned_addr learned my addr 172.21.15.46:0/4013230992 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:01.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f5e00640 1 -- 172.21.15.46:0/4013230992 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27f80ccf60 msgr2=0x7f27f80c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:01.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27efe00640 1 --2- 172.21.15.46:0/4013230992 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27f80ccf60 0x7f27f80c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:01.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f5e00640 1 --2- 172.21.15.46:0/4013230992 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27f80ccf60 0x7f27f80c9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f6800640 1 --2- 172.21.15.46:0/4013230992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27f815a9f0 0x7f27f80b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:01.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f5e00640 1 -- 172.21.15.46:0/4013230992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27f815a9f0 msgr2=0x7f27f80b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:01.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f5e00640 1 --2- 172.21.15.46:0/4013230992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27f815a9f0 0x7f27f80b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f5e00640 1 -- 172.21.15.46:0/4013230992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e0009420 con 0x7f27f80cb380 2024-03-29T08:20:01.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27efe00640 1 --2- 172.21.15.46:0/4013230992 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27f80ccf60 0x7f27f80c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:01.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f6800640 1 --2- 172.21.15.46:0/4013230992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27f815a9f0 0x7f27f80b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:01.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f5e00640 1 --2- 172.21.15.46:0/4013230992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f80cb380 0x7f27f80ca850 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f27e800c430 tx=0x7f27e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:01.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27ee000640 1 -- 172.21.15.46:0/4013230992 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e80052d0 con 0x7f27f80cb380 2024-03-29T08:20:01.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27ee000640 1 -- 172.21.15.46:0/4013230992 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e8005450 con 0x7f27f80cb380 2024-03-29T08:20:01.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27ee000640 1 -- 172.21.15.46:0/4013230992 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e8003840 con 0x7f27f80cb380 2024-03-29T08:20:01.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f7200640 1 -- 172.21.15.46:0/4013230992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f80c9d70 con 0x7f27f80cb380 2024-03-29T08:20:01.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f7200640 1 -- 172.21.15.46:0/4013230992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f80ca330 con 0x7f27f80cb380 2024-03-29T08:20:01.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27f7200640 1 -- 172.21.15.46:0/4013230992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f80ca520 con 0x7f27f80cb380 2024-03-29T08:20:01.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27ee000640 1 -- 172.21.15.46:0/4013230992 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f27e80039a0 con 0x7f27f80cb380 2024-03-29T08:20:01.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.131+0000 7f27ee000640 1 --2- 172.21.15.46:0/4013230992 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27d4066690 0x7f27d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:01.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.135+0000 7f27ee000640 1 -- 172.21.15.46:0/4013230992 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f27e8083470 con 0x7f27f80cb380 2024-03-29T08:20:01.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.135+0000 7f27efe00640 1 --2- 172.21.15.46:0/4013230992 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27d4066690 0x7f27d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:01.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.135+0000 7f27efe00640 1 --2- 172.21.15.46:0/4013230992 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27d4066690 0x7f27d4068b50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f27e00b2000 tx=0x7f27e00aa070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:01.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.139+0000 7f27ee000640 1 -- 172.21.15.46:0/4013230992 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f27e8050b50 con 0x7f27f80cb380 2024-03-29T08:20:01.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.299+0000 7f27f7200640 1 -- 172.21.15.46:0/4013230992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f27f804e040 con 0x7f27f80cb380 2024-03-29T08:20:01.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.303+0000 7f27ee000640 1 -- 172.21.15.46:0/4013230992 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f27e8054170 con 0x7f27f80cb380 2024-03-29T08:20:01.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.303+0000 7f27f7200640 1 -- 172.21.15.46:0/4013230992 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27d4066690 msgr2=0x7f27d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:01.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.303+0000 7f27f7200640 1 --2- 172.21.15.46:0/4013230992 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27d4066690 0x7f27d4068b50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f27e00b2000 tx=0x7f27e00aa070 comp rx=0 tx=0).stop 2024-03-29T08:20:01.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.303+0000 7f27f7200640 1 -- 172.21.15.46:0/4013230992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f80cb380 msgr2=0x7f27f80ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:01.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.303+0000 7f27f7200640 1 --2- 172.21.15.46:0/4013230992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f80cb380 0x7f27f80ca850 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f27e800c430 tx=0x7f27e800c8f0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.303+0000 7f27f7200640 1 -- 172.21.15.46:0/4013230992 shutdown_connections 2024-03-29T08:20:01.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.303+0000 7f27f7200640 1 --2- 172.21.15.46:0/4013230992 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27d4066690 0x7f27d4068b50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.303+0000 7f27f7200640 1 --2- 172.21.15.46:0/4013230992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27f815a9f0 0x7f27f80b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.303+0000 7f27f7200640 1 --2- 172.21.15.46:0/4013230992 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27f80ccf60 0x7f27f80c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.303+0000 7f27f7200640 1 --2- 172.21.15.46:0/4013230992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27f80cb380 0x7f27f80ca850 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.303+0000 7f27f7200640 1 -- 172.21.15.46:0/4013230992 >> 172.21.15.46:0/4013230992 conn(0x7f27f80d0550 msgr2=0x7f27f80d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:01.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.303+0000 7f27f7200640 1 -- 172.21.15.46:0/4013230992 shutdown_connections 2024-03-29T08:20:01.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.303+0000 7f27f7200640 1 -- 172.21.15.46:0/4013230992 wait complete. 2024-03-29T08:20:01.325 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T08:20:01.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:20:01.402 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:20:01.471 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T08:20:01.471 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:20:01.471 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:20:01.480 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:20:01.481 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:20:01.481 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:20:01.489 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:20:01.489 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:20:01.490 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:20:01.531 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:20:01.531 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:20:01.531 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:20:01.544 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:20:01.544 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-29T08:20:01.544 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:20:01.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.711+0000 7f78e0200640 1 Processor -- start 2024-03-29T08:20:01.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.711+0000 7f78e0200640 1 -- start start 2024-03-29T08:20:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78e0200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f78d815ab70 0x7f78d8098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78e0200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d8099390 con 0x7f78d815a790 2024-03-29T08:20:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78e0200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d80ce660 con 0x7f78d8098f80 2024-03-29T08:20:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78e0200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d80cf700 con 0x7f78d815ab70 2024-03-29T08:20:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78dd800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f78d815a790 0x7f78d80cbef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53542/0 (socket says 172.21.15.46:53542) 2024-03-29T08:20:01.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78dd800640 1 -- 172.21.15.46:0/3419404103 learned_addr learned my addr 172.21.15.46:0/3419404103 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78dce00640 1 --2- 172.21.15.46:0/3419404103 >> v2:172.21.15.73:3300/0 conn(0x7f78d815ab70 0x7f78d8098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78d7e00640 1 -- 172.21.15.46:0/3419404103 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273126270 0 0) 0x7f78d8099390 con 0x7f78d815a790 2024-03-29T08:20:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78d7e00640 1 -- 172.21.15.46:0/3419404103 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78c0003530 con 0x7f78d815a790 2024-03-29T08:20:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78d7e00640 1 -- 172.21.15.46:0/3419404103 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 464150193 0 0) 0x7f78d80ce660 con 0x7f78d8098f80 2024-03-29T08:20:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78d7e00640 1 -- 172.21.15.46:0/3419404103 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78d8099390 con 0x7f78d8098f80 2024-03-29T08:20:01.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78d7e00640 1 -- 172.21.15.46:0/3419404103 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1976314829 0 0) 0x7f78c0003530 con 0x7f78d815a790 2024-03-29T08:20:01.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78d7e00640 1 -- 172.21.15.46:0/3419404103 >> v2:172.21.15.73:3300/0 conn(0x7f78d815ab70 msgr2=0x7f78d8098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:01.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78d7e00640 1 --2- 172.21.15.46:0/3419404103 >> v2:172.21.15.73:3300/0 conn(0x7f78d815ab70 0x7f78d8098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78d7e00640 1 -- 172.21.15.46:0/3419404103 >> v1:172.21.15.73:6789/0 conn(0x7f78d8098f80 legacy=0x7f78d80b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:01.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78d7e00640 1 -- 172.21.15.46:0/3419404103 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d80ce940 con 0x7f78d815a790 2024-03-29T08:20:01.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78d7e00640 1 -- 172.21.15.46:0/3419404103 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f78cc001600 con 0x7f78d815a790 2024-03-29T08:20:01.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78d7e00640 1 -- 172.21.15.46:0/3419404103 >> v1:172.21.15.46:6789/0 conn(0x7f78d815a790 legacy=0x7f78d80cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:01.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78d7e00640 1 --2- 172.21.15.46:0/3419404103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78c0003b60 0x7f78c0003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:01.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78d7e00640 1 --2- 172.21.15.46:0/3419404103 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78c00047d0 0x7f78c0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:01.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78d7e00640 1 --2- 172.21.15.46:0/3419404103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78c0005190 0x7f78c000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:01.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78d7e00640 1 -- 172.21.15.46:0/3419404103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d80cf700 con 0x7f78c0003b60 2024-03-29T08:20:01.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78d7e00640 1 -- 172.21.15.46:0/3419404103 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c000c060 con 0x7f78c0005190 2024-03-29T08:20:01.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78d7e00640 1 -- 172.21.15.46:0/3419404103 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c000c220 con 0x7f78c00047d0 2024-03-29T08:20:01.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78dd800640 1 --2- 172.21.15.46:0/3419404103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78c0003b60 0x7f78c0003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:01.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78dce00640 1 --2- 172.21.15.46:0/3419404103 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78c00047d0 0x7f78c0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:01.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78de200640 1 --2- 172.21.15.46:0/3419404103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78c0005190 0x7f78c000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:01.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78dd800640 1 -- 172.21.15.46:0/3419404103 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78c00047d0 msgr2=0x7f78c0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:01.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78dd800640 1 --2- 172.21.15.46:0/3419404103 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78c00047d0 0x7f78c0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78dd800640 1 -- 172.21.15.46:0/3419404103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78c0005190 msgr2=0x7f78c000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:01.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78dd800640 1 --2- 172.21.15.46:0/3419404103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78c0005190 0x7f78c000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78dd800640 1 -- 172.21.15.46:0/3419404103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c0003530 con 0x7f78c0003b60 2024-03-29T08:20:01.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.715+0000 7f78de200640 1 -- 172.21.15.46:0/3419404103 reap_dead start 2024-03-29T08:20:01.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.719+0000 7f78dd800640 1 --2- 172.21.15.46:0/3419404103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78c0003b60 0x7f78c0003f60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f78cc007f50 tx=0x7f78cc01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997695c48aed4142 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:01.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.719+0000 7f78d7e00640 1 -- 172.21.15.46:0/3419404103 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc005ba0 con 0x7f78c0003b60 2024-03-29T08:20:01.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.719+0000 7f78d7e00640 1 -- 172.21.15.46:0/3419404103 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78cc005d20 con 0x7f78c0003b60 2024-03-29T08:20:01.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.719+0000 7f78d7e00640 1 -- 172.21.15.46:0/3419404103 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc0072b0 con 0x7f78c0003b60 2024-03-29T08:20:01.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.719+0000 7f78e0200640 1 -- 172.21.15.46:0/3419404103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78c0003b60 msgr2=0x7f78c0003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:01.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.719+0000 7f78e0200640 1 --2- 172.21.15.46:0/3419404103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78c0003b60 0x7f78c0003f60 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f78cc007f50 tx=0x7f78cc01ca90 comp rx=0 tx=0).stop 2024-03-29T08:20:01.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.719+0000 7f78e0200640 1 -- 172.21.15.46:0/3419404103 shutdown_connections 2024-03-29T08:20:01.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.719+0000 7f78e0200640 1 --2- 172.21.15.46:0/3419404103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78c0003b60 0x7f78c0003f60 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.719+0000 7f78e0200640 1 -- 172.21.15.46:0/3419404103 >> 172.21.15.46:0/3419404103 conn(0x7f78d80b21a0 msgr2=0x7f78d80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:01.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.719+0000 7f78e0200640 1 -- 172.21.15.46:0/3419404103 shutdown_connections 2024-03-29T08:20:01.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.719+0000 7f78e0200640 1 -- 172.21.15.46:0/3419404103 wait complete. 2024-03-29T08:20:01.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.719+0000 7f78e0200640 1 Processor -- start 2024-03-29T08:20:01.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.719+0000 7f78e0200640 1 -- start start 2024-03-29T08:20:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78e0200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78c0003b60 0x7f78d8157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78e0200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d815dad0 0x7f78d815ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78e0200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78d81f8820 0x7f78d81f8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78e0200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d81f91e0 con 0x7f78d815dad0 2024-03-29T08:20:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78e0200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d80c7780 con 0x7f78c0003b60 2024-03-29T08:20:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78e0200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d80c7a60 con 0x7f78d81f8820 2024-03-29T08:20:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78dce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d815dad0 0x7f78d815ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78dce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d815dad0 0x7f78d815ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54832/0 (socket says 172.21.15.46:54832) 2024-03-29T08:20:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78dce00640 1 -- 172.21.15.46:0/2018446289 learned_addr learned my addr 172.21.15.46:0/2018446289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78dd800640 1 --2- 172.21.15.46:0/2018446289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78c0003b60 0x7f78d8157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78de200640 1 --2- 172.21.15.46:0/2018446289 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78d81f8820 0x7f78d81f8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78dce00640 1 -- 172.21.15.46:0/2018446289 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78d81f8820 msgr2=0x7f78d81f8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78dce00640 1 --2- 172.21.15.46:0/2018446289 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78d81f8820 0x7f78d81f8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78dce00640 1 -- 172.21.15.46:0/2018446289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78c0003b60 msgr2=0x7f78d8157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78dce00640 1 --2- 172.21.15.46:0/2018446289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78c0003b60 0x7f78d8157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78dce00640 1 -- 172.21.15.46:0/2018446289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d80c7d40 con 0x7f78d815dad0 2024-03-29T08:20:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78de200640 1 --2- 172.21.15.46:0/2018446289 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78d81f8820 0x7f78d81f8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78dd800640 1 --2- 172.21.15.46:0/2018446289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78c0003b60 0x7f78d8157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78dce00640 1 --2- 172.21.15.46:0/2018446289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d815dad0 0x7f78d815ce30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f78c4011900 tx=0x7f78c4011dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78d5600640 1 -- 172.21.15.46:0/2018446289 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c40095b0 con 0x7f78d815dad0 2024-03-29T08:20:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78d5600640 1 -- 172.21.15.46:0/2018446289 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c4004240 con 0x7f78d815dad0 2024-03-29T08:20:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78d5600640 1 -- 172.21.15.46:0/2018446289 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c400f8e0 con 0x7f78d815dad0 2024-03-29T08:20:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78e0200640 1 -- 172.21.15.46:0/2018446289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d80c8020 con 0x7f78d815dad0 2024-03-29T08:20:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78e0200640 1 -- 172.21.15.46:0/2018446289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d80c4760 con 0x7f78d815dad0 2024-03-29T08:20:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.723+0000 7f78e0200640 1 -- 172.21.15.46:0/2018446289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d8066c90 con 0x7f78d815dad0 2024-03-29T08:20:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.727+0000 7f78d5600640 1 -- 172.21.15.46:0/2018446289 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f78c400fa40 con 0x7f78d815dad0 2024-03-29T08:20:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.727+0000 7f78d5600640 1 --2- 172.21.15.46:0/2018446289 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78b8066690 0x7f78b8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.727+0000 7f78d5600640 1 -- 172.21.15.46:0/2018446289 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f78c40845e0 con 0x7f78d815dad0 2024-03-29T08:20:01.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.727+0000 7f78dd800640 1 --2- 172.21.15.46:0/2018446289 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78b8066690 0x7f78b8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:01.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.727+0000 7f78dd800640 1 --2- 172.21.15.46:0/2018446289 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78b8066690 0x7f78b8068b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f78cc007f20 tx=0x7f78cc0b1040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:01.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.731+0000 7f78d5600640 1 -- 172.21.15.46:0/2018446289 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f78c4051bb0 con 0x7f78d815dad0 2024-03-29T08:20:01.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.923+0000 7f78e0200640 1 -- 172.21.15.46:0/2018446289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78d80b2ec0 con 0x7f78d815dad0 2024-03-29T08:20:01.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.923+0000 7f78d5600640 1 -- 172.21.15.46:0/2018446289 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f78c4023de0 con 0x7f78d815dad0 2024-03-29T08:20:01.927 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:01.927 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4964,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/4274575126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":4274575126},{"type":"v1","addr":"172.21.15.46:6834","nonce":4274575126}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5142,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3739899623","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3739899623},{"type":"v1","addr":"172.21.15.73:6837","nonce":3739899623}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1905427475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1905427475},{"type":"v1","addr":"172.21.15.46:6835","nonce":1905427475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3549352786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3549352786},{"type":"v1","addr":"172.21.15.73:6836","nonce":3549352786}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2024-03-29T08:20:01.928 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 18 2024-03-29T08:20:01.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.927+0000 7f78e0200640 1 -- 172.21.15.46:0/2018446289 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78b8066690 msgr2=0x7f78b8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:01.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.927+0000 7f78e0200640 1 --2- 172.21.15.46:0/2018446289 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78b8066690 0x7f78b8068b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f78cc007f20 tx=0x7f78cc0b1040 comp rx=0 tx=0).stop 2024-03-29T08:20:01.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.927+0000 7f78e0200640 1 -- 172.21.15.46:0/2018446289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d815dad0 msgr2=0x7f78d815ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:01.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.927+0000 7f78e0200640 1 --2- 172.21.15.46:0/2018446289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d815dad0 0x7f78d815ce30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f78c4011900 tx=0x7f78c4011dc0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.927+0000 7f78e0200640 1 -- 172.21.15.46:0/2018446289 shutdown_connections 2024-03-29T08:20:01.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.927+0000 7f78e0200640 1 --2- 172.21.15.46:0/2018446289 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78b8066690 0x7f78b8068b50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.927+0000 7f78e0200640 1 --2- 172.21.15.46:0/2018446289 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78d81f8820 0x7f78d81f8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.927+0000 7f78e0200640 1 --2- 172.21.15.46:0/2018446289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d815dad0 0x7f78d815ce30 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.927+0000 7f78e0200640 1 --2- 172.21.15.46:0/2018446289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78c0003b60 0x7f78d8157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:01.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.927+0000 7f78e0200640 1 -- 172.21.15.46:0/2018446289 >> 172.21.15.46:0/2018446289 conn(0x7f78d80b21a0 msgr2=0x7f78d81590a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:01.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.927+0000 7f78e0200640 1 -- 172.21.15.46:0/2018446289 shutdown_connections 2024-03-29T08:20:01.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:01.927+0000 7f78e0200640 1 -- 172.21.15.46:0/2018446289 wait complete. 2024-03-29T08:20:01.948 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T08:20:01.948 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-29T08:20:01.949 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T08:20:01.949 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:20:01.949 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T08:20:01.949 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-29T08:20:01.949 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T08:20:01.949 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:20:01.949 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T08:20:01.949 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-29T08:20:01.950 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T08:20:01.950 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:20:01.950 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T08:20:01.950 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-29T08:20:01.950 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T08:20:01.950 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:20:01.964 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T08:20:01.964 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T08:20:01.966 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T08:20:01.967 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T08:20:01.969 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T08:20:01.970 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T08:20:02.014 INFO:tasks.ceph.mds.b:Started 2024-03-29T08:20:02.014 DEBUG:teuthology.parallel:result is None 2024-03-29T08:20:02.018 INFO:tasks.ceph.mds.c:Started 2024-03-29T08:20:02.018 INFO:tasks.ceph.mds.d:Started 2024-03-29T08:20:02.019 DEBUG:teuthology.parallel:result is None 2024-03-29T08:20:02.019 DEBUG:teuthology.parallel:result is None 2024-03-29T08:20:02.098 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-03-29T08:20:02.106 INFO:tasks.ceph.mds.b.smithi073.stdout:starting mds.b at 2024-03-29T08:20:02.109 INFO:tasks.ceph.mds.d.smithi073.stdout:starting mds.d at 2024-03-29T08:20:06.834 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-03-29T08:20:07.963 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T08:20:07.964 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T08:20:07.968 INFO:tasks.ceph.mds.a:Started 2024-03-29T08:20:07.968 DEBUG:teuthology.parallel:result is None 2024-03-29T08:20:07.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-29T08:20:08.062 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-03-29T08:20:08.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.123+0000 7f3279e00640 1 Processor -- start 2024-03-29T08:20:08.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.123+0000 7f3279e00640 1 -- start start 2024-03-29T08:20:08.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.123+0000 7f3279e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f32740af530 0x7f32740af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:08.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.123+0000 7f3279e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32740702a0 0x7f3274070730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:08.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.123+0000 7f3279e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f327406c390 con 0x7f3274158300 2024-03-29T08:20:08.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.123+0000 7f3279e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3274070c70 con 0x7f32740702a0 2024-03-29T08:20:08.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.123+0000 7f3279e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32740afff0 con 0x7f32740af530 2024-03-29T08:20:08.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.123+0000 7f3273400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32740702a0 0x7f3274070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:08.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.123+0000 7f3273400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32740702a0 0x7f3274070730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37698/0 (socket says 172.21.15.46:37698) 2024-03-29T08:20:08.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.123+0000 7f3273400640 1 -- 172.21.15.46:0/3098524670 learned_addr learned my addr 172.21.15.46:0/3098524670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:08.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.123+0000 7f3272000640 1 --2- 172.21.15.46:0/3098524670 >> v2:172.21.15.73:3300/0 conn(0x7f32740af530 0x7f32740af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:08.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.127+0000 7f3273400640 1 -- 172.21.15.46:0/3098524670 >> v2:172.21.15.73:3300/0 conn(0x7f32740af530 msgr2=0x7f32740af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:08.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.127+0000 7f3273400640 1 --2- 172.21.15.46:0/3098524670 >> v2:172.21.15.73:3300/0 conn(0x7f32740af530 0x7f32740af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.127+0000 7f3273400640 1 -- 172.21.15.46:0/3098524670 >> v1:172.21.15.73:6789/0 conn(0x7f3274158300 legacy=0x7f327406d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:08.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.127+0000 7f3273400640 1 -- 172.21.15.46:0/3098524670 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327406c5e0 con 0x7f32740702a0 2024-03-29T08:20:08.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.127+0000 7f3273400640 1 --2- 172.21.15.46:0/3098524670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32740702a0 0x7f3274070730 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f32740665f0 tx=0x7f326c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02abe252baa972d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:08.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.127+0000 7f3271600640 1 -- 172.21.15.46:0/3098524670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c0a5760 con 0x7f32740702a0 2024-03-29T08:20:08.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.127+0000 7f3271600640 1 -- 172.21.15.46:0/3098524670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326c0a58c0 con 0x7f32740702a0 2024-03-29T08:20:08.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.127+0000 7f3271600640 1 -- 172.21.15.46:0/3098524670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c0a5c50 con 0x7f32740702a0 2024-03-29T08:20:08.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.127+0000 7f3279e00640 1 -- 172.21.15.46:0/3098524670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32740702a0 msgr2=0x7f3274070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:08.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.127+0000 7f3279e00640 1 --2- 172.21.15.46:0/3098524670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32740702a0 0x7f3274070730 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f32740665f0 tx=0x7f326c01c8a0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.127+0000 7f3279e00640 1 -- 172.21.15.46:0/3098524670 shutdown_connections 2024-03-29T08:20:08.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.127+0000 7f3279e00640 1 --2- 172.21.15.46:0/3098524670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32740702a0 0x7f3274070730 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.127+0000 7f3279e00640 1 --2- 172.21.15.46:0/3098524670 >> v2:172.21.15.73:3300/0 conn(0x7f32740af530 0x7f32740af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.127+0000 7f3279e00640 1 -- 172.21.15.46:0/3098524670 >> 172.21.15.46:0/3098524670 conn(0x7f3274066080 msgr2=0x7f3274069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:08.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.127+0000 7f3279e00640 1 -- 172.21.15.46:0/3098524670 shutdown_connections 2024-03-29T08:20:08.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.127+0000 7f3279e00640 1 -- 172.21.15.46:0/3098524670 wait complete. 2024-03-29T08:20:08.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3279e00640 1 Processor -- start 2024-03-29T08:20:08.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3279e00640 1 -- start start 2024-03-29T08:20:08.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3279e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32740702a0 0x7f3274147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:08.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3279e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f32740af530 0x7f3274155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:08.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3279e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3274158300 0x7f32741556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:08.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3279e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32741505f0 con 0x7f3274158300 2024-03-29T08:20:08.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3279e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32740b26a0 con 0x7f32740702a0 2024-03-29T08:20:08.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3279e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32740b2980 con 0x7f32740af530 2024-03-29T08:20:08.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3273400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3274158300 0x7f32741556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:08.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3273400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3274158300 0x7f32741556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37706/0 (socket says 172.21.15.46:37706) 2024-03-29T08:20:08.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3273400640 1 -- 172.21.15.46:0/1202089256 learned_addr learned my addr 172.21.15.46:0/1202089256 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:08.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3272000640 1 --2- 172.21.15.46:0/1202089256 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f32740af530 0x7f3274155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:08.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3272a00640 1 --2- 172.21.15.46:0/1202089256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32740702a0 0x7f3274147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:08.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3273400640 1 -- 172.21.15.46:0/1202089256 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f32740af530 msgr2=0x7f3274155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3273400640 1 --2- 172.21.15.46:0/1202089256 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f32740af530 0x7f3274155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3273400640 1 -- 172.21.15.46:0/1202089256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32740702a0 msgr2=0x7f3274147a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3273400640 1 --2- 172.21.15.46:0/1202089256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32740702a0 0x7f3274147a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3273400640 1 -- 172.21.15.46:0/1202089256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c009b30 con 0x7f3274158300 2024-03-29T08:20:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f3273400640 1 --2- 172.21.15.46:0/1202089256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3274158300 0x7f32741556a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f326c0af040 tx=0x7f326c019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.131+0000 7f326b400640 1 -- 172.21.15.46:0/1202089256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c0b8670 con 0x7f3274158300 2024-03-29T08:20:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.135+0000 7f326b400640 1 -- 172.21.15.46:0/1202089256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326c0b87d0 con 0x7f3274158300 2024-03-29T08:20:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.135+0000 7f326b400640 1 -- 172.21.15.46:0/1202089256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c0b8b60 con 0x7f3274158300 2024-03-29T08:20:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.135+0000 7f3279e00640 1 -- 172.21.15.46:0/1202089256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32740b2c60 con 0x7f3274158300 2024-03-29T08:20:08.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.135+0000 7f3279e00640 1 -- 172.21.15.46:0/1202089256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32740b3220 con 0x7f3274158300 2024-03-29T08:20:08.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.135+0000 7f3279e00640 1 -- 172.21.15.46:0/1202089256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3274071a90 con 0x7f3274158300 2024-03-29T08:20:08.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.135+0000 7f326b400640 1 -- 172.21.15.46:0/1202089256 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f326c0b8cc0 con 0x7f3274158300 2024-03-29T08:20:08.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.135+0000 7f326b400640 1 --2- 172.21.15.46:0/1202089256 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f32540666c0 0x7f3254068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:08.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.135+0000 7f326b400640 1 -- 172.21.15.46:0/1202089256 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f326c11f300 con 0x7f3274158300 2024-03-29T08:20:08.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.135+0000 7f3272a00640 1 --2- 172.21.15.46:0/1202089256 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f32540666c0 0x7f3254068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:08.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.135+0000 7f3272a00640 1 --2- 172.21.15.46:0/1202089256 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f32540666c0 0x7f3254068b80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3264000f70 tx=0x7f3264009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:08.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.139+0000 7f326b400640 1 -- 172.21.15.46:0/1202089256 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f326c0eca30 con 0x7f3274158300 2024-03-29T08:20:08.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:08.311+0000 7f3279e00640 1 -- 172.21.15.46:0/1202089256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f32740cd500 con 0x7f3274158300 2024-03-29T08:20:08.456 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:20:08.459 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:20:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.075+0000 7f326b400640 1 -- 172.21.15.46:0/1202089256 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v45) v1 ==== 97+0+0 (secure 0 0 0) 0x7f326c009510 con 0x7f3274158300 2024-03-29T08:20:09.080 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-03-29T08:20:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.079+0000 7f3279e00640 1 -- 172.21.15.46:0/1202089256 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f32540666c0 msgr2=0x7f3254068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.079+0000 7f3279e00640 1 --2- 172.21.15.46:0/1202089256 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f32540666c0 0x7f3254068b80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3264000f70 tx=0x7f3264009040 comp rx=0 tx=0).stop 2024-03-29T08:20:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.079+0000 7f3279e00640 1 -- 172.21.15.46:0/1202089256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3274158300 msgr2=0x7f32741556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.079+0000 7f3279e00640 1 --2- 172.21.15.46:0/1202089256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3274158300 0x7f32741556a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f326c0af040 tx=0x7f326c019640 comp rx=0 tx=0).stop 2024-03-29T08:20:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.079+0000 7f3279e00640 1 -- 172.21.15.46:0/1202089256 shutdown_connections 2024-03-29T08:20:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.079+0000 7f3279e00640 1 --2- 172.21.15.46:0/1202089256 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f32540666c0 0x7f3254068b80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.079+0000 7f3279e00640 1 --2- 172.21.15.46:0/1202089256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3274158300 0x7f32741556a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.079+0000 7f3279e00640 1 --2- 172.21.15.46:0/1202089256 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f32740af530 0x7f3274155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.079+0000 7f3279e00640 1 --2- 172.21.15.46:0/1202089256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32740702a0 0x7f3274147a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.079+0000 7f3279e00640 1 -- 172.21.15.46:0/1202089256 >> 172.21.15.46:0/1202089256 conn(0x7f3274066080 msgr2=0x7f32740cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.079+0000 7f3279e00640 1 -- 172.21.15.46:0/1202089256 shutdown_connections 2024-03-29T08:20:09.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.079+0000 7f3279e00640 1 -- 172.21.15.46:0/1202089256 wait complete. 2024-03-29T08:20:09.106 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-29T08:20:09.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.247+0000 7f5242200640 1 Processor -- start 2024-03-29T08:20:09.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.247+0000 7f5242200640 1 -- start start 2024-03-29T08:20:09.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.251+0000 7f5242200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f523c0ce8a0 0x7f523c0caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.251+0000 7f5242200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f523c0cb780 0x7f523c0aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.251+0000 7f5242200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f523c0cbc10 con 0x7f523c0ae810 2024-03-29T08:20:09.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.251+0000 7f5242200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f523c0c9bd0 con 0x7f523c0cb780 2024-03-29T08:20:09.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.251+0000 7f5242200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523c0c9eb0 con 0x7f523c0ce8a0 2024-03-29T08:20:09.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.251+0000 7f5233e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f523c0ae810 0x7f523c0cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33534/0 (socket says 172.21.15.46:33534) 2024-03-29T08:20:09.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.251+0000 7f5233e00640 1 -- 172.21.15.46:0/1306196696 learned_addr learned my addr 172.21.15.46:0/1306196696 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:09.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.251+0000 7f523be00640 1 --2- 172.21.15.46:0/1306196696 >> v2:172.21.15.73:3300/0 conn(0x7f523c0cb780 0x7f523c0aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.251+0000 7f523b400640 1 --2- 172.21.15.46:0/1306196696 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f523c0ce8a0 0x7f523c0caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.251+0000 7f523aa00640 1 -- 172.21.15.46:0/1306196696 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291233076 0 0) 0x7f523c0cbc10 con 0x7f523c0ae810 2024-03-29T08:20:09.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.251+0000 7f523aa00640 1 -- 172.21.15.46:0/1306196696 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5220003530 con 0x7f523c0ae810 2024-03-29T08:20:09.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.251+0000 7f523b400640 1 -- 172.21.15.46:0/1306196696 >> v2:172.21.15.73:3300/0 conn(0x7f523c0cb780 msgr2=0x7f523c0aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:09.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.251+0000 7f523b400640 1 --2- 172.21.15.46:0/1306196696 >> v2:172.21.15.73:3300/0 conn(0x7f523c0cb780 0x7f523c0aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.251+0000 7f523b400640 1 -- 172.21.15.46:0/1306196696 >> v1:172.21.15.46:6789/0 conn(0x7f523c0ae810 legacy=0x7f523c0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.251+0000 7f523b400640 1 -- 172.21.15.46:0/1306196696 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c0ca190 con 0x7f523c0ce8a0 2024-03-29T08:20:09.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.251+0000 7f523be00640 1 --2- 172.21.15.46:0/1306196696 >> v2:172.21.15.73:3300/0 conn(0x7f523c0cb780 0x7f523c0aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:20:09.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.255+0000 7f523b400640 1 --2- 172.21.15.46:0/1306196696 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f523c0ce8a0 0x7f523c0caf50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f522c0097f0 tx=0x7f522c01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b698ab2767b97868 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:09.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.255+0000 7f523aa00640 1 -- 172.21.15.46:0/1306196696 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c0afa20 con 0x7f523c0ce8a0 2024-03-29T08:20:09.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.255+0000 7f523aa00640 1 -- 172.21.15.46:0/1306196696 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522c0ab670 con 0x7f523c0ce8a0 2024-03-29T08:20:09.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.255+0000 7f523aa00640 1 -- 172.21.15.46:0/1306196696 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c0ab9e0 con 0x7f523c0ce8a0 2024-03-29T08:20:09.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.255+0000 7f5242200640 1 -- 172.21.15.46:0/1306196696 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f523c0ce8a0 msgr2=0x7f523c0caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.255+0000 7f5242200640 1 --2- 172.21.15.46:0/1306196696 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f523c0ce8a0 0x7f523c0caf50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f522c0097f0 tx=0x7f522c01c470 comp rx=0 tx=0).stop 2024-03-29T08:20:09.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.255+0000 7f5242200640 1 -- 172.21.15.46:0/1306196696 shutdown_connections 2024-03-29T08:20:09.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.255+0000 7f5242200640 1 --2- 172.21.15.46:0/1306196696 >> v2:172.21.15.73:3300/0 conn(0x7f523c0cb780 0x7f523c0aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.255+0000 7f5242200640 1 --2- 172.21.15.46:0/1306196696 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f523c0ce8a0 0x7f523c0caf50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.255+0000 7f5242200640 1 -- 172.21.15.46:0/1306196696 >> 172.21.15.46:0/1306196696 conn(0x7f523c098a40 msgr2=0x7f523c0cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:09.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.255+0000 7f5242200640 1 -- 172.21.15.46:0/1306196696 shutdown_connections 2024-03-29T08:20:09.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.255+0000 7f5242200640 1 -- 172.21.15.46:0/1306196696 wait complete. 2024-03-29T08:20:09.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.255+0000 7f5242200640 1 Processor -- start 2024-03-29T08:20:09.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.255+0000 7f5242200640 1 -- start start 2024-03-29T08:20:09.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f5242200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f523c0ae810 0x7f523c0c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f5242200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f523c0cb780 0x7f523c150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f5242200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f523c0ce8a0 0x7f523c0b4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f5242200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c0c49f0 con 0x7f523c0ce8a0 2024-03-29T08:20:09.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f5242200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523c150f60 con 0x7f523c0ae810 2024-03-29T08:20:09.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f5242200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523c0c3de0 con 0x7f523c0cb780 2024-03-29T08:20:09.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f523be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f523c0ce8a0 0x7f523c0b4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f523be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f523c0ce8a0 0x7f523c0b4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37720/0 (socket says 172.21.15.46:37720) 2024-03-29T08:20:09.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f523be00640 1 -- 172.21.15.46:0/2251034455 learned_addr learned my addr 172.21.15.46:0/2251034455 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:09.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f523b400640 1 --2- 172.21.15.46:0/2251034455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f523c0ae810 0x7f523c0c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f5233e00640 1 --2- 172.21.15.46:0/2251034455 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f523c0cb780 0x7f523c150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f523be00640 1 -- 172.21.15.46:0/2251034455 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f523c0cb780 msgr2=0x7f523c150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f523be00640 1 --2- 172.21.15.46:0/2251034455 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f523c0cb780 0x7f523c150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f523be00640 1 -- 172.21.15.46:0/2251034455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f523c0ae810 msgr2=0x7f523c0c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f523be00640 1 --2- 172.21.15.46:0/2251034455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f523c0ae810 0x7f523c0c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f523be00640 1 -- 172.21.15.46:0/2251034455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5234009a70 con 0x7f523c0ce8a0 2024-03-29T08:20:09.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f523b400640 1 --2- 172.21.15.46:0/2251034455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f523c0ae810 0x7f523c0c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:09.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f5233e00640 1 --2- 172.21.15.46:0/2251034455 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f523c0cb780 0x7f523c150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:09.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f523be00640 1 --2- 172.21.15.46:0/2251034455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f523c0ce8a0 0x7f523c0b4790 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f523400efc0 tx=0x7f5234009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:09.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f5233400640 1 -- 172.21.15.46:0/2251034455 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234009d70 con 0x7f523c0ce8a0 2024-03-29T08:20:09.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f5233400640 1 -- 172.21.15.46:0/2251034455 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523400f040 con 0x7f523c0ce8a0 2024-03-29T08:20:09.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f5242200640 1 -- 172.21.15.46:0/2251034455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522c009420 con 0x7f523c0ce8a0 2024-03-29T08:20:09.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f5233400640 1 -- 172.21.15.46:0/2251034455 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52340039a0 con 0x7f523c0ce8a0 2024-03-29T08:20:09.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f5242200640 1 -- 172.21.15.46:0/2251034455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523c0c43a0 con 0x7f523c0ce8a0 2024-03-29T08:20:09.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f5242200640 1 -- 172.21.15.46:0/2251034455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f523c066c90 con 0x7f523c0ce8a0 2024-03-29T08:20:09.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.259+0000 7f5233400640 1 -- 172.21.15.46:0/2251034455 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f52340144a0 con 0x7f523c0ce8a0 2024-03-29T08:20:09.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.263+0000 7f5233400640 1 --2- 172.21.15.46:0/2251034455 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52180666c0 0x7f5218068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.263+0000 7f5233400640 1 -- 172.21.15.46:0/2251034455 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5234084b20 con 0x7f523c0ce8a0 2024-03-29T08:20:09.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.263+0000 7f523b400640 1 --2- 172.21.15.46:0/2251034455 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52180666c0 0x7f5218068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.263+0000 7f523b400640 1 --2- 172.21.15.46:0/2251034455 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52180666c0 0x7f5218068b80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f522c0af000 tx=0x7f522c0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:09.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.267+0000 7f5233400640 1 -- 172.21.15.46:0/2251034455 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5234052140 con 0x7f523c0ce8a0 2024-03-29T08:20:09.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.483+0000 7f5242200640 1 -- 172.21.15.46:0/2251034455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f523c0b5400 con 0x7f523c0ce8a0 2024-03-29T08:20:09.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.483+0000 7f5233400640 1 -- 172.21.15.46:0/2251034455 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5234055760 con 0x7f523c0ce8a0 2024-03-29T08:20:09.486 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:09.486 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-03-29T08:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-03-29T08:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmlnXNJBAA7lI2lCBd0PeZudPsQyMOUQ==", 2024-03-29T08:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-03-29T08:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmbPkcMRAADnwJE17uILjJo14jghJuzg==", 2024-03-29T08:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.488 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-03-29T08:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmkCUVLBAAxcAOWGBDUCpEHeg/1OeQzA==", 2024-03-29T08:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-03-29T08:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZm7L11OBAAZ3fpXtEIYG1tGm9ZFswZ5Q==", 2024-03-29T08:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.489 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-03-29T08:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAIeQZmTHV8HhAAODJ+RP1Bx6k0D1153XgI0Q==", 2024-03-29T08:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:20:09.490 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-03-29T08:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQALeQZmZRD0JBAAjU6W/21CeqOMdIP5969iNA==", 2024-03-29T08:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-03-29T08:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAOeQZmSSP+KhAA26V8CByUJNfkVcG37AdlZQ==", 2024-03-29T08:20:09.492 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-03-29T08:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAReQZmb/e8LxAAV4yC1xlLwh8OAqOSPmcyyA==", 2024-03-29T08:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-03-29T08:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAXeQZmrLnzNRAAFPYQRs9sBNNKeyvcoI9rSg==", 2024-03-29T08:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.494 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-03-29T08:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAbeQZmjTPbABAAi4qTU5CBy9LtSSRf7cmfwg==", 2024-03-29T08:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-03-29T08:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAeeQZmo13eBRAAWY2VVBakLze9oA4ynp1rNA==", 2024-03-29T08:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:20:09.495 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-03-29T08:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAheQZmghpZCxAAdy5xI7qPwZb8LN8Rn6Ndaw==", 2024-03-29T08:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-03-29T08:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmyk22ABAA8Pn3OaIvxMQSLrKIM8FP4w==", 2024-03-29T08:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:20:09.496 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-03-29T08:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-29T08:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-03-29T08:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmG/E8ChAAJTcJ2j5xn0oUsmlV2trWdw==", 2024-03-29T08:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-03-29T08:20:09.497 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmhsqgBhAApqnO5qMG8VP27j9TfbZ8uw==", 2024-03-29T08:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-03-29T08:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmatlQEBAAAGKuGnAh4YXSk91TAzomXA==", 2024-03-29T08:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:20:09.498 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-03-29T08:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmp2B3CBAA4cfe/hUJeOg0SIr1KbWyUg==", 2024-03-29T08:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-03-29T08:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmyrJXIBAAv535Rz99UI3V7JvvzQ8oSw==", 2024-03-29T08:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-03-29T08:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-03-29T08:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZm72tiHBAA2N7qQnR5Ehu/7nP2WpklGg==", 2024-03-29T08:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:20:09.500 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-03-29T08:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-29T08:20:09.501 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:20:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.487+0000 7f5242200640 1 -- 172.21.15.46:0/2251034455 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52180666c0 msgr2=0x7f5218068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.487+0000 7f5242200640 1 --2- 172.21.15.46:0/2251034455 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52180666c0 0x7f5218068b80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f522c0af000 tx=0x7f522c0a7000 comp rx=0 tx=0).stop 2024-03-29T08:20:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.487+0000 7f5242200640 1 -- 172.21.15.46:0/2251034455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f523c0ce8a0 msgr2=0x7f523c0b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.487+0000 7f5242200640 1 --2- 172.21.15.46:0/2251034455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f523c0ce8a0 0x7f523c0b4790 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f523400efc0 tx=0x7f5234009f90 comp rx=0 tx=0).stop 2024-03-29T08:20:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.487+0000 7f5242200640 1 -- 172.21.15.46:0/2251034455 shutdown_connections 2024-03-29T08:20:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.487+0000 7f5242200640 1 --2- 172.21.15.46:0/2251034455 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52180666c0 0x7f5218068b80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.487+0000 7f5242200640 1 --2- 172.21.15.46:0/2251034455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f523c0ce8a0 0x7f523c0b4790 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.487+0000 7f5242200640 1 --2- 172.21.15.46:0/2251034455 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f523c0cb780 0x7f523c150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.487+0000 7f5242200640 1 --2- 172.21.15.46:0/2251034455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f523c0ae810 0x7f523c0c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.487+0000 7f5242200640 1 -- 172.21.15.46:0/2251034455 >> 172.21.15.46:0/2251034455 conn(0x7f523c098a40 msgr2=0x7f523c0cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:09.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.487+0000 7f5242200640 1 -- 172.21.15.46:0/2251034455 shutdown_connections 2024-03-29T08:20:09.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.487+0000 7f5242200640 1 -- 172.21.15.46:0/2251034455 wait complete. 2024-03-29T08:20:09.508 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:20:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.647+0000 7f23a1400640 1 Processor -- start 2024-03-29T08:20:09.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.647+0000 7f23a1400640 1 -- start start 2024-03-29T08:20:09.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.647+0000 7f23a1400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f239c0cb370 0x7f239c0cb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.647+0000 7f23a1400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f239c06a0f0 con 0x7f239c15bb20 2024-03-29T08:20:09.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.647+0000 7f23a1400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f239c15f150 con 0x7f239c15ec50 2024-03-29T08:20:09.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.647+0000 7f23a1400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239c15bf80 con 0x7f239c0cb370 2024-03-29T08:20:09.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.647+0000 7f2391e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f239c15bb20 0x7f239c15e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33538/0 (socket says 172.21.15.46:33538) 2024-03-29T08:20:09.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.647+0000 7f2391e00640 1 -- 172.21.15.46:0/570752249 learned_addr learned my addr 172.21.15.46:0/570752249 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:09.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.647+0000 7f239a800640 1 --2- 172.21.15.46:0/570752249 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f239c0cb370 0x7f239c0cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.647+0000 7f2399e00640 1 -- 172.21.15.46:0/570752249 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283769745 0 0) 0x7f239c06a0f0 con 0x7f239c15bb20 2024-03-29T08:20:09.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.647+0000 7f2399e00640 1 -- 172.21.15.46:0/570752249 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2380003530 con 0x7f239c15bb20 2024-03-29T08:20:09.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.647+0000 7f2399e00640 1 -- 172.21.15.46:0/570752249 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1022289096 0 0) 0x7f239c15f150 con 0x7f239c15ec50 2024-03-29T08:20:09.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.647+0000 7f2399e00640 1 -- 172.21.15.46:0/570752249 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f239c06a0f0 con 0x7f239c15ec50 2024-03-29T08:20:09.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.647+0000 7f239a800640 1 -- 172.21.15.46:0/570752249 >> v1:172.21.15.73:6789/0 conn(0x7f239c15ec50 legacy=0x7f239c15a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.647+0000 7f239a800640 1 -- 172.21.15.46:0/570752249 >> v1:172.21.15.46:6789/0 conn(0x7f239c15bb20 legacy=0x7f239c15e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.647+0000 7f239a800640 1 -- 172.21.15.46:0/570752249 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239c15f430 con 0x7f239c0cb370 2024-03-29T08:20:09.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.651+0000 7f239a800640 1 --2- 172.21.15.46:0/570752249 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f239c0cb370 0x7f239c0cb750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f238c0097f0 tx=0x7f238c01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d293ce0281283cb6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:09.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.651+0000 7f2399e00640 1 -- 172.21.15.46:0/570752249 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c0aa610 con 0x7f239c0cb370 2024-03-29T08:20:09.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.651+0000 7f2399e00640 1 -- 172.21.15.46:0/570752249 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238c0a6040 con 0x7f239c0cb370 2024-03-29T08:20:09.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.651+0000 7f2399e00640 1 -- 172.21.15.46:0/570752249 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c0aa970 con 0x7f239c0cb370 2024-03-29T08:20:09.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.651+0000 7f23a1400640 1 -- 172.21.15.46:0/570752249 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f239c0cb370 msgr2=0x7f239c0cb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.651+0000 7f23a1400640 1 --2- 172.21.15.46:0/570752249 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f239c0cb370 0x7f239c0cb750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f238c0097f0 tx=0x7f238c01c4b0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.651+0000 7f23a1400640 1 -- 172.21.15.46:0/570752249 shutdown_connections 2024-03-29T08:20:09.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.651+0000 7f23a1400640 1 --2- 172.21.15.46:0/570752249 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f239c0cb370 0x7f239c0cb750 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.651+0000 7f23a1400640 1 -- 172.21.15.46:0/570752249 >> 172.21.15.46:0/570752249 conn(0x7f239c098a40 msgr2=0x7f239c0cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:09.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.651+0000 7f23a1400640 1 -- 172.21.15.46:0/570752249 shutdown_connections 2024-03-29T08:20:09.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.651+0000 7f23a1400640 1 -- 172.21.15.46:0/570752249 wait complete. 2024-03-29T08:20:09.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f23a1400640 1 Processor -- start 2024-03-29T08:20:09.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f23a1400640 1 -- start start 2024-03-29T08:20:09.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f23a1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f239c0cb370 0x7f239c0c3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f23a1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f239c15bb20 0x7f239c15d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f23a1400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f239c15ec50 0x7f239c15d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f23a1400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c159530 con 0x7f239c0cb370 2024-03-29T08:20:09.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f23a1400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c158d60 con 0x7f239c15bb20 2024-03-29T08:20:09.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f23a1400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239c0c7570 con 0x7f239c15ec50 2024-03-29T08:20:09.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f239a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f239c0cb370 0x7f239c0c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f239a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f239c0cb370 0x7f239c0c3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37730/0 (socket says 172.21.15.46:37730) 2024-03-29T08:20:09.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f239a800640 1 -- 172.21.15.46:0/3147250896 learned_addr learned my addr 172.21.15.46:0/3147250896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:09.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f239b200640 1 --2- 172.21.15.46:0/3147250896 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f239c15ec50 0x7f239c15d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f2391e00640 1 --2- 172.21.15.46:0/3147250896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f239c15bb20 0x7f239c15d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f239a800640 1 -- 172.21.15.46:0/3147250896 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f239c15ec50 msgr2=0x7f239c15d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f239a800640 1 --2- 172.21.15.46:0/3147250896 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f239c15ec50 0x7f239c15d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f239a800640 1 -- 172.21.15.46:0/3147250896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f239c15bb20 msgr2=0x7f239c15d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f239a800640 1 --2- 172.21.15.46:0/3147250896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f239c15bb20 0x7f239c15d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f239a800640 1 -- 172.21.15.46:0/3147250896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238c009420 con 0x7f239c0cb370 2024-03-29T08:20:09.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f239a800640 1 --2- 172.21.15.46:0/3147250896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f239c0cb370 0x7f239c0c3030 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f238c000c10 tx=0x7f238c01acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:09.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f2393400640 1 -- 172.21.15.46:0/3147250896 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c006e20 con 0x7f239c0cb370 2024-03-29T08:20:09.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f2393400640 1 -- 172.21.15.46:0/3147250896 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238c0be410 con 0x7f239c0cb370 2024-03-29T08:20:09.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f23a1400640 1 -- 172.21.15.46:0/3147250896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239c158f70 con 0x7f239c0cb370 2024-03-29T08:20:09.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f2393400640 1 -- 172.21.15.46:0/3147250896 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c0be780 con 0x7f239c0cb370 2024-03-29T08:20:09.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f23a1400640 1 -- 172.21.15.46:0/3147250896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239c0c7200 con 0x7f239c0cb370 2024-03-29T08:20:09.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.655+0000 7f23a1400640 1 -- 172.21.15.46:0/3147250896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f239c0c7060 con 0x7f239c0cb370 2024-03-29T08:20:09.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.659+0000 7f2393400640 1 -- 172.21.15.46:0/3147250896 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f238c0be8e0 con 0x7f239c0cb370 2024-03-29T08:20:09.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.659+0000 7f2393400640 1 --2- 172.21.15.46:0/3147250896 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2378066690 0x7f2378068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.659+0000 7f2393400640 1 -- 172.21.15.46:0/3147250896 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f238c11e9c0 con 0x7f239c0cb370 2024-03-29T08:20:09.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.659+0000 7f2391e00640 1 --2- 172.21.15.46:0/3147250896 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2378066690 0x7f2378068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.659+0000 7f2391e00640 1 --2- 172.21.15.46:0/3147250896 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2378066690 0x7f2378068b50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f239c06f2b0 tx=0x7f2384009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:09.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.663+0000 7f2393400640 1 -- 172.21.15.46:0/3147250896 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f239c0c7060 con 0x7f239c0cb370 2024-03-29T08:20:09.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.823+0000 7f23a1400640 1 -- 172.21.15.46:0/3147250896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f239c0c7060 con 0x7f239c0cb370 2024-03-29T08:20:09.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.823+0000 7f2393400640 1 -- 172.21.15.46:0/3147250896 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+12341 (secure 0 0 0) 0x7f239c0c7060 con 0x7f239c0cb370 2024-03-29T08:20:09.827 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:09.827 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":45,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:20:09.077031+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:20:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.827+0000 7f23a1400640 1 -- 172.21.15.46:0/3147250896 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2378066690 msgr2=0x7f2378068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.827+0000 7f23a1400640 1 --2- 172.21.15.46:0/3147250896 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2378066690 0x7f2378068b50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f239c06f2b0 tx=0x7f2384009040 comp rx=0 tx=0).stop 2024-03-29T08:20:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.827+0000 7f23a1400640 1 -- 172.21.15.46:0/3147250896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f239c0cb370 msgr2=0x7f239c0c3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.827+0000 7f23a1400640 1 --2- 172.21.15.46:0/3147250896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f239c0cb370 0x7f239c0c3030 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f238c000c10 tx=0x7f238c01acf0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.827+0000 7f23a1400640 1 -- 172.21.15.46:0/3147250896 shutdown_connections 2024-03-29T08:20:09.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.827+0000 7f23a1400640 1 --2- 172.21.15.46:0/3147250896 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2378066690 0x7f2378068b50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.827+0000 7f23a1400640 1 --2- 172.21.15.46:0/3147250896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f239c15bb20 0x7f239c15d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.827+0000 7f23a1400640 1 --2- 172.21.15.46:0/3147250896 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f239c15ec50 0x7f239c15d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.827+0000 7f23a1400640 1 --2- 172.21.15.46:0/3147250896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f239c0cb370 0x7f239c0c3030 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.827+0000 7f23a1400640 1 -- 172.21.15.46:0/3147250896 >> 172.21.15.46:0/3147250896 conn(0x7f239c098a40 msgr2=0x7f239c0ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:09.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.827+0000 7f23a1400640 1 -- 172.21.15.46:0/3147250896 shutdown_connections 2024-03-29T08:20:09.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.827+0000 7f23a1400640 1 -- 172.21.15.46:0/3147250896 wait complete. 2024-03-29T08:20:09.849 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:20:09.849 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:20:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11c0c00640 1 Processor -- start 2024-03-29T08:20:09.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11c0c00640 1 -- start start 2024-03-29T08:20:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11c0c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f11bc142a50 0x7f11bc0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11c0c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11bc0ae990 0x7f11bc0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11c0c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11bc15d260 con 0x7f11bc15df90 2024-03-29T08:20:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11c0c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11bc0aefd0 con 0x7f11bc142a50 2024-03-29T08:20:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11c0c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc15d4b0 con 0x7f11bc0ae990 2024-03-29T08:20:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b9400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f11bc15df90 0x7f11bc0ae280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33552/0 (socket says 172.21.15.46:33552) 2024-03-29T08:20:09.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b9400640 1 -- 172.21.15.46:0/2588184080 learned_addr learned my addr 172.21.15.46:0/2588184080 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b9e00640 1 --2- 172.21.15.46:0/2588184080 >> v2:172.21.15.73:3300/0 conn(0x7f11bc142a50 0x7f11bc0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11ba800640 1 --2- 172.21.15.46:0/2588184080 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11bc0ae990 0x7f11bc0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b8a00640 1 -- 172.21.15.46:0/2588184080 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2049800851 0 0) 0x7f11bc15d260 con 0x7f11bc15df90 2024-03-29T08:20:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b8a00640 1 -- 172.21.15.46:0/2588184080 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11a0003530 con 0x7f11bc15df90 2024-03-29T08:20:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b8a00640 1 -- 172.21.15.46:0/2588184080 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2664957190 0 0) 0x7f11a0003530 con 0x7f11bc15df90 2024-03-29T08:20:09.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b8a00640 1 -- 172.21.15.46:0/2588184080 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11bc0ae990 msgr2=0x7f11bc0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b8a00640 1 --2- 172.21.15.46:0/2588184080 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11bc0ae990 0x7f11bc0cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b8a00640 1 -- 172.21.15.46:0/2588184080 >> v2:172.21.15.73:3300/0 conn(0x7f11bc142a50 msgr2=0x7f11bc0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b8a00640 1 --2- 172.21.15.46:0/2588184080 >> v2:172.21.15.73:3300/0 conn(0x7f11bc142a50 0x7f11bc0cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b8a00640 1 -- 172.21.15.46:0/2588184080 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11bc15d790 con 0x7f11bc15df90 2024-03-29T08:20:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b8a00640 1 -- 172.21.15.46:0/2588184080 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f11a8001680 con 0x7f11bc15df90 2024-03-29T08:20:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b8a00640 1 -- 172.21.15.46:0/2588184080 >> v1:172.21.15.46:6789/0 conn(0x7f11bc15df90 legacy=0x7f11bc0ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b8a00640 1 --2- 172.21.15.46:0/2588184080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11a0003980 0x7f11a0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b8a00640 1 --2- 172.21.15.46:0/2588184080 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11bc0ae990 0x7f11a00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b8a00640 1 --2- 172.21.15.46:0/2588184080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a0004d10 0x7f11a0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b8a00640 1 -- 172.21.15.46:0/2588184080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc0aefd0 con 0x7f11a0003980 2024-03-29T08:20:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b8a00640 1 -- 172.21.15.46:0/2588184080 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc15d4b0 con 0x7f11a0004d10 2024-03-29T08:20:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b8a00640 1 -- 172.21.15.46:0/2588184080 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a000b990 con 0x7f11bc0ae990 2024-03-29T08:20:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b9e00640 1 --2- 172.21.15.46:0/2588184080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11a0003980 0x7f11a0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b9400640 1 --2- 172.21.15.46:0/2588184080 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11bc0ae990 0x7f11a00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11ba800640 1 --2- 172.21.15.46:0/2588184080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a0004d10 0x7f11a0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b9e00640 1 -- 172.21.15.46:0/2588184080 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11bc0ae990 msgr2=0x7f11a00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b9e00640 1 --2- 172.21.15.46:0/2588184080 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11bc0ae990 0x7f11a00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b9e00640 1 -- 172.21.15.46:0/2588184080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a0004d10 msgr2=0x7f11a0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b9e00640 1 --2- 172.21.15.46:0/2588184080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a0004d10 0x7f11a0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11b9e00640 1 -- 172.21.15.46:0/2588184080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a0003530 con 0x7f11a0003980 2024-03-29T08:20:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.991+0000 7f11ba800640 1 --2- 172.21.15.46:0/2588184080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a0004d10 0x7f11a0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.995+0000 7f11b9e00640 1 --2- 172.21.15.46:0/2588184080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11a0003980 0x7f11a0003da0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009ae0 tx=0x7f11ac01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b27a593835f504 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.995+0000 7f11b8a00640 1 -- 172.21.15.46:0/2588184080 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac0b0400 con 0x7f11a0003980 2024-03-29T08:20:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.995+0000 7f11b8a00640 1 -- 172.21.15.46:0/2588184080 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ac0b0560 con 0x7f11a0003980 2024-03-29T08:20:09.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.995+0000 7f11b8a00640 1 -- 172.21.15.46:0/2588184080 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac0b0910 con 0x7f11a0003980 2024-03-29T08:20:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.995+0000 7f11c0c00640 1 -- 172.21.15.46:0/2588184080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11a0003980 msgr2=0x7f11a0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.995+0000 7f11c0c00640 1 --2- 172.21.15.46:0/2588184080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11a0003980 0x7f11a0003da0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009ae0 tx=0x7f11ac01cab0 comp rx=0 tx=0).stop 2024-03-29T08:20:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.995+0000 7f11ba800640 1 -- 172.21.15.46:0/2588184080 reap_dead start 2024-03-29T08:20:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.995+0000 7f11c0c00640 1 -- 172.21.15.46:0/2588184080 shutdown_connections 2024-03-29T08:20:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.995+0000 7f11c0c00640 1 -- 172.21.15.46:0/2588184080 >> 172.21.15.46:0/2588184080 conn(0x7f11bc06a390 msgr2=0x7f11bc0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.995+0000 7f11c0c00640 1 -- 172.21.15.46:0/2588184080 shutdown_connections 2024-03-29T08:20:09.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.995+0000 7f11c0c00640 1 -- 172.21.15.46:0/2588184080 wait complete. 2024-03-29T08:20:10.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11c0c00640 1 Processor -- start 2024-03-29T08:20:10.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11c0c00640 1 -- start start 2024-03-29T08:20:10.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11c0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a0003980 0x7f11bc165e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11c0c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11bc166380 0x7f11bc0ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11c0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11bc0acf30 0x7f11bc13db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11c0c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc0ac560 con 0x7f11bc0acf30 2024-03-29T08:20:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11c0c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc0ab950 con 0x7f11a0003980 2024-03-29T08:20:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11c0c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc0abc30 con 0x7f11bc166380 2024-03-29T08:20:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11ba800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11bc0acf30 0x7f11bc13db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11ba800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11bc0acf30 0x7f11bc13db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37752/0 (socket says 172.21.15.46:37752) 2024-03-29T08:20:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11ba800640 1 -- 172.21.15.46:0/1387348239 learned_addr learned my addr 172.21.15.46:0/1387348239 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11b9400640 1 --2- 172.21.15.46:0/1387348239 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11bc166380 0x7f11bc0ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11b9e00640 1 --2- 172.21.15.46:0/1387348239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a0003980 0x7f11bc165e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:10.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11ba800640 1 -- 172.21.15.46:0/1387348239 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11bc166380 msgr2=0x7f11bc0ac9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:10.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11ba800640 1 --2- 172.21.15.46:0/1387348239 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11bc166380 0x7f11bc0ac9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11ba800640 1 -- 172.21.15.46:0/1387348239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a0003980 msgr2=0x7f11bc165e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:10.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11ba800640 1 --2- 172.21.15.46:0/1387348239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a0003980 0x7f11bc165e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11ba800640 1 -- 172.21.15.46:0/1387348239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11bc0abf10 con 0x7f11bc0acf30 2024-03-29T08:20:10.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11ba800640 1 --2- 172.21.15.46:0/1387348239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11bc0acf30 0x7f11bc13db30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f11bc071720 tx=0x7f11b4010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:10.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11b2000640 1 -- 172.21.15.46:0/1387348239 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b401b070 con 0x7f11bc0acf30 2024-03-29T08:20:10.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11b2000640 1 -- 172.21.15.46:0/1387348239 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b400f040 con 0x7f11bc0acf30 2024-03-29T08:20:10.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11b2000640 1 -- 172.21.15.46:0/1387348239 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b400e050 con 0x7f11bc0acf30 2024-03-29T08:20:10.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11c0c00640 1 -- 172.21.15.46:0/1387348239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11bc0ac1f0 con 0x7f11bc0acf30 2024-03-29T08:20:10.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:09.999+0000 7f11c0c00640 1 -- 172.21.15.46:0/1387348239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11bc156f60 con 0x7f11bc0acf30 2024-03-29T08:20:10.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.003+0000 7f11c0c00640 1 -- 172.21.15.46:0/1387348239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11bc0ac3b0 con 0x7f11bc0acf30 2024-03-29T08:20:10.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.003+0000 7f11b2000640 1 -- 172.21.15.46:0/1387348239 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f11b4015440 con 0x7f11bc0acf30 2024-03-29T08:20:10.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.003+0000 7f11b2000640 1 --2- 172.21.15.46:0/1387348239 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f119c0666c0 0x7f119c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:10.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.003+0000 7f11b2000640 1 -- 172.21.15.46:0/1387348239 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f11b40840d0 con 0x7f11bc0acf30 2024-03-29T08:20:10.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.003+0000 7f11b9e00640 1 --2- 172.21.15.46:0/1387348239 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f119c0666c0 0x7f119c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:10.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.003+0000 7f11b9e00640 1 --2- 172.21.15.46:0/1387348239 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f119c0666c0 0x7f119c068b80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0af000 tx=0x7f11ac0046d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:10.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.007+0000 7f11b2000640 1 -- 172.21.15.46:0/1387348239 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f11b4051780 con 0x7f11bc0acf30 2024-03-29T08:20:10.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.167+0000 7f11c0c00640 1 -- 172.21.15.46:0/1387348239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11bc071a70 con 0x7f11bc0acf30 2024-03-29T08:20:10.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.167+0000 7f11b2000640 1 -- 172.21.15.46:0/1387348239 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11b4054da0 con 0x7f11bc0acf30 2024-03-29T08:20:10.171 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:20:10.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.171+0000 7f11c0c00640 1 -- 172.21.15.46:0/1387348239 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f119c0666c0 msgr2=0x7f119c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.171+0000 7f11c0c00640 1 --2- 172.21.15.46:0/1387348239 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f119c0666c0 0x7f119c068b80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0af000 tx=0x7f11ac0046d0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.171+0000 7f11c0c00640 1 -- 172.21.15.46:0/1387348239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11bc0acf30 msgr2=0x7f11bc13db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.171+0000 7f11c0c00640 1 --2- 172.21.15.46:0/1387348239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11bc0acf30 0x7f11bc13db30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f11bc071720 tx=0x7f11b4010590 comp rx=0 tx=0).stop 2024-03-29T08:20:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.171+0000 7f11c0c00640 1 -- 172.21.15.46:0/1387348239 shutdown_connections 2024-03-29T08:20:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.171+0000 7f11c0c00640 1 --2- 172.21.15.46:0/1387348239 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f119c0666c0 0x7f119c068b80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.171+0000 7f11c0c00640 1 --2- 172.21.15.46:0/1387348239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11bc0acf30 0x7f11bc13db30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.171+0000 7f11c0c00640 1 --2- 172.21.15.46:0/1387348239 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11bc166380 0x7f11bc0ac9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.171+0000 7f11c0c00640 1 --2- 172.21.15.46:0/1387348239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a0003980 0x7f11bc165e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.171+0000 7f11c0c00640 1 -- 172.21.15.46:0/1387348239 >> 172.21.15.46:0/1387348239 conn(0x7f11bc06a390 msgr2=0x7f11bc0cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.171+0000 7f11c0c00640 1 -- 172.21.15.46:0/1387348239 shutdown_connections 2024-03-29T08:20:10.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.171+0000 7f11c0c00640 1 -- 172.21.15.46:0/1387348239 wait complete. 2024-03-29T08:20:10.192 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T08:20:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5d0800640 1 Processor -- start 2024-03-29T08:20:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5d0800640 1 -- start start 2024-03-29T08:20:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5d0800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd5c80ce8a0 0x7fd5c80caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5d0800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd5c80cb780 0x7fd5c80aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5d0800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c80cbc10 con 0x7fd5c80ae810 2024-03-29T08:20:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5d0800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c80c9bd0 con 0x7fd5c80cb780 2024-03-29T08:20:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5d0800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c80c9eb0 con 0x7fd5c80ce8a0 2024-03-29T08:20:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5cd400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd5c80ae810 0x7fd5c80cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33558/0 (socket says 172.21.15.46:33558) 2024-03-29T08:20:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5cd400640 1 -- 172.21.15.46:0/3360953629 learned_addr learned my addr 172.21.15.46:0/3360953629 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5cde00640 1 --2- 172.21.15.46:0/3360953629 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd5c80ce8a0 0x7fd5c80caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5ce800640 1 --2- 172.21.15.46:0/3360953629 >> v2:172.21.15.73:3300/0 conn(0x7fd5c80cb780 0x7fd5c80aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5cca00640 1 -- 172.21.15.46:0/3360953629 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881944070 0 0) 0x7fd5c80cbc10 con 0x7fd5c80ae810 2024-03-29T08:20:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5cca00640 1 -- 172.21.15.46:0/3360953629 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5b0003530 con 0x7fd5c80ae810 2024-03-29T08:20:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5cde00640 1 -- 172.21.15.46:0/3360953629 >> v2:172.21.15.73:3300/0 conn(0x7fd5c80cb780 msgr2=0x7fd5c80aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5cde00640 1 --2- 172.21.15.46:0/3360953629 >> v2:172.21.15.73:3300/0 conn(0x7fd5c80cb780 0x7fd5c80aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5cde00640 1 -- 172.21.15.46:0/3360953629 >> v1:172.21.15.46:6789/0 conn(0x7fd5c80ae810 legacy=0x7fd5c80cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5cde00640 1 -- 172.21.15.46:0/3360953629 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c80ca190 con 0x7fd5c80ce8a0 2024-03-29T08:20:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.331+0000 7fd5ce800640 1 --2- 172.21.15.46:0/3360953629 >> v2:172.21.15.73:3300/0 conn(0x7fd5c80cb780 0x7fd5c80aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:20:10.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.335+0000 7fd5cde00640 1 --2- 172.21.15.46:0/3360953629 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd5c80ce8a0 0x7fd5c80caf50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0097f0 tx=0x7fd5bc01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea12be2240ba1e10 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.335+0000 7fd5cca00640 1 -- 172.21.15.46:0/3360953629 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c80afa20 con 0x7fd5c80ce8a0 2024-03-29T08:20:10.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.335+0000 7fd5cca00640 1 -- 172.21.15.46:0/3360953629 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5bc0ab670 con 0x7fd5c80ce8a0 2024-03-29T08:20:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.335+0000 7fd5cca00640 1 -- 172.21.15.46:0/3360953629 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc0aba00 con 0x7fd5c80ce8a0 2024-03-29T08:20:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.335+0000 7fd5d0800640 1 -- 172.21.15.46:0/3360953629 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd5c80ce8a0 msgr2=0x7fd5c80caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.335+0000 7fd5d0800640 1 --2- 172.21.15.46:0/3360953629 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd5c80ce8a0 0x7fd5c80caf50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc0097f0 tx=0x7fd5bc01c470 comp rx=0 tx=0).stop 2024-03-29T08:20:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.335+0000 7fd5d0800640 1 -- 172.21.15.46:0/3360953629 shutdown_connections 2024-03-29T08:20:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.335+0000 7fd5d0800640 1 --2- 172.21.15.46:0/3360953629 >> v2:172.21.15.73:3300/0 conn(0x7fd5c80cb780 0x7fd5c80aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.335+0000 7fd5d0800640 1 --2- 172.21.15.46:0/3360953629 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd5c80ce8a0 0x7fd5c80caf50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.335+0000 7fd5d0800640 1 -- 172.21.15.46:0/3360953629 >> 172.21.15.46:0/3360953629 conn(0x7fd5c8098a40 msgr2=0x7fd5c80cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:10.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.335+0000 7fd5d0800640 1 -- 172.21.15.46:0/3360953629 shutdown_connections 2024-03-29T08:20:10.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.335+0000 7fd5d0800640 1 -- 172.21.15.46:0/3360953629 wait complete. 2024-03-29T08:20:10.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5d0800640 1 Processor -- start 2024-03-29T08:20:10.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5d0800640 1 -- start start 2024-03-29T08:20:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5d0800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c80ae810 0x7fd5c80b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5d0800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd5c80cb780 0x7fd5c8150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5d0800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5c80ce8a0 0x7fd5c80c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5d0800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c80b31f0 con 0x7fd5c80ae810 2024-03-29T08:20:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5d0800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c81514b0 con 0x7fd5c80ce8a0 2024-03-29T08:20:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5d0800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c80c8dc0 con 0x7fd5c80cb780 2024-03-29T08:20:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5cde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c80ae810 0x7fd5c80b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5cde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c80ae810 0x7fd5c80b0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37766/0 (socket says 172.21.15.46:37766) 2024-03-29T08:20:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5cde00640 1 -- 172.21.15.46:0/3555905020 learned_addr learned my addr 172.21.15.46:0/3555905020 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5ce800640 1 --2- 172.21.15.46:0/3555905020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5c80ce8a0 0x7fd5c80c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5cd400640 1 --2- 172.21.15.46:0/3555905020 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd5c80cb780 0x7fd5c8150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5cde00640 1 -- 172.21.15.46:0/3555905020 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd5c80cb780 msgr2=0x7fd5c8150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5cde00640 1 --2- 172.21.15.46:0/3555905020 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd5c80cb780 0x7fd5c8150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5cde00640 1 -- 172.21.15.46:0/3555905020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5c80ce8a0 msgr2=0x7fd5c80c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5cde00640 1 --2- 172.21.15.46:0/3555905020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5c80ce8a0 0x7fd5c80c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5cde00640 1 -- 172.21.15.46:0/3555905020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c4009a70 con 0x7fd5c80ae810 2024-03-29T08:20:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5cde00640 1 --2- 172.21.15.46:0/3555905020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c80ae810 0x7fd5c80b0940 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc01ab20 tx=0x7fd5bc019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5c2000640 1 -- 172.21.15.46:0/3555905020 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc0abd30 con 0x7fd5c80ae810 2024-03-29T08:20:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5c2000640 1 -- 172.21.15.46:0/3555905020 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5bc0bf450 con 0x7fd5c80ae810 2024-03-29T08:20:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5c2000640 1 -- 172.21.15.46:0/3555905020 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc0bf800 con 0x7fd5c80ae810 2024-03-29T08:20:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5d0800640 1 -- 172.21.15.46:0/3555905020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5bc009420 con 0x7fd5c80ae810 2024-03-29T08:20:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.339+0000 7fd5d0800640 1 -- 172.21.15.46:0/3555905020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c80b1320 con 0x7fd5c80ae810 2024-03-29T08:20:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.343+0000 7fd5d0800640 1 -- 172.21.15.46:0/3555905020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c8066c90 con 0x7fd5c80ae810 2024-03-29T08:20:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.343+0000 7fd5c2000640 1 -- 172.21.15.46:0/3555905020 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd5bc0b43f0 con 0x7fd5c80ae810 2024-03-29T08:20:10.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.343+0000 7fd5c2000640 1 --2- 172.21.15.46:0/3555905020 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd5ac066690 0x7fd5ac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:10.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.343+0000 7fd5c2000640 1 -- 172.21.15.46:0/3555905020 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd5bc11e6d0 con 0x7fd5c80ae810 2024-03-29T08:20:10.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.343+0000 7fd5cd400640 1 --2- 172.21.15.46:0/3555905020 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd5ac066690 0x7fd5ac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:10.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.343+0000 7fd5cd400640 1 --2- 172.21.15.46:0/3555905020 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd5ac066690 0x7fd5ac068b50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80b2050 tx=0x7fd5b8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:10.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.347+0000 7fd5c2000640 1 -- 172.21.15.46:0/3555905020 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd5bc0ebea0 con 0x7fd5c80ae810 2024-03-29T08:20:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.511+0000 7fd5d0800640 1 -- 172.21.15.46:0/3555905020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd5c80b2fc0 con 0x7fd5c80ae810 2024-03-29T08:20:10.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.511+0000 7fd5c2000640 1 -- 172.21.15.46:0/3555905020 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v45) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd5bc0ef4c0 con 0x7fd5c80ae810 2024-03-29T08:20:10.515 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:10.515 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-03-29T08:20:10.515 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:10.515 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-03-29T08:20:10.516 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-03-29T08:20:10.516 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:10.516 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-03-29T08:20:10.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.515+0000 7fd5d0800640 1 -- 172.21.15.46:0/3555905020 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd5ac066690 msgr2=0x7fd5ac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:10.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.515+0000 7fd5d0800640 1 --2- 172.21.15.46:0/3555905020 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd5ac066690 0x7fd5ac068b50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80b2050 tx=0x7fd5b8009040 comp rx=0 tx=0).stop 2024-03-29T08:20:10.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.515+0000 7fd5d0800640 1 -- 172.21.15.46:0/3555905020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c80ae810 msgr2=0x7fd5c80b0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:10.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.515+0000 7fd5d0800640 1 --2- 172.21.15.46:0/3555905020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c80ae810 0x7fd5c80b0940 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc01ab20 tx=0x7fd5bc019210 comp rx=0 tx=0).stop 2024-03-29T08:20:10.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.515+0000 7fd5d0800640 1 -- 172.21.15.46:0/3555905020 shutdown_connections 2024-03-29T08:20:10.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.515+0000 7fd5d0800640 1 --2- 172.21.15.46:0/3555905020 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd5ac066690 0x7fd5ac068b50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.515+0000 7fd5d0800640 1 --2- 172.21.15.46:0/3555905020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5c80ce8a0 0x7fd5c80c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.515+0000 7fd5d0800640 1 --2- 172.21.15.46:0/3555905020 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd5c80cb780 0x7fd5c8150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.515+0000 7fd5d0800640 1 --2- 172.21.15.46:0/3555905020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5c80ae810 0x7fd5c80b0940 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.515+0000 7fd5d0800640 1 -- 172.21.15.46:0/3555905020 >> 172.21.15.46:0/3555905020 conn(0x7fd5c8098a40 msgr2=0x7fd5c81564a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:10.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.515+0000 7fd5d0800640 1 -- 172.21.15.46:0/3555905020 shutdown_connections 2024-03-29T08:20:10.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.515+0000 7fd5d0800640 1 -- 172.21.15.46:0/3555905020 wait complete. 2024-03-29T08:20:10.536 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T08:20:10.536 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T08:20:10.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.675+0000 7f33a4000640 1 Processor -- start 2024-03-29T08:20:10.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.679+0000 7f33a4000640 1 -- start start 2024-03-29T08:20:10.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.679+0000 7f33a4000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f339c15dc20 0x7f339c15e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:10.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.679+0000 7f33a4000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f339c15e000 0x7f339c15ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:10.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.679+0000 7f33a4000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f339c0cc390 con 0x7f339c0b19a0 2024-03-29T08:20:10.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.679+0000 7f33a4000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f339c15f100 con 0x7f339c15dc20 2024-03-29T08:20:10.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.679+0000 7f33a4000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339c15f3e0 con 0x7f339c15e000 2024-03-29T08:20:10.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.679+0000 7f33a1600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f339c15dc20 0x7f339c15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:10.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.679+0000 7f33a1600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f339c15dc20 0x7f339c15e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37780/0 (socket says 172.21.15.46:37780) 2024-03-29T08:20:10.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.679+0000 7f33a1600640 1 -- 172.21.15.46:0/1708508050 learned_addr learned my addr 172.21.15.46:0/1708508050 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:10.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.679+0000 7f33a2000640 1 --2- 172.21.15.46:0/1708508050 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f339c15e000 0x7f339c15ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:10.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.679+0000 7f33a1600640 1 -- 172.21.15.46:0/1708508050 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f339c15e000 msgr2=0x7f339c15ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:10.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.679+0000 7f33a1600640 1 --2- 172.21.15.46:0/1708508050 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f339c15e000 0x7f339c15ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.679+0000 7f33a1600640 1 -- 172.21.15.46:0/1708508050 >> v1:172.21.15.73:6789/0 conn(0x7f339c0b19a0 legacy=0x7f339c0ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:10.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.679+0000 7f33a1600640 1 -- 172.21.15.46:0/1708508050 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339c0cbd40 con 0x7f339c15dc20 2024-03-29T08:20:10.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.679+0000 7f33a2000640 1 --2- 172.21.15.46:0/1708508050 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f339c15e000 0x7f339c15ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:10.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.683+0000 7f33a1600640 1 --2- 172.21.15.46:0/1708508050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c15dc20 0x7f339c15e530 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3390009870 tx=0x7f339001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fc41c1b0951689 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:10.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.683+0000 7f339be00640 1 -- 172.21.15.46:0/1708508050 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33900ac640 con 0x7f339c15dc20 2024-03-29T08:20:10.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.683+0000 7f339be00640 1 -- 172.21.15.46:0/1708508050 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33900ac7a0 con 0x7f339c15dc20 2024-03-29T08:20:10.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.683+0000 7f339be00640 1 -- 172.21.15.46:0/1708508050 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33900acb30 con 0x7f339c15dc20 2024-03-29T08:20:10.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.683+0000 7f33a4000640 1 -- 172.21.15.46:0/1708508050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c15dc20 msgr2=0x7f339c15e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:10.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.683+0000 7f33a4000640 1 --2- 172.21.15.46:0/1708508050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c15dc20 0x7f339c15e530 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3390009870 tx=0x7f339001c4a0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.683+0000 7f33a4000640 1 -- 172.21.15.46:0/1708508050 shutdown_connections 2024-03-29T08:20:10.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.683+0000 7f33a4000640 1 --2- 172.21.15.46:0/1708508050 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f339c15e000 0x7f339c15ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.683+0000 7f33a4000640 1 --2- 172.21.15.46:0/1708508050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c15dc20 0x7f339c15e530 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.683+0000 7f33a4000640 1 -- 172.21.15.46:0/1708508050 >> 172.21.15.46:0/1708508050 conn(0x7f339c0b21b0 msgr2=0x7f339c0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:10.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.683+0000 7f33a4000640 1 -- 172.21.15.46:0/1708508050 shutdown_connections 2024-03-29T08:20:10.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.683+0000 7f33a4000640 1 -- 172.21.15.46:0/1708508050 wait complete. 2024-03-29T08:20:10.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.683+0000 7f33a4000640 1 Processor -- start 2024-03-29T08:20:10.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.683+0000 7f33a4000640 1 -- start start 2024-03-29T08:20:10.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a4000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c0b19a0 0x7f339c1476c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:10.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a4000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f339c15dc20 0x7f339c0c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:10.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a4000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f339c15e000 0x7f339c0affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:10.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a4000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339c0b0ad0 con 0x7f339c0b19a0 2024-03-29T08:20:10.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a4000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339c0c87e0 con 0x7f339c15dc20 2024-03-29T08:20:10.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a4000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339c0c8ac0 con 0x7f339c15e000 2024-03-29T08:20:10.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c0b19a0 0x7f339c1476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:10.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c0b19a0 0x7f339c1476c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37786/0 (socket says 172.21.15.46:37786) 2024-03-29T08:20:10.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a1600640 1 -- 172.21.15.46:0/2226950682 learned_addr learned my addr 172.21.15.46:0/2226950682 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:10.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a0c00640 1 --2- 172.21.15.46:0/2226950682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f339c15dc20 0x7f339c0c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:10.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a2000640 1 --2- 172.21.15.46:0/2226950682 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f339c15e000 0x7f339c0affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:10.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a1600640 1 -- 172.21.15.46:0/2226950682 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f339c15e000 msgr2=0x7f339c0affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:10.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a1600640 1 --2- 172.21.15.46:0/2226950682 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f339c15e000 0x7f339c0affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a1600640 1 -- 172.21.15.46:0/2226950682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f339c15dc20 msgr2=0x7f339c0c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:10.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a1600640 1 --2- 172.21.15.46:0/2226950682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f339c15dc20 0x7f339c0c6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a1600640 1 -- 172.21.15.46:0/2226950682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33900094d0 con 0x7f339c0b19a0 2024-03-29T08:20:10.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a1600640 1 --2- 172.21.15.46:0/2226950682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c0b19a0 0x7f339c1476c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f33900af040 tx=0x7f3390019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:10.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f3399600640 1 -- 172.21.15.46:0/2226950682 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33900b5830 con 0x7f339c0b19a0 2024-03-29T08:20:10.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f3399600640 1 -- 172.21.15.46:0/2226950682 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33900b59b0 con 0x7f339c0b19a0 2024-03-29T08:20:10.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a4000640 1 -- 172.21.15.46:0/2226950682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339c0c8da0 con 0x7f339c0b19a0 2024-03-29T08:20:10.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f3399600640 1 -- 172.21.15.46:0/2226950682 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33900b5d60 con 0x7f339c0b19a0 2024-03-29T08:20:10.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a4000640 1 -- 172.21.15.46:0/2226950682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f339c147c00 con 0x7f339c0b19a0 2024-03-29T08:20:10.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f33a4000640 1 -- 172.21.15.46:0/2226950682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339c066c90 con 0x7f339c0b19a0 2024-03-29T08:20:10.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.687+0000 7f3399600640 1 -- 172.21.15.46:0/2226950682 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f33900b3070 con 0x7f339c0b19a0 2024-03-29T08:20:10.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.691+0000 7f3399600640 1 --2- 172.21.15.46:0/2226950682 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f337c066690 0x7f337c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.691+0000 7f3399600640 1 -- 172.21.15.46:0/2226950682 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f339011ea30 con 0x7f339c0b19a0 2024-03-29T08:20:10.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.691+0000 7f33a0c00640 1 --2- 172.21.15.46:0/2226950682 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f337c066690 0x7f337c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:10.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.691+0000 7f33a0c00640 1 --2- 172.21.15.46:0/2226950682 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f337c066690 0x7f337c068b50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f339c1f8af0 tx=0x7f3388009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:10.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.695+0000 7f3399600640 1 -- 172.21.15.46:0/2226950682 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f33900ec100 con 0x7f339c0b19a0 2024-03-29T08:20:10.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.855+0000 7f33a4000640 1 -- 172.21.15.46:0/2226950682 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f339c0b0c10 con 0x7f339c0b19a0 2024-03-29T08:20:10.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.983+0000 7f3399600640 1 -- 172.21.15.46:0/2226950682 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v46) v1 ==== 138+0+0 (secure 0 0 0) 0x7f33900b1070 con 0x7f339c0b19a0 2024-03-29T08:20:10.988 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-03-29T08:20:10.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.991+0000 7f33a4000640 1 -- 172.21.15.46:0/2226950682 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f337c066690 msgr2=0x7f337c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:10.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.991+0000 7f33a4000640 1 --2- 172.21.15.46:0/2226950682 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f337c066690 0x7f337c068b50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f339c1f8af0 tx=0x7f3388009040 comp rx=0 tx=0).stop 2024-03-29T08:20:10.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.991+0000 7f33a4000640 1 -- 172.21.15.46:0/2226950682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c0b19a0 msgr2=0x7f339c1476c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:10.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.991+0000 7f33a4000640 1 --2- 172.21.15.46:0/2226950682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c0b19a0 0x7f339c1476c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f33900af040 tx=0x7f3390019240 comp rx=0 tx=0).stop 2024-03-29T08:20:10.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.991+0000 7f33a4000640 1 -- 172.21.15.46:0/2226950682 shutdown_connections 2024-03-29T08:20:10.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.991+0000 7f33a4000640 1 --2- 172.21.15.46:0/2226950682 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f337c066690 0x7f337c068b50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.991+0000 7f33a4000640 1 --2- 172.21.15.46:0/2226950682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f339c15dc20 0x7f339c0c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.991+0000 7f33a4000640 1 --2- 172.21.15.46:0/2226950682 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f339c15e000 0x7f339c0affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.991+0000 7f33a4000640 1 --2- 172.21.15.46:0/2226950682 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f339c0b19a0 0x7f339c1476c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:10.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.991+0000 7f33a4000640 1 -- 172.21.15.46:0/2226950682 >> 172.21.15.46:0/2226950682 conn(0x7f339c0b21b0 msgr2=0x7f339c0cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:10.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.991+0000 7f33a4000640 1 -- 172.21.15.46:0/2226950682 shutdown_connections 2024-03-29T08:20:10.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:10.991+0000 7f33a4000640 1 -- 172.21.15.46:0/2226950682 wait complete. 2024-03-29T08:20:11.013 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T08:20:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b0e600640 1 Processor -- start 2024-03-29T08:20:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b0e600640 1 -- start start 2024-03-29T08:20:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b0e600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4b08158300 0x7f4b0806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b0e600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b0806c390 con 0x7f4b080709b0 2024-03-29T08:20:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b0e600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b0806d600 con 0x7f4b080af530 2024-03-29T08:20:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b0e600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b080afff0 con 0x7f4b08158300 2024-03-29T08:20:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b07e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4b080709b0 0x7f4b0815bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33570/0 (socket says 172.21.15.46:33570) 2024-03-29T08:20:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b07e00640 1 -- 172.21.15.46:0/3200017886 learned_addr learned my addr 172.21.15.46:0/3200017886 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b07400640 1 --2- 172.21.15.46:0/3200017886 >> v2:172.21.15.73:3300/0 conn(0x7f4b08158300 0x7f4b0806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06000640 1 -- 172.21.15.46:0/3200017886 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164477695 0 0) 0x7f4b0806c390 con 0x7f4b080709b0 2024-03-29T08:20:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06000640 1 -- 172.21.15.46:0/3200017886 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4aec003530 con 0x7f4b080709b0 2024-03-29T08:20:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06000640 1 -- 172.21.15.46:0/3200017886 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2506934828 0 0) 0x7f4b0806d600 con 0x7f4b080af530 2024-03-29T08:20:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06000640 1 -- 172.21.15.46:0/3200017886 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b0806c390 con 0x7f4b080af530 2024-03-29T08:20:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06000640 1 -- 172.21.15.46:0/3200017886 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2455809226 0 0) 0x7f4aec003530 con 0x7f4b080709b0 2024-03-29T08:20:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06000640 1 -- 172.21.15.46:0/3200017886 >> v2:172.21.15.73:3300/0 conn(0x7f4b08158300 msgr2=0x7f4b0806d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06000640 1 --2- 172.21.15.46:0/3200017886 >> v2:172.21.15.73:3300/0 conn(0x7f4b08158300 0x7f4b0806d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06000640 1 -- 172.21.15.46:0/3200017886 >> v1:172.21.15.73:6789/0 conn(0x7f4b080af530 legacy=0x7f4b080702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06000640 1 -- 172.21.15.46:0/3200017886 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0806c5e0 con 0x7f4b080709b0 2024-03-29T08:20:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06000640 1 -- 172.21.15.46:0/3200017886 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f4b00001a00 con 0x7f4b080709b0 2024-03-29T08:20:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06000640 1 -- 172.21.15.46:0/3200017886 >> v1:172.21.15.46:6789/0 conn(0x7f4b080709b0 legacy=0x7f4b0815bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06000640 1 --2- 172.21.15.46:0/3200017886 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4aec003b10 0x7f4aec003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06000640 1 --2- 172.21.15.46:0/3200017886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aec0047d0 0x7f4aec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06000640 1 --2- 172.21.15.46:0/3200017886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aec005200 0x7f4aec00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06000640 1 -- 172.21.15.46:0/3200017886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080afff0 con 0x7f4aec005200 2024-03-29T08:20:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06000640 1 -- 172.21.15.46:0/3200017886 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec00c160 con 0x7f4aec0047d0 2024-03-29T08:20:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06000640 1 -- 172.21.15.46:0/3200017886 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec00c320 con 0x7f4aec003b10 2024-03-29T08:20:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b07e00640 1 --2- 172.21.15.46:0/3200017886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aec005200 0x7f4aec00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b07400640 1 --2- 172.21.15.46:0/3200017886 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4aec003b10 0x7f4aec003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b06a00640 1 --2- 172.21.15.46:0/3200017886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aec0047d0 0x7f4aec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:11.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b07400640 1 -- 172.21.15.46:0/3200017886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aec0047d0 msgr2=0x7f4aec004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:11.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b07400640 1 --2- 172.21.15.46:0/3200017886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aec0047d0 0x7f4aec004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:11.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b07400640 1 -- 172.21.15.46:0/3200017886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aec005200 msgr2=0x7f4aec00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:11.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b07400640 1 --2- 172.21.15.46:0/3200017886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aec005200 0x7f4aec00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:11.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b07400640 1 -- 172.21.15.46:0/3200017886 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec003530 con 0x7f4aec003b10 2024-03-29T08:20:11.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b07e00640 1 --2- 172.21.15.46:0/3200017886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4aec005200 0x7f4aec00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:20:11.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.163+0000 7f4b07e00640 1 -- 172.21.15.46:0/3200017886 reap_dead start 2024-03-29T08:20:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.167+0000 7f4b07400640 1 --2- 172.21.15.46:0/3200017886 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4aec003b10 0x7f4aec003f10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009ac0 tx=0x7f4af801ca20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73657ef74b5118a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.167+0000 7f4b06000640 1 -- 172.21.15.46:0/3200017886 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af80a5610 con 0x7f4aec003b10 2024-03-29T08:20:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.167+0000 7f4b06000640 1 -- 172.21.15.46:0/3200017886 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af80a5770 con 0x7f4aec003b10 2024-03-29T08:20:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.167+0000 7f4b06000640 1 -- 172.21.15.46:0/3200017886 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af80a5b00 con 0x7f4aec003b10 2024-03-29T08:20:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.167+0000 7f4b0e600640 1 -- 172.21.15.46:0/3200017886 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4aec003b10 msgr2=0x7f4aec003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.167+0000 7f4b0e600640 1 --2- 172.21.15.46:0/3200017886 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4aec003b10 0x7f4aec003f10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009ac0 tx=0x7f4af801ca20 comp rx=0 tx=0).stop 2024-03-29T08:20:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.167+0000 7f4b0e600640 1 -- 172.21.15.46:0/3200017886 shutdown_connections 2024-03-29T08:20:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.167+0000 7f4b0e600640 1 --2- 172.21.15.46:0/3200017886 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4aec003b10 0x7f4aec003f10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.167+0000 7f4b0e600640 1 -- 172.21.15.46:0/3200017886 >> 172.21.15.46:0/3200017886 conn(0x7f4b08066080 msgr2=0x7f4b08069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.167+0000 7f4b0e600640 1 -- 172.21.15.46:0/3200017886 shutdown_connections 2024-03-29T08:20:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.167+0000 7f4b0e600640 1 -- 172.21.15.46:0/3200017886 wait complete. 2024-03-29T08:20:11.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b0e600640 1 Processor -- start 2024-03-29T08:20:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b0e600640 1 -- start start 2024-03-29T08:20:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b0e600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aec003b10 0x7f4b080c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b0e600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b081405e0 0x7f4b081e72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b0e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b080cd890 0x7f4b080cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b0e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080ce280 con 0x7f4b080cd890 2024-03-29T08:20:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b0e600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08141580 con 0x7f4aec003b10 2024-03-29T08:20:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b0e600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080c4f30 con 0x7f4b081405e0 2024-03-29T08:20:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b080cd890 0x7f4b080cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b080cd890 0x7f4b080cdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37798/0 (socket says 172.21.15.46:37798) 2024-03-29T08:20:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b07e00640 1 -- 172.21.15.46:0/904341628 learned_addr learned my addr 172.21.15.46:0/904341628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b07400640 1 --2- 172.21.15.46:0/904341628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aec003b10 0x7f4b080c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b06a00640 1 --2- 172.21.15.46:0/904341628 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b081405e0 0x7f4b081e72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b07e00640 1 -- 172.21.15.46:0/904341628 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b081405e0 msgr2=0x7f4b081e72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b07e00640 1 --2- 172.21.15.46:0/904341628 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b081405e0 0x7f4b081e72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:11.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b07e00640 1 -- 172.21.15.46:0/904341628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aec003b10 msgr2=0x7f4b080c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b07e00640 1 --2- 172.21.15.46:0/904341628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aec003b10 0x7f4b080c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b07e00640 1 -- 172.21.15.46:0/904341628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b08140a60 con 0x7f4b080cd890 2024-03-29T08:20:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b07e00640 1 --2- 172.21.15.46:0/904341628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b080cd890 0x7f4b080cdd40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4b080a7550 tx=0x7f4b000074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4aff400640 1 -- 172.21.15.46:0/904341628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00015040 con 0x7f4b080cd890 2024-03-29T08:20:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4aff400640 1 -- 172.21.15.46:0/904341628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b00016650 con 0x7f4b080cd890 2024-03-29T08:20:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b0e600640 1 -- 172.21.15.46:0/904341628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b080c4320 con 0x7f4b080cd890 2024-03-29T08:20:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4aff400640 1 -- 172.21.15.46:0/904341628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b000169e0 con 0x7f4b080cd890 2024-03-29T08:20:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.171+0000 7f4b0e600640 1 -- 172.21.15.46:0/904341628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b080c48e0 con 0x7f4b080cd890 2024-03-29T08:20:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.175+0000 7f4b0e600640 1 -- 172.21.15.46:0/904341628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b080c50c0 con 0x7f4b080cd890 2024-03-29T08:20:11.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.175+0000 7f4aff400640 1 -- 172.21.15.46:0/904341628 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4b0000e0d0 con 0x7f4b080cd890 2024-03-29T08:20:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.175+0000 7f4aff400640 1 --2- 172.21.15.46:0/904341628 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ae80666c0 0x7f4ae8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.175+0000 7f4aff400640 1 -- 172.21.15.46:0/904341628 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f4b00088780 con 0x7f4b080cd890 2024-03-29T08:20:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.175+0000 7f4b07400640 1 --2- 172.21.15.46:0/904341628 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ae80666c0 0x7f4ae8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.179+0000 7f4b07400640 1 --2- 172.21.15.46:0/904341628 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ae80666c0 0x7f4ae8068b80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4af80af000 tx=0x7f4af80046d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:11.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.179+0000 7f4aff400640 1 -- 172.21.15.46:0/904341628 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4b00055cb0 con 0x7f4b080cd890 2024-03-29T08:20:11.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.339+0000 7f4b0e600640 1 -- 172.21.15.46:0/904341628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4b08071a90 con 0x7f4b080cd890 2024-03-29T08:20:11.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:11.991+0000 7f4aff400640 1 -- 172.21.15.46:0/904341628 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v47) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4b00028020 con 0x7f4b080cd890 2024-03-29T08:20:11.995 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-03-29T08:20:12.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.003+0000 7f4b0e600640 1 -- 172.21.15.46:0/904341628 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ae80666c0 msgr2=0x7f4ae8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.003+0000 7f4b0e600640 1 --2- 172.21.15.46:0/904341628 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ae80666c0 0x7f4ae8068b80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4af80af000 tx=0x7f4af80046d0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.003+0000 7f4b0e600640 1 -- 172.21.15.46:0/904341628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b080cd890 msgr2=0x7f4b080cdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.003+0000 7f4b0e600640 1 --2- 172.21.15.46:0/904341628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b080cd890 0x7f4b080cdd40 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4b080a7550 tx=0x7f4b000074f0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.003+0000 7f4b0e600640 1 -- 172.21.15.46:0/904341628 shutdown_connections 2024-03-29T08:20:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.003+0000 7f4b0e600640 1 --2- 172.21.15.46:0/904341628 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ae80666c0 0x7f4ae8068b80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.003+0000 7f4b0e600640 1 --2- 172.21.15.46:0/904341628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b080cd890 0x7f4b080cdd40 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.003+0000 7f4b0e600640 1 --2- 172.21.15.46:0/904341628 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b081405e0 0x7f4b081e72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.003+0000 7f4b0e600640 1 --2- 172.21.15.46:0/904341628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4aec003b10 0x7f4b080c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.003+0000 7f4b0e600640 1 -- 172.21.15.46:0/904341628 >> 172.21.15.46:0/904341628 conn(0x7f4b08066080 msgr2=0x7f4b08159e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:12.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.007+0000 7f4b0e600640 1 -- 172.21.15.46:0/904341628 shutdown_connections 2024-03-29T08:20:12.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.007+0000 7f4b0e600640 1 -- 172.21.15.46:0/904341628 wait complete. 2024-03-29T08:20:12.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T08:20:12.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.179+0000 7f4e07600640 1 Processor -- start 2024-03-29T08:20:12.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.179+0000 7f4e07600640 1 -- start start 2024-03-29T08:20:12.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.183+0000 7f4e07600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e000cbea0 0x7f4e000b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:12.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.183+0000 7f4e07600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e0015efe0 0x7f4e000b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:12.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.183+0000 7f4e07600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e000b20b0 con 0x7f4e00143eb0 2024-03-29T08:20:12.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.183+0000 7f4e07600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e000b2360 con 0x7f4e000cbea0 2024-03-29T08:20:12.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.183+0000 7f4e07600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0015e500 con 0x7f4e0015efe0 2024-03-29T08:20:12.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.183+0000 7f4df7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e000cbea0 0x7f4e000b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:12.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.183+0000 7f4df7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4e000cbea0 0x7f4e000b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37802/0 (socket says 172.21.15.46:37802) 2024-03-29T08:20:12.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.183+0000 7f4df7e00640 1 -- 172.21.15.46:0/160153996 learned_addr learned my addr 172.21.15.46:0/160153996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:12.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.183+0000 7f4e05600640 1 --2- 172.21.15.46:0/160153996 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e0015efe0 0x7f4e000b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:12.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.183+0000 7f4df7e00640 1 -- 172.21.15.46:0/160153996 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e0015efe0 msgr2=0x7f4e000b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:12.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.183+0000 7f4df7e00640 1 --2- 172.21.15.46:0/160153996 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e0015efe0 0x7f4e000b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.183+0000 7f4df7e00640 1 -- 172.21.15.46:0/160153996 >> v1:172.21.15.73:6789/0 conn(0x7f4e00143eb0 legacy=0x7f4e000cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.183+0000 7f4df7e00640 1 -- 172.21.15.46:0/160153996 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0015e7e0 con 0x7f4e000cbea0 2024-03-29T08:20:12.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.183+0000 7f4dffe00640 1 -- 172.21.15.46:0/160153996 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440821057 0 0) 0x7f4e000b20b0 con 0x7f4e00143eb0 2024-03-29T08:20:12.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.187+0000 7f4df7e00640 1 --2- 172.21.15.46:0/160153996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e000cbea0 0x7f4e000b1630 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009870 tx=0x7f4de801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b6b497a33a56f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:12.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.187+0000 7f4dffe00640 1 -- 172.21.15.46:0/160153996 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de80aa430 con 0x7f4e000cbea0 2024-03-29T08:20:12.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.187+0000 7f4dffe00640 1 -- 172.21.15.46:0/160153996 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de80aa590 con 0x7f4e000cbea0 2024-03-29T08:20:12.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.187+0000 7f4dffe00640 1 -- 172.21.15.46:0/160153996 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de80aa920 con 0x7f4e000cbea0 2024-03-29T08:20:12.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.187+0000 7f4e07600640 1 -- 172.21.15.46:0/160153996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e000cbea0 msgr2=0x7f4e000b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.187+0000 7f4e07600640 1 --2- 172.21.15.46:0/160153996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e000cbea0 0x7f4e000b1630 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009870 tx=0x7f4de801c530 comp rx=0 tx=0).stop 2024-03-29T08:20:12.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.187+0000 7f4e07600640 1 -- 172.21.15.46:0/160153996 shutdown_connections 2024-03-29T08:20:12.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.187+0000 7f4e07600640 1 --2- 172.21.15.46:0/160153996 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e0015efe0 0x7f4e000b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.187+0000 7f4e07600640 1 --2- 172.21.15.46:0/160153996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e000cbea0 0x7f4e000b1630 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.187+0000 7f4e07600640 1 -- 172.21.15.46:0/160153996 >> 172.21.15.46:0/160153996 conn(0x7f4e0006a390 msgr2=0x7f4e000cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.187+0000 7f4e07600640 1 -- 172.21.15.46:0/160153996 shutdown_connections 2024-03-29T08:20:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.187+0000 7f4e07600640 1 -- 172.21.15.46:0/160153996 wait complete. 2024-03-29T08:20:12.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.187+0000 7f4e07600640 1 Processor -- start 2024-03-29T08:20:12.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.187+0000 7f4e07600640 1 -- start start 2024-03-29T08:20:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e07600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e000cbea0 0x7f4e0006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e07600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e00143eb0 0x7f4e00137b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e07600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e0015efe0 0x7f4e00138040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e07600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e00142430 con 0x7f4e000cbea0 2024-03-29T08:20:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e07600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0013efe0 con 0x7f4e00143eb0 2024-03-29T08:20:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e07600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e00141b00 con 0x7f4e0015efe0 2024-03-29T08:20:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e04c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e000cbea0 0x7f4e0006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e04c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e000cbea0 0x7f4e0006c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37814/0 (socket says 172.21.15.46:37814) 2024-03-29T08:20:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e04c00640 1 -- 172.21.15.46:0/2083865425 learned_addr learned my addr 172.21.15.46:0/2083865425 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4df7e00640 1 --2- 172.21.15.46:0/2083865425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e00143eb0 0x7f4e00137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e05600640 1 --2- 172.21.15.46:0/2083865425 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e0015efe0 0x7f4e00138040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:12.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e04c00640 1 -- 172.21.15.46:0/2083865425 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e0015efe0 msgr2=0x7f4e00138040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e04c00640 1 --2- 172.21.15.46:0/2083865425 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e0015efe0 0x7f4e00138040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e04c00640 1 -- 172.21.15.46:0/2083865425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e00143eb0 msgr2=0x7f4e00137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e04c00640 1 --2- 172.21.15.46:0/2083865425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e00143eb0 0x7f4e00137b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e04c00640 1 -- 172.21.15.46:0/2083865425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de80094d0 con 0x7f4e000cbea0 2024-03-29T08:20:12.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e04c00640 1 --2- 172.21.15.46:0/2083865425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e000cbea0 0x7f4e0006c5d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4df000c4d0 tx=0x7f4df000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4dfd600640 1 -- 172.21.15.46:0/2083865425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df0005480 con 0x7f4e000cbea0 2024-03-29T08:20:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4dfd600640 1 -- 172.21.15.46:0/2083865425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df0003600 con 0x7f4e000cbea0 2024-03-29T08:20:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e07600640 1 -- 172.21.15.46:0/2083865425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e00141de0 con 0x7f4e000cbea0 2024-03-29T08:20:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4dfd600640 1 -- 172.21.15.46:0/2083865425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df00039b0 con 0x7f4e000cbea0 2024-03-29T08:20:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e07600640 1 -- 172.21.15.46:0/2083865425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0013ec70 con 0x7f4e000cbea0 2024-03-29T08:20:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.191+0000 7f4e07600640 1 -- 172.21.15.46:0/2083865425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e00071a70 con 0x7f4e000cbea0 2024-03-29T08:20:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.195+0000 7f4dfd600640 1 -- 172.21.15.46:0/2083865425 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4df0003b10 con 0x7f4e000cbea0 2024-03-29T08:20:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.195+0000 7f4dfd600640 1 --2- 172.21.15.46:0/2083865425 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ddc0666c0 0x7f4ddc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.195+0000 7f4dfd600640 1 -- 172.21.15.46:0/2083865425 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f4df0083670 con 0x7f4e000cbea0 2024-03-29T08:20:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.195+0000 7f4df7e00640 1 --2- 172.21.15.46:0/2083865425 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ddc0666c0 0x7f4ddc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.195+0000 7f4df7e00640 1 --2- 172.21.15.46:0/2083865425 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ddc0666c0 0x7f4ddc068b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4de80ae000 tx=0x7f4de80a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:12.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.199+0000 7f4dfd600640 1 -- 172.21.15.46:0/2083865425 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4df0050bb0 con 0x7f4e000cbea0 2024-03-29T08:20:12.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.395+0000 7f4e07600640 1 -- 172.21.15.46:0/2083865425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4e0004e120 con 0x7f4e000cbea0 2024-03-29T08:20:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.403+0000 7f4dfd600640 1 -- 172.21.15.46:0/2083865425 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4df0017020 con 0x7f4e000cbea0 2024-03-29T08:20:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 6 and data pool 7 2024-03-29T08:20:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.407+0000 7f4e07600640 1 -- 172.21.15.46:0/2083865425 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ddc0666c0 msgr2=0x7f4ddc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.407+0000 7f4e07600640 1 --2- 172.21.15.46:0/2083865425 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ddc0666c0 0x7f4ddc068b80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4de80ae000 tx=0x7f4de80a7000 comp rx=0 tx=0).stop 2024-03-29T08:20:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.407+0000 7f4e07600640 1 -- 172.21.15.46:0/2083865425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e000cbea0 msgr2=0x7f4e0006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.407+0000 7f4e07600640 1 --2- 172.21.15.46:0/2083865425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e000cbea0 0x7f4e0006c5d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4df000c4d0 tx=0x7f4df000c990 comp rx=0 tx=0).stop 2024-03-29T08:20:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.411+0000 7f4e07600640 1 -- 172.21.15.46:0/2083865425 shutdown_connections 2024-03-29T08:20:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.411+0000 7f4e07600640 1 --2- 172.21.15.46:0/2083865425 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ddc0666c0 0x7f4ddc068b80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.411+0000 7f4e07600640 1 --2- 172.21.15.46:0/2083865425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e00143eb0 0x7f4e00137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.411+0000 7f4e07600640 1 --2- 172.21.15.46:0/2083865425 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4e0015efe0 0x7f4e00138040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.411+0000 7f4e07600640 1 --2- 172.21.15.46:0/2083865425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e000cbea0 0x7f4e0006c5d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.411+0000 7f4e07600640 1 -- 172.21.15.46:0/2083865425 >> 172.21.15.46:0/2083865425 conn(0x7f4e0006a390 msgr2=0x7f4e000cd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.415+0000 7f4e07600640 1 -- 172.21.15.46:0/2083865425 shutdown_connections 2024-03-29T08:20:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.415+0000 7f4e07600640 1 -- 172.21.15.46:0/2083865425 wait complete. 2024-03-29T08:20:12.448 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:20:12.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.587+0000 7f9747200640 1 Processor -- start 2024-03-29T08:20:12.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.587+0000 7f9747200640 1 -- start start 2024-03-29T08:20:12.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.587+0000 7f9747200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f974015a940 0x7f97400afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:12.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.587+0000 7f9747200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97400b00d0 con 0x7f974015ad20 2024-03-29T08:20:12.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.587+0000 7f9747200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97400b0ab0 con 0x7f97400cf3c0 2024-03-29T08:20:12.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.587+0000 7f9747200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97400b0d90 con 0x7f974015a940 2024-03-29T08:20:12.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.587+0000 7f973fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f974015a940 0x7f97400afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:12.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.587+0000 7f9737400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f974015ad20 0x7f97400cecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33576/0 (socket says 172.21.15.46:33576) 2024-03-29T08:20:12.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.587+0000 7f9737400640 1 -- 172.21.15.46:0/535356930 learned_addr learned my addr 172.21.15.46:0/535356930 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:12.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.587+0000 7f973fe00640 1 -- 172.21.15.46:0/535356930 >> v1:172.21.15.73:6789/0 conn(0x7f97400cf3c0 legacy=0x7f97400cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:12.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.587+0000 7f973fe00640 1 -- 172.21.15.46:0/535356930 >> v1:172.21.15.46:6789/0 conn(0x7f974015ad20 legacy=0x7f97400cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.587+0000 7f973fe00640 1 -- 172.21.15.46:0/535356930 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97400b1070 con 0x7f974015a940 2024-03-29T08:20:12.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.587+0000 7f973f400640 1 -- 172.21.15.46:0/535356930 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1618300379 0 0) 0x7f97400b00d0 con 0x7f974015ad20 2024-03-29T08:20:12.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.591+0000 7f973fe00640 1 --2- 172.21.15.46:0/535356930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974015a940 0x7f97400afb90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f97300097c0 tx=0x7f973001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21940060712eb52 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:12.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.591+0000 7f973f400640 1 -- 172.21.15.46:0/535356930 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97300ac600 con 0x7f974015a940 2024-03-29T08:20:12.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.591+0000 7f973f400640 1 -- 172.21.15.46:0/535356930 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97300a8050 con 0x7f974015a940 2024-03-29T08:20:12.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.591+0000 7f973f400640 1 -- 172.21.15.46:0/535356930 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97300ac980 con 0x7f974015a940 2024-03-29T08:20:12.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.591+0000 7f9747200640 1 -- 172.21.15.46:0/535356930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974015a940 msgr2=0x7f97400afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.591+0000 7f9747200640 1 --2- 172.21.15.46:0/535356930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974015a940 0x7f97400afb90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f97300097c0 tx=0x7f973001c3a0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.591+0000 7f9747200640 1 -- 172.21.15.46:0/535356930 shutdown_connections 2024-03-29T08:20:12.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.591+0000 7f9747200640 1 --2- 172.21.15.46:0/535356930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974015a940 0x7f97400afb90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.591+0000 7f9747200640 1 -- 172.21.15.46:0/535356930 >> 172.21.15.46:0/535356930 conn(0x7f9740089fd0 msgr2=0x7f974008a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:12.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.591+0000 7f9747200640 1 -- 172.21.15.46:0/535356930 shutdown_connections 2024-03-29T08:20:12.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.591+0000 7f9747200640 1 -- 172.21.15.46:0/535356930 wait complete. 2024-03-29T08:20:12.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9747200640 1 Processor -- start 2024-03-29T08:20:12.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9747200640 1 -- start start 2024-03-29T08:20:12.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9747200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97400cf3c0 0x7f9740155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:12.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9747200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974015a940 0x7f97400c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:12.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9747200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f974015ad20 0x7f97400c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:12.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9747200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9740070180 con 0x7f974015a940 2024-03-29T08:20:12.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9747200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97400ae5e0 con 0x7f97400cf3c0 2024-03-29T08:20:12.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9747200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974006ced0 con 0x7f974015ad20 2024-03-29T08:20:12.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9737400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974015a940 0x7f97400c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:12.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9737400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974015a940 0x7f97400c63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37844/0 (socket says 172.21.15.46:37844) 2024-03-29T08:20:12.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9737400640 1 -- 172.21.15.46:0/3825840798 learned_addr learned my addr 172.21.15.46:0/3825840798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:12.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f973fe00640 1 --2- 172.21.15.46:0/3825840798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97400cf3c0 0x7f9740155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:12.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9745200640 1 --2- 172.21.15.46:0/3825840798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f974015ad20 0x7f97400c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:12.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9737400640 1 -- 172.21.15.46:0/3825840798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f974015ad20 msgr2=0x7f97400c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9737400640 1 --2- 172.21.15.46:0/3825840798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f974015ad20 0x7f97400c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9737400640 1 -- 172.21.15.46:0/3825840798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97400cf3c0 msgr2=0x7f9740155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9737400640 1 --2- 172.21.15.46:0/3825840798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97400cf3c0 0x7f9740155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9737400640 1 -- 172.21.15.46:0/3825840798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9730009420 con 0x7f974015a940 2024-03-29T08:20:12.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9745200640 1 --2- 172.21.15.46:0/3825840798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f974015ad20 0x7f97400c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:12.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9737400640 1 --2- 172.21.15.46:0/3825840798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974015a940 0x7f97400c63d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f972800c4b0 tx=0x7f972800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:12.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f973cc00640 1 -- 172.21.15.46:0/3825840798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97280074e0 con 0x7f974015a940 2024-03-29T08:20:12.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f973cc00640 1 -- 172.21.15.46:0/3825840798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9728007660 con 0x7f974015a940 2024-03-29T08:20:12.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f973cc00640 1 -- 172.21.15.46:0/3825840798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9728004a30 con 0x7f974015a940 2024-03-29T08:20:12.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9747200640 1 -- 172.21.15.46:0/3825840798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974006cbf0 con 0x7f974015a940 2024-03-29T08:20:12.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9747200640 1 -- 172.21.15.46:0/3825840798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9740165fe0 con 0x7f974015a940 2024-03-29T08:20:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.595+0000 7f9747200640 1 -- 172.21.15.46:0/3825840798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9740166200 con 0x7f974015a940 2024-03-29T08:20:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.599+0000 7f973cc00640 1 -- 172.21.15.46:0/3825840798 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9728004b90 con 0x7f974015a940 2024-03-29T08:20:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.599+0000 7f973cc00640 1 --2- 172.21.15.46:0/3825840798 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f971c066690 0x7f971c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.599+0000 7f973cc00640 1 -- 172.21.15.46:0/3825840798 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9728083260 con 0x7f974015a940 2024-03-29T08:20:12.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.599+0000 7f973fe00640 1 --2- 172.21.15.46:0/3825840798 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f971c066690 0x7f971c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:12.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.599+0000 7f973fe00640 1 --2- 172.21.15.46:0/3825840798 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f971c066690 0x7f971c068b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f97300ae000 tx=0x7f97300a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:12.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.603+0000 7f973cc00640 1 -- 172.21.15.46:0/3825840798 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9728054100 con 0x7f974015a940 2024-03-29T08:20:12.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.767+0000 7f9747200640 1 -- 172.21.15.46:0/3825840798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9740066c90 con 0x7f974015a940 2024-03-29T08:20:12.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.767+0000 7f973cc00640 1 -- 172.21.15.46:0/3825840798 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16006 (secure 0 0 0) 0x7f9728053c90 con 0x7f974015a940 2024-03-29T08:20:12.770 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:12.770 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":48,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:20:12.399799+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:20:10.906619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-29T08:20:11.389667+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:20:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.771+0000 7f9747200640 1 -- 172.21.15.46:0/3825840798 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f971c066690 msgr2=0x7f971c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.771+0000 7f9747200640 1 --2- 172.21.15.46:0/3825840798 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f971c066690 0x7f971c068b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f97300ae000 tx=0x7f97300a7000 comp rx=0 tx=0).stop 2024-03-29T08:20:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.771+0000 7f9747200640 1 -- 172.21.15.46:0/3825840798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974015a940 msgr2=0x7f97400c63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.771+0000 7f9747200640 1 --2- 172.21.15.46:0/3825840798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974015a940 0x7f97400c63d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f972800c4b0 tx=0x7f972800c970 comp rx=0 tx=0).stop 2024-03-29T08:20:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.771+0000 7f9747200640 1 -- 172.21.15.46:0/3825840798 shutdown_connections 2024-03-29T08:20:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.771+0000 7f9747200640 1 --2- 172.21.15.46:0/3825840798 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f971c066690 0x7f971c068b50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.771+0000 7f9747200640 1 --2- 172.21.15.46:0/3825840798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f974015ad20 0x7f97400c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.771+0000 7f9747200640 1 --2- 172.21.15.46:0/3825840798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f974015a940 0x7f97400c63d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.771+0000 7f9747200640 1 --2- 172.21.15.46:0/3825840798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97400cf3c0 0x7f9740155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.771+0000 7f9747200640 1 -- 172.21.15.46:0/3825840798 >> 172.21.15.46:0/3825840798 conn(0x7f9740089fd0 msgr2=0x7f97400cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:12.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.771+0000 7f9747200640 1 -- 172.21.15.46:0/3825840798 shutdown_connections 2024-03-29T08:20:12.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.771+0000 7f9747200640 1 -- 172.21.15.46:0/3825840798 wait complete. 2024-03-29T08:20:12.791 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:20:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.931+0000 7f5c5d600640 1 Processor -- start 2024-03-29T08:20:12.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.931+0000 7f5c5d600640 1 -- start start 2024-03-29T08:20:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.931+0000 7f5c5d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c5815a890 0x7f5c58098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.931+0000 7f5c5d600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c580cc2e0 con 0x7f5c58099690 2024-03-29T08:20:12.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.931+0000 7f5c5d600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c580cc580 con 0x7f5c580b2b30 2024-03-29T08:20:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.931+0000 7f5c5d600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c580cce20 con 0x7f5c5815a890 2024-03-29T08:20:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.931+0000 7f5c57e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c5815a890 0x7f5c58098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.931+0000 7f5c5cc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5c58099690 0x7f5c580b0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33590/0 (socket says 172.21.15.46:33590) 2024-03-29T08:20:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.931+0000 7f5c5cc00640 1 -- 172.21.15.46:0/4934210 learned_addr learned my addr 172.21.15.46:0/4934210 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.931+0000 7f5c57e00640 1 -- 172.21.15.46:0/4934210 >> v1:172.21.15.73:6789/0 conn(0x7f5c580b2b30 legacy=0x7f5c58098f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.931+0000 7f5c57e00640 1 -- 172.21.15.46:0/4934210 >> v1:172.21.15.46:6789/0 conn(0x7f5c58099690 legacy=0x7f5c580b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.931+0000 7f5c57e00640 1 -- 172.21.15.46:0/4934210 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c580cc860 con 0x7f5c5815a890 2024-03-29T08:20:12.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.931+0000 7f5c57400640 1 -- 172.21.15.46:0/4934210 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332105275 0 0) 0x7f5c580cc2e0 con 0x7f5c58099690 2024-03-29T08:20:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.935+0000 7f5c57e00640 1 --2- 172.21.15.46:0/4934210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c5815a890 0x7f5c58098a40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5c480097c0 tx=0x7f5c4801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f4c6979c63402a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.935+0000 7f5c57400640 1 -- 172.21.15.46:0/4934210 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c480ab600 con 0x7f5c5815a890 2024-03-29T08:20:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.935+0000 7f5c57400640 1 -- 172.21.15.46:0/4934210 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c480a7050 con 0x7f5c5815a890 2024-03-29T08:20:12.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.935+0000 7f5c57400640 1 -- 172.21.15.46:0/4934210 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c480ab980 con 0x7f5c5815a890 2024-03-29T08:20:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.935+0000 7f5c5d600640 1 -- 172.21.15.46:0/4934210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c5815a890 msgr2=0x7f5c58098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.935+0000 7f5c5d600640 1 --2- 172.21.15.46:0/4934210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c5815a890 0x7f5c58098a40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5c480097c0 tx=0x7f5c4801c3a0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.935+0000 7f5c5d600640 1 -- 172.21.15.46:0/4934210 shutdown_connections 2024-03-29T08:20:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.935+0000 7f5c5d600640 1 --2- 172.21.15.46:0/4934210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c5815a890 0x7f5c58098a40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.935+0000 7f5c5d600640 1 -- 172.21.15.46:0/4934210 >> 172.21.15.46:0/4934210 conn(0x7f5c580b2010 msgr2=0x7f5c580b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.935+0000 7f5c5d600640 1 -- 172.21.15.46:0/4934210 shutdown_connections 2024-03-29T08:20:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.935+0000 7f5c5d600640 1 -- 172.21.15.46:0/4934210 wait complete. 2024-03-29T08:20:12.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.935+0000 7f5c5d600640 1 Processor -- start 2024-03-29T08:20:12.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.935+0000 7f5c5d600640 1 -- start start 2024-03-29T08:20:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c5d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c58099690 0x7f5c58162e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c5d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c580b2b30 0x7f5c58153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c5d600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c5815a890 0x7f5c5806f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c5d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c58163360 con 0x7f5c580b2b30 2024-03-29T08:20:12.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c5d600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c580cbf30 con 0x7f5c58099690 2024-03-29T08:20:12.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c5d600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c58162b40 con 0x7f5c5815a890 2024-03-29T08:20:12.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c4f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c580b2b30 0x7f5c58153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:12.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c4f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c580b2b30 0x7f5c58153d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37868/0 (socket says 172.21.15.46:37868) 2024-03-29T08:20:12.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c4f400640 1 -- 172.21.15.46:0/502313780 learned_addr learned my addr 172.21.15.46:0/502313780 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:12.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c5cc00640 1 --2- 172.21.15.46:0/502313780 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c5815a890 0x7f5c5806f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:12.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c57e00640 1 --2- 172.21.15.46:0/502313780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c58099690 0x7f5c58162e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:12.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c4f400640 1 -- 172.21.15.46:0/502313780 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c5815a890 msgr2=0x7f5c5806f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c4f400640 1 --2- 172.21.15.46:0/502313780 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c5815a890 0x7f5c5806f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c4f400640 1 -- 172.21.15.46:0/502313780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c58099690 msgr2=0x7f5c58162e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:12.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c4f400640 1 --2- 172.21.15.46:0/502313780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c58099690 0x7f5c58162e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:12.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c4f400640 1 -- 172.21.15.46:0/502313780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c48009420 con 0x7f5c580b2b30 2024-03-29T08:20:12.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c4f400640 1 --2- 172.21.15.46:0/502313780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c580b2b30 0x7f5c58153d00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000ba80 tx=0x7f5c4000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:12.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c54c00640 1 -- 172.21.15.46:0/502313780 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4000cc00 con 0x7f5c580b2b30 2024-03-29T08:20:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c54c00640 1 -- 172.21.15.46:0/502313780 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4000cd60 con 0x7f5c580b2b30 2024-03-29T08:20:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c54c00640 1 -- 172.21.15.46:0/502313780 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c40003bc0 con 0x7f5c580b2b30 2024-03-29T08:20:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c5d600640 1 -- 172.21.15.46:0/502313780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c580cb970 con 0x7f5c580b2b30 2024-03-29T08:20:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c5d600640 1 -- 172.21.15.46:0/502313780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c58154240 con 0x7f5c580b2b30 2024-03-29T08:20:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c5d600640 1 -- 172.21.15.46:0/502313780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c580b3060 con 0x7f5c580b2b30 2024-03-29T08:20:12.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.939+0000 7f5c54c00640 1 -- 172.21.15.46:0/502313780 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5c40014440 con 0x7f5c580b2b30 2024-03-29T08:20:12.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.943+0000 7f5c54c00640 1 --2- 172.21.15.46:0/502313780 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c340666c0 0x7f5c34068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:12.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.943+0000 7f5c54c00640 1 -- 172.21.15.46:0/502313780 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c40083fa0 con 0x7f5c580b2b30 2024-03-29T08:20:12.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.943+0000 7f5c57e00640 1 --2- 172.21.15.46:0/502313780 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c340666c0 0x7f5c34068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:12.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.943+0000 7f5c57e00640 1 --2- 172.21.15.46:0/502313780 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c340666c0 0x7f5c34068b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5c480ad000 tx=0x7f5c480a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:12.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:12.947+0000 7f5c54c00640 1 -- 172.21.15.46:0/502313780 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5c580b3060 con 0x7f5c580b2b30 2024-03-29T08:20:13.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.111+0000 7f5c5d600640 1 -- 172.21.15.46:0/502313780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c580b3060 con 0x7f5c580b2b30 2024-03-29T08:20:13.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.111+0000 7f5c54c00640 1 -- 172.21.15.46:0/502313780 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16006 (secure 0 0 0) 0x7f5c580b3060 con 0x7f5c580b2b30 2024-03-29T08:20:13.115 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:13.115 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":48,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:20:12.399799+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:20:10.906619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-29T08:20:11.389667+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:20:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.115+0000 7f5c5d600640 1 -- 172.21.15.46:0/502313780 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c340666c0 msgr2=0x7f5c34068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.115+0000 7f5c5d600640 1 --2- 172.21.15.46:0/502313780 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c340666c0 0x7f5c34068b80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5c480ad000 tx=0x7f5c480a6000 comp rx=0 tx=0).stop 2024-03-29T08:20:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.115+0000 7f5c5d600640 1 -- 172.21.15.46:0/502313780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c580b2b30 msgr2=0x7f5c58153d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.115+0000 7f5c5d600640 1 --2- 172.21.15.46:0/502313780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c580b2b30 0x7f5c58153d00 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f5c4000ba80 tx=0x7f5c4000bf40 comp rx=0 tx=0).stop 2024-03-29T08:20:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.115+0000 7f5c5d600640 1 -- 172.21.15.46:0/502313780 shutdown_connections 2024-03-29T08:20:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.115+0000 7f5c5d600640 1 --2- 172.21.15.46:0/502313780 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c340666c0 0x7f5c34068b80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.115+0000 7f5c5d600640 1 --2- 172.21.15.46:0/502313780 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c5815a890 0x7f5c5806f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.115+0000 7f5c5d600640 1 --2- 172.21.15.46:0/502313780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c580b2b30 0x7f5c58153d00 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.115+0000 7f5c5d600640 1 --2- 172.21.15.46:0/502313780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c58099690 0x7f5c58162e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.115+0000 7f5c5d600640 1 -- 172.21.15.46:0/502313780 >> 172.21.15.46:0/502313780 conn(0x7f5c580b2010 msgr2=0x7f5c580b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.115+0000 7f5c5d600640 1 -- 172.21.15.46:0/502313780 shutdown_connections 2024-03-29T08:20:13.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.115+0000 7f5c5d600640 1 -- 172.21.15.46:0/502313780 wait complete. 2024-03-29T08:20:13.136 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T08:20:13.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.275+0000 7fca34200640 1 Processor -- start 2024-03-29T08:20:13.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.275+0000 7fca34200640 1 -- start start 2024-03-29T08:20:13.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.279+0000 7fca34200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca2c15c9c0 0x7fca2c0ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:13.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.279+0000 7fca34200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca2c06be50 0x7fca2c098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:13.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.279+0000 7fca34200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2c0cfe50 con 0x7fca2c098f80 2024-03-29T08:20:13.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.279+0000 7fca34200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca2c0cf030 con 0x7fca2c06be50 2024-03-29T08:20:13.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.279+0000 7fca34200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c0993f0 con 0x7fca2c15c9c0 2024-03-29T08:20:13.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.279+0000 7fca30e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca2c06be50 0x7fca2c098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:13.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.279+0000 7fca30e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca2c06be50 0x7fca2c098a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37876/0 (socket says 172.21.15.46:37876) 2024-03-29T08:20:13.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.279+0000 7fca30e00640 1 -- 172.21.15.46:0/2385891096 learned_addr learned my addr 172.21.15.46:0/2385891096 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:13.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.279+0000 7fca31800640 1 --2- 172.21.15.46:0/2385891096 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca2c15c9c0 0x7fca2c0ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:13.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.279+0000 7fca30e00640 1 -- 172.21.15.46:0/2385891096 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca2c15c9c0 msgr2=0x7fca2c0ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:13.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.279+0000 7fca30e00640 1 --2- 172.21.15.46:0/2385891096 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca2c15c9c0 0x7fca2c0ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:13.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.279+0000 7fca30e00640 1 -- 172.21.15.46:0/2385891096 >> v1:172.21.15.73:6789/0 conn(0x7fca2c098f80 legacy=0x7fca2c1605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:13.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.279+0000 7fca30e00640 1 -- 172.21.15.46:0/2385891096 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c0cf240 con 0x7fca2c06be50 2024-03-29T08:20:13.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.283+0000 7fca30e00640 1 --2- 172.21.15.46:0/2385891096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2c06be50 0x7fca2c098a40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fca18009870 tx=0x7fca1801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbc21a7dd9e267f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:13.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.283+0000 7fca2be00640 1 -- 172.21.15.46:0/2385891096 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca180ab600 con 0x7fca2c06be50 2024-03-29T08:20:13.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.283+0000 7fca2be00640 1 -- 172.21.15.46:0/2385891096 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca180a7050 con 0x7fca2c06be50 2024-03-29T08:20:13.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.283+0000 7fca2be00640 1 -- 172.21.15.46:0/2385891096 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca180ab980 con 0x7fca2c06be50 2024-03-29T08:20:13.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.283+0000 7fca34200640 1 -- 172.21.15.46:0/2385891096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2c06be50 msgr2=0x7fca2c098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:13.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.283+0000 7fca34200640 1 --2- 172.21.15.46:0/2385891096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2c06be50 0x7fca2c098a40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fca18009870 tx=0x7fca1801c560 comp rx=0 tx=0).stop 2024-03-29T08:20:13.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.283+0000 7fca34200640 1 -- 172.21.15.46:0/2385891096 shutdown_connections 2024-03-29T08:20:13.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.283+0000 7fca34200640 1 --2- 172.21.15.46:0/2385891096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2c06be50 0x7fca2c098a40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:13.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.283+0000 7fca34200640 1 --2- 172.21.15.46:0/2385891096 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca2c15c9c0 0x7fca2c0ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:13.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.283+0000 7fca34200640 1 -- 172.21.15.46:0/2385891096 >> 172.21.15.46:0/2385891096 conn(0x7fca2c0b2220 msgr2=0x7fca2c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:13.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.283+0000 7fca34200640 1 -- 172.21.15.46:0/2385891096 shutdown_connections 2024-03-29T08:20:13.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.283+0000 7fca34200640 1 -- 172.21.15.46:0/2385891096 wait complete. 2024-03-29T08:20:13.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.283+0000 7fca34200640 1 Processor -- start 2024-03-29T08:20:13.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.283+0000 7fca34200640 1 -- start start 2024-03-29T08:20:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.283+0000 7fca34200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2c06be50 0x7fca2c09df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca34200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca2c098f80 0x7fca2c154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca34200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca2c15c9c0 0x7fca2c154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca34200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c156100 con 0x7fca2c06be50 2024-03-29T08:20:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca34200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c155300 con 0x7fca2c15c9c0 2024-03-29T08:20:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca34200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c155580 con 0x7fca2c098f80 2024-03-29T08:20:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca31800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2c06be50 0x7fca2c09df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:13.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca31800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2c06be50 0x7fca2c09df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37892/0 (socket says 172.21.15.46:37892) 2024-03-29T08:20:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca31800640 1 -- 172.21.15.46:0/4254725858 learned_addr learned my addr 172.21.15.46:0/4254725858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca30e00640 1 --2- 172.21.15.46:0/4254725858 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca2c098f80 0x7fca2c154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca32200640 1 --2- 172.21.15.46:0/4254725858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca2c15c9c0 0x7fca2c154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca31800640 1 -- 172.21.15.46:0/4254725858 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca2c098f80 msgr2=0x7fca2c154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca31800640 1 --2- 172.21.15.46:0/4254725858 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca2c098f80 0x7fca2c154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca31800640 1 -- 172.21.15.46:0/4254725858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca2c15c9c0 msgr2=0x7fca2c154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca31800640 1 --2- 172.21.15.46:0/4254725858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca2c15c9c0 0x7fca2c154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:13.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca31800640 1 -- 172.21.15.46:0/4254725858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca180094d0 con 0x7fca2c06be50 2024-03-29T08:20:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca31800640 1 --2- 172.21.15.46:0/4254725858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2c06be50 0x7fca2c09df20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fca2000d830 tx=0x7fca2000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca29600640 1 -- 172.21.15.46:0/4254725858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000b6c0 con 0x7fca2c06be50 2024-03-29T08:20:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca29600640 1 -- 172.21.15.46:0/4254725858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2000b820 con 0x7fca2c06be50 2024-03-29T08:20:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca29600640 1 -- 172.21.15.46:0/4254725858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000bb90 con 0x7fca2c06be50 2024-03-29T08:20:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca34200640 1 -- 172.21.15.46:0/4254725858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2c155860 con 0x7fca2c06be50 2024-03-29T08:20:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca34200640 1 -- 172.21.15.46:0/4254725858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2c155e20 con 0x7fca2c06be50 2024-03-29T08:20:13.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca34200640 1 -- 172.21.15.46:0/4254725858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2c09e700 con 0x7fca2c06be50 2024-03-29T08:20:13.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.287+0000 7fca29600640 1 -- 172.21.15.46:0/4254725858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fca2001e460 con 0x7fca2c06be50 2024-03-29T08:20:13.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.291+0000 7fca29600640 1 --2- 172.21.15.46:0/4254725858 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca0c066690 0x7fca0c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:13.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.291+0000 7fca29600640 1 -- 172.21.15.46:0/4254725858 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca200847f0 con 0x7fca2c06be50 2024-03-29T08:20:13.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.291+0000 7fca30e00640 1 --2- 172.21.15.46:0/4254725858 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca0c066690 0x7fca0c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:13.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.291+0000 7fca30e00640 1 --2- 172.21.15.46:0/4254725858 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca0c066690 0x7fca0c068b50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fca180ad000 tx=0x7fca18008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:13.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.295+0000 7fca29600640 1 -- 172.21.15.46:0/4254725858 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fca20051c00 con 0x7fca2c06be50 2024-03-29T08:20:13.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:13.491+0000 7fca34200640 1 -- 172.21.15.46:0/4254725858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fca2c099d50 con 0x7fca2c06be50 2024-03-29T08:20:13.547 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-03-29T08:20:13.984 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-03-29T08:20:13.983+0000 7ff544a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:20:14.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.403+0000 7fca29600640 1 -- 172.21.15.46:0/4254725858 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7fca20055220 con 0x7fca2c06be50 2024-03-29T08:20:14.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.407+0000 7fca34200640 1 -- 172.21.15.46:0/4254725858 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca0c066690 msgr2=0x7fca0c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:14.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.407+0000 7fca34200640 1 --2- 172.21.15.46:0/4254725858 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca0c066690 0x7fca0c068b50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fca180ad000 tx=0x7fca18008040 comp rx=0 tx=0).stop 2024-03-29T08:20:14.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.407+0000 7fca34200640 1 -- 172.21.15.46:0/4254725858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2c06be50 msgr2=0x7fca2c09df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:14.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.407+0000 7fca34200640 1 --2- 172.21.15.46:0/4254725858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2c06be50 0x7fca2c09df20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fca2000d830 tx=0x7fca2000dcf0 comp rx=0 tx=0).stop 2024-03-29T08:20:14.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.407+0000 7fca34200640 1 -- 172.21.15.46:0/4254725858 shutdown_connections 2024-03-29T08:20:14.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.407+0000 7fca34200640 1 --2- 172.21.15.46:0/4254725858 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca0c066690 0x7fca0c068b50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:14.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.407+0000 7fca34200640 1 --2- 172.21.15.46:0/4254725858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca2c15c9c0 0x7fca2c154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:14.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.407+0000 7fca34200640 1 --2- 172.21.15.46:0/4254725858 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca2c098f80 0x7fca2c154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:14.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.407+0000 7fca34200640 1 --2- 172.21.15.46:0/4254725858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca2c06be50 0x7fca2c09df20 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:14.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.407+0000 7fca34200640 1 -- 172.21.15.46:0/4254725858 >> 172.21.15.46:0/4254725858 conn(0x7fca2c0b2220 msgr2=0x7fca2c15e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:14.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.407+0000 7fca34200640 1 -- 172.21.15.46:0/4254725858 shutdown_connections 2024-03-29T08:20:14.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.407+0000 7fca34200640 1 -- 172.21.15.46:0/4254725858 wait complete. 2024-03-29T08:20:14.431 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-29T08:20:14.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T08:20:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.579+0000 7f9168400640 1 Processor -- start 2024-03-29T08:20:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.579+0000 7f9168400640 1 -- start start 2024-03-29T08:20:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.579+0000 7f9168400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f91600cea30 0x7f91601606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.579+0000 7f9168400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f916015cd30 0x7f9160160be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.579+0000 7f9168400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91600cfb20 con 0x7f916015c950 2024-03-29T08:20:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.579+0000 7f9168400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91600cef10 con 0x7f916015cd30 2024-03-29T08:20:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.579+0000 7f9168400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91600cf1f0 con 0x7f91600cea30 2024-03-29T08:20:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.579+0000 7f9166400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f916015cd30 0x7f9160160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.579+0000 7f9166400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f916015cd30 0x7f9160160be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37894/0 (socket says 172.21.15.46:37894) 2024-03-29T08:20:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.579+0000 7f9166400640 1 -- 172.21.15.46:0/3651316612 learned_addr learned my addr 172.21.15.46:0/3651316612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.579+0000 7f9165000640 1 --2- 172.21.15.46:0/3651316612 >> v2:172.21.15.73:3300/0 conn(0x7f91600cea30 0x7f91601606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:14.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.579+0000 7f9166400640 1 -- 172.21.15.46:0/3651316612 >> v2:172.21.15.73:3300/0 conn(0x7f91600cea30 msgr2=0x7f91601606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:14.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.579+0000 7f9166400640 1 --2- 172.21.15.46:0/3651316612 >> v2:172.21.15.73:3300/0 conn(0x7f91600cea30 0x7f91601606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:14.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.579+0000 7f9166400640 1 -- 172.21.15.46:0/3651316612 >> v1:172.21.15.73:6789/0 conn(0x7f916015c950 legacy=0x7f916006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:14.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.579+0000 7f9166400640 1 -- 172.21.15.46:0/3651316612 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91600cf4d0 con 0x7f916015cd30 2024-03-29T08:20:14.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.579+0000 7f9165000640 1 --2- 172.21.15.46:0/3651316612 >> v2:172.21.15.73:3300/0 conn(0x7f91600cea30 0x7f91601606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:14.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.583+0000 7f9166400640 1 --2- 172.21.15.46:0/3651316612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916015cd30 0x7f9160160be0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f91600c41c0 tx=0x7f915801f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1e836d28147cae server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.583+0000 7f915fe00640 1 -- 172.21.15.46:0/3651316612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91580af400 con 0x7f916015cd30 2024-03-29T08:20:14.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.583+0000 7f915fe00640 1 -- 172.21.15.46:0/3651316612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91580ab050 con 0x7f916015cd30 2024-03-29T08:20:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.583+0000 7f915fe00640 1 -- 172.21.15.46:0/3651316612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91580af780 con 0x7f916015cd30 2024-03-29T08:20:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.583+0000 7f9168400640 1 -- 172.21.15.46:0/3651316612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916015cd30 msgr2=0x7f9160160be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.583+0000 7f9168400640 1 --2- 172.21.15.46:0/3651316612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916015cd30 0x7f9160160be0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f91600c41c0 tx=0x7f915801f1a0 comp rx=0 tx=0).stop 2024-03-29T08:20:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.583+0000 7f9168400640 1 -- 172.21.15.46:0/3651316612 shutdown_connections 2024-03-29T08:20:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.583+0000 7f9168400640 1 --2- 172.21.15.46:0/3651316612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916015cd30 0x7f9160160be0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.583+0000 7f9168400640 1 --2- 172.21.15.46:0/3651316612 >> v2:172.21.15.73:3300/0 conn(0x7f91600cea30 0x7f91601606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.583+0000 7f9168400640 1 -- 172.21.15.46:0/3651316612 >> 172.21.15.46:0/3651316612 conn(0x7f91600b21b0 msgr2=0x7f91600b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.583+0000 7f9168400640 1 -- 172.21.15.46:0/3651316612 shutdown_connections 2024-03-29T08:20:14.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.583+0000 7f9168400640 1 -- 172.21.15.46:0/3651316612 wait complete. 2024-03-29T08:20:14.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.583+0000 7f9168400640 1 Processor -- start 2024-03-29T08:20:14.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.583+0000 7f9168400640 1 -- start start 2024-03-29T08:20:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9168400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f91600cea30 0x7f91600b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9168400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916015c950 0x7f9160153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9168400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f916015cd30 0x7f91600b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9168400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91600b3a30 con 0x7f916015c950 2024-03-29T08:20:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9168400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91600b3d10 con 0x7f916015cd30 2024-03-29T08:20:14.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9168400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91600b3ff0 con 0x7f91600cea30 2024-03-29T08:20:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9165000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916015c950 0x7f9160153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9165000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916015c950 0x7f9160153fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37908/0 (socket says 172.21.15.46:37908) 2024-03-29T08:20:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9165000640 1 -- 172.21.15.46:0/2102543821 learned_addr learned my addr 172.21.15.46:0/2102543821 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9165a00640 1 --2- 172.21.15.46:0/2102543821 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f91600cea30 0x7f91600b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9166400640 1 --2- 172.21.15.46:0/2102543821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f916015cd30 0x7f91600b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9165000640 1 -- 172.21.15.46:0/2102543821 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f91600cea30 msgr2=0x7f91600b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9165000640 1 --2- 172.21.15.46:0/2102543821 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f91600cea30 0x7f91600b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9165000640 1 -- 172.21.15.46:0/2102543821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f916015cd30 msgr2=0x7f91600b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9165000640 1 --2- 172.21.15.46:0/2102543821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f916015cd30 0x7f91600b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9165000640 1 -- 172.21.15.46:0/2102543821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9158009b30 con 0x7f916015c950 2024-03-29T08:20:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9165000640 1 --2- 172.21.15.46:0/2102543821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916015c950 0x7f9160153fb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f914c00c7f0 tx=0x7f914c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f915d600640 1 -- 172.21.15.46:0/2102543821 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c00cea0 con 0x7f916015c950 2024-03-29T08:20:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f915d600640 1 -- 172.21.15.46:0/2102543821 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914c007a30 con 0x7f916015c950 2024-03-29T08:20:14.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f915d600640 1 -- 172.21.15.46:0/2102543821 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c007dc0 con 0x7f916015c950 2024-03-29T08:20:14.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9168400640 1 -- 172.21.15.46:0/2102543821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916006c470 con 0x7f916015c950 2024-03-29T08:20:14.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9168400640 1 -- 172.21.15.46:0/2102543821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91600b45b0 con 0x7f916015c950 2024-03-29T08:20:14.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f9168400640 1 -- 172.21.15.46:0/2102543821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91600cff90 con 0x7f916015c950 2024-03-29T08:20:14.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.587+0000 7f915d600640 1 -- 172.21.15.46:0/2102543821 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f914c0113f0 con 0x7f916015c950 2024-03-29T08:20:14.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.591+0000 7f915d600640 1 --2- 172.21.15.46:0/2102543821 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9140066690 0x7f9140068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:14.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.591+0000 7f915d600640 1 -- 172.21.15.46:0/2102543821 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f914c084320 con 0x7f916015c950 2024-03-29T08:20:14.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.591+0000 7f9165a00640 1 --2- 172.21.15.46:0/2102543821 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9140066690 0x7f9140068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:14.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.591+0000 7f9165a00640 1 --2- 172.21.15.46:0/2102543821 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9140066690 0x7f9140068b50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9154004e30 tx=0x7f9154009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:14.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.595+0000 7f915d600640 1 -- 172.21.15.46:0/2102543821 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f91600cff90 con 0x7f916015c950 2024-03-29T08:20:14.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:14.787+0000 7f9168400640 1 -- 172.21.15.46:0/2102543821 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f91600cff90 con 0x7f916015c950 2024-03-29T08:20:15.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.407+0000 7f915d600640 1 -- 172.21.15.46:0/2102543821 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f91600cff90 con 0x7f916015c950 2024-03-29T08:20:15.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.407+0000 7f9168400640 1 -- 172.21.15.46:0/2102543821 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9140066690 msgr2=0x7f9140068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:15.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.407+0000 7f9168400640 1 --2- 172.21.15.46:0/2102543821 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9140066690 0x7f9140068b50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9154004e30 tx=0x7f9154009040 comp rx=0 tx=0).stop 2024-03-29T08:20:15.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.407+0000 7f9168400640 1 -- 172.21.15.46:0/2102543821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916015c950 msgr2=0x7f9160153fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:15.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.407+0000 7f9168400640 1 --2- 172.21.15.46:0/2102543821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916015c950 0x7f9160153fb0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f914c00c7f0 tx=0x7f914c00ccb0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.407+0000 7f9168400640 1 -- 172.21.15.46:0/2102543821 shutdown_connections 2024-03-29T08:20:15.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.407+0000 7f9168400640 1 --2- 172.21.15.46:0/2102543821 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9140066690 0x7f9140068b50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.407+0000 7f9168400640 1 --2- 172.21.15.46:0/2102543821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f916015cd30 0x7f91600b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.407+0000 7f9168400640 1 --2- 172.21.15.46:0/2102543821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f916015c950 0x7f9160153fb0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.407+0000 7f9168400640 1 --2- 172.21.15.46:0/2102543821 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f91600cea30 0x7f91600b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.407+0000 7f9168400640 1 -- 172.21.15.46:0/2102543821 >> 172.21.15.46:0/2102543821 conn(0x7f91600b21b0 msgr2=0x7f916015d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:15.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.407+0000 7f9168400640 1 -- 172.21.15.46:0/2102543821 shutdown_connections 2024-03-29T08:20:15.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.407+0000 7f9168400640 1 -- 172.21.15.46:0/2102543821 wait complete. 2024-03-29T08:20:15.433 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T08:20:15.433 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:20:15.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.575+0000 7f3c42c00640 1 Processor -- start 2024-03-29T08:20:15.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.575+0000 7f3c42c00640 1 -- start start 2024-03-29T08:20:15.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.575+0000 7f3c42c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3c3c1593d0 0x7f3c3c06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:15.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.575+0000 7f3c42c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3c3c15d230 0x7f3c3c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:15.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.575+0000 7f3c42c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c3c06d130 con 0x7f3c3c15dbd0 2024-03-29T08:20:15.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.575+0000 7f3c42c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c3c15b8f0 con 0x7f3c3c1593d0 2024-03-29T08:20:15.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.575+0000 7f3c42c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c15ad50 con 0x7f3c3c15d230 2024-03-29T08:20:15.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.575+0000 7f3c3be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3c3c1593d0 0x7f3c3c06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:15.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.575+0000 7f3c3be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3c3c1593d0 0x7f3c3c06cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37918/0 (socket says 172.21.15.46:37918) 2024-03-29T08:20:15.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.575+0000 7f3c3be00640 1 -- 172.21.15.46:0/3117990999 learned_addr learned my addr 172.21.15.46:0/3117990999 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:15.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.575+0000 7f3c33e00640 1 --2- 172.21.15.46:0/3117990999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3c3c15d230 0x7f3c3c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:15.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.575+0000 7f3c3be00640 1 -- 172.21.15.46:0/3117990999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3c3c15d230 msgr2=0x7f3c3c15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:15.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.575+0000 7f3c3be00640 1 --2- 172.21.15.46:0/3117990999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3c3c15d230 0x7f3c3c15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.575+0000 7f3c3be00640 1 -- 172.21.15.46:0/3117990999 >> v1:172.21.15.46:6789/0 conn(0x7f3c3c15dbd0 legacy=0x7f3c3c15bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:15.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.575+0000 7f3c3be00640 1 -- 172.21.15.46:0/3117990999 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3c15e060 con 0x7f3c3c1593d0 2024-03-29T08:20:15.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.575+0000 7f3c3b400640 1 -- 172.21.15.46:0/3117990999 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2723096586 0 0) 0x7f3c3c06d130 con 0x7f3c3c15dbd0 2024-03-29T08:20:15.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.579+0000 7f3c3be00640 1 --2- 172.21.15.46:0/3117990999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3c1593d0 0x7f3c3c06cbf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009870 tx=0x7f3c2c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af4e8417da5faeb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:15.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.579+0000 7f3c3b400640 1 -- 172.21.15.46:0/3117990999 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c0a9620 con 0x7f3c3c1593d0 2024-03-29T08:20:15.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.579+0000 7f3c3b400640 1 -- 172.21.15.46:0/3117990999 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c0a9780 con 0x7f3c3c1593d0 2024-03-29T08:20:15.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.579+0000 7f3c3b400640 1 -- 172.21.15.46:0/3117990999 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c0a9b10 con 0x7f3c3c1593d0 2024-03-29T08:20:15.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.579+0000 7f3c42c00640 1 -- 172.21.15.46:0/3117990999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3c1593d0 msgr2=0x7f3c3c06cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:15.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.579+0000 7f3c42c00640 1 --2- 172.21.15.46:0/3117990999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3c1593d0 0x7f3c3c06cbf0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c009870 tx=0x7f3c2c01c560 comp rx=0 tx=0).stop 2024-03-29T08:20:15.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.579+0000 7f3c42c00640 1 -- 172.21.15.46:0/3117990999 shutdown_connections 2024-03-29T08:20:15.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.579+0000 7f3c42c00640 1 --2- 172.21.15.46:0/3117990999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3c3c15d230 0x7f3c3c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.579+0000 7f3c42c00640 1 --2- 172.21.15.46:0/3117990999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3c1593d0 0x7f3c3c06cbf0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.579+0000 7f3c42c00640 1 -- 172.21.15.46:0/3117990999 >> 172.21.15.46:0/3117990999 conn(0x7f3c3c066110 msgr2=0x7f3c3c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:15.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.579+0000 7f3c42c00640 1 -- 172.21.15.46:0/3117990999 shutdown_connections 2024-03-29T08:20:15.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.579+0000 7f3c42c00640 1 -- 172.21.15.46:0/3117990999 wait complete. 2024-03-29T08:20:15.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c42c00640 1 Processor -- start 2024-03-29T08:20:15.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c42c00640 1 -- start start 2024-03-29T08:20:15.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c42c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c3c1593d0 0x7f3c3c0cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:15.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c42c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3c15d230 0x7f3c3c150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:15.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c42c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3c3c15dbd0 0x7f3c3c150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:15.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c42c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c14dcc0 con 0x7f3c3c15d230 2024-03-29T08:20:15.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c42c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c0cd860 con 0x7f3c3c1593d0 2024-03-29T08:20:15.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c42c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c0cdb40 con 0x7f3c3c15dbd0 2024-03-29T08:20:15.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c33e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3c15d230 0x7f3c3c150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:15.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c33e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3c15d230 0x7f3c3c150230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37920/0 (socket says 172.21.15.46:37920) 2024-03-29T08:20:15.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c33e00640 1 -- 172.21.15.46:0/812150918 learned_addr learned my addr 172.21.15.46:0/812150918 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:15.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c3be00640 1 --2- 172.21.15.46:0/812150918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c3c1593d0 0x7f3c3c0cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:15.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c40c00640 1 --2- 172.21.15.46:0/812150918 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3c3c15dbd0 0x7f3c3c150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:15.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c33e00640 1 -- 172.21.15.46:0/812150918 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3c3c15dbd0 msgr2=0x7f3c3c150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:15.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c33e00640 1 --2- 172.21.15.46:0/812150918 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3c3c15dbd0 0x7f3c3c150770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c33e00640 1 -- 172.21.15.46:0/812150918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c3c1593d0 msgr2=0x7f3c3c0cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:15.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c33e00640 1 --2- 172.21.15.46:0/812150918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c3c1593d0 0x7f3c3c0cce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c33e00640 1 -- 172.21.15.46:0/812150918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c0094d0 con 0x7f3c3c15d230 2024-03-29T08:20:15.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c3be00640 1 --2- 172.21.15.46:0/812150918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c3c1593d0 0x7f3c3c0cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:15.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c40c00640 1 --2- 172.21.15.46:0/812150918 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3c3c15dbd0 0x7f3c3c150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:15.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c33e00640 1 --2- 172.21.15.46:0/812150918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3c15d230 0x7f3c3c150230 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400e870 tx=0x7f3c2400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:15.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c38c00640 1 -- 172.21.15.46:0/812150918 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c24009510 con 0x7f3c3c15d230 2024-03-29T08:20:15.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c38c00640 1 -- 172.21.15.46:0/812150918 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2400eec0 con 0x7f3c3c15d230 2024-03-29T08:20:15.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c38c00640 1 -- 172.21.15.46:0/812150918 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2400cb40 con 0x7f3c3c15d230 2024-03-29T08:20:15.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c42c00640 1 -- 172.21.15.46:0/812150918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3c0cde20 con 0x7f3c3c15d230 2024-03-29T08:20:15.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.583+0000 7f3c42c00640 1 -- 172.21.15.46:0/812150918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3c1463a0 con 0x7f3c3c15d230 2024-03-29T08:20:15.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.587+0000 7f3c42c00640 1 -- 172.21.15.46:0/812150918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c3c066c90 con 0x7f3c3c15d230 2024-03-29T08:20:15.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.587+0000 7f3c38c00640 1 -- 172.21.15.46:0/812150918 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3c2400ccc0 con 0x7f3c3c15d230 2024-03-29T08:20:15.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.587+0000 7f3c38c00640 1 --2- 172.21.15.46:0/812150918 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3c18066690 0x7f3c18068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:15.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.587+0000 7f3c38c00640 1 -- 172.21.15.46:0/812150918 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c24084260 con 0x7f3c3c15d230 2024-03-29T08:20:15.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.587+0000 7f3c3be00640 1 --2- 172.21.15.46:0/812150918 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3c18066690 0x7f3c18068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:15.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.587+0000 7f3c3be00640 1 --2- 172.21.15.46:0/812150918 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3c18066690 0x7f3c18068b50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0ae000 tx=0x7f3c2c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:15.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.591+0000 7f3c38c00640 1 -- 172.21.15.46:0/812150918 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3c240515c0 con 0x7f3c3c15d230 2024-03-29T08:20:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.787+0000 7f3c42c00640 1 -- 172.21.15.46:0/812150918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c3c1643a0 con 0x7f3c3c15d230 2024-03-29T08:20:15.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.791+0000 7f3c38c00640 1 -- 172.21.15.46:0/812150918 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4731 (secure 0 0 0) 0x7f3c24054be0 con 0x7f3c3c15d230 2024-03-29T08:20:15.792 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:15.792 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2702541693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2702541693},{"type":"v1","addr":"172.21.15.73:6837","nonce":2702541693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2507803771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2507803771},{"type":"v1","addr":"172.21.15.73:6835","nonce":2507803771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/494932499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":494932499},{"type":"v1","addr":"172.21.15.46:6833","nonce":494932499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:20:12.400114+0000","modified":"2024-03-29T08:20:15.405861+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3467089652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3467089652},{"type":"v1","addr":"172.21.15.46:6835","nonce":3467089652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-29T08:20:15.792 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-03-29T08:20:15.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.791+0000 7f3c42c00640 1 -- 172.21.15.46:0/812150918 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3c18066690 msgr2=0x7f3c18068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:15.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.791+0000 7f3c42c00640 1 --2- 172.21.15.46:0/812150918 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3c18066690 0x7f3c18068b50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c0ae000 tx=0x7f3c2c009040 comp rx=0 tx=0).stop 2024-03-29T08:20:15.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.791+0000 7f3c42c00640 1 -- 172.21.15.46:0/812150918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3c15d230 msgr2=0x7f3c3c150230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.791+0000 7f3c42c00640 1 --2- 172.21.15.46:0/812150918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3c15d230 0x7f3c3c150230 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3c2400e870 tx=0x7f3c2400ec70 comp rx=0 tx=0).stop 2024-03-29T08:20:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.791+0000 7f3c42c00640 1 -- 172.21.15.46:0/812150918 shutdown_connections 2024-03-29T08:20:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.791+0000 7f3c42c00640 1 --2- 172.21.15.46:0/812150918 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3c18066690 0x7f3c18068b50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.791+0000 7f3c42c00640 1 --2- 172.21.15.46:0/812150918 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3c3c15dbd0 0x7f3c3c150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.791+0000 7f3c42c00640 1 --2- 172.21.15.46:0/812150918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c3c15d230 0x7f3c3c150230 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.791+0000 7f3c42c00640 1 --2- 172.21.15.46:0/812150918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c3c1593d0 0x7f3c3c0cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.791+0000 7f3c42c00640 1 -- 172.21.15.46:0/812150918 >> 172.21.15.46:0/812150918 conn(0x7f3c3c066110 msgr2=0x7f3c3c0ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.791+0000 7f3c42c00640 1 -- 172.21.15.46:0/812150918 shutdown_connections 2024-03-29T08:20:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.791+0000 7f3c42c00640 1 -- 172.21.15.46:0/812150918 wait complete. 2024-03-29T08:20:15.812 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:20:15.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.951+0000 7f23f0c00640 1 Processor -- start 2024-03-29T08:20:15.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.951+0000 7f23f0c00640 1 -- start start 2024-03-29T08:20:15.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.951+0000 7f23f0c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23ec15ad20 0x7f23ec0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:15.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.951+0000 7f23f0c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f23ec0b0540 0x7f23ec0cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:15.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.951+0000 7f23f0c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23ec0b0ab0 con 0x7f23ec15a940 2024-03-29T08:20:15.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.951+0000 7f23f0c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23ec0b0d90 con 0x7f23ec0b0540 2024-03-29T08:20:15.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.951+0000 7f23f0c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec0b1070 con 0x7f23ec15ad20 2024-03-29T08:20:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.951+0000 7f23e1400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23ec15ad20 0x7f23ec0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.951+0000 7f23e1400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23ec15ad20 0x7f23ec0cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:47056/0 (socket says 172.21.15.46:47056) 2024-03-29T08:20:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.951+0000 7f23ea800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f23ec0b0540 0x7f23ec0cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.951+0000 7f23e1400640 1 -- 172.21.15.46:0/2496107020 learned_addr learned my addr 172.21.15.46:0/2496107020 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.951+0000 7f23e1400640 1 -- 172.21.15.46:0/2496107020 >> v2:172.21.15.73:3300/0 conn(0x7f23ec0b0540 msgr2=0x7f23ec0cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.951+0000 7f23e1400640 1 --2- 172.21.15.46:0/2496107020 >> v2:172.21.15.73:3300/0 conn(0x7f23ec0b0540 0x7f23ec0cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.951+0000 7f23e1400640 1 -- 172.21.15.46:0/2496107020 >> v1:172.21.15.73:6789/0 conn(0x7f23ec15a940 legacy=0x7f23ec0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:15.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.951+0000 7f23e9400640 1 -- 172.21.15.46:0/2496107020 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226064626 0 0) 0x7f23ec0b0ab0 con 0x7f23ec15a940 2024-03-29T08:20:15.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.951+0000 7f23e1400640 1 -- 172.21.15.46:0/2496107020 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ec0b1350 con 0x7f23ec15ad20 2024-03-29T08:20:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.955+0000 7f23e1400640 1 --2- 172.21.15.46:0/2496107020 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23ec15ad20 0x7f23ec0cecb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009870 tx=0x7f23d401c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2c924659bc73a6b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.955+0000 7f23e9400640 1 -- 172.21.15.46:0/2496107020 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d40a9610 con 0x7f23ec15ad20 2024-03-29T08:20:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.955+0000 7f23e9400640 1 -- 172.21.15.46:0/2496107020 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d40a9770 con 0x7f23ec15ad20 2024-03-29T08:20:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.955+0000 7f23e9400640 1 -- 172.21.15.46:0/2496107020 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d40a9b00 con 0x7f23ec15ad20 2024-03-29T08:20:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.955+0000 7f23f0c00640 1 -- 172.21.15.46:0/2496107020 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23ec15ad20 msgr2=0x7f23ec0cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.955+0000 7f23f0c00640 1 --2- 172.21.15.46:0/2496107020 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23ec15ad20 0x7f23ec0cecb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f23d4009870 tx=0x7f23d401c530 comp rx=0 tx=0).stop 2024-03-29T08:20:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.955+0000 7f23f0c00640 1 -- 172.21.15.46:0/2496107020 shutdown_connections 2024-03-29T08:20:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.955+0000 7f23f0c00640 1 --2- 172.21.15.46:0/2496107020 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23ec15ad20 0x7f23ec0cecb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.955+0000 7f23f0c00640 1 --2- 172.21.15.46:0/2496107020 >> v2:172.21.15.73:3300/0 conn(0x7f23ec0b0540 0x7f23ec0cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.955+0000 7f23f0c00640 1 -- 172.21.15.46:0/2496107020 >> 172.21.15.46:0/2496107020 conn(0x7f23ec089fd0 msgr2=0x7f23ec08a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:15.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.955+0000 7f23f0c00640 1 -- 172.21.15.46:0/2496107020 shutdown_connections 2024-03-29T08:20:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.955+0000 7f23f0c00640 1 -- 172.21.15.46:0/2496107020 wait complete. 2024-03-29T08:20:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23f0c00640 1 Processor -- start 2024-03-29T08:20:15.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23f0c00640 1 -- start start 2024-03-29T08:20:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23f0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23ec0b0540 0x7f23ec156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23f0c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23ec15a940 0x7f23ec0c4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23f0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23ec15ad20 0x7f23ec0add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23f0c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec157150 con 0x7f23ec15ad20 2024-03-29T08:20:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23f0c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec0b37b0 con 0x7f23ec0b0540 2024-03-29T08:20:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23f0c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ec0b4610 con 0x7f23ec15a940 2024-03-29T08:20:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23ea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23ec15ad20 0x7f23ec0add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23ea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23ec15ad20 0x7f23ec0add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37936/0 (socket says 172.21.15.46:37936) 2024-03-29T08:20:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23ea800640 1 -- 172.21.15.46:0/1698975243 learned_addr learned my addr 172.21.15.46:0/1698975243 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23e1400640 1 --2- 172.21.15.46:0/1698975243 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23ec15a940 0x7f23ec0c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23e9e00640 1 --2- 172.21.15.46:0/1698975243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23ec0b0540 0x7f23ec156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23ea800640 1 -- 172.21.15.46:0/1698975243 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23ec15a940 msgr2=0x7f23ec0c4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23ea800640 1 --2- 172.21.15.46:0/1698975243 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23ec15a940 0x7f23ec0c4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23ea800640 1 -- 172.21.15.46:0/1698975243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23ec0b0540 msgr2=0x7f23ec156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23ea800640 1 --2- 172.21.15.46:0/1698975243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23ec0b0540 0x7f23ec156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23ea800640 1 -- 172.21.15.46:0/1698975243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d40094d0 con 0x7f23ec15ad20 2024-03-29T08:20:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23ea800640 1 --2- 172.21.15.46:0/1698975243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23ec15ad20 0x7f23ec0add70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009dd0 tx=0x7f23e400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23e2a00640 1 -- 172.21.15.46:0/1698975243 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4014480 con 0x7f23ec15ad20 2024-03-29T08:20:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23e2a00640 1 -- 172.21.15.46:0/1698975243 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e400f040 con 0x7f23ec15ad20 2024-03-29T08:20:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23e2a00640 1 -- 172.21.15.46:0/1698975243 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e4014840 con 0x7f23ec15ad20 2024-03-29T08:20:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23f0c00640 1 -- 172.21.15.46:0/1698975243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23ec0b3a00 con 0x7f23ec15ad20 2024-03-29T08:20:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.959+0000 7f23f0c00640 1 -- 172.21.15.46:0/1698975243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ec0b3fc0 con 0x7f23ec15ad20 2024-03-29T08:20:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.963+0000 7f23f0c00640 1 -- 172.21.15.46:0/1698975243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23ec0b47a0 con 0x7f23ec15ad20 2024-03-29T08:20:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.963+0000 7f23e2a00640 1 -- 172.21.15.46:0/1698975243 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f23e40149a0 con 0x7f23ec15ad20 2024-03-29T08:20:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.963+0000 7f23e2a00640 1 --2- 172.21.15.46:0/1698975243 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23c8066690 0x7f23c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.963+0000 7f23e2a00640 1 -- 172.21.15.46:0/1698975243 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23e4084c00 con 0x7f23ec15ad20 2024-03-29T08:20:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.963+0000 7f23e9e00640 1 --2- 172.21.15.46:0/1698975243 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23c8066690 0x7f23c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.963+0000 7f23e9e00640 1 --2- 172.21.15.46:0/1698975243 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23c8066690 0x7f23c8068b50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f23dc000f70 tx=0x7f23dc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:15.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:15.967+0000 7f23e2a00640 1 -- 172.21.15.46:0/1698975243 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f23e4051f80 con 0x7f23ec15ad20 2024-03-29T08:20:16.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.135+0000 7f23f0c00640 1 -- 172.21.15.46:0/1698975243 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23ec066c90 con 0x7f23ec15ad20 2024-03-29T08:20:16.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.135+0000 7f23e2a00640 1 -- 172.21.15.46:0/1698975243 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f23e4018090 con 0x7f23ec15ad20 2024-03-29T08:20:16.138 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:16.138 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":49,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:20:13.402206+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:20:10.906619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-29T08:20:11.389667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:20:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.139+0000 7f23f0c00640 1 -- 172.21.15.46:0/1698975243 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23c8066690 msgr2=0x7f23c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.139+0000 7f23f0c00640 1 --2- 172.21.15.46:0/1698975243 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23c8066690 0x7f23c8068b50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f23dc000f70 tx=0x7f23dc009040 comp rx=0 tx=0).stop 2024-03-29T08:20:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.139+0000 7f23f0c00640 1 -- 172.21.15.46:0/1698975243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23ec15ad20 msgr2=0x7f23ec0add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.139+0000 7f23f0c00640 1 --2- 172.21.15.46:0/1698975243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23ec15ad20 0x7f23ec0add70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f23e4009dd0 tx=0x7f23e400c490 comp rx=0 tx=0).stop 2024-03-29T08:20:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.139+0000 7f23f0c00640 1 -- 172.21.15.46:0/1698975243 shutdown_connections 2024-03-29T08:20:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.139+0000 7f23f0c00640 1 --2- 172.21.15.46:0/1698975243 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23c8066690 0x7f23c8068b50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.139+0000 7f23f0c00640 1 --2- 172.21.15.46:0/1698975243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23ec15ad20 0x7f23ec0add70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.139+0000 7f23f0c00640 1 --2- 172.21.15.46:0/1698975243 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23ec15a940 0x7f23ec0c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.139+0000 7f23f0c00640 1 --2- 172.21.15.46:0/1698975243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23ec0b0540 0x7f23ec156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.139+0000 7f23f0c00640 1 -- 172.21.15.46:0/1698975243 >> 172.21.15.46:0/1698975243 conn(0x7f23ec089fd0 msgr2=0x7f23ec08a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.139+0000 7f23f0c00640 1 -- 172.21.15.46:0/1698975243 shutdown_connections 2024-03-29T08:20:16.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.139+0000 7f23f0c00640 1 -- 172.21.15.46:0/1698975243 wait complete. 2024-03-29T08:20:16.160 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T08:20:16.160 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:20:16.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.295+0000 7f5649800640 1 Processor -- start 2024-03-29T08:20:16.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.295+0000 7f5649800640 1 -- start start 2024-03-29T08:20:16.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.299+0000 7f5649800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5644144210 0x7f56440cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:16.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.299+0000 7f5649800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56440cb510 con 0x7f56440ceb20 2024-03-29T08:20:16.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.299+0000 7f5649800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5644070580 con 0x7f564415ada0 2024-03-29T08:20:16.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.299+0000 7f5649800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5644070860 con 0x7f5644144210 2024-03-29T08:20:16.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.299+0000 7f5643400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f56440ceb20 0x7f564406c950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33614/0 (socket says 172.21.15.46:33614) 2024-03-29T08:20:16.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.299+0000 7f5643400640 1 -- 172.21.15.46:0/2039963671 learned_addr learned my addr 172.21.15.46:0/2039963671 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:16.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.299+0000 7f5642a00640 1 --2- 172.21.15.46:0/2039963671 >> v2:172.21.15.73:3300/0 conn(0x7f5644144210 0x7f56440cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:16.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.299+0000 7f5642000640 1 -- 172.21.15.46:0/2039963671 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655109883 0 0) 0x7f56440cb510 con 0x7f56440ceb20 2024-03-29T08:20:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.299+0000 7f5642000640 1 -- 172.21.15.46:0/2039963671 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5628003530 con 0x7f56440ceb20 2024-03-29T08:20:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.299+0000 7f5642000640 1 -- 172.21.15.46:0/2039963671 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388406440 0 0) 0x7f5644070580 con 0x7f564415ada0 2024-03-29T08:20:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.299+0000 7f5642000640 1 -- 172.21.15.46:0/2039963671 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56440cb510 con 0x7f564415ada0 2024-03-29T08:20:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.299+0000 7f5642a00640 1 -- 172.21.15.46:0/2039963671 >> v1:172.21.15.73:6789/0 conn(0x7f564415ada0 legacy=0x7f56440ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.299+0000 7f5642a00640 1 -- 172.21.15.46:0/2039963671 >> v1:172.21.15.46:6789/0 conn(0x7f56440ceb20 legacy=0x7f564406c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.299+0000 7f5642000640 1 -- 172.21.15.46:0/2039963671 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1347742745 0 0) 0x7f5628003530 con 0x7f56440ceb20 2024-03-29T08:20:16.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.299+0000 7f5642a00640 1 -- 172.21.15.46:0/2039963671 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5644070b40 con 0x7f5644144210 2024-03-29T08:20:16.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.303+0000 7f5642a00640 1 --2- 172.21.15.46:0/2039963671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5644144210 0x7f56440cb8b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f56340097c0 tx=0x7f563401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab3041eb2db18f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:16.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.303+0000 7f5642000640 1 -- 172.21.15.46:0/2039963671 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c001a00 con 0x7f5644144210 2024-03-29T08:20:16.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.303+0000 7f5642000640 1 -- 172.21.15.46:0/2039963671 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56340a7050 con 0x7f5644144210 2024-03-29T08:20:16.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.303+0000 7f5642000640 1 -- 172.21.15.46:0/2039963671 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56340ab620 con 0x7f5644144210 2024-03-29T08:20:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.303+0000 7f5649800640 1 -- 172.21.15.46:0/2039963671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5644144210 msgr2=0x7f56440cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.303+0000 7f5649800640 1 --2- 172.21.15.46:0/2039963671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5644144210 0x7f56440cb8b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f56340097c0 tx=0x7f563401c3c0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.303+0000 7f5649800640 1 -- 172.21.15.46:0/2039963671 shutdown_connections 2024-03-29T08:20:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.303+0000 7f5649800640 1 --2- 172.21.15.46:0/2039963671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5644144210 0x7f56440cb8b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.303+0000 7f5649800640 1 -- 172.21.15.46:0/2039963671 >> 172.21.15.46:0/2039963671 conn(0x7f5644066110 msgr2=0x7f5644069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.303+0000 7f5649800640 1 -- 172.21.15.46:0/2039963671 shutdown_connections 2024-03-29T08:20:16.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.303+0000 7f5649800640 1 -- 172.21.15.46:0/2039963671 wait complete. 2024-03-29T08:20:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.303+0000 7f5649800640 1 Processor -- start 2024-03-29T08:20:16.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.303+0000 7f5649800640 1 -- start start 2024-03-29T08:20:16.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5649800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56440ceb20 0x7f56440aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:16.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5649800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5644144210 0x7f56441482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:16.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5649800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f564415ada0 0x7f56440b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:16.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5649800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56441487f0 con 0x7f564415ada0 2024-03-29T08:20:16.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5649800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56440af220 con 0x7f5644144210 2024-03-29T08:20:16.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5649800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56440af500 con 0x7f56440ceb20 2024-03-29T08:20:16.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5643400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f564415ada0 0x7f56440b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:16.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5643400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f564415ada0 0x7f56440b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37938/0 (socket says 172.21.15.46:37938) 2024-03-29T08:20:16.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5643400640 1 -- 172.21.15.46:0/618518237 learned_addr learned my addr 172.21.15.46:0/618518237 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:16.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f563be00640 1 --2- 172.21.15.46:0/618518237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5644144210 0x7f56441482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:16.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5642a00640 1 --2- 172.21.15.46:0/618518237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56440ceb20 0x7f56440aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:16.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5643400640 1 -- 172.21.15.46:0/618518237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56440ceb20 msgr2=0x7f56440aec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:16.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5643400640 1 --2- 172.21.15.46:0/618518237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56440ceb20 0x7f56440aec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5643400640 1 -- 172.21.15.46:0/618518237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5644144210 msgr2=0x7f56441482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:16.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5643400640 1 --2- 172.21.15.46:0/618518237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5644144210 0x7f56441482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5643400640 1 -- 172.21.15.46:0/618518237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5634009420 con 0x7f564415ada0 2024-03-29T08:20:16.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5643400640 1 --2- 172.21.15.46:0/618518237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f564415ada0 0x7f56440b2bc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f563c00cc90 tx=0x7f563c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f563aa00640 1 -- 172.21.15.46:0/618518237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c00cea0 con 0x7f564415ada0 2024-03-29T08:20:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f563aa00640 1 -- 172.21.15.46:0/618518237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563c012400 con 0x7f564415ada0 2024-03-29T08:20:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5649800640 1 -- 172.21.15.46:0/618518237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56440af7e0 con 0x7f564415ada0 2024-03-29T08:20:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f563aa00640 1 -- 172.21.15.46:0/618518237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c012770 con 0x7f564415ada0 2024-03-29T08:20:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5649800640 1 -- 172.21.15.46:0/618518237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56440b3100 con 0x7f564415ada0 2024-03-29T08:20:16.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.307+0000 7f5649800640 1 -- 172.21.15.46:0/618518237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56440ce280 con 0x7f564415ada0 2024-03-29T08:20:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.311+0000 7f563aa00640 1 -- 172.21.15.46:0/618518237 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f563c0128d0 con 0x7f564415ada0 2024-03-29T08:20:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.311+0000 7f563aa00640 1 --2- 172.21.15.46:0/618518237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5620066690 0x7f5620068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:16.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.311+0000 7f563aa00640 1 -- 172.21.15.46:0/618518237 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f563c084710 con 0x7f564415ada0 2024-03-29T08:20:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.311+0000 7f5642a00640 1 --2- 172.21.15.46:0/618518237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5620066690 0x7f5620068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:16.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.311+0000 7f5642a00640 1 --2- 172.21.15.46:0/618518237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5620066690 0x7f5620068b50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f56340ad000 tx=0x7f56340a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:16.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.315+0000 7f563aa00640 1 -- 172.21.15.46:0/618518237 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f563c051bc0 con 0x7f564415ada0 2024-03-29T08:20:16.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.475+0000 7f5649800640 1 -- 172.21.15.46:0/618518237 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56441686f0 con 0x7f5620066690 2024-03-29T08:20:16.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.487+0000 7f563aa00640 1 -- 172.21.15.46:0/618518237 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289123 (secure 0 0 0) 0x7f5634006e20 con 0x7f5620066690 2024-03-29T08:20:16.491 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:16.492 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:20:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.487+0000 7f5649800640 1 -- 172.21.15.46:0/618518237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5620066690 msgr2=0x7f5620068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.487+0000 7f5649800640 1 --2- 172.21.15.46:0/618518237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5620066690 0x7f5620068b50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f56340ad000 tx=0x7f56340a6000 comp rx=0 tx=0).stop 2024-03-29T08:20:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.487+0000 7f5649800640 1 -- 172.21.15.46:0/618518237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f564415ada0 msgr2=0x7f56440b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.487+0000 7f5649800640 1 --2- 172.21.15.46:0/618518237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f564415ada0 0x7f56440b2bc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f563c00cc90 tx=0x7f563c00a430 comp rx=0 tx=0).stop 2024-03-29T08:20:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.491+0000 7f5649800640 1 -- 172.21.15.46:0/618518237 shutdown_connections 2024-03-29T08:20:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.491+0000 7f5649800640 1 --2- 172.21.15.46:0/618518237 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5620066690 0x7f5620068b50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.491+0000 7f5649800640 1 --2- 172.21.15.46:0/618518237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5644144210 0x7f56441482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.491+0000 7f5649800640 1 --2- 172.21.15.46:0/618518237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f564415ada0 0x7f56440b2bc0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.491+0000 7f5649800640 1 --2- 172.21.15.46:0/618518237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f56440ceb20 0x7f56440aec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.491+0000 7f5649800640 1 -- 172.21.15.46:0/618518237 >> 172.21.15.46:0/618518237 conn(0x7f5644066110 msgr2=0x7f564413e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.491+0000 7f5649800640 1 -- 172.21.15.46:0/618518237 shutdown_connections 2024-03-29T08:20:16.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.491+0000 7f5649800640 1 -- 172.21.15.46:0/618518237 wait complete. 2024-03-29T08:20:16.511 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-03-29T08:20:16.069770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242360,"kb_used_data":5672,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754732360,"statfs":{"total":773094113280,"available":772845936640,"internally_reserved":0,"allocated":5808128,"data_stored":3039868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":242208313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.082492"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823754+0000","last_change":"2024-03-29T08:20:12.419302+0000","last_active":"2024-03-29T08:20:13.823754+0000","last_peered":"2024-03-29T08:20:13.823754+0000","last_clean":"2024-03-29T08:20:13.823754+0000","last_became_active":"2024-03-29T08:20:12.418550+0000","last_became_peered":"2024-03-29T08:20:12.418550+0000","last_unstale":"2024-03-29T08:20:13.823754+0000","last_undegraded":"2024-03-29T08:20:13.823754+0000","last_fullsized":"2024-03-29T08:20:13.823754+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:37:00.044842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823358+0000","last_change":"2024-03-29T08:20:12.437537+0000","last_active":"2024-03-29T08:20:13.823358+0000","last_peered":"2024-03-29T08:20:13.823358+0000","last_clean":"2024-03-29T08:20:13.823358+0000","last_became_active":"2024-03-29T08:20:12.430681+0000","last_became_peered":"2024-03-29T08:20:12.430681+0000","last_unstale":"2024-03-29T08:20:13.823358+0000","last_undegraded":"2024-03-29T08:20:13.823358+0000","last_fullsized":"2024-03-29T08:20:13.823358+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:37:00.044842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406798+0000","last_change":"2024-03-29T08:20:12.433309+0000","last_active":"2024-03-29T08:20:13.406798+0000","last_peered":"2024-03-29T08:20:13.406798+0000","last_clean":"2024-03-29T08:20:13.406798+0000","last_became_active":"2024-03-29T08:20:12.433048+0000","last_became_peered":"2024-03-29T08:20:12.433048+0000","last_unstale":"2024-03-29T08:20:13.406798+0000","last_undegraded":"2024-03-29T08:20:13.406798+0000","last_fullsized":"2024-03-29T08:20:13.406798+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:05:35.885318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823826+0000","last_change":"2024-03-29T08:20:12.431343+0000","last_active":"2024-03-29T08:20:13.823826+0000","last_peered":"2024-03-29T08:20:13.823826+0000","last_clean":"2024-03-29T08:20:13.823826+0000","last_became_active":"2024-03-29T08:20:12.430381+0000","last_became_peered":"2024-03-29T08:20:12.430381+0000","last_unstale":"2024-03-29T08:20:13.823826+0000","last_undegraded":"2024-03-29T08:20:13.823826+0000","last_fullsized":"2024-03-29T08:20:13.823826+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:12:57.396667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823109+0000","last_change":"2024-03-29T08:20:12.430861+0000","last_active":"2024-03-29T08:20:13.823109+0000","last_peered":"2024-03-29T08:20:13.823109+0000","last_clean":"2024-03-29T08:20:13.823109+0000","last_became_active":"2024-03-29T08:20:12.430619+0000","last_became_peered":"2024-03-29T08:20:12.430619+0000","last_unstale":"2024-03-29T08:20:13.823109+0000","last_undegraded":"2024-03-29T08:20:13.823109+0000","last_fullsized":"2024-03-29T08:20:13.823109+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:09:21.154366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823355+0000","last_change":"2024-03-29T08:20:12.439025+0000","last_active":"2024-03-29T08:20:13.823355+0000","last_peered":"2024-03-29T08:20:13.823355+0000","last_clean":"2024-03-29T08:20:13.823355+0000","last_became_active":"2024-03-29T08:20:12.438753+0000","last_became_peered":"2024-03-29T08:20:12.438753+0000","last_unstale":"2024-03-29T08:20:13.823355+0000","last_undegraded":"2024-03-29T08:20:13.823355+0000","last_fullsized":"2024-03-29T08:20:13.823355+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:55:17.168641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408321+0000","last_change":"2024-03-29T08:20:12.434732+0000","last_active":"2024-03-29T08:20:13.408321+0000","last_peered":"2024-03-29T08:20:13.408321+0000","last_clean":"2024-03-29T08:20:13.408321+0000","last_became_active":"2024-03-29T08:20:12.434504+0000","last_became_peered":"2024-03-29T08:20:12.434504+0000","last_unstale":"2024-03-29T08:20:13.408321+0000","last_undegraded":"2024-03-29T08:20:13.408321+0000","last_fullsized":"2024-03-29T08:20:13.408321+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:00:21.157051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823293+0000","last_change":"2024-03-29T08:20:12.432677+0000","last_active":"2024-03-29T08:20:13.823293+0000","last_peered":"2024-03-29T08:20:13.823293+0000","last_clean":"2024-03-29T08:20:13.823293+0000","last_became_active":"2024-03-29T08:20:12.431818+0000","last_became_peered":"2024-03-29T08:20:12.431818+0000","last_unstale":"2024-03-29T08:20:13.823293+0000","last_undegraded":"2024-03-29T08:20:13.823293+0000","last_fullsized":"2024-03-29T08:20:13.823293+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:04:28.146272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823919+0000","last_change":"2024-03-29T08:20:12.426774+0000","last_active":"2024-03-29T08:20:13.823919+0000","last_peered":"2024-03-29T08:20:13.823919+0000","last_clean":"2024-03-29T08:20:13.823919+0000","last_became_active":"2024-03-29T08:20:12.426540+0000","last_became_peered":"2024-03-29T08:20:12.426540+0000","last_unstale":"2024-03-29T08:20:13.823919+0000","last_undegraded":"2024-03-29T08:20:13.823919+0000","last_fullsized":"2024-03-29T08:20:13.823919+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:36:35.453659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.409123+0000","last_change":"2024-03-29T08:20:12.422005+0000","last_active":"2024-03-29T08:20:13.409123+0000","last_peered":"2024-03-29T08:20:13.409123+0000","last_clean":"2024-03-29T08:20:13.409123+0000","last_became_active":"2024-03-29T08:20:12.421722+0000","last_became_peered":"2024-03-29T08:20:12.421722+0000","last_unstale":"2024-03-29T08:20:13.409123+0000","last_undegraded":"2024-03-29T08:20:13.409123+0000","last_fullsized":"2024-03-29T08:20:13.409123+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:42:32.522198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823691+0000","last_change":"2024-03-29T08:20:12.431756+0000","last_active":"2024-03-29T08:20:13.823691+0000","last_peered":"2024-03-29T08:20:13.823691+0000","last_clean":"2024-03-29T08:20:13.823691+0000","last_became_active":"2024-03-29T08:20:12.431044+0000","last_became_peered":"2024-03-29T08:20:12.431044+0000","last_unstale":"2024-03-29T08:20:13.823691+0000","last_undegraded":"2024-03-29T08:20:13.823691+0000","last_fullsized":"2024-03-29T08:20:13.823691+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:28:16.416846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823236+0000","last_change":"2024-03-29T08:20:12.420989+0000","last_active":"2024-03-29T08:20:13.823236+0000","last_peered":"2024-03-29T08:20:13.823236+0000","last_clean":"2024-03-29T08:20:13.823236+0000","last_became_active":"2024-03-29T08:20:12.420743+0000","last_became_peered":"2024-03-29T08:20:12.420743+0000","last_unstale":"2024-03-29T08:20:13.823236+0000","last_undegraded":"2024-03-29T08:20:13.823236+0000","last_fullsized":"2024-03-29T08:20:13.823236+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:59.685322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823139+0000","last_change":"2024-03-29T08:20:12.439238+0000","last_active":"2024-03-29T08:20:13.823139+0000","last_peered":"2024-03-29T08:20:13.823139+0000","last_clean":"2024-03-29T08:20:13.823139+0000","last_became_active":"2024-03-29T08:20:12.438895+0000","last_became_peered":"2024-03-29T08:20:12.438895+0000","last_unstale":"2024-03-29T08:20:13.823139+0000","last_undegraded":"2024-03-29T08:20:13.823139+0000","last_fullsized":"2024-03-29T08:20:13.823139+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:59.685322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823619+0000","last_change":"2024-03-29T08:20:12.433228+0000","last_active":"2024-03-29T08:20:13.823619+0000","last_peered":"2024-03-29T08:20:13.823619+0000","last_clean":"2024-03-29T08:20:13.823619+0000","last_became_active":"2024-03-29T08:20:12.432959+0000","last_became_peered":"2024-03-29T08:20:12.432959+0000","last_unstale":"2024-03-29T08:20:13.823619+0000","last_undegraded":"2024-03-29T08:20:13.823619+0000","last_fullsized":"2024-03-29T08:20:13.823619+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:46.664716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823905+0000","last_change":"2024-03-29T08:20:12.431541+0000","last_active":"2024-03-29T08:20:13.823905+0000","last_peered":"2024-03-29T08:20:13.823905+0000","last_clean":"2024-03-29T08:20:13.823905+0000","last_became_active":"2024-03-29T08:20:12.431200+0000","last_became_peered":"2024-03-29T08:20:12.431200+0000","last_unstale":"2024-03-29T08:20:13.823905+0000","last_undegraded":"2024-03-29T08:20:13.823905+0000","last_fullsized":"2024-03-29T08:20:13.823905+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:40:51.379844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.822784+0000","last_change":"2024-03-29T08:20:12.434244+0000","last_active":"2024-03-29T08:20:13.822784+0000","last_peered":"2024-03-29T08:20:13.822784+0000","last_clean":"2024-03-29T08:20:13.822784+0000","last_became_active":"2024-03-29T08:20:12.433958+0000","last_became_peered":"2024-03-29T08:20:12.433958+0000","last_unstale":"2024-03-29T08:20:13.822784+0000","last_undegraded":"2024-03-29T08:20:13.822784+0000","last_fullsized":"2024-03-29T08:20:13.822784+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:43:50.843484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823301+0000","last_change":"2024-03-29T08:20:12.422488+0000","last_active":"2024-03-29T08:20:13.823301+0000","last_peered":"2024-03-29T08:20:13.823301+0000","last_clean":"2024-03-29T08:20:13.823301+0000","last_became_active":"2024-03-29T08:20:12.422205+0000","last_became_peered":"2024-03-29T08:20:12.422205+0000","last_unstale":"2024-03-29T08:20:13.823301+0000","last_undegraded":"2024-03-29T08:20:13.823301+0000","last_fullsized":"2024-03-29T08:20:13.823301+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:17:37.930817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406813+0000","last_change":"2024-03-29T08:20:12.435506+0000","last_active":"2024-03-29T08:20:13.406813+0000","last_peered":"2024-03-29T08:20:13.406813+0000","last_clean":"2024-03-29T08:20:13.406813+0000","last_became_active":"2024-03-29T08:20:12.435209+0000","last_became_peered":"2024-03-29T08:20:12.435209+0000","last_unstale":"2024-03-29T08:20:13.406813+0000","last_undegraded":"2024-03-29T08:20:13.406813+0000","last_fullsized":"2024-03-29T08:20:13.406813+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:07.507985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823638+0000","last_change":"2024-03-29T08:20:12.428939+0000","last_active":"2024-03-29T08:20:13.823638+0000","last_peered":"2024-03-29T08:20:13.823638+0000","last_clean":"2024-03-29T08:20:13.823638+0000","last_became_active":"2024-03-29T08:20:12.428490+0000","last_became_peered":"2024-03-29T08:20:12.428490+0000","last_unstale":"2024-03-29T08:20:13.823638+0000","last_undegraded":"2024-03-29T08:20:13.823638+0000","last_fullsized":"2024-03-29T08:20:13.823638+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:01:33.657226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408945+0000","last_change":"2024-03-29T08:20:12.427514+0000","last_active":"2024-03-29T08:20:13.408945+0000","last_peered":"2024-03-29T08:20:13.408945+0000","last_clean":"2024-03-29T08:20:13.408945+0000","last_became_active":"2024-03-29T08:20:12.425982+0000","last_became_peered":"2024-03-29T08:20:12.425982+0000","last_unstale":"2024-03-29T08:20:13.408945+0000","last_undegraded":"2024-03-29T08:20:13.408945+0000","last_fullsized":"2024-03-29T08:20:13.408945+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:43:22.612508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823880+0000","last_change":"2024-03-29T08:20:12.429011+0000","last_active":"2024-03-29T08:20:13.823880+0000","last_peered":"2024-03-29T08:20:13.823880+0000","last_clean":"2024-03-29T08:20:13.823880+0000","last_became_active":"2024-03-29T08:20:12.428723+0000","last_became_peered":"2024-03-29T08:20:12.428723+0000","last_unstale":"2024-03-29T08:20:13.823880+0000","last_undegraded":"2024-03-29T08:20:13.823880+0000","last_fullsized":"2024-03-29T08:20:13.823880+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:43:12.128937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.470008+0000","last_change":"2024-03-29T08:20:12.414219+0000","last_active":"2024-03-29T08:20:13.470008+0000","last_peered":"2024-03-29T08:20:13.470008+0000","last_clean":"2024-03-29T08:20:13.470008+0000","last_became_active":"2024-03-29T08:20:12.413962+0000","last_became_peered":"2024-03-29T08:20:12.413962+0000","last_unstale":"2024-03-29T08:20:13.470008+0000","last_undegraded":"2024-03-29T08:20:13.470008+0000","last_fullsized":"2024-03-29T08:20:13.470008+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:28:03.339574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408448+0000","last_change":"2024-03-29T08:20:12.428957+0000","last_active":"2024-03-29T08:20:13.408448+0000","last_peered":"2024-03-29T08:20:13.408448+0000","last_clean":"2024-03-29T08:20:13.408448+0000","last_became_active":"2024-03-29T08:20:12.428685+0000","last_became_peered":"2024-03-29T08:20:12.428685+0000","last_unstale":"2024-03-29T08:20:13.408448+0000","last_undegraded":"2024-03-29T08:20:13.408448+0000","last_fullsized":"2024-03-29T08:20:13.408448+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:43:33.099449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.822935+0000","last_change":"2024-03-29T08:20:12.437306+0000","last_active":"2024-03-29T08:20:13.822935+0000","last_peered":"2024-03-29T08:20:13.822935+0000","last_clean":"2024-03-29T08:20:13.822935+0000","last_became_active":"2024-03-29T08:20:12.437044+0000","last_became_peered":"2024-03-29T08:20:12.437044+0000","last_unstale":"2024-03-29T08:20:13.822935+0000","last_undegraded":"2024-03-29T08:20:13.822935+0000","last_fullsized":"2024-03-29T08:20:13.822935+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:09:21.154366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823853+0000","last_change":"2024-03-29T08:20:12.419515+0000","last_active":"2024-03-29T08:20:13.823853+0000","last_peered":"2024-03-29T08:20:13.823853+0000","last_clean":"2024-03-29T08:20:13.823853+0000","last_became_active":"2024-03-29T08:20:12.418873+0000","last_became_peered":"2024-03-29T08:20:12.418873+0000","last_unstale":"2024-03-29T08:20:13.823853+0000","last_undegraded":"2024-03-29T08:20:13.823853+0000","last_fullsized":"2024-03-29T08:20:13.823853+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:17:04.190507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823457+0000","last_change":"2024-03-29T08:20:12.431617+0000","last_active":"2024-03-29T08:20:13.823457+0000","last_peered":"2024-03-29T08:20:13.823457+0000","last_clean":"2024-03-29T08:20:13.823457+0000","last_became_active":"2024-03-29T08:20:12.430783+0000","last_became_peered":"2024-03-29T08:20:12.430783+0000","last_unstale":"2024-03-29T08:20:13.823457+0000","last_undegraded":"2024-03-29T08:20:13.823457+0000","last_fullsized":"2024-03-29T08:20:13.823457+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:06.183941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408697+0000","last_change":"2024-03-29T08:20:12.425409+0000","last_active":"2024-03-29T08:20:13.408697+0000","last_peered":"2024-03-29T08:20:13.408697+0000","last_clean":"2024-03-29T08:20:13.408697+0000","last_became_active":"2024-03-29T08:20:12.425173+0000","last_became_peered":"2024-03-29T08:20:12.425173+0000","last_unstale":"2024-03-29T08:20:13.408697+0000","last_undegraded":"2024-03-29T08:20:13.408697+0000","last_fullsized":"2024-03-29T08:20:13.408697+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:06:31.563790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408598+0000","last_change":"2024-03-29T08:20:12.423825+0000","last_active":"2024-03-29T08:20:13.408598+0000","last_peered":"2024-03-29T08:20:13.408598+0000","last_clean":"2024-03-29T08:20:13.408598+0000","last_became_active":"2024-03-29T08:20:12.423558+0000","last_became_peered":"2024-03-29T08:20:12.423558+0000","last_unstale":"2024-03-29T08:20:13.408598+0000","last_undegraded":"2024-03-29T08:20:13.408598+0000","last_fullsized":"2024-03-29T08:20:13.408598+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:41:36.745211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823203+0000","last_change":"2024-03-29T08:20:12.426763+0000","last_active":"2024-03-29T08:20:13.823203+0000","last_peered":"2024-03-29T08:20:13.823203+0000","last_clean":"2024-03-29T08:20:13.823203+0000","last_became_active":"2024-03-29T08:20:12.426496+0000","last_became_peered":"2024-03-29T08:20:12.426496+0000","last_unstale":"2024-03-29T08:20:13.823203+0000","last_undegraded":"2024-03-29T08:20:13.823203+0000","last_fullsized":"2024-03-29T08:20:13.823203+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:17:37.930817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.822902+0000","last_change":"2024-03-29T08:20:12.427375+0000","last_active":"2024-03-29T08:20:13.822902+0000","last_peered":"2024-03-29T08:20:13.822902+0000","last_clean":"2024-03-29T08:20:13.822902+0000","last_became_active":"2024-03-29T08:20:12.427061+0000","last_became_peered":"2024-03-29T08:20:12.427061+0000","last_unstale":"2024-03-29T08:20:13.822902+0000","last_undegraded":"2024-03-29T08:20:13.822902+0000","last_fullsized":"2024-03-29T08:20:13.822902+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:03:01.828915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823017+0000","last_change":"2024-03-29T08:20:12.435124+0000","last_active":"2024-03-29T08:20:13.823017+0000","last_peered":"2024-03-29T08:20:13.823017+0000","last_clean":"2024-03-29T08:20:13.823017+0000","last_became_active":"2024-03-29T08:20:12.434867+0000","last_became_peered":"2024-03-29T08:20:12.434867+0000","last_unstale":"2024-03-29T08:20:13.823017+0000","last_undegraded":"2024-03-29T08:20:13.823017+0000","last_fullsized":"2024-03-29T08:20:13.823017+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:23:02.288895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823731+0000","last_change":"2024-03-29T08:20:12.423256+0000","last_active":"2024-03-29T08:20:13.823731+0000","last_peered":"2024-03-29T08:20:13.823731+0000","last_clean":"2024-03-29T08:20:13.823731+0000","last_became_active":"2024-03-29T08:20:12.422967+0000","last_became_peered":"2024-03-29T08:20:12.422967+0000","last_unstale":"2024-03-29T08:20:13.823731+0000","last_undegraded":"2024-03-29T08:20:13.823731+0000","last_fullsized":"2024-03-29T08:20:13.823731+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:31:07.367334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.470227+0000","last_change":"2024-03-29T08:20:12.413552+0000","last_active":"2024-03-29T08:20:13.470227+0000","last_peered":"2024-03-29T08:20:13.470227+0000","last_clean":"2024-03-29T08:20:13.470227+0000","last_became_active":"2024-03-29T08:20:12.413236+0000","last_became_peered":"2024-03-29T08:20:12.413236+0000","last_unstale":"2024-03-29T08:20:13.470227+0000","last_undegraded":"2024-03-29T08:20:13.470227+0000","last_fullsized":"2024-03-29T08:20:13.470227+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:27.108985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"48'4","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823985+0000","last_change":"2024-03-29T08:20:12.018358+0000","last_active":"2024-03-29T08:20:13.823985+0000","last_peered":"2024-03-29T08:20:13.823985+0000","last_clean":"2024-03-29T08:20:13.823985+0000","last_became_active":"2024-03-29T08:20:12.018088+0000","last_became_peered":"2024-03-29T08:20:12.018088+0000","last_unstale":"2024-03-29T08:20:13.823985+0000","last_undegraded":"2024-03-29T08:20:13.823985+0000","last_fullsized":"2024-03-29T08:20:13.823985+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:03:50.843448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823832+0000","last_change":"2024-03-29T08:20:12.427585+0000","last_active":"2024-03-29T08:20:13.823832+0000","last_peered":"2024-03-29T08:20:13.823832+0000","last_clean":"2024-03-29T08:20:13.823832+0000","last_became_active":"2024-03-29T08:20:12.427358+0000","last_became_peered":"2024-03-29T08:20:12.427358+0000","last_unstale":"2024-03-29T08:20:13.823832+0000","last_undegraded":"2024-03-29T08:20:13.823832+0000","last_fullsized":"2024-03-29T08:20:13.823832+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:03:01.828915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408661+0000","last_change":"2024-03-29T08:20:12.012597+0000","last_active":"2024-03-29T08:20:13.408661+0000","last_peered":"2024-03-29T08:20:13.408661+0000","last_clean":"2024-03-29T08:20:13.408661+0000","last_became_active":"2024-03-29T08:20:12.012317+0000","last_became_peered":"2024-03-29T08:20:12.012317+0000","last_unstale":"2024-03-29T08:20:13.408661+0000","last_undegraded":"2024-03-29T08:20:13.408661+0000","last_fullsized":"2024-03-29T08:20:13.408661+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:42:31.516496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823311+0000","last_change":"2024-03-29T08:20:12.435704+0000","last_active":"2024-03-29T08:20:13.823311+0000","last_peered":"2024-03-29T08:20:13.823311+0000","last_clean":"2024-03-29T08:20:13.823311+0000","last_became_active":"2024-03-29T08:20:12.431038+0000","last_became_peered":"2024-03-29T08:20:12.431038+0000","last_unstale":"2024-03-29T08:20:13.823311+0000","last_undegraded":"2024-03-29T08:20:13.823311+0000","last_fullsized":"2024-03-29T08:20:13.823311+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:17:04.190507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823974+0000","last_change":"2024-03-29T08:20:12.018336+0000","last_active":"2024-03-29T08:20:13.823974+0000","last_peered":"2024-03-29T08:20:13.823974+0000","last_clean":"2024-03-29T08:20:13.823974+0000","last_became_active":"2024-03-29T08:20:12.018039+0000","last_became_peered":"2024-03-29T08:20:12.018039+0000","last_unstale":"2024-03-29T08:20:13.823974+0000","last_undegraded":"2024-03-29T08:20:13.823974+0000","last_fullsized":"2024-03-29T08:20:13.823974+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:55:32.286863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823929+0000","last_change":"2024-03-29T08:20:12.432800+0000","last_active":"2024-03-29T08:20:13.823929+0000","last_peered":"2024-03-29T08:20:13.823929+0000","last_clean":"2024-03-29T08:20:13.823929+0000","last_became_active":"2024-03-29T08:20:12.432474+0000","last_became_peered":"2024-03-29T08:20:12.432474+0000","last_unstale":"2024-03-29T08:20:13.823929+0000","last_undegraded":"2024-03-29T08:20:13.823929+0000","last_fullsized":"2024-03-29T08:20:13.823929+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:39:43.247395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408638+0000","last_change":"2024-03-29T08:20:12.012754+0000","last_active":"2024-03-29T08:20:13.408638+0000","last_peered":"2024-03-29T08:20:13.408638+0000","last_clean":"2024-03-29T08:20:13.408638+0000","last_became_active":"2024-03-29T08:20:12.012445+0000","last_became_peered":"2024-03-29T08:20:12.012445+0000","last_unstale":"2024-03-29T08:20:13.408638+0000","last_undegraded":"2024-03-29T08:20:13.408638+0000","last_fullsized":"2024-03-29T08:20:13.408638+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:10:00.416187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408438+0000","last_change":"2024-03-29T08:20:12.427236+0000","last_active":"2024-03-29T08:20:13.408438+0000","last_peered":"2024-03-29T08:20:13.408438+0000","last_clean":"2024-03-29T08:20:13.408438+0000","last_became_active":"2024-03-29T08:20:12.426971+0000","last_became_peered":"2024-03-29T08:20:12.426971+0000","last_unstale":"2024-03-29T08:20:13.408438+0000","last_undegraded":"2024-03-29T08:20:13.408438+0000","last_fullsized":"2024-03-29T08:20:13.408438+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:38:53.007190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823999+0000","last_change":"2024-03-29T08:20:12.019483+0000","last_active":"2024-03-29T08:20:13.823999+0000","last_peered":"2024-03-29T08:20:13.823999+0000","last_clean":"2024-03-29T08:20:13.823999+0000","last_became_active":"2024-03-29T08:20:12.019133+0000","last_became_peered":"2024-03-29T08:20:12.019133+0000","last_unstale":"2024-03-29T08:20:13.823999+0000","last_undegraded":"2024-03-29T08:20:13.823999+0000","last_fullsized":"2024-03-29T08:20:13.823999+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:47:05.848253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408927+0000","last_change":"2024-03-29T08:20:12.437431+0000","last_active":"2024-03-29T08:20:13.408927+0000","last_peered":"2024-03-29T08:20:13.408927+0000","last_clean":"2024-03-29T08:20:13.408927+0000","last_became_active":"2024-03-29T08:20:12.437089+0000","last_became_peered":"2024-03-29T08:20:12.437089+0000","last_unstale":"2024-03-29T08:20:13.408927+0000","last_undegraded":"2024-03-29T08:20:13.408927+0000","last_fullsized":"2024-03-29T08:20:13.408927+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:00:24.845050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408554+0000","last_change":"2024-03-29T08:20:12.009870+0000","last_active":"2024-03-29T08:20:13.408554+0000","last_peered":"2024-03-29T08:20:13.408554+0000","last_clean":"2024-03-29T08:20:13.408554+0000","last_became_active":"2024-03-29T08:20:12.009448+0000","last_became_peered":"2024-03-29T08:20:12.009448+0000","last_unstale":"2024-03-29T08:20:13.408554+0000","last_undegraded":"2024-03-29T08:20:13.408554+0000","last_fullsized":"2024-03-29T08:20:13.408554+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:48:19.548619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408497+0000","last_change":"2024-03-29T08:20:12.432064+0000","last_active":"2024-03-29T08:20:13.408497+0000","last_peered":"2024-03-29T08:20:13.408497+0000","last_clean":"2024-03-29T08:20:13.408497+0000","last_became_active":"2024-03-29T08:20:12.429253+0000","last_became_peered":"2024-03-29T08:20:12.429253+0000","last_unstale":"2024-03-29T08:20:13.408497+0000","last_undegraded":"2024-03-29T08:20:13.408497+0000","last_fullsized":"2024-03-29T08:20:13.408497+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:10:01.421888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823067+0000","last_change":"2024-03-29T08:20:12.018998+0000","last_active":"2024-03-29T08:20:13.823067+0000","last_peered":"2024-03-29T08:20:13.823067+0000","last_clean":"2024-03-29T08:20:13.823067+0000","last_became_active":"2024-03-29T08:20:12.018791+0000","last_became_peered":"2024-03-29T08:20:12.018791+0000","last_unstale":"2024-03-29T08:20:13.823067+0000","last_undegraded":"2024-03-29T08:20:13.823067+0000","last_fullsized":"2024-03-29T08:20:13.823067+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:03:50.843448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408343+0000","last_change":"2024-03-29T08:20:12.437175+0000","last_active":"2024-03-29T08:20:13.408343+0000","last_peered":"2024-03-29T08:20:13.408343+0000","last_clean":"2024-03-29T08:20:13.408343+0000","last_became_active":"2024-03-29T08:20:12.436905+0000","last_became_peered":"2024-03-29T08:20:12.436905+0000","last_unstale":"2024-03-29T08:20:13.408343+0000","last_undegraded":"2024-03-29T08:20:13.408343+0000","last_fullsized":"2024-03-29T08:20:13.408343+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:27:38.115643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408604+0000","last_change":"2024-03-29T08:20:12.015405+0000","last_active":"2024-03-29T08:20:13.408604+0000","last_peered":"2024-03-29T08:20:13.408604+0000","last_clean":"2024-03-29T08:20:13.408604+0000","last_became_active":"2024-03-29T08:20:12.015152+0000","last_became_peered":"2024-03-29T08:20:12.015152+0000","last_unstale":"2024-03-29T08:20:13.408604+0000","last_undegraded":"2024-03-29T08:20:13.408604+0000","last_fullsized":"2024-03-29T08:20:13.408604+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:58.679621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406843+0000","last_change":"2024-03-29T08:20:12.422275+0000","last_active":"2024-03-29T08:20:13.406843+0000","last_peered":"2024-03-29T08:20:13.406843+0000","last_clean":"2024-03-29T08:20:13.406843+0000","last_became_active":"2024-03-29T08:20:12.422016+0000","last_became_peered":"2024-03-29T08:20:12.422016+0000","last_unstale":"2024-03-29T08:20:13.406843+0000","last_undegraded":"2024-03-29T08:20:13.406843+0000","last_fullsized":"2024-03-29T08:20:13.406843+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:12:28.951099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406793+0000","last_change":"2024-03-29T08:20:12.011565+0000","last_active":"2024-03-29T08:20:13.406793+0000","last_peered":"2024-03-29T08:20:13.406793+0000","last_clean":"2024-03-29T08:20:13.406793+0000","last_became_active":"2024-03-29T08:20:12.011323+0000","last_became_peered":"2024-03-29T08:20:12.011323+0000","last_unstale":"2024-03-29T08:20:13.406793+0000","last_undegraded":"2024-03-29T08:20:13.406793+0000","last_fullsized":"2024-03-29T08:20:13.406793+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:22:49.764428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406759+0000","last_change":"2024-03-29T08:20:12.416154+0000","last_active":"2024-03-29T08:20:13.406759+0000","last_peered":"2024-03-29T08:20:13.406759+0000","last_clean":"2024-03-29T08:20:13.406759+0000","last_became_active":"2024-03-29T08:20:12.415899+0000","last_became_peered":"2024-03-29T08:20:12.415899+0000","last_unstale":"2024-03-29T08:20:13.406759+0000","last_undegraded":"2024-03-29T08:20:13.406759+0000","last_fullsized":"2024-03-29T08:20:13.406759+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:14:47.629162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.409086+0000","last_change":"2024-03-29T08:20:12.015929+0000","last_active":"2024-03-29T08:20:13.409086+0000","last_peered":"2024-03-29T08:20:13.409086+0000","last_clean":"2024-03-29T08:20:13.409086+0000","last_became_active":"2024-03-29T08:20:12.015333+0000","last_became_peered":"2024-03-29T08:20:12.015333+0000","last_unstale":"2024-03-29T08:20:13.409086+0000","last_undegraded":"2024-03-29T08:20:13.409086+0000","last_fullsized":"2024-03-29T08:20:13.409086+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:17:03.184805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823773+0000","last_change":"2024-03-29T08:20:12.435721+0000","last_active":"2024-03-29T08:20:13.823773+0000","last_peered":"2024-03-29T08:20:13.823773+0000","last_clean":"2024-03-29T08:20:13.823773+0000","last_became_active":"2024-03-29T08:20:12.432996+0000","last_became_peered":"2024-03-29T08:20:12.432996+0000","last_unstale":"2024-03-29T08:20:13.823773+0000","last_undegraded":"2024-03-29T08:20:13.823773+0000","last_fullsized":"2024-03-29T08:20:13.823773+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:34:51.649560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"48'3","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823322+0000","last_change":"2024-03-29T08:20:12.010462+0000","last_active":"2024-03-29T08:20:13.823322+0000","last_peered":"2024-03-29T08:20:13.823322+0000","last_clean":"2024-03-29T08:20:13.823322+0000","last_became_active":"2024-03-29T08:20:12.010170+0000","last_became_peered":"2024-03-29T08:20:12.010170+0000","last_unstale":"2024-03-29T08:20:13.823322+0000","last_undegraded":"2024-03-29T08:20:13.823322+0000","last_fullsized":"2024-03-29T08:20:13.823322+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:03:50.843448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408520+0000","last_change":"2024-03-29T08:20:12.440836+0000","last_active":"2024-03-29T08:20:13.408520+0000","last_peered":"2024-03-29T08:20:13.408520+0000","last_clean":"2024-03-29T08:20:13.408520+0000","last_became_active":"2024-03-29T08:20:12.440586+0000","last_became_peered":"2024-03-29T08:20:12.440586+0000","last_unstale":"2024-03-29T08:20:13.408520+0000","last_undegraded":"2024-03-29T08:20:13.408520+0000","last_fullsized":"2024-03-29T08:20:13.408520+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:55:32.718880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"48'6","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.409043+0000","last_change":"2024-03-29T08:20:12.010356+0000","last_active":"2024-03-29T08:20:13.409043+0000","last_peered":"2024-03-29T08:20:13.409043+0000","last_clean":"2024-03-29T08:20:13.409043+0000","last_became_active":"2024-03-29T08:20:12.010070+0000","last_became_peered":"2024-03-29T08:20:12.010070+0000","last_unstale":"2024-03-29T08:20:13.409043+0000","last_undegraded":"2024-03-29T08:20:13.409043+0000","last_fullsized":"2024-03-29T08:20:13.409043+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:09:20.148664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406718+0000","last_change":"2024-03-29T08:20:12.438799+0000","last_active":"2024-03-29T08:20:13.406718+0000","last_peered":"2024-03-29T08:20:13.406718+0000","last_clean":"2024-03-29T08:20:13.406718+0000","last_became_active":"2024-03-29T08:20:12.438409+0000","last_became_peered":"2024-03-29T08:20:12.438409+0000","last_unstale":"2024-03-29T08:20:13.406718+0000","last_undegraded":"2024-03-29T08:20:13.406718+0000","last_fullsized":"2024-03-29T08:20:13.406718+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:41.676339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.470179+0000","last_change":"2024-03-29T08:20:12.011291+0000","last_active":"2024-03-29T08:20:13.470179+0000","last_peered":"2024-03-29T08:20:13.470179+0000","last_clean":"2024-03-29T08:20:13.470179+0000","last_became_active":"2024-03-29T08:20:12.010954+0000","last_became_peered":"2024-03-29T08:20:12.010954+0000","last_unstale":"2024-03-29T08:20:13.470179+0000","last_undegraded":"2024-03-29T08:20:13.470179+0000","last_fullsized":"2024-03-29T08:20:13.470179+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:37.107521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.470108+0000","last_change":"2024-03-29T08:20:12.428277+0000","last_active":"2024-03-29T08:20:13.470108+0000","last_peered":"2024-03-29T08:20:13.470108+0000","last_clean":"2024-03-29T08:20:13.470108+0000","last_became_active":"2024-03-29T08:20:12.422163+0000","last_became_peered":"2024-03-29T08:20:12.422163+0000","last_unstale":"2024-03-29T08:20:13.470108+0000","last_undegraded":"2024-03-29T08:20:13.470108+0000","last_fullsized":"2024-03-29T08:20:13.470108+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:07.507985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823407+0000","last_change":"2024-03-29T08:20:12.012434+0000","last_active":"2024-03-29T08:20:13.823407+0000","last_peered":"2024-03-29T08:20:13.823407+0000","last_clean":"2024-03-29T08:20:13.823407+0000","last_became_active":"2024-03-29T08:20:12.012076+0000","last_became_peered":"2024-03-29T08:20:12.012076+0000","last_unstale":"2024-03-29T08:20:13.823407+0000","last_undegraded":"2024-03-29T08:20:13.823407+0000","last_fullsized":"2024-03-29T08:20:13.823407+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:11:08.174875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.469957+0000","last_change":"2024-03-29T08:20:12.428791+0000","last_active":"2024-03-29T08:20:13.469957+0000","last_peered":"2024-03-29T08:20:13.469957+0000","last_clean":"2024-03-29T08:20:13.469957+0000","last_became_active":"2024-03-29T08:20:12.428450+0000","last_became_peered":"2024-03-29T08:20:12.428450+0000","last_unstale":"2024-03-29T08:20:13.469957+0000","last_undegraded":"2024-03-29T08:20:13.469957+0000","last_fullsized":"2024-03-29T08:20:13.469957+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:53:42.654767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823112+0000","last_change":"2024-03-29T08:20:12.021393+0000","last_active":"2024-03-29T08:20:13.823112+0000","last_peered":"2024-03-29T08:20:13.823112+0000","last_clean":"2024-03-29T08:20:13.823112+0000","last_became_active":"2024-03-29T08:20:12.021116+0000","last_became_peered":"2024-03-29T08:20:12.021116+0000","last_unstale":"2024-03-29T08:20:13.823112+0000","last_undegraded":"2024-03-29T08:20:13.823112+0000","last_fullsized":"2024-03-29T08:20:13.823112+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:11:21.922581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.822854+0000","last_change":"2024-03-29T08:20:12.431560+0000","last_active":"2024-03-29T08:20:13.822854+0000","last_peered":"2024-03-29T08:20:13.822854+0000","last_clean":"2024-03-29T08:20:13.822854+0000","last_became_active":"2024-03-29T08:20:12.431308+0000","last_became_peered":"2024-03-29T08:20:12.431308+0000","last_unstale":"2024-03-29T08:20:13.822854+0000","last_undegraded":"2024-03-29T08:20:13.822854+0000","last_fullsized":"2024-03-29T08:20:13.822854+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:43:50.843484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"48'8","reported_seq":19,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408619+0000","last_change":"2024-03-29T08:20:12.007857+0000","last_active":"2024-03-29T08:20:13.408619+0000","last_peered":"2024-03-29T08:20:13.408619+0000","last_clean":"2024-03-29T08:20:13.408619+0000","last_became_active":"2024-03-29T08:20:12.007577+0000","last_became_peered":"2024-03-29T08:20:12.007577+0000","last_unstale":"2024-03-29T08:20:13.408619+0000","last_undegraded":"2024-03-29T08:20:13.408619+0000","last_fullsized":"2024-03-29T08:20:13.408619+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:17:36.925115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.470303+0000","last_change":"2024-03-29T08:20:12.428462+0000","last_active":"2024-03-29T08:20:13.470303+0000","last_peered":"2024-03-29T08:20:13.470303+0000","last_clean":"2024-03-29T08:20:13.470303+0000","last_became_active":"2024-03-29T08:20:12.425867+0000","last_became_peered":"2024-03-29T08:20:12.425867+0000","last_unstale":"2024-03-29T08:20:13.470303+0000","last_undegraded":"2024-03-29T08:20:13.470303+0000","last_fullsized":"2024-03-29T08:20:13.470303+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:26:26.576312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823450+0000","last_change":"2024-03-29T08:20:12.018885+0000","last_active":"2024-03-29T08:20:13.823450+0000","last_peered":"2024-03-29T08:20:13.823450+0000","last_clean":"2024-03-29T08:20:13.823450+0000","last_became_active":"2024-03-29T08:20:12.018553+0000","last_became_peered":"2024-03-29T08:20:12.018553+0000","last_unstale":"2024-03-29T08:20:13.823450+0000","last_undegraded":"2024-03-29T08:20:13.823450+0000","last_fullsized":"2024-03-29T08:20:13.823450+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:32:19.810173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.470172+0000","last_change":"2024-03-29T08:20:12.425777+0000","last_active":"2024-03-29T08:20:13.470172+0000","last_peered":"2024-03-29T08:20:13.470172+0000","last_clean":"2024-03-29T08:20:13.470172+0000","last_became_active":"2024-03-29T08:20:12.425438+0000","last_became_peered":"2024-03-29T08:20:12.425438+0000","last_unstale":"2024-03-29T08:20:13.470172+0000","last_undegraded":"2024-03-29T08:20:13.470172+0000","last_fullsized":"2024-03-29T08:20:13.470172+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:52:41.418480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406896+0000","last_change":"2024-03-29T08:20:12.019663+0000","last_active":"2024-03-29T08:20:13.406896+0000","last_peered":"2024-03-29T08:20:13.406896+0000","last_clean":"2024-03-29T08:20:13.406896+0000","last_became_active":"2024-03-29T08:20:12.016423+0000","last_became_peered":"2024-03-29T08:20:12.016423+0000","last_unstale":"2024-03-29T08:20:13.406896+0000","last_undegraded":"2024-03-29T08:20:13.406896+0000","last_fullsized":"2024-03-29T08:20:13.406896+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:19:44.116553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823392+0000","last_change":"2024-03-29T08:20:12.436125+0000","last_active":"2024-03-29T08:20:13.823392+0000","last_peered":"2024-03-29T08:20:13.823392+0000","last_clean":"2024-03-29T08:20:13.823392+0000","last_became_active":"2024-03-29T08:20:12.435846+0000","last_became_peered":"2024-03-29T08:20:12.435846+0000","last_unstale":"2024-03-29T08:20:13.823392+0000","last_undegraded":"2024-03-29T08:20:13.823392+0000","last_fullsized":"2024-03-29T08:20:13.823392+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:01:33.657226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406953+0000","last_change":"2024-03-29T08:20:12.018760+0000","last_active":"2024-03-29T08:20:13.406953+0000","last_peered":"2024-03-29T08:20:13.406953+0000","last_clean":"2024-03-29T08:20:13.406953+0000","last_became_active":"2024-03-29T08:20:12.018485+0000","last_became_peered":"2024-03-29T08:20:12.018485+0000","last_unstale":"2024-03-29T08:20:13.406953+0000","last_undegraded":"2024-03-29T08:20:13.406953+0000","last_fullsized":"2024-03-29T08:20:13.406953+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:27:28.312946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406918+0000","last_change":"2024-03-29T08:20:12.431691+0000","last_active":"2024-03-29T08:20:13.406918+0000","last_peered":"2024-03-29T08:20:13.406918+0000","last_clean":"2024-03-29T08:20:13.406918+0000","last_became_active":"2024-03-29T08:20:12.427172+0000","last_became_peered":"2024-03-29T08:20:12.427172+0000","last_unstale":"2024-03-29T08:20:13.406918+0000","last_undegraded":"2024-03-29T08:20:13.406918+0000","last_fullsized":"2024-03-29T08:20:13.406918+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:44:45.599624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823509+0000","last_change":"2024-03-29T08:20:12.021179+0000","last_active":"2024-03-29T08:20:13.823509+0000","last_peered":"2024-03-29T08:20:13.823509+0000","last_clean":"2024-03-29T08:20:13.823509+0000","last_became_active":"2024-03-29T08:20:12.020881+0000","last_became_peered":"2024-03-29T08:20:12.020881+0000","last_unstale":"2024-03-29T08:20:13.823509+0000","last_undegraded":"2024-03-29T08:20:13.823509+0000","last_fullsized":"2024-03-29T08:20:13.823509+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:41:38.371779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408769+0000","last_change":"2024-03-29T08:20:12.427190+0000","last_active":"2024-03-29T08:20:13.408769+0000","last_peered":"2024-03-29T08:20:13.408769+0000","last_clean":"2024-03-29T08:20:13.408769+0000","last_became_active":"2024-03-29T08:20:12.426912+0000","last_became_peered":"2024-03-29T08:20:12.426912+0000","last_unstale":"2024-03-29T08:20:13.408769+0000","last_undegraded":"2024-03-29T08:20:13.408769+0000","last_fullsized":"2024-03-29T08:20:13.408769+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:28.812148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.409033+0000","last_change":"2024-03-29T08:20:12.011532+0000","last_active":"2024-03-29T08:20:13.409033+0000","last_peered":"2024-03-29T08:20:13.409033+0000","last_clean":"2024-03-29T08:20:13.409033+0000","last_became_active":"2024-03-29T08:20:12.011280+0000","last_became_peered":"2024-03-29T08:20:12.011280+0000","last_unstale":"2024-03-29T08:20:13.409033+0000","last_undegraded":"2024-03-29T08:20:13.409033+0000","last_fullsized":"2024-03-29T08:20:13.409033+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:23:01.283193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823151+0000","last_change":"2024-03-29T08:20:12.424479+0000","last_active":"2024-03-29T08:20:13.823151+0000","last_peered":"2024-03-29T08:20:13.823151+0000","last_clean":"2024-03-29T08:20:13.823151+0000","last_became_active":"2024-03-29T08:20:12.423372+0000","last_became_peered":"2024-03-29T08:20:12.423372+0000","last_unstale":"2024-03-29T08:20:13.823151+0000","last_undegraded":"2024-03-29T08:20:13.823151+0000","last_fullsized":"2024-03-29T08:20:13.823151+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:17:04.190507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823517+0000","last_change":"2024-03-29T08:20:12.017378+0000","last_active":"2024-03-29T08:20:13.823517+0000","last_peered":"2024-03-29T08:20:13.823517+0000","last_clean":"2024-03-29T08:20:13.823517+0000","last_became_active":"2024-03-29T08:20:12.017137+0000","last_became_peered":"2024-03-29T08:20:12.017137+0000","last_unstale":"2024-03-29T08:20:13.823517+0000","last_undegraded":"2024-03-29T08:20:13.823517+0000","last_fullsized":"2024-03-29T08:20:13.823517+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:38:05.671123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823770+0000","last_change":"2024-03-29T08:20:12.428248+0000","last_active":"2024-03-29T08:20:13.823770+0000","last_peered":"2024-03-29T08:20:13.823770+0000","last_clean":"2024-03-29T08:20:13.823770+0000","last_became_active":"2024-03-29T08:20:12.427989+0000","last_became_peered":"2024-03-29T08:20:12.427989+0000","last_unstale":"2024-03-29T08:20:13.823770+0000","last_undegraded":"2024-03-29T08:20:13.823770+0000","last_fullsized":"2024-03-29T08:20:13.823770+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:37:40.314866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408596+0000","last_change":"2024-03-29T08:20:12.016530+0000","last_active":"2024-03-29T08:20:13.408596+0000","last_peered":"2024-03-29T08:20:13.408596+0000","last_clean":"2024-03-29T08:20:13.408596+0000","last_became_active":"2024-03-29T08:20:12.014858+0000","last_became_peered":"2024-03-29T08:20:12.014858+0000","last_unstale":"2024-03-29T08:20:13.408596+0000","last_undegraded":"2024-03-29T08:20:13.408596+0000","last_fullsized":"2024-03-29T08:20:13.408596+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:36:59.039140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823593+0000","last_change":"2024-03-29T08:20:12.433424+0000","last_active":"2024-03-29T08:20:13.823593+0000","last_peered":"2024-03-29T08:20:13.823593+0000","last_clean":"2024-03-29T08:20:13.823593+0000","last_became_active":"2024-03-29T08:20:12.431741+0000","last_became_peered":"2024-03-29T08:20:12.431741+0000","last_unstale":"2024-03-29T08:20:13.823593+0000","last_undegraded":"2024-03-29T08:20:13.823593+0000","last_fullsized":"2024-03-29T08:20:13.823593+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:27:34.663199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408317+0000","last_change":"2024-03-29T08:20:12.010852+0000","last_active":"2024-03-29T08:20:13.408317+0000","last_peered":"2024-03-29T08:20:13.408317+0000","last_clean":"2024-03-29T08:20:13.408317+0000","last_became_active":"2024-03-29T08:20:12.010602+0000","last_became_peered":"2024-03-29T08:20:12.010602+0000","last_unstale":"2024-03-29T08:20:13.408317+0000","last_undegraded":"2024-03-29T08:20:13.408317+0000","last_fullsized":"2024-03-29T08:20:13.408317+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:43:21.606807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408688+0000","last_change":"2024-03-29T08:20:12.429242+0000","last_active":"2024-03-29T08:20:13.408688+0000","last_peered":"2024-03-29T08:20:13.408688+0000","last_clean":"2024-03-29T08:20:13.408688+0000","last_became_active":"2024-03-29T08:20:12.428986+0000","last_became_peered":"2024-03-29T08:20:12.428986+0000","last_unstale":"2024-03-29T08:20:13.408688+0000","last_undegraded":"2024-03-29T08:20:13.408688+0000","last_fullsized":"2024-03-29T08:20:13.408688+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:58:06.775283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823609+0000","last_change":"2024-03-29T08:20:12.017438+0000","last_active":"2024-03-29T08:20:13.823609+0000","last_peered":"2024-03-29T08:20:13.823609+0000","last_clean":"2024-03-29T08:20:13.823609+0000","last_became_active":"2024-03-29T08:20:12.017132+0000","last_became_peered":"2024-03-29T08:20:12.017132+0000","last_unstale":"2024-03-29T08:20:13.823609+0000","last_undegraded":"2024-03-29T08:20:13.823609+0000","last_fullsized":"2024-03-29T08:20:13.823609+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:00:53.444271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.409150+0000","last_change":"2024-03-29T08:20:12.430375+0000","last_active":"2024-03-29T08:20:13.409150+0000","last_peered":"2024-03-29T08:20:13.409150+0000","last_clean":"2024-03-29T08:20:13.409150+0000","last_became_active":"2024-03-29T08:20:12.430021+0000","last_became_peered":"2024-03-29T08:20:12.430021+0000","last_unstale":"2024-03-29T08:20:13.409150+0000","last_undegraded":"2024-03-29T08:20:13.409150+0000","last_fullsized":"2024-03-29T08:20:13.409150+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:25:03.539726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823673+0000","last_change":"2024-03-29T08:20:12.018107+0000","last_active":"2024-03-29T08:20:13.823673+0000","last_peered":"2024-03-29T08:20:13.823673+0000","last_clean":"2024-03-29T08:20:13.823673+0000","last_became_active":"2024-03-29T08:20:12.017885+0000","last_became_peered":"2024-03-29T08:20:12.017885+0000","last_unstale":"2024-03-29T08:20:13.823673+0000","last_undegraded":"2024-03-29T08:20:13.823673+0000","last_fullsized":"2024-03-29T08:20:13.823673+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:11:21.922581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408627+0000","last_change":"2024-03-29T08:20:12.434892+0000","last_active":"2024-03-29T08:20:13.408627+0000","last_peered":"2024-03-29T08:20:13.408627+0000","last_clean":"2024-03-29T08:20:13.408627+0000","last_became_active":"2024-03-29T08:20:12.434621+0000","last_became_peered":"2024-03-29T08:20:12.434621+0000","last_unstale":"2024-03-29T08:20:13.408627+0000","last_undegraded":"2024-03-29T08:20:13.408627+0000","last_fullsized":"2024-03-29T08:20:13.408627+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:28.948477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.470189+0000","last_change":"2024-03-29T08:20:12.016011+0000","last_active":"2024-03-29T08:20:13.470189+0000","last_peered":"2024-03-29T08:20:13.470189+0000","last_clean":"2024-03-29T08:20:13.470189+0000","last_became_active":"2024-03-29T08:20:12.015613+0000","last_became_peered":"2024-03-29T08:20:12.015613+0000","last_unstale":"2024-03-29T08:20:13.470189+0000","last_undegraded":"2024-03-29T08:20:13.470189+0000","last_fullsized":"2024-03-29T08:20:13.470189+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:37.816754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408771+0000","last_change":"2024-03-29T08:20:12.426681+0000","last_active":"2024-03-29T08:20:13.408771+0000","last_peered":"2024-03-29T08:20:13.408771+0000","last_clean":"2024-03-29T08:20:13.408771+0000","last_became_active":"2024-03-29T08:20:12.426436+0000","last_became_peered":"2024-03-29T08:20:12.426436+0000","last_unstale":"2024-03-29T08:20:13.408771+0000","last_undegraded":"2024-03-29T08:20:13.408771+0000","last_fullsized":"2024-03-29T08:20:13.408771+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:09:27.388912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823545+0000","last_change":"2024-03-29T08:20:12.018569+0000","last_active":"2024-03-29T08:20:13.823545+0000","last_peered":"2024-03-29T08:20:13.823545+0000","last_clean":"2024-03-29T08:20:13.823545+0000","last_became_active":"2024-03-29T08:20:12.018354+0000","last_became_peered":"2024-03-29T08:20:12.018354+0000","last_unstale":"2024-03-29T08:20:13.823545+0000","last_undegraded":"2024-03-29T08:20:13.823545+0000","last_fullsized":"2024-03-29T08:20:13.823545+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:55:32.286863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823657+0000","last_change":"2024-03-29T08:20:12.430787+0000","last_active":"2024-03-29T08:20:13.823657+0000","last_peered":"2024-03-29T08:20:13.823657+0000","last_clean":"2024-03-29T08:20:13.823657+0000","last_became_active":"2024-03-29T08:20:12.430019+0000","last_became_peered":"2024-03-29T08:20:12.430019+0000","last_unstale":"2024-03-29T08:20:13.823657+0000","last_undegraded":"2024-03-29T08:20:13.823657+0000","last_fullsized":"2024-03-29T08:20:13.823657+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:49:17.040347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408554+0000","last_change":"2024-03-29T08:20:12.013272+0000","last_active":"2024-03-29T08:20:13.408554+0000","last_peered":"2024-03-29T08:20:13.408554+0000","last_clean":"2024-03-29T08:20:13.408554+0000","last_became_active":"2024-03-29T08:20:12.009964+0000","last_became_peered":"2024-03-29T08:20:12.009964+0000","last_unstale":"2024-03-29T08:20:13.408554+0000","last_undegraded":"2024-03-29T08:20:13.408554+0000","last_fullsized":"2024-03-29T08:20:13.408554+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:25:02.534024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406508+0000","last_change":"2024-03-29T08:20:12.435654+0000","last_active":"2024-03-29T08:20:13.406508+0000","last_peered":"2024-03-29T08:20:13.406508+0000","last_clean":"2024-03-29T08:20:13.406508+0000","last_became_active":"2024-03-29T08:20:12.435346+0000","last_became_peered":"2024-03-29T08:20:12.435346+0000","last_unstale":"2024-03-29T08:20:13.406508+0000","last_undegraded":"2024-03-29T08:20:13.406508+0000","last_fullsized":"2024-03-29T08:20:13.406508+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:52:41.418480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823121+0000","last_change":"2024-03-29T08:20:12.010149+0000","last_active":"2024-03-29T08:20:13.823121+0000","last_peered":"2024-03-29T08:20:13.823121+0000","last_clean":"2024-03-29T08:20:13.823121+0000","last_became_active":"2024-03-29T08:20:12.009873+0000","last_became_peered":"2024-03-29T08:20:12.009873+0000","last_unstale":"2024-03-29T08:20:13.823121+0000","last_undegraded":"2024-03-29T08:20:13.823121+0000","last_fullsized":"2024-03-29T08:20:13.823121+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:56:53.966923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.822953+0000","last_change":"2024-03-29T08:20:12.415800+0000","last_active":"2024-03-29T08:20:13.822953+0000","last_peered":"2024-03-29T08:20:13.822953+0000","last_clean":"2024-03-29T08:20:13.822953+0000","last_became_active":"2024-03-29T08:20:12.412788+0000","last_became_peered":"2024-03-29T08:20:12.412788+0000","last_unstale":"2024-03-29T08:20:13.822953+0000","last_undegraded":"2024-03-29T08:20:13.822953+0000","last_fullsized":"2024-03-29T08:20:13.822953+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:59:29.177991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.470121+0000","last_change":"2024-03-29T08:20:12.012788+0000","last_active":"2024-03-29T08:20:13.470121+0000","last_peered":"2024-03-29T08:20:13.470121+0000","last_clean":"2024-03-29T08:20:13.470121+0000","last_became_active":"2024-03-29T08:20:12.012542+0000","last_became_peered":"2024-03-29T08:20:12.012542+0000","last_unstale":"2024-03-29T08:20:13.470121+0000","last_undegraded":"2024-03-29T08:20:13.470121+0000","last_fullsized":"2024-03-29T08:20:13.470121+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:32:49.043368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823693+0000","last_change":"2024-03-29T08:20:12.427911+0000","last_active":"2024-03-29T08:20:13.823693+0000","last_peered":"2024-03-29T08:20:13.823693+0000","last_clean":"2024-03-29T08:20:13.823693+0000","last_became_active":"2024-03-29T08:20:12.427620+0000","last_became_peered":"2024-03-29T08:20:12.427620+0000","last_unstale":"2024-03-29T08:20:13.823693+0000","last_undegraded":"2024-03-29T08:20:13.823693+0000","last_fullsized":"2024-03-29T08:20:13.823693+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:08:40.657636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408716+0000","last_change":"2024-03-29T08:20:12.017983+0000","last_active":"2024-03-29T08:20:13.408716+0000","last_peered":"2024-03-29T08:20:13.408716+0000","last_clean":"2024-03-29T08:20:13.408716+0000","last_became_active":"2024-03-29T08:20:12.017785+0000","last_became_peered":"2024-03-29T08:20:12.017785+0000","last_unstale":"2024-03-29T08:20:13.408716+0000","last_undegraded":"2024-03-29T08:20:13.408716+0000","last_fullsized":"2024-03-29T08:20:13.408716+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:32:58.839816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":187,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-29T08:20:12.412429+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:20:12.412429+0000","last_peered":"2024-03-29T08:20:12.412429+0000","last_clean":"2024-03-29T08:20:12.412429+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:20:12.412429+0000","last_undegraded":"2024-03-29T08:20:12.412429+0000","last_fullsized":"2024-03-29T08:20:12.412429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":573440,"data_stored":231835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.65700000000000003,"15min":0.64500000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.67900000000000005,"15min":0.66900000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.75800000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.79300000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.63600000000000001,"15min":0.624},"min":{"1min":0.53000000000000003,"5min":0.503,"15min":0.503},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.66800000000000004,"15min":0.65600000000000003},"min":{"1min":0.57099999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.58499999999999996}]},{"osd":3,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.73199999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.70499999999999996,"15min":0.72099999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93000000000000005,"5min":1.123,"15min":1.123},"last":0.998}]},{"osd":5,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56999999999999995}]},{"osd":6,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72899999999999998,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.748,"15min":0.75600000000000001},"min":{"1min":0.46999999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.89300000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":577536,"data_stored":232417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.753},{"interface":"front","average":{"1min":0.754,"5min":0.69099999999999995,"15min":0.68100000000000005},"min":{"1min":0.52900000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.69999999999999996}]},{"osd":1,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61599999999999999,"15min":0.60599999999999998},"min":{"1min":0.438,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.61899999999999999},"min":{"1min":0.49199999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88900000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.71599999999999997}]},{"osd":2,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.68000000000000005,"15min":0.66800000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70299999999999996,"15min":0.69899999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.93200000000000005,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.77100000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70099999999999996,"15min":0.69599999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.51100000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.66800000000000004,"15min":0.67700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.96199999999999997,"5min":1.556,"15min":1.556},"last":0.505},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.72499999999999998,"15min":0.74299999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.93999999999999995,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.47599999999999998}]},{"osd":5,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.42899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92700000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.73099999999999998,"15min":0.73899999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.56799999999999995}]},{"osd":7,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.76200000000000001,"15min":0.77400000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97799999999999998,"5min":0.998,"15min":0.998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.77100000000000002,"15min":0.78500000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":1.024,"15min":1.024},"last":0.54800000000000004}]}]},{"osd":4,"up_from":15,"seq":64424509469,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":589824,"data_stored":233457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70499999999999996,"15min":0.70899999999999996},"min":{"1min":0.42499999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.014,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0489999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.70999999999999996}]},{"osd":1,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76100000000000001,"15min":0.76600000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.087,"5min":2.524,"15min":2.524},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.71399999999999997,"15min":0.71999999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91100000000000003,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.67400000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.498,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1060000000000001,"5min":2.363,"15min":2.363},"last":0.746},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76200000000000001,"15min":0.76100000000000001},"min":{"1min":0.54400000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.97599999999999998,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.83999999999999997}]},{"osd":3,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94299999999999995,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.81399999999999995,"15min":0.81899999999999995},"min":{"1min":0.63,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0700000000000001,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.89400000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.69499999999999995,"15min":0.70299999999999996},"min":{"1min":0.42499999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92400000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.45600000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.86799999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.82199999999999995}]},{"osd":6,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67200000000000004,"15min":0.67800000000000005},"min":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94899999999999995,"5min":0.996,"15min":0.996},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.51400000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.91500000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79400000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.89000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76200000000000001,"15min":0.77000000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.99199999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.92100000000000004}]}]},{"osd":3,"up_from":15,"seq":64424509472,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31696,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340144,"statfs":{"total":96636764160,"available":96604307456,"internally_reserved":0,"allocated":581632,"data_stored":233365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.45200000000000001,"15min":0.46200000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.61899999999999999,"5min":1.123,"15min":1.123},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.51400000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.32100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.52500000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.53700000000000003},"min":{"1min":0.32300000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.59299999999999997,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.55800000000000005,"15min":0.57399999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.65500000000000003,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.64500000000000002,"15min":0.66200000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.71199999999999997,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.78300000000000003,"15min":0.79900000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.071,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0609999999999999,"5min":1.391,"15min":1.391},"last":0.84299999999999997}]},{"osd":5,"last update":"Fri Mar 29 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.022,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.84899999999999998,"15min":0.86899999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.99399999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.77100000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.82199999999999995,"15min":0.84399999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.93000000000000005,"5min":1.669,"15min":1.669},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.81799999999999995,"15min":0.83699999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.081,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.68300000000000005}]},{"osd":7,"last update":"Fri Mar 29 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.044,"5min":1.887,"15min":1.887},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.84299999999999997,"15min":0.86799999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.01,"5min":2.044,"15min":2.044},"last":0.80300000000000005}]}]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":573440,"data_stored":231891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.504,"15min":0.50600000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.83399999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.316},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.46200000000000002}]},{"osd":1,"last update":"Fri Mar 29 08:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.497,"15min":0.497},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72899999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.502,"15min":0.51200000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72899999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.503}]},{"osd":3,"last update":"Fri Mar 29 08:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78200000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.57699999999999996}]},{"osd":4,"last update":"Fri Mar 29 08:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76100000000000001,"15min":0.76200000000000001},"min":{"1min":0.621,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.90600000000000003,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.76200000000000001,"15min":0.76700000000000002},"min":{"1min":0.502,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96099999999999997,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.435}]},{"osd":5,"last update":"Fri Mar 29 08:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.83499999999999996,"15min":0.84399999999999997},"min":{"1min":0.59399999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.997,"5min":1.306,"15min":1.306},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.85299999999999998,"15min":0.86499999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.008,"5min":1.526,"15min":1.526},"last":0.48799999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97699999999999998,"5min":1.095,"15min":1.095},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.78400000000000003,"15min":0.79300000000000004},"min":{"1min":0.623,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92700000000000005,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.68000000000000005}]},{"osd":7,"last update":"Fri Mar 29 08:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.56699999999999995,"5min":0.505,"15min":0.505},"max":{"1min":0.94399999999999995,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.626},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.58899999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.94699999999999995,"5min":1.153,"15min":1.153},"last":0.56499999999999995}]}]},{"osd":1,"up_from":15,"seq":64424509471,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1167360,"data_stored":822239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.61899999999999999,"15min":0.65200000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.58299999999999996,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.63600000000000001,"15min":0.66500000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.66100000000000003,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.29999999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.505,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64100000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.878,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.44}]},{"osd":3,"last update":"Fri Mar 29 08:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.44900000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.82399999999999995,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.38200000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.50900000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94599999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78200000000000003,"15min":0.79500000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98299999999999998,"5min":1.23,"15min":1.23},"last":0.78700000000000003}]},{"osd":5,"last update":"Fri Mar 29 08:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.77300000000000002,"15min":0.78300000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.93200000000000005,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.749,"15min":0.75900000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93999999999999995,"5min":1.107,"15min":1.107},"last":0.71799999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0049999999999999,"5min":1.026,"15min":1.026},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.85199999999999998,"15min":0.86799999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.02,"5min":1.196,"15min":1.196},"last":0.69399999999999995}]},{"osd":7,"last update":"Fri Mar 29 08:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.79200000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.86499999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.64300000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":577536,"data_stored":232473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.59599999999999997,"15min":0.628},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.61099999999999999,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.84099999999999997,"15min":0.90300000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.69499999999999995,"5min":8.0419999999999998,"15min":8.0419999999999998},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55600000000000005,"15min":0.56100000000000005},"min":{"1min":0.35899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72799999999999998,"5min":1.258,"15min":1.258},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.73299999999999998,"15min":0.76700000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78700000000000003,"5min":4.907,"15min":4.907},"last":0.47999999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.84199999999999997,"15min":0.90200000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.91000000000000003,"5min":7.9569999999999999,"15min":7.9569999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.65800000000000003,"15min":0.67700000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78300000000000003,"5min":1.397,"15min":1.397},"last":0.57699999999999996}]},{"osd":4,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.97999999999999998,"15min":1.022},"min":{"1min":0.61899999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94999999999999996,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.95499999999999996,"15min":0.99399999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.98399999999999999,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.71499999999999997}]},{"osd":5,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.92200000000000004,"15min":0.95399999999999996},"min":{"1min":0.56799999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.97899999999999998,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.85699999999999998,"15min":0.879},"min":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.98199999999999998,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.878}]},{"osd":6,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.85899999999999999,"15min":0.877},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96299999999999997,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.97799999999999998,"15min":1.016},"min":{"1min":0.54900000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.01,"5min":4.4100000000000001,"15min":4.4100000000000001},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.99299999999999999,"15min":1.0369999999999999},"min":{"1min":0.56999999999999995,"5min":0.496,"15min":0.496},"max":{"1min":1.0980000000000001,"5min":4.5209999999999999,"15min":4.5209999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.97699999999999998,"15min":1.0189999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.91300000000000003,"5min":4.4420000000000002,"15min":4.4420000000000002},"last":0.76300000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542175,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1167360,"data_stored":822191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.628,"15min":0.627},"min":{"1min":0.47099999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85199999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83399999999999996,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.72899999999999998,"15min":0.74199999999999999},"min":{"1min":0.48799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89700000000000002,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.5,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.621}]},{"osd":2,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69699999999999995,"15min":0.70099999999999996},"min":{"1min":0.47299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80500000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73899999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.56399999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.872,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.48499999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.88100000000000001,"5min":1.954,"15min":1.954},"last":0.68400000000000005}]},{"osd":4,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.64300000000000002,"15min":0.65100000000000002},"min":{"1min":0.42199999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.64900000000000002,"15min":0.66200000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72999999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67800000000000005,"15min":0.68600000000000005},"min":{"1min":0.443,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86899999999999999,"5min":1.071,"15min":1.071},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.69199999999999995,"15min":0.70099999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.85599999999999998,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.86899999999999999}]},{"osd":7,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.70899999999999996,"15min":0.71999999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84699999999999998,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.746,"15min":0.76000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90300000000000002,"5min":1.04,"15min":1.04},"last":0.85699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:20:16.517 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:20:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.691+0000 7f05b4200640 1 Processor -- start 2024-03-29T08:20:16.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.691+0000 7f05b4200640 1 -- start start 2024-03-29T08:20:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.695+0000 7f05b4200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f05ac15c9d0 0x7f05ac0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.695+0000 7f05b4200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ac0d0030 con 0x7f05ac099150 2024-03-29T08:20:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.695+0000 7f05b4200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ac0cf420 con 0x7f05ac0cef60 2024-03-29T08:20:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.695+0000 7f05b4200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ac0cf700 con 0x7f05ac15c9d0 2024-03-29T08:20:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.695+0000 7f05b2200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f05ac099150 0x7f05ac166a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59352/0 (socket says 172.21.15.46:59352) 2024-03-29T08:20:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.695+0000 7f05b2200640 1 -- 172.21.15.46:0/2029566272 learned_addr learned my addr 172.21.15.46:0/2029566272 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:16.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.695+0000 7f05b1800640 1 --2- 172.21.15.46:0/2029566272 >> v2:172.21.15.73:3300/0 conn(0x7f05ac15c9d0 0x7f05ac0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.695+0000 7f05abe00640 1 -- 172.21.15.46:0/2029566272 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1413308050 0 0) 0x7f05ac0d0030 con 0x7f05ac099150 2024-03-29T08:20:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.695+0000 7f05abe00640 1 -- 172.21.15.46:0/2029566272 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0594003530 con 0x7f05ac099150 2024-03-29T08:20:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.695+0000 7f05b1800640 1 -- 172.21.15.46:0/2029566272 >> v1:172.21.15.73:6789/0 conn(0x7f05ac0cef60 legacy=0x7f05ac098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.695+0000 7f05b1800640 1 -- 172.21.15.46:0/2029566272 >> v1:172.21.15.46:6789/0 conn(0x7f05ac099150 legacy=0x7f05ac166a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:16.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.695+0000 7f05b1800640 1 -- 172.21.15.46:0/2029566272 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac0cf9e0 con 0x7f05ac15c9d0 2024-03-29T08:20:16.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05b1800640 1 --2- 172.21.15.46:0/2029566272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05ac15c9d0 0x7f05ac0cea20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f05a00097c0 tx=0x7f05a001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d20b516e01f43bbd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05abe00640 1 -- 172.21.15.46:0/2029566272 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a00aa5f0 con 0x7f05ac15c9d0 2024-03-29T08:20:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05abe00640 1 -- 172.21.15.46:0/2029566272 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a00aa750 con 0x7f05ac15c9d0 2024-03-29T08:20:16.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05abe00640 1 -- 172.21.15.46:0/2029566272 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a00aaae0 con 0x7f05ac15c9d0 2024-03-29T08:20:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05b4200640 1 -- 172.21.15.46:0/2029566272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05ac15c9d0 msgr2=0x7f05ac0cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05b4200640 1 --2- 172.21.15.46:0/2029566272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05ac15c9d0 0x7f05ac0cea20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f05a00097c0 tx=0x7f05a001c470 comp rx=0 tx=0).stop 2024-03-29T08:20:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05b4200640 1 -- 172.21.15.46:0/2029566272 shutdown_connections 2024-03-29T08:20:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05b4200640 1 --2- 172.21.15.46:0/2029566272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05ac15c9d0 0x7f05ac0cea20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05b4200640 1 -- 172.21.15.46:0/2029566272 >> 172.21.15.46:0/2029566272 conn(0x7f05ac0b21a0 msgr2=0x7f05ac0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05b4200640 1 -- 172.21.15.46:0/2029566272 shutdown_connections 2024-03-29T08:20:16.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05b4200640 1 -- 172.21.15.46:0/2029566272 wait complete. 2024-03-29T08:20:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05b4200640 1 Processor -- start 2024-03-29T08:20:16.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05b4200640 1 -- start start 2024-03-29T08:20:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05b4200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05ac099150 0x7f05ac0b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05b4200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05ac0cef60 0x7f05ac15b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05b4200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ac15c9d0 0x7f05ac154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05b4200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0b4860 con 0x7f05ac15c9d0 2024-03-29T08:20:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05b4200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0b6090 con 0x7f05ac0cef60 2024-03-29T08:20:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.699+0000 7f05b4200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac0b4ab0 con 0x7f05ac099150 2024-03-29T08:20:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05b2200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ac15c9d0 0x7f05ac154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05b2200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ac15c9d0 0x7f05ac154bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47300/0 (socket says 172.21.15.46:47300) 2024-03-29T08:20:16.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05b2200640 1 -- 172.21.15.46:0/3348085716 learned_addr learned my addr 172.21.15.46:0/3348085716 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05b1800640 1 --2- 172.21.15.46:0/3348085716 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05ac099150 0x7f05ac0b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05b0e00640 1 --2- 172.21.15.46:0/3348085716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05ac0cef60 0x7f05ac15b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05b2200640 1 -- 172.21.15.46:0/3348085716 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05ac099150 msgr2=0x7f05ac0b5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05b2200640 1 --2- 172.21.15.46:0/3348085716 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05ac099150 0x7f05ac0b5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05b2200640 1 -- 172.21.15.46:0/3348085716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05ac0cef60 msgr2=0x7f05ac15b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05b2200640 1 --2- 172.21.15.46:0/3348085716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05ac0cef60 0x7f05ac15b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05b2200640 1 -- 172.21.15.46:0/3348085716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a0009420 con 0x7f05ac15c9d0 2024-03-29T08:20:16.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05b2200640 1 --2- 172.21.15.46:0/3348085716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ac15c9d0 0x7f05ac154bd0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f05a400cae0 tx=0x7f05a400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05a9600640 1 -- 172.21.15.46:0/3348085716 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a400ac40 con 0x7f05ac15c9d0 2024-03-29T08:20:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05a9600640 1 -- 172.21.15.46:0/3348085716 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a400ada0 con 0x7f05ac15c9d0 2024-03-29T08:20:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05a9600640 1 -- 172.21.15.46:0/3348085716 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a4012640 con 0x7f05ac15c9d0 2024-03-29T08:20:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05b4200640 1 -- 172.21.15.46:0/3348085716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ac0b4d90 con 0x7f05ac15c9d0 2024-03-29T08:20:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05b4200640 1 -- 172.21.15.46:0/3348085716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac0b5350 con 0x7f05ac15c9d0 2024-03-29T08:20:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05b4200640 1 -- 172.21.15.46:0/3348085716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ac1585b0 con 0x7f05ac15c9d0 2024-03-29T08:20:16.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05a9600640 1 -- 172.21.15.46:0/3348085716 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f05a4007070 con 0x7f05ac15c9d0 2024-03-29T08:20:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.703+0000 7f05a9600640 1 --2- 172.21.15.46:0/3348085716 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f058c066690 0x7f058c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.707+0000 7f05a9600640 1 -- 172.21.15.46:0/3348085716 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05a4084090 con 0x7f05ac15c9d0 2024-03-29T08:20:16.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.707+0000 7f05b1800640 1 --2- 172.21.15.46:0/3348085716 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f058c066690 0x7f058c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:16.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.707+0000 7f05b1800640 1 --2- 172.21.15.46:0/3348085716 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f058c066690 0x7f058c068b50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f05a00ae000 tx=0x7f05a00a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:16.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.707+0000 7f05a9600640 1 -- 172.21.15.46:0/3348085716 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f05a40514a0 con 0x7f05ac15c9d0 2024-03-29T08:20:16.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.871+0000 7f05b4200640 1 -- 172.21.15.46:0/3348085716 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05ac06b330 con 0x7f058c066690 2024-03-29T08:20:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.879+0000 7f05a9600640 1 -- 172.21.15.46:0/3348085716 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289123 (secure 0 0 0) 0x7f05a00aa8c0 con 0x7f058c066690 2024-03-29T08:20:16.882 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:20:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.883+0000 7f05b4200640 1 -- 172.21.15.46:0/3348085716 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f058c066690 msgr2=0x7f058c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.883+0000 7f05b4200640 1 --2- 172.21.15.46:0/3348085716 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f058c066690 0x7f058c068b50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f05a00ae000 tx=0x7f05a00a7000 comp rx=0 tx=0).stop 2024-03-29T08:20:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.883+0000 7f05b4200640 1 -- 172.21.15.46:0/3348085716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ac15c9d0 msgr2=0x7f05ac154bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.883+0000 7f05b4200640 1 --2- 172.21.15.46:0/3348085716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ac15c9d0 0x7f05ac154bd0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f05a400cae0 tx=0x7f05a400a060 comp rx=0 tx=0).stop 2024-03-29T08:20:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.883+0000 7f05b4200640 1 -- 172.21.15.46:0/3348085716 shutdown_connections 2024-03-29T08:20:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.883+0000 7f05b4200640 1 --2- 172.21.15.46:0/3348085716 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f058c066690 0x7f058c068b50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.883+0000 7f05b4200640 1 --2- 172.21.15.46:0/3348085716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05ac0cef60 0x7f05ac15b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.883+0000 7f05b4200640 1 --2- 172.21.15.46:0/3348085716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05ac15c9d0 0x7f05ac154bd0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.883+0000 7f05b4200640 1 --2- 172.21.15.46:0/3348085716 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05ac099150 0x7f05ac0b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:16.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.883+0000 7f05b4200640 1 -- 172.21.15.46:0/3348085716 >> 172.21.15.46:0/3348085716 conn(0x7f05ac0b21a0 msgr2=0x7f05ac0cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:16.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.883+0000 7f05b4200640 1 -- 172.21.15.46:0/3348085716 shutdown_connections 2024-03-29T08:20:16.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:16.883+0000 7f05b4200640 1 -- 172.21.15.46:0/3348085716 wait complete. 2024-03-29T08:20:16.903 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-03-29T08:20:16.069770+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242360,"kb_used_data":5672,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754732360,"statfs":{"total":773094113280,"available":772845936640,"internally_reserved":0,"allocated":5808128,"data_stored":3039868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":242208313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.082492"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823754+0000","last_change":"2024-03-29T08:20:12.419302+0000","last_active":"2024-03-29T08:20:13.823754+0000","last_peered":"2024-03-29T08:20:13.823754+0000","last_clean":"2024-03-29T08:20:13.823754+0000","last_became_active":"2024-03-29T08:20:12.418550+0000","last_became_peered":"2024-03-29T08:20:12.418550+0000","last_unstale":"2024-03-29T08:20:13.823754+0000","last_undegraded":"2024-03-29T08:20:13.823754+0000","last_fullsized":"2024-03-29T08:20:13.823754+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:37:00.044842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823358+0000","last_change":"2024-03-29T08:20:12.437537+0000","last_active":"2024-03-29T08:20:13.823358+0000","last_peered":"2024-03-29T08:20:13.823358+0000","last_clean":"2024-03-29T08:20:13.823358+0000","last_became_active":"2024-03-29T08:20:12.430681+0000","last_became_peered":"2024-03-29T08:20:12.430681+0000","last_unstale":"2024-03-29T08:20:13.823358+0000","last_undegraded":"2024-03-29T08:20:13.823358+0000","last_fullsized":"2024-03-29T08:20:13.823358+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:37:00.044842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406798+0000","last_change":"2024-03-29T08:20:12.433309+0000","last_active":"2024-03-29T08:20:13.406798+0000","last_peered":"2024-03-29T08:20:13.406798+0000","last_clean":"2024-03-29T08:20:13.406798+0000","last_became_active":"2024-03-29T08:20:12.433048+0000","last_became_peered":"2024-03-29T08:20:12.433048+0000","last_unstale":"2024-03-29T08:20:13.406798+0000","last_undegraded":"2024-03-29T08:20:13.406798+0000","last_fullsized":"2024-03-29T08:20:13.406798+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:05:35.885318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823826+0000","last_change":"2024-03-29T08:20:12.431343+0000","last_active":"2024-03-29T08:20:13.823826+0000","last_peered":"2024-03-29T08:20:13.823826+0000","last_clean":"2024-03-29T08:20:13.823826+0000","last_became_active":"2024-03-29T08:20:12.430381+0000","last_became_peered":"2024-03-29T08:20:12.430381+0000","last_unstale":"2024-03-29T08:20:13.823826+0000","last_undegraded":"2024-03-29T08:20:13.823826+0000","last_fullsized":"2024-03-29T08:20:13.823826+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:12:57.396667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823109+0000","last_change":"2024-03-29T08:20:12.430861+0000","last_active":"2024-03-29T08:20:13.823109+0000","last_peered":"2024-03-29T08:20:13.823109+0000","last_clean":"2024-03-29T08:20:13.823109+0000","last_became_active":"2024-03-29T08:20:12.430619+0000","last_became_peered":"2024-03-29T08:20:12.430619+0000","last_unstale":"2024-03-29T08:20:13.823109+0000","last_undegraded":"2024-03-29T08:20:13.823109+0000","last_fullsized":"2024-03-29T08:20:13.823109+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:09:21.154366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823355+0000","last_change":"2024-03-29T08:20:12.439025+0000","last_active":"2024-03-29T08:20:13.823355+0000","last_peered":"2024-03-29T08:20:13.823355+0000","last_clean":"2024-03-29T08:20:13.823355+0000","last_became_active":"2024-03-29T08:20:12.438753+0000","last_became_peered":"2024-03-29T08:20:12.438753+0000","last_unstale":"2024-03-29T08:20:13.823355+0000","last_undegraded":"2024-03-29T08:20:13.823355+0000","last_fullsized":"2024-03-29T08:20:13.823355+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:55:17.168641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408321+0000","last_change":"2024-03-29T08:20:12.434732+0000","last_active":"2024-03-29T08:20:13.408321+0000","last_peered":"2024-03-29T08:20:13.408321+0000","last_clean":"2024-03-29T08:20:13.408321+0000","last_became_active":"2024-03-29T08:20:12.434504+0000","last_became_peered":"2024-03-29T08:20:12.434504+0000","last_unstale":"2024-03-29T08:20:13.408321+0000","last_undegraded":"2024-03-29T08:20:13.408321+0000","last_fullsized":"2024-03-29T08:20:13.408321+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:00:21.157051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823293+0000","last_change":"2024-03-29T08:20:12.432677+0000","last_active":"2024-03-29T08:20:13.823293+0000","last_peered":"2024-03-29T08:20:13.823293+0000","last_clean":"2024-03-29T08:20:13.823293+0000","last_became_active":"2024-03-29T08:20:12.431818+0000","last_became_peered":"2024-03-29T08:20:12.431818+0000","last_unstale":"2024-03-29T08:20:13.823293+0000","last_undegraded":"2024-03-29T08:20:13.823293+0000","last_fullsized":"2024-03-29T08:20:13.823293+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:04:28.146272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823919+0000","last_change":"2024-03-29T08:20:12.426774+0000","last_active":"2024-03-29T08:20:13.823919+0000","last_peered":"2024-03-29T08:20:13.823919+0000","last_clean":"2024-03-29T08:20:13.823919+0000","last_became_active":"2024-03-29T08:20:12.426540+0000","last_became_peered":"2024-03-29T08:20:12.426540+0000","last_unstale":"2024-03-29T08:20:13.823919+0000","last_undegraded":"2024-03-29T08:20:13.823919+0000","last_fullsized":"2024-03-29T08:20:13.823919+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:36:35.453659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.409123+0000","last_change":"2024-03-29T08:20:12.422005+0000","last_active":"2024-03-29T08:20:13.409123+0000","last_peered":"2024-03-29T08:20:13.409123+0000","last_clean":"2024-03-29T08:20:13.409123+0000","last_became_active":"2024-03-29T08:20:12.421722+0000","last_became_peered":"2024-03-29T08:20:12.421722+0000","last_unstale":"2024-03-29T08:20:13.409123+0000","last_undegraded":"2024-03-29T08:20:13.409123+0000","last_fullsized":"2024-03-29T08:20:13.409123+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:42:32.522198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823691+0000","last_change":"2024-03-29T08:20:12.431756+0000","last_active":"2024-03-29T08:20:13.823691+0000","last_peered":"2024-03-29T08:20:13.823691+0000","last_clean":"2024-03-29T08:20:13.823691+0000","last_became_active":"2024-03-29T08:20:12.431044+0000","last_became_peered":"2024-03-29T08:20:12.431044+0000","last_unstale":"2024-03-29T08:20:13.823691+0000","last_undegraded":"2024-03-29T08:20:13.823691+0000","last_fullsized":"2024-03-29T08:20:13.823691+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:28:16.416846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823236+0000","last_change":"2024-03-29T08:20:12.420989+0000","last_active":"2024-03-29T08:20:13.823236+0000","last_peered":"2024-03-29T08:20:13.823236+0000","last_clean":"2024-03-29T08:20:13.823236+0000","last_became_active":"2024-03-29T08:20:12.420743+0000","last_became_peered":"2024-03-29T08:20:12.420743+0000","last_unstale":"2024-03-29T08:20:13.823236+0000","last_undegraded":"2024-03-29T08:20:13.823236+0000","last_fullsized":"2024-03-29T08:20:13.823236+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:59.685322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823139+0000","last_change":"2024-03-29T08:20:12.439238+0000","last_active":"2024-03-29T08:20:13.823139+0000","last_peered":"2024-03-29T08:20:13.823139+0000","last_clean":"2024-03-29T08:20:13.823139+0000","last_became_active":"2024-03-29T08:20:12.438895+0000","last_became_peered":"2024-03-29T08:20:12.438895+0000","last_unstale":"2024-03-29T08:20:13.823139+0000","last_undegraded":"2024-03-29T08:20:13.823139+0000","last_fullsized":"2024-03-29T08:20:13.823139+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:59.685322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823619+0000","last_change":"2024-03-29T08:20:12.433228+0000","last_active":"2024-03-29T08:20:13.823619+0000","last_peered":"2024-03-29T08:20:13.823619+0000","last_clean":"2024-03-29T08:20:13.823619+0000","last_became_active":"2024-03-29T08:20:12.432959+0000","last_became_peered":"2024-03-29T08:20:12.432959+0000","last_unstale":"2024-03-29T08:20:13.823619+0000","last_undegraded":"2024-03-29T08:20:13.823619+0000","last_fullsized":"2024-03-29T08:20:13.823619+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:53:46.664716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823905+0000","last_change":"2024-03-29T08:20:12.431541+0000","last_active":"2024-03-29T08:20:13.823905+0000","last_peered":"2024-03-29T08:20:13.823905+0000","last_clean":"2024-03-29T08:20:13.823905+0000","last_became_active":"2024-03-29T08:20:12.431200+0000","last_became_peered":"2024-03-29T08:20:12.431200+0000","last_unstale":"2024-03-29T08:20:13.823905+0000","last_undegraded":"2024-03-29T08:20:13.823905+0000","last_fullsized":"2024-03-29T08:20:13.823905+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:40:51.379844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.822784+0000","last_change":"2024-03-29T08:20:12.434244+0000","last_active":"2024-03-29T08:20:13.822784+0000","last_peered":"2024-03-29T08:20:13.822784+0000","last_clean":"2024-03-29T08:20:13.822784+0000","last_became_active":"2024-03-29T08:20:12.433958+0000","last_became_peered":"2024-03-29T08:20:12.433958+0000","last_unstale":"2024-03-29T08:20:13.822784+0000","last_undegraded":"2024-03-29T08:20:13.822784+0000","last_fullsized":"2024-03-29T08:20:13.822784+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:43:50.843484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823301+0000","last_change":"2024-03-29T08:20:12.422488+0000","last_active":"2024-03-29T08:20:13.823301+0000","last_peered":"2024-03-29T08:20:13.823301+0000","last_clean":"2024-03-29T08:20:13.823301+0000","last_became_active":"2024-03-29T08:20:12.422205+0000","last_became_peered":"2024-03-29T08:20:12.422205+0000","last_unstale":"2024-03-29T08:20:13.823301+0000","last_undegraded":"2024-03-29T08:20:13.823301+0000","last_fullsized":"2024-03-29T08:20:13.823301+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:17:37.930817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406813+0000","last_change":"2024-03-29T08:20:12.435506+0000","last_active":"2024-03-29T08:20:13.406813+0000","last_peered":"2024-03-29T08:20:13.406813+0000","last_clean":"2024-03-29T08:20:13.406813+0000","last_became_active":"2024-03-29T08:20:12.435209+0000","last_became_peered":"2024-03-29T08:20:12.435209+0000","last_unstale":"2024-03-29T08:20:13.406813+0000","last_undegraded":"2024-03-29T08:20:13.406813+0000","last_fullsized":"2024-03-29T08:20:13.406813+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:07.507985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823638+0000","last_change":"2024-03-29T08:20:12.428939+0000","last_active":"2024-03-29T08:20:13.823638+0000","last_peered":"2024-03-29T08:20:13.823638+0000","last_clean":"2024-03-29T08:20:13.823638+0000","last_became_active":"2024-03-29T08:20:12.428490+0000","last_became_peered":"2024-03-29T08:20:12.428490+0000","last_unstale":"2024-03-29T08:20:13.823638+0000","last_undegraded":"2024-03-29T08:20:13.823638+0000","last_fullsized":"2024-03-29T08:20:13.823638+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:01:33.657226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408945+0000","last_change":"2024-03-29T08:20:12.427514+0000","last_active":"2024-03-29T08:20:13.408945+0000","last_peered":"2024-03-29T08:20:13.408945+0000","last_clean":"2024-03-29T08:20:13.408945+0000","last_became_active":"2024-03-29T08:20:12.425982+0000","last_became_peered":"2024-03-29T08:20:12.425982+0000","last_unstale":"2024-03-29T08:20:13.408945+0000","last_undegraded":"2024-03-29T08:20:13.408945+0000","last_fullsized":"2024-03-29T08:20:13.408945+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:43:22.612508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823880+0000","last_change":"2024-03-29T08:20:12.429011+0000","last_active":"2024-03-29T08:20:13.823880+0000","last_peered":"2024-03-29T08:20:13.823880+0000","last_clean":"2024-03-29T08:20:13.823880+0000","last_became_active":"2024-03-29T08:20:12.428723+0000","last_became_peered":"2024-03-29T08:20:12.428723+0000","last_unstale":"2024-03-29T08:20:13.823880+0000","last_undegraded":"2024-03-29T08:20:13.823880+0000","last_fullsized":"2024-03-29T08:20:13.823880+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:43:12.128937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.470008+0000","last_change":"2024-03-29T08:20:12.414219+0000","last_active":"2024-03-29T08:20:13.470008+0000","last_peered":"2024-03-29T08:20:13.470008+0000","last_clean":"2024-03-29T08:20:13.470008+0000","last_became_active":"2024-03-29T08:20:12.413962+0000","last_became_peered":"2024-03-29T08:20:12.413962+0000","last_unstale":"2024-03-29T08:20:13.470008+0000","last_undegraded":"2024-03-29T08:20:13.470008+0000","last_fullsized":"2024-03-29T08:20:13.470008+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:28:03.339574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408448+0000","last_change":"2024-03-29T08:20:12.428957+0000","last_active":"2024-03-29T08:20:13.408448+0000","last_peered":"2024-03-29T08:20:13.408448+0000","last_clean":"2024-03-29T08:20:13.408448+0000","last_became_active":"2024-03-29T08:20:12.428685+0000","last_became_peered":"2024-03-29T08:20:12.428685+0000","last_unstale":"2024-03-29T08:20:13.408448+0000","last_undegraded":"2024-03-29T08:20:13.408448+0000","last_fullsized":"2024-03-29T08:20:13.408448+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:43:33.099449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.822935+0000","last_change":"2024-03-29T08:20:12.437306+0000","last_active":"2024-03-29T08:20:13.822935+0000","last_peered":"2024-03-29T08:20:13.822935+0000","last_clean":"2024-03-29T08:20:13.822935+0000","last_became_active":"2024-03-29T08:20:12.437044+0000","last_became_peered":"2024-03-29T08:20:12.437044+0000","last_unstale":"2024-03-29T08:20:13.822935+0000","last_undegraded":"2024-03-29T08:20:13.822935+0000","last_fullsized":"2024-03-29T08:20:13.822935+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:09:21.154366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823853+0000","last_change":"2024-03-29T08:20:12.419515+0000","last_active":"2024-03-29T08:20:13.823853+0000","last_peered":"2024-03-29T08:20:13.823853+0000","last_clean":"2024-03-29T08:20:13.823853+0000","last_became_active":"2024-03-29T08:20:12.418873+0000","last_became_peered":"2024-03-29T08:20:12.418873+0000","last_unstale":"2024-03-29T08:20:13.823853+0000","last_undegraded":"2024-03-29T08:20:13.823853+0000","last_fullsized":"2024-03-29T08:20:13.823853+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:17:04.190507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823457+0000","last_change":"2024-03-29T08:20:12.431617+0000","last_active":"2024-03-29T08:20:13.823457+0000","last_peered":"2024-03-29T08:20:13.823457+0000","last_clean":"2024-03-29T08:20:13.823457+0000","last_became_active":"2024-03-29T08:20:12.430783+0000","last_became_peered":"2024-03-29T08:20:12.430783+0000","last_unstale":"2024-03-29T08:20:13.823457+0000","last_undegraded":"2024-03-29T08:20:13.823457+0000","last_fullsized":"2024-03-29T08:20:13.823457+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:17:06.183941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408697+0000","last_change":"2024-03-29T08:20:12.425409+0000","last_active":"2024-03-29T08:20:13.408697+0000","last_peered":"2024-03-29T08:20:13.408697+0000","last_clean":"2024-03-29T08:20:13.408697+0000","last_became_active":"2024-03-29T08:20:12.425173+0000","last_became_peered":"2024-03-29T08:20:12.425173+0000","last_unstale":"2024-03-29T08:20:13.408697+0000","last_undegraded":"2024-03-29T08:20:13.408697+0000","last_fullsized":"2024-03-29T08:20:13.408697+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:06:31.563790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408598+0000","last_change":"2024-03-29T08:20:12.423825+0000","last_active":"2024-03-29T08:20:13.408598+0000","last_peered":"2024-03-29T08:20:13.408598+0000","last_clean":"2024-03-29T08:20:13.408598+0000","last_became_active":"2024-03-29T08:20:12.423558+0000","last_became_peered":"2024-03-29T08:20:12.423558+0000","last_unstale":"2024-03-29T08:20:13.408598+0000","last_undegraded":"2024-03-29T08:20:13.408598+0000","last_fullsized":"2024-03-29T08:20:13.408598+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:41:36.745211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823203+0000","last_change":"2024-03-29T08:20:12.426763+0000","last_active":"2024-03-29T08:20:13.823203+0000","last_peered":"2024-03-29T08:20:13.823203+0000","last_clean":"2024-03-29T08:20:13.823203+0000","last_became_active":"2024-03-29T08:20:12.426496+0000","last_became_peered":"2024-03-29T08:20:12.426496+0000","last_unstale":"2024-03-29T08:20:13.823203+0000","last_undegraded":"2024-03-29T08:20:13.823203+0000","last_fullsized":"2024-03-29T08:20:13.823203+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:17:37.930817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.822902+0000","last_change":"2024-03-29T08:20:12.427375+0000","last_active":"2024-03-29T08:20:13.822902+0000","last_peered":"2024-03-29T08:20:13.822902+0000","last_clean":"2024-03-29T08:20:13.822902+0000","last_became_active":"2024-03-29T08:20:12.427061+0000","last_became_peered":"2024-03-29T08:20:12.427061+0000","last_unstale":"2024-03-29T08:20:13.822902+0000","last_undegraded":"2024-03-29T08:20:13.822902+0000","last_fullsized":"2024-03-29T08:20:13.822902+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:03:01.828915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823017+0000","last_change":"2024-03-29T08:20:12.435124+0000","last_active":"2024-03-29T08:20:13.823017+0000","last_peered":"2024-03-29T08:20:13.823017+0000","last_clean":"2024-03-29T08:20:13.823017+0000","last_became_active":"2024-03-29T08:20:12.434867+0000","last_became_peered":"2024-03-29T08:20:12.434867+0000","last_unstale":"2024-03-29T08:20:13.823017+0000","last_undegraded":"2024-03-29T08:20:13.823017+0000","last_fullsized":"2024-03-29T08:20:13.823017+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:23:02.288895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823731+0000","last_change":"2024-03-29T08:20:12.423256+0000","last_active":"2024-03-29T08:20:13.823731+0000","last_peered":"2024-03-29T08:20:13.823731+0000","last_clean":"2024-03-29T08:20:13.823731+0000","last_became_active":"2024-03-29T08:20:12.422967+0000","last_became_peered":"2024-03-29T08:20:12.422967+0000","last_unstale":"2024-03-29T08:20:13.823731+0000","last_undegraded":"2024-03-29T08:20:13.823731+0000","last_fullsized":"2024-03-29T08:20:13.823731+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:31:07.367334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.470227+0000","last_change":"2024-03-29T08:20:12.413552+0000","last_active":"2024-03-29T08:20:13.470227+0000","last_peered":"2024-03-29T08:20:13.470227+0000","last_clean":"2024-03-29T08:20:13.470227+0000","last_became_active":"2024-03-29T08:20:12.413236+0000","last_became_peered":"2024-03-29T08:20:12.413236+0000","last_unstale":"2024-03-29T08:20:13.470227+0000","last_undegraded":"2024-03-29T08:20:13.470227+0000","last_fullsized":"2024-03-29T08:20:13.470227+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:27.108985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"48'4","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823985+0000","last_change":"2024-03-29T08:20:12.018358+0000","last_active":"2024-03-29T08:20:13.823985+0000","last_peered":"2024-03-29T08:20:13.823985+0000","last_clean":"2024-03-29T08:20:13.823985+0000","last_became_active":"2024-03-29T08:20:12.018088+0000","last_became_peered":"2024-03-29T08:20:12.018088+0000","last_unstale":"2024-03-29T08:20:13.823985+0000","last_undegraded":"2024-03-29T08:20:13.823985+0000","last_fullsized":"2024-03-29T08:20:13.823985+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:03:50.843448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823832+0000","last_change":"2024-03-29T08:20:12.427585+0000","last_active":"2024-03-29T08:20:13.823832+0000","last_peered":"2024-03-29T08:20:13.823832+0000","last_clean":"2024-03-29T08:20:13.823832+0000","last_became_active":"2024-03-29T08:20:12.427358+0000","last_became_peered":"2024-03-29T08:20:12.427358+0000","last_unstale":"2024-03-29T08:20:13.823832+0000","last_undegraded":"2024-03-29T08:20:13.823832+0000","last_fullsized":"2024-03-29T08:20:13.823832+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:03:01.828915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408661+0000","last_change":"2024-03-29T08:20:12.012597+0000","last_active":"2024-03-29T08:20:13.408661+0000","last_peered":"2024-03-29T08:20:13.408661+0000","last_clean":"2024-03-29T08:20:13.408661+0000","last_became_active":"2024-03-29T08:20:12.012317+0000","last_became_peered":"2024-03-29T08:20:12.012317+0000","last_unstale":"2024-03-29T08:20:13.408661+0000","last_undegraded":"2024-03-29T08:20:13.408661+0000","last_fullsized":"2024-03-29T08:20:13.408661+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:42:31.516496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823311+0000","last_change":"2024-03-29T08:20:12.435704+0000","last_active":"2024-03-29T08:20:13.823311+0000","last_peered":"2024-03-29T08:20:13.823311+0000","last_clean":"2024-03-29T08:20:13.823311+0000","last_became_active":"2024-03-29T08:20:12.431038+0000","last_became_peered":"2024-03-29T08:20:12.431038+0000","last_unstale":"2024-03-29T08:20:13.823311+0000","last_undegraded":"2024-03-29T08:20:13.823311+0000","last_fullsized":"2024-03-29T08:20:13.823311+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:17:04.190507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823974+0000","last_change":"2024-03-29T08:20:12.018336+0000","last_active":"2024-03-29T08:20:13.823974+0000","last_peered":"2024-03-29T08:20:13.823974+0000","last_clean":"2024-03-29T08:20:13.823974+0000","last_became_active":"2024-03-29T08:20:12.018039+0000","last_became_peered":"2024-03-29T08:20:12.018039+0000","last_unstale":"2024-03-29T08:20:13.823974+0000","last_undegraded":"2024-03-29T08:20:13.823974+0000","last_fullsized":"2024-03-29T08:20:13.823974+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:55:32.286863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823929+0000","last_change":"2024-03-29T08:20:12.432800+0000","last_active":"2024-03-29T08:20:13.823929+0000","last_peered":"2024-03-29T08:20:13.823929+0000","last_clean":"2024-03-29T08:20:13.823929+0000","last_became_active":"2024-03-29T08:20:12.432474+0000","last_became_peered":"2024-03-29T08:20:12.432474+0000","last_unstale":"2024-03-29T08:20:13.823929+0000","last_undegraded":"2024-03-29T08:20:13.823929+0000","last_fullsized":"2024-03-29T08:20:13.823929+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:39:43.247395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408638+0000","last_change":"2024-03-29T08:20:12.012754+0000","last_active":"2024-03-29T08:20:13.408638+0000","last_peered":"2024-03-29T08:20:13.408638+0000","last_clean":"2024-03-29T08:20:13.408638+0000","last_became_active":"2024-03-29T08:20:12.012445+0000","last_became_peered":"2024-03-29T08:20:12.012445+0000","last_unstale":"2024-03-29T08:20:13.408638+0000","last_undegraded":"2024-03-29T08:20:13.408638+0000","last_fullsized":"2024-03-29T08:20:13.408638+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:10:00.416187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408438+0000","last_change":"2024-03-29T08:20:12.427236+0000","last_active":"2024-03-29T08:20:13.408438+0000","last_peered":"2024-03-29T08:20:13.408438+0000","last_clean":"2024-03-29T08:20:13.408438+0000","last_became_active":"2024-03-29T08:20:12.426971+0000","last_became_peered":"2024-03-29T08:20:12.426971+0000","last_unstale":"2024-03-29T08:20:13.408438+0000","last_undegraded":"2024-03-29T08:20:13.408438+0000","last_fullsized":"2024-03-29T08:20:13.408438+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:38:53.007190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823999+0000","last_change":"2024-03-29T08:20:12.019483+0000","last_active":"2024-03-29T08:20:13.823999+0000","last_peered":"2024-03-29T08:20:13.823999+0000","last_clean":"2024-03-29T08:20:13.823999+0000","last_became_active":"2024-03-29T08:20:12.019133+0000","last_became_peered":"2024-03-29T08:20:12.019133+0000","last_unstale":"2024-03-29T08:20:13.823999+0000","last_undegraded":"2024-03-29T08:20:13.823999+0000","last_fullsized":"2024-03-29T08:20:13.823999+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:47:05.848253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408927+0000","last_change":"2024-03-29T08:20:12.437431+0000","last_active":"2024-03-29T08:20:13.408927+0000","last_peered":"2024-03-29T08:20:13.408927+0000","last_clean":"2024-03-29T08:20:13.408927+0000","last_became_active":"2024-03-29T08:20:12.437089+0000","last_became_peered":"2024-03-29T08:20:12.437089+0000","last_unstale":"2024-03-29T08:20:13.408927+0000","last_undegraded":"2024-03-29T08:20:13.408927+0000","last_fullsized":"2024-03-29T08:20:13.408927+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:00:24.845050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408554+0000","last_change":"2024-03-29T08:20:12.009870+0000","last_active":"2024-03-29T08:20:13.408554+0000","last_peered":"2024-03-29T08:20:13.408554+0000","last_clean":"2024-03-29T08:20:13.408554+0000","last_became_active":"2024-03-29T08:20:12.009448+0000","last_became_peered":"2024-03-29T08:20:12.009448+0000","last_unstale":"2024-03-29T08:20:13.408554+0000","last_undegraded":"2024-03-29T08:20:13.408554+0000","last_fullsized":"2024-03-29T08:20:13.408554+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:48:19.548619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408497+0000","last_change":"2024-03-29T08:20:12.432064+0000","last_active":"2024-03-29T08:20:13.408497+0000","last_peered":"2024-03-29T08:20:13.408497+0000","last_clean":"2024-03-29T08:20:13.408497+0000","last_became_active":"2024-03-29T08:20:12.429253+0000","last_became_peered":"2024-03-29T08:20:12.429253+0000","last_unstale":"2024-03-29T08:20:13.408497+0000","last_undegraded":"2024-03-29T08:20:13.408497+0000","last_fullsized":"2024-03-29T08:20:13.408497+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:10:01.421888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823067+0000","last_change":"2024-03-29T08:20:12.018998+0000","last_active":"2024-03-29T08:20:13.823067+0000","last_peered":"2024-03-29T08:20:13.823067+0000","last_clean":"2024-03-29T08:20:13.823067+0000","last_became_active":"2024-03-29T08:20:12.018791+0000","last_became_peered":"2024-03-29T08:20:12.018791+0000","last_unstale":"2024-03-29T08:20:13.823067+0000","last_undegraded":"2024-03-29T08:20:13.823067+0000","last_fullsized":"2024-03-29T08:20:13.823067+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:03:50.843448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408343+0000","last_change":"2024-03-29T08:20:12.437175+0000","last_active":"2024-03-29T08:20:13.408343+0000","last_peered":"2024-03-29T08:20:13.408343+0000","last_clean":"2024-03-29T08:20:13.408343+0000","last_became_active":"2024-03-29T08:20:12.436905+0000","last_became_peered":"2024-03-29T08:20:12.436905+0000","last_unstale":"2024-03-29T08:20:13.408343+0000","last_undegraded":"2024-03-29T08:20:13.408343+0000","last_fullsized":"2024-03-29T08:20:13.408343+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:27:38.115643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408604+0000","last_change":"2024-03-29T08:20:12.015405+0000","last_active":"2024-03-29T08:20:13.408604+0000","last_peered":"2024-03-29T08:20:13.408604+0000","last_clean":"2024-03-29T08:20:13.408604+0000","last_became_active":"2024-03-29T08:20:12.015152+0000","last_became_peered":"2024-03-29T08:20:12.015152+0000","last_unstale":"2024-03-29T08:20:13.408604+0000","last_undegraded":"2024-03-29T08:20:13.408604+0000","last_fullsized":"2024-03-29T08:20:13.408604+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:58.679621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406843+0000","last_change":"2024-03-29T08:20:12.422275+0000","last_active":"2024-03-29T08:20:13.406843+0000","last_peered":"2024-03-29T08:20:13.406843+0000","last_clean":"2024-03-29T08:20:13.406843+0000","last_became_active":"2024-03-29T08:20:12.422016+0000","last_became_peered":"2024-03-29T08:20:12.422016+0000","last_unstale":"2024-03-29T08:20:13.406843+0000","last_undegraded":"2024-03-29T08:20:13.406843+0000","last_fullsized":"2024-03-29T08:20:13.406843+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:12:28.951099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406793+0000","last_change":"2024-03-29T08:20:12.011565+0000","last_active":"2024-03-29T08:20:13.406793+0000","last_peered":"2024-03-29T08:20:13.406793+0000","last_clean":"2024-03-29T08:20:13.406793+0000","last_became_active":"2024-03-29T08:20:12.011323+0000","last_became_peered":"2024-03-29T08:20:12.011323+0000","last_unstale":"2024-03-29T08:20:13.406793+0000","last_undegraded":"2024-03-29T08:20:13.406793+0000","last_fullsized":"2024-03-29T08:20:13.406793+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:22:49.764428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406759+0000","last_change":"2024-03-29T08:20:12.416154+0000","last_active":"2024-03-29T08:20:13.406759+0000","last_peered":"2024-03-29T08:20:13.406759+0000","last_clean":"2024-03-29T08:20:13.406759+0000","last_became_active":"2024-03-29T08:20:12.415899+0000","last_became_peered":"2024-03-29T08:20:12.415899+0000","last_unstale":"2024-03-29T08:20:13.406759+0000","last_undegraded":"2024-03-29T08:20:13.406759+0000","last_fullsized":"2024-03-29T08:20:13.406759+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:14:47.629162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.409086+0000","last_change":"2024-03-29T08:20:12.015929+0000","last_active":"2024-03-29T08:20:13.409086+0000","last_peered":"2024-03-29T08:20:13.409086+0000","last_clean":"2024-03-29T08:20:13.409086+0000","last_became_active":"2024-03-29T08:20:12.015333+0000","last_became_peered":"2024-03-29T08:20:12.015333+0000","last_unstale":"2024-03-29T08:20:13.409086+0000","last_undegraded":"2024-03-29T08:20:13.409086+0000","last_fullsized":"2024-03-29T08:20:13.409086+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:17:03.184805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823773+0000","last_change":"2024-03-29T08:20:12.435721+0000","last_active":"2024-03-29T08:20:13.823773+0000","last_peered":"2024-03-29T08:20:13.823773+0000","last_clean":"2024-03-29T08:20:13.823773+0000","last_became_active":"2024-03-29T08:20:12.432996+0000","last_became_peered":"2024-03-29T08:20:12.432996+0000","last_unstale":"2024-03-29T08:20:13.823773+0000","last_undegraded":"2024-03-29T08:20:13.823773+0000","last_fullsized":"2024-03-29T08:20:13.823773+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:34:51.649560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"48'3","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823322+0000","last_change":"2024-03-29T08:20:12.010462+0000","last_active":"2024-03-29T08:20:13.823322+0000","last_peered":"2024-03-29T08:20:13.823322+0000","last_clean":"2024-03-29T08:20:13.823322+0000","last_became_active":"2024-03-29T08:20:12.010170+0000","last_became_peered":"2024-03-29T08:20:12.010170+0000","last_unstale":"2024-03-29T08:20:13.823322+0000","last_undegraded":"2024-03-29T08:20:13.823322+0000","last_fullsized":"2024-03-29T08:20:13.823322+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:03:50.843448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408520+0000","last_change":"2024-03-29T08:20:12.440836+0000","last_active":"2024-03-29T08:20:13.408520+0000","last_peered":"2024-03-29T08:20:13.408520+0000","last_clean":"2024-03-29T08:20:13.408520+0000","last_became_active":"2024-03-29T08:20:12.440586+0000","last_became_peered":"2024-03-29T08:20:12.440586+0000","last_unstale":"2024-03-29T08:20:13.408520+0000","last_undegraded":"2024-03-29T08:20:13.408520+0000","last_fullsized":"2024-03-29T08:20:13.408520+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:55:32.718880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"48'6","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.409043+0000","last_change":"2024-03-29T08:20:12.010356+0000","last_active":"2024-03-29T08:20:13.409043+0000","last_peered":"2024-03-29T08:20:13.409043+0000","last_clean":"2024-03-29T08:20:13.409043+0000","last_became_active":"2024-03-29T08:20:12.010070+0000","last_became_peered":"2024-03-29T08:20:12.010070+0000","last_unstale":"2024-03-29T08:20:13.409043+0000","last_undegraded":"2024-03-29T08:20:13.409043+0000","last_fullsized":"2024-03-29T08:20:13.409043+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:09:20.148664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406718+0000","last_change":"2024-03-29T08:20:12.438799+0000","last_active":"2024-03-29T08:20:13.406718+0000","last_peered":"2024-03-29T08:20:13.406718+0000","last_clean":"2024-03-29T08:20:13.406718+0000","last_became_active":"2024-03-29T08:20:12.438409+0000","last_became_peered":"2024-03-29T08:20:12.438409+0000","last_unstale":"2024-03-29T08:20:13.406718+0000","last_undegraded":"2024-03-29T08:20:13.406718+0000","last_fullsized":"2024-03-29T08:20:13.406718+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:41.676339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.470179+0000","last_change":"2024-03-29T08:20:12.011291+0000","last_active":"2024-03-29T08:20:13.470179+0000","last_peered":"2024-03-29T08:20:13.470179+0000","last_clean":"2024-03-29T08:20:13.470179+0000","last_became_active":"2024-03-29T08:20:12.010954+0000","last_became_peered":"2024-03-29T08:20:12.010954+0000","last_unstale":"2024-03-29T08:20:13.470179+0000","last_undegraded":"2024-03-29T08:20:13.470179+0000","last_fullsized":"2024-03-29T08:20:13.470179+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:39:37.107521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.470108+0000","last_change":"2024-03-29T08:20:12.428277+0000","last_active":"2024-03-29T08:20:13.470108+0000","last_peered":"2024-03-29T08:20:13.470108+0000","last_clean":"2024-03-29T08:20:13.470108+0000","last_became_active":"2024-03-29T08:20:12.422163+0000","last_became_peered":"2024-03-29T08:20:12.422163+0000","last_unstale":"2024-03-29T08:20:13.470108+0000","last_undegraded":"2024-03-29T08:20:13.470108+0000","last_fullsized":"2024-03-29T08:20:13.470108+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:07.507985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823407+0000","last_change":"2024-03-29T08:20:12.012434+0000","last_active":"2024-03-29T08:20:13.823407+0000","last_peered":"2024-03-29T08:20:13.823407+0000","last_clean":"2024-03-29T08:20:13.823407+0000","last_became_active":"2024-03-29T08:20:12.012076+0000","last_became_peered":"2024-03-29T08:20:12.012076+0000","last_unstale":"2024-03-29T08:20:13.823407+0000","last_undegraded":"2024-03-29T08:20:13.823407+0000","last_fullsized":"2024-03-29T08:20:13.823407+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:11:08.174875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.469957+0000","last_change":"2024-03-29T08:20:12.428791+0000","last_active":"2024-03-29T08:20:13.469957+0000","last_peered":"2024-03-29T08:20:13.469957+0000","last_clean":"2024-03-29T08:20:13.469957+0000","last_became_active":"2024-03-29T08:20:12.428450+0000","last_became_peered":"2024-03-29T08:20:12.428450+0000","last_unstale":"2024-03-29T08:20:13.469957+0000","last_undegraded":"2024-03-29T08:20:13.469957+0000","last_fullsized":"2024-03-29T08:20:13.469957+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:53:42.654767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823112+0000","last_change":"2024-03-29T08:20:12.021393+0000","last_active":"2024-03-29T08:20:13.823112+0000","last_peered":"2024-03-29T08:20:13.823112+0000","last_clean":"2024-03-29T08:20:13.823112+0000","last_became_active":"2024-03-29T08:20:12.021116+0000","last_became_peered":"2024-03-29T08:20:12.021116+0000","last_unstale":"2024-03-29T08:20:13.823112+0000","last_undegraded":"2024-03-29T08:20:13.823112+0000","last_fullsized":"2024-03-29T08:20:13.823112+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:11:21.922581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.822854+0000","last_change":"2024-03-29T08:20:12.431560+0000","last_active":"2024-03-29T08:20:13.822854+0000","last_peered":"2024-03-29T08:20:13.822854+0000","last_clean":"2024-03-29T08:20:13.822854+0000","last_became_active":"2024-03-29T08:20:12.431308+0000","last_became_peered":"2024-03-29T08:20:12.431308+0000","last_unstale":"2024-03-29T08:20:13.822854+0000","last_undegraded":"2024-03-29T08:20:13.822854+0000","last_fullsized":"2024-03-29T08:20:13.822854+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:43:50.843484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"48'8","reported_seq":19,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408619+0000","last_change":"2024-03-29T08:20:12.007857+0000","last_active":"2024-03-29T08:20:13.408619+0000","last_peered":"2024-03-29T08:20:13.408619+0000","last_clean":"2024-03-29T08:20:13.408619+0000","last_became_active":"2024-03-29T08:20:12.007577+0000","last_became_peered":"2024-03-29T08:20:12.007577+0000","last_unstale":"2024-03-29T08:20:13.408619+0000","last_undegraded":"2024-03-29T08:20:13.408619+0000","last_fullsized":"2024-03-29T08:20:13.408619+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:17:36.925115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.470303+0000","last_change":"2024-03-29T08:20:12.428462+0000","last_active":"2024-03-29T08:20:13.470303+0000","last_peered":"2024-03-29T08:20:13.470303+0000","last_clean":"2024-03-29T08:20:13.470303+0000","last_became_active":"2024-03-29T08:20:12.425867+0000","last_became_peered":"2024-03-29T08:20:12.425867+0000","last_unstale":"2024-03-29T08:20:13.470303+0000","last_undegraded":"2024-03-29T08:20:13.470303+0000","last_fullsized":"2024-03-29T08:20:13.470303+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:26:26.576312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823450+0000","last_change":"2024-03-29T08:20:12.018885+0000","last_active":"2024-03-29T08:20:13.823450+0000","last_peered":"2024-03-29T08:20:13.823450+0000","last_clean":"2024-03-29T08:20:13.823450+0000","last_became_active":"2024-03-29T08:20:12.018553+0000","last_became_peered":"2024-03-29T08:20:12.018553+0000","last_unstale":"2024-03-29T08:20:13.823450+0000","last_undegraded":"2024-03-29T08:20:13.823450+0000","last_fullsized":"2024-03-29T08:20:13.823450+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:32:19.810173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.470172+0000","last_change":"2024-03-29T08:20:12.425777+0000","last_active":"2024-03-29T08:20:13.470172+0000","last_peered":"2024-03-29T08:20:13.470172+0000","last_clean":"2024-03-29T08:20:13.470172+0000","last_became_active":"2024-03-29T08:20:12.425438+0000","last_became_peered":"2024-03-29T08:20:12.425438+0000","last_unstale":"2024-03-29T08:20:13.470172+0000","last_undegraded":"2024-03-29T08:20:13.470172+0000","last_fullsized":"2024-03-29T08:20:13.470172+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:52:41.418480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406896+0000","last_change":"2024-03-29T08:20:12.019663+0000","last_active":"2024-03-29T08:20:13.406896+0000","last_peered":"2024-03-29T08:20:13.406896+0000","last_clean":"2024-03-29T08:20:13.406896+0000","last_became_active":"2024-03-29T08:20:12.016423+0000","last_became_peered":"2024-03-29T08:20:12.016423+0000","last_unstale":"2024-03-29T08:20:13.406896+0000","last_undegraded":"2024-03-29T08:20:13.406896+0000","last_fullsized":"2024-03-29T08:20:13.406896+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:19:44.116553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823392+0000","last_change":"2024-03-29T08:20:12.436125+0000","last_active":"2024-03-29T08:20:13.823392+0000","last_peered":"2024-03-29T08:20:13.823392+0000","last_clean":"2024-03-29T08:20:13.823392+0000","last_became_active":"2024-03-29T08:20:12.435846+0000","last_became_peered":"2024-03-29T08:20:12.435846+0000","last_unstale":"2024-03-29T08:20:13.823392+0000","last_undegraded":"2024-03-29T08:20:13.823392+0000","last_fullsized":"2024-03-29T08:20:13.823392+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:01:33.657226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406953+0000","last_change":"2024-03-29T08:20:12.018760+0000","last_active":"2024-03-29T08:20:13.406953+0000","last_peered":"2024-03-29T08:20:13.406953+0000","last_clean":"2024-03-29T08:20:13.406953+0000","last_became_active":"2024-03-29T08:20:12.018485+0000","last_became_peered":"2024-03-29T08:20:12.018485+0000","last_unstale":"2024-03-29T08:20:13.406953+0000","last_undegraded":"2024-03-29T08:20:13.406953+0000","last_fullsized":"2024-03-29T08:20:13.406953+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:27:28.312946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406918+0000","last_change":"2024-03-29T08:20:12.431691+0000","last_active":"2024-03-29T08:20:13.406918+0000","last_peered":"2024-03-29T08:20:13.406918+0000","last_clean":"2024-03-29T08:20:13.406918+0000","last_became_active":"2024-03-29T08:20:12.427172+0000","last_became_peered":"2024-03-29T08:20:12.427172+0000","last_unstale":"2024-03-29T08:20:13.406918+0000","last_undegraded":"2024-03-29T08:20:13.406918+0000","last_fullsized":"2024-03-29T08:20:13.406918+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:44:45.599624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823509+0000","last_change":"2024-03-29T08:20:12.021179+0000","last_active":"2024-03-29T08:20:13.823509+0000","last_peered":"2024-03-29T08:20:13.823509+0000","last_clean":"2024-03-29T08:20:13.823509+0000","last_became_active":"2024-03-29T08:20:12.020881+0000","last_became_peered":"2024-03-29T08:20:12.020881+0000","last_unstale":"2024-03-29T08:20:13.823509+0000","last_undegraded":"2024-03-29T08:20:13.823509+0000","last_fullsized":"2024-03-29T08:20:13.823509+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:41:38.371779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408769+0000","last_change":"2024-03-29T08:20:12.427190+0000","last_active":"2024-03-29T08:20:13.408769+0000","last_peered":"2024-03-29T08:20:13.408769+0000","last_clean":"2024-03-29T08:20:13.408769+0000","last_became_active":"2024-03-29T08:20:12.426912+0000","last_became_peered":"2024-03-29T08:20:12.426912+0000","last_unstale":"2024-03-29T08:20:13.408769+0000","last_undegraded":"2024-03-29T08:20:13.408769+0000","last_fullsized":"2024-03-29T08:20:13.408769+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:28.812148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.409033+0000","last_change":"2024-03-29T08:20:12.011532+0000","last_active":"2024-03-29T08:20:13.409033+0000","last_peered":"2024-03-29T08:20:13.409033+0000","last_clean":"2024-03-29T08:20:13.409033+0000","last_became_active":"2024-03-29T08:20:12.011280+0000","last_became_peered":"2024-03-29T08:20:12.011280+0000","last_unstale":"2024-03-29T08:20:13.409033+0000","last_undegraded":"2024-03-29T08:20:13.409033+0000","last_fullsized":"2024-03-29T08:20:13.409033+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:23:01.283193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823151+0000","last_change":"2024-03-29T08:20:12.424479+0000","last_active":"2024-03-29T08:20:13.823151+0000","last_peered":"2024-03-29T08:20:13.823151+0000","last_clean":"2024-03-29T08:20:13.823151+0000","last_became_active":"2024-03-29T08:20:12.423372+0000","last_became_peered":"2024-03-29T08:20:12.423372+0000","last_unstale":"2024-03-29T08:20:13.823151+0000","last_undegraded":"2024-03-29T08:20:13.823151+0000","last_fullsized":"2024-03-29T08:20:13.823151+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:17:04.190507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823517+0000","last_change":"2024-03-29T08:20:12.017378+0000","last_active":"2024-03-29T08:20:13.823517+0000","last_peered":"2024-03-29T08:20:13.823517+0000","last_clean":"2024-03-29T08:20:13.823517+0000","last_became_active":"2024-03-29T08:20:12.017137+0000","last_became_peered":"2024-03-29T08:20:12.017137+0000","last_unstale":"2024-03-29T08:20:13.823517+0000","last_undegraded":"2024-03-29T08:20:13.823517+0000","last_fullsized":"2024-03-29T08:20:13.823517+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:38:05.671123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823770+0000","last_change":"2024-03-29T08:20:12.428248+0000","last_active":"2024-03-29T08:20:13.823770+0000","last_peered":"2024-03-29T08:20:13.823770+0000","last_clean":"2024-03-29T08:20:13.823770+0000","last_became_active":"2024-03-29T08:20:12.427989+0000","last_became_peered":"2024-03-29T08:20:12.427989+0000","last_unstale":"2024-03-29T08:20:13.823770+0000","last_undegraded":"2024-03-29T08:20:13.823770+0000","last_fullsized":"2024-03-29T08:20:13.823770+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:37:40.314866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408596+0000","last_change":"2024-03-29T08:20:12.016530+0000","last_active":"2024-03-29T08:20:13.408596+0000","last_peered":"2024-03-29T08:20:13.408596+0000","last_clean":"2024-03-29T08:20:13.408596+0000","last_became_active":"2024-03-29T08:20:12.014858+0000","last_became_peered":"2024-03-29T08:20:12.014858+0000","last_unstale":"2024-03-29T08:20:13.408596+0000","last_undegraded":"2024-03-29T08:20:13.408596+0000","last_fullsized":"2024-03-29T08:20:13.408596+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:36:59.039140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823593+0000","last_change":"2024-03-29T08:20:12.433424+0000","last_active":"2024-03-29T08:20:13.823593+0000","last_peered":"2024-03-29T08:20:13.823593+0000","last_clean":"2024-03-29T08:20:13.823593+0000","last_became_active":"2024-03-29T08:20:12.431741+0000","last_became_peered":"2024-03-29T08:20:12.431741+0000","last_unstale":"2024-03-29T08:20:13.823593+0000","last_undegraded":"2024-03-29T08:20:13.823593+0000","last_fullsized":"2024-03-29T08:20:13.823593+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:27:34.663199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408317+0000","last_change":"2024-03-29T08:20:12.010852+0000","last_active":"2024-03-29T08:20:13.408317+0000","last_peered":"2024-03-29T08:20:13.408317+0000","last_clean":"2024-03-29T08:20:13.408317+0000","last_became_active":"2024-03-29T08:20:12.010602+0000","last_became_peered":"2024-03-29T08:20:12.010602+0000","last_unstale":"2024-03-29T08:20:13.408317+0000","last_undegraded":"2024-03-29T08:20:13.408317+0000","last_fullsized":"2024-03-29T08:20:13.408317+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:43:21.606807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408688+0000","last_change":"2024-03-29T08:20:12.429242+0000","last_active":"2024-03-29T08:20:13.408688+0000","last_peered":"2024-03-29T08:20:13.408688+0000","last_clean":"2024-03-29T08:20:13.408688+0000","last_became_active":"2024-03-29T08:20:12.428986+0000","last_became_peered":"2024-03-29T08:20:12.428986+0000","last_unstale":"2024-03-29T08:20:13.408688+0000","last_undegraded":"2024-03-29T08:20:13.408688+0000","last_fullsized":"2024-03-29T08:20:13.408688+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:58:06.775283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"48'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823609+0000","last_change":"2024-03-29T08:20:12.017438+0000","last_active":"2024-03-29T08:20:13.823609+0000","last_peered":"2024-03-29T08:20:13.823609+0000","last_clean":"2024-03-29T08:20:13.823609+0000","last_became_active":"2024-03-29T08:20:12.017132+0000","last_became_peered":"2024-03-29T08:20:12.017132+0000","last_unstale":"2024-03-29T08:20:13.823609+0000","last_undegraded":"2024-03-29T08:20:13.823609+0000","last_fullsized":"2024-03-29T08:20:13.823609+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:00:53.444271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.409150+0000","last_change":"2024-03-29T08:20:12.430375+0000","last_active":"2024-03-29T08:20:13.409150+0000","last_peered":"2024-03-29T08:20:13.409150+0000","last_clean":"2024-03-29T08:20:13.409150+0000","last_became_active":"2024-03-29T08:20:12.430021+0000","last_became_peered":"2024-03-29T08:20:12.430021+0000","last_unstale":"2024-03-29T08:20:13.409150+0000","last_undegraded":"2024-03-29T08:20:13.409150+0000","last_fullsized":"2024-03-29T08:20:13.409150+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:25:03.539726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823673+0000","last_change":"2024-03-29T08:20:12.018107+0000","last_active":"2024-03-29T08:20:13.823673+0000","last_peered":"2024-03-29T08:20:13.823673+0000","last_clean":"2024-03-29T08:20:13.823673+0000","last_became_active":"2024-03-29T08:20:12.017885+0000","last_became_peered":"2024-03-29T08:20:12.017885+0000","last_unstale":"2024-03-29T08:20:13.823673+0000","last_undegraded":"2024-03-29T08:20:13.823673+0000","last_fullsized":"2024-03-29T08:20:13.823673+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:11:21.922581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408627+0000","last_change":"2024-03-29T08:20:12.434892+0000","last_active":"2024-03-29T08:20:13.408627+0000","last_peered":"2024-03-29T08:20:13.408627+0000","last_clean":"2024-03-29T08:20:13.408627+0000","last_became_active":"2024-03-29T08:20:12.434621+0000","last_became_peered":"2024-03-29T08:20:12.434621+0000","last_unstale":"2024-03-29T08:20:13.408627+0000","last_undegraded":"2024-03-29T08:20:13.408627+0000","last_fullsized":"2024-03-29T08:20:13.408627+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:28.948477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.470189+0000","last_change":"2024-03-29T08:20:12.016011+0000","last_active":"2024-03-29T08:20:13.470189+0000","last_peered":"2024-03-29T08:20:13.470189+0000","last_clean":"2024-03-29T08:20:13.470189+0000","last_became_active":"2024-03-29T08:20:12.015613+0000","last_became_peered":"2024-03-29T08:20:12.015613+0000","last_unstale":"2024-03-29T08:20:13.470189+0000","last_undegraded":"2024-03-29T08:20:13.470189+0000","last_fullsized":"2024-03-29T08:20:13.470189+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:06:37.816754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408771+0000","last_change":"2024-03-29T08:20:12.426681+0000","last_active":"2024-03-29T08:20:13.408771+0000","last_peered":"2024-03-29T08:20:13.408771+0000","last_clean":"2024-03-29T08:20:13.408771+0000","last_became_active":"2024-03-29T08:20:12.426436+0000","last_became_peered":"2024-03-29T08:20:12.426436+0000","last_unstale":"2024-03-29T08:20:13.408771+0000","last_undegraded":"2024-03-29T08:20:13.408771+0000","last_fullsized":"2024-03-29T08:20:13.408771+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:09:27.388912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823545+0000","last_change":"2024-03-29T08:20:12.018569+0000","last_active":"2024-03-29T08:20:13.823545+0000","last_peered":"2024-03-29T08:20:13.823545+0000","last_clean":"2024-03-29T08:20:13.823545+0000","last_became_active":"2024-03-29T08:20:12.018354+0000","last_became_peered":"2024-03-29T08:20:12.018354+0000","last_unstale":"2024-03-29T08:20:13.823545+0000","last_undegraded":"2024-03-29T08:20:13.823545+0000","last_fullsized":"2024-03-29T08:20:13.823545+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:55:32.286863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823657+0000","last_change":"2024-03-29T08:20:12.430787+0000","last_active":"2024-03-29T08:20:13.823657+0000","last_peered":"2024-03-29T08:20:13.823657+0000","last_clean":"2024-03-29T08:20:13.823657+0000","last_became_active":"2024-03-29T08:20:12.430019+0000","last_became_peered":"2024-03-29T08:20:12.430019+0000","last_unstale":"2024-03-29T08:20:13.823657+0000","last_undegraded":"2024-03-29T08:20:13.823657+0000","last_fullsized":"2024-03-29T08:20:13.823657+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:49:17.040347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408554+0000","last_change":"2024-03-29T08:20:12.013272+0000","last_active":"2024-03-29T08:20:13.408554+0000","last_peered":"2024-03-29T08:20:13.408554+0000","last_clean":"2024-03-29T08:20:13.408554+0000","last_became_active":"2024-03-29T08:20:12.009964+0000","last_became_peered":"2024-03-29T08:20:12.009964+0000","last_unstale":"2024-03-29T08:20:13.408554+0000","last_undegraded":"2024-03-29T08:20:13.408554+0000","last_fullsized":"2024-03-29T08:20:13.408554+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:25:02.534024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.406508+0000","last_change":"2024-03-29T08:20:12.435654+0000","last_active":"2024-03-29T08:20:13.406508+0000","last_peered":"2024-03-29T08:20:13.406508+0000","last_clean":"2024-03-29T08:20:13.406508+0000","last_became_active":"2024-03-29T08:20:12.435346+0000","last_became_peered":"2024-03-29T08:20:12.435346+0000","last_unstale":"2024-03-29T08:20:13.406508+0000","last_undegraded":"2024-03-29T08:20:13.406508+0000","last_fullsized":"2024-03-29T08:20:13.406508+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:52:41.418480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823121+0000","last_change":"2024-03-29T08:20:12.010149+0000","last_active":"2024-03-29T08:20:13.823121+0000","last_peered":"2024-03-29T08:20:13.823121+0000","last_clean":"2024-03-29T08:20:13.823121+0000","last_became_active":"2024-03-29T08:20:12.009873+0000","last_became_peered":"2024-03-29T08:20:12.009873+0000","last_unstale":"2024-03-29T08:20:13.823121+0000","last_undegraded":"2024-03-29T08:20:13.823121+0000","last_fullsized":"2024-03-29T08:20:13.823121+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:56:53.966923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.822953+0000","last_change":"2024-03-29T08:20:12.415800+0000","last_active":"2024-03-29T08:20:13.822953+0000","last_peered":"2024-03-29T08:20:13.822953+0000","last_clean":"2024-03-29T08:20:13.822953+0000","last_became_active":"2024-03-29T08:20:12.412788+0000","last_became_peered":"2024-03-29T08:20:12.412788+0000","last_unstale":"2024-03-29T08:20:13.822953+0000","last_undegraded":"2024-03-29T08:20:13.822953+0000","last_fullsized":"2024-03-29T08:20:13.822953+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:59:29.177991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.470121+0000","last_change":"2024-03-29T08:20:12.012788+0000","last_active":"2024-03-29T08:20:13.470121+0000","last_peered":"2024-03-29T08:20:13.470121+0000","last_clean":"2024-03-29T08:20:13.470121+0000","last_became_active":"2024-03-29T08:20:12.012542+0000","last_became_peered":"2024-03-29T08:20:12.012542+0000","last_unstale":"2024-03-29T08:20:13.470121+0000","last_undegraded":"2024-03-29T08:20:13.470121+0000","last_fullsized":"2024-03-29T08:20:13.470121+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:32:49.043368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.823693+0000","last_change":"2024-03-29T08:20:12.427911+0000","last_active":"2024-03-29T08:20:13.823693+0000","last_peered":"2024-03-29T08:20:13.823693+0000","last_clean":"2024-03-29T08:20:13.823693+0000","last_became_active":"2024-03-29T08:20:12.427620+0000","last_became_peered":"2024-03-29T08:20:12.427620+0000","last_unstale":"2024-03-29T08:20:13.823693+0000","last_undegraded":"2024-03-29T08:20:13.823693+0000","last_fullsized":"2024-03-29T08:20:13.823693+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:11.987220+0000","last_clean_scrub_stamp":"2024-03-29T08:20:11.987220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:08:40.657636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"48'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-03-29T08:20:13.408716+0000","last_change":"2024-03-29T08:20:12.017983+0000","last_active":"2024-03-29T08:20:13.408716+0000","last_peered":"2024-03-29T08:20:13.408716+0000","last_clean":"2024-03-29T08:20:13.408716+0000","last_became_active":"2024-03-29T08:20:12.017785+0000","last_became_peered":"2024-03-29T08:20:12.017785+0000","last_unstale":"2024-03-29T08:20:13.408716+0000","last_undegraded":"2024-03-29T08:20:13.408716+0000","last_fullsized":"2024-03-29T08:20:13.408716+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:20:10.981518+0000","last_clean_scrub_stamp":"2024-03-29T08:20:10.981518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:32:58.839816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":187,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-03-29T08:20:12.412429+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:20:12.412429+0000","last_peered":"2024-03-29T08:20:12.412429+0000","last_clean":"2024-03-29T08:20:12.412429+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:20:12.412429+0000","last_undegraded":"2024-03-29T08:20:12.412429+0000","last_fullsized":"2024-03-29T08:20:12.412429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27592,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":573440,"data_stored":231835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.65700000000000003,"15min":0.64500000000000002},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.67900000000000005,"15min":0.66900000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.75800000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.65100000000000002,"15min":0.64100000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.64100000000000001,"15min":0.625},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.79300000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.63600000000000001,"15min":0.624},"min":{"1min":0.53000000000000003,"5min":0.503,"15min":0.503},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.66800000000000004,"15min":0.65600000000000003},"min":{"1min":0.57099999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.58499999999999996}]},{"osd":3,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69699999999999995,"15min":0.69299999999999995},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.73599999999999999,"15min":0.72799999999999998},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.73199999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.70499999999999996,"15min":0.72099999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.93000000000000005,"5min":1.123,"15min":1.123},"last":0.998}]},{"osd":5,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.70899999999999996,"15min":0.71399999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70099999999999996,"15min":0.70299999999999996},"min":{"1min":0.44800000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.56999999999999995}]},{"osd":6,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72899999999999998,"15min":0.72799999999999998},"min":{"1min":0.53100000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.748,"15min":0.75600000000000001},"min":{"1min":0.46999999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.89300000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":577536,"data_stored":232417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.63500000000000001,"15min":0.625},"min":{"1min":0.49099999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.753},{"interface":"front","average":{"1min":0.754,"5min":0.69099999999999995,"15min":0.68100000000000005},"min":{"1min":0.52900000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.69999999999999996}]},{"osd":1,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.61599999999999999,"15min":0.60599999999999998},"min":{"1min":0.438,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.61899999999999999},"min":{"1min":0.49199999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88900000000000001,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.71599999999999997}]},{"osd":2,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.68000000000000005,"15min":0.66800000000000004},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70299999999999996,"15min":0.69899999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.93200000000000005,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.77100000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70099999999999996,"15min":0.69599999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.51100000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.66800000000000004,"15min":0.67700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.96199999999999997,"5min":1.556,"15min":1.556},"last":0.505},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.72499999999999998,"15min":0.74299999999999999},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.93999999999999995,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.47599999999999998}]},{"osd":5,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.69499999999999995,"15min":0.70899999999999996},"min":{"1min":0.42899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92700000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.73099999999999998,"15min":0.73899999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.56799999999999995}]},{"osd":7,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.76200000000000001,"15min":0.77400000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.97799999999999998,"5min":0.998,"15min":0.998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.77100000000000002,"15min":0.78500000000000003},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":1.024,"15min":1.024},"last":0.54800000000000004}]}]},{"osd":4,"up_from":15,"seq":64424509469,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":589824,"data_stored":233457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70499999999999996,"15min":0.70899999999999996},"min":{"1min":0.42499999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.014,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.73699999999999999,"15min":0.747},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0489999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.70999999999999996}]},{"osd":1,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.76100000000000001,"15min":0.76600000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.087,"5min":2.524,"15min":2.524},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.71399999999999997,"15min":0.71999999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.91100000000000003,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.67400000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73199999999999998,"15min":0.73599999999999999},"min":{"1min":0.498,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1060000000000001,"5min":2.363,"15min":2.363},"last":0.746},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76200000000000001,"15min":0.76100000000000001},"min":{"1min":0.54400000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.97599999999999998,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.83999999999999997}]},{"osd":3,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.75600000000000001,"15min":0.76000000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94299999999999995,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.81399999999999995,"15min":0.81899999999999995},"min":{"1min":0.63,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.0700000000000001,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.89400000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.69499999999999995,"15min":0.70299999999999996},"min":{"1min":0.42499999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92400000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.45600000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.86799999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.82199999999999995}]},{"osd":6,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.67200000000000004,"15min":0.67800000000000005},"min":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.94899999999999995,"5min":0.996,"15min":0.996},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.51400000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.91500000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79400000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.89000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.76200000000000001,"15min":0.77000000000000002},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.99199999999999999,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.92100000000000004}]}]},{"osd":3,"up_from":15,"seq":64424509472,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31696,"kb_used_data":568,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340144,"statfs":{"total":96636764160,"available":96604307456,"internally_reserved":0,"allocated":581632,"data_stored":233365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.45200000000000001,"15min":0.46200000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.61899999999999999,"5min":1.123,"15min":1.123},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.51400000000000001,"15min":0.52200000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.75,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.51400000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.32100000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.52500000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.52400000000000002,"15min":0.53700000000000003},"min":{"1min":0.32300000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.59299999999999997,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.48899999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.55800000000000005,"15min":0.57399999999999995},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.65500000000000003,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.64500000000000002,"15min":0.66200000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.71199999999999997,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.78300000000000003,"15min":0.79900000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.071,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0609999999999999,"5min":1.391,"15min":1.391},"last":0.84299999999999997}]},{"osd":5,"last update":"Fri Mar 29 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.79800000000000004,"15min":0.81000000000000005},"min":{"1min":0.57299999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.022,"5min":1.5669999999999999,"15min":1.5669999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.84899999999999998,"15min":0.86899999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.99399999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.77100000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.82199999999999995,"15min":0.84399999999999997},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.93000000000000005,"5min":1.669,"15min":1.669},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.81799999999999995,"15min":0.83699999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.081,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.68300000000000005}]},{"osd":7,"last update":"Fri Mar 29 08:20:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.044,"5min":1.887,"15min":1.887},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.84299999999999997,"15min":0.86799999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.01,"5min":2.044,"15min":2.044},"last":0.80300000000000005}]}]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31676,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340164,"statfs":{"total":96636764160,"available":96604327936,"internally_reserved":0,"allocated":573440,"data_stored":231891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.504,"15min":0.50600000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.83399999999999996,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.316},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.46200000000000002}]},{"osd":1,"last update":"Fri Mar 29 08:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.497,"15min":0.497},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72899999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.502,"15min":0.51200000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.72899999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.503}]},{"osd":3,"last update":"Fri Mar 29 08:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48899999999999999,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.57299999999999995},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78200000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.57699999999999996}]},{"osd":4,"last update":"Fri Mar 29 08:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.76100000000000001,"15min":0.76200000000000001},"min":{"1min":0.621,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.90600000000000003,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.76200000000000001,"15min":0.76700000000000002},"min":{"1min":0.502,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.96099999999999997,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.435}]},{"osd":5,"last update":"Fri Mar 29 08:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.83499999999999996,"15min":0.84399999999999997},"min":{"1min":0.59399999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.997,"5min":1.306,"15min":1.306},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.85299999999999998,"15min":0.86499999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.008,"5min":1.526,"15min":1.526},"last":0.48799999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97699999999999998,"5min":1.095,"15min":1.095},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.78400000000000003,"15min":0.79300000000000004},"min":{"1min":0.623,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.92700000000000005,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.68000000000000005}]},{"osd":7,"last update":"Fri Mar 29 08:20:01 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.56699999999999995,"5min":0.505,"15min":0.505},"max":{"1min":0.94399999999999995,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.626},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.58899999999999997,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.94699999999999995,"5min":1.153,"15min":1.153},"last":0.56499999999999995}]}]},{"osd":1,"up_from":15,"seq":64424509471,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1167360,"data_stored":822239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.61899999999999999,"15min":0.65200000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.58299999999999996,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.63600000000000001,"15min":0.66500000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.66100000000000003,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.29999999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.505,"15min":0.51000000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.64100000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53400000000000003,"15min":0.53700000000000003},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.878,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.44}]},{"osd":3,"last update":"Fri Mar 29 08:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.44900000000000001,"5min":0.433,"15min":0.433},"max":{"1min":0.82399999999999995,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.38200000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.50900000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94599999999999995,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78200000000000003,"15min":0.79500000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98299999999999998,"5min":1.23,"15min":1.23},"last":0.78700000000000003}]},{"osd":5,"last update":"Fri Mar 29 08:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.77300000000000002,"15min":0.78300000000000003},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.93200000000000005,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.749,"15min":0.75900000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93999999999999995,"5min":1.107,"15min":1.107},"last":0.71799999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76700000000000002,"15min":0.77100000000000002},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.0049999999999999,"5min":1.026,"15min":1.026},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.85199999999999998,"15min":0.86799999999999999},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.02,"5min":1.196,"15min":1.196},"last":0.69399999999999995}]},{"osd":7,"last update":"Fri Mar 29 08:20:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79300000000000004,"15min":0.80200000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.999,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.78000000000000003,"15min":0.79200000000000004},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":0.86499999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.64300000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31680,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340160,"statfs":{"total":96636764160,"available":96604323840,"internally_reserved":0,"allocated":577536,"data_stored":232473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.59599999999999997,"15min":0.628},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.61099999999999999,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.84099999999999997,"15min":0.90300000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.69499999999999995,"5min":8.0419999999999998,"15min":8.0419999999999998},"last":0.52100000000000002}]},{"osd":2,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.55600000000000005,"15min":0.56100000000000005},"min":{"1min":0.35899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72799999999999998,"5min":1.258,"15min":1.258},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.73299999999999998,"15min":0.76700000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78700000000000003,"5min":4.907,"15min":4.907},"last":0.47999999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.84199999999999997,"15min":0.90200000000000002},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.91000000000000003,"5min":7.9569999999999999,"15min":7.9569999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.65800000000000003,"15min":0.67700000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78300000000000003,"5min":1.397,"15min":1.397},"last":0.57699999999999996}]},{"osd":4,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.97999999999999998,"15min":1.022},"min":{"1min":0.61899999999999999,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.94999999999999996,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.95499999999999996,"15min":0.99399999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.98399999999999999,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.71499999999999997}]},{"osd":5,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.92200000000000004,"15min":0.95399999999999996},"min":{"1min":0.56799999999999995,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.97899999999999998,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.85699999999999998,"15min":0.879},"min":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.98199999999999998,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.878}]},{"osd":6,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.85899999999999999,"15min":0.877},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.96299999999999997,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.97799999999999998,"15min":1.016},"min":{"1min":0.54900000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.01,"5min":4.4100000000000001,"15min":4.4100000000000001},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:20:03 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.99299999999999999,"15min":1.0369999999999999},"min":{"1min":0.56999999999999995,"5min":0.496,"15min":0.496},"max":{"1min":1.0980000000000001,"5min":4.5209999999999999,"15min":4.5209999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.97699999999999998,"15min":1.0189999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.91300000000000003,"5min":4.4420000000000002,"15min":4.4420000000000002},"last":0.76300000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542175,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1167360,"data_stored":822191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.628,"15min":0.627},"min":{"1min":0.47099999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85199999999999998,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83399999999999996,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.72899999999999998,"15min":0.74199999999999999},"min":{"1min":0.48799999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89700000000000002,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.5,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.621}]},{"osd":2,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69699999999999995,"15min":0.70099999999999996},"min":{"1min":0.47299999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80500000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73899999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.71199999999999997,"15min":0.71399999999999997},"min":{"1min":0.56399999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.872,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.48499999999999999,"5min":0.435,"15min":0.435},"max":{"1min":0.88100000000000001,"5min":1.954,"15min":1.954},"last":0.68400000000000005}]},{"osd":4,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.64300000000000002,"15min":0.65100000000000002},"min":{"1min":0.42199999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81999999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.64900000000000002,"15min":0.66200000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72999999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81899999999999995}]},{"osd":6,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.67800000000000005,"15min":0.68600000000000005},"min":{"1min":0.443,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86899999999999999,"5min":1.071,"15min":1.071},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.69199999999999995,"15min":0.70099999999999996},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.85599999999999998,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.86899999999999999}]},{"osd":7,"last update":"Fri Mar 29 08:20:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.70899999999999996,"15min":0.71999999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.84699999999999998,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.746,"15min":0.76000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90300000000000002,"5min":1.04,"15min":1.04},"last":0.85699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:20:16.908 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T08:20:16.908 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-29T08:20:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.091+0000 7f368aa00640 1 Processor -- start 2024-03-29T08:20:17.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.091+0000 7f368aa00640 1 -- start start 2024-03-29T08:20:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.091+0000 7f368aa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f368415aab0 0x7f36840ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.091+0000 7f368aa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f36840cb120 0x7f36840cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.091+0000 7f368aa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36840ae120 con 0x7f36840ad450 2024-03-29T08:20:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.091+0000 7f368aa00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36840d1500 con 0x7f36840cb120 2024-03-29T08:20:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.091+0000 7f368aa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36840b01b0 con 0x7f368415aab0 2024-03-29T08:20:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.091+0000 7f3688a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f36840ad450 0x7f36840ad900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59360/0 (socket says 172.21.15.46:59360) 2024-03-29T08:20:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.091+0000 7f3688a00640 1 -- 172.21.15.46:0/1057728706 learned_addr learned my addr 172.21.15.46:0/1057728706 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.091+0000 7f3683e00640 1 --2- 172.21.15.46:0/1057728706 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f368415aab0 0x7f36840ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:17.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.091+0000 7f367b400640 1 --2- 172.21.15.46:0/1057728706 >> v2:172.21.15.73:3300/0 conn(0x7f36840cb120 0x7f36840cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:17.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.091+0000 7f3683400640 1 -- 172.21.15.46:0/1057728706 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2135378090 0 0) 0x7f36840ae120 con 0x7f36840ad450 2024-03-29T08:20:17.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.091+0000 7f3683400640 1 -- 172.21.15.46:0/1057728706 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3668003530 con 0x7f36840ad450 2024-03-29T08:20:17.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.091+0000 7f3683e00640 1 -- 172.21.15.46:0/1057728706 >> v2:172.21.15.73:3300/0 conn(0x7f36840cb120 msgr2=0x7f36840cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:17.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.091+0000 7f3683e00640 1 --2- 172.21.15.46:0/1057728706 >> v2:172.21.15.73:3300/0 conn(0x7f36840cb120 0x7f36840cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.091+0000 7f3683e00640 1 -- 172.21.15.46:0/1057728706 >> v1:172.21.15.46:6789/0 conn(0x7f36840ad450 legacy=0x7f36840ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:17.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.091+0000 7f3683e00640 1 -- 172.21.15.46:0/1057728706 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36840ae3e0 con 0x7f368415aab0 2024-03-29T08:20:17.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.095+0000 7f3683e00640 1 --2- 172.21.15.46:0/1057728706 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f368415aab0 0x7f36840ceb90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f36740097c0 tx=0x7f367401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c1233282d714420 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:17.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.095+0000 7f3683400640 1 -- 172.21.15.46:0/1057728706 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36740a6040 con 0x7f368415aab0 2024-03-29T08:20:17.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.095+0000 7f3683400640 1 -- 172.21.15.46:0/1057728706 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36740aa600 con 0x7f368415aab0 2024-03-29T08:20:17.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.095+0000 7f3683400640 1 -- 172.21.15.46:0/1057728706 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36740aa990 con 0x7f368415aab0 2024-03-29T08:20:17.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.095+0000 7f368aa00640 1 -- 172.21.15.46:0/1057728706 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f368415aab0 msgr2=0x7f36840ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:17.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.095+0000 7f368aa00640 1 --2- 172.21.15.46:0/1057728706 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f368415aab0 0x7f36840ceb90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f36740097c0 tx=0x7f367401c4a0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.095+0000 7f368aa00640 1 -- 172.21.15.46:0/1057728706 shutdown_connections 2024-03-29T08:20:17.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.095+0000 7f368aa00640 1 --2- 172.21.15.46:0/1057728706 >> v2:172.21.15.73:3300/0 conn(0x7f36840cb120 0x7f36840cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.095+0000 7f368aa00640 1 --2- 172.21.15.46:0/1057728706 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f368415aab0 0x7f36840ceb90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.095+0000 7f368aa00640 1 -- 172.21.15.46:0/1057728706 >> 172.21.15.46:0/1057728706 conn(0x7f3684099a10 msgr2=0x7f3684089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:17.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.095+0000 7f368aa00640 1 -- 172.21.15.46:0/1057728706 shutdown_connections 2024-03-29T08:20:17.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.095+0000 7f368aa00640 1 -- 172.21.15.46:0/1057728706 wait complete. 2024-03-29T08:20:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f368aa00640 1 Processor -- start 2024-03-29T08:20:17.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f368aa00640 1 -- start start 2024-03-29T08:20:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f368aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36840ad450 0x7f36840c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f368aa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36840cb120 0x7f36840ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f368aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368415aab0 0x7f36840b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f368aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3684158cc0 con 0x7f368415aab0 2024-03-29T08:20:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f368aa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36840c6de0 con 0x7f36840ad450 2024-03-29T08:20:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f368aa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36840b4980 con 0x7f36840cb120 2024-03-29T08:20:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f3688a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368415aab0 0x7f36840b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f3688a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368415aab0 0x7f36840b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47316/0 (socket says 172.21.15.46:47316) 2024-03-29T08:20:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f3688a00640 1 -- 172.21.15.46:0/243934955 learned_addr learned my addr 172.21.15.46:0/243934955 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f367b400640 1 --2- 172.21.15.46:0/243934955 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36840cb120 0x7f36840ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f3683e00640 1 --2- 172.21.15.46:0/243934955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36840ad450 0x7f36840c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f3688a00640 1 -- 172.21.15.46:0/243934955 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36840cb120 msgr2=0x7f36840ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f3688a00640 1 --2- 172.21.15.46:0/243934955 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36840cb120 0x7f36840ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f3688a00640 1 -- 172.21.15.46:0/243934955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36840ad450 msgr2=0x7f36840c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f3688a00640 1 --2- 172.21.15.46:0/243934955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36840ad450 0x7f36840c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f3688a00640 1 -- 172.21.15.46:0/243934955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3674009420 con 0x7f368415aab0 2024-03-29T08:20:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f3688a00640 1 --2- 172.21.15.46:0/243934955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368415aab0 0x7f36840b0bf0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f367c00cb40 tx=0x7f367c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:17.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f3680c00640 1 -- 172.21.15.46:0/243934955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c00aca0 con 0x7f368415aab0 2024-03-29T08:20:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f3680c00640 1 -- 172.21.15.46:0/243934955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367c00ae00 con 0x7f368415aab0 2024-03-29T08:20:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f368aa00640 1 -- 172.21.15.46:0/243934955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3684158140 con 0x7f368415aab0 2024-03-29T08:20:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f3680c00640 1 -- 172.21.15.46:0/243934955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c012620 con 0x7f368415aab0 2024-03-29T08:20:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f368aa00640 1 -- 172.21.15.46:0/243934955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3684158700 con 0x7f368415aab0 2024-03-29T08:20:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.099+0000 7f368aa00640 1 -- 172.21.15.46:0/243934955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368409e7d0 con 0x7f368415aab0 2024-03-29T08:20:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.103+0000 7f3680c00640 1 -- 172.21.15.46:0/243934955 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f367c007070 con 0x7f368415aab0 2024-03-29T08:20:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.103+0000 7f3680c00640 1 --2- 172.21.15.46:0/243934955 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36600666c0 0x7f3660068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.103+0000 7f3680c00640 1 -- 172.21.15.46:0/243934955 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f367c0841b0 con 0x7f368415aab0 2024-03-29T08:20:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.103+0000 7f3683e00640 1 --2- 172.21.15.46:0/243934955 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36600666c0 0x7f3660068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:17.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.103+0000 7f3683e00640 1 --2- 172.21.15.46:0/243934955 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36600666c0 0x7f3660068b80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f36740ae000 tx=0x7f36740a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:17.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.107+0000 7f3680c00640 1 -- 172.21.15.46:0/243934955 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f367c051510 con 0x7f368415aab0 2024-03-29T08:20:17.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.323+0000 7f368aa00640 1 -- 172.21.15.46:0/243934955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f36840c71c0 con 0x7f368415aab0 2024-03-29T08:20:17.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.327+0000 7f3680c00640 1 -- 172.21.15.46:0/243934955 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7f367c054b30 con 0x7f368415aab0 2024-03-29T08:20:17.329 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-03-29T08:20:17.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.327+0000 7f368aa00640 1 -- 172.21.15.46:0/243934955 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36600666c0 msgr2=0x7f3660068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:17.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.327+0000 7f368aa00640 1 --2- 172.21.15.46:0/243934955 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36600666c0 0x7f3660068b80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f36740ae000 tx=0x7f36740a7000 comp rx=0 tx=0).stop 2024-03-29T08:20:17.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.327+0000 7f368aa00640 1 -- 172.21.15.46:0/243934955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368415aab0 msgr2=0x7f36840b0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:17.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.327+0000 7f368aa00640 1 --2- 172.21.15.46:0/243934955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368415aab0 0x7f36840b0bf0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f367c00cb40 tx=0x7f367c00a060 comp rx=0 tx=0).stop 2024-03-29T08:20:17.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.327+0000 7f368aa00640 1 -- 172.21.15.46:0/243934955 shutdown_connections 2024-03-29T08:20:17.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.327+0000 7f368aa00640 1 --2- 172.21.15.46:0/243934955 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36600666c0 0x7f3660068b80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.327+0000 7f368aa00640 1 --2- 172.21.15.46:0/243934955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f368415aab0 0x7f36840b0bf0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.327+0000 7f368aa00640 1 --2- 172.21.15.46:0/243934955 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36840cb120 0x7f36840ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.327+0000 7f368aa00640 1 --2- 172.21.15.46:0/243934955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36840ad450 0x7f36840c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.327+0000 7f368aa00640 1 -- 172.21.15.46:0/243934955 >> 172.21.15.46:0/243934955 conn(0x7f3684099a10 msgr2=0x7f36841569d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:17.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.327+0000 7f368aa00640 1 -- 172.21.15.46:0/243934955 shutdown_connections 2024-03-29T08:20:17.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.327+0000 7f368aa00640 1 -- 172.21.15.46:0/243934955 wait complete. 2024-03-29T08:20:17.350 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:20:17.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.487+0000 7f3f25600640 1 Processor -- start 2024-03-29T08:20:17.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.487+0000 7f3f25600640 1 -- start start 2024-03-29T08:20:17.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f25600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3f201593d0 0x7f3f2006cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:17.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f25600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2006d130 con 0x7f3f2015cd90 2024-03-29T08:20:17.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f25600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f2015b8f0 con 0x7f3f2015d230 2024-03-29T08:20:17.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f25600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f2015ad50 con 0x7f3f201593d0 2024-03-29T08:20:17.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3f2015cd90 0x7f3f201597b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59366/0 (socket says 172.21.15.46:59366) 2024-03-29T08:20:17.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1f400640 1 -- 172.21.15.46:0/3961698705 learned_addr learned my addr 172.21.15.46:0/3961698705 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:17.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1ea00640 1 --2- 172.21.15.46:0/3961698705 >> v2:172.21.15.73:3300/0 conn(0x7f3f201593d0 0x7f3f2006cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:17.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1e000640 1 -- 172.21.15.46:0/3961698705 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3813437567 0 0) 0x7f3f2006d130 con 0x7f3f2015cd90 2024-03-29T08:20:17.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1e000640 1 -- 172.21.15.46:0/3961698705 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f04003530 con 0x7f3f2015cd90 2024-03-29T08:20:17.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1e000640 1 -- 172.21.15.46:0/3961698705 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756966052 0 0) 0x7f3f2015b8f0 con 0x7f3f2015d230 2024-03-29T08:20:17.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1e000640 1 -- 172.21.15.46:0/3961698705 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f2006d130 con 0x7f3f2015d230 2024-03-29T08:20:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1e000640 1 -- 172.21.15.46:0/3961698705 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3234801290 0 0) 0x7f3f04003530 con 0x7f3f2015cd90 2024-03-29T08:20:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1e000640 1 -- 172.21.15.46:0/3961698705 >> v2:172.21.15.73:3300/0 conn(0x7f3f201593d0 msgr2=0x7f3f2006cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1e000640 1 --2- 172.21.15.46:0/3961698705 >> v2:172.21.15.73:3300/0 conn(0x7f3f201593d0 0x7f3f2006cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1e000640 1 -- 172.21.15.46:0/3961698705 >> v1:172.21.15.73:6789/0 conn(0x7f3f2015d230 legacy=0x7f3f2015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1e000640 1 -- 172.21.15.46:0/3961698705 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2015afc0 con 0x7f3f2015cd90 2024-03-29T08:20:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1e000640 1 -- 172.21.15.46:0/3961698705 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f3f18001a00 con 0x7f3f2015cd90 2024-03-29T08:20:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1e000640 1 -- 172.21.15.46:0/3961698705 >> v1:172.21.15.46:6789/0 conn(0x7f3f2015cd90 legacy=0x7f3f201597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1e000640 1 --2- 172.21.15.46:0/3961698705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f04003b10 0x7f3f04003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1e000640 1 --2- 172.21.15.46:0/3961698705 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f040047d0 0x7f3f04004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1e000640 1 --2- 172.21.15.46:0/3961698705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f04005190 0x7f3f0400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1e000640 1 -- 172.21.15.46:0/3961698705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2015ad50 con 0x7f3f04005190 2024-03-29T08:20:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1e000640 1 -- 172.21.15.46:0/3961698705 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0400c010 con 0x7f3f04003b10 2024-03-29T08:20:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1e000640 1 -- 172.21.15.46:0/3961698705 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0400c1d0 con 0x7f3f040047d0 2024-03-29T08:20:17.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1f400640 1 --2- 172.21.15.46:0/3961698705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f04005190 0x7f3f0400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:17.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1ea00640 1 --2- 172.21.15.46:0/3961698705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f04003b10 0x7f3f04003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:17.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f16000640 1 --2- 172.21.15.46:0/3961698705 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f040047d0 0x7f3f04004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:17.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1f400640 1 -- 172.21.15.46:0/3961698705 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f040047d0 msgr2=0x7f3f04004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:17.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1f400640 1 --2- 172.21.15.46:0/3961698705 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f040047d0 0x7f3f04004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1f400640 1 -- 172.21.15.46:0/3961698705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f04003b10 msgr2=0x7f3f04003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:17.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1f400640 1 --2- 172.21.15.46:0/3961698705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f04003b10 0x7f3f04003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.491+0000 7f3f1f400640 1 -- 172.21.15.46:0/3961698705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f04003530 con 0x7f3f04005190 2024-03-29T08:20:17.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.495+0000 7f3f1f400640 1 -- 172.21.15.46:0/3961698705 reap_dead start 2024-03-29T08:20:17.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.495+0000 7f3f1f400640 1 --2- 172.21.15.46:0/3961698705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f04005190 0x7f3f0400b990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3f2015b8a0 tx=0x7f3f1801fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06280cf4466e122 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:17.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.495+0000 7f3f1e000640 1 -- 172.21.15.46:0/3961698705 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f180aa630 con 0x7f3f04005190 2024-03-29T08:20:17.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.495+0000 7f3f1e000640 1 -- 172.21.15.46:0/3961698705 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f180aa7b0 con 0x7f3f04005190 2024-03-29T08:20:17.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.495+0000 7f3f1e000640 1 -- 172.21.15.46:0/3961698705 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f180aab60 con 0x7f3f04005190 2024-03-29T08:20:17.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.495+0000 7f3f25600640 1 -- 172.21.15.46:0/3961698705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f04005190 msgr2=0x7f3f0400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:17.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.495+0000 7f3f25600640 1 --2- 172.21.15.46:0/3961698705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f04005190 0x7f3f0400b990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3f2015b8a0 tx=0x7f3f1801fa00 comp rx=0 tx=0).stop 2024-03-29T08:20:17.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.495+0000 7f3f25600640 1 -- 172.21.15.46:0/3961698705 shutdown_connections 2024-03-29T08:20:17.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.495+0000 7f3f25600640 1 --2- 172.21.15.46:0/3961698705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f04005190 0x7f3f0400b990 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.495+0000 7f3f25600640 1 -- 172.21.15.46:0/3961698705 >> 172.21.15.46:0/3961698705 conn(0x7f3f20066110 msgr2=0x7f3f20069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:17.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.495+0000 7f3f25600640 1 -- 172.21.15.46:0/3961698705 shutdown_connections 2024-03-29T08:20:17.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.495+0000 7f3f25600640 1 -- 172.21.15.46:0/3961698705 wait complete. 2024-03-29T08:20:17.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.495+0000 7f3f25600640 1 Processor -- start 2024-03-29T08:20:17.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f25600640 1 -- start start 2024-03-29T08:20:17.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f25600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f04005190 0x7f3f200ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f25600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f20142a50 0x7f3f200cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f25600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f20152900 0x7f3f20156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f25600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f200ce5e0 con 0x7f3f04005190 2024-03-29T08:20:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f25600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f200ccb30 con 0x7f3f20152900 2024-03-29T08:20:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f25600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f200cce10 con 0x7f3f20142a50 2024-03-29T08:20:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f1ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f04005190 0x7f3f200ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f1ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f04005190 0x7f3f200ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47332/0 (socket says 172.21.15.46:47332) 2024-03-29T08:20:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f1ea00640 1 -- 172.21.15.46:0/781484392 learned_addr learned my addr 172.21.15.46:0/781484392 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f16000640 1 --2- 172.21.15.46:0/781484392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f20142a50 0x7f3f200cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f1f400640 1 --2- 172.21.15.46:0/781484392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f20152900 0x7f3f20156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f1ea00640 1 -- 172.21.15.46:0/781484392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f20142a50 msgr2=0x7f3f200cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f1ea00640 1 --2- 172.21.15.46:0/781484392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f20142a50 0x7f3f200cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f1ea00640 1 -- 172.21.15.46:0/781484392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f20152900 msgr2=0x7f3f20156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f1ea00640 1 --2- 172.21.15.46:0/781484392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f20152900 0x7f3f20156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f1ea00640 1 -- 172.21.15.46:0/781484392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f200cd0f0 con 0x7f3f04005190 2024-03-29T08:20:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f16000640 1 --2- 172.21.15.46:0/781484392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f20142a50 0x7f3f200cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f1ea00640 1 --2- 172.21.15.46:0/781484392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f04005190 0x7f3f200ce0a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3f10011960 tx=0x7f3f10011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f17400640 1 -- 172.21.15.46:0/781484392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1000faf0 con 0x7f3f04005190 2024-03-29T08:20:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f17400640 1 -- 172.21.15.46:0/781484392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1000fc50 con 0x7f3f04005190 2024-03-29T08:20:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f17400640 1 -- 172.21.15.46:0/781484392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10014440 con 0x7f3f04005190 2024-03-29T08:20:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f25600640 1 -- 172.21.15.46:0/781484392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f20155250 con 0x7f3f04005190 2024-03-29T08:20:17.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f25600640 1 -- 172.21.15.46:0/781484392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f200cc820 con 0x7f3f04005190 2024-03-29T08:20:17.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f25600640 1 -- 172.21.15.46:0/781484392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f20066c90 con 0x7f3f04005190 2024-03-29T08:20:17.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.499+0000 7f3f17400640 1 -- 172.21.15.46:0/781484392 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3f100145a0 con 0x7f3f04005190 2024-03-29T08:20:17.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.503+0000 7f3f17400640 1 --2- 172.21.15.46:0/781484392 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3efc0666c0 0x7f3efc068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:17.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.503+0000 7f3f17400640 1 -- 172.21.15.46:0/781484392 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f100849b0 con 0x7f3f04005190 2024-03-29T08:20:17.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.503+0000 7f3f16000640 1 --2- 172.21.15.46:0/781484392 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3efc0666c0 0x7f3efc068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:17.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.503+0000 7f3f16000640 1 --2- 172.21.15.46:0/781484392 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3efc0666c0 0x7f3efc068b80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3f200afb70 tx=0x7f3f08009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:17.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.507+0000 7f3f17400640 1 -- 172.21.15.46:0/781484392 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3f10051c80 con 0x7f3f04005190 2024-03-29T08:20:17.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.703+0000 7f3f25600640 1 -- 172.21.15.46:0/781484392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f2006d3d0 con 0x7f3f04005190 2024-03-29T08:20:17.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.703+0000 7f3f17400640 1 -- 172.21.15.46:0/781484392 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4731 (secure 0 0 0) 0x7f3f10022090 con 0x7f3f04005190 2024-03-29T08:20:17.706 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:17.707 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2702541693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2702541693},{"type":"v1","addr":"172.21.15.73:6837","nonce":2702541693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2507803771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2507803771},{"type":"v1","addr":"172.21.15.73:6835","nonce":2507803771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/494932499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":494932499},{"type":"v1","addr":"172.21.15.46:6833","nonce":494932499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:20:12.400114+0000","modified":"2024-03-29T08:20:15.405861+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3467089652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3467089652},{"type":"v1","addr":"172.21.15.46:6835","nonce":3467089652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-29T08:20:17.707 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-03-29T08:20:17.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.707+0000 7f3f25600640 1 -- 172.21.15.46:0/781484392 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3efc0666c0 msgr2=0x7f3efc068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:17.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.707+0000 7f3f25600640 1 --2- 172.21.15.46:0/781484392 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3efc0666c0 0x7f3efc068b80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3f200afb70 tx=0x7f3f08009040 comp rx=0 tx=0).stop 2024-03-29T08:20:17.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.707+0000 7f3f25600640 1 -- 172.21.15.46:0/781484392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f04005190 msgr2=0x7f3f200ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.707+0000 7f3f25600640 1 --2- 172.21.15.46:0/781484392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f04005190 0x7f3f200ce0a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3f10011960 tx=0x7f3f10011e20 comp rx=0 tx=0).stop 2024-03-29T08:20:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.707+0000 7f3f25600640 1 -- 172.21.15.46:0/781484392 shutdown_connections 2024-03-29T08:20:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.707+0000 7f3f25600640 1 --2- 172.21.15.46:0/781484392 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3efc0666c0 0x7f3efc068b80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.707+0000 7f3f25600640 1 --2- 172.21.15.46:0/781484392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f20152900 0x7f3f20156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.707+0000 7f3f25600640 1 --2- 172.21.15.46:0/781484392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3f20142a50 0x7f3f200cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.707+0000 7f3f25600640 1 --2- 172.21.15.46:0/781484392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f04005190 0x7f3f200ce0a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.707+0000 7f3f25600640 1 -- 172.21.15.46:0/781484392 >> 172.21.15.46:0/781484392 conn(0x7f3f20066110 msgr2=0x7f3f200ae0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.707+0000 7f3f25600640 1 -- 172.21.15.46:0/781484392 shutdown_connections 2024-03-29T08:20:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:17.707+0000 7f3f25600640 1 -- 172.21.15.46:0/781484392 wait complete. 2024-03-29T08:20:18.131 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:20:12.400114+0000', 'modified': '2024-03-29T08:20:15.405861+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5631}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5631': {'gid': 5631, 'name': 'a', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/3467089652', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 3467089652}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 3467089652}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:20:18.131 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T08:20:18.131 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-29T08:20:18.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.271+0000 7f1c46000640 1 Processor -- start 2024-03-29T08:20:18.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.271+0000 7f1c46000640 1 -- start start 2024-03-29T08:20:18.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.275+0000 7f1c46000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c40142ab0 0x7f1c4006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.275+0000 7f1c46000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c4006de40 con 0x7f1c4006d9e0 2024-03-29T08:20:18.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.275+0000 7f1c46000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c40070c70 con 0x7f1c4015db10 2024-03-29T08:20:18.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.275+0000 7f1c46000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c40070340 con 0x7f1c40142ab0 2024-03-29T08:20:18.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.275+0000 7f1c37e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1c4006d9e0 0x7f1c4015d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59370/0 (socket says 172.21.15.46:59370) 2024-03-29T08:20:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.275+0000 7f1c37e00640 1 -- 172.21.15.46:0/3547939579 learned_addr learned my addr 172.21.15.46:0/3547939579 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.275+0000 7f1c3f400640 1 --2- 172.21.15.46:0/3547939579 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c40142ab0 0x7f1c4006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.275+0000 7f1c3ea00640 1 -- 172.21.15.46:0/3547939579 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221487204 0 0) 0x7f1c4006de40 con 0x7f1c4006d9e0 2024-03-29T08:20:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.275+0000 7f1c3ea00640 1 -- 172.21.15.46:0/3547939579 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c24003530 con 0x7f1c4006d9e0 2024-03-29T08:20:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.275+0000 7f1c3ea00640 1 -- 172.21.15.46:0/3547939579 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859463359 0 0) 0x7f1c40070c70 con 0x7f1c4015db10 2024-03-29T08:20:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.275+0000 7f1c3ea00640 1 -- 172.21.15.46:0/3547939579 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c4006de40 con 0x7f1c4015db10 2024-03-29T08:20:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.275+0000 7f1c3f400640 1 -- 172.21.15.46:0/3547939579 >> v1:172.21.15.73:6789/0 conn(0x7f1c4015db10 legacy=0x7f1c400b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.275+0000 7f1c3f400640 1 -- 172.21.15.46:0/3547939579 >> v1:172.21.15.46:6789/0 conn(0x7f1c4006d9e0 legacy=0x7f1c4015d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:18.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.275+0000 7f1c3f400640 1 -- 172.21.15.46:0/3547939579 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c40070620 con 0x7f1c40142ab0 2024-03-29T08:20:18.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.279+0000 7f1c3f400640 1 --2- 172.21.15.46:0/3547939579 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c40142ab0 0x7f1c4006d4a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1c300097f0 tx=0x7f1c3001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26e07b601c0674ed server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:18.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.279+0000 7f1c3ea00640 1 -- 172.21.15.46:0/3547939579 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c300ab400 con 0x7f1c40142ab0 2024-03-29T08:20:18.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.279+0000 7f1c3ea00640 1 -- 172.21.15.46:0/3547939579 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c300a7050 con 0x7f1c40142ab0 2024-03-29T08:20:18.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.279+0000 7f1c3ea00640 1 -- 172.21.15.46:0/3547939579 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c300ab780 con 0x7f1c40142ab0 2024-03-29T08:20:18.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.279+0000 7f1c46000640 1 -- 172.21.15.46:0/3547939579 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c40142ab0 msgr2=0x7f1c4006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:18.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.279+0000 7f1c46000640 1 --2- 172.21.15.46:0/3547939579 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c40142ab0 0x7f1c4006d4a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1c300097f0 tx=0x7f1c3001c4e0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.279+0000 7f1c46000640 1 -- 172.21.15.46:0/3547939579 shutdown_connections 2024-03-29T08:20:18.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.279+0000 7f1c46000640 1 --2- 172.21.15.46:0/3547939579 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c40142ab0 0x7f1c4006d4a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.279+0000 7f1c46000640 1 -- 172.21.15.46:0/3547939579 >> 172.21.15.46:0/3547939579 conn(0x7f1c40066080 msgr2=0x7f1c40069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:18.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.279+0000 7f1c46000640 1 -- 172.21.15.46:0/3547939579 shutdown_connections 2024-03-29T08:20:18.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.279+0000 7f1c46000640 1 -- 172.21.15.46:0/3547939579 wait complete. 2024-03-29T08:20:18.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.279+0000 7f1c46000640 1 Processor -- start 2024-03-29T08:20:18.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.279+0000 7f1c46000640 1 -- start start 2024-03-29T08:20:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c46000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c4006d9e0 0x7f1c4013ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c46000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c40142ab0 0x7f1c4015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c46000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c4015db10 0x7f1c4015ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c46000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c40160390 con 0x7f1c40142ab0 2024-03-29T08:20:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c46000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4015b650 con 0x7f1c4006d9e0 2024-03-29T08:20:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c46000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4015b930 con 0x7f1c4015db10 2024-03-29T08:20:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c40142ab0 0x7f1c4015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c40142ab0 0x7f1c4015e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47346/0 (socket says 172.21.15.46:47346) 2024-03-29T08:20:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c37e00640 1 -- 172.21.15.46:0/3021825174 learned_addr learned my addr 172.21.15.46:0/3021825174 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c3f400640 1 --2- 172.21.15.46:0/3021825174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c4006d9e0 0x7f1c4013ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c3fe00640 1 --2- 172.21.15.46:0/3021825174 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c4015db10 0x7f1c4015ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c37e00640 1 -- 172.21.15.46:0/3021825174 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c4015db10 msgr2=0x7f1c4015ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c37e00640 1 --2- 172.21.15.46:0/3021825174 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c4015db10 0x7f1c4015ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c37e00640 1 -- 172.21.15.46:0/3021825174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c4006d9e0 msgr2=0x7f1c4013ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c37e00640 1 --2- 172.21.15.46:0/3021825174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c4006d9e0 0x7f1c4013ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c37e00640 1 -- 172.21.15.46:0/3021825174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c30009420 con 0x7f1c40142ab0 2024-03-29T08:20:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c37e00640 1 --2- 172.21.15.46:0/3021825174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c40142ab0 0x7f1c4015e3c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800c420 tx=0x7f1c2800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c37400640 1 -- 172.21.15.46:0/3021825174 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28005cf0 con 0x7f1c40142ab0 2024-03-29T08:20:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c37400640 1 -- 172.21.15.46:0/3021825174 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c28005e50 con 0x7f1c40142ab0 2024-03-29T08:20:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c37400640 1 -- 172.21.15.46:0/3021825174 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28007310 con 0x7f1c40142ab0 2024-03-29T08:20:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c46000640 1 -- 172.21.15.46:0/3021825174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4015bc10 con 0x7f1c40142ab0 2024-03-29T08:20:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.283+0000 7f1c46000640 1 -- 172.21.15.46:0/3021825174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4015e900 con 0x7f1c40142ab0 2024-03-29T08:20:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c37400640 1 -- 172.21.15.46:0/3021825174 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1c28007470 con 0x7f1c40142ab0 2024-03-29T08:20:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c37400640 1 --2- 172.21.15.46:0/3021825174 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c1c066690 0x7f1c1c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c37400640 1 -- 172.21.15.46:0/3021825174 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c28083760 con 0x7f1c40142ab0 2024-03-29T08:20:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c3f400640 1 --2- 172.21.15.46:0/3021825174 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c1c066690 0x7f1c1c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c49445000 1 Processor -- start 2024-03-29T08:20:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c49445000 1 -- start start 2024-03-29T08:20:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c3f400640 1 --2- 172.21.15.46:0/3021825174 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c1c066690 0x7f1c1c068b50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1c300ad000 tx=0x7f1c30008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c49445000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8fddb0 0x55fd2f8fe190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c49445000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fe6d0 0x55fd2f8b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c49445000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8b44c0 0x55fd2f8b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c49445000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd2f77c5a0 con 0x55fd2f8b44c0 2024-03-29T08:20:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c49445000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd2f709af0 con 0x55fd2f8fe6d0 2024-03-29T08:20:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c49445000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd2f7070c0 con 0x55fd2f8fddb0 2024-03-29T08:20:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8b44c0 0x55fd2f8b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8b44c0 0x55fd2f8b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47358/0 (socket says 172.21.15.46:47358) 2024-03-29T08:20:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c3fe00640 1 -- 172.21.15.46:0/3022550651 learned_addr learned my addr 172.21.15.46:0/3022550651 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c3f400640 1 --2- 172.21.15.46:0/3022550651 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8fddb0 0x55fd2f8fe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c37e00640 1 --2- 172.21.15.46:0/3022550651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fe6d0 0x55fd2f8b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c3fe00640 1 -- 172.21.15.46:0/3022550651 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8fddb0 msgr2=0x55fd2f8fe190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c3fe00640 1 --2- 172.21.15.46:0/3022550651 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8fddb0 0x55fd2f8fe190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c3fe00640 1 -- 172.21.15.46:0/3022550651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fe6d0 msgr2=0x55fd2f8b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c3fe00640 1 --2- 172.21.15.46:0/3022550651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fe6d0 0x55fd2f8b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c3fe00640 1 -- 172.21.15.46:0/3022550651 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd2f3fe890 con 0x55fd2f8b44c0 2024-03-29T08:20:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c3f400640 1 --2- 172.21.15.46:0/3022550651 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8fddb0 0x55fd2f8fe190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c37e00640 1 --2- 172.21.15.46:0/3022550651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fe6d0 0x55fd2f8b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.287+0000 7f1c3fe00640 1 --2- 172.21.15.46:0/3022550651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8b44c0 0x55fd2f8b48a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1c38001dd0 tx=0x7f1c380016e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fb0429e3b2c471 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c2fe00640 1 -- 172.21.15.46:0/3022550651 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3800fbe0 con 0x55fd2f8b44c0 2024-03-29T08:20:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c2fe00640 1 -- 172.21.15.46:0/3022550651 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3800fd40 con 0x55fd2f8b44c0 2024-03-29T08:20:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c2fe00640 1 -- 172.21.15.46:0/3022550651 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38003bc0 con 0x55fd2f8b44c0 2024-03-29T08:20:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 -- 172.21.15.46:0/3022550651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8b44c0 msgr2=0x55fd2f8b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 --2- 172.21.15.46:0/3022550651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8b44c0 0x55fd2f8b48a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1c38001dd0 tx=0x7f1c380016e0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 -- 172.21.15.46:0/3022550651 shutdown_connections 2024-03-29T08:20:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 --2- 172.21.15.46:0/3022550651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fe6d0 0x55fd2f8b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 --2- 172.21.15.46:0/3022550651 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8b44c0 0x55fd2f8b48a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 --2- 172.21.15.46:0/3022550651 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8fddb0 0x55fd2f8fe190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 -- 172.21.15.46:0/3022550651 >> 172.21.15.46:0/3022550651 conn(0x55fd2f8ba830 msgr2=0x55fd2f8b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 -- 172.21.15.46:0/3022550651 shutdown_connections 2024-03-29T08:20:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 -- 172.21.15.46:0/3022550651 wait complete. 2024-03-29T08:20:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 -- start start 2024-03-29T08:20:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 Processor -- start 2024-03-29T08:20:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8b44c0 0x55fd2f8b8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8fddb0 0x55fd2f8b8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fe6d0 0x55fd2f8b8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd2f7029b0 con 0x55fd2f8fddb0 2024-03-29T08:20:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd2f6a1c00 con 0x55fd2f8fe6d0 2024-03-29T08:20:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd2f6c0070 con 0x55fd2f8b44c0 2024-03-29T08:20:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 10 client.0 _ll_register_callbacks cb 0x55fd2f8a00e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:20:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8fddb0 0x55fd2f8b8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8fddb0 0x55fd2f8b8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47368/0 (socket says 172.21.15.46:47368) 2024-03-29T08:20:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c37e00640 1 -- 172.21.15.46:0/3030494615 learned_addr learned my addr 172.21.15.46:0/3030494615 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c3fe00640 1 --2- 172.21.15.46:0/3030494615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fe6d0 0x55fd2f8b8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c3f400640 1 --2- 172.21.15.46:0/3030494615 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8b44c0 0x55fd2f8b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c37e00640 1 -- 172.21.15.46:0/3030494615 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8b44c0 msgr2=0x55fd2f8b8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c37e00640 1 --2- 172.21.15.46:0/3030494615 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8b44c0 0x55fd2f8b8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c37e00640 1 -- 172.21.15.46:0/3030494615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fe6d0 msgr2=0x55fd2f8b8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c37e00640 1 --2- 172.21.15.46:0/3030494615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fe6d0 0x55fd2f8b8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c37e00640 1 -- 172.21.15.46:0/3030494615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4006de40 con 0x55fd2f8fddb0 2024-03-29T08:20:18.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c37e00640 1 --2- 172.21.15.46:0/3030494615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8fddb0 0x55fd2f8b8770 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1c28081b90 tx=0x7f1c2805cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e16899a3b282b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:20:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c2cc00640 1 -- 172.21.15.46:0/3030494615 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2806c070 con 0x55fd2f8fddb0 2024-03-29T08:20:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c49445000 1 -- 172.21.15.46:0/3030494615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd2f6bf5f0 con 0x55fd2f8fddb0 2024-03-29T08:20:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.291+0000 7f1c2cc00640 1 -- 172.21.15.46:0/3030494615 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c2805c650 con 0x55fd2f8fddb0 2024-03-29T08:20:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c2cc00640 1 -- 172.21.15.46:0/3030494615 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c2805c810 con 0x55fd2f8fddb0 2024-03-29T08:20:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 10 client.5757 fetch_fsmap learned FSMap version 24 2024-03-29T08:20:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 1 -- 172.21.15.46:0/3030494615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55fd2f6c7ab0 con 0x55fd2f8fddb0 2024-03-29T08:20:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c2cc00640 1 -- 172.21.15.46:0/3030494615 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fd2f6bf5f0 con 0x55fd2f8fddb0 2024-03-29T08:20:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c2cc00640 1 -- 172.21.15.46:0/3030494615 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1c2807baf0 con 0x55fd2f8fddb0 2024-03-29T08:20:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 10 client.5757 fetch_fsmap finished waiting for FSMap version 24 2024-03-29T08:20:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 10 client.5757 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T08:20:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 20 client.5757 populate_metadata read hostname 'smithi046' 2024-03-29T08:20:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 1 --2- 172.21.15.46:0/3030494615 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55fd2f8b96c0 0x55fd2f90ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 4 client.5757 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:20:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 1 -- 172.21.15.46:0/3030494615 --> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd2f6ff790 con 0x55fd2f8b96c0 2024-03-29T08:20:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c3f400640 1 --2- 172.21.15.46:0/3030494615 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55fd2f8b96c0 0x55fd2f90ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c3f400640 1 --2- 172.21.15.46:0/3030494615 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55fd2f8b96c0 0x55fd2f90ee50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30e2c8cc2e4f2469 server_cookie=91ab277730adae5c in_seq=0 out_seq=0 2024-03-29T08:20:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c2cc00640 10 client.5757 ms_handle_connect on v2:172.21.15.46:6834/3467089652 2024-03-29T08:20:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c2cc00640 1 -- 172.21.15.46:0/3030494615 <== mds.0 v2:172.21.15.46:6834/3467089652 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fd2f6ff790 con 0x55fd2f8b96c0 2024-03-29T08:20:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c2cc00640 10 client.5757 handle_command_reply: tid=0 2024-03-29T08:20:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 1 client.5757 shutdown 2024-03-29T08:20:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 1 -- 172.21.15.46:0/3030494615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8fddb0 msgr2=0x55fd2f8b8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 1 --2- 172.21.15.46:0/3030494615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8fddb0 0x55fd2f8b8770 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1c28081b90 tx=0x7f1c2805cae0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 1 -- 172.21.15.46:0/3030494615 shutdown_connections 2024-03-29T08:20:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 1 --2- 172.21.15.46:0/3030494615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fe6d0 0x55fd2f8b8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 1 --2- 172.21.15.46:0/3030494615 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55fd2f8b96c0 0x55fd2f90ee50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 1 --2- 172.21.15.46:0/3030494615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8fddb0 0x55fd2f8b8770 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 1 --2- 172.21.15.46:0/3030494615 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8b44c0 0x55fd2f8b8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c2cc00640 0 client.5757 ms_handle_reset on v2:172.21.15.46:6834/3467089652 2024-03-29T08:20:18.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 1 -- 172.21.15.46:0/3030494615 >> 172.21.15.46:0/3030494615 conn(0x55fd2f8ba830 msgr2=0x55fd2f8b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:18.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 1 -- 172.21.15.46:0/3030494615 shutdown_connections 2024-03-29T08:20:18.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 1 -- 172.21.15.46:0/3030494615 wait complete. 2024-03-29T08:20:18.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.295+0000 7f1c49445000 20 client.5757 trim_cache size 0 max 16384 2024-03-29T08:20:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c49445000 1 Processor -- start 2024-03-29T08:20:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c49445000 1 -- start start 2024-03-29T08:20:18.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c49445000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8b44c0 0x55fd2f957a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c49445000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fddb0 0x55fd2f957f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c49445000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8fe6d0 0x55fd2f9584a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c49445000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd2f3bf940 con 0x55fd2f8fe6d0 2024-03-29T08:20:18.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c49445000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd2f7069b0 con 0x55fd2f8fddb0 2024-03-29T08:20:18.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c49445000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd2f6bbf80 con 0x55fd2f8b44c0 2024-03-29T08:20:18.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8fe6d0 0x55fd2f9584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c3fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8fe6d0 0x55fd2f9584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47376/0 (socket says 172.21.15.46:47376) 2024-03-29T08:20:18.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c3fe00640 1 -- 172.21.15.46:0/952959283 learned_addr learned my addr 172.21.15.46:0/952959283 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:18.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c37e00640 1 --2- 172.21.15.46:0/952959283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fddb0 0x55fd2f957f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c3f400640 1 --2- 172.21.15.46:0/952959283 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8b44c0 0x55fd2f957a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c3fe00640 1 -- 172.21.15.46:0/952959283 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8b44c0 msgr2=0x55fd2f957a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c3fe00640 1 --2- 172.21.15.46:0/952959283 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8b44c0 0x55fd2f957a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c3fe00640 1 -- 172.21.15.46:0/952959283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fddb0 msgr2=0x55fd2f957f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c3fe00640 1 --2- 172.21.15.46:0/952959283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fddb0 0x55fd2f957f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c3fe00640 1 -- 172.21.15.46:0/952959283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd2f6adc00 con 0x55fd2f8fe6d0 2024-03-29T08:20:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.311+0000 7f1c3fe00640 1 --2- 172.21.15.46:0/952959283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8fe6d0 0x55fd2f9584a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x55fd2f6bf470 tx=0x7f1c3800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46b03b269d62f5a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c34c00640 1 -- 172.21.15.46:0/952959283 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3800fde0 con 0x55fd2f8fe6d0 2024-03-29T08:20:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c34c00640 1 -- 172.21.15.46:0/952959283 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c38003980 con 0x55fd2f8fe6d0 2024-03-29T08:20:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c34c00640 1 -- 172.21.15.46:0/952959283 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38003d10 con 0x55fd2f8fe6d0 2024-03-29T08:20:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 -- 172.21.15.46:0/952959283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8fe6d0 msgr2=0x55fd2f9584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 --2- 172.21.15.46:0/952959283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8fe6d0 0x55fd2f9584a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x55fd2f6bf470 tx=0x7f1c3800a000 comp rx=0 tx=0).stop 2024-03-29T08:20:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 -- 172.21.15.46:0/952959283 shutdown_connections 2024-03-29T08:20:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 --2- 172.21.15.46:0/952959283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fddb0 0x55fd2f957f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 --2- 172.21.15.46:0/952959283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8fe6d0 0x55fd2f9584a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 --2- 172.21.15.46:0/952959283 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8b44c0 0x55fd2f957a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 -- 172.21.15.46:0/952959283 >> 172.21.15.46:0/952959283 conn(0x55fd2f8ba830 msgr2=0x55fd2f8fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 -- 172.21.15.46:0/952959283 shutdown_connections 2024-03-29T08:20:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 -- 172.21.15.46:0/952959283 wait complete. 2024-03-29T08:20:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 -- start start 2024-03-29T08:20:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 Processor -- start 2024-03-29T08:20:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8b44c0 0x55fd2f942de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fddb0 0x55fd2f8faf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8fe6d0 0x55fd2f8fb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd2f705510 con 0x55fd2f8b44c0 2024-03-29T08:20:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd2f745ac0 con 0x55fd2f8fddb0 2024-03-29T08:20:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fd2f6ad440 con 0x55fd2f8fe6d0 2024-03-29T08:20:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 10 client.0 _ll_register_callbacks cb 0x55fd2f6c7590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:20:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c3f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8b44c0 0x55fd2f942de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c3f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8b44c0 0x55fd2f942de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47386/0 (socket says 172.21.15.46:47386) 2024-03-29T08:20:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c3f400640 1 -- 172.21.15.46:0/1628446111 learned_addr learned my addr 172.21.15.46:0/1628446111 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c37e00640 1 --2- 172.21.15.46:0/1628446111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fddb0 0x55fd2f8faf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c3fe00640 1 --2- 172.21.15.46:0/1628446111 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8fe6d0 0x55fd2f8fb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c3f400640 1 -- 172.21.15.46:0/1628446111 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8fe6d0 msgr2=0x55fd2f8fb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c3f400640 1 --2- 172.21.15.46:0/1628446111 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8fe6d0 0x55fd2f8fb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c3f400640 1 -- 172.21.15.46:0/1628446111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fddb0 msgr2=0x55fd2f8faf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c3f400640 1 --2- 172.21.15.46:0/1628446111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fddb0 0x55fd2f8faf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c3f400640 1 -- 172.21.15.46:0/1628446111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd2f3fe890 con 0x55fd2f8b44c0 2024-03-29T08:20:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c3fe00640 1 --2- 172.21.15.46:0/1628446111 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8fe6d0 0x55fd2f8fb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c37e00640 1 --2- 172.21.15.46:0/1628446111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fddb0 0x55fd2f8faf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c3f400640 1 --2- 172.21.15.46:0/1628446111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8b44c0 0x55fd2f942de0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1c3001a110 tx=0x7f1c3001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4804702496ac6885 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c2cc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:20:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c2cc00640 1 -- 172.21.15.46:0/1628446111 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c300c8e20 con 0x55fd2f8b44c0 2024-03-29T08:20:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c2cc00640 1 -- 172.21.15.46:0/1628446111 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c300abb30 con 0x55fd2f8b44c0 2024-03-29T08:20:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.315+0000 7f1c49445000 1 -- 172.21.15.46:0/1628446111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd2f6d2460 con 0x55fd2f8b44c0 2024-03-29T08:20:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c2cc00640 1 -- 172.21.15.46:0/1628446111 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c300abcf0 con 0x55fd2f8b44c0 2024-03-29T08:20:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 10 client.5763 fetch_fsmap learned FSMap version 24 2024-03-29T08:20:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 1 -- 172.21.15.46:0/1628446111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55fd2f3fecc0 con 0x55fd2f8b44c0 2024-03-29T08:20:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c2cc00640 1 -- 172.21.15.46:0/1628446111 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fd2f6d2460 con 0x55fd2f8b44c0 2024-03-29T08:20:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c2cc00640 1 -- 172.21.15.46:0/1628446111 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1c300c2040 con 0x55fd2f8b44c0 2024-03-29T08:20:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 10 client.5763 fetch_fsmap finished waiting for FSMap version 24 2024-03-29T08:20:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 10 client.5763 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T08:20:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 20 client.5763 populate_metadata read hostname 'smithi046' 2024-03-29T08:20:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 1 --2- 172.21.15.46:0/1628446111 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55fd2f8fb9b0 0x55fd2f8fbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 4 client.5763 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T08:20:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 1 -- 172.21.15.46:0/1628446111 --> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fd2f6cb9c0 con 0x55fd2f8fb9b0 2024-03-29T08:20:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c37e00640 1 --2- 172.21.15.46:0/1628446111 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55fd2f8fb9b0 0x55fd2f8fbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c37e00640 1 --2- 172.21.15.46:0/1628446111 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55fd2f8fb9b0 0x55fd2f8fbd90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3e605ee158a3cb3 server_cookie=8f3da9d5f5bdf855 in_seq=0 out_seq=0 2024-03-29T08:20:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c2cc00640 10 client.5763 ms_handle_connect on v2:172.21.15.46:6834/3467089652 2024-03-29T08:20:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c2cc00640 1 -- 172.21.15.46:0/1628446111 <== mds.0 v2:172.21.15.46:6834/3467089652 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55fd2f6cb9c0 con 0x55fd2f8fb9b0 2024-03-29T08:20:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c2cc00640 10 client.5763 handle_command_reply: tid=0 2024-03-29T08:20:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 1 client.5763 shutdown 2024-03-29T08:20:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 1 -- 172.21.15.46:0/1628446111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8b44c0 msgr2=0x55fd2f942de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 1 --2- 172.21.15.46:0/1628446111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8b44c0 0x55fd2f942de0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1c3001a110 tx=0x7f1c3001ad20 comp rx=0 tx=0).stop 2024-03-29T08:20:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 1 -- 172.21.15.46:0/1628446111 shutdown_connections 2024-03-29T08:20:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 1 --2- 172.21.15.46:0/1628446111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd2f8fddb0 0x55fd2f8faf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 1 --2- 172.21.15.46:0/1628446111 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55fd2f8fb9b0 0x55fd2f8fbd90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 1 --2- 172.21.15.46:0/1628446111 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fd2f8fe6d0 0x55fd2f8fb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 1 --2- 172.21.15.46:0/1628446111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fd2f8b44c0 0x55fd2f942de0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c2cc00640 0 client.5763 ms_handle_reset on v2:172.21.15.46:6834/3467089652 2024-03-29T08:20:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 1 -- 172.21.15.46:0/1628446111 >> 172.21.15.46:0/1628446111 conn(0x55fd2f8ba830 msgr2=0x55fd2f8b77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 1 -- 172.21.15.46:0/1628446111 shutdown_connections 2024-03-29T08:20:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 1 -- 172.21.15.46:0/1628446111 wait complete. 2024-03-29T08:20:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c49445000 20 client.5763 trim_cache size 0 max 16384 2024-03-29T08:20:18.325 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:20:18.325 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "e9b7960f-ec1f-4787-95c3-c806094d37f0", 2024-03-29T08:20:18.325 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-03-29T08:20:18.325 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5631, 2024-03-29T08:20:18.325 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-29T08:20:18.326 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-29T08:20:18.326 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-29T08:20:18.326 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 5.9199743150000002, 2024-03-29T08:20:18.326 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 24, 2024-03-29T08:20:18.326 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 49, 2024-03-29T08:20:18.326 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 49, 2024-03-29T08:20:18.326 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 10.255094848000001 2024-03-29T08:20:18.326 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:20:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c46000640 1 -- 172.21.15.46:0/3021825174 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c1c066690 msgr2=0x7f1c1c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c46000640 1 --2- 172.21.15.46:0/3021825174 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c1c066690 0x7f1c1c068b50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1c300ad000 tx=0x7f1c30008040 comp rx=0 tx=0).stop 2024-03-29T08:20:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c46000640 1 -- 172.21.15.46:0/3021825174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c40142ab0 msgr2=0x7f1c4015e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c46000640 1 --2- 172.21.15.46:0/3021825174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c40142ab0 0x7f1c4015e3c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1c2800c420 tx=0x7f1c2800c8e0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c46000640 1 -- 172.21.15.46:0/3021825174 shutdown_connections 2024-03-29T08:20:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c46000640 1 --2- 172.21.15.46:0/3021825174 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c1c066690 0x7f1c1c068b50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c46000640 1 --2- 172.21.15.46:0/3021825174 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c4015db10 0x7f1c4015ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c46000640 1 --2- 172.21.15.46:0/3021825174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c40142ab0 0x7f1c4015e3c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c46000640 1 --2- 172.21.15.46:0/3021825174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c4006d9e0 0x7f1c4013ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c46000640 1 -- 172.21.15.46:0/3021825174 >> 172.21.15.46:0/3021825174 conn(0x7f1c40066080 msgr2=0x7f1c40141e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c46000640 1 -- 172.21.15.46:0/3021825174 shutdown_connections 2024-03-29T08:20:18.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.319+0000 7f1c46000640 1 -- 172.21.15.46:0/3021825174 wait complete. 2024-03-29T08:20:18.341 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:20:18.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.479+0000 7f92f9000640 1 Processor -- start 2024-03-29T08:20:18.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.479+0000 7f92f9000640 1 -- start start 2024-03-29T08:20:18.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.479+0000 7f92f9000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92f415c970 0x7f92f406b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.479+0000 7f92f9000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f92f406ba90 0x7f92f40d2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.479+0000 7f92f9000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f40d0900 con 0x7f92f40d2bb0 2024-03-29T08:20:18.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.479+0000 7f92f9000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f40d3040 con 0x7f92f406ba90 2024-03-29T08:20:18.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.483+0000 7f92f9000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f415cd50 con 0x7f92f415c970 2024-03-29T08:20:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.483+0000 7f92f3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f92f40d2bb0 0x7f92f40d0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59386/0 (socket says 172.21.15.46:59386) 2024-03-29T08:20:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.483+0000 7f92f3e00640 1 -- 172.21.15.46:0/3958421350 learned_addr learned my addr 172.21.15.46:0/3958421350 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.483+0000 7f92eaa00640 1 --2- 172.21.15.46:0/3958421350 >> v2:172.21.15.73:3300/0 conn(0x7f92f406ba90 0x7f92f40d2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.483+0000 7f92f3400640 1 --2- 172.21.15.46:0/3958421350 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92f415c970 0x7f92f406b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.483+0000 7f92f2a00640 1 -- 172.21.15.46:0/3958421350 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2350935097 0 0) 0x7f92f40d0900 con 0x7f92f40d2bb0 2024-03-29T08:20:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.483+0000 7f92f2a00640 1 -- 172.21.15.46:0/3958421350 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92d8003530 con 0x7f92f40d2bb0 2024-03-29T08:20:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.483+0000 7f92eaa00640 1 -- 172.21.15.46:0/3958421350 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92f415c970 msgr2=0x7f92f406b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.483+0000 7f92eaa00640 1 --2- 172.21.15.46:0/3958421350 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92f415c970 0x7f92f406b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.483+0000 7f92eaa00640 1 -- 172.21.15.46:0/3958421350 >> v1:172.21.15.46:6789/0 conn(0x7f92f40d2bb0 legacy=0x7f92f40d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.483+0000 7f92eaa00640 1 -- 172.21.15.46:0/3958421350 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f40ce240 con 0x7f92f406ba90 2024-03-29T08:20:18.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.483+0000 7f92eaa00640 1 --2- 172.21.15.46:0/3958421350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f406ba90 0x7f92f40d2670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009d90 tx=0x7f92dc01c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1235264ff5f85774 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:18.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f2a00640 1 -- 172.21.15.46:0/3958421350 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0a5a00 con 0x7f92f406ba90 2024-03-29T08:20:18.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f2a00640 1 -- 172.21.15.46:0/3958421350 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc0a5b60 con 0x7f92f406ba90 2024-03-29T08:20:18.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f2a00640 1 -- 172.21.15.46:0/3958421350 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0ad440 con 0x7f92f406ba90 2024-03-29T08:20:18.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f9000640 1 -- 172.21.15.46:0/3958421350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f406ba90 msgr2=0x7f92f40d2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:18.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f9000640 1 --2- 172.21.15.46:0/3958421350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f406ba90 0x7f92f40d2670 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009d90 tx=0x7f92dc01c940 comp rx=0 tx=0).stop 2024-03-29T08:20:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f9000640 1 -- 172.21.15.46:0/3958421350 shutdown_connections 2024-03-29T08:20:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f9000640 1 --2- 172.21.15.46:0/3958421350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f406ba90 0x7f92f40d2670 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f9000640 1 --2- 172.21.15.46:0/3958421350 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92f415c970 0x7f92f406b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f9000640 1 -- 172.21.15.46:0/3958421350 >> 172.21.15.46:0/3958421350 conn(0x7f92f409ba50 msgr2=0x7f92f4089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f9000640 1 -- 172.21.15.46:0/3958421350 shutdown_connections 2024-03-29T08:20:18.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f9000640 1 -- 172.21.15.46:0/3958421350 wait complete. 2024-03-29T08:20:18.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f9000640 1 Processor -- start 2024-03-29T08:20:18.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f9000640 1 -- start start 2024-03-29T08:20:18.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f9000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92f406ba90 0x7f92f40b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f40d2bb0 0x7f92f41525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f415c970 0x7f92f4152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f9000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f40cba50 con 0x7f92f40d2bb0 2024-03-29T08:20:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f9000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f4153060 con 0x7f92f415c970 2024-03-29T08:20:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f9000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f40cc3e0 con 0x7f92f406ba90 2024-03-29T08:20:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92eaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f40d2bb0 0x7f92f41525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92eaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f40d2bb0 0x7f92f41525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47388/0 (socket says 172.21.15.46:47388) 2024-03-29T08:20:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92eaa00640 1 -- 172.21.15.46:0/1085780228 learned_addr learned my addr 172.21.15.46:0/1085780228 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f3400640 1 --2- 172.21.15.46:0/1085780228 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92f406ba90 0x7f92f40b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.487+0000 7f92f3e00640 1 --2- 172.21.15.46:0/1085780228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f415c970 0x7f92f4152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.491+0000 7f92eaa00640 1 -- 172.21.15.46:0/1085780228 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92f406ba90 msgr2=0x7f92f40b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:18.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.491+0000 7f92eaa00640 1 --2- 172.21.15.46:0/1085780228 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92f406ba90 0x7f92f40b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.491+0000 7f92eaa00640 1 -- 172.21.15.46:0/1085780228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f415c970 msgr2=0x7f92f4152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.491+0000 7f92eaa00640 1 --2- 172.21.15.46:0/1085780228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f415c970 0x7f92f4152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.491+0000 7f92eaa00640 1 -- 172.21.15.46:0/1085780228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc009a70 con 0x7f92f40d2bb0 2024-03-29T08:20:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.491+0000 7f92eaa00640 1 --2- 172.21.15.46:0/1085780228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f40d2bb0 0x7f92f41525e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0b0040 tx=0x7f92dc0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.491+0000 7f92ebe00640 1 -- 172.21.15.46:0/1085780228 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0add60 con 0x7f92f40d2bb0 2024-03-29T08:20:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.491+0000 7f92ebe00640 1 -- 172.21.15.46:0/1085780228 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc009450 con 0x7f92f40d2bb0 2024-03-29T08:20:18.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.491+0000 7f92ebe00640 1 -- 172.21.15.46:0/1085780228 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0b9690 con 0x7f92f40d2bb0 2024-03-29T08:20:18.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.491+0000 7f92f9000640 1 -- 172.21.15.46:0/1085780228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f40cb390 con 0x7f92f40d2bb0 2024-03-29T08:20:18.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.491+0000 7f92f9000640 1 -- 172.21.15.46:0/1085780228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f40cc070 con 0x7f92f40d2bb0 2024-03-29T08:20:18.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.491+0000 7f92f9000640 1 -- 172.21.15.46:0/1085780228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f40cbb90 con 0x7f92f40d2bb0 2024-03-29T08:20:18.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.491+0000 7f92ebe00640 1 -- 172.21.15.46:0/1085780228 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f92dc0b97f0 con 0x7f92f40d2bb0 2024-03-29T08:20:18.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.491+0000 7f92ebe00640 1 --2- 172.21.15.46:0/1085780228 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f92d0066690 0x7f92d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:18.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.495+0000 7f92ebe00640 1 -- 172.21.15.46:0/1085780228 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92dc11fa80 con 0x7f92f40d2bb0 2024-03-29T08:20:18.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.495+0000 7f92f3400640 1 --2- 172.21.15.46:0/1085780228 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f92d0066690 0x7f92d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:18.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.495+0000 7f92f3400640 1 --2- 172.21.15.46:0/1085780228 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f92d0066690 0x7f92d0068b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009ac0 tx=0x7f92e4009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:18.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.499+0000 7f92ebe00640 1 -- 172.21.15.46:0/1085780228 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f92f40cbb90 con 0x7f92f40d2bb0 2024-03-29T08:20:18.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.691+0000 7f92f9000640 1 -- 172.21.15.46:0/1085780228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f92f40cbb90 con 0x7f92f40d2bb0 2024-03-29T08:20:18.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.691+0000 7f92ebe00640 1 -- 172.21.15.46:0/1085780228 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f92f40cbb90 con 0x7f92f40d2bb0 2024-03-29T08:20:18.695 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:20:18.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.695+0000 7f92f9000640 1 -- 172.21.15.46:0/1085780228 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f92d0066690 msgr2=0x7f92d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:18.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.695+0000 7f92f9000640 1 --2- 172.21.15.46:0/1085780228 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f92d0066690 0x7f92d0068b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009ac0 tx=0x7f92e4009190 comp rx=0 tx=0).stop 2024-03-29T08:20:18.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.695+0000 7f92f9000640 1 -- 172.21.15.46:0/1085780228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f40d2bb0 msgr2=0x7f92f41525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:18.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.695+0000 7f92f9000640 1 --2- 172.21.15.46:0/1085780228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f40d2bb0 0x7f92f41525e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0b0040 tx=0x7f92dc0196e0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.695+0000 7f92f9000640 1 -- 172.21.15.46:0/1085780228 shutdown_connections 2024-03-29T08:20:18.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.695+0000 7f92f9000640 1 --2- 172.21.15.46:0/1085780228 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f92d0066690 0x7f92d0068b50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.695+0000 7f92f9000640 1 --2- 172.21.15.46:0/1085780228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f415c970 0x7f92f4152b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.695+0000 7f92f9000640 1 --2- 172.21.15.46:0/1085780228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92f40d2bb0 0x7f92f41525e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.695+0000 7f92f9000640 1 --2- 172.21.15.46:0/1085780228 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92f406ba90 0x7f92f40b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:18.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.695+0000 7f92f9000640 1 -- 172.21.15.46:0/1085780228 >> 172.21.15.46:0/1085780228 conn(0x7f92f409ba50 msgr2=0x7f92f415baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:18.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.695+0000 7f92f9000640 1 -- 172.21.15.46:0/1085780228 shutdown_connections 2024-03-29T08:20:18.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:18.695+0000 7f92f9000640 1 -- 172.21.15.46:0/1085780228 wait complete. 2024-03-29T08:20:18.716 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:20:18.716 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:20:18.716 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-29T08:20:18.717 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T08:20:18.717 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:20:18.717 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:20:18.717 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:20:18.717 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:20:18.717 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:20:18.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:18.717 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-29T08:20:18.727 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:20:18.727 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:20:18.727 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:20:18.727 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:20:18.727 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:20:18.727 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:20:18.727 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:20:18.727 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:20:18.727 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-29T08:20:18.727 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:20:18.728 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:20:18.728 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-29T08:20:18.728 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:20:18.728 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:20:18.728 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-29T08:20:18.728 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T08:20:18.728 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42318sec preferred_lft 42318sec 2024-03-29T08:20:18.728 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-29T08:20:18.728 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:20:18.728 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:20:18.728 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:20:18.728 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-29T08:20:18.728 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:20:18.728 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:d9:58:09:4c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:20:18.729 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:20:18.729 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:20:18.729 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:20:18.729 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T08:20:18.729 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T08:20:18.729 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:20:18.729 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T08:20:18.729 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:20:18.729 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:20:18.729 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:20:18.729 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T08:20:18.729 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:20:18.729 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:20:18.729 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T08:20:18.730 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-29T08:20:18.730 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:20:18.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:18.730 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:20:18.789 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:20:18.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:18.791 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-29T08:20:18.844 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-29T08:20:18.844 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:20:18.844 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T08:20:18.844 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:20:18.844 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T08:20:18.844 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:20:18.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:18.845 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:20:18.845 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:20:18.845 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:20:18.845 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:20:18.845 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:20:18.845 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:20:18.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:18.944 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:20:18.952 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T08:20:18.952 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T08:20:18.953 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:20:18.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:18.953 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-03-29T08:20:19.015 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T08:20:19.015 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:20:19.015 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T08:20:19.067 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:20:19.067 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:20:19.067 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:20:19.122 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:20:19.184 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:20:19.184 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:20:19.245 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:20:19.247 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:20:19.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:19.247 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:20:19.299 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:20:19.299 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T08:20:19.346 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:20:19.367 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:20:19.367 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:20:19.430 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:20:19.431 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:20:19.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:19.432 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:20:19.442 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-03-29T08:20:19.439+0000 7f2e6ca33580 -1 init, newargv = 0x561c6f8dfe90 newargc=15 2024-03-29T08:20:19.443 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[38064]: starting ceph client 2024-03-29T08:20:19.449 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[38064]: starting fuse 2024-03-29T08:20:19.451 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:20:19.451 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-29T08:20:19.451 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:20:19.451 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-03-29T08:20:19.451 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-03-29T08:20:19.451 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:20:19.451 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-03-29T08:20:19.451 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:20:19.452 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-03-29T08:20:19.452 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T08:20:19.452 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-03-29T08:20:19.452 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:20:19.452 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:20:19.452 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-03-29T08:20:19.452 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-03-29T08:20:19.452 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-03-29T08:20:19.452 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-03-29T08:20:19.452 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:20:19.452 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-03-29T08:20:19.452 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:20:19.452 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:20:19.452 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:20:19.452 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-03-29T08:20:19.452 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-03-29T08:20:19.453 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-03-29T08:20:19.453 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:20:19.453 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:20:19.453 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-03-29T08:20:19.453 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:20:19.575 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.38064.asok 2024-03-29T08:20:19.760 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38064.asok 2024-03-29T08:20:19.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:19.760 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38064.asok status 2024-03-29T08:20:19.939 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:20:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-03-29T08:20:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:20:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:20:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-03-29T08:20:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-03-29T08:20:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T08:20:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "38064", 2024-03-29T08:20:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-03-29T08:20:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-03-29T08:20:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-03-29T08:20:19.940 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5775, 2024-03-29T08:20:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-03-29T08:20:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-03-29T08:20:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-03-29T08:20:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 5775 2024-03-29T08:20:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:20:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:20:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:20:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2772212383 2024-03-29T08:20:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:20:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:20:19.941 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:20:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2772212383 2024-03-29T08:20:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.5775 192.168.0.1:0/2772212383", 2024-03-29T08:20:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/2772212383", 2024-03-29T08:20:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-03-29T08:20:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 24, 2024-03-29T08:20:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 49, 2024-03-29T08:20:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:20:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-03-29T08:20:19.942 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-03-29T08:20:19.942 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:20:19.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:19.955 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-03-29T08:20:19.955 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:20:19.964 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-03-29T08:20:19.964 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T08:20:19.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:19.964 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:20:20.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:20.028 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-03-29T08:20:20.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:20.086 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-03-29T08:20:20.139 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:20:20.140 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:20:20.140 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:20:20.140 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:20:20.140 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-03-29T08:20:20.140 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:20:20.247 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627777 2024-03-29T08:20:20.436 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:20:20.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.619+0000 7f6f37e00640 1 Processor -- start 2024-03-29T08:20:20.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.619+0000 7f6f37e00640 1 -- start start 2024-03-29T08:20:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.619+0000 7f6f37e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6f3815a580 0x7f6f38160db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.619+0000 7f6f37e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6f380ce6f0 0x7f6f3815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.619+0000 7f6f37e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f3815de30 con 0x7f6f3815ce20 2024-03-29T08:20:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.619+0000 7f6f37e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f3815d870 con 0x7f6f3815a580 2024-03-29T08:20:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.619+0000 7f6f37e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f3815db50 con 0x7f6f380ce6f0 2024-03-29T08:20:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.619+0000 7f6f2fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6f3815ce20 0x7f6f380cdfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59396/0 (socket says 172.21.15.46:59396) 2024-03-29T08:20:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.619+0000 7f6f36a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6f3815a580 0x7f6f38160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:20.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.619+0000 7f6f2fe00640 1 -- 172.21.15.46:0/956103099 learned_addr learned my addr 172.21.15.46:0/956103099 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:20.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.619+0000 7f6f37400640 1 --2- 172.21.15.46:0/956103099 >> v2:172.21.15.73:3300/0 conn(0x7f6f380ce6f0 0x7f6f3815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:20.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.619+0000 7f6f36a00640 1 -- 172.21.15.46:0/956103099 >> v2:172.21.15.73:3300/0 conn(0x7f6f380ce6f0 msgr2=0x7f6f3815d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:20.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.619+0000 7f6f36a00640 1 --2- 172.21.15.46:0/956103099 >> v2:172.21.15.73:3300/0 conn(0x7f6f380ce6f0 0x7f6f3815d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:20.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.619+0000 7f6f36a00640 1 -- 172.21.15.46:0/956103099 >> v1:172.21.15.46:6789/0 conn(0x7f6f3815ce20 legacy=0x7f6f380cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.619+0000 7f6f36a00640 1 -- 172.21.15.46:0/956103099 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f381605b0 con 0x7f6f3815a580 2024-03-29T08:20:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.619+0000 7f6f36000640 1 -- 172.21.15.46:0/956103099 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078801830 0 0) 0x7f6f3815de30 con 0x7f6f3815ce20 2024-03-29T08:20:20.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.619+0000 7f6f37400640 1 --2- 172.21.15.46:0/956103099 >> v2:172.21.15.73:3300/0 conn(0x7f6f380ce6f0 0x7f6f3815d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.623+0000 7f6f36a00640 1 --2- 172.21.15.46:0/956103099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f3815a580 0x7f6f38160db0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6f3806dae0 tx=0x7f6f2801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548f550049ef30fb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:20.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.623+0000 7f6f36000640 1 -- 172.21.15.46:0/956103099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f280ab610 con 0x7f6f3815a580 2024-03-29T08:20:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.623+0000 7f6f36000640 1 -- 172.21.15.46:0/956103099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f280ab770 con 0x7f6f3815a580 2024-03-29T08:20:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.623+0000 7f6f36000640 1 -- 172.21.15.46:0/956103099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f280abb00 con 0x7f6f3815a580 2024-03-29T08:20:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.623+0000 7f6f37e00640 1 -- 172.21.15.46:0/956103099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f3815a580 msgr2=0x7f6f38160db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:20.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.623+0000 7f6f37e00640 1 --2- 172.21.15.46:0/956103099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f3815a580 0x7f6f38160db0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6f3806dae0 tx=0x7f6f2801c450 comp rx=0 tx=0).stop 2024-03-29T08:20:20.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.623+0000 7f6f37e00640 1 -- 172.21.15.46:0/956103099 shutdown_connections 2024-03-29T08:20:20.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.623+0000 7f6f37e00640 1 --2- 172.21.15.46:0/956103099 >> v2:172.21.15.73:3300/0 conn(0x7f6f380ce6f0 0x7f6f3815d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:20.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.623+0000 7f6f37e00640 1 --2- 172.21.15.46:0/956103099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f3815a580 0x7f6f38160db0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:20.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.623+0000 7f6f37e00640 1 -- 172.21.15.46:0/956103099 >> 172.21.15.46:0/956103099 conn(0x7f6f380ccf20 msgr2=0x7f6f380cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:20.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.623+0000 7f6f37e00640 1 -- 172.21.15.46:0/956103099 shutdown_connections 2024-03-29T08:20:20.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.623+0000 7f6f37e00640 1 -- 172.21.15.46:0/956103099 wait complete. 2024-03-29T08:20:20.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.623+0000 7f6f37e00640 1 Processor -- start 2024-03-29T08:20:20.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.623+0000 7f6f37e00640 1 -- start start 2024-03-29T08:20:20.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f380ce6f0 0x7f6f381558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f37e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f3815a580 0x7f6f38159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f37e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f3815ce20 0x7f6f3814a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f37e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f38153510 con 0x7f6f380ce6f0 2024-03-29T08:20:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f37e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f38156ca0 con 0x7f6f3815ce20 2024-03-29T08:20:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f37e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f38155e10 con 0x7f6f3815a580 2024-03-29T08:20:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f36a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f380ce6f0 0x7f6f381558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f36a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f380ce6f0 0x7f6f381558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47406/0 (socket says 172.21.15.46:47406) 2024-03-29T08:20:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f36a00640 1 -- 172.21.15.46:0/893266076 learned_addr learned my addr 172.21.15.46:0/893266076 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f2fe00640 1 --2- 172.21.15.46:0/893266076 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f3815a580 0x7f6f38159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f37400640 1 --2- 172.21.15.46:0/893266076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f3815ce20 0x7f6f3814a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f36a00640 1 -- 172.21.15.46:0/893266076 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f3815a580 msgr2=0x7f6f38159540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f36a00640 1 --2- 172.21.15.46:0/893266076 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f3815a580 0x7f6f38159540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f36a00640 1 -- 172.21.15.46:0/893266076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f3815ce20 msgr2=0x7f6f3814a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:20.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f36a00640 1 --2- 172.21.15.46:0/893266076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f3815ce20 0x7f6f3814a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:20.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f36a00640 1 -- 172.21.15.46:0/893266076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f28009420 con 0x7f6f380ce6f0 2024-03-29T08:20:20.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f37400640 1 --2- 172.21.15.46:0/893266076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f3815ce20 0x7f6f3814a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:20.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f36a00640 1 --2- 172.21.15.46:0/893266076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f380ce6f0 0x7f6f381558d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6f28019060 tx=0x7f6f2801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:20.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f2ea00640 1 -- 172.21.15.46:0/893266076 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f28004220 con 0x7f6f380ce6f0 2024-03-29T08:20:20.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f2ea00640 1 -- 172.21.15.46:0/893266076 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f280b9070 con 0x7f6f380ce6f0 2024-03-29T08:20:20.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f37e00640 1 -- 172.21.15.46:0/893266076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f38156090 con 0x7f6f380ce6f0 2024-03-29T08:20:20.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f2ea00640 1 -- 172.21.15.46:0/893266076 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f28003620 con 0x7f6f380ce6f0 2024-03-29T08:20:20.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f37e00640 1 -- 172.21.15.46:0/893266076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f38156650 con 0x7f6f380ce6f0 2024-03-29T08:20:20.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f37e00640 1 -- 172.21.15.46:0/893266076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f3804e040 con 0x7f6f380ce6f0 2024-03-29T08:20:20.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.627+0000 7f6f2ea00640 1 -- 172.21.15.46:0/893266076 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6f28003990 con 0x7f6f380ce6f0 2024-03-29T08:20:20.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.631+0000 7f6f2ea00640 1 --2- 172.21.15.46:0/893266076 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f14066690 0x7f6f14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:20.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.631+0000 7f6f2ea00640 1 -- 172.21.15.46:0/893266076 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f280bb070 con 0x7f6f380ce6f0 2024-03-29T08:20:20.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.631+0000 7f6f2fe00640 1 --2- 172.21.15.46:0/893266076 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f14066690 0x7f6f14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:20.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.631+0000 7f6f2fe00640 1 --2- 172.21.15.46:0/893266076 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f14066690 0x7f6f14068b50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6f381f80d0 tx=0x7f6f20007df0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:20.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.635+0000 7f6f2ea00640 1 -- 172.21.15.46:0/893266076 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6f280ec1d0 con 0x7f6f380ce6f0 2024-03-29T08:20:20.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.831+0000 7f6f37e00640 1 -- 172.21.15.46:0/893266076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f380706f0 con 0x7f6f380ce6f0 2024-03-29T08:20:20.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.831+0000 7f6f2ea00640 1 -- 172.21.15.46:0/893266076 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4731 (secure 0 0 0) 0x7f6f280c4030 con 0x7f6f380ce6f0 2024-03-29T08:20:20.835 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:20.836 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2702541693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2702541693},{"type":"v1","addr":"172.21.15.73:6837","nonce":2702541693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2507803771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2507803771},{"type":"v1","addr":"172.21.15.73:6835","nonce":2507803771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/494932499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":494932499},{"type":"v1","addr":"172.21.15.46:6833","nonce":494932499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:20:12.400114+0000","modified":"2024-03-29T08:20:15.405861+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3467089652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3467089652},{"type":"v1","addr":"172.21.15.46:6835","nonce":3467089652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-29T08:20:20.836 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-03-29T08:20:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.835+0000 7f6f37e00640 1 -- 172.21.15.46:0/893266076 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f14066690 msgr2=0x7f6f14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.835+0000 7f6f37e00640 1 --2- 172.21.15.46:0/893266076 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f14066690 0x7f6f14068b50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6f381f80d0 tx=0x7f6f20007df0 comp rx=0 tx=0).stop 2024-03-29T08:20:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.835+0000 7f6f37e00640 1 -- 172.21.15.46:0/893266076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f380ce6f0 msgr2=0x7f6f381558d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.835+0000 7f6f37e00640 1 --2- 172.21.15.46:0/893266076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f380ce6f0 0x7f6f381558d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6f28019060 tx=0x7f6f2801ac90 comp rx=0 tx=0).stop 2024-03-29T08:20:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.835+0000 7f6f37e00640 1 -- 172.21.15.46:0/893266076 shutdown_connections 2024-03-29T08:20:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.835+0000 7f6f37e00640 1 --2- 172.21.15.46:0/893266076 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f14066690 0x7f6f14068b50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.835+0000 7f6f37e00640 1 --2- 172.21.15.46:0/893266076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f3815ce20 0x7f6f3814a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.835+0000 7f6f37e00640 1 --2- 172.21.15.46:0/893266076 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f3815a580 0x7f6f38159540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:20.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.835+0000 7f6f37e00640 1 --2- 172.21.15.46:0/893266076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f380ce6f0 0x7f6f381558d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:20.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.835+0000 7f6f37e00640 1 -- 172.21.15.46:0/893266076 >> 172.21.15.46:0/893266076 conn(0x7f6f380ccf20 msgr2=0x7f6f380cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:20.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.835+0000 7f6f37e00640 1 -- 172.21.15.46:0/893266076 shutdown_connections 2024-03-29T08:20:20.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:20.835+0000 7f6f37e00640 1 -- 172.21.15.46:0/893266076 wait complete. 2024-03-29T08:20:20.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-03-29T08:20:21.024 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-03-29T08:20:21.025 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-29T08:20:21.025 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-03-29T08:20:21.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.067+0000 7fcd0a494080 1 Processor -- start 2024-03-29T08:20:21.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.067+0000 7fcd0a494080 1 -- start start 2024-03-29T08:20:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.067+0000 7fcd0a494080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d886c59440 0x55d886c6de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.067+0000 7fcd0a494080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d886ce5a30 0x55d886d99c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.067+0000 7fcd0a494080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d886ce5e10 0x55d886d9a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.067+0000 7fcd0a494080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d886bd91e0 con 0x55d886ce5e10 2024-03-29T08:20:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.067+0000 7fcd0a494080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d886b74960 con 0x55d886c59440 2024-03-29T08:20:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.067+0000 7fcd0a494080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d886b74600 con 0x55d886ce5a30 2024-03-29T08:20:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd08000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d886ce5e10 0x55d886d9a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd08000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d886ce5e10 0x55d886d9a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47432/0 (socket says 172.21.15.46:47432) 2024-03-29T08:20:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd08000640 1 -- 172.21.15.46:0/4274377597 learned_addr learned my addr 172.21.15.46:0/4274377597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd07600640 1 --2- 172.21.15.46:0/4274377597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d886c59440 0x55d886c6de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd06c00640 1 --2- 172.21.15.46:0/4274377597 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d886ce5a30 0x55d886d99c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd08000640 1 -- 172.21.15.46:0/4274377597 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d886ce5a30 msgr2=0x55d886d99c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd08000640 1 --2- 172.21.15.46:0/4274377597 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d886ce5a30 0x55d886d99c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd08000640 1 -- 172.21.15.46:0/4274377597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d886c59440 msgr2=0x55d886c6de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd08000640 1 --2- 172.21.15.46:0/4274377597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d886c59440 0x55d886c6de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd08000640 1 -- 172.21.15.46:0/4274377597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0000a100 con 0x55d886ce5e10 2024-03-29T08:20:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd07600640 1 --2- 172.21.15.46:0/4274377597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d886c59440 0x55d886c6de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd06c00640 1 --2- 172.21.15.46:0/4274377597 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d886ce5a30 0x55d886d99c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:21.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd08000640 1 --2- 172.21.15.46:0/4274377597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d886ce5e10 0x55d886d9a140 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcd0001b9c0 tx=0x7fcd000033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d287f5ffee328471 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd05800640 1 -- 172.21.15.46:0/4274377597 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd000b8070 con 0x55d886ce5e10 2024-03-29T08:20:21.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd05800640 1 -- 172.21.15.46:0/4274377597 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd000aa5c0 con 0x55d886ce5e10 2024-03-29T08:20:21.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd05800640 1 -- 172.21.15.46:0/4274377597 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd000aa930 con 0x55d886ce5e10 2024-03-29T08:20:21.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd0a494080 1 -- 172.21.15.46:0/4274377597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d886ce5e10 msgr2=0x55d886d9a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd0a494080 1 --2- 172.21.15.46:0/4274377597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d886ce5e10 0x55d886d9a140 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcd0001b9c0 tx=0x7fcd000033b0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd0a494080 1 -- 172.21.15.46:0/4274377597 shutdown_connections 2024-03-29T08:20:21.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd0a494080 1 --2- 172.21.15.46:0/4274377597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d886ce5e10 0x55d886d9a140 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd0a494080 1 --2- 172.21.15.46:0/4274377597 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d886ce5a30 0x55d886d99c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd0a494080 1 --2- 172.21.15.46:0/4274377597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d886c59440 0x55d886c6de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd0a494080 1 -- 172.21.15.46:0/4274377597 >> 172.21.15.46:0/4274377597 conn(0x55d886c57910 msgr2=0x55d886c30060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd0a494080 1 -- 172.21.15.46:0/4274377597 shutdown_connections 2024-03-29T08:20:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd0a494080 1 -- 172.21.15.46:0/4274377597 wait complete. 2024-03-29T08:20:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd0a494080 1 Processor -- start 2024-03-29T08:20:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd0a494080 1 -- start start 2024-03-29T08:20:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd0a494080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d886c59440 0x55d886d04520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd0a494080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d886ce5a30 0x55d886ce5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd0a494080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d886ce5e10 0x55d886c587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd0a494080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d886b6b690 con 0x55d886ce5e10 2024-03-29T08:20:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd0a494080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d886d02540 con 0x55d886c59440 2024-03-29T08:20:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd0a494080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d886d01ac0 con 0x55d886ce5a30 2024-03-29T08:20:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd08000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d886ce5e10 0x55d886c587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd08000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d886ce5e10 0x55d886c587e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47434/0 (socket says 172.21.15.46:47434) 2024-03-29T08:20:21.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd08000640 1 -- 172.21.15.46:0/3753841087 learned_addr learned my addr 172.21.15.46:0/3753841087 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd07600640 1 --2- 172.21.15.46:0/3753841087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d886c59440 0x55d886d04520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd08000640 1 -- 172.21.15.46:0/3753841087 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d886ce5a30 msgr2=0x55d886ce5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd08000640 1 --2- 172.21.15.46:0/3753841087 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d886ce5a30 0x55d886ce5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd08000640 1 -- 172.21.15.46:0/3753841087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d886c59440 msgr2=0x55d886d04520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd08000640 1 --2- 172.21.15.46:0/3753841087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d886c59440 0x55d886d04520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd08000640 1 -- 172.21.15.46:0/3753841087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d886b838c0 con 0x55d886ce5e10 2024-03-29T08:20:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.071+0000 7fcd07600640 1 --2- 172.21.15.46:0/3753841087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d886c59440 0x55d886d04520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.075+0000 7fcd08000640 1 --2- 172.21.15.46:0/3753841087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d886ce5e10 0x55d886c587e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000a3f0 tx=0x7fcd00018280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.075+0000 7fccffe00640 1 -- 172.21.15.46:0/3753841087 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd000b8040 con 0x55d886ce5e10 2024-03-29T08:20:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.075+0000 7fccffe00640 1 -- 172.21.15.46:0/3753841087 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd000bd850 con 0x55d886ce5e10 2024-03-29T08:20:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.075+0000 7fccffe00640 1 -- 172.21.15.46:0/3753841087 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd000bdc00 con 0x55d886ce5e10 2024-03-29T08:20:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.075+0000 7fcd0a494080 1 -- 172.21.15.46:0/3753841087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d886b76d00 con 0x55d886ce5e10 2024-03-29T08:20:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.075+0000 7fcd0a494080 1 -- 172.21.15.46:0/3753841087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d886b76760 con 0x55d886ce5e10 2024-03-29T08:20:21.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.075+0000 7fccffe00640 1 -- 172.21.15.46:0/3753841087 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcd000cb460 con 0x55d886ce5e10 2024-03-29T08:20:21.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.075+0000 7fccffe00640 1 --2- 172.21.15.46:0/3753841087 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcce80650a0 0x7fcce8067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.075+0000 7fccffe00640 1 -- 172.21.15.46:0/3753841087 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd0011edd0 con 0x55d886ce5e10 2024-03-29T08:20:21.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.075+0000 7fcd0a494080 1 --2- 172.21.15.46:0/3753841087 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x55d886ce5650 0x55d886d9dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.075+0000 7fcd0a494080 1 -- 172.21.15.46:0/3753841087 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49) v8 -- 0x55d886d9e1c0 con 0x55d886ce5650 2024-03-29T08:20:21.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.075+0000 7fcd07600640 1 --2- 172.21.15.46:0/3753841087 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcce80650a0 0x7fcce8067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.075+0000 7fcd06c00640 1 --2- 172.21.15.46:0/3753841087 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x55d886ce5650 0x55d886d9dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.079+0000 7fcd06c00640 1 --2- 172.21.15.46:0/3753841087 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x55d886ce5650 0x55d886d9dc80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.079+0000 7fcd07600640 1 --2- 172.21.15.46:0/3753841087 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcce80650a0 0x7fcce8067560 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fccf8000c10 tx=0x7fccf80015e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.079+0000 7fcd06c00640 1 -- 172.21.15.46:0/3753841087 <== osd.3 v2:172.21.15.46:6808/27426 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7fccf400c410 con 0x55d886ce5650 2024-03-29T08:20:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.079+0000 7fcd0a494080 1 -- 172.21.15.46:0/3753841087 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x55d886ce5650 msgr2=0x55d886d9dc80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.079+0000 7fcd0a494080 1 --2- 172.21.15.46:0/3753841087 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x55d886ce5650 0x55d886d9dc80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.079+0000 7fcd0a494080 1 -- 172.21.15.46:0/3753841087 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcce80650a0 msgr2=0x7fcce8067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.079+0000 7fcd0a494080 1 --2- 172.21.15.46:0/3753841087 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcce80650a0 0x7fcce8067560 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fccf8000c10 tx=0x7fccf80015e0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.079+0000 7fcd0a494080 1 -- 172.21.15.46:0/3753841087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d886ce5e10 msgr2=0x55d886c587e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.079+0000 7fcd0a494080 1 --2- 172.21.15.46:0/3753841087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d886ce5e10 0x55d886c587e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000a3f0 tx=0x7fcd00018280 comp rx=0 tx=0).stop 2024-03-29T08:20:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.079+0000 7fcd08000640 1 -- 172.21.15.46:0/3753841087 reap_dead start 2024-03-29T08:20:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.079+0000 7fcd0a494080 1 -- 172.21.15.46:0/3753841087 shutdown_connections 2024-03-29T08:20:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.079+0000 7fcd0a494080 1 -- 172.21.15.46:0/3753841087 >> 172.21.15.46:0/3753841087 conn(0x55d886c57910 msgr2=0x55d886c30060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.079+0000 7fcd0a494080 1 -- 172.21.15.46:0/3753841087 shutdown_connections 2024-03-29T08:20:21.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.079+0000 7fcd0a494080 1 -- 172.21.15.46:0/3753841087 wait complete. 2024-03-29T08:20:21.084 INFO:teuthology.orchestra.run.smithi046.stdout:Yalphaparent_a 2024-03-29T08:20:21.085 DEBUG:teuthology.orchestra.run.smithi046:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-03-29T08:20:21.177 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:20:21.178 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627777, 2024-03-29T08:20:21.178 INFO:teuthology.orchestra.run.smithi046.stdout: "ancestors": [ 2024-03-29T08:20:21.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:21.178 INFO:teuthology.orchestra.run.smithi046.stdout: "dirino": 1099511627776, 2024-03-29T08:20:21.178 INFO:teuthology.orchestra.run.smithi046.stdout: "dname": "alpha", 2024-03-29T08:20:21.178 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 2 2024-03-29T08:20:21.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:21.178 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:21.178 INFO:teuthology.orchestra.run.smithi046.stdout: "dirino": 1, 2024-03-29T08:20:21.178 INFO:teuthology.orchestra.run.smithi046.stdout: "dname": "parent_a", 2024-03-29T08:20:21.178 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 4 2024-03-29T08:20:21.178 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:21.179 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-29T08:20:21.179 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 7, 2024-03-29T08:20:21.179 INFO:teuthology.orchestra.run.smithi046.stdout: "old_pools": [] 2024-03-29T08:20:21.179 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:20:21.179 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:21.182 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-03-29T08:20:21.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.267+0000 7f0c55a0f080 1 Processor -- start 2024-03-29T08:20:21.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.267+0000 7f0c55a0f080 1 -- start start 2024-03-29T08:20:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.267+0000 7f0c55a0f080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55de1fdf3440 0x55de1fe9e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.267+0000 7f0c55a0f080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1fe7fa30 0x55de1fdc9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.267+0000 7f0c55a0f080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de1fe7fe10 0x55de1fe7bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.267+0000 7f0c55a0f080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1fd0e960 con 0x55de1fe7fa30 2024-03-29T08:20:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.267+0000 7f0c55a0f080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1fd0e600 con 0x55de1fe7fe10 2024-03-29T08:20:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.267+0000 7f0c55a0f080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de1fd05690 con 0x55de1fdf3440 2024-03-29T08:20:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c52200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1fe7fa30 0x55de1fdc9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c52200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1fe7fa30 0x55de1fdc9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47446/0 (socket says 172.21.15.46:47446) 2024-03-29T08:20:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c52200640 1 -- 172.21.15.46:0/3974466911 learned_addr learned my addr 172.21.15.46:0/3974466911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c52c00640 1 --2- 172.21.15.46:0/3974466911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55de1fdf3440 0x55de1fe9e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c53600640 1 --2- 172.21.15.46:0/3974466911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de1fe7fe10 0x55de1fe7bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c52200640 1 -- 172.21.15.46:0/3974466911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55de1fdf3440 msgr2=0x55de1fe9e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c52200640 1 --2- 172.21.15.46:0/3974466911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55de1fdf3440 0x55de1fe9e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c52200640 1 -- 172.21.15.46:0/3974466911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de1fe7fe10 msgr2=0x55de1fe7bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c52200640 1 --2- 172.21.15.46:0/3974466911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de1fe7fe10 0x55de1fe7bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c52200640 1 -- 172.21.15.46:0/3974466911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c44005d30 con 0x55de1fe7fa30 2024-03-29T08:20:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c52200640 1 --2- 172.21.15.46:0/3974466911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1fe7fa30 0x55de1fdc9660 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0c40001c80 tx=0x7f0c4000b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33336d421de65d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c50e00640 1 -- 172.21.15.46:0/3974466911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4000e480 con 0x55de1fe7fa30 2024-03-29T08:20:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c50e00640 1 -- 172.21.15.46:0/3974466911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c4000e5e0 con 0x55de1fe7fa30 2024-03-29T08:20:21.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c50e00640 1 -- 172.21.15.46:0/3974466911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4000e9d0 con 0x55de1fe7fa30 2024-03-29T08:20:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c55a0f080 1 -- 172.21.15.46:0/3974466911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1fe7fa30 msgr2=0x55de1fdc9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c55a0f080 1 --2- 172.21.15.46:0/3974466911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1fe7fa30 0x55de1fdc9660 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f0c40001c80 tx=0x7f0c4000b6d0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c55a0f080 1 -- 172.21.15.46:0/3974466911 shutdown_connections 2024-03-29T08:20:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c55a0f080 1 --2- 172.21.15.46:0/3974466911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de1fe7fe10 0x55de1fe7bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c55a0f080 1 --2- 172.21.15.46:0/3974466911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1fe7fa30 0x55de1fdc9660 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c55a0f080 1 --2- 172.21.15.46:0/3974466911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55de1fdf3440 0x55de1fe9e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c55a0f080 1 -- 172.21.15.46:0/3974466911 >> 172.21.15.46:0/3974466911 conn(0x55de1fdf1910 msgr2=0x55de1fdf7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c55a0f080 1 -- 172.21.15.46:0/3974466911 shutdown_connections 2024-03-29T08:20:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c55a0f080 1 -- 172.21.15.46:0/3974466911 wait complete. 2024-03-29T08:20:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c55a0f080 1 Processor -- start 2024-03-29T08:20:21.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c55a0f080 1 -- start start 2024-03-29T08:20:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c55a0f080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de1fdf3440 0x55de1fe0c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c55a0f080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1fe7fa30 0x55de1fea6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c55a0f080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55de1fe7fe10 0x55de1fe9aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c55a0f080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1fe0ddb0 con 0x55de1fe7fa30 2024-03-29T08:20:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c55a0f080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1fea7010 con 0x55de1fdf3440 2024-03-29T08:20:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c55a0f080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de1fe0ce90 con 0x55de1fe7fe10 2024-03-29T08:20:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c52200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1fe7fa30 0x55de1fea6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c52200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1fe7fa30 0x55de1fea6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47458/0 (socket says 172.21.15.46:47458) 2024-03-29T08:20:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c52200640 1 -- 172.21.15.46:0/3821447034 learned_addr learned my addr 172.21.15.46:0/3821447034 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c52c00640 1 --2- 172.21.15.46:0/3821447034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de1fdf3440 0x55de1fe0c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.271+0000 7f0c53600640 1 --2- 172.21.15.46:0/3821447034 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55de1fe7fe10 0x55de1fe9aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.275+0000 7f0c52200640 1 -- 172.21.15.46:0/3821447034 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55de1fe7fe10 msgr2=0x55de1fe9aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.275+0000 7f0c52200640 1 --2- 172.21.15.46:0/3821447034 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55de1fe7fe10 0x55de1fe9aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.275+0000 7f0c52200640 1 -- 172.21.15.46:0/3821447034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de1fdf3440 msgr2=0x55de1fe0c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.275+0000 7f0c52200640 1 --2- 172.21.15.46:0/3821447034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de1fdf3440 0x55de1fe0c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.275+0000 7f0c52200640 1 -- 172.21.15.46:0/3821447034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de1fd10970 con 0x55de1fe7fa30 2024-03-29T08:20:21.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.275+0000 7f0c52200640 1 --2- 172.21.15.46:0/3821447034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1fe7fa30 0x55de1fea6ad0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0c40019970 tx=0x7f0c40000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.275+0000 7f0c4b400640 1 -- 172.21.15.46:0/3821447034 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4001e070 con 0x55de1fe7fa30 2024-03-29T08:20:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.275+0000 7f0c4b400640 1 -- 172.21.15.46:0/3821447034 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c40019b50 con 0x55de1fe7fa30 2024-03-29T08:20:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.275+0000 7f0c4b400640 1 -- 172.21.15.46:0/3821447034 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4000e480 con 0x55de1fe7fa30 2024-03-29T08:20:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.275+0000 7f0c55a0f080 1 -- 172.21.15.46:0/3821447034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55de1fd1d8c0 con 0x55de1fe7fa30 2024-03-29T08:20:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.275+0000 7f0c55a0f080 1 -- 172.21.15.46:0/3821447034 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de1fd10d00 con 0x55de1fe7fa30 2024-03-29T08:20:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.275+0000 7f0c4b400640 1 -- 172.21.15.46:0/3821447034 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0c4000a040 con 0x55de1fe7fa30 2024-03-29T08:20:21.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.275+0000 7f0c4b400640 1 --2- 172.21.15.46:0/3821447034 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0c340650a0 0x7f0c34067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.275+0000 7f0c4b400640 1 -- 172.21.15.46:0/3821447034 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c40084b80 con 0x55de1fe7fa30 2024-03-29T08:20:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.275+0000 7f0c52c00640 1 --2- 172.21.15.46:0/3821447034 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0c340650a0 0x7f0c34067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.275+0000 7f0c55a0f080 1 --2- 172.21.15.46:0/3821447034 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x55de1ff41c10 0x55de1ff44000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.275+0000 7f0c55a0f080 1 -- 172.21.15.46:0/3821447034 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e49) v8 -- 0x55de1ff44540 con 0x55de1ff41c10 2024-03-29T08:20:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.279+0000 7f0c53600640 1 --2- 172.21.15.46:0/3821447034 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x55de1ff41c10 0x55de1ff44000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.279+0000 7f0c52c00640 1 --2- 172.21.15.46:0/3821447034 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0c340650a0 0x7f0c34067560 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0c440ae000 tx=0x7f0c440a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.279+0000 7f0c53600640 1 --2- 172.21.15.46:0/3821447034 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x55de1ff41c10 0x55de1ff44000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.279+0000 7f0c53600640 1 -- 172.21.15.46:0/3821447034 <== osd.3 v2:172.21.15.46:6808/27426 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v49'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f0c4c00da30 con 0x55de1ff41c10 2024-03-29T08:20:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.279+0000 7f0c55a0f080 1 -- 172.21.15.46:0/3821447034 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x55de1ff41c10 msgr2=0x55de1ff44000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.279+0000 7f0c55a0f080 1 --2- 172.21.15.46:0/3821447034 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x55de1ff41c10 0x55de1ff44000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.279+0000 7f0c55a0f080 1 -- 172.21.15.46:0/3821447034 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0c340650a0 msgr2=0x7f0c34067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.279+0000 7f0c55a0f080 1 --2- 172.21.15.46:0/3821447034 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0c340650a0 0x7f0c34067560 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0c440ae000 tx=0x7f0c440a6000 comp rx=0 tx=0).stop 2024-03-29T08:20:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.279+0000 7f0c55a0f080 1 -- 172.21.15.46:0/3821447034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1fe7fa30 msgr2=0x55de1fea6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.279+0000 7f0c55a0f080 1 --2- 172.21.15.46:0/3821447034 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55de1fe7fa30 0x55de1fea6ad0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0c40019970 tx=0x7f0c40000bd0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.279+0000 7f0c53600640 1 -- 172.21.15.46:0/3821447034 reap_dead start 2024-03-29T08:20:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.279+0000 7f0c55a0f080 1 -- 172.21.15.46:0/3821447034 shutdown_connections 2024-03-29T08:20:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.279+0000 7f0c55a0f080 1 -- 172.21.15.46:0/3821447034 >> 172.21.15.46:0/3821447034 conn(0x55de1fdf1910 msgr2=0x55de1fdc9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.279+0000 7f0c55a0f080 1 -- 172.21.15.46:0/3821447034 shutdown_connections 2024-03-29T08:20:21.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.279+0000 7f0c55a0f080 1 -- 172.21.15.46:0/3821447034 wait complete. 2024-03-29T08:20:21.287 DEBUG:teuthology.orchestra.run.smithi046:> sudo daemon-helper kill ceph --cluster ceph -w 2024-03-29T08:20:21.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:21.334 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-03-29T08:20:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.479+0000 7fe7ede00640 1 Processor -- start 2024-03-29T08:20:21.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.479+0000 7fe7ede00640 1 -- start start 2024-03-29T08:20:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.479+0000 7fe7ede00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe7e815a970 0x7fe7e80d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.479+0000 7fe7ede00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe7e80b0d00 0x7fe7e80cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.479+0000 7fe7ede00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e80cba70 con 0x7fe7e80d0b00 2024-03-29T08:20:21.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.479+0000 7fe7ede00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e80d0f60 con 0x7fe7e815a970 2024-03-29T08:20:21.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.479+0000 7fe7ede00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e80b1320 con 0x7fe7e80b0d00 2024-03-29T08:20:21.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.479+0000 7fe7e6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe7e815a970 0x7fe7e80d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.479+0000 7fe7e6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe7e815a970 0x7fe7e80d05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47468/0 (socket says 172.21.15.46:47468) 2024-03-29T08:20:21.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.479+0000 7fe7e6a00640 1 -- 172.21.15.46:0/2943299956 learned_addr learned my addr 172.21.15.46:0/2943299956 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.479+0000 7fe7e7400640 1 --2- 172.21.15.46:0/2943299956 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe7e80b0d00 0x7fe7e80cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.479+0000 7fe7e6a00640 1 -- 172.21.15.46:0/2943299956 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe7e80b0d00 msgr2=0x7fe7e80cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.479+0000 7fe7e6a00640 1 --2- 172.21.15.46:0/2943299956 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe7e80b0d00 0x7fe7e80cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.479+0000 7fe7e6a00640 1 -- 172.21.15.46:0/2943299956 >> v1:172.21.15.73:6789/0 conn(0x7fe7e80d0b00 legacy=0x7fe7e80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.479+0000 7fe7e6a00640 1 -- 172.21.15.46:0/2943299956 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e80cb420 con 0x7fe7e815a970 2024-03-29T08:20:21.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.479+0000 7fe7e7400640 1 --2- 172.21.15.46:0/2943299956 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe7e80b0d00 0x7fe7e80cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:21.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.483+0000 7fe7e6a00640 1 --2- 172.21.15.46:0/2943299956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e815a970 0x7fe7e80d05c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009870 tx=0x7fe7d801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49b7880c39d9ebd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.483+0000 7fe7e5600640 1 -- 172.21.15.46:0/2943299956 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80ab640 con 0x7fe7e815a970 2024-03-29T08:20:21.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.483+0000 7fe7e5600640 1 -- 172.21.15.46:0/2943299956 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d80a7070 con 0x7fe7e815a970 2024-03-29T08:20:21.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.483+0000 7fe7e5600640 1 -- 172.21.15.46:0/2943299956 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80ab9c0 con 0x7fe7e815a970 2024-03-29T08:20:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.483+0000 7fe7ede00640 1 -- 172.21.15.46:0/2943299956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e815a970 msgr2=0x7fe7e80d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.483+0000 7fe7ede00640 1 --2- 172.21.15.46:0/2943299956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e815a970 0x7fe7e80d05c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009870 tx=0x7fe7d801c530 comp rx=0 tx=0).stop 2024-03-29T08:20:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.483+0000 7fe7ede00640 1 -- 172.21.15.46:0/2943299956 shutdown_connections 2024-03-29T08:20:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.483+0000 7fe7ede00640 1 --2- 172.21.15.46:0/2943299956 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe7e80b0d00 0x7fe7e80cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.483+0000 7fe7ede00640 1 --2- 172.21.15.46:0/2943299956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e815a970 0x7fe7e80d05c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.483+0000 7fe7ede00640 1 -- 172.21.15.46:0/2943299956 >> 172.21.15.46:0/2943299956 conn(0x7fe7e8089fd0 msgr2=0x7fe7e808a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.483+0000 7fe7ede00640 1 -- 172.21.15.46:0/2943299956 shutdown_connections 2024-03-29T08:20:21.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.483+0000 7fe7ede00640 1 -- 172.21.15.46:0/2943299956 wait complete. 2024-03-29T08:20:21.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.483+0000 7fe7ede00640 1 Processor -- start 2024-03-29T08:20:21.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.483+0000 7fe7ede00640 1 -- start start 2024-03-29T08:20:21.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7ede00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe7e80b0d00 0x7fe7e806c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7ede00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7e80d0b00 0x7fe7e80ae060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7ede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e815a970 0x7fe7e8157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7ede00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e80aead0 con 0x7fe7e815a970 2024-03-29T08:20:21.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7ede00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e80aedb0 con 0x7fe7e80d0b00 2024-03-29T08:20:21.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7ede00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e80af090 con 0x7fe7e80b0d00 2024-03-29T08:20:21.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7e6a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe7e80b0d00 0x7fe7e806c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7e6a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe7e80b0d00 0x7fe7e806c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:37144/0 (socket says 172.21.15.46:37144) 2024-03-29T08:20:21.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7e6a00640 1 -- 172.21.15.46:0/2798581704 learned_addr learned my addr 172.21.15.46:0/2798581704 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7e7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e815a970 0x7fe7e8157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7e6000640 1 --2- 172.21.15.46:0/2798581704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7e80d0b00 0x7fe7e80ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7e6a00640 1 -- 172.21.15.46:0/2798581704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7e80d0b00 msgr2=0x7fe7e80ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7e6a00640 1 --2- 172.21.15.46:0/2798581704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7e80d0b00 0x7fe7e80ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7e6a00640 1 -- 172.21.15.46:0/2798581704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e815a970 msgr2=0x7fe7e8157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7e6a00640 1 --2- 172.21.15.46:0/2798581704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e815a970 0x7fe7e8157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7e6a00640 1 -- 172.21.15.46:0/2798581704 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d80094d0 con 0x7fe7e80b0d00 2024-03-29T08:20:21.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7e7400640 1 --2- 172.21.15.46:0/2798581704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e815a970 0x7fe7e8157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:20:21.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7e6a00640 1 --2- 172.21.15.46:0/2798581704 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe7e80b0d00 0x7fe7e806c260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80ad040 tx=0x7fe7d801ad70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7df400640 1 -- 172.21.15.46:0/2798581704 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80be3f0 con 0x7fe7e80b0d00 2024-03-29T08:20:21.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7df400640 1 -- 172.21.15.46:0/2798581704 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d80be570 con 0x7fe7e80b0d00 2024-03-29T08:20:21.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7ede00640 1 -- 172.21.15.46:0/2798581704 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e80af370 con 0x7fe7e80b0d00 2024-03-29T08:20:21.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7df400640 1 -- 172.21.15.46:0/2798581704 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80be920 con 0x7fe7e80b0d00 2024-03-29T08:20:21.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.487+0000 7fe7ede00640 1 -- 172.21.15.46:0/2798581704 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e806c7a0 con 0x7fe7e80b0d00 2024-03-29T08:20:21.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7df400640 1 -- 172.21.15.46:0/2798581704 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe7d80b43f0 con 0x7fe7e80b0d00 2024-03-29T08:20:21.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7df400640 1 --2- 172.21.15.46:0/2798581704 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe7c8066690 0x7fe7c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7df400640 1 -- 172.21.15.46:0/2798581704 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7d811e940 con 0x7fe7e80b0d00 2024-03-29T08:20:21.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7e6000640 1 --2- 172.21.15.46:0/2798581704 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe7c8066690 0x7fe7c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7f12b3000 1 Processor -- start 2024-03-29T08:20:21.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7f12b3000 1 -- start start 2024-03-29T08:20:21.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7e6000640 1 --2- 172.21.15.46:0/2798581704 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe7c8066690 0x7fe7c8068b50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8070080 tx=0x7fe7d4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7f12b3000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f0d30 0x55af4f6f1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7f12b3000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6f1650 0x55af4f6a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7f12b3000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6a7f60 0x55af4f6a8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7f12b3000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4f56f170 con 0x55af4f6f0d30 2024-03-29T08:20:21.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7f12b3000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4f4fcd80 con 0x55af4f6f1650 2024-03-29T08:20:21.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7f12b3000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af4f4fa350 con 0x55af4f6a7f60 2024-03-29T08:20:21.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7e6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f0d30 0x55af4f6f1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7e6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f0d30 0x55af4f6f1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47478/0 (socket says 172.21.15.46:47478) 2024-03-29T08:20:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7e6a00640 1 -- 172.21.15.46:0/335509838 learned_addr learned my addr 172.21.15.46:0/335509838 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7e6000640 1 --2- 172.21.15.46:0/335509838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6f1650 0x55af4f6a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7e7400640 1 --2- 172.21.15.46:0/335509838 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6a7f60 0x55af4f6a8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7e6a00640 1 -- 172.21.15.46:0/335509838 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6a7f60 msgr2=0x55af4f6a8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7e6a00640 1 --2- 172.21.15.46:0/335509838 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6a7f60 0x55af4f6a8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7e6a00640 1 -- 172.21.15.46:0/335509838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6f1650 msgr2=0x55af4f6a7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7e6a00640 1 --2- 172.21.15.46:0/335509838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6f1650 0x55af4f6a7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7e6a00640 1 -- 172.21.15.46:0/335509838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af4f1f2070 con 0x55af4f6f0d30 2024-03-29T08:20:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7e6a00640 1 --2- 172.21.15.46:0/335509838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f0d30 0x55af4f6f1110 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80ef9a0 tx=0x7fe7d80ef430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebf8c9ae854642f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7d3e00640 1 -- 172.21.15.46:0/335509838 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80f7d50 con 0x55af4f6f0d30 2024-03-29T08:20:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7d3e00640 1 -- 172.21.15.46:0/335509838 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d80eddb0 con 0x55af4f6f0d30 2024-03-29T08:20:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.491+0000 7fe7d3e00640 1 -- 172.21.15.46:0/335509838 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80eef00 con 0x55af4f6f0d30 2024-03-29T08:20:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 1 -- 172.21.15.46:0/335509838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f0d30 msgr2=0x55af4f6f1110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/335509838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f0d30 0x55af4f6f1110 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80ef9a0 tx=0x7fe7d80ef430 comp rx=0 tx=0).stop 2024-03-29T08:20:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 1 -- 172.21.15.46:0/335509838 shutdown_connections 2024-03-29T08:20:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/335509838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6f1650 0x55af4f6a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/335509838 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6a7f60 0x55af4f6a8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/335509838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f0d30 0x55af4f6f1110 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 1 -- 172.21.15.46:0/335509838 >> 172.21.15.46:0/335509838 conn(0x55af4f6a3f10 msgr2=0x55af4f6a6300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:21.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 1 -- 172.21.15.46:0/335509838 shutdown_connections 2024-03-29T08:20:21.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 1 -- 172.21.15.46:0/335509838 wait complete. 2024-03-29T08:20:21.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 1 -- start start 2024-03-29T08:20:21.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 1 Processor -- start 2024-03-29T08:20:21.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6a7f60 0x55af4f6ad970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6f0d30 0x55af4f6dea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f1650 0x55af4f6def40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4f4f5c40 con 0x55af4f6f1650 2024-03-29T08:20:21.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4f494fa0 con 0x55af4f6a7f60 2024-03-29T08:20:21.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af4f4b31c0 con 0x55af4f6f0d30 2024-03-29T08:20:21.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7f12b3000 10 client.0 _ll_register_callbacks cb 0x55af4f693060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:20:21.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7e7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f1650 0x55af4f6def40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7e6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6a7f60 0x55af4f6ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7e6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6a7f60 0x55af4f6ad970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40464/0 (socket says 172.21.15.46:40464) 2024-03-29T08:20:21.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7e6a00640 1 -- 172.21.15.46:0/947352759 learned_addr learned my addr 172.21.15.46:0/947352759 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7e6000640 1 --2- 172.21.15.46:0/947352759 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6f0d30 0x55af4f6dea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7e6a00640 1 -- 172.21.15.46:0/947352759 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6f0d30 msgr2=0x55af4f6dea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7e6a00640 1 --2- 172.21.15.46:0/947352759 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6f0d30 0x55af4f6dea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7e6a00640 1 -- 172.21.15.46:0/947352759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f1650 msgr2=0x55af4f6def40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7e6a00640 1 --2- 172.21.15.46:0/947352759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f1650 0x55af4f6def40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7e6a00640 1 -- 172.21.15.46:0/947352759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e806c7a0 con 0x55af4f6a7f60 2024-03-29T08:20:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7e7400640 1 --2- 172.21.15.46:0/947352759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f1650 0x55af4f6def40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:20:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7e6a00640 1 --2- 172.21.15.46:0/947352759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6a7f60 0x55af4f6ad970 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009870 tx=0x7fe7d8000bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7de3be2bee4f05a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.495+0000 7fe7d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-29T08:20:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7d0c00640 1 -- 172.21.15.46:0/947352759 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80ef100 con 0x55af4f6a7f60 2024-03-29T08:20:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7d0c00640 1 -- 172.21.15.46:0/947352759 <== mon.1 v2:172.21.15.73:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7d80f78c0 con 0x55af4f6a7f60 2024-03-29T08:20:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 1 -- 172.21.15.46:0/947352759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af4f4f2a20 con 0x55af4f6a7f60 2024-03-29T08:20:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7d0c00640 1 -- 172.21.15.46:0/947352759 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7d80f7a80 con 0x55af4f6a7f60 2024-03-29T08:20:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 10 client.5773 fetch_fsmap learned FSMap version 24 2024-03-29T08:20:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 1 -- 172.21.15.46:0/947352759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55af4f4baba0 con 0x55af4f6a7f60 2024-03-29T08:20:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7d0c00640 1 -- 172.21.15.46:0/947352759 <== mon.1 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af4f4f2a20 con 0x55af4f6a7f60 2024-03-29T08:20:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7d0c00640 1 -- 172.21.15.46:0/947352759 <== mon.1 v2:172.21.15.73:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe7d8107070 con 0x55af4f6a7f60 2024-03-29T08:20:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 10 client.5773 fetch_fsmap finished waiting for FSMap version 24 2024-03-29T08:20:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 10 client.5773 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-03-29T08:20:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 20 client.5773 populate_metadata read hostname 'smithi046' 2024-03-29T08:20:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/947352759 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55af4f6f7f20 0x55af4f6f8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 4 client.5773 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:20:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 1 -- 172.21.15.46:0/947352759 --> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af4f4f4590 con 0x55af4f6f7f20 2024-03-29T08:20:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7e6000640 1 --2- 172.21.15.46:0/947352759 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55af4f6f7f20 0x55af4f6f8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7e6000640 1 --2- 172.21.15.46:0/947352759 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55af4f6f7f20 0x55af4f6f8330 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e17e0832aca3ef88 server_cookie=c14803b638ba8b5b in_seq=0 out_seq=0 2024-03-29T08:20:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7d0c00640 10 client.5773 ms_handle_connect on v2:172.21.15.46:6834/3467089652 2024-03-29T08:20:21.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7d0c00640 1 -- 172.21.15.46:0/947352759 <== mds.0 v2:172.21.15.46:6834/3467089652 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55af4f4f4590 con 0x55af4f6f7f20 2024-03-29T08:20:21.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7d0c00640 10 client.5773 handle_command_reply: tid=0 2024-03-29T08:20:21.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 1 client.5773 shutdown 2024-03-29T08:20:21.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 1 -- 172.21.15.46:0/947352759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6a7f60 msgr2=0x55af4f6ad970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/947352759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6a7f60 0x55af4f6ad970 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009870 tx=0x7fe7d8000bd0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 1 -- 172.21.15.46:0/947352759 shutdown_connections 2024-03-29T08:20:21.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/947352759 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55af4f6f7f20 0x55af4f6f8330 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/947352759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f1650 0x55af4f6def40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/947352759 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6f0d30 0x55af4f6dea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/947352759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6a7f60 0x55af4f6ad970 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7d0c00640 0 client.5773 ms_handle_reset on v2:172.21.15.46:6834/3467089652 2024-03-29T08:20:21.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 1 -- 172.21.15.46:0/947352759 >> 172.21.15.46:0/947352759 conn(0x55af4f6a3f10 msgr2=0x55af4f6a6b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:21.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 1 -- 172.21.15.46:0/947352759 shutdown_connections 2024-03-29T08:20:21.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 1 -- 172.21.15.46:0/947352759 wait complete. 2024-03-29T08:20:21.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.499+0000 7fe7f12b3000 20 client.5773 trim_cache size 0 max 16384 2024-03-29T08:20:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.503+0000 7f95cc600640 1 Processor -- start 2024-03-29T08:20:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.503+0000 7f95cc600640 1 -- start start 2024-03-29T08:20:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.503+0000 7f95cc600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f95c40b6360 0x7f95c40cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.503+0000 7f95cc600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95c4099150 0x7f95c40cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.503+0000 7f95cc600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c406c160 con 0x7f95c40cfa30 2024-03-29T08:20:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.503+0000 7f95cc600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c41605b0 con 0x7f95c4099150 2024-03-29T08:20:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.503+0000 7f95cc600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c40d0090 con 0x7f95c40b6360 2024-03-29T08:20:21.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.503+0000 7f95ca600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95c4099150 0x7f95c40cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.503+0000 7f95c9c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f95c40b6360 0x7f95c40cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.503+0000 7f95ca600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95c4099150 0x7f95c40cbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47504/0 (socket says 172.21.15.46:47504) 2024-03-29T08:20:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.503+0000 7f95ca600640 1 -- 172.21.15.46:0/3869105104 learned_addr learned my addr 172.21.15.46:0/3869105104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.503+0000 7f95c9c00640 1 -- 172.21.15.46:0/3869105104 >> v2:172.21.15.46:3300/0 conn(0x7f95c4099150 msgr2=0x7f95c40cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.503+0000 7f95c9c00640 1 --2- 172.21.15.46:0/3869105104 >> v2:172.21.15.46:3300/0 conn(0x7f95c4099150 0x7f95c40cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.503+0000 7f95c9c00640 1 -- 172.21.15.46:0/3869105104 >> v1:172.21.15.73:6789/0 conn(0x7f95c40cfa30 legacy=0x7f95c4098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.503+0000 7f95c9c00640 1 -- 172.21.15.46:0/3869105104 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c40cae60 con 0x7f95c40b6360 2024-03-29T08:20:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.503+0000 7f95c3e00640 1 -- 172.21.15.46:0/3869105104 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3717831830 0 0) 0x7f95c406c160 con 0x7f95c40cfa30 2024-03-29T08:20:21.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.503+0000 7f95ca600640 1 --2- 172.21.15.46:0/3869105104 >> v2:172.21.15.46:3300/0 conn(0x7f95c4099150 0x7f95c40cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:20:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.507+0000 7f95c9c00640 1 --2- 172.21.15.46:0/3869105104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95c40b6360 0x7f95c40cf4f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f95b80097c0 tx=0x7f95b801c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b20ad75303547acf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.507+0000 7f95c3e00640 1 -- 172.21.15.46:0/3869105104 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b80ab600 con 0x7f95c40b6360 2024-03-29T08:20:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.507+0000 7f95c3e00640 1 -- 172.21.15.46:0/3869105104 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b80a7050 con 0x7f95c40b6360 2024-03-29T08:20:21.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.507+0000 7f95c3e00640 1 -- 172.21.15.46:0/3869105104 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b80ab980 con 0x7f95c40b6360 2024-03-29T08:20:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.507+0000 7f95cc600640 1 -- 172.21.15.46:0/3869105104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95c40b6360 msgr2=0x7f95c40cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.507+0000 7f95cc600640 1 --2- 172.21.15.46:0/3869105104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95c40b6360 0x7f95c40cf4f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f95b80097c0 tx=0x7f95b801c3d0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.507+0000 7f95cc600640 1 -- 172.21.15.46:0/3869105104 shutdown_connections 2024-03-29T08:20:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.507+0000 7f95cc600640 1 --2- 172.21.15.46:0/3869105104 >> v2:172.21.15.46:3300/0 conn(0x7f95c4099150 0x7f95c40cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.507+0000 7f95cc600640 1 --2- 172.21.15.46:0/3869105104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95c40b6360 0x7f95c40cf4f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.507+0000 7f95cc600640 1 -- 172.21.15.46:0/3869105104 >> 172.21.15.46:0/3869105104 conn(0x7f95c40b21d0 msgr2=0x7f95c40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.507+0000 7f95cc600640 1 -- 172.21.15.46:0/3869105104 shutdown_connections 2024-03-29T08:20:21.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.507+0000 7f95cc600640 1 -- 172.21.15.46:0/3869105104 wait complete. 2024-03-29T08:20:21.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.511+0000 7f95cc600640 1 Processor -- start 2024-03-29T08:20:21.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.511+0000 7f95cc600640 1 -- start start 2024-03-29T08:20:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.511+0000 7f95cc600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f95c4099150 0x7f95c41563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.511+0000 7f95cc600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95c40b6360 0x7f95c41588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.511+0000 7f95cc600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95c40cfa30 0x7f95c415e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.511+0000 7f95cc600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c4158410 con 0x7f95c40cfa30 2024-03-29T08:20:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.511+0000 7f95cc600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c4157800 con 0x7f95c40b6360 2024-03-29T08:20:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.511+0000 7f95cc600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c4157ae0 con 0x7f95c4099150 2024-03-29T08:20:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.511+0000 7f95ca600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95c40cfa30 0x7f95c415e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.511+0000 7f95ca600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95c40cfa30 0x7f95c415e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47506/0 (socket says 172.21.15.46:47506) 2024-03-29T08:20:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.511+0000 7f95ca600640 1 -- 172.21.15.46:0/3108593353 learned_addr learned my addr 172.21.15.46:0/3108593353 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.515+0000 7f95c9c00640 1 --2- 172.21.15.46:0/3108593353 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f95c4099150 0x7f95c41563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.515+0000 7f95c9200640 1 --2- 172.21.15.46:0/3108593353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95c40b6360 0x7f95c41588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.515+0000 7f95ca600640 1 -- 172.21.15.46:0/3108593353 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f95c4099150 msgr2=0x7f95c41563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.515+0000 7f95ca600640 1 --2- 172.21.15.46:0/3108593353 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f95c4099150 0x7f95c41563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.515+0000 7f95ca600640 1 -- 172.21.15.46:0/3108593353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95c40b6360 msgr2=0x7f95c41588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.515+0000 7f95ca600640 1 --2- 172.21.15.46:0/3108593353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95c40b6360 0x7f95c41588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.515+0000 7f95ca600640 1 -- 172.21.15.46:0/3108593353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc009b30 con 0x7f95c40cfa30 2024-03-29T08:20:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.515+0000 7f95ca600640 1 --2- 172.21.15.46:0/3108593353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95c40cfa30 0x7f95c415e0a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00ef90 tx=0x7f95bc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.515+0000 7f95c1600640 1 -- 172.21.15.46:0/3108593353 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00cd20 con 0x7f95c40cfa30 2024-03-29T08:20:21.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.515+0000 7f95c1600640 1 -- 172.21.15.46:0/3108593353 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95bc00f040 con 0x7f95c40cfa30 2024-03-29T08:20:21.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.515+0000 7f95c1600640 1 -- 172.21.15.46:0/3108593353 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00cea0 con 0x7f95c40cfa30 2024-03-29T08:20:21.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.515+0000 7f95cc600640 1 -- 172.21.15.46:0/3108593353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b8009420 con 0x7f95c40cfa30 2024-03-29T08:20:21.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.515+0000 7f95cc600640 1 -- 172.21.15.46:0/3108593353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c41580a0 con 0x7f95c40cfa30 2024-03-29T08:20:21.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.515+0000 7f95cc600640 1 -- 172.21.15.46:0/3108593353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f95c4158260 con 0x7f95c40cfa30 2024-03-29T08:20:21.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.515+0000 7f95c1600640 1 -- 172.21.15.46:0/3108593353 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f95bc003be0 con 0x7f95c40cfa30 2024-03-29T08:20:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7f95c1600640 1 --2- 172.21.15.46:0/3108593353 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f95a4066690 0x7f95a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7f95c1600640 1 -- 172.21.15.46:0/3108593353 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95bc085540 con 0x7f95c40cfa30 2024-03-29T08:20:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7f95c1600640 1 -- 172.21.15.46:0/3108593353 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+501 (secure 0 0 0) 0x7f95bc0858d0 con 0x7f95c40cfa30 2024-03-29T08:20:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7f95c9c00640 1 --2- 172.21.15.46:0/3108593353 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f95a4066690 0x7f95a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7f95cc600640 1 -- 172.21.15.46:0/3108593353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f95c409e6b0 con 0x7f95c40cfa30 2024-03-29T08:20:21.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7f95c9c00640 1 --2- 172.21.15.46:0/3108593353 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f95a4066690 0x7f95a4068b50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f95b80ad000 tx=0x7f95b8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7f12b3000 1 Processor -- start 2024-03-29T08:20:21.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7f12b3000 1 -- start start 2024-03-29T08:20:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7f12b3000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6a7f60 0x55af4f6f0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7f12b3000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6f0d30 0x55af4f6f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7f12b3000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6f1650 0x55af4f6f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7f12b3000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4f1b2cf0 con 0x55af4f6a7f60 2024-03-29T08:20:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7f12b3000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4f6abd80 con 0x55af4f6f0d30 2024-03-29T08:20:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7f12b3000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af4f56f170 con 0x55af4f6f1650 2024-03-29T08:20:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7e6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6a7f60 0x55af4f6f0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7e6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6a7f60 0x55af4f6f0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47520/0 (socket says 172.21.15.46:47520) 2024-03-29T08:20:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7e6a00640 1 -- 172.21.15.46:0/2497344676 learned_addr learned my addr 172.21.15.46:0/2497344676 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7e6000640 1 --2- 172.21.15.46:0/2497344676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6f0d30 0x55af4f6f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7e7400640 1 --2- 172.21.15.46:0/2497344676 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6f1650 0x55af4f6f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7e6a00640 1 -- 172.21.15.46:0/2497344676 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6f1650 msgr2=0x55af4f6f3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7e6a00640 1 --2- 172.21.15.46:0/2497344676 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6f1650 0x55af4f6f3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7e6a00640 1 -- 172.21.15.46:0/2497344676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6f0d30 msgr2=0x55af4f6f06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7e6a00640 1 --2- 172.21.15.46:0/2497344676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6f0d30 0x55af4f6f06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7e6a00640 1 -- 172.21.15.46:0/2497344676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af4f4a0ef0 con 0x55af4f6a7f60 2024-03-29T08:20:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7e7400640 1 --2- 172.21.15.46:0/2497344676 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6f1650 0x55af4f6f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7e6a00640 1 --2- 172.21.15.46:0/2497344676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6a7f60 0x55af4f6f0180 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80ef710 tx=0x7fe7d80f2070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c9af80eb1e7690 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7dcc00640 1 -- 172.21.15.46:0/2497344676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80ef550 con 0x55af4f6a7f60 2024-03-29T08:20:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7dcc00640 1 -- 172.21.15.46:0/2497344676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d80f78c0 con 0x55af4f6a7f60 2024-03-29T08:20:21.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.519+0000 7fe7dcc00640 1 -- 172.21.15.46:0/2497344676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80ef140 con 0x55af4f6a7f60 2024-03-29T08:20:21.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 -- 172.21.15.46:0/2497344676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6a7f60 msgr2=0x55af4f6f0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/2497344676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6a7f60 0x55af4f6f0180 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80ef710 tx=0x7fe7d80f2070 comp rx=0 tx=0).stop 2024-03-29T08:20:21.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 -- 172.21.15.46:0/2497344676 shutdown_connections 2024-03-29T08:20:21.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/2497344676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6f0d30 0x55af4f6f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/2497344676 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6f1650 0x55af4f6f3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/2497344676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6a7f60 0x55af4f6f0180 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 -- 172.21.15.46:0/2497344676 >> 172.21.15.46:0/2497344676 conn(0x55af4f6a3f10 msgr2=0x55af4f6efa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:21.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 -- 172.21.15.46:0/2497344676 shutdown_connections 2024-03-29T08:20:21.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 -- 172.21.15.46:0/2497344676 wait complete. 2024-03-29T08:20:21.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 -- start start 2024-03-29T08:20:21.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 Processor -- start 2024-03-29T08:20:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6a7f60 0x55af4f74f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f0d30 0x55af4f6f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6f1650 0x55af4f6f6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4f4c5330 con 0x55af4f6f0d30 2024-03-29T08:20:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af4f4fe010 con 0x55af4f6a7f60 2024-03-29T08:20:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af4f4f87a0 con 0x55af4f6f1650 2024-03-29T08:20:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 10 client.0 _ll_register_callbacks cb 0x55af4f56f0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:20:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7e6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f0d30 0x55af4f6f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7e6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f0d30 0x55af4f6f5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47524/0 (socket says 172.21.15.46:47524) 2024-03-29T08:20:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7e6000640 1 -- 172.21.15.46:0/2067074037 learned_addr learned my addr 172.21.15.46:0/2067074037 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7e6a00640 1 --2- 172.21.15.46:0/2067074037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6a7f60 0x55af4f74f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7e7400640 1 --2- 172.21.15.46:0/2067074037 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6f1650 0x55af4f6f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7e6000640 1 -- 172.21.15.46:0/2067074037 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6f1650 msgr2=0x55af4f6f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7e6000640 1 --2- 172.21.15.46:0/2067074037 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6f1650 0x55af4f6f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7e6000640 1 -- 172.21.15.46:0/2067074037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6a7f60 msgr2=0x55af4f74f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7e6000640 1 --2- 172.21.15.46:0/2067074037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6a7f60 0x55af4f74f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7e6000640 1 -- 172.21.15.46:0/2067074037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af4f4baba0 con 0x55af4f6f0d30 2024-03-29T08:20:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7e6000640 1 --2- 172.21.15.46:0/2067074037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f0d30 0x55af4f6f5d50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55af4f494fa0 tx=0x7fe7d4001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd384addc0cd853 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7d0c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:20:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7d0c00640 1 -- 172.21.15.46:0/2067074037 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4029540 con 0x55af4f6f0d30 2024-03-29T08:20:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7d0c00640 1 -- 172.21.15.46:0/2067074037 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7d401c040 con 0x55af4f6f0d30 2024-03-29T08:20:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 -- 172.21.15.46:0/2067074037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af4f707ab0 con 0x55af4f6f0d30 2024-03-29T08:20:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7d0c00640 1 -- 172.21.15.46:0/2067074037 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7d4014450 con 0x55af4f6f0d30 2024-03-29T08:20:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 10 client.5832 fetch_fsmap learned FSMap version 24 2024-03-29T08:20:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.523+0000 7fe7f12b3000 1 -- 172.21.15.46:0/2067074037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55af4f494a60 con 0x55af4f6f0d30 2024-03-29T08:20:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7d0c00640 1 -- 172.21.15.46:0/2067074037 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af4f707ab0 con 0x55af4f6f0d30 2024-03-29T08:20:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7d0c00640 1 -- 172.21.15.46:0/2067074037 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe7d4012070 con 0x55af4f6f0d30 2024-03-29T08:20:21.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 10 client.5832 fetch_fsmap finished waiting for FSMap version 24 2024-03-29T08:20:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 10 client.5832 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-03-29T08:20:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 20 client.5832 populate_metadata read hostname 'smithi046' 2024-03-29T08:20:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/2067074037 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55af4f6f67d0 0x55af4f6f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 4 client.5832 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-03-29T08:20:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 1 -- 172.21.15.46:0/2067074037 --> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55af4f4b25f0 con 0x55af4f6f67d0 2024-03-29T08:20:21.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7e6a00640 1 --2- 172.21.15.46:0/2067074037 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55af4f6f67d0 0x55af4f6f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7e6a00640 1 --2- 172.21.15.46:0/2067074037 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55af4f6f67d0 0x55af4f6f6be0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aab6ed0ef45659c server_cookie=d5452a3f8aa0201b in_seq=0 out_seq=0 2024-03-29T08:20:21.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7d0c00640 10 client.5832 ms_handle_connect on v2:172.21.15.46:6834/3467089652 2024-03-29T08:20:21.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7d0c00640 1 -- 172.21.15.46:0/2067074037 <== mds.0 v2:172.21.15.46:6834/3467089652 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55af4f4b25f0 con 0x55af4f6f67d0 2024-03-29T08:20:21.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7d0c00640 10 client.5832 handle_command_reply: tid=0 2024-03-29T08:20:21.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 1 client.5832 shutdown 2024-03-29T08:20:21.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 1 -- 172.21.15.46:0/2067074037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f0d30 msgr2=0x55af4f6f5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/2067074037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f0d30 0x55af4f6f5d50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55af4f494fa0 tx=0x7fe7d4001130 comp rx=0 tx=0).stop 2024-03-29T08:20:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 1 -- 172.21.15.46:0/2067074037 shutdown_connections 2024-03-29T08:20:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/2067074037 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55af4f6f67d0 0x55af4f6f6be0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/2067074037 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55af4f6f1650 0x55af4f6f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/2067074037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55af4f6f0d30 0x55af4f6f5d50 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7d0c00640 0 client.5832 ms_handle_reset on v2:172.21.15.46:6834/3467089652 2024-03-29T08:20:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 1 --2- 172.21.15.46:0/2067074037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55af4f6a7f60 0x55af4f74f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 1 -- 172.21.15.46:0/2067074037 >> 172.21.15.46:0/2067074037 conn(0x55af4f6a3f10 msgr2=0x55af4f6fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 1 -- 172.21.15.46:0/2067074037 shutdown_connections 2024-03-29T08:20:21.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 1 -- 172.21.15.46:0/2067074037 wait complete. 2024-03-29T08:20:21.533 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-03-29T08:20:21.527+0000 7ff540400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5775=0-4194304@1} caps={5775=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5775 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x559570b12100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-29T08:20:21.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7f12b3000 20 client.5832 trim_cache size 0 max 16384 2024-03-29T08:20:21.534 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:20:21.534 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-29T08:20:21.534 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "466a2478-b73d-43c0-9498-bb2ddd000464", 2024-03-29T08:20:21.534 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-29T08:20:21.534 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:20:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7ede00640 1 -- 172.21.15.46:0/2798581704 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe7c8066690 msgr2=0x7fe7c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7ede00640 1 --2- 172.21.15.46:0/2798581704 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe7c8066690 0x7fe7c8068b50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe7e8070080 tx=0x7fe7d4009040 comp rx=0 tx=0).stop 2024-03-29T08:20:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7ede00640 1 -- 172.21.15.46:0/2798581704 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe7e80b0d00 msgr2=0x7fe7e806c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7ede00640 1 --2- 172.21.15.46:0/2798581704 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe7e80b0d00 0x7fe7e806c260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe7d80ad040 tx=0x7fe7d801ad70 comp rx=0 tx=0).stop 2024-03-29T08:20:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7ede00640 1 -- 172.21.15.46:0/2798581704 shutdown_connections 2024-03-29T08:20:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7ede00640 1 --2- 172.21.15.46:0/2798581704 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe7c8066690 0x7fe7c8068b50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7ede00640 1 --2- 172.21.15.46:0/2798581704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7e80d0b00 0x7fe7e80ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7ede00640 1 --2- 172.21.15.46:0/2798581704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe7e815a970 0x7fe7e8157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7ede00640 1 --2- 172.21.15.46:0/2798581704 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe7e80b0d00 0x7fe7e806c260 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7ede00640 1 -- 172.21.15.46:0/2798581704 >> 172.21.15.46:0/2798581704 conn(0x7fe7e8089fd0 msgr2=0x7fe7e80b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7ede00640 1 -- 172.21.15.46:0/2798581704 shutdown_connections 2024-03-29T08:20:21.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.527+0000 7fe7ede00640 1 -- 172.21.15.46:0/2798581704 wait complete. 2024-03-29T08:20:21.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:21.550 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-03-29T08:20:21.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.691+0000 7f40ce800640 1 Processor -- start 2024-03-29T08:20:21.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.691+0000 7f40ce800640 1 -- start start 2024-03-29T08:20:21.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.691+0000 7f40ce800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f40c8143180 0x7f40c80adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.691+0000 7f40ce800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f40c80ae1e0 0x7f40c80cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.691+0000 7f40ce800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c80afe80 con 0x7f40c8142da0 2024-03-29T08:20:21.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.691+0000 7f40ce800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c806cd80 con 0x7f40c80ae1e0 2024-03-29T08:20:21.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.691+0000 7f40ce800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c806d020 con 0x7f40c8143180 2024-03-29T08:20:21.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.691+0000 7f40c7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f40c80ae1e0 0x7f40c80cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.691+0000 7f40bea00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f40c8143180 0x7f40c80adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.691+0000 7f40c7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f40c80ae1e0 0x7f40c80cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40516/0 (socket says 172.21.15.46:40516) 2024-03-29T08:20:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.691+0000 7f40c7e00640 1 -- 172.21.15.46:0/274577129 learned_addr learned my addr 172.21.15.46:0/274577129 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.691+0000 7f40c7e00640 1 -- 172.21.15.46:0/274577129 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f40c8143180 msgr2=0x7f40c80adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.691+0000 7f40c7e00640 1 --2- 172.21.15.46:0/274577129 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f40c8143180 0x7f40c80adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.691+0000 7f40c7e00640 1 -- 172.21.15.46:0/274577129 >> v1:172.21.15.73:6789/0 conn(0x7f40c8142da0 legacy=0x7f40c80abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.691+0000 7f40c7e00640 1 -- 172.21.15.46:0/274577129 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c80b00f0 con 0x7f40c80ae1e0 2024-03-29T08:20:21.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.691+0000 7f40c6a00640 1 -- 172.21.15.46:0/274577129 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211873028 0 0) 0x7f40c80afe80 con 0x7f40c8142da0 2024-03-29T08:20:21.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.695+0000 7f40c7e00640 1 --2- 172.21.15.46:0/274577129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40c80ae1e0 0x7f40c80cb790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009870 tx=0x7f40c001c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5f3c2b2751afaa3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.695+0000 7f40c6a00640 1 -- 172.21.15.46:0/274577129 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c00ab5f0 con 0x7f40c80ae1e0 2024-03-29T08:20:21.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.695+0000 7f40c6a00640 1 -- 172.21.15.46:0/274577129 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c00ab750 con 0x7f40c80ae1e0 2024-03-29T08:20:21.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.695+0000 7f40c6a00640 1 -- 172.21.15.46:0/274577129 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c00abae0 con 0x7f40c80ae1e0 2024-03-29T08:20:21.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.695+0000 7f40ce800640 1 -- 172.21.15.46:0/274577129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40c80ae1e0 msgr2=0x7f40c80cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.695+0000 7f40ce800640 1 --2- 172.21.15.46:0/274577129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40c80ae1e0 0x7f40c80cb790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f40c0009870 tx=0x7f40c001c560 comp rx=0 tx=0).stop 2024-03-29T08:20:21.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.695+0000 7f40ce800640 1 -- 172.21.15.46:0/274577129 shutdown_connections 2024-03-29T08:20:21.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.695+0000 7f40ce800640 1 --2- 172.21.15.46:0/274577129 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f40c8143180 0x7f40c80adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.695+0000 7f40ce800640 1 --2- 172.21.15.46:0/274577129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40c80ae1e0 0x7f40c80cb790 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.695+0000 7f40ce800640 1 -- 172.21.15.46:0/274577129 >> 172.21.15.46:0/274577129 conn(0x7f40c806a390 msgr2=0x7f40c80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:21.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.695+0000 7f40ce800640 1 -- 172.21.15.46:0/274577129 shutdown_connections 2024-03-29T08:20:21.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.695+0000 7f40ce800640 1 -- 172.21.15.46:0/274577129 wait complete. 2024-03-29T08:20:21.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40ce800640 1 Processor -- start 2024-03-29T08:20:21.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40ce800640 1 -- start start 2024-03-29T08:20:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40ce800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c80ae1e0 0x7f40c8140670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40ce800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40c8142da0 0x7f40c8145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40ce800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f40c8143180 0x7f40c813e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40ce800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c813def0 con 0x7f40c80ae1e0 2024-03-29T08:20:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40ce800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c813d270 con 0x7f40c8142da0 2024-03-29T08:20:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40ce800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c813d550 con 0x7f40c8143180 2024-03-29T08:20:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c80ae1e0 0x7f40c8140670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c80ae1e0 0x7f40c8140670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47526/0 (socket says 172.21.15.46:47526) 2024-03-29T08:20:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40c7400640 1 -- 172.21.15.46:0/483150110 learned_addr learned my addr 172.21.15.46:0/483150110 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40bea00640 1 --2- 172.21.15.46:0/483150110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40c8142da0 0x7f40c8145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40c7e00640 1 --2- 172.21.15.46:0/483150110 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f40c8143180 0x7f40c813e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40c7400640 1 -- 172.21.15.46:0/483150110 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f40c8143180 msgr2=0x7f40c813e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40c7400640 1 --2- 172.21.15.46:0/483150110 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f40c8143180 0x7f40c813e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40c7400640 1 -- 172.21.15.46:0/483150110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40c8142da0 msgr2=0x7f40c8145300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40c7400640 1 --2- 172.21.15.46:0/483150110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40c8142da0 0x7f40c8145300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40c7400640 1 -- 172.21.15.46:0/483150110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c00094d0 con 0x7f40c80ae1e0 2024-03-29T08:20:21.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40bea00640 1 --2- 172.21.15.46:0/483150110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40c8142da0 0x7f40c8145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:20:21.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40c7400640 1 --2- 172.21.15.46:0/483150110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c80ae1e0 0x7f40c8140670 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f40b800c4d0 tx=0x7f40b800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40bfe00640 1 -- 172.21.15.46:0/483150110 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8005480 con 0x7f40c80ae1e0 2024-03-29T08:20:21.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40bfe00640 1 -- 172.21.15.46:0/483150110 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b8003600 con 0x7f40c80ae1e0 2024-03-29T08:20:21.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40bfe00640 1 -- 172.21.15.46:0/483150110 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b80039b0 con 0x7f40c80ae1e0 2024-03-29T08:20:21.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40ce800640 1 -- 172.21.15.46:0/483150110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c813d830 con 0x7f40c80ae1e0 2024-03-29T08:20:21.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.699+0000 7f40ce800640 1 -- 172.21.15.46:0/483150110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c813ce90 con 0x7f40c80ae1e0 2024-03-29T08:20:21.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40bfe00640 1 -- 172.21.15.46:0/483150110 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f40b8003b10 con 0x7f40c80ae1e0 2024-03-29T08:20:21.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40bfe00640 1 --2- 172.21.15.46:0/483150110 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f40a4066690 0x7f40a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40bfe00640 1 -- 172.21.15.46:0/483150110 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40b8083790 con 0x7f40c80ae1e0 2024-03-29T08:20:21.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40d1bf6000 1 Processor -- start 2024-03-29T08:20:21.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40bea00640 1 --2- 172.21.15.46:0/483150110 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f40a4066690 0x7f40a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40d1bf6000 1 -- start start 2024-03-29T08:20:21.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40d1bf6000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd395a70 0x5586fd395e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40bea00640 1 --2- 172.21.15.46:0/483150110 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f40a4066690 0x7f40a4068b50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009fd0 tx=0x7f40b000b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40d1bf6000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd396390 0x5586fd3494c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40d1bf6000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd349a00 0x5586fd349de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40d1bf6000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586fd214630 con 0x5586fd349a00 2024-03-29T08:20:21.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40d1bf6000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586fd1a1bd0 con 0x5586fd395a70 2024-03-29T08:20:21.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40d1bf6000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586fd19f1a0 con 0x5586fd396390 2024-03-29T08:20:21.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd349a00 0x5586fd349de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40c7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd395a70 0x5586fd395e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40c7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd395a70 0x5586fd395e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:40536/0 (socket says 172.21.15.46:40536) 2024-03-29T08:20:21.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40c7400640 1 -- 172.21.15.46:0/1747815454 learned_addr learned my addr 172.21.15.46:0/1747815454 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.703+0000 7f40bea00640 1 --2- 172.21.15.46:0/1747815454 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd396390 0x5586fd3494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40c7400640 1 -- 172.21.15.46:0/1747815454 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd396390 msgr2=0x5586fd3494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40c7400640 1 --2- 172.21.15.46:0/1747815454 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd396390 0x5586fd3494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40c7400640 1 -- 172.21.15.46:0/1747815454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd349a00 msgr2=0x5586fd349de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40c7400640 1 --2- 172.21.15.46:0/1747815454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd349a00 0x5586fd349de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40c7400640 1 -- 172.21.15.46:0/1747815454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586fce96e20 con 0x5586fd395a70 2024-03-29T08:20:21.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40bea00640 1 --2- 172.21.15.46:0/1747815454 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd396390 0x5586fd3494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:21.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40c7e00640 1 --2- 172.21.15.46:0/1747815454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd349a00 0x5586fd349de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:20:21.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40c7400640 1 --2- 172.21.15.46:0/1747815454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd395a70 0x5586fd395e50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f40b8050d90 tx=0x7f40b80549d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d4bcc5381799367 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40b7e00640 1 -- 172.21.15.46:0/1747815454 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8052c30 con 0x5586fd395a70 2024-03-29T08:20:21.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40b7e00640 1 -- 172.21.15.46:0/1747815454 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b8050fa0 con 0x5586fd395a70 2024-03-29T08:20:21.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40b7e00640 1 -- 172.21.15.46:0/1747815454 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8053f80 con 0x5586fd395a70 2024-03-29T08:20:21.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 1 -- 172.21.15.46:0/1747815454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd395a70 msgr2=0x5586fd395e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/1747815454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd395a70 0x5586fd395e50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f40b8050d90 tx=0x7f40b80549d0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 1 -- 172.21.15.46:0/1747815454 shutdown_connections 2024-03-29T08:20:21.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/1747815454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd349a00 0x5586fd349de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/1747815454 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd396390 0x5586fd3494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/1747815454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd395a70 0x5586fd395e50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 1 -- 172.21.15.46:0/1747815454 >> 172.21.15.46:0/1747815454 conn(0x5586fd14a160 msgr2=0x5586fd378820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:21.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 1 -- 172.21.15.46:0/1747815454 shutdown_connections 2024-03-29T08:20:21.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 1 -- 172.21.15.46:0/1747815454 wait complete. 2024-03-29T08:20:21.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 1 -- start start 2024-03-29T08:20:21.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 1 Processor -- start 2024-03-29T08:20:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd349a00 0x5586fd3527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd395a70 0x5586fd379a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd396390 0x5586fd379f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586fd19aa90 con 0x5586fd395a70 2024-03-29T08:20:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586fd139d40 con 0x5586fd349a00 2024-03-29T08:20:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586fd157f30 con 0x5586fd396390 2024-03-29T08:20:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40d1bf6000 10 client.0 _ll_register_callbacks cb 0x5586fd1fc810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:20:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40bea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd395a70 0x5586fd379a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40bea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd395a70 0x5586fd379a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47536/0 (socket says 172.21.15.46:47536) 2024-03-29T08:20:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40c7e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd396390 0x5586fd379f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40bea00640 1 -- 172.21.15.46:0/3604881518 learned_addr learned my addr 172.21.15.46:0/3604881518 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.707+0000 7f40c7400640 1 --2- 172.21.15.46:0/3604881518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd349a00 0x5586fd3527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40bea00640 1 -- 172.21.15.46:0/3604881518 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd396390 msgr2=0x5586fd379f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40bea00640 1 --2- 172.21.15.46:0/3604881518 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd396390 0x5586fd379f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40bea00640 1 -- 172.21.15.46:0/3604881518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd349a00 msgr2=0x5586fd3527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40bea00640 1 --2- 172.21.15.46:0/3604881518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd349a00 0x5586fd3527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40bea00640 1 -- 172.21.15.46:0/3604881518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c80b00f0 con 0x5586fd395a70 2024-03-29T08:20:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40bea00640 1 --2- 172.21.15.46:0/3604881518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd395a70 0x5586fd379a50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f40b0011f20 tx=0x7f40b0004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fe11779263ec8d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:20:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40b4c00640 1 -- 172.21.15.46:0/3604881518 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b000e040 con 0x5586fd395a70 2024-03-29T08:20:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40b4c00640 1 -- 172.21.15.46:0/3604881518 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40b001c040 con 0x5586fd395a70 2024-03-29T08:20:21.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 1 -- 172.21.15.46:0/3604881518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586fd1993e0 con 0x5586fd395a70 2024-03-29T08:20:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40b4c00640 1 -- 172.21.15.46:0/3604881518 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40b0022820 con 0x5586fd395a70 2024-03-29T08:20:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 10 client.5841 fetch_fsmap learned FSMap version 24 2024-03-29T08:20:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 1 -- 172.21.15.46:0/3604881518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x7f40c813ce90 con 0x5586fd395a70 2024-03-29T08:20:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40b4c00640 1 -- 172.21.15.46:0/3604881518 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5586fd1993e0 con 0x5586fd395a70 2024-03-29T08:20:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40b4c00640 1 -- 172.21.15.46:0/3604881518 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f40b0022980 con 0x5586fd395a70 2024-03-29T08:20:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 10 client.5841 fetch_fsmap finished waiting for FSMap version 24 2024-03-29T08:20:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 10 client.5841 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-03-29T08:20:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 20 client.5841 populate_metadata read hostname 'smithi046' 2024-03-29T08:20:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/3604881518 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x5586fd39d2d0 0x5586fd39d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 4 client.5841 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:20:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 1 -- 172.21.15.46:0/3604881518 --> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586fd16ad00 con 0x5586fd39d2d0 2024-03-29T08:20:21.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40c7400640 1 --2- 172.21.15.46:0/3604881518 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x5586fd39d2d0 0x5586fd39d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40c7400640 1 --2- 172.21.15.46:0/3604881518 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x5586fd39d2d0 0x5586fd39d6e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cac2997a30d8f79 server_cookie=c824429f68963e64 in_seq=0 out_seq=0 2024-03-29T08:20:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40b4c00640 10 client.5841 ms_handle_connect on v2:172.21.15.46:6834/3467089652 2024-03-29T08:20:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40b4c00640 1 -- 172.21.15.46:0/3604881518 <== mds.0 v2:172.21.15.46:6834/3467089652 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5586fd16ad00 con 0x5586fd39d2d0 2024-03-29T08:20:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40b4c00640 10 client.5841 handle_command_reply: tid=0 2024-03-29T08:20:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 1 client.5841 shutdown 2024-03-29T08:20:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 1 -- 172.21.15.46:0/3604881518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd395a70 msgr2=0x5586fd379a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/3604881518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd395a70 0x5586fd379a50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f40b0011f20 tx=0x7f40b0004a50 comp rx=0 tx=0).stop 2024-03-29T08:20:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 1 -- 172.21.15.46:0/3604881518 shutdown_connections 2024-03-29T08:20:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/3604881518 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x5586fd39d2d0 0x5586fd39d6e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/3604881518 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd396390 0x5586fd379f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/3604881518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd395a70 0x5586fd379a50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/3604881518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd349a00 0x5586fd3527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40b4c00640 0 client.5841 ms_handle_reset on v2:172.21.15.46:6834/3467089652 2024-03-29T08:20:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 1 -- 172.21.15.46:0/3604881518 >> 172.21.15.46:0/3604881518 conn(0x5586fd14a160 msgr2=0x5586fd348bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:21.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 1 -- 172.21.15.46:0/3604881518 shutdown_connections 2024-03-29T08:20:21.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 1 -- 172.21.15.46:0/3604881518 wait complete. 2024-03-29T08:20:21.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.711+0000 7f40d1bf6000 20 client.5841 trim_cache size 0 max 16384 2024-03-29T08:20:21.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40d1bf6000 1 Processor -- start 2024-03-29T08:20:21.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40d1bf6000 1 -- start start 2024-03-29T08:20:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40d1bf6000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd349a00 0x5586fd348c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40d1bf6000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd395a70 0x5586fd349170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40d1bf6000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd396390 0x5586fd349de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40d1bf6000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586fce57aa0 con 0x5586fd395a70 2024-03-29T08:20:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40d1bf6000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586fd1982a0 con 0x5586fd396390 2024-03-29T08:20:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40d1bf6000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586fd19ea90 con 0x5586fd349a00 2024-03-29T08:20:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40bea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd395a70 0x5586fd349170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40bea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd395a70 0x5586fd349170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47552/0 (socket says 172.21.15.46:47552) 2024-03-29T08:20:21.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40bea00640 1 -- 172.21.15.46:0/3423246168 learned_addr learned my addr 172.21.15.46:0/3423246168 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40c7e00640 1 --2- 172.21.15.46:0/3423246168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd396390 0x5586fd349de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40c7400640 1 --2- 172.21.15.46:0/3423246168 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd349a00 0x5586fd348c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40bea00640 1 -- 172.21.15.46:0/3423246168 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd349a00 msgr2=0x5586fd348c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40bea00640 1 --2- 172.21.15.46:0/3423246168 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd349a00 0x5586fd348c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40bea00640 1 -- 172.21.15.46:0/3423246168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd396390 msgr2=0x5586fd349de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40bea00640 1 --2- 172.21.15.46:0/3423246168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd396390 0x5586fd349de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40bea00640 1 -- 172.21.15.46:0/3423246168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586fd15faa0 con 0x5586fd395a70 2024-03-29T08:20:21.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40c7e00640 1 --2- 172.21.15.46:0/3423246168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd396390 0x5586fd349de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:21.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40bea00640 1 --2- 172.21.15.46:0/3423246168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd395a70 0x5586fd349170 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f40b000ffb0 tx=0x7f40b0022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90e66543c8a5179 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40bcc00640 1 -- 172.21.15.46:0/3423246168 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b000e070 con 0x5586fd395a70 2024-03-29T08:20:21.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40bcc00640 1 -- 172.21.15.46:0/3423246168 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b000f040 con 0x5586fd395a70 2024-03-29T08:20:21.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.727+0000 7f40bcc00640 1 -- 172.21.15.46:0/3423246168 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b0022a80 con 0x5586fd395a70 2024-03-29T08:20:21.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 -- 172.21.15.46:0/3423246168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd395a70 msgr2=0x5586fd349170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/3423246168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd395a70 0x5586fd349170 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f40b000ffb0 tx=0x7f40b0022000 comp rx=0 tx=0).stop 2024-03-29T08:20:21.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 -- 172.21.15.46:0/3423246168 shutdown_connections 2024-03-29T08:20:21.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/3423246168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd396390 0x5586fd349de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/3423246168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd395a70 0x5586fd349170 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/3423246168 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd349a00 0x5586fd348c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 -- 172.21.15.46:0/3423246168 >> 172.21.15.46:0/3423246168 conn(0x5586fd14a160 msgr2=0x5586fd350fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:21.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 -- 172.21.15.46:0/3423246168 shutdown_connections 2024-03-29T08:20:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 -- 172.21.15.46:0/3423246168 wait complete. 2024-03-29T08:20:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 -- start start 2024-03-29T08:20:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 Processor -- start 2024-03-29T08:20:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd349a00 0x5586fd3f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd395a70 0x5586fd3f6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd396390 0x5586fd34c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586fd19c020 con 0x5586fd396390 2024-03-29T08:20:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586fd379c80 con 0x5586fd349a00 2024-03-29T08:20:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586fd3a3910 con 0x5586fd395a70 2024-03-29T08:20:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 10 client.0 _ll_register_callbacks cb 0x7f40b8083040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:20:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd396390 0x5586fd34c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd396390 0x5586fd34c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47558/0 (socket says 172.21.15.46:47558) 2024-03-29T08:20:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40c7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd349a00 0x5586fd3f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40bea00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd395a70 0x5586fd3f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40c7e00640 1 -- 172.21.15.46:0/982982464 learned_addr learned my addr 172.21.15.46:0/982982464 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:21.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40c7400640 1 -- 172.21.15.46:0/982982464 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd395a70 msgr2=0x5586fd3f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40c7400640 1 --2- 172.21.15.46:0/982982464 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd395a70 0x5586fd3f6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40c7400640 1 -- 172.21.15.46:0/982982464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd396390 msgr2=0x5586fd34c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40c7400640 1 --2- 172.21.15.46:0/982982464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd396390 0x5586fd34c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40c7400640 1 -- 172.21.15.46:0/982982464 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c813ce90 con 0x5586fd349a00 2024-03-29T08:20:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40bea00640 1 --2- 172.21.15.46:0/982982464 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd395a70 0x5586fd3f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:20:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40c7e00640 1 --2- 172.21.15.46:0/982982464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd396390 0x5586fd34c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:20:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40c7400640 1 --2- 172.21.15.46:0/982982464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd349a00 0x5586fd3f5be0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f40b800ce90 tx=0x7f40b801e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e28b44fac098020 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-29T08:20:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40b4c00640 1 -- 172.21.15.46:0/982982464 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8003e40 con 0x5586fd349a00 2024-03-29T08:20:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40b4c00640 1 -- 172.21.15.46:0/982982464 <== mon.1 v2:172.21.15.73:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40b8053d80 con 0x5586fd349a00 2024-03-29T08:20:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 -- 172.21.15.46:0/982982464 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5586fd37a360 con 0x5586fd349a00 2024-03-29T08:20:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40b4c00640 1 -- 172.21.15.46:0/982982464 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40b8053f60 con 0x5586fd349a00 2024-03-29T08:20:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 10 client.5809 fetch_fsmap learned FSMap version 24 2024-03-29T08:20:21.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.731+0000 7f40d1bf6000 1 -- 172.21.15.46:0/982982464 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5586fd145df0 con 0x5586fd349a00 2024-03-29T08:20:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40b4c00640 1 -- 172.21.15.46:0/982982464 <== mon.1 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5586fd37a360 con 0x5586fd349a00 2024-03-29T08:20:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40b4c00640 1 -- 172.21.15.46:0/982982464 <== mon.1 v2:172.21.15.73:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f40b805c740 con 0x5586fd349a00 2024-03-29T08:20:21.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 10 client.5809 fetch_fsmap finished waiting for FSMap version 24 2024-03-29T08:20:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 10 client.5809 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-03-29T08:20:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 20 client.5809 populate_metadata read hostname 'smithi046' 2024-03-29T08:20:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/982982464 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x5586fd34cb40 0x5586fd34cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 4 client.5809 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:20:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 1 -- 172.21.15.46:0/982982464 --> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5586fd157970 con 0x5586fd34cb40 2024-03-29T08:20:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40bea00640 1 --2- 172.21.15.46:0/982982464 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x5586fd34cb40 0x5586fd34cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40bea00640 1 --2- 172.21.15.46:0/982982464 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x5586fd34cb40 0x5586fd34cf50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b28e5058704c412 server_cookie=2c2aef40bcd7dabf in_seq=0 out_seq=0 2024-03-29T08:20:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40b4c00640 10 client.5809 ms_handle_connect on v2:172.21.15.46:6834/3467089652 2024-03-29T08:20:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40b4c00640 1 -- 172.21.15.46:0/982982464 <== mds.0 v2:172.21.15.46:6834/3467089652 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x5586fd157970 con 0x5586fd34cb40 2024-03-29T08:20:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40b4c00640 10 client.5809 handle_command_reply: tid=0 2024-03-29T08:20:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 1 client.5809 shutdown 2024-03-29T08:20:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 1 -- 172.21.15.46:0/982982464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd349a00 msgr2=0x5586fd3f5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/982982464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd349a00 0x5586fd3f5be0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f40b800ce90 tx=0x7f40b801e040 comp rx=0 tx=0).stop 2024-03-29T08:20:21.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 1 -- 172.21.15.46:0/982982464 shutdown_connections 2024-03-29T08:20:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/982982464 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x5586fd34cb40 0x5586fd34cf50 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/982982464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586fd396390 0x5586fd34c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/982982464 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5586fd395a70 0x5586fd3f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 1 --2- 172.21.15.46:0/982982464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5586fd349a00 0x5586fd3f5be0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40b4c00640 0 client.5809 ms_handle_reset on v2:172.21.15.46:6834/3467089652 2024-03-29T08:20:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 1 -- 172.21.15.46:0/982982464 >> 172.21.15.46:0/982982464 conn(0x5586fd14a160 msgr2=0x5586fd351840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 1 -- 172.21.15.46:0/982982464 shutdown_connections 2024-03-29T08:20:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 1 -- 172.21.15.46:0/982982464 wait complete. 2024-03-29T08:20:21.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40d1bf6000 20 client.5809 trim_cache size 0 max 16384 2024-03-29T08:20:21.741 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:20:21.741 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-29T08:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-29T08:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: "466a2478-b73d-43c0-9498-bb2ddd000464": { 2024-03-29T08:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-29T08:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "466a2478-b73d-43c0-9498-bb2ddd000464", 2024-03-29T08:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,repair" 2024-03-29T08:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:21.742 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:20:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40ce800640 1 -- 172.21.15.46:0/483150110 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f40a4066690 msgr2=0x7f40a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40ce800640 1 --2- 172.21.15.46:0/483150110 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f40a4066690 0x7f40a4068b50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009fd0 tx=0x7f40b000b040 comp rx=0 tx=0).stop 2024-03-29T08:20:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40ce800640 1 -- 172.21.15.46:0/483150110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c80ae1e0 msgr2=0x7f40c8140670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40ce800640 1 --2- 172.21.15.46:0/483150110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c80ae1e0 0x7f40c8140670 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f40b800c4d0 tx=0x7f40b800c990 comp rx=0 tx=0).stop 2024-03-29T08:20:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40ce800640 1 -- 172.21.15.46:0/483150110 shutdown_connections 2024-03-29T08:20:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40ce800640 1 --2- 172.21.15.46:0/483150110 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f40a4066690 0x7f40a4068b50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40ce800640 1 --2- 172.21.15.46:0/483150110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40c8142da0 0x7f40c8145300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40ce800640 1 --2- 172.21.15.46:0/483150110 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f40c8143180 0x7f40c813e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40ce800640 1 --2- 172.21.15.46:0/483150110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40c80ae1e0 0x7f40c8140670 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40ce800640 1 -- 172.21.15.46:0/483150110 >> 172.21.15.46:0/483150110 conn(0x7f40c806a390 msgr2=0x7f40c80cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40ce800640 1 -- 172.21.15.46:0/483150110 shutdown_connections 2024-03-29T08:20:21.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:21.735+0000 7f40ce800640 1 -- 172.21.15.46:0/483150110 wait complete. 2024-03-29T08:20:21.756 INFO:tasks.cephfs.filesystem:scrub status for tag:466a2478-b73d-43c0-9498-bb2ddd000464 - {'path': '/', 'tag': '466a2478-b73d-43c0-9498-bb2ddd000464', 'options': 'recursive,repair'} 2024-03-29T08:20:22.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:22.431+0000 7f95c1600640 1 -- 172.21.15.46:0/3108593353 <== mon.0 v2:172.21.15.46:3300/0 7 ==== log(6 entries from seq 1 at 2024-03-29T08:20:21.530766+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f95bc056bd0 con 0x7f95c40cfa30 2024-03-29T08:20:22.434 INFO:teuthology.orchestra.run.smithi046.stdout: cluster: 2024-03-29T08:20:22.434 INFO:teuthology.orchestra.run.smithi046.stdout: id: e9b7960f-ec1f-4787-95c3-c806094d37f0 2024-03-29T08:20:22.434 INFO:teuthology.orchestra.run.smithi046.stdout: health: HEALTH_OK 2024-03-29T08:20:22.434 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:22.435 INFO:teuthology.orchestra.run.smithi046.stdout: services: 2024-03-29T08:20:22.435 INFO:teuthology.orchestra.run.smithi046.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-03-29T08:20:22.435 INFO:teuthology.orchestra.run.smithi046.stdout: mgr: x(active, since 2m), standbys: y 2024-03-29T08:20:22.435 INFO:teuthology.orchestra.run.smithi046.stdout: mds: 1/1 daemons up, 3 standby 2024-03-29T08:20:22.435 INFO:teuthology.orchestra.run.smithi046.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-03-29T08:20:22.435 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:22.435 INFO:teuthology.orchestra.run.smithi046.stdout: data: 2024-03-29T08:20:22.435 INFO:teuthology.orchestra.run.smithi046.stdout: volumes: 1/1 healthy 2024-03-29T08:20:22.435 INFO:teuthology.orchestra.run.smithi046.stdout: pools: 3 pools, 97 pgs 2024-03-29T08:20:22.435 INFO:teuthology.orchestra.run.smithi046.stdout: objects: 24 objects, 579 KiB 2024-03-29T08:20:22.435 INFO:teuthology.orchestra.run.smithi046.stdout: usage: 237 MiB used, 720 GiB / 720 GiB avail 2024-03-29T08:20:22.435 INFO:teuthology.orchestra.run.smithi046.stdout: pgs: 97 active+clean 2024-03-29T08:20:22.435 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:22.435 INFO:teuthology.orchestra.run.smithi046.stdout: io: 2024-03-29T08:20:22.435 INFO:teuthology.orchestra.run.smithi046.stdout: client: 1.7 KiB/s wr, 0 op/s rd, 5 op/s wr 2024-03-29T08:20:22.435 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:22.436 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:22.436 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:20:21.530766+0000 mds.a [INF] scrub queued for path: / 2024-03-29T08:20:22.436 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:20:21.530770+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-03-29T08:20:22.436 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:20:21.530774+0000 mds.a [INF] scrub summary: active paths [/] 2024-03-29T08:20:22.436 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:20:21.531823+0000 mds.a [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-03-29T08:20:22.436 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:20:21.531830+0000 mds.a [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-03-29T08:20:22.436 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:20:21.532582+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-03-29T08:20:24.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:24.919+0000 7f95c1600640 1 -- 172.21.15.46:0/3108593353 <== mon.0 v2:172.21.15.46:3300/0 8 ==== log(1 entries from seq 7 at 2024-03-29T08:20:23.070868+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f95bc0564d0 con 0x7f95c40cfa30 2024-03-29T08:20:24.920 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:20:23.070868+0000 mds.a [INF] scrub summary: idle 2024-03-29T08:20:38.511 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:20:38.515 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:20:51.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:51.757 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-03-29T08:20:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.907+0000 7fa570800640 1 Processor -- start 2024-03-29T08:20:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.907+0000 7fa570800640 1 -- start start 2024-03-29T08:20:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.911+0000 7fa570800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa56815efe0 0x7fa5680b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.911+0000 7fa570800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5680b2360 con 0x7fa5680cbea0 2024-03-29T08:20:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.911+0000 7fa570800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56815e500 con 0x7fa568143eb0 2024-03-29T08:20:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.911+0000 7fa570800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56815e7e0 con 0x7fa56815efe0 2024-03-29T08:20:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.911+0000 7fa56e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa56815efe0 0x7fa5680b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.911+0000 7fa56d400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa5680cbea0 0x7fa5680b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48318/0 (socket says 172.21.15.46:48318) 2024-03-29T08:20:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.911+0000 7fa56e800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa56815efe0 0x7fa5680b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38450/0 (socket says 172.21.15.46:38450) 2024-03-29T08:20:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.911+0000 7fa56d400640 1 -- 172.21.15.46:0/883123599 learned_addr learned my addr 172.21.15.46:0/883123599 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.911+0000 7fa56e800640 1 -- 172.21.15.46:0/883123599 >> v1:172.21.15.73:6789/0 conn(0x7fa568143eb0 legacy=0x7fa5680cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:51.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.911+0000 7fa56e800640 1 -- 172.21.15.46:0/883123599 >> v1:172.21.15.46:6789/0 conn(0x7fa5680cbea0 legacy=0x7fa5680b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.911+0000 7fa56ca00640 1 -- 172.21.15.46:0/883123599 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87442502 0 0) 0x7fa5680b2360 con 0x7fa5680cbea0 2024-03-29T08:20:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.911+0000 7fa56e800640 1 -- 172.21.15.46:0/883123599 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56815eac0 con 0x7fa56815efe0 2024-03-29T08:20:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.915+0000 7fa56e800640 1 --2- 172.21.15.46:0/883123599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56815efe0 0x7fa5680b1d40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa564009d90 tx=0x7fa56401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59a14f4b745e33b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.915+0000 7fa56ca00640 1 -- 172.21.15.46:0/883123599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5640a5a50 con 0x7fa56815efe0 2024-03-29T08:20:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.915+0000 7fa56ca00640 1 -- 172.21.15.46:0/883123599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5640a5bb0 con 0x7fa56815efe0 2024-03-29T08:20:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.915+0000 7fa56ca00640 1 -- 172.21.15.46:0/883123599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5640ad440 con 0x7fa56815efe0 2024-03-29T08:20:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.915+0000 7fa570800640 1 -- 172.21.15.46:0/883123599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56815efe0 msgr2=0x7fa5680b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.915+0000 7fa570800640 1 --2- 172.21.15.46:0/883123599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56815efe0 0x7fa5680b1d40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa564009d90 tx=0x7fa56401c970 comp rx=0 tx=0).stop 2024-03-29T08:20:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.915+0000 7fa570800640 1 -- 172.21.15.46:0/883123599 shutdown_connections 2024-03-29T08:20:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.915+0000 7fa570800640 1 --2- 172.21.15.46:0/883123599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56815efe0 0x7fa5680b1d40 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.915+0000 7fa570800640 1 -- 172.21.15.46:0/883123599 >> 172.21.15.46:0/883123599 conn(0x7fa56806a390 msgr2=0x7fa5680cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:51.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.915+0000 7fa570800640 1 -- 172.21.15.46:0/883123599 shutdown_connections 2024-03-29T08:20:51.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.915+0000 7fa570800640 1 -- 172.21.15.46:0/883123599 wait complete. 2024-03-29T08:20:51.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.915+0000 7fa570800640 1 Processor -- start 2024-03-29T08:20:51.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.915+0000 7fa570800640 1 -- start start 2024-03-29T08:20:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa570800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5680cbea0 0x7fa568158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa570800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa568143eb0 0x7fa568158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa570800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa56815efe0 0x7fa568138560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa570800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56813f3d0 con 0x7fa568143eb0 2024-03-29T08:20:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa570800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa568139980 con 0x7fa56815efe0 2024-03-29T08:20:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa570800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56813f0f0 con 0x7fa5680cbea0 2024-03-29T08:20:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa56d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa568143eb0 0x7fa568158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa56de00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5680cbea0 0x7fa568158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa56d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa568143eb0 0x7fa568158ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38454/0 (socket says 172.21.15.46:38454) 2024-03-29T08:20:51.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa56d400640 1 -- 172.21.15.46:0/2253186401 learned_addr learned my addr 172.21.15.46:0/2253186401 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa56e800640 1 --2- 172.21.15.46:0/2253186401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa56815efe0 0x7fa568138560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa56d400640 1 -- 172.21.15.46:0/2253186401 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5680cbea0 msgr2=0x7fa568158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa56d400640 1 --2- 172.21.15.46:0/2253186401 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5680cbea0 0x7fa568158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa56d400640 1 -- 172.21.15.46:0/2253186401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa56815efe0 msgr2=0x7fa568138560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa56d400640 1 --2- 172.21.15.46:0/2253186401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa56815efe0 0x7fa568138560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa56d400640 1 -- 172.21.15.46:0/2253186401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa564009a70 con 0x7fa568143eb0 2024-03-29T08:20:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa56de00640 1 --2- 172.21.15.46:0/2253186401 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5680cbea0 0x7fa568158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:20:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa56d400640 1 --2- 172.21.15.46:0/2253186401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa568143eb0 0x7fa568158ed0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa55800c4a0 tx=0x7fa55800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa562000640 1 -- 172.21.15.46:0/2253186401 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5580074e0 con 0x7fa568143eb0 2024-03-29T08:20:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa562000640 1 -- 172.21.15.46:0/2253186401 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa558007640 con 0x7fa568143eb0 2024-03-29T08:20:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa562000640 1 -- 172.21.15.46:0/2253186401 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa558004a30 con 0x7fa568143eb0 2024-03-29T08:20:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa570800640 1 -- 172.21.15.46:0/2253186401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa568139c20 con 0x7fa568143eb0 2024-03-29T08:20:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa570800640 1 -- 172.21.15.46:0/2253186401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa568156430 con 0x7fa568143eb0 2024-03-29T08:20:51.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.919+0000 7fa562000640 1 -- 172.21.15.46:0/2253186401 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa558004b90 con 0x7fa568143eb0 2024-03-29T08:20:51.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa562000640 1 --2- 172.21.15.46:0/2253186401 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa54c066690 0x7fa54c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa562000640 1 -- 172.21.15.46:0/2253186401 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa558083360 con 0x7fa568143eb0 2024-03-29T08:20:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa573bd1000 1 Processor -- start 2024-03-29T08:20:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa56de00640 1 --2- 172.21.15.46:0/2253186401 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa54c066690 0x7fa54c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa573bd1000 1 -- start start 2024-03-29T08:20:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa56de00640 1 --2- 172.21.15.46:0/2253186401 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa54c066690 0x7fa54c068b50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa5681438d0 tx=0x7fa55c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa573bd1000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 0x55e9e9589240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa573bd1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e9589780 0x55e9e953c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa573bd1000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e953ce10 0x55e9e953d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa573bd1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e9403630 con 0x55e9e9589780 2024-03-29T08:20:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa573bd1000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e9390bd0 con 0x55e9e953ce10 2024-03-29T08:20:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa573bd1000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9e938e1a0 con 0x55e9e9588e60 2024-03-29T08:20:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa56d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e9589780 0x55e9e953c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa56d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e9589780 0x55e9e953c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38460/0 (socket says 172.21.15.46:38460) 2024-03-29T08:20:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa56d400640 1 -- 172.21.15.46:0/2066654306 learned_addr learned my addr 172.21.15.46:0/2066654306 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa56de00640 1 --2- 172.21.15.46:0/2066654306 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 0x55e9e9589240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa56e800640 1 --2- 172.21.15.46:0/2066654306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e953ce10 0x55e9e953d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa56d400640 1 -- 172.21.15.46:0/2066654306 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 msgr2=0x55e9e9589240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa56d400640 1 --2- 172.21.15.46:0/2066654306 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 0x55e9e9589240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa56d400640 1 -- 172.21.15.46:0/2066654306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e953ce10 msgr2=0x55e9e953d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa56d400640 1 --2- 172.21.15.46:0/2066654306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e953ce10 0x55e9e953d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa56d400640 1 -- 172.21.15.46:0/2066654306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9e9085e20 con 0x55e9e9589780 2024-03-29T08:20:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa56d400640 1 --2- 172.21.15.46:0/2066654306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e9589780 0x55e9e953c8d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa55805c240 tx=0x7fa5580540a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e4c2ba71c60c64 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa556a00640 1 -- 172.21.15.46:0/2066654306 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5580581b0 con 0x55e9e9589780 2024-03-29T08:20:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa556a00640 1 -- 172.21.15.46:0/2066654306 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55805c770 con 0x55e9e9589780 2024-03-29T08:20:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa556a00640 1 -- 172.21.15.46:0/2066654306 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5580830c0 con 0x55e9e9589780 2024-03-29T08:20:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa573bd1000 1 -- 172.21.15.46:0/2066654306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e9589780 msgr2=0x55e9e953c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa573bd1000 1 --2- 172.21.15.46:0/2066654306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e9589780 0x55e9e953c8d0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fa55805c240 tx=0x7fa5580540a0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa573bd1000 1 -- 172.21.15.46:0/2066654306 shutdown_connections 2024-03-29T08:20:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa573bd1000 1 --2- 172.21.15.46:0/2066654306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e953ce10 0x55e9e953d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa573bd1000 1 --2- 172.21.15.46:0/2066654306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e9589780 0x55e9e953c8d0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa573bd1000 1 --2- 172.21.15.46:0/2066654306 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 0x55e9e9589240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.923+0000 7fa573bd1000 1 -- 172.21.15.46:0/2066654306 >> 172.21.15.46:0/2066654306 conn(0x55e9e9339160 msgr2=0x55e9e9567820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:51.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 1 -- 172.21.15.46:0/2066654306 shutdown_connections 2024-03-29T08:20:51.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 1 -- 172.21.15.46:0/2066654306 wait complete. 2024-03-29T08:20:51.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 1 -- start start 2024-03-29T08:20:51.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 1 Processor -- start 2024-03-29T08:20:51.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e953ce10 0x55e9e95901c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 0x55e9e9590700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e9589780 0x55e9e956bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e9389a90 con 0x55e9e9589780 2024-03-29T08:20:51.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e9328d40 con 0x55e9e953ce10 2024-03-29T08:20:51.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9e9346f30 con 0x55e9e9588e60 2024-03-29T08:20:51.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 10 client.0 _ll_register_callbacks cb 0x55e9e93eb810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:20:51.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa56e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e9589780 0x55e9e956bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa56e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e9589780 0x55e9e956bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38474/0 (socket says 172.21.15.46:38474) 2024-03-29T08:20:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa56e800640 1 -- 172.21.15.46:0/418393324 learned_addr learned my addr 172.21.15.46:0/418393324 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa56de00640 1 --2- 172.21.15.46:0/418393324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e953ce10 0x55e9e95901c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa56d400640 1 --2- 172.21.15.46:0/418393324 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 0x55e9e9590700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa56e800640 1 -- 172.21.15.46:0/418393324 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 msgr2=0x55e9e9590700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa56e800640 1 --2- 172.21.15.46:0/418393324 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 0x55e9e9590700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa56e800640 1 -- 172.21.15.46:0/418393324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e953ce10 msgr2=0x55e9e95901c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa56e800640 1 --2- 172.21.15.46:0/418393324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e953ce10 0x55e9e95901c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa56e800640 1 -- 172.21.15.46:0/418393324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa568156430 con 0x55e9e9589780 2024-03-29T08:20:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa56de00640 1 --2- 172.21.15.46:0/418393324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e953ce10 0x55e9e95901c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa56e800640 1 --2- 172.21.15.46:0/418393324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e9589780 0x55e9e956bc60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa564009d90 tx=0x7fa56401b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8472d8838e0033ea server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa54b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:20:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa54b400640 1 -- 172.21.15.46:0/418393324 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5640a59f0 con 0x55e9e9589780 2024-03-29T08:20:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa54b400640 1 -- 172.21.15.46:0/418393324 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5640adc90 con 0x55e9e9589780 2024-03-29T08:20:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 1 -- 172.21.15.46:0/418393324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9e93883e0 con 0x55e9e9589780 2024-03-29T08:20:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa54b400640 1 -- 172.21.15.46:0/418393324 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5640ade50 con 0x55e9e9589780 2024-03-29T08:20:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 10 client.5862 fetch_fsmap learned FSMap version 24 2024-03-29T08:20:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 1 -- 172.21.15.46:0/418393324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55e9e934eaa0 con 0x55e9e9589780 2024-03-29T08:20:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa54b400640 1 -- 172.21.15.46:0/418393324 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e9e93883e0 con 0x55e9e9589780 2024-03-29T08:20:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa54b400640 1 -- 172.21.15.46:0/418393324 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa5640a5d20 con 0x55e9e9589780 2024-03-29T08:20:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 10 client.5862 fetch_fsmap finished waiting for FSMap version 24 2024-03-29T08:20:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 10 client.5862 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-03-29T08:20:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 20 client.5862 populate_metadata read hostname 'smithi046' 2024-03-29T08:20:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 1 --2- 172.21.15.46:0/418393324 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55e9e956ca00 0x55e9e956cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 4 client.5862 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:20:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa573bd1000 1 -- 172.21.15.46:0/418393324 --> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9e9359d00 con 0x55e9e956ca00 2024-03-29T08:20:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.927+0000 7fa56de00640 1 --2- 172.21.15.46:0/418393324 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55e9e956ca00 0x55e9e956cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:51.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.931+0000 7fa56de00640 1 --2- 172.21.15.46:0/418393324 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55e9e956ca00 0x55e9e956cde0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aace37d6e64c0db server_cookie=c256e4f7244bec3b in_seq=0 out_seq=0 2024-03-29T08:20:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.931+0000 7fa54b400640 10 client.5862 ms_handle_connect on v2:172.21.15.46:6834/3467089652 2024-03-29T08:20:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.931+0000 7fa54b400640 1 -- 172.21.15.46:0/418393324 <== mds.0 v2:172.21.15.46:6834/3467089652 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e9e9359d00 con 0x55e9e956ca00 2024-03-29T08:20:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.931+0000 7fa54b400640 10 client.5862 handle_command_reply: tid=0 2024-03-29T08:20:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.931+0000 7fa573bd1000 1 client.5862 shutdown 2024-03-29T08:20:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.931+0000 7fa573bd1000 1 -- 172.21.15.46:0/418393324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e9589780 msgr2=0x55e9e956bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.931+0000 7fa573bd1000 1 --2- 172.21.15.46:0/418393324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e9589780 0x55e9e956bc60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa564009d90 tx=0x7fa56401b1b0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.931+0000 7fa573bd1000 1 -- 172.21.15.46:0/418393324 shutdown_connections 2024-03-29T08:20:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.931+0000 7fa573bd1000 1 --2- 172.21.15.46:0/418393324 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55e9e956ca00 0x55e9e956cde0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.931+0000 7fa573bd1000 1 --2- 172.21.15.46:0/418393324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e9589780 0x55e9e956bc60 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.931+0000 7fa573bd1000 1 --2- 172.21.15.46:0/418393324 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 0x55e9e9590700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.931+0000 7fa573bd1000 1 --2- 172.21.15.46:0/418393324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e953ce10 0x55e9e95901c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.931+0000 7fa54b400640 0 client.5862 ms_handle_reset on v2:172.21.15.46:6834/3467089652 2024-03-29T08:20:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.931+0000 7fa573bd1000 1 -- 172.21.15.46:0/418393324 >> 172.21.15.46:0/418393324 conn(0x55e9e9339160 msgr2=0x55e9e9537bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.931+0000 7fa573bd1000 1 -- 172.21.15.46:0/418393324 shutdown_connections 2024-03-29T08:20:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.931+0000 7fa573bd1000 1 -- 172.21.15.46:0/418393324 wait complete. 2024-03-29T08:20:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.931+0000 7fa573bd1000 20 client.5862 trim_cache size 0 max 16384 2024-03-29T08:20:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 Processor -- start 2024-03-29T08:20:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 -- start start 2024-03-29T08:20:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e953ce10 0x55e9e95e2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 0x55e9e95e3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e9589780 0x55e9e95e3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e9046aa0 con 0x55e9e953ce10 2024-03-29T08:20:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e93590b0 con 0x55e9e9589780 2024-03-29T08:20:51.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9e959f580 con 0x55e9e9588e60 2024-03-29T08:20:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa56de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e953ce10 0x55e9e95e2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa56de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e953ce10 0x55e9e95e2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38484/0 (socket says 172.21.15.46:38484) 2024-03-29T08:20:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa56de00640 1 -- 172.21.15.46:0/158081404 learned_addr learned my addr 172.21.15.46:0/158081404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa56de00640 1 -- 172.21.15.46:0/158081404 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 msgr2=0x55e9e95e3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa56de00640 1 --2- 172.21.15.46:0/158081404 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 0x55e9e95e3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa56de00640 1 -- 172.21.15.46:0/158081404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e9589780 msgr2=0x55e9e95e3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa56de00640 1 --2- 172.21.15.46:0/158081404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e9589780 0x55e9e95e3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa56de00640 1 -- 172.21.15.46:0/158081404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9e9334df0 con 0x55e9e953ce10 2024-03-29T08:20:51.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa56de00640 1 --2- 172.21.15.46:0/158081404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e953ce10 0x55e9e95e2bc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0019f0 tx=0x7fa55c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a91f86bc08467d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa557400640 1 -- 172.21.15.46:0/158081404 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c006a40 con 0x55e9e953ce10 2024-03-29T08:20:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa557400640 1 -- 172.21.15.46:0/158081404 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c006ba0 con 0x55e9e953ce10 2024-03-29T08:20:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa557400640 1 -- 172.21.15.46:0/158081404 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0223f0 con 0x55e9e953ce10 2024-03-29T08:20:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 -- 172.21.15.46:0/158081404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e953ce10 msgr2=0x55e9e95e2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 --2- 172.21.15.46:0/158081404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e953ce10 0x55e9e95e2bc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0019f0 tx=0x7fa55c012000 comp rx=0 tx=0).stop 2024-03-29T08:20:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 -- 172.21.15.46:0/158081404 shutdown_connections 2024-03-29T08:20:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 --2- 172.21.15.46:0/158081404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e9589780 0x55e9e95e3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 --2- 172.21.15.46:0/158081404 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 0x55e9e95e3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 --2- 172.21.15.46:0/158081404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e953ce10 0x55e9e95e2bc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 -- 172.21.15.46:0/158081404 >> 172.21.15.46:0/158081404 conn(0x55e9e9339160 msgr2=0x55e9e956a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 -- 172.21.15.46:0/158081404 shutdown_connections 2024-03-29T08:20:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 -- 172.21.15.46:0/158081404 wait complete. 2024-03-29T08:20:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 -- start start 2024-03-29T08:20:51.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.947+0000 7fa573bd1000 1 Processor -- start 2024-03-29T08:20:51.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e953ce10 0x55e9e953f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 0x55e9e953f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e9589780 0x55e9e953fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e9596c00 con 0x55e9e953ce10 2024-03-29T08:20:51.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9e93e1920 con 0x55e9e9589780 2024-03-29T08:20:51.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9e938da90 con 0x55e9e9588e60 2024-03-29T08:20:51.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 10 client.0 _ll_register_callbacks cb 0x7fa55c007ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:20:51.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa56de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e953ce10 0x55e9e953f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:51.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa56de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e953ce10 0x55e9e953f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38486/0 (socket says 172.21.15.46:38486) 2024-03-29T08:20:51.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa56de00640 1 -- 172.21.15.46:0/2962655616 learned_addr learned my addr 172.21.15.46:0/2962655616 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:51.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa56d400640 1 --2- 172.21.15.46:0/2962655616 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 0x55e9e953f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:51.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa56e800640 1 --2- 172.21.15.46:0/2962655616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e9589780 0x55e9e953fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:51.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa56de00640 1 -- 172.21.15.46:0/2962655616 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 msgr2=0x55e9e953f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:51.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa56de00640 1 --2- 172.21.15.46:0/2962655616 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 0x55e9e953f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa56de00640 1 -- 172.21.15.46:0/2962655616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e9589780 msgr2=0x55e9e953fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:51.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa56de00640 1 --2- 172.21.15.46:0/2962655616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e9589780 0x55e9e953fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa56de00640 1 -- 172.21.15.46:0/2962655616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56815e500 con 0x55e9e953ce10 2024-03-29T08:20:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa56de00640 1 --2- 172.21.15.46:0/2962655616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e953ce10 0x55e9e953f150 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x55e9e938d480 tx=0x7fa55c001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317d12b6cebbc3f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa54b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:20:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa54b400640 1 -- 172.21.15.46:0/2962655616 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c030640 con 0x55e9e953ce10 2024-03-29T08:20:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa54b400640 1 -- 172.21.15.46:0/2962655616 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa55c01b040 con 0x55e9e953ce10 2024-03-29T08:20:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 -- 172.21.15.46:0/2962655616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e9e9346970 con 0x55e9e953ce10 2024-03-29T08:20:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa54b400640 1 -- 172.21.15.46:0/2962655616 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa55c0307a0 con 0x55e9e953ce10 2024-03-29T08:20:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 10 client.5868 fetch_fsmap learned FSMap version 24 2024-03-29T08:20:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 -- 172.21.15.46:0/2962655616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55e9e9328960 con 0x55e9e953ce10 2024-03-29T08:20:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa54b400640 1 -- 172.21.15.46:0/2962655616 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e9e9346970 con 0x55e9e953ce10 2024-03-29T08:20:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa54b400640 1 -- 172.21.15.46:0/2962655616 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa55c012070 con 0x55e9e953ce10 2024-03-29T08:20:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 10 client.5868 fetch_fsmap finished waiting for FSMap version 24 2024-03-29T08:20:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 10 client.5868 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-03-29T08:20:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 20 client.5868 populate_metadata read hostname 'smithi046' 2024-03-29T08:20:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 --2- 172.21.15.46:0/2962655616 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55e9e9588710 0x55e9e9540110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:51.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 4 client.5868 mds_command: new command op to 5631 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:20:51.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 -- 172.21.15.46:0/2962655616 --> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e9e93b8a20 con 0x55e9e9588710 2024-03-29T08:20:51.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa56d400640 1 --2- 172.21.15.46:0/2962655616 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55e9e9588710 0x55e9e9540110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:51.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa56d400640 1 --2- 172.21.15.46:0/2962655616 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55e9e9588710 0x55e9e9540110 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8ff350371e8d80d server_cookie=e07d73fc96a174e5 in_seq=0 out_seq=0 2024-03-29T08:20:51.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa54b400640 10 client.5868 ms_handle_connect on v2:172.21.15.46:6834/3467089652 2024-03-29T08:20:51.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa54b400640 1 -- 172.21.15.46:0/2962655616 <== mds.0 v2:172.21.15.46:6834/3467089652 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e9e93b8a20 con 0x55e9e9588710 2024-03-29T08:20:51.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa54b400640 10 client.5868 handle_command_reply: tid=0 2024-03-29T08:20:51.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 client.5868 shutdown 2024-03-29T08:20:51.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 -- 172.21.15.46:0/2962655616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e953ce10 msgr2=0x55e9e953f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:51.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 --2- 172.21.15.46:0/2962655616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e953ce10 0x55e9e953f150 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x55e9e938d480 tx=0x7fa55c001130 comp rx=0 tx=0).stop 2024-03-29T08:20:51.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 -- 172.21.15.46:0/2962655616 shutdown_connections 2024-03-29T08:20:51.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 --2- 172.21.15.46:0/2962655616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e9e9589780 0x55e9e953fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 --2- 172.21.15.46:0/2962655616 >> [v2:172.21.15.46:6834/3467089652,v1:172.21.15.46:6835/3467089652] conn(0x55e9e9588710 0x55e9e9540110 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 --2- 172.21.15.46:0/2962655616 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e9e9588e60 0x55e9e953f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 --2- 172.21.15.46:0/2962655616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e9e953ce10 0x55e9e953f150 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa54b400640 0 client.5868 ms_handle_reset on v2:172.21.15.46:6834/3467089652 2024-03-29T08:20:51.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 -- 172.21.15.46:0/2962655616 >> 172.21.15.46:0/2962655616 conn(0x55e9e9339160 msgr2=0x55e9e9537bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:51.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 -- 172.21.15.46:0/2962655616 shutdown_connections 2024-03-29T08:20:51.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.951+0000 7fa573bd1000 1 -- 172.21.15.46:0/2962655616 wait complete. 2024-03-29T08:20:51.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.955+0000 7fa573bd1000 20 client.5868 trim_cache size 0 max 16384 2024-03-29T08:20:51.959 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:20:51.959 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-29T08:20:51.959 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-29T08:20:51.959 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:20:51.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.955+0000 7fa570800640 1 -- 172.21.15.46:0/2253186401 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa54c066690 msgr2=0x7fa54c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:51.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.955+0000 7fa570800640 1 --2- 172.21.15.46:0/2253186401 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa54c066690 0x7fa54c068b50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa5681438d0 tx=0x7fa55c009040 comp rx=0 tx=0).stop 2024-03-29T08:20:51.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.955+0000 7fa570800640 1 -- 172.21.15.46:0/2253186401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa568143eb0 msgr2=0x7fa568158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:51.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.955+0000 7fa570800640 1 --2- 172.21.15.46:0/2253186401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa568143eb0 0x7fa568158ed0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa55800c4a0 tx=0x7fa55800c960 comp rx=0 tx=0).stop 2024-03-29T08:20:51.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.955+0000 7fa570800640 1 -- 172.21.15.46:0/2253186401 shutdown_connections 2024-03-29T08:20:51.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.955+0000 7fa570800640 1 --2- 172.21.15.46:0/2253186401 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa54c066690 0x7fa54c068b50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.955+0000 7fa570800640 1 --2- 172.21.15.46:0/2253186401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa56815efe0 0x7fa568138560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.955+0000 7fa570800640 1 --2- 172.21.15.46:0/2253186401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa568143eb0 0x7fa568158ed0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.955+0000 7fa570800640 1 --2- 172.21.15.46:0/2253186401 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5680cbea0 0x7fa568158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:51.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.955+0000 7fa570800640 1 -- 172.21.15.46:0/2253186401 >> 172.21.15.46:0/2253186401 conn(0x7fa56806a390 msgr2=0x7fa5680cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:51.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.955+0000 7fa570800640 1 -- 172.21.15.46:0/2253186401 shutdown_connections 2024-03-29T08:20:51.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:51.955+0000 7fa570800640 1 -- 172.21.15.46:0/2253186401 wait complete. 2024-03-29T08:20:51.977 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-29T08:20:52.078 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:20:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f0527600640 1 Processor -- start 2024-03-29T08:20:52.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f0527600640 1 -- start start 2024-03-29T08:20:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f0527600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f052015a940 0x7f05200afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f0527600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f052015ad20 0x7f05200cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f0527600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05200b0540 0x7f05200cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f0527600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05200b00d0 con 0x7f052015a940 2024-03-29T08:20:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f0527600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05200b0ab0 con 0x7f052015ad20 2024-03-29T08:20:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f0527600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05200b0d90 con 0x7f05200b0540 2024-03-29T08:20:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f0524c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f052015a940 0x7f05200afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f0524c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f052015a940 0x7f05200afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38500/0 (socket says 172.21.15.46:38500) 2024-03-29T08:20:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f0524c00640 1 -- 172.21.15.46:0/541305882 learned_addr learned my addr 172.21.15.46:0/541305882 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f0525600640 1 --2- 172.21.15.46:0/541305882 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05200b0540 0x7f05200cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f051fe00640 1 --2- 172.21.15.46:0/541305882 >> v2:172.21.15.73:3300/0 conn(0x7f052015ad20 0x7f05200cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f0524c00640 1 -- 172.21.15.46:0/541305882 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05200b0540 msgr2=0x7f05200cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f0524c00640 1 --2- 172.21.15.46:0/541305882 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05200b0540 0x7f05200cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f0524c00640 1 -- 172.21.15.46:0/541305882 >> v2:172.21.15.73:3300/0 conn(0x7f052015ad20 msgr2=0x7f05200cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:52.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f0524c00640 1 --2- 172.21.15.46:0/541305882 >> v2:172.21.15.73:3300/0 conn(0x7f052015ad20 0x7f05200cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f0524c00640 1 -- 172.21.15.46:0/541305882 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05200b1070 con 0x7f052015a940 2024-03-29T08:20:52.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.219+0000 7f051fe00640 1 --2- 172.21.15.46:0/541305882 >> v2:172.21.15.73:3300/0 conn(0x7f052015ad20 0x7f05200cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:52.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.223+0000 7f0524c00640 1 --2- 172.21.15.46:0/541305882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f052015a940 0x7f05200afb90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f05200c41d0 tx=0x7f051401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d8fb52d756d97 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.223+0000 7f051f400640 1 -- 172.21.15.46:0/541305882 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05140ab5f0 con 0x7f052015a940 2024-03-29T08:20:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.223+0000 7f051f400640 1 -- 172.21.15.46:0/541305882 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05140ab750 con 0x7f052015a940 2024-03-29T08:20:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.223+0000 7f051f400640 1 -- 172.21.15.46:0/541305882 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05140abae0 con 0x7f052015a940 2024-03-29T08:20:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.223+0000 7f0527600640 1 -- 172.21.15.46:0/541305882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f052015a940 msgr2=0x7f05200afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.223+0000 7f0527600640 1 --2- 172.21.15.46:0/541305882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f052015a940 0x7f05200afb90 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f05200c41d0 tx=0x7f051401c4c0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.223+0000 7f0527600640 1 -- 172.21.15.46:0/541305882 shutdown_connections 2024-03-29T08:20:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.223+0000 7f0527600640 1 --2- 172.21.15.46:0/541305882 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05200b0540 0x7f05200cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.223+0000 7f0527600640 1 --2- 172.21.15.46:0/541305882 >> v2:172.21.15.73:3300/0 conn(0x7f052015ad20 0x7f05200cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.223+0000 7f0527600640 1 --2- 172.21.15.46:0/541305882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f052015a940 0x7f05200afb90 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.223+0000 7f0527600640 1 -- 172.21.15.46:0/541305882 >> 172.21.15.46:0/541305882 conn(0x7f0520089fd0 msgr2=0x7f052008a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.223+0000 7f0527600640 1 -- 172.21.15.46:0/541305882 shutdown_connections 2024-03-29T08:20:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.223+0000 7f0527600640 1 -- 172.21.15.46:0/541305882 wait complete. 2024-03-29T08:20:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0527600640 1 Processor -- start 2024-03-29T08:20:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0527600640 1 -- start start 2024-03-29T08:20:52.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0527600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05200b0540 0x7f05201f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:52.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0527600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052015a940 0x7f05201540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:52.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0527600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f052015ad20 0x7f0520157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:52.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0527600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05201f9520 con 0x7f05200b0540 2024-03-29T08:20:52.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0527600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05201545e0 con 0x7f052015a940 2024-03-29T08:20:52.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0527600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05201f89a0 con 0x7f052015ad20 2024-03-29T08:20:52.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0524c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05200b0540 0x7f05201f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:52.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0524c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05200b0540 0x7f05201f4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38510/0 (socket says 172.21.15.46:38510) 2024-03-29T08:20:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0524c00640 1 -- 172.21.15.46:0/3428779828 learned_addr learned my addr 172.21.15.46:0/3428779828 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f051fe00640 1 --2- 172.21.15.46:0/3428779828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052015a940 0x7f05201540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0525600640 1 --2- 172.21.15.46:0/3428779828 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f052015ad20 0x7f0520157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0524c00640 1 -- 172.21.15.46:0/3428779828 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f052015ad20 msgr2=0x7f0520157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0524c00640 1 --2- 172.21.15.46:0/3428779828 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f052015ad20 0x7f0520157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0524c00640 1 -- 172.21.15.46:0/3428779828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052015a940 msgr2=0x7f05201540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0524c00640 1 --2- 172.21.15.46:0/3428779828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052015a940 0x7f05201540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0524c00640 1 -- 172.21.15.46:0/3428779828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05140094d0 con 0x7f05200b0540 2024-03-29T08:20:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0525600640 1 --2- 172.21.15.46:0/3428779828 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f052015ad20 0x7f0520157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0524c00640 1 --2- 172.21.15.46:0/3428779828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05200b0540 0x7f05201f4ac0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f051401ab70 tx=0x7f051401ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f051cc00640 1 -- 172.21.15.46:0/3428779828 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05140be5f0 con 0x7f05200b0540 2024-03-29T08:20:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f051cc00640 1 -- 172.21.15.46:0/3428779828 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05140be750 con 0x7f05200b0540 2024-03-29T08:20:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f051cc00640 1 -- 172.21.15.46:0/3428779828 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05140beae0 con 0x7f05200b0540 2024-03-29T08:20:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0527600640 1 -- 172.21.15.46:0/3428779828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05201f8c80 con 0x7f05200b0540 2024-03-29T08:20:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.227+0000 7f0527600640 1 -- 172.21.15.46:0/3428779828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05201f9240 con 0x7f05200b0540 2024-03-29T08:20:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.231+0000 7f0527600640 1 -- 172.21.15.46:0/3428779828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0520066c90 con 0x7f05200b0540 2024-03-29T08:20:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.231+0000 7f051cc00640 1 -- 172.21.15.46:0/3428779828 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f05140b43f0 con 0x7f05200b0540 2024-03-29T08:20:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.231+0000 7f051cc00640 1 --2- 172.21.15.46:0/3428779828 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0500066690 0x7f0500068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:52.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.231+0000 7f051cc00640 1 -- 172.21.15.46:0/3428779828 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05140a8050 con 0x7f05200b0540 2024-03-29T08:20:52.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.231+0000 7f051fe00640 1 --2- 172.21.15.46:0/3428779828 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0500066690 0x7f0500068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:52.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.231+0000 7f051fe00640 1 --2- 172.21.15.46:0/3428779828 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0500066690 0x7f0500068b50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f052009e650 tx=0x7f050c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:52.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.235+0000 7f051cc00640 1 -- 172.21.15.46:0/3428779828 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f05140ec0f0 con 0x7f05200b0540 2024-03-29T08:20:52.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.431+0000 7f0527600640 1 -- 172.21.15.46:0/3428779828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0520154770 con 0x7f05200b0540 2024-03-29T08:20:52.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.431+0000 7f051cc00640 1 -- 172.21.15.46:0/3428779828 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4731 (secure 0 0 0) 0x7f05140c4090 con 0x7f05200b0540 2024-03-29T08:20:52.435 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:52.435 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2702541693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2702541693},{"type":"v1","addr":"172.21.15.73:6837","nonce":2702541693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2507803771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2507803771},{"type":"v1","addr":"172.21.15.73:6835","nonce":2507803771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/494932499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":494932499},{"type":"v1","addr":"172.21.15.46:6833","nonce":494932499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:20:12.400114+0000","modified":"2024-03-29T08:20:15.405861+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3467089652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3467089652},{"type":"v1","addr":"172.21.15.46:6835","nonce":3467089652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-29T08:20:52.436 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-03-29T08:20:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.435+0000 7f0527600640 1 -- 172.21.15.46:0/3428779828 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0500066690 msgr2=0x7f0500068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.435+0000 7f0527600640 1 --2- 172.21.15.46:0/3428779828 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0500066690 0x7f0500068b50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f052009e650 tx=0x7f050c008040 comp rx=0 tx=0).stop 2024-03-29T08:20:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.435+0000 7f0527600640 1 -- 172.21.15.46:0/3428779828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05200b0540 msgr2=0x7f05201f4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.435+0000 7f0527600640 1 --2- 172.21.15.46:0/3428779828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05200b0540 0x7f05201f4ac0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f051401ab70 tx=0x7f051401ad00 comp rx=0 tx=0).stop 2024-03-29T08:20:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.435+0000 7f0527600640 1 -- 172.21.15.46:0/3428779828 shutdown_connections 2024-03-29T08:20:52.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.435+0000 7f0527600640 1 --2- 172.21.15.46:0/3428779828 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0500066690 0x7f0500068b50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.435+0000 7f0527600640 1 --2- 172.21.15.46:0/3428779828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f052015a940 0x7f05201540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.435+0000 7f0527600640 1 --2- 172.21.15.46:0/3428779828 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f052015ad20 0x7f0520157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.435+0000 7f0527600640 1 --2- 172.21.15.46:0/3428779828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05200b0540 0x7f05201f4ac0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.435+0000 7f0527600640 1 -- 172.21.15.46:0/3428779828 >> 172.21.15.46:0/3428779828 conn(0x7f0520089fd0 msgr2=0x7f052006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.435+0000 7f0527600640 1 -- 172.21.15.46:0/3428779828 shutdown_connections 2024-03-29T08:20:52.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.435+0000 7f0527600640 1 -- 172.21.15.46:0/3428779828 wait complete. 2024-03-29T08:20:52.458 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-03-29T08:20:52.624 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-03-29T08:20:52.624 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-29T08:20:52.624 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-03-29T08:20:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.667+0000 7f1053a53080 1 Processor -- start 2024-03-29T08:20:52.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.667+0000 7f1053a53080 1 -- start start 2024-03-29T08:20:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.667+0000 7f1053a53080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56060cad9e70 0x56060cb7b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.667+0000 7f1053a53080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56060cb7e610 0x56060cb81fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.667+0000 7f1053a53080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56060cb7e9f0 0x56060cad8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.667+0000 7f1053a53080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060c9f0960 con 0x56060cb7e9f0 2024-03-29T08:20:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.667+0000 7f1053a53080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060c9f0600 con 0x56060cad9e70 2024-03-29T08:20:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.667+0000 7f1053a53080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x56060c9e7690 con 0x56060cb7e610 2024-03-29T08:20:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.667+0000 7f1051600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56060cb7e9f0 0x56060cad8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.667+0000 7f1051600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56060cb7e9f0 0x56060cad8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38538/0 (socket says 172.21.15.46:38538) 2024-03-29T08:20:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.667+0000 7f1051600640 1 -- 172.21.15.46:0/4091435576 learned_addr learned my addr 172.21.15.46:0/4091435576 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.667+0000 7f104be00640 1 --2- 172.21.15.46:0/4091435576 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56060cb7e610 0x56060cb81fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.667+0000 7f1050c00640 1 --2- 172.21.15.46:0/4091435576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56060cad9e70 0x56060cb7b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.667+0000 7f1051600640 1 -- 172.21.15.46:0/4091435576 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56060cb7e610 msgr2=0x56060cb81fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1051600640 1 --2- 172.21.15.46:0/4091435576 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56060cb7e610 0x56060cb81fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1051600640 1 -- 172.21.15.46:0/4091435576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56060cad9e70 msgr2=0x56060cb7b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1051600640 1 --2- 172.21.15.46:0/4091435576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56060cad9e70 0x56060cb7b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1051600640 1 -- 172.21.15.46:0/4091435576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c00a100 con 0x56060cb7e9f0 2024-03-29T08:20:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1051600640 1 --2- 172.21.15.46:0/4091435576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56060cb7e9f0 0x56060cad8bb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f104c0190a0 tx=0x7f104c0033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf8d0fcc8dff99b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f104aa00640 1 -- 172.21.15.46:0/4091435576 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c0acb50 con 0x56060cb7e9f0 2024-03-29T08:20:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f104aa00640 1 -- 172.21.15.46:0/4091435576 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c0accb0 con 0x56060cb7e9f0 2024-03-29T08:20:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f104aa00640 1 -- 172.21.15.46:0/4091435576 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c0b5610 con 0x56060cb7e9f0 2024-03-29T08:20:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1053a53080 1 -- 172.21.15.46:0/4091435576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56060cb7e9f0 msgr2=0x56060cad8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1053a53080 1 --2- 172.21.15.46:0/4091435576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56060cb7e9f0 0x56060cad8bb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f104c0190a0 tx=0x7f104c0033b0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1053a53080 1 -- 172.21.15.46:0/4091435576 shutdown_connections 2024-03-29T08:20:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1053a53080 1 --2- 172.21.15.46:0/4091435576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56060cb7e9f0 0x56060cad8bb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1053a53080 1 --2- 172.21.15.46:0/4091435576 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56060cb7e610 0x56060cb81fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1053a53080 1 --2- 172.21.15.46:0/4091435576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56060cad9e70 0x56060cb7b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1053a53080 1 -- 172.21.15.46:0/4091435576 >> 172.21.15.46:0/4091435576 conn(0x56060caa9cf0 msgr2=0x56060cb81310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1053a53080 1 -- 172.21.15.46:0/4091435576 shutdown_connections 2024-03-29T08:20:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1053a53080 1 -- 172.21.15.46:0/4091435576 wait complete. 2024-03-29T08:20:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1053a53080 1 Processor -- start 2024-03-29T08:20:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1053a53080 1 -- start start 2024-03-29T08:20:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1053a53080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56060cad9e70 0x56060cb7abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1053a53080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56060cb7e610 0x56060cb86e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1053a53080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56060cb7e9f0 0x56060cad7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1053a53080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060cb877d0 con 0x56060cb7e610 2024-03-29T08:20:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1053a53080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56060cb87a70 con 0x56060cb7e9f0 2024-03-29T08:20:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1053a53080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x56060cb87d10 con 0x56060cad9e70 2024-03-29T08:20:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f104be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56060cb7e610 0x56060cb86e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f104be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56060cb7e610 0x56060cb86e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38542/0 (socket says 172.21.15.46:38542) 2024-03-29T08:20:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f104be00640 1 -- 172.21.15.46:0/1189740738 learned_addr learned my addr 172.21.15.46:0/1189740738 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1050c00640 1 --2- 172.21.15.46:0/1189740738 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56060cad9e70 0x56060cb7abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f1051600640 1 --2- 172.21.15.46:0/1189740738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56060cb7e9f0 0x56060cad7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f104be00640 1 -- 172.21.15.46:0/1189740738 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56060cad9e70 msgr2=0x56060cb7abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f104be00640 1 --2- 172.21.15.46:0/1189740738 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56060cad9e70 0x56060cb7abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f104be00640 1 -- 172.21.15.46:0/1189740738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56060cb7e9f0 msgr2=0x56060cad7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f104be00640 1 --2- 172.21.15.46:0/1189740738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56060cb7e9f0 0x56060cad7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f104be00640 1 -- 172.21.15.46:0/1189740738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56060ca19a60 con 0x56060cb7e610 2024-03-29T08:20:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.671+0000 7f104be00640 1 --2- 172.21.15.46:0/1189740738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56060cb7e610 0x56060cb86e50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f103c00dd10 tx=0x7f103c00b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:52.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.675+0000 7f1049600640 1 -- 172.21.15.46:0/1189740738 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c017470 con 0x56060cb7e610 2024-03-29T08:20:52.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.675+0000 7f1049600640 1 -- 172.21.15.46:0/1189740738 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103c0175d0 con 0x56060cb7e610 2024-03-29T08:20:52.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.675+0000 7f1049600640 1 -- 172.21.15.46:0/1189740738 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c017980 con 0x56060cb7e610 2024-03-29T08:20:52.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.675+0000 7f1053a53080 1 -- 172.21.15.46:0/1189740738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56060c9ff8c0 con 0x56060cb7e610 2024-03-29T08:20:52.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.675+0000 7f1053a53080 1 -- 172.21.15.46:0/1189740738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56060c9f2d00 con 0x56060cb7e610 2024-03-29T08:20:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.675+0000 7f1049600640 1 -- 172.21.15.46:0/1189740738 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f103c014070 con 0x56060cb7e610 2024-03-29T08:20:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.675+0000 7f1049600640 1 --2- 172.21.15.46:0/1189740738 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f10300650a0 0x7f1030067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.675+0000 7f1049600640 1 -- 172.21.15.46:0/1189740738 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f103c083130 con 0x56060cb7e610 2024-03-29T08:20:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.675+0000 7f1053a53080 1 --2- 172.21.15.46:0/1189740738 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x56060cad8c20 0x56060cb847d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.675+0000 7f1053a53080 1 -- 172.21.15.46:0/1189740738 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49) v8 -- 0x56060cb84d10 con 0x56060cad8c20 2024-03-29T08:20:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.675+0000 7f1050c00640 1 --2- 172.21.15.46:0/1189740738 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f10300650a0 0x7f1030067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.675+0000 7f1051600640 1 --2- 172.21.15.46:0/1189740738 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x56060cad8c20 0x56060cb847d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:52.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.675+0000 7f1051600640 1 --2- 172.21.15.46:0/1189740738 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x56060cad8c20 0x56060cb847d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:52.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.675+0000 7f1050c00640 1 --2- 172.21.15.46:0/1189740738 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f10300650a0 0x7f1030067560 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f10440097d0 tx=0x7f1044005b50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:52.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.679+0000 7f1051600640 1 -- 172.21.15.46:0/1189740738 <== osd.3 v2:172.21.15.46:6808/27426 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f104c0ac9d0 con 0x56060cad8c20 2024-03-29T08:20:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.679+0000 7f1053a53080 1 -- 172.21.15.46:0/1189740738 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x56060cad8c20 msgr2=0x56060cb847d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.679+0000 7f1053a53080 1 --2- 172.21.15.46:0/1189740738 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x56060cad8c20 0x56060cb847d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.679+0000 7f1053a53080 1 -- 172.21.15.46:0/1189740738 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f10300650a0 msgr2=0x7f1030067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.679+0000 7f1053a53080 1 --2- 172.21.15.46:0/1189740738 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f10300650a0 0x7f1030067560 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f10440097d0 tx=0x7f1044005b50 comp rx=0 tx=0).stop 2024-03-29T08:20:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.679+0000 7f1053a53080 1 -- 172.21.15.46:0/1189740738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56060cb7e610 msgr2=0x56060cb86e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.679+0000 7f1053a53080 1 --2- 172.21.15.46:0/1189740738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56060cb7e610 0x56060cb86e50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f103c00dd10 tx=0x7f103c00b6d0 comp rx=0 tx=0).stop 2024-03-29T08:20:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.679+0000 7f1051600640 1 -- 172.21.15.46:0/1189740738 reap_dead start 2024-03-29T08:20:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.679+0000 7f1053a53080 1 -- 172.21.15.46:0/1189740738 shutdown_connections 2024-03-29T08:20:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.679+0000 7f1053a53080 1 -- 172.21.15.46:0/1189740738 >> 172.21.15.46:0/1189740738 conn(0x56060caa9cf0 msgr2=0x56060caaa0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.679+0000 7f1053a53080 1 -- 172.21.15.46:0/1189740738 shutdown_connections 2024-03-29T08:20:52.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:52.679+0000 7f1053a53080 1 -- 172.21.15.46:0/1189740738 wait complete. 2024-03-29T08:20:52.684 INFO:teuthology.orchestra.run.smithi046.stdout:Yalphaparent_a 2024-03-29T08:20:52.684 DEBUG:teuthology.orchestra.run.smithi046:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-03-29T08:20:52.776 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:20:52.776 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627777, 2024-03-29T08:20:52.776 INFO:teuthology.orchestra.run.smithi046.stdout: "ancestors": [ 2024-03-29T08:20:52.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:52.776 INFO:teuthology.orchestra.run.smithi046.stdout: "dirino": 1099511627776, 2024-03-29T08:20:52.776 INFO:teuthology.orchestra.run.smithi046.stdout: "dname": "alpha", 2024-03-29T08:20:52.776 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 4 2024-03-29T08:20:52.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:20:52.776 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:20:52.776 INFO:teuthology.orchestra.run.smithi046.stdout: "dirino": 1, 2024-03-29T08:20:52.776 INFO:teuthology.orchestra.run.smithi046.stdout: "dname": "parent_a", 2024-03-29T08:20:52.777 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 7 2024-03-29T08:20:52.777 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:20:52.777 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-29T08:20:52.777 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 7, 2024-03-29T08:20:52.777 INFO:teuthology.orchestra.run.smithi046.stdout: "old_pools": [] 2024-03-29T08:20:52.777 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:20:52.777 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:52.781 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T08:20:52.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:20:52.886 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:20:52.962 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T08:20:52.963 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:20:52.963 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:20:52.972 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:20:53.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.155+0000 7ff918c00640 1 Processor -- start 2024-03-29T08:20:53.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.155+0000 7ff918c00640 1 -- start start 2024-03-29T08:20:53.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff918c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff9140b1d30 0x7ff9140b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff918c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff914066100 con 0x7ff914156e80 2024-03-29T08:20:53.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff918c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff914065e20 con 0x7ff91406a070 2024-03-29T08:20:53.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff918c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9140b2720 con 0x7ff9140b1d30 2024-03-29T08:20:53.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff913400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff914156e80 0x7ff91415d920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48324/0 (socket says 172.21.15.46:48324) 2024-03-29T08:20:53.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff913400640 1 -- 172.21.15.46:0/1461517940 learned_addr learned my addr 172.21.15.46:0/1461517940 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:53.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff90be00640 1 --2- 172.21.15.46:0/1461517940 >> v2:172.21.15.73:3300/0 conn(0x7ff9140b1d30 0x7ff9140b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff912a00640 1 -- 172.21.15.46:0/1461517940 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211210008 0 0) 0x7ff914066100 con 0x7ff914156e80 2024-03-29T08:20:53.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff912a00640 1 -- 172.21.15.46:0/1461517940 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8f8003530 con 0x7ff914156e80 2024-03-29T08:20:53.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff912a00640 1 -- 172.21.15.46:0/1461517940 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3656730554 0 0) 0x7ff8f8003530 con 0x7ff914156e80 2024-03-29T08:20:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff912a00640 1 -- 172.21.15.46:0/1461517940 >> v2:172.21.15.73:3300/0 conn(0x7ff9140b1d30 msgr2=0x7ff9140b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff912a00640 1 --2- 172.21.15.46:0/1461517940 >> v2:172.21.15.73:3300/0 conn(0x7ff9140b1d30 0x7ff9140b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff912a00640 1 -- 172.21.15.46:0/1461517940 >> v1:172.21.15.73:6789/0 conn(0x7ff91406a070 legacy=0x7ff91406a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff912a00640 1 -- 172.21.15.46:0/1461517940 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9140cb050 con 0x7ff914156e80 2024-03-29T08:20:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff912a00640 1 -- 172.21.15.46:0/1461517940 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7ff904001680 con 0x7ff914156e80 2024-03-29T08:20:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff912a00640 1 -- 172.21.15.46:0/1461517940 >> v1:172.21.15.46:6789/0 conn(0x7ff914156e80 legacy=0x7ff91415d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff912a00640 1 --2- 172.21.15.46:0/1461517940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f8003980 0x7ff8f8003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff912a00640 1 --2- 172.21.15.46:0/1461517940 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff8f80047d0 0x7ff8f8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff912a00640 1 --2- 172.21.15.46:0/1461517940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8f8005190 0x7ff8f800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff912a00640 1 -- 172.21.15.46:0/1461517940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9140b2720 con 0x7ff8f8005190 2024-03-29T08:20:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff912a00640 1 -- 172.21.15.46:0/1461517940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f800c090 con 0x7ff8f8003980 2024-03-29T08:20:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff912a00640 1 -- 172.21.15.46:0/1461517940 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f800c250 con 0x7ff8f80047d0 2024-03-29T08:20:53.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff913e00640 1 --2- 172.21.15.46:0/1461517940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8f8005190 0x7ff8f800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff90be00640 1 --2- 172.21.15.46:0/1461517940 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff8f80047d0 0x7ff8f8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff913400640 1 --2- 172.21.15.46:0/1461517940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f8003980 0x7ff8f8003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff913e00640 1 -- 172.21.15.46:0/1461517940 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff8f80047d0 msgr2=0x7ff8f8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:53.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff913e00640 1 --2- 172.21.15.46:0/1461517940 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff8f80047d0 0x7ff8f8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff913e00640 1 -- 172.21.15.46:0/1461517940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f8003980 msgr2=0x7ff8f8003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:53.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff913e00640 1 --2- 172.21.15.46:0/1461517940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f8003980 0x7ff8f8003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.159+0000 7ff913e00640 1 -- 172.21.15.46:0/1461517940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f8003530 con 0x7ff8f8005190 2024-03-29T08:20:53.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.163+0000 7ff913e00640 1 -- 172.21.15.46:0/1461517940 reap_dead start 2024-03-29T08:20:53.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.163+0000 7ff913e00640 1 --2- 172.21.15.46:0/1461517940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8f8005190 0x7ff8f800b990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff90c001f60 tx=0x7ff90c01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b2750a817cecb7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:53.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.163+0000 7ff912a00640 1 -- 172.21.15.46:0/1461517940 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91406b3e0 con 0x7ff8f8005190 2024-03-29T08:20:53.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.163+0000 7ff912a00640 1 -- 172.21.15.46:0/1461517940 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90c003b80 con 0x7ff8f8005190 2024-03-29T08:20:53.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.163+0000 7ff912a00640 1 -- 172.21.15.46:0/1461517940 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c0050d0 con 0x7ff8f8005190 2024-03-29T08:20:53.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.163+0000 7ff918c00640 1 -- 172.21.15.46:0/1461517940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8f8005190 msgr2=0x7ff8f800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:53.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.163+0000 7ff918c00640 1 --2- 172.21.15.46:0/1461517940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8f8005190 0x7ff8f800b990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff90c001f60 tx=0x7ff90c01c850 comp rx=0 tx=0).stop 2024-03-29T08:20:53.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.163+0000 7ff918c00640 1 -- 172.21.15.46:0/1461517940 shutdown_connections 2024-03-29T08:20:53.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.163+0000 7ff918c00640 1 --2- 172.21.15.46:0/1461517940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8f8005190 0x7ff8f800b990 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.163+0000 7ff918c00640 1 -- 172.21.15.46:0/1461517940 >> 172.21.15.46:0/1461517940 conn(0x7ff9140cb8a0 msgr2=0x7ff9140cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:53.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.163+0000 7ff918c00640 1 -- 172.21.15.46:0/1461517940 shutdown_connections 2024-03-29T08:20:53.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.163+0000 7ff918c00640 1 -- 172.21.15.46:0/1461517940 wait complete. 2024-03-29T08:20:53.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.163+0000 7ff918c00640 1 Processor -- start 2024-03-29T08:20:53.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.163+0000 7ff918c00640 1 -- start start 2024-03-29T08:20:53.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff918c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f8005190 0x7ff914137320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff918c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9141577c0 0x7ff91414c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff918c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff914157c40 0x7ff914149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff918c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91414d260 con 0x7ff9141577c0 2024-03-29T08:20:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff918c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91414d540 con 0x7ff8f8005190 2024-03-29T08:20:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff918c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91414d820 con 0x7ff914157c40 2024-03-29T08:20:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff90be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9141577c0 0x7ff91414c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff90be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9141577c0 0x7ff91414c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38558/0 (socket says 172.21.15.46:38558) 2024-03-29T08:20:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff90be00640 1 -- 172.21.15.46:0/2511470598 learned_addr learned my addr 172.21.15.46:0/2511470598 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff913400640 1 --2- 172.21.15.46:0/2511470598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f8005190 0x7ff914137320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff913e00640 1 --2- 172.21.15.46:0/2511470598 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff914157c40 0x7ff914149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff90be00640 1 -- 172.21.15.46:0/2511470598 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff914157c40 msgr2=0x7ff914149960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff90be00640 1 --2- 172.21.15.46:0/2511470598 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff914157c40 0x7ff914149960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff90be00640 1 -- 172.21.15.46:0/2511470598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f8005190 msgr2=0x7ff914137320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff90be00640 1 --2- 172.21.15.46:0/2511470598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f8005190 0x7ff914137320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff90be00640 1 -- 172.21.15.46:0/2511470598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91414db00 con 0x7ff9141577c0 2024-03-29T08:20:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff913400640 1 --2- 172.21.15.46:0/2511470598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f8005190 0x7ff914137320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff90be00640 1 --2- 172.21.15.46:0/2511470598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9141577c0 0x7ff91414c7c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc011960 tx=0x7ff8fc011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff90b400640 1 -- 172.21.15.46:0/2511470598 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc00faf0 con 0x7ff9141577c0 2024-03-29T08:20:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff90b400640 1 -- 172.21.15.46:0/2511470598 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8fc00fc50 con 0x7ff9141577c0 2024-03-29T08:20:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff918c00640 1 -- 172.21.15.46:0/2511470598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91414dde0 con 0x7ff9141577c0 2024-03-29T08:20:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff90b400640 1 -- 172.21.15.46:0/2511470598 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc014440 con 0x7ff9141577c0 2024-03-29T08:20:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff918c00640 1 -- 172.21.15.46:0/2511470598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff914137860 con 0x7ff9141577c0 2024-03-29T08:20:53.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.167+0000 7ff918c00640 1 -- 172.21.15.46:0/2511470598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91404e040 con 0x7ff9141577c0 2024-03-29T08:20:53.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.171+0000 7ff90b400640 1 -- 172.21.15.46:0/2511470598 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff8fc01d400 con 0x7ff9141577c0 2024-03-29T08:20:53.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.171+0000 7ff90b400640 1 --2- 172.21.15.46:0/2511470598 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff8f0066690 0x7ff8f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.171+0000 7ff90b400640 1 -- 172.21.15.46:0/2511470598 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8fc084ab0 con 0x7ff9141577c0 2024-03-29T08:20:53.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.171+0000 7ff913400640 1 --2- 172.21.15.46:0/2511470598 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff8f0066690 0x7ff8f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.171+0000 7ff913400640 1 --2- 172.21.15.46:0/2511470598 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff8f0066690 0x7ff8f0068b50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff904007bd0 tx=0x7ff904009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:53.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.175+0000 7ff90b400640 1 -- 172.21.15.46:0/2511470598 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff8fc051d80 con 0x7ff9141577c0 2024-03-29T08:20:53.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.335+0000 7ff918c00640 1 -- 172.21.15.46:0/2511470598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9140cc960 con 0x7ff9141577c0 2024-03-29T08:20:53.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.339+0000 7ff90b400640 1 -- 172.21.15.46:0/2511470598 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+15993 (secure 0 0 0) 0x7ff8fc0553a0 con 0x7ff9141577c0 2024-03-29T08:20:53.340 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:53.340 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":49,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:20:13.402206+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:20:10.906619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-29T08:20:11.389667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:20:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.339+0000 7ff918c00640 1 -- 172.21.15.46:0/2511470598 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff8f0066690 msgr2=0x7ff8f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.339+0000 7ff918c00640 1 --2- 172.21.15.46:0/2511470598 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff8f0066690 0x7ff8f0068b50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff904007bd0 tx=0x7ff904009040 comp rx=0 tx=0).stop 2024-03-29T08:20:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.339+0000 7ff918c00640 1 -- 172.21.15.46:0/2511470598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9141577c0 msgr2=0x7ff91414c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.339+0000 7ff918c00640 1 --2- 172.21.15.46:0/2511470598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9141577c0 0x7ff91414c7c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc011960 tx=0x7ff8fc011e20 comp rx=0 tx=0).stop 2024-03-29T08:20:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.339+0000 7ff918c00640 1 -- 172.21.15.46:0/2511470598 shutdown_connections 2024-03-29T08:20:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.339+0000 7ff918c00640 1 --2- 172.21.15.46:0/2511470598 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff8f0066690 0x7ff8f0068b50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.339+0000 7ff918c00640 1 --2- 172.21.15.46:0/2511470598 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff914157c40 0x7ff914149960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.339+0000 7ff918c00640 1 --2- 172.21.15.46:0/2511470598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9141577c0 0x7ff91414c7c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.339+0000 7ff918c00640 1 --2- 172.21.15.46:0/2511470598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8f8005190 0x7ff914137320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.339+0000 7ff918c00640 1 -- 172.21.15.46:0/2511470598 >> 172.21.15.46:0/2511470598 conn(0x7ff9140cb8a0 msgr2=0x7ff91415e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:53.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.339+0000 7ff918c00640 1 -- 172.21.15.46:0/2511470598 shutdown_connections 2024-03-29T08:20:53.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.339+0000 7ff918c00640 1 -- 172.21.15.46:0/2511470598 wait complete. 2024-03-29T08:20:53.361 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:20:10.906619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:20:11.389667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:20:53.362 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:20:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.503+0000 7f7535600640 1 Processor -- start 2024-03-29T08:20:53.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.503+0000 7f7535600640 1 -- start start 2024-03-29T08:20:53.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.503+0000 7f7535600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f753015c950 0x7f753006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.503+0000 7f7535600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f75300cea30 0x7f75301606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.507+0000 7f7535600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75300cfb20 con 0x7f753015cd30 2024-03-29T08:20:53.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.507+0000 7f7535600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75300cef10 con 0x7f75300cea30 2024-03-29T08:20:53.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.507+0000 7f7535600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75300cf1f0 con 0x7f753015c950 2024-03-29T08:20:53.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.507+0000 7f752f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f753015cd30 0x7f75300cad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48330/0 (socket says 172.21.15.46:48330) 2024-03-29T08:20:53.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.507+0000 7f7527e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f75300cea30 0x7f75301606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.507+0000 7f752f400640 1 -- 172.21.15.46:0/2031079418 learned_addr learned my addr 172.21.15.46:0/2031079418 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:53.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.507+0000 7f752ea00640 1 --2- 172.21.15.46:0/2031079418 >> v2:172.21.15.73:3300/0 conn(0x7f753015c950 0x7f753006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.507+0000 7f7527e00640 1 -- 172.21.15.46:0/2031079418 >> v2:172.21.15.73:3300/0 conn(0x7f753015c950 msgr2=0x7f753006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:53.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.507+0000 7f7527e00640 1 --2- 172.21.15.46:0/2031079418 >> v2:172.21.15.73:3300/0 conn(0x7f753015c950 0x7f753006bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.507+0000 7f7527e00640 1 -- 172.21.15.46:0/2031079418 >> v1:172.21.15.46:6789/0 conn(0x7f753015cd30 legacy=0x7f75300cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:53.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.507+0000 7f7527e00640 1 -- 172.21.15.46:0/2031079418 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75300cf4d0 con 0x7f75300cea30 2024-03-29T08:20:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.507+0000 7f752e000640 1 -- 172.21.15.46:0/2031079418 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491696333 0 0) 0x7f75300cfb20 con 0x7f753015cd30 2024-03-29T08:20:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.507+0000 7f7527e00640 1 --2- 172.21.15.46:0/2031079418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75300cea30 0x7f75301606a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7530071840 tx=0x7f751801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d65a2c281abfa1f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f752e000640 1 -- 172.21.15.46:0/2031079418 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75180a8970 con 0x7f75300cea30 2024-03-29T08:20:53.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f752e000640 1 -- 172.21.15.46:0/2031079418 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75180a8ad0 con 0x7f75300cea30 2024-03-29T08:20:53.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f752e000640 1 -- 172.21.15.46:0/2031079418 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75180a8e60 con 0x7f75300cea30 2024-03-29T08:20:53.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f7535600640 1 -- 172.21.15.46:0/2031079418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75300cea30 msgr2=0x7f75301606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:53.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f7535600640 1 --2- 172.21.15.46:0/2031079418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75300cea30 0x7f75301606a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f7530071840 tx=0x7f751801ec70 comp rx=0 tx=0).stop 2024-03-29T08:20:53.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f7535600640 1 -- 172.21.15.46:0/2031079418 shutdown_connections 2024-03-29T08:20:53.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f7535600640 1 --2- 172.21.15.46:0/2031079418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75300cea30 0x7f75301606a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f7535600640 1 --2- 172.21.15.46:0/2031079418 >> v2:172.21.15.73:3300/0 conn(0x7f753015c950 0x7f753006bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f7535600640 1 -- 172.21.15.46:0/2031079418 >> 172.21.15.46:0/2031079418 conn(0x7f75300b21b0 msgr2=0x7f75300b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f7535600640 1 -- 172.21.15.46:0/2031079418 shutdown_connections 2024-03-29T08:20:53.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f7535600640 1 -- 172.21.15.46:0/2031079418 wait complete. 2024-03-29T08:20:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f7535600640 1 Processor -- start 2024-03-29T08:20:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f7535600640 1 -- start start 2024-03-29T08:20:53.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f7535600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75300cea30 0x7f753009a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f7535600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f753015c950 0x7f75301e8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f7535600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f753015cd30 0x7f7530158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f7535600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75301e90a0 con 0x7f753015cd30 2024-03-29T08:20:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f7535600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7530158fc0 con 0x7f75300cea30 2024-03-29T08:20:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f7535600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753009d450 con 0x7f753015c950 2024-03-29T08:20:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f752f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f753015cd30 0x7f7530158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f752f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f753015cd30 0x7f7530158a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38576/0 (socket says 172.21.15.46:38576) 2024-03-29T08:20:53.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f752f400640 1 -- 172.21.15.46:0/3666851387 learned_addr learned my addr 172.21.15.46:0/3666851387 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f752ea00640 1 --2- 172.21.15.46:0/3666851387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75300cea30 0x7f753009a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f7527e00640 1 --2- 172.21.15.46:0/3666851387 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f753015c950 0x7f75301e8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f752f400640 1 -- 172.21.15.46:0/3666851387 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f753015c950 msgr2=0x7f75301e8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f752f400640 1 --2- 172.21.15.46:0/3666851387 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f753015c950 0x7f75301e8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f752f400640 1 -- 172.21.15.46:0/3666851387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75300cea30 msgr2=0x7f753009a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f752f400640 1 --2- 172.21.15.46:0/3666851387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75300cea30 0x7f753009a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.511+0000 7f752f400640 1 -- 172.21.15.46:0/3666851387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7518009420 con 0x7f753015cd30 2024-03-29T08:20:53.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.515+0000 7f752f400640 1 --2- 172.21.15.46:0/3666851387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f753015cd30 0x7f7530158a80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f752800c780 tx=0x7f752800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.515+0000 7f7526a00640 1 -- 172.21.15.46:0/3666851387 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752800ce90 con 0x7f753015cd30 2024-03-29T08:20:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.515+0000 7f7526a00640 1 -- 172.21.15.46:0/3666851387 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752800a640 con 0x7f753015cd30 2024-03-29T08:20:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.515+0000 7f7535600640 1 -- 172.21.15.46:0/3666851387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753009d730 con 0x7f753015cd30 2024-03-29T08:20:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.515+0000 7f7526a00640 1 -- 172.21.15.46:0/3666851387 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752800a9d0 con 0x7f753015cd30 2024-03-29T08:20:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.515+0000 7f7535600640 1 -- 172.21.15.46:0/3666851387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753009dcf0 con 0x7f753015cd30 2024-03-29T08:20:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.515+0000 7f7535600640 1 -- 172.21.15.46:0/3666851387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753009df10 con 0x7f753015cd30 2024-03-29T08:20:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.515+0000 7f7526a00640 1 -- 172.21.15.46:0/3666851387 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f752800ab30 con 0x7f753015cd30 2024-03-29T08:20:53.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.515+0000 7f7526a00640 1 --2- 172.21.15.46:0/3666851387 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f750c066690 0x7f750c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.519+0000 7f7526a00640 1 -- 172.21.15.46:0/3666851387 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7528083880 con 0x7f753015cd30 2024-03-29T08:20:53.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.519+0000 7f752ea00640 1 --2- 172.21.15.46:0/3666851387 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f750c066690 0x7f750c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.519+0000 7f752ea00640 1 --2- 172.21.15.46:0/3666851387 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f750c066690 0x7f750c068b50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7520007a60 tx=0x7f7520008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:53.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.523+0000 7f7526a00640 1 -- 172.21.15.46:0/3666851387 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7528050ca0 con 0x7f753015cd30 2024-03-29T08:20:53.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.683+0000 7f7535600640 1 -- 172.21.15.46:0/3666851387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f753009e690 con 0x7f753015cd30 2024-03-29T08:20:53.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.683+0000 7f7526a00640 1 -- 172.21.15.46:0/3666851387 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f75280542c0 con 0x7f753015cd30 2024-03-29T08:20:53.687 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:20:53.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.687+0000 7f7535600640 1 -- 172.21.15.46:0/3666851387 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f750c066690 msgr2=0x7f750c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:53.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.687+0000 7f7535600640 1 --2- 172.21.15.46:0/3666851387 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f750c066690 0x7f750c068b50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7520007a60 tx=0x7f7520008040 comp rx=0 tx=0).stop 2024-03-29T08:20:53.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.687+0000 7f7535600640 1 -- 172.21.15.46:0/3666851387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f753015cd30 msgr2=0x7f7530158a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:53.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.687+0000 7f7535600640 1 --2- 172.21.15.46:0/3666851387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f753015cd30 0x7f7530158a80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f752800c780 tx=0x7f752800cc40 comp rx=0 tx=0).stop 2024-03-29T08:20:53.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.687+0000 7f7535600640 1 -- 172.21.15.46:0/3666851387 shutdown_connections 2024-03-29T08:20:53.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.687+0000 7f7535600640 1 --2- 172.21.15.46:0/3666851387 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f750c066690 0x7f750c068b50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.687+0000 7f7535600640 1 --2- 172.21.15.46:0/3666851387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f753015cd30 0x7f7530158a80 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.687+0000 7f7535600640 1 --2- 172.21.15.46:0/3666851387 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f753015c950 0x7f75301e8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.687+0000 7f7535600640 1 --2- 172.21.15.46:0/3666851387 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75300cea30 0x7f753009a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.687+0000 7f7535600640 1 -- 172.21.15.46:0/3666851387 >> 172.21.15.46:0/3666851387 conn(0x7f75300b21b0 msgr2=0x7f753006b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.687+0000 7f7535600640 1 -- 172.21.15.46:0/3666851387 shutdown_connections 2024-03-29T08:20:53.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.687+0000 7f7535600640 1 -- 172.21.15.46:0/3666851387 wait complete. 2024-03-29T08:20:53.709 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:20:53.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.851+0000 7f899ca00640 1 Processor -- start 2024-03-29T08:20:53.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.851+0000 7f899ca00640 1 -- start start 2024-03-29T08:20:53.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.851+0000 7f899ca00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f899806f1c0 0x7f8998069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.851+0000 7f899ca00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f899806a2d0 0x7f899806d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.851+0000 7f899ca00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f899806a720 con 0x7f899815cad0 2024-03-29T08:20:53.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.851+0000 7f899ca00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f899815dd00 con 0x7f899806f1c0 2024-03-29T08:20:53.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.851+0000 7f899ca00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899815d080 con 0x7f899806a2d0 2024-03-29T08:20:53.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.851+0000 7f8996800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f899806a2d0 0x7f899806d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.851+0000 7f8995400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f899806f1c0 0x7f8998069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.851+0000 7f8996800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f899806a2d0 0x7f899806d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:35256/0 (socket says 172.21.15.46:35256) 2024-03-29T08:20:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.851+0000 7f8996800640 1 -- 172.21.15.46:0/4207626386 learned_addr learned my addr 172.21.15.46:0/4207626386 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.851+0000 7f8995e00640 1 --1- 172.21.15.46:0/4207626386 >> v1:172.21.15.73:6789/0 conn(0x7f899815cad0 0x7f899806eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:46108/0 (socket says 172.21.15.46:46108) 2024-03-29T08:20:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.851+0000 7f8996800640 1 -- 172.21.15.46:0/4207626386 >> v2:172.21.15.73:3300/0 conn(0x7f899806f1c0 msgr2=0x7f8998069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.851+0000 7f8996800640 1 --2- 172.21.15.46:0/4207626386 >> v2:172.21.15.73:3300/0 conn(0x7f899806f1c0 0x7f8998069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.851+0000 7f8996800640 1 -- 172.21.15.46:0/4207626386 >> v1:172.21.15.73:6789/0 conn(0x7f899815cad0 legacy=0x7f899806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:53.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.851+0000 7f8996800640 1 -- 172.21.15.46:0/4207626386 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899815d360 con 0x7f899806a2d0 2024-03-29T08:20:53.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.855+0000 7f8996800640 1 --2- 172.21.15.46:0/4207626386 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f899806a2d0 0x7f899806d470 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8990009e50 tx=0x7f899001c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3031149b0be5025 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:53.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.855+0000 7f8994a00640 1 -- 172.21.15.46:0/4207626386 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89900a5770 con 0x7f899806a2d0 2024-03-29T08:20:53.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.855+0000 7f8994a00640 1 -- 172.21.15.46:0/4207626386 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89900a58d0 con 0x7f899806a2d0 2024-03-29T08:20:53.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.855+0000 7f8994a00640 1 -- 172.21.15.46:0/4207626386 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89900a5c60 con 0x7f899806a2d0 2024-03-29T08:20:53.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.855+0000 7f899ca00640 1 -- 172.21.15.46:0/4207626386 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f899806a2d0 msgr2=0x7f899806d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:53.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.855+0000 7f899ca00640 1 --2- 172.21.15.46:0/4207626386 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f899806a2d0 0x7f899806d470 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8990009e50 tx=0x7f899001c9a0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.855+0000 7f899ca00640 1 -- 172.21.15.46:0/4207626386 shutdown_connections 2024-03-29T08:20:53.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.855+0000 7f899ca00640 1 --2- 172.21.15.46:0/4207626386 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f899806a2d0 0x7f899806d470 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.855+0000 7f899ca00640 1 --2- 172.21.15.46:0/4207626386 >> v2:172.21.15.73:3300/0 conn(0x7f899806f1c0 0x7f8998069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.855+0000 7f899ca00640 1 -- 172.21.15.46:0/4207626386 >> 172.21.15.46:0/4207626386 conn(0x7f8998098a40 msgr2=0x7f89980b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:53.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.855+0000 7f899ca00640 1 -- 172.21.15.46:0/4207626386 shutdown_connections 2024-03-29T08:20:53.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.855+0000 7f899ca00640 1 -- 172.21.15.46:0/4207626386 wait complete. 2024-03-29T08:20:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.855+0000 7f899ca00640 1 Processor -- start 2024-03-29T08:20:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f899ca00640 1 -- start start 2024-03-29T08:20:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f899ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f899806a2d0 0x7f899814e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f899ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899806f1c0 0x7f899814c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f899ca00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f899815cad0 0x7f899814f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f899ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8998157250 con 0x7f899806f1c0 2024-03-29T08:20:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f899ca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8998157530 con 0x7f899806a2d0 2024-03-29T08:20:53.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f899ca00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899814c780 con 0x7f899815cad0 2024-03-29T08:20:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f8995400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899806f1c0 0x7f899814c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f8995400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899806f1c0 0x7f899814c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38582/0 (socket says 172.21.15.46:38582) 2024-03-29T08:20:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f8995400640 1 -- 172.21.15.46:0/3823843734 learned_addr learned my addr 172.21.15.46:0/3823843734 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f8995e00640 1 --2- 172.21.15.46:0/3823843734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f899806a2d0 0x7f899814e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f8996800640 1 --2- 172.21.15.46:0/3823843734 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f899815cad0 0x7f899814f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f8995400640 1 -- 172.21.15.46:0/3823843734 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f899815cad0 msgr2=0x7f899814f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f8995400640 1 --2- 172.21.15.46:0/3823843734 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f899815cad0 0x7f899814f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f8995400640 1 -- 172.21.15.46:0/3823843734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f899806a2d0 msgr2=0x7f899814e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f8995400640 1 --2- 172.21.15.46:0/3823843734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f899806a2d0 0x7f899814e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f8995400640 1 -- 172.21.15.46:0/3823843734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8990009b30 con 0x7f899806f1c0 2024-03-29T08:20:53.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f8996800640 1 --2- 172.21.15.46:0/3823843734 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f899815cad0 0x7f899814f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f8995e00640 1 --2- 172.21.15.46:0/3823843734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f899806a2d0 0x7f899814e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f8995400640 1 --2- 172.21.15.46:0/3823843734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899806f1c0 0x7f899814c240 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f898400e960 tx=0x7f898400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f898e000640 1 -- 172.21.15.46:0/3823843734 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984009690 con 0x7f899806f1c0 2024-03-29T08:20:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f898e000640 1 -- 172.21.15.46:0/3823843734 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898400ca00 con 0x7f899806f1c0 2024-03-29T08:20:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f898e000640 1 -- 172.21.15.46:0/3823843734 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898400cd70 con 0x7f899806f1c0 2024-03-29T08:20:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f899ca00640 1 -- 172.21.15.46:0/3823843734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89980b3ce0 con 0x7f899806f1c0 2024-03-29T08:20:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f899ca00640 1 -- 172.21.15.46:0/3823843734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89980b42a0 con 0x7f899806f1c0 2024-03-29T08:20:53.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.859+0000 7f899ca00640 1 -- 172.21.15.46:0/3823843734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89980cd8f0 con 0x7f899806f1c0 2024-03-29T08:20:53.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.863+0000 7f898e000640 1 -- 172.21.15.46:0/3823843734 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8984013470 con 0x7f899806f1c0 2024-03-29T08:20:53.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.863+0000 7f898e000640 1 --2- 172.21.15.46:0/3823843734 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8978066690 0x7f8978068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:53.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.863+0000 7f898e000640 1 -- 172.21.15.46:0/3823843734 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89840844a0 con 0x7f899806f1c0 2024-03-29T08:20:53.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.863+0000 7f8995e00640 1 --2- 172.21.15.46:0/3823843734 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8978066690 0x7f8978068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:53.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.863+0000 7f8995e00640 1 --2- 172.21.15.46:0/3823843734 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8978066690 0x7f8978068b50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8988000f70 tx=0x7f8988009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:53.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:53.867+0000 7f898e000640 1 -- 172.21.15.46:0/3823843734 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8984051770 con 0x7f899806f1c0 2024-03-29T08:20:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.035+0000 7f899ca00640 1 -- 172.21.15.46:0/3823843734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f899809d700 con 0x7f899806f1c0 2024-03-29T08:20:54.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.035+0000 7f898e000640 1 -- 172.21.15.46:0/3823843734 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8984054d90 con 0x7f899806f1c0 2024-03-29T08:20:54.037 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:20:54.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.035+0000 7f899ca00640 1 -- 172.21.15.46:0/3823843734 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8978066690 msgr2=0x7f8978068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:54.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.035+0000 7f899ca00640 1 --2- 172.21.15.46:0/3823843734 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8978066690 0x7f8978068b50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8988000f70 tx=0x7f8988009040 comp rx=0 tx=0).stop 2024-03-29T08:20:54.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.035+0000 7f899ca00640 1 -- 172.21.15.46:0/3823843734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899806f1c0 msgr2=0x7f899814c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.035+0000 7f899ca00640 1 --2- 172.21.15.46:0/3823843734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899806f1c0 0x7f899814c240 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f898400e960 tx=0x7f898400ee20 comp rx=0 tx=0).stop 2024-03-29T08:20:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.035+0000 7f899ca00640 1 -- 172.21.15.46:0/3823843734 shutdown_connections 2024-03-29T08:20:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.035+0000 7f899ca00640 1 --2- 172.21.15.46:0/3823843734 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8978066690 0x7f8978068b50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.035+0000 7f899ca00640 1 --2- 172.21.15.46:0/3823843734 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f899815cad0 0x7f899814f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.035+0000 7f899ca00640 1 --2- 172.21.15.46:0/3823843734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f899806f1c0 0x7f899814c240 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.035+0000 7f899ca00640 1 --2- 172.21.15.46:0/3823843734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f899806a2d0 0x7f899814e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.035+0000 7f899ca00640 1 -- 172.21.15.46:0/3823843734 >> 172.21.15.46:0/3823843734 conn(0x7f8998098a40 msgr2=0x7f8998155bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.035+0000 7f899ca00640 1 -- 172.21.15.46:0/3823843734 shutdown_connections 2024-03-29T08:20:54.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.035+0000 7f899ca00640 1 -- 172.21.15.46:0/3823843734 wait complete. 2024-03-29T08:20:54.058 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:20:54.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.199+0000 7f6bd3200640 1 Processor -- start 2024-03-29T08:20:54.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.199+0000 7f6bd3200640 1 -- start start 2024-03-29T08:20:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.199+0000 7f6bd3200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6bcc06f1c0 0x7f6bcc069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.199+0000 7f6bd3200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bcc06a720 con 0x7f6bcc06a2d0 2024-03-29T08:20:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.199+0000 7f6bd3200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bcc15d080 con 0x7f6bcc15cad0 2024-03-29T08:20:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.199+0000 7f6bd3200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc15dd00 con 0x7f6bcc06f1c0 2024-03-29T08:20:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.199+0000 7f6bd1200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6bcc06a2d0 0x7f6bcc06d470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48334/0 (socket says 172.21.15.46:48334) 2024-03-29T08:20:54.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.199+0000 7f6bd1200640 1 -- 172.21.15.46:0/3669916801 learned_addr learned my addr 172.21.15.46:0/3669916801 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:54.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bc3e00640 1 --2- 172.21.15.46:0/3669916801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6bcc06f1c0 0x7f6bcc069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:54.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcb400640 1 -- 172.21.15.46:0/3669916801 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1112220183 0 0) 0x7f6bcc06a720 con 0x7f6bcc06a2d0 2024-03-29T08:20:54.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcb400640 1 -- 172.21.15.46:0/3669916801 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bb0003530 con 0x7f6bcc06a2d0 2024-03-29T08:20:54.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcb400640 1 -- 172.21.15.46:0/3669916801 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514102141 0 0) 0x7f6bcc15d080 con 0x7f6bcc15cad0 2024-03-29T08:20:54.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcb400640 1 -- 172.21.15.46:0/3669916801 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bcc06a720 con 0x7f6bcc15cad0 2024-03-29T08:20:54.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcb400640 1 -- 172.21.15.46:0/3669916801 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3418504394 0 0) 0x7f6bb0003530 con 0x7f6bcc06a2d0 2024-03-29T08:20:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcb400640 1 -- 172.21.15.46:0/3669916801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6bcc06f1c0 msgr2=0x7f6bcc069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcb400640 1 --2- 172.21.15.46:0/3669916801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6bcc06f1c0 0x7f6bcc069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcb400640 1 -- 172.21.15.46:0/3669916801 >> v1:172.21.15.73:6789/0 conn(0x7f6bcc15cad0 legacy=0x7f6bcc06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcb400640 1 -- 172.21.15.46:0/3669916801 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc15d360 con 0x7f6bcc06a2d0 2024-03-29T08:20:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcb400640 1 -- 172.21.15.46:0/3669916801 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f6bc4001a00 con 0x7f6bcc06a2d0 2024-03-29T08:20:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcb400640 1 -- 172.21.15.46:0/3669916801 >> v1:172.21.15.46:6789/0 conn(0x7f6bcc06a2d0 legacy=0x7f6bcc06d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcb400640 1 --2- 172.21.15.46:0/3669916801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6bcc06f1c0 0x7f6bb0003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcb400640 1 --2- 172.21.15.46:0/3669916801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb00040f0 0x7f6bb00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcb400640 1 --2- 172.21.15.46:0/3669916801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bb0004d10 0x7f6bb0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcb400640 1 -- 172.21.15.46:0/3669916801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc15dd00 con 0x7f6bb00040f0 2024-03-29T08:20:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcb400640 1 -- 172.21.15.46:0/3669916801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb000bb50 con 0x7f6bb0004d10 2024-03-29T08:20:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcb400640 1 -- 172.21.15.46:0/3669916801 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb000bd10 con 0x7f6bcc06f1c0 2024-03-29T08:20:54.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bc3e00640 1 --2- 172.21.15.46:0/3669916801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb00040f0 0x7f6bb00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bcbe00640 1 --2- 172.21.15.46:0/3669916801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6bcc06f1c0 0x7f6bb0003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bd1200640 1 --2- 172.21.15.46:0/3669916801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bb0004d10 0x7f6bb0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bc3e00640 1 -- 172.21.15.46:0/3669916801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6bcc06f1c0 msgr2=0x7f6bb0003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bc3e00640 1 --2- 172.21.15.46:0/3669916801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6bcc06f1c0 0x7f6bb0003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bc3e00640 1 -- 172.21.15.46:0/3669916801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bb0004d10 msgr2=0x7f6bb0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bc3e00640 1 --2- 172.21.15.46:0/3669916801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bb0004d10 0x7f6bb0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bc3e00640 1 -- 172.21.15.46:0/3669916801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb0003530 con 0x7f6bb00040f0 2024-03-29T08:20:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.203+0000 7f6bd1200640 1 --2- 172.21.15.46:0/3669916801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bb0004d10 0x7f6bb0005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:54.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bc3e00640 1 --2- 172.21.15.46:0/3669916801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb00040f0 0x7f6bb00047d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009ab0 tx=0x7f6bb401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1527fa9b9300ab2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:54.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bcb400640 1 -- 172.21.15.46:0/3669916801 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40a5530 con 0x7f6bb00040f0 2024-03-29T08:20:54.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bcb400640 1 -- 172.21.15.46:0/3669916801 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb40a5690 con 0x7f6bb00040f0 2024-03-29T08:20:54.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bcb400640 1 -- 172.21.15.46:0/3669916801 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40a5a40 con 0x7f6bb00040f0 2024-03-29T08:20:54.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bd3200640 1 -- 172.21.15.46:0/3669916801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb00040f0 msgr2=0x7f6bb00047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:54.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bd3200640 1 --2- 172.21.15.46:0/3669916801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bb00040f0 0x7f6bb00047d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4009ab0 tx=0x7f6bb401ca50 comp rx=0 tx=0).stop 2024-03-29T08:20:54.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bd1200640 1 -- 172.21.15.46:0/3669916801 reap_dead start 2024-03-29T08:20:54.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bd3200640 1 -- 172.21.15.46:0/3669916801 shutdown_connections 2024-03-29T08:20:54.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bd3200640 1 -- 172.21.15.46:0/3669916801 >> 172.21.15.46:0/3669916801 conn(0x7f6bcc098a40 msgr2=0x7f6bcc0b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:54.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bd3200640 1 -- 172.21.15.46:0/3669916801 shutdown_connections 2024-03-29T08:20:54.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bd3200640 1 -- 172.21.15.46:0/3669916801 wait complete. 2024-03-29T08:20:54.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bd3200640 1 Processor -- start 2024-03-29T08:20:54.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bd3200640 1 -- start start 2024-03-29T08:20:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bd3200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6bb00040f0 0x7f6bcc0c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bd3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bcc160a80 0x7f6bcc0c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bd3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bcc162990 0x7f6bcc162e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bd3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc163350 con 0x7f6bcc162990 2024-03-29T08:20:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bd3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc0b3ca0 con 0x7f6bcc160a80 2024-03-29T08:20:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.207+0000 7f6bd3200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc1560c0 con 0x7f6bb00040f0 2024-03-29T08:20:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bd1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bcc162990 0x7f6bcc162e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bd1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bcc162990 0x7f6bcc162e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38606/0 (socket says 172.21.15.46:38606) 2024-03-29T08:20:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bd1200640 1 -- 172.21.15.46:0/1309464717 learned_addr learned my addr 172.21.15.46:0/1309464717 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bc3e00640 1 --2- 172.21.15.46:0/1309464717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bcc160a80 0x7f6bcc0c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bcbe00640 1 --2- 172.21.15.46:0/1309464717 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6bb00040f0 0x7f6bcc0c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bd1200640 1 -- 172.21.15.46:0/1309464717 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6bb00040f0 msgr2=0x7f6bcc0c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:54.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bd1200640 1 --2- 172.21.15.46:0/1309464717 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6bb00040f0 0x7f6bcc0c4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bd1200640 1 -- 172.21.15.46:0/1309464717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bcc160a80 msgr2=0x7f6bcc0c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bd1200640 1 --2- 172.21.15.46:0/1309464717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bcc160a80 0x7f6bcc0c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bd1200640 1 -- 172.21.15.46:0/1309464717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc0c5440 con 0x7f6bcc162990 2024-03-29T08:20:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bd1200640 1 --2- 172.21.15.46:0/1309464717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bcc162990 0x7f6bcc162e10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc06d110 tx=0x7f6bc4007ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bc8c00640 1 -- 172.21.15.46:0/1309464717 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4017070 con 0x7f6bcc162990 2024-03-29T08:20:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bc8c00640 1 -- 172.21.15.46:0/1309464717 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc400e9c0 con 0x7f6bcc162990 2024-03-29T08:20:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bd3200640 1 -- 172.21.15.46:0/1309464717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bcc156310 con 0x7f6bcc162990 2024-03-29T08:20:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bc8c00640 1 -- 172.21.15.46:0/1309464717 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400eb50 con 0x7f6bcc162990 2024-03-29T08:20:54.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bd3200640 1 -- 172.21.15.46:0/1309464717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bcc0c6350 con 0x7f6bcc162990 2024-03-29T08:20:54.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bd3200640 1 -- 172.21.15.46:0/1309464717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bcc0cd8f0 con 0x7f6bcc162990 2024-03-29T08:20:54.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.211+0000 7f6bc8c00640 1 -- 172.21.15.46:0/1309464717 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6bc400ecb0 con 0x7f6bcc162990 2024-03-29T08:20:54.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.215+0000 7f6bc8c00640 1 --2- 172.21.15.46:0/1309464717 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ba8066690 0x7f6ba8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:54.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.215+0000 7f6bc8c00640 1 -- 172.21.15.46:0/1309464717 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bc4083b40 con 0x7f6bcc162990 2024-03-29T08:20:54.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.215+0000 7f6bcbe00640 1 --2- 172.21.15.46:0/1309464717 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ba8066690 0x7f6ba8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:54.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.215+0000 7f6bcbe00640 1 --2- 172.21.15.46:0/1309464717 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ba8066690 0x7f6ba8068b50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc007fd0 tx=0x7f6bbc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:54.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.219+0000 7f6bc8c00640 1 -- 172.21.15.46:0/1309464717 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6bc4050ec0 con 0x7f6bcc162990 2024-03-29T08:20:54.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.375+0000 7f6bd3200640 1 -- 172.21.15.46:0/1309464717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6bcc09d700 con 0x7f6bcc162990 2024-03-29T08:20:54.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.375+0000 7f6bc8c00640 1 -- 172.21.15.46:0/1309464717 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6bc40544e0 con 0x7f6bcc162990 2024-03-29T08:20:54.382 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:20:54.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.379+0000 7f6bd3200640 1 -- 172.21.15.46:0/1309464717 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ba8066690 msgr2=0x7f6ba8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:54.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.379+0000 7f6bd3200640 1 --2- 172.21.15.46:0/1309464717 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ba8066690 0x7f6ba8068b50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc007fd0 tx=0x7f6bbc009040 comp rx=0 tx=0).stop 2024-03-29T08:20:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.379+0000 7f6bd3200640 1 -- 172.21.15.46:0/1309464717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bcc162990 msgr2=0x7f6bcc162e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.379+0000 7f6bd3200640 1 --2- 172.21.15.46:0/1309464717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bcc162990 0x7f6bcc162e10 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc06d110 tx=0x7f6bc4007ba0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.379+0000 7f6bd3200640 1 -- 172.21.15.46:0/1309464717 shutdown_connections 2024-03-29T08:20:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.379+0000 7f6bd3200640 1 --2- 172.21.15.46:0/1309464717 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ba8066690 0x7f6ba8068b50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.379+0000 7f6bd3200640 1 --2- 172.21.15.46:0/1309464717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bcc160a80 0x7f6bcc0c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.379+0000 7f6bd3200640 1 --2- 172.21.15.46:0/1309464717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6bcc162990 0x7f6bcc162e10 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.379+0000 7f6bd3200640 1 --2- 172.21.15.46:0/1309464717 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6bb00040f0 0x7f6bcc0c4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.379+0000 7f6bd3200640 1 -- 172.21.15.46:0/1309464717 >> 172.21.15.46:0/1309464717 conn(0x7f6bcc098a40 msgr2=0x7f6bcc06d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.379+0000 7f6bd3200640 1 -- 172.21.15.46:0/1309464717 shutdown_connections 2024-03-29T08:20:54.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.379+0000 7f6bd3200640 1 -- 172.21.15.46:0/1309464717 wait complete. 2024-03-29T08:20:54.400 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:20:54.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.539+0000 7fe4a4400640 1 Processor -- start 2024-03-29T08:20:54.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.539+0000 7fe4a4400640 1 -- start start 2024-03-29T08:20:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.539+0000 7fe4a4400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe49c1461f0 0x7fe49c0702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.539+0000 7fe4a4400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe49c0707e0 0x7fe49c0b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:54.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.539+0000 7fe4a4400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49c1604b0 con 0x7fe49c0b3270 2024-03-29T08:20:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.539+0000 7fe4a4400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49c1465d0 con 0x7fe49c0707e0 2024-03-29T08:20:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.539+0000 7fe4a4400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c070c20 con 0x7fe49c1461f0 2024-03-29T08:20:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.539+0000 7fe4a1a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe49c1461f0 0x7fe49c0702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.539+0000 7fe4a1000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe49c0707e0 0x7fe49c0b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.539+0000 7fe4a1a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe49c1461f0 0x7fe49c0702a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:35290/0 (socket says 172.21.15.46:35290) 2024-03-29T08:20:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.539+0000 7fe4a1a00640 1 -- 172.21.15.46:0/210056442 learned_addr learned my addr 172.21.15.46:0/210056442 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.539+0000 7fe4a2400640 1 --1- 172.21.15.46:0/210056442 >> v1:172.21.15.73:6789/0 conn(0x7fe49c0b3270 0x7fe49c06c390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:46134/0 (socket says 172.21.15.46:46134) 2024-03-29T08:20:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.543+0000 7fe49be00640 1 -- 172.21.15.46:0/210056442 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 401677890 0 0) 0x7fe49c1604b0 con 0x7fe49c0b3270 2024-03-29T08:20:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.543+0000 7fe4a1000640 1 -- 172.21.15.46:0/210056442 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe49c1461f0 msgr2=0x7fe49c0702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.543+0000 7fe4a1000640 1 --2- 172.21.15.46:0/210056442 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe49c1461f0 0x7fe49c0702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.543+0000 7fe4a1000640 1 -- 172.21.15.46:0/210056442 >> v1:172.21.15.73:6789/0 conn(0x7fe49c0b3270 legacy=0x7fe49c06c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.543+0000 7fe4a1000640 1 -- 172.21.15.46:0/210056442 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c160700 con 0x7fe49c0707e0 2024-03-29T08:20:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.543+0000 7fe4a1a00640 1 --2- 172.21.15.46:0/210056442 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe49c1461f0 0x7fe49c0702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:20:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.543+0000 7fe4a1000640 1 --2- 172.21.15.46:0/210056442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe49c0707e0 0x7fe49c0b2d30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe4880097c0 tx=0x7fe48801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e18e6f57162c3fd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.543+0000 7fe49be00640 1 -- 172.21.15.46:0/210056442 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4880ac640 con 0x7fe49c0707e0 2024-03-29T08:20:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.543+0000 7fe49be00640 1 -- 172.21.15.46:0/210056442 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4880ac7a0 con 0x7fe49c0707e0 2024-03-29T08:20:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.543+0000 7fe49be00640 1 -- 172.21.15.46:0/210056442 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4880acb30 con 0x7fe49c0707e0 2024-03-29T08:20:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.543+0000 7fe4a4400640 1 -- 172.21.15.46:0/210056442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe49c0707e0 msgr2=0x7fe49c0b2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.543+0000 7fe4a4400640 1 --2- 172.21.15.46:0/210056442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe49c0707e0 0x7fe49c0b2d30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe4880097c0 tx=0x7fe48801c3e0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.543+0000 7fe4a4400640 1 -- 172.21.15.46:0/210056442 shutdown_connections 2024-03-29T08:20:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.543+0000 7fe4a4400640 1 --2- 172.21.15.46:0/210056442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe49c0707e0 0x7fe49c0b2d30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.543+0000 7fe4a4400640 1 --2- 172.21.15.46:0/210056442 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe49c1461f0 0x7fe49c0702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.543+0000 7fe4a4400640 1 -- 172.21.15.46:0/210056442 >> 172.21.15.46:0/210056442 conn(0x7fe49c066080 msgr2=0x7fe49c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a4400640 1 -- 172.21.15.46:0/210056442 shutdown_connections 2024-03-29T08:20:54.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a4400640 1 -- 172.21.15.46:0/210056442 wait complete. 2024-03-29T08:20:54.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a4400640 1 Processor -- start 2024-03-29T08:20:54.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a4400640 1 -- start start 2024-03-29T08:20:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a4400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe49c0707e0 0x7fe49c15f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a4400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe49c0b3270 0x7fe49c0ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a4400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe49c1461f0 0x7fe49c138ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a4400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c15dff0 con 0x7fe49c1461f0 2024-03-29T08:20:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a4400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c15fa80 con 0x7fe49c0707e0 2024-03-29T08:20:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a4400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c15e240 con 0x7fe49c0b3270 2024-03-29T08:20:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a2400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe49c1461f0 0x7fe49c138ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a2400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe49c1461f0 0x7fe49c138ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38622/0 (socket says 172.21.15.46:38622) 2024-03-29T08:20:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a2400640 1 -- 172.21.15.46:0/1677070900 learned_addr learned my addr 172.21.15.46:0/1677070900 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a1000640 1 --2- 172.21.15.46:0/1677070900 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe49c0b3270 0x7fe49c0ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a1a00640 1 --2- 172.21.15.46:0/1677070900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe49c0707e0 0x7fe49c15f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a2400640 1 -- 172.21.15.46:0/1677070900 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe49c0b3270 msgr2=0x7fe49c0ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:54.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a2400640 1 --2- 172.21.15.46:0/1677070900 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe49c0b3270 0x7fe49c0ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a2400640 1 -- 172.21.15.46:0/1677070900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe49c0707e0 msgr2=0x7fe49c15f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a2400640 1 --2- 172.21.15.46:0/1677070900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe49c0707e0 0x7fe49c15f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a2400640 1 -- 172.21.15.46:0/1677070900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4900094d0 con 0x7fe49c1461f0 2024-03-29T08:20:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a1000640 1 --2- 172.21.15.46:0/1677070900 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe49c0b3270 0x7fe49c0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a1a00640 1 --2- 172.21.15.46:0/1677070900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe49c0707e0 0x7fe49c15f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:54.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a2400640 1 --2- 172.21.15.46:0/1677070900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe49c1461f0 0x7fe49c138ca0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe494001570 tx=0x7fe49400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe499600640 1 -- 172.21.15.46:0/1677070900 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49400a640 con 0x7fe49c1461f0 2024-03-29T08:20:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe499600640 1 -- 172.21.15.46:0/1677070900 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49400a7c0 con 0x7fe49c1461f0 2024-03-29T08:20:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe4a4400640 1 -- 172.21.15.46:0/1677070900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe488009420 con 0x7fe49c1461f0 2024-03-29T08:20:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.547+0000 7fe499600640 1 -- 172.21.15.46:0/1677070900 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49400ab70 con 0x7fe49c1461f0 2024-03-29T08:20:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.551+0000 7fe4a4400640 1 -- 172.21.15.46:0/1677070900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49c15e800 con 0x7fe49c1461f0 2024-03-29T08:20:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.551+0000 7fe4a4400640 1 -- 172.21.15.46:0/1677070900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49c15e9f0 con 0x7fe49c1461f0 2024-03-29T08:20:54.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.551+0000 7fe499600640 1 -- 172.21.15.46:0/1677070900 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe49400acd0 con 0x7fe49c1461f0 2024-03-29T08:20:54.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.551+0000 7fe499600640 1 --2- 172.21.15.46:0/1677070900 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe47c066690 0x7fe47c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.551+0000 7fe499600640 1 -- 172.21.15.46:0/1677070900 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe494083a00 con 0x7fe49c1461f0 2024-03-29T08:20:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.551+0000 7fe4a1a00640 1 --2- 172.21.15.46:0/1677070900 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe47c066690 0x7fe47c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:54.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.555+0000 7fe4a1a00640 1 --2- 172.21.15.46:0/1677070900 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe47c066690 0x7fe47c068b50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe490009c60 tx=0x7fe490008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:54.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.555+0000 7fe499600640 1 -- 172.21.15.46:0/1677070900 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe494050d60 con 0x7fe49c1461f0 2024-03-29T08:20:54.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.731+0000 7fe4a4400640 1 -- 172.21.15.46:0/1677070900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe49c071a90 con 0x7fe49c1461f0 2024-03-29T08:20:54.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.731+0000 7fe499600640 1 -- 172.21.15.46:0/1677070900 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe494018020 con 0x7fe49c1461f0 2024-03-29T08:20:54.733 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:20:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.731+0000 7fe4a4400640 1 -- 172.21.15.46:0/1677070900 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe47c066690 msgr2=0x7fe47c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.731+0000 7fe4a4400640 1 --2- 172.21.15.46:0/1677070900 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe47c066690 0x7fe47c068b50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe490009c60 tx=0x7fe490008040 comp rx=0 tx=0).stop 2024-03-29T08:20:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.731+0000 7fe4a4400640 1 -- 172.21.15.46:0/1677070900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe49c1461f0 msgr2=0x7fe49c138ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.731+0000 7fe4a4400640 1 --2- 172.21.15.46:0/1677070900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe49c1461f0 0x7fe49c138ca0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe494001570 tx=0x7fe49400cca0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.731+0000 7fe4a4400640 1 -- 172.21.15.46:0/1677070900 shutdown_connections 2024-03-29T08:20:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.731+0000 7fe4a4400640 1 --2- 172.21.15.46:0/1677070900 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe47c066690 0x7fe47c068b50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.731+0000 7fe4a4400640 1 --2- 172.21.15.46:0/1677070900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe49c1461f0 0x7fe49c138ca0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.731+0000 7fe4a4400640 1 --2- 172.21.15.46:0/1677070900 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe49c0b3270 0x7fe49c0ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.731+0000 7fe4a4400640 1 --2- 172.21.15.46:0/1677070900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe49c0707e0 0x7fe49c15f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.731+0000 7fe4a4400640 1 -- 172.21.15.46:0/1677070900 >> 172.21.15.46:0/1677070900 conn(0x7fe49c066080 msgr2=0x7fe49c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:54.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.731+0000 7fe4a4400640 1 -- 172.21.15.46:0/1677070900 shutdown_connections 2024-03-29T08:20:54.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:54.731+0000 7fe4a4400640 1 -- 172.21.15.46:0/1677070900 wait complete. 2024-03-29T08:20:54.754 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-03-29T08:20:54.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:54.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T08:20:54.771 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[38064]: fuse finished with error 0 and tester_r 0 2024-03-29T08:20:54.774 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:20:54.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:54.774 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:20:54.874 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:20:54.874 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:20:54.883 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:20:54.883 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:20:54.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:54.884 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T08:20:54.935 INFO:teuthology.orchestra.run.smithi073.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-29T08:20:54.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:20:54.936 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:20:54.936 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:20:54.945 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:20:54.945 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:20:54.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:54.945 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-29T08:20:54.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:20:54.998 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-29T08:20:54.999 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:20:54.999 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:20:55.008 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:20:55.009 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:20:55.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:20:55.009 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-29T08:20:55.059 INFO:teuthology.orchestra.run.smithi073.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-29T08:20:55.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:20:55.061 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:20:55.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.199+0000 7fb229a00640 1 Processor -- start 2024-03-29T08:20:55.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.199+0000 7fb229a00640 1 -- start start 2024-03-29T08:20:55.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.199+0000 7fb229a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb2241593d0 0x7fb22406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:55.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.199+0000 7fb229a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb22415d230 0x7fb22415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:55.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.199+0000 7fb229a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb22406d130 con 0x7fb22415dbd0 2024-03-29T08:20:55.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.199+0000 7fb229a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb22415b8f0 con 0x7fb22415d230 2024-03-29T08:20:55.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.199+0000 7fb229a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22415ad50 con 0x7fb2241593d0 2024-03-29T08:20:55.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.203+0000 7fb222a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb2241593d0 0x7fb22406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:55.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.203+0000 7fb21a000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb22415d230 0x7fb22415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:55.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.203+0000 7fb21a000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb22415d230 0x7fb22415d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51850/0 (socket says 172.21.15.46:51850) 2024-03-29T08:20:55.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.203+0000 7fb21a000640 1 -- 172.21.15.46:0/2246098802 learned_addr learned my addr 172.21.15.46:0/2246098802 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:55.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.203+0000 7fb222a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb2241593d0 0x7fb22406cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:35314/0 (socket says 172.21.15.46:35314) 2024-03-29T08:20:55.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.203+0000 7fb222a00640 1 -- 172.21.15.46:0/2246098802 >> v2:172.21.15.73:3300/0 conn(0x7fb22415d230 msgr2=0x7fb22415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.203+0000 7fb222a00640 1 --2- 172.21.15.46:0/2246098802 >> v2:172.21.15.73:3300/0 conn(0x7fb22415d230 0x7fb22415d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.203+0000 7fb222a00640 1 -- 172.21.15.46:0/2246098802 >> v1:172.21.15.73:6789/0 conn(0x7fb22415dbd0 legacy=0x7fb22415bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.203+0000 7fb222a00640 1 -- 172.21.15.46:0/2246098802 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22415e060 con 0x7fb2241593d0 2024-03-29T08:20:55.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.203+0000 7fb21a000640 1 --2- 172.21.15.46:0/2246098802 >> v2:172.21.15.73:3300/0 conn(0x7fb22415d230 0x7fb22415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:20:55.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.203+0000 7fb222a00640 1 --2- 172.21.15.46:0/2246098802 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb2241593d0 0x7fb22406cbf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb214009d90 tx=0x7fb21401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75071483738cf8cc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.203+0000 7fb222000640 1 -- 172.21.15.46:0/2246098802 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2140a5a30 con 0x7fb2241593d0 2024-03-29T08:20:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.203+0000 7fb222000640 1 -- 172.21.15.46:0/2246098802 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2140a5b90 con 0x7fb2241593d0 2024-03-29T08:20:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.203+0000 7fb222000640 1 -- 172.21.15.46:0/2246098802 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2140ad440 con 0x7fb2241593d0 2024-03-29T08:20:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb229a00640 1 -- 172.21.15.46:0/2246098802 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb2241593d0 msgr2=0x7fb22406cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb229a00640 1 --2- 172.21.15.46:0/2246098802 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb2241593d0 0x7fb22406cbf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb214009d90 tx=0x7fb21401c970 comp rx=0 tx=0).stop 2024-03-29T08:20:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb229a00640 1 -- 172.21.15.46:0/2246098802 shutdown_connections 2024-03-29T08:20:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb229a00640 1 --2- 172.21.15.46:0/2246098802 >> v2:172.21.15.73:3300/0 conn(0x7fb22415d230 0x7fb22415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb229a00640 1 --2- 172.21.15.46:0/2246098802 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb2241593d0 0x7fb22406cbf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb229a00640 1 -- 172.21.15.46:0/2246098802 >> 172.21.15.46:0/2246098802 conn(0x7fb224066110 msgr2=0x7fb224069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb229a00640 1 -- 172.21.15.46:0/2246098802 shutdown_connections 2024-03-29T08:20:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb229a00640 1 -- 172.21.15.46:0/2246098802 wait complete. 2024-03-29T08:20:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb229a00640 1 Processor -- start 2024-03-29T08:20:55.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb229a00640 1 -- start start 2024-03-29T08:20:55.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb229a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2241593d0 0x7fb2241481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb229a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb22415d230 0x7fb22414ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb229a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb22415dbd0 0x7fb22414f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb229a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240cc650 con 0x7fb2241593d0 2024-03-29T08:20:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb229a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb224148700 con 0x7fb22415d230 2024-03-29T08:20:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb229a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240c8570 con 0x7fb22415dbd0 2024-03-29T08:20:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb222a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2241593d0 0x7fb2241481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb222a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2241593d0 0x7fb2241481c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38636/0 (socket says 172.21.15.46:38636) 2024-03-29T08:20:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb222a00640 1 -- 172.21.15.46:0/930840123 learned_addr learned my addr 172.21.15.46:0/930840123 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb21a000640 1 --2- 172.21.15.46:0/930840123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb22415d230 0x7fb22414ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb223400640 1 --2- 172.21.15.46:0/930840123 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb22415dbd0 0x7fb22414f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb222a00640 1 -- 172.21.15.46:0/930840123 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb22415dbd0 msgr2=0x7fb22414f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb222a00640 1 --2- 172.21.15.46:0/930840123 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb22415dbd0 0x7fb22414f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb222a00640 1 -- 172.21.15.46:0/930840123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb22415d230 msgr2=0x7fb22414ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:55.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb222a00640 1 --2- 172.21.15.46:0/930840123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb22415d230 0x7fb22414ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.207+0000 7fb222a00640 1 -- 172.21.15.46:0/930840123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c0094d0 con 0x7fb2241593d0 2024-03-29T08:20:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.211+0000 7fb222a00640 1 --2- 172.21.15.46:0/930840123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2241593d0 0x7fb2241481c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb21401a5a0 tx=0x7fb214019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:55.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.211+0000 7fb21b400640 1 -- 172.21.15.46:0/930840123 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb214009450 con 0x7fb2241593d0 2024-03-29T08:20:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.211+0000 7fb21b400640 1 -- 172.21.15.46:0/930840123 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2140adbf0 con 0x7fb2241593d0 2024-03-29T08:20:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.211+0000 7fb21b400640 1 -- 172.21.15.46:0/930840123 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2140b8420 con 0x7fb2241593d0 2024-03-29T08:20:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.211+0000 7fb229a00640 1 -- 172.21.15.46:0/930840123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb214009a70 con 0x7fb2241593d0 2024-03-29T08:20:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.211+0000 7fb229a00640 1 -- 172.21.15.46:0/930840123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2240c8b30 con 0x7fb2241593d0 2024-03-29T08:20:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.211+0000 7fb229a00640 1 -- 172.21.15.46:0/930840123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2240cc3b0 con 0x7fb2241593d0 2024-03-29T08:20:55.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.211+0000 7fb21b400640 1 -- 172.21.15.46:0/930840123 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb2140b8580 con 0x7fb2241593d0 2024-03-29T08:20:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.211+0000 7fb21b400640 1 --2- 172.21.15.46:0/930840123 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb200066690 0x7fb200068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.211+0000 7fb21b400640 1 -- 172.21.15.46:0/930840123 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb21411f530 con 0x7fb2241593d0 2024-03-29T08:20:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.211+0000 7fb21a000640 1 --2- 172.21.15.46:0/930840123 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb200066690 0x7fb200068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:55.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.215+0000 7fb21a000640 1 --2- 172.21.15.46:0/930840123 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb200066690 0x7fb200068b50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb22414a6c0 tx=0x7fb20c009240 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:55.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.215+0000 7fb21b400640 1 -- 172.21.15.46:0/930840123 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb2240cc3b0 con 0x7fb2241593d0 2024-03-29T08:20:55.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.411+0000 7fb229a00640 1 -- 172.21.15.46:0/930840123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2240cc3b0 con 0x7fb2241593d0 2024-03-29T08:20:55.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.411+0000 7fb21b400640 1 -- 172.21.15.46:0/930840123 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4731 (secure 0 0 0) 0x7fb2240cc3b0 con 0x7fb2241593d0 2024-03-29T08:20:55.413 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:55.413 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2702541693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2702541693},{"type":"v1","addr":"172.21.15.73:6837","nonce":2702541693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2507803771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2507803771},{"type":"v1","addr":"172.21.15.73:6835","nonce":2507803771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/494932499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":494932499},{"type":"v1","addr":"172.21.15.46:6833","nonce":494932499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:20:12.400114+0000","modified":"2024-03-29T08:20:15.405861+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3467089652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3467089652},{"type":"v1","addr":"172.21.15.46:6835","nonce":3467089652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-29T08:20:55.413 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-03-29T08:20:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.411+0000 7fb229a00640 1 -- 172.21.15.46:0/930840123 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb200066690 msgr2=0x7fb200068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.411+0000 7fb229a00640 1 --2- 172.21.15.46:0/930840123 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb200066690 0x7fb200068b50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb22414a6c0 tx=0x7fb20c009240 comp rx=0 tx=0).stop 2024-03-29T08:20:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.411+0000 7fb229a00640 1 -- 172.21.15.46:0/930840123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2241593d0 msgr2=0x7fb2241481c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.411+0000 7fb229a00640 1 --2- 172.21.15.46:0/930840123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2241593d0 0x7fb2241481c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb21401a5a0 tx=0x7fb214019710 comp rx=0 tx=0).stop 2024-03-29T08:20:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.411+0000 7fb229a00640 1 -- 172.21.15.46:0/930840123 shutdown_connections 2024-03-29T08:20:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.411+0000 7fb229a00640 1 --2- 172.21.15.46:0/930840123 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb200066690 0x7fb200068b50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.411+0000 7fb229a00640 1 --2- 172.21.15.46:0/930840123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb22415d230 0x7fb22414ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.411+0000 7fb229a00640 1 --2- 172.21.15.46:0/930840123 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb22415dbd0 0x7fb22414f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.411+0000 7fb229a00640 1 --2- 172.21.15.46:0/930840123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2241593d0 0x7fb2241481c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.411+0000 7fb229a00640 1 -- 172.21.15.46:0/930840123 >> 172.21.15.46:0/930840123 conn(0x7fb224066110 msgr2=0x7fb2240b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.411+0000 7fb229a00640 1 -- 172.21.15.46:0/930840123 shutdown_connections 2024-03-29T08:20:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.411+0000 7fb229a00640 1 -- 172.21.15.46:0/930840123 wait complete. 2024-03-29T08:20:55.435 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:20:55.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.571+0000 7f6610000640 1 Processor -- start 2024-03-29T08:20:55.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.571+0000 7f6610000640 1 -- start start 2024-03-29T08:20:55.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.575+0000 7f6610000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6608153d50 0x7f66081611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:55.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.575+0000 7f6610000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f660815d870 con 0x7f6608153970 2024-03-29T08:20:55.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.575+0000 7f6610000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f660815db50 con 0x7f66080ad3e0 2024-03-29T08:20:55.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.575+0000 7f6610000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f660815de30 con 0x7f6608153d50 2024-03-29T08:20:55.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.575+0000 7f660cc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6608153d50 0x7f66081611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:55.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.575+0000 7f660d600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6608153970 0x7f660815cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48342/0 (socket says 172.21.15.46:48342) 2024-03-29T08:20:55.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.575+0000 7f660d600640 1 -- 172.21.15.46:0/2789273951 learned_addr learned my addr 172.21.15.46:0/2789273951 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:55.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.575+0000 7f660cc00640 1 -- 172.21.15.46:0/2789273951 >> v1:172.21.15.73:6789/0 conn(0x7f66080ad3e0 legacy=0x7f66080ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:55.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.575+0000 7f660cc00640 1 -- 172.21.15.46:0/2789273951 >> v1:172.21.15.46:6789/0 conn(0x7f6608153970 legacy=0x7f660815cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:55.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.575+0000 7f660cc00640 1 -- 172.21.15.46:0/2789273951 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66080abea0 con 0x7f6608153d50 2024-03-29T08:20:55.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.575+0000 7f660cc00640 1 --2- 172.21.15.46:0/2789273951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6608153d50 0x7f66081611e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f65f40097c0 tx=0x7f65f401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dccd7e6bfb1b4232 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:55.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6607e00640 1 -- 172.21.15.46:0/2789273951 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f40ac410 con 0x7f6608153d50 2024-03-29T08:20:55.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6607e00640 1 -- 172.21.15.46:0/2789273951 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f40ac570 con 0x7f6608153d50 2024-03-29T08:20:55.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6607e00640 1 -- 172.21.15.46:0/2789273951 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f40ac900 con 0x7f6608153d50 2024-03-29T08:20:55.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6610000640 1 -- 172.21.15.46:0/2789273951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6608153d50 msgr2=0x7f66081611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:55.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6610000640 1 --2- 172.21.15.46:0/2789273951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6608153d50 0x7f66081611e0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f65f40097c0 tx=0x7f65f401c4a0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6610000640 1 -- 172.21.15.46:0/2789273951 shutdown_connections 2024-03-29T08:20:55.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6610000640 1 --2- 172.21.15.46:0/2789273951 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6608153d50 0x7f66081611e0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6610000640 1 -- 172.21.15.46:0/2789273951 >> 172.21.15.46:0/2789273951 conn(0x7f6608066110 msgr2=0x7f6608069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:55.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6610000640 1 -- 172.21.15.46:0/2789273951 shutdown_connections 2024-03-29T08:20:55.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6610000640 1 -- 172.21.15.46:0/2789273951 wait complete. 2024-03-29T08:20:55.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6610000640 1 Processor -- start 2024-03-29T08:20:55.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6610000640 1 -- start start 2024-03-29T08:20:55.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6610000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66080ad3e0 0x7f66080ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:55.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6610000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6608153970 0x7f6608147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:55.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6610000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f660814a550 0x7f66080b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:55.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6610000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66080cbca0 con 0x7f66080ad3e0 2024-03-29T08:20:55.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6610000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6608153580 con 0x7f6608153970 2024-03-29T08:20:55.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f6610000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66080cb810 con 0x7f660814a550 2024-03-29T08:20:55.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f660d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66080ad3e0 0x7f66080ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:55.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f660d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66080ad3e0 0x7f66080ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38654/0 (socket says 172.21.15.46:38654) 2024-03-29T08:20:55.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f660d600640 1 -- 172.21.15.46:0/2509876574 learned_addr learned my addr 172.21.15.46:0/2509876574 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:55.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f660cc00640 1 --2- 172.21.15.46:0/2509876574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6608153970 0x7f6608147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:55.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.579+0000 7f660e000640 1 --2- 172.21.15.46:0/2509876574 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f660814a550 0x7f66080b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:55.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.583+0000 7f660d600640 1 -- 172.21.15.46:0/2509876574 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f660814a550 msgr2=0x7f66080b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:55.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.583+0000 7f660d600640 1 --2- 172.21.15.46:0/2509876574 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f660814a550 0x7f66080b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.583+0000 7f660d600640 1 -- 172.21.15.46:0/2509876574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6608153970 msgr2=0x7f6608147d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:55.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.583+0000 7f660d600640 1 --2- 172.21.15.46:0/2509876574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6608153970 0x7f6608147d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.583+0000 7f660d600640 1 -- 172.21.15.46:0/2509876574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f4009420 con 0x7f66080ad3e0 2024-03-29T08:20:55.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.583+0000 7f660d600640 1 --2- 172.21.15.46:0/2509876574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66080ad3e0 0x7f66080ca710 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00c500 tx=0x7f65fc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:55.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.583+0000 7f6605600640 1 -- 172.21.15.46:0/2509876574 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc005cf0 con 0x7f66080ad3e0 2024-03-29T08:20:55.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.583+0000 7f6605600640 1 -- 172.21.15.46:0/2509876574 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc005e50 con 0x7f66080ad3e0 2024-03-29T08:20:55.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.583+0000 7f6605600640 1 -- 172.21.15.46:0/2509876574 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc0072f0 con 0x7f66080ad3e0 2024-03-29T08:20:55.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.583+0000 7f6610000640 1 -- 172.21.15.46:0/2509876574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66080cac50 con 0x7f66080ad3e0 2024-03-29T08:20:55.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.583+0000 7f6610000640 1 -- 172.21.15.46:0/2509876574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66080cb1c0 con 0x7f66080ad3e0 2024-03-29T08:20:55.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.583+0000 7f6610000640 1 -- 172.21.15.46:0/2509876574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66080cb020 con 0x7f66080ad3e0 2024-03-29T08:20:55.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.583+0000 7f6605600640 1 -- 172.21.15.46:0/2509876574 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f65fc007450 con 0x7f66080ad3e0 2024-03-29T08:20:55.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.583+0000 7f6605600640 1 --2- 172.21.15.46:0/2509876574 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f65e8066690 0x7f65e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:55.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.583+0000 7f6605600640 1 -- 172.21.15.46:0/2509876574 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65fc083620 con 0x7f66080ad3e0 2024-03-29T08:20:55.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.587+0000 7f660cc00640 1 --2- 172.21.15.46:0/2509876574 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f65e8066690 0x7f65e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:55.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.587+0000 7f660cc00640 1 --2- 172.21.15.46:0/2509876574 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f65e8066690 0x7f65e8068b50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f65f40ae000 tx=0x7f65f4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:55.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.587+0000 7f6605600640 1 -- 172.21.15.46:0/2509876574 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f66080cb020 con 0x7f66080ad3e0 2024-03-29T08:20:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.751+0000 7f6610000640 1 -- 172.21.15.46:0/2509876574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66080cb020 con 0x7f66080ad3e0 2024-03-29T08:20:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.751+0000 7f6605600640 1 -- 172.21.15.46:0/2509876574 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+15993 (secure 0 0 0) 0x7f66080cb020 con 0x7f66080ad3e0 2024-03-29T08:20:55.754 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:55.754 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":49,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:20:13.402206+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:20:10.906619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-29T08:20:11.389667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:20:55.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.755+0000 7f6610000640 1 -- 172.21.15.46:0/2509876574 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f65e8066690 msgr2=0x7f65e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:55.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.755+0000 7f6610000640 1 --2- 172.21.15.46:0/2509876574 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f65e8066690 0x7f65e8068b50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f65f40ae000 tx=0x7f65f4008040 comp rx=0 tx=0).stop 2024-03-29T08:20:55.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.755+0000 7f6610000640 1 -- 172.21.15.46:0/2509876574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66080ad3e0 msgr2=0x7f66080ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:55.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.755+0000 7f6610000640 1 --2- 172.21.15.46:0/2509876574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66080ad3e0 0x7f66080ca710 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f65fc00c500 tx=0x7f65fc00c9c0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.755+0000 7f6610000640 1 -- 172.21.15.46:0/2509876574 shutdown_connections 2024-03-29T08:20:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.755+0000 7f6610000640 1 --2- 172.21.15.46:0/2509876574 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f65e8066690 0x7f65e8068b50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.755+0000 7f6610000640 1 --2- 172.21.15.46:0/2509876574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6608153970 0x7f6608147d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.755+0000 7f6610000640 1 --2- 172.21.15.46:0/2509876574 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f660814a550 0x7f66080b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.755+0000 7f6610000640 1 --2- 172.21.15.46:0/2509876574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66080ad3e0 0x7f66080ca710 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.755+0000 7f6610000640 1 -- 172.21.15.46:0/2509876574 >> 172.21.15.46:0/2509876574 conn(0x7f6608066110 msgr2=0x7f660815eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.755+0000 7f6610000640 1 -- 172.21.15.46:0/2509876574 shutdown_connections 2024-03-29T08:20:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.755+0000 7f6610000640 1 -- 172.21.15.46:0/2509876574 wait complete. 2024-03-29T08:20:55.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:20:10.906619+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:20:11.389667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:20:55.776 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:20:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.915+0000 7fb53d800640 1 Processor -- start 2024-03-29T08:20:55.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.915+0000 7fb53d800640 1 -- start start 2024-03-29T08:20:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.919+0000 7fb53d800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb538142a50 0x7fb5380cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.919+0000 7fb53d800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb5380ae990 0x7fb5380cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.919+0000 7fb53d800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb53815d260 con 0x7fb53815df90 2024-03-29T08:20:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.919+0000 7fb53d800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5380aefd0 con 0x7fb5380ae990 2024-03-29T08:20:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.919+0000 7fb53d800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53815d4b0 con 0x7fb538142a50 2024-03-29T08:20:55.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.919+0000 7fb537400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb5380ae990 0x7fb5380cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.919+0000 7fb537400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb5380ae990 0x7fb5380cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38658/0 (socket says 172.21.15.46:38658) 2024-03-29T08:20:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.919+0000 7fb537400640 1 -- 172.21.15.46:0/1784965733 learned_addr learned my addr 172.21.15.46:0/1784965733 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.919+0000 7fb536a00640 1 --2- 172.21.15.46:0/1784965733 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb538142a50 0x7fb5380cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.919+0000 7fb537400640 1 -- 172.21.15.46:0/1784965733 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb538142a50 msgr2=0x7fb5380cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.919+0000 7fb537400640 1 --2- 172.21.15.46:0/1784965733 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb538142a50 0x7fb5380cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.919+0000 7fb537400640 1 -- 172.21.15.46:0/1784965733 >> v1:172.21.15.73:6789/0 conn(0x7fb53815df90 legacy=0x7fb5380ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.919+0000 7fb537400640 1 -- 172.21.15.46:0/1784965733 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb53815d790 con 0x7fb5380ae990 2024-03-29T08:20:55.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.919+0000 7fb536a00640 1 --2- 172.21.15.46:0/1784965733 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb538142a50 0x7fb5380cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:20:55.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb537400640 1 --2- 172.21.15.46:0/1784965733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5380ae990 0x7fb5380cbcd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb530009e50 tx=0x7fb53001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900167285833ea77 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb536000640 1 -- 172.21.15.46:0/1784965733 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300a5660 con 0x7fb5380ae990 2024-03-29T08:20:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb536000640 1 -- 172.21.15.46:0/1784965733 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5300a57c0 con 0x7fb5380ae990 2024-03-29T08:20:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb536000640 1 -- 172.21.15.46:0/1784965733 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300a5b50 con 0x7fb5380ae990 2024-03-29T08:20:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb53d800640 1 -- 172.21.15.46:0/1784965733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5380ae990 msgr2=0x7fb5380cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb53d800640 1 --2- 172.21.15.46:0/1784965733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5380ae990 0x7fb5380cbcd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb530009e50 tx=0x7fb53001c9a0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb53d800640 1 -- 172.21.15.46:0/1784965733 shutdown_connections 2024-03-29T08:20:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb53d800640 1 --2- 172.21.15.46:0/1784965733 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5380ae990 0x7fb5380cbcd0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb53d800640 1 --2- 172.21.15.46:0/1784965733 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb538142a50 0x7fb5380cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb53d800640 1 -- 172.21.15.46:0/1784965733 >> 172.21.15.46:0/1784965733 conn(0x7fb53806a390 msgr2=0x7fb5380cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:55.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb53d800640 1 -- 172.21.15.46:0/1784965733 shutdown_connections 2024-03-29T08:20:55.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb53d800640 1 -- 172.21.15.46:0/1784965733 wait complete. 2024-03-29T08:20:55.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb53d800640 1 Processor -- start 2024-03-29T08:20:55.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb53d800640 1 -- start start 2024-03-29T08:20:55.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb53d800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5380ae990 0x7fb5381f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:55.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb53d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb538142a50 0x7fb5380ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:55.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb53d800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53815df90 0x7fb5380abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:55.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb53d800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5381f8a60 con 0x7fb538142a50 2024-03-29T08:20:55.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb53d800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53815c820 con 0x7fb53815df90 2024-03-29T08:20:55.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.923+0000 7fb53d800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb53815a820 con 0x7fb5380ae990 2024-03-29T08:20:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb536a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5380ae990 0x7fb5381f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb52fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb538142a50 0x7fb5380ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb536a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5380ae990 0x7fb5381f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:35332/0 (socket says 172.21.15.46:35332) 2024-03-29T08:20:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb536a00640 1 -- 172.21.15.46:0/3272212504 learned_addr learned my addr 172.21.15.46:0/3272212504 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:55.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb537400640 1 --2- 172.21.15.46:0/3272212504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53815df90 0x7fb5380abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb536a00640 1 -- 172.21.15.46:0/3272212504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53815df90 msgr2=0x7fb5380abdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb536a00640 1 --2- 172.21.15.46:0/3272212504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53815df90 0x7fb5380abdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb536a00640 1 -- 172.21.15.46:0/3272212504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb538142a50 msgr2=0x7fb5380ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb536a00640 1 --2- 172.21.15.46:0/3272212504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb538142a50 0x7fb5380ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb536a00640 1 -- 172.21.15.46:0/3272212504 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb530009b30 con 0x7fb5380ae990 2024-03-29T08:20:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb52fe00640 1 --2- 172.21.15.46:0/3272212504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb538142a50 0x7fb5380ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:20:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb536a00640 1 --2- 172.21.15.46:0/3272212504 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5380ae990 0x7fb5381f56c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb52800e840 tx=0x7fb52800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:55.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb52ea00640 1 -- 172.21.15.46:0/3272212504 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52800c8d0 con 0x7fb5380ae990 2024-03-29T08:20:55.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb52ea00640 1 -- 172.21.15.46:0/3272212504 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52800ca30 con 0x7fb5380ae990 2024-03-29T08:20:55.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb53d800640 1 -- 172.21.15.46:0/3272212504 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb53815ab00 con 0x7fb5380ae990 2024-03-29T08:20:55.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb52ea00640 1 -- 172.21.15.46:0/3272212504 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52800cde0 con 0x7fb5380ae990 2024-03-29T08:20:55.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb53d800640 1 -- 172.21.15.46:0/3272212504 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb53815b0c0 con 0x7fb5380ae990 2024-03-29T08:20:55.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb53d800640 1 -- 172.21.15.46:0/3272212504 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb53815b2e0 con 0x7fb5380ae990 2024-03-29T08:20:55.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.927+0000 7fb52ea00640 1 -- 172.21.15.46:0/3272212504 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb528026480 con 0x7fb5380ae990 2024-03-29T08:20:55.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.931+0000 7fb52ea00640 1 --2- 172.21.15.46:0/3272212504 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb514066690 0x7fb514068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:55.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.931+0000 7fb52ea00640 1 -- 172.21.15.46:0/3272212504 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5280843b0 con 0x7fb5380ae990 2024-03-29T08:20:55.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.931+0000 7fb52fe00640 1 --2- 172.21.15.46:0/3272212504 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb514066690 0x7fb514068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:55.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.931+0000 7fb52fe00640 1 --2- 172.21.15.46:0/3272212504 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb514066690 0x7fb514068b50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb5380cadf0 tx=0x7fb520009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:55.935+0000 7fb52ea00640 1 -- 172.21.15.46:0/3272212504 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb528051840 con 0x7fb5380ae990 2024-03-29T08:20:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.103+0000 7fb53d800640 1 -- 172.21.15.46:0/3272212504 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb538071a70 con 0x7fb5380ae990 2024-03-29T08:20:56.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.103+0000 7fb52ea00640 1 -- 172.21.15.46:0/3272212504 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb528054e60 con 0x7fb5380ae990 2024-03-29T08:20:56.107 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:20:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.107+0000 7fb53d800640 1 -- 172.21.15.46:0/3272212504 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb514066690 msgr2=0x7fb514068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.107+0000 7fb53d800640 1 --2- 172.21.15.46:0/3272212504 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb514066690 0x7fb514068b50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb5380cadf0 tx=0x7fb520009040 comp rx=0 tx=0).stop 2024-03-29T08:20:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.107+0000 7fb53d800640 1 -- 172.21.15.46:0/3272212504 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5380ae990 msgr2=0x7fb5381f56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.107+0000 7fb53d800640 1 --2- 172.21.15.46:0/3272212504 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5380ae990 0x7fb5381f56c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb52800e840 tx=0x7fb52800ed00 comp rx=0 tx=0).stop 2024-03-29T08:20:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.107+0000 7fb53d800640 1 -- 172.21.15.46:0/3272212504 shutdown_connections 2024-03-29T08:20:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.107+0000 7fb53d800640 1 --2- 172.21.15.46:0/3272212504 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb514066690 0x7fb514068b50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.107+0000 7fb53d800640 1 --2- 172.21.15.46:0/3272212504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb53815df90 0x7fb5380abdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.107+0000 7fb53d800640 1 --2- 172.21.15.46:0/3272212504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb538142a50 0x7fb5380ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.107+0000 7fb53d800640 1 --2- 172.21.15.46:0/3272212504 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5380ae990 0x7fb5381f56c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.107+0000 7fb53d800640 1 -- 172.21.15.46:0/3272212504 >> 172.21.15.46:0/3272212504 conn(0x7fb53806a390 msgr2=0x7fb5380cdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.107+0000 7fb53d800640 1 -- 172.21.15.46:0/3272212504 shutdown_connections 2024-03-29T08:20:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.107+0000 7fb53d800640 1 -- 172.21.15.46:0/3272212504 wait complete. 2024-03-29T08:20:56.128 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:20:56.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.271+0000 7f0e9ae00640 1 Processor -- start 2024-03-29T08:20:56.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.271+0000 7f0e9ae00640 1 -- start start 2024-03-29T08:20:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.271+0000 7f0e9ae00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e940d0dc0 0x7f0e940cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.271+0000 7f0e9ae00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0e940cf820 0x7f0e940b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.271+0000 7f0e9ae00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e9415cf40 con 0x7f0e9415cb60 2024-03-29T08:20:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.271+0000 7f0e9ae00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e940b0250 con 0x7f0e940cf820 2024-03-29T08:20:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.271+0000 7f0e9ae00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e940b0810 con 0x7f0e940d0dc0 2024-03-29T08:20:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.271+0000 7f0e98e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0e940cf820 0x7f0e940b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.271+0000 7f0e98e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0e940cf820 0x7f0e940b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38670/0 (socket says 172.21.15.46:38670) 2024-03-29T08:20:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.271+0000 7f0e98e00640 1 -- 172.21.15.46:0/2643015322 learned_addr learned my addr 172.21.15.46:0/2643015322 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.271+0000 7f0e8b400640 1 --2- 172.21.15.46:0/2643015322 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e940d0dc0 0x7f0e940cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.271+0000 7f0e98e00640 1 -- 172.21.15.46:0/2643015322 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e940d0dc0 msgr2=0x7f0e940cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:56.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.271+0000 7f0e98e00640 1 --2- 172.21.15.46:0/2643015322 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e940d0dc0 0x7f0e940cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.271+0000 7f0e98e00640 1 -- 172.21.15.46:0/2643015322 >> v1:172.21.15.73:6789/0 conn(0x7f0e9415cb60 legacy=0x7f0e940d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.271+0000 7f0e98e00640 1 -- 172.21.15.46:0/2643015322 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e940d0010 con 0x7f0e940cf820 2024-03-29T08:20:56.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.275+0000 7f0e98e00640 1 --2- 172.21.15.46:0/2643015322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e940cf820 0x7f0e940b0ca0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009870 tx=0x7f0e8c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99c42831d37d37 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.275+0000 7f0e93400640 1 -- 172.21.15.46:0/2643015322 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c0a95f0 con 0x7f0e940cf820 2024-03-29T08:20:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.275+0000 7f0e93400640 1 -- 172.21.15.46:0/2643015322 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c0a9750 con 0x7f0e940cf820 2024-03-29T08:20:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.275+0000 7f0e93400640 1 -- 172.21.15.46:0/2643015322 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c0a9ae0 con 0x7f0e940cf820 2024-03-29T08:20:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.275+0000 7f0e9ae00640 1 -- 172.21.15.46:0/2643015322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e940cf820 msgr2=0x7f0e940b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.275+0000 7f0e9ae00640 1 --2- 172.21.15.46:0/2643015322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e940cf820 0x7f0e940b0ca0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009870 tx=0x7f0e8c01c530 comp rx=0 tx=0).stop 2024-03-29T08:20:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.275+0000 7f0e9ae00640 1 -- 172.21.15.46:0/2643015322 shutdown_connections 2024-03-29T08:20:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.275+0000 7f0e9ae00640 1 --2- 172.21.15.46:0/2643015322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e940cf820 0x7f0e940b0ca0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.275+0000 7f0e9ae00640 1 --2- 172.21.15.46:0/2643015322 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e940d0dc0 0x7f0e940cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.275+0000 7f0e9ae00640 1 -- 172.21.15.46:0/2643015322 >> 172.21.15.46:0/2643015322 conn(0x7f0e94099a10 msgr2=0x7f0e94089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.275+0000 7f0e9ae00640 1 -- 172.21.15.46:0/2643015322 shutdown_connections 2024-03-29T08:20:56.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.275+0000 7f0e9ae00640 1 -- 172.21.15.46:0/2643015322 wait complete. 2024-03-29T08:20:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e9ae00640 1 Processor -- start 2024-03-29T08:20:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e9ae00640 1 -- start start 2024-03-29T08:20:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e9ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e940cf820 0x7f0e94153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e9ae00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e940d0dc0 0x7f0e9414b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e9ae00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e9415cb60 0x7f0e9414bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e9ae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e94153580 con 0x7f0e940cf820 2024-03-29T08:20:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e9ae00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e941543c0 con 0x7f0e9415cb60 2024-03-29T08:20:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e9ae00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e941537b0 con 0x7f0e940d0dc0 2024-03-29T08:20:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e940cf820 0x7f0e94153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e940cf820 0x7f0e94153040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38684/0 (socket says 172.21.15.46:38684) 2024-03-29T08:20:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e93e00640 1 -- 172.21.15.46:0/1362709272 learned_addr learned my addr 172.21.15.46:0/1362709272 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:56.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e98e00640 1 --2- 172.21.15.46:0/1362709272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e9415cb60 0x7f0e9414bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e8b400640 1 --2- 172.21.15.46:0/1362709272 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e940d0dc0 0x7f0e9414b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e93e00640 1 -- 172.21.15.46:0/1362709272 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e940d0dc0 msgr2=0x7f0e9414b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e93e00640 1 --2- 172.21.15.46:0/1362709272 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e940d0dc0 0x7f0e9414b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e93e00640 1 -- 172.21.15.46:0/1362709272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e9415cb60 msgr2=0x7f0e9414bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e93e00640 1 --2- 172.21.15.46:0/1362709272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e9415cb60 0x7f0e9414bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e93e00640 1 -- 172.21.15.46:0/1362709272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8c0094d0 con 0x7f0e940cf820 2024-03-29T08:20:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e93e00640 1 --2- 172.21.15.46:0/1362709272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e940cf820 0x7f0e94153040 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400c7f0 tx=0x7f0e8400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e90c00640 1 -- 172.21.15.46:0/1362709272 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e84003b80 con 0x7f0e940cf820 2024-03-29T08:20:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e90c00640 1 -- 172.21.15.46:0/1362709272 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e84003ce0 con 0x7f0e940cf820 2024-03-29T08:20:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e9ae00640 1 -- 172.21.15.46:0/1362709272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e94153a90 con 0x7f0e940cf820 2024-03-29T08:20:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e90c00640 1 -- 172.21.15.46:0/1362709272 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8400aa90 con 0x7f0e940cf820 2024-03-29T08:20:56.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.279+0000 7f0e9ae00640 1 -- 172.21.15.46:0/1362709272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e94154050 con 0x7f0e940cf820 2024-03-29T08:20:56.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.283+0000 7f0e9ae00640 1 -- 172.21.15.46:0/1362709272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9409e7d0 con 0x7f0e940cf820 2024-03-29T08:20:56.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.283+0000 7f0e90c00640 1 -- 172.21.15.46:0/1362709272 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0e8400abf0 con 0x7f0e940cf820 2024-03-29T08:20:56.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.283+0000 7f0e90c00640 1 --2- 172.21.15.46:0/1362709272 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e70066710 0x7f0e70068bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.283+0000 7f0e90c00640 1 -- 172.21.15.46:0/1362709272 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f0e840840d0 con 0x7f0e940cf820 2024-03-29T08:20:56.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.283+0000 7f0e8b400640 1 --2- 172.21.15.46:0/1362709272 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e70066710 0x7f0e70068bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.283+0000 7f0e8b400640 1 --2- 172.21.15.46:0/1362709272 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e70066710 0x7f0e70068bd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0e94159860 tx=0x7f0e7c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:56.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.287+0000 7f0e90c00640 1 -- 172.21.15.46:0/1362709272 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0e84051390 con 0x7f0e940cf820 2024-03-29T08:20:56.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.451+0000 7f0e9ae00640 1 -- 172.21.15.46:0/1362709272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0e9404e120 con 0x7f0e940cf820 2024-03-29T08:20:56.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.451+0000 7f0e90c00640 1 -- 172.21.15.46:0/1362709272 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v50) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0e840549b0 con 0x7f0e940cf820 2024-03-29T08:20:56.453 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:20:56.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.451+0000 7f0e9ae00640 1 -- 172.21.15.46:0/1362709272 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e70066710 msgr2=0x7f0e70068bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.451+0000 7f0e9ae00640 1 --2- 172.21.15.46:0/1362709272 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e70066710 0x7f0e70068bd0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0e94159860 tx=0x7f0e7c008040 comp rx=0 tx=0).stop 2024-03-29T08:20:56.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.451+0000 7f0e9ae00640 1 -- 172.21.15.46:0/1362709272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e940cf820 msgr2=0x7f0e94153040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.451+0000 7f0e9ae00640 1 --2- 172.21.15.46:0/1362709272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e940cf820 0x7f0e94153040 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0e8400c7f0 tx=0x7f0e8400ccb0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.451+0000 7f0e9ae00640 1 -- 172.21.15.46:0/1362709272 shutdown_connections 2024-03-29T08:20:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.451+0000 7f0e9ae00640 1 --2- 172.21.15.46:0/1362709272 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e70066710 0x7f0e70068bd0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.451+0000 7f0e9ae00640 1 --2- 172.21.15.46:0/1362709272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e9415cb60 0x7f0e9414bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.451+0000 7f0e9ae00640 1 --2- 172.21.15.46:0/1362709272 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e940d0dc0 0x7f0e9414b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.451+0000 7f0e9ae00640 1 --2- 172.21.15.46:0/1362709272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e940cf820 0x7f0e94153040 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.451+0000 7f0e9ae00640 1 -- 172.21.15.46:0/1362709272 >> 172.21.15.46:0/1362709272 conn(0x7f0e94099a10 msgr2=0x7f0e940b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.451+0000 7f0e9ae00640 1 -- 172.21.15.46:0/1362709272 shutdown_connections 2024-03-29T08:20:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.451+0000 7f0e9ae00640 1 -- 172.21.15.46:0/1362709272 wait complete. 2024-03-29T08:20:56.473 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:20:56.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.615+0000 7faac7800640 1 Processor -- start 2024-03-29T08:20:56.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.615+0000 7faac7800640 1 -- start start 2024-03-29T08:20:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.615+0000 7faac7800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faac0142a50 0x7faac00cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.615+0000 7faac7800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faac015df90 0x7faac00cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.615+0000 7faac7800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac015d260 con 0x7faac00ae280 2024-03-29T08:20:56.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.615+0000 7faac7800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac00aefd0 con 0x7faac0142a50 2024-03-29T08:20:56.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.615+0000 7faac7800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac015d4b0 con 0x7faac015df90 2024-03-29T08:20:56.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.615+0000 7faac4e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faac0142a50 0x7faac00cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.615+0000 7faac4e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faac0142a50 0x7faac00cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45680/0 (socket says 172.21.15.46:45680) 2024-03-29T08:20:56.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.615+0000 7faac4e00640 1 -- 172.21.15.46:0/3538608092 learned_addr learned my addr 172.21.15.46:0/3538608092 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:56.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.615+0000 7faab7e00640 1 --2- 172.21.15.46:0/3538608092 >> v2:172.21.15.73:3300/0 conn(0x7faac015df90 0x7faac00cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.615+0000 7faac4e00640 1 -- 172.21.15.46:0/3538608092 >> v2:172.21.15.73:3300/0 conn(0x7faac015df90 msgr2=0x7faac00cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:56.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.615+0000 7faac4e00640 1 --2- 172.21.15.46:0/3538608092 >> v2:172.21.15.73:3300/0 conn(0x7faac015df90 0x7faac00cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.615+0000 7faac4e00640 1 -- 172.21.15.46:0/3538608092 >> v1:172.21.15.46:6789/0 conn(0x7faac00ae280 legacy=0x7faac00ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.615+0000 7faac4e00640 1 -- 172.21.15.46:0/3538608092 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac015d790 con 0x7faac0142a50 2024-03-29T08:20:56.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.615+0000 7faabfe00640 1 -- 172.21.15.46:0/3538608092 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916473425 0 0) 0x7faac015d260 con 0x7faac00ae280 2024-03-29T08:20:56.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.619+0000 7faac4e00640 1 --2- 172.21.15.46:0/3538608092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac0142a50 0x7faac00cb790 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7faac006e480 tx=0x7faab001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19ff4ffb29e53be server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.619+0000 7faabfe00640 1 -- 172.21.15.46:0/3538608092 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab00ab600 con 0x7faac0142a50 2024-03-29T08:20:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.619+0000 7faabfe00640 1 -- 172.21.15.46:0/3538608092 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab00a7050 con 0x7faac0142a50 2024-03-29T08:20:56.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.619+0000 7faabfe00640 1 -- 172.21.15.46:0/3538608092 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab00ab980 con 0x7faac0142a50 2024-03-29T08:20:56.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.619+0000 7faac7800640 1 -- 172.21.15.46:0/3538608092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac0142a50 msgr2=0x7faac00cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.619+0000 7faac7800640 1 --2- 172.21.15.46:0/3538608092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac0142a50 0x7faac00cb790 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7faac006e480 tx=0x7faab001c490 comp rx=0 tx=0).stop 2024-03-29T08:20:56.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.619+0000 7faac7800640 1 -- 172.21.15.46:0/3538608092 shutdown_connections 2024-03-29T08:20:56.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.619+0000 7faac7800640 1 --2- 172.21.15.46:0/3538608092 >> v2:172.21.15.73:3300/0 conn(0x7faac015df90 0x7faac00cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.619+0000 7faac7800640 1 --2- 172.21.15.46:0/3538608092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac0142a50 0x7faac00cb790 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.619+0000 7faac7800640 1 -- 172.21.15.46:0/3538608092 >> 172.21.15.46:0/3538608092 conn(0x7faac006a390 msgr2=0x7faac00cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:56.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.619+0000 7faac7800640 1 -- 172.21.15.46:0/3538608092 shutdown_connections 2024-03-29T08:20:56.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.619+0000 7faac7800640 1 -- 172.21.15.46:0/3538608092 wait complete. 2024-03-29T08:20:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faac7800640 1 Processor -- start 2024-03-29T08:20:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faac7800640 1 -- start start 2024-03-29T08:20:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faac7800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac00ae280 0x7faac0159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faac7800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac0142a50 0x7faac015bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faac7800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faac015df90 0x7faac0138140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faac7800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac015b3a0 con 0x7faac0142a50 2024-03-29T08:20:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faac7800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac015cbc0 con 0x7faac00ae280 2024-03-29T08:20:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faac7800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac015c600 con 0x7faac015df90 2024-03-29T08:20:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faab7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac0142a50 0x7faac015bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faab7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac0142a50 0x7faac015bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45692/0 (socket says 172.21.15.46:45692) 2024-03-29T08:20:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faab7e00640 1 -- 172.21.15.46:0/4174664938 learned_addr learned my addr 172.21.15.46:0/4174664938 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faac4e00640 1 --2- 172.21.15.46:0/4174664938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac00ae280 0x7faac0159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faac5800640 1 --2- 172.21.15.46:0/4174664938 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faac015df90 0x7faac0138140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faab7e00640 1 -- 172.21.15.46:0/4174664938 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faac015df90 msgr2=0x7faac0138140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faab7e00640 1 --2- 172.21.15.46:0/4174664938 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faac015df90 0x7faac0138140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faab7e00640 1 -- 172.21.15.46:0/4174664938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac00ae280 msgr2=0x7faac0159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faab7e00640 1 --2- 172.21.15.46:0/4174664938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac00ae280 0x7faac0159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faab7e00640 1 -- 172.21.15.46:0/4174664938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab00094d0 con 0x7faac0142a50 2024-03-29T08:20:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faab7e00640 1 --2- 172.21.15.46:0/4174664938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac0142a50 0x7faac015bc00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7faaa800e870 tx=0x7faaa800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faabd600640 1 -- 172.21.15.46:0/4174664938 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8009510 con 0x7faac0142a50 2024-03-29T08:20:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faabd600640 1 -- 172.21.15.46:0/4174664938 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa800eec0 con 0x7faac0142a50 2024-03-29T08:20:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faac7800640 1 -- 172.21.15.46:0/4174664938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac015c8e0 con 0x7faac0142a50 2024-03-29T08:20:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faabd600640 1 -- 172.21.15.46:0/4174664938 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa800cb40 con 0x7faac0142a50 2024-03-29T08:20:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faac7800640 1 -- 172.21.15.46:0/4174664938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac0141760 con 0x7faac0142a50 2024-03-29T08:20:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.623+0000 7faac7800640 1 -- 172.21.15.46:0/4174664938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac006ad60 con 0x7faac0142a50 2024-03-29T08:20:56.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.627+0000 7faabd600640 1 -- 172.21.15.46:0/4174664938 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7faaa800cca0 con 0x7faac0142a50 2024-03-29T08:20:56.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.627+0000 7faabd600640 1 --2- 172.21.15.46:0/4174664938 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faa9c066690 0x7faa9c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.627+0000 7faabd600640 1 -- 172.21.15.46:0/4174664938 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7faaa80842f0 con 0x7faac0142a50 2024-03-29T08:20:56.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.627+0000 7faac4e00640 1 --2- 172.21.15.46:0/4174664938 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faa9c066690 0x7faa9c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.627+0000 7faac4e00640 1 --2- 172.21.15.46:0/4174664938 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faa9c066690 0x7faa9c068b50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faab00ad000 tx=0x7faab0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:56.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.631+0000 7faabd600640 1 -- 172.21.15.46:0/4174664938 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7faaa80515e0 con 0x7faac0142a50 2024-03-29T08:20:56.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.791+0000 7faac7800640 1 -- 172.21.15.46:0/4174664938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faac0071a70 con 0x7faac0142a50 2024-03-29T08:20:56.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.791+0000 7faabd600640 1 -- 172.21.15.46:0/4174664938 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v50) v1 ==== 100+0+11 (secure 0 0 0) 0x7faaa8054c00 con 0x7faac0142a50 2024-03-29T08:20:56.794 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:20:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.795+0000 7faac7800640 1 -- 172.21.15.46:0/4174664938 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faa9c066690 msgr2=0x7faa9c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.795+0000 7faac7800640 1 --2- 172.21.15.46:0/4174664938 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faa9c066690 0x7faa9c068b50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faab00ad000 tx=0x7faab0008040 comp rx=0 tx=0).stop 2024-03-29T08:20:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.795+0000 7faac7800640 1 -- 172.21.15.46:0/4174664938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac0142a50 msgr2=0x7faac015bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.795+0000 7faac7800640 1 --2- 172.21.15.46:0/4174664938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac0142a50 0x7faac015bc00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7faaa800e870 tx=0x7faaa800ec70 comp rx=0 tx=0).stop 2024-03-29T08:20:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.795+0000 7faac7800640 1 -- 172.21.15.46:0/4174664938 shutdown_connections 2024-03-29T08:20:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.795+0000 7faac7800640 1 --2- 172.21.15.46:0/4174664938 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faa9c066690 0x7faa9c068b50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.795+0000 7faac7800640 1 --2- 172.21.15.46:0/4174664938 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faac015df90 0x7faac0138140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.795+0000 7faac7800640 1 --2- 172.21.15.46:0/4174664938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faac0142a50 0x7faac015bc00 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.795+0000 7faac7800640 1 --2- 172.21.15.46:0/4174664938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac00ae280 0x7faac0159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.795+0000 7faac7800640 1 -- 172.21.15.46:0/4174664938 >> 172.21.15.46:0/4174664938 conn(0x7faac006a390 msgr2=0x7faac00b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:56.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.795+0000 7faac7800640 1 -- 172.21.15.46:0/4174664938 shutdown_connections 2024-03-29T08:20:56.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.795+0000 7faac7800640 1 -- 172.21.15.46:0/4174664938 wait complete. 2024-03-29T08:20:56.815 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:20:56.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.955+0000 7f9568e00640 1 Processor -- start 2024-03-29T08:20:56.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.955+0000 7f9568e00640 1 -- start start 2024-03-29T08:20:56.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.955+0000 7f9568e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9564159a20 0x7f95640cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.955+0000 7f9568e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95640cc720 0x7f95640c3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.955+0000 7f9568e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95640c3610 con 0x7f95640d0270 2024-03-29T08:20:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.955+0000 7f9568e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95640cb5a0 con 0x7f95640cc720 2024-03-29T08:20:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.955+0000 7f9568e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95640cdf70 con 0x7f9564159a20 2024-03-29T08:20:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.955+0000 7f9562800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95640cc720 0x7f95640c3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.955+0000 7f9559400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f95640d0270 0x7f95640cc010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37294/0 (socket says 172.21.15.46:37294) 2024-03-29T08:20:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.955+0000 7f9559400640 1 -- 172.21.15.46:0/2858873409 learned_addr learned my addr 172.21.15.46:0/2858873409 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.955+0000 7f9561e00640 1 --2- 172.21.15.46:0/2858873409 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9564159a20 0x7f95640cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.955+0000 7f9562800640 1 -- 172.21.15.46:0/2858873409 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9564159a20 msgr2=0x7f95640cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.955+0000 7f9562800640 1 --2- 172.21.15.46:0/2858873409 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9564159a20 0x7f95640cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.955+0000 7f9562800640 1 -- 172.21.15.46:0/2858873409 >> v1:172.21.15.46:6789/0 conn(0x7f95640d0270 legacy=0x7f95640cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.955+0000 7f9562800640 1 -- 172.21.15.46:0/2858873409 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95640ce250 con 0x7f95640cc720 2024-03-29T08:20:56.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.955+0000 7f9561400640 1 -- 172.21.15.46:0/2858873409 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 8305689 0 0) 0x7f95640c3610 con 0x7f95640d0270 2024-03-29T08:20:56.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.959+0000 7f9562800640 1 --2- 172.21.15.46:0/2858873409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95640cc720 0x7f95640c3aa0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f955c009d90 tx=0x7f955c01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b869c2ef6d5e288 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.959+0000 7f9561400640 1 -- 172.21.15.46:0/2858873409 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c0a5900 con 0x7f95640cc720 2024-03-29T08:20:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.959+0000 7f9561400640 1 -- 172.21.15.46:0/2858873409 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955c0a5a60 con 0x7f95640cc720 2024-03-29T08:20:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.959+0000 7f9561400640 1 -- 172.21.15.46:0/2858873409 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955c0a5df0 con 0x7f95640cc720 2024-03-29T08:20:56.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.959+0000 7f9568e00640 1 -- 172.21.15.46:0/2858873409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95640cc720 msgr2=0x7f95640c3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.959+0000 7f9568e00640 1 --2- 172.21.15.46:0/2858873409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95640cc720 0x7f95640c3aa0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f955c009d90 tx=0x7f955c01c8b0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.959+0000 7f9568e00640 1 -- 172.21.15.46:0/2858873409 shutdown_connections 2024-03-29T08:20:56.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.959+0000 7f9568e00640 1 --2- 172.21.15.46:0/2858873409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95640cc720 0x7f95640c3aa0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.959+0000 7f9568e00640 1 --2- 172.21.15.46:0/2858873409 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9564159a20 0x7f95640cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.959+0000 7f9568e00640 1 -- 172.21.15.46:0/2858873409 >> 172.21.15.46:0/2858873409 conn(0x7f9564098a40 msgr2=0x7f95640cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:56.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.959+0000 7f9568e00640 1 -- 172.21.15.46:0/2858873409 shutdown_connections 2024-03-29T08:20:56.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.959+0000 7f9568e00640 1 -- 172.21.15.46:0/2858873409 wait complete. 2024-03-29T08:20:56.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9568e00640 1 Processor -- start 2024-03-29T08:20:56.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9568e00640 1 -- start start 2024-03-29T08:20:56.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9568e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95640cc720 0x7f95640c7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9568e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95640d0270 0x7f95640c5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9568e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9564159a20 0x7f95640b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9568e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9564150f40 con 0x7f95640cc720 2024-03-29T08:20:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9568e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95640c7640 con 0x7f95640d0270 2024-03-29T08:20:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9568e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95641511b0 con 0x7f9564159a20 2024-03-29T08:20:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9561e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95640cc720 0x7f95640c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9561e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95640cc720 0x7f95640c7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45710/0 (socket says 172.21.15.46:45710) 2024-03-29T08:20:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9561e00640 1 -- 172.21.15.46:0/2695079405 learned_addr learned my addr 172.21.15.46:0/2695079405 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9559400640 1 --2- 172.21.15.46:0/2695079405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95640d0270 0x7f95640c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9562800640 1 --2- 172.21.15.46:0/2695079405 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9564159a20 0x7f95640b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9561e00640 1 -- 172.21.15.46:0/2695079405 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9564159a20 msgr2=0x7f95640b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9561e00640 1 --2- 172.21.15.46:0/2695079405 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9564159a20 0x7f95640b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9561e00640 1 -- 172.21.15.46:0/2695079405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95640d0270 msgr2=0x7f95640c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:56.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9561e00640 1 --2- 172.21.15.46:0/2695079405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95640d0270 0x7f95640c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:56.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9561e00640 1 -- 172.21.15.46:0/2695079405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c009a70 con 0x7f95640cc720 2024-03-29T08:20:56.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9561e00640 1 --2- 172.21.15.46:0/2695079405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95640cc720 0x7f95640c7100 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f955400e870 tx=0x7f955400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:56.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f955aa00640 1 -- 172.21.15.46:0/2695079405 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9554009510 con 0x7f95640cc720 2024-03-29T08:20:56.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f955aa00640 1 -- 172.21.15.46:0/2695079405 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955400eec0 con 0x7f95640cc720 2024-03-29T08:20:56.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f955aa00640 1 -- 172.21.15.46:0/2695079405 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955400cb40 con 0x7f95640cc720 2024-03-29T08:20:56.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9568e00640 1 -- 172.21.15.46:0/2695079405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9564151490 con 0x7f95640cc720 2024-03-29T08:20:56.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9568e00640 1 -- 172.21.15.46:0/2695079405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95640c78c0 con 0x7f95640cc720 2024-03-29T08:20:56.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.963+0000 7f9568e00640 1 -- 172.21.15.46:0/2695079405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9564066c90 con 0x7f95640cc720 2024-03-29T08:20:56.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.967+0000 7f955aa00640 1 -- 172.21.15.46:0/2695079405 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f955400cca0 con 0x7f95640cc720 2024-03-29T08:20:56.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.967+0000 7f955aa00640 1 --2- 172.21.15.46:0/2695079405 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9540066690 0x7f9540068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:56.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.967+0000 7f955aa00640 1 -- 172.21.15.46:0/2695079405 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9554084330 con 0x7f95640cc720 2024-03-29T08:20:56.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.967+0000 7f9559400640 1 --2- 172.21.15.46:0/2695079405 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9540066690 0x7f9540068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:56.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.967+0000 7f9559400640 1 --2- 172.21.15.46:0/2695079405 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9540066690 0x7f9540068b50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f956406f0c0 tx=0x7f954c007b60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:56.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:56.971+0000 7f955aa00640 1 -- 172.21.15.46:0/2695079405 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9554051600 con 0x7f95640cc720 2024-03-29T08:20:57.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.171+0000 7f9568e00640 1 -- 172.21.15.46:0/2695079405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95641575b0 con 0x7f95640cc720 2024-03-29T08:20:57.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.171+0000 7f955aa00640 1 -- 172.21.15.46:0/2695079405 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4731 (secure 0 0 0) 0x7f9554054c20 con 0x7f95640cc720 2024-03-29T08:20:57.174 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:57.174 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2702541693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2702541693},{"type":"v1","addr":"172.21.15.73:6837","nonce":2702541693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2507803771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2507803771},{"type":"v1","addr":"172.21.15.73:6835","nonce":2507803771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/494932499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":494932499},{"type":"v1","addr":"172.21.15.46:6833","nonce":494932499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:20:12.400114+0000","modified":"2024-03-29T08:20:15.405861+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3467089652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3467089652},{"type":"v1","addr":"172.21.15.46:6835","nonce":3467089652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-29T08:20:57.175 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-03-29T08:20:57.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.175+0000 7f9568e00640 1 -- 172.21.15.46:0/2695079405 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9540066690 msgr2=0x7f9540068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:57.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.175+0000 7f9568e00640 1 --2- 172.21.15.46:0/2695079405 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9540066690 0x7f9540068b50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f956406f0c0 tx=0x7f954c007b60 comp rx=0 tx=0).stop 2024-03-29T08:20:57.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.175+0000 7f9568e00640 1 -- 172.21.15.46:0/2695079405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95640cc720 msgr2=0x7f95640c7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:57.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.175+0000 7f9568e00640 1 --2- 172.21.15.46:0/2695079405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95640cc720 0x7f95640c7100 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f955400e870 tx=0x7f955400ec70 comp rx=0 tx=0).stop 2024-03-29T08:20:57.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.175+0000 7f9568e00640 1 -- 172.21.15.46:0/2695079405 shutdown_connections 2024-03-29T08:20:57.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.175+0000 7f9568e00640 1 --2- 172.21.15.46:0/2695079405 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9540066690 0x7f9540068b50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.175+0000 7f9568e00640 1 --2- 172.21.15.46:0/2695079405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95640d0270 0x7f95640c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.175+0000 7f9568e00640 1 --2- 172.21.15.46:0/2695079405 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9564159a20 0x7f95640b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.175+0000 7f9568e00640 1 --2- 172.21.15.46:0/2695079405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95640cc720 0x7f95640c7100 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.175+0000 7f9568e00640 1 -- 172.21.15.46:0/2695079405 >> 172.21.15.46:0/2695079405 conn(0x7f9564098a40 msgr2=0x7f95640ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:57.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.175+0000 7f9568e00640 1 -- 172.21.15.46:0/2695079405 shutdown_connections 2024-03-29T08:20:57.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.175+0000 7f9568e00640 1 -- 172.21.15.46:0/2695079405 wait complete. 2024-03-29T08:20:57.196 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:20:57.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.335+0000 7f484f200640 1 Processor -- start 2024-03-29T08:20:57.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.335+0000 7f484f200640 1 -- start start 2024-03-29T08:20:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.335+0000 7f484f200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f484815a940 0x7f48480afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.335+0000 7f484f200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48480b00d0 con 0x7f48480cf3c0 2024-03-29T08:20:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.335+0000 7f484f200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48480b0ab0 con 0x7f484815ad20 2024-03-29T08:20:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.335+0000 7f484f200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48480b0d90 con 0x7f484815a940 2024-03-29T08:20:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.335+0000 7f484d200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f48480cf3c0 0x7f48480cb630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37310/0 (socket says 172.21.15.46:37310) 2024-03-29T08:20:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.335+0000 7f484d200640 1 -- 172.21.15.46:0/3099750164 learned_addr learned my addr 172.21.15.46:0/3099750164 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:57.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.335+0000 7f4847e00640 1 --2- 172.21.15.46:0/3099750164 >> v2:172.21.15.73:3300/0 conn(0x7f484815a940 0x7f48480afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.335+0000 7f4847400640 1 -- 172.21.15.46:0/3099750164 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339008429 0 0) 0x7f48480b00d0 con 0x7f48480cf3c0 2024-03-29T08:20:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.335+0000 7f4847400640 1 -- 172.21.15.46:0/3099750164 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f482c003530 con 0x7f48480cf3c0 2024-03-29T08:20:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.335+0000 7f4847e00640 1 -- 172.21.15.46:0/3099750164 >> v1:172.21.15.73:6789/0 conn(0x7f484815ad20 legacy=0x7f48480cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.335+0000 7f4847e00640 1 -- 172.21.15.46:0/3099750164 >> v1:172.21.15.46:6789/0 conn(0x7f48480cf3c0 legacy=0x7f48480cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:57.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.335+0000 7f4847e00640 1 -- 172.21.15.46:0/3099750164 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48480b1070 con 0x7f484815a940 2024-03-29T08:20:57.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.339+0000 7f4847e00640 1 --2- 172.21.15.46:0/3099750164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f484815a940 0x7f48480afb90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f48380097c0 tx=0x7f483801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f85a82641b3b0d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:57.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.339+0000 7f4847400640 1 -- 172.21.15.46:0/3099750164 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48380ac640 con 0x7f484815a940 2024-03-29T08:20:57.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.339+0000 7f4847400640 1 -- 172.21.15.46:0/3099750164 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48380ac7a0 con 0x7f484815a940 2024-03-29T08:20:57.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.339+0000 7f4847400640 1 -- 172.21.15.46:0/3099750164 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48380acb30 con 0x7f484815a940 2024-03-29T08:20:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.339+0000 7f484f200640 1 -- 172.21.15.46:0/3099750164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f484815a940 msgr2=0x7f48480afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.339+0000 7f484f200640 1 --2- 172.21.15.46:0/3099750164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f484815a940 0x7f48480afb90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f48380097c0 tx=0x7f483801c3e0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.339+0000 7f484f200640 1 -- 172.21.15.46:0/3099750164 shutdown_connections 2024-03-29T08:20:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.339+0000 7f484f200640 1 --2- 172.21.15.46:0/3099750164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f484815a940 0x7f48480afb90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.339+0000 7f484f200640 1 -- 172.21.15.46:0/3099750164 >> 172.21.15.46:0/3099750164 conn(0x7f4848089fd0 msgr2=0x7f484808a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.339+0000 7f484f200640 1 -- 172.21.15.46:0/3099750164 shutdown_connections 2024-03-29T08:20:57.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.339+0000 7f484f200640 1 -- 172.21.15.46:0/3099750164 wait complete. 2024-03-29T08:20:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f484f200640 1 Processor -- start 2024-03-29T08:20:57.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f484f200640 1 -- start start 2024-03-29T08:20:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f484f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48480cf3c0 0x7f4848152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f484f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f484815a940 0x7f484806b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f484f200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f484815ad20 0x7f484806be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f484f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48480c9080 con 0x7f48480cf3c0 2024-03-29T08:20:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f484f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48480c8850 con 0x7f484815a940 2024-03-29T08:20:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f484f200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4848152680 con 0x7f484815ad20 2024-03-29T08:20:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f4847e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48480cf3c0 0x7f4848152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f4847e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48480cf3c0 0x7f4848152920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45716/0 (socket says 172.21.15.46:45716) 2024-03-29T08:20:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f4847e00640 1 -- 172.21.15.46:0/965967141 learned_addr learned my addr 172.21.15.46:0/965967141 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f484d200640 1 --2- 172.21.15.46:0/965967141 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f484815ad20 0x7f484806be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f483f400640 1 --2- 172.21.15.46:0/965967141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f484815a940 0x7f484806b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f4847e00640 1 -- 172.21.15.46:0/965967141 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f484815ad20 msgr2=0x7f484806be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f4847e00640 1 --2- 172.21.15.46:0/965967141 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f484815ad20 0x7f484806be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f4847e00640 1 -- 172.21.15.46:0/965967141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f484815a940 msgr2=0x7f484806b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f4847e00640 1 --2- 172.21.15.46:0/965967141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f484815a940 0x7f484806b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f4847e00640 1 -- 172.21.15.46:0/965967141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4838009420 con 0x7f48480cf3c0 2024-03-29T08:20:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f483f400640 1 --2- 172.21.15.46:0/965967141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f484815a940 0x7f484806b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f484d200640 1 --2- 172.21.15.46:0/965967141 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f484815ad20 0x7f484806be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:20:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f4847e00640 1 --2- 172.21.15.46:0/965967141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48480cf3c0 0x7f4848152920 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f483801a010 tx=0x7f4838019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f4844c00640 1 -- 172.21.15.46:0/965967141 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48380b5600 con 0x7f48480cf3c0 2024-03-29T08:20:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f484f200640 1 -- 172.21.15.46:0/965967141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48480c8ac0 con 0x7f48480cf3c0 2024-03-29T08:20:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f4844c00640 1 -- 172.21.15.46:0/965967141 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48380b5780 con 0x7f48480cf3c0 2024-03-29T08:20:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f4844c00640 1 -- 172.21.15.46:0/965967141 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48380b5b30 con 0x7f48480cf3c0 2024-03-29T08:20:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.343+0000 7f484f200640 1 -- 172.21.15.46:0/965967141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4848152e60 con 0x7f48480cf3c0 2024-03-29T08:20:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.347+0000 7f484f200640 1 -- 172.21.15.46:0/965967141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484815c200 con 0x7f48480cf3c0 2024-03-29T08:20:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.347+0000 7f4844c00640 1 -- 172.21.15.46:0/965967141 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f48380b3040 con 0x7f48480cf3c0 2024-03-29T08:20:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.347+0000 7f4844c00640 1 --2- 172.21.15.46:0/965967141 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4824066690 0x7f4824068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.347+0000 7f4844c00640 1 -- 172.21.15.46:0/965967141 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f483811e710 con 0x7f48480cf3c0 2024-03-29T08:20:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.347+0000 7f483f400640 1 --2- 172.21.15.46:0/965967141 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4824066690 0x7f4824068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:57.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.347+0000 7f483f400640 1 --2- 172.21.15.46:0/965967141 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4824066690 0x7f4824068b50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f48480b0380 tx=0x7f4830009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:57.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.351+0000 7f4844c00640 1 -- 172.21.15.46:0/965967141 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f484815c200 con 0x7f48480cf3c0 2024-03-29T08:20:57.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.511+0000 7f484f200640 1 -- 172.21.15.46:0/965967141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f484815c200 con 0x7f48480cf3c0 2024-03-29T08:20:57.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.511+0000 7f4844c00640 1 -- 172.21.15.46:0/965967141 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16223 (secure 0 0 0) 0x7f48380b1090 con 0x7f48480cf3c0 2024-03-29T08:20:57.515 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:57.515 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":51,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:20:56.987721+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:20:10.906619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-29T08:20:11.389667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.3","mappings":[{"from":5,"to":7}]},{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.c","mappings":[{"from":5,"to":0}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.2b","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:20:57.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.515+0000 7f484f200640 1 -- 172.21.15.46:0/965967141 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4824066690 msgr2=0x7f4824068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.515+0000 7f484f200640 1 --2- 172.21.15.46:0/965967141 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4824066690 0x7f4824068b50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f48480b0380 tx=0x7f4830009040 comp rx=0 tx=0).stop 2024-03-29T08:20:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.515+0000 7f484f200640 1 -- 172.21.15.46:0/965967141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48480cf3c0 msgr2=0x7f4848152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.515+0000 7f484f200640 1 --2- 172.21.15.46:0/965967141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48480cf3c0 0x7f4848152920 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f483801a010 tx=0x7f4838019180 comp rx=0 tx=0).stop 2024-03-29T08:20:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.515+0000 7f484f200640 1 -- 172.21.15.46:0/965967141 shutdown_connections 2024-03-29T08:20:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.515+0000 7f484f200640 1 --2- 172.21.15.46:0/965967141 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4824066690 0x7f4824068b50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.515+0000 7f484f200640 1 --2- 172.21.15.46:0/965967141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f484815a940 0x7f484806b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.515+0000 7f484f200640 1 --2- 172.21.15.46:0/965967141 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f484815ad20 0x7f484806be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.515+0000 7f484f200640 1 --2- 172.21.15.46:0/965967141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48480cf3c0 0x7f4848152920 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.515+0000 7f484f200640 1 -- 172.21.15.46:0/965967141 >> 172.21.15.46:0/965967141 conn(0x7f4848089fd0 msgr2=0x7f48480cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.515+0000 7f484f200640 1 -- 172.21.15.46:0/965967141 shutdown_connections 2024-03-29T08:20:57.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.515+0000 7f484f200640 1 -- 172.21.15.46:0/965967141 wait complete. 2024-03-29T08:20:57.536 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T08:20:57.536 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:20:57.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.671+0000 7f7048600640 1 Processor -- start 2024-03-29T08:20:57.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.675+0000 7f7048600640 1 -- start start 2024-03-29T08:20:57.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.675+0000 7f7048600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f704015a940 0x7f70400afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:57.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.675+0000 7f7048600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70400b00d0 con 0x7f70400cf3c0 2024-03-29T08:20:57.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.675+0000 7f7048600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70400b0ab0 con 0x7f704015ad20 2024-03-29T08:20:57.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.675+0000 7f7048600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70400b0d90 con 0x7f704015a940 2024-03-29T08:20:57.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.675+0000 7f7046600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f70400cf3c0 0x7f70400cb630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37320/0 (socket says 172.21.15.46:37320) 2024-03-29T08:20:57.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.675+0000 7f7046600640 1 -- 172.21.15.46:0/2250549605 learned_addr learned my addr 172.21.15.46:0/2250549605 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:57.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.675+0000 7f7045c00640 1 --2- 172.21.15.46:0/2250549605 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f704015a940 0x7f70400afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:57.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.675+0000 7f703fe00640 1 -- 172.21.15.46:0/2250549605 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879019103 0 0) 0x7f70400b00d0 con 0x7f70400cf3c0 2024-03-29T08:20:57.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.675+0000 7f703fe00640 1 -- 172.21.15.46:0/2250549605 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7028003530 con 0x7f70400cf3c0 2024-03-29T08:20:57.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.675+0000 7f703fe00640 1 -- 172.21.15.46:0/2250549605 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4115525417 0 0) 0x7f70400b0ab0 con 0x7f704015ad20 2024-03-29T08:20:57.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.675+0000 7f703fe00640 1 -- 172.21.15.46:0/2250549605 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70400b00d0 con 0x7f704015ad20 2024-03-29T08:20:57.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.675+0000 7f703fe00640 1 -- 172.21.15.46:0/2250549605 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1468360512 0 0) 0x7f7028003530 con 0x7f70400cf3c0 2024-03-29T08:20:57.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.675+0000 7f7045c00640 1 -- 172.21.15.46:0/2250549605 >> v1:172.21.15.73:6789/0 conn(0x7f704015ad20 legacy=0x7f70400cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:57.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.675+0000 7f7045c00640 1 -- 172.21.15.46:0/2250549605 >> v1:172.21.15.46:6789/0 conn(0x7f70400cf3c0 legacy=0x7f70400cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:57.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.675+0000 7f7045c00640 1 -- 172.21.15.46:0/2250549605 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70400b1070 con 0x7f704015a940 2024-03-29T08:20:57.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.679+0000 7f7045c00640 1 --2- 172.21.15.46:0/2250549605 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f704015a940 0x7f70400afb90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f70340097f0 tx=0x7f703401c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e904a6cabcb59730 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:57.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.679+0000 7f703fe00640 1 -- 172.21.15.46:0/2250549605 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038001a00 con 0x7f704015a940 2024-03-29T08:20:57.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.679+0000 7f703fe00640 1 -- 172.21.15.46:0/2250549605 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70340a7050 con 0x7f704015a940 2024-03-29T08:20:57.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.679+0000 7f703fe00640 1 -- 172.21.15.46:0/2250549605 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70340ab600 con 0x7f704015a940 2024-03-29T08:20:57.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.679+0000 7f7048600640 1 -- 172.21.15.46:0/2250549605 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f704015a940 msgr2=0x7f70400afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:57.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.679+0000 7f7048600640 1 --2- 172.21.15.46:0/2250549605 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f704015a940 0x7f70400afb90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f70340097f0 tx=0x7f703401c340 comp rx=0 tx=0).stop 2024-03-29T08:20:57.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.679+0000 7f7048600640 1 -- 172.21.15.46:0/2250549605 shutdown_connections 2024-03-29T08:20:57.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.679+0000 7f7048600640 1 --2- 172.21.15.46:0/2250549605 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f704015a940 0x7f70400afb90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.679+0000 7f7048600640 1 -- 172.21.15.46:0/2250549605 >> 172.21.15.46:0/2250549605 conn(0x7f7040089fd0 msgr2=0x7f704008a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:57.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.679+0000 7f7048600640 1 -- 172.21.15.46:0/2250549605 shutdown_connections 2024-03-29T08:20:57.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.679+0000 7f7048600640 1 -- 172.21.15.46:0/2250549605 wait complete. 2024-03-29T08:20:57.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.679+0000 7f7048600640 1 Processor -- start 2024-03-29T08:20:57.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.679+0000 7f7048600640 1 -- start start 2024-03-29T08:20:57.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7048600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70400cf3c0 0x7f704015d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:57.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7048600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f704015a940 0x7f704015b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:57.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7048600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f704015ad20 0x7f704015b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:57.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7048600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70400ad8e0 con 0x7f704015ad20 2024-03-29T08:20:57.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7048600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704015df20 con 0x7f70400cf3c0 2024-03-29T08:20:57.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7048600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f704015d680 con 0x7f704015a940 2024-03-29T08:20:57.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7046600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f704015ad20 0x7f704015b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:57.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7046600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f704015ad20 0x7f704015b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45718/0 (socket says 172.21.15.46:45718) 2024-03-29T08:20:57.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7046600640 1 -- 172.21.15.46:0/3963989689 learned_addr learned my addr 172.21.15.46:0/3963989689 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:57.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7045200640 1 --2- 172.21.15.46:0/3963989689 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f704015a940 0x7f704015b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:57.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7045c00640 1 --2- 172.21.15.46:0/3963989689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70400cf3c0 0x7f704015d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:57.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7046600640 1 -- 172.21.15.46:0/3963989689 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f704015a940 msgr2=0x7f704015b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:57.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7046600640 1 --2- 172.21.15.46:0/3963989689 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f704015a940 0x7f704015b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7046600640 1 -- 172.21.15.46:0/3963989689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70400cf3c0 msgr2=0x7f704015d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:57.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7046600640 1 --2- 172.21.15.46:0/3963989689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70400cf3c0 0x7f704015d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7046600640 1 -- 172.21.15.46:0/3963989689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7034009420 con 0x7f704015ad20 2024-03-29T08:20:57.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7046600640 1 --2- 172.21.15.46:0/3963989689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f704015ad20 0x7f704015b660 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7038001f90 tx=0x7f703800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:57.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f703d600640 1 -- 172.21.15.46:0/3963989689 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7038012400 con 0x7f704015ad20 2024-03-29T08:20:57.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f703d600640 1 -- 172.21.15.46:0/3963989689 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7038012560 con 0x7f704015ad20 2024-03-29T08:20:57.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7048600640 1 -- 172.21.15.46:0/3963989689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704015d960 con 0x7f704015ad20 2024-03-29T08:20:57.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f703d600640 1 -- 172.21.15.46:0/3963989689 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70380128f0 con 0x7f704015ad20 2024-03-29T08:20:57.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7048600640 1 -- 172.21.15.46:0/3963989689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70400ad3e0 con 0x7f704015ad20 2024-03-29T08:20:57.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f7048600640 1 -- 172.21.15.46:0/3963989689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70400ada20 con 0x7f704015ad20 2024-03-29T08:20:57.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.683+0000 7f703d600640 1 -- 172.21.15.46:0/3963989689 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7038012a50 con 0x7f704015ad20 2024-03-29T08:20:57.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.687+0000 7f703d600640 1 --2- 172.21.15.46:0/3963989689 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7020066690 0x7f7020068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:57.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.687+0000 7f703d600640 1 -- 172.21.15.46:0/3963989689 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7038084bd0 con 0x7f704015ad20 2024-03-29T08:20:57.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.687+0000 7f7045c00640 1 --2- 172.21.15.46:0/3963989689 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7020066690 0x7f7020068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:57.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.687+0000 7f7045c00640 1 --2- 172.21.15.46:0/3963989689 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7020066690 0x7f7020068b50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f70340ad000 tx=0x7f70340a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:57.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.691+0000 7f703d600640 1 -- 172.21.15.46:0/3963989689 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f70400ada20 con 0x7f704015ad20 2024-03-29T08:20:57.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.891+0000 7f7048600640 1 -- 172.21.15.46:0/3963989689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70400ada20 con 0x7f704015ad20 2024-03-29T08:20:57.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.891+0000 7f703d600640 1 -- 172.21.15.46:0/3963989689 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4731 (secure 0 0 0) 0x7f70400ada20 con 0x7f704015ad20 2024-03-29T08:20:57.893 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:57.893 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2702541693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2702541693},{"type":"v1","addr":"172.21.15.73:6837","nonce":2702541693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2507803771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2507803771},{"type":"v1","addr":"172.21.15.73:6835","nonce":2507803771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/494932499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":494932499},{"type":"v1","addr":"172.21.15.46:6833","nonce":494932499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:20:12.400114+0000","modified":"2024-03-29T08:20:15.405861+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3467089652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3467089652},{"type":"v1","addr":"172.21.15.46:6835","nonce":3467089652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-29T08:20:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-03-29T08:20:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.891+0000 7f7048600640 1 -- 172.21.15.46:0/3963989689 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7020066690 msgr2=0x7f7020068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.891+0000 7f7048600640 1 --2- 172.21.15.46:0/3963989689 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7020066690 0x7f7020068b50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f70340ad000 tx=0x7f70340a6000 comp rx=0 tx=0).stop 2024-03-29T08:20:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.891+0000 7f7048600640 1 -- 172.21.15.46:0/3963989689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f704015ad20 msgr2=0x7f704015b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.891+0000 7f7048600640 1 --2- 172.21.15.46:0/3963989689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f704015ad20 0x7f704015b660 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f7038001f90 tx=0x7f703800a430 comp rx=0 tx=0).stop 2024-03-29T08:20:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.891+0000 7f7048600640 1 -- 172.21.15.46:0/3963989689 shutdown_connections 2024-03-29T08:20:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.891+0000 7f7048600640 1 --2- 172.21.15.46:0/3963989689 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7020066690 0x7f7020068b50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.891+0000 7f7048600640 1 --2- 172.21.15.46:0/3963989689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f704015ad20 0x7f704015b660 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.891+0000 7f7048600640 1 --2- 172.21.15.46:0/3963989689 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f704015a940 0x7f704015b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.891+0000 7f7048600640 1 --2- 172.21.15.46:0/3963989689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70400cf3c0 0x7f704015d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.891+0000 7f7048600640 1 -- 172.21.15.46:0/3963989689 >> 172.21.15.46:0/3963989689 conn(0x7f7040089fd0 msgr2=0x7f704006aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.891+0000 7f7048600640 1 -- 172.21.15.46:0/3963989689 shutdown_connections 2024-03-29T08:20:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:57.891+0000 7f7048600640 1 -- 172.21.15.46:0/3963989689 wait complete. 2024-03-29T08:20:57.914 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:20:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.055+0000 7f730ca00640 1 Processor -- start 2024-03-29T08:20:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.055+0000 7f730ca00640 1 -- start start 2024-03-29T08:20:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.055+0000 7f730ca00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7308165400 0x7f73081677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.055+0000 7f730ca00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f730806a5e0 con 0x7f73080b0310 2024-03-29T08:20:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.055+0000 7f730ca00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f730806a8c0 con 0x7f73080ce590 2024-03-29T08:20:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.055+0000 7f730ca00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730806aba0 con 0x7f7308165400 2024-03-29T08:20:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.055+0000 7f7307400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f73080b0310 0x7f73080b06f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37326/0 (socket says 172.21.15.46:37326) 2024-03-29T08:20:58.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.055+0000 7f7307400640 1 -- 172.21.15.46:0/1423871859 learned_addr learned my addr 172.21.15.46:0/1423871859 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:58.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.055+0000 7f7307e00640 1 --2- 172.21.15.46:0/1423871859 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7308165400 0x7f73081677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:58.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.055+0000 7f7306a00640 1 -- 172.21.15.46:0/1423871859 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2157010267 0 0) 0x7f730806a5e0 con 0x7f73080b0310 2024-03-29T08:20:58.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.055+0000 7f7306a00640 1 -- 172.21.15.46:0/1423871859 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72ec003530 con 0x7f73080b0310 2024-03-29T08:20:58.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.055+0000 7f7306a00640 1 -- 172.21.15.46:0/1423871859 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4195956638 0 0) 0x7f730806a8c0 con 0x7f73080ce590 2024-03-29T08:20:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.055+0000 7f7306a00640 1 -- 172.21.15.46:0/1423871859 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f730806a5e0 con 0x7f73080ce590 2024-03-29T08:20:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.055+0000 7f7306a00640 1 -- 172.21.15.46:0/1423871859 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2688082457 0 0) 0x7f72ec003530 con 0x7f73080b0310 2024-03-29T08:20:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.055+0000 7f7307e00640 1 -- 172.21.15.46:0/1423871859 >> v1:172.21.15.73:6789/0 conn(0x7f73080ce590 legacy=0x7f7308069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.055+0000 7f7307e00640 1 -- 172.21.15.46:0/1423871859 >> v1:172.21.15.46:6789/0 conn(0x7f73080b0310 legacy=0x7f73080b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.055+0000 7f7307e00640 1 -- 172.21.15.46:0/1423871859 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73080ae6e0 con 0x7f7308165400 2024-03-29T08:20:58.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.059+0000 7f7307e00640 1 --2- 172.21.15.46:0/1423871859 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7308165400 0x7f73081677f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7300009dc0 tx=0x7f730001c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f70ac3fbeb7f2ff server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:58.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.059+0000 7f7306a00640 1 -- 172.21.15.46:0/1423871859 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f8001680 con 0x7f7308165400 2024-03-29T08:20:58.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.059+0000 7f7306a00640 1 -- 172.21.15.46:0/1423871859 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73080ac520 con 0x7f7308165400 2024-03-29T08:20:58.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.059+0000 7f7306a00640 1 -- 172.21.15.46:0/1423871859 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73000a5820 con 0x7f7308165400 2024-03-29T08:20:58.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.059+0000 7f730ca00640 1 -- 172.21.15.46:0/1423871859 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7308165400 msgr2=0x7f73081677f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.059+0000 7f730ca00640 1 --2- 172.21.15.46:0/1423871859 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7308165400 0x7f73081677f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7300009dc0 tx=0x7f730001c8b0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.059+0000 7f730ca00640 1 -- 172.21.15.46:0/1423871859 shutdown_connections 2024-03-29T08:20:58.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.059+0000 7f730ca00640 1 --2- 172.21.15.46:0/1423871859 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7308165400 0x7f73081677f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.059+0000 7f730ca00640 1 -- 172.21.15.46:0/1423871859 >> 172.21.15.46:0/1423871859 conn(0x7f7308098a40 msgr2=0x7f73080cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:58.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.059+0000 7f730ca00640 1 -- 172.21.15.46:0/1423871859 shutdown_connections 2024-03-29T08:20:58.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.059+0000 7f730ca00640 1 -- 172.21.15.46:0/1423871859 wait complete. 2024-03-29T08:20:58.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f730ca00640 1 Processor -- start 2024-03-29T08:20:58.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f730ca00640 1 -- start start 2024-03-29T08:20:58.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f730ca00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f73080b0310 0x7f73080c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:58.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f730ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73080ce590 0x7f730815cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:58.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f730ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7308165400 0x7f73080b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:58.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f730ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73080afef0 con 0x7f7308165400 2024-03-29T08:20:58.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f730ca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730815ad50 con 0x7f73080ce590 2024-03-29T08:20:58.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f730ca00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7308157220 con 0x7f73080b0310 2024-03-29T08:20:58.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f7307e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7308165400 0x7f73080b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:58.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f7307e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7308165400 0x7f73080b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45720/0 (socket says 172.21.15.46:45720) 2024-03-29T08:20:58.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f7307e00640 1 -- 172.21.15.46:0/13119647 learned_addr learned my addr 172.21.15.46:0/13119647 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:58.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f7307400640 1 --2- 172.21.15.46:0/13119647 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f73080b0310 0x7f73080c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f72fea00640 1 --2- 172.21.15.46:0/13119647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73080ce590 0x7f730815cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f7307e00640 1 -- 172.21.15.46:0/13119647 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f73080b0310 msgr2=0x7f73080c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f7307e00640 1 --2- 172.21.15.46:0/13119647 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f73080b0310 0x7f73080c5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f7307e00640 1 -- 172.21.15.46:0/13119647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73080ce590 msgr2=0x7f730815cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f7307e00640 1 --2- 172.21.15.46:0/13119647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73080ce590 0x7f730815cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f7307e00640 1 -- 172.21.15.46:0/13119647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7300009a70 con 0x7f7308165400 2024-03-29T08:20:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f7307e00640 1 --2- 172.21.15.46:0/13119647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7308165400 0x7f73080b4580 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f730001cd90 tx=0x7f7300019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f72ffe00640 1 -- 172.21.15.46:0/13119647 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73000ad440 con 0x7f7308165400 2024-03-29T08:20:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f72ffe00640 1 -- 172.21.15.46:0/13119647 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73000ad5a0 con 0x7f7308165400 2024-03-29T08:20:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f730ca00640 1 -- 172.21.15.46:0/13119647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7308159550 con 0x7f7308165400 2024-03-29T08:20:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f72ffe00640 1 -- 172.21.15.46:0/13119647 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73000ad910 con 0x7f7308165400 2024-03-29T08:20:58.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f730ca00640 1 -- 172.21.15.46:0/13119647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73081589e0 con 0x7f7308165400 2024-03-29T08:20:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.063+0000 7f730ca00640 1 -- 172.21.15.46:0/13119647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73081596e0 con 0x7f7308165400 2024-03-29T08:20:58.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.067+0000 7f72ffe00640 1 -- 172.21.15.46:0/13119647 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f73000b4050 con 0x7f7308165400 2024-03-29T08:20:58.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.067+0000 7f72ffe00640 1 --2- 172.21.15.46:0/13119647 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f72e4066690 0x7f72e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:58.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.067+0000 7f72ffe00640 1 -- 172.21.15.46:0/13119647 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f730011ecb0 con 0x7f7308165400 2024-03-29T08:20:58.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.067+0000 7f7307400640 1 --2- 172.21.15.46:0/13119647 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f72e4066690 0x7f72e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:58.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.067+0000 7f7307400640 1 --2- 172.21.15.46:0/13119647 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f72e4066690 0x7f72e4068b50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f72f8000fe0 tx=0x7f72f8001cc0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:58.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.071+0000 7f72ffe00640 1 -- 172.21.15.46:0/13119647 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f73000ec150 con 0x7f7308165400 2024-03-29T08:20:58.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.263+0000 7f730ca00640 1 -- 172.21.15.46:0/13119647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7308158980 con 0x7f7308165400 2024-03-29T08:20:58.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.263+0000 7f72ffe00640 1 -- 172.21.15.46:0/13119647 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4731 (secure 0 0 0) 0x7f73000c7090 con 0x7f7308165400 2024-03-29T08:20:58.268 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:58.268 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2702541693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2702541693},{"type":"v1","addr":"172.21.15.73:6837","nonce":2702541693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2507803771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2507803771},{"type":"v1","addr":"172.21.15.73:6835","nonce":2507803771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/494932499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":494932499},{"type":"v1","addr":"172.21.15.46:6833","nonce":494932499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:20:12.400114+0000","modified":"2024-03-29T08:20:15.405861+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5631":{"gid":5631,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3467089652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3467089652},{"type":"v1","addr":"172.21.15.46:6835","nonce":3467089652}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-03-29T08:20:58.268 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-03-29T08:20:58.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.267+0000 7f730ca00640 1 -- 172.21.15.46:0/13119647 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f72e4066690 msgr2=0x7f72e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.267+0000 7f730ca00640 1 --2- 172.21.15.46:0/13119647 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f72e4066690 0x7f72e4068b50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f72f8000fe0 tx=0x7f72f8001cc0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.267+0000 7f730ca00640 1 -- 172.21.15.46:0/13119647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7308165400 msgr2=0x7f73080b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.267+0000 7f730ca00640 1 --2- 172.21.15.46:0/13119647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7308165400 0x7f73080b4580 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f730001cd90 tx=0x7f7300019650 comp rx=0 tx=0).stop 2024-03-29T08:20:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.267+0000 7f730ca00640 1 -- 172.21.15.46:0/13119647 shutdown_connections 2024-03-29T08:20:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.267+0000 7f730ca00640 1 --2- 172.21.15.46:0/13119647 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f72e4066690 0x7f72e4068b50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.267+0000 7f730ca00640 1 --2- 172.21.15.46:0/13119647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73080ce590 0x7f730815cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.267+0000 7f730ca00640 1 --2- 172.21.15.46:0/13119647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7308165400 0x7f73080b4580 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.267+0000 7f730ca00640 1 --2- 172.21.15.46:0/13119647 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f73080b0310 0x7f73080c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.267+0000 7f730ca00640 1 -- 172.21.15.46:0/13119647 >> 172.21.15.46:0/13119647 conn(0x7f7308098a40 msgr2=0x7f730806d340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.267+0000 7f730ca00640 1 -- 172.21.15.46:0/13119647 shutdown_connections 2024-03-29T08:20:58.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.267+0000 7f730ca00640 1 -- 172.21.15.46:0/13119647 wait complete. 2024-03-29T08:20:58.289 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:20:58.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.427+0000 7f0d8a600640 1 Processor -- start 2024-03-29T08:20:58.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.427+0000 7f0d8a600640 1 -- start start 2024-03-29T08:20:58.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.427+0000 7f0d8a600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0d84153d50 0x7f0d841611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:58.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.427+0000 7f0d8a600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d840ad3e0 0x7f0d840ad850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:58.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.427+0000 7f0d8a600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8415d870 con 0x7f0d84153970 2024-03-29T08:20:58.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.427+0000 7f0d8a600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d840add90 con 0x7f0d84153d50 2024-03-29T08:20:58.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.427+0000 7f0d8a600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8415db50 con 0x7f0d840ad3e0 2024-03-29T08:20:58.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.427+0000 7f0d83e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d840ad3e0 0x7f0d840ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:58.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.427+0000 7f0d83e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d840ad3e0 0x7f0d840ad850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:34986/0 (socket says 172.21.15.46:34986) 2024-03-29T08:20:58.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.427+0000 7f0d83e00640 1 -- 172.21.15.46:0/3757109997 learned_addr learned my addr 172.21.15.46:0/3757109997 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:58.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.427+0000 7f0d7aa00640 1 --2- 172.21.15.46:0/3757109997 >> v2:172.21.15.73:3300/0 conn(0x7f0d84153d50 0x7f0d841611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:58.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.431+0000 7f0d83e00640 1 -- 172.21.15.46:0/3757109997 >> v2:172.21.15.73:3300/0 conn(0x7f0d84153d50 msgr2=0x7f0d841611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:20:58.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.431+0000 7f0d83e00640 1 --2- 172.21.15.46:0/3757109997 >> v2:172.21.15.73:3300/0 conn(0x7f0d84153d50 0x7f0d841611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.431+0000 7f0d83e00640 1 -- 172.21.15.46:0/3757109997 >> v1:172.21.15.73:6789/0 conn(0x7f0d84153970 legacy=0x7f0d8415cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.431+0000 7f0d83e00640 1 -- 172.21.15.46:0/3757109997 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8415de30 con 0x7f0d840ad3e0 2024-03-29T08:20:58.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.431+0000 7f0d83e00640 1 --2- 172.21.15.46:0/3757109997 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d840ad3e0 0x7f0d840ad850 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009e50 tx=0x7f0d7c01ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98eb1e7c8492edf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:58.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.431+0000 7f0d82a00640 1 -- 172.21.15.46:0/3757109997 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c0a5870 con 0x7f0d840ad3e0 2024-03-29T08:20:58.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.431+0000 7f0d82a00640 1 -- 172.21.15.46:0/3757109997 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c0a59d0 con 0x7f0d840ad3e0 2024-03-29T08:20:58.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.431+0000 7f0d82a00640 1 -- 172.21.15.46:0/3757109997 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c0a5d60 con 0x7f0d840ad3e0 2024-03-29T08:20:58.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.431+0000 7f0d8a600640 1 -- 172.21.15.46:0/3757109997 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d840ad3e0 msgr2=0x7f0d840ad850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.431+0000 7f0d8a600640 1 --2- 172.21.15.46:0/3757109997 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d840ad3e0 0x7f0d840ad850 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009e50 tx=0x7f0d7c01ca00 comp rx=0 tx=0).stop 2024-03-29T08:20:58.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.431+0000 7f0d8a600640 1 -- 172.21.15.46:0/3757109997 shutdown_connections 2024-03-29T08:20:58.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.431+0000 7f0d8a600640 1 --2- 172.21.15.46:0/3757109997 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d840ad3e0 0x7f0d840ad850 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.431+0000 7f0d8a600640 1 --2- 172.21.15.46:0/3757109997 >> v2:172.21.15.73:3300/0 conn(0x7f0d84153d50 0x7f0d841611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.431+0000 7f0d8a600640 1 -- 172.21.15.46:0/3757109997 >> 172.21.15.46:0/3757109997 conn(0x7f0d84066110 msgr2=0x7f0d84069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:58.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.431+0000 7f0d8a600640 1 -- 172.21.15.46:0/3757109997 shutdown_connections 2024-03-29T08:20:58.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.431+0000 7f0d8a600640 1 -- 172.21.15.46:0/3757109997 wait complete. 2024-03-29T08:20:58.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d8a600640 1 Processor -- start 2024-03-29T08:20:58.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d8a600640 1 -- start start 2024-03-29T08:20:58.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d8a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d840ad3e0 0x7f0d841527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:58.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d8a600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d84153970 0x7f0d8413e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:58.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d8a600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d841467a0 0x7f0d84146c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:58.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d8a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d84147170 con 0x7f0d840ad3e0 2024-03-29T08:20:58.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d8a600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8413eb00 con 0x7f0d84153970 2024-03-29T08:20:58.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d8a600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d84151710 con 0x7f0d841467a0 2024-03-29T08:20:58.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d83400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d840ad3e0 0x7f0d841527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:58.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d83400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d840ad3e0 0x7f0d841527b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45732/0 (socket says 172.21.15.46:45732) 2024-03-29T08:20:58.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d83400640 1 -- 172.21.15.46:0/2373954419 learned_addr learned my addr 172.21.15.46:0/2373954419 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:58.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d83e00640 1 --2- 172.21.15.46:0/2373954419 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d841467a0 0x7f0d84146c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:58.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d7aa00640 1 --2- 172.21.15.46:0/2373954419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d84153970 0x7f0d8413e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:58.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d83400640 1 -- 172.21.15.46:0/2373954419 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d841467a0 msgr2=0x7f0d84146c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d83400640 1 --2- 172.21.15.46:0/2373954419 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d841467a0 0x7f0d84146c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d83400640 1 -- 172.21.15.46:0/2373954419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d84153970 msgr2=0x7f0d8413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d83400640 1 --2- 172.21.15.46:0/2373954419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d84153970 0x7f0d8413e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d83400640 1 -- 172.21.15.46:0/2373954419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c009b30 con 0x7f0d840ad3e0 2024-03-29T08:20:58.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d83400640 1 --2- 172.21.15.46:0/2373954419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d840ad3e0 0x7f0d841527b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00c500 tx=0x7f0d6c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:58.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d7be00640 1 -- 172.21.15.46:0/2373954419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c005220 con 0x7f0d840ad3e0 2024-03-29T08:20:58.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d7be00640 1 -- 172.21.15.46:0/2373954419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6c005380 con 0x7f0d840ad3e0 2024-03-29T08:20:58.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d8a600640 1 -- 172.21.15.46:0/2373954419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d841519f0 con 0x7f0d840ad3e0 2024-03-29T08:20:58.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d7be00640 1 -- 172.21.15.46:0/2373954419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6c007270 con 0x7f0d840ad3e0 2024-03-29T08:20:58.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.435+0000 7f0d8a600640 1 -- 172.21.15.46:0/2373954419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d84151fb0 con 0x7f0d840ad3e0 2024-03-29T08:20:58.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.439+0000 7f0d8a600640 1 -- 172.21.15.46:0/2373954419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d84066c90 con 0x7f0d840ad3e0 2024-03-29T08:20:58.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.439+0000 7f0d7be00640 1 -- 172.21.15.46:0/2373954419 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0d6c0073d0 con 0x7f0d840ad3e0 2024-03-29T08:20:58.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.439+0000 7f0d7be00640 1 --2- 172.21.15.46:0/2373954419 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0d600666c0 0x7f0d60068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:58.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.439+0000 7f0d7be00640 1 -- 172.21.15.46:0/2373954419 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f0d6c0839f0 con 0x7f0d840ad3e0 2024-03-29T08:20:58.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.439+0000 7f0d7aa00640 1 --2- 172.21.15.46:0/2373954419 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0d600666c0 0x7f0d60068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:58.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.439+0000 7f0d7aa00640 1 --2- 172.21.15.46:0/2373954419 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0d600666c0 0x7f0d60068b80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0d841613d0 tx=0x7f0d74009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:58.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.443+0000 7f0d7be00640 1 -- 172.21.15.46:0/2373954419 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0d6c050cf0 con 0x7f0d840ad3e0 2024-03-29T08:20:58.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.607+0000 7f0d8a600640 1 -- 172.21.15.46:0/2373954419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d8406c240 con 0x7f0d840ad3e0 2024-03-29T08:20:58.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.607+0000 7f0d7be00640 1 -- 172.21.15.46:0/2373954419 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16223 (secure 0 0 0) 0x7f0d6c054310 con 0x7f0d840ad3e0 2024-03-29T08:20:58.610 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:20:58.610 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":51,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:20:56.987721+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:20:10.906619+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-03-29T08:20:11.389667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.3","mappings":[{"from":5,"to":7}]},{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.c","mappings":[{"from":5,"to":0}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.2b","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:20:58.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.611+0000 7f0d8a600640 1 -- 172.21.15.46:0/2373954419 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0d600666c0 msgr2=0x7f0d60068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.611+0000 7f0d8a600640 1 --2- 172.21.15.46:0/2373954419 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0d600666c0 0x7f0d60068b80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0d841613d0 tx=0x7f0d74009190 comp rx=0 tx=0).stop 2024-03-29T08:20:58.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.611+0000 7f0d8a600640 1 -- 172.21.15.46:0/2373954419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d840ad3e0 msgr2=0x7f0d841527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.611+0000 7f0d8a600640 1 --2- 172.21.15.46:0/2373954419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d840ad3e0 0x7f0d841527b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c00c500 tx=0x7f0d6c00c9c0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.611+0000 7f0d8a600640 1 -- 172.21.15.46:0/2373954419 shutdown_connections 2024-03-29T08:20:58.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.611+0000 7f0d8a600640 1 --2- 172.21.15.46:0/2373954419 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0d600666c0 0x7f0d60068b80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.611+0000 7f0d8a600640 1 --2- 172.21.15.46:0/2373954419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d84153970 0x7f0d8413e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.611+0000 7f0d8a600640 1 --2- 172.21.15.46:0/2373954419 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0d841467a0 0x7f0d84146c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.611+0000 7f0d8a600640 1 --2- 172.21.15.46:0/2373954419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0d840ad3e0 0x7f0d841527b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.611+0000 7f0d8a600640 1 -- 172.21.15.46:0/2373954419 >> 172.21.15.46:0/2373954419 conn(0x7f0d84066110 msgr2=0x7f0d8406c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:58.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.611+0000 7f0d8a600640 1 -- 172.21.15.46:0/2373954419 shutdown_connections 2024-03-29T08:20:58.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.611+0000 7f0d8a600640 1 -- 172.21.15.46:0/2373954419 wait complete. 2024-03-29T08:20:58.632 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T08:20:58.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.771+0000 7f1031000640 1 Processor -- start 2024-03-29T08:20:58.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.771+0000 7f1031000640 1 -- start start 2024-03-29T08:20:58.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.771+0000 7f1031000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f102c06dbb0 0x7f102c15d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:58.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.771+0000 7f1031000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f102c070c70 con 0x7f102c142ab0 2024-03-29T08:20:58.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.771+0000 7f1031000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f102c070340 con 0x7f102c15d940 2024-03-29T08:20:58.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.771+0000 7f1031000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102c070620 con 0x7f102c06dbb0 2024-03-29T08:20:58.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.771+0000 7f1029e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f102c142ab0 0x7f102c06d4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37338/0 (socket says 172.21.15.46:37338) 2024-03-29T08:20:58.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.771+0000 7f1029e00640 1 -- 172.21.15.46:0/3802888671 learned_addr learned my addr 172.21.15.46:0/3802888671 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:20:58.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.771+0000 7f1023e00640 1 --2- 172.21.15.46:0/3802888671 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f102c06dbb0 0x7f102c15d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:58.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.771+0000 7f1029400640 1 -- 172.21.15.46:0/3802888671 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368201634 0 0) 0x7f102c070c70 con 0x7f102c142ab0 2024-03-29T08:20:58.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.771+0000 7f1029400640 1 -- 172.21.15.46:0/3802888671 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1010003530 con 0x7f102c142ab0 2024-03-29T08:20:58.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.771+0000 7f1029400640 1 -- 172.21.15.46:0/3802888671 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4016993439 0 0) 0x7f102c070340 con 0x7f102c15d940 2024-03-29T08:20:58.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.771+0000 7f1029400640 1 -- 172.21.15.46:0/3802888671 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f102c070c70 con 0x7f102c15d940 2024-03-29T08:20:58.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.771+0000 7f1023e00640 1 -- 172.21.15.46:0/3802888671 >> v1:172.21.15.73:6789/0 conn(0x7f102c15d940 legacy=0x7f102c0b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.771+0000 7f1023e00640 1 -- 172.21.15.46:0/3802888671 >> v1:172.21.15.46:6789/0 conn(0x7f102c142ab0 legacy=0x7f102c06d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.771+0000 7f1029400640 1 -- 172.21.15.46:0/3802888671 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 417526873 0 0) 0x7f1010003530 con 0x7f102c142ab0 2024-03-29T08:20:58.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.771+0000 7f1023e00640 1 -- 172.21.15.46:0/3802888671 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102c070900 con 0x7f102c06dbb0 2024-03-29T08:20:58.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.775+0000 7f1023e00640 1 --2- 172.21.15.46:0/3802888671 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f102c06dbb0 0x7f102c15d400 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f10140097f0 tx=0x7f101401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63f24acbe0517016 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:58.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.775+0000 7f1029400640 1 -- 172.21.15.46:0/3802888671 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c001680 con 0x7f102c06dbb0 2024-03-29T08:20:58.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.775+0000 7f1029400640 1 -- 172.21.15.46:0/3802888671 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10140a7050 con 0x7f102c06dbb0 2024-03-29T08:20:58.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.775+0000 7f1029400640 1 -- 172.21.15.46:0/3802888671 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10140ab400 con 0x7f102c06dbb0 2024-03-29T08:20:58.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.775+0000 7f1031000640 1 -- 172.21.15.46:0/3802888671 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f102c06dbb0 msgr2=0x7f102c15d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.775+0000 7f1031000640 1 --2- 172.21.15.46:0/3802888671 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f102c06dbb0 0x7f102c15d400 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f10140097f0 tx=0x7f101401c400 comp rx=0 tx=0).stop 2024-03-29T08:20:58.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.775+0000 7f1031000640 1 -- 172.21.15.46:0/3802888671 shutdown_connections 2024-03-29T08:20:58.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.775+0000 7f1031000640 1 --2- 172.21.15.46:0/3802888671 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f102c06dbb0 0x7f102c15d400 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.775+0000 7f1031000640 1 -- 172.21.15.46:0/3802888671 >> 172.21.15.46:0/3802888671 conn(0x7f102c066080 msgr2=0x7f102c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:58.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.775+0000 7f1031000640 1 -- 172.21.15.46:0/3802888671 shutdown_connections 2024-03-29T08:20:58.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.775+0000 7f1031000640 1 -- 172.21.15.46:0/3802888671 wait complete. 2024-03-29T08:20:58.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f1031000640 1 Processor -- start 2024-03-29T08:20:58.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f1031000640 1 -- start start 2024-03-29T08:20:58.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f1031000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f102c06dbb0 0x7f102c13a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:58.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f1031000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f102c142ab0 0x7f102c13b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:58.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f1031000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f102c15d940 0x7f102c13d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:58.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f1031000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102c158c70 con 0x7f102c15d940 2024-03-29T08:20:58.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f1031000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102c13bc30 con 0x7f102c06dbb0 2024-03-29T08:20:58.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f1031000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102c13b740 con 0x7f102c142ab0 2024-03-29T08:20:58.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f102a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f102c15d940 0x7f102c13d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:58.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f102a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f102c15d940 0x7f102c13d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45736/0 (socket says 172.21.15.46:45736) 2024-03-29T08:20:58.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f102a800640 1 -- 172.21.15.46:0/4175400144 learned_addr learned my addr 172.21.15.46:0/4175400144 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:58.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f1029e00640 1 --2- 172.21.15.46:0/4175400144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f102c06dbb0 0x7f102c13a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:58.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f1023e00640 1 --2- 172.21.15.46:0/4175400144 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f102c142ab0 0x7f102c13b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:58.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f102a800640 1 -- 172.21.15.46:0/4175400144 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f102c142ab0 msgr2=0x7f102c13b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f102a800640 1 --2- 172.21.15.46:0/4175400144 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f102c142ab0 0x7f102c13b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f102a800640 1 -- 172.21.15.46:0/4175400144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f102c06dbb0 msgr2=0x7f102c13a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f102a800640 1 --2- 172.21.15.46:0/4175400144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f102c06dbb0 0x7f102c13a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f102a800640 1 -- 172.21.15.46:0/4175400144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1014009420 con 0x7f102c15d940 2024-03-29T08:20:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f102a800640 1 --2- 172.21.15.46:0/4175400144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f102c15d940 0x7f102c13d1e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f102400c760 tx=0x7f102400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:58.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f1022000640 1 -- 172.21.15.46:0/4175400144 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024003b80 con 0x7f102c15d940 2024-03-29T08:20:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f1022000640 1 -- 172.21.15.46:0/4175400144 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1024003ce0 con 0x7f102c15d940 2024-03-29T08:20:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f1031000640 1 -- 172.21.15.46:0/4175400144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102c13b950 con 0x7f102c15d940 2024-03-29T08:20:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f1022000640 1 -- 172.21.15.46:0/4175400144 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102400aa90 con 0x7f102c15d940 2024-03-29T08:20:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.779+0000 7f1031000640 1 -- 172.21.15.46:0/4175400144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102c0a7920 con 0x7f102c15d940 2024-03-29T08:20:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.783+0000 7f1031000640 1 -- 172.21.15.46:0/4175400144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f102c0a74a0 con 0x7f102c15d940 2024-03-29T08:20:58.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.783+0000 7f1022000640 1 -- 172.21.15.46:0/4175400144 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f102400abf0 con 0x7f102c15d940 2024-03-29T08:20:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.783+0000 7f1022000640 1 --2- 172.21.15.46:0/4175400144 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f10080666c0 0x7f1008068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.783+0000 7f1022000640 1 -- 172.21.15.46:0/4175400144 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f10240840c0 con 0x7f102c15d940 2024-03-29T08:20:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.783+0000 7f1029e00640 1 --2- 172.21.15.46:0/4175400144 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f10080666c0 0x7f1008068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.783+0000 7f1029e00640 1 --2- 172.21.15.46:0/4175400144 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f10080666c0 0x7f1008068b80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f101c0011f0 tx=0x7f101c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:58.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.787+0000 7f1022000640 1 -- 172.21.15.46:0/4175400144 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f102c0a74a0 con 0x7f102c15d940 2024-03-29T08:20:58.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.983+0000 7f1031000640 1 -- 172.21.15.46:0/4175400144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f102c0a74a0 con 0x7f102c15d940 2024-03-29T08:20:58.987 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-03-29T08:20:58.983+0000 7fc16ae00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:20:58.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.991+0000 7f1022000640 1 -- 172.21.15.46:0/4175400144 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f102c0a74a0 con 0x7f102c15d940 2024-03-29T08:20:58.994 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T08:20:58.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.995+0000 7f1031000640 1 -- 172.21.15.46:0/4175400144 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f10080666c0 msgr2=0x7f1008068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.995+0000 7f1031000640 1 --2- 172.21.15.46:0/4175400144 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f10080666c0 0x7f1008068b80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f101c0011f0 tx=0x7f101c009040 comp rx=0 tx=0).stop 2024-03-29T08:20:58.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.995+0000 7f1031000640 1 -- 172.21.15.46:0/4175400144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f102c15d940 msgr2=0x7f102c13d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:58.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.995+0000 7f1031000640 1 --2- 172.21.15.46:0/4175400144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f102c15d940 0x7f102c13d1e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f102400c760 tx=0x7f102400cc20 comp rx=0 tx=0).stop 2024-03-29T08:20:58.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.995+0000 7f1031000640 1 -- 172.21.15.46:0/4175400144 shutdown_connections 2024-03-29T08:20:58.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.995+0000 7f1031000640 1 --2- 172.21.15.46:0/4175400144 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f10080666c0 0x7f1008068b80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.995+0000 7f1031000640 1 --2- 172.21.15.46:0/4175400144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f102c15d940 0x7f102c13d1e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.995+0000 7f1031000640 1 --2- 172.21.15.46:0/4175400144 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f102c142ab0 0x7f102c13b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.995+0000 7f1031000640 1 --2- 172.21.15.46:0/4175400144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f102c06dbb0 0x7f102c13a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:58.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.995+0000 7f1031000640 1 -- 172.21.15.46:0/4175400144 >> 172.21.15.46:0/4175400144 conn(0x7f102c066080 msgr2=0x7f102c15bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:58.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.995+0000 7f1031000640 1 -- 172.21.15.46:0/4175400144 shutdown_connections 2024-03-29T08:20:58.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:58.995+0000 7f1031000640 1 -- 172.21.15.46:0/4175400144 wait complete. 2024-03-29T08:20:59.003 INFO:tasks.ceph.mds.a.smithi046.stderr: -4732> 2024-03-29T08:20:13.983+0000 7ff544a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:20:59.007 INFO:tasks.ceph.mds.a.smithi046.stderr: -3047> 2024-03-29T08:20:21.527+0000 7ff540400640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5775=0-4194304@1} caps={5775=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=5775 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x559570b12100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-29T08:20:59.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T08:20:59.055 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-03-29T08:20:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.159+0000 7f0097200640 1 Processor -- start 2024-03-29T08:20:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.159+0000 7f0097200640 1 -- start start 2024-03-29T08:20:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.163+0000 7f0097200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f009815aca0 0x7f009806f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.163+0000 7f0097200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f009815b080 con 0x7f00980cd400 2024-03-29T08:20:59.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.163+0000 7f0097200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00980cf4c0 con 0x7f009806f780 2024-03-29T08:20:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.163+0000 7f0097200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00980ad150 con 0x7f009815aca0 2024-03-29T08:20:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.163+0000 7f0095e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f009815aca0 0x7f009806f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.163+0000 7f0095e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f009815aca0 0x7f009806f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45764/0 (socket says 172.21.15.46:45764) 2024-03-29T08:20:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.163+0000 7f0096800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f00980cd400 0x7f00980cb3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37366/0 (socket says 172.21.15.46:37366) 2024-03-29T08:20:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.163+0000 7f0095e00640 1 -- 172.21.15.46:0/2172387313 learned_addr learned my addr 172.21.15.46:0/2172387313 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.163+0000 7f008d400640 1 --1- 172.21.15.46:0/2172387313 >> v1:172.21.15.73:6789/0 conn(0x7f009806f780 0x7f00980cccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:47484/0 (socket says 172.21.15.46:47484) 2024-03-29T08:20:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.163+0000 7f0095e00640 1 -- 172.21.15.46:0/2172387313 >> v1:172.21.15.73:6789/0 conn(0x7f009806f780 legacy=0x7f00980cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.163+0000 7f0095e00640 1 -- 172.21.15.46:0/2172387313 >> v1:172.21.15.46:6789/0 conn(0x7f00980cd400 legacy=0x7f00980cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.163+0000 7f0095400640 1 -- 172.21.15.46:0/2172387313 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2877504722 0 0) 0x7f00900014a0 con 0x7f00980cd400 2024-03-29T08:20:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.163+0000 7f0095e00640 1 -- 172.21.15.46:0/2172387313 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00980d11a0 con 0x7f009815aca0 2024-03-29T08:20:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.163+0000 7f0095e00640 1 --2- 172.21.15.46:0/2172387313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009815aca0 0x7f009806f240 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f00880097f0 tx=0x7f008801bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24e1739e078dd63 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.163+0000 7f0095400640 1 -- 172.21.15.46:0/2172387313 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00880a4790 con 0x7f009815aca0 2024-03-29T08:20:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.163+0000 7f0095400640 1 -- 172.21.15.46:0/2172387313 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00880a48f0 con 0x7f009815aca0 2024-03-29T08:20:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0095400640 1 -- 172.21.15.46:0/2172387313 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00880a4c80 con 0x7f009815aca0 2024-03-29T08:20:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0097200640 1 -- 172.21.15.46:0/2172387313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009815aca0 msgr2=0x7f009806f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0097200640 1 --2- 172.21.15.46:0/2172387313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009815aca0 0x7f009806f240 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f00880097f0 tx=0x7f008801bfb0 comp rx=0 tx=0).stop 2024-03-29T08:20:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0097200640 1 -- 172.21.15.46:0/2172387313 shutdown_connections 2024-03-29T08:20:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0097200640 1 --2- 172.21.15.46:0/2172387313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f009815aca0 0x7f009806f240 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:59.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0097200640 1 -- 172.21.15.46:0/2172387313 >> 172.21.15.46:0/2172387313 conn(0x7f00980d0550 msgr2=0x7f00980d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:20:59.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0097200640 1 -- 172.21.15.46:0/2172387313 shutdown_connections 2024-03-29T08:20:59.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0097200640 1 -- 172.21.15.46:0/2172387313 wait complete. 2024-03-29T08:20:59.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0097200640 1 Processor -- start 2024-03-29T08:20:59.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0097200640 1 -- start start 2024-03-29T08:20:59.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0097200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f009806f780 0x7f009814f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:59.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0097200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00980cd400 0x7f00980c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:59.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0097200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f009815aca0 0x7f00980c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:59.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0097200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009815b7e0 con 0x7f00980cd400 2024-03-29T08:20:59.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0097200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009815b220 con 0x7f009815aca0 2024-03-29T08:20:59.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0097200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00980ca180 con 0x7f009806f780 2024-03-29T08:20:59.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f008d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00980cd400 0x7f00980c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:59.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f008d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00980cd400 0x7f00980c96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45776/0 (socket says 172.21.15.46:45776) 2024-03-29T08:20:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f008d400640 1 -- 172.21.15.46:0/4280564102 learned_addr learned my addr 172.21.15.46:0/4280564102 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:20:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0095e00640 1 --2- 172.21.15.46:0/4280564102 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f009806f780 0x7f009814f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f0096800640 1 --2- 172.21.15.46:0/4280564102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f009815aca0 0x7f00980c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f008d400640 1 -- 172.21.15.46:0/4280564102 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f009806f780 msgr2=0x7f009814f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f008d400640 1 --2- 172.21.15.46:0/4280564102 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f009806f780 0x7f009814f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f008d400640 1 -- 172.21.15.46:0/4280564102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f009815aca0 msgr2=0x7f00980c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:20:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f008d400640 1 --2- 172.21.15.46:0/4280564102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f009815aca0 0x7f00980c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:20:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.167+0000 7f008d400640 1 -- 172.21.15.46:0/4280564102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00880094d0 con 0x7f00980cd400 2024-03-29T08:20:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.171+0000 7f008d400640 1 --2- 172.21.15.46:0/4280564102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00980cd400 0x7f00980c96c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f008000c780 tx=0x7f008000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.171+0000 7f008ea00640 1 -- 172.21.15.46:0/4280564102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00800072d0 con 0x7f00980cd400 2024-03-29T08:20:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.171+0000 7f008ea00640 1 -- 172.21.15.46:0/4280564102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0080007450 con 0x7f00980cd400 2024-03-29T08:20:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.171+0000 7f0097200640 1 -- 172.21.15.46:0/4280564102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00980ca460 con 0x7f00980cd400 2024-03-29T08:20:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.171+0000 7f008ea00640 1 -- 172.21.15.46:0/4280564102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00800047f0 con 0x7f00980cd400 2024-03-29T08:20:59.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.171+0000 7f0097200640 1 -- 172.21.15.46:0/4280564102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009814fce0 con 0x7f00980cd400 2024-03-29T08:20:59.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.171+0000 7f0097200640 1 -- 172.21.15.46:0/4280564102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f009815b640 con 0x7f00980cd400 2024-03-29T08:20:59.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.171+0000 7f008ea00640 1 -- 172.21.15.46:0/4280564102 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0080004950 con 0x7f00980cd400 2024-03-29T08:20:59.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.171+0000 7f008ea00640 1 --2- 172.21.15.46:0/4280564102 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0074066690 0x7f0074068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:20:59.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.171+0000 7f008ea00640 1 -- 172.21.15.46:0/4280564102 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f0080083480 con 0x7f00980cd400 2024-03-29T08:20:59.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.171+0000 7f0095e00640 1 --2- 172.21.15.46:0/4280564102 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0074066690 0x7f0074068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:20:59.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.175+0000 7f0095e00640 1 --2- 172.21.15.46:0/4280564102 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0074066690 0x7f0074068b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f00880ae000 tx=0x7f00880a6070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:20:59.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.175+0000 7f008ea00640 1 -- 172.21.15.46:0/4280564102 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f009815b640 con 0x7f00980cd400 2024-03-29T08:20:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:20:59.371+0000 7f0097200640 1 -- 172.21.15.46:0/4280564102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f009815b640 con 0x7f00980cd400 2024-03-29T08:21:00.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.015+0000 7f008ea00640 1 -- 172.21.15.46:0/4280564102 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f009815b640 con 0x7f00980cd400 2024-03-29T08:21:00.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.019+0000 7f0097200640 1 -- 172.21.15.46:0/4280564102 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0074066690 msgr2=0x7f0074068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:00.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.019+0000 7f0097200640 1 --2- 172.21.15.46:0/4280564102 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0074066690 0x7f0074068b50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f00880ae000 tx=0x7f00880a6070 comp rx=0 tx=0).stop 2024-03-29T08:21:00.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.019+0000 7f0097200640 1 -- 172.21.15.46:0/4280564102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00980cd400 msgr2=0x7f00980c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:00.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.019+0000 7f0097200640 1 --2- 172.21.15.46:0/4280564102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00980cd400 0x7f00980c96c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f008000c780 tx=0x7f008000cc40 comp rx=0 tx=0).stop 2024-03-29T08:21:00.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.019+0000 7f0097200640 1 -- 172.21.15.46:0/4280564102 shutdown_connections 2024-03-29T08:21:00.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.019+0000 7f0097200640 1 --2- 172.21.15.46:0/4280564102 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0074066690 0x7f0074068b50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:00.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.019+0000 7f0097200640 1 --2- 172.21.15.46:0/4280564102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f009815aca0 0x7f00980c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:00.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.019+0000 7f0097200640 1 --2- 172.21.15.46:0/4280564102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00980cd400 0x7f00980c96c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.019+0000 7f0097200640 1 --2- 172.21.15.46:0/4280564102 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f009806f780 0x7f009814f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.019+0000 7f0097200640 1 -- 172.21.15.46:0/4280564102 >> 172.21.15.46:0/4280564102 conn(0x7f00980d0550 msgr2=0x7f00980ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.019+0000 7f0097200640 1 -- 172.21.15.46:0/4280564102 shutdown_connections 2024-03-29T08:21:00.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.019+0000 7f0097200640 1 -- 172.21.15.46:0/4280564102 wait complete. 2024-03-29T08:21:00.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T08:21:00.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f757ac00640 1 Processor -- start 2024-03-29T08:21:00.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f757ac00640 1 -- start start 2024-03-29T08:21:00.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f757ac00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f75741515e0 0x7f757406d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:00.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f757ac00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f75740afe60 0x7f75740b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:00.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f757ac00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75740c3670 0x7f75740c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:00.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f757ac00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75740b3300 con 0x7f75741515e0 2024-03-29T08:21:00.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f757ac00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75740b0830 con 0x7f75740afe60 2024-03-29T08:21:00.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f757ac00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75741519c0 con 0x7f75740c3670 2024-03-29T08:21:00.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f7579800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f75741515e0 0x7f757406d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:00.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f7579800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f75741515e0 0x7f757406d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45782/0 (socket says 172.21.15.46:45782) 2024-03-29T08:21:00.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f7579800640 1 -- 172.21.15.46:0/2548736808 learned_addr learned my addr 172.21.15.46:0/2548736808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:00.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f757a200640 1 --2- 172.21.15.46:0/2548736808 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75740c3670 0x7f75740c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:00.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f7578e00640 1 --2- 172.21.15.46:0/2548736808 >> v2:172.21.15.73:3300/0 conn(0x7f75740afe60 0x7f75740b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:00.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f7579800640 1 -- 172.21.15.46:0/2548736808 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75740c3670 msgr2=0x7f75740c3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:00.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f7579800640 1 --2- 172.21.15.46:0/2548736808 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75740c3670 0x7f75740c3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:00.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f7579800640 1 -- 172.21.15.46:0/2548736808 >> v2:172.21.15.73:3300/0 conn(0x7f75740afe60 msgr2=0x7f75740b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:00.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f7579800640 1 --2- 172.21.15.46:0/2548736808 >> v2:172.21.15.73:3300/0 conn(0x7f75740afe60 0x7f75740b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:00.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f7579800640 1 -- 172.21.15.46:0/2548736808 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75740b3020 con 0x7f75741515e0 2024-03-29T08:21:00.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.235+0000 7f7578e00640 1 --2- 172.21.15.46:0/2548736808 >> v2:172.21.15.73:3300/0 conn(0x7f75740afe60 0x7f75740b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:00.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.239+0000 7f7579800640 1 --2- 172.21.15.46:0/2548736808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75741515e0 0x7f757406d850 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f75740c5210 tx=0x7f756801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff61d1bee0b8998 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:00.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.239+0000 7f7573e00640 1 -- 172.21.15.46:0/2548736808 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75680aa610 con 0x7f75741515e0 2024-03-29T08:21:00.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.239+0000 7f7573e00640 1 -- 172.21.15.46:0/2548736808 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75680a6040 con 0x7f75741515e0 2024-03-29T08:21:00.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.239+0000 7f7573e00640 1 -- 172.21.15.46:0/2548736808 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75680aa990 con 0x7f75741515e0 2024-03-29T08:21:00.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.239+0000 7f757ac00640 1 -- 172.21.15.46:0/2548736808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75741515e0 msgr2=0x7f757406d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:00.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.239+0000 7f757ac00640 1 --2- 172.21.15.46:0/2548736808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75741515e0 0x7f757406d850 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f75740c5210 tx=0x7f756801c4f0 comp rx=0 tx=0).stop 2024-03-29T08:21:00.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.239+0000 7f757ac00640 1 -- 172.21.15.46:0/2548736808 shutdown_connections 2024-03-29T08:21:00.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.239+0000 7f757ac00640 1 --2- 172.21.15.46:0/2548736808 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75740c3670 0x7f75740c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:00.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.239+0000 7f757ac00640 1 --2- 172.21.15.46:0/2548736808 >> v2:172.21.15.73:3300/0 conn(0x7f75740afe60 0x7f75740b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:00.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.239+0000 7f757ac00640 1 --2- 172.21.15.46:0/2548736808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75741515e0 0x7f757406d850 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:00.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.239+0000 7f757ac00640 1 -- 172.21.15.46:0/2548736808 >> 172.21.15.46:0/2548736808 conn(0x7f7574069d90 msgr2=0x7f757406a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:00.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.239+0000 7f757ac00640 1 -- 172.21.15.46:0/2548736808 shutdown_connections 2024-03-29T08:21:00.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.239+0000 7f757ac00640 1 -- 172.21.15.46:0/2548736808 wait complete. 2024-03-29T08:21:00.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.239+0000 7f757ac00640 1 Processor -- start 2024-03-29T08:21:00.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f757ac00640 1 -- start start 2024-03-29T08:21:00.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f757ac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75740afe60 0x7f75740acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:00.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f757ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75740c3670 0x7f7574148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:00.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f757ac00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75741515e0 0x7f757415cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:00.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f757ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757414e1c0 con 0x7f75740c3670 2024-03-29T08:21:00.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f757ac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75740cce40 con 0x7f75740afe60 2024-03-29T08:21:00.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f757ac00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75740cc2c0 con 0x7f75741515e0 2024-03-29T08:21:00.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f7578e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75740c3670 0x7f7574148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f7578e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75740c3670 0x7f7574148930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45788/0 (socket says 172.21.15.46:45788) 2024-03-29T08:21:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f7578e00640 1 -- 172.21.15.46:0/999758025 learned_addr learned my addr 172.21.15.46:0/999758025 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f7579800640 1 --2- 172.21.15.46:0/999758025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75740afe60 0x7f75740acf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f757a200640 1 --2- 172.21.15.46:0/999758025 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75741515e0 0x7f757415cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f7578e00640 1 -- 172.21.15.46:0/999758025 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75741515e0 msgr2=0x7f757415cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f7578e00640 1 --2- 172.21.15.46:0/999758025 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75741515e0 0x7f757415cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f7578e00640 1 -- 172.21.15.46:0/999758025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75740afe60 msgr2=0x7f75740acf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f7578e00640 1 --2- 172.21.15.46:0/999758025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75740afe60 0x7f75740acf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:00.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f7578e00640 1 -- 172.21.15.46:0/999758025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75680094d0 con 0x7f75740c3670 2024-03-29T08:21:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f7578e00640 1 --2- 172.21.15.46:0/999758025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75740c3670 0x7f7574148930 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f756000d800 tx=0x7f756000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f7571600640 1 -- 172.21.15.46:0/999758025 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75600096c0 con 0x7f75740c3670 2024-03-29T08:21:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f7571600640 1 -- 172.21.15.46:0/999758025 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7560009820 con 0x7f75740c3670 2024-03-29T08:21:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f757ac00640 1 -- 172.21.15.46:0/999758025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75740cc5a0 con 0x7f75740c3670 2024-03-29T08:21:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f7571600640 1 -- 172.21.15.46:0/999758025 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560009bd0 con 0x7f75740c3670 2024-03-29T08:21:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f757ac00640 1 -- 172.21.15.46:0/999758025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75740ccb60 con 0x7f75740c3670 2024-03-29T08:21:00.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.243+0000 7f757ac00640 1 -- 172.21.15.46:0/999758025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757404e040 con 0x7f75740c3670 2024-03-29T08:21:00.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.247+0000 7f7571600640 1 -- 172.21.15.46:0/999758025 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7560009d30 con 0x7f75740c3670 2024-03-29T08:21:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.247+0000 7f7571600640 1 --2- 172.21.15.46:0/999758025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7554066690 0x7f7554068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.247+0000 7f7571600640 1 -- 172.21.15.46:0/999758025 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f7560084770 con 0x7f75740c3670 2024-03-29T08:21:00.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.247+0000 7f7579800640 1 --2- 172.21.15.46:0/999758025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7554066690 0x7f7554068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:00.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.247+0000 7f7579800640 1 --2- 172.21.15.46:0/999758025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7554066690 0x7f7554068b50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f75680ae000 tx=0x7f75680b5040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:00.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.251+0000 7f7571600640 1 -- 172.21.15.46:0/999758025 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7560051b30 con 0x7f75740c3670 2024-03-29T08:21:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:00.415+0000 7f757ac00640 1 -- 172.21.15.46:0/999758025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f757406a210 con 0x7f75740c3670 2024-03-29T08:21:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.019+0000 7f7571600640 1 -- 172.21.15.46:0/999758025 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v53) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7560055150 con 0x7f75740c3670 2024-03-29T08:21:01.021 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-03-29T08:21:01.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.023+0000 7f757ac00640 1 -- 172.21.15.46:0/999758025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7554066690 msgr2=0x7f7554068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:01.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.023+0000 7f757ac00640 1 --2- 172.21.15.46:0/999758025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7554066690 0x7f7554068b50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f75680ae000 tx=0x7f75680b5040 comp rx=0 tx=0).stop 2024-03-29T08:21:01.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.023+0000 7f757ac00640 1 -- 172.21.15.46:0/999758025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75740c3670 msgr2=0x7f7574148930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.023+0000 7f757ac00640 1 --2- 172.21.15.46:0/999758025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75740c3670 0x7f7574148930 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f756000d800 tx=0x7f756000dcc0 comp rx=0 tx=0).stop 2024-03-29T08:21:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.023+0000 7f757ac00640 1 -- 172.21.15.46:0/999758025 shutdown_connections 2024-03-29T08:21:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.023+0000 7f757ac00640 1 --2- 172.21.15.46:0/999758025 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7554066690 0x7f7554068b50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.023+0000 7f757ac00640 1 --2- 172.21.15.46:0/999758025 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75741515e0 0x7f757415cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.023+0000 7f757ac00640 1 --2- 172.21.15.46:0/999758025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75740c3670 0x7f7574148930 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.023+0000 7f757ac00640 1 --2- 172.21.15.46:0/999758025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75740afe60 0x7f75740acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.023+0000 7f757ac00640 1 -- 172.21.15.46:0/999758025 >> 172.21.15.46:0/999758025 conn(0x7f7574069d90 msgr2=0x7f75740ad690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.023+0000 7f757ac00640 1 -- 172.21.15.46:0/999758025 shutdown_connections 2024-03-29T08:21:01.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.023+0000 7f757ac00640 1 -- 172.21.15.46:0/999758025 wait complete. 2024-03-29T08:21:01.047 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T08:21:01.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.187+0000 7f467f400640 1 Processor -- start 2024-03-29T08:21:01.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.187+0000 7f467f400640 1 -- start start 2024-03-29T08:21:01.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.191+0000 7f467f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f46781598a0 0x7f46780cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:01.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.191+0000 7f467f400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f467815dc30 0x7f467806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:01.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.191+0000 7f467f400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46780b1bc0 con 0x7f46780b1470 2024-03-29T08:21:01.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.191+0000 7f467f400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46780b1ea0 con 0x7f46781598a0 2024-03-29T08:21:01.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.191+0000 7f467f400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46780b2180 con 0x7f467815dc30 2024-03-29T08:21:01.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.191+0000 7f467ca00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f46781598a0 0x7f46780cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:01.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.191+0000 7f467d400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f467815dc30 0x7f467806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:01.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.191+0000 7f467d400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f467815dc30 0x7f467806c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:35118/0 (socket says 172.21.15.46:35118) 2024-03-29T08:21:01.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.191+0000 7f467d400640 1 -- 172.21.15.46:0/1427563740 learned_addr learned my addr 172.21.15.46:0/1427563740 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:01.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.191+0000 7f466fe00640 1 --1- 172.21.15.46:0/1427563740 >> v1:172.21.15.73:6789/0 conn(0x7f46780b1470 0x7f467815d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:47492/0 (socket says 172.21.15.46:47492) 2024-03-29T08:21:01.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.191+0000 7f467ca00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f46781598a0 0x7f46780cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51664/0 (socket says 172.21.15.46:51664) 2024-03-29T08:21:01.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.191+0000 7f4677e00640 1 -- 172.21.15.46:0/1427563740 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695915205 0 0) 0x7f46780b1bc0 con 0x7f46780b1470 2024-03-29T08:21:01.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.191+0000 7f467d400640 1 -- 172.21.15.46:0/1427563740 >> v2:172.21.15.73:3300/0 conn(0x7f46781598a0 msgr2=0x7f46780cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:01.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.191+0000 7f467d400640 1 --2- 172.21.15.46:0/1427563740 >> v2:172.21.15.73:3300/0 conn(0x7f46781598a0 0x7f46780cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:01.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.191+0000 7f467d400640 1 -- 172.21.15.46:0/1427563740 >> v1:172.21.15.73:6789/0 conn(0x7f46780b1470 legacy=0x7f467815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:01.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.191+0000 7f467d400640 1 -- 172.21.15.46:0/1427563740 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46780b2460 con 0x7f467815dc30 2024-03-29T08:21:01.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.195+0000 7f467d400640 1 --2- 172.21.15.46:0/1427563740 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f467815dc30 0x7f467806c6c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f467000d150 tx=0x7f467001f660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67ff8b9fd452ad86 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:01.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.195+0000 7f4677e00640 1 -- 172.21.15.46:0/1427563740 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46700a7cf0 con 0x7f467815dc30 2024-03-29T08:21:01.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.195+0000 7f4677e00640 1 -- 172.21.15.46:0/1427563740 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46700a7e50 con 0x7f467815dc30 2024-03-29T08:21:01.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.195+0000 7f4677e00640 1 -- 172.21.15.46:0/1427563740 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467000b2f0 con 0x7f467815dc30 2024-03-29T08:21:01.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.195+0000 7f467f400640 1 -- 172.21.15.46:0/1427563740 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f467815dc30 msgr2=0x7f467806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:01.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.195+0000 7f467f400640 1 --2- 172.21.15.46:0/1427563740 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f467815dc30 0x7f467806c6c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f467000d150 tx=0x7f467001f660 comp rx=0 tx=0).stop 2024-03-29T08:21:01.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.195+0000 7f467f400640 1 -- 172.21.15.46:0/1427563740 shutdown_connections 2024-03-29T08:21:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.195+0000 7f467f400640 1 --2- 172.21.15.46:0/1427563740 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f467815dc30 0x7f467806c6c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.195+0000 7f467f400640 1 --2- 172.21.15.46:0/1427563740 >> v2:172.21.15.73:3300/0 conn(0x7f46781598a0 0x7f46780cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.195+0000 7f467f400640 1 -- 172.21.15.46:0/1427563740 >> 172.21.15.46:0/1427563740 conn(0x7f46780b42d0 msgr2=0x7f4678069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.195+0000 7f467f400640 1 -- 172.21.15.46:0/1427563740 shutdown_connections 2024-03-29T08:21:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.195+0000 7f467f400640 1 -- 172.21.15.46:0/1427563740 wait complete. 2024-03-29T08:21:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.195+0000 7f467f400640 1 Processor -- start 2024-03-29T08:21:01.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.195+0000 7f467f400640 1 -- start start 2024-03-29T08:21:01.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f467f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46780b1470 0x7f467814e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:01.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f467f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46781598a0 0x7f467814d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:01.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f467f400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f467815dc30 0x7f467815bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:01.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f467f400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467814f1a0 con 0x7f46781598a0 2024-03-29T08:21:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f467f400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467814d6d0 con 0x7f46780b1470 2024-03-29T08:21:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f467f400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46780cd9c0 con 0x7f467815dc30 2024-03-29T08:21:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f466fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46781598a0 0x7f467814d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f466fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46781598a0 0x7f467814d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45796/0 (socket says 172.21.15.46:45796) 2024-03-29T08:21:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f466fe00640 1 -- 172.21.15.46:0/2424819580 learned_addr learned my addr 172.21.15.46:0/2424819580 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f467d400640 1 --2- 172.21.15.46:0/2424819580 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f467815dc30 0x7f467815bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f467ca00640 1 --2- 172.21.15.46:0/2424819580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46780b1470 0x7f467814e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f466fe00640 1 -- 172.21.15.46:0/2424819580 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f467815dc30 msgr2=0x7f467815bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f466fe00640 1 --2- 172.21.15.46:0/2424819580 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f467815dc30 0x7f467815bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f466fe00640 1 -- 172.21.15.46:0/2424819580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46780b1470 msgr2=0x7f467814e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f466fe00640 1 --2- 172.21.15.46:0/2424819580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46780b1470 0x7f467814e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:01.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f466fe00640 1 -- 172.21.15.46:0/2424819580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467000bd40 con 0x7f46781598a0 2024-03-29T08:21:01.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f466fe00640 1 --2- 172.21.15.46:0/2424819580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46781598a0 0x7f467814d190 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f466000c4e0 tx=0x7f466000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:01.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f4675600640 1 -- 172.21.15.46:0/2424819580 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660007480 con 0x7f46781598a0 2024-03-29T08:21:01.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f4675600640 1 -- 172.21.15.46:0/2424819580 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4660007620 con 0x7f46781598a0 2024-03-29T08:21:01.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f4675600640 1 -- 172.21.15.46:0/2424819580 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660004a30 con 0x7f46781598a0 2024-03-29T08:21:01.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f467f400640 1 -- 172.21.15.46:0/2424819580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46780cdca0 con 0x7f46781598a0 2024-03-29T08:21:01.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f467f400640 1 -- 172.21.15.46:0/2424819580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46780ce260 con 0x7f46781598a0 2024-03-29T08:21:01.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f467f400640 1 -- 172.21.15.46:0/2424819580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4678066c90 con 0x7f46781598a0 2024-03-29T08:21:01.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.199+0000 7f4675600640 1 -- 172.21.15.46:0/2424819580 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4660004b90 con 0x7f46781598a0 2024-03-29T08:21:01.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.203+0000 7f4675600640 1 --2- 172.21.15.46:0/2424819580 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4654066690 0x7f4654068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:01.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.203+0000 7f4675600640 1 -- 172.21.15.46:0/2424819580 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f4660083270 con 0x7f46781598a0 2024-03-29T08:21:01.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.203+0000 7f467ca00640 1 --2- 172.21.15.46:0/2424819580 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4654066690 0x7f4654068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:01.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.203+0000 7f467ca00640 1 --2- 172.21.15.46:0/2424819580 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4654066690 0x7f4654068b50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f467815d390 tx=0x7f4668009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:01.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.207+0000 7f4675600640 1 -- 172.21.15.46:0/2424819580 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4660050810 con 0x7f46781598a0 2024-03-29T08:21:01.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:01.367+0000 7f467f400640 1 -- 172.21.15.46:0/2424819580 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4678164690 con 0x7f46781598a0 2024-03-29T08:21:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.027+0000 7f4675600640 1 -- 172.21.15.46:0/2424819580 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v54) v1 ==== 169+0+0 (secure 0 0 0) 0x7f466005a030 con 0x7f46781598a0 2024-03-29T08:21:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-03-29T08:21:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.031+0000 7f467f400640 1 -- 172.21.15.46:0/2424819580 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4654066690 msgr2=0x7f4654068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.031+0000 7f467f400640 1 --2- 172.21.15.46:0/2424819580 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4654066690 0x7f4654068b50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f467815d390 tx=0x7f4668009190 comp rx=0 tx=0).stop 2024-03-29T08:21:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.031+0000 7f467f400640 1 -- 172.21.15.46:0/2424819580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46781598a0 msgr2=0x7f467814d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.031+0000 7f467f400640 1 --2- 172.21.15.46:0/2424819580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46781598a0 0x7f467814d190 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f466000c4e0 tx=0x7f466000c9a0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.031+0000 7f467f400640 1 -- 172.21.15.46:0/2424819580 shutdown_connections 2024-03-29T08:21:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.031+0000 7f467f400640 1 --2- 172.21.15.46:0/2424819580 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4654066690 0x7f4654068b50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.031+0000 7f467f400640 1 --2- 172.21.15.46:0/2424819580 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f467815dc30 0x7f467815bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.031+0000 7f467f400640 1 --2- 172.21.15.46:0/2424819580 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46781598a0 0x7f467814d190 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.031+0000 7f467f400640 1 --2- 172.21.15.46:0/2424819580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46780b1470 0x7f467814e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.031+0000 7f467f400640 1 -- 172.21.15.46:0/2424819580 >> 172.21.15.46:0/2424819580 conn(0x7f46780b42d0 msgr2=0x7f46781523a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.031+0000 7f467f400640 1 -- 172.21.15.46:0/2424819580 shutdown_connections 2024-03-29T08:21:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.031+0000 7f467f400640 1 -- 172.21.15.46:0/2424819580 wait complete. 2024-03-29T08:21:02.057 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-29T08:21:02.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a7ac00640 1 Processor -- start 2024-03-29T08:21:02.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a7ac00640 1 -- start start 2024-03-29T08:21:02.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a7ac00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6a740d0dc0 0x7f6a740cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:02.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a7ac00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7415cf40 con 0x7f6a7415cb60 2024-03-29T08:21:02.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a7ac00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a740b0250 con 0x7f6a740cf820 2024-03-29T08:21:02.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a7ac00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a740b0810 con 0x7f6a740d0dc0 2024-03-29T08:21:02.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a73e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6a7415cb60 0x7f6a740d06b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37382/0 (socket says 172.21.15.46:37382) 2024-03-29T08:21:02.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a73e00640 1 -- 172.21.15.46:0/4148324414 learned_addr learned my addr 172.21.15.46:0/4148324414 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:02.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a6b400640 1 --2- 172.21.15.46:0/4148324414 >> v2:172.21.15.73:3300/0 conn(0x7f6a740d0dc0 0x7f6a740cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:02.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a73400640 1 -- 172.21.15.46:0/4148324414 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062932801 0 0) 0x7f6a7415cf40 con 0x7f6a7415cb60 2024-03-29T08:21:02.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a73400640 1 -- 172.21.15.46:0/4148324414 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a58003530 con 0x7f6a7415cb60 2024-03-29T08:21:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a73400640 1 -- 172.21.15.46:0/4148324414 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 780704555 0 0) 0x7f6a58003530 con 0x7f6a7415cb60 2024-03-29T08:21:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a73400640 1 -- 172.21.15.46:0/4148324414 >> v2:172.21.15.73:3300/0 conn(0x7f6a740d0dc0 msgr2=0x7f6a740cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a73400640 1 --2- 172.21.15.46:0/4148324414 >> v2:172.21.15.73:3300/0 conn(0x7f6a740d0dc0 0x7f6a740cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a73400640 1 -- 172.21.15.46:0/4148324414 >> v1:172.21.15.73:6789/0 conn(0x7f6a740cf820 legacy=0x7f6a740b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a73400640 1 -- 172.21.15.46:0/4148324414 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a740d0010 con 0x7f6a7415cb60 2024-03-29T08:21:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a73400640 1 -- 172.21.15.46:0/4148324414 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f6a64001680 con 0x7f6a7415cb60 2024-03-29T08:21:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a73400640 1 -- 172.21.15.46:0/4148324414 >> v1:172.21.15.46:6789/0 conn(0x7f6a7415cb60 legacy=0x7f6a740d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a73400640 1 --2- 172.21.15.46:0/4148324414 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6a58003980 0x7f6a58003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:02.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.203+0000 7f6a73400640 1 --2- 172.21.15.46:0/4148324414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a58004380 0x7f6a58004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a73400640 1 --2- 172.21.15.46:0/4148324414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a58005050 0x7f6a5800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a73400640 1 -- 172.21.15.46:0/4148324414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a740b0810 con 0x7f6a58005050 2024-03-29T08:21:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a73400640 1 -- 172.21.15.46:0/4148324414 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5800bfb0 con 0x7f6a58004380 2024-03-29T08:21:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a73400640 1 -- 172.21.15.46:0/4148324414 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5800c170 con 0x7f6a58003980 2024-03-29T08:21:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a78c00640 1 --2- 172.21.15.46:0/4148324414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a58005050 0x7f6a5800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a73e00640 1 --2- 172.21.15.46:0/4148324414 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6a58003980 0x7f6a58003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a6b400640 1 --2- 172.21.15.46:0/4148324414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a58004380 0x7f6a58004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a78c00640 1 -- 172.21.15.46:0/4148324414 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6a58003980 msgr2=0x7f6a58003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a78c00640 1 --2- 172.21.15.46:0/4148324414 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6a58003980 0x7f6a58003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a78c00640 1 -- 172.21.15.46:0/4148324414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a58004380 msgr2=0x7f6a58004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a78c00640 1 --2- 172.21.15.46:0/4148324414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a58004380 0x7f6a58004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a78c00640 1 -- 172.21.15.46:0/4148324414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a58003530 con 0x7f6a58005050 2024-03-29T08:21:02.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a6b400640 1 --2- 172.21.15.46:0/4148324414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a58004380 0x7f6a58004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:02.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a78c00640 1 -- 172.21.15.46:0/4148324414 reap_dead start 2024-03-29T08:21:02.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a78c00640 1 --2- 172.21.15.46:0/4148324414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a58005050 0x7f6a5800b990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c001f60 tx=0x7f6a6c01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec819fe5324e2bed server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:02.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a73400640 1 -- 172.21.15.46:0/4148324414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c003b80 con 0x7f6a58005050 2024-03-29T08:21:02.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a73400640 1 -- 172.21.15.46:0/4148324414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c003ce0 con 0x7f6a58005050 2024-03-29T08:21:02.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.207+0000 7f6a73400640 1 -- 172.21.15.46:0/4148324414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c0052f0 con 0x7f6a58005050 2024-03-29T08:21:02.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a7ac00640 1 -- 172.21.15.46:0/4148324414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a58005050 msgr2=0x7f6a5800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:02.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a7ac00640 1 --2- 172.21.15.46:0/4148324414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a58005050 0x7f6a5800b990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c001f60 tx=0x7f6a6c01c880 comp rx=0 tx=0).stop 2024-03-29T08:21:02.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a7ac00640 1 -- 172.21.15.46:0/4148324414 shutdown_connections 2024-03-29T08:21:02.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a7ac00640 1 --2- 172.21.15.46:0/4148324414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a58005050 0x7f6a5800b990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a7ac00640 1 -- 172.21.15.46:0/4148324414 >> 172.21.15.46:0/4148324414 conn(0x7f6a74099a10 msgr2=0x7f6a74089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:02.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a7ac00640 1 -- 172.21.15.46:0/4148324414 shutdown_connections 2024-03-29T08:21:02.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a7ac00640 1 -- 172.21.15.46:0/4148324414 wait complete. 2024-03-29T08:21:02.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a7ac00640 1 Processor -- start 2024-03-29T08:21:02.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a7ac00640 1 -- start start 2024-03-29T08:21:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a7ac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a58005050 0x7f6a740c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a7ac00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6a740ca180 0x7f6a7415f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a7ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7415f650 0x7f6a7414b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a7ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a741f1dd0 con 0x7f6a7415f650 2024-03-29T08:21:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a7ac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7415e9a0 con 0x7f6a58005050 2024-03-29T08:21:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a7ac00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7415ec80 con 0x7f6a740ca180 2024-03-29T08:21:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a78c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7415f650 0x7f6a7414b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a78c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7415f650 0x7f6a7414b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45814/0 (socket says 172.21.15.46:45814) 2024-03-29T08:21:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a78c00640 1 -- 172.21.15.46:0/3313629358 learned_addr learned my addr 172.21.15.46:0/3313629358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a78c00640 1 -- 172.21.15.46:0/3313629358 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6a740ca180 msgr2=0x7f6a7415f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a78c00640 1 --2- 172.21.15.46:0/3313629358 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6a740ca180 0x7f6a7415f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a78c00640 1 -- 172.21.15.46:0/3313629358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a58005050 msgr2=0x7f6a740c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:02.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a78c00640 1 --2- 172.21.15.46:0/3313629358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a58005050 0x7f6a740c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a78c00640 1 -- 172.21.15.46:0/3313629358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a740c6c70 con 0x7f6a7415f650 2024-03-29T08:21:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.211+0000 7f6a78c00640 1 --2- 172.21.15.46:0/3313629358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7415f650 0x7f6a7414b980 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c01c6b0 tx=0x7f6a58005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.215+0000 7f6a70c00640 1 -- 172.21.15.46:0/3313629358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c005830 con 0x7f6a7415f650 2024-03-29T08:21:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.215+0000 7f6a70c00640 1 -- 172.21.15.46:0/3313629358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c0a6470 con 0x7f6a7415f650 2024-03-29T08:21:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.215+0000 7f6a70c00640 1 -- 172.21.15.46:0/3313629358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c0a6820 con 0x7f6a7415f650 2024-03-29T08:21:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.215+0000 7f6a7ac00640 1 -- 172.21.15.46:0/3313629358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a740c6f50 con 0x7f6a7415f650 2024-03-29T08:21:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.215+0000 7f6a7ac00640 1 -- 172.21.15.46:0/3313629358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a740ca600 con 0x7f6a7415f650 2024-03-29T08:21:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.215+0000 7f6a7ac00640 1 -- 172.21.15.46:0/3313629358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7409e7d0 con 0x7f6a7415f650 2024-03-29T08:21:02.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.215+0000 7f6a70c00640 1 -- 172.21.15.46:0/3313629358 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6a6c0a6980 con 0x7f6a7415f650 2024-03-29T08:21:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.215+0000 7f6a70c00640 1 --2- 172.21.15.46:0/3313629358 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a500666c0 0x7f6a50068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:02.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.215+0000 7f6a70c00640 1 -- 172.21.15.46:0/3313629358 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6a6c0b0070 con 0x7f6a7415f650 2024-03-29T08:21:02.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.215+0000 7f6a73e00640 1 --2- 172.21.15.46:0/3313629358 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a500666c0 0x7f6a50068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:02.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.219+0000 7f6a73e00640 1 --2- 172.21.15.46:0/3313629358 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a500666c0 0x7f6a50068b80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6a64001ec0 tx=0x7f6a64001c60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:02.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.219+0000 7f6a70c00640 1 -- 172.21.15.46:0/3313629358 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6a6c0ea700 con 0x7f6a7415f650 2024-03-29T08:21:02.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.387+0000 7f6a7ac00640 1 -- 172.21.15.46:0/3313629358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6a7415edc0 con 0x7f6a7415f650 2024-03-29T08:21:02.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.391+0000 7f6a70c00640 1 -- 172.21.15.46:0/3313629358 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6a6c0edd20 con 0x7f6a7415f650 2024-03-29T08:21:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.391+0000 7f6a7ac00640 1 -- 172.21.15.46:0/3313629358 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a500666c0 msgr2=0x7f6a50068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.391+0000 7f6a7ac00640 1 --2- 172.21.15.46:0/3313629358 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a500666c0 0x7f6a50068b80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6a64001ec0 tx=0x7f6a64001c60 comp rx=0 tx=0).stop 2024-03-29T08:21:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.391+0000 7f6a7ac00640 1 -- 172.21.15.46:0/3313629358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7415f650 msgr2=0x7f6a7414b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.391+0000 7f6a7ac00640 1 --2- 172.21.15.46:0/3313629358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7415f650 0x7f6a7414b980 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c01c6b0 tx=0x7f6a58005800 comp rx=0 tx=0).stop 2024-03-29T08:21:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.391+0000 7f6a7ac00640 1 -- 172.21.15.46:0/3313629358 shutdown_connections 2024-03-29T08:21:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.391+0000 7f6a7ac00640 1 --2- 172.21.15.46:0/3313629358 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a500666c0 0x7f6a50068b80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.391+0000 7f6a7ac00640 1 --2- 172.21.15.46:0/3313629358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a7415f650 0x7f6a7414b980 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.391+0000 7f6a7ac00640 1 --2- 172.21.15.46:0/3313629358 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6a740ca180 0x7f6a7415f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.391+0000 7f6a7ac00640 1 --2- 172.21.15.46:0/3313629358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a58005050 0x7f6a740c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.395+0000 7f6a7ac00640 1 -- 172.21.15.46:0/3313629358 >> 172.21.15.46:0/3313629358 conn(0x7f6a74099a10 msgr2=0x7f6a740cec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.395+0000 7f6a7ac00640 1 -- 172.21.15.46:0/3313629358 shutdown_connections 2024-03-29T08:21:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.395+0000 7f6a7ac00640 1 -- 172.21.15.46:0/3313629358 wait complete. 2024-03-29T08:21:02.416 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-03-29T08:21:02.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03dce00640 1 Processor -- start 2024-03-29T08:21:02.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03dce00640 1 -- start start 2024-03-29T08:21:02.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03dce00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03d815cb60 0x7f03d806bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:02.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03dce00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d80b1610 con 0x7f03d80d0e00 2024-03-29T08:21:02.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03dce00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d80b18f0 con 0x7f03d80cae00 2024-03-29T08:21:02.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03dce00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d806c2f0 con 0x7f03d815cb60 2024-03-29T08:21:02.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d6800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f03d80d0e00 0x7f03d80af390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37384/0 (socket says 172.21.15.46:37384) 2024-03-29T08:21:02.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d6800640 1 -- 172.21.15.46:0/2382866302 learned_addr learned my addr 172.21.15.46:0/2382866302 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:02.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5e00640 1 --2- 172.21.15.46:0/2382866302 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03d815cb60 0x7f03d806bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:02.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5400640 1 -- 172.21.15.46:0/2382866302 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012360884 0 0) 0x7f03d80b1610 con 0x7f03d80d0e00 2024-03-29T08:21:02.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5400640 1 -- 172.21.15.46:0/2382866302 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03bc003530 con 0x7f03d80d0e00 2024-03-29T08:21:02.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5400640 1 -- 172.21.15.46:0/2382866302 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 649709700 0 0) 0x7f03d80b18f0 con 0x7f03d80cae00 2024-03-29T08:21:02.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5400640 1 -- 172.21.15.46:0/2382866302 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03d80b1610 con 0x7f03d80cae00 2024-03-29T08:21:02.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5400640 1 -- 172.21.15.46:0/2382866302 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1760750984 0 0) 0x7f03bc003530 con 0x7f03d80d0e00 2024-03-29T08:21:02.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5400640 1 -- 172.21.15.46:0/2382866302 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03d815cb60 msgr2=0x7f03d806bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:02.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5400640 1 --2- 172.21.15.46:0/2382866302 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03d815cb60 0x7f03d806bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5400640 1 -- 172.21.15.46:0/2382866302 >> v1:172.21.15.73:6789/0 conn(0x7f03d80cae00 legacy=0x7f03d80d06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:02.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5400640 1 -- 172.21.15.46:0/2382866302 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d80ced20 con 0x7f03d80d0e00 2024-03-29T08:21:02.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5400640 1 -- 172.21.15.46:0/2382866302 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f03d0001980 con 0x7f03d80d0e00 2024-03-29T08:21:02.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5e00640 1 --2- 172.21.15.46:0/2382866302 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03d815cb60 0x7f03d806bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:21:02.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5400640 1 -- 172.21.15.46:0/2382866302 >> v1:172.21.15.46:6789/0 conn(0x7f03d80d0e00 legacy=0x7f03d80af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:02.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5400640 1 --2- 172.21.15.46:0/2382866302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03bc003b40 0x7f03bc003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:02.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5400640 1 --2- 172.21.15.46:0/2382866302 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03d815cb60 0x7f03bc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:02.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5400640 1 --2- 172.21.15.46:0/2382866302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03bc004d10 0x7f03bc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:02.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5400640 1 -- 172.21.15.46:0/2382866302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d806c2f0 con 0x7f03bc004d10 2024-03-29T08:21:02.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5400640 1 -- 172.21.15.46:0/2382866302 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc00ba40 con 0x7f03bc003b40 2024-03-29T08:21:02.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5400640 1 -- 172.21.15.46:0/2382866302 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc00bc00 con 0x7f03d815cb60 2024-03-29T08:21:02.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d6800640 1 --2- 172.21.15.46:0/2382866302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03bc004d10 0x7f03bc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:02.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03cd400640 1 --2- 172.21.15.46:0/2382866302 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03d815cb60 0x7f03bc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:02.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d5e00640 1 --2- 172.21.15.46:0/2382866302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03bc003b40 0x7f03bc003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:02.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d6800640 1 -- 172.21.15.46:0/2382866302 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03d815cb60 msgr2=0x7f03bc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:02.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d6800640 1 --2- 172.21.15.46:0/2382866302 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03d815cb60 0x7f03bc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d6800640 1 -- 172.21.15.46:0/2382866302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03bc003b40 msgr2=0x7f03bc003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:02.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d6800640 1 --2- 172.21.15.46:0/2382866302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03bc003b40 0x7f03bc003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.555+0000 7f03d6800640 1 -- 172.21.15.46:0/2382866302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c8009420 con 0x7f03bc004d10 2024-03-29T08:21:02.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.559+0000 7f03d6800640 1 --2- 172.21.15.46:0/2382866302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03bc004d10 0x7f03bc005190 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f03d000a650 tx=0x7f03d001f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8c65154e52aa68 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:02.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.559+0000 7f03d5400640 1 -- 172.21.15.46:0/2382866302 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d00070d0 con 0x7f03bc004d10 2024-03-29T08:21:02.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.559+0000 7f03d5400640 1 -- 172.21.15.46:0/2382866302 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d0007230 con 0x7f03bc004d10 2024-03-29T08:21:02.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.559+0000 7f03d5400640 1 -- 172.21.15.46:0/2382866302 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d00075e0 con 0x7f03bc004d10 2024-03-29T08:21:02.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.559+0000 7f03dce00640 1 -- 172.21.15.46:0/2382866302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03bc004d10 msgr2=0x7f03bc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:02.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.559+0000 7f03dce00640 1 --2- 172.21.15.46:0/2382866302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03bc004d10 0x7f03bc005190 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f03d000a650 tx=0x7f03d001f8c0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.559+0000 7f03d6800640 1 -- 172.21.15.46:0/2382866302 reap_dead start 2024-03-29T08:21:02.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.559+0000 7f03dce00640 1 -- 172.21.15.46:0/2382866302 shutdown_connections 2024-03-29T08:21:02.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.559+0000 7f03dce00640 1 -- 172.21.15.46:0/2382866302 >> 172.21.15.46:0/2382866302 conn(0x7f03d8089fd0 msgr2=0x7f03d808a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:02.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.559+0000 7f03dce00640 1 -- 172.21.15.46:0/2382866302 shutdown_connections 2024-03-29T08:21:02.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.559+0000 7f03dce00640 1 -- 172.21.15.46:0/2382866302 wait complete. 2024-03-29T08:21:02.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03dce00640 1 Processor -- start 2024-03-29T08:21:02.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03dce00640 1 -- start start 2024-03-29T08:21:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03dce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03bc004d10 0x7f03d81f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03dce00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03d81586f0 0x7f03d81f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03dce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03d806ee30 0x7f03d806f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03dce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d806fe40 con 0x7f03d806ee30 2024-03-29T08:21:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03dce00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d81566e0 con 0x7f03bc004d10 2024-03-29T08:21:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03dce00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d806fb60 con 0x7f03d81586f0 2024-03-29T08:21:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03d6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03d806ee30 0x7f03d806f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03d6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03d806ee30 0x7f03d806f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45840/0 (socket says 172.21.15.46:45840) 2024-03-29T08:21:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03d6800640 1 -- 172.21.15.46:0/723198232 learned_addr learned my addr 172.21.15.46:0/723198232 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03d5e00640 1 --2- 172.21.15.46:0/723198232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03bc004d10 0x7f03d81f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03cd400640 1 --2- 172.21.15.46:0/723198232 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03d81586f0 0x7f03d81f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:02.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03d6800640 1 -- 172.21.15.46:0/723198232 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03d81586f0 msgr2=0x7f03d81f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03d6800640 1 --2- 172.21.15.46:0/723198232 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03d81586f0 0x7f03d81f5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03d6800640 1 -- 172.21.15.46:0/723198232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03bc004d10 msgr2=0x7f03d81f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03d6800640 1 --2- 172.21.15.46:0/723198232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03bc004d10 0x7f03d81f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03d6800640 1 -- 172.21.15.46:0/723198232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d8156090 con 0x7f03d806ee30 2024-03-29T08:21:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03d6800640 1 --2- 172.21.15.46:0/723198232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03d806ee30 0x7f03d806f2b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f03d0007040 tx=0x7f03bc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03cea00640 1 -- 172.21.15.46:0/723198232 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0001980 con 0x7f03d806ee30 2024-03-29T08:21:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03cea00640 1 -- 172.21.15.46:0/723198232 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d00011d0 con 0x7f03d806ee30 2024-03-29T08:21:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03cea00640 1 -- 172.21.15.46:0/723198232 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d0004db0 con 0x7f03d806ee30 2024-03-29T08:21:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03dce00640 1 -- 172.21.15.46:0/723198232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d8156370 con 0x7f03d806ee30 2024-03-29T08:21:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.563+0000 7f03dce00640 1 -- 172.21.15.46:0/723198232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d81f14c0 con 0x7f03d806ee30 2024-03-29T08:21:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.567+0000 7f03dce00640 1 -- 172.21.15.46:0/723198232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d8155ef0 con 0x7f03d806ee30 2024-03-29T08:21:02.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.567+0000 7f03cea00640 1 -- 172.21.15.46:0/723198232 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f03d00bd070 con 0x7f03d806ee30 2024-03-29T08:21:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.567+0000 7f03cea00640 1 --2- 172.21.15.46:0/723198232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f03b4066690 0x7f03b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.567+0000 7f03cea00640 1 -- 172.21.15.46:0/723198232 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f03d01215d0 con 0x7f03d806ee30 2024-03-29T08:21:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.567+0000 7f03d5e00640 1 --2- 172.21.15.46:0/723198232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f03b4066690 0x7f03b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:02.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.567+0000 7f03d5e00640 1 --2- 172.21.15.46:0/723198232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f03b4066690 0x7f03b4068b50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009cb0 tx=0x7f03c8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:02.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.571+0000 7f03cea00640 1 -- 172.21.15.46:0/723198232 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f03d8155ef0 con 0x7f03d806ee30 2024-03-29T08:21:02.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:02.795+0000 7f03dce00640 1 -- 172.21.15.46:0/723198232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f03d8155ef0 con 0x7f03d806ee30 2024-03-29T08:21:03.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.031+0000 7f03cea00640 1 -- 172.21.15.46:0/723198232 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v161) v1 ==== 149+0+0 (secure 0 0 0) 0x7f03d8155ef0 con 0x7f03d806ee30 2024-03-29T08:21:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.031+0000 7f03dce00640 1 -- 172.21.15.46:0/723198232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f03b4066690 msgr2=0x7f03b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.031+0000 7f03dce00640 1 --2- 172.21.15.46:0/723198232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f03b4066690 0x7f03b4068b50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009cb0 tx=0x7f03c8008040 comp rx=0 tx=0).stop 2024-03-29T08:21:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.031+0000 7f03dce00640 1 -- 172.21.15.46:0/723198232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03d806ee30 msgr2=0x7f03d806f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:03.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.031+0000 7f03dce00640 1 --2- 172.21.15.46:0/723198232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03d806ee30 0x7f03d806f2b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f03d0007040 tx=0x7f03bc005800 comp rx=0 tx=0).stop 2024-03-29T08:21:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.031+0000 7f03dce00640 1 -- 172.21.15.46:0/723198232 shutdown_connections 2024-03-29T08:21:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.031+0000 7f03dce00640 1 --2- 172.21.15.46:0/723198232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f03b4066690 0x7f03b4068b50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.031+0000 7f03dce00640 1 --2- 172.21.15.46:0/723198232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03d806ee30 0x7f03d806f2b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.031+0000 7f03dce00640 1 --2- 172.21.15.46:0/723198232 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03d81586f0 0x7f03d81f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.031+0000 7f03dce00640 1 --2- 172.21.15.46:0/723198232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03bc004d10 0x7f03d81f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.031+0000 7f03dce00640 1 -- 172.21.15.46:0/723198232 >> 172.21.15.46:0/723198232 conn(0x7f03d8089fd0 msgr2=0x7f03d80cbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.031+0000 7f03dce00640 1 -- 172.21.15.46:0/723198232 shutdown_connections 2024-03-29T08:21:03.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.031+0000 7f03dce00640 1 -- 172.21.15.46:0/723198232 wait complete. 2024-03-29T08:21:03.055 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-03-29T08:21:03.055 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-03-29T08:21:03.055 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-29T08:21:03.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-03-29T08:21:03.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.195+0000 7f97ee400640 1 Processor -- start 2024-03-29T08:21:03.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.195+0000 7f97ee400640 1 -- start start 2024-03-29T08:21:03.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.195+0000 7f97ee400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f97e80d0660 0x7f97e80d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:03.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.195+0000 7f97ee400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f97e806b8b0 0x7f97e80cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:03.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.195+0000 7f97ee400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e815cfb0 con 0x7f97e815dc00 2024-03-29T08:21:03.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.195+0000 7f97ee400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e806c100 con 0x7f97e806b8b0 2024-03-29T08:21:03.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.195+0000 7f97ee400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e80d0f80 con 0x7f97e80d0660 2024-03-29T08:21:03.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.195+0000 7f97e7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f97e815dc00 0x7f97e80cf7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37390/0 (socket says 172.21.15.46:37390) 2024-03-29T08:21:03.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.195+0000 7f97e7e00640 1 -- 172.21.15.46:0/2066951124 learned_addr learned my addr 172.21.15.46:0/2066951124 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:03.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.195+0000 7f97e7400640 1 --2- 172.21.15.46:0/2066951124 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f97e80d0660 0x7f97e80d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.195+0000 7f97dea00640 1 --2- 172.21.15.46:0/2066951124 >> v2:172.21.15.73:3300/0 conn(0x7f97e806b8b0 0x7f97e80cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.195+0000 7f97e6a00640 1 -- 172.21.15.46:0/2066951124 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202444479 0 0) 0x7f97e815cfb0 con 0x7f97e815dc00 2024-03-29T08:21:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.195+0000 7f97e6a00640 1 -- 172.21.15.46:0/2066951124 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97cc003530 con 0x7f97e815dc00 2024-03-29T08:21:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.195+0000 7f97e7400640 1 -- 172.21.15.46:0/2066951124 >> v2:172.21.15.73:3300/0 conn(0x7f97e806b8b0 msgr2=0x7f97e80cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.195+0000 7f97e7400640 1 --2- 172.21.15.46:0/2066951124 >> v2:172.21.15.73:3300/0 conn(0x7f97e806b8b0 0x7f97e80cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.195+0000 7f97e7400640 1 -- 172.21.15.46:0/2066951124 >> v1:172.21.15.46:6789/0 conn(0x7f97e815dc00 legacy=0x7f97e80cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:03.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.195+0000 7f97e7400640 1 -- 172.21.15.46:0/2066951124 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e806be20 con 0x7f97e80d0660 2024-03-29T08:21:03.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.199+0000 7f97e7400640 1 --2- 172.21.15.46:0/2066951124 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f97e80d0660 0x7f97e80d0a40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f97d80097c0 tx=0x7f97d801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc95aad299b12f38 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:03.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.199+0000 7f97e6a00640 1 -- 172.21.15.46:0/2066951124 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d80ab670 con 0x7f97e80d0660 2024-03-29T08:21:03.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.199+0000 7f97e6a00640 1 -- 172.21.15.46:0/2066951124 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d80ab7d0 con 0x7f97e80d0660 2024-03-29T08:21:03.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.199+0000 7f97e6a00640 1 -- 172.21.15.46:0/2066951124 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d80abb40 con 0x7f97e80d0660 2024-03-29T08:21:03.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.199+0000 7f97ee400640 1 -- 172.21.15.46:0/2066951124 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f97e80d0660 msgr2=0x7f97e80d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:03.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.199+0000 7f97ee400640 1 --2- 172.21.15.46:0/2066951124 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f97e80d0660 0x7f97e80d0a40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f97d80097c0 tx=0x7f97d801c470 comp rx=0 tx=0).stop 2024-03-29T08:21:03.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.199+0000 7f97ee400640 1 -- 172.21.15.46:0/2066951124 shutdown_connections 2024-03-29T08:21:03.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.199+0000 7f97ee400640 1 --2- 172.21.15.46:0/2066951124 >> v2:172.21.15.73:3300/0 conn(0x7f97e806b8b0 0x7f97e80cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:03.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.199+0000 7f97ee400640 1 --2- 172.21.15.46:0/2066951124 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f97e80d0660 0x7f97e80d0a40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:03.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.199+0000 7f97ee400640 1 -- 172.21.15.46:0/2066951124 >> 172.21.15.46:0/2066951124 conn(0x7f97e8089fd0 msgr2=0x7f97e808a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:03.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.199+0000 7f97ee400640 1 -- 172.21.15.46:0/2066951124 shutdown_connections 2024-03-29T08:21:03.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.199+0000 7f97ee400640 1 -- 172.21.15.46:0/2066951124 wait complete. 2024-03-29T08:21:03.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97ee400640 1 Processor -- start 2024-03-29T08:21:03.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97ee400640 1 -- start start 2024-03-29T08:21:03.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97ee400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e806b8b0 0x7f97e806e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:03.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97ee400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e80d0660 0x7f97e8152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:03.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97ee400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f97e815dc00 0x7f97e80b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:03.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97ee400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8153020 con 0x7f97e80d0660 2024-03-29T08:21:03.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97ee400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e80c4ff0 con 0x7f97e806b8b0 2024-03-29T08:21:03.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97ee400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8152d40 con 0x7f97e815dc00 2024-03-29T08:21:03.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97dea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e80d0660 0x7f97e8152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:03.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97dea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e80d0660 0x7f97e8152630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45850/0 (socket says 172.21.15.46:45850) 2024-03-29T08:21:03.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97dea00640 1 -- 172.21.15.46:0/3650180040 learned_addr learned my addr 172.21.15.46:0/3650180040 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:03.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97e7400640 1 --2- 172.21.15.46:0/3650180040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e806b8b0 0x7f97e806e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:03.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97e7e00640 1 --2- 172.21.15.46:0/3650180040 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f97e815dc00 0x7f97e80b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:03.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97dea00640 1 -- 172.21.15.46:0/3650180040 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f97e815dc00 msgr2=0x7f97e80b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:03.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97dea00640 1 --2- 172.21.15.46:0/3650180040 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f97e815dc00 0x7f97e80b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:03.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97dea00640 1 -- 172.21.15.46:0/3650180040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e806b8b0 msgr2=0x7f97e806e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:03.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97dea00640 1 --2- 172.21.15.46:0/3650180040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e806b8b0 0x7f97e806e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:03.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97dea00640 1 -- 172.21.15.46:0/3650180040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d8009420 con 0x7f97e80d0660 2024-03-29T08:21:03.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97e7e00640 1 --2- 172.21.15.46:0/3650180040 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f97e815dc00 0x7f97e80b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:03.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97dea00640 1 --2- 172.21.15.46:0/3650180040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e80d0660 0x7f97e8152630 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f97d000efc0 tx=0x7f97d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:03.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97dfe00640 1 -- 172.21.15.46:0/3650180040 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d0009b80 con 0x7f97e80d0660 2024-03-29T08:21:03.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97dfe00640 1 -- 172.21.15.46:0/3650180040 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d000f040 con 0x7f97e80d0660 2024-03-29T08:21:03.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97dfe00640 1 -- 172.21.15.46:0/3650180040 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d00146a0 con 0x7f97e80d0660 2024-03-29T08:21:03.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97ee400640 1 -- 172.21.15.46:0/3650180040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e80c5270 con 0x7f97e80d0660 2024-03-29T08:21:03.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.203+0000 7f97ee400640 1 -- 172.21.15.46:0/3650180040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e80c5830 con 0x7f97e80d0660 2024-03-29T08:21:03.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.207+0000 7f97ee400640 1 -- 172.21.15.46:0/3650180040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e8066c90 con 0x7f97e80d0660 2024-03-29T08:21:03.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.207+0000 7f97dfe00640 1 -- 172.21.15.46:0/3650180040 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f97d0014800 con 0x7f97e80d0660 2024-03-29T08:21:03.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.207+0000 7f97dfe00640 1 --2- 172.21.15.46:0/3650180040 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97c40666c0 0x7f97c4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:03.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.207+0000 7f97dfe00640 1 -- 172.21.15.46:0/3650180040 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f97d00848f0 con 0x7f97e80d0660 2024-03-29T08:21:03.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.207+0000 7f97e7400640 1 --2- 172.21.15.46:0/3650180040 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97c40666c0 0x7f97c4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:03.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.207+0000 7f97e7400640 1 --2- 172.21.15.46:0/3650180040 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97c40666c0 0x7f97c4068b80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f97d80af000 tx=0x7f97d80a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:03.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.211+0000 7f97dfe00640 1 -- 172.21.15.46:0/3650180040 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f97d0051f50 con 0x7f97e80d0660 2024-03-29T08:21:03.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:03.435+0000 7f97ee400640 1 -- 172.21.15.46:0/3650180040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f97e8157500 con 0x7f97e80d0660 2024-03-29T08:21:04.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.035+0000 7f97dfe00640 1 -- 172.21.15.46:0/3650180040 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v162) v1 ==== 172+0+0 (secure 0 0 0) 0x7f97d0055570 con 0x7f97e80d0660 2024-03-29T08:21:04.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.035+0000 7f97ee400640 1 -- 172.21.15.46:0/3650180040 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97c40666c0 msgr2=0x7f97c4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:04.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.035+0000 7f97ee400640 1 --2- 172.21.15.46:0/3650180040 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97c40666c0 0x7f97c4068b80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f97d80af000 tx=0x7f97d80a7000 comp rx=0 tx=0).stop 2024-03-29T08:21:04.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.035+0000 7f97ee400640 1 -- 172.21.15.46:0/3650180040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e80d0660 msgr2=0x7f97e8152630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:04.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.035+0000 7f97ee400640 1 --2- 172.21.15.46:0/3650180040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e80d0660 0x7f97e8152630 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f97d000efc0 tx=0x7f97d000c490 comp rx=0 tx=0).stop 2024-03-29T08:21:04.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.035+0000 7f97ee400640 1 -- 172.21.15.46:0/3650180040 shutdown_connections 2024-03-29T08:21:04.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.035+0000 7f97ee400640 1 --2- 172.21.15.46:0/3650180040 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97c40666c0 0x7f97c4068b80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.035+0000 7f97ee400640 1 --2- 172.21.15.46:0/3650180040 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f97e815dc00 0x7f97e80b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.035+0000 7f97ee400640 1 --2- 172.21.15.46:0/3650180040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e80d0660 0x7f97e8152630 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.035+0000 7f97ee400640 1 --2- 172.21.15.46:0/3650180040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e806b8b0 0x7f97e806e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.035+0000 7f97ee400640 1 -- 172.21.15.46:0/3650180040 >> 172.21.15.46:0/3650180040 conn(0x7f97e8089fd0 msgr2=0x7f97e806ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:04.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.035+0000 7f97ee400640 1 -- 172.21.15.46:0/3650180040 shutdown_connections 2024-03-29T08:21:04.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.035+0000 7f97ee400640 1 -- 172.21.15.46:0/3650180040 wait complete. 2024-03-29T08:21:04.061 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-29T08:21:04.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.207+0000 7f3e82600640 1 Processor -- start 2024-03-29T08:21:04.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.207+0000 7f3e82600640 1 -- start start 2024-03-29T08:21:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.207+0000 7f3e82600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e7c15dbb0 0x7f3e7c06ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.207+0000 7f3e82600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3e7c0b2440 0x7f3e7c06b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.207+0000 7f3e82600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e7c157490 con 0x7f3e7c0b3f20 2024-03-29T08:21:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.207+0000 7f3e82600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7c0b2c90 con 0x7f3e7c15dbb0 2024-03-29T08:21:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.207+0000 7f3e82600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c0b29b0 con 0x7f3e7c0b2440 2024-03-29T08:21:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.207+0000 7f3e7b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e7c15dbb0 0x7f3e7c06ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.207+0000 7f3e7be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3e7c0b2440 0x7f3e7c06b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.207+0000 7f3e7b400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e7c15dbb0 0x7f3e7c06ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51762/0 (socket says 172.21.15.46:51762) 2024-03-29T08:21:04.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.207+0000 7f3e7b400640 1 -- 172.21.15.46:0/382129939 learned_addr learned my addr 172.21.15.46:0/382129939 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:04.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.207+0000 7f3e7b400640 1 -- 172.21.15.46:0/382129939 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3e7c0b2440 msgr2=0x7f3e7c06b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:04.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.207+0000 7f3e7b400640 1 --2- 172.21.15.46:0/382129939 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3e7c0b2440 0x7f3e7c06b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.207+0000 7f3e7b400640 1 -- 172.21.15.46:0/382129939 >> v1:172.21.15.73:6789/0 conn(0x7f3e7c0b3f20 legacy=0x7f3e7c0b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:04.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.207+0000 7f3e7b400640 1 -- 172.21.15.46:0/382129939 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c0b3440 con 0x7f3e7c15dbb0 2024-03-29T08:21:04.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.211+0000 7f3e7b400640 1 --2- 172.21.15.46:0/382129939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e7c15dbb0 0x7f3e7c06ada0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009870 tx=0x7f3e6c01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bac7067255d74160 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:04.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.211+0000 7f3e7aa00640 1 -- 172.21.15.46:0/382129939 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c0aa430 con 0x7f3e7c15dbb0 2024-03-29T08:21:04.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.211+0000 7f3e7aa00640 1 -- 172.21.15.46:0/382129939 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c0aa590 con 0x7f3e7c15dbb0 2024-03-29T08:21:04.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.211+0000 7f3e7aa00640 1 -- 172.21.15.46:0/382129939 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c0aa920 con 0x7f3e7c15dbb0 2024-03-29T08:21:04.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.211+0000 7f3e82600640 1 -- 172.21.15.46:0/382129939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e7c15dbb0 msgr2=0x7f3e7c06ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:04.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.211+0000 7f3e82600640 1 --2- 172.21.15.46:0/382129939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e7c15dbb0 0x7f3e7c06ada0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009870 tx=0x7f3e6c01c4d0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.211+0000 7f3e82600640 1 -- 172.21.15.46:0/382129939 shutdown_connections 2024-03-29T08:21:04.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.211+0000 7f3e82600640 1 --2- 172.21.15.46:0/382129939 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3e7c0b2440 0x7f3e7c06b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.211+0000 7f3e82600640 1 --2- 172.21.15.46:0/382129939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e7c15dbb0 0x7f3e7c06ada0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.211+0000 7f3e82600640 1 -- 172.21.15.46:0/382129939 >> 172.21.15.46:0/382129939 conn(0x7f3e7c070ad0 msgr2=0x7f3e7c070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:04.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.211+0000 7f3e82600640 1 -- 172.21.15.46:0/382129939 shutdown_connections 2024-03-29T08:21:04.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.211+0000 7f3e82600640 1 -- 172.21.15.46:0/382129939 wait complete. 2024-03-29T08:21:04.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e82600640 1 Processor -- start 2024-03-29T08:21:04.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e82600640 1 -- start start 2024-03-29T08:21:04.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e82600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e7c0b2440 0x7f3e7c0ce500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:04.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e82600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3e7c0b3f20 0x7f3e7c14ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:04.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e82600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e7c15dbb0 0x7f3e7c14e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:04.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e82600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c14d390 con 0x7f3e7c0b2440 2024-03-29T08:21:04.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e82600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c14ee00 con 0x7f3e7c15dbb0 2024-03-29T08:21:04.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e82600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c14d540 con 0x7f3e7c0b3f20 2024-03-29T08:21:04.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e7b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e7c0b2440 0x7f3e7c0ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e7b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e7c0b2440 0x7f3e7c0ce500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45852/0 (socket says 172.21.15.46:45852) 2024-03-29T08:21:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e7b400640 1 -- 172.21.15.46:0/1837374573 learned_addr learned my addr 172.21.15.46:0/1837374573 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e7be00640 1 --2- 172.21.15.46:0/1837374573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e7c15dbb0 0x7f3e7c14e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e72a00640 1 --2- 172.21.15.46:0/1837374573 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3e7c0b3f20 0x7f3e7c14ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e7b400640 1 -- 172.21.15.46:0/1837374573 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3e7c0b3f20 msgr2=0x7f3e7c14ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e7b400640 1 --2- 172.21.15.46:0/1837374573 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3e7c0b3f20 0x7f3e7c14ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e7b400640 1 -- 172.21.15.46:0/1837374573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e7c15dbb0 msgr2=0x7f3e7c14e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e7b400640 1 --2- 172.21.15.46:0/1837374573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e7c15dbb0 0x7f3e7c14e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e7b400640 1 -- 172.21.15.46:0/1837374573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c0094d0 con 0x7f3e7c0b2440 2024-03-29T08:21:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e72a00640 1 --2- 172.21.15.46:0/1837374573 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3e7c0b3f20 0x7f3e7c14ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e7b400640 1 --2- 172.21.15.46:0/1837374573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e7c0b2440 0x7f3e7c0ce500 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0ac040 tx=0x7f3e6c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e73e00640 1 -- 172.21.15.46:0/1837374573 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c0aa430 con 0x7f3e7c0b2440 2024-03-29T08:21:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e73e00640 1 -- 172.21.15.46:0/1837374573 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c0b3800 con 0x7f3e7c0b2440 2024-03-29T08:21:04.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e73e00640 1 -- 172.21.15.46:0/1837374573 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c0b3bb0 con 0x7f3e7c0b2440 2024-03-29T08:21:04.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e82600640 1 -- 172.21.15.46:0/1837374573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e7c14d820 con 0x7f3e7c0b2440 2024-03-29T08:21:04.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e82600640 1 -- 172.21.15.46:0/1837374573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e7c14dde0 con 0x7f3e7c0b2440 2024-03-29T08:21:04.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.215+0000 7f3e82600640 1 -- 172.21.15.46:0/1837374573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7c09d100 con 0x7f3e7c0b2440 2024-03-29T08:21:04.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.219+0000 7f3e73e00640 1 -- 172.21.15.46:0/1837374573 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3e6c0b3d10 con 0x7f3e7c0b2440 2024-03-29T08:21:04.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.219+0000 7f3e73e00640 1 --2- 172.21.15.46:0/1837374573 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3e58066690 0x7f3e58068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:04.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.219+0000 7f3e73e00640 1 -- 172.21.15.46:0/1837374573 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3e6c11e730 con 0x7f3e7c0b2440 2024-03-29T08:21:04.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.219+0000 7f3e72a00640 1 --2- 172.21.15.46:0/1837374573 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3e58066690 0x7f3e58068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:04.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.219+0000 7f3e72a00640 1 --2- 172.21.15.46:0/1837374573 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3e58066690 0x7f3e58068b50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c0b0a00 tx=0x7f3e64009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:04.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.223+0000 7f3e73e00640 1 -- 172.21.15.46:0/1837374573 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3e6c0ebeb0 con 0x7f3e7c0b2440 2024-03-29T08:21:04.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.391+0000 7f3e82600640 1 -- 172.21.15.46:0/1837374573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3e7c04e120 con 0x7f3e7c0b2440 2024-03-29T08:21:04.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.395+0000 7f3e73e00640 1 -- 172.21.15.46:0/1837374573 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3e6c0ef4d0 con 0x7f3e7c0b2440 2024-03-29T08:21:04.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.399+0000 7f3e82600640 1 -- 172.21.15.46:0/1837374573 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3e58066690 msgr2=0x7f3e58068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:04.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.399+0000 7f3e82600640 1 --2- 172.21.15.46:0/1837374573 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3e58066690 0x7f3e58068b50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c0b0a00 tx=0x7f3e64009040 comp rx=0 tx=0).stop 2024-03-29T08:21:04.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.399+0000 7f3e82600640 1 -- 172.21.15.46:0/1837374573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e7c0b2440 msgr2=0x7f3e7c0ce500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:04.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.399+0000 7f3e82600640 1 --2- 172.21.15.46:0/1837374573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e7c0b2440 0x7f3e7c0ce500 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0ac040 tx=0x7f3e6c01ad10 comp rx=0 tx=0).stop 2024-03-29T08:21:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.399+0000 7f3e82600640 1 -- 172.21.15.46:0/1837374573 shutdown_connections 2024-03-29T08:21:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.399+0000 7f3e82600640 1 --2- 172.21.15.46:0/1837374573 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3e58066690 0x7f3e58068b50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.399+0000 7f3e82600640 1 --2- 172.21.15.46:0/1837374573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e7c15dbb0 0x7f3e7c14e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.399+0000 7f3e82600640 1 --2- 172.21.15.46:0/1837374573 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3e7c0b3f20 0x7f3e7c14ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.399+0000 7f3e82600640 1 --2- 172.21.15.46:0/1837374573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e7c0b2440 0x7f3e7c0ce500 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.399+0000 7f3e82600640 1 -- 172.21.15.46:0/1837374573 >> 172.21.15.46:0/1837374573 conn(0x7f3e7c070ad0 msgr2=0x7f3e7c06c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.399+0000 7f3e82600640 1 -- 172.21.15.46:0/1837374573 shutdown_connections 2024-03-29T08:21:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.399+0000 7f3e82600640 1 -- 172.21.15.46:0/1837374573 wait complete. 2024-03-29T08:21:04.421 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T08:21:04.421 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:21:04.510 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:21:04.558 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T08:21:04.559 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:21:04.559 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:21:04.567 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:21:04.567 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:21:04.568 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:21:04.577 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:21:04.577 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:21:04.577 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:21:04.619 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:21:04.619 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:21:04.620 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:21:04.628 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:21:04.629 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-29T08:21:04.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:21:04.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.803+0000 7fdf03e00640 1 Processor -- start 2024-03-29T08:21:04.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.803+0000 7fdf03e00640 1 -- start start 2024-03-29T08:21:04.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.803+0000 7fdf03e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdefc1593d0 0x7fdefc06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:04.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.803+0000 7fdf03e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdefc15cd90 0x7fdefc1597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:04.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.803+0000 7fdf03e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdefc06d130 con 0x7fdefc15d230 2024-03-29T08:21:04.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.803+0000 7fdf03e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdefc15b8f0 con 0x7fdefc1593d0 2024-03-29T08:21:04.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.803+0000 7fdf03e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc15ad50 con 0x7fdefc15cd90 2024-03-29T08:21:04.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.803+0000 7fdf01400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdefc1593d0 0x7fdefc06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:04.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.803+0000 7fdf01400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdefc1593d0 0x7fdefc06cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45864/0 (socket says 172.21.15.46:45864) 2024-03-29T08:21:04.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.803+0000 7fdf01400640 1 -- 172.21.15.46:0/578701418 learned_addr learned my addr 172.21.15.46:0/578701418 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:04.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.803+0000 7fdf01400640 1 -- 172.21.15.46:0/578701418 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdefc15cd90 msgr2=0x7fdefc1597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:04.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.803+0000 7fdf01400640 1 --2- 172.21.15.46:0/578701418 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdefc15cd90 0x7fdefc1597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.803+0000 7fdf01400640 1 -- 172.21.15.46:0/578701418 >> v1:172.21.15.73:6789/0 conn(0x7fdefc15d230 legacy=0x7fdefc15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:04.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.807+0000 7fdf01400640 1 -- 172.21.15.46:0/578701418 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc15afc0 con 0x7fdefc1593d0 2024-03-29T08:21:04.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.807+0000 7fdf01400640 1 --2- 172.21.15.46:0/578701418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdefc1593d0 0x7fdefc06cbf0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009870 tx=0x7fdef001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d4f1127aeda53 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:04.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.807+0000 7fdefbe00640 1 -- 172.21.15.46:0/578701418 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef00aa640 con 0x7fdefc1593d0 2024-03-29T08:21:04.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.807+0000 7fdefbe00640 1 -- 172.21.15.46:0/578701418 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef00aa7a0 con 0x7fdefc1593d0 2024-03-29T08:21:04.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.807+0000 7fdefbe00640 1 -- 172.21.15.46:0/578701418 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef00aab30 con 0x7fdefc1593d0 2024-03-29T08:21:04.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.807+0000 7fdf03e00640 1 -- 172.21.15.46:0/578701418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdefc1593d0 msgr2=0x7fdefc06cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:04.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.807+0000 7fdf03e00640 1 --2- 172.21.15.46:0/578701418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdefc1593d0 0x7fdefc06cbf0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009870 tx=0x7fdef001c520 comp rx=0 tx=0).stop 2024-03-29T08:21:04.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf03e00640 1 -- 172.21.15.46:0/578701418 shutdown_connections 2024-03-29T08:21:04.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf03e00640 1 --2- 172.21.15.46:0/578701418 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdefc15cd90 0x7fdefc1597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf03e00640 1 --2- 172.21.15.46:0/578701418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdefc1593d0 0x7fdefc06cbf0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf03e00640 1 -- 172.21.15.46:0/578701418 >> 172.21.15.46:0/578701418 conn(0x7fdefc066110 msgr2=0x7fdefc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:04.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf03e00640 1 -- 172.21.15.46:0/578701418 shutdown_connections 2024-03-29T08:21:04.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf03e00640 1 -- 172.21.15.46:0/578701418 wait complete. 2024-03-29T08:21:04.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf03e00640 1 Processor -- start 2024-03-29T08:21:04.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf03e00640 1 -- start start 2024-03-29T08:21:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf03e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdefc1593d0 0x7fdefc153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf03e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdefc15cd90 0x7fdefc0c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf03e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdefc15d230 0x7fdefc0c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf03e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc142540 con 0x7fdefc15cd90 2024-03-29T08:21:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf03e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc141f10 con 0x7fdefc15d230 2024-03-29T08:21:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf03e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdefc1583c0 con 0x7fdefc1593d0 2024-03-29T08:21:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf00a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdefc15cd90 0x7fdefc0c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf00a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdefc15cd90 0x7fdefc0c5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45870/0 (socket says 172.21.15.46:45870) 2024-03-29T08:21:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf00a00640 1 -- 172.21.15.46:0/4061754783 learned_addr learned my addr 172.21.15.46:0/4061754783 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf01e00640 1 --2- 172.21.15.46:0/4061754783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdefc15d230 0x7fdefc0c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:04.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf01400640 1 --2- 172.21.15.46:0/4061754783 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdefc1593d0 0x7fdefc153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf00a00640 1 -- 172.21.15.46:0/4061754783 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdefc1593d0 msgr2=0x7fdefc153ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf00a00640 1 --2- 172.21.15.46:0/4061754783 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdefc1593d0 0x7fdefc153ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf00a00640 1 -- 172.21.15.46:0/4061754783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdefc15d230 msgr2=0x7fdefc0c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf00a00640 1 --2- 172.21.15.46:0/4061754783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdefc15d230 0x7fdefc0c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf00a00640 1 -- 172.21.15.46:0/4061754783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef00094d0 con 0x7fdefc15cd90 2024-03-29T08:21:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf01e00640 1 --2- 172.21.15.46:0/4061754783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdefc15d230 0x7fdefc0c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf01400640 1 --2- 172.21.15.46:0/4061754783 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdefc1593d0 0x7fdefc153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdf00a00640 1 --2- 172.21.15.46:0/4061754783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdefc15cd90 0x7fdefc0c5a30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c490 tx=0x7fdee8001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdef9600640 1 -- 172.21.15.46:0/4061754783 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8003600 con 0x7fdefc15cd90 2024-03-29T08:21:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdef9600640 1 -- 172.21.15.46:0/4061754783 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee8003780 con 0x7fdefc15cd90 2024-03-29T08:21:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.811+0000 7fdef9600640 1 -- 172.21.15.46:0/4061754783 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8003b70 con 0x7fdefc15cd90 2024-03-29T08:21:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.815+0000 7fdf03e00640 1 -- 172.21.15.46:0/4061754783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdefc1421d0 con 0x7fdefc15cd90 2024-03-29T08:21:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.815+0000 7fdf03e00640 1 -- 172.21.15.46:0/4061754783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdefc1541d0 con 0x7fdefc15cd90 2024-03-29T08:21:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.815+0000 7fdf03e00640 1 -- 172.21.15.46:0/4061754783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdefc066c90 con 0x7fdefc15cd90 2024-03-29T08:21:04.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.815+0000 7fdef9600640 1 -- 172.21.15.46:0/4061754783 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdee8003cd0 con 0x7fdefc15cd90 2024-03-29T08:21:04.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.815+0000 7fdef9600640 1 --2- 172.21.15.46:0/4061754783 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdedc066690 0x7fdedc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:04.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.815+0000 7fdef9600640 1 -- 172.21.15.46:0/4061754783 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdee80830a0 con 0x7fdefc15cd90 2024-03-29T08:21:04.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.815+0000 7fdf01400640 1 --2- 172.21.15.46:0/4061754783 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdedc066690 0x7fdedc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:04.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.815+0000 7fdf01400640 1 --2- 172.21.15.46:0/4061754783 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdedc066690 0x7fdedc068b50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdef00ae000 tx=0x7fdef0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:04.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:04.819+0000 7fdef9600640 1 -- 172.21.15.46:0/4061754783 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdee8050780 con 0x7fdefc15cd90 2024-03-29T08:21:05.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.015+0000 7fdf03e00640 1 -- 172.21.15.46:0/4061754783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdefc06d3d0 con 0x7fdefc15cd90 2024-03-29T08:21:05.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.015+0000 7fdef9600640 1 -- 172.21.15.46:0/4061754783 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3523 (secure 0 0 0) 0x7fdee8017de0 con 0x7fdefc15cd90 2024-03-29T08:21:05.018 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:05.018 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5291,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2702541693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2702541693},{"type":"v1","addr":"172.21.15.73:6837","nonce":2702541693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5602,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2507803771","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2507803771},{"type":"v1","addr":"172.21.15.73:6835","nonce":2507803771}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5616,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/494932499","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":494932499},{"type":"v1","addr":"172.21.15.46:6833","nonce":494932499}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5988,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/753149609","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":753149609},{"type":"v1","addr":"172.21.15.46:6835","nonce":753149609}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2024-03-29T08:21:05.018 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 26 2024-03-29T08:21:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.015+0000 7fdf03e00640 1 -- 172.21.15.46:0/4061754783 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdedc066690 msgr2=0x7fdedc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.015+0000 7fdf03e00640 1 --2- 172.21.15.46:0/4061754783 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdedc066690 0x7fdedc068b50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdef00ae000 tx=0x7fdef0008040 comp rx=0 tx=0).stop 2024-03-29T08:21:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.015+0000 7fdf03e00640 1 -- 172.21.15.46:0/4061754783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdefc15cd90 msgr2=0x7fdefc0c5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.015+0000 7fdf03e00640 1 --2- 172.21.15.46:0/4061754783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdefc15cd90 0x7fdefc0c5a30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fdee800c490 tx=0x7fdee8001e80 comp rx=0 tx=0).stop 2024-03-29T08:21:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.015+0000 7fdf03e00640 1 -- 172.21.15.46:0/4061754783 shutdown_connections 2024-03-29T08:21:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.015+0000 7fdf03e00640 1 --2- 172.21.15.46:0/4061754783 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdedc066690 0x7fdedc068b50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.015+0000 7fdf03e00640 1 --2- 172.21.15.46:0/4061754783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdefc15d230 0x7fdefc0c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.015+0000 7fdf03e00640 1 --2- 172.21.15.46:0/4061754783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdefc15cd90 0x7fdefc0c5a30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.015+0000 7fdf03e00640 1 --2- 172.21.15.46:0/4061754783 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdefc1593d0 0x7fdefc153ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.015+0000 7fdf03e00640 1 -- 172.21.15.46:0/4061754783 >> 172.21.15.46:0/4061754783 conn(0x7fdefc066110 msgr2=0x7fdefc0b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.015+0000 7fdf03e00640 1 -- 172.21.15.46:0/4061754783 shutdown_connections 2024-03-29T08:21:05.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.015+0000 7fdf03e00640 1 -- 172.21.15.46:0/4061754783 wait complete. 2024-03-29T08:21:05.041 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T08:21:05.042 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-29T08:21:05.042 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T08:21:05.042 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:21:05.043 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T08:21:05.043 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-29T08:21:05.043 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T08:21:05.043 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:21:05.044 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T08:21:05.044 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-29T08:21:05.044 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T08:21:05.044 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:21:05.045 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T08:21:05.045 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-29T08:21:05.045 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T08:21:05.045 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:21:05.057 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T08:21:05.057 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T08:21:05.058 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T08:21:05.059 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T08:21:05.061 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T08:21:05.061 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T08:21:05.062 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T08:21:05.062 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T08:21:05.102 INFO:tasks.ceph.mds.a:Started 2024-03-29T08:21:05.102 DEBUG:teuthology.parallel:result is None 2024-03-29T08:21:05.104 INFO:tasks.ceph.mds.c:Started 2024-03-29T08:21:05.104 DEBUG:teuthology.parallel:result is None 2024-03-29T08:21:05.110 INFO:tasks.ceph.mds.b:Started 2024-03-29T08:21:05.110 DEBUG:teuthology.parallel:result is None 2024-03-29T08:21:05.111 INFO:tasks.ceph.mds.d:Started 2024-03-29T08:21:05.112 DEBUG:teuthology.parallel:result is None 2024-03-29T08:21:05.112 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-29T08:21:05.189 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-03-29T08:21:05.191 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-03-29T08:21:05.206 INFO:tasks.ceph.mds.b.smithi073.stdout:starting mds.b at 2024-03-29T08:21:05.206 INFO:tasks.ceph.mds.d.smithi073.stdout:starting mds.d at 2024-03-29T08:21:05.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.259+0000 7f98e0e00640 1 Processor -- start 2024-03-29T08:21:05.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.259+0000 7f98e0e00640 1 -- start start 2024-03-29T08:21:05.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.263+0000 7f98e0e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98dc160db0 0x7f98dc0cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:05.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.263+0000 7f98e0e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98dc0cfb20 con 0x7f98dc15c950 2024-03-29T08:21:05.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.263+0000 7f98e0e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98dc0cef10 con 0x7f98dc0cea30 2024-03-29T08:21:05.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.263+0000 7f98e0e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98dc0cf1f0 con 0x7f98dc160db0 2024-03-29T08:21:05.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.263+0000 7f98da800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98dc160db0 0x7f98dc0cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:05.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.263+0000 7f98da800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98dc160db0 0x7f98dc0cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45912/0 (socket says 172.21.15.46:45912) 2024-03-29T08:21:05.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.263+0000 7f98d9e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f98dc15c950 0x7f98dc06bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37418/0 (socket says 172.21.15.46:37418) 2024-03-29T08:21:05.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.263+0000 7f98da800640 1 -- 172.21.15.46:0/1588733444 learned_addr learned my addr 172.21.15.46:0/1588733444 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.263+0000 7f98da800640 1 -- 172.21.15.46:0/1588733444 >> v1:172.21.15.73:6789/0 conn(0x7f98dc0cea30 legacy=0x7f98dc1606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.263+0000 7f98da800640 1 -- 172.21.15.46:0/1588733444 >> v1:172.21.15.46:6789/0 conn(0x7f98dc15c950 legacy=0x7f98dc06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:05.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.263+0000 7f98da800640 1 -- 172.21.15.46:0/1588733444 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc0cf4d0 con 0x7f98dc160db0 2024-03-29T08:21:05.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98da800640 1 --2- 172.21.15.46:0/1588733444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98dc160db0 0x7f98dc0cad10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009e50 tx=0x7f98d401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28eb1820214e0685 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:05.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98d9400640 1 -- 172.21.15.46:0/1588733444 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d40af400 con 0x7f98dc160db0 2024-03-29T08:21:05.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98d9400640 1 -- 172.21.15.46:0/1588733444 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d40ab050 con 0x7f98dc160db0 2024-03-29T08:21:05.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98d9400640 1 -- 172.21.15.46:0/1588733444 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d40af780 con 0x7f98dc160db0 2024-03-29T08:21:05.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98e0e00640 1 -- 172.21.15.46:0/1588733444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98dc160db0 msgr2=0x7f98dc0cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:05.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98e0e00640 1 --2- 172.21.15.46:0/1588733444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98dc160db0 0x7f98dc0cad10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009e50 tx=0x7f98d401f210 comp rx=0 tx=0).stop 2024-03-29T08:21:05.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98e0e00640 1 -- 172.21.15.46:0/1588733444 shutdown_connections 2024-03-29T08:21:05.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98e0e00640 1 --2- 172.21.15.46:0/1588733444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98dc160db0 0x7f98dc0cad10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:05.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98e0e00640 1 -- 172.21.15.46:0/1588733444 >> 172.21.15.46:0/1588733444 conn(0x7f98dc0b21b0 msgr2=0x7f98dc0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:05.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98e0e00640 1 -- 172.21.15.46:0/1588733444 shutdown_connections 2024-03-29T08:21:05.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98e0e00640 1 -- 172.21.15.46:0/1588733444 wait complete. 2024-03-29T08:21:05.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98e0e00640 1 Processor -- start 2024-03-29T08:21:05.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98e0e00640 1 -- start start 2024-03-29T08:21:05.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98e0e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f98dc0cea30 0x7f98dc144960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:05.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98e0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98dc15c950 0x7f98dc06e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:05.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98e0e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98dc160db0 0x7f98dc06ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:05.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98e0e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc144ea0 con 0x7f98dc15c950 2024-03-29T08:21:05.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98e0e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc146040 con 0x7f98dc160db0 2024-03-29T08:21:05.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.267+0000 7f98e0e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc146320 con 0x7f98dc0cea30 2024-03-29T08:21:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98d1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98dc15c950 0x7f98dc06e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98d9e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f98dc0cea30 0x7f98dc144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98da800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98dc160db0 0x7f98dc06ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98d1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98dc15c950 0x7f98dc06e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45914/0 (socket says 172.21.15.46:45914) 2024-03-29T08:21:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98d1400640 1 -- 172.21.15.46:0/648567649 learned_addr learned my addr 172.21.15.46:0/648567649 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98d1400640 1 -- 172.21.15.46:0/648567649 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f98dc0cea30 msgr2=0x7f98dc144960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98d1400640 1 --2- 172.21.15.46:0/648567649 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f98dc0cea30 0x7f98dc144960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98d1400640 1 -- 172.21.15.46:0/648567649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98dc160db0 msgr2=0x7f98dc06ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:05.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98d1400640 1 --2- 172.21.15.46:0/648567649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98dc160db0 0x7f98dc06ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98d1400640 1 -- 172.21.15.46:0/648567649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d4009b30 con 0x7f98dc15c950 2024-03-29T08:21:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98d1400640 1 --2- 172.21.15.46:0/648567649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98dc15c950 0x7f98dc06e9d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f98c400c4d0 tx=0x7f98c400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98d2a00640 1 -- 172.21.15.46:0/648567649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c40052d0 con 0x7f98dc15c950 2024-03-29T08:21:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98d2a00640 1 -- 172.21.15.46:0/648567649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c4005430 con 0x7f98dc15c950 2024-03-29T08:21:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98d2a00640 1 -- 172.21.15.46:0/648567649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4012830 con 0x7f98dc15c950 2024-03-29T08:21:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98e0e00640 1 -- 172.21.15.46:0/648567649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98dc06c470 con 0x7f98dc15c950 2024-03-29T08:21:05.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98e0e00640 1 -- 172.21.15.46:0/648567649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98dc1468e0 con 0x7f98dc15c950 2024-03-29T08:21:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98e0e00640 1 -- 172.21.15.46:0/648567649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98dc06c270 con 0x7f98dc15c950 2024-03-29T08:21:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.271+0000 7f98d2a00640 1 -- 172.21.15.46:0/648567649 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f98c4012990 con 0x7f98dc15c950 2024-03-29T08:21:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.275+0000 7f98d2a00640 1 --2- 172.21.15.46:0/648567649 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f98b8066690 0x7f98b8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.275+0000 7f98d2a00640 1 -- 172.21.15.46:0/648567649 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f98c40838f0 con 0x7f98dc15c950 2024-03-29T08:21:05.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.275+0000 7f98d9e00640 1 --2- 172.21.15.46:0/648567649 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f98b8066690 0x7f98b8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:05.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.275+0000 7f98d9e00640 1 --2- 172.21.15.46:0/648567649 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f98b8066690 0x7f98b8068b50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f98cc000f70 tx=0x7f98cc007e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:05.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.279+0000 7f98d2a00640 1 -- 172.21.15.46:0/648567649 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f98c4050f70 con 0x7f98dc15c950 2024-03-29T08:21:05.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:05.451+0000 7f98e0e00640 1 -- 172.21.15.46:0/648567649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f98dc09e690 con 0x7f98dc15c950 2024-03-29T08:21:06.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.239+0000 7f98d2a00640 1 -- 172.21.15.46:0/648567649 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v59) v1 ==== 97+0+0 (secure 0 0 0) 0x7f98c4054590 con 0x7f98dc15c950 2024-03-29T08:21:06.242 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-03-29T08:21:06.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.243+0000 7f98e0e00640 1 -- 172.21.15.46:0/648567649 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f98b8066690 msgr2=0x7f98b8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:06.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.243+0000 7f98e0e00640 1 --2- 172.21.15.46:0/648567649 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f98b8066690 0x7f98b8068b50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f98cc000f70 tx=0x7f98cc007e50 comp rx=0 tx=0).stop 2024-03-29T08:21:06.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.243+0000 7f98e0e00640 1 -- 172.21.15.46:0/648567649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98dc15c950 msgr2=0x7f98dc06e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:06.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.243+0000 7f98e0e00640 1 --2- 172.21.15.46:0/648567649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98dc15c950 0x7f98dc06e9d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f98c400c4d0 tx=0x7f98c400c990 comp rx=0 tx=0).stop 2024-03-29T08:21:06.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.243+0000 7f98e0e00640 1 -- 172.21.15.46:0/648567649 shutdown_connections 2024-03-29T08:21:06.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.243+0000 7f98e0e00640 1 --2- 172.21.15.46:0/648567649 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f98b8066690 0x7f98b8068b50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.243+0000 7f98e0e00640 1 --2- 172.21.15.46:0/648567649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98dc160db0 0x7f98dc06ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.243+0000 7f98e0e00640 1 --2- 172.21.15.46:0/648567649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98dc15c950 0x7f98dc06e9d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.243+0000 7f98e0e00640 1 --2- 172.21.15.46:0/648567649 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f98dc0cea30 0x7f98dc144960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.243+0000 7f98e0e00640 1 -- 172.21.15.46:0/648567649 >> 172.21.15.46:0/648567649 conn(0x7f98dc0b21b0 msgr2=0x7f98dc15e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:06.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.243+0000 7f98e0e00640 1 -- 172.21.15.46:0/648567649 shutdown_connections 2024-03-29T08:21:06.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.243+0000 7f98e0e00640 1 -- 172.21.15.46:0/648567649 wait complete. 2024-03-29T08:21:06.263 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-29T08:21:06.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.403+0000 7f0f55c00640 1 Processor -- start 2024-03-29T08:21:06.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.403+0000 7f0f55c00640 1 -- start start 2024-03-29T08:21:06.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.403+0000 7f0f55c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f5006b8b0 0x7f0f500cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:06.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.403+0000 7f0f55c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5015cfb0 con 0x7f0f5015dc00 2024-03-29T08:21:06.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.403+0000 7f0f55c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5006be20 con 0x7f0f500d0660 2024-03-29T08:21:06.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.403+0000 7f0f55c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5006c100 con 0x7f0f5006b8b0 2024-03-29T08:21:06.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.403+0000 7f0f46000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f5006b8b0 0x7f0f500cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:06.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.403+0000 7f0f46000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f5006b8b0 0x7f0f500cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45926/0 (socket says 172.21.15.46:45926) 2024-03-29T08:21:06.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.403+0000 7f0f46000640 1 -- 172.21.15.46:0/2061685631 learned_addr learned my addr 172.21.15.46:0/2061685631 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:06.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.403+0000 7f0f4f400640 1 --1- 172.21.15.46:0/2061685631 >> v1:172.21.15.46:6789/0 conn(0x7f0f5015dc00 0x7f0f500cf7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37424/0 (socket says 172.21.15.46:37424) 2024-03-29T08:21:06.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.403+0000 7f0f46000640 1 -- 172.21.15.46:0/2061685631 >> v1:172.21.15.73:6789/0 conn(0x7f0f500d0660 legacy=0x7f0f500d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:06.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.403+0000 7f0f46000640 1 -- 172.21.15.46:0/2061685631 >> v1:172.21.15.46:6789/0 conn(0x7f0f5015dc00 legacy=0x7f0f500cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:06.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.403+0000 7f0f46000640 1 -- 172.21.15.46:0/2061685631 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f500cecf0 con 0x7f0f5006b8b0 2024-03-29T08:21:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.407+0000 7f0f46000640 1 --2- 172.21.15.46:0/2061685631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5006b8b0 0x7f0f500cad20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009e50 tx=0x7f0f3801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c691af7ff2f127b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.407+0000 7f0f4e000640 1 -- 172.21.15.46:0/2061685631 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f380a58d0 con 0x7f0f5006b8b0 2024-03-29T08:21:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.407+0000 7f0f4e000640 1 -- 172.21.15.46:0/2061685631 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f380a5a30 con 0x7f0f5006b8b0 2024-03-29T08:21:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.407+0000 7f0f4e000640 1 -- 172.21.15.46:0/2061685631 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f380a5dc0 con 0x7f0f5006b8b0 2024-03-29T08:21:06.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f55c00640 1 -- 172.21.15.46:0/2061685631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5006b8b0 msgr2=0x7f0f500cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f55c00640 1 --2- 172.21.15.46:0/2061685631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5006b8b0 0x7f0f500cad20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009e50 tx=0x7f0f3801ca00 comp rx=0 tx=0).stop 2024-03-29T08:21:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f55c00640 1 -- 172.21.15.46:0/2061685631 shutdown_connections 2024-03-29T08:21:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f55c00640 1 --2- 172.21.15.46:0/2061685631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5006b8b0 0x7f0f500cad20 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f55c00640 1 -- 172.21.15.46:0/2061685631 >> 172.21.15.46:0/2061685631 conn(0x7f0f50089fd0 msgr2=0x7f0f5008a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f55c00640 1 -- 172.21.15.46:0/2061685631 shutdown_connections 2024-03-29T08:21:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f55c00640 1 -- 172.21.15.46:0/2061685631 wait complete. 2024-03-29T08:21:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f55c00640 1 Processor -- start 2024-03-29T08:21:06.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f55c00640 1 -- start start 2024-03-29T08:21:06.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f55c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5006b8b0 0x7f0f501515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:06.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f55c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f500d0660 0x7f0f5014e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:06.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f55c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5015dc00 0x7f0f500c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:06.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f55c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f500c85f0 con 0x7f0f5006b8b0 2024-03-29T08:21:06.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f55c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f500c4200 con 0x7f0f500d0660 2024-03-29T08:21:06.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f55c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5014eec0 con 0x7f0f5015dc00 2024-03-29T08:21:06.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f4ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5006b8b0 0x7f0f501515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:06.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f4ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5006b8b0 0x7f0f501515d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45932/0 (socket says 172.21.15.46:45932) 2024-03-29T08:21:06.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f4ea00640 1 -- 172.21.15.46:0/354263593 learned_addr learned my addr 172.21.15.46:0/354263593 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:06.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f46000640 1 --2- 172.21.15.46:0/354263593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f500d0660 0x7f0f5014e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:06.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f4f400640 1 --2- 172.21.15.46:0/354263593 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5015dc00 0x7f0f500c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:06.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f4ea00640 1 -- 172.21.15.46:0/354263593 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5015dc00 msgr2=0x7f0f500c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:06.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f4ea00640 1 --2- 172.21.15.46:0/354263593 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5015dc00 0x7f0f500c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f4ea00640 1 -- 172.21.15.46:0/354263593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f500d0660 msgr2=0x7f0f5014e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:06.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f4ea00640 1 --2- 172.21.15.46:0/354263593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f500d0660 0x7f0f5014e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f4ea00640 1 -- 172.21.15.46:0/354263593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f38009b30 con 0x7f0f5006b8b0 2024-03-29T08:21:06.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.411+0000 7f0f4f400640 1 --2- 172.21.15.46:0/354263593 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5015dc00 0x7f0f500c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:06.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.415+0000 7f0f4ea00640 1 --2- 172.21.15.46:0/354263593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5006b8b0 0x7f0f501515d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000c500 tx=0x7f0f4000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:06.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.415+0000 7f0f47400640 1 -- 172.21.15.46:0/354263593 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40005220 con 0x7f0f5006b8b0 2024-03-29T08:21:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.415+0000 7f0f47400640 1 -- 172.21.15.46:0/354263593 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f40005380 con 0x7f0f5006b8b0 2024-03-29T08:21:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.415+0000 7f0f47400640 1 -- 172.21.15.46:0/354263593 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f40007270 con 0x7f0f5006b8b0 2024-03-29T08:21:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.415+0000 7f0f55c00640 1 -- 172.21.15.46:0/354263593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f500c4450 con 0x7f0f5006b8b0 2024-03-29T08:21:06.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.415+0000 7f0f55c00640 1 -- 172.21.15.46:0/354263593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f500c4a10 con 0x7f0f5006b8b0 2024-03-29T08:21:06.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.415+0000 7f0f55c00640 1 -- 172.21.15.46:0/354263593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f50066c90 con 0x7f0f5006b8b0 2024-03-29T08:21:06.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.415+0000 7f0f47400640 1 -- 172.21.15.46:0/354263593 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0f400073d0 con 0x7f0f5006b8b0 2024-03-29T08:21:06.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.415+0000 7f0f47400640 1 --2- 172.21.15.46:0/354263593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f2c066690 0x7f0f2c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:06.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.415+0000 7f0f47400640 1 -- 172.21.15.46:0/354263593 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0f40083780 con 0x7f0f5006b8b0 2024-03-29T08:21:06.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.415+0000 7f0f46000640 1 --2- 172.21.15.46:0/354263593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f2c066690 0x7f0f2c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:06.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.419+0000 7f0f46000640 1 --2- 172.21.15.46:0/354263593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f2c066690 0x7f0f2c068b50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0f380ae000 tx=0x7f0f380a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:06.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.419+0000 7f0f47400640 1 -- 172.21.15.46:0/354263593 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0f40050e50 con 0x7f0f5006b8b0 2024-03-29T08:21:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.635+0000 7f0f55c00640 1 -- 172.21.15.46:0/354263593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0f500d0ec0 con 0x7f0f5006b8b0 2024-03-29T08:21:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.635+0000 7f0f47400640 1 -- 172.21.15.46:0/354263593 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0f40054470 con 0x7f0f5006b8b0 2024-03-29T08:21:06.639 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:06.639 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:21:06.639 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-03-29T08:21:06.639 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.639 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-03-29T08:21:06.639 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmlnXNJBAA7lI2lCBd0PeZudPsQyMOUQ==", 2024-03-29T08:21:06.640 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.640 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:21:06.640 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:21:06.640 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:21:06.640 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:21:06.640 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.641 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.641 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-03-29T08:21:06.641 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmbPkcMRAADnwJE17uILjJo14jghJuzg==", 2024-03-29T08:21:06.641 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.641 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:21:06.642 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:21:06.642 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:21:06.642 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:21:06.643 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.643 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.643 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-03-29T08:21:06.644 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmkCUVLBAAxcAOWGBDUCpEHeg/1OeQzA==", 2024-03-29T08:21:06.644 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.644 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:21:06.644 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:21:06.645 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:21:06.645 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:21:06.645 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.645 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.646 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-03-29T08:21:06.646 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZm7L11OBAAZ3fpXtEIYG1tGm9ZFswZ5Q==", 2024-03-29T08:21:06.646 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.646 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:21:06.646 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:21:06.647 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:21:06.647 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:21:06.647 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.647 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-03-29T08:21:06.647 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAIeQZmTHV8HhAAODJ+RP1Bx6k0D1153XgI0Q==", 2024-03-29T08:21:06.647 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.647 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:21:06.647 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:21:06.647 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:21:06.647 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.648 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-03-29T08:21:06.648 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQALeQZmZRD0JBAAjU6W/21CeqOMdIP5969iNA==", 2024-03-29T08:21:06.648 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.648 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:21:06.648 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:21:06.648 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:21:06.648 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.648 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.648 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.648 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-03-29T08:21:06.648 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAOeQZmSSP+KhAA26V8CByUJNfkVcG37AdlZQ==", 2024-03-29T08:21:06.648 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.648 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:21:06.648 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:21:06.648 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:21:06.649 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.649 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-03-29T08:21:06.649 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAReQZmb/e8LxAAV4yC1xlLwh8OAqOSPmcyyA==", 2024-03-29T08:21:06.649 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.649 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:21:06.649 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:21:06.649 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:21:06.649 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.649 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.649 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-03-29T08:21:06.649 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAXeQZmrLnzNRAAFPYQRs9sBNNKeyvcoI9rSg==", 2024-03-29T08:21:06.649 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.650 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:21:06.650 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:21:06.650 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:21:06.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.650 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.650 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-03-29T08:21:06.650 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAbeQZmjTPbABAAi4qTU5CBy9LtSSRf7cmfwg==", 2024-03-29T08:21:06.650 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.650 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:21:06.650 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:21:06.650 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:21:06.650 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.651 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-03-29T08:21:06.651 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAeeQZmo13eBRAAWY2VVBakLze9oA4ynp1rNA==", 2024-03-29T08:21:06.651 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.651 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:21:06.651 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:21:06.651 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:21:06.651 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.651 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.651 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-03-29T08:21:06.651 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAheQZmghpZCxAAdy5xI7qPwZb8LN8Rn6Ndaw==", 2024-03-29T08:21:06.651 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.651 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:21:06.651 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:21:06.652 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:21:06.652 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.652 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-03-29T08:21:06.652 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmyk22ABAA8Pn3OaIvxMQSLrKIM8FP4w==", 2024-03-29T08:21:06.652 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.652 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:21:06.652 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-03-29T08:21:06.652 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-29T08:21:06.652 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.652 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-03-29T08:21:06.652 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmG/E8ChAAJTcJ2j5xn0oUsmlV2trWdw==", 2024-03-29T08:21:06.653 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.653 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:21:06.653 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:21:06.653 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:21:06.653 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:21:06.653 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.653 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.653 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-03-29T08:21:06.653 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmhsqgBhAApqnO5qMG8VP27j9TfbZ8uw==", 2024-03-29T08:21:06.653 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.653 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:21:06.653 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:21:06.653 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:21:06.654 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:21:06.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.654 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-03-29T08:21:06.654 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmatlQEBAAAGKuGnAh4YXSk91TAzomXA==", 2024-03-29T08:21:06.654 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.654 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:21:06.654 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:21:06.654 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:21:06.654 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:21:06.654 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.654 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.654 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-03-29T08:21:06.655 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmp2B3CBAA4cfe/hUJeOg0SIr1KbWyUg==", 2024-03-29T08:21:06.655 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.655 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:21:06.655 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:21:06.655 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:21:06.655 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:21:06.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.655 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.655 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-03-29T08:21:06.655 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmyrJXIBAAv535Rz99UI3V7JvvzQ8oSw==", 2024-03-29T08:21:06.655 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.655 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:21:06.655 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-03-29T08:21:06.655 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:21:06.655 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:06.656 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:06.656 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-03-29T08:21:06.656 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZm72tiHBAA2N7qQnR5Ehu/7nP2WpklGg==", 2024-03-29T08:21:06.656 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:21:06.656 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:21:06.656 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-03-29T08:21:06.656 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:21:06.656 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.656 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:06.656 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-29T08:21:06.656 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:21:06.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.639+0000 7f0f55c00640 1 -- 172.21.15.46:0/354263593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f2c066690 msgr2=0x7f0f2c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.639+0000 7f0f55c00640 1 --2- 172.21.15.46:0/354263593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f2c066690 0x7f0f2c068b50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0f380ae000 tx=0x7f0f380a7070 comp rx=0 tx=0).stop 2024-03-29T08:21:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.639+0000 7f0f55c00640 1 -- 172.21.15.46:0/354263593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5006b8b0 msgr2=0x7f0f501515d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.639+0000 7f0f55c00640 1 --2- 172.21.15.46:0/354263593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5006b8b0 0x7f0f501515d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000c500 tx=0x7f0f4000c9c0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.639+0000 7f0f55c00640 1 -- 172.21.15.46:0/354263593 shutdown_connections 2024-03-29T08:21:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.639+0000 7f0f55c00640 1 --2- 172.21.15.46:0/354263593 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f2c066690 0x7f0f2c068b50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.639+0000 7f0f55c00640 1 --2- 172.21.15.46:0/354263593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f500d0660 0x7f0f5014e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.639+0000 7f0f55c00640 1 --2- 172.21.15.46:0/354263593 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f5015dc00 0x7f0f500c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.639+0000 7f0f55c00640 1 --2- 172.21.15.46:0/354263593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f5006b8b0 0x7f0f501515d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.639+0000 7f0f55c00640 1 -- 172.21.15.46:0/354263593 >> 172.21.15.46:0/354263593 conn(0x7f0f50089fd0 msgr2=0x7f0f5006fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.639+0000 7f0f55c00640 1 -- 172.21.15.46:0/354263593 shutdown_connections 2024-03-29T08:21:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.639+0000 7f0f55c00640 1 -- 172.21.15.46:0/354263593 wait complete. 2024-03-29T08:21:06.660 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:21:06.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.799+0000 7f8550600640 1 Processor -- start 2024-03-29T08:21:06.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.799+0000 7f8550600640 1 -- start start 2024-03-29T08:21:06.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.799+0000 7f8550600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f85480d0660 0x7f85480d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:06.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.799+0000 7f8550600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f854815dc00 0x7f85480cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:06.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.799+0000 7f8550600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f854815cfb0 con 0x7f854806b8b0 2024-03-29T08:21:06.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.799+0000 7f8550600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f854806c100 con 0x7f854815dc00 2024-03-29T08:21:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.799+0000 7f8550600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85480d0f80 con 0x7f85480d0660 2024-03-29T08:21:06.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.799+0000 7f854e600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f854815dc00 0x7f85480cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.799+0000 7f854dc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f85480d0660 0x7f85480d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.799+0000 7f854dc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f85480d0660 0x7f85480d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:45436/0 (socket says 172.21.15.46:45436) 2024-03-29T08:21:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.799+0000 7f854dc00640 1 -- 172.21.15.46:0/2610084592 learned_addr learned my addr 172.21.15.46:0/2610084592 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.799+0000 7f854dc00640 1 -- 172.21.15.46:0/2610084592 >> v2:172.21.15.73:3300/0 conn(0x7f854815dc00 msgr2=0x7f85480cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.799+0000 7f854dc00640 1 --2- 172.21.15.46:0/2610084592 >> v2:172.21.15.73:3300/0 conn(0x7f854815dc00 0x7f85480cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.799+0000 7f854dc00640 1 -- 172.21.15.46:0/2610084592 >> v1:172.21.15.73:6789/0 conn(0x7f854806b8b0 legacy=0x7f85480cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.799+0000 7f854dc00640 1 -- 172.21.15.46:0/2610084592 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854806be20 con 0x7f85480d0660 2024-03-29T08:21:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.799+0000 7f854e600640 1 --2- 172.21.15.46:0/2610084592 >> v2:172.21.15.73:3300/0 conn(0x7f854815dc00 0x7f85480cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:21:06.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.803+0000 7f854dc00640 1 --2- 172.21.15.46:0/2610084592 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f85480d0660 0x7f85480d0a40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f853c009870 tx=0x7f853c01c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7998ee9be8509211 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.803+0000 7f8547e00640 1 -- 172.21.15.46:0/2610084592 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c0aa660 con 0x7f85480d0660 2024-03-29T08:21:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.803+0000 7f8547e00640 1 -- 172.21.15.46:0/2610084592 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853c0aa7c0 con 0x7f85480d0660 2024-03-29T08:21:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.803+0000 7f8547e00640 1 -- 172.21.15.46:0/2610084592 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c0aab50 con 0x7f85480d0660 2024-03-29T08:21:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.803+0000 7f8550600640 1 -- 172.21.15.46:0/2610084592 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f85480d0660 msgr2=0x7f85480d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.803+0000 7f8550600640 1 --2- 172.21.15.46:0/2610084592 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f85480d0660 0x7f85480d0a40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f853c009870 tx=0x7f853c01c530 comp rx=0 tx=0).stop 2024-03-29T08:21:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.803+0000 7f8550600640 1 -- 172.21.15.46:0/2610084592 shutdown_connections 2024-03-29T08:21:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.803+0000 7f8550600640 1 --2- 172.21.15.46:0/2610084592 >> v2:172.21.15.73:3300/0 conn(0x7f854815dc00 0x7f85480cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.803+0000 7f8550600640 1 --2- 172.21.15.46:0/2610084592 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f85480d0660 0x7f85480d0a40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.803+0000 7f8550600640 1 -- 172.21.15.46:0/2610084592 >> 172.21.15.46:0/2610084592 conn(0x7f8548089fd0 msgr2=0x7f854808a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:06.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.803+0000 7f8550600640 1 -- 172.21.15.46:0/2610084592 shutdown_connections 2024-03-29T08:21:06.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.803+0000 7f8550600640 1 -- 172.21.15.46:0/2610084592 wait complete. 2024-03-29T08:21:06.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f8550600640 1 Processor -- start 2024-03-29T08:21:06.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f8550600640 1 -- start start 2024-03-29T08:21:06.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f8550600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806b8b0 0x7f85480c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:06.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f8550600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85480d0660 0x7f8548152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f8550600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f854815dc00 0x7f85480c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f8550600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8548152d70 con 0x7f854806b8b0 2024-03-29T08:21:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f8550600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8548153050 con 0x7f85480d0660 2024-03-29T08:21:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f8550600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8548153330 con 0x7f854815dc00 2024-03-29T08:21:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f854dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806b8b0 0x7f85480c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f854dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806b8b0 0x7f85480c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54230/0 (socket says 172.21.15.46:54230) 2024-03-29T08:21:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f854dc00640 1 -- 172.21.15.46:0/1681035816 learned_addr learned my addr 172.21.15.46:0/1681035816 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f854e600640 1 --2- 172.21.15.46:0/1681035816 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f854815dc00 0x7f85480c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f854d200640 1 --2- 172.21.15.46:0/1681035816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85480d0660 0x7f8548152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f854dc00640 1 -- 172.21.15.46:0/1681035816 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f854815dc00 msgr2=0x7f85480c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f854dc00640 1 --2- 172.21.15.46:0/1681035816 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f854815dc00 0x7f85480c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f854dc00640 1 -- 172.21.15.46:0/1681035816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85480d0660 msgr2=0x7f8548152670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f854dc00640 1 --2- 172.21.15.46:0/1681035816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85480d0660 0x7f8548152670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f854dc00640 1 -- 172.21.15.46:0/1681035816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8540009a70 con 0x7f854806b8b0 2024-03-29T08:21:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f854e600640 1 --2- 172.21.15.46:0/1681035816 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f854815dc00 0x7f85480c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f854d200640 1 --2- 172.21.15.46:0/1681035816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85480d0660 0x7f8548152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f854dc00640 1 --2- 172.21.15.46:0/1681035816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806b8b0 0x7f85480c6d60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f853c0ac040 tx=0x7f853c01ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f8545600640 1 -- 172.21.15.46:0/1681035816 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c0b3800 con 0x7f854806b8b0 2024-03-29T08:21:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f8545600640 1 -- 172.21.15.46:0/1681035816 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853c0b3960 con 0x7f854806b8b0 2024-03-29T08:21:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f8545600640 1 -- 172.21.15.46:0/1681035816 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853c0b3d10 con 0x7f854806b8b0 2024-03-29T08:21:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f8550600640 1 -- 172.21.15.46:0/1681035816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853c0094d0 con 0x7f854806b8b0 2024-03-29T08:21:06.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f8550600640 1 -- 172.21.15.46:0/1681035816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85481538f0 con 0x7f854806b8b0 2024-03-29T08:21:06.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.807+0000 7f8550600640 1 -- 172.21.15.46:0/1681035816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8548153ae0 con 0x7f854806b8b0 2024-03-29T08:21:06.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.811+0000 7f8545600640 1 -- 172.21.15.46:0/1681035816 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f853c0b2380 con 0x7f854806b8b0 2024-03-29T08:21:06.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.811+0000 7f8545600640 1 --2- 172.21.15.46:0/1681035816 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8528066690 0x7f8528068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:06.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.811+0000 7f8545600640 1 -- 172.21.15.46:0/1681035816 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f853c0a6040 con 0x7f854806b8b0 2024-03-29T08:21:06.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.811+0000 7f854d200640 1 --2- 172.21.15.46:0/1681035816 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8528066690 0x7f8528068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:06.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.811+0000 7f854d200640 1 --2- 172.21.15.46:0/1681035816 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8528066690 0x7f8528068b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f854815c8f0 tx=0x7f8534009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:06.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.815+0000 7f8545600640 1 -- 172.21.15.46:0/1681035816 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f853c0ebe00 con 0x7f854806b8b0 2024-03-29T08:21:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.979+0000 7f8550600640 1 -- 172.21.15.46:0/1681035816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8548066c90 con 0x7f854806b8b0 2024-03-29T08:21:06.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.979+0000 7f8545600640 1 -- 172.21.15.46:0/1681035816 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+12341 (secure 0 0 0) 0x7f853c0ef420 con 0x7f854806b8b0 2024-03-29T08:21:06.982 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:06.982 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":59,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:21:06.239600+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:21:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.983+0000 7f8550600640 1 -- 172.21.15.46:0/1681035816 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8528066690 msgr2=0x7f8528068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.983+0000 7f8550600640 1 --2- 172.21.15.46:0/1681035816 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8528066690 0x7f8528068b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f854815c8f0 tx=0x7f8534009040 comp rx=0 tx=0).stop 2024-03-29T08:21:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.983+0000 7f8550600640 1 -- 172.21.15.46:0/1681035816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806b8b0 msgr2=0x7f85480c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.983+0000 7f8550600640 1 --2- 172.21.15.46:0/1681035816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806b8b0 0x7f85480c6d60 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f853c0ac040 tx=0x7f853c01ad70 comp rx=0 tx=0).stop 2024-03-29T08:21:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.983+0000 7f8550600640 1 -- 172.21.15.46:0/1681035816 shutdown_connections 2024-03-29T08:21:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.983+0000 7f8550600640 1 --2- 172.21.15.46:0/1681035816 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8528066690 0x7f8528068b50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.983+0000 7f8550600640 1 --2- 172.21.15.46:0/1681035816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85480d0660 0x7f8548152670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.983+0000 7f8550600640 1 --2- 172.21.15.46:0/1681035816 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f854815dc00 0x7f85480c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.983+0000 7f8550600640 1 --2- 172.21.15.46:0/1681035816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f854806b8b0 0x7f85480c6d60 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:06.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.983+0000 7f8550600640 1 -- 172.21.15.46:0/1681035816 >> 172.21.15.46:0/1681035816 conn(0x7f8548089fd0 msgr2=0x7f854806f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:06.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.983+0000 7f8550600640 1 -- 172.21.15.46:0/1681035816 shutdown_connections 2024-03-29T08:21:06.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:06.983+0000 7f8550600640 1 -- 172.21.15.46:0/1681035816 wait complete. 2024-03-29T08:21:07.004 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:21:07.004 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:21:07.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.147+0000 7f530be00640 1 Processor -- start 2024-03-29T08:21:07.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.147+0000 7f530be00640 1 -- start start 2024-03-29T08:21:07.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.147+0000 7f530be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f530c0cd3d0 0x7f530c0cba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:07.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.147+0000 7f530be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c06ff00 con 0x7f530c0ccf60 2024-03-29T08:21:07.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.147+0000 7f530be00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c0cf590 con 0x7f530c15a9f0 2024-03-29T08:21:07.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.147+0000 7f530be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530c06e130 con 0x7f530c0cd3d0 2024-03-29T08:21:07.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.147+0000 7f5303e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f530c0ccf60 0x7f530c0cb380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42060/0 (socket says 172.21.15.46:42060) 2024-03-29T08:21:07.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.147+0000 7f5303e00640 1 -- 172.21.15.46:0/2092890489 learned_addr learned my addr 172.21.15.46:0/2092890489 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:07.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.147+0000 7f530b400640 1 --2- 172.21.15.46:0/2092890489 >> v2:172.21.15.46:3300/0 conn(0x7f530c0cd3d0 0x7f530c0cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:07.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.147+0000 7f530a000640 1 -- 172.21.15.46:0/2092890489 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554866384 0 0) 0x7f530c06ff00 con 0x7f530c0ccf60 2024-03-29T08:21:07.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.147+0000 7f530a000640 1 -- 172.21.15.46:0/2092890489 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52f0003530 con 0x7f530c0ccf60 2024-03-29T08:21:07.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.147+0000 7f530b400640 1 -- 172.21.15.46:0/2092890489 >> v1:172.21.15.73:6789/0 conn(0x7f530c15a9f0 legacy=0x7f530c0ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.147+0000 7f530a000640 1 -- 172.21.15.46:0/2092890489 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849145373 0 0) 0x7f530c0cf590 con 0x7f530c15a9f0 2024-03-29T08:21:07.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.147+0000 7f530b400640 1 -- 172.21.15.46:0/2092890489 >> v1:172.21.15.46:6789/0 conn(0x7f530c0ccf60 legacy=0x7f530c0cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.147+0000 7f530b400640 1 -- 172.21.15.46:0/2092890489 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c06e410 con 0x7f530c0cd3d0 2024-03-29T08:21:07.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.151+0000 7f530b400640 1 --2- 172.21.15.46:0/2092890489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f530c0cd3d0 0x7f530c0cba90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5304009d90 tx=0x7f530401f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583ac485138e3e8b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:07.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.151+0000 7f530a000640 1 -- 172.21.15.46:0/2092890489 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53040af600 con 0x7f530c0cd3d0 2024-03-29T08:21:07.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.151+0000 7f530a000640 1 -- 172.21.15.46:0/2092890489 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53040ab050 con 0x7f530c0cd3d0 2024-03-29T08:21:07.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.151+0000 7f530a000640 1 -- 172.21.15.46:0/2092890489 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53040af980 con 0x7f530c0cd3d0 2024-03-29T08:21:07.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.151+0000 7f530be00640 1 -- 172.21.15.46:0/2092890489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f530c0cd3d0 msgr2=0x7f530c0cba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.151+0000 7f530be00640 1 --2- 172.21.15.46:0/2092890489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f530c0cd3d0 0x7f530c0cba90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5304009d90 tx=0x7f530401f150 comp rx=0 tx=0).stop 2024-03-29T08:21:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.151+0000 7f530be00640 1 -- 172.21.15.46:0/2092890489 shutdown_connections 2024-03-29T08:21:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.151+0000 7f530be00640 1 --2- 172.21.15.46:0/2092890489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f530c0cd3d0 0x7f530c0cba90 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.151+0000 7f530be00640 1 -- 172.21.15.46:0/2092890489 >> 172.21.15.46:0/2092890489 conn(0x7f530c0d0550 msgr2=0x7f530c0d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.151+0000 7f530be00640 1 -- 172.21.15.46:0/2092890489 shutdown_connections 2024-03-29T08:21:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.151+0000 7f530be00640 1 -- 172.21.15.46:0/2092890489 wait complete. 2024-03-29T08:21:07.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530be00640 1 Processor -- start 2024-03-29T08:21:07.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530be00640 1 -- start start 2024-03-29T08:21:07.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f530c0ccf60 0x7f530c0c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:07.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f530c0cd3d0 0x7f530c156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:07.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f530c15a9f0 0x7f530c0c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:07.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c0c88a0 con 0x7f530c15a9f0 2024-03-29T08:21:07.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c0c5e80 con 0x7f530c0ccf60 2024-03-29T08:21:07.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530be00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530c156ff0 con 0x7f530c0cd3d0 2024-03-29T08:21:07.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f530c15a9f0 0x7f530c0c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:07.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f530c15a9f0 0x7f530c0c6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54248/0 (socket says 172.21.15.46:54248) 2024-03-29T08:21:07.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530b400640 1 -- 172.21.15.46:0/1511210925 learned_addr learned my addr 172.21.15.46:0/1511210925 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:07.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530aa00640 1 --2- 172.21.15.46:0/1511210925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f530c0ccf60 0x7f530c0c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:07.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f5303e00640 1 --2- 172.21.15.46:0/1511210925 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f530c0cd3d0 0x7f530c156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530b400640 1 -- 172.21.15.46:0/1511210925 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f530c0cd3d0 msgr2=0x7f530c156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530b400640 1 --2- 172.21.15.46:0/1511210925 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f530c0cd3d0 0x7f530c156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530b400640 1 -- 172.21.15.46:0/1511210925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f530c0ccf60 msgr2=0x7f530c0c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530b400640 1 --2- 172.21.15.46:0/1511210925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f530c0ccf60 0x7f530c0c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530b400640 1 -- 172.21.15.46:0/1511210925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5304009a70 con 0x7f530c15a9f0 2024-03-29T08:21:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530b400640 1 --2- 172.21.15.46:0/1511210925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f530c15a9f0 0x7f530c0c6920 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5304009d90 tx=0x7f530401bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f5302a00640 1 -- 172.21.15.46:0/1511210925 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53040afc20 con 0x7f530c15a9f0 2024-03-29T08:21:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f5302a00640 1 -- 172.21.15.46:0/1511210925 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53040afd80 con 0x7f530c15a9f0 2024-03-29T08:21:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f5302a00640 1 -- 172.21.15.46:0/1511210925 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53040ba620 con 0x7f530c15a9f0 2024-03-29T08:21:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530be00640 1 -- 172.21.15.46:0/1511210925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530c0c6e60 con 0x7f530c15a9f0 2024-03-29T08:21:07.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.155+0000 7f530be00640 1 -- 172.21.15.46:0/1511210925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530c0c7f00 con 0x7f530c15a9f0 2024-03-29T08:21:07.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.159+0000 7f530be00640 1 -- 172.21.15.46:0/1511210925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530c0c89e0 con 0x7f530c15a9f0 2024-03-29T08:21:07.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.159+0000 7f5302a00640 1 -- 172.21.15.46:0/1511210925 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f53040ba780 con 0x7f530c15a9f0 2024-03-29T08:21:07.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.159+0000 7f5302a00640 1 --2- 172.21.15.46:0/1511210925 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52e8066690 0x7f52e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:07.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.159+0000 7f5302a00640 1 -- 172.21.15.46:0/1511210925 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f53041224e0 con 0x7f530c15a9f0 2024-03-29T08:21:07.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.159+0000 7f530aa00640 1 --2- 172.21.15.46:0/1511210925 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52e8066690 0x7f52e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:07.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.159+0000 7f530aa00640 1 --2- 172.21.15.46:0/1511210925 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52e8066690 0x7f52e8068b50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f52fc007fd0 tx=0x7f52fc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:07.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.163+0000 7f5302a00640 1 -- 172.21.15.46:0/1511210925 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f530c0c89e0 con 0x7f530c15a9f0 2024-03-29T08:21:07.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.323+0000 7f530be00640 1 -- 172.21.15.46:0/1511210925 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f530c0c89e0 con 0x7f530c15a9f0 2024-03-29T08:21:07.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.323+0000 7f5302a00640 1 -- 172.21.15.46:0/1511210925 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53040c8020 con 0x7f530c15a9f0 2024-03-29T08:21:07.327 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:21:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.327+0000 7f530be00640 1 -- 172.21.15.46:0/1511210925 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52e8066690 msgr2=0x7f52e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.327+0000 7f530be00640 1 --2- 172.21.15.46:0/1511210925 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52e8066690 0x7f52e8068b50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f52fc007fd0 tx=0x7f52fc009040 comp rx=0 tx=0).stop 2024-03-29T08:21:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.327+0000 7f530be00640 1 -- 172.21.15.46:0/1511210925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f530c15a9f0 msgr2=0x7f530c0c6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.327+0000 7f530be00640 1 --2- 172.21.15.46:0/1511210925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f530c15a9f0 0x7f530c0c6920 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5304009d90 tx=0x7f530401bef0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.327+0000 7f530be00640 1 -- 172.21.15.46:0/1511210925 shutdown_connections 2024-03-29T08:21:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.327+0000 7f530be00640 1 --2- 172.21.15.46:0/1511210925 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52e8066690 0x7f52e8068b50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.327+0000 7f530be00640 1 --2- 172.21.15.46:0/1511210925 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f530c15a9f0 0x7f530c0c6920 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.327+0000 7f530be00640 1 --2- 172.21.15.46:0/1511210925 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f530c0cd3d0 0x7f530c156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.327+0000 7f530be00640 1 --2- 172.21.15.46:0/1511210925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f530c0ccf60 0x7f530c0c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.327+0000 7f530be00640 1 -- 172.21.15.46:0/1511210925 >> 172.21.15.46:0/1511210925 conn(0x7f530c0d0550 msgr2=0x7f530c06c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:07.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.327+0000 7f530be00640 1 -- 172.21.15.46:0/1511210925 shutdown_connections 2024-03-29T08:21:07.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.327+0000 7f530be00640 1 -- 172.21.15.46:0/1511210925 wait complete. 2024-03-29T08:21:07.348 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T08:21:07.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.487+0000 7f24d1000640 1 Processor -- start 2024-03-29T08:21:07.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.487+0000 7f24d1000640 1 -- start start 2024-03-29T08:21:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.487+0000 7f24d1000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24cc15ad20 0x7f24cc0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.487+0000 7f24d1000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24cc0b0ab0 con 0x7f24cc0b0540 2024-03-29T08:21:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.487+0000 7f24d1000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24cc0b0d90 con 0x7f24cc15a940 2024-03-29T08:21:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.487+0000 7f24d1000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc0b1070 con 0x7f24cc15ad20 2024-03-29T08:21:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.487+0000 7f24ca800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f24cc0b0540 0x7f24cc0cb630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42074/0 (socket says 172.21.15.46:42074) 2024-03-29T08:21:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.487+0000 7f24ca800640 1 -- 172.21.15.46:0/2952280573 learned_addr learned my addr 172.21.15.46:0/2952280573 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:07.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.487+0000 7f24c1400640 1 --2- 172.21.15.46:0/2952280573 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24cc15ad20 0x7f24cc0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:07.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.487+0000 7f24c9400640 1 -- 172.21.15.46:0/2952280573 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2061832475 0 0) 0x7f24cc0b0ab0 con 0x7f24cc0b0540 2024-03-29T08:21:07.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.487+0000 7f24c9400640 1 -- 172.21.15.46:0/2952280573 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24b0003530 con 0x7f24cc0b0540 2024-03-29T08:21:07.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.487+0000 7f24c9400640 1 -- 172.21.15.46:0/2952280573 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3754067643 0 0) 0x7f24cc0b0d90 con 0x7f24cc15a940 2024-03-29T08:21:07.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.487+0000 7f24c1400640 1 -- 172.21.15.46:0/2952280573 >> v1:172.21.15.73:6789/0 conn(0x7f24cc15a940 legacy=0x7f24cc0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.487+0000 7f24c1400640 1 -- 172.21.15.46:0/2952280573 >> v1:172.21.15.46:6789/0 conn(0x7f24cc0b0540 legacy=0x7f24cc0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.487+0000 7f24c1400640 1 -- 172.21.15.46:0/2952280573 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc0b1350 con 0x7f24cc15ad20 2024-03-29T08:21:07.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.491+0000 7f24c1400640 1 --2- 172.21.15.46:0/2952280573 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24cc15ad20 0x7f24cc0cecb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f24b40097f0 tx=0x7f24b401c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bcfb7dbd52117c6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:07.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.491+0000 7f24c9400640 1 -- 172.21.15.46:0/2952280573 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40ac620 con 0x7f24cc15ad20 2024-03-29T08:21:07.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.491+0000 7f24c9400640 1 -- 172.21.15.46:0/2952280573 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b40ac780 con 0x7f24cc15ad20 2024-03-29T08:21:07.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.491+0000 7f24c9400640 1 -- 172.21.15.46:0/2952280573 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b40acb10 con 0x7f24cc15ad20 2024-03-29T08:21:07.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.491+0000 7f24d1000640 1 -- 172.21.15.46:0/2952280573 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24cc15ad20 msgr2=0x7f24cc0cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.491+0000 7f24d1000640 1 --2- 172.21.15.46:0/2952280573 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24cc15ad20 0x7f24cc0cecb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f24b40097f0 tx=0x7f24b401c4b0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.491+0000 7f24d1000640 1 -- 172.21.15.46:0/2952280573 shutdown_connections 2024-03-29T08:21:07.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.491+0000 7f24d1000640 1 --2- 172.21.15.46:0/2952280573 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24cc15ad20 0x7f24cc0cecb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.491+0000 7f24d1000640 1 -- 172.21.15.46:0/2952280573 >> 172.21.15.46:0/2952280573 conn(0x7f24cc089fd0 msgr2=0x7f24cc08a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:07.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.491+0000 7f24d1000640 1 -- 172.21.15.46:0/2952280573 shutdown_connections 2024-03-29T08:21:07.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.491+0000 7f24d1000640 1 -- 172.21.15.46:0/2952280573 wait complete. 2024-03-29T08:21:07.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24d1000640 1 Processor -- start 2024-03-29T08:21:07.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24d1000640 1 -- start start 2024-03-29T08:21:07.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24d1000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24cc0b0540 0x7f24cc15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:07.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24d1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24cc15a940 0x7f24cc151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:07.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24d1000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24cc15ad20 0x7f24cc0c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24d1000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc0c5d60 con 0x7f24cc15a940 2024-03-29T08:21:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24d1000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc15df40 con 0x7f24cc0b0540 2024-03-29T08:21:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24d1000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc15d980 con 0x7f24cc15ad20 2024-03-29T08:21:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24c1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24cc15a940 0x7f24cc151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24c1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24cc15a940 0x7f24cc151290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54260/0 (socket says 172.21.15.46:54260) 2024-03-29T08:21:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24c1400640 1 -- 172.21.15.46:0/33820923 learned_addr learned my addr 172.21.15.46:0/33820923 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24ca800640 1 --2- 172.21.15.46:0/33820923 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24cc15ad20 0x7f24cc0c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24c9e00640 1 --2- 172.21.15.46:0/33820923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24cc0b0540 0x7f24cc15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24c1400640 1 -- 172.21.15.46:0/33820923 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24cc15ad20 msgr2=0x7f24cc0c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24c1400640 1 --2- 172.21.15.46:0/33820923 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24cc15ad20 0x7f24cc0c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24c1400640 1 -- 172.21.15.46:0/33820923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24cc0b0540 msgr2=0x7f24cc15d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24c1400640 1 --2- 172.21.15.46:0/33820923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24cc0b0540 0x7f24cc15d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24c1400640 1 -- 172.21.15.46:0/33820923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b4009420 con 0x7f24cc15a940 2024-03-29T08:21:07.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24c1400640 1 --2- 172.21.15.46:0/33820923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24cc15a940 0x7f24cc151290 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f24b40097f0 tx=0x7f24b401acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:07.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24c2a00640 1 -- 172.21.15.46:0/33820923 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4004020 con 0x7f24cc15a940 2024-03-29T08:21:07.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24c2a00640 1 -- 172.21.15.46:0/33820923 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b4004180 con 0x7f24cc15a940 2024-03-29T08:21:07.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24c2a00640 1 -- 172.21.15.46:0/33820923 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b4004510 con 0x7f24cc15a940 2024-03-29T08:21:07.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24d1000640 1 -- 172.21.15.46:0/33820923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24cc15dc60 con 0x7f24cc15a940 2024-03-29T08:21:07.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.495+0000 7f24d1000640 1 -- 172.21.15.46:0/33820923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24cc0c6270 con 0x7f24cc15a940 2024-03-29T08:21:07.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.499+0000 7f24d1000640 1 -- 172.21.15.46:0/33820923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24cc066c90 con 0x7f24cc15a940 2024-03-29T08:21:07.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.499+0000 7f24c2a00640 1 -- 172.21.15.46:0/33820923 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f24b40aa070 con 0x7f24cc15a940 2024-03-29T08:21:07.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.499+0000 7f24c2a00640 1 --2- 172.21.15.46:0/33820923 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f24a8066690 0x7f24a8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:07.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.499+0000 7f24c2a00640 1 -- 172.21.15.46:0/33820923 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f24b411dff0 con 0x7f24cc15a940 2024-03-29T08:21:07.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.499+0000 7f24c9e00640 1 --2- 172.21.15.46:0/33820923 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f24a8066690 0x7f24a8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:07.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.499+0000 7f24c9e00640 1 --2- 172.21.15.46:0/33820923 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f24a8066690 0x7f24a8068b50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0012f0 tx=0x7f24bc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.503+0000 7f24c2a00640 1 -- 172.21.15.46:0/33820923 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f24b40eb820 con 0x7f24cc15a940 2024-03-29T08:21:07.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.667+0000 7f24d1000640 1 -- 172.21.15.46:0/33820923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f24cc0cb7d0 con 0x7f24cc15a940 2024-03-29T08:21:07.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.667+0000 7f24c2a00640 1 -- 172.21.15.46:0/33820923 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v59) v1 ==== 84+0+65 (secure 0 0 0) 0x7f24b40eee40 con 0x7f24cc15a940 2024-03-29T08:21:07.668 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:07.668 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-03-29T08:21:07.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:07.668 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-03-29T08:21:07.669 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-03-29T08:21:07.669 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:07.669 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-03-29T08:21:07.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.667+0000 7f24d1000640 1 -- 172.21.15.46:0/33820923 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f24a8066690 msgr2=0x7f24a8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.667+0000 7f24d1000640 1 --2- 172.21.15.46:0/33820923 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f24a8066690 0x7f24a8068b50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0012f0 tx=0x7f24bc009040 comp rx=0 tx=0).stop 2024-03-29T08:21:07.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.667+0000 7f24d1000640 1 -- 172.21.15.46:0/33820923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24cc15a940 msgr2=0x7f24cc151290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.667+0000 7f24d1000640 1 --2- 172.21.15.46:0/33820923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24cc15a940 0x7f24cc151290 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f24b40097f0 tx=0x7f24b401acf0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.667+0000 7f24d1000640 1 -- 172.21.15.46:0/33820923 shutdown_connections 2024-03-29T08:21:07.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.667+0000 7f24d1000640 1 --2- 172.21.15.46:0/33820923 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f24a8066690 0x7f24a8068b50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.667+0000 7f24d1000640 1 --2- 172.21.15.46:0/33820923 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24cc15ad20 0x7f24cc0c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.667+0000 7f24d1000640 1 --2- 172.21.15.46:0/33820923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24cc15a940 0x7f24cc151290 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.667+0000 7f24d1000640 1 --2- 172.21.15.46:0/33820923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24cc0b0540 0x7f24cc15d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.667+0000 7f24d1000640 1 -- 172.21.15.46:0/33820923 >> 172.21.15.46:0/33820923 conn(0x7f24cc089fd0 msgr2=0x7f24cc159b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:07.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.667+0000 7f24d1000640 1 -- 172.21.15.46:0/33820923 shutdown_connections 2024-03-29T08:21:07.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.667+0000 7f24d1000640 1 -- 172.21.15.46:0/33820923 wait complete. 2024-03-29T08:21:07.689 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T08:21:07.690 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T08:21:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.831+0000 7fa373200640 1 Processor -- start 2024-03-29T08:21:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.831+0000 7fa373200640 1 -- start start 2024-03-29T08:21:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.835+0000 7fa373200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3740d0260 0x7fa3740d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.835+0000 7fa373200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3740cd780 0x7fa37406c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.835+0000 7fa373200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa37406b5c0 con 0x7fa3740cdbe0 2024-03-29T08:21:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.835+0000 7fa373200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3740cf150 con 0x7fa3740cd780 2024-03-29T08:21:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.835+0000 7fa373200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37406b830 con 0x7fa3740d0260 2024-03-29T08:21:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.835+0000 7fa369400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3740cd780 0x7fa37406c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.835+0000 7fa369400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3740cd780 0x7fa37406c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54272/0 (socket says 172.21.15.46:54272) 2024-03-29T08:21:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.835+0000 7fa372800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa3740cdbe0 0x7fa3740afe50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42076/0 (socket says 172.21.15.46:42076) 2024-03-29T08:21:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.835+0000 7fa369400640 1 -- 172.21.15.46:0/4040212819 learned_addr learned my addr 172.21.15.46:0/4040212819 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.835+0000 7fa371e00640 1 --2- 172.21.15.46:0/4040212819 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3740d0260 0x7fa3740d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.835+0000 7fa369400640 1 -- 172.21.15.46:0/4040212819 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3740d0260 msgr2=0x7fa3740d0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.835+0000 7fa369400640 1 --2- 172.21.15.46:0/4040212819 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3740d0260 0x7fa3740d0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.835+0000 7fa369400640 1 -- 172.21.15.46:0/4040212819 >> v1:172.21.15.46:6789/0 conn(0x7fa3740cdbe0 legacy=0x7fa3740afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.835+0000 7fa369400640 1 -- 172.21.15.46:0/4040212819 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37406bb10 con 0x7fa3740cd780 2024-03-29T08:21:07.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.835+0000 7fa371400640 1 -- 172.21.15.46:0/4040212819 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771988655 0 0) 0x7fa37406b5c0 con 0x7fa3740cdbe0 2024-03-29T08:21:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.835+0000 7fa369400640 1 --2- 172.21.15.46:0/4040212819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3740cd780 0x7fa37406c030 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009870 tx=0x7fa35c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e504b3459bf0f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.835+0000 7fa371400640 1 -- 172.21.15.46:0/4040212819 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c0a9640 con 0x7fa3740cd780 2024-03-29T08:21:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa371400640 1 -- 172.21.15.46:0/4040212819 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c0a97a0 con 0x7fa3740cd780 2024-03-29T08:21:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa371400640 1 -- 172.21.15.46:0/4040212819 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c0a9b30 con 0x7fa3740cd780 2024-03-29T08:21:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa373200640 1 -- 172.21.15.46:0/4040212819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3740cd780 msgr2=0x7fa37406c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa373200640 1 --2- 172.21.15.46:0/4040212819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3740cd780 0x7fa37406c030 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009870 tx=0x7fa35c01c520 comp rx=0 tx=0).stop 2024-03-29T08:21:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa373200640 1 -- 172.21.15.46:0/4040212819 shutdown_connections 2024-03-29T08:21:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa373200640 1 --2- 172.21.15.46:0/4040212819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3740cd780 0x7fa37406c030 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa373200640 1 --2- 172.21.15.46:0/4040212819 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3740d0260 0x7fa3740d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa373200640 1 -- 172.21.15.46:0/4040212819 >> 172.21.15.46:0/4040212819 conn(0x7fa374099820 msgr2=0x7fa374089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa373200640 1 -- 172.21.15.46:0/4040212819 shutdown_connections 2024-03-29T08:21:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa373200640 1 -- 172.21.15.46:0/4040212819 wait complete. 2024-03-29T08:21:07.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa373200640 1 Processor -- start 2024-03-29T08:21:07.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa373200640 1 -- start start 2024-03-29T08:21:07.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa373200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3740cd780 0x7fa374152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:07.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa373200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3740d0260 0x7fa37406be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:07.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa373200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa374157bd0 0x7fa37406c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:07.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa373200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa374153170 con 0x7fa3740d0260 2024-03-29T08:21:07.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa373200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa374158060 con 0x7fa374157bd0 2024-03-29T08:21:07.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa373200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa374158340 con 0x7fa3740cd780 2024-03-29T08:21:07.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa369400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3740d0260 0x7fa37406be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:07.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa369400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3740d0260 0x7fa37406be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54278/0 (socket says 172.21.15.46:54278) 2024-03-29T08:21:07.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa369400640 1 -- 172.21.15.46:0/10294015 learned_addr learned my addr 172.21.15.46:0/10294015 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:07.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa369400640 1 -- 172.21.15.46:0/10294015 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3740cd780 msgr2=0x7fa374152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa369400640 1 --2- 172.21.15.46:0/10294015 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3740cd780 0x7fa374152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa369400640 1 -- 172.21.15.46:0/10294015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa374157bd0 msgr2=0x7fa37406c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:07.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa369400640 1 --2- 172.21.15.46:0/10294015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa374157bd0 0x7fa37406c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:07.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.839+0000 7fa369400640 1 -- 172.21.15.46:0/10294015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c0094d0 con 0x7fa3740d0260 2024-03-29T08:21:07.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.843+0000 7fa369400640 1 --2- 172.21.15.46:0/10294015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3740d0260 0x7fa37406be80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa35c01a150 tx=0x7fa35c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:07.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.843+0000 7fa36aa00640 1 -- 172.21.15.46:0/10294015 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c004220 con 0x7fa3740d0260 2024-03-29T08:21:07.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.843+0000 7fa36aa00640 1 -- 172.21.15.46:0/10294015 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c004380 con 0x7fa3740d0260 2024-03-29T08:21:07.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.843+0000 7fa36aa00640 1 -- 172.21.15.46:0/10294015 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c003820 con 0x7fa3740d0260 2024-03-29T08:21:07.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.843+0000 7fa373200640 1 -- 172.21.15.46:0/10294015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa37415d410 con 0x7fa3740d0260 2024-03-29T08:21:07.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.843+0000 7fa373200640 1 -- 172.21.15.46:0/10294015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa374158620 con 0x7fa3740d0260 2024-03-29T08:21:07.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.843+0000 7fa373200640 1 -- 172.21.15.46:0/10294015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3740c7430 con 0x7fa3740d0260 2024-03-29T08:21:07.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.843+0000 7fa36aa00640 1 -- 172.21.15.46:0/10294015 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa35c0a7070 con 0x7fa3740d0260 2024-03-29T08:21:07.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.843+0000 7fa36aa00640 1 --2- 172.21.15.46:0/10294015 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa350066690 0x7fa350068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:07.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.843+0000 7fa36aa00640 1 -- 172.21.15.46:0/10294015 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa35c11de90 con 0x7fa3740d0260 2024-03-29T08:21:07.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.843+0000 7fa371e00640 1 --2- 172.21.15.46:0/10294015 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa350066690 0x7fa350068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:07.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.847+0000 7fa371e00640 1 --2- 172.21.15.46:0/10294015 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa350066690 0x7fa350068b50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa364006aa0 tx=0x7fa364006a30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:07.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:07.847+0000 7fa36aa00640 1 -- 172.21.15.46:0/10294015 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa35c0eb4b0 con 0x7fa3740d0260 2024-03-29T08:21:08.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:08.011+0000 7fa373200640 1 -- 172.21.15.46:0/10294015 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa37404e040 con 0x7fa3740d0260 2024-03-29T08:21:08.584 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:21:08.588 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:21:08.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:08.923+0000 7fa36aa00640 1 -- 172.21.15.46:0/10294015 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v60) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa35c0044f0 con 0x7fa3740d0260 2024-03-29T08:21:08.927 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-03-29T08:21:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:08.927+0000 7fa373200640 1 -- 172.21.15.46:0/10294015 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa350066690 msgr2=0x7fa350068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:08.927+0000 7fa373200640 1 --2- 172.21.15.46:0/10294015 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa350066690 0x7fa350068b50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa364006aa0 tx=0x7fa364006a30 comp rx=0 tx=0).stop 2024-03-29T08:21:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:08.927+0000 7fa373200640 1 -- 172.21.15.46:0/10294015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3740d0260 msgr2=0x7fa37406be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:08.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:08.927+0000 7fa373200640 1 --2- 172.21.15.46:0/10294015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3740d0260 0x7fa37406be80 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fa35c01a150 tx=0x7fa35c0192c0 comp rx=0 tx=0).stop 2024-03-29T08:21:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:08.927+0000 7fa373200640 1 -- 172.21.15.46:0/10294015 shutdown_connections 2024-03-29T08:21:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:08.927+0000 7fa373200640 1 --2- 172.21.15.46:0/10294015 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa350066690 0x7fa350068b50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:08.927+0000 7fa373200640 1 --2- 172.21.15.46:0/10294015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa374157bd0 0x7fa37406c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:08.927+0000 7fa373200640 1 --2- 172.21.15.46:0/10294015 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3740d0260 0x7fa37406be80 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:08.927+0000 7fa373200640 1 --2- 172.21.15.46:0/10294015 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3740cd780 0x7fa374152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:08.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:08.927+0000 7fa373200640 1 -- 172.21.15.46:0/10294015 >> 172.21.15.46:0/10294015 conn(0x7fa374099820 msgr2=0x7fa37415f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:08.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:08.927+0000 7fa373200640 1 -- 172.21.15.46:0/10294015 shutdown_connections 2024-03-29T08:21:08.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:08.927+0000 7fa373200640 1 -- 172.21.15.46:0/10294015 wait complete. 2024-03-29T08:21:08.957 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T08:21:09.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.107+0000 7f27c7e00640 1 Processor -- start 2024-03-29T08:21:09.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.107+0000 7f27c7e00640 1 -- start start 2024-03-29T08:21:09.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.111+0000 7f27c7e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27c81597b0 0x7f27c815a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:09.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.111+0000 7f27c7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f27c80cc200 0x7f27c80ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:09.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.111+0000 7f27c7e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c815abe0 con 0x7f27c80b17b0 2024-03-29T08:21:09.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.111+0000 7f27c7e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c80cae20 con 0x7f27c80cc200 2024-03-29T08:21:09.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.111+0000 7f27c7e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c80cb2b0 con 0x7f27c81597b0 2024-03-29T08:21:09.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.111+0000 7f27c7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f27c80cc200 0x7f27c80ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:09.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.111+0000 7f27c7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f27c80cc200 0x7f27c80ccf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54282/0 (socket says 172.21.15.46:54282) 2024-03-29T08:21:09.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.111+0000 7f27c7400640 1 -- 172.21.15.46:0/4201720046 learned_addr learned my addr 172.21.15.46:0/4201720046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:09.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.111+0000 7f27c6a00640 1 --2- 172.21.15.46:0/4201720046 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27c81597b0 0x7f27c815a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:09.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.111+0000 7f27c7400640 1 -- 172.21.15.46:0/4201720046 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27c81597b0 msgr2=0x7f27c815a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:09.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.111+0000 7f27c7400640 1 --2- 172.21.15.46:0/4201720046 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27c81597b0 0x7f27c815a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:09.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.111+0000 7f27c7400640 1 -- 172.21.15.46:0/4201720046 >> v1:172.21.15.73:6789/0 conn(0x7f27c80b17b0 legacy=0x7f27c80cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:09.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.111+0000 7f27c7400640 1 -- 172.21.15.46:0/4201720046 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c80ca210 con 0x7f27c80cc200 2024-03-29T08:21:09.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.111+0000 7f27c7400640 1 --2- 172.21.15.46:0/4201720046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c80cc200 0x7f27c80ccf80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009870 tx=0x7f27c001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d7cdba36ab5f8f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:09.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c6000640 1 -- 172.21.15.46:0/4201720046 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c00a95f0 con 0x7f27c80cc200 2024-03-29T08:21:09.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c6000640 1 -- 172.21.15.46:0/4201720046 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c00a9750 con 0x7f27c80cc200 2024-03-29T08:21:09.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c6000640 1 -- 172.21.15.46:0/4201720046 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c00a9ae0 con 0x7f27c80cc200 2024-03-29T08:21:09.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7e00640 1 -- 172.21.15.46:0/4201720046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c80cc200 msgr2=0x7f27c80ccf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:09.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7e00640 1 --2- 172.21.15.46:0/4201720046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c80cc200 0x7f27c80ccf80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009870 tx=0x7f27c001c530 comp rx=0 tx=0).stop 2024-03-29T08:21:09.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7e00640 1 -- 172.21.15.46:0/4201720046 shutdown_connections 2024-03-29T08:21:09.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7e00640 1 --2- 172.21.15.46:0/4201720046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c80cc200 0x7f27c80ccf80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:09.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7e00640 1 --2- 172.21.15.46:0/4201720046 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27c81597b0 0x7f27c815a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:09.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7e00640 1 -- 172.21.15.46:0/4201720046 >> 172.21.15.46:0/4201720046 conn(0x7f27c8098a40 msgr2=0x7f27c80b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:09.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7e00640 1 -- 172.21.15.46:0/4201720046 shutdown_connections 2024-03-29T08:21:09.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7e00640 1 -- 172.21.15.46:0/4201720046 wait complete. 2024-03-29T08:21:09.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7e00640 1 Processor -- start 2024-03-29T08:21:09.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7e00640 1 -- start start 2024-03-29T08:21:09.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27c80b17b0 0x7f27c80b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:09.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27c80cc200 0x7f27c8148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:09.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c81597b0 0x7f27c80cf100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:09.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c81531d0 con 0x7f27c81597b0 2024-03-29T08:21:09.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c81534b0 con 0x7f27c80cc200 2024-03-29T08:21:09.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c8153790 con 0x7f27c80b17b0 2024-03-29T08:21:09.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c81597b0 0x7f27c80cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:09.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c81597b0 0x7f27c80cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54286/0 (socket says 172.21.15.46:54286) 2024-03-29T08:21:09.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c6a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27c80b17b0 0x7f27c80b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:09.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27c7400640 1 -- 172.21.15.46:0/2380997905 learned_addr learned my addr 172.21.15.46:0/2380997905 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:09.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.115+0000 7f27be000640 1 --2- 172.21.15.46:0/2380997905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27c80cc200 0x7f27c8148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:09.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.119+0000 7f27c7400640 1 -- 172.21.15.46:0/2380997905 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27c80b17b0 msgr2=0x7f27c80b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:09.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.119+0000 7f27c7400640 1 --2- 172.21.15.46:0/2380997905 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27c80b17b0 0x7f27c80b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:09.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.119+0000 7f27c7400640 1 -- 172.21.15.46:0/2380997905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27c80cc200 msgr2=0x7f27c8148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:09.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.119+0000 7f27c7400640 1 --2- 172.21.15.46:0/2380997905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27c80cc200 0x7f27c8148d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:09.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.119+0000 7f27c7400640 1 -- 172.21.15.46:0/2380997905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c00094d0 con 0x7f27c81597b0 2024-03-29T08:21:09.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.119+0000 7f27c6a00640 1 --2- 172.21.15.46:0/2380997905 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27c80b17b0 0x7f27c80b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:21:09.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.119+0000 7f27c7400640 1 --2- 172.21.15.46:0/2380997905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c81597b0 0x7f27c80cf100 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f27c00ab040 tx=0x7f27c001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.119+0000 7f27bf400640 1 -- 172.21.15.46:0/2380997905 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c00bc650 con 0x7f27c81597b0 2024-03-29T08:21:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.119+0000 7f27bf400640 1 -- 172.21.15.46:0/2380997905 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c00bc7b0 con 0x7f27c81597b0 2024-03-29T08:21:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.119+0000 7f27c7e00640 1 -- 172.21.15.46:0/2380997905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c8153a70 con 0x7f27c81597b0 2024-03-29T08:21:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.119+0000 7f27bf400640 1 -- 172.21.15.46:0/2380997905 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c00bcb60 con 0x7f27c81597b0 2024-03-29T08:21:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.119+0000 7f27c7e00640 1 -- 172.21.15.46:0/2380997905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c814c290 con 0x7f27c81597b0 2024-03-29T08:21:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.119+0000 7f27c7e00640 1 -- 172.21.15.46:0/2380997905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27c8153f90 con 0x7f27c81597b0 2024-03-29T08:21:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.119+0000 7f27bf400640 1 -- 172.21.15.46:0/2380997905 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f27c00bccc0 con 0x7f27c81597b0 2024-03-29T08:21:09.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.119+0000 7f27bf400640 1 --2- 172.21.15.46:0/2380997905 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27a40666c0 0x7f27a4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:09.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.123+0000 7f27bf400640 1 -- 172.21.15.46:0/2380997905 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f27c011eae0 con 0x7f27c81597b0 2024-03-29T08:21:09.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.123+0000 7f27c6a00640 1 --2- 172.21.15.46:0/2380997905 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27a40666c0 0x7f27a4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:09.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.123+0000 7f27c6a00640 1 --2- 172.21.15.46:0/2380997905 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27a40666c0 0x7f27a4068b80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009fd0 tx=0x7f27b800b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:09.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.127+0000 7f27bf400640 1 -- 172.21.15.46:0/2380997905 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f27c8153f90 con 0x7f27c81597b0 2024-03-29T08:21:09.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.287+0000 7f27c7e00640 1 -- 172.21.15.46:0/2380997905 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f27c8153f90 con 0x7f27c81597b0 2024-03-29T08:21:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.931+0000 7f27bf400640 1 -- 172.21.15.46:0/2380997905 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v61) v1 ==== 170+0+0 (secure 0 0 0) 0x7f27c8153f90 con 0x7f27c81597b0 2024-03-29T08:21:09.933 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-03-29T08:21:09.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.935+0000 7f27c7e00640 1 -- 172.21.15.46:0/2380997905 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27a40666c0 msgr2=0x7f27a4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:09.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.935+0000 7f27c7e00640 1 --2- 172.21.15.46:0/2380997905 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27a40666c0 0x7f27a4068b80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f27b8009fd0 tx=0x7f27b800b040 comp rx=0 tx=0).stop 2024-03-29T08:21:09.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.935+0000 7f27c7e00640 1 -- 172.21.15.46:0/2380997905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c81597b0 msgr2=0x7f27c80cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:09.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.935+0000 7f27c7e00640 1 --2- 172.21.15.46:0/2380997905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c81597b0 0x7f27c80cf100 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f27c00ab040 tx=0x7f27c001ad70 comp rx=0 tx=0).stop 2024-03-29T08:21:09.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.939+0000 7f27c7e00640 1 -- 172.21.15.46:0/2380997905 shutdown_connections 2024-03-29T08:21:09.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.939+0000 7f27c7e00640 1 --2- 172.21.15.46:0/2380997905 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27a40666c0 0x7f27a4068b80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:09.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.939+0000 7f27c7e00640 1 --2- 172.21.15.46:0/2380997905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27c80cc200 0x7f27c8148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:09.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.939+0000 7f27c7e00640 1 --2- 172.21.15.46:0/2380997905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27c81597b0 0x7f27c80cf100 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:09.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.939+0000 7f27c7e00640 1 --2- 172.21.15.46:0/2380997905 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f27c80b17b0 0x7f27c80b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:09.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.939+0000 7f27c7e00640 1 -- 172.21.15.46:0/2380997905 >> 172.21.15.46:0/2380997905 conn(0x7f27c8098a40 msgr2=0x7f27c815ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:09.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.939+0000 7f27c7e00640 1 -- 172.21.15.46:0/2380997905 shutdown_connections 2024-03-29T08:21:09.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:09.943+0000 7f27c7e00640 1 -- 172.21.15.46:0/2380997905 wait complete. 2024-03-29T08:21:09.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T08:21:10.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.119+0000 7f254a000640 1 Processor -- start 2024-03-29T08:21:10.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.119+0000 7f254a000640 1 -- start start 2024-03-29T08:21:10.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.119+0000 7f254a000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2544143eb0 0x7f25440cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.119+0000 7f254a000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25440b2360 con 0x7f25440b1d40 2024-03-29T08:21:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.119+0000 7f254a000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f254415e500 con 0x7f254415efe0 2024-03-29T08:21:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.119+0000 7f254a000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254415e7e0 con 0x7f2544143eb0 2024-03-29T08:21:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.119+0000 7f2543e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f25440b1d40 0x7f254415ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42082/0 (socket says 172.21.15.46:42082) 2024-03-29T08:21:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.119+0000 7f2543e00640 1 -- 172.21.15.46:0/621867516 learned_addr learned my addr 172.21.15.46:0/621867516 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:10.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.119+0000 7f2543400640 1 --2- 172.21.15.46:0/621867516 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2544143eb0 0x7f25440cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.119+0000 7f2542000640 1 -- 172.21.15.46:0/621867516 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866955135 0 0) 0x7f25440b2360 con 0x7f25440b1d40 2024-03-29T08:21:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.119+0000 7f2542000640 1 -- 172.21.15.46:0/621867516 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2528003530 con 0x7f25440b1d40 2024-03-29T08:21:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.123+0000 7f2542000640 1 -- 172.21.15.46:0/621867516 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3705029840 0 0) 0x7f254415e500 con 0x7f254415efe0 2024-03-29T08:21:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.123+0000 7f2542000640 1 -- 172.21.15.46:0/621867516 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25440b2360 con 0x7f254415efe0 2024-03-29T08:21:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.123+0000 7f2543400640 1 -- 172.21.15.46:0/621867516 >> v1:172.21.15.73:6789/0 conn(0x7f254415efe0 legacy=0x7f25440b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.123+0000 7f2543400640 1 -- 172.21.15.46:0/621867516 >> v1:172.21.15.46:6789/0 conn(0x7f25440b1d40 legacy=0x7f254415ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.123+0000 7f2543400640 1 -- 172.21.15.46:0/621867516 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254415eac0 con 0x7f2544143eb0 2024-03-29T08:21:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.123+0000 7f2543400640 1 --2- 172.21.15.46:0/621867516 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2544143eb0 0x7f25440cb790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f25340097f0 tx=0x7f253401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4041484f7ae09c57 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.123+0000 7f2542000640 1 -- 172.21.15.46:0/621867516 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25340ab600 con 0x7f2544143eb0 2024-03-29T08:21:10.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.123+0000 7f2542000640 1 -- 172.21.15.46:0/621867516 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25340a7050 con 0x7f2544143eb0 2024-03-29T08:21:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.123+0000 7f2542000640 1 -- 172.21.15.46:0/621867516 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25340ab980 con 0x7f2544143eb0 2024-03-29T08:21:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.123+0000 7f254a000640 1 -- 172.21.15.46:0/621867516 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2544143eb0 msgr2=0x7f25440cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.123+0000 7f254a000640 1 --2- 172.21.15.46:0/621867516 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2544143eb0 0x7f25440cb790 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f25340097f0 tx=0x7f253401c4e0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.123+0000 7f254a000640 1 -- 172.21.15.46:0/621867516 shutdown_connections 2024-03-29T08:21:10.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.123+0000 7f254a000640 1 --2- 172.21.15.46:0/621867516 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2544143eb0 0x7f25440cb790 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.123+0000 7f254a000640 1 -- 172.21.15.46:0/621867516 >> 172.21.15.46:0/621867516 conn(0x7f254406a390 msgr2=0x7f25440cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.123+0000 7f254a000640 1 -- 172.21.15.46:0/621867516 shutdown_connections 2024-03-29T08:21:10.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.123+0000 7f254a000640 1 -- 172.21.15.46:0/621867516 wait complete. 2024-03-29T08:21:10.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f254a000640 1 Processor -- start 2024-03-29T08:21:10.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f254a000640 1 -- start start 2024-03-29T08:21:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f254a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25440b1d40 0x7f254413f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f254a000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2544143eb0 0x7f254413d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f254a000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f254415efe0 0x7f2544139a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f254a000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254413f0a0 con 0x7f25440b1d40 2024-03-29T08:21:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f254a000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254413e490 con 0x7f2544143eb0 2024-03-29T08:21:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f254a000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254413e770 con 0x7f254415efe0 2024-03-29T08:21:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f2543400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25440b1d40 0x7f254413f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f2543400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25440b1d40 0x7f254413f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54296/0 (socket says 172.21.15.46:54296) 2024-03-29T08:21:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f2543400640 1 -- 172.21.15.46:0/2601312377 learned_addr learned my addr 172.21.15.46:0/2601312377 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f2543e00640 1 --2- 172.21.15.46:0/2601312377 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f254415efe0 0x7f2544139a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f2542a00640 1 --2- 172.21.15.46:0/2601312377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2544143eb0 0x7f254413d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f2543400640 1 -- 172.21.15.46:0/2601312377 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f254415efe0 msgr2=0x7f2544139a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f2543400640 1 --2- 172.21.15.46:0/2601312377 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f254415efe0 0x7f2544139a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f2543400640 1 -- 172.21.15.46:0/2601312377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2544143eb0 msgr2=0x7f254413d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f2543400640 1 --2- 172.21.15.46:0/2601312377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2544143eb0 0x7f254413d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f2543400640 1 -- 172.21.15.46:0/2601312377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2534009420 con 0x7f25440b1d40 2024-03-29T08:21:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f2543400640 1 --2- 172.21.15.46:0/2601312377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25440b1d40 0x7f254413f530 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f25340190f0 tx=0x7f253401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:10.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f253b400640 1 -- 172.21.15.46:0/2601312377 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25340b7650 con 0x7f25440b1d40 2024-03-29T08:21:10.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.127+0000 7f253b400640 1 -- 172.21.15.46:0/2601312377 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254415bc90 con 0x7f25440b1d40 2024-03-29T08:21:10.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.131+0000 7f254a000640 1 -- 172.21.15.46:0/2601312377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254413ea50 con 0x7f25440b1d40 2024-03-29T08:21:10.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.131+0000 7f253b400640 1 -- 172.21.15.46:0/2601312377 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25340b7a10 con 0x7f25440b1d40 2024-03-29T08:21:10.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.131+0000 7f254a000640 1 -- 172.21.15.46:0/2601312377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254413fa70 con 0x7f25440b1d40 2024-03-29T08:21:10.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.131+0000 7f254a000640 1 -- 172.21.15.46:0/2601312377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2544071a70 con 0x7f25440b1d40 2024-03-29T08:21:10.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.131+0000 7f253b400640 1 -- 172.21.15.46:0/2601312377 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f25340b7b70 con 0x7f25440b1d40 2024-03-29T08:21:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.131+0000 7f253b400640 1 --2- 172.21.15.46:0/2601312377 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f25240666e0 0x7f2524068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.131+0000 7f253b400640 1 -- 172.21.15.46:0/2601312377 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f253411f200 con 0x7f25440b1d40 2024-03-29T08:21:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.135+0000 7f2542a00640 1 --2- 172.21.15.46:0/2601312377 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f25240666e0 0x7f2524068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:10.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.135+0000 7f2542a00640 1 --2- 172.21.15.46:0/2601312377 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f25240666e0 0x7f2524068ba0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2544165d20 tx=0x7f2530009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:10.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.135+0000 7f253b400640 1 -- 172.21.15.46:0/2601312377 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f25340ec660 con 0x7f25440b1d40 2024-03-29T08:21:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.331+0000 7f254a000640 1 -- 172.21.15.46:0/2601312377 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f254413e8b0 con 0x7f25440b1d40 2024-03-29T08:21:10.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.343+0000 7f253b400640 1 -- 172.21.15.46:0/2601312377 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v28) v1 ==== 172+0+0 (secure 0 0 0) 0x7f254413e8b0 con 0x7f25440b1d40 2024-03-29T08:21:10.351 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 8 and data pool 9 2024-03-29T08:21:10.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.351+0000 7f254a000640 1 -- 172.21.15.46:0/2601312377 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f25240666e0 msgr2=0x7f2524068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.351+0000 7f254a000640 1 --2- 172.21.15.46:0/2601312377 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f25240666e0 0x7f2524068ba0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2544165d20 tx=0x7f2530009040 comp rx=0 tx=0).stop 2024-03-29T08:21:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.351+0000 7f254a000640 1 -- 172.21.15.46:0/2601312377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25440b1d40 msgr2=0x7f254413f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.351+0000 7f254a000640 1 --2- 172.21.15.46:0/2601312377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25440b1d40 0x7f254413f530 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f25340190f0 tx=0x7f253401ad20 comp rx=0 tx=0).stop 2024-03-29T08:21:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.351+0000 7f254a000640 1 -- 172.21.15.46:0/2601312377 shutdown_connections 2024-03-29T08:21:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.351+0000 7f254a000640 1 --2- 172.21.15.46:0/2601312377 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f25240666e0 0x7f2524068ba0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.351+0000 7f254a000640 1 --2- 172.21.15.46:0/2601312377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2544143eb0 0x7f254413d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.351+0000 7f254a000640 1 --2- 172.21.15.46:0/2601312377 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f254415efe0 0x7f2544139a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.351+0000 7f254a000640 1 --2- 172.21.15.46:0/2601312377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25440b1d40 0x7f254413f530 secure :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f25340190f0 tx=0x7f253401ad20 comp rx=0 tx=0).stop 2024-03-29T08:21:10.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.351+0000 7f254a000640 1 -- 172.21.15.46:0/2601312377 >> 172.21.15.46:0/2601312377 conn(0x7f254406a390 msgr2=0x7f25440b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.355+0000 7f254a000640 1 -- 172.21.15.46:0/2601312377 shutdown_connections 2024-03-29T08:21:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.355+0000 7f254a000640 1 -- 172.21.15.46:0/2601312377 wait complete. 2024-03-29T08:21:10.390 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:21:10.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.527+0000 7f4545000640 1 Processor -- start 2024-03-29T08:21:10.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.527+0000 7f4545000640 1 -- start start 2024-03-29T08:21:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.527+0000 7f4545000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f45400ccd10 0x7f4540098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.527+0000 7f4545000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f454015cdb0 0x7f4540098f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.527+0000 7f4545000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45400d0030 con 0x7f454015c9d0 2024-03-29T08:21:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.527+0000 7f4545000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45400cf420 con 0x7f45400ccd10 2024-03-29T08:21:10.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.527+0000 7f4545000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45400cf700 con 0x7f454015cdb0 2024-03-29T08:21:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.531+0000 7f453de00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f454015c9d0 0x7f45400cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42086/0 (socket says 172.21.15.46:42086) 2024-03-29T08:21:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.531+0000 7f453de00640 1 -- 172.21.15.46:0/2820194582 learned_addr learned my addr 172.21.15.46:0/2820194582 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.531+0000 7f4535400640 1 --2- 172.21.15.46:0/2820194582 >> v2:172.21.15.46:3300/0 conn(0x7f45400ccd10 0x7f4540098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.531+0000 7f453e800640 1 --2- 172.21.15.46:0/2820194582 >> v2:172.21.15.73:3300/0 conn(0x7f454015cdb0 0x7f4540098f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.531+0000 7f453d400640 1 -- 172.21.15.46:0/2820194582 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220036173 0 0) 0x7f45400d0030 con 0x7f454015c9d0 2024-03-29T08:21:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.531+0000 7f4535400640 1 -- 172.21.15.46:0/2820194582 >> v2:172.21.15.73:3300/0 conn(0x7f454015cdb0 msgr2=0x7f4540098f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.531+0000 7f4535400640 1 --2- 172.21.15.46:0/2820194582 >> v2:172.21.15.73:3300/0 conn(0x7f454015cdb0 0x7f4540098f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.531+0000 7f4535400640 1 -- 172.21.15.46:0/2820194582 >> v1:172.21.15.46:6789/0 conn(0x7f454015c9d0 legacy=0x7f45400cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.531+0000 7f4535400640 1 -- 172.21.15.46:0/2820194582 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45400cf9e0 con 0x7f45400ccd10 2024-03-29T08:21:10.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.531+0000 7f4535400640 1 --2- 172.21.15.46:0/2820194582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45400ccd10 0x7f4540098a40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4540071920 tx=0x7f452801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08dff0bb2cb12 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:10.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.531+0000 7f453d400640 1 -- 172.21.15.46:0/2820194582 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45280a7070 con 0x7f45400ccd10 2024-03-29T08:21:10.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.531+0000 7f453d400640 1 -- 172.21.15.46:0/2820194582 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45280ab430 con 0x7f45400ccd10 2024-03-29T08:21:10.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.531+0000 7f453d400640 1 -- 172.21.15.46:0/2820194582 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45280ab7c0 con 0x7f45400ccd10 2024-03-29T08:21:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f4545000640 1 -- 172.21.15.46:0/2820194582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45400ccd10 msgr2=0x7f4540098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f4545000640 1 --2- 172.21.15.46:0/2820194582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45400ccd10 0x7f4540098a40 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4540071920 tx=0x7f452801c450 comp rx=0 tx=0).stop 2024-03-29T08:21:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f4545000640 1 -- 172.21.15.46:0/2820194582 shutdown_connections 2024-03-29T08:21:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f4545000640 1 --2- 172.21.15.46:0/2820194582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45400ccd10 0x7f4540098a40 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f4545000640 1 --2- 172.21.15.46:0/2820194582 >> v2:172.21.15.73:3300/0 conn(0x7f454015cdb0 0x7f4540098f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f4545000640 1 -- 172.21.15.46:0/2820194582 >> 172.21.15.46:0/2820194582 conn(0x7f45400b21a0 msgr2=0x7f45400b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f4545000640 1 -- 172.21.15.46:0/2820194582 shutdown_connections 2024-03-29T08:21:10.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f4545000640 1 -- 172.21.15.46:0/2820194582 wait complete. 2024-03-29T08:21:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f4545000640 1 Processor -- start 2024-03-29T08:21:10.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f4545000640 1 -- start start 2024-03-29T08:21:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f4545000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45400ccd10 0x7f4540150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f4545000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f454015c9d0 0x7f454015afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f4545000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f454015cdb0 0x7f45400b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f4545000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4540151000 con 0x7f45400ccd10 2024-03-29T08:21:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f4545000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4540159cb0 con 0x7f454015c9d0 2024-03-29T08:21:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f4545000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4540159f90 con 0x7f454015cdb0 2024-03-29T08:21:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f453de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45400ccd10 0x7f4540150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f453de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45400ccd10 0x7f4540150ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54308/0 (socket says 172.21.15.46:54308) 2024-03-29T08:21:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f453de00640 1 -- 172.21.15.46:0/2590557952 learned_addr learned my addr 172.21.15.46:0/2590557952 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f453e800640 1 --2- 172.21.15.46:0/2590557952 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f454015cdb0 0x7f45400b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f4535400640 1 --2- 172.21.15.46:0/2590557952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f454015c9d0 0x7f454015afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:10.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f453de00640 1 -- 172.21.15.46:0/2590557952 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f454015cdb0 msgr2=0x7f45400b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f453de00640 1 --2- 172.21.15.46:0/2590557952 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f454015cdb0 0x7f45400b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f453de00640 1 -- 172.21.15.46:0/2590557952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f454015c9d0 msgr2=0x7f454015afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f453de00640 1 --2- 172.21.15.46:0/2590557952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f454015c9d0 0x7f454015afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.535+0000 7f453de00640 1 -- 172.21.15.46:0/2590557952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528009420 con 0x7f45400ccd10 2024-03-29T08:21:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.539+0000 7f453de00640 1 --2- 172.21.15.46:0/2590557952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45400ccd10 0x7f4540150ac0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f45300011f0 tx=0x7f453000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.539+0000 7f4536a00640 1 -- 172.21.15.46:0/2590557952 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45300074e0 con 0x7f45400ccd10 2024-03-29T08:21:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.539+0000 7f4536a00640 1 -- 172.21.15.46:0/2590557952 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4530007640 con 0x7f45400ccd10 2024-03-29T08:21:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.539+0000 7f4545000640 1 -- 172.21.15.46:0/2590557952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454015a270 con 0x7f45400ccd10 2024-03-29T08:21:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.539+0000 7f4536a00640 1 -- 172.21.15.46:0/2590557952 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530004a10 con 0x7f45400ccd10 2024-03-29T08:21:10.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.539+0000 7f4545000640 1 -- 172.21.15.46:0/2590557952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454015a830 con 0x7f45400ccd10 2024-03-29T08:21:10.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.539+0000 7f4545000640 1 -- 172.21.15.46:0/2590557952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45400c54b0 con 0x7f45400ccd10 2024-03-29T08:21:10.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.539+0000 7f4536a00640 1 -- 172.21.15.46:0/2590557952 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4530004b70 con 0x7f45400ccd10 2024-03-29T08:21:10.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.539+0000 7f4536a00640 1 --2- 172.21.15.46:0/2590557952 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f451c066690 0x7f451c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:10.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.539+0000 7f4536a00640 1 -- 172.21.15.46:0/2590557952 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45300836e0 con 0x7f45400ccd10 2024-03-29T08:21:10.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.539+0000 7f4535400640 1 --2- 172.21.15.46:0/2590557952 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f451c066690 0x7f451c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:10.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.543+0000 7f4535400640 1 --2- 172.21.15.46:0/2590557952 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f451c066690 0x7f451c068b50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f45280af000 tx=0x7f45280a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:10.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.543+0000 7f4536a00640 1 -- 172.21.15.46:0/2590557952 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4530050a60 con 0x7f45400ccd10 2024-03-29T08:21:10.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.707+0000 7f4545000640 1 -- 172.21.15.46:0/2590557952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4540066c90 con 0x7f45400ccd10 2024-03-29T08:21:10.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.711+0000 7f4536a00640 1 -- 172.21.15.46:0/2590557952 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16062 (secure 0 0 0) 0x7f4530054080 con 0x7f45400ccd10 2024-03-29T08:21:10.712 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:10.712 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":62,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:21:10.336843+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:21:08.061567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T08:21:09.336464+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:21:10.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.711+0000 7f4545000640 1 -- 172.21.15.46:0/2590557952 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f451c066690 msgr2=0x7f451c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.711+0000 7f4545000640 1 --2- 172.21.15.46:0/2590557952 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f451c066690 0x7f451c068b50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f45280af000 tx=0x7f45280a7000 comp rx=0 tx=0).stop 2024-03-29T08:21:10.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.711+0000 7f4545000640 1 -- 172.21.15.46:0/2590557952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45400ccd10 msgr2=0x7f4540150ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.711+0000 7f4545000640 1 --2- 172.21.15.46:0/2590557952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45400ccd10 0x7f4540150ac0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f45300011f0 tx=0x7f453000c9b0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.711+0000 7f4545000640 1 -- 172.21.15.46:0/2590557952 shutdown_connections 2024-03-29T08:21:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.711+0000 7f4545000640 1 --2- 172.21.15.46:0/2590557952 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f451c066690 0x7f451c068b50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.711+0000 7f4545000640 1 --2- 172.21.15.46:0/2590557952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f454015c9d0 0x7f454015afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.711+0000 7f4545000640 1 --2- 172.21.15.46:0/2590557952 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f454015cdb0 0x7f45400b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.711+0000 7f4545000640 1 --2- 172.21.15.46:0/2590557952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45400ccd10 0x7f4540150ac0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.711+0000 7f4545000640 1 -- 172.21.15.46:0/2590557952 >> 172.21.15.46:0/2590557952 conn(0x7f45400b21a0 msgr2=0x7f4540072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.711+0000 7f4545000640 1 -- 172.21.15.46:0/2590557952 shutdown_connections 2024-03-29T08:21:10.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.711+0000 7f4545000640 1 -- 172.21.15.46:0/2590557952 wait complete. 2024-03-29T08:21:10.734 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:21:10.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.871+0000 7f51c8400640 1 Processor -- start 2024-03-29T08:21:10.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.871+0000 7f51c8400640 1 -- start start 2024-03-29T08:21:10.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.871+0000 7f51c8400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51c01593d0 0x7f51c006cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:10.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.871+0000 7f51c8400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c006d130 con 0x7f51c015d230 2024-03-29T08:21:10.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.871+0000 7f51c8400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c015b8f0 con 0x7f51c015cd90 2024-03-29T08:21:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.871+0000 7f51c8400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c015ad50 con 0x7f51c01593d0 2024-03-29T08:21:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.871+0000 7f51c5000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f51c015d230 0x7f51c015d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42092/0 (socket says 172.21.15.46:42092) 2024-03-29T08:21:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.871+0000 7f51c5000640 1 -- 172.21.15.46:0/922214793 learned_addr learned my addr 172.21.15.46:0/922214793 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.875+0000 7f51c5a00640 1 --2- 172.21.15.46:0/922214793 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51c01593d0 0x7f51c006cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.875+0000 7f51bfe00640 1 -- 172.21.15.46:0/922214793 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798799809 0 0) 0x7f51c006d130 con 0x7f51c015d230 2024-03-29T08:21:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.875+0000 7f51bfe00640 1 -- 172.21.15.46:0/922214793 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51a8003530 con 0x7f51c015d230 2024-03-29T08:21:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.875+0000 7f51bfe00640 1 -- 172.21.15.46:0/922214793 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594568175 0 0) 0x7f51c015b8f0 con 0x7f51c015cd90 2024-03-29T08:21:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.875+0000 7f51bfe00640 1 -- 172.21.15.46:0/922214793 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51c006d130 con 0x7f51c015cd90 2024-03-29T08:21:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.875+0000 7f51c5a00640 1 -- 172.21.15.46:0/922214793 >> v1:172.21.15.73:6789/0 conn(0x7f51c015cd90 legacy=0x7f51c01597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.875+0000 7f51c5a00640 1 -- 172.21.15.46:0/922214793 >> v1:172.21.15.46:6789/0 conn(0x7f51c015d230 legacy=0x7f51c015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.875+0000 7f51bfe00640 1 -- 172.21.15.46:0/922214793 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 538012665 0 0) 0x7f51a8003530 con 0x7f51c015d230 2024-03-29T08:21:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.875+0000 7f51c5a00640 1 -- 172.21.15.46:0/922214793 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c015afc0 con 0x7f51c01593d0 2024-03-29T08:21:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.875+0000 7f51c5a00640 1 --2- 172.21.15.46:0/922214793 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51c01593d0 0x7f51c006cbf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f51b4009820 tx=0x7f51b401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ed91474e039204c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.875+0000 7f51bfe00640 1 -- 172.21.15.46:0/922214793 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac001680 con 0x7f51c01593d0 2024-03-29T08:21:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.875+0000 7f51bfe00640 1 -- 172.21.15.46:0/922214793 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b40a8050 con 0x7f51c01593d0 2024-03-29T08:21:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.875+0000 7f51bfe00640 1 -- 172.21.15.46:0/922214793 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b40ac620 con 0x7f51c01593d0 2024-03-29T08:21:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c8400640 1 -- 172.21.15.46:0/922214793 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51c01593d0 msgr2=0x7f51c006cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c8400640 1 --2- 172.21.15.46:0/922214793 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51c01593d0 0x7f51c006cbf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f51b4009820 tx=0x7f51b401c3e0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c8400640 1 -- 172.21.15.46:0/922214793 shutdown_connections 2024-03-29T08:21:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c8400640 1 --2- 172.21.15.46:0/922214793 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51c01593d0 0x7f51c006cbf0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c8400640 1 -- 172.21.15.46:0/922214793 >> 172.21.15.46:0/922214793 conn(0x7f51c0066110 msgr2=0x7f51c0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c8400640 1 -- 172.21.15.46:0/922214793 shutdown_connections 2024-03-29T08:21:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c8400640 1 -- 172.21.15.46:0/922214793 wait complete. 2024-03-29T08:21:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c8400640 1 Processor -- start 2024-03-29T08:21:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c8400640 1 -- start start 2024-03-29T08:21:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c8400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51c01593d0 0x7f51c0142fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c8400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51c015cd90 0x7f51c0151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c8400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51c015d230 0x7f51c0156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c8400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00cd3f0 con 0x7f51c015cd90 2024-03-29T08:21:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c8400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00cc770 con 0x7f51c01593d0 2024-03-29T08:21:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c8400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c00cca50 con 0x7f51c015d230 2024-03-29T08:21:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51c015cd90 0x7f51c0151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51c015cd90 0x7f51c0151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54322/0 (socket says 172.21.15.46:54322) 2024-03-29T08:21:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c5000640 1 -- 172.21.15.46:0/2666611302 learned_addr learned my addr 172.21.15.46:0/2666611302 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c5a00640 1 --2- 172.21.15.46:0/2666611302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51c01593d0 0x7f51c0142fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c6400640 1 --2- 172.21.15.46:0/2666611302 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51c015d230 0x7f51c0156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c5000640 1 -- 172.21.15.46:0/2666611302 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51c015d230 msgr2=0x7f51c0156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c5000640 1 --2- 172.21.15.46:0/2666611302 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51c015d230 0x7f51c0156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c5000640 1 -- 172.21.15.46:0/2666611302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51c01593d0 msgr2=0x7f51c0142fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c5000640 1 --2- 172.21.15.46:0/2666611302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51c01593d0 0x7f51c0142fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c5000640 1 -- 172.21.15.46:0/2666611302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b4009420 con 0x7f51c015cd90 2024-03-29T08:21:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c5a00640 1 --2- 172.21.15.46:0/2666611302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51c01593d0 0x7f51c0142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:10.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.879+0000 7f51c6400640 1 --2- 172.21.15.46:0/2666611302 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51c015d230 0x7f51c0156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.883+0000 7f51c5000640 1 --2- 172.21.15.46:0/2666611302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51c015cd90 0x7f51c0151b30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00c750 tx=0x7f51ac00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.883+0000 7f51bd600640 1 -- 172.21.15.46:0/2666611302 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac005ae0 con 0x7f51c015cd90 2024-03-29T08:21:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.883+0000 7f51bd600640 1 -- 172.21.15.46:0/2666611302 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac005c40 con 0x7f51c015cd90 2024-03-29T08:21:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.883+0000 7f51c8400640 1 -- 172.21.15.46:0/2666611302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c00ccd30 con 0x7f51c015cd90 2024-03-29T08:21:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.883+0000 7f51bd600640 1 -- 172.21.15.46:0/2666611302 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac007070 con 0x7f51c015cd90 2024-03-29T08:21:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.883+0000 7f51c8400640 1 -- 172.21.15.46:0/2666611302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c01434e0 con 0x7f51c015cd90 2024-03-29T08:21:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.883+0000 7f51c8400640 1 -- 172.21.15.46:0/2666611302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c0066c90 con 0x7f51c015cd90 2024-03-29T08:21:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.883+0000 7f51bd600640 1 -- 172.21.15.46:0/2666611302 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f51ac0071f0 con 0x7f51c015cd90 2024-03-29T08:21:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.883+0000 7f51bd600640 1 --2- 172.21.15.46:0/2666611302 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f51a0066690 0x7f51a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.883+0000 7f51bd600640 1 -- 172.21.15.46:0/2666611302 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51ac084060 con 0x7f51c015cd90 2024-03-29T08:21:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.883+0000 7f51c5a00640 1 --2- 172.21.15.46:0/2666611302 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f51a0066690 0x7f51a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.887+0000 7f51c5a00640 1 --2- 172.21.15.46:0/2666611302 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f51a0066690 0x7f51a0068b50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f51b40ae000 tx=0x7f51b4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:10.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:10.887+0000 7f51bd600640 1 -- 172.21.15.46:0/2666611302 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f51ac0b5980 con 0x7f51c015cd90 2024-03-29T08:21:11.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.055+0000 7f51c8400640 1 -- 172.21.15.46:0/2666611302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51c01653b0 con 0x7f51c015cd90 2024-03-29T08:21:11.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.055+0000 7f51bd600640 1 -- 172.21.15.46:0/2666611302 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16062 (secure 0 0 0) 0x7f51ac0512a0 con 0x7f51c015cd90 2024-03-29T08:21:11.058 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:11.058 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":62,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:21:10.336843+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:21:08.061567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T08:21:09.336464+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:21:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.059+0000 7f51c8400640 1 -- 172.21.15.46:0/2666611302 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f51a0066690 msgr2=0x7f51a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.059+0000 7f51c8400640 1 --2- 172.21.15.46:0/2666611302 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f51a0066690 0x7f51a0068b50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f51b40ae000 tx=0x7f51b4008040 comp rx=0 tx=0).stop 2024-03-29T08:21:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.059+0000 7f51c8400640 1 -- 172.21.15.46:0/2666611302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51c015cd90 msgr2=0x7f51c0151b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.059+0000 7f51c8400640 1 --2- 172.21.15.46:0/2666611302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51c015cd90 0x7f51c0151b30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00c750 tx=0x7f51ac00cc10 comp rx=0 tx=0).stop 2024-03-29T08:21:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.059+0000 7f51c8400640 1 -- 172.21.15.46:0/2666611302 shutdown_connections 2024-03-29T08:21:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.059+0000 7f51c8400640 1 --2- 172.21.15.46:0/2666611302 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f51a0066690 0x7f51a0068b50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.059+0000 7f51c8400640 1 --2- 172.21.15.46:0/2666611302 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f51c015d230 0x7f51c0156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:11.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.059+0000 7f51c8400640 1 --2- 172.21.15.46:0/2666611302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51c015cd90 0x7f51c0151b30 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.059+0000 7f51c8400640 1 --2- 172.21.15.46:0/2666611302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51c01593d0 0x7f51c0142fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.059+0000 7f51c8400640 1 -- 172.21.15.46:0/2666611302 >> 172.21.15.46:0/2666611302 conn(0x7f51c0066110 msgr2=0x7f51c00b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.059+0000 7f51c8400640 1 -- 172.21.15.46:0/2666611302 shutdown_connections 2024-03-29T08:21:11.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.059+0000 7f51c8400640 1 -- 172.21.15.46:0/2666611302 wait complete. 2024-03-29T08:21:11.081 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T08:21:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.223+0000 7fdf14e00640 1 Processor -- start 2024-03-29T08:21:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.223+0000 7fdf14e00640 1 -- start start 2024-03-29T08:21:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.223+0000 7fdf14e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf1015c950 0x7fdf1006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:11.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.223+0000 7fdf14e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf100cfb20 con 0x7fdf100cea30 2024-03-29T08:21:11.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.223+0000 7fdf14e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf100cef10 con 0x7fdf10160db0 2024-03-29T08:21:11.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.223+0000 7fdf14e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf100cf1f0 con 0x7fdf1015c950 2024-03-29T08:21:11.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.223+0000 7fdf07e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdf100cea30 0x7fdf101606a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42098/0 (socket says 172.21.15.46:42098) 2024-03-29T08:21:11.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.223+0000 7fdf07e00640 1 -- 172.21.15.46:0/1100212490 learned_addr learned my addr 172.21.15.46:0/1100212490 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:11.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.223+0000 7fdf0de00640 1 --2- 172.21.15.46:0/1100212490 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf1015c950 0x7fdf1006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:11.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.223+0000 7fdf0d400640 1 -- 172.21.15.46:0/1100212490 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410825019 0 0) 0x7fdf100cfb20 con 0x7fdf100cea30 2024-03-29T08:21:11.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.223+0000 7fdf0d400640 1 -- 172.21.15.46:0/1100212490 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdef4003530 con 0x7fdf100cea30 2024-03-29T08:21:11.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.223+0000 7fdf0d400640 1 -- 172.21.15.46:0/1100212490 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2022564742 0 0) 0x7fdf100cef10 con 0x7fdf10160db0 2024-03-29T08:21:11.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.223+0000 7fdf0d400640 1 -- 172.21.15.46:0/1100212490 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf100cfb20 con 0x7fdf10160db0 2024-03-29T08:21:11.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.223+0000 7fdf0de00640 1 -- 172.21.15.46:0/1100212490 >> v1:172.21.15.73:6789/0 conn(0x7fdf10160db0 legacy=0x7fdf100cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:11.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.223+0000 7fdf0de00640 1 -- 172.21.15.46:0/1100212490 >> v1:172.21.15.46:6789/0 conn(0x7fdf100cea30 legacy=0x7fdf101606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:11.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.223+0000 7fdf0de00640 1 -- 172.21.15.46:0/1100212490 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf100cf4d0 con 0x7fdf1015c950 2024-03-29T08:21:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.227+0000 7fdf0de00640 1 --2- 172.21.15.46:0/1100212490 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf1015c950 0x7fdf1006bbe0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdf000097f0 tx=0x7fdf0001ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3db76a7cb2b0a3e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.227+0000 7fdf0d400640 1 -- 172.21.15.46:0/1100212490 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf000a89d0 con 0x7fdf1015c950 2024-03-29T08:21:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.227+0000 7fdf0d400640 1 -- 172.21.15.46:0/1100212490 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf000a8b30 con 0x7fdf1015c950 2024-03-29T08:21:11.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.227+0000 7fdf0d400640 1 -- 172.21.15.46:0/1100212490 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf000b0440 con 0x7fdf1015c950 2024-03-29T08:21:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.227+0000 7fdf14e00640 1 -- 172.21.15.46:0/1100212490 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf1015c950 msgr2=0x7fdf1006bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.227+0000 7fdf14e00640 1 --2- 172.21.15.46:0/1100212490 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf1015c950 0x7fdf1006bbe0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdf000097f0 tx=0x7fdf0001ecd0 comp rx=0 tx=0).stop 2024-03-29T08:21:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.227+0000 7fdf14e00640 1 -- 172.21.15.46:0/1100212490 shutdown_connections 2024-03-29T08:21:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.227+0000 7fdf14e00640 1 --2- 172.21.15.46:0/1100212490 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf1015c950 0x7fdf1006bbe0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.227+0000 7fdf14e00640 1 -- 172.21.15.46:0/1100212490 >> 172.21.15.46:0/1100212490 conn(0x7fdf100b21b0 msgr2=0x7fdf100b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.227+0000 7fdf14e00640 1 -- 172.21.15.46:0/1100212490 shutdown_connections 2024-03-29T08:21:11.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.227+0000 7fdf14e00640 1 -- 172.21.15.46:0/1100212490 wait complete. 2024-03-29T08:21:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf14e00640 1 Processor -- start 2024-03-29T08:21:11.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf14e00640 1 -- start start 2024-03-29T08:21:11.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf14e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf100cea30 0x7fdf100b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:11.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf14e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1015c950 0x7fdf10159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:11.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf14e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf10160db0 0x7fdf100b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:11.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf14e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf10159d60 con 0x7fdf1015c950 2024-03-29T08:21:11.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf14e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf100b16e0 con 0x7fdf10160db0 2024-03-29T08:21:11.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf14e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf100b5180 con 0x7fdf100cea30 2024-03-29T08:21:11.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1015c950 0x7fdf10159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1015c950 0x7fdf10159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54326/0 (socket says 172.21.15.46:54326) 2024-03-29T08:21:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf07e00640 1 -- 172.21.15.46:0/3298516596 learned_addr learned my addr 172.21.15.46:0/3298516596 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf0de00640 1 --2- 172.21.15.46:0/3298516596 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf100cea30 0x7fdf100b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf0e800640 1 --2- 172.21.15.46:0/3298516596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf10160db0 0x7fdf100b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf07e00640 1 -- 172.21.15.46:0/3298516596 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf100cea30 msgr2=0x7fdf100b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf07e00640 1 --2- 172.21.15.46:0/3298516596 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf100cea30 0x7fdf100b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf07e00640 1 -- 172.21.15.46:0/3298516596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf10160db0 msgr2=0x7fdf100b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf07e00640 1 --2- 172.21.15.46:0/3298516596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf10160db0 0x7fdf100b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf07e00640 1 -- 172.21.15.46:0/3298516596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf00009420 con 0x7fdf1015c950 2024-03-29T08:21:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf07e00640 1 --2- 172.21.15.46:0/3298516596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1015c950 0x7fdf10159820 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdef800c4d0 tx=0x7fdef800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf06000640 1 -- 172.21.15.46:0/3298516596 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef8005cf0 con 0x7fdf1015c950 2024-03-29T08:21:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf06000640 1 -- 172.21.15.46:0/3298516596 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef8005e50 con 0x7fdf1015c950 2024-03-29T08:21:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf14e00640 1 -- 172.21.15.46:0/3298516596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf100b5460 con 0x7fdf1015c950 2024-03-29T08:21:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf06000640 1 -- 172.21.15.46:0/3298516596 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef80072b0 con 0x7fdf1015c950 2024-03-29T08:21:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.231+0000 7fdf14e00640 1 -- 172.21.15.46:0/3298516596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1009a420 con 0x7fdf1015c950 2024-03-29T08:21:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.235+0000 7fdf14e00640 1 -- 172.21.15.46:0/3298516596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1009e690 con 0x7fdf1015c950 2024-03-29T08:21:11.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.235+0000 7fdf06000640 1 -- 172.21.15.46:0/3298516596 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdef8007410 con 0x7fdf1015c950 2024-03-29T08:21:11.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.235+0000 7fdf06000640 1 --2- 172.21.15.46:0/3298516596 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeec066690 0x7fdeec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:11.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.235+0000 7fdf06000640 1 -- 172.21.15.46:0/3298516596 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdef8083740 con 0x7fdf1015c950 2024-03-29T08:21:11.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.235+0000 7fdf0de00640 1 --2- 172.21.15.46:0/3298516596 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeec066690 0x7fdeec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:11.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.235+0000 7fdf0de00640 1 --2- 172.21.15.46:0/3298516596 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeec066690 0x7fdeec068b50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdf000b2000 tx=0x7fdf00008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.239+0000 7fdf06000640 1 -- 172.21.15.46:0/3298516596 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdef8050b40 con 0x7fdf1015c950 2024-03-29T08:21:11.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:11.435+0000 7fdf14e00640 1 -- 172.21.15.46:0/3298516596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdf1006c600 con 0x7fdf1015c950 2024-03-29T08:21:12.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.347+0000 7fdf06000640 1 -- 172.21.15.46:0/3298516596 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdef80170b0 con 0x7fdf1015c950 2024-03-29T08:21:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.351+0000 7fdf14e00640 1 -- 172.21.15.46:0/3298516596 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeec066690 msgr2=0x7fdeec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.351+0000 7fdf14e00640 1 --2- 172.21.15.46:0/3298516596 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeec066690 0x7fdeec068b50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdf000b2000 tx=0x7fdf00008040 comp rx=0 tx=0).stop 2024-03-29T08:21:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.351+0000 7fdf14e00640 1 -- 172.21.15.46:0/3298516596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1015c950 msgr2=0x7fdf10159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.351+0000 7fdf14e00640 1 --2- 172.21.15.46:0/3298516596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1015c950 0x7fdf10159820 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdef800c4d0 tx=0x7fdef800c990 comp rx=0 tx=0).stop 2024-03-29T08:21:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.351+0000 7fdf14e00640 1 -- 172.21.15.46:0/3298516596 shutdown_connections 2024-03-29T08:21:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.351+0000 7fdf14e00640 1 --2- 172.21.15.46:0/3298516596 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeec066690 0x7fdeec068b50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.351+0000 7fdf14e00640 1 --2- 172.21.15.46:0/3298516596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf10160db0 0x7fdf100b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.351+0000 7fdf14e00640 1 --2- 172.21.15.46:0/3298516596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1015c950 0x7fdf10159820 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.351+0000 7fdf14e00640 1 --2- 172.21.15.46:0/3298516596 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf100cea30 0x7fdf100b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.351+0000 7fdf14e00640 1 -- 172.21.15.46:0/3298516596 >> 172.21.15.46:0/3298516596 conn(0x7fdf100b21b0 msgr2=0x7fdf1015e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.351+0000 7fdf14e00640 1 -- 172.21.15.46:0/3298516596 shutdown_connections 2024-03-29T08:21:12.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.351+0000 7fdf14e00640 1 -- 172.21.15.46:0/3298516596 wait complete. 2024-03-29T08:21:12.375 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-29T08:21:12.376 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T08:21:12.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53c7600640 1 Processor -- start 2024-03-29T08:21:12.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53c7600640 1 -- start start 2024-03-29T08:21:12.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53c7600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53c00cbea0 0x7f53c00b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:12.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53c7600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f53c015efe0 0x7f53c00b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53c7600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c00b20b0 con 0x7f53c0143eb0 2024-03-29T08:21:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53c7600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c00b2360 con 0x7f53c015efe0 2024-03-29T08:21:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53c7600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c015e500 con 0x7f53c00cbea0 2024-03-29T08:21:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53c4c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f53c0143eb0 0x7f53c00cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42112/0 (socket says 172.21.15.46:42112) 2024-03-29T08:21:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53c4c00640 1 -- 172.21.15.46:0/2686849625 learned_addr learned my addr 172.21.15.46:0/2686849625 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53b7e00640 1 --2- 172.21.15.46:0/2686849625 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53c00cbea0 0x7f53c00b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53c5600640 1 --2- 172.21.15.46:0/2686849625 >> v2:172.21.15.73:3300/0 conn(0x7f53c015efe0 0x7f53c00b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53bfe00640 1 -- 172.21.15.46:0/2686849625 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2596084035 0 0) 0x7f53c00b20b0 con 0x7f53c0143eb0 2024-03-29T08:21:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53bfe00640 1 -- 172.21.15.46:0/2686849625 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53a4003530 con 0x7f53c0143eb0 2024-03-29T08:21:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53bfe00640 1 -- 172.21.15.46:0/2686849625 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 630167421 0 0) 0x7f53a4003530 con 0x7f53c0143eb0 2024-03-29T08:21:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53bfe00640 1 -- 172.21.15.46:0/2686849625 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53c00cbea0 msgr2=0x7f53c00b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53bfe00640 1 --2- 172.21.15.46:0/2686849625 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53c00cbea0 0x7f53c00b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53bfe00640 1 -- 172.21.15.46:0/2686849625 >> v2:172.21.15.73:3300/0 conn(0x7f53c015efe0 msgr2=0x7f53c00b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:12.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53bfe00640 1 --2- 172.21.15.46:0/2686849625 >> v2:172.21.15.73:3300/0 conn(0x7f53c015efe0 0x7f53c00b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53bfe00640 1 -- 172.21.15.46:0/2686849625 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c015e7e0 con 0x7f53c0143eb0 2024-03-29T08:21:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53bfe00640 1 -- 172.21.15.46:0/2686849625 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f53b0001680 con 0x7f53c0143eb0 2024-03-29T08:21:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53bfe00640 1 -- 172.21.15.46:0/2686849625 >> v1:172.21.15.46:6789/0 conn(0x7f53c0143eb0 legacy=0x7f53c00cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53bfe00640 1 --2- 172.21.15.46:0/2686849625 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53c00cbea0 0x7f53a4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53bfe00640 1 --2- 172.21.15.46:0/2686849625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53a4003f00 0x7f53a40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53bfe00640 1 --2- 172.21.15.46:0/2686849625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53a4004d10 0x7f53a4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53bfe00640 1 -- 172.21.15.46:0/2686849625 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c00b2360 con 0x7f53a4003f00 2024-03-29T08:21:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53bfe00640 1 -- 172.21.15.46:0/2686849625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c015e500 con 0x7f53a4004d10 2024-03-29T08:21:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53bfe00640 1 -- 172.21.15.46:0/2686849625 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a400b990 con 0x7f53c00cbea0 2024-03-29T08:21:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53b7e00640 1 --2- 172.21.15.46:0/2686849625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53a4003f00 0x7f53a40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53c4c00640 1 --2- 172.21.15.46:0/2686849625 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53c00cbea0 0x7f53a4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.523+0000 7f53c5600640 1 --2- 172.21.15.46:0/2686849625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53a4004d10 0x7f53a4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.527+0000 7f53b7e00640 1 -- 172.21.15.46:0/2686849625 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53c00cbea0 msgr2=0x7f53a4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.527+0000 7f53b7e00640 1 --2- 172.21.15.46:0/2686849625 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53c00cbea0 0x7f53a4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.527+0000 7f53b7e00640 1 -- 172.21.15.46:0/2686849625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53a4004d10 msgr2=0x7f53a4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:12.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.527+0000 7f53b7e00640 1 --2- 172.21.15.46:0/2686849625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53a4004d10 0x7f53a4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.527+0000 7f53b7e00640 1 -- 172.21.15.46:0/2686849625 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a4003530 con 0x7f53a4003f00 2024-03-29T08:21:12.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.527+0000 7f53b7e00640 1 --2- 172.21.15.46:0/2686849625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53a4003f00 0x7f53a40047d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f53a8009ae0 tx=0x7f53a801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e051af5ca460d8e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:12.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.527+0000 7f53bfe00640 1 -- 172.21.15.46:0/2686849625 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a80a5660 con 0x7f53a4003f00 2024-03-29T08:21:12.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.527+0000 7f53bfe00640 1 -- 172.21.15.46:0/2686849625 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a80a57c0 con 0x7f53a4003f00 2024-03-29T08:21:12.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.527+0000 7f53bfe00640 1 -- 172.21.15.46:0/2686849625 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a80a5b70 con 0x7f53a4003f00 2024-03-29T08:21:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.527+0000 7f53c7600640 1 -- 172.21.15.46:0/2686849625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53a4003f00 msgr2=0x7f53a40047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.527+0000 7f53c7600640 1 --2- 172.21.15.46:0/2686849625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53a4003f00 0x7f53a40047d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f53a8009ae0 tx=0x7f53a801ca10 comp rx=0 tx=0).stop 2024-03-29T08:21:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.527+0000 7f53c5600640 1 -- 172.21.15.46:0/2686849625 reap_dead start 2024-03-29T08:21:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.527+0000 7f53c7600640 1 -- 172.21.15.46:0/2686849625 shutdown_connections 2024-03-29T08:21:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.527+0000 7f53c7600640 1 -- 172.21.15.46:0/2686849625 >> 172.21.15.46:0/2686849625 conn(0x7f53c006a390 msgr2=0x7f53c00cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c7600640 1 -- 172.21.15.46:0/2686849625 shutdown_connections 2024-03-29T08:21:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c7600640 1 -- 172.21.15.46:0/2686849625 wait complete. 2024-03-29T08:21:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c7600640 1 Processor -- start 2024-03-29T08:21:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c7600640 1 -- start start 2024-03-29T08:21:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c7600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53a4003f00 0x7f53c015cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c7600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53c0140490 0x7f53c0140910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c7600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c0159c80 0x7f53c015a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c7600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c015a640 con 0x7f53c0159c80 2024-03-29T08:21:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c7600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c01f2880 con 0x7f53c0140490 2024-03-29T08:21:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c7600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c0140f30 con 0x7f53a4003f00 2024-03-29T08:21:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c0159c80 0x7f53c015a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c0159c80 0x7f53c015a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54344/0 (socket says 172.21.15.46:54344) 2024-03-29T08:21:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c5600640 1 -- 172.21.15.46:0/2215507314 learned_addr learned my addr 172.21.15.46:0/2215507314 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c4c00640 1 --2- 172.21.15.46:0/2215507314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53a4003f00 0x7f53c015cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53b7e00640 1 --2- 172.21.15.46:0/2215507314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53c0140490 0x7f53c0140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c5600640 1 -- 172.21.15.46:0/2215507314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53a4003f00 msgr2=0x7f53c015cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c5600640 1 --2- 172.21.15.46:0/2215507314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53a4003f00 0x7f53c015cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c5600640 1 -- 172.21.15.46:0/2215507314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53c0140490 msgr2=0x7f53c0140910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:12.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c5600640 1 --2- 172.21.15.46:0/2215507314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53c0140490 0x7f53c0140910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:12.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c5600640 1 -- 172.21.15.46:0/2215507314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c0141210 con 0x7f53c0159c80 2024-03-29T08:21:12.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53c5600640 1 --2- 172.21.15.46:0/2215507314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c0159c80 0x7f53c015a100 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f53c00b43e0 tx=0x7f53b800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:12.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.531+0000 7f53bd600640 1 -- 172.21.15.46:0/2215507314 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b800e040 con 0x7f53c0159c80 2024-03-29T08:21:12.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.535+0000 7f53bd600640 1 -- 172.21.15.46:0/2215507314 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b800f040 con 0x7f53c0159c80 2024-03-29T08:21:12.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.535+0000 7f53c7600640 1 -- 172.21.15.46:0/2215507314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c01414f0 con 0x7f53c0159c80 2024-03-29T08:21:12.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.535+0000 7f53bd600640 1 -- 172.21.15.46:0/2215507314 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b8015430 con 0x7f53c0159c80 2024-03-29T08:21:12.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.535+0000 7f53c7600640 1 -- 172.21.15.46:0/2215507314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c01e4ce0 con 0x7f53c0159c80 2024-03-29T08:21:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.535+0000 7f53c7600640 1 -- 172.21.15.46:0/2215507314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c00b0f80 con 0x7f53c0159c80 2024-03-29T08:21:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.535+0000 7f53bd600640 1 -- 172.21.15.46:0/2215507314 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f53b8015590 con 0x7f53c0159c80 2024-03-29T08:21:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.535+0000 7f53bd600640 1 --2- 172.21.15.46:0/2215507314 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f539c0666c0 0x7f539c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.535+0000 7f53bd600640 1 -- 172.21.15.46:0/2215507314 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53b8084870 con 0x7f53c0159c80 2024-03-29T08:21:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.535+0000 7f53c4c00640 1 --2- 172.21.15.46:0/2215507314 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f539c0666c0 0x7f539c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.535+0000 7f53c4c00640 1 --2- 172.21.15.46:0/2215507314 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f539c0666c0 0x7f539c068b80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f53b0005180 tx=0x7f53b0007900 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:12.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.539+0000 7f53bd600640 1 -- 172.21.15.46:0/2215507314 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f53c00b0f80 con 0x7f53c0159c80 2024-03-29T08:21:12.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:12.735+0000 7f53c7600640 1 -- 172.21.15.46:0/2215507314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f53c00b0f80 con 0x7f53c0159c80 2024-03-29T08:21:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.347+0000 7f53bd600640 1 -- 172.21.15.46:0/2215507314 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f53c00b0f80 con 0x7f53c0159c80 2024-03-29T08:21:13.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.351+0000 7f53c7600640 1 -- 172.21.15.46:0/2215507314 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f539c0666c0 msgr2=0x7f539c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.351+0000 7f53c7600640 1 --2- 172.21.15.46:0/2215507314 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f539c0666c0 0x7f539c068b80 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f53b0005180 tx=0x7f53b0007900 comp rx=0 tx=0).stop 2024-03-29T08:21:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.351+0000 7f53c7600640 1 -- 172.21.15.46:0/2215507314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c0159c80 msgr2=0x7f53c015a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.351+0000 7f53c7600640 1 --2- 172.21.15.46:0/2215507314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c0159c80 0x7f53c015a100 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f53c00b43e0 tx=0x7f53b800f1b0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.351+0000 7f53c7600640 1 -- 172.21.15.46:0/2215507314 shutdown_connections 2024-03-29T08:21:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.351+0000 7f53c7600640 1 --2- 172.21.15.46:0/2215507314 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f539c0666c0 0x7f539c068b80 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.351+0000 7f53c7600640 1 --2- 172.21.15.46:0/2215507314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53c0140490 0x7f53c0140910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.351+0000 7f53c7600640 1 --2- 172.21.15.46:0/2215507314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53c0159c80 0x7f53c015a100 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.351+0000 7f53c7600640 1 --2- 172.21.15.46:0/2215507314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53a4003f00 0x7f53c015cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.351+0000 7f53c7600640 1 -- 172.21.15.46:0/2215507314 >> 172.21.15.46:0/2215507314 conn(0x7f53c006a390 msgr2=0x7f53c006cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.351+0000 7f53c7600640 1 -- 172.21.15.46:0/2215507314 shutdown_connections 2024-03-29T08:21:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.351+0000 7f53c7600640 1 -- 172.21.15.46:0/2215507314 wait complete. 2024-03-29T08:21:13.377 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T08:21:13.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:21:13.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1df800640 1 Processor -- start 2024-03-29T08:21:13.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1df800640 1 -- start start 2024-03-29T08:21:13.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1df800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1d815a790 0x7fb1d80cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1df800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1d815ab70 0x7fb1d8098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1df800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb1d80cc430 0x7fb1d8098f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1df800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d80cf700 con 0x7fb1d815a790 2024-03-29T08:21:13.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1df800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d80ce660 con 0x7fb1d80cc430 2024-03-29T08:21:13.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1df800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d80ce940 con 0x7fb1d815ab70 2024-03-29T08:21:13.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1dce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1d815a790 0x7fb1d80cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1dce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1d815a790 0x7fb1d80cbef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54346/0 (socket says 172.21.15.46:54346) 2024-03-29T08:21:13.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1dce00640 1 -- 172.21.15.46:0/2533301279 learned_addr learned my addr 172.21.15.46:0/2533301279 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:13.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1cfe00640 1 --2- 172.21.15.46:0/2533301279 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1d815ab70 0x7fb1d8098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1dd800640 1 --2- 172.21.15.46:0/2533301279 >> v2:172.21.15.73:3300/0 conn(0x7fb1d80cc430 0x7fb1d8098f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1dce00640 1 -- 172.21.15.46:0/2533301279 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1d815ab70 msgr2=0x7fb1d8098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:13.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1dce00640 1 --2- 172.21.15.46:0/2533301279 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1d815ab70 0x7fb1d8098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1dce00640 1 -- 172.21.15.46:0/2533301279 >> v2:172.21.15.73:3300/0 conn(0x7fb1d80cc430 msgr2=0x7fb1d8098f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:13.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1dce00640 1 --2- 172.21.15.46:0/2533301279 >> v2:172.21.15.73:3300/0 conn(0x7fb1d80cc430 0x7fb1d8098f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1dce00640 1 -- 172.21.15.46:0/2533301279 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d80cec20 con 0x7fb1d815a790 2024-03-29T08:21:13.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.519+0000 7fb1dd800640 1 --2- 172.21.15.46:0/2533301279 >> v2:172.21.15.73:3300/0 conn(0x7fb1d80cc430 0x7fb1d8098f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:13.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.523+0000 7fb1dce00640 1 --2- 172.21.15.46:0/2533301279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1d815a790 0x7fb1d80cbef0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8071b80 tx=0x7fb1c801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6ff38301f8cbb0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:13.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.523+0000 7fb1d7e00640 1 -- 172.21.15.46:0/2533301279 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80aa610 con 0x7fb1d815a790 2024-03-29T08:21:13.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.523+0000 7fb1d7e00640 1 -- 172.21.15.46:0/2533301279 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c80a6040 con 0x7fb1d815a790 2024-03-29T08:21:13.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.523+0000 7fb1d7e00640 1 -- 172.21.15.46:0/2533301279 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80aa990 con 0x7fb1d815a790 2024-03-29T08:21:13.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.523+0000 7fb1df800640 1 -- 172.21.15.46:0/2533301279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1d815a790 msgr2=0x7fb1d80cbef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.523+0000 7fb1df800640 1 --2- 172.21.15.46:0/2533301279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1d815a790 0x7fb1d80cbef0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8071b80 tx=0x7fb1c801c4f0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.523+0000 7fb1df800640 1 -- 172.21.15.46:0/2533301279 shutdown_connections 2024-03-29T08:21:13.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.523+0000 7fb1df800640 1 --2- 172.21.15.46:0/2533301279 >> v2:172.21.15.73:3300/0 conn(0x7fb1d80cc430 0x7fb1d8098f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.523+0000 7fb1df800640 1 --2- 172.21.15.46:0/2533301279 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1d815ab70 0x7fb1d8098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.523+0000 7fb1df800640 1 --2- 172.21.15.46:0/2533301279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1d815a790 0x7fb1d80cbef0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.523+0000 7fb1df800640 1 -- 172.21.15.46:0/2533301279 >> 172.21.15.46:0/2533301279 conn(0x7fb1d80b21a0 msgr2=0x7fb1d80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:13.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.523+0000 7fb1df800640 1 -- 172.21.15.46:0/2533301279 shutdown_connections 2024-03-29T08:21:13.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.523+0000 7fb1df800640 1 -- 172.21.15.46:0/2533301279 wait complete. 2024-03-29T08:21:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1df800640 1 Processor -- start 2024-03-29T08:21:13.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1df800640 1 -- start start 2024-03-29T08:21:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1df800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1d80cc430 0x7fb1d814cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1df800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1d815a790 0x7fb1d806aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1df800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1d815ab70 0x7fb1d80cc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1df800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d815c030 con 0x7fb1d80cc430 2024-03-29T08:21:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1df800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d806ce90 con 0x7fb1d815a790 2024-03-29T08:21:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1df800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d815d720 con 0x7fb1d815ab70 2024-03-29T08:21:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1dce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1d80cc430 0x7fb1d814cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1dce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1d80cc430 0x7fb1d814cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54362/0 (socket says 172.21.15.46:54362) 2024-03-29T08:21:13.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1dce00640 1 -- 172.21.15.46:0/2294845913 learned_addr learned my addr 172.21.15.46:0/2294845913 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1cfe00640 1 --2- 172.21.15.46:0/2294845913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1d815a790 0x7fb1d806aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1dd800640 1 --2- 172.21.15.46:0/2294845913 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1d815ab70 0x7fb1d80cc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1dce00640 1 -- 172.21.15.46:0/2294845913 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1d815ab70 msgr2=0x7fb1d80cc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1dce00640 1 --2- 172.21.15.46:0/2294845913 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1d815ab70 0x7fb1d80cc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1dce00640 1 -- 172.21.15.46:0/2294845913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1d815a790 msgr2=0x7fb1d806aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1dce00640 1 --2- 172.21.15.46:0/2294845913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1d815a790 0x7fb1d806aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1dce00640 1 -- 172.21.15.46:0/2294845913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c80094d0 con 0x7fb1d80cc430 2024-03-29T08:21:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1dd800640 1 --2- 172.21.15.46:0/2294845913 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1d815ab70 0x7fb1d80cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:13.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1dce00640 1 --2- 172.21.15.46:0/2294845913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1d80cc430 0x7fb1d814cb00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb1c801aba0 tx=0x7fb1c801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:13.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1d5600640 1 -- 172.21.15.46:0/2294845913 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80bd5f0 con 0x7fb1d80cc430 2024-03-29T08:21:13.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1d5600640 1 -- 172.21.15.46:0/2294845913 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c80bd750 con 0x7fb1d80cc430 2024-03-29T08:21:13.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1df800640 1 -- 172.21.15.46:0/2294845913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d815da00 con 0x7fb1d80cc430 2024-03-29T08:21:13.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1d5600640 1 -- 172.21.15.46:0/2294845913 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80bdb00 con 0x7fb1d80cc430 2024-03-29T08:21:13.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.527+0000 7fb1df800640 1 -- 172.21.15.46:0/2294845913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d814d040 con 0x7fb1d80cc430 2024-03-29T08:21:13.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.531+0000 7fb1df800640 1 -- 172.21.15.46:0/2294845913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d8066c90 con 0x7fb1d80cc430 2024-03-29T08:21:13.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.531+0000 7fb1d5600640 1 -- 172.21.15.46:0/2294845913 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb1c80b33f0 con 0x7fb1d80cc430 2024-03-29T08:21:13.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.531+0000 7fb1d5600640 1 --2- 172.21.15.46:0/2294845913 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb1b4066690 0x7fb1b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.531+0000 7fb1d5600640 1 -- 172.21.15.46:0/2294845913 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1c811e640 con 0x7fb1d80cc430 2024-03-29T08:21:13.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.531+0000 7fb1cfe00640 1 --2- 172.21.15.46:0/2294845913 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb1b4066690 0x7fb1b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.531+0000 7fb1cfe00640 1 --2- 172.21.15.46:0/2294845913 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb1b4066690 0x7fb1b4068b50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8159e90 tx=0x7fb1c0009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:13.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.535+0000 7fb1d5600640 1 -- 172.21.15.46:0/2294845913 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb1c80eba50 con 0x7fb1d80cc430 2024-03-29T08:21:13.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.731+0000 7fb1df800640 1 -- 172.21.15.46:0/2294845913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1d80b2ec0 con 0x7fb1d80cc430 2024-03-29T08:21:13.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.731+0000 7fb1d5600640 1 -- 172.21.15.46:0/2294845913 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4728 (secure 0 0 0) 0x7fb1c80ef070 con 0x7fb1d80cc430 2024-03-29T08:21:13.735 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:13.735 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2187826146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2187826146},{"type":"v1","addr":"172.21.15.73:6836","nonce":2187826146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/73382477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":73382477},{"type":"v1","addr":"172.21.15.73:6837","nonce":73382477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2874958170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2874958170},{"type":"v1","addr":"172.21.15.46:6834","nonce":2874958170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:21:10.337267+0000","modified":"2024-03-29T08:21:13.349387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6051},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6051":{"gid":6051,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3255474176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3255474176},{"type":"v1","addr":"172.21.15.46:6835","nonce":3255474176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-29T08:21:13.735 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 32 2024-03-29T08:21:13.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.735+0000 7fb1df800640 1 -- 172.21.15.46:0/2294845913 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb1b4066690 msgr2=0x7fb1b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.735+0000 7fb1df800640 1 --2- 172.21.15.46:0/2294845913 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb1b4066690 0x7fb1b4068b50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8159e90 tx=0x7fb1c0009190 comp rx=0 tx=0).stop 2024-03-29T08:21:13.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.735+0000 7fb1df800640 1 -- 172.21.15.46:0/2294845913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1d80cc430 msgr2=0x7fb1d814cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.735+0000 7fb1df800640 1 --2- 172.21.15.46:0/2294845913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1d80cc430 0x7fb1d814cb00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb1c801aba0 tx=0x7fb1c801ad30 comp rx=0 tx=0).stop 2024-03-29T08:21:13.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.735+0000 7fb1df800640 1 -- 172.21.15.46:0/2294845913 shutdown_connections 2024-03-29T08:21:13.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.735+0000 7fb1df800640 1 --2- 172.21.15.46:0/2294845913 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb1b4066690 0x7fb1b4068b50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.735+0000 7fb1df800640 1 --2- 172.21.15.46:0/2294845913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1d815a790 0x7fb1d806aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.735+0000 7fb1df800640 1 --2- 172.21.15.46:0/2294845913 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1d815ab70 0x7fb1d80cc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.735+0000 7fb1df800640 1 --2- 172.21.15.46:0/2294845913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1d80cc430 0x7fb1d814cb00 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.735+0000 7fb1df800640 1 -- 172.21.15.46:0/2294845913 >> 172.21.15.46:0/2294845913 conn(0x7fb1d80b21a0 msgr2=0x7fb1d81516b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:13.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.735+0000 7fb1df800640 1 -- 172.21.15.46:0/2294845913 shutdown_connections 2024-03-29T08:21:13.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.735+0000 7fb1df800640 1 -- 172.21.15.46:0/2294845913 wait complete. 2024-03-29T08:21:13.756 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:21:13.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.891+0000 7f3dd9800640 1 Processor -- start 2024-03-29T08:21:13.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.891+0000 7f3dd9800640 1 -- start start 2024-03-29T08:21:13.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.895+0000 7f3dd9800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3dd4144210 0x7f3dd40cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.895+0000 7f3dd9800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3dd415ada0 0x7f3dd40ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.895+0000 7f3dd9800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd40cb510 con 0x7f3dd40ce950 2024-03-29T08:21:13.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.895+0000 7f3dd9800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd40cede0 con 0x7f3dd415ada0 2024-03-29T08:21:13.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.895+0000 7f3dd9800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd406d960 con 0x7f3dd4144210 2024-03-29T08:21:13.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.895+0000 7f3dcbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3dd415ada0 0x7f3dd40ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.895+0000 7f3dcbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3dd415ada0 0x7f3dd40ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54366/0 (socket says 172.21.15.46:54366) 2024-03-29T08:21:13.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.895+0000 7f3dd3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3dd40ce950 0x7f3dd415e360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42124/0 (socket says 172.21.15.46:42124) 2024-03-29T08:21:13.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.895+0000 7f3dcbe00640 1 -- 172.21.15.46:0/2417726968 learned_addr learned my addr 172.21.15.46:0/2417726968 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:13.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.895+0000 7f3dd2a00640 1 --2- 172.21.15.46:0/2417726968 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3dd4144210 0x7f3dd40cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.895+0000 7f3dcbe00640 1 -- 172.21.15.46:0/2417726968 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3dd4144210 msgr2=0x7f3dd40cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:13.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.895+0000 7f3dcbe00640 1 --2- 172.21.15.46:0/2417726968 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3dd4144210 0x7f3dd40cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.895+0000 7f3dcbe00640 1 -- 172.21.15.46:0/2417726968 >> v1:172.21.15.46:6789/0 conn(0x7f3dd40ce950 legacy=0x7f3dd415e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.895+0000 7f3dcbe00640 1 -- 172.21.15.46:0/2417726968 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd4070580 con 0x7f3dd415ada0 2024-03-29T08:21:13.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.895+0000 7f3dd2000640 1 -- 172.21.15.46:0/2417726968 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378345644 0 0) 0x7f3dd40cb510 con 0x7f3dd40ce950 2024-03-29T08:21:13.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.899+0000 7f3dcbe00640 1 --2- 172.21.15.46:0/2417726968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dd415ada0 0x7f3dd40ce410 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009870 tx=0x7f3dbc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20f4db5cd34630a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:13.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.899+0000 7f3dd2000640 1 -- 172.21.15.46:0/2417726968 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc0ab600 con 0x7f3dd415ada0 2024-03-29T08:21:13.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.899+0000 7f3dd2000640 1 -- 172.21.15.46:0/2417726968 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc0a7050 con 0x7f3dd415ada0 2024-03-29T08:21:13.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.899+0000 7f3dd2000640 1 -- 172.21.15.46:0/2417726968 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc0ab980 con 0x7f3dd415ada0 2024-03-29T08:21:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.899+0000 7f3dd9800640 1 -- 172.21.15.46:0/2417726968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dd415ada0 msgr2=0x7f3dd40ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.899+0000 7f3dd9800640 1 --2- 172.21.15.46:0/2417726968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dd415ada0 0x7f3dd40ce410 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc009870 tx=0x7f3dbc01c560 comp rx=0 tx=0).stop 2024-03-29T08:21:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.899+0000 7f3dd9800640 1 -- 172.21.15.46:0/2417726968 shutdown_connections 2024-03-29T08:21:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.899+0000 7f3dd9800640 1 --2- 172.21.15.46:0/2417726968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dd415ada0 0x7f3dd40ce410 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.899+0000 7f3dd9800640 1 --2- 172.21.15.46:0/2417726968 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3dd4144210 0x7f3dd40cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.899+0000 7f3dd9800640 1 -- 172.21.15.46:0/2417726968 >> 172.21.15.46:0/2417726968 conn(0x7f3dd4066110 msgr2=0x7f3dd4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:13.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.899+0000 7f3dd9800640 1 -- 172.21.15.46:0/2417726968 shutdown_connections 2024-03-29T08:21:13.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.899+0000 7f3dd9800640 1 -- 172.21.15.46:0/2417726968 wait complete. 2024-03-29T08:21:13.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.899+0000 7f3dd9800640 1 Processor -- start 2024-03-29T08:21:13.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.899+0000 7f3dd9800640 1 -- start start 2024-03-29T08:21:13.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dd9800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3dd40ce950 0x7f3dd40b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dd9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dd4144210 0x7f3dd40acf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dd9800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dd415ada0 0x7f3dd41641e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dd9800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd40b3290 con 0x7f3dd4144210 2024-03-29T08:21:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dd9800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd4164720 con 0x7f3dd415ada0 2024-03-29T08:21:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dd9800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd40b2610 con 0x7f3dd40ce950 2024-03-29T08:21:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dcbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dd4144210 0x7f3dd40acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dcbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dd4144210 0x7f3dd40acf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54378/0 (socket says 172.21.15.46:54378) 2024-03-29T08:21:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dcbe00640 1 -- 172.21.15.46:0/213714856 learned_addr learned my addr 172.21.15.46:0/213714856 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dd2a00640 1 --2- 172.21.15.46:0/213714856 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3dd40ce950 0x7f3dd40b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dd3400640 1 --2- 172.21.15.46:0/213714856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dd415ada0 0x7f3dd41641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dcbe00640 1 -- 172.21.15.46:0/213714856 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3dd40ce950 msgr2=0x7f3dd40b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dcbe00640 1 --2- 172.21.15.46:0/213714856 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3dd40ce950 0x7f3dd40b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dcbe00640 1 -- 172.21.15.46:0/213714856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dd415ada0 msgr2=0x7f3dd41641e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:13.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dcbe00640 1 --2- 172.21.15.46:0/213714856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dd415ada0 0x7f3dd41641e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:13.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dcbe00640 1 -- 172.21.15.46:0/213714856 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc0094d0 con 0x7f3dd4144210 2024-03-29T08:21:13.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dcbe00640 1 --2- 172.21.15.46:0/213714856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dd4144210 0x7f3dd40acf90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc01ac10 tx=0x7f3dbc019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:13.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dcaa00640 1 -- 172.21.15.46:0/213714856 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc004020 con 0x7f3dd4144210 2024-03-29T08:21:13.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dcaa00640 1 -- 172.21.15.46:0/213714856 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc004180 con 0x7f3dd4144210 2024-03-29T08:21:13.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dcaa00640 1 -- 172.21.15.46:0/213714856 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc0044f0 con 0x7f3dd4144210 2024-03-29T08:21:13.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dd9800640 1 -- 172.21.15.46:0/213714856 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd40b28f0 con 0x7f3dd4144210 2024-03-29T08:21:13.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dd9800640 1 -- 172.21.15.46:0/213714856 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd40b2eb0 con 0x7f3dd4144210 2024-03-29T08:21:13.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dd9800640 1 -- 172.21.15.46:0/213714856 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd40ce280 con 0x7f3dd4144210 2024-03-29T08:21:13.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.903+0000 7f3dcaa00640 1 -- 172.21.15.46:0/213714856 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3dbc003620 con 0x7f3dd4144210 2024-03-29T08:21:13.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.907+0000 7f3dcaa00640 1 --2- 172.21.15.46:0/213714856 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3db0066690 0x7f3db0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:13.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.907+0000 7f3dcaa00640 1 -- 172.21.15.46:0/213714856 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3dbc11e3d0 con 0x7f3dd4144210 2024-03-29T08:21:13.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.907+0000 7f3dd2a00640 1 --2- 172.21.15.46:0/213714856 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3db0066690 0x7f3db0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:13.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.907+0000 7f3dd2a00640 1 --2- 172.21.15.46:0/213714856 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3db0066690 0x7f3db0068b50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009b60 tx=0x7f3dc4009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:13.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:13.911+0000 7f3dcaa00640 1 -- 172.21.15.46:0/213714856 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3dbc0eb730 con 0x7f3dd4144210 2024-03-29T08:21:14.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.071+0000 7f3dd9800640 1 -- 172.21.15.46:0/213714856 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3dd4066c90 con 0x7f3dd4144210 2024-03-29T08:21:14.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.071+0000 7f3dcaa00640 1 -- 172.21.15.46:0/213714856 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16049 (secure 0 0 0) 0x7f3dbc0eed50 con 0x7f3dd4144210 2024-03-29T08:21:14.073 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:14.074 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":63,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:21:11.339712+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:21:08.061567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T08:21:09.336464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:21:14.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.075+0000 7f3dd9800640 1 -- 172.21.15.46:0/213714856 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3db0066690 msgr2=0x7f3db0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.075+0000 7f3dd9800640 1 --2- 172.21.15.46:0/213714856 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3db0066690 0x7f3db0068b50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009b60 tx=0x7f3dc4009190 comp rx=0 tx=0).stop 2024-03-29T08:21:14.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.075+0000 7f3dd9800640 1 -- 172.21.15.46:0/213714856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dd4144210 msgr2=0x7f3dd40acf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.075+0000 7f3dd9800640 1 --2- 172.21.15.46:0/213714856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dd4144210 0x7f3dd40acf90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc01ac10 tx=0x7f3dbc019300 comp rx=0 tx=0).stop 2024-03-29T08:21:14.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.075+0000 7f3dd9800640 1 -- 172.21.15.46:0/213714856 shutdown_connections 2024-03-29T08:21:14.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.075+0000 7f3dd9800640 1 --2- 172.21.15.46:0/213714856 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3db0066690 0x7f3db0068b50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.075+0000 7f3dd9800640 1 --2- 172.21.15.46:0/213714856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dd415ada0 0x7f3dd41641e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.075+0000 7f3dd9800640 1 --2- 172.21.15.46:0/213714856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3dd4144210 0x7f3dd40acf90 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.075+0000 7f3dd9800640 1 --2- 172.21.15.46:0/213714856 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3dd40ce950 0x7f3dd40b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.075+0000 7f3dd9800640 1 -- 172.21.15.46:0/213714856 >> 172.21.15.46:0/213714856 conn(0x7f3dd4066110 msgr2=0x7f3dd4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:14.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.075+0000 7f3dd9800640 1 -- 172.21.15.46:0/213714856 shutdown_connections 2024-03-29T08:21:14.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.075+0000 7f3dd9800640 1 -- 172.21.15.46:0/213714856 wait complete. 2024-03-29T08:21:14.095 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T08:21:14.095 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:21:14.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.235+0000 7fd193200640 1 Processor -- start 2024-03-29T08:21:14.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.235+0000 7fd193200640 1 -- start start 2024-03-29T08:21:14.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.235+0000 7fd193200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd18c15a940 0x7fd18c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:14.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.235+0000 7fd193200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18c0b00d0 con 0x7fd18c0cf3c0 2024-03-29T08:21:14.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.235+0000 7fd193200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18c0b0ab0 con 0x7fd18c15ad20 2024-03-29T08:21:14.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.235+0000 7fd193200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c0b0d90 con 0x7fd18c15a940 2024-03-29T08:21:14.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.235+0000 7fd191200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd18c0cf3c0 0x7fd18c0cb630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42138/0 (socket says 172.21.15.46:42138) 2024-03-29T08:21:14.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.235+0000 7fd191200640 1 -- 172.21.15.46:0/3666209752 learned_addr learned my addr 172.21.15.46:0/3666209752 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:14.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.235+0000 7fd18be00640 1 --2- 172.21.15.46:0/3666209752 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd18c15a940 0x7fd18c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:14.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.235+0000 7fd18b400640 1 -- 172.21.15.46:0/3666209752 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594631728 0 0) 0x7fd18c0b00d0 con 0x7fd18c0cf3c0 2024-03-29T08:21:14.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.235+0000 7fd18b400640 1 -- 172.21.15.46:0/3666209752 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd170003530 con 0x7fd18c0cf3c0 2024-03-29T08:21:14.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.235+0000 7fd18be00640 1 -- 172.21.15.46:0/3666209752 >> v1:172.21.15.73:6789/0 conn(0x7fd18c15ad20 legacy=0x7fd18c0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.235+0000 7fd18b400640 1 -- 172.21.15.46:0/3666209752 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953452244 0 0) 0x7fd18c0b0ab0 con 0x7fd18c15ad20 2024-03-29T08:21:14.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.235+0000 7fd18be00640 1 -- 172.21.15.46:0/3666209752 >> v1:172.21.15.46:6789/0 conn(0x7fd18c0cf3c0 legacy=0x7fd18c0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.235+0000 7fd18be00640 1 -- 172.21.15.46:0/3666209752 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18c0b1070 con 0x7fd18c15a940 2024-03-29T08:21:14.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.239+0000 7fd18be00640 1 --2- 172.21.15.46:0/3666209752 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd18c15a940 0x7fd18c0afb90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009dc0 tx=0x7fd17c01c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9938f3906f44c9c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:14.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.239+0000 7fd18b400640 1 -- 172.21.15.46:0/3666209752 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0a5990 con 0x7fd18c15a940 2024-03-29T08:21:14.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.239+0000 7fd18b400640 1 -- 172.21.15.46:0/3666209752 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c0a5af0 con 0x7fd18c15a940 2024-03-29T08:21:14.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.239+0000 7fd18b400640 1 -- 172.21.15.46:0/3666209752 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c0a5e60 con 0x7fd18c15a940 2024-03-29T08:21:14.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.239+0000 7fd193200640 1 -- 172.21.15.46:0/3666209752 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd18c15a940 msgr2=0x7fd18c0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.239+0000 7fd193200640 1 --2- 172.21.15.46:0/3666209752 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd18c15a940 0x7fd18c0afb90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009dc0 tx=0x7fd17c01c8d0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.239+0000 7fd193200640 1 -- 172.21.15.46:0/3666209752 shutdown_connections 2024-03-29T08:21:14.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.239+0000 7fd193200640 1 --2- 172.21.15.46:0/3666209752 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd18c15a940 0x7fd18c0afb90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.239+0000 7fd193200640 1 -- 172.21.15.46:0/3666209752 >> 172.21.15.46:0/3666209752 conn(0x7fd18c089fd0 msgr2=0x7fd18c08a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:14.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.239+0000 7fd193200640 1 -- 172.21.15.46:0/3666209752 shutdown_connections 2024-03-29T08:21:14.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.239+0000 7fd193200640 1 -- 172.21.15.46:0/3666209752 wait complete. 2024-03-29T08:21:14.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd193200640 1 Processor -- start 2024-03-29T08:21:14.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd193200640 1 -- start start 2024-03-29T08:21:14.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd193200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd18c0cf3c0 0x7fd18c0c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:14.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd193200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c15a940 0x7fd18c15c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:14.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd193200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd18c15ad20 0x7fd18c154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:14.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd193200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c15d9b0 con 0x7fd18c15a940 2024-03-29T08:21:14.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd193200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c0c4fd0 con 0x7fd18c0cf3c0 2024-03-29T08:21:14.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd193200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18c15d3f0 con 0x7fd18c15ad20 2024-03-29T08:21:14.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd18be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd18c0cf3c0 0x7fd18c0c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:14.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd191200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd18c15ad20 0x7fd18c154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd18be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd18c0cf3c0 0x7fd18c0c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:37386/0 (socket says 172.21.15.46:37386) 2024-03-29T08:21:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd191200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd18c15ad20 0x7fd18c154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:45608/0 (socket says 172.21.15.46:45608) 2024-03-29T08:21:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd18be00640 1 -- 172.21.15.46:0/3495495433 learned_addr learned my addr 172.21.15.46:0/3495495433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd183400640 1 --2- 172.21.15.46:0/3495495433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c15a940 0x7fd18c15c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd191200640 1 -- 172.21.15.46:0/3495495433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd18c0cf3c0 msgr2=0x7fd18c0c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd191200640 1 --2- 172.21.15.46:0/3495495433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd18c0cf3c0 0x7fd18c0c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd191200640 1 -- 172.21.15.46:0/3495495433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c15a940 msgr2=0x7fd18c15c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd191200640 1 --2- 172.21.15.46:0/3495495433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c15a940 0x7fd18c15c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd191200640 1 -- 172.21.15.46:0/3495495433 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c009a70 con 0x7fd18c15ad20 2024-03-29T08:21:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd183400640 1 --2- 172.21.15.46:0/3495495433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c15a940 0x7fd18c15c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:21:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd191200640 1 --2- 172.21.15.46:0/3495495433 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd18c15ad20 0x7fd18c154330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c3c0 tx=0x7fd18400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd188c00640 1 -- 172.21.15.46:0/3495495433 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184005cf0 con 0x7fd18c15ad20 2024-03-29T08:21:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd188c00640 1 -- 172.21.15.46:0/3495495433 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd184005e50 con 0x7fd18c15ad20 2024-03-29T08:21:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd193200640 1 -- 172.21.15.46:0/3495495433 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18c15d6d0 con 0x7fd18c15ad20 2024-03-29T08:21:14.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd188c00640 1 -- 172.21.15.46:0/3495495433 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1840072d0 con 0x7fd18c15ad20 2024-03-29T08:21:14.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.243+0000 7fd193200640 1 -- 172.21.15.46:0/3495495433 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18c15bfb0 con 0x7fd18c15ad20 2024-03-29T08:21:14.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.247+0000 7fd193200640 1 -- 172.21.15.46:0/3495495433 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18c15bb30 con 0x7fd18c15ad20 2024-03-29T08:21:14.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.247+0000 7fd188c00640 1 -- 172.21.15.46:0/3495495433 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd184007430 con 0x7fd18c15ad20 2024-03-29T08:21:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.247+0000 7fd188c00640 1 --2- 172.21.15.46:0/3495495433 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd168066690 0x7fd168068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.247+0000 7fd188c00640 1 -- 172.21.15.46:0/3495495433 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1840836f0 con 0x7fd18c15ad20 2024-03-29T08:21:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.247+0000 7fd18be00640 1 --2- 172.21.15.46:0/3495495433 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd168066690 0x7fd168068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:14.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.247+0000 7fd18be00640 1 --2- 172.21.15.46:0/3495495433 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd168066690 0x7fd168068b50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd17c01c910 tx=0x7fd17c0a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:14.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.251+0000 7fd188c00640 1 -- 172.21.15.46:0/3495495433 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd18c15bb30 con 0x7fd18c15ad20 2024-03-29T08:21:14.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.411+0000 7fd193200640 1 -- 172.21.15.46:0/3495495433 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd18c168680 con 0x7fd168066690 2024-03-29T08:21:14.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.423+0000 7fd188c00640 1 -- 172.21.15.46:0/3495495433 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284914 (secure 0 0 0) 0x7fd17c0bd070 con 0x7fd168066690 2024-03-29T08:21:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:14.428 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:21:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.423+0000 7fd193200640 1 -- 172.21.15.46:0/3495495433 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd168066690 msgr2=0x7fd168068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.423+0000 7fd193200640 1 --2- 172.21.15.46:0/3495495433 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd168066690 0x7fd168068b50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd17c01c910 tx=0x7fd17c0a7070 comp rx=0 tx=0).stop 2024-03-29T08:21:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.423+0000 7fd193200640 1 -- 172.21.15.46:0/3495495433 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd18c15ad20 msgr2=0x7fd18c154330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.423+0000 7fd193200640 1 --2- 172.21.15.46:0/3495495433 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd18c15ad20 0x7fd18c154330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd18400c3c0 tx=0x7fd18400c880 comp rx=0 tx=0).stop 2024-03-29T08:21:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.423+0000 7fd193200640 1 -- 172.21.15.46:0/3495495433 shutdown_connections 2024-03-29T08:21:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.423+0000 7fd193200640 1 --2- 172.21.15.46:0/3495495433 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd168066690 0x7fd168068b50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.423+0000 7fd193200640 1 --2- 172.21.15.46:0/3495495433 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd18c15ad20 0x7fd18c154330 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.423+0000 7fd193200640 1 --2- 172.21.15.46:0/3495495433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd18c15a940 0x7fd18c15c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.423+0000 7fd193200640 1 --2- 172.21.15.46:0/3495495433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd18c0cf3c0 0x7fd18c0c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.423+0000 7fd193200640 1 -- 172.21.15.46:0/3495495433 >> 172.21.15.46:0/3495495433 conn(0x7fd18c089fd0 msgr2=0x7fd18c159b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:14.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.423+0000 7fd193200640 1 -- 172.21.15.46:0/3495495433 shutdown_connections 2024-03-29T08:21:14.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.423+0000 7fd193200640 1 -- 172.21.15.46:0/3495495433 wait complete. 2024-03-29T08:21:14.446 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-03-29T08:21:14.084517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243396,"kb_used_data":6752,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754731324,"statfs":{"total":773094113280,"available":772844875776,"internally_reserved":0,"allocated":6914048,"data_stored":3542294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":242208313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.157425"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.941725+0000","last_change":"2024-03-29T08:21:09.941725+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.941725+0000","last_undegraded":"2024-03-29T08:21:09.941725+0000","last_fullsized":"2024-03-29T08:21:09.941725+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:16:15.767217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936961+0000","last_change":"2024-03-29T08:21:09.936961+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936961+0000","last_undegraded":"2024-03-29T08:21:09.936961+0000","last_fullsized":"2024-03-29T08:21:09.936961+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:05:22.465098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938613+0000","last_change":"2024-03-29T08:21:09.938613+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938613+0000","last_undegraded":"2024-03-29T08:21:09.938613+0000","last_fullsized":"2024-03-29T08:21:09.938613+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:47:48.297292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.933020+0000","last_change":"2024-03-29T08:21:09.933020+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.933020+0000","last_undegraded":"2024-03-29T08:21:09.933020+0000","last_fullsized":"2024-03-29T08:21:09.933020+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:51.414360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936757+0000","last_change":"2024-03-29T08:21:09.936757+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936757+0000","last_undegraded":"2024-03-29T08:21:09.936757+0000","last_fullsized":"2024-03-29T08:21:09.936757+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:36:29.239728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939144+0000","last_change":"2024-03-29T08:21:09.939144+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939144+0000","last_undegraded":"2024-03-29T08:21:09.939144+0000","last_fullsized":"2024-03-29T08:21:09.939144+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:35:20.420999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.933702+0000","last_change":"2024-03-29T08:21:09.933702+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.933702+0000","last_undegraded":"2024-03-29T08:21:09.933702+0000","last_fullsized":"2024-03-29T08:21:09.933702+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:30:03.234601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939490+0000","last_change":"2024-03-29T08:21:09.939490+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939490+0000","last_undegraded":"2024-03-29T08:21:09.939490+0000","last_fullsized":"2024-03-29T08:21:09.939490+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:40.843359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.940289+0000","last_change":"2024-03-29T08:21:09.940289+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.940289+0000","last_undegraded":"2024-03-29T08:21:09.940289+0000","last_fullsized":"2024-03-29T08:21:09.940289+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:54.034461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.934375+0000","last_change":"2024-03-29T08:21:09.934375+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.934375+0000","last_undegraded":"2024-03-29T08:21:09.934375+0000","last_fullsized":"2024-03-29T08:21:09.934375+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:58.105380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.940141+0000","last_change":"2024-03-29T08:21:09.940141+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.940141+0000","last_undegraded":"2024-03-29T08:21:09.940141+0000","last_fullsized":"2024-03-29T08:21:09.940141+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:20:51.330512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.942430+0000","last_change":"2024-03-29T08:21:09.942430+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.942430+0000","last_undegraded":"2024-03-29T08:21:09.942430+0000","last_fullsized":"2024-03-29T08:21:09.942430+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:35:20.420999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939640+0000","last_change":"2024-03-29T08:21:09.939640+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939640+0000","last_undegraded":"2024-03-29T08:21:09.939640+0000","last_fullsized":"2024-03-29T08:21:09.939640+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:49:07.458847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.940293+0000","last_change":"2024-03-29T08:21:09.940293+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.940293+0000","last_undegraded":"2024-03-29T08:21:09.940293+0000","last_fullsized":"2024-03-29T08:21:09.940293+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:39.458643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938066+0000","last_change":"2024-03-29T08:21:09.938066+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938066+0000","last_undegraded":"2024-03-29T08:21:09.938066+0000","last_fullsized":"2024-03-29T08:21:09.938066+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:01:52.376061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.941263+0000","last_change":"2024-03-29T08:21:09.941263+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.941263+0000","last_undegraded":"2024-03-29T08:21:09.941263+0000","last_fullsized":"2024-03-29T08:21:09.941263+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:39.458643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939231+0000","last_change":"2024-03-29T08:21:09.939231+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939231+0000","last_undegraded":"2024-03-29T08:21:09.939231+0000","last_fullsized":"2024-03-29T08:21:09.939231+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:33:19.998323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.942709+0000","last_change":"2024-03-29T08:21:09.942709+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.942709+0000","last_undegraded":"2024-03-29T08:21:09.942709+0000","last_fullsized":"2024-03-29T08:21:09.942709+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:58:48.804311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.934214+0000","last_change":"2024-03-29T08:21:09.934214+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.934214+0000","last_undegraded":"2024-03-29T08:21:09.934214+0000","last_fullsized":"2024-03-29T08:21:09.934214+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:01.741332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.942033+0000","last_change":"2024-03-29T08:21:09.942033+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.942033+0000","last_undegraded":"2024-03-29T08:21:09.942033+0000","last_fullsized":"2024-03-29T08:21:09.942033+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:44.680234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.934012+0000","last_change":"2024-03-29T08:21:09.934012+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.934012+0000","last_undegraded":"2024-03-29T08:21:09.934012+0000","last_fullsized":"2024-03-29T08:21:09.934012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:15.253098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.933212+0000","last_change":"2024-03-29T08:21:09.933212+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.933212+0000","last_undegraded":"2024-03-29T08:21:09.933212+0000","last_fullsized":"2024-03-29T08:21:09.933212+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:25:00.595704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938923+0000","last_change":"2024-03-29T08:21:09.938923+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938923+0000","last_undegraded":"2024-03-29T08:21:09.938923+0000","last_fullsized":"2024-03-29T08:21:09.938923+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:58.831468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937337+0000","last_change":"2024-03-29T08:21:09.937337+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937337+0000","last_undegraded":"2024-03-29T08:21:09.937337+0000","last_fullsized":"2024-03-29T08:21:09.937337+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:59:24.384598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.943515+0000","last_change":"2024-03-29T08:21:09.943515+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.943515+0000","last_undegraded":"2024-03-29T08:21:09.943515+0000","last_fullsized":"2024-03-29T08:21:09.943515+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:53.450623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936629+0000","last_change":"2024-03-29T08:21:09.936629+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936629+0000","last_undegraded":"2024-03-29T08:21:09.936629+0000","last_fullsized":"2024-03-29T08:21:09.936629+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:49.605189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.932435+0000","last_change":"2024-03-29T08:21:09.932435+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.932435+0000","last_undegraded":"2024-03-29T08:21:09.932435+0000","last_fullsized":"2024-03-29T08:21:09.932435+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:45.698306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.943149+0000","last_change":"2024-03-29T08:21:09.943149+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.943149+0000","last_undegraded":"2024-03-29T08:21:09.943149+0000","last_fullsized":"2024-03-29T08:21:09.943149+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:01:19.390653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.934024+0000","last_change":"2024-03-29T08:21:09.934024+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.934024+0000","last_undegraded":"2024-03-29T08:21:09.934024+0000","last_fullsized":"2024-03-29T08:21:09.934024+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:54.034461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939827+0000","last_change":"2024-03-29T08:21:09.939827+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939827+0000","last_undegraded":"2024-03-29T08:21:09.939827+0000","last_fullsized":"2024-03-29T08:21:09.939827+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:53.450623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956239+0000","last_change":"2024-03-29T08:21:09.953242+0000","last_active":"2024-03-29T08:21:09.956239+0000","last_peered":"2024-03-29T08:21:09.956239+0000","last_clean":"2024-03-29T08:21:09.956239+0000","last_became_active":"2024-03-29T08:21:09.952978+0000","last_became_peered":"2024-03-29T08:21:09.952978+0000","last_unstale":"2024-03-29T08:21:09.956239+0000","last_undegraded":"2024-03-29T08:21:09.956239+0000","last_fullsized":"2024-03-29T08:21:09.956239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:43.677354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937754+0000","last_change":"2024-03-29T08:21:09.937754+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937754+0000","last_undegraded":"2024-03-29T08:21:09.937754+0000","last_fullsized":"2024-03-29T08:21:09.937754+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:40.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.957328+0000","last_change":"2024-03-29T08:21:09.955717+0000","last_active":"2024-03-29T08:21:09.957328+0000","last_peered":"2024-03-29T08:21:09.957328+0000","last_clean":"2024-03-29T08:21:09.957328+0000","last_became_active":"2024-03-29T08:21:09.955450+0000","last_became_peered":"2024-03-29T08:21:09.955450+0000","last_unstale":"2024-03-29T08:21:09.957328+0000","last_undegraded":"2024-03-29T08:21:09.957328+0000","last_fullsized":"2024-03-29T08:21:09.957328+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:47:40.877803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.952711+0000","last_change":"2024-03-29T08:21:09.950482+0000","last_active":"2024-03-29T08:21:09.952711+0000","last_peered":"2024-03-29T08:21:09.952711+0000","last_clean":"2024-03-29T08:21:09.952711+0000","last_became_active":"2024-03-29T08:21:09.950190+0000","last_became_peered":"2024-03-29T08:21:09.950190+0000","last_unstale":"2024-03-29T08:21:09.952711+0000","last_undegraded":"2024-03-29T08:21:09.952711+0000","last_fullsized":"2024-03-29T08:21:09.952711+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:07:20.233898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938835+0000","last_change":"2024-03-29T08:21:09.938835+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938835+0000","last_undegraded":"2024-03-29T08:21:09.938835+0000","last_fullsized":"2024-03-29T08:21:09.938835+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:21:23.293798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.944401+0000","last_change":"2024-03-29T08:21:09.942997+0000","last_active":"2024-03-29T08:21:09.944401+0000","last_peered":"2024-03-29T08:21:09.944401+0000","last_clean":"2024-03-29T08:21:09.944401+0000","last_became_active":"2024-03-29T08:21:09.942730+0000","last_became_peered":"2024-03-29T08:21:09.942730+0000","last_unstale":"2024-03-29T08:21:09.944401+0000","last_undegraded":"2024-03-29T08:21:09.944401+0000","last_fullsized":"2024-03-29T08:21:09.944401+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:42:48.592612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.947598+0000","last_change":"2024-03-29T08:21:09.944500+0000","last_active":"2024-03-29T08:21:09.947598+0000","last_peered":"2024-03-29T08:21:09.947598+0000","last_clean":"2024-03-29T08:21:09.947598+0000","last_became_active":"2024-03-29T08:21:09.944236+0000","last_became_peered":"2024-03-29T08:21:09.944236+0000","last_unstale":"2024-03-29T08:21:09.947598+0000","last_undegraded":"2024-03-29T08:21:09.947598+0000","last_fullsized":"2024-03-29T08:21:09.947598+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:06:08.420341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.935582+0000","last_change":"2024-03-29T08:21:09.935582+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.935582+0000","last_undegraded":"2024-03-29T08:21:09.935582+0000","last_fullsized":"2024-03-29T08:21:09.935582+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:05:09.416516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.952308+0000","last_change":"2024-03-29T08:21:09.946818+0000","last_active":"2024-03-29T08:21:09.952308+0000","last_peered":"2024-03-29T08:21:09.952308+0000","last_clean":"2024-03-29T08:21:09.952308+0000","last_became_active":"2024-03-29T08:21:09.946555+0000","last_became_peered":"2024-03-29T08:21:09.946555+0000","last_unstale":"2024-03-29T08:21:09.952308+0000","last_undegraded":"2024-03-29T08:21:09.952308+0000","last_fullsized":"2024-03-29T08:21:09.952308+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:27:46.452557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937385+0000","last_change":"2024-03-29T08:21:09.937385+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937385+0000","last_undegraded":"2024-03-29T08:21:09.937385+0000","last_fullsized":"2024-03-29T08:21:09.937385+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:58.105380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956350+0000","last_change":"2024-03-29T08:21:09.954162+0000","last_active":"2024-03-29T08:21:09.956350+0000","last_peered":"2024-03-29T08:21:09.956350+0000","last_clean":"2024-03-29T08:21:09.956350+0000","last_became_active":"2024-03-29T08:21:09.953849+0000","last_became_peered":"2024-03-29T08:21:09.953849+0000","last_unstale":"2024-03-29T08:21:09.956350+0000","last_undegraded":"2024-03-29T08:21:09.956350+0000","last_fullsized":"2024-03-29T08:21:09.956350+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:52.803477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937620+0000","last_change":"2024-03-29T08:21:09.937620+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937620+0000","last_undegraded":"2024-03-29T08:21:09.937620+0000","last_fullsized":"2024-03-29T08:21:09.937620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:19:16.700510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956277+0000","last_change":"2024-03-29T08:21:09.954120+0000","last_active":"2024-03-29T08:21:09.956277+0000","last_peered":"2024-03-29T08:21:09.956277+0000","last_clean":"2024-03-29T08:21:09.956277+0000","last_became_active":"2024-03-29T08:21:09.953375+0000","last_became_peered":"2024-03-29T08:21:09.953375+0000","last_unstale":"2024-03-29T08:21:09.956277+0000","last_undegraded":"2024-03-29T08:21:09.956277+0000","last_fullsized":"2024-03-29T08:21:09.956277+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:48.602309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.943037+0000","last_change":"2024-03-29T08:21:09.943037+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.943037+0000","last_undegraded":"2024-03-29T08:21:09.943037+0000","last_fullsized":"2024-03-29T08:21:09.943037+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:24.166586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.953570+0000","last_change":"2024-03-29T08:21:09.951304+0000","last_active":"2024-03-29T08:21:09.953570+0000","last_peered":"2024-03-29T08:21:09.953570+0000","last_clean":"2024-03-29T08:21:09.953570+0000","last_became_active":"2024-03-29T08:21:09.951067+0000","last_became_peered":"2024-03-29T08:21:09.951067+0000","last_unstale":"2024-03-29T08:21:09.953570+0000","last_undegraded":"2024-03-29T08:21:09.953570+0000","last_fullsized":"2024-03-29T08:21:09.953570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:13:08.676082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938641+0000","last_change":"2024-03-29T08:21:09.938641+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938641+0000","last_undegraded":"2024-03-29T08:21:09.938641+0000","last_fullsized":"2024-03-29T08:21:09.938641+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:08:39.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.939701+0000","last_change":"2024-03-29T08:21:08.939701+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.939701+0000","last_undegraded":"2024-03-29T08:21:08.939701+0000","last_fullsized":"2024-03-29T08:21:08.939701+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:08:44.015717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.945733+0000","last_change":"2024-03-29T08:21:09.944508+0000","last_active":"2024-03-29T08:21:09.945733+0000","last_peered":"2024-03-29T08:21:09.945733+0000","last_clean":"2024-03-29T08:21:09.945733+0000","last_became_active":"2024-03-29T08:21:09.944191+0000","last_became_peered":"2024-03-29T08:21:09.944191+0000","last_unstale":"2024-03-29T08:21:09.945733+0000","last_undegraded":"2024-03-29T08:21:09.945733+0000","last_fullsized":"2024-03-29T08:21:09.945733+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:25:36.003763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.932991+0000","last_change":"2024-03-29T08:21:09.932991+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.932991+0000","last_undegraded":"2024-03-29T08:21:09.932991+0000","last_fullsized":"2024-03-29T08:21:09.932991+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:56:39.434467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.955648+0000","last_change":"2024-03-29T08:21:09.953728+0000","last_active":"2024-03-29T08:21:09.955648+0000","last_peered":"2024-03-29T08:21:09.955648+0000","last_clean":"2024-03-29T08:21:09.955648+0000","last_became_active":"2024-03-29T08:21:09.953431+0000","last_became_peered":"2024-03-29T08:21:09.953431+0000","last_unstale":"2024-03-29T08:21:09.955648+0000","last_undegraded":"2024-03-29T08:21:09.955648+0000","last_fullsized":"2024-03-29T08:21:09.955648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:38:17.880237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939449+0000","last_change":"2024-03-29T08:21:09.939449+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939449+0000","last_undegraded":"2024-03-29T08:21:09.939449+0000","last_fullsized":"2024-03-29T08:21:09.939449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:52:34.720375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.934101+0000","last_change":"2024-03-29T08:21:08.934101+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.934101+0000","last_undegraded":"2024-03-29T08:21:08.934101+0000","last_fullsized":"2024-03-29T08:21:08.934101+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:03:13.729828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937954+0000","last_change":"2024-03-29T08:21:09.937954+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937954+0000","last_undegraded":"2024-03-29T08:21:09.937954+0000","last_fullsized":"2024-03-29T08:21:09.937954+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:21:28.741076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.950524+0000","last_change":"2024-03-29T08:21:09.948902+0000","last_active":"2024-03-29T08:21:09.950524+0000","last_peered":"2024-03-29T08:21:09.950524+0000","last_clean":"2024-03-29T08:21:09.950524+0000","last_became_active":"2024-03-29T08:21:09.948613+0000","last_became_peered":"2024-03-29T08:21:09.948613+0000","last_unstale":"2024-03-29T08:21:09.950524+0000","last_undegraded":"2024-03-29T08:21:09.950524+0000","last_fullsized":"2024-03-29T08:21:09.950524+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:00.619680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938522+0000","last_change":"2024-03-29T08:21:09.938522+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938522+0000","last_undegraded":"2024-03-29T08:21:09.938522+0000","last_fullsized":"2024-03-29T08:21:09.938522+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:01:48.373155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.947564+0000","last_change":"2024-03-29T08:21:09.946582+0000","last_active":"2024-03-29T08:21:09.947564+0000","last_peered":"2024-03-29T08:21:09.947564+0000","last_clean":"2024-03-29T08:21:09.947564+0000","last_became_active":"2024-03-29T08:21:09.946276+0000","last_became_peered":"2024-03-29T08:21:09.946276+0000","last_unstale":"2024-03-29T08:21:09.947564+0000","last_undegraded":"2024-03-29T08:21:09.947564+0000","last_fullsized":"2024-03-29T08:21:09.947564+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:01:32.117594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.932610+0000","last_change":"2024-03-29T08:21:09.932610+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.932610+0000","last_undegraded":"2024-03-29T08:21:09.932610+0000","last_fullsized":"2024-03-29T08:21:09.932610+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:11:50.905254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.955654+0000","last_change":"2024-03-29T08:21:09.953517+0000","last_active":"2024-03-29T08:21:09.955654+0000","last_peered":"2024-03-29T08:21:09.955654+0000","last_clean":"2024-03-29T08:21:09.955654+0000","last_became_active":"2024-03-29T08:21:09.953205+0000","last_became_peered":"2024-03-29T08:21:09.953205+0000","last_unstale":"2024-03-29T08:21:09.955654+0000","last_undegraded":"2024-03-29T08:21:09.955654+0000","last_fullsized":"2024-03-29T08:21:09.955654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:11:14.860644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936862+0000","last_change":"2024-03-29T08:21:09.936862+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936862+0000","last_undegraded":"2024-03-29T08:21:09.936862+0000","last_fullsized":"2024-03-29T08:21:09.936862+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:15.253098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.957279+0000","last_change":"2024-03-29T08:21:09.954705+0000","last_active":"2024-03-29T08:21:09.957279+0000","last_peered":"2024-03-29T08:21:09.957279+0000","last_clean":"2024-03-29T08:21:09.957279+0000","last_became_active":"2024-03-29T08:21:09.954314+0000","last_became_peered":"2024-03-29T08:21:09.954314+0000","last_unstale":"2024-03-29T08:21:09.957279+0000","last_undegraded":"2024-03-29T08:21:09.957279+0000","last_fullsized":"2024-03-29T08:21:09.957279+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:24:09.856034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936946+0000","last_change":"2024-03-29T08:21:09.936946+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936946+0000","last_undegraded":"2024-03-29T08:21:09.936946+0000","last_fullsized":"2024-03-29T08:21:09.936946+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:05:09.416516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.934157+0000","last_change":"2024-03-29T08:21:08.934157+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.934157+0000","last_undegraded":"2024-03-29T08:21:08.934157+0000","last_fullsized":"2024-03-29T08:21:08.934157+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:50:03.760523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.941996+0000","last_change":"2024-03-29T08:21:09.941996+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.941996+0000","last_undegraded":"2024-03-29T08:21:09.941996+0000","last_fullsized":"2024-03-29T08:21:09.941996+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:50:48.557652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.948833+0000","last_change":"2024-03-29T08:21:09.947148+0000","last_active":"2024-03-29T08:21:09.948833+0000","last_peered":"2024-03-29T08:21:09.948833+0000","last_clean":"2024-03-29T08:21:09.948833+0000","last_became_active":"2024-03-29T08:21:09.946610+0000","last_became_peered":"2024-03-29T08:21:09.946610+0000","last_unstale":"2024-03-29T08:21:09.948833+0000","last_undegraded":"2024-03-29T08:21:09.948833+0000","last_fullsized":"2024-03-29T08:21:09.948833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:51.605750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.932484+0000","last_change":"2024-03-29T08:21:09.932484+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.932484+0000","last_undegraded":"2024-03-29T08:21:09.932484+0000","last_fullsized":"2024-03-29T08:21:09.932484+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:37:31.250178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.936989+0000","last_change":"2024-03-29T08:21:08.936989+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.936989+0000","last_undegraded":"2024-03-29T08:21:08.936989+0000","last_fullsized":"2024-03-29T08:21:08.936989+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:26:31.314621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937474+0000","last_change":"2024-03-29T08:21:09.937474+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937474+0000","last_undegraded":"2024-03-29T08:21:09.937474+0000","last_fullsized":"2024-03-29T08:21:09.937474+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:24:10.858914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956169+0000","last_change":"2024-03-29T08:21:09.953120+0000","last_active":"2024-03-29T08:21:09.956169+0000","last_peered":"2024-03-29T08:21:09.956169+0000","last_clean":"2024-03-29T08:21:09.956169+0000","last_became_active":"2024-03-29T08:21:09.952860+0000","last_became_peered":"2024-03-29T08:21:09.952860+0000","last_unstale":"2024-03-29T08:21:09.956169+0000","last_undegraded":"2024-03-29T08:21:09.956169+0000","last_fullsized":"2024-03-29T08:21:09.956169+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:38.455763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.934110+0000","last_change":"2024-03-29T08:21:08.934110+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.934110+0000","last_undegraded":"2024-03-29T08:21:08.934110+0000","last_fullsized":"2024-03-29T08:21:08.934110+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:05:43.897022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.948860+0000","last_change":"2024-03-29T08:21:09.947277+0000","last_active":"2024-03-29T08:21:09.948860+0000","last_peered":"2024-03-29T08:21:09.948860+0000","last_clean":"2024-03-29T08:21:09.948860+0000","last_became_active":"2024-03-29T08:21:09.945398+0000","last_became_peered":"2024-03-29T08:21:09.945398+0000","last_unstale":"2024-03-29T08:21:09.948860+0000","last_undegraded":"2024-03-29T08:21:09.948860+0000","last_fullsized":"2024-03-29T08:21:09.948860+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:36:29.788331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.935957+0000","last_change":"2024-03-29T08:21:09.935957+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.935957+0000","last_undegraded":"2024-03-29T08:21:09.935957+0000","last_fullsized":"2024-03-29T08:21:09.935957+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:18:28.430663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.957304+0000","last_change":"2024-03-29T08:21:09.955367+0000","last_active":"2024-03-29T08:21:09.957304+0000","last_peered":"2024-03-29T08:21:09.957304+0000","last_clean":"2024-03-29T08:21:09.957304+0000","last_became_active":"2024-03-29T08:21:09.954965+0000","last_became_peered":"2024-03-29T08:21:09.954965+0000","last_unstale":"2024-03-29T08:21:09.957304+0000","last_undegraded":"2024-03-29T08:21:09.957304+0000","last_fullsized":"2024-03-29T08:21:09.957304+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:57.828588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936065+0000","last_change":"2024-03-29T08:21:09.936065+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936065+0000","last_undegraded":"2024-03-29T08:21:09.936065+0000","last_fullsized":"2024-03-29T08:21:09.936065+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:01.741332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.952538+0000","last_change":"2024-03-29T08:21:09.948192+0000","last_active":"2024-03-29T08:21:09.952538+0000","last_peered":"2024-03-29T08:21:09.952538+0000","last_clean":"2024-03-29T08:21:09.952538+0000","last_became_active":"2024-03-29T08:21:09.947926+0000","last_became_peered":"2024-03-29T08:21:09.947926+0000","last_unstale":"2024-03-29T08:21:09.952538+0000","last_undegraded":"2024-03-29T08:21:09.952538+0000","last_fullsized":"2024-03-29T08:21:09.952538+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:33:58.392690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936565+0000","last_change":"2024-03-29T08:21:09.936565+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936565+0000","last_undegraded":"2024-03-29T08:21:09.936565+0000","last_fullsized":"2024-03-29T08:21:09.936565+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:22:08.727894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.950245+0000","last_change":"2024-03-29T08:21:09.947847+0000","last_active":"2024-03-29T08:21:09.950245+0000","last_peered":"2024-03-29T08:21:09.950245+0000","last_clean":"2024-03-29T08:21:09.950245+0000","last_became_active":"2024-03-29T08:21:09.946845+0000","last_became_peered":"2024-03-29T08:21:09.946845+0000","last_unstale":"2024-03-29T08:21:09.950245+0000","last_undegraded":"2024-03-29T08:21:09.950245+0000","last_fullsized":"2024-03-29T08:21:09.950245+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:19:13.916686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.943473+0000","last_change":"2024-03-29T08:21:09.938352+0000","last_active":"2024-03-29T08:21:09.943473+0000","last_peered":"2024-03-29T08:21:09.943473+0000","last_clean":"2024-03-29T08:21:09.943473+0000","last_became_active":"2024-03-29T08:21:09.937930+0000","last_became_peered":"2024-03-29T08:21:09.937930+0000","last_unstale":"2024-03-29T08:21:09.943473+0000","last_undegraded":"2024-03-29T08:21:09.943473+0000","last_fullsized":"2024-03-29T08:21:09.943473+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:23.163706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938692+0000","last_change":"2024-03-29T08:21:09.938692+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938692+0000","last_undegraded":"2024-03-29T08:21:09.938692+0000","last_fullsized":"2024-03-29T08:21:09.938692+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:44.680234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956168+0000","last_change":"2024-03-29T08:21:09.953317+0000","last_active":"2024-03-29T08:21:09.956168+0000","last_peered":"2024-03-29T08:21:09.956168+0000","last_clean":"2024-03-29T08:21:09.956168+0000","last_became_active":"2024-03-29T08:21:09.953068+0000","last_became_peered":"2024-03-29T08:21:09.953068+0000","last_unstale":"2024-03-29T08:21:09.956168+0000","last_undegraded":"2024-03-29T08:21:09.956168+0000","last_fullsized":"2024-03-29T08:21:09.956168+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:39.840479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.941589+0000","last_change":"2024-03-29T08:21:09.941589+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.941589+0000","last_undegraded":"2024-03-29T08:21:09.941589+0000","last_fullsized":"2024-03-29T08:21:09.941589+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:40.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.952730+0000","last_change":"2024-03-29T08:21:09.950160+0000","last_active":"2024-03-29T08:21:09.952730+0000","last_peered":"2024-03-29T08:21:09.952730+0000","last_clean":"2024-03-29T08:21:09.952730+0000","last_became_active":"2024-03-29T08:21:09.949853+0000","last_became_peered":"2024-03-29T08:21:09.949853+0000","last_unstale":"2024-03-29T08:21:09.952730+0000","last_undegraded":"2024-03-29T08:21:09.952730+0000","last_fullsized":"2024-03-29T08:21:09.952730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:59:23.381718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.947411+0000","last_change":"2024-03-29T08:21:09.945534+0000","last_active":"2024-03-29T08:21:09.947411+0000","last_peered":"2024-03-29T08:21:09.947411+0000","last_clean":"2024-03-29T08:21:09.947411+0000","last_became_active":"2024-03-29T08:21:09.943748+0000","last_became_peered":"2024-03-29T08:21:09.943748+0000","last_unstale":"2024-03-29T08:21:09.947411+0000","last_undegraded":"2024-03-29T08:21:09.947411+0000","last_fullsized":"2024-03-29T08:21:09.947411+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:05:06.965072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936902+0000","last_change":"2024-03-29T08:21:09.936902+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936902+0000","last_undegraded":"2024-03-29T08:21:09.936902+0000","last_fullsized":"2024-03-29T08:21:09.936902+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:38:18.883117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.934050+0000","last_change":"2024-03-29T08:21:08.934050+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.934050+0000","last_undegraded":"2024-03-29T08:21:08.934050+0000","last_fullsized":"2024-03-29T08:21:08.934050+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:00:36.010161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.955570+0000","last_change":"2024-03-29T08:21:09.953812+0000","last_active":"2024-03-29T08:21:09.955570+0000","last_peered":"2024-03-29T08:21:09.955570+0000","last_clean":"2024-03-29T08:21:09.955570+0000","last_became_active":"2024-03-29T08:21:09.953475+0000","last_became_peered":"2024-03-29T08:21:09.953475+0000","last_unstale":"2024-03-29T08:21:09.955570+0000","last_undegraded":"2024-03-29T08:21:09.955570+0000","last_fullsized":"2024-03-29T08:21:09.955570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:05:21.462217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":196,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-03-29T08:21:08.935647+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:21:08.935647+0000","last_peered":"2024-03-29T08:21:08.935647+0000","last_clean":"2024-03-29T08:21:08.935647+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:21:08.935647+0000","last_undegraded":"2024-03-29T08:21:08.935647+0000","last_fullsized":"2024-03-29T08:21:08.935647+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":7},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":293626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67600000000000005,"15min":0.65200000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93400000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.69999999999999996,"15min":0.67600000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1200000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.68700000000000006}]},{"osd":1,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.67100000000000004,"15min":0.64800000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.66800000000000004,"15min":0.63400000000000001},"min":{"1min":0.52900000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.0189999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.82599999999999996}]},{"osd":2,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.66200000000000003,"15min":0.63300000000000001},"min":{"1min":0.55900000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.0089999999999999,"5min":1.085,"15min":1.085},"last":0.754},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.69299999999999995,"15min":0.66500000000000004},"min":{"1min":0.58499999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1000000000000001,"5min":1.288,"15min":1.288},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71099999999999997,"15min":0.69799999999999995},"min":{"1min":0.57899999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92800000000000005,"5min":1.018,"15min":1.018},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.745,"15min":0.73099999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1659999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.73499999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83499999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70199999999999996,"15min":0.71999999999999997},"min":{"1min":0.46100000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0009999999999999,"5min":1.123,"15min":1.123},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.95799999999999996,"5min":1.056,"15min":1.056},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.46300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73399999999999999,"15min":0.72999999999999998},"min":{"1min":0.48999999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.755},"min":{"1min":0.52300000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.84199999999999997}]}]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":293626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.66000000000000003,"15min":0.63300000000000001},"min":{"1min":0.54000000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70499999999999996,"15min":0.68500000000000005},"min":{"1min":0.57899999999999996,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.71399999999999997}]},{"osd":1,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.64100000000000001,"15min":0.61499999999999999},"min":{"1min":0.58599999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.64800000000000002,"15min":0.627},"min":{"1min":0.51600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.70299999999999996,"15min":0.67500000000000004},"min":{"1min":0.52300000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.54300000000000004,"5min":0.435,"15min":0.435},"max":{"1min":0.91200000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.81100000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71399999999999997,"15min":0.69999999999999996},"min":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94099999999999995,"5min":1.109,"15min":1.109},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.53200000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.84499999999999997}]},{"osd":4,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0620000000000001,"5min":1.556,"15min":1.556},"last":1.02},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70499999999999996,"15min":0.73699999999999999},"min":{"1min":0.44400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.0429999999999999,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.0429999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67800000000000005,"15min":0.70299999999999996},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.71599999999999997,"15min":0.73399999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.86299999999999999}]},{"osd":7,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.74199999999999999,"15min":0.76700000000000002},"min":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.92300000000000004,"5min":0.998,"15min":0.998},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.753,"15min":0.77900000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":1.109}]}]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":293626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.50800000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93100000000000005,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72699999999999998,"15min":0.74399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95199999999999996,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.73599999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.76500000000000001},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97799999999999998,"5min":2.524,"15min":2.524},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.54900000000000004,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93999999999999995,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.83699999999999997}]},{"osd":2,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84499999999999997,"5min":2.363,"15min":2.363},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0029999999999999,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.76900000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.747,"15min":0.75700000000000001},"min":{"1min":0.54700000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90000000000000002,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.753},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.81699999999999995},"min":{"1min":0.625,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.96199999999999997,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.86899999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68200000000000005,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88100000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.441,"5min":0.434,"15min":0.434},"max":{"1min":0.82199999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.65900000000000003}]},{"osd":6,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86799999999999999,"5min":0.996,"15min":0.996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72599999999999998,"15min":0.74399999999999999},"min":{"1min":0.52300000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.91600000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.63800000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71699999999999997,"15min":0.73799999999999999},"min":{"1min":0.51100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90900000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.46999999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92100000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.68999999999999995}]}]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":724992,"data_stored":299901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.46300000000000002},"min":{"1min":0.33700000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.72699999999999998,"5min":1.123,"15min":1.123},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52000000000000002,"15min":0.52400000000000002},"min":{"1min":0.432,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.871,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.437}]},{"osd":1,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44700000000000001,"15min":0.46100000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64800000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.51500000000000001,"15min":0.53400000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.68200000000000005,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.28599999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54700000000000004,"15min":0.56999999999999995},"min":{"1min":0.35999999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.70599999999999996,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63700000000000001,"15min":0.65900000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79700000000000004,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.41799999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.77400000000000002,"15min":0.79600000000000004},"min":{"1min":0.52100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1950000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.75900000000000001,"15min":0.77900000000000003},"min":{"1min":0.55700000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.004,"5min":1.391,"15min":1.391},"last":0.64600000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.80800000000000005},"min":{"1min":0.627,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84299999999999997,"15min":0.86699999999999999},"min":{"1min":0.64500000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6599999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.81000000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.79600000000000004,"15min":0.83599999999999997},"min":{"1min":0.51500000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.008,"5min":1.669,"15min":1.669},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.80300000000000005,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.212,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.76200000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82999999999999996,"15min":0.86399999999999999},"min":{"1min":0.54900000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4279999999999999,"5min":1.887,"15min":1.887},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.82099999999999995,"15min":0.86099999999999999},"min":{"1min":0.59899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98299999999999998,"5min":2.044,"15min":2.044},"last":0.70299999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":712704,"data_stored":293626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.48899999999999999,"15min":0.501},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69799999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66300000000000003,"5min":0.751,"15min":0.751},"last":0.65900000000000003}]},{"osd":1,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.34499999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72999999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.375},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.48999999999999999,"15min":0.50800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72499999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.39900000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75900000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40400000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69299999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.61599999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52600000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.052,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.75900000000000001,"15min":0.76600000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82999999999999996,"15min":0.84199999999999997},"min":{"1min":0.64600000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.081,"5min":1.306,"15min":1.306},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83999999999999997,"15min":0.85999999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.28,"5min":1.526,"15min":1.526},"last":0.55500000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.77700000000000002,"15min":0.77900000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.55600000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.121,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.53400000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.51000000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":0.79800000000000004},"min":{"1min":0.56499999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.71899999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1306624,"data_stored":883994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.56799999999999995,"15min":0.63500000000000001},"min":{"1min":0.33400000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.54800000000000004,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.60099999999999998,"15min":0.65300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71799999999999997,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.49399999999999999,"15min":0.50600000000000001},"min":{"1min":0.375,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73299999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53000000000000003,"15min":0.53600000000000003},"min":{"1min":0.39100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67000000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.435}]},{"osd":3,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.86099999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.75900000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.41299999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.53000000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95599999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.746},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.76300000000000001,"15min":0.78900000000000003},"min":{"1min":0.58099999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95199999999999996,"5min":1.23,"15min":1.23},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76100000000000001,"15min":0.77900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.89500000000000002,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.56499999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.873,"5min":1.107,"15min":1.107},"last":0.67800000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76000000000000001,"15min":0.76800000000000002},"min":{"1min":0.57399999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":1.026,"15min":1.026},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.81799999999999995,"15min":0.85699999999999998},"min":{"1min":0.55800000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.83199999999999996,"5min":1.196,"15min":1.196},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.78700000000000003,"15min":0.80000000000000004},"min":{"1min":0.57499999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.92900000000000005,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.78700000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91600000000000004,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.66100000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":724992,"data_stored":299901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.55300000000000005,"15min":0.61299999999999999},"min":{"1min":0.313,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.71099999999999997,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.753,"15min":0.873},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.73799999999999999,"5min":8.0419999999999998,"15min":8.0419999999999998},"last":0.34000000000000002}]},{"osd":2,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.53800000000000003,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.58899999999999997,"5min":1.258,"15min":1.258},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.67500000000000004,"15min":0.747},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63800000000000001,"5min":4.907,"15min":4.907},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.753,"15min":0.872},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.77000000000000002,"5min":7.9569999999999999,"15min":7.9569999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.624,"15min":0.66600000000000004},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.65100000000000002,"5min":1.397,"15min":1.397},"last":0.44900000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":1.002},"min":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91700000000000004,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.91500000000000004,"15min":0.98099999999999998},"min":{"1min":0.66700000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.97999999999999998,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.77000000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.879,"15min":0.93999999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.016,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.83299999999999996,"15min":0.871},"min":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.89900000000000002,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.754}]},{"osd":6,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.83899999999999997,"15min":0.87},"min":{"1min":0.64500000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95699999999999996,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.92900000000000005,"15min":0.999},"min":{"1min":0.623,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.097,"5min":4.4100000000000001,"15min":4.4100000000000001},"last":0.68100000000000005}]},{"osd":7,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.94599999999999995,"15min":1.0209999999999999},"min":{"1min":0.64100000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.038,"5min":4.5209999999999999,"15min":4.5209999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.92100000000000004,"15min":1},"min":{"1min":0.57699999999999996,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.099,"5min":4.4420000000000002,"15min":4.4420000000000002},"last":0.57699999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1306624,"data_stored":883994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.497,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89400000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.497}]},{"osd":1,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71599999999999997,"15min":0.73799999999999999},"min":{"1min":0.51700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.876,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67800000000000005,"15min":0.67700000000000005},"min":{"1min":0.44900000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89000000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.443,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84299999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.443},{"interface":"front","average":{"1min":0.748,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.59199999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90700000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.88600000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.54800000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91800000000000004,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.64500000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.90500000000000003,"5min":1.954,"15min":1.954},"last":0.80800000000000005}]},{"osd":4,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.50800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.51400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88700000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.82899999999999996}]},{"osd":6,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66700000000000004,"15min":0.68200000000000005},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82399999999999995,"5min":1.071,"15min":1.071},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.70099999999999996},"min":{"1min":0.53600000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.878,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92900000000000005,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.74099999999999999,"15min":0.75800000000000001},"min":{"1min":0.56799999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95199999999999996,"5min":1.04,"15min":1.04},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:21:14.452 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:21:14.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57c5800640 1 Processor -- start 2024-03-29T08:21:14.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57c5800640 1 -- start start 2024-03-29T08:21:14.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57c5800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f57c0144210 0x7f57c00cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:14.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57c5800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57c00ceb20 0x7f57c006c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:14.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57c5800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c00cb510 con 0x7f57c015ada0 2024-03-29T08:21:14.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57c5800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c0070580 con 0x7f57c0144210 2024-03-29T08:21:14.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57c5800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c0070860 con 0x7f57c00ceb20 2024-03-29T08:21:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57bf400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57c00ceb20 0x7f57c006c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57bea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f57c0144210 0x7f57c00cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57bea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f57c0144210 0x7f57c00cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:37392/0 (socket says 172.21.15.46:37392) 2024-03-29T08:21:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57bea00640 1 -- 172.21.15.46:0/630354741 learned_addr learned my addr 172.21.15.46:0/630354741 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57bf400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57c00ceb20 0x7f57c006c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:45614/0 (socket says 172.21.15.46:45614) 2024-03-29T08:21:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57be000640 1 --1- 172.21.15.46:0/630354741 >> v1:172.21.15.73:6789/0 conn(0x7f57c015ada0 0x7f57c00ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:50282/0 (socket says 172.21.15.46:50282) 2024-03-29T08:21:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57bea00640 1 -- 172.21.15.46:0/630354741 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57c00ceb20 msgr2=0x7f57c006c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57bea00640 1 --2- 172.21.15.46:0/630354741 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57c00ceb20 0x7f57c006c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57bea00640 1 -- 172.21.15.46:0/630354741 >> v1:172.21.15.73:6789/0 conn(0x7f57c015ada0 legacy=0x7f57c00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57bea00640 1 -- 172.21.15.46:0/630354741 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c0070b40 con 0x7f57c0144210 2024-03-29T08:21:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57bf400640 1 --2- 172.21.15.46:0/630354741 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57c00ceb20 0x7f57c006c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:21:14.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.631+0000 7f57bd600640 1 -- 172.21.15.46:0/630354741 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679980772 0 0) 0x7f57c00cb510 con 0x7f57c015ada0 2024-03-29T08:21:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.635+0000 7f57bea00640 1 --2- 172.21.15.46:0/630354741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c0144210 0x7f57c00cb8b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009870 tx=0x7f57b001c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa6de73c811b88ff server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.635+0000 7f57bd600640 1 -- 172.21.15.46:0/630354741 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b00ab670 con 0x7f57c0144210 2024-03-29T08:21:14.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.635+0000 7f57bd600640 1 -- 172.21.15.46:0/630354741 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b00ab7d0 con 0x7f57c0144210 2024-03-29T08:21:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.635+0000 7f57bd600640 1 -- 172.21.15.46:0/630354741 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b00abb60 con 0x7f57c0144210 2024-03-29T08:21:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.635+0000 7f57c5800640 1 -- 172.21.15.46:0/630354741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c0144210 msgr2=0x7f57c00cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.635+0000 7f57c5800640 1 --2- 172.21.15.46:0/630354741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c0144210 0x7f57c00cb8b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f57b0009870 tx=0x7f57b001c560 comp rx=0 tx=0).stop 2024-03-29T08:21:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.635+0000 7f57c5800640 1 -- 172.21.15.46:0/630354741 shutdown_connections 2024-03-29T08:21:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.635+0000 7f57c5800640 1 --2- 172.21.15.46:0/630354741 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57c00ceb20 0x7f57c006c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.635+0000 7f57c5800640 1 --2- 172.21.15.46:0/630354741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c0144210 0x7f57c00cb8b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.635+0000 7f57c5800640 1 -- 172.21.15.46:0/630354741 >> 172.21.15.46:0/630354741 conn(0x7f57c0066110 msgr2=0x7f57c0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.635+0000 7f57c5800640 1 -- 172.21.15.46:0/630354741 shutdown_connections 2024-03-29T08:21:14.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.635+0000 7f57c5800640 1 -- 172.21.15.46:0/630354741 wait complete. 2024-03-29T08:21:14.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57c5800640 1 Processor -- start 2024-03-29T08:21:14.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57c5800640 1 -- start start 2024-03-29T08:21:14.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57c5800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57c00ceb20 0x7f57c015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57c5800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c0144210 0x7f57c015b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57c5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c015ada0 0x7f57c0139b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57c5800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c013a0d0 con 0x7f57c015ada0 2024-03-29T08:21:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57c5800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c015cc60 con 0x7f57c0144210 2024-03-29T08:21:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57c5800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c015c050 con 0x7f57c00ceb20 2024-03-29T08:21:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c015ada0 0x7f57c0139b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c015ada0 0x7f57c0139b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54390/0 (socket says 172.21.15.46:54390) 2024-03-29T08:21:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57bf400640 1 -- 172.21.15.46:0/1367603036 learned_addr learned my addr 172.21.15.46:0/1367603036 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57bea00640 1 --2- 172.21.15.46:0/1367603036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57c00ceb20 0x7f57c015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57be000640 1 --2- 172.21.15.46:0/1367603036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c0144210 0x7f57c015b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57bf400640 1 -- 172.21.15.46:0/1367603036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57c00ceb20 msgr2=0x7f57c015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57bf400640 1 --2- 172.21.15.46:0/1367603036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57c00ceb20 0x7f57c015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57bf400640 1 -- 172.21.15.46:0/1367603036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c0144210 msgr2=0x7f57c015b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57bf400640 1 --2- 172.21.15.46:0/1367603036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c0144210 0x7f57c015b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57bf400640 1 -- 172.21.15.46:0/1367603036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b8009a70 con 0x7f57c015ada0 2024-03-29T08:21:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57bea00640 1 --2- 172.21.15.46:0/1367603036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57c00ceb20 0x7f57c015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57bf400640 1 --2- 172.21.15.46:0/1367603036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c015ada0 0x7f57c0139b90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f57b800efc0 tx=0x7f57b8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:14.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57b6a00640 1 -- 172.21.15.46:0/1367603036 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b8009d70 con 0x7f57c015ada0 2024-03-29T08:21:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57b6a00640 1 -- 172.21.15.46:0/1367603036 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b800f040 con 0x7f57c015ada0 2024-03-29T08:21:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57b6a00640 1 -- 172.21.15.46:0/1367603036 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b80039a0 con 0x7f57c015ada0 2024-03-29T08:21:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57c5800640 1 -- 172.21.15.46:0/1367603036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b00094d0 con 0x7f57c015ada0 2024-03-29T08:21:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57c5800640 1 -- 172.21.15.46:0/1367603036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c015c610 con 0x7f57c015ada0 2024-03-29T08:21:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.639+0000 7f57c5800640 1 -- 172.21.15.46:0/1367603036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c00ce280 con 0x7f57c015ada0 2024-03-29T08:21:14.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.643+0000 7f57b6a00640 1 -- 172.21.15.46:0/1367603036 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f57b80144a0 con 0x7f57c015ada0 2024-03-29T08:21:14.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.643+0000 7f57b6a00640 1 --2- 172.21.15.46:0/1367603036 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f57a0066690 0x7f57a0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:14.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.643+0000 7f57b6a00640 1 -- 172.21.15.46:0/1367603036 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57b8084f60 con 0x7f57c015ada0 2024-03-29T08:21:14.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.643+0000 7f57bea00640 1 --2- 172.21.15.46:0/1367603036 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f57a0066690 0x7f57a0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:14.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.643+0000 7f57bea00640 1 --2- 172.21.15.46:0/1367603036 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f57a0066690 0x7f57a0068b50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f57b00af000 tx=0x7f57b00a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:14.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.647+0000 7f57b6a00640 1 -- 172.21.15.46:0/1367603036 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f57b80521b0 con 0x7f57c015ada0 2024-03-29T08:21:14.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.803+0000 7f57c5800640 1 -- 172.21.15.46:0/1367603036 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f57c006d6a0 con 0x7f57a0066690 2024-03-29T08:21:14.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.811+0000 7f57b6a00640 1 -- 172.21.15.46:0/1367603036 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284914 (secure 0 0 0) 0x7f57b00ab9c0 con 0x7f57a0066690 2024-03-29T08:21:14.818 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:14.819 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:21:14.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.815+0000 7f57c5800640 1 -- 172.21.15.46:0/1367603036 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f57a0066690 msgr2=0x7f57a0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.815+0000 7f57c5800640 1 --2- 172.21.15.46:0/1367603036 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f57a0066690 0x7f57a0068b50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f57b00af000 tx=0x7f57b00a7000 comp rx=0 tx=0).stop 2024-03-29T08:21:14.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.815+0000 7f57c5800640 1 -- 172.21.15.46:0/1367603036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c015ada0 msgr2=0x7f57c0139b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:14.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.815+0000 7f57c5800640 1 --2- 172.21.15.46:0/1367603036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c015ada0 0x7f57c0139b90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f57b800efc0 tx=0x7f57b8009f90 comp rx=0 tx=0).stop 2024-03-29T08:21:14.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.815+0000 7f57c5800640 1 -- 172.21.15.46:0/1367603036 shutdown_connections 2024-03-29T08:21:14.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.815+0000 7f57c5800640 1 --2- 172.21.15.46:0/1367603036 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f57a0066690 0x7f57a0068b50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.815+0000 7f57c5800640 1 --2- 172.21.15.46:0/1367603036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c0144210 0x7f57c015b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.815+0000 7f57c5800640 1 --2- 172.21.15.46:0/1367603036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57c015ada0 0x7f57c0139b90 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.815+0000 7f57c5800640 1 --2- 172.21.15.46:0/1367603036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57c00ceb20 0x7f57c015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:14.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.815+0000 7f57c5800640 1 -- 172.21.15.46:0/1367603036 >> 172.21.15.46:0/1367603036 conn(0x7f57c0066110 msgr2=0x7f57c013c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:14.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.815+0000 7f57c5800640 1 -- 172.21.15.46:0/1367603036 shutdown_connections 2024-03-29T08:21:14.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:14.815+0000 7f57c5800640 1 -- 172.21.15.46:0/1367603036 wait complete. 2024-03-29T08:21:14.837 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-03-29T08:21:14.084517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243396,"kb_used_data":6752,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754731324,"statfs":{"total":773094113280,"available":772844875776,"internally_reserved":0,"allocated":6914048,"data_stored":3542294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":242208313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.157425"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.941725+0000","last_change":"2024-03-29T08:21:09.941725+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.941725+0000","last_undegraded":"2024-03-29T08:21:09.941725+0000","last_fullsized":"2024-03-29T08:21:09.941725+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:16:15.767217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936961+0000","last_change":"2024-03-29T08:21:09.936961+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936961+0000","last_undegraded":"2024-03-29T08:21:09.936961+0000","last_fullsized":"2024-03-29T08:21:09.936961+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:05:22.465098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938613+0000","last_change":"2024-03-29T08:21:09.938613+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938613+0000","last_undegraded":"2024-03-29T08:21:09.938613+0000","last_fullsized":"2024-03-29T08:21:09.938613+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:47:48.297292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.933020+0000","last_change":"2024-03-29T08:21:09.933020+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.933020+0000","last_undegraded":"2024-03-29T08:21:09.933020+0000","last_fullsized":"2024-03-29T08:21:09.933020+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:51.414360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936757+0000","last_change":"2024-03-29T08:21:09.936757+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936757+0000","last_undegraded":"2024-03-29T08:21:09.936757+0000","last_fullsized":"2024-03-29T08:21:09.936757+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:36:29.239728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939144+0000","last_change":"2024-03-29T08:21:09.939144+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939144+0000","last_undegraded":"2024-03-29T08:21:09.939144+0000","last_fullsized":"2024-03-29T08:21:09.939144+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:35:20.420999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.933702+0000","last_change":"2024-03-29T08:21:09.933702+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.933702+0000","last_undegraded":"2024-03-29T08:21:09.933702+0000","last_fullsized":"2024-03-29T08:21:09.933702+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:30:03.234601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939490+0000","last_change":"2024-03-29T08:21:09.939490+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939490+0000","last_undegraded":"2024-03-29T08:21:09.939490+0000","last_fullsized":"2024-03-29T08:21:09.939490+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:40.843359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.940289+0000","last_change":"2024-03-29T08:21:09.940289+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.940289+0000","last_undegraded":"2024-03-29T08:21:09.940289+0000","last_fullsized":"2024-03-29T08:21:09.940289+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:54.034461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.934375+0000","last_change":"2024-03-29T08:21:09.934375+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.934375+0000","last_undegraded":"2024-03-29T08:21:09.934375+0000","last_fullsized":"2024-03-29T08:21:09.934375+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:58.105380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.940141+0000","last_change":"2024-03-29T08:21:09.940141+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.940141+0000","last_undegraded":"2024-03-29T08:21:09.940141+0000","last_fullsized":"2024-03-29T08:21:09.940141+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:20:51.330512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.942430+0000","last_change":"2024-03-29T08:21:09.942430+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.942430+0000","last_undegraded":"2024-03-29T08:21:09.942430+0000","last_fullsized":"2024-03-29T08:21:09.942430+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:35:20.420999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939640+0000","last_change":"2024-03-29T08:21:09.939640+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939640+0000","last_undegraded":"2024-03-29T08:21:09.939640+0000","last_fullsized":"2024-03-29T08:21:09.939640+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:49:07.458847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.940293+0000","last_change":"2024-03-29T08:21:09.940293+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.940293+0000","last_undegraded":"2024-03-29T08:21:09.940293+0000","last_fullsized":"2024-03-29T08:21:09.940293+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:39.458643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938066+0000","last_change":"2024-03-29T08:21:09.938066+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938066+0000","last_undegraded":"2024-03-29T08:21:09.938066+0000","last_fullsized":"2024-03-29T08:21:09.938066+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:01:52.376061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.941263+0000","last_change":"2024-03-29T08:21:09.941263+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.941263+0000","last_undegraded":"2024-03-29T08:21:09.941263+0000","last_fullsized":"2024-03-29T08:21:09.941263+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:39.458643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939231+0000","last_change":"2024-03-29T08:21:09.939231+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939231+0000","last_undegraded":"2024-03-29T08:21:09.939231+0000","last_fullsized":"2024-03-29T08:21:09.939231+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:33:19.998323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.942709+0000","last_change":"2024-03-29T08:21:09.942709+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.942709+0000","last_undegraded":"2024-03-29T08:21:09.942709+0000","last_fullsized":"2024-03-29T08:21:09.942709+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:58:48.804311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.934214+0000","last_change":"2024-03-29T08:21:09.934214+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.934214+0000","last_undegraded":"2024-03-29T08:21:09.934214+0000","last_fullsized":"2024-03-29T08:21:09.934214+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:01.741332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.942033+0000","last_change":"2024-03-29T08:21:09.942033+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.942033+0000","last_undegraded":"2024-03-29T08:21:09.942033+0000","last_fullsized":"2024-03-29T08:21:09.942033+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:44.680234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.934012+0000","last_change":"2024-03-29T08:21:09.934012+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.934012+0000","last_undegraded":"2024-03-29T08:21:09.934012+0000","last_fullsized":"2024-03-29T08:21:09.934012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:15.253098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.933212+0000","last_change":"2024-03-29T08:21:09.933212+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.933212+0000","last_undegraded":"2024-03-29T08:21:09.933212+0000","last_fullsized":"2024-03-29T08:21:09.933212+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:25:00.595704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938923+0000","last_change":"2024-03-29T08:21:09.938923+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938923+0000","last_undegraded":"2024-03-29T08:21:09.938923+0000","last_fullsized":"2024-03-29T08:21:09.938923+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:58.831468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937337+0000","last_change":"2024-03-29T08:21:09.937337+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937337+0000","last_undegraded":"2024-03-29T08:21:09.937337+0000","last_fullsized":"2024-03-29T08:21:09.937337+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:59:24.384598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.943515+0000","last_change":"2024-03-29T08:21:09.943515+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.943515+0000","last_undegraded":"2024-03-29T08:21:09.943515+0000","last_fullsized":"2024-03-29T08:21:09.943515+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:53.450623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936629+0000","last_change":"2024-03-29T08:21:09.936629+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936629+0000","last_undegraded":"2024-03-29T08:21:09.936629+0000","last_fullsized":"2024-03-29T08:21:09.936629+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:49.605189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.932435+0000","last_change":"2024-03-29T08:21:09.932435+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.932435+0000","last_undegraded":"2024-03-29T08:21:09.932435+0000","last_fullsized":"2024-03-29T08:21:09.932435+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:45.698306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.943149+0000","last_change":"2024-03-29T08:21:09.943149+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.943149+0000","last_undegraded":"2024-03-29T08:21:09.943149+0000","last_fullsized":"2024-03-29T08:21:09.943149+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:01:19.390653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.934024+0000","last_change":"2024-03-29T08:21:09.934024+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.934024+0000","last_undegraded":"2024-03-29T08:21:09.934024+0000","last_fullsized":"2024-03-29T08:21:09.934024+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:54.034461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939827+0000","last_change":"2024-03-29T08:21:09.939827+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939827+0000","last_undegraded":"2024-03-29T08:21:09.939827+0000","last_fullsized":"2024-03-29T08:21:09.939827+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:53.450623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956239+0000","last_change":"2024-03-29T08:21:09.953242+0000","last_active":"2024-03-29T08:21:09.956239+0000","last_peered":"2024-03-29T08:21:09.956239+0000","last_clean":"2024-03-29T08:21:09.956239+0000","last_became_active":"2024-03-29T08:21:09.952978+0000","last_became_peered":"2024-03-29T08:21:09.952978+0000","last_unstale":"2024-03-29T08:21:09.956239+0000","last_undegraded":"2024-03-29T08:21:09.956239+0000","last_fullsized":"2024-03-29T08:21:09.956239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:43.677354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937754+0000","last_change":"2024-03-29T08:21:09.937754+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937754+0000","last_undegraded":"2024-03-29T08:21:09.937754+0000","last_fullsized":"2024-03-29T08:21:09.937754+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:40.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.957328+0000","last_change":"2024-03-29T08:21:09.955717+0000","last_active":"2024-03-29T08:21:09.957328+0000","last_peered":"2024-03-29T08:21:09.957328+0000","last_clean":"2024-03-29T08:21:09.957328+0000","last_became_active":"2024-03-29T08:21:09.955450+0000","last_became_peered":"2024-03-29T08:21:09.955450+0000","last_unstale":"2024-03-29T08:21:09.957328+0000","last_undegraded":"2024-03-29T08:21:09.957328+0000","last_fullsized":"2024-03-29T08:21:09.957328+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:47:40.877803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.952711+0000","last_change":"2024-03-29T08:21:09.950482+0000","last_active":"2024-03-29T08:21:09.952711+0000","last_peered":"2024-03-29T08:21:09.952711+0000","last_clean":"2024-03-29T08:21:09.952711+0000","last_became_active":"2024-03-29T08:21:09.950190+0000","last_became_peered":"2024-03-29T08:21:09.950190+0000","last_unstale":"2024-03-29T08:21:09.952711+0000","last_undegraded":"2024-03-29T08:21:09.952711+0000","last_fullsized":"2024-03-29T08:21:09.952711+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:07:20.233898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938835+0000","last_change":"2024-03-29T08:21:09.938835+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938835+0000","last_undegraded":"2024-03-29T08:21:09.938835+0000","last_fullsized":"2024-03-29T08:21:09.938835+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:21:23.293798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.944401+0000","last_change":"2024-03-29T08:21:09.942997+0000","last_active":"2024-03-29T08:21:09.944401+0000","last_peered":"2024-03-29T08:21:09.944401+0000","last_clean":"2024-03-29T08:21:09.944401+0000","last_became_active":"2024-03-29T08:21:09.942730+0000","last_became_peered":"2024-03-29T08:21:09.942730+0000","last_unstale":"2024-03-29T08:21:09.944401+0000","last_undegraded":"2024-03-29T08:21:09.944401+0000","last_fullsized":"2024-03-29T08:21:09.944401+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:42:48.592612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.947598+0000","last_change":"2024-03-29T08:21:09.944500+0000","last_active":"2024-03-29T08:21:09.947598+0000","last_peered":"2024-03-29T08:21:09.947598+0000","last_clean":"2024-03-29T08:21:09.947598+0000","last_became_active":"2024-03-29T08:21:09.944236+0000","last_became_peered":"2024-03-29T08:21:09.944236+0000","last_unstale":"2024-03-29T08:21:09.947598+0000","last_undegraded":"2024-03-29T08:21:09.947598+0000","last_fullsized":"2024-03-29T08:21:09.947598+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:06:08.420341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.935582+0000","last_change":"2024-03-29T08:21:09.935582+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.935582+0000","last_undegraded":"2024-03-29T08:21:09.935582+0000","last_fullsized":"2024-03-29T08:21:09.935582+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:05:09.416516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.952308+0000","last_change":"2024-03-29T08:21:09.946818+0000","last_active":"2024-03-29T08:21:09.952308+0000","last_peered":"2024-03-29T08:21:09.952308+0000","last_clean":"2024-03-29T08:21:09.952308+0000","last_became_active":"2024-03-29T08:21:09.946555+0000","last_became_peered":"2024-03-29T08:21:09.946555+0000","last_unstale":"2024-03-29T08:21:09.952308+0000","last_undegraded":"2024-03-29T08:21:09.952308+0000","last_fullsized":"2024-03-29T08:21:09.952308+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:27:46.452557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937385+0000","last_change":"2024-03-29T08:21:09.937385+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937385+0000","last_undegraded":"2024-03-29T08:21:09.937385+0000","last_fullsized":"2024-03-29T08:21:09.937385+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:58.105380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956350+0000","last_change":"2024-03-29T08:21:09.954162+0000","last_active":"2024-03-29T08:21:09.956350+0000","last_peered":"2024-03-29T08:21:09.956350+0000","last_clean":"2024-03-29T08:21:09.956350+0000","last_became_active":"2024-03-29T08:21:09.953849+0000","last_became_peered":"2024-03-29T08:21:09.953849+0000","last_unstale":"2024-03-29T08:21:09.956350+0000","last_undegraded":"2024-03-29T08:21:09.956350+0000","last_fullsized":"2024-03-29T08:21:09.956350+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:52.803477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937620+0000","last_change":"2024-03-29T08:21:09.937620+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937620+0000","last_undegraded":"2024-03-29T08:21:09.937620+0000","last_fullsized":"2024-03-29T08:21:09.937620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:19:16.700510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956277+0000","last_change":"2024-03-29T08:21:09.954120+0000","last_active":"2024-03-29T08:21:09.956277+0000","last_peered":"2024-03-29T08:21:09.956277+0000","last_clean":"2024-03-29T08:21:09.956277+0000","last_became_active":"2024-03-29T08:21:09.953375+0000","last_became_peered":"2024-03-29T08:21:09.953375+0000","last_unstale":"2024-03-29T08:21:09.956277+0000","last_undegraded":"2024-03-29T08:21:09.956277+0000","last_fullsized":"2024-03-29T08:21:09.956277+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:48.602309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.943037+0000","last_change":"2024-03-29T08:21:09.943037+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.943037+0000","last_undegraded":"2024-03-29T08:21:09.943037+0000","last_fullsized":"2024-03-29T08:21:09.943037+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:24.166586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.953570+0000","last_change":"2024-03-29T08:21:09.951304+0000","last_active":"2024-03-29T08:21:09.953570+0000","last_peered":"2024-03-29T08:21:09.953570+0000","last_clean":"2024-03-29T08:21:09.953570+0000","last_became_active":"2024-03-29T08:21:09.951067+0000","last_became_peered":"2024-03-29T08:21:09.951067+0000","last_unstale":"2024-03-29T08:21:09.953570+0000","last_undegraded":"2024-03-29T08:21:09.953570+0000","last_fullsized":"2024-03-29T08:21:09.953570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:13:08.676082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938641+0000","last_change":"2024-03-29T08:21:09.938641+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938641+0000","last_undegraded":"2024-03-29T08:21:09.938641+0000","last_fullsized":"2024-03-29T08:21:09.938641+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:08:39.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.939701+0000","last_change":"2024-03-29T08:21:08.939701+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.939701+0000","last_undegraded":"2024-03-29T08:21:08.939701+0000","last_fullsized":"2024-03-29T08:21:08.939701+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:08:44.015717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.945733+0000","last_change":"2024-03-29T08:21:09.944508+0000","last_active":"2024-03-29T08:21:09.945733+0000","last_peered":"2024-03-29T08:21:09.945733+0000","last_clean":"2024-03-29T08:21:09.945733+0000","last_became_active":"2024-03-29T08:21:09.944191+0000","last_became_peered":"2024-03-29T08:21:09.944191+0000","last_unstale":"2024-03-29T08:21:09.945733+0000","last_undegraded":"2024-03-29T08:21:09.945733+0000","last_fullsized":"2024-03-29T08:21:09.945733+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:25:36.003763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.932991+0000","last_change":"2024-03-29T08:21:09.932991+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.932991+0000","last_undegraded":"2024-03-29T08:21:09.932991+0000","last_fullsized":"2024-03-29T08:21:09.932991+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:56:39.434467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.955648+0000","last_change":"2024-03-29T08:21:09.953728+0000","last_active":"2024-03-29T08:21:09.955648+0000","last_peered":"2024-03-29T08:21:09.955648+0000","last_clean":"2024-03-29T08:21:09.955648+0000","last_became_active":"2024-03-29T08:21:09.953431+0000","last_became_peered":"2024-03-29T08:21:09.953431+0000","last_unstale":"2024-03-29T08:21:09.955648+0000","last_undegraded":"2024-03-29T08:21:09.955648+0000","last_fullsized":"2024-03-29T08:21:09.955648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:38:17.880237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939449+0000","last_change":"2024-03-29T08:21:09.939449+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939449+0000","last_undegraded":"2024-03-29T08:21:09.939449+0000","last_fullsized":"2024-03-29T08:21:09.939449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:52:34.720375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.934101+0000","last_change":"2024-03-29T08:21:08.934101+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.934101+0000","last_undegraded":"2024-03-29T08:21:08.934101+0000","last_fullsized":"2024-03-29T08:21:08.934101+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:03:13.729828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937954+0000","last_change":"2024-03-29T08:21:09.937954+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937954+0000","last_undegraded":"2024-03-29T08:21:09.937954+0000","last_fullsized":"2024-03-29T08:21:09.937954+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:21:28.741076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.950524+0000","last_change":"2024-03-29T08:21:09.948902+0000","last_active":"2024-03-29T08:21:09.950524+0000","last_peered":"2024-03-29T08:21:09.950524+0000","last_clean":"2024-03-29T08:21:09.950524+0000","last_became_active":"2024-03-29T08:21:09.948613+0000","last_became_peered":"2024-03-29T08:21:09.948613+0000","last_unstale":"2024-03-29T08:21:09.950524+0000","last_undegraded":"2024-03-29T08:21:09.950524+0000","last_fullsized":"2024-03-29T08:21:09.950524+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:00.619680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938522+0000","last_change":"2024-03-29T08:21:09.938522+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938522+0000","last_undegraded":"2024-03-29T08:21:09.938522+0000","last_fullsized":"2024-03-29T08:21:09.938522+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:01:48.373155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.947564+0000","last_change":"2024-03-29T08:21:09.946582+0000","last_active":"2024-03-29T08:21:09.947564+0000","last_peered":"2024-03-29T08:21:09.947564+0000","last_clean":"2024-03-29T08:21:09.947564+0000","last_became_active":"2024-03-29T08:21:09.946276+0000","last_became_peered":"2024-03-29T08:21:09.946276+0000","last_unstale":"2024-03-29T08:21:09.947564+0000","last_undegraded":"2024-03-29T08:21:09.947564+0000","last_fullsized":"2024-03-29T08:21:09.947564+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:01:32.117594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.932610+0000","last_change":"2024-03-29T08:21:09.932610+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.932610+0000","last_undegraded":"2024-03-29T08:21:09.932610+0000","last_fullsized":"2024-03-29T08:21:09.932610+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:11:50.905254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.955654+0000","last_change":"2024-03-29T08:21:09.953517+0000","last_active":"2024-03-29T08:21:09.955654+0000","last_peered":"2024-03-29T08:21:09.955654+0000","last_clean":"2024-03-29T08:21:09.955654+0000","last_became_active":"2024-03-29T08:21:09.953205+0000","last_became_peered":"2024-03-29T08:21:09.953205+0000","last_unstale":"2024-03-29T08:21:09.955654+0000","last_undegraded":"2024-03-29T08:21:09.955654+0000","last_fullsized":"2024-03-29T08:21:09.955654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:11:14.860644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936862+0000","last_change":"2024-03-29T08:21:09.936862+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936862+0000","last_undegraded":"2024-03-29T08:21:09.936862+0000","last_fullsized":"2024-03-29T08:21:09.936862+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:15.253098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.957279+0000","last_change":"2024-03-29T08:21:09.954705+0000","last_active":"2024-03-29T08:21:09.957279+0000","last_peered":"2024-03-29T08:21:09.957279+0000","last_clean":"2024-03-29T08:21:09.957279+0000","last_became_active":"2024-03-29T08:21:09.954314+0000","last_became_peered":"2024-03-29T08:21:09.954314+0000","last_unstale":"2024-03-29T08:21:09.957279+0000","last_undegraded":"2024-03-29T08:21:09.957279+0000","last_fullsized":"2024-03-29T08:21:09.957279+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:24:09.856034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936946+0000","last_change":"2024-03-29T08:21:09.936946+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936946+0000","last_undegraded":"2024-03-29T08:21:09.936946+0000","last_fullsized":"2024-03-29T08:21:09.936946+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:05:09.416516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.934157+0000","last_change":"2024-03-29T08:21:08.934157+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.934157+0000","last_undegraded":"2024-03-29T08:21:08.934157+0000","last_fullsized":"2024-03-29T08:21:08.934157+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:50:03.760523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.941996+0000","last_change":"2024-03-29T08:21:09.941996+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.941996+0000","last_undegraded":"2024-03-29T08:21:09.941996+0000","last_fullsized":"2024-03-29T08:21:09.941996+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:50:48.557652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.948833+0000","last_change":"2024-03-29T08:21:09.947148+0000","last_active":"2024-03-29T08:21:09.948833+0000","last_peered":"2024-03-29T08:21:09.948833+0000","last_clean":"2024-03-29T08:21:09.948833+0000","last_became_active":"2024-03-29T08:21:09.946610+0000","last_became_peered":"2024-03-29T08:21:09.946610+0000","last_unstale":"2024-03-29T08:21:09.948833+0000","last_undegraded":"2024-03-29T08:21:09.948833+0000","last_fullsized":"2024-03-29T08:21:09.948833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:51.605750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.932484+0000","last_change":"2024-03-29T08:21:09.932484+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.932484+0000","last_undegraded":"2024-03-29T08:21:09.932484+0000","last_fullsized":"2024-03-29T08:21:09.932484+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:37:31.250178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.936989+0000","last_change":"2024-03-29T08:21:08.936989+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.936989+0000","last_undegraded":"2024-03-29T08:21:08.936989+0000","last_fullsized":"2024-03-29T08:21:08.936989+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:26:31.314621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937474+0000","last_change":"2024-03-29T08:21:09.937474+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937474+0000","last_undegraded":"2024-03-29T08:21:09.937474+0000","last_fullsized":"2024-03-29T08:21:09.937474+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:24:10.858914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956169+0000","last_change":"2024-03-29T08:21:09.953120+0000","last_active":"2024-03-29T08:21:09.956169+0000","last_peered":"2024-03-29T08:21:09.956169+0000","last_clean":"2024-03-29T08:21:09.956169+0000","last_became_active":"2024-03-29T08:21:09.952860+0000","last_became_peered":"2024-03-29T08:21:09.952860+0000","last_unstale":"2024-03-29T08:21:09.956169+0000","last_undegraded":"2024-03-29T08:21:09.956169+0000","last_fullsized":"2024-03-29T08:21:09.956169+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:38.455763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.934110+0000","last_change":"2024-03-29T08:21:08.934110+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.934110+0000","last_undegraded":"2024-03-29T08:21:08.934110+0000","last_fullsized":"2024-03-29T08:21:08.934110+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:05:43.897022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.948860+0000","last_change":"2024-03-29T08:21:09.947277+0000","last_active":"2024-03-29T08:21:09.948860+0000","last_peered":"2024-03-29T08:21:09.948860+0000","last_clean":"2024-03-29T08:21:09.948860+0000","last_became_active":"2024-03-29T08:21:09.945398+0000","last_became_peered":"2024-03-29T08:21:09.945398+0000","last_unstale":"2024-03-29T08:21:09.948860+0000","last_undegraded":"2024-03-29T08:21:09.948860+0000","last_fullsized":"2024-03-29T08:21:09.948860+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:36:29.788331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.935957+0000","last_change":"2024-03-29T08:21:09.935957+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.935957+0000","last_undegraded":"2024-03-29T08:21:09.935957+0000","last_fullsized":"2024-03-29T08:21:09.935957+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:18:28.430663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.957304+0000","last_change":"2024-03-29T08:21:09.955367+0000","last_active":"2024-03-29T08:21:09.957304+0000","last_peered":"2024-03-29T08:21:09.957304+0000","last_clean":"2024-03-29T08:21:09.957304+0000","last_became_active":"2024-03-29T08:21:09.954965+0000","last_became_peered":"2024-03-29T08:21:09.954965+0000","last_unstale":"2024-03-29T08:21:09.957304+0000","last_undegraded":"2024-03-29T08:21:09.957304+0000","last_fullsized":"2024-03-29T08:21:09.957304+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:57.828588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936065+0000","last_change":"2024-03-29T08:21:09.936065+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936065+0000","last_undegraded":"2024-03-29T08:21:09.936065+0000","last_fullsized":"2024-03-29T08:21:09.936065+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:01.741332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.952538+0000","last_change":"2024-03-29T08:21:09.948192+0000","last_active":"2024-03-29T08:21:09.952538+0000","last_peered":"2024-03-29T08:21:09.952538+0000","last_clean":"2024-03-29T08:21:09.952538+0000","last_became_active":"2024-03-29T08:21:09.947926+0000","last_became_peered":"2024-03-29T08:21:09.947926+0000","last_unstale":"2024-03-29T08:21:09.952538+0000","last_undegraded":"2024-03-29T08:21:09.952538+0000","last_fullsized":"2024-03-29T08:21:09.952538+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:33:58.392690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936565+0000","last_change":"2024-03-29T08:21:09.936565+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936565+0000","last_undegraded":"2024-03-29T08:21:09.936565+0000","last_fullsized":"2024-03-29T08:21:09.936565+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:22:08.727894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.950245+0000","last_change":"2024-03-29T08:21:09.947847+0000","last_active":"2024-03-29T08:21:09.950245+0000","last_peered":"2024-03-29T08:21:09.950245+0000","last_clean":"2024-03-29T08:21:09.950245+0000","last_became_active":"2024-03-29T08:21:09.946845+0000","last_became_peered":"2024-03-29T08:21:09.946845+0000","last_unstale":"2024-03-29T08:21:09.950245+0000","last_undegraded":"2024-03-29T08:21:09.950245+0000","last_fullsized":"2024-03-29T08:21:09.950245+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:19:13.916686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.943473+0000","last_change":"2024-03-29T08:21:09.938352+0000","last_active":"2024-03-29T08:21:09.943473+0000","last_peered":"2024-03-29T08:21:09.943473+0000","last_clean":"2024-03-29T08:21:09.943473+0000","last_became_active":"2024-03-29T08:21:09.937930+0000","last_became_peered":"2024-03-29T08:21:09.937930+0000","last_unstale":"2024-03-29T08:21:09.943473+0000","last_undegraded":"2024-03-29T08:21:09.943473+0000","last_fullsized":"2024-03-29T08:21:09.943473+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:23.163706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938692+0000","last_change":"2024-03-29T08:21:09.938692+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938692+0000","last_undegraded":"2024-03-29T08:21:09.938692+0000","last_fullsized":"2024-03-29T08:21:09.938692+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:44.680234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956168+0000","last_change":"2024-03-29T08:21:09.953317+0000","last_active":"2024-03-29T08:21:09.956168+0000","last_peered":"2024-03-29T08:21:09.956168+0000","last_clean":"2024-03-29T08:21:09.956168+0000","last_became_active":"2024-03-29T08:21:09.953068+0000","last_became_peered":"2024-03-29T08:21:09.953068+0000","last_unstale":"2024-03-29T08:21:09.956168+0000","last_undegraded":"2024-03-29T08:21:09.956168+0000","last_fullsized":"2024-03-29T08:21:09.956168+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:39.840479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.941589+0000","last_change":"2024-03-29T08:21:09.941589+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.941589+0000","last_undegraded":"2024-03-29T08:21:09.941589+0000","last_fullsized":"2024-03-29T08:21:09.941589+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:40.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.952730+0000","last_change":"2024-03-29T08:21:09.950160+0000","last_active":"2024-03-29T08:21:09.952730+0000","last_peered":"2024-03-29T08:21:09.952730+0000","last_clean":"2024-03-29T08:21:09.952730+0000","last_became_active":"2024-03-29T08:21:09.949853+0000","last_became_peered":"2024-03-29T08:21:09.949853+0000","last_unstale":"2024-03-29T08:21:09.952730+0000","last_undegraded":"2024-03-29T08:21:09.952730+0000","last_fullsized":"2024-03-29T08:21:09.952730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:59:23.381718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.947411+0000","last_change":"2024-03-29T08:21:09.945534+0000","last_active":"2024-03-29T08:21:09.947411+0000","last_peered":"2024-03-29T08:21:09.947411+0000","last_clean":"2024-03-29T08:21:09.947411+0000","last_became_active":"2024-03-29T08:21:09.943748+0000","last_became_peered":"2024-03-29T08:21:09.943748+0000","last_unstale":"2024-03-29T08:21:09.947411+0000","last_undegraded":"2024-03-29T08:21:09.947411+0000","last_fullsized":"2024-03-29T08:21:09.947411+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:05:06.965072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936902+0000","last_change":"2024-03-29T08:21:09.936902+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936902+0000","last_undegraded":"2024-03-29T08:21:09.936902+0000","last_fullsized":"2024-03-29T08:21:09.936902+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:38:18.883117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.934050+0000","last_change":"2024-03-29T08:21:08.934050+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.934050+0000","last_undegraded":"2024-03-29T08:21:08.934050+0000","last_fullsized":"2024-03-29T08:21:08.934050+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:00:36.010161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.955570+0000","last_change":"2024-03-29T08:21:09.953812+0000","last_active":"2024-03-29T08:21:09.955570+0000","last_peered":"2024-03-29T08:21:09.955570+0000","last_clean":"2024-03-29T08:21:09.955570+0000","last_became_active":"2024-03-29T08:21:09.953475+0000","last_became_peered":"2024-03-29T08:21:09.953475+0000","last_unstale":"2024-03-29T08:21:09.955570+0000","last_undegraded":"2024-03-29T08:21:09.955570+0000","last_fullsized":"2024-03-29T08:21:09.955570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:05:21.462217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":196,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-03-29T08:21:08.935647+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:21:08.935647+0000","last_peered":"2024-03-29T08:21:08.935647+0000","last_clean":"2024-03-29T08:21:08.935647+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:21:08.935647+0000","last_undegraded":"2024-03-29T08:21:08.935647+0000","last_fullsized":"2024-03-29T08:21:08.935647+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":7},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":293626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67600000000000005,"15min":0.65200000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93400000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.69999999999999996,"15min":0.67600000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1200000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.68700000000000006}]},{"osd":1,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.67100000000000004,"15min":0.64800000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.66800000000000004,"15min":0.63400000000000001},"min":{"1min":0.52900000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.0189999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.82599999999999996}]},{"osd":2,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.66200000000000003,"15min":0.63300000000000001},"min":{"1min":0.55900000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.0089999999999999,"5min":1.085,"15min":1.085},"last":0.754},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.69299999999999995,"15min":0.66500000000000004},"min":{"1min":0.58499999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1000000000000001,"5min":1.288,"15min":1.288},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71099999999999997,"15min":0.69799999999999995},"min":{"1min":0.57899999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92800000000000005,"5min":1.018,"15min":1.018},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.745,"15min":0.73099999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1659999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.73499999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83499999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70199999999999996,"15min":0.71999999999999997},"min":{"1min":0.46100000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0009999999999999,"5min":1.123,"15min":1.123},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.95799999999999996,"5min":1.056,"15min":1.056},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.46300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73399999999999999,"15min":0.72999999999999998},"min":{"1min":0.48999999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.755},"min":{"1min":0.52300000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.84199999999999997}]}]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":293626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.66000000000000003,"15min":0.63300000000000001},"min":{"1min":0.54000000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70499999999999996,"15min":0.68500000000000005},"min":{"1min":0.57899999999999996,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.71399999999999997}]},{"osd":1,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.64100000000000001,"15min":0.61499999999999999},"min":{"1min":0.58599999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.64800000000000002,"15min":0.627},"min":{"1min":0.51600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.70299999999999996,"15min":0.67500000000000004},"min":{"1min":0.52300000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.54300000000000004,"5min":0.435,"15min":0.435},"max":{"1min":0.91200000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.81100000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71399999999999997,"15min":0.69999999999999996},"min":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94099999999999995,"5min":1.109,"15min":1.109},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.53200000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.84499999999999997}]},{"osd":4,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0620000000000001,"5min":1.556,"15min":1.556},"last":1.02},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70499999999999996,"15min":0.73699999999999999},"min":{"1min":0.44400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.0429999999999999,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.0429999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67800000000000005,"15min":0.70299999999999996},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.71599999999999997,"15min":0.73399999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.86299999999999999}]},{"osd":7,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.74199999999999999,"15min":0.76700000000000002},"min":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.92300000000000004,"5min":0.998,"15min":0.998},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.753,"15min":0.77900000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":1.109}]}]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":293626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.50800000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93100000000000005,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72699999999999998,"15min":0.74399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95199999999999996,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.73599999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.76500000000000001},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97799999999999998,"5min":2.524,"15min":2.524},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.54900000000000004,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93999999999999995,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.83699999999999997}]},{"osd":2,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84499999999999997,"5min":2.363,"15min":2.363},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0029999999999999,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.76900000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.747,"15min":0.75700000000000001},"min":{"1min":0.54700000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90000000000000002,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.753},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.81699999999999995},"min":{"1min":0.625,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.96199999999999997,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.86899999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68200000000000005,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88100000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.441,"5min":0.434,"15min":0.434},"max":{"1min":0.82199999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.65900000000000003}]},{"osd":6,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86799999999999999,"5min":0.996,"15min":0.996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72599999999999998,"15min":0.74399999999999999},"min":{"1min":0.52300000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.91600000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.63800000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71699999999999997,"15min":0.73799999999999999},"min":{"1min":0.51100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90900000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.46999999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92100000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.68999999999999995}]}]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":724992,"data_stored":299901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.46300000000000002},"min":{"1min":0.33700000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.72699999999999998,"5min":1.123,"15min":1.123},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52000000000000002,"15min":0.52400000000000002},"min":{"1min":0.432,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.871,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.437}]},{"osd":1,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44700000000000001,"15min":0.46100000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64800000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.51500000000000001,"15min":0.53400000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.68200000000000005,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.28599999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54700000000000004,"15min":0.56999999999999995},"min":{"1min":0.35999999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.70599999999999996,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63700000000000001,"15min":0.65900000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79700000000000004,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.41799999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.77400000000000002,"15min":0.79600000000000004},"min":{"1min":0.52100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1950000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.75900000000000001,"15min":0.77900000000000003},"min":{"1min":0.55700000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.004,"5min":1.391,"15min":1.391},"last":0.64600000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.80800000000000005},"min":{"1min":0.627,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84299999999999997,"15min":0.86699999999999999},"min":{"1min":0.64500000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6599999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.81000000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.79600000000000004,"15min":0.83599999999999997},"min":{"1min":0.51500000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.008,"5min":1.669,"15min":1.669},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.80300000000000005,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.212,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.76200000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82999999999999996,"15min":0.86399999999999999},"min":{"1min":0.54900000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4279999999999999,"5min":1.887,"15min":1.887},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.82099999999999995,"15min":0.86099999999999999},"min":{"1min":0.59899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98299999999999998,"5min":2.044,"15min":2.044},"last":0.70299999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":712704,"data_stored":293626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.48899999999999999,"15min":0.501},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69799999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66300000000000003,"5min":0.751,"15min":0.751},"last":0.65900000000000003}]},{"osd":1,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.34499999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72999999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.375},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.48999999999999999,"15min":0.50800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72499999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.39900000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75900000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40400000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69299999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.61599999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52600000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.052,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.75900000000000001,"15min":0.76600000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82999999999999996,"15min":0.84199999999999997},"min":{"1min":0.64600000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.081,"5min":1.306,"15min":1.306},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83999999999999997,"15min":0.85999999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.28,"5min":1.526,"15min":1.526},"last":0.55500000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.77700000000000002,"15min":0.77900000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.55600000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.121,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.53400000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.51000000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":0.79800000000000004},"min":{"1min":0.56499999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.71899999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1306624,"data_stored":883994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.56799999999999995,"15min":0.63500000000000001},"min":{"1min":0.33400000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.54800000000000004,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.60099999999999998,"15min":0.65300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71799999999999997,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.49399999999999999,"15min":0.50600000000000001},"min":{"1min":0.375,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73299999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53000000000000003,"15min":0.53600000000000003},"min":{"1min":0.39100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67000000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.435}]},{"osd":3,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.86099999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.75900000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.41299999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.53000000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95599999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.746},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.76300000000000001,"15min":0.78900000000000003},"min":{"1min":0.58099999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95199999999999996,"5min":1.23,"15min":1.23},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76100000000000001,"15min":0.77900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.89500000000000002,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.56499999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.873,"5min":1.107,"15min":1.107},"last":0.67800000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76000000000000001,"15min":0.76800000000000002},"min":{"1min":0.57399999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":1.026,"15min":1.026},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.81799999999999995,"15min":0.85699999999999998},"min":{"1min":0.55800000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.83199999999999996,"5min":1.196,"15min":1.196},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.78700000000000003,"15min":0.80000000000000004},"min":{"1min":0.57499999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.92900000000000005,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.78700000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91600000000000004,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.66100000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":724992,"data_stored":299901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.55300000000000005,"15min":0.61299999999999999},"min":{"1min":0.313,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.71099999999999997,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.753,"15min":0.873},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.73799999999999999,"5min":8.0419999999999998,"15min":8.0419999999999998},"last":0.34000000000000002}]},{"osd":2,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.53800000000000003,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.58899999999999997,"5min":1.258,"15min":1.258},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.67500000000000004,"15min":0.747},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63800000000000001,"5min":4.907,"15min":4.907},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.753,"15min":0.872},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.77000000000000002,"5min":7.9569999999999999,"15min":7.9569999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.624,"15min":0.66600000000000004},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.65100000000000002,"5min":1.397,"15min":1.397},"last":0.44900000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":1.002},"min":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91700000000000004,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.91500000000000004,"15min":0.98099999999999998},"min":{"1min":0.66700000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.97999999999999998,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.77000000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.879,"15min":0.93999999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.016,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.83299999999999996,"15min":0.871},"min":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.89900000000000002,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.754}]},{"osd":6,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.83899999999999997,"15min":0.87},"min":{"1min":0.64500000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95699999999999996,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.92900000000000005,"15min":0.999},"min":{"1min":0.623,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.097,"5min":4.4100000000000001,"15min":4.4100000000000001},"last":0.68100000000000005}]},{"osd":7,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.94599999999999995,"15min":1.0209999999999999},"min":{"1min":0.64100000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.038,"5min":4.5209999999999999,"15min":4.5209999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.92100000000000004,"15min":1},"min":{"1min":0.57699999999999996,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.099,"5min":4.4420000000000002,"15min":4.4420000000000002},"last":0.57699999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1306624,"data_stored":883994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.497,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89400000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.497}]},{"osd":1,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71599999999999997,"15min":0.73799999999999999},"min":{"1min":0.51700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.876,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67800000000000005,"15min":0.67700000000000005},"min":{"1min":0.44900000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89000000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.443,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84299999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.443},{"interface":"front","average":{"1min":0.748,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.59199999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90700000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.88600000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.54800000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91800000000000004,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.64500000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.90500000000000003,"5min":1.954,"15min":1.954},"last":0.80800000000000005}]},{"osd":4,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.50800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.51400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88700000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.82899999999999996}]},{"osd":6,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66700000000000004,"15min":0.68200000000000005},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82399999999999995,"5min":1.071,"15min":1.071},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.70099999999999996},"min":{"1min":0.53600000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.878,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92900000000000005,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.74099999999999999,"15min":0.75800000000000001},"min":{"1min":0.56799999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95199999999999996,"5min":1.04,"15min":1.04},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:21:14.842 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:21:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.015+0000 7f2d73600640 1 Processor -- start 2024-03-29T08:21:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.015+0000 7f2d73600640 1 -- start start 2024-03-29T08:21:15.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.019+0000 7f2d73600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d6c1569a0 0x7f2d6c06c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.019+0000 7f2d73600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2d6c06d910 0x7f2d6c0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.019+0000 7f2d73600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c15d7c0 con 0x7f2d6c06dd70 2024-03-29T08:21:15.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.019+0000 7f2d73600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6c0b21d0 con 0x7f2d6c06d910 2024-03-29T08:21:15.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.019+0000 7f2d73600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c15cc40 con 0x7f2d6c1569a0 2024-03-29T08:21:15.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.019+0000 7f2d6be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2d6c06d910 0x7f2d6c0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.019+0000 7f2d6be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2d6c06d910 0x7f2d6c0701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54396/0 (socket says 172.21.15.46:54396) 2024-03-29T08:21:15.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.019+0000 7f2d6be00640 1 -- 172.21.15.46:0/365609199 learned_addr learned my addr 172.21.15.46:0/365609199 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:15.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.019+0000 7f2d70c00640 1 --2- 172.21.15.46:0/365609199 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d6c1569a0 0x7f2d6c06c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.019+0000 7f2d6be00640 1 -- 172.21.15.46:0/365609199 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d6c1569a0 msgr2=0x7f2d6c06c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:15.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.019+0000 7f2d6be00640 1 --2- 172.21.15.46:0/365609199 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d6c1569a0 0x7f2d6c06c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.019+0000 7f2d6be00640 1 -- 172.21.15.46:0/365609199 >> v1:172.21.15.73:6789/0 conn(0x7f2d6c06dd70 legacy=0x7f2d6c1596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.019+0000 7f2d6be00640 1 -- 172.21.15.46:0/365609199 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c15cf20 con 0x7f2d6c06d910 2024-03-29T08:21:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.023+0000 7f2d6be00640 1 --2- 172.21.15.46:0/365609199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d6c06d910 0x7f2d6c0701c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009870 tx=0x7f2d5801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732290af78095f93 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:15.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.023+0000 7f2d6b400640 1 -- 172.21.15.46:0/365609199 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d580ad660 con 0x7f2d6c06d910 2024-03-29T08:21:15.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.023+0000 7f2d6b400640 1 -- 172.21.15.46:0/365609199 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d580ad7c0 con 0x7f2d6c06d910 2024-03-29T08:21:15.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.023+0000 7f2d6b400640 1 -- 172.21.15.46:0/365609199 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d580adb50 con 0x7f2d6c06d910 2024-03-29T08:21:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.023+0000 7f2d73600640 1 -- 172.21.15.46:0/365609199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d6c06d910 msgr2=0x7f2d6c0701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.023+0000 7f2d73600640 1 --2- 172.21.15.46:0/365609199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d6c06d910 0x7f2d6c0701c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009870 tx=0x7f2d5801c560 comp rx=0 tx=0).stop 2024-03-29T08:21:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.023+0000 7f2d73600640 1 -- 172.21.15.46:0/365609199 shutdown_connections 2024-03-29T08:21:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.023+0000 7f2d73600640 1 --2- 172.21.15.46:0/365609199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d6c06d910 0x7f2d6c0701c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.023+0000 7f2d73600640 1 --2- 172.21.15.46:0/365609199 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d6c1569a0 0x7f2d6c06c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.023+0000 7f2d73600640 1 -- 172.21.15.46:0/365609199 >> 172.21.15.46:0/365609199 conn(0x7f2d6c066110 msgr2=0x7f2d6c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.023+0000 7f2d73600640 1 -- 172.21.15.46:0/365609199 shutdown_connections 2024-03-29T08:21:15.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.023+0000 7f2d73600640 1 -- 172.21.15.46:0/365609199 wait complete. 2024-03-29T08:21:15.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.023+0000 7f2d73600640 1 Processor -- start 2024-03-29T08:21:15.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.023+0000 7f2d73600640 1 -- start start 2024-03-29T08:21:15.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d73600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d6c06d910 0x7f2d6c14f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d73600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d6c06dd70 0x7f2d6c158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d73600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d6c1569a0 0x7f2d6c151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d73600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c158360 con 0x7f2d6c1569a0 2024-03-29T08:21:15.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d73600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c1505a0 con 0x7f2d6c06d910 2024-03-29T08:21:15.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d73600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c14f870 con 0x7f2d6c06dd70 2024-03-29T08:21:15.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d71600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d6c1569a0 0x7f2d6c151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d71600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d6c1569a0 0x7f2d6c151990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54398/0 (socket says 172.21.15.46:54398) 2024-03-29T08:21:15.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d71600640 1 -- 172.21.15.46:0/2258744348 learned_addr learned my addr 172.21.15.46:0/2258744348 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:15.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d70c00640 1 --2- 172.21.15.46:0/2258744348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d6c06d910 0x7f2d6c14f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d6be00640 1 --2- 172.21.15.46:0/2258744348 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d6c06dd70 0x7f2d6c158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d71600640 1 -- 172.21.15.46:0/2258744348 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d6c06dd70 msgr2=0x7f2d6c158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d71600640 1 --2- 172.21.15.46:0/2258744348 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d6c06dd70 0x7f2d6c158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d71600640 1 -- 172.21.15.46:0/2258744348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d6c06d910 msgr2=0x7f2d6c14f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d71600640 1 --2- 172.21.15.46:0/2258744348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d6c06d910 0x7f2d6c14f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d71600640 1 -- 172.21.15.46:0/2258744348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d580094d0 con 0x7f2d6c1569a0 2024-03-29T08:21:15.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d71600640 1 --2- 172.21.15.46:0/2258744348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d6c1569a0 0x7f2d6c151990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400c7a0 tx=0x7f2d6400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:15.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d68c00640 1 -- 172.21.15.46:0/2258744348 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64003b80 con 0x7f2d6c1569a0 2024-03-29T08:21:15.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d68c00640 1 -- 172.21.15.46:0/2258744348 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d64003ce0 con 0x7f2d6c1569a0 2024-03-29T08:21:15.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d68c00640 1 -- 172.21.15.46:0/2258744348 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400aa30 con 0x7f2d6c1569a0 2024-03-29T08:21:15.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d73600640 1 -- 172.21.15.46:0/2258744348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c0afed0 con 0x7f2d6c1569a0 2024-03-29T08:21:15.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d73600640 1 -- 172.21.15.46:0/2258744348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c0afbf0 con 0x7f2d6c1569a0 2024-03-29T08:21:15.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d73600640 1 -- 172.21.15.46:0/2258744348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6c1584a0 con 0x7f2d6c1569a0 2024-03-29T08:21:15.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.027+0000 7f2d68c00640 1 -- 172.21.15.46:0/2258744348 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2d6400ab90 con 0x7f2d6c1569a0 2024-03-29T08:21:15.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.031+0000 7f2d68c00640 1 --2- 172.21.15.46:0/2258744348 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2d4c0666c0 0x7f2d4c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.031+0000 7f2d68c00640 1 -- 172.21.15.46:0/2258744348 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d64083dc0 con 0x7f2d6c1569a0 2024-03-29T08:21:15.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.031+0000 7f2d70c00640 1 --2- 172.21.15.46:0/2258744348 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2d4c0666c0 0x7f2d4c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.031+0000 7f2d70c00640 1 --2- 172.21.15.46:0/2258744348 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2d4c0666c0 0x7f2d4c068b80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0ccce0 tx=0x7f2d60008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:15.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.035+0000 7f2d68c00640 1 -- 172.21.15.46:0/2258744348 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2d6c1584a0 con 0x7f2d6c1569a0 2024-03-29T08:21:15.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.191+0000 7f2d73600640 1 -- 172.21.15.46:0/2258744348 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d6c15a1e0 con 0x7f2d4c0666c0 2024-03-29T08:21:15.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.199+0000 7f2d68c00640 1 -- 172.21.15.46:0/2258744348 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284914 (secure 0 0 0) 0x7f2d600108d0 con 0x7f2d4c0666c0 2024-03-29T08:21:15.206 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:15.208 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:21:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.203+0000 7f2d73600640 1 -- 172.21.15.46:0/2258744348 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2d4c0666c0 msgr2=0x7f2d4c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.203+0000 7f2d73600640 1 --2- 172.21.15.46:0/2258744348 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2d4c0666c0 0x7f2d4c068b80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0ccce0 tx=0x7f2d60008040 comp rx=0 tx=0).stop 2024-03-29T08:21:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.203+0000 7f2d73600640 1 -- 172.21.15.46:0/2258744348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d6c1569a0 msgr2=0x7f2d6c151990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.203+0000 7f2d73600640 1 --2- 172.21.15.46:0/2258744348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d6c1569a0 0x7f2d6c151990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400c7a0 tx=0x7f2d6400cc60 comp rx=0 tx=0).stop 2024-03-29T08:21:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.203+0000 7f2d73600640 1 -- 172.21.15.46:0/2258744348 shutdown_connections 2024-03-29T08:21:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.203+0000 7f2d73600640 1 --2- 172.21.15.46:0/2258744348 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2d4c0666c0 0x7f2d4c068b80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.203+0000 7f2d73600640 1 --2- 172.21.15.46:0/2258744348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d6c1569a0 0x7f2d6c151990 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.203+0000 7f2d73600640 1 --2- 172.21.15.46:0/2258744348 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d6c06dd70 0x7f2d6c158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.203+0000 7f2d73600640 1 --2- 172.21.15.46:0/2258744348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d6c06d910 0x7f2d6c14f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.203+0000 7f2d73600640 1 -- 172.21.15.46:0/2258744348 >> 172.21.15.46:0/2258744348 conn(0x7f2d6c066110 msgr2=0x7f2d6c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:15.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.203+0000 7f2d73600640 1 -- 172.21.15.46:0/2258744348 shutdown_connections 2024-03-29T08:21:15.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.203+0000 7f2d73600640 1 -- 172.21.15.46:0/2258744348 wait complete. 2024-03-29T08:21:15.227 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-03-29T08:21:14.084517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243396,"kb_used_data":6752,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754731324,"statfs":{"total":773094113280,"available":772844875776,"internally_reserved":0,"allocated":6914048,"data_stored":3542294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":242208313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.157425"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.941725+0000","last_change":"2024-03-29T08:21:09.941725+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.941725+0000","last_undegraded":"2024-03-29T08:21:09.941725+0000","last_fullsized":"2024-03-29T08:21:09.941725+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:16:15.767217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936961+0000","last_change":"2024-03-29T08:21:09.936961+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936961+0000","last_undegraded":"2024-03-29T08:21:09.936961+0000","last_fullsized":"2024-03-29T08:21:09.936961+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:05:22.465098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938613+0000","last_change":"2024-03-29T08:21:09.938613+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938613+0000","last_undegraded":"2024-03-29T08:21:09.938613+0000","last_fullsized":"2024-03-29T08:21:09.938613+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:47:48.297292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.933020+0000","last_change":"2024-03-29T08:21:09.933020+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.933020+0000","last_undegraded":"2024-03-29T08:21:09.933020+0000","last_fullsized":"2024-03-29T08:21:09.933020+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:51.414360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936757+0000","last_change":"2024-03-29T08:21:09.936757+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936757+0000","last_undegraded":"2024-03-29T08:21:09.936757+0000","last_fullsized":"2024-03-29T08:21:09.936757+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:36:29.239728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939144+0000","last_change":"2024-03-29T08:21:09.939144+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939144+0000","last_undegraded":"2024-03-29T08:21:09.939144+0000","last_fullsized":"2024-03-29T08:21:09.939144+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:35:20.420999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.933702+0000","last_change":"2024-03-29T08:21:09.933702+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.933702+0000","last_undegraded":"2024-03-29T08:21:09.933702+0000","last_fullsized":"2024-03-29T08:21:09.933702+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:30:03.234601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939490+0000","last_change":"2024-03-29T08:21:09.939490+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939490+0000","last_undegraded":"2024-03-29T08:21:09.939490+0000","last_fullsized":"2024-03-29T08:21:09.939490+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:40.843359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.940289+0000","last_change":"2024-03-29T08:21:09.940289+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.940289+0000","last_undegraded":"2024-03-29T08:21:09.940289+0000","last_fullsized":"2024-03-29T08:21:09.940289+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:54.034461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.934375+0000","last_change":"2024-03-29T08:21:09.934375+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.934375+0000","last_undegraded":"2024-03-29T08:21:09.934375+0000","last_fullsized":"2024-03-29T08:21:09.934375+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:58.105380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.940141+0000","last_change":"2024-03-29T08:21:09.940141+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.940141+0000","last_undegraded":"2024-03-29T08:21:09.940141+0000","last_fullsized":"2024-03-29T08:21:09.940141+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:20:51.330512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.942430+0000","last_change":"2024-03-29T08:21:09.942430+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.942430+0000","last_undegraded":"2024-03-29T08:21:09.942430+0000","last_fullsized":"2024-03-29T08:21:09.942430+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:35:20.420999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939640+0000","last_change":"2024-03-29T08:21:09.939640+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939640+0000","last_undegraded":"2024-03-29T08:21:09.939640+0000","last_fullsized":"2024-03-29T08:21:09.939640+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:49:07.458847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.940293+0000","last_change":"2024-03-29T08:21:09.940293+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.940293+0000","last_undegraded":"2024-03-29T08:21:09.940293+0000","last_fullsized":"2024-03-29T08:21:09.940293+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:39.458643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938066+0000","last_change":"2024-03-29T08:21:09.938066+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938066+0000","last_undegraded":"2024-03-29T08:21:09.938066+0000","last_fullsized":"2024-03-29T08:21:09.938066+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:01:52.376061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.941263+0000","last_change":"2024-03-29T08:21:09.941263+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.941263+0000","last_undegraded":"2024-03-29T08:21:09.941263+0000","last_fullsized":"2024-03-29T08:21:09.941263+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:39.458643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939231+0000","last_change":"2024-03-29T08:21:09.939231+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939231+0000","last_undegraded":"2024-03-29T08:21:09.939231+0000","last_fullsized":"2024-03-29T08:21:09.939231+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:33:19.998323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.942709+0000","last_change":"2024-03-29T08:21:09.942709+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.942709+0000","last_undegraded":"2024-03-29T08:21:09.942709+0000","last_fullsized":"2024-03-29T08:21:09.942709+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:58:48.804311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.934214+0000","last_change":"2024-03-29T08:21:09.934214+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.934214+0000","last_undegraded":"2024-03-29T08:21:09.934214+0000","last_fullsized":"2024-03-29T08:21:09.934214+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:01.741332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.942033+0000","last_change":"2024-03-29T08:21:09.942033+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.942033+0000","last_undegraded":"2024-03-29T08:21:09.942033+0000","last_fullsized":"2024-03-29T08:21:09.942033+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:44.680234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.934012+0000","last_change":"2024-03-29T08:21:09.934012+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.934012+0000","last_undegraded":"2024-03-29T08:21:09.934012+0000","last_fullsized":"2024-03-29T08:21:09.934012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:15.253098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.933212+0000","last_change":"2024-03-29T08:21:09.933212+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.933212+0000","last_undegraded":"2024-03-29T08:21:09.933212+0000","last_fullsized":"2024-03-29T08:21:09.933212+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:25:00.595704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938923+0000","last_change":"2024-03-29T08:21:09.938923+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938923+0000","last_undegraded":"2024-03-29T08:21:09.938923+0000","last_fullsized":"2024-03-29T08:21:09.938923+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:58.831468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937337+0000","last_change":"2024-03-29T08:21:09.937337+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937337+0000","last_undegraded":"2024-03-29T08:21:09.937337+0000","last_fullsized":"2024-03-29T08:21:09.937337+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:59:24.384598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.943515+0000","last_change":"2024-03-29T08:21:09.943515+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.943515+0000","last_undegraded":"2024-03-29T08:21:09.943515+0000","last_fullsized":"2024-03-29T08:21:09.943515+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:53.450623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936629+0000","last_change":"2024-03-29T08:21:09.936629+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936629+0000","last_undegraded":"2024-03-29T08:21:09.936629+0000","last_fullsized":"2024-03-29T08:21:09.936629+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:49.605189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.932435+0000","last_change":"2024-03-29T08:21:09.932435+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.932435+0000","last_undegraded":"2024-03-29T08:21:09.932435+0000","last_fullsized":"2024-03-29T08:21:09.932435+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:45.698306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.943149+0000","last_change":"2024-03-29T08:21:09.943149+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.943149+0000","last_undegraded":"2024-03-29T08:21:09.943149+0000","last_fullsized":"2024-03-29T08:21:09.943149+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:01:19.390653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.934024+0000","last_change":"2024-03-29T08:21:09.934024+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.934024+0000","last_undegraded":"2024-03-29T08:21:09.934024+0000","last_fullsized":"2024-03-29T08:21:09.934024+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:54.034461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939827+0000","last_change":"2024-03-29T08:21:09.939827+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939827+0000","last_undegraded":"2024-03-29T08:21:09.939827+0000","last_fullsized":"2024-03-29T08:21:09.939827+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:53.450623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956239+0000","last_change":"2024-03-29T08:21:09.953242+0000","last_active":"2024-03-29T08:21:09.956239+0000","last_peered":"2024-03-29T08:21:09.956239+0000","last_clean":"2024-03-29T08:21:09.956239+0000","last_became_active":"2024-03-29T08:21:09.952978+0000","last_became_peered":"2024-03-29T08:21:09.952978+0000","last_unstale":"2024-03-29T08:21:09.956239+0000","last_undegraded":"2024-03-29T08:21:09.956239+0000","last_fullsized":"2024-03-29T08:21:09.956239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:43.677354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937754+0000","last_change":"2024-03-29T08:21:09.937754+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937754+0000","last_undegraded":"2024-03-29T08:21:09.937754+0000","last_fullsized":"2024-03-29T08:21:09.937754+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:40.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.957328+0000","last_change":"2024-03-29T08:21:09.955717+0000","last_active":"2024-03-29T08:21:09.957328+0000","last_peered":"2024-03-29T08:21:09.957328+0000","last_clean":"2024-03-29T08:21:09.957328+0000","last_became_active":"2024-03-29T08:21:09.955450+0000","last_became_peered":"2024-03-29T08:21:09.955450+0000","last_unstale":"2024-03-29T08:21:09.957328+0000","last_undegraded":"2024-03-29T08:21:09.957328+0000","last_fullsized":"2024-03-29T08:21:09.957328+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:47:40.877803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.952711+0000","last_change":"2024-03-29T08:21:09.950482+0000","last_active":"2024-03-29T08:21:09.952711+0000","last_peered":"2024-03-29T08:21:09.952711+0000","last_clean":"2024-03-29T08:21:09.952711+0000","last_became_active":"2024-03-29T08:21:09.950190+0000","last_became_peered":"2024-03-29T08:21:09.950190+0000","last_unstale":"2024-03-29T08:21:09.952711+0000","last_undegraded":"2024-03-29T08:21:09.952711+0000","last_fullsized":"2024-03-29T08:21:09.952711+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:07:20.233898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938835+0000","last_change":"2024-03-29T08:21:09.938835+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938835+0000","last_undegraded":"2024-03-29T08:21:09.938835+0000","last_fullsized":"2024-03-29T08:21:09.938835+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:21:23.293798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.944401+0000","last_change":"2024-03-29T08:21:09.942997+0000","last_active":"2024-03-29T08:21:09.944401+0000","last_peered":"2024-03-29T08:21:09.944401+0000","last_clean":"2024-03-29T08:21:09.944401+0000","last_became_active":"2024-03-29T08:21:09.942730+0000","last_became_peered":"2024-03-29T08:21:09.942730+0000","last_unstale":"2024-03-29T08:21:09.944401+0000","last_undegraded":"2024-03-29T08:21:09.944401+0000","last_fullsized":"2024-03-29T08:21:09.944401+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:42:48.592612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.947598+0000","last_change":"2024-03-29T08:21:09.944500+0000","last_active":"2024-03-29T08:21:09.947598+0000","last_peered":"2024-03-29T08:21:09.947598+0000","last_clean":"2024-03-29T08:21:09.947598+0000","last_became_active":"2024-03-29T08:21:09.944236+0000","last_became_peered":"2024-03-29T08:21:09.944236+0000","last_unstale":"2024-03-29T08:21:09.947598+0000","last_undegraded":"2024-03-29T08:21:09.947598+0000","last_fullsized":"2024-03-29T08:21:09.947598+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:06:08.420341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.935582+0000","last_change":"2024-03-29T08:21:09.935582+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.935582+0000","last_undegraded":"2024-03-29T08:21:09.935582+0000","last_fullsized":"2024-03-29T08:21:09.935582+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:05:09.416516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.952308+0000","last_change":"2024-03-29T08:21:09.946818+0000","last_active":"2024-03-29T08:21:09.952308+0000","last_peered":"2024-03-29T08:21:09.952308+0000","last_clean":"2024-03-29T08:21:09.952308+0000","last_became_active":"2024-03-29T08:21:09.946555+0000","last_became_peered":"2024-03-29T08:21:09.946555+0000","last_unstale":"2024-03-29T08:21:09.952308+0000","last_undegraded":"2024-03-29T08:21:09.952308+0000","last_fullsized":"2024-03-29T08:21:09.952308+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:27:46.452557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937385+0000","last_change":"2024-03-29T08:21:09.937385+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937385+0000","last_undegraded":"2024-03-29T08:21:09.937385+0000","last_fullsized":"2024-03-29T08:21:09.937385+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:58.105380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956350+0000","last_change":"2024-03-29T08:21:09.954162+0000","last_active":"2024-03-29T08:21:09.956350+0000","last_peered":"2024-03-29T08:21:09.956350+0000","last_clean":"2024-03-29T08:21:09.956350+0000","last_became_active":"2024-03-29T08:21:09.953849+0000","last_became_peered":"2024-03-29T08:21:09.953849+0000","last_unstale":"2024-03-29T08:21:09.956350+0000","last_undegraded":"2024-03-29T08:21:09.956350+0000","last_fullsized":"2024-03-29T08:21:09.956350+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:52.803477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937620+0000","last_change":"2024-03-29T08:21:09.937620+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937620+0000","last_undegraded":"2024-03-29T08:21:09.937620+0000","last_fullsized":"2024-03-29T08:21:09.937620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:19:16.700510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956277+0000","last_change":"2024-03-29T08:21:09.954120+0000","last_active":"2024-03-29T08:21:09.956277+0000","last_peered":"2024-03-29T08:21:09.956277+0000","last_clean":"2024-03-29T08:21:09.956277+0000","last_became_active":"2024-03-29T08:21:09.953375+0000","last_became_peered":"2024-03-29T08:21:09.953375+0000","last_unstale":"2024-03-29T08:21:09.956277+0000","last_undegraded":"2024-03-29T08:21:09.956277+0000","last_fullsized":"2024-03-29T08:21:09.956277+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:48.602309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.943037+0000","last_change":"2024-03-29T08:21:09.943037+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.943037+0000","last_undegraded":"2024-03-29T08:21:09.943037+0000","last_fullsized":"2024-03-29T08:21:09.943037+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:24.166586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.953570+0000","last_change":"2024-03-29T08:21:09.951304+0000","last_active":"2024-03-29T08:21:09.953570+0000","last_peered":"2024-03-29T08:21:09.953570+0000","last_clean":"2024-03-29T08:21:09.953570+0000","last_became_active":"2024-03-29T08:21:09.951067+0000","last_became_peered":"2024-03-29T08:21:09.951067+0000","last_unstale":"2024-03-29T08:21:09.953570+0000","last_undegraded":"2024-03-29T08:21:09.953570+0000","last_fullsized":"2024-03-29T08:21:09.953570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:13:08.676082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938641+0000","last_change":"2024-03-29T08:21:09.938641+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938641+0000","last_undegraded":"2024-03-29T08:21:09.938641+0000","last_fullsized":"2024-03-29T08:21:09.938641+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:08:39.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.939701+0000","last_change":"2024-03-29T08:21:08.939701+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.939701+0000","last_undegraded":"2024-03-29T08:21:08.939701+0000","last_fullsized":"2024-03-29T08:21:08.939701+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:08:44.015717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.945733+0000","last_change":"2024-03-29T08:21:09.944508+0000","last_active":"2024-03-29T08:21:09.945733+0000","last_peered":"2024-03-29T08:21:09.945733+0000","last_clean":"2024-03-29T08:21:09.945733+0000","last_became_active":"2024-03-29T08:21:09.944191+0000","last_became_peered":"2024-03-29T08:21:09.944191+0000","last_unstale":"2024-03-29T08:21:09.945733+0000","last_undegraded":"2024-03-29T08:21:09.945733+0000","last_fullsized":"2024-03-29T08:21:09.945733+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:25:36.003763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.932991+0000","last_change":"2024-03-29T08:21:09.932991+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.932991+0000","last_undegraded":"2024-03-29T08:21:09.932991+0000","last_fullsized":"2024-03-29T08:21:09.932991+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:56:39.434467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.955648+0000","last_change":"2024-03-29T08:21:09.953728+0000","last_active":"2024-03-29T08:21:09.955648+0000","last_peered":"2024-03-29T08:21:09.955648+0000","last_clean":"2024-03-29T08:21:09.955648+0000","last_became_active":"2024-03-29T08:21:09.953431+0000","last_became_peered":"2024-03-29T08:21:09.953431+0000","last_unstale":"2024-03-29T08:21:09.955648+0000","last_undegraded":"2024-03-29T08:21:09.955648+0000","last_fullsized":"2024-03-29T08:21:09.955648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:38:17.880237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939449+0000","last_change":"2024-03-29T08:21:09.939449+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939449+0000","last_undegraded":"2024-03-29T08:21:09.939449+0000","last_fullsized":"2024-03-29T08:21:09.939449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:52:34.720375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.934101+0000","last_change":"2024-03-29T08:21:08.934101+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.934101+0000","last_undegraded":"2024-03-29T08:21:08.934101+0000","last_fullsized":"2024-03-29T08:21:08.934101+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:03:13.729828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937954+0000","last_change":"2024-03-29T08:21:09.937954+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937954+0000","last_undegraded":"2024-03-29T08:21:09.937954+0000","last_fullsized":"2024-03-29T08:21:09.937954+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:21:28.741076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.950524+0000","last_change":"2024-03-29T08:21:09.948902+0000","last_active":"2024-03-29T08:21:09.950524+0000","last_peered":"2024-03-29T08:21:09.950524+0000","last_clean":"2024-03-29T08:21:09.950524+0000","last_became_active":"2024-03-29T08:21:09.948613+0000","last_became_peered":"2024-03-29T08:21:09.948613+0000","last_unstale":"2024-03-29T08:21:09.950524+0000","last_undegraded":"2024-03-29T08:21:09.950524+0000","last_fullsized":"2024-03-29T08:21:09.950524+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:00.619680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938522+0000","last_change":"2024-03-29T08:21:09.938522+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938522+0000","last_undegraded":"2024-03-29T08:21:09.938522+0000","last_fullsized":"2024-03-29T08:21:09.938522+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:01:48.373155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.947564+0000","last_change":"2024-03-29T08:21:09.946582+0000","last_active":"2024-03-29T08:21:09.947564+0000","last_peered":"2024-03-29T08:21:09.947564+0000","last_clean":"2024-03-29T08:21:09.947564+0000","last_became_active":"2024-03-29T08:21:09.946276+0000","last_became_peered":"2024-03-29T08:21:09.946276+0000","last_unstale":"2024-03-29T08:21:09.947564+0000","last_undegraded":"2024-03-29T08:21:09.947564+0000","last_fullsized":"2024-03-29T08:21:09.947564+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:01:32.117594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.932610+0000","last_change":"2024-03-29T08:21:09.932610+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.932610+0000","last_undegraded":"2024-03-29T08:21:09.932610+0000","last_fullsized":"2024-03-29T08:21:09.932610+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:11:50.905254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.955654+0000","last_change":"2024-03-29T08:21:09.953517+0000","last_active":"2024-03-29T08:21:09.955654+0000","last_peered":"2024-03-29T08:21:09.955654+0000","last_clean":"2024-03-29T08:21:09.955654+0000","last_became_active":"2024-03-29T08:21:09.953205+0000","last_became_peered":"2024-03-29T08:21:09.953205+0000","last_unstale":"2024-03-29T08:21:09.955654+0000","last_undegraded":"2024-03-29T08:21:09.955654+0000","last_fullsized":"2024-03-29T08:21:09.955654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:11:14.860644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936862+0000","last_change":"2024-03-29T08:21:09.936862+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936862+0000","last_undegraded":"2024-03-29T08:21:09.936862+0000","last_fullsized":"2024-03-29T08:21:09.936862+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:15.253098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.957279+0000","last_change":"2024-03-29T08:21:09.954705+0000","last_active":"2024-03-29T08:21:09.957279+0000","last_peered":"2024-03-29T08:21:09.957279+0000","last_clean":"2024-03-29T08:21:09.957279+0000","last_became_active":"2024-03-29T08:21:09.954314+0000","last_became_peered":"2024-03-29T08:21:09.954314+0000","last_unstale":"2024-03-29T08:21:09.957279+0000","last_undegraded":"2024-03-29T08:21:09.957279+0000","last_fullsized":"2024-03-29T08:21:09.957279+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:24:09.856034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936946+0000","last_change":"2024-03-29T08:21:09.936946+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936946+0000","last_undegraded":"2024-03-29T08:21:09.936946+0000","last_fullsized":"2024-03-29T08:21:09.936946+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:05:09.416516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.934157+0000","last_change":"2024-03-29T08:21:08.934157+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.934157+0000","last_undegraded":"2024-03-29T08:21:08.934157+0000","last_fullsized":"2024-03-29T08:21:08.934157+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:50:03.760523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.941996+0000","last_change":"2024-03-29T08:21:09.941996+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.941996+0000","last_undegraded":"2024-03-29T08:21:09.941996+0000","last_fullsized":"2024-03-29T08:21:09.941996+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:50:48.557652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.948833+0000","last_change":"2024-03-29T08:21:09.947148+0000","last_active":"2024-03-29T08:21:09.948833+0000","last_peered":"2024-03-29T08:21:09.948833+0000","last_clean":"2024-03-29T08:21:09.948833+0000","last_became_active":"2024-03-29T08:21:09.946610+0000","last_became_peered":"2024-03-29T08:21:09.946610+0000","last_unstale":"2024-03-29T08:21:09.948833+0000","last_undegraded":"2024-03-29T08:21:09.948833+0000","last_fullsized":"2024-03-29T08:21:09.948833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:51.605750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.932484+0000","last_change":"2024-03-29T08:21:09.932484+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.932484+0000","last_undegraded":"2024-03-29T08:21:09.932484+0000","last_fullsized":"2024-03-29T08:21:09.932484+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:37:31.250178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.936989+0000","last_change":"2024-03-29T08:21:08.936989+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.936989+0000","last_undegraded":"2024-03-29T08:21:08.936989+0000","last_fullsized":"2024-03-29T08:21:08.936989+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:26:31.314621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937474+0000","last_change":"2024-03-29T08:21:09.937474+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937474+0000","last_undegraded":"2024-03-29T08:21:09.937474+0000","last_fullsized":"2024-03-29T08:21:09.937474+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:24:10.858914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956169+0000","last_change":"2024-03-29T08:21:09.953120+0000","last_active":"2024-03-29T08:21:09.956169+0000","last_peered":"2024-03-29T08:21:09.956169+0000","last_clean":"2024-03-29T08:21:09.956169+0000","last_became_active":"2024-03-29T08:21:09.952860+0000","last_became_peered":"2024-03-29T08:21:09.952860+0000","last_unstale":"2024-03-29T08:21:09.956169+0000","last_undegraded":"2024-03-29T08:21:09.956169+0000","last_fullsized":"2024-03-29T08:21:09.956169+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:38.455763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.934110+0000","last_change":"2024-03-29T08:21:08.934110+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.934110+0000","last_undegraded":"2024-03-29T08:21:08.934110+0000","last_fullsized":"2024-03-29T08:21:08.934110+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:05:43.897022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.948860+0000","last_change":"2024-03-29T08:21:09.947277+0000","last_active":"2024-03-29T08:21:09.948860+0000","last_peered":"2024-03-29T08:21:09.948860+0000","last_clean":"2024-03-29T08:21:09.948860+0000","last_became_active":"2024-03-29T08:21:09.945398+0000","last_became_peered":"2024-03-29T08:21:09.945398+0000","last_unstale":"2024-03-29T08:21:09.948860+0000","last_undegraded":"2024-03-29T08:21:09.948860+0000","last_fullsized":"2024-03-29T08:21:09.948860+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:36:29.788331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.935957+0000","last_change":"2024-03-29T08:21:09.935957+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.935957+0000","last_undegraded":"2024-03-29T08:21:09.935957+0000","last_fullsized":"2024-03-29T08:21:09.935957+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:18:28.430663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.957304+0000","last_change":"2024-03-29T08:21:09.955367+0000","last_active":"2024-03-29T08:21:09.957304+0000","last_peered":"2024-03-29T08:21:09.957304+0000","last_clean":"2024-03-29T08:21:09.957304+0000","last_became_active":"2024-03-29T08:21:09.954965+0000","last_became_peered":"2024-03-29T08:21:09.954965+0000","last_unstale":"2024-03-29T08:21:09.957304+0000","last_undegraded":"2024-03-29T08:21:09.957304+0000","last_fullsized":"2024-03-29T08:21:09.957304+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:57.828588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936065+0000","last_change":"2024-03-29T08:21:09.936065+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936065+0000","last_undegraded":"2024-03-29T08:21:09.936065+0000","last_fullsized":"2024-03-29T08:21:09.936065+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:01.741332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.952538+0000","last_change":"2024-03-29T08:21:09.948192+0000","last_active":"2024-03-29T08:21:09.952538+0000","last_peered":"2024-03-29T08:21:09.952538+0000","last_clean":"2024-03-29T08:21:09.952538+0000","last_became_active":"2024-03-29T08:21:09.947926+0000","last_became_peered":"2024-03-29T08:21:09.947926+0000","last_unstale":"2024-03-29T08:21:09.952538+0000","last_undegraded":"2024-03-29T08:21:09.952538+0000","last_fullsized":"2024-03-29T08:21:09.952538+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:33:58.392690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936565+0000","last_change":"2024-03-29T08:21:09.936565+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936565+0000","last_undegraded":"2024-03-29T08:21:09.936565+0000","last_fullsized":"2024-03-29T08:21:09.936565+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:22:08.727894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.950245+0000","last_change":"2024-03-29T08:21:09.947847+0000","last_active":"2024-03-29T08:21:09.950245+0000","last_peered":"2024-03-29T08:21:09.950245+0000","last_clean":"2024-03-29T08:21:09.950245+0000","last_became_active":"2024-03-29T08:21:09.946845+0000","last_became_peered":"2024-03-29T08:21:09.946845+0000","last_unstale":"2024-03-29T08:21:09.950245+0000","last_undegraded":"2024-03-29T08:21:09.950245+0000","last_fullsized":"2024-03-29T08:21:09.950245+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:19:13.916686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.943473+0000","last_change":"2024-03-29T08:21:09.938352+0000","last_active":"2024-03-29T08:21:09.943473+0000","last_peered":"2024-03-29T08:21:09.943473+0000","last_clean":"2024-03-29T08:21:09.943473+0000","last_became_active":"2024-03-29T08:21:09.937930+0000","last_became_peered":"2024-03-29T08:21:09.937930+0000","last_unstale":"2024-03-29T08:21:09.943473+0000","last_undegraded":"2024-03-29T08:21:09.943473+0000","last_fullsized":"2024-03-29T08:21:09.943473+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:23.163706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938692+0000","last_change":"2024-03-29T08:21:09.938692+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938692+0000","last_undegraded":"2024-03-29T08:21:09.938692+0000","last_fullsized":"2024-03-29T08:21:09.938692+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:44.680234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956168+0000","last_change":"2024-03-29T08:21:09.953317+0000","last_active":"2024-03-29T08:21:09.956168+0000","last_peered":"2024-03-29T08:21:09.956168+0000","last_clean":"2024-03-29T08:21:09.956168+0000","last_became_active":"2024-03-29T08:21:09.953068+0000","last_became_peered":"2024-03-29T08:21:09.953068+0000","last_unstale":"2024-03-29T08:21:09.956168+0000","last_undegraded":"2024-03-29T08:21:09.956168+0000","last_fullsized":"2024-03-29T08:21:09.956168+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:39.840479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.941589+0000","last_change":"2024-03-29T08:21:09.941589+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.941589+0000","last_undegraded":"2024-03-29T08:21:09.941589+0000","last_fullsized":"2024-03-29T08:21:09.941589+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:40.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.952730+0000","last_change":"2024-03-29T08:21:09.950160+0000","last_active":"2024-03-29T08:21:09.952730+0000","last_peered":"2024-03-29T08:21:09.952730+0000","last_clean":"2024-03-29T08:21:09.952730+0000","last_became_active":"2024-03-29T08:21:09.949853+0000","last_became_peered":"2024-03-29T08:21:09.949853+0000","last_unstale":"2024-03-29T08:21:09.952730+0000","last_undegraded":"2024-03-29T08:21:09.952730+0000","last_fullsized":"2024-03-29T08:21:09.952730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:59:23.381718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.947411+0000","last_change":"2024-03-29T08:21:09.945534+0000","last_active":"2024-03-29T08:21:09.947411+0000","last_peered":"2024-03-29T08:21:09.947411+0000","last_clean":"2024-03-29T08:21:09.947411+0000","last_became_active":"2024-03-29T08:21:09.943748+0000","last_became_peered":"2024-03-29T08:21:09.943748+0000","last_unstale":"2024-03-29T08:21:09.947411+0000","last_undegraded":"2024-03-29T08:21:09.947411+0000","last_fullsized":"2024-03-29T08:21:09.947411+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:05:06.965072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936902+0000","last_change":"2024-03-29T08:21:09.936902+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936902+0000","last_undegraded":"2024-03-29T08:21:09.936902+0000","last_fullsized":"2024-03-29T08:21:09.936902+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:38:18.883117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.934050+0000","last_change":"2024-03-29T08:21:08.934050+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.934050+0000","last_undegraded":"2024-03-29T08:21:08.934050+0000","last_fullsized":"2024-03-29T08:21:08.934050+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:00:36.010161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.955570+0000","last_change":"2024-03-29T08:21:09.953812+0000","last_active":"2024-03-29T08:21:09.955570+0000","last_peered":"2024-03-29T08:21:09.955570+0000","last_clean":"2024-03-29T08:21:09.955570+0000","last_became_active":"2024-03-29T08:21:09.953475+0000","last_became_peered":"2024-03-29T08:21:09.953475+0000","last_unstale":"2024-03-29T08:21:09.955570+0000","last_undegraded":"2024-03-29T08:21:09.955570+0000","last_fullsized":"2024-03-29T08:21:09.955570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:05:21.462217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":196,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-03-29T08:21:08.935647+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:21:08.935647+0000","last_peered":"2024-03-29T08:21:08.935647+0000","last_clean":"2024-03-29T08:21:08.935647+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:21:08.935647+0000","last_undegraded":"2024-03-29T08:21:08.935647+0000","last_fullsized":"2024-03-29T08:21:08.935647+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":7},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":293626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67600000000000005,"15min":0.65200000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93400000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.69999999999999996,"15min":0.67600000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1200000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.68700000000000006}]},{"osd":1,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.67100000000000004,"15min":0.64800000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.66800000000000004,"15min":0.63400000000000001},"min":{"1min":0.52900000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.0189999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.82599999999999996}]},{"osd":2,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.66200000000000003,"15min":0.63300000000000001},"min":{"1min":0.55900000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.0089999999999999,"5min":1.085,"15min":1.085},"last":0.754},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.69299999999999995,"15min":0.66500000000000004},"min":{"1min":0.58499999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1000000000000001,"5min":1.288,"15min":1.288},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71099999999999997,"15min":0.69799999999999995},"min":{"1min":0.57899999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92800000000000005,"5min":1.018,"15min":1.018},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.745,"15min":0.73099999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1659999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.73499999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83499999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70199999999999996,"15min":0.71999999999999997},"min":{"1min":0.46100000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0009999999999999,"5min":1.123,"15min":1.123},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.95799999999999996,"5min":1.056,"15min":1.056},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.46300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73399999999999999,"15min":0.72999999999999998},"min":{"1min":0.48999999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.755},"min":{"1min":0.52300000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.84199999999999997}]}]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":293626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.66000000000000003,"15min":0.63300000000000001},"min":{"1min":0.54000000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70499999999999996,"15min":0.68500000000000005},"min":{"1min":0.57899999999999996,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.71399999999999997}]},{"osd":1,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.64100000000000001,"15min":0.61499999999999999},"min":{"1min":0.58599999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.64800000000000002,"15min":0.627},"min":{"1min":0.51600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.70299999999999996,"15min":0.67500000000000004},"min":{"1min":0.52300000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.54300000000000004,"5min":0.435,"15min":0.435},"max":{"1min":0.91200000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.81100000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71399999999999997,"15min":0.69999999999999996},"min":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94099999999999995,"5min":1.109,"15min":1.109},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.53200000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.84499999999999997}]},{"osd":4,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0620000000000001,"5min":1.556,"15min":1.556},"last":1.02},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70499999999999996,"15min":0.73699999999999999},"min":{"1min":0.44400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.0429999999999999,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.0429999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67800000000000005,"15min":0.70299999999999996},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.71599999999999997,"15min":0.73399999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.86299999999999999}]},{"osd":7,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.74199999999999999,"15min":0.76700000000000002},"min":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.92300000000000004,"5min":0.998,"15min":0.998},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.753,"15min":0.77900000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":1.109}]}]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":293626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.50800000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93100000000000005,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72699999999999998,"15min":0.74399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95199999999999996,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.73599999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.76500000000000001},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97799999999999998,"5min":2.524,"15min":2.524},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.54900000000000004,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93999999999999995,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.83699999999999997}]},{"osd":2,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84499999999999997,"5min":2.363,"15min":2.363},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0029999999999999,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.76900000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.747,"15min":0.75700000000000001},"min":{"1min":0.54700000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90000000000000002,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.753},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.81699999999999995},"min":{"1min":0.625,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.96199999999999997,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.86899999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68200000000000005,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88100000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.441,"5min":0.434,"15min":0.434},"max":{"1min":0.82199999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.65900000000000003}]},{"osd":6,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86799999999999999,"5min":0.996,"15min":0.996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72599999999999998,"15min":0.74399999999999999},"min":{"1min":0.52300000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.91600000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.63800000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71699999999999997,"15min":0.73799999999999999},"min":{"1min":0.51100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90900000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.46999999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92100000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.68999999999999995}]}]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":724992,"data_stored":299901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.46300000000000002},"min":{"1min":0.33700000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.72699999999999998,"5min":1.123,"15min":1.123},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52000000000000002,"15min":0.52400000000000002},"min":{"1min":0.432,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.871,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.437}]},{"osd":1,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44700000000000001,"15min":0.46100000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64800000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.51500000000000001,"15min":0.53400000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.68200000000000005,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.28599999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54700000000000004,"15min":0.56999999999999995},"min":{"1min":0.35999999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.70599999999999996,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63700000000000001,"15min":0.65900000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79700000000000004,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.41799999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.77400000000000002,"15min":0.79600000000000004},"min":{"1min":0.52100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1950000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.75900000000000001,"15min":0.77900000000000003},"min":{"1min":0.55700000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.004,"5min":1.391,"15min":1.391},"last":0.64600000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.80800000000000005},"min":{"1min":0.627,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84299999999999997,"15min":0.86699999999999999},"min":{"1min":0.64500000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6599999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.81000000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.79600000000000004,"15min":0.83599999999999997},"min":{"1min":0.51500000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.008,"5min":1.669,"15min":1.669},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.80300000000000005,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.212,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.76200000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82999999999999996,"15min":0.86399999999999999},"min":{"1min":0.54900000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4279999999999999,"5min":1.887,"15min":1.887},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.82099999999999995,"15min":0.86099999999999999},"min":{"1min":0.59899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98299999999999998,"5min":2.044,"15min":2.044},"last":0.70299999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":712704,"data_stored":293626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.48899999999999999,"15min":0.501},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69799999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66300000000000003,"5min":0.751,"15min":0.751},"last":0.65900000000000003}]},{"osd":1,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.34499999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72999999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.375},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.48999999999999999,"15min":0.50800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72499999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.39900000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75900000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40400000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69299999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.61599999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52600000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.052,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.75900000000000001,"15min":0.76600000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82999999999999996,"15min":0.84199999999999997},"min":{"1min":0.64600000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.081,"5min":1.306,"15min":1.306},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83999999999999997,"15min":0.85999999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.28,"5min":1.526,"15min":1.526},"last":0.55500000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.77700000000000002,"15min":0.77900000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.55600000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.121,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.53400000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.51000000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":0.79800000000000004},"min":{"1min":0.56499999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.71899999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1306624,"data_stored":883994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.56799999999999995,"15min":0.63500000000000001},"min":{"1min":0.33400000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.54800000000000004,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.60099999999999998,"15min":0.65300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71799999999999997,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.49399999999999999,"15min":0.50600000000000001},"min":{"1min":0.375,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73299999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53000000000000003,"15min":0.53600000000000003},"min":{"1min":0.39100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67000000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.435}]},{"osd":3,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.86099999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.75900000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.41299999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.53000000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95599999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.746},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.76300000000000001,"15min":0.78900000000000003},"min":{"1min":0.58099999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95199999999999996,"5min":1.23,"15min":1.23},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76100000000000001,"15min":0.77900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.89500000000000002,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.56499999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.873,"5min":1.107,"15min":1.107},"last":0.67800000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76000000000000001,"15min":0.76800000000000002},"min":{"1min":0.57399999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":1.026,"15min":1.026},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.81799999999999995,"15min":0.85699999999999998},"min":{"1min":0.55800000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.83199999999999996,"5min":1.196,"15min":1.196},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.78700000000000003,"15min":0.80000000000000004},"min":{"1min":0.57499999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.92900000000000005,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.78700000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91600000000000004,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.66100000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":724992,"data_stored":299901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.55300000000000005,"15min":0.61299999999999999},"min":{"1min":0.313,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.71099999999999997,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.753,"15min":0.873},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.73799999999999999,"5min":8.0419999999999998,"15min":8.0419999999999998},"last":0.34000000000000002}]},{"osd":2,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.53800000000000003,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.58899999999999997,"5min":1.258,"15min":1.258},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.67500000000000004,"15min":0.747},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63800000000000001,"5min":4.907,"15min":4.907},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.753,"15min":0.872},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.77000000000000002,"5min":7.9569999999999999,"15min":7.9569999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.624,"15min":0.66600000000000004},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.65100000000000002,"5min":1.397,"15min":1.397},"last":0.44900000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":1.002},"min":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91700000000000004,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.91500000000000004,"15min":0.98099999999999998},"min":{"1min":0.66700000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.97999999999999998,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.77000000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.879,"15min":0.93999999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.016,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.83299999999999996,"15min":0.871},"min":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.89900000000000002,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.754}]},{"osd":6,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.83899999999999997,"15min":0.87},"min":{"1min":0.64500000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95699999999999996,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.92900000000000005,"15min":0.999},"min":{"1min":0.623,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.097,"5min":4.4100000000000001,"15min":4.4100000000000001},"last":0.68100000000000005}]},{"osd":7,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.94599999999999995,"15min":1.0209999999999999},"min":{"1min":0.64100000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.038,"5min":4.5209999999999999,"15min":4.5209999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.92100000000000004,"15min":1},"min":{"1min":0.57699999999999996,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.099,"5min":4.4420000000000002,"15min":4.4420000000000002},"last":0.57699999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1306624,"data_stored":883994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.497,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89400000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.497}]},{"osd":1,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71599999999999997,"15min":0.73799999999999999},"min":{"1min":0.51700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.876,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67800000000000005,"15min":0.67700000000000005},"min":{"1min":0.44900000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89000000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.443,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84299999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.443},{"interface":"front","average":{"1min":0.748,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.59199999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90700000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.88600000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.54800000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91800000000000004,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.64500000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.90500000000000003,"5min":1.954,"15min":1.954},"last":0.80800000000000005}]},{"osd":4,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.50800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.51400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88700000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.82899999999999996}]},{"osd":6,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66700000000000004,"15min":0.68200000000000005},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82399999999999995,"5min":1.071,"15min":1.071},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.70099999999999996},"min":{"1min":0.53600000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.878,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92900000000000005,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.74099999999999999,"15min":0.75800000000000001},"min":{"1min":0.56799999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95199999999999996,"5min":1.04,"15min":1.04},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:21:15.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3f is not active+clean 2024-03-29T08:21:15.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.941725+0000', 'last_change': '2024-03-29T08:21:09.941725+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.941725+0000', 'last_undegraded': '2024-03-29T08:21:09.941725+0000', 'last_fullsized': '2024-03-29T08:21:09.941725+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T09:16:15.767217+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T08:21:15.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3e is not active+clean 2024-03-29T08:21:15.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.936961+0000', 'last_change': '2024-03-29T08:21:09.936961+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.936961+0000', 'last_undegraded': '2024-03-29T08:21:09.936961+0000', 'last_fullsized': '2024-03-29T08:21:09.936961+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T15:05:22.465098+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T08:21:15.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3d is not active+clean 2024-03-29T08:21:15.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.938613+0000', 'last_change': '2024-03-29T08:21:09.938613+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.938613+0000', 'last_undegraded': '2024-03-29T08:21:09.938613+0000', 'last_fullsized': '2024-03-29T08:21:09.938613+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T12:47:48.297292+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T08:21:15.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2024-03-29T08:21:15.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.933020+0000', 'last_change': '2024-03-29T08:21:09.933020+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.933020+0000', 'last_undegraded': '2024-03-29T08:21:09.933020+0000', 'last_fullsized': '2024-03-29T08:21:09.933020+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T09:27:51.414360+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T08:21:15.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3b is not active+clean 2024-03-29T08:21:15.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.936757+0000', 'last_change': '2024-03-29T08:21:09.936757+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.936757+0000', 'last_undegraded': '2024-03-29T08:21:09.936757+0000', 'last_fullsized': '2024-03-29T08:21:09.936757+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T13:36:29.239728+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T08:21:15.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3a is not active+clean 2024-03-29T08:21:15.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.939144+0000', 'last_change': '2024-03-29T08:21:09.939144+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.939144+0000', 'last_undegraded': '2024-03-29T08:21:09.939144+0000', 'last_fullsized': '2024-03-29T08:21:09.939144+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T14:35:20.420999+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T08:21:15.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.39 is not active+clean 2024-03-29T08:21:15.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.933702+0000', 'last_change': '2024-03-29T08:21:09.933702+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.933702+0000', 'last_undegraded': '2024-03-29T08:21:09.933702+0000', 'last_fullsized': '2024-03-29T08:21:09.933702+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T18:30:03.234601+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T08:21:15.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.38 is not active+clean 2024-03-29T08:21:15.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T08:21:09.925622+0000', 'last_change': '2024-03-29T08:21:09.925622+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.925622+0000', 'last_undegraded': '2024-03-29T08:21:09.925622+0000', 'last_fullsized': '2024-03-29T08:21:09.925622+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T08:21:15.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.37 is not active+clean 2024-03-29T08:21:15.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.939490+0000', 'last_change': '2024-03-29T08:21:09.939490+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.939490+0000', 'last_undegraded': '2024-03-29T08:21:09.939490+0000', 'last_fullsized': '2024-03-29T08:21:09.939490+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T09:40:40.843359+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-29T08:21:15.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.36 is not active+clean 2024-03-29T08:21:15.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.940289+0000', 'last_change': '2024-03-29T08:21:09.940289+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.940289+0000', 'last_undegraded': '2024-03-29T08:21:09.940289+0000', 'last_fullsized': '2024-03-29T08:21:09.940289+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T10:07:54.034461+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T08:21:15.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.35 is not active+clean 2024-03-29T08:21:15.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.934375+0000', 'last_change': '2024-03-29T08:21:09.934375+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.934375+0000', 'last_undegraded': '2024-03-29T08:21:09.934375+0000', 'last_fullsized': '2024-03-29T08:21:09.934375+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T11:18:58.105380+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T08:21:15.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2024-03-29T08:21:15.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.940141+0000', 'last_change': '2024-03-29T08:21:09.940141+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.940141+0000', 'last_undegraded': '2024-03-29T08:21:09.940141+0000', 'last_fullsized': '2024-03-29T08:21:09.940141+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T13:20:51.330512+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-29T08:21:15.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2024-03-29T08:21:15.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.942430+0000', 'last_change': '2024-03-29T08:21:09.942430+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.942430+0000', 'last_undegraded': '2024-03-29T08:21:09.942430+0000', 'last_fullsized': '2024-03-29T08:21:09.942430+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T14:35:20.420999+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-29T08:21:15.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.32 is not active+clean 2024-03-29T08:21:15.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.939640+0000', 'last_change': '2024-03-29T08:21:09.939640+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.939640+0000', 'last_undegraded': '2024-03-29T08:21:09.939640+0000', 'last_fullsized': '2024-03-29T08:21:09.939640+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T15:49:07.458847+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T08:21:15.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.31 is not active+clean 2024-03-29T08:21:15.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.940293+0000', 'last_change': '2024-03-29T08:21:09.940293+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.940293+0000', 'last_undegraded': '2024-03-29T08:21:09.940293+0000', 'last_fullsized': '2024-03-29T08:21:09.940293+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T15:01:39.458643+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-29T08:21:15.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.30 is not active+clean 2024-03-29T08:21:15.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.938066+0000', 'last_change': '2024-03-29T08:21:09.938066+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.938066+0000', 'last_undegraded': '2024-03-29T08:21:09.938066+0000', 'last_fullsized': '2024-03-29T08:21:09.938066+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T11:01:52.376061+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T08:21:15.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2f is not active+clean 2024-03-29T08:21:15.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.941263+0000', 'last_change': '2024-03-29T08:21:09.941263+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.941263+0000', 'last_undegraded': '2024-03-29T08:21:09.941263+0000', 'last_fullsized': '2024-03-29T08:21:09.941263+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T15:01:39.458643+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T08:21:15.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2e is not active+clean 2024-03-29T08:21:15.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.939231+0000', 'last_change': '2024-03-29T08:21:09.939231+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.939231+0000', 'last_undegraded': '2024-03-29T08:21:09.939231+0000', 'last_fullsized': '2024-03-29T08:21:09.939231+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T10:33:19.998323+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T08:21:15.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2d is not active+clean 2024-03-29T08:21:15.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.942709+0000', 'last_change': '2024-03-29T08:21:09.942709+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.942709+0000', 'last_undegraded': '2024-03-29T08:21:09.942709+0000', 'last_fullsized': '2024-03-29T08:21:09.942709+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T12:58:48.804311+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T08:21:15.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2c is not active+clean 2024-03-29T08:21:15.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.934214+0000', 'last_change': '2024-03-29T08:21:09.934214+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.934214+0000', 'last_undegraded': '2024-03-29T08:21:09.934214+0000', 'last_fullsized': '2024-03-29T08:21:09.934214+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T09:02:01.741332+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T08:21:15.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2b is not active+clean 2024-03-29T08:21:15.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.942033+0000', 'last_change': '2024-03-29T08:21:09.942033+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.942033+0000', 'last_undegraded': '2024-03-29T08:21:09.942033+0000', 'last_fullsized': '2024-03-29T08:21:09.942033+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T19:46:44.680234+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-29T08:21:15.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2a is not active+clean 2024-03-29T08:21:15.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.934012+0000', 'last_change': '2024-03-29T08:21:09.934012+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.934012+0000', 'last_undegraded': '2024-03-29T08:21:09.934012+0000', 'last_fullsized': '2024-03-29T08:21:09.934012+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T11:15:15.253098+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T08:21:15.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.29 is not active+clean 2024-03-29T08:21:15.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.933212+0000', 'last_change': '2024-03-29T08:21:09.933212+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.933212+0000', 'last_undegraded': '2024-03-29T08:21:09.933212+0000', 'last_fullsized': '2024-03-29T08:21:09.933212+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T08:25:00.595704+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T08:21:15.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2024-03-29T08:21:15.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.938923+0000', 'last_change': '2024-03-29T08:21:09.938923+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.938923+0000', 'last_undegraded': '2024-03-29T08:21:09.938923+0000', 'last_fullsized': '2024-03-29T08:21:09.938923+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T10:38:58.831468+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-29T08:21:15.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.27 is not active+clean 2024-03-29T08:21:15.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.937337+0000', 'last_change': '2024-03-29T08:21:09.937337+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.937337+0000', 'last_undegraded': '2024-03-29T08:21:09.937337+0000', 'last_fullsized': '2024-03-29T08:21:09.937337+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T16:59:24.384598+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T08:21:15.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2024-03-29T08:21:15.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.943515+0000', 'last_change': '2024-03-29T08:21:09.943515+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.943515+0000', 'last_undegraded': '2024-03-29T08:21:09.943515+0000', 'last_fullsized': '2024-03-29T08:21:09.943515+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T17:39:53.450623+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-29T08:21:15.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.25 is not active+clean 2024-03-29T08:21:15.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.936629+0000', 'last_change': '2024-03-29T08:21:09.936629+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.936629+0000', 'last_undegraded': '2024-03-29T08:21:09.936629+0000', 'last_fullsized': '2024-03-29T08:21:09.936629+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T19:13:49.605189+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T08:21:15.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2024-03-29T08:21:15.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T08:21:09.925622+0000', 'last_change': '2024-03-29T08:21:09.925622+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.925622+0000', 'last_undegraded': '2024-03-29T08:21:09.925622+0000', 'last_fullsized': '2024-03-29T08:21:09.925622+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T08:21:15.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.23 is not active+clean 2024-03-29T08:21:15.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T08:21:09.925622+0000', 'last_change': '2024-03-29T08:21:09.925622+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.925622+0000', 'last_undegraded': '2024-03-29T08:21:09.925622+0000', 'last_fullsized': '2024-03-29T08:21:09.925622+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T08:21:15.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.22 is not active+clean 2024-03-29T08:21:15.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.932435+0000', 'last_change': '2024-03-29T08:21:09.932435+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.932435+0000', 'last_undegraded': '2024-03-29T08:21:09.932435+0000', 'last_fullsized': '2024-03-29T08:21:09.932435+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T14:59:45.698306+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T08:21:15.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.21 is not active+clean 2024-03-29T08:21:15.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.943149+0000', 'last_change': '2024-03-29T08:21:09.943149+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.943149+0000', 'last_undegraded': '2024-03-29T08:21:09.943149+0000', 'last_fullsized': '2024-03-29T08:21:09.943149+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T17:01:19.390653+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T08:21:15.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.20 is not active+clean 2024-03-29T08:21:15.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.934024+0000', 'last_change': '2024-03-29T08:21:09.934024+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.934024+0000', 'last_undegraded': '2024-03-29T08:21:09.934024+0000', 'last_fullsized': '2024-03-29T08:21:09.934024+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T10:07:54.034461+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T08:21:15.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2024-03-29T08:21:15.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.939827+0000', 'last_change': '2024-03-29T08:21:09.939827+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.939827+0000', 'last_undegraded': '2024-03-29T08:21:09.939827+0000', 'last_fullsized': '2024-03-29T08:21:09.939827+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T17:39:53.450623+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T08:21:15.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2024-03-29T08:21:15.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.937754+0000', 'last_change': '2024-03-29T08:21:09.937754+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.937754+0000', 'last_undegraded': '2024-03-29T08:21:09.937754+0000', 'last_fullsized': '2024-03-29T08:21:09.937754+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T11:18:40.333881+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T08:21:15.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2024-03-29T08:21:15.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T08:21:09.925622+0000', 'last_change': '2024-03-29T08:21:09.925622+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.925622+0000', 'last_undegraded': '2024-03-29T08:21:09.925622+0000', 'last_fullsized': '2024-03-29T08:21:09.925622+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T08:21:15.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2024-03-29T08:21:15.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.938835+0000', 'last_change': '2024-03-29T08:21:09.938835+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.938835+0000', 'last_undegraded': '2024-03-29T08:21:09.938835+0000', 'last_fullsized': '2024-03-29T08:21:09.938835+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T12:21:23.293798+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T08:21:15.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2024-03-29T08:21:15.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T08:21:09.925622+0000', 'last_change': '2024-03-29T08:21:09.925622+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.925622+0000', 'last_undegraded': '2024-03-29T08:21:09.925622+0000', 'last_fullsized': '2024-03-29T08:21:09.925622+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T08:21:15.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2024-03-29T08:21:15.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.935582+0000', 'last_change': '2024-03-29T08:21:09.935582+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.935582+0000', 'last_undegraded': '2024-03-29T08:21:09.935582+0000', 'last_fullsized': '2024-03-29T08:21:09.935582+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T09:05:09.416516+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-03-29T08:21:15.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2024-03-29T08:21:15.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.937385+0000', 'last_change': '2024-03-29T08:21:09.937385+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.937385+0000', 'last_undegraded': '2024-03-29T08:21:09.937385+0000', 'last_fullsized': '2024-03-29T08:21:09.937385+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T11:18:58.105380+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T08:21:15.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2024-03-29T08:21:15.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.937620+0000', 'last_change': '2024-03-29T08:21:09.937620+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.937620+0000', 'last_undegraded': '2024-03-29T08:21:09.937620+0000', 'last_fullsized': '2024-03-29T08:21:09.937620+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T18:19:16.700510+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T08:21:15.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2024-03-29T08:21:15.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.943037+0000', 'last_change': '2024-03-29T08:21:09.943037+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.943037+0000', 'last_undegraded': '2024-03-29T08:21:09.943037+0000', 'last_fullsized': '2024-03-29T08:21:09.943037+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T12:35:24.166586+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-29T08:21:15.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2024-03-29T08:21:15.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.938641+0000', 'last_change': '2024-03-29T08:21:09.938641+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.938641+0000', 'last_undegraded': '2024-03-29T08:21:09.938641+0000', 'last_fullsized': '2024-03-29T08:21:09.938641+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T17:08:39.271148+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T08:21:15.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-03-29T08:21:15.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:08.939701+0000', 'last_change': '2024-03-29T08:21:08.939701+0000', 'last_active': '2024-03-29T08:21:08.922741+0000', 'last_peered': '2024-03-29T08:21:08.922741+0000', 'last_clean': '2024-03-29T08:21:08.922741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:08.939701+0000', 'last_undegraded': '2024-03-29T08:21:08.939701+0000', 'last_fullsized': '2024-03-29T08:21:08.939701+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T20:08:44.015717+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:21:15.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2024-03-29T08:21:15.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T08:21:09.925622+0000', 'last_change': '2024-03-29T08:21:09.925622+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.925622+0000', 'last_undegraded': '2024-03-29T08:21:09.925622+0000', 'last_fullsized': '2024-03-29T08:21:09.925622+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T08:21:15.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2024-03-29T08:21:15.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.932991+0000', 'last_change': '2024-03-29T08:21:09.932991+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.932991+0000', 'last_undegraded': '2024-03-29T08:21:09.932991+0000', 'last_fullsized': '2024-03-29T08:21:09.932991+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T10:56:39.434467+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T08:21:15.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2024-03-29T08:21:15.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.939449+0000', 'last_change': '2024-03-29T08:21:09.939449+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.939449+0000', 'last_undegraded': '2024-03-29T08:21:09.939449+0000', 'last_fullsized': '2024-03-29T08:21:09.939449+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T16:52:34.720375+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-29T08:21:15.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-03-29T08:21:15.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:08.934101+0000', 'last_change': '2024-03-29T08:21:08.934101+0000', 'last_active': '2024-03-29T08:21:08.922741+0000', 'last_peered': '2024-03-29T08:21:08.922741+0000', 'last_clean': '2024-03-29T08:21:08.922741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:08.934101+0000', 'last_undegraded': '2024-03-29T08:21:08.934101+0000', 'last_fullsized': '2024-03-29T08:21:08.934101+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T09:03:13.729828+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:21:15.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2024-03-29T08:21:15.240 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.937954+0000', 'last_change': '2024-03-29T08:21:09.937954+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.937954+0000', 'last_undegraded': '2024-03-29T08:21:09.937954+0000', 'last_fullsized': '2024-03-29T08:21:09.937954+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T16:21:28.741076+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T08:21:15.240 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2024-03-29T08:21:15.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.938522+0000', 'last_change': '2024-03-29T08:21:09.938522+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.938522+0000', 'last_undegraded': '2024-03-29T08:21:09.938522+0000', 'last_fullsized': '2024-03-29T08:21:09.938522+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T17:01:48.373155+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-29T08:21:15.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2024-03-29T08:21:15.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.932610+0000', 'last_change': '2024-03-29T08:21:09.932610+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.932610+0000', 'last_undegraded': '2024-03-29T08:21:09.932610+0000', 'last_fullsized': '2024-03-29T08:21:09.932610+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T10:11:50.905254+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T08:21:15.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2024-03-29T08:21:15.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.936862+0000', 'last_change': '2024-03-29T08:21:09.936862+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.936862+0000', 'last_undegraded': '2024-03-29T08:21:09.936862+0000', 'last_fullsized': '2024-03-29T08:21:09.936862+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T11:15:15.253098+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T08:21:15.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2024-03-29T08:21:15.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.936946+0000', 'last_change': '2024-03-29T08:21:09.936946+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.936946+0000', 'last_undegraded': '2024-03-29T08:21:09.936946+0000', 'last_fullsized': '2024-03-29T08:21:09.936946+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T09:05:09.416516+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T08:21:15.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2024-03-29T08:21:15.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:08.934157+0000', 'last_change': '2024-03-29T08:21:08.934157+0000', 'last_active': '2024-03-29T08:21:08.922741+0000', 'last_peered': '2024-03-29T08:21:08.922741+0000', 'last_clean': '2024-03-29T08:21:08.922741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:08.934157+0000', 'last_undegraded': '2024-03-29T08:21:08.934157+0000', 'last_fullsized': '2024-03-29T08:21:08.934157+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T10:50:03.760523+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:21:15.241 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2024-03-29T08:21:15.241 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.941996+0000', 'last_change': '2024-03-29T08:21:09.941996+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.941996+0000', 'last_undegraded': '2024-03-29T08:21:09.941996+0000', 'last_fullsized': '2024-03-29T08:21:09.941996+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T17:50:48.557652+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T08:21:15.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2024-03-29T08:21:15.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.932484+0000', 'last_change': '2024-03-29T08:21:09.932484+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.932484+0000', 'last_undegraded': '2024-03-29T08:21:09.932484+0000', 'last_fullsized': '2024-03-29T08:21:09.932484+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T11:37:31.250178+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-03-29T08:21:15.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2024-03-29T08:21:15.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:08.936989+0000', 'last_change': '2024-03-29T08:21:08.936989+0000', 'last_active': '2024-03-29T08:21:08.922741+0000', 'last_peered': '2024-03-29T08:21:08.922741+0000', 'last_clean': '2024-03-29T08:21:08.922741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:08.936989+0000', 'last_undegraded': '2024-03-29T08:21:08.936989+0000', 'last_fullsized': '2024-03-29T08:21:08.936989+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T11:26:31.314621+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:21:15.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2024-03-29T08:21:15.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.937474+0000', 'last_change': '2024-03-29T08:21:09.937474+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.937474+0000', 'last_undegraded': '2024-03-29T08:21:09.937474+0000', 'last_fullsized': '2024-03-29T08:21:09.937474+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T17:24:10.858914+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-03-29T08:21:15.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2024-03-29T08:21:15.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T08:21:09.925622+0000', 'last_change': '2024-03-29T08:21:09.925622+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.925622+0000', 'last_undegraded': '2024-03-29T08:21:09.925622+0000', 'last_fullsized': '2024-03-29T08:21:09.925622+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T08:21:15.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2024-03-29T08:21:15.242 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:08.934110+0000', 'last_change': '2024-03-29T08:21:08.934110+0000', 'last_active': '2024-03-29T08:21:08.922741+0000', 'last_peered': '2024-03-29T08:21:08.922741+0000', 'last_clean': '2024-03-29T08:21:08.922741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:08.934110+0000', 'last_undegraded': '2024-03-29T08:21:08.934110+0000', 'last_fullsized': '2024-03-29T08:21:08.934110+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T13:05:43.897022+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:21:15.242 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2024-03-29T08:21:15.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T08:21:09.925622+0000', 'last_change': '2024-03-29T08:21:09.925622+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.925622+0000', 'last_undegraded': '2024-03-29T08:21:09.925622+0000', 'last_fullsized': '2024-03-29T08:21:09.925622+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T08:21:15.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2024-03-29T08:21:15.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.935957+0000', 'last_change': '2024-03-29T08:21:09.935957+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.935957+0000', 'last_undegraded': '2024-03-29T08:21:09.935957+0000', 'last_fullsized': '2024-03-29T08:21:09.935957+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T13:18:28.430663+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T08:21:15.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2024-03-29T08:21:15.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.936065+0000', 'last_change': '2024-03-29T08:21:09.936065+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.936065+0000', 'last_undegraded': '2024-03-29T08:21:09.936065+0000', 'last_fullsized': '2024-03-29T08:21:09.936065+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T09:02:01.741332+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-03-29T08:21:15.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2024-03-29T08:21:15.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.936565+0000', 'last_change': '2024-03-29T08:21:09.936565+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.936565+0000', 'last_undegraded': '2024-03-29T08:21:09.936565+0000', 'last_fullsized': '2024-03-29T08:21:09.936565+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T10:22:08.727894+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T08:21:15.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2024-03-29T08:21:15.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T08:21:09.925622+0000', 'last_change': '2024-03-29T08:21:09.925622+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.925622+0000', 'last_undegraded': '2024-03-29T08:21:09.925622+0000', 'last_fullsized': '2024-03-29T08:21:09.925622+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T08:21:15.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2024-03-29T08:21:15.243 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.938692+0000', 'last_change': '2024-03-29T08:21:09.938692+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.938692+0000', 'last_undegraded': '2024-03-29T08:21:09.938692+0000', 'last_fullsized': '2024-03-29T08:21:09.938692+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T19:46:44.680234+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-03-29T08:21:15.243 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2024-03-29T08:21:15.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.941589+0000', 'last_change': '2024-03-29T08:21:09.941589+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.941589+0000', 'last_undegraded': '2024-03-29T08:21:09.941589+0000', 'last_fullsized': '2024-03-29T08:21:09.941589+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T11:18:40.333881+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-03-29T08:21:15.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2024-03-29T08:21:15.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T08:21:09.925622+0000', 'last_change': '2024-03-29T08:21:09.925622+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.925622+0000', 'last_undegraded': '2024-03-29T08:21:09.925622+0000', 'last_fullsized': '2024-03-29T08:21:09.925622+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T08:21:15.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2024-03-29T08:21:15.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 61, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:09.936902+0000', 'last_change': '2024-03-29T08:21:09.936902+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.936902+0000', 'last_undegraded': '2024-03-29T08:21:09.936902+0000', 'last_fullsized': '2024-03-29T08:21:09.936902+0000', 'mapping_epoch': 61, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 61, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T14:38:18.883117+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-03-29T08:21:15.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-03-29T08:21:15.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 60, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:21:08.934050+0000', 'last_change': '2024-03-29T08:21:08.934050+0000', 'last_active': '2024-03-29T08:21:08.922741+0000', 'last_peered': '2024-03-29T08:21:08.922741+0000', 'last_clean': '2024-03-29T08:21:08.922741+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:08.934050+0000', 'last_undegraded': '2024-03-29T08:21:08.934050+0000', 'last_fullsized': '2024-03-29T08:21:08.934050+0000', 'mapping_epoch': 60, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 60, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:08.922741+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T11:00:36.010161+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:21:15.244 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2024-03-29T08:21:15.244 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-03-29T08:21:09.925622+0000', 'last_change': '2024-03-29T08:21:09.925622+0000', 'last_active': '2024-03-29T08:21:09.925622+0000', 'last_peered': '2024-03-29T08:21:09.925622+0000', 'last_clean': '2024-03-29T08:21:09.925622+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:21:09.925622+0000', 'last_undegraded': '2024-03-29T08:21:09.925622+0000', 'last_fullsized': '2024-03-29T08:21:09.925622+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'last_clean_scrub_stamp': '2024-03-29T08:21:09.925622+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-03-29T08:21:15.245 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-29T08:21:15.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.407+0000 7ff4efa00640 1 Processor -- start 2024-03-29T08:21:15.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.407+0000 7ff4efa00640 1 -- start start 2024-03-29T08:21:15.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.407+0000 7ff4efa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff4e8144210 0x7ff4e80cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.407+0000 7ff4efa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4e80ceb20 0x7ff4e806c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.407+0000 7ff4efa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e80cb510 con 0x7ff4e815ada0 2024-03-29T08:21:15.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.407+0000 7ff4efa00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e8070580 con 0x7ff4e80ceb20 2024-03-29T08:21:15.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.407+0000 7ff4efa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e8070860 con 0x7ff4e8144210 2024-03-29T08:21:15.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.407+0000 7ff4eda00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4e80ceb20 0x7ff4e806c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.407+0000 7ff4e7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff4e815ada0 0x7ff4e80ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42142/0 (socket says 172.21.15.46:42142) 2024-03-29T08:21:15.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.407+0000 7ff4e7e00640 1 -- 172.21.15.46:0/2612757204 learned_addr learned my addr 172.21.15.46:0/2612757204 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:15.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.407+0000 7ff4eda00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4e80ceb20 0x7ff4e806c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54408/0 (socket says 172.21.15.46:54408) 2024-03-29T08:21:15.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.407+0000 7ff4ed000640 1 --2- 172.21.15.46:0/2612757204 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff4e8144210 0x7ff4e80cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.407+0000 7ff4eda00640 1 -- 172.21.15.46:0/2612757204 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff4e8144210 msgr2=0x7ff4e80cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:15.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.407+0000 7ff4eda00640 1 --2- 172.21.15.46:0/2612757204 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff4e8144210 0x7ff4e80cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.407+0000 7ff4eda00640 1 -- 172.21.15.46:0/2612757204 >> v1:172.21.15.46:6789/0 conn(0x7ff4e815ada0 legacy=0x7ff4e80ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.407+0000 7ff4eda00640 1 -- 172.21.15.46:0/2612757204 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e8070b40 con 0x7ff4e80ceb20 2024-03-29T08:21:15.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.407+0000 7ff4e7400640 1 -- 172.21.15.46:0/2612757204 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2757136285 0 0) 0x7ff4e80cb510 con 0x7ff4e815ada0 2024-03-29T08:21:15.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.411+0000 7ff4eda00640 1 --2- 172.21.15.46:0/2612757204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e80ceb20 0x7ff4e806c950 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0009d90 tx=0x7ff4e001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74bd5cb1252d1e27 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.411+0000 7ff4e7400640 1 -- 172.21.15.46:0/2612757204 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00a5940 con 0x7ff4e80ceb20 2024-03-29T08:21:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.411+0000 7ff4e7400640 1 -- 172.21.15.46:0/2612757204 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e00a5aa0 con 0x7ff4e80ceb20 2024-03-29T08:21:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.411+0000 7ff4e7400640 1 -- 172.21.15.46:0/2612757204 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e00a5e30 con 0x7ff4e80ceb20 2024-03-29T08:21:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.411+0000 7ff4efa00640 1 -- 172.21.15.46:0/2612757204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e80ceb20 msgr2=0x7ff4e806c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.411+0000 7ff4efa00640 1 --2- 172.21.15.46:0/2612757204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e80ceb20 0x7ff4e806c950 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0009d90 tx=0x7ff4e001c970 comp rx=0 tx=0).stop 2024-03-29T08:21:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.411+0000 7ff4efa00640 1 -- 172.21.15.46:0/2612757204 shutdown_connections 2024-03-29T08:21:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.411+0000 7ff4efa00640 1 --2- 172.21.15.46:0/2612757204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e80ceb20 0x7ff4e806c950 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.411+0000 7ff4efa00640 1 --2- 172.21.15.46:0/2612757204 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff4e8144210 0x7ff4e80cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.411+0000 7ff4efa00640 1 -- 172.21.15.46:0/2612757204 >> 172.21.15.46:0/2612757204 conn(0x7ff4e8066110 msgr2=0x7ff4e8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.411+0000 7ff4efa00640 1 -- 172.21.15.46:0/2612757204 shutdown_connections 2024-03-29T08:21:15.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.411+0000 7ff4efa00640 1 -- 172.21.15.46:0/2612757204 wait complete. 2024-03-29T08:21:15.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4efa00640 1 Processor -- start 2024-03-29T08:21:15.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4efa00640 1 -- start start 2024-03-29T08:21:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4efa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e80ceb20 0x7ff4e815b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4efa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff4e8144210 0x7ff4e8143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4efa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e815ada0 0x7ff4e813bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4efa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e813f870 con 0x7ff4e80ceb20 2024-03-29T08:21:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4efa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e815f1d0 con 0x7ff4e815ada0 2024-03-29T08:21:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4efa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e813ebf0 con 0x7ff4e8144210 2024-03-29T08:21:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4ed000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e80ceb20 0x7ff4e815b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4ed000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e80ceb20 0x7ff4e815b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54416/0 (socket says 172.21.15.46:54416) 2024-03-29T08:21:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4ed000640 1 -- 172.21.15.46:0/3605621170 learned_addr learned my addr 172.21.15.46:0/3605621170 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:15.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4ed000640 1 -- 172.21.15.46:0/3605621170 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff4e8144210 msgr2=0x7ff4e8143180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4ed000640 1 --2- 172.21.15.46:0/3605621170 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff4e8144210 0x7ff4e8143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4ed000640 1 -- 172.21.15.46:0/3605621170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e815ada0 msgr2=0x7ff4e813bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4ed000640 1 --2- 172.21.15.46:0/3605621170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e815ada0 0x7ff4e813bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4ed000640 1 -- 172.21.15.46:0/3605621170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e0009a70 con 0x7ff4e80ceb20 2024-03-29T08:21:15.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4ed000640 1 --2- 172.21.15.46:0/3605621170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e80ceb20 0x7ff4e815b200 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00e930 tx=0x7ff4dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:15.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4e4c00640 1 -- 172.21.15.46:0/3605621170 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc009690 con 0x7ff4e80ceb20 2024-03-29T08:21:15.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4e4c00640 1 -- 172.21.15.46:0/3605621170 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4dc00ca00 con 0x7ff4e80ceb20 2024-03-29T08:21:15.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4e4c00640 1 -- 172.21.15.46:0/3605621170 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc00cd70 con 0x7ff4e80ceb20 2024-03-29T08:21:15.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4efa00640 1 -- 172.21.15.46:0/3605621170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e813eed0 con 0x7ff4e80ceb20 2024-03-29T08:21:15.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4efa00640 1 -- 172.21.15.46:0/3605621170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e813f490 con 0x7ff4e80ceb20 2024-03-29T08:21:15.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.415+0000 7ff4efa00640 1 -- 172.21.15.46:0/3605621170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e80ce280 con 0x7ff4e80ceb20 2024-03-29T08:21:15.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.419+0000 7ff4e4c00640 1 -- 172.21.15.46:0/3605621170 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff4dc013470 con 0x7ff4e80ceb20 2024-03-29T08:21:15.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.419+0000 7ff4e4c00640 1 --2- 172.21.15.46:0/3605621170 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff4c8066690 0x7ff4c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.419+0000 7ff4e4c00640 1 -- 172.21.15.46:0/3605621170 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4dc0843d0 con 0x7ff4e80ceb20 2024-03-29T08:21:15.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.419+0000 7ff4e7e00640 1 --2- 172.21.15.46:0/3605621170 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff4c8066690 0x7ff4c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.419+0000 7ff4e7e00640 1 --2- 172.21.15.46:0/3605621170 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff4c8066690 0x7ff4c8068b50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80ccdd0 tx=0x7ff4d4007970 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:15.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.423+0000 7ff4e4c00640 1 -- 172.21.15.46:0/3605621170 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff4dc051890 con 0x7ff4e80ceb20 2024-03-29T08:21:15.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.639+0000 7ff4efa00640 1 -- 172.21.15.46:0/3605621170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff4e8066c90 con 0x7ff4e80ceb20 2024-03-29T08:21:15.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.639+0000 7ff4e4c00640 1 -- 172.21.15.46:0/3605621170 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1270 (secure 0 0 0) 0x7ff4dc054eb0 con 0x7ff4e80ceb20 2024-03-29T08:21:15.643 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:15.643 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":207,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":63,"num_osds":8,"num_up_osds":8,"osd_up_since":1711700279,"num_in_osds":8,"osd_in_since":1711700270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":59},{"state_name":"active+clean","count":27},{"state_name":"unknown","count":11}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":249237504,"bytes_avail":772844875776,"bytes_total":773094113280,"unknown_pgs_ratio":0.11340206116437912,"inactive_pgs_ratio":0.60824739933013916},"fsmap":{"epoch":32,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":6051}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-03-29T08:21:04.082552+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-29T08:21:15.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.643+0000 7ff4efa00640 1 -- 172.21.15.46:0/3605621170 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff4c8066690 msgr2=0x7ff4c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.643+0000 7ff4efa00640 1 --2- 172.21.15.46:0/3605621170 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff4c8066690 0x7ff4c8068b50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80ccdd0 tx=0x7ff4d4007970 comp rx=0 tx=0).stop 2024-03-29T08:21:15.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.643+0000 7ff4efa00640 1 -- 172.21.15.46:0/3605621170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e80ceb20 msgr2=0x7ff4e815b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.643+0000 7ff4efa00640 1 --2- 172.21.15.46:0/3605621170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e80ceb20 0x7ff4e815b200 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc00e930 tx=0x7ff4dc00edf0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.643+0000 7ff4efa00640 1 -- 172.21.15.46:0/3605621170 shutdown_connections 2024-03-29T08:21:15.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.643+0000 7ff4efa00640 1 --2- 172.21.15.46:0/3605621170 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff4c8066690 0x7ff4c8068b50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.643+0000 7ff4efa00640 1 --2- 172.21.15.46:0/3605621170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e815ada0 0x7ff4e813bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.643+0000 7ff4efa00640 1 --2- 172.21.15.46:0/3605621170 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff4e8144210 0x7ff4e8143180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.643+0000 7ff4efa00640 1 --2- 172.21.15.46:0/3605621170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4e80ceb20 0x7ff4e815b200 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.643+0000 7ff4efa00640 1 -- 172.21.15.46:0/3605621170 >> 172.21.15.46:0/3605621170 conn(0x7ff4e8066110 msgr2=0x7ff4e813d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:15.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.643+0000 7ff4efa00640 1 -- 172.21.15.46:0/3605621170 shutdown_connections 2024-03-29T08:21:15.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.643+0000 7ff4efa00640 1 -- 172.21.15.46:0/3605621170 wait complete. 2024-03-29T08:21:15.664 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-29T08:21:15.664 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:21:15.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38ca400640 1 Processor -- start 2024-03-29T08:21:15.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38ca400640 1 -- start start 2024-03-29T08:21:15.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38ca400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f38c4144210 0x7f38c40cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38ca400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38c415ada0 0x7f38c40ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38ca400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38c40ce950 0x7f38c415e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38ca400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c40cede0 con 0x7f38c415ada0 2024-03-29T08:21:15.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38ca400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c40cb510 con 0x7f38c4144210 2024-03-29T08:21:15.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38ca400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c4070580 con 0x7f38c40ce950 2024-03-29T08:21:15.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38c2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38c415ada0 0x7f38c40ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38c2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38c415ada0 0x7f38c40ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54424/0 (socket says 172.21.15.46:54424) 2024-03-29T08:21:15.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38c2a00640 1 -- 172.21.15.46:0/2197604678 learned_addr learned my addr 172.21.15.46:0/2197604678 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:15.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38c3e00640 1 --2- 172.21.15.46:0/2197604678 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38c40ce950 0x7f38c415e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38c3400640 1 --2- 172.21.15.46:0/2197604678 >> v2:172.21.15.73:3300/0 conn(0x7f38c4144210 0x7f38c40cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38c2a00640 1 -- 172.21.15.46:0/2197604678 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38c40ce950 msgr2=0x7f38c415e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:15.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38c2a00640 1 --2- 172.21.15.46:0/2197604678 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38c40ce950 0x7f38c415e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38c2a00640 1 -- 172.21.15.46:0/2197604678 >> v2:172.21.15.73:3300/0 conn(0x7f38c4144210 msgr2=0x7f38c40cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:15.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38c2a00640 1 --2- 172.21.15.46:0/2197604678 >> v2:172.21.15.73:3300/0 conn(0x7f38c4144210 0x7f38c40cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.803+0000 7f38c2a00640 1 -- 172.21.15.46:0/2197604678 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c4070860 con 0x7f38c415ada0 2024-03-29T08:21:15.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.807+0000 7f38c2a00640 1 --2- 172.21.15.46:0/2197604678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c415ada0 0x7f38c40ce410 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f38c4071360 tx=0x7f38b001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368a691997cb291d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:15.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.807+0000 7f38c2000640 1 -- 172.21.15.46:0/2197604678 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b00ab600 con 0x7f38c415ada0 2024-03-29T08:21:15.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.807+0000 7f38c2000640 1 -- 172.21.15.46:0/2197604678 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b00a7050 con 0x7f38c415ada0 2024-03-29T08:21:15.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.807+0000 7f38c2000640 1 -- 172.21.15.46:0/2197604678 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b00ab980 con 0x7f38c415ada0 2024-03-29T08:21:15.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.807+0000 7f38ca400640 1 -- 172.21.15.46:0/2197604678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c415ada0 msgr2=0x7f38c40ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.807+0000 7f38ca400640 1 --2- 172.21.15.46:0/2197604678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c415ada0 0x7f38c40ce410 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f38c4071360 tx=0x7f38b001c260 comp rx=0 tx=0).stop 2024-03-29T08:21:15.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.807+0000 7f38ca400640 1 -- 172.21.15.46:0/2197604678 shutdown_connections 2024-03-29T08:21:15.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.807+0000 7f38ca400640 1 --2- 172.21.15.46:0/2197604678 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38c40ce950 0x7f38c415e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.807+0000 7f38ca400640 1 --2- 172.21.15.46:0/2197604678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c415ada0 0x7f38c40ce410 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.807+0000 7f38ca400640 1 --2- 172.21.15.46:0/2197604678 >> v2:172.21.15.73:3300/0 conn(0x7f38c4144210 0x7f38c40cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.807+0000 7f38ca400640 1 -- 172.21.15.46:0/2197604678 >> 172.21.15.46:0/2197604678 conn(0x7f38c4066110 msgr2=0x7f38c4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:15.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.807+0000 7f38ca400640 1 -- 172.21.15.46:0/2197604678 shutdown_connections 2024-03-29T08:21:15.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.807+0000 7f38ca400640 1 -- 172.21.15.46:0/2197604678 wait complete. 2024-03-29T08:21:15.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.807+0000 7f38ca400640 1 Processor -- start 2024-03-29T08:21:15.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38ca400640 1 -- start start 2024-03-29T08:21:15.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38ca400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38c40ce950 0x7f38c41650a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38ca400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38c4144210 0x7f38c40ac300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38ca400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c415ada0 0x7f38c4159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38ca400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c40ad540 con 0x7f38c415ada0 2024-03-29T08:21:15.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38ca400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c40ab8b0 con 0x7f38c40ce950 2024-03-29T08:21:15.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38ca400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c40ac840 con 0x7f38c4144210 2024-03-29T08:21:15.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c415ada0 0x7f38c4159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c415ada0 0x7f38c4159190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54438/0 (socket says 172.21.15.46:54438) 2024-03-29T08:21:15.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38c3e00640 1 -- 172.21.15.46:0/1000391746 learned_addr learned my addr 172.21.15.46:0/1000391746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:15.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38c3400640 1 --2- 172.21.15.46:0/1000391746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38c40ce950 0x7f38c41650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38c2a00640 1 --2- 172.21.15.46:0/1000391746 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38c4144210 0x7f38c40ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38c3e00640 1 -- 172.21.15.46:0/1000391746 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38c4144210 msgr2=0x7f38c40ac300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38c3e00640 1 --2- 172.21.15.46:0/1000391746 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38c4144210 0x7f38c40ac300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38c3e00640 1 -- 172.21.15.46:0/1000391746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38c40ce950 msgr2=0x7f38c41650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38c3e00640 1 --2- 172.21.15.46:0/1000391746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38c40ce950 0x7f38c41650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38c3e00640 1 -- 172.21.15.46:0/1000391746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b00094d0 con 0x7f38c415ada0 2024-03-29T08:21:15.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38c3e00640 1 --2- 172.21.15.46:0/1000391746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c415ada0 0x7f38c4159190 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f38bc00eed0 tx=0x7f38bc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:15.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38bb400640 1 -- 172.21.15.46:0/1000391746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc010040 con 0x7f38c415ada0 2024-03-29T08:21:15.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38bb400640 1 -- 172.21.15.46:0/1000391746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc014400 con 0x7f38c415ada0 2024-03-29T08:21:15.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38bb400640 1 -- 172.21.15.46:0/1000391746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc009ea0 con 0x7f38c415ada0 2024-03-29T08:21:15.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38ca400640 1 -- 172.21.15.46:0/1000391746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c413eaf0 con 0x7f38c415ada0 2024-03-29T08:21:15.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38ca400640 1 -- 172.21.15.46:0/1000391746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c40abb00 con 0x7f38c415ada0 2024-03-29T08:21:15.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.811+0000 7f38ca400640 1 -- 172.21.15.46:0/1000391746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c40ce280 con 0x7f38c415ada0 2024-03-29T08:21:15.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.815+0000 7f38bb400640 1 -- 172.21.15.46:0/1000391746 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f38bc0147b0 con 0x7f38c415ada0 2024-03-29T08:21:15.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.815+0000 7f38bb400640 1 --2- 172.21.15.46:0/1000391746 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38a40666c0 0x7f38a4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:15.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.815+0000 7f38bb400640 1 -- 172.21.15.46:0/1000391746 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38bc0847b0 con 0x7f38c415ada0 2024-03-29T08:21:15.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.815+0000 7f38c3400640 1 --2- 172.21.15.46:0/1000391746 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38a40666c0 0x7f38a4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:15.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.815+0000 7f38c3400640 1 --2- 172.21.15.46:0/1000391746 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38a40666c0 0x7f38a4068b80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009b60 tx=0x7f38b4009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:15.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.819+0000 7f38bb400640 1 -- 172.21.15.46:0/1000391746 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f38bc051b30 con 0x7f38c415ada0 2024-03-29T08:21:15.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.975+0000 7f38ca400640 1 -- 172.21.15.46:0/1000391746 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38c40cbdf0 con 0x7f38a40666c0 2024-03-29T08:21:15.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.987+0000 7f38bb400640 1 -- 172.21.15.46:0/1000391746 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284914 (secure 0 0 0) 0x7f38b4011ac0 con 0x7f38a40666c0 2024-03-29T08:21:15.990 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:15.992 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:21:15.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.987+0000 7f38ca400640 1 -- 172.21.15.46:0/1000391746 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38a40666c0 msgr2=0x7f38a4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.987+0000 7f38ca400640 1 --2- 172.21.15.46:0/1000391746 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38a40666c0 0x7f38a4068b80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f38b4009b60 tx=0x7f38b4009190 comp rx=0 tx=0).stop 2024-03-29T08:21:15.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.987+0000 7f38ca400640 1 -- 172.21.15.46:0/1000391746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c415ada0 msgr2=0x7f38c4159190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:15.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.987+0000 7f38ca400640 1 --2- 172.21.15.46:0/1000391746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c415ada0 0x7f38c4159190 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f38bc00eed0 tx=0x7f38bc00b480 comp rx=0 tx=0).stop 2024-03-29T08:21:15.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.987+0000 7f38ca400640 1 -- 172.21.15.46:0/1000391746 shutdown_connections 2024-03-29T08:21:15.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.987+0000 7f38ca400640 1 --2- 172.21.15.46:0/1000391746 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38a40666c0 0x7f38a4068b80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.987+0000 7f38ca400640 1 --2- 172.21.15.46:0/1000391746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38c415ada0 0x7f38c4159190 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.987+0000 7f38ca400640 1 --2- 172.21.15.46:0/1000391746 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38c4144210 0x7f38c40ac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.987+0000 7f38ca400640 1 --2- 172.21.15.46:0/1000391746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38c40ce950 0x7f38c41650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:15.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.987+0000 7f38ca400640 1 -- 172.21.15.46:0/1000391746 >> 172.21.15.46:0/1000391746 conn(0x7f38c4066110 msgr2=0x7f38c4141910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:15.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.987+0000 7f38ca400640 1 -- 172.21.15.46:0/1000391746 shutdown_connections 2024-03-29T08:21:15.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:15.987+0000 7f38ca400640 1 -- 172.21.15.46:0/1000391746 wait complete. 2024-03-29T08:21:16.011 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-03-29T08:21:14.084517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":172,"acting":172,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243396,"kb_used_data":6752,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754731324,"statfs":{"total":773094113280,"available":772844875776,"internally_reserved":0,"allocated":6914048,"data_stored":3542294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":242208313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.157425"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.941725+0000","last_change":"2024-03-29T08:21:09.941725+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.941725+0000","last_undegraded":"2024-03-29T08:21:09.941725+0000","last_fullsized":"2024-03-29T08:21:09.941725+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:16:15.767217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936961+0000","last_change":"2024-03-29T08:21:09.936961+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936961+0000","last_undegraded":"2024-03-29T08:21:09.936961+0000","last_fullsized":"2024-03-29T08:21:09.936961+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:05:22.465098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938613+0000","last_change":"2024-03-29T08:21:09.938613+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938613+0000","last_undegraded":"2024-03-29T08:21:09.938613+0000","last_fullsized":"2024-03-29T08:21:09.938613+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:47:48.297292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.933020+0000","last_change":"2024-03-29T08:21:09.933020+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.933020+0000","last_undegraded":"2024-03-29T08:21:09.933020+0000","last_fullsized":"2024-03-29T08:21:09.933020+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:51.414360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936757+0000","last_change":"2024-03-29T08:21:09.936757+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936757+0000","last_undegraded":"2024-03-29T08:21:09.936757+0000","last_fullsized":"2024-03-29T08:21:09.936757+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:36:29.239728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939144+0000","last_change":"2024-03-29T08:21:09.939144+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939144+0000","last_undegraded":"2024-03-29T08:21:09.939144+0000","last_fullsized":"2024-03-29T08:21:09.939144+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:35:20.420999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.933702+0000","last_change":"2024-03-29T08:21:09.933702+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.933702+0000","last_undegraded":"2024-03-29T08:21:09.933702+0000","last_fullsized":"2024-03-29T08:21:09.933702+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:30:03.234601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939490+0000","last_change":"2024-03-29T08:21:09.939490+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939490+0000","last_undegraded":"2024-03-29T08:21:09.939490+0000","last_fullsized":"2024-03-29T08:21:09.939490+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:40.843359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.940289+0000","last_change":"2024-03-29T08:21:09.940289+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.940289+0000","last_undegraded":"2024-03-29T08:21:09.940289+0000","last_fullsized":"2024-03-29T08:21:09.940289+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:54.034461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.934375+0000","last_change":"2024-03-29T08:21:09.934375+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.934375+0000","last_undegraded":"2024-03-29T08:21:09.934375+0000","last_fullsized":"2024-03-29T08:21:09.934375+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:58.105380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.940141+0000","last_change":"2024-03-29T08:21:09.940141+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.940141+0000","last_undegraded":"2024-03-29T08:21:09.940141+0000","last_fullsized":"2024-03-29T08:21:09.940141+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:20:51.330512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.942430+0000","last_change":"2024-03-29T08:21:09.942430+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.942430+0000","last_undegraded":"2024-03-29T08:21:09.942430+0000","last_fullsized":"2024-03-29T08:21:09.942430+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:35:20.420999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939640+0000","last_change":"2024-03-29T08:21:09.939640+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939640+0000","last_undegraded":"2024-03-29T08:21:09.939640+0000","last_fullsized":"2024-03-29T08:21:09.939640+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:49:07.458847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.940293+0000","last_change":"2024-03-29T08:21:09.940293+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.940293+0000","last_undegraded":"2024-03-29T08:21:09.940293+0000","last_fullsized":"2024-03-29T08:21:09.940293+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:39.458643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938066+0000","last_change":"2024-03-29T08:21:09.938066+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938066+0000","last_undegraded":"2024-03-29T08:21:09.938066+0000","last_fullsized":"2024-03-29T08:21:09.938066+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:01:52.376061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.941263+0000","last_change":"2024-03-29T08:21:09.941263+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.941263+0000","last_undegraded":"2024-03-29T08:21:09.941263+0000","last_fullsized":"2024-03-29T08:21:09.941263+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:39.458643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939231+0000","last_change":"2024-03-29T08:21:09.939231+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939231+0000","last_undegraded":"2024-03-29T08:21:09.939231+0000","last_fullsized":"2024-03-29T08:21:09.939231+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:33:19.998323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.942709+0000","last_change":"2024-03-29T08:21:09.942709+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.942709+0000","last_undegraded":"2024-03-29T08:21:09.942709+0000","last_fullsized":"2024-03-29T08:21:09.942709+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:58:48.804311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.934214+0000","last_change":"2024-03-29T08:21:09.934214+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.934214+0000","last_undegraded":"2024-03-29T08:21:09.934214+0000","last_fullsized":"2024-03-29T08:21:09.934214+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:01.741332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.942033+0000","last_change":"2024-03-29T08:21:09.942033+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.942033+0000","last_undegraded":"2024-03-29T08:21:09.942033+0000","last_fullsized":"2024-03-29T08:21:09.942033+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:44.680234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.934012+0000","last_change":"2024-03-29T08:21:09.934012+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.934012+0000","last_undegraded":"2024-03-29T08:21:09.934012+0000","last_fullsized":"2024-03-29T08:21:09.934012+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:15.253098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.933212+0000","last_change":"2024-03-29T08:21:09.933212+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.933212+0000","last_undegraded":"2024-03-29T08:21:09.933212+0000","last_fullsized":"2024-03-29T08:21:09.933212+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:25:00.595704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938923+0000","last_change":"2024-03-29T08:21:09.938923+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938923+0000","last_undegraded":"2024-03-29T08:21:09.938923+0000","last_fullsized":"2024-03-29T08:21:09.938923+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:58.831468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937337+0000","last_change":"2024-03-29T08:21:09.937337+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937337+0000","last_undegraded":"2024-03-29T08:21:09.937337+0000","last_fullsized":"2024-03-29T08:21:09.937337+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:59:24.384598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.943515+0000","last_change":"2024-03-29T08:21:09.943515+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.943515+0000","last_undegraded":"2024-03-29T08:21:09.943515+0000","last_fullsized":"2024-03-29T08:21:09.943515+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:53.450623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936629+0000","last_change":"2024-03-29T08:21:09.936629+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936629+0000","last_undegraded":"2024-03-29T08:21:09.936629+0000","last_fullsized":"2024-03-29T08:21:09.936629+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:49.605189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.932435+0000","last_change":"2024-03-29T08:21:09.932435+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.932435+0000","last_undegraded":"2024-03-29T08:21:09.932435+0000","last_fullsized":"2024-03-29T08:21:09.932435+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:45.698306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.943149+0000","last_change":"2024-03-29T08:21:09.943149+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.943149+0000","last_undegraded":"2024-03-29T08:21:09.943149+0000","last_fullsized":"2024-03-29T08:21:09.943149+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:01:19.390653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.934024+0000","last_change":"2024-03-29T08:21:09.934024+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.934024+0000","last_undegraded":"2024-03-29T08:21:09.934024+0000","last_fullsized":"2024-03-29T08:21:09.934024+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:54.034461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939827+0000","last_change":"2024-03-29T08:21:09.939827+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939827+0000","last_undegraded":"2024-03-29T08:21:09.939827+0000","last_fullsized":"2024-03-29T08:21:09.939827+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:53.450623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956239+0000","last_change":"2024-03-29T08:21:09.953242+0000","last_active":"2024-03-29T08:21:09.956239+0000","last_peered":"2024-03-29T08:21:09.956239+0000","last_clean":"2024-03-29T08:21:09.956239+0000","last_became_active":"2024-03-29T08:21:09.952978+0000","last_became_peered":"2024-03-29T08:21:09.952978+0000","last_unstale":"2024-03-29T08:21:09.956239+0000","last_undegraded":"2024-03-29T08:21:09.956239+0000","last_fullsized":"2024-03-29T08:21:09.956239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:43.677354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937754+0000","last_change":"2024-03-29T08:21:09.937754+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937754+0000","last_undegraded":"2024-03-29T08:21:09.937754+0000","last_fullsized":"2024-03-29T08:21:09.937754+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:40.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.957328+0000","last_change":"2024-03-29T08:21:09.955717+0000","last_active":"2024-03-29T08:21:09.957328+0000","last_peered":"2024-03-29T08:21:09.957328+0000","last_clean":"2024-03-29T08:21:09.957328+0000","last_became_active":"2024-03-29T08:21:09.955450+0000","last_became_peered":"2024-03-29T08:21:09.955450+0000","last_unstale":"2024-03-29T08:21:09.957328+0000","last_undegraded":"2024-03-29T08:21:09.957328+0000","last_fullsized":"2024-03-29T08:21:09.957328+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:47:40.877803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.952711+0000","last_change":"2024-03-29T08:21:09.950482+0000","last_active":"2024-03-29T08:21:09.952711+0000","last_peered":"2024-03-29T08:21:09.952711+0000","last_clean":"2024-03-29T08:21:09.952711+0000","last_became_active":"2024-03-29T08:21:09.950190+0000","last_became_peered":"2024-03-29T08:21:09.950190+0000","last_unstale":"2024-03-29T08:21:09.952711+0000","last_undegraded":"2024-03-29T08:21:09.952711+0000","last_fullsized":"2024-03-29T08:21:09.952711+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:07:20.233898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938835+0000","last_change":"2024-03-29T08:21:09.938835+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938835+0000","last_undegraded":"2024-03-29T08:21:09.938835+0000","last_fullsized":"2024-03-29T08:21:09.938835+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:21:23.293798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.944401+0000","last_change":"2024-03-29T08:21:09.942997+0000","last_active":"2024-03-29T08:21:09.944401+0000","last_peered":"2024-03-29T08:21:09.944401+0000","last_clean":"2024-03-29T08:21:09.944401+0000","last_became_active":"2024-03-29T08:21:09.942730+0000","last_became_peered":"2024-03-29T08:21:09.942730+0000","last_unstale":"2024-03-29T08:21:09.944401+0000","last_undegraded":"2024-03-29T08:21:09.944401+0000","last_fullsized":"2024-03-29T08:21:09.944401+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:42:48.592612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.947598+0000","last_change":"2024-03-29T08:21:09.944500+0000","last_active":"2024-03-29T08:21:09.947598+0000","last_peered":"2024-03-29T08:21:09.947598+0000","last_clean":"2024-03-29T08:21:09.947598+0000","last_became_active":"2024-03-29T08:21:09.944236+0000","last_became_peered":"2024-03-29T08:21:09.944236+0000","last_unstale":"2024-03-29T08:21:09.947598+0000","last_undegraded":"2024-03-29T08:21:09.947598+0000","last_fullsized":"2024-03-29T08:21:09.947598+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:06:08.420341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.935582+0000","last_change":"2024-03-29T08:21:09.935582+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.935582+0000","last_undegraded":"2024-03-29T08:21:09.935582+0000","last_fullsized":"2024-03-29T08:21:09.935582+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:05:09.416516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.952308+0000","last_change":"2024-03-29T08:21:09.946818+0000","last_active":"2024-03-29T08:21:09.952308+0000","last_peered":"2024-03-29T08:21:09.952308+0000","last_clean":"2024-03-29T08:21:09.952308+0000","last_became_active":"2024-03-29T08:21:09.946555+0000","last_became_peered":"2024-03-29T08:21:09.946555+0000","last_unstale":"2024-03-29T08:21:09.952308+0000","last_undegraded":"2024-03-29T08:21:09.952308+0000","last_fullsized":"2024-03-29T08:21:09.952308+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:27:46.452557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937385+0000","last_change":"2024-03-29T08:21:09.937385+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937385+0000","last_undegraded":"2024-03-29T08:21:09.937385+0000","last_fullsized":"2024-03-29T08:21:09.937385+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:58.105380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956350+0000","last_change":"2024-03-29T08:21:09.954162+0000","last_active":"2024-03-29T08:21:09.956350+0000","last_peered":"2024-03-29T08:21:09.956350+0000","last_clean":"2024-03-29T08:21:09.956350+0000","last_became_active":"2024-03-29T08:21:09.953849+0000","last_became_peered":"2024-03-29T08:21:09.953849+0000","last_unstale":"2024-03-29T08:21:09.956350+0000","last_undegraded":"2024-03-29T08:21:09.956350+0000","last_fullsized":"2024-03-29T08:21:09.956350+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:52.803477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937620+0000","last_change":"2024-03-29T08:21:09.937620+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937620+0000","last_undegraded":"2024-03-29T08:21:09.937620+0000","last_fullsized":"2024-03-29T08:21:09.937620+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:19:16.700510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956277+0000","last_change":"2024-03-29T08:21:09.954120+0000","last_active":"2024-03-29T08:21:09.956277+0000","last_peered":"2024-03-29T08:21:09.956277+0000","last_clean":"2024-03-29T08:21:09.956277+0000","last_became_active":"2024-03-29T08:21:09.953375+0000","last_became_peered":"2024-03-29T08:21:09.953375+0000","last_unstale":"2024-03-29T08:21:09.956277+0000","last_undegraded":"2024-03-29T08:21:09.956277+0000","last_fullsized":"2024-03-29T08:21:09.956277+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:48.602309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.943037+0000","last_change":"2024-03-29T08:21:09.943037+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.943037+0000","last_undegraded":"2024-03-29T08:21:09.943037+0000","last_fullsized":"2024-03-29T08:21:09.943037+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:24.166586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.953570+0000","last_change":"2024-03-29T08:21:09.951304+0000","last_active":"2024-03-29T08:21:09.953570+0000","last_peered":"2024-03-29T08:21:09.953570+0000","last_clean":"2024-03-29T08:21:09.953570+0000","last_became_active":"2024-03-29T08:21:09.951067+0000","last_became_peered":"2024-03-29T08:21:09.951067+0000","last_unstale":"2024-03-29T08:21:09.953570+0000","last_undegraded":"2024-03-29T08:21:09.953570+0000","last_fullsized":"2024-03-29T08:21:09.953570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:13:08.676082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938641+0000","last_change":"2024-03-29T08:21:09.938641+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938641+0000","last_undegraded":"2024-03-29T08:21:09.938641+0000","last_fullsized":"2024-03-29T08:21:09.938641+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:08:39.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.939701+0000","last_change":"2024-03-29T08:21:08.939701+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.939701+0000","last_undegraded":"2024-03-29T08:21:08.939701+0000","last_fullsized":"2024-03-29T08:21:08.939701+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:08:44.015717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.945733+0000","last_change":"2024-03-29T08:21:09.944508+0000","last_active":"2024-03-29T08:21:09.945733+0000","last_peered":"2024-03-29T08:21:09.945733+0000","last_clean":"2024-03-29T08:21:09.945733+0000","last_became_active":"2024-03-29T08:21:09.944191+0000","last_became_peered":"2024-03-29T08:21:09.944191+0000","last_unstale":"2024-03-29T08:21:09.945733+0000","last_undegraded":"2024-03-29T08:21:09.945733+0000","last_fullsized":"2024-03-29T08:21:09.945733+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:25:36.003763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.932991+0000","last_change":"2024-03-29T08:21:09.932991+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.932991+0000","last_undegraded":"2024-03-29T08:21:09.932991+0000","last_fullsized":"2024-03-29T08:21:09.932991+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:56:39.434467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.955648+0000","last_change":"2024-03-29T08:21:09.953728+0000","last_active":"2024-03-29T08:21:09.955648+0000","last_peered":"2024-03-29T08:21:09.955648+0000","last_clean":"2024-03-29T08:21:09.955648+0000","last_became_active":"2024-03-29T08:21:09.953431+0000","last_became_peered":"2024-03-29T08:21:09.953431+0000","last_unstale":"2024-03-29T08:21:09.955648+0000","last_undegraded":"2024-03-29T08:21:09.955648+0000","last_fullsized":"2024-03-29T08:21:09.955648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:38:17.880237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.939449+0000","last_change":"2024-03-29T08:21:09.939449+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.939449+0000","last_undegraded":"2024-03-29T08:21:09.939449+0000","last_fullsized":"2024-03-29T08:21:09.939449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:52:34.720375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.934101+0000","last_change":"2024-03-29T08:21:08.934101+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.934101+0000","last_undegraded":"2024-03-29T08:21:08.934101+0000","last_fullsized":"2024-03-29T08:21:08.934101+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:03:13.729828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937954+0000","last_change":"2024-03-29T08:21:09.937954+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937954+0000","last_undegraded":"2024-03-29T08:21:09.937954+0000","last_fullsized":"2024-03-29T08:21:09.937954+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:21:28.741076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.950524+0000","last_change":"2024-03-29T08:21:09.948902+0000","last_active":"2024-03-29T08:21:09.950524+0000","last_peered":"2024-03-29T08:21:09.950524+0000","last_clean":"2024-03-29T08:21:09.950524+0000","last_became_active":"2024-03-29T08:21:09.948613+0000","last_became_peered":"2024-03-29T08:21:09.948613+0000","last_unstale":"2024-03-29T08:21:09.950524+0000","last_undegraded":"2024-03-29T08:21:09.950524+0000","last_fullsized":"2024-03-29T08:21:09.950524+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:00.619680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938522+0000","last_change":"2024-03-29T08:21:09.938522+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938522+0000","last_undegraded":"2024-03-29T08:21:09.938522+0000","last_fullsized":"2024-03-29T08:21:09.938522+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:01:48.373155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.947564+0000","last_change":"2024-03-29T08:21:09.946582+0000","last_active":"2024-03-29T08:21:09.947564+0000","last_peered":"2024-03-29T08:21:09.947564+0000","last_clean":"2024-03-29T08:21:09.947564+0000","last_became_active":"2024-03-29T08:21:09.946276+0000","last_became_peered":"2024-03-29T08:21:09.946276+0000","last_unstale":"2024-03-29T08:21:09.947564+0000","last_undegraded":"2024-03-29T08:21:09.947564+0000","last_fullsized":"2024-03-29T08:21:09.947564+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:01:32.117594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.932610+0000","last_change":"2024-03-29T08:21:09.932610+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.932610+0000","last_undegraded":"2024-03-29T08:21:09.932610+0000","last_fullsized":"2024-03-29T08:21:09.932610+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:11:50.905254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.955654+0000","last_change":"2024-03-29T08:21:09.953517+0000","last_active":"2024-03-29T08:21:09.955654+0000","last_peered":"2024-03-29T08:21:09.955654+0000","last_clean":"2024-03-29T08:21:09.955654+0000","last_became_active":"2024-03-29T08:21:09.953205+0000","last_became_peered":"2024-03-29T08:21:09.953205+0000","last_unstale":"2024-03-29T08:21:09.955654+0000","last_undegraded":"2024-03-29T08:21:09.955654+0000","last_fullsized":"2024-03-29T08:21:09.955654+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:11:14.860644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936862+0000","last_change":"2024-03-29T08:21:09.936862+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936862+0000","last_undegraded":"2024-03-29T08:21:09.936862+0000","last_fullsized":"2024-03-29T08:21:09.936862+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:15.253098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.957279+0000","last_change":"2024-03-29T08:21:09.954705+0000","last_active":"2024-03-29T08:21:09.957279+0000","last_peered":"2024-03-29T08:21:09.957279+0000","last_clean":"2024-03-29T08:21:09.957279+0000","last_became_active":"2024-03-29T08:21:09.954314+0000","last_became_peered":"2024-03-29T08:21:09.954314+0000","last_unstale":"2024-03-29T08:21:09.957279+0000","last_undegraded":"2024-03-29T08:21:09.957279+0000","last_fullsized":"2024-03-29T08:21:09.957279+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:24:09.856034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936946+0000","last_change":"2024-03-29T08:21:09.936946+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936946+0000","last_undegraded":"2024-03-29T08:21:09.936946+0000","last_fullsized":"2024-03-29T08:21:09.936946+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:05:09.416516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.934157+0000","last_change":"2024-03-29T08:21:08.934157+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.934157+0000","last_undegraded":"2024-03-29T08:21:08.934157+0000","last_fullsized":"2024-03-29T08:21:08.934157+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:50:03.760523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.941996+0000","last_change":"2024-03-29T08:21:09.941996+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.941996+0000","last_undegraded":"2024-03-29T08:21:09.941996+0000","last_fullsized":"2024-03-29T08:21:09.941996+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:50:48.557652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.948833+0000","last_change":"2024-03-29T08:21:09.947148+0000","last_active":"2024-03-29T08:21:09.948833+0000","last_peered":"2024-03-29T08:21:09.948833+0000","last_clean":"2024-03-29T08:21:09.948833+0000","last_became_active":"2024-03-29T08:21:09.946610+0000","last_became_peered":"2024-03-29T08:21:09.946610+0000","last_unstale":"2024-03-29T08:21:09.948833+0000","last_undegraded":"2024-03-29T08:21:09.948833+0000","last_fullsized":"2024-03-29T08:21:09.948833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:51.605750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.932484+0000","last_change":"2024-03-29T08:21:09.932484+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.932484+0000","last_undegraded":"2024-03-29T08:21:09.932484+0000","last_fullsized":"2024-03-29T08:21:09.932484+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:37:31.250178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.936989+0000","last_change":"2024-03-29T08:21:08.936989+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.936989+0000","last_undegraded":"2024-03-29T08:21:08.936989+0000","last_fullsized":"2024-03-29T08:21:08.936989+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:26:31.314621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.937474+0000","last_change":"2024-03-29T08:21:09.937474+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.937474+0000","last_undegraded":"2024-03-29T08:21:09.937474+0000","last_fullsized":"2024-03-29T08:21:09.937474+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:24:10.858914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956169+0000","last_change":"2024-03-29T08:21:09.953120+0000","last_active":"2024-03-29T08:21:09.956169+0000","last_peered":"2024-03-29T08:21:09.956169+0000","last_clean":"2024-03-29T08:21:09.956169+0000","last_became_active":"2024-03-29T08:21:09.952860+0000","last_became_peered":"2024-03-29T08:21:09.952860+0000","last_unstale":"2024-03-29T08:21:09.956169+0000","last_undegraded":"2024-03-29T08:21:09.956169+0000","last_fullsized":"2024-03-29T08:21:09.956169+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:38.455763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.934110+0000","last_change":"2024-03-29T08:21:08.934110+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.934110+0000","last_undegraded":"2024-03-29T08:21:08.934110+0000","last_fullsized":"2024-03-29T08:21:08.934110+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:05:43.897022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.948860+0000","last_change":"2024-03-29T08:21:09.947277+0000","last_active":"2024-03-29T08:21:09.948860+0000","last_peered":"2024-03-29T08:21:09.948860+0000","last_clean":"2024-03-29T08:21:09.948860+0000","last_became_active":"2024-03-29T08:21:09.945398+0000","last_became_peered":"2024-03-29T08:21:09.945398+0000","last_unstale":"2024-03-29T08:21:09.948860+0000","last_undegraded":"2024-03-29T08:21:09.948860+0000","last_fullsized":"2024-03-29T08:21:09.948860+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:36:29.788331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.935957+0000","last_change":"2024-03-29T08:21:09.935957+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.935957+0000","last_undegraded":"2024-03-29T08:21:09.935957+0000","last_fullsized":"2024-03-29T08:21:09.935957+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:18:28.430663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.957304+0000","last_change":"2024-03-29T08:21:09.955367+0000","last_active":"2024-03-29T08:21:09.957304+0000","last_peered":"2024-03-29T08:21:09.957304+0000","last_clean":"2024-03-29T08:21:09.957304+0000","last_became_active":"2024-03-29T08:21:09.954965+0000","last_became_peered":"2024-03-29T08:21:09.954965+0000","last_unstale":"2024-03-29T08:21:09.957304+0000","last_undegraded":"2024-03-29T08:21:09.957304+0000","last_fullsized":"2024-03-29T08:21:09.957304+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:57.828588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936065+0000","last_change":"2024-03-29T08:21:09.936065+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936065+0000","last_undegraded":"2024-03-29T08:21:09.936065+0000","last_fullsized":"2024-03-29T08:21:09.936065+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:01.741332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.952538+0000","last_change":"2024-03-29T08:21:09.948192+0000","last_active":"2024-03-29T08:21:09.952538+0000","last_peered":"2024-03-29T08:21:09.952538+0000","last_clean":"2024-03-29T08:21:09.952538+0000","last_became_active":"2024-03-29T08:21:09.947926+0000","last_became_peered":"2024-03-29T08:21:09.947926+0000","last_unstale":"2024-03-29T08:21:09.952538+0000","last_undegraded":"2024-03-29T08:21:09.952538+0000","last_fullsized":"2024-03-29T08:21:09.952538+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:33:58.392690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936565+0000","last_change":"2024-03-29T08:21:09.936565+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936565+0000","last_undegraded":"2024-03-29T08:21:09.936565+0000","last_fullsized":"2024-03-29T08:21:09.936565+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:22:08.727894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.950245+0000","last_change":"2024-03-29T08:21:09.947847+0000","last_active":"2024-03-29T08:21:09.950245+0000","last_peered":"2024-03-29T08:21:09.950245+0000","last_clean":"2024-03-29T08:21:09.950245+0000","last_became_active":"2024-03-29T08:21:09.946845+0000","last_became_peered":"2024-03-29T08:21:09.946845+0000","last_unstale":"2024-03-29T08:21:09.950245+0000","last_undegraded":"2024-03-29T08:21:09.950245+0000","last_fullsized":"2024-03-29T08:21:09.950245+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:19:13.916686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.943473+0000","last_change":"2024-03-29T08:21:09.938352+0000","last_active":"2024-03-29T08:21:09.943473+0000","last_peered":"2024-03-29T08:21:09.943473+0000","last_clean":"2024-03-29T08:21:09.943473+0000","last_became_active":"2024-03-29T08:21:09.937930+0000","last_became_peered":"2024-03-29T08:21:09.937930+0000","last_unstale":"2024-03-29T08:21:09.943473+0000","last_undegraded":"2024-03-29T08:21:09.943473+0000","last_fullsized":"2024-03-29T08:21:09.943473+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:23.163706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.938692+0000","last_change":"2024-03-29T08:21:09.938692+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.938692+0000","last_undegraded":"2024-03-29T08:21:09.938692+0000","last_fullsized":"2024-03-29T08:21:09.938692+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:44.680234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.956168+0000","last_change":"2024-03-29T08:21:09.953317+0000","last_active":"2024-03-29T08:21:09.956168+0000","last_peered":"2024-03-29T08:21:09.956168+0000","last_clean":"2024-03-29T08:21:09.956168+0000","last_became_active":"2024-03-29T08:21:09.953068+0000","last_became_peered":"2024-03-29T08:21:09.953068+0000","last_unstale":"2024-03-29T08:21:09.956168+0000","last_undegraded":"2024-03-29T08:21:09.956168+0000","last_fullsized":"2024-03-29T08:21:09.956168+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:39.840479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.941589+0000","last_change":"2024-03-29T08:21:09.941589+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.941589+0000","last_undegraded":"2024-03-29T08:21:09.941589+0000","last_fullsized":"2024-03-29T08:21:09.941589+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:40.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.952730+0000","last_change":"2024-03-29T08:21:09.950160+0000","last_active":"2024-03-29T08:21:09.952730+0000","last_peered":"2024-03-29T08:21:09.952730+0000","last_clean":"2024-03-29T08:21:09.952730+0000","last_became_active":"2024-03-29T08:21:09.949853+0000","last_became_peered":"2024-03-29T08:21:09.949853+0000","last_unstale":"2024-03-29T08:21:09.952730+0000","last_undegraded":"2024-03-29T08:21:09.952730+0000","last_fullsized":"2024-03-29T08:21:09.952730+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:59:23.381718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.947411+0000","last_change":"2024-03-29T08:21:09.945534+0000","last_active":"2024-03-29T08:21:09.947411+0000","last_peered":"2024-03-29T08:21:09.947411+0000","last_clean":"2024-03-29T08:21:09.947411+0000","last_became_active":"2024-03-29T08:21:09.943748+0000","last_became_peered":"2024-03-29T08:21:09.943748+0000","last_unstale":"2024-03-29T08:21:09.947411+0000","last_undegraded":"2024-03-29T08:21:09.947411+0000","last_fullsized":"2024-03-29T08:21:09.947411+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:05:06.965072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-03-29T08:21:09.936902+0000","last_change":"2024-03-29T08:21:09.936902+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.936902+0000","last_undegraded":"2024-03-29T08:21:09.936902+0000","last_fullsized":"2024-03-29T08:21:09.936902+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:38:18.883117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":60,"state":"creating+peering","last_fresh":"2024-03-29T08:21:08.934050+0000","last_change":"2024-03-29T08:21:08.934050+0000","last_active":"2024-03-29T08:21:08.922741+0000","last_peered":"2024-03-29T08:21:08.922741+0000","last_clean":"2024-03-29T08:21:08.922741+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:08.934050+0000","last_undegraded":"2024-03-29T08:21:08.934050+0000","last_fullsized":"2024-03-29T08:21:08.934050+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:00:36.010161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-29T08:21:09.925622+0000","last_change":"2024-03-29T08:21:09.925622+0000","last_active":"2024-03-29T08:21:09.925622+0000","last_peered":"2024-03-29T08:21:09.925622+0000","last_clean":"2024-03-29T08:21:09.925622+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:21:09.925622+0000","last_undegraded":"2024-03-29T08:21:09.925622+0000","last_fullsized":"2024-03-29T08:21:09.925622+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-03-29T08:21:09.955570+0000","last_change":"2024-03-29T08:21:09.953812+0000","last_active":"2024-03-29T08:21:09.955570+0000","last_peered":"2024-03-29T08:21:09.955570+0000","last_clean":"2024-03-29T08:21:09.955570+0000","last_became_active":"2024-03-29T08:21:09.953475+0000","last_became_peered":"2024-03-29T08:21:09.953475+0000","last_unstale":"2024-03-29T08:21:09.955570+0000","last_undegraded":"2024-03-29T08:21:09.955570+0000","last_fullsized":"2024-03-29T08:21:09.955570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:05:21.462217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":196,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-03-29T08:21:08.935647+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:21:08.935647+0000","last_peered":"2024-03-29T08:21:08.935647+0000","last_clean":"2024-03-29T08:21:08.935647+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:21:08.935647+0000","last_undegraded":"2024-03-29T08:21:08.935647+0000","last_fullsized":"2024-03-29T08:21:08.935647+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":106,"acting":106,"num_store_stats":7},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":293626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67600000000000005,"15min":0.65200000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93400000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.69999999999999996,"15min":0.67600000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1200000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.68700000000000006}]},{"osd":1,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.67100000000000004,"15min":0.64800000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.66800000000000004,"15min":0.63400000000000001},"min":{"1min":0.52900000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.0189999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.82599999999999996}]},{"osd":2,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.66200000000000003,"15min":0.63300000000000001},"min":{"1min":0.55900000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.0089999999999999,"5min":1.085,"15min":1.085},"last":0.754},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.69299999999999995,"15min":0.66500000000000004},"min":{"1min":0.58499999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1000000000000001,"5min":1.288,"15min":1.288},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71099999999999997,"15min":0.69799999999999995},"min":{"1min":0.57899999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92800000000000005,"5min":1.018,"15min":1.018},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.745,"15min":0.73099999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1659999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.73499999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83499999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70199999999999996,"15min":0.71999999999999997},"min":{"1min":0.46100000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0009999999999999,"5min":1.123,"15min":1.123},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.95799999999999996,"5min":1.056,"15min":1.056},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.46300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.77200000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73399999999999999,"15min":0.72999999999999998},"min":{"1min":0.48999999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.755},"min":{"1min":0.52300000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.84199999999999997}]}]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":293626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.66000000000000003,"15min":0.63300000000000001},"min":{"1min":0.54000000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70499999999999996,"15min":0.68500000000000005},"min":{"1min":0.57899999999999996,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.71399999999999997}]},{"osd":1,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.64100000000000001,"15min":0.61499999999999999},"min":{"1min":0.58599999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.64800000000000002,"15min":0.627},"min":{"1min":0.51600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.70299999999999996,"15min":0.67500000000000004},"min":{"1min":0.52300000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.54300000000000004,"5min":0.435,"15min":0.435},"max":{"1min":0.91200000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.81100000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71399999999999997,"15min":0.69999999999999996},"min":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94099999999999995,"5min":1.109,"15min":1.109},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.53200000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.84499999999999997}]},{"osd":4,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0620000000000001,"5min":1.556,"15min":1.556},"last":1.02},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70499999999999996,"15min":0.73699999999999999},"min":{"1min":0.44400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.0429999999999999,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":1.0429999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67800000000000005,"15min":0.70299999999999996},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.71599999999999997,"15min":0.73399999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.86299999999999999}]},{"osd":7,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.74199999999999999,"15min":0.76700000000000002},"min":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.92300000000000004,"5min":0.998,"15min":0.998},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.753,"15min":0.77900000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":1.109}]}]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27716,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344124,"statfs":{"total":96636764160,"available":96608382976,"internally_reserved":0,"allocated":712704,"data_stored":293626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.50800000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93100000000000005,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72699999999999998,"15min":0.74399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95199999999999996,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.73599999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.76500000000000001},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97799999999999998,"5min":2.524,"15min":2.524},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.54900000000000004,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93999999999999995,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.83699999999999997}]},{"osd":2,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84499999999999997,"5min":2.363,"15min":2.363},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0029999999999999,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.76900000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.747,"15min":0.75700000000000001},"min":{"1min":0.54700000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90000000000000002,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.753},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.81699999999999995},"min":{"1min":0.625,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.96199999999999997,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.86899999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68200000000000005,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88100000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.441,"5min":0.434,"15min":0.434},"max":{"1min":0.82199999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.65900000000000003}]},{"osd":6,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86799999999999999,"5min":0.996,"15min":0.996},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72599999999999998,"15min":0.74399999999999999},"min":{"1min":0.52300000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.91600000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.63800000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71699999999999997,"15min":0.73799999999999999},"min":{"1min":0.51100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90900000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.46999999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92100000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.68999999999999995}]}]},{"osd":3,"up_from":15,"seq":64424509483,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31828,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340012,"statfs":{"total":96636764160,"available":96604172288,"internally_reserved":0,"allocated":724992,"data_stored":299901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.46300000000000002},"min":{"1min":0.33700000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.72699999999999998,"5min":1.123,"15min":1.123},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52000000000000002,"15min":0.52400000000000002},"min":{"1min":0.432,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.871,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.437}]},{"osd":1,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44700000000000001,"15min":0.46100000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64800000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.502,"5min":0.51500000000000001,"15min":0.53400000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.68200000000000005,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.28599999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54700000000000004,"15min":0.56999999999999995},"min":{"1min":0.35999999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.70599999999999996,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63700000000000001,"15min":0.65900000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79700000000000004,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.41799999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.77400000000000002,"15min":0.79600000000000004},"min":{"1min":0.52100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1950000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.75900000000000001,"15min":0.77900000000000003},"min":{"1min":0.55700000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.004,"5min":1.391,"15min":1.391},"last":0.64600000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.80800000000000005},"min":{"1min":0.627,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84299999999999997,"15min":0.86699999999999999},"min":{"1min":0.64500000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6599999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.81000000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.79600000000000004,"15min":0.83599999999999997},"min":{"1min":0.51500000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.008,"5min":1.669,"15min":1.669},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.80300000000000005,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.212,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.76200000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82999999999999996,"15min":0.86399999999999999},"min":{"1min":0.54900000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4279999999999999,"5min":1.887,"15min":1.887},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.82099999999999995,"15min":0.86099999999999999},"min":{"1min":0.59899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98299999999999998,"5min":2.044,"15min":2.044},"last":0.70299999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":712704,"data_stored":293626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.48899999999999999,"15min":0.501},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69799999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66300000000000003,"5min":0.751,"15min":0.751},"last":0.65900000000000003}]},{"osd":1,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.34499999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72999999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.375},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.48999999999999999,"15min":0.50800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72499999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.39900000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75900000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40400000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69299999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.61599999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52600000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.052,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.75900000000000001,"15min":0.76600000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82999999999999996,"15min":0.84199999999999997},"min":{"1min":0.64600000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.081,"5min":1.306,"15min":1.306},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83999999999999997,"15min":0.85999999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.28,"5min":1.526,"15min":1.526},"last":0.55500000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.77700000000000002,"15min":0.77900000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.55600000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.121,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.53400000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.51000000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":0.79800000000000004},"min":{"1min":0.56499999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.71899999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509482,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1306624,"data_stored":883994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.56799999999999995,"15min":0.63500000000000001},"min":{"1min":0.33400000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.54800000000000004,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.60099999999999998,"15min":0.65300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71799999999999997,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.49399999999999999,"15min":0.50600000000000001},"min":{"1min":0.375,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73299999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53000000000000003,"15min":0.53600000000000003},"min":{"1min":0.39100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67000000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.435}]},{"osd":3,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.86099999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.75900000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.41299999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.53000000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95599999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.746},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.76300000000000001,"15min":0.78900000000000003},"min":{"1min":0.58099999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95199999999999996,"5min":1.23,"15min":1.23},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76100000000000001,"15min":0.77900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.89500000000000002,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.56499999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.873,"5min":1.107,"15min":1.107},"last":0.67800000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76000000000000001,"15min":0.76800000000000002},"min":{"1min":0.57399999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":1.026,"15min":1.026},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.81799999999999995,"15min":0.85699999999999998},"min":{"1min":0.55800000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.83199999999999996,"5min":1.196,"15min":1.196},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.78700000000000003,"15min":0.80000000000000004},"min":{"1min":0.57499999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.92900000000000005,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.78700000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91600000000000004,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.66100000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509481,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31824,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340016,"statfs":{"total":96636764160,"available":96604176384,"internally_reserved":0,"allocated":724992,"data_stored":299901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.55300000000000005,"15min":0.61299999999999999},"min":{"1min":0.313,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.71099999999999997,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.753,"15min":0.873},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.73799999999999999,"5min":8.0419999999999998,"15min":8.0419999999999998},"last":0.34000000000000002}]},{"osd":2,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.53800000000000003,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.58899999999999997,"5min":1.258,"15min":1.258},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.67500000000000004,"15min":0.747},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63800000000000001,"5min":4.907,"15min":4.907},"last":0.48099999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.753,"15min":0.872},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.77000000000000002,"5min":7.9569999999999999,"15min":7.9569999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.624,"15min":0.66600000000000004},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.65100000000000002,"5min":1.397,"15min":1.397},"last":0.44900000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":1.002},"min":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91700000000000004,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.91500000000000004,"15min":0.98099999999999998},"min":{"1min":0.66700000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.97999999999999998,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.77000000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.879,"15min":0.93999999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.016,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.83299999999999996,"15min":0.871},"min":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.89900000000000002,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.754}]},{"osd":6,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.83899999999999997,"15min":0.87},"min":{"1min":0.64500000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95699999999999996,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.92900000000000005,"15min":0.999},"min":{"1min":0.623,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.097,"5min":4.4100000000000001,"15min":4.4100000000000001},"last":0.68100000000000005}]},{"osd":7,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.94599999999999995,"15min":1.0209999999999999},"min":{"1min":0.64100000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.038,"5min":4.5209999999999999,"15min":4.5209999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.92100000000000004,"15min":1},"min":{"1min":0.57699999999999996,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.099,"5min":4.4420000000000002,"15min":4.4420000000000002},"last":0.57699999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542186,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1306624,"data_stored":883994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.497,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89400000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.497}]},{"osd":1,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71599999999999997,"15min":0.73799999999999999},"min":{"1min":0.51700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.876,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67800000000000005,"15min":0.67700000000000005},"min":{"1min":0.44900000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89000000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.69099999999999995}]},{"osd":2,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.443,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84299999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.443},{"interface":"front","average":{"1min":0.748,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.59199999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90700000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.88600000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.54800000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91800000000000004,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.64500000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.90500000000000003,"5min":1.954,"15min":1.954},"last":0.80800000000000005}]},{"osd":4,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.50800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.51400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88700000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.82899999999999996}]},{"osd":6,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66700000000000004,"15min":0.68200000000000005},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82399999999999995,"5min":1.071,"15min":1.071},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.70099999999999996},"min":{"1min":0.53600000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.878,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92900000000000005,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.74099999999999999,"15min":0.75800000000000001},"min":{"1min":0.56799999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95199999999999996,"5min":1.04,"15min":1.04},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:21:16.054 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-03-29T08:21:16.051+0000 7f1cc6c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:21:19.017 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:21:19.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.167+0000 7fd45d400640 1 Processor -- start 2024-03-29T08:21:19.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.167+0000 7fd45d400640 1 -- start start 2024-03-29T08:21:19.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.171+0000 7fd45d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd458099150 0x7fd458167a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:19.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.171+0000 7fd45d400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4580d0030 con 0x7fd4580ccd10 2024-03-29T08:21:19.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.171+0000 7fd45d400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4580cf420 con 0x7fd45815c9d0 2024-03-29T08:21:19.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.171+0000 7fd45d400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4580cf700 con 0x7fd458099150 2024-03-29T08:21:19.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.171+0000 7fd457200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd458099150 0x7fd458167a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:19.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.171+0000 7fd44fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd4580ccd10 0x7fd458098a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45116/0 (socket says 172.21.15.46:45116) 2024-03-29T08:21:19.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.171+0000 7fd44fe00640 1 -- 172.21.15.46:0/1029866571 learned_addr learned my addr 172.21.15.46:0/1029866571 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:19.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.171+0000 7fd457200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd458099150 0x7fd458167a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44176/0 (socket says 172.21.15.46:44176) 2024-03-29T08:21:19.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.171+0000 7fd457200640 1 -- 172.21.15.46:0/1029866571 >> v1:172.21.15.73:6789/0 conn(0x7fd45815c9d0 legacy=0x7fd4580cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:19.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.171+0000 7fd457200640 1 -- 172.21.15.46:0/1029866571 >> v1:172.21.15.46:6789/0 conn(0x7fd4580ccd10 legacy=0x7fd458098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:19.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.171+0000 7fd457200640 1 -- 172.21.15.46:0/1029866571 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4580cf9e0 con 0x7fd458099150 2024-03-29T08:21:19.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.171+0000 7fd455e00640 1 -- 172.21.15.46:0/1029866571 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934690887 0 0) 0x7fd4580d0030 con 0x7fd4580ccd10 2024-03-29T08:21:19.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.171+0000 7fd457200640 1 --2- 172.21.15.46:0/1029866571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd458099150 0x7fd458167a40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd450009d90 tx=0x7fd45001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c57d33aa27a9ec8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:19.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.171+0000 7fd455e00640 1 -- 172.21.15.46:0/1029866571 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4500a59e0 con 0x7fd458099150 2024-03-29T08:21:19.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.171+0000 7fd455e00640 1 -- 172.21.15.46:0/1029866571 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4500a5b40 con 0x7fd458099150 2024-03-29T08:21:19.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.171+0000 7fd455e00640 1 -- 172.21.15.46:0/1029866571 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4500ad440 con 0x7fd458099150 2024-03-29T08:21:19.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd45d400640 1 -- 172.21.15.46:0/1029866571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd458099150 msgr2=0x7fd458167a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:19.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd45d400640 1 --2- 172.21.15.46:0/1029866571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd458099150 0x7fd458167a40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd450009d90 tx=0x7fd45001c940 comp rx=0 tx=0).stop 2024-03-29T08:21:19.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd45d400640 1 -- 172.21.15.46:0/1029866571 shutdown_connections 2024-03-29T08:21:19.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd45d400640 1 --2- 172.21.15.46:0/1029866571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd458099150 0x7fd458167a40 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd45d400640 1 -- 172.21.15.46:0/1029866571 >> 172.21.15.46:0/1029866571 conn(0x7fd4580b21a0 msgr2=0x7fd4580b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:19.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd45d400640 1 -- 172.21.15.46:0/1029866571 shutdown_connections 2024-03-29T08:21:19.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd45d400640 1 -- 172.21.15.46:0/1029866571 wait complete. 2024-03-29T08:21:19.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd45d400640 1 Processor -- start 2024-03-29T08:21:19.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd45d400640 1 -- start start 2024-03-29T08:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd45d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd458099150 0x7fd4580ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd45d400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580ccd10 0x7fd458154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd45d400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd45815c9d0 0x7fd45814d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd45d400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4580c6c70 con 0x7fd458099150 2024-03-29T08:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd45d400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4580c5900 con 0x7fd4580ccd10 2024-03-29T08:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd45d400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4580c5380 con 0x7fd45815c9d0 2024-03-29T08:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd456800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd458099150 0x7fd4580ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd456800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd458099150 0x7fd4580ca180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44178/0 (socket says 172.21.15.46:44178) 2024-03-29T08:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd456800640 1 -- 172.21.15.46:0/1379319820 learned_addr learned my addr 172.21.15.46:0/1379319820 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd457200640 1 --2- 172.21.15.46:0/1379319820 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd45815c9d0 0x7fd45814d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd44fe00640 1 --2- 172.21.15.46:0/1379319820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580ccd10 0x7fd458154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd456800640 1 -- 172.21.15.46:0/1379319820 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd45815c9d0 msgr2=0x7fd45814d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:19.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd456800640 1 --2- 172.21.15.46:0/1379319820 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd45815c9d0 0x7fd45814d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd456800640 1 -- 172.21.15.46:0/1379319820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580ccd10 msgr2=0x7fd458154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd456800640 1 --2- 172.21.15.46:0/1379319820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580ccd10 0x7fd458154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.175+0000 7fd456800640 1 -- 172.21.15.46:0/1379319820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd450009a70 con 0x7fd458099150 2024-03-29T08:21:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.179+0000 7fd456800640 1 --2- 172.21.15.46:0/1379319820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd458099150 0x7fd4580ca180 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd44800c3f0 tx=0x7fd44800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.179+0000 7fd44ea00640 1 -- 172.21.15.46:0/1379319820 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4480052d0 con 0x7fd458099150 2024-03-29T08:21:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.179+0000 7fd44ea00640 1 -- 172.21.15.46:0/1379319820 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd448005430 con 0x7fd458099150 2024-03-29T08:21:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.179+0000 7fd44ea00640 1 -- 172.21.15.46:0/1379319820 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd448003840 con 0x7fd458099150 2024-03-29T08:21:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.179+0000 7fd45d400640 1 -- 172.21.15.46:0/1379319820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4580c6990 con 0x7fd458099150 2024-03-29T08:21:19.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.179+0000 7fd45d400640 1 -- 172.21.15.46:0/1379319820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45814dc90 con 0x7fd458099150 2024-03-29T08:21:19.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.179+0000 7fd45d400640 1 -- 172.21.15.46:0/1379319820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4580c51e0 con 0x7fd458099150 2024-03-29T08:21:19.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.179+0000 7fd44ea00640 1 -- 172.21.15.46:0/1379319820 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd4480039a0 con 0x7fd458099150 2024-03-29T08:21:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.179+0000 7fd44ea00640 1 --2- 172.21.15.46:0/1379319820 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd4340666c0 0x7fd434068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.179+0000 7fd44ea00640 1 -- 172.21.15.46:0/1379319820 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd448083630 con 0x7fd458099150 2024-03-29T08:21:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.179+0000 7fd44fe00640 1 --2- 172.21.15.46:0/1379319820 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd4340666c0 0x7fd434068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:19.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.183+0000 7fd44fe00640 1 --2- 172.21.15.46:0/1379319820 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd4340666c0 0x7fd434068b80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd440007fd0 tx=0x7fd440007ba0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:19.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.183+0000 7fd44ea00640 1 -- 172.21.15.46:0/1379319820 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd4580c51e0 con 0x7fd458099150 2024-03-29T08:21:19.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.343+0000 7fd45d400640 1 -- 172.21.15.46:0/1379319820 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd45814f330 con 0x7fd4340666c0 2024-03-29T08:21:19.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.351+0000 7fd44ea00640 1 -- 172.21.15.46:0/1379319820 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289167 (secure 0 0 0) 0x7fd45814f330 con 0x7fd4340666c0 2024-03-29T08:21:19.357 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:19.359 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:21:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.355+0000 7fd45d400640 1 -- 172.21.15.46:0/1379319820 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd4340666c0 msgr2=0x7fd434068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.355+0000 7fd45d400640 1 --2- 172.21.15.46:0/1379319820 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd4340666c0 0x7fd434068b80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd440007fd0 tx=0x7fd440007ba0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.355+0000 7fd45d400640 1 -- 172.21.15.46:0/1379319820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd458099150 msgr2=0x7fd4580ca180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.355+0000 7fd45d400640 1 --2- 172.21.15.46:0/1379319820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd458099150 0x7fd4580ca180 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd44800c3f0 tx=0x7fd44800c8b0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.355+0000 7fd45d400640 1 -- 172.21.15.46:0/1379319820 shutdown_connections 2024-03-29T08:21:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.355+0000 7fd45d400640 1 --2- 172.21.15.46:0/1379319820 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd4340666c0 0x7fd434068b80 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.355+0000 7fd45d400640 1 --2- 172.21.15.46:0/1379319820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580ccd10 0x7fd458154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.355+0000 7fd45d400640 1 --2- 172.21.15.46:0/1379319820 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd45815c9d0 0x7fd45814d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.355+0000 7fd45d400640 1 --2- 172.21.15.46:0/1379319820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd458099150 0x7fd4580ca180 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.355+0000 7fd45d400640 1 -- 172.21.15.46:0/1379319820 >> 172.21.15.46:0/1379319820 conn(0x7fd4580b21a0 msgr2=0x7fd45806fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.355+0000 7fd45d400640 1 -- 172.21.15.46:0/1379319820 shutdown_connections 2024-03-29T08:21:19.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.355+0000 7fd45d400640 1 -- 172.21.15.46:0/1379319820 wait complete. 2024-03-29T08:21:19.377 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-03-29T08:21:18.085527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243736,"kb_used_data":7080,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754730984,"statfs":{"total":773094113280,"available":772844527616,"internally_reserved":0,"allocated":7249920,"data_stored":3689268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":242208313},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001873"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.376719+0000","last_change":"2024-03-29T08:21:10.373320+0000","last_active":"2024-03-29T08:21:10.376719+0000","last_peered":"2024-03-29T08:21:10.376719+0000","last_clean":"2024-03-29T08:21:10.376719+0000","last_became_active":"2024-03-29T08:21:10.373048+0000","last_became_peered":"2024-03-29T08:21:10.373048+0000","last_unstale":"2024-03-29T08:21:10.376719+0000","last_undegraded":"2024-03-29T08:21:10.376719+0000","last_fullsized":"2024-03-29T08:21:10.376719+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:16:15.767217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.373169+0000","last_change":"2024-03-29T08:21:10.368483+0000","last_active":"2024-03-29T08:21:10.373169+0000","last_peered":"2024-03-29T08:21:10.373169+0000","last_clean":"2024-03-29T08:21:10.373169+0000","last_became_active":"2024-03-29T08:21:10.368219+0000","last_became_peered":"2024-03-29T08:21:10.368219+0000","last_unstale":"2024-03-29T08:21:10.373169+0000","last_undegraded":"2024-03-29T08:21:10.373169+0000","last_fullsized":"2024-03-29T08:21:10.373169+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:05:22.465098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.363027+0000","last_change":"2024-03-29T08:21:10.361063+0000","last_active":"2024-03-29T08:21:10.363027+0000","last_peered":"2024-03-29T08:21:10.363027+0000","last_clean":"2024-03-29T08:21:10.363027+0000","last_became_active":"2024-03-29T08:21:10.360622+0000","last_became_peered":"2024-03-29T08:21:10.360622+0000","last_unstale":"2024-03-29T08:21:10.363027+0000","last_undegraded":"2024-03-29T08:21:10.363027+0000","last_fullsized":"2024-03-29T08:21:10.363027+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:47:48.297292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.369963+0000","last_change":"2024-03-29T08:21:10.365519+0000","last_active":"2024-03-29T08:21:10.369963+0000","last_peered":"2024-03-29T08:21:10.369963+0000","last_clean":"2024-03-29T08:21:10.369963+0000","last_became_active":"2024-03-29T08:21:10.364955+0000","last_became_peered":"2024-03-29T08:21:10.364955+0000","last_unstale":"2024-03-29T08:21:10.369963+0000","last_undegraded":"2024-03-29T08:21:10.369963+0000","last_fullsized":"2024-03-29T08:21:10.369963+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:51.414360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.375745+0000","last_change":"2024-03-29T08:21:10.367208+0000","last_active":"2024-03-29T08:21:10.375745+0000","last_peered":"2024-03-29T08:21:10.375745+0000","last_clean":"2024-03-29T08:21:10.375745+0000","last_became_active":"2024-03-29T08:21:10.366929+0000","last_became_peered":"2024-03-29T08:21:10.366929+0000","last_unstale":"2024-03-29T08:21:10.375745+0000","last_undegraded":"2024-03-29T08:21:10.375745+0000","last_fullsized":"2024-03-29T08:21:10.375745+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:36:29.239728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.365061+0000","last_change":"2024-03-29T08:21:10.360462+0000","last_active":"2024-03-29T08:21:10.365061+0000","last_peered":"2024-03-29T08:21:10.365061+0000","last_clean":"2024-03-29T08:21:10.365061+0000","last_became_active":"2024-03-29T08:21:10.359795+0000","last_became_peered":"2024-03-29T08:21:10.359795+0000","last_unstale":"2024-03-29T08:21:10.365061+0000","last_undegraded":"2024-03-29T08:21:10.365061+0000","last_fullsized":"2024-03-29T08:21:10.365061+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:35:20.420999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.368728+0000","last_change":"2024-03-29T08:21:10.366090+0000","last_active":"2024-03-29T08:21:10.368728+0000","last_peered":"2024-03-29T08:21:10.368728+0000","last_clean":"2024-03-29T08:21:10.368728+0000","last_became_active":"2024-03-29T08:21:10.365262+0000","last_became_peered":"2024-03-29T08:21:10.365262+0000","last_unstale":"2024-03-29T08:21:10.368728+0000","last_undegraded":"2024-03-29T08:21:10.368728+0000","last_fullsized":"2024-03-29T08:21:10.368728+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:30:03.234601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-29T08:21:11.537740+0000","last_change":"2024-03-29T08:21:10.360297+0000","last_active":"2024-03-29T08:21:11.537740+0000","last_peered":"2024-03-29T08:21:11.537740+0000","last_clean":"2024-03-29T08:21:11.537740+0000","last_became_active":"2024-03-29T08:21:10.357163+0000","last_became_peered":"2024-03-29T08:21:10.357163+0000","last_unstale":"2024-03-29T08:21:11.537740+0000","last_undegraded":"2024-03-29T08:21:11.537740+0000","last_fullsized":"2024-03-29T08:21:11.537740+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:31:08.299748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.365543+0000","last_change":"2024-03-29T08:21:10.360976+0000","last_active":"2024-03-29T08:21:10.365543+0000","last_peered":"2024-03-29T08:21:10.365543+0000","last_clean":"2024-03-29T08:21:10.365543+0000","last_became_active":"2024-03-29T08:21:10.360719+0000","last_became_peered":"2024-03-29T08:21:10.360719+0000","last_unstale":"2024-03-29T08:21:10.365543+0000","last_undegraded":"2024-03-29T08:21:10.365543+0000","last_fullsized":"2024-03-29T08:21:10.365543+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:40.843359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.365109+0000","last_change":"2024-03-29T08:21:10.361065+0000","last_active":"2024-03-29T08:21:10.365109+0000","last_peered":"2024-03-29T08:21:10.365109+0000","last_clean":"2024-03-29T08:21:10.365109+0000","last_became_active":"2024-03-29T08:21:10.360809+0000","last_became_peered":"2024-03-29T08:21:10.360809+0000","last_unstale":"2024-03-29T08:21:10.365109+0000","last_undegraded":"2024-03-29T08:21:10.365109+0000","last_fullsized":"2024-03-29T08:21:10.365109+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:54.034461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.361823+0000","last_change":"2024-03-29T08:21:10.356483+0000","last_active":"2024-03-29T08:21:10.361823+0000","last_peered":"2024-03-29T08:21:10.361823+0000","last_clean":"2024-03-29T08:21:10.361823+0000","last_became_active":"2024-03-29T08:21:10.356172+0000","last_became_peered":"2024-03-29T08:21:10.356172+0000","last_unstale":"2024-03-29T08:21:10.361823+0000","last_undegraded":"2024-03-29T08:21:10.361823+0000","last_fullsized":"2024-03-29T08:21:10.361823+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:58.105380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.356786+0000","last_change":"2024-03-29T08:21:10.352867+0000","last_active":"2024-03-29T08:21:10.356786+0000","last_peered":"2024-03-29T08:21:10.356786+0000","last_clean":"2024-03-29T08:21:10.356786+0000","last_became_active":"2024-03-29T08:21:10.352572+0000","last_became_peered":"2024-03-29T08:21:10.352572+0000","last_unstale":"2024-03-29T08:21:10.356786+0000","last_undegraded":"2024-03-29T08:21:10.356786+0000","last_fullsized":"2024-03-29T08:21:10.356786+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:20:51.330512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.370170+0000","last_change":"2024-03-29T08:21:10.366232+0000","last_active":"2024-03-29T08:21:10.370170+0000","last_peered":"2024-03-29T08:21:10.370170+0000","last_clean":"2024-03-29T08:21:10.370170+0000","last_became_active":"2024-03-29T08:21:10.365968+0000","last_became_peered":"2024-03-29T08:21:10.365968+0000","last_unstale":"2024-03-29T08:21:10.370170+0000","last_undegraded":"2024-03-29T08:21:10.370170+0000","last_fullsized":"2024-03-29T08:21:10.370170+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:35:20.420999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.378633+0000","last_change":"2024-03-29T08:21:10.377982+0000","last_active":"2024-03-29T08:21:10.378633+0000","last_peered":"2024-03-29T08:21:10.378633+0000","last_clean":"2024-03-29T08:21:10.378633+0000","last_became_active":"2024-03-29T08:21:10.377632+0000","last_became_peered":"2024-03-29T08:21:10.377632+0000","last_unstale":"2024-03-29T08:21:10.378633+0000","last_undegraded":"2024-03-29T08:21:10.378633+0000","last_fullsized":"2024-03-29T08:21:10.378633+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:49:07.458847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.356812+0000","last_change":"2024-03-29T08:21:10.354113+0000","last_active":"2024-03-29T08:21:10.356812+0000","last_peered":"2024-03-29T08:21:10.356812+0000","last_clean":"2024-03-29T08:21:10.356812+0000","last_became_active":"2024-03-29T08:21:10.353618+0000","last_became_peered":"2024-03-29T08:21:10.353618+0000","last_unstale":"2024-03-29T08:21:10.356812+0000","last_undegraded":"2024-03-29T08:21:10.356812+0000","last_fullsized":"2024-03-29T08:21:10.356812+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:39.458643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.375780+0000","last_change":"2024-03-29T08:21:10.368789+0000","last_active":"2024-03-29T08:21:10.375780+0000","last_peered":"2024-03-29T08:21:10.375780+0000","last_clean":"2024-03-29T08:21:10.375780+0000","last_became_active":"2024-03-29T08:21:10.368349+0000","last_became_peered":"2024-03-29T08:21:10.368349+0000","last_unstale":"2024-03-29T08:21:10.375780+0000","last_undegraded":"2024-03-29T08:21:10.375780+0000","last_fullsized":"2024-03-29T08:21:10.375780+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:01:52.376061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.376738+0000","last_change":"2024-03-29T08:21:10.373831+0000","last_active":"2024-03-29T08:21:10.376738+0000","last_peered":"2024-03-29T08:21:10.376738+0000","last_clean":"2024-03-29T08:21:10.376738+0000","last_became_active":"2024-03-29T08:21:10.373597+0000","last_became_peered":"2024-03-29T08:21:10.373597+0000","last_unstale":"2024-03-29T08:21:10.376738+0000","last_undegraded":"2024-03-29T08:21:10.376738+0000","last_fullsized":"2024-03-29T08:21:10.376738+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:39.458643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.375007+0000","last_change":"2024-03-29T08:21:10.369918+0000","last_active":"2024-03-29T08:21:10.375007+0000","last_peered":"2024-03-29T08:21:10.375007+0000","last_clean":"2024-03-29T08:21:10.375007+0000","last_became_active":"2024-03-29T08:21:10.369740+0000","last_became_peered":"2024-03-29T08:21:10.369740+0000","last_unstale":"2024-03-29T08:21:10.375007+0000","last_undegraded":"2024-03-29T08:21:10.375007+0000","last_fullsized":"2024-03-29T08:21:10.375007+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:33:19.998323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.379425+0000","last_change":"2024-03-29T08:21:10.372426+0000","last_active":"2024-03-29T08:21:10.379425+0000","last_peered":"2024-03-29T08:21:10.379425+0000","last_clean":"2024-03-29T08:21:10.379425+0000","last_became_active":"2024-03-29T08:21:10.372096+0000","last_became_peered":"2024-03-29T08:21:10.372096+0000","last_unstale":"2024-03-29T08:21:10.379425+0000","last_undegraded":"2024-03-29T08:21:10.379425+0000","last_fullsized":"2024-03-29T08:21:10.379425+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:58:48.804311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.376199+0000","last_change":"2024-03-29T08:21:10.370488+0000","last_active":"2024-03-29T08:21:10.376199+0000","last_peered":"2024-03-29T08:21:10.376199+0000","last_clean":"2024-03-29T08:21:10.376199+0000","last_became_active":"2024-03-29T08:21:10.369952+0000","last_became_peered":"2024-03-29T08:21:10.369952+0000","last_unstale":"2024-03-29T08:21:10.376199+0000","last_undegraded":"2024-03-29T08:21:10.376199+0000","last_fullsized":"2024-03-29T08:21:10.376199+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:01.741332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.370186+0000","last_change":"2024-03-29T08:21:10.368973+0000","last_active":"2024-03-29T08:21:10.370186+0000","last_peered":"2024-03-29T08:21:10.370186+0000","last_clean":"2024-03-29T08:21:10.370186+0000","last_became_active":"2024-03-29T08:21:10.368751+0000","last_became_peered":"2024-03-29T08:21:10.368751+0000","last_unstale":"2024-03-29T08:21:10.370186+0000","last_undegraded":"2024-03-29T08:21:10.370186+0000","last_fullsized":"2024-03-29T08:21:10.370186+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:44.680234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.372278+0000","last_change":"2024-03-29T08:21:10.362564+0000","last_active":"2024-03-29T08:21:10.372278+0000","last_peered":"2024-03-29T08:21:10.372278+0000","last_clean":"2024-03-29T08:21:10.372278+0000","last_became_active":"2024-03-29T08:21:10.362306+0000","last_became_peered":"2024-03-29T08:21:10.362306+0000","last_unstale":"2024-03-29T08:21:10.372278+0000","last_undegraded":"2024-03-29T08:21:10.372278+0000","last_fullsized":"2024-03-29T08:21:10.372278+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:15.253098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.368745+0000","last_change":"2024-03-29T08:21:10.366281+0000","last_active":"2024-03-29T08:21:10.368745+0000","last_peered":"2024-03-29T08:21:10.368745+0000","last_clean":"2024-03-29T08:21:10.368745+0000","last_became_active":"2024-03-29T08:21:10.365907+0000","last_became_peered":"2024-03-29T08:21:10.365907+0000","last_unstale":"2024-03-29T08:21:10.368745+0000","last_undegraded":"2024-03-29T08:21:10.368745+0000","last_fullsized":"2024-03-29T08:21:10.368745+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:25:00.595704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.357419+0000","last_change":"2024-03-29T08:21:10.353995+0000","last_active":"2024-03-29T08:21:10.357419+0000","last_peered":"2024-03-29T08:21:10.357419+0000","last_clean":"2024-03-29T08:21:10.357419+0000","last_became_active":"2024-03-29T08:21:10.353509+0000","last_became_peered":"2024-03-29T08:21:10.353509+0000","last_unstale":"2024-03-29T08:21:10.357419+0000","last_undegraded":"2024-03-29T08:21:10.357419+0000","last_fullsized":"2024-03-29T08:21:10.357419+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:58.831468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.365475+0000","last_change":"2024-03-29T08:21:10.361686+0000","last_active":"2024-03-29T08:21:10.365475+0000","last_peered":"2024-03-29T08:21:10.365475+0000","last_clean":"2024-03-29T08:21:10.365475+0000","last_became_active":"2024-03-29T08:21:10.361365+0000","last_became_peered":"2024-03-29T08:21:10.361365+0000","last_unstale":"2024-03-29T08:21:10.365475+0000","last_undegraded":"2024-03-29T08:21:10.365475+0000","last_fullsized":"2024-03-29T08:21:10.365475+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:59:24.384598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.352986+0000","last_change":"2024-03-29T08:21:10.351292+0000","last_active":"2024-03-29T08:21:10.352986+0000","last_peered":"2024-03-29T08:21:10.352986+0000","last_clean":"2024-03-29T08:21:10.352986+0000","last_became_active":"2024-03-29T08:21:10.350578+0000","last_became_peered":"2024-03-29T08:21:10.350578+0000","last_unstale":"2024-03-29T08:21:10.352986+0000","last_undegraded":"2024-03-29T08:21:10.352986+0000","last_fullsized":"2024-03-29T08:21:10.352986+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:53.450623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.374806+0000","last_change":"2024-03-29T08:21:10.369925+0000","last_active":"2024-03-29T08:21:10.374806+0000","last_peered":"2024-03-29T08:21:10.374806+0000","last_clean":"2024-03-29T08:21:10.374806+0000","last_became_active":"2024-03-29T08:21:10.369632+0000","last_became_peered":"2024-03-29T08:21:10.369632+0000","last_unstale":"2024-03-29T08:21:10.374806+0000","last_undegraded":"2024-03-29T08:21:10.374806+0000","last_fullsized":"2024-03-29T08:21:10.374806+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:49.605189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-29T08:21:11.537940+0000","last_change":"2024-03-29T08:21:10.354706+0000","last_active":"2024-03-29T08:21:11.537940+0000","last_peered":"2024-03-29T08:21:11.537940+0000","last_clean":"2024-03-29T08:21:11.537940+0000","last_became_active":"2024-03-29T08:21:10.354300+0000","last_became_peered":"2024-03-29T08:21:10.354300+0000","last_unstale":"2024-03-29T08:21:11.537940+0000","last_undegraded":"2024-03-29T08:21:11.537940+0000","last_fullsized":"2024-03-29T08:21:11.537940+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:16:37.374572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-29T08:21:11.537954+0000","last_change":"2024-03-29T08:21:10.364304+0000","last_active":"2024-03-29T08:21:11.537954+0000","last_peered":"2024-03-29T08:21:11.537954+0000","last_clean":"2024-03-29T08:21:11.537954+0000","last_became_active":"2024-03-29T08:21:10.357285+0000","last_became_peered":"2024-03-29T08:21:10.357285+0000","last_unstale":"2024-03-29T08:21:11.537954+0000","last_undegraded":"2024-03-29T08:21:11.537954+0000","last_fullsized":"2024-03-29T08:21:11.537954+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:53.946448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.349345+0000","last_change":"2024-03-29T08:21:10.346102+0000","last_active":"2024-03-29T08:21:10.349345+0000","last_peered":"2024-03-29T08:21:10.349345+0000","last_clean":"2024-03-29T08:21:10.349345+0000","last_became_active":"2024-03-29T08:21:10.345854+0000","last_became_peered":"2024-03-29T08:21:10.345854+0000","last_unstale":"2024-03-29T08:21:10.349345+0000","last_undegraded":"2024-03-29T08:21:10.349345+0000","last_fullsized":"2024-03-29T08:21:10.349345+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:59:45.698306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.371289+0000","last_change":"2024-03-29T08:21:10.364033+0000","last_active":"2024-03-29T08:21:10.371289+0000","last_peered":"2024-03-29T08:21:10.371289+0000","last_clean":"2024-03-29T08:21:10.371289+0000","last_became_active":"2024-03-29T08:21:10.363759+0000","last_became_peered":"2024-03-29T08:21:10.363759+0000","last_unstale":"2024-03-29T08:21:10.371289+0000","last_undegraded":"2024-03-29T08:21:10.371289+0000","last_fullsized":"2024-03-29T08:21:10.371289+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:01:19.390653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.375845+0000","last_change":"2024-03-29T08:21:10.365615+0000","last_active":"2024-03-29T08:21:10.375845+0000","last_peered":"2024-03-29T08:21:10.375845+0000","last_clean":"2024-03-29T08:21:10.375845+0000","last_became_active":"2024-03-29T08:21:10.365345+0000","last_became_peered":"2024-03-29T08:21:10.365345+0000","last_unstale":"2024-03-29T08:21:10.375845+0000","last_undegraded":"2024-03-29T08:21:10.375845+0000","last_fullsized":"2024-03-29T08:21:10.375845+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:54.034461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.365127+0000","last_change":"2024-03-29T08:21:10.361710+0000","last_active":"2024-03-29T08:21:10.365127+0000","last_peered":"2024-03-29T08:21:10.365127+0000","last_clean":"2024-03-29T08:21:10.365127+0000","last_became_active":"2024-03-29T08:21:10.360475+0000","last_became_peered":"2024-03-29T08:21:10.360475+0000","last_unstale":"2024-03-29T08:21:10.365127+0000","last_undegraded":"2024-03-29T08:21:10.365127+0000","last_fullsized":"2024-03-29T08:21:10.365127+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:53.450623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"62'4","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.369112+0000","last_change":"2024-03-29T08:21:09.953242+0000","last_active":"2024-03-29T08:21:10.369112+0000","last_peered":"2024-03-29T08:21:10.369112+0000","last_clean":"2024-03-29T08:21:10.369112+0000","last_became_active":"2024-03-29T08:21:09.952978+0000","last_became_peered":"2024-03-29T08:21:09.952978+0000","last_unstale":"2024-03-29T08:21:10.369112+0000","last_undegraded":"2024-03-29T08:21:10.369112+0000","last_fullsized":"2024-03-29T08:21:10.369112+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:43.677354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.375653+0000","last_change":"2024-03-29T08:21:10.365222+0000","last_active":"2024-03-29T08:21:10.375653+0000","last_peered":"2024-03-29T08:21:10.375653+0000","last_clean":"2024-03-29T08:21:10.375653+0000","last_became_active":"2024-03-29T08:21:10.364960+0000","last_became_peered":"2024-03-29T08:21:10.364960+0000","last_unstale":"2024-03-29T08:21:10.375653+0000","last_undegraded":"2024-03-29T08:21:10.375653+0000","last_fullsized":"2024-03-29T08:21:10.375653+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:40.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.373059+0000","last_change":"2024-03-29T08:21:09.955717+0000","last_active":"2024-03-29T08:21:10.373059+0000","last_peered":"2024-03-29T08:21:10.373059+0000","last_clean":"2024-03-29T08:21:10.373059+0000","last_became_active":"2024-03-29T08:21:09.955450+0000","last_became_peered":"2024-03-29T08:21:09.955450+0000","last_unstale":"2024-03-29T08:21:10.373059+0000","last_undegraded":"2024-03-29T08:21:10.373059+0000","last_fullsized":"2024-03-29T08:21:10.373059+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:47:40.877803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-29T08:21:11.538215+0000","last_change":"2024-03-29T08:21:10.365123+0000","last_active":"2024-03-29T08:21:11.538215+0000","last_peered":"2024-03-29T08:21:11.538215+0000","last_clean":"2024-03-29T08:21:11.538215+0000","last_became_active":"2024-03-29T08:21:10.364865+0000","last_became_peered":"2024-03-29T08:21:10.364865+0000","last_unstale":"2024-03-29T08:21:11.538215+0000","last_undegraded":"2024-03-29T08:21:11.538215+0000","last_fullsized":"2024-03-29T08:21:11.538215+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:53:40.109459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.380078+0000","last_change":"2024-03-29T08:21:09.950482+0000","last_active":"2024-03-29T08:21:10.380078+0000","last_peered":"2024-03-29T08:21:10.380078+0000","last_clean":"2024-03-29T08:21:10.380078+0000","last_became_active":"2024-03-29T08:21:09.950190+0000","last_became_peered":"2024-03-29T08:21:09.950190+0000","last_unstale":"2024-03-29T08:21:10.380078+0000","last_undegraded":"2024-03-29T08:21:10.380078+0000","last_fullsized":"2024-03-29T08:21:10.380078+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:07:20.233898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.375673+0000","last_change":"2024-03-29T08:21:10.367127+0000","last_active":"2024-03-29T08:21:10.375673+0000","last_peered":"2024-03-29T08:21:10.375673+0000","last_clean":"2024-03-29T08:21:10.375673+0000","last_became_active":"2024-03-29T08:21:10.366057+0000","last_became_peered":"2024-03-29T08:21:10.366057+0000","last_unstale":"2024-03-29T08:21:10.375673+0000","last_undegraded":"2024-03-29T08:21:10.375673+0000","last_fullsized":"2024-03-29T08:21:10.375673+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:21:23.293798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.376789+0000","last_change":"2024-03-29T08:21:09.942997+0000","last_active":"2024-03-29T08:21:10.376789+0000","last_peered":"2024-03-29T08:21:10.376789+0000","last_clean":"2024-03-29T08:21:10.376789+0000","last_became_active":"2024-03-29T08:21:09.942730+0000","last_became_peered":"2024-03-29T08:21:09.942730+0000","last_unstale":"2024-03-29T08:21:10.376789+0000","last_undegraded":"2024-03-29T08:21:10.376789+0000","last_fullsized":"2024-03-29T08:21:10.376789+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:42:48.592612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-29T08:21:11.537771+0000","last_change":"2024-03-29T08:21:10.354837+0000","last_active":"2024-03-29T08:21:11.537771+0000","last_peered":"2024-03-29T08:21:11.537771+0000","last_clean":"2024-03-29T08:21:11.537771+0000","last_became_active":"2024-03-29T08:21:10.354095+0000","last_became_peered":"2024-03-29T08:21:10.354095+0000","last_unstale":"2024-03-29T08:21:11.537771+0000","last_undegraded":"2024-03-29T08:21:11.537771+0000","last_fullsized":"2024-03-29T08:21:11.537771+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:44:19.470211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.368177+0000","last_change":"2024-03-29T08:21:09.944500+0000","last_active":"2024-03-29T08:21:10.368177+0000","last_peered":"2024-03-29T08:21:10.368177+0000","last_clean":"2024-03-29T08:21:10.368177+0000","last_became_active":"2024-03-29T08:21:09.944236+0000","last_became_peered":"2024-03-29T08:21:09.944236+0000","last_unstale":"2024-03-29T08:21:10.368177+0000","last_undegraded":"2024-03-29T08:21:10.368177+0000","last_fullsized":"2024-03-29T08:21:10.368177+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:06:08.420341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.372269+0000","last_change":"2024-03-29T08:21:10.366636+0000","last_active":"2024-03-29T08:21:10.372269+0000","last_peered":"2024-03-29T08:21:10.372269+0000","last_clean":"2024-03-29T08:21:10.372269+0000","last_became_active":"2024-03-29T08:21:10.366204+0000","last_became_peered":"2024-03-29T08:21:10.366204+0000","last_unstale":"2024-03-29T08:21:10.372269+0000","last_undegraded":"2024-03-29T08:21:10.372269+0000","last_fullsized":"2024-03-29T08:21:10.372269+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:05:09.416516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.376592+0000","last_change":"2024-03-29T08:21:09.946818+0000","last_active":"2024-03-29T08:21:10.376592+0000","last_peered":"2024-03-29T08:21:10.376592+0000","last_clean":"2024-03-29T08:21:10.376592+0000","last_became_active":"2024-03-29T08:21:09.946555+0000","last_became_peered":"2024-03-29T08:21:09.946555+0000","last_unstale":"2024-03-29T08:21:10.376592+0000","last_undegraded":"2024-03-29T08:21:10.376592+0000","last_fullsized":"2024-03-29T08:21:10.376592+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:27:46.452557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.374711+0000","last_change":"2024-03-29T08:21:10.371486+0000","last_active":"2024-03-29T08:21:10.374711+0000","last_peered":"2024-03-29T08:21:10.374711+0000","last_clean":"2024-03-29T08:21:10.374711+0000","last_became_active":"2024-03-29T08:21:10.371258+0000","last_became_peered":"2024-03-29T08:21:10.371258+0000","last_unstale":"2024-03-29T08:21:10.374711+0000","last_undegraded":"2024-03-29T08:21:10.374711+0000","last_fullsized":"2024-03-29T08:21:10.374711+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:58.105380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.346717+0000","last_change":"2024-03-29T08:21:09.954162+0000","last_active":"2024-03-29T08:21:10.346717+0000","last_peered":"2024-03-29T08:21:10.346717+0000","last_clean":"2024-03-29T08:21:10.346717+0000","last_became_active":"2024-03-29T08:21:09.953849+0000","last_became_peered":"2024-03-29T08:21:09.953849+0000","last_unstale":"2024-03-29T08:21:10.346717+0000","last_undegraded":"2024-03-29T08:21:10.346717+0000","last_fullsized":"2024-03-29T08:21:10.346717+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:52.803477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.375778+0000","last_change":"2024-03-29T08:21:10.367730+0000","last_active":"2024-03-29T08:21:10.375778+0000","last_peered":"2024-03-29T08:21:10.375778+0000","last_clean":"2024-03-29T08:21:10.375778+0000","last_became_active":"2024-03-29T08:21:10.367483+0000","last_became_peered":"2024-03-29T08:21:10.367483+0000","last_unstale":"2024-03-29T08:21:10.375778+0000","last_undegraded":"2024-03-29T08:21:10.375778+0000","last_fullsized":"2024-03-29T08:21:10.375778+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:19:16.700510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.341905+0000","last_change":"2024-03-29T08:21:09.954120+0000","last_active":"2024-03-29T08:21:10.341905+0000","last_peered":"2024-03-29T08:21:10.341905+0000","last_clean":"2024-03-29T08:21:10.341905+0000","last_became_active":"2024-03-29T08:21:09.953375+0000","last_became_peered":"2024-03-29T08:21:09.953375+0000","last_unstale":"2024-03-29T08:21:10.341905+0000","last_undegraded":"2024-03-29T08:21:10.341905+0000","last_fullsized":"2024-03-29T08:21:10.341905+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:48.602309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.352948+0000","last_change":"2024-03-29T08:21:10.350179+0000","last_active":"2024-03-29T08:21:10.352948+0000","last_peered":"2024-03-29T08:21:10.352948+0000","last_clean":"2024-03-29T08:21:10.352948+0000","last_became_active":"2024-03-29T08:21:10.347566+0000","last_became_peered":"2024-03-29T08:21:10.347566+0000","last_unstale":"2024-03-29T08:21:10.352948+0000","last_undegraded":"2024-03-29T08:21:10.352948+0000","last_fullsized":"2024-03-29T08:21:10.352948+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:24.166586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.348398+0000","last_change":"2024-03-29T08:21:09.951304+0000","last_active":"2024-03-29T08:21:10.348398+0000","last_peered":"2024-03-29T08:21:10.348398+0000","last_clean":"2024-03-29T08:21:10.348398+0000","last_became_active":"2024-03-29T08:21:09.951067+0000","last_became_peered":"2024-03-29T08:21:09.951067+0000","last_unstale":"2024-03-29T08:21:10.348398+0000","last_undegraded":"2024-03-29T08:21:10.348398+0000","last_fullsized":"2024-03-29T08:21:10.348398+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:13:08.676082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.376644+0000","last_change":"2024-03-29T08:21:10.374978+0000","last_active":"2024-03-29T08:21:10.376644+0000","last_peered":"2024-03-29T08:21:10.376644+0000","last_clean":"2024-03-29T08:21:10.376644+0000","last_became_active":"2024-03-29T08:21:10.374420+0000","last_became_peered":"2024-03-29T08:21:10.374420+0000","last_unstale":"2024-03-29T08:21:10.376644+0000","last_undegraded":"2024-03-29T08:21:10.376644+0000","last_fullsized":"2024-03-29T08:21:10.376644+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:08:39.271148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.349965+0000","last_change":"2024-03-29T08:21:09.952367+0000","last_active":"2024-03-29T08:21:10.349965+0000","last_peered":"2024-03-29T08:21:10.349965+0000","last_clean":"2024-03-29T08:21:10.349965+0000","last_became_active":"2024-03-29T08:21:09.952089+0000","last_became_peered":"2024-03-29T08:21:09.952089+0000","last_unstale":"2024-03-29T08:21:10.349965+0000","last_undegraded":"2024-03-29T08:21:10.349965+0000","last_fullsized":"2024-03-29T08:21:10.349965+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:08:44.015717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-29T08:21:11.538193+0000","last_change":"2024-03-29T08:21:10.369746+0000","last_active":"2024-03-29T08:21:11.538193+0000","last_peered":"2024-03-29T08:21:11.538193+0000","last_clean":"2024-03-29T08:21:11.538193+0000","last_became_active":"2024-03-29T08:21:10.369491+0000","last_became_peered":"2024-03-29T08:21:10.369491+0000","last_unstale":"2024-03-29T08:21:11.538193+0000","last_undegraded":"2024-03-29T08:21:11.538193+0000","last_fullsized":"2024-03-29T08:21:11.538193+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:19:35.191446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"62'3","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.376579+0000","last_change":"2024-03-29T08:21:09.944508+0000","last_active":"2024-03-29T08:21:10.376579+0000","last_peered":"2024-03-29T08:21:10.376579+0000","last_clean":"2024-03-29T08:21:10.376579+0000","last_became_active":"2024-03-29T08:21:09.944191+0000","last_became_peered":"2024-03-29T08:21:09.944191+0000","last_unstale":"2024-03-29T08:21:10.376579+0000","last_undegraded":"2024-03-29T08:21:10.376579+0000","last_fullsized":"2024-03-29T08:21:10.376579+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:25:36.003763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.365297+0000","last_change":"2024-03-29T08:21:10.360534+0000","last_active":"2024-03-29T08:21:10.365297+0000","last_peered":"2024-03-29T08:21:10.365297+0000","last_clean":"2024-03-29T08:21:10.365297+0000","last_became_active":"2024-03-29T08:21:10.359874+0000","last_became_peered":"2024-03-29T08:21:10.359874+0000","last_unstale":"2024-03-29T08:21:10.365297+0000","last_undegraded":"2024-03-29T08:21:10.365297+0000","last_fullsized":"2024-03-29T08:21:10.365297+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:56:39.434467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"62'6","reported_seq":17,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.378427+0000","last_change":"2024-03-29T08:21:09.953728+0000","last_active":"2024-03-29T08:21:10.378427+0000","last_peered":"2024-03-29T08:21:10.378427+0000","last_clean":"2024-03-29T08:21:10.378427+0000","last_became_active":"2024-03-29T08:21:09.953431+0000","last_became_peered":"2024-03-29T08:21:09.953431+0000","last_unstale":"2024-03-29T08:21:10.378427+0000","last_undegraded":"2024-03-29T08:21:10.378427+0000","last_fullsized":"2024-03-29T08:21:10.378427+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:38:17.880237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.367625+0000","last_change":"2024-03-29T08:21:10.363180+0000","last_active":"2024-03-29T08:21:10.367625+0000","last_peered":"2024-03-29T08:21:10.367625+0000","last_clean":"2024-03-29T08:21:10.367625+0000","last_became_active":"2024-03-29T08:21:10.362956+0000","last_became_peered":"2024-03-29T08:21:10.362956+0000","last_unstale":"2024-03-29T08:21:10.367625+0000","last_undegraded":"2024-03-29T08:21:10.367625+0000","last_fullsized":"2024-03-29T08:21:10.367625+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:52:34.720375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.377526+0000","last_change":"2024-03-29T08:21:09.954743+0000","last_active":"2024-03-29T08:21:10.377526+0000","last_peered":"2024-03-29T08:21:10.377526+0000","last_clean":"2024-03-29T08:21:10.377526+0000","last_became_active":"2024-03-29T08:21:09.954486+0000","last_became_peered":"2024-03-29T08:21:09.954486+0000","last_unstale":"2024-03-29T08:21:10.377526+0000","last_undegraded":"2024-03-29T08:21:10.377526+0000","last_fullsized":"2024-03-29T08:21:10.377526+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:03:13.729828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.363054+0000","last_change":"2024-03-29T08:21:10.361667+0000","last_active":"2024-03-29T08:21:10.363054+0000","last_peered":"2024-03-29T08:21:10.363054+0000","last_clean":"2024-03-29T08:21:10.363054+0000","last_became_active":"2024-03-29T08:21:10.361419+0000","last_became_peered":"2024-03-29T08:21:10.361419+0000","last_unstale":"2024-03-29T08:21:10.363054+0000","last_undegraded":"2024-03-29T08:21:10.363054+0000","last_fullsized":"2024-03-29T08:21:10.363054+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:21:28.741076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.343726+0000","last_change":"2024-03-29T08:21:09.948902+0000","last_active":"2024-03-29T08:21:10.343726+0000","last_peered":"2024-03-29T08:21:10.343726+0000","last_clean":"2024-03-29T08:21:10.343726+0000","last_became_active":"2024-03-29T08:21:09.948613+0000","last_became_peered":"2024-03-29T08:21:09.948613+0000","last_unstale":"2024-03-29T08:21:10.343726+0000","last_undegraded":"2024-03-29T08:21:10.343726+0000","last_fullsized":"2024-03-29T08:21:10.343726+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:57:00.619680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.357400+0000","last_change":"2024-03-29T08:21:10.353834+0000","last_active":"2024-03-29T08:21:10.357400+0000","last_peered":"2024-03-29T08:21:10.357400+0000","last_clean":"2024-03-29T08:21:10.357400+0000","last_became_active":"2024-03-29T08:21:10.353157+0000","last_became_peered":"2024-03-29T08:21:10.353157+0000","last_unstale":"2024-03-29T08:21:10.357400+0000","last_undegraded":"2024-03-29T08:21:10.357400+0000","last_fullsized":"2024-03-29T08:21:10.357400+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:01:48.373155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.344421+0000","last_change":"2024-03-29T08:21:09.946582+0000","last_active":"2024-03-29T08:21:10.344421+0000","last_peered":"2024-03-29T08:21:10.344421+0000","last_clean":"2024-03-29T08:21:10.344421+0000","last_became_active":"2024-03-29T08:21:09.946276+0000","last_became_peered":"2024-03-29T08:21:09.946276+0000","last_unstale":"2024-03-29T08:21:10.344421+0000","last_undegraded":"2024-03-29T08:21:10.344421+0000","last_fullsized":"2024-03-29T08:21:10.344421+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:01:32.117594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.368765+0000","last_change":"2024-03-29T08:21:10.366291+0000","last_active":"2024-03-29T08:21:10.368765+0000","last_peered":"2024-03-29T08:21:10.368765+0000","last_clean":"2024-03-29T08:21:10.368765+0000","last_became_active":"2024-03-29T08:21:10.365935+0000","last_became_peered":"2024-03-29T08:21:10.365935+0000","last_unstale":"2024-03-29T08:21:10.368765+0000","last_undegraded":"2024-03-29T08:21:10.368765+0000","last_fullsized":"2024-03-29T08:21:10.368765+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:11:50.905254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"62'8","reported_seq":19,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.370667+0000","last_change":"2024-03-29T08:21:09.953517+0000","last_active":"2024-03-29T08:21:10.370667+0000","last_peered":"2024-03-29T08:21:10.370667+0000","last_clean":"2024-03-29T08:21:10.370667+0000","last_became_active":"2024-03-29T08:21:09.953205+0000","last_became_peered":"2024-03-29T08:21:09.953205+0000","last_unstale":"2024-03-29T08:21:10.370667+0000","last_undegraded":"2024-03-29T08:21:10.370667+0000","last_fullsized":"2024-03-29T08:21:10.370667+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:11:14.860644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.376896+0000","last_change":"2024-03-29T08:21:10.375135+0000","last_active":"2024-03-29T08:21:10.376896+0000","last_peered":"2024-03-29T08:21:10.376896+0000","last_clean":"2024-03-29T08:21:10.376896+0000","last_became_active":"2024-03-29T08:21:10.374792+0000","last_became_peered":"2024-03-29T08:21:10.374792+0000","last_unstale":"2024-03-29T08:21:10.376896+0000","last_undegraded":"2024-03-29T08:21:10.376896+0000","last_fullsized":"2024-03-29T08:21:10.376896+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:15:15.253098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.379971+0000","last_change":"2024-03-29T08:21:09.954705+0000","last_active":"2024-03-29T08:21:10.379971+0000","last_peered":"2024-03-29T08:21:10.379971+0000","last_clean":"2024-03-29T08:21:10.379971+0000","last_became_active":"2024-03-29T08:21:09.954314+0000","last_became_peered":"2024-03-29T08:21:09.954314+0000","last_unstale":"2024-03-29T08:21:10.379971+0000","last_undegraded":"2024-03-29T08:21:10.379971+0000","last_fullsized":"2024-03-29T08:21:10.379971+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:24:09.856034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.376923+0000","last_change":"2024-03-29T08:21:10.375314+0000","last_active":"2024-03-29T08:21:10.376923+0000","last_peered":"2024-03-29T08:21:10.376923+0000","last_clean":"2024-03-29T08:21:10.376923+0000","last_became_active":"2024-03-29T08:21:10.374032+0000","last_became_peered":"2024-03-29T08:21:10.374032+0000","last_unstale":"2024-03-29T08:21:10.376923+0000","last_undegraded":"2024-03-29T08:21:10.376923+0000","last_fullsized":"2024-03-29T08:21:10.376923+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:05:09.416516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.374406+0000","last_change":"2024-03-29T08:21:09.947621+0000","last_active":"2024-03-29T08:21:10.374406+0000","last_peered":"2024-03-29T08:21:10.374406+0000","last_clean":"2024-03-29T08:21:10.374406+0000","last_became_active":"2024-03-29T08:21:09.945817+0000","last_became_peered":"2024-03-29T08:21:09.945817+0000","last_unstale":"2024-03-29T08:21:10.374406+0000","last_undegraded":"2024-03-29T08:21:10.374406+0000","last_fullsized":"2024-03-29T08:21:10.374406+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:50:03.760523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.365492+0000","last_change":"2024-03-29T08:21:10.361875+0000","last_active":"2024-03-29T08:21:10.365492+0000","last_peered":"2024-03-29T08:21:10.365492+0000","last_clean":"2024-03-29T08:21:10.365492+0000","last_became_active":"2024-03-29T08:21:10.361470+0000","last_became_peered":"2024-03-29T08:21:10.361470+0000","last_unstale":"2024-03-29T08:21:10.365492+0000","last_undegraded":"2024-03-29T08:21:10.365492+0000","last_fullsized":"2024-03-29T08:21:10.365492+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:50:48.557652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"62'1","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.368054+0000","last_change":"2024-03-29T08:21:09.947148+0000","last_active":"2024-03-29T08:21:10.368054+0000","last_peered":"2024-03-29T08:21:10.368054+0000","last_clean":"2024-03-29T08:21:10.368054+0000","last_became_active":"2024-03-29T08:21:09.946610+0000","last_became_peered":"2024-03-29T08:21:09.946610+0000","last_unstale":"2024-03-29T08:21:10.368054+0000","last_undegraded":"2024-03-29T08:21:10.368054+0000","last_fullsized":"2024-03-29T08:21:10.368054+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:51.605750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.368615+0000","last_change":"2024-03-29T08:21:10.365113+0000","last_active":"2024-03-29T08:21:10.368615+0000","last_peered":"2024-03-29T08:21:10.368615+0000","last_clean":"2024-03-29T08:21:10.368615+0000","last_became_active":"2024-03-29T08:21:10.364857+0000","last_became_peered":"2024-03-29T08:21:10.364857+0000","last_unstale":"2024-03-29T08:21:10.368615+0000","last_undegraded":"2024-03-29T08:21:10.368615+0000","last_fullsized":"2024-03-29T08:21:10.368615+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:37:31.250178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.351091+0000","last_change":"2024-03-29T08:21:09.945407+0000","last_active":"2024-03-29T08:21:10.351091+0000","last_peered":"2024-03-29T08:21:10.351091+0000","last_clean":"2024-03-29T08:21:10.351091+0000","last_became_active":"2024-03-29T08:21:09.945100+0000","last_became_peered":"2024-03-29T08:21:09.945100+0000","last_unstale":"2024-03-29T08:21:10.351091+0000","last_undegraded":"2024-03-29T08:21:10.351091+0000","last_fullsized":"2024-03-29T08:21:10.351091+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:26:31.314621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.367608+0000","last_change":"2024-03-29T08:21:10.362422+0000","last_active":"2024-03-29T08:21:10.367608+0000","last_peered":"2024-03-29T08:21:10.367608+0000","last_clean":"2024-03-29T08:21:10.367608+0000","last_became_active":"2024-03-29T08:21:10.362194+0000","last_became_peered":"2024-03-29T08:21:10.362194+0000","last_unstale":"2024-03-29T08:21:10.367608+0000","last_undegraded":"2024-03-29T08:21:10.367608+0000","last_fullsized":"2024-03-29T08:21:10.367608+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:24:10.858914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.369809+0000","last_change":"2024-03-29T08:21:09.953120+0000","last_active":"2024-03-29T08:21:10.369809+0000","last_peered":"2024-03-29T08:21:10.369809+0000","last_clean":"2024-03-29T08:21:10.369809+0000","last_became_active":"2024-03-29T08:21:09.952860+0000","last_became_peered":"2024-03-29T08:21:09.952860+0000","last_unstale":"2024-03-29T08:21:10.369809+0000","last_undegraded":"2024-03-29T08:21:10.369809+0000","last_fullsized":"2024-03-29T08:21:10.369809+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:01:38.455763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-29T08:21:11.538013+0000","last_change":"2024-03-29T08:21:10.369151+0000","last_active":"2024-03-29T08:21:11.538013+0000","last_peered":"2024-03-29T08:21:11.538013+0000","last_clean":"2024-03-29T08:21:11.538013+0000","last_became_active":"2024-03-29T08:21:10.368169+0000","last_became_peered":"2024-03-29T08:21:10.368169+0000","last_unstale":"2024-03-29T08:21:11.538013+0000","last_undegraded":"2024-03-29T08:21:11.538013+0000","last_fullsized":"2024-03-29T08:21:11.538013+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:34:51.485832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.351208+0000","last_change":"2024-03-29T08:21:09.952305+0000","last_active":"2024-03-29T08:21:10.351208+0000","last_peered":"2024-03-29T08:21:10.351208+0000","last_clean":"2024-03-29T08:21:10.351208+0000","last_became_active":"2024-03-29T08:21:09.952036+0000","last_became_peered":"2024-03-29T08:21:09.952036+0000","last_unstale":"2024-03-29T08:21:10.351208+0000","last_undegraded":"2024-03-29T08:21:10.351208+0000","last_fullsized":"2024-03-29T08:21:10.351208+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:05:43.897022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-29T08:21:11.537748+0000","last_change":"2024-03-29T08:21:10.352341+0000","last_active":"2024-03-29T08:21:11.537748+0000","last_peered":"2024-03-29T08:21:11.537748+0000","last_clean":"2024-03-29T08:21:11.537748+0000","last_became_active":"2024-03-29T08:21:10.352075+0000","last_became_peered":"2024-03-29T08:21:10.352075+0000","last_unstale":"2024-03-29T08:21:11.537748+0000","last_undegraded":"2024-03-29T08:21:11.537748+0000","last_fullsized":"2024-03-29T08:21:11.537748+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:17:39.223026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.346817+0000","last_change":"2024-03-29T08:21:09.947277+0000","last_active":"2024-03-29T08:21:10.346817+0000","last_peered":"2024-03-29T08:21:10.346817+0000","last_clean":"2024-03-29T08:21:10.346817+0000","last_became_active":"2024-03-29T08:21:09.945398+0000","last_became_peered":"2024-03-29T08:21:09.945398+0000","last_unstale":"2024-03-29T08:21:10.346817+0000","last_undegraded":"2024-03-29T08:21:10.346817+0000","last_fullsized":"2024-03-29T08:21:10.346817+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:36:29.788331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.374961+0000","last_change":"2024-03-29T08:21:10.369441+0000","last_active":"2024-03-29T08:21:10.374961+0000","last_peered":"2024-03-29T08:21:10.374961+0000","last_clean":"2024-03-29T08:21:10.374961+0000","last_became_active":"2024-03-29T08:21:10.368081+0000","last_became_peered":"2024-03-29T08:21:10.368081+0000","last_unstale":"2024-03-29T08:21:10.374961+0000","last_undegraded":"2024-03-29T08:21:10.374961+0000","last_fullsized":"2024-03-29T08:21:10.374961+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:18:28.430663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"62'2","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.376939+0000","last_change":"2024-03-29T08:21:09.955367+0000","last_active":"2024-03-29T08:21:10.376939+0000","last_peered":"2024-03-29T08:21:10.376939+0000","last_clean":"2024-03-29T08:21:10.376939+0000","last_became_active":"2024-03-29T08:21:09.954965+0000","last_became_peered":"2024-03-29T08:21:09.954965+0000","last_unstale":"2024-03-29T08:21:10.376939+0000","last_undegraded":"2024-03-29T08:21:10.376939+0000","last_fullsized":"2024-03-29T08:21:10.376939+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:57.828588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.375063+0000","last_change":"2024-03-29T08:21:10.367736+0000","last_active":"2024-03-29T08:21:10.375063+0000","last_peered":"2024-03-29T08:21:10.375063+0000","last_clean":"2024-03-29T08:21:10.375063+0000","last_became_active":"2024-03-29T08:21:10.367428+0000","last_became_peered":"2024-03-29T08:21:10.367428+0000","last_unstale":"2024-03-29T08:21:10.375063+0000","last_undegraded":"2024-03-29T08:21:10.375063+0000","last_fullsized":"2024-03-29T08:21:10.375063+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:02:01.741332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.357898+0000","last_change":"2024-03-29T08:21:09.948192+0000","last_active":"2024-03-29T08:21:10.357898+0000","last_peered":"2024-03-29T08:21:10.357898+0000","last_clean":"2024-03-29T08:21:10.357898+0000","last_became_active":"2024-03-29T08:21:09.947926+0000","last_became_peered":"2024-03-29T08:21:09.947926+0000","last_unstale":"2024-03-29T08:21:10.357898+0000","last_undegraded":"2024-03-29T08:21:10.357898+0000","last_fullsized":"2024-03-29T08:21:10.357898+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:33:58.392690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.379478+0000","last_change":"2024-03-29T08:21:10.372604+0000","last_active":"2024-03-29T08:21:10.379478+0000","last_peered":"2024-03-29T08:21:10.379478+0000","last_clean":"2024-03-29T08:21:10.379478+0000","last_became_active":"2024-03-29T08:21:10.372326+0000","last_became_peered":"2024-03-29T08:21:10.372326+0000","last_unstale":"2024-03-29T08:21:10.379478+0000","last_undegraded":"2024-03-29T08:21:10.379478+0000","last_fullsized":"2024-03-29T08:21:10.379478+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:22:08.727894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.347818+0000","last_change":"2024-03-29T08:21:09.947847+0000","last_active":"2024-03-29T08:21:10.347818+0000","last_peered":"2024-03-29T08:21:10.347818+0000","last_clean":"2024-03-29T08:21:10.347818+0000","last_became_active":"2024-03-29T08:21:09.946845+0000","last_became_peered":"2024-03-29T08:21:09.946845+0000","last_unstale":"2024-03-29T08:21:10.347818+0000","last_undegraded":"2024-03-29T08:21:10.347818+0000","last_fullsized":"2024-03-29T08:21:10.347818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:19:13.916686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-29T08:21:11.538156+0000","last_change":"2024-03-29T08:21:10.356822+0000","last_active":"2024-03-29T08:21:11.538156+0000","last_peered":"2024-03-29T08:21:11.538156+0000","last_clean":"2024-03-29T08:21:11.538156+0000","last_became_active":"2024-03-29T08:21:10.356531+0000","last_became_peered":"2024-03-29T08:21:10.356531+0000","last_unstale":"2024-03-29T08:21:11.538156+0000","last_undegraded":"2024-03-29T08:21:11.538156+0000","last_fullsized":"2024-03-29T08:21:11.538156+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:41:23.588728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.356444+0000","last_change":"2024-03-29T08:21:09.938352+0000","last_active":"2024-03-29T08:21:10.356444+0000","last_peered":"2024-03-29T08:21:10.356444+0000","last_clean":"2024-03-29T08:21:10.356444+0000","last_became_active":"2024-03-29T08:21:09.937930+0000","last_became_peered":"2024-03-29T08:21:09.937930+0000","last_unstale":"2024-03-29T08:21:10.356444+0000","last_undegraded":"2024-03-29T08:21:10.356444+0000","last_fullsized":"2024-03-29T08:21:10.356444+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:23.163706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.365041+0000","last_change":"2024-03-29T08:21:10.360378+0000","last_active":"2024-03-29T08:21:10.365041+0000","last_peered":"2024-03-29T08:21:10.365041+0000","last_clean":"2024-03-29T08:21:10.365041+0000","last_became_active":"2024-03-29T08:21:10.360083+0000","last_became_peered":"2024-03-29T08:21:10.360083+0000","last_unstale":"2024-03-29T08:21:10.365041+0000","last_undegraded":"2024-03-29T08:21:10.365041+0000","last_fullsized":"2024-03-29T08:21:10.365041+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:46:44.680234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.358703+0000","last_change":"2024-03-29T08:21:09.953317+0000","last_active":"2024-03-29T08:21:10.358703+0000","last_peered":"2024-03-29T08:21:10.358703+0000","last_clean":"2024-03-29T08:21:10.358703+0000","last_became_active":"2024-03-29T08:21:09.953068+0000","last_became_peered":"2024-03-29T08:21:09.953068+0000","last_unstale":"2024-03-29T08:21:10.358703+0000","last_undegraded":"2024-03-29T08:21:10.358703+0000","last_fullsized":"2024-03-29T08:21:10.358703+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:40:39.840479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.357220+0000","last_change":"2024-03-29T08:21:10.354408+0000","last_active":"2024-03-29T08:21:10.357220+0000","last_peered":"2024-03-29T08:21:10.357220+0000","last_clean":"2024-03-29T08:21:10.357220+0000","last_became_active":"2024-03-29T08:21:10.354059+0000","last_became_peered":"2024-03-29T08:21:10.354059+0000","last_unstale":"2024-03-29T08:21:10.357220+0000","last_undegraded":"2024-03-29T08:21:10.357220+0000","last_fullsized":"2024-03-29T08:21:10.357220+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:18:40.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.358965+0000","last_change":"2024-03-29T08:21:09.950160+0000","last_active":"2024-03-29T08:21:10.358965+0000","last_peered":"2024-03-29T08:21:10.358965+0000","last_clean":"2024-03-29T08:21:10.358965+0000","last_became_active":"2024-03-29T08:21:09.949853+0000","last_became_peered":"2024-03-29T08:21:09.949853+0000","last_unstale":"2024-03-29T08:21:10.358965+0000","last_undegraded":"2024-03-29T08:21:10.358965+0000","last_fullsized":"2024-03-29T08:21:10.358965+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:59:23.381718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-29T08:21:11.537746+0000","last_change":"2024-03-29T08:21:10.353284+0000","last_active":"2024-03-29T08:21:11.537746+0000","last_peered":"2024-03-29T08:21:11.537746+0000","last_clean":"2024-03-29T08:21:11.537746+0000","last_became_active":"2024-03-29T08:21:10.353014+0000","last_became_peered":"2024-03-29T08:21:10.353014+0000","last_unstale":"2024-03-29T08:21:11.537746+0000","last_undegraded":"2024-03-29T08:21:11.537746+0000","last_fullsized":"2024-03-29T08:21:11.537746+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:16.094618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.357432+0000","last_change":"2024-03-29T08:21:09.945534+0000","last_active":"2024-03-29T08:21:10.357432+0000","last_peered":"2024-03-29T08:21:10.357432+0000","last_clean":"2024-03-29T08:21:10.357432+0000","last_became_active":"2024-03-29T08:21:09.943748+0000","last_became_peered":"2024-03-29T08:21:09.943748+0000","last_unstale":"2024-03-29T08:21:10.357432+0000","last_undegraded":"2024-03-29T08:21:10.357432+0000","last_fullsized":"2024-03-29T08:21:10.357432+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:05:06.965072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.379629+0000","last_change":"2024-03-29T08:21:10.373545+0000","last_active":"2024-03-29T08:21:10.379629+0000","last_peered":"2024-03-29T08:21:10.379629+0000","last_clean":"2024-03-29T08:21:10.379629+0000","last_became_active":"2024-03-29T08:21:10.372308+0000","last_became_peered":"2024-03-29T08:21:10.372308+0000","last_unstale":"2024-03-29T08:21:10.379629+0000","last_undegraded":"2024-03-29T08:21:10.379629+0000","last_fullsized":"2024-03-29T08:21:10.379629+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:38:18.883117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.349706+0000","last_change":"2024-03-29T08:21:09.946729+0000","last_active":"2024-03-29T08:21:10.349706+0000","last_peered":"2024-03-29T08:21:10.349706+0000","last_clean":"2024-03-29T08:21:10.349706+0000","last_became_active":"2024-03-29T08:21:09.946469+0000","last_became_peered":"2024-03-29T08:21:09.946469+0000","last_unstale":"2024-03-29T08:21:10.349706+0000","last_undegraded":"2024-03-29T08:21:10.349706+0000","last_fullsized":"2024-03-29T08:21:10.349706+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:00:36.010161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-03-29T08:21:11.537970+0000","last_change":"2024-03-29T08:21:10.364039+0000","last_active":"2024-03-29T08:21:11.537970+0000","last_peered":"2024-03-29T08:21:11.537970+0000","last_clean":"2024-03-29T08:21:11.537970+0000","last_became_active":"2024-03-29T08:21:10.363644+0000","last_became_peered":"2024-03-29T08:21:10.363644+0000","last_unstale":"2024-03-29T08:21:11.537970+0000","last_undegraded":"2024-03-29T08:21:11.537970+0000","last_fullsized":"2024-03-29T08:21:11.537970+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:09.925622+0000","last_clean_scrub_stamp":"2024-03-29T08:21:09.925622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:14:19.704327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.342022+0000","last_change":"2024-03-29T08:21:09.953812+0000","last_active":"2024-03-29T08:21:10.342022+0000","last_peered":"2024-03-29T08:21:10.342022+0000","last_clean":"2024-03-29T08:21:10.342022+0000","last_became_active":"2024-03-29T08:21:09.953475+0000","last_became_peered":"2024-03-29T08:21:09.953475+0000","last_unstale":"2024-03-29T08:21:10.342022+0000","last_undegraded":"2024-03-29T08:21:10.342022+0000","last_fullsized":"2024-03-29T08:21:10.342022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:21:08.922741+0000","last_clean_scrub_stamp":"2024-03-29T08:21:08.922741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:05:21.462217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":198,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-03-29T08:21:10.346901+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:21:10.346901+0000","last_peered":"2024-03-29T08:21:10.346901+0000","last_clean":"2024-03-29T08:21:10.346901+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:21:10.346901+0000","last_undegraded":"2024-03-29T08:21:10.346901+0000","last_fullsized":"2024-03-29T08:21:10.346901+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":313068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67600000000000005,"15min":0.65200000000000002},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.93400000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.69999999999999996,"15min":0.67600000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1200000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.83399999999999996}]},{"osd":1,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.67100000000000004,"15min":0.64800000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.66800000000000004,"15min":0.63400000000000001},"min":{"1min":0.52900000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.0189999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.86799999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.66200000000000003,"15min":0.63300000000000001},"min":{"1min":0.55900000000000005,"5min":0.503,"15min":0.503},"max":{"1min":1.0089999999999999,"5min":1.085,"15min":1.085},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.69299999999999995,"15min":0.66500000000000004},"min":{"1min":0.58499999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1000000000000001,"5min":1.288,"15min":1.288},"last":0.85299999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.71099999999999997,"15min":0.69799999999999995},"min":{"1min":0.57899999999999996,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.92800000000000005,"5min":1.018,"15min":1.018},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.745,"15min":0.73099999999999998},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.1659999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.68100000000000005}]},{"osd":4,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83499999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.70199999999999996,"15min":0.71999999999999997},"min":{"1min":0.46100000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0009999999999999,"5min":1.123,"15min":1.123},"last":0.50600000000000001}]},{"osd":5,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.70199999999999996,"15min":0.71199999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.95799999999999996,"5min":1.056,"15min":1.056},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.69499999999999995,"15min":0.70099999999999996},"min":{"1min":0.46300000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.89200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.73399999999999999,"15min":0.72999999999999998},"min":{"1min":0.48999999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.755},"min":{"1min":0.52300000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.56399999999999995}]}]},{"osd":6,"up_from":15,"seq":64424509482,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":753664,"data_stored":312998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.66000000000000003,"15min":0.63300000000000001},"min":{"1min":0.54000000000000004,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70499999999999996,"15min":0.68500000000000005},"min":{"1min":0.57899999999999996,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.64100000000000001,"15min":0.61499999999999999},"min":{"1min":0.58599999999999997,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.64800000000000002,"15min":0.627},"min":{"1min":0.51600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.436}]},{"osd":2,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.70299999999999996,"15min":0.67500000000000004},"min":{"1min":0.52300000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.71599999999999997,"15min":0.70399999999999996},"min":{"1min":0.54300000000000004,"5min":0.435,"15min":0.435},"max":{"1min":0.91200000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.65600000000000003}]},{"osd":3,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.71399999999999997,"15min":0.69999999999999996},"min":{"1min":0.56299999999999994,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94099999999999995,"5min":1.109,"15min":1.109},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70199999999999996,"15min":0.69399999999999995},"min":{"1min":0.53200000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.0620000000000001,"5min":1.556,"15min":1.556},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70499999999999996,"15min":0.73699999999999999},"min":{"1min":0.44400000000000001,"5min":0.371,"15min":0.371},"max":{"1min":1.0429999999999999,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.63}]},{"osd":5,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67800000000000005,"15min":0.70299999999999996},"min":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.71599999999999997,"15min":0.73399999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.85699999999999998}]},{"osd":7,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.74199999999999999,"15min":0.76700000000000002},"min":{"1min":0.49099999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.92300000000000004,"5min":0.998,"15min":0.998},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.753,"15min":0.77900000000000003},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.68200000000000005}]}]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":313592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70399999999999996,"15min":0.70799999999999996},"min":{"1min":0.50800000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93100000000000005,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.72699999999999998,"15min":0.74399999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95199999999999996,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.747}]},{"osd":1,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.75800000000000001,"15min":0.76500000000000001},"min":{"1min":0.57299999999999995,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.97799999999999998,"5min":2.524,"15min":2.524},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71299999999999997,"15min":0.71899999999999997},"min":{"1min":0.54900000000000004,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.93999999999999995,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.55400000000000005}]},{"osd":2,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.72699999999999998,"15min":0.73399999999999999},"min":{"1min":0.55300000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.84499999999999997,"5min":2.363,"15min":2.363},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.52600000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0029999999999999,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.63100000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.747,"15min":0.75700000000000001},"min":{"1min":0.54700000000000004,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.90000000000000002,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.80800000000000005,"15min":0.81699999999999995},"min":{"1min":0.625,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":0.96199999999999997,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.69499999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68200000000000005,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88100000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.441,"5min":0.434,"15min":0.434},"max":{"1min":0.82199999999999995,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.65300000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.66100000000000003,"15min":0.67500000000000004},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.86799999999999999,"5min":0.996,"15min":0.996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.72599999999999998,"15min":0.74399999999999999},"min":{"1min":0.52300000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.91600000000000004,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.78000000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:21:04 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.71699999999999997,"15min":0.73799999999999999},"min":{"1min":0.51100000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.90900000000000003,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.748,"15min":0.76600000000000001},"min":{"1min":0.46999999999999997,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.92100000000000004,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.73399999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509484,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":753664,"data_stored":313066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.45600000000000002,"15min":0.46300000000000002},"min":{"1min":0.33700000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.72699999999999998,"5min":1.123,"15min":1.123},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.52000000000000002,"15min":0.52400000000000002},"min":{"1min":0.432,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.871,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.45300000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.44700000000000001,"15min":0.46100000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64800000000000002,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.51500000000000001,"15min":0.53400000000000003},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.68200000000000005,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.48099999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54700000000000004,"15min":0.56999999999999995},"min":{"1min":0.35999999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.70599999999999996,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63700000000000001,"15min":0.65900000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79700000000000004,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.52600000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.77400000000000002,"15min":0.79600000000000004},"min":{"1min":0.52100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1950000000000001,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.75900000000000001,"15min":0.77900000000000003},"min":{"1min":0.55700000000000005,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.004,"5min":1.391,"15min":1.391},"last":0.46999999999999997}]},{"osd":5,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.80800000000000005},"min":{"1min":0.627,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84299999999999997,"15min":0.86699999999999999},"min":{"1min":0.64500000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.6599999999999999,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.68500000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.79600000000000004,"15min":0.83599999999999997},"min":{"1min":0.51500000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.008,"5min":1.669,"15min":1.669},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.80300000000000005,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.212,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.64800000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.82999999999999996,"15min":0.86399999999999999},"min":{"1min":0.54900000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4279999999999999,"5min":1.887,"15min":1.887},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.82099999999999995,"15min":0.86099999999999999},"min":{"1min":0.59899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.98299999999999998,"5min":2.044,"15min":2.044},"last":0.70199999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509483,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":770048,"data_stored":314700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.48899999999999999,"15min":0.501},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69799999999999995,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.502,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.66300000000000003,"5min":0.751,"15min":0.751},"last":0.42899999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.49399999999999999,"15min":0.496},"min":{"1min":0.34499999999999997,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.72999999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.48999999999999999,"15min":0.50800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72499999999999998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.40400000000000003}]},{"osd":3,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.49399999999999999,"15min":0.48799999999999999},"min":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75900000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.504},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40400000000000003,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.69299999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.48299999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.52600000000000002,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.052,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.755,"5min":0.75900000000000001,"15min":0.76600000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.76200000000000001}]},{"osd":5,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82999999999999996,"15min":0.84199999999999997},"min":{"1min":0.64600000000000002,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.081,"5min":1.306,"15min":1.306},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.83999999999999997,"15min":0.85999999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.28,"5min":1.526,"15min":1.526},"last":0.86299999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.77700000000000002,"15min":0.77900000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.55600000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.121,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.78900000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80100000000000005,"15min":0.80900000000000005},"min":{"1min":0.51000000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.78500000000000003,"15min":0.79800000000000004},"min":{"1min":0.56499999999999995,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.80700000000000005}]}]},{"osd":1,"up_from":15,"seq":64424509483,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1347584,"data_stored":903926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.56799999999999995,"15min":0.63500000000000001},"min":{"1min":0.33400000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.54800000000000004,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.60099999999999998,"15min":0.65300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71799999999999997,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.69199999999999995}]},{"osd":2,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.49399999999999999,"15min":0.50600000000000001},"min":{"1min":0.375,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.73299999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.53000000000000003,"15min":0.53600000000000003},"min":{"1min":0.39100000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.67000000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.49199999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.57899999999999996},"min":{"1min":0.48099999999999998,"5min":0.433,"15min":0.433},"max":{"1min":0.86099999999999999,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.75900000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.78500000000000003}]},{"osd":4,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72799999999999998,"15min":0.73599999999999999},"min":{"1min":0.53000000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.95599999999999996,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.76300000000000001,"15min":0.78900000000000003},"min":{"1min":0.58099999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":0.95199999999999996,"5min":1.23,"15min":1.23},"last":0.58499999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.76100000000000001,"15min":0.77900000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.89500000000000002,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.56499999999999995,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.873,"5min":1.107,"15min":1.107},"last":0.82199999999999995}]},{"osd":6,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76000000000000001,"15min":0.76800000000000002},"min":{"1min":0.57399999999999995,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.97199999999999998,"5min":1.026,"15min":1.026},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.81799999999999995,"15min":0.85699999999999998},"min":{"1min":0.55800000000000005,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.83199999999999996,"5min":1.196,"15min":1.196},"last":0.77300000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:21:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.78700000000000003,"15min":0.80000000000000004},"min":{"1min":0.57499999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.92900000000000005,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.76700000000000002,"15min":0.78700000000000003},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.91600000000000004,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.56799999999999995}]}]},{"osd":0,"up_from":15,"seq":64424509482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":761856,"data_stored":314540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.55300000000000005,"15min":0.61299999999999999},"min":{"1min":0.313,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.71099999999999997,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.316},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.753,"15min":0.873},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.73799999999999999,"5min":8.0419999999999998,"15min":8.0419999999999998},"last":0.379}]},{"osd":2,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.53800000000000003,"15min":0.55500000000000005},"min":{"1min":0.35899999999999999,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.58899999999999997,"5min":1.258,"15min":1.258},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.67500000000000004,"15min":0.747},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.63800000000000001,"5min":4.907,"15min":4.907},"last":0.42999999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.753,"15min":0.872},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.77000000000000002,"5min":7.9569999999999999,"15min":7.9569999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.624,"15min":0.66600000000000004},"min":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.65100000000000002,"5min":1.397,"15min":1.397},"last":0.505}]},{"osd":4,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.92000000000000004,"15min":1.002},"min":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.91700000000000004,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.91500000000000004,"15min":0.98099999999999998},"min":{"1min":0.66700000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.97999999999999998,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.73999999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.879,"15min":0.93999999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.016,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.83299999999999996,"15min":0.871},"min":{"1min":0.57999999999999996,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.89900000000000002,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.76400000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.83899999999999997,"15min":0.87},"min":{"1min":0.64500000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.95699999999999996,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.92900000000000005,"15min":0.999},"min":{"1min":0.623,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.097,"5min":4.4100000000000001,"15min":4.4100000000000001},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Mar 29 08:21:06 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.94599999999999995,"15min":1.0209999999999999},"min":{"1min":0.64100000000000001,"5min":0.496,"15min":0.496},"max":{"1min":1.038,"5min":4.5209999999999999,"15min":4.5209999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.92100000000000004,"15min":1},"min":{"1min":0.57699999999999996,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.099,"5min":4.4420000000000002,"15min":4.4420000000000002},"last":0.81499999999999995}]}]},{"osd":5,"up_from":14,"seq":60129542187,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1347584,"data_stored":903378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.65500000000000003,"15min":0.65300000000000002},"min":{"1min":0.497,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.89400000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.67900000000000005}]},{"osd":1,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71599999999999997,"15min":0.73799999999999999},"min":{"1min":0.51700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.876,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.67800000000000005,"15min":0.67700000000000005},"min":{"1min":0.44900000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.89000000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.83399999999999996}]},{"osd":2,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.443,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84299999999999997,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.70599999999999996,"15min":0.70399999999999996},"min":{"1min":0.59199999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90700000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.81799999999999995}]},{"osd":3,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.54800000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.91800000000000004,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.74199999999999999,"15min":0.75},"min":{"1min":0.64500000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.90500000000000003,"5min":1.954,"15min":1.954},"last":0.70099999999999996}]},{"osd":4,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.65300000000000002,"15min":0.65400000000000003},"min":{"1min":0.50800000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.755},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.65300000000000002,"15min":0.66300000000000003},"min":{"1min":0.51400000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88700000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.878}]},{"osd":6,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66700000000000004,"15min":0.68200000000000005},"min":{"1min":0.48299999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.82399999999999995,"5min":1.071,"15min":1.071},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.69199999999999995,"15min":0.70099999999999996},"min":{"1min":0.53600000000000003,"5min":0.432,"15min":0.432},"max":{"1min":0.878,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Fri Mar 29 08:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.48399999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.92900000000000005,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.74099999999999999,"15min":0.75800000000000001},"min":{"1min":0.56799999999999995,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.95199999999999996,"5min":1.04,"15min":1.04},"last":0.80300000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:21:19.382 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T08:21:19.382 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-29T08:21:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.563+0000 7fe730c00640 1 Processor -- start 2024-03-29T08:21:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.563+0000 7fe730c00640 1 -- start start 2024-03-29T08:21:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.563+0000 7fe730c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe72c0b3850 0x7fe72c15d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.563+0000 7fe730c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe72c15d9c0 con 0x7fe72c0b1630 2024-03-29T08:21:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.563+0000 7fe730c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe72c15ca30 con 0x7fe72c071240 2024-03-29T08:21:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.563+0000 7fe730c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe72c0b20f0 con 0x7fe72c0b3850 2024-03-29T08:21:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.563+0000 7fe729e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe72c0b3850 0x7fe72c15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.563+0000 7fe721400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe72c0b1630 0x7fe72c070b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45130/0 (socket says 172.21.15.46:45130) 2024-03-29T08:21:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.563+0000 7fe721400640 1 -- 172.21.15.46:0/3502926069 learned_addr learned my addr 172.21.15.46:0/3502926069 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.567+0000 7fe729e00640 1 -- 172.21.15.46:0/3502926069 >> v1:172.21.15.73:6789/0 conn(0x7fe72c071240 legacy=0x7fe72c1594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.567+0000 7fe729e00640 1 -- 172.21.15.46:0/3502926069 >> v1:172.21.15.46:6789/0 conn(0x7fe72c0b1630 legacy=0x7fe72c070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.567+0000 7fe729e00640 1 -- 172.21.15.46:0/3502926069 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72c0b23d0 con 0x7fe72c0b3850 2024-03-29T08:21:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.567+0000 7fe729400640 1 -- 172.21.15.46:0/3502926069 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629931174 0 0) 0x7fe714001120 con 0x7fe72c0b1630 2024-03-29T08:21:19.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.567+0000 7fe729e00640 1 --2- 172.21.15.46:0/3502926069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c0b3850 0x7fe72c15d480 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009d90 tx=0x7fe71c01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd674cad3c47292 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:19.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.567+0000 7fe729400640 1 -- 172.21.15.46:0/3502926069 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c0a5840 con 0x7fe72c0b3850 2024-03-29T08:21:19.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.567+0000 7fe729400640 1 -- 172.21.15.46:0/3502926069 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71c0a59a0 con 0x7fe72c0b3850 2024-03-29T08:21:19.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.567+0000 7fe729400640 1 -- 172.21.15.46:0/3502926069 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c0a5d30 con 0x7fe72c0b3850 2024-03-29T08:21:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.567+0000 7fe730c00640 1 -- 172.21.15.46:0/3502926069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c0b3850 msgr2=0x7fe72c15d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.567+0000 7fe730c00640 1 --2- 172.21.15.46:0/3502926069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c0b3850 0x7fe72c15d480 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe71c009d90 tx=0x7fe71c01c870 comp rx=0 tx=0).stop 2024-03-29T08:21:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.567+0000 7fe730c00640 1 -- 172.21.15.46:0/3502926069 shutdown_connections 2024-03-29T08:21:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.567+0000 7fe730c00640 1 --2- 172.21.15.46:0/3502926069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c0b3850 0x7fe72c15d480 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.567+0000 7fe730c00640 1 -- 172.21.15.46:0/3502926069 >> 172.21.15.46:0/3502926069 conn(0x7fe72c066080 msgr2=0x7fe72c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe730c00640 1 -- 172.21.15.46:0/3502926069 shutdown_connections 2024-03-29T08:21:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe730c00640 1 -- 172.21.15.46:0/3502926069 wait complete. 2024-03-29T08:21:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe730c00640 1 Processor -- start 2024-03-29T08:21:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe730c00640 1 -- start start 2024-03-29T08:21:19.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe730c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c071240 0x7fe72c142a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:19.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe730c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe72c0b1630 0x7fe72c14a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:19.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe730c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe72c0b3850 0x7fe72c152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:19.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe730c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c0ad3c0 con 0x7fe72c071240 2024-03-29T08:21:19.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe730c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c14a930 con 0x7fe72c0b1630 2024-03-29T08:21:19.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe730c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c0ac960 con 0x7fe72c0b3850 2024-03-29T08:21:19.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe729e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c071240 0x7fe72c142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe729e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c071240 0x7fe72c142a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44194/0 (socket says 172.21.15.46:44194) 2024-03-29T08:21:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe729e00640 1 -- 172.21.15.46:0/3180241940 learned_addr learned my addr 172.21.15.46:0/3180241940 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe721400640 1 --2- 172.21.15.46:0/3180241940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe72c0b1630 0x7fe72c14a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe72a800640 1 --2- 172.21.15.46:0/3180241940 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe72c0b3850 0x7fe72c152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe729e00640 1 -- 172.21.15.46:0/3180241940 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe72c0b3850 msgr2=0x7fe72c152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe729e00640 1 --2- 172.21.15.46:0/3180241940 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe72c0b3850 0x7fe72c152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe729e00640 1 -- 172.21.15.46:0/3180241940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe72c0b1630 msgr2=0x7fe72c14a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe729e00640 1 --2- 172.21.15.46:0/3180241940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe72c0b1630 0x7fe72c14a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe729e00640 1 -- 172.21.15.46:0/3180241940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c009a70 con 0x7fe72c071240 2024-03-29T08:21:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe721400640 1 --2- 172.21.15.46:0/3180241940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe72c0b1630 0x7fe72c14a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe729e00640 1 --2- 172.21.15.46:0/3180241940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c071240 0x7fe72c142a50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0b0040 tx=0x7fe71c019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe722a00640 1 -- 172.21.15.46:0/3180241940 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c0ad850 con 0x7fe72c071240 2024-03-29T08:21:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe722a00640 1 -- 172.21.15.46:0/3180241940 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71c0ad9b0 con 0x7fe72c071240 2024-03-29T08:21:19.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe722a00640 1 -- 172.21.15.46:0/3180241940 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71c0add60 con 0x7fe72c071240 2024-03-29T08:21:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.571+0000 7fe730c00640 1 -- 172.21.15.46:0/3180241940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72c0acc40 con 0x7fe72c071240 2024-03-29T08:21:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.575+0000 7fe730c00640 1 -- 172.21.15.46:0/3180241940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72c14c3f0 con 0x7fe72c071240 2024-03-29T08:21:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.575+0000 7fe730c00640 1 -- 172.21.15.46:0/3180241940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe72c14c5e0 con 0x7fe72c071240 2024-03-29T08:21:19.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.575+0000 7fe722a00640 1 -- 172.21.15.46:0/3180241940 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe71c0b4050 con 0x7fe72c071240 2024-03-29T08:21:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.575+0000 7fe722a00640 1 --2- 172.21.15.46:0/3180241940 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe708066690 0x7fe708068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.575+0000 7fe722a00640 1 -- 172.21.15.46:0/3180241940 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe71c11f140 con 0x7fe72c071240 2024-03-29T08:21:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.575+0000 7fe721400640 1 --2- 172.21.15.46:0/3180241940 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe708066690 0x7fe708068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.575+0000 7fe721400640 1 --2- 172.21.15.46:0/3180241940 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe708066690 0x7fe708068b50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe72c15a870 tx=0x7fe714007cb0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:19.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.579+0000 7fe722a00640 1 -- 172.21.15.46:0/3180241940 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe72c14c5e0 con 0x7fe72c071240 2024-03-29T08:21:19.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.795+0000 7fe730c00640 1 -- 172.21.15.46:0/3180241940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe72c14c5e0 con 0x7fe72c071240 2024-03-29T08:21:19.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.799+0000 7fe722a00640 1 -- 172.21.15.46:0/3180241940 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe71c0ec550 con 0x7fe72c071240 2024-03-29T08:21:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-03-29T08:21:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.803+0000 7fe730c00640 1 -- 172.21.15.46:0/3180241940 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe708066690 msgr2=0x7fe708068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.803+0000 7fe730c00640 1 --2- 172.21.15.46:0/3180241940 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe708066690 0x7fe708068b50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe72c15a870 tx=0x7fe714007cb0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.803+0000 7fe730c00640 1 -- 172.21.15.46:0/3180241940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c071240 msgr2=0x7fe72c142a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.803+0000 7fe730c00640 1 --2- 172.21.15.46:0/3180241940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c071240 0x7fe72c142a50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0b0040 tx=0x7fe71c019610 comp rx=0 tx=0).stop 2024-03-29T08:21:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.803+0000 7fe730c00640 1 -- 172.21.15.46:0/3180241940 shutdown_connections 2024-03-29T08:21:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.803+0000 7fe730c00640 1 --2- 172.21.15.46:0/3180241940 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe708066690 0x7fe708068b50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.803+0000 7fe730c00640 1 --2- 172.21.15.46:0/3180241940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe72c0b1630 0x7fe72c14a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.803+0000 7fe730c00640 1 --2- 172.21.15.46:0/3180241940 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe72c0b3850 0x7fe72c152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.803+0000 7fe730c00640 1 --2- 172.21.15.46:0/3180241940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe72c071240 0x7fe72c142a50 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.803+0000 7fe730c00640 1 -- 172.21.15.46:0/3180241940 >> 172.21.15.46:0/3180241940 conn(0x7fe72c066080 msgr2=0x7fe72c1518b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:19.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.803+0000 7fe730c00640 1 -- 172.21.15.46:0/3180241940 shutdown_connections 2024-03-29T08:21:19.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.803+0000 7fe730c00640 1 -- 172.21.15.46:0/3180241940 wait complete. 2024-03-29T08:21:19.823 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:21:19.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.963+0000 7fda5b800640 1 Processor -- start 2024-03-29T08:21:19.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.963+0000 7fda5b800640 1 -- start start 2024-03-29T08:21:19.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.963+0000 7fda5b800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda5415a940 0x7fda540afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:19.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.963+0000 7fda5b800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda5415ad20 0x7fda540cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:19.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.963+0000 7fda5b800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda540b00d0 con 0x7fda540b0540 2024-03-29T08:21:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.963+0000 7fda5b800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda540b0ab0 con 0x7fda5415ad20 2024-03-29T08:21:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.963+0000 7fda5b800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda540b0d90 con 0x7fda5415a940 2024-03-29T08:21:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.963+0000 7fda4be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fda5415ad20 0x7fda540cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.963+0000 7fda59800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fda540b0540 0x7fda540cd070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45144/0 (socket says 172.21.15.46:45144) 2024-03-29T08:21:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.963+0000 7fda59800640 1 -- 172.21.15.46:0/3907372491 learned_addr learned my addr 172.21.15.46:0/3907372491 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.963+0000 7fda58e00640 1 --2- 172.21.15.46:0/3907372491 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda5415a940 0x7fda540afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.963+0000 7fda4be00640 1 -- 172.21.15.46:0/3907372491 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda5415a940 msgr2=0x7fda540afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.963+0000 7fda4be00640 1 --2- 172.21.15.46:0/3907372491 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda5415a940 0x7fda540afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.963+0000 7fda4be00640 1 -- 172.21.15.46:0/3907372491 >> v1:172.21.15.46:6789/0 conn(0x7fda540b0540 legacy=0x7fda540cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:19.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.963+0000 7fda4be00640 1 -- 172.21.15.46:0/3907372491 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda540b1070 con 0x7fda5415ad20 2024-03-29T08:21:19.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.967+0000 7fda4be00640 1 --2- 172.21.15.46:0/3907372491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda5415ad20 0x7fda540cecb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fda3c0097c0 tx=0x7fda3c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2f66b07340af17 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:19.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.967+0000 7fda53e00640 1 -- 172.21.15.46:0/3907372491 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0ab670 con 0x7fda5415ad20 2024-03-29T08:21:19.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.967+0000 7fda53e00640 1 -- 172.21.15.46:0/3907372491 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c0ab7d0 con 0x7fda5415ad20 2024-03-29T08:21:19.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.967+0000 7fda53e00640 1 -- 172.21.15.46:0/3907372491 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0abb60 con 0x7fda5415ad20 2024-03-29T08:21:19.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.967+0000 7fda5b800640 1 -- 172.21.15.46:0/3907372491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda5415ad20 msgr2=0x7fda540cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:19.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.967+0000 7fda5b800640 1 --2- 172.21.15.46:0/3907372491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda5415ad20 0x7fda540cecb0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fda3c0097c0 tx=0x7fda3c01c470 comp rx=0 tx=0).stop 2024-03-29T08:21:19.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.967+0000 7fda5b800640 1 -- 172.21.15.46:0/3907372491 shutdown_connections 2024-03-29T08:21:19.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.967+0000 7fda5b800640 1 --2- 172.21.15.46:0/3907372491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda5415ad20 0x7fda540cecb0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.967+0000 7fda5b800640 1 --2- 172.21.15.46:0/3907372491 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda5415a940 0x7fda540afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.967+0000 7fda5b800640 1 -- 172.21.15.46:0/3907372491 >> 172.21.15.46:0/3907372491 conn(0x7fda54089fd0 msgr2=0x7fda5408a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:19.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.967+0000 7fda5b800640 1 -- 172.21.15.46:0/3907372491 shutdown_connections 2024-03-29T08:21:19.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.967+0000 7fda5b800640 1 -- 172.21.15.46:0/3907372491 wait complete. 2024-03-29T08:21:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda5b800640 1 Processor -- start 2024-03-29T08:21:19.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda5b800640 1 -- start start 2024-03-29T08:21:19.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda5b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda540b0540 0x7fda54152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:19.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda5b800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda5415a940 0x7fda54162770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda5b800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda5415ad20 0x7fda54162cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda5b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5415a250 con 0x7fda540b0540 2024-03-29T08:21:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda5b800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5406b060 con 0x7fda5415a940 2024-03-29T08:21:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda5b800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda54152570 con 0x7fda5415ad20 2024-03-29T08:21:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda58e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda540b0540 0x7fda54152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda58e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda540b0540 0x7fda54152030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44210/0 (socket says 172.21.15.46:44210) 2024-03-29T08:21:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda58e00640 1 -- 172.21.15.46:0/2047028652 learned_addr learned my addr 172.21.15.46:0/2047028652 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda4be00640 1 --2- 172.21.15.46:0/2047028652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda5415a940 0x7fda54162770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda59800640 1 --2- 172.21.15.46:0/2047028652 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda5415ad20 0x7fda54162cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda58e00640 1 -- 172.21.15.46:0/2047028652 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda5415ad20 msgr2=0x7fda54162cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda58e00640 1 --2- 172.21.15.46:0/2047028652 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda5415ad20 0x7fda54162cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda58e00640 1 -- 172.21.15.46:0/2047028652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda5415a940 msgr2=0x7fda54162770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:19.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda58e00640 1 --2- 172.21.15.46:0/2047028652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda5415a940 0x7fda54162770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:19.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda58e00640 1 -- 172.21.15.46:0/2047028652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c009420 con 0x7fda540b0540 2024-03-29T08:21:19.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda59800640 1 --2- 172.21.15.46:0/2047028652 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda5415ad20 0x7fda54162cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:19.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda58e00640 1 --2- 172.21.15.46:0/2047028652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda540b0540 0x7fda54152030 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fda4400d800 tx=0x7fda4400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:19.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda51600640 1 -- 172.21.15.46:0/2047028652 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4400b6c0 con 0x7fda540b0540 2024-03-29T08:21:19.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda51600640 1 -- 172.21.15.46:0/2047028652 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4400b820 con 0x7fda540b0540 2024-03-29T08:21:19.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda51600640 1 -- 172.21.15.46:0/2047028652 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4400bb90 con 0x7fda540b0540 2024-03-29T08:21:19.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda5b800640 1 -- 172.21.15.46:0/2047028652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda54158950 con 0x7fda540b0540 2024-03-29T08:21:19.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.971+0000 7fda5b800640 1 -- 172.21.15.46:0/2047028652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda54152ef0 con 0x7fda540b0540 2024-03-29T08:21:19.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.975+0000 7fda5b800640 1 -- 172.21.15.46:0/2047028652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda54066c90 con 0x7fda540b0540 2024-03-29T08:21:19.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.975+0000 7fda51600640 1 -- 172.21.15.46:0/2047028652 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fda4400bcf0 con 0x7fda540b0540 2024-03-29T08:21:19.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.975+0000 7fda51600640 1 --2- 172.21.15.46:0/2047028652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fda30066690 0x7fda30068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:19.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.975+0000 7fda51600640 1 -- 172.21.15.46:0/2047028652 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda44084710 con 0x7fda540b0540 2024-03-29T08:21:19.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.975+0000 7fda4be00640 1 --2- 172.21.15.46:0/2047028652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fda30066690 0x7fda30068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:19.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.975+0000 7fda4be00640 1 --2- 172.21.15.46:0/2047028652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fda30066690 0x7fda30068b50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fda3c0af000 tx=0x7fda3c0b6040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:19.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:19.979+0000 7fda51600640 1 -- 172.21.15.46:0/2047028652 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fda44051a70 con 0x7fda540b0540 2024-03-29T08:21:20.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.179+0000 7fda5b800640 1 -- 172.21.15.46:0/2047028652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda540c3fb0 con 0x7fda540b0540 2024-03-29T08:21:20.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.179+0000 7fda51600640 1 -- 172.21.15.46:0/2047028652 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4728 (secure 0 0 0) 0x7fda44055090 con 0x7fda540b0540 2024-03-29T08:21:20.181 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:20.181 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2187826146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2187826146},{"type":"v1","addr":"172.21.15.73:6836","nonce":2187826146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/73382477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":73382477},{"type":"v1","addr":"172.21.15.73:6837","nonce":73382477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2874958170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2874958170},{"type":"v1","addr":"172.21.15.46:6834","nonce":2874958170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:21:10.337267+0000","modified":"2024-03-29T08:21:13.349387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6051},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6051":{"gid":6051,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3255474176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3255474176},{"type":"v1","addr":"172.21.15.46:6835","nonce":3255474176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-29T08:21:20.181 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 32 2024-03-29T08:21:20.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.179+0000 7fda5b800640 1 -- 172.21.15.46:0/2047028652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fda30066690 msgr2=0x7fda30068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:20.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.179+0000 7fda5b800640 1 --2- 172.21.15.46:0/2047028652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fda30066690 0x7fda30068b50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fda3c0af000 tx=0x7fda3c0b6040 comp rx=0 tx=0).stop 2024-03-29T08:21:20.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.179+0000 7fda5b800640 1 -- 172.21.15.46:0/2047028652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda540b0540 msgr2=0x7fda54152030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:20.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.179+0000 7fda5b800640 1 --2- 172.21.15.46:0/2047028652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda540b0540 0x7fda54152030 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fda4400d800 tx=0x7fda4400dcc0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.179+0000 7fda5b800640 1 -- 172.21.15.46:0/2047028652 shutdown_connections 2024-03-29T08:21:20.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.179+0000 7fda5b800640 1 --2- 172.21.15.46:0/2047028652 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fda30066690 0x7fda30068b50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.179+0000 7fda5b800640 1 --2- 172.21.15.46:0/2047028652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda5415a940 0x7fda54162770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.179+0000 7fda5b800640 1 --2- 172.21.15.46:0/2047028652 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fda5415ad20 0x7fda54162cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.179+0000 7fda5b800640 1 --2- 172.21.15.46:0/2047028652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fda540b0540 0x7fda54152030 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.179+0000 7fda5b800640 1 -- 172.21.15.46:0/2047028652 >> 172.21.15.46:0/2047028652 conn(0x7fda54089fd0 msgr2=0x7fda541571b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.179+0000 7fda5b800640 1 -- 172.21.15.46:0/2047028652 shutdown_connections 2024-03-29T08:21:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.179+0000 7fda5b800640 1 -- 172.21.15.46:0/2047028652 wait complete. 2024-03-29T08:21:20.606 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:21:10.337267+0000', 'modified': '2024-03-29T08:21:13.349387+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6051}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6051': {'gid': 6051, 'name': 'c', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/3255474176', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 3255474176}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 3255474176}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:21:20.606 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T08:21:20.606 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-29T08:21:20.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.747+0000 7f17d4c00640 1 Processor -- start 2024-03-29T08:21:20.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.747+0000 7f17d4c00640 1 -- start start 2024-03-29T08:21:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.751+0000 7f17d4c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f17d00709b0 0x7f17d00af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.751+0000 7f17d4c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d006c390 con 0x7f17d00af530 2024-03-29T08:21:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.751+0000 7f17d4c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d00afff0 con 0x7f17d0158300 2024-03-29T08:21:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.751+0000 7f17d4c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17d006c5e0 con 0x7f17d00709b0 2024-03-29T08:21:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.751+0000 7f17ce800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f17d00709b0 0x7f17d00af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.751+0000 7f17ce800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f17d00709b0 0x7f17d00af9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44220/0 (socket says 172.21.15.46:44220) 2024-03-29T08:21:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.751+0000 7f17ce800640 1 -- 172.21.15.46:0/2954211659 learned_addr learned my addr 172.21.15.46:0/2954211659 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.751+0000 7f17ce800640 1 -- 172.21.15.46:0/2954211659 >> v1:172.21.15.73:6789/0 conn(0x7f17d0158300 legacy=0x7f17d006d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:20.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.751+0000 7f17ce800640 1 -- 172.21.15.46:0/2954211659 >> v1:172.21.15.46:6789/0 conn(0x7f17d00af530 legacy=0x7f17d00702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:20.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.751+0000 7f17ce800640 1 -- 172.21.15.46:0/2954211659 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d006c8c0 con 0x7f17d00709b0 2024-03-29T08:21:20.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.751+0000 7f17ce800640 1 --2- 172.21.15.46:0/2954211659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d00709b0 0x7f17d00af9a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009e50 tx=0x7f17c801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72ffac52134a8c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:20.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.751+0000 7f17cd400640 1 -- 172.21.15.46:0/2954211659 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c80a5740 con 0x7f17d00709b0 2024-03-29T08:21:20.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.751+0000 7f17cd400640 1 -- 172.21.15.46:0/2954211659 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c80a58a0 con 0x7f17d00709b0 2024-03-29T08:21:20.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.751+0000 7f17cd400640 1 -- 172.21.15.46:0/2954211659 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c80a5c30 con 0x7f17d00709b0 2024-03-29T08:21:20.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17d4c00640 1 -- 172.21.15.46:0/2954211659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d00709b0 msgr2=0x7f17d00af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:20.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17d4c00640 1 --2- 172.21.15.46:0/2954211659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d00709b0 0x7f17d00af9a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f17c8009e50 tx=0x7f17c801c8a0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17d4c00640 1 -- 172.21.15.46:0/2954211659 shutdown_connections 2024-03-29T08:21:20.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17d4c00640 1 --2- 172.21.15.46:0/2954211659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d00709b0 0x7f17d00af9a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17d4c00640 1 -- 172.21.15.46:0/2954211659 >> 172.21.15.46:0/2954211659 conn(0x7f17d0066080 msgr2=0x7f17d0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:20.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17d4c00640 1 -- 172.21.15.46:0/2954211659 shutdown_connections 2024-03-29T08:21:20.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17d4c00640 1 -- 172.21.15.46:0/2954211659 wait complete. 2024-03-29T08:21:20.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17d4c00640 1 Processor -- start 2024-03-29T08:21:20.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17d4c00640 1 -- start start 2024-03-29T08:21:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17d4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17d00709b0 0x7f17d014cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17d4c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f17d00af530 0x7f17d014a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17d4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d0158300 0x7f17d014e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17d4c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d014d6f0 con 0x7f17d0158300 2024-03-29T08:21:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17d4c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d014d9d0 con 0x7f17d00709b0 2024-03-29T08:21:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17d4c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d014dcb0 con 0x7f17d00af530 2024-03-29T08:21:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17ce800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d0158300 0x7f17d014e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17ce800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d0158300 0x7f17d014e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44224/0 (socket says 172.21.15.46:44224) 2024-03-29T08:21:20.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17ce800640 1 -- 172.21.15.46:0/4086756840 learned_addr learned my addr 172.21.15.46:0/4086756840 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17cde00640 1 --2- 172.21.15.46:0/4086756840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17d00709b0 0x7f17d014cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17c5400640 1 --2- 172.21.15.46:0/4086756840 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f17d00af530 0x7f17d014a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17ce800640 1 -- 172.21.15.46:0/4086756840 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f17d00af530 msgr2=0x7f17d014a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17ce800640 1 --2- 172.21.15.46:0/4086756840 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f17d00af530 0x7f17d014a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17ce800640 1 -- 172.21.15.46:0/4086756840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17d00709b0 msgr2=0x7f17d014cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17ce800640 1 --2- 172.21.15.46:0/4086756840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17d00709b0 0x7f17d014cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17ce800640 1 -- 172.21.15.46:0/4086756840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c8009b30 con 0x7f17d0158300 2024-03-29T08:21:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17c5400640 1 --2- 172.21.15.46:0/4086756840 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f17d00af530 0x7f17d014a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.755+0000 7f17cde00640 1 --2- 172.21.15.46:0/4086756840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17d00709b0 0x7f17d014cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.759+0000 7f17ce800640 1 --2- 172.21.15.46:0/4086756840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d0158300 0x7f17d014e790 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f17c80af040 tx=0x7f17c8019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.759+0000 7f17c6a00640 1 -- 172.21.15.46:0/4086756840 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c80b8670 con 0x7f17d0158300 2024-03-29T08:21:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.759+0000 7f17c6a00640 1 -- 172.21.15.46:0/4086756840 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c80b87d0 con 0x7f17d0158300 2024-03-29T08:21:20.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.759+0000 7f17c6a00640 1 -- 172.21.15.46:0/4086756840 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c80b8b60 con 0x7f17d0158300 2024-03-29T08:21:20.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.759+0000 7f17d4c00640 1 -- 172.21.15.46:0/4086756840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d014df90 con 0x7f17d0158300 2024-03-29T08:21:20.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.759+0000 7f17d4c00640 1 -- 172.21.15.46:0/4086756840 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d014d400 con 0x7f17d0158300 2024-03-29T08:21:20.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.759+0000 7f17c6a00640 1 -- 172.21.15.46:0/4086756840 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f17c80b8cc0 con 0x7f17d0158300 2024-03-29T08:21:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17c6a00640 1 --2- 172.21.15.46:0/4086756840 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f17ac066690 0x7f17ac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17c6a00640 1 -- 172.21.15.46:0/4086756840 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17c811f810 con 0x7f17d0158300 2024-03-29T08:21:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17d815b000 1 Processor -- start 2024-03-29T08:21:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17cde00640 1 --2- 172.21.15.46:0/4086756840 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f17ac066690 0x7f17ac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17d815b000 1 -- start start 2024-03-29T08:21:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17cde00640 1 --2- 172.21.15.46:0/4086756840 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f17ac066690 0x7f17ac068b50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f17c00016e0 tx=0x7f17c0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17d815b000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3c3a210 0x55e2c3c3a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17d815b000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3c3ab30 0x55e2c3bf0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17d815b000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3bf08d0 0x55e2c3bf0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17d815b000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2c3ab45a0 con 0x55e2c3c3a210 2024-03-29T08:21:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17d815b000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2c3a41af0 con 0x55e2c3c3ab30 2024-03-29T08:21:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17d815b000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2c3a3f0c0 con 0x55e2c3bf08d0 2024-03-29T08:21:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17cde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3c3a210 0x55e2c3c3a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17cde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3c3a210 0x55e2c3c3a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44228/0 (socket says 172.21.15.46:44228) 2024-03-29T08:21:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17c5400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3c3ab30 0x55e2c3bf0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17cde00640 1 -- 172.21.15.46:0/380697097 learned_addr learned my addr 172.21.15.46:0/380697097 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17ce800640 1 --2- 172.21.15.46:0/380697097 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3bf08d0 0x55e2c3bf0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17cde00640 1 -- 172.21.15.46:0/380697097 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3bf08d0 msgr2=0x55e2c3bf0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17cde00640 1 --2- 172.21.15.46:0/380697097 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3bf08d0 0x55e2c3bf0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17cde00640 1 -- 172.21.15.46:0/380697097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3c3ab30 msgr2=0x55e2c3bf0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17cde00640 1 --2- 172.21.15.46:0/380697097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3c3ab30 0x55e2c3bf0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17cde00640 1 -- 172.21.15.46:0/380697097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2c3736890 con 0x55e2c3c3a210 2024-03-29T08:21:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17cde00640 1 --2- 172.21.15.46:0/380697097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3c3a210 0x55e2c3c3a5f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f17c0005360 tx=0x7f17c0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3fdabe4d688a6c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17bea00640 1 -- 172.21.15.46:0/380697097 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0003a10 con 0x55e2c3c3a210 2024-03-29T08:21:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17bea00640 1 -- 172.21.15.46:0/380697097 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c0003b70 con 0x55e2c3c3a210 2024-03-29T08:21:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17bea00640 1 -- 172.21.15.46:0/380697097 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0020450 con 0x55e2c3c3a210 2024-03-29T08:21:20.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17d815b000 1 -- 172.21.15.46:0/380697097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3c3a210 msgr2=0x55e2c3c3a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17d815b000 1 --2- 172.21.15.46:0/380697097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3c3a210 0x55e2c3c3a5f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f17c0005360 tx=0x7f17c0010000 comp rx=0 tx=0).stop 2024-03-29T08:21:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17d815b000 1 -- 172.21.15.46:0/380697097 shutdown_connections 2024-03-29T08:21:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17d815b000 1 --2- 172.21.15.46:0/380697097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3c3ab30 0x55e2c3bf0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17d815b000 1 --2- 172.21.15.46:0/380697097 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3bf08d0 0x55e2c3bf0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17d815b000 1 --2- 172.21.15.46:0/380697097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3c3a210 0x55e2c3c3a5f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17d815b000 1 -- 172.21.15.46:0/380697097 >> 172.21.15.46:0/380697097 conn(0x55e2c3bf2830 msgr2=0x55e2c3bea860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17d815b000 1 -- 172.21.15.46:0/380697097 shutdown_connections 2024-03-29T08:21:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.763+0000 7f17d815b000 1 -- 172.21.15.46:0/380697097 wait complete. 2024-03-29T08:21:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 1 -- start start 2024-03-29T08:21:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 1 Processor -- start 2024-03-29T08:21:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3bf08d0 0x55e2c3c280f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3c3a210 0x55e2c3c28630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3c3ab30 0x55e2c3c28b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2c3a3a9b0 con 0x55e2c3bf08d0 2024-03-29T08:21:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2c39d9c00 con 0x55e2c3c3a210 2024-03-29T08:21:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2c39f8070 con 0x55e2c3c3ab30 2024-03-29T08:21:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 10 client.0 _ll_register_callbacks cb 0x55e2c3bd80e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:21:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17cde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3bf08d0 0x55e2c3c280f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17cde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3bf08d0 0x55e2c3c280f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44238/0 (socket says 172.21.15.46:44238) 2024-03-29T08:21:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17cde00640 1 -- 172.21.15.46:0/1845487763 learned_addr learned my addr 172.21.15.46:0/1845487763 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17ce800640 1 --2- 172.21.15.46:0/1845487763 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3c3ab30 0x55e2c3c28b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17c5400640 1 --2- 172.21.15.46:0/1845487763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3c3a210 0x55e2c3c28630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17cde00640 1 -- 172.21.15.46:0/1845487763 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3c3ab30 msgr2=0x55e2c3c28b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17cde00640 1 --2- 172.21.15.46:0/1845487763 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3c3ab30 0x55e2c3c28b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17cde00640 1 -- 172.21.15.46:0/1845487763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3c3a210 msgr2=0x55e2c3c28630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17cde00640 1 --2- 172.21.15.46:0/1845487763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3c3a210 0x55e2c3c28630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17cde00640 1 -- 172.21.15.46:0/1845487763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c00050d0 con 0x55e2c3bf08d0 2024-03-29T08:21:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17cde00640 1 --2- 172.21.15.46:0/1845487763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3bf08d0 0x55e2c3c280f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f17c0003db0 tx=0x7f17c0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be072c9399e56ee5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17b3400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:21:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17b3400640 1 -- 172.21.15.46:0/1845487763 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0027070 con 0x55e2c3bf08d0 2024-03-29T08:21:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17b3400640 1 -- 172.21.15.46:0/1845487763 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17c0022070 con 0x55e2c3bf08d0 2024-03-29T08:21:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 1 -- 172.21.15.46:0/1845487763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2c39f75f0 con 0x55e2c3bf08d0 2024-03-29T08:21:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17b3400640 1 -- 172.21.15.46:0/1845487763 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17c0022230 con 0x55e2c3bf08d0 2024-03-29T08:21:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 10 client.6231 fetch_fsmap learned FSMap version 32 2024-03-29T08:21:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 1 -- 172.21.15.46:0/1845487763 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55e2c39ffab0 con 0x55e2c3bf08d0 2024-03-29T08:21:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17b3400640 1 -- 172.21.15.46:0/1845487763 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e2c39f75f0 con 0x55e2c3bf08d0 2024-03-29T08:21:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17b3400640 1 -- 172.21.15.46:0/1845487763 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 32) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f17c0022c00 con 0x55e2c3bf08d0 2024-03-29T08:21:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 10 client.6231 fetch_fsmap finished waiting for FSMap version 32 2024-03-29T08:21:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 10 client.6231 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:21:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 20 client.6231 populate_metadata read hostname 'smithi046' 2024-03-29T08:21:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 1 --2- 172.21.15.46:0/1845487763 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55e2c3c29410 0x55e2c3c4b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 4 client.6231 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:21:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17d815b000 1 -- 172.21.15.46:0/1845487763 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e2c3a37790 con 0x55e2c3c29410 2024-03-29T08:21:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17c5400640 1 --2- 172.21.15.46:0/1845487763 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55e2c3c29410 0x55e2c3c4b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17c5400640 1 --2- 172.21.15.46:0/1845487763 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55e2c3c29410 0x55e2c3c4b780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1894b658b44c9b98 server_cookie=f789964af5b1612 in_seq=0 out_seq=0 2024-03-29T08:21:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.767+0000 7f17b3400640 10 client.6231 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.771+0000 7f17b3400640 1 -- 172.21.15.46:0/1845487763 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e2c3a37790 con 0x55e2c3c29410 2024-03-29T08:21:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.771+0000 7f17b3400640 10 client.6231 handle_command_reply: tid=0 2024-03-29T08:21:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.771+0000 7f17d815b000 1 client.6231 shutdown 2024-03-29T08:21:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.771+0000 7f17d815b000 1 -- 172.21.15.46:0/1845487763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3bf08d0 msgr2=0x55e2c3c280f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.771+0000 7f17d815b000 1 --2- 172.21.15.46:0/1845487763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3bf08d0 0x55e2c3c280f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f17c0003db0 tx=0x7f17c0012000 comp rx=0 tx=0).stop 2024-03-29T08:21:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.771+0000 7f17d815b000 1 -- 172.21.15.46:0/1845487763 shutdown_connections 2024-03-29T08:21:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.771+0000 7f17d815b000 1 --2- 172.21.15.46:0/1845487763 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55e2c3c29410 0x55e2c3c4b780 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.771+0000 7f17d815b000 1 --2- 172.21.15.46:0/1845487763 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3c3a210 0x55e2c3c28630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.771+0000 7f17d815b000 1 --2- 172.21.15.46:0/1845487763 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3c3ab30 0x55e2c3c28b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.771+0000 7f17b3400640 0 client.6231 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.771+0000 7f17d815b000 1 --2- 172.21.15.46:0/1845487763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3bf08d0 0x55e2c3c280f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.771+0000 7f17d815b000 1 -- 172.21.15.46:0/1845487763 >> 172.21.15.46:0/1845487763 conn(0x55e2c3bf2830 msgr2=0x55e2c3bf1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.771+0000 7f17d815b000 1 -- 172.21.15.46:0/1845487763 shutdown_connections 2024-03-29T08:21:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.771+0000 7f17d815b000 1 -- 172.21.15.46:0/1845487763 wait complete. 2024-03-29T08:21:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.771+0000 7f17d815b000 20 client.6231 trim_cache size 0 max 16384 2024-03-29T08:21:20.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 Processor -- start 2024-03-29T08:21:20.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 -- start start 2024-03-29T08:21:20.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3bf2830 0x55e2c3bea6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3bf08d0 0x55e2c3beac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3c3a210 0x55e2c3c49770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2c36f7940 con 0x55e2c3bf2830 2024-03-29T08:21:20.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2c3a3e9b0 con 0x55e2c3bf08d0 2024-03-29T08:21:20.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2c3a37db0 con 0x55e2c3c3a210 2024-03-29T08:21:20.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17cde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3bf2830 0x55e2c3bea6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17cde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3bf2830 0x55e2c3bea6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44252/0 (socket says 172.21.15.46:44252) 2024-03-29T08:21:20.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17cde00640 1 -- 172.21.15.46:0/1077586162 learned_addr learned my addr 172.21.15.46:0/1077586162 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:20.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17ce800640 1 --2- 172.21.15.46:0/1077586162 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3c3a210 0x55e2c3c49770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17c5400640 1 --2- 172.21.15.46:0/1077586162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3bf08d0 0x55e2c3beac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17cde00640 1 -- 172.21.15.46:0/1077586162 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3c3a210 msgr2=0x55e2c3c49770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:20.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17cde00640 1 --2- 172.21.15.46:0/1077586162 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3c3a210 0x55e2c3c49770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17cde00640 1 -- 172.21.15.46:0/1077586162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3bf08d0 msgr2=0x55e2c3beac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:20.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17cde00640 1 --2- 172.21.15.46:0/1077586162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3bf08d0 0x55e2c3beac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17cde00640 1 -- 172.21.15.46:0/1077586162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2c39e5c00 con 0x55e2c3bf2830 2024-03-29T08:21:20.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17c5400640 1 --2- 172.21.15.46:0/1077586162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3bf08d0 0x55e2c3beac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:20.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17ce800640 1 --2- 172.21.15.46:0/1077586162 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3c3a210 0x55e2c3c49770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:20.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17cde00640 1 --2- 172.21.15.46:0/1077586162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3bf2830 0x55e2c3bea6e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f17c000d5c0 tx=0x7f17c001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c0059d7664f6db server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:20.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17bf400640 1 -- 172.21.15.46:0/1077586162 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0022840 con 0x55e2c3bf2830 2024-03-29T08:21:20.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17bf400640 1 -- 172.21.15.46:0/1077586162 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c0018030 con 0x55e2c3bf2830 2024-03-29T08:21:20.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17bf400640 1 -- 172.21.15.46:0/1077586162 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0022070 con 0x55e2c3bf2830 2024-03-29T08:21:20.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 -- 172.21.15.46:0/1077586162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3bf2830 msgr2=0x55e2c3bea6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:20.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 --2- 172.21.15.46:0/1077586162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3bf2830 0x55e2c3bea6e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f17c000d5c0 tx=0x7f17c001c000 comp rx=0 tx=0).stop 2024-03-29T08:21:20.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 -- 172.21.15.46:0/1077586162 shutdown_connections 2024-03-29T08:21:20.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 --2- 172.21.15.46:0/1077586162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3bf08d0 0x55e2c3beac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 --2- 172.21.15.46:0/1077586162 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3c3a210 0x55e2c3c49770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 --2- 172.21.15.46:0/1077586162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3bf2830 0x55e2c3bea6e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 -- 172.21.15.46:0/1077586162 >> 172.21.15.46:0/1077586162 conn(0x55e2c3c39e30 msgr2=0x55e2c3be9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:20.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 -- 172.21.15.46:0/1077586162 shutdown_connections 2024-03-29T08:21:20.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 -- 172.21.15.46:0/1077586162 wait complete. 2024-03-29T08:21:20.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 -- start start 2024-03-29T08:21:20.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.787+0000 7f17d815b000 1 Processor -- start 2024-03-29T08:21:20.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3bf08d0 0x55e2c3c97bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3bf2830 0x55e2c3c4bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3c3a210 0x55e2c3c4c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2c3a7dac0 con 0x55e2c3c3a210 2024-03-29T08:21:20.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2c39f3f80 con 0x55e2c3bf08d0 2024-03-29T08:21:20.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e2c3be3cd0 con 0x55e2c3bf2830 2024-03-29T08:21:20.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 10 client.0 _ll_register_callbacks cb 0x55e2c39ff590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:21:20.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17ce800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3c3a210 0x55e2c3c4c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17ce800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3c3a210 0x55e2c3c4c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44268/0 (socket says 172.21.15.46:44268) 2024-03-29T08:21:20.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17ce800640 1 -- 172.21.15.46:0/661532524 learned_addr learned my addr 172.21.15.46:0/661532524 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:20.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17cde00640 1 --2- 172.21.15.46:0/661532524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3bf08d0 0x55e2c3c97bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17c5400640 1 --2- 172.21.15.46:0/661532524 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3bf2830 0x55e2c3c4bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17ce800640 1 -- 172.21.15.46:0/661532524 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3bf2830 msgr2=0x55e2c3c4bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:20.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17ce800640 1 --2- 172.21.15.46:0/661532524 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3bf2830 0x55e2c3c4bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17ce800640 1 -- 172.21.15.46:0/661532524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3bf08d0 msgr2=0x55e2c3c97bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:20.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17ce800640 1 --2- 172.21.15.46:0/661532524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3bf08d0 0x55e2c3c97bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17ce800640 1 -- 172.21.15.46:0/661532524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2c39ffab0 con 0x55e2c3c3a210 2024-03-29T08:21:20.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17cde00640 1 --2- 172.21.15.46:0/661532524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3bf08d0 0x55e2c3c97bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:20.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17ce800640 1 --2- 172.21.15.46:0/661532524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3c3a210 0x55e2c3c4c420 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f17c80f0660 tx=0x7f17c80f0aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a887c08faec9d20a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:20.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17b3400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:21:20.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17b3400640 1 -- 172.21.15.46:0/661532524 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c80f0dc0 con 0x55e2c3c3a210 2024-03-29T08:21:20.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17b3400640 1 -- 172.21.15.46:0/661532524 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17c80eedc0 con 0x55e2c3c3a210 2024-03-29T08:21:20.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 -- 172.21.15.46:0/661532524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e2c3a0a460 con 0x55e2c3c3a210 2024-03-29T08:21:20.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17b3400640 1 -- 172.21.15.46:0/661532524 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f17c80efe50 con 0x55e2c3c3a210 2024-03-29T08:21:20.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 10 client.6237 fetch_fsmap learned FSMap version 32 2024-03-29T08:21:20.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 -- 172.21.15.46:0/661532524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55e2c3736cc0 con 0x55e2c3c3a210 2024-03-29T08:21:20.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17b3400640 1 -- 172.21.15.46:0/661532524 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e2c3a0a460 con 0x55e2c3c3a210 2024-03-29T08:21:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17b3400640 1 -- 172.21.15.46:0/661532524 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 32) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f17c8005d20 con 0x55e2c3c3a210 2024-03-29T08:21:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 10 client.6237 fetch_fsmap finished waiting for FSMap version 32 2024-03-29T08:21:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 10 client.6237 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:21:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 20 client.6237 populate_metadata read hostname 'smithi046' 2024-03-29T08:21:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 --2- 172.21.15.46:0/661532524 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55e2c3c3ab30 0x55e2c3c4c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 4 client.6237 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T08:21:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 -- 172.21.15.46:0/661532524 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e2c3a039c0 con 0x55e2c3c3ab30 2024-03-29T08:21:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17cde00640 1 --2- 172.21.15.46:0/661532524 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55e2c3c3ab30 0x55e2c3c4c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17cde00640 1 --2- 172.21.15.46:0/661532524 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55e2c3c3ab30 0x55e2c3c4c960 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=326e6d8695f83368 server_cookie=f72d88f0c040b858 in_seq=0 out_seq=0 2024-03-29T08:21:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17b3400640 10 client.6237 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17b3400640 1 -- 172.21.15.46:0/661532524 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55e2c3a039c0 con 0x55e2c3c3ab30 2024-03-29T08:21:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17b3400640 10 client.6237 handle_command_reply: tid=0 2024-03-29T08:21:20.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 client.6237 shutdown 2024-03-29T08:21:20.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 -- 172.21.15.46:0/661532524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3c3a210 msgr2=0x55e2c3c4c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:20.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 --2- 172.21.15.46:0/661532524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3c3a210 0x55e2c3c4c420 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f17c80f0660 tx=0x7f17c80f0aa0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 -- 172.21.15.46:0/661532524 shutdown_connections 2024-03-29T08:21:20.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 --2- 172.21.15.46:0/661532524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e2c3c3a210 0x55e2c3c4c420 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 --2- 172.21.15.46:0/661532524 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e2c3bf2830 0x55e2c3c4bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 --2- 172.21.15.46:0/661532524 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55e2c3c3ab30 0x55e2c3c4c960 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 --2- 172.21.15.46:0/661532524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e2c3bf08d0 0x55e2c3c97bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 -- 172.21.15.46:0/661532524 >> 172.21.15.46:0/661532524 conn(0x55e2c3c39e30 msgr2=0x55e2c3bea860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:20.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17b3400640 0 client.6237 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:20.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 -- 172.21.15.46:0/661532524 shutdown_connections 2024-03-29T08:21:20.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.791+0000 7f17d815b000 1 -- 172.21.15.46:0/661532524 wait complete. 2024-03-29T08:21:20.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.795+0000 7f17d815b000 20 client.6237 trim_cache size 0 max 16384 2024-03-29T08:21:20.801 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:21:20.801 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "e9b7960f-ec1f-4787-95c3-c806094d37f0", 2024-03-29T08:21:20.801 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-03-29T08:21:20.801 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6051, 2024-03-29T08:21:20.801 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-29T08:21:20.801 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-29T08:21:20.801 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-29T08:21:20.801 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 10.456979928000001, 2024-03-29T08:21:20.801 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 32, 2024-03-29T08:21:20.801 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 63, 2024-03-29T08:21:20.801 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 63, 2024-03-29T08:21:20.801 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 15.600576316 2024-03-29T08:21:20.802 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:21:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.795+0000 7f17d4c00640 1 -- 172.21.15.46:0/4086756840 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f17ac066690 msgr2=0x7f17ac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.795+0000 7f17d4c00640 1 --2- 172.21.15.46:0/4086756840 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f17ac066690 0x7f17ac068b50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f17c00016e0 tx=0x7f17c0009040 comp rx=0 tx=0).stop 2024-03-29T08:21:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.795+0000 7f17d4c00640 1 -- 172.21.15.46:0/4086756840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d0158300 msgr2=0x7f17d014e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.795+0000 7f17d4c00640 1 --2- 172.21.15.46:0/4086756840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d0158300 0x7f17d014e790 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f17c80af040 tx=0x7f17c8019640 comp rx=0 tx=0).stop 2024-03-29T08:21:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.795+0000 7f17d4c00640 1 -- 172.21.15.46:0/4086756840 shutdown_connections 2024-03-29T08:21:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.795+0000 7f17d4c00640 1 --2- 172.21.15.46:0/4086756840 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f17ac066690 0x7f17ac068b50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.795+0000 7f17d4c00640 1 --2- 172.21.15.46:0/4086756840 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17d0158300 0x7f17d014e790 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.795+0000 7f17d4c00640 1 --2- 172.21.15.46:0/4086756840 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f17d00af530 0x7f17d014a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.795+0000 7f17d4c00640 1 --2- 172.21.15.46:0/4086756840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17d00709b0 0x7f17d014cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.795+0000 7f17d4c00640 1 -- 172.21.15.46:0/4086756840 >> 172.21.15.46:0/4086756840 conn(0x7f17d0066080 msgr2=0x7f17d00702a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.795+0000 7f17d4c00640 1 -- 172.21.15.46:0/4086756840 shutdown_connections 2024-03-29T08:21:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.795+0000 7f17d4c00640 1 -- 172.21.15.46:0/4086756840 wait complete. 2024-03-29T08:21:20.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:21:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.951+0000 7fc9f3200640 1 Processor -- start 2024-03-29T08:21:20.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.951+0000 7fc9f3200640 1 -- start start 2024-03-29T08:21:20.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.955+0000 7fc9f3200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc9ec1593d0 0x7fc9ec06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.955+0000 7fc9f3200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9ec15d230 0x7fc9ec15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.955+0000 7fc9f3200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ec06d130 con 0x7fc9ec15dbd0 2024-03-29T08:21:20.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.955+0000 7fc9f3200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ec15b8f0 con 0x7fc9ec1593d0 2024-03-29T08:21:20.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.955+0000 7fc9f3200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec15ad50 con 0x7fc9ec15d230 2024-03-29T08:21:20.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.955+0000 7fc9f1200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc9ec15dbd0 0x7fc9ec15bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45160/0 (socket says 172.21.15.46:45160) 2024-03-29T08:21:20.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.955+0000 7fc9f1200640 1 -- 172.21.15.46:0/2594927619 learned_addr learned my addr 172.21.15.46:0/2594927619 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:20.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.955+0000 7fc9ebe00640 1 --2- 172.21.15.46:0/2594927619 >> v2:172.21.15.73:3300/0 conn(0x7fc9ec1593d0 0x7fc9ec06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.955+0000 7fc9e3e00640 1 --2- 172.21.15.46:0/2594927619 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9ec15d230 0x7fc9ec15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.955+0000 7fc9eb400640 1 -- 172.21.15.46:0/2594927619 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236174853 0 0) 0x7fc9ec06d130 con 0x7fc9ec15dbd0 2024-03-29T08:21:20.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.955+0000 7fc9eb400640 1 -- 172.21.15.46:0/2594927619 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9d0003530 con 0x7fc9ec15dbd0 2024-03-29T08:21:20.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.955+0000 7fc9ebe00640 1 -- 172.21.15.46:0/2594927619 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9ec15d230 msgr2=0x7fc9ec15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:20.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.955+0000 7fc9ebe00640 1 --2- 172.21.15.46:0/2594927619 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9ec15d230 0x7fc9ec15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.955+0000 7fc9ebe00640 1 -- 172.21.15.46:0/2594927619 >> v1:172.21.15.46:6789/0 conn(0x7fc9ec15dbd0 legacy=0x7fc9ec15bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:20.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.955+0000 7fc9ebe00640 1 -- 172.21.15.46:0/2594927619 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ec15e060 con 0x7fc9ec1593d0 2024-03-29T08:21:20.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9ebe00640 1 --2- 172.21.15.46:0/2594927619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9ec1593d0 0x7fc9ec06cbf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0097c0 tx=0x7fc9dc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da3bed5f3fcdec73 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:20.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9eb400640 1 -- 172.21.15.46:0/2594927619 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0a6040 con 0x7fc9ec1593d0 2024-03-29T08:21:20.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9eb400640 1 -- 172.21.15.46:0/2594927619 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9dc0aa600 con 0x7fc9ec1593d0 2024-03-29T08:21:20.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9eb400640 1 -- 172.21.15.46:0/2594927619 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0aa970 con 0x7fc9ec1593d0 2024-03-29T08:21:20.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9f3200640 1 -- 172.21.15.46:0/2594927619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9ec1593d0 msgr2=0x7fc9ec06cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:20.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9f3200640 1 --2- 172.21.15.46:0/2594927619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9ec1593d0 0x7fc9ec06cbf0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0097c0 tx=0x7fc9dc01c4a0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9f3200640 1 -- 172.21.15.46:0/2594927619 shutdown_connections 2024-03-29T08:21:20.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9f3200640 1 --2- 172.21.15.46:0/2594927619 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9ec15d230 0x7fc9ec15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9f3200640 1 --2- 172.21.15.46:0/2594927619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9ec1593d0 0x7fc9ec06cbf0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9f3200640 1 -- 172.21.15.46:0/2594927619 >> 172.21.15.46:0/2594927619 conn(0x7fc9ec066110 msgr2=0x7fc9ec069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:20.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9f3200640 1 -- 172.21.15.46:0/2594927619 shutdown_connections 2024-03-29T08:21:20.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9f3200640 1 -- 172.21.15.46:0/2594927619 wait complete. 2024-03-29T08:21:20.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9f3200640 1 Processor -- start 2024-03-29T08:21:20.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9f3200640 1 -- start start 2024-03-29T08:21:20.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9f3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9ec1593d0 0x7fc9ec145910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9f3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9ec15d230 0x7fc9ec147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9f3200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9ec15dbd0 0x7fc9ec14e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9f3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec1468d0 con 0x7fc9ec15d230 2024-03-29T08:21:20.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9f3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec145e50 con 0x7fc9ec1593d0 2024-03-29T08:21:20.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9f3200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec147d70 con 0x7fc9ec15dbd0 2024-03-29T08:21:20.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9ec15d230 0x7fc9ec147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.959+0000 7fc9e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9ec15d230 0x7fc9ec147830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44284/0 (socket says 172.21.15.46:44284) 2024-03-29T08:21:20.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9e3e00640 1 -- 172.21.15.46:0/3288861808 learned_addr learned my addr 172.21.15.46:0/3288861808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:20.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9ebe00640 1 --2- 172.21.15.46:0/3288861808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9ec1593d0 0x7fc9ec145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9f1200640 1 --2- 172.21.15.46:0/3288861808 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9ec15dbd0 0x7fc9ec14e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9e3e00640 1 -- 172.21.15.46:0/3288861808 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9ec15dbd0 msgr2=0x7fc9ec14e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:20.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9e3e00640 1 --2- 172.21.15.46:0/3288861808 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9ec15dbd0 0x7fc9ec14e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9e3e00640 1 -- 172.21.15.46:0/3288861808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9ec1593d0 msgr2=0x7fc9ec145910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:20.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9e3e00640 1 --2- 172.21.15.46:0/3288861808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9ec1593d0 0x7fc9ec145910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:20.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9e3e00640 1 -- 172.21.15.46:0/3288861808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc009420 con 0x7fc9ec15d230 2024-03-29T08:21:20.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9e3e00640 1 --2- 172.21.15.46:0/3288861808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9ec15d230 0x7fc9ec147830 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400e870 tx=0x7fc9d400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:20.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9e8c00640 1 -- 172.21.15.46:0/3288861808 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4009510 con 0x7fc9ec15d230 2024-03-29T08:21:20.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9e8c00640 1 -- 172.21.15.46:0/3288861808 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d400eec0 con 0x7fc9ec15d230 2024-03-29T08:21:20.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9e8c00640 1 -- 172.21.15.46:0/3288861808 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d400cb40 con 0x7fc9ec15d230 2024-03-29T08:21:20.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9f3200640 1 -- 172.21.15.46:0/3288861808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ec0af910 con 0x7fc9ec15d230 2024-03-29T08:21:20.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9f3200640 1 -- 172.21.15.46:0/3288861808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ec14b3b0 con 0x7fc9ec15d230 2024-03-29T08:21:20.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9f3200640 1 -- 172.21.15.46:0/3288861808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ec066c90 con 0x7fc9ec15d230 2024-03-29T08:21:20.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9e8c00640 1 -- 172.21.15.46:0/3288861808 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc9d400cca0 con 0x7fc9ec15d230 2024-03-29T08:21:20.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9e8c00640 1 --2- 172.21.15.46:0/3288861808 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc9c80666c0 0x7fc9c8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:20.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.963+0000 7fc9e8c00640 1 -- 172.21.15.46:0/3288861808 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9d40841c0 con 0x7fc9ec15d230 2024-03-29T08:21:20.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.967+0000 7fc9ebe00640 1 --2- 172.21.15.46:0/3288861808 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc9c80666c0 0x7fc9c8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:20.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.967+0000 7fc9ebe00640 1 --2- 172.21.15.46:0/3288861808 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc9c80666c0 0x7fc9c8068b80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0ae000 tx=0x7fc9dc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:20.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:20.971+0000 7fc9e8c00640 1 -- 172.21.15.46:0/3288861808 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc9d40515e0 con 0x7fc9ec15d230 2024-03-29T08:21:21.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:21.167+0000 7fc9f3200640 1 -- 172.21.15.46:0/3288861808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc9ec06d3d0 con 0x7fc9ec15d230 2024-03-29T08:21:21.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:21.167+0000 7fc9e8c00640 1 -- 172.21.15.46:0/3288861808 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc9d4054c00 con 0x7fc9ec15d230 2024-03-29T08:21:21.171 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:21:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:21.171+0000 7fc9f3200640 1 -- 172.21.15.46:0/3288861808 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc9c80666c0 msgr2=0x7fc9c8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:21.171+0000 7fc9f3200640 1 --2- 172.21.15.46:0/3288861808 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc9c80666c0 0x7fc9c8068b80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0ae000 tx=0x7fc9dc008040 comp rx=0 tx=0).stop 2024-03-29T08:21:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:21.171+0000 7fc9f3200640 1 -- 172.21.15.46:0/3288861808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9ec15d230 msgr2=0x7fc9ec147830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:21.171+0000 7fc9f3200640 1 --2- 172.21.15.46:0/3288861808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9ec15d230 0x7fc9ec147830 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400e870 tx=0x7fc9d400ec70 comp rx=0 tx=0).stop 2024-03-29T08:21:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:21.171+0000 7fc9f3200640 1 -- 172.21.15.46:0/3288861808 shutdown_connections 2024-03-29T08:21:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:21.171+0000 7fc9f3200640 1 --2- 172.21.15.46:0/3288861808 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc9c80666c0 0x7fc9c8068b80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:21.171+0000 7fc9f3200640 1 --2- 172.21.15.46:0/3288861808 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9ec15dbd0 0x7fc9ec14e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:21.171+0000 7fc9f3200640 1 --2- 172.21.15.46:0/3288861808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9ec15d230 0x7fc9ec147830 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:21.171+0000 7fc9f3200640 1 --2- 172.21.15.46:0/3288861808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9ec1593d0 0x7fc9ec145910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:21.171+0000 7fc9f3200640 1 -- 172.21.15.46:0/3288861808 >> 172.21.15.46:0/3288861808 conn(0x7fc9ec066110 msgr2=0x7fc9ec0709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:21.171+0000 7fc9f3200640 1 -- 172.21.15.46:0/3288861808 shutdown_connections 2024-03-29T08:21:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:21.171+0000 7fc9f3200640 1 -- 172.21.15.46:0/3288861808 wait complete. 2024-03-29T08:21:21.192 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:21:21.192 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:21:21.192 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-29T08:21:21.192 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T08:21:21.192 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:21:21.193 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:21:21.193 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:21:21.193 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:21:21.193 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:21:21.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:21.193 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-29T08:21:21.203 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:21:21.203 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:21:21.203 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:21:21.203 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:21:21.203 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:21:21.203 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:21:21.203 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:21:21.204 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:21:21.204 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-29T08:21:21.204 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:21:21.204 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:21:21.204 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-29T08:21:21.204 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:21:21.204 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:21:21.204 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-29T08:21:21.204 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T08:21:21.204 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42256sec preferred_lft 42256sec 2024-03-29T08:21:21.204 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-29T08:21:21.204 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:21:21.204 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:21:21.204 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:21:21.204 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-29T08:21:21.205 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:21:21.205 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:d9:58:09:4c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:21:21.205 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:21:21.205 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:21:21.205 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:21:21.205 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T08:21:21.205 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T08:21:21.205 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:21:21.205 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T08:21:21.205 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:21:21.205 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:21:21.205 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:21:21.205 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T08:21:21.206 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:21:21.206 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:21:21.206 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T08:21:21.206 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-29T08:21:21.206 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:21:21.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:21.206 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:21:21.265 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:21:21.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:21.267 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-29T08:21:21.321 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-29T08:21:21.321 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:21:21.321 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T08:21:21.322 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:21:21.322 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T08:21:21.322 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:21:21.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:21.322 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:21:21.322 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:21:21.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:21:21.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:21:21.323 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:21:21.323 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:21:21.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:21.413 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:21:21.422 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T08:21:21.422 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T08:21:21.422 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:21:21.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:21.423 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-03-29T08:21:21.483 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T08:21:21.483 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:21:21.483 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T08:21:21.535 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:21:21.535 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:21:21.535 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:21:21.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:21:21.647 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:21:21.648 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:21:21.709 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:21:21.710 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:21:21.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:21.711 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:21:21.763 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:21:21.763 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T08:21:21.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:21:21.836 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:21:21.836 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:21:21.902 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:21:21.904 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:21:21.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:21.905 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:21:21.910 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-03-29T08:21:21.907+0000 7f29e0816580 -1 init, newargv = 0x55fe275a2130 newargc=15 2024-03-29T08:21:21.911 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[39828]: starting ceph client 2024-03-29T08:21:21.917 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[39828]: starting fuse 2024-03-29T08:21:21.919 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:21:21.919 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-29T08:21:21.920 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:21:21.920 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-03-29T08:21:21.920 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-03-29T08:21:21.920 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:21:21.920 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-03-29T08:21:21.920 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:21:21.920 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-03-29T08:21:21.920 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T08:21:21.920 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-03-29T08:21:21.920 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:21:21.920 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:21:21.921 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-03-29T08:21:21.921 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-03-29T08:21:21.921 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-03-29T08:21:21.921 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-03-29T08:21:21.921 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:21:21.921 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-03-29T08:21:21.921 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:21:21.921 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:21:21.921 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:21:21.921 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-03-29T08:21:21.921 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-03-29T08:21:21.921 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-03-29T08:21:21.921 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:21:21.921 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:21:21.921 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-03-29T08:21:21.922 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:21:22.047 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.39828.asok 2024-03-29T08:21:22.232 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39828.asok 2024-03-29T08:21:22.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:22.232 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39828.asok status 2024-03-29T08:21:22.408 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:21:22.408 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-03-29T08:21:22.408 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:21:22.408 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:21:22.408 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-03-29T08:21:22.409 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-03-29T08:21:22.409 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T08:21:22.409 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "39828", 2024-03-29T08:21:22.409 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-03-29T08:21:22.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:22.409 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-03-29T08:21:22.409 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-03-29T08:21:22.409 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6249, 2024-03-29T08:21:22.409 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-03-29T08:21:22.409 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-03-29T08:21:22.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-03-29T08:21:22.409 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 6249 2024-03-29T08:21:22.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:22.409 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:21:22.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:21:22.410 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:21:22.410 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3234928501 2024-03-29T08:21:22.410 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:22.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:22.410 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:21:22.410 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:21:22.410 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:21:22.410 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3234928501 2024-03-29T08:21:22.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:22.410 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.6249 192.168.0.1:0/3234928501", 2024-03-29T08:21:22.410 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/3234928501", 2024-03-29T08:21:22.410 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-03-29T08:21:22.411 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 32, 2024-03-29T08:21:22.411 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 63, 2024-03-29T08:21:22.411 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:21:22.411 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-03-29T08:21:22.411 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-03-29T08:21:22.411 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:21:22.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:22.423 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-03-29T08:21:22.423 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:21:22.432 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-03-29T08:21:22.432 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T08:21:22.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:22.433 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:21:22.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:22.495 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-03-29T08:21:22.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:22.554 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-03-29T08:21:22.607 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:21:22.607 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:21:22.607 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:21:22.607 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:21:22.607 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-03-29T08:21:22.607 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:21:22.718 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627777 2024-03-29T08:21:22.908 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:21:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.091+0000 7f2af3200640 1 Processor -- start 2024-03-29T08:21:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.091+0000 7f2af3200640 1 -- start start 2024-03-29T08:21:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.091+0000 7f2af3200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2af4099150 0x7f2af40cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.091+0000 7f2af3200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2af4099810 0x7f2af40cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.091+0000 7f2af3200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af40cb5d0 con 0x7f2af415cb00 2024-03-29T08:21:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.091+0000 7f2af3200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af4099d10 con 0x7f2af4099150 2024-03-29T08:21:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.091+0000 7f2af3200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40caca0 con 0x7f2af4099810 2024-03-29T08:21:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.091+0000 7f2aebe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2af4099150 0x7f2af40cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.091+0000 7f2af2800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2af4099810 0x7f2af40cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.091+0000 7f2af2800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2af4099810 0x7f2af40cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:46520/0 (socket says 172.21.15.46:46520) 2024-03-29T08:21:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.091+0000 7f2aebe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2af4099150 0x7f2af40cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47240/0 (socket says 172.21.15.46:47240) 2024-03-29T08:21:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.091+0000 7f2af2800640 1 -- 172.21.15.46:0/609659262 learned_addr learned my addr 172.21.15.46:0/609659262 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.091+0000 7f2af2800640 1 -- 172.21.15.46:0/609659262 >> v2:172.21.15.73:3300/0 conn(0x7f2af4099150 msgr2=0x7f2af40cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.091+0000 7f2af2800640 1 --2- 172.21.15.46:0/609659262 >> v2:172.21.15.73:3300/0 conn(0x7f2af4099150 0x7f2af40cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.091+0000 7f2af2800640 1 -- 172.21.15.46:0/609659262 >> v1:172.21.15.73:6789/0 conn(0x7f2af415cb00 legacy=0x7f2af4098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.091+0000 7f2af2800640 1 -- 172.21.15.46:0/609659262 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af40caf80 con 0x7f2af4099810 2024-03-29T08:21:23.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.091+0000 7f2aebe00640 1 --2- 172.21.15.46:0/609659262 >> v2:172.21.15.73:3300/0 conn(0x7f2af4099150 0x7f2af40cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:21:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.095+0000 7f2af2800640 1 --2- 172.21.15.46:0/609659262 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2af4099810 0x7f2af40cbfa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009e50 tx=0x7f2aec01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f83d484f1dbb249a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.095+0000 7f2af1400640 1 -- 172.21.15.46:0/609659262 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0a57d0 con 0x7f2af4099810 2024-03-29T08:21:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.095+0000 7f2af1400640 1 -- 172.21.15.46:0/609659262 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec0a5930 con 0x7f2af4099810 2024-03-29T08:21:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.095+0000 7f2af1400640 1 -- 172.21.15.46:0/609659262 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0a5cc0 con 0x7f2af4099810 2024-03-29T08:21:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.095+0000 7f2af3200640 1 -- 172.21.15.46:0/609659262 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2af4099810 msgr2=0x7f2af40cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.095+0000 7f2af3200640 1 --2- 172.21.15.46:0/609659262 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2af4099810 0x7f2af40cbfa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009e50 tx=0x7f2aec01c970 comp rx=0 tx=0).stop 2024-03-29T08:21:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.095+0000 7f2af3200640 1 -- 172.21.15.46:0/609659262 shutdown_connections 2024-03-29T08:21:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.095+0000 7f2af3200640 1 --2- 172.21.15.46:0/609659262 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2af4099810 0x7f2af40cbfa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.095+0000 7f2af3200640 1 --2- 172.21.15.46:0/609659262 >> v2:172.21.15.73:3300/0 conn(0x7f2af4099150 0x7f2af40cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.095+0000 7f2af3200640 1 -- 172.21.15.46:0/609659262 >> 172.21.15.46:0/609659262 conn(0x7f2af40b2010 msgr2=0x7f2af40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.095+0000 7f2af3200640 1 -- 172.21.15.46:0/609659262 shutdown_connections 2024-03-29T08:21:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.095+0000 7f2af3200640 1 -- 172.21.15.46:0/609659262 wait complete. 2024-03-29T08:21:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.095+0000 7f2af3200640 1 Processor -- start 2024-03-29T08:21:23.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2af3200640 1 -- start start 2024-03-29T08:21:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2af3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af4099150 0x7f2af414b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2af3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af4099810 0x7f2af40c4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2af3200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2af415cb00 0x7f2af40aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2af3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af4159b70 con 0x7f2af4099810 2024-03-29T08:21:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2af3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af4159e50 con 0x7f2af4099150 2024-03-29T08:21:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2af3200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af415a130 con 0x7f2af415cb00 2024-03-29T08:21:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2aebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af4099810 0x7f2af40c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2aebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af4099810 0x7f2af40c4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44286/0 (socket says 172.21.15.46:44286) 2024-03-29T08:21:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2aebe00640 1 -- 172.21.15.46:0/1248730684 learned_addr learned my addr 172.21.15.46:0/1248730684 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2af2800640 1 --2- 172.21.15.46:0/1248730684 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2af415cb00 0x7f2af40aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2af1e00640 1 --2- 172.21.15.46:0/1248730684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af4099150 0x7f2af414b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2aebe00640 1 -- 172.21.15.46:0/1248730684 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2af415cb00 msgr2=0x7f2af40aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2aebe00640 1 --2- 172.21.15.46:0/1248730684 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2af415cb00 0x7f2af40aeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2aebe00640 1 -- 172.21.15.46:0/1248730684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af4099150 msgr2=0x7f2af414b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2aebe00640 1 --2- 172.21.15.46:0/1248730684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af4099150 0x7f2af414b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2aebe00640 1 -- 172.21.15.46:0/1248730684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc009420 con 0x7f2af4099810 2024-03-29T08:21:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2af1e00640 1 --2- 172.21.15.46:0/1248730684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af4099150 0x7f2af414b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2aebe00640 1 --2- 172.21.15.46:0/1248730684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af4099810 0x7f2af40c4050 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00ebe0 tx=0x7f2adc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2aea000640 1 -- 172.21.15.46:0/1248730684 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc00eee0 con 0x7f2af4099810 2024-03-29T08:21:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2aea000640 1 -- 172.21.15.46:0/1248730684 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2adc00cc30 con 0x7f2af4099810 2024-03-29T08:21:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2aea000640 1 -- 172.21.15.46:0/1248730684 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc013470 con 0x7f2af4099810 2024-03-29T08:21:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2af3200640 1 -- 172.21.15.46:0/1248730684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aec009b30 con 0x7f2af4099810 2024-03-29T08:21:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2af3200640 1 -- 172.21.15.46:0/1248730684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af415a6f0 con 0x7f2af4099810 2024-03-29T08:21:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.099+0000 7f2af3200640 1 -- 172.21.15.46:0/1248730684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af404e040 con 0x7f2af4099810 2024-03-29T08:21:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.103+0000 7f2aea000640 1 -- 172.21.15.46:0/1248730684 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2adc0135d0 con 0x7f2af4099810 2024-03-29T08:21:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.103+0000 7f2aea000640 1 --2- 172.21.15.46:0/1248730684 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2ad00666c0 0x7f2ad0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.103+0000 7f2aea000640 1 -- 172.21.15.46:0/1248730684 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2adc084430 con 0x7f2af4099810 2024-03-29T08:21:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.103+0000 7f2af1e00640 1 --2- 172.21.15.46:0/1248730684 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2ad00666c0 0x7f2ad0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.103+0000 7f2af1e00640 1 --2- 172.21.15.46:0/1248730684 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2ad00666c0 0x7f2ad0068b80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4000f70 tx=0x7f2ae4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.107+0000 7f2aea000640 1 -- 172.21.15.46:0/1248730684 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2adc0517b0 con 0x7f2af4099810 2024-03-29T08:21:23.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.299+0000 7f2af3200640 1 -- 172.21.15.46:0/1248730684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2af40cded0 con 0x7f2af4099810 2024-03-29T08:21:23.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.303+0000 7f2aea000640 1 -- 172.21.15.46:0/1248730684 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4728 (secure 0 0 0) 0x7f2adc054dd0 con 0x7f2af4099810 2024-03-29T08:21:23.304 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:23.305 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2187826146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2187826146},{"type":"v1","addr":"172.21.15.73:6836","nonce":2187826146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/73382477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":73382477},{"type":"v1","addr":"172.21.15.73:6837","nonce":73382477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2874958170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2874958170},{"type":"v1","addr":"172.21.15.46:6834","nonce":2874958170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:21:10.337267+0000","modified":"2024-03-29T08:21:13.349387+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6051},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6051":{"gid":6051,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/3255474176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3255474176},{"type":"v1","addr":"172.21.15.46:6835","nonce":3255474176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-29T08:21:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 32 2024-03-29T08:21:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.303+0000 7f2af3200640 1 -- 172.21.15.46:0/1248730684 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2ad00666c0 msgr2=0x7f2ad0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.303+0000 7f2af3200640 1 --2- 172.21.15.46:0/1248730684 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2ad00666c0 0x7f2ad0068b80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2ae4000f70 tx=0x7f2ae4009040 comp rx=0 tx=0).stop 2024-03-29T08:21:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.303+0000 7f2af3200640 1 -- 172.21.15.46:0/1248730684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af4099810 msgr2=0x7f2af40c4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.303+0000 7f2af3200640 1 --2- 172.21.15.46:0/1248730684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af4099810 0x7f2af40c4050 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2adc00ebe0 tx=0x7f2adc00c520 comp rx=0 tx=0).stop 2024-03-29T08:21:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.303+0000 7f2af3200640 1 -- 172.21.15.46:0/1248730684 shutdown_connections 2024-03-29T08:21:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.303+0000 7f2af3200640 1 --2- 172.21.15.46:0/1248730684 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2ad00666c0 0x7f2ad0068b80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.303+0000 7f2af3200640 1 --2- 172.21.15.46:0/1248730684 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2af415cb00 0x7f2af40aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.303+0000 7f2af3200640 1 --2- 172.21.15.46:0/1248730684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af4099810 0x7f2af40c4050 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.303+0000 7f2af3200640 1 --2- 172.21.15.46:0/1248730684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af4099150 0x7f2af414b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.303+0000 7f2af3200640 1 -- 172.21.15.46:0/1248730684 >> 172.21.15.46:0/1248730684 conn(0x7f2af40b2010 msgr2=0x7f2af40b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.303+0000 7f2af3200640 1 -- 172.21.15.46:0/1248730684 shutdown_connections 2024-03-29T08:21:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.303+0000 7f2af3200640 1 -- 172.21.15.46:0/1248730684 wait complete. 2024-03-29T08:21:23.326 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-03-29T08:21:23.494 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-03-29T08:21:23.494 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-29T08:21:23.495 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-03-29T08:21:23.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a9709b080 1 Processor -- start 2024-03-29T08:21:23.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a9709b080 1 -- start start 2024-03-29T08:21:23.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a9709b080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557c297f9e70 0x557c298a8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a9709b080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c2989e610 0x557c298a0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a9709b080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2989e9f0 0x557c2989cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a9709b080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c297751e0 con 0x557c2989e9f0 2024-03-29T08:21:23.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a9709b080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c29710960 con 0x557c2989e610 2024-03-29T08:21:23.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a9709b080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c29710600 con 0x557c297f9e70 2024-03-29T08:21:23.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a94c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2989e9f0 0x557c2989cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a94c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2989e9f0 0x557c2989cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44298/0 (socket says 172.21.15.46:44298) 2024-03-29T08:21:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a94c00640 1 -- 172.21.15.46:0/1530087855 learned_addr learned my addr 172.21.15.46:0/1530087855 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a94c00640 1 -- 172.21.15.46:0/1530087855 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557c297f9e70 msgr2=0x557c298a8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a94c00640 1 --2- 172.21.15.46:0/1530087855 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557c297f9e70 0x557c298a8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a94c00640 1 -- 172.21.15.46:0/1530087855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c2989e610 msgr2=0x557c298a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a94c00640 1 --2- 172.21.15.46:0/1530087855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c2989e610 0x557c298a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a94c00640 1 -- 172.21.15.46:0/1530087855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8c00a100 con 0x557c2989e9f0 2024-03-29T08:21:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a94c00640 1 --2- 172.21.15.46:0/1530087855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2989e9f0 0x557c2989cd60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0ad040 tx=0x7f9a8c003340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc220b57cdd1256 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a92400640 1 -- 172.21.15.46:0/1530087855 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0aa680 con 0x557c2989e9f0 2024-03-29T08:21:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a92400640 1 -- 172.21.15.46:0/1530087855 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c0aa7e0 con 0x557c2989e9f0 2024-03-29T08:21:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a92400640 1 -- 172.21.15.46:0/1530087855 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0aab50 con 0x557c2989e9f0 2024-03-29T08:21:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a9709b080 1 -- 172.21.15.46:0/1530087855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2989e9f0 msgr2=0x557c2989cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a9709b080 1 --2- 172.21.15.46:0/1530087855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2989e9f0 0x557c2989cd60 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c0ad040 tx=0x7f9a8c003340 comp rx=0 tx=0).stop 2024-03-29T08:21:23.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a9709b080 1 -- 172.21.15.46:0/1530087855 shutdown_connections 2024-03-29T08:21:23.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a9709b080 1 --2- 172.21.15.46:0/1530087855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c2989e610 0x557c298a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a9709b080 1 --2- 172.21.15.46:0/1530087855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2989e9f0 0x557c2989cd60 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a9709b080 1 --2- 172.21.15.46:0/1530087855 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557c297f9e70 0x557c298a8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a9709b080 1 -- 172.21.15.46:0/1530087855 >> 172.21.15.46:0/1530087855 conn(0x557c297c9cf0 msgr2=0x557c298a1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:23.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a9709b080 1 -- 172.21.15.46:0/1530087855 shutdown_connections 2024-03-29T08:21:23.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.539+0000 7f9a9709b080 1 -- 172.21.15.46:0/1530087855 wait complete. 2024-03-29T08:21:23.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a9709b080 1 Processor -- start 2024-03-29T08:21:23.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a9709b080 1 -- start start 2024-03-29T08:21:23.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a9709b080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c297f9e70 0x557c29881e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a9709b080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2989e610 0x557c29897410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a9709b080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557c2989e9f0 0x557c29897950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a9709b080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c29707690 con 0x557c2989e610 2024-03-29T08:21:23.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a9709b080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c29885ed0 con 0x557c297f9e70 2024-03-29T08:21:23.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a9709b080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c29886170 con 0x557c2989e9f0 2024-03-29T08:21:23.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a93800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2989e610 0x557c29897410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a93800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2989e610 0x557c29897410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44300/0 (socket says 172.21.15.46:44300) 2024-03-29T08:21:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a93800640 1 -- 172.21.15.46:0/852908178 learned_addr learned my addr 172.21.15.46:0/852908178 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a94c00640 1 --2- 172.21.15.46:0/852908178 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557c2989e9f0 0x557c29897950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a94200640 1 --2- 172.21.15.46:0/852908178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c297f9e70 0x557c29881e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a93800640 1 -- 172.21.15.46:0/852908178 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557c2989e9f0 msgr2=0x557c29897950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a93800640 1 --2- 172.21.15.46:0/852908178 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557c2989e9f0 0x557c29897950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a93800640 1 -- 172.21.15.46:0/852908178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c297f9e70 msgr2=0x557c29881e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a93800640 1 --2- 172.21.15.46:0/852908178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c297f9e70 0x557c29881e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a93800640 1 -- 172.21.15.46:0/852908178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c2971f8c0 con 0x557c2989e610 2024-03-29T08:21:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a94200640 1 --2- 172.21.15.46:0/852908178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c297f9e70 0x557c29881e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a93800640 1 --2- 172.21.15.46:0/852908178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2989e610 0x557c29897410 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000bd30 tx=0x7f9a80009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a91000640 1 -- 172.21.15.46:0/852908178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8000e480 con 0x557c2989e610 2024-03-29T08:21:23.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a91000640 1 -- 172.21.15.46:0/852908178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8000e5e0 con 0x557c2989e610 2024-03-29T08:21:23.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a91000640 1 -- 172.21.15.46:0/852908178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8000e990 con 0x557c2989e610 2024-03-29T08:21:23.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a9709b080 1 -- 172.21.15.46:0/852908178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557c29712d00 con 0x557c2989e610 2024-03-29T08:21:23.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.543+0000 7f9a9709b080 1 -- 172.21.15.46:0/852908178 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c29712760 con 0x557c2989e610 2024-03-29T08:21:23.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a91000640 1 -- 172.21.15.46:0/852908178 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9a8000a040 con 0x557c2989e610 2024-03-29T08:21:23.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a91000640 1 --2- 172.21.15.46:0/852908178 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9a740650a0 0x7f9a74067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a91000640 1 -- 172.21.15.46:0/852908178 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a80082d30 con 0x557c2989e610 2024-03-29T08:21:23.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a9709b080 1 --2- 172.21.15.46:0/852908178 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x557c298823c0 0x557c29886b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a9709b080 1 -- 172.21.15.46:0/852908178 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e63) v8 -- 0x557c29887090 con 0x557c298823c0 2024-03-29T08:21:23.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a94200640 1 --2- 172.21.15.46:0/852908178 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9a740650a0 0x7f9a74067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a94c00640 1 --2- 172.21.15.46:0/852908178 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x557c298823c0 0x557c29886b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a94200640 1 --2- 172.21.15.46:0/852908178 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9a740650a0 0x7f9a74067560 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400d410 tx=0x7f9a84001660 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a94c00640 1 --2- 172.21.15.46:0/852908178 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x557c298823c0 0x557c29886b50 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a94c00640 1 -- 172.21.15.46:0/852908178 <== osd.2 v2:172.21.15.46:6809/27428 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7f9a8c0aa9d0 con 0x557c298823c0 2024-03-29T08:21:23.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a9709b080 1 -- 172.21.15.46:0/852908178 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x557c298823c0 msgr2=0x557c29886b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a9709b080 1 --2- 172.21.15.46:0/852908178 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x557c298823c0 0x557c29886b50 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a9709b080 1 -- 172.21.15.46:0/852908178 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9a740650a0 msgr2=0x7f9a74067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a9709b080 1 --2- 172.21.15.46:0/852908178 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9a740650a0 0x7f9a74067560 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400d410 tx=0x7f9a84001660 comp rx=0 tx=0).stop 2024-03-29T08:21:23.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a9709b080 1 -- 172.21.15.46:0/852908178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2989e610 msgr2=0x557c29897410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a9709b080 1 --2- 172.21.15.46:0/852908178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2989e610 0x557c29897410 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9a8000bd30 tx=0x7f9a80009650 comp rx=0 tx=0).stop 2024-03-29T08:21:23.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a94c00640 1 -- 172.21.15.46:0/852908178 reap_dead start 2024-03-29T08:21:23.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a9709b080 1 -- 172.21.15.46:0/852908178 shutdown_connections 2024-03-29T08:21:23.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a9709b080 1 -- 172.21.15.46:0/852908178 >> 172.21.15.46:0/852908178 conn(0x557c297c9cf0 msgr2=0x557c298a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:23.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a9709b080 1 -- 172.21.15.46:0/852908178 shutdown_connections 2024-03-29T08:21:23.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.547+0000 7f9a9709b080 1 -- 172.21.15.46:0/852908178 wait complete. 2024-03-29T08:21:23.555 INFO:teuthology.orchestra.run.smithi046.stdout:Xfiledir_test 2024-03-29T08:21:23.556 DEBUG:teuthology.orchestra.run.smithi046:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-03-29T08:21:23.644 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:21:23.644 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627777, 2024-03-29T08:21:23.644 INFO:teuthology.orchestra.run.smithi046.stdout: "ancestors": [ 2024-03-29T08:21:23.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:23.644 INFO:teuthology.orchestra.run.smithi046.stdout: "dirino": 1099511627776, 2024-03-29T08:21:23.644 INFO:teuthology.orchestra.run.smithi046.stdout: "dname": "file", 2024-03-29T08:21:23.644 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 2 2024-03-29T08:21:23.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:21:23.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:23.644 INFO:teuthology.orchestra.run.smithi046.stdout: "dirino": 1, 2024-03-29T08:21:23.644 INFO:teuthology.orchestra.run.smithi046.stdout: "dname": "dir_test", 2024-03-29T08:21:23.645 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 4 2024-03-29T08:21:23.645 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:23.645 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-29T08:21:23.645 INFO:teuthology.orchestra.run.smithi046.stdout: "pool": 9, 2024-03-29T08:21:23.645 INFO:teuthology.orchestra.run.smithi046.stdout: "old_pools": [] 2024-03-29T08:21:23.645 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:21:23.645 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:23.648 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-03-29T08:21:23.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.735+0000 7f16a3c47080 1 Processor -- start 2024-03-29T08:21:23.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 -- start start 2024-03-29T08:21:23.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f25096030 0x559f2506cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559f250983b0 0x559f250b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f2513e720 0x559f250b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f250151e0 con 0x559f25096030 2024-03-29T08:21:23.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f24fb0960 con 0x559f2513e720 2024-03-29T08:21:23.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f24fb0600 con 0x559f250983b0 2024-03-29T08:21:23.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a0200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f25096030 0x559f2506cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a0200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f25096030 0x559f2506cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44310/0 (socket says 172.21.15.46:44310) 2024-03-29T08:21:23.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a0200640 1 -- 172.21.15.46:0/3772657929 learned_addr learned my addr 172.21.15.46:0/3772657929 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:23.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f169f800640 1 --2- 172.21.15.46:0/3772657929 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559f250983b0 0x559f250b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a0c00640 1 --2- 172.21.15.46:0/3772657929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f2513e720 0x559f250b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a0200640 1 -- 172.21.15.46:0/3772657929 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559f250983b0 msgr2=0x559f250b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:23.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a0200640 1 --2- 172.21.15.46:0/3772657929 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559f250983b0 0x559f250b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a0200640 1 -- 172.21.15.46:0/3772657929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f2513e720 msgr2=0x559f250b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:23.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a0200640 1 --2- 172.21.15.46:0/3772657929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f2513e720 0x559f250b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a0200640 1 -- 172.21.15.46:0/3772657929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1690005d30 con 0x559f25096030 2024-03-29T08:21:23.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a0c00640 1 --2- 172.21.15.46:0/3772657929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f2513e720 0x559f250b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:23.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f169f800640 1 --2- 172.21.15.46:0/3772657929 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559f250983b0 0x559f250b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:23.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a0200640 1 --2- 172.21.15.46:0/3772657929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f25096030 0x559f2506cc50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f16900af040 tx=0x7f1690003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be116225d65c460 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f169e400640 1 -- 172.21.15.46:0/3772657929 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169000a2f0 con 0x559f25096030 2024-03-29T08:21:23.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f169e400640 1 -- 172.21.15.46:0/3772657929 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16900a5740 con 0x559f25096030 2024-03-29T08:21:23.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f169e400640 1 -- 172.21.15.46:0/3772657929 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16900a5ad0 con 0x559f25096030 2024-03-29T08:21:23.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 -- 172.21.15.46:0/3772657929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f25096030 msgr2=0x559f2506cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 --2- 172.21.15.46:0/3772657929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f25096030 0x559f2506cc50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f16900af040 tx=0x7f1690003030 comp rx=0 tx=0).stop 2024-03-29T08:21:23.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 -- 172.21.15.46:0/3772657929 shutdown_connections 2024-03-29T08:21:23.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 --2- 172.21.15.46:0/3772657929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f2513e720 0x559f250b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 --2- 172.21.15.46:0/3772657929 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559f250983b0 0x559f250b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 --2- 172.21.15.46:0/3772657929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f25096030 0x559f2506cc50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 -- 172.21.15.46:0/3772657929 >> 172.21.15.46:0/3772657929 conn(0x559f25097290 msgr2=0x559f2511d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:23.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 -- 172.21.15.46:0/3772657929 shutdown_connections 2024-03-29T08:21:23.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 -- 172.21.15.46:0/3772657929 wait complete. 2024-03-29T08:21:23.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 Processor -- start 2024-03-29T08:21:23.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.739+0000 7f16a3c47080 1 -- start start 2024-03-29T08:21:23.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a3c47080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f25096030 0x559f2506b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a3c47080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f250983b0 0x559f250af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a3c47080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559f2513e720 0x559f250afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a3c47080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f24fa7690 con 0x559f25096030 2024-03-29T08:21:23.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a3c47080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f25091040 con 0x559f250983b0 2024-03-29T08:21:23.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a3c47080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f2506b720 con 0x559f2513e720 2024-03-29T08:21:23.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a0200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f25096030 0x559f2506b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a0200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f25096030 0x559f2506b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44322/0 (socket says 172.21.15.46:44322) 2024-03-29T08:21:23.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a0200640 1 -- 172.21.15.46:0/3521039866 learned_addr learned my addr 172.21.15.46:0/3521039866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:23.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a0c00640 1 --2- 172.21.15.46:0/3521039866 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559f2513e720 0x559f250afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f169f800640 1 --2- 172.21.15.46:0/3521039866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f250983b0 0x559f250af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a0200640 1 -- 172.21.15.46:0/3521039866 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559f2513e720 msgr2=0x559f250afe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a0200640 1 --2- 172.21.15.46:0/3521039866 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559f2513e720 0x559f250afe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a0200640 1 -- 172.21.15.46:0/3521039866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f250983b0 msgr2=0x559f250af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a0200640 1 --2- 172.21.15.46:0/3521039866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f250983b0 0x559f250af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a0200640 1 -- 172.21.15.46:0/3521039866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f24fbf8c0 con 0x559f25096030 2024-03-29T08:21:23.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f169f800640 1 --2- 172.21.15.46:0/3521039866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559f250983b0 0x559f250af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:23.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a0200640 1 --2- 172.21.15.46:0/3521039866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f25096030 0x559f2506b1e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f169001bb40 tx=0x7f1690018400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f169d000640 1 -- 172.21.15.46:0/3521039866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16900b7a70 con 0x559f25096030 2024-03-29T08:21:23.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f169d000640 1 -- 172.21.15.46:0/3521039866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16900b7bd0 con 0x559f25096030 2024-03-29T08:21:23.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f169d000640 1 -- 172.21.15.46:0/3521039866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16900ad420 con 0x559f25096030 2024-03-29T08:21:23.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a3c47080 1 -- 172.21.15.46:0/3521039866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559f24fb2d00 con 0x559f25096030 2024-03-29T08:21:23.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f16a3c47080 1 -- 172.21.15.46:0/3521039866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f24fb2760 con 0x559f25096030 2024-03-29T08:21:23.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.743+0000 7f169d000640 1 -- 172.21.15.46:0/3521039866 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f16900c4030 con 0x559f25096030 2024-03-29T08:21:23.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f169d000640 1 --2- 172.21.15.46:0/3521039866 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1680065120 0x7f16800675e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f169d000640 1 -- 172.21.15.46:0/3521039866 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16900c2070 con 0x559f25096030 2024-03-29T08:21:23.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f16a3c47080 1 --2- 172.21.15.46:0/3521039866 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x559f250929f0 0x559f251e4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f16a3c47080 1 -- 172.21.15.46:0/3521039866 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63) v8 -- 0x559f251e5420 con 0x559f250929f0 2024-03-29T08:21:23.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f169f800640 1 --2- 172.21.15.46:0/3521039866 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1680065120 0x7f16800675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f16a0c00640 1 --2- 172.21.15.46:0/3521039866 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x559f250929f0 0x559f251e4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f16a0c00640 1 --2- 172.21.15.46:0/3521039866 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x559f250929f0 0x559f251e4ee0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f169f800640 1 --2- 172.21.15.46:0/3521039866 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1680065120 0x7f16800675e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f168c000c10 tx=0x7f168c003750 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f16a0c00640 1 -- 172.21.15.46:0/3521039866 <== osd.2 v2:172.21.15.46:6809/27428 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v63'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f16980058a0 con 0x559f250929f0 2024-03-29T08:21:23.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f16a3c47080 1 -- 172.21.15.46:0/3521039866 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x559f250929f0 msgr2=0x559f251e4ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f16a3c47080 1 --2- 172.21.15.46:0/3521039866 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x559f250929f0 0x559f251e4ee0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f16a3c47080 1 -- 172.21.15.46:0/3521039866 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1680065120 msgr2=0x7f16800675e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f16a3c47080 1 --2- 172.21.15.46:0/3521039866 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1680065120 0x7f16800675e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f168c000c10 tx=0x7f168c003750 comp rx=0 tx=0).stop 2024-03-29T08:21:23.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f16a3c47080 1 -- 172.21.15.46:0/3521039866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f25096030 msgr2=0x559f2506b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f16a3c47080 1 --2- 172.21.15.46:0/3521039866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f25096030 0x559f2506b1e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f169001bb40 tx=0x7f1690018400 comp rx=0 tx=0).stop 2024-03-29T08:21:23.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f16a0c00640 1 -- 172.21.15.46:0/3521039866 reap_dead start 2024-03-29T08:21:23.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f16a3c47080 1 -- 172.21.15.46:0/3521039866 shutdown_connections 2024-03-29T08:21:23.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f16a3c47080 1 -- 172.21.15.46:0/3521039866 >> 172.21.15.46:0/3521039866 conn(0x559f25097290 msgr2=0x559f250aa630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:23.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f16a3c47080 1 -- 172.21.15.46:0/3521039866 shutdown_connections 2024-03-29T08:21:23.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.747+0000 7f16a3c47080 1 -- 172.21.15.46:0/3521039866 wait complete. 2024-03-29T08:21:23.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:23.755 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-03-29T08:21:23.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.895+0000 7f5134a00640 1 Processor -- start 2024-03-29T08:21:23.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.895+0000 7f5134a00640 1 -- start start 2024-03-29T08:21:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.899+0000 7f5134a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f51300cec80 0x7f5130141770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.899+0000 7f5134a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513006cd80 con 0x7f5130140f40 2024-03-29T08:21:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.899+0000 7f5134a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513006d060 con 0x7f5130141cb0 2024-03-29T08:21:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.899+0000 7f5134a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51300cbff0 con 0x7f51300cec80 2024-03-29T08:21:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.899+0000 7f5127e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f51300cec80 0x7f5130141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.899+0000 7f512de00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5130140f40 0x7f51300cb8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45180/0 (socket says 172.21.15.46:45180) 2024-03-29T08:21:23.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.899+0000 7f512de00640 1 -- 172.21.15.46:0/2304433722 learned_addr learned my addr 172.21.15.46:0/2304433722 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:23.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.899+0000 7f512d400640 1 -- 172.21.15.46:0/2304433722 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 108032527 0 0) 0x7f513006cd80 con 0x7f5130140f40 2024-03-29T08:21:23.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.899+0000 7f5127e00640 1 -- 172.21.15.46:0/2304433722 >> v1:172.21.15.73:6789/0 conn(0x7f5130141cb0 legacy=0x7f513006db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.899+0000 7f5127e00640 1 -- 172.21.15.46:0/2304433722 >> v1:172.21.15.46:6789/0 conn(0x7f5130140f40 legacy=0x7f51300cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.899+0000 7f5127e00640 1 -- 172.21.15.46:0/2304433722 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513006d340 con 0x7f51300cec80 2024-03-29T08:21:23.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.903+0000 7f5127e00640 1 --2- 172.21.15.46:0/2304433722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51300cec80 0x7f5130141770 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f51180097c0 tx=0x7f511801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1097552a3c56410 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.903+0000 7f512d400640 1 -- 172.21.15.46:0/2304433722 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180aa5f0 con 0x7f51300cec80 2024-03-29T08:21:23.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.903+0000 7f512d400640 1 -- 172.21.15.46:0/2304433722 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51180aa750 con 0x7f51300cec80 2024-03-29T08:21:23.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.903+0000 7f512d400640 1 -- 172.21.15.46:0/2304433722 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180aaae0 con 0x7f51300cec80 2024-03-29T08:21:23.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.903+0000 7f5134a00640 1 -- 172.21.15.46:0/2304433722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51300cec80 msgr2=0x7f5130141770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.903+0000 7f5134a00640 1 --2- 172.21.15.46:0/2304433722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51300cec80 0x7f5130141770 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f51180097c0 tx=0x7f511801c470 comp rx=0 tx=0).stop 2024-03-29T08:21:23.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.903+0000 7f5134a00640 1 -- 172.21.15.46:0/2304433722 shutdown_connections 2024-03-29T08:21:23.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.903+0000 7f5134a00640 1 --2- 172.21.15.46:0/2304433722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51300cec80 0x7f5130141770 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.903+0000 7f5134a00640 1 -- 172.21.15.46:0/2304433722 >> 172.21.15.46:0/2304433722 conn(0x7f51300b42d0 msgr2=0x7f5130069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:23.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.903+0000 7f5134a00640 1 -- 172.21.15.46:0/2304433722 shutdown_connections 2024-03-29T08:21:23.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.903+0000 7f5134a00640 1 -- 172.21.15.46:0/2304433722 wait complete. 2024-03-29T08:21:23.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.903+0000 7f5134a00640 1 Processor -- start 2024-03-29T08:21:23.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.903+0000 7f5134a00640 1 -- start start 2024-03-29T08:21:23.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5134a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51300cec80 0x7f5130157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5134a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5130140f40 0x7f5130159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5134a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5130141cb0 0x7f5130140080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5134a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51300a8250 con 0x7f5130140f40 2024-03-29T08:21:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5134a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5130157d80 con 0x7f51300cec80 2024-03-29T08:21:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5134a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5130159bc0 con 0x7f5130141cb0 2024-03-29T08:21:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5127e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5130140f40 0x7f5130159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5127e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5130140f40 0x7f5130159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44342/0 (socket says 172.21.15.46:44342) 2024-03-29T08:21:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5127e00640 1 -- 172.21.15.46:0/3601869289 learned_addr learned my addr 172.21.15.46:0/3601869289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f512e800640 1 --2- 172.21.15.46:0/3601869289 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5130141cb0 0x7f5130140080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f512de00640 1 --2- 172.21.15.46:0/3601869289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51300cec80 0x7f5130157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5127e00640 1 -- 172.21.15.46:0/3601869289 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5130141cb0 msgr2=0x7f5130140080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5127e00640 1 --2- 172.21.15.46:0/3601869289 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5130141cb0 0x7f5130140080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5127e00640 1 -- 172.21.15.46:0/3601869289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51300cec80 msgr2=0x7f5130157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5127e00640 1 --2- 172.21.15.46:0/3601869289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51300cec80 0x7f5130157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5127e00640 1 -- 172.21.15.46:0/3601869289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5118009420 con 0x7f5130140f40 2024-03-29T08:21:23.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5127e00640 1 --2- 172.21.15.46:0/3601869289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5130140f40 0x7f5130159680 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f51180ac040 tx=0x7f511801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5126000640 1 -- 172.21.15.46:0/3601869289 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180b4810 con 0x7f5130140f40 2024-03-29T08:21:23.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5126000640 1 -- 172.21.15.46:0/3601869289 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51180b4990 con 0x7f5130140f40 2024-03-29T08:21:23.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5126000640 1 -- 172.21.15.46:0/3601869289 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180b4d40 con 0x7f5130140f40 2024-03-29T08:21:23.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5134a00640 1 -- 172.21.15.46:0/3601869289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51300a7640 con 0x7f5130140f40 2024-03-29T08:21:23.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.907+0000 7f5134a00640 1 -- 172.21.15.46:0/3601869289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51300a7c00 con 0x7f5130140f40 2024-03-29T08:21:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f5126000640 1 -- 172.21.15.46:0/3601869289 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f51180b3390 con 0x7f5130140f40 2024-03-29T08:21:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f5126000640 1 --2- 172.21.15.46:0/3601869289 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f510c066690 0x7f510c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f5126000640 1 -- 172.21.15.46:0/3601869289 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51180a7070 con 0x7f5130140f40 2024-03-29T08:21:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f5137d7d000 1 Processor -- start 2024-03-29T08:21:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f512de00640 1 --2- 172.21.15.46:0/3601869289 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f510c066690 0x7f510c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f5137d7d000 1 -- start start 2024-03-29T08:21:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f512de00640 1 --2- 172.21.15.46:0/3601869289 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f510c066690 0x7f510c068b50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5120007a90 tx=0x7f51200079e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f5137d7d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848507c10 0x559848507ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f5137d7d000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559848508530 0x5598484bed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f5137d7d000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598484bf260 0x5598484bf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f5137d7d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559848383100 con 0x559848507c10 2024-03-29T08:21:23.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f5137d7d000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559848310d10 con 0x5598484bf260 2024-03-29T08:21:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f5137d7d000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55984830e2e0 con 0x559848508530 2024-03-29T08:21:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f512de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848507c10 0x559848507ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f512de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848507c10 0x559848507ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44356/0 (socket says 172.21.15.46:44356) 2024-03-29T08:21:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f512de00640 1 -- 172.21.15.46:0/2762578980 learned_addr learned my addr 172.21.15.46:0/2762578980 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f512e800640 1 --2- 172.21.15.46:0/2762578980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598484bf260 0x5598484bf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f5127e00640 1 --2- 172.21.15.46:0/2762578980 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559848508530 0x5598484bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f512de00640 1 -- 172.21.15.46:0/2762578980 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559848508530 msgr2=0x5598484bed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f512de00640 1 --2- 172.21.15.46:0/2762578980 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559848508530 0x5598484bed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f512de00640 1 -- 172.21.15.46:0/2762578980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598484bf260 msgr2=0x5598484bf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f512de00640 1 --2- 172.21.15.46:0/2762578980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598484bf260 0x5598484bf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f512de00640 1 -- 172.21.15.46:0/2762578980 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559848005fb0 con 0x559848507c10 2024-03-29T08:21:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f5127e00640 1 --2- 172.21.15.46:0/2762578980 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559848508530 0x5598484bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f512de00640 1 --2- 172.21.15.46:0/2762578980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848507c10 0x559848507ff0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f51200019c0 tx=0x7f5120013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fadf066fa4c6ffd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f511ea00640 1 -- 172.21.15.46:0/2762578980 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120013b80 con 0x559848507c10 2024-03-29T08:21:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f511ea00640 1 -- 172.21.15.46:0/2762578980 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5120013ce0 con 0x559848507c10 2024-03-29T08:21:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.911+0000 7f511ea00640 1 -- 172.21.15.46:0/2762578980 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120012550 con 0x559848507c10 2024-03-29T08:21:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 -- 172.21.15.46:0/2762578980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848507c10 msgr2=0x559848507ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 --2- 172.21.15.46:0/2762578980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848507c10 0x559848507ff0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f51200019c0 tx=0x7f5120013000 comp rx=0 tx=0).stop 2024-03-29T08:21:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 -- 172.21.15.46:0/2762578980 shutdown_connections 2024-03-29T08:21:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 --2- 172.21.15.46:0/2762578980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598484bf260 0x5598484bf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 --2- 172.21.15.46:0/2762578980 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559848508530 0x5598484bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 --2- 172.21.15.46:0/2762578980 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848507c10 0x559848507ff0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 -- 172.21.15.46:0/2762578980 >> 172.21.15.46:0/2762578980 conn(0x5598482b9280 msgr2=0x5598484b91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 -- 172.21.15.46:0/2762578980 shutdown_connections 2024-03-29T08:21:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 -- 172.21.15.46:0/2762578980 wait complete. 2024-03-29T08:21:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 -- start start 2024-03-29T08:21:23.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 Processor -- start 2024-03-29T08:21:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5598484bf260 0x5598484f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559848507c10 0x5598484f2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848508530 0x5598484f3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559848309bd0 con 0x559848508530 2024-03-29T08:21:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598482a8f60 con 0x559848507c10 2024-03-29T08:21:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598482c7180 con 0x5598484bf260 2024-03-29T08:21:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 10 client.0 _ll_register_callbacks cb 0x559848303240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:21:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f512e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848508530 0x5598484f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f512e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848508530 0x5598484f3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44368/0 (socket says 172.21.15.46:44368) 2024-03-29T08:21:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f512e800640 1 -- 172.21.15.46:0/2587422281 learned_addr learned my addr 172.21.15.46:0/2587422281 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f512de00640 1 --2- 172.21.15.46:0/2587422281 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5598484bf260 0x5598484f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5127e00640 1 --2- 172.21.15.46:0/2587422281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559848507c10 0x5598484f2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f512e800640 1 -- 172.21.15.46:0/2587422281 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5598484bf260 msgr2=0x5598484f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f512e800640 1 --2- 172.21.15.46:0/2587422281 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5598484bf260 0x5598484f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f512e800640 1 -- 172.21.15.46:0/2587422281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559848507c10 msgr2=0x5598484f2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f512e800640 1 --2- 172.21.15.46:0/2587422281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559848507c10 0x5598484f2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f512e800640 1 -- 172.21.15.46:0/2587422281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5120004d40 con 0x559848508530 2024-03-29T08:21:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f512e800640 1 --2- 172.21.15.46:0/2587422281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848508530 0x5598484f3430 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5128003fd0 tx=0x7f5128001c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc3cdf222fd0f2b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5113400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:21:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5113400640 1 -- 172.21.15.46:0/2587422281 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128013600 con 0x559848508530 2024-03-29T08:21:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5113400640 1 -- 172.21.15.46:0/2587422281 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5128013760 con 0x559848508530 2024-03-29T08:21:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 -- 172.21.15.46:0/2587422281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598482d9850 con 0x559848508530 2024-03-29T08:21:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5113400640 1 -- 172.21.15.46:0/2587422281 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5128013960 con 0x559848508530 2024-03-29T08:21:23.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 10 client.6285 fetch_fsmap learned FSMap version 32 2024-03-29T08:21:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5137d7d000 1 -- 172.21.15.46:0/2587422281 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5598482cebc0 con 0x559848508530 2024-03-29T08:21:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.915+0000 7f5113400640 1 -- 172.21.15.46:0/2587422281 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5598482d9850 con 0x559848508530 2024-03-29T08:21:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5113400640 1 -- 172.21.15.46:0/2587422281 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 32) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5128013ac0 con 0x559848508530 2024-03-29T08:21:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 10 client.6285 fetch_fsmap finished waiting for FSMap version 32 2024-03-29T08:21:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 10 client.6285 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-03-29T08:21:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 20 client.6285 populate_metadata read hostname 'smithi046' 2024-03-29T08:21:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 1 --2- 172.21.15.46:0/2587422281 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5598484f3e40 0x5598484f4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 4 client.6285 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:21:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 1 -- 172.21.15.46:0/2587422281 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598483069b0 con 0x5598484f3e40 2024-03-29T08:21:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f512de00640 1 --2- 172.21.15.46:0/2587422281 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5598484f3e40 0x5598484f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f512de00640 1 --2- 172.21.15.46:0/2587422281 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5598484f3e40 0x5598484f4250 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6af12e0fd44b3691 server_cookie=ff512bd9d6eb130b in_seq=0 out_seq=0 2024-03-29T08:21:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5113400640 10 client.6285 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5113400640 1 -- 172.21.15.46:0/2587422281 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5598483069b0 con 0x5598484f3e40 2024-03-29T08:21:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5113400640 10 client.6285 handle_command_reply: tid=0 2024-03-29T08:21:23.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 1 client.6285 shutdown 2024-03-29T08:21:23.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 1 -- 172.21.15.46:0/2587422281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848508530 msgr2=0x5598484f3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 1 --2- 172.21.15.46:0/2587422281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848508530 0x5598484f3430 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5128003fd0 tx=0x7f5128001c70 comp rx=0 tx=0).stop 2024-03-29T08:21:23.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 1 -- 172.21.15.46:0/2587422281 shutdown_connections 2024-03-29T08:21:23.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 1 --2- 172.21.15.46:0/2587422281 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5598484f3e40 0x5598484f4250 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 1 --2- 172.21.15.46:0/2587422281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559848507c10 0x5598484f2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 1 --2- 172.21.15.46:0/2587422281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848508530 0x5598484f3430 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 1 --2- 172.21.15.46:0/2587422281 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5598484bf260 0x5598484f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 1 -- 172.21.15.46:0/2587422281 >> 172.21.15.46:0/2587422281 conn(0x5598482b9280 msgr2=0x5598484c05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:23.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5113400640 0 client.6285 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:23.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 1 -- 172.21.15.46:0/2587422281 shutdown_connections 2024-03-29T08:21:23.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 1 -- 172.21.15.46:0/2587422281 wait complete. 2024-03-29T08:21:23.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.919+0000 7f5137d7d000 20 client.6285 trim_cache size 0 max 16384 2024-03-29T08:21:23.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f5137d7d000 1 Processor -- start 2024-03-29T08:21:23.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f5137d7d000 1 -- start start 2024-03-29T08:21:23.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f5137d7d000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5598484bf260 0x5598484c0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f5137d7d000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559848507c10 0x5598484c0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f5137d7d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848508530 0x5598485190b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f5137d7d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559847fc6c30 con 0x559848508530 2024-03-29T08:21:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f5137d7d000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559848293e80 con 0x559848507c10 2024-03-29T08:21:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f5137d7d000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598484c6520 con 0x5598484bf260 2024-03-29T08:21:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f512e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848508530 0x5598485190b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f512de00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5598484bf260 0x5598484c0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f512e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848508530 0x5598485190b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44384/0 (socket says 172.21.15.46:44384) 2024-03-29T08:21:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f512e800640 1 -- 172.21.15.46:0/806480 learned_addr learned my addr 172.21.15.46:0/806480 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f5127e00640 1 --2- 172.21.15.46:0/806480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559848507c10 0x5598484c0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f512e800640 1 -- 172.21.15.46:0/806480 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5598484bf260 msgr2=0x5598484c0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f512e800640 1 --2- 172.21.15.46:0/806480 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5598484bf260 0x5598484c0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f512e800640 1 -- 172.21.15.46:0/806480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559848507c10 msgr2=0x5598484c0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f512e800640 1 --2- 172.21.15.46:0/806480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559848507c10 0x5598484c0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f512e800640 1 -- 172.21.15.46:0/806480 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598482b4f10 con 0x559848508530 2024-03-29T08:21:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f512e800640 1 --2- 172.21.15.46:0/806480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848508530 0x5598485190b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5128007df0 tx=0x7f512800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d90e951735223e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f511f400640 1 -- 172.21.15.46:0/806480 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128005310 con 0x559848508530 2024-03-29T08:21:23.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f511f400640 1 -- 172.21.15.46:0/806480 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5128015670 con 0x559848508530 2024-03-29T08:21:23.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f511f400640 1 -- 172.21.15.46:0/806480 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5128015bc0 con 0x559848508530 2024-03-29T08:21:23.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f5137d7d000 1 -- 172.21.15.46:0/806480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848508530 msgr2=0x5598485190b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f5137d7d000 1 --2- 172.21.15.46:0/806480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848508530 0x5598485190b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5128007df0 tx=0x7f512800f000 comp rx=0 tx=0).stop 2024-03-29T08:21:23.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f5137d7d000 1 -- 172.21.15.46:0/806480 shutdown_connections 2024-03-29T08:21:23.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f5137d7d000 1 --2- 172.21.15.46:0/806480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559848507c10 0x5598484c0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f5137d7d000 1 --2- 172.21.15.46:0/806480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559848508530 0x5598485190b0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f5137d7d000 1 --2- 172.21.15.46:0/806480 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5598484bf260 0x5598484c0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.935+0000 7f5137d7d000 1 -- 172.21.15.46:0/806480 >> 172.21.15.46:0/806480 conn(0x5598482b9280 msgr2=0x5598485071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:23.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 1 -- 172.21.15.46:0/806480 shutdown_connections 2024-03-29T08:21:23.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 1 -- 172.21.15.46:0/806480 wait complete. 2024-03-29T08:21:23.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 1 -- start start 2024-03-29T08:21:23.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 1 Processor -- start 2024-03-29T08:21:23.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598484bf260 0x55984851e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559848507c10 0x55984851eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559848508530 0x55984851f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598483b5ac0 con 0x5598484bf260 2024-03-29T08:21:23.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559848303df0 con 0x559848508530 2024-03-29T08:21:23.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598483073e0 con 0x559848507c10 2024-03-29T08:21:23.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 10 client.0 _ll_register_callbacks cb 0x5598482ba7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:21:23.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f512de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598484bf260 0x55984851e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f512de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598484bf260 0x55984851e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44394/0 (socket says 172.21.15.46:44394) 2024-03-29T08:21:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f512de00640 1 -- 172.21.15.46:0/1722748404 learned_addr learned my addr 172.21.15.46:0/1722748404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5127e00640 1 --2- 172.21.15.46:0/1722748404 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559848507c10 0x55984851eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f512e800640 1 --2- 172.21.15.46:0/1722748404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559848508530 0x55984851f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f512de00640 1 -- 172.21.15.46:0/1722748404 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559848507c10 msgr2=0x55984851eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f512de00640 1 --2- 172.21.15.46:0/1722748404 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559848507c10 0x55984851eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f512de00640 1 -- 172.21.15.46:0/1722748404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559848508530 msgr2=0x55984851f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f512de00640 1 --2- 172.21.15.46:0/1722748404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559848508530 0x55984851f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f512de00640 1 -- 172.21.15.46:0/1722748404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598482cebc0 con 0x5598484bf260 2024-03-29T08:21:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f512de00640 1 --2- 172.21.15.46:0/1722748404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598484bf260 0x55984851e1a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5120024040 tx=0x7f5120007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc92ca4e2001b6c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5113400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:21:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5113400640 1 -- 172.21.15.46:0/1722748404 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51200323c0 con 0x5598484bf260 2024-03-29T08:21:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5113400640 1 -- 172.21.15.46:0/1722748404 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f512001b050 con 0x5598484bf260 2024-03-29T08:21:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 1 -- 172.21.15.46:0/1722748404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5598482d9570 con 0x5598484bf260 2024-03-29T08:21:23.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5113400640 1 -- 172.21.15.46:0/1722748404 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5120032520 con 0x5598484bf260 2024-03-29T08:21:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 10 client.6291 fetch_fsmap learned FSMap version 32 2024-03-29T08:21:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 1 -- 172.21.15.46:0/1722748404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5598482a8a80 con 0x5598484bf260 2024-03-29T08:21:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5113400640 1 -- 172.21.15.46:0/1722748404 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5598482d9570 con 0x5598484bf260 2024-03-29T08:21:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5113400640 1 -- 172.21.15.46:0/1722748404 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 32) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5120012070 con 0x5598484bf260 2024-03-29T08:21:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 10 client.6291 fetch_fsmap finished waiting for FSMap version 32 2024-03-29T08:21:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 10 client.6291 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-03-29T08:21:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 20 client.6291 populate_metadata read hostname 'smithi046' 2024-03-29T08:21:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 1 --2- 172.21.15.46:0/1722748404 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55984851c2d0 0x55984851c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 4 client.6291 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-03-29T08:21:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.939+0000 7f5137d7d000 1 -- 172.21.15.46:0/1722748404 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x559848308520 con 0x55984851c2d0 2024-03-29T08:21:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5127e00640 1 --2- 172.21.15.46:0/1722748404 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55984851c2d0 0x55984851c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5127e00640 1 --2- 172.21.15.46:0/1722748404 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55984851c2d0 0x55984851c6e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ed7fc0530666dda server_cookie=22f81a2e111dc277 in_seq=0 out_seq=0 2024-03-29T08:21:23.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5113400640 10 client.6291 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:23.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5113400640 1 -- 172.21.15.46:0/1722748404 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559848308520 con 0x55984851c2d0 2024-03-29T08:21:23.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5113400640 10 client.6291 handle_command_reply: tid=0 2024-03-29T08:21:23.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5137d7d000 1 client.6291 shutdown 2024-03-29T08:21:23.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5137d7d000 1 -- 172.21.15.46:0/1722748404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598484bf260 msgr2=0x55984851e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5137d7d000 1 --2- 172.21.15.46:0/1722748404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598484bf260 0x55984851e1a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f5120024040 tx=0x7f5120007d20 comp rx=0 tx=0).stop 2024-03-29T08:21:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5137d7d000 1 -- 172.21.15.46:0/1722748404 shutdown_connections 2024-03-29T08:21:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5137d7d000 1 --2- 172.21.15.46:0/1722748404 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55984851c2d0 0x55984851c6e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5137d7d000 1 --2- 172.21.15.46:0/1722748404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559848508530 0x55984851f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5137d7d000 1 --2- 172.21.15.46:0/1722748404 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559848507c10 0x55984851eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5137d7d000 1 --2- 172.21.15.46:0/1722748404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598484bf260 0x55984851e1a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5137d7d000 1 -- 172.21.15.46:0/1722748404 >> 172.21.15.46:0/1722748404 conn(0x5598482b9280 msgr2=0x5598484bf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:23.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5113400640 0 client.6291 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:23.948 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f1cc2600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6249=0-4194304@1} caps={6249=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6249 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x5598e82ca100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-29T08:21:23.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5137d7d000 1 -- 172.21.15.46:0/1722748404 shutdown_connections 2024-03-29T08:21:23.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5137d7d000 1 -- 172.21.15.46:0/1722748404 wait complete. 2024-03-29T08:21:23.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5137d7d000 20 client.6291 trim_cache size 0 max 16384 2024-03-29T08:21:23.949 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:21:23.949 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-29T08:21:23.949 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "1124f5d4-176a-4398-b011-fe43515cb9d8", 2024-03-29T08:21:23.949 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-29T08:21:23.949 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:21:23.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5134a00640 1 -- 172.21.15.46:0/3601869289 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f510c066690 msgr2=0x7f510c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5134a00640 1 --2- 172.21.15.46:0/3601869289 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f510c066690 0x7f510c068b50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5120007a90 tx=0x7f51200079e0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5134a00640 1 -- 172.21.15.46:0/3601869289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5130140f40 msgr2=0x7f5130159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:23.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5134a00640 1 --2- 172.21.15.46:0/3601869289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5130140f40 0x7f5130159680 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f51180ac040 tx=0x7f511801acb0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5134a00640 1 -- 172.21.15.46:0/3601869289 shutdown_connections 2024-03-29T08:21:23.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5134a00640 1 --2- 172.21.15.46:0/3601869289 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f510c066690 0x7f510c068b50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5134a00640 1 --2- 172.21.15.46:0/3601869289 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5130141cb0 0x7f5130140080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5134a00640 1 --2- 172.21.15.46:0/3601869289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5130140f40 0x7f5130159680 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5134a00640 1 --2- 172.21.15.46:0/3601869289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51300cec80 0x7f5130157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:23.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5134a00640 1 -- 172.21.15.46:0/3601869289 >> 172.21.15.46:0/3601869289 conn(0x7f51300b42d0 msgr2=0x7f51300cb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:23.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5134a00640 1 -- 172.21.15.46:0/3601869289 shutdown_connections 2024-03-29T08:21:23.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:23.943+0000 7f5134a00640 1 -- 172.21.15.46:0/3601869289 wait complete. 2024-03-29T08:21:23.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:23.965 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-29T08:21:24.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.103+0000 7ff281a00640 1 Processor -- start 2024-03-29T08:21:24.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.103+0000 7ff281a00640 1 -- start start 2024-03-29T08:21:24.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.107+0000 7ff281a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff27c15dbb0 0x7ff27c06ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.107+0000 7ff281a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff27c0b3f20 0x7ff27c06b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.107+0000 7ff281a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff27c0b1d30 0x7ff27c0b2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.107+0000 7ff281a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27c157490 con 0x7ff27c0b1d30 2024-03-29T08:21:24.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.107+0000 7ff281a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27c0b2c90 con 0x7ff27c15dbb0 2024-03-29T08:21:24.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.107+0000 7ff281a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c0b29b0 con 0x7ff27c0b3f20 2024-03-29T08:21:24.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.107+0000 7ff27b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff27c0b1d30 0x7ff27c0b2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.107+0000 7ff27b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff27c0b1d30 0x7ff27c0b2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44402/0 (socket says 172.21.15.46:44402) 2024-03-29T08:21:24.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.107+0000 7ff27b400640 1 -- 172.21.15.46:0/4222884317 learned_addr learned my addr 172.21.15.46:0/4222884317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:24.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.107+0000 7ff272000640 1 --2- 172.21.15.46:0/4222884317 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff27c0b3f20 0x7ff27c06b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.107+0000 7ff27aa00640 1 --2- 172.21.15.46:0/4222884317 >> v2:172.21.15.73:3300/0 conn(0x7ff27c15dbb0 0x7ff27c06ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.107+0000 7ff27b400640 1 -- 172.21.15.46:0/4222884317 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff27c0b3f20 msgr2=0x7ff27c06b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:24.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.107+0000 7ff27b400640 1 --2- 172.21.15.46:0/4222884317 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff27c0b3f20 0x7ff27c06b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.107+0000 7ff27b400640 1 -- 172.21.15.46:0/4222884317 >> v2:172.21.15.73:3300/0 conn(0x7ff27c15dbb0 msgr2=0x7ff27c06ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:24.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.107+0000 7ff27b400640 1 --2- 172.21.15.46:0/4222884317 >> v2:172.21.15.73:3300/0 conn(0x7ff27c15dbb0 0x7ff27c06ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.107+0000 7ff27b400640 1 -- 172.21.15.46:0/4222884317 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c0b3440 con 0x7ff27c0b1d30 2024-03-29T08:21:24.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.107+0000 7ff27aa00640 1 --2- 172.21.15.46:0/4222884317 >> v2:172.21.15.73:3300/0 conn(0x7ff27c15dbb0 0x7ff27c06ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.111+0000 7ff27b400640 1 --2- 172.21.15.46:0/4222884317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27c0b1d30 0x7ff27c0b2160 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff27c066210 tx=0x7ff27401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0323c7803799433 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.111+0000 7ff27a000640 1 -- 172.21.15.46:0/4222884317 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2740ab650 con 0x7ff27c0b1d30 2024-03-29T08:21:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.111+0000 7ff27a000640 1 -- 172.21.15.46:0/4222884317 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2740ab7b0 con 0x7ff27c0b1d30 2024-03-29T08:21:24.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.111+0000 7ff27a000640 1 -- 172.21.15.46:0/4222884317 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2740abb40 con 0x7ff27c0b1d30 2024-03-29T08:21:24.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.111+0000 7ff281a00640 1 -- 172.21.15.46:0/4222884317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27c0b1d30 msgr2=0x7ff27c0b2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:24.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.111+0000 7ff281a00640 1 --2- 172.21.15.46:0/4222884317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27c0b1d30 0x7ff27c0b2160 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff27c066210 tx=0x7ff27401c490 comp rx=0 tx=0).stop 2024-03-29T08:21:24.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.111+0000 7ff281a00640 1 -- 172.21.15.46:0/4222884317 shutdown_connections 2024-03-29T08:21:24.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.111+0000 7ff281a00640 1 --2- 172.21.15.46:0/4222884317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27c0b1d30 0x7ff27c0b2160 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.111+0000 7ff281a00640 1 --2- 172.21.15.46:0/4222884317 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff27c0b3f20 0x7ff27c06b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.111+0000 7ff281a00640 1 --2- 172.21.15.46:0/4222884317 >> v2:172.21.15.73:3300/0 conn(0x7ff27c15dbb0 0x7ff27c06ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.111+0000 7ff281a00640 1 -- 172.21.15.46:0/4222884317 >> 172.21.15.46:0/4222884317 conn(0x7ff27c070ad0 msgr2=0x7ff27c070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:24.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.111+0000 7ff281a00640 1 -- 172.21.15.46:0/4222884317 shutdown_connections 2024-03-29T08:21:24.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.111+0000 7ff281a00640 1 -- 172.21.15.46:0/4222884317 wait complete. 2024-03-29T08:21:24.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.111+0000 7ff281a00640 1 Processor -- start 2024-03-29T08:21:24.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.111+0000 7ff281a00640 1 -- start start 2024-03-29T08:21:24.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff281a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27c0b1d30 0x7ff27c155f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff281a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff27c0b3f20 0x7ff27c15edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff281a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff27c15dbb0 0x7ff27c14e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff281a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c15f330 con 0x7ff27c0b1d30 2024-03-29T08:21:24.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff281a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c15fe50 con 0x7ff27c15dbb0 2024-03-29T08:21:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff281a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c14b240 con 0x7ff27c0b3f20 2024-03-29T08:21:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff27aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27c0b1d30 0x7ff27c155f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff27aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27c0b1d30 0x7ff27c155f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44412/0 (socket says 172.21.15.46:44412) 2024-03-29T08:21:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff27aa00640 1 -- 172.21.15.46:0/1868962586 learned_addr learned my addr 172.21.15.46:0/1868962586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff272000640 1 --2- 172.21.15.46:0/1868962586 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff27c0b3f20 0x7ff27c15edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff27b400640 1 --2- 172.21.15.46:0/1868962586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff27c15dbb0 0x7ff27c14e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff27aa00640 1 -- 172.21.15.46:0/1868962586 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff27c0b3f20 msgr2=0x7ff27c15edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff27aa00640 1 --2- 172.21.15.46:0/1868962586 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff27c0b3f20 0x7ff27c15edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff27aa00640 1 -- 172.21.15.46:0/1868962586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff27c15dbb0 msgr2=0x7ff27c14e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff27aa00640 1 --2- 172.21.15.46:0/1868962586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff27c15dbb0 0x7ff27c14e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff27aa00640 1 -- 172.21.15.46:0/1868962586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2740094d0 con 0x7ff27c0b1d30 2024-03-29T08:21:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff27aa00640 1 --2- 172.21.15.46:0/1868962586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27c0b1d30 0x7ff27c155f70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff26c007ef0 tx=0x7ff26c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:24.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff273400640 1 -- 172.21.15.46:0/1868962586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c003ba0 con 0x7ff27c0b1d30 2024-03-29T08:21:24.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff273400640 1 -- 172.21.15.46:0/1868962586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26c00f040 con 0x7ff27c0b1d30 2024-03-29T08:21:24.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff281a00640 1 -- 172.21.15.46:0/1868962586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27c15f5b0 con 0x7ff27c0b1d30 2024-03-29T08:21:24.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff273400640 1 -- 172.21.15.46:0/1868962586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c00de80 con 0x7ff27c0b1d30 2024-03-29T08:21:24.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff281a00640 1 -- 172.21.15.46:0/1868962586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27c15fb70 con 0x7ff27c0b1d30 2024-03-29T08:21:24.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.115+0000 7ff273400640 1 -- 172.21.15.46:0/1868962586 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff26c004030 con 0x7ff27c0b1d30 2024-03-29T08:21:24.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff273400640 1 --2- 172.21.15.46:0/1868962586 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff258066690 0x7ff258068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff273400640 1 -- 172.21.15.46:0/1868962586 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff26c084490 con 0x7ff27c0b1d30 2024-03-29T08:21:24.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff272000640 1 --2- 172.21.15.46:0/1868962586 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff258066690 0x7ff258068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff284e06000 1 Processor -- start 2024-03-29T08:21:24.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff284e06000 1 -- start start 2024-03-29T08:21:24.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff272000640 1 --2- 172.21.15.46:0/1868962586 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff258066690 0x7ff258068b50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff27c06a810 tx=0x7ff264009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:24.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff284e06000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d64e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff284e06000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d65390 0x5633b2d184c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff284e06000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d18a00 0x5633b2d18de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff284e06000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b2be3630 con 0x5633b2d64a70 2024-03-29T08:21:24.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff284e06000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b2b70bd0 con 0x5633b2d18a00 2024-03-29T08:21:24.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff284e06000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633b2b6e1a0 con 0x5633b2d65390 2024-03-29T08:21:24.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff27aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d64e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff27aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d64e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44420/0 (socket says 172.21.15.46:44420) 2024-03-29T08:21:24.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff27aa00640 1 -- 172.21.15.46:0/2328775894 learned_addr learned my addr 172.21.15.46:0/2328775894 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:24.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff27aa00640 1 -- 172.21.15.46:0/2328775894 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d65390 msgr2=0x5633b2d184c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:24.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff272000640 1 --2- 172.21.15.46:0/2328775894 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d65390 0x5633b2d184c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff27aa00640 1 --2- 172.21.15.46:0/2328775894 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d65390 0x5633b2d184c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff27aa00640 1 -- 172.21.15.46:0/2328775894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d18a00 msgr2=0x5633b2d18de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:24.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff27aa00640 1 --2- 172.21.15.46:0/2328775894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d18a00 0x5633b2d18de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff27aa00640 1 -- 172.21.15.46:0/2328775894 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633b2865e20 con 0x5633b2d64a70 2024-03-29T08:21:24.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff27aa00640 1 --2- 172.21.15.46:0/2328775894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d64e50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff26c055260 tx=0x7ff26c054f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f96a9528a326ead4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:24.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff26b400640 1 -- 172.21.15.46:0/2328775894 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c05d8a0 con 0x5633b2d64a70 2024-03-29T08:21:24.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff26b400640 1 -- 172.21.15.46:0/2328775894 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26c053900 con 0x5633b2d64a70 2024-03-29T08:21:24.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff26b400640 1 -- 172.21.15.46:0/2328775894 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c0840d0 con 0x5633b2d64a70 2024-03-29T08:21:24.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff284e06000 1 -- 172.21.15.46:0/2328775894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 msgr2=0x5633b2d64e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:24.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff284e06000 1 --2- 172.21.15.46:0/2328775894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d64e50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff26c055260 tx=0x7ff26c054f80 comp rx=0 tx=0).stop 2024-03-29T08:21:24.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff284e06000 1 -- 172.21.15.46:0/2328775894 shutdown_connections 2024-03-29T08:21:24.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff284e06000 1 --2- 172.21.15.46:0/2328775894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d18a00 0x5633b2d18de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff284e06000 1 --2- 172.21.15.46:0/2328775894 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d65390 0x5633b2d184c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff284e06000 1 --2- 172.21.15.46:0/2328775894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d64e50 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff284e06000 1 -- 172.21.15.46:0/2328775894 >> 172.21.15.46:0/2328775894 conn(0x5633b2b19160 msgr2=0x5633b2d47820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:24.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff284e06000 1 -- 172.21.15.46:0/2328775894 shutdown_connections 2024-03-29T08:21:24.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.119+0000 7ff284e06000 1 -- 172.21.15.46:0/2328775894 wait complete. 2024-03-29T08:21:24.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 1 -- start start 2024-03-29T08:21:24.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 1 Processor -- start 2024-03-29T08:21:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d18a00 0x5633b2d217b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d64a70 0x5633b2d48a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d65390 0x5633b2d48f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b2b69a90 con 0x5633b2d65390 2024-03-29T08:21:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b2b08d40 con 0x5633b2d18a00 2024-03-29T08:21:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633b2b26f30 con 0x5633b2d64a70 2024-03-29T08:21:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 10 client.0 _ll_register_callbacks cb 0x5633b2bcb810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:21:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff27b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d65390 0x5633b2d48f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff27b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d65390 0x5633b2d48f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44424/0 (socket says 172.21.15.46:44424) 2024-03-29T08:21:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff27b400640 1 -- 172.21.15.46:0/2728941160 learned_addr learned my addr 172.21.15.46:0/2728941160 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:24.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff272000640 1 --2- 172.21.15.46:0/2728941160 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d64a70 0x5633b2d48a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff27aa00640 1 --2- 172.21.15.46:0/2728941160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d18a00 0x5633b2d217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff27b400640 1 -- 172.21.15.46:0/2728941160 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d64a70 msgr2=0x5633b2d48a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff27b400640 1 --2- 172.21.15.46:0/2728941160 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d64a70 0x5633b2d48a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff27b400640 1 -- 172.21.15.46:0/2728941160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d18a00 msgr2=0x5633b2d217b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff27b400640 1 --2- 172.21.15.46:0/2728941160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d18a00 0x5633b2d217b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff27b400640 1 -- 172.21.15.46:0/2728941160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27c15fb70 con 0x5633b2d65390 2024-03-29T08:21:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff27b400640 1 --2- 172.21.15.46:0/2728941160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d65390 0x5633b2d48f90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff2740ad040 tx=0x7ff27401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c6b3984f596825 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff25fe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:21:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff25fe00640 1 -- 172.21.15.46:0/2728941160 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2740abbd0 con 0x5633b2d65390 2024-03-29T08:21:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff25fe00640 1 -- 172.21.15.46:0/2728941160 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2740abd30 con 0x5633b2d65390 2024-03-29T08:21:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 1 -- 172.21.15.46:0/2728941160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633b2b683e0 con 0x5633b2d65390 2024-03-29T08:21:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff25fe00640 1 -- 172.21.15.46:0/2728941160 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2740b43f0 con 0x5633b2d65390 2024-03-29T08:21:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 10 client.6303 fetch_fsmap learned FSMap version 32 2024-03-29T08:21:24.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 1 -- 172.21.15.46:0/2728941160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5633b2b2eaa0 con 0x5633b2d65390 2024-03-29T08:21:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff25fe00640 1 -- 172.21.15.46:0/2728941160 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5633b2b683e0 con 0x5633b2d65390 2024-03-29T08:21:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff25fe00640 1 -- 172.21.15.46:0/2728941160 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 32) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff2740b4550 con 0x5633b2d65390 2024-03-29T08:21:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 10 client.6303 fetch_fsmap finished waiting for FSMap version 32 2024-03-29T08:21:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 10 client.6303 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-03-29T08:21:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 20 client.6303 populate_metadata read hostname 'smithi046' 2024-03-29T08:21:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 1 --2- 172.21.15.46:0/2728941160 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5633b2d6c2d0 0x5633b2d6c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 4 client.6303 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:21:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff284e06000 1 -- 172.21.15.46:0/2728941160 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633b2b39d00 con 0x5633b2d6c2d0 2024-03-29T08:21:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.123+0000 7ff27aa00640 1 --2- 172.21.15.46:0/2728941160 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5633b2d6c2d0 0x5633b2d6c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.127+0000 7ff27aa00640 1 --2- 172.21.15.46:0/2728941160 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5633b2d6c2d0 0x5633b2d6c6b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a01d143929cb666 server_cookie=6e5547e00870202b in_seq=0 out_seq=0 2024-03-29T08:21:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.127+0000 7ff25fe00640 10 client.6303 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.127+0000 7ff25fe00640 1 -- 172.21.15.46:0/2728941160 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5633b2b39d00 con 0x5633b2d6c2d0 2024-03-29T08:21:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.127+0000 7ff25fe00640 10 client.6303 handle_command_reply: tid=0 2024-03-29T08:21:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.127+0000 7ff284e06000 1 client.6303 shutdown 2024-03-29T08:21:24.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.127+0000 7ff284e06000 1 -- 172.21.15.46:0/2728941160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d65390 msgr2=0x5633b2d48f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.127+0000 7ff284e06000 1 --2- 172.21.15.46:0/2728941160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d65390 0x5633b2d48f90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff2740ad040 tx=0x7ff27401acd0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.127+0000 7ff284e06000 1 -- 172.21.15.46:0/2728941160 shutdown_connections 2024-03-29T08:21:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.127+0000 7ff284e06000 1 --2- 172.21.15.46:0/2728941160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d65390 0x5633b2d48f90 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.127+0000 7ff284e06000 1 --2- 172.21.15.46:0/2728941160 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d64a70 0x5633b2d48a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.127+0000 7ff284e06000 1 --2- 172.21.15.46:0/2728941160 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5633b2d6c2d0 0x5633b2d6c6b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.127+0000 7ff284e06000 1 --2- 172.21.15.46:0/2728941160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d18a00 0x5633b2d217b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.127+0000 7ff284e06000 1 -- 172.21.15.46:0/2728941160 >> 172.21.15.46:0/2728941160 conn(0x5633b2b19160 msgr2=0x5633b2d17bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.127+0000 7ff25fe00640 0 client.6303 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.127+0000 7ff284e06000 1 -- 172.21.15.46:0/2728941160 shutdown_connections 2024-03-29T08:21:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.127+0000 7ff284e06000 1 -- 172.21.15.46:0/2728941160 wait complete. 2024-03-29T08:21:24.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.127+0000 7ff284e06000 20 client.6303 trim_cache size 0 max 16384 2024-03-29T08:21:24.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff284e06000 1 Processor -- start 2024-03-29T08:21:24.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff284e06000 1 -- start start 2024-03-29T08:21:24.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff284e06000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d18a00 0x5633b2d17c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff284e06000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d18170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff284e06000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d65390 0x5633b2d18de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff284e06000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b2826aa0 con 0x5633b2d64a70 2024-03-29T08:21:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff284e06000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b2b2ac60 con 0x5633b2d65390 2024-03-29T08:21:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff284e06000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633b2b33640 con 0x5633b2d18a00 2024-03-29T08:21:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff272000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d18170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff272000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d18170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44430/0 (socket says 172.21.15.46:44430) 2024-03-29T08:21:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff272000640 1 -- 172.21.15.46:0/2451200495 learned_addr learned my addr 172.21.15.46:0/2451200495 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff27aa00640 1 --2- 172.21.15.46:0/2451200495 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d18a00 0x5633b2d17c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff27b400640 1 --2- 172.21.15.46:0/2451200495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d65390 0x5633b2d18de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff272000640 1 -- 172.21.15.46:0/2451200495 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d18a00 msgr2=0x5633b2d17c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff272000640 1 --2- 172.21.15.46:0/2451200495 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d18a00 0x5633b2d17c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff272000640 1 -- 172.21.15.46:0/2451200495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d65390 msgr2=0x5633b2d18de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:24.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff272000640 1 --2- 172.21.15.46:0/2451200495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d65390 0x5633b2d18de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff272000640 1 -- 172.21.15.46:0/2451200495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633b2b14df0 con 0x5633b2d64a70 2024-03-29T08:21:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff27b400640 1 --2- 172.21.15.46:0/2451200495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d65390 0x5633b2d18de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:24.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff272000640 1 --2- 172.21.15.46:0/2451200495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d18170 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff2640044b0 tx=0x7ff264014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbef4d6e07cf1de2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff26be00640 1 -- 172.21.15.46:0/2451200495 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff264003830 con 0x5633b2d64a70 2024-03-29T08:21:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff26be00640 1 -- 172.21.15.46:0/2451200495 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff264003990 con 0x5633b2d64a70 2024-03-29T08:21:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff26be00640 1 -- 172.21.15.46:0/2451200495 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff264003d00 con 0x5633b2d64a70 2024-03-29T08:21:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff284e06000 1 -- 172.21.15.46:0/2451200495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 msgr2=0x5633b2d18170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff284e06000 1 --2- 172.21.15.46:0/2451200495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d18170 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff2640044b0 tx=0x7ff264014000 comp rx=0 tx=0).stop 2024-03-29T08:21:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff284e06000 1 -- 172.21.15.46:0/2451200495 shutdown_connections 2024-03-29T08:21:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff284e06000 1 --2- 172.21.15.46:0/2451200495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d65390 0x5633b2d18de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff284e06000 1 --2- 172.21.15.46:0/2451200495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d18170 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff284e06000 1 --2- 172.21.15.46:0/2451200495 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d18a00 0x5633b2d17c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff284e06000 1 -- 172.21.15.46:0/2451200495 >> 172.21.15.46:0/2451200495 conn(0x5633b2b19160 msgr2=0x5633b2d20810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff284e06000 1 -- 172.21.15.46:0/2451200495 shutdown_connections 2024-03-29T08:21:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.143+0000 7ff284e06000 1 -- 172.21.15.46:0/2451200495 wait complete. 2024-03-29T08:21:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 1 -- start start 2024-03-29T08:21:24.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 1 Processor -- start 2024-03-29T08:21:24.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d18a00 0x5633b2dc45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d1c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d65390 0x5633b2d1c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b2b35390 con 0x5633b2d64a70 2024-03-29T08:21:24.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633b2b6c5f0 con 0x5633b2d18a00 2024-03-29T08:21:24.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633b2b6da90 con 0x5633b2d65390 2024-03-29T08:21:24.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 10 client.0 _ll_register_callbacks cb 0x7ff264009060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:21:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff272000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d1c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff272000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d1c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44432/0 (socket says 172.21.15.46:44432) 2024-03-29T08:21:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff272000640 1 -- 172.21.15.46:0/1121596291 learned_addr learned my addr 172.21.15.46:0/1121596291 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff27aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d18a00 0x5633b2dc45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff27b400640 1 --2- 172.21.15.46:0/1121596291 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d65390 0x5633b2d1c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff272000640 1 -- 172.21.15.46:0/1121596291 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d65390 msgr2=0x5633b2d1c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff272000640 1 --2- 172.21.15.46:0/1121596291 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d65390 0x5633b2d1c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff272000640 1 -- 172.21.15.46:0/1121596291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d18a00 msgr2=0x5633b2dc45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff272000640 1 --2- 172.21.15.46:0/1121596291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d18a00 0x5633b2dc45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff272000640 1 -- 172.21.15.46:0/1121596291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff264004160 con 0x5633b2d64a70 2024-03-29T08:21:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff27aa00640 1 --2- 172.21.15.46:0/1121596291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d18a00 0x5633b2dc45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:21:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff272000640 1 --2- 172.21.15.46:0/1121596291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d1c000 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff26400a010 tx=0x7ff264004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc28fe8dde38f3c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff25fe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:21:24.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff25fe00640 1 -- 172.21.15.46:0/1121596291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff264003830 con 0x5633b2d64a70 2024-03-29T08:21:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff25fe00640 1 -- 172.21.15.46:0/1121596291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff264038af0 con 0x5633b2d64a70 2024-03-29T08:21:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 1 -- 172.21.15.46:0/1121596291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633b2b26970 con 0x5633b2d64a70 2024-03-29T08:21:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff25fe00640 1 -- 172.21.15.46:0/1121596291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff264038cf0 con 0x5633b2d64a70 2024-03-29T08:21:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 10 client.6309 fetch_fsmap learned FSMap version 32 2024-03-29T08:21:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 1 -- 172.21.15.46:0/1121596291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5633b2b08960 con 0x5633b2d64a70 2024-03-29T08:21:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff25fe00640 1 -- 172.21.15.46:0/1121596291 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26403e070 con 0x5633b2d64a70 2024-03-29T08:21:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff25fe00640 1 -- 172.21.15.46:0/1121596291 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 32) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff2640059a0 con 0x5633b2d64a70 2024-03-29T08:21:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 10 client.6309 fetch_fsmap finished waiting for FSMap version 32 2024-03-29T08:21:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 10 client.6309 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-03-29T08:21:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 20 client.6309 populate_metadata read hostname 'smithi046' 2024-03-29T08:21:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 1 --2- 172.21.15.46:0/1121596291 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5633b2d801e0 0x5633b2d805f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 4 client.6309 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:21:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff284e06000 1 -- 172.21.15.46:0/1121596291 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5633b2b98a20 con 0x5633b2d801e0 2024-03-29T08:21:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.147+0000 7ff27aa00640 1 --2- 172.21.15.46:0/1121596291 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5633b2d801e0 0x5633b2d805f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:24.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff27aa00640 1 --2- 172.21.15.46:0/1121596291 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5633b2d801e0 0x5633b2d805f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8778a145e64366 server_cookie=e4d413822b4ce66f in_seq=0 out_seq=0 2024-03-29T08:21:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff25fe00640 10 client.6309 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff25fe00640 1 -- 172.21.15.46:0/1121596291 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x5633b2b98a20 con 0x5633b2d801e0 2024-03-29T08:21:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff25fe00640 10 client.6309 handle_command_reply: tid=0 2024-03-29T08:21:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff284e06000 1 client.6309 shutdown 2024-03-29T08:21:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff284e06000 1 -- 172.21.15.46:0/1121596291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 msgr2=0x5633b2d1c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff284e06000 1 --2- 172.21.15.46:0/1121596291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d1c000 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff26400a010 tx=0x7ff264004740 comp rx=0 tx=0).stop 2024-03-29T08:21:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff284e06000 1 -- 172.21.15.46:0/1121596291 shutdown_connections 2024-03-29T08:21:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff284e06000 1 --2- 172.21.15.46:0/1121596291 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5633b2d65390 0x5633b2d1c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff284e06000 1 --2- 172.21.15.46:0/1121596291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5633b2d64a70 0x5633b2d1c000 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff284e06000 1 --2- 172.21.15.46:0/1121596291 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5633b2d801e0 0x5633b2d805f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff284e06000 1 --2- 172.21.15.46:0/1121596291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5633b2d18a00 0x5633b2dc45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff284e06000 1 -- 172.21.15.46:0/1121596291 >> 172.21.15.46:0/1121596291 conn(0x5633b2b19160 msgr2=0x5633b2d16c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff25fe00640 0 client.6309 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:24.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff284e06000 1 -- 172.21.15.46:0/1121596291 shutdown_connections 2024-03-29T08:21:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff284e06000 1 -- 172.21.15.46:0/1121596291 wait complete. 2024-03-29T08:21:24.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff284e06000 20 client.6309 trim_cache size 0 max 16384 2024-03-29T08:21:24.156 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:21:24.156 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-29T08:21:24.156 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-29T08:21:24.156 INFO:teuthology.orchestra.run.smithi046.stdout: "1124f5d4-176a-4398-b011-fe43515cb9d8": { 2024-03-29T08:21:24.156 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-29T08:21:24.156 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "1124f5d4-176a-4398-b011-fe43515cb9d8", 2024-03-29T08:21:24.156 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive" 2024-03-29T08:21:24.156 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:24.156 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:24.157 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:21:24.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff281a00640 1 -- 172.21.15.46:0/1868962586 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff258066690 msgr2=0x7ff258068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:24.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff281a00640 1 --2- 172.21.15.46:0/1868962586 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff258066690 0x7ff258068b50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff27c06a810 tx=0x7ff264009190 comp rx=0 tx=0).stop 2024-03-29T08:21:24.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff281a00640 1 -- 172.21.15.46:0/1868962586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27c0b1d30 msgr2=0x7ff27c155f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:24.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff281a00640 1 --2- 172.21.15.46:0/1868962586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27c0b1d30 0x7ff27c155f70 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff26c007ef0 tx=0x7ff26c00d4a0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff281a00640 1 -- 172.21.15.46:0/1868962586 shutdown_connections 2024-03-29T08:21:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff281a00640 1 --2- 172.21.15.46:0/1868962586 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff258066690 0x7ff258068b50 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff281a00640 1 --2- 172.21.15.46:0/1868962586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff27c15dbb0 0x7ff27c14e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff281a00640 1 --2- 172.21.15.46:0/1868962586 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff27c0b3f20 0x7ff27c15edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff281a00640 1 --2- 172.21.15.46:0/1868962586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff27c0b1d30 0x7ff27c155f70 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff281a00640 1 -- 172.21.15.46:0/1868962586 >> 172.21.15.46:0/1868962586 conn(0x7ff27c070ad0 msgr2=0x7ff27c15ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff281a00640 1 -- 172.21.15.46:0/1868962586 shutdown_connections 2024-03-29T08:21:24.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:24.151+0000 7ff281a00640 1 -- 172.21.15.46:0/1868962586 wait complete. 2024-03-29T08:21:24.171 INFO:tasks.cephfs.filesystem:scrub status for tag:1124f5d4-176a-4398-b011-fe43515cb9d8 - {'path': '/', 'tag': '1124f5d4-176a-4398-b011-fe43515cb9d8', 'options': 'recursive'} 2024-03-29T08:21:26.944 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-03-29T08:21:26.943+0000 7fc16e000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-29T08:21:38.641 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:21:38.645 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:21:54.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:54.172 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-29T08:21:54.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.322+0000 7fc7a1c00640 1 Processor -- start 2024-03-29T08:21:54.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.322+0000 7fc7a1c00640 1 -- start start 2024-03-29T08:21:54.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc7a1c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc79c142da0 0x7fc79c0adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc7a1c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c0afe80 con 0x7fc79c143180 2024-03-29T08:21:54.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc7a1c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc79c06cd80 con 0x7fc79c0ae1e0 2024-03-29T08:21:54.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc7a1c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c06d020 con 0x7fc79c142da0 2024-03-29T08:21:54.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc793e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc79c143180 0x7fc79c0abac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36200/0 (socket says 172.21.15.46:36200) 2024-03-29T08:21:54.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc793e00640 1 -- 172.21.15.46:0/272746159 learned_addr learned my addr 172.21.15.46:0/272746159 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:54.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79aa00640 1 --2- 172.21.15.46:0/272746159 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc79c142da0 0x7fc79c0adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79a000640 1 -- 172.21.15.46:0/272746159 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669287356 0 0) 0x7fc79c0afe80 con 0x7fc79c143180 2024-03-29T08:21:54.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79a000640 1 -- 172.21.15.46:0/272746159 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc780003530 con 0x7fc79c143180 2024-03-29T08:21:54.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79a000640 1 -- 172.21.15.46:0/272746159 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1299767649 0 0) 0x7fc780003530 con 0x7fc79c143180 2024-03-29T08:21:54.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79a000640 1 -- 172.21.15.46:0/272746159 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc79c142da0 msgr2=0x7fc79c0adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79a000640 1 --2- 172.21.15.46:0/272746159 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc79c142da0 0x7fc79c0adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79a000640 1 -- 172.21.15.46:0/272746159 >> v1:172.21.15.73:6789/0 conn(0x7fc79c0ae1e0 legacy=0x7fc79c0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79a000640 1 -- 172.21.15.46:0/272746159 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c0b00f0 con 0x7fc79c143180 2024-03-29T08:21:54.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79a000640 1 -- 172.21.15.46:0/272746159 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fc784001680 con 0x7fc79c143180 2024-03-29T08:21:54.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79a000640 1 -- 172.21.15.46:0/272746159 >> v1:172.21.15.46:6789/0 conn(0x7fc79c143180 legacy=0x7fc79c0abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79a000640 1 --2- 172.21.15.46:0/272746159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7800039b0 0x7fc780003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79a000640 1 --2- 172.21.15.46:0/272746159 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc79c142da0 0x7fc7800047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79a000640 1 --2- 172.21.15.46:0/272746159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc780004d10 0x7fc780005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79a000640 1 -- 172.21.15.46:0/272746159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c06d020 con 0x7fc7800039b0 2024-03-29T08:21:54.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79a000640 1 -- 172.21.15.46:0/272746159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78000b990 con 0x7fc780004d10 2024-03-29T08:21:54.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79a000640 1 -- 172.21.15.46:0/272746159 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78000bb50 con 0x7fc79c142da0 2024-03-29T08:21:54.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79aa00640 1 --2- 172.21.15.46:0/272746159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7800039b0 0x7fc780003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc793e00640 1 --2- 172.21.15.46:0/272746159 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc79c142da0 0x7fc7800047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79b400640 1 --2- 172.21.15.46:0/272746159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc780004d10 0x7fc780005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79aa00640 1 -- 172.21.15.46:0/272746159 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc79c142da0 msgr2=0x7fc7800047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79aa00640 1 --2- 172.21.15.46:0/272746159 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc79c142da0 0x7fc7800047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79aa00640 1 -- 172.21.15.46:0/272746159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc780004d10 msgr2=0x7fc780005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79aa00640 1 --2- 172.21.15.46:0/272746159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc780004d10 0x7fc780005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.326+0000 7fc79aa00640 1 -- 172.21.15.46:0/272746159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc780003530 con 0x7fc7800039b0 2024-03-29T08:21:54.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.330+0000 7fc79aa00640 1 --2- 172.21.15.46:0/272746159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7800039b0 0x7fc780003d90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009ae0 tx=0x7fc78c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5497516de2912bc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:54.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.330+0000 7fc79a000640 1 -- 172.21.15.46:0/272746159 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0a55a0 con 0x7fc7800039b0 2024-03-29T08:21:54.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.330+0000 7fc79a000640 1 -- 172.21.15.46:0/272746159 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c0a5700 con 0x7fc7800039b0 2024-03-29T08:21:54.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.330+0000 7fc79a000640 1 -- 172.21.15.46:0/272746159 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0a5ab0 con 0x7fc7800039b0 2024-03-29T08:21:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.330+0000 7fc7a1c00640 1 -- 172.21.15.46:0/272746159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7800039b0 msgr2=0x7fc780003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.330+0000 7fc7a1c00640 1 --2- 172.21.15.46:0/272746159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7800039b0 0x7fc780003d90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009ae0 tx=0x7fc78c01c9b0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.330+0000 7fc79b400640 1 -- 172.21.15.46:0/272746159 reap_dead start 2024-03-29T08:21:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.330+0000 7fc7a1c00640 1 -- 172.21.15.46:0/272746159 shutdown_connections 2024-03-29T08:21:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.330+0000 7fc7a1c00640 1 -- 172.21.15.46:0/272746159 >> 172.21.15.46:0/272746159 conn(0x7fc79c06a390 msgr2=0x7fc79c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.330+0000 7fc7a1c00640 1 -- 172.21.15.46:0/272746159 shutdown_connections 2024-03-29T08:21:54.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.330+0000 7fc7a1c00640 1 -- 172.21.15.46:0/272746159 wait complete. 2024-03-29T08:21:54.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc7a1c00640 1 Processor -- start 2024-03-29T08:21:54.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc7a1c00640 1 -- start start 2024-03-29T08:21:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc7a1c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc7800039b0 0x7fc79c15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc7a1c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc79c139370 0x7fc79c1397f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc7a1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc79c145380 0x7fc79c145800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc7a1c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c1f2830 con 0x7fc79c145380 2024-03-29T08:21:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc7a1c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c15c930 con 0x7fc79c139370 2024-03-29T08:21:54.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc7a1c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79c145d40 con 0x7fc7800039b0 2024-03-29T08:21:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc79b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc79c145380 0x7fc79c145800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc79aa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc7800039b0 0x7fc79c15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc79b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc79c145380 0x7fc79c145800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54888/0 (socket says 172.21.15.46:54888) 2024-03-29T08:21:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc79b400640 1 -- 172.21.15.46:0/3945876909 learned_addr learned my addr 172.21.15.46:0/3945876909 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc793e00640 1 --2- 172.21.15.46:0/3945876909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc79c139370 0x7fc79c1397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc79b400640 1 -- 172.21.15.46:0/3945876909 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc7800039b0 msgr2=0x7fc79c15c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc79b400640 1 --2- 172.21.15.46:0/3945876909 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc7800039b0 0x7fc79c15c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc79b400640 1 -- 172.21.15.46:0/3945876909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc79c139370 msgr2=0x7fc79c1397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc79b400640 1 --2- 172.21.15.46:0/3945876909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc79c139370 0x7fc79c1397f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc79b400640 1 -- 172.21.15.46:0/3945876909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c138f30 con 0x7fc79c145380 2024-03-29T08:21:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc79b400640 1 --2- 172.21.15.46:0/3945876909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc79c145380 0x7fc79c145800 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc79c06e5a0 tx=0x7fc79400aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc792a00640 1 -- 172.21.15.46:0/3945876909 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794006d70 con 0x7fc79c145380 2024-03-29T08:21:54.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc792a00640 1 -- 172.21.15.46:0/3945876909 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc794003980 con 0x7fc79c145380 2024-03-29T08:21:54.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc7a1c00640 1 -- 172.21.15.46:0/3945876909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79c138320 con 0x7fc79c145380 2024-03-29T08:21:54.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc792a00640 1 -- 172.21.15.46:0/3945876909 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794003d30 con 0x7fc79c145380 2024-03-29T08:21:54.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.334+0000 7fc7a1c00640 1 -- 172.21.15.46:0/3945876909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79c1388e0 con 0x7fc79c145380 2024-03-29T08:21:54.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc792a00640 1 -- 172.21.15.46:0/3945876909 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc79400f820 con 0x7fc79c145380 2024-03-29T08:21:54.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc792a00640 1 --2- 172.21.15.46:0/3945876909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc778066690 0x7fc778068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc792a00640 1 -- 172.21.15.46:0/3945876909 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7940869e0 con 0x7fc79c145380 2024-03-29T08:21:54.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc79aa00640 1 --2- 172.21.15.46:0/3945876909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc778066690 0x7fc778068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc7a5062000 1 Processor -- start 2024-03-29T08:21:54.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc7a5062000 1 -- start start 2024-03-29T08:21:54.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc79aa00640 1 --2- 172.21.15.46:0/3945876909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc778066690 0x7fc778068b50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0af000 tx=0x7fc78c0046d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc7a5062000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dcdfa70 0x55ce9dcdfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc7a5062000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dce0390 0x55ce9dc934c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc7a5062000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dc93a00 0x55ce9dc93de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc7a5062000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce9db5e630 con 0x55ce9dcdfa70 2024-03-29T08:21:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc7a5062000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce9daebbd0 con 0x55ce9dce0390 2024-03-29T08:21:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc7a5062000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce9dae91a0 con 0x55ce9dc93a00 2024-03-29T08:21:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc79aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dcdfa70 0x55ce9dcdfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc79aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dcdfa70 0x55ce9dcdfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54896/0 (socket says 172.21.15.46:54896) 2024-03-29T08:21:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc79aa00640 1 -- 172.21.15.46:0/965617479 learned_addr learned my addr 172.21.15.46:0/965617479 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc793e00640 1 --2- 172.21.15.46:0/965617479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dce0390 0x55ce9dc934c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc79b400640 1 --2- 172.21.15.46:0/965617479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dc93a00 0x55ce9dc93de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc79aa00640 1 -- 172.21.15.46:0/965617479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dc93a00 msgr2=0x55ce9dc93de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc79aa00640 1 --2- 172.21.15.46:0/965617479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dc93a00 0x55ce9dc93de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc79aa00640 1 -- 172.21.15.46:0/965617479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dce0390 msgr2=0x55ce9dc934c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc79aa00640 1 --2- 172.21.15.46:0/965617479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dce0390 0x55ce9dc934c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc79aa00640 1 -- 172.21.15.46:0/965617479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce9d7e0e20 con 0x55ce9dcdfa70 2024-03-29T08:21:54.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.338+0000 7fc79b400640 1 --2- 172.21.15.46:0/965617479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dc93a00 0x55ce9dc93de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:54.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc79aa00640 1 --2- 172.21.15.46:0/965617479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dcdfa70 0x55ce9dcdfe50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc78c000c10 tx=0x7fc78c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634f90396bce8996 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:54.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc78b400640 1 -- 172.21.15.46:0/965617479 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0ad640 con 0x55ce9dcdfa70 2024-03-29T08:21:54.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc78b400640 1 -- 172.21.15.46:0/965617479 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c0ad7a0 con 0x55ce9dcdfa70 2024-03-29T08:21:54.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc78b400640 1 -- 172.21.15.46:0/965617479 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0adb10 con 0x55ce9dcdfa70 2024-03-29T08:21:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 1 -- 172.21.15.46:0/965617479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dcdfa70 msgr2=0x55ce9dcdfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 1 --2- 172.21.15.46:0/965617479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dcdfa70 0x55ce9dcdfe50 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc78c000c10 tx=0x7fc78c004740 comp rx=0 tx=0).stop 2024-03-29T08:21:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 1 -- 172.21.15.46:0/965617479 shutdown_connections 2024-03-29T08:21:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 1 --2- 172.21.15.46:0/965617479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dce0390 0x55ce9dc934c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 1 --2- 172.21.15.46:0/965617479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dc93a00 0x55ce9dc93de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 1 --2- 172.21.15.46:0/965617479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dcdfa70 0x55ce9dcdfe50 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 1 -- 172.21.15.46:0/965617479 >> 172.21.15.46:0/965617479 conn(0x55ce9da94160 msgr2=0x55ce9dcc2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 1 -- 172.21.15.46:0/965617479 shutdown_connections 2024-03-29T08:21:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 1 -- 172.21.15.46:0/965617479 wait complete. 2024-03-29T08:21:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 1 -- start start 2024-03-29T08:21:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 1 Processor -- start 2024-03-29T08:21:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dc93a00 0x55ce9dc9c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dcdfa70 0x55ce9dcc3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dce0390 0x55ce9dcc3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce9dae4a90 con 0x55ce9dcdfa70 2024-03-29T08:21:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce9da83d40 con 0x55ce9dce0390 2024-03-29T08:21:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce9daa1f30 con 0x55ce9dc93a00 2024-03-29T08:21:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc7a5062000 10 client.0 _ll_register_callbacks cb 0x55ce9db46810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:21:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc793e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dcdfa70 0x55ce9dcc3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc79aa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dc93a00 0x55ce9dc9c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc793e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dcdfa70 0x55ce9dcc3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54910/0 (socket says 172.21.15.46:54910) 2024-03-29T08:21:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc793e00640 1 -- 172.21.15.46:0/1103981712 learned_addr learned my addr 172.21.15.46:0/1103981712 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc79b400640 1 --2- 172.21.15.46:0/1103981712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dce0390 0x55ce9dcc3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc793e00640 1 -- 172.21.15.46:0/1103981712 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dc93a00 msgr2=0x55ce9dc9c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc793e00640 1 --2- 172.21.15.46:0/1103981712 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dc93a00 0x55ce9dc9c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc793e00640 1 -- 172.21.15.46:0/1103981712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dce0390 msgr2=0x55ce9dcc3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc793e00640 1 --2- 172.21.15.46:0/1103981712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dce0390 0x55ce9dcc3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.342+0000 7fc793e00640 1 -- 172.21.15.46:0/1103981712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc780003530 con 0x55ce9dcdfa70 2024-03-29T08:21:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc793e00640 1 --2- 172.21.15.46:0/1103981712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dcdfa70 0x55ce9dcc3a50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc784004dc0 tx=0x7fc784007410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd366ab98a6c473 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc77fe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:21:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc77fe00640 1 -- 172.21.15.46:0/1103981712 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc784007750 con 0x55ce9dcdfa70 2024-03-29T08:21:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc77fe00640 1 -- 172.21.15.46:0/1103981712 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc784015910 con 0x55ce9dcdfa70 2024-03-29T08:21:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 1 -- 172.21.15.46:0/1103981712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce9dae33e0 con 0x55ce9dcdfa70 2024-03-29T08:21:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc77fe00640 1 -- 172.21.15.46:0/1103981712 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc784015ad0 con 0x55ce9dcdfa70 2024-03-29T08:21:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 10 client.6321 fetch_fsmap learned FSMap version 33 2024-03-29T08:21:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 1 -- 172.21.15.46:0/1103981712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55ce9daa9aa0 con 0x55ce9dcdfa70 2024-03-29T08:21:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc77fe00640 1 -- 172.21.15.46:0/1103981712 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ce9dae33e0 con 0x55ce9dcdfa70 2024-03-29T08:21:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc77fe00640 1 -- 172.21.15.46:0/1103981712 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc784012050 con 0x55ce9dcdfa70 2024-03-29T08:21:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 10 client.6321 fetch_fsmap finished waiting for FSMap version 33 2024-03-29T08:21:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 10 client.6321 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-03-29T08:21:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 20 client.6321 populate_metadata read hostname 'smithi046' 2024-03-29T08:21:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 1 --2- 172.21.15.46:0/1103981712 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55ce9dce70c0 0x55ce9dce74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 4 client.6321 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:21:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 1 -- 172.21.15.46:0/1103981712 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce9dab4d00 con 0x55ce9dce70c0 2024-03-29T08:21:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc79aa00640 1 --2- 172.21.15.46:0/1103981712 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55ce9dce70c0 0x55ce9dce74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc77fe00640 10 client.6321 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:54.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc79aa00640 1 --2- 172.21.15.46:0/1103981712 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55ce9dce70c0 0x55ce9dce74a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d01f127c0f7431d0 server_cookie=a7a782b22e4f969d in_seq=0 out_seq=0 2024-03-29T08:21:54.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc77fe00640 1 -- 172.21.15.46:0/1103981712 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ce9dab4d00 con 0x55ce9dce70c0 2024-03-29T08:21:54.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc77fe00640 10 client.6321 handle_command_reply: tid=0 2024-03-29T08:21:54.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 1 client.6321 shutdown 2024-03-29T08:21:54.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 1 -- 172.21.15.46:0/1103981712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dcdfa70 msgr2=0x55ce9dcc3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 1 --2- 172.21.15.46:0/1103981712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dcdfa70 0x55ce9dcc3a50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc784004dc0 tx=0x7fc784007410 comp rx=0 tx=0).stop 2024-03-29T08:21:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 1 -- 172.21.15.46:0/1103981712 shutdown_connections 2024-03-29T08:21:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 1 --2- 172.21.15.46:0/1103981712 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55ce9dce70c0 0x55ce9dce74a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 1 --2- 172.21.15.46:0/1103981712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dce0390 0x55ce9dcc3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 1 --2- 172.21.15.46:0/1103981712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dcdfa70 0x55ce9dcc3a50 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 1 --2- 172.21.15.46:0/1103981712 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dc93a00 0x55ce9dc9c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 1 -- 172.21.15.46:0/1103981712 >> 172.21.15.46:0/1103981712 conn(0x55ce9da94160 msgr2=0x55ce9dc92bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc77fe00640 0 client.6321 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 1 -- 172.21.15.46:0/1103981712 shutdown_connections 2024-03-29T08:21:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 1 -- 172.21.15.46:0/1103981712 wait complete. 2024-03-29T08:21:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.346+0000 7fc7a5062000 20 client.6321 trim_cache size 0 max 16384 2024-03-29T08:21:54.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc7a5062000 1 Processor -- start 2024-03-29T08:21:54.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc7a5062000 1 -- start start 2024-03-29T08:21:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc7a5062000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dc93a00 0x55ce9dc9bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc7a5062000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dcdfa70 0x55ce9dc92c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc7a5062000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dce0390 0x55ce9dc93170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc7a5062000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce9d7a1aa0 con 0x55ce9dce0390 2024-03-29T08:21:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc7a5062000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce9dae0280 con 0x55ce9dc93a00 2024-03-29T08:21:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc7a5062000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce9dae75f0 con 0x55ce9dcdfa70 2024-03-29T08:21:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc79b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dce0390 0x55ce9dc93170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc79b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dce0390 0x55ce9dc93170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54918/0 (socket says 172.21.15.46:54918) 2024-03-29T08:21:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc79b400640 1 -- 172.21.15.46:0/3563091136 learned_addr learned my addr 172.21.15.46:0/3563091136 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc79b400640 1 -- 172.21.15.46:0/3563091136 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dcdfa70 msgr2=0x55ce9dc92c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc79b400640 1 --2- 172.21.15.46:0/3563091136 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dcdfa70 0x55ce9dc92c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc79aa00640 1 --2- 172.21.15.46:0/3563091136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dc93a00 0x55ce9dc9bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc79b400640 1 -- 172.21.15.46:0/3563091136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dc93a00 msgr2=0x55ce9dc9bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc79b400640 1 --2- 172.21.15.46:0/3563091136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dc93a00 0x55ce9dc9bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc79b400640 1 -- 172.21.15.46:0/3563091136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce9da8fdf0 con 0x55ce9dce0390 2024-03-29T08:21:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc79aa00640 1 --2- 172.21.15.46:0/3563091136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dc93a00 0x55ce9dc9bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:21:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.362+0000 7fc79b400640 1 --2- 172.21.15.46:0/3563091136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dce0390 0x55ce9dc93170 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc794007fd0 tx=0x7fc794057490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf384451116eca3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:54.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc78be00640 1 -- 172.21.15.46:0/3563091136 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79405fd60 con 0x55ce9dce0390 2024-03-29T08:21:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc78be00640 1 -- 172.21.15.46:0/3563091136 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79406f070 con 0x55ce9dce0390 2024-03-29T08:21:54.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc78be00640 1 -- 172.21.15.46:0/3563091136 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc794054130 con 0x55ce9dce0390 2024-03-29T08:21:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 -- 172.21.15.46:0/3563091136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dce0390 msgr2=0x55ce9dc93170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 --2- 172.21.15.46:0/3563091136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dce0390 0x55ce9dc93170 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc794007fd0 tx=0x7fc794057490 comp rx=0 tx=0).stop 2024-03-29T08:21:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 -- 172.21.15.46:0/3563091136 shutdown_connections 2024-03-29T08:21:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 --2- 172.21.15.46:0/3563091136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dce0390 0x55ce9dc93170 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 --2- 172.21.15.46:0/3563091136 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dcdfa70 0x55ce9dc92c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 --2- 172.21.15.46:0/3563091136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dc93a00 0x55ce9dc9bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 -- 172.21.15.46:0/3563091136 >> 172.21.15.46:0/3563091136 conn(0x55ce9da94160 msgr2=0x55ce9dc9afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 -- 172.21.15.46:0/3563091136 shutdown_connections 2024-03-29T08:21:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 -- 172.21.15.46:0/3563091136 wait complete. 2024-03-29T08:21:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 -- start start 2024-03-29T08:21:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 Processor -- start 2024-03-29T08:21:54.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dc93a00 0x55ce9dd3c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dcdfa70 0x55ce9dd22680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dce0390 0x55ce9dd22bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce9dab40b0 con 0x55ce9dce0390 2024-03-29T08:21:54.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce9daae640 con 0x55ce9dc93a00 2024-03-29T08:21:54.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce9dc9c860 con 0x55ce9dcdfa70 2024-03-29T08:21:54.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 10 client.0 _ll_register_callbacks cb 0x7fc78c0bb050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:21:54.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc79b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dce0390 0x55ce9dd22bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc79b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dce0390 0x55ce9dd22bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54934/0 (socket says 172.21.15.46:54934) 2024-03-29T08:21:54.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc79b400640 1 -- 172.21.15.46:0/2730512002 learned_addr learned my addr 172.21.15.46:0/2730512002 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:54.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc79aa00640 1 --2- 172.21.15.46:0/2730512002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dc93a00 0x55ce9dd3c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc793e00640 1 --2- 172.21.15.46:0/2730512002 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dcdfa70 0x55ce9dd22680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc79b400640 1 -- 172.21.15.46:0/2730512002 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dcdfa70 msgr2=0x55ce9dd22680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc79b400640 1 --2- 172.21.15.46:0/2730512002 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dcdfa70 0x55ce9dd22680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc79b400640 1 -- 172.21.15.46:0/2730512002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dc93a00 msgr2=0x55ce9dd3c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc79b400640 1 --2- 172.21.15.46:0/2730512002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dc93a00 0x55ce9dd3c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc79b400640 1 -- 172.21.15.46:0/2730512002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79c1388e0 con 0x55ce9dce0390 2024-03-29T08:21:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc79b400640 1 --2- 172.21.15.46:0/2730512002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dce0390 0x55ce9dd22bc0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc794053f20 tx=0x7fc794006f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ea69e3fdf2f426 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc77fe00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:21:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc77fe00640 1 -- 172.21.15.46:0/2730512002 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79405f8d0 con 0x55ce9dce0390 2024-03-29T08:21:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc77fe00640 1 -- 172.21.15.46:0/2730512002 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc79405fa30 con 0x55ce9dce0390 2024-03-29T08:21:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 -- 172.21.15.46:0/2730512002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce9daa1970 con 0x55ce9dce0390 2024-03-29T08:21:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc77fe00640 1 -- 172.21.15.46:0/2730512002 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc794053810 con 0x55ce9dce0390 2024-03-29T08:21:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 10 client.6327 fetch_fsmap learned FSMap version 33 2024-03-29T08:21:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc7a5062000 1 -- 172.21.15.46:0/2730512002 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55ce9da83960 con 0x55ce9dce0390 2024-03-29T08:21:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.366+0000 7fc77fe00640 1 -- 172.21.15.46:0/2730512002 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ce9daa1970 con 0x55ce9dce0390 2024-03-29T08:21:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc77fe00640 1 -- 172.21.15.46:0/2730512002 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc794053970 con 0x55ce9dce0390 2024-03-29T08:21:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 10 client.6327 fetch_fsmap finished waiting for FSMap version 33 2024-03-29T08:21:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 10 client.6327 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-03-29T08:21:54.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 20 client.6327 populate_metadata read hostname 'smithi046' 2024-03-29T08:21:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 1 --2- 172.21.15.46:0/2730512002 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55ce9dd3d0b0 0x55ce9dd3d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 4 client.6327 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:21:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 1 -- 172.21.15.46:0/2730512002 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ce9db13a20 con 0x55ce9dd3d0b0 2024-03-29T08:21:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc79aa00640 1 --2- 172.21.15.46:0/2730512002 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55ce9dd3d0b0 0x55ce9dd3d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc79aa00640 1 --2- 172.21.15.46:0/2730512002 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55ce9dd3d0b0 0x55ce9dd3d4c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86a03ffeeb145259 server_cookie=17984a30bffcdf05 in_seq=0 out_seq=0 2024-03-29T08:21:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc77fe00640 10 client.6327 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc77fe00640 1 -- 172.21.15.46:0/2730512002 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ce9db13a20 con 0x55ce9dd3d0b0 2024-03-29T08:21:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc77fe00640 10 client.6327 handle_command_reply: tid=0 2024-03-29T08:21:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 1 client.6327 shutdown 2024-03-29T08:21:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 1 -- 172.21.15.46:0/2730512002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dce0390 msgr2=0x55ce9dd22bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 1 --2- 172.21.15.46:0/2730512002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dce0390 0x55ce9dd22bc0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fc794053f20 tx=0x7fc794006f50 comp rx=0 tx=0).stop 2024-03-29T08:21:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 1 -- 172.21.15.46:0/2730512002 shutdown_connections 2024-03-29T08:21:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 1 --2- 172.21.15.46:0/2730512002 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ce9dce0390 0x55ce9dd22bc0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 1 --2- 172.21.15.46:0/2730512002 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ce9dcdfa70 0x55ce9dd22680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 1 --2- 172.21.15.46:0/2730512002 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55ce9dd3d0b0 0x55ce9dd3d4c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 1 --2- 172.21.15.46:0/2730512002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ce9dc93a00 0x55ce9dd3c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 1 -- 172.21.15.46:0/2730512002 >> 172.21.15.46:0/2730512002 conn(0x55ce9da94160 msgr2=0x55ce9dc9b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:54.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc77fe00640 0 client.6327 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:54.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 1 -- 172.21.15.46:0/2730512002 shutdown_connections 2024-03-29T08:21:54.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 1 -- 172.21.15.46:0/2730512002 wait complete. 2024-03-29T08:21:54.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a5062000 20 client.6327 trim_cache size 0 max 16384 2024-03-29T08:21:54.377 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:21:54.377 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-29T08:21:54.377 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-29T08:21:54.378 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:21:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a1c00640 1 -- 172.21.15.46:0/3945876909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc778066690 msgr2=0x7fc778068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a1c00640 1 --2- 172.21.15.46:0/3945876909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc778066690 0x7fc778068b50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0af000 tx=0x7fc78c0046d0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a1c00640 1 -- 172.21.15.46:0/3945876909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc79c145380 msgr2=0x7fc79c145800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a1c00640 1 --2- 172.21.15.46:0/3945876909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc79c145380 0x7fc79c145800 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc79c06e5a0 tx=0x7fc79400aec0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a1c00640 1 -- 172.21.15.46:0/3945876909 shutdown_connections 2024-03-29T08:21:54.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a1c00640 1 --2- 172.21.15.46:0/3945876909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc778066690 0x7fc778068b50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a1c00640 1 --2- 172.21.15.46:0/3945876909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc79c139370 0x7fc79c1397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a1c00640 1 --2- 172.21.15.46:0/3945876909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc79c145380 0x7fc79c145800 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a1c00640 1 --2- 172.21.15.46:0/3945876909 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc7800039b0 0x7fc79c15c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a1c00640 1 -- 172.21.15.46:0/3945876909 >> 172.21.15.46:0/3945876909 conn(0x7fc79c06a390 msgr2=0x7fc79c1604b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:54.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a1c00640 1 -- 172.21.15.46:0/3945876909 shutdown_connections 2024-03-29T08:21:54.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.370+0000 7fc7a1c00640 1 -- 172.21.15.46:0/3945876909 wait complete. 2024-03-29T08:21:54.392 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-29T08:21:54.392 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:21:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.530+0000 7fb048c00640 1 Processor -- start 2024-03-29T08:21:54.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.530+0000 7fb048c00640 1 -- start start 2024-03-29T08:21:54.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.530+0000 7fb048c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb04415feb0 0x7fb044071370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.530+0000 7fb048c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb0441605f0 0x7fb044069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.530+0000 7fb048c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb044161020 con 0x7fb044160a80 2024-03-29T08:21:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.530+0000 7fb048c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb044161300 con 0x7fb0441605f0 2024-03-29T08:21:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.530+0000 7fb048c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb044070b70 con 0x7fb04415feb0 2024-03-29T08:21:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.530+0000 7fb042800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb044160a80 0x7fb04406d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36204/0 (socket says 172.21.15.46:36204) 2024-03-29T08:21:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.530+0000 7fb042800640 1 -- 172.21.15.46:0/3506448534 learned_addr learned my addr 172.21.15.46:0/3506448534 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:54.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.530+0000 7fb039400640 1 --2- 172.21.15.46:0/3506448534 >> v2:172.21.15.46:3300/0 conn(0x7fb0441605f0 0x7fb044069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.530+0000 7fb041e00640 1 --2- 172.21.15.46:0/3506448534 >> v2:172.21.15.73:3300/0 conn(0x7fb04415feb0 0x7fb044071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.530+0000 7fb039400640 1 -- 172.21.15.46:0/3506448534 >> v2:172.21.15.73:3300/0 conn(0x7fb04415feb0 msgr2=0x7fb044071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.530+0000 7fb039400640 1 --2- 172.21.15.46:0/3506448534 >> v2:172.21.15.73:3300/0 conn(0x7fb04415feb0 0x7fb044071370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.530+0000 7fb039400640 1 -- 172.21.15.46:0/3506448534 >> v1:172.21.15.46:6789/0 conn(0x7fb044160a80 legacy=0x7fb04406d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.534+0000 7fb039400640 1 -- 172.21.15.46:0/3506448534 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb044070e50 con 0x7fb0441605f0 2024-03-29T08:21:54.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.530+0000 7fb041400640 1 -- 172.21.15.46:0/3506448534 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322630633 0 0) 0x7fb044161020 con 0x7fb044160a80 2024-03-29T08:21:54.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.534+0000 7fb039400640 1 --2- 172.21.15.46:0/3506448534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0441605f0 0x7fb044069d90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0097c0 tx=0x7fb02c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10190f34fa06d67c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:54.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.534+0000 7fb041400640 1 -- 172.21.15.46:0/3506448534 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c0ab600 con 0x7fb0441605f0 2024-03-29T08:21:54.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.534+0000 7fb041400640 1 -- 172.21.15.46:0/3506448534 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c0a7050 con 0x7fb0441605f0 2024-03-29T08:21:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.534+0000 7fb041400640 1 -- 172.21.15.46:0/3506448534 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c0ab980 con 0x7fb0441605f0 2024-03-29T08:21:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.534+0000 7fb048c00640 1 -- 172.21.15.46:0/3506448534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0441605f0 msgr2=0x7fb044069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.534+0000 7fb048c00640 1 --2- 172.21.15.46:0/3506448534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0441605f0 0x7fb044069d90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0097c0 tx=0x7fb02c01c3d0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.534+0000 7fb048c00640 1 -- 172.21.15.46:0/3506448534 shutdown_connections 2024-03-29T08:21:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.534+0000 7fb048c00640 1 --2- 172.21.15.46:0/3506448534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0441605f0 0x7fb044069d90 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.534+0000 7fb048c00640 1 --2- 172.21.15.46:0/3506448534 >> v2:172.21.15.73:3300/0 conn(0x7fb04415feb0 0x7fb044071370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.534+0000 7fb048c00640 1 -- 172.21.15.46:0/3506448534 >> 172.21.15.46:0/3506448534 conn(0x7fb0440ced50 msgr2=0x7fb0440cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:54.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.534+0000 7fb048c00640 1 -- 172.21.15.46:0/3506448534 shutdown_connections 2024-03-29T08:21:54.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.534+0000 7fb048c00640 1 -- 172.21.15.46:0/3506448534 wait complete. 2024-03-29T08:21:54.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb048c00640 1 Processor -- start 2024-03-29T08:21:54.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb048c00640 1 -- start start 2024-03-29T08:21:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb048c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04415feb0 0x7fb044151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb048c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0441605f0 0x7fb044152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb048c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb044160a80 0x7fb04414d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb048c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0441512c0 con 0x7fb0441605f0 2024-03-29T08:21:54.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb048c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04414f6d0 con 0x7fb04415feb0 2024-03-29T08:21:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb048c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440ade50 con 0x7fb044160a80 2024-03-29T08:21:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb039400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0441605f0 0x7fb044152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb039400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0441605f0 0x7fb044152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54954/0 (socket says 172.21.15.46:54954) 2024-03-29T08:21:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb039400640 1 -- 172.21.15.46:0/8879133 learned_addr learned my addr 172.21.15.46:0/8879133 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb041e00640 1 --2- 172.21.15.46:0/8879133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04415feb0 0x7fb044151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb042800640 1 --2- 172.21.15.46:0/8879133 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb044160a80 0x7fb04414d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb039400640 1 -- 172.21.15.46:0/8879133 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb044160a80 msgr2=0x7fb04414d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb039400640 1 --2- 172.21.15.46:0/8879133 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb044160a80 0x7fb04414d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb039400640 1 -- 172.21.15.46:0/8879133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04415feb0 msgr2=0x7fb044151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb039400640 1 --2- 172.21.15.46:0/8879133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04415feb0 0x7fb044151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb039400640 1 -- 172.21.15.46:0/8879133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c009420 con 0x7fb0441605f0 2024-03-29T08:21:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb039400640 1 --2- 172.21.15.46:0/8879133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0441605f0 0x7fb044152f80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb02c01a000 tx=0x7fb02c019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:54.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb03aa00640 1 -- 172.21.15.46:0/8879133 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c006e20 con 0x7fb0441605f0 2024-03-29T08:21:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb03aa00640 1 -- 172.21.15.46:0/8879133 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c004020 con 0x7fb0441605f0 2024-03-29T08:21:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb03aa00640 1 -- 172.21.15.46:0/8879133 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c004390 con 0x7fb0441605f0 2024-03-29T08:21:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb048c00640 1 -- 172.21.15.46:0/8879133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0440b2ea0 con 0x7fb0441605f0 2024-03-29T08:21:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.538+0000 7fb048c00640 1 -- 172.21.15.46:0/8879133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0440b3460 con 0x7fb0441605f0 2024-03-29T08:21:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.542+0000 7fb048c00640 1 -- 172.21.15.46:0/8879133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb044066c90 con 0x7fb0441605f0 2024-03-29T08:21:54.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.542+0000 7fb03aa00640 1 -- 172.21.15.46:0/8879133 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb02c0b2040 con 0x7fb0441605f0 2024-03-29T08:21:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.542+0000 7fb03aa00640 1 --2- 172.21.15.46:0/8879133 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb020066690 0x7fb020068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.542+0000 7fb03aa00640 1 -- 172.21.15.46:0/8879133 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb02c11dfd0 con 0x7fb0441605f0 2024-03-29T08:21:54.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.542+0000 7fb041e00640 1 --2- 172.21.15.46:0/8879133 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb020066690 0x7fb020068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.542+0000 7fb041e00640 1 --2- 172.21.15.46:0/8879133 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb020066690 0x7fb020068b50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb034007a60 tx=0x7fb034008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:54.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.546+0000 7fb03aa00640 1 -- 172.21.15.46:0/8879133 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb02c0eb3e0 con 0x7fb0441605f0 2024-03-29T08:21:54.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.746+0000 7fb048c00640 1 -- 172.21.15.46:0/8879133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb0440cb540 con 0x7fb0441605f0 2024-03-29T08:21:54.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.746+0000 7fb03aa00640 1 -- 172.21.15.46:0/8879133 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4728 (secure 0 0 0) 0x7fb02c0b0020 con 0x7fb0441605f0 2024-03-29T08:21:54.748 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:54.749 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2187826146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2187826146},{"type":"v1","addr":"172.21.15.73:6836","nonce":2187826146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/73382477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":73382477},{"type":"v1","addr":"172.21.15.73:6837","nonce":73382477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2874958170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2874958170},{"type":"v1","addr":"172.21.15.46:6834","nonce":2874958170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:21:10.337267+0000","modified":"2024-03-29T08:21:26.945919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6051},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6051":{"gid":6051,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":7,"addr":"172.21.15.46:6835/3255474176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3255474176},{"type":"v1","addr":"172.21.15.46:6835","nonce":3255474176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-29T08:21:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 33 2024-03-29T08:21:54.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.746+0000 7fb048c00640 1 -- 172.21.15.46:0/8879133 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb020066690 msgr2=0x7fb020068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.746+0000 7fb048c00640 1 --2- 172.21.15.46:0/8879133 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb020066690 0x7fb020068b50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb034007a60 tx=0x7fb034008040 comp rx=0 tx=0).stop 2024-03-29T08:21:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.746+0000 7fb048c00640 1 -- 172.21.15.46:0/8879133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0441605f0 msgr2=0x7fb044152f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.746+0000 7fb048c00640 1 --2- 172.21.15.46:0/8879133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0441605f0 0x7fb044152f80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb02c01a000 tx=0x7fb02c019170 comp rx=0 tx=0).stop 2024-03-29T08:21:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.746+0000 7fb048c00640 1 -- 172.21.15.46:0/8879133 shutdown_connections 2024-03-29T08:21:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.746+0000 7fb048c00640 1 --2- 172.21.15.46:0/8879133 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb020066690 0x7fb020068b50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.746+0000 7fb048c00640 1 --2- 172.21.15.46:0/8879133 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb044160a80 0x7fb04414d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.746+0000 7fb048c00640 1 --2- 172.21.15.46:0/8879133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0441605f0 0x7fb044152f80 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.746+0000 7fb048c00640 1 --2- 172.21.15.46:0/8879133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04415feb0 0x7fb044151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.746+0000 7fb048c00640 1 -- 172.21.15.46:0/8879133 >> 172.21.15.46:0/8879133 conn(0x7fb0440ced50 msgr2=0x7fb044158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.746+0000 7fb048c00640 1 -- 172.21.15.46:0/8879133 shutdown_connections 2024-03-29T08:21:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.746+0000 7fb048c00640 1 -- 172.21.15.46:0/8879133 wait complete. 2024-03-29T08:21:54.770 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-03-29T08:21:54.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.910+0000 7f0ed9800640 1 Processor -- start 2024-03-29T08:21:54.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.910+0000 7f0ed9800640 1 -- start start 2024-03-29T08:21:54.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.910+0000 7f0ed9800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0ed406f1c0 0x7f0ed4069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.910+0000 7f0ed9800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed406a720 con 0x7f0ed406a2d0 2024-03-29T08:21:54.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.910+0000 7f0ed9800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed415d080 con 0x7f0ed415cad0 2024-03-29T08:21:54.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.910+0000 7f0ed9800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed415dd00 con 0x7f0ed406f1c0 2024-03-29T08:21:54.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.910+0000 7f0ed3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0ed406a2d0 0x7f0ed406d470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36208/0 (socket says 172.21.15.46:36208) 2024-03-29T08:21:54.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.910+0000 7f0ed3400640 1 -- 172.21.15.46:0/3140557557 learned_addr learned my addr 172.21.15.46:0/3140557557 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:54.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.910+0000 7f0ed2000640 1 --2- 172.21.15.46:0/3140557557 >> v2:172.21.15.46:3300/0 conn(0x7f0ed406f1c0 0x7f0ed4069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.910+0000 7f0ed1600640 1 -- 172.21.15.46:0/3140557557 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 572207313 0 0) 0x7f0ed406a720 con 0x7f0ed406a2d0 2024-03-29T08:21:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.910+0000 7f0ed1600640 1 -- 172.21.15.46:0/3140557557 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0eb8003530 con 0x7f0ed406a2d0 2024-03-29T08:21:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.910+0000 7f0ed2000640 1 -- 172.21.15.46:0/3140557557 >> v1:172.21.15.73:6789/0 conn(0x7f0ed415cad0 legacy=0x7f0ed406eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.910+0000 7f0ed2000640 1 -- 172.21.15.46:0/3140557557 >> v1:172.21.15.46:6789/0 conn(0x7f0ed406a2d0 legacy=0x7f0ed406d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.910+0000 7f0ed2000640 1 -- 172.21.15.46:0/3140557557 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed415d360 con 0x7f0ed406f1c0 2024-03-29T08:21:54.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.914+0000 7f0ed2000640 1 --2- 172.21.15.46:0/3140557557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed406f1c0 0x7f0ed4069d90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00097c0 tx=0x7f0ec001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1e97631c538410 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:54.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.914+0000 7f0ed1600640 1 -- 172.21.15.46:0/3140557557 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec00ab600 con 0x7f0ed406f1c0 2024-03-29T08:21:54.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.914+0000 7f0ed1600640 1 -- 172.21.15.46:0/3140557557 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec00a7050 con 0x7f0ed406f1c0 2024-03-29T08:21:54.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.914+0000 7f0ed1600640 1 -- 172.21.15.46:0/3140557557 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec00ab960 con 0x7f0ed406f1c0 2024-03-29T08:21:54.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.914+0000 7f0ed9800640 1 -- 172.21.15.46:0/3140557557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed406f1c0 msgr2=0x7f0ed4069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.914+0000 7f0ed9800640 1 --2- 172.21.15.46:0/3140557557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed406f1c0 0x7f0ed4069d90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0ec00097c0 tx=0x7f0ec001c410 comp rx=0 tx=0).stop 2024-03-29T08:21:54.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.914+0000 7f0ed9800640 1 -- 172.21.15.46:0/3140557557 shutdown_connections 2024-03-29T08:21:54.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.914+0000 7f0ed9800640 1 --2- 172.21.15.46:0/3140557557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed406f1c0 0x7f0ed4069d90 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.914+0000 7f0ed9800640 1 -- 172.21.15.46:0/3140557557 >> 172.21.15.46:0/3140557557 conn(0x7f0ed4098a40 msgr2=0x7f0ed40b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:54.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.914+0000 7f0ed9800640 1 -- 172.21.15.46:0/3140557557 shutdown_connections 2024-03-29T08:21:54.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.914+0000 7f0ed9800640 1 -- 172.21.15.46:0/3140557557 wait complete. 2024-03-29T08:21:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.914+0000 7f0ed9800640 1 Processor -- start 2024-03-29T08:21:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.914+0000 7f0ed9800640 1 -- start start 2024-03-29T08:21:54.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed9800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ed406a2d0 0x7f0ed4148330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed9800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ed406f1c0 0x7f0ed41547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed415cad0 0x7f0ed4154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed9800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40c5be0 con 0x7f0ed415cad0 2024-03-29T08:21:54.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed9800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed415bee0 con 0x7f0ed406a2d0 2024-03-29T08:21:54.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed9800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed40ac500 con 0x7f0ed406f1c0 2024-03-29T08:21:54.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed415cad0 0x7f0ed4154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed415cad0 0x7f0ed4154ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54974/0 (socket says 172.21.15.46:54974) 2024-03-29T08:21:54.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed3400640 1 -- 172.21.15.46:0/371467878 learned_addr learned my addr 172.21.15.46:0/371467878 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:54.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed2000640 1 --2- 172.21.15.46:0/371467878 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ed406f1c0 0x7f0ed41547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed2a00640 1 --2- 172.21.15.46:0/371467878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ed406a2d0 0x7f0ed4148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed3400640 1 -- 172.21.15.46:0/371467878 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ed406f1c0 msgr2=0x7f0ed41547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed3400640 1 --2- 172.21.15.46:0/371467878 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ed406f1c0 0x7f0ed41547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed3400640 1 -- 172.21.15.46:0/371467878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ed406a2d0 msgr2=0x7f0ed4148330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed3400640 1 --2- 172.21.15.46:0/371467878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ed406a2d0 0x7f0ed4148330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed3400640 1 -- 172.21.15.46:0/371467878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec0009420 con 0x7f0ed415cad0 2024-03-29T08:21:54.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed3400640 1 --2- 172.21.15.46:0/371467878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed415cad0 0x7f0ed4154ce0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00cb40 tx=0x7f0ecc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:54.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ecaa00640 1 -- 172.21.15.46:0/371467878 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc00aca0 con 0x7f0ed415cad0 2024-03-29T08:21:54.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ecaa00640 1 -- 172.21.15.46:0/371467878 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ecc00ae00 con 0x7f0ed415cad0 2024-03-29T08:21:54.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ecaa00640 1 -- 172.21.15.46:0/371467878 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc012620 con 0x7f0ed415cad0 2024-03-29T08:21:54.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed9800640 1 -- 172.21.15.46:0/371467878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed40ac7e0 con 0x7f0ed415cad0 2024-03-29T08:21:54.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.918+0000 7f0ed9800640 1 -- 172.21.15.46:0/371467878 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed4148870 con 0x7f0ed415cad0 2024-03-29T08:21:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0ecaa00640 1 -- 172.21.15.46:0/371467878 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0ecc007070 con 0x7f0ed415cad0 2024-03-29T08:21:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0ecaa00640 1 --2- 172.21.15.46:0/371467878 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0eb40666c0 0x7f0eb4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0ecaa00640 1 -- 172.21.15.46:0/371467878 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ecc084190 con 0x7f0ed415cad0 2024-03-29T08:21:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0ed2a00640 1 --2- 172.21.15.46:0/371467878 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0eb40666c0 0x7f0eb4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0edccff000 1 Processor -- start 2024-03-29T08:21:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0edccff000 1 -- start start 2024-03-29T08:21:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0ed2a00640 1 --2- 172.21.15.46:0/371467878 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0eb40666c0 0x7f0eb4068b80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001770 tx=0x7f0ec4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0edccff000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252cfe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0edccff000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252cfebb0 0x55d252cb4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0edccff000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cb51b0 0x55d252cb5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0edccff000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d252b7d340 con 0x55d252cfe290 2024-03-29T08:21:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0edccff000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d252b09cd0 con 0x55d252cfebb0 2024-03-29T08:21:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0edccff000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d252b072a0 con 0x55d252cb51b0 2024-03-29T08:21:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0ed2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252cfe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0ed2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252cfe670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54988/0 (socket says 172.21.15.46:54988) 2024-03-29T08:21:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0ed2a00640 1 -- 172.21.15.46:0/1892741502 learned_addr learned my addr 172.21.15.46:0/1892741502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0ed2000640 1 --2- 172.21.15.46:0/1892741502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252cfebb0 0x55d252cb4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0ed3400640 1 --2- 172.21.15.46:0/1892741502 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cb51b0 0x55d252cb5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0ed2a00640 1 -- 172.21.15.46:0/1892741502 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cb51b0 msgr2=0x55d252cb5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0ed2a00640 1 --2- 172.21.15.46:0/1892741502 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cb51b0 0x55d252cb5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0ed2a00640 1 -- 172.21.15.46:0/1892741502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252cfebb0 msgr2=0x55d252cb4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0ed2a00640 1 --2- 172.21.15.46:0/1892741502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252cfebb0 0x55d252cb4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.922+0000 7f0ed2a00640 1 -- 172.21.15.46:0/1892741502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2527f54b0 con 0x55d252cfe290 2024-03-29T08:21:54.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0ed2a00640 1 --2- 172.21.15.46:0/1892741502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252cfe670 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400f850 tx=0x7f0ec4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c850c32bbb3b0b09 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0ebf400640 1 -- 172.21.15.46:0/1892741502 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40159e0 con 0x55d252cfe290 2024-03-29T08:21:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0ebf400640 1 -- 172.21.15.46:0/1892741502 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec4015b40 con 0x55d252cfe290 2024-03-29T08:21:54.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0ebf400640 1 -- 172.21.15.46:0/1892741502 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4020400 con 0x55d252cfe290 2024-03-29T08:21:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 1 -- 172.21.15.46:0/1892741502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 msgr2=0x55d252cfe670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 1 --2- 172.21.15.46:0/1892741502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252cfe670 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0ec400f850 tx=0x7f0ec4000ee0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 1 -- 172.21.15.46:0/1892741502 shutdown_connections 2024-03-29T08:21:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 1 --2- 172.21.15.46:0/1892741502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252cfebb0 0x55d252cb4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 1 --2- 172.21.15.46:0/1892741502 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cb51b0 0x55d252cb5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 1 --2- 172.21.15.46:0/1892741502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252cfe670 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 1 -- 172.21.15.46:0/1892741502 >> 172.21.15.46:0/1892741502 conn(0x55d252cb1160 msgr2=0x55d252cb3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 1 -- 172.21.15.46:0/1892741502 shutdown_connections 2024-03-29T08:21:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 1 -- 172.21.15.46:0/1892741502 wait complete. 2024-03-29T08:21:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 1 -- start start 2024-03-29T08:21:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 1 Processor -- start 2024-03-29T08:21:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cb51b0 0x55d252cba700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252cbac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252cfebb0 0x55d252cbfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d252b02b90 con 0x55d252cfe290 2024-03-29T08:21:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d252ac00a0 con 0x55d252cfebb0 2024-03-29T08:21:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d252abfd40 con 0x55d252cb51b0 2024-03-29T08:21:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0edccff000 10 client.0 _ll_register_callbacks cb 0x55d252bb7c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:21:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0ed2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252cbac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0ed2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252cbac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55002/0 (socket says 172.21.15.46:55002) 2024-03-29T08:21:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0ed2000640 1 -- 172.21.15.46:0/394718578 learned_addr learned my addr 172.21.15.46:0/394718578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0ed2a00640 1 --2- 172.21.15.46:0/394718578 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cb51b0 0x55d252cba700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0ed3400640 1 --2- 172.21.15.46:0/394718578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252cfebb0 0x55d252cbfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0ed2000640 1 -- 172.21.15.46:0/394718578 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cb51b0 msgr2=0x55d252cba700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0ed2000640 1 --2- 172.21.15.46:0/394718578 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cb51b0 0x55d252cba700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0ed2000640 1 -- 172.21.15.46:0/394718578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252cfebb0 msgr2=0x55d252cbfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0ed2000640 1 --2- 172.21.15.46:0/394718578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252cfebb0 0x55d252cbfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.926+0000 7f0ed2000640 1 -- 172.21.15.46:0/394718578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed415d080 con 0x55d252cfe290 2024-03-29T08:21:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0ed2000640 1 --2- 172.21.15.46:0/394718578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252cbac40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0ec001c8f0 tx=0x7f0ec001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b18411c1e5579b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0eb3e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:21:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0eb3e00640 1 -- 172.21.15.46:0/394718578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec0003820 con 0x55d252cfe290 2024-03-29T08:21:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0eb3e00640 1 -- 172.21.15.46:0/394718578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ec0003980 con 0x55d252cfe290 2024-03-29T08:21:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 1 -- 172.21.15.46:0/394718578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d252ad2550 con 0x55d252cfe290 2024-03-29T08:21:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0eb3e00640 1 -- 172.21.15.46:0/394718578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ec0003b40 con 0x55d252cfe290 2024-03-29T08:21:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 10 client.6351 fetch_fsmap learned FSMap version 33 2024-03-29T08:21:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 1 -- 172.21.15.46:0/394718578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55d252ac7ba0 con 0x55d252cfe290 2024-03-29T08:21:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0eb3e00640 1 -- 172.21.15.46:0/394718578 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d252ad2550 con 0x55d252cfe290 2024-03-29T08:21:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0eb3e00640 1 -- 172.21.15.46:0/394718578 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0ec0003ca0 con 0x55d252cfe290 2024-03-29T08:21:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 10 client.6351 fetch_fsmap finished waiting for FSMap version 33 2024-03-29T08:21:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 10 client.6351 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:21:54.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 20 client.6351 populate_metadata read hostname 'smithi046' 2024-03-29T08:21:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 1 --2- 172.21.15.46:0/394718578 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55d252d0d320 0x55d252d0f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 4 client.6351 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:21:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 1 -- 172.21.15.46:0/394718578 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d252aff970 con 0x55d252d0d320 2024-03-29T08:21:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0ed2a00640 1 --2- 172.21.15.46:0/394718578 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55d252d0d320 0x55d252d0f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0ed2a00640 1 --2- 172.21.15.46:0/394718578 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55d252d0d320 0x55d252d0f710 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=924f2fbdd054688b server_cookie=7d26b92dfdf5d9ea in_seq=0 out_seq=0 2024-03-29T08:21:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0eb3e00640 10 client.6351 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0eb3e00640 1 -- 172.21.15.46:0/394718578 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d252aff970 con 0x55d252d0d320 2024-03-29T08:21:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0eb3e00640 10 client.6351 handle_command_reply: tid=0 2024-03-29T08:21:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 1 client.6351 shutdown 2024-03-29T08:21:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 1 -- 172.21.15.46:0/394718578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 msgr2=0x55d252cbac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 1 --2- 172.21.15.46:0/394718578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252cbac40 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0ec001c8f0 tx=0x7f0ec001ac50 comp rx=0 tx=0).stop 2024-03-29T08:21:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 1 -- 172.21.15.46:0/394718578 shutdown_connections 2024-03-29T08:21:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 1 --2- 172.21.15.46:0/394718578 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55d252d0d320 0x55d252d0f710 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 1 --2- 172.21.15.46:0/394718578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252cfebb0 0x55d252cbfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 1 --2- 172.21.15.46:0/394718578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252cbac40 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 1 --2- 172.21.15.46:0/394718578 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cb51b0 0x55d252cba700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0eb3e00640 0 client.6351 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 1 -- 172.21.15.46:0/394718578 >> 172.21.15.46:0/394718578 conn(0x55d252cb1160 msgr2=0x55d252cb3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 1 -- 172.21.15.46:0/394718578 shutdown_connections 2024-03-29T08:21:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 1 -- 172.21.15.46:0/394718578 wait complete. 2024-03-29T08:21:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.930+0000 7f0edccff000 20 client.6351 trim_cache size 0 max 16384 2024-03-29T08:21:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.942+0000 7f0edccff000 1 Processor -- start 2024-03-29T08:21:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.942+0000 7f0edccff000 1 -- start start 2024-03-29T08:21:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0edccff000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cb51b0 0x55d252cbac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0edccff000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252cfe290 0x55d252cb96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0edccff000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cfebb0 0x55d252cb9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0edccff000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2527bfc40 con 0x55d252cb51b0 2024-03-29T08:21:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0edccff000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d252acdad0 con 0x55d252cfe290 2024-03-29T08:21:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0edccff000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d252b06b90 con 0x55d252cfebb0 2024-03-29T08:21:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0ed2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cb51b0 0x55d252cbac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0ed2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cb51b0 0x55d252cbac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55016/0 (socket says 172.21.15.46:55016) 2024-03-29T08:21:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0ed2a00640 1 -- 172.21.15.46:0/3329566092 learned_addr learned my addr 172.21.15.46:0/3329566092 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0ed3400640 1 --2- 172.21.15.46:0/3329566092 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cfebb0 0x55d252cb9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0ed2000640 1 --2- 172.21.15.46:0/3329566092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252cfe290 0x55d252cb96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0ed2a00640 1 -- 172.21.15.46:0/3329566092 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cfebb0 msgr2=0x55d252cb9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0ed2a00640 1 --2- 172.21.15.46:0/3329566092 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cfebb0 0x55d252cb9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0ed2a00640 1 -- 172.21.15.46:0/3329566092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252cfe290 msgr2=0x55d252cb96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0ed2a00640 1 --2- 172.21.15.46:0/3329566092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252cfe290 0x55d252cb96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0ed2a00640 1 -- 172.21.15.46:0/3329566092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d252aadef0 con 0x55d252cb51b0 2024-03-29T08:21:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0ed2a00640 1 --2- 172.21.15.46:0/3329566092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cb51b0 0x55d252cbac30 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001580 tx=0x7f0ec4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa9b0a65c74a289 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0ebfe00640 1 -- 172.21.15.46:0/3329566092 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec40069e0 con 0x55d252cb51b0 2024-03-29T08:21:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0ebfe00640 1 -- 172.21.15.46:0/3329566092 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec4001d50 con 0x55d252cb51b0 2024-03-29T08:21:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0ebfe00640 1 -- 172.21.15.46:0/3329566092 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4006d80 con 0x55d252cb51b0 2024-03-29T08:21:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0edccff000 1 -- 172.21.15.46:0/3329566092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cb51b0 msgr2=0x55d252cbac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0edccff000 1 --2- 172.21.15.46:0/3329566092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cb51b0 0x55d252cbac30 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001580 tx=0x7f0ec4014000 comp rx=0 tx=0).stop 2024-03-29T08:21:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0edccff000 1 -- 172.21.15.46:0/3329566092 shutdown_connections 2024-03-29T08:21:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0edccff000 1 --2- 172.21.15.46:0/3329566092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252cfe290 0x55d252cb96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0edccff000 1 --2- 172.21.15.46:0/3329566092 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cfebb0 0x55d252cb9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0edccff000 1 --2- 172.21.15.46:0/3329566092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cb51b0 0x55d252cbac30 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0edccff000 1 -- 172.21.15.46:0/3329566092 >> 172.21.15.46:0/3329566092 conn(0x55d252cb1160 msgr2=0x55d252cb48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0edccff000 1 -- 172.21.15.46:0/3329566092 shutdown_connections 2024-03-29T08:21:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0edccff000 1 -- 172.21.15.46:0/3329566092 wait complete. 2024-03-29T08:21:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0edccff000 1 -- start start 2024-03-29T08:21:54.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.946+0000 7f0edccff000 1 Processor -- start 2024-03-29T08:21:54.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252d0fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cfebb0 0x55d252d103b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252d108f0 0x55d252d582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d252b01a20 con 0x55d252cfe290 2024-03-29T08:21:54.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d252abc300 con 0x55d252d108f0 2024-03-29T08:21:54.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d252cac5e0 con 0x55d252cfebb0 2024-03-29T08:21:54.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 10 client.0 _ll_register_callbacks cb 0x55d252ba0870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:21:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0ed2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252d0fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0ed2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252d0fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55018/0 (socket says 172.21.15.46:55018) 2024-03-29T08:21:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0ed2a00640 1 -- 172.21.15.46:0/38162410 learned_addr learned my addr 172.21.15.46:0/38162410 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0ed2000640 1 --2- 172.21.15.46:0/38162410 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cfebb0 0x55d252d103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0ed3400640 1 --2- 172.21.15.46:0/38162410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252d108f0 0x55d252d582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0ed2a00640 1 -- 172.21.15.46:0/38162410 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cfebb0 msgr2=0x55d252d103b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0ed2a00640 1 --2- 172.21.15.46:0/38162410 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cfebb0 0x55d252d103b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0ed2a00640 1 -- 172.21.15.46:0/38162410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252d108f0 msgr2=0x55d252d582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0ed2a00640 1 --2- 172.21.15.46:0/38162410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252d108f0 0x55d252d582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0ed2a00640 1 -- 172.21.15.46:0/38162410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2527f54b0 con 0x55d252cfe290 2024-03-29T08:21:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0ed3400640 1 --2- 172.21.15.46:0/38162410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252d108f0 0x55d252d582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0ed2000640 1 --2- 172.21.15.46:0/38162410 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cfebb0 0x55d252d103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:54.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0ed2a00640 1 --2- 172.21.15.46:0/38162410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252d0fe70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001ec0 tx=0x7f0ec4000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187398f638b0b57d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0eb3e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:21:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0eb3e00640 1 -- 172.21.15.46:0/38162410 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4006d70 con 0x55d252cfe290 2024-03-29T08:21:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0eb3e00640 1 -- 172.21.15.46:0/38162410 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ec4007020 con 0x55d252cfe290 2024-03-29T08:21:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 -- 172.21.15.46:0/38162410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d252ab2620 con 0x55d252cfe290 2024-03-29T08:21:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0eb3e00640 1 -- 172.21.15.46:0/38162410 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ec4007240 con 0x55d252cfe290 2024-03-29T08:21:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 10 client.6357 fetch_fsmap learned FSMap version 33 2024-03-29T08:21:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 -- 172.21.15.46:0/38162410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55d2527fefc0 con 0x55d252cfe290 2024-03-29T08:21:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0eb3e00640 1 -- 172.21.15.46:0/38162410 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d252ab2620 con 0x55d252cfe290 2024-03-29T08:21:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0eb3e00640 1 -- 172.21.15.46:0/38162410 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0ec4012070 con 0x55d252cfe290 2024-03-29T08:21:54.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 10 client.6357 fetch_fsmap finished waiting for FSMap version 33 2024-03-29T08:21:54.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 10 client.6357 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:21:54.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 20 client.6357 populate_metadata read hostname 'smithi046' 2024-03-29T08:21:54.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 --2- 172.21.15.46:0/38162410 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55d252d58820 0x55d252d58c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:54.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 4 client.6357 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-29T08:21:54.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 -- 172.21.15.46:0/38162410 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55d252b014e0 con 0x55d252d58820 2024-03-29T08:21:54.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0ed2000640 1 --2- 172.21.15.46:0/38162410 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55d252d58820 0x55d252d58c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:54.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0eb3e00640 10 client.6357 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:54.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0ed2000640 1 --2- 172.21.15.46:0/38162410 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55d252d58820 0x55d252d58c00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=718bbf28d538f991 server_cookie=1a3ae7c6b9a23115 in_seq=0 out_seq=0 2024-03-29T08:21:54.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0eb3e00640 1 -- 172.21.15.46:0/38162410 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x55d252b014e0 con 0x55d252d58820 2024-03-29T08:21:54.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0eb3e00640 10 client.6357 handle_command_reply: tid=0 2024-03-29T08:21:54.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 client.6357 shutdown 2024-03-29T08:21:54.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 -- 172.21.15.46:0/38162410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 msgr2=0x55d252d0fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 --2- 172.21.15.46:0/38162410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252d0fe70 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001ec0 tx=0x7f0ec4000bd0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 -- 172.21.15.46:0/38162410 shutdown_connections 2024-03-29T08:21:54.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 --2- 172.21.15.46:0/38162410 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55d252d58820 0x55d252d58c00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 --2- 172.21.15.46:0/38162410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d252d108f0 0x55d252d582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 --2- 172.21.15.46:0/38162410 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d252cfebb0 0x55d252d103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 --2- 172.21.15.46:0/38162410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d252cfe290 0x55d252d0fe70 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 -- 172.21.15.46:0/38162410 >> 172.21.15.46:0/38162410 conn(0x55d252cb1160 msgr2=0x55d252cfcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:54.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0eb3e00640 0 client.6357 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:54.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 -- 172.21.15.46:0/38162410 shutdown_connections 2024-03-29T08:21:54.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 1 -- 172.21.15.46:0/38162410 wait complete. 2024-03-29T08:21:54.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.950+0000 7f0edccff000 20 client.6357 trim_cache size 0 max 16384 2024-03-29T08:21:54.971 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:54.971 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-03-29T08:21:54.972 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:21:54.972 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-03-29T08:21:54.972 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 2281190255, 2024-03-29T08:21:54.972 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627777, 2024-03-29T08:21:54.972 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/dir_test/file" 2024-03-29T08:21:54.972 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:54.973 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-03-29T08:21:54.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.954+0000 7f0ed9800640 1 -- 172.21.15.46:0/371467878 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0eb40666c0 msgr2=0x7f0eb4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.954+0000 7f0ed9800640 1 --2- 172.21.15.46:0/371467878 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0eb40666c0 0x7f0eb4068b80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001770 tx=0x7f0ec4009040 comp rx=0 tx=0).stop 2024-03-29T08:21:54.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.954+0000 7f0ed9800640 1 -- 172.21.15.46:0/371467878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed415cad0 msgr2=0x7f0ed4154ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:54.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.954+0000 7f0ed9800640 1 --2- 172.21.15.46:0/371467878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed415cad0 0x7f0ed4154ce0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc00cb40 tx=0x7f0ecc00a060 comp rx=0 tx=0).stop 2024-03-29T08:21:54.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.954+0000 7f0ed9800640 1 -- 172.21.15.46:0/371467878 shutdown_connections 2024-03-29T08:21:54.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.954+0000 7f0ed9800640 1 --2- 172.21.15.46:0/371467878 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0eb40666c0 0x7f0eb4068b80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.954+0000 7f0ed9800640 1 --2- 172.21.15.46:0/371467878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ed415cad0 0x7f0ed4154ce0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.954+0000 7f0ed9800640 1 --2- 172.21.15.46:0/371467878 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0ed406f1c0 0x7f0ed41547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.954+0000 7f0ed9800640 1 --2- 172.21.15.46:0/371467878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ed406a2d0 0x7f0ed4148330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:54.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.954+0000 7f0ed9800640 1 -- 172.21.15.46:0/371467878 >> 172.21.15.46:0/371467878 conn(0x7f0ed4098a40 msgr2=0x7f0ed40affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:54.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.954+0000 7f0ed9800640 1 -- 172.21.15.46:0/371467878 shutdown_connections 2024-03-29T08:21:54.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:54.954+0000 7f0ed9800640 1 -- 172.21.15.46:0/371467878 wait complete. 2024-03-29T08:21:54.978 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-29T08:21:55.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.114+0000 7f3075800640 1 Processor -- start 2024-03-29T08:21:55.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.114+0000 7f3075800640 1 -- start start 2024-03-29T08:21:55.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.114+0000 7f3075800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f307015ab70 0x7f3070098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.114+0000 7f3075800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3070098f80 0x7f30700b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.114+0000 7f3075800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30700993d0 con 0x7f307015a790 2024-03-29T08:21:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.114+0000 7f3075800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30700cf700 con 0x7f307015ab70 2024-03-29T08:21:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.114+0000 7f3075800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30700ce660 con 0x7f3070098f80 2024-03-29T08:21:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.114+0000 7f3067e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f307015ab70 0x7f3070098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.114+0000 7f3067e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f307015ab70 0x7f3070098a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55026/0 (socket says 172.21.15.46:55026) 2024-03-29T08:21:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.114+0000 7f3067e00640 1 -- 172.21.15.46:0/1705048691 learned_addr learned my addr 172.21.15.46:0/1705048691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.118+0000 7f306f400640 1 --2- 172.21.15.46:0/1705048691 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3070098f80 0x7f30700b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.118+0000 7f3067e00640 1 -- 172.21.15.46:0/1705048691 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3070098f80 msgr2=0x7f30700b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.118+0000 7f3067e00640 1 --2- 172.21.15.46:0/1705048691 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3070098f80 0x7f30700b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.118+0000 7f3067e00640 1 -- 172.21.15.46:0/1705048691 >> v1:172.21.15.73:6789/0 conn(0x7f307015a790 legacy=0x7f30700cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.118+0000 7f3067e00640 1 -- 172.21.15.46:0/1705048691 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30700ce940 con 0x7f307015ab70 2024-03-29T08:21:55.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.118+0000 7f3067e00640 1 --2- 172.21.15.46:0/1705048691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307015ab70 0x7f3070098a40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3058009870 tx=0x7f305801c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc64a68316508d03 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:55.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.118+0000 7f306e000640 1 -- 172.21.15.46:0/1705048691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058004230 con 0x7f307015ab70 2024-03-29T08:21:55.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.118+0000 7f306e000640 1 -- 172.21.15.46:0/1705048691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30580a7050 con 0x7f307015ab70 2024-03-29T08:21:55.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.118+0000 7f306e000640 1 -- 172.21.15.46:0/1705048691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058003620 con 0x7f307015ab70 2024-03-29T08:21:55.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3075800640 1 -- 172.21.15.46:0/1705048691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307015ab70 msgr2=0x7f3070098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3075800640 1 --2- 172.21.15.46:0/1705048691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307015ab70 0x7f3070098a40 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f3058009870 tx=0x7f305801c350 comp rx=0 tx=0).stop 2024-03-29T08:21:55.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3075800640 1 -- 172.21.15.46:0/1705048691 shutdown_connections 2024-03-29T08:21:55.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3075800640 1 --2- 172.21.15.46:0/1705048691 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3070098f80 0x7f30700b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3075800640 1 --2- 172.21.15.46:0/1705048691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307015ab70 0x7f3070098a40 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3075800640 1 -- 172.21.15.46:0/1705048691 >> 172.21.15.46:0/1705048691 conn(0x7f30700b21a0 msgr2=0x7f30700b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:55.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3075800640 1 -- 172.21.15.46:0/1705048691 shutdown_connections 2024-03-29T08:21:55.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3075800640 1 -- 172.21.15.46:0/1705048691 wait complete. 2024-03-29T08:21:55.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3075800640 1 Processor -- start 2024-03-29T08:21:55.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3075800640 1 -- start start 2024-03-29T08:21:55.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3075800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3070098f80 0x7f307006aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3075800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307015a790 0x7f307015d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3075800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f307015ab70 0x7f30701488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3075800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307006c330 con 0x7f307015a790 2024-03-29T08:21:55.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3075800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307006b3e0 con 0x7f307015ab70 2024-03-29T08:21:55.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3075800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307006b6b0 con 0x7f3070098f80 2024-03-29T08:21:55.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3067e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307015a790 0x7f307015d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3067e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307015a790 0x7f307015d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55032/0 (socket says 172.21.15.46:55032) 2024-03-29T08:21:55.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3067e00640 1 -- 172.21.15.46:0/3743596661 learned_addr learned my addr 172.21.15.46:0/3743596661 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f306f400640 1 --2- 172.21.15.46:0/3743596661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f307015ab70 0x7f30701488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f306ea00640 1 --2- 172.21.15.46:0/3743596661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3070098f80 0x7f307006aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3067e00640 1 -- 172.21.15.46:0/3743596661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3070098f80 msgr2=0x7f307006aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3067e00640 1 --2- 172.21.15.46:0/3743596661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3070098f80 0x7f307006aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3067e00640 1 -- 172.21.15.46:0/3743596661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f307015ab70 msgr2=0x7f30701488d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3067e00640 1 --2- 172.21.15.46:0/3743596661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f307015ab70 0x7f30701488d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3067e00640 1 -- 172.21.15.46:0/3743596661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30580094d0 con 0x7f307015a790 2024-03-29T08:21:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f306ea00640 1 --2- 172.21.15.46:0/3743596661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3070098f80 0x7f307006aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f306f400640 1 --2- 172.21.15.46:0/3743596661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f307015ab70 0x7f30701488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:55.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.122+0000 7f3067e00640 1 --2- 172.21.15.46:0/3743596661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307015a790 0x7f307015d460 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3058019f80 tx=0x7f30580190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.126+0000 7f3066a00640 1 -- 172.21.15.46:0/3743596661 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058003cf0 con 0x7f307015a790 2024-03-29T08:21:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.126+0000 7f3066a00640 1 -- 172.21.15.46:0/3743596661 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3058003e50 con 0x7f307015a790 2024-03-29T08:21:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.126+0000 7f3066a00640 1 -- 172.21.15.46:0/3743596661 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30580b6660 con 0x7f307015a790 2024-03-29T08:21:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.126+0000 7f3075800640 1 -- 172.21.15.46:0/3743596661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307006b990 con 0x7f307015a790 2024-03-29T08:21:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.126+0000 7f3075800640 1 -- 172.21.15.46:0/3743596661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307006bf50 con 0x7f307015a790 2024-03-29T08:21:55.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.126+0000 7f3075800640 1 -- 172.21.15.46:0/3743596661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3070066c90 con 0x7f307015a790 2024-03-29T08:21:55.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.126+0000 7f3066a00640 1 -- 172.21.15.46:0/3743596661 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f30580b2040 con 0x7f307015a790 2024-03-29T08:21:55.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.126+0000 7f3066a00640 1 --2- 172.21.15.46:0/3743596661 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f304c0666c0 0x7f304c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.126+0000 7f3066a00640 1 -- 172.21.15.46:0/3743596661 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f305811e550 con 0x7f307015a790 2024-03-29T08:21:55.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.126+0000 7f306ea00640 1 --2- 172.21.15.46:0/3743596661 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f304c0666c0 0x7f304c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.126+0000 7f306ea00640 1 --2- 172.21.15.46:0/3743596661 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f304c0666c0 0x7f304c068b80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3060000f70 tx=0x7f3060009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:55.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.130+0000 7f3066a00640 1 -- 172.21.15.46:0/3743596661 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f30580eb8b0 con 0x7f307015a790 2024-03-29T08:21:55.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.326+0000 7f3075800640 1 -- 172.21.15.46:0/3743596661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3070159ec0 con 0x7f307015a790 2024-03-29T08:21:55.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.326+0000 7f3066a00640 1 -- 172.21.15.46:0/3743596661 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f30580eeed0 con 0x7f307015a790 2024-03-29T08:21:55.330 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:21:55.330 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-03-29T08:21:55.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.330+0000 7f3075800640 1 -- 172.21.15.46:0/3743596661 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f304c0666c0 msgr2=0x7f304c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.330+0000 7f3075800640 1 --2- 172.21.15.46:0/3743596661 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f304c0666c0 0x7f304c068b80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3060000f70 tx=0x7f3060009040 comp rx=0 tx=0).stop 2024-03-29T08:21:55.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.330+0000 7f3075800640 1 -- 172.21.15.46:0/3743596661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307015a790 msgr2=0x7f307015d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.330+0000 7f3075800640 1 --2- 172.21.15.46:0/3743596661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307015a790 0x7f307015d460 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3058019f80 tx=0x7f30580190f0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.330+0000 7f3075800640 1 -- 172.21.15.46:0/3743596661 shutdown_connections 2024-03-29T08:21:55.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.330+0000 7f3075800640 1 --2- 172.21.15.46:0/3743596661 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f304c0666c0 0x7f304c068b80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.330+0000 7f3075800640 1 --2- 172.21.15.46:0/3743596661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f307015ab70 0x7f30701488d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.330+0000 7f3075800640 1 --2- 172.21.15.46:0/3743596661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307015a790 0x7f307015d460 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.330+0000 7f3075800640 1 --2- 172.21.15.46:0/3743596661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3070098f80 0x7f307006aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.330+0000 7f3075800640 1 -- 172.21.15.46:0/3743596661 >> 172.21.15.46:0/3743596661 conn(0x7f30700b21a0 msgr2=0x7f30700b44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:55.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.330+0000 7f3075800640 1 -- 172.21.15.46:0/3743596661 shutdown_connections 2024-03-29T08:21:55.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.330+0000 7f3075800640 1 -- 172.21.15.46:0/3743596661 wait complete. 2024-03-29T08:21:55.351 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-29T08:21:55.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:55.352 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-03-29T08:21:55.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.490+0000 7f8a2fc00640 1 Processor -- start 2024-03-29T08:21:55.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.490+0000 7f8a2fc00640 1 -- start start 2024-03-29T08:21:55.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.494+0000 7f8a2fc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8a28071100 0x7f8a28161100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.494+0000 7f8a2fc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a281606b0 con 0x7f8a2815cc50 2024-03-29T08:21:55.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.494+0000 7f8a2fc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a28160c70 con 0x7f8a2806b470 2024-03-29T08:21:55.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.494+0000 7f8a2fc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a2815d030 con 0x7f8a28071100 2024-03-29T08:21:55.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.494+0000 7f8a27e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8a28071100 0x7f8a28161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.494+0000 7f8a2d200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8a2815cc50 0x7f8a280709f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36214/0 (socket says 172.21.15.46:36214) 2024-03-29T08:21:55.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.494+0000 7f8a2d200640 1 -- 172.21.15.46:0/3911588737 learned_addr learned my addr 172.21.15.46:0/3911588737 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:21:55.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.494+0000 7f8a27e00640 1 -- 172.21.15.46:0/3911588737 >> v1:172.21.15.73:6789/0 conn(0x7f8a2806b470 legacy=0x7f8a2806d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.494+0000 7f8a27e00640 1 -- 172.21.15.46:0/3911588737 >> v1:172.21.15.46:6789/0 conn(0x7f8a2815cc50 legacy=0x7f8a280709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.494+0000 7f8a27e00640 1 -- 172.21.15.46:0/3911588737 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a280ce790 con 0x7f8a28071100 2024-03-29T08:21:55.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.494+0000 7f8a27400640 1 -- 172.21.15.46:0/3911588737 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2446051562 0 0) 0x7f8a281606b0 con 0x7f8a2815cc50 2024-03-29T08:21:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.498+0000 7f8a27e00640 1 --2- 172.21.15.46:0/3911588737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a28071100 0x7f8a28161100 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8a140097c0 tx=0x7f8a2806e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08b1f619949af25 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.498+0000 7f8a27400640 1 -- 172.21.15.46:0/3911588737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a140a8050 con 0x7f8a28071100 2024-03-29T08:21:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.498+0000 7f8a27400640 1 -- 172.21.15.46:0/3911588737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a140ac5f0 con 0x7f8a28071100 2024-03-29T08:21:55.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.498+0000 7f8a27400640 1 -- 172.21.15.46:0/3911588737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a140ac980 con 0x7f8a28071100 2024-03-29T08:21:55.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.498+0000 7f8a2fc00640 1 -- 172.21.15.46:0/3911588737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a28071100 msgr2=0x7f8a28161100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.498+0000 7f8a2fc00640 1 --2- 172.21.15.46:0/3911588737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a28071100 0x7f8a28161100 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8a140097c0 tx=0x7f8a2806e7e0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.498+0000 7f8a2fc00640 1 -- 172.21.15.46:0/3911588737 shutdown_connections 2024-03-29T08:21:55.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.498+0000 7f8a2fc00640 1 --2- 172.21.15.46:0/3911588737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a28071100 0x7f8a28161100 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.498+0000 7f8a2fc00640 1 -- 172.21.15.46:0/3911588737 >> 172.21.15.46:0/3911588737 conn(0x7f8a280ced50 msgr2=0x7f8a280cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:55.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.498+0000 7f8a2fc00640 1 -- 172.21.15.46:0/3911588737 shutdown_connections 2024-03-29T08:21:55.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.498+0000 7f8a2fc00640 1 -- 172.21.15.46:0/3911588737 wait complete. 2024-03-29T08:21:55.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.498+0000 7f8a2fc00640 1 Processor -- start 2024-03-29T08:21:55.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.498+0000 7f8a2fc00640 1 -- start start 2024-03-29T08:21:55.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.498+0000 7f8a2fc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a2806b470 0x7f8a280ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.498+0000 7f8a2fc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a28071100 0x7f8a2815f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a2fc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a2815cc50 0x7f8a28148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a2fc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a280ac600 con 0x7f8a28071100 2024-03-29T08:21:55.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a2fc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2815fe90 con 0x7f8a2815cc50 2024-03-29T08:21:55.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a2fc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a280ad1c0 con 0x7f8a2806b470 2024-03-29T08:21:55.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a27e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a28071100 0x7f8a2815f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a27e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a28071100 0x7f8a2815f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55062/0 (socket says 172.21.15.46:55062) 2024-03-29T08:21:55.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a27e00640 1 -- 172.21.15.46:0/3968730661 learned_addr learned my addr 172.21.15.46:0/3968730661 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:55.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a2d200640 1 --2- 172.21.15.46:0/3968730661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a2806b470 0x7f8a280ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a2dc00640 1 --2- 172.21.15.46:0/3968730661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a2815cc50 0x7f8a28148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a27e00640 1 -- 172.21.15.46:0/3968730661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a2806b470 msgr2=0x7f8a280ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a27e00640 1 --2- 172.21.15.46:0/3968730661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a2806b470 0x7f8a280ac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a27e00640 1 -- 172.21.15.46:0/3968730661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a2815cc50 msgr2=0x7f8a28148570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a27e00640 1 --2- 172.21.15.46:0/3968730661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a2815cc50 0x7f8a28148570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a27e00640 1 -- 172.21.15.46:0/3968730661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a14009420 con 0x7f8a28071100 2024-03-29T08:21:55.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a2d200640 1 --2- 172.21.15.46:0/3968730661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a2806b470 0x7f8a280ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:55.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a27e00640 1 --2- 172.21.15.46:0/3968730661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a28071100 0x7f8a2815f950 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8a1401a0d0 tx=0x7f8a1401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:55.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a24c00640 1 -- 172.21.15.46:0/3968730661 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a140b7860 con 0x7f8a28071100 2024-03-29T08:21:55.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a24c00640 1 -- 172.21.15.46:0/3968730661 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a140b7a00 con 0x7f8a28071100 2024-03-29T08:21:55.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a24c00640 1 -- 172.21.15.46:0/3968730661 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a140b7db0 con 0x7f8a28071100 2024-03-29T08:21:55.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a2fc00640 1 -- 172.21.15.46:0/3968730661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a281600a0 con 0x7f8a28071100 2024-03-29T08:21:55.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a2fc00640 1 -- 172.21.15.46:0/3968730661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a280acb70 con 0x7f8a28071100 2024-03-29T08:21:55.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.502+0000 7f8a24c00640 1 -- 172.21.15.46:0/3968730661 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8a140b3070 con 0x7f8a28071100 2024-03-29T08:21:55.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a24c00640 1 --2- 172.21.15.46:0/3968730661 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8a08066690 0x7f8a08068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a24c00640 1 -- 172.21.15.46:0/3968730661 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a1411e530 con 0x7f8a28071100 2024-03-29T08:21:55.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a2d200640 1 --2- 172.21.15.46:0/3968730661 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8a08066690 0x7f8a08068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a330e6000 1 Processor -- start 2024-03-29T08:21:55.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a330e6000 1 -- start start 2024-03-29T08:21:55.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a2d200640 1 --2- 172.21.15.46:0/3968730661 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8a08066690 0x7f8a08068b50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c000f70 tx=0x7f8a1c007df0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:55.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a330e6000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950391b20 0x560950391f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a330e6000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950392440 0x560950347940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a330e6000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950347e80 0x560950348260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a330e6000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609502105c0 con 0x560950391b20 2024-03-29T08:21:55.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a330e6000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56095019ca00 con 0x560950347e80 2024-03-29T08:21:55.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a330e6000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560950199fd0 con 0x560950392440 2024-03-29T08:21:55.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a2d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950391b20 0x560950391f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a2d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950391b20 0x560950391f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55066/0 (socket says 172.21.15.46:55066) 2024-03-29T08:21:55.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a2d200640 1 -- 172.21.15.46:0/3373202208 learned_addr learned my addr 172.21.15.46:0/3373202208 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:55.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a2dc00640 1 --2- 172.21.15.46:0/3373202208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950347e80 0x560950348260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a27e00640 1 --2- 172.21.15.46:0/3373202208 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950392440 0x560950347940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a2d200640 1 -- 172.21.15.46:0/3373202208 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950392440 msgr2=0x560950347940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a2d200640 1 --2- 172.21.15.46:0/3373202208 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950392440 0x560950347940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a2d200640 1 -- 172.21.15.46:0/3373202208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950347e80 msgr2=0x560950348260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a2d200640 1 --2- 172.21.15.46:0/3373202208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950347e80 0x560950348260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a2d200640 1 -- 172.21.15.46:0/3373202208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609501368c0 con 0x560950391b20 2024-03-29T08:21:55.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a2d200640 1 --2- 172.21.15.46:0/3373202208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950391b20 0x560950391f00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c007800 tx=0x7f8a1c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7f980cd1b0faab server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a19600640 1 -- 172.21.15.46:0/3373202208 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c004e00 con 0x560950391b20 2024-03-29T08:21:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a19600640 1 -- 172.21.15.46:0/3373202208 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1c005050 con 0x560950391b20 2024-03-29T08:21:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a19600640 1 -- 172.21.15.46:0/3373202208 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c005400 con 0x560950391b20 2024-03-29T08:21:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a330e6000 1 -- 172.21.15.46:0/3373202208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950391b20 msgr2=0x560950391f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a330e6000 1 --2- 172.21.15.46:0/3373202208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950391b20 0x560950391f00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c007800 tx=0x7f8a1c011000 comp rx=0 tx=0).stop 2024-03-29T08:21:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a330e6000 1 -- 172.21.15.46:0/3373202208 shutdown_connections 2024-03-29T08:21:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a330e6000 1 --2- 172.21.15.46:0/3373202208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950347e80 0x560950348260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a330e6000 1 --2- 172.21.15.46:0/3373202208 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950392440 0x560950347940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a330e6000 1 --2- 172.21.15.46:0/3373202208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950391b20 0x560950391f00 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a330e6000 1 -- 172.21.15.46:0/3373202208 >> 172.21.15.46:0/3373202208 conn(0x56095034e400 msgr2=0x560950346220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a330e6000 1 -- 172.21.15.46:0/3373202208 shutdown_connections 2024-03-29T08:21:55.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.506+0000 7f8a330e6000 1 -- 172.21.15.46:0/3373202208 wait complete. 2024-03-29T08:21:55.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 1 -- start start 2024-03-29T08:21:55.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 1 Processor -- start 2024-03-29T08:21:55.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950347e80 0x560950352cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950391b20 0x560950353230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950392440 0x560950353770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609501958c0 con 0x560950392440 2024-03-29T08:21:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560950135160 con 0x560950391b20 2024-03-29T08:21:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560950134e00 con 0x560950347e80 2024-03-29T08:21:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 10 client.0 _ll_register_callbacks cb 0x560950333820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:21:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a2dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950392440 0x560950353770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a2d200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950347e80 0x560950352cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a2dc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950392440 0x560950353770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55078/0 (socket says 172.21.15.46:55078) 2024-03-29T08:21:55.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a2dc00640 1 -- 172.21.15.46:0/1845587746 learned_addr learned my addr 172.21.15.46:0/1845587746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:55.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a27e00640 1 --2- 172.21.15.46:0/1845587746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950391b20 0x560950353230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a2dc00640 1 -- 172.21.15.46:0/1845587746 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950347e80 msgr2=0x560950352cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a2dc00640 1 --2- 172.21.15.46:0/1845587746 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950347e80 0x560950352cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a2dc00640 1 -- 172.21.15.46:0/1845587746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950391b20 msgr2=0x560950353230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a2dc00640 1 --2- 172.21.15.46:0/1845587746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950391b20 0x560950353230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a2dc00640 1 -- 172.21.15.46:0/1845587746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1c00f040 con 0x560950392440 2024-03-29T08:21:55.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a2dc00640 1 --2- 172.21.15.46:0/1845587746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950392440 0x560950353770 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8a20011580 tx=0x7f8a20011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ca15395547fe21 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a0e000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:21:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a0e000640 1 -- 172.21.15.46:0/1845587746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2001a070 con 0x560950392440 2024-03-29T08:21:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a0e000640 1 -- 172.21.15.46:0/1845587746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a20006e30 con 0x560950392440 2024-03-29T08:21:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 1 -- 172.21.15.46:0/1845587746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560950194210 con 0x560950392440 2024-03-29T08:21:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a0e000640 1 -- 172.21.15.46:0/1845587746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a20003980 con 0x560950392440 2024-03-29T08:21:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 10 client.6378 fetch_fsmap learned FSMap version 33 2024-03-29T08:21:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 1 -- 172.21.15.46:0/1845587746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x56094fe91d50 con 0x560950392440 2024-03-29T08:21:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a0e000640 1 -- 172.21.15.46:0/1845587746 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a20024020 con 0x560950392440 2024-03-29T08:21:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a0e000640 1 -- 172.21.15.46:0/1845587746 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f8a2001d020 con 0x560950392440 2024-03-29T08:21:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 10 client.6378 fetch_fsmap finished waiting for FSMap version 33 2024-03-29T08:21:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 10 client.6378 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-03-29T08:21:55.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 20 client.6378 populate_metadata read hostname 'smithi046' 2024-03-29T08:21:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 1 --2- 172.21.15.46:0/1845587746 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5609503a0dc0 0x5609503a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 4 client.6378 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:21:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a330e6000 1 -- 172.21.15.46:0/1845587746 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560950153aa0 con 0x5609503a0dc0 2024-03-29T08:21:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.510+0000 7f8a2d200640 1 --2- 172.21.15.46:0/1845587746 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5609503a0dc0 0x5609503a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.514+0000 7f8a0e000640 10 client.6378 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.514+0000 7f8a2d200640 1 --2- 172.21.15.46:0/1845587746 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5609503a0dc0 0x5609503a31b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3f0c72f5989fa0f server_cookie=2316d39352de08c8 in_seq=0 out_seq=0 2024-03-29T08:21:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.514+0000 7f8a0e000640 1 -- 172.21.15.46:0/1845587746 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560950153aa0 con 0x5609503a0dc0 2024-03-29T08:21:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.514+0000 7f8a0e000640 10 client.6378 handle_command_reply: tid=0 2024-03-29T08:21:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.514+0000 7f8a330e6000 1 client.6378 shutdown 2024-03-29T08:21:55.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.514+0000 7f8a330e6000 1 -- 172.21.15.46:0/1845587746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950392440 msgr2=0x560950353770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.514+0000 7f8a330e6000 1 --2- 172.21.15.46:0/1845587746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950392440 0x560950353770 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8a20011580 tx=0x7f8a20011a40 comp rx=0 tx=0).stop 2024-03-29T08:21:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.514+0000 7f8a330e6000 1 -- 172.21.15.46:0/1845587746 shutdown_connections 2024-03-29T08:21:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.514+0000 7f8a330e6000 1 --2- 172.21.15.46:0/1845587746 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5609503a0dc0 0x5609503a31b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.514+0000 7f8a330e6000 1 --2- 172.21.15.46:0/1845587746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950391b20 0x560950353230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.514+0000 7f8a330e6000 1 --2- 172.21.15.46:0/1845587746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950392440 0x560950353770 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.514+0000 7f8a0e000640 0 client.6378 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.514+0000 7f8a330e6000 1 --2- 172.21.15.46:0/1845587746 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950347e80 0x560950352cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.514+0000 7f8a330e6000 1 -- 172.21.15.46:0/1845587746 >> 172.21.15.46:0/1845587746 conn(0x56095034e400 msgr2=0x560950346920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.514+0000 7f8a330e6000 1 -- 172.21.15.46:0/1845587746 shutdown_connections 2024-03-29T08:21:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.514+0000 7f8a330e6000 1 -- 172.21.15.46:0/1845587746 wait complete. 2024-03-29T08:21:55.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.514+0000 7f8a330e6000 20 client.6378 trim_cache size 0 max 16384 2024-03-29T08:21:55.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 Processor -- start 2024-03-29T08:21:55.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 -- start start 2024-03-29T08:21:55.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950347e80 0x5609503ebb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950391b20 0x5609503ec0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950392440 0x5609503ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094fe529d0 con 0x560950391b20 2024-03-29T08:21:55.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560950198420 con 0x560950347e80 2024-03-29T08:21:55.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560950132ea0 con 0x560950392440 2024-03-29T08:21:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a27e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950391b20 0x5609503ec0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a27e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950391b20 0x5609503ec0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55094/0 (socket says 172.21.15.46:55094) 2024-03-29T08:21:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a27e00640 1 -- 172.21.15.46:0/1818196473 learned_addr learned my addr 172.21.15.46:0/1818196473 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a2d200640 1 --2- 172.21.15.46:0/1818196473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950347e80 0x5609503ebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a2dc00640 1 --2- 172.21.15.46:0/1818196473 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950392440 0x5609503ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a27e00640 1 -- 172.21.15.46:0/1818196473 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950392440 msgr2=0x5609503ec5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a27e00640 1 --2- 172.21.15.46:0/1818196473 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950392440 0x5609503ec5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a27e00640 1 -- 172.21.15.46:0/1818196473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950347e80 msgr2=0x5609503ebb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a27e00640 1 --2- 172.21.15.46:0/1818196473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950347e80 0x5609503ebb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a27e00640 1 -- 172.21.15.46:0/1818196473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56095015aba0 con 0x560950391b20 2024-03-29T08:21:55.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a2d200640 1 --2- 172.21.15.46:0/1818196473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950347e80 0x5609503ebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:55.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a27e00640 1 --2- 172.21.15.46:0/1818196473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950391b20 0x5609503ec0b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8a1401c980 tx=0x7f8a1411e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed4307ebe271d3ba server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:55.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a1a000640 1 -- 172.21.15.46:0/1818196473 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a140a8020 con 0x560950391b20 2024-03-29T08:21:55.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a1a000640 1 -- 172.21.15.46:0/1818196473 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a140ef5f0 con 0x560950391b20 2024-03-29T08:21:55.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a1a000640 1 -- 172.21.15.46:0/1818196473 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a140f3420 con 0x560950391b20 2024-03-29T08:21:55.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 -- 172.21.15.46:0/1818196473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950391b20 msgr2=0x5609503ec0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 --2- 172.21.15.46:0/1818196473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950391b20 0x5609503ec0b0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8a1401c980 tx=0x7f8a1411e040 comp rx=0 tx=0).stop 2024-03-29T08:21:55.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 -- 172.21.15.46:0/1818196473 shutdown_connections 2024-03-29T08:21:55.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 --2- 172.21.15.46:0/1818196473 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950392440 0x5609503ec5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 --2- 172.21.15.46:0/1818196473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950391b20 0x5609503ec0b0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 --2- 172.21.15.46:0/1818196473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950347e80 0x5609503ebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 -- 172.21.15.46:0/1818196473 >> 172.21.15.46:0/1818196473 conn(0x56095034e400 msgr2=0x5609503a3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:55.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 -- 172.21.15.46:0/1818196473 shutdown_connections 2024-03-29T08:21:55.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 -- 172.21.15.46:0/1818196473 wait complete. 2024-03-29T08:21:55.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 -- start start 2024-03-29T08:21:55.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.530+0000 7f8a330e6000 1 Processor -- start 2024-03-29T08:21:55.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a330e6000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950347e80 0x5609503f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a330e6000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950391b20 0x5609503f0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a330e6000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950392440 0x56095038f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a330e6000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56095034aca0 con 0x560950347e80 2024-03-29T08:21:55.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a330e6000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56095011ca80 con 0x560950391b20 2024-03-29T08:21:55.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a330e6000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x56095015cef0 con 0x560950392440 2024-03-29T08:21:55.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a330e6000 10 client.0 _ll_register_callbacks cb 0x560950143f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:21:55.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a2d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950347e80 0x5609503f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a2d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950347e80 0x5609503f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55108/0 (socket says 172.21.15.46:55108) 2024-03-29T08:21:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a2d200640 1 -- 172.21.15.46:0/989900199 learned_addr learned my addr 172.21.15.46:0/989900199 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a27e00640 1 --2- 172.21.15.46:0/989900199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950391b20 0x5609503f0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a2dc00640 1 --2- 172.21.15.46:0/989900199 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950392440 0x56095038f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a2d200640 1 -- 172.21.15.46:0/989900199 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950392440 msgr2=0x56095038f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a2d200640 1 --2- 172.21.15.46:0/989900199 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950392440 0x56095038f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a2d200640 1 -- 172.21.15.46:0/989900199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950391b20 msgr2=0x5609503f0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a2d200640 1 --2- 172.21.15.46:0/989900199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950391b20 0x5609503f0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a2d200640 1 -- 172.21.15.46:0/989900199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a280acb70 con 0x560950347e80 2024-03-29T08:21:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a2d200640 1 --2- 172.21.15.46:0/989900199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950347e80 0x5609503f0430 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c000c10 tx=0x7f8a1c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6edb7f665e01d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:55.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a0e000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:21:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a0e000640 1 -- 172.21.15.46:0/989900199 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c027040 con 0x560950347e80 2024-03-29T08:21:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a0e000640 1 -- 172.21.15.46:0/989900199 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a1c02c900 con 0x560950347e80 2024-03-29T08:21:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a330e6000 1 -- 172.21.15.46:0/989900199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560950353b40 con 0x560950347e80 2024-03-29T08:21:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a0e000640 1 -- 172.21.15.46:0/989900199 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a1c02c5d0 con 0x560950347e80 2024-03-29T08:21:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a330e6000 10 client.6384 fetch_fsmap learned FSMap version 33 2024-03-29T08:21:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a330e6000 1 -- 172.21.15.46:0/989900199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x560950140cf0 con 0x560950347e80 2024-03-29T08:21:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a0e000640 1 -- 172.21.15.46:0/989900199 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560950353b40 con 0x560950347e80 2024-03-29T08:21:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a0e000640 1 -- 172.21.15.46:0/989900199 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f8a1c02cac0 con 0x560950347e80 2024-03-29T08:21:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a330e6000 10 client.6384 fetch_fsmap finished waiting for FSMap version 33 2024-03-29T08:21:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a330e6000 10 client.6384 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-03-29T08:21:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a330e6000 20 client.6384 populate_metadata read hostname 'smithi046' 2024-03-29T08:21:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a330e6000 1 --2- 172.21.15.46:0/989900199 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5609503905b0 0x5609503909c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a330e6000 4 client.6384 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-03-29T08:21:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a330e6000 1 -- 172.21.15.46:0/989900199 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) v1 -- 0x5609501926a0 con 0x5609503905b0 2024-03-29T08:21:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a27e00640 1 --2- 172.21.15.46:0/989900199 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5609503905b0 0x5609503909c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a27e00640 1 --2- 172.21.15.46:0/989900199 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5609503905b0 0x5609503909c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ebe3a12fab64cbe server_cookie=6dd041244f8a3af0 in_seq=0 out_seq=0 2024-03-29T08:21:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a0e000640 10 client.6384 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a0e000640 1 -- 172.21.15.46:0/989900199 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5609501926a0 con 0x5609503905b0 2024-03-29T08:21:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a0e000640 10 client.6384 handle_command_reply: tid=0 2024-03-29T08:21:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.534+0000 7f8a330e6000 1 client.6384 shutdown 2024-03-29T08:21:55.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a330e6000 1 -- 172.21.15.46:0/989900199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950347e80 msgr2=0x5609503f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a330e6000 1 --2- 172.21.15.46:0/989900199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950347e80 0x5609503f0430 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c000c10 tx=0x7f8a1c013000 comp rx=0 tx=0).stop 2024-03-29T08:21:55.545 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f1cc2600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-03-29T08:21:22.608086+0000 1=1+0) (iversion lock) caps={6249=pAsLsXsFs/-@2},l=6249(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x5598e82ca100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-29T08:21:55.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a330e6000 1 -- 172.21.15.46:0/989900199 shutdown_connections 2024-03-29T08:21:55.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a330e6000 1 --2- 172.21.15.46:0/989900199 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5609503905b0 0x5609503909c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a330e6000 1 --2- 172.21.15.46:0/989900199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560950391b20 0x5609503f0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a330e6000 1 --2- 172.21.15.46:0/989900199 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560950392440 0x56095038f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a330e6000 1 --2- 172.21.15.46:0/989900199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560950347e80 0x5609503f0430 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a330e6000 1 -- 172.21.15.46:0/989900199 >> 172.21.15.46:0/989900199 conn(0x56095034e400 msgr2=0x5609503a4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:55.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a0e000640 0 client.6384 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:55.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a330e6000 1 -- 172.21.15.46:0/989900199 shutdown_connections 2024-03-29T08:21:55.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a330e6000 1 -- 172.21.15.46:0/989900199 wait complete. 2024-03-29T08:21:55.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a330e6000 20 client.6384 trim_cache size 0 max 16384 2024-03-29T08:21:55.547 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:21:55.548 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-29T08:21:55.548 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "534f32e7-216b-4f5d-a059-67ee65cc24df", 2024-03-29T08:21:55.548 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-29T08:21:55.548 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:21:55.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a2fc00640 1 -- 172.21.15.46:0/3968730661 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8a08066690 msgr2=0x7f8a08068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a2fc00640 1 --2- 172.21.15.46:0/3968730661 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8a08066690 0x7f8a08068b50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c000f70 tx=0x7f8a1c007df0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a2fc00640 1 -- 172.21.15.46:0/3968730661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a28071100 msgr2=0x7f8a2815f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a2fc00640 1 --2- 172.21.15.46:0/3968730661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a28071100 0x7f8a2815f950 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8a1401a0d0 tx=0x7f8a1401ace0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a2fc00640 1 -- 172.21.15.46:0/3968730661 shutdown_connections 2024-03-29T08:21:55.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a2fc00640 1 --2- 172.21.15.46:0/3968730661 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8a08066690 0x7f8a08068b50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a2fc00640 1 --2- 172.21.15.46:0/3968730661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a2815cc50 0x7f8a28148570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a2fc00640 1 --2- 172.21.15.46:0/3968730661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a28071100 0x7f8a2815f950 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a2fc00640 1 --2- 172.21.15.46:0/3968730661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a2806b470 0x7f8a280ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a2fc00640 1 -- 172.21.15.46:0/3968730661 >> 172.21.15.46:0/3968730661 conn(0x7f8a280ced50 msgr2=0x7f8a281514f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:55.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a2fc00640 1 -- 172.21.15.46:0/3968730661 shutdown_connections 2024-03-29T08:21:55.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.538+0000 7f8a2fc00640 1 -- 172.21.15.46:0/3968730661 wait complete. 2024-03-29T08:21:55.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:21:55.560 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-29T08:21:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.702+0000 7f1a44e00640 1 Processor -- start 2024-03-29T08:21:55.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.702+0000 7f1a44e00640 1 -- start start 2024-03-29T08:21:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.702+0000 7f1a44e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1a4015a9c0 0x7f1a400cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.702+0000 7f1a44e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1a40099150 0x7f1a400b3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.702+0000 7f1a44e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a400ce290 con 0x7f1a400cc370 2024-03-29T08:21:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.702+0000 7f1a44e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a400ce570 con 0x7f1a4015a9c0 2024-03-29T08:21:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.702+0000 7f1a44e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a400ce850 con 0x7f1a40099150 2024-03-29T08:21:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.702+0000 7f1a3de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1a4015a9c0 0x7f1a400cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.702+0000 7f1a3de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1a4015a9c0 0x7f1a400cbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55122/0 (socket says 172.21.15.46:55122) 2024-03-29T08:21:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.702+0000 7f1a3de00640 1 -- 172.21.15.46:0/2935473426 learned_addr learned my addr 172.21.15.46:0/2935473426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.702+0000 7f1a3e800640 1 --2- 172.21.15.46:0/2935473426 >> v2:172.21.15.73:3300/0 conn(0x7f1a40099150 0x7f1a400b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.702+0000 7f1a3de00640 1 -- 172.21.15.46:0/2935473426 >> v2:172.21.15.73:3300/0 conn(0x7f1a40099150 msgr2=0x7f1a400b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.702+0000 7f1a3de00640 1 --2- 172.21.15.46:0/2935473426 >> v2:172.21.15.73:3300/0 conn(0x7f1a40099150 0x7f1a400b3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.702+0000 7f1a3de00640 1 -- 172.21.15.46:0/2935473426 >> v1:172.21.15.73:6789/0 conn(0x7f1a400cc370 legacy=0x7f1a40098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.702+0000 7f1a3de00640 1 -- 172.21.15.46:0/2935473426 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a400ceb30 con 0x7f1a4015a9c0 2024-03-29T08:21:55.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.706+0000 7f1a3de00640 1 --2- 172.21.15.46:0/2935473426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a4015a9c0 0x7f1a400cbe30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1a4015f640 tx=0x7f1a3001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3c8cebcf9fdca3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:55.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.706+0000 7f1a3d400640 1 -- 172.21.15.46:0/2935473426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a300ad680 con 0x7f1a4015a9c0 2024-03-29T08:21:55.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.706+0000 7f1a3d400640 1 -- 172.21.15.46:0/2935473426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a300ad7e0 con 0x7f1a4015a9c0 2024-03-29T08:21:55.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.706+0000 7f1a3d400640 1 -- 172.21.15.46:0/2935473426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a300adb70 con 0x7f1a4015a9c0 2024-03-29T08:21:55.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.706+0000 7f1a44e00640 1 -- 172.21.15.46:0/2935473426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a4015a9c0 msgr2=0x7f1a400cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.706+0000 7f1a44e00640 1 --2- 172.21.15.46:0/2935473426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a4015a9c0 0x7f1a400cbe30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1a4015f640 tx=0x7f1a3001c490 comp rx=0 tx=0).stop 2024-03-29T08:21:55.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.706+0000 7f1a44e00640 1 -- 172.21.15.46:0/2935473426 shutdown_connections 2024-03-29T08:21:55.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.706+0000 7f1a44e00640 1 --2- 172.21.15.46:0/2935473426 >> v2:172.21.15.73:3300/0 conn(0x7f1a40099150 0x7f1a400b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.706+0000 7f1a44e00640 1 --2- 172.21.15.46:0/2935473426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a4015a9c0 0x7f1a400cbe30 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.706+0000 7f1a44e00640 1 -- 172.21.15.46:0/2935473426 >> 172.21.15.46:0/2935473426 conn(0x7f1a400b21b0 msgr2=0x7f1a400b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:55.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.706+0000 7f1a44e00640 1 -- 172.21.15.46:0/2935473426 shutdown_connections 2024-03-29T08:21:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.706+0000 7f1a44e00640 1 -- 172.21.15.46:0/2935473426 wait complete. 2024-03-29T08:21:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a44e00640 1 Processor -- start 2024-03-29T08:21:55.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a44e00640 1 -- start start 2024-03-29T08:21:55.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a44e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a40099150 0x7f1a400c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a44e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a400cc370 0x7f1a4014cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a44e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1a4015a9c0 0x7f1a4014a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a44e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4014d130 con 0x7f1a400cc370 2024-03-29T08:21:55.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a44e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4014d6b0 con 0x7f1a40099150 2024-03-29T08:21:55.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a44e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4014d3d0 con 0x7f1a4015a9c0 2024-03-29T08:21:55.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a35400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a400cc370 0x7f1a4014cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a35400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a400cc370 0x7f1a4014cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55126/0 (socket says 172.21.15.46:55126) 2024-03-29T08:21:55.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a35400640 1 -- 172.21.15.46:0/674430328 learned_addr learned my addr 172.21.15.46:0/674430328 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:55.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a3de00640 1 --2- 172.21.15.46:0/674430328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a40099150 0x7f1a400c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a3e800640 1 --2- 172.21.15.46:0/674430328 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1a4015a9c0 0x7f1a4014a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a35400640 1 -- 172.21.15.46:0/674430328 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1a4015a9c0 msgr2=0x7f1a4014a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a35400640 1 --2- 172.21.15.46:0/674430328 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1a4015a9c0 0x7f1a4014a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a35400640 1 -- 172.21.15.46:0/674430328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a40099150 msgr2=0x7f1a400c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a35400640 1 --2- 172.21.15.46:0/674430328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a40099150 0x7f1a400c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a35400640 1 -- 172.21.15.46:0/674430328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a300094d0 con 0x7f1a400cc370 2024-03-29T08:21:55.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a35400640 1 --2- 172.21.15.46:0/674430328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a400cc370 0x7f1a4014cbf0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800c500 tx=0x7f1a2800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:55.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a36a00640 1 -- 172.21.15.46:0/674430328 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a28005220 con 0x7f1a400cc370 2024-03-29T08:21:55.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a36a00640 1 -- 172.21.15.46:0/674430328 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a28005380 con 0x7f1a400cc370 2024-03-29T08:21:55.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a36a00640 1 -- 172.21.15.46:0/674430328 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a28007270 con 0x7f1a400cc370 2024-03-29T08:21:55.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a44e00640 1 -- 172.21.15.46:0/674430328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a400cf790 con 0x7f1a400cc370 2024-03-29T08:21:55.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.710+0000 7f1a44e00640 1 -- 172.21.15.46:0/674430328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4006ef80 con 0x7f1a400cc370 2024-03-29T08:21:55.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a36a00640 1 -- 172.21.15.46:0/674430328 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1a280073d0 con 0x7f1a400cc370 2024-03-29T08:21:55.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a36a00640 1 --2- 172.21.15.46:0/674430328 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1a1c066690 0x7f1a1c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a36a00640 1 -- 172.21.15.46:0/674430328 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a280839b0 con 0x7f1a400cc370 2024-03-29T08:21:55.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a48202000 1 Processor -- start 2024-03-29T08:21:55.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a3de00640 1 --2- 172.21.15.46:0/674430328 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1a1c066690 0x7f1a1c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a48202000 1 -- start start 2024-03-29T08:21:55.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a3de00640 1 --2- 172.21.15.46:0/674430328 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1a1c066690 0x7f1a1c068b50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1a300af000 tx=0x7f1a300a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:55.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a48202000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a8a4ae60 0x5572a8a4b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a48202000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a8a4b780 0x5572a89fe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a48202000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a89fee10 0x5572a89ff1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a48202000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a88c5630 con 0x5572a8a4b780 2024-03-29T08:21:55.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a48202000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a8852bd0 con 0x5572a89fee10 2024-03-29T08:21:55.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a48202000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572a88501a0 con 0x5572a8a4ae60 2024-03-29T08:21:55.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a35400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a8a4b780 0x5572a89fe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a35400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a8a4b780 0x5572a89fe8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55136/0 (socket says 172.21.15.46:55136) 2024-03-29T08:21:55.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a35400640 1 -- 172.21.15.46:0/188161983 learned_addr learned my addr 172.21.15.46:0/188161983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:55.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a3de00640 1 --2- 172.21.15.46:0/188161983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a8a4ae60 0x5572a8a4b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a3e800640 1 --2- 172.21.15.46:0/188161983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a89fee10 0x5572a89ff1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a35400640 1 -- 172.21.15.46:0/188161983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a8a4ae60 msgr2=0x5572a8a4b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a35400640 1 --2- 172.21.15.46:0/188161983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a8a4ae60 0x5572a8a4b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a35400640 1 -- 172.21.15.46:0/188161983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a89fee10 msgr2=0x5572a89ff1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a35400640 1 --2- 172.21.15.46:0/188161983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a89fee10 0x5572a89ff1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.714+0000 7f1a35400640 1 -- 172.21.15.46:0/188161983 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572a8547e20 con 0x5572a8a4b780 2024-03-29T08:21:55.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a35400640 1 --2- 172.21.15.46:0/188161983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a8a4b780 0x5572a89fe8d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1a28054990 tx=0x7f1a28054670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115647062948ed2b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:55.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a2ea00640 1 -- 172.21.15.46:0/188161983 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2805cd40 con 0x5572a8a4b780 2024-03-29T08:21:55.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a2ea00640 1 -- 172.21.15.46:0/188161983 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a28052da0 con 0x5572a8a4b780 2024-03-29T08:21:55.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a2ea00640 1 -- 172.21.15.46:0/188161983 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2806c0d0 con 0x5572a8a4b780 2024-03-29T08:21:55.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 -- 172.21.15.46:0/188161983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a8a4b780 msgr2=0x5572a89fe8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 --2- 172.21.15.46:0/188161983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a8a4b780 0x5572a89fe8d0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1a28054990 tx=0x7f1a28054670 comp rx=0 tx=0).stop 2024-03-29T08:21:55.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 -- 172.21.15.46:0/188161983 shutdown_connections 2024-03-29T08:21:55.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 --2- 172.21.15.46:0/188161983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a89fee10 0x5572a89ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 --2- 172.21.15.46:0/188161983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a8a4b780 0x5572a89fe8d0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 --2- 172.21.15.46:0/188161983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a8a4ae60 0x5572a8a4b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 -- 172.21.15.46:0/188161983 >> 172.21.15.46:0/188161983 conn(0x5572a87fb160 msgr2=0x5572a8a29820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:55.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 -- 172.21.15.46:0/188161983 shutdown_connections 2024-03-29T08:21:55.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 -- 172.21.15.46:0/188161983 wait complete. 2024-03-29T08:21:55.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 -- start start 2024-03-29T08:21:55.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 Processor -- start 2024-03-29T08:21:55.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a89fee10 0x5572a8a29e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a8a4ae60 0x5572a8a4bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a8a4b780 0x5572a8a4c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a884ba90 con 0x5572a8a4ae60 2024-03-29T08:21:55.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a87ead40 con 0x5572a8a4b780 2024-03-29T08:21:55.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572a8808f30 con 0x5572a89fee10 2024-03-29T08:21:55.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 10 client.0 _ll_register_callbacks cb 0x5572a88ad810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:21:55.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a35400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a8a4ae60 0x5572a8a4bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a35400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a8a4ae60 0x5572a8a4bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55142/0 (socket says 172.21.15.46:55142) 2024-03-29T08:21:55.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a35400640 1 -- 172.21.15.46:0/978160630 learned_addr learned my addr 172.21.15.46:0/978160630 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:55.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a3de00640 1 --2- 172.21.15.46:0/978160630 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a89fee10 0x5572a8a29e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a3e800640 1 --2- 172.21.15.46:0/978160630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a8a4b780 0x5572a8a4c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a35400640 1 -- 172.21.15.46:0/978160630 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a89fee10 msgr2=0x5572a8a29e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a35400640 1 --2- 172.21.15.46:0/978160630 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a89fee10 0x5572a8a29e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a35400640 1 -- 172.21.15.46:0/978160630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a8a4b780 msgr2=0x5572a8a4c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a35400640 1 --2- 172.21.15.46:0/978160630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a8a4b780 0x5572a8a4c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a35400640 1 -- 172.21.15.46:0/978160630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4006ef80 con 0x5572a8a4ae60 2024-03-29T08:21:55.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a35400640 1 --2- 172.21.15.46:0/978160630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a8a4ae60 0x5572a8a4bb60 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1a28007820 tx=0x7f1a28001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d5d3476eee5bec server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:55.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a23400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:21:55.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a23400640 1 -- 172.21.15.46:0/978160630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2805cd40 con 0x5572a8a4ae60 2024-03-29T08:21:55.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a23400640 1 -- 172.21.15.46:0/978160630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a28051110 con 0x5572a8a4ae60 2024-03-29T08:21:55.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.718+0000 7f1a48202000 1 -- 172.21.15.46:0/978160630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572a884a3e0 con 0x5572a8a4ae60 2024-03-29T08:21:55.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a23400640 1 -- 172.21.15.46:0/978160630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a280512d0 con 0x5572a8a4ae60 2024-03-29T08:21:55.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 10 client.6396 fetch_fsmap learned FSMap version 33 2024-03-29T08:21:55.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 1 -- 172.21.15.46:0/978160630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5572a8810aa0 con 0x5572a8a4ae60 2024-03-29T08:21:55.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a23400640 1 -- 172.21.15.46:0/978160630 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5572a884a3e0 con 0x5572a8a4ae60 2024-03-29T08:21:55.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a23400640 1 -- 172.21.15.46:0/978160630 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1a28019030 con 0x5572a8a4ae60 2024-03-29T08:21:55.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 10 client.6396 fetch_fsmap finished waiting for FSMap version 33 2024-03-29T08:21:55.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 10 client.6396 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-03-29T08:21:55.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 20 client.6396 populate_metadata read hostname 'smithi046' 2024-03-29T08:21:55.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 1 --2- 172.21.15.46:0/978160630 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5572a8a4cab0 0x5572a8a4cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 4 client.6396 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:21:55.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 1 -- 172.21.15.46:0/978160630 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5572a881bd00 con 0x5572a8a4cab0 2024-03-29T08:21:55.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a3de00640 1 --2- 172.21.15.46:0/978160630 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5572a8a4cab0 0x5572a8a4cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a3de00640 1 --2- 172.21.15.46:0/978160630 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5572a8a4cab0 0x5572a8a4cec0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73f15008ffb77cc6 server_cookie=72832f3a3b19de1 in_seq=0 out_seq=0 2024-03-29T08:21:55.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a23400640 10 client.6396 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:55.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a23400640 1 -- 172.21.15.46:0/978160630 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5572a881bd00 con 0x5572a8a4cab0 2024-03-29T08:21:55.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a23400640 10 client.6396 handle_command_reply: tid=0 2024-03-29T08:21:55.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 1 client.6396 shutdown 2024-03-29T08:21:55.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 1 -- 172.21.15.46:0/978160630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a8a4ae60 msgr2=0x5572a8a4bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 1 --2- 172.21.15.46:0/978160630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a8a4ae60 0x5572a8a4bb60 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1a28007820 tx=0x7f1a28001130 comp rx=0 tx=0).stop 2024-03-29T08:21:55.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 1 -- 172.21.15.46:0/978160630 shutdown_connections 2024-03-29T08:21:55.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 1 --2- 172.21.15.46:0/978160630 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5572a8a4cab0 0x5572a8a4cec0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 1 --2- 172.21.15.46:0/978160630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a8a4b780 0x5572a8a4c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 1 --2- 172.21.15.46:0/978160630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a8a4ae60 0x5572a8a4bb60 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 1 --2- 172.21.15.46:0/978160630 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a89fee10 0x5572a8a29e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a23400640 0 client.6396 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:55.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 1 -- 172.21.15.46:0/978160630 >> 172.21.15.46:0/978160630 conn(0x5572a87fb160 msgr2=0x5572a89f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:55.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 1 -- 172.21.15.46:0/978160630 shutdown_connections 2024-03-29T08:21:55.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 1 -- 172.21.15.46:0/978160630 wait complete. 2024-03-29T08:21:55.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.722+0000 7f1a48202000 20 client.6396 trim_cache size 0 max 16384 2024-03-29T08:21:55.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a48202000 1 Processor -- start 2024-03-29T08:21:55.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a48202000 1 -- start start 2024-03-29T08:21:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a48202000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a89fee10 0x5572a8a28c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a48202000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a8a4ae60 0x5572a8a29160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a48202000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a8a4b780 0x5572a8a4ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a48202000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a8508aa0 con 0x5572a89fee10 2024-03-29T08:21:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a48202000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a884e5f0 con 0x5572a8a4ae60 2024-03-29T08:21:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a48202000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572a8a2aeb0 con 0x5572a8a4b780 2024-03-29T08:21:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a3de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a89fee10 0x5572a8a28c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a3de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a89fee10 0x5572a8a28c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55144/0 (socket says 172.21.15.46:55144) 2024-03-29T08:21:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a3de00640 1 -- 172.21.15.46:0/282835831 learned_addr learned my addr 172.21.15.46:0/282835831 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a3de00640 1 -- 172.21.15.46:0/282835831 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a8a4b780 msgr2=0x5572a8a4ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a3de00640 1 --2- 172.21.15.46:0/282835831 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a8a4b780 0x5572a8a4ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a3de00640 1 -- 172.21.15.46:0/282835831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a8a4ae60 msgr2=0x5572a8a29160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a3de00640 1 --2- 172.21.15.46:0/282835831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a8a4ae60 0x5572a8a29160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a3de00640 1 -- 172.21.15.46:0/282835831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5572a87f6df0 con 0x5572a89fee10 2024-03-29T08:21:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a3de00640 1 --2- 172.21.15.46:0/282835831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a89fee10 0x5572a8a28c20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1a300190a0 tx=0x7f1a30009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ceb76ec1a4294 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a2f400640 1 -- 172.21.15.46:0/282835831 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a300cbbc0 con 0x5572a89fee10 2024-03-29T08:21:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a2f400640 1 -- 172.21.15.46:0/282835831 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a300be040 con 0x5572a89fee10 2024-03-29T08:21:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a2f400640 1 -- 172.21.15.46:0/282835831 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a300cb160 con 0x5572a89fee10 2024-03-29T08:21:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a48202000 1 -- 172.21.15.46:0/282835831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a89fee10 msgr2=0x5572a8a28c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a48202000 1 --2- 172.21.15.46:0/282835831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a89fee10 0x5572a8a28c20 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1a300190a0 tx=0x7f1a30009240 comp rx=0 tx=0).stop 2024-03-29T08:21:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a48202000 1 -- 172.21.15.46:0/282835831 shutdown_connections 2024-03-29T08:21:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a48202000 1 --2- 172.21.15.46:0/282835831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a8a4ae60 0x5572a8a29160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a48202000 1 --2- 172.21.15.46:0/282835831 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a8a4b780 0x5572a8a4ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a48202000 1 --2- 172.21.15.46:0/282835831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a89fee10 0x5572a8a28c20 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a48202000 1 -- 172.21.15.46:0/282835831 >> 172.21.15.46:0/282835831 conn(0x5572a87fb160 msgr2=0x5572a8a28510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a48202000 1 -- 172.21.15.46:0/282835831 shutdown_connections 2024-03-29T08:21:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.738+0000 7f1a48202000 1 -- 172.21.15.46:0/282835831 wait complete. 2024-03-29T08:21:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 1 -- start start 2024-03-29T08:21:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 1 Processor -- start 2024-03-29T08:21:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a89fee10 0x5572a8a8d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a8a4ae60 0x5572a8a4f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a8a4b780 0x5572a8a4fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a8a29e70 con 0x5572a89fee10 2024-03-29T08:21:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572a8a611d0 con 0x5572a8a4b780 2024-03-29T08:21:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5572a87f1070 con 0x5572a8a4ae60 2024-03-29T08:21:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 10 client.0 _ll_register_callbacks cb 0x7f1a28050d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:21:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a3de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a89fee10 0x5572a8a8d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a3de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a89fee10 0x5572a8a8d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55150/0 (socket says 172.21.15.46:55150) 2024-03-29T08:21:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a3de00640 1 -- 172.21.15.46:0/1358456196 learned_addr learned my addr 172.21.15.46:0/1358456196 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:21:55.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a35400640 1 --2- 172.21.15.46:0/1358456196 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a8a4ae60 0x5572a8a4f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a3e800640 1 --2- 172.21.15.46:0/1358456196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a8a4b780 0x5572a8a4fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a3de00640 1 -- 172.21.15.46:0/1358456196 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a8a4ae60 msgr2=0x5572a8a4f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a3de00640 1 --2- 172.21.15.46:0/1358456196 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a8a4ae60 0x5572a8a4f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a3de00640 1 -- 172.21.15.46:0/1358456196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a8a4b780 msgr2=0x5572a8a4fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:21:55.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a3de00640 1 --2- 172.21.15.46:0/1358456196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a8a4b780 0x5572a8a4fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a3de00640 1 -- 172.21.15.46:0/1358456196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a400ceb30 con 0x5572a89fee10 2024-03-29T08:21:55.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a3e800640 1 --2- 172.21.15.46:0/1358456196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a8a4b780 0x5572a8a4fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:21:55.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a3de00640 1 --2- 172.21.15.46:0/1358456196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a89fee10 0x5572a8a8d810 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1a300190a0 tx=0x7f1a300046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415ea5ec544503de server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:21:55.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a23400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:21:55.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a23400640 1 -- 172.21.15.46:0/1358456196 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a300cb4f0 con 0x5572a89fee10 2024-03-29T08:21:55.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a23400640 1 -- 172.21.15.46:0/1358456196 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(63..63 src has 1..63) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a300cbbc0 con 0x5572a89fee10 2024-03-29T08:21:55.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 1 -- 172.21.15.46:0/1358456196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572a8808970 con 0x5572a89fee10 2024-03-29T08:21:55.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a23400640 1 -- 172.21.15.46:0/1358456196 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a300cbd80 con 0x5572a89fee10 2024-03-29T08:21:55.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 10 client.6402 fetch_fsmap learned FSMap version 33 2024-03-29T08:21:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 1 -- 172.21.15.46:0/1358456196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5572a87ea960 con 0x5572a89fee10 2024-03-29T08:21:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a23400640 1 -- 172.21.15.46:0/1358456196 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5572a8808970 con 0x5572a89fee10 2024-03-29T08:21:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a23400640 1 -- 172.21.15.46:0/1358456196 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 33) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1a300c3040 con 0x5572a89fee10 2024-03-29T08:21:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 10 client.6402 fetch_fsmap finished waiting for FSMap version 33 2024-03-29T08:21:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 10 client.6402 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-03-29T08:21:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 20 client.6402 populate_metadata read hostname 'smithi046' 2024-03-29T08:21:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 1 --2- 172.21.15.46:0/1358456196 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5572a8a50130 0x5572a8a50510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:21:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 4 client.6402 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:21:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a48202000 1 -- 172.21.15.46:0/1358456196 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5572a887aa20 con 0x5572a8a50130 2024-03-29T08:21:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a35400640 1 --2- 172.21.15.46:0/1358456196 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5572a8a50130 0x5572a8a50510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:21:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a35400640 1 --2- 172.21.15.46:0/1358456196 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5572a8a50130 0x5572a8a50510 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a51b1e869a785ced server_cookie=c5238c27ad02149f in_seq=0 out_seq=0 2024-03-29T08:21:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.742+0000 7f1a23400640 10 client.6402 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a23400640 1 -- 172.21.15.46:0/1358456196 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x5572a887aa20 con 0x5572a8a50130 2024-03-29T08:21:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a23400640 10 client.6402 handle_command_reply: tid=0 2024-03-29T08:21:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a48202000 1 client.6402 shutdown 2024-03-29T08:21:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a48202000 1 -- 172.21.15.46:0/1358456196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a89fee10 msgr2=0x5572a8a8d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a48202000 1 --2- 172.21.15.46:0/1358456196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a89fee10 0x5572a8a8d810 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f1a300190a0 tx=0x7f1a300046d0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a48202000 1 -- 172.21.15.46:0/1358456196 shutdown_connections 2024-03-29T08:21:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a48202000 1 --2- 172.21.15.46:0/1358456196 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x5572a8a50130 0x5572a8a50510 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a48202000 1 --2- 172.21.15.46:0/1358456196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5572a8a4b780 0x5572a8a4fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a48202000 1 --2- 172.21.15.46:0/1358456196 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5572a8a4ae60 0x5572a8a4f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a23400640 0 client.6402 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:21:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a48202000 1 --2- 172.21.15.46:0/1358456196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5572a89fee10 0x5572a8a8d810 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a48202000 1 -- 172.21.15.46:0/1358456196 >> 172.21.15.46:0/1358456196 conn(0x5572a87fb160 msgr2=0x5572a89f94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a48202000 1 -- 172.21.15.46:0/1358456196 shutdown_connections 2024-03-29T08:21:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a48202000 1 -- 172.21.15.46:0/1358456196 wait complete. 2024-03-29T08:21:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a48202000 20 client.6402 trim_cache size 0 max 16384 2024-03-29T08:21:55.755 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:21:55.755 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-29T08:21:55.755 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-29T08:21:55.755 INFO:teuthology.orchestra.run.smithi046.stdout: "534f32e7-216b-4f5d-a059-67ee65cc24df": { 2024-03-29T08:21:55.755 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-29T08:21:55.755 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "534f32e7-216b-4f5d-a059-67ee65cc24df", 2024-03-29T08:21:55.755 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,repair,force" 2024-03-29T08:21:55.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:55.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:21:55.756 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:21:55.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a44e00640 1 -- 172.21.15.46:0/674430328 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1a1c066690 msgr2=0x7f1a1c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a44e00640 1 --2- 172.21.15.46:0/674430328 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1a1c066690 0x7f1a1c068b50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1a300af000 tx=0x7f1a300a7000 comp rx=0 tx=0).stop 2024-03-29T08:21:55.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a44e00640 1 -- 172.21.15.46:0/674430328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a400cc370 msgr2=0x7f1a4014cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:21:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a44e00640 1 --2- 172.21.15.46:0/674430328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a400cc370 0x7f1a4014cbf0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800c500 tx=0x7f1a2800c9c0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a44e00640 1 -- 172.21.15.46:0/674430328 shutdown_connections 2024-03-29T08:21:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a44e00640 1 --2- 172.21.15.46:0/674430328 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1a1c066690 0x7f1a1c068b50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a44e00640 1 --2- 172.21.15.46:0/674430328 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1a4015a9c0 0x7f1a4014a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a44e00640 1 --2- 172.21.15.46:0/674430328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a400cc370 0x7f1a4014cbf0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a44e00640 1 --2- 172.21.15.46:0/674430328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a40099150 0x7f1a400c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:21:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a44e00640 1 -- 172.21.15.46:0/674430328 >> 172.21.15.46:0/674430328 conn(0x7f1a400b21b0 msgr2=0x7f1a400cdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:21:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a44e00640 1 -- 172.21.15.46:0/674430328 shutdown_connections 2024-03-29T08:21:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:21:55.746+0000 7f1a44e00640 1 -- 172.21.15.46:0/674430328 wait complete. 2024-03-29T08:21:55.767 INFO:tasks.cephfs.filesystem:scrub status for tag:534f32e7-216b-4f5d-a059-67ee65cc24df - {'path': '/', 'tag': '534f32e7-216b-4f5d-a059-67ee65cc24df', 'options': 'recursive,repair,force'} 2024-03-29T08:22:08.699 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:22:08.703 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:22:25.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:25.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-03-29T08:22:25.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.918+0000 7fd3c7400640 1 Processor -- start 2024-03-29T08:22:25.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.918+0000 7fd3c7400640 1 -- start start 2024-03-29T08:22:25.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3c7400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd3c00cbea0 0x7fd3c01613a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3c7400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c006c370 con 0x7fd3c006d0a0 2024-03-29T08:22:25.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3c7400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c00ae100 con 0x7fd3c00add20 2024-03-29T08:22:25.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3c7400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c0158260 con 0x7fd3c00cbea0 2024-03-29T08:22:25.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3c5400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd3c006d0a0 0x7fd3c01618e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41452/0 (socket says 172.21.15.46:41452) 2024-03-29T08:22:25.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3c5400640 1 -- 172.21.15.46:0/439880578 learned_addr learned my addr 172.21.15.46:0/439880578 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:25.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3b7e00640 1 --2- 172.21.15.46:0/439880578 >> v2:172.21.15.73:3300/0 conn(0x7fd3c00cbea0 0x7fd3c01613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3bfe00640 1 -- 172.21.15.46:0/439880578 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 292184477 0 0) 0x7fd3c006c370 con 0x7fd3c006d0a0 2024-03-29T08:22:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3bfe00640 1 -- 172.21.15.46:0/439880578 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3a4003530 con 0x7fd3c006d0a0 2024-03-29T08:22:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3bfe00640 1 -- 172.21.15.46:0/439880578 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4266958398 0 0) 0x7fd3c00ae100 con 0x7fd3c00add20 2024-03-29T08:22:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3bfe00640 1 -- 172.21.15.46:0/439880578 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3c006c370 con 0x7fd3c00add20 2024-03-29T08:22:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3bfe00640 1 -- 172.21.15.46:0/439880578 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1229907041 0 0) 0x7fd3a4003530 con 0x7fd3c006d0a0 2024-03-29T08:22:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3bfe00640 1 -- 172.21.15.46:0/439880578 >> v2:172.21.15.73:3300/0 conn(0x7fd3c00cbea0 msgr2=0x7fd3c01613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3bfe00640 1 --2- 172.21.15.46:0/439880578 >> v2:172.21.15.73:3300/0 conn(0x7fd3c00cbea0 0x7fd3c01613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3bfe00640 1 -- 172.21.15.46:0/439880578 >> v1:172.21.15.73:6789/0 conn(0x7fd3c00add20 legacy=0x7fd3c00cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3bfe00640 1 -- 172.21.15.46:0/439880578 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c0162110 con 0x7fd3c006d0a0 2024-03-29T08:22:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3bfe00640 1 -- 172.21.15.46:0/439880578 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fd3b8001a00 con 0x7fd3c006d0a0 2024-03-29T08:22:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3b7e00640 1 --2- 172.21.15.46:0/439880578 >> v2:172.21.15.73:3300/0 conn(0x7fd3c00cbea0 0x7fd3c01613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:22:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3bfe00640 1 -- 172.21.15.46:0/439880578 >> v1:172.21.15.46:6789/0 conn(0x7fd3c006d0a0 legacy=0x7fd3c01618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3bfe00640 1 --2- 172.21.15.46:0/439880578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a4003b10 0x7fd3a4003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3bfe00640 1 --2- 172.21.15.46:0/439880578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3a40047d0 0x7fd3a4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3bfe00640 1 --2- 172.21.15.46:0/439880578 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3a4005190 0x7fd3a400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3bfe00640 1 -- 172.21.15.46:0/439880578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c0158260 con 0x7fd3a40047d0 2024-03-29T08:22:25.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3bfe00640 1 -- 172.21.15.46:0/439880578 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a400c010 con 0x7fd3a4003b10 2024-03-29T08:22:25.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3bfe00640 1 -- 172.21.15.46:0/439880578 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a400c1d0 con 0x7fd3a4005190 2024-03-29T08:22:25.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3b7e00640 1 --2- 172.21.15.46:0/439880578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3a40047d0 0x7fd3a4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3c4a00640 1 --2- 172.21.15.46:0/439880578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a4003b10 0x7fd3a4003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3c5400640 1 --2- 172.21.15.46:0/439880578 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3a4005190 0x7fd3a400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3b7e00640 1 -- 172.21.15.46:0/439880578 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3a4005190 msgr2=0x7fd3a400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:25.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3b7e00640 1 --2- 172.21.15.46:0/439880578 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3a4005190 0x7fd3a400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3b7e00640 1 -- 172.21.15.46:0/439880578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a4003b10 msgr2=0x7fd3a4003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:25.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3b7e00640 1 --2- 172.21.15.46:0/439880578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a4003b10 0x7fd3a4003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3b7e00640 1 -- 172.21.15.46:0/439880578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a4003530 con 0x7fd3a40047d0 2024-03-29T08:22:25.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.922+0000 7fd3c5400640 1 -- 172.21.15.46:0/439880578 reap_dead start 2024-03-29T08:22:25.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.926+0000 7fd3b7e00640 1 --2- 172.21.15.46:0/439880578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3a40047d0 0x7fd3a4004c50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009c80 tx=0x7fd3a801cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f700f75464db8b6d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.926+0000 7fd3bfe00640 1 -- 172.21.15.46:0/439880578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a80a5890 con 0x7fd3a40047d0 2024-03-29T08:22:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.926+0000 7fd3bfe00640 1 -- 172.21.15.46:0/439880578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a80a59f0 con 0x7fd3a40047d0 2024-03-29T08:22:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.926+0000 7fd3bfe00640 1 -- 172.21.15.46:0/439880578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a80a5de0 con 0x7fd3a40047d0 2024-03-29T08:22:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.926+0000 7fd3c7400640 1 -- 172.21.15.46:0/439880578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3a40047d0 msgr2=0x7fd3a4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.926+0000 7fd3c7400640 1 --2- 172.21.15.46:0/439880578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3a40047d0 0x7fd3a4004c50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8009c80 tx=0x7fd3a801cbd0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.926+0000 7fd3c7400640 1 -- 172.21.15.46:0/439880578 shutdown_connections 2024-03-29T08:22:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.926+0000 7fd3c7400640 1 --2- 172.21.15.46:0/439880578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3a40047d0 0x7fd3a4004c50 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.926+0000 7fd3c7400640 1 -- 172.21.15.46:0/439880578 >> 172.21.15.46:0/439880578 conn(0x7fd3c006a390 msgr2=0x7fd3c00cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.926+0000 7fd3c7400640 1 -- 172.21.15.46:0/439880578 shutdown_connections 2024-03-29T08:22:25.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.926+0000 7fd3c7400640 1 -- 172.21.15.46:0/439880578 wait complete. 2024-03-29T08:22:25.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.926+0000 7fd3c7400640 1 Processor -- start 2024-03-29T08:22:25.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c7400640 1 -- start start 2024-03-29T08:22:25.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c7400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a40047d0 0x7fd3c01f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c7400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3c015b2d0 0x7fd3c01f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3c0150cc0 0x7fd3c0151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c7400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c01516b0 con 0x7fd3c0150cc0 2024-03-29T08:22:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c7400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c01f7a60 con 0x7fd3a40047d0 2024-03-29T08:22:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c7400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c0154fe0 con 0x7fd3c015b2d0 2024-03-29T08:22:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3c0150cc0 0x7fd3c0151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3c0150cc0 0x7fd3c0151170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49962/0 (socket says 172.21.15.46:49962) 2024-03-29T08:22:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c5400640 1 -- 172.21.15.46:0/3583059365 learned_addr learned my addr 172.21.15.46:0/3583059365 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c4a00640 1 --2- 172.21.15.46:0/3583059365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a40047d0 0x7fd3c01f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3b7e00640 1 --2- 172.21.15.46:0/3583059365 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3c015b2d0 0x7fd3c01f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c5400640 1 -- 172.21.15.46:0/3583059365 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3c015b2d0 msgr2=0x7fd3c01f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c5400640 1 --2- 172.21.15.46:0/3583059365 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3c015b2d0 0x7fd3c01f69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c5400640 1 -- 172.21.15.46:0/3583059365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a40047d0 msgr2=0x7fd3c01f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:25.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c5400640 1 --2- 172.21.15.46:0/3583059365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a40047d0 0x7fd3c01f3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c5400640 1 -- 172.21.15.46:0/3583059365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c01552c0 con 0x7fd3c0150cc0 2024-03-29T08:22:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c5400640 1 --2- 172.21.15.46:0/3583059365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3c0150cc0 0x7fd3c0151170 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4004750 tx=0x7fd3b800a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3bd600640 1 -- 172.21.15.46:0/3583059365 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8007af0 con 0x7fd3c0150cc0 2024-03-29T08:22:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3bd600640 1 -- 172.21.15.46:0/3583059365 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b80133f0 con 0x7fd3c0150cc0 2024-03-29T08:22:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3bd600640 1 -- 172.21.15.46:0/3583059365 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b80137a0 con 0x7fd3c0150cc0 2024-03-29T08:22:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c7400640 1 -- 172.21.15.46:0/3583059365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c01f43e0 con 0x7fd3c0150cc0 2024-03-29T08:22:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.930+0000 7fd3c7400640 1 -- 172.21.15.46:0/3583059365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c01f71c0 con 0x7fd3c0150cc0 2024-03-29T08:22:25.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3bd600640 1 -- 172.21.15.46:0/3583059365 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd3b8013900 con 0x7fd3c0150cc0 2024-03-29T08:22:25.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3bd600640 1 --2- 172.21.15.46:0/3583059365 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd39c0666c0 0x7fd39c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3bd600640 1 -- 172.21.15.46:0/3583059365 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd3b80848a0 con 0x7fd3c0150cc0 2024-03-29T08:22:25.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3c4a00640 1 --2- 172.21.15.46:0/3583059365 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd39c0666c0 0x7fd39c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3ca7de000 1 Processor -- start 2024-03-29T08:22:25.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3ca7de000 1 -- start start 2024-03-29T08:22:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3c4a00640 1 --2- 172.21.15.46:0/3583059365 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd39c0666c0 0x7fd39c068b80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0005630 tx=0x7fd3b0011330 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3ca7de000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca26e6a70 0x560ca26e6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3ca7de000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca26e7390 0x560ca269a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3ca7de000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca269aa00 0x560ca269ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3ca7de000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca2565630 con 0x560ca269aa00 2024-03-29T08:22:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3ca7de000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca24f2bd0 con 0x560ca26e7390 2024-03-29T08:22:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3ca7de000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ca24f01a0 con 0x560ca26e6a70 2024-03-29T08:22:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3c5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca269aa00 0x560ca269ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3c5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca269aa00 0x560ca269ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49964/0 (socket says 172.21.15.46:49964) 2024-03-29T08:22:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3c5400640 1 -- 172.21.15.46:0/4006952417 learned_addr learned my addr 172.21.15.46:0/4006952417 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3c4a00640 1 --2- 172.21.15.46:0/4006952417 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca26e6a70 0x560ca26e6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3b7e00640 1 --2- 172.21.15.46:0/4006952417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca26e7390 0x560ca269a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3c5400640 1 -- 172.21.15.46:0/4006952417 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca26e6a70 msgr2=0x560ca26e6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3c5400640 1 --2- 172.21.15.46:0/4006952417 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca26e6a70 0x560ca26e6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3c5400640 1 -- 172.21.15.46:0/4006952417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca26e7390 msgr2=0x560ca269a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3c5400640 1 --2- 172.21.15.46:0/4006952417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca26e7390 0x560ca269a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3c5400640 1 -- 172.21.15.46:0/4006952417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ca21e7e20 con 0x560ca269aa00 2024-03-29T08:22:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3c5400640 1 --2- 172.21.15.46:0/4006952417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca269aa00 0x560ca269ade0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8082d30 tx=0x7fd3b8055940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63781377b2ad7551 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3b5600640 1 -- 172.21.15.46:0/4006952417 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b806d070 con 0x560ca269aa00 2024-03-29T08:22:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3b5600640 1 -- 172.21.15.46:0/4006952417 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b806d1d0 con 0x560ca269aa00 2024-03-29T08:22:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.934+0000 7fd3b5600640 1 -- 172.21.15.46:0/4006952417 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8052050 con 0x560ca269aa00 2024-03-29T08:22:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 -- 172.21.15.46:0/4006952417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca269aa00 msgr2=0x560ca269ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/4006952417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca269aa00 0x560ca269ade0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8082d30 tx=0x7fd3b8055940 comp rx=0 tx=0).stop 2024-03-29T08:22:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 -- 172.21.15.46:0/4006952417 shutdown_connections 2024-03-29T08:22:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/4006952417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca26e7390 0x560ca269a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/4006952417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca269aa00 0x560ca269ade0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/4006952417 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca26e6a70 0x560ca26e6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 -- 172.21.15.46:0/4006952417 >> 172.21.15.46:0/4006952417 conn(0x560ca249b160 msgr2=0x560ca26c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:25.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 -- 172.21.15.46:0/4006952417 shutdown_connections 2024-03-29T08:22:25.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 -- 172.21.15.46:0/4006952417 wait complete. 2024-03-29T08:22:25.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 -- start start 2024-03-29T08:22:25.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 Processor -- start 2024-03-29T08:22:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca269aa00 0x560ca26a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca26e6a70 0x560ca26caa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca26e7390 0x560ca26caf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca24eba90 con 0x560ca26e7390 2024-03-29T08:22:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca248ad40 con 0x560ca26e6a70 2024-03-29T08:22:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ca24a8f30 con 0x560ca269aa00 2024-03-29T08:22:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 10 client.0 _ll_register_callbacks cb 0x560ca254d810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:22:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3c5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca26e7390 0x560ca26caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3c5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca26e7390 0x560ca26caf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49976/0 (socket says 172.21.15.46:49976) 2024-03-29T08:22:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3c5400640 1 -- 172.21.15.46:0/96703008 learned_addr learned my addr 172.21.15.46:0/96703008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3c4a00640 1 --2- 172.21.15.46:0/96703008 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca269aa00 0x560ca26a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3b7e00640 1 --2- 172.21.15.46:0/96703008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca26e6a70 0x560ca26caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3c5400640 1 -- 172.21.15.46:0/96703008 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca269aa00 msgr2=0x560ca26a37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3c5400640 1 --2- 172.21.15.46:0/96703008 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca269aa00 0x560ca26a37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3c5400640 1 -- 172.21.15.46:0/96703008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca26e6a70 msgr2=0x560ca26caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3c5400640 1 --2- 172.21.15.46:0/96703008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca26e6a70 0x560ca26caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3c5400640 1 -- 172.21.15.46:0/96703008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c01f71c0 con 0x560ca26e7390 2024-03-29T08:22:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3b7e00640 1 --2- 172.21.15.46:0/96703008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca26e6a70 0x560ca26caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3c5400640 1 --2- 172.21.15.46:0/96703008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca26e7390 0x560ca26caf90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8082de0 tx=0x7fd3b8058070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d908b330b0394e47 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ae000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:22:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ae000640 1 -- 172.21.15.46:0/96703008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b806d050 con 0x560ca26e7390 2024-03-29T08:22:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ae000640 1 -- 172.21.15.46:0/96703008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd3b8053da0 con 0x560ca26e7390 2024-03-29T08:22:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 -- 172.21.15.46:0/96703008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ca24ea3e0 con 0x560ca26e7390 2024-03-29T08:22:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ae000640 1 -- 172.21.15.46:0/96703008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3b805d9f0 con 0x560ca26e7390 2024-03-29T08:22:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 10 client.6414 fetch_fsmap learned FSMap version 34 2024-03-29T08:22:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ca7de000 1 -- 172.21.15.46:0/96703008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x560ca24b0aa0 con 0x560ca26e7390 2024-03-29T08:22:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.938+0000 7fd3ae000640 1 -- 172.21.15.46:0/96703008 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560ca24ea3e0 con 0x560ca26e7390 2024-03-29T08:22:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ae000640 1 -- 172.21.15.46:0/96703008 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd3b8012070 con 0x560ca26e7390 2024-03-29T08:22:25.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 10 client.6414 fetch_fsmap finished waiting for FSMap version 34 2024-03-29T08:22:25.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 10 client.6414 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-03-29T08:22:25.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 20 client.6414 populate_metadata read hostname 'smithi046' 2024-03-29T08:22:25.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/96703008 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x560ca26ede00 0x560ca26ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 4 client.6414 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:22:25.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 1 -- 172.21.15.46:0/96703008 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ca24bbd00 con 0x560ca26ede00 2024-03-29T08:22:25.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3c4a00640 1 --2- 172.21.15.46:0/96703008 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x560ca26ede00 0x560ca26ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ae000640 10 client.6414 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:22:25.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3c4a00640 1 --2- 172.21.15.46:0/96703008 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x560ca26ede00 0x560ca26ee1e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c870b8c31f248f75 server_cookie=54ccf2cb637daee8 in_seq=0 out_seq=0 2024-03-29T08:22:25.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ae000640 1 -- 172.21.15.46:0/96703008 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560ca24bbd00 con 0x560ca26ede00 2024-03-29T08:22:25.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ae000640 10 client.6414 handle_command_reply: tid=0 2024-03-29T08:22:25.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 1 client.6414 shutdown 2024-03-29T08:22:25.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 1 -- 172.21.15.46:0/96703008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca26e7390 msgr2=0x560ca26caf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:25.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/96703008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca26e7390 0x560ca26caf90 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8082de0 tx=0x7fd3b8058070 comp rx=0 tx=0).stop 2024-03-29T08:22:25.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 1 -- 172.21.15.46:0/96703008 shutdown_connections 2024-03-29T08:22:25.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/96703008 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x560ca26ede00 0x560ca26ee1e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/96703008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca26e6a70 0x560ca26caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/96703008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca26e7390 0x560ca26caf90 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ae000640 0 client.6414 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:22:25.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/96703008 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca269aa00 0x560ca26a37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 1 -- 172.21.15.46:0/96703008 >> 172.21.15.46:0/96703008 conn(0x560ca249b160 msgr2=0x560ca2699bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:25.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 1 -- 172.21.15.46:0/96703008 shutdown_connections 2024-03-29T08:22:25.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 1 -- 172.21.15.46:0/96703008 wait complete. 2024-03-29T08:22:25.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.942+0000 7fd3ca7de000 20 client.6414 trim_cache size 0 max 16384 2024-03-29T08:22:25.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3ca7de000 1 Processor -- start 2024-03-29T08:22:25.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3ca7de000 1 -- start start 2024-03-29T08:22:25.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3ca7de000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca269aa00 0x560ca2699c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3ca7de000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca26e6a70 0x560ca269a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3ca7de000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca26e7390 0x560ca269ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3ca7de000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca21a8aa0 con 0x560ca269aa00 2024-03-29T08:22:25.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3ca7de000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca2481a90 con 0x560ca26e7390 2024-03-29T08:22:25.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3ca7de000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ca24ee5f0 con 0x560ca26e6a70 2024-03-29T08:22:25.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3c4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca269aa00 0x560ca2699c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3c4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca269aa00 0x560ca2699c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49982/0 (socket says 172.21.15.46:49982) 2024-03-29T08:22:25.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3c4a00640 1 -- 172.21.15.46:0/12580470 learned_addr learned my addr 172.21.15.46:0/12580470 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:25.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3b7e00640 1 --2- 172.21.15.46:0/12580470 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca26e6a70 0x560ca269a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3c5400640 1 --2- 172.21.15.46:0/12580470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca26e7390 0x560ca269ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3c4a00640 1 -- 172.21.15.46:0/12580470 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca26e6a70 msgr2=0x560ca269a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:25.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3c4a00640 1 --2- 172.21.15.46:0/12580470 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca26e6a70 0x560ca269a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3c4a00640 1 -- 172.21.15.46:0/12580470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca26e7390 msgr2=0x560ca269ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:25.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3c4a00640 1 --2- 172.21.15.46:0/12580470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca26e7390 0x560ca269ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3c4a00640 1 -- 172.21.15.46:0/12580470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ca2496df0 con 0x560ca269aa00 2024-03-29T08:22:25.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3c4a00640 1 --2- 172.21.15.46:0/12580470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca269aa00 0x560ca2699c30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd3b00138d0 tx=0x7fd3b000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fe108c9de968cb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:25.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3b6000640 1 -- 172.21.15.46:0/12580470 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b002b070 con 0x560ca269aa00 2024-03-29T08:22:25.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3b6000640 1 -- 172.21.15.46:0/12580470 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b0013e20 con 0x560ca269aa00 2024-03-29T08:22:25.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3b6000640 1 -- 172.21.15.46:0/12580470 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b001b040 con 0x560ca269aa00 2024-03-29T08:22:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3ca7de000 1 -- 172.21.15.46:0/12580470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca269aa00 msgr2=0x560ca2699c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/12580470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca269aa00 0x560ca2699c30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd3b00138d0 tx=0x7fd3b000c000 comp rx=0 tx=0).stop 2024-03-29T08:22:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3ca7de000 1 -- 172.21.15.46:0/12580470 shutdown_connections 2024-03-29T08:22:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/12580470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca26e7390 0x560ca269ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/12580470 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca26e6a70 0x560ca269a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/12580470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca269aa00 0x560ca2699c30 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3ca7de000 1 -- 172.21.15.46:0/12580470 >> 172.21.15.46:0/12580470 conn(0x560ca249b160 msgr2=0x560ca26a2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3ca7de000 1 -- 172.21.15.46:0/12580470 shutdown_connections 2024-03-29T08:22:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.958+0000 7fd3ca7de000 1 -- 172.21.15.46:0/12580470 wait complete. 2024-03-29T08:22:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 1 -- start start 2024-03-29T08:22:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 1 Processor -- start 2024-03-29T08:22:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca269aa00 0x560ca269dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca26e6a70 0x560ca269e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca26e7390 0x560ca2742b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca24ed020 con 0x560ca26e7390 2024-03-29T08:22:25.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ca24f3e60 con 0x560ca269aa00 2024-03-29T08:22:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560ca24bb0b0 con 0x560ca26e6a70 2024-03-29T08:22:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 10 client.0 _ll_register_callbacks cb 0x7fd3b8051c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:22:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3c5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca26e7390 0x560ca2742b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3c5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca26e7390 0x560ca2742b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49986/0 (socket says 172.21.15.46:49986) 2024-03-29T08:22:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3c5400640 1 -- 172.21.15.46:0/3241069165 learned_addr learned my addr 172.21.15.46:0/3241069165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3c4a00640 1 --2- 172.21.15.46:0/3241069165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca269aa00 0x560ca269dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3b7e00640 1 --2- 172.21.15.46:0/3241069165 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca26e6a70 0x560ca269e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3c5400640 1 -- 172.21.15.46:0/3241069165 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca26e6a70 msgr2=0x560ca269e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3c5400640 1 --2- 172.21.15.46:0/3241069165 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca26e6a70 0x560ca269e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3c5400640 1 -- 172.21.15.46:0/3241069165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca269aa00 msgr2=0x560ca269dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3c5400640 1 --2- 172.21.15.46:0/3241069165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca269aa00 0x560ca269dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3c5400640 1 -- 172.21.15.46:0/3241069165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c006c370 con 0x560ca26e7390 2024-03-29T08:22:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3b7e00640 1 --2- 172.21.15.46:0/3241069165 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca26e6a70 0x560ca269e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3c5400640 1 --2- 172.21.15.46:0/3241069165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca26e7390 0x560ca2742b90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8055640 tx=0x7fd3b8001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23184c1f93fe9fb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:25.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ae000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:22:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ae000640 1 -- 172.21.15.46:0/3241069165 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b806d050 con 0x560ca26e7390 2024-03-29T08:22:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ae000640 1 -- 172.21.15.46:0/3241069165 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd3b8053da0 con 0x560ca26e7390 2024-03-29T08:22:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 1 -- 172.21.15.46:0/3241069165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ca24a8970 con 0x560ca26e7390 2024-03-29T08:22:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ae000640 1 -- 172.21.15.46:0/3241069165 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3b8059140 con 0x560ca26e7390 2024-03-29T08:22:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 10 client.6420 fetch_fsmap learned FSMap version 34 2024-03-29T08:22:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 1 -- 172.21.15.46:0/3241069165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x560ca248a960 con 0x560ca26e7390 2024-03-29T08:22:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ae000640 1 -- 172.21.15.46:0/3241069165 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560ca24a8970 con 0x560ca26e7390 2024-03-29T08:22:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ae000640 1 -- 172.21.15.46:0/3241069165 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd3b807a020 con 0x560ca26e7390 2024-03-29T08:22:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 10 client.6420 fetch_fsmap finished waiting for FSMap version 34 2024-03-29T08:22:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 10 client.6420 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-03-29T08:22:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 20 client.6420 populate_metadata read hostname 'smithi046' 2024-03-29T08:22:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/3241069165 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x560ca27430d0 0x560ca27434e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:25.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 4 client.6420 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:22:25.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 1 -- 172.21.15.46:0/3241069165 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560ca251aa20 con 0x560ca27430d0 2024-03-29T08:22:25.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3c4a00640 1 --2- 172.21.15.46:0/3241069165 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x560ca27430d0 0x560ca27434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:25.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3c4a00640 1 --2- 172.21.15.46:0/3241069165 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x560ca27430d0 0x560ca27434e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1b8e5557cd0d54c server_cookie=b5d5c7ccf50c6ff7 in_seq=0 out_seq=0 2024-03-29T08:22:25.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ae000640 10 client.6420 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:22:25.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ae000640 1 -- 172.21.15.46:0/3241069165 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560ca251aa20 con 0x560ca27430d0 2024-03-29T08:22:25.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ae000640 10 client.6420 handle_command_reply: tid=0 2024-03-29T08:22:25.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.962+0000 7fd3ca7de000 1 client.6420 shutdown 2024-03-29T08:22:25.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3ca7de000 1 -- 172.21.15.46:0/3241069165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca26e7390 msgr2=0x560ca2742b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:25.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/3241069165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca26e7390 0x560ca2742b90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8055640 tx=0x7fd3b8001ac0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3ca7de000 1 -- 172.21.15.46:0/3241069165 shutdown_connections 2024-03-29T08:22:25.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/3241069165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560ca26e7390 0x560ca2742b90 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/3241069165 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560ca26e6a70 0x560ca269e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/3241069165 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x560ca27430d0 0x560ca27434e0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3ca7de000 1 --2- 172.21.15.46:0/3241069165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ca269aa00 0x560ca269dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3ae000640 0 client.6420 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:22:25.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3ca7de000 1 -- 172.21.15.46:0/3241069165 >> 172.21.15.46:0/3241069165 conn(0x560ca249b160 msgr2=0x560ca2698c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:25.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3ca7de000 1 -- 172.21.15.46:0/3241069165 shutdown_connections 2024-03-29T08:22:25.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3ca7de000 1 -- 172.21.15.46:0/3241069165 wait complete. 2024-03-29T08:22:25.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3ca7de000 20 client.6420 trim_cache size 0 max 16384 2024-03-29T08:22:25.972 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:22:25.972 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-29T08:22:25.973 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-29T08:22:25.973 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:22:25.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3c7400640 1 -- 172.21.15.46:0/3583059365 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd39c0666c0 msgr2=0x7fd39c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:25.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3c7400640 1 --2- 172.21.15.46:0/3583059365 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd39c0666c0 0x7fd39c068b80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd3b0005630 tx=0x7fd3b0011330 comp rx=0 tx=0).stop 2024-03-29T08:22:25.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3c7400640 1 -- 172.21.15.46:0/3583059365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3c0150cc0 msgr2=0x7fd3c0151170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:25.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3c7400640 1 --2- 172.21.15.46:0/3583059365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3c0150cc0 0x7fd3c0151170 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4004750 tx=0x7fd3b800a6a0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3c7400640 1 -- 172.21.15.46:0/3583059365 shutdown_connections 2024-03-29T08:22:25.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3c7400640 1 --2- 172.21.15.46:0/3583059365 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd39c0666c0 0x7fd39c068b80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3c7400640 1 --2- 172.21.15.46:0/3583059365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3c0150cc0 0x7fd3c0151170 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3c7400640 1 --2- 172.21.15.46:0/3583059365 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3c015b2d0 0x7fd3c01f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3c7400640 1 --2- 172.21.15.46:0/3583059365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a40047d0 0x7fd3c01f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:25.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3c7400640 1 -- 172.21.15.46:0/3583059365 >> 172.21.15.46:0/3583059365 conn(0x7fd3c006a390 msgr2=0x7fd3c00ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:25.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3c7400640 1 -- 172.21.15.46:0/3583059365 shutdown_connections 2024-03-29T08:22:25.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:25.966+0000 7fd3c7400640 1 -- 172.21.15.46:0/3583059365 wait complete. 2024-03-29T08:22:25.987 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-29T08:22:25.988 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:22:26.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd9600640 1 Processor -- start 2024-03-29T08:22:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd9600640 1 -- start start 2024-03-29T08:22:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd9600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2fd40adaf0 0x7f2fd41611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd9600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd415d870 con 0x7f2fd4153d50 2024-03-29T08:22:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd9600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd415db50 con 0x7f2fd4153970 2024-03-29T08:22:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd9600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd415de30 con 0x7f2fd40adaf0 2024-03-29T08:22:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fca000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2fd4153d50 0x7f2fd40ad3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41460/0 (socket says 172.21.15.46:41460) 2024-03-29T08:22:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fca000640 1 -- 172.21.15.46:0/3333379784 learned_addr learned my addr 172.21.15.46:0/3333379784 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd3400640 1 --2- 172.21.15.46:0/3333379784 >> v2:172.21.15.73:3300/0 conn(0x7f2fd40adaf0 0x7f2fd41611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd2000640 1 -- 172.21.15.46:0/3333379784 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769925414 0 0) 0x7f2fd415d870 con 0x7f2fd4153d50 2024-03-29T08:22:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd2000640 1 -- 172.21.15.46:0/3333379784 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fb8003530 con 0x7f2fd4153d50 2024-03-29T08:22:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd2000640 1 -- 172.21.15.46:0/3333379784 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3003673474 0 0) 0x7f2fb8003530 con 0x7f2fd4153d50 2024-03-29T08:22:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd2000640 1 -- 172.21.15.46:0/3333379784 >> v2:172.21.15.73:3300/0 conn(0x7f2fd40adaf0 msgr2=0x7f2fd41611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd2000640 1 --2- 172.21.15.46:0/3333379784 >> v2:172.21.15.73:3300/0 conn(0x7f2fd40adaf0 0x7f2fd41611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd2000640 1 -- 172.21.15.46:0/3333379784 >> v1:172.21.15.73:6789/0 conn(0x7f2fd4153970 legacy=0x7f2fd415cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd2000640 1 -- 172.21.15.46:0/3333379784 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd40abea0 con 0x7f2fd4153d50 2024-03-29T08:22:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd2000640 1 -- 172.21.15.46:0/3333379784 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f2fbc001680 con 0x7f2fd4153d50 2024-03-29T08:22:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd2000640 1 -- 172.21.15.46:0/3333379784 >> v1:172.21.15.46:6789/0 conn(0x7f2fd4153d50 legacy=0x7f2fd40ad3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd2000640 1 --2- 172.21.15.46:0/3333379784 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2fb8003980 0x7f2fb8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd2000640 1 --2- 172.21.15.46:0/3333379784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fb80047d0 0x7f2fb8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd2000640 1 --2- 172.21.15.46:0/3333379784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fb8005190 0x7f2fb800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd2000640 1 -- 172.21.15.46:0/3333379784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd415de30 con 0x7f2fb80047d0 2024-03-29T08:22:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd2000640 1 -- 172.21.15.46:0/3333379784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb800c130 con 0x7f2fb8005190 2024-03-29T08:22:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd2000640 1 -- 172.21.15.46:0/3333379784 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb800c2f0 con 0x7f2fb8003980 2024-03-29T08:22:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fca000640 1 --2- 172.21.15.46:0/3333379784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fb80047d0 0x7f2fb8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd2a00640 1 --2- 172.21.15.46:0/3333379784 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2fb8003980 0x7f2fb8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.126+0000 7f2fd3400640 1 --2- 172.21.15.46:0/3333379784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fb8005190 0x7f2fb800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.130+0000 7f2fca000640 1 -- 172.21.15.46:0/3333379784 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2fb8003980 msgr2=0x7f2fb8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.130+0000 7f2fca000640 1 --2- 172.21.15.46:0/3333379784 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2fb8003980 0x7f2fb8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.130+0000 7f2fca000640 1 -- 172.21.15.46:0/3333379784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fb8005190 msgr2=0x7f2fb800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:26.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.130+0000 7f2fca000640 1 --2- 172.21.15.46:0/3333379784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fb8005190 0x7f2fb800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.130+0000 7f2fca000640 1 -- 172.21.15.46:0/3333379784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb8003530 con 0x7f2fb80047d0 2024-03-29T08:22:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.130+0000 7f2fd3400640 1 -- 172.21.15.46:0/3333379784 reap_dead start 2024-03-29T08:22:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.130+0000 7f2fca000640 1 --2- 172.21.15.46:0/3333379784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fb80047d0 0x7f2fb8004c50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00c240 tx=0x7f2fbc01cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a51b0c7e91f170 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:26.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.130+0000 7f2fd2000640 1 -- 172.21.15.46:0/3333379784 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc005ae0 con 0x7f2fb80047d0 2024-03-29T08:22:26.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.130+0000 7f2fd2000640 1 -- 172.21.15.46:0/3333379784 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fbc005c40 con 0x7f2fb80047d0 2024-03-29T08:22:26.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.130+0000 7f2fd2000640 1 -- 172.21.15.46:0/3333379784 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc0072b0 con 0x7f2fb80047d0 2024-03-29T08:22:26.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.130+0000 7f2fd9600640 1 -- 172.21.15.46:0/3333379784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fb80047d0 msgr2=0x7f2fb8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.130+0000 7f2fd9600640 1 --2- 172.21.15.46:0/3333379784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fb80047d0 0x7f2fb8004c50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc00c240 tx=0x7f2fbc01cbe0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.130+0000 7f2fd9600640 1 -- 172.21.15.46:0/3333379784 shutdown_connections 2024-03-29T08:22:26.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.130+0000 7f2fd9600640 1 --2- 172.21.15.46:0/3333379784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fb80047d0 0x7f2fb8004c50 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.130+0000 7f2fd9600640 1 -- 172.21.15.46:0/3333379784 >> 172.21.15.46:0/3333379784 conn(0x7f2fd4066110 msgr2=0x7f2fd4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:26.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.130+0000 7f2fd9600640 1 -- 172.21.15.46:0/3333379784 shutdown_connections 2024-03-29T08:22:26.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd9600640 1 -- 172.21.15.46:0/3333379784 wait complete. 2024-03-29T08:22:26.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd9600640 1 Processor -- start 2024-03-29T08:22:26.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd9600640 1 -- start start 2024-03-29T08:22:26.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd9600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fb80047d0 0x7f2fd413e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd9600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2fd4147fe0 0x7f2fd41f9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd4155e70 0x7f2fd40ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd9600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd41f9210 con 0x7f2fd4155e70 2024-03-29T08:22:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd9600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd41567e0 con 0x7f2fb80047d0 2024-03-29T08:22:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd9600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd4156ac0 con 0x7f2fd4147fe0 2024-03-29T08:22:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fb80047d0 0x7f2fd413e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd4155e70 0x7f2fd40ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd4155e70 0x7f2fd40ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50010/0 (socket says 172.21.15.46:50010) 2024-03-29T08:22:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd3400640 1 -- 172.21.15.46:0/855757227 learned_addr learned my addr 172.21.15.46:0/855757227 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fca000640 1 --2- 172.21.15.46:0/855757227 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2fd4147fe0 0x7f2fd41f9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd3400640 1 -- 172.21.15.46:0/855757227 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2fd4147fe0 msgr2=0x7f2fd41f9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd3400640 1 --2- 172.21.15.46:0/855757227 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2fd4147fe0 0x7f2fd41f9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd3400640 1 -- 172.21.15.46:0/855757227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fb80047d0 msgr2=0x7f2fd413e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd3400640 1 --2- 172.21.15.46:0/855757227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fb80047d0 0x7f2fd413e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd3400640 1 -- 172.21.15.46:0/855757227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd41f9420 con 0x7f2fd4155e70 2024-03-29T08:22:26.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd2a00640 1 --2- 172.21.15.46:0/855757227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fb80047d0 0x7f2fd413e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:22:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fd3400640 1 --2- 172.21.15.46:0/855757227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd4155e70 0x7f2fd40ccdc0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2fd406e5a0 tx=0x7f2fcc00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.134+0000 7f2fcb400640 1 -- 172.21.15.46:0/855757227 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc00e040 con 0x7f2fd4155e70 2024-03-29T08:22:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.138+0000 7f2fcb400640 1 -- 172.21.15.46:0/855757227 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc00f040 con 0x7f2fd4155e70 2024-03-29T08:22:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.138+0000 7f2fcb400640 1 -- 172.21.15.46:0/855757227 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc015430 con 0x7f2fd4155e70 2024-03-29T08:22:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.138+0000 7f2fd9600640 1 -- 172.21.15.46:0/855757227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd41f9700 con 0x7f2fd4155e70 2024-03-29T08:22:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.138+0000 7f2fd9600640 1 -- 172.21.15.46:0/855757227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd41564f0 con 0x7f2fd4155e70 2024-03-29T08:22:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.138+0000 7f2fd9600640 1 -- 172.21.15.46:0/855757227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd4156c80 con 0x7f2fd4155e70 2024-03-29T08:22:26.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.138+0000 7f2fcb400640 1 -- 172.21.15.46:0/855757227 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2fcc015590 con 0x7f2fd4155e70 2024-03-29T08:22:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.138+0000 7f2fcb400640 1 --2- 172.21.15.46:0/855757227 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2fb00666c0 0x7f2fb0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.138+0000 7f2fcb400640 1 -- 172.21.15.46:0/855757227 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2fcc084950 con 0x7f2fd4155e70 2024-03-29T08:22:26.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.138+0000 7f2fd2a00640 1 --2- 172.21.15.46:0/855757227 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2fb00666c0 0x7f2fb0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.138+0000 7f2fd2a00640 1 --2- 172.21.15.46:0/855757227 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2fb00666c0 0x7f2fb0068b80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4067340 tx=0x7f2fc4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.142+0000 7f2fcb400640 1 -- 172.21.15.46:0/855757227 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2fcc051d20 con 0x7f2fd4155e70 2024-03-29T08:22:26.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.338+0000 7f2fd9600640 1 -- 172.21.15.46:0/855757227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fd4066c90 con 0x7f2fd4155e70 2024-03-29T08:22:26.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.338+0000 7f2fcb400640 1 -- 172.21.15.46:0/855757227 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4729 (secure 0 0 0) 0x7f2fcc055340 con 0x7f2fd4155e70 2024-03-29T08:22:26.343 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:26.344 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2187826146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2187826146},{"type":"v1","addr":"172.21.15.73:6836","nonce":2187826146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/73382477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":73382477},{"type":"v1","addr":"172.21.15.73:6837","nonce":73382477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2874958170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2874958170},{"type":"v1","addr":"172.21.15.46:6834","nonce":2874958170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:21:10.337267+0000","modified":"2024-03-29T08:22:02.946081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6051},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6051":{"gid":6051,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":16,"addr":"172.21.15.46:6835/3255474176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3255474176},{"type":"v1","addr":"172.21.15.46:6835","nonce":3255474176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-29T08:22:26.344 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2024-03-29T08:22:26.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.342+0000 7f2fd9600640 1 -- 172.21.15.46:0/855757227 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2fb00666c0 msgr2=0x7f2fb0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.342+0000 7f2fd9600640 1 --2- 172.21.15.46:0/855757227 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2fb00666c0 0x7f2fb0068b80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2fd4067340 tx=0x7f2fc4009040 comp rx=0 tx=0).stop 2024-03-29T08:22:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.342+0000 7f2fd9600640 1 -- 172.21.15.46:0/855757227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd4155e70 msgr2=0x7f2fd40ccdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.342+0000 7f2fd9600640 1 --2- 172.21.15.46:0/855757227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd4155e70 0x7f2fd40ccdc0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f2fd406e5a0 tx=0x7f2fcc00f580 comp rx=0 tx=0).stop 2024-03-29T08:22:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.342+0000 7f2fd9600640 1 -- 172.21.15.46:0/855757227 shutdown_connections 2024-03-29T08:22:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.342+0000 7f2fd9600640 1 --2- 172.21.15.46:0/855757227 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2fb00666c0 0x7f2fb0068b80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.342+0000 7f2fd9600640 1 --2- 172.21.15.46:0/855757227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2fd4155e70 0x7f2fd40ccdc0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.342+0000 7f2fd9600640 1 --2- 172.21.15.46:0/855757227 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2fd4147fe0 0x7f2fd41f9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.342+0000 7f2fd9600640 1 --2- 172.21.15.46:0/855757227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2fb80047d0 0x7f2fd413e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.342+0000 7f2fd9600640 1 -- 172.21.15.46:0/855757227 >> 172.21.15.46:0/855757227 conn(0x7f2fd4066110 msgr2=0x7f2fd4160950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.342+0000 7f2fd9600640 1 -- 172.21.15.46:0/855757227 shutdown_connections 2024-03-29T08:22:26.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.342+0000 7f2fd9600640 1 -- 172.21.15.46:0/855757227 wait complete. 2024-03-29T08:22:26.364 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-03-29T08:22:26.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.502+0000 7fb33c400640 1 Processor -- start 2024-03-29T08:22:26.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.502+0000 7fb33c400640 1 -- start start 2024-03-29T08:22:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.506+0000 7fb33c400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb33415dbb0 0x7fb33406ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.506+0000 7fb33c400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb3340b2440 0x7fb33406b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.506+0000 7fb33c400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb334157490 con 0x7fb3340b3f20 2024-03-29T08:22:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.506+0000 7fb33c400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3340b2c90 con 0x7fb3340b2440 2024-03-29T08:22:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.506+0000 7fb33c400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3340b29b0 con 0x7fb33415dbb0 2024-03-29T08:22:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.506+0000 7fb339000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb3340b3f20 0x7fb3340b1d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47962/0 (socket says 172.21.15.46:47962) 2024-03-29T08:22:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.506+0000 7fb339000640 1 -- 172.21.15.46:0/3319452114 learned_addr learned my addr 172.21.15.46:0/3319452114 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.506+0000 7fb33a400640 1 --2- 172.21.15.46:0/3319452114 >> v2:172.21.15.46:3300/0 conn(0x7fb3340b2440 0x7fb33406b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.506+0000 7fb339a00640 1 --2- 172.21.15.46:0/3319452114 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb33415dbb0 0x7fb33406ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.506+0000 7fb333e00640 1 -- 172.21.15.46:0/3319452114 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1847497740 0 0) 0x7fb334157490 con 0x7fb3340b3f20 2024-03-29T08:22:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.506+0000 7fb333e00640 1 -- 172.21.15.46:0/3319452114 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb31c003530 con 0x7fb3340b3f20 2024-03-29T08:22:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.506+0000 7fb33a400640 1 -- 172.21.15.46:0/3319452114 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb33415dbb0 msgr2=0x7fb33406ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.506+0000 7fb33a400640 1 --2- 172.21.15.46:0/3319452114 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb33415dbb0 0x7fb33406ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.506+0000 7fb33a400640 1 -- 172.21.15.46:0/3319452114 >> v1:172.21.15.46:6789/0 conn(0x7fb3340b3f20 legacy=0x7fb3340b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.506+0000 7fb33a400640 1 -- 172.21.15.46:0/3319452114 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3340b3440 con 0x7fb3340b2440 2024-03-29T08:22:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb33a400640 1 --2- 172.21.15.46:0/3319452114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3340b2440 0x7fb33406b2e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009d90 tx=0x7fb32c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6acecd14b2f7839 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb333e00640 1 -- 172.21.15.46:0/3319452114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c0a59a0 con 0x7fb3340b2440 2024-03-29T08:22:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb333e00640 1 -- 172.21.15.46:0/3319452114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c0a5b00 con 0x7fb3340b2440 2024-03-29T08:22:26.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb333e00640 1 -- 172.21.15.46:0/3319452114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c0a5e90 con 0x7fb3340b2440 2024-03-29T08:22:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb33c400640 1 -- 172.21.15.46:0/3319452114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3340b2440 msgr2=0x7fb33406b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb33c400640 1 --2- 172.21.15.46:0/3319452114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3340b2440 0x7fb33406b2e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009d90 tx=0x7fb32c01c8e0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb33c400640 1 -- 172.21.15.46:0/3319452114 shutdown_connections 2024-03-29T08:22:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb33c400640 1 --2- 172.21.15.46:0/3319452114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3340b2440 0x7fb33406b2e0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb33c400640 1 --2- 172.21.15.46:0/3319452114 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb33415dbb0 0x7fb33406ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb33c400640 1 -- 172.21.15.46:0/3319452114 >> 172.21.15.46:0/3319452114 conn(0x7fb334070ad0 msgr2=0x7fb334070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb33c400640 1 -- 172.21.15.46:0/3319452114 shutdown_connections 2024-03-29T08:22:26.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb33c400640 1 -- 172.21.15.46:0/3319452114 wait complete. 2024-03-29T08:22:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb33c400640 1 Processor -- start 2024-03-29T08:22:26.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb33c400640 1 -- start start 2024-03-29T08:22:26.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb33c400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb3340b2440 0x7fb33414d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb33c400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3340b3f20 0x7fb334155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb33c400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb33415dbb0 0x7fb334156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb33c400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb334151bb0 con 0x7fb3340b3f20 2024-03-29T08:22:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb33c400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb334152430 con 0x7fb33415dbb0 2024-03-29T08:22:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.510+0000 7fb33c400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb334151e40 con 0x7fb3340b2440 2024-03-29T08:22:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.514+0000 7fb339000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3340b3f20 0x7fb334155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.514+0000 7fb339a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb3340b2440 0x7fb33414d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.514+0000 7fb339000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3340b3f20 0x7fb334155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42000/0 (socket says 172.21.15.46:42000) 2024-03-29T08:22:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.514+0000 7fb339000640 1 -- 172.21.15.46:0/2191011024 learned_addr learned my addr 172.21.15.46:0/2191011024 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.514+0000 7fb33a400640 1 --2- 172.21.15.46:0/2191011024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb33415dbb0 0x7fb334156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.514+0000 7fb339000640 1 -- 172.21.15.46:0/2191011024 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb3340b2440 msgr2=0x7fb33414d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.514+0000 7fb339000640 1 --2- 172.21.15.46:0/2191011024 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb3340b2440 0x7fb33414d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.514+0000 7fb339000640 1 -- 172.21.15.46:0/2191011024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb33415dbb0 msgr2=0x7fb334156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.514+0000 7fb339000640 1 --2- 172.21.15.46:0/2191011024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb33415dbb0 0x7fb334156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.514+0000 7fb339000640 1 -- 172.21.15.46:0/2191011024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c009a70 con 0x7fb3340b3f20 2024-03-29T08:22:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.514+0000 7fb339000640 1 --2- 172.21.15.46:0/2191011024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3340b3f20 0x7fb334155b20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb32000c450 tx=0x7fb32000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:26.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.514+0000 7fb331600640 1 -- 172.21.15.46:0/2191011024 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320005cf0 con 0x7fb3340b3f20 2024-03-29T08:22:26.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.514+0000 7fb331600640 1 -- 172.21.15.46:0/2191011024 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb320005e50 con 0x7fb3340b3f20 2024-03-29T08:22:26.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.514+0000 7fb331600640 1 -- 172.21.15.46:0/2191011024 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320007270 con 0x7fb3340b3f20 2024-03-29T08:22:26.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.514+0000 7fb33c400640 1 -- 172.21.15.46:0/2191011024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3341515f0 con 0x7fb3340b3f20 2024-03-29T08:22:26.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.514+0000 7fb33c400640 1 -- 172.21.15.46:0/2191011024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb334152050 con 0x7fb3340b3f20 2024-03-29T08:22:26.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.514+0000 7fb331600640 1 -- 172.21.15.46:0/2191011024 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb3200073d0 con 0x7fb3340b3f20 2024-03-29T08:22:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb331600640 1 --2- 172.21.15.46:0/2191011024 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb314066690 0x7fb314068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb331600640 1 -- 172.21.15.46:0/2191011024 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb320083600 con 0x7fb3340b3f20 2024-03-29T08:22:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb339a00640 1 --2- 172.21.15.46:0/2191011024 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb314066690 0x7fb314068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb33f8cb000 1 Processor -- start 2024-03-29T08:22:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb33f8cb000 1 -- start start 2024-03-29T08:22:26.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb339a00640 1 --2- 172.21.15.46:0/2191011024 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb314066690 0x7fb314068b50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb328009b60 tx=0x7fb328009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:26.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb33f8cb000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d63a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb33f8cb000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d63abb0 0x55774d5f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb33f8cb000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d5f11b0 0x55774d5f1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb33f8cb000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774d4b9340 con 0x55774d63a290 2024-03-29T08:22:26.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb33f8cb000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774d445cd0 con 0x55774d63abb0 2024-03-29T08:22:26.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb33f8cb000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55774d4432a0 con 0x55774d5f11b0 2024-03-29T08:22:26.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb339a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d63a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb339a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d63a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42012/0 (socket says 172.21.15.46:42012) 2024-03-29T08:22:26.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb339a00640 1 -- 172.21.15.46:0/3357587042 learned_addr learned my addr 172.21.15.46:0/3357587042 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb339000640 1 --2- 172.21.15.46:0/3357587042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d63abb0 0x55774d5f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb33a400640 1 --2- 172.21.15.46:0/3357587042 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d5f11b0 0x55774d5f1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb339a00640 1 -- 172.21.15.46:0/3357587042 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d5f11b0 msgr2=0x55774d5f1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb339a00640 1 --2- 172.21.15.46:0/3357587042 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d5f11b0 0x55774d5f1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb339a00640 1 -- 172.21.15.46:0/3357587042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d63abb0 msgr2=0x55774d5f0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb339a00640 1 --2- 172.21.15.46:0/3357587042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d63abb0 0x55774d5f0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb339a00640 1 -- 172.21.15.46:0/3357587042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55774d1314b0 con 0x55774d63a290 2024-03-29T08:22:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb339a00640 1 --2- 172.21.15.46:0/3357587042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d63a670 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb328011ec0 tx=0x7fb328014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0325d68947bea7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb326000640 1 -- 172.21.15.46:0/3357587042 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32801c040 con 0x55774d63a290 2024-03-29T08:22:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb326000640 1 -- 172.21.15.46:0/3357587042 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb328003830 con 0x55774d63a290 2024-03-29T08:22:26.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.518+0000 7fb326000640 1 -- 172.21.15.46:0/3357587042 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328003ba0 con 0x55774d63a290 2024-03-29T08:22:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 -- 172.21.15.46:0/3357587042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 msgr2=0x55774d63a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/3357587042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d63a670 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fb328011ec0 tx=0x7fb328014000 comp rx=0 tx=0).stop 2024-03-29T08:22:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 -- 172.21.15.46:0/3357587042 shutdown_connections 2024-03-29T08:22:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/3357587042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d63abb0 0x55774d5f0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/3357587042 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d5f11b0 0x55774d5f1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/3357587042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d63a670 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 -- 172.21.15.46:0/3357587042 >> 172.21.15.46:0/3357587042 conn(0x55774d5ed160 msgr2=0x55774d5ef550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 -- 172.21.15.46:0/3357587042 shutdown_connections 2024-03-29T08:22:26.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 -- 172.21.15.46:0/3357587042 wait complete. 2024-03-29T08:22:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 -- start start 2024-03-29T08:22:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 Processor -- start 2024-03-29T08:22:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d5f11b0 0x55774d5f6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d5f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d63abb0 0x55774d5fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774d43eb90 con 0x55774d63a290 2024-03-29T08:22:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774d3fc0a0 con 0x55774d5f11b0 2024-03-29T08:22:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55774d3fbd40 con 0x55774d63abb0 2024-03-29T08:22:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 10 client.0 _ll_register_callbacks cb 0x55774d4f3c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:22:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb339000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d5f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb339000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d5f6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42026/0 (socket says 172.21.15.46:42026) 2024-03-29T08:22:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb339000640 1 -- 172.21.15.46:0/289151432 learned_addr learned my addr 172.21.15.46:0/289151432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb339a00640 1 --2- 172.21.15.46:0/289151432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d5f11b0 0x55774d5f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33a400640 1 --2- 172.21.15.46:0/289151432 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d63abb0 0x55774d5fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb339000640 1 -- 172.21.15.46:0/289151432 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d63abb0 msgr2=0x55774d5fbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb339000640 1 --2- 172.21.15.46:0/289151432 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d63abb0 0x55774d5fbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb339000640 1 -- 172.21.15.46:0/289151432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d5f11b0 msgr2=0x55774d5f6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb339000640 1 --2- 172.21.15.46:0/289151432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d5f11b0 0x55774d5f6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb339000640 1 -- 172.21.15.46:0/289151432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb328011ba0 con 0x55774d63a290 2024-03-29T08:22:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb339a00640 1 --2- 172.21.15.46:0/289151432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d5f11b0 0x55774d5f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:22:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb339000640 1 --2- 172.21.15.46:0/289151432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d5f6c50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb32000ce10 tx=0x7fb32005ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb26411b2c62b19c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb31aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:22:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb31aa00640 1 -- 172.21.15.46:0/289151432 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320053b80 con 0x55774d63a290 2024-03-29T08:22:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb31aa00640 1 -- 172.21.15.46:0/289151432 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb320053ce0 con 0x55774d63a290 2024-03-29T08:22:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 -- 172.21.15.46:0/289151432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55774d40e550 con 0x55774d63a290 2024-03-29T08:22:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb31aa00640 1 -- 172.21.15.46:0/289151432 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb32005c540 con 0x55774d63a290 2024-03-29T08:22:26.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 10 client.6441 fetch_fsmap learned FSMap version 34 2024-03-29T08:22:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 -- 172.21.15.46:0/289151432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55774d403ba0 con 0x55774d63a290 2024-03-29T08:22:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb31aa00640 1 -- 172.21.15.46:0/289151432 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55774d40e550 con 0x55774d63a290 2024-03-29T08:22:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb31aa00640 1 -- 172.21.15.46:0/289151432 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 34) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb3200830d0 con 0x55774d63a290 2024-03-29T08:22:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 10 client.6441 fetch_fsmap finished waiting for FSMap version 34 2024-03-29T08:22:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 10 client.6441 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:22:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 20 client.6441 populate_metadata read hostname 'smithi046' 2024-03-29T08:22:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/289151432 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55774d649320 0x55774d64b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 4 client.6441 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:22:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.522+0000 7fb33f8cb000 1 -- 172.21.15.46:0/289151432 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55774d43b970 con 0x55774d649320 2024-03-29T08:22:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb339a00640 1 --2- 172.21.15.46:0/289151432 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55774d649320 0x55774d64b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb339a00640 1 --2- 172.21.15.46:0/289151432 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55774d649320 0x55774d64b710 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=296138bab6228327 server_cookie=44430f338c2b0c01 in_seq=0 out_seq=0 2024-03-29T08:22:26.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb31aa00640 10 client.6441 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:22:26.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb31aa00640 1 -- 172.21.15.46:0/289151432 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55774d43b970 con 0x55774d649320 2024-03-29T08:22:26.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb31aa00640 10 client.6441 handle_command_reply: tid=0 2024-03-29T08:22:26.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb33f8cb000 1 client.6441 shutdown 2024-03-29T08:22:26.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb33f8cb000 1 -- 172.21.15.46:0/289151432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 msgr2=0x55774d5f6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/289151432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d5f6c50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fb32000ce10 tx=0x7fb32005ca90 comp rx=0 tx=0).stop 2024-03-29T08:22:26.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb33f8cb000 1 -- 172.21.15.46:0/289151432 shutdown_connections 2024-03-29T08:22:26.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/289151432 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d63abb0 0x55774d5fbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/289151432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d5f6c50 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/289151432 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55774d649320 0x55774d64b710 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/289151432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d5f11b0 0x55774d5f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb31aa00640 0 client.6441 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:22:26.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb33f8cb000 1 -- 172.21.15.46:0/289151432 >> 172.21.15.46:0/289151432 conn(0x55774d5ed160 msgr2=0x55774d5efde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:26.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb33f8cb000 1 -- 172.21.15.46:0/289151432 shutdown_connections 2024-03-29T08:22:26.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb33f8cb000 1 -- 172.21.15.46:0/289151432 wait complete. 2024-03-29T08:22:26.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.526+0000 7fb33f8cb000 20 client.6441 trim_cache size 0 max 16384 2024-03-29T08:22:26.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb33f8cb000 1 Processor -- start 2024-03-29T08:22:26.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb33f8cb000 1 -- start start 2024-03-29T08:22:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb33f8cb000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d5f11b0 0x55774d5f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb33f8cb000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d5f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb33f8cb000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d63abb0 0x55774d5f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb33f8cb000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774d0fbc40 con 0x55774d63a290 2024-03-29T08:22:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb33f8cb000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774d3dba40 con 0x55774d63abb0 2024-03-29T08:22:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb33f8cb000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55774d5e85e0 con 0x55774d5f11b0 2024-03-29T08:22:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb339000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d5f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb339000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d5f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42028/0 (socket says 172.21.15.46:42028) 2024-03-29T08:22:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb339000640 1 -- 172.21.15.46:0/3067260767 learned_addr learned my addr 172.21.15.46:0/3067260767 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:26.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb339a00640 1 --2- 172.21.15.46:0/3067260767 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d5f11b0 0x55774d5f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb33a400640 1 --2- 172.21.15.46:0/3067260767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d63abb0 0x55774d5f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb339000640 1 -- 172.21.15.46:0/3067260767 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d5f11b0 msgr2=0x55774d5f6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:26.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb339000640 1 --2- 172.21.15.46:0/3067260767 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d5f11b0 0x55774d5f6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb339000640 1 -- 172.21.15.46:0/3067260767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d63abb0 msgr2=0x55774d5f5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:26.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb339000640 1 --2- 172.21.15.46:0/3067260767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d63abb0 0x55774d5f5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb339000640 1 -- 172.21.15.46:0/3067260767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55774d3e9ef0 con 0x55774d63a290 2024-03-29T08:22:26.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb339000640 1 --2- 172.21.15.46:0/3067260767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d5f56c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb320054620 tx=0x7fb320057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0f1782ab674fe0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:26.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb326a00640 1 -- 172.21.15.46:0/3067260767 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb320054710 con 0x55774d63a290 2024-03-29T08:22:26.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb326a00640 1 -- 172.21.15.46:0/3067260767 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32000e040 con 0x55774d63a290 2024-03-29T08:22:26.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.538+0000 7fb326a00640 1 -- 172.21.15.46:0/3067260767 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32005c540 con 0x55774d63a290 2024-03-29T08:22:26.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 -- 172.21.15.46:0/3067260767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 msgr2=0x55774d5f56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/3067260767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d5f56c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fb320054620 tx=0x7fb320057000 comp rx=0 tx=0).stop 2024-03-29T08:22:26.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 -- 172.21.15.46:0/3067260767 shutdown_connections 2024-03-29T08:22:26.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/3067260767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d63abb0 0x55774d5f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/3067260767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63a290 0x55774d5f56c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/3067260767 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d5f11b0 0x55774d5f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 -- 172.21.15.46:0/3067260767 >> 172.21.15.46:0/3067260767 conn(0x55774d5ed160 msgr2=0x55774d5f08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 -- 172.21.15.46:0/3067260767 shutdown_connections 2024-03-29T08:22:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 -- 172.21.15.46:0/3067260767 wait complete. 2024-03-29T08:22:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 -- start start 2024-03-29T08:22:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 Processor -- start 2024-03-29T08:22:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d5f11b0 0x55774d693bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d63a290 0x55774d64ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63abb0 0x55774d64bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774d3d4cb0 con 0x55774d63abb0 2024-03-29T08:22:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55774d4416f0 con 0x55774d5f11b0 2024-03-29T08:22:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55774d5fa600 con 0x55774d63a290 2024-03-29T08:22:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 10 client.0 _ll_register_callbacks cb 0x55774d4dc870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:22:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb339a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d5f11b0 0x55774d693bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63abb0 0x55774d64bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb339a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d5f11b0 0x55774d693bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:48092/0 (socket says 172.21.15.46:48092) 2024-03-29T08:22:26.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb339a00640 1 -- 172.21.15.46:0/1800458408 learned_addr learned my addr 172.21.15.46:0/1800458408 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:26.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb339000640 1 --2- 172.21.15.46:0/1800458408 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d63a290 0x55774d64ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb339a00640 1 -- 172.21.15.46:0/1800458408 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d63a290 msgr2=0x55774d64ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:26.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb339a00640 1 --2- 172.21.15.46:0/1800458408 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d63a290 0x55774d64ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb339a00640 1 -- 172.21.15.46:0/1800458408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63abb0 msgr2=0x55774d64bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:26.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb339a00640 1 --2- 172.21.15.46:0/1800458408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63abb0 0x55774d64bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb339a00640 1 -- 172.21.15.46:0/1800458408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55774d403ba0 con 0x55774d5f11b0 2024-03-29T08:22:26.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb339a00640 1 --2- 172.21.15.46:0/1800458408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d5f11b0 0x55774d693bd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb328011750 tx=0x7fb328004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60f2b586b90df811 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:26.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb31aa00640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-29T08:22:26.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb31aa00640 1 -- 172.21.15.46:0/1800458408 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328035440 con 0x55774d5f11b0 2024-03-29T08:22:26.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb31aa00640 1 -- 172.21.15.46:0/1800458408 <== mon.1 v2:172.21.15.73:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb32803fb30 con 0x55774d5f11b0 2024-03-29T08:22:26.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 -- 172.21.15.46:0/1800458408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55774d43d4e0 con 0x55774d5f11b0 2024-03-29T08:22:26.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb31aa00640 1 -- 172.21.15.46:0/1800458408 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3280355a0 con 0x55774d5f11b0 2024-03-29T08:22:26.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 10 client.6331 fetch_fsmap learned FSMap version 34 2024-03-29T08:22:26.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 1 -- 172.21.15.46:0/1800458408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55774d13afc0 con 0x55774d5f11b0 2024-03-29T08:22:26.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb31aa00640 1 -- 172.21.15.46:0/1800458408 <== mon.1 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55774d43d4e0 con 0x55774d5f11b0 2024-03-29T08:22:26.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb31aa00640 1 -- 172.21.15.46:0/1800458408 <== mon.1 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb32803fcf0 con 0x55774d5f11b0 2024-03-29T08:22:26.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 10 client.6331 fetch_fsmap finished waiting for FSMap version 34 2024-03-29T08:22:26.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 10 client.6331 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:22:26.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.542+0000 7fb33f8cb000 20 client.6331 populate_metadata read hostname 'smithi046' 2024-03-29T08:22:26.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/1800458408 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55774d64c490 0x55774d64c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33f8cb000 4 client.6331 mds_command: new command op to 6051 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-29T08:22:26.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33f8cb000 1 -- 172.21.15.46:0/1800458408 --> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55774d437650 con 0x55774d64c490 2024-03-29T08:22:26.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb339000640 1 --2- 172.21.15.46:0/1800458408 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55774d64c490 0x55774d64c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb339000640 1 --2- 172.21.15.46:0/1800458408 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55774d64c490 0x55774d64c870 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f3dcc19474fc639 server_cookie=de295cef6288abb in_seq=0 out_seq=0 2024-03-29T08:22:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb31aa00640 10 client.6331 ms_handle_connect on v2:172.21.15.46:6833/3255474176 2024-03-29T08:22:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb31aa00640 1 -- 172.21.15.46:0/1800458408 <== mds.0 v2:172.21.15.46:6833/3255474176 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55774d437650 con 0x55774d64c490 2024-03-29T08:22:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb31aa00640 10 client.6331 handle_command_reply: tid=0 2024-03-29T08:22:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33f8cb000 1 client.6331 shutdown 2024-03-29T08:22:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33f8cb000 1 -- 172.21.15.46:0/1800458408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d5f11b0 msgr2=0x55774d693bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/1800458408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d5f11b0 0x55774d693bd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb328011750 tx=0x7fb328004740 comp rx=0 tx=0).stop 2024-03-29T08:22:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33f8cb000 1 -- 172.21.15.46:0/1800458408 shutdown_connections 2024-03-29T08:22:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/1800458408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55774d63abb0 0x55774d64bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/1800458408 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55774d63a290 0x55774d64ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/1800458408 >> [v2:172.21.15.46:6833/3255474176,v1:172.21.15.46:6835/3255474176] conn(0x55774d64c490 0x55774d64c870 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33f8cb000 1 --2- 172.21.15.46:0/1800458408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55774d5f11b0 0x55774d693bd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb31aa00640 0 client.6331 ms_handle_reset on v2:172.21.15.46:6833/3255474176 2024-03-29T08:22:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33f8cb000 1 -- 172.21.15.46:0/1800458408 >> 172.21.15.46:0/1800458408 conn(0x55774d5ed160 msgr2=0x55774d638d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:26.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33f8cb000 1 -- 172.21.15.46:0/1800458408 shutdown_connections 2024-03-29T08:22:26.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33f8cb000 1 -- 172.21.15.46:0/1800458408 wait complete. 2024-03-29T08:22:26.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33f8cb000 20 client.6331 trim_cache size 0 max 16384 2024-03-29T08:22:26.553 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:26.553 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-29T08:22:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33c400640 1 -- 172.21.15.46:0/2191011024 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb314066690 msgr2=0x7fb314068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33c400640 1 --2- 172.21.15.46:0/2191011024 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb314066690 0x7fb314068b50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb328009b60 tx=0x7fb328009190 comp rx=0 tx=0).stop 2024-03-29T08:22:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33c400640 1 -- 172.21.15.46:0/2191011024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3340b3f20 msgr2=0x7fb334155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33c400640 1 --2- 172.21.15.46:0/2191011024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3340b3f20 0x7fb334155b20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb32000c450 tx=0x7fb32000c910 comp rx=0 tx=0).stop 2024-03-29T08:22:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33c400640 1 -- 172.21.15.46:0/2191011024 shutdown_connections 2024-03-29T08:22:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33c400640 1 --2- 172.21.15.46:0/2191011024 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb314066690 0x7fb314068b50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33c400640 1 --2- 172.21.15.46:0/2191011024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb33415dbb0 0x7fb334156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33c400640 1 --2- 172.21.15.46:0/2191011024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3340b3f20 0x7fb334155b20 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33c400640 1 --2- 172.21.15.46:0/2191011024 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb3340b2440 0x7fb33414d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33c400640 1 -- 172.21.15.46:0/2191011024 >> 172.21.15.46:0/2191011024 conn(0x7fb334070ad0 msgr2=0x7fb334150400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:26.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33c400640 1 -- 172.21.15.46:0/2191011024 shutdown_connections 2024-03-29T08:22:26.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.546+0000 7fb33c400640 1 -- 172.21.15.46:0/2191011024 wait complete. 2024-03-29T08:22:26.568 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-29T08:22:26.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.702+0000 7f723aa00640 1 Processor -- start 2024-03-29T08:22:26.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.702+0000 7f723aa00640 1 -- start start 2024-03-29T08:22:26.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.706+0000 7f723aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f723415ada0 0x7f72340ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.706+0000 7f723aa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f72340ce950 0x7f723406c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.706+0000 7f723aa00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72340cb510 con 0x7f7234144210 2024-03-29T08:22:26.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.706+0000 7f723aa00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72340cede0 con 0x7f723415ada0 2024-03-29T08:22:26.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.706+0000 7f723aa00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7234070580 con 0x7f72340ce950 2024-03-29T08:22:26.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.706+0000 7f722b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f723415ada0 0x7f72340ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.706+0000 7f722b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f723415ada0 0x7f72340ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42038/0 (socket says 172.21.15.46:42038) 2024-03-29T08:22:26.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.706+0000 7f722b400640 1 -- 172.21.15.46:0/367003897 learned_addr learned my addr 172.21.15.46:0/367003897 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:26.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.706+0000 7f7238a00640 1 --2- 172.21.15.46:0/367003897 >> v2:172.21.15.73:3300/0 conn(0x7f72340ce950 0x7f723406c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.706+0000 7f722b400640 1 -- 172.21.15.46:0/367003897 >> v2:172.21.15.73:3300/0 conn(0x7f72340ce950 msgr2=0x7f723406c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:26.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.706+0000 7f722b400640 1 --2- 172.21.15.46:0/367003897 >> v2:172.21.15.73:3300/0 conn(0x7f72340ce950 0x7f723406c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.706+0000 7f722b400640 1 -- 172.21.15.46:0/367003897 >> v1:172.21.15.73:6789/0 conn(0x7f7234144210 legacy=0x7f72340cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.706+0000 7f722b400640 1 -- 172.21.15.46:0/367003897 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7234070860 con 0x7f723415ada0 2024-03-29T08:22:26.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.706+0000 7f7238a00640 1 --2- 172.21.15.46:0/367003897 >> v2:172.21.15.73:3300/0 conn(0x7f72340ce950 0x7f723406c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:26.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.710+0000 7f722b400640 1 --2- 172.21.15.46:0/367003897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723415ada0 0x7f72340ce410 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f72340cf170 tx=0x7f721c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2fcadfcea2fe05 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:26.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.710+0000 7f7233400640 1 -- 172.21.15.46:0/367003897 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c0ab610 con 0x7f723415ada0 2024-03-29T08:22:26.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.710+0000 7f7233400640 1 -- 172.21.15.46:0/367003897 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c0ab770 con 0x7f723415ada0 2024-03-29T08:22:26.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.710+0000 7f7233400640 1 -- 172.21.15.46:0/367003897 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c0abb00 con 0x7f723415ada0 2024-03-29T08:22:26.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.710+0000 7f723aa00640 1 -- 172.21.15.46:0/367003897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723415ada0 msgr2=0x7f72340ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.710+0000 7f723aa00640 1 --2- 172.21.15.46:0/367003897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723415ada0 0x7f72340ce410 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f72340cf170 tx=0x7f721c01c520 comp rx=0 tx=0).stop 2024-03-29T08:22:26.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.710+0000 7f723aa00640 1 -- 172.21.15.46:0/367003897 shutdown_connections 2024-03-29T08:22:26.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.710+0000 7f723aa00640 1 --2- 172.21.15.46:0/367003897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723415ada0 0x7f72340ce410 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.710+0000 7f723aa00640 1 --2- 172.21.15.46:0/367003897 >> v2:172.21.15.73:3300/0 conn(0x7f72340ce950 0x7f723406c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.710+0000 7f723aa00640 1 -- 172.21.15.46:0/367003897 >> 172.21.15.46:0/367003897 conn(0x7f7234066110 msgr2=0x7f7234069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:26.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.710+0000 7f723aa00640 1 -- 172.21.15.46:0/367003897 shutdown_connections 2024-03-29T08:22:26.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.710+0000 7f723aa00640 1 -- 172.21.15.46:0/367003897 wait complete. 2024-03-29T08:22:26.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.710+0000 7f723aa00640 1 Processor -- start 2024-03-29T08:22:26.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.710+0000 7f723aa00640 1 -- start start 2024-03-29T08:22:26.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f723aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72340ce950 0x7f723415b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f723aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7234144210 0x7f723415d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f723aa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f723415ada0 0x7f723415d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f723aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723415dd00 con 0x7f7234144210 2024-03-29T08:22:26.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f723aa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723415dfe0 con 0x7f72340ce950 2024-03-29T08:22:26.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f723aa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723415e2c0 con 0x7f723415ada0 2024-03-29T08:22:26.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f722b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7234144210 0x7f723415d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f722b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7234144210 0x7f723415d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42040/0 (socket says 172.21.15.46:42040) 2024-03-29T08:22:26.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f722b400640 1 -- 172.21.15.46:0/3847051736 learned_addr learned my addr 172.21.15.46:0/3847051736 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:26.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f7233e00640 1 --2- 172.21.15.46:0/3847051736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72340ce950 0x7f723415b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f7238a00640 1 --2- 172.21.15.46:0/3847051736 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f723415ada0 0x7f723415d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f722b400640 1 -- 172.21.15.46:0/3847051736 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f723415ada0 msgr2=0x7f723415d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f722b400640 1 --2- 172.21.15.46:0/3847051736 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f723415ada0 0x7f723415d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f722b400640 1 -- 172.21.15.46:0/3847051736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72340ce950 msgr2=0x7f723415b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f722b400640 1 --2- 172.21.15.46:0/3847051736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72340ce950 0x7f723415b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f722b400640 1 -- 172.21.15.46:0/3847051736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c0094d0 con 0x7f7234144210 2024-03-29T08:22:26.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f7233e00640 1 --2- 172.21.15.46:0/3847051736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72340ce950 0x7f723415b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:26.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f7238a00640 1 --2- 172.21.15.46:0/3847051736 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f723415ada0 0x7f723415d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:26.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f722b400640 1 --2- 172.21.15.46:0/3847051736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7234144210 0x7f723415d1b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f721c019130 tx=0x7f721c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:26.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f7230c00640 1 -- 172.21.15.46:0/3847051736 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c0b4800 con 0x7f7234144210 2024-03-29T08:22:26.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f7230c00640 1 -- 172.21.15.46:0/3847051736 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c0b4980 con 0x7f7234144210 2024-03-29T08:22:26.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f7230c00640 1 -- 172.21.15.46:0/3847051736 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c0b4d30 con 0x7f7234144210 2024-03-29T08:22:26.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f723aa00640 1 -- 172.21.15.46:0/3847051736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f723415e5a0 con 0x7f7234144210 2024-03-29T08:22:26.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f723aa00640 1 -- 172.21.15.46:0/3847051736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f723415a1f0 con 0x7f7234144210 2024-03-29T08:22:26.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f723aa00640 1 -- 172.21.15.46:0/3847051736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72340ce280 con 0x7f7234144210 2024-03-29T08:22:26.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.714+0000 7f7230c00640 1 -- 172.21.15.46:0/3847051736 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f721c0b3380 con 0x7f7234144210 2024-03-29T08:22:26.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.718+0000 7f7230c00640 1 --2- 172.21.15.46:0/3847051736 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7210066690 0x7f7210068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:26.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.718+0000 7f7230c00640 1 -- 172.21.15.46:0/3847051736 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f721c11e7e0 con 0x7f7234144210 2024-03-29T08:22:26.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.718+0000 7f7233e00640 1 --2- 172.21.15.46:0/3847051736 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7210066690 0x7f7210068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:26.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.718+0000 7f7233e00640 1 --2- 172.21.15.46:0/3847051736 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7210066690 0x7f7210068b50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7234141450 tx=0x7f7224009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:26.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.722+0000 7f7230c00640 1 -- 172.21.15.46:0/3847051736 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f721c0ebb20 con 0x7f7234144210 2024-03-29T08:22:26.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.914+0000 7f723aa00640 1 -- 172.21.15.46:0/3847051736 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7234066c90 con 0x7f7234144210 2024-03-29T08:22:26.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.914+0000 7f7230c00640 1 -- 172.21.15.46:0/3847051736 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f721c0ef140 con 0x7f7234144210 2024-03-29T08:22:26.918 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:26.918 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-29T08:22:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.914+0000 7f723aa00640 1 -- 172.21.15.46:0/3847051736 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7210066690 msgr2=0x7f7210068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.914+0000 7f723aa00640 1 --2- 172.21.15.46:0/3847051736 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7210066690 0x7f7210068b50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7234141450 tx=0x7f7224009040 comp rx=0 tx=0).stop 2024-03-29T08:22:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.918+0000 7f723aa00640 1 -- 172.21.15.46:0/3847051736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7234144210 msgr2=0x7f723415d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.918+0000 7f723aa00640 1 --2- 172.21.15.46:0/3847051736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7234144210 0x7f723415d1b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f721c019130 tx=0x7f721c01ad60 comp rx=0 tx=0).stop 2024-03-29T08:22:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.918+0000 7f723aa00640 1 -- 172.21.15.46:0/3847051736 shutdown_connections 2024-03-29T08:22:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.918+0000 7f723aa00640 1 --2- 172.21.15.46:0/3847051736 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7210066690 0x7f7210068b50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.918+0000 7f723aa00640 1 --2- 172.21.15.46:0/3847051736 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f723415ada0 0x7f723415d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.918+0000 7f723aa00640 1 --2- 172.21.15.46:0/3847051736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7234144210 0x7f723415d1b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.918+0000 7f723aa00640 1 --2- 172.21.15.46:0/3847051736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72340ce950 0x7f723415b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.918+0000 7f723aa00640 1 -- 172.21.15.46:0/3847051736 >> 172.21.15.46:0/3847051736 conn(0x7f7234066110 msgr2=0x7f7234069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.918+0000 7f723aa00640 1 -- 172.21.15.46:0/3847051736 shutdown_connections 2024-03-29T08:22:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:26.918+0000 7f723aa00640 1 -- 172.21.15.46:0/3847051736 wait complete. 2024-03-29T08:22:26.938 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-29T08:22:26.939 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T08:22:26.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:22:27.022 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:22:27.102 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T08:22:27.102 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:22:27.102 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:22:27.111 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:22:27.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.290+0000 7fe8b6e00640 1 Processor -- start 2024-03-29T08:22:27.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.290+0000 7fe8b6e00640 1 -- start start 2024-03-29T08:22:27.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.294+0000 7fe8b6e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe8b015c9d0 0x7fe8b00cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.294+0000 7fe8b6e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe8b00cef60 0x7fe8b0098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.294+0000 7fe8b6e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe8b00ccd10 0x7fe8b0098f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.294+0000 7fe8b6e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b00d0030 con 0x7fe8b00cef60 2024-03-29T08:22:27.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.294+0000 7fe8b6e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b00cf420 con 0x7fe8b00ccd10 2024-03-29T08:22:27.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.294+0000 7fe8b6e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00cf700 con 0x7fe8b015c9d0 2024-03-29T08:22:27.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.294+0000 7fe8a7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe8b00cef60 0x7fe8b0098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.294+0000 7fe8b4e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe8b00ccd10 0x7fe8b0098f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.294+0000 7fe8afe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe8b015c9d0 0x7fe8b00cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.294+0000 7fe8a7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe8b00cef60 0x7fe8b0098a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42052/0 (socket says 172.21.15.46:42052) 2024-03-29T08:22:27.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.294+0000 7fe8a7400640 1 -- 172.21.15.46:0/2247025300 learned_addr learned my addr 172.21.15.46:0/2247025300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:27.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.294+0000 7fe8a7400640 1 -- 172.21.15.46:0/2247025300 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe8b015c9d0 msgr2=0x7fe8b00cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:27.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.294+0000 7fe8a7400640 1 --2- 172.21.15.46:0/2247025300 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe8b015c9d0 0x7fe8b00cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.294+0000 7fe8a7400640 1 -- 172.21.15.46:0/2247025300 >> v2:172.21.15.73:3300/0 conn(0x7fe8b00ccd10 msgr2=0x7fe8b0098f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:27.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.294+0000 7fe8a7400640 1 --2- 172.21.15.46:0/2247025300 >> v2:172.21.15.73:3300/0 conn(0x7fe8b00ccd10 0x7fe8b0098f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.294+0000 7fe8a7400640 1 -- 172.21.15.46:0/2247025300 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b00cf9e0 con 0x7fe8b00cef60 2024-03-29T08:22:27.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.294+0000 7fe8a7400640 1 --2- 172.21.15.46:0/2247025300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b00cef60 0x7fe8b0098a40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe8b01617a0 tx=0x7fe89801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b995cde14db2231 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:27.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8af400640 1 -- 172.21.15.46:0/2247025300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8980aa430 con 0x7fe8b00cef60 2024-03-29T08:22:27.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8af400640 1 -- 172.21.15.46:0/2247025300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8980aa590 con 0x7fe8b00cef60 2024-03-29T08:22:27.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8af400640 1 -- 172.21.15.46:0/2247025300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8980aa920 con 0x7fe8b00cef60 2024-03-29T08:22:27.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8b6e00640 1 -- 172.21.15.46:0/2247025300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b00cef60 msgr2=0x7fe8b0098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8b6e00640 1 --2- 172.21.15.46:0/2247025300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b00cef60 0x7fe8b0098a40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fe8b01617a0 tx=0x7fe89801c4f0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8b6e00640 1 -- 172.21.15.46:0/2247025300 shutdown_connections 2024-03-29T08:22:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8b6e00640 1 --2- 172.21.15.46:0/2247025300 >> v2:172.21.15.73:3300/0 conn(0x7fe8b00ccd10 0x7fe8b0098f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8b6e00640 1 --2- 172.21.15.46:0/2247025300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b00cef60 0x7fe8b0098a40 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8b6e00640 1 --2- 172.21.15.46:0/2247025300 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe8b015c9d0 0x7fe8b00cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8b6e00640 1 -- 172.21.15.46:0/2247025300 >> 172.21.15.46:0/2247025300 conn(0x7fe8b00b21a0 msgr2=0x7fe8b00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8b6e00640 1 -- 172.21.15.46:0/2247025300 shutdown_connections 2024-03-29T08:22:27.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8b6e00640 1 -- 172.21.15.46:0/2247025300 wait complete. 2024-03-29T08:22:27.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8b6e00640 1 Processor -- start 2024-03-29T08:22:27.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8b6e00640 1 -- start start 2024-03-29T08:22:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8b6e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b00ccd10 0x7fe8b00ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8b6e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe8b00cef60 0x7fe8b0155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8b6e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8b015c9d0 0x7fe8b014c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8b6e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b0156080 con 0x7fe8b00ccd10 2024-03-29T08:22:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8b6e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b015d650 con 0x7fe8b015c9d0 2024-03-29T08:22:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8b6e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00c9540 con 0x7fe8b00cef60 2024-03-29T08:22:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b00ccd10 0x7fe8b00ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.298+0000 7fe8afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b00ccd10 0x7fe8b00ca4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42060/0 (socket says 172.21.15.46:42060) 2024-03-29T08:22:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.302+0000 7fe8afe00640 1 -- 172.21.15.46:0/2643204432 learned_addr learned my addr 172.21.15.46:0/2643204432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.302+0000 7fe8a7400640 1 --2- 172.21.15.46:0/2643204432 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe8b00cef60 0x7fe8b0155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.302+0000 7fe8b4e00640 1 --2- 172.21.15.46:0/2643204432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8b015c9d0 0x7fe8b014c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.302+0000 7fe8afe00640 1 -- 172.21.15.46:0/2643204432 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe8b00cef60 msgr2=0x7fe8b0155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:27.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.302+0000 7fe8afe00640 1 --2- 172.21.15.46:0/2643204432 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe8b00cef60 0x7fe8b0155b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.302+0000 7fe8afe00640 1 -- 172.21.15.46:0/2643204432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8b015c9d0 msgr2=0x7fe8b014c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:27.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.302+0000 7fe8afe00640 1 --2- 172.21.15.46:0/2643204432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8b015c9d0 0x7fe8b014c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.302+0000 7fe8afe00640 1 -- 172.21.15.46:0/2643204432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8980094d0 con 0x7fe8b00ccd10 2024-03-29T08:22:27.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.302+0000 7fe8afe00640 1 --2- 172.21.15.46:0/2643204432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b00ccd10 0x7fe8b00ca4c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000e960 tx=0x7fe8a000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:27.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.302+0000 7fe8acc00640 1 -- 172.21.15.46:0/2643204432 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0009690 con 0x7fe8b00ccd10 2024-03-29T08:22:27.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.302+0000 7fe8acc00640 1 -- 172.21.15.46:0/2643204432 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a000ca00 con 0x7fe8b00ccd10 2024-03-29T08:22:27.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.302+0000 7fe8b6e00640 1 -- 172.21.15.46:0/2643204432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b00c9820 con 0x7fe8b00ccd10 2024-03-29T08:22:27.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.302+0000 7fe8acc00640 1 -- 172.21.15.46:0/2643204432 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a000cd70 con 0x7fe8b00ccd10 2024-03-29T08:22:27.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.302+0000 7fe8b6e00640 1 -- 172.21.15.46:0/2643204432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b015d8a0 con 0x7fe8b00ccd10 2024-03-29T08:22:27.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.302+0000 7fe8b6e00640 1 -- 172.21.15.46:0/2643204432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b00c9d40 con 0x7fe8b00ccd10 2024-03-29T08:22:27.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.302+0000 7fe8acc00640 1 -- 172.21.15.46:0/2643204432 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe8a0013470 con 0x7fe8b00ccd10 2024-03-29T08:22:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.302+0000 7fe8acc00640 1 --2- 172.21.15.46:0/2643204432 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe88c066690 0x7fe88c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.306+0000 7fe8acc00640 1 -- 172.21.15.46:0/2643204432 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe8a0084270 con 0x7fe8b00ccd10 2024-03-29T08:22:27.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.306+0000 7fe8a7400640 1 --2- 172.21.15.46:0/2643204432 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe88c066690 0x7fe88c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.306+0000 7fe8a7400640 1 --2- 172.21.15.46:0/2643204432 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe88c066690 0x7fe88c068b50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe8980ae000 tx=0x7fe8980b5040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:27.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.306+0000 7fe8acc00640 1 -- 172.21.15.46:0/2643204432 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe8a00516e0 con 0x7fe8b00ccd10 2024-03-29T08:22:27.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.470+0000 7fe8b6e00640 1 -- 172.21.15.46:0/2643204432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe8b0066c90 con 0x7fe8b00ccd10 2024-03-29T08:22:27.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.470+0000 7fe8acc00640 1 -- 172.21.15.46:0/2643204432 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16145 (secure 0 0 0) 0x7fe8a0054d00 con 0x7fe8b00ccd10 2024-03-29T08:22:27.475 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:27.475 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":65,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:21:58.933917+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:21:08.061567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T08:21:09.336464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.2f","mappings":[{"from":6,"to":1}]},{"pgid":"9.3b","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:22:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.474+0000 7fe8b6e00640 1 -- 172.21.15.46:0/2643204432 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe88c066690 msgr2=0x7fe88c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.474+0000 7fe8b6e00640 1 --2- 172.21.15.46:0/2643204432 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe88c066690 0x7fe88c068b50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe8980ae000 tx=0x7fe8980b5040 comp rx=0 tx=0).stop 2024-03-29T08:22:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.474+0000 7fe8b6e00640 1 -- 172.21.15.46:0/2643204432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b00ccd10 msgr2=0x7fe8b00ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.474+0000 7fe8b6e00640 1 --2- 172.21.15.46:0/2643204432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b00ccd10 0x7fe8b00ca4c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000e960 tx=0x7fe8a000ee20 comp rx=0 tx=0).stop 2024-03-29T08:22:27.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.474+0000 7fe8b6e00640 1 -- 172.21.15.46:0/2643204432 shutdown_connections 2024-03-29T08:22:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.474+0000 7fe8b6e00640 1 --2- 172.21.15.46:0/2643204432 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe88c066690 0x7fe88c068b50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.474+0000 7fe8b6e00640 1 --2- 172.21.15.46:0/2643204432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8b015c9d0 0x7fe8b014c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.474+0000 7fe8b6e00640 1 --2- 172.21.15.46:0/2643204432 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe8b00cef60 0x7fe8b0155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.474+0000 7fe8b6e00640 1 --2- 172.21.15.46:0/2643204432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8b00ccd10 0x7fe8b00ca4c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.474+0000 7fe8b6e00640 1 -- 172.21.15.46:0/2643204432 >> 172.21.15.46:0/2643204432 conn(0x7fe8b00b21a0 msgr2=0x7fe8b00cace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.474+0000 7fe8b6e00640 1 -- 172.21.15.46:0/2643204432 shutdown_connections 2024-03-29T08:22:27.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.474+0000 7fe8b6e00640 1 -- 172.21.15.46:0/2643204432 wait complete. 2024-03-29T08:22:27.497 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:21:08.061567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:21:09.336464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:22:27.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:22:27.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.638+0000 7f2b63000640 1 Processor -- start 2024-03-29T08:22:27.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.638+0000 7f2b63000640 1 -- start start 2024-03-29T08:22:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.638+0000 7f2b63000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2b5c15dfb0 0x7f2b5c0b1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.638+0000 7f2b63000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b5c098a40 0x7f2b5c098eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.638+0000 7f2b63000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b5c0cb3f0 con 0x7f2b5c0ced10 2024-03-29T08:22:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.638+0000 7f2b63000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b5c0b1e00 con 0x7f2b5c15dfb0 2024-03-29T08:22:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.638+0000 7f2b63000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c0993f0 con 0x7f2b5c098a40 2024-03-29T08:22:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.638+0000 7f2b53400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2b5c0ced10 0x7f2b5c0b3580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47974/0 (socket says 172.21.15.46:47974) 2024-03-29T08:22:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.638+0000 7f2b53400640 1 -- 172.21.15.46:0/3945896244 learned_addr learned my addr 172.21.15.46:0/3945896244 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.638+0000 7f2b5be00640 1 --2- 172.21.15.46:0/3945896244 >> v2:172.21.15.46:3300/0 conn(0x7f2b5c15dfb0 0x7f2b5c0b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.638+0000 7f2b61000640 1 --2- 172.21.15.46:0/3945896244 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b5c098a40 0x7f2b5c098eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.638+0000 7f2b5b400640 1 -- 172.21.15.46:0/3945896244 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525232050 0 0) 0x7f2b5c0cb3f0 con 0x7f2b5c0ced10 2024-03-29T08:22:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.638+0000 7f2b5b400640 1 -- 172.21.15.46:0/3945896244 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b40003530 con 0x7f2b5c0ced10 2024-03-29T08:22:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.638+0000 7f2b5be00640 1 -- 172.21.15.46:0/3945896244 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b5c098a40 msgr2=0x7f2b5c098eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.638+0000 7f2b5be00640 1 --2- 172.21.15.46:0/3945896244 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b5c098a40 0x7f2b5c098eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.638+0000 7f2b5be00640 1 -- 172.21.15.46:0/3945896244 >> v1:172.21.15.46:6789/0 conn(0x7f2b5c0ced10 legacy=0x7f2b5c0b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.638+0000 7f2b5be00640 1 -- 172.21.15.46:0/3945896244 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c0cb620 con 0x7f2b5c15dfb0 2024-03-29T08:22:27.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.642+0000 7f2b5be00640 1 --2- 172.21.15.46:0/3945896244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b5c15dfb0 0x7f2b5c0b1570 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0097c0 tx=0x7f2b4c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aed484174922dc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.642+0000 7f2b5b400640 1 -- 172.21.15.46:0/3945896244 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c0ab670 con 0x7f2b5c15dfb0 2024-03-29T08:22:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.642+0000 7f2b5b400640 1 -- 172.21.15.46:0/3945896244 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4c0ab7d0 con 0x7f2b5c15dfb0 2024-03-29T08:22:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.642+0000 7f2b5b400640 1 -- 172.21.15.46:0/3945896244 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c0abb60 con 0x7f2b5c15dfb0 2024-03-29T08:22:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.642+0000 7f2b63000640 1 -- 172.21.15.46:0/3945896244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b5c15dfb0 msgr2=0x7f2b5c0b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.642+0000 7f2b63000640 1 --2- 172.21.15.46:0/3945896244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b5c15dfb0 0x7f2b5c0b1570 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0097c0 tx=0x7f2b4c01c470 comp rx=0 tx=0).stop 2024-03-29T08:22:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.642+0000 7f2b63000640 1 -- 172.21.15.46:0/3945896244 shutdown_connections 2024-03-29T08:22:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.642+0000 7f2b63000640 1 --2- 172.21.15.46:0/3945896244 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b5c098a40 0x7f2b5c098eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.642+0000 7f2b63000640 1 --2- 172.21.15.46:0/3945896244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b5c15dfb0 0x7f2b5c0b1570 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.642+0000 7f2b63000640 1 -- 172.21.15.46:0/3945896244 >> 172.21.15.46:0/3945896244 conn(0x7f2b5c0b2220 msgr2=0x7f2b5c0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.642+0000 7f2b63000640 1 -- 172.21.15.46:0/3945896244 shutdown_connections 2024-03-29T08:22:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.642+0000 7f2b63000640 1 -- 172.21.15.46:0/3945896244 wait complete. 2024-03-29T08:22:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b63000640 1 Processor -- start 2024-03-29T08:22:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b63000640 1 -- start start 2024-03-29T08:22:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b63000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b5c098a40 0x7f2b5c144960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b63000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b5c0ced10 0x7f2b5c14cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b63000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b5c15dfb0 0x7f2b5c147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b63000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c144ea0 con 0x7f2b5c098a40 2024-03-29T08:22:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b63000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c14d3e0 con 0x7f2b5c15dfb0 2024-03-29T08:22:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b63000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c070340 con 0x7f2b5c0ced10 2024-03-29T08:22:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b5c098a40 0x7f2b5c144960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b5c098a40 0x7f2b5c144960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42070/0 (socket says 172.21.15.46:42070) 2024-03-29T08:22:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b5be00640 1 -- 172.21.15.46:0/3958323422 learned_addr learned my addr 172.21.15.46:0/3958323422 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b53400640 1 --2- 172.21.15.46:0/3958323422 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b5c0ced10 0x7f2b5c14cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b61000640 1 --2- 172.21.15.46:0/3958323422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b5c15dfb0 0x7f2b5c147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b5be00640 1 -- 172.21.15.46:0/3958323422 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b5c0ced10 msgr2=0x7f2b5c14cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b5be00640 1 --2- 172.21.15.46:0/3958323422 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b5c0ced10 0x7f2b5c14cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b5be00640 1 -- 172.21.15.46:0/3958323422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b5c15dfb0 msgr2=0x7f2b5c147d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b5be00640 1 --2- 172.21.15.46:0/3958323422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b5c15dfb0 0x7f2b5c147d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b5be00640 1 -- 172.21.15.46:0/3958323422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4c009420 con 0x7f2b5c098a40 2024-03-29T08:22:27.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b5be00640 1 --2- 172.21.15.46:0/3958323422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b5c098a40 0x7f2b5c144960 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0ad040 tx=0x7f2b4c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:27.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b58c00640 1 -- 172.21.15.46:0/3958323422 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c0bf610 con 0x7f2b5c098a40 2024-03-29T08:22:27.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b58c00640 1 -- 172.21.15.46:0/3958323422 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4c0bf7b0 con 0x7f2b5c098a40 2024-03-29T08:22:27.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b58c00640 1 -- 172.21.15.46:0/3958323422 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c0bfb60 con 0x7f2b5c098a40 2024-03-29T08:22:27.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b63000640 1 -- 172.21.15.46:0/3958323422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5c070060 con 0x7f2b5c098a40 2024-03-29T08:22:27.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b63000640 1 -- 172.21.15.46:0/3958323422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5c145760 con 0x7f2b5c098a40 2024-03-29T08:22:27.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.646+0000 7f2b63000640 1 -- 172.21.15.46:0/3958323422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b5c09e700 con 0x7f2b5c098a40 2024-03-29T08:22:27.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.650+0000 7f2b58c00640 1 -- 172.21.15.46:0/3958323422 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2b4c0bfcc0 con 0x7f2b5c098a40 2024-03-29T08:22:27.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.650+0000 7f2b58c00640 1 --2- 172.21.15.46:0/3958323422 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2b38066690 0x7f2b38068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.650+0000 7f2b58c00640 1 -- 172.21.15.46:0/3958323422 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2b4c11ef90 con 0x7f2b5c098a40 2024-03-29T08:22:27.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.650+0000 7f2b53400640 1 --2- 172.21.15.46:0/3958323422 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2b38066690 0x7f2b38068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.650+0000 7f2b53400640 1 --2- 172.21.15.46:0/3958323422 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2b38066690 0x7f2b38068b50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c14acf0 tx=0x7f2b44009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:27.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.654+0000 7f2b58c00640 1 -- 172.21.15.46:0/3958323422 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2b4c0ec2b0 con 0x7f2b5c098a40 2024-03-29T08:22:27.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.814+0000 7f2b63000640 1 -- 172.21.15.46:0/3958323422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2b5c04e120 con 0x7f2b5c098a40 2024-03-29T08:22:27.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.814+0000 7f2b58c00640 1 -- 172.21.15.46:0/3958323422 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b4c0ef8d0 con 0x7f2b5c098a40 2024-03-29T08:22:27.817 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:22:27.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.814+0000 7f2b63000640 1 -- 172.21.15.46:0/3958323422 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2b38066690 msgr2=0x7f2b38068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:27.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.814+0000 7f2b63000640 1 --2- 172.21.15.46:0/3958323422 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2b38066690 0x7f2b38068b50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c14acf0 tx=0x7f2b44009040 comp rx=0 tx=0).stop 2024-03-29T08:22:27.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.814+0000 7f2b63000640 1 -- 172.21.15.46:0/3958323422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b5c098a40 msgr2=0x7f2b5c144960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:27.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.814+0000 7f2b63000640 1 --2- 172.21.15.46:0/3958323422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b5c098a40 0x7f2b5c144960 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0ad040 tx=0x7f2b4c01acb0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.814+0000 7f2b63000640 1 -- 172.21.15.46:0/3958323422 shutdown_connections 2024-03-29T08:22:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.814+0000 7f2b63000640 1 --2- 172.21.15.46:0/3958323422 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2b38066690 0x7f2b38068b50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.814+0000 7f2b63000640 1 --2- 172.21.15.46:0/3958323422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b5c15dfb0 0x7f2b5c147d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.814+0000 7f2b63000640 1 --2- 172.21.15.46:0/3958323422 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b5c0ced10 0x7f2b5c14cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.814+0000 7f2b63000640 1 --2- 172.21.15.46:0/3958323422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b5c098a40 0x7f2b5c144960 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.814+0000 7f2b63000640 1 -- 172.21.15.46:0/3958323422 >> 172.21.15.46:0/3958323422 conn(0x7f2b5c0b2220 msgr2=0x7f2b5c0b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.814+0000 7f2b63000640 1 -- 172.21.15.46:0/3958323422 shutdown_connections 2024-03-29T08:22:27.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.814+0000 7f2b63000640 1 -- 172.21.15.46:0/3958323422 wait complete. 2024-03-29T08:22:27.839 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:22:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.978+0000 7ff704a00640 1 Processor -- start 2024-03-29T08:22:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.978+0000 7ff704a00640 1 -- start start 2024-03-29T08:22:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.982+0000 7ff704a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff7000cd3d0 0x7ff7000cba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.982+0000 7ff704a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70006ff00 con 0x7ff7000ccf60 2024-03-29T08:22:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.982+0000 7ff704a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7000cf590 con 0x7ff70015a9f0 2024-03-29T08:22:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.982+0000 7ff704a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70006e130 con 0x7ff7000cd3d0 2024-03-29T08:22:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.982+0000 7ff6f5e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff7000ccf60 0x7ff7000cb380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47984/0 (socket says 172.21.15.46:47984) 2024-03-29T08:22:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.982+0000 7ff6f5e00640 1 -- 172.21.15.46:0/1606877215 learned_addr learned my addr 172.21.15.46:0/1606877215 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:27.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.982+0000 7ff6ff200640 1 --2- 172.21.15.46:0/1606877215 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff7000cd3d0 0x7ff7000cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.982+0000 7ff6fde00640 1 -- 172.21.15.46:0/1606877215 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505683592 0 0) 0x7ff70006ff00 con 0x7ff7000ccf60 2024-03-29T08:22:27.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.982+0000 7ff6fde00640 1 -- 172.21.15.46:0/1606877215 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6e4003530 con 0x7ff7000ccf60 2024-03-29T08:22:27.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.982+0000 7ff6fde00640 1 -- 172.21.15.46:0/1606877215 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285137962 0 0) 0x7ff7000cf590 con 0x7ff70015a9f0 2024-03-29T08:22:27.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.982+0000 7ff6fde00640 1 -- 172.21.15.46:0/1606877215 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff70006ff00 con 0x7ff70015a9f0 2024-03-29T08:22:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.982+0000 7ff6ff200640 1 -- 172.21.15.46:0/1606877215 >> v1:172.21.15.73:6789/0 conn(0x7ff70015a9f0 legacy=0x7ff7000ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.982+0000 7ff6ff200640 1 -- 172.21.15.46:0/1606877215 >> v1:172.21.15.46:6789/0 conn(0x7ff7000ccf60 legacy=0x7ff7000cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:27.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.982+0000 7ff6ff200640 1 -- 172.21.15.46:0/1606877215 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70006e410 con 0x7ff7000cd3d0 2024-03-29T08:22:27.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.986+0000 7ff6ff200640 1 --2- 172.21.15.46:0/1606877215 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff7000cd3d0 0x7ff7000cba90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80097f0 tx=0x7ff6f801ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd65064df171929a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:27.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.986+0000 7ff6fde00640 1 -- 172.21.15.46:0/1606877215 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f80a89d0 con 0x7ff7000cd3d0 2024-03-29T08:22:27.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.986+0000 7ff6fde00640 1 -- 172.21.15.46:0/1606877215 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f80a8b30 con 0x7ff7000cd3d0 2024-03-29T08:22:27.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.986+0000 7ff6fde00640 1 -- 172.21.15.46:0/1606877215 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f80b0440 con 0x7ff7000cd3d0 2024-03-29T08:22:27.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.986+0000 7ff704a00640 1 -- 172.21.15.46:0/1606877215 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff7000cd3d0 msgr2=0x7ff7000cba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:27.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.986+0000 7ff704a00640 1 --2- 172.21.15.46:0/1606877215 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff7000cd3d0 0x7ff7000cba90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff6f80097f0 tx=0x7ff6f801ecd0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.986+0000 7ff704a00640 1 -- 172.21.15.46:0/1606877215 shutdown_connections 2024-03-29T08:22:27.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.986+0000 7ff704a00640 1 --2- 172.21.15.46:0/1606877215 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff7000cd3d0 0x7ff7000cba90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.986+0000 7ff704a00640 1 -- 172.21.15.46:0/1606877215 >> 172.21.15.46:0/1606877215 conn(0x7ff7000d0550 msgr2=0x7ff7000d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:27.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.986+0000 7ff704a00640 1 -- 172.21.15.46:0/1606877215 shutdown_connections 2024-03-29T08:22:27.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.986+0000 7ff704a00640 1 -- 172.21.15.46:0/1606877215 wait complete. 2024-03-29T08:22:27.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.986+0000 7ff704a00640 1 Processor -- start 2024-03-29T08:22:27.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.986+0000 7ff704a00640 1 -- start start 2024-03-29T08:22:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.986+0000 7ff704a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7000ccf60 0x7ff700159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.986+0000 7ff704a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff7000cd3d0 0x7ff7001ef620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff704a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff70015a9f0 0x7ff7001582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff704a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7000c8510 con 0x7ff7000ccf60 2024-03-29T08:22:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff704a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7001595d0 con 0x7ff70015a9f0 2024-03-29T08:22:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff704a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7000c7620 con 0x7ff7000cd3d0 2024-03-29T08:22:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff6fe800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7000ccf60 0x7ff700159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff6fe800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7000ccf60 0x7ff700159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42074/0 (socket says 172.21.15.46:42074) 2024-03-29T08:22:27.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff6fe800640 1 -- 172.21.15.46:0/4082806968 learned_addr learned my addr 172.21.15.46:0/4082806968 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff6f5e00640 1 --2- 172.21.15.46:0/4082806968 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff7000cd3d0 0x7ff7001ef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff6ff200640 1 --2- 172.21.15.46:0/4082806968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff70015a9f0 0x7ff7001582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff6fe800640 1 -- 172.21.15.46:0/4082806968 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff7000cd3d0 msgr2=0x7ff7001ef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff6fe800640 1 --2- 172.21.15.46:0/4082806968 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff7000cd3d0 0x7ff7001ef620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff6fe800640 1 -- 172.21.15.46:0/4082806968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff70015a9f0 msgr2=0x7ff7001582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff6fe800640 1 --2- 172.21.15.46:0/4082806968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff70015a9f0 0x7ff7001582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff6fe800640 1 -- 172.21.15.46:0/4082806968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f8009420 con 0x7ff7000ccf60 2024-03-29T08:22:27.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff6fe800640 1 --2- 172.21.15.46:0/4082806968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7000ccf60 0x7ff700159090 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000c870 tx=0x7ff6f000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:27.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff6f7400640 1 -- 172.21.15.46:0/4082806968 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0003b80 con 0x7ff7000ccf60 2024-03-29T08:22:27.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff6f7400640 1 -- 172.21.15.46:0/4082806968 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f0003ce0 con 0x7ff7000ccf60 2024-03-29T08:22:27.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff6f7400640 1 -- 172.21.15.46:0/4082806968 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f000aa30 con 0x7ff7000ccf60 2024-03-29T08:22:27.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff704a00640 1 -- 172.21.15.46:0/4082806968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7000c7900 con 0x7ff7000ccf60 2024-03-29T08:22:27.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff704a00640 1 -- 172.21.15.46:0/4082806968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7000c7ec0 con 0x7ff7000ccf60 2024-03-29T08:22:27.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff704a00640 1 -- 172.21.15.46:0/4082806968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff70004e040 con 0x7ff7000ccf60 2024-03-29T08:22:27.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff6f7400640 1 -- 172.21.15.46:0/4082806968 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff6f000ab90 con 0x7ff7000ccf60 2024-03-29T08:22:27.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff6f7400640 1 --2- 172.21.15.46:0/4082806968 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff6dc066690 0x7ff6dc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:27.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.990+0000 7ff6f7400640 1 -- 172.21.15.46:0/4082806968 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff6f0084070 con 0x7ff7000ccf60 2024-03-29T08:22:27.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.994+0000 7ff6f5e00640 1 --2- 172.21.15.46:0/4082806968 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff6dc066690 0x7ff6dc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:27.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.994+0000 7ff6f5e00640 1 --2- 172.21.15.46:0/4082806968 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff6dc066690 0x7ff6dc068b50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff7001f3360 tx=0x7ff6e8007d40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:27.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:27.998+0000 7ff6f7400640 1 -- 172.21.15.46:0/4082806968 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff6f00b5980 con 0x7ff7000ccf60 2024-03-29T08:22:28.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.154+0000 7ff704a00640 1 -- 172.21.15.46:0/4082806968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff700152990 con 0x7ff7000ccf60 2024-03-29T08:22:28.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.154+0000 7ff6f7400640 1 -- 172.21.15.46:0/4082806968 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff6f00513b0 con 0x7ff7000ccf60 2024-03-29T08:22:28.159 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:22:28.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.158+0000 7ff704a00640 1 -- 172.21.15.46:0/4082806968 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff6dc066690 msgr2=0x7ff6dc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:28.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.158+0000 7ff704a00640 1 --2- 172.21.15.46:0/4082806968 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff6dc066690 0x7ff6dc068b50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff7001f3360 tx=0x7ff6e8007d40 comp rx=0 tx=0).stop 2024-03-29T08:22:28.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.158+0000 7ff704a00640 1 -- 172.21.15.46:0/4082806968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7000ccf60 msgr2=0x7ff700159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:28.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.158+0000 7ff704a00640 1 --2- 172.21.15.46:0/4082806968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7000ccf60 0x7ff700159090 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff6f000c870 tx=0x7ff6f000cd30 comp rx=0 tx=0).stop 2024-03-29T08:22:28.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.158+0000 7ff704a00640 1 -- 172.21.15.46:0/4082806968 shutdown_connections 2024-03-29T08:22:28.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.158+0000 7ff704a00640 1 --2- 172.21.15.46:0/4082806968 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff6dc066690 0x7ff6dc068b50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.158+0000 7ff704a00640 1 --2- 172.21.15.46:0/4082806968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff70015a9f0 0x7ff7001582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.158+0000 7ff704a00640 1 --2- 172.21.15.46:0/4082806968 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff7000cd3d0 0x7ff7001ef620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.158+0000 7ff704a00640 1 --2- 172.21.15.46:0/4082806968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7000ccf60 0x7ff700159090 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.158+0000 7ff704a00640 1 -- 172.21.15.46:0/4082806968 >> 172.21.15.46:0/4082806968 conn(0x7ff7000d0550 msgr2=0x7ff700155370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:28.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.158+0000 7ff704a00640 1 -- 172.21.15.46:0/4082806968 shutdown_connections 2024-03-29T08:22:28.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.158+0000 7ff704a00640 1 -- 172.21.15.46:0/4082806968 wait complete. 2024-03-29T08:22:28.181 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:22:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.322+0000 7f8d70800640 1 Processor -- start 2024-03-29T08:22:28.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.322+0000 7f8d70800640 1 -- start start 2024-03-29T08:22:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.322+0000 7f8d70800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d6815dfb0 0x7f8d680b1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.322+0000 7f8d70800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d680cb3f0 con 0x7f8d680ced10 2024-03-29T08:22:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.322+0000 7f8d70800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d680cb620 con 0x7f8d68098a40 2024-03-29T08:22:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.322+0000 7f8d70800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d680b1e00 con 0x7f8d6815dfb0 2024-03-29T08:22:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.322+0000 7f8d6d400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8d680ced10 0x7f8d680b3580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47986/0 (socket says 172.21.15.46:47986) 2024-03-29T08:22:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.322+0000 7f8d6d400640 1 -- 172.21.15.46:0/612378168 learned_addr learned my addr 172.21.15.46:0/612378168 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:28.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.322+0000 7f8d6de00640 1 --2- 172.21.15.46:0/612378168 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d6815dfb0 0x7f8d680b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.322+0000 7f8d6ca00640 1 -- 172.21.15.46:0/612378168 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 748844185 0 0) 0x7f8d680cb3f0 con 0x7f8d680ced10 2024-03-29T08:22:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.322+0000 7f8d6ca00640 1 -- 172.21.15.46:0/612378168 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d50003530 con 0x7f8d680ced10 2024-03-29T08:22:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.322+0000 7f8d6ca00640 1 -- 172.21.15.46:0/612378168 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4040671967 0 0) 0x7f8d680cb620 con 0x7f8d68098a40 2024-03-29T08:22:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.322+0000 7f8d6de00640 1 -- 172.21.15.46:0/612378168 >> v1:172.21.15.73:6789/0 conn(0x7f8d68098a40 legacy=0x7f8d680cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.322+0000 7f8d6de00640 1 -- 172.21.15.46:0/612378168 >> v1:172.21.15.46:6789/0 conn(0x7f8d680ced10 legacy=0x7f8d680b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:28.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.322+0000 7f8d6de00640 1 -- 172.21.15.46:0/612378168 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d680cf4f0 con 0x7f8d6815dfb0 2024-03-29T08:22:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.326+0000 7f8d6de00640 1 --2- 172.21.15.46:0/612378168 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d6815dfb0 0x7f8d680b1570 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c0097f0 tx=0x7f8d5c01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a4407e767bbc05 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:28.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.326+0000 7f8d6ca00640 1 -- 172.21.15.46:0/612378168 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c0ac620 con 0x7f8d6815dfb0 2024-03-29T08:22:28.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.326+0000 7f8d6ca00640 1 -- 172.21.15.46:0/612378168 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5c0ac780 con 0x7f8d6815dfb0 2024-03-29T08:22:28.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.326+0000 7f8d6ca00640 1 -- 172.21.15.46:0/612378168 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5c0acb10 con 0x7f8d6815dfb0 2024-03-29T08:22:28.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.326+0000 7f8d70800640 1 -- 172.21.15.46:0/612378168 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d6815dfb0 msgr2=0x7f8d680b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:28.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.326+0000 7f8d70800640 1 --2- 172.21.15.46:0/612378168 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d6815dfb0 0x7f8d680b1570 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c0097f0 tx=0x7f8d5c01c4b0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.326+0000 7f8d70800640 1 -- 172.21.15.46:0/612378168 shutdown_connections 2024-03-29T08:22:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.326+0000 7f8d70800640 1 --2- 172.21.15.46:0/612378168 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d6815dfb0 0x7f8d680b1570 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.326+0000 7f8d70800640 1 -- 172.21.15.46:0/612378168 >> 172.21.15.46:0/612378168 conn(0x7f8d680b2220 msgr2=0x7f8d680b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.326+0000 7f8d70800640 1 -- 172.21.15.46:0/612378168 shutdown_connections 2024-03-29T08:22:28.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.326+0000 7f8d70800640 1 -- 172.21.15.46:0/612378168 wait complete. 2024-03-29T08:22:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d70800640 1 Processor -- start 2024-03-29T08:22:28.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d70800640 1 -- start start 2024-03-29T08:22:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d70800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d68098a40 0x7f8d680c6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d70800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d680ced10 0x7f8d6815d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d70800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d6815dfb0 0x7f8d6806b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d70800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d680c78a0 con 0x7f8d680ced10 2024-03-29T08:22:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d70800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d680c7b80 con 0x7f8d68098a40 2024-03-29T08:22:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d70800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d680c7e60 con 0x7f8d6815dfb0 2024-03-29T08:22:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d6d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d680ced10 0x7f8d6815d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d6d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d680ced10 0x7f8d6815d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42088/0 (socket says 172.21.15.46:42088) 2024-03-29T08:22:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d6d400640 1 -- 172.21.15.46:0/3538208943 learned_addr learned my addr 172.21.15.46:0/3538208943 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d6de00640 1 --2- 172.21.15.46:0/3538208943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d68098a40 0x7f8d680c6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:28.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d6e800640 1 --2- 172.21.15.46:0/3538208943 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d6815dfb0 0x7f8d6806b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d6d400640 1 -- 172.21.15.46:0/3538208943 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d6815dfb0 msgr2=0x7f8d6806b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d6d400640 1 --2- 172.21.15.46:0/3538208943 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d6815dfb0 0x7f8d6806b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d6d400640 1 -- 172.21.15.46:0/3538208943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d68098a40 msgr2=0x7f8d680c6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d6d400640 1 --2- 172.21.15.46:0/3538208943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d68098a40 0x7f8d680c6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d6d400640 1 -- 172.21.15.46:0/3538208943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5c009420 con 0x7f8d680ced10 2024-03-29T08:22:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d6de00640 1 --2- 172.21.15.46:0/3538208943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d68098a40 0x7f8d680c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:22:28.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d6d400640 1 --2- 172.21.15.46:0/3538208943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d680ced10 0x7f8d6815d0a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800c450 tx=0x7f8d5800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d62000640 1 -- 172.21.15.46:0/3538208943 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58005cf0 con 0x7f8d680ced10 2024-03-29T08:22:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d62000640 1 -- 172.21.15.46:0/3538208943 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d58005e50 con 0x7f8d680ced10 2024-03-29T08:22:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d62000640 1 -- 172.21.15.46:0/3538208943 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58007290 con 0x7f8d680ced10 2024-03-29T08:22:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d70800640 1 -- 172.21.15.46:0/3538208943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d680c8140 con 0x7f8d680ced10 2024-03-29T08:22:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d70800640 1 -- 172.21.15.46:0/3538208943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d680c7010 con 0x7f8d680ced10 2024-03-29T08:22:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.330+0000 7f8d70800640 1 -- 172.21.15.46:0/3538208943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d68072dc0 con 0x7f8d680ced10 2024-03-29T08:22:28.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.334+0000 7f8d62000640 1 -- 172.21.15.46:0/3538208943 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8d580073f0 con 0x7f8d680ced10 2024-03-29T08:22:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.334+0000 7f8d62000640 1 --2- 172.21.15.46:0/3538208943 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8d4c066690 0x7f8d4c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.334+0000 7f8d62000640 1 -- 172.21.15.46:0/3538208943 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8d580837b0 con 0x7f8d680ced10 2024-03-29T08:22:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.334+0000 7f8d6de00640 1 --2- 172.21.15.46:0/3538208943 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8d4c066690 0x7f8d4c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:28.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.334+0000 7f8d6de00640 1 --2- 172.21.15.46:0/3538208943 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8d4c066690 0x7f8d4c068b50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c0ae000 tx=0x7f8d5c0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:28.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.338+0000 7f8d62000640 1 -- 172.21.15.46:0/3538208943 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8d58050af0 con 0x7f8d680ced10 2024-03-29T08:22:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.498+0000 7f8d70800640 1 -- 172.21.15.46:0/3538208943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8d6809e700 con 0x7f8d680ced10 2024-03-29T08:22:28.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.498+0000 7f8d62000640 1 -- 172.21.15.46:0/3538208943 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8d58017020 con 0x7f8d680ced10 2024-03-29T08:22:28.502 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:22:28.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.502+0000 7f8d70800640 1 -- 172.21.15.46:0/3538208943 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8d4c066690 msgr2=0x7f8d4c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:28.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.502+0000 7f8d70800640 1 --2- 172.21.15.46:0/3538208943 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8d4c066690 0x7f8d4c068b50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c0ae000 tx=0x7f8d5c0a7000 comp rx=0 tx=0).stop 2024-03-29T08:22:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.502+0000 7f8d70800640 1 -- 172.21.15.46:0/3538208943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d680ced10 msgr2=0x7f8d6815d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.502+0000 7f8d70800640 1 --2- 172.21.15.46:0/3538208943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d680ced10 0x7f8d6815d0a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800c450 tx=0x7f8d5800c910 comp rx=0 tx=0).stop 2024-03-29T08:22:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.502+0000 7f8d70800640 1 -- 172.21.15.46:0/3538208943 shutdown_connections 2024-03-29T08:22:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.502+0000 7f8d70800640 1 --2- 172.21.15.46:0/3538208943 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8d4c066690 0x7f8d4c068b50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.502+0000 7f8d70800640 1 --2- 172.21.15.46:0/3538208943 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8d6815dfb0 0x7f8d6806b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.502+0000 7f8d70800640 1 --2- 172.21.15.46:0/3538208943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d680ced10 0x7f8d6815d0a0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.502+0000 7f8d70800640 1 --2- 172.21.15.46:0/3538208943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d68098a40 0x7f8d680c6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.502+0000 7f8d70800640 1 -- 172.21.15.46:0/3538208943 >> 172.21.15.46:0/3538208943 conn(0x7f8d680b2220 msgr2=0x7f8d68154400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.502+0000 7f8d70800640 1 -- 172.21.15.46:0/3538208943 shutdown_connections 2024-03-29T08:22:28.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.502+0000 7f8d70800640 1 -- 172.21.15.46:0/3538208943 wait complete. 2024-03-29T08:22:28.524 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:22:28.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d10e00640 1 Processor -- start 2024-03-29T08:22:28.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d10e00640 1 -- start start 2024-03-29T08:22:28.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d10e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6d0c159970 0x7f6d0c0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:28.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d10e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d0c0cbf60 0x7f6d0c0cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:28.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d10e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6d0c0ce100 0x7f6d0c0ca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:28.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d10e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d0c0ce590 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d10e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d0c0c34c0 con 0x7f6d0c0ce100 2024-03-29T08:22:28.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d10e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c0cc460 con 0x7f6d0c159970 2024-03-29T08:22:28.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d01400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d0c0cbf60 0x7f6d0c0cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:28.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d01400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d0c0cbf60 0x7f6d0c0cdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42090/0 (socket says 172.21.15.46:42090) 2024-03-29T08:22:28.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d01400640 1 -- 172.21.15.46:0/155367198 learned_addr learned my addr 172.21.15.46:0/155367198 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:28.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d0a800640 1 --2- 172.21.15.46:0/155367198 >> v2:172.21.15.73:3300/0 conn(0x7f6d0c0ce100 0x7f6d0c0ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:28.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d09e00640 1 --2- 172.21.15.46:0/155367198 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6d0c159970 0x7f6d0c0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:28.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d01400640 1 -- 172.21.15.46:0/155367198 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6d0c159970 msgr2=0x7f6d0c0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:28.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d01400640 1 --2- 172.21.15.46:0/155367198 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6d0c159970 0x7f6d0c0cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d01400640 1 -- 172.21.15.46:0/155367198 >> v2:172.21.15.73:3300/0 conn(0x7f6d0c0ce100 msgr2=0x7f6d0c0ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:28.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d01400640 1 --2- 172.21.15.46:0/155367198 >> v2:172.21.15.73:3300/0 conn(0x7f6d0c0ce100 0x7f6d0c0ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.662+0000 7f6d01400640 1 -- 172.21.15.46:0/155367198 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0c0cc740 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.666+0000 7f6d01400640 1 --2- 172.21.15.46:0/155367198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d0c0cbf60 0x7f6d0c0cdbc0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c072000 tx=0x7f6cf401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eacd0b6d4c21d77d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:28.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.666+0000 7f6d09400640 1 -- 172.21.15.46:0/155367198 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40aa610 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.666+0000 7f6d09400640 1 -- 172.21.15.46:0/155367198 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf40a6040 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.666+0000 7f6d09400640 1 -- 172.21.15.46:0/155367198 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf40aa990 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.666+0000 7f6d10e00640 1 -- 172.21.15.46:0/155367198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d0c0cbf60 msgr2=0x7f6d0c0cdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:28.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.666+0000 7f6d10e00640 1 --2- 172.21.15.46:0/155367198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d0c0cbf60 0x7f6d0c0cdbc0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c072000 tx=0x7f6cf401c4f0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.666+0000 7f6d10e00640 1 -- 172.21.15.46:0/155367198 shutdown_connections 2024-03-29T08:22:28.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.666+0000 7f6d10e00640 1 --2- 172.21.15.46:0/155367198 >> v2:172.21.15.73:3300/0 conn(0x7f6d0c0ce100 0x7f6d0c0ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.666+0000 7f6d10e00640 1 --2- 172.21.15.46:0/155367198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d0c0cbf60 0x7f6d0c0cdbc0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.666+0000 7f6d10e00640 1 --2- 172.21.15.46:0/155367198 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6d0c159970 0x7f6d0c0cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.666+0000 7f6d10e00640 1 -- 172.21.15.46:0/155367198 >> 172.21.15.46:0/155367198 conn(0x7f6d0c098a40 msgr2=0x7f6d0c0cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:28.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.666+0000 7f6d10e00640 1 -- 172.21.15.46:0/155367198 shutdown_connections 2024-03-29T08:22:28.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.666+0000 7f6d10e00640 1 -- 172.21.15.46:0/155367198 wait complete. 2024-03-29T08:22:28.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d10e00640 1 Processor -- start 2024-03-29T08:22:28.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d10e00640 1 -- start start 2024-03-29T08:22:28.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d10e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d0c0cbf60 0x7f6d0c150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:28.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d10e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6d0c0ce100 0x7f6d0c0c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d10e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d0c159970 0x7f6d0c152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d10e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c0c3f10 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d10e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c0c57a0 con 0x7f6d0c159970 2024-03-29T08:22:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d10e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c0c5a80 con 0x7f6d0c0ce100 2024-03-29T08:22:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d09e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d0c0cbf60 0x7f6d0c150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d09e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d0c0cbf60 0x7f6d0c150c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42100/0 (socket says 172.21.15.46:42100) 2024-03-29T08:22:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d09e00640 1 -- 172.21.15.46:0/1890774182 learned_addr learned my addr 172.21.15.46:0/1890774182 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d01400640 1 --2- 172.21.15.46:0/1890774182 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6d0c0ce100 0x7f6d0c0c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d0a800640 1 --2- 172.21.15.46:0/1890774182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d0c159970 0x7f6d0c152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d09e00640 1 -- 172.21.15.46:0/1890774182 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6d0c0ce100 msgr2=0x7f6d0c0c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d09e00640 1 --2- 172.21.15.46:0/1890774182 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6d0c0ce100 0x7f6d0c0c4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d09e00640 1 -- 172.21.15.46:0/1890774182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d0c159970 msgr2=0x7f6d0c152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d09e00640 1 --2- 172.21.15.46:0/1890774182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d0c159970 0x7f6d0c152a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d09e00640 1 -- 172.21.15.46:0/1890774182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf40094d0 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d09e00640 1 --2- 172.21.15.46:0/1890774182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d0c0cbf60 0x7f6d0c150c30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc007e40 tx=0x7f6cfc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d02a00640 1 -- 172.21.15.46:0/1890774182 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc014620 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d02a00640 1 -- 172.21.15.46:0/1890774182 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cfc00ee90 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d02a00640 1 -- 172.21.15.46:0/1890774182 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc0149c0 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d10e00640 1 -- 172.21.15.46:0/1890774182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0c0c5d60 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.670+0000 7f6d10e00640 1 -- 172.21.15.46:0/1890774182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0c151170 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.674+0000 7f6d10e00640 1 -- 172.21.15.46:0/1890774182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0c099080 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.674+0000 7f6d02a00640 1 -- 172.21.15.46:0/1890774182 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6cfc014b20 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.674+0000 7f6d02a00640 1 --2- 172.21.15.46:0/1890774182 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ce8066690 0x7f6ce8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:28.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.674+0000 7f6d02a00640 1 -- 172.21.15.46:0/1890774182 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6cfc084bb0 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.674+0000 7f6d01400640 1 --2- 172.21.15.46:0/1890774182 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ce8066690 0x7f6ce8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:28.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.674+0000 7f6d01400640 1 --2- 172.21.15.46:0/1890774182 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ce8066690 0x7f6ce8068b50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40ae000 tx=0x7f6cf40a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:28.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.678+0000 7f6d02a00640 1 -- 172.21.15.46:0/1890774182 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6cfc051e40 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.850+0000 7f6d10e00640 1 -- 172.21.15.46:0/1890774182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6d0c0c5f50 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.850+0000 7f6d02a00640 1 -- 172.21.15.46:0/1890774182 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v65) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6cfc055460 con 0x7f6d0c0cbf60 2024-03-29T08:22:28.852 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:22:28.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.850+0000 7f6d10e00640 1 -- 172.21.15.46:0/1890774182 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ce8066690 msgr2=0x7f6ce8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:28.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.850+0000 7f6d10e00640 1 --2- 172.21.15.46:0/1890774182 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ce8066690 0x7f6ce8068b50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40ae000 tx=0x7f6cf40a7000 comp rx=0 tx=0).stop 2024-03-29T08:22:28.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.850+0000 7f6d10e00640 1 -- 172.21.15.46:0/1890774182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d0c0cbf60 msgr2=0x7f6d0c150c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:28.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.850+0000 7f6d10e00640 1 --2- 172.21.15.46:0/1890774182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d0c0cbf60 0x7f6d0c150c30 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc007e40 tx=0x7f6cfc00e4b0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.850+0000 7f6d10e00640 1 -- 172.21.15.46:0/1890774182 shutdown_connections 2024-03-29T08:22:28.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.850+0000 7f6d10e00640 1 --2- 172.21.15.46:0/1890774182 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ce8066690 0x7f6ce8068b50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.850+0000 7f6d10e00640 1 --2- 172.21.15.46:0/1890774182 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d0c159970 0x7f6d0c152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.850+0000 7f6d10e00640 1 --2- 172.21.15.46:0/1890774182 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6d0c0ce100 0x7f6d0c0c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.850+0000 7f6d10e00640 1 --2- 172.21.15.46:0/1890774182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d0c0cbf60 0x7f6d0c150c30 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:28.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.850+0000 7f6d10e00640 1 -- 172.21.15.46:0/1890774182 >> 172.21.15.46:0/1890774182 conn(0x7f6d0c098a40 msgr2=0x7f6d0c06dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:28.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.850+0000 7f6d10e00640 1 -- 172.21.15.46:0/1890774182 shutdown_connections 2024-03-29T08:22:28.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:28.850+0000 7f6d10e00640 1 -- 172.21.15.46:0/1890774182 wait complete. 2024-03-29T08:22:28.874 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-03-29T08:22:28.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:28.875 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T08:22:28.894 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[39828]: fuse finished with error 0 and tester_r 0 2024-03-29T08:22:28.895 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:22:28.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:28.896 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:22:28.989 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:22:28.989 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:22:28.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:22:29.000 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:22:29.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:29.000 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T08:22:29.055 INFO:teuthology.orchestra.run.smithi073.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-29T08:22:29.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:22:29.056 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:22:29.056 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:22:29.065 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:22:29.065 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:22:29.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:29.066 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-29T08:22:29.114 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-29T08:22:29.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:22:29.115 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:22:29.116 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:22:29.125 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:22:29.125 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:22:29.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:29.125 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-29T08:22:29.174 INFO:teuthology.orchestra.run.smithi073.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-29T08:22:29.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:22:29.175 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:22:29.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.310+0000 7f863c400640 1 Processor -- start 2024-03-29T08:22:29.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.314+0000 7f863c400640 1 -- start start 2024-03-29T08:22:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.314+0000 7f863c400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f863415ad20 0x7f86340cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.314+0000 7f863c400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f86340b0540 0x7f86340cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.314+0000 7f863c400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86340b0ab0 con 0x7f863415a940 2024-03-29T08:22:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.314+0000 7f863c400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86340b0d90 con 0x7f86340b0540 2024-03-29T08:22:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.314+0000 7f863c400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86340b1070 con 0x7f863415ad20 2024-03-29T08:22:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.314+0000 7f8639000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f863415ad20 0x7f86340cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.314+0000 7f863a400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f86340b0540 0x7f86340cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.314+0000 7f8639000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f863415ad20 0x7f86340cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:54278/0 (socket says 172.21.15.46:54278) 2024-03-29T08:22:29.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.314+0000 7f8639000640 1 -- 172.21.15.46:0/4086703255 learned_addr learned my addr 172.21.15.46:0/4086703255 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:29.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.314+0000 7f8639000640 1 -- 172.21.15.46:0/4086703255 >> v2:172.21.15.73:3300/0 conn(0x7f86340b0540 msgr2=0x7f86340cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:29.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.314+0000 7f8639000640 1 --2- 172.21.15.46:0/4086703255 >> v2:172.21.15.73:3300/0 conn(0x7f86340b0540 0x7f86340cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.314+0000 7f8639000640 1 -- 172.21.15.46:0/4086703255 >> v1:172.21.15.73:6789/0 conn(0x7f863415a940 legacy=0x7f86340afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:29.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.314+0000 7f8639000640 1 -- 172.21.15.46:0/4086703255 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86340b1350 con 0x7f863415ad20 2024-03-29T08:22:29.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.318+0000 7f8639000640 1 --2- 172.21.15.46:0/4086703255 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f863415ad20 0x7f86340cecb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8620009870 tx=0x7f862001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4c1ddaf747da9d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:29.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.318+0000 7f8633e00640 1 -- 172.21.15.46:0/4086703255 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86200a9610 con 0x7f863415ad20 2024-03-29T08:22:29.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.318+0000 7f8633e00640 1 -- 172.21.15.46:0/4086703255 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86200a9770 con 0x7f863415ad20 2024-03-29T08:22:29.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.318+0000 7f8633e00640 1 -- 172.21.15.46:0/4086703255 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86200a9b00 con 0x7f863415ad20 2024-03-29T08:22:29.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.318+0000 7f863c400640 1 -- 172.21.15.46:0/4086703255 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f863415ad20 msgr2=0x7f86340cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:29.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.318+0000 7f863c400640 1 --2- 172.21.15.46:0/4086703255 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f863415ad20 0x7f86340cecb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8620009870 tx=0x7f862001c530 comp rx=0 tx=0).stop 2024-03-29T08:22:29.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.318+0000 7f863c400640 1 -- 172.21.15.46:0/4086703255 shutdown_connections 2024-03-29T08:22:29.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.318+0000 7f863c400640 1 --2- 172.21.15.46:0/4086703255 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f863415ad20 0x7f86340cecb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.318+0000 7f863c400640 1 --2- 172.21.15.46:0/4086703255 >> v2:172.21.15.73:3300/0 conn(0x7f86340b0540 0x7f86340cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.318+0000 7f863c400640 1 -- 172.21.15.46:0/4086703255 >> 172.21.15.46:0/4086703255 conn(0x7f8634089fd0 msgr2=0x7f863408a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:29.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.318+0000 7f863c400640 1 -- 172.21.15.46:0/4086703255 shutdown_connections 2024-03-29T08:22:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.318+0000 7f863c400640 1 -- 172.21.15.46:0/4086703255 wait complete. 2024-03-29T08:22:29.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.318+0000 7f863c400640 1 Processor -- start 2024-03-29T08:22:29.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.318+0000 7f863c400640 1 -- start start 2024-03-29T08:22:29.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863c400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86340b0540 0x7f8634150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:29.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863c400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f863415a940 0x7f86340cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:29.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863c400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f863415ad20 0x7f86340cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:29.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863c400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863406cb70 con 0x7f863415ad20 2024-03-29T08:22:29.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863c400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863406bef0 con 0x7f863415a940 2024-03-29T08:22:29.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863c400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863406c1d0 con 0x7f86340b0540 2024-03-29T08:22:29.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f863415ad20 0x7f86340cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:29.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f863415ad20 0x7f86340cd610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42116/0 (socket says 172.21.15.46:42116) 2024-03-29T08:22:29.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863a400640 1 -- 172.21.15.46:0/3479097207 learned_addr learned my addr 172.21.15.46:0/3479097207 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:29.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f8639000640 1 --2- 172.21.15.46:0/3479097207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f863415a940 0x7f86340cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:29.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f8639a00640 1 --2- 172.21.15.46:0/3479097207 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86340b0540 0x7f8634150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:29.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863a400640 1 -- 172.21.15.46:0/3479097207 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86340b0540 msgr2=0x7f8634150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:29.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863a400640 1 --2- 172.21.15.46:0/3479097207 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86340b0540 0x7f8634150d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863a400640 1 -- 172.21.15.46:0/3479097207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f863415a940 msgr2=0x7f86340cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:29.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863a400640 1 --2- 172.21.15.46:0/3479097207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f863415a940 0x7f86340cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863a400640 1 -- 172.21.15.46:0/3479097207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86200094d0 con 0x7f863415ad20 2024-03-29T08:22:29.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f8639000640 1 --2- 172.21.15.46:0/3479097207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f863415a940 0x7f86340cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:22:29.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863a400640 1 --2- 172.21.15.46:0/3479097207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f863415ad20 0x7f86340cd610 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f862c00efc0 tx=0x7f862c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:29.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f8631600640 1 -- 172.21.15.46:0/3479097207 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c014480 con 0x7f863415ad20 2024-03-29T08:22:29.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f8631600640 1 -- 172.21.15.46:0/3479097207 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862c00f040 con 0x7f863415ad20 2024-03-29T08:22:29.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f8631600640 1 -- 172.21.15.46:0/3479097207 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c014820 con 0x7f863415ad20 2024-03-29T08:22:29.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863c400640 1 -- 172.21.15.46:0/3479097207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f863406c4b0 con 0x7f863415ad20 2024-03-29T08:22:29.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863c400640 1 -- 172.21.15.46:0/3479097207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863406b180 con 0x7f863415ad20 2024-03-29T08:22:29.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f863c400640 1 -- 172.21.15.46:0/3479097207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f863406ccb0 con 0x7f863415ad20 2024-03-29T08:22:29.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.322+0000 7f8631600640 1 -- 172.21.15.46:0/3479097207 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f862c014980 con 0x7f863415ad20 2024-03-29T08:22:29.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.326+0000 7f8631600640 1 --2- 172.21.15.46:0/3479097207 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8614066690 0x7f8614068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:29.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.326+0000 7f8631600640 1 -- 172.21.15.46:0/3479097207 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f862c084c60 con 0x7f863415ad20 2024-03-29T08:22:29.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.326+0000 7f8639a00640 1 --2- 172.21.15.46:0/3479097207 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8614066690 0x7f8614068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:29.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.326+0000 7f8639a00640 1 --2- 172.21.15.46:0/3479097207 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8614066690 0x7f8614068b50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8628007fd0 tx=0x7f8628009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:29.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.330+0000 7f8631600640 1 -- 172.21.15.46:0/3479097207 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f863406ccb0 con 0x7f863415ad20 2024-03-29T08:22:29.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.522+0000 7f863c400640 1 -- 172.21.15.46:0/3479097207 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f863406ccb0 con 0x7f863415ad20 2024-03-29T08:22:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.522+0000 7f8631600640 1 -- 172.21.15.46:0/3479097207 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4729 (secure 0 0 0) 0x7f863406ccb0 con 0x7f863415ad20 2024-03-29T08:22:29.527 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:29.527 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2187826146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2187826146},{"type":"v1","addr":"172.21.15.73:6836","nonce":2187826146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/73382477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":73382477},{"type":"v1","addr":"172.21.15.73:6837","nonce":73382477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2874958170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2874958170},{"type":"v1","addr":"172.21.15.46:6834","nonce":2874958170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:21:10.337267+0000","modified":"2024-03-29T08:22:02.946081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6051},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6051":{"gid":6051,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":16,"addr":"172.21.15.46:6835/3255474176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3255474176},{"type":"v1","addr":"172.21.15.46:6835","nonce":3255474176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-29T08:22:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2024-03-29T08:22:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.526+0000 7f863c400640 1 -- 172.21.15.46:0/3479097207 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8614066690 msgr2=0x7f8614068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.526+0000 7f863c400640 1 --2- 172.21.15.46:0/3479097207 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8614066690 0x7f8614068b50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8628007fd0 tx=0x7f8628009040 comp rx=0 tx=0).stop 2024-03-29T08:22:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.526+0000 7f863c400640 1 -- 172.21.15.46:0/3479097207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f863415ad20 msgr2=0x7f86340cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.526+0000 7f863c400640 1 --2- 172.21.15.46:0/3479097207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f863415ad20 0x7f86340cd610 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f862c00efc0 tx=0x7f862c00c490 comp rx=0 tx=0).stop 2024-03-29T08:22:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.526+0000 7f863c400640 1 -- 172.21.15.46:0/3479097207 shutdown_connections 2024-03-29T08:22:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.526+0000 7f863c400640 1 --2- 172.21.15.46:0/3479097207 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8614066690 0x7f8614068b50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.526+0000 7f863c400640 1 --2- 172.21.15.46:0/3479097207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f863415a940 0x7f86340cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.526+0000 7f863c400640 1 --2- 172.21.15.46:0/3479097207 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f863415ad20 0x7f86340cd610 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.526+0000 7f863c400640 1 --2- 172.21.15.46:0/3479097207 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86340b0540 0x7f8634150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.526+0000 7f863c400640 1 -- 172.21.15.46:0/3479097207 >> 172.21.15.46:0/3479097207 conn(0x7f8634089fd0 msgr2=0x7f863406f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:29.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.526+0000 7f863c400640 1 -- 172.21.15.46:0/3479097207 shutdown_connections 2024-03-29T08:22:29.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.526+0000 7f863c400640 1 -- 172.21.15.46:0/3479097207 wait complete. 2024-03-29T08:22:29.548 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:22:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6f1c00640 1 Processor -- start 2024-03-29T08:22:29.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6f1c00640 1 -- start start 2024-03-29T08:22:29.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6f1c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb6ec15a940 0x7fb6ec0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6f1c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6ec15ad20 0x7fb6ec0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6f1c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6ec0b0540 0x7fb6ec0cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6f1c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ec0b00d0 con 0x7fb6ec15ad20 2024-03-29T08:22:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6f1c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ec0b0ab0 con 0x7fb6ec15a940 2024-03-29T08:22:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6f1c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec0b0d90 con 0x7fb6ec0b0540 2024-03-29T08:22:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6e2000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6ec15ad20 0x7fb6ec0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6e2000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6ec15ad20 0x7fb6ec0cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42122/0 (socket says 172.21.15.46:42122) 2024-03-29T08:22:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6e2000640 1 -- 172.21.15.46:0/2671515917 learned_addr learned my addr 172.21.15.46:0/2671515917 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6eaa00640 1 --2- 172.21.15.46:0/2671515917 >> v2:172.21.15.73:3300/0 conn(0x7fb6ec15a940 0x7fb6ec0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6eb400640 1 --2- 172.21.15.46:0/2671515917 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6ec0b0540 0x7fb6ec0cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6e2000640 1 -- 172.21.15.46:0/2671515917 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6ec0b0540 msgr2=0x7fb6ec0cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6e2000640 1 --2- 172.21.15.46:0/2671515917 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6ec0b0540 0x7fb6ec0cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6e2000640 1 -- 172.21.15.46:0/2671515917 >> v2:172.21.15.73:3300/0 conn(0x7fb6ec15a940 msgr2=0x7fb6ec0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:29.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6e2000640 1 --2- 172.21.15.46:0/2671515917 >> v2:172.21.15.73:3300/0 conn(0x7fb6ec15a940 0x7fb6ec0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.686+0000 7fb6e2000640 1 -- 172.21.15.46:0/2671515917 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ec0b1070 con 0x7fb6ec15ad20 2024-03-29T08:22:29.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.690+0000 7fb6e2000640 1 --2- 172.21.15.46:0/2671515917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6ec15ad20 0x7fb6ec0cecb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec070d60 tx=0x7fb6d401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab9cdda4e0ebaea server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:29.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.690+0000 7fb6ea000640 1 -- 172.21.15.46:0/2671515917 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40a7070 con 0x7fb6ec15ad20 2024-03-29T08:22:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.690+0000 7fb6ea000640 1 -- 172.21.15.46:0/2671515917 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d40ab430 con 0x7fb6ec15ad20 2024-03-29T08:22:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.690+0000 7fb6ea000640 1 -- 172.21.15.46:0/2671515917 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40ab7c0 con 0x7fb6ec15ad20 2024-03-29T08:22:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.690+0000 7fb6f1c00640 1 -- 172.21.15.46:0/2671515917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6ec15ad20 msgr2=0x7fb6ec0cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.690+0000 7fb6f1c00640 1 --2- 172.21.15.46:0/2671515917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6ec15ad20 0x7fb6ec0cecb0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec070d60 tx=0x7fb6d401c4c0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.690+0000 7fb6f1c00640 1 -- 172.21.15.46:0/2671515917 shutdown_connections 2024-03-29T08:22:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.690+0000 7fb6f1c00640 1 --2- 172.21.15.46:0/2671515917 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6ec0b0540 0x7fb6ec0cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.690+0000 7fb6f1c00640 1 --2- 172.21.15.46:0/2671515917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6ec15ad20 0x7fb6ec0cecb0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.690+0000 7fb6f1c00640 1 --2- 172.21.15.46:0/2671515917 >> v2:172.21.15.73:3300/0 conn(0x7fb6ec15a940 0x7fb6ec0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.690+0000 7fb6f1c00640 1 -- 172.21.15.46:0/2671515917 >> 172.21.15.46:0/2671515917 conn(0x7fb6ec089fd0 msgr2=0x7fb6ec08a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.690+0000 7fb6f1c00640 1 -- 172.21.15.46:0/2671515917 shutdown_connections 2024-03-29T08:22:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.690+0000 7fb6f1c00640 1 -- 172.21.15.46:0/2671515917 wait complete. 2024-03-29T08:22:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6f1c00640 1 Processor -- start 2024-03-29T08:22:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6f1c00640 1 -- start start 2024-03-29T08:22:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6f1c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6ec0b0540 0x7fb6ec15d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6f1c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6ec15a940 0x7fb6ec163bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6f1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6ec15ad20 0x7fb6ec06c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6f1c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec15d290 con 0x7fb6ec15ad20 2024-03-29T08:22:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6f1c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec164130 con 0x7fb6ec0b0540 2024-03-29T08:22:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6f1c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec15c680 con 0x7fb6ec15a940 2024-03-29T08:22:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6eb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6ec15ad20 0x7fb6ec06c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6eb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6ec15ad20 0x7fb6ec06c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42132/0 (socket says 172.21.15.46:42132) 2024-03-29T08:22:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6eb400640 1 -- 172.21.15.46:0/3608444074 learned_addr learned my addr 172.21.15.46:0/3608444074 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6eaa00640 1 --2- 172.21.15.46:0/3608444074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6ec0b0540 0x7fb6ec15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:29.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6e2000640 1 --2- 172.21.15.46:0/3608444074 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6ec15a940 0x7fb6ec163bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6eb400640 1 -- 172.21.15.46:0/3608444074 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6ec15a940 msgr2=0x7fb6ec163bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6eb400640 1 --2- 172.21.15.46:0/3608444074 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6ec15a940 0x7fb6ec163bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6eb400640 1 -- 172.21.15.46:0/3608444074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6ec0b0540 msgr2=0x7fb6ec15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6eb400640 1 --2- 172.21.15.46:0/3608444074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6ec0b0540 0x7fb6ec15d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6eb400640 1 -- 172.21.15.46:0/3608444074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d40094d0 con 0x7fb6ec15ad20 2024-03-29T08:22:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6eb400640 1 --2- 172.21.15.46:0/3608444074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6ec15ad20 0x7fb6ec06c870 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4007d20 tx=0x7fb6e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6e3400640 1 -- 172.21.15.46:0/3608444074 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4014420 con 0x7fb6ec15ad20 2024-03-29T08:22:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6e3400640 1 -- 172.21.15.46:0/3608444074 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e40145a0 con 0x7fb6ec15ad20 2024-03-29T08:22:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6f1c00640 1 -- 172.21.15.46:0/3608444074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ec15c960 con 0x7fb6ec15ad20 2024-03-29T08:22:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6e3400640 1 -- 172.21.15.46:0/3608444074 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4014950 con 0x7fb6ec15ad20 2024-03-29T08:22:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.694+0000 7fb6f1c00640 1 -- 172.21.15.46:0/3608444074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ec15cf20 con 0x7fb6ec15ad20 2024-03-29T08:22:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.698+0000 7fb6f1c00640 1 -- 172.21.15.46:0/3608444074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ec15d400 con 0x7fb6ec15ad20 2024-03-29T08:22:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.698+0000 7fb6e3400640 1 -- 172.21.15.46:0/3608444074 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb6e4014ab0 con 0x7fb6ec15ad20 2024-03-29T08:22:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.698+0000 7fb6e3400640 1 --2- 172.21.15.46:0/3608444074 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6c80666c0 0x7fb6c8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:29.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.698+0000 7fb6e3400640 1 -- 172.21.15.46:0/3608444074 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb6e4084920 con 0x7fb6ec15ad20 2024-03-29T08:22:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.698+0000 7fb6eaa00640 1 --2- 172.21.15.46:0/3608444074 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6c80666c0 0x7fb6c8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.698+0000 7fb6eaa00640 1 --2- 172.21.15.46:0/3608444074 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6c80666c0 0x7fb6c8068b80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc007a60 tx=0x7fb6dc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.702+0000 7fb6e3400640 1 -- 172.21.15.46:0/3608444074 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb6e4051cf0 con 0x7fb6ec15ad20 2024-03-29T08:22:29.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.866+0000 7fb6f1c00640 1 -- 172.21.15.46:0/3608444074 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6ec066c90 con 0x7fb6ec15ad20 2024-03-29T08:22:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.870+0000 7fb6e3400640 1 -- 172.21.15.46:0/3608444074 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16145 (secure 0 0 0) 0x7fb6e4055310 con 0x7fb6ec15ad20 2024-03-29T08:22:29.872 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:29.872 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":65,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:21:58.933917+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:21:08.061567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T08:21:09.336464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.2f","mappings":[{"from":6,"to":1}]},{"pgid":"9.3b","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:22:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.870+0000 7fb6f1c00640 1 -- 172.21.15.46:0/3608444074 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6c80666c0 msgr2=0x7fb6c8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.870+0000 7fb6f1c00640 1 --2- 172.21.15.46:0/3608444074 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6c80666c0 0x7fb6c8068b80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc007a60 tx=0x7fb6dc008040 comp rx=0 tx=0).stop 2024-03-29T08:22:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.870+0000 7fb6f1c00640 1 -- 172.21.15.46:0/3608444074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6ec15ad20 msgr2=0x7fb6ec06c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.870+0000 7fb6f1c00640 1 --2- 172.21.15.46:0/3608444074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6ec15ad20 0x7fb6ec06c870 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4007d20 tx=0x7fb6e400e4b0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.870+0000 7fb6f1c00640 1 -- 172.21.15.46:0/3608444074 shutdown_connections 2024-03-29T08:22:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.870+0000 7fb6f1c00640 1 --2- 172.21.15.46:0/3608444074 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6c80666c0 0x7fb6c8068b80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.870+0000 7fb6f1c00640 1 --2- 172.21.15.46:0/3608444074 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6ec15ad20 0x7fb6ec06c870 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.870+0000 7fb6f1c00640 1 --2- 172.21.15.46:0/3608444074 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6ec15a940 0x7fb6ec163bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.870+0000 7fb6f1c00640 1 --2- 172.21.15.46:0/3608444074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6ec0b0540 0x7fb6ec15d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.870+0000 7fb6f1c00640 1 -- 172.21.15.46:0/3608444074 >> 172.21.15.46:0/3608444074 conn(0x7fb6ec089fd0 msgr2=0x7fb6ec159750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.870+0000 7fb6f1c00640 1 -- 172.21.15.46:0/3608444074 shutdown_connections 2024-03-29T08:22:29.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:29.870+0000 7fb6f1c00640 1 -- 172.21.15.46:0/3608444074 wait complete. 2024-03-29T08:22:29.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:21:08.061567+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:21:09.336464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:22:29.893 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:22:30.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.034+0000 7efc15e00640 1 Processor -- start 2024-03-29T08:22:30.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.034+0000 7efc15e00640 1 -- start start 2024-03-29T08:22:30.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.034+0000 7efc15e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc100cea30 0x7efc101606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:30.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.034+0000 7efc15e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc100cfb20 con 0x7efc1015cd30 2024-03-29T08:22:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.034+0000 7efc15e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc100cef10 con 0x7efc1015c950 2024-03-29T08:22:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.034+0000 7efc15e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc100cf1f0 con 0x7efc100cea30 2024-03-29T08:22:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.034+0000 7efc07e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc100cea30 0x7efc101606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.034+0000 7efc0f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efc1015cd30 0x7efc100cad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47990/0 (socket says 172.21.15.46:47990) 2024-03-29T08:22:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.034+0000 7efc0f400640 1 -- 172.21.15.46:0/3917732025 learned_addr learned my addr 172.21.15.46:0/3917732025 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.034+0000 7efc07e00640 1 -- 172.21.15.46:0/3917732025 >> v1:172.21.15.73:6789/0 conn(0x7efc1015c950 legacy=0x7efc1006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.034+0000 7efc07e00640 1 -- 172.21.15.46:0/3917732025 >> v1:172.21.15.46:6789/0 conn(0x7efc1015cd30 legacy=0x7efc100cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.034+0000 7efc07e00640 1 -- 172.21.15.46:0/3917732025 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc100cf4d0 con 0x7efc100cea30 2024-03-29T08:22:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.038+0000 7efc07e00640 1 --2- 172.21.15.46:0/3917732025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc100cea30 0x7efc101606a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7efbf80097c0 tx=0x7efbf801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc3a3a0834ac8cd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.038+0000 7efc0e000640 1 -- 172.21.15.46:0/3917732025 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf80a8990 con 0x7efc100cea30 2024-03-29T08:22:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.038+0000 7efc0e000640 1 -- 172.21.15.46:0/3917732025 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf80a8af0 con 0x7efc100cea30 2024-03-29T08:22:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.038+0000 7efc0e000640 1 -- 172.21.15.46:0/3917732025 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf80a8e80 con 0x7efc100cea30 2024-03-29T08:22:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.038+0000 7efc15e00640 1 -- 172.21.15.46:0/3917732025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc100cea30 msgr2=0x7efc101606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.038+0000 7efc15e00640 1 --2- 172.21.15.46:0/3917732025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc100cea30 0x7efc101606a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7efbf80097c0 tx=0x7efbf801ec90 comp rx=0 tx=0).stop 2024-03-29T08:22:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.038+0000 7efc15e00640 1 -- 172.21.15.46:0/3917732025 shutdown_connections 2024-03-29T08:22:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.038+0000 7efc15e00640 1 --2- 172.21.15.46:0/3917732025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc100cea30 0x7efc101606a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.038+0000 7efc15e00640 1 -- 172.21.15.46:0/3917732025 >> 172.21.15.46:0/3917732025 conn(0x7efc100b21b0 msgr2=0x7efc100b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.038+0000 7efc15e00640 1 -- 172.21.15.46:0/3917732025 shutdown_connections 2024-03-29T08:22:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.038+0000 7efc15e00640 1 -- 172.21.15.46:0/3917732025 wait complete. 2024-03-29T08:22:30.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc15e00640 1 Processor -- start 2024-03-29T08:22:30.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc15e00640 1 -- start start 2024-03-29T08:22:30.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc15e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc100cea30 0x7efc1009de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:30.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc15e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1015c950 0x7efc10153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:30.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc15e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efc1015cd30 0x7efc10098a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc15e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1009b960 con 0x7efc1015c950 2024-03-29T08:22:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc15e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1009ce20 con 0x7efc100cea30 2024-03-29T08:22:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc15e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc10153790 con 0x7efc1015cd30 2024-03-29T08:22:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1015c950 0x7efc10153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1015c950 0x7efc10153250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42138/0 (socket says 172.21.15.46:42138) 2024-03-29T08:22:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc07e00640 1 -- 172.21.15.46:0/1625879215 learned_addr learned my addr 172.21.15.46:0/1625879215 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc0ea00640 1 --2- 172.21.15.46:0/1625879215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc100cea30 0x7efc1009de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc0f400640 1 --2- 172.21.15.46:0/1625879215 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efc1015cd30 0x7efc10098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc07e00640 1 -- 172.21.15.46:0/1625879215 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efc1015cd30 msgr2=0x7efc10098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc07e00640 1 --2- 172.21.15.46:0/1625879215 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efc1015cd30 0x7efc10098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc07e00640 1 -- 172.21.15.46:0/1625879215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc100cea30 msgr2=0x7efc1009de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc07e00640 1 --2- 172.21.15.46:0/1625879215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc100cea30 0x7efc1009de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc07e00640 1 -- 172.21.15.46:0/1625879215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf8009420 con 0x7efc1015c950 2024-03-29T08:22:30.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc07e00640 1 --2- 172.21.15.46:0/1625879215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1015c950 0x7efc10153250 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7efbf80b3040 tx=0x7efbf801ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc07400640 1 -- 172.21.15.46:0/1625879215 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf80b9630 con 0x7efc1015c950 2024-03-29T08:22:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc07400640 1 -- 172.21.15.46:0/1625879215 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf80b97b0 con 0x7efc1015c950 2024-03-29T08:22:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc07400640 1 -- 172.21.15.46:0/1625879215 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf80b9b60 con 0x7efc1015c950 2024-03-29T08:22:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc15e00640 1 -- 172.21.15.46:0/1625879215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc1006c470 con 0x7efc1015c950 2024-03-29T08:22:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.042+0000 7efc15e00640 1 -- 172.21.15.46:0/1625879215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc1009d370 con 0x7efc1015c950 2024-03-29T08:22:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.046+0000 7efc15e00640 1 -- 172.21.15.46:0/1625879215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc1009e690 con 0x7efc1015c950 2024-03-29T08:22:30.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.046+0000 7efc07400640 1 -- 172.21.15.46:0/1625879215 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efbf80b7040 con 0x7efc1015c950 2024-03-29T08:22:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.046+0000 7efc07400640 1 --2- 172.21.15.46:0/1625879215 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efbec066690 0x7efbec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.046+0000 7efc07400640 1 -- 172.21.15.46:0/1625879215 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7efbf8122190 con 0x7efc1015c950 2024-03-29T08:22:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.046+0000 7efc0ea00640 1 --2- 172.21.15.46:0/1625879215 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efbec066690 0x7efbec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:30.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.046+0000 7efc0ea00640 1 --2- 172.21.15.46:0/1625879215 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efbec066690 0x7efbec068b50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efc100b45e0 tx=0x7efc00009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:30.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.050+0000 7efc07400640 1 -- 172.21.15.46:0/1625879215 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efbf80ef4b0 con 0x7efc1015c950 2024-03-29T08:22:30.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.214+0000 7efc15e00640 1 -- 172.21.15.46:0/1625879215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc10070260 con 0x7efc1015c950 2024-03-29T08:22:30.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.214+0000 7efc07400640 1 -- 172.21.15.46:0/1625879215 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7efbf80f2ad0 con 0x7efc1015c950 2024-03-29T08:22:30.217 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:22:30.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.214+0000 7efc15e00640 1 -- 172.21.15.46:0/1625879215 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efbec066690 msgr2=0x7efbec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.214+0000 7efc15e00640 1 --2- 172.21.15.46:0/1625879215 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efbec066690 0x7efbec068b50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7efc100b45e0 tx=0x7efc00009040 comp rx=0 tx=0).stop 2024-03-29T08:22:30.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.214+0000 7efc15e00640 1 -- 172.21.15.46:0/1625879215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1015c950 msgr2=0x7efc10153250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.214+0000 7efc15e00640 1 --2- 172.21.15.46:0/1625879215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1015c950 0x7efc10153250 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7efbf80b3040 tx=0x7efbf801ba30 comp rx=0 tx=0).stop 2024-03-29T08:22:30.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.214+0000 7efc15e00640 1 -- 172.21.15.46:0/1625879215 shutdown_connections 2024-03-29T08:22:30.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.214+0000 7efc15e00640 1 --2- 172.21.15.46:0/1625879215 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efbec066690 0x7efbec068b50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.214+0000 7efc15e00640 1 --2- 172.21.15.46:0/1625879215 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efc1015cd30 0x7efc10098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.214+0000 7efc15e00640 1 --2- 172.21.15.46:0/1625879215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc1015c950 0x7efc10153250 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.214+0000 7efc15e00640 1 --2- 172.21.15.46:0/1625879215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc100cea30 0x7efc1009de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.214+0000 7efc15e00640 1 -- 172.21.15.46:0/1625879215 >> 172.21.15.46:0/1625879215 conn(0x7efc100b21b0 msgr2=0x7efc1015ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:30.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.214+0000 7efc15e00640 1 -- 172.21.15.46:0/1625879215 shutdown_connections 2024-03-29T08:22:30.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.214+0000 7efc15e00640 1 -- 172.21.15.46:0/1625879215 wait complete. 2024-03-29T08:22:30.238 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:22:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.382+0000 7ff2bda00640 1 Processor -- start 2024-03-29T08:22:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.382+0000 7ff2bda00640 1 -- start start 2024-03-29T08:22:30.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.382+0000 7ff2bda00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff2b80cea30 0x7ff2b81606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:30.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.382+0000 7ff2bda00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff2b815cd30 0x7ff2b8160be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.382+0000 7ff2bda00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b80cfb20 con 0x7ff2b815c950 2024-03-29T08:22:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.382+0000 7ff2bda00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b80cef10 con 0x7ff2b80cea30 2024-03-29T08:22:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.382+0000 7ff2bda00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b80cf1f0 con 0x7ff2b815cd30 2024-03-29T08:22:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.382+0000 7ff2b6a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff2b815c950 0x7ff2b806bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48006/0 (socket says 172.21.15.46:48006) 2024-03-29T08:22:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.382+0000 7ff2b6a00640 1 -- 172.21.15.46:0/55130828 learned_addr learned my addr 172.21.15.46:0/55130828 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.382+0000 7ff2ae000640 1 --2- 172.21.15.46:0/55130828 >> v2:172.21.15.46:3300/0 conn(0x7ff2b80cea30 0x7ff2b81606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.382+0000 7ff2b7400640 1 --2- 172.21.15.46:0/55130828 >> v2:172.21.15.73:3300/0 conn(0x7ff2b815cd30 0x7ff2b8160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.382+0000 7ff2ae000640 1 -- 172.21.15.46:0/55130828 >> v2:172.21.15.73:3300/0 conn(0x7ff2b815cd30 msgr2=0x7ff2b8160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.382+0000 7ff2ae000640 1 --2- 172.21.15.46:0/55130828 >> v2:172.21.15.73:3300/0 conn(0x7ff2b815cd30 0x7ff2b8160be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.382+0000 7ff2ae000640 1 -- 172.21.15.46:0/55130828 >> v1:172.21.15.46:6789/0 conn(0x7ff2b815c950 legacy=0x7ff2b806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.382+0000 7ff2b6000640 1 -- 172.21.15.46:0/55130828 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2296318441 0 0) 0x7ff2b80cfb20 con 0x7ff2b815c950 2024-03-29T08:22:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.382+0000 7ff2ae000640 1 -- 172.21.15.46:0/55130828 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b80cf4d0 con 0x7ff2b80cea30 2024-03-29T08:22:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.386+0000 7ff2ae000640 1 --2- 172.21.15.46:0/55130828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2b80cea30 0x7ff2b81606a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80723a0 tx=0x7ff2a001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1dc4598e14cd0fb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.386+0000 7ff2b6000640 1 -- 172.21.15.46:0/55130828 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00a8970 con 0x7ff2b80cea30 2024-03-29T08:22:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.386+0000 7ff2b6000640 1 -- 172.21.15.46:0/55130828 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a00a8ad0 con 0x7ff2b80cea30 2024-03-29T08:22:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.386+0000 7ff2b6000640 1 -- 172.21.15.46:0/55130828 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00a8e60 con 0x7ff2b80cea30 2024-03-29T08:22:30.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.386+0000 7ff2bda00640 1 -- 172.21.15.46:0/55130828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2b80cea30 msgr2=0x7ff2b81606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.386+0000 7ff2bda00640 1 --2- 172.21.15.46:0/55130828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2b80cea30 0x7ff2b81606a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80723a0 tx=0x7ff2a001ec70 comp rx=0 tx=0).stop 2024-03-29T08:22:30.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.386+0000 7ff2bda00640 1 -- 172.21.15.46:0/55130828 shutdown_connections 2024-03-29T08:22:30.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.386+0000 7ff2bda00640 1 --2- 172.21.15.46:0/55130828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2b80cea30 0x7ff2b81606a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.386+0000 7ff2bda00640 1 --2- 172.21.15.46:0/55130828 >> v2:172.21.15.73:3300/0 conn(0x7ff2b815cd30 0x7ff2b8160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.386+0000 7ff2bda00640 1 -- 172.21.15.46:0/55130828 >> 172.21.15.46:0/55130828 conn(0x7ff2b80b21b0 msgr2=0x7ff2b80b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:30.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.386+0000 7ff2bda00640 1 -- 172.21.15.46:0/55130828 shutdown_connections 2024-03-29T08:22:30.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.386+0000 7ff2bda00640 1 -- 172.21.15.46:0/55130828 wait complete. 2024-03-29T08:22:30.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2bda00640 1 Processor -- start 2024-03-29T08:22:30.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2bda00640 1 -- start start 2024-03-29T08:22:30.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2bda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2b80cea30 0x7ff2b806f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:30.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2bda00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2b815c950 0x7ff2b8154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:30.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2bda00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2b815cd30 0x7ff2b8154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:30.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2bda00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b80c59c0 con 0x7ff2b80cea30 2024-03-29T08:22:30.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2bda00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b8155780 con 0x7ff2b815cd30 2024-03-29T08:22:30.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2bda00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b80c4cf0 con 0x7ff2b815c950 2024-03-29T08:22:30.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2b6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2b80cea30 0x7ff2b806f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2b6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2b80cea30 0x7ff2b806f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42160/0 (socket says 172.21.15.46:42160) 2024-03-29T08:22:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2b6a00640 1 -- 172.21.15.46:0/640408915 learned_addr learned my addr 172.21.15.46:0/640408915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2ae000640 1 --2- 172.21.15.46:0/640408915 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2b815c950 0x7ff2b8154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2b7400640 1 --2- 172.21.15.46:0/640408915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2b815cd30 0x7ff2b8154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2b6a00640 1 -- 172.21.15.46:0/640408915 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2b815c950 msgr2=0x7ff2b8154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2b6a00640 1 --2- 172.21.15.46:0/640408915 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2b815c950 0x7ff2b8154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2b6a00640 1 -- 172.21.15.46:0/640408915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2b815cd30 msgr2=0x7ff2b8154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2b6a00640 1 --2- 172.21.15.46:0/640408915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2b815cd30 0x7ff2b8154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2b6a00640 1 -- 172.21.15.46:0/640408915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a0009420 con 0x7ff2b80cea30 2024-03-29T08:22:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2ae000640 1 --2- 172.21.15.46:0/640408915 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2b815c950 0x7ff2b8154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2b7400640 1 --2- 172.21.15.46:0/640408915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2b815cd30 0x7ff2b8154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2b6a00640 1 --2- 172.21.15.46:0/640408915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2b80cea30 0x7ff2b806f470 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8001570 tx=0x7ff2a800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:30.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2af400640 1 -- 172.21.15.46:0/640408915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a800a640 con 0x7ff2b80cea30 2024-03-29T08:22:30.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2af400640 1 -- 172.21.15.46:0/640408915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a800a7c0 con 0x7ff2b80cea30 2024-03-29T08:22:30.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2af400640 1 -- 172.21.15.46:0/640408915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a800ab70 con 0x7ff2b80cea30 2024-03-29T08:22:30.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2bda00640 1 -- 172.21.15.46:0/640408915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b806c470 con 0x7ff2b80cea30 2024-03-29T08:22:30.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2bda00640 1 -- 172.21.15.46:0/640408915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b80c52b0 con 0x7ff2b80cea30 2024-03-29T08:22:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.390+0000 7ff2bda00640 1 -- 172.21.15.46:0/640408915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b80cff90 con 0x7ff2b80cea30 2024-03-29T08:22:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.394+0000 7ff2af400640 1 -- 172.21.15.46:0/640408915 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff2a800acd0 con 0x7ff2b80cea30 2024-03-29T08:22:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.394+0000 7ff2af400640 1 --2- 172.21.15.46:0/640408915 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff294066690 0x7ff294068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.394+0000 7ff2af400640 1 -- 172.21.15.46:0/640408915 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff2a8083840 con 0x7ff2b80cea30 2024-03-29T08:22:30.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.394+0000 7ff2ae000640 1 --2- 172.21.15.46:0/640408915 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff294066690 0x7ff294068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:30.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.394+0000 7ff2ae000640 1 --2- 172.21.15.46:0/640408915 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff294066690 0x7ff294068b50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00b2000 tx=0x7ff2a0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:30.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.398+0000 7ff2af400640 1 -- 172.21.15.46:0/640408915 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff2b80cff90 con 0x7ff2b80cea30 2024-03-29T08:22:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.558+0000 7ff2bda00640 1 -- 172.21.15.46:0/640408915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff2b80cff90 con 0x7ff2b80cea30 2024-03-29T08:22:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.558+0000 7ff2af400640 1 -- 172.21.15.46:0/640408915 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff2b80cff90 con 0x7ff2b80cea30 2024-03-29T08:22:30.561 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:22:30.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.558+0000 7ff2bda00640 1 -- 172.21.15.46:0/640408915 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff294066690 msgr2=0x7ff294068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.558+0000 7ff2bda00640 1 --2- 172.21.15.46:0/640408915 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff294066690 0x7ff294068b50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00b2000 tx=0x7ff2a0008040 comp rx=0 tx=0).stop 2024-03-29T08:22:30.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.558+0000 7ff2bda00640 1 -- 172.21.15.46:0/640408915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2b80cea30 msgr2=0x7ff2b806f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.558+0000 7ff2bda00640 1 --2- 172.21.15.46:0/640408915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2b80cea30 0x7ff2b806f470 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8001570 tx=0x7ff2a800cd20 comp rx=0 tx=0).stop 2024-03-29T08:22:30.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.558+0000 7ff2bda00640 1 -- 172.21.15.46:0/640408915 shutdown_connections 2024-03-29T08:22:30.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.558+0000 7ff2bda00640 1 --2- 172.21.15.46:0/640408915 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff294066690 0x7ff294068b50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.558+0000 7ff2bda00640 1 --2- 172.21.15.46:0/640408915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2b815cd30 0x7ff2b8154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.558+0000 7ff2bda00640 1 --2- 172.21.15.46:0/640408915 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2b815c950 0x7ff2b8154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.558+0000 7ff2bda00640 1 --2- 172.21.15.46:0/640408915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2b80cea30 0x7ff2b806f470 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.558+0000 7ff2bda00640 1 -- 172.21.15.46:0/640408915 >> 172.21.15.46:0/640408915 conn(0x7ff2b80b21b0 msgr2=0x7ff2b815d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:30.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.558+0000 7ff2bda00640 1 -- 172.21.15.46:0/640408915 shutdown_connections 2024-03-29T08:22:30.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.558+0000 7ff2bda00640 1 -- 172.21.15.46:0/640408915 wait complete. 2024-03-29T08:22:30.581 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:22:30.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.722+0000 7fdaac600640 1 Processor -- start 2024-03-29T08:22:30.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.722+0000 7fdaac600640 1 -- start start 2024-03-29T08:22:30.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.722+0000 7fdaac600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdaa406f1c0 0x7fdaa4069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:30.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.722+0000 7fdaac600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa406a720 con 0x7fdaa415cad0 2024-03-29T08:22:30.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.722+0000 7fdaac600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa415d080 con 0x7fdaa406a2d0 2024-03-29T08:22:30.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.722+0000 7fdaac600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa415dd00 con 0x7fdaa406f1c0 2024-03-29T08:22:30.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.722+0000 7fdaa9200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdaa406f1c0 0x7fdaa4069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:30.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.722+0000 7fdaa9200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdaa406f1c0 0x7fdaa4069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42164/0 (socket says 172.21.15.46:42164) 2024-03-29T08:22:30.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.722+0000 7fdaa9c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdaa415cad0 0x7fdaa406eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48014/0 (socket says 172.21.15.46:48014) 2024-03-29T08:22:30.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.722+0000 7fdaa9200640 1 -- 172.21.15.46:0/1247557361 learned_addr learned my addr 172.21.15.46:0/1247557361 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:30.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.722+0000 7fdaa9200640 1 -- 172.21.15.46:0/1247557361 >> v1:172.21.15.73:6789/0 conn(0x7fdaa406a2d0 legacy=0x7fdaa406d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.722+0000 7fdaa9200640 1 -- 172.21.15.46:0/1247557361 >> v1:172.21.15.46:6789/0 conn(0x7fdaa415cad0 legacy=0x7fdaa406eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.722+0000 7fdaa9200640 1 -- 172.21.15.46:0/1247557361 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa415d360 con 0x7fdaa406f1c0 2024-03-29T08:22:30.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.726+0000 7fdaa9200640 1 --2- 172.21.15.46:0/1247557361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa406f1c0 0x7fdaa4069d90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fda90009870 tx=0x7fda9001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2076827907064693 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.726+0000 7fdaa3e00640 1 -- 172.21.15.46:0/1247557361 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda900ac640 con 0x7fdaa406f1c0 2024-03-29T08:22:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.726+0000 7fdaa3e00640 1 -- 172.21.15.46:0/1247557361 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda900ac7a0 con 0x7fdaa406f1c0 2024-03-29T08:22:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.726+0000 7fdaa3e00640 1 -- 172.21.15.46:0/1247557361 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda900acb30 con 0x7fdaa406f1c0 2024-03-29T08:22:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.726+0000 7fdaac600640 1 -- 172.21.15.46:0/1247557361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa406f1c0 msgr2=0x7fdaa4069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.726+0000 7fdaac600640 1 --2- 172.21.15.46:0/1247557361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa406f1c0 0x7fdaa4069d90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fda90009870 tx=0x7fda9001c4d0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.726+0000 7fdaac600640 1 -- 172.21.15.46:0/1247557361 shutdown_connections 2024-03-29T08:22:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.726+0000 7fdaac600640 1 --2- 172.21.15.46:0/1247557361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa406f1c0 0x7fdaa4069d90 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.726+0000 7fdaac600640 1 -- 172.21.15.46:0/1247557361 >> 172.21.15.46:0/1247557361 conn(0x7fdaa4098a40 msgr2=0x7fdaa40b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.726+0000 7fdaac600640 1 -- 172.21.15.46:0/1247557361 shutdown_connections 2024-03-29T08:22:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.726+0000 7fdaac600640 1 -- 172.21.15.46:0/1247557361 wait complete. 2024-03-29T08:22:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.726+0000 7fdaac600640 1 Processor -- start 2024-03-29T08:22:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaac600640 1 -- start start 2024-03-29T08:22:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaac600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa406a2d0 0x7fdaa41519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaac600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa406f1c0 0x7fdaa4159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaac600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdaa415cad0 0x7fdaa415fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaac600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa40c33d0 con 0x7fdaa406a2d0 2024-03-29T08:22:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaac600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa40c36b0 con 0x7fdaa406f1c0 2024-03-29T08:22:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaac600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa40c3990 con 0x7fdaa415cad0 2024-03-29T08:22:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaa9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa406a2d0 0x7fdaa41519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaa9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa406a2d0 0x7fdaa41519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42174/0 (socket says 172.21.15.46:42174) 2024-03-29T08:22:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaa9c00640 1 -- 172.21.15.46:0/4160027532 learned_addr learned my addr 172.21.15.46:0/4160027532 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaa9200640 1 --2- 172.21.15.46:0/4160027532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa406f1c0 0x7fdaa4159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaaa600640 1 --2- 172.21.15.46:0/4160027532 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdaa415cad0 0x7fdaa415fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaa9c00640 1 -- 172.21.15.46:0/4160027532 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdaa415cad0 msgr2=0x7fdaa415fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaa9c00640 1 --2- 172.21.15.46:0/4160027532 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdaa415cad0 0x7fdaa415fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaa9c00640 1 -- 172.21.15.46:0/4160027532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa406f1c0 msgr2=0x7fdaa4159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaa9c00640 1 --2- 172.21.15.46:0/4160027532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa406f1c0 0x7fdaa4159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaa9c00640 1 -- 172.21.15.46:0/4160027532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda900094d0 con 0x7fdaa406a2d0 2024-03-29T08:22:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaa9c00640 1 --2- 172.21.15.46:0/4160027532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa406a2d0 0x7fdaa41519b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fda9800c430 tx=0x7fda9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaa1600640 1 -- 172.21.15.46:0/4160027532 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda980074e0 con 0x7fdaa406a2d0 2024-03-29T08:22:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaa1600640 1 -- 172.21.15.46:0/4160027532 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda98007640 con 0x7fdaa406a2d0 2024-03-29T08:22:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaac600640 1 -- 172.21.15.46:0/4160027532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa40c3c70 con 0x7fdaa406a2d0 2024-03-29T08:22:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaa1600640 1 -- 172.21.15.46:0/4160027532 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98004a10 con 0x7fdaa406a2d0 2024-03-29T08:22:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaac600640 1 -- 172.21.15.46:0/4160027532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa4151ef0 con 0x7fdaa406a2d0 2024-03-29T08:22:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.730+0000 7fdaac600640 1 -- 172.21.15.46:0/4160027532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa40cd8f0 con 0x7fdaa406a2d0 2024-03-29T08:22:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.734+0000 7fdaa1600640 1 -- 172.21.15.46:0/4160027532 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fda98004b70 con 0x7fdaa406a2d0 2024-03-29T08:22:30.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.734+0000 7fdaa1600640 1 --2- 172.21.15.46:0/4160027532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fda84066690 0x7fda84068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:30.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.734+0000 7fdaa1600640 1 -- 172.21.15.46:0/4160027532 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fda98083aa0 con 0x7fdaa406a2d0 2024-03-29T08:22:30.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.734+0000 7fdaa9200640 1 --2- 172.21.15.46:0/4160027532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fda84066690 0x7fda84068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:30.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.734+0000 7fdaa9200640 1 --2- 172.21.15.46:0/4160027532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fda84066690 0x7fda84068b50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fda900ae000 tx=0x7fda900a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:30.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.738+0000 7fdaa1600640 1 -- 172.21.15.46:0/4160027532 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fda98050e80 con 0x7fdaa406a2d0 2024-03-29T08:22:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.898+0000 7fdaac600640 1 -- 172.21.15.46:0/4160027532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdaa409d700 con 0x7fdaa406a2d0 2024-03-29T08:22:30.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.898+0000 7fdaa1600640 1 -- 172.21.15.46:0/4160027532 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda980544a0 con 0x7fdaa406a2d0 2024-03-29T08:22:30.900 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:22:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.898+0000 7fdaac600640 1 -- 172.21.15.46:0/4160027532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fda84066690 msgr2=0x7fda84068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.898+0000 7fdaac600640 1 --2- 172.21.15.46:0/4160027532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fda84066690 0x7fda84068b50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fda900ae000 tx=0x7fda900a7000 comp rx=0 tx=0).stop 2024-03-29T08:22:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.898+0000 7fdaac600640 1 -- 172.21.15.46:0/4160027532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa406a2d0 msgr2=0x7fdaa41519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.898+0000 7fdaac600640 1 --2- 172.21.15.46:0/4160027532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa406a2d0 0x7fdaa41519b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fda9800c430 tx=0x7fda9800c8f0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.898+0000 7fdaac600640 1 -- 172.21.15.46:0/4160027532 shutdown_connections 2024-03-29T08:22:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.898+0000 7fdaac600640 1 --2- 172.21.15.46:0/4160027532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fda84066690 0x7fda84068b50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.898+0000 7fdaac600640 1 --2- 172.21.15.46:0/4160027532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa406f1c0 0x7fdaa4159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.898+0000 7fdaac600640 1 --2- 172.21.15.46:0/4160027532 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdaa415cad0 0x7fdaa415fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.898+0000 7fdaac600640 1 --2- 172.21.15.46:0/4160027532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdaa406a2d0 0x7fdaa41519b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.898+0000 7fdaac600640 1 -- 172.21.15.46:0/4160027532 >> 172.21.15.46:0/4160027532 conn(0x7fdaa4098a40 msgr2=0x7fdaa40b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.898+0000 7fdaac600640 1 -- 172.21.15.46:0/4160027532 shutdown_connections 2024-03-29T08:22:30.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:30.898+0000 7fdaac600640 1 -- 172.21.15.46:0/4160027532 wait complete. 2024-03-29T08:22:30.924 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:22:31.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.062+0000 7ff87fe00640 1 Processor -- start 2024-03-29T08:22:31.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.062+0000 7ff87fe00640 1 -- start start 2024-03-29T08:22:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff87815c9d0 0x7ff8780cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff8780cef60 0x7ff878098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff8780ccd10 0x7ff878098f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8780d0030 con 0x7ff8780cef60 2024-03-29T08:22:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87fe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8780cf420 con 0x7ff8780ccd10 2024-03-29T08:22:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87fe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780cf700 con 0x7ff87815c9d0 2024-03-29T08:22:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff8780cef60 0x7ff878098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff8780cef60 0x7ff878098a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42176/0 (socket says 172.21.15.46:42176) 2024-03-29T08:22:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87ca00640 1 -- 172.21.15.46:0/2810638798 learned_addr learned my addr 172.21.15.46:0/2810638798 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87d400640 1 --2- 172.21.15.46:0/2810638798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff87815c9d0 0x7ff8780cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87de00640 1 --2- 172.21.15.46:0/2810638798 >> v2:172.21.15.73:3300/0 conn(0x7ff8780ccd10 0x7ff878098f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87ca00640 1 -- 172.21.15.46:0/2810638798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff87815c9d0 msgr2=0x7ff8780cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87ca00640 1 --2- 172.21.15.46:0/2810638798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff87815c9d0 0x7ff8780cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87ca00640 1 -- 172.21.15.46:0/2810638798 >> v2:172.21.15.73:3300/0 conn(0x7ff8780ccd10 msgr2=0x7ff878098f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87ca00640 1 --2- 172.21.15.46:0/2810638798 >> v2:172.21.15.73:3300/0 conn(0x7ff8780ccd10 0x7ff878098f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87ca00640 1 -- 172.21.15.46:0/2810638798 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8780cf9e0 con 0x7ff8780cef60 2024-03-29T08:22:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87de00640 1 --2- 172.21.15.46:0/2810638798 >> v2:172.21.15.73:3300/0 conn(0x7ff8780ccd10 0x7ff878098f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:31.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.066+0000 7ff87ca00640 1 --2- 172.21.15.46:0/2810638798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8780cef60 0x7ff878098a40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff8781617a0 tx=0x7ff86401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547b4e5c04bf17d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff877e00640 1 -- 172.21.15.46:0/2810638798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8640aa430 con 0x7ff8780cef60 2024-03-29T08:22:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff877e00640 1 -- 172.21.15.46:0/2810638798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8640aa590 con 0x7ff8780cef60 2024-03-29T08:22:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff877e00640 1 -- 172.21.15.46:0/2810638798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8640aa920 con 0x7ff8780cef60 2024-03-29T08:22:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87fe00640 1 -- 172.21.15.46:0/2810638798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8780cef60 msgr2=0x7ff878098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87fe00640 1 --2- 172.21.15.46:0/2810638798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8780cef60 0x7ff878098a40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff8781617a0 tx=0x7ff86401c4f0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87fe00640 1 -- 172.21.15.46:0/2810638798 shutdown_connections 2024-03-29T08:22:31.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87fe00640 1 --2- 172.21.15.46:0/2810638798 >> v2:172.21.15.73:3300/0 conn(0x7ff8780ccd10 0x7ff878098f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87fe00640 1 --2- 172.21.15.46:0/2810638798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8780cef60 0x7ff878098a40 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87fe00640 1 --2- 172.21.15.46:0/2810638798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff87815c9d0 0x7ff8780cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87fe00640 1 -- 172.21.15.46:0/2810638798 >> 172.21.15.46:0/2810638798 conn(0x7ff8780b21a0 msgr2=0x7ff8780b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:31.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87fe00640 1 -- 172.21.15.46:0/2810638798 shutdown_connections 2024-03-29T08:22:31.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87fe00640 1 -- 172.21.15.46:0/2810638798 wait complete. 2024-03-29T08:22:31.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87fe00640 1 Processor -- start 2024-03-29T08:22:31.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87fe00640 1 -- start start 2024-03-29T08:22:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8780ccd10 0x7ff87814e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8780cef60 0x7ff8780c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff87815c9d0 0x7ff87815b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780c99b0 con 0x7ff8780ccd10 2024-03-29T08:22:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87815d200 con 0x7ff8780cef60 2024-03-29T08:22:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87fe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8780c96d0 con 0x7ff87815c9d0 2024-03-29T08:22:31.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8780ccd10 0x7ff87814e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8780ccd10 0x7ff87814e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42184/0 (socket says 172.21.15.46:42184) 2024-03-29T08:22:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87d400640 1 -- 172.21.15.46:0/2202116215 learned_addr learned my addr 172.21.15.46:0/2202116215 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.070+0000 7ff87ca00640 1 --2- 172.21.15.46:0/2202116215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8780cef60 0x7ff8780c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff87de00640 1 --2- 172.21.15.46:0/2202116215 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff87815c9d0 0x7ff87815b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff87d400640 1 -- 172.21.15.46:0/2202116215 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff87815c9d0 msgr2=0x7ff87815b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff87d400640 1 --2- 172.21.15.46:0/2202116215 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff87815c9d0 0x7ff87815b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff87d400640 1 -- 172.21.15.46:0/2202116215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8780cef60 msgr2=0x7ff8780c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff87d400640 1 --2- 172.21.15.46:0/2202116215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8780cef60 0x7ff8780c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff87d400640 1 -- 172.21.15.46:0/2202116215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8640094d0 con 0x7ff8780ccd10 2024-03-29T08:22:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff87ca00640 1 --2- 172.21.15.46:0/2202116215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8780cef60 0x7ff8780c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff87d400640 1 --2- 172.21.15.46:0/2202116215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8780ccd10 0x7ff87814e380 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00d800 tx=0x7ff86c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff875600640 1 -- 172.21.15.46:0/2202116215 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c00b6c0 con 0x7ff8780ccd10 2024-03-29T08:22:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff875600640 1 -- 172.21.15.46:0/2202116215 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c00b820 con 0x7ff8780ccd10 2024-03-29T08:22:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff875600640 1 -- 172.21.15.46:0/2202116215 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c00bb90 con 0x7ff8780ccd10 2024-03-29T08:22:31.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff87fe00640 1 -- 172.21.15.46:0/2202116215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87815d410 con 0x7ff8780ccd10 2024-03-29T08:22:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff87fe00640 1 -- 172.21.15.46:0/2202116215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87815d9d0 con 0x7ff8780ccd10 2024-03-29T08:22:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff87fe00640 1 -- 172.21.15.46:0/2202116215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff878066c90 con 0x7ff8780ccd10 2024-03-29T08:22:31.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff875600640 1 -- 172.21.15.46:0/2202116215 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff86c00bcf0 con 0x7ff8780ccd10 2024-03-29T08:22:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff875600640 1 --2- 172.21.15.46:0/2202116215 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff858066690 0x7ff858068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff875600640 1 -- 172.21.15.46:0/2202116215 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff86c084760 con 0x7ff8780ccd10 2024-03-29T08:22:31.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.074+0000 7ff87ca00640 1 --2- 172.21.15.46:0/2202116215 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff858066690 0x7ff858068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.078+0000 7ff87ca00640 1 --2- 172.21.15.46:0/2202116215 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff858066690 0x7ff858068b50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff8640ae000 tx=0x7ff8640a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:31.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.078+0000 7ff875600640 1 -- 172.21.15.46:0/2202116215 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff86c0519f0 con 0x7ff8780ccd10 2024-03-29T08:22:31.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.278+0000 7ff87fe00640 1 -- 172.21.15.46:0/2202116215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8780b2ec0 con 0x7ff8780ccd10 2024-03-29T08:22:31.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.278+0000 7ff875600640 1 -- 172.21.15.46:0/2202116215 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4729 (secure 0 0 0) 0x7ff86c055010 con 0x7ff8780ccd10 2024-03-29T08:22:31.282 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:31.283 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2187826146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2187826146},{"type":"v1","addr":"172.21.15.73:6836","nonce":2187826146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/73382477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":73382477},{"type":"v1","addr":"172.21.15.73:6837","nonce":73382477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2874958170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2874958170},{"type":"v1","addr":"172.21.15.46:6834","nonce":2874958170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:21:10.337267+0000","modified":"2024-03-29T08:22:02.946081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6051},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6051":{"gid":6051,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":16,"addr":"172.21.15.46:6835/3255474176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3255474176},{"type":"v1","addr":"172.21.15.46:6835","nonce":3255474176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-29T08:22:31.283 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2024-03-29T08:22:31.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.282+0000 7ff87fe00640 1 -- 172.21.15.46:0/2202116215 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff858066690 msgr2=0x7ff858068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:31.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.282+0000 7ff87fe00640 1 --2- 172.21.15.46:0/2202116215 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff858066690 0x7ff858068b50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff8640ae000 tx=0x7ff8640a7000 comp rx=0 tx=0).stop 2024-03-29T08:22:31.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.282+0000 7ff87fe00640 1 -- 172.21.15.46:0/2202116215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8780ccd10 msgr2=0x7ff87814e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:31.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.282+0000 7ff87fe00640 1 --2- 172.21.15.46:0/2202116215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8780ccd10 0x7ff87814e380 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00d800 tx=0x7ff86c00dcc0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.282+0000 7ff87fe00640 1 -- 172.21.15.46:0/2202116215 shutdown_connections 2024-03-29T08:22:31.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.282+0000 7ff87fe00640 1 --2- 172.21.15.46:0/2202116215 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff858066690 0x7ff858068b50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.282+0000 7ff87fe00640 1 --2- 172.21.15.46:0/2202116215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8780cef60 0x7ff8780c8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.282+0000 7ff87fe00640 1 --2- 172.21.15.46:0/2202116215 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff87815c9d0 0x7ff87815b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.282+0000 7ff87fe00640 1 --2- 172.21.15.46:0/2202116215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8780ccd10 0x7ff87814e380 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.282+0000 7ff87fe00640 1 -- 172.21.15.46:0/2202116215 >> 172.21.15.46:0/2202116215 conn(0x7ff8780b21a0 msgr2=0x7ff878072e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:31.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.282+0000 7ff87fe00640 1 -- 172.21.15.46:0/2202116215 shutdown_connections 2024-03-29T08:22:31.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.282+0000 7ff87fe00640 1 -- 172.21.15.46:0/2202116215 wait complete. 2024-03-29T08:22:31.304 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:22:31.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.442+0000 7fdcc2200640 1 Processor -- start 2024-03-29T08:22:31.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.442+0000 7fdcc2200640 1 -- start start 2024-03-29T08:22:31.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.442+0000 7fdcc2200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdcbc0d0660 0x7fdcbc0d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.442+0000 7fdcc2200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdcbc15dc00 0x7fdcbc0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.442+0000 7fdcc2200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcbc15cfb0 con 0x7fdcbc06b8b0 2024-03-29T08:22:31.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.442+0000 7fdcc2200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcbc06c100 con 0x7fdcbc15dc00 2024-03-29T08:22:31.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.442+0000 7fdcc2200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcbc0d0f80 con 0x7fdcbc0d0660 2024-03-29T08:22:31.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.442+0000 7fdcbbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdcbc15dc00 0x7fdcbc0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.442+0000 7fdcbbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdcbc15dc00 0x7fdcbc0cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42198/0 (socket says 172.21.15.46:42198) 2024-03-29T08:22:31.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.442+0000 7fdcbbe00640 1 -- 172.21.15.46:0/998934051 learned_addr learned my addr 172.21.15.46:0/998934051 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:31.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.442+0000 7fdcbb400640 1 --2- 172.21.15.46:0/998934051 >> v2:172.21.15.73:3300/0 conn(0x7fdcbc0d0660 0x7fdcbc0d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.442+0000 7fdcbbe00640 1 -- 172.21.15.46:0/998934051 >> v2:172.21.15.73:3300/0 conn(0x7fdcbc0d0660 msgr2=0x7fdcbc0d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:31.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.442+0000 7fdcbbe00640 1 --2- 172.21.15.46:0/998934051 >> v2:172.21.15.73:3300/0 conn(0x7fdcbc0d0660 0x7fdcbc0d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.442+0000 7fdcbbe00640 1 -- 172.21.15.46:0/998934051 >> v1:172.21.15.73:6789/0 conn(0x7fdcbc06b8b0 legacy=0x7fdcbc0cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:31.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.442+0000 7fdcbbe00640 1 -- 172.21.15.46:0/998934051 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc06be20 con 0x7fdcbc15dc00 2024-03-29T08:22:31.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.446+0000 7fdcbbe00640 1 --2- 172.21.15.46:0/998934051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15dc00 0x7fdcbc0cf7d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc071dd0 tx=0x7fdcb401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11806c45c6af316 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:31.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.446+0000 7fdcbaa00640 1 -- 172.21.15.46:0/998934051 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40a7070 con 0x7fdcbc15dc00 2024-03-29T08:22:31.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.446+0000 7fdcbaa00640 1 -- 172.21.15.46:0/998934051 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb40ab430 con 0x7fdcbc15dc00 2024-03-29T08:22:31.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.446+0000 7fdcbaa00640 1 -- 172.21.15.46:0/998934051 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40ab7c0 con 0x7fdcbc15dc00 2024-03-29T08:22:31.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.446+0000 7fdcc2200640 1 -- 172.21.15.46:0/998934051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15dc00 msgr2=0x7fdcbc0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:31.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.446+0000 7fdcc2200640 1 --2- 172.21.15.46:0/998934051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15dc00 0x7fdcbc0cf7d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc071dd0 tx=0x7fdcb401c4c0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.446+0000 7fdcc2200640 1 -- 172.21.15.46:0/998934051 shutdown_connections 2024-03-29T08:22:31.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.446+0000 7fdcc2200640 1 --2- 172.21.15.46:0/998934051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15dc00 0x7fdcbc0cf7d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.446+0000 7fdcc2200640 1 --2- 172.21.15.46:0/998934051 >> v2:172.21.15.73:3300/0 conn(0x7fdcbc0d0660 0x7fdcbc0d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.446+0000 7fdcc2200640 1 -- 172.21.15.46:0/998934051 >> 172.21.15.46:0/998934051 conn(0x7fdcbc089fd0 msgr2=0x7fdcbc08a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:31.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.446+0000 7fdcc2200640 1 -- 172.21.15.46:0/998934051 shutdown_connections 2024-03-29T08:22:31.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.446+0000 7fdcc2200640 1 -- 172.21.15.46:0/998934051 wait complete. 2024-03-29T08:22:31.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcc2200640 1 Processor -- start 2024-03-29T08:22:31.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcc2200640 1 -- start start 2024-03-29T08:22:31.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcc2200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdcbc06b8b0 0x7fdcbc0ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcc2200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcbc0d0660 0x7fdcbc151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcc2200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15dc00 0x7fdcbc1f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcc2200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc0c4d70 con 0x7fdcbc15dc00 2024-03-29T08:22:31.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcc2200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc157240 con 0x7fdcbc0d0660 2024-03-29T08:22:31.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcc2200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc157520 con 0x7fdcbc06b8b0 2024-03-29T08:22:31.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcbbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15dc00 0x7fdcbc1f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcbbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15dc00 0x7fdcbc1f8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42214/0 (socket says 172.21.15.46:42214) 2024-03-29T08:22:31.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcbbe00640 1 -- 172.21.15.46:0/2508706510 learned_addr learned my addr 172.21.15.46:0/2508706510 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:31.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcb2a00640 1 --2- 172.21.15.46:0/2508706510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcbc0d0660 0x7fdcbc151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcbb400640 1 --2- 172.21.15.46:0/2508706510 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdcbc06b8b0 0x7fdcbc0ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcbbe00640 1 -- 172.21.15.46:0/2508706510 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdcbc06b8b0 msgr2=0x7fdcbc0ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:31.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcbbe00640 1 --2- 172.21.15.46:0/2508706510 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdcbc06b8b0 0x7fdcbc0ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcbbe00640 1 -- 172.21.15.46:0/2508706510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcbc0d0660 msgr2=0x7fdcbc151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:31.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcbbe00640 1 --2- 172.21.15.46:0/2508706510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcbc0d0660 0x7fdcbc151450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcbbe00640 1 -- 172.21.15.46:0/2508706510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb40094d0 con 0x7fdcbc15dc00 2024-03-29T08:22:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcb2a00640 1 --2- 172.21.15.46:0/2508706510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcbc0d0660 0x7fdcbc151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:22:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcbb400640 1 --2- 172.21.15.46:0/2508706510 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdcbc06b8b0 0x7fdcbc0ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:22:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcbbe00640 1 --2- 172.21.15.46:0/2508706510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15dc00 0x7fdcbc1f8690 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40ad040 tx=0x7fdcb4019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcb3e00640 1 -- 172.21.15.46:0/2508706510 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40b4620 con 0x7fdcbc15dc00 2024-03-29T08:22:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcb3e00640 1 -- 172.21.15.46:0/2508706510 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb40b47a0 con 0x7fdcbc15dc00 2024-03-29T08:22:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcb3e00640 1 -- 172.21.15.46:0/2508706510 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb40b4b50 con 0x7fdcbc15dc00 2024-03-29T08:22:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcc2200640 1 -- 172.21.15.46:0/2508706510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcbc0c4720 con 0x7fdcbc15dc00 2024-03-29T08:22:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.450+0000 7fdcc2200640 1 -- 172.21.15.46:0/2508706510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcbc0ca700 con 0x7fdcbc15dc00 2024-03-29T08:22:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.454+0000 7fdcc2200640 1 -- 172.21.15.46:0/2508706510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcbc0c4f20 con 0x7fdcbc15dc00 2024-03-29T08:22:31.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.454+0000 7fdcb3e00640 1 -- 172.21.15.46:0/2508706510 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdcb40b4cb0 con 0x7fdcbc15dc00 2024-03-29T08:22:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.454+0000 7fdcb3e00640 1 --2- 172.21.15.46:0/2508706510 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdc980666e0 0x7fdc98068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.454+0000 7fdcb3e00640 1 -- 172.21.15.46:0/2508706510 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fdcb411e500 con 0x7fdcbc15dc00 2024-03-29T08:22:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.454+0000 7fdcbb400640 1 --2- 172.21.15.46:0/2508706510 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdc980666e0 0x7fdc98068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.454+0000 7fdcbb400640 1 --2- 172.21.15.46:0/2508706510 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdc980666e0 0x7fdc98068ba0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdcac007330 tx=0x7fdcac00b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:31.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.458+0000 7fdcb3e00640 1 -- 172.21.15.46:0/2508706510 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdcbc0c4f20 con 0x7fdcbc15dc00 2024-03-29T08:22:31.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.622+0000 7fdcc2200640 1 -- 172.21.15.46:0/2508706510 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcbc0c4f20 con 0x7fdcbc15dc00 2024-03-29T08:22:31.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.622+0000 7fdcb3e00640 1 -- 172.21.15.46:0/2508706510 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16145 (secure 0 0 0) 0x7fdcbc0c4f20 con 0x7fdcbc15dc00 2024-03-29T08:22:31.627 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:31.627 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":65,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:21:58.933917+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:21:08.061567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T08:21:09.336464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.2f","mappings":[{"from":6,"to":1}]},{"pgid":"9.3b","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:22:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.626+0000 7fdcc2200640 1 -- 172.21.15.46:0/2508706510 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdc980666e0 msgr2=0x7fdc98068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:31.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.626+0000 7fdcc2200640 1 --2- 172.21.15.46:0/2508706510 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdc980666e0 0x7fdc98068ba0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fdcac007330 tx=0x7fdcac00b040 comp rx=0 tx=0).stop 2024-03-29T08:22:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.626+0000 7fdcc2200640 1 -- 172.21.15.46:0/2508706510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15dc00 msgr2=0x7fdcbc1f8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.626+0000 7fdcc2200640 1 --2- 172.21.15.46:0/2508706510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15dc00 0x7fdcbc1f8690 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fdcb40ad040 tx=0x7fdcb4019260 comp rx=0 tx=0).stop 2024-03-29T08:22:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.626+0000 7fdcc2200640 1 -- 172.21.15.46:0/2508706510 shutdown_connections 2024-03-29T08:22:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.626+0000 7fdcc2200640 1 --2- 172.21.15.46:0/2508706510 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdc980666e0 0x7fdc98068ba0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.626+0000 7fdcc2200640 1 --2- 172.21.15.46:0/2508706510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcbc0d0660 0x7fdcbc151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.626+0000 7fdcc2200640 1 --2- 172.21.15.46:0/2508706510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcbc15dc00 0x7fdcbc1f8690 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.626+0000 7fdcc2200640 1 --2- 172.21.15.46:0/2508706510 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdcbc06b8b0 0x7fdcbc0ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.626+0000 7fdcc2200640 1 -- 172.21.15.46:0/2508706510 >> 172.21.15.46:0/2508706510 conn(0x7fdcbc089fd0 msgr2=0x7fdcbc06f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.626+0000 7fdcc2200640 1 -- 172.21.15.46:0/2508706510 shutdown_connections 2024-03-29T08:22:31.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.626+0000 7fdcc2200640 1 -- 172.21.15.46:0/2508706510 wait complete. 2024-03-29T08:22:31.649 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T08:22:31.650 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:22:31.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.786+0000 7f4ee6600640 1 Processor -- start 2024-03-29T08:22:31.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.786+0000 7f4ee6600640 1 -- start start 2024-03-29T08:22:31.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.786+0000 7f4ee6600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4ee00d0b00 0x7f4ee00cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.786+0000 7f4ee6600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4ee00cb900 0x7f4ee00b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.786+0000 7f4ee6600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee00ad370 con 0x7f4ee015aa20 2024-03-29T08:22:31.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.786+0000 7f4ee6600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee00cc130 con 0x7f4ee00d0b00 2024-03-29T08:22:31.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.786+0000 7f4ee6600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee00cc410 con 0x7f4ee00cb900 2024-03-29T08:22:31.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.786+0000 7f4edf400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4ee015aa20 0x7f4ee00d03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48018/0 (socket says 172.21.15.46:48018) 2024-03-29T08:22:31.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.786+0000 7f4edf400640 1 -- 172.21.15.46:0/2646660489 learned_addr learned my addr 172.21.15.46:0/2646660489 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:31.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.786+0000 7f4ed6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4ee00d0b00 0x7f4ee00cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.786+0000 7f4edfe00640 1 --2- 172.21.15.46:0/2646660489 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4ee00cb900 0x7f4ee00b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.786+0000 7f4ed6a00640 1 -- 172.21.15.46:0/2646660489 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4ee00cb900 msgr2=0x7f4ee00b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:31.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.786+0000 7f4ed6a00640 1 --2- 172.21.15.46:0/2646660489 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4ee00cb900 0x7f4ee00b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.786+0000 7f4ed6a00640 1 -- 172.21.15.46:0/2646660489 >> v1:172.21.15.46:6789/0 conn(0x7f4ee015aa20 legacy=0x7f4ee00d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:31.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.786+0000 7f4ed6a00640 1 -- 172.21.15.46:0/2646660489 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee00b05c0 con 0x7f4ee00d0b00 2024-03-29T08:22:31.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.790+0000 7f4ed6a00640 1 --2- 172.21.15.46:0/2646660489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee00d0b00 0x7f4ee00cb3c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80097c0 tx=0x7f4ec801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcabe4bbbfbfc530 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:31.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.790+0000 7f4edea00640 1 -- 172.21.15.46:0/2646660489 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80aa5f0 con 0x7f4ee00d0b00 2024-03-29T08:22:31.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.790+0000 7f4edea00640 1 -- 172.21.15.46:0/2646660489 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee00cbf20 con 0x7f4ee00d0b00 2024-03-29T08:22:31.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.790+0000 7f4edea00640 1 -- 172.21.15.46:0/2646660489 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80aa970 con 0x7f4ee00d0b00 2024-03-29T08:22:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.790+0000 7f4ee6600640 1 -- 172.21.15.46:0/2646660489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee00d0b00 msgr2=0x7f4ee00cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.790+0000 7f4ee6600640 1 --2- 172.21.15.46:0/2646660489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee00d0b00 0x7f4ee00cb3c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80097c0 tx=0x7f4ec801c470 comp rx=0 tx=0).stop 2024-03-29T08:22:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.790+0000 7f4ee6600640 1 -- 172.21.15.46:0/2646660489 shutdown_connections 2024-03-29T08:22:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.790+0000 7f4ee6600640 1 --2- 172.21.15.46:0/2646660489 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4ee00cb900 0x7f4ee00b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.790+0000 7f4ee6600640 1 --2- 172.21.15.46:0/2646660489 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee00d0b00 0x7f4ee00cb3c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.790+0000 7f4ee6600640 1 -- 172.21.15.46:0/2646660489 >> 172.21.15.46:0/2646660489 conn(0x7f4ee0089fd0 msgr2=0x7f4ee008a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.790+0000 7f4ee6600640 1 -- 172.21.15.46:0/2646660489 shutdown_connections 2024-03-29T08:22:31.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.790+0000 7f4ee6600640 1 -- 172.21.15.46:0/2646660489 wait complete. 2024-03-29T08:22:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4ee6600640 1 Processor -- start 2024-03-29T08:22:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4ee6600640 1 -- start start 2024-03-29T08:22:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4ee6600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee00cb900 0x7f4ee0163d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4ee6600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ee00d0b00 0x7f4ee00c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4ee6600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4ee015aa20 0x7f4ee00c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4ee6600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee00c8ef0 con 0x7f4ee00cb900 2024-03-29T08:22:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4ee6600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee00c9a30 con 0x7f4ee00d0b00 2024-03-29T08:22:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4ee6600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee00c9100 con 0x7f4ee015aa20 2024-03-29T08:22:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4edf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee00cb900 0x7f4ee0163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4edf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee00cb900 0x7f4ee0163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42238/0 (socket says 172.21.15.46:42238) 2024-03-29T08:22:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4edf400640 1 -- 172.21.15.46:0/2143354604 learned_addr learned my addr 172.21.15.46:0/2143354604 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4edfe00640 1 --2- 172.21.15.46:0/2143354604 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4ee015aa20 0x7f4ee00c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4ed6a00640 1 --2- 172.21.15.46:0/2143354604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ee00d0b00 0x7f4ee00c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4edf400640 1 -- 172.21.15.46:0/2143354604 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4ee015aa20 msgr2=0x7f4ee00c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4edf400640 1 --2- 172.21.15.46:0/2143354604 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4ee015aa20 0x7f4ee00c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4edf400640 1 -- 172.21.15.46:0/2143354604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ee00d0b00 msgr2=0x7f4ee00c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:31.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4edf400640 1 --2- 172.21.15.46:0/2143354604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ee00d0b00 0x7f4ee00c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:31.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4edf400640 1 -- 172.21.15.46:0/2143354604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec8009420 con 0x7f4ee00cb900 2024-03-29T08:22:31.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4edf400640 1 --2- 172.21.15.46:0/2143354604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee00cb900 0x7f4ee0163d90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000cbc0 tx=0x7f4ed000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:31.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4ed7e00640 1 -- 172.21.15.46:0/2143354604 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed000aca0 con 0x7f4ee00cb900 2024-03-29T08:22:31.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4ed7e00640 1 -- 172.21.15.46:0/2143354604 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed000ae00 con 0x7f4ee00cb900 2024-03-29T08:22:31.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4ed7e00640 1 -- 172.21.15.46:0/2143354604 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed0012600 con 0x7f4ee00cb900 2024-03-29T08:22:31.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4ee6600640 1 -- 172.21.15.46:0/2143354604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee00c93e0 con 0x7f4ee00cb900 2024-03-29T08:22:31.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4ee6600640 1 -- 172.21.15.46:0/2143354604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee01642d0 con 0x7f4ee00cb900 2024-03-29T08:22:31.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.794+0000 7f4ee6600640 1 -- 172.21.15.46:0/2143354604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee0066c90 con 0x7f4ee00cb900 2024-03-29T08:22:31.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.798+0000 7f4ed7e00640 1 -- 172.21.15.46:0/2143354604 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4ed0007070 con 0x7f4ee00cb900 2024-03-29T08:22:31.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.798+0000 7f4ed7e00640 1 --2- 172.21.15.46:0/2143354604 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ebc066690 0x7f4ebc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:31.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.798+0000 7f4ed7e00640 1 -- 172.21.15.46:0/2143354604 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4ed0084120 con 0x7f4ee00cb900 2024-03-29T08:22:31.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.798+0000 7f4ed6a00640 1 --2- 172.21.15.46:0/2143354604 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ebc066690 0x7f4ebc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:31.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.798+0000 7f4ed6a00640 1 --2- 172.21.15.46:0/2143354604 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ebc066690 0x7f4ebc068b50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80ae000 tx=0x7f4ec80a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:31.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:31.802+0000 7f4ed7e00640 1 -- 172.21.15.46:0/2143354604 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4ed0051460 con 0x7f4ee00cb900 2024-03-29T08:22:32.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.002+0000 7f4ee6600640 1 -- 172.21.15.46:0/2143354604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ee006e650 con 0x7f4ee00cb900 2024-03-29T08:22:32.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.002+0000 7f4ed7e00640 1 -- 172.21.15.46:0/2143354604 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4729 (secure 0 0 0) 0x7f4ed0054a80 con 0x7f4ee00cb900 2024-03-29T08:22:32.004 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:32.004 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2187826146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2187826146},{"type":"v1","addr":"172.21.15.73:6836","nonce":2187826146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/73382477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":73382477},{"type":"v1","addr":"172.21.15.73:6837","nonce":73382477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2874958170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2874958170},{"type":"v1","addr":"172.21.15.46:6834","nonce":2874958170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:21:10.337267+0000","modified":"2024-03-29T08:22:02.946081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6051},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6051":{"gid":6051,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":16,"addr":"172.21.15.46:6835/3255474176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3255474176},{"type":"v1","addr":"172.21.15.46:6835","nonce":3255474176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-29T08:22:32.004 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2024-03-29T08:22:32.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.002+0000 7f4ee6600640 1 -- 172.21.15.46:0/2143354604 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ebc066690 msgr2=0x7f4ebc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.002+0000 7f4ee6600640 1 --2- 172.21.15.46:0/2143354604 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ebc066690 0x7f4ebc068b50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80ae000 tx=0x7f4ec80a7000 comp rx=0 tx=0).stop 2024-03-29T08:22:32.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.002+0000 7f4ee6600640 1 -- 172.21.15.46:0/2143354604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee00cb900 msgr2=0x7f4ee0163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.002+0000 7f4ee6600640 1 --2- 172.21.15.46:0/2143354604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee00cb900 0x7f4ee0163d90 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000cbc0 tx=0x7f4ed000a060 comp rx=0 tx=0).stop 2024-03-29T08:22:32.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.002+0000 7f4ee6600640 1 -- 172.21.15.46:0/2143354604 shutdown_connections 2024-03-29T08:22:32.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.002+0000 7f4ee6600640 1 --2- 172.21.15.46:0/2143354604 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ebc066690 0x7f4ebc068b50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.002+0000 7f4ee6600640 1 --2- 172.21.15.46:0/2143354604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ee00d0b00 0x7f4ee00c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.002+0000 7f4ee6600640 1 --2- 172.21.15.46:0/2143354604 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4ee015aa20 0x7f4ee00c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.002+0000 7f4ee6600640 1 --2- 172.21.15.46:0/2143354604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ee00cb900 0x7f4ee0163d90 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.002+0000 7f4ee6600640 1 -- 172.21.15.46:0/2143354604 >> 172.21.15.46:0/2143354604 conn(0x7f4ee0089fd0 msgr2=0x7f4ee0164bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:32.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.002+0000 7f4ee6600640 1 -- 172.21.15.46:0/2143354604 shutdown_connections 2024-03-29T08:22:32.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.002+0000 7f4ee6600640 1 -- 172.21.15.46:0/2143354604 wait complete. 2024-03-29T08:22:32.025 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:22:32.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.162+0000 7f7fb2200640 1 Processor -- start 2024-03-29T08:22:32.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.162+0000 7f7fb2200640 1 -- start start 2024-03-29T08:22:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.162+0000 7f7fb2200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7fac15a790 0x7f7fac0cbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7fb2200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fac0ce660 con 0x7f7fac15ab70 2024-03-29T08:22:32.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7fb2200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fac0ce940 con 0x7f7fac099150 2024-03-29T08:22:32.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7fb2200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fac0cf700 con 0x7f7fac15a790 2024-03-29T08:22:32.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7fa3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7fac15ab70 0x7f7fac098a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48020/0 (socket says 172.21.15.46:48020) 2024-03-29T08:22:32.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7fa3e00640 1 -- 172.21.15.46:0/3357088611 learned_addr learned my addr 172.21.15.46:0/3357088611 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:32.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7fab400640 1 --2- 172.21.15.46:0/3357088611 >> v2:172.21.15.73:3300/0 conn(0x7f7fac15a790 0x7f7fac0cbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7faaa00640 1 -- 172.21.15.46:0/3357088611 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592626948 0 0) 0x7f7fac0ce660 con 0x7f7fac15ab70 2024-03-29T08:22:32.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7faaa00640 1 -- 172.21.15.46:0/3357088611 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f90003530 con 0x7f7fac15ab70 2024-03-29T08:22:32.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7faaa00640 1 -- 172.21.15.46:0/3357088611 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3219130383 0 0) 0x7f7f90003530 con 0x7f7fac15ab70 2024-03-29T08:22:32.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7faaa00640 1 -- 172.21.15.46:0/3357088611 >> v2:172.21.15.73:3300/0 conn(0x7f7fac15a790 msgr2=0x7f7fac0cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:32.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7faaa00640 1 --2- 172.21.15.46:0/3357088611 >> v2:172.21.15.73:3300/0 conn(0x7f7fac15a790 0x7f7fac0cbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7faaa00640 1 -- 172.21.15.46:0/3357088611 >> v1:172.21.15.73:6789/0 conn(0x7f7fac099150 legacy=0x7f7fac0b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7faaa00640 1 -- 172.21.15.46:0/3357088611 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fac0cec20 con 0x7f7fac15ab70 2024-03-29T08:22:32.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7faaa00640 1 -- 172.21.15.46:0/3357088611 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f7f94001680 con 0x7f7fac15ab70 2024-03-29T08:22:32.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7faaa00640 1 -- 172.21.15.46:0/3357088611 >> v1:172.21.15.46:6789/0 conn(0x7f7fac15ab70 legacy=0x7f7fac098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7faaa00640 1 --2- 172.21.15.46:0/3357088611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f90003980 0x7f7f90003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7faaa00640 1 --2- 172.21.15.46:0/3357088611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f900047d0 0x7f7f90004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7faaa00640 1 --2- 172.21.15.46:0/3357088611 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f90005190 0x7f7f9000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7faaa00640 1 -- 172.21.15.46:0/3357088611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac0cf700 con 0x7f7f900047d0 2024-03-29T08:22:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7faaa00640 1 -- 172.21.15.46:0/3357088611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9000c030 con 0x7f7f90003980 2024-03-29T08:22:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7faaa00640 1 -- 172.21.15.46:0/3357088611 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9000c1f0 con 0x7f7f90005190 2024-03-29T08:22:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/3357088611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f900047d0 0x7f7f90004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7fab400640 1 --2- 172.21.15.46:0/3357088611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f90003980 0x7f7f90003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7fabe00640 1 --2- 172.21.15.46:0/3357088611 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f90005190 0x7f7f9000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7fa3e00640 1 -- 172.21.15.46:0/3357088611 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f90005190 msgr2=0x7f7f9000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/3357088611 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f90005190 0x7f7f9000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7fa3e00640 1 -- 172.21.15.46:0/3357088611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f90003980 msgr2=0x7f7f90003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/3357088611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f90003980 0x7f7f90003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7fa3e00640 1 -- 172.21.15.46:0/3357088611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f90003530 con 0x7f7f900047d0 2024-03-29T08:22:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7fabe00640 1 --2- 172.21.15.46:0/3357088611 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f90005190 0x7f7f9000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.166+0000 7f7fabe00640 1 -- 172.21.15.46:0/3357088611 reap_dead start 2024-03-29T08:22:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.170+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/3357088611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f900047d0 0x7f7f90004c50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400c240 tx=0x7f7f9401f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6092b694ffd4b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.170+0000 7f7faaa00640 1 -- 172.21.15.46:0/3357088611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94005ae0 con 0x7f7f900047d0 2024-03-29T08:22:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.170+0000 7f7faaa00640 1 -- 172.21.15.46:0/3357088611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f940ab070 con 0x7f7f900047d0 2024-03-29T08:22:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.170+0000 7f7faaa00640 1 -- 172.21.15.46:0/3357088611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94007070 con 0x7f7f900047d0 2024-03-29T08:22:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.170+0000 7f7fb2200640 1 -- 172.21.15.46:0/3357088611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f900047d0 msgr2=0x7f7f90004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.170+0000 7f7fb2200640 1 --2- 172.21.15.46:0/3357088611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f900047d0 0x7f7f90004c50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7f9400c240 tx=0x7f7f9401f3d0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.170+0000 7f7fb2200640 1 -- 172.21.15.46:0/3357088611 shutdown_connections 2024-03-29T08:22:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.170+0000 7f7fb2200640 1 --2- 172.21.15.46:0/3357088611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f900047d0 0x7f7f90004c50 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.170+0000 7f7fb2200640 1 -- 172.21.15.46:0/3357088611 >> 172.21.15.46:0/3357088611 conn(0x7f7fac0b21a0 msgr2=0x7f7fac0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.170+0000 7f7fb2200640 1 -- 172.21.15.46:0/3357088611 shutdown_connections 2024-03-29T08:22:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.170+0000 7f7fb2200640 1 -- 172.21.15.46:0/3357088611 wait complete. 2024-03-29T08:22:32.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.170+0000 7f7fb2200640 1 Processor -- start 2024-03-29T08:22:32.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.170+0000 7f7fb2200640 1 -- start start 2024-03-29T08:22:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fb2200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f900047d0 0x7f7fac0c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fb2200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fac06c3d0 0x7f7fac06f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fb2200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fac153a20 0x7f7fac15d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fb2200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac0c7cc0 con 0x7f7fac06c3d0 2024-03-29T08:22:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fb2200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac0c4c90 con 0x7f7fac153a20 2024-03-29T08:22:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fb2200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac0c4f70 con 0x7f7f900047d0 2024-03-29T08:22:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fa3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fac06c3d0 0x7f7fac06f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fa3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fac06c3d0 0x7f7fac06f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42258/0 (socket says 172.21.15.46:42258) 2024-03-29T08:22:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1040815969 learned_addr learned my addr 172.21.15.46:0/1040815969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fab400640 1 --2- 172.21.15.46:0/1040815969 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f900047d0 0x7f7fac0c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fabe00640 1 --2- 172.21.15.46:0/1040815969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fac153a20 0x7f7fac15d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1040815969 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f900047d0 msgr2=0x7f7fac0c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/1040815969 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f900047d0 0x7f7fac0c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1040815969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fac153a20 msgr2=0x7f7fac15d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/1040815969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fac153a20 0x7f7fac15d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fa3e00640 1 -- 172.21.15.46:0/1040815969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fac0c5250 con 0x7f7fac06c3d0 2024-03-29T08:22:32.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fa3e00640 1 --2- 172.21.15.46:0/1040815969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fac06c3d0 0x7f7fac06f470 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7f9401f200 tx=0x7f7f90005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:32.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fa3400640 1 -- 172.21.15.46:0/1040815969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94001640 con 0x7f7fac06c3d0 2024-03-29T08:22:32.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fa3400640 1 -- 172.21.15.46:0/1040815969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f94007070 con 0x7f7fac06c3d0 2024-03-29T08:22:32.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fa3400640 1 -- 172.21.15.46:0/1040815969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f94007420 con 0x7f7fac06c3d0 2024-03-29T08:22:32.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fb2200640 1 -- 172.21.15.46:0/1040815969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fac0c5530 con 0x7f7fac06c3d0 2024-03-29T08:22:32.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fb2200640 1 -- 172.21.15.46:0/1040815969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fac06f9b0 con 0x7f7fac06c3d0 2024-03-29T08:22:32.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fb2200640 1 -- 172.21.15.46:0/1040815969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fac066c90 con 0x7f7fac06c3d0 2024-03-29T08:22:32.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.174+0000 7f7fa3400640 1 -- 172.21.15.46:0/1040815969 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7f940b9030 con 0x7f7fac06c3d0 2024-03-29T08:22:32.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.178+0000 7f7fa3400640 1 --2- 172.21.15.46:0/1040815969 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7f88066690 0x7f7f88068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.178+0000 7f7fa3400640 1 -- 172.21.15.46:0/1040815969 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7f940b7070 con 0x7f7fac06c3d0 2024-03-29T08:22:32.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.178+0000 7f7fab400640 1 --2- 172.21.15.46:0/1040815969 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7f88066690 0x7f7f88068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.178+0000 7f7fab400640 1 --2- 172.21.15.46:0/1040815969 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7f88066690 0x7f7f88068b50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009c60 tx=0x7f7f9c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:32.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.182+0000 7f7fa3400640 1 -- 172.21.15.46:0/1040815969 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7f940ee200 con 0x7f7fac06c3d0 2024-03-29T08:22:32.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.374+0000 7f7fb2200640 1 -- 172.21.15.46:0/1040815969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fac0b2ec0 con 0x7f7fac06c3d0 2024-03-29T08:22:32.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.374+0000 7f7fa3400640 1 -- 172.21.15.46:0/1040815969 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4729 (secure 0 0 0) 0x7f7f940c1030 con 0x7f7fac06c3d0 2024-03-29T08:22:32.379 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:32.380 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2187826146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2187826146},{"type":"v1","addr":"172.21.15.73:6836","nonce":2187826146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/73382477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":73382477},{"type":"v1","addr":"172.21.15.73:6837","nonce":73382477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2874958170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2874958170},{"type":"v1","addr":"172.21.15.46:6834","nonce":2874958170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:21:10.337267+0000","modified":"2024-03-29T08:22:02.946081+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6051},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6051":{"gid":6051,"name":"c","rank":0,"incarnation":28,"state":"up:active","state_seq":16,"addr":"172.21.15.46:6835/3255474176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3255474176},{"type":"v1","addr":"172.21.15.46:6835","nonce":3255474176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-03-29T08:22:32.380 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 34 2024-03-29T08:22:32.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.378+0000 7f7fb2200640 1 -- 172.21.15.46:0/1040815969 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7f88066690 msgr2=0x7f7f88068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.378+0000 7f7fb2200640 1 --2- 172.21.15.46:0/1040815969 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7f88066690 0x7f7f88068b50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009c60 tx=0x7f7f9c009190 comp rx=0 tx=0).stop 2024-03-29T08:22:32.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.378+0000 7f7fb2200640 1 -- 172.21.15.46:0/1040815969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fac06c3d0 msgr2=0x7f7fac06f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.378+0000 7f7fb2200640 1 --2- 172.21.15.46:0/1040815969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fac06c3d0 0x7f7fac06f470 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7f9401f200 tx=0x7f7f90005800 comp rx=0 tx=0).stop 2024-03-29T08:22:32.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.378+0000 7f7fb2200640 1 -- 172.21.15.46:0/1040815969 shutdown_connections 2024-03-29T08:22:32.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.378+0000 7f7fb2200640 1 --2- 172.21.15.46:0/1040815969 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7f88066690 0x7f7f88068b50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.378+0000 7f7fb2200640 1 --2- 172.21.15.46:0/1040815969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fac153a20 0x7f7fac15d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.378+0000 7f7fb2200640 1 --2- 172.21.15.46:0/1040815969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7fac06c3d0 0x7f7fac06f470 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.378+0000 7f7fb2200640 1 --2- 172.21.15.46:0/1040815969 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7f900047d0 0x7f7fac0c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.378+0000 7f7fb2200640 1 -- 172.21.15.46:0/1040815969 >> 172.21.15.46:0/1040815969 conn(0x7f7fac0b21a0 msgr2=0x7f7fac0cc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:32.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.378+0000 7f7fb2200640 1 -- 172.21.15.46:0/1040815969 shutdown_connections 2024-03-29T08:22:32.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.378+0000 7f7fb2200640 1 -- 172.21.15.46:0/1040815969 wait complete. 2024-03-29T08:22:32.400 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:22:32.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.538+0000 7fb0e6c00640 1 Processor -- start 2024-03-29T08:22:32.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.538+0000 7fb0e6c00640 1 -- start start 2024-03-29T08:22:32.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.538+0000 7fb0e6c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb0e015cd90 0x7fb0e006cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.538+0000 7fb0e6c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb0e015d940 0x7fb0e01597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.538+0000 7fb0e6c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e006d130 con 0x7fb0e01593d0 2024-03-29T08:22:32.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.538+0000 7fb0e6c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e015b8f0 con 0x7fb0e015d940 2024-03-29T08:22:32.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.538+0000 7fb0e6c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e015ad50 con 0x7fb0e015cd90 2024-03-29T08:22:32.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.538+0000 7fb0e4c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb0e015d940 0x7fb0e01597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.538+0000 7fb0e4c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb0e015d940 0x7fb0e01597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42264/0 (socket says 172.21.15.46:42264) 2024-03-29T08:22:32.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.538+0000 7fb0e4c00640 1 -- 172.21.15.46:0/3139904987 learned_addr learned my addr 172.21.15.46:0/3139904987 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:32.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.538+0000 7fb0d7400640 1 --2- 172.21.15.46:0/3139904987 >> v2:172.21.15.73:3300/0 conn(0x7fb0e015cd90 0x7fb0e006cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.538+0000 7fb0e4c00640 1 -- 172.21.15.46:0/3139904987 >> v2:172.21.15.73:3300/0 conn(0x7fb0e015cd90 msgr2=0x7fb0e006cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:32.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.538+0000 7fb0e4c00640 1 --2- 172.21.15.46:0/3139904987 >> v2:172.21.15.73:3300/0 conn(0x7fb0e015cd90 0x7fb0e006cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.538+0000 7fb0e4c00640 1 -- 172.21.15.46:0/3139904987 >> v1:172.21.15.73:6789/0 conn(0x7fb0e01593d0 legacy=0x7fb0e015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.538+0000 7fb0e4c00640 1 -- 172.21.15.46:0/3139904987 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e015afc0 con 0x7fb0e015d940 2024-03-29T08:22:32.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.542+0000 7fb0e4c00640 1 --2- 172.21.15.46:0/3139904987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0e015d940 0x7fb0e01597b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00cf180 tx=0x7fb0d801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd70d856c15681c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:32.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.542+0000 7fb0df400640 1 -- 172.21.15.46:0/3139904987 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80a58b0 con 0x7fb0e015d940 2024-03-29T08:22:32.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.542+0000 7fb0df400640 1 -- 172.21.15.46:0/3139904987 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d80a5a10 con 0x7fb0e015d940 2024-03-29T08:22:32.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.542+0000 7fb0df400640 1 -- 172.21.15.46:0/3139904987 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80a5da0 con 0x7fb0e015d940 2024-03-29T08:22:32.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.542+0000 7fb0e6c00640 1 -- 172.21.15.46:0/3139904987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0e015d940 msgr2=0x7fb0e01597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.542+0000 7fb0e6c00640 1 --2- 172.21.15.46:0/3139904987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0e015d940 0x7fb0e01597b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb0e00cf180 tx=0x7fb0d801c9f0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.542+0000 7fb0e6c00640 1 -- 172.21.15.46:0/3139904987 shutdown_connections 2024-03-29T08:22:32.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.542+0000 7fb0e6c00640 1 --2- 172.21.15.46:0/3139904987 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0e015d940 0x7fb0e01597b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.542+0000 7fb0e6c00640 1 --2- 172.21.15.46:0/3139904987 >> v2:172.21.15.73:3300/0 conn(0x7fb0e015cd90 0x7fb0e006cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.542+0000 7fb0e6c00640 1 -- 172.21.15.46:0/3139904987 >> 172.21.15.46:0/3139904987 conn(0x7fb0e0066110 msgr2=0x7fb0e0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:32.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.542+0000 7fb0e6c00640 1 -- 172.21.15.46:0/3139904987 shutdown_connections 2024-03-29T08:22:32.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.542+0000 7fb0e6c00640 1 -- 172.21.15.46:0/3139904987 wait complete. 2024-03-29T08:22:32.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e6c00640 1 Processor -- start 2024-03-29T08:22:32.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e6c00640 1 -- start start 2024-03-29T08:22:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e6c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0e01593d0 0x7fb0e014d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e6c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb0e015cd90 0x7fb0e0152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e6c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0e015d940 0x7fb0e0152670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e6c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e014dc80 con 0x7fb0e015d940 2024-03-29T08:22:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e6c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e014df50 con 0x7fb0e01593d0 2024-03-29T08:22:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e6c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e014e230 con 0x7fb0e015cd90 2024-03-29T08:22:32.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0e015d940 0x7fb0e0152670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0e015d940 0x7fb0e0152670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42266/0 (socket says 172.21.15.46:42266) 2024-03-29T08:22:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e4c00640 1 -- 172.21.15.46:0/2832711413 learned_addr learned my addr 172.21.15.46:0/2832711413 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0dfe00640 1 --2- 172.21.15.46:0/2832711413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0e01593d0 0x7fb0e014d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0d7400640 1 --2- 172.21.15.46:0/2832711413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb0e015cd90 0x7fb0e0152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e4c00640 1 -- 172.21.15.46:0/2832711413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb0e015cd90 msgr2=0x7fb0e0152130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e4c00640 1 --2- 172.21.15.46:0/2832711413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb0e015cd90 0x7fb0e0152130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e4c00640 1 -- 172.21.15.46:0/2832711413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0e01593d0 msgr2=0x7fb0e014d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e4c00640 1 --2- 172.21.15.46:0/2832711413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0e01593d0 0x7fb0e014d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e4c00640 1 -- 172.21.15.46:0/2832711413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d8009b30 con 0x7fb0e015d940 2024-03-29T08:22:32.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e4c00640 1 --2- 172.21.15.46:0/2832711413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0e015d940 0x7fb0e0152670 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8019600 tx=0x7fb0d8019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:32.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0dcc00640 1 -- 172.21.15.46:0/2832711413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80b8670 con 0x7fb0e015d940 2024-03-29T08:22:32.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0dcc00640 1 -- 172.21.15.46:0/2832711413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d80b87f0 con 0x7fb0e015d940 2024-03-29T08:22:32.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0dcc00640 1 -- 172.21.15.46:0/2832711413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80b8ba0 con 0x7fb0e015d940 2024-03-29T08:22:32.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e6c00640 1 -- 172.21.15.46:0/2832711413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e014e510 con 0x7fb0e015d940 2024-03-29T08:22:32.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.546+0000 7fb0e6c00640 1 -- 172.21.15.46:0/2832711413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e014ead0 con 0x7fb0e015d940 2024-03-29T08:22:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.550+0000 7fb0e6c00640 1 -- 172.21.15.46:0/2832711413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e014ecc0 con 0x7fb0e015d940 2024-03-29T08:22:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.550+0000 7fb0dcc00640 1 -- 172.21.15.46:0/2832711413 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb0d80b8d00 con 0x7fb0e015d940 2024-03-29T08:22:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.550+0000 7fb0dcc00640 1 --2- 172.21.15.46:0/2832711413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb0bc066690 0x7fb0bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.550+0000 7fb0dcc00640 1 -- 172.21.15.46:0/2832711413 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb0d811f5e0 con 0x7fb0e015d940 2024-03-29T08:22:32.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.550+0000 7fb0dfe00640 1 --2- 172.21.15.46:0/2832711413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb0bc066690 0x7fb0bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.550+0000 7fb0dfe00640 1 --2- 172.21.15.46:0/2832711413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb0bc066690 0x7fb0bc068b50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0007fd0 tx=0x7fb0d0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:32.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.554+0000 7fb0dcc00640 1 -- 172.21.15.46:0/2832711413 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb0d80ec920 con 0x7fb0e015d940 2024-03-29T08:22:32.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.718+0000 7fb0e6c00640 1 -- 172.21.15.46:0/2832711413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0e0066c90 con 0x7fb0e015d940 2024-03-29T08:22:32.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.718+0000 7fb0dcc00640 1 -- 172.21.15.46:0/2832711413 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16145 (secure 0 0 0) 0x7fb0d80eff40 con 0x7fb0e015d940 2024-03-29T08:22:32.722 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:32.722 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":65,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:21:58.933917+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:21:08.061567+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-03-29T08:21:09.336464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.2f","mappings":[{"from":6,"to":1}]},{"pgid":"9.3b","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:22:32.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.722+0000 7fb0e6c00640 1 -- 172.21.15.46:0/2832711413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb0bc066690 msgr2=0x7fb0bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.722+0000 7fb0e6c00640 1 --2- 172.21.15.46:0/2832711413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb0bc066690 0x7fb0bc068b50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0007fd0 tx=0x7fb0d0009040 comp rx=0 tx=0).stop 2024-03-29T08:22:32.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.722+0000 7fb0e6c00640 1 -- 172.21.15.46:0/2832711413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0e015d940 msgr2=0x7fb0e0152670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.722+0000 7fb0e6c00640 1 --2- 172.21.15.46:0/2832711413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0e015d940 0x7fb0e0152670 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8019600 tx=0x7fb0d8019790 comp rx=0 tx=0).stop 2024-03-29T08:22:32.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.722+0000 7fb0e6c00640 1 -- 172.21.15.46:0/2832711413 shutdown_connections 2024-03-29T08:22:32.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.722+0000 7fb0e6c00640 1 --2- 172.21.15.46:0/2832711413 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb0bc066690 0x7fb0bc068b50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.722+0000 7fb0e6c00640 1 --2- 172.21.15.46:0/2832711413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb0e015d940 0x7fb0e0152670 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.722+0000 7fb0e6c00640 1 --2- 172.21.15.46:0/2832711413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb0e015cd90 0x7fb0e0152130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.722+0000 7fb0e6c00640 1 --2- 172.21.15.46:0/2832711413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0e01593d0 0x7fb0e014d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.722+0000 7fb0e6c00640 1 -- 172.21.15.46:0/2832711413 >> 172.21.15.46:0/2832711413 conn(0x7fb0e0066110 msgr2=0x7fb0e0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:32.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.722+0000 7fb0e6c00640 1 -- 172.21.15.46:0/2832711413 shutdown_connections 2024-03-29T08:22:32.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.722+0000 7fb0e6c00640 1 -- 172.21.15.46:0/2832711413 wait complete. 2024-03-29T08:22:32.743 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T08:22:32.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.882+0000 7f0a26800640 1 Processor -- start 2024-03-29T08:22:32.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.882+0000 7f0a26800640 1 -- start start 2024-03-29T08:22:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.882+0000 7f0a26800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0a2015c950 0x7f0a2006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.882+0000 7f0a26800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0a20160db0 0x7f0a200cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.882+0000 7f0a26800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a200cfb20 con 0x7f0a200cea30 2024-03-29T08:22:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.882+0000 7f0a26800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a200cef10 con 0x7f0a2015c950 2024-03-29T08:22:32.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.882+0000 7f0a26800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a200cf1f0 con 0x7f0a20160db0 2024-03-29T08:22:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.882+0000 7f0a1f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0a2015c950 0x7f0a2006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.882+0000 7f0a1fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0a20160db0 0x7f0a200cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.882+0000 7f0a1fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0a20160db0 0x7f0a200cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:54416/0 (socket says 172.21.15.46:54416) 2024-03-29T08:22:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.882+0000 7f0a1fe00640 1 -- 172.21.15.46:0/97952514 learned_addr learned my addr 172.21.15.46:0/97952514 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.882+0000 7f0a1f400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0a2015c950 0x7f0a2006bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:48330/0 (socket says 172.21.15.46:48330) 2024-03-29T08:22:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.882+0000 7f0a1f400640 1 -- 172.21.15.46:0/97952514 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0a20160db0 msgr2=0x7f0a200cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.882+0000 7f0a1f400640 1 --2- 172.21.15.46:0/97952514 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0a20160db0 0x7f0a200cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.882+0000 7f0a1f400640 1 -- 172.21.15.46:0/97952514 >> v1:172.21.15.73:6789/0 conn(0x7f0a200cea30 legacy=0x7f0a201606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.882+0000 7f0a1f400640 1 -- 172.21.15.46:0/97952514 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a200cf4d0 con 0x7f0a2015c950 2024-03-29T08:22:32.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.882+0000 7f0a1fe00640 1 --2- 172.21.15.46:0/97952514 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0a20160db0 0x7f0a200cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:22:32.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.886+0000 7f0a1f400640 1 --2- 172.21.15.46:0/97952514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a2015c950 0x7f0a2006bbe0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0a100097c0 tx=0x7f0a1001ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47ea3d8c2735bed5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:32.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.886+0000 7f0a1e000640 1 -- 172.21.15.46:0/97952514 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a100a8990 con 0x7f0a2015c950 2024-03-29T08:22:32.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.886+0000 7f0a1e000640 1 -- 172.21.15.46:0/97952514 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a100a8af0 con 0x7f0a2015c950 2024-03-29T08:22:32.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.886+0000 7f0a1e000640 1 -- 172.21.15.46:0/97952514 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a100a8e80 con 0x7f0a2015c950 2024-03-29T08:22:32.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.886+0000 7f0a26800640 1 -- 172.21.15.46:0/97952514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a2015c950 msgr2=0x7f0a2006bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.886+0000 7f0a26800640 1 --2- 172.21.15.46:0/97952514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a2015c950 0x7f0a2006bbe0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f0a100097c0 tx=0x7f0a1001ec90 comp rx=0 tx=0).stop 2024-03-29T08:22:32.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.886+0000 7f0a26800640 1 -- 172.21.15.46:0/97952514 shutdown_connections 2024-03-29T08:22:32.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.886+0000 7f0a26800640 1 --2- 172.21.15.46:0/97952514 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0a20160db0 0x7f0a200cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.886+0000 7f0a26800640 1 --2- 172.21.15.46:0/97952514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a2015c950 0x7f0a2006bbe0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.886+0000 7f0a26800640 1 -- 172.21.15.46:0/97952514 >> 172.21.15.46:0/97952514 conn(0x7f0a200b21b0 msgr2=0x7f0a200b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:32.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.886+0000 7f0a26800640 1 -- 172.21.15.46:0/97952514 shutdown_connections 2024-03-29T08:22:32.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.886+0000 7f0a26800640 1 -- 172.21.15.46:0/97952514 wait complete. 2024-03-29T08:22:32.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.886+0000 7f0a26800640 1 Processor -- start 2024-03-29T08:22:32.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a26800640 1 -- start start 2024-03-29T08:22:32.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a26800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0a200cea30 0x7f0a200c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a26800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a2015c950 0x7f0a201581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a26800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a20160db0 0x7f0a200b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a26800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2015a9c0 con 0x7f0a20160db0 2024-03-29T08:22:32.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a26800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a200b4ec0 con 0x7f0a2015c950 2024-03-29T08:22:32.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a26800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a200c5190 con 0x7f0a200cea30 2024-03-29T08:22:32.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a1fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a20160db0 0x7f0a200b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a1fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a20160db0 0x7f0a200b4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42278/0 (socket says 172.21.15.46:42278) 2024-03-29T08:22:32.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a1fe00640 1 -- 172.21.15.46:0/836601345 learned_addr learned my addr 172.21.15.46:0/836601345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:32.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a1ea00640 1 --2- 172.21.15.46:0/836601345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a2015c950 0x7f0a201581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a1f400640 1 --2- 172.21.15.46:0/836601345 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0a200cea30 0x7f0a200c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a1fe00640 1 -- 172.21.15.46:0/836601345 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0a200cea30 msgr2=0x7f0a200c61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a1fe00640 1 --2- 172.21.15.46:0/836601345 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0a200cea30 0x7f0a200c61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a1fe00640 1 -- 172.21.15.46:0/836601345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a2015c950 msgr2=0x7f0a201581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:32.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a1fe00640 1 --2- 172.21.15.46:0/836601345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a2015c950 0x7f0a201581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:32.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a1fe00640 1 -- 172.21.15.46:0/836601345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a18009b30 con 0x7f0a20160db0 2024-03-29T08:22:32.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a1fe00640 1 --2- 172.21.15.46:0/836601345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a20160db0 0x7f0a200b4980 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800ef90 tx=0x7f0a1800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:32.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a17400640 1 -- 172.21.15.46:0/836601345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a18018070 con 0x7f0a20160db0 2024-03-29T08:22:32.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a17400640 1 -- 172.21.15.46:0/836601345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1800f040 con 0x7f0a20160db0 2024-03-29T08:22:32.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a17400640 1 -- 172.21.15.46:0/836601345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1800ce20 con 0x7f0a20160db0 2024-03-29T08:22:32.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a26800640 1 -- 172.21.15.46:0/836601345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a10009420 con 0x7f0a20160db0 2024-03-29T08:22:32.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a26800640 1 -- 172.21.15.46:0/836601345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a200c5710 con 0x7f0a20160db0 2024-03-29T08:22:32.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.890+0000 7f0a26800640 1 -- 172.21.15.46:0/836601345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a200ccba0 con 0x7f0a20160db0 2024-03-29T08:22:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.894+0000 7f0a17400640 1 -- 172.21.15.46:0/836601345 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0a18014480 con 0x7f0a20160db0 2024-03-29T08:22:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.894+0000 7f0a17400640 1 --2- 172.21.15.46:0/836601345 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0a000666c0 0x7f0a00068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:32.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.894+0000 7f0a17400640 1 -- 172.21.15.46:0/836601345 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0a180842d0 con 0x7f0a20160db0 2024-03-29T08:22:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.894+0000 7f0a1f400640 1 --2- 172.21.15.46:0/836601345 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0a000666c0 0x7f0a00068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:32.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.894+0000 7f0a1f400640 1 --2- 172.21.15.46:0/836601345 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0a000666c0 0x7f0a00068b80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0a100b2000 tx=0x7f0a10008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:32.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:32.898+0000 7f0a17400640 1 -- 172.21.15.46:0/836601345 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0a180516a0 con 0x7f0a20160db0 2024-03-29T08:22:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.094+0000 7f0a26800640 1 -- 172.21.15.46:0/836601345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0a2009e690 con 0x7f0a20160db0 2024-03-29T08:22:33.098 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-03-29T08:22:33.094+0000 7fc16ae00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:22:33.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.102+0000 7f0a17400640 1 -- 172.21.15.46:0/836601345 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0a18054cc0 con 0x7f0a20160db0 2024-03-29T08:22:33.107 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T08:22:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.106+0000 7f0a26800640 1 -- 172.21.15.46:0/836601345 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0a000666c0 msgr2=0x7f0a00068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.106+0000 7f0a26800640 1 --2- 172.21.15.46:0/836601345 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0a000666c0 0x7f0a00068b80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0a100b2000 tx=0x7f0a10008040 comp rx=0 tx=0).stop 2024-03-29T08:22:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.106+0000 7f0a26800640 1 -- 172.21.15.46:0/836601345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a20160db0 msgr2=0x7f0a200b4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.106+0000 7f0a26800640 1 --2- 172.21.15.46:0/836601345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a20160db0 0x7f0a200b4980 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800ef90 tx=0x7f0a1800c490 comp rx=0 tx=0).stop 2024-03-29T08:22:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.106+0000 7f0a26800640 1 -- 172.21.15.46:0/836601345 shutdown_connections 2024-03-29T08:22:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.106+0000 7f0a26800640 1 --2- 172.21.15.46:0/836601345 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0a000666c0 0x7f0a00068b80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.106+0000 7f0a26800640 1 --2- 172.21.15.46:0/836601345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a2015c950 0x7f0a201581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.106+0000 7f0a26800640 1 --2- 172.21.15.46:0/836601345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a20160db0 0x7f0a200b4980 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:33.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.106+0000 7f0a26800640 1 --2- 172.21.15.46:0/836601345 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0a200cea30 0x7f0a200c61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.106+0000 7f0a26800640 1 -- 172.21.15.46:0/836601345 >> 172.21.15.46:0/836601345 conn(0x7f0a200b21b0 msgr2=0x7f0a2015d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.106+0000 7f0a26800640 1 -- 172.21.15.46:0/836601345 shutdown_connections 2024-03-29T08:22:33.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.106+0000 7f0a26800640 1 -- 172.21.15.46:0/836601345 wait complete. 2024-03-29T08:22:33.123 INFO:tasks.ceph.mds.c.smithi046.stderr: -6191> 2024-03-29T08:21:16.051+0000 7f1cc6c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:22:33.126 INFO:tasks.ceph.mds.c.smithi046.stderr: -4553> 2024-03-29T08:21:23.943+0000 7f1cc2600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6249=0-4194304@1} caps={6249=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6249 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x5598e82ca100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-29T08:22:33.130 INFO:tasks.ceph.mds.c.smithi046.stderr: -2731> 2024-03-29T08:21:55.538+0000 7f1cc2600640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-03-29T08:21:22.608086+0000 1=1+0) (iversion lock) caps={6249=pAsLsXsFs/-@2},l=6249(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x5598e82ca100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-29T08:22:33.136 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T08:22:33.175 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-03-29T08:22:33.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.278+0000 7f8b64800640 1 Processor -- start 2024-03-29T08:22:33.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.278+0000 7f8b64800640 1 -- start start 2024-03-29T08:22:33.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.278+0000 7f8b64800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b5c15c950 0x7f8b5c06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:33.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.278+0000 7f8b64800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8b5c0cea30 0x7f8b5c1606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:33.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.278+0000 7f8b64800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b5c0cfb20 con 0x7f8b5c15cd30 2024-03-29T08:22:33.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.278+0000 7f8b64800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b5c0cef10 con 0x7f8b5c15c950 2024-03-29T08:22:33.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.278+0000 7f8b64800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b5c0cf1f0 con 0x7f8b5c0cea30 2024-03-29T08:22:33.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.278+0000 7f8b61e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b5c15c950 0x7f8b5c06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:33.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.278+0000 7f8b61e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8b5c15c950 0x7f8b5c06bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42304/0 (socket says 172.21.15.46:42304) 2024-03-29T08:22:33.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.278+0000 7f8b61e00640 1 -- 172.21.15.46:0/3184250671 learned_addr learned my addr 172.21.15.46:0/3184250671 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.278+0000 7f8b61e00640 1 -- 172.21.15.46:0/3184250671 >> v2:172.21.15.73:3300/0 conn(0x7f8b5c0cea30 msgr2=0x7f8b5c1606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.278+0000 7f8b61e00640 1 --2- 172.21.15.46:0/3184250671 >> v2:172.21.15.73:3300/0 conn(0x7f8b5c0cea30 0x7f8b5c1606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.278+0000 7f8b61e00640 1 -- 172.21.15.46:0/3184250671 >> v1:172.21.15.73:6789/0 conn(0x7f8b5c15cd30 legacy=0x7f8b5c0cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:33.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.278+0000 7f8b61e00640 1 -- 172.21.15.46:0/3184250671 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c0cf4d0 con 0x7f8b5c15c950 2024-03-29T08:22:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.282+0000 7f8b61e00640 1 --2- 172.21.15.46:0/3184250671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b5c15c950 0x7f8b5c06bbe0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0c41c0 tx=0x7f8b5001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6903f956e3bf4c1d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.282+0000 7f8b60a00640 1 -- 172.21.15.46:0/3184250671 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b500a89d0 con 0x7f8b5c15c950 2024-03-29T08:22:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.282+0000 7f8b60a00640 1 -- 172.21.15.46:0/3184250671 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b500a8b30 con 0x7f8b5c15c950 2024-03-29T08:22:33.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.282+0000 7f8b60a00640 1 -- 172.21.15.46:0/3184250671 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b500b0440 con 0x7f8b5c15c950 2024-03-29T08:22:33.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.282+0000 7f8b64800640 1 -- 172.21.15.46:0/3184250671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b5c15c950 msgr2=0x7f8b5c06bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:33.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.282+0000 7f8b64800640 1 --2- 172.21.15.46:0/3184250671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b5c15c950 0x7f8b5c06bbe0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0c41c0 tx=0x7f8b5001ecd0 comp rx=0 tx=0).stop 2024-03-29T08:22:33.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.282+0000 7f8b64800640 1 -- 172.21.15.46:0/3184250671 shutdown_connections 2024-03-29T08:22:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.282+0000 7f8b64800640 1 --2- 172.21.15.46:0/3184250671 >> v2:172.21.15.73:3300/0 conn(0x7f8b5c0cea30 0x7f8b5c1606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.282+0000 7f8b64800640 1 --2- 172.21.15.46:0/3184250671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b5c15c950 0x7f8b5c06bbe0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.282+0000 7f8b64800640 1 -- 172.21.15.46:0/3184250671 >> 172.21.15.46:0/3184250671 conn(0x7f8b5c0b21b0 msgr2=0x7f8b5c0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.282+0000 7f8b64800640 1 -- 172.21.15.46:0/3184250671 shutdown_connections 2024-03-29T08:22:33.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.282+0000 7f8b64800640 1 -- 172.21.15.46:0/3184250671 wait complete. 2024-03-29T08:22:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.282+0000 7f8b64800640 1 Processor -- start 2024-03-29T08:22:33.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.282+0000 7f8b64800640 1 -- start start 2024-03-29T08:22:33.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b64800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b5c0cea30 0x7f8b5c153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b64800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8b5c15c950 0x7f8b5c0b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b64800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b5c15cd30 0x7f8b5c1f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b64800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c155550 con 0x7f8b5c0cea30 2024-03-29T08:22:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b64800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c1557a0 con 0x7f8b5c15cd30 2024-03-29T08:22:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b64800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c155a80 con 0x7f8b5c15c950 2024-03-29T08:22:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b61e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b5c0cea30 0x7f8b5c153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b61e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b5c0cea30 0x7f8b5c153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42314/0 (socket says 172.21.15.46:42314) 2024-03-29T08:22:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b61e00640 1 -- 172.21.15.46:0/494832093 learned_addr learned my addr 172.21.15.46:0/494832093 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b62800640 1 --2- 172.21.15.46:0/494832093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b5c15cd30 0x7f8b5c1f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b61400640 1 --2- 172.21.15.46:0/494832093 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8b5c15c950 0x7f8b5c0b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b61e00640 1 -- 172.21.15.46:0/494832093 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8b5c15c950 msgr2=0x7f8b5c0b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b61e00640 1 --2- 172.21.15.46:0/494832093 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8b5c15c950 0x7f8b5c0b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:33.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b61e00640 1 -- 172.21.15.46:0/494832093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b5c15cd30 msgr2=0x7f8b5c1f2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:33.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b61e00640 1 --2- 172.21.15.46:0/494832093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b5c15cd30 0x7f8b5c1f2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:33.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b61e00640 1 -- 172.21.15.46:0/494832093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b500094d0 con 0x7f8b5c0cea30 2024-03-29T08:22:33.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b62800640 1 --2- 172.21.15.46:0/494832093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b5c15cd30 0x7f8b5c1f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:33.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b61e00640 1 --2- 172.21.15.46:0/494832093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b5c0cea30 0x7f8b5c153730 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8b500b3040 tx=0x7f8b5001ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:33.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b56000640 1 -- 172.21.15.46:0/494832093 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b500b0e00 con 0x7f8b5c0cea30 2024-03-29T08:22:33.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b56000640 1 -- 172.21.15.46:0/494832093 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b500bb470 con 0x7f8b5c0cea30 2024-03-29T08:22:33.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b56000640 1 -- 172.21.15.46:0/494832093 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b500bb820 con 0x7f8b5c0cea30 2024-03-29T08:22:33.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b64800640 1 -- 172.21.15.46:0/494832093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5c155d60 con 0x7f8b5c0cea30 2024-03-29T08:22:33.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b64800640 1 -- 172.21.15.46:0/494832093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5c156320 con 0x7f8b5c0cea30 2024-03-29T08:22:33.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.286+0000 7f8b64800640 1 -- 172.21.15.46:0/494832093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b5c09e690 con 0x7f8b5c0cea30 2024-03-29T08:22:33.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.290+0000 7f8b56000640 1 -- 172.21.15.46:0/494832093 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8b500b7070 con 0x7f8b5c0cea30 2024-03-29T08:22:33.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.290+0000 7f8b56000640 1 --2- 172.21.15.46:0/494832093 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8b40066690 0x7f8b40068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:33.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.290+0000 7f8b56000640 1 -- 172.21.15.46:0/494832093 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f8b50122700 con 0x7f8b5c0cea30 2024-03-29T08:22:33.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.290+0000 7f8b61400640 1 --2- 172.21.15.46:0/494832093 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8b40066690 0x7f8b40068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:33.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.290+0000 7f8b61400640 1 --2- 172.21.15.46:0/494832093 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8b40066690 0x7f8b40068b50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c006fd0 tx=0x7f8b4c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:33.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.294+0000 7f8b56000640 1 -- 172.21.15.46:0/494832093 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8b500ef950 con 0x7f8b5c0cea30 2024-03-29T08:22:33.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:33.490+0000 7f8b64800640 1 -- 172.21.15.46:0/494832093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8b5c09e690 con 0x7f8b5c0cea30 2024-03-29T08:22:34.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.106+0000 7f8b56000640 1 -- 172.21.15.46:0/494832093 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8b500f2f70 con 0x7f8b5c0cea30 2024-03-29T08:22:34.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.110+0000 7f8b64800640 1 -- 172.21.15.46:0/494832093 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8b40066690 msgr2=0x7f8b40068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:34.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.110+0000 7f8b64800640 1 --2- 172.21.15.46:0/494832093 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8b40066690 0x7f8b40068b50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c006fd0 tx=0x7f8b4c008040 comp rx=0 tx=0).stop 2024-03-29T08:22:34.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.110+0000 7f8b64800640 1 -- 172.21.15.46:0/494832093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b5c0cea30 msgr2=0x7f8b5c153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:34.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.110+0000 7f8b64800640 1 --2- 172.21.15.46:0/494832093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b5c0cea30 0x7f8b5c153730 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8b500b3040 tx=0x7f8b5001ba70 comp rx=0 tx=0).stop 2024-03-29T08:22:34.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.110+0000 7f8b64800640 1 -- 172.21.15.46:0/494832093 shutdown_connections 2024-03-29T08:22:34.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.110+0000 7f8b64800640 1 --2- 172.21.15.46:0/494832093 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8b40066690 0x7f8b40068b50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:34.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.110+0000 7f8b64800640 1 --2- 172.21.15.46:0/494832093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b5c15cd30 0x7f8b5c1f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:34.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.110+0000 7f8b64800640 1 --2- 172.21.15.46:0/494832093 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8b5c15c950 0x7f8b5c0b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:34.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.110+0000 7f8b64800640 1 --2- 172.21.15.46:0/494832093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b5c0cea30 0x7f8b5c153730 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:34.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.110+0000 7f8b64800640 1 -- 172.21.15.46:0/494832093 >> 172.21.15.46:0/494832093 conn(0x7f8b5c0b21b0 msgr2=0x7f8b5c15f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:34.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.110+0000 7f8b64800640 1 -- 172.21.15.46:0/494832093 shutdown_connections 2024-03-29T08:22:34.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.110+0000 7f8b64800640 1 -- 172.21.15.46:0/494832093 wait complete. 2024-03-29T08:22:34.135 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T08:22:34.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.318+0000 7ff4be400640 1 Processor -- start 2024-03-29T08:22:34.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.318+0000 7ff4be400640 1 -- start start 2024-03-29T08:22:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.322+0000 7ff4be400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4b81590c0 0x7ff4b806d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.322+0000 7ff4be400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff4b80b21a0 0x7ff4b8070180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.322+0000 7ff4be400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b80b1470 con 0x7ff4b806d8e0 2024-03-29T08:22:34.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.322+0000 7ff4be400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b806de60 con 0x7ff4b81590c0 2024-03-29T08:22:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.322+0000 7ff4be400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b80b16c0 con 0x7ff4b80b21a0 2024-03-29T08:22:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.322+0000 7ff4b7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4b81590c0 0x7ff4b806d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.322+0000 7ff4b7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff4b81590c0 0x7ff4b806d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42330/0 (socket says 172.21.15.46:42330) 2024-03-29T08:22:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.322+0000 7ff4b7400640 1 -- 172.21.15.46:0/4029925656 learned_addr learned my addr 172.21.15.46:0/4029925656 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.322+0000 7ff4aea00640 1 --2- 172.21.15.46:0/4029925656 >> v2:172.21.15.73:3300/0 conn(0x7ff4b80b21a0 0x7ff4b8070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.322+0000 7ff4b7400640 1 -- 172.21.15.46:0/4029925656 >> v2:172.21.15.73:3300/0 conn(0x7ff4b80b21a0 msgr2=0x7ff4b8070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.322+0000 7ff4b7400640 1 --2- 172.21.15.46:0/4029925656 >> v2:172.21.15.73:3300/0 conn(0x7ff4b80b21a0 0x7ff4b8070180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.322+0000 7ff4b7400640 1 -- 172.21.15.46:0/4029925656 >> v1:172.21.15.73:6789/0 conn(0x7ff4b806d8e0 legacy=0x7ff4b815cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.322+0000 7ff4b7400640 1 -- 172.21.15.46:0/4029925656 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b80b19a0 con 0x7ff4b81590c0 2024-03-29T08:22:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4b7400640 1 --2- 172.21.15.46:0/4029925656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b81590c0 0x7ff4b806d3a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff4b806e690 tx=0x7ff4a801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ebb392b306dc1a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4b6a00640 1 -- 172.21.15.46:0/4029925656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80a6430 con 0x7ff4b81590c0 2024-03-29T08:22:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4b6a00640 1 -- 172.21.15.46:0/4029925656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a80a6590 con 0x7ff4b81590c0 2024-03-29T08:22:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4b6a00640 1 -- 172.21.15.46:0/4029925656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80a6920 con 0x7ff4b81590c0 2024-03-29T08:22:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4be400640 1 -- 172.21.15.46:0/4029925656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b81590c0 msgr2=0x7ff4b806d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4be400640 1 --2- 172.21.15.46:0/4029925656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b81590c0 0x7ff4b806d3a0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff4b806e690 tx=0x7ff4a801c9f0 comp rx=0 tx=0).stop 2024-03-29T08:22:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4be400640 1 -- 172.21.15.46:0/4029925656 shutdown_connections 2024-03-29T08:22:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4be400640 1 --2- 172.21.15.46:0/4029925656 >> v2:172.21.15.73:3300/0 conn(0x7ff4b80b21a0 0x7ff4b8070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4be400640 1 --2- 172.21.15.46:0/4029925656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b81590c0 0x7ff4b806d3a0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4be400640 1 -- 172.21.15.46:0/4029925656 >> 172.21.15.46:0/4029925656 conn(0x7ff4b80b42d0 msgr2=0x7ff4b8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4be400640 1 -- 172.21.15.46:0/4029925656 shutdown_connections 2024-03-29T08:22:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4be400640 1 -- 172.21.15.46:0/4029925656 wait complete. 2024-03-29T08:22:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4be400640 1 Processor -- start 2024-03-29T08:22:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4be400640 1 -- start start 2024-03-29T08:22:34.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4be400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b806d8e0 0x7ff4b815bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4be400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff4b80b21a0 0x7ff4b8150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4be400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b81590c0 0x7ff4b81511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4be400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b814c1c0 con 0x7ff4b806d8e0 2024-03-29T08:22:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4be400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b814c410 con 0x7ff4b81590c0 2024-03-29T08:22:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4be400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b814c6f0 con 0x7ff4b80b21a0 2024-03-29T08:22:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.326+0000 7ff4b7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b806d8e0 0x7ff4b815bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4b7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b806d8e0 0x7ff4b815bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42340/0 (socket says 172.21.15.46:42340) 2024-03-29T08:22:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4b7400640 1 -- 172.21.15.46:0/981603338 learned_addr learned my addr 172.21.15.46:0/981603338 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4aea00640 1 --2- 172.21.15.46:0/981603338 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff4b80b21a0 0x7ff4b8150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4b7e00640 1 --2- 172.21.15.46:0/981603338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b81590c0 0x7ff4b81511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4b7400640 1 -- 172.21.15.46:0/981603338 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff4b80b21a0 msgr2=0x7ff4b8150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4b7400640 1 --2- 172.21.15.46:0/981603338 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff4b80b21a0 0x7ff4b8150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4b7400640 1 -- 172.21.15.46:0/981603338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b81590c0 msgr2=0x7ff4b81511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4b7400640 1 --2- 172.21.15.46:0/981603338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b81590c0 0x7ff4b81511d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4b7400640 1 -- 172.21.15.46:0/981603338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a8009b30 con 0x7ff4b806d8e0 2024-03-29T08:22:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4b7400640 1 --2- 172.21.15.46:0/981603338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b806d8e0 0x7ff4b815bee0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8019600 tx=0x7ff4a8019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4afe00640 1 -- 172.21.15.46:0/981603338 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80a6430 con 0x7ff4b806d8e0 2024-03-29T08:22:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4afe00640 1 -- 172.21.15.46:0/981603338 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a80bd650 con 0x7ff4b806d8e0 2024-03-29T08:22:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4afe00640 1 -- 172.21.15.46:0/981603338 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a80bda00 con 0x7ff4b806d8e0 2024-03-29T08:22:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4be400640 1 -- 172.21.15.46:0/981603338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b814c9d0 con 0x7ff4b806d8e0 2024-03-29T08:22:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4be400640 1 -- 172.21.15.46:0/981603338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b814cf90 con 0x7ff4b806d8e0 2024-03-29T08:22:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4be400640 1 -- 172.21.15.46:0/981603338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b8066c90 con 0x7ff4b806d8e0 2024-03-29T08:22:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4afe00640 1 -- 172.21.15.46:0/981603338 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff4a80bdb60 con 0x7ff4b806d8e0 2024-03-29T08:22:34.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.330+0000 7ff4afe00640 1 --2- 172.21.15.46:0/981603338 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff494066690 0x7ff494068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:34.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.334+0000 7ff4afe00640 1 -- 172.21.15.46:0/981603338 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff4a811ea40 con 0x7ff4b806d8e0 2024-03-29T08:22:34.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.334+0000 7ff4aea00640 1 --2- 172.21.15.46:0/981603338 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff494066690 0x7ff494068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:34.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.334+0000 7ff4aea00640 1 --2- 172.21.15.46:0/981603338 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff494066690 0x7ff494068b50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff4b81f6a40 tx=0x7ff4a0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:34.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.334+0000 7ff4afe00640 1 -- 172.21.15.46:0/981603338 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff4a80ebdb0 con 0x7ff4b806d8e0 2024-03-29T08:22:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:34.498+0000 7ff4be400640 1 -- 172.21.15.46:0/981603338 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff4b806a260 con 0x7ff4b806d8e0 2024-03-29T08:22:35.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.106+0000 7ff4afe00640 1 -- 172.21.15.46:0/981603338 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff4a80c20d0 con 0x7ff4b806d8e0 2024-03-29T08:22:35.112 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-03-29T08:22:35.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.110+0000 7ff4be400640 1 -- 172.21.15.46:0/981603338 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff494066690 msgr2=0x7ff494068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:35.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.110+0000 7ff4be400640 1 --2- 172.21.15.46:0/981603338 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff494066690 0x7ff494068b50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff4b81f6a40 tx=0x7ff4a0008040 comp rx=0 tx=0).stop 2024-03-29T08:22:35.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.110+0000 7ff4be400640 1 -- 172.21.15.46:0/981603338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b806d8e0 msgr2=0x7ff4b815bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:35.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.110+0000 7ff4be400640 1 --2- 172.21.15.46:0/981603338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b806d8e0 0x7ff4b815bee0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8019600 tx=0x7ff4a8019790 comp rx=0 tx=0).stop 2024-03-29T08:22:35.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.110+0000 7ff4be400640 1 -- 172.21.15.46:0/981603338 shutdown_connections 2024-03-29T08:22:35.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.110+0000 7ff4be400640 1 --2- 172.21.15.46:0/981603338 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff494066690 0x7ff494068b50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:35.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.110+0000 7ff4be400640 1 --2- 172.21.15.46:0/981603338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b81590c0 0x7ff4b81511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:35.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.110+0000 7ff4be400640 1 --2- 172.21.15.46:0/981603338 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff4b80b21a0 0x7ff4b8150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:35.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.110+0000 7ff4be400640 1 --2- 172.21.15.46:0/981603338 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4b806d8e0 0x7ff4b815bee0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:35.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.110+0000 7ff4be400640 1 -- 172.21.15.46:0/981603338 >> 172.21.15.46:0/981603338 conn(0x7ff4b80b42d0 msgr2=0x7ff4b80ccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:35.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.110+0000 7ff4be400640 1 -- 172.21.15.46:0/981603338 shutdown_connections 2024-03-29T08:22:35.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.110+0000 7ff4be400640 1 -- 172.21.15.46:0/981603338 wait complete. 2024-03-29T08:22:35.138 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T08:22:35.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.282+0000 7fa719c00640 1 Processor -- start 2024-03-29T08:22:35.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.282+0000 7fa719c00640 1 -- start start 2024-03-29T08:22:35.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.282+0000 7fa719c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa714158b80 0x7fa714069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:35.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.282+0000 7fa719c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71406a720 con 0x7fa714073420 2024-03-29T08:22:35.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.282+0000 7fa719c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71415cb60 con 0x7fa71406a2d0 2024-03-29T08:22:35.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.282+0000 7fa719c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa71415c5e0 con 0x7fa714158b80 2024-03-29T08:22:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.286+0000 7fa712a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa714073420 0x7fa71415cff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48032/0 (socket says 172.21.15.46:48032) 2024-03-29T08:22:35.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.286+0000 7fa712a00640 1 -- 172.21.15.46:0/1829143738 learned_addr learned my addr 172.21.15.46:0/1829143738 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.286+0000 7fa70a000640 1 --2- 172.21.15.46:0/1829143738 >> v2:172.21.15.73:3300/0 conn(0x7fa714158b80 0x7fa714069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.286+0000 7fa70a000640 1 -- 172.21.15.46:0/1829143738 >> v1:172.21.15.73:6789/0 conn(0x7fa71406a2d0 legacy=0x7fa7140cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.286+0000 7fa70a000640 1 -- 172.21.15.46:0/1829143738 >> v1:172.21.15.46:6789/0 conn(0x7fa714073420 legacy=0x7fa71415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:35.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.286+0000 7fa70a000640 1 -- 172.21.15.46:0/1829143738 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71406dcb0 con 0x7fa714158b80 2024-03-29T08:22:35.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.290+0000 7fa70a000640 1 --2- 172.21.15.46:0/1829143738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa714158b80 0x7fa714069d90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0097c0 tx=0x7fa6fc01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8495205122a71df4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:35.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.290+0000 7fa712000640 1 -- 172.21.15.46:0/1829143738 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc0ac640 con 0x7fa714158b80 2024-03-29T08:22:35.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.290+0000 7fa712000640 1 -- 172.21.15.46:0/1829143738 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6fc0ac7a0 con 0x7fa714158b80 2024-03-29T08:22:35.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.290+0000 7fa712000640 1 -- 172.21.15.46:0/1829143738 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc0acb30 con 0x7fa714158b80 2024-03-29T08:22:35.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.294+0000 7fa719c00640 1 -- 172.21.15.46:0/1829143738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa714158b80 msgr2=0x7fa714069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:35.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.294+0000 7fa719c00640 1 --2- 172.21.15.46:0/1829143738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa714158b80 0x7fa714069d90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0097c0 tx=0x7fa6fc01c3e0 comp rx=0 tx=0).stop 2024-03-29T08:22:35.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.294+0000 7fa719c00640 1 -- 172.21.15.46:0/1829143738 shutdown_connections 2024-03-29T08:22:35.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.294+0000 7fa719c00640 1 --2- 172.21.15.46:0/1829143738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa714158b80 0x7fa714069d90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:35.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.294+0000 7fa719c00640 1 -- 172.21.15.46:0/1829143738 >> 172.21.15.46:0/1829143738 conn(0x7fa7140b05d0 msgr2=0x7fa7140b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:35.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.294+0000 7fa719c00640 1 -- 172.21.15.46:0/1829143738 shutdown_connections 2024-03-29T08:22:35.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.294+0000 7fa719c00640 1 -- 172.21.15.46:0/1829143738 wait complete. 2024-03-29T08:22:35.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.294+0000 7fa719c00640 1 Processor -- start 2024-03-29T08:22:35.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.294+0000 7fa719c00640 1 -- start start 2024-03-29T08:22:35.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa719c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa71406a2d0 0x7fa714148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:35.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa719c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714073420 0x7fa714157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:35.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa719c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa714158b80 0x7fa7141690b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:35.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa719c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7141500c0 con 0x7fa714073420 2024-03-29T08:22:35.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa719c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa714149510 con 0x7fa714158b80 2024-03-29T08:22:35.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa719c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa714150310 con 0x7fa71406a2d0 2024-03-29T08:22:35.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa70a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714073420 0x7fa714157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:35.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa70a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714073420 0x7fa714157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42350/0 (socket says 172.21.15.46:42350) 2024-03-29T08:22:35.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa70a000640 1 -- 172.21.15.46:0/484690230 learned_addr learned my addr 172.21.15.46:0/484690230 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:35.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa712a00640 1 --2- 172.21.15.46:0/484690230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa71406a2d0 0x7fa714148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:35.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa713400640 1 --2- 172.21.15.46:0/484690230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa714158b80 0x7fa7141690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:35.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa70a000640 1 -- 172.21.15.46:0/484690230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa71406a2d0 msgr2=0x7fa714148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:35.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa70a000640 1 --2- 172.21.15.46:0/484690230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa71406a2d0 0x7fa714148fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:35.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa70a000640 1 -- 172.21.15.46:0/484690230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa714158b80 msgr2=0x7fa7141690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:35.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa70a000640 1 --2- 172.21.15.46:0/484690230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa714158b80 0x7fa7141690b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:35.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa70a000640 1 -- 172.21.15.46:0/484690230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc009420 con 0x7fa714073420 2024-03-29T08:22:35.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa70a000640 1 --2- 172.21.15.46:0/484690230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714073420 0x7fa714157770 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc018ff0 tx=0x7fa6fc01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:35.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa70b400640 1 -- 172.21.15.46:0/484690230 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc0b7860 con 0x7fa714073420 2024-03-29T08:22:35.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa70b400640 1 -- 172.21.15.46:0/484690230 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6fc0b79e0 con 0x7fa714073420 2024-03-29T08:22:35.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa70b400640 1 -- 172.21.15.46:0/484690230 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc0b7d90 con 0x7fa714073420 2024-03-29T08:22:35.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa719c00640 1 -- 172.21.15.46:0/484690230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7141505f0 con 0x7fa714073420 2024-03-29T08:22:35.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa719c00640 1 -- 172.21.15.46:0/484690230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa714150bb0 con 0x7fa714073420 2024-03-29T08:22:35.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa719c00640 1 -- 172.21.15.46:0/484690230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa714150da0 con 0x7fa714073420 2024-03-29T08:22:35.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.298+0000 7fa70b400640 1 -- 172.21.15.46:0/484690230 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa6fc0b3070 con 0x7fa714073420 2024-03-29T08:22:35.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.302+0000 7fa70b400640 1 --2- 172.21.15.46:0/484690230 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6f0066690 0x7fa6f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:35.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.302+0000 7fa70b400640 1 -- 172.21.15.46:0/484690230 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 5415+0+0 (secure 0 0 0) 0x7fa6fc11eab0 con 0x7fa714073420 2024-03-29T08:22:35.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.302+0000 7fa712a00640 1 --2- 172.21.15.46:0/484690230 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6f0066690 0x7fa6f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:35.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.302+0000 7fa712a00640 1 --2- 172.21.15.46:0/484690230 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6f0066690 0x7fa6f0068b50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa704005890 tx=0x7fa704009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:35.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.306+0000 7fa70b400640 1 -- 172.21.15.46:0/484690230 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa6fc0ebf50 con 0x7fa714073420 2024-03-29T08:22:35.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:35.466+0000 7fa719c00640 1 -- 172.21.15.46:0/484690230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa714150450 con 0x7fa714073420 2024-03-29T08:22:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.114+0000 7fa70b400640 1 -- 172.21.15.46:0/484690230 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa714150450 con 0x7fa714073420 2024-03-29T08:22:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-03-29T08:22:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.118+0000 7fa719c00640 1 -- 172.21.15.46:0/484690230 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6f0066690 msgr2=0x7fa6f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.118+0000 7fa719c00640 1 --2- 172.21.15.46:0/484690230 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6f0066690 0x7fa6f0068b50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa704005890 tx=0x7fa704009040 comp rx=0 tx=0).stop 2024-03-29T08:22:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.118+0000 7fa719c00640 1 -- 172.21.15.46:0/484690230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714073420 msgr2=0x7fa714157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.118+0000 7fa719c00640 1 --2- 172.21.15.46:0/484690230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714073420 0x7fa714157770 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc018ff0 tx=0x7fa6fc01ac20 comp rx=0 tx=0).stop 2024-03-29T08:22:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.122+0000 7fa719c00640 1 -- 172.21.15.46:0/484690230 shutdown_connections 2024-03-29T08:22:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.122+0000 7fa719c00640 1 --2- 172.21.15.46:0/484690230 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6f0066690 0x7fa6f0068b50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.122+0000 7fa719c00640 1 --2- 172.21.15.46:0/484690230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa714158b80 0x7fa7141690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.122+0000 7fa719c00640 1 --2- 172.21.15.46:0/484690230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714073420 0x7fa714157770 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.122+0000 7fa719c00640 1 --2- 172.21.15.46:0/484690230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa71406a2d0 0x7fa714148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.122+0000 7fa719c00640 1 -- 172.21.15.46:0/484690230 >> 172.21.15.46:0/484690230 conn(0x7fa7140b05d0 msgr2=0x7fa714159060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:36.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.122+0000 7fa719c00640 1 -- 172.21.15.46:0/484690230 shutdown_connections 2024-03-29T08:22:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.122+0000 7fa719c00640 1 -- 172.21.15.46:0/484690230 wait complete. 2024-03-29T08:22:36.152 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-29T08:22:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.294+0000 7f1fbdc00640 1 Processor -- start 2024-03-29T08:22:36.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.294+0000 7f1fbdc00640 1 -- start start 2024-03-29T08:22:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.294+0000 7f1fbdc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1fb80d1070 0x7f1fb80cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.294+0000 7f1fbdc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fb815c1c0 0x7f1fb815c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.298+0000 7f1fbdc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb806e150 con 0x7f1fb80ccdd0 2024-03-29T08:22:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.298+0000 7f1fbdc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb80afe30 con 0x7f1fb80d1070 2024-03-29T08:22:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.298+0000 7f1fbdc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb80cfab0 con 0x7f1fb815c1c0 2024-03-29T08:22:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.298+0000 7f1fafe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1fb80d1070 0x7f1fb80cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.298+0000 7f1fafe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1fb80d1070 0x7f1fb80cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42354/0 (socket says 172.21.15.46:42354) 2024-03-29T08:22:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.298+0000 7f1fafe00640 1 -- 172.21.15.46:0/2722490247 learned_addr learned my addr 172.21.15.46:0/2722490247 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:36.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.298+0000 7f1fb7e00640 1 --1- 172.21.15.46:0/2722490247 >> v1:172.21.15.46:6789/0 conn(0x7f1fb80ccdd0 0x7f1fb80cd1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48036/0 (socket says 172.21.15.46:48036) 2024-03-29T08:22:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.298+0000 7f1fbd200640 1 --2- 172.21.15.46:0/2722490247 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fb815c1c0 0x7f1fb815c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.298+0000 7f1fafe00640 1 -- 172.21.15.46:0/2722490247 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fb815c1c0 msgr2=0x7f1fb815c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.298+0000 7f1fafe00640 1 --2- 172.21.15.46:0/2722490247 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fb815c1c0 0x7f1fb815c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.298+0000 7f1fafe00640 1 -- 172.21.15.46:0/2722490247 >> v1:172.21.15.46:6789/0 conn(0x7f1fb80ccdd0 legacy=0x7f1fb80cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.298+0000 7f1fafe00640 1 -- 172.21.15.46:0/2722490247 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb80cad40 con 0x7f1fb80d1070 2024-03-29T08:22:36.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.298+0000 7f1fb7400640 1 -- 172.21.15.46:0/2722490247 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084042032 0 0) 0x7f1fb806e150 con 0x7f1fb80ccdd0 2024-03-29T08:22:36.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.298+0000 7f1fafe00640 1 --2- 172.21.15.46:0/2722490247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb80d1070 0x7f1fb80cf3d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00098a0 tx=0x7f1fa001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f828bffa16083cf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.298+0000 7f1fb7400640 1 -- 172.21.15.46:0/2722490247 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00ad640 con 0x7f1fb80d1070 2024-03-29T08:22:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.298+0000 7f1fb7400640 1 -- 172.21.15.46:0/2722490247 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa00ad7a0 con 0x7f1fb80d1070 2024-03-29T08:22:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.298+0000 7f1fb7400640 1 -- 172.21.15.46:0/2722490247 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa00adb30 con 0x7f1fb80d1070 2024-03-29T08:22:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fbdc00640 1 -- 172.21.15.46:0/2722490247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb80d1070 msgr2=0x7f1fb80cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fbdc00640 1 --2- 172.21.15.46:0/2722490247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb80d1070 0x7f1fb80cf3d0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00098a0 tx=0x7f1fa001c480 comp rx=0 tx=0).stop 2024-03-29T08:22:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fbdc00640 1 -- 172.21.15.46:0/2722490247 shutdown_connections 2024-03-29T08:22:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fbdc00640 1 --2- 172.21.15.46:0/2722490247 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fb815c1c0 0x7f1fb815c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fbdc00640 1 --2- 172.21.15.46:0/2722490247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb80d1070 0x7f1fb80cf3d0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fbdc00640 1 -- 172.21.15.46:0/2722490247 >> 172.21.15.46:0/2722490247 conn(0x7f1fb80d0550 msgr2=0x7f1fb80d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fbdc00640 1 -- 172.21.15.46:0/2722490247 shutdown_connections 2024-03-29T08:22:36.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fbdc00640 1 -- 172.21.15.46:0/2722490247 wait complete. 2024-03-29T08:22:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fbdc00640 1 Processor -- start 2024-03-29T08:22:36.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fbdc00640 1 -- start start 2024-03-29T08:22:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fbdc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb80ccdd0 0x7f1fb80c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fbdc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fb80d1070 0x7f1fb80c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fbdc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fb815c1c0 0x7f1fb815a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fbdc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb8153f00 con 0x7f1fb80ccdd0 2024-03-29T08:22:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fbdc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb80c61f0 con 0x7f1fb80d1070 2024-03-29T08:22:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fbdc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb80c6460 con 0x7f1fb815c1c0 2024-03-29T08:22:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fb7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb80ccdd0 0x7f1fb80c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fb7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb80ccdd0 0x7f1fb80c5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42362/0 (socket says 172.21.15.46:42362) 2024-03-29T08:22:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fb7e00640 1 -- 172.21.15.46:0/4178170073 learned_addr learned my addr 172.21.15.46:0/4178170073 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fb7e00640 1 -- 172.21.15.46:0/4178170073 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fb815c1c0 msgr2=0x7f1fb815a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fb7e00640 1 --2- 172.21.15.46:0/4178170073 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fb815c1c0 0x7f1fb815a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fb7e00640 1 -- 172.21.15.46:0/4178170073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fb80d1070 msgr2=0x7f1fb80c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fb7e00640 1 --2- 172.21.15.46:0/4178170073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fb80d1070 0x7f1fb80c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.302+0000 7f1fb7e00640 1 -- 172.21.15.46:0/4178170073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa00094d0 con 0x7f1fb80ccdd0 2024-03-29T08:22:36.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.306+0000 7f1fb7e00640 1 --2- 172.21.15.46:0/4178170073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb80ccdd0 0x7f1fb80c5770 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800c4a0 tx=0x7f1fa800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.306+0000 7f1fb4c00640 1 -- 172.21.15.46:0/4178170073 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa80074e0 con 0x7f1fb80ccdd0 2024-03-29T08:22:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.306+0000 7f1fb4c00640 1 -- 172.21.15.46:0/4178170073 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa8007640 con 0x7f1fb80ccdd0 2024-03-29T08:22:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.306+0000 7f1fb4c00640 1 -- 172.21.15.46:0/4178170073 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa80049f0 con 0x7f1fb80ccdd0 2024-03-29T08:22:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.306+0000 7f1fbdc00640 1 -- 172.21.15.46:0/4178170073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb80c6740 con 0x7f1fb80ccdd0 2024-03-29T08:22:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.306+0000 7f1fbdc00640 1 -- 172.21.15.46:0/4178170073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb8153b20 con 0x7f1fb80ccdd0 2024-03-29T08:22:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.306+0000 7f1fbdc00640 1 -- 172.21.15.46:0/4178170073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb8154040 con 0x7f1fb80ccdd0 2024-03-29T08:22:36.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.306+0000 7f1fb4c00640 1 -- 172.21.15.46:0/4178170073 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1fa8004b50 con 0x7f1fb80ccdd0 2024-03-29T08:22:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.306+0000 7f1fb4c00640 1 --2- 172.21.15.46:0/4178170073 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f94066690 0x7f1f94068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:36.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.306+0000 7f1fb4c00640 1 -- 172.21.15.46:0/4178170073 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1fa8082f80 con 0x7f1fb80ccdd0 2024-03-29T08:22:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.306+0000 7f1fafe00640 1 --2- 172.21.15.46:0/4178170073 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f94066690 0x7f1f94068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:36.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.310+0000 7f1fafe00640 1 --2- 172.21.15.46:0/4178170073 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f94066690 0x7f1f94068b50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00af000 tx=0x7f1fa0004740 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:36.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.310+0000 7f1fb4c00640 1 -- 172.21.15.46:0/4178170073 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1fa8050690 con 0x7f1fb80ccdd0 2024-03-29T08:22:36.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.470+0000 7f1fbdc00640 1 -- 172.21.15.46:0/4178170073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1fb804e040 con 0x7f1fb80ccdd0 2024-03-29T08:22:36.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.474+0000 7f1fb4c00640 1 -- 172.21.15.46:0/4178170073 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1fa80180a0 con 0x7f1fb80ccdd0 2024-03-29T08:22:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.478+0000 7f1fbdc00640 1 -- 172.21.15.46:0/4178170073 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f94066690 msgr2=0x7f1f94068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:36.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.478+0000 7f1fbdc00640 1 --2- 172.21.15.46:0/4178170073 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f94066690 0x7f1f94068b50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00af000 tx=0x7f1fa0004740 comp rx=0 tx=0).stop 2024-03-29T08:22:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.478+0000 7f1fbdc00640 1 -- 172.21.15.46:0/4178170073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb80ccdd0 msgr2=0x7f1fb80c5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.478+0000 7f1fbdc00640 1 --2- 172.21.15.46:0/4178170073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb80ccdd0 0x7f1fb80c5770 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800c4a0 tx=0x7f1fa800c960 comp rx=0 tx=0).stop 2024-03-29T08:22:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.478+0000 7f1fbdc00640 1 -- 172.21.15.46:0/4178170073 shutdown_connections 2024-03-29T08:22:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.478+0000 7f1fbdc00640 1 --2- 172.21.15.46:0/4178170073 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f94066690 0x7f1f94068b50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.478+0000 7f1fbdc00640 1 --2- 172.21.15.46:0/4178170073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fb80d1070 0x7f1fb80c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.478+0000 7f1fbdc00640 1 --2- 172.21.15.46:0/4178170073 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1fb815c1c0 0x7f1fb815a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.478+0000 7f1fbdc00640 1 --2- 172.21.15.46:0/4178170073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fb80ccdd0 0x7f1fb80c5770 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.478+0000 7f1fbdc00640 1 -- 172.21.15.46:0/4178170073 >> 172.21.15.46:0/4178170073 conn(0x7f1fb80d0550 msgr2=0x7f1fb80aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.478+0000 7f1fbdc00640 1 -- 172.21.15.46:0/4178170073 shutdown_connections 2024-03-29T08:22:36.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.478+0000 7f1fbdc00640 1 -- 172.21.15.46:0/4178170073 wait complete. 2024-03-29T08:22:36.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-03-29T08:22:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd45e400640 1 Processor -- start 2024-03-29T08:22:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd45e400640 1 -- start start 2024-03-29T08:22:36.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd45e400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd458099810 0x7fd4580cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:36.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd45e400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4580cb5d0 con 0x7fd45815cb00 2024-03-29T08:22:36.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd45e400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4580caca0 con 0x7fd458099150 2024-03-29T08:22:36.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd45e400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd458099d10 con 0x7fd458099810 2024-03-29T08:22:36.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd45d000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd45815cb00 0x7fd458098a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38072/0 (socket says 172.21.15.46:38072) 2024-03-29T08:22:36.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd45d000640 1 -- 172.21.15.46:0/22165066 learned_addr learned my addr 172.21.15.46:0/22165066 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd45da00640 1 --2- 172.21.15.46:0/22165066 >> v2:172.21.15.73:3300/0 conn(0x7fd458099810 0x7fd4580cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd457400640 1 -- 172.21.15.46:0/22165066 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945415785 0 0) 0x7fd4580cb5d0 con 0x7fd45815cb00 2024-03-29T08:22:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd457400640 1 -- 172.21.15.46:0/22165066 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd440003530 con 0x7fd45815cb00 2024-03-29T08:22:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd457400640 1 -- 172.21.15.46:0/22165066 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1215540640 0 0) 0x7fd440003530 con 0x7fd45815cb00 2024-03-29T08:22:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd457400640 1 -- 172.21.15.46:0/22165066 >> v2:172.21.15.73:3300/0 conn(0x7fd458099810 msgr2=0x7fd4580cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd457400640 1 --2- 172.21.15.46:0/22165066 >> v2:172.21.15.73:3300/0 conn(0x7fd458099810 0x7fd4580cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd457400640 1 -- 172.21.15.46:0/22165066 >> v1:172.21.15.73:6789/0 conn(0x7fd458099150 legacy=0x7fd4580cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd457400640 1 -- 172.21.15.46:0/22165066 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4580caf80 con 0x7fd45815cb00 2024-03-29T08:22:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd457400640 1 -- 172.21.15.46:0/22165066 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fd44c001680 con 0x7fd45815cb00 2024-03-29T08:22:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd457400640 1 -- 172.21.15.46:0/22165066 >> v1:172.21.15.46:6789/0 conn(0x7fd45815cb00 legacy=0x7fd458098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd457400640 1 --2- 172.21.15.46:0/22165066 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd440003980 0x7fd440003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd457400640 1 --2- 172.21.15.46:0/22165066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd440004380 0x7fd440004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd457400640 1 --2- 172.21.15.46:0/22165066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4400050f0 0x7fd44000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd457400640 1 -- 172.21.15.46:0/22165066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd458099d10 con 0x7fd4400050f0 2024-03-29T08:22:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd457400640 1 -- 172.21.15.46:0/22165066 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44000c030 con 0x7fd440004380 2024-03-29T08:22:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd457400640 1 -- 172.21.15.46:0/22165066 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44000c1f0 con 0x7fd440003980 2024-03-29T08:22:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd45da00640 1 --2- 172.21.15.46:0/22165066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4400050f0 0x7fd44000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd45d000640 1 --2- 172.21.15.46:0/22165066 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd440003980 0x7fd440003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd457e00640 1 --2- 172.21.15.46:0/22165066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd440004380 0x7fd440004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd45da00640 1 -- 172.21.15.46:0/22165066 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd440003980 msgr2=0x7fd440003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd45da00640 1 --2- 172.21.15.46:0/22165066 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd440003980 0x7fd440003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd45da00640 1 -- 172.21.15.46:0/22165066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd440004380 msgr2=0x7fd440004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd45da00640 1 --2- 172.21.15.46:0/22165066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd440004380 0x7fd440004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.638+0000 7fd45da00640 1 -- 172.21.15.46:0/22165066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd440003530 con 0x7fd4400050f0 2024-03-29T08:22:36.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.642+0000 7fd45da00640 1 -- 172.21.15.46:0/22165066 reap_dead start 2024-03-29T08:22:36.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.642+0000 7fd45da00640 1 --2- 172.21.15.46:0/22165066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4400050f0 0x7fd44000b990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd450009f90 tx=0x7fd45001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72cd4d563e295e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:36.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.642+0000 7fd457400640 1 -- 172.21.15.46:0/22165066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4500a5c90 con 0x7fd4400050f0 2024-03-29T08:22:36.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.642+0000 7fd457400640 1 -- 172.21.15.46:0/22165066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45000f040 con 0x7fd4400050f0 2024-03-29T08:22:36.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.642+0000 7fd457400640 1 -- 172.21.15.46:0/22165066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4500039a0 con 0x7fd4400050f0 2024-03-29T08:22:36.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.642+0000 7fd45e400640 1 -- 172.21.15.46:0/22165066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4400050f0 msgr2=0x7fd44000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:36.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.642+0000 7fd45e400640 1 --2- 172.21.15.46:0/22165066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4400050f0 0x7fd44000b990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd450009f90 tx=0x7fd45001cc80 comp rx=0 tx=0).stop 2024-03-29T08:22:36.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.642+0000 7fd45e400640 1 -- 172.21.15.46:0/22165066 shutdown_connections 2024-03-29T08:22:36.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.642+0000 7fd45e400640 1 --2- 172.21.15.46:0/22165066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4400050f0 0x7fd44000b990 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.642+0000 7fd45e400640 1 -- 172.21.15.46:0/22165066 >> 172.21.15.46:0/22165066 conn(0x7fd4580b2010 msgr2=0x7fd4580b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:36.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.642+0000 7fd45e400640 1 -- 172.21.15.46:0/22165066 shutdown_connections 2024-03-29T08:22:36.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.642+0000 7fd45e400640 1 -- 172.21.15.46:0/22165066 wait complete. 2024-03-29T08:22:36.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45e400640 1 Processor -- start 2024-03-29T08:22:36.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45e400640 1 -- start start 2024-03-29T08:22:36.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4400050f0 0x7fd4580aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:36.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45e400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd4580b4590 0x7fd4580b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45e400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580b0560 0x7fd4580b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4580b0f20 con 0x7fd4400050f0 2024-03-29T08:22:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45e400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4580ad590 con 0x7fd4580b0560 2024-03-29T08:22:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45e400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4580ad870 con 0x7fd4580b4590 2024-03-29T08:22:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4400050f0 0x7fd4580aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4400050f0 0x7fd4580aea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60264/0 (socket says 172.21.15.46:60264) 2024-03-29T08:22:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45d000640 1 -- 172.21.15.46:0/125364876 learned_addr learned my addr 172.21.15.46:0/125364876 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd457e00640 1 --2- 172.21.15.46:0/125364876 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd4580b4590 0x7fd4580b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45d000640 1 -- 172.21.15.46:0/125364876 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd4580b4590 msgr2=0x7fd4580b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45da00640 1 --2- 172.21.15.46:0/125364876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580b0560 0x7fd4580b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45d000640 1 --2- 172.21.15.46:0/125364876 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd4580b4590 0x7fd4580b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45d000640 1 -- 172.21.15.46:0/125364876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580b0560 msgr2=0x7fd4580b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45d000640 1 --2- 172.21.15.46:0/125364876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580b0560 0x7fd4580b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:36.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45d000640 1 -- 172.21.15.46:0/125364876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4580adb50 con 0x7fd4400050f0 2024-03-29T08:22:36.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd457e00640 1 --2- 172.21.15.46:0/125364876 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd4580b4590 0x7fd4580b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:22:36.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45da00640 1 --2- 172.21.15.46:0/125364876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580b0560 0x7fd4580b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:22:36.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45d000640 1 --2- 172.21.15.46:0/125364876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4400050f0 0x7fd4580aea70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00e780 tx=0x7fd44c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:36.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd454c00640 1 -- 172.21.15.46:0/125364876 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c00c6c0 con 0x7fd4400050f0 2024-03-29T08:22:36.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd454c00640 1 -- 172.21.15.46:0/125364876 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c00c820 con 0x7fd4400050f0 2024-03-29T08:22:36.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45e400640 1 -- 172.21.15.46:0/125364876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4580ade30 con 0x7fd4400050f0 2024-03-29T08:22:36.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd454c00640 1 -- 172.21.15.46:0/125364876 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c00cbb0 con 0x7fd4400050f0 2024-03-29T08:22:36.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.646+0000 7fd45e400640 1 -- 172.21.15.46:0/125364876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4580aefb0 con 0x7fd4400050f0 2024-03-29T08:22:36.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.650+0000 7fd45e400640 1 -- 172.21.15.46:0/125364876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45804e040 con 0x7fd4400050f0 2024-03-29T08:22:36.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.650+0000 7fd454c00640 1 -- 172.21.15.46:0/125364876 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd44c00cd10 con 0x7fd4400050f0 2024-03-29T08:22:36.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.650+0000 7fd454c00640 1 --2- 172.21.15.46:0/125364876 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd438066690 0x7fd438068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:36.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.650+0000 7fd454c00640 1 -- 172.21.15.46:0/125364876 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd44c083ce0 con 0x7fd4400050f0 2024-03-29T08:22:36.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.650+0000 7fd457e00640 1 --2- 172.21.15.46:0/125364876 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd438066690 0x7fd438068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:36.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.650+0000 7fd457e00640 1 --2- 172.21.15.46:0/125364876 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd438066690 0x7fd438068b50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd458160570 tx=0x7fd44400ab10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:36.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.654+0000 7fd454c00640 1 -- 172.21.15.46:0/125364876 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd44c051340 con 0x7fd4400050f0 2024-03-29T08:22:36.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:36.874+0000 7fd45e400640 1 -- 172.21.15.46:0/125364876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7fd4581f1c20 con 0x7fd4400050f0 2024-03-29T08:22:37.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.114+0000 7fd454c00640 1 -- 172.21.15.46:0/125364876 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v227) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd44c054960 con 0x7fd4400050f0 2024-03-29T08:22:37.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.118+0000 7fd45e400640 1 -- 172.21.15.46:0/125364876 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd438066690 msgr2=0x7fd438068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:37.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.118+0000 7fd45e400640 1 --2- 172.21.15.46:0/125364876 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd438066690 0x7fd438068b50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd458160570 tx=0x7fd44400ab10 comp rx=0 tx=0).stop 2024-03-29T08:22:37.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.118+0000 7fd45e400640 1 -- 172.21.15.46:0/125364876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4400050f0 msgr2=0x7fd4580aea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:37.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.118+0000 7fd45e400640 1 --2- 172.21.15.46:0/125364876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4400050f0 0x7fd4580aea70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd44c00e780 tx=0x7fd44c00ec40 comp rx=0 tx=0).stop 2024-03-29T08:22:37.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.118+0000 7fd45e400640 1 -- 172.21.15.46:0/125364876 shutdown_connections 2024-03-29T08:22:37.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.118+0000 7fd45e400640 1 --2- 172.21.15.46:0/125364876 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd438066690 0x7fd438068b50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:37.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.118+0000 7fd45e400640 1 --2- 172.21.15.46:0/125364876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580b0560 0x7fd4580b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:37.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.118+0000 7fd45e400640 1 --2- 172.21.15.46:0/125364876 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd4580b4590 0x7fd4580b4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:37.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.118+0000 7fd45e400640 1 --2- 172.21.15.46:0/125364876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4400050f0 0x7fd4580aea70 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:37.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.118+0000 7fd45e400640 1 -- 172.21.15.46:0/125364876 >> 172.21.15.46:0/125364876 conn(0x7fd4580b2010 msgr2=0x7fd45815a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:37.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.118+0000 7fd45e400640 1 -- 172.21.15.46:0/125364876 shutdown_connections 2024-03-29T08:22:37.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.118+0000 7fd45e400640 1 -- 172.21.15.46:0/125364876 wait complete. 2024-03-29T08:22:37.141 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-03-29T08:22:37.141 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-03-29T08:22:37.142 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-29T08:22:37.142 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-03-29T08:22:37.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.278+0000 7f0817c00640 1 Processor -- start 2024-03-29T08:22:37.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.278+0000 7f0817c00640 1 -- start start 2024-03-29T08:22:37.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.278+0000 7f0817c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0810144210 0x7f08100cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:37.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.282+0000 7f0817c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f081015ada0 0x7f08100ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:37.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.282+0000 7f0817c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100cb510 con 0x7f08100ce950 2024-03-29T08:22:37.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.282+0000 7f0817c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08100cede0 con 0x7f081015ada0 2024-03-29T08:22:37.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.282+0000 7f0817c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081006d960 con 0x7f0810144210 2024-03-29T08:22:37.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.282+0000 7f0815c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f08100ce950 0x7f081015e360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38084/0 (socket says 172.21.15.46:38084) 2024-03-29T08:22:37.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.282+0000 7f0815c00640 1 -- 172.21.15.46:0/3460865424 learned_addr learned my addr 172.21.15.46:0/3460865424 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:37.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.282+0000 7f0815200640 1 --2- 172.21.15.46:0/3460865424 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0810144210 0x7f08100cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:37.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.282+0000 7f0807e00640 1 --2- 172.21.15.46:0/3460865424 >> v2:172.21.15.73:3300/0 conn(0x7f081015ada0 0x7f08100ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:37.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.282+0000 7f080fe00640 1 -- 172.21.15.46:0/3460865424 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910337533 0 0) 0x7f08100cb510 con 0x7f08100ce950 2024-03-29T08:22:37.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.282+0000 7f080fe00640 1 -- 172.21.15.46:0/3460865424 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07f4003530 con 0x7f08100ce950 2024-03-29T08:22:37.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.282+0000 7f0815200640 1 -- 172.21.15.46:0/3460865424 >> v2:172.21.15.73:3300/0 conn(0x7f081015ada0 msgr2=0x7f08100ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:37.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.282+0000 7f0815200640 1 --2- 172.21.15.46:0/3460865424 >> v2:172.21.15.73:3300/0 conn(0x7f081015ada0 0x7f08100ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:37.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.282+0000 7f0815200640 1 -- 172.21.15.46:0/3460865424 >> v1:172.21.15.46:6789/0 conn(0x7f08100ce950 legacy=0x7f081015e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:37.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.282+0000 7f0815200640 1 -- 172.21.15.46:0/3460865424 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0810070580 con 0x7f0810144210 2024-03-29T08:22:37.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0815200640 1 --2- 172.21.15.46:0/3460865424 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0810144210 0x7f08100cb8b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f08000097c0 tx=0x7f080001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5093a6bcd93923b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:37.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f080fe00640 1 -- 172.21.15.46:0/3460865424 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08000ab600 con 0x7f0810144210 2024-03-29T08:22:37.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f080fe00640 1 -- 172.21.15.46:0/3460865424 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08000a7050 con 0x7f0810144210 2024-03-29T08:22:37.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f080fe00640 1 -- 172.21.15.46:0/3460865424 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08000ab980 con 0x7f0810144210 2024-03-29T08:22:37.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0817c00640 1 -- 172.21.15.46:0/3460865424 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0810144210 msgr2=0x7f08100cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0817c00640 1 --2- 172.21.15.46:0/3460865424 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0810144210 0x7f08100cb8b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f08000097c0 tx=0x7f080001c4a0 comp rx=0 tx=0).stop 2024-03-29T08:22:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0817c00640 1 -- 172.21.15.46:0/3460865424 shutdown_connections 2024-03-29T08:22:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0817c00640 1 --2- 172.21.15.46:0/3460865424 >> v2:172.21.15.73:3300/0 conn(0x7f081015ada0 0x7f08100ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0817c00640 1 --2- 172.21.15.46:0/3460865424 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0810144210 0x7f08100cb8b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0817c00640 1 -- 172.21.15.46:0/3460865424 >> 172.21.15.46:0/3460865424 conn(0x7f0810066110 msgr2=0x7f0810069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0817c00640 1 -- 172.21.15.46:0/3460865424 shutdown_connections 2024-03-29T08:22:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0817c00640 1 -- 172.21.15.46:0/3460865424 wait complete. 2024-03-29T08:22:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0817c00640 1 Processor -- start 2024-03-29T08:22:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0817c00640 1 -- start start 2024-03-29T08:22:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0817c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08100ce950 0x7f0810159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0817c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0810144210 0x7f081013bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0817c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f081015ada0 0x7f0810142ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0817c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081015b550 con 0x7f081015ada0 2024-03-29T08:22:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0817c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0810141ca0 con 0x7f08100ce950 2024-03-29T08:22:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0817c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0810141f80 con 0x7f0810144210 2024-03-29T08:22:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0815c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f081015ada0 0x7f0810142ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0815c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f081015ada0 0x7f0810142ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60276/0 (socket says 172.21.15.46:60276) 2024-03-29T08:22:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0815c00640 1 -- 172.21.15.46:0/2728174030 learned_addr learned my addr 172.21.15.46:0/2728174030 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0807e00640 1 --2- 172.21.15.46:0/2728174030 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0810144210 0x7f081013bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.286+0000 7f0815200640 1 --2- 172.21.15.46:0/2728174030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08100ce950 0x7f0810159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.290+0000 7f0815c00640 1 -- 172.21.15.46:0/2728174030 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0810144210 msgr2=0x7f081013bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.290+0000 7f0815c00640 1 --2- 172.21.15.46:0/2728174030 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0810144210 0x7f081013bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.290+0000 7f0815c00640 1 -- 172.21.15.46:0/2728174030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08100ce950 msgr2=0x7f0810159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.290+0000 7f0815c00640 1 --2- 172.21.15.46:0/2728174030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08100ce950 0x7f0810159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.290+0000 7f0815c00640 1 -- 172.21.15.46:0/2728174030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0800009420 con 0x7f081015ada0 2024-03-29T08:22:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.290+0000 7f0807e00640 1 --2- 172.21.15.46:0/2728174030 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0810144210 0x7f081013bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:22:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.290+0000 7f0815c00640 1 --2- 172.21.15.46:0/2728174030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f081015ada0 0x7f0810142ec0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f080800caa0 tx=0x7f080800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.290+0000 7f080d600640 1 -- 172.21.15.46:0/2728174030 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080800ac00 con 0x7f081015ada0 2024-03-29T08:22:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.290+0000 7f080d600640 1 -- 172.21.15.46:0/2728174030 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080800ad60 con 0x7f081015ada0 2024-03-29T08:22:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.290+0000 7f080d600640 1 -- 172.21.15.46:0/2728174030 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0808012620 con 0x7f081015ada0 2024-03-29T08:22:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.290+0000 7f0817c00640 1 -- 172.21.15.46:0/2728174030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f081013b2e0 con 0x7f081015ada0 2024-03-29T08:22:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.290+0000 7f0817c00640 1 -- 172.21.15.46:0/2728174030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0810142260 con 0x7f081015ada0 2024-03-29T08:22:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.290+0000 7f0817c00640 1 -- 172.21.15.46:0/2728174030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08100ce280 con 0x7f081015ada0 2024-03-29T08:22:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.290+0000 7f080d600640 1 -- 172.21.15.46:0/2728174030 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0808007070 con 0x7f081015ada0 2024-03-29T08:22:37.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.290+0000 7f080d600640 1 --2- 172.21.15.46:0/2728174030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07ec066690 0x7f07ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:37.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.290+0000 7f080d600640 1 -- 172.21.15.46:0/2728174030 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0808083d30 con 0x7f081015ada0 2024-03-29T08:22:37.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.294+0000 7f0815200640 1 --2- 172.21.15.46:0/2728174030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07ec066690 0x7f07ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.294+0000 7f0815200640 1 --2- 172.21.15.46:0/2728174030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07ec066690 0x7f07ec068b50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f08000ad000 tx=0x7f08000a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.298+0000 7f080d600640 1 -- 172.21.15.46:0/2728174030 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f08080514e0 con 0x7f081015ada0 2024-03-29T08:22:37.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:37.514+0000 7f0817c00640 1 -- 172.21.15.46:0/2728174030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7f081013b800 con 0x7f081015ada0 2024-03-29T08:22:38.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.118+0000 7f080d600640 1 -- 172.21.15.46:0/2728174030 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v228) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0808054b00 con 0x7f081015ada0 2024-03-29T08:22:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.118+0000 7f0817c00640 1 -- 172.21.15.46:0/2728174030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07ec066690 msgr2=0x7f07ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.118+0000 7f0817c00640 1 --2- 172.21.15.46:0/2728174030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07ec066690 0x7f07ec068b50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f08000ad000 tx=0x7f08000a6000 comp rx=0 tx=0).stop 2024-03-29T08:22:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.118+0000 7f0817c00640 1 -- 172.21.15.46:0/2728174030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f081015ada0 msgr2=0x7f0810142ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.118+0000 7f0817c00640 1 --2- 172.21.15.46:0/2728174030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f081015ada0 0x7f0810142ec0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f080800caa0 tx=0x7f080800cf60 comp rx=0 tx=0).stop 2024-03-29T08:22:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.118+0000 7f0817c00640 1 -- 172.21.15.46:0/2728174030 shutdown_connections 2024-03-29T08:22:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.118+0000 7f0817c00640 1 --2- 172.21.15.46:0/2728174030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07ec066690 0x7f07ec068b50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.118+0000 7f0817c00640 1 --2- 172.21.15.46:0/2728174030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f081015ada0 0x7f0810142ec0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.118+0000 7f0817c00640 1 --2- 172.21.15.46:0/2728174030 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0810144210 0x7f081013bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.118+0000 7f0817c00640 1 --2- 172.21.15.46:0/2728174030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08100ce950 0x7f0810159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.118+0000 7f0817c00640 1 -- 172.21.15.46:0/2728174030 >> 172.21.15.46:0/2728174030 conn(0x7f0810066110 msgr2=0x7f08100b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.118+0000 7f0817c00640 1 -- 172.21.15.46:0/2728174030 shutdown_connections 2024-03-29T08:22:38.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.118+0000 7f0817c00640 1 -- 172.21.15.46:0/2728174030 wait complete. 2024-03-29T08:22:38.145 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-29T08:22:38.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.286+0000 7f9e05400640 1 Processor -- start 2024-03-29T08:22:38.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.286+0000 7f9e05400640 1 -- start start 2024-03-29T08:22:38.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.290+0000 7f9e05400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9e0015a9f0 0x7f9e000ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:38.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.290+0000 7f9e05400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9e000ccf60 0x7f9e000cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:38.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.290+0000 7f9e05400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e0006eef0 0x7f9e0006f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:38.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.290+0000 7f9e05400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e000cd910 con 0x7f9e0006eef0 2024-03-29T08:22:38.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.290+0000 7f9e05400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e000cf590 con 0x7f9e0015a9f0 2024-03-29T08:22:38.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.290+0000 7f9e05400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0006e130 con 0x7f9e000ccf60 2024-03-29T08:22:38.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.290+0000 7f9e04a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e0006eef0 0x7f9e0006f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:38.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.290+0000 7f9e04a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e0006eef0 0x7f9e0006f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60286/0 (socket says 172.21.15.46:60286) 2024-03-29T08:22:38.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.290+0000 7f9e04a00640 1 -- 172.21.15.46:0/2832579279 learned_addr learned my addr 172.21.15.46:0/2832579279 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:38.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.290+0000 7f9df7e00640 1 --2- 172.21.15.46:0/2832579279 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9e000ccf60 0x7f9e000cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:38.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.290+0000 7f9dffe00640 1 --2- 172.21.15.46:0/2832579279 >> v2:172.21.15.73:3300/0 conn(0x7f9e0015a9f0 0x7f9e000ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:38.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.290+0000 7f9e04a00640 1 -- 172.21.15.46:0/2832579279 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9e000ccf60 msgr2=0x7f9e000cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:38.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.290+0000 7f9e04a00640 1 --2- 172.21.15.46:0/2832579279 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9e000ccf60 0x7f9e000cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.290+0000 7f9e04a00640 1 -- 172.21.15.46:0/2832579279 >> v2:172.21.15.73:3300/0 conn(0x7f9e0015a9f0 msgr2=0x7f9e000ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:38.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.290+0000 7f9e04a00640 1 --2- 172.21.15.46:0/2832579279 >> v2:172.21.15.73:3300/0 conn(0x7f9e0015a9f0 0x7f9e000ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.290+0000 7f9e04a00640 1 -- 172.21.15.46:0/2832579279 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0006e410 con 0x7f9e0006eef0 2024-03-29T08:22:38.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e04a00640 1 --2- 172.21.15.46:0/2832579279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e0006eef0 0x7f9e0006f360 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9e00071a30 tx=0x7f9df801f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724cc60a538a5c06 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:38.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9dff400640 1 -- 172.21.15.46:0/2832579279 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df80af400 con 0x7f9e0006eef0 2024-03-29T08:22:38.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9dff400640 1 -- 172.21.15.46:0/2832579279 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df80ab050 con 0x7f9e0006eef0 2024-03-29T08:22:38.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9dff400640 1 -- 172.21.15.46:0/2832579279 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df80af780 con 0x7f9e0006eef0 2024-03-29T08:22:38.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e05400640 1 -- 172.21.15.46:0/2832579279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e0006eef0 msgr2=0x7f9e0006f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:38.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e05400640 1 --2- 172.21.15.46:0/2832579279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e0006eef0 0x7f9e0006f360 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9e00071a30 tx=0x7f9df801f1e0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e05400640 1 -- 172.21.15.46:0/2832579279 shutdown_connections 2024-03-29T08:22:38.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e05400640 1 --2- 172.21.15.46:0/2832579279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e0006eef0 0x7f9e0006f360 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e05400640 1 --2- 172.21.15.46:0/2832579279 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9e000ccf60 0x7f9e000cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e05400640 1 --2- 172.21.15.46:0/2832579279 >> v2:172.21.15.73:3300/0 conn(0x7f9e0015a9f0 0x7f9e000ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e05400640 1 -- 172.21.15.46:0/2832579279 >> 172.21.15.46:0/2832579279 conn(0x7f9e000d0550 msgr2=0x7f9e000d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:38.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e05400640 1 -- 172.21.15.46:0/2832579279 shutdown_connections 2024-03-29T08:22:38.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e05400640 1 -- 172.21.15.46:0/2832579279 wait complete. 2024-03-29T08:22:38.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e05400640 1 Processor -- start 2024-03-29T08:22:38.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e05400640 1 -- start start 2024-03-29T08:22:38.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e05400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e0006eef0 0x7f9e001f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:38.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e05400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e000ccf60 0x7f9e000b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:38.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e05400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9e0015a9f0 0x7f9e000b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:38.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e05400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e000b32f0 con 0x7f9e0006eef0 2024-03-29T08:22:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e05400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e000b35d0 con 0x7f9e000ccf60 2024-03-29T08:22:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9e05400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e000b38b0 con 0x7f9e0015a9f0 2024-03-29T08:22:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.294+0000 7f9dffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e0006eef0 0x7f9e001f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9df7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e000ccf60 0x7f9e000b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9e04a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9e0015a9f0 0x7f9e000b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9dffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e0006eef0 0x7f9e001f2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60300/0 (socket says 172.21.15.46:60300) 2024-03-29T08:22:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9df7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e000ccf60 0x7f9e000b2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:53878/0 (socket says 172.21.15.46:53878) 2024-03-29T08:22:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9dffe00640 1 -- 172.21.15.46:0/3490696893 learned_addr learned my addr 172.21.15.46:0/3490696893 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9dffe00640 1 -- 172.21.15.46:0/3490696893 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9e0015a9f0 msgr2=0x7f9e000b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9dffe00640 1 --2- 172.21.15.46:0/3490696893 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9e0015a9f0 0x7f9e000b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9dffe00640 1 -- 172.21.15.46:0/3490696893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e000ccf60 msgr2=0x7f9e000b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9dffe00640 1 --2- 172.21.15.46:0/3490696893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e000ccf60 0x7f9e000b2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9dffe00640 1 -- 172.21.15.46:0/3490696893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df8009b30 con 0x7f9e0006eef0 2024-03-29T08:22:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9dffe00640 1 --2- 172.21.15.46:0/3490696893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e0006eef0 0x7f9e001f2b30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9df000d800 tx=0x7f9df000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:38.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9dfcc00640 1 -- 172.21.15.46:0/3490696893 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df000b6c0 con 0x7f9e0006eef0 2024-03-29T08:22:38.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9dfcc00640 1 -- 172.21.15.46:0/3490696893 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df000b820 con 0x7f9e0006eef0 2024-03-29T08:22:38.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9dfcc00640 1 -- 172.21.15.46:0/3490696893 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df000bb90 con 0x7f9e0006eef0 2024-03-29T08:22:38.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9e05400640 1 -- 172.21.15.46:0/3490696893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e000b3b90 con 0x7f9e0006eef0 2024-03-29T08:22:38.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9e05400640 1 -- 172.21.15.46:0/3490696893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e001f3070 con 0x7f9e0006eef0 2024-03-29T08:22:38.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9e05400640 1 -- 172.21.15.46:0/3490696893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e0004e040 con 0x7f9e0006eef0 2024-03-29T08:22:38.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9dfcc00640 1 -- 172.21.15.46:0/3490696893 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9df000bcf0 con 0x7f9e0006eef0 2024-03-29T08:22:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9dfcc00640 1 --2- 172.21.15.46:0/3490696893 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9ddc066690 0x7f9ddc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.298+0000 7f9dfcc00640 1 -- 172.21.15.46:0/3490696893 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9df00842f0 con 0x7f9e0006eef0 2024-03-29T08:22:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.302+0000 7f9df7e00640 1 --2- 172.21.15.46:0/3490696893 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9ddc066690 0x7f9ddc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:38.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.302+0000 7f9df7e00640 1 --2- 172.21.15.46:0/3490696893 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9ddc066690 0x7f9ddc068b50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9e000b62a0 tx=0x7f9de8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.306+0000 7f9dfcc00640 1 -- 172.21.15.46:0/3490696893 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9df0051a00 con 0x7f9e0006eef0 2024-03-29T08:22:38.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.478+0000 7f9e05400640 1 -- 172.21.15.46:0/3490696893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9e000cfed0 con 0x7f9e0006eef0 2024-03-29T08:22:38.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.482+0000 7f9dfcc00640 1 -- 172.21.15.46:0/3490696893 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9df0055020 con 0x7f9e0006eef0 2024-03-29T08:22:38.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.486+0000 7f9e05400640 1 -- 172.21.15.46:0/3490696893 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9ddc066690 msgr2=0x7f9ddc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:38.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.486+0000 7f9e05400640 1 --2- 172.21.15.46:0/3490696893 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9ddc066690 0x7f9ddc068b50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9e000b62a0 tx=0x7f9de8008040 comp rx=0 tx=0).stop 2024-03-29T08:22:38.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.486+0000 7f9e05400640 1 -- 172.21.15.46:0/3490696893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e0006eef0 msgr2=0x7f9e001f2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:38.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.486+0000 7f9e05400640 1 --2- 172.21.15.46:0/3490696893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e0006eef0 0x7f9e001f2b30 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9df000d800 tx=0x7f9df000dcc0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.486+0000 7f9e05400640 1 -- 172.21.15.46:0/3490696893 shutdown_connections 2024-03-29T08:22:38.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.486+0000 7f9e05400640 1 --2- 172.21.15.46:0/3490696893 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9ddc066690 0x7f9ddc068b50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.486+0000 7f9e05400640 1 --2- 172.21.15.46:0/3490696893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9e000ccf60 0x7f9e000b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.486+0000 7f9e05400640 1 --2- 172.21.15.46:0/3490696893 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9e0015a9f0 0x7f9e000b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.486+0000 7f9e05400640 1 --2- 172.21.15.46:0/3490696893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e0006eef0 0x7f9e001f2b30 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.486+0000 7f9e05400640 1 -- 172.21.15.46:0/3490696893 >> 172.21.15.46:0/3490696893 conn(0x7f9e000d0550 msgr2=0x7f9e0006c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:38.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.486+0000 7f9e05400640 1 -- 172.21.15.46:0/3490696893 shutdown_connections 2024-03-29T08:22:38.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.486+0000 7f9e05400640 1 -- 172.21.15.46:0/3490696893 wait complete. 2024-03-29T08:22:38.507 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T08:22:38.507 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:22:38.562 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:22:38.658 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T08:22:38.658 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:22:38.658 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:22:38.667 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:22:38.667 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:22:38.667 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:22:38.676 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:22:38.676 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:22:38.676 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:22:38.719 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:22:38.719 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:22:38.720 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:22:38.728 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:22:38.728 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-29T08:22:38.729 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:22:38.760 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:22:38.767 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:22:38.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.898+0000 7fa4f1200640 1 Processor -- start 2024-03-29T08:22:38.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.898+0000 7fa4f1200640 1 -- start start 2024-03-29T08:22:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.902+0000 7fa4f1200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa4ec153d50 0x7fa4ec1611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.902+0000 7fa4f1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4ec0ad3e0 0x7fa4ec0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.902+0000 7fa4f1200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4ec15d870 con 0x7fa4ec153970 2024-03-29T08:22:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.902+0000 7fa4f1200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4ec0addb0 con 0x7fa4ec0ad3e0 2024-03-29T08:22:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.902+0000 7fa4f1200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec15db50 con 0x7fa4ec153d50 2024-03-29T08:22:38.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.902+0000 7fa4ea800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4ec0ad3e0 0x7fa4ec0ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.902+0000 7fa4ea800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4ec0ad3e0 0x7fa4ec0ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60310/0 (socket says 172.21.15.46:60310) 2024-03-29T08:22:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.902+0000 7fa4ea800640 1 -- 172.21.15.46:0/2475413040 learned_addr learned my addr 172.21.15.46:0/2475413040 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.902+0000 7fa4e3e00640 1 --2- 172.21.15.46:0/2475413040 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa4ec153d50 0x7fa4ec1611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.902+0000 7fa4ea800640 1 -- 172.21.15.46:0/2475413040 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa4ec153d50 msgr2=0x7fa4ec1611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.902+0000 7fa4ea800640 1 --2- 172.21.15.46:0/2475413040 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa4ec153d50 0x7fa4ec1611e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.902+0000 7fa4ea800640 1 -- 172.21.15.46:0/2475413040 >> v1:172.21.15.73:6789/0 conn(0x7fa4ec153970 legacy=0x7fa4ec15cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:38.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.902+0000 7fa4ea800640 1 -- 172.21.15.46:0/2475413040 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ec15de30 con 0x7fa4ec0ad3e0 2024-03-29T08:22:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.902+0000 7fa4ea800640 1 --2- 172.21.15.46:0/2475413040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4ec0ad3e0 0x7fa4ec0ad870 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009e50 tx=0x7fa4e401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e83a4edef7540d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4e9400640 1 -- 172.21.15.46:0/2475413040 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e40a5870 con 0x7fa4ec0ad3e0 2024-03-29T08:22:38.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4e9400640 1 -- 172.21.15.46:0/2475413040 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e40a59d0 con 0x7fa4ec0ad3e0 2024-03-29T08:22:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4e9400640 1 -- 172.21.15.46:0/2475413040 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e40a5d60 con 0x7fa4ec0ad3e0 2024-03-29T08:22:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4f1200640 1 -- 172.21.15.46:0/2475413040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4ec0ad3e0 msgr2=0x7fa4ec0ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4f1200640 1 --2- 172.21.15.46:0/2475413040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4ec0ad3e0 0x7fa4ec0ad870 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa4e4009e50 tx=0x7fa4e401ca00 comp rx=0 tx=0).stop 2024-03-29T08:22:38.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4f1200640 1 -- 172.21.15.46:0/2475413040 shutdown_connections 2024-03-29T08:22:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4f1200640 1 --2- 172.21.15.46:0/2475413040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4ec0ad3e0 0x7fa4ec0ad870 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4f1200640 1 --2- 172.21.15.46:0/2475413040 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa4ec153d50 0x7fa4ec1611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4f1200640 1 -- 172.21.15.46:0/2475413040 >> 172.21.15.46:0/2475413040 conn(0x7fa4ec066110 msgr2=0x7fa4ec069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4f1200640 1 -- 172.21.15.46:0/2475413040 shutdown_connections 2024-03-29T08:22:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4f1200640 1 -- 172.21.15.46:0/2475413040 wait complete. 2024-03-29T08:22:38.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4f1200640 1 Processor -- start 2024-03-29T08:22:38.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4f1200640 1 -- start start 2024-03-29T08:22:38.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4f1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4ec0ad3e0 0x7fa4ec0cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:38.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4f1200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa4ec153970 0x7fa4ec13e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:38.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4f1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4ec141880 0x7fa4ec142c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:38.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4f1200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec0cda40 con 0x7fa4ec141880 2024-03-29T08:22:38.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4f1200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec150e40 con 0x7fa4ec0ad3e0 2024-03-29T08:22:38.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4f1200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ec0cb840 con 0x7fa4ec153970 2024-03-29T08:22:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4ea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4ec141880 0x7fa4ec142c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4ea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4ec141880 0x7fa4ec142c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60318/0 (socket says 172.21.15.46:60318) 2024-03-29T08:22:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4ea800640 1 -- 172.21.15.46:0/2663132762 learned_addr learned my addr 172.21.15.46:0/2663132762 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4e3e00640 1 --2- 172.21.15.46:0/2663132762 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa4ec153970 0x7fa4ec13e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.906+0000 7fa4e9e00640 1 --2- 172.21.15.46:0/2663132762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4ec0ad3e0 0x7fa4ec0cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.910+0000 7fa4ea800640 1 -- 172.21.15.46:0/2663132762 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa4ec153970 msgr2=0x7fa4ec13e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.910+0000 7fa4ea800640 1 --2- 172.21.15.46:0/2663132762 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa4ec153970 0x7fa4ec13e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.910+0000 7fa4ea800640 1 -- 172.21.15.46:0/2663132762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4ec0ad3e0 msgr2=0x7fa4ec0cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.910+0000 7fa4ea800640 1 --2- 172.21.15.46:0/2663132762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4ec0ad3e0 0x7fa4ec0cd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.910+0000 7fa4ea800640 1 -- 172.21.15.46:0/2663132762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e4009b30 con 0x7fa4ec141880 2024-03-29T08:22:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.910+0000 7fa4e9e00640 1 --2- 172.21.15.46:0/2663132762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4ec0ad3e0 0x7fa4ec0cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:38.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.910+0000 7fa4ea800640 1 --2- 172.21.15.46:0/2663132762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4ec141880 0x7fa4ec142c10 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa4e40af040 tx=0x7fa4e40197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.910+0000 7fa4e2000640 1 -- 172.21.15.46:0/2663132762 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e40b8670 con 0x7fa4ec141880 2024-03-29T08:22:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.910+0000 7fa4e2000640 1 -- 172.21.15.46:0/2663132762 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e40b87f0 con 0x7fa4ec141880 2024-03-29T08:22:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.910+0000 7fa4e2000640 1 -- 172.21.15.46:0/2663132762 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e40b8ba0 con 0x7fa4ec141880 2024-03-29T08:22:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.910+0000 7fa4f1200640 1 -- 172.21.15.46:0/2663132762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4ec0cbb20 con 0x7fa4ec141880 2024-03-29T08:22:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.910+0000 7fa4f1200640 1 -- 172.21.15.46:0/2663132762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ec151100 con 0x7fa4ec141880 2024-03-29T08:22:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.910+0000 7fa4f1200640 1 -- 172.21.15.46:0/2663132762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4ec0cbfd0 con 0x7fa4ec141880 2024-03-29T08:22:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.910+0000 7fa4e2000640 1 -- 172.21.15.46:0/2663132762 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa4e40b8d00 con 0x7fa4ec141880 2024-03-29T08:22:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.910+0000 7fa4e2000640 1 --2- 172.21.15.46:0/2663132762 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa4c8066690 0x7fa4c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.910+0000 7fa4e2000640 1 -- 172.21.15.46:0/2663132762 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa4e411f4e0 con 0x7fa4ec141880 2024-03-29T08:22:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.910+0000 7fa4e9e00640 1 --2- 172.21.15.46:0/2663132762 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa4c8066690 0x7fa4c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:38.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.914+0000 7fa4e9e00640 1 --2- 172.21.15.46:0/2663132762 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa4c8066690 0x7fa4c8068b50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc000f70 tx=0x7fa4dc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:38.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:38.914+0000 7fa4e2000640 1 -- 172.21.15.46:0/2663132762 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa4e40ecbd0 con 0x7fa4ec141880 2024-03-29T08:22:39.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.114+0000 7fa4f1200640 1 -- 172.21.15.46:0/2663132762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4ec066c90 con 0x7fa4ec141880 2024-03-29T08:22:39.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.114+0000 7fa4e2000640 1 -- 172.21.15.46:0/2663132762 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+3523 (secure 0 0 0) 0x7fa4e40f01f0 con 0x7fa4ec141880 2024-03-29T08:22:39.117 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:39.118 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6013,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2187826146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2187826146},{"type":"v1","addr":"172.21.15.73:6836","nonce":2187826146}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6016,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/73382477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":73382477},{"type":"v1","addr":"172.21.15.73:6837","nonce":73382477}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2874958170","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2874958170},{"type":"v1","addr":"172.21.15.46:6834","nonce":2874958170}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6564,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3624999276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":3624999276},{"type":"v1","addr":"172.21.15.46:6835","nonce":3624999276}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36}],"filesystems":[]} 2024-03-29T08:22:39.118 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 36 2024-03-29T08:22:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.114+0000 7fa4f1200640 1 -- 172.21.15.46:0/2663132762 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa4c8066690 msgr2=0x7fa4c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.114+0000 7fa4f1200640 1 --2- 172.21.15.46:0/2663132762 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa4c8066690 0x7fa4c8068b50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc000f70 tx=0x7fa4dc009040 comp rx=0 tx=0).stop 2024-03-29T08:22:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.114+0000 7fa4f1200640 1 -- 172.21.15.46:0/2663132762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4ec141880 msgr2=0x7fa4ec142c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.114+0000 7fa4f1200640 1 --2- 172.21.15.46:0/2663132762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4ec141880 0x7fa4ec142c10 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa4e40af040 tx=0x7fa4e40197a0 comp rx=0 tx=0).stop 2024-03-29T08:22:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.114+0000 7fa4f1200640 1 -- 172.21.15.46:0/2663132762 shutdown_connections 2024-03-29T08:22:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.114+0000 7fa4f1200640 1 --2- 172.21.15.46:0/2663132762 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa4c8066690 0x7fa4c8068b50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.114+0000 7fa4f1200640 1 --2- 172.21.15.46:0/2663132762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4ec141880 0x7fa4ec142c10 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.114+0000 7fa4f1200640 1 --2- 172.21.15.46:0/2663132762 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa4ec153970 0x7fa4ec13e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.114+0000 7fa4f1200640 1 --2- 172.21.15.46:0/2663132762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4ec0ad3e0 0x7fa4ec0cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.114+0000 7fa4f1200640 1 -- 172.21.15.46:0/2663132762 >> 172.21.15.46:0/2663132762 conn(0x7fa4ec066110 msgr2=0x7fa4ec06d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.118+0000 7fa4f1200640 1 -- 172.21.15.46:0/2663132762 shutdown_connections 2024-03-29T08:22:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.118+0000 7fa4f1200640 1 -- 172.21.15.46:0/2663132762 wait complete. 2024-03-29T08:22:39.139 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T08:22:39.139 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-29T08:22:39.139 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T08:22:39.139 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:22:39.139 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T08:22:39.140 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-29T08:22:39.140 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T08:22:39.140 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:22:39.140 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T08:22:39.140 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-29T08:22:39.140 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T08:22:39.140 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:22:39.141 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T08:22:39.141 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-29T08:22:39.141 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T08:22:39.141 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:22:39.155 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T08:22:39.155 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T08:22:39.156 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T08:22:39.156 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T08:22:39.157 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T08:22:39.157 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T08:22:39.157 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T08:22:39.158 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T08:22:39.201 INFO:tasks.ceph.mds.b:Started 2024-03-29T08:22:39.202 DEBUG:teuthology.parallel:result is None 2024-03-29T08:22:39.202 INFO:tasks.ceph.mds.a:Started 2024-03-29T08:22:39.202 DEBUG:teuthology.parallel:result is None 2024-03-29T08:22:39.203 INFO:tasks.ceph.mds.d:Started 2024-03-29T08:22:39.203 DEBUG:teuthology.parallel:result is None 2024-03-29T08:22:39.204 INFO:tasks.ceph.mds.c:Started 2024-03-29T08:22:39.204 DEBUG:teuthology.parallel:result is None 2024-03-29T08:22:39.204 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-29T08:22:39.290 INFO:tasks.ceph.mds.b.smithi073.stdout:starting mds.b at 2024-03-29T08:22:39.290 INFO:tasks.ceph.mds.d.smithi073.stdout:starting mds.d at 2024-03-29T08:22:39.290 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-03-29T08:22:39.292 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-03-29T08:22:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.350+0000 7fd865200640 1 Processor -- start 2024-03-29T08:22:39.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.350+0000 7fd865200640 1 -- start start 2024-03-29T08:22:39.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.350+0000 7fd865200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd86015a5d0 0x7fd8600b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:39.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.350+0000 7fd865200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8600b1640 0x7fd8600ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:39.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.350+0000 7fd865200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8600ce040 con 0x7fd8600cd500 2024-03-29T08:22:39.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.350+0000 7fd865200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8600b0ce0 con 0x7fd8600b1640 2024-03-29T08:22:39.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.350+0000 7fd865200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600cda80 con 0x7fd86015a5d0 2024-03-29T08:22:39.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.350+0000 7fd855400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8600b1640 0x7fd8600ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:39.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.350+0000 7fd855400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8600b1640 0x7fd8600ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60364/0 (socket says 172.21.15.46:60364) 2024-03-29T08:22:39.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.350+0000 7fd855400640 1 -- 172.21.15.46:0/3508892220 learned_addr learned my addr 172.21.15.46:0/3508892220 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:39.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.350+0000 7fd85de00640 1 --2- 172.21.15.46:0/3508892220 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd86015a5d0 0x7fd8600b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:39.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.350+0000 7fd85e800640 1 --1- 172.21.15.46:0/3508892220 >> v1:172.21.15.46:6789/0 conn(0x7fd8600cd500 0x7fd86006ae80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38090/0 (socket says 172.21.15.46:38090) 2024-03-29T08:22:39.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.350+0000 7fd855400640 1 -- 172.21.15.46:0/3508892220 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd86015a5d0 msgr2=0x7fd8600b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:39.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.350+0000 7fd855400640 1 --2- 172.21.15.46:0/3508892220 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd86015a5d0 0x7fd8600b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:39.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.350+0000 7fd855400640 1 -- 172.21.15.46:0/3508892220 >> v1:172.21.15.46:6789/0 conn(0x7fd8600cd500 legacy=0x7fd86006ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:39.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.350+0000 7fd855400640 1 -- 172.21.15.46:0/3508892220 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8600cdd60 con 0x7fd8600b1640 2024-03-29T08:22:39.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.354+0000 7fd855400640 1 --2- 172.21.15.46:0/3508892220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8600b1640 0x7fd8600ccfc0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd848009870 tx=0x7fd84801cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f9e0b6943870d8d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:39.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.354+0000 7fd85d400640 1 -- 172.21.15.46:0/3508892220 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480aa620 con 0x7fd8600b1640 2024-03-29T08:22:39.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.354+0000 7fd85d400640 1 -- 172.21.15.46:0/3508892220 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8480aa780 con 0x7fd8600b1640 2024-03-29T08:22:39.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.354+0000 7fd85d400640 1 -- 172.21.15.46:0/3508892220 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480aab10 con 0x7fd8600b1640 2024-03-29T08:22:39.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.354+0000 7fd865200640 1 -- 172.21.15.46:0/3508892220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8600b1640 msgr2=0x7fd8600ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:39.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.354+0000 7fd865200640 1 --2- 172.21.15.46:0/3508892220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8600b1640 0x7fd8600ccfc0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd848009870 tx=0x7fd84801cf00 comp rx=0 tx=0).stop 2024-03-29T08:22:39.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.354+0000 7fd865200640 1 -- 172.21.15.46:0/3508892220 shutdown_connections 2024-03-29T08:22:39.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.354+0000 7fd865200640 1 --2- 172.21.15.46:0/3508892220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8600b1640 0x7fd8600ccfc0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:39.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.354+0000 7fd865200640 1 --2- 172.21.15.46:0/3508892220 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd86015a5d0 0x7fd8600b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:39.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.354+0000 7fd865200640 1 -- 172.21.15.46:0/3508892220 >> 172.21.15.46:0/3508892220 conn(0x7fd860099a10 msgr2=0x7fd860089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:39.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.354+0000 7fd865200640 1 -- 172.21.15.46:0/3508892220 shutdown_connections 2024-03-29T08:22:39.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.354+0000 7fd865200640 1 -- 172.21.15.46:0/3508892220 wait complete. 2024-03-29T08:22:39.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd865200640 1 Processor -- start 2024-03-29T08:22:39.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd865200640 1 -- start start 2024-03-29T08:22:39.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd865200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8600b1640 0x7fd86015d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:39.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd865200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8600cd500 0x7fd86015fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:39.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd865200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd86015a5d0 0x7fd86006e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:39.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd865200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8601600c0 con 0x7fd8600b1640 2024-03-29T08:22:39.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd865200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8600aed70 con 0x7fd86015a5d0 2024-03-29T08:22:39.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd865200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd860153930 con 0x7fd8600cd500 2024-03-29T08:22:39.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd85de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8600b1640 0x7fd86015d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:39.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd85de00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8600b1640 0x7fd86015d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60376/0 (socket says 172.21.15.46:60376) 2024-03-29T08:22:39.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd85de00640 1 -- 172.21.15.46:0/1474810960 learned_addr learned my addr 172.21.15.46:0/1474810960 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:39.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd855400640 1 --2- 172.21.15.46:0/1474810960 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8600cd500 0x7fd86015fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:39.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd85e800640 1 --2- 172.21.15.46:0/1474810960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd86015a5d0 0x7fd86006e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:39.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd85de00640 1 -- 172.21.15.46:0/1474810960 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8600cd500 msgr2=0x7fd86015fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:39.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd85de00640 1 --2- 172.21.15.46:0/1474810960 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8600cd500 0x7fd86015fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:39.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd85de00640 1 -- 172.21.15.46:0/1474810960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd86015a5d0 msgr2=0x7fd86006e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:39.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd85de00640 1 --2- 172.21.15.46:0/1474810960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd86015a5d0 0x7fd86006e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:39.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd85de00640 1 -- 172.21.15.46:0/1474810960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8480094d0 con 0x7fd8600b1640 2024-03-29T08:22:39.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd85e800640 1 --2- 172.21.15.46:0/1474810960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd86015a5d0 0x7fd86006e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:39.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd85de00640 1 --2- 172.21.15.46:0/1474810960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8600b1640 0x7fd86015d160 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd85000e870 tx=0x7fd85000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:39.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd856a00640 1 -- 172.21.15.46:0/1474810960 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850009510 con 0x7fd8600b1640 2024-03-29T08:22:39.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd856a00640 1 -- 172.21.15.46:0/1474810960 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85000eec0 con 0x7fd8600b1640 2024-03-29T08:22:39.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd865200640 1 -- 172.21.15.46:0/1474810960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd860153370 con 0x7fd8600b1640 2024-03-29T08:22:39.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd856a00640 1 -- 172.21.15.46:0/1474810960 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000cae0 con 0x7fd8600b1640 2024-03-29T08:22:39.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.358+0000 7fd865200640 1 -- 172.21.15.46:0/1474810960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8600aefa0 con 0x7fd8600b1640 2024-03-29T08:22:39.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.362+0000 7fd865200640 1 -- 172.21.15.46:0/1474810960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86009e7d0 con 0x7fd8600b1640 2024-03-29T08:22:39.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.362+0000 7fd856a00640 1 -- 172.21.15.46:0/1474810960 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd85000cc40 con 0x7fd8600b1640 2024-03-29T08:22:39.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.362+0000 7fd856a00640 1 --2- 172.21.15.46:0/1474810960 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd83c0666c0 0x7fd83c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:39.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.362+0000 7fd856a00640 1 -- 172.21.15.46:0/1474810960 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd850083e80 con 0x7fd8600b1640 2024-03-29T08:22:39.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.362+0000 7fd855400640 1 --2- 172.21.15.46:0/1474810960 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd83c0666c0 0x7fd83c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:39.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.362+0000 7fd855400640 1 --2- 172.21.15.46:0/1474810960 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd83c0666c0 0x7fd83c068b80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd8480ae000 tx=0x7fd8480b5040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:39.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.366+0000 7fd856a00640 1 -- 172.21.15.46:0/1474810960 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd850051590 con 0x7fd8600b1640 2024-03-29T08:22:39.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:39.538+0000 7fd865200640 1 -- 172.21.15.46:0/1474810960 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd86004e120 con 0x7fd8600b1640 2024-03-29T08:22:40.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.318+0000 7fd856a00640 1 -- 172.21.15.46:0/1474810960 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v73) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd850054bb0 con 0x7fd8600b1640 2024-03-29T08:22:40.323 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-03-29T08:22:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.322+0000 7fd865200640 1 -- 172.21.15.46:0/1474810960 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd83c0666c0 msgr2=0x7fd83c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.322+0000 7fd865200640 1 --2- 172.21.15.46:0/1474810960 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd83c0666c0 0x7fd83c068b80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd8480ae000 tx=0x7fd8480b5040 comp rx=0 tx=0).stop 2024-03-29T08:22:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.322+0000 7fd865200640 1 -- 172.21.15.46:0/1474810960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8600b1640 msgr2=0x7fd86015d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.322+0000 7fd865200640 1 --2- 172.21.15.46:0/1474810960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8600b1640 0x7fd86015d160 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fd85000e870 tx=0x7fd85000ec70 comp rx=0 tx=0).stop 2024-03-29T08:22:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.322+0000 7fd865200640 1 -- 172.21.15.46:0/1474810960 shutdown_connections 2024-03-29T08:22:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.322+0000 7fd865200640 1 --2- 172.21.15.46:0/1474810960 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd83c0666c0 0x7fd83c068b80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.322+0000 7fd865200640 1 --2- 172.21.15.46:0/1474810960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd86015a5d0 0x7fd86006e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.322+0000 7fd865200640 1 --2- 172.21.15.46:0/1474810960 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8600cd500 0x7fd86015fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:40.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.322+0000 7fd865200640 1 --2- 172.21.15.46:0/1474810960 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8600b1640 0x7fd86015d160 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.322+0000 7fd865200640 1 -- 172.21.15.46:0/1474810960 >> 172.21.15.46:0/1474810960 conn(0x7fd860099a10 msgr2=0x7fd86008a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.322+0000 7fd865200640 1 -- 172.21.15.46:0/1474810960 shutdown_connections 2024-03-29T08:22:40.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.322+0000 7fd865200640 1 -- 172.21.15.46:0/1474810960 wait complete. 2024-03-29T08:22:40.348 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-29T08:22:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.486+0000 7f251f400640 1 Processor -- start 2024-03-29T08:22:40.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.486+0000 7f251f400640 1 -- start start 2024-03-29T08:22:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.490+0000 7f251f400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f251815dc00 0x7f25180cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.490+0000 7f251f400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251815cfb0 con 0x7f25180d0660 2024-03-29T08:22:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.490+0000 7f251f400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251806be20 con 0x7f251806b8b0 2024-03-29T08:22:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.490+0000 7f251f400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251806c100 con 0x7f251815dc00 2024-03-29T08:22:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.490+0000 7f251ca00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f25180d0660 0x7f25180d0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38102/0 (socket says 172.21.15.46:38102) 2024-03-29T08:22:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.490+0000 7f251ca00640 1 -- 172.21.15.46:0/1622910378 learned_addr learned my addr 172.21.15.46:0/1622910378 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.490+0000 7f251d400640 1 --2- 172.21.15.46:0/1622910378 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f251815dc00 0x7f25180cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:40.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.490+0000 7f2517400640 1 -- 172.21.15.46:0/1622910378 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3233442543 0 0) 0x7f251815cfb0 con 0x7f25180d0660 2024-03-29T08:22:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.490+0000 7f2517400640 1 -- 172.21.15.46:0/1622910378 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2500003530 con 0x7f25180d0660 2024-03-29T08:22:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.490+0000 7f2517400640 1 -- 172.21.15.46:0/1622910378 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425241154 0 0) 0x7f251806be20 con 0x7f251806b8b0 2024-03-29T08:22:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.490+0000 7f2517400640 1 -- 172.21.15.46:0/1622910378 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f251815cfb0 con 0x7f251806b8b0 2024-03-29T08:22:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.490+0000 7f2517400640 1 -- 172.21.15.46:0/1622910378 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3983122139 0 0) 0x7f2500003530 con 0x7f25180d0660 2024-03-29T08:22:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.490+0000 7f251d400640 1 -- 172.21.15.46:0/1622910378 >> v1:172.21.15.73:6789/0 conn(0x7f251806b8b0 legacy=0x7f25180cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.490+0000 7f251d400640 1 -- 172.21.15.46:0/1622910378 >> v1:172.21.15.46:6789/0 conn(0x7f25180d0660 legacy=0x7f25180d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:40.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.490+0000 7f251d400640 1 -- 172.21.15.46:0/1622910378 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25180cecf0 con 0x7f251815dc00 2024-03-29T08:22:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.494+0000 7f251d400640 1 --2- 172.21.15.46:0/1622910378 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f251815dc00 0x7f25180cf7d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2510009dc0 tx=0x7f251001c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ce567c85e70a6e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.494+0000 7f2517400640 1 -- 172.21.15.46:0/1622910378 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c001680 con 0x7f251815dc00 2024-03-29T08:22:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.494+0000 7f2517400640 1 -- 172.21.15.46:0/1622910378 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25100a55c0 con 0x7f251815dc00 2024-03-29T08:22:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.494+0000 7f2517400640 1 -- 172.21.15.46:0/1622910378 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25100a5950 con 0x7f251815dc00 2024-03-29T08:22:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.494+0000 7f251f400640 1 -- 172.21.15.46:0/1622910378 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f251815dc00 msgr2=0x7f25180cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.494+0000 7f251f400640 1 --2- 172.21.15.46:0/1622910378 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f251815dc00 0x7f25180cf7d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2510009dc0 tx=0x7f251001c850 comp rx=0 tx=0).stop 2024-03-29T08:22:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.494+0000 7f251f400640 1 -- 172.21.15.46:0/1622910378 shutdown_connections 2024-03-29T08:22:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.494+0000 7f251f400640 1 --2- 172.21.15.46:0/1622910378 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f251815dc00 0x7f25180cf7d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.494+0000 7f251f400640 1 -- 172.21.15.46:0/1622910378 >> 172.21.15.46:0/1622910378 conn(0x7f2518089fd0 msgr2=0x7f251808a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.494+0000 7f251f400640 1 -- 172.21.15.46:0/1622910378 shutdown_connections 2024-03-29T08:22:40.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.494+0000 7f251f400640 1 -- 172.21.15.46:0/1622910378 wait complete. 2024-03-29T08:22:40.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.494+0000 7f251f400640 1 Processor -- start 2024-03-29T08:22:40.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.494+0000 7f251f400640 1 -- start start 2024-03-29T08:22:40.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f251f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f251806b8b0 0x7f25180c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:40.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f251f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25180d0660 0x7f25181518b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:40.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f251f400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f251815dc00 0x7f2518157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:40.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f251f400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2518158ce0 con 0x7f25180d0660 2024-03-29T08:22:40.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f251f400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25180c8900 con 0x7f251806b8b0 2024-03-29T08:22:40.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f251f400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25180ca290 con 0x7f251815dc00 2024-03-29T08:22:40.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f2517e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25180d0660 0x7f25181518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:40.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f251ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f251806b8b0 0x7f25180c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:40.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f2517e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25180d0660 0x7f25181518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60378/0 (socket says 172.21.15.46:60378) 2024-03-29T08:22:40.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f2517e00640 1 -- 172.21.15.46:0/3412490723 learned_addr learned my addr 172.21.15.46:0/3412490723 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:40.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f251d400640 1 --2- 172.21.15.46:0/3412490723 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f251815dc00 0x7f2518157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:40.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f251ca00640 1 -- 172.21.15.46:0/3412490723 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f251815dc00 msgr2=0x7f2518157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:40.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f251ca00640 1 --2- 172.21.15.46:0/3412490723 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f251815dc00 0x7f2518157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:40.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f251ca00640 1 -- 172.21.15.46:0/3412490723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25180d0660 msgr2=0x7f25181518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:40.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f251ca00640 1 --2- 172.21.15.46:0/3412490723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25180d0660 0x7f25181518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:40.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f251ca00640 1 -- 172.21.15.46:0/3412490723 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2510009a70 con 0x7f251806b8b0 2024-03-29T08:22:40.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f2517e00640 1 --2- 172.21.15.46:0/3412490723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25180d0660 0x7f25181518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:22:40.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f251ca00640 1 --2- 172.21.15.46:0/3412490723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f251806b8b0 0x7f25180c83c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f250c00c580 tx=0x7f250c00ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:40.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f2514c00640 1 -- 172.21.15.46:0/3412490723 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c005cf0 con 0x7f251806b8b0 2024-03-29T08:22:40.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f2514c00640 1 -- 172.21.15.46:0/3412490723 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250c005e50 con 0x7f251806b8b0 2024-03-29T08:22:40.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f251f400640 1 -- 172.21.15.46:0/3412490723 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2518158f80 con 0x7f251806b8b0 2024-03-29T08:22:40.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f2514c00640 1 -- 172.21.15.46:0/3412490723 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c007330 con 0x7f251806b8b0 2024-03-29T08:22:40.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f251f400640 1 -- 172.21.15.46:0/3412490723 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25180ca570 con 0x7f251806b8b0 2024-03-29T08:22:40.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f251f400640 1 -- 172.21.15.46:0/3412490723 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2518066c90 con 0x7f251806b8b0 2024-03-29T08:22:40.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.498+0000 7f2514c00640 1 -- 172.21.15.46:0/3412490723 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f250c007490 con 0x7f251806b8b0 2024-03-29T08:22:40.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.502+0000 7f2514c00640 1 --2- 172.21.15.46:0/3412490723 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f24f80666c0 0x7f24f8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:40.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.502+0000 7f2514c00640 1 -- 172.21.15.46:0/3412490723 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f250c0834f0 con 0x7f251806b8b0 2024-03-29T08:22:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.502+0000 7f2517e00640 1 --2- 172.21.15.46:0/3412490723 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f24f80666c0 0x7f24f8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:40.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.502+0000 7f2517e00640 1 --2- 172.21.15.46:0/3412490723 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f24f80666c0 0x7f24f8068b80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f251815bc30 tx=0x7f2504009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:40.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.506+0000 7f2514c00640 1 -- 172.21.15.46:0/3412490723 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f250c050c50 con 0x7f251806b8b0 2024-03-29T08:22:40.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.722+0000 7f251f400640 1 -- 172.21.15.46:0/3412490723 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f251815d520 con 0x7f251806b8b0 2024-03-29T08:22:40.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.726+0000 7f2514c00640 1 -- 172.21.15.46:0/3412490723 <== mon.1 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f250c017090 con 0x7f251806b8b0 2024-03-29T08:22:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:40.728 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:22:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-03-29T08:22:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-03-29T08:22:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmlnXNJBAA7lI2lCBd0PeZudPsQyMOUQ==", 2024-03-29T08:22:40.728 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:22:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:22:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:22:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:22:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-03-29T08:22:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmbPkcMRAADnwJE17uILjJo14jghJuzg==", 2024-03-29T08:22:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:22:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:22:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:22:40.729 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:22:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-03-29T08:22:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmkCUVLBAAxcAOWGBDUCpEHeg/1OeQzA==", 2024-03-29T08:22:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:22:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:22:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:22:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:22:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-03-29T08:22:40.730 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZm7L11OBAAZ3fpXtEIYG1tGm9ZFswZ5Q==", 2024-03-29T08:22:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:22:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:22:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:22:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:22:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-03-29T08:22:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAIeQZmTHV8HhAAODJ+RP1Bx6k0D1153XgI0Q==", 2024-03-29T08:22:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:22:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:22:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:22:40.731 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-03-29T08:22:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQALeQZmZRD0JBAAjU6W/21CeqOMdIP5969iNA==", 2024-03-29T08:22:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:22:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:22:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:22:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-03-29T08:22:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAOeQZmSSP+KhAA26V8CByUJNfkVcG37AdlZQ==", 2024-03-29T08:22:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.732 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:22:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:22:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:22:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-03-29T08:22:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAReQZmb/e8LxAAV4yC1xlLwh8OAqOSPmcyyA==", 2024-03-29T08:22:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:22:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:22:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:22:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.733 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-03-29T08:22:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAXeQZmrLnzNRAAFPYQRs9sBNNKeyvcoI9rSg==", 2024-03-29T08:22:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:22:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:22:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:22:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-03-29T08:22:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAbeQZmjTPbABAAi4qTU5CBy9LtSSRf7cmfwg==", 2024-03-29T08:22:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:22:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:22:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:22:40.734 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-03-29T08:22:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAeeQZmo13eBRAAWY2VVBakLze9oA4ynp1rNA==", 2024-03-29T08:22:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:22:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:22:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:22:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-03-29T08:22:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAheQZmghpZCxAAdy5xI7qPwZb8LN8Rn6Ndaw==", 2024-03-29T08:22:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.735 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:22:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:22:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:22:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-03-29T08:22:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmyk22ABAA8Pn3OaIvxMQSLrKIM8FP4w==", 2024-03-29T08:22:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:22:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-03-29T08:22:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-29T08:22:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.736 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-03-29T08:22:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmG/E8ChAAJTcJ2j5xn0oUsmlV2trWdw==", 2024-03-29T08:22:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:22:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:22:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:22:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:22:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-03-29T08:22:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmhsqgBhAApqnO5qMG8VP27j9TfbZ8uw==", 2024-03-29T08:22:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:22:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:22:40.737 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:22:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:22:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-03-29T08:22:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmatlQEBAAAGKuGnAh4YXSk91TAzomXA==", 2024-03-29T08:22:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:22:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:22:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:22:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:22:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.738 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-03-29T08:22:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmp2B3CBAA4cfe/hUJeOg0SIr1KbWyUg==", 2024-03-29T08:22:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:22:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:22:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:22:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:22:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-03-29T08:22:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmyrJXIBAAv535Rz99UI3V7JvvzQ8oSw==", 2024-03-29T08:22:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:22:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-03-29T08:22:40.739 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:22:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-03-29T08:22:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZm72tiHBAA2N7qQnR5Ehu/7nP2WpklGg==", 2024-03-29T08:22:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:22:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:22:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-03-29T08:22:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:22:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:40.740 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-29T08:22:40.740 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:22:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.726+0000 7f251f400640 1 -- 172.21.15.46:0/3412490723 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f24f80666c0 msgr2=0x7f24f8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.726+0000 7f251f400640 1 --2- 172.21.15.46:0/3412490723 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f24f80666c0 0x7f24f8068b80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f251815bc30 tx=0x7f2504009040 comp rx=0 tx=0).stop 2024-03-29T08:22:40.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.726+0000 7f251f400640 1 -- 172.21.15.46:0/3412490723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f251806b8b0 msgr2=0x7f25180c83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.726+0000 7f251f400640 1 --2- 172.21.15.46:0/3412490723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f251806b8b0 0x7f25180c83c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f250c00c580 tx=0x7f250c00ca40 comp rx=0 tx=0).stop 2024-03-29T08:22:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.726+0000 7f251f400640 1 -- 172.21.15.46:0/3412490723 shutdown_connections 2024-03-29T08:22:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.726+0000 7f251f400640 1 --2- 172.21.15.46:0/3412490723 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f24f80666c0 0x7f24f8068b80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.726+0000 7f251f400640 1 --2- 172.21.15.46:0/3412490723 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f251815dc00 0x7f2518157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.726+0000 7f251f400640 1 --2- 172.21.15.46:0/3412490723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25180d0660 0x7f25181518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.726+0000 7f251f400640 1 --2- 172.21.15.46:0/3412490723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f251806b8b0 0x7f25180c83c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.726+0000 7f251f400640 1 -- 172.21.15.46:0/3412490723 >> 172.21.15.46:0/3412490723 conn(0x7f2518089fd0 msgr2=0x7f25180d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.726+0000 7f251f400640 1 -- 172.21.15.46:0/3412490723 shutdown_connections 2024-03-29T08:22:40.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.726+0000 7f251f400640 1 -- 172.21.15.46:0/3412490723 wait complete. 2024-03-29T08:22:40.749 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:22:40.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.886+0000 7f6aea200640 1 Processor -- start 2024-03-29T08:22:40.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.886+0000 7f6aea200640 1 -- start start 2024-03-29T08:22:40.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6aea200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ae40ce8a0 0x7f6ae40caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:40.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6aea200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ae40ae810 0x7f6ae40cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:40.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6aea200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ae40c9bd0 con 0x7f6ae40cec80 2024-03-29T08:22:40.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6aea200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ae40cbcc0 con 0x7f6ae40ce8a0 2024-03-29T08:22:40.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6aea200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40c9eb0 con 0x7f6ae40ae810 2024-03-29T08:22:40.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6ae3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ae40ce8a0 0x7f6ae40caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:40.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6ae3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ae40ce8a0 0x7f6ae40caf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60384/0 (socket says 172.21.15.46:60384) 2024-03-29T08:22:40.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6ae3400640 1 -- 172.21.15.46:0/65440161 learned_addr learned my addr 172.21.15.46:0/65440161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:40.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6adbe00640 1 --2- 172.21.15.46:0/65440161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ae40ae810 0x7f6ae40cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:40.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6ae3400640 1 -- 172.21.15.46:0/65440161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ae40ae810 msgr2=0x7f6ae40cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:40.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6ae3400640 1 --2- 172.21.15.46:0/65440161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ae40ae810 0x7f6ae40cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:40.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6ae3400640 1 -- 172.21.15.46:0/65440161 >> v1:172.21.15.46:6789/0 conn(0x7f6ae40cec80 legacy=0x7f6ae40aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:40.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6ae3400640 1 -- 172.21.15.46:0/65440161 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae40ca190 con 0x7f6ae40ce8a0 2024-03-29T08:22:40.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6adbe00640 1 --2- 172.21.15.46:0/65440161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ae40ae810 0x7f6ae40cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:40.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6ae3400640 1 --2- 172.21.15.46:0/65440161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae40ce8a0 0x7f6ae40caf50 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009870 tx=0x7f6ad401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14bd88a1fbb4bbc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:40.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6ae2a00640 1 -- 172.21.15.46:0/65440161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad40a95f0 con 0x7f6ae40ce8a0 2024-03-29T08:22:40.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6ae2a00640 1 -- 172.21.15.46:0/65440161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad40a9750 con 0x7f6ae40ce8a0 2024-03-29T08:22:40.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.890+0000 7f6ae2a00640 1 -- 172.21.15.46:0/65440161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad40a9ae0 con 0x7f6ae40ce8a0 2024-03-29T08:22:40.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6aea200640 1 -- 172.21.15.46:0/65440161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae40ce8a0 msgr2=0x7f6ae40caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:40.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6aea200640 1 --2- 172.21.15.46:0/65440161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae40ce8a0 0x7f6ae40caf50 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4009870 tx=0x7f6ad401c530 comp rx=0 tx=0).stop 2024-03-29T08:22:40.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6aea200640 1 -- 172.21.15.46:0/65440161 shutdown_connections 2024-03-29T08:22:40.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6aea200640 1 --2- 172.21.15.46:0/65440161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ae40ae810 0x7f6ae40cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:40.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6aea200640 1 --2- 172.21.15.46:0/65440161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae40ce8a0 0x7f6ae40caf50 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:40.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6aea200640 1 -- 172.21.15.46:0/65440161 >> 172.21.15.46:0/65440161 conn(0x7f6ae4098a40 msgr2=0x7f6ae40cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:40.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6aea200640 1 -- 172.21.15.46:0/65440161 shutdown_connections 2024-03-29T08:22:40.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6aea200640 1 -- 172.21.15.46:0/65440161 wait complete. 2024-03-29T08:22:40.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6aea200640 1 Processor -- start 2024-03-29T08:22:40.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6aea200640 1 -- start start 2024-03-29T08:22:40.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6aea200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae40ae810 0x7f6ae40c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:40.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6aea200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ae40ce8a0 0x7f6ae40c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:40.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6aea200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ae40cec80 0x7f6ae40cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:40.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6aea200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40b2bb0 con 0x7f6ae40ae810 2024-03-29T08:22:40.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6aea200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40c49b0 con 0x7f6ae40cec80 2024-03-29T08:22:40.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6aea200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae40c4c90 con 0x7f6ae40ce8a0 2024-03-29T08:22:40.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6ae3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae40ae810 0x7f6ae40c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:40.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6ae3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae40ae810 0x7f6ae40c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60394/0 (socket says 172.21.15.46:60394) 2024-03-29T08:22:40.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6ae3400640 1 -- 172.21.15.46:0/817861907 learned_addr learned my addr 172.21.15.46:0/817861907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:40.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6adbe00640 1 --2- 172.21.15.46:0/817861907 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ae40ce8a0 0x7f6ae40c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:40.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6ae3e00640 1 --2- 172.21.15.46:0/817861907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ae40cec80 0x7f6ae40cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:40.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6ae3400640 1 -- 172.21.15.46:0/817861907 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ae40ce8a0 msgr2=0x7f6ae40c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:40.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6ae3400640 1 --2- 172.21.15.46:0/817861907 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ae40ce8a0 0x7f6ae40c7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:40.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6ae3400640 1 -- 172.21.15.46:0/817861907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ae40cec80 msgr2=0x7f6ae40cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:40.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6ae3400640 1 --2- 172.21.15.46:0/817861907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ae40cec80 0x7f6ae40cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:40.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6ae3400640 1 -- 172.21.15.46:0/817861907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad40094d0 con 0x7f6ae40ae810 2024-03-29T08:22:40.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.894+0000 7f6ae3e00640 1 --2- 172.21.15.46:0/817861907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ae40cec80 0x7f6ae40cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:22:40.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.898+0000 7f6ae3400640 1 --2- 172.21.15.46:0/817861907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae40ae810 0x7f6ae40c42d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40ab040 tx=0x7f6ad401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:40.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.898+0000 7f6adb400640 1 -- 172.21.15.46:0/817861907 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad40b1800 con 0x7f6ae40ae810 2024-03-29T08:22:40.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.898+0000 7f6adb400640 1 -- 172.21.15.46:0/817861907 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad40b1980 con 0x7f6ae40ae810 2024-03-29T08:22:40.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.898+0000 7f6adb400640 1 -- 172.21.15.46:0/817861907 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad40b1d30 con 0x7f6ae40ae810 2024-03-29T08:22:40.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.898+0000 7f6aea200640 1 -- 172.21.15.46:0/817861907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae40c4f70 con 0x7f6ae40ae810 2024-03-29T08:22:40.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.898+0000 7f6aea200640 1 -- 172.21.15.46:0/817861907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae40cb6f0 con 0x7f6ae40ae810 2024-03-29T08:22:40.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.898+0000 7f6aea200640 1 -- 172.21.15.46:0/817861907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae40c4dd0 con 0x7f6ae40ae810 2024-03-29T08:22:40.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.898+0000 7f6adb400640 1 -- 172.21.15.46:0/817861907 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6ad40bc450 con 0x7f6ae40ae810 2024-03-29T08:22:40.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.898+0000 7f6adb400640 1 --2- 172.21.15.46:0/817861907 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ac0066690 0x7f6ac0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:40.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.898+0000 7f6adb400640 1 -- 172.21.15.46:0/817861907 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6ad411e870 con 0x7f6ae40ae810 2024-03-29T08:22:40.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.898+0000 7f6adbe00640 1 --2- 172.21.15.46:0/817861907 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ac0066690 0x7f6ac0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:40.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.902+0000 7f6adbe00640 1 --2- 172.21.15.46:0/817861907 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ac0066690 0x7f6ac0068b50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40c9650 tx=0x7f6acc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:40.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:40.902+0000 7f6adb400640 1 -- 172.21.15.46:0/817861907 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6ae40c4dd0 con 0x7f6ae40ae810 2024-03-29T08:22:41.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.066+0000 7f6aea200640 1 -- 172.21.15.46:0/817861907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ae40c4dd0 con 0x7f6ae40ae810 2024-03-29T08:22:41.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.066+0000 7f6adb400640 1 -- 172.21.15.46:0/817861907 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+12341 (secure 0 0 0) 0x7f6ae40c4dd0 con 0x7f6ae40ae810 2024-03-29T08:22:41.069 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:41.069 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":73,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:22:40.318824+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:22:41.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.066+0000 7f6aea200640 1 -- 172.21.15.46:0/817861907 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ac0066690 msgr2=0x7f6ac0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.066+0000 7f6aea200640 1 --2- 172.21.15.46:0/817861907 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ac0066690 0x7f6ac0068b50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6ae40c9650 tx=0x7f6acc008040 comp rx=0 tx=0).stop 2024-03-29T08:22:41.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.066+0000 7f6aea200640 1 -- 172.21.15.46:0/817861907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae40ae810 msgr2=0x7f6ae40c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.066+0000 7f6aea200640 1 --2- 172.21.15.46:0/817861907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae40ae810 0x7f6ae40c42d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40ab040 tx=0x7f6ad401ad70 comp rx=0 tx=0).stop 2024-03-29T08:22:41.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.066+0000 7f6aea200640 1 -- 172.21.15.46:0/817861907 shutdown_connections 2024-03-29T08:22:41.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.066+0000 7f6aea200640 1 --2- 172.21.15.46:0/817861907 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6ac0066690 0x7f6ac0068b50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.066+0000 7f6aea200640 1 --2- 172.21.15.46:0/817861907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ae40cec80 0x7f6ae40cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.066+0000 7f6aea200640 1 --2- 172.21.15.46:0/817861907 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ae40ce8a0 0x7f6ae40c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.066+0000 7f6aea200640 1 --2- 172.21.15.46:0/817861907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ae40ae810 0x7f6ae40c42d0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.066+0000 7f6aea200640 1 -- 172.21.15.46:0/817861907 >> 172.21.15.46:0/817861907 conn(0x7f6ae4098a40 msgr2=0x7f6ae4072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:41.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.066+0000 7f6aea200640 1 -- 172.21.15.46:0/817861907 shutdown_connections 2024-03-29T08:22:41.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.066+0000 7f6aea200640 1 -- 172.21.15.46:0/817861907 wait complete. 2024-03-29T08:22:41.089 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:22:41.090 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:22:41.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.234+0000 7f030e000640 1 Processor -- start 2024-03-29T08:22:41.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.234+0000 7f030e000640 1 -- start start 2024-03-29T08:22:41.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.234+0000 7f030e000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f030806b4b0 0x7f03080b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:41.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.234+0000 7f030e000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f03080b3f20 0x7f03080b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:41.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.234+0000 7f030e000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0308157490 con 0x7f030815dbb0 2024-03-29T08:22:41.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.234+0000 7f030e000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03080b27b0 con 0x7f03080b3f20 2024-03-29T08:22:41.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.234+0000 7f030e000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03080b2c90 con 0x7f030806b4b0 2024-03-29T08:22:41.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.234+0000 7f02fea00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f030806b4b0 0x7f03080b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:41.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.234+0000 7f0307e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f03080b3f20 0x7f03080b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:41.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.234+0000 7f02fea00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f030806b4b0 0x7f03080b1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:47408/0 (socket says 172.21.15.46:47408) 2024-03-29T08:22:41.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.234+0000 7f02fea00640 1 -- 172.21.15.46:0/1214799370 learned_addr learned my addr 172.21.15.46:0/1214799370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:41.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.234+0000 7f0307400640 1 --1- 172.21.15.46:0/1214799370 >> v1:172.21.15.73:6789/0 conn(0x7f030815dbb0 0x7f030806ada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:52520/0 (socket says 172.21.15.46:52520) 2024-03-29T08:22:41.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.234+0000 7f0307e00640 1 -- 172.21.15.46:0/1214799370 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f030806b4b0 msgr2=0x7f03080b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:41.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.234+0000 7f0307e00640 1 --2- 172.21.15.46:0/1214799370 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f030806b4b0 0x7f03080b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.234+0000 7f0307e00640 1 -- 172.21.15.46:0/1214799370 >> v1:172.21.15.73:6789/0 conn(0x7f030815dbb0 legacy=0x7f030806ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.234+0000 7f0307e00640 1 -- 172.21.15.46:0/1214799370 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03080b29b0 con 0x7f03080b3f20 2024-03-29T08:22:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.238+0000 7f0307e00640 1 --2- 172.21.15.46:0/1214799370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03080b3f20 0x7f03080b2270 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0300009d90 tx=0x7f030001c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f23f0396452454f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.238+0000 7f0306a00640 1 -- 172.21.15.46:0/1214799370 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03000a59a0 con 0x7f03080b3f20 2024-03-29T08:22:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.238+0000 7f0306a00640 1 -- 172.21.15.46:0/1214799370 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03000a5b00 con 0x7f03080b3f20 2024-03-29T08:22:41.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.238+0000 7f0306a00640 1 -- 172.21.15.46:0/1214799370 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03000a5e90 con 0x7f03080b3f20 2024-03-29T08:22:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.238+0000 7f030e000640 1 -- 172.21.15.46:0/1214799370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03080b3f20 msgr2=0x7f03080b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.238+0000 7f030e000640 1 --2- 172.21.15.46:0/1214799370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03080b3f20 0x7f03080b2270 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0300009d90 tx=0x7f030001c8e0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.238+0000 7f030e000640 1 -- 172.21.15.46:0/1214799370 shutdown_connections 2024-03-29T08:22:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.238+0000 7f030e000640 1 --2- 172.21.15.46:0/1214799370 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f030806b4b0 0x7f03080b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.238+0000 7f030e000640 1 --2- 172.21.15.46:0/1214799370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03080b3f20 0x7f03080b2270 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.238+0000 7f030e000640 1 -- 172.21.15.46:0/1214799370 >> 172.21.15.46:0/1214799370 conn(0x7f0308070ad0 msgr2=0x7f0308070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.238+0000 7f030e000640 1 -- 172.21.15.46:0/1214799370 shutdown_connections 2024-03-29T08:22:41.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.238+0000 7f030e000640 1 -- 172.21.15.46:0/1214799370 wait complete. 2024-03-29T08:22:41.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.238+0000 7f030e000640 1 Processor -- start 2024-03-29T08:22:41.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f030e000640 1 -- start start 2024-03-29T08:22:41.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f030e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030806b4b0 0x7f03080ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:41.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f030e000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03080b3f20 0x7f0308152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:41.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f030e000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f030815dbb0 0x7f0308155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:41.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f030e000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0308151c90 con 0x7f030806b4b0 2024-03-29T08:22:41.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f030e000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03080b04f0 con 0x7f03080b3f20 2024-03-29T08:22:41.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f030e000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0308151640 con 0x7f030815dbb0 2024-03-29T08:22:41.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f0307400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030806b4b0 0x7f03080ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f0307400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030806b4b0 0x7f03080ce1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60396/0 (socket says 172.21.15.46:60396) 2024-03-29T08:22:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f0307400640 1 -- 172.21.15.46:0/2227141443 learned_addr learned my addr 172.21.15.46:0/2227141443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f02fea00640 1 --2- 172.21.15.46:0/2227141443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03080b3f20 0x7f0308152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f0307e00640 1 --2- 172.21.15.46:0/2227141443 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f030815dbb0 0x7f0308155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f0307400640 1 -- 172.21.15.46:0/2227141443 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f030815dbb0 msgr2=0x7f0308155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f0307400640 1 --2- 172.21.15.46:0/2227141443 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f030815dbb0 0x7f0308155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f0307400640 1 -- 172.21.15.46:0/2227141443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03080b3f20 msgr2=0x7f0308152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f0307400640 1 --2- 172.21.15.46:0/2227141443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03080b3f20 0x7f0308152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f0307400640 1 -- 172.21.15.46:0/2227141443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0300009a70 con 0x7f030806b4b0 2024-03-29T08:22:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f0307400640 1 --2- 172.21.15.46:0/2227141443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030806b4b0 0x7f03080ce1d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f02f800c430 tx=0x7f02f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f02ffe00640 1 -- 172.21.15.46:0/2227141443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f80074e0 con 0x7f030806b4b0 2024-03-29T08:22:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f02ffe00640 1 -- 172.21.15.46:0/2227141443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f8007640 con 0x7f030806b4b0 2024-03-29T08:22:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f02ffe00640 1 -- 172.21.15.46:0/2227141443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8004a10 con 0x7f030806b4b0 2024-03-29T08:22:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f030e000640 1 -- 172.21.15.46:0/2227141443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0308151920 con 0x7f030806b4b0 2024-03-29T08:22:41.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f030e000640 1 -- 172.21.15.46:0/2227141443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0308152660 con 0x7f030806b4b0 2024-03-29T08:22:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.242+0000 7f030e000640 1 -- 172.21.15.46:0/2227141443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f030809d100 con 0x7f030806b4b0 2024-03-29T08:22:41.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.246+0000 7f02ffe00640 1 -- 172.21.15.46:0/2227141443 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f02f8004b70 con 0x7f030806b4b0 2024-03-29T08:22:41.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.246+0000 7f02ffe00640 1 --2- 172.21.15.46:0/2227141443 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f02e40666c0 0x7f02e4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:41.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.246+0000 7f02ffe00640 1 -- 172.21.15.46:0/2227141443 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f02f80836b0 con 0x7f030806b4b0 2024-03-29T08:22:41.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.246+0000 7f02fea00640 1 --2- 172.21.15.46:0/2227141443 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f02e40666c0 0x7f02e4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:41.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.246+0000 7f02fea00640 1 --2- 172.21.15.46:0/2227141443 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f02e40666c0 0x7f02e4068b80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f03081f8de0 tx=0x7f02f0009240 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:41.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.250+0000 7f02ffe00640 1 -- 172.21.15.46:0/2227141443 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f02f8050e10 con 0x7f030806b4b0 2024-03-29T08:22:41.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.406+0000 7f030e000640 1 -- 172.21.15.46:0/2227141443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f03081676e0 con 0x7f030806b4b0 2024-03-29T08:22:41.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.410+0000 7f02ffe00640 1 -- 172.21.15.46:0/2227141443 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f02f8054430 con 0x7f030806b4b0 2024-03-29T08:22:41.412 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:22:41.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.410+0000 7f030e000640 1 -- 172.21.15.46:0/2227141443 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f02e40666c0 msgr2=0x7f02e4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.410+0000 7f030e000640 1 --2- 172.21.15.46:0/2227141443 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f02e40666c0 0x7f02e4068b80 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f03081f8de0 tx=0x7f02f0009240 comp rx=0 tx=0).stop 2024-03-29T08:22:41.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.410+0000 7f030e000640 1 -- 172.21.15.46:0/2227141443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030806b4b0 msgr2=0x7f03080ce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.410+0000 7f030e000640 1 --2- 172.21.15.46:0/2227141443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030806b4b0 0x7f03080ce1d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f02f800c430 tx=0x7f02f800c8f0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.410+0000 7f030e000640 1 -- 172.21.15.46:0/2227141443 shutdown_connections 2024-03-29T08:22:41.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.410+0000 7f030e000640 1 --2- 172.21.15.46:0/2227141443 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f02e40666c0 0x7f02e4068b80 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.410+0000 7f030e000640 1 --2- 172.21.15.46:0/2227141443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03080b3f20 0x7f0308152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.410+0000 7f030e000640 1 --2- 172.21.15.46:0/2227141443 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f030815dbb0 0x7f0308155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.410+0000 7f030e000640 1 --2- 172.21.15.46:0/2227141443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f030806b4b0 0x7f03080ce1d0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.410+0000 7f030e000640 1 -- 172.21.15.46:0/2227141443 >> 172.21.15.46:0/2227141443 conn(0x7f0308070ad0 msgr2=0x7f0308160500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:41.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.410+0000 7f030e000640 1 -- 172.21.15.46:0/2227141443 shutdown_connections 2024-03-29T08:22:41.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.410+0000 7f030e000640 1 -- 172.21.15.46:0/2227141443 wait complete. 2024-03-29T08:22:41.433 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T08:22:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.570+0000 7fe933000640 1 Processor -- start 2024-03-29T08:22:41.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.570+0000 7fe933000640 1 -- start start 2024-03-29T08:22:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.570+0000 7fe933000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe92c144210 0x7fe92c0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.570+0000 7fe933000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92c0cb510 con 0x7fe92c0ceb20 2024-03-29T08:22:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.570+0000 7fe933000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92c070580 con 0x7fe92c15ada0 2024-03-29T08:22:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.570+0000 7fe933000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92c070860 con 0x7fe92c144210 2024-03-29T08:22:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.570+0000 7fe931000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe92c0ceb20 0x7fe92c06c950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38130/0 (socket says 172.21.15.46:38130) 2024-03-29T08:22:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.570+0000 7fe931000640 1 -- 172.21.15.46:0/1121174064 learned_addr learned my addr 172.21.15.46:0/1121174064 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.574+0000 7fe92be00640 1 --2- 172.21.15.46:0/1121174064 >> v2:172.21.15.73:3300/0 conn(0x7fe92c144210 0x7fe92c0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.574+0000 7fe92b400640 1 -- 172.21.15.46:0/1121174064 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2373932408 0 0) 0x7fe92c0cb510 con 0x7fe92c0ceb20 2024-03-29T08:22:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.574+0000 7fe92b400640 1 -- 172.21.15.46:0/1121174064 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe910003530 con 0x7fe92c0ceb20 2024-03-29T08:22:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.574+0000 7fe92be00640 1 -- 172.21.15.46:0/1121174064 >> v1:172.21.15.73:6789/0 conn(0x7fe92c15ada0 legacy=0x7fe92c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.574+0000 7fe92be00640 1 -- 172.21.15.46:0/1121174064 >> v1:172.21.15.46:6789/0 conn(0x7fe92c0ceb20 legacy=0x7fe92c06c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.574+0000 7fe92be00640 1 -- 172.21.15.46:0/1121174064 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c070b40 con 0x7fe92c144210 2024-03-29T08:22:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.574+0000 7fe92be00640 1 --2- 172.21.15.46:0/1121174064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92c144210 0x7fe92c0cb8b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0097c0 tx=0x7fe91c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acb842733746cc99 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.574+0000 7fe92b400640 1 -- 172.21.15.46:0/1121174064 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c0ac410 con 0x7fe92c144210 2024-03-29T08:22:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.574+0000 7fe92b400640 1 -- 172.21.15.46:0/1121174064 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91c0ac570 con 0x7fe92c144210 2024-03-29T08:22:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.574+0000 7fe92b400640 1 -- 172.21.15.46:0/1121174064 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c0ac900 con 0x7fe92c144210 2024-03-29T08:22:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe933000640 1 -- 172.21.15.46:0/1121174064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92c144210 msgr2=0x7fe92c0cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe933000640 1 --2- 172.21.15.46:0/1121174064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92c144210 0x7fe92c0cb8b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0097c0 tx=0x7fe91c01c4a0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe933000640 1 -- 172.21.15.46:0/1121174064 shutdown_connections 2024-03-29T08:22:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe933000640 1 --2- 172.21.15.46:0/1121174064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92c144210 0x7fe92c0cb8b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe933000640 1 -- 172.21.15.46:0/1121174064 >> 172.21.15.46:0/1121174064 conn(0x7fe92c066110 msgr2=0x7fe92c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe933000640 1 -- 172.21.15.46:0/1121174064 shutdown_connections 2024-03-29T08:22:41.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe933000640 1 -- 172.21.15.46:0/1121174064 wait complete. 2024-03-29T08:22:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe933000640 1 Processor -- start 2024-03-29T08:22:41.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe933000640 1 -- start start 2024-03-29T08:22:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe933000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe92c0ceb20 0x7fe92c159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe933000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92c144210 0x7fe92c13d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe933000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c15ada0 0x7fe92c0aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe933000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0ae6b0 con 0x7fe92c15ada0 2024-03-29T08:22:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe933000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c13e1f0 con 0x7fe92c144210 2024-03-29T08:22:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe933000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c0adde0 con 0x7fe92c0ceb20 2024-03-29T08:22:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe931000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c15ada0 0x7fe92c0aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe931000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c15ada0 0x7fe92c0aca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60398/0 (socket says 172.21.15.46:60398) 2024-03-29T08:22:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe931000640 1 -- 172.21.15.46:0/2811875545 learned_addr learned my addr 172.21.15.46:0/2811875545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe92be00640 1 --2- 172.21.15.46:0/2811875545 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe92c0ceb20 0x7fe92c159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:41.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe923400640 1 --2- 172.21.15.46:0/2811875545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92c144210 0x7fe92c13d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe931000640 1 -- 172.21.15.46:0/2811875545 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe92c0ceb20 msgr2=0x7fe92c159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe931000640 1 --2- 172.21.15.46:0/2811875545 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe92c0ceb20 0x7fe92c159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe931000640 1 -- 172.21.15.46:0/2811875545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92c144210 msgr2=0x7fe92c13d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe931000640 1 --2- 172.21.15.46:0/2811875545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92c144210 0x7fe92c13d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.578+0000 7fe931000640 1 -- 172.21.15.46:0/2811875545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe91c009420 con 0x7fe92c15ada0 2024-03-29T08:22:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.582+0000 7fe931000640 1 --2- 172.21.15.46:0/2811875545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c15ada0 0x7fe92c0aca30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe92400cbb0 tx=0x7fe92400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.582+0000 7fe928c00640 1 -- 172.21.15.46:0/2811875545 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92400ad40 con 0x7fe92c15ada0 2024-03-29T08:22:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.582+0000 7fe928c00640 1 -- 172.21.15.46:0/2811875545 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe924012400 con 0x7fe92c15ada0 2024-03-29T08:22:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.582+0000 7fe928c00640 1 -- 172.21.15.46:0/2811875545 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92400aea0 con 0x7fe92c15ada0 2024-03-29T08:22:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.582+0000 7fe933000640 1 -- 172.21.15.46:0/2811875545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92c15b180 con 0x7fe92c15ada0 2024-03-29T08:22:41.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.582+0000 7fe933000640 1 -- 172.21.15.46:0/2811875545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92c0adff0 con 0x7fe92c15ada0 2024-03-29T08:22:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.582+0000 7fe933000640 1 -- 172.21.15.46:0/2811875545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92c0ce280 con 0x7fe92c15ada0 2024-03-29T08:22:41.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.582+0000 7fe928c00640 1 -- 172.21.15.46:0/2811875545 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe924007070 con 0x7fe92c15ada0 2024-03-29T08:22:41.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.582+0000 7fe928c00640 1 --2- 172.21.15.46:0/2811875545 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe908066690 0x7fe908068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:41.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.582+0000 7fe928c00640 1 -- 172.21.15.46:0/2811875545 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe924083d20 con 0x7fe92c15ada0 2024-03-29T08:22:41.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.582+0000 7fe92be00640 1 --2- 172.21.15.46:0/2811875545 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe908066690 0x7fe908068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:41.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.586+0000 7fe92be00640 1 --2- 172.21.15.46:0/2811875545 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe908066690 0x7fe908068b50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0ae000 tx=0x7fe91c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:41.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.586+0000 7fe928c00640 1 -- 172.21.15.46:0/2811875545 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe9240513f0 con 0x7fe92c15ada0 2024-03-29T08:22:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.754+0000 7fe933000640 1 -- 172.21.15.46:0/2811875545 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe92c0ae7f0 con 0x7fe92c15ada0 2024-03-29T08:22:41.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.754+0000 7fe928c00640 1 -- 172.21.15.46:0/2811875545 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe92c0ae7f0 con 0x7fe92c15ada0 2024-03-29T08:22:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:41.757 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-03-29T08:22:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:22:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-03-29T08:22:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-03-29T08:22:41.757 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:41.757 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-03-29T08:22:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.754+0000 7fe933000640 1 -- 172.21.15.46:0/2811875545 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe908066690 msgr2=0x7fe908068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.754+0000 7fe933000640 1 --2- 172.21.15.46:0/2811875545 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe908066690 0x7fe908068b50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0ae000 tx=0x7fe91c008040 comp rx=0 tx=0).stop 2024-03-29T08:22:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.754+0000 7fe933000640 1 -- 172.21.15.46:0/2811875545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c15ada0 msgr2=0x7fe92c0aca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.754+0000 7fe933000640 1 --2- 172.21.15.46:0/2811875545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c15ada0 0x7fe92c0aca30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe92400cbb0 tx=0x7fe92400a430 comp rx=0 tx=0).stop 2024-03-29T08:22:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.754+0000 7fe933000640 1 -- 172.21.15.46:0/2811875545 shutdown_connections 2024-03-29T08:22:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.754+0000 7fe933000640 1 --2- 172.21.15.46:0/2811875545 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe908066690 0x7fe908068b50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.754+0000 7fe933000640 1 --2- 172.21.15.46:0/2811875545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe92c144210 0x7fe92c13d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.754+0000 7fe933000640 1 --2- 172.21.15.46:0/2811875545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe92c15ada0 0x7fe92c0aca30 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.754+0000 7fe933000640 1 --2- 172.21.15.46:0/2811875545 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe92c0ceb20 0x7fe92c159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.754+0000 7fe933000640 1 -- 172.21.15.46:0/2811875545 >> 172.21.15.46:0/2811875545 conn(0x7fe92c066110 msgr2=0x7fe92c1416b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:41.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.754+0000 7fe933000640 1 -- 172.21.15.46:0/2811875545 shutdown_connections 2024-03-29T08:22:41.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.754+0000 7fe933000640 1 -- 172.21.15.46:0/2811875545 wait complete. 2024-03-29T08:22:41.778 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T08:22:41.778 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T08:22:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.918+0000 7f9fe9200640 1 Processor -- start 2024-03-29T08:22:41.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.918+0000 7f9fe9200640 1 -- start start 2024-03-29T08:22:41.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.918+0000 7f9fe9200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9fe41598a0 0x7f9fe40cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.918+0000 7f9fe9200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9fe415dc30 0x7f9fe406c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.918+0000 7f9fe9200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe40b1bc0 con 0x7f9fe40b1470 2024-03-29T08:22:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.918+0000 7f9fe9200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe40b1ea0 con 0x7f9fe41598a0 2024-03-29T08:22:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.918+0000 7f9fe9200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe40b2180 con 0x7f9fe415dc30 2024-03-29T08:22:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.918+0000 7f9fe1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9fe41598a0 0x7f9fe40cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.918+0000 7f9fe1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9fe41598a0 0x7f9fe40cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60408/0 (socket says 172.21.15.46:60408) 2024-03-29T08:22:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.918+0000 7f9fdbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9fe40b1470 0x7f9fe415d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38144/0 (socket says 172.21.15.46:38144) 2024-03-29T08:22:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.918+0000 7f9fe1e00640 1 -- 172.21.15.46:0/4029543329 learned_addr learned my addr 172.21.15.46:0/4029543329 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.922+0000 7f9fe2800640 1 --2- 172.21.15.46:0/4029543329 >> v2:172.21.15.73:3300/0 conn(0x7f9fe415dc30 0x7f9fe406c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.922+0000 7f9fe1e00640 1 -- 172.21.15.46:0/4029543329 >> v2:172.21.15.73:3300/0 conn(0x7f9fe415dc30 msgr2=0x7f9fe406c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.922+0000 7f9fe1e00640 1 --2- 172.21.15.46:0/4029543329 >> v2:172.21.15.73:3300/0 conn(0x7f9fe415dc30 0x7f9fe406c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.922+0000 7f9fe1e00640 1 -- 172.21.15.46:0/4029543329 >> v1:172.21.15.46:6789/0 conn(0x7f9fe40b1470 legacy=0x7f9fe415d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.922+0000 7f9fe1e00640 1 -- 172.21.15.46:0/4029543329 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe40b2460 con 0x7f9fe41598a0 2024-03-29T08:22:41.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.922+0000 7f9fe2800640 1 --2- 172.21.15.46:0/4029543329 >> v2:172.21.15.73:3300/0 conn(0x7f9fe415dc30 0x7f9fe406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:41.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.922+0000 7f9fe1e00640 1 --2- 172.21.15.46:0/4029543329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe41598a0 0x7f9fe40cadb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40b2b70 tx=0x7f9fd401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b5ce182f083ce server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.922+0000 7f9fe1400640 1 -- 172.21.15.46:0/4029543329 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40ab680 con 0x7f9fe41598a0 2024-03-29T08:22:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.922+0000 7f9fe1400640 1 -- 172.21.15.46:0/4029543329 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd40ab7e0 con 0x7f9fe41598a0 2024-03-29T08:22:41.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.922+0000 7f9fe1400640 1 -- 172.21.15.46:0/4029543329 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd40abb70 con 0x7f9fe41598a0 2024-03-29T08:22:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.922+0000 7f9fe9200640 1 -- 172.21.15.46:0/4029543329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe41598a0 msgr2=0x7f9fe40cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fe9200640 1 --2- 172.21.15.46:0/4029543329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe41598a0 0x7f9fe40cadb0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40b2b70 tx=0x7f9fd401c4f0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fe9200640 1 -- 172.21.15.46:0/4029543329 shutdown_connections 2024-03-29T08:22:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fe9200640 1 --2- 172.21.15.46:0/4029543329 >> v2:172.21.15.73:3300/0 conn(0x7f9fe415dc30 0x7f9fe406c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fe9200640 1 --2- 172.21.15.46:0/4029543329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe41598a0 0x7f9fe40cadb0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fe9200640 1 -- 172.21.15.46:0/4029543329 >> 172.21.15.46:0/4029543329 conn(0x7f9fe40b42d0 msgr2=0x7f9fe4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fe9200640 1 -- 172.21.15.46:0/4029543329 shutdown_connections 2024-03-29T08:22:41.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fe9200640 1 -- 172.21.15.46:0/4029543329 wait complete. 2024-03-29T08:22:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fe9200640 1 Processor -- start 2024-03-29T08:22:41.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fe9200640 1 -- start start 2024-03-29T08:22:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fe9200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9fe40b1470 0x7f9fe415c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fe9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe41598a0 0x7f9fe4144350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fe9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fe415dc30 0x7f9fe40ce150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fe9200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe41583e0 con 0x7f9fe41598a0 2024-03-29T08:22:41.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fe9200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe41f8dc0 con 0x7f9fe415dc30 2024-03-29T08:22:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fe9200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe4151670 con 0x7f9fe40b1470 2024-03-29T08:22:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fdbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe41598a0 0x7f9fe4144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fdbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe41598a0 0x7f9fe4144350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60410/0 (socket says 172.21.15.46:60410) 2024-03-29T08:22:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fdbe00640 1 -- 172.21.15.46:0/2450441655 learned_addr learned my addr 172.21.15.46:0/2450441655 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fe1e00640 1 --2- 172.21.15.46:0/2450441655 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9fe40b1470 0x7f9fe415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fe2800640 1 --2- 172.21.15.46:0/2450441655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fe415dc30 0x7f9fe40ce150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fdbe00640 1 -- 172.21.15.46:0/2450441655 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9fe40b1470 msgr2=0x7f9fe415c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fdbe00640 1 --2- 172.21.15.46:0/2450441655 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9fe40b1470 0x7f9fe415c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fdbe00640 1 -- 172.21.15.46:0/2450441655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fe415dc30 msgr2=0x7f9fe40ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fdbe00640 1 --2- 172.21.15.46:0/2450441655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fe415dc30 0x7f9fe40ce150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fdbe00640 1 -- 172.21.15.46:0/2450441655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd40094d0 con 0x7f9fe41598a0 2024-03-29T08:22:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.926+0000 7f9fdbe00640 1 --2- 172.21.15.46:0/2450441655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe41598a0 0x7f9fe4144350 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00c490 tx=0x7f9fcc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.930+0000 7f9fda000640 1 -- 172.21.15.46:0/2450441655 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc007480 con 0x7f9fe41598a0 2024-03-29T08:22:41.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.930+0000 7f9fda000640 1 -- 172.21.15.46:0/2450441655 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc0075e0 con 0x7f9fe41598a0 2024-03-29T08:22:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.930+0000 7f9fda000640 1 -- 172.21.15.46:0/2450441655 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc004a10 con 0x7f9fe41598a0 2024-03-29T08:22:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.930+0000 7f9fe9200640 1 -- 172.21.15.46:0/2450441655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe415cf30 con 0x7f9fe41598a0 2024-03-29T08:22:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.930+0000 7f9fe9200640 1 -- 172.21.15.46:0/2450441655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe41f8770 con 0x7f9fe41598a0 2024-03-29T08:22:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.930+0000 7f9fe9200640 1 -- 172.21.15.46:0/2450441655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe4066c90 con 0x7f9fe41598a0 2024-03-29T08:22:41.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.930+0000 7f9fda000640 1 -- 172.21.15.46:0/2450441655 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9fcc004b70 con 0x7f9fe41598a0 2024-03-29T08:22:41.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.930+0000 7f9fda000640 1 --2- 172.21.15.46:0/2450441655 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9fc0066690 0x7f9fc0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:41.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.930+0000 7f9fda000640 1 -- 172.21.15.46:0/2450441655 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9fcc082fa0 con 0x7f9fe41598a0 2024-03-29T08:22:41.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.930+0000 7f9fe1e00640 1 --2- 172.21.15.46:0/2450441655 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9fc0066690 0x7f9fc0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:41.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.930+0000 7f9fe1e00640 1 --2- 172.21.15.46:0/2450441655 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9fc0066690 0x7f9fc0068b50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9fd40af000 tx=0x7f9fd40b6040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:41.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:41.934+0000 7f9fda000640 1 -- 172.21.15.46:0/2450441655 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9fcc0b4980 con 0x7f9fe41598a0 2024-03-29T08:22:42.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.098+0000 7f9fe9200640 1 -- 172.21.15.46:0/2450441655 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9fe40b2bd0 con 0x7f9fe41598a0 2024-03-29T08:22:42.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.198+0000 7f9fda000640 1 -- 172.21.15.46:0/2450441655 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9fcc018020 con 0x7f9fe41598a0 2024-03-29T08:22:42.200 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-03-29T08:22:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.202+0000 7f9fe9200640 1 -- 172.21.15.46:0/2450441655 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9fc0066690 msgr2=0x7f9fc0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.202+0000 7f9fe9200640 1 --2- 172.21.15.46:0/2450441655 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9fc0066690 0x7f9fc0068b50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9fd40af000 tx=0x7f9fd40b6040 comp rx=0 tx=0).stop 2024-03-29T08:22:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.202+0000 7f9fe9200640 1 -- 172.21.15.46:0/2450441655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe41598a0 msgr2=0x7f9fe4144350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.202+0000 7f9fe9200640 1 --2- 172.21.15.46:0/2450441655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe41598a0 0x7f9fe4144350 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00c490 tx=0x7f9fcc00c950 comp rx=0 tx=0).stop 2024-03-29T08:22:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.202+0000 7f9fe9200640 1 -- 172.21.15.46:0/2450441655 shutdown_connections 2024-03-29T08:22:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.202+0000 7f9fe9200640 1 --2- 172.21.15.46:0/2450441655 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9fc0066690 0x7f9fc0068b50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.202+0000 7f9fe9200640 1 --2- 172.21.15.46:0/2450441655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fe415dc30 0x7f9fe40ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.202+0000 7f9fe9200640 1 --2- 172.21.15.46:0/2450441655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fe41598a0 0x7f9fe4144350 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.202+0000 7f9fe9200640 1 --2- 172.21.15.46:0/2450441655 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9fe40b1470 0x7f9fe415c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.202+0000 7f9fe9200640 1 -- 172.21.15.46:0/2450441655 >> 172.21.15.46:0/2450441655 conn(0x7f9fe40b42d0 msgr2=0x7f9fe40b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.202+0000 7f9fe9200640 1 -- 172.21.15.46:0/2450441655 shutdown_connections 2024-03-29T08:22:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.202+0000 7f9fe9200640 1 -- 172.21.15.46:0/2450441655 wait complete. 2024-03-29T08:22:42.227 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T08:22:42.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.370+0000 7ff136e00640 1 Processor -- start 2024-03-29T08:22:42.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.370+0000 7ff136e00640 1 -- start start 2024-03-29T08:22:42.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.374+0000 7ff136e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff130098a40 0x7ff130098ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:42.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.374+0000 7ff136e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff1300ad040 0x7ff1300ad490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:42.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.374+0000 7ff136e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1300993e0 con 0x7ff13015ca70 2024-03-29T08:22:42.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.374+0000 7ff136e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1300cfc30 con 0x7ff130098a40 2024-03-29T08:22:42.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.374+0000 7ff136e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1300cf090 con 0x7ff1300ad040 2024-03-29T08:22:42.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.374+0000 7ff135000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff130098a40 0x7ff130098ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:42.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.374+0000 7ff135a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff13015ca70 0x7ff1300cf520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38148/0 (socket says 172.21.15.46:38148) 2024-03-29T08:22:42.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.374+0000 7ff135a00640 1 -- 172.21.15.46:0/2912883275 learned_addr learned my addr 172.21.15.46:0/2912883275 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:42.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.374+0000 7ff136400640 1 --2- 172.21.15.46:0/2912883275 >> v2:172.21.15.73:3300/0 conn(0x7ff1300ad040 0x7ff1300ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:42.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.374+0000 7ff12fe00640 1 -- 172.21.15.46:0/2912883275 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518992681 0 0) 0x7ff1300993e0 con 0x7ff13015ca70 2024-03-29T08:22:42.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.374+0000 7ff135000640 1 -- 172.21.15.46:0/2912883275 >> v2:172.21.15.73:3300/0 conn(0x7ff1300ad040 msgr2=0x7ff1300ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:42.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.374+0000 7ff135000640 1 --2- 172.21.15.46:0/2912883275 >> v2:172.21.15.73:3300/0 conn(0x7ff1300ad040 0x7ff1300ad490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:42.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.374+0000 7ff135000640 1 -- 172.21.15.46:0/2912883275 >> v1:172.21.15.46:6789/0 conn(0x7ff13015ca70 legacy=0x7ff1300cf520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:42.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.374+0000 7ff135000640 1 -- 172.21.15.46:0/2912883275 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1300ce7e0 con 0x7ff130098a40 2024-03-29T08:22:42.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff135000640 1 --2- 172.21.15.46:0/2912883275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff130098a40 0x7ff130098ea0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff13006dc20 tx=0x7ff11c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2bcb36e3a6927b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:42.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff12fe00640 1 -- 172.21.15.46:0/2912883275 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c0ab600 con 0x7ff130098a40 2024-03-29T08:22:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff12fe00640 1 -- 172.21.15.46:0/2912883275 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11c0a7050 con 0x7ff130098a40 2024-03-29T08:22:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff12fe00640 1 -- 172.21.15.46:0/2912883275 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c0ab980 con 0x7ff130098a40 2024-03-29T08:22:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff136e00640 1 -- 172.21.15.46:0/2912883275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff130098a40 msgr2=0x7ff130098ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff136e00640 1 --2- 172.21.15.46:0/2912883275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff130098a40 0x7ff130098ea0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff13006dc20 tx=0x7ff11c01c480 comp rx=0 tx=0).stop 2024-03-29T08:22:42.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff136e00640 1 -- 172.21.15.46:0/2912883275 shutdown_connections 2024-03-29T08:22:42.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff136e00640 1 --2- 172.21.15.46:0/2912883275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff130098a40 0x7ff130098ea0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:42.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff136e00640 1 --2- 172.21.15.46:0/2912883275 >> v2:172.21.15.73:3300/0 conn(0x7ff1300ad040 0x7ff1300ad490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:42.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff136e00640 1 -- 172.21.15.46:0/2912883275 >> 172.21.15.46:0/2912883275 conn(0x7ff1300b1fa0 msgr2=0x7ff1300b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:42.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff136e00640 1 -- 172.21.15.46:0/2912883275 shutdown_connections 2024-03-29T08:22:42.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff136e00640 1 -- 172.21.15.46:0/2912883275 wait complete. 2024-03-29T08:22:42.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff136e00640 1 Processor -- start 2024-03-29T08:22:42.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff136e00640 1 -- start start 2024-03-29T08:22:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff136e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff130098a40 0x7ff13015f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff136e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1300ad040 0x7ff1301423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff136e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff13015ca70 0x7ff130142910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff136e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13015e8f0 con 0x7ff1300ad040 2024-03-29T08:22:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff136e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13015ebd0 con 0x7ff13015ca70 2024-03-29T08:22:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff136e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13015eeb0 con 0x7ff130098a40 2024-03-29T08:22:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff135000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1300ad040 0x7ff1301423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff135000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1300ad040 0x7ff1301423d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60426/0 (socket says 172.21.15.46:60426) 2024-03-29T08:22:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.378+0000 7ff135000640 1 -- 172.21.15.46:0/405485169 learned_addr learned my addr 172.21.15.46:0/405485169 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.382+0000 7ff135a00640 1 --2- 172.21.15.46:0/405485169 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff130098a40 0x7ff13015f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.382+0000 7ff136400640 1 --2- 172.21.15.46:0/405485169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff13015ca70 0x7ff130142910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:42.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.382+0000 7ff135000640 1 -- 172.21.15.46:0/405485169 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff130098a40 msgr2=0x7ff13015f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:42.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.382+0000 7ff135000640 1 --2- 172.21.15.46:0/405485169 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff130098a40 0x7ff13015f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:42.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.382+0000 7ff135000640 1 -- 172.21.15.46:0/405485169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff13015ca70 msgr2=0x7ff130142910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:42.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.382+0000 7ff135000640 1 --2- 172.21.15.46:0/405485169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff13015ca70 0x7ff130142910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:42.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.382+0000 7ff135000640 1 -- 172.21.15.46:0/405485169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c009420 con 0x7ff1300ad040 2024-03-29T08:22:42.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.382+0000 7ff135000640 1 --2- 172.21.15.46:0/405485169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1300ad040 0x7ff1301423d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0ae040 tx=0x7ff11c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.382+0000 7ff12d600640 1 -- 172.21.15.46:0/405485169 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c0b4840 con 0x7ff1300ad040 2024-03-29T08:22:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.382+0000 7ff12d600640 1 -- 172.21.15.46:0/405485169 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11c0b49c0 con 0x7ff1300ad040 2024-03-29T08:22:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.382+0000 7ff136e00640 1 -- 172.21.15.46:0/405485169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13015f190 con 0x7ff1300ad040 2024-03-29T08:22:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.382+0000 7ff12d600640 1 -- 172.21.15.46:0/405485169 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c0b4d70 con 0x7ff1300ad040 2024-03-29T08:22:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.382+0000 7ff136e00640 1 -- 172.21.15.46:0/405485169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1301547f0 con 0x7ff1300ad040 2024-03-29T08:22:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.382+0000 7ff136e00640 1 -- 172.21.15.46:0/405485169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13004e040 con 0x7ff1300ad040 2024-03-29T08:22:42.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.382+0000 7ff12d600640 1 -- 172.21.15.46:0/405485169 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff11c0b2040 con 0x7ff1300ad040 2024-03-29T08:22:42.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.382+0000 7ff12d600640 1 --2- 172.21.15.46:0/405485169 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff110066690 0x7ff110068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:42.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.386+0000 7ff12d600640 1 -- 172.21.15.46:0/405485169 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff11c11e290 con 0x7ff1300ad040 2024-03-29T08:22:42.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.386+0000 7ff135a00640 1 --2- 172.21.15.46:0/405485169 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff110066690 0x7ff110068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:42.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.386+0000 7ff135a00640 1 --2- 172.21.15.46:0/405485169 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff110066690 0x7ff110068b50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff1240010f0 tx=0x7ff124009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:42.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.390+0000 7ff12d600640 1 -- 172.21.15.46:0/405485169 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff11c0eb770 con 0x7ff1300ad040 2024-03-29T08:22:42.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:42.550+0000 7ff136e00640 1 -- 172.21.15.46:0/405485169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff13015f640 con 0x7ff1300ad040 2024-03-29T08:22:43.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.202+0000 7ff12d600640 1 -- 172.21.15.46:0/405485169 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff11c0eed90 con 0x7ff1300ad040 2024-03-29T08:22:43.204 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-03-29T08:22:43.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.206+0000 7ff136e00640 1 -- 172.21.15.46:0/405485169 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff110066690 msgr2=0x7ff110068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:43.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.206+0000 7ff136e00640 1 --2- 172.21.15.46:0/405485169 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff110066690 0x7ff110068b50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff1240010f0 tx=0x7ff124009040 comp rx=0 tx=0).stop 2024-03-29T08:22:43.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.206+0000 7ff136e00640 1 -- 172.21.15.46:0/405485169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1300ad040 msgr2=0x7ff1301423d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:43.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.206+0000 7ff136e00640 1 --2- 172.21.15.46:0/405485169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1300ad040 0x7ff1301423d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0ae040 tx=0x7ff11c019220 comp rx=0 tx=0).stop 2024-03-29T08:22:43.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.206+0000 7ff136e00640 1 -- 172.21.15.46:0/405485169 shutdown_connections 2024-03-29T08:22:43.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.206+0000 7ff136e00640 1 --2- 172.21.15.46:0/405485169 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff110066690 0x7ff110068b50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.206+0000 7ff136e00640 1 --2- 172.21.15.46:0/405485169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff13015ca70 0x7ff130142910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.206+0000 7ff136e00640 1 --2- 172.21.15.46:0/405485169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1300ad040 0x7ff1301423d0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.206+0000 7ff136e00640 1 --2- 172.21.15.46:0/405485169 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff130098a40 0x7ff13015f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.206+0000 7ff136e00640 1 -- 172.21.15.46:0/405485169 >> 172.21.15.46:0/405485169 conn(0x7ff1300b1fa0 msgr2=0x7ff130098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:43.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.206+0000 7ff136e00640 1 -- 172.21.15.46:0/405485169 shutdown_connections 2024-03-29T08:22:43.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.206+0000 7ff136e00640 1 -- 172.21.15.46:0/405485169 wait complete. 2024-03-29T08:22:43.245 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T08:22:43.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.386+0000 7fbee6e00640 1 Processor -- start 2024-03-29T08:22:43.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.386+0000 7fbee6e00640 1 -- start start 2024-03-29T08:22:43.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.390+0000 7fbee6e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbee0142da0 0x7fbee00adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:43.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.390+0000 7fbee6e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbee0143180 0x7fbee00ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:43.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.390+0000 7fbee6e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbee00ae720 con 0x7fbee00abac0 2024-03-29T08:22:43.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.390+0000 7fbee6e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbee00afe80 con 0x7fbee0143180 2024-03-29T08:22:43.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.390+0000 7fbee6e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee006cd80 con 0x7fbee0142da0 2024-03-29T08:22:43.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.390+0000 7fbed7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbee0143180 0x7fbee00ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:43.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.390+0000 7fbed7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbee0143180 0x7fbee00ae1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60434/0 (socket says 172.21.15.46:60434) 2024-03-29T08:22:43.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.390+0000 7fbed7e00640 1 -- 172.21.15.46:0/629823287 learned_addr learned my addr 172.21.15.46:0/629823287 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:43.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.390+0000 7fbee4e00640 1 --1- 172.21.15.46:0/629823287 >> v1:172.21.15.46:6789/0 conn(0x7fbee00abac0 0x7fbee00cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38152/0 (socket says 172.21.15.46:38152) 2024-03-29T08:22:43.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.390+0000 7fbedfe00640 1 --2- 172.21.15.46:0/629823287 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbee0142da0 0x7fbee00adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:43.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.390+0000 7fbed7e00640 1 -- 172.21.15.46:0/629823287 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbee0142da0 msgr2=0x7fbee00adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:43.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.390+0000 7fbed7e00640 1 --2- 172.21.15.46:0/629823287 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbee0142da0 0x7fbee00adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.390+0000 7fbed7e00640 1 -- 172.21.15.46:0/629823287 >> v1:172.21.15.46:6789/0 conn(0x7fbee00abac0 legacy=0x7fbee00cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:43.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.390+0000 7fbed7e00640 1 -- 172.21.15.46:0/629823287 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee006d020 con 0x7fbee0143180 2024-03-29T08:22:43.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.390+0000 7fbed7e00640 1 --2- 172.21.15.46:0/629823287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee0143180 0x7fbee00ae1e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009870 tx=0x7fbec801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aeef709f17fe0f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:43.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbedf400640 1 -- 172.21.15.46:0/629823287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec80a9610 con 0x7fbee0143180 2024-03-29T08:22:43.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbedf400640 1 -- 172.21.15.46:0/629823287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec80a9770 con 0x7fbee0143180 2024-03-29T08:22:43.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbedf400640 1 -- 172.21.15.46:0/629823287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec80a9b00 con 0x7fbee0143180 2024-03-29T08:22:43.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee6e00640 1 -- 172.21.15.46:0/629823287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee0143180 msgr2=0x7fbee00ae1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:43.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee6e00640 1 --2- 172.21.15.46:0/629823287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee0143180 0x7fbee00ae1e0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009870 tx=0x7fbec801c530 comp rx=0 tx=0).stop 2024-03-29T08:22:43.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee6e00640 1 -- 172.21.15.46:0/629823287 shutdown_connections 2024-03-29T08:22:43.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee6e00640 1 --2- 172.21.15.46:0/629823287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee0143180 0x7fbee00ae1e0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee6e00640 1 --2- 172.21.15.46:0/629823287 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbee0142da0 0x7fbee00adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee6e00640 1 -- 172.21.15.46:0/629823287 >> 172.21.15.46:0/629823287 conn(0x7fbee006a390 msgr2=0x7fbee00cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:43.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee6e00640 1 -- 172.21.15.46:0/629823287 shutdown_connections 2024-03-29T08:22:43.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee6e00640 1 -- 172.21.15.46:0/629823287 wait complete. 2024-03-29T08:22:43.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee6e00640 1 Processor -- start 2024-03-29T08:22:43.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee6e00640 1 -- start start 2024-03-29T08:22:43.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee6e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbee00abac0 0x7fbee013b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:43.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee6e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbee0142da0 0x7fbee015f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:43.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee6e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee0143180 0x7fbee0145bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:43.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee6e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee015ce60 con 0x7fbee0143180 2024-03-29T08:22:43.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee6e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee015d140 con 0x7fbee0142da0 2024-03-29T08:22:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee6e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbee015d420 con 0x7fbee00abac0 2024-03-29T08:22:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee0143180 0x7fbee0145bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee4e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee0143180 0x7fbee0145bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60440/0 (socket says 172.21.15.46:60440) 2024-03-29T08:22:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee4e00640 1 -- 172.21.15.46:0/3451666184 learned_addr learned my addr 172.21.15.46:0/3451666184 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbedfe00640 1 --2- 172.21.15.46:0/3451666184 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbee00abac0 0x7fbee013b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbed7e00640 1 --2- 172.21.15.46:0/3451666184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbee0142da0 0x7fbee015f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee4e00640 1 -- 172.21.15.46:0/3451666184 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbee00abac0 msgr2=0x7fbee013b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee4e00640 1 --2- 172.21.15.46:0/3451666184 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbee00abac0 0x7fbee013b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee4e00640 1 -- 172.21.15.46:0/3451666184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbee0142da0 msgr2=0x7fbee015f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee4e00640 1 --2- 172.21.15.46:0/3451666184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbee0142da0 0x7fbee015f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.394+0000 7fbee4e00640 1 -- 172.21.15.46:0/3451666184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec80094d0 con 0x7fbee0143180 2024-03-29T08:22:43.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.398+0000 7fbee4e00640 1 --2- 172.21.15.46:0/3451666184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee0143180 0x7fbee0145bb0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbed800c770 tx=0x7fbed800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:43.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.398+0000 7fbedcc00640 1 -- 172.21.15.46:0/3451666184 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed800ce80 con 0x7fbee0143180 2024-03-29T08:22:43.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.398+0000 7fbedcc00640 1 -- 172.21.15.46:0/3451666184 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbed800a640 con 0x7fbee0143180 2024-03-29T08:22:43.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.398+0000 7fbedcc00640 1 -- 172.21.15.46:0/3451666184 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed800a9f0 con 0x7fbee0143180 2024-03-29T08:22:43.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.398+0000 7fbee6e00640 1 -- 172.21.15.46:0/3451666184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee015d700 con 0x7fbee0143180 2024-03-29T08:22:43.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.398+0000 7fbee6e00640 1 -- 172.21.15.46:0/3451666184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee01460f0 con 0x7fbee0143180 2024-03-29T08:22:43.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.398+0000 7fbee6e00640 1 -- 172.21.15.46:0/3451666184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee015dc20 con 0x7fbee0143180 2024-03-29T08:22:43.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.398+0000 7fbedcc00640 1 -- 172.21.15.46:0/3451666184 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbed800ab50 con 0x7fbee0143180 2024-03-29T08:22:43.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.398+0000 7fbedcc00640 1 --2- 172.21.15.46:0/3451666184 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbebc066690 0x7fbebc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:43.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.398+0000 7fbedcc00640 1 -- 172.21.15.46:0/3451666184 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fbed8083980 con 0x7fbee0143180 2024-03-29T08:22:43.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.398+0000 7fbedfe00640 1 --2- 172.21.15.46:0/3451666184 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbebc066690 0x7fbebc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:43.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.402+0000 7fbedfe00640 1 --2- 172.21.15.46:0/3451666184 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbebc066690 0x7fbebc068b50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbee00b1350 tx=0x7fbed0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:43.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.406+0000 7fbedcc00640 1 -- 172.21.15.46:0/3451666184 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbee015dc20 con 0x7fbee0143180 2024-03-29T08:22:43.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.598+0000 7fbee6e00640 1 -- 172.21.15.46:0/3451666184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbee015dc20 con 0x7fbee0143180 2024-03-29T08:22:43.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.610+0000 7fbedcc00640 1 -- 172.21.15.46:0/3451666184 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v38) v1 ==== 174+0+0 (secure 0 0 0) 0x7fbee015dc20 con 0x7fbee0143180 2024-03-29T08:22:43.612 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 10 and data pool 11 2024-03-29T08:22:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.614+0000 7fbee6e00640 1 -- 172.21.15.46:0/3451666184 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbebc066690 msgr2=0x7fbebc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.614+0000 7fbee6e00640 1 --2- 172.21.15.46:0/3451666184 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbebc066690 0x7fbebc068b50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbee00b1350 tx=0x7fbed0008040 comp rx=0 tx=0).stop 2024-03-29T08:22:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.614+0000 7fbee6e00640 1 -- 172.21.15.46:0/3451666184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee0143180 msgr2=0x7fbee0145bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:43.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.614+0000 7fbee6e00640 1 --2- 172.21.15.46:0/3451666184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee0143180 0x7fbee0145bb0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fbed800c770 tx=0x7fbed800cc30 comp rx=0 tx=0).stop 2024-03-29T08:22:43.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.614+0000 7fbee6e00640 1 -- 172.21.15.46:0/3451666184 shutdown_connections 2024-03-29T08:22:43.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.614+0000 7fbee6e00640 1 --2- 172.21.15.46:0/3451666184 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbebc066690 0x7fbebc068b50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.614+0000 7fbee6e00640 1 --2- 172.21.15.46:0/3451666184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbee0142da0 0x7fbee015f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.614+0000 7fbee6e00640 1 --2- 172.21.15.46:0/3451666184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbee0143180 0x7fbee0145bb0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.614+0000 7fbee6e00640 1 --2- 172.21.15.46:0/3451666184 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbee00abac0 0x7fbee013b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.614+0000 7fbee6e00640 1 -- 172.21.15.46:0/3451666184 >> 172.21.15.46:0/3451666184 conn(0x7fbee006a390 msgr2=0x7fbee01604b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:43.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.618+0000 7fbee6e00640 1 -- 172.21.15.46:0/3451666184 shutdown_connections 2024-03-29T08:22:43.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.618+0000 7fbee6e00640 1 -- 172.21.15.46:0/3451666184 wait complete. 2024-03-29T08:22:43.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:22:43.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.798+0000 7f0465400640 1 Processor -- start 2024-03-29T08:22:43.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.798+0000 7f0465400640 1 -- start start 2024-03-29T08:22:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.802+0000 7f0465400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f046015feb0 0x7f0460071370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.802+0000 7f0465400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f04601605f0 0x7f0460069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.802+0000 7f0465400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0460160a50 0x7f046006a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.802+0000 7f0465400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0460161020 con 0x7f0460160a50 2024-03-29T08:22:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.802+0000 7f0465400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0460161300 con 0x7f04601605f0 2024-03-29T08:22:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.802+0000 7f0465400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0460070b70 con 0x7f046015feb0 2024-03-29T08:22:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.802+0000 7f045f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0460160a50 0x7f046006a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:43.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.802+0000 7f045f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0460160a50 0x7f046006a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60448/0 (socket says 172.21.15.46:60448) 2024-03-29T08:22:43.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.802+0000 7f045f200640 1 -- 172.21.15.46:0/45058973 learned_addr learned my addr 172.21.15.46:0/45058973 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:43.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.802+0000 7f045e800640 1 --2- 172.21.15.46:0/45058973 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f046015feb0 0x7f0460071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:43.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.802+0000 7f0457e00640 1 --2- 172.21.15.46:0/45058973 >> v2:172.21.15.73:3300/0 conn(0x7f04601605f0 0x7f0460069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:43.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.802+0000 7f045f200640 1 -- 172.21.15.46:0/45058973 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f046015feb0 msgr2=0x7f0460071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:43.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.802+0000 7f045f200640 1 --2- 172.21.15.46:0/45058973 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f046015feb0 0x7f0460071370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.802+0000 7f045f200640 1 -- 172.21.15.46:0/45058973 >> v2:172.21.15.73:3300/0 conn(0x7f04601605f0 msgr2=0x7f0460069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:43.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.802+0000 7f045f200640 1 --2- 172.21.15.46:0/45058973 >> v2:172.21.15.73:3300/0 conn(0x7f04601605f0 0x7f0460069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.802+0000 7f045f200640 1 -- 172.21.15.46:0/45058973 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0460070e50 con 0x7f0460160a50 2024-03-29T08:22:43.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.802+0000 7f0457e00640 1 --2- 172.21.15.46:0/45058973 >> v2:172.21.15.73:3300/0 conn(0x7f04601605f0 0x7f0460069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:43.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.806+0000 7f045f200640 1 --2- 172.21.15.46:0/45058973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0460160a50 0x7f046006a2d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f04600ccb20 tx=0x7f045801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19f1f4a2505a2f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:43.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.806+0000 7f045de00640 1 -- 172.21.15.46:0/45058973 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04580a5800 con 0x7f0460160a50 2024-03-29T08:22:43.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.806+0000 7f045de00640 1 -- 172.21.15.46:0/45058973 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04580a5960 con 0x7f0460160a50 2024-03-29T08:22:43.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.806+0000 7f045de00640 1 -- 172.21.15.46:0/45058973 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04580a5cf0 con 0x7f0460160a50 2024-03-29T08:22:43.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.806+0000 7f0465400640 1 -- 172.21.15.46:0/45058973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0460160a50 msgr2=0x7f046006a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:43.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.806+0000 7f0465400640 1 --2- 172.21.15.46:0/45058973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0460160a50 0x7f046006a2d0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f04600ccb20 tx=0x7f045801c960 comp rx=0 tx=0).stop 2024-03-29T08:22:43.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.806+0000 7f0465400640 1 -- 172.21.15.46:0/45058973 shutdown_connections 2024-03-29T08:22:43.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.806+0000 7f0465400640 1 --2- 172.21.15.46:0/45058973 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0460160a50 0x7f046006a2d0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.806+0000 7f0465400640 1 --2- 172.21.15.46:0/45058973 >> v2:172.21.15.73:3300/0 conn(0x7f04601605f0 0x7f0460069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.806+0000 7f0465400640 1 --2- 172.21.15.46:0/45058973 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f046015feb0 0x7f0460071370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.806+0000 7f0465400640 1 -- 172.21.15.46:0/45058973 >> 172.21.15.46:0/45058973 conn(0x7f04600ced50 msgr2=0x7f04600cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:43.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.806+0000 7f0465400640 1 -- 172.21.15.46:0/45058973 shutdown_connections 2024-03-29T08:22:43.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.806+0000 7f0465400640 1 -- 172.21.15.46:0/45058973 wait complete. 2024-03-29T08:22:43.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.806+0000 7f0465400640 1 Processor -- start 2024-03-29T08:22:43.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.806+0000 7f0465400640 1 -- start start 2024-03-29T08:22:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0465400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f046015feb0 0x7f04601515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0465400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04601605f0 0x7f04600cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0465400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0460160a50 0x7f046014cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0465400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04600b19a0 con 0x7f04601605f0 2024-03-29T08:22:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0465400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0460151f50 con 0x7f0460160a50 2024-03-29T08:22:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0465400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0460152230 con 0x7f046015feb0 2024-03-29T08:22:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0457e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04601605f0 0x7f04600cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:43.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0457e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04601605f0 0x7f04600cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60454/0 (socket says 172.21.15.46:60454) 2024-03-29T08:22:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0457e00640 1 -- 172.21.15.46:0/1074396923 learned_addr learned my addr 172.21.15.46:0/1074396923 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f045e800640 1 --2- 172.21.15.46:0/1074396923 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f046015feb0 0x7f04601515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f045f200640 1 --2- 172.21.15.46:0/1074396923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0460160a50 0x7f046014cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0457e00640 1 -- 172.21.15.46:0/1074396923 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f046015feb0 msgr2=0x7f04601515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0457e00640 1 --2- 172.21.15.46:0/1074396923 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f046015feb0 0x7f04601515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0457e00640 1 -- 172.21.15.46:0/1074396923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0460160a50 msgr2=0x7f046014cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0457e00640 1 --2- 172.21.15.46:0/1074396923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0460160a50 0x7f046014cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0457e00640 1 -- 172.21.15.46:0/1074396923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0458009b30 con 0x7f04601605f0 2024-03-29T08:22:43.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0457e00640 1 --2- 172.21.15.46:0/1074396923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04601605f0 0x7f04600cd140 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f044800d800 tx=0x7f044800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:43.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0456a00640 1 -- 172.21.15.46:0/1074396923 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044800b6c0 con 0x7f04601605f0 2024-03-29T08:22:43.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0456a00640 1 -- 172.21.15.46:0/1074396923 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044800b820 con 0x7f04601605f0 2024-03-29T08:22:43.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0456a00640 1 -- 172.21.15.46:0/1074396923 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044800bbd0 con 0x7f04601605f0 2024-03-29T08:22:43.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0465400640 1 -- 172.21.15.46:0/1074396923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0460152510 con 0x7f04601605f0 2024-03-29T08:22:43.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0465400640 1 -- 172.21.15.46:0/1074396923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0460151b20 con 0x7f04601605f0 2024-03-29T08:22:43.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0465400640 1 -- 172.21.15.46:0/1074396923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0460066c90 con 0x7f04601605f0 2024-03-29T08:22:43.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.810+0000 7f0456a00640 1 -- 172.21.15.46:0/1074396923 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f044800bd30 con 0x7f04601605f0 2024-03-29T08:22:43.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.814+0000 7f0456a00640 1 --2- 172.21.15.46:0/1074396923 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f043c066690 0x7f043c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:43.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.814+0000 7f0456a00640 1 -- 172.21.15.46:0/1074396923 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0448084580 con 0x7f04601605f0 2024-03-29T08:22:43.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.814+0000 7f045e800640 1 --2- 172.21.15.46:0/1074396923 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f043c066690 0x7f043c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:43.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.814+0000 7f045e800640 1 --2- 172.21.15.46:0/1074396923 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f043c066690 0x7f043c068b50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f04500079c0 tx=0x7f0450008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:43.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.818+0000 7f0456a00640 1 -- 172.21.15.46:0/1074396923 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0448051900 con 0x7f04601605f0 2024-03-29T08:22:43.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.982+0000 7f0465400640 1 -- 172.21.15.46:0/1074396923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04600cb540 con 0x7f04601605f0 2024-03-29T08:22:43.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.982+0000 7f0456a00640 1 -- 172.21.15.46:0/1074396923 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16005 (secure 0 0 0) 0x7f0448054f20 con 0x7f04601605f0 2024-03-29T08:22:43.986 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:43.986 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":76,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:22:43.605132+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:22:42.147364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-29T08:22:42.602168+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:22:43.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.986+0000 7f0465400640 1 -- 172.21.15.46:0/1074396923 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f043c066690 msgr2=0x7f043c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:43.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.986+0000 7f0465400640 1 --2- 172.21.15.46:0/1074396923 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f043c066690 0x7f043c068b50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f04500079c0 tx=0x7f0450008040 comp rx=0 tx=0).stop 2024-03-29T08:22:43.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.986+0000 7f0465400640 1 -- 172.21.15.46:0/1074396923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04601605f0 msgr2=0x7f04600cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:43.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.986+0000 7f0465400640 1 --2- 172.21.15.46:0/1074396923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04601605f0 0x7f04600cd140 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f044800d800 tx=0x7f044800dcc0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.986+0000 7f0465400640 1 -- 172.21.15.46:0/1074396923 shutdown_connections 2024-03-29T08:22:43.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.986+0000 7f0465400640 1 --2- 172.21.15.46:0/1074396923 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f043c066690 0x7f043c068b50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.986+0000 7f0465400640 1 --2- 172.21.15.46:0/1074396923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0460160a50 0x7f046014cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.986+0000 7f0465400640 1 --2- 172.21.15.46:0/1074396923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04601605f0 0x7f04600cd140 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.986+0000 7f0465400640 1 --2- 172.21.15.46:0/1074396923 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f046015feb0 0x7f04601515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:43.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.986+0000 7f0465400640 1 -- 172.21.15.46:0/1074396923 >> 172.21.15.46:0/1074396923 conn(0x7f04600ced50 msgr2=0x7f0460155150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:43.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.986+0000 7f0465400640 1 -- 172.21.15.46:0/1074396923 shutdown_connections 2024-03-29T08:22:43.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:43.986+0000 7f0465400640 1 -- 172.21.15.46:0/1074396923 wait complete. 2024-03-29T08:22:44.007 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:22:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4af000640 1 Processor -- start 2024-03-29T08:22:44.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4af000640 1 -- start start 2024-03-29T08:22:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4af000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb4a80d2670 0x7fb4a80d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4af000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a80d0900 con 0x7fb4a815c970 2024-03-29T08:22:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4af000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a815cd50 con 0x7fb4a80d0d90 2024-03-29T08:22:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4af000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a80d3040 con 0x7fb4a80d2670 2024-03-29T08:22:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4adc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb4a815c970 0x7fb4a806b550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38162/0 (socket says 172.21.15.46:38162) 2024-03-29T08:22:44.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4adc00640 1 -- 172.21.15.46:0/440804327 learned_addr learned my addr 172.21.15.46:0/440804327 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4ad200640 1 --2- 172.21.15.46:0/440804327 >> v2:172.21.15.73:3300/0 conn(0x7fb4a80d2670 0x7fb4a80d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4a7e00640 1 -- 172.21.15.46:0/440804327 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1014445566 0 0) 0x7fb4a80d0900 con 0x7fb4a815c970 2024-03-29T08:22:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4a7e00640 1 -- 172.21.15.46:0/440804327 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb490003530 con 0x7fb4a815c970 2024-03-29T08:22:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4a7e00640 1 -- 172.21.15.46:0/440804327 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1203318929 0 0) 0x7fb490003530 con 0x7fb4a815c970 2024-03-29T08:22:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4a7e00640 1 -- 172.21.15.46:0/440804327 >> v2:172.21.15.73:3300/0 conn(0x7fb4a80d2670 msgr2=0x7fb4a80d2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4a7e00640 1 --2- 172.21.15.46:0/440804327 >> v2:172.21.15.73:3300/0 conn(0x7fb4a80d2670 0x7fb4a80d2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4a7e00640 1 -- 172.21.15.46:0/440804327 >> v1:172.21.15.73:6789/0 conn(0x7fb4a80d0d90 legacy=0x7fb4a80d1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:44.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4a7e00640 1 -- 172.21.15.46:0/440804327 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a80ce240 con 0x7fb4a815c970 2024-03-29T08:22:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4a7e00640 1 -- 172.21.15.46:0/440804327 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fb49c001680 con 0x7fb4a815c970 2024-03-29T08:22:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4a7e00640 1 -- 172.21.15.46:0/440804327 >> v1:172.21.15.46:6789/0 conn(0x7fb4a815c970 legacy=0x7fb4a806b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4a7e00640 1 --2- 172.21.15.46:0/440804327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490003980 0x7fb490003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4a7e00640 1 --2- 172.21.15.46:0/440804327 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4900047d0 0x7fb490004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4a7e00640 1 --2- 172.21.15.46:0/440804327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb490005190 0x7fb49000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4a7e00640 1 -- 172.21.15.46:0/440804327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a80d3040 con 0x7fb490003980 2024-03-29T08:22:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4a7e00640 1 -- 172.21.15.46:0/440804327 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49000c030 con 0x7fb490005190 2024-03-29T08:22:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4a7e00640 1 -- 172.21.15.46:0/440804327 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49000c1f0 con 0x7fb4900047d0 2024-03-29T08:22:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4adc00640 1 --2- 172.21.15.46:0/440804327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490003980 0x7fb490003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4ae600640 1 --2- 172.21.15.46:0/440804327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb490005190 0x7fb49000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4ad200640 1 --2- 172.21.15.46:0/440804327 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4900047d0 0x7fb490004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4adc00640 1 -- 172.21.15.46:0/440804327 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4900047d0 msgr2=0x7fb490004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4adc00640 1 --2- 172.21.15.46:0/440804327 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4900047d0 0x7fb490004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4adc00640 1 -- 172.21.15.46:0/440804327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb490005190 msgr2=0x7fb49000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:44.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4adc00640 1 --2- 172.21.15.46:0/440804327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb490005190 0x7fb49000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4ae600640 1 -- 172.21.15.46:0/440804327 reap_dead start 2024-03-29T08:22:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.146+0000 7fb4adc00640 1 -- 172.21.15.46:0/440804327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb490003530 con 0x7fb490003980 2024-03-29T08:22:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.150+0000 7fb4adc00640 1 --2- 172.21.15.46:0/440804327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490003980 0x7fb490003da0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00c240 tx=0x7fb49c01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69dd9d9381b3bbf9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:44.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.150+0000 7fb4a7e00640 1 -- 172.21.15.46:0/440804327 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c005ae0 con 0x7fb490003980 2024-03-29T08:22:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.150+0000 7fb4a7e00640 1 -- 172.21.15.46:0/440804327 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49c005c40 con 0x7fb490003980 2024-03-29T08:22:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.150+0000 7fb4a7e00640 1 -- 172.21.15.46:0/440804327 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c0a5a60 con 0x7fb490003980 2024-03-29T08:22:44.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.150+0000 7fb4af000640 1 -- 172.21.15.46:0/440804327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490003980 msgr2=0x7fb490003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.150+0000 7fb4af000640 1 --2- 172.21.15.46:0/440804327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490003980 0x7fb490003da0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00c240 tx=0x7fb49c01cc30 comp rx=0 tx=0).stop 2024-03-29T08:22:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.150+0000 7fb4af000640 1 -- 172.21.15.46:0/440804327 shutdown_connections 2024-03-29T08:22:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.150+0000 7fb4af000640 1 --2- 172.21.15.46:0/440804327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490003980 0x7fb490003da0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.150+0000 7fb4af000640 1 -- 172.21.15.46:0/440804327 >> 172.21.15.46:0/440804327 conn(0x7fb4a809ba50 msgr2=0x7fb4a8089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.150+0000 7fb4af000640 1 -- 172.21.15.46:0/440804327 shutdown_connections 2024-03-29T08:22:44.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.150+0000 7fb4af000640 1 -- 172.21.15.46:0/440804327 wait complete. 2024-03-29T08:22:44.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4af000640 1 Processor -- start 2024-03-29T08:22:44.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4af000640 1 -- start start 2024-03-29T08:22:44.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4af000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490003980 0x7fb4a80b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:44.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4af000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4a80b6fd0 0x7fb4a8150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:44.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4af000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4a80b7e60 0x7fb4a8160400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:44.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4af000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a80b7d00 con 0x7fb490003980 2024-03-29T08:22:44.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4af000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a80b8360 con 0x7fb4a80b6fd0 2024-03-29T08:22:44.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4af000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a80b64f0 con 0x7fb4a80b7e60 2024-03-29T08:22:44.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4adc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490003980 0x7fb4a80b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:44.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4adc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490003980 0x7fb4a80b77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60474/0 (socket says 172.21.15.46:60474) 2024-03-29T08:22:44.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4adc00640 1 -- 172.21.15.46:0/2183108568 learned_addr learned my addr 172.21.15.46:0/2183108568 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:44.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4ad200640 1 --2- 172.21.15.46:0/2183108568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4a80b6fd0 0x7fb4a8150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:44.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4ae600640 1 --2- 172.21.15.46:0/2183108568 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4a80b7e60 0x7fb4a8160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:44.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4adc00640 1 -- 172.21.15.46:0/2183108568 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4a80b7e60 msgr2=0x7fb4a8160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:44.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4adc00640 1 --2- 172.21.15.46:0/2183108568 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4a80b7e60 0x7fb4a8160400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:44.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4adc00640 1 -- 172.21.15.46:0/2183108568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4a80b6fd0 msgr2=0x7fb4a8150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:44.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4adc00640 1 --2- 172.21.15.46:0/2183108568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4a80b6fd0 0x7fb4a8150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:44.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4adc00640 1 -- 172.21.15.46:0/2183108568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a80b67d0 con 0x7fb490003980 2024-03-29T08:22:44.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4adc00640 1 --2- 172.21.15.46:0/2183108568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490003980 0x7fb4a80b77c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb49c01ca60 tx=0x7fb490005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:44.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4a5600640 1 -- 172.21.15.46:0/2183108568 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c0b4070 con 0x7fb490003980 2024-03-29T08:22:44.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4a5600640 1 -- 172.21.15.46:0/2183108568 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49c001440 con 0x7fb490003980 2024-03-29T08:22:44.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4a5600640 1 -- 172.21.15.46:0/2183108568 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c0a5820 con 0x7fb490003980 2024-03-29T08:22:44.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4af000640 1 -- 172.21.15.46:0/2183108568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a80b6ab0 con 0x7fb490003980 2024-03-29T08:22:44.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.154+0000 7fb4af000640 1 -- 172.21.15.46:0/2183108568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a8150f40 con 0x7fb490003980 2024-03-29T08:22:44.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.158+0000 7fb4af000640 1 -- 172.21.15.46:0/2183108568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a80b6350 con 0x7fb490003980 2024-03-29T08:22:44.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.158+0000 7fb4a5600640 1 -- 172.21.15.46:0/2183108568 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb49c0a5980 con 0x7fb490003980 2024-03-29T08:22:44.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.158+0000 7fb4a5600640 1 --2- 172.21.15.46:0/2183108568 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb4880666c0 0x7fb488068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:44.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.158+0000 7fb4a5600640 1 -- 172.21.15.46:0/2183108568 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb49c11dbd0 con 0x7fb490003980 2024-03-29T08:22:44.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.158+0000 7fb4ad200640 1 --2- 172.21.15.46:0/2183108568 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb4880666c0 0x7fb488068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:44.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.158+0000 7fb4ad200640 1 --2- 172.21.15.46:0/2183108568 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb4880666c0 0x7fb488068b80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8160f50 tx=0x7fb494009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:44.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.162+0000 7fb4a5600640 1 -- 172.21.15.46:0/2183108568 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb4a80b6350 con 0x7fb490003980 2024-03-29T08:22:44.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.326+0000 7fb4af000640 1 -- 172.21.15.46:0/2183108568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4a80b6350 con 0x7fb490003980 2024-03-29T08:22:44.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.326+0000 7fb4a5600640 1 -- 172.21.15.46:0/2183108568 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16005 (secure 0 0 0) 0x7fb4a80b6350 con 0x7fb490003980 2024-03-29T08:22:44.330 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:44.331 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":76,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:22:43.605132+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:22:42.147364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-29T08:22:42.602168+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:22:44.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.330+0000 7fb4af000640 1 -- 172.21.15.46:0/2183108568 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb4880666c0 msgr2=0x7fb488068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:44.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.330+0000 7fb4af000640 1 --2- 172.21.15.46:0/2183108568 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb4880666c0 0x7fb488068b80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8160f50 tx=0x7fb494009190 comp rx=0 tx=0).stop 2024-03-29T08:22:44.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.330+0000 7fb4af000640 1 -- 172.21.15.46:0/2183108568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490003980 msgr2=0x7fb4a80b77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:44.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.330+0000 7fb4af000640 1 --2- 172.21.15.46:0/2183108568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490003980 0x7fb4a80b77c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb49c01ca60 tx=0x7fb490005800 comp rx=0 tx=0).stop 2024-03-29T08:22:44.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.330+0000 7fb4af000640 1 -- 172.21.15.46:0/2183108568 shutdown_connections 2024-03-29T08:22:44.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.330+0000 7fb4af000640 1 --2- 172.21.15.46:0/2183108568 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb4880666c0 0x7fb488068b80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:44.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.330+0000 7fb4af000640 1 --2- 172.21.15.46:0/2183108568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4a80b6fd0 0x7fb4a8150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:44.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.330+0000 7fb4af000640 1 --2- 172.21.15.46:0/2183108568 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb4a80b7e60 0x7fb4a8160400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:44.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.330+0000 7fb4af000640 1 --2- 172.21.15.46:0/2183108568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490003980 0x7fb4a80b77c0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:44.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.330+0000 7fb4af000640 1 -- 172.21.15.46:0/2183108568 >> 172.21.15.46:0/2183108568 conn(0x7fb4a809ba50 msgr2=0x7fb4a80cbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:44.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.330+0000 7fb4af000640 1 -- 172.21.15.46:0/2183108568 shutdown_connections 2024-03-29T08:22:44.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.330+0000 7fb4af000640 1 -- 172.21.15.46:0/2183108568 wait complete. 2024-03-29T08:22:44.351 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T08:22:44.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.490+0000 7ffafae00640 1 Processor -- start 2024-03-29T08:22:44.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.490+0000 7ffafae00640 1 -- start start 2024-03-29T08:22:44.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.494+0000 7ffafae00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffaf40b1570 0x7ffaf40b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:44.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.494+0000 7ffafae00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffaf4098a40 0x7ffaf4098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:44.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.494+0000 7ffafae00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf40cb3f0 con 0x7ffaf415dfb0 2024-03-29T08:22:44.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.494+0000 7ffafae00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf4099410 con 0x7ffaf4098a40 2024-03-29T08:22:44.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.494+0000 7ffafae00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf40b1e00 con 0x7ffaf40b1570 2024-03-29T08:22:44.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.494+0000 7ffaf3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffaf415dfb0 0x7ffaf40ced10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38166/0 (socket says 172.21.15.46:38166) 2024-03-29T08:22:44.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.494+0000 7ffaf3e00640 1 -- 172.21.15.46:0/3470932184 learned_addr learned my addr 172.21.15.46:0/3470932184 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:44.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.494+0000 7ffaf8e00640 1 --2- 172.21.15.46:0/3470932184 >> v2:172.21.15.46:3300/0 conn(0x7ffaf4098a40 0x7ffaf4098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:44.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.494+0000 7ffaeb400640 1 --2- 172.21.15.46:0/3470932184 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffaf40b1570 0x7ffaf40b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.494+0000 7ffaf8e00640 1 -- 172.21.15.46:0/3470932184 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffaf40b1570 msgr2=0x7ffaf40b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.494+0000 7ffaf8e00640 1 --2- 172.21.15.46:0/3470932184 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffaf40b1570 0x7ffaf40b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.494+0000 7ffaf8e00640 1 -- 172.21.15.46:0/3470932184 >> v1:172.21.15.46:6789/0 conn(0x7ffaf415dfb0 legacy=0x7ffaf40ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.494+0000 7ffaf8e00640 1 -- 172.21.15.46:0/3470932184 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf40cb620 con 0x7ffaf4098a40 2024-03-29T08:22:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.494+0000 7ffaeb400640 1 --2- 172.21.15.46:0/3470932184 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffaf40b1570 0x7ffaf40b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:44.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.494+0000 7ffaf3400640 1 -- 172.21.15.46:0/3470932184 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480985708 0 0) 0x7ffaf40cb3f0 con 0x7ffaf415dfb0 2024-03-29T08:22:44.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.498+0000 7ffaf8e00640 1 --2- 172.21.15.46:0/3470932184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf4098a40 0x7ffaf4098ed0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009d90 tx=0x7ffaec01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4c42d908436fc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:44.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.498+0000 7ffaf3400640 1 -- 172.21.15.46:0/3470932184 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec0a5910 con 0x7ffaf4098a40 2024-03-29T08:22:44.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.498+0000 7ffaf3400640 1 -- 172.21.15.46:0/3470932184 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaec0a5a70 con 0x7ffaf4098a40 2024-03-29T08:22:44.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.498+0000 7ffaf3400640 1 -- 172.21.15.46:0/3470932184 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec0a5e00 con 0x7ffaf4098a40 2024-03-29T08:22:44.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.498+0000 7ffafae00640 1 -- 172.21.15.46:0/3470932184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf4098a40 msgr2=0x7ffaf4098ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:44.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.498+0000 7ffafae00640 1 --2- 172.21.15.46:0/3470932184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf4098a40 0x7ffaf4098ed0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009d90 tx=0x7ffaec01c940 comp rx=0 tx=0).stop 2024-03-29T08:22:44.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.498+0000 7ffafae00640 1 -- 172.21.15.46:0/3470932184 shutdown_connections 2024-03-29T08:22:44.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.498+0000 7ffafae00640 1 --2- 172.21.15.46:0/3470932184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf4098a40 0x7ffaf4098ed0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:44.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.498+0000 7ffafae00640 1 --2- 172.21.15.46:0/3470932184 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffaf40b1570 0x7ffaf40b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:44.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.498+0000 7ffafae00640 1 -- 172.21.15.46:0/3470932184 >> 172.21.15.46:0/3470932184 conn(0x7ffaf40b2220 msgr2=0x7ffaf40b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:44.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.498+0000 7ffafae00640 1 -- 172.21.15.46:0/3470932184 shutdown_connections 2024-03-29T08:22:44.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.498+0000 7ffafae00640 1 -- 172.21.15.46:0/3470932184 wait complete. 2024-03-29T08:22:44.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.498+0000 7ffafae00640 1 Processor -- start 2024-03-29T08:22:44.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.498+0000 7ffafae00640 1 -- start start 2024-03-29T08:22:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.498+0000 7ffafae00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffaf4098a40 0x7ffaf4154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffafae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf40b1570 0x7ffaf4153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffafae00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaf415dfb0 0x7ffaf406c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffafae00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf406bef0 con 0x7ffaf40b1570 2024-03-29T08:22:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffafae00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf4164b70 con 0x7ffaf415dfb0 2024-03-29T08:22:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffafae00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf41642d0 con 0x7ffaf4098a40 2024-03-29T08:22:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffaeb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf40b1570 0x7ffaf4153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:44.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffaeb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf40b1570 0x7ffaf4153c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60480/0 (socket says 172.21.15.46:60480) 2024-03-29T08:22:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffaeb400640 1 -- 172.21.15.46:0/4121435804 learned_addr learned my addr 172.21.15.46:0/4121435804 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffaf8e00640 1 --2- 172.21.15.46:0/4121435804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaf415dfb0 0x7ffaf406c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffaf3e00640 1 --2- 172.21.15.46:0/4121435804 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffaf4098a40 0x7ffaf4154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffaeb400640 1 -- 172.21.15.46:0/4121435804 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffaf4098a40 msgr2=0x7ffaf4154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffaeb400640 1 --2- 172.21.15.46:0/4121435804 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffaf4098a40 0x7ffaf4154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffaeb400640 1 -- 172.21.15.46:0/4121435804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaf415dfb0 msgr2=0x7ffaf406c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffaeb400640 1 --2- 172.21.15.46:0/4121435804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaf415dfb0 0x7ffaf406c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffaeb400640 1 -- 172.21.15.46:0/4121435804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec009a70 con 0x7ffaf40b1570 2024-03-29T08:22:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffaeb400640 1 --2- 172.21.15.46:0/4121435804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf40b1570 0x7ffaf4153c80 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00e810 tx=0x7ffadc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffaf0c00640 1 -- 172.21.15.46:0/4121435804 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc00c8d0 con 0x7ffaf40b1570 2024-03-29T08:22:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffaf0c00640 1 -- 172.21.15.46:0/4121435804 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffadc00ca30 con 0x7ffaf40b1570 2024-03-29T08:22:44.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffaf0c00640 1 -- 172.21.15.46:0/4121435804 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc00cdc0 con 0x7ffaf40b1570 2024-03-29T08:22:44.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffafae00640 1 -- 172.21.15.46:0/4121435804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf41645b0 con 0x7ffaf40b1570 2024-03-29T08:22:44.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffafae00640 1 -- 172.21.15.46:0/4121435804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf4163ef0 con 0x7ffaf40b1570 2024-03-29T08:22:44.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffafae00640 1 -- 172.21.15.46:0/4121435804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf406c060 con 0x7ffaf40b1570 2024-03-29T08:22:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.502+0000 7ffaf0c00640 1 -- 172.21.15.46:0/4121435804 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ffadc013410 con 0x7ffaf40b1570 2024-03-29T08:22:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.506+0000 7ffaf0c00640 1 --2- 172.21.15.46:0/4121435804 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffad00666c0 0x7ffad0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.506+0000 7ffaf0c00640 1 -- 172.21.15.46:0/4121435804 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffadc083ed0 con 0x7ffaf40b1570 2024-03-29T08:22:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.506+0000 7ffaf3e00640 1 --2- 172.21.15.46:0/4121435804 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffad00666c0 0x7ffad0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:44.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.506+0000 7ffaf3e00640 1 --2- 172.21.15.46:0/4121435804 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffad00666c0 0x7ffad0068b80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffae4000f70 tx=0x7ffae4007b80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:44.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.506+0000 7ffaf0c00640 1 -- 172.21.15.46:0/4121435804 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ffaf406c060 con 0x7ffaf40b1570 2024-03-29T08:22:44.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:44.702+0000 7ffafae00640 1 -- 172.21.15.46:0/4121435804 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffaf415d830 con 0x7ffaf40b1570 2024-03-29T08:22:45.132 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-03-29T08:22:45.130+0000 7fded1000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:22:45.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.614+0000 7ffaf0c00640 1 -- 172.21.15.46:0/4121435804 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v41) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffadc0512e0 con 0x7ffaf40b1570 2024-03-29T08:22:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.618+0000 7ffafae00640 1 -- 172.21.15.46:0/4121435804 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffad00666c0 msgr2=0x7ffad0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.618+0000 7ffafae00640 1 --2- 172.21.15.46:0/4121435804 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffad00666c0 0x7ffad0068b80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffae4000f70 tx=0x7ffae4007b80 comp rx=0 tx=0).stop 2024-03-29T08:22:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.618+0000 7ffafae00640 1 -- 172.21.15.46:0/4121435804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf40b1570 msgr2=0x7ffaf4153c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.618+0000 7ffafae00640 1 --2- 172.21.15.46:0/4121435804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf40b1570 0x7ffaf4153c80 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00e810 tx=0x7ffadc00ecd0 comp rx=0 tx=0).stop 2024-03-29T08:22:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.618+0000 7ffafae00640 1 -- 172.21.15.46:0/4121435804 shutdown_connections 2024-03-29T08:22:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.618+0000 7ffafae00640 1 --2- 172.21.15.46:0/4121435804 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffad00666c0 0x7ffad0068b80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.618+0000 7ffafae00640 1 --2- 172.21.15.46:0/4121435804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaf415dfb0 0x7ffaf406c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.618+0000 7ffafae00640 1 --2- 172.21.15.46:0/4121435804 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaf40b1570 0x7ffaf4153c80 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.618+0000 7ffafae00640 1 --2- 172.21.15.46:0/4121435804 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffaf4098a40 0x7ffaf4154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.618+0000 7ffafae00640 1 -- 172.21.15.46:0/4121435804 >> 172.21.15.46:0/4121435804 conn(0x7ffaf40b2220 msgr2=0x7ffaf406fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.618+0000 7ffafae00640 1 -- 172.21.15.46:0/4121435804 shutdown_connections 2024-03-29T08:22:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.618+0000 7ffafae00640 1 -- 172.21.15.46:0/4121435804 wait complete. 2024-03-29T08:22:45.645 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-29T08:22:45.645 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T08:22:45.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.790+0000 7f0127a00640 1 Processor -- start 2024-03-29T08:22:45.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.790+0000 7f0127a00640 1 -- start start 2024-03-29T08:22:45.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.790+0000 7f0127a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0120142a50 0x7f01200cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:45.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.790+0000 7f0127a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f012015df90 0x7f01200cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:45.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.790+0000 7f0127a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f01200ae280 0x7f01200ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:45.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.790+0000 7f0127a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f012015d260 con 0x7f0120142a50 2024-03-29T08:22:45.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.790+0000 7f0127a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01200aefd0 con 0x7f01200ae280 2024-03-29T08:22:45.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.790+0000 7f0127a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012015d4b0 con 0x7f012015df90 2024-03-29T08:22:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.790+0000 7f0125000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0120142a50 0x7f01200cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.790+0000 7f0125000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0120142a50 0x7f01200cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60484/0 (socket says 172.21.15.46:60484) 2024-03-29T08:22:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.790+0000 7f0125000640 1 -- 172.21.15.46:0/4036428256 learned_addr learned my addr 172.21.15.46:0/4036428256 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.790+0000 7f0125000640 1 -- 172.21.15.46:0/4036428256 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f012015df90 msgr2=0x7f01200cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.790+0000 7f0125000640 1 --2- 172.21.15.46:0/4036428256 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f012015df90 0x7f01200cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.790+0000 7f0125000640 1 -- 172.21.15.46:0/4036428256 >> v2:172.21.15.73:3300/0 conn(0x7f01200ae280 msgr2=0x7f01200ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.790+0000 7f0125000640 1 --2- 172.21.15.46:0/4036428256 >> v2:172.21.15.73:3300/0 conn(0x7f01200ae280 0x7f01200ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.790+0000 7f0125000640 1 -- 172.21.15.46:0/4036428256 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012015d790 con 0x7f0120142a50 2024-03-29T08:22:45.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.794+0000 7f0125000640 1 --2- 172.21.15.46:0/4036428256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0120142a50 0x7f01200cb790 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f012006e580 tx=0x7f011401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94959edd838da87b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:45.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.794+0000 7f011f400640 1 -- 172.21.15.46:0/4036428256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01140ad660 con 0x7f0120142a50 2024-03-29T08:22:45.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.794+0000 7f011f400640 1 -- 172.21.15.46:0/4036428256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01140ad7c0 con 0x7f0120142a50 2024-03-29T08:22:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.794+0000 7f011f400640 1 -- 172.21.15.46:0/4036428256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01140adb50 con 0x7f0120142a50 2024-03-29T08:22:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.794+0000 7f0127a00640 1 -- 172.21.15.46:0/4036428256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0120142a50 msgr2=0x7f01200cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.794+0000 7f0127a00640 1 --2- 172.21.15.46:0/4036428256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0120142a50 0x7f01200cb790 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f012006e580 tx=0x7f011401c490 comp rx=0 tx=0).stop 2024-03-29T08:22:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.794+0000 7f0127a00640 1 -- 172.21.15.46:0/4036428256 shutdown_connections 2024-03-29T08:22:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.794+0000 7f0127a00640 1 --2- 172.21.15.46:0/4036428256 >> v2:172.21.15.73:3300/0 conn(0x7f01200ae280 0x7f01200ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:45.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.794+0000 7f0127a00640 1 --2- 172.21.15.46:0/4036428256 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f012015df90 0x7f01200cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:45.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.794+0000 7f0127a00640 1 --2- 172.21.15.46:0/4036428256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0120142a50 0x7f01200cb790 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:45.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.794+0000 7f0127a00640 1 -- 172.21.15.46:0/4036428256 >> 172.21.15.46:0/4036428256 conn(0x7f012006a390 msgr2=0x7f01200cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:45.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.794+0000 7f0127a00640 1 -- 172.21.15.46:0/4036428256 shutdown_connections 2024-03-29T08:22:45.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.794+0000 7f0127a00640 1 -- 172.21.15.46:0/4036428256 wait complete. 2024-03-29T08:22:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0127a00640 1 Processor -- start 2024-03-29T08:22:45.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0127a00640 1 -- start start 2024-03-29T08:22:45.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0127a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01200ae280 0x7f01201f5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:45.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0127a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0120142a50 0x7f01201f4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:45.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0127a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f012015df90 0x7f012015a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:45.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0127a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01201f6110 con 0x7f012015df90 2024-03-29T08:22:45.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0127a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01201f5090 con 0x7f01200ae280 2024-03-29T08:22:45.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0127a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0120140b40 con 0x7f0120142a50 2024-03-29T08:22:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0125a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f012015df90 0x7f012015a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0125a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f012015df90 0x7f012015a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60494/0 (socket says 172.21.15.46:60494) 2024-03-29T08:22:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0125a00640 1 -- 172.21.15.46:0/3649438569 learned_addr learned my addr 172.21.15.46:0/3649438569 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f011fe00640 1 --2- 172.21.15.46:0/3649438569 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0120142a50 0x7f01201f4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0125000640 1 --2- 172.21.15.46:0/3649438569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01200ae280 0x7f01201f5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0125a00640 1 -- 172.21.15.46:0/3649438569 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0120142a50 msgr2=0x7f01201f4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0125a00640 1 --2- 172.21.15.46:0/3649438569 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0120142a50 0x7f01201f4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0125a00640 1 -- 172.21.15.46:0/3649438569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01200ae280 msgr2=0x7f01201f5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0125a00640 1 --2- 172.21.15.46:0/3649438569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01200ae280 0x7f01201f5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0125a00640 1 -- 172.21.15.46:0/3649438569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01140094d0 con 0x7f012015df90 2024-03-29T08:22:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0125a00640 1 --2- 172.21.15.46:0/3649438569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f012015df90 0x7f012015a1b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f011800b9a0 tx=0x7f011800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f011cc00640 1 -- 172.21.15.46:0/3649438569 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011800cba0 con 0x7f012015df90 2024-03-29T08:22:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f011cc00640 1 -- 172.21.15.46:0/3649438569 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011800cd20 con 0x7f012015df90 2024-03-29T08:22:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f011cc00640 1 -- 172.21.15.46:0/3649438569 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0118014680 con 0x7f012015df90 2024-03-29T08:22:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.798+0000 7f0127a00640 1 -- 172.21.15.46:0/3649438569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f012013ebc0 con 0x7f012015df90 2024-03-29T08:22:45.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.802+0000 7f0127a00640 1 -- 172.21.15.46:0/3649438569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012013f3e0 con 0x7f012015df90 2024-03-29T08:22:45.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.802+0000 7f0127a00640 1 -- 172.21.15.46:0/3649438569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0120071a70 con 0x7f012015df90 2024-03-29T08:22:45.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.802+0000 7f011cc00640 1 -- 172.21.15.46:0/3649438569 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f011801d3b0 con 0x7f012015df90 2024-03-29T08:22:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.802+0000 7f011cc00640 1 --2- 172.21.15.46:0/3649438569 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0100066690 0x7f0100068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.802+0000 7f011cc00640 1 -- 172.21.15.46:0/3649438569 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01180840d0 con 0x7f012015df90 2024-03-29T08:22:45.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.802+0000 7f0125000640 1 --2- 172.21.15.46:0/3649438569 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0100066690 0x7f0100068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:45.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.802+0000 7f0125000640 1 --2- 172.21.15.46:0/3649438569 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0100066690 0x7f0100068b50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f01140af000 tx=0x7f01140a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:45.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:45.806+0000 7f011cc00640 1 -- 172.21.15.46:0/3649438569 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0118051450 con 0x7f012015df90 2024-03-29T08:22:46.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.002+0000 7f0127a00640 1 -- 172.21.15.46:0/3649438569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0120159f90 con 0x7f012015df90 2024-03-29T08:22:46.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.622+0000 7f011cc00640 1 -- 172.21.15.46:0/3649438569 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v42) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0118054a70 con 0x7f012015df90 2024-03-29T08:22:46.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.622+0000 7f0127a00640 1 -- 172.21.15.46:0/3649438569 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0100066690 msgr2=0x7f0100068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:46.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.622+0000 7f0127a00640 1 --2- 172.21.15.46:0/3649438569 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0100066690 0x7f0100068b50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f01140af000 tx=0x7f01140a7000 comp rx=0 tx=0).stop 2024-03-29T08:22:46.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.622+0000 7f0127a00640 1 -- 172.21.15.46:0/3649438569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f012015df90 msgr2=0x7f012015a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:46.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.622+0000 7f0127a00640 1 --2- 172.21.15.46:0/3649438569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f012015df90 0x7f012015a1b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f011800b9a0 tx=0x7f011800be60 comp rx=0 tx=0).stop 2024-03-29T08:22:46.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.626+0000 7f0127a00640 1 -- 172.21.15.46:0/3649438569 shutdown_connections 2024-03-29T08:22:46.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.626+0000 7f0127a00640 1 --2- 172.21.15.46:0/3649438569 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0100066690 0x7f0100068b50 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:46.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.626+0000 7f0127a00640 1 --2- 172.21.15.46:0/3649438569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f012015df90 0x7f012015a1b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:46.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.626+0000 7f0127a00640 1 --2- 172.21.15.46:0/3649438569 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0120142a50 0x7f01201f4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:46.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.626+0000 7f0127a00640 1 --2- 172.21.15.46:0/3649438569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01200ae280 0x7f01201f5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:46.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.626+0000 7f0127a00640 1 -- 172.21.15.46:0/3649438569 >> 172.21.15.46:0/3649438569 conn(0x7f012006a390 msgr2=0x7f01200cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:46.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.626+0000 7f0127a00640 1 -- 172.21.15.46:0/3649438569 shutdown_connections 2024-03-29T08:22:46.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.626+0000 7f0127a00640 1 -- 172.21.15.46:0/3649438569 wait complete. 2024-03-29T08:22:46.649 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T08:22:46.649 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:22:46.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.786+0000 7face8c00640 1 Processor -- start 2024-03-29T08:22:46.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.786+0000 7face8c00640 1 -- start start 2024-03-29T08:22:46.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face8c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7face41593d0 0x7face406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:46.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face8c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7face415d230 0x7face415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:46.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face8c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face406d130 con 0x7face415dbd0 2024-03-29T08:22:46.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face8c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face415b8f0 con 0x7face415d230 2024-03-29T08:22:46.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face8c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face415ad50 con 0x7face41593d0 2024-03-29T08:22:46.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face2800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7face415dbd0 0x7face415bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57524/0 (socket says 172.21.15.46:57524) 2024-03-29T08:22:46.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face2800640 1 -- 172.21.15.46:0/2080336446 learned_addr learned my addr 172.21.15.46:0/2080336446 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:46.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1e00640 1 --2- 172.21.15.46:0/2080336446 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7face41593d0 0x7face406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:46.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7facdbe00640 1 --2- 172.21.15.46:0/2080336446 >> v2:172.21.15.73:3300/0 conn(0x7face415d230 0x7face415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:46.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1400640 1 -- 172.21.15.46:0/2080336446 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805272922 0 0) 0x7face406d130 con 0x7face415dbd0 2024-03-29T08:22:46.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1400640 1 -- 172.21.15.46:0/2080336446 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7facc8003530 con 0x7face415dbd0 2024-03-29T08:22:46.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1400640 1 -- 172.21.15.46:0/2080336446 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2194361552 0 0) 0x7facc8003530 con 0x7face415dbd0 2024-03-29T08:22:46.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1400640 1 -- 172.21.15.46:0/2080336446 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7face41593d0 msgr2=0x7face406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:46.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1400640 1 --2- 172.21.15.46:0/2080336446 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7face41593d0 0x7face406cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:46.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1400640 1 -- 172.21.15.46:0/2080336446 >> v2:172.21.15.73:3300/0 conn(0x7face415d230 msgr2=0x7face415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:46.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1400640 1 --2- 172.21.15.46:0/2080336446 >> v2:172.21.15.73:3300/0 conn(0x7face415d230 0x7face415d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:46.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1400640 1 -- 172.21.15.46:0/2080336446 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face415e060 con 0x7face415dbd0 2024-03-29T08:22:46.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1400640 1 -- 172.21.15.46:0/2080336446 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7facdc001a00 con 0x7face415dbd0 2024-03-29T08:22:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1400640 1 -- 172.21.15.46:0/2080336446 >> v1:172.21.15.46:6789/0 conn(0x7face415dbd0 legacy=0x7face415bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1400640 1 --2- 172.21.15.46:0/2080336446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7facc8003980 0x7facc8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1400640 1 --2- 172.21.15.46:0/2080336446 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7face41593d0 0x7facc80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1400640 1 --2- 172.21.15.46:0/2080336446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facc8004d10 0x7facc8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1400640 1 -- 172.21.15.46:0/2080336446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face415b8f0 con 0x7facc8004d10 2024-03-29T08:22:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1400640 1 -- 172.21.15.46:0/2080336446 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face415ad50 con 0x7facc8003980 2024-03-29T08:22:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1400640 1 -- 172.21.15.46:0/2080336446 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc800b990 con 0x7face41593d0 2024-03-29T08:22:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face2800640 1 --2- 172.21.15.46:0/2080336446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facc8004d10 0x7facc8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7facdbe00640 1 --2- 172.21.15.46:0/2080336446 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7face41593d0 0x7facc80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face1e00640 1 --2- 172.21.15.46:0/2080336446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7facc8003980 0x7facc8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face2800640 1 -- 172.21.15.46:0/2080336446 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7face41593d0 msgr2=0x7facc80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face2800640 1 --2- 172.21.15.46:0/2080336446 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7face41593d0 0x7facc80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face2800640 1 -- 172.21.15.46:0/2080336446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7facc8003980 msgr2=0x7facc8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face2800640 1 --2- 172.21.15.46:0/2080336446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7facc8003980 0x7facc8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.790+0000 7face2800640 1 -- 172.21.15.46:0/2080336446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc8003530 con 0x7facc8004d10 2024-03-29T08:22:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.794+0000 7face2800640 1 --2- 172.21.15.46:0/2080336446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facc8004d10 0x7facc8005190 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7facdc00a830 tx=0x7facdc01d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3903f0cb834a7e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.794+0000 7face1400640 1 -- 172.21.15.46:0/2080336446 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc0ad630 con 0x7facc8004d10 2024-03-29T08:22:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.794+0000 7face1400640 1 -- 172.21.15.46:0/2080336446 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc0a8070 con 0x7facc8004d10 2024-03-29T08:22:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.794+0000 7face1400640 1 -- 172.21.15.46:0/2080336446 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc0ad9f0 con 0x7facc8004d10 2024-03-29T08:22:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.794+0000 7face8c00640 1 -- 172.21.15.46:0/2080336446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facc8004d10 msgr2=0x7facc8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.794+0000 7face8c00640 1 --2- 172.21.15.46:0/2080336446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7facc8004d10 0x7facc8005190 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7facdc00a830 tx=0x7facdc01d380 comp rx=0 tx=0).stop 2024-03-29T08:22:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.794+0000 7face2800640 1 -- 172.21.15.46:0/2080336446 reap_dead start 2024-03-29T08:22:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.794+0000 7face8c00640 1 -- 172.21.15.46:0/2080336446 shutdown_connections 2024-03-29T08:22:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.794+0000 7face8c00640 1 -- 172.21.15.46:0/2080336446 >> 172.21.15.46:0/2080336446 conn(0x7face4066110 msgr2=0x7face4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.794+0000 7face8c00640 1 -- 172.21.15.46:0/2080336446 shutdown_connections 2024-03-29T08:22:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.794+0000 7face8c00640 1 -- 172.21.15.46:0/2080336446 wait complete. 2024-03-29T08:22:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7face8c00640 1 Processor -- start 2024-03-29T08:22:46.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7face8c00640 1 -- start start 2024-03-29T08:22:46.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7face8c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7facc8004d10 0x7face414fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:46.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7face8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7face40cb300 0x7face4165350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:46.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7face8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7face40cb780 0x7face4165890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:46.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7face8c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face414cd70 con 0x7face40cb300 2024-03-29T08:22:46.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7face8c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face4165ef0 con 0x7face40cb780 2024-03-29T08:22:46.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7face8c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face41661d0 con 0x7facc8004d10 2024-03-29T08:22:46.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7facdbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7face40cb300 0x7face4165350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:46.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7facdbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7face40cb300 0x7face4165350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54514/0 (socket says 172.21.15.46:54514) 2024-03-29T08:22:46.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7facdbe00640 1 -- 172.21.15.46:0/568359653 learned_addr learned my addr 172.21.15.46:0/568359653 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:46.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7face2800640 1 --2- 172.21.15.46:0/568359653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7face40cb780 0x7face4165890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:46.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7face1e00640 1 --2- 172.21.15.46:0/568359653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7facc8004d10 0x7face414fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:46.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7facdbe00640 1 -- 172.21.15.46:0/568359653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7facc8004d10 msgr2=0x7face414fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:46.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7facdbe00640 1 --2- 172.21.15.46:0/568359653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7facc8004d10 0x7face414fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:46.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7facdbe00640 1 -- 172.21.15.46:0/568359653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7face40cb780 msgr2=0x7face4165890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:46.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7facdbe00640 1 --2- 172.21.15.46:0/568359653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7face40cb780 0x7face4165890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:46.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7facdbe00640 1 -- 172.21.15.46:0/568359653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face41664b0 con 0x7face40cb300 2024-03-29T08:22:46.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7face2800640 1 --2- 172.21.15.46:0/568359653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7face40cb780 0x7face4165890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:46.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7facdbe00640 1 --2- 172.21.15.46:0/568359653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7face40cb300 0x7face4165350 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7faccc011980 tx=0x7faccc011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:46.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7facda000640 1 -- 172.21.15.46:0/568359653 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc00faf0 con 0x7face40cb300 2024-03-29T08:22:46.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7facda000640 1 -- 172.21.15.46:0/568359653 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faccc00fc50 con 0x7face40cb300 2024-03-29T08:22:46.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7facda000640 1 -- 172.21.15.46:0/568359653 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faccc014440 con 0x7face40cb300 2024-03-29T08:22:46.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7face8c00640 1 -- 172.21.15.46:0/568359653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face414d010 con 0x7face40cb300 2024-03-29T08:22:46.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7face8c00640 1 -- 172.21.15.46:0/568359653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face414e510 con 0x7face40cb300 2024-03-29T08:22:46.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.798+0000 7face8c00640 1 -- 172.21.15.46:0/568359653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face4066c90 con 0x7face40cb300 2024-03-29T08:22:46.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.802+0000 7facda000640 1 -- 172.21.15.46:0/568359653 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7faccc0145a0 con 0x7face40cb300 2024-03-29T08:22:46.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.802+0000 7facda000640 1 --2- 172.21.15.46:0/568359653 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7facc0066690 0x7facc0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:46.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.802+0000 7facda000640 1 -- 172.21.15.46:0/568359653 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faccc084b00 con 0x7face40cb300 2024-03-29T08:22:46.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.802+0000 7face1e00640 1 --2- 172.21.15.46:0/568359653 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7facc0066690 0x7facc0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:46.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.802+0000 7face1e00640 1 --2- 172.21.15.46:0/568359653 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7facc0066690 0x7facc0068b50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7facd4009b20 tx=0x7facd4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:46.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.806+0000 7facda000640 1 -- 172.21.15.46:0/568359653 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7faccc086e60 con 0x7face40cb300 2024-03-29T08:22:47.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.998+0000 7face8c00640 1 -- 172.21.15.46:0/568359653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7face4158b90 con 0x7face40cb300 2024-03-29T08:22:47.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:46.998+0000 7facda000640 1 -- 172.21.15.46:0/568359653 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4727 (secure 0 0 0) 0x7faccc022020 con 0x7face40cb300 2024-03-29T08:22:47.002 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:47.003 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3456771376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3456771376},{"type":"v1","addr":"172.21.15.73:6836","nonce":3456771376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2671538025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2671538025},{"type":"v1","addr":"172.21.15.73:6837","nonce":2671538025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3196015693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3196015693},{"type":"v1","addr":"172.21.15.46:6833","nonce":3196015693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:22:43.605457+0000","modified":"2024-03-29T08:22:46.620088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/9617787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":9617787},{"type":"v1","addr":"172.21.15.46:6835","nonce":9617787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-29T08:22:47.003 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 42 2024-03-29T08:22:47.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.002+0000 7face8c00640 1 -- 172.21.15.46:0/568359653 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7facc0066690 msgr2=0x7facc0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.002+0000 7face8c00640 1 --2- 172.21.15.46:0/568359653 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7facc0066690 0x7facc0068b50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7facd4009b20 tx=0x7facd4008040 comp rx=0 tx=0).stop 2024-03-29T08:22:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.002+0000 7face8c00640 1 -- 172.21.15.46:0/568359653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7face40cb300 msgr2=0x7face4165350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.002+0000 7face8c00640 1 --2- 172.21.15.46:0/568359653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7face40cb300 0x7face4165350 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7faccc011980 tx=0x7faccc011e40 comp rx=0 tx=0).stop 2024-03-29T08:22:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.002+0000 7face8c00640 1 -- 172.21.15.46:0/568359653 shutdown_connections 2024-03-29T08:22:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.002+0000 7face8c00640 1 --2- 172.21.15.46:0/568359653 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7facc0066690 0x7facc0068b50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.002+0000 7face8c00640 1 --2- 172.21.15.46:0/568359653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7face40cb780 0x7face4165890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.002+0000 7face8c00640 1 --2- 172.21.15.46:0/568359653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7face40cb300 0x7face4165350 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.002+0000 7face8c00640 1 --2- 172.21.15.46:0/568359653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7facc8004d10 0x7face414fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.002+0000 7face8c00640 1 -- 172.21.15.46:0/568359653 >> 172.21.15.46:0/568359653 conn(0x7face4066110 msgr2=0x7face4152650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.002+0000 7face8c00640 1 -- 172.21.15.46:0/568359653 shutdown_connections 2024-03-29T08:22:47.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.002+0000 7face8c00640 1 -- 172.21.15.46:0/568359653 wait complete. 2024-03-29T08:22:47.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:22:47.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f1542400640 1 Processor -- start 2024-03-29T08:22:47.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f1542400640 1 -- start start 2024-03-29T08:22:47.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f1542400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f153c1593d0 0x7f153c06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f1542400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f153c15d230 0x7f153c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f1542400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153c06d130 con 0x7f153c15dbd0 2024-03-29T08:22:47.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f1542400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f153c15b8f0 con 0x7f153c1593d0 2024-03-29T08:22:47.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f1542400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153c15ad50 con 0x7f153c15d230 2024-03-29T08:22:47.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f153be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f153c15dbd0 0x7f153c15bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57540/0 (socket says 172.21.15.46:57540) 2024-03-29T08:22:47.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f153be00640 1 -- 172.21.15.46:0/287390296 learned_addr learned my addr 172.21.15.46:0/287390296 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:47.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f153b400640 1 --2- 172.21.15.46:0/287390296 >> v2:172.21.15.73:3300/0 conn(0x7f153c1593d0 0x7f153c06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f1533e00640 1 --2- 172.21.15.46:0/287390296 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f153c15d230 0x7f153c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f153aa00640 1 -- 172.21.15.46:0/287390296 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3241472803 0 0) 0x7f153c06d130 con 0x7f153c15dbd0 2024-03-29T08:22:47.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f153aa00640 1 -- 172.21.15.46:0/287390296 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1520003530 con 0x7f153c15dbd0 2024-03-29T08:22:47.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f153b400640 1 -- 172.21.15.46:0/287390296 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f153c15d230 msgr2=0x7f153c15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:47.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f153b400640 1 --2- 172.21.15.46:0/287390296 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f153c15d230 0x7f153c15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f153b400640 1 -- 172.21.15.46:0/287390296 >> v1:172.21.15.46:6789/0 conn(0x7f153c15dbd0 legacy=0x7f153c15bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f153b400640 1 -- 172.21.15.46:0/287390296 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c15e060 con 0x7f153c1593d0 2024-03-29T08:22:47.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.162+0000 7f1533e00640 1 --2- 172.21.15.46:0/287390296 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f153c15d230 0x7f153c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:22:47.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.166+0000 7f153b400640 1 --2- 172.21.15.46:0/287390296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f153c1593d0 0x7f153c06cbf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f152c0097c0 tx=0x7f152c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c675d6d4ab0232fc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:47.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.166+0000 7f153aa00640 1 -- 172.21.15.46:0/287390296 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c0a6040 con 0x7f153c1593d0 2024-03-29T08:22:47.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.166+0000 7f153aa00640 1 -- 172.21.15.46:0/287390296 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152c0aa600 con 0x7f153c1593d0 2024-03-29T08:22:47.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.166+0000 7f153aa00640 1 -- 172.21.15.46:0/287390296 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c0aa970 con 0x7f153c1593d0 2024-03-29T08:22:47.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.166+0000 7f1542400640 1 -- 172.21.15.46:0/287390296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f153c1593d0 msgr2=0x7f153c06cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.166+0000 7f1542400640 1 --2- 172.21.15.46:0/287390296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f153c1593d0 0x7f153c06cbf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f152c0097c0 tx=0x7f152c01c4a0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.166+0000 7f1542400640 1 -- 172.21.15.46:0/287390296 shutdown_connections 2024-03-29T08:22:47.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.166+0000 7f1542400640 1 --2- 172.21.15.46:0/287390296 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f153c15d230 0x7f153c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.166+0000 7f1542400640 1 --2- 172.21.15.46:0/287390296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f153c1593d0 0x7f153c06cbf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.166+0000 7f1542400640 1 -- 172.21.15.46:0/287390296 >> 172.21.15.46:0/287390296 conn(0x7f153c066110 msgr2=0x7f153c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:47.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.166+0000 7f1542400640 1 -- 172.21.15.46:0/287390296 shutdown_connections 2024-03-29T08:22:47.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.166+0000 7f1542400640 1 -- 172.21.15.46:0/287390296 wait complete. 2024-03-29T08:22:47.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f1542400640 1 Processor -- start 2024-03-29T08:22:47.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f1542400640 1 -- start start 2024-03-29T08:22:47.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f1542400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f153c1593d0 0x7f153c14de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f1542400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f153c15d230 0x7f153c0acb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f1542400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f153c15dbd0 0x7f153c14c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f1542400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c0ab8b0 con 0x7f153c15dbd0 2024-03-29T08:22:47.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f1542400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c14c740 con 0x7f153c15d230 2024-03-29T08:22:47.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f1542400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153c153360 con 0x7f153c1593d0 2024-03-29T08:22:47.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f153be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f153c15dbd0 0x7f153c14c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f153be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f153c15dbd0 0x7f153c14c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54518/0 (socket says 172.21.15.46:54518) 2024-03-29T08:22:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f153be00640 1 -- 172.21.15.46:0/4161801849 learned_addr learned my addr 172.21.15.46:0/4161801849 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f153b400640 1 --2- 172.21.15.46:0/4161801849 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f153c1593d0 0x7f153c14de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f1533e00640 1 --2- 172.21.15.46:0/4161801849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f153c15d230 0x7f153c0acb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f153be00640 1 -- 172.21.15.46:0/4161801849 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f153c1593d0 msgr2=0x7f153c14de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f153be00640 1 --2- 172.21.15.46:0/4161801849 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f153c1593d0 0x7f153c14de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f153be00640 1 -- 172.21.15.46:0/4161801849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f153c15d230 msgr2=0x7f153c0acb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f153be00640 1 --2- 172.21.15.46:0/4161801849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f153c15d230 0x7f153c0acb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f153be00640 1 -- 172.21.15.46:0/4161801849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1524009420 con 0x7f153c15dbd0 2024-03-29T08:22:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f153be00640 1 --2- 172.21.15.46:0/4161801849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f153c15dbd0 0x7f153c14c200 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f153400cb40 tx=0x7f153400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f1533400640 1 -- 172.21.15.46:0/4161801849 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153400aca0 con 0x7f153c15dbd0 2024-03-29T08:22:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f1533400640 1 -- 172.21.15.46:0/4161801849 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153400ae00 con 0x7f153c15dbd0 2024-03-29T08:22:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f1542400640 1 -- 172.21.15.46:0/4161801849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152c009420 con 0x7f153c15dbd0 2024-03-29T08:22:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f1533400640 1 -- 172.21.15.46:0/4161801849 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1534012620 con 0x7f153c15dbd0 2024-03-29T08:22:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f1542400640 1 -- 172.21.15.46:0/4161801849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153c0ad4d0 con 0x7f153c15dbd0 2024-03-29T08:22:47.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.170+0000 7f1542400640 1 -- 172.21.15.46:0/4161801849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153c066c90 con 0x7f153c15dbd0 2024-03-29T08:22:47.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.174+0000 7f1533400640 1 -- 172.21.15.46:0/4161801849 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1534007070 con 0x7f153c15dbd0 2024-03-29T08:22:47.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.174+0000 7f1533400640 1 --2- 172.21.15.46:0/4161801849 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15180666c0 0x7f1518068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.174+0000 7f1533400640 1 -- 172.21.15.46:0/4161801849 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1534084100 con 0x7f153c15dbd0 2024-03-29T08:22:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.174+0000 7f153b400640 1 --2- 172.21.15.46:0/4161801849 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15180666c0 0x7f1518068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.174+0000 7f153b400640 1 --2- 172.21.15.46:0/4161801849 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15180666c0 0x7f1518068b80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f152c0ae000 tx=0x7f152c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:47.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.178+0000 7f1533400640 1 -- 172.21.15.46:0/4161801849 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1534051510 con 0x7f153c15dbd0 2024-03-29T08:22:47.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.342+0000 7f1542400640 1 -- 172.21.15.46:0/4161801849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f153c06d3d0 con 0x7f153c15dbd0 2024-03-29T08:22:47.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.342+0000 7f1533400640 1 -- 172.21.15.46:0/4161801849 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+15992 (secure 0 0 0) 0x7f1534054b30 con 0x7f153c15dbd0 2024-03-29T08:22:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:47.345 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":77,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:22:44.608017+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:22:42.147364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-29T08:22:42.602168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:22:47.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.342+0000 7f1542400640 1 -- 172.21.15.46:0/4161801849 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15180666c0 msgr2=0x7f1518068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.342+0000 7f1542400640 1 --2- 172.21.15.46:0/4161801849 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15180666c0 0x7f1518068b80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f152c0ae000 tx=0x7f152c008040 comp rx=0 tx=0).stop 2024-03-29T08:22:47.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.342+0000 7f1542400640 1 -- 172.21.15.46:0/4161801849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f153c15dbd0 msgr2=0x7f153c14c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.342+0000 7f1542400640 1 --2- 172.21.15.46:0/4161801849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f153c15dbd0 0x7f153c14c200 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f153400cb40 tx=0x7f153400a060 comp rx=0 tx=0).stop 2024-03-29T08:22:47.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.342+0000 7f1542400640 1 -- 172.21.15.46:0/4161801849 shutdown_connections 2024-03-29T08:22:47.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.342+0000 7f1542400640 1 --2- 172.21.15.46:0/4161801849 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15180666c0 0x7f1518068b80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.342+0000 7f1542400640 1 --2- 172.21.15.46:0/4161801849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f153c15d230 0x7f153c0acb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.342+0000 7f1542400640 1 --2- 172.21.15.46:0/4161801849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f153c15dbd0 0x7f153c14c200 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.342+0000 7f1542400640 1 --2- 172.21.15.46:0/4161801849 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f153c1593d0 0x7f153c14de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.342+0000 7f1542400640 1 -- 172.21.15.46:0/4161801849 >> 172.21.15.46:0/4161801849 conn(0x7f153c066110 msgr2=0x7f153c1583a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:47.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.342+0000 7f1542400640 1 -- 172.21.15.46:0/4161801849 shutdown_connections 2024-03-29T08:22:47.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.342+0000 7f1542400640 1 -- 172.21.15.46:0/4161801849 wait complete. 2024-03-29T08:22:47.367 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T08:22:47.367 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:22:47.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.502+0000 7f1c61c00640 1 Processor -- start 2024-03-29T08:22:47.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.502+0000 7f1c61c00640 1 -- start start 2024-03-29T08:22:47.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.502+0000 7f1c61c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1c5c15cd90 0x7f1c5c06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.502+0000 7f1c61c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1c5c15d940 0x7f1c5c1597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.502+0000 7f1c61c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5c06d130 con 0x7f1c5c1593d0 2024-03-29T08:22:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.502+0000 7f1c61c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5c15b8f0 con 0x7f1c5c15d940 2024-03-29T08:22:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.502+0000 7f1c61c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5c15ad50 con 0x7f1c5c15cd90 2024-03-29T08:22:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.506+0000 7f1c5b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1c5c15d940 0x7f1c5c1597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.506+0000 7f1c5aa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1c5c1593d0 0x7f1c5c15d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57554/0 (socket says 172.21.15.46:57554) 2024-03-29T08:22:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.506+0000 7f1c5aa00640 1 -- 172.21.15.46:0/3183048619 learned_addr learned my addr 172.21.15.46:0/3183048619 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.506+0000 7f1c52000640 1 --2- 172.21.15.46:0/3183048619 >> v2:172.21.15.73:3300/0 conn(0x7f1c5c15cd90 0x7f1c5c06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.506+0000 7f1c5b400640 1 -- 172.21.15.46:0/3183048619 >> v2:172.21.15.73:3300/0 conn(0x7f1c5c15cd90 msgr2=0x7f1c5c06cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.506+0000 7f1c5b400640 1 --2- 172.21.15.46:0/3183048619 >> v2:172.21.15.73:3300/0 conn(0x7f1c5c15cd90 0x7f1c5c06cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.506+0000 7f1c5b400640 1 -- 172.21.15.46:0/3183048619 >> v1:172.21.15.46:6789/0 conn(0x7f1c5c1593d0 legacy=0x7f1c5c15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.506+0000 7f1c5b400640 1 -- 172.21.15.46:0/3183048619 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c15afc0 con 0x7f1c5c15d940 2024-03-29T08:22:47.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.506+0000 7f1c5b400640 1 --2- 172.21.15.46:0/3183048619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15d940 0x7f1c5c1597b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0cf180 tx=0x7f1c5401c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da327d66e8d9dbd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.506+0000 7f1c5a000640 1 -- 172.21.15.46:0/3183048619 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c540a5800 con 0x7f1c5c15d940 2024-03-29T08:22:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.506+0000 7f1c5a000640 1 -- 172.21.15.46:0/3183048619 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c540a5960 con 0x7f1c5c15d940 2024-03-29T08:22:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.506+0000 7f1c5a000640 1 -- 172.21.15.46:0/3183048619 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c540a5cf0 con 0x7f1c5c15d940 2024-03-29T08:22:47.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c61c00640 1 -- 172.21.15.46:0/3183048619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15d940 msgr2=0x7f1c5c1597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c61c00640 1 --2- 172.21.15.46:0/3183048619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15d940 0x7f1c5c1597b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0cf180 tx=0x7f1c5401c7c0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c61c00640 1 -- 172.21.15.46:0/3183048619 shutdown_connections 2024-03-29T08:22:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c61c00640 1 --2- 172.21.15.46:0/3183048619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15d940 0x7f1c5c1597b0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c61c00640 1 --2- 172.21.15.46:0/3183048619 >> v2:172.21.15.73:3300/0 conn(0x7f1c5c15cd90 0x7f1c5c06cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c61c00640 1 -- 172.21.15.46:0/3183048619 >> 172.21.15.46:0/3183048619 conn(0x7f1c5c066110 msgr2=0x7f1c5c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c61c00640 1 -- 172.21.15.46:0/3183048619 shutdown_connections 2024-03-29T08:22:47.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c61c00640 1 -- 172.21.15.46:0/3183048619 wait complete. 2024-03-29T08:22:47.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c61c00640 1 Processor -- start 2024-03-29T08:22:47.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c61c00640 1 -- start start 2024-03-29T08:22:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c61c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c5c1593d0 0x7f1c5c14cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c61c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c5c15cd90 0x7f1c5c1580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c61c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15d940 0x7f1c5c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c61c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c151660 con 0x7f1c5c15d940 2024-03-29T08:22:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c61c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c151c20 con 0x7f1c5c15cd90 2024-03-29T08:22:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c61c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c1632e0 con 0x7f1c5c1593d0 2024-03-29T08:22:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c5b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15d940 0x7f1c5c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c5b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15d940 0x7f1c5c0ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54542/0 (socket says 172.21.15.46:54542) 2024-03-29T08:22:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c5b400640 1 -- 172.21.15.46:0/2694419177 learned_addr learned my addr 172.21.15.46:0/2694419177 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:47.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c5aa00640 1 --2- 172.21.15.46:0/2694419177 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c5c1593d0 0x7f1c5c14cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c52000640 1 --2- 172.21.15.46:0/2694419177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c5c15cd90 0x7f1c5c1580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c5b400640 1 -- 172.21.15.46:0/2694419177 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c5c1593d0 msgr2=0x7f1c5c14cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c5b400640 1 --2- 172.21.15.46:0/2694419177 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c5c1593d0 0x7f1c5c14cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c5b400640 1 -- 172.21.15.46:0/2694419177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c5c15cd90 msgr2=0x7f1c5c1580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c5b400640 1 --2- 172.21.15.46:0/2694419177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c5c15cd90 0x7f1c5c1580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c5b400640 1 -- 172.21.15.46:0/2694419177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c54009a70 con 0x7f1c5c15d940 2024-03-29T08:22:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c52000640 1 --2- 172.21.15.46:0/2694419177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c5c15cd90 0x7f1c5c1580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:47.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.510+0000 7f1c5b400640 1 --2- 172.21.15.46:0/2694419177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15d940 0x7f1c5c0ae8b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1c5401a3f0 tx=0x7f1c54019560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.514+0000 7f1c53400640 1 -- 172.21.15.46:0/2694419177 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c540ad640 con 0x7f1c5c15d940 2024-03-29T08:22:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.514+0000 7f1c53400640 1 -- 172.21.15.46:0/2694419177 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c540ad7a0 con 0x7f1c5c15d940 2024-03-29T08:22:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.514+0000 7f1c53400640 1 -- 172.21.15.46:0/2694419177 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c540adb50 con 0x7f1c5c15d940 2024-03-29T08:22:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.514+0000 7f1c61c00640 1 -- 172.21.15.46:0/2694419177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c1635c0 con 0x7f1c5c15d940 2024-03-29T08:22:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.514+0000 7f1c61c00640 1 -- 172.21.15.46:0/2694419177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c14d300 con 0x7f1c5c15d940 2024-03-29T08:22:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.514+0000 7f1c61c00640 1 -- 172.21.15.46:0/2694419177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5c0cc3b0 con 0x7f1c5c15d940 2024-03-29T08:22:47.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.514+0000 7f1c53400640 1 -- 172.21.15.46:0/2694419177 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1c540b4050 con 0x7f1c5c15d940 2024-03-29T08:22:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.514+0000 7f1c53400640 1 --2- 172.21.15.46:0/2694419177 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c38066690 0x7f1c38068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.514+0000 7f1c53400640 1 -- 172.21.15.46:0/2694419177 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c5411f250 con 0x7f1c5c15d940 2024-03-29T08:22:47.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.514+0000 7f1c5aa00640 1 --2- 172.21.15.46:0/2694419177 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c38066690 0x7f1c38068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.518+0000 7f1c5aa00640 1 --2- 172.21.15.46:0/2694419177 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c38066690 0x7f1c38068b50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c000f70 tx=0x7f1c4c007e10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:47.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.518+0000 7f1c53400640 1 -- 172.21.15.46:0/2694419177 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1c540ec5e0 con 0x7f1c5c15d940 2024-03-29T08:22:47.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.678+0000 7f1c61c00640 1 -- 172.21.15.46:0/2694419177 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1c5c066c90 con 0x7f1c38066690 2024-03-29T08:22:47.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.690+0000 7f1c53400640 1 -- 172.21.15.46:0/2694419177 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289360 (secure 0 0 0) 0x7f1c5c066c90 con 0x7f1c38066690 2024-03-29T08:22:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:47.697 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:22:47.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.694+0000 7f1c61c00640 1 -- 172.21.15.46:0/2694419177 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c38066690 msgr2=0x7f1c38068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.694+0000 7f1c61c00640 1 --2- 172.21.15.46:0/2694419177 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c38066690 0x7f1c38068b50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c000f70 tx=0x7f1c4c007e10 comp rx=0 tx=0).stop 2024-03-29T08:22:47.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.694+0000 7f1c61c00640 1 -- 172.21.15.46:0/2694419177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15d940 msgr2=0x7f1c5c0ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.694+0000 7f1c61c00640 1 --2- 172.21.15.46:0/2694419177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15d940 0x7f1c5c0ae8b0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1c5401a3f0 tx=0x7f1c54019560 comp rx=0 tx=0).stop 2024-03-29T08:22:47.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.694+0000 7f1c61c00640 1 -- 172.21.15.46:0/2694419177 shutdown_connections 2024-03-29T08:22:47.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.694+0000 7f1c61c00640 1 --2- 172.21.15.46:0/2694419177 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1c38066690 0x7f1c38068b50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.694+0000 7f1c61c00640 1 --2- 172.21.15.46:0/2694419177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c5c15cd90 0x7f1c5c1580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.694+0000 7f1c61c00640 1 --2- 172.21.15.46:0/2694419177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c5c15d940 0x7f1c5c0ae8b0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.694+0000 7f1c61c00640 1 --2- 172.21.15.46:0/2694419177 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1c5c1593d0 0x7f1c5c14cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.694+0000 7f1c61c00640 1 -- 172.21.15.46:0/2694419177 >> 172.21.15.46:0/2694419177 conn(0x7f1c5c066110 msgr2=0x7f1c5c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:47.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.694+0000 7f1c61c00640 1 -- 172.21.15.46:0/2694419177 shutdown_connections 2024-03-29T08:22:47.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.694+0000 7f1c61c00640 1 -- 172.21.15.46:0/2694419177 wait complete. 2024-03-29T08:22:47.715 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-03-29T08:22:46.108607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249228,"kb_used_data":8452,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754725492,"statfs":{"total":773094113280,"available":772838903808,"internally_reserved":0,"allocated":8654848,"data_stored":4315688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":246402617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001056"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617012+0000","last_change":"2024-03-29T08:22:43.633225+0000","last_active":"2024-03-29T08:22:44.617012+0000","last_peered":"2024-03-29T08:22:44.617012+0000","last_clean":"2024-03-29T08:22:44.617012+0000","last_became_active":"2024-03-29T08:22:43.632973+0000","last_became_peered":"2024-03-29T08:22:43.632973+0000","last_unstale":"2024-03-29T08:22:44.617012+0000","last_undegraded":"2024-03-29T08:22:44.617012+0000","last_fullsized":"2024-03-29T08:22:44.617012+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:25:25.960847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805627+0000","last_change":"2024-03-29T08:22:43.629895+0000","last_active":"2024-03-29T08:22:44.805627+0000","last_peered":"2024-03-29T08:22:44.805627+0000","last_clean":"2024-03-29T08:22:44.805627+0000","last_became_active":"2024-03-29T08:22:43.629559+0000","last_became_peered":"2024-03-29T08:22:43.629559+0000","last_unstale":"2024-03-29T08:22:44.805627+0000","last_undegraded":"2024-03-29T08:22:44.805627+0000","last_fullsized":"2024-03-29T08:22:44.805627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:02.860218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806460+0000","last_change":"2024-03-29T08:22:43.642933+0000","last_active":"2024-03-29T08:22:44.806460+0000","last_peered":"2024-03-29T08:22:44.806460+0000","last_clean":"2024-03-29T08:22:44.806460+0000","last_became_active":"2024-03-29T08:22:43.642635+0000","last_became_peered":"2024-03-29T08:22:43.642635+0000","last_unstale":"2024-03-29T08:22:44.806460+0000","last_undegraded":"2024-03-29T08:22:44.806460+0000","last_fullsized":"2024-03-29T08:22:44.806460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:48.131948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806561+0000","last_change":"2024-03-29T08:22:43.643005+0000","last_active":"2024-03-29T08:22:44.806561+0000","last_peered":"2024-03-29T08:22:44.806561+0000","last_clean":"2024-03-29T08:22:44.806561+0000","last_became_active":"2024-03-29T08:22:43.642661+0000","last_became_peered":"2024-03-29T08:22:43.642661+0000","last_unstale":"2024-03-29T08:22:44.806561+0000","last_undegraded":"2024-03-29T08:22:44.806561+0000","last_fullsized":"2024-03-29T08:22:44.806561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:00:32.361795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.628169+0000","last_change":"2024-03-29T08:22:43.619735+0000","last_active":"2024-03-29T08:22:43.628169+0000","last_peered":"2024-03-29T08:22:43.628169+0000","last_clean":"2024-03-29T08:22:43.628169+0000","last_became_active":"2024-03-29T08:22:43.619437+0000","last_became_peered":"2024-03-29T08:22:43.619437+0000","last_unstale":"2024-03-29T08:22:43.628169+0000","last_undegraded":"2024-03-29T08:22:43.628169+0000","last_fullsized":"2024-03-29T08:22:43.628169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:58:46.838515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615649+0000","last_change":"2024-03-29T08:22:43.631163+0000","last_active":"2024-03-29T08:22:44.615649+0000","last_peered":"2024-03-29T08:22:44.615649+0000","last_clean":"2024-03-29T08:22:44.615649+0000","last_became_active":"2024-03-29T08:22:43.630891+0000","last_became_peered":"2024-03-29T08:22:43.630891+0000","last_unstale":"2024-03-29T08:22:44.615649+0000","last_undegraded":"2024-03-29T08:22:44.615649+0000","last_fullsized":"2024-03-29T08:22:44.615649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:52.022173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805402+0000","last_change":"2024-03-29T08:22:43.639229+0000","last_active":"2024-03-29T08:22:44.805402+0000","last_peered":"2024-03-29T08:22:44.805402+0000","last_clean":"2024-03-29T08:22:44.805402+0000","last_became_active":"2024-03-29T08:22:43.638986+0000","last_became_peered":"2024-03-29T08:22:43.638986+0000","last_unstale":"2024-03-29T08:22:44.805402+0000","last_undegraded":"2024-03-29T08:22:44.805402+0000","last_fullsized":"2024-03-29T08:22:44.805402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:59.296230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616317+0000","last_change":"2024-03-29T08:22:43.626805+0000","last_active":"2024-03-29T08:22:44.616317+0000","last_peered":"2024-03-29T08:22:44.616317+0000","last_clean":"2024-03-29T08:22:44.616317+0000","last_became_active":"2024-03-29T08:22:43.626525+0000","last_became_peered":"2024-03-29T08:22:43.626525+0000","last_unstale":"2024-03-29T08:22:44.616317+0000","last_undegraded":"2024-03-29T08:22:44.616317+0000","last_fullsized":"2024-03-29T08:22:44.616317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:33:05.663906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806898+0000","last_change":"2024-03-29T08:22:43.626927+0000","last_active":"2024-03-29T08:22:44.806898+0000","last_peered":"2024-03-29T08:22:44.806898+0000","last_clean":"2024-03-29T08:22:44.806898+0000","last_became_active":"2024-03-29T08:22:43.626679+0000","last_became_peered":"2024-03-29T08:22:43.626679+0000","last_unstale":"2024-03-29T08:22:44.806898+0000","last_undegraded":"2024-03-29T08:22:44.806898+0000","last_fullsized":"2024-03-29T08:22:44.806898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:27:12.482046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805938+0000","last_change":"2024-03-29T08:22:43.637486+0000","last_active":"2024-03-29T08:22:44.805938+0000","last_peered":"2024-03-29T08:22:44.805938+0000","last_clean":"2024-03-29T08:22:44.805938+0000","last_became_active":"2024-03-29T08:22:43.637013+0000","last_became_peered":"2024-03-29T08:22:43.637013+0000","last_unstale":"2024-03-29T08:22:44.805938+0000","last_undegraded":"2024-03-29T08:22:44.805938+0000","last_fullsized":"2024-03-29T08:22:44.805938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:25:32.682463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616821+0000","last_change":"2024-03-29T08:22:43.632983+0000","last_active":"2024-03-29T08:22:44.616821+0000","last_peered":"2024-03-29T08:22:44.616821+0000","last_clean":"2024-03-29T08:22:44.616821+0000","last_became_active":"2024-03-29T08:22:43.632646+0000","last_became_peered":"2024-03-29T08:22:43.632646+0000","last_unstale":"2024-03-29T08:22:44.616821+0000","last_undegraded":"2024-03-29T08:22:44.616821+0000","last_fullsized":"2024-03-29T08:22:44.616821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:51:19.001329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617961+0000","last_change":"2024-03-29T08:22:43.631539+0000","last_active":"2024-03-29T08:22:44.617961+0000","last_peered":"2024-03-29T08:22:44.617961+0000","last_clean":"2024-03-29T08:22:44.617961+0000","last_became_active":"2024-03-29T08:22:43.631243+0000","last_became_peered":"2024-03-29T08:22:43.631243+0000","last_unstale":"2024-03-29T08:22:44.617961+0000","last_undegraded":"2024-03-29T08:22:44.617961+0000","last_fullsized":"2024-03-29T08:22:44.617961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:34:54.150064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806665+0000","last_change":"2024-03-29T08:22:43.631647+0000","last_active":"2024-03-29T08:22:44.806665+0000","last_peered":"2024-03-29T08:22:44.806665+0000","last_clean":"2024-03-29T08:22:44.806665+0000","last_became_active":"2024-03-29T08:22:43.631176+0000","last_became_peered":"2024-03-29T08:22:43.631176+0000","last_unstale":"2024-03-29T08:22:44.806665+0000","last_undegraded":"2024-03-29T08:22:44.806665+0000","last_fullsized":"2024-03-29T08:22:44.806665+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:16:06.248835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617737+0000","last_change":"2024-03-29T08:22:43.639032+0000","last_active":"2024-03-29T08:22:44.617737+0000","last_peered":"2024-03-29T08:22:44.617737+0000","last_clean":"2024-03-29T08:22:44.617737+0000","last_became_active":"2024-03-29T08:22:43.638727+0000","last_became_peered":"2024-03-29T08:22:43.638727+0000","last_unstale":"2024-03-29T08:22:44.617737+0000","last_undegraded":"2024-03-29T08:22:44.617737+0000","last_fullsized":"2024-03-29T08:22:44.617737+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:08.118565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617792+0000","last_change":"2024-03-29T08:22:43.632746+0000","last_active":"2024-03-29T08:22:44.617792+0000","last_peered":"2024-03-29T08:22:44.617792+0000","last_clean":"2024-03-29T08:22:44.617792+0000","last_became_active":"2024-03-29T08:22:43.632424+0000","last_became_peered":"2024-03-29T08:22:43.632424+0000","last_unstale":"2024-03-29T08:22:44.617792+0000","last_undegraded":"2024-03-29T08:22:44.617792+0000","last_fullsized":"2024-03-29T08:22:44.617792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:55:24.078237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616344+0000","last_change":"2024-03-29T08:22:43.626771+0000","last_active":"2024-03-29T08:22:44.616344+0000","last_peered":"2024-03-29T08:22:44.616344+0000","last_clean":"2024-03-29T08:22:44.616344+0000","last_became_active":"2024-03-29T08:22:43.626161+0000","last_became_peered":"2024-03-29T08:22:43.626161+0000","last_unstale":"2024-03-29T08:22:44.616344+0000","last_undegraded":"2024-03-29T08:22:44.616344+0000","last_fullsized":"2024-03-29T08:22:44.616344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:22:34.783161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.632632+0000","last_change":"2024-03-29T08:22:43.624629+0000","last_active":"2024-03-29T08:22:43.632632+0000","last_peered":"2024-03-29T08:22:43.632632+0000","last_clean":"2024-03-29T08:22:43.632632+0000","last_became_active":"2024-03-29T08:22:43.624368+0000","last_became_peered":"2024-03-29T08:22:43.624368+0000","last_unstale":"2024-03-29T08:22:43.632632+0000","last_undegraded":"2024-03-29T08:22:43.632632+0000","last_fullsized":"2024-03-29T08:22:43.632632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:13:33.671724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.636068+0000","last_change":"2024-03-29T08:22:43.630276+0000","last_active":"2024-03-29T08:22:43.636068+0000","last_peered":"2024-03-29T08:22:43.636068+0000","last_clean":"2024-03-29T08:22:43.636068+0000","last_became_active":"2024-03-29T08:22:43.630025+0000","last_became_peered":"2024-03-29T08:22:43.630025+0000","last_unstale":"2024-03-29T08:22:43.636068+0000","last_undegraded":"2024-03-29T08:22:43.636068+0000","last_fullsized":"2024-03-29T08:22:43.636068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:46:02.004983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616698+0000","last_change":"2024-03-29T08:22:43.629031+0000","last_active":"2024-03-29T08:22:44.616698+0000","last_peered":"2024-03-29T08:22:44.616698+0000","last_clean":"2024-03-29T08:22:44.616698+0000","last_became_active":"2024-03-29T08:22:43.628689+0000","last_became_peered":"2024-03-29T08:22:43.628689+0000","last_unstale":"2024-03-29T08:22:44.616698+0000","last_undegraded":"2024-03-29T08:22:44.616698+0000","last_fullsized":"2024-03-29T08:22:44.616698+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:00:24.800667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.633178+0000","last_change":"2024-03-29T08:22:43.625172+0000","last_active":"2024-03-29T08:22:43.633178+0000","last_peered":"2024-03-29T08:22:43.633178+0000","last_clean":"2024-03-29T08:22:43.633178+0000","last_became_active":"2024-03-29T08:22:43.624925+0000","last_became_peered":"2024-03-29T08:22:43.624925+0000","last_unstale":"2024-03-29T08:22:43.633178+0000","last_undegraded":"2024-03-29T08:22:43.633178+0000","last_fullsized":"2024-03-29T08:22:43.633178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:15:04.871776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805217+0000","last_change":"2024-03-29T08:22:43.622948+0000","last_active":"2024-03-29T08:22:44.805217+0000","last_peered":"2024-03-29T08:22:44.805217+0000","last_clean":"2024-03-29T08:22:44.805217+0000","last_became_active":"2024-03-29T08:22:43.621966+0000","last_became_peered":"2024-03-29T08:22:43.621966+0000","last_unstale":"2024-03-29T08:22:44.805217+0000","last_undegraded":"2024-03-29T08:22:44.805217+0000","last_fullsized":"2024-03-29T08:22:44.805217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:36:34.102967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615742+0000","last_change":"2024-03-29T08:22:43.633739+0000","last_active":"2024-03-29T08:22:44.615742+0000","last_peered":"2024-03-29T08:22:44.615742+0000","last_clean":"2024-03-29T08:22:44.615742+0000","last_became_active":"2024-03-29T08:22:43.633467+0000","last_became_peered":"2024-03-29T08:22:43.633467+0000","last_unstale":"2024-03-29T08:22:44.615742+0000","last_undegraded":"2024-03-29T08:22:44.615742+0000","last_fullsized":"2024-03-29T08:22:44.615742+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:52:34.595634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805418+0000","last_change":"2024-03-29T08:22:43.642181+0000","last_active":"2024-03-29T08:22:44.805418+0000","last_peered":"2024-03-29T08:22:44.805418+0000","last_clean":"2024-03-29T08:22:44.805418+0000","last_became_active":"2024-03-29T08:22:43.641702+0000","last_became_peered":"2024-03-29T08:22:43.641702+0000","last_unstale":"2024-03-29T08:22:44.805418+0000","last_undegraded":"2024-03-29T08:22:44.805418+0000","last_fullsized":"2024-03-29T08:22:44.805418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:47:44.670409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616420+0000","last_change":"2024-03-29T08:22:43.634630+0000","last_active":"2024-03-29T08:22:44.616420+0000","last_peered":"2024-03-29T08:22:44.616420+0000","last_clean":"2024-03-29T08:22:44.616420+0000","last_became_active":"2024-03-29T08:22:43.634436+0000","last_became_peered":"2024-03-29T08:22:43.634436+0000","last_unstale":"2024-03-29T08:22:44.616420+0000","last_undegraded":"2024-03-29T08:22:44.616420+0000","last_fullsized":"2024-03-29T08:22:44.616420+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:03:30.338339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805760+0000","last_change":"2024-03-29T08:22:43.642436+0000","last_active":"2024-03-29T08:22:44.805760+0000","last_peered":"2024-03-29T08:22:44.805760+0000","last_clean":"2024-03-29T08:22:44.805760+0000","last_became_active":"2024-03-29T08:22:43.642105+0000","last_became_peered":"2024-03-29T08:22:43.642105+0000","last_unstale":"2024-03-29T08:22:44.805760+0000","last_undegraded":"2024-03-29T08:22:44.805760+0000","last_fullsized":"2024-03-29T08:22:44.805760+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:54:37.580597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616780+0000","last_change":"2024-03-29T08:22:43.634174+0000","last_active":"2024-03-29T08:22:44.616780+0000","last_peered":"2024-03-29T08:22:44.616780+0000","last_clean":"2024-03-29T08:22:44.616780+0000","last_became_active":"2024-03-29T08:22:43.633910+0000","last_became_peered":"2024-03-29T08:22:43.633910+0000","last_unstale":"2024-03-29T08:22:44.616780+0000","last_undegraded":"2024-03-29T08:22:44.616780+0000","last_fullsized":"2024-03-29T08:22:44.616780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:50:45.140035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.639079+0000","last_change":"2024-03-29T08:22:43.631752+0000","last_active":"2024-03-29T08:22:43.639079+0000","last_peered":"2024-03-29T08:22:43.639079+0000","last_clean":"2024-03-29T08:22:43.639079+0000","last_became_active":"2024-03-29T08:22:43.629604+0000","last_became_peered":"2024-03-29T08:22:43.629604+0000","last_unstale":"2024-03-29T08:22:43.639079+0000","last_undegraded":"2024-03-29T08:22:43.639079+0000","last_fullsized":"2024-03-29T08:22:43.639079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:21:30.129175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805243+0000","last_change":"2024-03-29T08:22:43.630773+0000","last_active":"2024-03-29T08:22:44.805243+0000","last_peered":"2024-03-29T08:22:44.805243+0000","last_clean":"2024-03-29T08:22:44.805243+0000","last_became_active":"2024-03-29T08:22:43.630334+0000","last_became_peered":"2024-03-29T08:22:43.630334+0000","last_unstale":"2024-03-29T08:22:44.805243+0000","last_undegraded":"2024-03-29T08:22:44.805243+0000","last_fullsized":"2024-03-29T08:22:44.805243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:18:54.747124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806761+0000","last_change":"2024-03-29T08:22:43.634991+0000","last_active":"2024-03-29T08:22:44.806761+0000","last_peered":"2024-03-29T08:22:44.806761+0000","last_clean":"2024-03-29T08:22:44.806761+0000","last_became_active":"2024-03-29T08:22:43.634738+0000","last_became_peered":"2024-03-29T08:22:43.634738+0000","last_unstale":"2024-03-29T08:22:44.806761+0000","last_undegraded":"2024-03-29T08:22:44.806761+0000","last_fullsized":"2024-03-29T08:22:44.806761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:47:44.670409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616463+0000","last_change":"2024-03-29T08:22:43.629100+0000","last_active":"2024-03-29T08:22:44.616463+0000","last_peered":"2024-03-29T08:22:44.616463+0000","last_clean":"2024-03-29T08:22:44.616463+0000","last_became_active":"2024-03-29T08:22:43.627042+0000","last_became_peered":"2024-03-29T08:22:43.627042+0000","last_unstale":"2024-03-29T08:22:44.616463+0000","last_undegraded":"2024-03-29T08:22:44.616463+0000","last_fullsized":"2024-03-29T08:22:44.616463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:50:59.701887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805098+0000","last_change":"2024-03-29T08:22:43.631317+0000","last_active":"2024-03-29T08:22:44.805098+0000","last_peered":"2024-03-29T08:22:44.805098+0000","last_clean":"2024-03-29T08:22:44.805098+0000","last_became_active":"2024-03-29T08:22:43.630869+0000","last_became_peered":"2024-03-29T08:22:43.630869+0000","last_unstale":"2024-03-29T08:22:44.805098+0000","last_undegraded":"2024-03-29T08:22:44.805098+0000","last_fullsized":"2024-03-29T08:22:44.805098+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:27:27.391136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805053+0000","last_change":"2024-03-29T08:22:43.630349+0000","last_active":"2024-03-29T08:22:44.805053+0000","last_peered":"2024-03-29T08:22:44.805053+0000","last_clean":"2024-03-29T08:22:44.805053+0000","last_became_active":"2024-03-29T08:22:43.630024+0000","last_became_peered":"2024-03-29T08:22:43.630024+0000","last_unstale":"2024-03-29T08:22:44.805053+0000","last_undegraded":"2024-03-29T08:22:44.805053+0000","last_fullsized":"2024-03-29T08:22:44.805053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:50.979084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616776+0000","last_change":"2024-03-29T08:22:43.630622+0000","last_active":"2024-03-29T08:22:44.616776+0000","last_peered":"2024-03-29T08:22:44.616776+0000","last_clean":"2024-03-29T08:22:44.616776+0000","last_became_active":"2024-03-29T08:22:43.630260+0000","last_became_peered":"2024-03-29T08:22:43.630260+0000","last_unstale":"2024-03-29T08:22:44.616776+0000","last_undegraded":"2024-03-29T08:22:44.616776+0000","last_fullsized":"2024-03-29T08:22:44.616776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:30:20.394649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"76'4","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615889+0000","last_change":"2024-03-29T08:22:43.215321+0000","last_active":"2024-03-29T08:22:44.615889+0000","last_peered":"2024-03-29T08:22:44.615889+0000","last_clean":"2024-03-29T08:22:44.615889+0000","last_became_active":"2024-03-29T08:22:43.215046+0000","last_became_peered":"2024-03-29T08:22:43.215046+0000","last_unstale":"2024-03-29T08:22:44.615889+0000","last_undegraded":"2024-03-29T08:22:44.615889+0000","last_fullsized":"2024-03-29T08:22:44.615889+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:27:36.616878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806949+0000","last_change":"2024-03-29T08:22:43.628977+0000","last_active":"2024-03-29T08:22:44.806949+0000","last_peered":"2024-03-29T08:22:44.806949+0000","last_clean":"2024-03-29T08:22:44.806949+0000","last_became_active":"2024-03-29T08:22:43.628685+0000","last_became_peered":"2024-03-29T08:22:43.628685+0000","last_unstale":"2024-03-29T08:22:44.806949+0000","last_undegraded":"2024-03-29T08:22:44.806949+0000","last_fullsized":"2024-03-29T08:22:44.806949+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:25:32.682463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.639267+0000","last_change":"2024-03-29T08:22:43.221426+0000","last_active":"2024-03-29T08:22:43.639267+0000","last_peered":"2024-03-29T08:22:43.639267+0000","last_clean":"2024-03-29T08:22:43.639267+0000","last_became_active":"2024-03-29T08:22:43.221108+0000","last_became_peered":"2024-03-29T08:22:43.221108+0000","last_unstale":"2024-03-29T08:22:43.639267+0000","last_undegraded":"2024-03-29T08:22:43.639267+0000","last_fullsized":"2024-03-29T08:22:43.639267+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:23:23.298782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615876+0000","last_change":"2024-03-29T08:22:43.624975+0000","last_active":"2024-03-29T08:22:44.615876+0000","last_peered":"2024-03-29T08:22:44.615876+0000","last_clean":"2024-03-29T08:22:44.615876+0000","last_became_active":"2024-03-29T08:22:43.624703+0000","last_became_peered":"2024-03-29T08:22:43.624703+0000","last_unstale":"2024-03-29T08:22:44.615876+0000","last_undegraded":"2024-03-29T08:22:44.615876+0000","last_fullsized":"2024-03-29T08:22:44.615876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:36.191598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"76'2","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616830+0000","last_change":"2024-03-29T08:22:43.230592+0000","last_active":"2024-03-29T08:22:44.616830+0000","last_peered":"2024-03-29T08:22:44.616830+0000","last_clean":"2024-03-29T08:22:44.616830+0000","last_became_active":"2024-03-29T08:22:43.230283+0000","last_became_peered":"2024-03-29T08:22:43.230283+0000","last_unstale":"2024-03-29T08:22:44.616830+0000","last_undegraded":"2024-03-29T08:22:44.616830+0000","last_fullsized":"2024-03-29T08:22:44.616830+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:13:53.448045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805702+0000","last_change":"2024-03-29T08:22:43.639138+0000","last_active":"2024-03-29T08:22:44.805702+0000","last_peered":"2024-03-29T08:22:44.805702+0000","last_clean":"2024-03-29T08:22:44.805702+0000","last_became_active":"2024-03-29T08:22:43.638843+0000","last_became_peered":"2024-03-29T08:22:43.638843+0000","last_unstale":"2024-03-29T08:22:44.805702+0000","last_undegraded":"2024-03-29T08:22:44.805702+0000","last_fullsized":"2024-03-29T08:22:44.805702+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:48.131948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"76'2","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615961+0000","last_change":"2024-03-29T08:22:43.224111+0000","last_active":"2024-03-29T08:22:44.615961+0000","last_peered":"2024-03-29T08:22:44.615961+0000","last_clean":"2024-03-29T08:22:44.615961+0000","last_became_active":"2024-03-29T08:22:43.223482+0000","last_became_peered":"2024-03-29T08:22:43.223482+0000","last_unstale":"2024-03-29T08:22:44.615961+0000","last_undegraded":"2024-03-29T08:22:44.615961+0000","last_fullsized":"2024-03-29T08:22:44.615961+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:09.901410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616558+0000","last_change":"2024-03-29T08:22:43.632353+0000","last_active":"2024-03-29T08:22:44.616558+0000","last_peered":"2024-03-29T08:22:44.616558+0000","last_clean":"2024-03-29T08:22:44.616558+0000","last_became_active":"2024-03-29T08:22:43.630798+0000","last_became_peered":"2024-03-29T08:22:43.630798+0000","last_unstale":"2024-03-29T08:22:44.616558+0000","last_undegraded":"2024-03-29T08:22:44.616558+0000","last_fullsized":"2024-03-29T08:22:44.616558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:12:56.478370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"76'1","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.618099+0000","last_change":"2024-03-29T08:22:43.217434+0000","last_active":"2024-03-29T08:22:44.618099+0000","last_peered":"2024-03-29T08:22:44.618099+0000","last_clean":"2024-03-29T08:22:44.618099+0000","last_became_active":"2024-03-29T08:22:43.217181+0000","last_became_peered":"2024-03-29T08:22:43.217181+0000","last_unstale":"2024-03-29T08:22:44.618099+0000","last_undegraded":"2024-03-29T08:22:44.618099+0000","last_fullsized":"2024-03-29T08:22:44.618099+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:27:26.388890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615732+0000","last_change":"2024-03-29T08:22:43.637920+0000","last_active":"2024-03-29T08:22:44.615732+0000","last_peered":"2024-03-29T08:22:44.615732+0000","last_clean":"2024-03-29T08:22:44.615732+0000","last_became_active":"2024-03-29T08:22:43.637352+0000","last_became_peered":"2024-03-29T08:22:43.637352+0000","last_unstale":"2024-03-29T08:22:44.615732+0000","last_undegraded":"2024-03-29T08:22:44.615732+0000","last_fullsized":"2024-03-29T08:22:44.615732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:55:24.078237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"76'1","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805505+0000","last_change":"2024-03-29T08:22:43.226767+0000","last_active":"2024-03-29T08:22:44.805505+0000","last_peered":"2024-03-29T08:22:44.805505+0000","last_clean":"2024-03-29T08:22:44.805505+0000","last_became_active":"2024-03-29T08:22:43.226508+0000","last_became_peered":"2024-03-29T08:22:43.226508+0000","last_unstale":"2024-03-29T08:22:44.805505+0000","last_undegraded":"2024-03-29T08:22:44.805505+0000","last_fullsized":"2024-03-29T08:22:44.805505+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:25:10.869415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806689+0000","last_change":"2024-03-29T08:22:43.635841+0000","last_active":"2024-03-29T08:22:44.806689+0000","last_peered":"2024-03-29T08:22:44.806689+0000","last_clean":"2024-03-29T08:22:44.806689+0000","last_became_active":"2024-03-29T08:22:43.635564+0000","last_became_peered":"2024-03-29T08:22:43.635564+0000","last_unstale":"2024-03-29T08:22:44.806689+0000","last_undegraded":"2024-03-29T08:22:44.806689+0000","last_fullsized":"2024-03-29T08:22:44.806689+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:27:27.391136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806906+0000","last_change":"2024-03-29T08:22:43.230874+0000","last_active":"2024-03-29T08:22:44.806906+0000","last_peered":"2024-03-29T08:22:44.806906+0000","last_clean":"2024-03-29T08:22:44.806906+0000","last_became_active":"2024-03-29T08:22:43.230488+0000","last_became_peered":"2024-03-29T08:22:43.230488+0000","last_unstale":"2024-03-29T08:22:44.806906+0000","last_undegraded":"2024-03-29T08:22:44.806906+0000","last_fullsized":"2024-03-29T08:22:44.806906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:16:38.717323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617804+0000","last_change":"2024-03-29T08:22:43.629776+0000","last_active":"2024-03-29T08:22:44.617804+0000","last_peered":"2024-03-29T08:22:44.617804+0000","last_clean":"2024-03-29T08:22:44.617804+0000","last_became_active":"2024-03-29T08:22:43.629522+0000","last_became_peered":"2024-03-29T08:22:43.629522+0000","last_unstale":"2024-03-29T08:22:44.617804+0000","last_undegraded":"2024-03-29T08:22:44.617804+0000","last_fullsized":"2024-03-29T08:22:44.617804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:19:37.662168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617702+0000","last_change":"2024-03-29T08:22:43.215440+0000","last_active":"2024-03-29T08:22:44.617702+0000","last_peered":"2024-03-29T08:22:44.617702+0000","last_clean":"2024-03-29T08:22:44.617702+0000","last_became_active":"2024-03-29T08:22:43.215169+0000","last_became_peered":"2024-03-29T08:22:43.215169+0000","last_unstale":"2024-03-29T08:22:44.617702+0000","last_undegraded":"2024-03-29T08:22:44.617702+0000","last_fullsized":"2024-03-29T08:22:44.617702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:18:53.744878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805986+0000","last_change":"2024-03-29T08:22:43.642081+0000","last_active":"2024-03-29T08:22:44.805986+0000","last_peered":"2024-03-29T08:22:44.805986+0000","last_clean":"2024-03-29T08:22:44.805986+0000","last_became_active":"2024-03-29T08:22:43.641828+0000","last_became_peered":"2024-03-29T08:22:43.641828+0000","last_unstale":"2024-03-29T08:22:44.805986+0000","last_undegraded":"2024-03-29T08:22:44.805986+0000","last_fullsized":"2024-03-29T08:22:44.805986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:02:13.271666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616953+0000","last_change":"2024-03-29T08:22:43.232918+0000","last_active":"2024-03-29T08:22:44.616953+0000","last_peered":"2024-03-29T08:22:44.616953+0000","last_clean":"2024-03-29T08:22:44.616953+0000","last_became_active":"2024-03-29T08:22:43.232348+0000","last_became_peered":"2024-03-29T08:22:43.232348+0000","last_unstale":"2024-03-29T08:22:44.616953+0000","last_undegraded":"2024-03-29T08:22:44.616953+0000","last_fullsized":"2024-03-29T08:22:44.616953+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:48:19.039327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805896+0000","last_change":"2024-03-29T08:22:43.632088+0000","last_active":"2024-03-29T08:22:44.805896+0000","last_peered":"2024-03-29T08:22:44.805896+0000","last_clean":"2024-03-29T08:22:44.805896+0000","last_became_active":"2024-03-29T08:22:43.629957+0000","last_became_peered":"2024-03-29T08:22:43.629957+0000","last_unstale":"2024-03-29T08:22:44.805896+0000","last_undegraded":"2024-03-29T08:22:44.805896+0000","last_fullsized":"2024-03-29T08:22:44.805896+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:04:10.246667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805702+0000","last_change":"2024-03-29T08:22:43.219123+0000","last_active":"2024-03-29T08:22:44.805702+0000","last_peered":"2024-03-29T08:22:44.805702+0000","last_clean":"2024-03-29T08:22:44.805702+0000","last_became_active":"2024-03-29T08:22:43.218666+0000","last_became_peered":"2024-03-29T08:22:43.218666+0000","last_unstale":"2024-03-29T08:22:44.805702+0000","last_undegraded":"2024-03-29T08:22:44.805702+0000","last_fullsized":"2024-03-29T08:22:44.805702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:47:43.668163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616541+0000","last_change":"2024-03-29T08:22:43.639765+0000","last_active":"2024-03-29T08:22:44.616541+0000","last_peered":"2024-03-29T08:22:44.616541+0000","last_clean":"2024-03-29T08:22:44.616541+0000","last_became_active":"2024-03-29T08:22:43.639508+0000","last_became_peered":"2024-03-29T08:22:43.639508+0000","last_unstale":"2024-03-29T08:22:44.616541+0000","last_undegraded":"2024-03-29T08:22:44.616541+0000","last_fullsized":"2024-03-29T08:22:44.616541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:59:13.224813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"76'3","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617867+0000","last_change":"2024-03-29T08:22:43.222223+0000","last_active":"2024-03-29T08:22:44.617867+0000","last_peered":"2024-03-29T08:22:44.617867+0000","last_clean":"2024-03-29T08:22:44.617867+0000","last_became_active":"2024-03-29T08:22:43.221960+0000","last_became_peered":"2024-03-29T08:22:43.221960+0000","last_unstale":"2024-03-29T08:22:44.617867+0000","last_undegraded":"2024-03-29T08:22:44.617867+0000","last_fullsized":"2024-03-29T08:22:44.617867+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:06:49.605118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.639060+0000","last_change":"2024-03-29T08:22:43.631179+0000","last_active":"2024-03-29T08:22:43.639060+0000","last_peered":"2024-03-29T08:22:43.639060+0000","last_clean":"2024-03-29T08:22:43.639060+0000","last_became_active":"2024-03-29T08:22:43.629676+0000","last_became_peered":"2024-03-29T08:22:43.629676+0000","last_unstale":"2024-03-29T08:22:43.639060+0000","last_undegraded":"2024-03-29T08:22:43.639060+0000","last_fullsized":"2024-03-29T08:22:43.639060+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:04:11.342755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"76'6","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.807478+0000","last_change":"2024-03-29T08:22:43.225095+0000","last_active":"2024-03-29T08:22:44.807478+0000","last_peered":"2024-03-29T08:22:44.807478+0000","last_clean":"2024-03-29T08:22:44.807478+0000","last_became_active":"2024-03-29T08:22:43.224653+0000","last_became_peered":"2024-03-29T08:22:43.224653+0000","last_unstale":"2024-03-29T08:22:44.807478+0000","last_undegraded":"2024-03-29T08:22:44.807478+0000","last_fullsized":"2024-03-29T08:22:44.807478+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:27:57.366351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615744+0000","last_change":"2024-03-29T08:22:43.639987+0000","last_active":"2024-03-29T08:22:44.615744+0000","last_peered":"2024-03-29T08:22:44.615744+0000","last_clean":"2024-03-29T08:22:44.615744+0000","last_became_active":"2024-03-29T08:22:43.633583+0000","last_became_peered":"2024-03-29T08:22:43.633583+0000","last_unstale":"2024-03-29T08:22:44.615744+0000","last_undegraded":"2024-03-29T08:22:44.615744+0000","last_fullsized":"2024-03-29T08:22:44.615744+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:07:31.259509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"76'2","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.807341+0000","last_change":"2024-03-29T08:22:43.233812+0000","last_active":"2024-03-29T08:22:44.807341+0000","last_peered":"2024-03-29T08:22:44.807341+0000","last_clean":"2024-03-29T08:22:44.807341+0000","last_became_active":"2024-03-29T08:22:43.233506+0000","last_became_peered":"2024-03-29T08:22:43.233506+0000","last_unstale":"2024-03-29T08:22:44.807341+0000","last_undegraded":"2024-03-29T08:22:44.807341+0000","last_fullsized":"2024-03-29T08:22:44.807341+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:33:08.692539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805885+0000","last_change":"2024-03-29T08:22:43.635709+0000","last_active":"2024-03-29T08:22:44.805885+0000","last_peered":"2024-03-29T08:22:44.805885+0000","last_clean":"2024-03-29T08:22:44.805885+0000","last_became_active":"2024-03-29T08:22:43.635450+0000","last_became_peered":"2024-03-29T08:22:43.635450+0000","last_unstale":"2024-03-29T08:22:44.805885+0000","last_undegraded":"2024-03-29T08:22:44.805885+0000","last_fullsized":"2024-03-29T08:22:44.805885+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:56:48.996098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805615+0000","last_change":"2024-03-29T08:22:43.222275+0000","last_active":"2024-03-29T08:22:44.805615+0000","last_peered":"2024-03-29T08:22:44.805615+0000","last_clean":"2024-03-29T08:22:44.805615+0000","last_became_active":"2024-03-29T08:22:43.221852+0000","last_became_peered":"2024-03-29T08:22:43.221852+0000","last_unstale":"2024-03-29T08:22:44.805615+0000","last_undegraded":"2024-03-29T08:22:44.805615+0000","last_fullsized":"2024-03-29T08:22:44.805615+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:41:44.709879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.635930+0000","last_change":"2024-03-29T08:22:43.626610+0000","last_active":"2024-03-29T08:22:43.635930+0000","last_peered":"2024-03-29T08:22:43.635930+0000","last_clean":"2024-03-29T08:22:43.635930+0000","last_became_active":"2024-03-29T08:22:43.626357+0000","last_became_peered":"2024-03-29T08:22:43.626357+0000","last_unstale":"2024-03-29T08:22:43.635930+0000","last_undegraded":"2024-03-29T08:22:43.635930+0000","last_fullsized":"2024-03-29T08:22:43.635930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:30:19.739080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805489+0000","last_change":"2024-03-29T08:22:43.222495+0000","last_active":"2024-03-29T08:22:44.805489+0000","last_peered":"2024-03-29T08:22:44.805489+0000","last_clean":"2024-03-29T08:22:44.805489+0000","last_became_active":"2024-03-29T08:22:43.222233+0000","last_became_peered":"2024-03-29T08:22:43.222233+0000","last_unstale":"2024-03-29T08:22:44.805489+0000","last_undegraded":"2024-03-29T08:22:44.805489+0000","last_fullsized":"2024-03-29T08:22:44.805489+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:29:13.574338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806420+0000","last_change":"2024-03-29T08:22:43.629285+0000","last_active":"2024-03-29T08:22:44.806420+0000","last_peered":"2024-03-29T08:22:44.806420+0000","last_clean":"2024-03-29T08:22:44.806420+0000","last_became_active":"2024-03-29T08:22:43.628997+0000","last_became_peered":"2024-03-29T08:22:43.628997+0000","last_unstale":"2024-03-29T08:22:44.806420+0000","last_undegraded":"2024-03-29T08:22:44.806420+0000","last_fullsized":"2024-03-29T08:22:44.806420+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:26:04.212702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"76'8","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805269+0000","last_change":"2024-03-29T08:22:43.225743+0000","last_active":"2024-03-29T08:22:44.805269+0000","last_peered":"2024-03-29T08:22:44.805269+0000","last_clean":"2024-03-29T08:22:44.805269+0000","last_became_active":"2024-03-29T08:22:43.225356+0000","last_became_peered":"2024-03-29T08:22:43.225356+0000","last_unstale":"2024-03-29T08:22:44.805269+0000","last_undegraded":"2024-03-29T08:22:44.805269+0000","last_fullsized":"2024-03-29T08:22:44.805269+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:22.771461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805650+0000","last_change":"2024-03-29T08:22:43.621232+0000","last_active":"2024-03-29T08:22:44.805650+0000","last_peered":"2024-03-29T08:22:44.805650+0000","last_clean":"2024-03-29T08:22:44.805650+0000","last_became_active":"2024-03-29T08:22:43.620968+0000","last_became_peered":"2024-03-29T08:22:43.620968+0000","last_unstale":"2024-03-29T08:22:44.805650+0000","last_undegraded":"2024-03-29T08:22:44.805650+0000","last_fullsized":"2024-03-29T08:22:44.805650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:06:50.607364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"76'1","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616977+0000","last_change":"2024-03-29T08:22:43.236922+0000","last_active":"2024-03-29T08:22:44.616977+0000","last_peered":"2024-03-29T08:22:44.616977+0000","last_clean":"2024-03-29T08:22:44.616977+0000","last_became_active":"2024-03-29T08:22:43.236713+0000","last_became_peered":"2024-03-29T08:22:43.236713+0000","last_unstale":"2024-03-29T08:22:44.616977+0000","last_undegraded":"2024-03-29T08:22:44.616977+0000","last_fullsized":"2024-03-29T08:22:44.616977+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:10:18.639596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805919+0000","last_change":"2024-03-29T08:22:43.630230+0000","last_active":"2024-03-29T08:22:44.805919+0000","last_peered":"2024-03-29T08:22:44.805919+0000","last_clean":"2024-03-29T08:22:44.805919+0000","last_became_active":"2024-03-29T08:22:43.629583+0000","last_became_peered":"2024-03-29T08:22:43.629583+0000","last_unstale":"2024-03-29T08:22:44.805919+0000","last_undegraded":"2024-03-29T08:22:44.805919+0000","last_fullsized":"2024-03-29T08:22:44.805919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:01.351488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"76'2","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805727+0000","last_change":"2024-03-29T08:22:43.226205+0000","last_active":"2024-03-29T08:22:44.805727+0000","last_peered":"2024-03-29T08:22:44.805727+0000","last_clean":"2024-03-29T08:22:44.805727+0000","last_became_active":"2024-03-29T08:22:43.225945+0000","last_became_peered":"2024-03-29T08:22:43.225945+0000","last_unstale":"2024-03-29T08:22:44.805727+0000","last_undegraded":"2024-03-29T08:22:44.805727+0000","last_fullsized":"2024-03-29T08:22:44.805727+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:54:36.578351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.807087+0000","last_change":"2024-03-29T08:22:43.625891+0000","last_active":"2024-03-29T08:22:44.807087+0000","last_peered":"2024-03-29T08:22:44.807087+0000","last_clean":"2024-03-29T08:22:44.807087+0000","last_became_active":"2024-03-29T08:22:43.625550+0000","last_became_peered":"2024-03-29T08:22:43.625550+0000","last_unstale":"2024-03-29T08:22:44.807087+0000","last_undegraded":"2024-03-29T08:22:44.807087+0000","last_fullsized":"2024-03-29T08:22:44.807087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:55.452693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"76'1","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805359+0000","last_change":"2024-03-29T08:22:43.230782+0000","last_active":"2024-03-29T08:22:44.805359+0000","last_peered":"2024-03-29T08:22:44.805359+0000","last_clean":"2024-03-29T08:22:44.805359+0000","last_became_active":"2024-03-29T08:22:43.230501+0000","last_became_peered":"2024-03-29T08:22:43.230501+0000","last_unstale":"2024-03-29T08:22:44.805359+0000","last_undegraded":"2024-03-29T08:22:44.805359+0000","last_fullsized":"2024-03-29T08:22:44.805359+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:58.293985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806472+0000","last_change":"2024-03-29T08:22:43.627874+0000","last_active":"2024-03-29T08:22:44.806472+0000","last_peered":"2024-03-29T08:22:44.806472+0000","last_clean":"2024-03-29T08:22:44.806472+0000","last_became_active":"2024-03-29T08:22:43.627509+0000","last_became_peered":"2024-03-29T08:22:43.627509+0000","last_unstale":"2024-03-29T08:22:44.806472+0000","last_undegraded":"2024-03-29T08:22:44.806472+0000","last_fullsized":"2024-03-29T08:22:44.806472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:59.296230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616035+0000","last_change":"2024-03-29T08:22:43.231245+0000","last_active":"2024-03-29T08:22:44.616035+0000","last_peered":"2024-03-29T08:22:44.616035+0000","last_clean":"2024-03-29T08:22:44.616035+0000","last_became_active":"2024-03-29T08:22:43.231003+0000","last_became_peered":"2024-03-29T08:22:43.231003+0000","last_unstale":"2024-03-29T08:22:44.616035+0000","last_undegraded":"2024-03-29T08:22:44.616035+0000","last_fullsized":"2024-03-29T08:22:44.616035+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:00.349242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.807450+0000","last_change":"2024-03-29T08:22:43.633345+0000","last_active":"2024-03-29T08:22:44.807450+0000","last_peered":"2024-03-29T08:22:44.807450+0000","last_clean":"2024-03-29T08:22:44.807450+0000","last_became_active":"2024-03-29T08:22:43.632738+0000","last_became_peered":"2024-03-29T08:22:43.632738+0000","last_unstale":"2024-03-29T08:22:44.807450+0000","last_undegraded":"2024-03-29T08:22:44.807450+0000","last_fullsized":"2024-03-29T08:22:44.807450+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:27:37.619123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615857+0000","last_change":"2024-03-29T08:22:43.215283+0000","last_active":"2024-03-29T08:22:44.615857+0000","last_peered":"2024-03-29T08:22:44.615857+0000","last_clean":"2024-03-29T08:22:44.615857+0000","last_became_active":"2024-03-29T08:22:43.214394+0000","last_became_peered":"2024-03-29T08:22:43.214394+0000","last_unstale":"2024-03-29T08:22:44.615857+0000","last_undegraded":"2024-03-29T08:22:44.615857+0000","last_fullsized":"2024-03-29T08:22:44.615857+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:36:33.100721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806789+0000","last_change":"2024-03-29T08:22:43.632269+0000","last_active":"2024-03-29T08:22:44.806789+0000","last_peered":"2024-03-29T08:22:44.806789+0000","last_clean":"2024-03-29T08:22:44.806789+0000","last_became_active":"2024-03-29T08:22:43.631956+0000","last_became_peered":"2024-03-29T08:22:43.631956+0000","last_unstale":"2024-03-29T08:22:44.806789+0000","last_undegraded":"2024-03-29T08:22:44.806789+0000","last_fullsized":"2024-03-29T08:22:44.806789+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:18:54.747124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805203+0000","last_change":"2024-03-29T08:22:43.214550+0000","last_active":"2024-03-29T08:22:44.805203+0000","last_peered":"2024-03-29T08:22:44.805203+0000","last_clean":"2024-03-29T08:22:44.805203+0000","last_became_active":"2024-03-29T08:22:43.214148+0000","last_became_peered":"2024-03-29T08:22:43.214148+0000","last_unstale":"2024-03-29T08:22:44.805203+0000","last_undegraded":"2024-03-29T08:22:44.805203+0000","last_fullsized":"2024-03-29T08:22:44.805203+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:58:28.598523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617916+0000","last_change":"2024-03-29T08:22:43.626561+0000","last_active":"2024-03-29T08:22:44.617916+0000","last_peered":"2024-03-29T08:22:44.617916+0000","last_clean":"2024-03-29T08:22:44.617916+0000","last_became_active":"2024-03-29T08:22:43.625552+0000","last_became_peered":"2024-03-29T08:22:43.625552+0000","last_unstale":"2024-03-29T08:22:44.617916+0000","last_undegraded":"2024-03-29T08:22:44.617916+0000","last_fullsized":"2024-03-29T08:22:44.617916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:36.191598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.614165+0000","last_change":"2024-03-29T08:22:43.221653+0000","last_active":"2024-03-29T08:22:43.614165+0000","last_peered":"2024-03-29T08:22:43.614165+0000","last_clean":"2024-03-29T08:22:43.614165+0000","last_became_active":"2024-03-29T08:22:43.221327+0000","last_became_peered":"2024-03-29T08:22:43.221327+0000","last_unstale":"2024-03-29T08:22:43.614165+0000","last_undegraded":"2024-03-29T08:22:43.614165+0000","last_fullsized":"2024-03-29T08:22:43.614165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:27:29.994458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.635984+0000","last_change":"2024-03-29T08:22:43.628071+0000","last_active":"2024-03-29T08:22:43.635984+0000","last_peered":"2024-03-29T08:22:43.635984+0000","last_clean":"2024-03-29T08:22:43.635984+0000","last_became_active":"2024-03-29T08:22:43.627813+0000","last_became_peered":"2024-03-29T08:22:43.627813+0000","last_unstale":"2024-03-29T08:22:43.635984+0000","last_undegraded":"2024-03-29T08:22:43.635984+0000","last_fullsized":"2024-03-29T08:22:43.635984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:56:43.668049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"76'2","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617031+0000","last_change":"2024-03-29T08:22:43.229417+0000","last_active":"2024-03-29T08:22:44.617031+0000","last_peered":"2024-03-29T08:22:44.617031+0000","last_clean":"2024-03-29T08:22:44.617031+0000","last_became_active":"2024-03-29T08:22:43.229140+0000","last_became_peered":"2024-03-29T08:22:43.229140+0000","last_unstale":"2024-03-29T08:22:44.617031+0000","last_undegraded":"2024-03-29T08:22:44.617031+0000","last_fullsized":"2024-03-29T08:22:44.617031+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:58:00.887698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805886+0000","last_change":"2024-03-29T08:22:43.628003+0000","last_active":"2024-03-29T08:22:44.805886+0000","last_peered":"2024-03-29T08:22:44.805886+0000","last_clean":"2024-03-29T08:22:44.805886+0000","last_became_active":"2024-03-29T08:22:43.627801+0000","last_became_peered":"2024-03-29T08:22:43.627801+0000","last_unstale":"2024-03-29T08:22:44.805886+0000","last_undegraded":"2024-03-29T08:22:44.805886+0000","last_fullsized":"2024-03-29T08:22:44.805886+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:00:32.361795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616868+0000","last_change":"2024-03-29T08:22:43.233295+0000","last_active":"2024-03-29T08:22:44.616868+0000","last_peered":"2024-03-29T08:22:44.616868+0000","last_clean":"2024-03-29T08:22:44.616868+0000","last_became_active":"2024-03-29T08:22:43.230783+0000","last_became_peered":"2024-03-29T08:22:43.230783+0000","last_unstale":"2024-03-29T08:22:44.616868+0000","last_undegraded":"2024-03-29T08:22:44.616868+0000","last_fullsized":"2024-03-29T08:22:44.616868+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:31:09.745872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805346+0000","last_change":"2024-03-29T08:22:43.630577+0000","last_active":"2024-03-29T08:22:44.805346+0000","last_peered":"2024-03-29T08:22:44.805346+0000","last_clean":"2024-03-29T08:22:44.805346+0000","last_became_active":"2024-03-29T08:22:43.629939+0000","last_became_peered":"2024-03-29T08:22:43.629939+0000","last_unstale":"2024-03-29T08:22:44.805346+0000","last_undegraded":"2024-03-29T08:22:44.805346+0000","last_fullsized":"2024-03-29T08:22:44.805346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:54:42.264431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617887+0000","last_change":"2024-03-29T08:22:43.222144+0000","last_active":"2024-03-29T08:22:44.617887+0000","last_peered":"2024-03-29T08:22:44.617887+0000","last_clean":"2024-03-29T08:22:44.617887+0000","last_became_active":"2024-03-29T08:22:43.221340+0000","last_became_peered":"2024-03-29T08:22:43.221340+0000","last_unstale":"2024-03-29T08:22:44.617887+0000","last_undegraded":"2024-03-29T08:22:44.617887+0000","last_fullsized":"2024-03-29T08:22:44.617887+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:49.976839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805729+0000","last_change":"2024-03-29T08:22:43.632501+0000","last_active":"2024-03-29T08:22:44.805729+0000","last_peered":"2024-03-29T08:22:44.805729+0000","last_clean":"2024-03-29T08:22:44.805729+0000","last_became_active":"2024-03-29T08:22:43.632223+0000","last_became_peered":"2024-03-29T08:22:43.632223+0000","last_unstale":"2024-03-29T08:22:44.805729+0000","last_undegraded":"2024-03-29T08:22:44.805729+0000","last_fullsized":"2024-03-29T08:22:44.805729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:58:29.600769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.618010+0000","last_change":"2024-03-29T08:22:43.220688+0000","last_active":"2024-03-29T08:22:44.618010+0000","last_peered":"2024-03-29T08:22:44.618010+0000","last_clean":"2024-03-29T08:22:44.618010+0000","last_became_active":"2024-03-29T08:22:43.219423+0000","last_became_peered":"2024-03-29T08:22:43.219423+0000","last_unstale":"2024-03-29T08:22:44.618010+0000","last_undegraded":"2024-03-29T08:22:44.618010+0000","last_fullsized":"2024-03-29T08:22:44.618010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:54:41.262185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.627857+0000","last_change":"2024-03-29T08:22:43.619914+0000","last_active":"2024-03-29T08:22:43.627857+0000","last_peered":"2024-03-29T08:22:43.627857+0000","last_clean":"2024-03-29T08:22:43.627857+0000","last_became_active":"2024-03-29T08:22:43.619631+0000","last_became_peered":"2024-03-29T08:22:43.619631+0000","last_unstale":"2024-03-29T08:22:43.627857+0000","last_undegraded":"2024-03-29T08:22:43.627857+0000","last_fullsized":"2024-03-29T08:22:43.627857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:16.123192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617860+0000","last_change":"2024-03-29T08:22:43.220643+0000","last_active":"2024-03-29T08:22:44.617860+0000","last_peered":"2024-03-29T08:22:44.617860+0000","last_clean":"2024-03-29T08:22:44.617860+0000","last_became_active":"2024-03-29T08:22:43.220364+0000","last_became_peered":"2024-03-29T08:22:43.220364+0000","last_unstale":"2024-03-29T08:22:44.617860+0000","last_undegraded":"2024-03-29T08:22:44.617860+0000","last_fullsized":"2024-03-29T08:22:44.617860+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:09.901410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616728+0000","last_change":"2024-03-29T08:22:43.636809+0000","last_active":"2024-03-29T08:22:44.616728+0000","last_peered":"2024-03-29T08:22:44.616728+0000","last_clean":"2024-03-29T08:22:44.616728+0000","last_became_active":"2024-03-29T08:22:43.632811+0000","last_became_peered":"2024-03-29T08:22:43.632811+0000","last_unstale":"2024-03-29T08:22:44.616728+0000","last_undegraded":"2024-03-29T08:22:44.616728+0000","last_fullsized":"2024-03-29T08:22:44.616728+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:12.985868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.614086+0000","last_change":"2024-03-29T08:22:43.226092+0000","last_active":"2024-03-29T08:22:43.614086+0000","last_peered":"2024-03-29T08:22:43.614086+0000","last_clean":"2024-03-29T08:22:43.614086+0000","last_became_active":"2024-03-29T08:22:43.225879+0000","last_became_peered":"2024-03-29T08:22:43.225879+0000","last_unstale":"2024-03-29T08:22:43.614086+0000","last_undegraded":"2024-03-29T08:22:43.614086+0000","last_fullsized":"2024-03-29T08:22:43.614086+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:05:45.594351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806869+0000","last_change":"2024-03-29T08:22:43.629920+0000","last_active":"2024-03-29T08:22:44.806869+0000","last_peered":"2024-03-29T08:22:44.806869+0000","last_clean":"2024-03-29T08:22:44.806869+0000","last_became_active":"2024-03-29T08:22:43.629622+0000","last_became_peered":"2024-03-29T08:22:43.629622+0000","last_unstale":"2024-03-29T08:22:44.806869+0000","last_undegraded":"2024-03-29T08:22:44.806869+0000","last_fullsized":"2024-03-29T08:22:44.806869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:10.903656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805210+0000","last_change":"2024-03-29T08:22:43.219117+0000","last_active":"2024-03-29T08:22:44.805210+0000","last_peered":"2024-03-29T08:22:44.805210+0000","last_clean":"2024-03-29T08:22:44.805210+0000","last_became_active":"2024-03-29T08:22:43.218434+0000","last_became_peered":"2024-03-29T08:22:43.218434+0000","last_unstale":"2024-03-29T08:22:44.805210+0000","last_undegraded":"2024-03-29T08:22:44.805210+0000","last_fullsized":"2024-03-29T08:22:44.805210+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:02:12.269420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805105+0000","last_change":"2024-03-29T08:22:43.631161+0000","last_active":"2024-03-29T08:22:44.805105+0000","last_peered":"2024-03-29T08:22:44.805105+0000","last_clean":"2024-03-29T08:22:44.805105+0000","last_became_active":"2024-03-29T08:22:43.630549+0000","last_became_peered":"2024-03-29T08:22:43.630549+0000","last_unstale":"2024-03-29T08:22:44.805105+0000","last_undegraded":"2024-03-29T08:22:44.805105+0000","last_fullsized":"2024-03-29T08:22:44.805105+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:59:11.546657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615628+0000","last_change":"2024-03-29T08:22:43.218296+0000","last_active":"2024-03-29T08:22:44.615628+0000","last_peered":"2024-03-29T08:22:44.615628+0000","last_clean":"2024-03-29T08:22:44.615628+0000","last_became_active":"2024-03-29T08:22:43.218020+0000","last_became_peered":"2024-03-29T08:22:43.218020+0000","last_unstale":"2024-03-29T08:22:44.615628+0000","last_undegraded":"2024-03-29T08:22:44.615628+0000","last_fullsized":"2024-03-29T08:22:44.615628+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:27:26.388890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.636022+0000","last_change":"2024-03-29T08:22:43.628211+0000","last_active":"2024-03-29T08:22:43.636022+0000","last_peered":"2024-03-29T08:22:43.636022+0000","last_clean":"2024-03-29T08:22:43.636022+0000","last_became_active":"2024-03-29T08:22:43.627639+0000","last_became_peered":"2024-03-29T08:22:43.627639+0000","last_unstale":"2024-03-29T08:22:43.636022+0000","last_undegraded":"2024-03-29T08:22:43.636022+0000","last_fullsized":"2024-03-29T08:22:43.636022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:20:20.240883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"76'2","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805581+0000","last_change":"2024-03-29T08:22:43.223307+0000","last_active":"2024-03-29T08:22:44.805581+0000","last_peered":"2024-03-29T08:22:44.805581+0000","last_clean":"2024-03-29T08:22:44.805581+0000","last_became_active":"2024-03-29T08:22:43.223039+0000","last_became_peered":"2024-03-29T08:22:43.223039+0000","last_unstale":"2024-03-29T08:22:44.805581+0000","last_undegraded":"2024-03-29T08:22:44.805581+0000","last_fullsized":"2024-03-29T08:22:44.805581+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:26.031908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":211,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.622161+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:22:43.622161+0000","last_peered":"2024-03-29T08:22:43.622161+0000","last_clean":"2024-03-29T08:22:43.622161+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:22:43.622161+0000","last_undegraded":"2024-03-29T08:22:43.622161+0000","last_fullsized":"2024-03-29T08:22:43.622161+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509500,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":395205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:22:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.70099999999999996,"15min":0.66000000000000003},"min":{"1min":0.56599999999999995,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.94799999999999995,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.5},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.72399999999999998,"15min":0.68400000000000005},"min":{"1min":0.628,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96199999999999997,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.68899999999999995}]},{"osd":1,"last update":"Fri Mar 29 08:22:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68600000000000005,"15min":0.65300000000000002},"min":{"1min":0.52000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90700000000000003,"5min":1.202,"15min":1.202},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.69399999999999995,"15min":0.64200000000000002},"min":{"1min":0.57999999999999996,"5min":0.441,"15min":0.441},"max":{"1min":0.89700000000000002,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.748}]},{"osd":2,"last update":"Fri Mar 29 08:22:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.68000000000000005,"15min":0.63900000000000001},"min":{"1min":0.54600000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.86699999999999999,"5min":1.085,"15min":1.085},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.71399999999999997,"15min":0.67200000000000004},"min":{"1min":0.55500000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":1.288,"15min":1.288},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:22:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71899999999999997,"15min":0.70099999999999996},"min":{"1min":0.56200000000000006,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89300000000000002,"5min":1.018,"15min":1.018},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.748,"15min":0.73199999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.97099999999999997,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.86599999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:22:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61899999999999999,"15min":0.629},"min":{"1min":0.433,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.67700000000000005,"15min":0.71099999999999997},"min":{"1min":0.45400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":1.123,"15min":1.123},"last":0.80000000000000004}]},{"osd":5,"last update":"Fri Mar 29 08:22:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68000000000000005,"15min":0.70399999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.91100000000000003,"5min":1.056,"15min":1.056},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68600000000000005,"15min":0.69799999999999995},"min":{"1min":0.45200000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.88700000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:22:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72899999999999998,"15min":0.72799999999999998},"min":{"1min":0.52000000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92600000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.71699999999999997,"15min":0.746},"min":{"1min":0.502,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.88400000000000001,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.84999999999999998}]}]},{"osd":6,"up_from":15,"seq":64424509500,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":946176,"data_stored":395239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.67700000000000005,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86299999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72099999999999997,"15min":0.69099999999999995},"min":{"1min":0.64200000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94199999999999995,"5min":1.079,"15min":1.079},"last":0.83799999999999997}]},{"osd":1,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65100000000000002,"15min":0.61799999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82299999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.66500000000000004,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.99399999999999999,"5min":1.079,"15min":1.079},"last":0.754}]},{"osd":2,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72399999999999998,"15min":0.68200000000000005},"min":{"1min":0.60599999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.95799999999999996,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72399999999999998,"15min":0.70599999999999996},"min":{"1min":0.60699999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.90400000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.63200000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70999999999999996,"15min":0.69899999999999995},"min":{"1min":0.48499999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0089999999999999,"5min":1.109,"15min":1.109},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70399999999999996,"15min":0.69499999999999995},"min":{"1min":0.50700000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91400000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73599999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64900000000000002,"15min":0.67100000000000004},"min":{"1min":0.39400000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.002,"5min":1.556,"15min":1.556},"last":0.437},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.67500000000000004,"15min":0.72699999999999998},"min":{"1min":0.437,"5min":0.371,"15min":0.371},"max":{"1min":0.81499999999999995,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.56999999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.65400000000000003,"15min":0.69499999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91000000000000003,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.69099999999999995,"15min":0.72599999999999998},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85699999999999998,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.65200000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.71599999999999997,"15min":0.75800000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.875,"5min":0.998,"15min":0.998},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.72599999999999998,"15min":0.77000000000000002},"min":{"1min":0.501,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.84999999999999998,"5min":1.109,"15min":1.109},"last":0.67500000000000004}]}]},{"osd":4,"up_from":15,"seq":64424509499,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":929792,"data_stored":388035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85799999999999998,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71999999999999997,"15min":0.74199999999999999},"min":{"1min":0.54700000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.90200000000000002,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.85899999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.624,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92700000000000005,"5min":2.524,"15min":2.524},"last":1.042},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.71799999999999997},"min":{"1min":0.44500000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90400000000000003,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.67600000000000005}]},{"osd":2,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72199999999999998,"15min":0.73199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91300000000000003,"5min":2.363,"15min":2.363},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.60599999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.87,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.76600000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74199999999999999,"15min":0.755},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89300000000000002,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.79400000000000004,"15min":0.81200000000000006},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95199999999999996,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.95599999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66900000000000004,"15min":0.69399999999999995},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.64300000000000002,"15min":0.65000000000000002},"min":{"1min":0.44600000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.79900000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.88700000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65100000000000002,"15min":0.67100000000000004},"min":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90600000000000003,"5min":0.996,"15min":0.996},"last":1.016},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71899999999999997,"15min":0.74199999999999999},"min":{"1min":0.51500000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.88,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":1.1220000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70599999999999996,"15min":0.73399999999999999},"min":{"1min":0.53200000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96399999999999997,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74199999999999999,"15min":0.76400000000000001},"min":{"1min":0.55400000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97799999999999998,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.0660000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509502,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":929792,"data_stored":387467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45500000000000002,"15min":0.46300000000000002},"min":{"1min":0.26600000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.84799999999999998,"5min":1.123,"15min":1.123},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.371,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73799999999999999,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64100000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.44700000000000001,"15min":0.46100000000000002},"min":{"1min":0.31,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.78900000000000003,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51100000000000001,"15min":0.53300000000000003},"min":{"1min":0.38500000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.004,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.61599999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53700000000000003,"15min":0.56699999999999995},"min":{"1min":0.33200000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.80900000000000005,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.625,"15min":0.65500000000000003},"min":{"1min":0.47399999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97699999999999998,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.73299999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.75700000000000001,"15min":0.79000000000000004},"min":{"1min":0.55900000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93000000000000005,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.748,"15min":0.77600000000000002},"min":{"1min":0.46999999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0629999999999999,"5min":1.391,"15min":1.391},"last":0.90500000000000003}]},{"osd":5,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78200000000000003,"15min":0.80400000000000005},"min":{"1min":0.56999999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0409999999999999,"5min":1.671,"15min":1.671},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.82299999999999995,"15min":0.85999999999999999},"min":{"1min":0.58199999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.032,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.0389999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.76600000000000001,"15min":0.82599999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98099999999999998,"5min":1.669,"15min":1.669},"last":0.995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.82499999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.075,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.91600000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.80700000000000005,"15min":0.85699999999999998},"min":{"1min":0.56000000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98899999999999999,"5min":1.887,"15min":1.887},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.79700000000000004,"15min":0.85299999999999998},"min":{"1min":0.53700000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0089999999999999,"5min":2.044,"15min":2.044},"last":0.93899999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509501,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":933888,"data_stored":394223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47699999999999998,"15min":0.497},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.68500000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.42899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72599999999999998,"5min":0.751,"15min":0.751},"last":0.54600000000000004}]},{"osd":1,"last update":"Fri Mar 29 08:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.496,"15min":0.497},"min":{"1min":0.34300000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.61899999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.74299999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.33400000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51400000000000001,"15min":0.495},"min":{"1min":0.44700000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.46500000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78500000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.44700000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.77000000000000002,"15min":0.76500000000000001},"min":{"1min":0.59799999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0089999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.76500000000000001},"min":{"1min":0.48599999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.044,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.82599999999999996,"15min":0.84099999999999997},"min":{"1min":0.66200000000000003,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0109999999999999,"5min":1.306,"15min":1.306},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.82299999999999995,"15min":0.85499999999999998},"min":{"1min":0.55500000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.995,"5min":1.526,"15min":1.526},"last":0.81200000000000006}]},{"osd":6,"last update":"Fri Mar 29 08:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.58699999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97199999999999998,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77000000000000002,"15min":0.78800000000000003},"min":{"1min":0.53400000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.97399999999999998,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.79500000000000004,"15min":0.80700000000000005},"min":{"1min":0.59099999999999997,"5min":0.505,"15min":0.505},"max":{"1min":0.99199999999999999,"5min":1.319,"15min":1.319},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.77700000000000002,"15min":0.79500000000000004},"min":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.026,"5min":1.216,"15min":1.216},"last":0.60099999999999998}]}]},{"osd":1,"up_from":15,"seq":64424509501,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1523712,"data_stored":984009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.52300000000000002,"15min":0.62},"min":{"1min":0.33100000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.57699999999999996,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56899999999999995,"15min":0.64300000000000002},"min":{"1min":0.32900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.84299999999999997,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.50600000000000001},"min":{"1min":0.41499999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52800000000000002,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66500000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.496}]},{"osd":3,"last update":"Fri Mar 29 08:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91000000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78500000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46899999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.755,"15min":0.78600000000000003},"min":{"1min":0.51800000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0229999999999999,"5min":1.23,"15min":1.23},"last":0.71899999999999997}]},{"osd":5,"last update":"Fri Mar 29 08:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.76100000000000001,"15min":0.77900000000000003},"min":{"1min":0.55200000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.093,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.749,"15min":0.75900000000000001},"min":{"1min":0.53400000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Mar 29 08:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80700000000000005,"15min":0.85299999999999998},"min":{"1min":0.58499999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0509999999999999,"5min":1.196,"15min":1.196},"last":0.624}]},{"osd":7,"last update":"Fri Mar 29 08:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.79000000000000004,"15min":0.80100000000000005},"min":{"1min":0.57199999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77000000000000002,"15min":0.78800000000000003},"min":{"1min":0.56799999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.74299999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509500,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":933888,"data_stored":393711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.50700000000000001,"15min":0.59799999999999998},"min":{"1min":0.30299999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58699999999999997,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.66300000000000003,"15min":0.84299999999999997},"min":{"1min":0.33100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.67500000000000004,"5min":8.0419999999999998,"15min":8.0419999999999998},"last":0.32200000000000001}]},{"osd":2,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.51900000000000002,"15min":0.54900000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.61599999999999999,"5min":1.258,"15min":1.258},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.621,"15min":0.72899999999999998},"min":{"1min":0.40000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.63,"5min":4.907,"15min":4.907},"last":0.47599999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.66400000000000003,"15min":0.84199999999999997},"min":{"1min":0.33100000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.72699999999999998,"5min":7.9569999999999999,"15min":7.9569999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60299999999999998,"15min":0.65900000000000003},"min":{"1min":0.46400000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.71299999999999997,"5min":1.397,"15min":1.397},"last":0.36299999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.874,"15min":0.98599999999999999},"min":{"1min":0.59699999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0029999999999999,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.85699999999999998,"15min":0.96099999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0209999999999999,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.56599999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.84599999999999997,"15min":0.92900000000000005},"min":{"1min":0.60999999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0880000000000001,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80800000000000005,"15min":0.86199999999999999},"min":{"1min":0.60799999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.94999999999999996,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.82199999999999995,"15min":0.86399999999999999},"min":{"1min":0.64500000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.98399999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.88200000000000001,"15min":0.98399999999999999},"min":{"1min":0.64500000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0389999999999999,"5min":4.4100000000000001,"15min":4.4100000000000001},"last":0.68600000000000005}]},{"osd":7,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89600000000000002,"15min":1.004},"min":{"1min":0.59799999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.0669999999999999,"5min":4.5209999999999999,"15min":4.5209999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.872,"15min":0.98399999999999999},"min":{"1min":0.63,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.006,"5min":4.4420000000000002,"15min":4.4420000000000002},"last":0.58499999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542205,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1515520,"data_stored":977799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.45000000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85999999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65700000000000003,"15min":0.65400000000000003},"min":{"1min":0.48699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80500000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.75600000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.69799999999999995,"15min":0.73199999999999998},"min":{"1min":0.505,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88400000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.626},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.54900000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88100000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.46700000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81599999999999995,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70499999999999996,"15min":0.70399999999999996},"min":{"1min":0.55900000000000005,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91400000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.68999999999999995}]},{"osd":3,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71099999999999997,"15min":0.71399999999999997},"min":{"1min":0.54900000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88700000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.52300000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.85999999999999999,"5min":1.954,"15min":1.954},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.46100000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83399999999999996,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64700000000000002,"15min":0.66100000000000003},"min":{"1min":0.442,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88300000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.746}]},{"osd":6,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65500000000000003,"15min":0.67800000000000005},"min":{"1min":0.42899999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81799999999999995,"5min":1.071,"15min":1.071},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.5,"5min":0.432,"15min":0.432},"max":{"1min":0.80000000000000004,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.65000000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69599999999999995,"15min":0.71599999999999997},"min":{"1min":0.495,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85199999999999998,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.72599999999999998,"15min":0.753},"min":{"1min":0.52300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.872,"5min":1.04,"15min":1.04},"last":0.81499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:22:47.721 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:22:47.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.902+0000 7fb5da600640 1 Processor -- start 2024-03-29T08:22:47.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.902+0000 7fb5da600640 1 -- start start 2024-03-29T08:22:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.902+0000 7fb5da600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb5d40ce590 0x7fb5d4069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.902+0000 7fb5da600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d40aeee0 0x7fb5d40af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.902+0000 7fb5da600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d406a5e0 con 0x7fb5d40b0310 2024-03-29T08:22:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.902+0000 7fb5da600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d406aba0 con 0x7fb5d40ce590 2024-03-29T08:22:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.902+0000 7fb5da600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d40ce9f0 con 0x7fb5d40aeee0 2024-03-29T08:22:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.902+0000 7fb5d3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb5d40ce590 0x7fb5d4069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.902+0000 7fb5d3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb5d40ce590 0x7fb5d4069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54558/0 (socket says 172.21.15.46:54558) 2024-03-29T08:22:47.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.902+0000 7fb5d3e00640 1 -- 172.21.15.46:0/1916317685 learned_addr learned my addr 172.21.15.46:0/1916317685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:47.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.902+0000 7fb5d3e00640 1 -- 172.21.15.46:0/1916317685 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d40aeee0 msgr2=0x7fb5d40af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:47.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.902+0000 7fb5d9c00640 1 --2- 172.21.15.46:0/1916317685 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d40aeee0 0x7fb5d40af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.902+0000 7fb5d3400640 1 -- 172.21.15.46:0/1916317685 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176456003 0 0) 0x7fb5d406a5e0 con 0x7fb5d40b0310 2024-03-29T08:22:47.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.902+0000 7fb5d3e00640 1 --2- 172.21.15.46:0/1916317685 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d40aeee0 0x7fb5d40af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.902+0000 7fb5d3e00640 1 -- 172.21.15.46:0/1916317685 >> v1:172.21.15.46:6789/0 conn(0x7fb5d40b0310 legacy=0x7fb5d40b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.902+0000 7fb5d3e00640 1 -- 172.21.15.46:0/1916317685 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d406a8c0 con 0x7fb5d40ce590 2024-03-29T08:22:47.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.902+0000 7fb5d9c00640 1 --2- 172.21.15.46:0/1916317685 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d40aeee0 0x7fb5d40af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:22:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.906+0000 7fb5d3e00640 1 --2- 172.21.15.46:0/1916317685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d40ce590 0x7fb5d4069d90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009870 tx=0x7fb5c001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a074a89e149906 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.906+0000 7fb5d3400640 1 -- 172.21.15.46:0/1916317685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00a95f0 con 0x7fb5d40ce590 2024-03-29T08:22:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.906+0000 7fb5d3400640 1 -- 172.21.15.46:0/1916317685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c00a9750 con 0x7fb5d40ce590 2024-03-29T08:22:47.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.906+0000 7fb5d3400640 1 -- 172.21.15.46:0/1916317685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00a9ae0 con 0x7fb5d40ce590 2024-03-29T08:22:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.906+0000 7fb5da600640 1 -- 172.21.15.46:0/1916317685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d40ce590 msgr2=0x7fb5d4069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.906+0000 7fb5da600640 1 --2- 172.21.15.46:0/1916317685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d40ce590 0x7fb5d4069d90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009870 tx=0x7fb5c001c530 comp rx=0 tx=0).stop 2024-03-29T08:22:47.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.906+0000 7fb5da600640 1 -- 172.21.15.46:0/1916317685 shutdown_connections 2024-03-29T08:22:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.906+0000 7fb5da600640 1 --2- 172.21.15.46:0/1916317685 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d40aeee0 0x7fb5d40af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.906+0000 7fb5da600640 1 --2- 172.21.15.46:0/1916317685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d40ce590 0x7fb5d4069d90 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.906+0000 7fb5da600640 1 -- 172.21.15.46:0/1916317685 >> 172.21.15.46:0/1916317685 conn(0x7fb5d4098a40 msgr2=0x7fb5d40cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.906+0000 7fb5da600640 1 -- 172.21.15.46:0/1916317685 shutdown_connections 2024-03-29T08:22:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.906+0000 7fb5da600640 1 -- 172.21.15.46:0/1916317685 wait complete. 2024-03-29T08:22:47.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.906+0000 7fb5da600640 1 Processor -- start 2024-03-29T08:22:47.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.906+0000 7fb5da600640 1 -- start start 2024-03-29T08:22:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5da600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d40aeee0 0x7fb5d4147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5da600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d40b0310 0x7fb5d4099420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5da600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d40ce590 0x7fb5d40cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5da600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d4156a10 con 0x7fb5d40ce590 2024-03-29T08:22:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5da600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d415db70 con 0x7fb5d40aeee0 2024-03-29T08:22:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5da600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d4155e00 con 0x7fb5d40b0310 2024-03-29T08:22:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5d9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d40ce590 0x7fb5d40cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5d9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d40ce590 0x7fb5d40cf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54560/0 (socket says 172.21.15.46:54560) 2024-03-29T08:22:47.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5d9c00640 1 -- 172.21.15.46:0/625464384 learned_addr learned my addr 172.21.15.46:0/625464384 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5d9200640 1 --2- 172.21.15.46:0/625464384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d40aeee0 0x7fb5d4147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5d3e00640 1 --2- 172.21.15.46:0/625464384 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d40b0310 0x7fb5d4099420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5d9c00640 1 -- 172.21.15.46:0/625464384 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d40b0310 msgr2=0x7fb5d4099420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5d9c00640 1 --2- 172.21.15.46:0/625464384 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d40b0310 0x7fb5d4099420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5d9c00640 1 -- 172.21.15.46:0/625464384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d40aeee0 msgr2=0x7fb5d4147b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5d9c00640 1 --2- 172.21.15.46:0/625464384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d40aeee0 0x7fb5d4147b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5d9c00640 1 -- 172.21.15.46:0/625464384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c00094d0 con 0x7fb5d40ce590 2024-03-29T08:22:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5d9c00640 1 --2- 172.21.15.46:0/625464384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d40ce590 0x7fb5d40cf0c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00cd50 tx=0x7fb5cc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5d0c00640 1 -- 172.21.15.46:0/625464384 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc00f680 con 0x7fb5d40ce590 2024-03-29T08:22:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5d0c00640 1 -- 172.21.15.46:0/625464384 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5cc00f800 con 0x7fb5d40ce590 2024-03-29T08:22:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5da600640 1 -- 172.21.15.46:0/625464384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d41560e0 con 0x7fb5d40ce590 2024-03-29T08:22:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5d0c00640 1 -- 172.21.15.46:0/625464384 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc00fbb0 con 0x7fb5d40ce590 2024-03-29T08:22:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5da600640 1 -- 172.21.15.46:0/625464384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d41566a0 con 0x7fb5d40ce590 2024-03-29T08:22:47.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5da600640 1 -- 172.21.15.46:0/625464384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d4156b80 con 0x7fb5d40ce590 2024-03-29T08:22:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.910+0000 7fb5d0c00640 1 -- 172.21.15.46:0/625464384 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb5cc00b040 con 0x7fb5d40ce590 2024-03-29T08:22:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.914+0000 7fb5d0c00640 1 --2- 172.21.15.46:0/625464384 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb5b4066690 0x7fb5b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.914+0000 7fb5d0c00640 1 -- 172.21.15.46:0/625464384 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5cc084ad0 con 0x7fb5d40ce590 2024-03-29T08:22:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.914+0000 7fb5d9200640 1 --2- 172.21.15.46:0/625464384 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb5b4066690 0x7fb5b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:47.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.914+0000 7fb5d9200640 1 --2- 172.21.15.46:0/625464384 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb5b4066690 0x7fb5b4068b50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8000f70 tx=0x7fb5c8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:47.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:47.918+0000 7fb5d0c00640 1 -- 172.21.15.46:0/625464384 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb5cc051ed0 con 0x7fb5d40ce590 2024-03-29T08:22:48.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.078+0000 7fb5da600640 1 -- 172.21.15.46:0/625464384 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb5d40ac310 con 0x7fb5b4066690 2024-03-29T08:22:48.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.086+0000 7fb5d0c00640 1 -- 172.21.15.46:0/625464384 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289360 (secure 0 0 0) 0x7fb5c8001d00 con 0x7fb5b4066690 2024-03-29T08:22:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:48.093 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:22:48.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.090+0000 7fb5da600640 1 -- 172.21.15.46:0/625464384 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb5b4066690 msgr2=0x7fb5b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:48.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.090+0000 7fb5da600640 1 --2- 172.21.15.46:0/625464384 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb5b4066690 0x7fb5b4068b50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8000f70 tx=0x7fb5c8009040 comp rx=0 tx=0).stop 2024-03-29T08:22:48.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.090+0000 7fb5da600640 1 -- 172.21.15.46:0/625464384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d40ce590 msgr2=0x7fb5d40cf0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:48.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.090+0000 7fb5da600640 1 --2- 172.21.15.46:0/625464384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d40ce590 0x7fb5d40cf0c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00cd50 tx=0x7fb5cc00a650 comp rx=0 tx=0).stop 2024-03-29T08:22:48.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.090+0000 7fb5da600640 1 -- 172.21.15.46:0/625464384 shutdown_connections 2024-03-29T08:22:48.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.090+0000 7fb5da600640 1 --2- 172.21.15.46:0/625464384 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb5b4066690 0x7fb5b4068b50 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.090+0000 7fb5da600640 1 --2- 172.21.15.46:0/625464384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d40ce590 0x7fb5d40cf0c0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.090+0000 7fb5da600640 1 --2- 172.21.15.46:0/625464384 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d40b0310 0x7fb5d4099420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.090+0000 7fb5da600640 1 --2- 172.21.15.46:0/625464384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d40aeee0 0x7fb5d4147b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.090+0000 7fb5da600640 1 -- 172.21.15.46:0/625464384 >> 172.21.15.46:0/625464384 conn(0x7fb5d4098a40 msgr2=0x7fb5d406e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:48.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.090+0000 7fb5da600640 1 -- 172.21.15.46:0/625464384 shutdown_connections 2024-03-29T08:22:48.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.090+0000 7fb5da600640 1 -- 172.21.15.46:0/625464384 wait complete. 2024-03-29T08:22:48.112 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-03-29T08:22:46.108607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249228,"kb_used_data":8452,"kb_used_omap":12,"kb_used_meta":240627,"kb_avail":754725492,"statfs":{"total":773094113280,"available":772838903808,"internally_reserved":0,"allocated":8654848,"data_stored":4315688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":246402617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001056"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617012+0000","last_change":"2024-03-29T08:22:43.633225+0000","last_active":"2024-03-29T08:22:44.617012+0000","last_peered":"2024-03-29T08:22:44.617012+0000","last_clean":"2024-03-29T08:22:44.617012+0000","last_became_active":"2024-03-29T08:22:43.632973+0000","last_became_peered":"2024-03-29T08:22:43.632973+0000","last_unstale":"2024-03-29T08:22:44.617012+0000","last_undegraded":"2024-03-29T08:22:44.617012+0000","last_fullsized":"2024-03-29T08:22:44.617012+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:25:25.960847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805627+0000","last_change":"2024-03-29T08:22:43.629895+0000","last_active":"2024-03-29T08:22:44.805627+0000","last_peered":"2024-03-29T08:22:44.805627+0000","last_clean":"2024-03-29T08:22:44.805627+0000","last_became_active":"2024-03-29T08:22:43.629559+0000","last_became_peered":"2024-03-29T08:22:43.629559+0000","last_unstale":"2024-03-29T08:22:44.805627+0000","last_undegraded":"2024-03-29T08:22:44.805627+0000","last_fullsized":"2024-03-29T08:22:44.805627+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:02.860218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806460+0000","last_change":"2024-03-29T08:22:43.642933+0000","last_active":"2024-03-29T08:22:44.806460+0000","last_peered":"2024-03-29T08:22:44.806460+0000","last_clean":"2024-03-29T08:22:44.806460+0000","last_became_active":"2024-03-29T08:22:43.642635+0000","last_became_peered":"2024-03-29T08:22:43.642635+0000","last_unstale":"2024-03-29T08:22:44.806460+0000","last_undegraded":"2024-03-29T08:22:44.806460+0000","last_fullsized":"2024-03-29T08:22:44.806460+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:48.131948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806561+0000","last_change":"2024-03-29T08:22:43.643005+0000","last_active":"2024-03-29T08:22:44.806561+0000","last_peered":"2024-03-29T08:22:44.806561+0000","last_clean":"2024-03-29T08:22:44.806561+0000","last_became_active":"2024-03-29T08:22:43.642661+0000","last_became_peered":"2024-03-29T08:22:43.642661+0000","last_unstale":"2024-03-29T08:22:44.806561+0000","last_undegraded":"2024-03-29T08:22:44.806561+0000","last_fullsized":"2024-03-29T08:22:44.806561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:00:32.361795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.628169+0000","last_change":"2024-03-29T08:22:43.619735+0000","last_active":"2024-03-29T08:22:43.628169+0000","last_peered":"2024-03-29T08:22:43.628169+0000","last_clean":"2024-03-29T08:22:43.628169+0000","last_became_active":"2024-03-29T08:22:43.619437+0000","last_became_peered":"2024-03-29T08:22:43.619437+0000","last_unstale":"2024-03-29T08:22:43.628169+0000","last_undegraded":"2024-03-29T08:22:43.628169+0000","last_fullsized":"2024-03-29T08:22:43.628169+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:58:46.838515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615649+0000","last_change":"2024-03-29T08:22:43.631163+0000","last_active":"2024-03-29T08:22:44.615649+0000","last_peered":"2024-03-29T08:22:44.615649+0000","last_clean":"2024-03-29T08:22:44.615649+0000","last_became_active":"2024-03-29T08:22:43.630891+0000","last_became_peered":"2024-03-29T08:22:43.630891+0000","last_unstale":"2024-03-29T08:22:44.615649+0000","last_undegraded":"2024-03-29T08:22:44.615649+0000","last_fullsized":"2024-03-29T08:22:44.615649+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:52.022173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805402+0000","last_change":"2024-03-29T08:22:43.639229+0000","last_active":"2024-03-29T08:22:44.805402+0000","last_peered":"2024-03-29T08:22:44.805402+0000","last_clean":"2024-03-29T08:22:44.805402+0000","last_became_active":"2024-03-29T08:22:43.638986+0000","last_became_peered":"2024-03-29T08:22:43.638986+0000","last_unstale":"2024-03-29T08:22:44.805402+0000","last_undegraded":"2024-03-29T08:22:44.805402+0000","last_fullsized":"2024-03-29T08:22:44.805402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:59.296230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616317+0000","last_change":"2024-03-29T08:22:43.626805+0000","last_active":"2024-03-29T08:22:44.616317+0000","last_peered":"2024-03-29T08:22:44.616317+0000","last_clean":"2024-03-29T08:22:44.616317+0000","last_became_active":"2024-03-29T08:22:43.626525+0000","last_became_peered":"2024-03-29T08:22:43.626525+0000","last_unstale":"2024-03-29T08:22:44.616317+0000","last_undegraded":"2024-03-29T08:22:44.616317+0000","last_fullsized":"2024-03-29T08:22:44.616317+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:33:05.663906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806898+0000","last_change":"2024-03-29T08:22:43.626927+0000","last_active":"2024-03-29T08:22:44.806898+0000","last_peered":"2024-03-29T08:22:44.806898+0000","last_clean":"2024-03-29T08:22:44.806898+0000","last_became_active":"2024-03-29T08:22:43.626679+0000","last_became_peered":"2024-03-29T08:22:43.626679+0000","last_unstale":"2024-03-29T08:22:44.806898+0000","last_undegraded":"2024-03-29T08:22:44.806898+0000","last_fullsized":"2024-03-29T08:22:44.806898+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:27:12.482046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805938+0000","last_change":"2024-03-29T08:22:43.637486+0000","last_active":"2024-03-29T08:22:44.805938+0000","last_peered":"2024-03-29T08:22:44.805938+0000","last_clean":"2024-03-29T08:22:44.805938+0000","last_became_active":"2024-03-29T08:22:43.637013+0000","last_became_peered":"2024-03-29T08:22:43.637013+0000","last_unstale":"2024-03-29T08:22:44.805938+0000","last_undegraded":"2024-03-29T08:22:44.805938+0000","last_fullsized":"2024-03-29T08:22:44.805938+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:25:32.682463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616821+0000","last_change":"2024-03-29T08:22:43.632983+0000","last_active":"2024-03-29T08:22:44.616821+0000","last_peered":"2024-03-29T08:22:44.616821+0000","last_clean":"2024-03-29T08:22:44.616821+0000","last_became_active":"2024-03-29T08:22:43.632646+0000","last_became_peered":"2024-03-29T08:22:43.632646+0000","last_unstale":"2024-03-29T08:22:44.616821+0000","last_undegraded":"2024-03-29T08:22:44.616821+0000","last_fullsized":"2024-03-29T08:22:44.616821+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:51:19.001329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617961+0000","last_change":"2024-03-29T08:22:43.631539+0000","last_active":"2024-03-29T08:22:44.617961+0000","last_peered":"2024-03-29T08:22:44.617961+0000","last_clean":"2024-03-29T08:22:44.617961+0000","last_became_active":"2024-03-29T08:22:43.631243+0000","last_became_peered":"2024-03-29T08:22:43.631243+0000","last_unstale":"2024-03-29T08:22:44.617961+0000","last_undegraded":"2024-03-29T08:22:44.617961+0000","last_fullsized":"2024-03-29T08:22:44.617961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:34:54.150064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806665+0000","last_change":"2024-03-29T08:22:43.631647+0000","last_active":"2024-03-29T08:22:44.806665+0000","last_peered":"2024-03-29T08:22:44.806665+0000","last_clean":"2024-03-29T08:22:44.806665+0000","last_became_active":"2024-03-29T08:22:43.631176+0000","last_became_peered":"2024-03-29T08:22:43.631176+0000","last_unstale":"2024-03-29T08:22:44.806665+0000","last_undegraded":"2024-03-29T08:22:44.806665+0000","last_fullsized":"2024-03-29T08:22:44.806665+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:16:06.248835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617737+0000","last_change":"2024-03-29T08:22:43.639032+0000","last_active":"2024-03-29T08:22:44.617737+0000","last_peered":"2024-03-29T08:22:44.617737+0000","last_clean":"2024-03-29T08:22:44.617737+0000","last_became_active":"2024-03-29T08:22:43.638727+0000","last_became_peered":"2024-03-29T08:22:43.638727+0000","last_unstale":"2024-03-29T08:22:44.617737+0000","last_undegraded":"2024-03-29T08:22:44.617737+0000","last_fullsized":"2024-03-29T08:22:44.617737+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:51:08.118565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617792+0000","last_change":"2024-03-29T08:22:43.632746+0000","last_active":"2024-03-29T08:22:44.617792+0000","last_peered":"2024-03-29T08:22:44.617792+0000","last_clean":"2024-03-29T08:22:44.617792+0000","last_became_active":"2024-03-29T08:22:43.632424+0000","last_became_peered":"2024-03-29T08:22:43.632424+0000","last_unstale":"2024-03-29T08:22:44.617792+0000","last_undegraded":"2024-03-29T08:22:44.617792+0000","last_fullsized":"2024-03-29T08:22:44.617792+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:55:24.078237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616344+0000","last_change":"2024-03-29T08:22:43.626771+0000","last_active":"2024-03-29T08:22:44.616344+0000","last_peered":"2024-03-29T08:22:44.616344+0000","last_clean":"2024-03-29T08:22:44.616344+0000","last_became_active":"2024-03-29T08:22:43.626161+0000","last_became_peered":"2024-03-29T08:22:43.626161+0000","last_unstale":"2024-03-29T08:22:44.616344+0000","last_undegraded":"2024-03-29T08:22:44.616344+0000","last_fullsized":"2024-03-29T08:22:44.616344+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:22:34.783161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.632632+0000","last_change":"2024-03-29T08:22:43.624629+0000","last_active":"2024-03-29T08:22:43.632632+0000","last_peered":"2024-03-29T08:22:43.632632+0000","last_clean":"2024-03-29T08:22:43.632632+0000","last_became_active":"2024-03-29T08:22:43.624368+0000","last_became_peered":"2024-03-29T08:22:43.624368+0000","last_unstale":"2024-03-29T08:22:43.632632+0000","last_undegraded":"2024-03-29T08:22:43.632632+0000","last_fullsized":"2024-03-29T08:22:43.632632+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:13:33.671724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.636068+0000","last_change":"2024-03-29T08:22:43.630276+0000","last_active":"2024-03-29T08:22:43.636068+0000","last_peered":"2024-03-29T08:22:43.636068+0000","last_clean":"2024-03-29T08:22:43.636068+0000","last_became_active":"2024-03-29T08:22:43.630025+0000","last_became_peered":"2024-03-29T08:22:43.630025+0000","last_unstale":"2024-03-29T08:22:43.636068+0000","last_undegraded":"2024-03-29T08:22:43.636068+0000","last_fullsized":"2024-03-29T08:22:43.636068+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:46:02.004983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616698+0000","last_change":"2024-03-29T08:22:43.629031+0000","last_active":"2024-03-29T08:22:44.616698+0000","last_peered":"2024-03-29T08:22:44.616698+0000","last_clean":"2024-03-29T08:22:44.616698+0000","last_became_active":"2024-03-29T08:22:43.628689+0000","last_became_peered":"2024-03-29T08:22:43.628689+0000","last_unstale":"2024-03-29T08:22:44.616698+0000","last_undegraded":"2024-03-29T08:22:44.616698+0000","last_fullsized":"2024-03-29T08:22:44.616698+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:00:24.800667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.633178+0000","last_change":"2024-03-29T08:22:43.625172+0000","last_active":"2024-03-29T08:22:43.633178+0000","last_peered":"2024-03-29T08:22:43.633178+0000","last_clean":"2024-03-29T08:22:43.633178+0000","last_became_active":"2024-03-29T08:22:43.624925+0000","last_became_peered":"2024-03-29T08:22:43.624925+0000","last_unstale":"2024-03-29T08:22:43.633178+0000","last_undegraded":"2024-03-29T08:22:43.633178+0000","last_fullsized":"2024-03-29T08:22:43.633178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:15:04.871776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805217+0000","last_change":"2024-03-29T08:22:43.622948+0000","last_active":"2024-03-29T08:22:44.805217+0000","last_peered":"2024-03-29T08:22:44.805217+0000","last_clean":"2024-03-29T08:22:44.805217+0000","last_became_active":"2024-03-29T08:22:43.621966+0000","last_became_peered":"2024-03-29T08:22:43.621966+0000","last_unstale":"2024-03-29T08:22:44.805217+0000","last_undegraded":"2024-03-29T08:22:44.805217+0000","last_fullsized":"2024-03-29T08:22:44.805217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:36:34.102967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615742+0000","last_change":"2024-03-29T08:22:43.633739+0000","last_active":"2024-03-29T08:22:44.615742+0000","last_peered":"2024-03-29T08:22:44.615742+0000","last_clean":"2024-03-29T08:22:44.615742+0000","last_became_active":"2024-03-29T08:22:43.633467+0000","last_became_peered":"2024-03-29T08:22:43.633467+0000","last_unstale":"2024-03-29T08:22:44.615742+0000","last_undegraded":"2024-03-29T08:22:44.615742+0000","last_fullsized":"2024-03-29T08:22:44.615742+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:52:34.595634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805418+0000","last_change":"2024-03-29T08:22:43.642181+0000","last_active":"2024-03-29T08:22:44.805418+0000","last_peered":"2024-03-29T08:22:44.805418+0000","last_clean":"2024-03-29T08:22:44.805418+0000","last_became_active":"2024-03-29T08:22:43.641702+0000","last_became_peered":"2024-03-29T08:22:43.641702+0000","last_unstale":"2024-03-29T08:22:44.805418+0000","last_undegraded":"2024-03-29T08:22:44.805418+0000","last_fullsized":"2024-03-29T08:22:44.805418+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:47:44.670409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616420+0000","last_change":"2024-03-29T08:22:43.634630+0000","last_active":"2024-03-29T08:22:44.616420+0000","last_peered":"2024-03-29T08:22:44.616420+0000","last_clean":"2024-03-29T08:22:44.616420+0000","last_became_active":"2024-03-29T08:22:43.634436+0000","last_became_peered":"2024-03-29T08:22:43.634436+0000","last_unstale":"2024-03-29T08:22:44.616420+0000","last_undegraded":"2024-03-29T08:22:44.616420+0000","last_fullsized":"2024-03-29T08:22:44.616420+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:03:30.338339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805760+0000","last_change":"2024-03-29T08:22:43.642436+0000","last_active":"2024-03-29T08:22:44.805760+0000","last_peered":"2024-03-29T08:22:44.805760+0000","last_clean":"2024-03-29T08:22:44.805760+0000","last_became_active":"2024-03-29T08:22:43.642105+0000","last_became_peered":"2024-03-29T08:22:43.642105+0000","last_unstale":"2024-03-29T08:22:44.805760+0000","last_undegraded":"2024-03-29T08:22:44.805760+0000","last_fullsized":"2024-03-29T08:22:44.805760+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:54:37.580597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616780+0000","last_change":"2024-03-29T08:22:43.634174+0000","last_active":"2024-03-29T08:22:44.616780+0000","last_peered":"2024-03-29T08:22:44.616780+0000","last_clean":"2024-03-29T08:22:44.616780+0000","last_became_active":"2024-03-29T08:22:43.633910+0000","last_became_peered":"2024-03-29T08:22:43.633910+0000","last_unstale":"2024-03-29T08:22:44.616780+0000","last_undegraded":"2024-03-29T08:22:44.616780+0000","last_fullsized":"2024-03-29T08:22:44.616780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:50:45.140035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.639079+0000","last_change":"2024-03-29T08:22:43.631752+0000","last_active":"2024-03-29T08:22:43.639079+0000","last_peered":"2024-03-29T08:22:43.639079+0000","last_clean":"2024-03-29T08:22:43.639079+0000","last_became_active":"2024-03-29T08:22:43.629604+0000","last_became_peered":"2024-03-29T08:22:43.629604+0000","last_unstale":"2024-03-29T08:22:43.639079+0000","last_undegraded":"2024-03-29T08:22:43.639079+0000","last_fullsized":"2024-03-29T08:22:43.639079+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:21:30.129175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805243+0000","last_change":"2024-03-29T08:22:43.630773+0000","last_active":"2024-03-29T08:22:44.805243+0000","last_peered":"2024-03-29T08:22:44.805243+0000","last_clean":"2024-03-29T08:22:44.805243+0000","last_became_active":"2024-03-29T08:22:43.630334+0000","last_became_peered":"2024-03-29T08:22:43.630334+0000","last_unstale":"2024-03-29T08:22:44.805243+0000","last_undegraded":"2024-03-29T08:22:44.805243+0000","last_fullsized":"2024-03-29T08:22:44.805243+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:18:54.747124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806761+0000","last_change":"2024-03-29T08:22:43.634991+0000","last_active":"2024-03-29T08:22:44.806761+0000","last_peered":"2024-03-29T08:22:44.806761+0000","last_clean":"2024-03-29T08:22:44.806761+0000","last_became_active":"2024-03-29T08:22:43.634738+0000","last_became_peered":"2024-03-29T08:22:43.634738+0000","last_unstale":"2024-03-29T08:22:44.806761+0000","last_undegraded":"2024-03-29T08:22:44.806761+0000","last_fullsized":"2024-03-29T08:22:44.806761+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:47:44.670409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616463+0000","last_change":"2024-03-29T08:22:43.629100+0000","last_active":"2024-03-29T08:22:44.616463+0000","last_peered":"2024-03-29T08:22:44.616463+0000","last_clean":"2024-03-29T08:22:44.616463+0000","last_became_active":"2024-03-29T08:22:43.627042+0000","last_became_peered":"2024-03-29T08:22:43.627042+0000","last_unstale":"2024-03-29T08:22:44.616463+0000","last_undegraded":"2024-03-29T08:22:44.616463+0000","last_fullsized":"2024-03-29T08:22:44.616463+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:50:59.701887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805098+0000","last_change":"2024-03-29T08:22:43.631317+0000","last_active":"2024-03-29T08:22:44.805098+0000","last_peered":"2024-03-29T08:22:44.805098+0000","last_clean":"2024-03-29T08:22:44.805098+0000","last_became_active":"2024-03-29T08:22:43.630869+0000","last_became_peered":"2024-03-29T08:22:43.630869+0000","last_unstale":"2024-03-29T08:22:44.805098+0000","last_undegraded":"2024-03-29T08:22:44.805098+0000","last_fullsized":"2024-03-29T08:22:44.805098+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:27:27.391136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805053+0000","last_change":"2024-03-29T08:22:43.630349+0000","last_active":"2024-03-29T08:22:44.805053+0000","last_peered":"2024-03-29T08:22:44.805053+0000","last_clean":"2024-03-29T08:22:44.805053+0000","last_became_active":"2024-03-29T08:22:43.630024+0000","last_became_peered":"2024-03-29T08:22:43.630024+0000","last_unstale":"2024-03-29T08:22:44.805053+0000","last_undegraded":"2024-03-29T08:22:44.805053+0000","last_fullsized":"2024-03-29T08:22:44.805053+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:50.979084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616776+0000","last_change":"2024-03-29T08:22:43.630622+0000","last_active":"2024-03-29T08:22:44.616776+0000","last_peered":"2024-03-29T08:22:44.616776+0000","last_clean":"2024-03-29T08:22:44.616776+0000","last_became_active":"2024-03-29T08:22:43.630260+0000","last_became_peered":"2024-03-29T08:22:43.630260+0000","last_unstale":"2024-03-29T08:22:44.616776+0000","last_undegraded":"2024-03-29T08:22:44.616776+0000","last_fullsized":"2024-03-29T08:22:44.616776+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:30:20.394649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"76'4","reported_seq":15,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615889+0000","last_change":"2024-03-29T08:22:43.215321+0000","last_active":"2024-03-29T08:22:44.615889+0000","last_peered":"2024-03-29T08:22:44.615889+0000","last_clean":"2024-03-29T08:22:44.615889+0000","last_became_active":"2024-03-29T08:22:43.215046+0000","last_became_peered":"2024-03-29T08:22:43.215046+0000","last_unstale":"2024-03-29T08:22:44.615889+0000","last_undegraded":"2024-03-29T08:22:44.615889+0000","last_fullsized":"2024-03-29T08:22:44.615889+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:27:36.616878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806949+0000","last_change":"2024-03-29T08:22:43.628977+0000","last_active":"2024-03-29T08:22:44.806949+0000","last_peered":"2024-03-29T08:22:44.806949+0000","last_clean":"2024-03-29T08:22:44.806949+0000","last_became_active":"2024-03-29T08:22:43.628685+0000","last_became_peered":"2024-03-29T08:22:43.628685+0000","last_unstale":"2024-03-29T08:22:44.806949+0000","last_undegraded":"2024-03-29T08:22:44.806949+0000","last_fullsized":"2024-03-29T08:22:44.806949+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:25:32.682463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.639267+0000","last_change":"2024-03-29T08:22:43.221426+0000","last_active":"2024-03-29T08:22:43.639267+0000","last_peered":"2024-03-29T08:22:43.639267+0000","last_clean":"2024-03-29T08:22:43.639267+0000","last_became_active":"2024-03-29T08:22:43.221108+0000","last_became_peered":"2024-03-29T08:22:43.221108+0000","last_unstale":"2024-03-29T08:22:43.639267+0000","last_undegraded":"2024-03-29T08:22:43.639267+0000","last_fullsized":"2024-03-29T08:22:43.639267+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:23:23.298782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615876+0000","last_change":"2024-03-29T08:22:43.624975+0000","last_active":"2024-03-29T08:22:44.615876+0000","last_peered":"2024-03-29T08:22:44.615876+0000","last_clean":"2024-03-29T08:22:44.615876+0000","last_became_active":"2024-03-29T08:22:43.624703+0000","last_became_peered":"2024-03-29T08:22:43.624703+0000","last_unstale":"2024-03-29T08:22:44.615876+0000","last_undegraded":"2024-03-29T08:22:44.615876+0000","last_fullsized":"2024-03-29T08:22:44.615876+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:36.191598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"76'2","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616830+0000","last_change":"2024-03-29T08:22:43.230592+0000","last_active":"2024-03-29T08:22:44.616830+0000","last_peered":"2024-03-29T08:22:44.616830+0000","last_clean":"2024-03-29T08:22:44.616830+0000","last_became_active":"2024-03-29T08:22:43.230283+0000","last_became_peered":"2024-03-29T08:22:43.230283+0000","last_unstale":"2024-03-29T08:22:44.616830+0000","last_undegraded":"2024-03-29T08:22:44.616830+0000","last_fullsized":"2024-03-29T08:22:44.616830+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:13:53.448045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805702+0000","last_change":"2024-03-29T08:22:43.639138+0000","last_active":"2024-03-29T08:22:44.805702+0000","last_peered":"2024-03-29T08:22:44.805702+0000","last_clean":"2024-03-29T08:22:44.805702+0000","last_became_active":"2024-03-29T08:22:43.638843+0000","last_became_peered":"2024-03-29T08:22:43.638843+0000","last_unstale":"2024-03-29T08:22:44.805702+0000","last_undegraded":"2024-03-29T08:22:44.805702+0000","last_fullsized":"2024-03-29T08:22:44.805702+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:48.131948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"76'2","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615961+0000","last_change":"2024-03-29T08:22:43.224111+0000","last_active":"2024-03-29T08:22:44.615961+0000","last_peered":"2024-03-29T08:22:44.615961+0000","last_clean":"2024-03-29T08:22:44.615961+0000","last_became_active":"2024-03-29T08:22:43.223482+0000","last_became_peered":"2024-03-29T08:22:43.223482+0000","last_unstale":"2024-03-29T08:22:44.615961+0000","last_undegraded":"2024-03-29T08:22:44.615961+0000","last_fullsized":"2024-03-29T08:22:44.615961+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:09.901410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616558+0000","last_change":"2024-03-29T08:22:43.632353+0000","last_active":"2024-03-29T08:22:44.616558+0000","last_peered":"2024-03-29T08:22:44.616558+0000","last_clean":"2024-03-29T08:22:44.616558+0000","last_became_active":"2024-03-29T08:22:43.630798+0000","last_became_peered":"2024-03-29T08:22:43.630798+0000","last_unstale":"2024-03-29T08:22:44.616558+0000","last_undegraded":"2024-03-29T08:22:44.616558+0000","last_fullsized":"2024-03-29T08:22:44.616558+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:12:56.478370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"76'1","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.618099+0000","last_change":"2024-03-29T08:22:43.217434+0000","last_active":"2024-03-29T08:22:44.618099+0000","last_peered":"2024-03-29T08:22:44.618099+0000","last_clean":"2024-03-29T08:22:44.618099+0000","last_became_active":"2024-03-29T08:22:43.217181+0000","last_became_peered":"2024-03-29T08:22:43.217181+0000","last_unstale":"2024-03-29T08:22:44.618099+0000","last_undegraded":"2024-03-29T08:22:44.618099+0000","last_fullsized":"2024-03-29T08:22:44.618099+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:27:26.388890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615732+0000","last_change":"2024-03-29T08:22:43.637920+0000","last_active":"2024-03-29T08:22:44.615732+0000","last_peered":"2024-03-29T08:22:44.615732+0000","last_clean":"2024-03-29T08:22:44.615732+0000","last_became_active":"2024-03-29T08:22:43.637352+0000","last_became_peered":"2024-03-29T08:22:43.637352+0000","last_unstale":"2024-03-29T08:22:44.615732+0000","last_undegraded":"2024-03-29T08:22:44.615732+0000","last_fullsized":"2024-03-29T08:22:44.615732+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:55:24.078237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"76'1","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805505+0000","last_change":"2024-03-29T08:22:43.226767+0000","last_active":"2024-03-29T08:22:44.805505+0000","last_peered":"2024-03-29T08:22:44.805505+0000","last_clean":"2024-03-29T08:22:44.805505+0000","last_became_active":"2024-03-29T08:22:43.226508+0000","last_became_peered":"2024-03-29T08:22:43.226508+0000","last_unstale":"2024-03-29T08:22:44.805505+0000","last_undegraded":"2024-03-29T08:22:44.805505+0000","last_fullsized":"2024-03-29T08:22:44.805505+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:25:10.869415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806689+0000","last_change":"2024-03-29T08:22:43.635841+0000","last_active":"2024-03-29T08:22:44.806689+0000","last_peered":"2024-03-29T08:22:44.806689+0000","last_clean":"2024-03-29T08:22:44.806689+0000","last_became_active":"2024-03-29T08:22:43.635564+0000","last_became_peered":"2024-03-29T08:22:43.635564+0000","last_unstale":"2024-03-29T08:22:44.806689+0000","last_undegraded":"2024-03-29T08:22:44.806689+0000","last_fullsized":"2024-03-29T08:22:44.806689+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:27:27.391136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806906+0000","last_change":"2024-03-29T08:22:43.230874+0000","last_active":"2024-03-29T08:22:44.806906+0000","last_peered":"2024-03-29T08:22:44.806906+0000","last_clean":"2024-03-29T08:22:44.806906+0000","last_became_active":"2024-03-29T08:22:43.230488+0000","last_became_peered":"2024-03-29T08:22:43.230488+0000","last_unstale":"2024-03-29T08:22:44.806906+0000","last_undegraded":"2024-03-29T08:22:44.806906+0000","last_fullsized":"2024-03-29T08:22:44.806906+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:16:38.717323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617804+0000","last_change":"2024-03-29T08:22:43.629776+0000","last_active":"2024-03-29T08:22:44.617804+0000","last_peered":"2024-03-29T08:22:44.617804+0000","last_clean":"2024-03-29T08:22:44.617804+0000","last_became_active":"2024-03-29T08:22:43.629522+0000","last_became_peered":"2024-03-29T08:22:43.629522+0000","last_unstale":"2024-03-29T08:22:44.617804+0000","last_undegraded":"2024-03-29T08:22:44.617804+0000","last_fullsized":"2024-03-29T08:22:44.617804+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:19:37.662168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617702+0000","last_change":"2024-03-29T08:22:43.215440+0000","last_active":"2024-03-29T08:22:44.617702+0000","last_peered":"2024-03-29T08:22:44.617702+0000","last_clean":"2024-03-29T08:22:44.617702+0000","last_became_active":"2024-03-29T08:22:43.215169+0000","last_became_peered":"2024-03-29T08:22:43.215169+0000","last_unstale":"2024-03-29T08:22:44.617702+0000","last_undegraded":"2024-03-29T08:22:44.617702+0000","last_fullsized":"2024-03-29T08:22:44.617702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:18:53.744878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805986+0000","last_change":"2024-03-29T08:22:43.642081+0000","last_active":"2024-03-29T08:22:44.805986+0000","last_peered":"2024-03-29T08:22:44.805986+0000","last_clean":"2024-03-29T08:22:44.805986+0000","last_became_active":"2024-03-29T08:22:43.641828+0000","last_became_peered":"2024-03-29T08:22:43.641828+0000","last_unstale":"2024-03-29T08:22:44.805986+0000","last_undegraded":"2024-03-29T08:22:44.805986+0000","last_fullsized":"2024-03-29T08:22:44.805986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:02:13.271666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616953+0000","last_change":"2024-03-29T08:22:43.232918+0000","last_active":"2024-03-29T08:22:44.616953+0000","last_peered":"2024-03-29T08:22:44.616953+0000","last_clean":"2024-03-29T08:22:44.616953+0000","last_became_active":"2024-03-29T08:22:43.232348+0000","last_became_peered":"2024-03-29T08:22:43.232348+0000","last_unstale":"2024-03-29T08:22:44.616953+0000","last_undegraded":"2024-03-29T08:22:44.616953+0000","last_fullsized":"2024-03-29T08:22:44.616953+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:48:19.039327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805896+0000","last_change":"2024-03-29T08:22:43.632088+0000","last_active":"2024-03-29T08:22:44.805896+0000","last_peered":"2024-03-29T08:22:44.805896+0000","last_clean":"2024-03-29T08:22:44.805896+0000","last_became_active":"2024-03-29T08:22:43.629957+0000","last_became_peered":"2024-03-29T08:22:43.629957+0000","last_unstale":"2024-03-29T08:22:44.805896+0000","last_undegraded":"2024-03-29T08:22:44.805896+0000","last_fullsized":"2024-03-29T08:22:44.805896+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:04:10.246667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805702+0000","last_change":"2024-03-29T08:22:43.219123+0000","last_active":"2024-03-29T08:22:44.805702+0000","last_peered":"2024-03-29T08:22:44.805702+0000","last_clean":"2024-03-29T08:22:44.805702+0000","last_became_active":"2024-03-29T08:22:43.218666+0000","last_became_peered":"2024-03-29T08:22:43.218666+0000","last_unstale":"2024-03-29T08:22:44.805702+0000","last_undegraded":"2024-03-29T08:22:44.805702+0000","last_fullsized":"2024-03-29T08:22:44.805702+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:47:43.668163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616541+0000","last_change":"2024-03-29T08:22:43.639765+0000","last_active":"2024-03-29T08:22:44.616541+0000","last_peered":"2024-03-29T08:22:44.616541+0000","last_clean":"2024-03-29T08:22:44.616541+0000","last_became_active":"2024-03-29T08:22:43.639508+0000","last_became_peered":"2024-03-29T08:22:43.639508+0000","last_unstale":"2024-03-29T08:22:44.616541+0000","last_undegraded":"2024-03-29T08:22:44.616541+0000","last_fullsized":"2024-03-29T08:22:44.616541+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:59:13.224813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"76'3","reported_seq":14,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617867+0000","last_change":"2024-03-29T08:22:43.222223+0000","last_active":"2024-03-29T08:22:44.617867+0000","last_peered":"2024-03-29T08:22:44.617867+0000","last_clean":"2024-03-29T08:22:44.617867+0000","last_became_active":"2024-03-29T08:22:43.221960+0000","last_became_peered":"2024-03-29T08:22:43.221960+0000","last_unstale":"2024-03-29T08:22:44.617867+0000","last_undegraded":"2024-03-29T08:22:44.617867+0000","last_fullsized":"2024-03-29T08:22:44.617867+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:06:49.605118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.639060+0000","last_change":"2024-03-29T08:22:43.631179+0000","last_active":"2024-03-29T08:22:43.639060+0000","last_peered":"2024-03-29T08:22:43.639060+0000","last_clean":"2024-03-29T08:22:43.639060+0000","last_became_active":"2024-03-29T08:22:43.629676+0000","last_became_peered":"2024-03-29T08:22:43.629676+0000","last_unstale":"2024-03-29T08:22:43.639060+0000","last_undegraded":"2024-03-29T08:22:43.639060+0000","last_fullsized":"2024-03-29T08:22:43.639060+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:04:11.342755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"76'6","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.807478+0000","last_change":"2024-03-29T08:22:43.225095+0000","last_active":"2024-03-29T08:22:44.807478+0000","last_peered":"2024-03-29T08:22:44.807478+0000","last_clean":"2024-03-29T08:22:44.807478+0000","last_became_active":"2024-03-29T08:22:43.224653+0000","last_became_peered":"2024-03-29T08:22:43.224653+0000","last_unstale":"2024-03-29T08:22:44.807478+0000","last_undegraded":"2024-03-29T08:22:44.807478+0000","last_fullsized":"2024-03-29T08:22:44.807478+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:27:57.366351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615744+0000","last_change":"2024-03-29T08:22:43.639987+0000","last_active":"2024-03-29T08:22:44.615744+0000","last_peered":"2024-03-29T08:22:44.615744+0000","last_clean":"2024-03-29T08:22:44.615744+0000","last_became_active":"2024-03-29T08:22:43.633583+0000","last_became_peered":"2024-03-29T08:22:43.633583+0000","last_unstale":"2024-03-29T08:22:44.615744+0000","last_undegraded":"2024-03-29T08:22:44.615744+0000","last_fullsized":"2024-03-29T08:22:44.615744+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:07:31.259509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"76'2","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.807341+0000","last_change":"2024-03-29T08:22:43.233812+0000","last_active":"2024-03-29T08:22:44.807341+0000","last_peered":"2024-03-29T08:22:44.807341+0000","last_clean":"2024-03-29T08:22:44.807341+0000","last_became_active":"2024-03-29T08:22:43.233506+0000","last_became_peered":"2024-03-29T08:22:43.233506+0000","last_unstale":"2024-03-29T08:22:44.807341+0000","last_undegraded":"2024-03-29T08:22:44.807341+0000","last_fullsized":"2024-03-29T08:22:44.807341+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:33:08.692539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805885+0000","last_change":"2024-03-29T08:22:43.635709+0000","last_active":"2024-03-29T08:22:44.805885+0000","last_peered":"2024-03-29T08:22:44.805885+0000","last_clean":"2024-03-29T08:22:44.805885+0000","last_became_active":"2024-03-29T08:22:43.635450+0000","last_became_peered":"2024-03-29T08:22:43.635450+0000","last_unstale":"2024-03-29T08:22:44.805885+0000","last_undegraded":"2024-03-29T08:22:44.805885+0000","last_fullsized":"2024-03-29T08:22:44.805885+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:56:48.996098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805615+0000","last_change":"2024-03-29T08:22:43.222275+0000","last_active":"2024-03-29T08:22:44.805615+0000","last_peered":"2024-03-29T08:22:44.805615+0000","last_clean":"2024-03-29T08:22:44.805615+0000","last_became_active":"2024-03-29T08:22:43.221852+0000","last_became_peered":"2024-03-29T08:22:43.221852+0000","last_unstale":"2024-03-29T08:22:44.805615+0000","last_undegraded":"2024-03-29T08:22:44.805615+0000","last_fullsized":"2024-03-29T08:22:44.805615+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:41:44.709879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.635930+0000","last_change":"2024-03-29T08:22:43.626610+0000","last_active":"2024-03-29T08:22:43.635930+0000","last_peered":"2024-03-29T08:22:43.635930+0000","last_clean":"2024-03-29T08:22:43.635930+0000","last_became_active":"2024-03-29T08:22:43.626357+0000","last_became_peered":"2024-03-29T08:22:43.626357+0000","last_unstale":"2024-03-29T08:22:43.635930+0000","last_undegraded":"2024-03-29T08:22:43.635930+0000","last_fullsized":"2024-03-29T08:22:43.635930+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:30:19.739080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805489+0000","last_change":"2024-03-29T08:22:43.222495+0000","last_active":"2024-03-29T08:22:44.805489+0000","last_peered":"2024-03-29T08:22:44.805489+0000","last_clean":"2024-03-29T08:22:44.805489+0000","last_became_active":"2024-03-29T08:22:43.222233+0000","last_became_peered":"2024-03-29T08:22:43.222233+0000","last_unstale":"2024-03-29T08:22:44.805489+0000","last_undegraded":"2024-03-29T08:22:44.805489+0000","last_fullsized":"2024-03-29T08:22:44.805489+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:29:13.574338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806420+0000","last_change":"2024-03-29T08:22:43.629285+0000","last_active":"2024-03-29T08:22:44.806420+0000","last_peered":"2024-03-29T08:22:44.806420+0000","last_clean":"2024-03-29T08:22:44.806420+0000","last_became_active":"2024-03-29T08:22:43.628997+0000","last_became_peered":"2024-03-29T08:22:43.628997+0000","last_unstale":"2024-03-29T08:22:44.806420+0000","last_undegraded":"2024-03-29T08:22:44.806420+0000","last_fullsized":"2024-03-29T08:22:44.806420+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:26:04.212702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"76'8","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805269+0000","last_change":"2024-03-29T08:22:43.225743+0000","last_active":"2024-03-29T08:22:44.805269+0000","last_peered":"2024-03-29T08:22:44.805269+0000","last_clean":"2024-03-29T08:22:44.805269+0000","last_became_active":"2024-03-29T08:22:43.225356+0000","last_became_peered":"2024-03-29T08:22:43.225356+0000","last_unstale":"2024-03-29T08:22:44.805269+0000","last_undegraded":"2024-03-29T08:22:44.805269+0000","last_fullsized":"2024-03-29T08:22:44.805269+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:26:22.771461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805650+0000","last_change":"2024-03-29T08:22:43.621232+0000","last_active":"2024-03-29T08:22:44.805650+0000","last_peered":"2024-03-29T08:22:44.805650+0000","last_clean":"2024-03-29T08:22:44.805650+0000","last_became_active":"2024-03-29T08:22:43.620968+0000","last_became_peered":"2024-03-29T08:22:43.620968+0000","last_unstale":"2024-03-29T08:22:44.805650+0000","last_undegraded":"2024-03-29T08:22:44.805650+0000","last_fullsized":"2024-03-29T08:22:44.805650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:06:50.607364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"76'1","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616977+0000","last_change":"2024-03-29T08:22:43.236922+0000","last_active":"2024-03-29T08:22:44.616977+0000","last_peered":"2024-03-29T08:22:44.616977+0000","last_clean":"2024-03-29T08:22:44.616977+0000","last_became_active":"2024-03-29T08:22:43.236713+0000","last_became_peered":"2024-03-29T08:22:43.236713+0000","last_unstale":"2024-03-29T08:22:44.616977+0000","last_undegraded":"2024-03-29T08:22:44.616977+0000","last_fullsized":"2024-03-29T08:22:44.616977+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:10:18.639596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805919+0000","last_change":"2024-03-29T08:22:43.630230+0000","last_active":"2024-03-29T08:22:44.805919+0000","last_peered":"2024-03-29T08:22:44.805919+0000","last_clean":"2024-03-29T08:22:44.805919+0000","last_became_active":"2024-03-29T08:22:43.629583+0000","last_became_peered":"2024-03-29T08:22:43.629583+0000","last_unstale":"2024-03-29T08:22:44.805919+0000","last_undegraded":"2024-03-29T08:22:44.805919+0000","last_fullsized":"2024-03-29T08:22:44.805919+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:01.351488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"76'2","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805727+0000","last_change":"2024-03-29T08:22:43.226205+0000","last_active":"2024-03-29T08:22:44.805727+0000","last_peered":"2024-03-29T08:22:44.805727+0000","last_clean":"2024-03-29T08:22:44.805727+0000","last_became_active":"2024-03-29T08:22:43.225945+0000","last_became_peered":"2024-03-29T08:22:43.225945+0000","last_unstale":"2024-03-29T08:22:44.805727+0000","last_undegraded":"2024-03-29T08:22:44.805727+0000","last_fullsized":"2024-03-29T08:22:44.805727+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:54:36.578351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.807087+0000","last_change":"2024-03-29T08:22:43.625891+0000","last_active":"2024-03-29T08:22:44.807087+0000","last_peered":"2024-03-29T08:22:44.807087+0000","last_clean":"2024-03-29T08:22:44.807087+0000","last_became_active":"2024-03-29T08:22:43.625550+0000","last_became_peered":"2024-03-29T08:22:43.625550+0000","last_unstale":"2024-03-29T08:22:44.807087+0000","last_undegraded":"2024-03-29T08:22:44.807087+0000","last_fullsized":"2024-03-29T08:22:44.807087+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:55.452693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"76'1","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805359+0000","last_change":"2024-03-29T08:22:43.230782+0000","last_active":"2024-03-29T08:22:44.805359+0000","last_peered":"2024-03-29T08:22:44.805359+0000","last_clean":"2024-03-29T08:22:44.805359+0000","last_became_active":"2024-03-29T08:22:43.230501+0000","last_became_peered":"2024-03-29T08:22:43.230501+0000","last_unstale":"2024-03-29T08:22:44.805359+0000","last_undegraded":"2024-03-29T08:22:44.805359+0000","last_fullsized":"2024-03-29T08:22:44.805359+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:58.293985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806472+0000","last_change":"2024-03-29T08:22:43.627874+0000","last_active":"2024-03-29T08:22:44.806472+0000","last_peered":"2024-03-29T08:22:44.806472+0000","last_clean":"2024-03-29T08:22:44.806472+0000","last_became_active":"2024-03-29T08:22:43.627509+0000","last_became_peered":"2024-03-29T08:22:43.627509+0000","last_unstale":"2024-03-29T08:22:44.806472+0000","last_undegraded":"2024-03-29T08:22:44.806472+0000","last_fullsized":"2024-03-29T08:22:44.806472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:53:59.296230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616035+0000","last_change":"2024-03-29T08:22:43.231245+0000","last_active":"2024-03-29T08:22:44.616035+0000","last_peered":"2024-03-29T08:22:44.616035+0000","last_clean":"2024-03-29T08:22:44.616035+0000","last_became_active":"2024-03-29T08:22:43.231003+0000","last_became_peered":"2024-03-29T08:22:43.231003+0000","last_unstale":"2024-03-29T08:22:44.616035+0000","last_undegraded":"2024-03-29T08:22:44.616035+0000","last_fullsized":"2024-03-29T08:22:44.616035+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:13:00.349242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.807450+0000","last_change":"2024-03-29T08:22:43.633345+0000","last_active":"2024-03-29T08:22:44.807450+0000","last_peered":"2024-03-29T08:22:44.807450+0000","last_clean":"2024-03-29T08:22:44.807450+0000","last_became_active":"2024-03-29T08:22:43.632738+0000","last_became_peered":"2024-03-29T08:22:43.632738+0000","last_unstale":"2024-03-29T08:22:44.807450+0000","last_undegraded":"2024-03-29T08:22:44.807450+0000","last_fullsized":"2024-03-29T08:22:44.807450+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:27:37.619123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615857+0000","last_change":"2024-03-29T08:22:43.215283+0000","last_active":"2024-03-29T08:22:44.615857+0000","last_peered":"2024-03-29T08:22:44.615857+0000","last_clean":"2024-03-29T08:22:44.615857+0000","last_became_active":"2024-03-29T08:22:43.214394+0000","last_became_peered":"2024-03-29T08:22:43.214394+0000","last_unstale":"2024-03-29T08:22:44.615857+0000","last_undegraded":"2024-03-29T08:22:44.615857+0000","last_fullsized":"2024-03-29T08:22:44.615857+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:36:33.100721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806789+0000","last_change":"2024-03-29T08:22:43.632269+0000","last_active":"2024-03-29T08:22:44.806789+0000","last_peered":"2024-03-29T08:22:44.806789+0000","last_clean":"2024-03-29T08:22:44.806789+0000","last_became_active":"2024-03-29T08:22:43.631956+0000","last_became_peered":"2024-03-29T08:22:43.631956+0000","last_unstale":"2024-03-29T08:22:44.806789+0000","last_undegraded":"2024-03-29T08:22:44.806789+0000","last_fullsized":"2024-03-29T08:22:44.806789+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:18:54.747124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805203+0000","last_change":"2024-03-29T08:22:43.214550+0000","last_active":"2024-03-29T08:22:44.805203+0000","last_peered":"2024-03-29T08:22:44.805203+0000","last_clean":"2024-03-29T08:22:44.805203+0000","last_became_active":"2024-03-29T08:22:43.214148+0000","last_became_peered":"2024-03-29T08:22:43.214148+0000","last_unstale":"2024-03-29T08:22:44.805203+0000","last_undegraded":"2024-03-29T08:22:44.805203+0000","last_fullsized":"2024-03-29T08:22:44.805203+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:58:28.598523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617916+0000","last_change":"2024-03-29T08:22:43.626561+0000","last_active":"2024-03-29T08:22:44.617916+0000","last_peered":"2024-03-29T08:22:44.617916+0000","last_clean":"2024-03-29T08:22:44.617916+0000","last_became_active":"2024-03-29T08:22:43.625552+0000","last_became_peered":"2024-03-29T08:22:43.625552+0000","last_unstale":"2024-03-29T08:22:44.617916+0000","last_undegraded":"2024-03-29T08:22:44.617916+0000","last_fullsized":"2024-03-29T08:22:44.617916+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:36.191598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.614165+0000","last_change":"2024-03-29T08:22:43.221653+0000","last_active":"2024-03-29T08:22:43.614165+0000","last_peered":"2024-03-29T08:22:43.614165+0000","last_clean":"2024-03-29T08:22:43.614165+0000","last_became_active":"2024-03-29T08:22:43.221327+0000","last_became_peered":"2024-03-29T08:22:43.221327+0000","last_unstale":"2024-03-29T08:22:43.614165+0000","last_undegraded":"2024-03-29T08:22:43.614165+0000","last_fullsized":"2024-03-29T08:22:43.614165+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:27:29.994458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.635984+0000","last_change":"2024-03-29T08:22:43.628071+0000","last_active":"2024-03-29T08:22:43.635984+0000","last_peered":"2024-03-29T08:22:43.635984+0000","last_clean":"2024-03-29T08:22:43.635984+0000","last_became_active":"2024-03-29T08:22:43.627813+0000","last_became_peered":"2024-03-29T08:22:43.627813+0000","last_unstale":"2024-03-29T08:22:43.635984+0000","last_undegraded":"2024-03-29T08:22:43.635984+0000","last_fullsized":"2024-03-29T08:22:43.635984+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:56:43.668049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"76'2","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617031+0000","last_change":"2024-03-29T08:22:43.229417+0000","last_active":"2024-03-29T08:22:44.617031+0000","last_peered":"2024-03-29T08:22:44.617031+0000","last_clean":"2024-03-29T08:22:44.617031+0000","last_became_active":"2024-03-29T08:22:43.229140+0000","last_became_peered":"2024-03-29T08:22:43.229140+0000","last_unstale":"2024-03-29T08:22:44.617031+0000","last_undegraded":"2024-03-29T08:22:44.617031+0000","last_fullsized":"2024-03-29T08:22:44.617031+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:58:00.887698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805886+0000","last_change":"2024-03-29T08:22:43.628003+0000","last_active":"2024-03-29T08:22:44.805886+0000","last_peered":"2024-03-29T08:22:44.805886+0000","last_clean":"2024-03-29T08:22:44.805886+0000","last_became_active":"2024-03-29T08:22:43.627801+0000","last_became_peered":"2024-03-29T08:22:43.627801+0000","last_unstale":"2024-03-29T08:22:44.805886+0000","last_undegraded":"2024-03-29T08:22:44.805886+0000","last_fullsized":"2024-03-29T08:22:44.805886+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:00:32.361795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616868+0000","last_change":"2024-03-29T08:22:43.233295+0000","last_active":"2024-03-29T08:22:44.616868+0000","last_peered":"2024-03-29T08:22:44.616868+0000","last_clean":"2024-03-29T08:22:44.616868+0000","last_became_active":"2024-03-29T08:22:43.230783+0000","last_became_peered":"2024-03-29T08:22:43.230783+0000","last_unstale":"2024-03-29T08:22:44.616868+0000","last_undegraded":"2024-03-29T08:22:44.616868+0000","last_fullsized":"2024-03-29T08:22:44.616868+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:31:09.745872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805346+0000","last_change":"2024-03-29T08:22:43.630577+0000","last_active":"2024-03-29T08:22:44.805346+0000","last_peered":"2024-03-29T08:22:44.805346+0000","last_clean":"2024-03-29T08:22:44.805346+0000","last_became_active":"2024-03-29T08:22:43.629939+0000","last_became_peered":"2024-03-29T08:22:43.629939+0000","last_unstale":"2024-03-29T08:22:44.805346+0000","last_undegraded":"2024-03-29T08:22:44.805346+0000","last_fullsized":"2024-03-29T08:22:44.805346+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:54:42.264431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617887+0000","last_change":"2024-03-29T08:22:43.222144+0000","last_active":"2024-03-29T08:22:44.617887+0000","last_peered":"2024-03-29T08:22:44.617887+0000","last_clean":"2024-03-29T08:22:44.617887+0000","last_became_active":"2024-03-29T08:22:43.221340+0000","last_became_peered":"2024-03-29T08:22:43.221340+0000","last_unstale":"2024-03-29T08:22:44.617887+0000","last_undegraded":"2024-03-29T08:22:44.617887+0000","last_fullsized":"2024-03-29T08:22:44.617887+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:49.976839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805729+0000","last_change":"2024-03-29T08:22:43.632501+0000","last_active":"2024-03-29T08:22:44.805729+0000","last_peered":"2024-03-29T08:22:44.805729+0000","last_clean":"2024-03-29T08:22:44.805729+0000","last_became_active":"2024-03-29T08:22:43.632223+0000","last_became_peered":"2024-03-29T08:22:43.632223+0000","last_unstale":"2024-03-29T08:22:44.805729+0000","last_undegraded":"2024-03-29T08:22:44.805729+0000","last_fullsized":"2024-03-29T08:22:44.805729+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:58:29.600769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.618010+0000","last_change":"2024-03-29T08:22:43.220688+0000","last_active":"2024-03-29T08:22:44.618010+0000","last_peered":"2024-03-29T08:22:44.618010+0000","last_clean":"2024-03-29T08:22:44.618010+0000","last_became_active":"2024-03-29T08:22:43.219423+0000","last_became_peered":"2024-03-29T08:22:43.219423+0000","last_unstale":"2024-03-29T08:22:44.618010+0000","last_undegraded":"2024-03-29T08:22:44.618010+0000","last_fullsized":"2024-03-29T08:22:44.618010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:54:41.262185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.627857+0000","last_change":"2024-03-29T08:22:43.619914+0000","last_active":"2024-03-29T08:22:43.627857+0000","last_peered":"2024-03-29T08:22:43.627857+0000","last_clean":"2024-03-29T08:22:43.627857+0000","last_became_active":"2024-03-29T08:22:43.619631+0000","last_became_peered":"2024-03-29T08:22:43.619631+0000","last_unstale":"2024-03-29T08:22:43.627857+0000","last_undegraded":"2024-03-29T08:22:43.627857+0000","last_fullsized":"2024-03-29T08:22:43.627857+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:42:16.123192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.617860+0000","last_change":"2024-03-29T08:22:43.220643+0000","last_active":"2024-03-29T08:22:44.617860+0000","last_peered":"2024-03-29T08:22:44.617860+0000","last_clean":"2024-03-29T08:22:44.617860+0000","last_became_active":"2024-03-29T08:22:43.220364+0000","last_became_peered":"2024-03-29T08:22:43.220364+0000","last_unstale":"2024-03-29T08:22:44.617860+0000","last_undegraded":"2024-03-29T08:22:44.617860+0000","last_fullsized":"2024-03-29T08:22:44.617860+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:09.901410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.616728+0000","last_change":"2024-03-29T08:22:43.636809+0000","last_active":"2024-03-29T08:22:44.616728+0000","last_peered":"2024-03-29T08:22:44.616728+0000","last_clean":"2024-03-29T08:22:44.616728+0000","last_became_active":"2024-03-29T08:22:43.632811+0000","last_became_peered":"2024-03-29T08:22:43.632811+0000","last_unstale":"2024-03-29T08:22:44.616728+0000","last_undegraded":"2024-03-29T08:22:44.616728+0000","last_fullsized":"2024-03-29T08:22:44.616728+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:12.985868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.614086+0000","last_change":"2024-03-29T08:22:43.226092+0000","last_active":"2024-03-29T08:22:43.614086+0000","last_peered":"2024-03-29T08:22:43.614086+0000","last_clean":"2024-03-29T08:22:43.614086+0000","last_became_active":"2024-03-29T08:22:43.225879+0000","last_became_peered":"2024-03-29T08:22:43.225879+0000","last_unstale":"2024-03-29T08:22:43.614086+0000","last_undegraded":"2024-03-29T08:22:43.614086+0000","last_fullsized":"2024-03-29T08:22:43.614086+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:05:45.594351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.806869+0000","last_change":"2024-03-29T08:22:43.629920+0000","last_active":"2024-03-29T08:22:44.806869+0000","last_peered":"2024-03-29T08:22:44.806869+0000","last_clean":"2024-03-29T08:22:44.806869+0000","last_became_active":"2024-03-29T08:22:43.629622+0000","last_became_peered":"2024-03-29T08:22:43.629622+0000","last_unstale":"2024-03-29T08:22:44.806869+0000","last_undegraded":"2024-03-29T08:22:44.806869+0000","last_fullsized":"2024-03-29T08:22:44.806869+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:38:10.903656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805210+0000","last_change":"2024-03-29T08:22:43.219117+0000","last_active":"2024-03-29T08:22:44.805210+0000","last_peered":"2024-03-29T08:22:44.805210+0000","last_clean":"2024-03-29T08:22:44.805210+0000","last_became_active":"2024-03-29T08:22:43.218434+0000","last_became_peered":"2024-03-29T08:22:43.218434+0000","last_unstale":"2024-03-29T08:22:44.805210+0000","last_undegraded":"2024-03-29T08:22:44.805210+0000","last_fullsized":"2024-03-29T08:22:44.805210+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:02:12.269420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805105+0000","last_change":"2024-03-29T08:22:43.631161+0000","last_active":"2024-03-29T08:22:44.805105+0000","last_peered":"2024-03-29T08:22:44.805105+0000","last_clean":"2024-03-29T08:22:44.805105+0000","last_became_active":"2024-03-29T08:22:43.630549+0000","last_became_peered":"2024-03-29T08:22:43.630549+0000","last_unstale":"2024-03-29T08:22:44.805105+0000","last_undegraded":"2024-03-29T08:22:44.805105+0000","last_fullsized":"2024-03-29T08:22:44.805105+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:59:11.546657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.615628+0000","last_change":"2024-03-29T08:22:43.218296+0000","last_active":"2024-03-29T08:22:44.615628+0000","last_peered":"2024-03-29T08:22:44.615628+0000","last_clean":"2024-03-29T08:22:44.615628+0000","last_became_active":"2024-03-29T08:22:43.218020+0000","last_became_peered":"2024-03-29T08:22:43.218020+0000","last_unstale":"2024-03-29T08:22:44.615628+0000","last_undegraded":"2024-03-29T08:22:44.615628+0000","last_fullsized":"2024-03-29T08:22:44.615628+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:27:26.388890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.636022+0000","last_change":"2024-03-29T08:22:43.628211+0000","last_active":"2024-03-29T08:22:43.636022+0000","last_peered":"2024-03-29T08:22:43.636022+0000","last_clean":"2024-03-29T08:22:43.636022+0000","last_became_active":"2024-03-29T08:22:43.627639+0000","last_became_peered":"2024-03-29T08:22:43.627639+0000","last_unstale":"2024-03-29T08:22:43.636022+0000","last_undegraded":"2024-03-29T08:22:43.636022+0000","last_fullsized":"2024-03-29T08:22:43.636022+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:43.199839+0000","last_clean_scrub_stamp":"2024-03-29T08:22:43.199839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:20:20.240883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"76'2","reported_seq":13,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-03-29T08:22:44.805581+0000","last_change":"2024-03-29T08:22:43.223307+0000","last_active":"2024-03-29T08:22:44.805581+0000","last_peered":"2024-03-29T08:22:44.805581+0000","last_clean":"2024-03-29T08:22:44.805581+0000","last_became_active":"2024-03-29T08:22:43.223039+0000","last_became_peered":"2024-03-29T08:22:43.223039+0000","last_unstale":"2024-03-29T08:22:44.805581+0000","last_undegraded":"2024-03-29T08:22:44.805581+0000","last_fullsized":"2024-03-29T08:22:44.805581+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:22:42.197593+0000","last_clean_scrub_stamp":"2024-03-29T08:22:42.197593+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:26.031908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":211,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-03-29T08:22:43.622161+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:22:43.622161+0000","last_peered":"2024-03-29T08:22:43.622161+0000","last_clean":"2024-03-29T08:22:43.622161+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:22:43.622161+0000","last_undegraded":"2024-03-29T08:22:43.622161+0000","last_fullsized":"2024-03-29T08:22:43.622161+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509500,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":942080,"data_stored":395205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:22:05 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.70099999999999996,"15min":0.66000000000000003},"min":{"1min":0.56599999999999995,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.94799999999999995,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.5},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.72399999999999998,"15min":0.68400000000000005},"min":{"1min":0.628,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.96199999999999997,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.68899999999999995}]},{"osd":1,"last update":"Fri Mar 29 08:22:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68600000000000005,"15min":0.65300000000000002},"min":{"1min":0.52000000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.90700000000000003,"5min":1.202,"15min":1.202},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.69399999999999995,"15min":0.64200000000000002},"min":{"1min":0.57999999999999996,"5min":0.441,"15min":0.441},"max":{"1min":0.89700000000000002,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.748}]},{"osd":2,"last update":"Fri Mar 29 08:22:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.68000000000000005,"15min":0.63900000000000001},"min":{"1min":0.54600000000000004,"5min":0.503,"15min":0.503},"max":{"1min":0.86699999999999999,"5min":1.085,"15min":1.085},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.71399999999999997,"15min":0.67200000000000004},"min":{"1min":0.55500000000000005,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.94199999999999995,"5min":1.288,"15min":1.288},"last":0.72999999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:22:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71899999999999997,"15min":0.70099999999999996},"min":{"1min":0.56200000000000006,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89300000000000002,"5min":1.018,"15min":1.018},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.748,"15min":0.73199999999999998},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.97099999999999997,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.86599999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:22:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61899999999999999,"15min":0.629},"min":{"1min":0.433,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.67700000000000005,"15min":0.71099999999999997},"min":{"1min":0.45400000000000001,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82399999999999995,"5min":1.123,"15min":1.123},"last":0.80000000000000004}]},{"osd":5,"last update":"Fri Mar 29 08:22:05 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68000000000000005,"15min":0.70399999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.91100000000000003,"5min":1.056,"15min":1.056},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.68600000000000005,"15min":0.69799999999999995},"min":{"1min":0.45200000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.88700000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:22:05 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.72899999999999998,"15min":0.72799999999999998},"min":{"1min":0.52000000000000002,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92600000000000005,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.71699999999999997,"15min":0.746},"min":{"1min":0.502,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.88400000000000001,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.84999999999999998}]}]},{"osd":6,"up_from":15,"seq":64424509500,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":946176,"data_stored":395239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.67700000000000005,"15min":0.63900000000000001},"min":{"1min":0.47099999999999997,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86299999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72099999999999997,"15min":0.69099999999999995},"min":{"1min":0.64200000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94199999999999995,"5min":1.079,"15min":1.079},"last":0.83799999999999997}]},{"osd":1,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65100000000000002,"15min":0.61799999999999999},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82299999999999995,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.66500000000000004,"15min":0.63300000000000001},"min":{"1min":0.436,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.99399999999999999,"5min":1.079,"15min":1.079},"last":0.754}]},{"osd":2,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72399999999999998,"15min":0.68200000000000005},"min":{"1min":0.60599999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.95799999999999996,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72399999999999998,"15min":0.70599999999999996},"min":{"1min":0.60699999999999998,"5min":0.435,"15min":0.435},"max":{"1min":0.90400000000000003,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.63200000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70999999999999996,"15min":0.69899999999999995},"min":{"1min":0.48499999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0089999999999999,"5min":1.109,"15min":1.109},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70399999999999996,"15min":0.69499999999999995},"min":{"1min":0.50700000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91400000000000003,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.73599999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64900000000000002,"15min":0.67100000000000004},"min":{"1min":0.39400000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.002,"5min":1.556,"15min":1.556},"last":0.437},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.67500000000000004,"15min":0.72699999999999998},"min":{"1min":0.437,"5min":0.371,"15min":0.371},"max":{"1min":0.81499999999999995,"5min":2.0539999999999998,"15min":2.0539999999999998},"last":0.56999999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.65400000000000003,"15min":0.69499999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.91000000000000003,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.69099999999999995,"15min":0.72599999999999998},"min":{"1min":0.433,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.85699999999999998,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.65200000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.71599999999999997,"15min":0.75800000000000001},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.875,"5min":0.998,"15min":0.998},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.72599999999999998,"15min":0.77000000000000002},"min":{"1min":0.501,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.84999999999999998,"5min":1.109,"15min":1.109},"last":0.67500000000000004}]}]},{"osd":4,"up_from":15,"seq":64424509499,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":929792,"data_stored":388035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85799999999999998,"5min":2.1709999999999998,"15min":2.1709999999999998},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71999999999999997,"15min":0.74199999999999999},"min":{"1min":0.54700000000000004,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.90200000000000002,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.85899999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.624,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.92700000000000005,"5min":2.524,"15min":2.524},"last":1.042},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.71799999999999997},"min":{"1min":0.44500000000000001,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90400000000000003,"5min":2.2679999999999998,"15min":2.2679999999999998},"last":0.67600000000000005}]},{"osd":2,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.72199999999999998,"15min":0.73199999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91300000000000003,"5min":2.363,"15min":2.363},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.746,"5min":0.75800000000000001,"15min":0.76000000000000001},"min":{"1min":0.60599999999999998,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.87,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.76600000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74199999999999999,"15min":0.755},"min":{"1min":0.495,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89300000000000002,"5min":2.3029999999999999,"15min":2.3029999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.755,"5min":0.79400000000000004,"15min":0.81200000000000006},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95199999999999996,"5min":2.2549999999999999,"15min":2.2549999999999999},"last":0.95599999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.66900000000000004,"15min":0.69399999999999995},"min":{"1min":0.46500000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.64300000000000002,"15min":0.65000000000000002},"min":{"1min":0.44600000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.79900000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.88700000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.65100000000000002,"15min":0.67100000000000004},"min":{"1min":0.46999999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.90600000000000003,"5min":0.996,"15min":0.996},"last":1.016},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71899999999999997,"15min":0.74199999999999999},"min":{"1min":0.51500000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.88,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":1.1220000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70599999999999996,"15min":0.73399999999999999},"min":{"1min":0.53200000000000003,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96399999999999997,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74199999999999999,"15min":0.76400000000000001},"min":{"1min":0.55400000000000005,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97799999999999998,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":1.0660000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509502,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":929792,"data_stored":387467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45500000000000002,"15min":0.46300000000000002},"min":{"1min":0.26600000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.84799999999999998,"5min":1.123,"15min":1.123},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.371,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.73799999999999999,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64100000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.44700000000000001,"15min":0.46100000000000002},"min":{"1min":0.31,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.78900000000000003,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.51100000000000001,"15min":0.53300000000000003},"min":{"1min":0.38500000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.004,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.61599999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53700000000000003,"15min":0.56699999999999995},"min":{"1min":0.33200000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.80900000000000005,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.625,"15min":0.65500000000000003},"min":{"1min":0.47399999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.97699999999999998,"5min":1.8580000000000001,"15min":1.8580000000000001},"last":0.73299999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.75700000000000001,"15min":0.79000000000000004},"min":{"1min":0.55900000000000005,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93000000000000005,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.748,"15min":0.77600000000000002},"min":{"1min":0.46999999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.0629999999999999,"5min":1.391,"15min":1.391},"last":0.90500000000000003}]},{"osd":5,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78200000000000003,"15min":0.80400000000000005},"min":{"1min":0.56999999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0409999999999999,"5min":1.671,"15min":1.671},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.82299999999999995,"15min":0.85999999999999999},"min":{"1min":0.58199999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.032,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.0389999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.76600000000000001,"15min":0.82599999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98099999999999998,"5min":1.669,"15min":1.669},"last":0.995},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.78100000000000003,"15min":0.82499999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.075,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.91600000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.80700000000000005,"15min":0.85699999999999998},"min":{"1min":0.56000000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.98899999999999999,"5min":1.887,"15min":1.887},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.79700000000000004,"15min":0.85299999999999998},"min":{"1min":0.53700000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0089999999999999,"5min":2.044,"15min":2.044},"last":0.93899999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509501,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":933888,"data_stored":394223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.47699999999999998,"15min":0.497},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.68500000000000005,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.42899999999999999,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.72599999999999998,"5min":0.751,"15min":0.751},"last":0.54600000000000004}]},{"osd":1,"last update":"Fri Mar 29 08:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.496,"15min":0.497},"min":{"1min":0.34300000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.61899999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.32000000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.74299999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.33400000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.51400000000000001,"15min":0.495},"min":{"1min":0.44700000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.46500000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78500000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.44700000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.77000000000000002,"15min":0.76500000000000001},"min":{"1min":0.59799999999999998,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0089999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.76500000000000001},"min":{"1min":0.48599999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.044,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.82599999999999996,"15min":0.84099999999999997},"min":{"1min":0.66200000000000003,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0109999999999999,"5min":1.306,"15min":1.306},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.82299999999999995,"15min":0.85499999999999998},"min":{"1min":0.55500000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.995,"5min":1.526,"15min":1.526},"last":0.81200000000000006}]},{"osd":6,"last update":"Fri Mar 29 08:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77500000000000002,"15min":0.77900000000000003},"min":{"1min":0.58699999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.97199999999999998,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.77000000000000002,"15min":0.78800000000000003},"min":{"1min":0.53400000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.97399999999999998,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:22:04 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.79500000000000004,"15min":0.80700000000000005},"min":{"1min":0.59099999999999997,"5min":0.505,"15min":0.505},"max":{"1min":0.99199999999999999,"5min":1.319,"15min":1.319},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.77700000000000002,"15min":0.79500000000000004},"min":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.026,"5min":1.216,"15min":1.216},"last":0.60099999999999998}]}]},{"osd":1,"up_from":15,"seq":64424509501,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1523712,"data_stored":984009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.52300000000000002,"15min":0.62},"min":{"1min":0.33100000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.57699999999999996,"5min":4.2539999999999996,"15min":4.2539999999999996},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56899999999999995,"15min":0.64300000000000002},"min":{"1min":0.32900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.84299999999999997,"5min":4.1909999999999998,"15min":4.1909999999999998},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.49299999999999999,"15min":0.50600000000000001},"min":{"1min":0.41499999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52800000000000002,"15min":0.53500000000000003},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66500000000000004,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.496}]},{"osd":3,"last update":"Fri Mar 29 08:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.58299999999999996,"15min":0.58099999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91000000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.78500000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.40200000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.46899999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.755,"15min":0.78600000000000003},"min":{"1min":0.51800000000000002,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0229999999999999,"5min":1.23,"15min":1.23},"last":0.71899999999999997}]},{"osd":5,"last update":"Fri Mar 29 08:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.76100000000000001,"15min":0.77900000000000003},"min":{"1min":0.55200000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.093,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.749,"15min":0.75900000000000001},"min":{"1min":0.53400000000000003,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Mar 29 08:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.80700000000000005,"15min":0.85299999999999998},"min":{"1min":0.58499999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0509999999999999,"5min":1.196,"15min":1.196},"last":0.624}]},{"osd":7,"last update":"Fri Mar 29 08:22:10 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.79000000000000004,"15min":0.80100000000000005},"min":{"1min":0.57199999999999995,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77000000000000002,"15min":0.78800000000000003},"min":{"1min":0.56799999999999995,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.74299999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509500,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":933888,"data_stored":393711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.50700000000000001,"15min":0.59799999999999998},"min":{"1min":0.30299999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58699999999999997,"5min":3.9359999999999999,"15min":3.9359999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.66300000000000003,"15min":0.84299999999999997},"min":{"1min":0.33100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.67500000000000004,"5min":8.0419999999999998,"15min":8.0419999999999998},"last":0.32200000000000001}]},{"osd":2,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.51900000000000002,"15min":0.54900000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.61599999999999999,"5min":1.258,"15min":1.258},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.621,"15min":0.72899999999999998},"min":{"1min":0.40000000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.63,"5min":4.907,"15min":4.907},"last":0.47599999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.66400000000000003,"15min":0.84199999999999997},"min":{"1min":0.33100000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.72699999999999998,"5min":7.9569999999999999,"15min":7.9569999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.60299999999999998,"15min":0.65900000000000003},"min":{"1min":0.46400000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.71299999999999997,"5min":1.397,"15min":1.397},"last":0.36299999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.874,"15min":0.98599999999999999},"min":{"1min":0.59699999999999998,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.0029999999999999,"5min":4.4720000000000004,"15min":4.4720000000000004},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.85699999999999998,"15min":0.96099999999999997},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.0209999999999999,"5min":4.3920000000000003,"15min":4.3920000000000003},"last":0.56599999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.84599999999999997,"15min":0.92900000000000005},"min":{"1min":0.60999999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0880000000000001,"5min":4.3200000000000003,"15min":4.3200000000000003},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80800000000000005,"15min":0.86199999999999999},"min":{"1min":0.60799999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.94999999999999996,"5min":1.6759999999999999,"15min":1.6759999999999999},"last":0.64600000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.82199999999999995,"15min":0.86399999999999999},"min":{"1min":0.64500000000000002,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.98399999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.88200000000000001,"15min":0.98399999999999999},"min":{"1min":0.64500000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0389999999999999,"5min":4.4100000000000001,"15min":4.4100000000000001},"last":0.68600000000000005}]},{"osd":7,"last update":"Fri Mar 29 08:22:08 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.89600000000000002,"15min":1.004},"min":{"1min":0.59799999999999998,"5min":0.496,"15min":0.496},"max":{"1min":1.0669999999999999,"5min":4.5209999999999999,"15min":4.5209999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.872,"15min":0.98399999999999999},"min":{"1min":0.63,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.006,"5min":4.4420000000000002,"15min":4.4420000000000002},"last":0.58499999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542205,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1515520,"data_stored":977799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64900000000000002,"15min":0.63400000000000001},"min":{"1min":0.45000000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.85999999999999999,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65700000000000003,"15min":0.65400000000000003},"min":{"1min":0.48699999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80500000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.75600000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.69799999999999995,"15min":0.73199999999999998},"min":{"1min":0.505,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88400000000000001,"5min":2.5129999999999999,"15min":2.5129999999999999},"last":0.626},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.54900000000000004,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.88100000000000001,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67100000000000004,"15min":0.67300000000000004},"min":{"1min":0.46700000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.81599999999999995,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70499999999999996,"15min":0.70399999999999996},"min":{"1min":0.55900000000000005,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91400000000000003,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.68999999999999995}]},{"osd":3,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.71099999999999997,"15min":0.71399999999999997},"min":{"1min":0.54900000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88700000000000001,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.72699999999999998,"15min":0.745},"min":{"1min":0.52300000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.85999999999999999,"5min":1.954,"15min":1.954},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64500000000000002,"15min":0.65100000000000002},"min":{"1min":0.46100000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83399999999999996,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64700000000000002,"15min":0.66100000000000003},"min":{"1min":0.442,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88300000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.746}]},{"osd":6,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65500000000000003,"15min":0.67800000000000005},"min":{"1min":0.42899999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.81799999999999995,"5min":1.071,"15min":1.071},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.68100000000000005,"15min":0.69699999999999995},"min":{"1min":0.5,"5min":0.432,"15min":0.432},"max":{"1min":0.80000000000000004,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.65000000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:22:07 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69599999999999995,"15min":0.71599999999999997},"min":{"1min":0.495,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.85199999999999998,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.72599999999999998,"15min":0.753},"min":{"1min":0.52300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.872,"5min":1.04,"15min":1.04},"last":0.81499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:22:48.147 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T08:22:48.147 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-29T08:22:48.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd94200640 1 Processor -- start 2024-03-29T08:22:48.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd94200640 1 -- start start 2024-03-29T08:22:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd94200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdd8c0cb120 0x7fdd8c1654f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd94200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8c0b01b0 con 0x7fdd8c0cf2a0 2024-03-29T08:22:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd94200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8c0ae120 con 0x7fdd8c15aab0 2024-03-29T08:22:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd94200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd8c0d1500 con 0x7fdd8c0cb120 2024-03-29T08:22:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd90e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdd8c0cf2a0 0x7fdd8c0ad450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57570/0 (socket says 172.21.15.46:57570) 2024-03-29T08:22:48.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd90e00640 1 -- 172.21.15.46:0/1358742200 learned_addr learned my addr 172.21.15.46:0/1358742200 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd92200640 1 --2- 172.21.15.46:0/1358742200 >> v2:172.21.15.73:3300/0 conn(0x7fdd8c0cb120 0x7fdd8c1654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd8be00640 1 -- 172.21.15.46:0/1358742200 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628299372 0 0) 0x7fdd8c0b01b0 con 0x7fdd8c0cf2a0 2024-03-29T08:22:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd8be00640 1 -- 172.21.15.46:0/1358742200 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd74003530 con 0x7fdd8c0cf2a0 2024-03-29T08:22:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd8be00640 1 -- 172.21.15.46:0/1358742200 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592442747 0 0) 0x7fdd8c0ae120 con 0x7fdd8c15aab0 2024-03-29T08:22:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd8be00640 1 -- 172.21.15.46:0/1358742200 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd8c0b01b0 con 0x7fdd8c15aab0 2024-03-29T08:22:48.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd8be00640 1 -- 172.21.15.46:0/1358742200 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1362776252 0 0) 0x7fdd74003530 con 0x7fdd8c0cf2a0 2024-03-29T08:22:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd8be00640 1 -- 172.21.15.46:0/1358742200 >> v2:172.21.15.73:3300/0 conn(0x7fdd8c0cb120 msgr2=0x7fdd8c1654f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd8be00640 1 --2- 172.21.15.46:0/1358742200 >> v2:172.21.15.73:3300/0 conn(0x7fdd8c0cb120 0x7fdd8c1654f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd8be00640 1 -- 172.21.15.46:0/1358742200 >> v1:172.21.15.73:6789/0 conn(0x7fdd8c15aab0 legacy=0x7fdd8c0ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd8be00640 1 -- 172.21.15.46:0/1358742200 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c0ae3e0 con 0x7fdd8c0cf2a0 2024-03-29T08:22:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd8be00640 1 -- 172.21.15.46:0/1358742200 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fdd78001680 con 0x7fdd8c0cf2a0 2024-03-29T08:22:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd92200640 1 --2- 172.21.15.46:0/1358742200 >> v2:172.21.15.73:3300/0 conn(0x7fdd8c0cb120 0x7fdd8c1654f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:22:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.302+0000 7fdd8be00640 1 -- 172.21.15.46:0/1358742200 >> v1:172.21.15.46:6789/0 conn(0x7fdd8c0cf2a0 legacy=0x7fdd8c0ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd8be00640 1 --2- 172.21.15.46:0/1358742200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd74003b10 0x7fdd74003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd8be00640 1 --2- 172.21.15.46:0/1358742200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd740047d0 0x7fdd74004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd8be00640 1 --2- 172.21.15.46:0/1358742200 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd74005190 0x7fdd7400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd8be00640 1 -- 172.21.15.46:0/1358742200 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c0d1500 con 0x7fdd74003b10 2024-03-29T08:22:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd8be00640 1 -- 172.21.15.46:0/1358742200 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7400c010 con 0x7fdd740047d0 2024-03-29T08:22:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd8be00640 1 -- 172.21.15.46:0/1358742200 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7400c1d0 con 0x7fdd74005190 2024-03-29T08:22:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd91800640 1 --2- 172.21.15.46:0/1358742200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd74003b10 0x7fdd74003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:48.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd90e00640 1 --2- 172.21.15.46:0/1358742200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd740047d0 0x7fdd74004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd92200640 1 --2- 172.21.15.46:0/1358742200 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd74005190 0x7fdd7400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd91800640 1 -- 172.21.15.46:0/1358742200 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd74005190 msgr2=0x7fdd7400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd91800640 1 --2- 172.21.15.46:0/1358742200 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd74005190 0x7fdd7400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd91800640 1 -- 172.21.15.46:0/1358742200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd740047d0 msgr2=0x7fdd74004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd91800640 1 --2- 172.21.15.46:0/1358742200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd740047d0 0x7fdd74004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd91800640 1 -- 172.21.15.46:0/1358742200 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd74003530 con 0x7fdd74003b10 2024-03-29T08:22:48.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd92200640 1 -- 172.21.15.46:0/1358742200 reap_dead start 2024-03-29T08:22:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd91800640 1 --2- 172.21.15.46:0/1358742200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd74003b10 0x7fdd74003f10 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fdd800018d0 tx=0x7fdd8001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec2c2ce6dfc2f42a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd8be00640 1 -- 172.21.15.46:0/1358742200 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd800052d0 con 0x7fdd74003b10 2024-03-29T08:22:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd8be00640 1 -- 172.21.15.46:0/1358742200 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd80005450 con 0x7fdd74003b10 2024-03-29T08:22:48.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.306+0000 7fdd8be00640 1 -- 172.21.15.46:0/1358742200 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80003840 con 0x7fdd74003b10 2024-03-29T08:22:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd94200640 1 -- 172.21.15.46:0/1358742200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd74003b10 msgr2=0x7fdd74003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd94200640 1 --2- 172.21.15.46:0/1358742200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd74003b10 0x7fdd74003f10 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fdd800018d0 tx=0x7fdd8001c540 comp rx=0 tx=0).stop 2024-03-29T08:22:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd94200640 1 -- 172.21.15.46:0/1358742200 shutdown_connections 2024-03-29T08:22:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd94200640 1 --2- 172.21.15.46:0/1358742200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd74003b10 0x7fdd74003f10 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd94200640 1 -- 172.21.15.46:0/1358742200 >> 172.21.15.46:0/1358742200 conn(0x7fdd8c099a10 msgr2=0x7fdd8c089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd94200640 1 -- 172.21.15.46:0/1358742200 shutdown_connections 2024-03-29T08:22:48.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd94200640 1 -- 172.21.15.46:0/1358742200 wait complete. 2024-03-29T08:22:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd94200640 1 Processor -- start 2024-03-29T08:22:48.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd94200640 1 -- start start 2024-03-29T08:22:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd94200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd74003b10 0x7fdd8c06c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd94200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd8c0c7810 0x7fdd8c0c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd94200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd8c154880 0x7fdd8c154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd94200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c0c8f90 con 0x7fdd8c154880 2024-03-29T08:22:48.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd94200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c155240 con 0x7fdd74003b10 2024-03-29T08:22:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd94200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c06cd50 con 0x7fdd8c0c7810 2024-03-29T08:22:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd92200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd8c154880 0x7fdd8c154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd91800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd74003b10 0x7fdd8c06c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd92200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd8c154880 0x7fdd8c154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54578/0 (socket says 172.21.15.46:54578) 2024-03-29T08:22:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd92200640 1 -- 172.21.15.46:0/2743101865 learned_addr learned my addr 172.21.15.46:0/2743101865 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd90e00640 1 --2- 172.21.15.46:0/2743101865 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd8c0c7810 0x7fdd8c0c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd92200640 1 -- 172.21.15.46:0/2743101865 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd8c0c7810 msgr2=0x7fdd8c0c7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd92200640 1 --2- 172.21.15.46:0/2743101865 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd8c0c7810 0x7fdd8c0c7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd92200640 1 -- 172.21.15.46:0/2743101865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd74003b10 msgr2=0x7fdd8c06c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.310+0000 7fdd92200640 1 --2- 172.21.15.46:0/2743101865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd74003b10 0x7fdd8c06c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.314+0000 7fdd92200640 1 -- 172.21.15.46:0/2743101865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c0c8360 con 0x7fdd8c154880 2024-03-29T08:22:48.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.314+0000 7fdd92200640 1 --2- 172.21.15.46:0/2743101865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd8c154880 0x7fdd8c154d00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c0ce880 tx=0x7fdd8400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.314+0000 7fdd89600640 1 -- 172.21.15.46:0/2743101865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400e040 con 0x7fdd8c154880 2024-03-29T08:22:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.314+0000 7fdd89600640 1 -- 172.21.15.46:0/2743101865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8400f040 con 0x7fdd8c154880 2024-03-29T08:22:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.314+0000 7fdd89600640 1 -- 172.21.15.46:0/2743101865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84015670 con 0x7fdd8c154880 2024-03-29T08:22:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.314+0000 7fdd94200640 1 -- 172.21.15.46:0/2743101865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8c0c8640 con 0x7fdd8c154880 2024-03-29T08:22:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.314+0000 7fdd94200640 1 -- 172.21.15.46:0/2743101865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8c0c8c20 con 0x7fdd8c154880 2024-03-29T08:22:48.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.314+0000 7fdd94200640 1 -- 172.21.15.46:0/2743101865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8c0c9140 con 0x7fdd8c154880 2024-03-29T08:22:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.314+0000 7fdd89600640 1 -- 172.21.15.46:0/2743101865 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdd840157d0 con 0x7fdd8c154880 2024-03-29T08:22:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.314+0000 7fdd89600640 1 --2- 172.21.15.46:0/2743101865 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd6c0666c0 0x7fdd6c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:48.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.314+0000 7fdd89600640 1 -- 172.21.15.46:0/2743101865 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd84084650 con 0x7fdd8c154880 2024-03-29T08:22:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.314+0000 7fdd91800640 1 --2- 172.21.15.46:0/2743101865 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd6c0666c0 0x7fdd6c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.318+0000 7fdd91800640 1 --2- 172.21.15.46:0/2743101865 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd6c0666c0 0x7fdd6c068b80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdd800ad000 tx=0x7fdd800a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.318+0000 7fdd89600640 1 -- 172.21.15.46:0/2743101865 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdd8c0c9140 con 0x7fdd8c154880 2024-03-29T08:22:48.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.534+0000 7fdd94200640 1 -- 172.21.15.46:0/2743101865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdd8c0c9140 con 0x7fdd8c154880 2024-03-29T08:22:48.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.538+0000 7fdd89600640 1 -- 172.21.15.46:0/2743101865 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7fdd8c0c9140 con 0x7fdd8c154880 2024-03-29T08:22:48.543 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-03-29T08:22:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.542+0000 7fdd94200640 1 -- 172.21.15.46:0/2743101865 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd6c0666c0 msgr2=0x7fdd6c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.542+0000 7fdd94200640 1 --2- 172.21.15.46:0/2743101865 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd6c0666c0 0x7fdd6c068b80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdd800ad000 tx=0x7fdd800a7000 comp rx=0 tx=0).stop 2024-03-29T08:22:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.542+0000 7fdd94200640 1 -- 172.21.15.46:0/2743101865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd8c154880 msgr2=0x7fdd8c154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.542+0000 7fdd94200640 1 --2- 172.21.15.46:0/2743101865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd8c154880 0x7fdd8c154d00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c0ce880 tx=0x7fdd8400f580 comp rx=0 tx=0).stop 2024-03-29T08:22:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.542+0000 7fdd94200640 1 -- 172.21.15.46:0/2743101865 shutdown_connections 2024-03-29T08:22:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.542+0000 7fdd94200640 1 --2- 172.21.15.46:0/2743101865 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd6c0666c0 0x7fdd6c068b80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.542+0000 7fdd94200640 1 --2- 172.21.15.46:0/2743101865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd8c154880 0x7fdd8c154d00 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.542+0000 7fdd94200640 1 --2- 172.21.15.46:0/2743101865 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd8c0c7810 0x7fdd8c0c7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.542+0000 7fdd94200640 1 --2- 172.21.15.46:0/2743101865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd74003b10 0x7fdd8c06c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.542+0000 7fdd94200640 1 -- 172.21.15.46:0/2743101865 >> 172.21.15.46:0/2743101865 conn(0x7fdd8c099a10 msgr2=0x7fdd8c089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.542+0000 7fdd94200640 1 -- 172.21.15.46:0/2743101865 shutdown_connections 2024-03-29T08:22:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.542+0000 7fdd94200640 1 -- 172.21.15.46:0/2743101865 wait complete. 2024-03-29T08:22:48.565 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:22:48.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70d2c00640 1 Processor -- start 2024-03-29T08:22:48.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70d2c00640 1 -- start start 2024-03-29T08:22:48.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70d2c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f70cc0ce8a0 0x7f70cc0caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:48.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70d2c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70cc0ae810 0x7f70cc0cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:48.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70d2c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70cc0c9bd0 con 0x7f70cc0cec80 2024-03-29T08:22:48.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70d2c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70cc0cbcc0 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70d2c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc0c9eb0 con 0x7f70cc0ae810 2024-03-29T08:22:48.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70d0c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f70cc0cec80 0x7f70cc0aecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57582/0 (socket says 172.21.15.46:57582) 2024-03-29T08:22:48.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70d0c00640 1 -- 172.21.15.46:0/888226807 learned_addr learned my addr 172.21.15.46:0/888226807 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:48.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70cbe00640 1 --2- 172.21.15.46:0/888226807 >> v2:172.21.15.73:3300/0 conn(0x7f70cc0ce8a0 0x7f70cc0caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:48.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70c3400640 1 --2- 172.21.15.46:0/888226807 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70cc0ae810 0x7f70cc0cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70cb400640 1 -- 172.21.15.46:0/888226807 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029782274 0 0) 0x7f70cc0c9bd0 con 0x7f70cc0cec80 2024-03-29T08:22:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70cb400640 1 -- 172.21.15.46:0/888226807 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70b0003530 con 0x7f70cc0cec80 2024-03-29T08:22:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70cbe00640 1 -- 172.21.15.46:0/888226807 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70cc0ae810 msgr2=0x7f70cc0cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70cbe00640 1 --2- 172.21.15.46:0/888226807 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70cc0ae810 0x7f70cc0cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70cbe00640 1 -- 172.21.15.46:0/888226807 >> v1:172.21.15.46:6789/0 conn(0x7f70cc0cec80 legacy=0x7f70cc0aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70cbe00640 1 -- 172.21.15.46:0/888226807 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc0ca190 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.702+0000 7f70c3400640 1 --2- 172.21.15.46:0/888226807 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70cc0ae810 0x7f70cc0cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:22:48.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.706+0000 7f70cbe00640 1 --2- 172.21.15.46:0/888226807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70cc0ce8a0 0x7f70cc0caf50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0097c0 tx=0x7f70bc01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88336f1a30bac2c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.706+0000 7f70cb400640 1 -- 172.21.15.46:0/888226807 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc0ab670 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.706+0000 7f70cb400640 1 -- 172.21.15.46:0/888226807 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70bc0ab7d0 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.706+0000 7f70cb400640 1 -- 172.21.15.46:0/888226807 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70bc0abb60 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.706+0000 7f70d2c00640 1 -- 172.21.15.46:0/888226807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70cc0ce8a0 msgr2=0x7f70cc0caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:48.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.706+0000 7f70d2c00640 1 --2- 172.21.15.46:0/888226807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70cc0ce8a0 0x7f70cc0caf50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0097c0 tx=0x7f70bc01c470 comp rx=0 tx=0).stop 2024-03-29T08:22:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.706+0000 7f70d2c00640 1 -- 172.21.15.46:0/888226807 shutdown_connections 2024-03-29T08:22:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.706+0000 7f70d2c00640 1 --2- 172.21.15.46:0/888226807 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70cc0ae810 0x7f70cc0cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.706+0000 7f70d2c00640 1 --2- 172.21.15.46:0/888226807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70cc0ce8a0 0x7f70cc0caf50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.706+0000 7f70d2c00640 1 -- 172.21.15.46:0/888226807 >> 172.21.15.46:0/888226807 conn(0x7f70cc098a40 msgr2=0x7f70cc0cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.706+0000 7f70d2c00640 1 -- 172.21.15.46:0/888226807 shutdown_connections 2024-03-29T08:22:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.706+0000 7f70d2c00640 1 -- 172.21.15.46:0/888226807 wait complete. 2024-03-29T08:22:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70d2c00640 1 Processor -- start 2024-03-29T08:22:48.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70d2c00640 1 -- start start 2024-03-29T08:22:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70d2c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70cc0ae810 0x7f70cc14de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70d2c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70cc0ce8a0 0x7f70cc0c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70d2c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70cc0cec80 0x7f70cc0b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70d2c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc0c6d50 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70d2c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc0c7030 con 0x7f70cc0ae810 2024-03-29T08:22:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70d2c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc0af900 con 0x7f70cc0cec80 2024-03-29T08:22:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70cc0ce8a0 0x7f70cc0c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70c3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70cc0ce8a0 0x7f70cc0c6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54584/0 (socket says 172.21.15.46:54584) 2024-03-29T08:22:48.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70c3400640 1 -- 172.21.15.46:0/899118406 learned_addr learned my addr 172.21.15.46:0/899118406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70d0c00640 1 --2- 172.21.15.46:0/899118406 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70cc0cec80 0x7f70cc0b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70cbe00640 1 --2- 172.21.15.46:0/899118406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70cc0ae810 0x7f70cc14de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70c3400640 1 -- 172.21.15.46:0/899118406 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70cc0cec80 msgr2=0x7f70cc0b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70c3400640 1 --2- 172.21.15.46:0/899118406 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70cc0cec80 0x7f70cc0b1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70c3400640 1 -- 172.21.15.46:0/899118406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70cc0ae810 msgr2=0x7f70cc14de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70c3400640 1 --2- 172.21.15.46:0/899118406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70cc0ae810 0x7f70cc14de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70c3400640 1 -- 172.21.15.46:0/899118406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b4009420 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70cbe00640 1 --2- 172.21.15.46:0/899118406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70cc0ae810 0x7f70cc14de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70c3400640 1 --2- 172.21.15.46:0/899118406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70cc0ce8a0 0x7f70cc0c6650 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f70b400e980 tx=0x7f70b400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70c8c00640 1 -- 172.21.15.46:0/899118406 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b400ca00 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70c8c00640 1 -- 172.21.15.46:0/899118406 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b400cb60 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70c8c00640 1 -- 172.21.15.46:0/899118406 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b4013470 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70d2c00640 1 -- 172.21.15.46:0/899118406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70bc009420 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.710+0000 7f70d2c00640 1 -- 172.21.15.46:0/899118406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70cc0c7310 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.714+0000 7f70d2c00640 1 -- 172.21.15.46:0/899118406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70cc066c90 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.714+0000 7f70c8c00640 1 -- 172.21.15.46:0/899118406 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f70b40135d0 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.714+0000 7f70c8c00640 1 --2- 172.21.15.46:0/899118406 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f70a8066690 0x7f70a8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:48.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.714+0000 7f70c8c00640 1 -- 172.21.15.46:0/899118406 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70b4084500 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.714+0000 7f70cbe00640 1 --2- 172.21.15.46:0/899118406 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f70a8066690 0x7f70a8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:48.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.714+0000 7f70cbe00640 1 --2- 172.21.15.46:0/899118406 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f70a8066690 0x7f70a8068b50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0af000 tx=0x7f70bc0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:48.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.718+0000 7f70c8c00640 1 -- 172.21.15.46:0/899118406 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f70b40517d0 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.914+0000 7f70d2c00640 1 -- 172.21.15.46:0/899118406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70cc152680 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.914+0000 7f70c8c00640 1 -- 172.21.15.46:0/899118406 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4727 (secure 0 0 0) 0x7f70b4054df0 con 0x7f70cc0ce8a0 2024-03-29T08:22:48.919 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:48.919 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3456771376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3456771376},{"type":"v1","addr":"172.21.15.73:6836","nonce":3456771376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2671538025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2671538025},{"type":"v1","addr":"172.21.15.73:6837","nonce":2671538025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3196015693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3196015693},{"type":"v1","addr":"172.21.15.46:6833","nonce":3196015693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:22:43.605457+0000","modified":"2024-03-29T08:22:46.620088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/9617787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":9617787},{"type":"v1","addr":"172.21.15.46:6835","nonce":9617787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-29T08:22:48.920 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 42 2024-03-29T08:22:48.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.918+0000 7f70d2c00640 1 -- 172.21.15.46:0/899118406 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f70a8066690 msgr2=0x7f70a8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:48.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.918+0000 7f70d2c00640 1 --2- 172.21.15.46:0/899118406 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f70a8066690 0x7f70a8068b50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0af000 tx=0x7f70bc0a7000 comp rx=0 tx=0).stop 2024-03-29T08:22:48.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.918+0000 7f70d2c00640 1 -- 172.21.15.46:0/899118406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70cc0ce8a0 msgr2=0x7f70cc0c6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:48.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.918+0000 7f70d2c00640 1 --2- 172.21.15.46:0/899118406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70cc0ce8a0 0x7f70cc0c6650 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f70b400e980 tx=0x7f70b400ee40 comp rx=0 tx=0).stop 2024-03-29T08:22:48.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.918+0000 7f70d2c00640 1 -- 172.21.15.46:0/899118406 shutdown_connections 2024-03-29T08:22:48.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.918+0000 7f70d2c00640 1 --2- 172.21.15.46:0/899118406 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f70a8066690 0x7f70a8068b50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.918+0000 7f70d2c00640 1 --2- 172.21.15.46:0/899118406 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70cc0cec80 0x7f70cc0b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.918+0000 7f70d2c00640 1 --2- 172.21.15.46:0/899118406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70cc0ce8a0 0x7f70cc0c6650 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.918+0000 7f70d2c00640 1 --2- 172.21.15.46:0/899118406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70cc0ae810 0x7f70cc14de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:48.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.918+0000 7f70d2c00640 1 -- 172.21.15.46:0/899118406 >> 172.21.15.46:0/899118406 conn(0x7f70cc098a40 msgr2=0x7f70cc15b990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:48.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.918+0000 7f70d2c00640 1 -- 172.21.15.46:0/899118406 shutdown_connections 2024-03-29T08:22:48.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:48.918+0000 7f70d2c00640 1 -- 172.21.15.46:0/899118406 wait complete. 2024-03-29T08:22:49.341 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 42, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:22:43.605457+0000', 'modified': '2024-03-29T08:22:46.620088+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6636}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6636': {'gid': 6636, 'name': 'c', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/9617787', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 9617787}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 9617787}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:22:49.341 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T08:22:49.342 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-29T08:22:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.482+0000 7f35f6000640 1 Processor -- start 2024-03-29T08:22:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.482+0000 7f35f6000640 1 -- start start 2024-03-29T08:22:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.482+0000 7f35f6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f35f015c950 0x7f35f006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.482+0000 7f35f6000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f35f0160db0 0x7f35f00cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.482+0000 7f35f6000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f00cfb20 con 0x7f35f00cea30 2024-03-29T08:22:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.482+0000 7f35f6000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f00cef10 con 0x7f35f015c950 2024-03-29T08:22:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.482+0000 7f35f6000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f00cf1f0 con 0x7f35f0160db0 2024-03-29T08:22:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.482+0000 7f35ef400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f35f015c950 0x7f35f006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.482+0000 7f35ef400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f35f015c950 0x7f35f006bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54600/0 (socket says 172.21.15.46:54600) 2024-03-29T08:22:49.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.482+0000 7f35ef400640 1 -- 172.21.15.46:0/2436863201 learned_addr learned my addr 172.21.15.46:0/2436863201 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:49.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.482+0000 7f35ef400640 1 -- 172.21.15.46:0/2436863201 >> v2:172.21.15.73:3300/0 conn(0x7f35f0160db0 msgr2=0x7f35f00cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:49.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.482+0000 7f35efe00640 1 --2- 172.21.15.46:0/2436863201 >> v2:172.21.15.73:3300/0 conn(0x7f35f0160db0 0x7f35f00cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.482+0000 7f35ef400640 1 --2- 172.21.15.46:0/2436863201 >> v2:172.21.15.73:3300/0 conn(0x7f35f0160db0 0x7f35f00cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.482+0000 7f35ef400640 1 -- 172.21.15.46:0/2436863201 >> v1:172.21.15.73:6789/0 conn(0x7f35f00cea30 legacy=0x7f35f01606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:49.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.482+0000 7f35ef400640 1 -- 172.21.15.46:0/2436863201 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f00cf4d0 con 0x7f35f015c950 2024-03-29T08:22:49.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.482+0000 7f35efe00640 1 --2- 172.21.15.46:0/2436863201 >> v2:172.21.15.73:3300/0 conn(0x7f35f0160db0 0x7f35f00cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:22:49.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.486+0000 7f35ef400640 1 --2- 172.21.15.46:0/2436863201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35f015c950 0x7f35f006bbe0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f35f00723a0 tx=0x7f35e001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c08f915b2246cf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:49.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.486+0000 7f35eea00640 1 -- 172.21.15.46:0/2436863201 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e00a89d0 con 0x7f35f015c950 2024-03-29T08:22:49.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.486+0000 7f35eea00640 1 -- 172.21.15.46:0/2436863201 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e00a8b30 con 0x7f35f015c950 2024-03-29T08:22:49.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.486+0000 7f35eea00640 1 -- 172.21.15.46:0/2436863201 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e00b0440 con 0x7f35f015c950 2024-03-29T08:22:49.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.486+0000 7f35f6000640 1 -- 172.21.15.46:0/2436863201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35f015c950 msgr2=0x7f35f006bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:49.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.486+0000 7f35f6000640 1 --2- 172.21.15.46:0/2436863201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35f015c950 0x7f35f006bbe0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f35f00723a0 tx=0x7f35e001ecd0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.486+0000 7f35f6000640 1 -- 172.21.15.46:0/2436863201 shutdown_connections 2024-03-29T08:22:49.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.486+0000 7f35f6000640 1 --2- 172.21.15.46:0/2436863201 >> v2:172.21.15.73:3300/0 conn(0x7f35f0160db0 0x7f35f00cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.486+0000 7f35f6000640 1 --2- 172.21.15.46:0/2436863201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35f015c950 0x7f35f006bbe0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.486+0000 7f35f6000640 1 -- 172.21.15.46:0/2436863201 >> 172.21.15.46:0/2436863201 conn(0x7f35f00b21b0 msgr2=0x7f35f00b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:49.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.486+0000 7f35f6000640 1 -- 172.21.15.46:0/2436863201 shutdown_connections 2024-03-29T08:22:49.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.486+0000 7f35f6000640 1 -- 172.21.15.46:0/2436863201 wait complete. 2024-03-29T08:22:49.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35f6000640 1 Processor -- start 2024-03-29T08:22:49.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35f6000640 1 -- start start 2024-03-29T08:22:49.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35f6000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35f00cea30 0x7f35f00c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35f6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35f015c950 0x7f35f0153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35f6000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f35f0160db0 0x7f35f00ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35f6000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f01589d0 con 0x7f35f015c950 2024-03-29T08:22:49.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35f6000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f0159830 con 0x7f35f00cea30 2024-03-29T08:22:49.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35f6000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f0159cc0 con 0x7f35f0160db0 2024-03-29T08:22:49.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35f015c950 0x7f35f0153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35f015c950 0x7f35f0153950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54608/0 (socket says 172.21.15.46:54608) 2024-03-29T08:22:49.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35e7e00640 1 -- 172.21.15.46:0/1247197945 learned_addr learned my addr 172.21.15.46:0/1247197945 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:49.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35ef400640 1 --2- 172.21.15.46:0/1247197945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35f00cea30 0x7f35f00c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35efe00640 1 --2- 172.21.15.46:0/1247197945 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f35f0160db0 0x7f35f00ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35e7e00640 1 -- 172.21.15.46:0/1247197945 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f35f0160db0 msgr2=0x7f35f00ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:49.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35e7e00640 1 --2- 172.21.15.46:0/1247197945 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f35f0160db0 0x7f35f00ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35e7e00640 1 -- 172.21.15.46:0/1247197945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35f00cea30 msgr2=0x7f35f00c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:49.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35e7e00640 1 --2- 172.21.15.46:0/1247197945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35f00cea30 0x7f35f00c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35e7e00640 1 -- 172.21.15.46:0/1247197945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e00094d0 con 0x7f35f015c950 2024-03-29T08:22:49.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35ef400640 1 --2- 172.21.15.46:0/1247197945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35f00cea30 0x7f35f00c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:49.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35e7e00640 1 --2- 172.21.15.46:0/1247197945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35f015c950 0x7f35f0153950 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f35d800c410 tx=0x7f35d800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:49.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35e7400640 1 -- 172.21.15.46:0/1247197945 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d8011790 con 0x7f35f015c950 2024-03-29T08:22:49.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35e7400640 1 -- 172.21.15.46:0/1247197945 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d80118f0 con 0x7f35f015c950 2024-03-29T08:22:49.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35f6000640 1 -- 172.21.15.46:0/1247197945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f0158c20 con 0x7f35f015c950 2024-03-29T08:22:49.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35e7400640 1 -- 172.21.15.46:0/1247197945 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d8011ce0 con 0x7f35f015c950 2024-03-29T08:22:49.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.490+0000 7f35f6000640 1 -- 172.21.15.46:0/1247197945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f01591e0 con 0x7f35f015c950 2024-03-29T08:22:49.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35e7400640 1 -- 172.21.15.46:0/1247197945 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f35d801c460 con 0x7f35f015c950 2024-03-29T08:22:49.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35e7400640 1 --2- 172.21.15.46:0/1247197945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35cc066690 0x7f35cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35e7400640 1 -- 172.21.15.46:0/1247197945 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35d8084190 con 0x7f35f015c950 2024-03-29T08:22:49.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35ef400640 1 --2- 172.21.15.46:0/1247197945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35cc066690 0x7f35cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35f94aa000 1 Processor -- start 2024-03-29T08:22:49.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35f94aa000 1 -- start start 2024-03-29T08:22:49.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35ef400640 1 --2- 172.21.15.46:0/1247197945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35cc066690 0x7f35cc068b50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f35e00b2000 tx=0x7f35e0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:49.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35f94aa000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f73db0 0x560c70f74190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35f94aa000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f746d0 0x560c70f29f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35f94aa000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f2a4c0 0x560c70f2a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35f94aa000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c70df25a0 con 0x560c70f746d0 2024-03-29T08:22:49.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35f94aa000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c70d7faf0 con 0x560c70f73db0 2024-03-29T08:22:49.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35f94aa000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c70d7d0c0 con 0x560c70f2a4c0 2024-03-29T08:22:49.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f746d0 0x560c70f29f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35ef400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f73db0 0x560c70f74190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f746d0 0x560c70f29f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54612/0 (socket says 172.21.15.46:54612) 2024-03-29T08:22:49.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35e7e00640 1 -- 172.21.15.46:0/619971070 learned_addr learned my addr 172.21.15.46:0/619971070 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:49.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35efe00640 1 --2- 172.21.15.46:0/619971070 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f2a4c0 0x560c70f2a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35e7e00640 1 -- 172.21.15.46:0/619971070 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f2a4c0 msgr2=0x560c70f2a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:49.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35e7e00640 1 --2- 172.21.15.46:0/619971070 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f2a4c0 0x560c70f2a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35e7e00640 1 -- 172.21.15.46:0/619971070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f73db0 msgr2=0x560c70f74190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:49.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.494+0000 7f35e7e00640 1 --2- 172.21.15.46:0/619971070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f73db0 0x560c70f74190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35e7e00640 1 -- 172.21.15.46:0/619971070 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c70a74890 con 0x560c70f746d0 2024-03-29T08:22:49.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35e7e00640 1 --2- 172.21.15.46:0/619971070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f746d0 0x560c70f29f80 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f35d805cfe0 tx=0x7f35d8055820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad66da01c153f7d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:49.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35dfe00640 1 -- 172.21.15.46:0/619971070 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d8057070 con 0x560c70f746d0 2024-03-29T08:22:49.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35dfe00640 1 -- 172.21.15.46:0/619971070 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d8058f50 con 0x560c70f746d0 2024-03-29T08:22:49.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35dfe00640 1 -- 172.21.15.46:0/619971070 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d805d510 con 0x560c70f746d0 2024-03-29T08:22:49.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 1 -- 172.21.15.46:0/619971070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f746d0 msgr2=0x560c70f29f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:49.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 1 --2- 172.21.15.46:0/619971070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f746d0 0x560c70f29f80 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f35d805cfe0 tx=0x7f35d8055820 comp rx=0 tx=0).stop 2024-03-29T08:22:49.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 1 -- 172.21.15.46:0/619971070 shutdown_connections 2024-03-29T08:22:49.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 1 --2- 172.21.15.46:0/619971070 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f2a4c0 0x560c70f2a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 1 --2- 172.21.15.46:0/619971070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f746d0 0x560c70f29f80 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 1 --2- 172.21.15.46:0/619971070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f73db0 0x560c70f74190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 1 -- 172.21.15.46:0/619971070 >> 172.21.15.46:0/619971070 conn(0x560c70f30830 msgr2=0x560c70f28860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:49.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 1 -- 172.21.15.46:0/619971070 shutdown_connections 2024-03-29T08:22:49.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 1 -- 172.21.15.46:0/619971070 wait complete. 2024-03-29T08:22:49.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 1 -- start start 2024-03-29T08:22:49.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 1 Processor -- start 2024-03-29T08:22:49.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70f7a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f73db0 0x560c70f7a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f746d0 0x560c70f2e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c70d789b0 con 0x560c70f2a4c0 2024-03-29T08:22:49.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c70d17c00 con 0x560c70f746d0 2024-03-29T08:22:49.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c70d36070 con 0x560c70f73db0 2024-03-29T08:22:49.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35f94aa000 10 client.0 _ll_register_callbacks cb 0x560c70f160e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:22:49.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70f7a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70f7a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54616/0 (socket says 172.21.15.46:54616) 2024-03-29T08:22:49.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35ef400640 1 -- 172.21.15.46:0/1117711535 learned_addr learned my addr 172.21.15.46:0/1117711535 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:49.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35e7e00640 1 --2- 172.21.15.46:0/1117711535 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f73db0 0x560c70f7a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35efe00640 1 --2- 172.21.15.46:0/1117711535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f746d0 0x560c70f2e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35ef400640 1 -- 172.21.15.46:0/1117711535 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f73db0 msgr2=0x560c70f7a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:49.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35ef400640 1 --2- 172.21.15.46:0/1117711535 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f73db0 0x560c70f7a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35ef400640 1 -- 172.21.15.46:0/1117711535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f746d0 msgr2=0x560c70f2e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:49.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35ef400640 1 --2- 172.21.15.46:0/1117711535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f746d0 0x560c70f2e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35ef400640 1 -- 172.21.15.46:0/1117711535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d8059e70 con 0x560c70f2a4c0 2024-03-29T08:22:49.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.498+0000 7f35efe00640 1 --2- 172.21.15.46:0/1117711535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f746d0 0x560c70f2e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:49.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35ef400640 1 --2- 172.21.15.46:0/1117711535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70f7a4a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f35e0008010 tx=0x7f35e00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdda75842b76aab server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:49.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:22:49.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35dcc00640 1 -- 172.21.15.46:0/1117711535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e00c1040 con 0x560c70f2a4c0 2024-03-29T08:22:49.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35dcc00640 1 -- 172.21.15.46:0/1117711535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35e00c7630 con 0x560c70f2a4c0 2024-03-29T08:22:49.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 1 -- 172.21.15.46:0/1117711535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c70d355f0 con 0x560c70f2a4c0 2024-03-29T08:22:49.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35dcc00640 1 -- 172.21.15.46:0/1117711535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35e00bb420 con 0x560c70f2a4c0 2024-03-29T08:22:49.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 10 client.6777 fetch_fsmap learned FSMap version 42 2024-03-29T08:22:49.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 1 -- 172.21.15.46:0/1117711535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x560c70d3dab0 con 0x560c70f2a4c0 2024-03-29T08:22:49.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35dcc00640 1 -- 172.21.15.46:0/1117711535 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560c70d355f0 con 0x560c70f2a4c0 2024-03-29T08:22:49.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35dcc00640 1 -- 172.21.15.46:0/1117711535 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 42) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f35e00c77f0 con 0x560c70f2a4c0 2024-03-29T08:22:49.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 10 client.6777 fetch_fsmap finished waiting for FSMap version 42 2024-03-29T08:22:49.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 10 client.6777 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:22:49.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 20 client.6777 populate_metadata read hostname 'smithi046' 2024-03-29T08:22:49.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 1 --2- 172.21.15.46:0/1117711535 >> [v2:172.21.15.46:6834/9617787,v1:172.21.15.46:6835/9617787] conn(0x560c70f2ec40 0x560c70f2f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 4 client.6777 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:22:49.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 1 -- 172.21.15.46:0/1117711535 --> [v2:172.21.15.46:6834/9617787,v1:172.21.15.46:6835/9617787] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c70d75790 con 0x560c70f2ec40 2024-03-29T08:22:49.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35e7e00640 1 --2- 172.21.15.46:0/1117711535 >> [v2:172.21.15.46:6834/9617787,v1:172.21.15.46:6835/9617787] conn(0x560c70f2ec40 0x560c70f2f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35e7e00640 1 --2- 172.21.15.46:0/1117711535 >> [v2:172.21.15.46:6834/9617787,v1:172.21.15.46:6835/9617787] conn(0x560c70f2ec40 0x560c70f2f020 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8807f303e3a9a23c server_cookie=884fd1053f0a51cf in_seq=0 out_seq=0 2024-03-29T08:22:49.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35dcc00640 10 client.6777 ms_handle_connect on v2:172.21.15.46:6834/9617787 2024-03-29T08:22:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35dcc00640 1 -- 172.21.15.46:0/1117711535 <== mds.0 v2:172.21.15.46:6834/9617787 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560c70d75790 con 0x560c70f2ec40 2024-03-29T08:22:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35dcc00640 10 client.6777 handle_command_reply: tid=0 2024-03-29T08:22:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 1 client.6777 shutdown 2024-03-29T08:22:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 1 -- 172.21.15.46:0/1117711535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 msgr2=0x560c70f7a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 1 --2- 172.21.15.46:0/1117711535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70f7a4a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f35e0008010 tx=0x7f35e00046d0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 1 -- 172.21.15.46:0/1117711535 shutdown_connections 2024-03-29T08:22:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 1 --2- 172.21.15.46:0/1117711535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f746d0 0x560c70f2e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 1 --2- 172.21.15.46:0/1117711535 >> [v2:172.21.15.46:6834/9617787,v1:172.21.15.46:6835/9617787] conn(0x560c70f2ec40 0x560c70f2f020 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 1 --2- 172.21.15.46:0/1117711535 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f73db0 0x560c70f7a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 1 --2- 172.21.15.46:0/1117711535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70f7a4a0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35dcc00640 0 client.6777 ms_handle_reset on v2:172.21.15.46:6834/9617787 2024-03-29T08:22:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 1 -- 172.21.15.46:0/1117711535 >> 172.21.15.46:0/1117711535 conn(0x560c70f30830 msgr2=0x560c70f28f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 1 -- 172.21.15.46:0/1117711535 shutdown_connections 2024-03-29T08:22:49.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 1 -- 172.21.15.46:0/1117711535 wait complete. 2024-03-29T08:22:49.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.502+0000 7f35f94aa000 20 client.6777 trim_cache size 0 max 16384 2024-03-29T08:22:49.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.518+0000 7f35f94aa000 1 Processor -- start 2024-03-29T08:22:49.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.518+0000 7f35f94aa000 1 -- start start 2024-03-29T08:22:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.518+0000 7f35f94aa000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70f72eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.518+0000 7f35f94aa000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f73db0 0x560c70f733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.518+0000 7f35f94aa000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f746d0 0x560c70f74ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.518+0000 7f35f94aa000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c70a35940 con 0x560c70f2a4c0 2024-03-29T08:22:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.518+0000 7f35f94aa000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c70f80980 con 0x560c70f73db0 2024-03-29T08:22:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.518+0000 7f35f94aa000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c70e493e0 con 0x560c70f746d0 2024-03-29T08:22:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.518+0000 7f35ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70f72eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.518+0000 7f35ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70f72eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54624/0 (socket says 172.21.15.46:54624) 2024-03-29T08:22:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.518+0000 7f35ef400640 1 -- 172.21.15.46:0/3002166501 learned_addr learned my addr 172.21.15.46:0/3002166501 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35efe00640 1 --2- 172.21.15.46:0/3002166501 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f746d0 0x560c70f74ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35e7e00640 1 --2- 172.21.15.46:0/3002166501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f73db0 0x560c70f733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35ef400640 1 -- 172.21.15.46:0/3002166501 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f746d0 msgr2=0x560c70f74ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35ef400640 1 --2- 172.21.15.46:0/3002166501 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f746d0 0x560c70f74ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35ef400640 1 -- 172.21.15.46:0/3002166501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f73db0 msgr2=0x560c70f733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:49.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35ef400640 1 --2- 172.21.15.46:0/3002166501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f73db0 0x560c70f733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35ef400640 1 -- 172.21.15.46:0/3002166501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c70d23c00 con 0x560c70f2a4c0 2024-03-29T08:22:49.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35ef400640 1 --2- 172.21.15.46:0/3002166501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70f72eb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f35e0000c10 tx=0x7f35e00c2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f06aff9c40fdc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:49.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35e4c00640 1 -- 172.21.15.46:0/3002166501 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e00a89d0 con 0x560c70f2a4c0 2024-03-29T08:22:49.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35e4c00640 1 -- 172.21.15.46:0/3002166501 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e00b0a40 con 0x560c70f2a4c0 2024-03-29T08:22:49.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35e4c00640 1 -- 172.21.15.46:0/3002166501 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e00c7630 con 0x560c70f2a4c0 2024-03-29T08:22:49.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 1 -- 172.21.15.46:0/3002166501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 msgr2=0x560c70f72eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:49.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 1 --2- 172.21.15.46:0/3002166501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70f72eb0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f35e0000c10 tx=0x7f35e00c2000 comp rx=0 tx=0).stop 2024-03-29T08:22:49.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 1 -- 172.21.15.46:0/3002166501 shutdown_connections 2024-03-29T08:22:49.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 1 --2- 172.21.15.46:0/3002166501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f73db0 0x560c70f733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 1 --2- 172.21.15.46:0/3002166501 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f746d0 0x560c70f74ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 1 --2- 172.21.15.46:0/3002166501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70f72eb0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 1 -- 172.21.15.46:0/3002166501 >> 172.21.15.46:0/3002166501 conn(0x560c70f30830 msgr2=0x560c70f71790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:49.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 1 -- 172.21.15.46:0/3002166501 shutdown_connections 2024-03-29T08:22:49.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 1 -- 172.21.15.46:0/3002166501 wait complete. 2024-03-29T08:22:49.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 1 -- start start 2024-03-29T08:22:49.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 1 Processor -- start 2024-03-29T08:22:49.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70fce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f73db0 0x560c70f77730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f746d0 0x560c70f77c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c70d7c9b0 con 0x560c70f2a4c0 2024-03-29T08:22:49.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c70d77660 con 0x560c70f746d0 2024-03-29T08:22:49.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c70dbbac0 con 0x560c70f73db0 2024-03-29T08:22:49.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35f94aa000 10 client.0 _ll_register_callbacks cb 0x560c70d3d590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:22:49.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70fce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70fce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54630/0 (socket says 172.21.15.46:54630) 2024-03-29T08:22:49.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35ef400640 1 -- 172.21.15.46:0/2060686653 learned_addr learned my addr 172.21.15.46:0/2060686653 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:49.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35e7e00640 1 --2- 172.21.15.46:0/2060686653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f73db0 0x560c70f77730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.522+0000 7f35efe00640 1 --2- 172.21.15.46:0/2060686653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f746d0 0x560c70f77c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35ef400640 1 -- 172.21.15.46:0/2060686653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f73db0 msgr2=0x560c70f77730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:49.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35ef400640 1 --2- 172.21.15.46:0/2060686653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f73db0 0x560c70f77730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35ef400640 1 -- 172.21.15.46:0/2060686653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f746d0 msgr2=0x560c70f77c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35ef400640 1 --2- 172.21.15.46:0/2060686653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f746d0 0x560c70f77c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35ef400640 1 -- 172.21.15.46:0/2060686653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c70d3dab0 con 0x560c70f2a4c0 2024-03-29T08:22:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35efe00640 1 --2- 172.21.15.46:0/2060686653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f746d0 0x560c70f77c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35ef400640 1 --2- 172.21.15.46:0/2060686653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70fce0b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f35e00203b0 tx=0x7f35e00aa000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d28c4b20a70b37c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:22:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35dcc00640 1 -- 172.21.15.46:0/2060686653 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e00c1030 con 0x560c70f2a4c0 2024-03-29T08:22:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 1 -- 172.21.15.46:0/2060686653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c70d48460 con 0x560c70f2a4c0 2024-03-29T08:22:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35dcc00640 1 -- 172.21.15.46:0/2060686653 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35e00c7640 con 0x560c70f2a4c0 2024-03-29T08:22:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35dcc00640 1 -- 172.21.15.46:0/2060686653 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35e00c7800 con 0x560c70f2a4c0 2024-03-29T08:22:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 10 client.6783 fetch_fsmap learned FSMap version 42 2024-03-29T08:22:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 1 -- 172.21.15.46:0/2060686653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=2+}) v3 -- 0x560c70a74cc0 con 0x560c70f2a4c0 2024-03-29T08:22:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35dcc00640 1 -- 172.21.15.46:0/2060686653 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560c70d48460 con 0x560c70f2a4c0 2024-03-29T08:22:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35dcc00640 1 -- 172.21.15.46:0/2060686653 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 42) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f35e00bb420 con 0x560c70f2a4c0 2024-03-29T08:22:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 10 client.6783 fetch_fsmap finished waiting for FSMap version 42 2024-03-29T08:22:49.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 10 client.6783 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:22:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 20 client.6783 populate_metadata read hostname 'smithi046' 2024-03-29T08:22:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 1 --2- 172.21.15.46:0/2060686653 >> [v2:172.21.15.46:6834/9617787,v1:172.21.15.46:6835/9617787] conn(0x560c70f781b0 0x560c70f78590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 4 client.6783 mds_command: new command op to 6636 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T08:22:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 1 -- 172.21.15.46:0/2060686653 --> [v2:172.21.15.46:6834/9617787,v1:172.21.15.46:6835/9617787] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560c70d419c0 con 0x560c70f781b0 2024-03-29T08:22:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35e7e00640 1 --2- 172.21.15.46:0/2060686653 >> [v2:172.21.15.46:6834/9617787,v1:172.21.15.46:6835/9617787] conn(0x560c70f781b0 0x560c70f78590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35e7e00640 1 --2- 172.21.15.46:0/2060686653 >> [v2:172.21.15.46:6834/9617787,v1:172.21.15.46:6835/9617787] conn(0x560c70f781b0 0x560c70f78590 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21394410e862c3a5 server_cookie=543512d5fffc50d5 in_seq=0 out_seq=0 2024-03-29T08:22:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35dcc00640 10 client.6783 ms_handle_connect on v2:172.21.15.46:6834/9617787 2024-03-29T08:22:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35dcc00640 1 -- 172.21.15.46:0/2060686653 <== mds.0 v2:172.21.15.46:6834/9617787 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x560c70d419c0 con 0x560c70f781b0 2024-03-29T08:22:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35dcc00640 10 client.6783 handle_command_reply: tid=0 2024-03-29T08:22:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 1 client.6783 shutdown 2024-03-29T08:22:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 1 -- 172.21.15.46:0/2060686653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 msgr2=0x560c70fce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 1 --2- 172.21.15.46:0/2060686653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70fce0b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f35e00203b0 tx=0x7f35e00aa000 comp rx=0 tx=0).stop 2024-03-29T08:22:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 1 -- 172.21.15.46:0/2060686653 shutdown_connections 2024-03-29T08:22:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 1 --2- 172.21.15.46:0/2060686653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c70f746d0 0x560c70f77c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 1 --2- 172.21.15.46:0/2060686653 >> [v2:172.21.15.46:6834/9617787,v1:172.21.15.46:6835/9617787] conn(0x560c70f781b0 0x560c70f78590 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 1 --2- 172.21.15.46:0/2060686653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560c70f73db0 0x560c70f77730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 1 --2- 172.21.15.46:0/2060686653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560c70f2a4c0 0x560c70fce0b0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35dcc00640 0 client.6783 ms_handle_reset on v2:172.21.15.46:6834/9617787 2024-03-29T08:22:49.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 1 -- 172.21.15.46:0/2060686653 >> 172.21.15.46:0/2060686653 conn(0x560c70f30830 msgr2=0x560c70f72020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:49.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 1 -- 172.21.15.46:0/2060686653 shutdown_connections 2024-03-29T08:22:49.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 1 -- 172.21.15.46:0/2060686653 wait complete. 2024-03-29T08:22:49.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f94aa000 20 client.6783 trim_cache size 0 max 16384 2024-03-29T08:22:49.534 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:22:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "e9b7960f-ec1f-4787-95c3-c806094d37f0", 2024-03-29T08:22:49.534 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-03-29T08:22:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6636, 2024-03-29T08:22:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-29T08:22:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-29T08:22:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-29T08:22:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 5.9235332859999996, 2024-03-29T08:22:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 42, 2024-03-29T08:22:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 77, 2024-03-29T08:22:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 77, 2024-03-29T08:22:49.535 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 10.231820059 2024-03-29T08:22:49.535 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:22:49.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f6000640 1 -- 172.21.15.46:0/1247197945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35cc066690 msgr2=0x7f35cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:49.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f6000640 1 --2- 172.21.15.46:0/1247197945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35cc066690 0x7f35cc068b50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f35e00b2000 tx=0x7f35e0008040 comp rx=0 tx=0).stop 2024-03-29T08:22:49.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f6000640 1 -- 172.21.15.46:0/1247197945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35f015c950 msgr2=0x7f35f0153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:49.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f6000640 1 --2- 172.21.15.46:0/1247197945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35f015c950 0x7f35f0153950 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f35d800c410 tx=0x7f35d800c8d0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f6000640 1 -- 172.21.15.46:0/1247197945 shutdown_connections 2024-03-29T08:22:49.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f6000640 1 --2- 172.21.15.46:0/1247197945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35cc066690 0x7f35cc068b50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f6000640 1 --2- 172.21.15.46:0/1247197945 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f35f0160db0 0x7f35f00ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f6000640 1 --2- 172.21.15.46:0/1247197945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35f015c950 0x7f35f0153950 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f6000640 1 --2- 172.21.15.46:0/1247197945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35f00cea30 0x7f35f00c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f6000640 1 -- 172.21.15.46:0/1247197945 >> 172.21.15.46:0/1247197945 conn(0x7f35f00b21b0 msgr2=0x7f35f015dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:49.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f6000640 1 -- 172.21.15.46:0/1247197945 shutdown_connections 2024-03-29T08:22:49.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.526+0000 7f35f6000640 1 -- 172.21.15.46:0/1247197945 wait complete. 2024-03-29T08:22:49.549 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:22:49.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.686+0000 7f53edc00640 1 Processor -- start 2024-03-29T08:22:49.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.686+0000 7f53edc00640 1 -- start start 2024-03-29T08:22:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.686+0000 7f53edc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f53e815cd90 0x7f53e806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.686+0000 7f53edc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53e815d940 0x7f53e81597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.686+0000 7f53edc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e806d130 con 0x7f53e81593d0 2024-03-29T08:22:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.686+0000 7f53edc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e815b8f0 con 0x7f53e815cd90 2024-03-29T08:22:49.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.686+0000 7f53edc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e815ad50 con 0x7f53e815d940 2024-03-29T08:22:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.686+0000 7f53de000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f53e815cd90 0x7f53e806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.686+0000 7f53e7400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53e815d940 0x7f53e81597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.686+0000 7f53e6a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f53e81593d0 0x7f53e815d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:48872/0 (socket says 172.21.15.46:48872) 2024-03-29T08:22:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.686+0000 7f53e6a00640 1 -- 172.21.15.46:0/2245204060 learned_addr learned my addr 172.21.15.46:0/2245204060 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.686+0000 7f53de000640 1 -- 172.21.15.46:0/2245204060 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53e815d940 msgr2=0x7f53e81597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:49.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.686+0000 7f53de000640 1 --2- 172.21.15.46:0/2245204060 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53e815d940 0x7f53e81597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.686+0000 7f53de000640 1 -- 172.21.15.46:0/2245204060 >> v1:172.21.15.73:6789/0 conn(0x7f53e81593d0 legacy=0x7f53e815d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:49.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.686+0000 7f53de000640 1 -- 172.21.15.46:0/2245204060 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e815afc0 con 0x7f53e815cd90 2024-03-29T08:22:49.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.690+0000 7f53de000640 1 --2- 172.21.15.46:0/2245204060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53e815cd90 0x7f53e806cbf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f53d00097c0 tx=0x7f53d001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb5d8c93a7fd3ca server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.690+0000 7f53e6000640 1 -- 172.21.15.46:0/2245204060 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d00a6040 con 0x7f53e815cd90 2024-03-29T08:22:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.690+0000 7f53e6000640 1 -- 172.21.15.46:0/2245204060 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d00aa600 con 0x7f53e815cd90 2024-03-29T08:22:49.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.690+0000 7f53e6000640 1 -- 172.21.15.46:0/2245204060 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d00aa990 con 0x7f53e815cd90 2024-03-29T08:22:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.690+0000 7f53edc00640 1 -- 172.21.15.46:0/2245204060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53e815cd90 msgr2=0x7f53e806cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.690+0000 7f53edc00640 1 --2- 172.21.15.46:0/2245204060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53e815cd90 0x7f53e806cbf0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f53d00097c0 tx=0x7f53d001c4a0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.690+0000 7f53edc00640 1 -- 172.21.15.46:0/2245204060 shutdown_connections 2024-03-29T08:22:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.690+0000 7f53edc00640 1 --2- 172.21.15.46:0/2245204060 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53e815d940 0x7f53e81597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.690+0000 7f53edc00640 1 --2- 172.21.15.46:0/2245204060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53e815cd90 0x7f53e806cbf0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.690+0000 7f53edc00640 1 -- 172.21.15.46:0/2245204060 >> 172.21.15.46:0/2245204060 conn(0x7f53e8066110 msgr2=0x7f53e8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.690+0000 7f53edc00640 1 -- 172.21.15.46:0/2245204060 shutdown_connections 2024-03-29T08:22:49.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.690+0000 7f53edc00640 1 -- 172.21.15.46:0/2245204060 wait complete. 2024-03-29T08:22:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53edc00640 1 Processor -- start 2024-03-29T08:22:49.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53edc00640 1 -- start start 2024-03-29T08:22:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53edc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53e81593d0 0x7f53e813de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53edc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53e815cd90 0x7f53e80af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53edc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e815d940 0x7f53e80af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53edc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e8151d70 con 0x7f53e815d940 2024-03-29T08:22:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53edc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e813e3d0 con 0x7f53e81593d0 2024-03-29T08:22:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53edc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e8151110 con 0x7f53e815cd90 2024-03-29T08:22:49.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53e7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e815d940 0x7f53e80af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53e6a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53e81593d0 0x7f53e813de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53e7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e815d940 0x7f53e80af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54644/0 (socket says 172.21.15.46:54644) 2024-03-29T08:22:49.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53e7400640 1 -- 172.21.15.46:0/3995735774 learned_addr learned my addr 172.21.15.46:0/3995735774 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:49.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53de000640 1 --2- 172.21.15.46:0/3995735774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53e815cd90 0x7f53e80af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53e7400640 1 -- 172.21.15.46:0/3995735774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53e815cd90 msgr2=0x7f53e80af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:49.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53e7400640 1 --2- 172.21.15.46:0/3995735774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53e815cd90 0x7f53e80af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53e7400640 1 -- 172.21.15.46:0/3995735774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53e81593d0 msgr2=0x7f53e813de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:49.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53e7400640 1 --2- 172.21.15.46:0/3995735774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53e81593d0 0x7f53e813de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53e7400640 1 -- 172.21.15.46:0/3995735774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d0009420 con 0x7f53e815d940 2024-03-29T08:22:49.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53e7400640 1 --2- 172.21.15.46:0/3995735774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e815d940 0x7f53e80af8a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f53e000efc0 tx=0x7f53e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:49.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53df400640 1 -- 172.21.15.46:0/3995735774 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0014480 con 0x7f53e815d940 2024-03-29T08:22:49.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53df400640 1 -- 172.21.15.46:0/3995735774 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e000f040 con 0x7f53e815d940 2024-03-29T08:22:49.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53edc00640 1 -- 172.21.15.46:0/3995735774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e81513d0 con 0x7f53e815d940 2024-03-29T08:22:49.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53df400640 1 -- 172.21.15.46:0/3995735774 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0014820 con 0x7f53e815d940 2024-03-29T08:22:49.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.694+0000 7f53edc00640 1 -- 172.21.15.46:0/3995735774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e8151990 con 0x7f53e815d940 2024-03-29T08:22:49.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.698+0000 7f53edc00640 1 -- 172.21.15.46:0/3995735774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e8151eb0 con 0x7f53e815d940 2024-03-29T08:22:49.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.698+0000 7f53df400640 1 -- 172.21.15.46:0/3995735774 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f53e0014980 con 0x7f53e815d940 2024-03-29T08:22:49.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.698+0000 7f53df400640 1 --2- 172.21.15.46:0/3995735774 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53c40666c0 0x7f53c4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:49.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.698+0000 7f53df400640 1 -- 172.21.15.46:0/3995735774 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53e0084b80 con 0x7f53e815d940 2024-03-29T08:22:49.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.698+0000 7f53e6a00640 1 --2- 172.21.15.46:0/3995735774 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53c40666c0 0x7f53c4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:49.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.698+0000 7f53e6a00640 1 --2- 172.21.15.46:0/3995735774 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53c40666c0 0x7f53c4068b80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f53d8000f70 tx=0x7f53d8007e90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:49.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.702+0000 7f53df400640 1 -- 172.21.15.46:0/3995735774 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f53e0051f90 con 0x7f53e815d940 2024-03-29T08:22:49.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.898+0000 7f53edc00640 1 -- 172.21.15.46:0/3995735774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f53e8066c90 con 0x7f53e815d940 2024-03-29T08:22:49.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.898+0000 7f53df400640 1 -- 172.21.15.46:0/3995735774 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v42) v1 ==== 53+0+73 (secure 0 0 0) 0x7f53e0084e60 con 0x7f53e815d940 2024-03-29T08:22:49.903 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:22:49.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.902+0000 7f53edc00640 1 -- 172.21.15.46:0/3995735774 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53c40666c0 msgr2=0x7f53c4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:49.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.902+0000 7f53edc00640 1 --2- 172.21.15.46:0/3995735774 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53c40666c0 0x7f53c4068b80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f53d8000f70 tx=0x7f53d8007e90 comp rx=0 tx=0).stop 2024-03-29T08:22:49.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.902+0000 7f53edc00640 1 -- 172.21.15.46:0/3995735774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e815d940 msgr2=0x7f53e80af8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:49.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.902+0000 7f53edc00640 1 --2- 172.21.15.46:0/3995735774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e815d940 0x7f53e80af8a0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f53e000efc0 tx=0x7f53e000c490 comp rx=0 tx=0).stop 2024-03-29T08:22:49.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.902+0000 7f53edc00640 1 -- 172.21.15.46:0/3995735774 shutdown_connections 2024-03-29T08:22:49.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.902+0000 7f53edc00640 1 --2- 172.21.15.46:0/3995735774 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53c40666c0 0x7f53c4068b80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.902+0000 7f53edc00640 1 --2- 172.21.15.46:0/3995735774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53e815d940 0x7f53e80af8a0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.902+0000 7f53edc00640 1 --2- 172.21.15.46:0/3995735774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53e815cd90 0x7f53e80af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.902+0000 7f53edc00640 1 --2- 172.21.15.46:0/3995735774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53e81593d0 0x7f53e813de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:49.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.902+0000 7f53edc00640 1 -- 172.21.15.46:0/3995735774 >> 172.21.15.46:0/3995735774 conn(0x7f53e8066110 msgr2=0x7f53e8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:49.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.902+0000 7f53edc00640 1 -- 172.21.15.46:0/3995735774 shutdown_connections 2024-03-29T08:22:49.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:49.902+0000 7f53edc00640 1 -- 172.21.15.46:0/3995735774 wait complete. 2024-03-29T08:22:49.923 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:22:49.924 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:22:49.924 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-29T08:22:49.924 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T08:22:49.924 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:22:49.924 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:22:49.924 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:22:49.924 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:22:49.924 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:22:49.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:49.924 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-29T08:22:49.936 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:22:49.937 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:22:49.937 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:22:49.937 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:22:49.937 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:22:49.938 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:22:49.938 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:22:49.938 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:22:49.939 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-29T08:22:49.939 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:22:49.939 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:22:49.939 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-29T08:22:49.940 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:22:49.940 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:22:49.940 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-29T08:22:49.941 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T08:22:49.941 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42167sec preferred_lft 42167sec 2024-03-29T08:22:49.941 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-29T08:22:49.941 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:22:49.941 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:22:49.941 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:22:49.941 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-29T08:22:49.941 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:22:49.941 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:d9:58:09:4c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:22:49.941 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:22:49.941 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:22:49.941 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:22:49.941 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T08:22:49.942 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T08:22:49.942 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:22:49.942 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T08:22:49.942 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:22:49.942 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:22:49.942 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:22:49.942 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T08:22:49.942 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:22:49.942 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:22:49.942 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T08:22:49.942 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-29T08:22:49.942 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:22:49.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:49.943 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:22:49.997 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:22:49.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:49.999 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-29T08:22:50.052 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-29T08:22:50.052 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:22:50.052 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T08:22:50.052 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:22:50.052 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T08:22:50.052 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:22:50.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:50.053 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:22:50.053 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:22:50.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:22:50.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:22:50.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:22:50.053 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:22:50.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:50.146 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:22:50.155 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T08:22:50.155 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T08:22:50.156 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:22:50.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:50.156 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-03-29T08:22:50.219 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T08:22:50.219 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:22:50.219 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T08:22:50.271 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:22:50.272 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:22:50.272 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:22:50.327 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:22:50.394 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:22:50.394 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:22:50.457 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:22:50.458 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:22:50.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:50.459 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:22:50.512 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:22:50.512 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T08:22:50.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:22:50.578 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:22:50.578 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:22:50.638 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:22:50.640 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:22:50.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:50.641 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:22:50.652 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-03-29T08:22:50.650+0000 7fc69f079580 -1 init, newargv = 0x5601dec4d040 newargc=15 2024-03-29T08:22:50.652 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[41766]: starting ceph client 2024-03-29T08:22:50.659 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[41766]: starting fuse 2024-03-29T08:22:50.660 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:22:50.661 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-29T08:22:50.661 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:22:50.661 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-03-29T08:22:50.661 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-03-29T08:22:50.661 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:22:50.661 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-03-29T08:22:50.661 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:22:50.661 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-03-29T08:22:50.661 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T08:22:50.661 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-03-29T08:22:50.661 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:22:50.662 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:22:50.662 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-03-29T08:22:50.662 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-03-29T08:22:50.662 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-03-29T08:22:50.662 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-03-29T08:22:50.662 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:22:50.662 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-03-29T08:22:50.662 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:22:50.662 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:22:50.662 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:22:50.662 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-03-29T08:22:50.662 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-03-29T08:22:50.662 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-03-29T08:22:50.662 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:22:50.663 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:22:50.663 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-03-29T08:22:50.663 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:22:50.784 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.41766.asok 2024-03-29T08:22:50.967 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41766.asok 2024-03-29T08:22:50.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:50.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41766.asok status 2024-03-29T08:22:51.143 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:22:51.143 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-03-29T08:22:51.143 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:22:51.143 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:22:51.143 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-03-29T08:22:51.143 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-03-29T08:22:51.143 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T08:22:51.143 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "41766", 2024-03-29T08:22:51.143 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-03-29T08:22:51.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:51.144 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-03-29T08:22:51.144 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-03-29T08:22:51.144 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6795, 2024-03-29T08:22:51.144 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-03-29T08:22:51.144 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-03-29T08:22:51.144 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-03-29T08:22:51.144 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 6795 2024-03-29T08:22:51.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:51.144 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:22:51.144 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:22:51.144 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:22:51.144 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3229319948 2024-03-29T08:22:51.144 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:22:51.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:51.145 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:22:51.145 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:22:51.145 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:22:51.145 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3229319948 2024-03-29T08:22:51.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:22:51.145 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.6795 192.168.0.1:0/3229319948", 2024-03-29T08:22:51.145 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/3229319948", 2024-03-29T08:22:51.145 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-03-29T08:22:51.145 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 42, 2024-03-29T08:22:51.145 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 77, 2024-03-29T08:22:51.145 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:22:51.145 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-03-29T08:22:51.146 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-03-29T08:22:51.146 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:22:51.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:51.158 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-03-29T08:22:51.158 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:22:51.167 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-03-29T08:22:51.167 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T08:22:51.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:51.168 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:22:51.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:51.232 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-03-29T08:22:51.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:51.289 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-03-29T08:22:51.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:51.343 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-03-29T08:22:51.397 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:22:51.397 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:22:51.398 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:22:51.398 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:22:51.398 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-03-29T08:22:51.398 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:22:51.506 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627776 2024-03-29T08:22:51.696 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:22:51.696 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:22:51.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:22:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.930+0000 7f9de3000640 1 Processor -- start 2024-03-29T08:22:51.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.930+0000 7f9de3000640 1 -- start start 2024-03-29T08:22:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.934+0000 7f9de3000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ddc15ada0 0x7f9ddc0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.934+0000 7f9de3000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ddc0ce950 0x7f9ddc06c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.934+0000 7f9de3000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ddc0cb510 con 0x7f9ddc144210 2024-03-29T08:22:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.934+0000 7f9de3000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ddc0cede0 con 0x7f9ddc15ada0 2024-03-29T08:22:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.934+0000 7f9de3000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc070580 con 0x7f9ddc0ce950 2024-03-29T08:22:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.934+0000 7f9dd3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ddc15ada0 0x7f9ddc0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.934+0000 7f9ddbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9ddc144210 0x7f9ddc0cb8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57598/0 (socket says 172.21.15.46:57598) 2024-03-29T08:22:51.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.934+0000 7f9ddbe00640 1 -- 172.21.15.46:0/1660180649 learned_addr learned my addr 172.21.15.46:0/1660180649 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.934+0000 7f9dd3e00640 1 -- 172.21.15.46:0/1660180649 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ddc0ce950 msgr2=0x7f9ddc06c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.934+0000 7f9de1000640 1 --2- 172.21.15.46:0/1660180649 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ddc0ce950 0x7f9ddc06c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.934+0000 7f9dd3e00640 1 --2- 172.21.15.46:0/1660180649 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ddc0ce950 0x7f9ddc06c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.934+0000 7f9dd3e00640 1 -- 172.21.15.46:0/1660180649 >> v1:172.21.15.46:6789/0 conn(0x7f9ddc144210 legacy=0x7f9ddc0cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:51.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.934+0000 7f9ddb400640 1 -- 172.21.15.46:0/1660180649 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14141562 0 0) 0x7f9ddc0cb510 con 0x7f9ddc144210 2024-03-29T08:22:51.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.934+0000 7f9dd3e00640 1 -- 172.21.15.46:0/1660180649 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc070860 con 0x7f9ddc15ada0 2024-03-29T08:22:51.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.934+0000 7f9de1000640 1 --2- 172.21.15.46:0/1660180649 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ddc0ce950 0x7f9ddc06c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:22:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9dd3e00640 1 --2- 172.21.15.46:0/1660180649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ddc15ada0 0x7f9ddc0ce410 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40097c0 tx=0x7f9dc401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36bf766a4cf175be server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9ddb400640 1 -- 172.21.15.46:0/1660180649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40ad640 con 0x7f9ddc15ada0 2024-03-29T08:22:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9ddb400640 1 -- 172.21.15.46:0/1660180649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc40ad7a0 con 0x7f9ddc15ada0 2024-03-29T08:22:51.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9ddb400640 1 -- 172.21.15.46:0/1660180649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc40adb30 con 0x7f9ddc15ada0 2024-03-29T08:22:51.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9de3000640 1 -- 172.21.15.46:0/1660180649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ddc15ada0 msgr2=0x7f9ddc0ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:51.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9de3000640 1 --2- 172.21.15.46:0/1660180649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ddc15ada0 0x7f9ddc0ce410 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40097c0 tx=0x7f9dc401c4a0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9de3000640 1 -- 172.21.15.46:0/1660180649 shutdown_connections 2024-03-29T08:22:51.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9de3000640 1 --2- 172.21.15.46:0/1660180649 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ddc0ce950 0x7f9ddc06c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9de3000640 1 --2- 172.21.15.46:0/1660180649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ddc15ada0 0x7f9ddc0ce410 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9de3000640 1 -- 172.21.15.46:0/1660180649 >> 172.21.15.46:0/1660180649 conn(0x7f9ddc066110 msgr2=0x7f9ddc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:51.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9de3000640 1 -- 172.21.15.46:0/1660180649 shutdown_connections 2024-03-29T08:22:51.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9de3000640 1 -- 172.21.15.46:0/1660180649 wait complete. 2024-03-29T08:22:51.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9de3000640 1 Processor -- start 2024-03-29T08:22:51.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9de3000640 1 -- start start 2024-03-29T08:22:51.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9de3000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ddc0ce950 0x7f9ddc138b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:51.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9de3000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ddc144210 0x7f9ddc13e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:51.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9de3000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ddc15ada0 0x7f9ddc154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:51.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9de3000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc164290 con 0x7f9ddc144210 2024-03-29T08:22:51.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9de3000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc0b3000 con 0x7f9ddc0ce950 2024-03-29T08:22:51.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9de3000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc163980 con 0x7f9ddc15ada0 2024-03-29T08:22:51.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9dd3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ddc144210 0x7f9ddc13e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:51.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9dd3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ddc144210 0x7f9ddc13e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54660/0 (socket says 172.21.15.46:54660) 2024-03-29T08:22:51.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9dd3e00640 1 -- 172.21.15.46:0/927448551 learned_addr learned my addr 172.21.15.46:0/927448551 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:51.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9de1000640 1 --2- 172.21.15.46:0/927448551 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ddc15ada0 0x7f9ddc154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:51.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.938+0000 7f9ddbe00640 1 --2- 172.21.15.46:0/927448551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ddc0ce950 0x7f9ddc138b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:51.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.942+0000 7f9dd3e00640 1 -- 172.21.15.46:0/927448551 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ddc15ada0 msgr2=0x7f9ddc154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:51.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.942+0000 7f9dd3e00640 1 --2- 172.21.15.46:0/927448551 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ddc15ada0 0x7f9ddc154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.942+0000 7f9dd3e00640 1 -- 172.21.15.46:0/927448551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ddc0ce950 msgr2=0x7f9ddc138b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:51.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.942+0000 7f9dd3e00640 1 --2- 172.21.15.46:0/927448551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ddc0ce950 0x7f9ddc138b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:51.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.942+0000 7f9dd3e00640 1 -- 172.21.15.46:0/927448551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc4009420 con 0x7f9ddc144210 2024-03-29T08:22:51.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.942+0000 7f9dd3e00640 1 --2- 172.21.15.46:0/927448551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ddc144210 0x7f9ddc13e750 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40097c0 tx=0x7f9dc401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:51.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.942+0000 7f9dd8c00640 1 -- 172.21.15.46:0/927448551 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4004230 con 0x7f9ddc144210 2024-03-29T08:22:51.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.942+0000 7f9dd8c00640 1 -- 172.21.15.46:0/927448551 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc40a6070 con 0x7f9ddc144210 2024-03-29T08:22:51.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.942+0000 7f9dd8c00640 1 -- 172.21.15.46:0/927448551 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4003620 con 0x7f9ddc144210 2024-03-29T08:22:51.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.942+0000 7f9de3000640 1 -- 172.21.15.46:0/927448551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ddc0b2d20 con 0x7f9ddc144210 2024-03-29T08:22:51.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.942+0000 7f9de3000640 1 -- 172.21.15.46:0/927448551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ddc163eb0 con 0x7f9ddc144210 2024-03-29T08:22:51.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.942+0000 7f9de3000640 1 -- 172.21.15.46:0/927448551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ddc0ce280 con 0x7f9ddc144210 2024-03-29T08:22:51.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.942+0000 7f9dd8c00640 1 -- 172.21.15.46:0/927448551 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9dc4003780 con 0x7f9ddc144210 2024-03-29T08:22:51.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.942+0000 7f9dd8c00640 1 --2- 172.21.15.46:0/927448551 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9db8066690 0x7f9db8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:51.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.942+0000 7f9dd8c00640 1 -- 172.21.15.46:0/927448551 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9dc411dcc0 con 0x7f9ddc144210 2024-03-29T08:22:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.946+0000 7f9ddbe00640 1 --2- 172.21.15.46:0/927448551 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9db8066690 0x7f9db8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:51.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.946+0000 7f9ddbe00640 1 --2- 172.21.15.46:0/927448551 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9db8066690 0x7f9db8068b50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc007fd0 tx=0x7f9dcc007d60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:51.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:51.946+0000 7f9dd8c00640 1 -- 172.21.15.46:0/927448551 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9dc4124080 con 0x7f9ddc144210 2024-03-29T08:22:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.110+0000 7f9de3000640 1 -- 172.21.15.46:0/927448551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ddc1643d0 con 0x7f9ddc144210 2024-03-29T08:22:52.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.110+0000 7f9dd8c00640 1 -- 172.21.15.46:0/927448551 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+15992 (secure 0 0 0) 0x7f9ddc1643d0 con 0x7f9ddc144210 2024-03-29T08:22:52.115 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:52.115 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":77,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:22:44.608017+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:22:42.147364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-29T08:22:42.602168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:22:52.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.114+0000 7f9de3000640 1 -- 172.21.15.46:0/927448551 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9db8066690 msgr2=0x7f9db8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.114+0000 7f9de3000640 1 --2- 172.21.15.46:0/927448551 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9db8066690 0x7f9db8068b50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc007fd0 tx=0x7f9dcc007d60 comp rx=0 tx=0).stop 2024-03-29T08:22:52.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.114+0000 7f9de3000640 1 -- 172.21.15.46:0/927448551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ddc144210 msgr2=0x7f9ddc13e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.114+0000 7f9de3000640 1 --2- 172.21.15.46:0/927448551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ddc144210 0x7f9ddc13e750 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40097c0 tx=0x7f9dc401ace0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.114+0000 7f9de3000640 1 -- 172.21.15.46:0/927448551 shutdown_connections 2024-03-29T08:22:52.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.114+0000 7f9de3000640 1 --2- 172.21.15.46:0/927448551 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9db8066690 0x7f9db8068b50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.114+0000 7f9de3000640 1 --2- 172.21.15.46:0/927448551 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ddc15ada0 0x7f9ddc154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.114+0000 7f9de3000640 1 --2- 172.21.15.46:0/927448551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ddc144210 0x7f9ddc13e750 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.114+0000 7f9de3000640 1 --2- 172.21.15.46:0/927448551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ddc0ce950 0x7f9ddc138b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.114+0000 7f9de3000640 1 -- 172.21.15.46:0/927448551 >> 172.21.15.46:0/927448551 conn(0x7f9ddc066110 msgr2=0x7f9ddc141f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.114+0000 7f9de3000640 1 -- 172.21.15.46:0/927448551 shutdown_connections 2024-03-29T08:22:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.114+0000 7f9de3000640 1 -- 172.21.15.46:0/927448551 wait complete. 2024-03-29T08:22:52.137 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:22:42.147364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:22:42.602168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:22:52.137 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:22:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f237f400640 1 Processor -- start 2024-03-29T08:22:52.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f237f400640 1 -- start start 2024-03-29T08:22:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f237f400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23780cbea0 0x7f23780adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f237f400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23780afe80 con 0x7f2378143180 2024-03-29T08:22:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f237f400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237806cd80 con 0x7f2378142da0 2024-03-29T08:22:52.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f237f400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237806d020 con 0x7f23780cbea0 2024-03-29T08:22:52.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f2377e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2378143180 0x7f23780cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57612/0 (socket says 172.21.15.46:57612) 2024-03-29T08:22:52.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f2377e00640 1 -- 172.21.15.46:0/3182469479 learned_addr learned my addr 172.21.15.46:0/3182469479 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:52.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f237d400640 1 --2- 172.21.15.46:0/3182469479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23780cbea0 0x7f23780adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f2377400640 1 -- 172.21.15.46:0/3182469479 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080893711 0 0) 0x7f23780afe80 con 0x7f2378143180 2024-03-29T08:22:52.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f2377400640 1 -- 172.21.15.46:0/3182469479 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2360003530 con 0x7f2378143180 2024-03-29T08:22:52.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f2377400640 1 -- 172.21.15.46:0/3182469479 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146978627 0 0) 0x7f237806cd80 con 0x7f2378142da0 2024-03-29T08:22:52.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f2377400640 1 -- 172.21.15.46:0/3182469479 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23780afe80 con 0x7f2378142da0 2024-03-29T08:22:52.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f2377400640 1 -- 172.21.15.46:0/3182469479 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 891325410 0 0) 0x7f2360003530 con 0x7f2378143180 2024-03-29T08:22:52.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f2377400640 1 -- 172.21.15.46:0/3182469479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23780cbea0 msgr2=0x7f23780adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:52.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f2377400640 1 --2- 172.21.15.46:0/3182469479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23780cbea0 0x7f23780adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f2377400640 1 -- 172.21.15.46:0/3182469479 >> v1:172.21.15.73:6789/0 conn(0x7f2378142da0 legacy=0x7f23780abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f2377400640 1 -- 172.21.15.46:0/3182469479 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23780b00f0 con 0x7f2378143180 2024-03-29T08:22:52.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f2377400640 1 -- 172.21.15.46:0/3182469479 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f2364001680 con 0x7f2378143180 2024-03-29T08:22:52.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f2377400640 1 -- 172.21.15.46:0/3182469479 >> v1:172.21.15.46:6789/0 conn(0x7f2378143180 legacy=0x7f23780cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.278+0000 7f2377400640 1 --2- 172.21.15.46:0/3182469479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2360003b40 0x7f2360003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.282+0000 7f2377400640 1 --2- 172.21.15.46:0/3182469479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23780cbea0 0x7f23600047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.282+0000 7f2377400640 1 --2- 172.21.15.46:0/3182469479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2360004d10 0x7f2360005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.282+0000 7f2377400640 1 -- 172.21.15.46:0/3182469479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237806d020 con 0x7f2360003b40 2024-03-29T08:22:52.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.282+0000 7f2377400640 1 -- 172.21.15.46:0/3182469479 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236000ba40 con 0x7f2360004d10 2024-03-29T08:22:52.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.282+0000 7f2377400640 1 -- 172.21.15.46:0/3182469479 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236000bc00 con 0x7f23780cbea0 2024-03-29T08:22:52.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.282+0000 7f237ca00640 1 --2- 172.21.15.46:0/3182469479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2360003b40 0x7f2360003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.282+0000 7f2377e00640 1 --2- 172.21.15.46:0/3182469479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23780cbea0 0x7f23600047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.282+0000 7f237d400640 1 --2- 172.21.15.46:0/3182469479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2360004d10 0x7f2360005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.282+0000 7f237ca00640 1 -- 172.21.15.46:0/3182469479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23780cbea0 msgr2=0x7f23600047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:52.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.282+0000 7f237ca00640 1 --2- 172.21.15.46:0/3182469479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23780cbea0 0x7f23600047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.282+0000 7f237ca00640 1 -- 172.21.15.46:0/3182469479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2360004d10 msgr2=0x7f2360005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:52.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.282+0000 7f237ca00640 1 --2- 172.21.15.46:0/3182469479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2360004d10 0x7f2360005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.282+0000 7f237ca00640 1 -- 172.21.15.46:0/3182469479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2360003530 con 0x7f2360003b40 2024-03-29T08:22:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.282+0000 7f237ca00640 1 --2- 172.21.15.46:0/3182469479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2360003b40 0x7f2360003f40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f236c0018d0 tx=0x7f236c01c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1133dbb48604d8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.282+0000 7f2377400640 1 -- 172.21.15.46:0/3182469479 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c0050d0 con 0x7f2360003b40 2024-03-29T08:22:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.282+0000 7f2377400640 1 -- 172.21.15.46:0/3182469479 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c005250 con 0x7f2360003b40 2024-03-29T08:22:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.282+0000 7f2377400640 1 -- 172.21.15.46:0/3182469479 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c003600 con 0x7f2360003b40 2024-03-29T08:22:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237f400640 1 -- 172.21.15.46:0/3182469479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2360003b40 msgr2=0x7f2360003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237f400640 1 --2- 172.21.15.46:0/3182469479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2360003b40 0x7f2360003f40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f236c0018d0 tx=0x7f236c01c580 comp rx=0 tx=0).stop 2024-03-29T08:22:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237d400640 1 -- 172.21.15.46:0/3182469479 reap_dead start 2024-03-29T08:22:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237f400640 1 -- 172.21.15.46:0/3182469479 shutdown_connections 2024-03-29T08:22:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237f400640 1 -- 172.21.15.46:0/3182469479 >> 172.21.15.46:0/3182469479 conn(0x7f237806a390 msgr2=0x7f23780cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237f400640 1 -- 172.21.15.46:0/3182469479 shutdown_connections 2024-03-29T08:22:52.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237f400640 1 -- 172.21.15.46:0/3182469479 wait complete. 2024-03-29T08:22:52.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237f400640 1 Processor -- start 2024-03-29T08:22:52.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237f400640 1 -- start start 2024-03-29T08:22:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2360003b40 0x7f23780b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23780cd810 0x7f23780cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237f400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23780b1590 0x7f23780b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237f400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237813ba60 con 0x7f2360003b40 2024-03-29T08:22:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237f400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23780ce1d0 con 0x7f23780cd810 2024-03-29T08:22:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237f400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237813ade0 con 0x7f23780b1590 2024-03-29T08:22:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2360003b40 0x7f23780b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2360003b40 0x7f23780b3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54682/0 (socket says 172.21.15.46:54682) 2024-03-29T08:22:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237ca00640 1 -- 172.21.15.46:0/71367849 learned_addr learned my addr 172.21.15.46:0/71367849 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f2377e00640 1 --2- 172.21.15.46:0/71367849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23780cd810 0x7f23780cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237d400640 1 --2- 172.21.15.46:0/71367849 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23780b1590 0x7f23780b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237ca00640 1 -- 172.21.15.46:0/71367849 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23780b1590 msgr2=0x7f23780b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237ca00640 1 --2- 172.21.15.46:0/71367849 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23780b1590 0x7f23780b1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237ca00640 1 -- 172.21.15.46:0/71367849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23780cd810 msgr2=0x7f23780cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237ca00640 1 --2- 172.21.15.46:0/71367849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23780cd810 0x7f23780cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.286+0000 7f237ca00640 1 -- 172.21.15.46:0/71367849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237813b0c0 con 0x7f2360003b40 2024-03-29T08:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.290+0000 7f237ca00640 1 --2- 172.21.15.46:0/71367849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2360003b40 0x7f23780b3ea0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f236c005040 tx=0x7f2360005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.290+0000 7f2374c00640 1 -- 172.21.15.46:0/71367849 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c0050d0 con 0x7f2360003b40 2024-03-29T08:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.290+0000 7f2374c00640 1 -- 172.21.15.46:0/71367849 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c003cc0 con 0x7f2360003b40 2024-03-29T08:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.290+0000 7f2374c00640 1 -- 172.21.15.46:0/71367849 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c0069e0 con 0x7f2360003b40 2024-03-29T08:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.290+0000 7f237f400640 1 -- 172.21.15.46:0/71367849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237813b3a0 con 0x7f2360003b40 2024-03-29T08:22:52.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.290+0000 7f237f400640 1 -- 172.21.15.46:0/71367849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23780b1f50 con 0x7f2360003b40 2024-03-29T08:22:52.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.290+0000 7f237f400640 1 -- 172.21.15.46:0/71367849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23780b2140 con 0x7f2360003b40 2024-03-29T08:22:52.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.290+0000 7f2374c00640 1 -- 172.21.15.46:0/71367849 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f236c006b40 con 0x7f2360003b40 2024-03-29T08:22:52.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.290+0000 7f2374c00640 1 --2- 172.21.15.46:0/71367849 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2358066690 0x7f2358068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.290+0000 7f2374c00640 1 -- 172.21.15.46:0/71367849 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f236c11e0e0 con 0x7f2360003b40 2024-03-29T08:22:52.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.290+0000 7f2377e00640 1 --2- 172.21.15.46:0/71367849 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2358066690 0x7f2358068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.294+0000 7f2377e00640 1 --2- 172.21.15.46:0/71367849 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2358066690 0x7f2358068b50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2378158760 tx=0x7f2364009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:52.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.294+0000 7f2374c00640 1 -- 172.21.15.46:0/71367849 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f236c0eb560 con 0x7f2360003b40 2024-03-29T08:22:52.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.458+0000 7f237f400640 1 -- 172.21.15.46:0/71367849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23781355f0 con 0x7f2360003b40 2024-03-29T08:22:52.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.458+0000 7f2374c00640 1 -- 172.21.15.46:0/71367849 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f236c0eeb80 con 0x7f2360003b40 2024-03-29T08:22:52.462 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:22:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.462+0000 7f237f400640 1 -- 172.21.15.46:0/71367849 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2358066690 msgr2=0x7f2358068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.462+0000 7f237f400640 1 --2- 172.21.15.46:0/71367849 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2358066690 0x7f2358068b50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2378158760 tx=0x7f2364009040 comp rx=0 tx=0).stop 2024-03-29T08:22:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.462+0000 7f237f400640 1 -- 172.21.15.46:0/71367849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2360003b40 msgr2=0x7f23780b3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.462+0000 7f237f400640 1 --2- 172.21.15.46:0/71367849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2360003b40 0x7f23780b3ea0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f236c005040 tx=0x7f2360005800 comp rx=0 tx=0).stop 2024-03-29T08:22:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.462+0000 7f237f400640 1 -- 172.21.15.46:0/71367849 shutdown_connections 2024-03-29T08:22:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.462+0000 7f237f400640 1 --2- 172.21.15.46:0/71367849 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2358066690 0x7f2358068b50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.462+0000 7f237f400640 1 --2- 172.21.15.46:0/71367849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23780cd810 0x7f23780cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.462+0000 7f237f400640 1 --2- 172.21.15.46:0/71367849 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23780b1590 0x7f23780b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.462+0000 7f237f400640 1 --2- 172.21.15.46:0/71367849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2360003b40 0x7f23780b3ea0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.462+0000 7f237f400640 1 -- 172.21.15.46:0/71367849 >> 172.21.15.46:0/71367849 conn(0x7f237806a390 msgr2=0x7f237815c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:52.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.462+0000 7f237f400640 1 -- 172.21.15.46:0/71367849 shutdown_connections 2024-03-29T08:22:52.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.462+0000 7f237f400640 1 -- 172.21.15.46:0/71367849 wait complete. 2024-03-29T08:22:52.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:22:52.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.622+0000 7fbc7b200640 1 Processor -- start 2024-03-29T08:22:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.622+0000 7fbc7b200640 1 -- start start 2024-03-29T08:22:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.622+0000 7fbc7b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc7415cb60 0x7fbc740d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.622+0000 7fbc7b200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbc740d0bf0 0x7fbc740cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.622+0000 7fbc7b200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc7415cf40 con 0x7fbc740cf820 2024-03-29T08:22:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.622+0000 7fbc7b200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc740b0250 con 0x7fbc7415cb60 2024-03-29T08:22:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.622+0000 7fbc7b200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc740b0810 con 0x7fbc740d0bf0 2024-03-29T08:22:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.622+0000 7fbc73e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc7415cb60 0x7fbc740d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.622+0000 7fbc79200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbc740cf820 0x7fbc740ccfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57618/0 (socket says 172.21.15.46:57618) 2024-03-29T08:22:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.622+0000 7fbc79200640 1 -- 172.21.15.46:0/451268540 learned_addr learned my addr 172.21.15.46:0/451268540 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:52.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.622+0000 7fbc6b400640 1 --2- 172.21.15.46:0/451268540 >> v2:172.21.15.73:3300/0 conn(0x7fbc740d0bf0 0x7fbc740cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.622+0000 7fbc73e00640 1 -- 172.21.15.46:0/451268540 >> v2:172.21.15.73:3300/0 conn(0x7fbc740d0bf0 msgr2=0x7fbc740cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.622+0000 7fbc73e00640 1 --2- 172.21.15.46:0/451268540 >> v2:172.21.15.73:3300/0 conn(0x7fbc740d0bf0 0x7fbc740cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.622+0000 7fbc73e00640 1 -- 172.21.15.46:0/451268540 >> v1:172.21.15.46:6789/0 conn(0x7fbc740cf820 legacy=0x7fbc740ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.622+0000 7fbc73400640 1 -- 172.21.15.46:0/451268540 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661127834 0 0) 0x7fbc7415cf40 con 0x7fbc740cf820 2024-03-29T08:22:52.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.622+0000 7fbc73e00640 1 -- 172.21.15.46:0/451268540 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc740d1050 con 0x7fbc7415cb60 2024-03-29T08:22:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.626+0000 7fbc73e00640 1 --2- 172.21.15.46:0/451268540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc7415cb60 0x7fbc740d06b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fbc74160530 tx=0x7fbc6401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865a1cf272decaf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.626+0000 7fbc73400640 1 -- 172.21.15.46:0/451268540 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc640a59c0 con 0x7fbc7415cb60 2024-03-29T08:22:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.626+0000 7fbc73400640 1 -- 172.21.15.46:0/451268540 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc640a5b20 con 0x7fbc7415cb60 2024-03-29T08:22:52.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.626+0000 7fbc73400640 1 -- 172.21.15.46:0/451268540 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc640ad440 con 0x7fbc7415cb60 2024-03-29T08:22:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.626+0000 7fbc7b200640 1 -- 172.21.15.46:0/451268540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc7415cb60 msgr2=0x7fbc740d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.626+0000 7fbc7b200640 1 --2- 172.21.15.46:0/451268540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc7415cb60 0x7fbc740d06b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fbc74160530 tx=0x7fbc6401c900 comp rx=0 tx=0).stop 2024-03-29T08:22:52.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.626+0000 7fbc7b200640 1 -- 172.21.15.46:0/451268540 shutdown_connections 2024-03-29T08:22:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.626+0000 7fbc7b200640 1 --2- 172.21.15.46:0/451268540 >> v2:172.21.15.73:3300/0 conn(0x7fbc740d0bf0 0x7fbc740cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.626+0000 7fbc7b200640 1 --2- 172.21.15.46:0/451268540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc7415cb60 0x7fbc740d06b0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.626+0000 7fbc7b200640 1 -- 172.21.15.46:0/451268540 >> 172.21.15.46:0/451268540 conn(0x7fbc74099a10 msgr2=0x7fbc74089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.626+0000 7fbc7b200640 1 -- 172.21.15.46:0/451268540 shutdown_connections 2024-03-29T08:22:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.626+0000 7fbc7b200640 1 -- 172.21.15.46:0/451268540 wait complete. 2024-03-29T08:22:52.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc7b200640 1 Processor -- start 2024-03-29T08:22:52.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc7b200640 1 -- start start 2024-03-29T08:22:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc7b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc740cf820 0x7fbc741545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc7b200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc740d0bf0 0x7fbc7414a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc7b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc7415cb60 0x7fbc740ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc7b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc74153890 con 0x7fbc7415cb60 2024-03-29T08:22:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc7b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc74154130 con 0x7fbc740cf820 2024-03-29T08:22:52.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc7b200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc740afe20 con 0x7fbc740d0bf0 2024-03-29T08:22:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc79200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc7415cb60 0x7fbc740ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc79200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc7415cb60 0x7fbc740ae120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54704/0 (socket says 172.21.15.46:54704) 2024-03-29T08:22:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc79200640 1 -- 172.21.15.46:0/4262127407 learned_addr learned my addr 172.21.15.46:0/4262127407 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc6b400640 1 --2- 172.21.15.46:0/4262127407 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc740d0bf0 0x7fbc7414a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc73e00640 1 --2- 172.21.15.46:0/4262127407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc740cf820 0x7fbc741545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc79200640 1 -- 172.21.15.46:0/4262127407 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc740d0bf0 msgr2=0x7fbc7414a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc79200640 1 --2- 172.21.15.46:0/4262127407 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc740d0bf0 0x7fbc7414a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc79200640 1 -- 172.21.15.46:0/4262127407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc740cf820 msgr2=0x7fbc741545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc79200640 1 --2- 172.21.15.46:0/4262127407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc740cf820 0x7fbc741545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc79200640 1 -- 172.21.15.46:0/4262127407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc64009a70 con 0x7fbc7415cb60 2024-03-29T08:22:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc6b400640 1 --2- 172.21.15.46:0/4262127407 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc740d0bf0 0x7fbc7414a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:22:52.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc79200640 1 --2- 172.21.15.46:0/4262127407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc7415cb60 0x7fbc740ae120 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c00c4b0 tx=0x7fbc6c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc70c00640 1 -- 172.21.15.46:0/4262127407 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0074e0 con 0x7fbc7415cb60 2024-03-29T08:22:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc70c00640 1 -- 172.21.15.46:0/4262127407 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6c007640 con 0x7fbc7415cb60 2024-03-29T08:22:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc7b200640 1 -- 172.21.15.46:0/4262127407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc74153ae0 con 0x7fbc7415cb60 2024-03-29T08:22:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc70c00640 1 -- 172.21.15.46:0/4262127407 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c004a30 con 0x7fbc7415cb60 2024-03-29T08:22:52.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.630+0000 7fbc7b200640 1 -- 172.21.15.46:0/4262127407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc740afa10 con 0x7fbc7415cb60 2024-03-29T08:22:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.634+0000 7fbc7b200640 1 -- 172.21.15.46:0/4262127407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc741542c0 con 0x7fbc7415cb60 2024-03-29T08:22:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.634+0000 7fbc70c00640 1 -- 172.21.15.46:0/4262127407 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbc6c004b90 con 0x7fbc7415cb60 2024-03-29T08:22:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.634+0000 7fbc70c00640 1 --2- 172.21.15.46:0/4262127407 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc50066690 0x7fbc50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.634+0000 7fbc70c00640 1 -- 172.21.15.46:0/4262127407 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc6c0836b0 con 0x7fbc7415cb60 2024-03-29T08:22:52.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.634+0000 7fbc73e00640 1 --2- 172.21.15.46:0/4262127407 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc50066690 0x7fbc50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.634+0000 7fbc73e00640 1 --2- 172.21.15.46:0/4262127407 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc50066690 0x7fbc50068b50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbc640af000 tx=0x7fbc640a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:52.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.638+0000 7fbc70c00640 1 -- 172.21.15.46:0/4262127407 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbc6c050980 con 0x7fbc7415cb60 2024-03-29T08:22:52.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.798+0000 7fbc7b200640 1 -- 172.21.15.46:0/4262127407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbc7409e7d0 con 0x7fbc7415cb60 2024-03-29T08:22:52.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.798+0000 7fbc70c00640 1 -- 172.21.15.46:0/4262127407 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbc6c053fa0 con 0x7fbc7415cb60 2024-03-29T08:22:52.801 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:22:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.798+0000 7fbc7b200640 1 -- 172.21.15.46:0/4262127407 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc50066690 msgr2=0x7fbc50068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.798+0000 7fbc7b200640 1 --2- 172.21.15.46:0/4262127407 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc50066690 0x7fbc50068b50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbc640af000 tx=0x7fbc640a7070 comp rx=0 tx=0).stop 2024-03-29T08:22:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.798+0000 7fbc7b200640 1 -- 172.21.15.46:0/4262127407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc7415cb60 msgr2=0x7fbc740ae120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.798+0000 7fbc7b200640 1 --2- 172.21.15.46:0/4262127407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc7415cb60 0x7fbc740ae120 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c00c4b0 tx=0x7fbc6c00c970 comp rx=0 tx=0).stop 2024-03-29T08:22:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.798+0000 7fbc7b200640 1 -- 172.21.15.46:0/4262127407 shutdown_connections 2024-03-29T08:22:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.798+0000 7fbc7b200640 1 --2- 172.21.15.46:0/4262127407 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc50066690 0x7fbc50068b50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.798+0000 7fbc7b200640 1 --2- 172.21.15.46:0/4262127407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc7415cb60 0x7fbc740ae120 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.798+0000 7fbc7b200640 1 --2- 172.21.15.46:0/4262127407 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc740d0bf0 0x7fbc7414a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.798+0000 7fbc7b200640 1 --2- 172.21.15.46:0/4262127407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc740cf820 0x7fbc741545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.798+0000 7fbc7b200640 1 -- 172.21.15.46:0/4262127407 >> 172.21.15.46:0/4262127407 conn(0x7fbc74099a10 msgr2=0x7fbc740b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:52.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.798+0000 7fbc7b200640 1 -- 172.21.15.46:0/4262127407 shutdown_connections 2024-03-29T08:22:52.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.798+0000 7fbc7b200640 1 -- 172.21.15.46:0/4262127407 wait complete. 2024-03-29T08:22:52.821 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:22:52.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b26800640 1 Processor -- start 2024-03-29T08:22:52.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b26800640 1 -- start start 2024-03-29T08:22:52.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b26800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b200b1810 0x7f7b200cad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b26800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7b200cb2c0 0x7f7b200ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b26800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b201605e0 con 0x7f7b20160200 2024-03-29T08:22:52.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b26800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b2006b490 con 0x7f7b200cb2c0 2024-03-29T08:22:52.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b26800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b20098b70 con 0x7f7b200b1810 2024-03-29T08:22:52.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b1f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7b20160200 0x7f7b20099930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57630/0 (socket says 172.21.15.46:57630) 2024-03-29T08:22:52.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b1f400640 1 -- 172.21.15.46:0/3091612790 learned_addr learned my addr 172.21.15.46:0/3091612790 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:52.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b1fe00640 1 --2- 172.21.15.46:0/3091612790 >> v2:172.21.15.73:3300/0 conn(0x7f7b200cb2c0 0x7f7b200ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b1ea00640 1 --2- 172.21.15.46:0/3091612790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b200b1810 0x7f7b200cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b1e000640 1 -- 172.21.15.46:0/3091612790 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266942798 0 0) 0x7f7b201605e0 con 0x7f7b20160200 2024-03-29T08:22:52.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b1e000640 1 -- 172.21.15.46:0/3091612790 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b04003530 con 0x7f7b20160200 2024-03-29T08:22:52.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b1fe00640 1 -- 172.21.15.46:0/3091612790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b200b1810 msgr2=0x7f7b200cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:52.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b1e000640 1 -- 172.21.15.46:0/3091612790 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1700574711 0 0) 0x7f7b04003530 con 0x7f7b20160200 2024-03-29T08:22:52.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b1fe00640 1 --2- 172.21.15.46:0/3091612790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b200b1810 0x7f7b200cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b1fe00640 1 -- 172.21.15.46:0/3091612790 >> v1:172.21.15.46:6789/0 conn(0x7f7b20160200 legacy=0x7f7b20099930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.966+0000 7f7b1fe00640 1 -- 172.21.15.46:0/3091612790 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b20098e50 con 0x7f7b200cb2c0 2024-03-29T08:22:52.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.970+0000 7f7b1fe00640 1 --2- 172.21.15.46:0/3091612790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b200cb2c0 0x7f7b200ce8e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009d90 tx=0x7f7b1801c7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dfb77f37912f95a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:52.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.970+0000 7f7b1e000640 1 -- 172.21.15.46:0/3091612790 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b10001680 con 0x7f7b200cb2c0 2024-03-29T08:22:52.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.970+0000 7f7b1e000640 1 -- 172.21.15.46:0/3091612790 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b180a55d0 con 0x7f7b200cb2c0 2024-03-29T08:22:52.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.970+0000 7f7b1e000640 1 -- 172.21.15.46:0/3091612790 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b180a5960 con 0x7f7b200cb2c0 2024-03-29T08:22:52.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.970+0000 7f7b26800640 1 -- 172.21.15.46:0/3091612790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b200cb2c0 msgr2=0x7f7b200ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.970+0000 7f7b26800640 1 --2- 172.21.15.46:0/3091612790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b200cb2c0 0x7f7b200ce8e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009d90 tx=0x7f7b1801c7d0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.970+0000 7f7b26800640 1 -- 172.21.15.46:0/3091612790 shutdown_connections 2024-03-29T08:22:52.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.970+0000 7f7b26800640 1 --2- 172.21.15.46:0/3091612790 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b200b1810 0x7f7b200cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.970+0000 7f7b26800640 1 --2- 172.21.15.46:0/3091612790 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b200cb2c0 0x7f7b200ce8e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.970+0000 7f7b26800640 1 -- 172.21.15.46:0/3091612790 >> 172.21.15.46:0/3091612790 conn(0x7f7b200b2220 msgr2=0x7f7b200b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:52.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.970+0000 7f7b26800640 1 -- 172.21.15.46:0/3091612790 shutdown_connections 2024-03-29T08:22:52.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.970+0000 7f7b26800640 1 -- 172.21.15.46:0/3091612790 wait complete. 2024-03-29T08:22:52.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b26800640 1 Processor -- start 2024-03-29T08:22:52.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b26800640 1 -- start start 2024-03-29T08:22:52.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b26800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b200b1810 0x7f7b20154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b26800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b200cb2c0 0x7f7b2015e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b26800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b20160200 0x7f7b2015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b26800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2015baf0 con 0x7f7b200cb2c0 2024-03-29T08:22:52.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b26800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2015bdd0 con 0x7f7b200b1810 2024-03-29T08:22:52.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b26800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2015c0b0 con 0x7f7b20160200 2024-03-29T08:22:52.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b1ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b200cb2c0 0x7f7b2015e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b1ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b200cb2c0 0x7f7b2015e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54714/0 (socket says 172.21.15.46:54714) 2024-03-29T08:22:52.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b1ea00640 1 -- 172.21.15.46:0/3772277654 learned_addr learned my addr 172.21.15.46:0/3772277654 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:52.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b1f400640 1 --2- 172.21.15.46:0/3772277654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b200b1810 0x7f7b20154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b1fe00640 1 --2- 172.21.15.46:0/3772277654 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b20160200 0x7f7b2015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b1ea00640 1 -- 172.21.15.46:0/3772277654 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b20160200 msgr2=0x7f7b2015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b1ea00640 1 --2- 172.21.15.46:0/3772277654 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b20160200 0x7f7b2015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b1ea00640 1 -- 172.21.15.46:0/3772277654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b200b1810 msgr2=0x7f7b20154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:52.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b1ea00640 1 --2- 172.21.15.46:0/3772277654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b200b1810 0x7f7b20154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:52.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b1ea00640 1 -- 172.21.15.46:0/3772277654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b18009a70 con 0x7f7b200cb2c0 2024-03-29T08:22:52.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b1f400640 1 --2- 172.21.15.46:0/3772277654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b200b1810 0x7f7b20154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:22:52.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b1fe00640 1 --2- 172.21.15.46:0/3772277654 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b20160200 0x7f7b2015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:22:52.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b1ea00640 1 --2- 172.21.15.46:0/3772277654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b200cb2c0 0x7f7b2015e850 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00e930 tx=0x7f7b0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:52.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b17400640 1 -- 172.21.15.46:0/3772277654 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c009690 con 0x7f7b200cb2c0 2024-03-29T08:22:52.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b17400640 1 -- 172.21.15.46:0/3772277654 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0c00ca00 con 0x7f7b200cb2c0 2024-03-29T08:22:52.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b26800640 1 -- 172.21.15.46:0/3772277654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2015c390 con 0x7f7b200cb2c0 2024-03-29T08:22:52.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b17400640 1 -- 172.21.15.46:0/3772277654 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c00cd70 con 0x7f7b200cb2c0 2024-03-29T08:22:52.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b26800640 1 -- 172.21.15.46:0/3772277654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b20155340 con 0x7f7b200cb2c0 2024-03-29T08:22:52.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.974+0000 7f7b26800640 1 -- 172.21.15.46:0/3772277654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b2004e120 con 0x7f7b200cb2c0 2024-03-29T08:22:52.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.978+0000 7f7b17400640 1 -- 172.21.15.46:0/3772277654 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7b0c013470 con 0x7f7b200cb2c0 2024-03-29T08:22:52.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.978+0000 7f7b17400640 1 --2- 172.21.15.46:0/3772277654 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b00066690 0x7f7b00068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:52.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.978+0000 7f7b17400640 1 -- 172.21.15.46:0/3772277654 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b0c084520 con 0x7f7b200cb2c0 2024-03-29T08:22:52.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.978+0000 7f7b1f400640 1 --2- 172.21.15.46:0/3772277654 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b00066690 0x7f7b00068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:52.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.978+0000 7f7b1f400640 1 --2- 172.21.15.46:0/3772277654 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b00066690 0x7f7b00068b50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7b100011f0 tx=0x7f7b10009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:52.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:52.982+0000 7f7b17400640 1 -- 172.21.15.46:0/3772277654 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7b0c0518b0 con 0x7f7b200cb2c0 2024-03-29T08:22:53.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.142+0000 7f7b26800640 1 -- 172.21.15.46:0/3772277654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b2006b690 con 0x7f7b200cb2c0 2024-03-29T08:22:53.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.142+0000 7f7b17400640 1 -- 172.21.15.46:0/3772277654 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7b0c054ed0 con 0x7f7b200cb2c0 2024-03-29T08:22:53.146 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:22:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.146+0000 7f7b26800640 1 -- 172.21.15.46:0/3772277654 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b00066690 msgr2=0x7f7b00068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.146+0000 7f7b26800640 1 --2- 172.21.15.46:0/3772277654 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b00066690 0x7f7b00068b50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7b100011f0 tx=0x7f7b10009040 comp rx=0 tx=0).stop 2024-03-29T08:22:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.146+0000 7f7b26800640 1 -- 172.21.15.46:0/3772277654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b200cb2c0 msgr2=0x7f7b2015e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.146+0000 7f7b26800640 1 --2- 172.21.15.46:0/3772277654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b200cb2c0 0x7f7b2015e850 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00e930 tx=0x7f7b0c00edf0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.146+0000 7f7b26800640 1 -- 172.21.15.46:0/3772277654 shutdown_connections 2024-03-29T08:22:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.146+0000 7f7b26800640 1 --2- 172.21.15.46:0/3772277654 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b00066690 0x7f7b00068b50 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.146+0000 7f7b26800640 1 --2- 172.21.15.46:0/3772277654 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b20160200 0x7f7b2015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.146+0000 7f7b26800640 1 --2- 172.21.15.46:0/3772277654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b200cb2c0 0x7f7b2015e850 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.146+0000 7f7b26800640 1 --2- 172.21.15.46:0/3772277654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b200b1810 0x7f7b20154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.146+0000 7f7b26800640 1 -- 172.21.15.46:0/3772277654 >> 172.21.15.46:0/3772277654 conn(0x7f7b200b2220 msgr2=0x7f7b2006e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.146+0000 7f7b26800640 1 -- 172.21.15.46:0/3772277654 shutdown_connections 2024-03-29T08:22:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.146+0000 7f7b26800640 1 -- 172.21.15.46:0/3772277654 wait complete. 2024-03-29T08:22:53.167 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:22:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.306+0000 7fb683400640 1 Processor -- start 2024-03-29T08:22:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.306+0000 7fb683400640 1 -- start start 2024-03-29T08:22:53.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.306+0000 7fb683400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb67c06dbb0 0x7fb67c15d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.306+0000 7fb683400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb67c15d940 0x7fb67c0b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.306+0000 7fb683400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67c070c70 con 0x7fb67c142ab0 2024-03-29T08:22:53.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.306+0000 7fb683400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67c070340 con 0x7fb67c15d940 2024-03-29T08:22:53.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.306+0000 7fb683400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c070620 con 0x7fb67c06dbb0 2024-03-29T08:22:53.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.306+0000 7fb67be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb67c06dbb0 0x7fb67c15d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.306+0000 7fb680a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb67c142ab0 0x7fb67c06d4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:48892/0 (socket says 172.21.15.46:48892) 2024-03-29T08:22:53.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.306+0000 7fb681400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb67c15d940 0x7fb67c0b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.306+0000 7fb680a00640 1 -- 172.21.15.46:0/4002179138 learned_addr learned my addr 172.21.15.46:0/4002179138 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:53.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.306+0000 7fb67be00640 1 -- 172.21.15.46:0/4002179138 >> v2:172.21.15.73:3300/0 conn(0x7fb67c15d940 msgr2=0x7fb67c0b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:53.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.306+0000 7fb67be00640 1 --2- 172.21.15.46:0/4002179138 >> v2:172.21.15.73:3300/0 conn(0x7fb67c15d940 0x7fb67c0b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.306+0000 7fb67be00640 1 -- 172.21.15.46:0/4002179138 >> v1:172.21.15.73:6789/0 conn(0x7fb67c142ab0 legacy=0x7fb67c06d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.306+0000 7fb67be00640 1 -- 172.21.15.46:0/4002179138 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c070900 con 0x7fb67c06dbb0 2024-03-29T08:22:53.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.306+0000 7fb67b400640 1 -- 172.21.15.46:0/4002179138 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1337040341 0 0) 0x7fb67c070c70 con 0x7fb67c142ab0 2024-03-29T08:22:53.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.310+0000 7fb67be00640 1 --2- 172.21.15.46:0/4002179138 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb67c06dbb0 0x7fb67c15d400 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb6680097c0 tx=0x7fb66801e6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11ba93521951df8a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:53.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.310+0000 7fb67b400640 1 -- 172.21.15.46:0/4002179138 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6680ad600 con 0x7fb67c06dbb0 2024-03-29T08:22:53.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.310+0000 7fb67b400640 1 -- 172.21.15.46:0/4002179138 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6680a9050 con 0x7fb67c06dbb0 2024-03-29T08:22:53.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.310+0000 7fb67b400640 1 -- 172.21.15.46:0/4002179138 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6680ad980 con 0x7fb67c06dbb0 2024-03-29T08:22:53.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.310+0000 7fb683400640 1 -- 172.21.15.46:0/4002179138 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb67c06dbb0 msgr2=0x7fb67c15d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.310+0000 7fb683400640 1 --2- 172.21.15.46:0/4002179138 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb67c06dbb0 0x7fb67c15d400 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb6680097c0 tx=0x7fb66801e6b0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.310+0000 7fb683400640 1 -- 172.21.15.46:0/4002179138 shutdown_connections 2024-03-29T08:22:53.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.310+0000 7fb683400640 1 --2- 172.21.15.46:0/4002179138 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb67c06dbb0 0x7fb67c15d400 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.310+0000 7fb683400640 1 --2- 172.21.15.46:0/4002179138 >> v2:172.21.15.73:3300/0 conn(0x7fb67c15d940 0x7fb67c0b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.310+0000 7fb683400640 1 -- 172.21.15.46:0/4002179138 >> 172.21.15.46:0/4002179138 conn(0x7fb67c066080 msgr2=0x7fb67c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:53.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.310+0000 7fb683400640 1 -- 172.21.15.46:0/4002179138 shutdown_connections 2024-03-29T08:22:53.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.310+0000 7fb683400640 1 -- 172.21.15.46:0/4002179138 wait complete. 2024-03-29T08:22:53.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb683400640 1 Processor -- start 2024-03-29T08:22:53.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb683400640 1 -- start start 2024-03-29T08:22:53.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb683400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb67c06dbb0 0x7fb67c1630b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb683400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb67c142ab0 0x7fb67c1612c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb683400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c15d940 0x7fb67c0ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb683400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0ac4f0 con 0x7fb67c142ab0 2024-03-29T08:22:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb683400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c142670 con 0x7fb67c15d940 2024-03-29T08:22:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb683400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c1420b0 con 0x7fb67c06dbb0 2024-03-29T08:22:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb67be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb67c142ab0 0x7fb67c1612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb67be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb67c142ab0 0x7fb67c1612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54722/0 (socket says 172.21.15.46:54722) 2024-03-29T08:22:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb67be00640 1 -- 172.21.15.46:0/3830926972 learned_addr learned my addr 172.21.15.46:0/3830926972 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb680a00640 1 --2- 172.21.15.46:0/3830926972 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb67c06dbb0 0x7fb67c1630b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb681400640 1 --2- 172.21.15.46:0/3830926972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c15d940 0x7fb67c0ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb67be00640 1 -- 172.21.15.46:0/3830926972 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb67c06dbb0 msgr2=0x7fb67c1630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb67be00640 1 --2- 172.21.15.46:0/3830926972 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb67c06dbb0 0x7fb67c1630b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb67be00640 1 -- 172.21.15.46:0/3830926972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c15d940 msgr2=0x7fb67c0ac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb67be00640 1 --2- 172.21.15.46:0/3830926972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c15d940 0x7fb67c0ac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb67be00640 1 -- 172.21.15.46:0/3830926972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb668009420 con 0x7fb67c142ab0 2024-03-29T08:22:53.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb67be00640 1 --2- 172.21.15.46:0/3830926972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb67c142ab0 0x7fb67c1612c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb6680b0040 tx=0x7fb66801b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:53.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb678c00640 1 -- 172.21.15.46:0/3830926972 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb668006e20 con 0x7fb67c142ab0 2024-03-29T08:22:53.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb678c00640 1 -- 172.21.15.46:0/3830926972 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6680a6e90 con 0x7fb67c142ab0 2024-03-29T08:22:53.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb678c00640 1 -- 172.21.15.46:0/3830926972 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6680b6690 con 0x7fb67c142ab0 2024-03-29T08:22:53.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb683400640 1 -- 172.21.15.46:0/3830926972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67c142390 con 0x7fb67c142ab0 2024-03-29T08:22:53.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.314+0000 7fb683400640 1 -- 172.21.15.46:0/3830926972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67c161800 con 0x7fb67c142ab0 2024-03-29T08:22:53.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.318+0000 7fb683400640 1 -- 172.21.15.46:0/3830926972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67c1619f0 con 0x7fb67c142ab0 2024-03-29T08:22:53.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.318+0000 7fb678c00640 1 -- 172.21.15.46:0/3830926972 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb6680b53b0 con 0x7fb67c142ab0 2024-03-29T08:22:53.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.318+0000 7fb678c00640 1 --2- 172.21.15.46:0/3830926972 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb65c066690 0x7fb65c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.318+0000 7fb678c00640 1 -- 172.21.15.46:0/3830926972 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb66811d670 con 0x7fb67c142ab0 2024-03-29T08:22:53.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.318+0000 7fb680a00640 1 --2- 172.21.15.46:0/3830926972 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb65c066690 0x7fb65c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.318+0000 7fb680a00640 1 --2- 172.21.15.46:0/3830926972 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb65c066690 0x7fb65c068b50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb670007fd0 tx=0x7fb670007d60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:53.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.322+0000 7fb678c00640 1 -- 172.21.15.46:0/3830926972 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb67c1619f0 con 0x7fb67c142ab0 2024-03-29T08:22:53.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.494+0000 7fb683400640 1 -- 172.21.15.46:0/3830926972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb67c0ac2c0 con 0x7fb67c142ab0 2024-03-29T08:22:53.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.494+0000 7fb678c00640 1 -- 172.21.15.46:0/3830926972 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v77) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb67c0ac2c0 con 0x7fb67c142ab0 2024-03-29T08:22:53.497 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:22:53.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.494+0000 7fb683400640 1 -- 172.21.15.46:0/3830926972 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb65c066690 msgr2=0x7fb65c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.494+0000 7fb683400640 1 --2- 172.21.15.46:0/3830926972 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb65c066690 0x7fb65c068b50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb670007fd0 tx=0x7fb670007d60 comp rx=0 tx=0).stop 2024-03-29T08:22:53.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.494+0000 7fb683400640 1 -- 172.21.15.46:0/3830926972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb67c142ab0 msgr2=0x7fb67c1612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.494+0000 7fb683400640 1 --2- 172.21.15.46:0/3830926972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb67c142ab0 0x7fb67c1612c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fb6680b0040 tx=0x7fb66801b450 comp rx=0 tx=0).stop 2024-03-29T08:22:53.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.494+0000 7fb683400640 1 -- 172.21.15.46:0/3830926972 shutdown_connections 2024-03-29T08:22:53.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.494+0000 7fb683400640 1 --2- 172.21.15.46:0/3830926972 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb65c066690 0x7fb65c068b50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.494+0000 7fb683400640 1 --2- 172.21.15.46:0/3830926972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c15d940 0x7fb67c0ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.494+0000 7fb683400640 1 --2- 172.21.15.46:0/3830926972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb67c142ab0 0x7fb67c1612c0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.494+0000 7fb683400640 1 --2- 172.21.15.46:0/3830926972 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb67c06dbb0 0x7fb67c1630b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.494+0000 7fb683400640 1 -- 172.21.15.46:0/3830926972 >> 172.21.15.46:0/3830926972 conn(0x7fb67c066080 msgr2=0x7fb67c15a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:53.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.494+0000 7fb683400640 1 -- 172.21.15.46:0/3830926972 shutdown_connections 2024-03-29T08:22:53.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.494+0000 7fb683400640 1 -- 172.21.15.46:0/3830926972 wait complete. 2024-03-29T08:22:53.518 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:22:53.518 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:22:53.527 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:22:53.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.702+0000 7f4b03600640 1 Processor -- start 2024-03-29T08:22:53.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.702+0000 7f4b03600640 1 -- start start 2024-03-29T08:22:53.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.702+0000 7f4b03600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4afc1593d0 0x7f4afc06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.702+0000 7f4b03600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4afc15d230 0x7f4afc15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.702+0000 7f4b03600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4afc15dbd0 0x7f4afc15bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.702+0000 7f4b03600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4afc06d130 con 0x7f4afc1593d0 2024-03-29T08:22:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.702+0000 7f4b03600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4afc15b8f0 con 0x7f4afc15dbd0 2024-03-29T08:22:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.702+0000 7f4b03600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc15ad50 con 0x7f4afc15d230 2024-03-29T08:22:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.702+0000 7f4b00c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4afc1593d0 0x7f4afc06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.702+0000 7f4b00c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4afc1593d0 0x7f4afc06cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54728/0 (socket says 172.21.15.46:54728) 2024-03-29T08:22:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.702+0000 7f4b00c00640 1 -- 172.21.15.46:0/4039513449 learned_addr learned my addr 172.21.15.46:0/4039513449 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.706+0000 7f4b01600640 1 --2- 172.21.15.46:0/4039513449 >> v2:172.21.15.73:3300/0 conn(0x7f4afc15dbd0 0x7f4afc15bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.706+0000 7f4af3e00640 1 --2- 172.21.15.46:0/4039513449 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4afc15d230 0x7f4afc15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.706+0000 7f4b00c00640 1 -- 172.21.15.46:0/4039513449 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4afc15d230 msgr2=0x7f4afc15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:53.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.706+0000 7f4b00c00640 1 --2- 172.21.15.46:0/4039513449 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4afc15d230 0x7f4afc15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.706+0000 7f4b00c00640 1 -- 172.21.15.46:0/4039513449 >> v2:172.21.15.73:3300/0 conn(0x7f4afc15dbd0 msgr2=0x7f4afc15bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:53.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.706+0000 7f4b00c00640 1 --2- 172.21.15.46:0/4039513449 >> v2:172.21.15.73:3300/0 conn(0x7f4afc15dbd0 0x7f4afc15bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.706+0000 7f4b00c00640 1 -- 172.21.15.46:0/4039513449 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc15e060 con 0x7f4afc1593d0 2024-03-29T08:22:53.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.706+0000 7f4b00c00640 1 --2- 172.21.15.46:0/4039513449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc1593d0 0x7f4afc06cbf0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4afc15b860 tx=0x7f4aec01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407fded093fc6287 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.706+0000 7f4afbe00640 1 -- 172.21.15.46:0/4039513449 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec0aa640 con 0x7f4afc1593d0 2024-03-29T08:22:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.706+0000 7f4afbe00640 1 -- 172.21.15.46:0/4039513449 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aec0aa7a0 con 0x7f4afc1593d0 2024-03-29T08:22:53.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.706+0000 7f4afbe00640 1 -- 172.21.15.46:0/4039513449 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec0aab30 con 0x7f4afc1593d0 2024-03-29T08:22:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.706+0000 7f4b03600640 1 -- 172.21.15.46:0/4039513449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc1593d0 msgr2=0x7f4afc06cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.706+0000 7f4b03600640 1 --2- 172.21.15.46:0/4039513449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc1593d0 0x7f4afc06cbf0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4afc15b860 tx=0x7f4aec01c520 comp rx=0 tx=0).stop 2024-03-29T08:22:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4b03600640 1 -- 172.21.15.46:0/4039513449 shutdown_connections 2024-03-29T08:22:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4b03600640 1 --2- 172.21.15.46:0/4039513449 >> v2:172.21.15.73:3300/0 conn(0x7f4afc15dbd0 0x7f4afc15bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4b03600640 1 --2- 172.21.15.46:0/4039513449 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4afc15d230 0x7f4afc15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4b03600640 1 --2- 172.21.15.46:0/4039513449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc1593d0 0x7f4afc06cbf0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4b03600640 1 -- 172.21.15.46:0/4039513449 >> 172.21.15.46:0/4039513449 conn(0x7f4afc066110 msgr2=0x7f4afc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:53.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4b03600640 1 -- 172.21.15.46:0/4039513449 shutdown_connections 2024-03-29T08:22:53.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4b03600640 1 -- 172.21.15.46:0/4039513449 wait complete. 2024-03-29T08:22:53.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4b03600640 1 Processor -- start 2024-03-29T08:22:53.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4b03600640 1 -- start start 2024-03-29T08:22:53.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4b03600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4afc1593d0 0x7f4afc14c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4b03600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc15d230 0x7f4afc153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4b03600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4afc15dbd0 0x7f4afc163060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4b03600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc156240 con 0x7f4afc15d230 2024-03-29T08:22:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4b03600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc156510 con 0x7f4afc1593d0 2024-03-29T08:22:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4b03600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc1567f0 con 0x7f4afc15dbd0 2024-03-29T08:22:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4af3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc15d230 0x7f4afc153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4b00c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4afc1593d0 0x7f4afc14c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4af3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc15d230 0x7f4afc153e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54742/0 (socket says 172.21.15.46:54742) 2024-03-29T08:22:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4af3e00640 1 -- 172.21.15.46:0/2552837780 learned_addr learned my addr 172.21.15.46:0/2552837780 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4b01600640 1 --2- 172.21.15.46:0/2552837780 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4afc15dbd0 0x7f4afc163060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4af3e00640 1 -- 172.21.15.46:0/2552837780 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4afc15dbd0 msgr2=0x7f4afc163060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4af3e00640 1 --2- 172.21.15.46:0/2552837780 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4afc15dbd0 0x7f4afc163060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4af3e00640 1 -- 172.21.15.46:0/2552837780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4afc1593d0 msgr2=0x7f4afc14c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4af3e00640 1 --2- 172.21.15.46:0/2552837780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4afc1593d0 0x7f4afc14c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4af3e00640 1 -- 172.21.15.46:0/2552837780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec0094d0 con 0x7f4afc15d230 2024-03-29T08:22:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.710+0000 7f4af3e00640 1 --2- 172.21.15.46:0/2552837780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc15d230 0x7f4afc153e50 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400d800 tx=0x7f4ae400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.714+0000 7f4af9600640 1 -- 172.21.15.46:0/2552837780 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae40096a0 con 0x7f4afc15d230 2024-03-29T08:22:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.714+0000 7f4af9600640 1 -- 172.21.15.46:0/2552837780 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae4009800 con 0x7f4afc15d230 2024-03-29T08:22:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.714+0000 7f4af9600640 1 -- 172.21.15.46:0/2552837780 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4009990 con 0x7f4afc15d230 2024-03-29T08:22:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.714+0000 7f4b03600640 1 -- 172.21.15.46:0/2552837780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4afc156ad0 con 0x7f4afc15d230 2024-03-29T08:22:53.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.714+0000 7f4b03600640 1 -- 172.21.15.46:0/2552837780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4afc157090 con 0x7f4afc15d230 2024-03-29T08:22:53.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.714+0000 7f4b03600640 1 -- 172.21.15.46:0/2552837780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4afc066c90 con 0x7f4afc15d230 2024-03-29T08:22:53.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.714+0000 7f4af9600640 1 -- 172.21.15.46:0/2552837780 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4ae4009af0 con 0x7f4afc15d230 2024-03-29T08:22:53.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.714+0000 7f4af9600640 1 --2- 172.21.15.46:0/2552837780 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ad80666c0 0x7f4ad8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:53.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.714+0000 7f4af9600640 1 -- 172.21.15.46:0/2552837780 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ae4084470 con 0x7f4afc15d230 2024-03-29T08:22:53.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.714+0000 7f4b00c00640 1 --2- 172.21.15.46:0/2552837780 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ad80666c0 0x7f4ad8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:53.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.718+0000 7f4b00c00640 1 --2- 172.21.15.46:0/2552837780 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ad80666c0 0x7f4ad8068b80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0ae000 tx=0x7f4aec0b5040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:53.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.718+0000 7f4af9600640 1 -- 172.21.15.46:0/2552837780 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4ae4051950 con 0x7f4afc15d230 2024-03-29T08:22:53.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.882+0000 7f4b03600640 1 -- 172.21.15.46:0/2552837780 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4afc06d3d0 con 0x7f4afc15d230 2024-03-29T08:22:53.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.882+0000 7f4af9600640 1 -- 172.21.15.46:0/2552837780 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+15992 (secure 0 0 0) 0x7f4ae400b3a0 con 0x7f4afc15d230 2024-03-29T08:22:53.886 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:22:53.886 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":77,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:22:44.608017+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:22:42.147364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-29T08:22:42.602168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:22:53.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.886+0000 7f4b03600640 1 -- 172.21.15.46:0/2552837780 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ad80666c0 msgr2=0x7f4ad8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.886+0000 7f4b03600640 1 --2- 172.21.15.46:0/2552837780 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ad80666c0 0x7f4ad8068b80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0ae000 tx=0x7f4aec0b5040 comp rx=0 tx=0).stop 2024-03-29T08:22:53.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.886+0000 7f4b03600640 1 -- 172.21.15.46:0/2552837780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc15d230 msgr2=0x7f4afc153e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:53.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.886+0000 7f4b03600640 1 --2- 172.21.15.46:0/2552837780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc15d230 0x7f4afc153e50 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400d800 tx=0x7f4ae400dcc0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.886+0000 7f4b03600640 1 -- 172.21.15.46:0/2552837780 shutdown_connections 2024-03-29T08:22:53.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.886+0000 7f4b03600640 1 --2- 172.21.15.46:0/2552837780 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4ad80666c0 0x7f4ad8068b80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.886+0000 7f4b03600640 1 --2- 172.21.15.46:0/2552837780 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4afc15dbd0 0x7f4afc163060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.886+0000 7f4b03600640 1 --2- 172.21.15.46:0/2552837780 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4afc15d230 0x7f4afc153e50 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.886+0000 7f4b03600640 1 --2- 172.21.15.46:0/2552837780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4afc1593d0 0x7f4afc14c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:53.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.886+0000 7f4b03600640 1 -- 172.21.15.46:0/2552837780 >> 172.21.15.46:0/2552837780 conn(0x7f4afc066110 msgr2=0x7f4afc163c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:53.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.886+0000 7f4b03600640 1 -- 172.21.15.46:0/2552837780 shutdown_connections 2024-03-29T08:22:53.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:53.886+0000 7f4b03600640 1 -- 172.21.15.46:0/2552837780 wait complete. 2024-03-29T08:22:53.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:22:42.147364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:22:42.602168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:22:53.908 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:22:54.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.046+0000 7fb1b4000640 1 Processor -- start 2024-03-29T08:22:54.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.046+0000 7fb1b4000640 1 -- start start 2024-03-29T08:22:54.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.050+0000 7fb1b4000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb1ac159970 0x7fb1ac06d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:54.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.050+0000 7fb1b4000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1ac0711f0 0x7fb1ac070650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:54.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.050+0000 7fb1b4000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1ac070b90 0x7fb1ac0b2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:54.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.050+0000 7fb1b4000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ac071020 con 0x7fb1ac070b90 2024-03-29T08:22:54.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.050+0000 7fb1b4000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ac0b1940 con 0x7fb1ac159970 2024-03-29T08:22:54.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.050+0000 7fb1b4000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac06ddc0 con 0x7fb1ac0711f0 2024-03-29T08:22:54.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.050+0000 7fb1b2000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1ac070b90 0x7fb1ac0b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:54.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.050+0000 7fb1b2000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1ac070b90 0x7fb1ac0b2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54748/0 (socket says 172.21.15.46:54748) 2024-03-29T08:22:54.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.050+0000 7fb1b2000640 1 -- 172.21.15.46:0/2717293892 learned_addr learned my addr 172.21.15.46:0/2717293892 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:54.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.050+0000 7fb1b0c00640 1 --2- 172.21.15.46:0/2717293892 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1ac0711f0 0x7fb1ac070650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:54.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.050+0000 7fb1b1600640 1 --2- 172.21.15.46:0/2717293892 >> v2:172.21.15.73:3300/0 conn(0x7fb1ac159970 0x7fb1ac06d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:54.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.050+0000 7fb1b2000640 1 -- 172.21.15.46:0/2717293892 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1ac0711f0 msgr2=0x7fb1ac070650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:54.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.050+0000 7fb1b2000640 1 --2- 172.21.15.46:0/2717293892 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1ac0711f0 0x7fb1ac070650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.050+0000 7fb1b2000640 1 -- 172.21.15.46:0/2717293892 >> v2:172.21.15.73:3300/0 conn(0x7fb1ac159970 msgr2=0x7fb1ac06d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:54.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.050+0000 7fb1b2000640 1 --2- 172.21.15.46:0/2717293892 >> v2:172.21.15.73:3300/0 conn(0x7fb1ac159970 0x7fb1ac06d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.050+0000 7fb1b2000640 1 -- 172.21.15.46:0/2717293892 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac06e0a0 con 0x7fb1ac070b90 2024-03-29T08:22:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.054+0000 7fb1b2000640 1 --2- 172.21.15.46:0/2717293892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1ac070b90 0x7fb1ac0b2070 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac06aee0 tx=0x7fb1a401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d477dbfac03a71fc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.054+0000 7fb1abe00640 1 -- 172.21.15.46:0/2717293892 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a40a5760 con 0x7fb1ac070b90 2024-03-29T08:22:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.054+0000 7fb1abe00640 1 -- 172.21.15.46:0/2717293892 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a40a58c0 con 0x7fb1ac070b90 2024-03-29T08:22:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.054+0000 7fb1abe00640 1 -- 172.21.15.46:0/2717293892 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a40a5c50 con 0x7fb1ac070b90 2024-03-29T08:22:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.054+0000 7fb1b4000640 1 -- 172.21.15.46:0/2717293892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1ac070b90 msgr2=0x7fb1ac0b2070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.054+0000 7fb1b4000640 1 --2- 172.21.15.46:0/2717293892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1ac070b90 0x7fb1ac0b2070 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac06aee0 tx=0x7fb1a401c9f0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.054+0000 7fb1b4000640 1 -- 172.21.15.46:0/2717293892 shutdown_connections 2024-03-29T08:22:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.054+0000 7fb1b4000640 1 --2- 172.21.15.46:0/2717293892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1ac070b90 0x7fb1ac0b2070 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.054+0000 7fb1b4000640 1 --2- 172.21.15.46:0/2717293892 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1ac0711f0 0x7fb1ac070650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.054+0000 7fb1b4000640 1 --2- 172.21.15.46:0/2717293892 >> v2:172.21.15.73:3300/0 conn(0x7fb1ac159970 0x7fb1ac06d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.054+0000 7fb1b4000640 1 -- 172.21.15.46:0/2717293892 >> 172.21.15.46:0/2717293892 conn(0x7fb1ac0b0640 msgr2=0x7fb1ac0b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.054+0000 7fb1b4000640 1 -- 172.21.15.46:0/2717293892 shutdown_connections 2024-03-29T08:22:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.054+0000 7fb1b4000640 1 -- 172.21.15.46:0/2717293892 wait complete. 2024-03-29T08:22:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.054+0000 7fb1b4000640 1 Processor -- start 2024-03-29T08:22:54.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.054+0000 7fb1b4000640 1 -- start start 2024-03-29T08:22:54.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b4000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1ac070b90 0x7fb1ac150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:54.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b4000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1ac0711f0 0x7fb1ac14ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:54.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b4000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1ac159970 0x7fb1ac151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:54.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b4000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac14e600 con 0x7fb1ac159970 2024-03-29T08:22:54.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b4000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac151520 con 0x7fb1ac0711f0 2024-03-29T08:22:54.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b4000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ac14b180 con 0x7fb1ac070b90 2024-03-29T08:22:54.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1ac159970 0x7fb1ac151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:54.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1ac159970 0x7fb1ac151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54760/0 (socket says 172.21.15.46:54760) 2024-03-29T08:22:54.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b2000640 1 -- 172.21.15.46:0/3117281429 learned_addr learned my addr 172.21.15.46:0/3117281429 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b1600640 1 --2- 172.21.15.46:0/3117281429 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1ac070b90 0x7fb1ac150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b0c00640 1 --2- 172.21.15.46:0/3117281429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1ac0711f0 0x7fb1ac14ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b2000640 1 -- 172.21.15.46:0/3117281429 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1ac070b90 msgr2=0x7fb1ac150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b2000640 1 --2- 172.21.15.46:0/3117281429 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1ac070b90 0x7fb1ac150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b2000640 1 -- 172.21.15.46:0/3117281429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1ac0711f0 msgr2=0x7fb1ac14ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b2000640 1 --2- 172.21.15.46:0/3117281429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1ac0711f0 0x7fb1ac14ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b2000640 1 -- 172.21.15.46:0/3117281429 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a4009b30 con 0x7fb1ac159970 2024-03-29T08:22:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b2000640 1 --2- 172.21.15.46:0/3117281429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1ac159970 0x7fb1ac151760 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40af040 tx=0x7fb1a4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1a9600640 1 -- 172.21.15.46:0/3117281429 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a40b8670 con 0x7fb1ac159970 2024-03-29T08:22:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1a9600640 1 -- 172.21.15.46:0/3117281429 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a40b87f0 con 0x7fb1ac159970 2024-03-29T08:22:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1a9600640 1 -- 172.21.15.46:0/3117281429 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a40b8ba0 con 0x7fb1ac159970 2024-03-29T08:22:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b4000640 1 -- 172.21.15.46:0/3117281429 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ac14da80 con 0x7fb1ac159970 2024-03-29T08:22:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b4000640 1 -- 172.21.15.46:0/3117281429 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ac14e040 con 0x7fb1ac159970 2024-03-29T08:22:54.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1b4000640 1 -- 172.21.15.46:0/3117281429 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1ac0667c0 con 0x7fb1ac159970 2024-03-29T08:22:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.058+0000 7fb1a9600640 1 -- 172.21.15.46:0/3117281429 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb1a40b8d00 con 0x7fb1ac159970 2024-03-29T08:22:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.062+0000 7fb1a9600640 1 --2- 172.21.15.46:0/3117281429 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb18c066690 0x7fb18c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.062+0000 7fb1a9600640 1 -- 172.21.15.46:0/3117281429 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1a411f520 con 0x7fb1ac159970 2024-03-29T08:22:54.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.062+0000 7fb1b1600640 1 --2- 172.21.15.46:0/3117281429 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb18c066690 0x7fb18c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:54.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.062+0000 7fb1b1600640 1 --2- 172.21.15.46:0/3117281429 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb18c066690 0x7fb18c068b50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009ac0 tx=0x7fb1a0009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:54.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.066+0000 7fb1a9600640 1 -- 172.21.15.46:0/3117281429 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb1a40ec9c0 con 0x7fb1ac159970 2024-03-29T08:22:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.226+0000 7fb1b4000640 1 -- 172.21.15.46:0/3117281429 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1ac04e120 con 0x7fb1ac159970 2024-03-29T08:22:54.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.226+0000 7fb1a9600640 1 -- 172.21.15.46:0/3117281429 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb1a4009510 con 0x7fb1ac159970 2024-03-29T08:22:54.230 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:22:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.230+0000 7fb1b4000640 1 -- 172.21.15.46:0/3117281429 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb18c066690 msgr2=0x7fb18c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.230+0000 7fb1b4000640 1 --2- 172.21.15.46:0/3117281429 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb18c066690 0x7fb18c068b50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0009ac0 tx=0x7fb1a0009190 comp rx=0 tx=0).stop 2024-03-29T08:22:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.230+0000 7fb1b4000640 1 -- 172.21.15.46:0/3117281429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1ac159970 msgr2=0x7fb1ac151760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.230+0000 7fb1b4000640 1 --2- 172.21.15.46:0/3117281429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1ac159970 0x7fb1ac151760 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40af040 tx=0x7fb1a4019790 comp rx=0 tx=0).stop 2024-03-29T08:22:54.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.230+0000 7fb1b4000640 1 -- 172.21.15.46:0/3117281429 shutdown_connections 2024-03-29T08:22:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.230+0000 7fb1b4000640 1 --2- 172.21.15.46:0/3117281429 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb18c066690 0x7fb18c068b50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.230+0000 7fb1b4000640 1 --2- 172.21.15.46:0/3117281429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1ac0711f0 0x7fb1ac14ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.230+0000 7fb1b4000640 1 --2- 172.21.15.46:0/3117281429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1ac159970 0x7fb1ac151760 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.230+0000 7fb1b4000640 1 --2- 172.21.15.46:0/3117281429 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb1ac070b90 0x7fb1ac150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.230+0000 7fb1b4000640 1 -- 172.21.15.46:0/3117281429 >> 172.21.15.46:0/3117281429 conn(0x7fb1ac0b0640 msgr2=0x7fb1ac072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.230+0000 7fb1b4000640 1 -- 172.21.15.46:0/3117281429 shutdown_connections 2024-03-29T08:22:54.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.230+0000 7fb1b4000640 1 -- 172.21.15.46:0/3117281429 wait complete. 2024-03-29T08:22:54.252 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:22:54.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.390+0000 7faea2a00640 1 Processor -- start 2024-03-29T08:22:54.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.390+0000 7faea2a00640 1 -- start start 2024-03-29T08:22:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.394+0000 7faea2a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fae9c06b4b0 0x7fae9c0b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.394+0000 7faea2a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9c157490 con 0x7fae9c15dbb0 2024-03-29T08:22:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.394+0000 7faea2a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9c0b2c90 con 0x7fae9c0b3f20 2024-03-29T08:22:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.394+0000 7faea2a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c0b29b0 con 0x7fae9c06b4b0 2024-03-29T08:22:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.394+0000 7fae9be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fae9c15dbb0 0x7fae9c06ada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57640/0 (socket says 172.21.15.46:57640) 2024-03-29T08:22:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.394+0000 7fae9be00640 1 -- 172.21.15.46:0/4155139610 learned_addr learned my addr 172.21.15.46:0/4155139610 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.394+0000 7fae93e00640 1 --2- 172.21.15.46:0/4155139610 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fae9c06b4b0 0x7fae9c0b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:54.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.394+0000 7fae9b400640 1 -- 172.21.15.46:0/4155139610 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882615204 0 0) 0x7fae9c157490 con 0x7fae9c15dbb0 2024-03-29T08:22:54.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.394+0000 7fae9b400640 1 -- 172.21.15.46:0/4155139610 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae80003530 con 0x7fae9c15dbb0 2024-03-29T08:22:54.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.394+0000 7fae9b400640 1 -- 172.21.15.46:0/4155139610 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152685935 0 0) 0x7fae9c0b2c90 con 0x7fae9c0b3f20 2024-03-29T08:22:54.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.394+0000 7fae9b400640 1 -- 172.21.15.46:0/4155139610 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae9c157490 con 0x7fae9c0b3f20 2024-03-29T08:22:54.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.394+0000 7fae93e00640 1 -- 172.21.15.46:0/4155139610 >> v1:172.21.15.73:6789/0 conn(0x7fae9c0b3f20 legacy=0x7fae9c0ac450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.394+0000 7fae93e00640 1 -- 172.21.15.46:0/4155139610 >> v1:172.21.15.46:6789/0 conn(0x7fae9c15dbb0 legacy=0x7fae9c06ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.394+0000 7fae93e00640 1 -- 172.21.15.46:0/4155139610 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9c0b3440 con 0x7fae9c06b4b0 2024-03-29T08:22:54.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.398+0000 7fae93e00640 1 --2- 172.21.15.46:0/4155139610 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fae9c06b4b0 0x7fae9c0b1d30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fae840097f0 tx=0x7fae8401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8b342c97f0124fb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:54.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.398+0000 7fae9b400640 1 -- 172.21.15.46:0/4155139610 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae840ab670 con 0x7fae9c06b4b0 2024-03-29T08:22:54.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.398+0000 7fae9b400640 1 -- 172.21.15.46:0/4155139610 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae840ab7d0 con 0x7fae9c06b4b0 2024-03-29T08:22:54.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.398+0000 7fae9b400640 1 -- 172.21.15.46:0/4155139610 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae840abb60 con 0x7fae9c06b4b0 2024-03-29T08:22:54.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.398+0000 7faea2a00640 1 -- 172.21.15.46:0/4155139610 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fae9c06b4b0 msgr2=0x7fae9c0b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.398+0000 7faea2a00640 1 --2- 172.21.15.46:0/4155139610 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fae9c06b4b0 0x7fae9c0b1d30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fae840097f0 tx=0x7fae8401c450 comp rx=0 tx=0).stop 2024-03-29T08:22:54.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.398+0000 7faea2a00640 1 -- 172.21.15.46:0/4155139610 shutdown_connections 2024-03-29T08:22:54.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.398+0000 7faea2a00640 1 --2- 172.21.15.46:0/4155139610 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fae9c06b4b0 0x7fae9c0b1d30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.398+0000 7faea2a00640 1 -- 172.21.15.46:0/4155139610 >> 172.21.15.46:0/4155139610 conn(0x7fae9c070ad0 msgr2=0x7fae9c070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.398+0000 7faea2a00640 1 -- 172.21.15.46:0/4155139610 shutdown_connections 2024-03-29T08:22:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.398+0000 7faea2a00640 1 -- 172.21.15.46:0/4155139610 wait complete. 2024-03-29T08:22:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.398+0000 7faea2a00640 1 Processor -- start 2024-03-29T08:22:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.398+0000 7faea2a00640 1 -- start start 2024-03-29T08:22:54.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7faea2a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fae9c06b4b0 0x7fae9c0cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:54.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7faea2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae9c0b3f20 0x7fae9c15be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:54.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7faea2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae9c15dbb0 0x7fae9c14fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:54.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7faea2a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c15f7f0 con 0x7fae9c0b3f20 2024-03-29T08:22:54.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7faea2a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c153080 con 0x7fae9c15dbb0 2024-03-29T08:22:54.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7faea2a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae9c153bc0 con 0x7fae9c06b4b0 2024-03-29T08:22:54.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7fae93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae9c0b3f20 0x7fae9c15be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:54.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7fae93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae9c0b3f20 0x7fae9c15be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54762/0 (socket says 172.21.15.46:54762) 2024-03-29T08:22:54.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7fae93e00640 1 -- 172.21.15.46:0/1577438698 learned_addr learned my addr 172.21.15.46:0/1577438698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7fae9be00640 1 --2- 172.21.15.46:0/1577438698 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fae9c06b4b0 0x7fae9c0cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7faea0a00640 1 --2- 172.21.15.46:0/1577438698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae9c15dbb0 0x7fae9c14fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7fae93e00640 1 -- 172.21.15.46:0/1577438698 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fae9c06b4b0 msgr2=0x7fae9c0cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7fae93e00640 1 --2- 172.21.15.46:0/1577438698 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fae9c06b4b0 0x7fae9c0cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7fae93e00640 1 -- 172.21.15.46:0/1577438698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae9c15dbb0 msgr2=0x7fae9c14fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7fae93e00640 1 --2- 172.21.15.46:0/1577438698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae9c15dbb0 0x7fae9c14fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7fae93e00640 1 -- 172.21.15.46:0/1577438698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae84009420 con 0x7fae9c0b3f20 2024-03-29T08:22:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7faea0a00640 1 --2- 172.21.15.46:0/1577438698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae9c15dbb0 0x7fae9c14fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7fae93e00640 1 --2- 172.21.15.46:0/1577438698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae9c0b3f20 0x7fae9c15be10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fae84019060 tx=0x7fae8401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7fae98c00640 1 -- 172.21.15.46:0/1577438698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae840bf610 con 0x7fae9c0b3f20 2024-03-29T08:22:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7fae98c00640 1 -- 172.21.15.46:0/1577438698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae840bf790 con 0x7fae9c0b3f20 2024-03-29T08:22:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7faea2a00640 1 -- 172.21.15.46:0/1577438698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae9c153290 con 0x7fae9c0b3f20 2024-03-29T08:22:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7fae98c00640 1 -- 172.21.15.46:0/1577438698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae840bfb40 con 0x7fae9c0b3f20 2024-03-29T08:22:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7faea2a00640 1 -- 172.21.15.46:0/1577438698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae9c153850 con 0x7fae9c0b3f20 2024-03-29T08:22:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.402+0000 7faea2a00640 1 -- 172.21.15.46:0/1577438698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae9c15e0b0 con 0x7fae9c0b3f20 2024-03-29T08:22:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.406+0000 7fae98c00640 1 -- 172.21.15.46:0/1577438698 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fae840b5400 con 0x7fae9c0b3f20 2024-03-29T08:22:54.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.406+0000 7fae98c00640 1 --2- 172.21.15.46:0/1577438698 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fae78066690 0x7fae78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:54.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.406+0000 7fae98c00640 1 -- 172.21.15.46:0/1577438698 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae8411eef0 con 0x7fae9c0b3f20 2024-03-29T08:22:54.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.406+0000 7fae9be00640 1 --2- 172.21.15.46:0/1577438698 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fae78066690 0x7fae78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:54.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.406+0000 7fae9be00640 1 --2- 172.21.15.46:0/1577438698 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fae78066690 0x7fae78068b50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fae8c007fd0 tx=0x7fae8c007d80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:54.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.410+0000 7fae98c00640 1 -- 172.21.15.46:0/1577438698 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fae840ec390 con 0x7fae9c0b3f20 2024-03-29T08:22:54.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.570+0000 7faea2a00640 1 -- 172.21.15.46:0/1577438698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae9c09d100 con 0x7fae9c0b3f20 2024-03-29T08:22:54.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.570+0000 7fae98c00640 1 -- 172.21.15.46:0/1577438698 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+11 (secure 0 0 0) 0x7fae840ef9b0 con 0x7fae9c0b3f20 2024-03-29T08:22:54.574 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:22:54.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.574+0000 7faea2a00640 1 -- 172.21.15.46:0/1577438698 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fae78066690 msgr2=0x7fae78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.574+0000 7faea2a00640 1 --2- 172.21.15.46:0/1577438698 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fae78066690 0x7fae78068b50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fae8c007fd0 tx=0x7fae8c007d80 comp rx=0 tx=0).stop 2024-03-29T08:22:54.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.574+0000 7faea2a00640 1 -- 172.21.15.46:0/1577438698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae9c0b3f20 msgr2=0x7fae9c15be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.574+0000 7faea2a00640 1 --2- 172.21.15.46:0/1577438698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae9c0b3f20 0x7fae9c15be10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fae84019060 tx=0x7fae8401ac90 comp rx=0 tx=0).stop 2024-03-29T08:22:54.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.574+0000 7faea2a00640 1 -- 172.21.15.46:0/1577438698 shutdown_connections 2024-03-29T08:22:54.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.574+0000 7faea2a00640 1 --2- 172.21.15.46:0/1577438698 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fae78066690 0x7fae78068b50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.574+0000 7faea2a00640 1 --2- 172.21.15.46:0/1577438698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae9c15dbb0 0x7fae9c14fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.574+0000 7faea2a00640 1 --2- 172.21.15.46:0/1577438698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae9c0b3f20 0x7fae9c15be10 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.574+0000 7faea2a00640 1 --2- 172.21.15.46:0/1577438698 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fae9c06b4b0 0x7fae9c0cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.574+0000 7faea2a00640 1 -- 172.21.15.46:0/1577438698 >> 172.21.15.46:0/1577438698 conn(0x7fae9c070ad0 msgr2=0x7fae9c06cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:54.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.574+0000 7faea2a00640 1 -- 172.21.15.46:0/1577438698 shutdown_connections 2024-03-29T08:22:54.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.574+0000 7faea2a00640 1 -- 172.21.15.46:0/1577438698 wait complete. 2024-03-29T08:22:54.595 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:22:54.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.734+0000 7f9ceba00640 1 Processor -- start 2024-03-29T08:22:54.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.734+0000 7f9ceba00640 1 -- start start 2024-03-29T08:22:54.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.738+0000 7f9ceba00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ce415efe0 0x7f9ce40b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:54.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.738+0000 7f9ceba00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce40b2360 con 0x7f9ce4143eb0 2024-03-29T08:22:54.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.738+0000 7f9ceba00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce415e500 con 0x7f9ce40cbea0 2024-03-29T08:22:54.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.738+0000 7f9ceba00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce415e7e0 con 0x7f9ce415efe0 2024-03-29T08:22:54.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.738+0000 7f9ce9000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9ce4143eb0 0x7f9ce40cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57654/0 (socket says 172.21.15.46:57654) 2024-03-29T08:22:54.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.738+0000 7f9ce9000640 1 -- 172.21.15.46:0/29650923 learned_addr learned my addr 172.21.15.46:0/29650923 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:54.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.738+0000 7f9ce9a00640 1 --2- 172.21.15.46:0/29650923 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ce415efe0 0x7f9ce40b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:54.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.738+0000 7f9ce3e00640 1 -- 172.21.15.46:0/29650923 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2504880644 0 0) 0x7f9ce40b2360 con 0x7f9ce4143eb0 2024-03-29T08:22:54.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.738+0000 7f9ce3e00640 1 -- 172.21.15.46:0/29650923 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cc8003530 con 0x7f9ce4143eb0 2024-03-29T08:22:54.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.738+0000 7f9ce3e00640 1 -- 172.21.15.46:0/29650923 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376935393 0 0) 0x7f9ce415e500 con 0x7f9ce40cbea0 2024-03-29T08:22:54.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.738+0000 7f9ce3e00640 1 -- 172.21.15.46:0/29650923 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce40b2360 con 0x7f9ce40cbea0 2024-03-29T08:22:54.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.738+0000 7f9ce9a00640 1 -- 172.21.15.46:0/29650923 >> v1:172.21.15.73:6789/0 conn(0x7f9ce40cbea0 legacy=0x7f9ce40b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.738+0000 7f9ce9a00640 1 -- 172.21.15.46:0/29650923 >> v1:172.21.15.46:6789/0 conn(0x7f9ce4143eb0 legacy=0x7f9ce40cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.738+0000 7f9ce9a00640 1 -- 172.21.15.46:0/29650923 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce415eac0 con 0x7f9ce415efe0 2024-03-29T08:22:54.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.742+0000 7f9ce9a00640 1 --2- 172.21.15.46:0/29650923 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ce415efe0 0x7f9ce40b1d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009dc0 tx=0x7f9cdc01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a237a33c5c9f96ed server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:54.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.742+0000 7f9ce3e00640 1 -- 172.21.15.46:0/29650923 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0a5a70 con 0x7f9ce415efe0 2024-03-29T08:22:54.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.742+0000 7f9ce3e00640 1 -- 172.21.15.46:0/29650923 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc0a5bd0 con 0x7f9ce415efe0 2024-03-29T08:22:54.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.742+0000 7f9ce3e00640 1 -- 172.21.15.46:0/29650923 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0ad440 con 0x7f9ce415efe0 2024-03-29T08:22:54.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.742+0000 7f9ceba00640 1 -- 172.21.15.46:0/29650923 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ce415efe0 msgr2=0x7f9ce40b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.742+0000 7f9ceba00640 1 --2- 172.21.15.46:0/29650923 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ce415efe0 0x7f9ce40b1d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009dc0 tx=0x7f9cdc01c990 comp rx=0 tx=0).stop 2024-03-29T08:22:54.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.742+0000 7f9ceba00640 1 -- 172.21.15.46:0/29650923 shutdown_connections 2024-03-29T08:22:54.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.742+0000 7f9ceba00640 1 --2- 172.21.15.46:0/29650923 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ce415efe0 0x7f9ce40b1d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.742+0000 7f9ceba00640 1 -- 172.21.15.46:0/29650923 >> 172.21.15.46:0/29650923 conn(0x7f9ce406a390 msgr2=0x7f9ce40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:54.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.742+0000 7f9ceba00640 1 -- 172.21.15.46:0/29650923 shutdown_connections 2024-03-29T08:22:54.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.742+0000 7f9ceba00640 1 -- 172.21.15.46:0/29650923 wait complete. 2024-03-29T08:22:54.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.742+0000 7f9ceba00640 1 Processor -- start 2024-03-29T08:22:54.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.742+0000 7f9ceba00640 1 -- start start 2024-03-29T08:22:54.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ceba00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ce40cbea0 0x7f9ce413b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ceba00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ce4143eb0 0x7f9ce4155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ceba00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce415efe0 0x7f9ce4138670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ceba00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce4155c40 con 0x7f9ce415efe0 2024-03-29T08:22:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ceba00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce4155f20 con 0x7f9ce4143eb0 2024-03-29T08:22:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ceba00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce4156200 con 0x7f9ce40cbea0 2024-03-29T08:22:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ce9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce415efe0 0x7f9ce4138670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ce9a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce415efe0 0x7f9ce4138670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54776/0 (socket says 172.21.15.46:54776) 2024-03-29T08:22:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ce9a00640 1 -- 172.21.15.46:0/2441318532 learned_addr learned my addr 172.21.15.46:0/2441318532 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9cdbe00640 1 --2- 172.21.15.46:0/2441318532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ce4143eb0 0x7f9ce4155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ce9000640 1 --2- 172.21.15.46:0/2441318532 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ce40cbea0 0x7f9ce413b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ce9a00640 1 -- 172.21.15.46:0/2441318532 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ce40cbea0 msgr2=0x7f9ce413b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ce9a00640 1 --2- 172.21.15.46:0/2441318532 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ce40cbea0 0x7f9ce413b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ce9a00640 1 -- 172.21.15.46:0/2441318532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ce4143eb0 msgr2=0x7f9ce4155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ce9a00640 1 --2- 172.21.15.46:0/2441318532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ce4143eb0 0x7f9ce4155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ce9a00640 1 -- 172.21.15.46:0/2441318532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cdc009a70 con 0x7f9ce415efe0 2024-03-29T08:22:54.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9cdbe00640 1 --2- 172.21.15.46:0/2441318532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ce4143eb0 0x7f9ce4155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:22:54.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ce9a00640 1 --2- 172.21.15.46:0/2441318532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce415efe0 0x7f9ce4138670 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc01a5c0 tx=0x7f9cdc019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:54.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ce1600640 1 -- 172.21.15.46:0/2441318532 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0a5a70 con 0x7f9ce415efe0 2024-03-29T08:22:54.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ce1600640 1 -- 172.21.15.46:0/2441318532 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc009450 con 0x7f9ce415efe0 2024-03-29T08:22:54.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ce1600640 1 -- 172.21.15.46:0/2441318532 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc0b7650 con 0x7f9ce415efe0 2024-03-29T08:22:54.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ceba00640 1 -- 172.21.15.46:0/2441318532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce41564e0 con 0x7f9ce415efe0 2024-03-29T08:22:54.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ceba00640 1 -- 172.21.15.46:0/2441318532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce4155550 con 0x7f9ce415efe0 2024-03-29T08:22:54.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ceba00640 1 -- 172.21.15.46:0/2441318532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce4071a70 con 0x7f9ce415efe0 2024-03-29T08:22:54.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.746+0000 7f9ce1600640 1 -- 172.21.15.46:0/2441318532 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9cdc0b4050 con 0x7f9ce415efe0 2024-03-29T08:22:54.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.750+0000 7f9ce1600640 1 --2- 172.21.15.46:0/2441318532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9cc0066690 0x7f9cc0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:54.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.750+0000 7f9ce1600640 1 -- 172.21.15.46:0/2441318532 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9cdc11f3c0 con 0x7f9ce415efe0 2024-03-29T08:22:54.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.750+0000 7f9ce9000640 1 --2- 172.21.15.46:0/2441318532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9cc0066690 0x7f9cc0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:54.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.750+0000 7f9ce9000640 1 --2- 172.21.15.46:0/2441318532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9cc0066690 0x7f9cc0068b50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4000f70 tx=0x7f9cd4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:54.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.754+0000 7f9ce1600640 1 -- 172.21.15.46:0/2441318532 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9cdc0ec7f0 con 0x7f9ce415efe0 2024-03-29T08:22:54.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.914+0000 7f9ceba00640 1 -- 172.21.15.46:0/2441318532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ce4156990 con 0x7f9ce415efe0 2024-03-29T08:22:54.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.914+0000 7f9ce1600640 1 -- 172.21.15.46:0/2441318532 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9cdc0efe10 con 0x7f9ce415efe0 2024-03-29T08:22:54.918 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:22:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.918+0000 7f9ceba00640 1 -- 172.21.15.46:0/2441318532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9cc0066690 msgr2=0x7f9cc0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.918+0000 7f9ceba00640 1 --2- 172.21.15.46:0/2441318532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9cc0066690 0x7f9cc0068b50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4000f70 tx=0x7f9cd4009040 comp rx=0 tx=0).stop 2024-03-29T08:22:54.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.918+0000 7f9ceba00640 1 -- 172.21.15.46:0/2441318532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce415efe0 msgr2=0x7f9ce4138670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:54.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.918+0000 7f9ceba00640 1 --2- 172.21.15.46:0/2441318532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce415efe0 0x7f9ce4138670 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc01a5c0 tx=0x7f9cdc019730 comp rx=0 tx=0).stop 2024-03-29T08:22:54.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.918+0000 7f9ceba00640 1 -- 172.21.15.46:0/2441318532 shutdown_connections 2024-03-29T08:22:54.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.918+0000 7f9ceba00640 1 --2- 172.21.15.46:0/2441318532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9cc0066690 0x7f9cc0068b50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.918+0000 7f9ceba00640 1 --2- 172.21.15.46:0/2441318532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ce4143eb0 0x7f9ce4155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.918+0000 7f9ceba00640 1 --2- 172.21.15.46:0/2441318532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ce415efe0 0x7f9ce4138670 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.918+0000 7f9ceba00640 1 --2- 172.21.15.46:0/2441318532 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ce40cbea0 0x7f9ce413b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:54.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.918+0000 7f9ceba00640 1 -- 172.21.15.46:0/2441318532 >> 172.21.15.46:0/2441318532 conn(0x7f9ce406a390 msgr2=0x7f9ce40cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:54.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.918+0000 7f9ceba00640 1 -- 172.21.15.46:0/2441318532 shutdown_connections 2024-03-29T08:22:54.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:54.918+0000 7f9ceba00640 1 -- 172.21.15.46:0/2441318532 wait complete. 2024-03-29T08:22:54.940 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:22:55.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.078+0000 7f78f8a00640 1 Processor -- start 2024-03-29T08:22:55.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.078+0000 7f78f8a00640 1 -- start start 2024-03-29T08:22:55.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.078+0000 7f78f8a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f78f4142ab0 0x7f78f406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:55.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.078+0000 7f78f8a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f406d9e0 0x7f78f415d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:55.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.078+0000 7f78f8a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f406de40 con 0x7f78f415d940 2024-03-29T08:22:55.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.078+0000 7f78f8a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f4070c70 con 0x7f78f4142ab0 2024-03-29T08:22:55.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.078+0000 7f78f8a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f4070340 con 0x7f78f406d9e0 2024-03-29T08:22:55.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.082+0000 7f78e9400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f406d9e0 0x7f78f415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:55.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.082+0000 7f78f2800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f78f415d940 0x7f78f40ae210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:48910/0 (socket says 172.21.15.46:48910) 2024-03-29T08:22:55.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.082+0000 7f78f2800640 1 -- 172.21.15.46:0/2915084560 learned_addr learned my addr 172.21.15.46:0/2915084560 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:22:55.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.082+0000 7f78f1e00640 1 --2- 172.21.15.46:0/2915084560 >> v2:172.21.15.73:3300/0 conn(0x7f78f4142ab0 0x7f78f406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:55.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.082+0000 7f78f1400640 1 -- 172.21.15.46:0/2915084560 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350094340 0 0) 0x7f78f406de40 con 0x7f78f415d940 2024-03-29T08:22:55.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.082+0000 7f78f1400640 1 -- 172.21.15.46:0/2915084560 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78d8003530 con 0x7f78f415d940 2024-03-29T08:22:55.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.082+0000 7f78f1e00640 1 -- 172.21.15.46:0/2915084560 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f406d9e0 msgr2=0x7f78f415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:22:55.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.082+0000 7f78f1e00640 1 --2- 172.21.15.46:0/2915084560 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f406d9e0 0x7f78f415d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:55.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.082+0000 7f78f1e00640 1 -- 172.21.15.46:0/2915084560 >> v1:172.21.15.73:6789/0 conn(0x7f78f415d940 legacy=0x7f78f40ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:55.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.082+0000 7f78f1e00640 1 -- 172.21.15.46:0/2915084560 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f4070620 con 0x7f78f4142ab0 2024-03-29T08:22:55.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.082+0000 7f78e9400640 1 --2- 172.21.15.46:0/2915084560 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f406d9e0 0x7f78f415d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:22:55.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.082+0000 7f78f1e00640 1 --2- 172.21.15.46:0/2915084560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f4142ab0 0x7f78f406d4a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f78e40097c0 tx=0x7f78e401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ec4407191b6c13b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:55.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f1400640 1 -- 172.21.15.46:0/2915084560 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e40ab600 con 0x7f78f4142ab0 2024-03-29T08:22:55.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f1400640 1 -- 172.21.15.46:0/2915084560 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e40a7050 con 0x7f78f4142ab0 2024-03-29T08:22:55.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f1400640 1 -- 172.21.15.46:0/2915084560 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e40ab960 con 0x7f78f4142ab0 2024-03-29T08:22:55.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f8a00640 1 -- 172.21.15.46:0/2915084560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f4142ab0 msgr2=0x7f78f406d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:55.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f8a00640 1 --2- 172.21.15.46:0/2915084560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f4142ab0 0x7f78f406d4a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f78e40097c0 tx=0x7f78e401c4a0 comp rx=0 tx=0).stop 2024-03-29T08:22:55.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f8a00640 1 -- 172.21.15.46:0/2915084560 shutdown_connections 2024-03-29T08:22:55.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f8a00640 1 --2- 172.21.15.46:0/2915084560 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f406d9e0 0x7f78f415d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:55.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f8a00640 1 --2- 172.21.15.46:0/2915084560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f4142ab0 0x7f78f406d4a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:55.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f8a00640 1 -- 172.21.15.46:0/2915084560 >> 172.21.15.46:0/2915084560 conn(0x7f78f4066080 msgr2=0x7f78f4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:55.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f8a00640 1 -- 172.21.15.46:0/2915084560 shutdown_connections 2024-03-29T08:22:55.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f8a00640 1 -- 172.21.15.46:0/2915084560 wait complete. 2024-03-29T08:22:55.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f8a00640 1 Processor -- start 2024-03-29T08:22:55.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f8a00640 1 -- start start 2024-03-29T08:22:55.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f8a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f406d9e0 0x7f78f40a7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:55.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f4142ab0 0x7f78f4163160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:55.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f415d940 0x7f78f41f8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:55.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f8a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f413ef60 con 0x7f78f415d940 2024-03-29T08:22:55.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f8a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f41f6210 con 0x7f78f4142ab0 2024-03-29T08:22:55.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f8a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f4163a20 con 0x7f78f406d9e0 2024-03-29T08:22:55.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f415d940 0x7f78f41f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:55.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f1e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f406d9e0 0x7f78f40a7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:55.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78e9400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f4142ab0 0x7f78f4163160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:55.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f415d940 0x7f78f41f8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54784/0 (socket says 172.21.15.46:54784) 2024-03-29T08:22:55.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f2800640 1 -- 172.21.15.46:0/1922705278 learned_addr learned my addr 172.21.15.46:0/1922705278 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:22:55.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.086+0000 7f78f1e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f406d9e0 0x7f78f40a7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:55658/0 (socket says 172.21.15.46:55658) 2024-03-29T08:22:55.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.090+0000 7f78f1e00640 1 -- 172.21.15.46:0/1922705278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f4142ab0 msgr2=0x7f78f4163160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:55.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.090+0000 7f78f1e00640 1 --2- 172.21.15.46:0/1922705278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f4142ab0 0x7f78f4163160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:55.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.090+0000 7f78f1e00640 1 -- 172.21.15.46:0/1922705278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f415d940 msgr2=0x7f78f41f8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:55.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.090+0000 7f78f1e00640 1 --2- 172.21.15.46:0/1922705278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f415d940 0x7f78f41f8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:55.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.090+0000 7f78f1e00640 1 -- 172.21.15.46:0/1922705278 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78dc00bc80 con 0x7f78f406d9e0 2024-03-29T08:22:55.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.090+0000 7f78f2800640 1 --2- 172.21.15.46:0/1922705278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f415d940 0x7f78f41f8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:22:55.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.090+0000 7f78f1e00640 1 --2- 172.21.15.46:0/1922705278 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f406d9e0 0x7f78f40a7ad0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f78f406e430 tx=0x7f78e4019240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:55.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.090+0000 7f78eaa00640 1 -- 172.21.15.46:0/1922705278 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e40b4400 con 0x7f78f406d9e0 2024-03-29T08:22:55.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.090+0000 7f78eaa00640 1 -- 172.21.15.46:0/1922705278 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e40b4560 con 0x7f78f406d9e0 2024-03-29T08:22:55.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.090+0000 7f78eaa00640 1 -- 172.21.15.46:0/1922705278 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e40b48f0 con 0x7f78f406d9e0 2024-03-29T08:22:55.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.090+0000 7f78f8a00640 1 -- 172.21.15.46:0/1922705278 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e4009420 con 0x7f78f406d9e0 2024-03-29T08:22:55.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.090+0000 7f78f8a00640 1 -- 172.21.15.46:0/1922705278 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f4163fe0 con 0x7f78f406d9e0 2024-03-29T08:22:55.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.090+0000 7f78f8a00640 1 -- 172.21.15.46:0/1922705278 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f4071a90 con 0x7f78f406d9e0 2024-03-29T08:22:55.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.090+0000 7f78eaa00640 1 -- 172.21.15.46:0/1922705278 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f78e40b2040 con 0x7f78f406d9e0 2024-03-29T08:22:55.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.094+0000 7f78eaa00640 1 --2- 172.21.15.46:0/1922705278 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78d0066690 0x7f78d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:22:55.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.094+0000 7f78eaa00640 1 -- 172.21.15.46:0/1922705278 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78e411e950 con 0x7f78f406d9e0 2024-03-29T08:22:55.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.094+0000 7f78e9400640 1 --2- 172.21.15.46:0/1922705278 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78d0066690 0x7f78d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:22:55.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.094+0000 7f78e9400640 1 --2- 172.21.15.46:0/1922705278 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78d0066690 0x7f78d0068b50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f78f40b3920 tx=0x7f78dc00d040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:22:55.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.098+0000 7f78eaa00640 1 -- 172.21.15.46:0/1922705278 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f78e40ebd50 con 0x7f78f406d9e0 2024-03-29T08:22:55.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.270+0000 7f78f8a00640 1 -- 172.21.15.46:0/1922705278 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f78f41f6650 con 0x7f78f406d9e0 2024-03-29T08:22:55.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.270+0000 7f78eaa00640 1 -- 172.21.15.46:0/1922705278 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v77) v1 ==== 80+0+0 (secure 0 0 0) 0x7f78e40b0080 con 0x7f78f406d9e0 2024-03-29T08:22:55.274 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:22:55.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.274+0000 7f78f8a00640 1 -- 172.21.15.46:0/1922705278 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78d0066690 msgr2=0x7f78d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:55.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.274+0000 7f78f8a00640 1 --2- 172.21.15.46:0/1922705278 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78d0066690 0x7f78d0068b50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f78f40b3920 tx=0x7f78dc00d040 comp rx=0 tx=0).stop 2024-03-29T08:22:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.274+0000 7f78f8a00640 1 -- 172.21.15.46:0/1922705278 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f406d9e0 msgr2=0x7f78f40a7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:22:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.274+0000 7f78f8a00640 1 --2- 172.21.15.46:0/1922705278 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f406d9e0 0x7f78f40a7ad0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f78f406e430 tx=0x7f78e4019240 comp rx=0 tx=0).stop 2024-03-29T08:22:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.274+0000 7f78f8a00640 1 -- 172.21.15.46:0/1922705278 shutdown_connections 2024-03-29T08:22:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.274+0000 7f78f8a00640 1 --2- 172.21.15.46:0/1922705278 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78d0066690 0x7f78d0068b50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.274+0000 7f78f8a00640 1 --2- 172.21.15.46:0/1922705278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f4142ab0 0x7f78f4163160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.274+0000 7f78f8a00640 1 --2- 172.21.15.46:0/1922705278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f415d940 0x7f78f41f8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.274+0000 7f78f8a00640 1 --2- 172.21.15.46:0/1922705278 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f406d9e0 0x7f78f40a7ad0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:22:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.274+0000 7f78f8a00640 1 -- 172.21.15.46:0/1922705278 >> 172.21.15.46:0/1922705278 conn(0x7f78f4066080 msgr2=0x7f78f4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:22:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.274+0000 7f78f8a00640 1 -- 172.21.15.46:0/1922705278 shutdown_connections 2024-03-29T08:22:55.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:22:55.274+0000 7f78f8a00640 1 -- 172.21.15.46:0/1922705278 wait complete. 2024-03-29T08:22:55.296 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-03-29T08:22:55.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:22:55.296 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T08:22:55.314 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[41766]: fuse finished with error 0 and tester_r 0 2024-03-29T08:22:55.317 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:23:01.418 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:23:01.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:01.418 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:23:01.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:23:01.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.622+0000 7f799e200640 1 Processor -- start 2024-03-29T08:23:01.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.622+0000 7f799e200640 1 -- start start 2024-03-29T08:23:01.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.622+0000 7f799e200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7998161170 0x7f79980ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:01.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.622+0000 7f799e200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f79980cb550 0x7f799815c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:01.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.622+0000 7f799e200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7998160440 con 0x7f7998069d90 2024-03-29T08:23:01.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.622+0000 7f799e200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7998160ce0 con 0x7f7998161170 2024-03-29T08:23:01.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.622+0000 7f799e200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79980cefa0 con 0x7f79980cb550 2024-03-29T08:23:01.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.622+0000 7f7997e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7998161170 0x7f79980ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:01.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.622+0000 7f799ce00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7998069d90 0x7f799806a170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45430/0 (socket says 172.21.15.46:45430) 2024-03-29T08:23:01.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.622+0000 7f799ce00640 1 -- 172.21.15.46:0/131150759 learned_addr learned my addr 172.21.15.46:0/131150759 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:01.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.622+0000 7f799d800640 1 --2- 172.21.15.46:0/131150759 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f79980cb550 0x7f799815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:01.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.626+0000 7f7997e00640 1 -- 172.21.15.46:0/131150759 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f79980cb550 msgr2=0x7f799815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:01.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.626+0000 7f7997e00640 1 --2- 172.21.15.46:0/131150759 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f79980cb550 0x7f799815c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:01.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.626+0000 7f7997e00640 1 -- 172.21.15.46:0/131150759 >> v1:172.21.15.46:6789/0 conn(0x7f7998069d90 legacy=0x7f799806a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.626+0000 7f7997e00640 1 -- 172.21.15.46:0/131150759 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7998160690 con 0x7f7998161170 2024-03-29T08:23:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.626+0000 7f7997400640 1 -- 172.21.15.46:0/131150759 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 650484335 0 0) 0x7f7998160440 con 0x7f7998069d90 2024-03-29T08:23:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.626+0000 7f7997e00640 1 --2- 172.21.15.46:0/131150759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998161170 0x7f79980ce670 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f79840097c0 tx=0x7f798401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92674b772d7c171d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.626+0000 7f7997400640 1 -- 172.21.15.46:0/131150759 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79840ab600 con 0x7f7998161170 2024-03-29T08:23:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.626+0000 7f7997400640 1 -- 172.21.15.46:0/131150759 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79840ab760 con 0x7f7998161170 2024-03-29T08:23:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.626+0000 7f7997400640 1 -- 172.21.15.46:0/131150759 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79840abaf0 con 0x7f7998161170 2024-03-29T08:23:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.626+0000 7f799e200640 1 -- 172.21.15.46:0/131150759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998161170 msgr2=0x7f79980ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.626+0000 7f799e200640 1 --2- 172.21.15.46:0/131150759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998161170 0x7f79980ce670 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f79840097c0 tx=0x7f798401c3e0 comp rx=0 tx=0).stop 2024-03-29T08:23:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.626+0000 7f799e200640 1 -- 172.21.15.46:0/131150759 shutdown_connections 2024-03-29T08:23:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.626+0000 7f799e200640 1 --2- 172.21.15.46:0/131150759 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f79980cb550 0x7f799815c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.626+0000 7f799e200640 1 --2- 172.21.15.46:0/131150759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998161170 0x7f79980ce670 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.626+0000 7f799e200640 1 -- 172.21.15.46:0/131150759 >> 172.21.15.46:0/131150759 conn(0x7f7998098a40 msgr2=0x7f79980cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:01.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799e200640 1 -- 172.21.15.46:0/131150759 shutdown_connections 2024-03-29T08:23:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799e200640 1 -- 172.21.15.46:0/131150759 wait complete. 2024-03-29T08:23:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799e200640 1 Processor -- start 2024-03-29T08:23:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799e200640 1 -- start start 2024-03-29T08:23:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998069d90 0x7f7998151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79980cb550 0x7f7998157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799e200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7998161170 0x7f799814e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7998156600 con 0x7f7998069d90 2024-03-29T08:23:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799e200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79980c5e50 con 0x7f79980cb550 2024-03-29T08:23:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799e200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7998155ff0 con 0x7f7998161170 2024-03-29T08:23:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998069d90 0x7f7998151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998069d90 0x7f7998151680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38966/0 (socket says 172.21.15.46:38966) 2024-03-29T08:23:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799ce00640 1 -- 172.21.15.46:0/1572584115 learned_addr learned my addr 172.21.15.46:0/1572584115 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:01.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799d800640 1 --2- 172.21.15.46:0/1572584115 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7998161170 0x7f799814e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f7997e00640 1 --2- 172.21.15.46:0/1572584115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79980cb550 0x7f7998157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799ce00640 1 -- 172.21.15.46:0/1572584115 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7998161170 msgr2=0x7f799814e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799ce00640 1 --2- 172.21.15.46:0/1572584115 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7998161170 0x7f799814e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799ce00640 1 -- 172.21.15.46:0/1572584115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79980cb550 msgr2=0x7f7998157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799ce00640 1 --2- 172.21.15.46:0/1572584115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79980cb550 0x7f7998157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799ce00640 1 -- 172.21.15.46:0/1572584115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7984009420 con 0x7f7998069d90 2024-03-29T08:23:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799ce00640 1 --2- 172.21.15.46:0/1572584115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998069d90 0x7f7998151680 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f798c0011f0 tx=0x7f798c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f7994c00640 1 -- 172.21.15.46:0/1572584115 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c0074e0 con 0x7f7998069d90 2024-03-29T08:23:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f7994c00640 1 -- 172.21.15.46:0/1572584115 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798c007640 con 0x7f7998069d90 2024-03-29T08:23:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f7994c00640 1 -- 172.21.15.46:0/1572584115 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798c004a10 con 0x7f7998069d90 2024-03-29T08:23:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.630+0000 7f799e200640 1 -- 172.21.15.46:0/1572584115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799814ef30 con 0x7f7998069d90 2024-03-29T08:23:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.634+0000 7f799e200640 1 -- 172.21.15.46:0/1572584115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7998156290 con 0x7f7998069d90 2024-03-29T08:23:01.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.634+0000 7f799e200640 1 -- 172.21.15.46:0/1572584115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799804e040 con 0x7f7998069d90 2024-03-29T08:23:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.634+0000 7f7994c00640 1 -- 172.21.15.46:0/1572584115 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f798c004b70 con 0x7f7998069d90 2024-03-29T08:23:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.634+0000 7f7994c00640 1 --2- 172.21.15.46:0/1572584115 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f79780666c0 0x7f7978068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.634+0000 7f7994c00640 1 -- 172.21.15.46:0/1572584115 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f798c083310 con 0x7f7998069d90 2024-03-29T08:23:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.634+0000 7f7997e00640 1 --2- 172.21.15.46:0/1572584115 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f79780666c0 0x7f7978068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.634+0000 7f7997e00640 1 --2- 172.21.15.46:0/1572584115 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f79780666c0 0x7f7978068b80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f79840af000 tx=0x7f79840b6040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:01.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.638+0000 7f7994c00640 1 -- 172.21.15.46:0/1572584115 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f798c0506e0 con 0x7f7998069d90 2024-03-29T08:23:01.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.830+0000 7f799e200640 1 -- 172.21.15.46:0/1572584115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7998066ea0 con 0x7f7998069d90 2024-03-29T08:23:01.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.834+0000 7f7994c00640 1 -- 172.21.15.46:0/1572584115 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+4727 (secure 0 0 0) 0x7f798c053d00 con 0x7f7998069d90 2024-03-29T08:23:01.836 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:23:01.836 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":42,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3456771376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3456771376},{"type":"v1","addr":"172.21.15.73:6836","nonce":3456771376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2671538025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2671538025},{"type":"v1","addr":"172.21.15.73:6837","nonce":2671538025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3196015693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3196015693},{"type":"v1","addr":"172.21.15.46:6833","nonce":3196015693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":42,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:22:43.605457+0000","modified":"2024-03-29T08:22:46.620088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/9617787","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":9617787},{"type":"v1","addr":"172.21.15.46:6835","nonce":9617787}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-29T08:23:01.836 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 42 2024-03-29T08:23:01.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.834+0000 7f799e200640 1 -- 172.21.15.46:0/1572584115 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f79780666c0 msgr2=0x7f7978068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:01.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.834+0000 7f799e200640 1 --2- 172.21.15.46:0/1572584115 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f79780666c0 0x7f7978068b80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f79840af000 tx=0x7f79840b6040 comp rx=0 tx=0).stop 2024-03-29T08:23:01.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.834+0000 7f799e200640 1 -- 172.21.15.46:0/1572584115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998069d90 msgr2=0x7f7998151680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:01.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.834+0000 7f799e200640 1 --2- 172.21.15.46:0/1572584115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998069d90 0x7f7998151680 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f798c0011f0 tx=0x7f798c00c9b0 comp rx=0 tx=0).stop 2024-03-29T08:23:01.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.834+0000 7f799e200640 1 -- 172.21.15.46:0/1572584115 shutdown_connections 2024-03-29T08:23:01.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.834+0000 7f799e200640 1 --2- 172.21.15.46:0/1572584115 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f79780666c0 0x7f7978068b80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:01.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.834+0000 7f799e200640 1 --2- 172.21.15.46:0/1572584115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79980cb550 0x7f7998157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:01.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.834+0000 7f799e200640 1 --2- 172.21.15.46:0/1572584115 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7998161170 0x7f799814e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:01.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.834+0000 7f799e200640 1 --2- 172.21.15.46:0/1572584115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998069d90 0x7f7998151680 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:01.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.834+0000 7f799e200640 1 -- 172.21.15.46:0/1572584115 >> 172.21.15.46:0/1572584115 conn(0x7f7998098a40 msgr2=0x7f79980c9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:01.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.834+0000 7f799e200640 1 -- 172.21.15.46:0/1572584115 shutdown_connections 2024-03-29T08:23:01.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:01.834+0000 7f799e200640 1 -- 172.21.15.46:0/1572584115 wait complete. 2024-03-29T08:23:01.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-03-29T08:23:02.032 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-03-29T08:23:02.033 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-29T08:23:02.033 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T08:23:02.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.170+0000 7f1d93600640 1 Processor -- start 2024-03-29T08:23:02.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.170+0000 7f1d93600640 1 -- start start 2024-03-29T08:23:02.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.174+0000 7f1d93600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1d8c15cbf0 0x7f1d8c0ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.174+0000 7f1d93600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1d8c06f140 0x7f1d8c0ac660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.174+0000 7f1d93600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d8c0cf3f0 con 0x7f1d8c0b1cf0 2024-03-29T08:23:02.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.174+0000 7f1d93600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d8c0ceac0 con 0x7f1d8c06f140 2024-03-29T08:23:02.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.174+0000 7f1d93600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c0ceda0 con 0x7f1d8c15cbf0 2024-03-29T08:23:02.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.174+0000 7f1d83e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1d8c0b1cf0 0x7f1d8c06ea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45446/0 (socket says 172.21.15.46:45446) 2024-03-29T08:23:02.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.174+0000 7f1d83e00640 1 -- 172.21.15.46:0/2563161284 learned_addr learned my addr 172.21.15.46:0/2563161284 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:02.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.174+0000 7f1d90c00640 1 --2- 172.21.15.46:0/2563161284 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1d8c15cbf0 0x7f1d8c0ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.174+0000 7f1d91600640 1 --2- 172.21.15.46:0/2563161284 >> v2:172.21.15.73:3300/0 conn(0x7f1d8c06f140 0x7f1d8c0ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.174+0000 7f1d8be00640 1 -- 172.21.15.46:0/2563161284 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1680096014 0 0) 0x7f1d74001120 con 0x7f1d8c0b1cf0 2024-03-29T08:23:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.174+0000 7f1d8be00640 1 -- 172.21.15.46:0/2563161284 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d70003530 con 0x7f1d8c0b1cf0 2024-03-29T08:23:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.174+0000 7f1d8be00640 1 -- 172.21.15.46:0/2563161284 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3858635132 0 0) 0x7f1d70003530 con 0x7f1d8c0b1cf0 2024-03-29T08:23:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.174+0000 7f1d90c00640 1 -- 172.21.15.46:0/2563161284 >> v2:172.21.15.73:3300/0 conn(0x7f1d8c06f140 msgr2=0x7f1d8c0ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.174+0000 7f1d90c00640 1 --2- 172.21.15.46:0/2563161284 >> v2:172.21.15.73:3300/0 conn(0x7f1d8c06f140 0x7f1d8c0ac660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.174+0000 7f1d90c00640 1 -- 172.21.15.46:0/2563161284 >> v1:172.21.15.46:6789/0 conn(0x7f1d8c0b1cf0 legacy=0x7f1d8c06ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.174+0000 7f1d90c00640 1 -- 172.21.15.46:0/2563161284 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c0cf080 con 0x7f1d8c15cbf0 2024-03-29T08:23:02.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.174+0000 7f1d91600640 1 --2- 172.21.15.46:0/2563161284 >> v2:172.21.15.73:3300/0 conn(0x7f1d8c06f140 0x7f1d8c0ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:23:02.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.178+0000 7f1d90c00640 1 --2- 172.21.15.46:0/2563161284 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1d8c15cbf0 0x7f1d8c0ae5b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0097c0 tx=0x7f1d7c01c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfc14cc0984829 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:02.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.178+0000 7f1d8be00640 1 -- 172.21.15.46:0/2563161284 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74001810 con 0x7f1d8c15cbf0 2024-03-29T08:23:02.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.178+0000 7f1d8be00640 1 -- 172.21.15.46:0/2563161284 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7c0a7050 con 0x7f1d8c15cbf0 2024-03-29T08:23:02.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.178+0000 7f1d8be00640 1 -- 172.21.15.46:0/2563161284 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7c0ab620 con 0x7f1d8c15cbf0 2024-03-29T08:23:02.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.178+0000 7f1d93600640 1 -- 172.21.15.46:0/2563161284 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1d8c15cbf0 msgr2=0x7f1d8c0ae5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.178+0000 7f1d93600640 1 --2- 172.21.15.46:0/2563161284 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1d8c15cbf0 0x7f1d8c0ae5b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0097c0 tx=0x7f1d7c01c3c0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.178+0000 7f1d93600640 1 -- 172.21.15.46:0/2563161284 shutdown_connections 2024-03-29T08:23:02.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.178+0000 7f1d93600640 1 --2- 172.21.15.46:0/2563161284 >> v2:172.21.15.73:3300/0 conn(0x7f1d8c06f140 0x7f1d8c0ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.178+0000 7f1d93600640 1 --2- 172.21.15.46:0/2563161284 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1d8c15cbf0 0x7f1d8c0ae5b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.178+0000 7f1d93600640 1 -- 172.21.15.46:0/2563161284 >> 172.21.15.46:0/2563161284 conn(0x7f1d8c098a40 msgr2=0x7f1d8c0b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:02.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.178+0000 7f1d93600640 1 -- 172.21.15.46:0/2563161284 shutdown_connections 2024-03-29T08:23:02.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.178+0000 7f1d93600640 1 -- 172.21.15.46:0/2563161284 wait complete. 2024-03-29T08:23:02.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.178+0000 7f1d93600640 1 Processor -- start 2024-03-29T08:23:02.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.178+0000 7f1d93600640 1 -- start start 2024-03-29T08:23:02.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d93600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d8c06f140 0x7f1d8c14a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d93600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d8c0b1cf0 0x7f1d8c157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d93600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1d8c15cbf0 0x7f1d8c153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d93600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c0ccc90 con 0x7f1d8c0b1cf0 2024-03-29T08:23:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d93600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c0c6f00 con 0x7f1d8c06f140 2024-03-29T08:23:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d93600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c0c71e0 con 0x7f1d8c15cbf0 2024-03-29T08:23:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d8c0b1cf0 0x7f1d8c157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d8c0b1cf0 0x7f1d8c157b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38980/0 (socket says 172.21.15.46:38980) 2024-03-29T08:23:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d83e00640 1 -- 172.21.15.46:0/3843593058 learned_addr learned my addr 172.21.15.46:0/3843593058 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d91600640 1 --2- 172.21.15.46:0/3843593058 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1d8c15cbf0 0x7f1d8c153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d90c00640 1 --2- 172.21.15.46:0/3843593058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d8c06f140 0x7f1d8c14a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d83e00640 1 -- 172.21.15.46:0/3843593058 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1d8c15cbf0 msgr2=0x7f1d8c153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d83e00640 1 --2- 172.21.15.46:0/3843593058 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1d8c15cbf0 0x7f1d8c153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d83e00640 1 -- 172.21.15.46:0/3843593058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d8c06f140 msgr2=0x7f1d8c14a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d83e00640 1 --2- 172.21.15.46:0/3843593058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d8c06f140 0x7f1d8c14a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d83e00640 1 -- 172.21.15.46:0/3843593058 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d84009a70 con 0x7f1d8c0b1cf0 2024-03-29T08:23:02.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d83e00640 1 --2- 172.21.15.46:0/3843593058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d8c0b1cf0 0x7f1d8c157b40 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400c780 tx=0x7f1d7400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:02.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d89600640 1 -- 172.21.15.46:0/3843593058 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74005ae0 con 0x7f1d8c0b1cf0 2024-03-29T08:23:02.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d89600640 1 -- 172.21.15.46:0/3843593058 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d74005c60 con 0x7f1d8c0b1cf0 2024-03-29T08:23:02.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d89600640 1 -- 172.21.15.46:0/3843593058 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74007070 con 0x7f1d8c0b1cf0 2024-03-29T08:23:02.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d93600640 1 -- 172.21.15.46:0/3843593058 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7c009420 con 0x7f1d8c0b1cf0 2024-03-29T08:23:02.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d93600640 1 -- 172.21.15.46:0/3843593058 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8c0c77a0 con 0x7f1d8c0b1cf0 2024-03-29T08:23:02.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d93600640 1 -- 172.21.15.46:0/3843593058 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d8c09d700 con 0x7f1d8c0b1cf0 2024-03-29T08:23:02.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.182+0000 7f1d89600640 1 -- 172.21.15.46:0/3843593058 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1d740071d0 con 0x7f1d8c0b1cf0 2024-03-29T08:23:02.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.186+0000 7f1d89600640 1 --2- 172.21.15.46:0/3843593058 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1d68066690 0x7f1d68068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.186+0000 7f1d89600640 1 -- 172.21.15.46:0/3843593058 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1d74083cc0 con 0x7f1d8c0b1cf0 2024-03-29T08:23:02.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.186+0000 7f1d90c00640 1 --2- 172.21.15.46:0/3843593058 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1d68066690 0x7f1d68068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.186+0000 7f1d90c00640 1 --2- 172.21.15.46:0/3843593058 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1d68066690 0x7f1d68068b50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0ad000 tx=0x7f1d7c0a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:02.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.190+0000 7f1d89600640 1 -- 172.21.15.46:0/3843593058 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1d74050ff0 con 0x7f1d8c0b1cf0 2024-03-29T08:23:02.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.386+0000 7f1d93600640 1 -- 172.21.15.46:0/3843593058 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1d8c0cf3f0 con 0x7f1d8c0b1cf0 2024-03-29T08:23:02.389 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-03-29T08:23:02.386+0000 7fc16ae00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:23:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.394+0000 7f1d89600640 1 -- 172.21.15.46:0/3843593058 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v43) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1d8c0cf3f0 con 0x7f1d8c0b1cf0 2024-03-29T08:23:02.396 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T08:23:02.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.394+0000 7f1d93600640 1 -- 172.21.15.46:0/3843593058 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1d68066690 msgr2=0x7f1d68068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.394+0000 7f1d93600640 1 --2- 172.21.15.46:0/3843593058 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1d68066690 0x7f1d68068b50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0ad000 tx=0x7f1d7c0a6000 comp rx=0 tx=0).stop 2024-03-29T08:23:02.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.394+0000 7f1d93600640 1 -- 172.21.15.46:0/3843593058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d8c0b1cf0 msgr2=0x7f1d8c157b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.394+0000 7f1d93600640 1 --2- 172.21.15.46:0/3843593058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d8c0b1cf0 0x7f1d8c157b40 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1d7400c780 tx=0x7f1d7400cb80 comp rx=0 tx=0).stop 2024-03-29T08:23:02.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.394+0000 7f1d93600640 1 -- 172.21.15.46:0/3843593058 shutdown_connections 2024-03-29T08:23:02.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.394+0000 7f1d93600640 1 --2- 172.21.15.46:0/3843593058 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1d68066690 0x7f1d68068b50 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.394+0000 7f1d93600640 1 --2- 172.21.15.46:0/3843593058 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1d8c15cbf0 0x7f1d8c153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.394+0000 7f1d93600640 1 --2- 172.21.15.46:0/3843593058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d8c0b1cf0 0x7f1d8c157b40 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.394+0000 7f1d93600640 1 --2- 172.21.15.46:0/3843593058 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d8c06f140 0x7f1d8c14a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.394+0000 7f1d93600640 1 -- 172.21.15.46:0/3843593058 >> 172.21.15.46:0/3843593058 conn(0x7f1d8c098a40 msgr2=0x7f1d8c155d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:02.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.394+0000 7f1d93600640 1 -- 172.21.15.46:0/3843593058 shutdown_connections 2024-03-29T08:23:02.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.394+0000 7f1d93600640 1 -- 172.21.15.46:0/3843593058 wait complete. 2024-03-29T08:23:02.407 INFO:tasks.ceph.mds.c.smithi046.stderr: -3488> 2024-03-29T08:22:45.130+0000 7fded1000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:23:02.428 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-03-29T08:23:02.464 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-03-29T08:23:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 Processor -- start 2024-03-29T08:23:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 -- start start 2024-03-29T08:23:02.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558f261c4440 0x558f2626f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f26250a30 0x558f2626d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f26250e10 0x558f2626bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f260df960 con 0x558f26250a30 2024-03-29T08:23:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f260df600 con 0x558f26250e10 2024-03-29T08:23:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f260d6690 con 0x558f261c4440 2024-03-29T08:23:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc089400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f26250a30 0x558f2626d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc089400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f26250a30 0x558f2626d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39016/0 (socket says 172.21.15.46:39016) 2024-03-29T08:23:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc089400640 1 -- 172.21.15.46:0/3420054053 learned_addr learned my addr 172.21.15.46:0/3420054053 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc089e00640 1 --2- 172.21.15.46:0/3420054053 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558f261c4440 0x558f2626f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08a800640 1 --2- 172.21.15.46:0/3420054053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f26250e10 0x558f2626bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc089400640 1 -- 172.21.15.46:0/3420054053 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558f261c4440 msgr2=0x558f2626f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc089400640 1 --2- 172.21.15.46:0/3420054053 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558f261c4440 0x558f2626f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc089400640 1 -- 172.21.15.46:0/3420054053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f26250e10 msgr2=0x558f2626bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc089400640 1 --2- 172.21.15.46:0/3420054053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f26250e10 0x558f2626bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc089400640 1 -- 172.21.15.46:0/3420054053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c005df0 con 0x558f26250a30 2024-03-29T08:23:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc089400640 1 --2- 172.21.15.46:0/3420054053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f26250a30 0x558f2626d7b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fc078001da0 tx=0x7fc07800b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d961da2beadcde server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc083e00640 1 -- 172.21.15.46:0/3420054053 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078014070 con 0x558f26250a30 2024-03-29T08:23:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc083e00640 1 -- 172.21.15.46:0/3420054053 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07800f460 con 0x558f26250a30 2024-03-29T08:23:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc083e00640 1 -- 172.21.15.46:0/3420054053 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078007040 con 0x558f26250a30 2024-03-29T08:23:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 -- 172.21.15.46:0/3420054053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f26250a30 msgr2=0x558f2626d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 --2- 172.21.15.46:0/3420054053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f26250a30 0x558f2626d7b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fc078001da0 tx=0x7fc07800b610 comp rx=0 tx=0).stop 2024-03-29T08:23:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 -- 172.21.15.46:0/3420054053 shutdown_connections 2024-03-29T08:23:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 --2- 172.21.15.46:0/3420054053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f26250e10 0x558f2626bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 --2- 172.21.15.46:0/3420054053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f26250a30 0x558f2626d7b0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 --2- 172.21.15.46:0/3420054053 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558f261c4440 0x558f2626f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 -- 172.21.15.46:0/3420054053 >> 172.21.15.46:0/3420054053 conn(0x558f261c2910 msgr2=0x558f2626df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 -- 172.21.15.46:0/3420054053 shutdown_connections 2024-03-29T08:23:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 -- 172.21.15.46:0/3420054053 wait complete. 2024-03-29T08:23:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 Processor -- start 2024-03-29T08:23:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.474+0000 7fc08cc69080 1 -- start start 2024-03-29T08:23:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc08cc69080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f261c4440 0x558f262501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc08cc69080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f26250a30 0x558f26268d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc08cc69080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558f26250e10 0x558f261c3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc08cc69080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f26269680 con 0x558f26250a30 2024-03-29T08:23:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc08cc69080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f26269920 con 0x558f261c4440 2024-03-29T08:23:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc08cc69080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f26269bc0 con 0x558f26250e10 2024-03-29T08:23:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc089400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f26250a30 0x558f26268d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc089400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f26250a30 0x558f26268d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39028/0 (socket says 172.21.15.46:39028) 2024-03-29T08:23:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc089400640 1 -- 172.21.15.46:0/1185001012 learned_addr learned my addr 172.21.15.46:0/1185001012 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc089e00640 1 --2- 172.21.15.46:0/1185001012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f261c4440 0x558f262501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc08a800640 1 --2- 172.21.15.46:0/1185001012 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558f26250e10 0x558f261c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc089400640 1 -- 172.21.15.46:0/1185001012 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558f26250e10 msgr2=0x558f261c3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc089400640 1 --2- 172.21.15.46:0/1185001012 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x558f26250e10 0x558f261c3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc089400640 1 -- 172.21.15.46:0/1185001012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f261c4440 msgr2=0x558f262501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc089400640 1 --2- 172.21.15.46:0/1185001012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558f261c4440 0x558f262501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc089400640 1 -- 172.21.15.46:0/1185001012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f260e1970 con 0x558f26250a30 2024-03-29T08:23:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc089400640 1 --2- 172.21.15.46:0/1185001012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f26250a30 0x558f26268d00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fc07800bb10 tx=0x7fc078000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc082a00640 1 -- 172.21.15.46:0/1185001012 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078014040 con 0x558f26250a30 2024-03-29T08:23:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc082a00640 1 -- 172.21.15.46:0/1185001012 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07800f5d0 con 0x558f26250a30 2024-03-29T08:23:02.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc08cc69080 1 -- 172.21.15.46:0/1185001012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558f260ee8c0 con 0x558f26250a30 2024-03-29T08:23:02.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc082a00640 1 -- 172.21.15.46:0/1185001012 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07800f9c0 con 0x558f26250a30 2024-03-29T08:23:02.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc08cc69080 1 -- 172.21.15.46:0/1185001012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f260e1d00 con 0x558f26250a30 2024-03-29T08:23:02.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.478+0000 7fc082a00640 1 -- 172.21.15.46:0/1185001012 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc07800fb20 con 0x558f26250a30 2024-03-29T08:23:02.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.482+0000 7fc082a00640 1 --2- 172.21.15.46:0/1185001012 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc06c065070 0x7fc06c067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.482+0000 7fc082a00640 1 -- 172.21.15.46:0/1185001012 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc078006070 con 0x558f26250a30 2024-03-29T08:23:02.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.482+0000 7fc08cc69080 1 --2- 172.21.15.46:0/1185001012 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x558f261dd5f0 0x558f261dd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.482+0000 7fc089e00640 1 --2- 172.21.15.46:0/1185001012 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc06c065070 0x7fc06c067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.482+0000 7fc08cc69080 1 -- 172.21.15.46:0/1185001012 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80) v8 -- 0x558f261ddf10 con 0x558f261dd5f0 2024-03-29T08:23:02.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.482+0000 7fc08a800640 1 --2- 172.21.15.46:0/1185001012 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x558f261dd5f0 0x558f261dd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.482+0000 7fc089e00640 1 --2- 172.21.15.46:0/1185001012 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc06c065070 0x7fc06c067530 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0ae000 tx=0x7fc07c0a6070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:02.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.482+0000 7fc08a800640 1 --2- 172.21.15.46:0/1185001012 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x558f261dd5f0 0x558f261dd9d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:02.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.486+0000 7fc08a800640 1 -- 172.21.15.46:0/1185001012 <== osd.1 v2:172.21.15.46:6824/27427 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v80'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fc08400da30 con 0x558f261dd5f0 2024-03-29T08:23:02.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.486+0000 7fc08cc69080 1 -- 172.21.15.46:0/1185001012 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x558f261dd5f0 msgr2=0x558f261dd9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.486+0000 7fc08cc69080 1 --2- 172.21.15.46:0/1185001012 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x558f261dd5f0 0x558f261dd9d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.486+0000 7fc08cc69080 1 -- 172.21.15.46:0/1185001012 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc06c065070 msgr2=0x7fc06c067530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.486+0000 7fc08cc69080 1 --2- 172.21.15.46:0/1185001012 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc06c065070 0x7fc06c067530 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc07c0ae000 tx=0x7fc07c0a6070 comp rx=0 tx=0).stop 2024-03-29T08:23:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.486+0000 7fc08cc69080 1 -- 172.21.15.46:0/1185001012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f26250a30 msgr2=0x558f26268d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.486+0000 7fc08cc69080 1 --2- 172.21.15.46:0/1185001012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f26250a30 0x558f26268d00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fc07800bb10 tx=0x7fc078000bd0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.486+0000 7fc08a800640 1 -- 172.21.15.46:0/1185001012 reap_dead start 2024-03-29T08:23:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.486+0000 7fc08cc69080 1 -- 172.21.15.46:0/1185001012 shutdown_connections 2024-03-29T08:23:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.486+0000 7fc08cc69080 1 -- 172.21.15.46:0/1185001012 >> 172.21.15.46:0/1185001012 conn(0x558f261c2910 msgr2=0x558f2626f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.486+0000 7fc08cc69080 1 -- 172.21.15.46:0/1185001012 shutdown_connections 2024-03-29T08:23:02.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.486+0000 7fc08cc69080 1 -- 172.21.15.46:0/1185001012 wait complete. 2024-03-29T08:23:02.494 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-29T08:23:02.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.678+0000 7f6fb0c00640 1 Processor -- start 2024-03-29T08:23:02.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.678+0000 7f6fb0c00640 1 -- start start 2024-03-29T08:23:02.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.678+0000 7f6fb0c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6fac142a50 0x7f6fac0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.678+0000 7f6fb0c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac15d260 con 0x7f6fac15df90 2024-03-29T08:23:02.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.678+0000 7f6fb0c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fac0aefd0 con 0x7f6fac0ae990 2024-03-29T08:23:02.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.678+0000 7f6fb0c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fac15d4b0 con 0x7f6fac142a50 2024-03-29T08:23:02.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.682+0000 7f6fa9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6fac142a50 0x7f6fac0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.682+0000 7f6fa3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6fac15df90 0x7f6fac0ae280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45460/0 (socket says 172.21.15.46:45460) 2024-03-29T08:23:02.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.682+0000 7f6fa3e00640 1 -- 172.21.15.46:0/3708649700 learned_addr learned my addr 172.21.15.46:0/3708649700 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:02.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.682+0000 7f6fa9e00640 1 -- 172.21.15.46:0/3708649700 >> v1:172.21.15.73:6789/0 conn(0x7f6fac0ae990 legacy=0x7f6fac0af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.682+0000 7f6fa9e00640 1 -- 172.21.15.46:0/3708649700 >> v1:172.21.15.46:6789/0 conn(0x7f6fac15df90 legacy=0x7f6fac0ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.682+0000 7f6fa9e00640 1 -- 172.21.15.46:0/3708649700 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fac15d790 con 0x7f6fac142a50 2024-03-29T08:23:02.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.682+0000 7f6fa9e00640 1 --2- 172.21.15.46:0/3708649700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fac142a50 0x7f6fac0cb790 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0097c0 tx=0x7f6f9c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce9aa84edcebf59 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:02.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.682+0000 7f6fa9400640 1 -- 172.21.15.46:0/3708649700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0a4ea0 con 0x7f6fac142a50 2024-03-29T08:23:02.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.682+0000 7f6fa9400640 1 -- 172.21.15.46:0/3708649700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c0a7050 con 0x7f6fac142a50 2024-03-29T08:23:02.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.682+0000 7f6fa9400640 1 -- 172.21.15.46:0/3708649700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0ab620 con 0x7f6fac142a50 2024-03-29T08:23:02.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fb0c00640 1 -- 172.21.15.46:0/3708649700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fac142a50 msgr2=0x7f6fac0cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fb0c00640 1 --2- 172.21.15.46:0/3708649700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fac142a50 0x7f6fac0cb790 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c0097c0 tx=0x7f6f9c01c410 comp rx=0 tx=0).stop 2024-03-29T08:23:02.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fb0c00640 1 -- 172.21.15.46:0/3708649700 shutdown_connections 2024-03-29T08:23:02.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fb0c00640 1 --2- 172.21.15.46:0/3708649700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fac142a50 0x7f6fac0cb790 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fb0c00640 1 -- 172.21.15.46:0/3708649700 >> 172.21.15.46:0/3708649700 conn(0x7f6fac06a390 msgr2=0x7f6fac0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:02.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fb0c00640 1 -- 172.21.15.46:0/3708649700 shutdown_connections 2024-03-29T08:23:02.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fb0c00640 1 -- 172.21.15.46:0/3708649700 wait complete. 2024-03-29T08:23:02.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fb0c00640 1 Processor -- start 2024-03-29T08:23:02.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fb0c00640 1 -- start start 2024-03-29T08:23:02.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fb0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fac0ae990 0x7f6fac1f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fb0c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fac142a50 0x7f6fac0ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fb0c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fac15df90 0x7f6fac1589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fb0c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac1f6560 con 0x7f6fac0ae990 2024-03-29T08:23:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fb0c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac13f430 con 0x7f6fac142a50 2024-03-29T08:23:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fb0c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fac13f710 con 0x7f6fac15df90 2024-03-29T08:23:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fa9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fac0ae990 0x7f6fac1f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fa9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fac0ae990 0x7f6fac1f6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39046/0 (socket says 172.21.15.46:39046) 2024-03-29T08:23:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fa9e00640 1 -- 172.21.15.46:0/831780908 learned_addr learned my addr 172.21.15.46:0/831780908 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fa3e00640 1 --2- 172.21.15.46:0/831780908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fac142a50 0x7f6fac0ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6faa800640 1 --2- 172.21.15.46:0/831780908 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fac15df90 0x7f6fac1589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fa9e00640 1 -- 172.21.15.46:0/831780908 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fac15df90 msgr2=0x7f6fac1589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fa9e00640 1 --2- 172.21.15.46:0/831780908 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fac15df90 0x7f6fac1589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fa9e00640 1 -- 172.21.15.46:0/831780908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fac142a50 msgr2=0x7f6fac0ccf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fa9e00640 1 --2- 172.21.15.46:0/831780908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fac142a50 0x7f6fac0ccf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fa9e00640 1 -- 172.21.15.46:0/831780908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c009420 con 0x7f6fac0ae990 2024-03-29T08:23:02.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fa3e00640 1 --2- 172.21.15.46:0/831780908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fac142a50 0x7f6fac0ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:02.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6faa800640 1 --2- 172.21.15.46:0/831780908 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fac15df90 0x7f6fac1589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:02.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.686+0000 7f6fa9e00640 1 --2- 172.21.15.46:0/831780908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fac0ae990 0x7f6fac1f6020 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c019020 tx=0x7f6f9c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:02.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.690+0000 7f6fa2000640 1 -- 172.21.15.46:0/831780908 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0b4820 con 0x7f6fac0ae990 2024-03-29T08:23:02.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.690+0000 7f6fa2000640 1 -- 172.21.15.46:0/831780908 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c0abe90 con 0x7f6fac0ae990 2024-03-29T08:23:02.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.690+0000 7f6fb0c00640 1 -- 172.21.15.46:0/831780908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fac13f9f0 con 0x7f6fac0ae990 2024-03-29T08:23:02.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.690+0000 7f6fa2000640 1 -- 172.21.15.46:0/831780908 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0b4be0 con 0x7f6fac0ae990 2024-03-29T08:23:02.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.690+0000 7f6fb0c00640 1 -- 172.21.15.46:0/831780908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fac13ffb0 con 0x7f6fac0ae990 2024-03-29T08:23:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.690+0000 7f6fb0c00640 1 -- 172.21.15.46:0/831780908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fac0733a0 con 0x7f6fac0ae990 2024-03-29T08:23:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.690+0000 7f6fa2000640 1 -- 172.21.15.46:0/831780908 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6f9c0b2040 con 0x7f6fac0ae990 2024-03-29T08:23:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.690+0000 7f6fa2000640 1 --2- 172.21.15.46:0/831780908 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f88066690 0x7f6f88068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.690+0000 7f6fa2000640 1 -- 172.21.15.46:0/831780908 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6f9c11e0b0 con 0x7f6fac0ae990 2024-03-29T08:23:02.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.690+0000 7f6fa3e00640 1 --2- 172.21.15.46:0/831780908 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f88066690 0x7f6f88068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:02.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.694+0000 7f6fa3e00640 1 --2- 172.21.15.46:0/831780908 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f88066690 0x7f6f88068b50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6f94007fd0 tx=0x7f6f94009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:02.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.694+0000 7f6fa2000640 1 -- 172.21.15.46:0/831780908 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6f9c0eb2f0 con 0x7f6fac0ae990 2024-03-29T08:23:02.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.890+0000 7f6fb0c00640 1 -- 172.21.15.46:0/831780908 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f6fac067570 con 0x7f6fac0ae990 2024-03-29T08:23:02.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.946+0000 7f6fa2000640 1 -- 172.21.15.46:0/831780908 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v44) v1 ==== 162+0+0 (secure 0 0 0) 0x7f6f9c006e20 con 0x7f6fac0ae990 2024-03-29T08:23:02.949 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-29T08:23:02.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.946+0000 7f6fb0c00640 1 -- 172.21.15.46:0/831780908 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f88066690 msgr2=0x7f6f88068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.946+0000 7f6fb0c00640 1 --2- 172.21.15.46:0/831780908 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f88066690 0x7f6f88068b50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6f94007fd0 tx=0x7f6f94009040 comp rx=0 tx=0).stop 2024-03-29T08:23:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.946+0000 7f6fb0c00640 1 -- 172.21.15.46:0/831780908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fac0ae990 msgr2=0x7f6fac1f6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.946+0000 7f6fb0c00640 1 --2- 172.21.15.46:0/831780908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fac0ae990 0x7f6fac1f6020 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c019020 tx=0x7f6f9c0191b0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.946+0000 7f6fb0c00640 1 -- 172.21.15.46:0/831780908 shutdown_connections 2024-03-29T08:23:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.946+0000 7f6fb0c00640 1 --2- 172.21.15.46:0/831780908 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f88066690 0x7f6f88068b50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.946+0000 7f6fb0c00640 1 --2- 172.21.15.46:0/831780908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fac142a50 0x7f6fac0ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.946+0000 7f6fb0c00640 1 --2- 172.21.15.46:0/831780908 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fac15df90 0x7f6fac1589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.946+0000 7f6fb0c00640 1 --2- 172.21.15.46:0/831780908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fac0ae990 0x7f6fac1f6020 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.946+0000 7f6fb0c00640 1 -- 172.21.15.46:0/831780908 >> 172.21.15.46:0/831780908 conn(0x7f6fac06a390 msgr2=0x7f6fac13dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.946+0000 7f6fb0c00640 1 -- 172.21.15.46:0/831780908 shutdown_connections 2024-03-29T08:23:02.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:02.946+0000 7f6fb0c00640 1 -- 172.21.15.46:0/831780908 wait complete. 2024-03-29T08:23:02.974 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:23:03.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.114+0000 7fe9fa600640 1 Processor -- start 2024-03-29T08:23:03.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.114+0000 7fe9fa600640 1 -- start start 2024-03-29T08:23:03.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.114+0000 7fe9fa600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe9f415a940 0x7fe9f40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:03.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.114+0000 7fe9fa600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe9f40cf3c0 0x7fe9f40cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:03.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.114+0000 7fe9fa600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f40b00d0 con 0x7fe9f415ad20 2024-03-29T08:23:03.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.114+0000 7fe9fa600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f40b0ab0 con 0x7fe9f415a940 2024-03-29T08:23:03.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.114+0000 7fe9fa600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f40b0d90 con 0x7fe9f40cf3c0 2024-03-29T08:23:03.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.114+0000 7fe9f3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe9f415a940 0x7fe9f40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:03.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.114+0000 7fe9eaa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe9f415ad20 0x7fe9f40cecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45476/0 (socket says 172.21.15.46:45476) 2024-03-29T08:23:03.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.114+0000 7fe9eaa00640 1 -- 172.21.15.46:0/1023024675 learned_addr learned my addr 172.21.15.46:0/1023024675 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:03.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.114+0000 7fe9f3e00640 1 --2- 172.21.15.46:0/1023024675 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe9f40cf3c0 0x7fe9f40cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:03.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.114+0000 7fe9f3400640 1 -- 172.21.15.46:0/1023024675 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe9f40cf3c0 msgr2=0x7fe9f40cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:03.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.114+0000 7fe9f3400640 1 --2- 172.21.15.46:0/1023024675 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe9f40cf3c0 0x7fe9f40cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:03.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.114+0000 7fe9f3400640 1 -- 172.21.15.46:0/1023024675 >> v1:172.21.15.46:6789/0 conn(0x7fe9f415ad20 legacy=0x7fe9f40cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:03.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.114+0000 7fe9f3400640 1 -- 172.21.15.46:0/1023024675 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f40b1070 con 0x7fe9f415a940 2024-03-29T08:23:03.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.114+0000 7fe9f2a00640 1 -- 172.21.15.46:0/1023024675 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2328953732 0 0) 0x7fe9f40b00d0 con 0x7fe9f415ad20 2024-03-29T08:23:03.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.114+0000 7fe9f3e00640 1 --2- 172.21.15.46:0/1023024675 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe9f40cf3c0 0x7fe9f40cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:23:03.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.118+0000 7fe9f3400640 1 --2- 172.21.15.46:0/1023024675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f415a940 0x7fe9f40afb90 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40097c0 tx=0x7fe9e401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4c8317e9349443 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:03.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.118+0000 7fe9f2a00640 1 -- 172.21.15.46:0/1023024675 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40ac640 con 0x7fe9f415a940 2024-03-29T08:23:03.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.118+0000 7fe9f2a00640 1 -- 172.21.15.46:0/1023024675 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e40ac7a0 con 0x7fe9f415a940 2024-03-29T08:23:03.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.118+0000 7fe9f2a00640 1 -- 172.21.15.46:0/1023024675 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e40acb30 con 0x7fe9f415a940 2024-03-29T08:23:03.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.118+0000 7fe9fa600640 1 -- 172.21.15.46:0/1023024675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f415a940 msgr2=0x7fe9f40afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:03.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.118+0000 7fe9fa600640 1 --2- 172.21.15.46:0/1023024675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f415a940 0x7fe9f40afb90 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40097c0 tx=0x7fe9e401c3e0 comp rx=0 tx=0).stop 2024-03-29T08:23:03.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.118+0000 7fe9fa600640 1 -- 172.21.15.46:0/1023024675 shutdown_connections 2024-03-29T08:23:03.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.118+0000 7fe9fa600640 1 --2- 172.21.15.46:0/1023024675 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe9f40cf3c0 0x7fe9f40cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:03.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.118+0000 7fe9fa600640 1 --2- 172.21.15.46:0/1023024675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f415a940 0x7fe9f40afb90 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.118+0000 7fe9fa600640 1 -- 172.21.15.46:0/1023024675 >> 172.21.15.46:0/1023024675 conn(0x7fe9f4089fd0 msgr2=0x7fe9f408a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.118+0000 7fe9fa600640 1 -- 172.21.15.46:0/1023024675 shutdown_connections 2024-03-29T08:23:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.118+0000 7fe9fa600640 1 -- 172.21.15.46:0/1023024675 wait complete. 2024-03-29T08:23:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9fa600640 1 Processor -- start 2024-03-29T08:23:03.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9fa600640 1 -- start start 2024-03-29T08:23:03.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9fa600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe9f40cf3c0 0x7fe9f406b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9fa600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9f415a940 0x7fe9f41534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9fa600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f415ad20 0x7fe9f415d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9fa600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f406be90 con 0x7fe9f415ad20 2024-03-29T08:23:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9fa600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f406c170 con 0x7fe9f415a940 2024-03-29T08:23:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9fa600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f406c450 con 0x7fe9f40cf3c0 2024-03-29T08:23:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9f3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f415ad20 0x7fe9f415d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9f3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f415ad20 0x7fe9f415d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39062/0 (socket says 172.21.15.46:39062) 2024-03-29T08:23:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9f3e00640 1 -- 172.21.15.46:0/3410997842 learned_addr learned my addr 172.21.15.46:0/3410997842 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9f3400640 1 --2- 172.21.15.46:0/3410997842 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe9f40cf3c0 0x7fe9f406b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9eaa00640 1 --2- 172.21.15.46:0/3410997842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9f415a940 0x7fe9f41534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9f3e00640 1 -- 172.21.15.46:0/3410997842 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe9f40cf3c0 msgr2=0x7fe9f406b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9f3e00640 1 --2- 172.21.15.46:0/3410997842 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe9f40cf3c0 0x7fe9f406b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:03.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9f3e00640 1 -- 172.21.15.46:0/3410997842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9f415a940 msgr2=0x7fe9f41534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9f3e00640 1 --2- 172.21.15.46:0/3410997842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9f415a940 0x7fe9f41534e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9f3e00640 1 -- 172.21.15.46:0/3410997842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e4009420 con 0x7fe9f415ad20 2024-03-29T08:23:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9f3e00640 1 --2- 172.21.15.46:0/3410997842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f415ad20 0x7fe9f415d2b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00c7f0 tx=0x7fe9ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9ebe00640 1 -- 172.21.15.46:0/3410997842 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec004220 con 0x7fe9f415ad20 2024-03-29T08:23:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9ebe00640 1 -- 172.21.15.46:0/3410997842 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ec0043a0 con 0x7fe9f415ad20 2024-03-29T08:23:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9ebe00640 1 -- 172.21.15.46:0/3410997842 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec003860 con 0x7fe9f415ad20 2024-03-29T08:23:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9fa600640 1 -- 172.21.15.46:0/3410997842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f406c730 con 0x7fe9f415ad20 2024-03-29T08:23:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9fa600640 1 -- 172.21.15.46:0/3410997842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f4153fb0 con 0x7fe9f415ad20 2024-03-29T08:23:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.122+0000 7fe9fa600640 1 -- 172.21.15.46:0/3410997842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f4066c90 con 0x7fe9f415ad20 2024-03-29T08:23:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.126+0000 7fe9ebe00640 1 -- 172.21.15.46:0/3410997842 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe9ec003bd0 con 0x7fe9f415ad20 2024-03-29T08:23:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.126+0000 7fe9ebe00640 1 --2- 172.21.15.46:0/3410997842 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe9d0066690 0x7fe9d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.126+0000 7fe9ebe00640 1 -- 172.21.15.46:0/3410997842 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe9ec0842d0 con 0x7fe9f415ad20 2024-03-29T08:23:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.126+0000 7fe9f3400640 1 --2- 172.21.15.46:0/3410997842 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe9d0066690 0x7fe9d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.126+0000 7fe9f3400640 1 --2- 172.21.15.46:0/3410997842 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe9d0066690 0x7fe9d0068b50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40ae000 tx=0x7fe9e40a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:03.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.130+0000 7fe9ebe00640 1 -- 172.21.15.46:0/3410997842 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe9ec051510 con 0x7fe9f415ad20 2024-03-29T08:23:03.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.326+0000 7fe9fa600640 1 -- 172.21.15.46:0/3410997842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9f406cbe0 con 0x7fe9f415ad20 2024-03-29T08:23:03.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.326+0000 7fe9ebe00640 1 -- 172.21.15.46:0/3410997842 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4737 (secure 0 0 0) 0x7fe9ec003e80 con 0x7fe9f415ad20 2024-03-29T08:23:03.328 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:23:03.329 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3456771376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3456771376},{"type":"v1","addr":"172.21.15.73:6836","nonce":3456771376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2671538025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2671538025},{"type":"v1","addr":"172.21.15.73:6837","nonce":2671538025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3196015693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3196015693},{"type":"v1","addr":"172.21.15.46:6833","nonce":3196015693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:22:43.605457+0000","modified":"2024-03-29T08:23:02.946899+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":44,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6835/1672348161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1672348161},{"type":"v1","addr":"172.21.15.46:6835","nonce":1672348161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-29T08:23:03.329 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 44 2024-03-29T08:23:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.326+0000 7fe9fa600640 1 -- 172.21.15.46:0/3410997842 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe9d0066690 msgr2=0x7fe9d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.326+0000 7fe9fa600640 1 --2- 172.21.15.46:0/3410997842 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe9d0066690 0x7fe9d0068b50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe9e40ae000 tx=0x7fe9e40a7000 comp rx=0 tx=0).stop 2024-03-29T08:23:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.326+0000 7fe9fa600640 1 -- 172.21.15.46:0/3410997842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f415ad20 msgr2=0x7fe9f415d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.326+0000 7fe9fa600640 1 --2- 172.21.15.46:0/3410997842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f415ad20 0x7fe9f415d2b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00c7f0 tx=0x7fe9ec00ccb0 comp rx=0 tx=0).stop 2024-03-29T08:23:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.326+0000 7fe9fa600640 1 -- 172.21.15.46:0/3410997842 shutdown_connections 2024-03-29T08:23:03.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.326+0000 7fe9fa600640 1 --2- 172.21.15.46:0/3410997842 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe9d0066690 0x7fe9d0068b50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.326+0000 7fe9fa600640 1 --2- 172.21.15.46:0/3410997842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe9f415a940 0x7fe9f41534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.326+0000 7fe9fa600640 1 --2- 172.21.15.46:0/3410997842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9f415ad20 0x7fe9f415d2b0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.326+0000 7fe9fa600640 1 --2- 172.21.15.46:0/3410997842 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe9f40cf3c0 0x7fe9f406b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.326+0000 7fe9fa600640 1 -- 172.21.15.46:0/3410997842 >> 172.21.15.46:0/3410997842 conn(0x7fe9f4089fd0 msgr2=0x7fe9f408a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.326+0000 7fe9fa600640 1 -- 172.21.15.46:0/3410997842 shutdown_connections 2024-03-29T08:23:03.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:03.326+0000 7fe9fa600640 1 -- 172.21.15.46:0/3410997842 wait complete. 2024-03-29T08:23:03.752 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:22:43.605457+0000', 'modified': '2024-03-29T08:23:02.946899+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6876}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6876': {'gid': 6876, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.46:6835/1672348161', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 1672348161}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1672348161}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:23:03.752 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6876:up:replay 2024-03-29T08:23:04.753 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:23:04.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.902+0000 7f592fe00640 1 Processor -- start 2024-03-29T08:23:04.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.902+0000 7f592fe00640 1 -- start start 2024-03-29T08:23:04.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.902+0000 7f592fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59281593d0 0x7f592806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.902+0000 7f592fe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f592815d230 0x7f592815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.902+0000 7f592fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f592815dbd0 0x7f592815bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.902+0000 7f592fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592806d130 con 0x7f592815dbd0 2024-03-29T08:23:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.902+0000 7f592fe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592815b8f0 con 0x7f592815d230 2024-03-29T08:23:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.902+0000 7f592fe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592815ad50 con 0x7f59281593d0 2024-03-29T08:23:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.902+0000 7f592de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f592815dbd0 0x7f592815bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.902+0000 7f592de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f592815dbd0 0x7f592815bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39064/0 (socket says 172.21.15.46:39064) 2024-03-29T08:23:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.902+0000 7f592de00640 1 -- 172.21.15.46:0/651381309 learned_addr learned my addr 172.21.15.46:0/651381309 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.906+0000 7f592d400640 1 --2- 172.21.15.46:0/651381309 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59281593d0 0x7f592806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.906+0000 7f592ca00640 1 --2- 172.21.15.46:0/651381309 >> v2:172.21.15.73:3300/0 conn(0x7f592815d230 0x7f592815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.906+0000 7f592de00640 1 -- 172.21.15.46:0/651381309 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59281593d0 msgr2=0x7f592806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.906+0000 7f592de00640 1 --2- 172.21.15.46:0/651381309 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59281593d0 0x7f592806cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.906+0000 7f592de00640 1 -- 172.21.15.46:0/651381309 >> v2:172.21.15.73:3300/0 conn(0x7f592815d230 msgr2=0x7f592815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:04.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.906+0000 7f592de00640 1 --2- 172.21.15.46:0/651381309 >> v2:172.21.15.73:3300/0 conn(0x7f592815d230 0x7f592815d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:04.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.906+0000 7f592de00640 1 -- 172.21.15.46:0/651381309 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592815e060 con 0x7f592815dbd0 2024-03-29T08:23:04.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.906+0000 7f592de00640 1 --2- 172.21.15.46:0/651381309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592815dbd0 0x7f592815bd80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5928071520 tx=0x7f592001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65d6a35c45dc1fc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:04.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.906+0000 7f5927e00640 1 -- 172.21.15.46:0/651381309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200a5870 con 0x7f592815dbd0 2024-03-29T08:23:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.906+0000 7f5927e00640 1 -- 172.21.15.46:0/651381309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59200a59d0 con 0x7f592815dbd0 2024-03-29T08:23:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.906+0000 7f5927e00640 1 -- 172.21.15.46:0/651381309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59200a5d60 con 0x7f592815dbd0 2024-03-29T08:23:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.906+0000 7f592fe00640 1 -- 172.21.15.46:0/651381309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592815dbd0 msgr2=0x7f592815bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:04.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.906+0000 7f592fe00640 1 --2- 172.21.15.46:0/651381309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592815dbd0 0x7f592815bd80 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f5928071520 tx=0x7f592001c9f0 comp rx=0 tx=0).stop 2024-03-29T08:23:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592fe00640 1 -- 172.21.15.46:0/651381309 shutdown_connections 2024-03-29T08:23:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592fe00640 1 --2- 172.21.15.46:0/651381309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f592815dbd0 0x7f592815bd80 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592fe00640 1 --2- 172.21.15.46:0/651381309 >> v2:172.21.15.73:3300/0 conn(0x7f592815d230 0x7f592815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592fe00640 1 --2- 172.21.15.46:0/651381309 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59281593d0 0x7f592806cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592fe00640 1 -- 172.21.15.46:0/651381309 >> 172.21.15.46:0/651381309 conn(0x7f5928066110 msgr2=0x7f5928069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592fe00640 1 -- 172.21.15.46:0/651381309 shutdown_connections 2024-03-29T08:23:04.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592fe00640 1 -- 172.21.15.46:0/651381309 wait complete. 2024-03-29T08:23:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592fe00640 1 Processor -- start 2024-03-29T08:23:04.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592fe00640 1 -- start start 2024-03-29T08:23:04.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59281593d0 0x7f5928163190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f592815d230 0x7f5928156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592815dbd0 0x7f5928157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5928158040 con 0x7f59281593d0 2024-03-29T08:23:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59281577a0 con 0x7f592815dbd0 2024-03-29T08:23:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592fe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5928157a80 con 0x7f592815d230 2024-03-29T08:23:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59281593d0 0x7f5928163190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59281593d0 0x7f5928163190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39072/0 (socket says 172.21.15.46:39072) 2024-03-29T08:23:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592d400640 1 -- 172.21.15.46:0/2845706964 learned_addr learned my addr 172.21.15.46:0/2845706964 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592ca00640 1 --2- 172.21.15.46:0/2845706964 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f592815d230 0x7f5928156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592de00640 1 --2- 172.21.15.46:0/2845706964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592815dbd0 0x7f5928157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592d400640 1 -- 172.21.15.46:0/2845706964 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f592815d230 msgr2=0x7f5928156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:04.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592d400640 1 --2- 172.21.15.46:0/2845706964 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f592815d230 0x7f5928156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592d400640 1 -- 172.21.15.46:0/2845706964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592815dbd0 msgr2=0x7f5928157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592d400640 1 --2- 172.21.15.46:0/2845706964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592815dbd0 0x7f5928157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592d400640 1 -- 172.21.15.46:0/2845706964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5920009b30 con 0x7f59281593d0 2024-03-29T08:23:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592de00640 1 --2- 172.21.15.46:0/2845706964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592815dbd0 0x7f5928157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592ca00640 1 --2- 172.21.15.46:0/2845706964 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f592815d230 0x7f5928156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.910+0000 7f592d400640 1 --2- 172.21.15.46:0/2845706964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59281593d0 0x7f5928163190 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f591c00d800 tx=0x7f591c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.914+0000 7f5925600640 1 -- 172.21.15.46:0/2845706964 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c0096a0 con 0x7f59281593d0 2024-03-29T08:23:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.914+0000 7f5925600640 1 -- 172.21.15.46:0/2845706964 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591c009800 con 0x7f59281593d0 2024-03-29T08:23:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.914+0000 7f592fe00640 1 -- 172.21.15.46:0/2845706964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5928157d60 con 0x7f59281593d0 2024-03-29T08:23:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.914+0000 7f5925600640 1 -- 172.21.15.46:0/2845706964 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c009990 con 0x7f59281593d0 2024-03-29T08:23:04.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.914+0000 7f592fe00640 1 -- 172.21.15.46:0/2845706964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59281f5360 con 0x7f59281593d0 2024-03-29T08:23:04.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.914+0000 7f592fe00640 1 -- 172.21.15.46:0/2845706964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59280cc2e0 con 0x7f59281593d0 2024-03-29T08:23:04.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.914+0000 7f5925600640 1 -- 172.21.15.46:0/2845706964 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f591c009af0 con 0x7f59281593d0 2024-03-29T08:23:04.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.914+0000 7f5925600640 1 --2- 172.21.15.46:0/2845706964 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5908066690 0x7f5908068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:04.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.914+0000 7f5925600640 1 -- 172.21.15.46:0/2845706964 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f591c084750 con 0x7f59281593d0 2024-03-29T08:23:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.914+0000 7f592ca00640 1 --2- 172.21.15.46:0/2845706964 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5908066690 0x7f5908068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:04.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.918+0000 7f592ca00640 1 --2- 172.21.15.46:0/2845706964 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5908066690 0x7f5908068b50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f59281f9710 tx=0x7f5914008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:04.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:04.918+0000 7f5925600640 1 -- 172.21.15.46:0/2845706964 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f59280cc2e0 con 0x7f59281593d0 2024-03-29T08:23:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:05.114+0000 7f592fe00640 1 -- 172.21.15.46:0/2845706964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59280cc2e0 con 0x7f59281593d0 2024-03-29T08:23:05.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:05.114+0000 7f5925600640 1 -- 172.21.15.46:0/2845706964 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f59280cc2e0 con 0x7f59281593d0 2024-03-29T08:23:05.119 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:23:05.119 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3456771376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3456771376},{"type":"v1","addr":"172.21.15.73:6836","nonce":3456771376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2671538025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2671538025},{"type":"v1","addr":"172.21.15.73:6837","nonce":2671538025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3196015693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3196015693},{"type":"v1","addr":"172.21.15.46:6833","nonce":3196015693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:22:43.605457+0000","modified":"2024-03-29T08:23:04.956481+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":44,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.46:6835/1672348161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1672348161},{"type":"v1","addr":"172.21.15.46:6835","nonce":1672348161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-29T08:23:05.119 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 46 2024-03-29T08:23:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:05.118+0000 7f592fe00640 1 -- 172.21.15.46:0/2845706964 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5908066690 msgr2=0x7f5908068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:05.118+0000 7f592fe00640 1 --2- 172.21.15.46:0/2845706964 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5908066690 0x7f5908068b50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f59281f9710 tx=0x7f5914008040 comp rx=0 tx=0).stop 2024-03-29T08:23:05.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:05.118+0000 7f592fe00640 1 -- 172.21.15.46:0/2845706964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59281593d0 msgr2=0x7f5928163190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:05.118+0000 7f592fe00640 1 --2- 172.21.15.46:0/2845706964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59281593d0 0x7f5928163190 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f591c00d800 tx=0x7f591c00dcc0 comp rx=0 tx=0).stop 2024-03-29T08:23:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:05.118+0000 7f592fe00640 1 -- 172.21.15.46:0/2845706964 shutdown_connections 2024-03-29T08:23:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:05.118+0000 7f592fe00640 1 --2- 172.21.15.46:0/2845706964 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5908066690 0x7f5908068b50 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:05.118+0000 7f592fe00640 1 --2- 172.21.15.46:0/2845706964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592815dbd0 0x7f5928157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:05.118+0000 7f592fe00640 1 --2- 172.21.15.46:0/2845706964 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f592815d230 0x7f5928156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:05.118+0000 7f592fe00640 1 --2- 172.21.15.46:0/2845706964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59281593d0 0x7f5928163190 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:05.118+0000 7f592fe00640 1 -- 172.21.15.46:0/2845706964 >> 172.21.15.46:0/2845706964 conn(0x7f5928066110 msgr2=0x7f59280cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:05.118+0000 7f592fe00640 1 -- 172.21.15.46:0/2845706964 shutdown_connections 2024-03-29T08:23:05.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:05.118+0000 7f592fe00640 1 -- 172.21.15.46:0/2845706964 wait complete. 2024-03-29T08:23:05.542 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:22:43.605457+0000', 'modified': '2024-03-29T08:23:04.956481+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6876}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6876': {'gid': 6876, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.46:6835/1672348161', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 1672348161}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1672348161}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:23:05.543 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6876:up:rejoin 2024-03-29T08:23:06.543 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:23:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.694+0000 7f3609c00640 1 Processor -- start 2024-03-29T08:23:06.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.694+0000 7f3609c00640 1 -- start start 2024-03-29T08:23:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.694+0000 7f3609c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f36040d0660 0x7f36040d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.694+0000 7f3609c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360415cfb0 con 0x7f360415dc00 2024-03-29T08:23:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.694+0000 7f3609c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36040d0f80 con 0x7f360406b8b0 2024-03-29T08:23:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.694+0000 7f3609c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360406c100 con 0x7f36040d0660 2024-03-29T08:23:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.694+0000 7f3603400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f360415dc00 0x7f36040cf7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34030/0 (socket says 172.21.15.46:34030) 2024-03-29T08:23:06.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.694+0000 7f3603400640 1 -- 172.21.15.46:0/594409607 learned_addr learned my addr 172.21.15.46:0/594409607 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:06.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.694+0000 7f3602a00640 1 --2- 172.21.15.46:0/594409607 >> v2:172.21.15.73:3300/0 conn(0x7f36040d0660 0x7f36040d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:06.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.694+0000 7f3602000640 1 -- 172.21.15.46:0/594409607 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198169525 0 0) 0x7f360415cfb0 con 0x7f360415dc00 2024-03-29T08:23:06.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.694+0000 7f3602000640 1 -- 172.21.15.46:0/594409607 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35e8003530 con 0x7f360415dc00 2024-03-29T08:23:06.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.694+0000 7f3602a00640 1 -- 172.21.15.46:0/594409607 >> v1:172.21.15.73:6789/0 conn(0x7f360406b8b0 legacy=0x7f36040cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:06.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.694+0000 7f3602000640 1 -- 172.21.15.46:0/594409607 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892961999 0 0) 0x7f36040d0f80 con 0x7f360406b8b0 2024-03-29T08:23:06.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.694+0000 7f3602a00640 1 -- 172.21.15.46:0/594409607 >> v1:172.21.15.46:6789/0 conn(0x7f360415dc00 legacy=0x7f36040cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:06.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.694+0000 7f3602a00640 1 -- 172.21.15.46:0/594409607 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360406be20 con 0x7f36040d0660 2024-03-29T08:23:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.698+0000 7f3602a00640 1 --2- 172.21.15.46:0/594409607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36040d0660 0x7f36040d0a40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f35f40097c0 tx=0x7f35f401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5651410107c1ac6d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.698+0000 7f3602000640 1 -- 172.21.15.46:0/594409607 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f40ab670 con 0x7f36040d0660 2024-03-29T08:23:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.698+0000 7f3602000640 1 -- 172.21.15.46:0/594409607 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f40ab7d0 con 0x7f36040d0660 2024-03-29T08:23:06.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.698+0000 7f3602000640 1 -- 172.21.15.46:0/594409607 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f40abb60 con 0x7f36040d0660 2024-03-29T08:23:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.698+0000 7f3609c00640 1 -- 172.21.15.46:0/594409607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36040d0660 msgr2=0x7f36040d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.698+0000 7f3609c00640 1 --2- 172.21.15.46:0/594409607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36040d0660 0x7f36040d0a40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f35f40097c0 tx=0x7f35f401c470 comp rx=0 tx=0).stop 2024-03-29T08:23:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.698+0000 7f3609c00640 1 -- 172.21.15.46:0/594409607 shutdown_connections 2024-03-29T08:23:06.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.698+0000 7f3609c00640 1 --2- 172.21.15.46:0/594409607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36040d0660 0x7f36040d0a40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.698+0000 7f3609c00640 1 -- 172.21.15.46:0/594409607 >> 172.21.15.46:0/594409607 conn(0x7f3604089fd0 msgr2=0x7f360408a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.698+0000 7f3609c00640 1 -- 172.21.15.46:0/594409607 shutdown_connections 2024-03-29T08:23:06.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.698+0000 7f3609c00640 1 -- 172.21.15.46:0/594409607 wait complete. 2024-03-29T08:23:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3609c00640 1 Processor -- start 2024-03-29T08:23:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3609c00640 1 -- start start 2024-03-29T08:23:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3609c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f360406b8b0 0x7f3604145c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3609c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36040d0660 0x7f36040b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:06.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3609c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f360415dc00 0x7f3604157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:06.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3609c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3604155920 con 0x7f360415dc00 2024-03-29T08:23:06.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3609c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3604146180 con 0x7f36040d0660 2024-03-29T08:23:06.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3609c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36040b5a20 con 0x7f360406b8b0 2024-03-29T08:23:06.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3603400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f360415dc00 0x7f3604157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:06.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3602a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f360406b8b0 0x7f3604145c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:06.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3602a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f360406b8b0 0x7f3604145c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:43702/0 (socket says 172.21.15.46:43702) 2024-03-29T08:23:06.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3602a00640 1 -- 172.21.15.46:0/1793216435 learned_addr learned my addr 172.21.15.46:0/1793216435 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:06.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f35fbe00640 1 --2- 172.21.15.46:0/1793216435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36040d0660 0x7f36040b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3602a00640 1 -- 172.21.15.46:0/1793216435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36040d0660 msgr2=0x7f36040b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3602a00640 1 --2- 172.21.15.46:0/1793216435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36040d0660 0x7f36040b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3602a00640 1 -- 172.21.15.46:0/1793216435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f360415dc00 msgr2=0x7f3604157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3602a00640 1 --2- 172.21.15.46:0/1793216435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f360415dc00 0x7f3604157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3602a00640 1 -- 172.21.15.46:0/1793216435 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f4009420 con 0x7f360406b8b0 2024-03-29T08:23:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3602a00640 1 --2- 172.21.15.46:0/1793216435 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f360406b8b0 0x7f3604145c40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f35f40097c0 tx=0x7f35f401acb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f35faa00640 1 -- 172.21.15.46:0/1793216435 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f40bf610 con 0x7f360406b8b0 2024-03-29T08:23:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f35faa00640 1 -- 172.21.15.46:0/1793216435 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f40bf770 con 0x7f360406b8b0 2024-03-29T08:23:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3609c00640 1 -- 172.21.15.46:0/1793216435 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3604154d10 con 0x7f360406b8b0 2024-03-29T08:23:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f35faa00640 1 -- 172.21.15.46:0/1793216435 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f40bfb00 con 0x7f360406b8b0 2024-03-29T08:23:06.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.702+0000 7f3609c00640 1 -- 172.21.15.46:0/1793216435 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36041552d0 con 0x7f360406b8b0 2024-03-29T08:23:06.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.706+0000 7f3609c00640 1 -- 172.21.15.46:0/1793216435 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3604066c90 con 0x7f360406b8b0 2024-03-29T08:23:06.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.706+0000 7f35faa00640 1 -- 172.21.15.46:0/1793216435 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f35f40de430 con 0x7f360406b8b0 2024-03-29T08:23:06.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.706+0000 7f35faa00640 1 --2- 172.21.15.46:0/1793216435 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35e0066690 0x7f35e0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:06.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.706+0000 7f35faa00640 1 -- 172.21.15.46:0/1793216435 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f35f411f230 con 0x7f360406b8b0 2024-03-29T08:23:06.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.706+0000 7f35fbe00640 1 --2- 172.21.15.46:0/1793216435 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35e0066690 0x7f35e0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:06.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.706+0000 7f35fbe00640 1 --2- 172.21.15.46:0/1793216435 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35e0066690 0x7f35e0068b50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f36040cd0a0 tx=0x7f35ec009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:06.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.710+0000 7f35faa00640 1 -- 172.21.15.46:0/1793216435 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f35f40ec750 con 0x7f360406b8b0 2024-03-29T08:23:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.906+0000 7f3609c00640 1 -- 172.21.15.46:0/1793216435 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36040cdd30 con 0x7f360406b8b0 2024-03-29T08:23:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.906+0000 7f35faa00640 1 -- 172.21.15.46:0/1793216435 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f35f40efd70 con 0x7f360406b8b0 2024-03-29T08:23:06.911 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:23:06.911 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3456771376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3456771376},{"type":"v1","addr":"172.21.15.73:6836","nonce":3456771376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2671538025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2671538025},{"type":"v1","addr":"172.21.15.73:6837","nonce":2671538025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3196015693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3196015693},{"type":"v1","addr":"172.21.15.46:6833","nonce":3196015693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:22:43.605457+0000","modified":"2024-03-29T08:23:05.958069+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1672348161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1672348161},{"type":"v1","addr":"172.21.15.46:6835","nonce":1672348161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-29T08:23:06.911 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 47 2024-03-29T08:23:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.910+0000 7f3609c00640 1 -- 172.21.15.46:0/1793216435 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35e0066690 msgr2=0x7f35e0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.910+0000 7f3609c00640 1 --2- 172.21.15.46:0/1793216435 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35e0066690 0x7f35e0068b50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f36040cd0a0 tx=0x7f35ec009040 comp rx=0 tx=0).stop 2024-03-29T08:23:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.910+0000 7f3609c00640 1 -- 172.21.15.46:0/1793216435 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f360406b8b0 msgr2=0x7f3604145c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.910+0000 7f3609c00640 1 --2- 172.21.15.46:0/1793216435 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f360406b8b0 0x7f3604145c40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f35f40097c0 tx=0x7f35f401acb0 comp rx=0 tx=0).stop 2024-03-29T08:23:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.910+0000 7f3609c00640 1 -- 172.21.15.46:0/1793216435 shutdown_connections 2024-03-29T08:23:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.910+0000 7f3609c00640 1 --2- 172.21.15.46:0/1793216435 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f35e0066690 0x7f35e0068b50 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.910+0000 7f3609c00640 1 --2- 172.21.15.46:0/1793216435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36040d0660 0x7f36040b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:06.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.910+0000 7f3609c00640 1 --2- 172.21.15.46:0/1793216435 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f360415dc00 0x7f3604157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:06.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.910+0000 7f3609c00640 1 --2- 172.21.15.46:0/1793216435 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f360406b8b0 0x7f3604145c40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:06.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.910+0000 7f3609c00640 1 -- 172.21.15.46:0/1793216435 >> 172.21.15.46:0/1793216435 conn(0x7f3604089fd0 msgr2=0x7f360406e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:06.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.910+0000 7f3609c00640 1 -- 172.21.15.46:0/1793216435 shutdown_connections 2024-03-29T08:23:06.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:06.910+0000 7f3609c00640 1 -- 172.21.15.46:0/1793216435 wait complete. 2024-03-29T08:23:07.335 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:22:43.605457+0000', 'modified': '2024-03-29T08:23:05.958069+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6876}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6876': {'gid': 6876, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6835/1672348161', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 1672348161}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1672348161}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:23:07.335 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T08:23:07.335 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-29T08:23:07.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.474+0000 7f988f800640 1 Processor -- start 2024-03-29T08:23:07.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.474+0000 7f988f800640 1 -- start start 2024-03-29T08:23:07.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.478+0000 7f988f800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9888142ab0 0x7f988806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.478+0000 7f988f800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988806de40 con 0x7f988806d9e0 2024-03-29T08:23:07.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.478+0000 7f988f800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9888070c70 con 0x7f988815db10 2024-03-29T08:23:07.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.478+0000 7f988f800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9888070340 con 0x7f9888142ab0 2024-03-29T08:23:07.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.478+0000 7f9887e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f988806d9e0 0x7f988815d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34036/0 (socket says 172.21.15.46:34036) 2024-03-29T08:23:07.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.478+0000 7f9887e00640 1 -- 172.21.15.46:0/3797480547 learned_addr learned my addr 172.21.15.46:0/3797480547 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:07.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.478+0000 7f988ce00640 1 --2- 172.21.15.46:0/3797480547 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9888142ab0 0x7f988806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.478+0000 7f9887400640 1 -- 172.21.15.46:0/3797480547 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828590972 0 0) 0x7f988806de40 con 0x7f988806d9e0 2024-03-29T08:23:07.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.478+0000 7f9887400640 1 -- 172.21.15.46:0/3797480547 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9870003530 con 0x7f988806d9e0 2024-03-29T08:23:07.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.478+0000 7f9887400640 1 -- 172.21.15.46:0/3797480547 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084815268 0 0) 0x7f9888070c70 con 0x7f988815db10 2024-03-29T08:23:07.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.478+0000 7f9887400640 1 -- 172.21.15.46:0/3797480547 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f988806de40 con 0x7f988815db10 2024-03-29T08:23:07.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.478+0000 7f988ce00640 1 -- 172.21.15.46:0/3797480547 >> v1:172.21.15.73:6789/0 conn(0x7f988815db10 legacy=0x7f98880b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:07.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.478+0000 7f988ce00640 1 -- 172.21.15.46:0/3797480547 >> v1:172.21.15.46:6789/0 conn(0x7f988806d9e0 legacy=0x7f988815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:07.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.478+0000 7f988ce00640 1 -- 172.21.15.46:0/3797480547 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9888070620 con 0x7f9888142ab0 2024-03-29T08:23:07.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f988ce00640 1 --2- 172.21.15.46:0/3797480547 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9888142ab0 0x7f988806d4a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f987c0097f0 tx=0x7f987c01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=559c1f3f231fa39f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:07.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f9887400640 1 -- 172.21.15.46:0/3797480547 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c0ab400 con 0x7f9888142ab0 2024-03-29T08:23:07.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f9887400640 1 -- 172.21.15.46:0/3797480547 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c0a7050 con 0x7f9888142ab0 2024-03-29T08:23:07.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f9887400640 1 -- 172.21.15.46:0/3797480547 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c0ab780 con 0x7f9888142ab0 2024-03-29T08:23:07.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f988f800640 1 -- 172.21.15.46:0/3797480547 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9888142ab0 msgr2=0x7f988806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:07.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f988f800640 1 --2- 172.21.15.46:0/3797480547 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9888142ab0 0x7f988806d4a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f987c0097f0 tx=0x7f987c01c4e0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f988f800640 1 -- 172.21.15.46:0/3797480547 shutdown_connections 2024-03-29T08:23:07.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f988f800640 1 --2- 172.21.15.46:0/3797480547 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9888142ab0 0x7f988806d4a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f988f800640 1 -- 172.21.15.46:0/3797480547 >> 172.21.15.46:0/3797480547 conn(0x7f9888066080 msgr2=0x7f9888069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:07.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f988f800640 1 -- 172.21.15.46:0/3797480547 shutdown_connections 2024-03-29T08:23:07.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f988f800640 1 -- 172.21.15.46:0/3797480547 wait complete. 2024-03-29T08:23:07.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f988f800640 1 Processor -- start 2024-03-29T08:23:07.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f988f800640 1 -- start start 2024-03-29T08:23:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f988f800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f988806d9e0 0x7f9888159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f988f800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9888142ab0 0x7f988815a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f988f800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f988815db10 0x7f988815a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f988f800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9888160170 con 0x7f9888142ab0 2024-03-29T08:23:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f988f800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988815f0d0 con 0x7f988806d9e0 2024-03-29T08:23:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f988f800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988815f3b0 con 0x7f988815db10 2024-03-29T08:23:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f9887e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9888142ab0 0x7f988815a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f9887e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9888142ab0 0x7f988815a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40344/0 (socket says 172.21.15.46:40344) 2024-03-29T08:23:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f9887e00640 1 -- 172.21.15.46:0/2101291399 learned_addr learned my addr 172.21.15.46:0/2101291399 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.482+0000 7f988ce00640 1 --2- 172.21.15.46:0/2101291399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f988806d9e0 0x7f9888159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.486+0000 7f988d800640 1 --2- 172.21.15.46:0/2101291399 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f988815db10 0x7f988815a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.486+0000 7f9887e00640 1 -- 172.21.15.46:0/2101291399 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f988815db10 msgr2=0x7f988815a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:07.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.486+0000 7f9887e00640 1 --2- 172.21.15.46:0/2101291399 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f988815db10 0x7f988815a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.486+0000 7f9887e00640 1 -- 172.21.15.46:0/2101291399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f988806d9e0 msgr2=0x7f9888159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:07.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.486+0000 7f9887e00640 1 --2- 172.21.15.46:0/2101291399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f988806d9e0 0x7f9888159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.486+0000 7f9887e00640 1 -- 172.21.15.46:0/2101291399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987c009420 con 0x7f9888142ab0 2024-03-29T08:23:07.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.486+0000 7f9887e00640 1 --2- 172.21.15.46:0/2101291399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9888142ab0 0x7f988815a420 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f987400c420 tx=0x7f987400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.486+0000 7f9884c00640 1 -- 172.21.15.46:0/2101291399 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874005cf0 con 0x7f9888142ab0 2024-03-29T08:23:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.486+0000 7f9884c00640 1 -- 172.21.15.46:0/2101291399 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9874005e50 con 0x7f9888142ab0 2024-03-29T08:23:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.486+0000 7f9884c00640 1 -- 172.21.15.46:0/2101291399 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98740072d0 con 0x7f9888142ab0 2024-03-29T08:23:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.486+0000 7f988f800640 1 -- 172.21.15.46:0/2101291399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988815f690 con 0x7f9888142ab0 2024-03-29T08:23:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.486+0000 7f988f800640 1 -- 172.21.15.46:0/2101291399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988815fc50 con 0x7f9888142ab0 2024-03-29T08:23:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.486+0000 7f9884c00640 1 -- 172.21.15.46:0/2101291399 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9874007430 con 0x7f9888142ab0 2024-03-29T08:23:07.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.486+0000 7f9884c00640 1 --2- 172.21.15.46:0/2101291399 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9868066690 0x7f9868068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9884c00640 1 -- 172.21.15.46:0/2101291399 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f98740837b0 con 0x7f9888142ab0 2024-03-29T08:23:07.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9892b98000 1 Processor -- start 2024-03-29T08:23:07.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f988ce00640 1 --2- 172.21.15.46:0/2101291399 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9868066690 0x7f9868068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9892b98000 1 -- start start 2024-03-29T08:23:07.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f988ce00640 1 --2- 172.21.15.46:0/2101291399 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9868066690 0x7f9868068b50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f987c0ad000 tx=0x7f987c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:07.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9892b98000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06bc9db0 0x556c06bca190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9892b98000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bca6d0 0x556c06b7ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9892b98000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06b804c0 0x556c06b808a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9892b98000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c06a485a0 con 0x556c06bca6d0 2024-03-29T08:23:07.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9892b98000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c069d5af0 con 0x556c06bc9db0 2024-03-29T08:23:07.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9892b98000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c069d30c0 con 0x556c06b804c0 2024-03-29T08:23:07.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9887e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bca6d0 0x556c06b7ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9887e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bca6d0 0x556c06b7ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40352/0 (socket says 172.21.15.46:40352) 2024-03-29T08:23:07.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9887e00640 1 -- 172.21.15.46:0/1925725398 learned_addr learned my addr 172.21.15.46:0/1925725398 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:07.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f988d800640 1 --2- 172.21.15.46:0/1925725398 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06b804c0 0x556c06b808a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f988ce00640 1 --2- 172.21.15.46:0/1925725398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06bc9db0 0x556c06bca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9887e00640 1 -- 172.21.15.46:0/1925725398 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06b804c0 msgr2=0x556c06b808a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:07.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9887e00640 1 --2- 172.21.15.46:0/1925725398 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06b804c0 0x556c06b808a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9887e00640 1 -- 172.21.15.46:0/1925725398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06bc9db0 msgr2=0x556c06bca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:07.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9887e00640 1 --2- 172.21.15.46:0/1925725398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06bc9db0 0x556c06bca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9887e00640 1 -- 172.21.15.46:0/1925725398 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c066ca890 con 0x556c06bca6d0 2024-03-29T08:23:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9887e00640 1 --2- 172.21.15.46:0/1925725398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bca6d0 0x556c06b7ff80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f987405c570 tx=0x7f987400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15c302b714ff8c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9879600640 1 -- 172.21.15.46:0/1925725398 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98740076e0 con 0x556c06bca6d0 2024-03-29T08:23:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9879600640 1 -- 172.21.15.46:0/1925725398 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9874052b00 con 0x556c06bca6d0 2024-03-29T08:23:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9879600640 1 -- 172.21.15.46:0/1925725398 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874050e70 con 0x556c06bca6d0 2024-03-29T08:23:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9892b98000 1 -- 172.21.15.46:0/1925725398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bca6d0 msgr2=0x556c06b7ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9892b98000 1 --2- 172.21.15.46:0/1925725398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bca6d0 0x556c06b7ff80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f987405c570 tx=0x7f987400cf90 comp rx=0 tx=0).stop 2024-03-29T08:23:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9892b98000 1 -- 172.21.15.46:0/1925725398 shutdown_connections 2024-03-29T08:23:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9892b98000 1 --2- 172.21.15.46:0/1925725398 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06b804c0 0x556c06b808a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9892b98000 1 --2- 172.21.15.46:0/1925725398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bca6d0 0x556c06b7ff80 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9892b98000 1 --2- 172.21.15.46:0/1925725398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06bc9db0 0x556c06bca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9892b98000 1 -- 172.21.15.46:0/1925725398 >> 172.21.15.46:0/1925725398 conn(0x556c06b86830 msgr2=0x556c06b7e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9892b98000 1 -- 172.21.15.46:0/1925725398 shutdown_connections 2024-03-29T08:23:07.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.490+0000 7f9892b98000 1 -- 172.21.15.46:0/1925725398 wait complete. 2024-03-29T08:23:07.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 1 -- start start 2024-03-29T08:23:07.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 1 Processor -- start 2024-03-29T08:23:07.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06b804c0 0x556c06bc95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06bc9db0 0x556c06b84230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bca6d0 0x556c06b84770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c069ce9b0 con 0x556c06bca6d0 2024-03-29T08:23:07.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0696dc00 con 0x556c06b804c0 2024-03-29T08:23:07.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c0698c070 con 0x556c06bc9db0 2024-03-29T08:23:07.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 10 client.0 _ll_register_callbacks cb 0x556c06b6c0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:23:07.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f988d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bca6d0 0x556c06b84770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f988d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bca6d0 0x556c06b84770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40362/0 (socket says 172.21.15.46:40362) 2024-03-29T08:23:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f988d800640 1 -- 172.21.15.46:0/3276440155 learned_addr learned my addr 172.21.15.46:0/3276440155 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9887e00640 1 --2- 172.21.15.46:0/3276440155 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06bc9db0 0x556c06b84230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f988ce00640 1 --2- 172.21.15.46:0/3276440155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06b804c0 0x556c06bc95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f988d800640 1 -- 172.21.15.46:0/3276440155 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06bc9db0 msgr2=0x556c06b84230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f988d800640 1 --2- 172.21.15.46:0/3276440155 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06bc9db0 0x556c06b84230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f988d800640 1 -- 172.21.15.46:0/3276440155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06b804c0 msgr2=0x556c06bc95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f988d800640 1 --2- 172.21.15.46:0/3276440155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06b804c0 0x556c06bc95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f988d800640 1 -- 172.21.15.46:0/3276440155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988815fc50 con 0x556c06bca6d0 2024-03-29T08:23:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f988d800640 1 --2- 172.21.15.46:0/3276440155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bca6d0 0x556c06b84770 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9880012660 tx=0x7f9880012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60811da5f77feda server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f986e000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:23:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f986e000640 1 -- 172.21.15.46:0/3276440155 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9880003980 con 0x556c06bca6d0 2024-03-29T08:23:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f986e000640 1 -- 172.21.15.46:0/3276440155 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9880003ae0 con 0x556c06bca6d0 2024-03-29T08:23:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 1 -- 172.21.15.46:0/3276440155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c0698b5f0 con 0x556c06bca6d0 2024-03-29T08:23:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f986e000640 1 -- 172.21.15.46:0/3276440155 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9880003ce0 con 0x556c06bca6d0 2024-03-29T08:23:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 10 client.6927 fetch_fsmap learned FSMap version 47 2024-03-29T08:23:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 1 -- 172.21.15.46:0/3276440155 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x556c06993ab0 con 0x556c06bca6d0 2024-03-29T08:23:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f986e000640 1 -- 172.21.15.46:0/3276440155 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556c0698b5f0 con 0x556c06bca6d0 2024-03-29T08:23:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f986e000640 1 -- 172.21.15.46:0/3276440155 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9880010820 con 0x556c06bca6d0 2024-03-29T08:23:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 10 client.6927 fetch_fsmap finished waiting for FSMap version 47 2024-03-29T08:23:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 10 client.6927 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:23:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 20 client.6927 populate_metadata read hostname 'smithi046' 2024-03-29T08:23:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 1 --2- 172.21.15.46:0/3276440155 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x556c06b85180 0x556c06b85560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 4 client.6927 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:23:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.494+0000 7f9892b98000 1 -- 172.21.15.46:0/3276440155 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c069cb790 con 0x556c06b85180 2024-03-29T08:23:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f988ce00640 1 --2- 172.21.15.46:0/3276440155 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x556c06b85180 0x556c06b85560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f988ce00640 1 --2- 172.21.15.46:0/3276440155 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x556c06b85180 0x556c06b85560 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0699b5fa90c1df6 server_cookie=cfe1113fc2e62239 in_seq=0 out_seq=0 2024-03-29T08:23:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f986e000640 10 client.6927 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f986e000640 1 -- 172.21.15.46:0/3276440155 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556c069cb790 con 0x556c06b85180 2024-03-29T08:23:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f986e000640 10 client.6927 handle_command_reply: tid=0 2024-03-29T08:23:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f9892b98000 1 client.6927 shutdown 2024-03-29T08:23:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f9892b98000 1 -- 172.21.15.46:0/3276440155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bca6d0 msgr2=0x556c06b84770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f9892b98000 1 --2- 172.21.15.46:0/3276440155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bca6d0 0x556c06b84770 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9880012660 tx=0x7f9880012b20 comp rx=0 tx=0).stop 2024-03-29T08:23:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f9892b98000 1 -- 172.21.15.46:0/3276440155 shutdown_connections 2024-03-29T08:23:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f9892b98000 1 --2- 172.21.15.46:0/3276440155 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x556c06b85180 0x556c06b85560 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f9892b98000 1 --2- 172.21.15.46:0/3276440155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bca6d0 0x556c06b84770 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f9892b98000 1 --2- 172.21.15.46:0/3276440155 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06bc9db0 0x556c06b84230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f9892b98000 1 --2- 172.21.15.46:0/3276440155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06b804c0 0x556c06bc95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f986e000640 0 client.6927 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f9892b98000 1 -- 172.21.15.46:0/3276440155 >> 172.21.15.46:0/3276440155 conn(0x556c06b86830 msgr2=0x556c06b7ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f9892b98000 1 -- 172.21.15.46:0/3276440155 shutdown_connections 2024-03-29T08:23:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f9892b98000 1 -- 172.21.15.46:0/3276440155 wait complete. 2024-03-29T08:23:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.498+0000 7f9892b98000 20 client.6927 trim_cache size 0 max 16384 2024-03-29T08:23:07.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f9892b98000 1 Processor -- start 2024-03-29T08:23:07.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f9892b98000 1 -- start start 2024-03-29T08:23:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f9892b98000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06b804c0 0x556c06bc3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f9892b98000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06bc9db0 0x556c06bc3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f9892b98000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06bca6d0 0x556c06bc41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f9892b98000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0668b940 con 0x556c06b804c0 2024-03-29T08:23:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f9892b98000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c0696bc10 con 0x556c06bca6d0 2024-03-29T08:23:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f9892b98000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c06995c60 con 0x556c06bc9db0 2024-03-29T08:23:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f988ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06b804c0 0x556c06bc3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f988ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06b804c0 0x556c06bc3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40366/0 (socket says 172.21.15.46:40366) 2024-03-29T08:23:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f988ce00640 1 -- 172.21.15.46:0/1513339558 learned_addr learned my addr 172.21.15.46:0/1513339558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f9887e00640 1 --2- 172.21.15.46:0/1513339558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06bc9db0 0x556c06bc3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f988d800640 1 --2- 172.21.15.46:0/1513339558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06bca6d0 0x556c06bc41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f988ce00640 1 -- 172.21.15.46:0/1513339558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06bc9db0 msgr2=0x556c06bc3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f988ce00640 1 --2- 172.21.15.46:0/1513339558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06bc9db0 0x556c06bc3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f988ce00640 1 -- 172.21.15.46:0/1513339558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06bca6d0 msgr2=0x556c06bc41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:07.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f988ce00640 1 --2- 172.21.15.46:0/1513339558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06bca6d0 0x556c06bc41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f988ce00640 1 -- 172.21.15.46:0/1513339558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c06979c00 con 0x556c06b804c0 2024-03-29T08:23:07.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f988ce00640 1 --2- 172.21.15.46:0/1513339558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06b804c0 0x556c06bc3770 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f987c0190f0 tx=0x7f987c0a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5163557f5bc687 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:07.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f987a000640 1 -- 172.21.15.46:0/1513339558 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c0ca270 con 0x556c06b804c0 2024-03-29T08:23:07.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f987a000640 1 -- 172.21.15.46:0/1513339558 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c0bd040 con 0x556c06b804c0 2024-03-29T08:23:07.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.514+0000 7f987a000640 1 -- 172.21.15.46:0/1513339558 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c0ca5d0 con 0x556c06b804c0 2024-03-29T08:23:07.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 -- 172.21.15.46:0/1513339558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06b804c0 msgr2=0x556c06bc3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:07.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 --2- 172.21.15.46:0/1513339558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06b804c0 0x556c06bc3770 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f987c0190f0 tx=0x7f987c0a6000 comp rx=0 tx=0).stop 2024-03-29T08:23:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 -- 172.21.15.46:0/1513339558 shutdown_connections 2024-03-29T08:23:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 --2- 172.21.15.46:0/1513339558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06bca6d0 0x556c06bc41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 --2- 172.21.15.46:0/1513339558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06bc9db0 0x556c06bc3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 --2- 172.21.15.46:0/1513339558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06b804c0 0x556c06bc3770 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 -- 172.21.15.46:0/1513339558 >> 172.21.15.46:0/1513339558 conn(0x556c06b86830 msgr2=0x556c06bce080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 -- 172.21.15.46:0/1513339558 shutdown_connections 2024-03-29T08:23:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 -- 172.21.15.46:0/1513339558 wait complete. 2024-03-29T08:23:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 -- start start 2024-03-29T08:23:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 Processor -- start 2024-03-29T08:23:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06b804c0 0x556c06c20050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bc9db0 0x556c06bc6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06bca6d0 0x556c06bc7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c06a9a8f0 con 0x556c06bc9db0 2024-03-29T08:23:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c069cc980 con 0x556c06bca6d0 2024-03-29T08:23:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c069557c0 con 0x556c06b804c0 2024-03-29T08:23:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 10 client.0 _ll_register_callbacks cb 0x556c06993590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:23:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9887e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bc9db0 0x556c06bc6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9887e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bc9db0 0x556c06bc6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40382/0 (socket says 172.21.15.46:40382) 2024-03-29T08:23:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9887e00640 1 -- 172.21.15.46:0/1577551503 learned_addr learned my addr 172.21.15.46:0/1577551503 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f988ce00640 1 --2- 172.21.15.46:0/1577551503 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06b804c0 0x556c06c20050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f988d800640 1 --2- 172.21.15.46:0/1577551503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06bca6d0 0x556c06bc7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9887e00640 1 -- 172.21.15.46:0/1577551503 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06b804c0 msgr2=0x556c06c20050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9887e00640 1 --2- 172.21.15.46:0/1577551503 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06b804c0 0x556c06c20050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9887e00640 1 -- 172.21.15.46:0/1577551503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06bca6d0 msgr2=0x556c06bc7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9887e00640 1 --2- 172.21.15.46:0/1577551503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06bca6d0 0x556c06bc7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9887e00640 1 -- 172.21.15.46:0/1577551503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9888070620 con 0x556c06bc9db0 2024-03-29T08:23:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f988ce00640 1 --2- 172.21.15.46:0/1577551503 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06b804c0 0x556c06c20050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:23:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9887e00640 1 --2- 172.21.15.46:0/1577551503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bc9db0 0x556c06bc6f30 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9874054920 tx=0x7f9874014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a09c274856b43f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f986e000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:23:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f986e000640 1 -- 172.21.15.46:0/1577551503 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9874050e70 con 0x556c06bc9db0 2024-03-29T08:23:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f986e000640 1 -- 172.21.15.46:0/1577551503 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9874050fd0 con 0x556c06bc9db0 2024-03-29T08:23:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 -- 172.21.15.46:0/1577551503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c0699e460 con 0x556c06bc9db0 2024-03-29T08:23:07.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f986e000640 1 -- 172.21.15.46:0/1577551503 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f987405c610 con 0x556c06bc9db0 2024-03-29T08:23:07.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 10 client.6933 fetch_fsmap learned FSMap version 47 2024-03-29T08:23:07.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f9892b98000 1 -- 172.21.15.46:0/1577551503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x556c066cacc0 con 0x556c06bc9db0 2024-03-29T08:23:07.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.518+0000 7f986e000640 1 -- 172.21.15.46:0/1577551503 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556c0699e460 con 0x556c06bc9db0 2024-03-29T08:23:07.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f986e000640 1 -- 172.21.15.46:0/1577551503 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 47) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9874050550 con 0x556c06bc9db0 2024-03-29T08:23:07.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 10 client.6933 fetch_fsmap finished waiting for FSMap version 47 2024-03-29T08:23:07.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 10 client.6933 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:23:07.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 20 client.6933 populate_metadata read hostname 'smithi046' 2024-03-29T08:23:07.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 1 --2- 172.21.15.46:0/1577551503 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x556c06bc79b0 0x556c06bc7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 4 client.6933 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T08:23:07.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 1 -- 172.21.15.46:0/1577551503 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556c069979c0 con 0x556c06bc79b0 2024-03-29T08:23:07.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f988ce00640 1 --2- 172.21.15.46:0/1577551503 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x556c06bc79b0 0x556c06bc7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f988ce00640 1 --2- 172.21.15.46:0/1577551503 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x556c06bc79b0 0x556c06bc7d90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=751b5b7b59ba6491 server_cookie=2c818ef3299c697 in_seq=0 out_seq=0 2024-03-29T08:23:07.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f986e000640 10 client.6933 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:07.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f986e000640 1 -- 172.21.15.46:0/1577551503 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x556c069979c0 con 0x556c06bc79b0 2024-03-29T08:23:07.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f986e000640 10 client.6933 handle_command_reply: tid=0 2024-03-29T08:23:07.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 1 client.6933 shutdown 2024-03-29T08:23:07.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 1 -- 172.21.15.46:0/1577551503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bc9db0 msgr2=0x556c06bc6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:07.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 1 --2- 172.21.15.46:0/1577551503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bc9db0 0x556c06bc6f30 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9874054920 tx=0x7f9874014070 comp rx=0 tx=0).stop 2024-03-29T08:23:07.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 1 -- 172.21.15.46:0/1577551503 shutdown_connections 2024-03-29T08:23:07.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 1 --2- 172.21.15.46:0/1577551503 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x556c06bc79b0 0x556c06bc7d90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 1 --2- 172.21.15.46:0/1577551503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556c06bca6d0 0x556c06bc7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 1 --2- 172.21.15.46:0/1577551503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c06bc9db0 0x556c06bc6f30 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 1 --2- 172.21.15.46:0/1577551503 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556c06b804c0 0x556c06c20050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f986e000640 0 client.6933 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:07.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 1 -- 172.21.15.46:0/1577551503 >> 172.21.15.46:0/1577551503 conn(0x556c06b86830 msgr2=0x556c06bce780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:07.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 1 -- 172.21.15.46:0/1577551503 shutdown_connections 2024-03-29T08:23:07.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 1 -- 172.21.15.46:0/1577551503 wait complete. 2024-03-29T08:23:07.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f9892b98000 20 client.6933 trim_cache size 0 max 16384 2024-03-29T08:23:07.529 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:23:07.529 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "e9b7960f-ec1f-4787-95c3-c806094d37f0", 2024-03-29T08:23:07.529 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-03-29T08:23:07.529 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6876, 2024-03-29T08:23:07.529 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-29T08:23:07.529 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-29T08:23:07.529 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-29T08:23:07.529 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 4.5772414179999998, 2024-03-29T08:23:07.530 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 47, 2024-03-29T08:23:07.530 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 80, 2024-03-29T08:23:07.530 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 80, 2024-03-29T08:23:07.530 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 5.057175193 2024-03-29T08:23:07.530 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:23:07.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f988f800640 1 -- 172.21.15.46:0/2101291399 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9868066690 msgr2=0x7f9868068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:07.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f988f800640 1 --2- 172.21.15.46:0/2101291399 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9868066690 0x7f9868068b50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f987c0ad000 tx=0x7f987c008040 comp rx=0 tx=0).stop 2024-03-29T08:23:07.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f988f800640 1 -- 172.21.15.46:0/2101291399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9888142ab0 msgr2=0x7f988815a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:07.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f988f800640 1 --2- 172.21.15.46:0/2101291399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9888142ab0 0x7f988815a420 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f987400c420 tx=0x7f987400c8e0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f988f800640 1 -- 172.21.15.46:0/2101291399 shutdown_connections 2024-03-29T08:23:07.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f988f800640 1 --2- 172.21.15.46:0/2101291399 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9868066690 0x7f9868068b50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f988f800640 1 --2- 172.21.15.46:0/2101291399 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f988815db10 0x7f988815a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f988f800640 1 --2- 172.21.15.46:0/2101291399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9888142ab0 0x7f988815a420 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f988f800640 1 --2- 172.21.15.46:0/2101291399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f988806d9e0 0x7f9888159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f988f800640 1 -- 172.21.15.46:0/2101291399 >> 172.21.15.46:0/2101291399 conn(0x7f9888066080 msgr2=0x7f98880b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:07.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f988f800640 1 -- 172.21.15.46:0/2101291399 shutdown_connections 2024-03-29T08:23:07.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.522+0000 7f988f800640 1 -- 172.21.15.46:0/2101291399 wait complete. 2024-03-29T08:23:07.545 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:23:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.682+0000 7fb65ca00640 1 Processor -- start 2024-03-29T08:23:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.686+0000 7fb65ca00640 1 -- start start 2024-03-29T08:23:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.686+0000 7fb65ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb65815ad20 0x7fb6580cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.686+0000 7fb65ca00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb6580b0540 0x7fb6580cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.686+0000 7fb65ca00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6580b0ab0 con 0x7fb65815a940 2024-03-29T08:23:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.686+0000 7fb65ca00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6580b0d90 con 0x7fb65815ad20 2024-03-29T08:23:07.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.686+0000 7fb65ca00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6580b1070 con 0x7fb6580b0540 2024-03-29T08:23:07.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.686+0000 7fb64fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb65815ad20 0x7fb6580cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.686+0000 7fb64fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb65815ad20 0x7fb6580cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40396/0 (socket says 172.21.15.46:40396) 2024-03-29T08:23:07.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.686+0000 7fb64fe00640 1 -- 172.21.15.46:0/3225026464 learned_addr learned my addr 172.21.15.46:0/3225026464 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.686+0000 7fb656800640 1 --2- 172.21.15.46:0/3225026464 >> v2:172.21.15.73:3300/0 conn(0x7fb6580b0540 0x7fb6580cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.686+0000 7fb64fe00640 1 -- 172.21.15.46:0/3225026464 >> v2:172.21.15.73:3300/0 conn(0x7fb6580b0540 msgr2=0x7fb6580cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.686+0000 7fb64fe00640 1 --2- 172.21.15.46:0/3225026464 >> v2:172.21.15.73:3300/0 conn(0x7fb6580b0540 0x7fb6580cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.686+0000 7fb64fe00640 1 -- 172.21.15.46:0/3225026464 >> v1:172.21.15.73:6789/0 conn(0x7fb65815a940 legacy=0x7fb6580afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:07.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.686+0000 7fb64fe00640 1 -- 172.21.15.46:0/3225026464 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6580b1350 con 0x7fb65815ad20 2024-03-29T08:23:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.690+0000 7fb64fe00640 1 --2- 172.21.15.46:0/3225026464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb65815ad20 0x7fb6580cecb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb6580c7750 tx=0x7fb64001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae2a867c1b349e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.690+0000 7fb655400640 1 -- 172.21.15.46:0/3225026464 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6400ab680 con 0x7fb65815ad20 2024-03-29T08:23:07.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.690+0000 7fb655400640 1 -- 172.21.15.46:0/3225026464 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6400ab7e0 con 0x7fb65815ad20 2024-03-29T08:23:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.690+0000 7fb655400640 1 -- 172.21.15.46:0/3225026464 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6400abb70 con 0x7fb65815ad20 2024-03-29T08:23:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.690+0000 7fb65ca00640 1 -- 172.21.15.46:0/3225026464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb65815ad20 msgr2=0x7fb6580cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.690+0000 7fb65ca00640 1 --2- 172.21.15.46:0/3225026464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb65815ad20 0x7fb6580cecb0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb6580c7750 tx=0x7fb64001c460 comp rx=0 tx=0).stop 2024-03-29T08:23:07.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.690+0000 7fb65ca00640 1 -- 172.21.15.46:0/3225026464 shutdown_connections 2024-03-29T08:23:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.690+0000 7fb65ca00640 1 --2- 172.21.15.46:0/3225026464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb65815ad20 0x7fb6580cecb0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.690+0000 7fb65ca00640 1 --2- 172.21.15.46:0/3225026464 >> v2:172.21.15.73:3300/0 conn(0x7fb6580b0540 0x7fb6580cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.690+0000 7fb65ca00640 1 -- 172.21.15.46:0/3225026464 >> 172.21.15.46:0/3225026464 conn(0x7fb658089fd0 msgr2=0x7fb65808a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.690+0000 7fb65ca00640 1 -- 172.21.15.46:0/3225026464 shutdown_connections 2024-03-29T08:23:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.690+0000 7fb65ca00640 1 -- 172.21.15.46:0/3225026464 wait complete. 2024-03-29T08:23:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.690+0000 7fb65ca00640 1 Processor -- start 2024-03-29T08:23:07.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.690+0000 7fb65ca00640 1 -- start start 2024-03-29T08:23:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb65ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6580b0540 0x7fb65815bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb65ca00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb65815a940 0x7fb6580c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb65ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb65815ad20 0x7fb6580b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb65ca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65815c100 con 0x7fb65815ad20 2024-03-29T08:23:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb65ca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65815c3e0 con 0x7fb6580b0540 2024-03-29T08:23:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb65ca00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65815c6c0 con 0x7fb65815a940 2024-03-29T08:23:07.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb656800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb65815ad20 0x7fb6580b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb656800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb65815ad20 0x7fb6580b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40400/0 (socket says 172.21.15.46:40400) 2024-03-29T08:23:07.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb656800640 1 -- 172.21.15.46:0/2901174954 learned_addr learned my addr 172.21.15.46:0/2901174954 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:07.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb655e00640 1 --2- 172.21.15.46:0/2901174954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6580b0540 0x7fb65815bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb64fe00640 1 --2- 172.21.15.46:0/2901174954 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb65815a940 0x7fb6580c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb656800640 1 -- 172.21.15.46:0/2901174954 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb65815a940 msgr2=0x7fb6580c77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:07.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb656800640 1 --2- 172.21.15.46:0/2901174954 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb65815a940 0x7fb6580c77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb656800640 1 -- 172.21.15.46:0/2901174954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6580b0540 msgr2=0x7fb65815bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:07.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb656800640 1 --2- 172.21.15.46:0/2901174954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6580b0540 0x7fb65815bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb656800640 1 -- 172.21.15.46:0/2901174954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6400094d0 con 0x7fb65815ad20 2024-03-29T08:23:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb656800640 1 --2- 172.21.15.46:0/2901174954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb65815ad20 0x7fb6580b5a90 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fb650007e40 tx=0x7fb65000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb64e000640 1 -- 172.21.15.46:0/2901174954 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650014620 con 0x7fb65815ad20 2024-03-29T08:23:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb64e000640 1 -- 172.21.15.46:0/2901174954 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65000ee90 con 0x7fb65815ad20 2024-03-29T08:23:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb65ca00640 1 -- 172.21.15.46:0/2901174954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65815c9a0 con 0x7fb65815ad20 2024-03-29T08:23:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb64e000640 1 -- 172.21.15.46:0/2901174954 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6500149e0 con 0x7fb65815ad20 2024-03-29T08:23:07.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb65ca00640 1 -- 172.21.15.46:0/2901174954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6580b5fd0 con 0x7fb65815ad20 2024-03-29T08:23:07.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb65ca00640 1 -- 172.21.15.46:0/2901174954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65815cec0 con 0x7fb65815ad20 2024-03-29T08:23:07.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.694+0000 7fb64e000640 1 -- 172.21.15.46:0/2901174954 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb650014b40 con 0x7fb65815ad20 2024-03-29T08:23:07.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.698+0000 7fb64e000640 1 --2- 172.21.15.46:0/2901174954 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb634066690 0x7fb634068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:07.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.698+0000 7fb64e000640 1 -- 172.21.15.46:0/2901174954 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb650084b70 con 0x7fb65815ad20 2024-03-29T08:23:07.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.698+0000 7fb655e00640 1 --2- 172.21.15.46:0/2901174954 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb634066690 0x7fb634068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:07.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.698+0000 7fb655e00640 1 --2- 172.21.15.46:0/2901174954 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb634066690 0x7fb634068b50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb648007fd0 tx=0x7fb648009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:07.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.702+0000 7fb64e000640 1 -- 172.21.15.46:0/2901174954 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb65815cec0 con 0x7fb65815ad20 2024-03-29T08:23:07.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.898+0000 7fb65ca00640 1 -- 172.21.15.46:0/2901174954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb65815cec0 con 0x7fb65815ad20 2024-03-29T08:23:07.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.898+0000 7fb64e000640 1 -- 172.21.15.46:0/2901174954 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb65815cec0 con 0x7fb65815ad20 2024-03-29T08:23:07.901 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:23:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.898+0000 7fb65ca00640 1 -- 172.21.15.46:0/2901174954 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb634066690 msgr2=0x7fb634068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:07.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.898+0000 7fb65ca00640 1 --2- 172.21.15.46:0/2901174954 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb634066690 0x7fb634068b50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb648007fd0 tx=0x7fb648009040 comp rx=0 tx=0).stop 2024-03-29T08:23:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.898+0000 7fb65ca00640 1 -- 172.21.15.46:0/2901174954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb65815ad20 msgr2=0x7fb6580b5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.898+0000 7fb65ca00640 1 --2- 172.21.15.46:0/2901174954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb65815ad20 0x7fb6580b5a90 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fb650007e40 tx=0x7fb65000e4b0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.898+0000 7fb65ca00640 1 -- 172.21.15.46:0/2901174954 shutdown_connections 2024-03-29T08:23:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.898+0000 7fb65ca00640 1 --2- 172.21.15.46:0/2901174954 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb634066690 0x7fb634068b50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.898+0000 7fb65ca00640 1 --2- 172.21.15.46:0/2901174954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb65815ad20 0x7fb6580b5a90 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.898+0000 7fb65ca00640 1 --2- 172.21.15.46:0/2901174954 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb65815a940 0x7fb6580c77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.898+0000 7fb65ca00640 1 --2- 172.21.15.46:0/2901174954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6580b0540 0x7fb65815bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.898+0000 7fb65ca00640 1 -- 172.21.15.46:0/2901174954 >> 172.21.15.46:0/2901174954 conn(0x7fb658089fd0 msgr2=0x7fb65806f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.902+0000 7fb65ca00640 1 -- 172.21.15.46:0/2901174954 shutdown_connections 2024-03-29T08:23:07.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:07.902+0000 7fb65ca00640 1 -- 172.21.15.46:0/2901174954 wait complete. 2024-03-29T08:23:07.922 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:23:07.922 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:23:07.922 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-29T08:23:07.922 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T08:23:07.922 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:23:07.922 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:23:07.922 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:23:07.922 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:23:07.923 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:23:07.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:07.923 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-29T08:23:07.932 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:23:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:23:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:23:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:23:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:23:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:23:07.933 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:23:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:23:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-29T08:23:07.933 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:23:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:23:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-29T08:23:07.933 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:23:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:23:07.933 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-29T08:23:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T08:23:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42149sec preferred_lft 42149sec 2024-03-29T08:23:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-29T08:23:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:23:07.934 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:23:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:23:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-29T08:23:07.934 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:23:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:d9:58:09:4c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:23:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:23:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:23:07.934 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:23:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T08:23:07.934 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T08:23:07.935 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:23:07.935 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T08:23:07.935 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:23:07.935 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:23:07.935 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:23:07.935 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T08:23:07.935 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:23:07.935 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:23:07.935 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T08:23:07.935 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-29T08:23:07.935 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:23:07.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:07.936 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:23:07.997 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:23:07.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:07.999 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-29T08:23:08.052 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-29T08:23:08.052 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:23:08.053 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T08:23:08.053 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:23:08.053 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T08:23:08.053 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:23:08.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:08.053 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:23:08.053 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:23:08.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:23:08.054 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:23:08.054 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:23:08.054 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:23:08.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:08.146 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:23:08.156 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T08:23:08.156 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T08:23:08.156 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:23:08.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:08.156 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-03-29T08:23:08.219 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T08:23:08.219 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:23:08.219 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T08:23:08.323 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:23:08.324 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:23:08.324 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:23:08.332 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:23:08.395 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:23:08.396 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:23:08.461 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:23:08.463 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:23:08.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:08.463 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:23:08.519 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:23:08.519 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T08:23:08.566 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:23:08.587 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:23:08.620 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:23:08.655 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:23:08.657 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:23:08.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:08.658 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:23:08.660 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-03-29T08:23:08.658+0000 7fad7364e580 -1 init, newargv = 0x565466625ec0 newargc=15 2024-03-29T08:23:08.662 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[42408]: starting ceph client 2024-03-29T08:23:08.669 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[42408]: starting fuse 2024-03-29T08:23:08.675 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:23:08.676 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-29T08:23:08.676 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:23:08.676 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-03-29T08:23:08.676 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-03-29T08:23:08.677 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:23:08.677 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-03-29T08:23:08.677 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:23:08.677 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-03-29T08:23:08.677 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T08:23:08.677 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-03-29T08:23:08.677 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:23:08.677 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:23:08.677 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-03-29T08:23:08.678 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-03-29T08:23:08.678 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-03-29T08:23:08.678 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-03-29T08:23:08.678 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:23:08.678 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-03-29T08:23:08.678 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:23:08.678 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:23:08.679 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:23:08.679 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-03-29T08:23:08.679 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-03-29T08:23:08.679 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-03-29T08:23:08.679 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:23:08.679 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:23:08.679 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-03-29T08:23:08.679 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:23:08.810 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.42408.asok 2024-03-29T08:23:08.817 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:23:08.823 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:23:08.991 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42408.asok 2024-03-29T08:23:08.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:08.992 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42408.asok status 2024-03-29T08:23:09.123 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:23:09.123 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-03-29T08:23:09.123 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:23:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:23:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-03-29T08:23:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-03-29T08:23:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T08:23:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "42408", 2024-03-29T08:23:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-03-29T08:23:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:23:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-03-29T08:23:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-03-29T08:23:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 6945, 2024-03-29T08:23:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-03-29T08:23:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-03-29T08:23:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-03-29T08:23:09.124 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 6945 2024-03-29T08:23:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:23:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:23:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:23:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:23:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 704241615 2024-03-29T08:23:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:23:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:23:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:23:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:23:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:23:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 704241615 2024-03-29T08:23:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:23:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.6945 192.168.0.1:0/704241615", 2024-03-29T08:23:09.125 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/704241615", 2024-03-29T08:23:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-03-29T08:23:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 47, 2024-03-29T08:23:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 80, 2024-03-29T08:23:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:23:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-03-29T08:23:09.126 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-03-29T08:23:09.126 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:23:09.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:09.139 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-03-29T08:23:09.139 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:23:09.148 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-03-29T08:23:09.149 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T08:23:09.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:09.149 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:23:09.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:09.210 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-03-29T08:23:09.264 INFO:teuthology.orchestra.run.smithi046.stdout:file_undamaged 2024-03-29T08:23:09.265 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:23:09.265 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:23:09.266 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:23:09.266 DEBUG:teuthology.orchestra.run.smithi046:> import json 2024-03-29T08:23:09.266 DEBUG:teuthology.orchestra.run.smithi046:> import sys 2024-03-29T08:23:09.266 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:23:09.266 DEBUG:teuthology.orchestra.run.smithi046:> try: 2024-03-29T08:23:09.266 DEBUG:teuthology.orchestra.run.smithi046:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-03-29T08:23:09.266 DEBUG:teuthology.orchestra.run.smithi046:> except OSError as e: 2024-03-29T08:23:09.266 DEBUG:teuthology.orchestra.run.smithi046:> sys.exit(e.errno) 2024-03-29T08:23:09.266 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:23:09.266 DEBUG:teuthology.orchestra.run.smithi046:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-03-29T08:23:09.266 DEBUG:teuthology.orchestra.run.smithi046:> print(json.dumps( 2024-03-29T08:23:09.266 DEBUG:teuthology.orchestra.run.smithi046:> dict([(a, getattr(s, a)) for a in attrs]), 2024-03-29T08:23:09.266 DEBUG:teuthology.orchestra.run.smithi046:> indent=2)) 2024-03-29T08:23:09.266 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:23:09.557 INFO:teuthology.orchestra.run.smithi046.stderr:daemon-helper: command failed with exit status 2 2024-03-29T08:23:09.564 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-03-29T08:23:09.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:09.565 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-03-29T08:23:09.631 INFO:teuthology.orchestra.run.smithi046.stdout:2 2024-03-29T08:23:09.632 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:23:09.632 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:23:09.684 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:23:09.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.862+0000 7fbb03e00640 1 Processor -- start 2024-03-29T08:23:09.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.862+0000 7fbb03e00640 1 -- start start 2024-03-29T08:23:09.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.862+0000 7fbb03e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbb040ce680 0x7fbb040cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:09.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.862+0000 7fbb03e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb04099a50 con 0x7fbb040b1570 2024-03-29T08:23:09.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.866+0000 7fbb03e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb040cf3f0 con 0x7fbb040cefa0 2024-03-29T08:23:09.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.866+0000 7fbb03e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb0415f5f0 con 0x7fbb040ce680 2024-03-29T08:23:09.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.866+0000 7fbb02a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbb040ce680 0x7fbb040cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:09.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.866+0000 7fbafbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbb040b1570 0x7fbb0415e7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34044/0 (socket says 172.21.15.46:34044) 2024-03-29T08:23:09.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.866+0000 7fbafbe00640 1 -- 172.21.15.46:0/3206436494 learned_addr learned my addr 172.21.15.46:0/3206436494 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:09.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.866+0000 7fbb02a00640 1 -- 172.21.15.46:0/3206436494 >> v1:172.21.15.73:6789/0 conn(0x7fbb040cefa0 legacy=0x7fbb0415eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:09.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.866+0000 7fbb02a00640 1 -- 172.21.15.46:0/3206436494 >> v1:172.21.15.46:6789/0 conn(0x7fbb040b1570 legacy=0x7fbb0415e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:09.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.866+0000 7fbb02a00640 1 -- 172.21.15.46:0/3206436494 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb04099c80 con 0x7fbb040ce680 2024-03-29T08:23:09.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.866+0000 7fbb02000640 1 -- 172.21.15.46:0/3206436494 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702532883 0 0) 0x7fbb04099a50 con 0x7fbb040b1570 2024-03-29T08:23:09.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.866+0000 7fbb02a00640 1 --2- 172.21.15.46:0/3206436494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040ce680 0x7fbb040cea60 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40097c0 tx=0x7fbaf401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991392f043dbf395 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:09.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.866+0000 7fbb02000640 1 -- 172.21.15.46:0/3206436494 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf40ab600 con 0x7fbb040ce680 2024-03-29T08:23:09.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.866+0000 7fbb02000640 1 -- 172.21.15.46:0/3206436494 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf40a7050 con 0x7fbb040ce680 2024-03-29T08:23:09.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.866+0000 7fbb02000640 1 -- 172.21.15.46:0/3206436494 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf40ab980 con 0x7fbb040ce680 2024-03-29T08:23:09.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb03e00640 1 -- 172.21.15.46:0/3206436494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040ce680 msgr2=0x7fbb040cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:09.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb03e00640 1 --2- 172.21.15.46:0/3206436494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040ce680 0x7fbb040cea60 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40097c0 tx=0x7fbaf401c3d0 comp rx=0 tx=0).stop 2024-03-29T08:23:09.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb03e00640 1 -- 172.21.15.46:0/3206436494 shutdown_connections 2024-03-29T08:23:09.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb03e00640 1 --2- 172.21.15.46:0/3206436494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040ce680 0x7fbb040cea60 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:09.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb03e00640 1 -- 172.21.15.46:0/3206436494 >> 172.21.15.46:0/3206436494 conn(0x7fbb040b2010 msgr2=0x7fbb040b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:09.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb03e00640 1 -- 172.21.15.46:0/3206436494 shutdown_connections 2024-03-29T08:23:09.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb03e00640 1 -- 172.21.15.46:0/3206436494 wait complete. 2024-03-29T08:23:09.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb03e00640 1 Processor -- start 2024-03-29T08:23:09.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb03e00640 1 -- start start 2024-03-29T08:23:09.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb03e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040b1570 0x7fbb0409c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:09.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb03e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb040ce680 0x7fbb0409d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:09.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb03e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb040cefa0 0x7fbb04152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:09.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb03e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0409c6d0 con 0x7fbb040b1570 2024-03-29T08:23:09.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb03e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0409c950 con 0x7fbb040ce680 2024-03-29T08:23:09.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb03e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0409cc30 con 0x7fbb040cefa0 2024-03-29T08:23:09.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb02a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040b1570 0x7fbb0409c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:09.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb02a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040b1570 0x7fbb0409c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40428/0 (socket says 172.21.15.46:40428) 2024-03-29T08:23:09.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb02a00640 1 -- 172.21.15.46:0/3940899181 learned_addr learned my addr 172.21.15.46:0/3940899181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:09.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbafbe00640 1 --2- 172.21.15.46:0/3940899181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb040ce680 0x7fbb0409d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:09.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb03400640 1 --2- 172.21.15.46:0/3940899181 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb040cefa0 0x7fbb04152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:09.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb02a00640 1 -- 172.21.15.46:0/3940899181 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb040cefa0 msgr2=0x7fbb04152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:09.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb02a00640 1 --2- 172.21.15.46:0/3940899181 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb040cefa0 0x7fbb04152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:09.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb02a00640 1 -- 172.21.15.46:0/3940899181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb040ce680 msgr2=0x7fbb0409d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:09.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb02a00640 1 --2- 172.21.15.46:0/3940899181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb040ce680 0x7fbb0409d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:09.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb02a00640 1 -- 172.21.15.46:0/3940899181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf4009420 con 0x7fbb040b1570 2024-03-29T08:23:09.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbafbe00640 1 --2- 172.21.15.46:0/3940899181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb040ce680 0x7fbb0409d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:09.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.870+0000 7fbb02a00640 1 --2- 172.21.15.46:0/3940899181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040b1570 0x7fbb0409c190 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40ae040 tx=0x7fbaf401ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:09.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.874+0000 7fbafb400640 1 -- 172.21.15.46:0/3940899181 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf40b7650 con 0x7fbb040b1570 2024-03-29T08:23:09.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.874+0000 7fbafb400640 1 -- 172.21.15.46:0/3940899181 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaf40b77b0 con 0x7fbb040b1570 2024-03-29T08:23:09.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.874+0000 7fbafb400640 1 -- 172.21.15.46:0/3940899181 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf40b7b60 con 0x7fbb040b1570 2024-03-29T08:23:09.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.874+0000 7fbb03e00640 1 -- 172.21.15.46:0/3940899181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb0409cf10 con 0x7fbb040b1570 2024-03-29T08:23:09.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.874+0000 7fbb03e00640 1 -- 172.21.15.46:0/3940899181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0409d4d0 con 0x7fbb040b1570 2024-03-29T08:23:09.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.874+0000 7fbb03e00640 1 -- 172.21.15.46:0/3940899181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb0404e040 con 0x7fbb040b1570 2024-03-29T08:23:09.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.874+0000 7fbafb400640 1 -- 172.21.15.46:0/3940899181 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbaf40a4d50 con 0x7fbb040b1570 2024-03-29T08:23:09.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.874+0000 7fbafb400640 1 --2- 172.21.15.46:0/3940899181 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbae0066690 0x7fbae0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:09.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.874+0000 7fbafb400640 1 -- 172.21.15.46:0/3940899181 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fbaf411ea10 con 0x7fbb040b1570 2024-03-29T08:23:09.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.874+0000 7fbafbe00640 1 --2- 172.21.15.46:0/3940899181 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbae0066690 0x7fbae0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:09.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.874+0000 7fbafbe00640 1 --2- 172.21.15.46:0/3940899181 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbae0066690 0x7fbae0068b50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbaec007fd0 tx=0x7fbaec007c20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:09.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:09.878+0000 7fbafb400640 1 -- 172.21.15.46:0/3940899181 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbaf40ebd70 con 0x7fbb040b1570 2024-03-29T08:23:10.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.042+0000 7fbb03e00640 1 -- 172.21.15.46:0/3940899181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb040cd020 con 0x7fbb040b1570 2024-03-29T08:23:10.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.042+0000 7fbafb400640 1 -- 172.21.15.46:0/3940899181 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16213 (secure 0 0 0) 0x7fbaf40b0090 con 0x7fbb040b1570 2024-03-29T08:23:10.047 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:23:10.047 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:23:02.390005+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:22:42.147364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-29T08:22:42.602168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1f","mappings":[{"from":2,"to":7}]},{"pgid":"11.27","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/9617787":"2024-03-30T08:23:02.389981+0000","172.21.15.46:6834/9617787":"2024-03-30T08:23:02.389981+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:23:10.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.046+0000 7fbb03e00640 1 -- 172.21.15.46:0/3940899181 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbae0066690 msgr2=0x7fbae0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.046+0000 7fbb03e00640 1 --2- 172.21.15.46:0/3940899181 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbae0066690 0x7fbae0068b50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbaec007fd0 tx=0x7fbaec007c20 comp rx=0 tx=0).stop 2024-03-29T08:23:10.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.046+0000 7fbb03e00640 1 -- 172.21.15.46:0/3940899181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040b1570 msgr2=0x7fbb0409c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.046+0000 7fbb03e00640 1 --2- 172.21.15.46:0/3940899181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040b1570 0x7fbb0409c190 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fbaf40ae040 tx=0x7fbaf401ac10 comp rx=0 tx=0).stop 2024-03-29T08:23:10.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.046+0000 7fbb03e00640 1 -- 172.21.15.46:0/3940899181 shutdown_connections 2024-03-29T08:23:10.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.046+0000 7fbb03e00640 1 --2- 172.21.15.46:0/3940899181 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbae0066690 0x7fbae0068b50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.046+0000 7fbb03e00640 1 --2- 172.21.15.46:0/3940899181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb040ce680 0x7fbb0409d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.046+0000 7fbb03e00640 1 --2- 172.21.15.46:0/3940899181 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb040cefa0 0x7fbb04152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.046+0000 7fbb03e00640 1 --2- 172.21.15.46:0/3940899181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb040b1570 0x7fbb0409c190 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.046+0000 7fbb03e00640 1 -- 172.21.15.46:0/3940899181 >> 172.21.15.46:0/3940899181 conn(0x7fbb040b2010 msgr2=0x7fbb04098c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:10.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.046+0000 7fbb03e00640 1 -- 172.21.15.46:0/3940899181 shutdown_connections 2024-03-29T08:23:10.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.046+0000 7fbb03e00640 1 -- 172.21.15.46:0/3940899181 wait complete. 2024-03-29T08:23:10.068 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:22:42.147364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:22:42.602168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:23:10.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:23:10.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.210+0000 7f0e37800640 1 Processor -- start 2024-03-29T08:23:10.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.210+0000 7f0e37800640 1 -- start start 2024-03-29T08:23:10.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.210+0000 7f0e37800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e3015dbb0 0x7f0e3006ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.210+0000 7f0e37800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0e300b3f20 0x7f0e3006b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.210+0000 7f0e37800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e30157490 con 0x7f0e300b1d30 2024-03-29T08:23:10.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.210+0000 7f0e37800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e300b2c90 con 0x7f0e300b3f20 2024-03-29T08:23:10.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.210+0000 7f0e37800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e300b29b0 con 0x7f0e3015dbb0 2024-03-29T08:23:10.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.210+0000 7f0e27e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0e300b3f20 0x7f0e3006b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.210+0000 7f0e27e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0e300b3f20 0x7f0e3006b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40432/0 (socket says 172.21.15.46:40432) 2024-03-29T08:23:10.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.210+0000 7f0e27e00640 1 -- 172.21.15.46:0/1065110964 learned_addr learned my addr 172.21.15.46:0/1065110964 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:10.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.210+0000 7f0e34e00640 1 --2- 172.21.15.46:0/1065110964 >> v2:172.21.15.73:3300/0 conn(0x7f0e3015dbb0 0x7f0e3006ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.210+0000 7f0e27e00640 1 -- 172.21.15.46:0/1065110964 >> v2:172.21.15.73:3300/0 conn(0x7f0e3015dbb0 msgr2=0x7f0e3006ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:10.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.210+0000 7f0e27e00640 1 --2- 172.21.15.46:0/1065110964 >> v2:172.21.15.73:3300/0 conn(0x7f0e3015dbb0 0x7f0e3006ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.210+0000 7f0e27e00640 1 -- 172.21.15.46:0/1065110964 >> v1:172.21.15.73:6789/0 conn(0x7f0e300b1d30 legacy=0x7f0e300b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:10.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.210+0000 7f0e27e00640 1 -- 172.21.15.46:0/1065110964 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e300b3440 con 0x7f0e300b3f20 2024-03-29T08:23:10.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.210+0000 7f0e34e00640 1 --2- 172.21.15.46:0/1065110964 >> v2:172.21.15.73:3300/0 conn(0x7f0e3015dbb0 0x7f0e3006ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:23:10.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.214+0000 7f0e27e00640 1 --2- 172.21.15.46:0/1065110964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e300b3f20 0x7f0e3006b2e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f0e30066210 tx=0x7f0e1801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0ed11405477554 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.214+0000 7f0e2fe00640 1 -- 172.21.15.46:0/1065110964 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e180ab650 con 0x7f0e300b3f20 2024-03-29T08:23:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.214+0000 7f0e2fe00640 1 -- 172.21.15.46:0/1065110964 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e180ab7b0 con 0x7f0e300b3f20 2024-03-29T08:23:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.214+0000 7f0e2fe00640 1 -- 172.21.15.46:0/1065110964 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e180abb40 con 0x7f0e300b3f20 2024-03-29T08:23:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.214+0000 7f0e37800640 1 -- 172.21.15.46:0/1065110964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e300b3f20 msgr2=0x7f0e3006b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.214+0000 7f0e37800640 1 --2- 172.21.15.46:0/1065110964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e300b3f20 0x7f0e3006b2e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f0e30066210 tx=0x7f0e1801c450 comp rx=0 tx=0).stop 2024-03-29T08:23:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.214+0000 7f0e37800640 1 -- 172.21.15.46:0/1065110964 shutdown_connections 2024-03-29T08:23:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.214+0000 7f0e37800640 1 --2- 172.21.15.46:0/1065110964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e300b3f20 0x7f0e3006b2e0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.214+0000 7f0e37800640 1 --2- 172.21.15.46:0/1065110964 >> v2:172.21.15.73:3300/0 conn(0x7f0e3015dbb0 0x7f0e3006ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.214+0000 7f0e37800640 1 -- 172.21.15.46:0/1065110964 >> 172.21.15.46:0/1065110964 conn(0x7f0e30070ad0 msgr2=0x7f0e30070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:10.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.214+0000 7f0e37800640 1 -- 172.21.15.46:0/1065110964 shutdown_connections 2024-03-29T08:23:10.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.214+0000 7f0e37800640 1 -- 172.21.15.46:0/1065110964 wait complete. 2024-03-29T08:23:10.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.214+0000 7f0e37800640 1 Processor -- start 2024-03-29T08:23:10.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.214+0000 7f0e37800640 1 -- start start 2024-03-29T08:23:10.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e37800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e300b1d30 0x7f0e30152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e37800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e300b3f20 0x7f0e3014cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e37800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e3015dbb0 0x7f0e3006d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e37800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30152400 con 0x7f0e300b1d30 2024-03-29T08:23:10.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e37800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3014d880 con 0x7f0e3015dbb0 2024-03-29T08:23:10.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e37800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e301517f0 con 0x7f0e300b3f20 2024-03-29T08:23:10.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e34e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e300b1d30 0x7f0e30152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e34e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e300b1d30 0x7f0e30152890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40444/0 (socket says 172.21.15.46:40444) 2024-03-29T08:23:10.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e34e00640 1 -- 172.21.15.46:0/1846549190 learned_addr learned my addr 172.21.15.46:0/1846549190 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:10.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e35800640 1 --2- 172.21.15.46:0/1846549190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e3015dbb0 0x7f0e3006d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e27e00640 1 --2- 172.21.15.46:0/1846549190 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e300b3f20 0x7f0e3014cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e34e00640 1 -- 172.21.15.46:0/1846549190 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e300b3f20 msgr2=0x7f0e3014cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e34e00640 1 --2- 172.21.15.46:0/1846549190 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e300b3f20 0x7f0e3014cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e34e00640 1 -- 172.21.15.46:0/1846549190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e3015dbb0 msgr2=0x7f0e3006d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e34e00640 1 --2- 172.21.15.46:0/1846549190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e3015dbb0 0x7f0e3006d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e34e00640 1 -- 172.21.15.46:0/1846549190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e180094d0 con 0x7f0e300b1d30 2024-03-29T08:23:10.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e27e00640 1 --2- 172.21.15.46:0/1846549190 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e300b3f20 0x7f0e3014cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:23:10.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e35800640 1 --2- 172.21.15.46:0/1846549190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e3015dbb0 0x7f0e3006d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:23:10.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e34e00640 1 --2- 172.21.15.46:0/1846549190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e300b1d30 0x7f0e30152890 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000c7f0 tx=0x7f0e2000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:10.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e2d600640 1 -- 172.21.15.46:0/1846549190 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e20007820 con 0x7f0e300b1d30 2024-03-29T08:23:10.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e2d600640 1 -- 172.21.15.46:0/1846549190 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e20007980 con 0x7f0e300b1d30 2024-03-29T08:23:10.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e2d600640 1 -- 172.21.15.46:0/1846549190 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e20007d30 con 0x7f0e300b1d30 2024-03-29T08:23:10.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e37800640 1 -- 172.21.15.46:0/1846549190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e30151ad0 con 0x7f0e300b1d30 2024-03-29T08:23:10.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e37800640 1 -- 172.21.15.46:0/1846549190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e30152090 con 0x7f0e300b1d30 2024-03-29T08:23:10.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e37800640 1 -- 172.21.15.46:0/1846549190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3009d100 con 0x7f0e300b1d30 2024-03-29T08:23:10.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.218+0000 7f0e2d600640 1 -- 172.21.15.46:0/1846549190 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0e200113f0 con 0x7f0e300b1d30 2024-03-29T08:23:10.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.222+0000 7f0e2d600640 1 --2- 172.21.15.46:0/1846549190 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e0c0666c0 0x7f0e0c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.222+0000 7f0e2d600640 1 -- 172.21.15.46:0/1846549190 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f0e20084580 con 0x7f0e300b1d30 2024-03-29T08:23:10.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.222+0000 7f0e27e00640 1 --2- 172.21.15.46:0/1846549190 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e0c0666c0 0x7f0e0c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.222+0000 7f0e27e00640 1 --2- 172.21.15.46:0/1846549190 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e0c0666c0 0x7f0e0c068b80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0e180af000 tx=0x7f0e18008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:10.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.226+0000 7f0e2d600640 1 -- 172.21.15.46:0/1846549190 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0e20051850 con 0x7f0e300b1d30 2024-03-29T08:23:10.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.386+0000 7f0e37800640 1 -- 172.21.15.46:0/1846549190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e301676e0 con 0x7f0e300b1d30 2024-03-29T08:23:10.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.386+0000 7f0e2d600640 1 -- 172.21.15.46:0/1846549190 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e20054e70 con 0x7f0e300b1d30 2024-03-29T08:23:10.390 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:23:10.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.390+0000 7f0e37800640 1 -- 172.21.15.46:0/1846549190 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e0c0666c0 msgr2=0x7f0e0c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.390+0000 7f0e37800640 1 --2- 172.21.15.46:0/1846549190 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e0c0666c0 0x7f0e0c068b80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0e180af000 tx=0x7f0e18008040 comp rx=0 tx=0).stop 2024-03-29T08:23:10.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.390+0000 7f0e37800640 1 -- 172.21.15.46:0/1846549190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e300b1d30 msgr2=0x7f0e30152890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.390+0000 7f0e37800640 1 --2- 172.21.15.46:0/1846549190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e300b1d30 0x7f0e30152890 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000c7f0 tx=0x7f0e2000ccb0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.390+0000 7f0e37800640 1 -- 172.21.15.46:0/1846549190 shutdown_connections 2024-03-29T08:23:10.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.390+0000 7f0e37800640 1 --2- 172.21.15.46:0/1846549190 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e0c0666c0 0x7f0e0c068b80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.390+0000 7f0e37800640 1 --2- 172.21.15.46:0/1846549190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e3015dbb0 0x7f0e3006d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.390+0000 7f0e37800640 1 --2- 172.21.15.46:0/1846549190 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e300b3f20 0x7f0e3014cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.390+0000 7f0e37800640 1 --2- 172.21.15.46:0/1846549190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e300b1d30 0x7f0e30152890 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.390+0000 7f0e37800640 1 -- 172.21.15.46:0/1846549190 >> 172.21.15.46:0/1846549190 conn(0x7f0e30070ad0 msgr2=0x7f0e300b01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:10.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.390+0000 7f0e37800640 1 -- 172.21.15.46:0/1846549190 shutdown_connections 2024-03-29T08:23:10.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.390+0000 7f0e37800640 1 -- 172.21.15.46:0/1846549190 wait complete. 2024-03-29T08:23:10.412 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:23:10.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.550+0000 7f2d6b600640 1 Processor -- start 2024-03-29T08:23:10.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.550+0000 7f2d6b600640 1 -- start start 2024-03-29T08:23:10.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d6b600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2d6415efe0 0x7f2d640b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d6b600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d640b2360 con 0x7f2d640cbea0 2024-03-29T08:23:10.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d6b600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6415e500 con 0x7f2d64143eb0 2024-03-29T08:23:10.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d6b600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6415e7e0 con 0x7f2d6415efe0 2024-03-29T08:23:10.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d5be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2d640cbea0 0x7f2d640b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34054/0 (socket says 172.21.15.46:34054) 2024-03-29T08:23:10.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d5be00640 1 -- 172.21.15.46:0/981352007 learned_addr learned my addr 172.21.15.46:0/981352007 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:10.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d69600640 1 --2- 172.21.15.46:0/981352007 >> v2:172.21.15.73:3300/0 conn(0x7f2d6415efe0 0x7f2d640b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d63e00640 1 -- 172.21.15.46:0/981352007 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406634590 0 0) 0x7f2d640b2360 con 0x7f2d640cbea0 2024-03-29T08:23:10.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d63e00640 1 -- 172.21.15.46:0/981352007 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d48003530 con 0x7f2d640cbea0 2024-03-29T08:23:10.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d63e00640 1 -- 172.21.15.46:0/981352007 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1550117751 0 0) 0x7f2d48003530 con 0x7f2d640cbea0 2024-03-29T08:23:10.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d63e00640 1 -- 172.21.15.46:0/981352007 >> v2:172.21.15.73:3300/0 conn(0x7f2d6415efe0 msgr2=0x7f2d640b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:10.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d63e00640 1 --2- 172.21.15.46:0/981352007 >> v2:172.21.15.73:3300/0 conn(0x7f2d6415efe0 0x7f2d640b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d63e00640 1 -- 172.21.15.46:0/981352007 >> v1:172.21.15.73:6789/0 conn(0x7f2d64143eb0 legacy=0x7f2d640cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d63e00640 1 -- 172.21.15.46:0/981352007 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6415eac0 con 0x7f2d640cbea0 2024-03-29T08:23:10.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d63e00640 1 -- 172.21.15.46:0/981352007 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f2d4c001680 con 0x7f2d640cbea0 2024-03-29T08:23:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d63e00640 1 -- 172.21.15.46:0/981352007 >> v1:172.21.15.46:6789/0 conn(0x7f2d640cbea0 legacy=0x7f2d640b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d63e00640 1 --2- 172.21.15.46:0/981352007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d48003980 0x7f2d48003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d63e00640 1 --2- 172.21.15.46:0/981352007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d48004380 0x7f2d48004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d63e00640 1 --2- 172.21.15.46:0/981352007 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d48005050 0x7f2d4800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d63e00640 1 -- 172.21.15.46:0/981352007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6415e7e0 con 0x7f2d48004380 2024-03-29T08:23:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d63e00640 1 -- 172.21.15.46:0/981352007 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4800bfb0 con 0x7f2d48003980 2024-03-29T08:23:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d63e00640 1 -- 172.21.15.46:0/981352007 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4800c170 con 0x7f2d48005050 2024-03-29T08:23:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d5be00640 1 --2- 172.21.15.46:0/981352007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d48004380 0x7f2d48004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d68c00640 1 --2- 172.21.15.46:0/981352007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d48003980 0x7f2d48003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d69600640 1 --2- 172.21.15.46:0/981352007 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d48005050 0x7f2d4800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d5be00640 1 -- 172.21.15.46:0/981352007 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d48005050 msgr2=0x7f2d4800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d5be00640 1 --2- 172.21.15.46:0/981352007 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d48005050 0x7f2d4800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d5be00640 1 -- 172.21.15.46:0/981352007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d48003980 msgr2=0x7f2d48003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:10.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d5be00640 1 --2- 172.21.15.46:0/981352007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d48003980 0x7f2d48003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d5be00640 1 -- 172.21.15.46:0/981352007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d48003530 con 0x7f2d48004380 2024-03-29T08:23:10.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.554+0000 7f2d69600640 1 -- 172.21.15.46:0/981352007 reap_dead start 2024-03-29T08:23:10.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.558+0000 7f2d5be00640 1 --2- 172.21.15.46:0/981352007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d48004380 0x7f2d48004850 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00c190 tx=0x7f2d4c01cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd346b8a04318328 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:10.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.558+0000 7f2d63e00640 1 -- 172.21.15.46:0/981352007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c005ae0 con 0x7f2d48004380 2024-03-29T08:23:10.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.558+0000 7f2d63e00640 1 -- 172.21.15.46:0/981352007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4c005c60 con 0x7f2d48004380 2024-03-29T08:23:10.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.558+0000 7f2d63e00640 1 -- 172.21.15.46:0/981352007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c0a5aa0 con 0x7f2d48004380 2024-03-29T08:23:10.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.558+0000 7f2d6b600640 1 -- 172.21.15.46:0/981352007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d48004380 msgr2=0x7f2d48004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.558+0000 7f2d6b600640 1 --2- 172.21.15.46:0/981352007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d48004380 0x7f2d48004850 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c00c190 tx=0x7f2d4c01cb60 comp rx=0 tx=0).stop 2024-03-29T08:23:10.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.558+0000 7f2d6b600640 1 -- 172.21.15.46:0/981352007 shutdown_connections 2024-03-29T08:23:10.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.558+0000 7f2d6b600640 1 --2- 172.21.15.46:0/981352007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d48004380 0x7f2d48004850 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.558+0000 7f2d6b600640 1 -- 172.21.15.46:0/981352007 >> 172.21.15.46:0/981352007 conn(0x7f2d6406a390 msgr2=0x7f2d640cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:10.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.558+0000 7f2d6b600640 1 -- 172.21.15.46:0/981352007 shutdown_connections 2024-03-29T08:23:10.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.558+0000 7f2d6b600640 1 -- 172.21.15.46:0/981352007 wait complete. 2024-03-29T08:23:10.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.558+0000 7f2d6b600640 1 Processor -- start 2024-03-29T08:23:10.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.558+0000 7f2d6b600640 1 -- start start 2024-03-29T08:23:10.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d6b600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d48004380 0x7f2d640a9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d6b600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d641f3740 0x7f2d64141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d6b600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d640ab7a0 0x7f2d6413ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d6b600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d641f4020 con 0x7f2d641f3740 2024-03-29T08:23:10.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d6b600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d641f4300 con 0x7f2d48004380 2024-03-29T08:23:10.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d6b600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d641f45e0 con 0x7f2d640ab7a0 2024-03-29T08:23:10.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d641f3740 0x7f2d64141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d641f3740 0x7f2d64141ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40462/0 (socket says 172.21.15.46:40462) 2024-03-29T08:23:10.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d5be00640 1 -- 172.21.15.46:0/1937772360 learned_addr learned my addr 172.21.15.46:0/1937772360 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:10.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d68c00640 1 --2- 172.21.15.46:0/1937772360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d48004380 0x7f2d640a9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d69600640 1 --2- 172.21.15.46:0/1937772360 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d640ab7a0 0x7f2d6413ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d5be00640 1 -- 172.21.15.46:0/1937772360 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d640ab7a0 msgr2=0x7f2d6413ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d5be00640 1 --2- 172.21.15.46:0/1937772360 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d640ab7a0 0x7f2d6413ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d5be00640 1 -- 172.21.15.46:0/1937772360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d48004380 msgr2=0x7f2d640a9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d5be00640 1 --2- 172.21.15.46:0/1937772360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d48004380 0x7f2d640a9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d5be00640 1 -- 172.21.15.46:0/1937772360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d640a7770 con 0x7f2d641f3740 2024-03-29T08:23:10.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d68c00640 1 --2- 172.21.15.46:0/1937772360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d48004380 0x7f2d640a9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:23:10.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d69600640 1 --2- 172.21.15.46:0/1937772360 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d640ab7a0 0x7f2d6413ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:23:10.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d5be00640 1 --2- 172.21.15.46:0/1937772360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d641f3740 0x7f2d64141ed0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c01c990 tx=0x7f2d48005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:10.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d61600640 1 -- 172.21.15.46:0/1937772360 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c0b4070 con 0x7f2d641f3740 2024-03-29T08:23:10.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d61600640 1 -- 172.21.15.46:0/1937772360 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4c0a5a70 con 0x7f2d641f3740 2024-03-29T08:23:10.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d61600640 1 -- 172.21.15.46:0/1937772360 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c0a5de0 con 0x7f2d641f3740 2024-03-29T08:23:10.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d6b600640 1 -- 172.21.15.46:0/1937772360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d640a7a50 con 0x7f2d641f3740 2024-03-29T08:23:10.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d6b600640 1 -- 172.21.15.46:0/1937772360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d641425e0 con 0x7f2d641f3740 2024-03-29T08:23:10.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d6b600640 1 -- 172.21.15.46:0/1937772360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d640a7be0 con 0x7f2d641f3740 2024-03-29T08:23:10.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.562+0000 7f2d61600640 1 -- 172.21.15.46:0/1937772360 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2d4c007070 con 0x7f2d641f3740 2024-03-29T08:23:10.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.566+0000 7f2d61600640 1 --2- 172.21.15.46:0/1937772360 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2d400666c0 0x7f2d40068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.566+0000 7f2d61600640 1 -- 172.21.15.46:0/1937772360 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2d4c11e380 con 0x7f2d641f3740 2024-03-29T08:23:10.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.566+0000 7f2d68c00640 1 --2- 172.21.15.46:0/1937772360 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2d400666c0 0x7f2d40068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.566+0000 7f2d68c00640 1 --2- 172.21.15.46:0/1937772360 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2d400666c0 0x7f2d40068b80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2d54005630 tx=0x7f2d54007cd0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:10.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.570+0000 7f2d61600640 1 -- 172.21.15.46:0/1937772360 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2d4c0eb650 con 0x7f2d641f3740 2024-03-29T08:23:10.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.726+0000 7f2d6b600640 1 -- 172.21.15.46:0/1937772360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d64071a70 con 0x7f2d641f3740 2024-03-29T08:23:10.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.730+0000 7f2d61600640 1 -- 172.21.15.46:0/1937772360 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2d4c0eec70 con 0x7f2d641f3740 2024-03-29T08:23:10.732 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:23:10.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.730+0000 7f2d6b600640 1 -- 172.21.15.46:0/1937772360 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2d400666c0 msgr2=0x7f2d40068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.730+0000 7f2d6b600640 1 --2- 172.21.15.46:0/1937772360 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2d400666c0 0x7f2d40068b80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2d54005630 tx=0x7f2d54007cd0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.730+0000 7f2d6b600640 1 -- 172.21.15.46:0/1937772360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d641f3740 msgr2=0x7f2d64141ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.730+0000 7f2d6b600640 1 --2- 172.21.15.46:0/1937772360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d641f3740 0x7f2d64141ed0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c01c990 tx=0x7f2d48005800 comp rx=0 tx=0).stop 2024-03-29T08:23:10.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.730+0000 7f2d6b600640 1 -- 172.21.15.46:0/1937772360 shutdown_connections 2024-03-29T08:23:10.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.730+0000 7f2d6b600640 1 --2- 172.21.15.46:0/1937772360 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2d400666c0 0x7f2d40068b80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.730+0000 7f2d6b600640 1 --2- 172.21.15.46:0/1937772360 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2d640ab7a0 0x7f2d6413ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.730+0000 7f2d6b600640 1 --2- 172.21.15.46:0/1937772360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d641f3740 0x7f2d64141ed0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.730+0000 7f2d6b600640 1 --2- 172.21.15.46:0/1937772360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d48004380 0x7f2d640a9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.730+0000 7f2d6b600640 1 -- 172.21.15.46:0/1937772360 >> 172.21.15.46:0/1937772360 conn(0x7f2d6406a390 msgr2=0x7f2d640707f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:10.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.730+0000 7f2d6b600640 1 -- 172.21.15.46:0/1937772360 shutdown_connections 2024-03-29T08:23:10.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.730+0000 7f2d6b600640 1 -- 172.21.15.46:0/1937772360 wait complete. 2024-03-29T08:23:10.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:23:10.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.894+0000 7f9c42a00640 1 Processor -- start 2024-03-29T08:23:10.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.894+0000 7f9c42a00640 1 -- start start 2024-03-29T08:23:10.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.894+0000 7f9c42a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9c3c143eb0 0x7f9c3c0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.894+0000 7f9c42a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9c3c15efe0 0x7f9c3c0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.894+0000 7f9c42a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3c0b2360 con 0x7f9c3c0b1630 2024-03-29T08:23:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.894+0000 7f9c42a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c15e500 con 0x7f9c3c143eb0 2024-03-29T08:23:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.894+0000 7f9c42a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c15e7e0 con 0x7f9c3c15efe0 2024-03-29T08:23:10.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.894+0000 7f9c40a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9c3c0b1630 0x7f9c3c0b1a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34056/0 (socket says 172.21.15.46:34056) 2024-03-29T08:23:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.894+0000 7f9c40a00640 1 -- 172.21.15.46:0/2647021493 learned_addr learned my addr 172.21.15.46:0/2647021493 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.894+0000 7f9c3be00640 1 --2- 172.21.15.46:0/2647021493 >> v2:172.21.15.46:3300/0 conn(0x7f9c3c143eb0 0x7f9c3c0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.894+0000 7f9c3b400640 1 --2- 172.21.15.46:0/2647021493 >> v2:172.21.15.73:3300/0 conn(0x7f9c3c15efe0 0x7f9c3c0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.894+0000 7f9c3be00640 1 -- 172.21.15.46:0/2647021493 >> v2:172.21.15.73:3300/0 conn(0x7f9c3c15efe0 msgr2=0x7f9c3c0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.894+0000 7f9c3be00640 1 --2- 172.21.15.46:0/2647021493 >> v2:172.21.15.73:3300/0 conn(0x7f9c3c15efe0 0x7f9c3c0cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.894+0000 7f9c3be00640 1 -- 172.21.15.46:0/2647021493 >> v1:172.21.15.46:6789/0 conn(0x7f9c3c0b1630 legacy=0x7f9c3c0b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.894+0000 7f9c3aa00640 1 -- 172.21.15.46:0/2647021493 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217969785 0 0) 0x7f9c3c0b2360 con 0x7f9c3c0b1630 2024-03-29T08:23:10.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.894+0000 7f9c3be00640 1 -- 172.21.15.46:0/2647021493 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c15eac0 con 0x7f9c3c143eb0 2024-03-29T08:23:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.898+0000 7f9c3be00640 1 --2- 172.21.15.46:0/2647021493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c143eb0 0x7f9c3c0cb790 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c158930 tx=0x7f9c2c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277d232bf32c5a86 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.898+0000 7f9c3aa00640 1 -- 172.21.15.46:0/2647021493 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0a8050 con 0x7f9c3c143eb0 2024-03-29T08:23:10.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.898+0000 7f9c3aa00640 1 -- 172.21.15.46:0/2647021493 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c0ac5f0 con 0x7f9c3c143eb0 2024-03-29T08:23:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.898+0000 7f9c3aa00640 1 -- 172.21.15.46:0/2647021493 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0ac980 con 0x7f9c3c143eb0 2024-03-29T08:23:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.898+0000 7f9c42a00640 1 -- 172.21.15.46:0/2647021493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c143eb0 msgr2=0x7f9c3c0cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.898+0000 7f9c42a00640 1 --2- 172.21.15.46:0/2647021493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c143eb0 0x7f9c3c0cb790 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c158930 tx=0x7f9c2c01c480 comp rx=0 tx=0).stop 2024-03-29T08:23:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.898+0000 7f9c42a00640 1 -- 172.21.15.46:0/2647021493 shutdown_connections 2024-03-29T08:23:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.898+0000 7f9c42a00640 1 --2- 172.21.15.46:0/2647021493 >> v2:172.21.15.73:3300/0 conn(0x7f9c3c15efe0 0x7f9c3c0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.898+0000 7f9c42a00640 1 --2- 172.21.15.46:0/2647021493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c143eb0 0x7f9c3c0cb790 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.898+0000 7f9c42a00640 1 -- 172.21.15.46:0/2647021493 >> 172.21.15.46:0/2647021493 conn(0x7f9c3c06a390 msgr2=0x7f9c3c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.898+0000 7f9c42a00640 1 -- 172.21.15.46:0/2647021493 shutdown_connections 2024-03-29T08:23:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.898+0000 7f9c42a00640 1 -- 172.21.15.46:0/2647021493 wait complete. 2024-03-29T08:23:10.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c42a00640 1 Processor -- start 2024-03-29T08:23:10.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c42a00640 1 -- start start 2024-03-29T08:23:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c42a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c0b1630 0x7f9c3c0c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c42a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c3c143eb0 0x7f9c3c13e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c42a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c3c15efe0 0x7f9c3c140340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c42a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c13be10 con 0x7f9c3c0b1630 2024-03-29T08:23:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c42a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0ca460 con 0x7f9c3c15efe0 2024-03-29T08:23:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c42a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c13af20 con 0x7f9c3c143eb0 2024-03-29T08:23:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c3be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c0b1630 0x7f9c3c0c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c3be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c0b1630 0x7f9c3c0c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40480/0 (socket says 172.21.15.46:40480) 2024-03-29T08:23:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c3be00640 1 -- 172.21.15.46:0/721137932 learned_addr learned my addr 172.21.15.46:0/721137932 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c40a00640 1 --2- 172.21.15.46:0/721137932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c3c15efe0 0x7f9c3c140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c3b400640 1 --2- 172.21.15.46:0/721137932 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c3c143eb0 0x7f9c3c13e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c3be00640 1 -- 172.21.15.46:0/721137932 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c3c143eb0 msgr2=0x7f9c3c13e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c3be00640 1 --2- 172.21.15.46:0/721137932 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c3c143eb0 0x7f9c3c13e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c3be00640 1 -- 172.21.15.46:0/721137932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c3c15efe0 msgr2=0x7f9c3c140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c3be00640 1 --2- 172.21.15.46:0/721137932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c3c15efe0 0x7f9c3c140340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c3be00640 1 -- 172.21.15.46:0/721137932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c009420 con 0x7f9c3c0b1630 2024-03-29T08:23:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c40a00640 1 --2- 172.21.15.46:0/721137932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c3c15efe0 0x7f9c3c140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:23:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c3b400640 1 --2- 172.21.15.46:0/721137932 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c3c143eb0 0x7f9c3c13e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:23:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c3be00640 1 --2- 172.21.15.46:0/721137932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c0b1630 0x7f9c3c0c9f20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c01c960 tx=0x7f9c2c019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c33e00640 1 -- 172.21.15.46:0/721137932 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0a8040 con 0x7f9c3c0b1630 2024-03-29T08:23:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c33e00640 1 -- 172.21.15.46:0/721137932 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c0b5620 con 0x7f9c3c0b1630 2024-03-29T08:23:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c33e00640 1 -- 172.21.15.46:0/721137932 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0b59b0 con 0x7f9c3c0b1630 2024-03-29T08:23:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c42a00640 1 -- 172.21.15.46:0/721137932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c13b200 con 0x7f9c3c0b1630 2024-03-29T08:23:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c42a00640 1 -- 172.21.15.46:0/721137932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c13b7c0 con 0x7f9c3c0b1630 2024-03-29T08:23:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.902+0000 7f9c42a00640 1 -- 172.21.15.46:0/721137932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3c0b0fc0 con 0x7f9c3c0b1630 2024-03-29T08:23:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.906+0000 7f9c33e00640 1 -- 172.21.15.46:0/721137932 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9c2c0b3040 con 0x7f9c3c0b1630 2024-03-29T08:23:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.906+0000 7f9c33e00640 1 --2- 172.21.15.46:0/721137932 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c1c0666c0 0x7f9c1c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.906+0000 7f9c33e00640 1 -- 172.21.15.46:0/721137932 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9c2c11e2a0 con 0x7f9c3c0b1630 2024-03-29T08:23:10.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.906+0000 7f9c3b400640 1 --2- 172.21.15.46:0/721137932 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c1c0666c0 0x7f9c1c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.906+0000 7f9c3b400640 1 --2- 172.21.15.46:0/721137932 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c1c0666c0 0x7f9c1c068b80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0b39d0 tx=0x7f9c28008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:10.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:10.910+0000 7f9c33e00640 1 -- 172.21.15.46:0/721137932 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9c2c0eb570 con 0x7f9c3c0b1630 2024-03-29T08:23:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.070+0000 7f9c42a00640 1 -- 172.21.15.46:0/721137932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9c3c071a70 con 0x7f9c3c0b1630 2024-03-29T08:23:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.070+0000 7f9c33e00640 1 -- 172.21.15.46:0/721137932 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9c2c0eeb90 con 0x7f9c3c0b1630 2024-03-29T08:23:11.075 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:23:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.074+0000 7f9c42a00640 1 -- 172.21.15.46:0/721137932 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c1c0666c0 msgr2=0x7f9c1c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.074+0000 7f9c42a00640 1 --2- 172.21.15.46:0/721137932 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c1c0666c0 0x7f9c1c068b80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c0b39d0 tx=0x7f9c28008040 comp rx=0 tx=0).stop 2024-03-29T08:23:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.074+0000 7f9c42a00640 1 -- 172.21.15.46:0/721137932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c0b1630 msgr2=0x7f9c3c0c9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.074+0000 7f9c42a00640 1 --2- 172.21.15.46:0/721137932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c0b1630 0x7f9c3c0c9f20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c01c960 tx=0x7f9c2c019220 comp rx=0 tx=0).stop 2024-03-29T08:23:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.074+0000 7f9c42a00640 1 -- 172.21.15.46:0/721137932 shutdown_connections 2024-03-29T08:23:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.074+0000 7f9c42a00640 1 --2- 172.21.15.46:0/721137932 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c1c0666c0 0x7f9c1c068b80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.074+0000 7f9c42a00640 1 --2- 172.21.15.46:0/721137932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c3c15efe0 0x7f9c3c140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.074+0000 7f9c42a00640 1 --2- 172.21.15.46:0/721137932 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c3c143eb0 0x7f9c3c13e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.074+0000 7f9c42a00640 1 --2- 172.21.15.46:0/721137932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c3c0b1630 0x7f9c3c0c9f20 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.074+0000 7f9c42a00640 1 -- 172.21.15.46:0/721137932 >> 172.21.15.46:0/721137932 conn(0x7f9c3c06a390 msgr2=0x7f9c3c0b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.074+0000 7f9c42a00640 1 -- 172.21.15.46:0/721137932 shutdown_connections 2024-03-29T08:23:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.074+0000 7f9c42a00640 1 -- 172.21.15.46:0/721137932 wait complete. 2024-03-29T08:23:11.096 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:23:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.234+0000 7f4333600640 1 Processor -- start 2024-03-29T08:23:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.234+0000 7f4333600640 1 -- start start 2024-03-29T08:23:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.234+0000 7f4333600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f432c0af530 0x7f432c0af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.234+0000 7f4333600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f432c0702a0 0x7f432c070730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.234+0000 7f4333600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432c06c390 con 0x7f432c158300 2024-03-29T08:23:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.234+0000 7f4333600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f432c070c70 con 0x7f432c0702a0 2024-03-29T08:23:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.234+0000 7f4333600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0afff0 con 0x7f432c0af530 2024-03-29T08:23:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.234+0000 7f4330c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f432c158300 0x7f432c06d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34058/0 (socket says 172.21.15.46:34058) 2024-03-29T08:23:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.234+0000 7f4330c00640 1 -- 172.21.15.46:0/3915260053 learned_addr learned my addr 172.21.15.46:0/3915260053 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.234+0000 7f4323e00640 1 --2- 172.21.15.46:0/3915260053 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f432c0af530 0x7f432c0af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.234+0000 7f4331600640 1 --2- 172.21.15.46:0/3915260053 >> v2:172.21.15.73:3300/0 conn(0x7f432c0702a0 0x7f432c070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.234+0000 7f432be00640 1 -- 172.21.15.46:0/3915260053 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467005696 0 0) 0x7f432c06c390 con 0x7f432c158300 2024-03-29T08:23:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f432be00640 1 -- 172.21.15.46:0/3915260053 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4310003530 con 0x7f432c158300 2024-03-29T08:23:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f432be00640 1 -- 172.21.15.46:0/3915260053 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 181038983 0 0) 0x7f4310003530 con 0x7f432c158300 2024-03-29T08:23:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f432be00640 1 -- 172.21.15.46:0/3915260053 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f432c0af530 msgr2=0x7f432c0af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f432be00640 1 --2- 172.21.15.46:0/3915260053 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f432c0af530 0x7f432c0af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f432be00640 1 -- 172.21.15.46:0/3915260053 >> v2:172.21.15.73:3300/0 conn(0x7f432c0702a0 msgr2=0x7f432c070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f432be00640 1 --2- 172.21.15.46:0/3915260053 >> v2:172.21.15.73:3300/0 conn(0x7f432c0702a0 0x7f432c070730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f432be00640 1 -- 172.21.15.46:0/3915260053 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c06c5e0 con 0x7f432c158300 2024-03-29T08:23:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f432be00640 1 -- 172.21.15.46:0/3915260053 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f431c001680 con 0x7f432c158300 2024-03-29T08:23:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f432be00640 1 -- 172.21.15.46:0/3915260053 >> v1:172.21.15.46:6789/0 conn(0x7f432c158300 legacy=0x7f432c06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f432be00640 1 --2- 172.21.15.46:0/3915260053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4310003980 0x7f4310003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f432be00640 1 --2- 172.21.15.46:0/3915260053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43100047d0 0x7f4310004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f432be00640 1 --2- 172.21.15.46:0/3915260053 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f432c0af530 0x7f4310005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f432be00640 1 -- 172.21.15.46:0/3915260053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c070c70 con 0x7f4310003980 2024-03-29T08:23:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f432be00640 1 -- 172.21.15.46:0/3915260053 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0afff0 con 0x7f43100047d0 2024-03-29T08:23:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f432be00640 1 -- 172.21.15.46:0/3915260053 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431000ba10 con 0x7f432c0af530 2024-03-29T08:23:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f4330c00640 1 --2- 172.21.15.46:0/3915260053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4310003980 0x7f4310003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f4323e00640 1 --2- 172.21.15.46:0/3915260053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43100047d0 0x7f4310004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f4331600640 1 --2- 172.21.15.46:0/3915260053 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f432c0af530 0x7f4310005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f4330c00640 1 -- 172.21.15.46:0/3915260053 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f432c0af530 msgr2=0x7f4310005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f4330c00640 1 --2- 172.21.15.46:0/3915260053 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f432c0af530 0x7f4310005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f4330c00640 1 -- 172.21.15.46:0/3915260053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43100047d0 msgr2=0x7f4310004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f4330c00640 1 --2- 172.21.15.46:0/3915260053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43100047d0 0x7f4310004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f4330c00640 1 -- 172.21.15.46:0/3915260053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4310003530 con 0x7f4310003980 2024-03-29T08:23:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f4331600640 1 --2- 172.21.15.46:0/3915260053 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f432c0af530 0x7f4310005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.238+0000 7f4330c00640 1 --2- 172.21.15.46:0/3915260053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4310003980 0x7f4310003da0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f431c00c210 tx=0x7f431c01c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba776c91032854d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f432be00640 1 -- 172.21.15.46:0/3915260053 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c005ae0 con 0x7f4310003980 2024-03-29T08:23:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f432be00640 1 -- 172.21.15.46:0/3915260053 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c0a7070 con 0x7f4310003980 2024-03-29T08:23:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f432be00640 1 -- 172.21.15.46:0/3915260053 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c005e80 con 0x7f4310003980 2024-03-29T08:23:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4333600640 1 -- 172.21.15.46:0/3915260053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4310003980 msgr2=0x7f4310003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4333600640 1 --2- 172.21.15.46:0/3915260053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4310003980 0x7f4310003da0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f431c00c210 tx=0x7f431c01c8f0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4331600640 1 -- 172.21.15.46:0/3915260053 reap_dead start 2024-03-29T08:23:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4333600640 1 -- 172.21.15.46:0/3915260053 shutdown_connections 2024-03-29T08:23:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4333600640 1 -- 172.21.15.46:0/3915260053 >> 172.21.15.46:0/3915260053 conn(0x7f432c066080 msgr2=0x7f432c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4333600640 1 -- 172.21.15.46:0/3915260053 shutdown_connections 2024-03-29T08:23:11.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4333600640 1 -- 172.21.15.46:0/3915260053 wait complete. 2024-03-29T08:23:11.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4333600640 1 Processor -- start 2024-03-29T08:23:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4333600640 1 -- start start 2024-03-29T08:23:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4333600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4310003980 0x7f432c141ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4333600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f432c0b2440 0x7f432c0b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4333600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f432c0abf00 0x7f432c1604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4333600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0b1680 con 0x7f4310003980 2024-03-29T08:23:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4333600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0b1960 con 0x7f432c0b2440 2024-03-29T08:23:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4333600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432c0b1c40 con 0x7f432c0abf00 2024-03-29T08:23:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4330c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4310003980 0x7f432c141ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4330c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4310003980 0x7f432c141ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40500/0 (socket says 172.21.15.46:40500) 2024-03-29T08:23:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4330c00640 1 -- 172.21.15.46:0/3248604440 learned_addr learned my addr 172.21.15.46:0/3248604440 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4323e00640 1 --2- 172.21.15.46:0/3248604440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f432c0b2440 0x7f432c0b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.242+0000 7f4331600640 1 --2- 172.21.15.46:0/3248604440 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f432c0abf00 0x7f432c1604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.246+0000 7f4330c00640 1 -- 172.21.15.46:0/3248604440 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f432c0abf00 msgr2=0x7f432c1604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.246+0000 7f4330c00640 1 --2- 172.21.15.46:0/3248604440 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f432c0abf00 0x7f432c1604f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.246+0000 7f4330c00640 1 -- 172.21.15.46:0/3248604440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f432c0b2440 msgr2=0x7f432c0b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.246+0000 7f4330c00640 1 --2- 172.21.15.46:0/3248604440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f432c0b2440 0x7f432c0b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.246+0000 7f4330c00640 1 -- 172.21.15.46:0/3248604440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c0b1f20 con 0x7f4310003980 2024-03-29T08:23:11.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.246+0000 7f4330c00640 1 --2- 172.21.15.46:0/3248604440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4310003980 0x7f432c141ea0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f431c01cdd0 tx=0x7f4310005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.246+0000 7f4329600640 1 -- 172.21.15.46:0/3248604440 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c0b3070 con 0x7f4310003980 2024-03-29T08:23:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.246+0000 7f4329600640 1 -- 172.21.15.46:0/3248604440 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c0047f0 con 0x7f4310003980 2024-03-29T08:23:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.246+0000 7f4333600640 1 -- 172.21.15.46:0/3248604440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432c1427a0 con 0x7f4310003980 2024-03-29T08:23:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.246+0000 7f4329600640 1 -- 172.21.15.46:0/3248604440 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c004b60 con 0x7f4310003980 2024-03-29T08:23:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.246+0000 7f4333600640 1 -- 172.21.15.46:0/3248604440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432c0b35a0 con 0x7f4310003980 2024-03-29T08:23:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.246+0000 7f4333600640 1 -- 172.21.15.46:0/3248604440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f432c0b17c0 con 0x7f4310003980 2024-03-29T08:23:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.246+0000 7f4329600640 1 -- 172.21.15.46:0/3248604440 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f431c0a9020 con 0x7f4310003980 2024-03-29T08:23:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.246+0000 7f4329600640 1 --2- 172.21.15.46:0/3248604440 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4308066690 0x7f4308068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.246+0000 7f4329600640 1 -- 172.21.15.46:0/3248604440 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f431c11caa0 con 0x7f4310003980 2024-03-29T08:23:11.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.246+0000 7f4323e00640 1 --2- 172.21.15.46:0/3248604440 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4308066690 0x7f4308068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.250+0000 7f4323e00640 1 --2- 172.21.15.46:0/3248604440 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4308066690 0x7f4308068b50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f432c15ba20 tx=0x7f4314009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:11.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.250+0000 7f4329600640 1 -- 172.21.15.46:0/3248604440 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f432c0b17c0 con 0x7f4310003980 2024-03-29T08:23:11.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.426+0000 7f4333600640 1 -- 172.21.15.46:0/3248604440 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f432c0b17c0 con 0x7f4310003980 2024-03-29T08:23:11.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.426+0000 7f4329600640 1 -- 172.21.15.46:0/3248604440 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v80) v1 ==== 80+0+116 (secure 0 0 0) 0x7f432c0b17c0 con 0x7f4310003980 2024-03-29T08:23:11.431 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6835/9617787 2024-03-30T08:23:02.389981+0000 2024-03-29T08:23:11.431 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6834/9617787 2024-03-30T08:23:02.389981+0000 2024-03-29T08:23:11.431 INFO:teuthology.orchestra.run.smithi046.stderr:listed 2 entries 2024-03-29T08:23:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.430+0000 7f4333600640 1 -- 172.21.15.46:0/3248604440 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4308066690 msgr2=0x7f4308068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.430+0000 7f4333600640 1 --2- 172.21.15.46:0/3248604440 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4308066690 0x7f4308068b50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f432c15ba20 tx=0x7f4314009190 comp rx=0 tx=0).stop 2024-03-29T08:23:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.430+0000 7f4333600640 1 -- 172.21.15.46:0/3248604440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4310003980 msgr2=0x7f432c141ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.430+0000 7f4333600640 1 --2- 172.21.15.46:0/3248604440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4310003980 0x7f432c141ea0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f431c01cdd0 tx=0x7f4310005800 comp rx=0 tx=0).stop 2024-03-29T08:23:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.430+0000 7f4333600640 1 -- 172.21.15.46:0/3248604440 shutdown_connections 2024-03-29T08:23:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.430+0000 7f4333600640 1 --2- 172.21.15.46:0/3248604440 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4308066690 0x7f4308068b50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.430+0000 7f4333600640 1 --2- 172.21.15.46:0/3248604440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f432c0b2440 0x7f432c0b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.430+0000 7f4333600640 1 --2- 172.21.15.46:0/3248604440 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f432c0abf00 0x7f432c1604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.430+0000 7f4333600640 1 --2- 172.21.15.46:0/3248604440 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4310003980 0x7f432c141ea0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.430+0000 7f4333600640 1 -- 172.21.15.46:0/3248604440 >> 172.21.15.46:0/3248604440 conn(0x7f432c066080 msgr2=0x7f432c15de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.430+0000 7f4333600640 1 -- 172.21.15.46:0/3248604440 shutdown_connections 2024-03-29T08:23:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.430+0000 7f4333600640 1 -- 172.21.15.46:0/3248604440 wait complete. 2024-03-29T08:23:11.452 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:23:11.453 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:23:11.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:23:11.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.638+0000 7fdf5f800640 1 Processor -- start 2024-03-29T08:23:11.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.638+0000 7fdf5f800640 1 -- start start 2024-03-29T08:23:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.638+0000 7fdf5f800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf58144210 0x7fdf580cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.638+0000 7fdf5f800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf5815ada0 0x7fdf580ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.638+0000 7fdf5f800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf580cb510 con 0x7fdf580ce950 2024-03-29T08:23:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.638+0000 7fdf5f800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf580cede0 con 0x7fdf58144210 2024-03-29T08:23:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.638+0000 7fdf5f800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5806d960 con 0x7fdf5815ada0 2024-03-29T08:23:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.638+0000 7fdf5ce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf58144210 0x7fdf580cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.638+0000 7fdf5d800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdf580ce950 0x7fdf5815e360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34074/0 (socket says 172.21.15.46:34074) 2024-03-29T08:23:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.638+0000 7fdf5d800640 1 -- 172.21.15.46:0/439342333 learned_addr learned my addr 172.21.15.46:0/439342333 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.642+0000 7fdf57e00640 1 --2- 172.21.15.46:0/439342333 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf5815ada0 0x7fdf580ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.642+0000 7fdf5ce00640 1 -- 172.21.15.46:0/439342333 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf5815ada0 msgr2=0x7fdf580ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.642+0000 7fdf5ce00640 1 --2- 172.21.15.46:0/439342333 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf5815ada0 0x7fdf580ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.642+0000 7fdf5ce00640 1 -- 172.21.15.46:0/439342333 >> v1:172.21.15.46:6789/0 conn(0x7fdf580ce950 legacy=0x7fdf5815e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.642+0000 7fdf5ce00640 1 -- 172.21.15.46:0/439342333 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf58070580 con 0x7fdf58144210 2024-03-29T08:23:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.642+0000 7fdf57400640 1 -- 172.21.15.46:0/439342333 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689870292 0 0) 0x7fdf580cb510 con 0x7fdf580ce950 2024-03-29T08:23:11.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.642+0000 7fdf5ce00640 1 --2- 172.21.15.46:0/439342333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf58144210 0x7fdf580cb8b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0097c0 tx=0x7fdf4c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d818cf6ba953a75f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:11.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.642+0000 7fdf57400640 1 -- 172.21.15.46:0/439342333 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0ab600 con 0x7fdf58144210 2024-03-29T08:23:11.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.642+0000 7fdf57400640 1 -- 172.21.15.46:0/439342333 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4c0a7050 con 0x7fdf58144210 2024-03-29T08:23:11.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.642+0000 7fdf57400640 1 -- 172.21.15.46:0/439342333 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0ab980 con 0x7fdf58144210 2024-03-29T08:23:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5f800640 1 -- 172.21.15.46:0/439342333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf58144210 msgr2=0x7fdf580cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5f800640 1 --2- 172.21.15.46:0/439342333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf58144210 0x7fdf580cb8b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0097c0 tx=0x7fdf4c01c4a0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5f800640 1 -- 172.21.15.46:0/439342333 shutdown_connections 2024-03-29T08:23:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5f800640 1 --2- 172.21.15.46:0/439342333 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf5815ada0 0x7fdf580ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5f800640 1 --2- 172.21.15.46:0/439342333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf58144210 0x7fdf580cb8b0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5f800640 1 -- 172.21.15.46:0/439342333 >> 172.21.15.46:0/439342333 conn(0x7fdf58066110 msgr2=0x7fdf58069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5f800640 1 -- 172.21.15.46:0/439342333 shutdown_connections 2024-03-29T08:23:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5f800640 1 -- 172.21.15.46:0/439342333 wait complete. 2024-03-29T08:23:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5f800640 1 Processor -- start 2024-03-29T08:23:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5f800640 1 -- start start 2024-03-29T08:23:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5f800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf580ce950 0x7fdf5813fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5f800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf58144210 0x7fdf5815f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5f800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf5815ada0 0x7fdf581600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5f800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf58165540 con 0x7fdf58144210 2024-03-29T08:23:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5f800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf58164d30 con 0x7fdf580ce950 2024-03-29T08:23:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5f800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf58164f80 con 0x7fdf5815ada0 2024-03-29T08:23:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf57e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf58144210 0x7fdf5815f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf57e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf58144210 0x7fdf5815f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40522/0 (socket says 172.21.15.46:40522) 2024-03-29T08:23:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf57e00640 1 -- 172.21.15.46:0/3321621725 learned_addr learned my addr 172.21.15.46:0/3321621725 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5ce00640 1 --2- 172.21.15.46:0/3321621725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf580ce950 0x7fdf5813fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf5d800640 1 --2- 172.21.15.46:0/3321621725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf5815ada0 0x7fdf581600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf57e00640 1 -- 172.21.15.46:0/3321621725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf5815ada0 msgr2=0x7fdf581600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf57e00640 1 --2- 172.21.15.46:0/3321621725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf5815ada0 0x7fdf581600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf57e00640 1 -- 172.21.15.46:0/3321621725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf580ce950 msgr2=0x7fdf5813fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf57e00640 1 --2- 172.21.15.46:0/3321621725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf580ce950 0x7fdf5813fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.646+0000 7fdf57e00640 1 -- 172.21.15.46:0/3321621725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c009420 con 0x7fdf58144210 2024-03-29T08:23:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.650+0000 7fdf57e00640 1 --2- 172.21.15.46:0/3321621725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf58144210 0x7fdf5815f450 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400d5f0 tx=0x7fdf4400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.650+0000 7fdf54c00640 1 -- 172.21.15.46:0/3321621725 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf440125f0 con 0x7fdf58144210 2024-03-29T08:23:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.650+0000 7fdf54c00640 1 -- 172.21.15.46:0/3321621725 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf44012750 con 0x7fdf58144210 2024-03-29T08:23:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.650+0000 7fdf54c00640 1 -- 172.21.15.46:0/3321621725 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf44012ae0 con 0x7fdf58144210 2024-03-29T08:23:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.650+0000 7fdf5f800640 1 -- 172.21.15.46:0/3321621725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf58165260 con 0x7fdf58144210 2024-03-29T08:23:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.650+0000 7fdf5f800640 1 -- 172.21.15.46:0/3321621725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5815b600 con 0x7fdf58144210 2024-03-29T08:23:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.650+0000 7fdf5f800640 1 -- 172.21.15.46:0/3321621725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5815b7c0 con 0x7fdf58144210 2024-03-29T08:23:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.650+0000 7fdf54c00640 1 -- 172.21.15.46:0/3321621725 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdf44012c40 con 0x7fdf58144210 2024-03-29T08:23:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.650+0000 7fdf54c00640 1 --2- 172.21.15.46:0/3321621725 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdf38066690 0x7fdf38068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.650+0000 7fdf54c00640 1 -- 172.21.15.46:0/3321621725 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fdf44084580 con 0x7fdf58144210 2024-03-29T08:23:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.654+0000 7fdf5ce00640 1 --2- 172.21.15.46:0/3321621725 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdf38066690 0x7fdf38068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.654+0000 7fdf5ce00640 1 --2- 172.21.15.46:0/3321621725 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdf38066690 0x7fdf38068b50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0ad000 tx=0x7fdf4c0a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:11.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.658+0000 7fdf54c00640 1 -- 172.21.15.46:0/3321621725 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdf44051910 con 0x7fdf58144210 2024-03-29T08:23:11.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.818+0000 7fdf5f800640 1 -- 172.21.15.46:0/3321621725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf580ce280 con 0x7fdf58144210 2024-03-29T08:23:11.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.818+0000 7fdf54c00640 1 -- 172.21.15.46:0/3321621725 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16213 (secure 0 0 0) 0x7fdf44054f30 con 0x7fdf58144210 2024-03-29T08:23:11.823 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:23:11.823 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:23:02.390005+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:22:42.147364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-29T08:22:42.602168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1f","mappings":[{"from":2,"to":7}]},{"pgid":"11.27","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/9617787":"2024-03-30T08:23:02.389981+0000","172.21.15.46:6834/9617787":"2024-03-30T08:23:02.389981+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:23:11.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.822+0000 7fdf5f800640 1 -- 172.21.15.46:0/3321621725 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdf38066690 msgr2=0x7fdf38068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.822+0000 7fdf5f800640 1 --2- 172.21.15.46:0/3321621725 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdf38066690 0x7fdf38068b50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0ad000 tx=0x7fdf4c0a6000 comp rx=0 tx=0).stop 2024-03-29T08:23:11.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.822+0000 7fdf5f800640 1 -- 172.21.15.46:0/3321621725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf58144210 msgr2=0x7fdf5815f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.822+0000 7fdf5f800640 1 --2- 172.21.15.46:0/3321621725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf58144210 0x7fdf5815f450 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fdf4400d5f0 tx=0x7fdf4400dab0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.822+0000 7fdf5f800640 1 -- 172.21.15.46:0/3321621725 shutdown_connections 2024-03-29T08:23:11.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.822+0000 7fdf5f800640 1 --2- 172.21.15.46:0/3321621725 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdf38066690 0x7fdf38068b50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.822+0000 7fdf5f800640 1 --2- 172.21.15.46:0/3321621725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdf5815ada0 0x7fdf581600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.822+0000 7fdf5f800640 1 --2- 172.21.15.46:0/3321621725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf58144210 0x7fdf5815f450 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.822+0000 7fdf5f800640 1 --2- 172.21.15.46:0/3321621725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf580ce950 0x7fdf5813fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.822+0000 7fdf5f800640 1 -- 172.21.15.46:0/3321621725 >> 172.21.15.46:0/3321621725 conn(0x7fdf58066110 msgr2=0x7fdf58140650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:11.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.822+0000 7fdf5f800640 1 -- 172.21.15.46:0/3321621725 shutdown_connections 2024-03-29T08:23:11.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.822+0000 7fdf5f800640 1 -- 172.21.15.46:0/3321621725 wait complete. 2024-03-29T08:23:11.844 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:22:42.147364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:22:42.602168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:23:11.845 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:23:11.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.986+0000 7faf94e00640 1 Processor -- start 2024-03-29T08:23:11.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.986+0000 7faf94e00640 1 -- start start 2024-03-29T08:23:11.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.986+0000 7faf94e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faf900af680 0x7faf9015b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.986+0000 7faf94e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faf900b0860 0x7faf901510a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.986+0000 7faf94e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9015e880 con 0x7faf900afa60 2024-03-29T08:23:11.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.986+0000 7faf94e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9015eb60 con 0x7faf900b0860 2024-03-29T08:23:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.986+0000 7faf94e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9015ee40 con 0x7faf900af680 2024-03-29T08:23:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.986+0000 7faf87e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7faf900afa60 0x7faf900b0150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34080/0 (socket says 172.21.15.46:34080) 2024-03-29T08:23:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.986+0000 7faf8fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faf900b0860 0x7faf901510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.986+0000 7faf87e00640 1 -- 172.21.15.46:0/3136628631 learned_addr learned my addr 172.21.15.46:0/3136628631 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.986+0000 7faf8f400640 1 --2- 172.21.15.46:0/3136628631 >> v2:172.21.15.73:3300/0 conn(0x7faf900af680 0x7faf9015b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.986+0000 7faf8fe00640 1 -- 172.21.15.46:0/3136628631 >> v2:172.21.15.73:3300/0 conn(0x7faf900af680 msgr2=0x7faf9015b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.986+0000 7faf8fe00640 1 --2- 172.21.15.46:0/3136628631 >> v2:172.21.15.73:3300/0 conn(0x7faf900af680 0x7faf9015b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.986+0000 7faf8fe00640 1 -- 172.21.15.46:0/3136628631 >> v1:172.21.15.46:6789/0 conn(0x7faf900afa60 legacy=0x7faf900b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.986+0000 7faf8fe00640 1 -- 172.21.15.46:0/3136628631 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9015f120 con 0x7faf900b0860 2024-03-29T08:23:11.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.990+0000 7faf8fe00640 1 --2- 172.21.15.46:0/3136628631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf900b0860 0x7faf901510a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7faf900722d0 tx=0x7faf8801c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c155e978dcdea server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:11.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.990+0000 7faf8ea00640 1 -- 172.21.15.46:0/3136628631 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf880a58f0 con 0x7faf900b0860 2024-03-29T08:23:11.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.990+0000 7faf8ea00640 1 -- 172.21.15.46:0/3136628631 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf880a5a50 con 0x7faf900b0860 2024-03-29T08:23:11.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.990+0000 7faf8ea00640 1 -- 172.21.15.46:0/3136628631 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf880a5de0 con 0x7faf900b0860 2024-03-29T08:23:11.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.990+0000 7faf94e00640 1 -- 172.21.15.46:0/3136628631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf900b0860 msgr2=0x7faf901510a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.990+0000 7faf94e00640 1 --2- 172.21.15.46:0/3136628631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf900b0860 0x7faf901510a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7faf900722d0 tx=0x7faf8801c830 comp rx=0 tx=0).stop 2024-03-29T08:23:11.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.990+0000 7faf94e00640 1 -- 172.21.15.46:0/3136628631 shutdown_connections 2024-03-29T08:23:11.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.990+0000 7faf94e00640 1 --2- 172.21.15.46:0/3136628631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf900b0860 0x7faf901510a0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.990+0000 7faf94e00640 1 --2- 172.21.15.46:0/3136628631 >> v2:172.21.15.73:3300/0 conn(0x7faf900af680 0x7faf9015b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.990+0000 7faf94e00640 1 -- 172.21.15.46:0/3136628631 >> 172.21.15.46:0/3136628631 conn(0x7faf90098a40 msgr2=0x7faf900cf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:11.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.990+0000 7faf94e00640 1 -- 172.21.15.46:0/3136628631 shutdown_connections 2024-03-29T08:23:11.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.990+0000 7faf94e00640 1 -- 172.21.15.46:0/3136628631 wait complete. 2024-03-29T08:23:11.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.990+0000 7faf94e00640 1 Processor -- start 2024-03-29T08:23:11.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.990+0000 7faf94e00640 1 -- start start 2024-03-29T08:23:11.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf94e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf900af680 0x7faf901472c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf94e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf900afa60 0x7faf900ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf94e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf900b0860 0x7faf9015d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:11.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf94e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf90147800 con 0x7faf900af680 2024-03-29T08:23:11.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf94e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf900cee70 con 0x7faf900afa60 2024-03-29T08:23:11.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf94e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf900cf0a0 con 0x7faf900b0860 2024-03-29T08:23:11.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf8f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf900af680 0x7faf901472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf8f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf900af680 0x7faf901472c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40550/0 (socket says 172.21.15.46:40550) 2024-03-29T08:23:11.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf8f400640 1 -- 172.21.15.46:0/2241293748 learned_addr learned my addr 172.21.15.46:0/2241293748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:11.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf87e00640 1 --2- 172.21.15.46:0/2241293748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf900afa60 0x7faf900ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf8fe00640 1 --2- 172.21.15.46:0/2241293748 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf900b0860 0x7faf9015d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:11.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf8f400640 1 -- 172.21.15.46:0/2241293748 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf900b0860 msgr2=0x7faf9015d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf8f400640 1 --2- 172.21.15.46:0/2241293748 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf900b0860 0x7faf9015d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf8f400640 1 -- 172.21.15.46:0/2241293748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf900afa60 msgr2=0x7faf900ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:11.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf8f400640 1 --2- 172.21.15.46:0/2241293748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf900afa60 0x7faf900ce930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:11.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf8f400640 1 -- 172.21.15.46:0/2241293748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf88009a70 con 0x7faf900af680 2024-03-29T08:23:11.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf87e00640 1 --2- 172.21.15.46:0/2241293748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf900afa60 0x7faf900ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:11.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf8f400640 1 --2- 172.21.15.46:0/2241293748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf900af680 0x7faf901472c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7faf8000e960 tx=0x7faf8000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:11.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf87400640 1 -- 172.21.15.46:0/2241293748 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80009690 con 0x7faf900af680 2024-03-29T08:23:11.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf87400640 1 -- 172.21.15.46:0/2241293748 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8000ca00 con 0x7faf900af680 2024-03-29T08:23:11.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf87400640 1 -- 172.21.15.46:0/2241293748 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8000cd70 con 0x7faf900af680 2024-03-29T08:23:11.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf94e00640 1 -- 172.21.15.46:0/2241293748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf901588d0 con 0x7faf900af680 2024-03-29T08:23:11.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf94e00640 1 -- 172.21.15.46:0/2241293748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf90158e90 con 0x7faf900af680 2024-03-29T08:23:11.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.994+0000 7faf94e00640 1 -- 172.21.15.46:0/2241293748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf90067170 con 0x7faf900af680 2024-03-29T08:23:11.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.998+0000 7faf87400640 1 -- 172.21.15.46:0/2241293748 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7faf80013470 con 0x7faf900af680 2024-03-29T08:23:12.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.998+0000 7faf87400640 1 --2- 172.21.15.46:0/2241293748 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf6c066690 0x7faf6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:12.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.998+0000 7faf87400640 1 -- 172.21.15.46:0/2241293748 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7faf800844d0 con 0x7faf900af680 2024-03-29T08:23:12.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.998+0000 7faf87e00640 1 --2- 172.21.15.46:0/2241293748 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf6c066690 0x7faf6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:12.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:11.998+0000 7faf87e00640 1 --2- 172.21.15.46:0/2241293748 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf6c066690 0x7faf6c068b50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7faf900b5320 tx=0x7faf78009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:12.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.002+0000 7faf87400640 1 -- 172.21.15.46:0/2241293748 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7faf80051790 con 0x7faf900af680 2024-03-29T08:23:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.162+0000 7faf94e00640 1 -- 172.21.15.46:0/2241293748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf9004e040 con 0x7faf900af680 2024-03-29T08:23:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.162+0000 7faf87400640 1 -- 172.21.15.46:0/2241293748 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf80054db0 con 0x7faf900af680 2024-03-29T08:23:12.165 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:23:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.162+0000 7faf94e00640 1 -- 172.21.15.46:0/2241293748 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf6c066690 msgr2=0x7faf6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.162+0000 7faf94e00640 1 --2- 172.21.15.46:0/2241293748 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf6c066690 0x7faf6c068b50 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7faf900b5320 tx=0x7faf78009040 comp rx=0 tx=0).stop 2024-03-29T08:23:12.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.162+0000 7faf94e00640 1 -- 172.21.15.46:0/2241293748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf900af680 msgr2=0x7faf901472c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:12.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.162+0000 7faf94e00640 1 --2- 172.21.15.46:0/2241293748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf900af680 0x7faf901472c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7faf8000e960 tx=0x7faf8000ee20 comp rx=0 tx=0).stop 2024-03-29T08:23:12.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.162+0000 7faf94e00640 1 -- 172.21.15.46:0/2241293748 shutdown_connections 2024-03-29T08:23:12.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.162+0000 7faf94e00640 1 --2- 172.21.15.46:0/2241293748 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf6c066690 0x7faf6c068b50 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.162+0000 7faf94e00640 1 --2- 172.21.15.46:0/2241293748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf900afa60 0x7faf900ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.162+0000 7faf94e00640 1 --2- 172.21.15.46:0/2241293748 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf900b0860 0x7faf9015d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.162+0000 7faf94e00640 1 --2- 172.21.15.46:0/2241293748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf900af680 0x7faf901472c0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.162+0000 7faf94e00640 1 -- 172.21.15.46:0/2241293748 >> 172.21.15.46:0/2241293748 conn(0x7faf90098a40 msgr2=0x7faf90150cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:12.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.166+0000 7faf94e00640 1 -- 172.21.15.46:0/2241293748 shutdown_connections 2024-03-29T08:23:12.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.166+0000 7faf94e00640 1 -- 172.21.15.46:0/2241293748 wait complete. 2024-03-29T08:23:12.187 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:23:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.326+0000 7f23cca00640 1 Processor -- start 2024-03-29T08:23:12.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.326+0000 7f23cca00640 1 -- start start 2024-03-29T08:23:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.330+0000 7f23cca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f23c815dcd0 0x7f23c806ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.330+0000 7f23cca00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23c80cd260 0x7f23c806d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.330+0000 7f23cca00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c806b590 con 0x7f23c8069e00 2024-03-29T08:23:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.330+0000 7f23cca00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c806c990 con 0x7f23c815dcd0 2024-03-29T08:23:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.330+0000 7f23cca00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c806cc70 con 0x7f23c80cd260 2024-03-29T08:23:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.330+0000 7f23bea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f23c8069e00 0x7f23c80ccb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34094/0 (socket says 172.21.15.46:34094) 2024-03-29T08:23:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.330+0000 7f23bea00640 1 -- 172.21.15.46:0/4104270575 learned_addr learned my addr 172.21.15.46:0/4104270575 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.330+0000 7f23c7400640 1 --2- 172.21.15.46:0/4104270575 >> v2:172.21.15.46:3300/0 conn(0x7f23c815dcd0 0x7f23c806ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.330+0000 7f23c7e00640 1 --2- 172.21.15.46:0/4104270575 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23c80cd260 0x7f23c806d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.330+0000 7f23c7400640 1 -- 172.21.15.46:0/4104270575 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23c80cd260 msgr2=0x7f23c806d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.330+0000 7f23c7400640 1 --2- 172.21.15.46:0/4104270575 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23c80cd260 0x7f23c806d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.330+0000 7f23c7400640 1 -- 172.21.15.46:0/4104270575 >> v1:172.21.15.46:6789/0 conn(0x7f23c8069e00 legacy=0x7f23c80ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.330+0000 7f23c7400640 1 -- 172.21.15.46:0/4104270575 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c806cf50 con 0x7f23c815dcd0 2024-03-29T08:23:12.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.330+0000 7f23c6a00640 1 -- 172.21.15.46:0/4104270575 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683269238 0 0) 0x7f23c806b590 con 0x7f23c8069e00 2024-03-29T08:23:12.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23c7400640 1 --2- 172.21.15.46:0/4104270575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23c815dcd0 0x7f23c806ac80 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f23b80097c0 tx=0x7f23b801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b95bb9fa04b17b96 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:12.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23c6a00640 1 -- 172.21.15.46:0/4104270575 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b80ab600 con 0x7f23c815dcd0 2024-03-29T08:23:12.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23c6a00640 1 -- 172.21.15.46:0/4104270575 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b80a7050 con 0x7f23c815dcd0 2024-03-29T08:23:12.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23c6a00640 1 -- 172.21.15.46:0/4104270575 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b80ab980 con 0x7f23c815dcd0 2024-03-29T08:23:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23cca00640 1 -- 172.21.15.46:0/4104270575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23c815dcd0 msgr2=0x7f23c806ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23cca00640 1 --2- 172.21.15.46:0/4104270575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23c815dcd0 0x7f23c806ac80 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f23b80097c0 tx=0x7f23b801c3e0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23cca00640 1 -- 172.21.15.46:0/4104270575 shutdown_connections 2024-03-29T08:23:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23cca00640 1 --2- 172.21.15.46:0/4104270575 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23c80cd260 0x7f23c806d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23cca00640 1 --2- 172.21.15.46:0/4104270575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23c815dcd0 0x7f23c806ac80 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23cca00640 1 -- 172.21.15.46:0/4104270575 >> 172.21.15.46:0/4104270575 conn(0x7f23c80cb6d0 msgr2=0x7f23c80cbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23cca00640 1 -- 172.21.15.46:0/4104270575 shutdown_connections 2024-03-29T08:23:12.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23cca00640 1 -- 172.21.15.46:0/4104270575 wait complete. 2024-03-29T08:23:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23cca00640 1 Processor -- start 2024-03-29T08:23:12.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23cca00640 1 -- start start 2024-03-29T08:23:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23cca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23c8069e00 0x7f23c814fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:12.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23cca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23c80cd260 0x7f23c81f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23cca00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23c815dcd0 0x7f23c8154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23cca00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c8150cc0 con 0x7f23c80cd260 2024-03-29T08:23:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23cca00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c8150fa0 con 0x7f23c8069e00 2024-03-29T08:23:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.334+0000 7f23cca00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c8151280 con 0x7f23c815dcd0 2024-03-29T08:23:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23bea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23c80cd260 0x7f23c81f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23bea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23c80cd260 0x7f23c81f5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40566/0 (socket says 172.21.15.46:40566) 2024-03-29T08:23:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23bea00640 1 -- 172.21.15.46:0/233612147 learned_addr learned my addr 172.21.15.46:0/233612147 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23c7400640 1 --2- 172.21.15.46:0/233612147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23c8069e00 0x7f23c814fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:12.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23c7e00640 1 --2- 172.21.15.46:0/233612147 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23c815dcd0 0x7f23c8154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23bea00640 1 -- 172.21.15.46:0/233612147 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23c815dcd0 msgr2=0x7f23c8154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:12.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23bea00640 1 --2- 172.21.15.46:0/233612147 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23c815dcd0 0x7f23c8154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23bea00640 1 -- 172.21.15.46:0/233612147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23c8069e00 msgr2=0x7f23c814fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23bea00640 1 --2- 172.21.15.46:0/233612147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23c8069e00 0x7f23c814fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23bea00640 1 -- 172.21.15.46:0/233612147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b8009420 con 0x7f23c80cd260 2024-03-29T08:23:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23c7400640 1 --2- 172.21.15.46:0/233612147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23c8069e00 0x7f23c814fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:23:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23bea00640 1 --2- 172.21.15.46:0/233612147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23c80cd260 0x7f23c81f5db0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f23b00011f0 tx=0x7f23b000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23bfe00640 1 -- 172.21.15.46:0/233612147 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b00074e0 con 0x7f23c80cd260 2024-03-29T08:23:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23bfe00640 1 -- 172.21.15.46:0/233612147 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b0007640 con 0x7f23c80cd260 2024-03-29T08:23:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23bfe00640 1 -- 172.21.15.46:0/233612147 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0004a10 con 0x7f23c80cd260 2024-03-29T08:23:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23cca00640 1 -- 172.21.15.46:0/233612147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c8151560 con 0x7f23c80cd260 2024-03-29T08:23:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23cca00640 1 -- 172.21.15.46:0/233612147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c81501e0 con 0x7f23c80cd260 2024-03-29T08:23:12.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23cca00640 1 -- 172.21.15.46:0/233612147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c8151a80 con 0x7f23c80cd260 2024-03-29T08:23:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.338+0000 7f23bfe00640 1 -- 172.21.15.46:0/233612147 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f23b0004b70 con 0x7f23c80cd260 2024-03-29T08:23:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.342+0000 7f23bfe00640 1 --2- 172.21.15.46:0/233612147 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23a4066690 0x7f23a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:12.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.342+0000 7f23bfe00640 1 -- 172.21.15.46:0/233612147 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f23b00834f0 con 0x7f23c80cd260 2024-03-29T08:23:12.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.342+0000 7f23c7400640 1 --2- 172.21.15.46:0/233612147 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23a4066690 0x7f23a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:12.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.342+0000 7f23c7400640 1 --2- 172.21.15.46:0/233612147 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23a4066690 0x7f23a4068b50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f23b80ad000 tx=0x7f23b8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:12.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.346+0000 7f23bfe00640 1 -- 172.21.15.46:0/233612147 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f23c8151a80 con 0x7f23c80cd260 2024-03-29T08:23:12.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.506+0000 7f23cca00640 1 -- 172.21.15.46:0/233612147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f23c8151a80 con 0x7f23c80cd260 2024-03-29T08:23:12.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.506+0000 7f23bfe00640 1 -- 172.21.15.46:0/233612147 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f23b0018020 con 0x7f23c80cd260 2024-03-29T08:23:12.508 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:23:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.506+0000 7f23cca00640 1 -- 172.21.15.46:0/233612147 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23a4066690 msgr2=0x7f23a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.506+0000 7f23cca00640 1 --2- 172.21.15.46:0/233612147 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23a4066690 0x7f23a4068b50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f23b80ad000 tx=0x7f23b8008040 comp rx=0 tx=0).stop 2024-03-29T08:23:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.506+0000 7f23cca00640 1 -- 172.21.15.46:0/233612147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23c80cd260 msgr2=0x7f23c81f5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.506+0000 7f23cca00640 1 --2- 172.21.15.46:0/233612147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23c80cd260 0x7f23c81f5db0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f23b00011f0 tx=0x7f23b000c9b0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.506+0000 7f23cca00640 1 -- 172.21.15.46:0/233612147 shutdown_connections 2024-03-29T08:23:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.506+0000 7f23cca00640 1 --2- 172.21.15.46:0/233612147 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f23a4066690 0x7f23a4068b50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.506+0000 7f23cca00640 1 --2- 172.21.15.46:0/233612147 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f23c815dcd0 0x7f23c8154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.506+0000 7f23cca00640 1 --2- 172.21.15.46:0/233612147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23c80cd260 0x7f23c81f5db0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.506+0000 7f23cca00640 1 --2- 172.21.15.46:0/233612147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23c8069e00 0x7f23c814fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.506+0000 7f23cca00640 1 -- 172.21.15.46:0/233612147 >> 172.21.15.46:0/233612147 conn(0x7f23c80cb6d0 msgr2=0x7f23c80cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.506+0000 7f23cca00640 1 -- 172.21.15.46:0/233612147 shutdown_connections 2024-03-29T08:23:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.506+0000 7f23cca00640 1 -- 172.21.15.46:0/233612147 wait complete. 2024-03-29T08:23:12.528 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:23:12.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.666+0000 7f2a1e400640 1 Processor -- start 2024-03-29T08:23:12.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.666+0000 7f2a1e400640 1 -- start start 2024-03-29T08:23:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.670+0000 7f2a1e400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2a180cbea0 0x7f2a180b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.670+0000 7f2a1e400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a180b2360 con 0x7f2a18143eb0 2024-03-29T08:23:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.670+0000 7f2a1e400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a1815e500 con 0x7f2a1815efe0 2024-03-29T08:23:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.670+0000 7f2a1e400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a1815e7e0 con 0x7f2a180cbea0 2024-03-29T08:23:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.670+0000 7f2a17400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2a18143eb0 0x7f2a180cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34108/0 (socket says 172.21.15.46:34108) 2024-03-29T08:23:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.670+0000 7f2a17400640 1 -- 172.21.15.46:0/1875031021 learned_addr learned my addr 172.21.15.46:0/1875031021 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:12.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.670+0000 7f2a0fe00640 1 --2- 172.21.15.46:0/1875031021 >> v2:172.21.15.46:3300/0 conn(0x7f2a180cbea0 0x7f2a180b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:12.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.670+0000 7f2a16a00640 1 -- 172.21.15.46:0/1875031021 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 130716139 0 0) 0x7f2a180b2360 con 0x7f2a18143eb0 2024-03-29T08:23:12.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.670+0000 7f2a16a00640 1 -- 172.21.15.46:0/1875031021 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29fc003530 con 0x7f2a18143eb0 2024-03-29T08:23:12.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.670+0000 7f2a0fe00640 1 -- 172.21.15.46:0/1875031021 >> v1:172.21.15.73:6789/0 conn(0x7f2a1815efe0 legacy=0x7f2a180b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:12.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.670+0000 7f2a0fe00640 1 -- 172.21.15.46:0/1875031021 >> v1:172.21.15.46:6789/0 conn(0x7f2a18143eb0 legacy=0x7f2a180cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:12.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.670+0000 7f2a0fe00640 1 -- 172.21.15.46:0/1875031021 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1815eac0 con 0x7f2a180cbea0 2024-03-29T08:23:12.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a0fe00640 1 --2- 172.21.15.46:0/1875031021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cbea0 0x7f2a180b1630 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2a000097c0 tx=0x7f2a0001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e43c4e12d581772 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:12.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a16a00640 1 -- 172.21.15.46:0/1875031021 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a000aa620 con 0x7f2a180cbea0 2024-03-29T08:23:12.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a16a00640 1 -- 172.21.15.46:0/1875031021 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a000aa780 con 0x7f2a180cbea0 2024-03-29T08:23:12.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a16a00640 1 -- 172.21.15.46:0/1875031021 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a000aab10 con 0x7f2a180cbea0 2024-03-29T08:23:12.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a1e400640 1 -- 172.21.15.46:0/1875031021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cbea0 msgr2=0x7f2a180b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:12.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a1e400640 1 --2- 172.21.15.46:0/1875031021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cbea0 0x7f2a180b1630 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2a000097c0 tx=0x7f2a0001c4a0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a1e400640 1 -- 172.21.15.46:0/1875031021 shutdown_connections 2024-03-29T08:23:12.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a1e400640 1 --2- 172.21.15.46:0/1875031021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cbea0 0x7f2a180b1630 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a1e400640 1 -- 172.21.15.46:0/1875031021 >> 172.21.15.46:0/1875031021 conn(0x7f2a1806a390 msgr2=0x7f2a180cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:12.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a1e400640 1 -- 172.21.15.46:0/1875031021 shutdown_connections 2024-03-29T08:23:12.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a1e400640 1 -- 172.21.15.46:0/1875031021 wait complete. 2024-03-29T08:23:12.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a1e400640 1 Processor -- start 2024-03-29T08:23:12.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a1e400640 1 -- start start 2024-03-29T08:23:12.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a1e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cbea0 0x7f2a1815d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:12.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a1e400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a18143eb0 0x7f2a1815aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:12.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a1e400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a1815efe0 0x7f2a18138b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:12.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a1e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a18142130 con 0x7f2a180cbea0 2024-03-29T08:23:12.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a1e400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a181423d0 con 0x7f2a18143eb0 2024-03-29T08:23:12.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a1e400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a181426b0 con 0x7f2a1815efe0 2024-03-29T08:23:12.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a17400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cbea0 0x7f2a1815d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:12.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a17400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cbea0 0x7f2a1815d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40574/0 (socket says 172.21.15.46:40574) 2024-03-29T08:23:12.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.674+0000 7f2a17400640 1 -- 172.21.15.46:0/509332601 learned_addr learned my addr 172.21.15.46:0/509332601 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:12.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.678+0000 7f2a0fe00640 1 --2- 172.21.15.46:0/509332601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a18143eb0 0x7f2a1815aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:12.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.678+0000 7f2a17e00640 1 --2- 172.21.15.46:0/509332601 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a1815efe0 0x7f2a18138b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.678+0000 7f2a17400640 1 -- 172.21.15.46:0/509332601 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a1815efe0 msgr2=0x7f2a18138b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.678+0000 7f2a17400640 1 --2- 172.21.15.46:0/509332601 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a1815efe0 0x7f2a18138b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.678+0000 7f2a17400640 1 -- 172.21.15.46:0/509332601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a18143eb0 msgr2=0x7f2a1815aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.678+0000 7f2a17400640 1 --2- 172.21.15.46:0/509332601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a18143eb0 0x7f2a1815aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.678+0000 7f2a17400640 1 -- 172.21.15.46:0/509332601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a00009420 con 0x7f2a180cbea0 2024-03-29T08:23:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.678+0000 7f2a17400640 1 --2- 172.21.15.46:0/509332601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cbea0 0x7f2a1815d880 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2a080011f0 tx=0x7f2a0800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:12.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.678+0000 7f2a0f400640 1 -- 172.21.15.46:0/509332601 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a08005cf0 con 0x7f2a180cbea0 2024-03-29T08:23:12.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.678+0000 7f2a0f400640 1 -- 172.21.15.46:0/509332601 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a08005e50 con 0x7f2a180cbea0 2024-03-29T08:23:12.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.678+0000 7f2a1e400640 1 -- 172.21.15.46:0/509332601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a18142990 con 0x7f2a180cbea0 2024-03-29T08:23:12.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.678+0000 7f2a0f400640 1 -- 172.21.15.46:0/509332601 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a08007290 con 0x7f2a180cbea0 2024-03-29T08:23:12.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.678+0000 7f2a1e400640 1 -- 172.21.15.46:0/509332601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a18142f50 con 0x7f2a180cbea0 2024-03-29T08:23:12.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.678+0000 7f2a1e400640 1 -- 172.21.15.46:0/509332601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a1806e220 con 0x7f2a180cbea0 2024-03-29T08:23:12.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.678+0000 7f2a0f400640 1 -- 172.21.15.46:0/509332601 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2a080073f0 con 0x7f2a180cbea0 2024-03-29T08:23:12.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.678+0000 7f2a0f400640 1 --2- 172.21.15.46:0/509332601 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f29f4066690 0x7f29f4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:12.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.678+0000 7f2a0f400640 1 -- 172.21.15.46:0/509332601 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2a08083d80 con 0x7f2a180cbea0 2024-03-29T08:23:12.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.682+0000 7f2a0fe00640 1 --2- 172.21.15.46:0/509332601 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f29f4066690 0x7f29f4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:12.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.682+0000 7f2a0fe00640 1 --2- 172.21.15.46:0/509332601 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f29f4066690 0x7f29f4068b50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2a000ae000 tx=0x7f2a000a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:12.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.686+0000 7f2a0f400640 1 -- 172.21.15.46:0/509332601 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2a08051070 con 0x7f2a180cbea0 2024-03-29T08:23:12.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.842+0000 7f2a1e400640 1 -- 172.21.15.46:0/509332601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2a18143140 con 0x7f2a180cbea0 2024-03-29T08:23:12.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.842+0000 7f2a0f400640 1 -- 172.21.15.46:0/509332601 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2a08054690 con 0x7f2a180cbea0 2024-03-29T08:23:12.847 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:23:12.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.846+0000 7f2a1e400640 1 -- 172.21.15.46:0/509332601 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f29f4066690 msgr2=0x7f29f4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:12.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.846+0000 7f2a1e400640 1 --2- 172.21.15.46:0/509332601 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f29f4066690 0x7f29f4068b50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2a000ae000 tx=0x7f2a000a7000 comp rx=0 tx=0).stop 2024-03-29T08:23:12.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.846+0000 7f2a1e400640 1 -- 172.21.15.46:0/509332601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cbea0 msgr2=0x7f2a1815d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:12.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.846+0000 7f2a1e400640 1 --2- 172.21.15.46:0/509332601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cbea0 0x7f2a1815d880 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f2a080011f0 tx=0x7f2a0800c980 comp rx=0 tx=0).stop 2024-03-29T08:23:12.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.846+0000 7f2a1e400640 1 -- 172.21.15.46:0/509332601 shutdown_connections 2024-03-29T08:23:12.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.846+0000 7f2a1e400640 1 --2- 172.21.15.46:0/509332601 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f29f4066690 0x7f29f4068b50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.846+0000 7f2a1e400640 1 --2- 172.21.15.46:0/509332601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a18143eb0 0x7f2a1815aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.846+0000 7f2a1e400640 1 --2- 172.21.15.46:0/509332601 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a1815efe0 0x7f2a18138b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.846+0000 7f2a1e400640 1 --2- 172.21.15.46:0/509332601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a180cbea0 0x7f2a1815d880 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:12.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.846+0000 7f2a1e400640 1 -- 172.21.15.46:0/509332601 >> 172.21.15.46:0/509332601 conn(0x7f2a1806a390 msgr2=0x7f2a1815a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:12.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.846+0000 7f2a1e400640 1 -- 172.21.15.46:0/509332601 shutdown_connections 2024-03-29T08:23:12.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:12.846+0000 7f2a1e400640 1 -- 172.21.15.46:0/509332601 wait complete. 2024-03-29T08:23:12.868 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:23:13.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.006+0000 7f3064400640 1 Processor -- start 2024-03-29T08:23:13.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.006+0000 7f3064400640 1 -- start start 2024-03-29T08:23:13.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.006+0000 7f3064400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f305c15d480 0x7f305c070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:13.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.006+0000 7f3064400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305c071520 con 0x7f305c0b3850 2024-03-29T08:23:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.006+0000 7f3064400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f305c15ca30 con 0x7f305c071070 2024-03-29T08:23:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.006+0000 7f3064400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f305c0b20f0 con 0x7f305c15d480 2024-03-29T08:23:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.006+0000 7f3061a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f305c0b3850 0x7f305c0b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34116/0 (socket says 172.21.15.46:34116) 2024-03-29T08:23:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.006+0000 7f3061a00640 1 -- 172.21.15.46:0/48339626 learned_addr learned my addr 172.21.15.46:0/48339626 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:13.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.006+0000 7f3061000640 1 --2- 172.21.15.46:0/48339626 >> v2:172.21.15.73:3300/0 conn(0x7f305c15d480 0x7f305c070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.006+0000 7f305be00640 1 -- 172.21.15.46:0/48339626 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508562322 0 0) 0x7f3050001120 con 0x7f305c0b3850 2024-03-29T08:23:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.006+0000 7f305be00640 1 -- 172.21.15.46:0/48339626 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3044003530 con 0x7f305c0b3850 2024-03-29T08:23:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.006+0000 7f305be00640 1 -- 172.21.15.46:0/48339626 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2257226910 0 0) 0x7f305c15ca30 con 0x7f305c071070 2024-03-29T08:23:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.006+0000 7f305be00640 1 -- 172.21.15.46:0/48339626 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3050001120 con 0x7f305c071070 2024-03-29T08:23:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.006+0000 7f3061000640 1 -- 172.21.15.46:0/48339626 >> v1:172.21.15.73:6789/0 conn(0x7f305c071070 legacy=0x7f305c1594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.010+0000 7f3061000640 1 -- 172.21.15.46:0/48339626 >> v1:172.21.15.46:6789/0 conn(0x7f305c0b3850 legacy=0x7f305c0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:13.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.010+0000 7f3061000640 1 -- 172.21.15.46:0/48339626 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c0b23d0 con 0x7f305c15d480 2024-03-29T08:23:13.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.010+0000 7f3061000640 1 --2- 172.21.15.46:0/48339626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f305c15d480 0x7f305c070b30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f30480097c0 tx=0x7f304801c220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aab1365a060d86f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.010+0000 7f305be00640 1 -- 172.21.15.46:0/48339626 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050001810 con 0x7f305c15d480 2024-03-29T08:23:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.010+0000 7f305be00640 1 -- 172.21.15.46:0/48339626 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30480a7050 con 0x7f305c15d480 2024-03-29T08:23:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.010+0000 7f305be00640 1 -- 172.21.15.46:0/48339626 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30480ab620 con 0x7f305c15d480 2024-03-29T08:23:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.010+0000 7f3064400640 1 -- 172.21.15.46:0/48339626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f305c15d480 msgr2=0x7f305c070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:13.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.010+0000 7f3064400640 1 --2- 172.21.15.46:0/48339626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f305c15d480 0x7f305c070b30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f30480097c0 tx=0x7f304801c220 comp rx=0 tx=0).stop 2024-03-29T08:23:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.010+0000 7f3064400640 1 -- 172.21.15.46:0/48339626 shutdown_connections 2024-03-29T08:23:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.010+0000 7f3064400640 1 --2- 172.21.15.46:0/48339626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f305c15d480 0x7f305c070b30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.010+0000 7f3064400640 1 -- 172.21.15.46:0/48339626 >> 172.21.15.46:0/48339626 conn(0x7f305c066080 msgr2=0x7f305c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.010+0000 7f3064400640 1 -- 172.21.15.46:0/48339626 shutdown_connections 2024-03-29T08:23:13.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.010+0000 7f3064400640 1 -- 172.21.15.46:0/48339626 wait complete. 2024-03-29T08:23:13.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3064400640 1 Processor -- start 2024-03-29T08:23:13.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3064400640 1 -- start start 2024-03-29T08:23:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3064400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f305c071070 0x7f305c1557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3064400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f305c0b3850 0x7f305c15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3064400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f305c15d480 0x7f305c153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3064400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c15e470 con 0x7f305c0b3850 2024-03-29T08:23:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3064400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305c155d10 con 0x7f305c15d480 2024-03-29T08:23:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3064400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f305c1588b0 con 0x7f305c071070 2024-03-29T08:23:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3061000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f305c0b3850 0x7f305c15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3061000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f305c0b3850 0x7f305c15b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40588/0 (socket says 172.21.15.46:40588) 2024-03-29T08:23:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3061000640 1 -- 172.21.15.46:0/1981938621 learned_addr learned my addr 172.21.15.46:0/1981938621 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3062400640 1 --2- 172.21.15.46:0/1981938621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f305c15d480 0x7f305c153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3061a00640 1 --2- 172.21.15.46:0/1981938621 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f305c071070 0x7f305c1557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:13.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3061000640 1 -- 172.21.15.46:0/1981938621 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f305c071070 msgr2=0x7f305c1557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:13.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3061000640 1 --2- 172.21.15.46:0/1981938621 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f305c071070 0x7f305c1557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:13.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3061000640 1 -- 172.21.15.46:0/1981938621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f305c15d480 msgr2=0x7f305c153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:13.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3061000640 1 --2- 172.21.15.46:0/1981938621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f305c15d480 0x7f305c153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:13.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3061000640 1 -- 172.21.15.46:0/1981938621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3048009420 con 0x7f305c0b3850 2024-03-29T08:23:13.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3062400640 1 --2- 172.21.15.46:0/1981938621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f305c15d480 0x7f305c153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:13.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3061000640 1 --2- 172.21.15.46:0/1981938621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f305c0b3850 0x7f305c15b1a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f3048018e30 tx=0x7f304801aa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:13.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3059600640 1 -- 172.21.15.46:0/1981938621 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30480abd20 con 0x7f305c0b3850 2024-03-29T08:23:13.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3059600640 1 -- 172.21.15.46:0/1981938621 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30480b7450 con 0x7f305c0b3850 2024-03-29T08:23:13.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3059600640 1 -- 172.21.15.46:0/1981938621 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30480abea0 con 0x7f305c0b3850 2024-03-29T08:23:13.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3064400640 1 -- 172.21.15.46:0/1981938621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305c1575d0 con 0x7f305c0b3850 2024-03-29T08:23:13.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.014+0000 7f3064400640 1 -- 172.21.15.46:0/1981938621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305c157b90 con 0x7f305c0b3850 2024-03-29T08:23:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.018+0000 7f3064400640 1 -- 172.21.15.46:0/1981938621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f305c15e600 con 0x7f305c0b3850 2024-03-29T08:23:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.018+0000 7f3059600640 1 -- 172.21.15.46:0/1981938621 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f30480b7800 con 0x7f305c0b3850 2024-03-29T08:23:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.018+0000 7f3059600640 1 --2- 172.21.15.46:0/1981938621 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f303c066690 0x7f303c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.018+0000 7f3059600640 1 -- 172.21.15.46:0/1981938621 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f304811e880 con 0x7f305c0b3850 2024-03-29T08:23:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.018+0000 7f3061a00640 1 --2- 172.21.15.46:0/1981938621 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f303c066690 0x7f303c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.018+0000 7f3061a00640 1 --2- 172.21.15.46:0/1981938621 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f303c066690 0x7f303c068b50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3050000f70 tx=0x7f3050001e00 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:13.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.022+0000 7f3059600640 1 -- 172.21.15.46:0/1981938621 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f305c15e600 con 0x7f305c0b3850 2024-03-29T08:23:13.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.194+0000 7f3064400640 1 -- 172.21.15.46:0/1981938621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f305c15e600 con 0x7f305c0b3850 2024-03-29T08:23:13.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.194+0000 7f3059600640 1 -- 172.21.15.46:0/1981938621 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v80) v1 ==== 80+0+116 (secure 0 0 0) 0x7f30480b0090 con 0x7f305c0b3850 2024-03-29T08:23:13.199 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6835/9617787 2024-03-30T08:23:02.389981+0000 2024-03-29T08:23:13.199 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6834/9617787 2024-03-30T08:23:02.389981+0000 2024-03-29T08:23:13.200 INFO:teuthology.orchestra.run.smithi046.stderr:listed 2 entries 2024-03-29T08:23:13.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.198+0000 7f3064400640 1 -- 172.21.15.46:0/1981938621 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f303c066690 msgr2=0x7f303c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:13.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.198+0000 7f3064400640 1 --2- 172.21.15.46:0/1981938621 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f303c066690 0x7f303c068b50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3050000f70 tx=0x7f3050001e00 comp rx=0 tx=0).stop 2024-03-29T08:23:13.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.198+0000 7f3064400640 1 -- 172.21.15.46:0/1981938621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f305c0b3850 msgr2=0x7f305c15b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:13.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.198+0000 7f3064400640 1 --2- 172.21.15.46:0/1981938621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f305c0b3850 0x7f305c15b1a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f3048018e30 tx=0x7f304801aa60 comp rx=0 tx=0).stop 2024-03-29T08:23:13.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.198+0000 7f3064400640 1 -- 172.21.15.46:0/1981938621 shutdown_connections 2024-03-29T08:23:13.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.198+0000 7f3064400640 1 --2- 172.21.15.46:0/1981938621 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f303c066690 0x7f303c068b50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:13.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.198+0000 7f3064400640 1 --2- 172.21.15.46:0/1981938621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f305c15d480 0x7f305c153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:13.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.198+0000 7f3064400640 1 --2- 172.21.15.46:0/1981938621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f305c0b3850 0x7f305c15b1a0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:13.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.198+0000 7f3064400640 1 --2- 172.21.15.46:0/1981938621 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f305c071070 0x7f305c1557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:13.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.198+0000 7f3064400640 1 -- 172.21.15.46:0/1981938621 >> 172.21.15.46:0/1981938621 conn(0x7f305c066080 msgr2=0x7f305c15a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:13.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.198+0000 7f3064400640 1 -- 172.21.15.46:0/1981938621 shutdown_connections 2024-03-29T08:23:13.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:13.198+0000 7f3064400640 1 -- 172.21.15.46:0/1981938621 wait complete. 2024-03-29T08:23:13.220 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-03-29T08:23:13.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:13.220 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T08:23:13.238 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[42408]: fuse finished with error 0 and tester_r 0 2024-03-29T08:23:13.240 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:23:13.951 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-03-29T08:23:13.950+0000 7fc16e000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-29T08:23:19.342 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:23:19.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:19.342 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:23:19.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:19.354 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-03-29T08:23:19.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.550+0000 7f087f600640 1 Processor -- start 2024-03-29T08:23:19.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.550+0000 7f087f600640 1 -- start start 2024-03-29T08:23:19.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.550+0000 7f087f600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08780cec80 0x7f0878141770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.550+0000 7f087f600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0878141cb0 0x7f087806db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.550+0000 7f087f600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087806cd80 con 0x7f0878140f40 2024-03-29T08:23:19.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.550+0000 7f087f600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08780cbff0 con 0x7f0878141cb0 2024-03-29T08:23:19.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.550+0000 7f087f600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087806d060 con 0x7f08780cec80 2024-03-29T08:23:19.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f087cc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0878140f40 0x7f08780cb8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44326/0 (socket says 172.21.15.46:44326) 2024-03-29T08:23:19.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f087cc00640 1 -- 172.21.15.46:0/753671228 learned_addr learned my addr 172.21.15.46:0/753671228 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f087d600640 1 --2- 172.21.15.46:0/753671228 >> v2:172.21.15.73:3300/0 conn(0x7f0878141cb0 0x7f087806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f086fe00640 1 --2- 172.21.15.46:0/753671228 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08780cec80 0x7f0878141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f0877e00640 1 -- 172.21.15.46:0/753671228 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266789493 0 0) 0x7f087806cd80 con 0x7f0878140f40 2024-03-29T08:23:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f0877e00640 1 -- 172.21.15.46:0/753671228 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f085c003530 con 0x7f0878140f40 2024-03-29T08:23:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f0877e00640 1 -- 172.21.15.46:0/753671228 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3615284167 0 0) 0x7f085c003530 con 0x7f0878140f40 2024-03-29T08:23:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f0877e00640 1 -- 172.21.15.46:0/753671228 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08780cec80 msgr2=0x7f0878141770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f0877e00640 1 --2- 172.21.15.46:0/753671228 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08780cec80 0x7f0878141770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f0877e00640 1 -- 172.21.15.46:0/753671228 >> v2:172.21.15.73:3300/0 conn(0x7f0878141cb0 msgr2=0x7f087806db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f0877e00640 1 --2- 172.21.15.46:0/753671228 >> v2:172.21.15.73:3300/0 conn(0x7f0878141cb0 0x7f087806db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f0877e00640 1 -- 172.21.15.46:0/753671228 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087806d340 con 0x7f0878140f40 2024-03-29T08:23:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f0877e00640 1 -- 172.21.15.46:0/753671228 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f0868001680 con 0x7f0878140f40 2024-03-29T08:23:19.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f0877e00640 1 -- 172.21.15.46:0/753671228 >> v1:172.21.15.46:6789/0 conn(0x7f0878140f40 legacy=0x7f08780cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f0877e00640 1 --2- 172.21.15.46:0/753671228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c003980 0x7f085c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f0877e00640 1 --2- 172.21.15.46:0/753671228 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08780cec80 0x7f085c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f0877e00640 1 --2- 172.21.15.46:0/753671228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c004d10 0x7f085c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f0877e00640 1 -- 172.21.15.46:0/753671228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087806d060 con 0x7f085c003980 2024-03-29T08:23:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f0877e00640 1 -- 172.21.15.46:0/753671228 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c00b990 con 0x7f085c004d10 2024-03-29T08:23:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f0877e00640 1 -- 172.21.15.46:0/753671228 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085c00bad0 con 0x7f08780cec80 2024-03-29T08:23:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f087cc00640 1 --2- 172.21.15.46:0/753671228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c003980 0x7f085c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f086fe00640 1 --2- 172.21.15.46:0/753671228 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08780cec80 0x7f085c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f087d600640 1 --2- 172.21.15.46:0/753671228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c004d10 0x7f085c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f087cc00640 1 -- 172.21.15.46:0/753671228 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08780cec80 msgr2=0x7f085c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f087cc00640 1 --2- 172.21.15.46:0/753671228 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08780cec80 0x7f085c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f087cc00640 1 -- 172.21.15.46:0/753671228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c004d10 msgr2=0x7f085c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f087cc00640 1 --2- 172.21.15.46:0/753671228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f085c004d10 0x7f085c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.554+0000 7f087cc00640 1 -- 172.21.15.46:0/753671228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085c003530 con 0x7f085c003980 2024-03-29T08:23:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f087cc00640 1 --2- 172.21.15.46:0/753671228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c003980 0x7f085c003da0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f086800c110 tx=0x7f086801cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c66b2f7c37ba1a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f0877e00640 1 -- 172.21.15.46:0/753671228 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08680058e0 con 0x7f085c003980 2024-03-29T08:23:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f0877e00640 1 -- 172.21.15.46:0/753671228 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0868005a60 con 0x7f085c003980 2024-03-29T08:23:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f0877e00640 1 -- 172.21.15.46:0/753671228 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0868005e10 con 0x7f085c003980 2024-03-29T08:23:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f087f600640 1 -- 172.21.15.46:0/753671228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c003980 msgr2=0x7f085c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f087f600640 1 --2- 172.21.15.46:0/753671228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c003980 0x7f085c003da0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f086800c110 tx=0x7f086801cb90 comp rx=0 tx=0).stop 2024-03-29T08:23:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f087d600640 1 -- 172.21.15.46:0/753671228 reap_dead start 2024-03-29T08:23:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f087f600640 1 -- 172.21.15.46:0/753671228 shutdown_connections 2024-03-29T08:23:19.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f087f600640 1 -- 172.21.15.46:0/753671228 >> 172.21.15.46:0/753671228 conn(0x7f08780b42d0 msgr2=0x7f0878069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f087f600640 1 -- 172.21.15.46:0/753671228 shutdown_connections 2024-03-29T08:23:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f087f600640 1 -- 172.21.15.46:0/753671228 wait complete. 2024-03-29T08:23:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f087f600640 1 Processor -- start 2024-03-29T08:23:19.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f087f600640 1 -- start start 2024-03-29T08:23:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f087f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c003980 0x7f08780acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f087f600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08781f3220 0x7f0878134f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f087f600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0878135480 0x7f087813dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f087f600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0878135ee0 con 0x7f085c003980 2024-03-29T08:23:19.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f087f600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0878136970 con 0x7f08781f3220 2024-03-29T08:23:19.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.558+0000 7f087f600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08780ad310 con 0x7f0878135480 2024-03-29T08:23:19.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.562+0000 7f087cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c003980 0x7f08780acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.562+0000 7f087cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c003980 0x7f08780acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51530/0 (socket says 172.21.15.46:51530) 2024-03-29T08:23:19.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.562+0000 7f087cc00640 1 -- 172.21.15.46:0/1134804267 learned_addr learned my addr 172.21.15.46:0/1134804267 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:19.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.562+0000 7f087d600640 1 --2- 172.21.15.46:0/1134804267 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0878135480 0x7f087813dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.562+0000 7f086fe00640 1 --2- 172.21.15.46:0/1134804267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08781f3220 0x7f0878134f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.562+0000 7f087cc00640 1 -- 172.21.15.46:0/1134804267 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0878135480 msgr2=0x7f087813dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.562+0000 7f087cc00640 1 --2- 172.21.15.46:0/1134804267 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0878135480 0x7f087813dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.562+0000 7f087cc00640 1 -- 172.21.15.46:0/1134804267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08781f3220 msgr2=0x7f0878134f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.562+0000 7f087cc00640 1 --2- 172.21.15.46:0/1134804267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08781f3220 0x7f0878134f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.562+0000 7f087cc00640 1 -- 172.21.15.46:0/1134804267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0878135920 con 0x7f085c003980 2024-03-29T08:23:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.562+0000 7f087cc00640 1 --2- 172.21.15.46:0/1134804267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c003980 0x7f08780acdd0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0868007040 tx=0x7f085c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.562+0000 7f0875600640 1 -- 172.21.15.46:0/1134804267 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08680047f0 con 0x7f085c003980 2024-03-29T08:23:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.562+0000 7f0875600640 1 -- 172.21.15.46:0/1134804267 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0868004950 con 0x7f085c003980 2024-03-29T08:23:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.562+0000 7f0875600640 1 -- 172.21.15.46:0/1134804267 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0868004d00 con 0x7f085c003980 2024-03-29T08:23:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.562+0000 7f087f600640 1 -- 172.21.15.46:0/1134804267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0878135c00 con 0x7f085c003980 2024-03-29T08:23:19.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.562+0000 7f087f600640 1 -- 172.21.15.46:0/1134804267 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087813b590 con 0x7f085c003980 2024-03-29T08:23:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.562+0000 7f0875600640 1 -- 172.21.15.46:0/1134804267 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f08680070d0 con 0x7f085c003980 2024-03-29T08:23:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0875600640 1 --2- 172.21.15.46:0/1134804267 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0854066690 0x7f0854068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0875600640 1 -- 172.21.15.46:0/1134804267 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f086811d710 con 0x7f085c003980 2024-03-29T08:23:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0882ad1000 1 Processor -- start 2024-03-29T08:23:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f086fe00640 1 --2- 172.21.15.46:0/1134804267 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0854066690 0x7f0854068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0882ad1000 1 -- start start 2024-03-29T08:23:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f086fe00640 1 --2- 172.21.15.46:0/1134804267 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0854066690 0x7f0854068b50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f087813e760 tx=0x7f0860009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0882ad1000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da6d00 0x55ceb5da70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0882ad1000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5da7620 0x55ceb5d5d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0882ad1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5d5d640 0x55ceb5d5da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0882ad1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb5c213a0 con 0x55ceb5d5d640 2024-03-29T08:23:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0882ad1000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb5baeab0 con 0x55ceb5da7620 2024-03-29T08:23:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0882ad1000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ceb5bac080 con 0x55ceb5da6d00 2024-03-29T08:23:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f087d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5d5d640 0x55ceb5d5da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f087d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5d5d640 0x55ceb5d5da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51536/0 (socket says 172.21.15.46:51536) 2024-03-29T08:23:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f087d600640 1 -- 172.21.15.46:0/1858526637 learned_addr learned my addr 172.21.15.46:0/1858526637 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f086fe00640 1 --2- 172.21.15.46:0/1858526637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5da7620 0x55ceb5d5d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f087cc00640 1 --2- 172.21.15.46:0/1858526637 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da6d00 0x55ceb5da70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f087d600640 1 -- 172.21.15.46:0/1858526637 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da6d00 msgr2=0x55ceb5da70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f087d600640 1 --2- 172.21.15.46:0/1858526637 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da6d00 0x55ceb5da70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f087d600640 1 -- 172.21.15.46:0/1858526637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5da7620 msgr2=0x55ceb5d5d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f087d600640 1 --2- 172.21.15.46:0/1858526637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5da7620 0x55ceb5d5d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f087d600640 1 -- 172.21.15.46:0/1858526637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ceb58a4000 con 0x55ceb5d5d640 2024-03-29T08:23:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f087d600640 1 --2- 172.21.15.46:0/1858526637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5d5d640 0x55ceb5d5da20 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f08780ccca0 tx=0x7f0870009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13807f39e43fca6c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f086d600640 1 -- 172.21.15.46:0/1858526637 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000e040 con 0x55ceb5d5d640 2024-03-29T08:23:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f086d600640 1 -- 172.21.15.46:0/1858526637 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087000f040 con 0x55ceb5d5d640 2024-03-29T08:23:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f086d600640 1 -- 172.21.15.46:0/1858526637 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870003da0 con 0x55ceb5d5d640 2024-03-29T08:23:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0882ad1000 1 -- 172.21.15.46:0/1858526637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5d5d640 msgr2=0x55ceb5d5da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0882ad1000 1 --2- 172.21.15.46:0/1858526637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5d5d640 0x55ceb5d5da20 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f08780ccca0 tx=0x7f0870009750 comp rx=0 tx=0).stop 2024-03-29T08:23:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0882ad1000 1 -- 172.21.15.46:0/1858526637 shutdown_connections 2024-03-29T08:23:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0882ad1000 1 --2- 172.21.15.46:0/1858526637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5da7620 0x55ceb5d5d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0882ad1000 1 --2- 172.21.15.46:0/1858526637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5d5d640 0x55ceb5d5da20 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0882ad1000 1 --2- 172.21.15.46:0/1858526637 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da6d00 0x55ceb5da70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0882ad1000 1 -- 172.21.15.46:0/1858526637 >> 172.21.15.46:0/1858526637 conn(0x55ceb5d5f5a0 msgr2=0x55ceb5d575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0882ad1000 1 -- 172.21.15.46:0/1858526637 shutdown_connections 2024-03-29T08:23:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.566+0000 7f0882ad1000 1 -- 172.21.15.46:0/1858526637 wait complete. 2024-03-29T08:23:19.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 1 -- start start 2024-03-29T08:23:19.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 1 Processor -- start 2024-03-29T08:23:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5d5d640 0x55ceb5d94310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da6d00 0x55ceb5d94850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5da7620 0x55ceb5d94d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb5ba7970 con 0x55ceb5d5d640 2024-03-29T08:23:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb5b47a00 con 0x55ceb5da7620 2024-03-29T08:23:19.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ceb5b476a0 con 0x55ceb5da6d00 2024-03-29T08:23:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 10 client.0 _ll_register_callbacks cb 0x55ceb5d44bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:23:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f087cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5d5d640 0x55ceb5d94310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f087cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5d5d640 0x55ceb5d94310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51548/0 (socket says 172.21.15.46:51548) 2024-03-29T08:23:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f087cc00640 1 -- 172.21.15.46:0/2199789331 learned_addr learned my addr 172.21.15.46:0/2199789331 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f086fe00640 1 --2- 172.21.15.46:0/2199789331 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da6d00 0x55ceb5d94850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f087d600640 1 --2- 172.21.15.46:0/2199789331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5da7620 0x55ceb5d94d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f087cc00640 1 -- 172.21.15.46:0/2199789331 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da6d00 msgr2=0x55ceb5d94850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f087cc00640 1 --2- 172.21.15.46:0/2199789331 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da6d00 0x55ceb5d94850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f087cc00640 1 -- 172.21.15.46:0/2199789331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5da7620 msgr2=0x55ceb5d94d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f087cc00640 1 --2- 172.21.15.46:0/2199789331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5da7620 0x55ceb5d94d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f087cc00640 1 -- 172.21.15.46:0/2199789331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0870009cc0 con 0x55ceb5d5d640 2024-03-29T08:23:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f086fe00640 1 --2- 172.21.15.46:0/2199789331 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da6d00 0x55ceb5d94850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f087cc00640 1 --2- 172.21.15.46:0/2199789331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5d5d640 0x55ceb5d94310 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f086801cbd0 tx=0x7f0868004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c0a04c09b8f2a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0866000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:23:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0866000640 1 -- 172.21.15.46:0/2199789331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08680f2440 con 0x55ceb5d5d640 2024-03-29T08:23:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0866000640 1 -- 172.21.15.46:0/2199789331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f08680f6a00 con 0x55ceb5d5d640 2024-03-29T08:23:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 1 -- 172.21.15.46:0/2199789331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ceb5b57370 con 0x55ceb5d5d640 2024-03-29T08:23:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0866000640 1 -- 172.21.15.46:0/2199789331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08680f2160 con 0x55ceb5d5d640 2024-03-29T08:23:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 10 client.7035 fetch_fsmap learned FSMap version 48 2024-03-29T08:23:19.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 1 -- 172.21.15.46:0/2199789331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55ceb5b6c8f0 con 0x55ceb5d5d640 2024-03-29T08:23:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0866000640 1 -- 172.21.15.46:0/2199789331 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ceb5b57370 con 0x55ceb5d5d640 2024-03-29T08:23:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0866000640 1 -- 172.21.15.46:0/2199789331 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 48) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f08680c1020 con 0x55ceb5d5d640 2024-03-29T08:23:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 10 client.7035 fetch_fsmap finished waiting for FSMap version 48 2024-03-29T08:23:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 10 client.7035 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-29T08:23:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 20 client.7035 populate_metadata read hostname 'smithi046' 2024-03-29T08:23:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 1 --2- 172.21.15.46:0/2199789331 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55ceb5db5d90 0x55ceb5db8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 4 client.7035 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:23:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f0882ad1000 1 -- 172.21.15.46:0/2199789331 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ceb5ba62c0 con 0x55ceb5db5d90 2024-03-29T08:23:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.570+0000 7f086fe00640 1 --2- 172.21.15.46:0/2199789331 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55ceb5db5d90 0x55ceb5db8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.574+0000 7f086fe00640 1 --2- 172.21.15.46:0/2199789331 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55ceb5db5d90 0x55ceb5db8180 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a21edf02a94ed7 server_cookie=59c859aff9d05f91 in_seq=0 out_seq=0 2024-03-29T08:23:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.574+0000 7f0866000640 10 client.7035 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.574+0000 7f0866000640 1 -- 172.21.15.46:0/2199789331 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55ceb5ba62c0 con 0x55ceb5db5d90 2024-03-29T08:23:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.574+0000 7f0866000640 10 client.7035 handle_command_reply: tid=0 2024-03-29T08:23:19.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.574+0000 7f0882ad1000 1 client.7035 shutdown 2024-03-29T08:23:19.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.574+0000 7f0882ad1000 1 -- 172.21.15.46:0/2199789331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5d5d640 msgr2=0x55ceb5d94310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.574+0000 7f0882ad1000 1 --2- 172.21.15.46:0/2199789331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5d5d640 0x55ceb5d94310 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f086801cbd0 tx=0x7f0868004f90 comp rx=0 tx=0).stop 2024-03-29T08:23:19.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.574+0000 7f0882ad1000 1 -- 172.21.15.46:0/2199789331 shutdown_connections 2024-03-29T08:23:19.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.574+0000 7f0882ad1000 1 --2- 172.21.15.46:0/2199789331 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55ceb5db5d90 0x55ceb5db8180 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.574+0000 7f0882ad1000 1 --2- 172.21.15.46:0/2199789331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5da7620 0x55ceb5d94d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.574+0000 7f0882ad1000 1 --2- 172.21.15.46:0/2199789331 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da6d00 0x55ceb5d94850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.574+0000 7f0882ad1000 1 --2- 172.21.15.46:0/2199789331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5d5d640 0x55ceb5d94310 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.574+0000 7f0866000640 0 client.7035 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:19.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.574+0000 7f0882ad1000 1 -- 172.21.15.46:0/2199789331 >> 172.21.15.46:0/2199789331 conn(0x55ceb5d5f5a0 msgr2=0x55ceb5d5ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:19.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.574+0000 7f0882ad1000 1 -- 172.21.15.46:0/2199789331 shutdown_connections 2024-03-29T08:23:19.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.574+0000 7f0882ad1000 1 -- 172.21.15.46:0/2199789331 wait complete. 2024-03-29T08:23:19.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.574+0000 7f0882ad1000 20 client.7035 trim_cache size 0 max 16384 2024-03-29T08:23:19.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 Processor -- start 2024-03-29T08:23:19.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 -- start start 2024-03-29T08:23:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5d5d640 0x55ceb5d5ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5da6d00 0x55ceb5d56d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da7620 0x55ceb5d57280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb5864c80 con 0x55ceb5da6d00 2024-03-29T08:23:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb5ba3160 con 0x55ceb5d5d640 2024-03-29T08:23:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ceb5dbcb10 con 0x55ceb5da7620 2024-03-29T08:23:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f086fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5da6d00 0x55ceb5d56d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f086fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5da6d00 0x55ceb5d56d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51550/0 (socket says 172.21.15.46:51550) 2024-03-29T08:23:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f086fe00640 1 -- 172.21.15.46:0/4127923867 learned_addr learned my addr 172.21.15.46:0/4127923867 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f086fe00640 1 -- 172.21.15.46:0/4127923867 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da7620 msgr2=0x55ceb5d57280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f086fe00640 1 --2- 172.21.15.46:0/4127923867 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da7620 0x55ceb5d57280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f086fe00640 1 -- 172.21.15.46:0/4127923867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5d5d640 msgr2=0x55ceb5d5ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f086fe00640 1 --2- 172.21.15.46:0/4127923867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5d5d640 0x55ceb5d5ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f086fe00640 1 -- 172.21.15.46:0/4127923867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ceb5b52c40 con 0x55ceb5da6d00 2024-03-29T08:23:19.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f086fe00640 1 --2- 172.21.15.46:0/4127923867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5da6d00 0x55ceb5d56d40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f0860004360 tx=0x7f0860017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fb1a2b66fa2625 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f086e000640 1 -- 172.21.15.46:0/4127923867 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860004450 con 0x55ceb5da6d00 2024-03-29T08:23:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f086e000640 1 -- 172.21.15.46:0/4127923867 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0860027040 con 0x55ceb5da6d00 2024-03-29T08:23:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f086e000640 1 -- 172.21.15.46:0/4127923867 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860003620 con 0x55ceb5da6d00 2024-03-29T08:23:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 -- 172.21.15.46:0/4127923867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5da6d00 msgr2=0x55ceb5d56d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 --2- 172.21.15.46:0/4127923867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5da6d00 0x55ceb5d56d40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f0860004360 tx=0x7f0860017000 comp rx=0 tx=0).stop 2024-03-29T08:23:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 -- 172.21.15.46:0/4127923867 shutdown_connections 2024-03-29T08:23:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 --2- 172.21.15.46:0/4127923867 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da7620 0x55ceb5d57280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 --2- 172.21.15.46:0/4127923867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5da6d00 0x55ceb5d56d40 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 --2- 172.21.15.46:0/4127923867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5d5d640 0x55ceb5d5ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 -- 172.21.15.46:0/4127923867 >> 172.21.15.46:0/4127923867 conn(0x55ceb5d5f5a0 msgr2=0x55ceb5d92cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 -- 172.21.15.46:0/4127923867 shutdown_connections 2024-03-29T08:23:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 -- 172.21.15.46:0/4127923867 wait complete. 2024-03-29T08:23:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 -- start start 2024-03-29T08:23:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.590+0000 7f0882ad1000 1 Processor -- start 2024-03-29T08:23:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0882ad1000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5d5d640 0x55ceb5db9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0882ad1000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da6d00 0x55ceb5e05290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0882ad1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5da7620 0x55ceb5e057d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0882ad1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb5b50270 con 0x55ceb5da7620 2024-03-29T08:23:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0882ad1000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ceb5b65b20 con 0x55ceb5d5d640 2024-03-29T08:23:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0882ad1000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ceb5beb2e0 con 0x55ceb5da6d00 2024-03-29T08:23:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0882ad1000 10 client.0 _ll_register_callbacks cb 0x55ceb5b46110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:23:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f087d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5da7620 0x55ceb5e057d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f087d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5da7620 0x55ceb5e057d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51560/0 (socket says 172.21.15.46:51560) 2024-03-29T08:23:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f087d600640 1 -- 172.21.15.46:0/1022760379 learned_addr learned my addr 172.21.15.46:0/1022760379 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:19.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f087cc00640 1 --2- 172.21.15.46:0/1022760379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5d5d640 0x55ceb5db9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f086fe00640 1 --2- 172.21.15.46:0/1022760379 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da6d00 0x55ceb5e05290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f087d600640 1 -- 172.21.15.46:0/1022760379 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da6d00 msgr2=0x55ceb5e05290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f087d600640 1 --2- 172.21.15.46:0/1022760379 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da6d00 0x55ceb5e05290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f087d600640 1 -- 172.21.15.46:0/1022760379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5d5d640 msgr2=0x55ceb5db9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f087d600640 1 --2- 172.21.15.46:0/1022760379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5d5d640 0x55ceb5db9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f087d600640 1 -- 172.21.15.46:0/1022760379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0860014dc0 con 0x55ceb5da7620 2024-03-29T08:23:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f087d600640 1 --2- 172.21.15.46:0/1022760379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5da7620 0x55ceb5e057d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f08781420b0 tx=0x7f0870004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74acc8872490cbf2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0866000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:23:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0866000640 1 -- 172.21.15.46:0/1022760379 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000e040 con 0x55ceb5da7620 2024-03-29T08:23:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0866000640 1 -- 172.21.15.46:0/1022760379 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f087001a070 con 0x55ceb5da7620 2024-03-29T08:23:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0882ad1000 1 -- 172.21.15.46:0/1022760379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ceb5ba4750 con 0x55ceb5da7620 2024-03-29T08:23:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0866000640 1 -- 172.21.15.46:0/1022760379 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0870003e50 con 0x55ceb5da7620 2024-03-29T08:23:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0882ad1000 10 client.7041 fetch_fsmap learned FSMap version 48 2024-03-29T08:23:19.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0882ad1000 1 -- 172.21.15.46:0/1022760379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55ceb5b46970 con 0x55ceb5da7620 2024-03-29T08:23:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0866000640 1 -- 172.21.15.46:0/1022760379 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ceb5ba4750 con 0x55ceb5da7620 2024-03-29T08:23:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0866000640 1 -- 172.21.15.46:0/1022760379 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 48) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0870005d20 con 0x55ceb5da7620 2024-03-29T08:23:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0882ad1000 10 client.7041 fetch_fsmap finished waiting for FSMap version 48 2024-03-29T08:23:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0882ad1000 10 client.7041 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-29T08:23:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0882ad1000 20 client.7041 populate_metadata read hostname 'smithi046' 2024-03-29T08:23:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0882ad1000 1 --2- 172.21.15.46:0/1022760379 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55ceb5dc1540 0x55ceb5dc1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0882ad1000 4 client.7041 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-03-29T08:23:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0882ad1000 1 -- 172.21.15.46:0/1022760379 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) v1 -- 0x55ceb5b6fed0 con 0x55ceb5dc1540 2024-03-29T08:23:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f087cc00640 1 --2- 172.21.15.46:0/1022760379 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55ceb5dc1540 0x55ceb5dc1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f087cc00640 1 --2- 172.21.15.46:0/1022760379 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55ceb5dc1540 0x55ceb5dc1950 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=673d043c10cf9f7c server_cookie=592cb8d2a982c810 in_seq=0 out_seq=0 2024-03-29T08:23:19.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.594+0000 7f0866000640 10 client.7041 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f0866000640 1 -- 172.21.15.46:0/1022760379 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ceb5b6fed0 con 0x55ceb5dc1540 2024-03-29T08:23:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f0866000640 10 client.7041 handle_command_reply: tid=0 2024-03-29T08:23:19.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f0882ad1000 1 client.7041 shutdown 2024-03-29T08:23:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f0882ad1000 1 -- 172.21.15.46:0/1022760379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5da7620 msgr2=0x55ceb5e057d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f0882ad1000 1 --2- 172.21.15.46:0/1022760379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5da7620 0x55ceb5e057d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f08781420b0 tx=0x7f0870004a50 comp rx=0 tx=0).stop 2024-03-29T08:23:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f0882ad1000 1 -- 172.21.15.46:0/1022760379 shutdown_connections 2024-03-29T08:23:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f0882ad1000 1 --2- 172.21.15.46:0/1022760379 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55ceb5dc1540 0x55ceb5dc1950 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f0882ad1000 1 --2- 172.21.15.46:0/1022760379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ceb5da7620 0x55ceb5e057d0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f0882ad1000 1 --2- 172.21.15.46:0/1022760379 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ceb5da6d00 0x55ceb5e05290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f0882ad1000 1 --2- 172.21.15.46:0/1022760379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ceb5d5d640 0x55ceb5db9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f0882ad1000 1 -- 172.21.15.46:0/1022760379 >> 172.21.15.46:0/1022760379 conn(0x55ceb5d5f5a0 msgr2=0x55ceb5d933b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f0866000640 0 client.7041 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f0882ad1000 1 -- 172.21.15.46:0/1022760379 shutdown_connections 2024-03-29T08:23:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f0882ad1000 1 -- 172.21.15.46:0/1022760379 wait complete. 2024-03-29T08:23:19.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f0882ad1000 20 client.7041 trim_cache size 0 max 16384 2024-03-29T08:23:19.605 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:23:19.605 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-29T08:23:19.605 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "444816b4-a10f-4031-8a61-decf5cbfdf81", 2024-03-29T08:23:19.605 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-29T08:23:19.605 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:23:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f087f600640 1 -- 172.21.15.46:0/1134804267 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0854066690 msgr2=0x7f0854068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f087f600640 1 --2- 172.21.15.46:0/1134804267 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0854066690 0x7f0854068b50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f087813e760 tx=0x7f0860009190 comp rx=0 tx=0).stop 2024-03-29T08:23:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f087f600640 1 -- 172.21.15.46:0/1134804267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c003980 msgr2=0x7f08780acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f087f600640 1 --2- 172.21.15.46:0/1134804267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c003980 0x7f08780acdd0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0868007040 tx=0x7f085c005800 comp rx=0 tx=0).stop 2024-03-29T08:23:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f087f600640 1 -- 172.21.15.46:0/1134804267 shutdown_connections 2024-03-29T08:23:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f087f600640 1 --2- 172.21.15.46:0/1134804267 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0854066690 0x7f0854068b50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f087f600640 1 --2- 172.21.15.46:0/1134804267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08781f3220 0x7f0878134f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f087f600640 1 --2- 172.21.15.46:0/1134804267 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0878135480 0x7f087813dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f087f600640 1 --2- 172.21.15.46:0/1134804267 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f085c003980 0x7f08780acdd0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f087f600640 1 -- 172.21.15.46:0/1134804267 >> 172.21.15.46:0/1134804267 conn(0x7f08780b42d0 msgr2=0x7f0878143ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:19.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f087f600640 1 -- 172.21.15.46:0/1134804267 shutdown_connections 2024-03-29T08:23:19.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.598+0000 7f087f600640 1 -- 172.21.15.46:0/1134804267 wait complete. 2024-03-29T08:23:19.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:19.620 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-29T08:23:19.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.758+0000 7fefb6200640 1 Processor -- start 2024-03-29T08:23:19.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.758+0000 7fefb6200640 1 -- start start 2024-03-29T08:23:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.762+0000 7fefb6200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefb00cea30 0x7fefb01606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.762+0000 7fefb6200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fefb015cd30 0x7fefb0160be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.762+0000 7fefb6200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb00cfb20 con 0x7fefb015c950 2024-03-29T08:23:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.762+0000 7fefb6200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb00cef10 con 0x7fefb015cd30 2024-03-29T08:23:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.762+0000 7fefb6200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb00cf1f0 con 0x7fefb00cea30 2024-03-29T08:23:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.762+0000 7fefaf400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fefb015c950 0x7fefb006bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:44342/0 (socket says 172.21.15.46:44342) 2024-03-29T08:23:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.762+0000 7fefaf400640 1 -- 172.21.15.46:0/4144071545 learned_addr learned my addr 172.21.15.46:0/4144071545 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:19.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.762+0000 7fefafe00640 1 --2- 172.21.15.46:0/4144071545 >> v2:172.21.15.73:3300/0 conn(0x7fefb015cd30 0x7fefb0160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.762+0000 7fefaea00640 1 --2- 172.21.15.46:0/4144071545 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefb00cea30 0x7fefb01606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.762+0000 7fefae000640 1 -- 172.21.15.46:0/4144071545 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36002189 0 0) 0x7fefb00cfb20 con 0x7fefb015c950 2024-03-29T08:23:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.762+0000 7fefae000640 1 -- 172.21.15.46:0/4144071545 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef94003530 con 0x7fefb015c950 2024-03-29T08:23:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.762+0000 7fefafe00640 1 -- 172.21.15.46:0/4144071545 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefb00cea30 msgr2=0x7fefb01606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.762+0000 7fefafe00640 1 --2- 172.21.15.46:0/4144071545 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefb00cea30 0x7fefb01606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.762+0000 7fefafe00640 1 -- 172.21.15.46:0/4144071545 >> v1:172.21.15.46:6789/0 conn(0x7fefb015c950 legacy=0x7fefb006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.762+0000 7fefafe00640 1 -- 172.21.15.46:0/4144071545 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb00cf4d0 con 0x7fefb015cd30 2024-03-29T08:23:19.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.766+0000 7fefafe00640 1 --2- 172.21.15.46:0/4144071545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb015cd30 0x7fefb0160be0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009d90 tx=0x7fefa801f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e54797e1e96e0900 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.766+0000 7fefae000640 1 -- 172.21.15.46:0/4144071545 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa80af600 con 0x7fefb015cd30 2024-03-29T08:23:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.766+0000 7fefae000640 1 -- 172.21.15.46:0/4144071545 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa80ab050 con 0x7fefb015cd30 2024-03-29T08:23:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.766+0000 7fefae000640 1 -- 172.21.15.46:0/4144071545 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa80af960 con 0x7fefb015cd30 2024-03-29T08:23:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.766+0000 7fefb6200640 1 -- 172.21.15.46:0/4144071545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb015cd30 msgr2=0x7fefb0160be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.766+0000 7fefb6200640 1 --2- 172.21.15.46:0/4144071545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb015cd30 0x7fefb0160be0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009d90 tx=0x7fefa801f150 comp rx=0 tx=0).stop 2024-03-29T08:23:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.766+0000 7fefb6200640 1 -- 172.21.15.46:0/4144071545 shutdown_connections 2024-03-29T08:23:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.766+0000 7fefb6200640 1 --2- 172.21.15.46:0/4144071545 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefb00cea30 0x7fefb01606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.766+0000 7fefb6200640 1 --2- 172.21.15.46:0/4144071545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb015cd30 0x7fefb0160be0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.766+0000 7fefb6200640 1 -- 172.21.15.46:0/4144071545 >> 172.21.15.46:0/4144071545 conn(0x7fefb00b21b0 msgr2=0x7fefb00b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.766+0000 7fefb6200640 1 -- 172.21.15.46:0/4144071545 shutdown_connections 2024-03-29T08:23:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.766+0000 7fefb6200640 1 -- 172.21.15.46:0/4144071545 wait complete. 2024-03-29T08:23:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.766+0000 7fefb6200640 1 Processor -- start 2024-03-29T08:23:19.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.766+0000 7fefb6200640 1 -- start start 2024-03-29T08:23:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefb6200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb00cea30 0x7fefb01596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefb6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb015c950 0x7fefb015f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefb6200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefb015cd30 0x7fefb00b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefb6200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb0159c20 con 0x7fefb015c950 2024-03-29T08:23:19.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefb6200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb009c720 con 0x7fefb00cea30 2024-03-29T08:23:19.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefb6200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb009ca00 con 0x7fefb015cd30 2024-03-29T08:23:19.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefaea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb015c950 0x7fefb015f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefaea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb015c950 0x7fefb015f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51568/0 (socket says 172.21.15.46:51568) 2024-03-29T08:23:19.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefaea00640 1 -- 172.21.15.46:0/3019130096 learned_addr learned my addr 172.21.15.46:0/3019130096 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:19.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefaf400640 1 --2- 172.21.15.46:0/3019130096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb00cea30 0x7fefb01596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefafe00640 1 --2- 172.21.15.46:0/3019130096 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefb015cd30 0x7fefb00b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefaea00640 1 -- 172.21.15.46:0/3019130096 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefb015cd30 msgr2=0x7fefb00b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefaea00640 1 --2- 172.21.15.46:0/3019130096 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefb015cd30 0x7fefb00b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefaea00640 1 -- 172.21.15.46:0/3019130096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb00cea30 msgr2=0x7fefb01596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefaea00640 1 --2- 172.21.15.46:0/3019130096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb00cea30 0x7fefb01596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefaea00640 1 -- 172.21.15.46:0/3019130096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa8009a70 con 0x7fefb015c950 2024-03-29T08:23:19.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefaea00640 1 --2- 172.21.15.46:0/3019130096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb015c950 0x7fefb015f470 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00e930 tx=0x7fef9c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefa7400640 1 -- 172.21.15.46:0/3019130096 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c009690 con 0x7fefb015c950 2024-03-29T08:23:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefa7400640 1 -- 172.21.15.46:0/3019130096 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9c00ca00 con 0x7fefb015c950 2024-03-29T08:23:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefb6200640 1 -- 172.21.15.46:0/3019130096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb006c470 con 0x7fefb015c950 2024-03-29T08:23:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefa7400640 1 -- 172.21.15.46:0/3019130096 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00cd70 con 0x7fefb015c950 2024-03-29T08:23:19.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefb6200640 1 -- 172.21.15.46:0/3019130096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb009cfc0 con 0x7fefb015c950 2024-03-29T08:23:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.770+0000 7fefa7400640 1 -- 172.21.15.46:0/3019130096 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fef9c013470 con 0x7fefb015c950 2024-03-29T08:23:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefa7400640 1 --2- 172.21.15.46:0/3019130096 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fef90066690 0x7fef90068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefa7400640 1 -- 172.21.15.46:0/3019130096 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fef9c0844e0 con 0x7fefb015c950 2024-03-29T08:23:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefaf400640 1 --2- 172.21.15.46:0/3019130096 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fef90066690 0x7fef90068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefb967f000 1 Processor -- start 2024-03-29T08:23:19.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefb967f000 1 -- start start 2024-03-29T08:23:19.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefaf400640 1 --2- 172.21.15.46:0/3019130096 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fef90066690 0x7fef90068b50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fefa0007fd0 tx=0x7fefa0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:19.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefb967f000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f5a70 0x55b3462f5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefb967f000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462f6390 0x55b3462a94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefb967f000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462a9a00 0x55b3462a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefb967f000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b346174630 con 0x55b3462f6390 2024-03-29T08:23:19.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefb967f000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b346101bd0 con 0x55b3462f5a70 2024-03-29T08:23:19.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefb967f000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3460ff1a0 con 0x55b3462a9a00 2024-03-29T08:23:19.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefaea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462f6390 0x55b3462a94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefaea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462f6390 0x55b3462a94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51576/0 (socket says 172.21.15.46:51576) 2024-03-29T08:23:19.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefaea00640 1 -- 172.21.15.46:0/3233148197 learned_addr learned my addr 172.21.15.46:0/3233148197 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:19.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefaf400640 1 --2- 172.21.15.46:0/3233148197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f5a70 0x55b3462f5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefafe00640 1 --2- 172.21.15.46:0/3233148197 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462a9a00 0x55b3462a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefaea00640 1 -- 172.21.15.46:0/3233148197 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462a9a00 msgr2=0x55b3462a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefaea00640 1 --2- 172.21.15.46:0/3233148197 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462a9a00 0x55b3462a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefaea00640 1 -- 172.21.15.46:0/3233148197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f5a70 msgr2=0x55b3462f5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefaea00640 1 --2- 172.21.15.46:0/3233148197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f5a70 0x55b3462f5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefaea00640 1 -- 172.21.15.46:0/3233148197 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b345df6e20 con 0x55b3462f6390 2024-03-29T08:23:19.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefaf400640 1 --2- 172.21.15.46:0/3233148197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f5a70 0x55b3462f5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:19.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefaea00640 1 --2- 172.21.15.46:0/3233148197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462f6390 0x55b3462a94c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fef9c082850 tx=0x7fef9c054fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9d2c5ca92430ed server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fef9be00640 1 -- 172.21.15.46:0/3233148197 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c0538f0 con 0x55b3462f6390 2024-03-29T08:23:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fef9be00640 1 -- 172.21.15.46:0/3233148197 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9c051c60 con 0x55b3462f6390 2024-03-29T08:23:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fef9be00640 1 -- 172.21.15.46:0/3233148197 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c0842b0 con 0x55b3462f6390 2024-03-29T08:23:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefb967f000 1 -- 172.21.15.46:0/3233148197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462f6390 msgr2=0x55b3462a94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefb967f000 1 --2- 172.21.15.46:0/3233148197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462f6390 0x55b3462a94c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fef9c082850 tx=0x7fef9c054fe0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefb967f000 1 -- 172.21.15.46:0/3233148197 shutdown_connections 2024-03-29T08:23:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefb967f000 1 --2- 172.21.15.46:0/3233148197 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462a9a00 0x55b3462a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefb967f000 1 --2- 172.21.15.46:0/3233148197 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462f6390 0x55b3462a94c0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefb967f000 1 --2- 172.21.15.46:0/3233148197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f5a70 0x55b3462f5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefb967f000 1 -- 172.21.15.46:0/3233148197 >> 172.21.15.46:0/3233148197 conn(0x55b3460aa160 msgr2=0x55b3462d8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefb967f000 1 -- 172.21.15.46:0/3233148197 shutdown_connections 2024-03-29T08:23:19.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.774+0000 7fefb967f000 1 -- 172.21.15.46:0/3233148197 wait complete. 2024-03-29T08:23:19.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 1 -- start start 2024-03-29T08:23:19.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 1 Processor -- start 2024-03-29T08:23:19.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462f5a70 0x55b3462d9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f6390 0x55b3462d9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3460faa90 con 0x55b3462a9a00 2024-03-29T08:23:19.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b346099d40 con 0x55b3462f6390 2024-03-29T08:23:19.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3460b7f30 con 0x55b3462f5a70 2024-03-29T08:23:19.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 10 client.0 _ll_register_callbacks cb 0x55b34615c810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:23:19.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefaf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefaf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51590/0 (socket says 172.21.15.46:51590) 2024-03-29T08:23:19.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefaf400640 1 -- 172.21.15.46:0/1261142248 learned_addr learned my addr 172.21.15.46:0/1261142248 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:19.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefafe00640 1 --2- 172.21.15.46:0/1261142248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f6390 0x55b3462d9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefaea00640 1 --2- 172.21.15.46:0/1261142248 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462f5a70 0x55b3462d9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefaf400640 1 -- 172.21.15.46:0/1261142248 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462f5a70 msgr2=0x55b3462d9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefaf400640 1 --2- 172.21.15.46:0/1261142248 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462f5a70 0x55b3462d9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefaf400640 1 -- 172.21.15.46:0/1261142248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f6390 msgr2=0x55b3462d9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefaf400640 1 --2- 172.21.15.46:0/1261142248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f6390 0x55b3462d9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefaf400640 1 -- 172.21.15.46:0/1261142248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb009cfc0 con 0x55b3462a9a00 2024-03-29T08:23:19.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefaea00640 1 --2- 172.21.15.46:0/1261142248 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462f5a70 0x55b3462d9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:23:19.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefaf400640 1 --2- 172.21.15.46:0/1261142248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462b27b0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fefa0015760 tx=0x7fefa00010b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3841bacd6597e4e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:19.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fef98c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:23:19.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fef98c00640 1 -- 172.21.15.46:0/1261142248 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0013680 con 0x55b3462a9a00 2024-03-29T08:23:19.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 1 -- 172.21.15.46:0/1261142248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3460f93e0 con 0x55b3462a9a00 2024-03-29T08:23:19.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fef98c00640 1 -- 172.21.15.46:0/1261142248 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fefa00137e0 con 0x55b3462a9a00 2024-03-29T08:23:19.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fef98c00640 1 -- 172.21.15.46:0/1261142248 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefa00139a0 con 0x55b3462a9a00 2024-03-29T08:23:19.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 10 client.7053 fetch_fsmap learned FSMap version 48 2024-03-29T08:23:19.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 1 -- 172.21.15.46:0/1261142248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55b3460bfaa0 con 0x55b3462a9a00 2024-03-29T08:23:19.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fef98c00640 1 -- 172.21.15.46:0/1261142248 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b3460f93e0 con 0x55b3462a9a00 2024-03-29T08:23:19.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fef98c00640 1 -- 172.21.15.46:0/1261142248 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 48) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fefa0012070 con 0x55b3462a9a00 2024-03-29T08:23:19.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 10 client.7053 fetch_fsmap finished waiting for FSMap version 48 2024-03-29T08:23:19.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 10 client.7053 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-29T08:23:19.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 20 client.7053 populate_metadata read hostname 'smithi046' 2024-03-29T08:23:19.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 1 --2- 172.21.15.46:0/1261142248 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55b3462fd2d0 0x55b3462fd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 4 client.7053 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:23:19.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefb967f000 1 -- 172.21.15.46:0/1261142248 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3460cad00 con 0x55b3462fd2d0 2024-03-29T08:23:19.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.778+0000 7fefaea00640 1 --2- 172.21.15.46:0/1261142248 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55b3462fd2d0 0x55b3462fd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.782+0000 7fef98c00640 10 client.7053 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:19.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.782+0000 7fefaea00640 1 --2- 172.21.15.46:0/1261142248 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55b3462fd2d0 0x55b3462fd6b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b689d9a3507c1f server_cookie=e6bc49818c0e42bf in_seq=0 out_seq=0 2024-03-29T08:23:19.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.782+0000 7fef98c00640 1 -- 172.21.15.46:0/1261142248 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b3460cad00 con 0x55b3462fd2d0 2024-03-29T08:23:19.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.782+0000 7fef98c00640 10 client.7053 handle_command_reply: tid=0 2024-03-29T08:23:19.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.782+0000 7fefb967f000 1 client.7053 shutdown 2024-03-29T08:23:19.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.782+0000 7fefb967f000 1 -- 172.21.15.46:0/1261142248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 msgr2=0x55b3462b27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.782+0000 7fefb967f000 1 --2- 172.21.15.46:0/1261142248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462b27b0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fefa0015760 tx=0x7fefa00010b0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.782+0000 7fefb967f000 1 -- 172.21.15.46:0/1261142248 shutdown_connections 2024-03-29T08:23:19.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.782+0000 7fefb967f000 1 --2- 172.21.15.46:0/1261142248 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55b3462fd2d0 0x55b3462fd6b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.782+0000 7fefb967f000 1 --2- 172.21.15.46:0/1261142248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f6390 0x55b3462d9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.782+0000 7fef98c00640 0 client.7053 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:19.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.782+0000 7fefb967f000 1 --2- 172.21.15.46:0/1261142248 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462f5a70 0x55b3462d9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.782+0000 7fefb967f000 1 --2- 172.21.15.46:0/1261142248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462b27b0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.782+0000 7fefb967f000 1 -- 172.21.15.46:0/1261142248 >> 172.21.15.46:0/1261142248 conn(0x55b3460aa160 msgr2=0x55b3462a8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:19.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.782+0000 7fefb967f000 1 -- 172.21.15.46:0/1261142248 shutdown_connections 2024-03-29T08:23:19.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.782+0000 7fefb967f000 1 -- 172.21.15.46:0/1261142248 wait complete. 2024-03-29T08:23:19.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.782+0000 7fefb967f000 20 client.7053 trim_cache size 0 max 16384 2024-03-29T08:23:19.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 Processor -- start 2024-03-29T08:23:19.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 -- start start 2024-03-29T08:23:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462a8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462f5a70 0x55b3462a9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f6390 0x55b3462a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b345db7aa0 con 0x55b3462a9a00 2024-03-29T08:23:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3460c6390 con 0x55b3462f6390 2024-03-29T08:23:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b346102e60 con 0x55b3462f5a70 2024-03-29T08:23:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefaf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462a8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefaf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462a8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51594/0 (socket says 172.21.15.46:51594) 2024-03-29T08:23:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefaf400640 1 -- 172.21.15.46:0/2985882005 learned_addr learned my addr 172.21.15.46:0/2985882005 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefafe00640 1 --2- 172.21.15.46:0/2985882005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f6390 0x55b3462a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefaea00640 1 --2- 172.21.15.46:0/2985882005 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462f5a70 0x55b3462a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefaf400640 1 -- 172.21.15.46:0/2985882005 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462f5a70 msgr2=0x55b3462a9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefaf400640 1 --2- 172.21.15.46:0/2985882005 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462f5a70 0x55b3462a9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefaf400640 1 -- 172.21.15.46:0/2985882005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f6390 msgr2=0x55b3462a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefaf400640 1 --2- 172.21.15.46:0/2985882005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f6390 0x55b3462a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefaf400640 1 -- 172.21.15.46:0/2985882005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3460a5df0 con 0x55b3462a9a00 2024-03-29T08:23:19.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefafe00640 1 --2- 172.21.15.46:0/2985882005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f6390 0x55b3462a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:19.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefaf400640 1 --2- 172.21.15.46:0/2985882005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462a8c30 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fefa00068a0 tx=0x7fefa0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f962e6c9b2ba4f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:19.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefa4c00640 1 -- 172.21.15.46:0/2985882005 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0001070 con 0x55b3462a9a00 2024-03-29T08:23:19.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefa4c00640 1 -- 172.21.15.46:0/2985882005 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa000f5f0 con 0x55b3462a9a00 2024-03-29T08:23:19.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefa4c00640 1 -- 172.21.15.46:0/2985882005 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0006110 con 0x55b3462a9a00 2024-03-29T08:23:19.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 -- 172.21.15.46:0/2985882005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 msgr2=0x55b3462a8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 --2- 172.21.15.46:0/2985882005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462a8c30 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fefa00068a0 tx=0x7fefa0014000 comp rx=0 tx=0).stop 2024-03-29T08:23:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 -- 172.21.15.46:0/2985882005 shutdown_connections 2024-03-29T08:23:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 --2- 172.21.15.46:0/2985882005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f6390 0x55b3462a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 --2- 172.21.15.46:0/2985882005 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462f5a70 0x55b3462a9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 --2- 172.21.15.46:0/2985882005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462a8c30 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 -- 172.21.15.46:0/2985882005 >> 172.21.15.46:0/2985882005 conn(0x55b3460aa160 msgr2=0x55b3462b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 -- 172.21.15.46:0/2985882005 shutdown_connections 2024-03-29T08:23:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 -- 172.21.15.46:0/2985882005 wait complete. 2024-03-29T08:23:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 -- start start 2024-03-29T08:23:19.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.798+0000 7fefb967f000 1 Processor -- start 2024-03-29T08:23:19.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462acf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f5a70 0x55b3463111e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462f6390 0x55b346311720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3462da0e0 con 0x55b3462a9a00 2024-03-29T08:23:19.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3460c4640 con 0x55b3462f5a70 2024-03-29T08:23:19.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3460fd5f0 con 0x55b3462f6390 2024-03-29T08:23:19.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 10 client.0 _ll_register_callbacks cb 0x7fefa80b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:23:19.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefaf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefaf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462acf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51610/0 (socket says 172.21.15.46:51610) 2024-03-29T08:23:19.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefaf400640 1 -- 172.21.15.46:0/3964159083 learned_addr learned my addr 172.21.15.46:0/3964159083 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:19.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefafe00640 1 --2- 172.21.15.46:0/3964159083 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462f6390 0x55b346311720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefaea00640 1 --2- 172.21.15.46:0/3964159083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f5a70 0x55b3463111e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefaf400640 1 -- 172.21.15.46:0/3964159083 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462f6390 msgr2=0x55b346311720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefaf400640 1 --2- 172.21.15.46:0/3964159083 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462f6390 0x55b346311720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefaf400640 1 -- 172.21.15.46:0/3964159083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f5a70 msgr2=0x55b3463111e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:19.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefaf400640 1 --2- 172.21.15.46:0/3964159083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f5a70 0x55b3463111e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefaf400640 1 -- 172.21.15.46:0/3964159083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3460bfaa0 con 0x55b3462a9a00 2024-03-29T08:23:19.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefaf400640 1 --2- 172.21.15.46:0/3964159083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462acf90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fefa0011010 tx=0x7fefa0000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd9eb1130495cc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:19.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fef98c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:23:19.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fef98c00640 1 -- 172.21.15.46:0/3964159083 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa00062a0 con 0x55b3462a9a00 2024-03-29T08:23:19.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fef98c00640 1 -- 172.21.15.46:0/3964159083 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fefa0006400 con 0x55b3462a9a00 2024-03-29T08:23:19.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 -- 172.21.15.46:0/3964159083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3460b7970 con 0x55b3462a9a00 2024-03-29T08:23:19.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fef98c00640 1 -- 172.21.15.46:0/3964159083 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefa00065c0 con 0x55b3462a9a00 2024-03-29T08:23:19.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 10 client.7059 fetch_fsmap learned FSMap version 48 2024-03-29T08:23:19.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 -- 172.21.15.46:0/3964159083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55b346099960 con 0x55b3462a9a00 2024-03-29T08:23:19.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fef98c00640 1 -- 172.21.15.46:0/3964159083 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b3460b7970 con 0x55b3462a9a00 2024-03-29T08:23:19.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fef98c00640 1 -- 172.21.15.46:0/3964159083 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 48) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fefa00047f0 con 0x55b3462a9a00 2024-03-29T08:23:19.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 10 client.7059 fetch_fsmap finished waiting for FSMap version 48 2024-03-29T08:23:19.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 10 client.7059 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-29T08:23:19.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 20 client.7059 populate_metadata read hostname 'smithi046' 2024-03-29T08:23:19.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 --2- 172.21.15.46:0/3964159083 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55b3462afff0 0x55b346311c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:19.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 4 client.7059 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:23:19.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 -- 172.21.15.46:0/3964159083 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b346129a20 con 0x55b3462afff0 2024-03-29T08:23:19.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefaea00640 1 --2- 172.21.15.46:0/3964159083 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55b3462afff0 0x55b346311c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:19.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fef98c00640 10 client.7059 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:19.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefaea00640 1 --2- 172.21.15.46:0/3964159083 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55b3462afff0 0x55b346311c60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fec80a0a82fa1e03 server_cookie=b4da575581d0c5ea in_seq=0 out_seq=0 2024-03-29T08:23:19.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fef98c00640 1 -- 172.21.15.46:0/3964159083 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b346129a20 con 0x55b3462afff0 2024-03-29T08:23:19.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fef98c00640 10 client.7059 handle_command_reply: tid=0 2024-03-29T08:23:19.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 client.7059 shutdown 2024-03-29T08:23:19.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 -- 172.21.15.46:0/3964159083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 msgr2=0x55b3462acf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 --2- 172.21.15.46:0/3964159083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462acf90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fefa0011010 tx=0x7fefa0000bd0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 -- 172.21.15.46:0/3964159083 shutdown_connections 2024-03-29T08:23:19.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 --2- 172.21.15.46:0/3964159083 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55b3462afff0 0x55b346311c60 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 --2- 172.21.15.46:0/3964159083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3462f5a70 0x55b3463111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 --2- 172.21.15.46:0/3964159083 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b3462f6390 0x55b346311720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 --2- 172.21.15.46:0/3964159083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b3462a9a00 0x55b3462acf90 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fef98c00640 0 client.7059 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:19.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 -- 172.21.15.46:0/3964159083 >> 172.21.15.46:0/3964159083 conn(0x55b3460aa160 msgr2=0x55b3462a7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:19.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.802+0000 7fefb967f000 1 -- 172.21.15.46:0/3964159083 shutdown_connections 2024-03-29T08:23:19.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.806+0000 7fefb967f000 1 -- 172.21.15.46:0/3964159083 wait complete. 2024-03-29T08:23:19.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.806+0000 7fefb967f000 20 client.7059 trim_cache size 0 max 16384 2024-03-29T08:23:19.815 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:23:19.815 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-29T08:23:19.816 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-29T08:23:19.816 INFO:teuthology.orchestra.run.smithi046.stdout: "444816b4-a10f-4031-8a61-decf5cbfdf81": { 2024-03-29T08:23:19.816 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/subdir", 2024-03-29T08:23:19.816 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "444816b4-a10f-4031-8a61-decf5cbfdf81", 2024-03-29T08:23:19.816 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive" 2024-03-29T08:23:19.816 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:23:19.817 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:23:19.817 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:23:19.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.806+0000 7fefb6200640 1 -- 172.21.15.46:0/3019130096 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fef90066690 msgr2=0x7fef90068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.806+0000 7fefb6200640 1 --2- 172.21.15.46:0/3019130096 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fef90066690 0x7fef90068b50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fefa0007fd0 tx=0x7fefa0009040 comp rx=0 tx=0).stop 2024-03-29T08:23:19.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.806+0000 7fefb6200640 1 -- 172.21.15.46:0/3019130096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb015c950 msgr2=0x7fefb015f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:19.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.806+0000 7fefb6200640 1 --2- 172.21.15.46:0/3019130096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb015c950 0x7fefb015f470 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00e930 tx=0x7fef9c00edf0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.806+0000 7fefb6200640 1 -- 172.21.15.46:0/3019130096 shutdown_connections 2024-03-29T08:23:19.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.806+0000 7fefb6200640 1 --2- 172.21.15.46:0/3019130096 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fef90066690 0x7fef90068b50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.806+0000 7fefb6200640 1 --2- 172.21.15.46:0/3019130096 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefb015cd30 0x7fefb00b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.806+0000 7fefb6200640 1 --2- 172.21.15.46:0/3019130096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb015c950 0x7fefb015f470 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.806+0000 7fefb6200640 1 --2- 172.21.15.46:0/3019130096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefb00cea30 0x7fefb01596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:19.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.806+0000 7fefb6200640 1 -- 172.21.15.46:0/3019130096 >> 172.21.15.46:0/3019130096 conn(0x7fefb00b21b0 msgr2=0x7fefb015ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:19.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.806+0000 7fefb6200640 1 -- 172.21.15.46:0/3019130096 shutdown_connections 2024-03-29T08:23:19.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:19.806+0000 7fefb6200640 1 -- 172.21.15.46:0/3019130096 wait complete. 2024-03-29T08:23:19.827 INFO:tasks.cephfs.filesystem:scrub status for tag:444816b4-a10f-4031-8a61-decf5cbfdf81 - {'path': '/subdir', 'tag': '444816b4-a10f-4031-8a61-decf5cbfdf81', 'options': 'recursive'} 2024-03-29T08:23:38.881 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:23:38.885 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:23:49.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:49.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-29T08:23:49.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.981+0000 7fd8c1e00640 1 Processor -- start 2024-03-29T08:23:49.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.981+0000 7fd8c1e00640 1 -- start start 2024-03-29T08:23:49.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.981+0000 7fd8c1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8bc142da0 0x7fd8bc0adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:49.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.981+0000 7fd8c1e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd8bc0ae1e0 0x7fd8bc0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:49.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.981+0000 7fd8c1e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8bc0afe80 con 0x7fd8bc143180 2024-03-29T08:23:49.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.981+0000 7fd8c1e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8bc06cd80 con 0x7fd8bc142da0 2024-03-29T08:23:49.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.981+0000 7fd8c1e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8bc06d020 con 0x7fd8bc0ae1e0 2024-03-29T08:23:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.981+0000 7fd8baa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8bc142da0 0x7fd8bc0adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.981+0000 7fd8baa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd8bc142da0 0x7fd8bc0adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51984/0 (socket says 172.21.15.46:51984) 2024-03-29T08:23:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.981+0000 7fd8baa00640 1 -- 172.21.15.46:0/1593466572 learned_addr learned my addr 172.21.15.46:0/1593466572 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.981+0000 7fd8bb400640 1 --2- 172.21.15.46:0/1593466572 >> v2:172.21.15.73:3300/0 conn(0x7fd8bc0ae1e0 0x7fd8bc0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.981+0000 7fd8baa00640 1 -- 172.21.15.46:0/1593466572 >> v2:172.21.15.73:3300/0 conn(0x7fd8bc0ae1e0 msgr2=0x7fd8bc0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.981+0000 7fd8baa00640 1 --2- 172.21.15.46:0/1593466572 >> v2:172.21.15.73:3300/0 conn(0x7fd8bc0ae1e0 0x7fd8bc0cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:49.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.981+0000 7fd8baa00640 1 -- 172.21.15.46:0/1593466572 >> v1:172.21.15.73:6789/0 conn(0x7fd8bc143180 legacy=0x7fd8bc0abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.981+0000 7fd8baa00640 1 -- 172.21.15.46:0/1593466572 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc0b00f0 con 0x7fd8bc142da0 2024-03-29T08:23:49.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.981+0000 7fd8bb400640 1 --2- 172.21.15.46:0/1593466572 >> v2:172.21.15.73:3300/0 conn(0x7fd8bc0ae1e0 0x7fd8bc0cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:23:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.985+0000 7fd8baa00640 1 --2- 172.21.15.46:0/1593466572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8bc142da0 0x7fd8bc0adb80 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc06dc80 tx=0x7fd8ac01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e447a5e6a6a6d93 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.985+0000 7fd8ba000640 1 -- 172.21.15.46:0/1593466572 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0ab5f0 con 0x7fd8bc142da0 2024-03-29T08:23:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.985+0000 7fd8ba000640 1 -- 172.21.15.46:0/1593466572 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac0ab750 con 0x7fd8bc142da0 2024-03-29T08:23:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.985+0000 7fd8ba000640 1 -- 172.21.15.46:0/1593466572 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0abae0 con 0x7fd8bc142da0 2024-03-29T08:23:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.985+0000 7fd8c1e00640 1 -- 172.21.15.46:0/1593466572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8bc142da0 msgr2=0x7fd8bc0adb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:49.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.985+0000 7fd8c1e00640 1 --2- 172.21.15.46:0/1593466572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8bc142da0 0x7fd8bc0adb80 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc06dc80 tx=0x7fd8ac01c4e0 comp rx=0 tx=0).stop 2024-03-29T08:23:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.985+0000 7fd8c1e00640 1 -- 172.21.15.46:0/1593466572 shutdown_connections 2024-03-29T08:23:49.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.985+0000 7fd8c1e00640 1 --2- 172.21.15.46:0/1593466572 >> v2:172.21.15.73:3300/0 conn(0x7fd8bc0ae1e0 0x7fd8bc0cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.985+0000 7fd8c1e00640 1 --2- 172.21.15.46:0/1593466572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8bc142da0 0x7fd8bc0adb80 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.985+0000 7fd8c1e00640 1 -- 172.21.15.46:0/1593466572 >> 172.21.15.46:0/1593466572 conn(0x7fd8bc06a390 msgr2=0x7fd8bc0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.985+0000 7fd8c1e00640 1 -- 172.21.15.46:0/1593466572 shutdown_connections 2024-03-29T08:23:49.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.985+0000 7fd8c1e00640 1 -- 172.21.15.46:0/1593466572 wait complete. 2024-03-29T08:23:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8c1e00640 1 Processor -- start 2024-03-29T08:23:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8c1e00640 1 -- start start 2024-03-29T08:23:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8c1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8bc0ae1e0 0x7fd8bc15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8c1e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8bc142da0 0x7fd8bc141a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8c1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8bc143180 0x7fd8bc141f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8c1e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc13bc50 con 0x7fd8bc143180 2024-03-29T08:23:49.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8c1e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc13bf30 con 0x7fd8bc0ae1e0 2024-03-29T08:23:49.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8c1e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8bc13c210 con 0x7fd8bc142da0 2024-03-29T08:23:49.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8bb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8bc143180 0x7fd8bc141f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:49.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8bb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8bc143180 0x7fd8bc141f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51986/0 (socket says 172.21.15.46:51986) 2024-03-29T08:23:49.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8bb400640 1 -- 172.21.15.46:0/2178274529 learned_addr learned my addr 172.21.15.46:0/2178274529 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:49.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8baa00640 1 --2- 172.21.15.46:0/2178274529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8bc0ae1e0 0x7fd8bc15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:49.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8b2000640 1 --2- 172.21.15.46:0/2178274529 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8bc142da0 0x7fd8bc141a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:49.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8bb400640 1 -- 172.21.15.46:0/2178274529 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8bc142da0 msgr2=0x7fd8bc141a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:49.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8bb400640 1 --2- 172.21.15.46:0/2178274529 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8bc142da0 0x7fd8bc141a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:49.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8bb400640 1 -- 172.21.15.46:0/2178274529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8bc0ae1e0 msgr2=0x7fd8bc15aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:49.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8bb400640 1 --2- 172.21.15.46:0/2178274529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8bc0ae1e0 0x7fd8bc15aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:49.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8bb400640 1 -- 172.21.15.46:0/2178274529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac0094d0 con 0x7fd8bc143180 2024-03-29T08:23:49.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8b2000640 1 --2- 172.21.15.46:0/2178274529 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8bc142da0 0x7fd8bc141a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:49.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8bb400640 1 --2- 172.21.15.46:0/2178274529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8bc143180 0x7fd8bc141f90 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400cef0 tx=0x7fd8b400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:49.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8b3e00640 1 -- 172.21.15.46:0/2178274529 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b400d840 con 0x7fd8bc143180 2024-03-29T08:23:49.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8b3e00640 1 -- 172.21.15.46:0/2178274529 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b400d9c0 con 0x7fd8bc143180 2024-03-29T08:23:49.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8c1e00640 1 -- 172.21.15.46:0/2178274529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8bc13c4f0 con 0x7fd8bc143180 2024-03-29T08:23:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8b3e00640 1 -- 172.21.15.46:0/2178274529 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b400dd70 con 0x7fd8bc143180 2024-03-29T08:23:49.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.989+0000 7fd8c1e00640 1 -- 172.21.15.46:0/2178274529 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8bc15c2c0 con 0x7fd8bc143180 2024-03-29T08:23:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.993+0000 7fd8c5204000 1 Processor -- start 2024-03-29T08:23:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.993+0000 7fd8b3e00640 1 -- 172.21.15.46:0/2178274529 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd8b401e3b0 con 0x7fd8bc143180 2024-03-29T08:23:49.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.993+0000 7fd8c5204000 1 -- start start 2024-03-29T08:23:50.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8b3e00640 1 --2- 172.21.15.46:0/2178274529 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd898066690 0x7fd898068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8b3e00640 1 -- 172.21.15.46:0/2178274529 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd8b4084cd0 con 0x7fd8bc143180 2024-03-29T08:23:50.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8c5204000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f299635a70 0x55f299635e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8c5204000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f299636390 0x55f2995e94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8c5204000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995e9a00 0x55f2995e9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8c5204000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2994b4630 con 0x55f2995e9a00 2024-03-29T08:23:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8c5204000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f299441bd0 con 0x55f299636390 2024-03-29T08:23:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8c5204000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f29943f1a0 con 0x55f299635a70 2024-03-29T08:23:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8b2000640 1 --2- 172.21.15.46:0/2178274529 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd898066690 0x7fd898068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8baa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f299635a70 0x55f299635e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8bb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995e9a00 0x55f2995e9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8b2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f299636390 0x55f2995e94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8b2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f299636390 0x55f2995e94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:45182/0 (socket says 172.21.15.46:45182) 2024-03-29T08:23:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8baa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f299635a70 0x55f299635e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:60564/0 (socket says 172.21.15.46:60564) 2024-03-29T08:23:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8b2000640 1 -- 172.21.15.46:0/1917609055 learned_addr learned my addr 172.21.15.46:0/1917609055 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8bb400640 1 -- 172.21.15.46:0/1917609055 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f299635a70 msgr2=0x55f299635e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8bb400640 1 --2- 172.21.15.46:0/1917609055 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f299635a70 0x55f299635e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8bb400640 1 -- 172.21.15.46:0/1917609055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f299636390 msgr2=0x55f2995e94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8bb400640 1 --2- 172.21.15.46:0/1917609055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f299636390 0x55f2995e94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8bb400640 1 -- 172.21.15.46:0/1917609055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f299136e20 con 0x55f2995e9a00 2024-03-29T08:23:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8b2000640 1 --2- 172.21.15.46:0/2178274529 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd898066690 0x7fd898068b50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4014610 tx=0x7fd8a400a8d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8bb400640 1 --2- 172.21.15.46:0/1917609055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995e9a00 0x55f2995e9de0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4055a40 tx=0x7fd8b4052100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2a6d1f501e7d60 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8abe00640 1 -- 172.21.15.46:0/1917609055 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b40540e0 con 0x55f2995e9a00 2024-03-29T08:23:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8abe00640 1 -- 172.21.15.46:0/1917609055 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b4052450 con 0x55f2995e9a00 2024-03-29T08:23:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8abe00640 1 -- 172.21.15.46:0/1917609055 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4055430 con 0x55f2995e9a00 2024-03-29T08:23:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8c5204000 1 -- 172.21.15.46:0/1917609055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995e9a00 msgr2=0x55f2995e9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8c5204000 1 --2- 172.21.15.46:0/1917609055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995e9a00 0x55f2995e9de0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4055a40 tx=0x7fd8b4052100 comp rx=0 tx=0).stop 2024-03-29T08:23:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8c5204000 1 -- 172.21.15.46:0/1917609055 shutdown_connections 2024-03-29T08:23:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8c5204000 1 --2- 172.21.15.46:0/1917609055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f299636390 0x55f2995e94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8c5204000 1 --2- 172.21.15.46:0/1917609055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995e9a00 0x55f2995e9de0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8c5204000 1 --2- 172.21.15.46:0/1917609055 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f299635a70 0x55f299635e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8c5204000 1 -- 172.21.15.46:0/1917609055 >> 172.21.15.46:0/1917609055 conn(0x55f2993ea160 msgr2=0x55f299618820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8c5204000 1 -- 172.21.15.46:0/1917609055 shutdown_connections 2024-03-29T08:23:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8c5204000 1 -- 172.21.15.46:0/1917609055 wait complete. 2024-03-29T08:23:50.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8c5204000 1 -- start start 2024-03-29T08:23:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:49.997+0000 7fd8c5204000 1 Processor -- start 2024-03-29T08:23:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8c5204000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995e9a00 0x55f2995f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8c5204000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f299635a70 0x55f299619a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8c5204000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f299636390 0x55f299619f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8c5204000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f29943aa90 con 0x55f2995e9a00 2024-03-29T08:23:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8c5204000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2993d9d40 con 0x55f299636390 2024-03-29T08:23:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8c5204000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f2993f7f30 con 0x55f299635a70 2024-03-29T08:23:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8c5204000 10 client.0 _ll_register_callbacks cb 0x55f29949c810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:23:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8baa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995e9a00 0x55f2995f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8baa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995e9a00 0x55f2995f1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52006/0 (socket says 172.21.15.46:52006) 2024-03-29T08:23:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8b2000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f299635a70 0x55f299619a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8baa00640 1 -- 172.21.15.46:0/3443223725 learned_addr learned my addr 172.21.15.46:0/3443223725 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8bb400640 1 --2- 172.21.15.46:0/3443223725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f299636390 0x55f299619f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8baa00640 1 -- 172.21.15.46:0/3443223725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f299635a70 msgr2=0x55f299619a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8baa00640 1 --2- 172.21.15.46:0/3443223725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f299635a70 0x55f299619a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8baa00640 1 -- 172.21.15.46:0/3443223725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f299636390 msgr2=0x55f299619f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:50.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8baa00640 1 --2- 172.21.15.46:0/3443223725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f299636390 0x55f299619f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8baa00640 1 -- 172.21.15.46:0/3443223725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8bc15c2c0 con 0x55f2995e9a00 2024-03-29T08:23:50.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8baa00640 1 --2- 172.21.15.46:0/3443223725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995e9a00 0x55f2995f1ad0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac01ab90 tx=0x7fd8ac0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0737e375afe147 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:23:50.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8a8c00640 1 -- 172.21.15.46:0/3443223725 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0c3070 con 0x55f2995e9a00 2024-03-29T08:23:50.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8a8c00640 1 -- 172.21.15.46:0/3443223725 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd8ac0b9040 con 0x55f2995e9a00 2024-03-29T08:23:50.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8c5204000 1 -- 172.21.15.46:0/3443223725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2994393e0 con 0x55f2995e9a00 2024-03-29T08:23:50.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8a8c00640 1 -- 172.21.15.46:0/3443223725 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8ac0ab4a0 con 0x55f2995e9a00 2024-03-29T08:23:50.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8c5204000 10 client.7071 fetch_fsmap learned FSMap version 48 2024-03-29T08:23:50.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8c5204000 1 -- 172.21.15.46:0/3443223725 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55f2993ffaa0 con 0x55f2995e9a00 2024-03-29T08:23:50.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8a8c00640 1 -- 172.21.15.46:0/3443223725 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f2994393e0 con 0x55f2995e9a00 2024-03-29T08:23:50.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8a8c00640 1 -- 172.21.15.46:0/3443223725 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 48) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd8ac0bd050 con 0x55f2995e9a00 2024-03-29T08:23:50.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8c5204000 10 client.7071 fetch_fsmap finished waiting for FSMap version 48 2024-03-29T08:23:50.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8c5204000 10 client.7071 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-29T08:23:50.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8c5204000 20 client.7071 populate_metadata read hostname 'smithi046' 2024-03-29T08:23:50.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8c5204000 1 --2- 172.21.15.46:0/3443223725 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55f29963ceb0 0x55f29963d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8c5204000 4 client.7071 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:23:50.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8c5204000 1 -- 172.21.15.46:0/3443223725 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f29940ad00 con 0x55f29963ceb0 2024-03-29T08:23:50.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8b2000640 1 --2- 172.21.15.46:0/3443223725 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55f29963ceb0 0x55f29963d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8a8c00640 10 client.7071 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:50.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8b2000640 1 --2- 172.21.15.46:0/3443223725 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55f29963ceb0 0x55f29963d2c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ede55d63232fb6a server_cookie=c965f2bbe2b1bd6b in_seq=0 out_seq=0 2024-03-29T08:23:50.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8a8c00640 1 -- 172.21.15.46:0/3443223725 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55f29940ad00 con 0x55f29963ceb0 2024-03-29T08:23:50.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8a8c00640 10 client.7071 handle_command_reply: tid=0 2024-03-29T08:23:50.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.001+0000 7fd8c5204000 1 client.7071 shutdown 2024-03-29T08:23:50.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.005+0000 7fd8c5204000 1 -- 172.21.15.46:0/3443223725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995e9a00 msgr2=0x55f2995f1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.005+0000 7fd8c5204000 1 --2- 172.21.15.46:0/3443223725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995e9a00 0x55f2995f1ad0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac01ab90 tx=0x7fd8ac0046d0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.005+0000 7fd8c5204000 1 -- 172.21.15.46:0/3443223725 shutdown_connections 2024-03-29T08:23:50.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.005+0000 7fd8c5204000 1 --2- 172.21.15.46:0/3443223725 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55f29963ceb0 0x55f29963d2c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.005+0000 7fd8c5204000 1 --2- 172.21.15.46:0/3443223725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f299636390 0x55f299619f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.005+0000 7fd8c5204000 1 --2- 172.21.15.46:0/3443223725 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f299635a70 0x55f299619a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.005+0000 7fd8a8c00640 0 client.7071 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:50.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.005+0000 7fd8c5204000 1 --2- 172.21.15.46:0/3443223725 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995e9a00 0x55f2995f1ad0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.005+0000 7fd8c5204000 1 -- 172.21.15.46:0/3443223725 >> 172.21.15.46:0/3443223725 conn(0x55f2993ea160 msgr2=0x55f2995e8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:50.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.005+0000 7fd8c5204000 1 -- 172.21.15.46:0/3443223725 shutdown_connections 2024-03-29T08:23:50.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.005+0000 7fd8c5204000 1 -- 172.21.15.46:0/3443223725 wait complete. 2024-03-29T08:23:50.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.005+0000 7fd8c5204000 20 client.7071 trim_cache size 0 max 16384 2024-03-29T08:23:50.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.017+0000 7fd8c5204000 1 Processor -- start 2024-03-29T08:23:50.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.017+0000 7fd8c5204000 1 -- start start 2024-03-29T08:23:50.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995f1c60 0x55f2995f09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f2995f2040 0x55f2995f0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f2993ea160 0x55f2995e9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2990f7aa0 con 0x55f2995f1c60 2024-03-29T08:23:50.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f299439920 con 0x55f2993ea160 2024-03-29T08:23:50.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f29943d5f0 con 0x55f2995f2040 2024-03-29T08:23:50.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8baa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995f1c60 0x55f2995f09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8baa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995f1c60 0x55f2995f09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52012/0 (socket says 172.21.15.46:52012) 2024-03-29T08:23:50.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8bb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f2993ea160 0x55f2995e9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8baa00640 1 -- 172.21.15.46:0/3927683674 learned_addr learned my addr 172.21.15.46:0/3927683674 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:50.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8b2000640 1 --2- 172.21.15.46:0/3927683674 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f2995f2040 0x55f2995f0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8baa00640 1 -- 172.21.15.46:0/3927683674 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f2995f2040 msgr2=0x55f2995f0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:50.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8baa00640 1 --2- 172.21.15.46:0/3927683674 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f2995f2040 0x55f2995f0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8baa00640 1 -- 172.21.15.46:0/3927683674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f2993ea160 msgr2=0x55f2995e9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:50.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8baa00640 1 --2- 172.21.15.46:0/3927683674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f2993ea160 0x55f2995e9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8baa00640 1 -- 172.21.15.46:0/3927683674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2993e5df0 con 0x55f2995f1c60 2024-03-29T08:23:50.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8baa00640 1 --2- 172.21.15.46:0/3927683674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995f1c60 0x55f2995f09e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0190f0 tx=0x7fd8ac0b6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556e18f28898abd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8b0c00640 1 -- 172.21.15.46:0/3927683674 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0c3070 con 0x55f2995f1c60 2024-03-29T08:23:50.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8b0c00640 1 -- 172.21.15.46:0/3927683674 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac006b50 con 0x55f2995f1c60 2024-03-29T08:23:50.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8b0c00640 1 -- 172.21.15.46:0/3927683674 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac004240 con 0x55f2995f1c60 2024-03-29T08:23:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 -- 172.21.15.46:0/3927683674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995f1c60 msgr2=0x55f2995f09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 --2- 172.21.15.46:0/3927683674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995f1c60 0x55f2995f09e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac0190f0 tx=0x7fd8ac0b6040 comp rx=0 tx=0).stop 2024-03-29T08:23:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 -- 172.21.15.46:0/3927683674 shutdown_connections 2024-03-29T08:23:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 --2- 172.21.15.46:0/3927683674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f2993ea160 0x55f2995e9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 --2- 172.21.15.46:0/3927683674 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f2995f2040 0x55f2995f0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 --2- 172.21.15.46:0/3927683674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995f1c60 0x55f2995f09e0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 -- 172.21.15.46:0/3927683674 >> 172.21.15.46:0/3927683674 conn(0x55f299635330 msgr2=0x55f2995e8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 -- 172.21.15.46:0/3927683674 shutdown_connections 2024-03-29T08:23:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 -- 172.21.15.46:0/3927683674 wait complete. 2024-03-29T08:23:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 -- start start 2024-03-29T08:23:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 Processor -- start 2024-03-29T08:23:50.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f2993ea160 0x55f2995ece50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f2995f1c60 0x55f299694d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995f2040 0x55f2996952d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.021+0000 7fd8c5204000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f299400b10 con 0x55f2995f2040 2024-03-29T08:23:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f299619c80 con 0x55f2995f1c60 2024-03-29T08:23:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f299437e90 con 0x55f2993ea160 2024-03-29T08:23:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 10 client.0 _ll_register_callbacks cb 0x7fd8b4007a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:23:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8bb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995f2040 0x55f2996952d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8bb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995f2040 0x55f2996952d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52018/0 (socket says 172.21.15.46:52018) 2024-03-29T08:23:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8bb400640 1 -- 172.21.15.46:0/1765003640 learned_addr learned my addr 172.21.15.46:0/1765003640 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:50.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8b2000640 1 --2- 172.21.15.46:0/1765003640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f2995f1c60 0x55f299694d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8baa00640 1 --2- 172.21.15.46:0/1765003640 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f2993ea160 0x55f2995ece50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8bb400640 1 -- 172.21.15.46:0/1765003640 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f2993ea160 msgr2=0x55f2995ece50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:50.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8bb400640 1 --2- 172.21.15.46:0/1765003640 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f2993ea160 0x55f2995ece50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8bb400640 1 -- 172.21.15.46:0/1765003640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f2995f1c60 msgr2=0x55f299694d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:50.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8bb400640 1 --2- 172.21.15.46:0/1765003640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f2995f1c60 0x55f299694d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8bb400640 1 -- 172.21.15.46:0/1765003640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2993ffaa0 con 0x55f2995f2040 2024-03-29T08:23:50.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8bb400640 1 --2- 172.21.15.46:0/1765003640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995f2040 0x55f2996952d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4055a40 tx=0x7fd8b4021070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e29fa36b35871c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8a8c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:23:50.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8a8c00640 1 -- 172.21.15.46:0/1765003640 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4055430 con 0x55f2995f2040 2024-03-29T08:23:50.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8a8c00640 1 -- 172.21.15.46:0/1765003640 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd8b4081040 con 0x55f2995f2040 2024-03-29T08:23:50.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 1 -- 172.21.15.46:0/1765003640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2993f7970 con 0x55f2995f2040 2024-03-29T08:23:50.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8a8c00640 1 -- 172.21.15.46:0/1765003640 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8b405dbf0 con 0x55f2995f2040 2024-03-29T08:23:50.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 10 client.7077 fetch_fsmap learned FSMap version 48 2024-03-29T08:23:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 1 -- 172.21.15.46:0/1765003640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55f2993d9960 con 0x55f2995f2040 2024-03-29T08:23:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8a8c00640 1 -- 172.21.15.46:0/1765003640 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f2993f7970 con 0x55f2995f2040 2024-03-29T08:23:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8a8c00640 1 -- 172.21.15.46:0/1765003640 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 48) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fd8b4005d20 con 0x55f2995f2040 2024-03-29T08:23:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 10 client.7077 fetch_fsmap finished waiting for FSMap version 48 2024-03-29T08:23:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 10 client.7077 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-29T08:23:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 20 client.7077 populate_metadata read hostname 'smithi046' 2024-03-29T08:23:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 1 --2- 172.21.15.46:0/1765003640 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55f2995e9a00 0x55f299650dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 4 client.7077 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:23:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 1 -- 172.21.15.46:0/1765003640 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f299469a20 con 0x55f2995e9a00 2024-03-29T08:23:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8baa00640 1 --2- 172.21.15.46:0/1765003640 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55f2995e9a00 0x55f299650dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8a8c00640 10 client.7077 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8baa00640 1 --2- 172.21.15.46:0/1765003640 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55f2995e9a00 0x55f299650dc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91d2b17af7bb9fad server_cookie=36a8e760bb3a4068 in_seq=0 out_seq=0 2024-03-29T08:23:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8a8c00640 1 -- 172.21.15.46:0/1765003640 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f299469a20 con 0x55f2995e9a00 2024-03-29T08:23:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8a8c00640 10 client.7077 handle_command_reply: tid=0 2024-03-29T08:23:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 1 client.7077 shutdown 2024-03-29T08:23:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 1 -- 172.21.15.46:0/1765003640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995f2040 msgr2=0x55f2996952d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 1 --2- 172.21.15.46:0/1765003640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995f2040 0x55f2996952d0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4055a40 tx=0x7fd8b4021070 comp rx=0 tx=0).stop 2024-03-29T08:23:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 1 -- 172.21.15.46:0/1765003640 shutdown_connections 2024-03-29T08:23:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 1 --2- 172.21.15.46:0/1765003640 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55f2995e9a00 0x55f299650dc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 1 --2- 172.21.15.46:0/1765003640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f2995f1c60 0x55f299694d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 1 --2- 172.21.15.46:0/1765003640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f2995f2040 0x55f2996952d0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 1 --2- 172.21.15.46:0/1765003640 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f2993ea160 0x55f2995ece50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8a8c00640 0 client.7077 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:50.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 1 -- 172.21.15.46:0/1765003640 >> 172.21.15.46:0/1765003640 conn(0x55f299635330 msgr2=0x55f2995ef9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:50.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 1 -- 172.21.15.46:0/1765003640 shutdown_connections 2024-03-29T08:23:50.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 1 -- 172.21.15.46:0/1765003640 wait complete. 2024-03-29T08:23:50.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c5204000 20 client.7077 trim_cache size 0 max 16384 2024-03-29T08:23:50.042 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:23:50.043 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-29T08:23:50.043 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-29T08:23:50.043 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:23:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c1e00640 1 -- 172.21.15.46:0/2178274529 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd898066690 msgr2=0x7fd898068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c1e00640 1 --2- 172.21.15.46:0/2178274529 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd898066690 0x7fd898068b50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4014610 tx=0x7fd8a400a8d0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c1e00640 1 -- 172.21.15.46:0/2178274529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8bc143180 msgr2=0x7fd8bc141f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c1e00640 1 --2- 172.21.15.46:0/2178274529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8bc143180 0x7fd8bc141f90 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400cef0 tx=0x7fd8b400a710 comp rx=0 tx=0).stop 2024-03-29T08:23:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c1e00640 1 -- 172.21.15.46:0/2178274529 shutdown_connections 2024-03-29T08:23:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c1e00640 1 --2- 172.21.15.46:0/2178274529 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd898066690 0x7fd898068b50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c1e00640 1 --2- 172.21.15.46:0/2178274529 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8bc143180 0x7fd8bc141f90 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c1e00640 1 --2- 172.21.15.46:0/2178274529 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8bc142da0 0x7fd8bc141a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c1e00640 1 --2- 172.21.15.46:0/2178274529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8bc0ae1e0 0x7fd8bc15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.025+0000 7fd8c1e00640 1 -- 172.21.15.46:0/2178274529 >> 172.21.15.46:0/2178274529 conn(0x7fd8bc06a390 msgr2=0x7fd8bc160460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.029+0000 7fd8c1e00640 1 -- 172.21.15.46:0/2178274529 shutdown_connections 2024-03-29T08:23:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.029+0000 7fd8c1e00640 1 -- 172.21.15.46:0/2178274529 wait complete. 2024-03-29T08:23:50.051 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-29T08:23:50.051 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:23:50.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.189+0000 7f9516c00640 1 Processor -- start 2024-03-29T08:23:50.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.189+0000 7f9516c00640 1 -- start start 2024-03-29T08:23:50.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.189+0000 7f9516c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95100ae810 0x7f95100caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.189+0000 7f9516c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f95100cb780 0x7f95100aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.189+0000 7f9516c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95100cbc10 con 0x7f95100ce8a0 2024-03-29T08:23:50.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.189+0000 7f9516c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95100c9bd0 con 0x7f95100ae810 2024-03-29T08:23:50.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.189+0000 7f9516c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95100c9eb0 con 0x7f95100cb780 2024-03-29T08:23:50.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.189+0000 7f9507e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95100ae810 0x7f95100caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.189+0000 7f950fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f95100ce8a0 0x7f95100cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41108/0 (socket says 172.21.15.46:41108) 2024-03-29T08:23:50.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.189+0000 7f9507e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f95100ae810 0x7f95100caf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52028/0 (socket says 172.21.15.46:52028) 2024-03-29T08:23:50.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.189+0000 7f950fe00640 1 -- 172.21.15.46:0/258278443 learned_addr learned my addr 172.21.15.46:0/258278443 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.189+0000 7f9514c00640 1 --2- 172.21.15.46:0/258278443 >> v2:172.21.15.73:3300/0 conn(0x7f95100cb780 0x7f95100aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.189+0000 7f9507e00640 1 -- 172.21.15.46:0/258278443 >> v2:172.21.15.73:3300/0 conn(0x7f95100cb780 msgr2=0x7f95100aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.189+0000 7f9507e00640 1 --2- 172.21.15.46:0/258278443 >> v2:172.21.15.73:3300/0 conn(0x7f95100cb780 0x7f95100aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.189+0000 7f9507e00640 1 -- 172.21.15.46:0/258278443 >> v1:172.21.15.46:6789/0 conn(0x7f95100ce8a0 legacy=0x7f95100cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.189+0000 7f950f400640 1 -- 172.21.15.46:0/258278443 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765453109 0 0) 0x7f95100cbc10 con 0x7f95100ce8a0 2024-03-29T08:23:50.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.189+0000 7f9507e00640 1 -- 172.21.15.46:0/258278443 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95100ca190 con 0x7f95100ae810 2024-03-29T08:23:50.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.193+0000 7f9507e00640 1 --2- 172.21.15.46:0/258278443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95100ae810 0x7f95100caf50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f9510070890 tx=0x7f94f801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4a7377bc9d8f66 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.193+0000 7f950f400640 1 -- 172.21.15.46:0/258278443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95100afa20 con 0x7f95100ae810 2024-03-29T08:23:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.193+0000 7f950f400640 1 -- 172.21.15.46:0/258278443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f80ab650 con 0x7f95100ae810 2024-03-29T08:23:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.193+0000 7f950f400640 1 -- 172.21.15.46:0/258278443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f80ab9e0 con 0x7f95100ae810 2024-03-29T08:23:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.193+0000 7f9516c00640 1 -- 172.21.15.46:0/258278443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95100ae810 msgr2=0x7f95100caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.193+0000 7f9516c00640 1 --2- 172.21.15.46:0/258278443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95100ae810 0x7f95100caf50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f9510070890 tx=0x7f94f801c450 comp rx=0 tx=0).stop 2024-03-29T08:23:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.193+0000 7f9516c00640 1 -- 172.21.15.46:0/258278443 shutdown_connections 2024-03-29T08:23:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.193+0000 7f9516c00640 1 --2- 172.21.15.46:0/258278443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95100ae810 0x7f95100caf50 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.193+0000 7f9516c00640 1 --2- 172.21.15.46:0/258278443 >> v2:172.21.15.73:3300/0 conn(0x7f95100cb780 0x7f95100aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.193+0000 7f9516c00640 1 -- 172.21.15.46:0/258278443 >> 172.21.15.46:0/258278443 conn(0x7f9510098a40 msgr2=0x7f95100cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.193+0000 7f9516c00640 1 -- 172.21.15.46:0/258278443 shutdown_connections 2024-03-29T08:23:50.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.193+0000 7f9516c00640 1 -- 172.21.15.46:0/258278443 wait complete. 2024-03-29T08:23:50.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f9516c00640 1 Processor -- start 2024-03-29T08:23:50.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f9516c00640 1 -- start start 2024-03-29T08:23:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f9516c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95100ae810 0x7f951014e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f9516c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95100cb780 0x7f9510151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f9516c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f95100ce8a0 0x7f95101529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f9516c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9510159790 con 0x7f95100ae810 2024-03-29T08:23:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f9516c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95100afcc0 con 0x7f95100cb780 2024-03-29T08:23:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f9516c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95101591d0 con 0x7f95100ce8a0 2024-03-29T08:23:50.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f950fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95100ae810 0x7f951014e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f950fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95100ae810 0x7f951014e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52036/0 (socket says 172.21.15.46:52036) 2024-03-29T08:23:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f950fe00640 1 -- 172.21.15.46:0/2874618658 learned_addr learned my addr 172.21.15.46:0/2874618658 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f9507e00640 1 --2- 172.21.15.46:0/2874618658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95100cb780 0x7f9510151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f9514c00640 1 --2- 172.21.15.46:0/2874618658 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f95100ce8a0 0x7f95101529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f950fe00640 1 -- 172.21.15.46:0/2874618658 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f95100ce8a0 msgr2=0x7f95101529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f950fe00640 1 --2- 172.21.15.46:0/2874618658 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f95100ce8a0 0x7f95101529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f950fe00640 1 -- 172.21.15.46:0/2874618658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95100cb780 msgr2=0x7f9510151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f950fe00640 1 --2- 172.21.15.46:0/2874618658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95100cb780 0x7f9510151b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f950fe00640 1 -- 172.21.15.46:0/2874618658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f8009420 con 0x7f95100ae810 2024-03-29T08:23:50.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f9514c00640 1 --2- 172.21.15.46:0/2874618658 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f95100ce8a0 0x7f95101529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:50.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f950fe00640 1 --2- 172.21.15.46:0/2874618658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95100ae810 0x7f951014e380 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f950000c480 tx=0x7f950000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f950cc00640 1 -- 172.21.15.46:0/2874618658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95000074e0 con 0x7f95100ae810 2024-03-29T08:23:50.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f950cc00640 1 -- 172.21.15.46:0/2874618658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9500007660 con 0x7f95100ae810 2024-03-29T08:23:50.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f950cc00640 1 -- 172.21.15.46:0/2874618658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9500004a30 con 0x7f95100ae810 2024-03-29T08:23:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f9516c00640 1 -- 172.21.15.46:0/2874618658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95101594b0 con 0x7f95100ae810 2024-03-29T08:23:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f9516c00640 1 -- 172.21.15.46:0/2874618658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95101515f0 con 0x7f95100ae810 2024-03-29T08:23:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.197+0000 7f9516c00640 1 -- 172.21.15.46:0/2874618658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9510066c90 con 0x7f95100ae810 2024-03-29T08:23:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.201+0000 7f950cc00640 1 -- 172.21.15.46:0/2874618658 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9500004b90 con 0x7f95100ae810 2024-03-29T08:23:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.201+0000 7f950cc00640 1 --2- 172.21.15.46:0/2874618658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f94ec066690 0x7f94ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.201+0000 7f950cc00640 1 -- 172.21.15.46:0/2874618658 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9500083340 con 0x7f95100ae810 2024-03-29T08:23:50.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.201+0000 7f9507e00640 1 --2- 172.21.15.46:0/2874618658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f94ec066690 0x7f94ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.201+0000 7f9507e00640 1 --2- 172.21.15.46:0/2874618658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f94ec066690 0x7f94ec068b50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f94f80af000 tx=0x7f94f80a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.205+0000 7f950cc00640 1 -- 172.21.15.46:0/2874618658 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9500054150 con 0x7f95100ae810 2024-03-29T08:23:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.405+0000 7f9516c00640 1 -- 172.21.15.46:0/2874618658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f951006e750 con 0x7f95100ae810 2024-03-29T08:23:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.405+0000 7f950cc00640 1 -- 172.21.15.46:0/2874618658 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f9500053ce0 con 0x7f95100ae810 2024-03-29T08:23:50.409 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:23:50.409 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3456771376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3456771376},{"type":"v1","addr":"172.21.15.73:6836","nonce":3456771376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2671538025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2671538025},{"type":"v1","addr":"172.21.15.73:6837","nonce":2671538025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3196015693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3196015693},{"type":"v1","addr":"172.21.15.46:6833","nonce":3196015693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:22:43.605457+0000","modified":"2024-03-29T08:23:13.950990+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":6,"addr":"172.21.15.46:6835/1672348161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1672348161},{"type":"v1","addr":"172.21.15.46:6835","nonce":1672348161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-29T08:23:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 48 2024-03-29T08:23:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.405+0000 7f9516c00640 1 -- 172.21.15.46:0/2874618658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f94ec066690 msgr2=0x7f94ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.405+0000 7f9516c00640 1 --2- 172.21.15.46:0/2874618658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f94ec066690 0x7f94ec068b50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f94f80af000 tx=0x7f94f80a7000 comp rx=0 tx=0).stop 2024-03-29T08:23:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.405+0000 7f9516c00640 1 -- 172.21.15.46:0/2874618658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95100ae810 msgr2=0x7f951014e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.405+0000 7f9516c00640 1 --2- 172.21.15.46:0/2874618658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95100ae810 0x7f951014e380 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f950000c480 tx=0x7f950000c940 comp rx=0 tx=0).stop 2024-03-29T08:23:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.405+0000 7f9516c00640 1 -- 172.21.15.46:0/2874618658 shutdown_connections 2024-03-29T08:23:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.405+0000 7f9516c00640 1 --2- 172.21.15.46:0/2874618658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f94ec066690 0x7f94ec068b50 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.405+0000 7f9516c00640 1 --2- 172.21.15.46:0/2874618658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95100cb780 0x7f9510151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.405+0000 7f9516c00640 1 --2- 172.21.15.46:0/2874618658 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f95100ce8a0 0x7f95101529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.405+0000 7f9516c00640 1 --2- 172.21.15.46:0/2874618658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95100ae810 0x7f951014e380 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.405+0000 7f9516c00640 1 -- 172.21.15.46:0/2874618658 >> 172.21.15.46:0/2874618658 conn(0x7f9510098a40 msgr2=0x7f95100caf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.405+0000 7f9516c00640 1 -- 172.21.15.46:0/2874618658 shutdown_connections 2024-03-29T08:23:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.409+0000 7f9516c00640 1 -- 172.21.15.46:0/2874618658 wait complete. 2024-03-29T08:23:50.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-03-29T08:23:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.569+0000 7f33bb400640 1 Processor -- start 2024-03-29T08:23:50.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.569+0000 7f33bb400640 1 -- start start 2024-03-29T08:23:50.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.569+0000 7f33bb400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f33b40cbea0 0x7f33b40ae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.569+0000 7f33bb400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b415d260 con 0x7f33b415df90 2024-03-29T08:23:50.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.569+0000 7f33bb400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b40aefd0 con 0x7f33b4142a50 2024-03-29T08:23:50.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.569+0000 7f33bb400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b415d4b0 con 0x7f33b40cbea0 2024-03-29T08:23:50.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.569+0000 7f33b9400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f33b40cbea0 0x7f33b40ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.569+0000 7f33abe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f33b415df90 0x7f33b40ae280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41110/0 (socket says 172.21.15.46:41110) 2024-03-29T08:23:50.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.569+0000 7f33abe00640 1 -- 172.21.15.46:0/3814515712 learned_addr learned my addr 172.21.15.46:0/3814515712 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:50.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.573+0000 7f33b9400640 1 -- 172.21.15.46:0/3814515712 >> v1:172.21.15.73:6789/0 conn(0x7f33b4142a50 legacy=0x7f33b40cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.573+0000 7f33b9400640 1 -- 172.21.15.46:0/3814515712 >> v1:172.21.15.46:6789/0 conn(0x7f33b415df90 legacy=0x7f33b40ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.573+0000 7f33b9400640 1 -- 172.21.15.46:0/3814515712 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b415d790 con 0x7f33b40cbea0 2024-03-29T08:23:50.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.573+0000 7f33b3e00640 1 -- 172.21.15.46:0/3814515712 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638085511 0 0) 0x7f33b415d260 con 0x7f33b415df90 2024-03-29T08:23:50.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.573+0000 7f33b9400640 1 --2- 172.21.15.46:0/3814515712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b40cbea0 0x7f33b40ae990 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009d90 tx=0x7f33ac01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c257cec31c32e534 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.573+0000 7f33b3e00640 1 -- 172.21.15.46:0/3814515712 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0a57a0 con 0x7f33b40cbea0 2024-03-29T08:23:50.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.573+0000 7f33b3e00640 1 -- 172.21.15.46:0/3814515712 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ac0a5900 con 0x7f33b40cbea0 2024-03-29T08:23:50.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.573+0000 7f33b3e00640 1 -- 172.21.15.46:0/3814515712 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0a5c90 con 0x7f33b40cbea0 2024-03-29T08:23:50.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.573+0000 7f33bb400640 1 -- 172.21.15.46:0/3814515712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b40cbea0 msgr2=0x7f33b40ae990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.573+0000 7f33bb400640 1 --2- 172.21.15.46:0/3814515712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b40cbea0 0x7f33b40ae990 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f33ac009d90 tx=0x7f33ac01c8e0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.573+0000 7f33bb400640 1 -- 172.21.15.46:0/3814515712 shutdown_connections 2024-03-29T08:23:50.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.573+0000 7f33bb400640 1 --2- 172.21.15.46:0/3814515712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b40cbea0 0x7f33b40ae990 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.573+0000 7f33bb400640 1 -- 172.21.15.46:0/3814515712 >> 172.21.15.46:0/3814515712 conn(0x7f33b406a390 msgr2=0x7f33b40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:50.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.573+0000 7f33bb400640 1 -- 172.21.15.46:0/3814515712 shutdown_connections 2024-03-29T08:23:50.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.573+0000 7f33bb400640 1 -- 172.21.15.46:0/3814515712 wait complete. 2024-03-29T08:23:50.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33bb400640 1 Processor -- start 2024-03-29T08:23:50.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33bb400640 1 -- start start 2024-03-29T08:23:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33bb400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f33b40cbea0 0x7f33b4159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33bb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33b4142a50 0x7f33b4158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33bb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b415df90 0x7f33b415b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33bb400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b415a530 con 0x7f33b415df90 2024-03-29T08:23:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33bb400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b415a810 con 0x7f33b4142a50 2024-03-29T08:23:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33bb400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b415aaf0 con 0x7f33b40cbea0 2024-03-29T08:23:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33b9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b415df90 0x7f33b415b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33b9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b415df90 0x7f33b415b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52044/0 (socket says 172.21.15.46:52044) 2024-03-29T08:23:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33b9400640 1 -- 172.21.15.46:0/187076116 learned_addr learned my addr 172.21.15.46:0/187076116 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:50.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33abe00640 1 --2- 172.21.15.46:0/187076116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33b4142a50 0x7f33b4158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33b8a00640 1 --2- 172.21.15.46:0/187076116 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f33b40cbea0 0x7f33b4159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33b9400640 1 -- 172.21.15.46:0/187076116 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f33b40cbea0 msgr2=0x7f33b4159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33b9400640 1 --2- 172.21.15.46:0/187076116 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f33b40cbea0 0x7f33b4159ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33b9400640 1 -- 172.21.15.46:0/187076116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33b4142a50 msgr2=0x7f33b4158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33b9400640 1 --2- 172.21.15.46:0/187076116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33b4142a50 0x7f33b4158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33b9400640 1 -- 172.21.15.46:0/187076116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ac009a70 con 0x7f33b415df90 2024-03-29T08:23:50.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33b8a00640 1 --2- 172.21.15.46:0/187076116 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f33b40cbea0 0x7f33b4159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:50.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33b9400640 1 --2- 172.21.15.46:0/187076116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b415df90 0x7f33b415b5d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0194f0 tx=0x7f33ac019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33b1600640 1 -- 172.21.15.46:0/187076116 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0ad8d0 con 0x7f33b415df90 2024-03-29T08:23:50.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33b1600640 1 -- 172.21.15.46:0/187076116 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ac0ada50 con 0x7f33b415df90 2024-03-29T08:23:50.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33b1600640 1 -- 172.21.15.46:0/187076116 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0ade00 con 0x7f33b415df90 2024-03-29T08:23:50.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33bb400640 1 -- 172.21.15.46:0/187076116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b415add0 con 0x7f33b415df90 2024-03-29T08:23:50.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.577+0000 7f33bb400640 1 -- 172.21.15.46:0/187076116 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b415a220 con 0x7f33b415df90 2024-03-29T08:23:50.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.581+0000 7f33b1600640 1 -- 172.21.15.46:0/187076116 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f33ac0b9450 con 0x7f33b415df90 2024-03-29T08:23:50.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.581+0000 7f33b1600640 1 --2- 172.21.15.46:0/187076116 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3390066690 0x7f3390068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.581+0000 7f33b1600640 1 -- 172.21.15.46:0/187076116 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f33ac11f7a0 con 0x7f33b415df90 2024-03-29T08:23:50.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.581+0000 7f33b8a00640 1 --2- 172.21.15.46:0/187076116 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3390066690 0x7f3390068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.581+0000 7f33be8f1000 1 Processor -- start 2024-03-29T08:23:50.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.581+0000 7f33be8f1000 1 -- start start 2024-03-29T08:23:50.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.581+0000 7f33b8a00640 1 --2- 172.21.15.46:0/187076116 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3390066690 0x7f3390068b50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f33a4000f70 tx=0x7f33a4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31947c290 0x55e31947c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947cbb0 0x55e319432c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e3194331b0 0x55e319433590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3192fb340 con 0x55e31947cbb0 2024-03-29T08:23:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e319287cd0 con 0x55e31947c290 2024-03-29T08:23:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3192852a0 con 0x55e3194331b0 2024-03-29T08:23:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947cbb0 0x55e319432c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947cbb0 0x55e319432c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52056/0 (socket says 172.21.15.46:52056) 2024-03-29T08:23:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33abe00640 1 -- 172.21.15.46:0/1746022016 learned_addr learned my addr 172.21.15.46:0/1746022016 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33b8a00640 1 --2- 172.21.15.46:0/1746022016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31947c290 0x55e31947c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33b9400640 1 --2- 172.21.15.46:0/1746022016 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e3194331b0 0x55e319433590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33abe00640 1 -- 172.21.15.46:0/1746022016 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e3194331b0 msgr2=0x55e319433590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33abe00640 1 --2- 172.21.15.46:0/1746022016 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e3194331b0 0x55e319433590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33abe00640 1 -- 172.21.15.46:0/1746022016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31947c290 msgr2=0x55e31947c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33abe00640 1 --2- 172.21.15.46:0/1746022016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31947c290 0x55e31947c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33abe00640 1 -- 172.21.15.46:0/1746022016 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e318f734b0 con 0x55e31947cbb0 2024-03-29T08:23:50.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33abe00640 1 --2- 172.21.15.46:0/1746022016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947cbb0 0x55e319432c70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f339c0076c0 tx=0x7f339c004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a379af14887f0a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33a9600640 1 -- 172.21.15.46:0/1746022016 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c007240 con 0x55e31947cbb0 2024-03-29T08:23:50.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33a9600640 1 -- 172.21.15.46:0/1746022016 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c0017f0 con 0x55e31947cbb0 2024-03-29T08:23:50.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33a9600640 1 -- 172.21.15.46:0/1746022016 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c001b80 con 0x55e31947cbb0 2024-03-29T08:23:50.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 -- 172.21.15.46:0/1746022016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947cbb0 msgr2=0x55e319432c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 --2- 172.21.15.46:0/1746022016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947cbb0 0x55e319432c70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f339c0076c0 tx=0x7f339c004bc0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 -- 172.21.15.46:0/1746022016 shutdown_connections 2024-03-29T08:23:50.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 --2- 172.21.15.46:0/1746022016 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e3194331b0 0x55e319433590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 --2- 172.21.15.46:0/1746022016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947cbb0 0x55e319432c70 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 --2- 172.21.15.46:0/1746022016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31947c290 0x55e31947c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 -- 172.21.15.46:0/1746022016 >> 172.21.15.46:0/1746022016 conn(0x55e31942f160 msgr2=0x55e319431550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:50.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 -- 172.21.15.46:0/1746022016 shutdown_connections 2024-03-29T08:23:50.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 -- 172.21.15.46:0/1746022016 wait complete. 2024-03-29T08:23:50.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 -- start start 2024-03-29T08:23:50.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 Processor -- start 2024-03-29T08:23:50.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e3194331b0 0x55e3194386d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e31947c290 0x55e319438c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947cbb0 0x55e31943de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e319280b90 con 0x55e31947cbb0 2024-03-29T08:23:50.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31923e0a0 con 0x55e3194331b0 2024-03-29T08:23:50.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.585+0000 7f33be8f1000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e31923dd40 con 0x55e31947c290 2024-03-29T08:23:50.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 10 client.0 _ll_register_callbacks cb 0x55e319335c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:23:50.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33b9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947cbb0 0x55e31943de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33b9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947cbb0 0x55e31943de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52070/0 (socket says 172.21.15.46:52070) 2024-03-29T08:23:50.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33b9400640 1 -- 172.21.15.46:0/1390802578 learned_addr learned my addr 172.21.15.46:0/1390802578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:50.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33b8a00640 1 --2- 172.21.15.46:0/1390802578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e3194331b0 0x55e3194386d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33abe00640 1 --2- 172.21.15.46:0/1390802578 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e31947c290 0x55e319438c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33b9400640 1 -- 172.21.15.46:0/1390802578 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e31947c290 msgr2=0x55e319438c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:50.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33b9400640 1 --2- 172.21.15.46:0/1390802578 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e31947c290 0x55e319438c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33b9400640 1 -- 172.21.15.46:0/1390802578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e3194331b0 msgr2=0x55e3194386d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:50.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33b9400640 1 --2- 172.21.15.46:0/1390802578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e3194331b0 0x55e3194386d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33b9400640 1 -- 172.21.15.46:0/1390802578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f339c0073a0 con 0x55e31947cbb0 2024-03-29T08:23:50.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33b9400640 1 --2- 172.21.15.46:0/1390802578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947cbb0 0x55e31943de40 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0a58d0 tx=0x7f33ac0f0760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9706faca3a89a6a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33a2000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:23:50.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33a2000640 1 -- 172.21.15.46:0/1390802578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac0f0b00 con 0x55e31947cbb0 2024-03-29T08:23:50.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33a2000640 1 -- 172.21.15.46:0/1390802578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f33ac0efb90 con 0x55e31947cbb0 2024-03-29T08:23:50.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 1 -- 172.21.15.46:0/1390802578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e319250550 con 0x55e31947cbb0 2024-03-29T08:23:50.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33a2000640 1 -- 172.21.15.46:0/1390802578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33ac0efd50 con 0x55e31947cbb0 2024-03-29T08:23:50.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 10 client.7101 fetch_fsmap learned FSMap version 48 2024-03-29T08:23:50.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 1 -- 172.21.15.46:0/1390802578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55e319245ba0 con 0x55e31947cbb0 2024-03-29T08:23:50.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33a2000640 1 -- 172.21.15.46:0/1390802578 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e319250550 con 0x55e31947cbb0 2024-03-29T08:23:50.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33a2000640 1 -- 172.21.15.46:0/1390802578 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 48) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f33ac005d20 con 0x55e31947cbb0 2024-03-29T08:23:50.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 10 client.7101 fetch_fsmap finished waiting for FSMap version 48 2024-03-29T08:23:50.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 10 client.7101 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:23:50.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 20 client.7101 populate_metadata read hostname 'smithi046' 2024-03-29T08:23:50.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 1 --2- 172.21.15.46:0/1390802578 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55e31948b320 0x55e31948d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 4 client.7101 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:23:50.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 1 -- 172.21.15.46:0/1390802578 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e31927d970 con 0x55e31948b320 2024-03-29T08:23:50.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33b8a00640 1 --2- 172.21.15.46:0/1390802578 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55e31948b320 0x55e31948d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33b8a00640 1 --2- 172.21.15.46:0/1390802578 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55e31948b320 0x55e31948d710 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e2f344c4185cfc server_cookie=20fd16e50be07255 in_seq=0 out_seq=0 2024-03-29T08:23:50.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33a2000640 10 client.7101 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:50.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33a2000640 1 -- 172.21.15.46:0/1390802578 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55e31927d970 con 0x55e31948b320 2024-03-29T08:23:50.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33a2000640 10 client.7101 handle_command_reply: tid=0 2024-03-29T08:23:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 1 client.7101 shutdown 2024-03-29T08:23:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 1 -- 172.21.15.46:0/1390802578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947cbb0 msgr2=0x55e31943de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 1 --2- 172.21.15.46:0/1390802578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947cbb0 0x55e31943de40 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0a58d0 tx=0x7f33ac0f0760 comp rx=0 tx=0).stop 2024-03-29T08:23:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 1 -- 172.21.15.46:0/1390802578 shutdown_connections 2024-03-29T08:23:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 1 --2- 172.21.15.46:0/1390802578 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55e31948b320 0x55e31948d710 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 1 --2- 172.21.15.46:0/1390802578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947cbb0 0x55e31943de40 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 1 --2- 172.21.15.46:0/1390802578 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e31947c290 0x55e319438c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 1 --2- 172.21.15.46:0/1390802578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e3194331b0 0x55e3194386d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33a2000640 0 client.7101 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 1 -- 172.21.15.46:0/1390802578 >> 172.21.15.46:0/1390802578 conn(0x55e31942f160 msgr2=0x55e319431de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 1 -- 172.21.15.46:0/1390802578 shutdown_connections 2024-03-29T08:23:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 1 -- 172.21.15.46:0/1390802578 wait complete. 2024-03-29T08:23:50.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.589+0000 7f33be8f1000 20 client.7101 trim_cache size 0 max 16384 2024-03-29T08:23:50.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.601+0000 7f33be8f1000 1 Processor -- start 2024-03-29T08:23:50.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.601+0000 7f33be8f1000 1 -- start start 2024-03-29T08:23:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33be8f1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3194331b0 0x55e31947b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33be8f1000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e31947c290 0x55e31947a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33be8f1000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31947cbb0 0x55e31947a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33be8f1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e318f3dc40 con 0x55e3194331b0 2024-03-29T08:23:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33be8f1000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e319284b90 con 0x55e31947cbb0 2024-03-29T08:23:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33be8f1000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e319288f60 con 0x55e31947c290 2024-03-29T08:23:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33b8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3194331b0 0x55e31947b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33b8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3194331b0 0x55e31947b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52080/0 (socket says 172.21.15.46:52080) 2024-03-29T08:23:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33b8a00640 1 -- 172.21.15.46:0/3276503643 learned_addr learned my addr 172.21.15.46:0/3276503643 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33b9400640 1 --2- 172.21.15.46:0/3276503643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31947cbb0 0x55e31947a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33abe00640 1 --2- 172.21.15.46:0/3276503643 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e31947c290 0x55e31947a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33b8a00640 1 -- 172.21.15.46:0/3276503643 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e31947c290 msgr2=0x55e31947a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33b8a00640 1 --2- 172.21.15.46:0/3276503643 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e31947c290 0x55e31947a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33b8a00640 1 -- 172.21.15.46:0/3276503643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31947cbb0 msgr2=0x55e31947a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33b8a00640 1 --2- 172.21.15.46:0/3276503643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31947cbb0 0x55e31947a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33b8a00640 1 -- 172.21.15.46:0/3276503643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e31922bef0 con 0x55e3194331b0 2024-03-29T08:23:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33b8a00640 1 --2- 172.21.15.46:0/3276503643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3194331b0 0x55e31947b810 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f33a400e530 tx=0x7f33a4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92dda3a55ed16088 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33aa000640 1 -- 172.21.15.46:0/3276503643 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4005190 con 0x55e3194331b0 2024-03-29T08:23:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33aa000640 1 -- 172.21.15.46:0/3276503643 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a402a070 con 0x55e3194331b0 2024-03-29T08:23:50.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33aa000640 1 -- 172.21.15.46:0/3276503643 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4026440 con 0x55e3194331b0 2024-03-29T08:23:50.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33be8f1000 1 -- 172.21.15.46:0/3276503643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3194331b0 msgr2=0x55e31947b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33be8f1000 1 --2- 172.21.15.46:0/3276503643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3194331b0 0x55e31947b810 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f33a400e530 tx=0x7f33a4012000 comp rx=0 tx=0).stop 2024-03-29T08:23:50.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33be8f1000 1 -- 172.21.15.46:0/3276503643 shutdown_connections 2024-03-29T08:23:50.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33be8f1000 1 --2- 172.21.15.46:0/3276503643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e31947cbb0 0x55e31947a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33be8f1000 1 --2- 172.21.15.46:0/3276503643 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e31947c290 0x55e31947a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33be8f1000 1 --2- 172.21.15.46:0/3276503643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3194331b0 0x55e31947b810 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33be8f1000 1 -- 172.21.15.46:0/3276503643 >> 172.21.15.46:0/3276503643 conn(0x55e31942f160 msgr2=0x55e319478d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:50.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33be8f1000 1 -- 172.21.15.46:0/3276503643 shutdown_connections 2024-03-29T08:23:50.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33be8f1000 1 -- 172.21.15.46:0/3276503643 wait complete. 2024-03-29T08:23:50.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33be8f1000 1 -- start start 2024-03-29T08:23:50.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.605+0000 7f33be8f1000 1 Processor -- start 2024-03-29T08:23:50.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e3194331b0 0x55e31948e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947c290 0x55e31948e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e31947cbb0 0x55e319496ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e319436860 con 0x55e31947c290 2024-03-29T08:23:50.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e31927e3a0 con 0x55e3194331b0 2024-03-29T08:23:50.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e31927fa20 con 0x55e31947cbb0 2024-03-29T08:23:50.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 10 client.0 _ll_register_callbacks cb 0x55e31931e870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:23:50.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947c290 0x55e31948e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33abe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947c290 0x55e31948e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52090/0 (socket says 172.21.15.46:52090) 2024-03-29T08:23:50.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33abe00640 1 -- 172.21.15.46:0/1091212237 learned_addr learned my addr 172.21.15.46:0/1091212237 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:50.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33b8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e3194331b0 0x55e31948e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33b9400640 1 --2- 172.21.15.46:0/1091212237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e31947cbb0 0x55e319496ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33abe00640 1 -- 172.21.15.46:0/1091212237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e31947cbb0 msgr2=0x55e319496ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:50.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33abe00640 1 --2- 172.21.15.46:0/1091212237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e31947cbb0 0x55e319496ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33abe00640 1 -- 172.21.15.46:0/1091212237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e3194331b0 msgr2=0x55e31948e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:50.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33abe00640 1 --2- 172.21.15.46:0/1091212237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e3194331b0 0x55e31948e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33abe00640 1 -- 172.21.15.46:0/1091212237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a401d040 con 0x55e31947c290 2024-03-29T08:23:50.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33b8a00640 1 --2- 172.21.15.46:0/1091212237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e3194331b0 0x55e31948e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:23:50.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33abe00640 1 --2- 172.21.15.46:0/1091212237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947c290 0x55e31948e8c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f339c007a50 tx=0x7f339c007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f698d517fd3cb005 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33a2000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:23:50.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33a2000640 1 -- 172.21.15.46:0/1091212237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c014860 con 0x55e31947c290 2024-03-29T08:23:50.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33a2000640 1 -- 172.21.15.46:0/1091212237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f339c0149c0 con 0x55e31947c290 2024-03-29T08:23:50.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 -- 172.21.15.46:0/1091212237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e319230620 con 0x55e31947c290 2024-03-29T08:23:50.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33a2000640 1 -- 172.21.15.46:0/1091212237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f339c014bc0 con 0x55e31947c290 2024-03-29T08:23:50.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 10 client.7107 fetch_fsmap learned FSMap version 48 2024-03-29T08:23:50.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 -- 172.21.15.46:0/1091212237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55e318f7cfc0 con 0x55e31947c290 2024-03-29T08:23:50.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33a2000640 1 -- 172.21.15.46:0/1091212237 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e319230620 con 0x55e31947c290 2024-03-29T08:23:50.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33a2000640 1 -- 172.21.15.46:0/1091212237 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 48) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f339c01f040 con 0x55e31947c290 2024-03-29T08:23:50.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 10 client.7107 fetch_fsmap finished waiting for FSMap version 48 2024-03-29T08:23:50.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 10 client.7107 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:23:50.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 20 client.7107 populate_metadata read hostname 'smithi046' 2024-03-29T08:23:50.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 --2- 172.21.15.46:0/1091212237 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55e3194d9c10 0x55e319497010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 4 client.7107 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-29T08:23:50.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 -- 172.21.15.46:0/1091212237 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55e31927f4e0 con 0x55e3194d9c10 2024-03-29T08:23:50.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33b8a00640 1 --2- 172.21.15.46:0/1091212237 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55e3194d9c10 0x55e319497010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33b8a00640 1 --2- 172.21.15.46:0/1091212237 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55e3194d9c10 0x55e319497010 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ca41f3e636eb4a8 server_cookie=f942187ddfa7603c in_seq=0 out_seq=0 2024-03-29T08:23:50.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33a2000640 10 client.7107 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:50.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33a2000640 1 -- 172.21.15.46:0/1091212237 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+237 (crc 0 0 0) 0x55e31927f4e0 con 0x55e3194d9c10 2024-03-29T08:23:50.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33a2000640 10 client.7107 handle_command_reply: tid=0 2024-03-29T08:23:50.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 client.7107 shutdown 2024-03-29T08:23:50.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 -- 172.21.15.46:0/1091212237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947c290 msgr2=0x55e31948e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 --2- 172.21.15.46:0/1091212237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947c290 0x55e31948e8c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f339c007a50 tx=0x7f339c007e50 comp rx=0 tx=0).stop 2024-03-29T08:23:50.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 -- 172.21.15.46:0/1091212237 shutdown_connections 2024-03-29T08:23:50.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 --2- 172.21.15.46:0/1091212237 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x55e3194d9c10 0x55e319497010 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 --2- 172.21.15.46:0/1091212237 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55e31947cbb0 0x55e319496ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 --2- 172.21.15.46:0/1091212237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e31947c290 0x55e31948e8c0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33a2000640 0 client.7107 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:50.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 --2- 172.21.15.46:0/1091212237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e3194331b0 0x55e31948e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 -- 172.21.15.46:0/1091212237 >> 172.21.15.46:0/1091212237 conn(0x55e31942f160 msgr2=0x55e31947ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:50.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 -- 172.21.15.46:0/1091212237 shutdown_connections 2024-03-29T08:23:50.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 1 -- 172.21.15.46:0/1091212237 wait complete. 2024-03-29T08:23:50.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.609+0000 7f33be8f1000 20 client.7107 trim_cache size 0 max 16384 2024-03-29T08:23:50.619 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:23:50.619 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-03-29T08:23:50.619 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:23:50.619 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "dentry", 2024-03-29T08:23:50.619 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 1933737389, 2024-03-29T08:23:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627776, 2024-03-29T08:23:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: "frag": "*", 2024-03-29T08:23:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: "dname": "file_to_be_damaged", 2024-03-29T08:23:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_id": "head", 2024-03-29T08:23:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/subdir/file_to_be_damaged" 2024-03-29T08:23:50.620 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:23:50.620 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-03-29T08:23:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.613+0000 7f33bb400640 1 -- 172.21.15.46:0/187076116 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3390066690 msgr2=0x7f3390068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.613+0000 7f33bb400640 1 --2- 172.21.15.46:0/187076116 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3390066690 0x7f3390068b50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f33a4000f70 tx=0x7f33a4009040 comp rx=0 tx=0).stop 2024-03-29T08:23:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.613+0000 7f33bb400640 1 -- 172.21.15.46:0/187076116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b415df90 msgr2=0x7f33b415b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.613+0000 7f33bb400640 1 --2- 172.21.15.46:0/187076116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b415df90 0x7f33b415b5d0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f33ac0194f0 tx=0x7f33ac019680 comp rx=0 tx=0).stop 2024-03-29T08:23:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.613+0000 7f33bb400640 1 -- 172.21.15.46:0/187076116 shutdown_connections 2024-03-29T08:23:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.613+0000 7f33bb400640 1 --2- 172.21.15.46:0/187076116 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3390066690 0x7f3390068b50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.613+0000 7f33bb400640 1 --2- 172.21.15.46:0/187076116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33b4142a50 0x7f33b4158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.613+0000 7f33bb400640 1 --2- 172.21.15.46:0/187076116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33b415df90 0x7f33b415b5d0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.613+0000 7f33bb400640 1 --2- 172.21.15.46:0/187076116 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f33b40cbea0 0x7f33b4159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.613+0000 7f33bb400640 1 -- 172.21.15.46:0/187076116 >> 172.21.15.46:0/187076116 conn(0x7f33b406a390 msgr2=0x7f33b40cd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:50.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.613+0000 7f33bb400640 1 -- 172.21.15.46:0/187076116 shutdown_connections 2024-03-29T08:23:50.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.613+0000 7f33bb400640 1 -- 172.21.15.46:0/187076116 wait complete. 2024-03-29T08:23:50.635 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-29T08:23:50.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.769+0000 7fea4ac00640 1 Processor -- start 2024-03-29T08:23:50.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.769+0000 7fea4ac00640 1 -- start start 2024-03-29T08:23:50.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.773+0000 7fea4ac00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fea440b0310 0x7fea440ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.773+0000 7fea4ac00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4406a5e0 con 0x7fea440aeee0 2024-03-29T08:23:50.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.773+0000 7fea4ac00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4406a8c0 con 0x7fea440b06f0 2024-03-29T08:23:50.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.773+0000 7fea4ac00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4406aba0 con 0x7fea440b0310 2024-03-29T08:23:50.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.773+0000 7fea49800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fea440b0310 0x7fea440ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.773+0000 7fea49800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fea440b0310 0x7fea440ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52094/0 (socket says 172.21.15.46:52094) 2024-03-29T08:23:50.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.773+0000 7fea49800640 1 -- 172.21.15.46:0/3315597735 learned_addr learned my addr 172.21.15.46:0/3315597735 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:50.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.773+0000 7fea49800640 1 -- 172.21.15.46:0/3315597735 >> v1:172.21.15.73:6789/0 conn(0x7fea440b06f0 legacy=0x7fea44069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.773+0000 7fea49800640 1 -- 172.21.15.46:0/3315597735 >> v1:172.21.15.46:6789/0 conn(0x7fea440aeee0 legacy=0x7fea440af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.773+0000 7fea49800640 1 -- 172.21.15.46:0/3315597735 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea440ae6e0 con 0x7fea440b0310 2024-03-29T08:23:50.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.773+0000 7fea49800640 1 --2- 172.21.15.46:0/3315597735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea440b0310 0x7fea440ce590 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fea38009870 tx=0x7fea3801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3b75320228d303 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea43e00640 1 -- 172.21.15.46:0/3315597735 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea380ab640 con 0x7fea440b0310 2024-03-29T08:23:50.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea43e00640 1 -- 172.21.15.46:0/3315597735 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea380a7070 con 0x7fea440b0310 2024-03-29T08:23:50.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea43e00640 1 -- 172.21.15.46:0/3315597735 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea380ab9c0 con 0x7fea440b0310 2024-03-29T08:23:50.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4ac00640 1 -- 172.21.15.46:0/3315597735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea440b0310 msgr2=0x7fea440ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4ac00640 1 --2- 172.21.15.46:0/3315597735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea440b0310 0x7fea440ce590 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fea38009870 tx=0x7fea3801c530 comp rx=0 tx=0).stop 2024-03-29T08:23:50.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4ac00640 1 -- 172.21.15.46:0/3315597735 shutdown_connections 2024-03-29T08:23:50.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4ac00640 1 --2- 172.21.15.46:0/3315597735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea440b0310 0x7fea440ce590 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4ac00640 1 -- 172.21.15.46:0/3315597735 >> 172.21.15.46:0/3315597735 conn(0x7fea44098a40 msgr2=0x7fea440cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:50.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4ac00640 1 -- 172.21.15.46:0/3315597735 shutdown_connections 2024-03-29T08:23:50.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4ac00640 1 -- 172.21.15.46:0/3315597735 wait complete. 2024-03-29T08:23:50.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4ac00640 1 Processor -- start 2024-03-29T08:23:50.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4ac00640 1 -- start start 2024-03-29T08:23:50.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4ac00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fea440aeee0 0x7fea440c5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4ac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea440b0310 0x7fea441575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea440b06f0 0x7fea440c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea44157b10 con 0x7fea440b06f0 2024-03-29T08:23:50.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4ac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea440c39b0 con 0x7fea440b0310 2024-03-29T08:23:50.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4ac00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea440c3c90 con 0x7fea440aeee0 2024-03-29T08:23:50.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea440b06f0 0x7fea440c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea48e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea440b0310 0x7fea441575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea49800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fea440aeee0 0x7fea440c5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea440b06f0 0x7fea440c5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52102/0 (socket says 172.21.15.46:52102) 2024-03-29T08:23:50.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.777+0000 7fea4a200640 1 -- 172.21.15.46:0/1051660461 learned_addr learned my addr 172.21.15.46:0/1051660461 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:50.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.781+0000 7fea4a200640 1 -- 172.21.15.46:0/1051660461 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fea440aeee0 msgr2=0x7fea440c5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.781+0000 7fea4a200640 1 --2- 172.21.15.46:0/1051660461 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fea440aeee0 0x7fea440c5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.781+0000 7fea4a200640 1 -- 172.21.15.46:0/1051660461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea440b0310 msgr2=0x7fea441575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.781+0000 7fea4a200640 1 --2- 172.21.15.46:0/1051660461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea440b0310 0x7fea441575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.781+0000 7fea4a200640 1 -- 172.21.15.46:0/1051660461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea380094d0 con 0x7fea440b06f0 2024-03-29T08:23:50.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.781+0000 7fea4a200640 1 --2- 172.21.15.46:0/1051660461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea440b06f0 0x7fea440c5020 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c730 tx=0x7fea3c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.781+0000 7fea41600640 1 -- 172.21.15.46:0/1051660461 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c003b80 con 0x7fea440b06f0 2024-03-29T08:23:50.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.781+0000 7fea41600640 1 -- 172.21.15.46:0/1051660461 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c003ce0 con 0x7fea440b06f0 2024-03-29T08:23:50.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.781+0000 7fea41600640 1 -- 172.21.15.46:0/1051660461 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c00aa30 con 0x7fea440b06f0 2024-03-29T08:23:50.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.781+0000 7fea4ac00640 1 -- 172.21.15.46:0/1051660461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea440c3f70 con 0x7fea440b06f0 2024-03-29T08:23:50.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.781+0000 7fea4ac00640 1 -- 172.21.15.46:0/1051660461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea440c4530 con 0x7fea440b06f0 2024-03-29T08:23:50.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.781+0000 7fea4ac00640 1 -- 172.21.15.46:0/1051660461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea441601d0 con 0x7fea440b06f0 2024-03-29T08:23:50.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.781+0000 7fea41600640 1 -- 172.21.15.46:0/1051660461 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fea3c00ab90 con 0x7fea440b06f0 2024-03-29T08:23:50.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.781+0000 7fea41600640 1 --2- 172.21.15.46:0/1051660461 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fea24066690 0x7fea24068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:50.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.785+0000 7fea41600640 1 -- 172.21.15.46:0/1051660461 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fea3c083f50 con 0x7fea440b06f0 2024-03-29T08:23:50.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.785+0000 7fea49800640 1 --2- 172.21.15.46:0/1051660461 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fea24066690 0x7fea24068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:50.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.785+0000 7fea49800640 1 --2- 172.21.15.46:0/1051660461 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fea24066690 0x7fea24068b50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fea380af000 tx=0x7fea380b6040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:50.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.789+0000 7fea41600640 1 -- 172.21.15.46:0/1051660461 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fea3c051240 con 0x7fea440b06f0 2024-03-29T08:23:50.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.977+0000 7fea4ac00640 1 -- 172.21.15.46:0/1051660461 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fea440c4720 con 0x7fea440b06f0 2024-03-29T08:23:50.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.981+0000 7fea41600640 1 -- 172.21.15.46:0/1051660461 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fea3c054860 con 0x7fea440b06f0 2024-03-29T08:23:50.984 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:23:50.984 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-03-29T08:23:50.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.981+0000 7fea4ac00640 1 -- 172.21.15.46:0/1051660461 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fea24066690 msgr2=0x7fea24068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.981+0000 7fea4ac00640 1 --2- 172.21.15.46:0/1051660461 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fea24066690 0x7fea24068b50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fea380af000 tx=0x7fea380b6040 comp rx=0 tx=0).stop 2024-03-29T08:23:50.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.981+0000 7fea4ac00640 1 -- 172.21.15.46:0/1051660461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea440b06f0 msgr2=0x7fea440c5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:50.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.981+0000 7fea4ac00640 1 --2- 172.21.15.46:0/1051660461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea440b06f0 0x7fea440c5020 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fea3c00c730 tx=0x7fea3c00cbf0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.981+0000 7fea4ac00640 1 -- 172.21.15.46:0/1051660461 shutdown_connections 2024-03-29T08:23:50.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.981+0000 7fea4ac00640 1 --2- 172.21.15.46:0/1051660461 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fea24066690 0x7fea24068b50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.981+0000 7fea4ac00640 1 --2- 172.21.15.46:0/1051660461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea440b0310 0x7fea441575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.981+0000 7fea4ac00640 1 --2- 172.21.15.46:0/1051660461 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea440b06f0 0x7fea440c5020 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.981+0000 7fea4ac00640 1 --2- 172.21.15.46:0/1051660461 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fea440aeee0 0x7fea440c5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:50.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.981+0000 7fea4ac00640 1 -- 172.21.15.46:0/1051660461 >> 172.21.15.46:0/1051660461 conn(0x7fea44098a40 msgr2=0x7fea440cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:50.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.981+0000 7fea4ac00640 1 -- 172.21.15.46:0/1051660461 shutdown_connections 2024-03-29T08:23:50.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:50.981+0000 7fea4ac00640 1 -- 172.21.15.46:0/1051660461 wait complete. 2024-03-29T08:23:51.007 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-29T08:23:51.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:51.007 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-03-29T08:23:51.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b9e00640 1 Processor -- start 2024-03-29T08:23:51.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b9e00640 1 -- start start 2024-03-29T08:23:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b9e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc0b40b21a0 0x7fc0b4070180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b9e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b40b1470 con 0x7fc0b40706c0 2024-03-29T08:23:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b9e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b406de60 con 0x7fc0b41590c0 2024-03-29T08:23:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b9e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b4070b50 con 0x7fc0b40b21a0 2024-03-29T08:23:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc0b40706c0 0x7fc0b40cbde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41118/0 (socket says 172.21.15.46:41118) 2024-03-29T08:23:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b3400640 1 -- 172.21.15.46:0/4260188904 learned_addr learned my addr 172.21.15.46:0/4260188904 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:23:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0aa000640 1 --2- 172.21.15.46:0/4260188904 >> v2:172.21.15.73:3300/0 conn(0x7fc0b40b21a0 0x7fc0b4070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b2000640 1 -- 172.21.15.46:0/4260188904 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 202217599 0 0) 0x7fc0b40b1470 con 0x7fc0b40706c0 2024-03-29T08:23:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b2000640 1 -- 172.21.15.46:0/4260188904 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc098004640 con 0x7fc0b40706c0 2024-03-29T08:23:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b2000640 1 -- 172.21.15.46:0/4260188904 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84679777 0 0) 0x7fc0b406de60 con 0x7fc0b41590c0 2024-03-29T08:23:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b2000640 1 -- 172.21.15.46:0/4260188904 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0b40b1470 con 0x7fc0b41590c0 2024-03-29T08:23:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b2000640 1 -- 172.21.15.46:0/4260188904 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1905530119 0 0) 0x7fc098004640 con 0x7fc0b40706c0 2024-03-29T08:23:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b2000640 1 -- 172.21.15.46:0/4260188904 >> v2:172.21.15.73:3300/0 conn(0x7fc0b40b21a0 msgr2=0x7fc0b4070180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b2000640 1 --2- 172.21.15.46:0/4260188904 >> v2:172.21.15.73:3300/0 conn(0x7fc0b40b21a0 0x7fc0b4070180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b2000640 1 -- 172.21.15.46:0/4260188904 >> v1:172.21.15.73:6789/0 conn(0x7fc0b41590c0 legacy=0x7fc0b406d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b2000640 1 -- 172.21.15.46:0/4260188904 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b40b16c0 con 0x7fc0b40706c0 2024-03-29T08:23:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b2000640 1 -- 172.21.15.46:0/4260188904 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fc0ac001a00 con 0x7fc0b40706c0 2024-03-29T08:23:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b2000640 1 -- 172.21.15.46:0/4260188904 >> v1:172.21.15.46:6789/0 conn(0x7fc0b40706c0 legacy=0x7fc0b40cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.149+0000 7fc0b2000640 1 --2- 172.21.15.46:0/4260188904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc098004c20 0x7fc098005020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0b2000640 1 --2- 172.21.15.46:0/4260188904 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0980058e0 0x7fc098005d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0b2000640 1 --2- 172.21.15.46:0/4260188904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0980062a0 0x7fc09800caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0b2000640 1 -- 172.21.15.46:0/4260188904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b4070b50 con 0x7fc098004c20 2024-03-29T08:23:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0b2000640 1 -- 172.21.15.46:0/4260188904 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09800d120 con 0x7fc0980062a0 2024-03-29T08:23:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0b2000640 1 -- 172.21.15.46:0/4260188904 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980034d0 con 0x7fc0980058e0 2024-03-29T08:23:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0b2a00640 1 --2- 172.21.15.46:0/4260188904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc098004c20 0x7fc098005020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0b3400640 1 --2- 172.21.15.46:0/4260188904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0980062a0 0x7fc09800caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0aa000640 1 --2- 172.21.15.46:0/4260188904 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0980058e0 0x7fc098005d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0b2a00640 1 -- 172.21.15.46:0/4260188904 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0980058e0 msgr2=0x7fc098005d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0b2a00640 1 --2- 172.21.15.46:0/4260188904 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0980058e0 0x7fc098005d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0b2a00640 1 -- 172.21.15.46:0/4260188904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0980062a0 msgr2=0x7fc09800caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0b2a00640 1 --2- 172.21.15.46:0/4260188904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0980062a0 0x7fc09800caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0b2a00640 1 -- 172.21.15.46:0/4260188904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc098004640 con 0x7fc098004c20 2024-03-29T08:23:51.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0b3400640 1 -- 172.21.15.46:0/4260188904 reap_dead start 2024-03-29T08:23:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0b2a00640 1 --2- 172.21.15.46:0/4260188904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc098004c20 0x7fc098005020 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40018d0 tx=0x7fc0a401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da41fd775bcadb69 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0b2000640 1 -- 172.21.15.46:0/4260188904 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a40052d0 con 0x7fc098004c20 2024-03-29T08:23:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0b2000640 1 -- 172.21.15.46:0/4260188904 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a40a7050 con 0x7fc098004c20 2024-03-29T08:23:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.153+0000 7fc0b2000640 1 -- 172.21.15.46:0/4260188904 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4003600 con 0x7fc098004c20 2024-03-29T08:23:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b9e00640 1 -- 172.21.15.46:0/4260188904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc098004c20 msgr2=0x7fc098005020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b9e00640 1 --2- 172.21.15.46:0/4260188904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc098004c20 0x7fc098005020 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40018d0 tx=0x7fc0a401c560 comp rx=0 tx=0).stop 2024-03-29T08:23:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b9e00640 1 -- 172.21.15.46:0/4260188904 shutdown_connections 2024-03-29T08:23:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b9e00640 1 --2- 172.21.15.46:0/4260188904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc098004c20 0x7fc098005020 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b9e00640 1 -- 172.21.15.46:0/4260188904 >> 172.21.15.46:0/4260188904 conn(0x7fc0b40b42d0 msgr2=0x7fc0b4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b9e00640 1 -- 172.21.15.46:0/4260188904 shutdown_connections 2024-03-29T08:23:51.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b9e00640 1 -- 172.21.15.46:0/4260188904 wait complete. 2024-03-29T08:23:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b9e00640 1 Processor -- start 2024-03-29T08:23:51.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b9e00640 1 -- start start 2024-03-29T08:23:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc098004c20 0x7fc0b41559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0b4152390 0x7fc0b41f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b9e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0b41575b0 0x7fc0b4157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b9e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b4157f70 con 0x7fc0b4152390 2024-03-29T08:23:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b9e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b41fab10 con 0x7fc098004c20 2024-03-29T08:23:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b9e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b41f9f00 con 0x7fc0b41575b0 2024-03-29T08:23:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0aa000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0b4152390 0x7fc0b41f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0aa000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0b4152390 0x7fc0b41f96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52114/0 (socket says 172.21.15.46:52114) 2024-03-29T08:23:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0aa000640 1 -- 172.21.15.46:0/913446961 learned_addr learned my addr 172.21.15.46:0/913446961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b2a00640 1 --2- 172.21.15.46:0/913446961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc098004c20 0x7fc0b41559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b3400640 1 --2- 172.21.15.46:0/913446961 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0b41575b0 0x7fc0b4157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0aa000640 1 -- 172.21.15.46:0/913446961 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0b41575b0 msgr2=0x7fc0b4157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0aa000640 1 --2- 172.21.15.46:0/913446961 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0b41575b0 0x7fc0b4157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0aa000640 1 -- 172.21.15.46:0/913446961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc098004c20 msgr2=0x7fc0b41559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0aa000640 1 --2- 172.21.15.46:0/913446961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc098004c20 0x7fc0b41559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0aa000640 1 -- 172.21.15.46:0/913446961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b41fa1e0 con 0x7fc0b4152390 2024-03-29T08:23:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.157+0000 7fc0b2a00640 1 --2- 172.21.15.46:0/913446961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc098004c20 0x7fc0b41559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.161+0000 7fc0aa000640 1 --2- 172.21.15.46:0/913446961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0b4152390 0x7fc0b41f96c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc09c011930 tx=0x7fc09c011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.161+0000 7fc0abe00640 1 -- 172.21.15.46:0/913446961 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c0095b0 con 0x7fc0b4152390 2024-03-29T08:23:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.161+0000 7fc0abe00640 1 -- 172.21.15.46:0/913446961 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09c004240 con 0x7fc0b4152390 2024-03-29T08:23:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.161+0000 7fc0b9e00640 1 -- 172.21.15.46:0/913446961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b41fa4c0 con 0x7fc0b4152390 2024-03-29T08:23:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.161+0000 7fc0abe00640 1 -- 172.21.15.46:0/913446961 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c00f8e0 con 0x7fc0b4152390 2024-03-29T08:23:51.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.161+0000 7fc0b9e00640 1 -- 172.21.15.46:0/913446961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b41f9c00 con 0x7fc0b4152390 2024-03-29T08:23:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.161+0000 7fc0abe00640 1 -- 172.21.15.46:0/913446961 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc09c00fa40 con 0x7fc0b4152390 2024-03-29T08:23:51.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.161+0000 7fc0abe00640 1 --2- 172.21.15.46:0/913446961 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0900666c0 0x7fc090068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.161+0000 7fc0abe00640 1 -- 172.21.15.46:0/913446961 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc09c084ae0 con 0x7fc0b4152390 2024-03-29T08:23:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.161+0000 7fc0bd1d9000 1 Processor -- start 2024-03-29T08:23:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.161+0000 7fc0b2a00640 1 --2- 172.21.15.46:0/913446961 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0900666c0 0x7fc090068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 -- start start 2024-03-29T08:23:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0b2a00640 1 --2- 172.21.15.46:0/913446961 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0900666c0 0x7fc090068b80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40ac000 tx=0x7fc0a40a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15f3a00 0x559fa15f3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f4320 0x559fa15a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15a9d70 0x559fa15aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fa1471820 con 0x559fa15f4320 2024-03-29T08:23:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fa13fea20 con 0x559fa15f3a00 2024-03-29T08:23:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fa13fbff0 con 0x559fa15a9d70 2024-03-29T08:23:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0aa000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f4320 0x559fa15a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0aa000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f4320 0x559fa15a9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52120/0 (socket says 172.21.15.46:52120) 2024-03-29T08:23:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0aa000640 1 -- 172.21.15.46:0/2116051151 learned_addr learned my addr 172.21.15.46:0/2116051151 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0b3400640 1 --2- 172.21.15.46:0/2116051151 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15a9d70 0x559fa15aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0b2a00640 1 --2- 172.21.15.46:0/2116051151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15f3a00 0x559fa15f3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0aa000640 1 -- 172.21.15.46:0/2116051151 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15a9d70 msgr2=0x559fa15aa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0aa000640 1 --2- 172.21.15.46:0/2116051151 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15a9d70 0x559fa15aa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0aa000640 1 -- 172.21.15.46:0/2116051151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15f3a00 msgr2=0x559fa15f3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0aa000640 1 --2- 172.21.15.46:0/2116051151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15f3a00 0x559fa15f3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0aa000640 1 -- 172.21.15.46:0/2116051151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fa10f3c90 con 0x559fa15f4320 2024-03-29T08:23:51.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0b3400640 1 --2- 172.21.15.46:0/2116051151 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15a9d70 0x559fa15aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:23:51.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0aa000640 1 --2- 172.21.15.46:0/2116051151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f4320 0x559fa15a9830 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0555d0 tx=0x7fc09c004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8711757c7a2d0194 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:51.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0a3e00640 1 -- 172.21.15.46:0/2116051151 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c059700 con 0x559fa15f4320 2024-03-29T08:23:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0a3e00640 1 -- 172.21.15.46:0/2116051151 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc09c05dd50 con 0x559fa15f4320 2024-03-29T08:23:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0a3e00640 1 -- 172.21.15.46:0/2116051151 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c051fe0 con 0x559fa15f4320 2024-03-29T08:23:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/2116051151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f4320 msgr2=0x559fa15a9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/2116051151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f4320 0x559fa15a9830 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0555d0 tx=0x7fc09c004700 comp rx=0 tx=0).stop 2024-03-29T08:23:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/2116051151 shutdown_connections 2024-03-29T08:23:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/2116051151 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15a9d70 0x559fa15aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/2116051151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f4320 0x559fa15a9830 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/2116051151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15f3a00 0x559fa15f3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/2116051151 >> 172.21.15.46:0/2116051151 conn(0x559fa15b02f0 msgr2=0x559fa15a8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/2116051151 shutdown_connections 2024-03-29T08:23:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/2116051151 wait complete. 2024-03-29T08:23:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 -- start start 2024-03-29T08:23:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.165+0000 7fc0bd1d9000 1 Processor -- start 2024-03-29T08:23:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0bd1d9000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15a9d70 0x559fa15f3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0bd1d9000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f3a00 0x559fa15aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0bd1d9000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15f4320 0x559fa15af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0bd1d9000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fa13f78e0 con 0x559fa15f3a00 2024-03-29T08:23:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0bd1d9000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fa1397230 con 0x559fa15a9d70 2024-03-29T08:23:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0bd1d9000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fa1396ed0 con 0x559fa15f4320 2024-03-29T08:23:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0bd1d9000 10 client.0 _ll_register_callbacks cb 0x559fa1595700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:23:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0aa000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f3a00 0x559fa15aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0aa000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f3a00 0x559fa15aeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52130/0 (socket says 172.21.15.46:52130) 2024-03-29T08:23:51.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0aa000640 1 -- 172.21.15.46:0/4027769796 learned_addr learned my addr 172.21.15.46:0/4027769796 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0b3400640 1 --2- 172.21.15.46:0/4027769796 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15f4320 0x559fa15af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0aa000640 1 -- 172.21.15.46:0/4027769796 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15f4320 msgr2=0x559fa15af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0b2a00640 1 --2- 172.21.15.46:0/4027769796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15a9d70 0x559fa15f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0aa000640 1 --2- 172.21.15.46:0/4027769796 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15f4320 0x559fa15af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0aa000640 1 -- 172.21.15.46:0/4027769796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15a9d70 msgr2=0x559fa15f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0aa000640 1 --2- 172.21.15.46:0/4027769796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15a9d70 0x559fa15f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0aa000640 1 -- 172.21.15.46:0/4027769796 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b40b16c0 con 0x559fa15f3a00 2024-03-29T08:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0b3400640 1 --2- 172.21.15.46:0/4027769796 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15f4320 0x559fa15af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0aa000640 1 --2- 172.21.15.46:0/4027769796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f3a00 0x559fa15aeec0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc09c05d820 tx=0x7fc09c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4245d960666e4053 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0a0c00640 1 -- 172.21.15.46:0/4027769796 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c05dd50 con 0x559fa15f3a00 2024-03-29T08:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0a0c00640 1 -- 172.21.15.46:0/4027769796 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc09c059700 con 0x559fa15f3a00 2024-03-29T08:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/4027769796 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fa13b06c0 con 0x559fa15f3a00 2024-03-29T08:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0a0c00640 1 -- 172.21.15.46:0/4027769796 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc09c06d1c0 con 0x559fa15f3a00 2024-03-29T08:23:51.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0bd1d9000 10 client.7128 fetch_fsmap learned FSMap version 48 2024-03-29T08:23:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/4027769796 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x7fc0b41f9c00 con 0x559fa15f3a00 2024-03-29T08:23:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0a0c00640 1 -- 172.21.15.46:0/4027769796 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559fa13b06c0 con 0x559fa15f3a00 2024-03-29T08:23:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0a0c00640 1 -- 172.21.15.46:0/4027769796 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 48) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc09c054dc0 con 0x559fa15f3a00 2024-03-29T08:23:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0bd1d9000 10 client.7128 fetch_fsmap finished waiting for FSMap version 48 2024-03-29T08:23:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0bd1d9000 10 client.7128 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-29T08:23:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0bd1d9000 20 client.7128 populate_metadata read hostname 'smithi046' 2024-03-29T08:23:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/4027769796 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x559fa15b6480 0x559fa1605cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0bd1d9000 4 client.7128 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:23:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/4027769796 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fa13c0b40 con 0x559fa15b6480 2024-03-29T08:23:51.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0b2a00640 1 --2- 172.21.15.46:0/4027769796 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x559fa15b6480 0x559fa1605cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0b2a00640 1 --2- 172.21.15.46:0/4027769796 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x559fa15b6480 0x559fa1605cc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c2cf7ee87e00b2d server_cookie=770de0f184205cac in_seq=0 out_seq=0 2024-03-29T08:23:51.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0a0c00640 10 client.7128 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0a0c00640 1 -- 172.21.15.46:0/4027769796 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559fa13c0b40 con 0x559fa15b6480 2024-03-29T08:23:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0a0c00640 10 client.7128 handle_command_reply: tid=0 2024-03-29T08:23:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.169+0000 7fc0bd1d9000 1 client.7128 shutdown 2024-03-29T08:23:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.173+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/4027769796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f3a00 msgr2=0x559fa15aeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.173+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/4027769796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f3a00 0x559fa15aeec0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc09c05d820 tx=0x7fc09c008040 comp rx=0 tx=0).stop 2024-03-29T08:23:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.173+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/4027769796 shutdown_connections 2024-03-29T08:23:51.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.173+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/4027769796 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x559fa15b6480 0x559fa1605cc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.173+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/4027769796 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15f4320 0x559fa15af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.173+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/4027769796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f3a00 0x559fa15aeec0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.173+0000 7fc0a0c00640 0 client.7128 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.173+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/4027769796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15a9d70 0x559fa15f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.173+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/4027769796 >> 172.21.15.46:0/4027769796 conn(0x559fa15b02f0 msgr2=0x559fa15a8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.173+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/4027769796 shutdown_connections 2024-03-29T08:23:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.173+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/4027769796 wait complete. 2024-03-29T08:23:51.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.173+0000 7fc0bd1d9000 20 client.7128 trim_cache size 0 max 16384 2024-03-29T08:23:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.185+0000 7fc0bd1d9000 1 Processor -- start 2024-03-29T08:23:51.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.185+0000 7fc0bd1d9000 1 -- start start 2024-03-29T08:23:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.185+0000 7fc0bd1d9000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15a9d70 0x559fa15f2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.185+0000 7fc0bd1d9000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15f3a00 0x559fa15f2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.185+0000 7fc0bd1d9000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f4320 0x559fa1603cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.185+0000 7fc0bd1d9000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fa15b5820 con 0x559fa15f4320 2024-03-29T08:23:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.185+0000 7fc0bd1d9000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fa10b4d40 con 0x559fa15a9d70 2024-03-29T08:23:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.185+0000 7fc0bd1d9000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fa13c1590 con 0x559fa15f3a00 2024-03-29T08:23:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f4320 0x559fa1603cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f4320 0x559fa1603cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52140/0 (socket says 172.21.15.46:52140) 2024-03-29T08:23:51.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0b3400640 1 -- 172.21.15.46:0/2458366263 learned_addr learned my addr 172.21.15.46:0/2458366263 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0aa000640 1 --2- 172.21.15.46:0/2458366263 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15f3a00 0x559fa15f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0b2a00640 1 --2- 172.21.15.46:0/2458366263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15a9d70 0x559fa15f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0b3400640 1 -- 172.21.15.46:0/2458366263 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15f3a00 msgr2=0x559fa15f2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0b3400640 1 --2- 172.21.15.46:0/2458366263 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15f3a00 0x559fa15f2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0b3400640 1 -- 172.21.15.46:0/2458366263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15a9d70 msgr2=0x559fa15f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0b3400640 1 --2- 172.21.15.46:0/2458366263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15a9d70 0x559fa15f2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0b3400640 1 -- 172.21.15.46:0/2458366263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fa13bcbe0 con 0x559fa15f4320 2024-03-29T08:23:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0aa000640 1 --2- 172.21.15.46:0/2458366263 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15f3a00 0x559fa15f2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:23:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0b3400640 1 --2- 172.21.15.46:0/2458366263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f4320 0x559fa1603cb0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc0b40b3830 tx=0x7fc0ac002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0ab6af009820de server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0a8c00640 1 -- 172.21.15.46:0/2458366263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac016420 con 0x559fa15f4320 2024-03-29T08:23:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0a8c00640 1 -- 172.21.15.46:0/2458366263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ac016580 con 0x559fa15f4320 2024-03-29T08:23:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0a8c00640 1 -- 172.21.15.46:0/2458366263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac0168f0 con 0x559fa15f4320 2024-03-29T08:23:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/2458366263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f4320 msgr2=0x559fa1603cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/2458366263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f4320 0x559fa1603cb0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc0b40b3830 tx=0x7fc0ac002190 comp rx=0 tx=0).stop 2024-03-29T08:23:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/2458366263 shutdown_connections 2024-03-29T08:23:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/2458366263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f4320 0x559fa1603cb0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/2458366263 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15f3a00 0x559fa15f2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/2458366263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15a9d70 0x559fa15f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/2458366263 >> 172.21.15.46:0/2458366263 conn(0x559fa15b02f0 msgr2=0x559fa15f0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/2458366263 shutdown_connections 2024-03-29T08:23:51.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/2458366263 wait complete. 2024-03-29T08:23:51.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 1 -- start start 2024-03-29T08:23:51.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 1 Processor -- start 2024-03-29T08:23:51.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15a9d70 0x559fa16072c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f3a00 0x559fa160f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15f4320 0x559fa160f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fa13b8ed0 con 0x559fa15f3a00 2024-03-29T08:23:51.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fa15f34a0 con 0x559fa15f4320 2024-03-29T08:23:51.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fa13f30d0 con 0x559fa15a9d70 2024-03-29T08:23:51.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0bd1d9000 10 client.0 _ll_register_callbacks cb 0x559fa1575b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:23:51.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0aa000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f3a00 0x559fa160f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0aa000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f3a00 0x559fa160f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52148/0 (socket says 172.21.15.46:52148) 2024-03-29T08:23:51.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0aa000640 1 -- 172.21.15.46:0/1538251457 learned_addr learned my addr 172.21.15.46:0/1538251457 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0b3400640 1 --2- 172.21.15.46:0/1538251457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15f4320 0x559fa160f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0b2a00640 1 --2- 172.21.15.46:0/1538251457 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15a9d70 0x559fa16072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0aa000640 1 -- 172.21.15.46:0/1538251457 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15a9d70 msgr2=0x559fa16072c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0aa000640 1 --2- 172.21.15.46:0/1538251457 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15a9d70 0x559fa16072c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0aa000640 1 -- 172.21.15.46:0/1538251457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15f4320 msgr2=0x559fa160f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0aa000640 1 --2- 172.21.15.46:0/1538251457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15f4320 0x559fa160f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0aa000640 1 -- 172.21.15.46:0/1538251457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ac001980 con 0x559fa15f3a00 2024-03-29T08:23:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.189+0000 7fc0b2a00640 1 --2- 172.21.15.46:0/1538251457 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15a9d70 0x559fa16072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0aa000640 1 --2- 172.21.15.46:0/1538251457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f3a00 0x559fa160f080 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0522f0 tx=0x7fc09c057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07e1ca56d1f6882 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:23:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0a0c00640 1 -- 172.21.15.46:0/1538251457 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09c06d1c0 con 0x559fa15f3a00 2024-03-29T08:23:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0a0c00640 1 -- 172.21.15.46:0/1538251457 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc09c059700 con 0x559fa15f3a00 2024-03-29T08:23:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/1538251457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fa139dfa0 con 0x559fa15f3a00 2024-03-29T08:23:51.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0a0c00640 1 -- 172.21.15.46:0/1538251457 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc09c055230 con 0x559fa15f3a00 2024-03-29T08:23:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 10 client.7134 fetch_fsmap learned FSMap version 48 2024-03-29T08:23:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/1538251457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x559fa13a2d30 con 0x559fa15f3a00 2024-03-29T08:23:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0a0c00640 1 -- 172.21.15.46:0/1538251457 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559fa139dfa0 con 0x559fa15f3a00 2024-03-29T08:23:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0a0c00640 1 -- 172.21.15.46:0/1538251457 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 48) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc09c0516c0 con 0x559fa15f3a00 2024-03-29T08:23:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 10 client.7134 fetch_fsmap finished waiting for FSMap version 48 2024-03-29T08:23:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 10 client.7134 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-29T08:23:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 20 client.7134 populate_metadata read hostname 'smithi046' 2024-03-29T08:23:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/1538251457 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x559fa16584e0 0x559fa165a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 4 client.7134 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-03-29T08:23:51.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/1538251457 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) v1 -- 0x559fa13f46c0 con 0x559fa16584e0 2024-03-29T08:23:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0b2a00640 1 --2- 172.21.15.46:0/1538251457 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x559fa16584e0 0x559fa165a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0a0c00640 10 client.7134 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0b2a00640 1 --2- 172.21.15.46:0/1538251457 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x559fa16584e0 0x559fa165a8d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13b0cdc52a679fcb server_cookie=6648de24dabe9536 in_seq=0 out_seq=0 2024-03-29T08:23:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0a0c00640 1 -- 172.21.15.46:0/1538251457 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559fa13f46c0 con 0x559fa16584e0 2024-03-29T08:23:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0a0c00640 10 client.7134 handle_command_reply: tid=0 2024-03-29T08:23:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 1 client.7134 shutdown 2024-03-29T08:23:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/1538251457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f3a00 msgr2=0x559fa160f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/1538251457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f3a00 0x559fa160f080 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0522f0 tx=0x7fc09c057000 comp rx=0 tx=0).stop 2024-03-29T08:23:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/1538251457 shutdown_connections 2024-03-29T08:23:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/1538251457 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x559fa16584e0 0x559fa165a8d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/1538251457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559fa15f4320 0x559fa160f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/1538251457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fa15f3a00 0x559fa160f080 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 1 --2- 172.21.15.46:0/1538251457 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559fa15a9d70 0x559fa16072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0a0c00640 0 client.7134 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:51.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/1538251457 >> 172.21.15.46:0/1538251457 conn(0x559fa15b02f0 msgr2=0x559fa15f12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:51.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/1538251457 shutdown_connections 2024-03-29T08:23:51.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 1 -- 172.21.15.46:0/1538251457 wait complete. 2024-03-29T08:23:51.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0bd1d9000 20 client.7134 trim_cache size 0 max 16384 2024-03-29T08:23:51.204 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:23:51.204 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-29T08:23:51.204 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "591a7d29-d60b-47ec-a847-06b8fc3962fa", 2024-03-29T08:23:51.204 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-29T08:23:51.204 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:23:51.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0b9e00640 1 -- 172.21.15.46:0/913446961 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0900666c0 msgr2=0x7fc090068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0b9e00640 1 --2- 172.21.15.46:0/913446961 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0900666c0 0x7fc090068b80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40ac000 tx=0x7fc0a40a6000 comp rx=0 tx=0).stop 2024-03-29T08:23:51.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0b9e00640 1 -- 172.21.15.46:0/913446961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0b4152390 msgr2=0x7fc0b41f96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0b9e00640 1 --2- 172.21.15.46:0/913446961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0b4152390 0x7fc0b41f96c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc09c011930 tx=0x7fc09c011df0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0b9e00640 1 -- 172.21.15.46:0/913446961 shutdown_connections 2024-03-29T08:23:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0b9e00640 1 --2- 172.21.15.46:0/913446961 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0900666c0 0x7fc090068b80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0b9e00640 1 --2- 172.21.15.46:0/913446961 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0b41575b0 0x7fc0b4157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0b9e00640 1 --2- 172.21.15.46:0/913446961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0b4152390 0x7fc0b41f96c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0b9e00640 1 --2- 172.21.15.46:0/913446961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc098004c20 0x7fc0b41559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0b9e00640 1 -- 172.21.15.46:0/913446961 >> 172.21.15.46:0/913446961 conn(0x7fc0b40b42d0 msgr2=0x7fc0b40cd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:51.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0b9e00640 1 -- 172.21.15.46:0/913446961 shutdown_connections 2024-03-29T08:23:51.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.193+0000 7fc0b9e00640 1 -- 172.21.15.46:0/913446961 wait complete. 2024-03-29T08:23:51.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:23:51.218 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-29T08:23:51.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.357+0000 7f4663200640 1 Processor -- start 2024-03-29T08:23:51.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.357+0000 7f4663200640 1 -- start start 2024-03-29T08:23:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.357+0000 7f4663200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f46640d1070 0x7f46640cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.357+0000 7f4663200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f466415c1c0 0x7f466415c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.357+0000 7f4663200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466406e150 con 0x7f46640ccdd0 2024-03-29T08:23:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.357+0000 7f4663200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46640afe30 con 0x7f466415c1c0 2024-03-29T08:23:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.357+0000 7f4663200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46640cfab0 con 0x7f46640d1070 2024-03-29T08:23:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.361+0000 7f4662800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f466415c1c0 0x7f466415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.361+0000 7f4662800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f466415c1c0 0x7f466415c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52158/0 (socket says 172.21.15.46:52158) 2024-03-29T08:23:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.361+0000 7f4662800640 1 -- 172.21.15.46:0/1121884053 learned_addr learned my addr 172.21.15.46:0/1121884053 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:51.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.361+0000 7f465be00640 1 --2- 172.21.15.46:0/1121884053 >> v2:172.21.15.73:3300/0 conn(0x7f46640d1070 0x7f46640cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.361+0000 7f4662800640 1 -- 172.21.15.46:0/1121884053 >> v2:172.21.15.73:3300/0 conn(0x7f46640d1070 msgr2=0x7f46640cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.361+0000 7f4662800640 1 --2- 172.21.15.46:0/1121884053 >> v2:172.21.15.73:3300/0 conn(0x7f46640d1070 0x7f46640cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.361+0000 7f4662800640 1 -- 172.21.15.46:0/1121884053 >> v1:172.21.15.73:6789/0 conn(0x7f46640ccdd0 legacy=0x7f46640cd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.361+0000 7f4662800640 1 -- 172.21.15.46:0/1121884053 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46640cad40 con 0x7f466415c1c0 2024-03-29T08:23:51.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.361+0000 7f4662800640 1 --2- 172.21.15.46:0/1121884053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466415c1c0 0x7f466415c630 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f465c009e50 tx=0x7f465c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2c474a39481789 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:51.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.361+0000 7f4661400640 1 -- 172.21.15.46:0/1121884053 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0a5810 con 0x7f466415c1c0 2024-03-29T08:23:51.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.361+0000 7f4661400640 1 -- 172.21.15.46:0/1121884053 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c0a5970 con 0x7f466415c1c0 2024-03-29T08:23:51.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.361+0000 7f4661400640 1 -- 172.21.15.46:0/1121884053 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0a5d00 con 0x7f466415c1c0 2024-03-29T08:23:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4663200640 1 -- 172.21.15.46:0/1121884053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466415c1c0 msgr2=0x7f466415c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4663200640 1 --2- 172.21.15.46:0/1121884053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466415c1c0 0x7f466415c630 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f465c009e50 tx=0x7f465c01c930 comp rx=0 tx=0).stop 2024-03-29T08:23:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4663200640 1 -- 172.21.15.46:0/1121884053 shutdown_connections 2024-03-29T08:23:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4663200640 1 --2- 172.21.15.46:0/1121884053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f466415c1c0 0x7f466415c630 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4663200640 1 --2- 172.21.15.46:0/1121884053 >> v2:172.21.15.73:3300/0 conn(0x7f46640d1070 0x7f46640cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4663200640 1 -- 172.21.15.46:0/1121884053 >> 172.21.15.46:0/1121884053 conn(0x7f46640d0550 msgr2=0x7f46640d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4663200640 1 -- 172.21.15.46:0/1121884053 shutdown_connections 2024-03-29T08:23:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4663200640 1 -- 172.21.15.46:0/1121884053 wait complete. 2024-03-29T08:23:51.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4663200640 1 Processor -- start 2024-03-29T08:23:51.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4663200640 1 -- start start 2024-03-29T08:23:51.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4663200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46640ccdd0 0x7f46641535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4663200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f46640d1070 0x7f4664156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4663200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f466415c1c0 0x7f46640c7620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4663200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46640ca990 con 0x7f46640ccdd0 2024-03-29T08:23:51.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4663200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4664157260 con 0x7f466415c1c0 2024-03-29T08:23:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4663200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46640c98f0 con 0x7f46640d1070 2024-03-29T08:23:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4661e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46640ccdd0 0x7f46641535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f465be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f46640d1070 0x7f4664156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4661e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46640ccdd0 0x7f46641535a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52162/0 (socket says 172.21.15.46:52162) 2024-03-29T08:23:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4661e00640 1 -- 172.21.15.46:0/1619331999 learned_addr learned my addr 172.21.15.46:0/1619331999 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4662800640 1 --2- 172.21.15.46:0/1619331999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f466415c1c0 0x7f46640c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4661e00640 1 -- 172.21.15.46:0/1619331999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f46640d1070 msgr2=0x7f4664156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4661e00640 1 --2- 172.21.15.46:0/1619331999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f46640d1070 0x7f4664156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4661e00640 1 -- 172.21.15.46:0/1619331999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f466415c1c0 msgr2=0x7f46640c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4661e00640 1 --2- 172.21.15.46:0/1619331999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f466415c1c0 0x7f46640c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4661e00640 1 -- 172.21.15.46:0/1619331999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c009b30 con 0x7f46640ccdd0 2024-03-29T08:23:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.365+0000 7f4661e00640 1 --2- 172.21.15.46:0/1619331999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46640ccdd0 0x7f46641535a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f465400c500 tx=0x7f465400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.369+0000 7f465a000640 1 -- 172.21.15.46:0/1619331999 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4654005220 con 0x7f46640ccdd0 2024-03-29T08:23:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.369+0000 7f465a000640 1 -- 172.21.15.46:0/1619331999 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4654005380 con 0x7f46640ccdd0 2024-03-29T08:23:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.369+0000 7f465a000640 1 -- 172.21.15.46:0/1619331999 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4654007270 con 0x7f46640ccdd0 2024-03-29T08:23:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.369+0000 7f4663200640 1 -- 172.21.15.46:0/1619331999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46640c9bd0 con 0x7f46640ccdd0 2024-03-29T08:23:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.369+0000 7f4663200640 1 -- 172.21.15.46:0/1619331999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46640ca190 con 0x7f46640ccdd0 2024-03-29T08:23:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.369+0000 7f465a000640 1 -- 172.21.15.46:0/1619331999 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f46540073d0 con 0x7f46640ccdd0 2024-03-29T08:23:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.369+0000 7f465a000640 1 --2- 172.21.15.46:0/1619331999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4640066690 0x7f4640068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.369+0000 7f465a000640 1 -- 172.21.15.46:0/1619331999 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4654083a30 con 0x7f46640ccdd0 2024-03-29T08:23:51.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.369+0000 7f466c0cb000 1 Processor -- start 2024-03-29T08:23:51.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.369+0000 7f465be00640 1 --2- 172.21.15.46:0/1619331999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4640066690 0x7f4640068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.369+0000 7f466c0cb000 1 -- start start 2024-03-29T08:23:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f465be00640 1 --2- 172.21.15.46:0/1619331999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4640066690 0x7f4640068b50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4664067790 tx=0x7f464c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f466c0cb000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae3a70 0x560b32ae3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f466c0cb000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32ae4390 0x560b32a974c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f466c0cb000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32a97a00 0x560b32a97de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f466c0cb000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b32962630 con 0x560b32ae3a70 2024-03-29T08:23:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f466c0cb000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b328efbd0 con 0x560b32ae4390 2024-03-29T08:23:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f466c0cb000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b328ed1a0 con 0x560b32a97a00 2024-03-29T08:23:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f4661e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae3a70 0x560b32ae3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f4661e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae3a70 0x560b32ae3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52170/0 (socket says 172.21.15.46:52170) 2024-03-29T08:23:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f4661e00640 1 -- 172.21.15.46:0/3048424881 learned_addr learned my addr 172.21.15.46:0/3048424881 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f465be00640 1 --2- 172.21.15.46:0/3048424881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32ae4390 0x560b32a974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f4662800640 1 --2- 172.21.15.46:0/3048424881 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32a97a00 0x560b32a97de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f4661e00640 1 -- 172.21.15.46:0/3048424881 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32a97a00 msgr2=0x560b32a97de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f4661e00640 1 --2- 172.21.15.46:0/3048424881 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32a97a00 0x560b32a97de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f4661e00640 1 -- 172.21.15.46:0/3048424881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32ae4390 msgr2=0x560b32a974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f4661e00640 1 --2- 172.21.15.46:0/3048424881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32ae4390 0x560b32a974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f4661e00640 1 -- 172.21.15.46:0/3048424881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b325e4e20 con 0x560b32ae3a70 2024-03-29T08:23:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f4661e00640 1 --2- 172.21.15.46:0/3048424881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae3a70 0x560b32ae3e50 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f465405c800 tx=0x7f4654007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439c2a09f4a890d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f4652a00640 1 -- 172.21.15.46:0/3048424881 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4654058770 con 0x560b32ae3a70 2024-03-29T08:23:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f4652a00640 1 -- 172.21.15.46:0/3048424881 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465405cd30 con 0x560b32ae3a70 2024-03-29T08:23:51.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f4652a00640 1 -- 172.21.15.46:0/3048424881 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4654051100 con 0x560b32ae3a70 2024-03-29T08:23:51.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f466c0cb000 1 -- 172.21.15.46:0/3048424881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae3a70 msgr2=0x560b32ae3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f466c0cb000 1 --2- 172.21.15.46:0/3048424881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae3a70 0x560b32ae3e50 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f465405c800 tx=0x7f4654007680 comp rx=0 tx=0).stop 2024-03-29T08:23:51.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f466c0cb000 1 -- 172.21.15.46:0/3048424881 shutdown_connections 2024-03-29T08:23:51.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f466c0cb000 1 --2- 172.21.15.46:0/3048424881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32ae4390 0x560b32a974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f466c0cb000 1 --2- 172.21.15.46:0/3048424881 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32a97a00 0x560b32a97de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f466c0cb000 1 --2- 172.21.15.46:0/3048424881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae3a70 0x560b32ae3e50 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f466c0cb000 1 -- 172.21.15.46:0/3048424881 >> 172.21.15.46:0/3048424881 conn(0x560b32898160 msgr2=0x560b32ac6820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:51.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f466c0cb000 1 -- 172.21.15.46:0/3048424881 shutdown_connections 2024-03-29T08:23:51.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f466c0cb000 1 -- 172.21.15.46:0/3048424881 wait complete. 2024-03-29T08:23:51.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f466c0cb000 1 -- start start 2024-03-29T08:23:51.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.373+0000 7f466c0cb000 1 Processor -- start 2024-03-29T08:23:51.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32a97a00 0x560b32aa0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32ae3a70 0x560b32ac7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae4390 0x560b32ac7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b328e8a90 con 0x560b32ae4390 2024-03-29T08:23:51.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b32887d40 con 0x560b32a97a00 2024-03-29T08:23:51.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b328a5f30 con 0x560b32ae3a70 2024-03-29T08:23:51.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 10 client.0 _ll_register_callbacks cb 0x560b3294a810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:23:51.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4662800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae4390 0x560b32ac7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4662800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae4390 0x560b32ac7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52178/0 (socket says 172.21.15.46:52178) 2024-03-29T08:23:51.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4662800640 1 -- 172.21.15.46:0/752732209 learned_addr learned my addr 172.21.15.46:0/752732209 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:51.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4661e00640 1 --2- 172.21.15.46:0/752732209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32a97a00 0x560b32aa0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f465be00640 1 --2- 172.21.15.46:0/752732209 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32ae3a70 0x560b32ac7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4662800640 1 -- 172.21.15.46:0/752732209 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32ae3a70 msgr2=0x560b32ac7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4662800640 1 --2- 172.21.15.46:0/752732209 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32ae3a70 0x560b32ac7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4662800640 1 -- 172.21.15.46:0/752732209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32a97a00 msgr2=0x560b32aa0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4662800640 1 --2- 172.21.15.46:0/752732209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32a97a00 0x560b32aa0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4662800640 1 -- 172.21.15.46:0/752732209 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46640ca190 con 0x560b32ae4390 2024-03-29T08:23:51.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f465be00640 1 --2- 172.21.15.46:0/752732209 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32ae3a70 0x560b32ac7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:51.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4662800640 1 --2- 172.21.15.46:0/752732209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae4390 0x560b32ac7f90 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f465c0af040 tx=0x7f465c01b170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1a6e9c12413d4d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:51.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4647400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:23:51.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4647400640 1 -- 172.21.15.46:0/752732209 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0c1450 con 0x560b32ae4390 2024-03-29T08:23:51.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 -- 172.21.15.46:0/752732209 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b328e73e0 con 0x560b32ae4390 2024-03-29T08:23:51.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4647400640 1 -- 172.21.15.46:0/752732209 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f465c0c15b0 con 0x560b32ae4390 2024-03-29T08:23:51.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4647400640 1 -- 172.21.15.46:0/752732209 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f465c0c1770 con 0x560b32ae4390 2024-03-29T08:23:51.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 10 client.7146 fetch_fsmap learned FSMap version 48 2024-03-29T08:23:51.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 -- 172.21.15.46:0/752732209 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x560b328adaa0 con 0x560b32ae4390 2024-03-29T08:23:51.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4647400640 1 -- 172.21.15.46:0/752732209 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b328e73e0 con 0x560b32ae4390 2024-03-29T08:23:51.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4647400640 1 -- 172.21.15.46:0/752732209 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 48) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f465c0c18d0 con 0x560b32ae4390 2024-03-29T08:23:51.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 10 client.7146 fetch_fsmap finished waiting for FSMap version 48 2024-03-29T08:23:51.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 10 client.7146 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-29T08:23:51.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 20 client.7146 populate_metadata read hostname 'smithi046' 2024-03-29T08:23:51.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 --2- 172.21.15.46:0/752732209 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x560b32aeb2d0 0x560b32aeb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 4 client.7146 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:23:51.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 -- 172.21.15.46:0/752732209 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b328b8d00 con 0x560b32aeb2d0 2024-03-29T08:23:51.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4661e00640 1 --2- 172.21.15.46:0/752732209 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x560b32aeb2d0 0x560b32aeb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4661e00640 1 --2- 172.21.15.46:0/752732209 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x560b32aeb2d0 0x560b32aeb6b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a2eacc3f40fa0f server_cookie=1fc4215b0bc3d930 in_seq=0 out_seq=0 2024-03-29T08:23:51.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4647400640 10 client.7146 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:51.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4647400640 1 -- 172.21.15.46:0/752732209 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560b328b8d00 con 0x560b32aeb2d0 2024-03-29T08:23:51.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4647400640 10 client.7146 handle_command_reply: tid=0 2024-03-29T08:23:51.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 client.7146 shutdown 2024-03-29T08:23:51.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 -- 172.21.15.46:0/752732209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae4390 msgr2=0x560b32ac7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 --2- 172.21.15.46:0/752732209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae4390 0x560b32ac7f90 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f465c0af040 tx=0x7f465c01b170 comp rx=0 tx=0).stop 2024-03-29T08:23:51.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 -- 172.21.15.46:0/752732209 shutdown_connections 2024-03-29T08:23:51.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 --2- 172.21.15.46:0/752732209 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x560b32aeb2d0 0x560b32aeb6b0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 --2- 172.21.15.46:0/752732209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae4390 0x560b32ac7f90 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 --2- 172.21.15.46:0/752732209 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32ae3a70 0x560b32ac7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 --2- 172.21.15.46:0/752732209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32a97a00 0x560b32aa0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f4647400640 0 client.7146 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:51.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 -- 172.21.15.46:0/752732209 >> 172.21.15.46:0/752732209 conn(0x560b32898160 msgr2=0x560b32a96bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:51.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 -- 172.21.15.46:0/752732209 shutdown_connections 2024-03-29T08:23:51.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.377+0000 7f466c0cb000 1 -- 172.21.15.46:0/752732209 wait complete. 2024-03-29T08:23:51.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.381+0000 7f466c0cb000 20 client.7146 trim_cache size 0 max 16384 2024-03-29T08:23:51.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.393+0000 7f466c0cb000 1 Processor -- start 2024-03-29T08:23:51.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.393+0000 7f466c0cb000 1 -- start start 2024-03-29T08:23:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.393+0000 7f466c0cb000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32a97a00 0x560b32a9f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.393+0000 7f466c0cb000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32ae3a70 0x560b32a9fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.393+0000 7f466c0cb000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32ae4390 0x560b32a97de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.393+0000 7f466c0cb000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b325a5aa0 con 0x560b32a97a00 2024-03-29T08:23:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.393+0000 7f466c0cb000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b328f0e60 con 0x560b32ae4390 2024-03-29T08:23:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.393+0000 7f466c0cb000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b328e62a0 con 0x560b32ae3a70 2024-03-29T08:23:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.393+0000 7f4661e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32a97a00 0x560b32a9f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.393+0000 7f4661e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32a97a00 0x560b32a9f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52188/0 (socket says 172.21.15.46:52188) 2024-03-29T08:23:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.393+0000 7f4661e00640 1 -- 172.21.15.46:0/3810478148 learned_addr learned my addr 172.21.15.46:0/3810478148 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f465be00640 1 --2- 172.21.15.46:0/3810478148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32ae3a70 0x560b32a9fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f4662800640 1 --2- 172.21.15.46:0/3810478148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32ae4390 0x560b32a97de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f4661e00640 1 -- 172.21.15.46:0/3810478148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32ae3a70 msgr2=0x560b32a9fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f4661e00640 1 --2- 172.21.15.46:0/3810478148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32ae3a70 0x560b32a9fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f4661e00640 1 -- 172.21.15.46:0/3810478148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32ae4390 msgr2=0x560b32a97de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f4661e00640 1 --2- 172.21.15.46:0/3810478148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32ae4390 0x560b32a97de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f4661e00640 1 -- 172.21.15.46:0/3810478148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b32893df0 con 0x560b32a97a00 2024-03-29T08:23:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f465be00640 1 --2- 172.21.15.46:0/3810478148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32ae3a70 0x560b32a9fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:23:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f4661e00640 1 --2- 172.21.15.46:0/3810478148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32a97a00 0x560b32a9f6f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4654050ef0 tx=0x7f4654058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae682285b3c70b2d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f4653400640 1 -- 172.21.15.46:0/3810478148 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4654071040 con 0x560b32a97a00 2024-03-29T08:23:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f4653400640 1 -- 172.21.15.46:0/3810478148 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4654052d90 con 0x560b32a97a00 2024-03-29T08:23:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f4653400640 1 -- 172.21.15.46:0/3810478148 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4654051100 con 0x560b32a97a00 2024-03-29T08:23:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 1 -- 172.21.15.46:0/3810478148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32a97a00 msgr2=0x560b32a9f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 1 --2- 172.21.15.46:0/3810478148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32a97a00 0x560b32a9f6f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4654050ef0 tx=0x7f4654058000 comp rx=0 tx=0).stop 2024-03-29T08:23:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 1 -- 172.21.15.46:0/3810478148 shutdown_connections 2024-03-29T08:23:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 1 --2- 172.21.15.46:0/3810478148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32ae4390 0x560b32a97de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 1 --2- 172.21.15.46:0/3810478148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32ae3a70 0x560b32a9fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 1 --2- 172.21.15.46:0/3810478148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32a97a00 0x560b32a9f6f0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 1 -- 172.21.15.46:0/3810478148 >> 172.21.15.46:0/3810478148 conn(0x560b32898160 msgr2=0x560b32a9efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:51.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 1 -- 172.21.15.46:0/3810478148 shutdown_connections 2024-03-29T08:23:51.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 1 -- 172.21.15.46:0/3810478148 wait complete. 2024-03-29T08:23:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 1 -- start start 2024-03-29T08:23:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 1 Processor -- start 2024-03-29T08:23:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32a97a00 0x560b32a9ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae3a70 0x560b32b3f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32ae4390 0x560b32b3fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b32a91930 con 0x560b32ae3a70 2024-03-29T08:23:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b328b4390 con 0x560b32a97a00 2024-03-29T08:23:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b329b9470 con 0x560b32ae4390 2024-03-29T08:23:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f466c0cb000 10 client.0 _ll_register_callbacks cb 0x7f4654050d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:23:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f465be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae3a70 0x560b32b3f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f465be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae3a70 0x560b32b3f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52190/0 (socket says 172.21.15.46:52190) 2024-03-29T08:23:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f465be00640 1 -- 172.21.15.46:0/1862428786 learned_addr learned my addr 172.21.15.46:0/1862428786 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:23:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f4661e00640 1 --2- 172.21.15.46:0/1862428786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32a97a00 0x560b32a9ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f4662800640 1 --2- 172.21.15.46:0/1862428786 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32ae4390 0x560b32b3fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f465be00640 1 -- 172.21.15.46:0/1862428786 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32ae4390 msgr2=0x560b32b3fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f465be00640 1 --2- 172.21.15.46:0/1862428786 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32ae4390 0x560b32b3fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f465be00640 1 -- 172.21.15.46:0/1862428786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32a97a00 msgr2=0x560b32a9ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:23:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f465be00640 1 --2- 172.21.15.46:0/1862428786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32a97a00 0x560b32a9ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.397+0000 7f465be00640 1 -- 172.21.15.46:0/1862428786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b325e4e20 con 0x560b32ae3a70 2024-03-29T08:23:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f465be00640 1 --2- 172.21.15.46:0/1862428786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae3a70 0x560b32b3f8d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f464c0036d0 tx=0x7f464c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30520bf5f42899 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:23:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4647400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:23:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4647400640 1 -- 172.21.15.46:0/1862428786 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f464c01c040 con 0x560b32ae3a70 2024-03-29T08:23:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4647400640 1 -- 172.21.15.46:0/1862428786 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f464c003e00 con 0x560b32ae3a70 2024-03-29T08:23:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 1 -- 172.21.15.46:0/1862428786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b328a5970 con 0x560b32ae3a70 2024-03-29T08:23:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4647400640 1 -- 172.21.15.46:0/1862428786 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f464c014650 con 0x560b32ae3a70 2024-03-29T08:23:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 10 client.7152 fetch_fsmap learned FSMap version 48 2024-03-29T08:23:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 1 -- 172.21.15.46:0/1862428786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x560b32887960 con 0x560b32ae3a70 2024-03-29T08:23:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4647400640 1 -- 172.21.15.46:0/1862428786 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560b328a5970 con 0x560b32ae3a70 2024-03-29T08:23:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4647400640 1 -- 172.21.15.46:0/1862428786 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 48) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f464c02a020 con 0x560b32ae3a70 2024-03-29T08:23:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 10 client.7152 fetch_fsmap finished waiting for FSMap version 48 2024-03-29T08:23:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 10 client.7152 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-29T08:23:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 20 client.7152 populate_metadata read hostname 'smithi046' 2024-03-29T08:23:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 1 --2- 172.21.15.46:0/1862428786 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x560b32b40350 0x560b32b40760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:23:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 4 client.7152 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:23:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 1 -- 172.21.15.46:0/1862428786 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560b32917a20 con 0x560b32b40350 2024-03-29T08:23:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4661e00640 1 --2- 172.21.15.46:0/1862428786 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x560b32b40350 0x560b32b40760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:23:51.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4661e00640 1 --2- 172.21.15.46:0/1862428786 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x560b32b40350 0x560b32b40760 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e2d30f4791df6d7 server_cookie=b8fa3d0b0595111c in_seq=0 out_seq=0 2024-03-29T08:23:51.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4647400640 10 client.7152 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:51.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4647400640 1 -- 172.21.15.46:0/1862428786 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x560b32917a20 con 0x560b32b40350 2024-03-29T08:23:51.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4647400640 10 client.7152 handle_command_reply: tid=0 2024-03-29T08:23:51.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 1 client.7152 shutdown 2024-03-29T08:23:51.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 1 -- 172.21.15.46:0/1862428786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae3a70 msgr2=0x560b32b3f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 1 --2- 172.21.15.46:0/1862428786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae3a70 0x560b32b3f8d0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f464c0036d0 tx=0x7f464c0046d0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 1 -- 172.21.15.46:0/1862428786 shutdown_connections 2024-03-29T08:23:51.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 1 --2- 172.21.15.46:0/1862428786 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x560b32b40350 0x560b32b40760 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 1 --2- 172.21.15.46:0/1862428786 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560b32ae4390 0x560b32b3fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 1 --2- 172.21.15.46:0/1862428786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560b32ae3a70 0x560b32b3f8d0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 1 --2- 172.21.15.46:0/1862428786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b32a97a00 0x560b32a9ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4647400640 0 client.7152 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:23:51.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 1 -- 172.21.15.46:0/1862428786 >> 172.21.15.46:0/1862428786 conn(0x560b32898160 msgr2=0x560b32a9f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:51.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 1 -- 172.21.15.46:0/1862428786 shutdown_connections 2024-03-29T08:23:51.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 1 -- 172.21.15.46:0/1862428786 wait complete. 2024-03-29T08:23:51.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f466c0cb000 20 client.7152 trim_cache size 0 max 16384 2024-03-29T08:23:51.412 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:23:51.412 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-29T08:23:51.412 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-29T08:23:51.413 INFO:teuthology.orchestra.run.smithi046.stdout: "591a7d29-d60b-47ec-a847-06b8fc3962fa": { 2024-03-29T08:23:51.413 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/subdir", 2024-03-29T08:23:51.413 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "591a7d29-d60b-47ec-a847-06b8fc3962fa", 2024-03-29T08:23:51.413 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,repair" 2024-03-29T08:23:51.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:23:51.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:23:51.413 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:23:51.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4663200640 1 -- 172.21.15.46:0/1619331999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4640066690 msgr2=0x7f4640068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4663200640 1 --2- 172.21.15.46:0/1619331999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4640066690 0x7f4640068b50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4664067790 tx=0x7f464c009190 comp rx=0 tx=0).stop 2024-03-29T08:23:51.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4663200640 1 -- 172.21.15.46:0/1619331999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46640ccdd0 msgr2=0x7f46641535a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:23:51.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4663200640 1 --2- 172.21.15.46:0/1619331999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46640ccdd0 0x7f46641535a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f465400c500 tx=0x7f465400c9c0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4663200640 1 -- 172.21.15.46:0/1619331999 shutdown_connections 2024-03-29T08:23:51.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4663200640 1 --2- 172.21.15.46:0/1619331999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4640066690 0x7f4640068b50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4663200640 1 --2- 172.21.15.46:0/1619331999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f466415c1c0 0x7f46640c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4663200640 1 --2- 172.21.15.46:0/1619331999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f46640d1070 0x7f4664156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4663200640 1 --2- 172.21.15.46:0/1619331999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46640ccdd0 0x7f46641535a0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:23:51.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4663200640 1 -- 172.21.15.46:0/1619331999 >> 172.21.15.46:0/1619331999 conn(0x7f46640d0550 msgr2=0x7f46640cd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:23:51.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4663200640 1 -- 172.21.15.46:0/1619331999 shutdown_connections 2024-03-29T08:23:51.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:23:51.401+0000 7f4663200640 1 -- 172.21.15.46:0/1619331999 wait complete. 2024-03-29T08:23:51.425 INFO:tasks.cephfs.filesystem:scrub status for tag:591a7d29-d60b-47ec-a847-06b8fc3962fa - {'path': '/subdir', 'tag': '591a7d29-d60b-47ec-a847-06b8fc3962fa', 'options': 'recursive,repair'} 2024-03-29T08:24:08.945 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:24:08.949 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:24:21.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:21.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-03-29T08:24:21.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.581+0000 7fb875a00640 1 Processor -- start 2024-03-29T08:24:21.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.581+0000 7fb875a00640 1 -- start start 2024-03-29T08:24:21.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.581+0000 7fb875a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb87015aea0 0x7fb8700cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.581+0000 7fb875a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb87006a890 0x7fb8700cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.581+0000 7fb875a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8700cb210 con 0x7fb8700b04f0 2024-03-29T08:24:21.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.581+0000 7fb875a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb87015bee0 con 0x7fb87015aea0 2024-03-29T08:24:21.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.581+0000 7fb875a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8701553c0 con 0x7fb87006a890 2024-03-29T08:24:21.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.585+0000 7fb866000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb87015aea0 0x7fb8700cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.585+0000 7fb866000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb87015aea0 0x7fb8700cff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58756/0 (socket says 172.21.15.46:58756) 2024-03-29T08:24:21.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.585+0000 7fb866000640 1 -- 172.21.15.46:0/2021448046 learned_addr learned my addr 172.21.15.46:0/2021448046 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:21.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.585+0000 7fb86f400640 1 --2- 172.21.15.46:0/2021448046 >> v2:172.21.15.73:3300/0 conn(0x7fb87006a890 0x7fb8700cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.585+0000 7fb866000640 1 -- 172.21.15.46:0/2021448046 >> v2:172.21.15.73:3300/0 conn(0x7fb87006a890 msgr2=0x7fb8700cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:21.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.585+0000 7fb866000640 1 --2- 172.21.15.46:0/2021448046 >> v2:172.21.15.73:3300/0 conn(0x7fb87006a890 0x7fb8700cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.585+0000 7fb866000640 1 -- 172.21.15.46:0/2021448046 >> v1:172.21.15.73:6789/0 conn(0x7fb8700b04f0 legacy=0x7fb8700b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:21.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.585+0000 7fb866000640 1 -- 172.21.15.46:0/2021448046 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87015b360 con 0x7fb87015aea0 2024-03-29T08:24:21.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.585+0000 7fb866000640 1 --2- 172.21.15.46:0/2021448046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb87015aea0 0x7fb8700cff40 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb8700cd060 tx=0x7fb85801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e45dddba0c77c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:21.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.585+0000 7fb86e000640 1 -- 172.21.15.46:0/2021448046 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8580ab600 con 0x7fb87015aea0 2024-03-29T08:24:21.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.585+0000 7fb86e000640 1 -- 172.21.15.46:0/2021448046 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8580a7050 con 0x7fb87015aea0 2024-03-29T08:24:21.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.585+0000 7fb86e000640 1 -- 172.21.15.46:0/2021448046 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8580ab980 con 0x7fb87015aea0 2024-03-29T08:24:21.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb875a00640 1 -- 172.21.15.46:0/2021448046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb87015aea0 msgr2=0x7fb8700cff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb875a00640 1 --2- 172.21.15.46:0/2021448046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb87015aea0 0x7fb8700cff40 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb8700cd060 tx=0x7fb85801c450 comp rx=0 tx=0).stop 2024-03-29T08:24:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb875a00640 1 -- 172.21.15.46:0/2021448046 shutdown_connections 2024-03-29T08:24:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb875a00640 1 --2- 172.21.15.46:0/2021448046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb87015aea0 0x7fb8700cff40 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb875a00640 1 --2- 172.21.15.46:0/2021448046 >> v2:172.21.15.73:3300/0 conn(0x7fb87006a890 0x7fb8700cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb875a00640 1 -- 172.21.15.46:0/2021448046 >> 172.21.15.46:0/2021448046 conn(0x7fb870098a40 msgr2=0x7fb8700cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb875a00640 1 -- 172.21.15.46:0/2021448046 shutdown_connections 2024-03-29T08:24:21.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb875a00640 1 -- 172.21.15.46:0/2021448046 wait complete. 2024-03-29T08:24:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb875a00640 1 Processor -- start 2024-03-29T08:24:21.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb875a00640 1 -- start start 2024-03-29T08:24:21.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb875a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb87006a890 0x7fb8700ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb875a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8700b04f0 0x7fb87014d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb875a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb87015aea0 0x7fb87014ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb875a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8700cdf10 con 0x7fb87006a890 2024-03-29T08:24:21.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb875a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8700ce720 con 0x7fb8700b04f0 2024-03-29T08:24:21.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb875a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8700ce160 con 0x7fb87015aea0 2024-03-29T08:24:21.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb86ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb87006a890 0x7fb8700ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb86ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb87006a890 0x7fb8700ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58770/0 (socket says 172.21.15.46:58770) 2024-03-29T08:24:21.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb86ea00640 1 -- 172.21.15.46:0/3408356563 learned_addr learned my addr 172.21.15.46:0/3408356563 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:21.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb866000640 1 --2- 172.21.15.46:0/3408356563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8700b04f0 0x7fb87014d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb86f400640 1 --2- 172.21.15.46:0/3408356563 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb87015aea0 0x7fb87014ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb86ea00640 1 -- 172.21.15.46:0/3408356563 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb87015aea0 msgr2=0x7fb87014ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:21.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb86ea00640 1 --2- 172.21.15.46:0/3408356563 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb87015aea0 0x7fb87014ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb86ea00640 1 -- 172.21.15.46:0/3408356563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8700b04f0 msgr2=0x7fb87014d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb86ea00640 1 --2- 172.21.15.46:0/3408356563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8700b04f0 0x7fb87014d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb86ea00640 1 -- 172.21.15.46:0/3408356563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8580094d0 con 0x7fb87006a890 2024-03-29T08:24:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb86f400640 1 --2- 172.21.15.46:0/3408356563 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb87015aea0 0x7fb87014ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.589+0000 7fb86ea00640 1 --2- 172.21.15.46:0/3408356563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb87006a890 0x7fb8700ceeb0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fb86000c7c0 tx=0x7fb86000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.593+0000 7fb867400640 1 -- 172.21.15.46:0/3408356563 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86000aa50 con 0x7fb87006a890 2024-03-29T08:24:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.593+0000 7fb867400640 1 -- 172.21.15.46:0/3408356563 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86000abb0 con 0x7fb87006a890 2024-03-29T08:24:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.593+0000 7fb867400640 1 -- 172.21.15.46:0/3408356563 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860012420 con 0x7fb87006a890 2024-03-29T08:24:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.593+0000 7fb875a00640 1 -- 172.21.15.46:0/3408356563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8700ce440 con 0x7fb87006a890 2024-03-29T08:24:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.593+0000 7fb875a00640 1 -- 172.21.15.46:0/3408356563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87014f770 con 0x7fb87006a890 2024-03-29T08:24:21.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.593+0000 7fb867400640 1 -- 172.21.15.46:0/3408356563 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb860012580 con 0x7fb87006a890 2024-03-29T08:24:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.593+0000 7fb867400640 1 --2- 172.21.15.46:0/3408356563 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb84c066690 0x7fb84c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.593+0000 7fb867400640 1 -- 172.21.15.46:0/3408356563 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb8600844d0 con 0x7fb87006a890 2024-03-29T08:24:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.593+0000 7fb866000640 1 --2- 172.21.15.46:0/3408356563 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb84c066690 0x7fb84c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.593+0000 7fb878d67000 1 Processor -- start 2024-03-29T08:24:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.593+0000 7fb878d67000 1 -- start start 2024-03-29T08:24:21.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb866000640 1 --2- 172.21.15.46:0/3408356563 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb84c066690 0x7fb84c068b50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb8700cf560 tx=0x7fb858008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb878d67000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e60e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb878d67000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e61390 0x5604b8e144c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb878d67000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e14a00 0x5604b8e14de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb878d67000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b8cdf630 con 0x5604b8e60a70 2024-03-29T08:24:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb878d67000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b8c6cbd0 con 0x5604b8e14a00 2024-03-29T08:24:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb878d67000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5604b8c6a1a0 con 0x5604b8e61390 2024-03-29T08:24:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb86ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e60e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb86ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e60e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58784/0 (socket says 172.21.15.46:58784) 2024-03-29T08:24:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb86ea00640 1 -- 172.21.15.46:0/2916863912 learned_addr learned my addr 172.21.15.46:0/2916863912 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb86f400640 1 --2- 172.21.15.46:0/2916863912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e14a00 0x5604b8e14de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb866000640 1 --2- 172.21.15.46:0/2916863912 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e61390 0x5604b8e144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb86ea00640 1 -- 172.21.15.46:0/2916863912 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e61390 msgr2=0x5604b8e144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb86ea00640 1 --2- 172.21.15.46:0/2916863912 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e61390 0x5604b8e144c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb86ea00640 1 -- 172.21.15.46:0/2916863912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e14a00 msgr2=0x5604b8e14de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:21.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb86ea00640 1 --2- 172.21.15.46:0/2916863912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e14a00 0x5604b8e14de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb86ea00640 1 -- 172.21.15.46:0/2916863912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604b8961e20 con 0x5604b8e60a70 2024-03-29T08:24:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb86ea00640 1 --2- 172.21.15.46:0/2916863912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e60e50 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fb86005d320 tx=0x7fb860055c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b94e6164aff96d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb85f400640 1 -- 172.21.15.46:0/2916863912 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86005d850 con 0x5604b8e60a70 2024-03-29T08:24:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb85f400640 1 -- 172.21.15.46:0/2916863912 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb860053800 con 0x5604b8e60a70 2024-03-29T08:24:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb85f400640 1 -- 172.21.15.46:0/2916863912 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb860054950 con 0x5604b8e60a70 2024-03-29T08:24:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb878d67000 1 -- 172.21.15.46:0/2916863912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 msgr2=0x5604b8e60e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb878d67000 1 --2- 172.21.15.46:0/2916863912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e60e50 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fb86005d320 tx=0x7fb860055c20 comp rx=0 tx=0).stop 2024-03-29T08:24:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb878d67000 1 -- 172.21.15.46:0/2916863912 shutdown_connections 2024-03-29T08:24:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb878d67000 1 --2- 172.21.15.46:0/2916863912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e14a00 0x5604b8e14de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb878d67000 1 --2- 172.21.15.46:0/2916863912 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e61390 0x5604b8e144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb878d67000 1 --2- 172.21.15.46:0/2916863912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e60e50 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb878d67000 1 -- 172.21.15.46:0/2916863912 >> 172.21.15.46:0/2916863912 conn(0x5604b8c15160 msgr2=0x5604b8e43820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb878d67000 1 -- 172.21.15.46:0/2916863912 shutdown_connections 2024-03-29T08:24:21.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb878d67000 1 -- 172.21.15.46:0/2916863912 wait complete. 2024-03-29T08:24:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb878d67000 1 -- start start 2024-03-29T08:24:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.597+0000 7fb878d67000 1 Processor -- start 2024-03-29T08:24:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb878d67000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e14a00 0x5604b8e1d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb878d67000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e44a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb878d67000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e61390 0x5604b8e44f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb878d67000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b8c65a90 con 0x5604b8e60a70 2024-03-29T08:24:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb878d67000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b8c04d40 con 0x5604b8e61390 2024-03-29T08:24:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb878d67000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5604b8c22f30 con 0x5604b8e14a00 2024-03-29T08:24:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb878d67000 10 client.0 _ll_register_callbacks cb 0x5604b8cc7810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:24:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb866000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e44a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb866000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e44a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58786/0 (socket says 172.21.15.46:58786) 2024-03-29T08:24:21.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb866000640 1 -- 172.21.15.46:0/1814089706 learned_addr learned my addr 172.21.15.46:0/1814089706 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb86f400640 1 --2- 172.21.15.46:0/1814089706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e61390 0x5604b8e44f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb86ea00640 1 --2- 172.21.15.46:0/1814089706 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e14a00 0x5604b8e1d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb866000640 1 -- 172.21.15.46:0/1814089706 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e14a00 msgr2=0x5604b8e1d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb866000640 1 --2- 172.21.15.46:0/1814089706 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e14a00 0x5604b8e1d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb866000640 1 -- 172.21.15.46:0/1814089706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e61390 msgr2=0x5604b8e44f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb866000640 1 --2- 172.21.15.46:0/1814089706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e61390 0x5604b8e44f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb866000640 1 -- 172.21.15.46:0/1814089706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87014f770 con 0x5604b8e60a70 2024-03-29T08:24:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb866000640 1 --2- 172.21.15.46:0/1814089706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e44a50 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb85801ab00 tx=0x7fb85801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f73b3c52c7ce9e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb853e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:24:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb853e00640 1 -- 172.21.15.46:0/1814089706 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8580acd50 con 0x5604b8e60a70 2024-03-29T08:24:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb853e00640 1 -- 172.21.15.46:0/1814089706 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb8580b4430 con 0x5604b8e60a70 2024-03-29T08:24:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb878d67000 1 -- 172.21.15.46:0/1814089706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604b8c643e0 con 0x5604b8e60a70 2024-03-29T08:24:21.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb853e00640 1 -- 172.21.15.46:0/1814089706 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8580aceb0 con 0x5604b8e60a70 2024-03-29T08:24:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb878d67000 10 client.7164 fetch_fsmap learned FSMap version 49 2024-03-29T08:24:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb878d67000 1 -- 172.21.15.46:0/1814089706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x5604b8c2aaa0 con 0x5604b8e60a70 2024-03-29T08:24:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb853e00640 1 -- 172.21.15.46:0/1814089706 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5604b8c643e0 con 0x5604b8e60a70 2024-03-29T08:24:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb853e00640 1 -- 172.21.15.46:0/1814089706 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb8580b45f0 con 0x5604b8e60a70 2024-03-29T08:24:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb878d67000 10 client.7164 fetch_fsmap finished waiting for FSMap version 49 2024-03-29T08:24:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb878d67000 10 client.7164 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-29T08:24:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb878d67000 20 client.7164 populate_metadata read hostname 'smithi046' 2024-03-29T08:24:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb878d67000 1 --2- 172.21.15.46:0/1814089706 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x5604b8e682d0 0x5604b8e686b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb878d67000 4 client.7164 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:24:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb878d67000 1 -- 172.21.15.46:0/1814089706 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604b8c35d00 con 0x5604b8e682d0 2024-03-29T08:24:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb86ea00640 1 --2- 172.21.15.46:0/1814089706 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x5604b8e682d0 0x5604b8e686b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb86ea00640 1 --2- 172.21.15.46:0/1814089706 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x5604b8e682d0 0x5604b8e686b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e144ec8ce2fbff51 server_cookie=fea1f308fa31b2ed in_seq=0 out_seq=0 2024-03-29T08:24:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb853e00640 10 client.7164 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:24:21.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb853e00640 1 -- 172.21.15.46:0/1814089706 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5604b8c35d00 con 0x5604b8e682d0 2024-03-29T08:24:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb853e00640 10 client.7164 handle_command_reply: tid=0 2024-03-29T08:24:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.601+0000 7fb878d67000 1 client.7164 shutdown 2024-03-29T08:24:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.605+0000 7fb878d67000 1 -- 172.21.15.46:0/1814089706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 msgr2=0x5604b8e44a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.605+0000 7fb878d67000 1 --2- 172.21.15.46:0/1814089706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e44a50 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb85801ab00 tx=0x7fb85801ac90 comp rx=0 tx=0).stop 2024-03-29T08:24:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.605+0000 7fb878d67000 1 -- 172.21.15.46:0/1814089706 shutdown_connections 2024-03-29T08:24:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.605+0000 7fb878d67000 1 --2- 172.21.15.46:0/1814089706 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x5604b8e682d0 0x5604b8e686b0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.605+0000 7fb878d67000 1 --2- 172.21.15.46:0/1814089706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e61390 0x5604b8e44f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.605+0000 7fb878d67000 1 --2- 172.21.15.46:0/1814089706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e44a50 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.605+0000 7fb878d67000 1 --2- 172.21.15.46:0/1814089706 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e14a00 0x5604b8e1d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.605+0000 7fb853e00640 0 client.7164 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:24:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.605+0000 7fb878d67000 1 -- 172.21.15.46:0/1814089706 >> 172.21.15.46:0/1814089706 conn(0x5604b8c15160 msgr2=0x5604b8e13bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.605+0000 7fb878d67000 1 -- 172.21.15.46:0/1814089706 shutdown_connections 2024-03-29T08:24:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.605+0000 7fb878d67000 1 -- 172.21.15.46:0/1814089706 wait complete. 2024-03-29T08:24:21.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.605+0000 7fb878d67000 20 client.7164 trim_cache size 0 max 16384 2024-03-29T08:24:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.617+0000 7fb878d67000 1 Processor -- start 2024-03-29T08:24:21.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.617+0000 7fb878d67000 1 -- start start 2024-03-29T08:24:21.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.617+0000 7fb878d67000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e14a00 0x5604b8e1c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.617+0000 7fb878d67000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e1cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.617+0000 7fb878d67000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e61390 0x5604b8e14de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.617+0000 7fb878d67000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b8922aa0 con 0x5604b8e60a70 2024-03-29T08:24:21.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.617+0000 7fb878d67000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b8c6de60 con 0x5604b8e61390 2024-03-29T08:24:21.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.617+0000 7fb878d67000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5604b8d1eec0 con 0x5604b8e14a00 2024-03-29T08:24:21.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb866000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e1cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb866000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e1cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58800/0 (socket says 172.21.15.46:58800) 2024-03-29T08:24:21.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb866000640 1 -- 172.21.15.46:0/556608811 learned_addr learned my addr 172.21.15.46:0/556608811 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:21.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb86ea00640 1 --2- 172.21.15.46:0/556608811 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e14a00 0x5604b8e1c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb86f400640 1 --2- 172.21.15.46:0/556608811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e61390 0x5604b8e14de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb866000640 1 -- 172.21.15.46:0/556608811 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e14a00 msgr2=0x5604b8e1c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb866000640 1 --2- 172.21.15.46:0/556608811 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e14a00 0x5604b8e1c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb866000640 1 -- 172.21.15.46:0/556608811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e61390 msgr2=0x5604b8e14de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb866000640 1 --2- 172.21.15.46:0/556608811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e61390 0x5604b8e14de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb866000640 1 -- 172.21.15.46:0/556608811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604b8c10df0 con 0x5604b8e60a70 2024-03-29T08:24:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb866000640 1 --2- 172.21.15.46:0/556608811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e1cc00 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fb85801acd0 tx=0x7fb8580b8070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a2921993c4b06b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb85fe00640 1 -- 172.21.15.46:0/556608811 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8580b4630 con 0x5604b8e60a70 2024-03-29T08:24:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb85fe00640 1 -- 172.21.15.46:0/556608811 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8580be040 con 0x5604b8e60a70 2024-03-29T08:24:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb85fe00640 1 -- 172.21.15.46:0/556608811 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8580b6d30 con 0x5604b8e60a70 2024-03-29T08:24:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 1 -- 172.21.15.46:0/556608811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 msgr2=0x5604b8e1cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:21.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 1 --2- 172.21.15.46:0/556608811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e1cc00 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fb85801acd0 tx=0x7fb8580b8070 comp rx=0 tx=0).stop 2024-03-29T08:24:21.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 1 -- 172.21.15.46:0/556608811 shutdown_connections 2024-03-29T08:24:21.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 1 --2- 172.21.15.46:0/556608811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e61390 0x5604b8e14de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 1 --2- 172.21.15.46:0/556608811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e60a70 0x5604b8e1cc00 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 1 --2- 172.21.15.46:0/556608811 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e14a00 0x5604b8e1c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 1 -- 172.21.15.46:0/556608811 >> 172.21.15.46:0/556608811 conn(0x5604b8c15160 msgr2=0x5604b8e1bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:21.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 1 -- 172.21.15.46:0/556608811 shutdown_connections 2024-03-29T08:24:21.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 1 -- 172.21.15.46:0/556608811 wait complete. 2024-03-29T08:24:21.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 1 -- start start 2024-03-29T08:24:21.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 1 Processor -- start 2024-03-29T08:24:21.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e14a00 0x5604b8ec0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e60a70 0x5604b8ec0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e61390 0x5604b8e17600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b8e44c80 con 0x5604b8e61390 2024-03-29T08:24:21.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604b8e45220 con 0x5604b8e60a70 2024-03-29T08:24:21.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5604b8e1a2e0 con 0x5604b8e14a00 2024-03-29T08:24:21.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb878d67000 10 client.0 _ll_register_callbacks cb 0x7fb8600517a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:24:21.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb86f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e61390 0x5604b8e17600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb86ea00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e14a00 0x5604b8ec0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb866000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e60a70 0x5604b8ec0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb86f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e61390 0x5604b8e17600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58812/0 (socket says 172.21.15.46:58812) 2024-03-29T08:24:21.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb86ea00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e14a00 0x5604b8ec0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:59052/0 (socket says 172.21.15.46:59052) 2024-03-29T08:24:21.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.621+0000 7fb86f400640 1 -- 172.21.15.46:0/3517942274 learned_addr learned my addr 172.21.15.46:0/3517942274 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:21.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb86f400640 1 -- 172.21.15.46:0/3517942274 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e14a00 msgr2=0x5604b8ec0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:21.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb86f400640 1 --2- 172.21.15.46:0/3517942274 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e14a00 0x5604b8ec0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb86f400640 1 -- 172.21.15.46:0/3517942274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e60a70 msgr2=0x5604b8ec0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:21.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb86f400640 1 --2- 172.21.15.46:0/3517942274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e60a70 0x5604b8ec0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb86f400640 1 -- 172.21.15.46:0/3517942274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604b8c2aaa0 con 0x5604b8e61390 2024-03-29T08:24:21.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb86f400640 1 --2- 172.21.15.46:0/3517942274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e61390 0x5604b8e17600 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb8680176c0 tx=0x7fb868017b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e4fc3be9445c07 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:21.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb853e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:24:21.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb853e00640 1 -- 172.21.15.46:0/3517942274 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86800b720 con 0x5604b8e61390 2024-03-29T08:24:21.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb853e00640 1 -- 172.21.15.46:0/3517942274 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb868021b80 con 0x5604b8e61390 2024-03-29T08:24:21.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 1 -- 172.21.15.46:0/3517942274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604b8e45360 con 0x5604b8e61390 2024-03-29T08:24:21.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb853e00640 1 -- 172.21.15.46:0/3517942274 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb868021d60 con 0x5604b8e61390 2024-03-29T08:24:21.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 10 client.7170 fetch_fsmap learned FSMap version 49 2024-03-29T08:24:21.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 1 -- 172.21.15.46:0/3517942274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x5604b8c04960 con 0x5604b8e61390 2024-03-29T08:24:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb853e00640 1 -- 172.21.15.46:0/3517942274 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5604b8e45360 con 0x5604b8e61390 2024-03-29T08:24:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb853e00640 1 -- 172.21.15.46:0/3517942274 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb868015b30 con 0x5604b8e61390 2024-03-29T08:24:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 10 client.7170 fetch_fsmap finished waiting for FSMap version 49 2024-03-29T08:24:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 10 client.7170 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-03-29T08:24:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 20 client.7170 populate_metadata read hostname 'smithi046' 2024-03-29T08:24:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 1 --2- 172.21.15.46:0/3517942274 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x5604b8e17b40 0x5604b8e17f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 4 client.7170 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:24:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 1 -- 172.21.15.46:0/3517942274 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5604b8c22970 con 0x5604b8e17b40 2024-03-29T08:24:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb86ea00640 1 --2- 172.21.15.46:0/3517942274 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x5604b8e17b40 0x5604b8e17f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb853e00640 10 client.7170 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:24:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb86ea00640 1 --2- 172.21.15.46:0/3517942274 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x5604b8e17b40 0x5604b8e17f50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0150328e6b44ac8 server_cookie=96591ef5109cb795 in_seq=0 out_seq=0 2024-03-29T08:24:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb853e00640 1 -- 172.21.15.46:0/3517942274 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5604b8c22970 con 0x5604b8e17b40 2024-03-29T08:24:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb853e00640 10 client.7170 handle_command_reply: tid=0 2024-03-29T08:24:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 1 client.7170 shutdown 2024-03-29T08:24:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 1 -- 172.21.15.46:0/3517942274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e61390 msgr2=0x5604b8e17600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 1 --2- 172.21.15.46:0/3517942274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e61390 0x5604b8e17600 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fb8680176c0 tx=0x7fb868017b80 comp rx=0 tx=0).stop 2024-03-29T08:24:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 1 -- 172.21.15.46:0/3517942274 shutdown_connections 2024-03-29T08:24:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 1 --2- 172.21.15.46:0/3517942274 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x5604b8e17b40 0x5604b8e17f50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 1 --2- 172.21.15.46:0/3517942274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5604b8e60a70 0x5604b8ec0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 1 --2- 172.21.15.46:0/3517942274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604b8e61390 0x5604b8e17600 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 1 --2- 172.21.15.46:0/3517942274 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5604b8e14a00 0x5604b8ec0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb853e00640 0 client.7170 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:24:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 1 -- 172.21.15.46:0/3517942274 >> 172.21.15.46:0/3517942274 conn(0x5604b8c15160 msgr2=0x5604b8e1c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 1 -- 172.21.15.46:0/3517942274 shutdown_connections 2024-03-29T08:24:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 1 -- 172.21.15.46:0/3517942274 wait complete. 2024-03-29T08:24:21.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb878d67000 20 client.7170 trim_cache size 0 max 16384 2024-03-29T08:24:21.631 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:24:21.631 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-29T08:24:21.631 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-29T08:24:21.631 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:24:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb875a00640 1 -- 172.21.15.46:0/3408356563 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb84c066690 msgr2=0x7fb84c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb875a00640 1 --2- 172.21.15.46:0/3408356563 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb84c066690 0x7fb84c068b50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb8700cf560 tx=0x7fb858008040 comp rx=0 tx=0).stop 2024-03-29T08:24:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb875a00640 1 -- 172.21.15.46:0/3408356563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb87006a890 msgr2=0x7fb8700ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb875a00640 1 --2- 172.21.15.46:0/3408356563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb87006a890 0x7fb8700ceeb0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fb86000c7c0 tx=0x7fb86000cc80 comp rx=0 tx=0).stop 2024-03-29T08:24:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb875a00640 1 -- 172.21.15.46:0/3408356563 shutdown_connections 2024-03-29T08:24:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb875a00640 1 --2- 172.21.15.46:0/3408356563 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb84c066690 0x7fb84c068b50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb875a00640 1 --2- 172.21.15.46:0/3408356563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8700b04f0 0x7fb87014d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb875a00640 1 --2- 172.21.15.46:0/3408356563 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb87015aea0 0x7fb87014ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb875a00640 1 --2- 172.21.15.46:0/3408356563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb87006a890 0x7fb8700ceeb0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb875a00640 1 -- 172.21.15.46:0/3408356563 >> 172.21.15.46:0/3408356563 conn(0x7fb870098a40 msgr2=0x7fb870151740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb875a00640 1 -- 172.21.15.46:0/3408356563 shutdown_connections 2024-03-29T08:24:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.625+0000 7fb875a00640 1 -- 172.21.15.46:0/3408356563 wait complete. 2024-03-29T08:24:21.646 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-29T08:24:21.646 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:24:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.785+0000 7fdfe9e00640 1 Processor -- start 2024-03-29T08:24:21.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe9e00640 1 -- start start 2024-03-29T08:24:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdfe40d0660 0x7fdfe40d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe9e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdfe406b8b0 0x7fdfe40cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe9e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdfe415dc00 0x7fdfe40cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe9e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe406c100 con 0x7fdfe40d0660 2024-03-29T08:24:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe9e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe415cfb0 con 0x7fdfe415dc00 2024-03-29T08:24:21.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe9e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe40d0f80 con 0x7fdfe406b8b0 2024-03-29T08:24:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdfe40d0660 0x7fdfe40d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdfe40d0660 0x7fdfe40d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58826/0 (socket says 172.21.15.46:58826) 2024-03-29T08:24:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe2a00640 1 -- 172.21.15.46:0/2407432998 learned_addr learned my addr 172.21.15.46:0/2407432998 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe2000640 1 --2- 172.21.15.46:0/2407432998 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdfe406b8b0 0x7fdfe40cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe3400640 1 --2- 172.21.15.46:0/2407432998 >> v2:172.21.15.73:3300/0 conn(0x7fdfe415dc00 0x7fdfe40cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe2a00640 1 -- 172.21.15.46:0/2407432998 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdfe406b8b0 msgr2=0x7fdfe40cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe2a00640 1 --2- 172.21.15.46:0/2407432998 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdfe406b8b0 0x7fdfe40cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe2a00640 1 -- 172.21.15.46:0/2407432998 >> v2:172.21.15.73:3300/0 conn(0x7fdfe415dc00 msgr2=0x7fdfe40cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe2a00640 1 --2- 172.21.15.46:0/2407432998 >> v2:172.21.15.73:3300/0 conn(0x7fdfe415dc00 0x7fdfe40cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe2a00640 1 -- 172.21.15.46:0/2407432998 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe406be20 con 0x7fdfe40d0660 2024-03-29T08:24:21.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.789+0000 7fdfe2000640 1 --2- 172.21.15.46:0/2407432998 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdfe406b8b0 0x7fdfe40cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:21.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.793+0000 7fdfe2a00640 1 --2- 172.21.15.46:0/2407432998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfe40d0660 0x7fdfe40d0a40 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fdfe409e600 tx=0x7fdfd401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=470782bd6014547 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.793+0000 7fdfe1600640 1 -- 172.21.15.46:0/2407432998 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd40ab680 con 0x7fdfe40d0660 2024-03-29T08:24:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.793+0000 7fdfe1600640 1 -- 172.21.15.46:0/2407432998 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd40ab7e0 con 0x7fdfe40d0660 2024-03-29T08:24:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.793+0000 7fdfe1600640 1 -- 172.21.15.46:0/2407432998 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd40abb70 con 0x7fdfe40d0660 2024-03-29T08:24:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.793+0000 7fdfe9e00640 1 -- 172.21.15.46:0/2407432998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfe40d0660 msgr2=0x7fdfe40d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.793+0000 7fdfe9e00640 1 --2- 172.21.15.46:0/2407432998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfe40d0660 0x7fdfe40d0a40 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fdfe409e600 tx=0x7fdfd401c4f0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.793+0000 7fdfe9e00640 1 -- 172.21.15.46:0/2407432998 shutdown_connections 2024-03-29T08:24:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.793+0000 7fdfe9e00640 1 --2- 172.21.15.46:0/2407432998 >> v2:172.21.15.73:3300/0 conn(0x7fdfe415dc00 0x7fdfe40cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.793+0000 7fdfe9e00640 1 --2- 172.21.15.46:0/2407432998 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdfe406b8b0 0x7fdfe40cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.793+0000 7fdfe9e00640 1 --2- 172.21.15.46:0/2407432998 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfe40d0660 0x7fdfe40d0a40 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.793+0000 7fdfe9e00640 1 -- 172.21.15.46:0/2407432998 >> 172.21.15.46:0/2407432998 conn(0x7fdfe4089fd0 msgr2=0x7fdfe408a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:21.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.793+0000 7fdfe9e00640 1 -- 172.21.15.46:0/2407432998 shutdown_connections 2024-03-29T08:24:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.793+0000 7fdfe9e00640 1 -- 172.21.15.46:0/2407432998 wait complete. 2024-03-29T08:24:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.793+0000 7fdfe9e00640 1 Processor -- start 2024-03-29T08:24:21.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.793+0000 7fdfe9e00640 1 -- start start 2024-03-29T08:24:21.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe9e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdfe406b8b0 0x7fdfe41505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe9e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfe40d0660 0x7fdfe40c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfe415dc00 0x7fdfe41516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe9e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe41f60a0 con 0x7fdfe415dc00 2024-03-29T08:24:21.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe9e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe40b32c0 con 0x7fdfe40d0660 2024-03-29T08:24:21.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe9e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe40b3880 con 0x7fdfe406b8b0 2024-03-29T08:24:21.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfe415dc00 0x7fdfe41516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfe415dc00 0x7fdfe41516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58838/0 (socket says 172.21.15.46:58838) 2024-03-29T08:24:21.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe3400640 1 -- 172.21.15.46:0/3293857308 learned_addr learned my addr 172.21.15.46:0/3293857308 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:21.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe2a00640 1 --2- 172.21.15.46:0/3293857308 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdfe406b8b0 0x7fdfe41505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe2000640 1 --2- 172.21.15.46:0/3293857308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfe40d0660 0x7fdfe40c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe3400640 1 -- 172.21.15.46:0/3293857308 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdfe406b8b0 msgr2=0x7fdfe41505f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:21.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe3400640 1 --2- 172.21.15.46:0/3293857308 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdfe406b8b0 0x7fdfe41505f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe3400640 1 -- 172.21.15.46:0/3293857308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfe40d0660 msgr2=0x7fdfe40c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:21.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe3400640 1 --2- 172.21.15.46:0/3293857308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfe40d0660 0x7fdfe40c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:21.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe3400640 1 -- 172.21.15.46:0/3293857308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd40094d0 con 0x7fdfe415dc00 2024-03-29T08:24:21.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe2a00640 1 --2- 172.21.15.46:0/3293857308 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdfe406b8b0 0x7fdfe41505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:21.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe3400640 1 --2- 172.21.15.46:0/3293857308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfe415dc00 0x7fdfe41516e0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc007e90 tx=0x7fdfdc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:21.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfdb400640 1 -- 172.21.15.46:0/3293857308 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc014420 con 0x7fdfe415dc00 2024-03-29T08:24:21.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfdb400640 1 -- 172.21.15.46:0/3293857308 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfdc014580 con 0x7fdfe415dc00 2024-03-29T08:24:21.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfdb400640 1 -- 172.21.15.46:0/3293857308 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc014930 con 0x7fdfe415dc00 2024-03-29T08:24:21.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe9e00640 1 -- 172.21.15.46:0/3293857308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe4150b30 con 0x7fdfe415dc00 2024-03-29T08:24:21.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe9e00640 1 -- 172.21.15.46:0/3293857308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe41f65d0 con 0x7fdfe415dc00 2024-03-29T08:24:21.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfe9e00640 1 -- 172.21.15.46:0/3293857308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe4066c90 con 0x7fdfe415dc00 2024-03-29T08:24:21.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.797+0000 7fdfdb400640 1 -- 172.21.15.46:0/3293857308 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdfdc014a90 con 0x7fdfe415dc00 2024-03-29T08:24:21.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.801+0000 7fdfdb400640 1 --2- 172.21.15.46:0/3293857308 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdfc4066690 0x7fdfc4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:21.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.801+0000 7fdfdb400640 1 -- 172.21.15.46:0/3293857308 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fdfdc084aa0 con 0x7fdfe415dc00 2024-03-29T08:24:21.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.801+0000 7fdfe2a00640 1 --2- 172.21.15.46:0/3293857308 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdfc4066690 0x7fdfc4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:21.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.801+0000 7fdfe2a00640 1 --2- 172.21.15.46:0/3293857308 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdfc4066690 0x7fdfc4068b50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40af000 tx=0x7fdfd40a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:21.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:21.805+0000 7fdfdb400640 1 -- 172.21.15.46:0/3293857308 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdfdc051d80 con 0x7fdfe415dc00 2024-03-29T08:24:22.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.005+0000 7fdfe9e00640 1 -- 172.21.15.46:0/3293857308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfe40b39c0 con 0x7fdfe415dc00 2024-03-29T08:24:22.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.005+0000 7fdfdb400640 1 -- 172.21.15.46:0/3293857308 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4738 (secure 0 0 0) 0x7fdfdc0553a0 con 0x7fdfe415dc00 2024-03-29T08:24:22.005 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:22.005 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3456771376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3456771376},{"type":"v1","addr":"172.21.15.73:6836","nonce":3456771376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2671538025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2671538025},{"type":"v1","addr":"172.21.15.73:6837","nonce":2671538025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3196015693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3196015693},{"type":"v1","addr":"172.21.15.46:6833","nonce":3196015693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:22:43.605457+0000","modified":"2024-03-29T08:23:53.005710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":16,"addr":"172.21.15.46:6835/1672348161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1672348161},{"type":"v1","addr":"172.21.15.46:6835","nonce":1672348161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-29T08:24:22.005 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 49 2024-03-29T08:24:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.005+0000 7fdfe9e00640 1 -- 172.21.15.46:0/3293857308 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdfc4066690 msgr2=0x7fdfc4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.005+0000 7fdfe9e00640 1 --2- 172.21.15.46:0/3293857308 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdfc4066690 0x7fdfc4068b50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40af000 tx=0x7fdfd40a7000 comp rx=0 tx=0).stop 2024-03-29T08:24:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.005+0000 7fdfe9e00640 1 -- 172.21.15.46:0/3293857308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfe415dc00 msgr2=0x7fdfe41516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.005+0000 7fdfe9e00640 1 --2- 172.21.15.46:0/3293857308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfe415dc00 0x7fdfe41516e0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc007e90 tx=0x7fdfdc00e4b0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.005+0000 7fdfe9e00640 1 -- 172.21.15.46:0/3293857308 shutdown_connections 2024-03-29T08:24:22.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.005+0000 7fdfe9e00640 1 --2- 172.21.15.46:0/3293857308 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdfc4066690 0x7fdfc4068b50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.005+0000 7fdfe9e00640 1 --2- 172.21.15.46:0/3293857308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfe40d0660 0x7fdfe40c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.005+0000 7fdfe9e00640 1 --2- 172.21.15.46:0/3293857308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdfe415dc00 0x7fdfe41516e0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.005+0000 7fdfe9e00640 1 --2- 172.21.15.46:0/3293857308 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdfe406b8b0 0x7fdfe41505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.005+0000 7fdfe9e00640 1 -- 172.21.15.46:0/3293857308 >> 172.21.15.46:0/3293857308 conn(0x7fdfe4089fd0 msgr2=0x7fdfe406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:22.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.005+0000 7fdfe9e00640 1 -- 172.21.15.46:0/3293857308 shutdown_connections 2024-03-29T08:24:22.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.005+0000 7fdfe9e00640 1 -- 172.21.15.46:0/3293857308 wait complete. 2024-03-29T08:24:22.026 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-03-29T08:24:22.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.169+0000 7f3a4da00640 1 Processor -- start 2024-03-29T08:24:22.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.169+0000 7f3a4da00640 1 -- start start 2024-03-29T08:24:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.169+0000 7f3a4da00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a4815df90 0x7f3a480ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.169+0000 7f3a4da00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3a480cbea0 0x7f3a480ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.169+0000 7f3a4da00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a4815d260 con 0x7f3a48142a50 2024-03-29T08:24:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.169+0000 7f3a4da00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a480aed00 con 0x7f3a4815df90 2024-03-29T08:24:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.169+0000 7f3a4da00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a480aefd0 con 0x7f3a480cbea0 2024-03-29T08:24:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.169+0000 7f3a3e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a4815df90 0x7f3a480ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.169+0000 7f3a3e000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3a4815df90 0x7f3a480ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58840/0 (socket says 172.21.15.46:58840) 2024-03-29T08:24:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.169+0000 7f3a3e000640 1 -- 172.21.15.46:0/218429597 learned_addr learned my addr 172.21.15.46:0/218429597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.169+0000 7f3a46a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3a48142a50 0x7f3a480cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41072/0 (socket says 172.21.15.46:41072) 2024-03-29T08:24:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.169+0000 7f3a47400640 1 --2- 172.21.15.46:0/218429597 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3a480cbea0 0x7f3a480ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.169+0000 7f3a3e000640 1 -- 172.21.15.46:0/218429597 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3a480cbea0 msgr2=0x7f3a480ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.169+0000 7f3a3e000640 1 --2- 172.21.15.46:0/218429597 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3a480cbea0 0x7f3a480ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.169+0000 7f3a3e000640 1 -- 172.21.15.46:0/218429597 >> v1:172.21.15.46:6789/0 conn(0x7f3a48142a50 legacy=0x7f3a480cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.169+0000 7f3a3e000640 1 -- 172.21.15.46:0/218429597 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4815d4b0 con 0x7f3a4815df90 2024-03-29T08:24:22.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.173+0000 7f3a3e000640 1 --2- 172.21.15.46:0/218429597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a4815df90 0x7f3a480ae280 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009870 tx=0x7f3a3001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ecdf7c14d766dd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:22.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.173+0000 7f3a46000640 1 -- 172.21.15.46:0/218429597 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a300ab600 con 0x7f3a4815df90 2024-03-29T08:24:22.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.173+0000 7f3a46000640 1 -- 172.21.15.46:0/218429597 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a300a7050 con 0x7f3a4815df90 2024-03-29T08:24:22.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.173+0000 7f3a46000640 1 -- 172.21.15.46:0/218429597 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a300ab980 con 0x7f3a4815df90 2024-03-29T08:24:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.173+0000 7f3a4da00640 1 -- 172.21.15.46:0/218429597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a4815df90 msgr2=0x7f3a480ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.173+0000 7f3a4da00640 1 --2- 172.21.15.46:0/218429597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a4815df90 0x7f3a480ae280 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009870 tx=0x7f3a3001c4d0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.173+0000 7f3a4da00640 1 -- 172.21.15.46:0/218429597 shutdown_connections 2024-03-29T08:24:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.173+0000 7f3a4da00640 1 --2- 172.21.15.46:0/218429597 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3a480cbea0 0x7f3a480ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.173+0000 7f3a4da00640 1 --2- 172.21.15.46:0/218429597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a4815df90 0x7f3a480ae280 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.173+0000 7f3a4da00640 1 -- 172.21.15.46:0/218429597 >> 172.21.15.46:0/218429597 conn(0x7f3a4806a390 msgr2=0x7f3a480cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.173+0000 7f3a4da00640 1 -- 172.21.15.46:0/218429597 shutdown_connections 2024-03-29T08:24:22.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.173+0000 7f3a4da00640 1 -- 172.21.15.46:0/218429597 wait complete. 2024-03-29T08:24:22.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a4da00640 1 Processor -- start 2024-03-29T08:24:22.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a4da00640 1 -- start start 2024-03-29T08:24:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a4da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a480cbea0 0x7f3a48157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a4da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a48142a50 0x7f3a4813ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a4da00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3a4815df90 0x7f3a4815ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a4da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a48158410 con 0x7f3a48142a50 2024-03-29T08:24:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a4da00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4813f5e0 con 0x7f3a480cbea0 2024-03-29T08:24:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a4da00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4813f8c0 con 0x7f3a4815df90 2024-03-29T08:24:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a3e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a48142a50 0x7f3a4813ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a3e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a48142a50 0x7f3a4813ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58842/0 (socket says 172.21.15.46:58842) 2024-03-29T08:24:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a3e000640 1 -- 172.21.15.46:0/992272828 learned_addr learned my addr 172.21.15.46:0/992272828 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a46a00640 1 --2- 172.21.15.46:0/992272828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a480cbea0 0x7f3a48157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a47400640 1 --2- 172.21.15.46:0/992272828 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3a4815df90 0x7f3a4815ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a3e000640 1 -- 172.21.15.46:0/992272828 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3a4815df90 msgr2=0x7f3a4815ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a3e000640 1 --2- 172.21.15.46:0/992272828 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3a4815df90 0x7f3a4815ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a3e000640 1 -- 172.21.15.46:0/992272828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a480cbea0 msgr2=0x7f3a48157ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a3e000640 1 --2- 172.21.15.46:0/992272828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a480cbea0 0x7f3a48157ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a3e000640 1 -- 172.21.15.46:0/992272828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a300094d0 con 0x7f3a48142a50 2024-03-29T08:24:22.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a3e000640 1 --2- 172.21.15.46:0/992272828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a48142a50 0x7f3a4813ecd0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3a300ae040 tx=0x7f3a30019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a3f400640 1 -- 172.21.15.46:0/992272828 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a300acea0 con 0x7f3a48142a50 2024-03-29T08:24:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a3f400640 1 -- 172.21.15.46:0/992272828 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a300b6640 con 0x7f3a48142a50 2024-03-29T08:24:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a3f400640 1 -- 172.21.15.46:0/992272828 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a300b69f0 con 0x7f3a48142a50 2024-03-29T08:24:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a4da00640 1 -- 172.21.15.46:0/992272828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a4813fba0 con 0x7f3a48142a50 2024-03-29T08:24:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.177+0000 7f3a4da00640 1 -- 172.21.15.46:0/992272828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a481f3640 con 0x7f3a48142a50 2024-03-29T08:24:22.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.181+0000 7f3a3f400640 1 -- 172.21.15.46:0/992272828 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3a300b2040 con 0x7f3a48142a50 2024-03-29T08:24:22.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.181+0000 7f3a3f400640 1 --2- 172.21.15.46:0/992272828 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3a240666c0 0x7f3a24068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.181+0000 7f3a3f400640 1 -- 172.21.15.46:0/992272828 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f3a3011e410 con 0x7f3a48142a50 2024-03-29T08:24:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.181+0000 7f3a46a00640 1 --2- 172.21.15.46:0/992272828 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3a240666c0 0x7f3a24068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.181+0000 7f3a50ecf000 1 Processor -- start 2024-03-29T08:24:22.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.181+0000 7f3a50ecf000 1 -- start start 2024-03-29T08:24:22.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.181+0000 7f3a46a00640 1 --2- 172.21.15.46:0/992272828 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3a240666c0 0x7f3a24068b80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3a380012f0 tx=0x7f3a38001700 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:22.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.181+0000 7f3a50ecf000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09290 0x5593a2a09670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.181+0000 7f3a50ecf000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a2a09bb0 0x5593a29bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.181+0000 7f3a50ecf000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a29c01b0 0x5593a29c0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.181+0000 7f3a50ecf000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593a2888340 con 0x5593a2a09bb0 2024-03-29T08:24:22.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.181+0000 7f3a50ecf000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593a2814cd0 con 0x5593a29c01b0 2024-03-29T08:24:22.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.181+0000 7f3a50ecf000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593a28122a0 con 0x5593a2a09290 2024-03-29T08:24:22.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.181+0000 7f3a3e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a2a09bb0 0x5593a29bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.181+0000 7f3a3e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a2a09bb0 0x5593a29bfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58856/0 (socket says 172.21.15.46:58856) 2024-03-29T08:24:22.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a3e000640 1 -- 172.21.15.46:0/3486163668 learned_addr learned my addr 172.21.15.46:0/3486163668 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:22.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a46a00640 1 --2- 172.21.15.46:0/3486163668 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09290 0x5593a2a09670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a47400640 1 --2- 172.21.15.46:0/3486163668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a29c01b0 0x5593a29c0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a3e000640 1 -- 172.21.15.46:0/3486163668 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09290 msgr2=0x5593a2a09670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:22.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a3e000640 1 --2- 172.21.15.46:0/3486163668 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09290 0x5593a2a09670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a3e000640 1 -- 172.21.15.46:0/3486163668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a29c01b0 msgr2=0x5593a29c0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:22.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a3e000640 1 --2- 172.21.15.46:0/3486163668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a29c01b0 0x5593a29c0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a3e000640 1 -- 172.21.15.46:0/3486163668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593a25004b0 con 0x5593a2a09bb0 2024-03-29T08:24:22.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a3e000640 1 --2- 172.21.15.46:0/3486163668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a2a09bb0 0x5593a29bfc70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3a300ef0c0 tx=0x7f3a300ef2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ebe97a50ab1f7f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:22.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a37400640 1 -- 172.21.15.46:0/3486163668 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a300f7700 con 0x5593a2a09bb0 2024-03-29T08:24:22.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a37400640 1 -- 172.21.15.46:0/3486163668 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a300ed760 con 0x5593a2a09bb0 2024-03-29T08:24:22.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a37400640 1 -- 172.21.15.46:0/3486163668 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a300ebad0 con 0x5593a2a09bb0 2024-03-29T08:24:22.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 1 -- 172.21.15.46:0/3486163668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a2a09bb0 msgr2=0x5593a29bfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/3486163668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a2a09bb0 0x5593a29bfc70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3a300ef0c0 tx=0x7f3a300ef2c0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 1 -- 172.21.15.46:0/3486163668 shutdown_connections 2024-03-29T08:24:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/3486163668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a29c01b0 0x5593a29c0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/3486163668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a2a09bb0 0x5593a29bfc70 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/3486163668 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09290 0x5593a2a09670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 1 -- 172.21.15.46:0/3486163668 >> 172.21.15.46:0/3486163668 conn(0x5593a29bc160 msgr2=0x5593a29be550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 1 -- 172.21.15.46:0/3486163668 shutdown_connections 2024-03-29T08:24:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 1 -- 172.21.15.46:0/3486163668 wait complete. 2024-03-29T08:24:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 1 -- start start 2024-03-29T08:24:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 1 Processor -- start 2024-03-29T08:24:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a29c01b0 0x5593a2a08c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09290 0x5593a29c5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a2a09bb0 0x5593a29c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593a280db90 con 0x5593a29c01b0 2024-03-29T08:24:22.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593a27cb0a0 con 0x5593a2a09bb0 2024-03-29T08:24:22.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593a27cad40 con 0x5593a2a09290 2024-03-29T08:24:22.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a50ecf000 10 client.0 _ll_register_callbacks cb 0x5593a28c2c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:24:22.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a46a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a29c01b0 0x5593a2a08c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a46a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a29c01b0 0x5593a2a08c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58866/0 (socket says 172.21.15.46:58866) 2024-03-29T08:24:22.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a46a00640 1 -- 172.21.15.46:0/1109164526 learned_addr learned my addr 172.21.15.46:0/1109164526 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:22.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a47400640 1 --2- 172.21.15.46:0/1109164526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a2a09bb0 0x5593a29c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a3e000640 1 --2- 172.21.15.46:0/1109164526 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09290 0x5593a29c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a46a00640 1 -- 172.21.15.46:0/1109164526 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09290 msgr2=0x5593a29c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:22.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a46a00640 1 --2- 172.21.15.46:0/1109164526 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09290 0x5593a29c5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a46a00640 1 -- 172.21.15.46:0/1109164526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a2a09bb0 msgr2=0x5593a29c5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:22.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a46a00640 1 --2- 172.21.15.46:0/1109164526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a2a09bb0 0x5593a29c5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a46a00640 1 -- 172.21.15.46:0/1109164526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a481f3640 con 0x5593a29c01b0 2024-03-29T08:24:22.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.185+0000 7f3a47400640 1 --2- 172.21.15.46:0/1109164526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a2a09bb0 0x5593a29c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:22.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a46a00640 1 --2- 172.21.15.46:0/1109164526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a29c01b0 0x5593a2a08c30 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3a38013a00 tx=0x7f3a38001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d508cb7a14adaa46 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:22.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a2be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:24:22.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a2be00640 1 -- 172.21.15.46:0/1109164526 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a38012ae0 con 0x5593a29c01b0 2024-03-29T08:24:22.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a2be00640 1 -- 172.21.15.46:0/1109164526 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f3a3801b070 con 0x5593a29c01b0 2024-03-29T08:24:22.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 1 -- 172.21.15.46:0/1109164526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593a27dd550 con 0x5593a29c01b0 2024-03-29T08:24:22.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a2be00640 1 -- 172.21.15.46:0/1109164526 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a38012c40 con 0x5593a29c01b0 2024-03-29T08:24:22.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 10 client.7191 fetch_fsmap learned FSMap version 49 2024-03-29T08:24:22.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 1 -- 172.21.15.46:0/1109164526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x5593a27d2ba0 con 0x5593a29c01b0 2024-03-29T08:24:22.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a2be00640 1 -- 172.21.15.46:0/1109164526 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5593a27dd550 con 0x5593a29c01b0 2024-03-29T08:24:22.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a2be00640 1 -- 172.21.15.46:0/1109164526 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f3a38012070 con 0x5593a29c01b0 2024-03-29T08:24:22.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 10 client.7191 fetch_fsmap finished waiting for FSMap version 49 2024-03-29T08:24:22.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 10 client.7191 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:24:22.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 20 client.7191 populate_metadata read hostname 'smithi046' 2024-03-29T08:24:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/1109164526 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x5593a2a0a7f0 0x5593a2a1ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 4 client.7191 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:24:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 1 -- 172.21.15.46:0/1109164526 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593a280a970 con 0x5593a2a0a7f0 2024-03-29T08:24:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a3e000640 1 --2- 172.21.15.46:0/1109164526 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x5593a2a0a7f0 0x5593a2a1ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a3e000640 1 --2- 172.21.15.46:0/1109164526 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x5593a2a0a7f0 0x5593a2a1ec30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b86e190926a5e39e server_cookie=4e0f9bcd0d3e46ed in_seq=0 out_seq=0 2024-03-29T08:24:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a2be00640 10 client.7191 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:24:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a2be00640 1 -- 172.21.15.46:0/1109164526 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5593a280a970 con 0x5593a2a0a7f0 2024-03-29T08:24:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a2be00640 10 client.7191 handle_command_reply: tid=0 2024-03-29T08:24:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 1 client.7191 shutdown 2024-03-29T08:24:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 1 -- 172.21.15.46:0/1109164526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a29c01b0 msgr2=0x5593a2a08c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/1109164526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a29c01b0 0x5593a2a08c30 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f3a38013a00 tx=0x7f3a38001260 comp rx=0 tx=0).stop 2024-03-29T08:24:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 1 -- 172.21.15.46:0/1109164526 shutdown_connections 2024-03-29T08:24:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/1109164526 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x5593a2a0a7f0 0x5593a2a1ec30 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/1109164526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a2a09bb0 0x5593a29c5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/1109164526 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09290 0x5593a29c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/1109164526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a29c01b0 0x5593a2a08c30 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a2be00640 0 client.7191 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:24:22.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 1 -- 172.21.15.46:0/1109164526 >> 172.21.15.46:0/1109164526 conn(0x5593a29bc160 msgr2=0x5593a29bede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:22.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 1 -- 172.21.15.46:0/1109164526 shutdown_connections 2024-03-29T08:24:22.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 1 -- 172.21.15.46:0/1109164526 wait complete. 2024-03-29T08:24:22.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.189+0000 7f3a50ecf000 20 client.7191 trim_cache size 0 max 16384 2024-03-29T08:24:22.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.201+0000 7f3a50ecf000 1 Processor -- start 2024-03-29T08:24:22.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.201+0000 7f3a50ecf000 1 -- start start 2024-03-29T08:24:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.201+0000 7f3a50ecf000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a29c01b0 0x5593a29be550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09290 0x5593a29bea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a2a09bb0 0x5593a29bf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593a24cac40 con 0x5593a29c01b0 2024-03-29T08:24:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593a2a08eb0 con 0x5593a2a09bb0 2024-03-29T08:24:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593a280ca20 con 0x5593a2a09290 2024-03-29T08:24:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a46a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a29c01b0 0x5593a29be550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a46a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a29c01b0 0x5593a29be550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58878/0 (socket says 172.21.15.46:58878) 2024-03-29T08:24:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a46a00640 1 -- 172.21.15.46:0/3273453715 learned_addr learned my addr 172.21.15.46:0/3273453715 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a3e000640 1 --2- 172.21.15.46:0/3273453715 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09290 0x5593a29bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a47400640 1 --2- 172.21.15.46:0/3273453715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a2a09bb0 0x5593a29bf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a46a00640 1 -- 172.21.15.46:0/3273453715 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09290 msgr2=0x5593a29bea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a46a00640 1 --2- 172.21.15.46:0/3273453715 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09290 0x5593a29bea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a46a00640 1 -- 172.21.15.46:0/3273453715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a2a09bb0 msgr2=0x5593a29bf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a46a00640 1 --2- 172.21.15.46:0/3273453715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a2a09bb0 0x5593a29bf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a46a00640 1 -- 172.21.15.46:0/3273453715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593a27b8ef0 con 0x5593a29c01b0 2024-03-29T08:24:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a46a00640 1 --2- 172.21.15.46:0/3273453715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a29c01b0 0x5593a29be550 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3a380074d0 tx=0x7f3a38021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11b7a14416c8a49 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:22.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a37e00640 1 -- 172.21.15.46:0/3273453715 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a38036030 con 0x5593a29c01b0 2024-03-29T08:24:22.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a37e00640 1 -- 172.21.15.46:0/3273453715 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a38012ae0 con 0x5593a29c01b0 2024-03-29T08:24:22.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a37e00640 1 -- 172.21.15.46:0/3273453715 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a38021410 con 0x5593a29c01b0 2024-03-29T08:24:22.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 -- 172.21.15.46:0/3273453715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a29c01b0 msgr2=0x5593a29be550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/3273453715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a29c01b0 0x5593a29be550 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3a380074d0 tx=0x7f3a38021000 comp rx=0 tx=0).stop 2024-03-29T08:24:22.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 -- 172.21.15.46:0/3273453715 shutdown_connections 2024-03-29T08:24:22.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/3273453715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a2a09bb0 0x5593a29bf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/3273453715 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09290 0x5593a29bea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/3273453715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a29c01b0 0x5593a29be550 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 -- 172.21.15.46:0/3273453715 >> 172.21.15.46:0/3273453715 conn(0x5593a29bc160 msgr2=0x5593a29c4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:22.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 -- 172.21.15.46:0/3273453715 shutdown_connections 2024-03-29T08:24:22.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 -- 172.21.15.46:0/3273453715 wait complete. 2024-03-29T08:24:22.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 -- start start 2024-03-29T08:24:22.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 Processor -- start 2024-03-29T08:24:22.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a29c01b0 0x5593a2a0eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a2a09290 0x5593a2a0f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09bb0 0x5593a2a68540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593a2811b90 con 0x5593a2a09290 2024-03-29T08:24:22.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593a2a22e80 con 0x5593a29c01b0 2024-03-29T08:24:22.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593a280b3a0 con 0x5593a2a09bb0 2024-03-29T08:24:22.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a50ecf000 10 client.0 _ll_register_callbacks cb 0x5593a28ab870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:24:22.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a3e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a2a09290 0x5593a2a0f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a3e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a2a09290 0x5593a2a0f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58884/0 (socket says 172.21.15.46:58884) 2024-03-29T08:24:22.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.205+0000 7f3a3e000640 1 -- 172.21.15.46:0/1458192164 learned_addr learned my addr 172.21.15.46:0/1458192164 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:22.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a46a00640 1 --2- 172.21.15.46:0/1458192164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a29c01b0 0x5593a2a0eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a47400640 1 --2- 172.21.15.46:0/1458192164 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09bb0 0x5593a2a68540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a3e000640 1 -- 172.21.15.46:0/1458192164 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09bb0 msgr2=0x5593a2a68540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:22.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a3e000640 1 --2- 172.21.15.46:0/1458192164 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09bb0 0x5593a2a68540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a3e000640 1 -- 172.21.15.46:0/1458192164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a29c01b0 msgr2=0x5593a2a0eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:22.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a3e000640 1 --2- 172.21.15.46:0/1458192164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a29c01b0 0x5593a2a0eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a3e000640 1 -- 172.21.15.46:0/1458192164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593a27d2ba0 con 0x5593a2a09290 2024-03-29T08:24:22.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a47400640 1 --2- 172.21.15.46:0/1458192164 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09bb0 0x5593a2a68540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:22.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a3e000640 1 --2- 172.21.15.46:0/1458192164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a2a09290 0x5593a2a0f430 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3a3001ca40 tx=0x7f3a3011e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900c07674b4679b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:22.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a2be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:24:22.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a2be00640 1 -- 172.21.15.46:0/1458192164 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a300ef190 con 0x5593a2a09290 2024-03-29T08:24:22.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a2be00640 1 -- 172.21.15.46:0/1458192164 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f3a3011b040 con 0x5593a2a09290 2024-03-29T08:24:22.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 1 -- 172.21.15.46:0/1458192164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593a27bd620 con 0x5593a2a09290 2024-03-29T08:24:22.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a2be00640 1 -- 172.21.15.46:0/1458192164 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a3011e0e0 con 0x5593a2a09290 2024-03-29T08:24:22.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 10 client.7197 fetch_fsmap learned FSMap version 49 2024-03-29T08:24:22.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 1 -- 172.21.15.46:0/1458192164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x5593a2509fc0 con 0x5593a2a09290 2024-03-29T08:24:22.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a2be00640 1 -- 172.21.15.46:0/1458192164 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5593a27bd620 con 0x5593a2a09290 2024-03-29T08:24:22.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a2be00640 1 -- 172.21.15.46:0/1458192164 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 49) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f3a300c6020 con 0x5593a2a09290 2024-03-29T08:24:22.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 10 client.7197 fetch_fsmap finished waiting for FSMap version 49 2024-03-29T08:24:22.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 10 client.7197 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:24:22.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 20 client.7197 populate_metadata read hostname 'smithi046' 2024-03-29T08:24:22.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/1458192164 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x5593a2a68e10 0x5593a2a74650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 4 client.7197 mds_command: new command op to 6876 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-29T08:24:22.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 1 -- 172.21.15.46:0/1458192164 --> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5593a280c4e0 con 0x5593a2a68e10 2024-03-29T08:24:22.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a46a00640 1 --2- 172.21.15.46:0/1458192164 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x5593a2a68e10 0x5593a2a74650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a46a00640 1 --2- 172.21.15.46:0/1458192164 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x5593a2a68e10 0x5593a2a74650 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74cabed7824530d6 server_cookie=ae4f67a98502737c in_seq=0 out_seq=0 2024-03-29T08:24:22.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a2be00640 10 client.7197 ms_handle_connect on v2:172.21.15.46:6834/1672348161 2024-03-29T08:24:22.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a2be00640 1 -- 172.21.15.46:0/1458192164 <== mds.0 v2:172.21.15.46:6834/1672348161 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5593a280c4e0 con 0x5593a2a68e10 2024-03-29T08:24:22.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a2be00640 10 client.7197 handle_command_reply: tid=0 2024-03-29T08:24:22.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 1 client.7197 shutdown 2024-03-29T08:24:22.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 1 -- 172.21.15.46:0/1458192164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a2a09290 msgr2=0x5593a2a0f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/1458192164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a2a09290 0x5593a2a0f430 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3a3001ca40 tx=0x7f3a3011e070 comp rx=0 tx=0).stop 2024-03-29T08:24:22.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 1 -- 172.21.15.46:0/1458192164 shutdown_connections 2024-03-29T08:24:22.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/1458192164 >> [v2:172.21.15.46:6834/1672348161,v1:172.21.15.46:6835/1672348161] conn(0x5593a2a68e10 0x5593a2a74650 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/1458192164 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5593a2a09bb0 0x5593a2a68540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/1458192164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593a2a09290 0x5593a2a0f430 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 1 --2- 172.21.15.46:0/1458192164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593a29c01b0 0x5593a2a0eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 1 -- 172.21.15.46:0/1458192164 >> 172.21.15.46:0/1458192164 conn(0x5593a29bc160 msgr2=0x5593a29c4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:22.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a2be00640 0 client.7197 ms_handle_reset on v2:172.21.15.46:6834/1672348161 2024-03-29T08:24:22.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 1 -- 172.21.15.46:0/1458192164 shutdown_connections 2024-03-29T08:24:22.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 1 -- 172.21.15.46:0/1458192164 wait complete. 2024-03-29T08:24:22.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a50ecf000 20 client.7197 trim_cache size 0 max 16384 2024-03-29T08:24:22.215 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:22.215 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-29T08:24:22.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a4da00640 1 -- 172.21.15.46:0/992272828 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3a240666c0 msgr2=0x7f3a24068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a4da00640 1 --2- 172.21.15.46:0/992272828 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3a240666c0 0x7f3a24068b80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3a380012f0 tx=0x7f3a38001700 comp rx=0 tx=0).stop 2024-03-29T08:24:22.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a4da00640 1 -- 172.21.15.46:0/992272828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a48142a50 msgr2=0x7f3a4813ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a4da00640 1 --2- 172.21.15.46:0/992272828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a48142a50 0x7f3a4813ecd0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3a300ae040 tx=0x7f3a30019270 comp rx=0 tx=0).stop 2024-03-29T08:24:22.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a4da00640 1 -- 172.21.15.46:0/992272828 shutdown_connections 2024-03-29T08:24:22.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a4da00640 1 --2- 172.21.15.46:0/992272828 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3a240666c0 0x7f3a24068b80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a4da00640 1 --2- 172.21.15.46:0/992272828 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3a4815df90 0x7f3a4815ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a4da00640 1 --2- 172.21.15.46:0/992272828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3a48142a50 0x7f3a4813ecd0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a4da00640 1 --2- 172.21.15.46:0/992272828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a480cbea0 0x7f3a48157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a4da00640 1 -- 172.21.15.46:0/992272828 >> 172.21.15.46:0/992272828 conn(0x7f3a4806a390 msgr2=0x7f3a4815bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:22.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a4da00640 1 -- 172.21.15.46:0/992272828 shutdown_connections 2024-03-29T08:24:22.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.209+0000 7f3a4da00640 1 -- 172.21.15.46:0/992272828 wait complete. 2024-03-29T08:24:22.229 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-29T08:24:22.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.369+0000 7ff3e8200640 1 Processor -- start 2024-03-29T08:24:22.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.369+0000 7ff3e8200640 1 -- start start 2024-03-29T08:24:22.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.369+0000 7ff3e8200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff3e015ab70 0x7ff3e0098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.369+0000 7ff3e8200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff3e0098f80 0x7ff3e00b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.369+0000 7ff3e8200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e00ce660 con 0x7ff3e015a790 2024-03-29T08:24:22.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.369+0000 7ff3e8200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e00cf700 con 0x7ff3e0098f80 2024-03-29T08:24:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.369+0000 7ff3e8200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e00993f0 con 0x7ff3e015ab70 2024-03-29T08:24:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.369+0000 7ff3e6200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff3e0098f80 0x7ff3e00b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.369+0000 7ff3e6200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff3e0098f80 0x7ff3e00b5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58894/0 (socket says 172.21.15.46:58894) 2024-03-29T08:24:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.369+0000 7ff3e6200640 1 -- 172.21.15.46:0/714878394 learned_addr learned my addr 172.21.15.46:0/714878394 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.369+0000 7ff3e4e00640 1 --2- 172.21.15.46:0/714878394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff3e015ab70 0x7ff3e0098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.373+0000 7ff3e6200640 1 -- 172.21.15.46:0/714878394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff3e015ab70 msgr2=0x7ff3e0098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:22.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.373+0000 7ff3e6200640 1 --2- 172.21.15.46:0/714878394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff3e015ab70 0x7ff3e0098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.373+0000 7ff3e6200640 1 -- 172.21.15.46:0/714878394 >> v1:172.21.15.73:6789/0 conn(0x7ff3e015a790 legacy=0x7ff3e00cbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.373+0000 7ff3e6200640 1 -- 172.21.15.46:0/714878394 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e00ce940 con 0x7ff3e0098f80 2024-03-29T08:24:22.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.373+0000 7ff3dfe00640 1 -- 172.21.15.46:0/714878394 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769662061 0 0) 0x7ff3e00ce660 con 0x7ff3e015a790 2024-03-29T08:24:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.373+0000 7ff3e6200640 1 --2- 172.21.15.46:0/714878394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3e0098f80 0x7ff3e00b5860 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8009e80 tx=0x7ff3d801c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e2ca6d29280c8d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.373+0000 7ff3dfe00640 1 -- 172.21.15.46:0/714878394 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d80a56c0 con 0x7ff3e0098f80 2024-03-29T08:24:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.373+0000 7ff3dfe00640 1 -- 172.21.15.46:0/714878394 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d80a5820 con 0x7ff3e0098f80 2024-03-29T08:24:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.373+0000 7ff3dfe00640 1 -- 172.21.15.46:0/714878394 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d80a5bb0 con 0x7ff3e0098f80 2024-03-29T08:24:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.373+0000 7ff3e8200640 1 -- 172.21.15.46:0/714878394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3e0098f80 msgr2=0x7ff3e00b5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.373+0000 7ff3e8200640 1 --2- 172.21.15.46:0/714878394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3e0098f80 0x7ff3e00b5860 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7ff3d8009e80 tx=0x7ff3d801c800 comp rx=0 tx=0).stop 2024-03-29T08:24:22.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.373+0000 7ff3e8200640 1 -- 172.21.15.46:0/714878394 shutdown_connections 2024-03-29T08:24:22.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.373+0000 7ff3e8200640 1 --2- 172.21.15.46:0/714878394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3e0098f80 0x7ff3e00b5860 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.373+0000 7ff3e8200640 1 --2- 172.21.15.46:0/714878394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff3e015ab70 0x7ff3e0098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.373+0000 7ff3e8200640 1 -- 172.21.15.46:0/714878394 >> 172.21.15.46:0/714878394 conn(0x7ff3e00b21a0 msgr2=0x7ff3e00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:22.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e8200640 1 -- 172.21.15.46:0/714878394 shutdown_connections 2024-03-29T08:24:22.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e8200640 1 -- 172.21.15.46:0/714878394 wait complete. 2024-03-29T08:24:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e8200640 1 Processor -- start 2024-03-29T08:24:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e8200640 1 -- start start 2024-03-29T08:24:22.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e8200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff3e0098f80 0x7ff3e014e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e8200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3e015a790 0x7ff3e014bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e8200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3e015ab70 0x7ff3e006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e8200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e014d5f0 con 0x7ff3e015ab70 2024-03-29T08:24:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e8200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e006bdf0 con 0x7ff3e015a790 2024-03-29T08:24:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e8200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e014de50 con 0x7ff3e0098f80 2024-03-29T08:24:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3e015ab70 0x7ff3e006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e6200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3e015ab70 0x7ff3e006b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58898/0 (socket says 172.21.15.46:58898) 2024-03-29T08:24:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e6200640 1 -- 172.21.15.46:0/3018865352 learned_addr learned my addr 172.21.15.46:0/3018865352 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e5800640 1 --2- 172.21.15.46:0/3018865352 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff3e0098f80 0x7ff3e014e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e4e00640 1 --2- 172.21.15.46:0/3018865352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3e015a790 0x7ff3e014bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e6200640 1 -- 172.21.15.46:0/3018865352 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff3e0098f80 msgr2=0x7ff3e014e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e6200640 1 --2- 172.21.15.46:0/3018865352 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff3e0098f80 0x7ff3e014e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e6200640 1 -- 172.21.15.46:0/3018865352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3e015a790 msgr2=0x7ff3e014bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e6200640 1 --2- 172.21.15.46:0/3018865352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3e015a790 0x7ff3e014bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e6200640 1 -- 172.21.15.46:0/3018865352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d8009b30 con 0x7ff3e015ab70 2024-03-29T08:24:22.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e6200640 1 --2- 172.21.15.46:0/3018865352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3e015ab70 0x7ff3e006b8b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7ff3d801a430 tx=0x7ff3d80195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3dd600640 1 -- 172.21.15.46:0/3018865352 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d80b8670 con 0x7ff3e015ab70 2024-03-29T08:24:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3dd600640 1 -- 172.21.15.46:0/3018865352 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d80b87d0 con 0x7ff3e015ab70 2024-03-29T08:24:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3dd600640 1 -- 172.21.15.46:0/3018865352 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d80b8b80 con 0x7ff3e015ab70 2024-03-29T08:24:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.377+0000 7ff3e8200640 1 -- 172.21.15.46:0/3018865352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e006c9d0 con 0x7ff3e015ab70 2024-03-29T08:24:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.381+0000 7ff3e8200640 1 -- 172.21.15.46:0/3018865352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e014dae0 con 0x7ff3e015ab70 2024-03-29T08:24:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.381+0000 7ff3e8200640 1 -- 172.21.15.46:0/3018865352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e0066c90 con 0x7ff3e015ab70 2024-03-29T08:24:22.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.381+0000 7ff3dd600640 1 -- 172.21.15.46:0/3018865352 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff3d80b8ce0 con 0x7ff3e015ab70 2024-03-29T08:24:22.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.381+0000 7ff3dd600640 1 --2- 172.21.15.46:0/3018865352 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3c0066690 0x7ff3c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.381+0000 7ff3dd600640 1 -- 172.21.15.46:0/3018865352 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff3d811f740 con 0x7ff3e015ab70 2024-03-29T08:24:22.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.381+0000 7ff3e5800640 1 --2- 172.21.15.46:0/3018865352 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3c0066690 0x7ff3c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.385+0000 7ff3e5800640 1 --2- 172.21.15.46:0/3018865352 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3c0066690 0x7ff3c0068b50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4000f70 tx=0x7ff3d4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:22.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.385+0000 7ff3dd600640 1 -- 172.21.15.46:0/3018865352 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff3d80ecaa0 con 0x7ff3e015ab70 2024-03-29T08:24:22.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.581+0000 7ff3e8200640 1 -- 172.21.15.46:0/3018865352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff3e014df90 con 0x7ff3e015ab70 2024-03-29T08:24:22.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.581+0000 7ff3dd600640 1 -- 172.21.15.46:0/3018865352 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ff3d8009510 con 0x7ff3e015ab70 2024-03-29T08:24:22.581 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:22.581 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-29T08:24:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.581+0000 7ff3e8200640 1 -- 172.21.15.46:0/3018865352 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3c0066690 msgr2=0x7ff3c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.581+0000 7ff3e8200640 1 --2- 172.21.15.46:0/3018865352 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3c0066690 0x7ff3c0068b50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4000f70 tx=0x7ff3d4009040 comp rx=0 tx=0).stop 2024-03-29T08:24:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.581+0000 7ff3e8200640 1 -- 172.21.15.46:0/3018865352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3e015ab70 msgr2=0x7ff3e006b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.581+0000 7ff3e8200640 1 --2- 172.21.15.46:0/3018865352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3e015ab70 0x7ff3e006b8b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7ff3d801a430 tx=0x7ff3d80195a0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.581+0000 7ff3e8200640 1 -- 172.21.15.46:0/3018865352 shutdown_connections 2024-03-29T08:24:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.581+0000 7ff3e8200640 1 --2- 172.21.15.46:0/3018865352 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3c0066690 0x7ff3c0068b50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.581+0000 7ff3e8200640 1 --2- 172.21.15.46:0/3018865352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3e015a790 0x7ff3e014bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.581+0000 7ff3e8200640 1 --2- 172.21.15.46:0/3018865352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3e015ab70 0x7ff3e006b8b0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.581+0000 7ff3e8200640 1 --2- 172.21.15.46:0/3018865352 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff3e0098f80 0x7ff3e014e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.581+0000 7ff3e8200640 1 -- 172.21.15.46:0/3018865352 >> 172.21.15.46:0/3018865352 conn(0x7ff3e00b21a0 msgr2=0x7ff3e00b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:22.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.581+0000 7ff3e8200640 1 -- 172.21.15.46:0/3018865352 shutdown_connections 2024-03-29T08:24:22.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.581+0000 7ff3e8200640 1 -- 172.21.15.46:0/3018865352 wait complete. 2024-03-29T08:24:22.602 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-29T08:24:22.602 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:24:22.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.741+0000 7f2b43000640 1 Processor -- start 2024-03-29T08:24:22.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.741+0000 7f2b43000640 1 -- start start 2024-03-29T08:24:22.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b43000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2b3c15a7c0 0x7f2b3c06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b43000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b3c06d130 con 0x7f2b3c15cd90 2024-03-29T08:24:22.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b43000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b3c15b8f0 con 0x7f2b3c1593d0 2024-03-29T08:24:22.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b43000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b3c15ad50 con 0x7f2b3c15a7c0 2024-03-29T08:24:22.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b33400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2b3c15cd90 0x7f2b3c15d940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41082/0 (socket says 172.21.15.46:41082) 2024-03-29T08:24:22.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b33400640 1 -- 172.21.15.46:0/2953303947 learned_addr learned my addr 172.21.15.46:0/2953303947 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b41000640 1 --2- 172.21.15.46:0/2953303947 >> v2:172.21.15.73:3300/0 conn(0x7f2b3c15a7c0 0x7f2b3c06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3b400640 1 -- 172.21.15.46:0/2953303947 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1945401722 0 0) 0x7f2b3c06d130 con 0x7f2b3c15cd90 2024-03-29T08:24:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3b400640 1 -- 172.21.15.46:0/2953303947 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b20003530 con 0x7f2b3c15cd90 2024-03-29T08:24:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3b400640 1 -- 172.21.15.46:0/2953303947 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3082945245 0 0) 0x7f2b20003530 con 0x7f2b3c15cd90 2024-03-29T08:24:22.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3b400640 1 -- 172.21.15.46:0/2953303947 >> v2:172.21.15.73:3300/0 conn(0x7f2b3c15a7c0 msgr2=0x7f2b3c06cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3b400640 1 --2- 172.21.15.46:0/2953303947 >> v2:172.21.15.73:3300/0 conn(0x7f2b3c15a7c0 0x7f2b3c06cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3b400640 1 -- 172.21.15.46:0/2953303947 >> v1:172.21.15.73:6789/0 conn(0x7f2b3c1593d0 legacy=0x7f2b3c15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3b400640 1 -- 172.21.15.46:0/2953303947 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c15e050 con 0x7f2b3c15cd90 2024-03-29T08:24:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3b400640 1 -- 172.21.15.46:0/2953303947 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f2b24001680 con 0x7f2b3c15cd90 2024-03-29T08:24:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b41000640 1 --2- 172.21.15.46:0/2953303947 >> v2:172.21.15.73:3300/0 conn(0x7f2b3c15a7c0 0x7f2b3c06cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:24:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3b400640 1 -- 172.21.15.46:0/2953303947 >> v1:172.21.15.46:6789/0 conn(0x7f2b3c15cd90 legacy=0x7f2b3c15d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3b400640 1 --2- 172.21.15.46:0/2953303947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b200039d0 0x7f2b20003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3b400640 1 --2- 172.21.15.46:0/2953303947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b200047d0 0x7f2b20004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3b400640 1 --2- 172.21.15.46:0/2953303947 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b20005190 0x7f2b2000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3b400640 1 -- 172.21.15.46:0/2953303947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c15ad50 con 0x7f2b200039d0 2024-03-29T08:24:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3b400640 1 -- 172.21.15.46:0/2953303947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2000c090 con 0x7f2b200047d0 2024-03-29T08:24:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3b400640 1 -- 172.21.15.46:0/2953303947 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2000e460 con 0x7f2b20005190 2024-03-29T08:24:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3be00640 1 --2- 172.21.15.46:0/2953303947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b200039d0 0x7f2b20003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b33400640 1 --2- 172.21.15.46:0/2953303947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b200047d0 0x7f2b20004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b41000640 1 --2- 172.21.15.46:0/2953303947 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b20005190 0x7f2b2000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3be00640 1 -- 172.21.15.46:0/2953303947 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b20005190 msgr2=0x7f2b2000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3be00640 1 --2- 172.21.15.46:0/2953303947 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b20005190 0x7f2b2000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3be00640 1 -- 172.21.15.46:0/2953303947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b200047d0 msgr2=0x7f2b20004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3be00640 1 --2- 172.21.15.46:0/2953303947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b200047d0 0x7f2b20004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b3be00640 1 -- 172.21.15.46:0/2953303947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b20003530 con 0x7f2b200039d0 2024-03-29T08:24:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b41000640 1 --2- 172.21.15.46:0/2953303947 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b20005190 0x7f2b2000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:22.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.745+0000 7f2b41000640 1 -- 172.21.15.46:0/2953303947 reap_dead start 2024-03-29T08:24:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.749+0000 7f2b3be00640 1 --2- 172.21.15.46:0/2953303947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b200039d0 0x7f2b20003df0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c001780 tx=0x7f2b2c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e679c926f4da64 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.749+0000 7f2b3b400640 1 -- 172.21.15.46:0/2953303947 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c0052d0 con 0x7f2b200039d0 2024-03-29T08:24:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.749+0000 7f2b3b400640 1 -- 172.21.15.46:0/2953303947 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c0a7050 con 0x7f2b200039d0 2024-03-29T08:24:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.749+0000 7f2b3b400640 1 -- 172.21.15.46:0/2953303947 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c003600 con 0x7f2b200039d0 2024-03-29T08:24:22.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.749+0000 7f2b43000640 1 -- 172.21.15.46:0/2953303947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b200039d0 msgr2=0x7f2b20003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.749+0000 7f2b43000640 1 --2- 172.21.15.46:0/2953303947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b200039d0 0x7f2b20003df0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c001780 tx=0x7f2b2c01c400 comp rx=0 tx=0).stop 2024-03-29T08:24:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.749+0000 7f2b43000640 1 -- 172.21.15.46:0/2953303947 shutdown_connections 2024-03-29T08:24:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.749+0000 7f2b43000640 1 --2- 172.21.15.46:0/2953303947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b200039d0 0x7f2b20003df0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.749+0000 7f2b43000640 1 -- 172.21.15.46:0/2953303947 >> 172.21.15.46:0/2953303947 conn(0x7f2b3c066110 msgr2=0x7f2b3c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.749+0000 7f2b43000640 1 -- 172.21.15.46:0/2953303947 shutdown_connections 2024-03-29T08:24:22.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.749+0000 7f2b43000640 1 -- 172.21.15.46:0/2953303947 wait complete. 2024-03-29T08:24:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.749+0000 7f2b43000640 1 Processor -- start 2024-03-29T08:24:22.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b43000640 1 -- start start 2024-03-29T08:24:22.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b43000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b200039d0 0x7f2b3c0ac740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b43000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b3c1f9150 0x7f2b3c0ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b43000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b3c1f95b0 0x7f2b3c0aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b43000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c1f5bd0 con 0x7f2b3c1f9150 2024-03-29T08:24:22.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b43000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c1f5eb0 con 0x7f2b3c1f95b0 2024-03-29T08:24:22.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b43000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3c1f6190 con 0x7f2b200039d0 2024-03-29T08:24:22.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b33400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b3c1f9150 0x7f2b3c0ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b33400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b3c1f9150 0x7f2b3c0ae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58914/0 (socket says 172.21.15.46:58914) 2024-03-29T08:24:22.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b33400640 1 -- 172.21.15.46:0/98081667 learned_addr learned my addr 172.21.15.46:0/98081667 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:22.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b3be00640 1 --2- 172.21.15.46:0/98081667 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b200039d0 0x7f2b3c0ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b41000640 1 --2- 172.21.15.46:0/98081667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b3c1f95b0 0x7f2b3c0aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b33400640 1 -- 172.21.15.46:0/98081667 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b200039d0 msgr2=0x7f2b3c0ac740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b33400640 1 --2- 172.21.15.46:0/98081667 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b200039d0 0x7f2b3c0ac740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b33400640 1 -- 172.21.15.46:0/98081667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b3c1f95b0 msgr2=0x7f2b3c0aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b33400640 1 --2- 172.21.15.46:0/98081667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b3c1f95b0 0x7f2b3c0aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b33400640 1 -- 172.21.15.46:0/98081667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3c1f8950 con 0x7f2b3c1f9150 2024-03-29T08:24:22.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b3be00640 1 --2- 172.21.15.46:0/98081667 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b200039d0 0x7f2b3c0ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:22.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b33400640 1 --2- 172.21.15.46:0/98081667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b3c1f9150 0x7f2b3c0ae680 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2b24000c10 tx=0x7f2b24004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:22.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b38c00640 1 -- 172.21.15.46:0/98081667 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2400f710 con 0x7f2b3c1f9150 2024-03-29T08:24:22.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b38c00640 1 -- 172.21.15.46:0/98081667 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2400f8b0 con 0x7f2b3c1f9150 2024-03-29T08:24:22.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b43000640 1 -- 172.21.15.46:0/98081667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b3c1f8c30 con 0x7f2b3c1f9150 2024-03-29T08:24:22.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b38c00640 1 -- 172.21.15.46:0/98081667 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2400fc60 con 0x7f2b3c1f9150 2024-03-29T08:24:22.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b43000640 1 -- 172.21.15.46:0/98081667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3c155a00 con 0x7f2b3c1f9150 2024-03-29T08:24:22.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.753+0000 7f2b43000640 1 -- 172.21.15.46:0/98081667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b3c066c90 con 0x7f2b3c1f9150 2024-03-29T08:24:22.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.757+0000 7f2b38c00640 1 -- 172.21.15.46:0/98081667 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2b2401d460 con 0x7f2b3c1f9150 2024-03-29T08:24:22.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.757+0000 7f2b38c00640 1 --2- 172.21.15.46:0/98081667 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2b180666c0 0x7f2b18068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:22.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.757+0000 7f2b38c00640 1 -- 172.21.15.46:0/98081667 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2b240842c0 con 0x7f2b3c1f9150 2024-03-29T08:24:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.757+0000 7f2b3be00640 1 --2- 172.21.15.46:0/98081667 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2b180666c0 0x7f2b18068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:22.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.757+0000 7f2b3be00640 1 --2- 172.21.15.46:0/98081667 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2b180666c0 0x7f2b18068b80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c001750 tx=0x7f2b2c0011a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:22.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.761+0000 7f2b38c00640 1 -- 172.21.15.46:0/98081667 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2b24051590 con 0x7f2b3c1f9150 2024-03-29T08:24:22.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.961+0000 7f2b43000640 1 -- 172.21.15.46:0/98081667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2b3c0ce5b0 con 0x7f2b3c1f9150 2024-03-29T08:24:22.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.961+0000 7f2b38c00640 1 -- 172.21.15.46:0/98081667 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2b24054bb0 con 0x7f2b3c1f9150 2024-03-29T08:24:22.962 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:24:22.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.965+0000 7f2b43000640 1 -- 172.21.15.46:0/98081667 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2b180666c0 msgr2=0x7f2b18068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.965+0000 7f2b43000640 1 --2- 172.21.15.46:0/98081667 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2b180666c0 0x7f2b18068b80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c001750 tx=0x7f2b2c0011a0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.965+0000 7f2b43000640 1 -- 172.21.15.46:0/98081667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b3c1f9150 msgr2=0x7f2b3c0ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.965+0000 7f2b43000640 1 --2- 172.21.15.46:0/98081667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b3c1f9150 0x7f2b3c0ae680 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2b24000c10 tx=0x7f2b24004f90 comp rx=0 tx=0).stop 2024-03-29T08:24:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.965+0000 7f2b43000640 1 -- 172.21.15.46:0/98081667 shutdown_connections 2024-03-29T08:24:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.965+0000 7f2b43000640 1 --2- 172.21.15.46:0/98081667 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2b180666c0 0x7f2b18068b80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.965+0000 7f2b43000640 1 --2- 172.21.15.46:0/98081667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b3c1f95b0 0x7f2b3c0aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.965+0000 7f2b43000640 1 --2- 172.21.15.46:0/98081667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b3c1f9150 0x7f2b3c0ae680 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.965+0000 7f2b43000640 1 --2- 172.21.15.46:0/98081667 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b200039d0 0x7f2b3c0ac740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.965+0000 7f2b43000640 1 -- 172.21.15.46:0/98081667 >> 172.21.15.46:0/98081667 conn(0x7f2b3c066110 msgr2=0x7f2b3c0ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.965+0000 7f2b43000640 1 -- 172.21.15.46:0/98081667 shutdown_connections 2024-03-29T08:24:22.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:22.965+0000 7f2b43000640 1 -- 172.21.15.46:0/98081667 wait complete. 2024-03-29T08:24:22.984 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:24:22.985 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:24:22.985 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-29T08:24:22.985 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T08:24:22.985 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:24:22.985 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:24:22.985 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:24:22.985 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:24:22.985 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:24:22.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:22.985 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-29T08:24:22.994 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:24:22.994 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:24:22.995 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:24:22.995 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:24:22.995 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:24:22.995 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:24:22.995 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:24:22.995 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:24:22.995 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-29T08:24:22.995 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:24:22.995 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:24:22.995 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-29T08:24:22.995 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:24:22.995 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:24:22.995 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-29T08:24:22.995 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T08:24:22.995 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42074sec preferred_lft 42074sec 2024-03-29T08:24:22.996 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-29T08:24:22.996 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:24:22.996 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:24:22.996 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:24:22.996 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-29T08:24:22.996 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:24:22.996 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:d9:58:09:4c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:24:22.996 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:24:22.996 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:24:22.996 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:24:22.996 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T08:24:22.996 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T08:24:22.996 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:24:22.996 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T08:24:22.996 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:24:22.997 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:24:22.997 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:24:22.997 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T08:24:22.997 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:24:22.997 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:24:22.997 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T08:24:22.997 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-29T08:24:22.997 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:24:22.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:22.998 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:24:23.061 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:24:23.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:23.064 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-29T08:24:23.116 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-29T08:24:23.116 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:24:23.116 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T08:24:23.116 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:24:23.116 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T08:24:23.117 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:24:23.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:23.117 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:24:23.117 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:24:23.117 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:24:23.117 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:24:23.117 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:24:23.117 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:24:23.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:23.213 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:24:23.222 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T08:24:23.223 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T08:24:23.223 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:24:23.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:23.223 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-03-29T08:24:23.287 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T08:24:23.287 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:24:23.287 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T08:24:23.340 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:24:23.340 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:24:23.340 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:24:23.390 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:24:23.452 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:24:23.452 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:24:23.513 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:24:23.515 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:24:23.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:23.516 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:24:23.566 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:24:23.567 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T08:24:23.610 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:24:23.630 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:24:23.630 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:24:23.689 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:24:23.691 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:24:23.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:23.692 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:24:23.703 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-03-29T08:24:23.705+0000 7f0fb392d580 -1 init, newargv = 0x557812134170 newargc=15 2024-03-29T08:24:23.704 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[43296]: starting ceph client 2024-03-29T08:24:23.712 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[43296]: starting fuse 2024-03-29T08:24:24.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:24:24.839 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:24:24.839 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:24:24.899 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:24:24.901 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:24:24.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:24.902 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:24:24.956 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:24:24.956 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-29T08:24:24.956 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:24:24.956 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-03-29T08:24:24.956 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-03-29T08:24:24.956 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:24:24.956 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-03-29T08:24:24.956 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:24:24.957 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-03-29T08:24:24.957 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T08:24:24.957 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-03-29T08:24:24.957 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:24:24.957 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:24:24.957 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-03-29T08:24:24.957 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-03-29T08:24:24.957 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-03-29T08:24:24.957 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-03-29T08:24:24.957 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:24:24.957 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-03-29T08:24:24.957 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:24:24.957 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:24:24.957 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:24:24.957 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-03-29T08:24:24.958 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-03-29T08:24:24.958 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-03-29T08:24:24.958 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:24:24.958 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:24:24.958 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-03-29T08:24:24.958 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:24:25.090 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.43296.asok 2024-03-29T08:24:25.273 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43296.asok 2024-03-29T08:24:25.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:25.273 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43296.asok status 2024-03-29T08:24:25.455 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:24:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-03-29T08:24:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:24:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:24:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-03-29T08:24:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-03-29T08:24:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T08:24:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "43296", 2024-03-29T08:24:25.455 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-03-29T08:24:25.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:25.456 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-03-29T08:24:25.456 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-03-29T08:24:25.456 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 7215, 2024-03-29T08:24:25.456 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-03-29T08:24:25.456 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-03-29T08:24:25.456 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-03-29T08:24:25.456 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 7215 2024-03-29T08:24:25.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:25.456 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:24:25.456 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:24:25.456 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:24:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1968491565 2024-03-29T08:24:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:24:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:24:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:24:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1968491565 2024-03-29T08:24:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.7215 192.168.0.1:0/1968491565", 2024-03-29T08:24:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/1968491565", 2024-03-29T08:24:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-03-29T08:24:25.457 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 49, 2024-03-29T08:24:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 80, 2024-03-29T08:24:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:24:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-03-29T08:24:25.458 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-03-29T08:24:25.458 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:24:25.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:25.471 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-03-29T08:24:25.471 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:24:25.481 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-03-29T08:24:25.481 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T08:24:25.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:25.482 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:24:25.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:25.543 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-03-29T08:24:25.594 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:24:25.595 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-03-29T08:24:25.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.681+0000 7fa34beeb080 1 Processor -- start 2024-03-29T08:24:25.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 -- start start 2024-03-29T08:24:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa330003a80 0x5590bd67b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590bd683a50 0x5590bd67cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5590bd67a200 0x5590bd67a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590bd4ee960 con 0x5590bd683a50 2024-03-29T08:24:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590bd4ee600 con 0x5590bd67a200 2024-03-29T08:24:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590bd4e5690 con 0x7fa330003a80 2024-03-29T08:24:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa343e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590bd683a50 0x5590bd67cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa343e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590bd683a50 0x5590bd67cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58924/0 (socket says 172.21.15.46:58924) 2024-03-29T08:24:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa343e00640 1 -- 172.21.15.46:0/1091280105 learned_addr learned my addr 172.21.15.46:0/1091280105 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:25.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa349c00640 1 --2- 172.21.15.46:0/1091280105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5590bd67a200 0x5590bd67a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa349200640 1 --2- 172.21.15.46:0/1091280105 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa330003a80 0x5590bd67b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa343e00640 1 -- 172.21.15.46:0/1091280105 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa330003a80 msgr2=0x5590bd67b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa343e00640 1 --2- 172.21.15.46:0/1091280105 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa330003a80 0x5590bd67b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa343e00640 1 -- 172.21.15.46:0/1091280105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5590bd67a200 msgr2=0x5590bd67a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa343e00640 1 --2- 172.21.15.46:0/1091280105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5590bd67a200 0x5590bd67a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa343e00640 1 -- 172.21.15.46:0/1091280105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590bd4fd8c0 con 0x5590bd683a50 2024-03-29T08:24:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa343e00640 1 --2- 172.21.15.46:0/1091280105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590bd683a50 0x5590bd67cef0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa334009750 tx=0x7fa334005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa8fd280064e2c26 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa342a00640 1 -- 172.21.15.46:0/1091280105 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33400f040 con 0x5590bd683a50 2024-03-29T08:24:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa342a00640 1 -- 172.21.15.46:0/1091280105 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa334008b20 con 0x5590bd683a50 2024-03-29T08:24:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa342a00640 1 -- 172.21.15.46:0/1091280105 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa334008e90 con 0x5590bd683a50 2024-03-29T08:24:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 -- 172.21.15.46:0/1091280105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590bd683a50 msgr2=0x5590bd67cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:25.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 --2- 172.21.15.46:0/1091280105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590bd683a50 0x5590bd67cef0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa334009750 tx=0x7fa334005a10 comp rx=0 tx=0).stop 2024-03-29T08:24:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 -- 172.21.15.46:0/1091280105 shutdown_connections 2024-03-29T08:24:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 --2- 172.21.15.46:0/1091280105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5590bd67a200 0x5590bd67a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 --2- 172.21.15.46:0/1091280105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590bd683a50 0x5590bd67cef0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 --2- 172.21.15.46:0/1091280105 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa330003a80 0x5590bd67b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 -- 172.21.15.46:0/1091280105 >> 172.21.15.46:0/1091280105 conn(0x5590bd5ad550 msgr2=0x5590bd5ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 -- 172.21.15.46:0/1091280105 shutdown_connections 2024-03-29T08:24:25.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 -- 172.21.15.46:0/1091280105 wait complete. 2024-03-29T08:24:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 Processor -- start 2024-03-29T08:24:25.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.685+0000 7fa34beeb080 1 -- start start 2024-03-29T08:24:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa34beeb080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5590bd67a200 0x5590bd66e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa34beeb080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5590bd683a50 0x5590bd67bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa34beeb080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa330003a80 0x5590bd67c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa34beeb080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590bd66c050 con 0x7fa330003a80 2024-03-29T08:24:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa34beeb080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590bd66c2f0 con 0x5590bd683a50 2024-03-29T08:24:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa34beeb080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590bd66c590 con 0x5590bd67a200 2024-03-29T08:24:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa349c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa330003a80 0x5590bd67c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa349c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa330003a80 0x5590bd67c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58940/0 (socket says 172.21.15.46:58940) 2024-03-29T08:24:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa349c00640 1 -- 172.21.15.46:0/164795156 learned_addr learned my addr 172.21.15.46:0/164795156 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa349200640 1 --2- 172.21.15.46:0/164795156 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5590bd67a200 0x5590bd66e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:25.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa343e00640 1 --2- 172.21.15.46:0/164795156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5590bd683a50 0x5590bd67bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa349c00640 1 -- 172.21.15.46:0/164795156 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5590bd67a200 msgr2=0x5590bd66e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa349c00640 1 --2- 172.21.15.46:0/164795156 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5590bd67a200 0x5590bd66e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa349c00640 1 -- 172.21.15.46:0/164795156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5590bd683a50 msgr2=0x5590bd67bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa349c00640 1 --2- 172.21.15.46:0/164795156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5590bd683a50 0x5590bd67bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa349c00640 1 -- 172.21.15.46:0/164795156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa334009420 con 0x7fa330003a80 2024-03-29T08:24:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa349c00640 1 --2- 172.21.15.46:0/164795156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa330003a80 0x5590bd67c1b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa34400a7c0 tx=0x7fa344005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa341600640 1 -- 172.21.15.46:0/164795156 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa344005390 con 0x7fa330003a80 2024-03-29T08:24:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa341600640 1 -- 172.21.15.46:0/164795156 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa344005510 con 0x7fa330003a80 2024-03-29T08:24:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa341600640 1 -- 172.21.15.46:0/164795156 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3440a8740 con 0x7fa330003a80 2024-03-29T08:24:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa34beeb080 1 -- 172.21.15.46:0/164795156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5590bd4f0d00 con 0x7fa330003a80 2024-03-29T08:24:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa34beeb080 1 -- 172.21.15.46:0/164795156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590bd4f0760 con 0x7fa330003a80 2024-03-29T08:24:25.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.689+0000 7fa341600640 1 -- 172.21.15.46:0/164795156 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa3440a88a0 con 0x7fa330003a80 2024-03-29T08:24:25.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa341600640 1 --2- 172.21.15.46:0/164795156 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa328065070 0x7fa328067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:25.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa341600640 1 -- 172.21.15.46:0/164795156 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa34411dc40 con 0x7fa330003a80 2024-03-29T08:24:25.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa34beeb080 1 --2- 172.21.15.46:0/164795156 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5590bd66ccd0 0x5590bd66d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:25.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa34beeb080 1 -- 172.21.15.46:0/164795156 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80) v8 -- 0x5590bd66d5f0 con 0x5590bd66ccd0 2024-03-29T08:24:25.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa343e00640 1 --2- 172.21.15.46:0/164795156 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5590bd66ccd0 0x5590bd66d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:25.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa349200640 1 --2- 172.21.15.46:0/164795156 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa328065070 0x7fa328067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:25.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa343e00640 1 --2- 172.21.15.46:0/164795156 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5590bd66ccd0 0x5590bd66d0b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:25.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa349200640 1 --2- 172.21.15.46:0/164795156 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa328065070 0x7fa328067530 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa33c000c10 tx=0x7fa33c010040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:25.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa343e00640 1 -- 172.21.15.46:0/164795156 <== osd.1 v2:172.21.15.46:6824/27427 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v80'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa33401b020 con 0x5590bd66ccd0 2024-03-29T08:24:25.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa34beeb080 1 -- 172.21.15.46:0/164795156 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5590bd66ccd0 msgr2=0x5590bd66d0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:25.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa34beeb080 1 --2- 172.21.15.46:0/164795156 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5590bd66ccd0 0x5590bd66d0b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:25.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa34beeb080 1 -- 172.21.15.46:0/164795156 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa328065070 msgr2=0x7fa328067530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:25.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa34beeb080 1 --2- 172.21.15.46:0/164795156 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa328065070 0x7fa328067530 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa33c000c10 tx=0x7fa33c010040 comp rx=0 tx=0).stop 2024-03-29T08:24:25.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa34beeb080 1 -- 172.21.15.46:0/164795156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa330003a80 msgr2=0x5590bd67c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:25.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa34beeb080 1 --2- 172.21.15.46:0/164795156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa330003a80 0x5590bd67c1b0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa34400a7c0 tx=0x7fa344005d90 comp rx=0 tx=0).stop 2024-03-29T08:24:25.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa349c00640 1 -- 172.21.15.46:0/164795156 reap_dead start 2024-03-29T08:24:25.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa34beeb080 1 -- 172.21.15.46:0/164795156 shutdown_connections 2024-03-29T08:24:25.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa34beeb080 1 -- 172.21.15.46:0/164795156 >> 172.21.15.46:0/164795156 conn(0x5590bd5ad550 msgr2=0x5590bd5da440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:25.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa34beeb080 1 -- 172.21.15.46:0/164795156 shutdown_connections 2024-03-29T08:24:25.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:25.693+0000 7fa34beeb080 1 -- 172.21.15.46:0/164795156 wait complete. 2024-03-29T08:24:25.699 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T08:24:25.699 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:24:25.778 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:24:25.852 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T08:24:25.852 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:24:25.852 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:24:25.861 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:24:26.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.045+0000 7f2f2a800640 1 Processor -- start 2024-03-29T08:24:26.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.045+0000 7f2f2a800640 1 -- start start 2024-03-29T08:24:26.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.045+0000 7f2f2a800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2f24144210 0x7f2f240cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:26.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.045+0000 7f2f2a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f240ceb20 0x7f2f2406c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:26.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.045+0000 7f2f2a800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f240cb510 con 0x7f2f2415ada0 2024-03-29T08:24:26.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.045+0000 7f2f2a800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f24070580 con 0x7f2f240ceb20 2024-03-29T08:24:26.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.045+0000 7f2f2a800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f24070860 con 0x7f2f24144210 2024-03-29T08:24:26.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.045+0000 7f2f23e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f240ceb20 0x7f2f2406c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:26.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.045+0000 7f2f23e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f240ceb20 0x7f2f2406c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58954/0 (socket says 172.21.15.46:58954) 2024-03-29T08:24:26.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.045+0000 7f2f23e00640 1 -- 172.21.15.46:0/2159218054 learned_addr learned my addr 172.21.15.46:0/2159218054 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:26.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.045+0000 7f2f23400640 1 --2- 172.21.15.46:0/2159218054 >> v2:172.21.15.73:3300/0 conn(0x7f2f24144210 0x7f2f240cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:26.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.045+0000 7f2f23e00640 1 -- 172.21.15.46:0/2159218054 >> v2:172.21.15.73:3300/0 conn(0x7f2f24144210 msgr2=0x7f2f240cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:26.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.045+0000 7f2f23e00640 1 --2- 172.21.15.46:0/2159218054 >> v2:172.21.15.73:3300/0 conn(0x7f2f24144210 0x7f2f240cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.045+0000 7f2f23e00640 1 -- 172.21.15.46:0/2159218054 >> v1:172.21.15.73:6789/0 conn(0x7f2f2415ada0 legacy=0x7f2f240ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:26.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.045+0000 7f2f23e00640 1 -- 172.21.15.46:0/2159218054 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f24070b40 con 0x7f2f240ceb20 2024-03-29T08:24:26.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.045+0000 7f2f23400640 1 --2- 172.21.15.46:0/2159218054 >> v2:172.21.15.73:3300/0 conn(0x7f2f24144210 0x7f2f240cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:24:26.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.049+0000 7f2f23e00640 1 --2- 172.21.15.46:0/2159218054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f240ceb20 0x7f2f2406c950 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2f24071360 tx=0x7f2f1c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d30ff53e305219 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:26.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.049+0000 7f2f22a00640 1 -- 172.21.15.46:0/2159218054 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c0a5780 con 0x7f2f240ceb20 2024-03-29T08:24:26.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.049+0000 7f2f22a00640 1 -- 172.21.15.46:0/2159218054 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c0a58e0 con 0x7f2f240ceb20 2024-03-29T08:24:26.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.049+0000 7f2f22a00640 1 -- 172.21.15.46:0/2159218054 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c0a5c70 con 0x7f2f240ceb20 2024-03-29T08:24:26.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.049+0000 7f2f2a800640 1 -- 172.21.15.46:0/2159218054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f240ceb20 msgr2=0x7f2f2406c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.049+0000 7f2f2a800640 1 --2- 172.21.15.46:0/2159218054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f240ceb20 0x7f2f2406c950 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2f24071360 tx=0x7f2f1c01c9b0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.049+0000 7f2f2a800640 1 -- 172.21.15.46:0/2159218054 shutdown_connections 2024-03-29T08:24:26.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.049+0000 7f2f2a800640 1 --2- 172.21.15.46:0/2159218054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f240ceb20 0x7f2f2406c950 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.049+0000 7f2f2a800640 1 --2- 172.21.15.46:0/2159218054 >> v2:172.21.15.73:3300/0 conn(0x7f2f24144210 0x7f2f240cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.049+0000 7f2f2a800640 1 -- 172.21.15.46:0/2159218054 >> 172.21.15.46:0/2159218054 conn(0x7f2f24066110 msgr2=0x7f2f24069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:26.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.049+0000 7f2f2a800640 1 -- 172.21.15.46:0/2159218054 shutdown_connections 2024-03-29T08:24:26.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.049+0000 7f2f2a800640 1 -- 172.21.15.46:0/2159218054 wait complete. 2024-03-29T08:24:26.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f2a800640 1 Processor -- start 2024-03-29T08:24:26.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f2a800640 1 -- start start 2024-03-29T08:24:26.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f2a800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f240ceb20 0x7f2f2413acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:26.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f2a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f24144210 0x7f2f241f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:26.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f2a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f2415ada0 0x7f2f241f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:26.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f2a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f240aff20 con 0x7f2f24144210 2024-03-29T08:24:26.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f2a800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f24140430 con 0x7f2f2415ada0 2024-03-29T08:24:26.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f2a800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f240afc40 con 0x7f2f240ceb20 2024-03-29T08:24:26.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f1be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f24144210 0x7f2f241f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:26.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f1be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f24144210 0x7f2f241f6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58956/0 (socket says 172.21.15.46:58956) 2024-03-29T08:24:26.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f1be00640 1 -- 172.21.15.46:0/2298546486 learned_addr learned my addr 172.21.15.46:0/2298546486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:26.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f23400640 1 --2- 172.21.15.46:0/2298546486 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f240ceb20 0x7f2f2413acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:26.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f23e00640 1 --2- 172.21.15.46:0/2298546486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f2415ada0 0x7f2f241f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:26.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f1be00640 1 -- 172.21.15.46:0/2298546486 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f240ceb20 msgr2=0x7f2f2413acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f1be00640 1 --2- 172.21.15.46:0/2298546486 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f240ceb20 0x7f2f2413acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f1be00640 1 -- 172.21.15.46:0/2298546486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f2415ada0 msgr2=0x7f2f241f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f1be00640 1 --2- 172.21.15.46:0/2298546486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f2415ada0 0x7f2f241f6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f1be00640 1 -- 172.21.15.46:0/2298546486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c009b30 con 0x7f2f24144210 2024-03-29T08:24:26.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f1be00640 1 --2- 172.21.15.46:0/2298546486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f24144210 0x7f2f241f6ab0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00b410 tx=0x7f2f0c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:26.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f1b400640 1 -- 172.21.15.46:0/2298546486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c004020 con 0x7f2f24144210 2024-03-29T08:24:26.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f1b400640 1 -- 172.21.15.46:0/2298546486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0c004180 con 0x7f2f24144210 2024-03-29T08:24:26.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f1b400640 1 -- 172.21.15.46:0/2298546486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c011630 con 0x7f2f24144210 2024-03-29T08:24:26.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f2a800640 1 -- 172.21.15.46:0/2298546486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2413fde0 con 0x7f2f24144210 2024-03-29T08:24:26.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f2a800640 1 -- 172.21.15.46:0/2298546486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2413c310 con 0x7f2f24144210 2024-03-29T08:24:26.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.053+0000 7f2f2a800640 1 -- 172.21.15.46:0/2298546486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f240ce280 con 0x7f2f24144210 2024-03-29T08:24:26.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.057+0000 7f2f1b400640 1 -- 172.21.15.46:0/2298546486 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2f0c011790 con 0x7f2f24144210 2024-03-29T08:24:26.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.057+0000 7f2f1b400640 1 --2- 172.21.15.46:0/2298546486 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f00066690 0x7f2f00068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:26.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.057+0000 7f2f1b400640 1 -- 172.21.15.46:0/2298546486 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2f0c083e00 con 0x7f2f24144210 2024-03-29T08:24:26.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.057+0000 7f2f23400640 1 --2- 172.21.15.46:0/2298546486 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f00066690 0x7f2f00068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:26.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.057+0000 7f2f23400640 1 --2- 172.21.15.46:0/2298546486 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f00066690 0x7f2f00068b50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400ab60 tx=0x7f2f14008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:26.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.061+0000 7f2f1b400640 1 -- 172.21.15.46:0/2298546486 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2f0c051180 con 0x7f2f24144210 2024-03-29T08:24:26.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.221+0000 7f2f2a800640 1 -- 172.21.15.46:0/2298546486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f2413fc40 con 0x7f2f24144210 2024-03-29T08:24:26.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.225+0000 7f2f1b400640 1 -- 172.21.15.46:0/2298546486 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16213 (secure 0 0 0) 0x7f2f2413fc40 con 0x7f2f24144210 2024-03-29T08:24:26.224 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:26.224 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:23:02.390005+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:22:42.147364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-29T08:22:42.602168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1f","mappings":[{"from":2,"to":7}]},{"pgid":"11.27","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/9617787":"2024-03-30T08:23:02.389981+0000","172.21.15.46:6834/9617787":"2024-03-30T08:23:02.389981+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:24:26.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.225+0000 7f2f2a800640 1 -- 172.21.15.46:0/2298546486 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f00066690 msgr2=0x7f2f00068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.225+0000 7f2f2a800640 1 --2- 172.21.15.46:0/2298546486 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f00066690 0x7f2f00068b50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400ab60 tx=0x7f2f14008040 comp rx=0 tx=0).stop 2024-03-29T08:24:26.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.225+0000 7f2f2a800640 1 -- 172.21.15.46:0/2298546486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f24144210 msgr2=0x7f2f241f6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.225+0000 7f2f2a800640 1 --2- 172.21.15.46:0/2298546486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f24144210 0x7f2f241f6ab0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00b410 tx=0x7f2f0c00b8d0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.225+0000 7f2f2a800640 1 -- 172.21.15.46:0/2298546486 shutdown_connections 2024-03-29T08:24:26.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.225+0000 7f2f2a800640 1 --2- 172.21.15.46:0/2298546486 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f00066690 0x7f2f00068b50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.225+0000 7f2f2a800640 1 --2- 172.21.15.46:0/2298546486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f2415ada0 0x7f2f241f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.225+0000 7f2f2a800640 1 --2- 172.21.15.46:0/2298546486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f24144210 0x7f2f241f6ab0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.225+0000 7f2f2a800640 1 --2- 172.21.15.46:0/2298546486 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f240ceb20 0x7f2f2413acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.225+0000 7f2f2a800640 1 -- 172.21.15.46:0/2298546486 >> 172.21.15.46:0/2298546486 conn(0x7f2f24066110 msgr2=0x7f2f240ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:26.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.225+0000 7f2f2a800640 1 -- 172.21.15.46:0/2298546486 shutdown_connections 2024-03-29T08:24:26.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.225+0000 7f2f2a800640 1 -- 172.21.15.46:0/2298546486 wait complete. 2024-03-29T08:24:26.245 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:22:42.147364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:22:42.602168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:24:26.245 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:24:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.389+0000 7fc50aa00640 1 Processor -- start 2024-03-29T08:24:26.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.389+0000 7fc50aa00640 1 -- start start 2024-03-29T08:24:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.389+0000 7fc50aa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc504143eb0 0x7fc5040cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.389+0000 7fc50aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc50415efe0 0x7fc5040cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.389+0000 7fc50aa00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5040b2360 con 0x7fc5040b1630 2024-03-29T08:24:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.389+0000 7fc50aa00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc50415e500 con 0x7fc50415efe0 2024-03-29T08:24:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.389+0000 7fc50aa00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc50415e7e0 con 0x7fc504143eb0 2024-03-29T08:24:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.389+0000 7fc508a00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc5040b1630 0x7fc5040b1a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:41178/0 (socket says 172.21.15.46:41178) 2024-03-29T08:24:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.389+0000 7fc508a00640 1 -- 172.21.15.46:0/177918908 learned_addr learned my addr 172.21.15.46:0/177918908 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.389+0000 7fc503e00640 1 --2- 172.21.15.46:0/177918908 >> v2:172.21.15.73:3300/0 conn(0x7fc504143eb0 0x7fc5040cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.389+0000 7fc4fb400640 1 --2- 172.21.15.46:0/177918908 >> v2:172.21.15.46:3300/0 conn(0x7fc50415efe0 0x7fc5040cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.389+0000 7fc503400640 1 -- 172.21.15.46:0/177918908 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940329334 0 0) 0x7fc5040b2360 con 0x7fc5040b1630 2024-03-29T08:24:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.389+0000 7fc503400640 1 -- 172.21.15.46:0/177918908 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4e8003530 con 0x7fc5040b1630 2024-03-29T08:24:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.389+0000 7fc4fb400640 1 -- 172.21.15.46:0/177918908 >> v2:172.21.15.73:3300/0 conn(0x7fc504143eb0 msgr2=0x7fc5040cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.389+0000 7fc4fb400640 1 --2- 172.21.15.46:0/177918908 >> v2:172.21.15.73:3300/0 conn(0x7fc504143eb0 0x7fc5040cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.389+0000 7fc4fb400640 1 -- 172.21.15.46:0/177918908 >> v1:172.21.15.73:6789/0 conn(0x7fc5040b1630 legacy=0x7fc5040b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.389+0000 7fc4fb400640 1 -- 172.21.15.46:0/177918908 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50415eac0 con 0x7fc50415efe0 2024-03-29T08:24:26.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.393+0000 7fc4fb400640 1 --2- 172.21.15.46:0/177918908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50415efe0 0x7fc5040cbcd0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc5040b0b80 tx=0x7fc4ec01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b5558cf1353d0c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:26.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.393+0000 7fc503400640 1 -- 172.21.15.46:0/177918908 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec0a8050 con 0x7fc50415efe0 2024-03-29T08:24:26.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.393+0000 7fc503400640 1 -- 172.21.15.46:0/177918908 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ec0ac5f0 con 0x7fc50415efe0 2024-03-29T08:24:26.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.393+0000 7fc503400640 1 -- 172.21.15.46:0/177918908 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec0ac980 con 0x7fc50415efe0 2024-03-29T08:24:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.393+0000 7fc50aa00640 1 -- 172.21.15.46:0/177918908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50415efe0 msgr2=0x7fc5040cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.393+0000 7fc50aa00640 1 --2- 172.21.15.46:0/177918908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50415efe0 0x7fc5040cbcd0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fc5040b0b80 tx=0x7fc4ec01c480 comp rx=0 tx=0).stop 2024-03-29T08:24:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.393+0000 7fc50aa00640 1 -- 172.21.15.46:0/177918908 shutdown_connections 2024-03-29T08:24:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.393+0000 7fc50aa00640 1 --2- 172.21.15.46:0/177918908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc50415efe0 0x7fc5040cbcd0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.393+0000 7fc50aa00640 1 --2- 172.21.15.46:0/177918908 >> v2:172.21.15.73:3300/0 conn(0x7fc504143eb0 0x7fc5040cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.393+0000 7fc50aa00640 1 -- 172.21.15.46:0/177918908 >> 172.21.15.46:0/177918908 conn(0x7fc50406a390 msgr2=0x7fc5040cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.393+0000 7fc50aa00640 1 -- 172.21.15.46:0/177918908 shutdown_connections 2024-03-29T08:24:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.393+0000 7fc50aa00640 1 -- 172.21.15.46:0/177918908 wait complete. 2024-03-29T08:24:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc50aa00640 1 Processor -- start 2024-03-29T08:24:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc50aa00640 1 -- start start 2024-03-29T08:24:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc50aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5040b1630 0x7fc504163a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc50aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc504143eb0 0x7fc504142510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc50aa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc50415efe0 0x7fc504142a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc50aa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc504141a00 con 0x7fc5040b1630 2024-03-29T08:24:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc50aa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc504142f90 con 0x7fc504143eb0 2024-03-29T08:24:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc50aa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc504140df0 con 0x7fc50415efe0 2024-03-29T08:24:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc503e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5040b1630 0x7fc504163a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc503e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5040b1630 0x7fc504163a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58982/0 (socket says 172.21.15.46:58982) 2024-03-29T08:24:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc503e00640 1 -- 172.21.15.46:0/580457414 learned_addr learned my addr 172.21.15.46:0/580457414 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc4fb400640 1 --2- 172.21.15.46:0/580457414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc504143eb0 0x7fc504142510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc508a00640 1 --2- 172.21.15.46:0/580457414 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc50415efe0 0x7fc504142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc503e00640 1 -- 172.21.15.46:0/580457414 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc50415efe0 msgr2=0x7fc504142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc503e00640 1 --2- 172.21.15.46:0/580457414 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc50415efe0 0x7fc504142a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc503e00640 1 -- 172.21.15.46:0/580457414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc504143eb0 msgr2=0x7fc504142510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc503e00640 1 --2- 172.21.15.46:0/580457414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc504143eb0 0x7fc504142510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc503e00640 1 -- 172.21.15.46:0/580457414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ec009420 con 0x7fc5040b1630 2024-03-29T08:24:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc4fb400640 1 --2- 172.21.15.46:0/580457414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc504143eb0 0x7fc504142510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc503e00640 1 --2- 172.21.15.46:0/580457414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5040b1630 0x7fc504163a80 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400e930 tx=0x7fc4f400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc500c00640 1 -- 172.21.15.46:0/580457414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4009690 con 0x7fc5040b1630 2024-03-29T08:24:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc500c00640 1 -- 172.21.15.46:0/580457414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f400ca00 con 0x7fc5040b1630 2024-03-29T08:24:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc500c00640 1 -- 172.21.15.46:0/580457414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f400cd70 con 0x7fc5040b1630 2024-03-29T08:24:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc50aa00640 1 -- 172.21.15.46:0/580457414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5041410d0 con 0x7fc5040b1630 2024-03-29T08:24:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc50aa00640 1 -- 172.21.15.46:0/580457414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc504141690 con 0x7fc5040b1630 2024-03-29T08:24:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.397+0000 7fc50aa00640 1 -- 172.21.15.46:0/580457414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc504071a70 con 0x7fc5040b1630 2024-03-29T08:24:26.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.401+0000 7fc500c00640 1 -- 172.21.15.46:0/580457414 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc4f4013470 con 0x7fc5040b1630 2024-03-29T08:24:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.401+0000 7fc500c00640 1 --2- 172.21.15.46:0/580457414 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc4e0066690 0x7fc4e0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.401+0000 7fc500c00640 1 -- 172.21.15.46:0/580457414 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc4f40844e0 con 0x7fc5040b1630 2024-03-29T08:24:26.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.401+0000 7fc4fb400640 1 --2- 172.21.15.46:0/580457414 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc4e0066690 0x7fc4e0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:26.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.401+0000 7fc4fb400640 1 --2- 172.21.15.46:0/580457414 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc4e0066690 0x7fc4e0068b50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec0ae000 tx=0x7fc4ec0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:26.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.405+0000 7fc500c00640 1 -- 172.21.15.46:0/580457414 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc4f4051870 con 0x7fc5040b1630 2024-03-29T08:24:26.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.565+0000 7fc50aa00640 1 -- 172.21.15.46:0/580457414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc50404e120 con 0x7fc5040b1630 2024-03-29T08:24:26.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.565+0000 7fc500c00640 1 -- 172.21.15.46:0/580457414 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4f4054e90 con 0x7fc5040b1630 2024-03-29T08:24:26.566 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:24:26.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.565+0000 7fc50aa00640 1 -- 172.21.15.46:0/580457414 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc4e0066690 msgr2=0x7fc4e0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.565+0000 7fc50aa00640 1 --2- 172.21.15.46:0/580457414 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc4e0066690 0x7fc4e0068b50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec0ae000 tx=0x7fc4ec0a7000 comp rx=0 tx=0).stop 2024-03-29T08:24:26.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.565+0000 7fc50aa00640 1 -- 172.21.15.46:0/580457414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5040b1630 msgr2=0x7fc504163a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.565+0000 7fc50aa00640 1 --2- 172.21.15.46:0/580457414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5040b1630 0x7fc504163a80 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fc4f400e930 tx=0x7fc4f400edf0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.569+0000 7fc50aa00640 1 -- 172.21.15.46:0/580457414 shutdown_connections 2024-03-29T08:24:26.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.569+0000 7fc50aa00640 1 --2- 172.21.15.46:0/580457414 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc4e0066690 0x7fc4e0068b50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.569+0000 7fc50aa00640 1 --2- 172.21.15.46:0/580457414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc504143eb0 0x7fc504142510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.569+0000 7fc50aa00640 1 --2- 172.21.15.46:0/580457414 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc50415efe0 0x7fc504142a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.569+0000 7fc50aa00640 1 --2- 172.21.15.46:0/580457414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5040b1630 0x7fc504163a80 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.569+0000 7fc50aa00640 1 -- 172.21.15.46:0/580457414 >> 172.21.15.46:0/580457414 conn(0x7fc50406a390 msgr2=0x7fc5040b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:26.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.569+0000 7fc50aa00640 1 -- 172.21.15.46:0/580457414 shutdown_connections 2024-03-29T08:24:26.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.569+0000 7fc50aa00640 1 -- 172.21.15.46:0/580457414 wait complete. 2024-03-29T08:24:26.587 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:24:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.729+0000 7f0627a00640 1 Processor -- start 2024-03-29T08:24:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.729+0000 7f0627a00640 1 -- start start 2024-03-29T08:24:26.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.729+0000 7f0627a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f062015c950 0x7f062006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:26.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.729+0000 7f0627a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06200cfb20 con 0x7f0620160db0 2024-03-29T08:24:26.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.729+0000 7f0627a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06200cef10 con 0x7f06200cea30 2024-03-29T08:24:26.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.729+0000 7f0627a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06200cf1f0 con 0x7f062015c950 2024-03-29T08:24:26.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.729+0000 7f0625a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0620160db0 0x7f06200cad10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57736/0 (socket says 172.21.15.46:57736) 2024-03-29T08:24:26.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.729+0000 7f0625a00640 1 -- 172.21.15.46:0/4150201101 learned_addr learned my addr 172.21.15.46:0/4150201101 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.733+0000 7f0625000640 1 --2- 172.21.15.46:0/4150201101 >> v2:172.21.15.73:3300/0 conn(0x7f062015c950 0x7f062006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.733+0000 7f061f400640 1 -- 172.21.15.46:0/4150201101 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539204464 0 0) 0x7f06200cfb20 con 0x7f0620160db0 2024-03-29T08:24:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.733+0000 7f061f400640 1 -- 172.21.15.46:0/4150201101 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0608003530 con 0x7f0620160db0 2024-03-29T08:24:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.733+0000 7f061f400640 1 -- 172.21.15.46:0/4150201101 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986587437 0 0) 0x7f06200cef10 con 0x7f06200cea30 2024-03-29T08:24:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.733+0000 7f061f400640 1 -- 172.21.15.46:0/4150201101 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06200cfb20 con 0x7f06200cea30 2024-03-29T08:24:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.733+0000 7f061f400640 1 -- 172.21.15.46:0/4150201101 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1085381612 0 0) 0x7f0608003530 con 0x7f0620160db0 2024-03-29T08:24:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.733+0000 7f0625000640 1 -- 172.21.15.46:0/4150201101 >> v1:172.21.15.73:6789/0 conn(0x7f06200cea30 legacy=0x7f06201606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.733+0000 7f0625000640 1 -- 172.21.15.46:0/4150201101 >> v1:172.21.15.46:6789/0 conn(0x7f0620160db0 legacy=0x7f06200cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.733+0000 7f0625000640 1 -- 172.21.15.46:0/4150201101 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06200cf4d0 con 0x7f062015c950 2024-03-29T08:24:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.733+0000 7f0625000640 1 --2- 172.21.15.46:0/4150201101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f062015c950 0x7f062006bbe0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f06140097c0 tx=0x7f061401ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a2dd8bd472a5e41 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.733+0000 7f061f400640 1 -- 172.21.15.46:0/4150201101 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0618001a00 con 0x7f062015c950 2024-03-29T08:24:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.733+0000 7f061f400640 1 -- 172.21.15.46:0/4150201101 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06140a8630 con 0x7f062015c950 2024-03-29T08:24:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.733+0000 7f061f400640 1 -- 172.21.15.46:0/4150201101 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06140a89a0 con 0x7f062015c950 2024-03-29T08:24:26.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f0627a00640 1 -- 172.21.15.46:0/4150201101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f062015c950 msgr2=0x7f062006bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f0627a00640 1 --2- 172.21.15.46:0/4150201101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f062015c950 0x7f062006bbe0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f06140097c0 tx=0x7f061401ebb0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f0627a00640 1 -- 172.21.15.46:0/4150201101 shutdown_connections 2024-03-29T08:24:26.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f0627a00640 1 --2- 172.21.15.46:0/4150201101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f062015c950 0x7f062006bbe0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f0627a00640 1 -- 172.21.15.46:0/4150201101 >> 172.21.15.46:0/4150201101 conn(0x7f06200b21b0 msgr2=0x7f06200b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:26.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f0627a00640 1 -- 172.21.15.46:0/4150201101 shutdown_connections 2024-03-29T08:24:26.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f0627a00640 1 -- 172.21.15.46:0/4150201101 wait complete. 2024-03-29T08:24:26.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f0627a00640 1 Processor -- start 2024-03-29T08:24:26.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f0627a00640 1 -- start start 2024-03-29T08:24:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f0627a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f06200cea30 0x7f0620156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f0627a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f062015c950 0x7f06201e6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f0627a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0620160db0 0x7f06200b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f0627a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06201e8060 con 0x7f062015c950 2024-03-29T08:24:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f0627a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06201f2670 con 0x7f0620160db0 2024-03-29T08:24:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f0627a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06201e77c0 con 0x7f06200cea30 2024-03-29T08:24:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f061fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f062015c950 0x7f06201e6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f061fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f062015c950 0x7f06201e6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48818/0 (socket says 172.21.15.46:48818) 2024-03-29T08:24:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f061fe00640 1 -- 172.21.15.46:0/1682828516 learned_addr learned my addr 172.21.15.46:0/1682828516 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:26.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f0625a00640 1 --2- 172.21.15.46:0/1682828516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0620160db0 0x7f06200b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:26.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f0625000640 1 --2- 172.21.15.46:0/1682828516 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f06200cea30 0x7f0620156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:26.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f061fe00640 1 -- 172.21.15.46:0/1682828516 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f06200cea30 msgr2=0x7f0620156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f061fe00640 1 --2- 172.21.15.46:0/1682828516 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f06200cea30 0x7f0620156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f061fe00640 1 -- 172.21.15.46:0/1682828516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0620160db0 msgr2=0x7f06200b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.737+0000 7f061fe00640 1 --2- 172.21.15.46:0/1682828516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0620160db0 0x7f06200b4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.741+0000 7f061fe00640 1 -- 172.21.15.46:0/1682828516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0614009420 con 0x7f062015c950 2024-03-29T08:24:26.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.741+0000 7f061fe00640 1 --2- 172.21.15.46:0/1682828516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f062015c950 0x7f06201e6de0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f060c00c410 tx=0x7f060c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:26.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.741+0000 7f061cc00640 1 -- 172.21.15.46:0/1682828516 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c0052d0 con 0x7f062015c950 2024-03-29T08:24:26.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.741+0000 7f061cc00640 1 -- 172.21.15.46:0/1682828516 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060c005450 con 0x7f062015c950 2024-03-29T08:24:26.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.741+0000 7f061cc00640 1 -- 172.21.15.46:0/1682828516 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060c003840 con 0x7f062015c950 2024-03-29T08:24:26.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.741+0000 7f0627a00640 1 -- 172.21.15.46:0/1682828516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06201e7aa0 con 0x7f062015c950 2024-03-29T08:24:26.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.741+0000 7f0627a00640 1 -- 172.21.15.46:0/1682828516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06201f2880 con 0x7f062015c950 2024-03-29T08:24:26.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.741+0000 7f0627a00640 1 -- 172.21.15.46:0/1682828516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062009e690 con 0x7f062015c950 2024-03-29T08:24:26.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.741+0000 7f061cc00640 1 -- 172.21.15.46:0/1682828516 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f060c0039a0 con 0x7f062015c950 2024-03-29T08:24:26.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.741+0000 7f061cc00640 1 --2- 172.21.15.46:0/1682828516 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0600066690 0x7f0600068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:26.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.741+0000 7f061cc00640 1 -- 172.21.15.46:0/1682828516 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f060c0838a0 con 0x7f062015c950 2024-03-29T08:24:26.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.745+0000 7f0625000640 1 --2- 172.21.15.46:0/1682828516 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0600066690 0x7f0600068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:26.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.745+0000 7f0625000640 1 --2- 172.21.15.46:0/1682828516 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0600066690 0x7f0600068b50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f06140b2000 tx=0x7f06140aa070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:26.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.749+0000 7f061cc00640 1 -- 172.21.15.46:0/1682828516 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f060c050b90 con 0x7f062015c950 2024-03-29T08:24:26.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.905+0000 7f0627a00640 1 -- 172.21.15.46:0/1682828516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f062006c600 con 0x7f062015c950 2024-03-29T08:24:26.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.909+0000 7f061cc00640 1 -- 172.21.15.46:0/1682828516 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f060c017020 con 0x7f062015c950 2024-03-29T08:24:26.908 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:24:26.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.909+0000 7f0627a00640 1 -- 172.21.15.46:0/1682828516 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0600066690 msgr2=0x7f0600068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.909+0000 7f0627a00640 1 --2- 172.21.15.46:0/1682828516 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0600066690 0x7f0600068b50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f06140b2000 tx=0x7f06140aa070 comp rx=0 tx=0).stop 2024-03-29T08:24:26.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.909+0000 7f0627a00640 1 -- 172.21.15.46:0/1682828516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f062015c950 msgr2=0x7f06201e6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:26.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.909+0000 7f0627a00640 1 --2- 172.21.15.46:0/1682828516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f062015c950 0x7f06201e6de0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f060c00c410 tx=0x7f060c00c8d0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.909+0000 7f0627a00640 1 -- 172.21.15.46:0/1682828516 shutdown_connections 2024-03-29T08:24:26.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.909+0000 7f0627a00640 1 --2- 172.21.15.46:0/1682828516 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0600066690 0x7f0600068b50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.909+0000 7f0627a00640 1 --2- 172.21.15.46:0/1682828516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0620160db0 0x7f06200b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.909+0000 7f0627a00640 1 --2- 172.21.15.46:0/1682828516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f062015c950 0x7f06201e6de0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.909+0000 7f0627a00640 1 --2- 172.21.15.46:0/1682828516 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f06200cea30 0x7f0620156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:26.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.909+0000 7f0627a00640 1 -- 172.21.15.46:0/1682828516 >> 172.21.15.46:0/1682828516 conn(0x7f06200b21b0 msgr2=0x7f062015d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:26.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.909+0000 7f0627a00640 1 -- 172.21.15.46:0/1682828516 shutdown_connections 2024-03-29T08:24:26.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:26.909+0000 7f0627a00640 1 -- 172.21.15.46:0/1682828516 wait complete. 2024-03-29T08:24:26.929 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:24:27.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.073+0000 7fa727800640 1 Processor -- start 2024-03-29T08:24:27.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.073+0000 7fa727800640 1 -- start start 2024-03-29T08:24:27.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.073+0000 7fa727800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa72015df90 0x7fa7200ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:27.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.073+0000 7fa727800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa7200cbea0 0x7fa7200ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:27.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.073+0000 7fa727800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72015d260 con 0x7fa720142a50 2024-03-29T08:24:27.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.073+0000 7fa727800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7200aed00 con 0x7fa72015df90 2024-03-29T08:24:27.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.073+0000 7fa727800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7200aefd0 con 0x7fa7200cbea0 2024-03-29T08:24:27.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.073+0000 7fa717e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa72015df90 0x7fa7200ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:27.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.073+0000 7fa717e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa72015df90 0x7fa7200ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48824/0 (socket says 172.21.15.46:48824) 2024-03-29T08:24:27.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.073+0000 7fa717e00640 1 -- 172.21.15.46:0/4201316093 learned_addr learned my addr 172.21.15.46:0/4201316093 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:27.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.073+0000 7fa725800640 1 --2- 172.21.15.46:0/4201316093 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa7200cbea0 0x7fa7200ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:27.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.073+0000 7fa717e00640 1 -- 172.21.15.46:0/4201316093 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa7200cbea0 msgr2=0x7fa7200ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:27.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.073+0000 7fa717e00640 1 --2- 172.21.15.46:0/4201316093 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa7200cbea0 0x7fa7200ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.073+0000 7fa717e00640 1 -- 172.21.15.46:0/4201316093 >> v1:172.21.15.73:6789/0 conn(0x7fa720142a50 legacy=0x7fa7200cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:27.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.077+0000 7fa717e00640 1 -- 172.21.15.46:0/4201316093 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72015d4b0 con 0x7fa72015df90 2024-03-29T08:24:27.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.077+0000 7fa717e00640 1 --2- 172.21.15.46:0/4201316093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa72015df90 0x7fa7200ae280 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fa708009870 tx=0x7fa70801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef3314124828be6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:27.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.077+0000 7fa71fe00640 1 -- 172.21.15.46:0/4201316093 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080ab600 con 0x7fa72015df90 2024-03-29T08:24:27.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.077+0000 7fa71fe00640 1 -- 172.21.15.46:0/4201316093 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7080a7050 con 0x7fa72015df90 2024-03-29T08:24:27.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.077+0000 7fa71fe00640 1 -- 172.21.15.46:0/4201316093 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080ab980 con 0x7fa72015df90 2024-03-29T08:24:27.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.077+0000 7fa727800640 1 -- 172.21.15.46:0/4201316093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa72015df90 msgr2=0x7fa7200ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:27.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.077+0000 7fa727800640 1 --2- 172.21.15.46:0/4201316093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa72015df90 0x7fa7200ae280 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fa708009870 tx=0x7fa70801c4d0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.077+0000 7fa727800640 1 -- 172.21.15.46:0/4201316093 shutdown_connections 2024-03-29T08:24:27.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.077+0000 7fa727800640 1 --2- 172.21.15.46:0/4201316093 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa7200cbea0 0x7fa7200ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.077+0000 7fa727800640 1 --2- 172.21.15.46:0/4201316093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa72015df90 0x7fa7200ae280 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.077+0000 7fa727800640 1 -- 172.21.15.46:0/4201316093 >> 172.21.15.46:0/4201316093 conn(0x7fa72006a390 msgr2=0x7fa7200cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:27.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.077+0000 7fa727800640 1 -- 172.21.15.46:0/4201316093 shutdown_connections 2024-03-29T08:24:27.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.077+0000 7fa727800640 1 -- 172.21.15.46:0/4201316093 wait complete. 2024-03-29T08:24:27.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa727800640 1 Processor -- start 2024-03-29T08:24:27.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa727800640 1 -- start start 2024-03-29T08:24:27.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa727800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7200cbea0 0x7fa720139ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:27.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa727800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa720142a50 0x7fa720159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:27.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa727800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa72015df90 0x7fa72006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:27.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa727800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa720163130 con 0x7fa72015df90 2024-03-29T08:24:27.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa727800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa720162840 con 0x7fa7200cbea0 2024-03-29T08:24:27.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa727800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa720162ae0 con 0x7fa720142a50 2024-03-29T08:24:27.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa725800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa72015df90 0x7fa72006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:27.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa725800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa72015df90 0x7fa72006c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48832/0 (socket says 172.21.15.46:48832) 2024-03-29T08:24:27.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa725800640 1 -- 172.21.15.46:0/3780351516 learned_addr learned my addr 172.21.15.46:0/3780351516 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:27.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa717e00640 1 --2- 172.21.15.46:0/3780351516 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa720142a50 0x7fa720159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:27.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa724e00640 1 --2- 172.21.15.46:0/3780351516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7200cbea0 0x7fa720139ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:27.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa725800640 1 -- 172.21.15.46:0/3780351516 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa720142a50 msgr2=0x7fa720159ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:27.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa725800640 1 --2- 172.21.15.46:0/3780351516 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa720142a50 0x7fa720159ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa725800640 1 -- 172.21.15.46:0/3780351516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7200cbea0 msgr2=0x7fa720139ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:27.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa725800640 1 --2- 172.21.15.46:0/3780351516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7200cbea0 0x7fa720139ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa725800640 1 -- 172.21.15.46:0/3780351516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7080094d0 con 0x7fa72015df90 2024-03-29T08:24:27.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa725800640 1 --2- 172.21.15.46:0/3780351516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa72015df90 0x7fa72006c370 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fa718007de0 tx=0x7fa71800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:27.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa71d600640 1 -- 172.21.15.46:0/3780351516 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718014620 con 0x7fa72015df90 2024-03-29T08:24:27.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa71d600640 1 -- 172.21.15.46:0/3780351516 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71800ee90 con 0x7fa72015df90 2024-03-29T08:24:27.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa71d600640 1 -- 172.21.15.46:0/3780351516 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7180149e0 con 0x7fa72015df90 2024-03-29T08:24:27.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa727800640 1 -- 172.21.15.46:0/3780351516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa720162dc0 con 0x7fa72015df90 2024-03-29T08:24:27.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.081+0000 7fa727800640 1 -- 172.21.15.46:0/3780351516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7201635c0 con 0x7fa72015df90 2024-03-29T08:24:27.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.085+0000 7fa727800640 1 -- 172.21.15.46:0/3780351516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa720162f50 con 0x7fa72015df90 2024-03-29T08:24:27.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.085+0000 7fa71d600640 1 -- 172.21.15.46:0/3780351516 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa718014b40 con 0x7fa72015df90 2024-03-29T08:24:27.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.085+0000 7fa71d600640 1 --2- 172.21.15.46:0/3780351516 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6fc066690 0x7fa6fc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:27.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.085+0000 7fa71d600640 1 -- 172.21.15.46:0/3780351516 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa718084c40 con 0x7fa72015df90 2024-03-29T08:24:27.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.085+0000 7fa724e00640 1 --2- 172.21.15.46:0/3780351516 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6fc066690 0x7fa6fc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:27.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.085+0000 7fa724e00640 1 --2- 172.21.15.46:0/3780351516 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6fc066690 0x7fa6fc068b50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa710000f70 tx=0x7fa710009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:27.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.089+0000 7fa71d600640 1 -- 172.21.15.46:0/3780351516 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa718051f10 con 0x7fa72015df90 2024-03-29T08:24:27.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.249+0000 7fa727800640 1 -- 172.21.15.46:0/3780351516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa720071a70 con 0x7fa72015df90 2024-03-29T08:24:27.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.249+0000 7fa71d600640 1 -- 172.21.15.46:0/3780351516 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa718055530 con 0x7fa72015df90 2024-03-29T08:24:27.251 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:24:27.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.253+0000 7fa727800640 1 -- 172.21.15.46:0/3780351516 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6fc066690 msgr2=0x7fa6fc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:27.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.253+0000 7fa727800640 1 --2- 172.21.15.46:0/3780351516 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6fc066690 0x7fa6fc068b50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa710000f70 tx=0x7fa710009040 comp rx=0 tx=0).stop 2024-03-29T08:24:27.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.253+0000 7fa727800640 1 -- 172.21.15.46:0/3780351516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa72015df90 msgr2=0x7fa72006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:27.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.253+0000 7fa727800640 1 --2- 172.21.15.46:0/3780351516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa72015df90 0x7fa72006c370 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fa718007de0 tx=0x7fa71800e4b0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.253+0000 7fa727800640 1 -- 172.21.15.46:0/3780351516 shutdown_connections 2024-03-29T08:24:27.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.253+0000 7fa727800640 1 --2- 172.21.15.46:0/3780351516 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6fc066690 0x7fa6fc068b50 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.253+0000 7fa727800640 1 --2- 172.21.15.46:0/3780351516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa72015df90 0x7fa72006c370 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.253+0000 7fa727800640 1 --2- 172.21.15.46:0/3780351516 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa720142a50 0x7fa720159ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.253+0000 7fa727800640 1 --2- 172.21.15.46:0/3780351516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7200cbea0 0x7fa720139ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.253+0000 7fa727800640 1 -- 172.21.15.46:0/3780351516 >> 172.21.15.46:0/3780351516 conn(0x7fa72006a390 msgr2=0x7fa7200cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:27.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.253+0000 7fa727800640 1 -- 172.21.15.46:0/3780351516 shutdown_connections 2024-03-29T08:24:27.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.253+0000 7fa727800640 1 -- 172.21.15.46:0/3780351516 wait complete. 2024-03-29T08:24:27.272 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:24:27.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc914000640 1 Processor -- start 2024-03-29T08:24:27.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc914000640 1 -- start start 2024-03-29T08:24:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc914000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc90c158300 0x7fc90c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc914000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc90c0af530 0x7fc90c0af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc914000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90c06c390 con 0x7fc90c0702a0 2024-03-29T08:24:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc914000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90c06d600 con 0x7fc90c0af530 2024-03-29T08:24:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc914000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0afff0 con 0x7fc90c158300 2024-03-29T08:24:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc912000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc90c0702a0 0x7fc90c15e390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57752/0 (socket says 172.21.15.46:57752) 2024-03-29T08:24:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc912000640 1 -- 172.21.15.46:0/3718031962 learned_addr learned my addr 172.21.15.46:0/3718031962 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc911600640 1 --2- 172.21.15.46:0/3718031962 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc90c158300 0x7fc90c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc910c00640 1 --2- 172.21.15.46:0/3718031962 >> v2:172.21.15.73:3300/0 conn(0x7fc90c0af530 0x7fc90c0af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc90be00640 1 -- 172.21.15.46:0/3718031962 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511865818 0 0) 0x7fc90c06c390 con 0x7fc90c0702a0 2024-03-29T08:24:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc90be00640 1 -- 172.21.15.46:0/3718031962 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8f4003530 con 0x7fc90c0702a0 2024-03-29T08:24:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc90be00640 1 -- 172.21.15.46:0/3718031962 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 695340465 0 0) 0x7fc8f4003530 con 0x7fc90c0702a0 2024-03-29T08:24:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc90be00640 1 -- 172.21.15.46:0/3718031962 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc90c158300 msgr2=0x7fc90c06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc90be00640 1 --2- 172.21.15.46:0/3718031962 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc90c158300 0x7fc90c06d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc90be00640 1 -- 172.21.15.46:0/3718031962 >> v2:172.21.15.73:3300/0 conn(0x7fc90c0af530 msgr2=0x7fc90c0af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc90be00640 1 --2- 172.21.15.46:0/3718031962 >> v2:172.21.15.73:3300/0 conn(0x7fc90c0af530 0x7fc90c0af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc90be00640 1 -- 172.21.15.46:0/3718031962 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c06c5e0 con 0x7fc90c0702a0 2024-03-29T08:24:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc90be00640 1 -- 172.21.15.46:0/3718031962 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fc904001a00 con 0x7fc90c0702a0 2024-03-29T08:24:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc90be00640 1 -- 172.21.15.46:0/3718031962 >> v1:172.21.15.46:6789/0 conn(0x7fc90c0702a0 legacy=0x7fc90c15e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.413+0000 7fc90be00640 1 --2- 172.21.15.46:0/3718031962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4003980 0x7fc8f4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.417+0000 7fc90be00640 1 --2- 172.21.15.46:0/3718031962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8f40047d0 0x7fc8f4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.417+0000 7fc90be00640 1 --2- 172.21.15.46:0/3718031962 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc90c158300 0x7fc8f4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.417+0000 7fc90be00640 1 -- 172.21.15.46:0/3718031962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c06d600 con 0x7fc8f4003980 2024-03-29T08:24:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.417+0000 7fc90be00640 1 -- 172.21.15.46:0/3718031962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0afff0 con 0x7fc8f40047d0 2024-03-29T08:24:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.417+0000 7fc90be00640 1 -- 172.21.15.46:0/3718031962 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f400ba70 con 0x7fc90c158300 2024-03-29T08:24:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.417+0000 7fc911600640 1 --2- 172.21.15.46:0/3718031962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4003980 0x7fc8f4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.417+0000 7fc910c00640 1 --2- 172.21.15.46:0/3718031962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8f40047d0 0x7fc8f4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.417+0000 7fc912000640 1 --2- 172.21.15.46:0/3718031962 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc90c158300 0x7fc8f4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.417+0000 7fc911600640 1 -- 172.21.15.46:0/3718031962 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc90c158300 msgr2=0x7fc8f4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.417+0000 7fc911600640 1 --2- 172.21.15.46:0/3718031962 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc90c158300 0x7fc8f4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.417+0000 7fc911600640 1 -- 172.21.15.46:0/3718031962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8f40047d0 msgr2=0x7fc8f4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.417+0000 7fc911600640 1 --2- 172.21.15.46:0/3718031962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8f40047d0 0x7fc8f4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.417+0000 7fc911600640 1 -- 172.21.15.46:0/3718031962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f4003530 con 0x7fc8f4003980 2024-03-29T08:24:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.417+0000 7fc911600640 1 --2- 172.21.15.46:0/3718031962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4003980 0x7fc8f4003da0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc900009ab0 tx=0x7fc90001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdff22429a4b5283 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.417+0000 7fc90be00640 1 -- 172.21.15.46:0/3718031962 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9000a64f0 con 0x7fc8f4003980 2024-03-29T08:24:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.417+0000 7fc90be00640 1 -- 172.21.15.46:0/3718031962 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9000a6650 con 0x7fc8f4003980 2024-03-29T08:24:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.417+0000 7fc90be00640 1 -- 172.21.15.46:0/3718031962 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9000a6a00 con 0x7fc8f4003980 2024-03-29T08:24:27.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc914000640 1 -- 172.21.15.46:0/3718031962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4003980 msgr2=0x7fc8f4003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:27.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc914000640 1 --2- 172.21.15.46:0/3718031962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4003980 0x7fc8f4003da0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fc900009ab0 tx=0x7fc90001ca00 comp rx=0 tx=0).stop 2024-03-29T08:24:27.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc912000640 1 -- 172.21.15.46:0/3718031962 reap_dead start 2024-03-29T08:24:27.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc914000640 1 -- 172.21.15.46:0/3718031962 shutdown_connections 2024-03-29T08:24:27.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc914000640 1 -- 172.21.15.46:0/3718031962 >> 172.21.15.46:0/3718031962 conn(0x7fc90c066080 msgr2=0x7fc90c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:27.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc914000640 1 -- 172.21.15.46:0/3718031962 shutdown_connections 2024-03-29T08:24:27.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc914000640 1 -- 172.21.15.46:0/3718031962 wait complete. 2024-03-29T08:24:27.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc914000640 1 Processor -- start 2024-03-29T08:24:27.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc914000640 1 -- start start 2024-03-29T08:24:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc914000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4003980 0x7fc90c1563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc914000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc90c155ec0 0x7fc90c164760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc914000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc90c164ca0 0x7fc90c1f8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc914000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c165150 con 0x7fc8f4003980 2024-03-29T08:24:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc914000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0b37c0 con 0x7fc90c164ca0 2024-03-29T08:24:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc914000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0abb50 con 0x7fc90c155ec0 2024-03-29T08:24:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc911600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4003980 0x7fc90c1563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc911600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4003980 0x7fc90c1563f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48854/0 (socket says 172.21.15.46:48854) 2024-03-29T08:24:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc911600640 1 -- 172.21.15.46:0/850769788 learned_addr learned my addr 172.21.15.46:0/850769788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc912000640 1 --2- 172.21.15.46:0/850769788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc90c164ca0 0x7fc90c1f8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc910c00640 1 --2- 172.21.15.46:0/850769788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc90c155ec0 0x7fc90c164760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc911600640 1 -- 172.21.15.46:0/850769788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc90c155ec0 msgr2=0x7fc90c164760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc911600640 1 --2- 172.21.15.46:0/850769788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc90c155ec0 0x7fc90c164760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc911600640 1 -- 172.21.15.46:0/850769788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc90c164ca0 msgr2=0x7fc90c1f8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc911600640 1 --2- 172.21.15.46:0/850769788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc90c164ca0 0x7fc90c1f8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc911600640 1 -- 172.21.15.46:0/850769788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c070c40 con 0x7fc8f4003980 2024-03-29T08:24:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc912000640 1 --2- 172.21.15.46:0/850769788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc90c164ca0 0x7fc90c1f8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.421+0000 7fc910c00640 1 --2- 172.21.15.46:0/850769788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc90c155ec0 0x7fc90c164760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.425+0000 7fc911600640 1 --2- 172.21.15.46:0/850769788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4003980 0x7fc90c1563f0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc9000b0040 tx=0x7fc8f4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.425+0000 7fc909600640 1 -- 172.21.15.46:0/850769788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90000f0c0 con 0x7fc8f4003980 2024-03-29T08:24:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.425+0000 7fc909600640 1 -- 172.21.15.46:0/850769788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9000a67c0 con 0x7fc8f4003980 2024-03-29T08:24:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.425+0000 7fc909600640 1 -- 172.21.15.46:0/850769788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9000a6b70 con 0x7fc8f4003980 2024-03-29T08:24:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.425+0000 7fc914000640 1 -- 172.21.15.46:0/850769788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90c0ace40 con 0x7fc8f4003980 2024-03-29T08:24:27.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.425+0000 7fc914000640 1 -- 172.21.15.46:0/850769788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c0ac080 con 0x7fc8f4003980 2024-03-29T08:24:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.425+0000 7fc914000640 1 -- 172.21.15.46:0/850769788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90c071a90 con 0x7fc8f4003980 2024-03-29T08:24:27.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.425+0000 7fc909600640 1 -- 172.21.15.46:0/850769788 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc9000a6cd0 con 0x7fc8f4003980 2024-03-29T08:24:27.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.425+0000 7fc909600640 1 --2- 172.21.15.46:0/850769788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8ec0666c0 0x7fc8ec068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:27.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.425+0000 7fc909600640 1 -- 172.21.15.46:0/850769788 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc9000aa070 con 0x7fc8f4003980 2024-03-29T08:24:27.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.429+0000 7fc910c00640 1 --2- 172.21.15.46:0/850769788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8ec0666c0 0x7fc8ec068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:27.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.429+0000 7fc910c00640 1 --2- 172.21.15.46:0/850769788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8ec0666c0 0x7fc8ec068b80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0b0840 tx=0x7fc8f8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:27.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.429+0000 7fc909600640 1 -- 172.21.15.46:0/850769788 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc9000ec0d0 con 0x7fc8f4003980 2024-03-29T08:24:27.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.601+0000 7fc914000640 1 -- 172.21.15.46:0/850769788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc90c0b2cb0 con 0x7fc8f4003980 2024-03-29T08:24:27.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.601+0000 7fc909600640 1 -- 172.21.15.46:0/850769788 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v80) v1 ==== 80+0+116 (secure 0 0 0) 0x7fc9000ef6f0 con 0x7fc8f4003980 2024-03-29T08:24:27.603 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6835/9617787 2024-03-30T08:23:02.389981+0000 2024-03-29T08:24:27.603 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6834/9617787 2024-03-30T08:23:02.389981+0000 2024-03-29T08:24:27.603 INFO:teuthology.orchestra.run.smithi046.stderr:listed 2 entries 2024-03-29T08:24:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.605+0000 7fc914000640 1 -- 172.21.15.46:0/850769788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8ec0666c0 msgr2=0x7fc8ec068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.605+0000 7fc914000640 1 --2- 172.21.15.46:0/850769788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8ec0666c0 0x7fc8ec068b80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0b0840 tx=0x7fc8f8008040 comp rx=0 tx=0).stop 2024-03-29T08:24:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.605+0000 7fc914000640 1 -- 172.21.15.46:0/850769788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4003980 msgr2=0x7fc90c1563f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.605+0000 7fc914000640 1 --2- 172.21.15.46:0/850769788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4003980 0x7fc90c1563f0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fc9000b0040 tx=0x7fc8f4005800 comp rx=0 tx=0).stop 2024-03-29T08:24:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.605+0000 7fc914000640 1 -- 172.21.15.46:0/850769788 shutdown_connections 2024-03-29T08:24:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.605+0000 7fc914000640 1 --2- 172.21.15.46:0/850769788 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8ec0666c0 0x7fc8ec068b80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.605+0000 7fc914000640 1 --2- 172.21.15.46:0/850769788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc90c164ca0 0x7fc90c1f8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.605+0000 7fc914000640 1 --2- 172.21.15.46:0/850769788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc90c155ec0 0x7fc90c164760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.605+0000 7fc914000640 1 --2- 172.21.15.46:0/850769788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8f4003980 0x7fc90c1563f0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.605+0000 7fc914000640 1 -- 172.21.15.46:0/850769788 >> 172.21.15.46:0/850769788 conn(0x7fc90c066080 msgr2=0x7fc90c158dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.605+0000 7fc914000640 1 -- 172.21.15.46:0/850769788 shutdown_connections 2024-03-29T08:24:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:27.605+0000 7fc914000640 1 -- 172.21.15.46:0/850769788 wait complete. 2024-03-29T08:24:27.623 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-03-29T08:24:27.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:27.623 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T08:24:27.644 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[43296]: fuse finished with error 0 and tester_r 0 2024-03-29T08:24:27.646 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:24:27.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:27.646 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:24:27.695 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:24:27.696 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:24:27.705 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:24:27.705 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:24:27.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:27.705 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T08:24:27.757 INFO:teuthology.orchestra.run.smithi073.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-29T08:24:27.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:24:27.758 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:24:27.758 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:24:27.767 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:24:27.767 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:24:27.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:27.767 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-29T08:24:27.814 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-29T08:24:27.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:24:27.815 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:24:27.816 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:24:27.824 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:24:27.825 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:24:27.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:27.825 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-29T08:24:27.873 INFO:teuthology.orchestra.run.smithi073.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-29T08:24:27.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:24:27.877 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:24:28.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.017+0000 7f295d600640 1 Processor -- start 2024-03-29T08:24:28.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.017+0000 7f295d600640 1 -- start start 2024-03-29T08:24:28.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.021+0000 7f295d600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2958159530 0x7f295806d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:28.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.021+0000 7f295d600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2958070a70 0x7f2958159e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:28.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.021+0000 7f295d600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f295815d4b0 con 0x7f295806ddd0 2024-03-29T08:24:28.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.021+0000 7f295d600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f295815d790 con 0x7f2958159530 2024-03-29T08:24:28.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.021+0000 7f295d600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295815da70 con 0x7f2958070a70 2024-03-29T08:24:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.021+0000 7f295cc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2958070a70 0x7f2958159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.021+0000 7f295cc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2958070a70 0x7f2958159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:49950/0 (socket says 172.21.15.46:49950) 2024-03-29T08:24:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.021+0000 7f295cc00640 1 -- 172.21.15.46:0/3496404432 learned_addr learned my addr 172.21.15.46:0/3496404432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.021+0000 7f2957400640 1 --1- 172.21.15.46:0/3496404432 >> v1:172.21.15.73:6789/0 conn(0x7f295806ddd0 0x7f2958070360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:45686/0 (socket says 172.21.15.46:45686) 2024-03-29T08:24:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.021+0000 7f2957e00640 1 --2- 172.21.15.46:0/3496404432 >> v2:172.21.15.73:3300/0 conn(0x7f2958159530 0x7f295806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.021+0000 7f295cc00640 1 -- 172.21.15.46:0/3496404432 >> v2:172.21.15.73:3300/0 conn(0x7f2958159530 msgr2=0x7f295806d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.021+0000 7f295cc00640 1 --2- 172.21.15.46:0/3496404432 >> v2:172.21.15.73:3300/0 conn(0x7f2958159530 0x7f295806d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.021+0000 7f295cc00640 1 -- 172.21.15.46:0/3496404432 >> v1:172.21.15.73:6789/0 conn(0x7f295806ddd0 legacy=0x7f2958070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.021+0000 7f295cc00640 1 -- 172.21.15.46:0/3496404432 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295815dd50 con 0x7f2958070a70 2024-03-29T08:24:28.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.021+0000 7f2957e00640 1 --2- 172.21.15.46:0/3496404432 >> v2:172.21.15.73:3300/0 conn(0x7f2958159530 0x7f295806d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:24:28.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.025+0000 7f295cc00640 1 --2- 172.21.15.46:0/3496404432 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2958070a70 0x7f2958159e60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f295000d150 tx=0x7f295001ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=377d01f78224f442 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.025+0000 7f2956a00640 1 -- 172.21.15.46:0/3496404432 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29500a7ce0 con 0x7f2958070a70 2024-03-29T08:24:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.025+0000 7f2956a00640 1 -- 172.21.15.46:0/3496404432 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29500a7e40 con 0x7f2958070a70 2024-03-29T08:24:28.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.025+0000 7f2956a00640 1 -- 172.21.15.46:0/3496404432 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295000b2f0 con 0x7f2958070a70 2024-03-29T08:24:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.025+0000 7f295d600640 1 -- 172.21.15.46:0/3496404432 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2958070a70 msgr2=0x7f2958159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.025+0000 7f295d600640 1 --2- 172.21.15.46:0/3496404432 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2958070a70 0x7f2958159e60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f295000d150 tx=0x7f295001ec30 comp rx=0 tx=0).stop 2024-03-29T08:24:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.025+0000 7f295d600640 1 -- 172.21.15.46:0/3496404432 shutdown_connections 2024-03-29T08:24:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.025+0000 7f295d600640 1 --2- 172.21.15.46:0/3496404432 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2958070a70 0x7f2958159e60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.025+0000 7f295d600640 1 --2- 172.21.15.46:0/3496404432 >> v2:172.21.15.73:3300/0 conn(0x7f2958159530 0x7f295806d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.025+0000 7f295d600640 1 -- 172.21.15.46:0/3496404432 >> 172.21.15.46:0/3496404432 conn(0x7f2958065e80 msgr2=0x7f2958069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.025+0000 7f295d600640 1 -- 172.21.15.46:0/3496404432 shutdown_connections 2024-03-29T08:24:28.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.025+0000 7f295d600640 1 -- 172.21.15.46:0/3496404432 wait complete. 2024-03-29T08:24:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.025+0000 7f295d600640 1 Processor -- start 2024-03-29T08:24:28.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.025+0000 7f295d600640 1 -- start start 2024-03-29T08:24:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.025+0000 7f295d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f295806ddd0 0x7f29581519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f295d600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2958070a70 0x7f295814dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f295d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2958159530 0x7f295814eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f295d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295814e8e0 con 0x7f2958159530 2024-03-29T08:24:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f295d600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2958152730 con 0x7f295806ddd0 2024-03-29T08:24:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f295d600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2958156bd0 con 0x7f2958070a70 2024-03-29T08:24:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f295cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2958159530 0x7f295814eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f295cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2958159530 0x7f295814eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48868/0 (socket says 172.21.15.46:48868) 2024-03-29T08:24:28.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f295cc00640 1 -- 172.21.15.46:0/2263111904 learned_addr learned my addr 172.21.15.46:0/2263111904 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f2957e00640 1 --2- 172.21.15.46:0/2263111904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f295806ddd0 0x7f29581519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f2957400640 1 --2- 172.21.15.46:0/2263111904 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2958070a70 0x7f295814dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f295cc00640 1 -- 172.21.15.46:0/2263111904 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2958070a70 msgr2=0x7f295814dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f295cc00640 1 --2- 172.21.15.46:0/2263111904 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2958070a70 0x7f295814dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f295cc00640 1 -- 172.21.15.46:0/2263111904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f295806ddd0 msgr2=0x7f29581519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f295cc00640 1 --2- 172.21.15.46:0/2263111904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f295806ddd0 0x7f29581519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f295cc00640 1 -- 172.21.15.46:0/2263111904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2948009420 con 0x7f2958159530 2024-03-29T08:24:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f295cc00640 1 --2- 172.21.15.46:0/2263111904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2958159530 0x7f295814eb30 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f29500b1040 tx=0x7f295001b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f294fe00640 1 -- 172.21.15.46:0/2263111904 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950009800 con 0x7f2958159530 2024-03-29T08:24:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f294fe00640 1 -- 172.21.15.46:0/2263111904 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2950009980 con 0x7f2958159530 2024-03-29T08:24:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f294fe00640 1 -- 172.21.15.46:0/2263111904 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950009d30 con 0x7f2958159530 2024-03-29T08:24:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f295d600640 1 -- 172.21.15.46:0/2263111904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295000bd40 con 0x7f2958159530 2024-03-29T08:24:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f295d600640 1 -- 172.21.15.46:0/2263111904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2958157190 con 0x7f2958159530 2024-03-29T08:24:28.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f295d600640 1 -- 172.21.15.46:0/2263111904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2958157380 con 0x7f2958159530 2024-03-29T08:24:28.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.029+0000 7f294fe00640 1 -- 172.21.15.46:0/2263111904 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f29500c4460 con 0x7f2958159530 2024-03-29T08:24:28.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.033+0000 7f294fe00640 1 --2- 172.21.15.46:0/2263111904 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2938066690 0x7f2938068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:28.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.033+0000 7f294fe00640 1 -- 172.21.15.46:0/2263111904 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f295011fab0 con 0x7f2958159530 2024-03-29T08:24:28.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.033+0000 7f2957e00640 1 --2- 172.21.15.46:0/2263111904 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2938066690 0x7f2938068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:28.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.033+0000 7f2957e00640 1 --2- 172.21.15.46:0/2263111904 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2938066690 0x7f2938068b50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2958156860 tx=0x7f2948009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:28.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.037+0000 7f294fe00640 1 -- 172.21.15.46:0/2263111904 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f29500ecec0 con 0x7f2958159530 2024-03-29T08:24:28.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.233+0000 7f295d600640 1 -- 172.21.15.46:0/2263111904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f295804e040 con 0x7f2958159530 2024-03-29T08:24:28.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.233+0000 7f294fe00640 1 -- 172.21.15.46:0/2263111904 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f29500f04e0 con 0x7f2958159530 2024-03-29T08:24:28.234 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:28.234 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3456771376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3456771376},{"type":"v1","addr":"172.21.15.73:6836","nonce":3456771376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2671538025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2671538025},{"type":"v1","addr":"172.21.15.73:6837","nonce":2671538025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3196015693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3196015693},{"type":"v1","addr":"172.21.15.46:6833","nonce":3196015693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:22:43.605457+0000","modified":"2024-03-29T08:23:53.005710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":16,"addr":"172.21.15.46:6835/1672348161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1672348161},{"type":"v1","addr":"172.21.15.46:6835","nonce":1672348161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-29T08:24:28.234 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 49 2024-03-29T08:24:28.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.237+0000 7f295d600640 1 -- 172.21.15.46:0/2263111904 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2938066690 msgr2=0x7f2938068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.237+0000 7f295d600640 1 --2- 172.21.15.46:0/2263111904 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2938066690 0x7f2938068b50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f2958156860 tx=0x7f2948009190 comp rx=0 tx=0).stop 2024-03-29T08:24:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.237+0000 7f295d600640 1 -- 172.21.15.46:0/2263111904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2958159530 msgr2=0x7f295814eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.237+0000 7f295d600640 1 --2- 172.21.15.46:0/2263111904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2958159530 0x7f295814eb30 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f29500b1040 tx=0x7f295001b9d0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.237+0000 7f295d600640 1 -- 172.21.15.46:0/2263111904 shutdown_connections 2024-03-29T08:24:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.237+0000 7f295d600640 1 --2- 172.21.15.46:0/2263111904 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2938066690 0x7f2938068b50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.237+0000 7f295d600640 1 --2- 172.21.15.46:0/2263111904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2958159530 0x7f295814eb30 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.237+0000 7f295d600640 1 --2- 172.21.15.46:0/2263111904 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2958070a70 0x7f295814dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.237+0000 7f295d600640 1 --2- 172.21.15.46:0/2263111904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f295806ddd0 0x7f29581519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.237+0000 7f295d600640 1 -- 172.21.15.46:0/2263111904 >> 172.21.15.46:0/2263111904 conn(0x7f2958065e80 msgr2=0x7f29580b02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.237+0000 7f295d600640 1 -- 172.21.15.46:0/2263111904 shutdown_connections 2024-03-29T08:24:28.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.237+0000 7f295d600640 1 -- 172.21.15.46:0/2263111904 wait complete. 2024-03-29T08:24:28.255 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:24:28.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.397+0000 7f035c800640 1 Processor -- start 2024-03-29T08:24:28.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.397+0000 7f035c800640 1 -- start start 2024-03-29T08:24:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.397+0000 7f035c800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f03540b07e0 0x7f03540cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.397+0000 7f035c800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03540cfb20 0x7f035415eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.397+0000 7f035c800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03540d02a0 con 0x7f03540b0bc0 2024-03-29T08:24:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.397+0000 7f035c800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f035415b160 con 0x7f03540b07e0 2024-03-29T08:24:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.397+0000 7f035c800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035415b440 con 0x7f03540cfb20 2024-03-29T08:24:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.397+0000 7f035a800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03540cfb20 0x7f035415eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.397+0000 7f0359e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f03540b07e0 0x7f03540cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.397+0000 7f0359400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f03540b0bc0 0x7f035415e470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:45690/0 (socket says 172.21.15.46:45690) 2024-03-29T08:24:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.397+0000 7f0359400640 1 -- 172.21.15.46:0/2358832646 learned_addr learned my addr 172.21.15.46:0/2358832646 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.397+0000 7f035a800640 1 -- 172.21.15.46:0/2358832646 >> v2:172.21.15.73:3300/0 conn(0x7f03540b07e0 msgr2=0x7f03540cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.397+0000 7f0358a00640 1 -- 172.21.15.46:0/2358832646 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3405060566 0 0) 0x7f03540d02a0 con 0x7f03540b0bc0 2024-03-29T08:24:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.397+0000 7f035a800640 1 --2- 172.21.15.46:0/2358832646 >> v2:172.21.15.73:3300/0 conn(0x7f03540b07e0 0x7f03540cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.397+0000 7f035a800640 1 -- 172.21.15.46:0/2358832646 >> v1:172.21.15.73:6789/0 conn(0x7f03540b0bc0 legacy=0x7f035415e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.397+0000 7f035a800640 1 -- 172.21.15.46:0/2358832646 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035415b720 con 0x7f03540cfb20 2024-03-29T08:24:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.397+0000 7f0359e00640 1 --2- 172.21.15.46:0/2358832646 >> v2:172.21.15.73:3300/0 conn(0x7f03540b07e0 0x7f03540cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:24:28.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.401+0000 7f035a800640 1 --2- 172.21.15.46:0/2358832646 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03540cfb20 0x7f035415eb80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0350009d90 tx=0x7f035001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=273bb983e732d7b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:28.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.401+0000 7f0358a00640 1 -- 172.21.15.46:0/2358832646 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035415be50 con 0x7f03540cfb20 2024-03-29T08:24:28.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.401+0000 7f0358a00640 1 -- 172.21.15.46:0/2358832646 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03500a59d0 con 0x7f03540cfb20 2024-03-29T08:24:28.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.401+0000 7f0358a00640 1 -- 172.21.15.46:0/2358832646 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03500a5d60 con 0x7f03540cfb20 2024-03-29T08:24:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.401+0000 7f035c800640 1 -- 172.21.15.46:0/2358832646 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03540cfb20 msgr2=0x7f035415eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.401+0000 7f035c800640 1 --2- 172.21.15.46:0/2358832646 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03540cfb20 0x7f035415eb80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0350009d90 tx=0x7f035001c970 comp rx=0 tx=0).stop 2024-03-29T08:24:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.401+0000 7f035c800640 1 -- 172.21.15.46:0/2358832646 shutdown_connections 2024-03-29T08:24:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.401+0000 7f035c800640 1 --2- 172.21.15.46:0/2358832646 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03540cfb20 0x7f035415eb80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.401+0000 7f035c800640 1 --2- 172.21.15.46:0/2358832646 >> v2:172.21.15.73:3300/0 conn(0x7f03540b07e0 0x7f03540cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.401+0000 7f035c800640 1 -- 172.21.15.46:0/2358832646 >> 172.21.15.46:0/2358832646 conn(0x7f0354071810 msgr2=0x7f0354098a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.401+0000 7f035c800640 1 -- 172.21.15.46:0/2358832646 shutdown_connections 2024-03-29T08:24:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.401+0000 7f035c800640 1 -- 172.21.15.46:0/2358832646 wait complete. 2024-03-29T08:24:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035c800640 1 Processor -- start 2024-03-29T08:24:28.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035c800640 1 -- start start 2024-03-29T08:24:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035c800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03540b07e0 0x7f03540b4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035c800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03540b0bc0 0x7f03540ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035c800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03540cfb20 0x7f03541536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035c800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03540cf2c0 con 0x7f03540cfb20 2024-03-29T08:24:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035c800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03540ceab0 con 0x7f03540b0bc0 2024-03-29T08:24:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035c800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03540ced00 con 0x7f03540b07e0 2024-03-29T08:24:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03540cfb20 0x7f03541536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03540cfb20 0x7f03541536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48872/0 (socket says 172.21.15.46:48872) 2024-03-29T08:24:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035a800640 1 -- 172.21.15.46:0/1994882463 learned_addr learned my addr 172.21.15.46:0/1994882463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f0359e00640 1 --2- 172.21.15.46:0/1994882463 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03540b07e0 0x7f03540b4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f0359400640 1 --2- 172.21.15.46:0/1994882463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03540b0bc0 0x7f03540ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035a800640 1 -- 172.21.15.46:0/1994882463 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03540b07e0 msgr2=0x7f03540b4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035a800640 1 --2- 172.21.15.46:0/1994882463 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03540b07e0 0x7f03540b4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035a800640 1 -- 172.21.15.46:0/1994882463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03540b0bc0 msgr2=0x7f03540ce2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035a800640 1 --2- 172.21.15.46:0/1994882463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03540b0bc0 0x7f03540ce2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035a800640 1 -- 172.21.15.46:0/1994882463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0348009420 con 0x7f03540cfb20 2024-03-29T08:24:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035a800640 1 --2- 172.21.15.46:0/1994882463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03540cfb20 0x7f03541536d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f035001a5a0 tx=0x7f0350019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f034e000640 1 -- 172.21.15.46:0/1994882463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03500ad850 con 0x7f03540cfb20 2024-03-29T08:24:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f034e000640 1 -- 172.21.15.46:0/1994882463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03500ad9b0 con 0x7f03540cfb20 2024-03-29T08:24:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035c800640 1 -- 172.21.15.46:0/1994882463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0350009a70 con 0x7f03540cfb20 2024-03-29T08:24:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f034e000640 1 -- 172.21.15.46:0/1994882463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03500adda0 con 0x7f03540cfb20 2024-03-29T08:24:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035c800640 1 -- 172.21.15.46:0/1994882463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03540cddd0 con 0x7f03540cfb20 2024-03-29T08:24:28.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.405+0000 7f035c800640 1 -- 172.21.15.46:0/1994882463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03540cdc30 con 0x7f03540cfb20 2024-03-29T08:24:28.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.409+0000 7f034e000640 1 -- 172.21.15.46:0/1994882463 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f03500b4070 con 0x7f03540cfb20 2024-03-29T08:24:28.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.409+0000 7f034e000640 1 --2- 172.21.15.46:0/1994882463 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0338066690 0x7f0338068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:28.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.409+0000 7f034e000640 1 -- 172.21.15.46:0/1994882463 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f035011edc0 con 0x7f03540cfb20 2024-03-29T08:24:28.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.409+0000 7f0359e00640 1 --2- 172.21.15.46:0/1994882463 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0338066690 0x7f0338068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:28.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.409+0000 7f0359e00640 1 --2- 172.21.15.46:0/1994882463 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0338066690 0x7f0338068b50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0348009cf0 tx=0x7f0348009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.413+0000 7f034e000640 1 -- 172.21.15.46:0/1994882463 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f03540cdc30 con 0x7f03540cfb20 2024-03-29T08:24:28.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.573+0000 7f035c800640 1 -- 172.21.15.46:0/1994882463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03540cdc30 con 0x7f03540cfb20 2024-03-29T08:24:28.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.577+0000 7f034e000640 1 -- 172.21.15.46:0/1994882463 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16213 (secure 0 0 0) 0x7f03540cdc30 con 0x7f03540cfb20 2024-03-29T08:24:28.576 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:28.576 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:23:02.390005+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:22:42.147364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-29T08:22:42.602168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1f","mappings":[{"from":2,"to":7}]},{"pgid":"11.27","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/9617787":"2024-03-30T08:23:02.389981+0000","172.21.15.46:6834/9617787":"2024-03-30T08:23:02.389981+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:24:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.577+0000 7f035c800640 1 -- 172.21.15.46:0/1994882463 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0338066690 msgr2=0x7f0338068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.577+0000 7f035c800640 1 --2- 172.21.15.46:0/1994882463 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0338066690 0x7f0338068b50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0348009cf0 tx=0x7f0348009190 comp rx=0 tx=0).stop 2024-03-29T08:24:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.577+0000 7f035c800640 1 -- 172.21.15.46:0/1994882463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03540cfb20 msgr2=0x7f03541536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.577+0000 7f035c800640 1 --2- 172.21.15.46:0/1994882463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03540cfb20 0x7f03541536d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f035001a5a0 tx=0x7f0350019710 comp rx=0 tx=0).stop 2024-03-29T08:24:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.577+0000 7f035c800640 1 -- 172.21.15.46:0/1994882463 shutdown_connections 2024-03-29T08:24:28.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.577+0000 7f035c800640 1 --2- 172.21.15.46:0/1994882463 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0338066690 0x7f0338068b50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.577+0000 7f035c800640 1 --2- 172.21.15.46:0/1994882463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03540b0bc0 0x7f03540ce2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.577+0000 7f035c800640 1 --2- 172.21.15.46:0/1994882463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03540cfb20 0x7f03541536d0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.577+0000 7f035c800640 1 --2- 172.21.15.46:0/1994882463 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f03540b07e0 0x7f03540b4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.577+0000 7f035c800640 1 -- 172.21.15.46:0/1994882463 >> 172.21.15.46:0/1994882463 conn(0x7f0354071810 msgr2=0x7f035406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.577+0000 7f035c800640 1 -- 172.21.15.46:0/1994882463 shutdown_connections 2024-03-29T08:24:28.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.577+0000 7f035c800640 1 -- 172.21.15.46:0/1994882463 wait complete. 2024-03-29T08:24:28.597 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:22:42.147364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:22:42.602168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:24:28.597 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:24:28.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.741+0000 7f96cf200640 1 Processor -- start 2024-03-29T08:24:28.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.741+0000 7f96cf200640 1 -- start start 2024-03-29T08:24:28.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.741+0000 7f96cf200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96d0099780 0x7f96d00b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:28.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.741+0000 7f96cf200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d006b150 con 0x7f96d0098a40 2024-03-29T08:24:28.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.741+0000 7f96cf200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d015e190 con 0x7f96d015ee70 2024-03-29T08:24:28.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.741+0000 7f96cf200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d006b6d0 con 0x7f96d0099780 2024-03-29T08:24:28.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.741+0000 7f96cde00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f96d0098a40 0x7f96d0098e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57760/0 (socket says 172.21.15.46:57760) 2024-03-29T08:24:28.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.741+0000 7f96cde00640 1 -- 172.21.15.46:0/2883301661 learned_addr learned my addr 172.21.15.46:0/2883301661 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:28.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.741+0000 7f96c7e00640 1 --2- 172.21.15.46:0/2883301661 >> v2:172.21.15.46:3300/0 conn(0x7f96d0099780 0x7f96d00b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:28.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.741+0000 7f96cd400640 1 -- 172.21.15.46:0/2883301661 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227773559 0 0) 0x7f96d006b150 con 0x7f96d0098a40 2024-03-29T08:24:28.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.741+0000 7f96cd400640 1 -- 172.21.15.46:0/2883301661 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96b4003530 con 0x7f96d0098a40 2024-03-29T08:24:28.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.741+0000 7f96cd400640 1 -- 172.21.15.46:0/2883301661 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2982155325 0 0) 0x7f96d015e190 con 0x7f96d015ee70 2024-03-29T08:24:28.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.741+0000 7f96c7e00640 1 -- 172.21.15.46:0/2883301661 >> v1:172.21.15.73:6789/0 conn(0x7f96d015ee70 legacy=0x7f96d015f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.741+0000 7f96c7e00640 1 -- 172.21.15.46:0/2883301661 >> v1:172.21.15.46:6789/0 conn(0x7f96d0098a40 legacy=0x7f96d0098e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.741+0000 7f96c7e00640 1 -- 172.21.15.46:0/2883301661 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d006b3f0 con 0x7f96d0099780 2024-03-29T08:24:28.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.745+0000 7f96c7e00640 1 --2- 172.21.15.46:0/2883301661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96d0099780 0x7f96d00b1420 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f96b80097f0 tx=0x7f96b801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e90e4ddfb3ce9e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:28.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.745+0000 7f96cd400640 1 -- 172.21.15.46:0/2883301661 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b80ac640 con 0x7f96d0099780 2024-03-29T08:24:28.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.745+0000 7f96cd400640 1 -- 172.21.15.46:0/2883301661 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b80ac7a0 con 0x7f96d0099780 2024-03-29T08:24:28.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.745+0000 7f96cd400640 1 -- 172.21.15.46:0/2883301661 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b80acb30 con 0x7f96d0099780 2024-03-29T08:24:28.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.745+0000 7f96cf200640 1 -- 172.21.15.46:0/2883301661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96d0099780 msgr2=0x7f96d00b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.745+0000 7f96cf200640 1 --2- 172.21.15.46:0/2883301661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96d0099780 0x7f96d00b1420 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f96b80097f0 tx=0x7f96b801c3e0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.745+0000 7f96cf200640 1 -- 172.21.15.46:0/2883301661 shutdown_connections 2024-03-29T08:24:28.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.745+0000 7f96cf200640 1 --2- 172.21.15.46:0/2883301661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96d0099780 0x7f96d00b1420 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.745+0000 7f96cf200640 1 -- 172.21.15.46:0/2883301661 >> 172.21.15.46:0/2883301661 conn(0x7f96d00b1fb0 msgr2=0x7f96d00b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:28.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.745+0000 7f96cf200640 1 -- 172.21.15.46:0/2883301661 shutdown_connections 2024-03-29T08:24:28.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.745+0000 7f96cf200640 1 -- 172.21.15.46:0/2883301661 wait complete. 2024-03-29T08:24:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cf200640 1 Processor -- start 2024-03-29T08:24:28.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cf200640 1 -- start start 2024-03-29T08:24:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96d0098a40 0x7f96d0151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cf200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96d0099780 0x7f96d01414d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cf200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96d015ee70 0x7f96d0156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cf200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d00c59b0 con 0x7f96d0098a40 2024-03-29T08:24:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cf200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d00c5c90 con 0x7f96d015ee70 2024-03-29T08:24:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cf200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d00c5f70 con 0x7f96d0099780 2024-03-29T08:24:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96d0098a40 0x7f96d0151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96d0098a40 0x7f96d0151890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48900/0 (socket says 172.21.15.46:48900) 2024-03-29T08:24:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cde00640 1 -- 172.21.15.46:0/3924565437 learned_addr learned my addr 172.21.15.46:0/3924565437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96c7e00640 1 --2- 172.21.15.46:0/3924565437 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96d0099780 0x7f96d01414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:28.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96ce800640 1 --2- 172.21.15.46:0/3924565437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96d015ee70 0x7f96d0156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cde00640 1 -- 172.21.15.46:0/3924565437 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96d0099780 msgr2=0x7f96d01414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cde00640 1 --2- 172.21.15.46:0/3924565437 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96d0099780 0x7f96d01414d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cde00640 1 -- 172.21.15.46:0/3924565437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96d015ee70 msgr2=0x7f96d0156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cde00640 1 --2- 172.21.15.46:0/3924565437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96d015ee70 0x7f96d0156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cde00640 1 -- 172.21.15.46:0/3924565437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b8009420 con 0x7f96d0098a40 2024-03-29T08:24:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cde00640 1 --2- 172.21.15.46:0/3924565437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96d0098a40 0x7f96d0151890 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f96c000c3c0 tx=0x7f96c000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96c6000640 1 -- 172.21.15.46:0/3924565437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c0005cf0 con 0x7f96d0098a40 2024-03-29T08:24:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96c6000640 1 -- 172.21.15.46:0/3924565437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c0005e50 con 0x7f96d0098a40 2024-03-29T08:24:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96c6000640 1 -- 172.21.15.46:0/3924565437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c00072f0 con 0x7f96d0098a40 2024-03-29T08:24:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cf200640 1 -- 172.21.15.46:0/3924565437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d00c6250 con 0x7f96d0098a40 2024-03-29T08:24:28.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cf200640 1 -- 172.21.15.46:0/3924565437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d0144240 con 0x7f96d0098a40 2024-03-29T08:24:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.749+0000 7f96cf200640 1 -- 172.21.15.46:0/3924565437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d00c6490 con 0x7f96d0098a40 2024-03-29T08:24:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.753+0000 7f96c6000640 1 -- 172.21.15.46:0/3924565437 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f96c0007450 con 0x7f96d0098a40 2024-03-29T08:24:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.753+0000 7f96c6000640 1 --2- 172.21.15.46:0/3924565437 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f96ac066690 0x7f96ac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:28.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.753+0000 7f96c6000640 1 -- 172.21.15.46:0/3924565437 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f96c00837a0 con 0x7f96d0098a40 2024-03-29T08:24:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.753+0000 7f96c7e00640 1 --2- 172.21.15.46:0/3924565437 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f96ac066690 0x7f96ac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:28.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.753+0000 7f96c7e00640 1 --2- 172.21.15.46:0/3924565437 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f96ac066690 0x7f96ac068b50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f96b80ae000 tx=0x7f96b80a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:28.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.757+0000 7f96c6000640 1 -- 172.21.15.46:0/3924565437 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f96d00c6490 con 0x7f96d0098a40 2024-03-29T08:24:28.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.917+0000 7f96cf200640 1 -- 172.21.15.46:0/3924565437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f96d00c6490 con 0x7f96d0098a40 2024-03-29T08:24:28.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.917+0000 7f96c6000640 1 -- 172.21.15.46:0/3924565437 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f96d00c6490 con 0x7f96d0098a40 2024-03-29T08:24:28.917 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:24:28.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.917+0000 7f96cf200640 1 -- 172.21.15.46:0/3924565437 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f96ac066690 msgr2=0x7f96ac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.917+0000 7f96cf200640 1 --2- 172.21.15.46:0/3924565437 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f96ac066690 0x7f96ac068b50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f96b80ae000 tx=0x7f96b80a7000 comp rx=0 tx=0).stop 2024-03-29T08:24:28.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.917+0000 7f96cf200640 1 -- 172.21.15.46:0/3924565437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96d0098a40 msgr2=0x7f96d0151890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:28.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.917+0000 7f96cf200640 1 --2- 172.21.15.46:0/3924565437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96d0098a40 0x7f96d0151890 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f96c000c3c0 tx=0x7f96c000c880 comp rx=0 tx=0).stop 2024-03-29T08:24:28.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.917+0000 7f96cf200640 1 -- 172.21.15.46:0/3924565437 shutdown_connections 2024-03-29T08:24:28.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.917+0000 7f96cf200640 1 --2- 172.21.15.46:0/3924565437 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f96ac066690 0x7f96ac068b50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.917+0000 7f96cf200640 1 --2- 172.21.15.46:0/3924565437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96d015ee70 0x7f96d0156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.917+0000 7f96cf200640 1 --2- 172.21.15.46:0/3924565437 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96d0099780 0x7f96d01414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.917+0000 7f96cf200640 1 --2- 172.21.15.46:0/3924565437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96d0098a40 0x7f96d0151890 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:28.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.917+0000 7f96cf200640 1 -- 172.21.15.46:0/3924565437 >> 172.21.15.46:0/3924565437 conn(0x7f96d00b1fb0 msgr2=0x7f96d015d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:28.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.917+0000 7f96cf200640 1 -- 172.21.15.46:0/3924565437 shutdown_connections 2024-03-29T08:24:28.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:28.917+0000 7f96cf200640 1 -- 172.21.15.46:0/3924565437 wait complete. 2024-03-29T08:24:28.937 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:24:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.081+0000 7fc8bb800640 1 Processor -- start 2024-03-29T08:24:29.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.081+0000 7fc8bb800640 1 -- start start 2024-03-29T08:24:29.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.081+0000 7fc8bb800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8b4143eb0 0x7fc8b40cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.081+0000 7fc8bb800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc8b415efe0 0x7fc8b40cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.081+0000 7fc8bb800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b40b2360 con 0x7fc8b40b1630 2024-03-29T08:24:29.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.081+0000 7fc8bb800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b415e500 con 0x7fc8b415efe0 2024-03-29T08:24:29.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.081+0000 7fc8bb800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b415e7e0 con 0x7fc8b4143eb0 2024-03-29T08:24:29.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.081+0000 7fc8b9800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc8b40b1630 0x7fc8b40b1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57764/0 (socket says 172.21.15.46:57764) 2024-03-29T08:24:29.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.081+0000 7fc8b9800640 1 -- 172.21.15.46:0/3690073230 learned_addr learned my addr 172.21.15.46:0/3690073230 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:29.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.081+0000 7fc8abe00640 1 --2- 172.21.15.46:0/3690073230 >> v2:172.21.15.73:3300/0 conn(0x7fc8b415efe0 0x7fc8b40cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.081+0000 7fc8b8e00640 1 --2- 172.21.15.46:0/3690073230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8b4143eb0 0x7fc8b40cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.081+0000 7fc8b3e00640 1 -- 172.21.15.46:0/3690073230 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500991206 0 0) 0x7fc8b40b2360 con 0x7fc8b40b1630 2024-03-29T08:24:29.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.081+0000 7fc8b3e00640 1 -- 172.21.15.46:0/3690073230 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc898003530 con 0x7fc8b40b1630 2024-03-29T08:24:29.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8b3e00640 1 -- 172.21.15.46:0/3690073230 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 941507906 0 0) 0x7fc898003530 con 0x7fc8b40b1630 2024-03-29T08:24:29.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8b3e00640 1 -- 172.21.15.46:0/3690073230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8b4143eb0 msgr2=0x7fc8b40cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:29.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8b3e00640 1 --2- 172.21.15.46:0/3690073230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8b4143eb0 0x7fc8b40cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8b3e00640 1 -- 172.21.15.46:0/3690073230 >> v2:172.21.15.73:3300/0 conn(0x7fc8b415efe0 msgr2=0x7fc8b40cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:29.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8b3e00640 1 --2- 172.21.15.46:0/3690073230 >> v2:172.21.15.73:3300/0 conn(0x7fc8b415efe0 0x7fc8b40cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8b3e00640 1 -- 172.21.15.46:0/3690073230 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b415eac0 con 0x7fc8b40b1630 2024-03-29T08:24:29.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8b3e00640 1 -- 172.21.15.46:0/3690073230 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fc8ac001a00 con 0x7fc8b40b1630 2024-03-29T08:24:29.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8abe00640 1 --2- 172.21.15.46:0/3690073230 >> v2:172.21.15.73:3300/0 conn(0x7fc8b415efe0 0x7fc8b40cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:24:29.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8b3e00640 1 -- 172.21.15.46:0/3690073230 >> v1:172.21.15.46:6789/0 conn(0x7fc8b40b1630 legacy=0x7fc8b40b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8b3e00640 1 --2- 172.21.15.46:0/3690073230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8b4143eb0 0x7fc898003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8b3e00640 1 --2- 172.21.15.46:0/3690073230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc898003f50 0x7fc8980047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8b3e00640 1 --2- 172.21.15.46:0/3690073230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc898004d10 0x7fc898005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8b3e00640 1 -- 172.21.15.46:0/3690073230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b415e500 con 0x7fc898003f50 2024-03-29T08:24:29.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8b3e00640 1 -- 172.21.15.46:0/3690073230 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b415e7e0 con 0x7fc898004d10 2024-03-29T08:24:29.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8b3e00640 1 -- 172.21.15.46:0/3690073230 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89800b990 con 0x7fc8b4143eb0 2024-03-29T08:24:29.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8abe00640 1 --2- 172.21.15.46:0/3690073230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc898003f50 0x7fc8980047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8b8e00640 1 --2- 172.21.15.46:0/3690073230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8b4143eb0 0x7fc898003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8b9800640 1 --2- 172.21.15.46:0/3690073230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc898004d10 0x7fc898005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8abe00640 1 -- 172.21.15.46:0/3690073230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8b4143eb0 msgr2=0x7fc898003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:29.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8abe00640 1 --2- 172.21.15.46:0/3690073230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8b4143eb0 0x7fc898003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8abe00640 1 -- 172.21.15.46:0/3690073230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc898004d10 msgr2=0x7fc898005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:29.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8abe00640 1 --2- 172.21.15.46:0/3690073230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc898004d10 0x7fc898005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8abe00640 1 -- 172.21.15.46:0/3690073230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc898003530 con 0x7fc898003f50 2024-03-29T08:24:29.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8b9800640 1 --2- 172.21.15.46:0/3690073230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc898004d10 0x7fc898005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:29.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.085+0000 7fc8abe00640 1 --2- 172.21.15.46:0/3690073230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc898003f50 0x7fc8980047d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009ce0 tx=0x7fc89c01cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7eb53ee95fc1849 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:29.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8b3e00640 1 -- 172.21.15.46:0/3690073230 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c0a5820 con 0x7fc898003f50 2024-03-29T08:24:29.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8b3e00640 1 -- 172.21.15.46:0/3690073230 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89c0a5980 con 0x7fc898003f50 2024-03-29T08:24:29.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8b3e00640 1 -- 172.21.15.46:0/3690073230 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c0a5d70 con 0x7fc898003f50 2024-03-29T08:24:29.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8bb800640 1 -- 172.21.15.46:0/3690073230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc898003f50 msgr2=0x7fc8980047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8bb800640 1 --2- 172.21.15.46:0/3690073230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc898003f50 0x7fc8980047d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009ce0 tx=0x7fc89c01cc70 comp rx=0 tx=0).stop 2024-03-29T08:24:29.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8b9800640 1 -- 172.21.15.46:0/3690073230 reap_dead start 2024-03-29T08:24:29.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8bb800640 1 -- 172.21.15.46:0/3690073230 shutdown_connections 2024-03-29T08:24:29.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8bb800640 1 -- 172.21.15.46:0/3690073230 >> 172.21.15.46:0/3690073230 conn(0x7fc8b406a390 msgr2=0x7fc8b40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:29.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8bb800640 1 -- 172.21.15.46:0/3690073230 shutdown_connections 2024-03-29T08:24:29.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8bb800640 1 -- 172.21.15.46:0/3690073230 wait complete. 2024-03-29T08:24:29.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8bb800640 1 Processor -- start 2024-03-29T08:24:29.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8bb800640 1 -- start start 2024-03-29T08:24:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8bb800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc898003f50 0x7fc8b40ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8bb800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b41f2740 0x7fc8b41413a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8bb800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8b413dc40 0x7fc8b413cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8bb800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b413edb0 con 0x7fc8b413dc40 2024-03-29T08:24:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8bb800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40ae510 con 0x7fc8b41f2740 2024-03-29T08:24:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8bb800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b40adf50 con 0x7fc898003f50 2024-03-29T08:24:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8b9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8b413dc40 0x7fc8b413cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8b9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8b413dc40 0x7fc8b413cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48912/0 (socket says 172.21.15.46:48912) 2024-03-29T08:24:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8b9800640 1 -- 172.21.15.46:0/3983572857 learned_addr learned my addr 172.21.15.46:0/3983572857 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8b8e00640 1 --2- 172.21.15.46:0/3983572857 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc898003f50 0x7fc8b40ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.089+0000 7fc8abe00640 1 --2- 172.21.15.46:0/3983572857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b41f2740 0x7fc8b41413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.093+0000 7fc8b9800640 1 -- 172.21.15.46:0/3983572857 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc898003f50 msgr2=0x7fc8b40ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.093+0000 7fc8b9800640 1 --2- 172.21.15.46:0/3983572857 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc898003f50 0x7fc8b40ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.093+0000 7fc8b9800640 1 -- 172.21.15.46:0/3983572857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b41f2740 msgr2=0x7fc8b41413a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.093+0000 7fc8b9800640 1 --2- 172.21.15.46:0/3983572857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b41f2740 0x7fc8b41413a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.093+0000 7fc8b9800640 1 -- 172.21.15.46:0/3983572857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b40ae230 con 0x7fc8b413dc40 2024-03-29T08:24:29.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.093+0000 7fc8b9800640 1 --2- 172.21.15.46:0/3983572857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8b413dc40 0x7fc8b413cfe0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc8b406ad70 tx=0x7fc8ac011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:29.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.093+0000 7fc8b1600640 1 -- 172.21.15.46:0/3983572857 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac0137a0 con 0x7fc8b413dc40 2024-03-29T08:24:29.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.093+0000 7fc8b1600640 1 -- 172.21.15.46:0/3983572857 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ac013920 con 0x7fc8b413dc40 2024-03-29T08:24:29.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.093+0000 7fc8b1600640 1 -- 172.21.15.46:0/3983572857 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ac013cd0 con 0x7fc8b413dc40 2024-03-29T08:24:29.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.093+0000 7fc8bb800640 1 -- 172.21.15.46:0/3983572857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b413ea40 con 0x7fc8b413dc40 2024-03-29T08:24:29.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.093+0000 7fc8bb800640 1 -- 172.21.15.46:0/3983572857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b41418e0 con 0x7fc8b413dc40 2024-03-29T08:24:29.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.093+0000 7fc8bb800640 1 -- 172.21.15.46:0/3983572857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b4141ad0 con 0x7fc8b413dc40 2024-03-29T08:24:29.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.093+0000 7fc8b1600640 1 -- 172.21.15.46:0/3983572857 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc8ac01f460 con 0x7fc8b413dc40 2024-03-29T08:24:29.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.093+0000 7fc8b1600640 1 --2- 172.21.15.46:0/3983572857 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc890066690 0x7fc890068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.093+0000 7fc8b1600640 1 -- 172.21.15.46:0/3983572857 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc8ac087190 con 0x7fc8b413dc40 2024-03-29T08:24:29.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.093+0000 7fc8b8e00640 1 --2- 172.21.15.46:0/3983572857 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc890066690 0x7fc890068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.097+0000 7fc8b8e00640 1 --2- 172.21.15.46:0/3983572857 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc890066690 0x7fc890068b50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009b00 tx=0x7fc8a4009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:29.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.097+0000 7fc8b1600640 1 -- 172.21.15.46:0/3983572857 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc8b4141ad0 con 0x7fc8b413dc40 2024-03-29T08:24:29.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.257+0000 7fc8bb800640 1 -- 172.21.15.46:0/3983572857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc8b413e8a0 con 0x7fc8b413dc40 2024-03-29T08:24:29.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.257+0000 7fc8b1600640 1 -- 172.21.15.46:0/3983572857 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc8b413e8a0 con 0x7fc8b413dc40 2024-03-29T08:24:29.259 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:24:29.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.261+0000 7fc8bb800640 1 -- 172.21.15.46:0/3983572857 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc890066690 msgr2=0x7fc890068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.261+0000 7fc8bb800640 1 --2- 172.21.15.46:0/3983572857 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc890066690 0x7fc890068b50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fc8a4009b00 tx=0x7fc8a4009190 comp rx=0 tx=0).stop 2024-03-29T08:24:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.261+0000 7fc8bb800640 1 -- 172.21.15.46:0/3983572857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8b413dc40 msgr2=0x7fc8b413cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.261+0000 7fc8bb800640 1 --2- 172.21.15.46:0/3983572857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8b413dc40 0x7fc8b413cfe0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fc8b406ad70 tx=0x7fc8ac011b60 comp rx=0 tx=0).stop 2024-03-29T08:24:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.261+0000 7fc8bb800640 1 -- 172.21.15.46:0/3983572857 shutdown_connections 2024-03-29T08:24:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.261+0000 7fc8bb800640 1 --2- 172.21.15.46:0/3983572857 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc890066690 0x7fc890068b50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.261+0000 7fc8bb800640 1 --2- 172.21.15.46:0/3983572857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8b41f2740 0x7fc8b41413a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.261+0000 7fc8bb800640 1 --2- 172.21.15.46:0/3983572857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8b413dc40 0x7fc8b413cfe0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.261+0000 7fc8bb800640 1 --2- 172.21.15.46:0/3983572857 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc898003f50 0x7fc8b40ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.261+0000 7fc8bb800640 1 -- 172.21.15.46:0/3983572857 >> 172.21.15.46:0/3983572857 conn(0x7fc8b406a390 msgr2=0x7fc8b4070180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.261+0000 7fc8bb800640 1 -- 172.21.15.46:0/3983572857 shutdown_connections 2024-03-29T08:24:29.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.261+0000 7fc8bb800640 1 -- 172.21.15.46:0/3983572857 wait complete. 2024-03-29T08:24:29.279 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:24:29.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.421+0000 7fc83d800640 1 Processor -- start 2024-03-29T08:24:29.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.421+0000 7fc83d800640 1 -- start start 2024-03-29T08:24:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.425+0000 7fc83d800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc83815cad0 0x7fc83806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.425+0000 7fc83d800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc83806a4a0 0x7fc83806d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.425+0000 7fc83d800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc83815d080 con 0x7fc83806eff0 2024-03-29T08:24:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.425+0000 7fc83d800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc83815dd00 con 0x7fc83815cad0 2024-03-29T08:24:29.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.425+0000 7fc83d800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83806f480 con 0x7fc83806a4a0 2024-03-29T08:24:29.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.425+0000 7fc837400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc83806a4a0 0x7fc83806d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.425+0000 7fc836a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc83815cad0 0x7fc83806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.425+0000 7fc837400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc83806a4a0 0x7fc83806d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:50028/0 (socket says 172.21.15.46:50028) 2024-03-29T08:24:29.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.425+0000 7fc82e000640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc83806eff0 0x7fc838069d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:45714/0 (socket says 172.21.15.46:45714) 2024-03-29T08:24:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.425+0000 7fc837400640 1 -- 172.21.15.46:0/746287175 learned_addr learned my addr 172.21.15.46:0/746287175 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.425+0000 7fc837400640 1 -- 172.21.15.46:0/746287175 >> v2:172.21.15.73:3300/0 conn(0x7fc83815cad0 msgr2=0x7fc83806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.425+0000 7fc837400640 1 --2- 172.21.15.46:0/746287175 >> v2:172.21.15.73:3300/0 conn(0x7fc83815cad0 0x7fc83806eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.425+0000 7fc837400640 1 -- 172.21.15.46:0/746287175 >> v1:172.21.15.73:6789/0 conn(0x7fc83806eff0 legacy=0x7fc838069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.425+0000 7fc837400640 1 -- 172.21.15.46:0/746287175 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc83815d360 con 0x7fc83806a4a0 2024-03-29T08:24:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc837400640 1 --2- 172.21.15.46:0/746287175 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc83806a4a0 0x7fc83806d470 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc830009870 tx=0x7fc83001c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1765470996d615d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc836000640 1 -- 172.21.15.46:0/746287175 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8300ac640 con 0x7fc83806a4a0 2024-03-29T08:24:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc836000640 1 -- 172.21.15.46:0/746287175 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8300ac7a0 con 0x7fc83806a4a0 2024-03-29T08:24:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc836000640 1 -- 172.21.15.46:0/746287175 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8300acb30 con 0x7fc83806a4a0 2024-03-29T08:24:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc83d800640 1 -- 172.21.15.46:0/746287175 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc83806a4a0 msgr2=0x7fc83806d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc83d800640 1 --2- 172.21.15.46:0/746287175 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc83806a4a0 0x7fc83806d470 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc830009870 tx=0x7fc83001c4d0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc83d800640 1 -- 172.21.15.46:0/746287175 shutdown_connections 2024-03-29T08:24:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc83d800640 1 --2- 172.21.15.46:0/746287175 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc83806a4a0 0x7fc83806d470 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc83d800640 1 --2- 172.21.15.46:0/746287175 >> v2:172.21.15.73:3300/0 conn(0x7fc83815cad0 0x7fc83806eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc83d800640 1 -- 172.21.15.46:0/746287175 >> 172.21.15.46:0/746287175 conn(0x7fc838098a40 msgr2=0x7fc8380b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc83d800640 1 -- 172.21.15.46:0/746287175 shutdown_connections 2024-03-29T08:24:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc83d800640 1 -- 172.21.15.46:0/746287175 wait complete. 2024-03-29T08:24:29.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc83d800640 1 Processor -- start 2024-03-29T08:24:29.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc83d800640 1 -- start start 2024-03-29T08:24:29.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc83d800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc83806a4a0 0x7fc83814f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc83d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc83806eff0 0x7fc83815dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc83d800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc83815cad0 0x7fc8380ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc83d800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc838160c20 con 0x7fc83806eff0 2024-03-29T08:24:29.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc83d800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8381512e0 con 0x7fc83806a4a0 2024-03-29T08:24:29.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc83d800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83815ffa0 con 0x7fc83815cad0 2024-03-29T08:24:29.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc82e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc83806eff0 0x7fc83815dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc836a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc83806a4a0 0x7fc83814f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.429+0000 7fc836a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc83806a4a0 0x7fc83814f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:37812/0 (socket says 172.21.15.46:37812) 2024-03-29T08:24:29.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.433+0000 7fc836a00640 1 -- 172.21.15.46:0/3217886059 learned_addr learned my addr 172.21.15.46:0/3217886059 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:29.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.433+0000 7fc837400640 1 --2- 172.21.15.46:0/3217886059 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc83815cad0 0x7fc8380ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.433+0000 7fc836a00640 1 -- 172.21.15.46:0/3217886059 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc83815cad0 msgr2=0x7fc8380ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.433+0000 7fc836a00640 1 --2- 172.21.15.46:0/3217886059 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc83815cad0 0x7fc8380ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.433+0000 7fc836a00640 1 -- 172.21.15.46:0/3217886059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc83806eff0 msgr2=0x7fc83815dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.433+0000 7fc836a00640 1 --2- 172.21.15.46:0/3217886059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc83806eff0 0x7fc83815dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.433+0000 7fc836a00640 1 -- 172.21.15.46:0/3217886059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8300094d0 con 0x7fc83806a4a0 2024-03-29T08:24:29.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.433+0000 7fc836a00640 1 --2- 172.21.15.46:0/3217886059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc83806a4a0 0x7fc83814f290 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc82800efc0 tx=0x7fc82800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:29.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.433+0000 7fc82f400640 1 -- 172.21.15.46:0/3217886059 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc828009b80 con 0x7fc83806a4a0 2024-03-29T08:24:29.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.433+0000 7fc82f400640 1 -- 172.21.15.46:0/3217886059 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82800f040 con 0x7fc83806a4a0 2024-03-29T08:24:29.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.433+0000 7fc82f400640 1 -- 172.21.15.46:0/3217886059 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8280146a0 con 0x7fc83806a4a0 2024-03-29T08:24:29.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.433+0000 7fc83d800640 1 -- 172.21.15.46:0/3217886059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc838160280 con 0x7fc83806a4a0 2024-03-29T08:24:29.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.433+0000 7fc83d800640 1 -- 172.21.15.46:0/3217886059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc838160840 con 0x7fc83806a4a0 2024-03-29T08:24:29.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.433+0000 7fc83d800640 1 -- 172.21.15.46:0/3217886059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8380cd8f0 con 0x7fc83806a4a0 2024-03-29T08:24:29.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.433+0000 7fc82f400640 1 -- 172.21.15.46:0/3217886059 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc828014800 con 0x7fc83806a4a0 2024-03-29T08:24:29.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.437+0000 7fc82f400640 1 --2- 172.21.15.46:0/3217886059 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc814066690 0x7fc814068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.437+0000 7fc82f400640 1 -- 172.21.15.46:0/3217886059 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc828084ff0 con 0x7fc83806a4a0 2024-03-29T08:24:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.437+0000 7fc82e000640 1 --2- 172.21.15.46:0/3217886059 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc814066690 0x7fc814068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.437+0000 7fc82e000640 1 --2- 172.21.15.46:0/3217886059 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc814066690 0x7fc814068b50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc83815b8c0 tx=0x7fc820007e10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:29.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.437+0000 7fc82f400640 1 -- 172.21.15.46:0/3217886059 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc8280523f0 con 0x7fc83806a4a0 2024-03-29T08:24:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.597+0000 7fc83d800640 1 -- 172.21.15.46:0/3217886059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc83809d700 con 0x7fc83806a4a0 2024-03-29T08:24:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.597+0000 7fc82f400640 1 -- 172.21.15.46:0/3217886059 <== mon.1 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc828055a10 con 0x7fc83806a4a0 2024-03-29T08:24:29.599 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:24:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.601+0000 7fc83d800640 1 -- 172.21.15.46:0/3217886059 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc814066690 msgr2=0x7fc814068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.601+0000 7fc83d800640 1 --2- 172.21.15.46:0/3217886059 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc814066690 0x7fc814068b50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc83815b8c0 tx=0x7fc820007e10 comp rx=0 tx=0).stop 2024-03-29T08:24:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.601+0000 7fc83d800640 1 -- 172.21.15.46:0/3217886059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc83806a4a0 msgr2=0x7fc83814f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.601+0000 7fc83d800640 1 --2- 172.21.15.46:0/3217886059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc83806a4a0 0x7fc83814f290 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc82800efc0 tx=0x7fc82800c490 comp rx=0 tx=0).stop 2024-03-29T08:24:29.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.601+0000 7fc83d800640 1 -- 172.21.15.46:0/3217886059 shutdown_connections 2024-03-29T08:24:29.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.601+0000 7fc83d800640 1 --2- 172.21.15.46:0/3217886059 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc814066690 0x7fc814068b50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.601+0000 7fc83d800640 1 --2- 172.21.15.46:0/3217886059 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc83815cad0 0x7fc8380ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.601+0000 7fc83d800640 1 --2- 172.21.15.46:0/3217886059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc83806eff0 0x7fc83815dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.601+0000 7fc83d800640 1 --2- 172.21.15.46:0/3217886059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc83806a4a0 0x7fc83814f290 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.601+0000 7fc83d800640 1 -- 172.21.15.46:0/3217886059 >> 172.21.15.46:0/3217886059 conn(0x7fc838098a40 msgr2=0x7fc8380b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:29.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.601+0000 7fc83d800640 1 -- 172.21.15.46:0/3217886059 shutdown_connections 2024-03-29T08:24:29.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.601+0000 7fc83d800640 1 -- 172.21.15.46:0/3217886059 wait complete. 2024-03-29T08:24:29.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:24:29.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.761+0000 7f836b800640 1 Processor -- start 2024-03-29T08:24:29.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.761+0000 7f836b800640 1 -- start start 2024-03-29T08:24:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.761+0000 7f836b800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f836415ad20 0x7f83640cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.761+0000 7f836b800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83640b0ab0 con 0x7f836415a940 2024-03-29T08:24:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.761+0000 7f836b800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83640b0d90 con 0x7f83640b0540 2024-03-29T08:24:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.761+0000 7f836b800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83640b1070 con 0x7f836415ad20 2024-03-29T08:24:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.761+0000 7f8368e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f836415a940 0x7f83640afb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57768/0 (socket says 172.21.15.46:57768) 2024-03-29T08:24:29.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.761+0000 7f8368e00640 1 -- 172.21.15.46:0/549794480 learned_addr learned my addr 172.21.15.46:0/549794480 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.761+0000 7f835be00640 1 --2- 172.21.15.46:0/549794480 >> v2:172.21.15.73:3300/0 conn(0x7f836415ad20 0x7f83640cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.761+0000 7f8363e00640 1 -- 172.21.15.46:0/549794480 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341163735 0 0) 0x7f83640b0ab0 con 0x7f836415a940 2024-03-29T08:24:29.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.761+0000 7f8363e00640 1 -- 172.21.15.46:0/549794480 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8348003530 con 0x7f836415a940 2024-03-29T08:24:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.761+0000 7f8363e00640 1 -- 172.21.15.46:0/549794480 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474812016 0 0) 0x7f83640b0d90 con 0x7f83640b0540 2024-03-29T08:24:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.761+0000 7f8363e00640 1 -- 172.21.15.46:0/549794480 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83640b0ab0 con 0x7f83640b0540 2024-03-29T08:24:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.761+0000 7f835be00640 1 -- 172.21.15.46:0/549794480 >> v1:172.21.15.73:6789/0 conn(0x7f83640b0540 legacy=0x7f83640cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.761+0000 7f835be00640 1 -- 172.21.15.46:0/549794480 >> v1:172.21.15.46:6789/0 conn(0x7f836415a940 legacy=0x7f83640afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.761+0000 7f835be00640 1 -- 172.21.15.46:0/549794480 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83640b1350 con 0x7f836415ad20 2024-03-29T08:24:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.765+0000 7f835be00640 1 --2- 172.21.15.46:0/549794480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f836415ad20 0x7f83640cecb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f834c0097c0 tx=0x7f834c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8fe735cae98bad5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.765+0000 7f8363e00640 1 -- 172.21.15.46:0/549794480 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c0ab600 con 0x7f836415ad20 2024-03-29T08:24:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.765+0000 7f8363e00640 1 -- 172.21.15.46:0/549794480 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834c0a7050 con 0x7f836415ad20 2024-03-29T08:24:29.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.765+0000 7f8363e00640 1 -- 172.21.15.46:0/549794480 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c0ab980 con 0x7f836415ad20 2024-03-29T08:24:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.765+0000 7f836b800640 1 -- 172.21.15.46:0/549794480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f836415ad20 msgr2=0x7f83640cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.765+0000 7f836b800640 1 --2- 172.21.15.46:0/549794480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f836415ad20 0x7f83640cecb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f834c0097c0 tx=0x7f834c01c3e0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.765+0000 7f836b800640 1 -- 172.21.15.46:0/549794480 shutdown_connections 2024-03-29T08:24:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.765+0000 7f836b800640 1 --2- 172.21.15.46:0/549794480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f836415ad20 0x7f83640cecb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.765+0000 7f836b800640 1 -- 172.21.15.46:0/549794480 >> 172.21.15.46:0/549794480 conn(0x7f8364089fd0 msgr2=0x7f836408a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.765+0000 7f836b800640 1 -- 172.21.15.46:0/549794480 shutdown_connections 2024-03-29T08:24:29.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.765+0000 7f836b800640 1 -- 172.21.15.46:0/549794480 wait complete. 2024-03-29T08:24:29.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f836b800640 1 Processor -- start 2024-03-29T08:24:29.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f836b800640 1 -- start start 2024-03-29T08:24:29.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f836b800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f83640b0540 0x7f8364159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f836b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415a940 0x7f83640c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f836b800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f836415ad20 0x7f83640c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f836b800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83640c6dc0 con 0x7f836415a940 2024-03-29T08:24:29.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f836b800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8364156ef0 con 0x7f836415ad20 2024-03-29T08:24:29.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f836b800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8364156ba0 con 0x7f83640b0540 2024-03-29T08:24:29.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f835be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415a940 0x7f83640c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f835be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415a940 0x7f83640c87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48938/0 (socket says 172.21.15.46:48938) 2024-03-29T08:24:29.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f835be00640 1 -- 172.21.15.46:0/1385980643 learned_addr learned my addr 172.21.15.46:0/1385980643 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:29.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f8368e00640 1 --2- 172.21.15.46:0/1385980643 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f83640b0540 0x7f8364159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f8369800640 1 --2- 172.21.15.46:0/1385980643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f836415ad20 0x7f83640c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f835be00640 1 -- 172.21.15.46:0/1385980643 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f83640b0540 msgr2=0x7f8364159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f835be00640 1 --2- 172.21.15.46:0/1385980643 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f83640b0540 0x7f8364159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f835be00640 1 -- 172.21.15.46:0/1385980643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f836415ad20 msgr2=0x7f83640c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f835be00640 1 --2- 172.21.15.46:0/1385980643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f836415ad20 0x7f83640c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f835be00640 1 -- 172.21.15.46:0/1385980643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c009420 con 0x7f836415a940 2024-03-29T08:24:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f8369800640 1 --2- 172.21.15.46:0/1385980643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f836415ad20 0x7f83640c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:24:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f835be00640 1 --2- 172.21.15.46:0/1385980643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415a940 0x7f83640c87f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f834c018ff0 tx=0x7f834c019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f8361600640 1 -- 172.21.15.46:0/1385980643 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c0b4840 con 0x7f836415a940 2024-03-29T08:24:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f8361600640 1 -- 172.21.15.46:0/1385980643 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834c0b49c0 con 0x7f836415a940 2024-03-29T08:24:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f8361600640 1 -- 172.21.15.46:0/1385980643 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c0b4d70 con 0x7f836415a940 2024-03-29T08:24:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f836b800640 1 -- 172.21.15.46:0/1385980643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83640b5e70 con 0x7f836415a940 2024-03-29T08:24:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.769+0000 7f836b800640 1 -- 172.21.15.46:0/1385980643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8364158f70 con 0x7f836415a940 2024-03-29T08:24:29.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.773+0000 7f836b800640 1 -- 172.21.15.46:0/1385980643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8364066c90 con 0x7f836415a940 2024-03-29T08:24:29.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.773+0000 7f8361600640 1 -- 172.21.15.46:0/1385980643 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f834c0b2040 con 0x7f836415a940 2024-03-29T08:24:29.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.773+0000 7f8361600640 1 --2- 172.21.15.46:0/1385980643 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f83400666c0 0x7f8340068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:29.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.773+0000 7f8361600640 1 -- 172.21.15.46:0/1385980643 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f834c11e490 con 0x7f836415a940 2024-03-29T08:24:29.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.773+0000 7f8368e00640 1 --2- 172.21.15.46:0/1385980643 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f83400666c0 0x7f8340068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:29.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.773+0000 7f8368e00640 1 --2- 172.21.15.46:0/1385980643 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f83400666c0 0x7f8340068b80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8354000f70 tx=0x7f8354009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:29.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.777+0000 7f8361600640 1 -- 172.21.15.46:0/1385980643 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f834c0eb6e0 con 0x7f836415a940 2024-03-29T08:24:29.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.969+0000 7f836b800640 1 -- 172.21.15.46:0/1385980643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83640cd550 con 0x7f836415a940 2024-03-29T08:24:29.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.969+0000 7f8361600640 1 -- 172.21.15.46:0/1385980643 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f834c0b0020 con 0x7f836415a940 2024-03-29T08:24:29.971 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:29.971 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3456771376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3456771376},{"type":"v1","addr":"172.21.15.73:6836","nonce":3456771376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2671538025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2671538025},{"type":"v1","addr":"172.21.15.73:6837","nonce":2671538025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3196015693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3196015693},{"type":"v1","addr":"172.21.15.46:6833","nonce":3196015693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:22:43.605457+0000","modified":"2024-03-29T08:23:53.005710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":16,"addr":"172.21.15.46:6835/1672348161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1672348161},{"type":"v1","addr":"172.21.15.46:6835","nonce":1672348161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-29T08:24:29.971 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 49 2024-03-29T08:24:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.973+0000 7f836b800640 1 -- 172.21.15.46:0/1385980643 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f83400666c0 msgr2=0x7f8340068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.973+0000 7f836b800640 1 --2- 172.21.15.46:0/1385980643 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f83400666c0 0x7f8340068b80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8354000f70 tx=0x7f8354009040 comp rx=0 tx=0).stop 2024-03-29T08:24:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.973+0000 7f836b800640 1 -- 172.21.15.46:0/1385980643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415a940 msgr2=0x7f83640c87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.973+0000 7f836b800640 1 --2- 172.21.15.46:0/1385980643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415a940 0x7f83640c87f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f834c018ff0 tx=0x7f834c019180 comp rx=0 tx=0).stop 2024-03-29T08:24:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.973+0000 7f836b800640 1 -- 172.21.15.46:0/1385980643 shutdown_connections 2024-03-29T08:24:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.973+0000 7f836b800640 1 --2- 172.21.15.46:0/1385980643 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f83400666c0 0x7f8340068b80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.973+0000 7f836b800640 1 --2- 172.21.15.46:0/1385980643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f836415ad20 0x7f83640c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.973+0000 7f836b800640 1 --2- 172.21.15.46:0/1385980643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415a940 0x7f83640c87f0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.973+0000 7f836b800640 1 --2- 172.21.15.46:0/1385980643 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f83640b0540 0x7f8364159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.973+0000 7f836b800640 1 -- 172.21.15.46:0/1385980643 >> 172.21.15.46:0/1385980643 conn(0x7f8364089fd0 msgr2=0x7f83640cc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.973+0000 7f836b800640 1 -- 172.21.15.46:0/1385980643 shutdown_connections 2024-03-29T08:24:29.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:29.973+0000 7f836b800640 1 -- 172.21.15.46:0/1385980643 wait complete. 2024-03-29T08:24:29.992 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:24:30.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.133+0000 7f7b11e00640 1 Processor -- start 2024-03-29T08:24:30.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.133+0000 7f7b11e00640 1 -- start start 2024-03-29T08:24:30.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.133+0000 7f7b11e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7b0c0cf3c0 0x7f7b0c165660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.133+0000 7f7b11e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0c0b0ab0 con 0x7f7b0c15ad20 2024-03-29T08:24:30.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.133+0000 7f7b11e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0c0b0d90 con 0x7f7b0c15a940 2024-03-29T08:24:30.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.133+0000 7f7b11e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0c0b1070 con 0x7f7b0c0cf3c0 2024-03-29T08:24:30.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.133+0000 7f7b0b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7b0c0cf3c0 0x7f7b0c165660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.133+0000 7f7b0b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7b0c0cf3c0 0x7f7b0c165660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48948/0 (socket says 172.21.15.46:48948) 2024-03-29T08:24:30.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.133+0000 7f7b0b400640 1 -- 172.21.15.46:0/3078885734 learned_addr learned my addr 172.21.15.46:0/3078885734 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:30.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.133+0000 7f7b0b400640 1 -- 172.21.15.46:0/3078885734 >> v1:172.21.15.73:6789/0 conn(0x7f7b0c15a940 legacy=0x7f7b0c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.133+0000 7f7b0b400640 1 -- 172.21.15.46:0/3078885734 >> v1:172.21.15.46:6789/0 conn(0x7f7b0c15ad20 legacy=0x7f7b0c0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.133+0000 7f7b0b400640 1 -- 172.21.15.46:0/3078885734 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0c0b1350 con 0x7f7b0c0cf3c0 2024-03-29T08:24:30.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.133+0000 7f7b0a000640 1 -- 172.21.15.46:0/3078885734 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 998901872 0 0) 0x7f7b0c0b0ab0 con 0x7f7b0c15ad20 2024-03-29T08:24:30.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.137+0000 7f7b0b400640 1 --2- 172.21.15.46:0/3078885734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c0cf3c0 0x7f7b0c165660 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7b04009e50 tx=0x7f7b0401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c95048bc46e60d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:30.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.137+0000 7f7b0a000640 1 -- 172.21.15.46:0/3078885734 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b040a5850 con 0x7f7b0c0cf3c0 2024-03-29T08:24:30.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.137+0000 7f7b0a000640 1 -- 172.21.15.46:0/3078885734 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b040a59b0 con 0x7f7b0c0cf3c0 2024-03-29T08:24:30.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.137+0000 7f7b0a000640 1 -- 172.21.15.46:0/3078885734 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b040a5d40 con 0x7f7b0c0cf3c0 2024-03-29T08:24:30.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.137+0000 7f7b11e00640 1 -- 172.21.15.46:0/3078885734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c0cf3c0 msgr2=0x7f7b0c165660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.137+0000 7f7b11e00640 1 --2- 172.21.15.46:0/3078885734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c0cf3c0 0x7f7b0c165660 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7b04009e50 tx=0x7f7b0401c970 comp rx=0 tx=0).stop 2024-03-29T08:24:30.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.137+0000 7f7b11e00640 1 -- 172.21.15.46:0/3078885734 shutdown_connections 2024-03-29T08:24:30.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.137+0000 7f7b11e00640 1 --2- 172.21.15.46:0/3078885734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c0cf3c0 0x7f7b0c165660 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.137+0000 7f7b11e00640 1 -- 172.21.15.46:0/3078885734 >> 172.21.15.46:0/3078885734 conn(0x7f7b0c089fd0 msgr2=0x7f7b0c08a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:30.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.137+0000 7f7b11e00640 1 -- 172.21.15.46:0/3078885734 shutdown_connections 2024-03-29T08:24:30.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.137+0000 7f7b11e00640 1 -- 172.21.15.46:0/3078885734 wait complete. 2024-03-29T08:24:30.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b11e00640 1 Processor -- start 2024-03-29T08:24:30.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b11e00640 1 -- start start 2024-03-29T08:24:30.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b11e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b0c0cf3c0 0x7f7b0c14a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b11e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c15a940 0x7f7b0c15b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b11e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b0c15ad20 0x7f7b0c1521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b11e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c1570a0 con 0x7f7b0c15a940 2024-03-29T08:24:30.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b11e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c152f60 con 0x7f7b0c0cf3c0 2024-03-29T08:24:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b11e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0c157560 con 0x7f7b0c15ad20 2024-03-29T08:24:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b03e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c15a940 0x7f7b0c15b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b03e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c15a940 0x7f7b0c15b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48952/0 (socket says 172.21.15.46:48952) 2024-03-29T08:24:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b03e00640 1 -- 172.21.15.46:0/1662035104 learned_addr learned my addr 172.21.15.46:0/1662035104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b0b400640 1 --2- 172.21.15.46:0/1662035104 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b0c15ad20 0x7f7b0c1521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b0aa00640 1 --2- 172.21.15.46:0/1662035104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b0c0cf3c0 0x7f7b0c14a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b03e00640 1 -- 172.21.15.46:0/1662035104 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b0c15ad20 msgr2=0x7f7b0c1521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b03e00640 1 --2- 172.21.15.46:0/1662035104 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b0c15ad20 0x7f7b0c1521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b03e00640 1 -- 172.21.15.46:0/1662035104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b0c0cf3c0 msgr2=0x7f7b0c14a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b03e00640 1 --2- 172.21.15.46:0/1662035104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b0c0cf3c0 0x7f7b0c14a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b03e00640 1 -- 172.21.15.46:0/1662035104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b04009b30 con 0x7f7b0c15a940 2024-03-29T08:24:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b0aa00640 1 --2- 172.21.15.46:0/1662035104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b0c0cf3c0 0x7f7b0c14a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b03e00640 1 --2- 172.21.15.46:0/1662035104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c15a940 0x7f7b0c15b120 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7af4001ec0 tx=0x7f7af400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b03400640 1 -- 172.21.15.46:0/1662035104 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40054e0 con 0x7f7b0c15a940 2024-03-29T08:24:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b03400640 1 -- 172.21.15.46:0/1662035104 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af4003600 con 0x7f7b0c15a940 2024-03-29T08:24:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b03400640 1 -- 172.21.15.46:0/1662035104 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40039b0 con 0x7f7b0c15a940 2024-03-29T08:24:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b11e00640 1 -- 172.21.15.46:0/1662035104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0c1566f0 con 0x7f7b0c15a940 2024-03-29T08:24:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.141+0000 7f7b11e00640 1 -- 172.21.15.46:0/1662035104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0c157330 con 0x7f7b0c15a940 2024-03-29T08:24:30.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.145+0000 7f7b11e00640 1 -- 172.21.15.46:0/1662035104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0c0add10 con 0x7f7b0c15a940 2024-03-29T08:24:30.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.145+0000 7f7b03400640 1 -- 172.21.15.46:0/1662035104 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7af4003b10 con 0x7f7b0c15a940 2024-03-29T08:24:30.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.145+0000 7f7b03400640 1 --2- 172.21.15.46:0/1662035104 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ae80666c0 0x7f7ae8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.145+0000 7f7b03400640 1 -- 172.21.15.46:0/1662035104 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f7af40837b0 con 0x7f7b0c15a940 2024-03-29T08:24:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.145+0000 7f7b0aa00640 1 --2- 172.21.15.46:0/1662035104 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ae80666c0 0x7f7ae8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.145+0000 7f7b0aa00640 1 --2- 172.21.15.46:0/1662035104 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ae80666c0 0x7f7ae8068b80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7afc001710 tx=0x7f7afc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:30.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.149+0000 7f7b03400640 1 -- 172.21.15.46:0/1662035104 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7b0c0add10 con 0x7f7b0c15a940 2024-03-29T08:24:30.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.309+0000 7f7b11e00640 1 -- 172.21.15.46:0/1662035104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b0c0add10 con 0x7f7b0c15a940 2024-03-29T08:24:30.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.313+0000 7f7b03400640 1 -- 172.21.15.46:0/1662035104 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16213 (secure 0 0 0) 0x7f7b0c0add10 con 0x7f7b0c15a940 2024-03-29T08:24:30.312 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:30.312 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:23:02.390005+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:22:42.147364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-29T08:22:42.602168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1f","mappings":[{"from":2,"to":7}]},{"pgid":"11.27","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/9617787":"2024-03-30T08:23:02.389981+0000","172.21.15.46:6834/9617787":"2024-03-30T08:23:02.389981+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:24:30.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.313+0000 7f7b11e00640 1 -- 172.21.15.46:0/1662035104 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ae80666c0 msgr2=0x7f7ae8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.313+0000 7f7b11e00640 1 --2- 172.21.15.46:0/1662035104 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ae80666c0 0x7f7ae8068b80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7afc001710 tx=0x7f7afc009040 comp rx=0 tx=0).stop 2024-03-29T08:24:30.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.313+0000 7f7b11e00640 1 -- 172.21.15.46:0/1662035104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c15a940 msgr2=0x7f7b0c15b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.313+0000 7f7b11e00640 1 --2- 172.21.15.46:0/1662035104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c15a940 0x7f7b0c15b120 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7af4001ec0 tx=0x7f7af400c8e0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.313+0000 7f7b11e00640 1 -- 172.21.15.46:0/1662035104 shutdown_connections 2024-03-29T08:24:30.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.313+0000 7f7b11e00640 1 --2- 172.21.15.46:0/1662035104 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ae80666c0 0x7f7ae8068b80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.313+0000 7f7b11e00640 1 --2- 172.21.15.46:0/1662035104 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b0c15ad20 0x7f7b0c1521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.313+0000 7f7b11e00640 1 --2- 172.21.15.46:0/1662035104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b0c15a940 0x7f7b0c15b120 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.313+0000 7f7b11e00640 1 --2- 172.21.15.46:0/1662035104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b0c0cf3c0 0x7f7b0c14a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.313+0000 7f7b11e00640 1 -- 172.21.15.46:0/1662035104 >> 172.21.15.46:0/1662035104 conn(0x7f7b0c089fd0 msgr2=0x7f7b0c0cb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:30.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.313+0000 7f7b11e00640 1 -- 172.21.15.46:0/1662035104 shutdown_connections 2024-03-29T08:24:30.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.313+0000 7f7b11e00640 1 -- 172.21.15.46:0/1662035104 wait complete. 2024-03-29T08:24:30.334 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T08:24:30.334 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:24:30.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.473+0000 7fa26be00640 1 Processor -- start 2024-03-29T08:24:30.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.473+0000 7fa26be00640 1 -- start start 2024-03-29T08:24:30.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.477+0000 7fa26be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa26415cd90 0x7fa26406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.477+0000 7fa26be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa26415d940 0x7fa2641597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.477+0000 7fa26be00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa26406d130 con 0x7fa2641593d0 2024-03-29T08:24:30.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.477+0000 7fa26be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26415b8f0 con 0x7fa26415d940 2024-03-29T08:24:30.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.477+0000 7fa26be00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26415ad50 con 0x7fa26415cd90 2024-03-29T08:24:30.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.477+0000 7fa269e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa26415d940 0x7fa2641597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.477+0000 7fa269e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa26415d940 0x7fa2641597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48964/0 (socket says 172.21.15.46:48964) 2024-03-29T08:24:30.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.477+0000 7fa269e00640 1 -- 172.21.15.46:0/3386748379 learned_addr learned my addr 172.21.15.46:0/3386748379 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:30.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.477+0000 7fa269400640 1 --1- 172.21.15.46:0/3386748379 >> v1:172.21.15.46:6789/0 conn(0x7fa2641593d0 0x7fa26415d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57796/0 (socket says 172.21.15.46:57796) 2024-03-29T08:24:30.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.477+0000 7fa268a00640 1 --2- 172.21.15.46:0/3386748379 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa26415cd90 0x7fa26406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.477+0000 7fa269e00640 1 -- 172.21.15.46:0/3386748379 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa26415cd90 msgr2=0x7fa26406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:30.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.477+0000 7fa269e00640 1 --2- 172.21.15.46:0/3386748379 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa26415cd90 0x7fa26406cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.477+0000 7fa269e00640 1 -- 172.21.15.46:0/3386748379 >> v1:172.21.15.46:6789/0 conn(0x7fa2641593d0 legacy=0x7fa26415d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.477+0000 7fa269e00640 1 -- 172.21.15.46:0/3386748379 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26415afc0 con 0x7fa26415d940 2024-03-29T08:24:30.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.477+0000 7fa263e00640 1 -- 172.21.15.46:0/3386748379 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1636151455 0 0) 0x7fa26406d130 con 0x7fa2641593d0 2024-03-29T08:24:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.481+0000 7fa269e00640 1 --2- 172.21.15.46:0/3386748379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa26415d940 0x7fa2641597b0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00d150 tx=0x7fa25c01ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a2aaaca85cea6a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.481+0000 7fa263e00640 1 -- 172.21.15.46:0/3386748379 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c0a7d00 con 0x7fa26415d940 2024-03-29T08:24:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.481+0000 7fa263e00640 1 -- 172.21.15.46:0/3386748379 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25c0a7e60 con 0x7fa26415d940 2024-03-29T08:24:30.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.481+0000 7fa263e00640 1 -- 172.21.15.46:0/3386748379 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c00b2f0 con 0x7fa26415d940 2024-03-29T08:24:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.481+0000 7fa26be00640 1 -- 172.21.15.46:0/3386748379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa26415d940 msgr2=0x7fa2641597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.481+0000 7fa26be00640 1 --2- 172.21.15.46:0/3386748379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa26415d940 0x7fa2641597b0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00d150 tx=0x7fa25c01ec30 comp rx=0 tx=0).stop 2024-03-29T08:24:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.481+0000 7fa26be00640 1 -- 172.21.15.46:0/3386748379 shutdown_connections 2024-03-29T08:24:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.481+0000 7fa26be00640 1 --2- 172.21.15.46:0/3386748379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa26415d940 0x7fa2641597b0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.481+0000 7fa26be00640 1 --2- 172.21.15.46:0/3386748379 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa26415cd90 0x7fa26406cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.481+0000 7fa26be00640 1 -- 172.21.15.46:0/3386748379 >> 172.21.15.46:0/3386748379 conn(0x7fa264066110 msgr2=0x7fa264069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.481+0000 7fa26be00640 1 -- 172.21.15.46:0/3386748379 shutdown_connections 2024-03-29T08:24:30.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.481+0000 7fa26be00640 1 -- 172.21.15.46:0/3386748379 wait complete. 2024-03-29T08:24:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.481+0000 7fa26be00640 1 Processor -- start 2024-03-29T08:24:30.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.481+0000 7fa26be00640 1 -- start start 2024-03-29T08:24:30.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa26be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa2641593d0 0x7fa2641576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa26be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa26415cd90 0x7fa264152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa26be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa26415d940 0x7fa264153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa26be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa264157bf0 con 0x7fa26415cd90 2024-03-29T08:24:30.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa26be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa264156610 con 0x7fa26415d940 2024-03-29T08:24:30.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa26be00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2641568f0 con 0x7fa2641593d0 2024-03-29T08:24:30.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa268a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa26415cd90 0x7fa264152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa268a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa26415cd90 0x7fa264152490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48972/0 (socket says 172.21.15.46:48972) 2024-03-29T08:24:30.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa268a00640 1 -- 172.21.15.46:0/3809038287 learned_addr learned my addr 172.21.15.46:0/3809038287 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:30.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa269400640 1 --2- 172.21.15.46:0/3809038287 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa2641593d0 0x7fa2641576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa269e00640 1 --2- 172.21.15.46:0/3809038287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa26415d940 0x7fa264153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa268a00640 1 -- 172.21.15.46:0/3809038287 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa2641593d0 msgr2=0x7fa2641576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa268a00640 1 --2- 172.21.15.46:0/3809038287 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa2641593d0 0x7fa2641576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa268a00640 1 -- 172.21.15.46:0/3809038287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa26415d940 msgr2=0x7fa264153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa268a00640 1 --2- 172.21.15.46:0/3809038287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa26415d940 0x7fa264153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa268a00640 1 -- 172.21.15.46:0/3809038287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c00bd40 con 0x7fa26415cd90 2024-03-29T08:24:30.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa268a00640 1 --2- 172.21.15.46:0/3809038287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa26415cd90 0x7fa264152490 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fa25000e870 tx=0x7fa25000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:30.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa261600640 1 -- 172.21.15.46:0/3809038287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250009510 con 0x7fa26415cd90 2024-03-29T08:24:30.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa261600640 1 -- 172.21.15.46:0/3809038287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25000eec0 con 0x7fa26415cd90 2024-03-29T08:24:30.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa26be00640 1 -- 172.21.15.46:0/3809038287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa264156bd0 con 0x7fa26415cd90 2024-03-29T08:24:30.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa261600640 1 -- 172.21.15.46:0/3809038287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25000cb40 con 0x7fa26415cd90 2024-03-29T08:24:30.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa26be00640 1 -- 172.21.15.46:0/3809038287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa264157190 con 0x7fa26415cd90 2024-03-29T08:24:30.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa26be00640 1 -- 172.21.15.46:0/3809038287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa264070b70 con 0x7fa26415cd90 2024-03-29T08:24:30.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.485+0000 7fa261600640 1 -- 172.21.15.46:0/3809038287 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa25000cca0 con 0x7fa26415cd90 2024-03-29T08:24:30.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.489+0000 7fa261600640 1 --2- 172.21.15.46:0/3809038287 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa244066690 0x7fa244068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.489+0000 7fa261600640 1 -- 172.21.15.46:0/3809038287 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa250084280 con 0x7fa26415cd90 2024-03-29T08:24:30.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.489+0000 7fa269400640 1 --2- 172.21.15.46:0/3809038287 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa244066690 0x7fa244068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.489+0000 7fa269400640 1 --2- 172.21.15.46:0/3809038287 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa244066690 0x7fa244068b50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa258000f70 tx=0x7fa258007df0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:30.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.493+0000 7fa261600640 1 -- 172.21.15.46:0/3809038287 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa250051600 con 0x7fa26415cd90 2024-03-29T08:24:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.689+0000 7fa26be00640 1 -- 172.21.15.46:0/3809038287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa264066c90 con 0x7fa26415cd90 2024-03-29T08:24:30.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.689+0000 7fa261600640 1 -- 172.21.15.46:0/3809038287 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4738 (secure 0 0 0) 0x7fa250054c20 con 0x7fa26415cd90 2024-03-29T08:24:30.689 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:30.689 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3456771376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3456771376},{"type":"v1","addr":"172.21.15.73:6836","nonce":3456771376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2671538025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2671538025},{"type":"v1","addr":"172.21.15.73:6837","nonce":2671538025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3196015693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3196015693},{"type":"v1","addr":"172.21.15.46:6833","nonce":3196015693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:22:43.605457+0000","modified":"2024-03-29T08:23:53.005710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":16,"addr":"172.21.15.46:6835/1672348161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1672348161},{"type":"v1","addr":"172.21.15.46:6835","nonce":1672348161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-29T08:24:30.689 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 49 2024-03-29T08:24:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.689+0000 7fa26be00640 1 -- 172.21.15.46:0/3809038287 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa244066690 msgr2=0x7fa244068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.689+0000 7fa26be00640 1 --2- 172.21.15.46:0/3809038287 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa244066690 0x7fa244068b50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa258000f70 tx=0x7fa258007df0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.689+0000 7fa26be00640 1 -- 172.21.15.46:0/3809038287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa26415cd90 msgr2=0x7fa264152490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.689+0000 7fa26be00640 1 --2- 172.21.15.46:0/3809038287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa26415cd90 0x7fa264152490 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fa25000e870 tx=0x7fa25000ec70 comp rx=0 tx=0).stop 2024-03-29T08:24:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.689+0000 7fa26be00640 1 -- 172.21.15.46:0/3809038287 shutdown_connections 2024-03-29T08:24:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.689+0000 7fa26be00640 1 --2- 172.21.15.46:0/3809038287 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa244066690 0x7fa244068b50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.689+0000 7fa26be00640 1 --2- 172.21.15.46:0/3809038287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa26415d940 0x7fa264153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.689+0000 7fa26be00640 1 --2- 172.21.15.46:0/3809038287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa26415cd90 0x7fa264152490 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.689+0000 7fa26be00640 1 --2- 172.21.15.46:0/3809038287 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa2641593d0 0x7fa2641576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.689+0000 7fa26be00640 1 -- 172.21.15.46:0/3809038287 >> 172.21.15.46:0/3809038287 conn(0x7fa264066110 msgr2=0x7fa2640ad800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.689+0000 7fa26be00640 1 -- 172.21.15.46:0/3809038287 shutdown_connections 2024-03-29T08:24:30.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.689+0000 7fa26be00640 1 -- 172.21.15.46:0/3809038287 wait complete. 2024-03-29T08:24:30.710 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:24:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.849+0000 7ffa07c00640 1 Processor -- start 2024-03-29T08:24:30.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.849+0000 7ffa07c00640 1 -- start start 2024-03-29T08:24:30.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.849+0000 7ffa07c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa00153970 0x7ffa001611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.849+0000 7ffa07c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ffa00153d50 0x7ffa0015cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.849+0000 7ffa07c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0015d870 con 0x7ffa000ad3e0 2024-03-29T08:24:30.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.849+0000 7ffa07c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0015db50 con 0x7ffa00153970 2024-03-29T08:24:30.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.849+0000 7ffa07c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0015de30 con 0x7ffa00153d50 2024-03-29T08:24:30.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.849+0000 7ffa05200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa00153970 0x7ffa001611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.849+0000 7ffa05200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffa00153970 0x7ffa001611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48978/0 (socket says 172.21.15.46:48978) 2024-03-29T08:24:30.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.849+0000 7ffa05200640 1 -- 172.21.15.46:0/2271007884 learned_addr learned my addr 172.21.15.46:0/2271007884 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:30.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.849+0000 7ffa05c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffa000ad3e0 0x7ffa000ac3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57802/0 (socket says 172.21.15.46:57802) 2024-03-29T08:24:30.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.849+0000 7ff9f7e00640 1 --2- 172.21.15.46:0/2271007884 >> v2:172.21.15.73:3300/0 conn(0x7ffa00153d50 0x7ffa0015cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.849+0000 7ffa05200640 1 -- 172.21.15.46:0/2271007884 >> v2:172.21.15.73:3300/0 conn(0x7ffa00153d50 msgr2=0x7ffa0015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:30.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.849+0000 7ffa05200640 1 --2- 172.21.15.46:0/2271007884 >> v2:172.21.15.73:3300/0 conn(0x7ffa00153d50 0x7ffa0015cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.849+0000 7ffa05200640 1 -- 172.21.15.46:0/2271007884 >> v1:172.21.15.46:6789/0 conn(0x7ffa000ad3e0 legacy=0x7ffa000ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.849+0000 7ffa05200640 1 -- 172.21.15.46:0/2271007884 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa000abea0 con 0x7ffa00153970 2024-03-29T08:24:30.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.849+0000 7ff9ffe00640 1 -- 172.21.15.46:0/2271007884 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3418262008 0 0) 0x7ffa0015d870 con 0x7ffa000ad3e0 2024-03-29T08:24:30.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.853+0000 7ffa05200640 1 --2- 172.21.15.46:0/2271007884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa00153970 0x7ffa001611e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ffa0006e480 tx=0x7ff9f001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f21696638ca5ea server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:30.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.853+0000 7ff9ffe00640 1 -- 172.21.15.46:0/2271007884 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00a7070 con 0x7ffa00153970 2024-03-29T08:24:30.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.853+0000 7ff9ffe00640 1 -- 172.21.15.46:0/2271007884 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f00ab430 con 0x7ffa00153970 2024-03-29T08:24:30.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.853+0000 7ff9ffe00640 1 -- 172.21.15.46:0/2271007884 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00ab7c0 con 0x7ffa00153970 2024-03-29T08:24:30.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.853+0000 7ffa07c00640 1 -- 172.21.15.46:0/2271007884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa00153970 msgr2=0x7ffa001611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.853+0000 7ffa07c00640 1 --2- 172.21.15.46:0/2271007884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa00153970 0x7ffa001611e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ffa0006e480 tx=0x7ff9f001c520 comp rx=0 tx=0).stop 2024-03-29T08:24:30.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.853+0000 7ffa07c00640 1 -- 172.21.15.46:0/2271007884 shutdown_connections 2024-03-29T08:24:30.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.853+0000 7ffa07c00640 1 --2- 172.21.15.46:0/2271007884 >> v2:172.21.15.73:3300/0 conn(0x7ffa00153d50 0x7ffa0015cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.853+0000 7ffa07c00640 1 --2- 172.21.15.46:0/2271007884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa00153970 0x7ffa001611e0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.853+0000 7ffa07c00640 1 -- 172.21.15.46:0/2271007884 >> 172.21.15.46:0/2271007884 conn(0x7ffa00066110 msgr2=0x7ffa00069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:30.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.853+0000 7ffa07c00640 1 -- 172.21.15.46:0/2271007884 shutdown_connections 2024-03-29T08:24:30.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.853+0000 7ffa07c00640 1 -- 172.21.15.46:0/2271007884 wait complete. 2024-03-29T08:24:30.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa07c00640 1 Processor -- start 2024-03-29T08:24:30.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa07c00640 1 -- start start 2024-03-29T08:24:30.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa07c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa000ad3e0 0x7ffa000ae730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa07c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffa00153970 0x7ffa0014fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa07c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0013bde0 0x7ffa00155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa07c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000af3c0 con 0x7ffa0013bde0 2024-03-29T08:24:30.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa07c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000cc330 con 0x7ffa000ad3e0 2024-03-29T08:24:30.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa07c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa001500b0 con 0x7ffa00153970 2024-03-29T08:24:30.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa05c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0013bde0 0x7ffa00155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa05c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0013bde0 0x7ffa00155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48980/0 (socket says 172.21.15.46:48980) 2024-03-29T08:24:30.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa05c00640 1 -- 172.21.15.46:0/1373308165 learned_addr learned my addr 172.21.15.46:0/1373308165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:30.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa05200640 1 --2- 172.21.15.46:0/1373308165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa000ad3e0 0x7ffa000ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ff9f7e00640 1 --2- 172.21.15.46:0/1373308165 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffa00153970 0x7ffa0014fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa05c00640 1 -- 172.21.15.46:0/1373308165 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffa00153970 msgr2=0x7ffa0014fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa05c00640 1 --2- 172.21.15.46:0/1373308165 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffa00153970 0x7ffa0014fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa05c00640 1 -- 172.21.15.46:0/1373308165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa000ad3e0 msgr2=0x7ffa000ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:30.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa05c00640 1 --2- 172.21.15.46:0/1373308165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa000ad3e0 0x7ffa000ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:30.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa05c00640 1 -- 172.21.15.46:0/1373308165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f00094d0 con 0x7ffa0013bde0 2024-03-29T08:24:30.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa05c00640 1 --2- 172.21.15.46:0/1373308165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0013bde0 0x7ffa00155710 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8001570 tx=0x7ff9f800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:30.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ff9fd600640 1 -- 172.21.15.46:0/1373308165 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f800a640 con 0x7ffa0013bde0 2024-03-29T08:24:30.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ff9fd600640 1 -- 172.21.15.46:0/1373308165 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f800a7c0 con 0x7ffa0013bde0 2024-03-29T08:24:30.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa07c00640 1 -- 172.21.15.46:0/1373308165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa000aee00 con 0x7ffa0013bde0 2024-03-29T08:24:30.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ff9fd600640 1 -- 172.21.15.46:0/1373308165 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f800ab70 con 0x7ffa0013bde0 2024-03-29T08:24:30.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.857+0000 7ffa07c00640 1 -- 172.21.15.46:0/1373308165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa000cbf50 con 0x7ffa0013bde0 2024-03-29T08:24:30.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.861+0000 7ffa07c00640 1 -- 172.21.15.46:0/1373308165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa000cbdb0 con 0x7ffa0013bde0 2024-03-29T08:24:30.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.861+0000 7ff9fd600640 1 -- 172.21.15.46:0/1373308165 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff9f800acd0 con 0x7ffa0013bde0 2024-03-29T08:24:30.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.861+0000 7ff9fd600640 1 --2- 172.21.15.46:0/1373308165 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff9dc066690 0x7ff9dc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:30.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.861+0000 7ff9fd600640 1 -- 172.21.15.46:0/1373308165 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff9f8083a10 con 0x7ffa0013bde0 2024-03-29T08:24:30.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.861+0000 7ffa05200640 1 --2- 172.21.15.46:0/1373308165 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff9dc066690 0x7ff9dc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:30.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.861+0000 7ffa05200640 1 --2- 172.21.15.46:0/1373308165 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff9dc066690 0x7ff9dc068b50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00af000 tx=0x7ff9f00b6040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:30.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:30.865+0000 7ff9fd600640 1 -- 172.21.15.46:0/1373308165 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ffa000cbdb0 con 0x7ffa0013bde0 2024-03-29T08:24:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.065+0000 7ffa07c00640 1 -- 172.21.15.46:0/1373308165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa000cc470 con 0x7ffa0013bde0 2024-03-29T08:24:31.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.065+0000 7ff9fd600640 1 -- 172.21.15.46:0/1373308165 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4738 (secure 0 0 0) 0x7ffa000cc470 con 0x7ffa0013bde0 2024-03-29T08:24:31.066 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:31.066 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3456771376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3456771376},{"type":"v1","addr":"172.21.15.73:6836","nonce":3456771376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2671538025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2671538025},{"type":"v1","addr":"172.21.15.73:6837","nonce":2671538025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3196015693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3196015693},{"type":"v1","addr":"172.21.15.46:6833","nonce":3196015693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:22:43.605457+0000","modified":"2024-03-29T08:23:53.005710+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6876},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6876":{"gid":6876,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":16,"addr":"172.21.15.46:6835/1672348161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1672348161},{"type":"v1","addr":"172.21.15.46:6835","nonce":1672348161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-03-29T08:24:31.066 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 49 2024-03-29T08:24:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.069+0000 7ffa07c00640 1 -- 172.21.15.46:0/1373308165 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff9dc066690 msgr2=0x7ff9dc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.069+0000 7ffa07c00640 1 --2- 172.21.15.46:0/1373308165 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff9dc066690 0x7ff9dc068b50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00af000 tx=0x7ff9f00b6040 comp rx=0 tx=0).stop 2024-03-29T08:24:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.069+0000 7ffa07c00640 1 -- 172.21.15.46:0/1373308165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0013bde0 msgr2=0x7ffa00155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.069+0000 7ffa07c00640 1 --2- 172.21.15.46:0/1373308165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0013bde0 0x7ffa00155710 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8001570 tx=0x7ff9f800cca0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.069+0000 7ffa07c00640 1 -- 172.21.15.46:0/1373308165 shutdown_connections 2024-03-29T08:24:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.069+0000 7ffa07c00640 1 --2- 172.21.15.46:0/1373308165 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff9dc066690 0x7ff9dc068b50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.069+0000 7ffa07c00640 1 --2- 172.21.15.46:0/1373308165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffa0013bde0 0x7ffa00155710 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.069+0000 7ffa07c00640 1 --2- 172.21.15.46:0/1373308165 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffa00153970 0x7ffa0014fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.069+0000 7ffa07c00640 1 --2- 172.21.15.46:0/1373308165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa000ad3e0 0x7ffa000ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.069+0000 7ffa07c00640 1 -- 172.21.15.46:0/1373308165 >> 172.21.15.46:0/1373308165 conn(0x7ffa00066110 msgr2=0x7ffa0015fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.069+0000 7ffa07c00640 1 -- 172.21.15.46:0/1373308165 shutdown_connections 2024-03-29T08:24:31.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.069+0000 7ffa07c00640 1 -- 172.21.15.46:0/1373308165 wait complete. 2024-03-29T08:24:31.087 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:24:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1eeb200640 1 Processor -- start 2024-03-29T08:24:31.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1eeb200640 1 -- start start 2024-03-29T08:24:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1eeb200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1eec15a6b0 0x7f1eec15e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1eeb200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1eec0b0220 0x7f1eec0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1eeb200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eec0b0bb0 con 0x7f1eec15b5a0 2024-03-29T08:24:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1eeb200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eec15aa90 con 0x7f1eec15a6b0 2024-03-29T08:24:31.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1eeb200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec0aef50 con 0x7f1eec0b0220 2024-03-29T08:24:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1eea800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1eec0b0220 0x7f1eec0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1eea800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1eec0b0220 0x7f1eec0b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:50090/0 (socket says 172.21.15.46:50090) 2024-03-29T08:24:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1eea800640 1 -- 172.21.15.46:0/1512498638 learned_addr learned my addr 172.21.15.46:0/1512498638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1ee9e00640 1 --2- 172.21.15.46:0/1512498638 >> v2:172.21.15.73:3300/0 conn(0x7f1eec15a6b0 0x7f1eec15e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1ee9400640 1 -- 172.21.15.46:0/1512498638 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3211531901 0 0) 0x7f1ed4001120 con 0x7f1eec15b5a0 2024-03-29T08:24:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1ee9400640 1 -- 172.21.15.46:0/1512498638 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ed0003530 con 0x7f1eec15b5a0 2024-03-29T08:24:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1ee9e00640 1 -- 172.21.15.46:0/1512498638 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1eec0b0220 msgr2=0x7f1eec0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1ee9e00640 1 --2- 172.21.15.46:0/1512498638 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1eec0b0220 0x7f1eec0b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1ee9e00640 1 -- 172.21.15.46:0/1512498638 >> v1:172.21.15.73:6789/0 conn(0x7f1eec15b5a0 legacy=0x7f1eec0caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1ee9e00640 1 -- 172.21.15.46:0/1512498638 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec15f090 con 0x7f1eec15a6b0 2024-03-29T08:24:31.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.229+0000 7f1eea800640 1 --2- 172.21.15.46:0/1512498638 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1eec0b0220 0x7f1eec0b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:24:31.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.233+0000 7f1ee9e00640 1 --2- 172.21.15.46:0/1512498638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec15a6b0 0x7f1eec15e160 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0097c0 tx=0x7f1edc01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccada29e846b7fc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:31.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.233+0000 7f1ee9400640 1 -- 172.21.15.46:0/1512498638 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0ab670 con 0x7f1eec15a6b0 2024-03-29T08:24:31.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.233+0000 7f1ee9400640 1 -- 172.21.15.46:0/1512498638 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc0ab7d0 con 0x7f1eec15a6b0 2024-03-29T08:24:31.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.233+0000 7f1ee9400640 1 -- 172.21.15.46:0/1512498638 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0abb60 con 0x7f1eec15a6b0 2024-03-29T08:24:31.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.233+0000 7f1eeb200640 1 -- 172.21.15.46:0/1512498638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec15a6b0 msgr2=0x7f1eec15e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.233+0000 7f1eeb200640 1 --2- 172.21.15.46:0/1512498638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec15a6b0 0x7f1eec15e160 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0097c0 tx=0x7f1edc01c470 comp rx=0 tx=0).stop 2024-03-29T08:24:31.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.233+0000 7f1eeb200640 1 -- 172.21.15.46:0/1512498638 shutdown_connections 2024-03-29T08:24:31.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.233+0000 7f1eeb200640 1 --2- 172.21.15.46:0/1512498638 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1eec0b0220 0x7f1eec0b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.233+0000 7f1eeb200640 1 --2- 172.21.15.46:0/1512498638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec15a6b0 0x7f1eec15e160 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.233+0000 7f1eeb200640 1 -- 172.21.15.46:0/1512498638 >> 172.21.15.46:0/1512498638 conn(0x7f1eec098a40 msgr2=0x7f1eec0cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:31.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.233+0000 7f1eeb200640 1 -- 172.21.15.46:0/1512498638 shutdown_connections 2024-03-29T08:24:31.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.233+0000 7f1eeb200640 1 -- 172.21.15.46:0/1512498638 wait complete. 2024-03-29T08:24:31.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.233+0000 7f1eeb200640 1 Processor -- start 2024-03-29T08:24:31.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eeb200640 1 -- start start 2024-03-29T08:24:31.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eeb200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec0b0220 0x7f1eec1508b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eeb200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1eec15a6b0 0x7f1eec0c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eeb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eec15b5a0 0x7f1eec1510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eeb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec157d90 con 0x7f1eec15b5a0 2024-03-29T08:24:31.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eeb200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec0c9650 con 0x7f1eec0b0220 2024-03-29T08:24:31.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eeb200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec0c9090 con 0x7f1eec15a6b0 2024-03-29T08:24:31.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eec15b5a0 0x7f1eec1510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eec15b5a0 0x7f1eec1510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48986/0 (socket says 172.21.15.46:48986) 2024-03-29T08:24:31.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eea800640 1 -- 172.21.15.46:0/1900718513 learned_addr learned my addr 172.21.15.46:0/1900718513 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:31.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1ee3e00640 1 --2- 172.21.15.46:0/1900718513 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1eec15a6b0 0x7f1eec0c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1ee9e00640 1 --2- 172.21.15.46:0/1900718513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec0b0220 0x7f1eec1508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eea800640 1 -- 172.21.15.46:0/1900718513 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1eec15a6b0 msgr2=0x7f1eec0c56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eea800640 1 --2- 172.21.15.46:0/1900718513 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1eec15a6b0 0x7f1eec0c56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eea800640 1 -- 172.21.15.46:0/1900718513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec0b0220 msgr2=0x7f1eec1508b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eea800640 1 --2- 172.21.15.46:0/1900718513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec0b0220 0x7f1eec1508b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eea800640 1 -- 172.21.15.46:0/1900718513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee4009b30 con 0x7f1eec15b5a0 2024-03-29T08:24:31.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eea800640 1 --2- 172.21.15.46:0/1900718513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eec15b5a0 0x7f1eec1510f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400efc0 tx=0x7f1ee400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:31.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1ee2000640 1 -- 172.21.15.46:0/1900718513 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4018070 con 0x7f1eec15b5a0 2024-03-29T08:24:31.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1ee2000640 1 -- 172.21.15.46:0/1900718513 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee400f040 con 0x7f1eec15b5a0 2024-03-29T08:24:31.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eeb200640 1 -- 172.21.15.46:0/1900718513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1edc009420 con 0x7f1eec15b5a0 2024-03-29T08:24:31.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1ee2000640 1 -- 172.21.15.46:0/1900718513 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee400ce00 con 0x7f1eec15b5a0 2024-03-29T08:24:31.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eeb200640 1 -- 172.21.15.46:0/1900718513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eec157a20 con 0x7f1eec15b5a0 2024-03-29T08:24:31.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.237+0000 7f1eeb200640 1 -- 172.21.15.46:0/1900718513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eec157f40 con 0x7f1eec15b5a0 2024-03-29T08:24:31.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.241+0000 7f1ee2000640 1 -- 172.21.15.46:0/1900718513 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1ee4014480 con 0x7f1eec15b5a0 2024-03-29T08:24:31.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.241+0000 7f1ee2000640 1 --2- 172.21.15.46:0/1900718513 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1ec80666c0 0x7f1ec8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.241+0000 7f1ee2000640 1 -- 172.21.15.46:0/1900718513 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1ee40843c0 con 0x7f1eec15b5a0 2024-03-29T08:24:31.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.241+0000 7f1ee9e00640 1 --2- 172.21.15.46:0/1900718513 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1ec80666c0 0x7f1ec8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.241+0000 7f1ee9e00640 1 --2- 172.21.15.46:0/1900718513 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1ec80666c0 0x7f1ec8068b80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0af000 tx=0x7f1edc0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:31.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.245+0000 7f1ee2000640 1 -- 172.21.15.46:0/1900718513 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1eec157f40 con 0x7f1eec15b5a0 2024-03-29T08:24:31.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.409+0000 7f1eeb200640 1 -- 172.21.15.46:0/1900718513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1eec04e040 con 0x7f1eec15b5a0 2024-03-29T08:24:31.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.409+0000 7f1ee2000640 1 -- 172.21.15.46:0/1900718513 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16213 (secure 0 0 0) 0x7f1ee40551f0 con 0x7f1eec15b5a0 2024-03-29T08:24:31.410 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:31.410 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:23:02.390005+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:22:42.147364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-03-29T08:22:42.602168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.1f","mappings":[{"from":2,"to":7}]},{"pgid":"11.27","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/9617787":"2024-03-30T08:23:02.389981+0000","172.21.15.46:6834/9617787":"2024-03-30T08:23:02.389981+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:24:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.413+0000 7f1eeb200640 1 -- 172.21.15.46:0/1900718513 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1ec80666c0 msgr2=0x7f1ec8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.413+0000 7f1eeb200640 1 --2- 172.21.15.46:0/1900718513 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1ec80666c0 0x7f1ec8068b80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0af000 tx=0x7f1edc0a7000 comp rx=0 tx=0).stop 2024-03-29T08:24:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.413+0000 7f1eeb200640 1 -- 172.21.15.46:0/1900718513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eec15b5a0 msgr2=0x7f1eec1510f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.413+0000 7f1eeb200640 1 --2- 172.21.15.46:0/1900718513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eec15b5a0 0x7f1eec1510f0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400efc0 tx=0x7f1ee400c490 comp rx=0 tx=0).stop 2024-03-29T08:24:31.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.413+0000 7f1eeb200640 1 -- 172.21.15.46:0/1900718513 shutdown_connections 2024-03-29T08:24:31.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.413+0000 7f1eeb200640 1 --2- 172.21.15.46:0/1900718513 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1ec80666c0 0x7f1ec8068b80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.413+0000 7f1eeb200640 1 --2- 172.21.15.46:0/1900718513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1eec15b5a0 0x7f1eec1510f0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.413+0000 7f1eeb200640 1 --2- 172.21.15.46:0/1900718513 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1eec15a6b0 0x7f1eec0c56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.413+0000 7f1eeb200640 1 --2- 172.21.15.46:0/1900718513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1eec0b0220 0x7f1eec1508b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.413+0000 7f1eeb200640 1 -- 172.21.15.46:0/1900718513 >> 172.21.15.46:0/1900718513 conn(0x7f1eec098a40 msgr2=0x7f1eec0cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:31.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.413+0000 7f1eeb200640 1 -- 172.21.15.46:0/1900718513 shutdown_connections 2024-03-29T08:24:31.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.413+0000 7f1eeb200640 1 -- 172.21.15.46:0/1900718513 wait complete. 2024-03-29T08:24:31.433 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T08:24:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.573+0000 7ff5a2800640 1 Processor -- start 2024-03-29T08:24:31.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.573+0000 7ff5a2800640 1 -- start start 2024-03-29T08:24:31.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.573+0000 7ff5a2800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff59c0afa20 0x7ff59c0cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.573+0000 7ff5a2800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff59c0b0cc0 0x7ff59c0b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.573+0000 7ff5a2800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59c0ce7a0 con 0x7ff59c0b1f40 2024-03-29T08:24:31.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.573+0000 7ff5a2800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59c06b470 con 0x7ff59c0afa20 2024-03-29T08:24:31.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.573+0000 7ff5a2800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c0b2910 con 0x7ff59c0b0cc0 2024-03-29T08:24:31.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.577+0000 7ff5a1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff59c0afa20 0x7ff59c0cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.577+0000 7ff5a1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff59c0afa20 0x7ff59c0cf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49000/0 (socket says 172.21.15.46:49000) 2024-03-29T08:24:31.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.577+0000 7ff5a1400640 1 -- 172.21.15.46:0/1255403904 learned_addr learned my addr 172.21.15.46:0/1255403904 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:31.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.577+0000 7ff5a1e00640 1 --2- 172.21.15.46:0/1255403904 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff59c0b0cc0 0x7ff59c0b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.577+0000 7ff5a1400640 1 -- 172.21.15.46:0/1255403904 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff59c0b0cc0 msgr2=0x7ff59c0b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:31.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.577+0000 7ff5a1400640 1 --2- 172.21.15.46:0/1255403904 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff59c0b0cc0 0x7ff59c0b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.577+0000 7ff5a1400640 1 -- 172.21.15.46:0/1255403904 >> v1:172.21.15.73:6789/0 conn(0x7ff59c0b1f40 legacy=0x7ff59c0b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.577+0000 7ff5a1400640 1 -- 172.21.15.46:0/1255403904 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c1613c0 con 0x7ff59c0afa20 2024-03-29T08:24:31.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.577+0000 7ff5a1400640 1 --2- 172.21.15.46:0/1255403904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff59c0afa20 0x7ff59c0cf470 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff590009870 tx=0x7ff59001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1882729e565c981c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:31.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.577+0000 7ff59be00640 1 -- 172.21.15.46:0/1255403904 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5900ab600 con 0x7ff59c0afa20 2024-03-29T08:24:31.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.577+0000 7ff59be00640 1 -- 172.21.15.46:0/1255403904 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5900a7050 con 0x7ff59c0afa20 2024-03-29T08:24:31.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.577+0000 7ff59be00640 1 -- 172.21.15.46:0/1255403904 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5900ab980 con 0x7ff59c0afa20 2024-03-29T08:24:31.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a2800640 1 -- 172.21.15.46:0/1255403904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff59c0afa20 msgr2=0x7ff59c0cf470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a2800640 1 --2- 172.21.15.46:0/1255403904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff59c0afa20 0x7ff59c0cf470 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff590009870 tx=0x7ff59001c540 comp rx=0 tx=0).stop 2024-03-29T08:24:31.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a2800640 1 -- 172.21.15.46:0/1255403904 shutdown_connections 2024-03-29T08:24:31.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a2800640 1 --2- 172.21.15.46:0/1255403904 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff59c0b0cc0 0x7ff59c0b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a2800640 1 --2- 172.21.15.46:0/1255403904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff59c0afa20 0x7ff59c0cf470 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a2800640 1 -- 172.21.15.46:0/1255403904 >> 172.21.15.46:0/1255403904 conn(0x7ff59c0d04c0 msgr2=0x7ff59c0d08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:31.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a2800640 1 -- 172.21.15.46:0/1255403904 shutdown_connections 2024-03-29T08:24:31.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a2800640 1 -- 172.21.15.46:0/1255403904 wait complete. 2024-03-29T08:24:31.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a2800640 1 Processor -- start 2024-03-29T08:24:31.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a2800640 1 -- start start 2024-03-29T08:24:31.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a2800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff59c0afa20 0x7ff59c15fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff59c0b0cc0 0x7ff59c0b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a2800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff59c0b1f40 0x7ff59c0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a2800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c151db0 con 0x7ff59c0b0cc0 2024-03-29T08:24:31.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a2800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c0b6010 con 0x7ff59c0afa20 2024-03-29T08:24:31.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a2800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c15dc40 con 0x7ff59c0b1f40 2024-03-29T08:24:31.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff59c0b0cc0 0x7ff59c0b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff59c0b0cc0 0x7ff59c0b5ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49014/0 (socket says 172.21.15.46:49014) 2024-03-29T08:24:31.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a0a00640 1 -- 172.21.15.46:0/118463830 learned_addr learned my addr 172.21.15.46:0/118463830 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:31.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a1400640 1 --2- 172.21.15.46:0/118463830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff59c0afa20 0x7ff59c15fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a1e00640 1 --2- 172.21.15.46:0/118463830 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff59c0b1f40 0x7ff59c0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a0a00640 1 -- 172.21.15.46:0/118463830 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff59c0b1f40 msgr2=0x7ff59c0b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a0a00640 1 --2- 172.21.15.46:0/118463830 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff59c0b1f40 0x7ff59c0b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a0a00640 1 -- 172.21.15.46:0/118463830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff59c0afa20 msgr2=0x7ff59c15fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a0a00640 1 --2- 172.21.15.46:0/118463830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff59c0afa20 0x7ff59c15fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a0a00640 1 -- 172.21.15.46:0/118463830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff590009480 con 0x7ff59c0b0cc0 2024-03-29T08:24:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.581+0000 7ff5a0a00640 1 --2- 172.21.15.46:0/118463830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff59c0b0cc0 0x7ff59c0b5ad0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff58800c3c0 tx=0x7ff58800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.585+0000 7ff599600640 1 -- 172.21.15.46:0/118463830 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5880052d0 con 0x7ff59c0b0cc0 2024-03-29T08:24:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.585+0000 7ff599600640 1 -- 172.21.15.46:0/118463830 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff588005430 con 0x7ff59c0b0cc0 2024-03-29T08:24:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.585+0000 7ff599600640 1 -- 172.21.15.46:0/118463830 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588003840 con 0x7ff59c0b0cc0 2024-03-29T08:24:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.585+0000 7ff5a2800640 1 -- 172.21.15.46:0/118463830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59c15df20 con 0x7ff59c0b0cc0 2024-03-29T08:24:31.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.585+0000 7ff5a2800640 1 -- 172.21.15.46:0/118463830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59c0cda70 con 0x7ff59c0b0cc0 2024-03-29T08:24:31.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.585+0000 7ff5a2800640 1 -- 172.21.15.46:0/118463830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff59c151f20 con 0x7ff59c0b0cc0 2024-03-29T08:24:31.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.585+0000 7ff599600640 1 -- 172.21.15.46:0/118463830 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff5880039a0 con 0x7ff59c0b0cc0 2024-03-29T08:24:31.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.585+0000 7ff599600640 1 --2- 172.21.15.46:0/118463830 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff57c066690 0x7ff57c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.585+0000 7ff599600640 1 -- 172.21.15.46:0/118463830 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff5880836e0 con 0x7ff59c0b0cc0 2024-03-29T08:24:31.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.585+0000 7ff5a1400640 1 --2- 172.21.15.46:0/118463830 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff57c066690 0x7ff57c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.589+0000 7ff5a1400640 1 --2- 172.21.15.46:0/118463830 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff57c066690 0x7ff57c068b50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff5900ad000 tx=0x7ff5900a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:31.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.589+0000 7ff599600640 1 -- 172.21.15.46:0/118463830 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff588050a60 con 0x7ff59c0b0cc0 2024-03-29T08:24:31.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.785+0000 7ff5a2800640 1 -- 172.21.15.46:0/118463830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff59c04e040 con 0x7ff59c0b0cc0 2024-03-29T08:24:31.785 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-03-29T08:24:31.785+0000 7fc16ae00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:24:31.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.793+0000 7ff599600640 1 -- 172.21.15.46:0/118463830 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff588054080 con 0x7ff59c0b0cc0 2024-03-29T08:24:31.793 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T08:24:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.797+0000 7ff5a2800640 1 -- 172.21.15.46:0/118463830 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff57c066690 msgr2=0x7ff57c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.797+0000 7ff5a2800640 1 --2- 172.21.15.46:0/118463830 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff57c066690 0x7ff57c068b50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff5900ad000 tx=0x7ff5900a6000 comp rx=0 tx=0).stop 2024-03-29T08:24:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.797+0000 7ff5a2800640 1 -- 172.21.15.46:0/118463830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff59c0b0cc0 msgr2=0x7ff59c0b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.797+0000 7ff5a2800640 1 --2- 172.21.15.46:0/118463830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff59c0b0cc0 0x7ff59c0b5ad0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7ff58800c3c0 tx=0x7ff58800c880 comp rx=0 tx=0).stop 2024-03-29T08:24:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.797+0000 7ff5a2800640 1 -- 172.21.15.46:0/118463830 shutdown_connections 2024-03-29T08:24:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.797+0000 7ff5a2800640 1 --2- 172.21.15.46:0/118463830 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff57c066690 0x7ff57c068b50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.797+0000 7ff5a2800640 1 --2- 172.21.15.46:0/118463830 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff59c0b1f40 0x7ff59c0b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.797+0000 7ff5a2800640 1 --2- 172.21.15.46:0/118463830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff59c0b0cc0 0x7ff59c0b5ad0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.797+0000 7ff5a2800640 1 --2- 172.21.15.46:0/118463830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff59c0afa20 0x7ff59c15fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.797+0000 7ff5a2800640 1 -- 172.21.15.46:0/118463830 >> 172.21.15.46:0/118463830 conn(0x7ff59c0d04c0 msgr2=0x7ff59c0aeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.797+0000 7ff5a2800640 1 -- 172.21.15.46:0/118463830 shutdown_connections 2024-03-29T08:24:31.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.797+0000 7ff5a2800640 1 -- 172.21.15.46:0/118463830 wait complete. 2024-03-29T08:24:31.823 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T08:24:31.863 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-03-29T08:24:31.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.969+0000 7f36cc800640 1 Processor -- start 2024-03-29T08:24:31.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.969+0000 7f36cc800640 1 -- start start 2024-03-29T08:24:31.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.969+0000 7f36cc800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f36c40cbea0 0x7f36c40b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.969+0000 7f36cc800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c40b2360 con 0x7f36c4143eb0 2024-03-29T08:24:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.969+0000 7f36cc800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c415e500 con 0x7f36c415efe0 2024-03-29T08:24:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.969+0000 7f36cc800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c415e7e0 con 0x7f36c40cbea0 2024-03-29T08:24:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.969+0000 7f36c9400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f36c40cbea0 0x7f36c40b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.969+0000 7f36c9400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f36c40cbea0 0x7f36c40b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49024/0 (socket says 172.21.15.46:49024) 2024-03-29T08:24:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.969+0000 7f36c9e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f36c4143eb0 0x7f36c40cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57812/0 (socket says 172.21.15.46:57812) 2024-03-29T08:24:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.969+0000 7f36c9400640 1 -- 172.21.15.46:0/878606751 learned_addr learned my addr 172.21.15.46:0/878606751 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.969+0000 7f36c9400640 1 -- 172.21.15.46:0/878606751 >> v1:172.21.15.73:6789/0 conn(0x7f36c415efe0 legacy=0x7f36c40b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.969+0000 7f36c9400640 1 -- 172.21.15.46:0/878606751 >> v1:172.21.15.46:6789/0 conn(0x7f36c4143eb0 legacy=0x7f36c40cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.969+0000 7f36c9400640 1 -- 172.21.15.46:0/878606751 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c415eac0 con 0x7f36c40cbea0 2024-03-29T08:24:31.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.969+0000 7f36c8a00640 1 -- 172.21.15.46:0/878606751 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309454384 0 0) 0x7f36c40b2360 con 0x7f36c4143eb0 2024-03-29T08:24:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.973+0000 7f36c9400640 1 --2- 172.21.15.46:0/878606751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c40cbea0 0x7f36c40b1630 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009870 tx=0x7f36b401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b17009fd9f8ce29 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.973+0000 7f36c8a00640 1 -- 172.21.15.46:0/878606751 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b40a9640 con 0x7f36c40cbea0 2024-03-29T08:24:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.973+0000 7f36c8a00640 1 -- 172.21.15.46:0/878606751 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b40a97a0 con 0x7f36c40cbea0 2024-03-29T08:24:31.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.973+0000 7f36c8a00640 1 -- 172.21.15.46:0/878606751 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b40a9b30 con 0x7f36c40cbea0 2024-03-29T08:24:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.973+0000 7f36cc800640 1 -- 172.21.15.46:0/878606751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c40cbea0 msgr2=0x7f36c40b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.973+0000 7f36cc800640 1 --2- 172.21.15.46:0/878606751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c40cbea0 0x7f36c40b1630 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009870 tx=0x7f36b401c560 comp rx=0 tx=0).stop 2024-03-29T08:24:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.973+0000 7f36cc800640 1 -- 172.21.15.46:0/878606751 shutdown_connections 2024-03-29T08:24:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.973+0000 7f36cc800640 1 --2- 172.21.15.46:0/878606751 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c40cbea0 0x7f36c40b1630 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.973+0000 7f36cc800640 1 -- 172.21.15.46:0/878606751 >> 172.21.15.46:0/878606751 conn(0x7f36c406a390 msgr2=0x7f36c40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.973+0000 7f36cc800640 1 -- 172.21.15.46:0/878606751 shutdown_connections 2024-03-29T08:24:31.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.973+0000 7f36cc800640 1 -- 172.21.15.46:0/878606751 wait complete. 2024-03-29T08:24:31.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36cc800640 1 Processor -- start 2024-03-29T08:24:31.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36cc800640 1 -- start start 2024-03-29T08:24:31.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36cc800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c40cbea0 0x7f36c415c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36cc800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36c4143eb0 0x7f36c415d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36cc800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36c415efe0 0x7f36c40afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36cc800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c415c930 con 0x7f36c40cbea0 2024-03-29T08:24:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36cc800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c415cc10 con 0x7f36c415efe0 2024-03-29T08:24:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36cc800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c415cef0 con 0x7f36c4143eb0 2024-03-29T08:24:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36c9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c40cbea0 0x7f36c415c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36c9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c40cbea0 0x7f36c415c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49034/0 (socket says 172.21.15.46:49034) 2024-03-29T08:24:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36c9e00640 1 -- 172.21.15.46:0/1274771685 learned_addr learned my addr 172.21.15.46:0/1274771685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36ca800640 1 --2- 172.21.15.46:0/1274771685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36c415efe0 0x7f36c40afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36c9400640 1 --2- 172.21.15.46:0/1274771685 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36c4143eb0 0x7f36c415d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36c9e00640 1 -- 172.21.15.46:0/1274771685 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36c4143eb0 msgr2=0x7f36c415d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36c9e00640 1 --2- 172.21.15.46:0/1274771685 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36c4143eb0 0x7f36c415d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36c9e00640 1 -- 172.21.15.46:0/1274771685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36c415efe0 msgr2=0x7f36c40afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36c9e00640 1 --2- 172.21.15.46:0/1274771685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36c415efe0 0x7f36c40afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36c9e00640 1 -- 172.21.15.46:0/1274771685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b40094d0 con 0x7f36c40cbea0 2024-03-29T08:24:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36c9e00640 1 --2- 172.21.15.46:0/1274771685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c40cbea0 0x7f36c415c350 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f36b800c4a0 tx=0x7f36b800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36be000640 1 -- 172.21.15.46:0/1274771685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b80074e0 con 0x7f36c40cbea0 2024-03-29T08:24:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36be000640 1 -- 172.21.15.46:0/1274771685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b8007640 con 0x7f36c40cbea0 2024-03-29T08:24:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36be000640 1 -- 172.21.15.46:0/1274771685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8004a30 con 0x7f36c40cbea0 2024-03-29T08:24:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36cc800640 1 -- 172.21.15.46:0/1274771685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c415d1d0 con 0x7f36c40cbea0 2024-03-29T08:24:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36cc800640 1 -- 172.21.15.46:0/1274771685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c4141490 con 0x7f36c40cbea0 2024-03-29T08:24:31.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.977+0000 7f36cc800640 1 -- 172.21.15.46:0/1274771685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c4071a70 con 0x7f36c40cbea0 2024-03-29T08:24:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.981+0000 7f36be000640 1 -- 172.21.15.46:0/1274771685 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f36b8004b90 con 0x7f36c40cbea0 2024-03-29T08:24:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.981+0000 7f36be000640 1 --2- 172.21.15.46:0/1274771685 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36a8066690 0x7f36a8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:31.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.981+0000 7f36be000640 1 -- 172.21.15.46:0/1274771685 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f36b80834e0 con 0x7f36c40cbea0 2024-03-29T08:24:31.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.981+0000 7f36c9400640 1 --2- 172.21.15.46:0/1274771685 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36a8066690 0x7f36a8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:31.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.981+0000 7f36c9400640 1 --2- 172.21.15.46:0/1274771685 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36a8066690 0x7f36a8068b50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f36b40ae000 tx=0x7f36b40a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:31.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:31.985+0000 7f36be000640 1 -- 172.21.15.46:0/1274771685 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f36b8050770 con 0x7f36c40cbea0 2024-03-29T08:24:32.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:32.185+0000 7f36cc800640 1 -- 172.21.15.46:0/1274771685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f36c404e120 con 0x7f36c40cbea0 2024-03-29T08:24:32.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:32.797+0000 7f36be000640 1 -- 172.21.15.46:0/1274771685 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7f36b8053d90 con 0x7f36c40cbea0 2024-03-29T08:24:32.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:32.797+0000 7f36cc800640 1 -- 172.21.15.46:0/1274771685 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36a8066690 msgr2=0x7f36a8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:32.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:32.797+0000 7f36cc800640 1 --2- 172.21.15.46:0/1274771685 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36a8066690 0x7f36a8068b50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f36b40ae000 tx=0x7f36b40a7000 comp rx=0 tx=0).stop 2024-03-29T08:24:32.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:32.797+0000 7f36cc800640 1 -- 172.21.15.46:0/1274771685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c40cbea0 msgr2=0x7f36c415c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:32.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:32.797+0000 7f36cc800640 1 --2- 172.21.15.46:0/1274771685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c40cbea0 0x7f36c415c350 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f36b800c4a0 tx=0x7f36b800c960 comp rx=0 tx=0).stop 2024-03-29T08:24:32.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:32.797+0000 7f36cc800640 1 -- 172.21.15.46:0/1274771685 shutdown_connections 2024-03-29T08:24:32.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:32.797+0000 7f36cc800640 1 --2- 172.21.15.46:0/1274771685 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f36a8066690 0x7f36a8068b50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:32.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:32.797+0000 7f36cc800640 1 --2- 172.21.15.46:0/1274771685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36c415efe0 0x7f36c40afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:32.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:32.797+0000 7f36cc800640 1 --2- 172.21.15.46:0/1274771685 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f36c4143eb0 0x7f36c415d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:32.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:32.797+0000 7f36cc800640 1 --2- 172.21.15.46:0/1274771685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c40cbea0 0x7f36c415c350 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:32.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:32.797+0000 7f36cc800640 1 -- 172.21.15.46:0/1274771685 >> 172.21.15.46:0/1274771685 conn(0x7f36c406a390 msgr2=0x7f36c406cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:32.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:32.797+0000 7f36cc800640 1 -- 172.21.15.46:0/1274771685 shutdown_connections 2024-03-29T08:24:32.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:32.797+0000 7f36cc800640 1 -- 172.21.15.46:0/1274771685 wait complete. 2024-03-29T08:24:32.820 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T08:24:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.009+0000 7fb16d200640 1 Processor -- start 2024-03-29T08:24:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.009+0000 7fb16d200640 1 -- start start 2024-03-29T08:24:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.009+0000 7fb16d200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb168140f40 0x7fb1680cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.009+0000 7fb16d200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1680cec80 0x7fb168141770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.009+0000 7fb16d200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb16806cd80 con 0x7fb168141cb0 2024-03-29T08:24:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.009+0000 7fb16d200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb16806d060 con 0x7fb1680cec80 2024-03-29T08:24:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.009+0000 7fb16d200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb16806d340 con 0x7fb168140f40 2024-03-29T08:24:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.009+0000 7fb15fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1680cec80 0x7fb168141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.009+0000 7fb15fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1680cec80 0x7fb168141770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49050/0 (socket says 172.21.15.46:49050) 2024-03-29T08:24:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.009+0000 7fb15fe00640 1 -- 172.21.15.46:0/4025938317 learned_addr learned my addr 172.21.15.46:0/4025938317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.009+0000 7fb165e00640 1 --2- 172.21.15.46:0/4025938317 >> v2:172.21.15.73:3300/0 conn(0x7fb168140f40 0x7fb1680cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.009+0000 7fb15fe00640 1 -- 172.21.15.46:0/4025938317 >> v2:172.21.15.73:3300/0 conn(0x7fb168140f40 msgr2=0x7fb1680cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.009+0000 7fb15fe00640 1 --2- 172.21.15.46:0/4025938317 >> v2:172.21.15.73:3300/0 conn(0x7fb168140f40 0x7fb1680cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.009+0000 7fb15fe00640 1 -- 172.21.15.46:0/4025938317 >> v1:172.21.15.73:6789/0 conn(0x7fb168141cb0 legacy=0x7fb16806db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:33.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.009+0000 7fb15fe00640 1 -- 172.21.15.46:0/4025938317 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16806d620 con 0x7fb1680cec80 2024-03-29T08:24:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.013+0000 7fb15fe00640 1 --2- 172.21.15.46:0/4025938317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1680cec80 0x7fb168141770 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb1680ce460 tx=0x7fb15001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af1a42e50de52c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.013+0000 7fb165400640 1 -- 172.21.15.46:0/4025938317 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1500ab5f0 con 0x7fb1680cec80 2024-03-29T08:24:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.013+0000 7fb165400640 1 -- 172.21.15.46:0/4025938317 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1500ab750 con 0x7fb1680cec80 2024-03-29T08:24:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.013+0000 7fb165400640 1 -- 172.21.15.46:0/4025938317 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1500abae0 con 0x7fb1680cec80 2024-03-29T08:24:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.013+0000 7fb16d200640 1 -- 172.21.15.46:0/4025938317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1680cec80 msgr2=0x7fb168141770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.013+0000 7fb16d200640 1 --2- 172.21.15.46:0/4025938317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1680cec80 0x7fb168141770 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb1680ce460 tx=0x7fb15001c4e0 comp rx=0 tx=0).stop 2024-03-29T08:24:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.013+0000 7fb16d200640 1 -- 172.21.15.46:0/4025938317 shutdown_connections 2024-03-29T08:24:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.013+0000 7fb16d200640 1 --2- 172.21.15.46:0/4025938317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1680cec80 0x7fb168141770 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.013+0000 7fb16d200640 1 --2- 172.21.15.46:0/4025938317 >> v2:172.21.15.73:3300/0 conn(0x7fb168140f40 0x7fb1680cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.013+0000 7fb16d200640 1 -- 172.21.15.46:0/4025938317 >> 172.21.15.46:0/4025938317 conn(0x7fb1680b42d0 msgr2=0x7fb168069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.013+0000 7fb16d200640 1 -- 172.21.15.46:0/4025938317 shutdown_connections 2024-03-29T08:24:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.013+0000 7fb16d200640 1 -- 172.21.15.46:0/4025938317 wait complete. 2024-03-29T08:24:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb16d200640 1 Processor -- start 2024-03-29T08:24:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb16d200640 1 -- start start 2024-03-29T08:24:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb16d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1680cec80 0x7fb16813b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb16d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb168140f40 0x7fb16815a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb16d200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb168141cb0 0x7fb168139ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb16d200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16813a420 con 0x7fb1680cec80 2024-03-29T08:24:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb16d200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16813a610 con 0x7fb168140f40 2024-03-29T08:24:33.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb16d200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb16813a8f0 con 0x7fb168141cb0 2024-03-29T08:24:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb165e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1680cec80 0x7fb16813b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb165e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1680cec80 0x7fb16813b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49056/0 (socket says 172.21.15.46:49056) 2024-03-29T08:24:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb165e00640 1 -- 172.21.15.46:0/2382143999 learned_addr learned my addr 172.21.15.46:0/2382143999 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb166800640 1 --2- 172.21.15.46:0/2382143999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb168141cb0 0x7fb168139ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb15fe00640 1 --2- 172.21.15.46:0/2382143999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb168140f40 0x7fb16815a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb165e00640 1 -- 172.21.15.46:0/2382143999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb168141cb0 msgr2=0x7fb168139ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb165e00640 1 --2- 172.21.15.46:0/2382143999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb168141cb0 0x7fb168139ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb165e00640 1 -- 172.21.15.46:0/2382143999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb168140f40 msgr2=0x7fb16815a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb165e00640 1 --2- 172.21.15.46:0/2382143999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb168140f40 0x7fb16815a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb165e00640 1 -- 172.21.15.46:0/2382143999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1500094d0 con 0x7fb1680cec80 2024-03-29T08:24:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb165e00640 1 --2- 172.21.15.46:0/2382143999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1680cec80 0x7fb16813b6b0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb15800d800 tx=0x7fb15800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb15e000640 1 -- 172.21.15.46:0/2382143999 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15800b6c0 con 0x7fb1680cec80 2024-03-29T08:24:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb15e000640 1 -- 172.21.15.46:0/2382143999 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb15800b820 con 0x7fb1680cec80 2024-03-29T08:24:33.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb15e000640 1 -- 172.21.15.46:0/2382143999 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15800bb90 con 0x7fb1680cec80 2024-03-29T08:24:33.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb16d200640 1 -- 172.21.15.46:0/2382143999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16813abd0 con 0x7fb1680cec80 2024-03-29T08:24:33.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.017+0000 7fb16d200640 1 -- 172.21.15.46:0/2382143999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16813b190 con 0x7fb1680cec80 2024-03-29T08:24:33.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.021+0000 7fb16d200640 1 -- 172.21.15.46:0/2382143999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb168066c90 con 0x7fb1680cec80 2024-03-29T08:24:33.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.021+0000 7fb15e000640 1 -- 172.21.15.46:0/2382143999 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb15800bcf0 con 0x7fb1680cec80 2024-03-29T08:24:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.021+0000 7fb15e000640 1 --2- 172.21.15.46:0/2382143999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb144066690 0x7fb144068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.021+0000 7fb15e000640 1 -- 172.21.15.46:0/2382143999 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fb1580848b0 con 0x7fb1680cec80 2024-03-29T08:24:33.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.021+0000 7fb15fe00640 1 --2- 172.21.15.46:0/2382143999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb144066690 0x7fb144068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:33.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.021+0000 7fb15fe00640 1 --2- 172.21.15.46:0/2382143999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb144066690 0x7fb144068b50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb1500af000 tx=0x7fb1500a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:33.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.025+0000 7fb15e000640 1 -- 172.21.15.46:0/2382143999 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb158051a90 con 0x7fb1680cec80 2024-03-29T08:24:33.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:33.185+0000 7fb16d200640 1 -- 172.21.15.46:0/2382143999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb1681424d0 con 0x7fb1680cec80 2024-03-29T08:24:34.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.045+0000 7fb15e000640 1 -- 172.21.15.46:0/2382143999 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v82) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb158011340 con 0x7fb1680cec80 2024-03-29T08:24:34.047 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-03-29T08:24:34.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.049+0000 7fb16d200640 1 -- 172.21.15.46:0/2382143999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb144066690 msgr2=0x7fb144068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:34.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.049+0000 7fb16d200640 1 --2- 172.21.15.46:0/2382143999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb144066690 0x7fb144068b50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb1500af000 tx=0x7fb1500a7000 comp rx=0 tx=0).stop 2024-03-29T08:24:34.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.049+0000 7fb16d200640 1 -- 172.21.15.46:0/2382143999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1680cec80 msgr2=0x7fb16813b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:34.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.049+0000 7fb16d200640 1 --2- 172.21.15.46:0/2382143999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1680cec80 0x7fb16813b6b0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb15800d800 tx=0x7fb15800dcc0 comp rx=0 tx=0).stop 2024-03-29T08:24:34.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.053+0000 7fb16d200640 1 -- 172.21.15.46:0/2382143999 shutdown_connections 2024-03-29T08:24:34.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.053+0000 7fb16d200640 1 --2- 172.21.15.46:0/2382143999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb144066690 0x7fb144068b50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:34.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.053+0000 7fb16d200640 1 --2- 172.21.15.46:0/2382143999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb168140f40 0x7fb16815a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:34.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.053+0000 7fb16d200640 1 --2- 172.21.15.46:0/2382143999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb168141cb0 0x7fb168139ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:34.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.053+0000 7fb16d200640 1 --2- 172.21.15.46:0/2382143999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1680cec80 0x7fb16813b6b0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:34.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.053+0000 7fb16d200640 1 -- 172.21.15.46:0/2382143999 >> 172.21.15.46:0/2382143999 conn(0x7fb1680b42d0 msgr2=0x7fb168070180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:34.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.053+0000 7fb16d200640 1 -- 172.21.15.46:0/2382143999 shutdown_connections 2024-03-29T08:24:34.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.053+0000 7fb16d200640 1 -- 172.21.15.46:0/2382143999 wait complete. 2024-03-29T08:24:34.079 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T08:24:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.233+0000 7f0f6ec00640 1 Processor -- start 2024-03-29T08:24:34.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.233+0000 7f0f6ec00640 1 -- start start 2024-03-29T08:24:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.233+0000 7f0f6ec00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f6815cc50 0x7f0f68161100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:34.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.233+0000 7f0f6ec00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f68071100 0x7f0f680ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.233+0000 7f0f6ec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f681606b0 con 0x7f0f6806b470 2024-03-29T08:24:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.233+0000 7f0f6ec00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f6815d030 con 0x7f0f6815cc50 2024-03-29T08:24:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.233+0000 7f0f6ec00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f68160c70 con 0x7f0f68071100 2024-03-29T08:24:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.233+0000 7f0f67e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0f6815cc50 0x7f0f68161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.233+0000 7f0f67400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0f6806b470 0x7f0f680709f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57822/0 (socket says 172.21.15.46:57822) 2024-03-29T08:24:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.233+0000 7f0f67400640 1 -- 172.21.15.46:0/1410176350 learned_addr learned my addr 172.21.15.46:0/1410176350 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.233+0000 7f0f6cc00640 1 --2- 172.21.15.46:0/1410176350 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f68071100 0x7f0f680ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.233+0000 7f0f67e00640 1 -- 172.21.15.46:0/1410176350 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f68071100 msgr2=0x7f0f680ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.233+0000 7f0f67e00640 1 --2- 172.21.15.46:0/1410176350 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f68071100 0x7f0f680ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.233+0000 7f0f67e00640 1 -- 172.21.15.46:0/1410176350 >> v1:172.21.15.46:6789/0 conn(0x7f0f6806b470 legacy=0x7f0f680709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.233+0000 7f0f67e00640 1 -- 172.21.15.46:0/1410176350 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f680cd190 con 0x7f0f6815cc50 2024-03-29T08:24:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.233+0000 7f0f66a00640 1 -- 172.21.15.46:0/1410176350 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123767904 0 0) 0x7f0f681606b0 con 0x7f0f6806b470 2024-03-29T08:24:34.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.233+0000 7f0f6cc00640 1 --2- 172.21.15.46:0/1410176350 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f68071100 0x7f0f680ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:34.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.237+0000 7f0f67e00640 1 --2- 172.21.15.46:0/1410176350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f6815cc50 0x7f0f68161100 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f0f580097c0 tx=0x7f0f5801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6817236c6dd275e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:34.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.237+0000 7f0f66a00640 1 -- 172.21.15.46:0/1410176350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f580ad640 con 0x7f0f6815cc50 2024-03-29T08:24:34.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.237+0000 7f0f66a00640 1 -- 172.21.15.46:0/1410176350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f580ad7a0 con 0x7f0f6815cc50 2024-03-29T08:24:34.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.237+0000 7f0f66a00640 1 -- 172.21.15.46:0/1410176350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f580adb30 con 0x7f0f6815cc50 2024-03-29T08:24:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.237+0000 7f0f6ec00640 1 -- 172.21.15.46:0/1410176350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f6815cc50 msgr2=0x7f0f68161100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.237+0000 7f0f6ec00640 1 --2- 172.21.15.46:0/1410176350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f6815cc50 0x7f0f68161100 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f0f580097c0 tx=0x7f0f5801c4a0 comp rx=0 tx=0).stop 2024-03-29T08:24:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.237+0000 7f0f6ec00640 1 -- 172.21.15.46:0/1410176350 shutdown_connections 2024-03-29T08:24:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.237+0000 7f0f6ec00640 1 --2- 172.21.15.46:0/1410176350 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f68071100 0x7f0f680ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.237+0000 7f0f6ec00640 1 --2- 172.21.15.46:0/1410176350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f6815cc50 0x7f0f68161100 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.237+0000 7f0f6ec00640 1 -- 172.21.15.46:0/1410176350 >> 172.21.15.46:0/1410176350 conn(0x7f0f680ced50 msgr2=0x7f0f680cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.237+0000 7f0f6ec00640 1 -- 172.21.15.46:0/1410176350 shutdown_connections 2024-03-29T08:24:34.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.237+0000 7f0f6ec00640 1 -- 172.21.15.46:0/1410176350 wait complete. 2024-03-29T08:24:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.237+0000 7f0f6ec00640 1 Processor -- start 2024-03-29T08:24:34.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f6ec00640 1 -- start start 2024-03-29T08:24:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f6ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f6806b470 0x7f0f680ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f6ec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f68071100 0x7f0f6814c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f6ec00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f6815cc50 0x7f0f6814ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f6ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f680aed70 con 0x7f0f6806b470 2024-03-29T08:24:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f6ec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f680acf50 con 0x7f0f68071100 2024-03-29T08:24:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f6ec00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f680ad230 con 0x7f0f6815cc50 2024-03-29T08:24:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f67e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f6806b470 0x7f0f680ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f67e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f6806b470 0x7f0f680ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49072/0 (socket says 172.21.15.46:49072) 2024-03-29T08:24:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f67e00640 1 -- 172.21.15.46:0/4207964391 learned_addr learned my addr 172.21.15.46:0/4207964391 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f6cc00640 1 --2- 172.21.15.46:0/4207964391 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f6815cc50 0x7f0f6814ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f67400640 1 --2- 172.21.15.46:0/4207964391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f68071100 0x7f0f6814c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:34.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f67e00640 1 -- 172.21.15.46:0/4207964391 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f6815cc50 msgr2=0x7f0f6814ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:34.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f67e00640 1 --2- 172.21.15.46:0/4207964391 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f6815cc50 0x7f0f6814ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:34.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f67e00640 1 -- 172.21.15.46:0/4207964391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f68071100 msgr2=0x7f0f6814c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:34.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f67e00640 1 --2- 172.21.15.46:0/4207964391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f68071100 0x7f0f6814c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:34.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f67e00640 1 -- 172.21.15.46:0/4207964391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f58009420 con 0x7f0f6806b470 2024-03-29T08:24:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f67e00640 1 --2- 172.21.15.46:0/4207964391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f6806b470 0x7f0f680ae830 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f0f58000c10 tx=0x7f0f58019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f5fe00640 1 -- 172.21.15.46:0/4207964391 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f58004230 con 0x7f0f6806b470 2024-03-29T08:24:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f5fe00640 1 -- 172.21.15.46:0/4207964391 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f580a6040 con 0x7f0f6806b470 2024-03-29T08:24:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f6ec00640 1 -- 172.21.15.46:0/4207964391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f6815e350 con 0x7f0f6806b470 2024-03-29T08:24:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f5fe00640 1 -- 172.21.15.46:0/4207964391 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f58003620 con 0x7f0f6806b470 2024-03-29T08:24:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f6ec00640 1 -- 172.21.15.46:0/4207964391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f6815e910 con 0x7f0f6806b470 2024-03-29T08:24:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.241+0000 7f0f6ec00640 1 -- 172.21.15.46:0/4207964391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f68066c90 con 0x7f0f6806b470 2024-03-29T08:24:34.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.245+0000 7f0f5fe00640 1 -- 172.21.15.46:0/4207964391 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0f580b4040 con 0x7f0f6806b470 2024-03-29T08:24:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.245+0000 7f0f5fe00640 1 --2- 172.21.15.46:0/4207964391 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f48066690 0x7f0f48068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.245+0000 7f0f5fe00640 1 -- 172.21.15.46:0/4207964391 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5472+0+0 (secure 0 0 0) 0x7f0f5811d7a0 con 0x7f0f6806b470 2024-03-29T08:24:34.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.245+0000 7f0f67400640 1 --2- 172.21.15.46:0/4207964391 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f48066690 0x7f0f48068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:34.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.245+0000 7f0f67400640 1 --2- 172.21.15.46:0/4207964391 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f48066690 0x7f0f48068b50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0f680b3840 tx=0x7f0f54007d60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:34.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.249+0000 7f0f5fe00640 1 -- 172.21.15.46:0/4207964391 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0f580ead50 con 0x7f0f6806b470 2024-03-29T08:24:34.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:34.409+0000 7f0f6ec00640 1 -- 172.21.15.46:0/4207964391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0f68065fd0 con 0x7f0f6806b470 2024-03-29T08:24:35.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.053+0000 7f0f5fe00640 1 -- 172.21.15.46:0/4207964391 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v83) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0f580ee370 con 0x7f0f6806b470 2024-03-29T08:24:35.055 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-03-29T08:24:35.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.057+0000 7f0f6ec00640 1 -- 172.21.15.46:0/4207964391 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f48066690 msgr2=0x7f0f48068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:35.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.057+0000 7f0f6ec00640 1 --2- 172.21.15.46:0/4207964391 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f48066690 0x7f0f48068b50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f0f680b3840 tx=0x7f0f54007d60 comp rx=0 tx=0).stop 2024-03-29T08:24:35.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.057+0000 7f0f6ec00640 1 -- 172.21.15.46:0/4207964391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f6806b470 msgr2=0x7f0f680ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:35.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.057+0000 7f0f6ec00640 1 --2- 172.21.15.46:0/4207964391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f6806b470 0x7f0f680ae830 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f0f58000c10 tx=0x7f0f58019240 comp rx=0 tx=0).stop 2024-03-29T08:24:35.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.057+0000 7f0f6ec00640 1 -- 172.21.15.46:0/4207964391 shutdown_connections 2024-03-29T08:24:35.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.057+0000 7f0f6ec00640 1 --2- 172.21.15.46:0/4207964391 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f48066690 0x7f0f48068b50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:35.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.057+0000 7f0f6ec00640 1 --2- 172.21.15.46:0/4207964391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0f68071100 0x7f0f6814c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:35.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.057+0000 7f0f6ec00640 1 --2- 172.21.15.46:0/4207964391 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0f6815cc50 0x7f0f6814ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:35.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.057+0000 7f0f6ec00640 1 --2- 172.21.15.46:0/4207964391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f6806b470 0x7f0f680ae830 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:35.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.057+0000 7f0f6ec00640 1 -- 172.21.15.46:0/4207964391 >> 172.21.15.46:0/4207964391 conn(0x7f0f680ced50 msgr2=0x7f0f680b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:35.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.061+0000 7f0f6ec00640 1 -- 172.21.15.46:0/4207964391 shutdown_connections 2024-03-29T08:24:35.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.061+0000 7f0f6ec00640 1 -- 172.21.15.46:0/4207964391 wait complete. 2024-03-29T08:24:35.082 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-29T08:24:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.237+0000 7f0bc0400640 1 Processor -- start 2024-03-29T08:24:35.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.237+0000 7f0bc0400640 1 -- start start 2024-03-29T08:24:35.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.237+0000 7f0bc0400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0bb80b1810 0x7f0bb80cad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:35.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.237+0000 7f0bc0400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb81605e0 con 0x7f0bb80cb2c0 2024-03-29T08:24:35.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.237+0000 7f0bc0400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb806b490 con 0x7f0bb8160200 2024-03-29T08:24:35.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.237+0000 7f0bc0400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb8098b70 con 0x7f0bb80b1810 2024-03-29T08:24:35.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.237+0000 7f0bbe400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0bb80cb2c0 0x7f0bb80ce8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57832/0 (socket says 172.21.15.46:57832) 2024-03-29T08:24:35.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.237+0000 7f0bbe400640 1 -- 172.21.15.46:0/2047330592 learned_addr learned my addr 172.21.15.46:0/2047330592 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:35.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.237+0000 7f0bbd000640 1 --2- 172.21.15.46:0/2047330592 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0bb80b1810 0x7f0bb80cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:35.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.237+0000 7f0bb7e00640 1 -- 172.21.15.46:0/2047330592 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360870345 0 0) 0x7f0bb81605e0 con 0x7f0bb80cb2c0 2024-03-29T08:24:35.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.241+0000 7f0bb7e00640 1 -- 172.21.15.46:0/2047330592 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ba0003530 con 0x7f0bb80cb2c0 2024-03-29T08:24:35.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.241+0000 7f0bb7e00640 1 -- 172.21.15.46:0/2047330592 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263064469 0 0) 0x7f0bb806b490 con 0x7f0bb8160200 2024-03-29T08:24:35.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.241+0000 7f0bb7e00640 1 -- 172.21.15.46:0/2047330592 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bb81605e0 con 0x7f0bb8160200 2024-03-29T08:24:35.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.241+0000 7f0bbd000640 1 -- 172.21.15.46:0/2047330592 >> v1:172.21.15.73:6789/0 conn(0x7f0bb8160200 legacy=0x7f0bb8099930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:35.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.241+0000 7f0bbd000640 1 -- 172.21.15.46:0/2047330592 >> v1:172.21.15.46:6789/0 conn(0x7f0bb80cb2c0 legacy=0x7f0bb80ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:35.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.241+0000 7f0bbd000640 1 -- 172.21.15.46:0/2047330592 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb8098e50 con 0x7f0bb80b1810 2024-03-29T08:24:35.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.241+0000 7f0bbd000640 1 --2- 172.21.15.46:0/2047330592 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0bb80b1810 0x7f0bb80cad80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40097f0 tx=0x7f0ba401c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7981c98689a5d942 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:35.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.241+0000 7f0bb7e00640 1 -- 172.21.15.46:0/2047330592 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40ab620 con 0x7f0bb80b1810 2024-03-29T08:24:35.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.241+0000 7f0bb7e00640 1 -- 172.21.15.46:0/2047330592 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba40ab780 con 0x7f0bb80b1810 2024-03-29T08:24:35.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.241+0000 7f0bb7e00640 1 -- 172.21.15.46:0/2047330592 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba40abaf0 con 0x7f0bb80b1810 2024-03-29T08:24:35.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.241+0000 7f0bc0400640 1 -- 172.21.15.46:0/2047330592 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0bb80b1810 msgr2=0x7f0bb80cad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:35.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.241+0000 7f0bc0400640 1 --2- 172.21.15.46:0/2047330592 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0bb80b1810 0x7f0bb80cad80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40097f0 tx=0x7f0ba401c420 comp rx=0 tx=0).stop 2024-03-29T08:24:35.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.241+0000 7f0bc0400640 1 -- 172.21.15.46:0/2047330592 shutdown_connections 2024-03-29T08:24:35.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.241+0000 7f0bc0400640 1 --2- 172.21.15.46:0/2047330592 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0bb80b1810 0x7f0bb80cad80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:35.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.241+0000 7f0bc0400640 1 -- 172.21.15.46:0/2047330592 >> 172.21.15.46:0/2047330592 conn(0x7f0bb80b2220 msgr2=0x7f0bb80b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:35.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bc0400640 1 -- 172.21.15.46:0/2047330592 shutdown_connections 2024-03-29T08:24:35.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bc0400640 1 -- 172.21.15.46:0/2047330592 wait complete. 2024-03-29T08:24:35.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bc0400640 1 Processor -- start 2024-03-29T08:24:35.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bc0400640 1 -- start start 2024-03-29T08:24:35.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bc0400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb80b1810 0x7f0bb814ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:35.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bc0400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0bb80cb2c0 0x7f0bb8157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:35.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bc0400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0bb8160200 0x7f0bb81560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:35.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bc0400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb8156620 con 0x7f0bb80b1810 2024-03-29T08:24:35.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bc0400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb81568c0 con 0x7f0bb80cb2c0 2024-03-29T08:24:35.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bc0400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb8156cf0 con 0x7f0bb8160200 2024-03-29T08:24:35.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bbda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb80b1810 0x7f0bb814ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bbda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb80b1810 0x7f0bb814ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49080/0 (socket says 172.21.15.46:49080) 2024-03-29T08:24:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bbda00640 1 -- 172.21.15.46:0/2306990550 learned_addr learned my addr 172.21.15.46:0/2306990550 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bbd000640 1 --2- 172.21.15.46:0/2306990550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0bb80cb2c0 0x7f0bb8157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bbe400640 1 --2- 172.21.15.46:0/2306990550 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0bb8160200 0x7f0bb81560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bbda00640 1 -- 172.21.15.46:0/2306990550 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0bb8160200 msgr2=0x7f0bb81560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bbda00640 1 --2- 172.21.15.46:0/2306990550 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0bb8160200 0x7f0bb81560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bbda00640 1 -- 172.21.15.46:0/2306990550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0bb80cb2c0 msgr2=0x7f0bb8157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bbda00640 1 --2- 172.21.15.46:0/2306990550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0bb80cb2c0 0x7f0bb8157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bbda00640 1 -- 172.21.15.46:0/2306990550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba4009420 con 0x7f0bb80b1810 2024-03-29T08:24:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bbd000640 1 --2- 172.21.15.46:0/2306990550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0bb80cb2c0 0x7f0bb8157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bbda00640 1 --2- 172.21.15.46:0/2306990550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb80b1810 0x7f0bb814ab10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00c490 tx=0x7f0bac00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bb5600640 1 -- 172.21.15.46:0/2306990550 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac0052d0 con 0x7f0bb80b1810 2024-03-29T08:24:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bb5600640 1 -- 172.21.15.46:0/2306990550 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bac005430 con 0x7f0bb80b1810 2024-03-29T08:24:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bb5600640 1 -- 172.21.15.46:0/2306990550 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bac012870 con 0x7f0bb80b1810 2024-03-29T08:24:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.245+0000 7f0bc0400640 1 -- 172.21.15.46:0/2306990550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb8156fd0 con 0x7f0bb80b1810 2024-03-29T08:24:35.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.249+0000 7f0bc0400640 1 -- 172.21.15.46:0/2306990550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb8157590 con 0x7f0bb80b1810 2024-03-29T08:24:35.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.249+0000 7f0bc0400640 1 -- 172.21.15.46:0/2306990550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb80671b0 con 0x7f0bb80b1810 2024-03-29T08:24:35.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.249+0000 7f0bb5600640 1 -- 172.21.15.46:0/2306990550 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0bac0129d0 con 0x7f0bb80b1810 2024-03-29T08:24:35.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.249+0000 7f0bb5600640 1 --2- 172.21.15.46:0/2306990550 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0b980666c0 0x7f0b98068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:35.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.249+0000 7f0bb5600640 1 -- 172.21.15.46:0/2306990550 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f0bac0839f0 con 0x7f0bb80b1810 2024-03-29T08:24:35.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.249+0000 7f0bbd000640 1 --2- 172.21.15.46:0/2306990550 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0b980666c0 0x7f0b98068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:35.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.249+0000 7f0bbd000640 1 --2- 172.21.15.46:0/2306990550 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0b980666c0 0x7f0b98068b80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40af000 tx=0x7f0ba40a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:35.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.253+0000 7f0bb5600640 1 -- 172.21.15.46:0/2306990550 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0bac051090 con 0x7f0bb80b1810 2024-03-29T08:24:35.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.417+0000 7f0bc0400640 1 -- 172.21.15.46:0/2306990550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0bb804e120 con 0x7f0bb80b1810 2024-03-29T08:24:35.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.421+0000 7f0bb5600640 1 -- 172.21.15.46:0/2306990550 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0bac0546b0 con 0x7f0bb80b1810 2024-03-29T08:24:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.425+0000 7f0bc0400640 1 -- 172.21.15.46:0/2306990550 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0b980666c0 msgr2=0x7f0b98068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.425+0000 7f0bc0400640 1 --2- 172.21.15.46:0/2306990550 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0b980666c0 0x7f0b98068b80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40af000 tx=0x7f0ba40a7000 comp rx=0 tx=0).stop 2024-03-29T08:24:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.425+0000 7f0bc0400640 1 -- 172.21.15.46:0/2306990550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb80b1810 msgr2=0x7f0bb814ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.425+0000 7f0bc0400640 1 --2- 172.21.15.46:0/2306990550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb80b1810 0x7f0bb814ab10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00c490 tx=0x7f0bac00c950 comp rx=0 tx=0).stop 2024-03-29T08:24:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.425+0000 7f0bc0400640 1 -- 172.21.15.46:0/2306990550 shutdown_connections 2024-03-29T08:24:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.425+0000 7f0bc0400640 1 --2- 172.21.15.46:0/2306990550 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0b980666c0 0x7f0b98068b80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.425+0000 7f0bc0400640 1 --2- 172.21.15.46:0/2306990550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0bb80cb2c0 0x7f0bb8157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.425+0000 7f0bc0400640 1 --2- 172.21.15.46:0/2306990550 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0bb8160200 0x7f0bb81560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.425+0000 7f0bc0400640 1 --2- 172.21.15.46:0/2306990550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0bb80b1810 0x7f0bb814ab10 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.425+0000 7f0bc0400640 1 -- 172.21.15.46:0/2306990550 >> 172.21.15.46:0/2306990550 conn(0x7f0bb80b2220 msgr2=0x7f0bb806e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:35.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.425+0000 7f0bc0400640 1 -- 172.21.15.46:0/2306990550 shutdown_connections 2024-03-29T08:24:35.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.425+0000 7f0bc0400640 1 -- 172.21.15.46:0/2306990550 wait complete. 2024-03-29T08:24:35.444 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-03-29T08:24:35.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeb7000640 1 Processor -- start 2024-03-29T08:24:35.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeb7000640 1 -- start start 2024-03-29T08:24:35.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeb7000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efeb0160090 0x7efeb006aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:35.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeb7000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efeb0099150 0x7efeb015c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:35.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeb7000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb00cec50 con 0x7efeb006b3e0 2024-03-29T08:24:35.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeb7000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb00d00f0 con 0x7efeb0160090 2024-03-29T08:24:35.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeb7000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb006b960 con 0x7efeb0099150 2024-03-29T08:24:35.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efea7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7efeb006b3e0 0x7efeb0098a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57836/0 (socket says 172.21.15.46:57836) 2024-03-29T08:24:35.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efea7e00640 1 -- 172.21.15.46:0/1565153139 learned_addr learned my addr 172.21.15.46:0/1565153139 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:35.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeb5000640 1 --2- 172.21.15.46:0/1565153139 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efeb0099150 0x7efeb015c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:35.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeafe00640 1 --2- 172.21.15.46:0/1565153139 >> v2:172.21.15.73:3300/0 conn(0x7efeb0160090 0x7efeb006aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeaf400640 1 -- 172.21.15.46:0/1565153139 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2802604126 0 0) 0x7efeb00cec50 con 0x7efeb006b3e0 2024-03-29T08:24:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeaf400640 1 -- 172.21.15.46:0/1565153139 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe94003530 con 0x7efeb006b3e0 2024-03-29T08:24:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeaf400640 1 -- 172.21.15.46:0/1565153139 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2464795071 0 0) 0x7efe94003530 con 0x7efeb006b3e0 2024-03-29T08:24:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeaf400640 1 -- 172.21.15.46:0/1565153139 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efeb0099150 msgr2=0x7efeb015c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeaf400640 1 --2- 172.21.15.46:0/1565153139 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efeb0099150 0x7efeb015c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeaf400640 1 -- 172.21.15.46:0/1565153139 >> v2:172.21.15.73:3300/0 conn(0x7efeb0160090 msgr2=0x7efeb006aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:35.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeaf400640 1 --2- 172.21.15.46:0/1565153139 >> v2:172.21.15.73:3300/0 conn(0x7efeb0160090 0x7efeb006aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeaf400640 1 -- 172.21.15.46:0/1565153139 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb006bc40 con 0x7efeb006b3e0 2024-03-29T08:24:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeaf400640 1 -- 172.21.15.46:0/1565153139 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7efe98001680 con 0x7efeb006b3e0 2024-03-29T08:24:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeaf400640 1 -- 172.21.15.46:0/1565153139 >> v1:172.21.15.46:6789/0 conn(0x7efeb006b3e0 legacy=0x7efeb0098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeaf400640 1 --2- 172.21.15.46:0/1565153139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe94003980 0x7efe94003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeaf400640 1 --2- 172.21.15.46:0/1565153139 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efeb0099150 0x7efe940047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeaf400640 1 --2- 172.21.15.46:0/1565153139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe94004d10 0x7efe94005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeaf400640 1 -- 172.21.15.46:0/1565153139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00d00f0 con 0x7efe94004d10 2024-03-29T08:24:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeaf400640 1 -- 172.21.15.46:0/1565153139 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb006b960 con 0x7efe94003980 2024-03-29T08:24:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeaf400640 1 -- 172.21.15.46:0/1565153139 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9400ba90 con 0x7efeb0099150 2024-03-29T08:24:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeb5000640 1 --2- 172.21.15.46:0/1565153139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe94004d10 0x7efe94005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeafe00640 1 --2- 172.21.15.46:0/1565153139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe94003980 0x7efe94003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efea7e00640 1 --2- 172.21.15.46:0/1565153139 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efeb0099150 0x7efe940047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeb5000640 1 -- 172.21.15.46:0/1565153139 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efeb0099150 msgr2=0x7efe940047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:35.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeb5000640 1 --2- 172.21.15.46:0/1565153139 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efeb0099150 0x7efe940047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeb5000640 1 -- 172.21.15.46:0/1565153139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe94003980 msgr2=0x7efe94003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeb5000640 1 --2- 172.21.15.46:0/1565153139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe94003980 0x7efe94003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efeb5000640 1 -- 172.21.15.46:0/1565153139 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe94003530 con 0x7efe94004d10 2024-03-29T08:24:35.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.585+0000 7efea7e00640 1 --2- 172.21.15.46:0/1565153139 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efeb0099150 0x7efe940047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.589+0000 7efeb5000640 1 --2- 172.21.15.46:0/1565153139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe94004d10 0x7efe94005190 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7efea8009f90 tx=0x7efea801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367b9cd300fc092d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.589+0000 7efeaf400640 1 -- 172.21.15.46:0/1565153139 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea80a5c20 con 0x7efe94004d10 2024-03-29T08:24:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.589+0000 7efeaf400640 1 -- 172.21.15.46:0/1565153139 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea800f040 con 0x7efe94004d10 2024-03-29T08:24:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.589+0000 7efeaf400640 1 -- 172.21.15.46:0/1565153139 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea80ad440 con 0x7efe94004d10 2024-03-29T08:24:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.589+0000 7efeb7000640 1 -- 172.21.15.46:0/1565153139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe94004d10 msgr2=0x7efe94005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.589+0000 7efeb7000640 1 --2- 172.21.15.46:0/1565153139 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe94004d10 0x7efe94005190 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7efea8009f90 tx=0x7efea801cc10 comp rx=0 tx=0).stop 2024-03-29T08:24:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.589+0000 7efeb5000640 1 -- 172.21.15.46:0/1565153139 reap_dead start 2024-03-29T08:24:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.589+0000 7efeb7000640 1 -- 172.21.15.46:0/1565153139 shutdown_connections 2024-03-29T08:24:35.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.589+0000 7efeb7000640 1 -- 172.21.15.46:0/1565153139 >> 172.21.15.46:0/1565153139 conn(0x7efeb00b21a0 msgr2=0x7efeb00b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.589+0000 7efeb7000640 1 -- 172.21.15.46:0/1565153139 shutdown_connections 2024-03-29T08:24:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.589+0000 7efeb7000640 1 -- 172.21.15.46:0/1565153139 wait complete. 2024-03-29T08:24:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efeb7000640 1 Processor -- start 2024-03-29T08:24:35.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efeb7000640 1 -- start start 2024-03-29T08:24:35.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efeb7000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efeb014bfb0 0x7efeb00b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efeb7000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efeb014c640 0x7efeb00b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efeb7000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efeb0155b40 0x7efeb015ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efeb7000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb014c390 con 0x7efeb014c640 2024-03-29T08:24:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efeb7000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb0156040 con 0x7efeb014bfb0 2024-03-29T08:24:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efeb7000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb014ee20 con 0x7efeb0155b40 2024-03-29T08:24:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efea7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efeb014c640 0x7efeb00b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efea7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efeb014c640 0x7efeb00b32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49096/0 (socket says 172.21.15.46:49096) 2024-03-29T08:24:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efea7e00640 1 -- 172.21.15.46:0/3461414358 learned_addr learned my addr 172.21.15.46:0/3461414358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efeafe00640 1 --2- 172.21.15.46:0/3461414358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efeb014bfb0 0x7efeb00b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efeb5000640 1 --2- 172.21.15.46:0/3461414358 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efeb0155b40 0x7efeb015ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efea7e00640 1 -- 172.21.15.46:0/3461414358 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efeb0155b40 msgr2=0x7efeb015ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efea7e00640 1 --2- 172.21.15.46:0/3461414358 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efeb0155b40 0x7efeb015ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:35.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efea7e00640 1 -- 172.21.15.46:0/3461414358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efeb014bfb0 msgr2=0x7efeb00b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:35.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efea7e00640 1 --2- 172.21.15.46:0/3461414358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efeb014bfb0 0x7efeb00b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:35.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efea7e00640 1 -- 172.21.15.46:0/3461414358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb014f100 con 0x7efeb014c640 2024-03-29T08:24:35.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efeafe00640 1 --2- 172.21.15.46:0/3461414358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efeb014bfb0 0x7efeb00b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:24:35.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efeb5000640 1 --2- 172.21.15.46:0/3461414358 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efeb0155b40 0x7efeb015ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:24:35.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efea7e00640 1 --2- 172.21.15.46:0/3461414358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efeb014c640 0x7efeb00b32d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7efe9800e870 tx=0x7efe9800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:35.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efeacc00640 1 -- 172.21.15.46:0/3461414358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9800c870 con 0x7efeb014c640 2024-03-29T08:24:35.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efeacc00640 1 -- 172.21.15.46:0/3461414358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9800c9d0 con 0x7efeb014c640 2024-03-29T08:24:35.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efeacc00640 1 -- 172.21.15.46:0/3461414358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9800cd60 con 0x7efeb014c640 2024-03-29T08:24:35.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efeb7000640 1 -- 172.21.15.46:0/3461414358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb014f3e0 con 0x7efeb014c640 2024-03-29T08:24:35.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.593+0000 7efeb7000640 1 -- 172.21.15.46:0/3461414358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb00b3810 con 0x7efeb014c640 2024-03-29T08:24:35.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.597+0000 7efeb7000640 1 -- 172.21.15.46:0/3461414358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb0066c90 con 0x7efeb014c640 2024-03-29T08:24:35.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.597+0000 7efeacc00640 1 -- 172.21.15.46:0/3461414358 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efe9801d460 con 0x7efeb014c640 2024-03-29T08:24:35.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.597+0000 7efeacc00640 1 --2- 172.21.15.46:0/3461414358 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe8c066690 0x7efe8c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:35.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.597+0000 7efeacc00640 1 -- 172.21.15.46:0/3461414358 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5029+0+0 (secure 0 0 0) 0x7efe98083fe0 con 0x7efeb014c640 2024-03-29T08:24:35.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.597+0000 7efeafe00640 1 --2- 172.21.15.46:0/3461414358 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe8c066690 0x7efe8c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:35.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.597+0000 7efeafe00640 1 --2- 172.21.15.46:0/3461414358 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe8c066690 0x7efe8c068b50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efea0009ad0 tx=0x7efea0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:35.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.601+0000 7efeacc00640 1 -- 172.21.15.46:0/3461414358 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efe98051590 con 0x7efeb014c640 2024-03-29T08:24:35.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:35.817+0000 7efeb7000640 1 -- 172.21.15.46:0/3461414358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7efeb0099570 con 0x7efeb014c640 2024-03-29T08:24:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.053+0000 7efeacc00640 1 -- 172.21.15.46:0/3461414358 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v310) v1 ==== 166+0+0 (secure 0 0 0) 0x7efe98054bb0 con 0x7efeb014c640 2024-03-29T08:24:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.057+0000 7efeb7000640 1 -- 172.21.15.46:0/3461414358 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe8c066690 msgr2=0x7efe8c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.057+0000 7efeb7000640 1 --2- 172.21.15.46:0/3461414358 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe8c066690 0x7efe8c068b50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efea0009ad0 tx=0x7efea0008040 comp rx=0 tx=0).stop 2024-03-29T08:24:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.057+0000 7efeb7000640 1 -- 172.21.15.46:0/3461414358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efeb014c640 msgr2=0x7efeb00b32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.057+0000 7efeb7000640 1 --2- 172.21.15.46:0/3461414358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efeb014c640 0x7efeb00b32d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7efe9800e870 tx=0x7efe9800ed30 comp rx=0 tx=0).stop 2024-03-29T08:24:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.057+0000 7efeb7000640 1 -- 172.21.15.46:0/3461414358 shutdown_connections 2024-03-29T08:24:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.057+0000 7efeb7000640 1 --2- 172.21.15.46:0/3461414358 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe8c066690 0x7efe8c068b50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.057+0000 7efeb7000640 1 --2- 172.21.15.46:0/3461414358 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efeb0155b40 0x7efeb015ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.057+0000 7efeb7000640 1 --2- 172.21.15.46:0/3461414358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efeb014c640 0x7efeb00b32d0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.057+0000 7efeb7000640 1 --2- 172.21.15.46:0/3461414358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efeb014bfb0 0x7efeb00b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.057+0000 7efeb7000640 1 -- 172.21.15.46:0/3461414358 >> 172.21.15.46:0/3461414358 conn(0x7efeb00b21a0 msgr2=0x7efeb006e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.057+0000 7efeb7000640 1 -- 172.21.15.46:0/3461414358 shutdown_connections 2024-03-29T08:24:36.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.057+0000 7efeb7000640 1 -- 172.21.15.46:0/3461414358 wait complete. 2024-03-29T08:24:36.077 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-03-29T08:24:36.077 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-03-29T08:24:36.077 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-29T08:24:36.077 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-03-29T08:24:36.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.217+0000 7f4b39600640 1 Processor -- start 2024-03-29T08:24:36.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.217+0000 7f4b39600640 1 -- start start 2024-03-29T08:24:36.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.221+0000 7f4b39600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4b341593d0 0x7f4b3406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:36.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.221+0000 7f4b39600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b3415cd90 0x7f4b341597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:36.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.221+0000 7f4b39600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b3406d130 con 0x7f4b3415d230 2024-03-29T08:24:36.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.221+0000 7f4b39600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b3415b8f0 con 0x7f4b341593d0 2024-03-29T08:24:36.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.221+0000 7f4b39600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3415ad50 con 0x7f4b3415cd90 2024-03-29T08:24:36.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.221+0000 7f4b32a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4b341593d0 0x7f4b3406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:36.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.221+0000 7f4b2be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4b3415d230 0x7f4b3415d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:45784/0 (socket says 172.21.15.46:45784) 2024-03-29T08:24:36.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.221+0000 7f4b2be00640 1 -- 172.21.15.46:0/87801934 learned_addr learned my addr 172.21.15.46:0/87801934 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:36.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.221+0000 7f4b33400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b3415cd90 0x7f4b341597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:36.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.221+0000 7f4b32a00640 1 -- 172.21.15.46:0/87801934 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b3415cd90 msgr2=0x7f4b341597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:36.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.221+0000 7f4b32a00640 1 --2- 172.21.15.46:0/87801934 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b3415cd90 0x7f4b341597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:36.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.221+0000 7f4b32a00640 1 -- 172.21.15.46:0/87801934 >> v1:172.21.15.73:6789/0 conn(0x7f4b3415d230 legacy=0x7f4b3415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:36.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.221+0000 7f4b32a00640 1 -- 172.21.15.46:0/87801934 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3415afc0 con 0x7f4b341593d0 2024-03-29T08:24:36.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.225+0000 7f4b32a00640 1 --2- 172.21.15.46:0/87801934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b341593d0 0x7f4b3406cbf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4b240097c0 tx=0x7f4b2401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=181403fe33d4bd6b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:36.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.225+0000 7f4b32000640 1 -- 172.21.15.46:0/87801934 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b240a6040 con 0x7f4b341593d0 2024-03-29T08:24:36.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.225+0000 7f4b32000640 1 -- 172.21.15.46:0/87801934 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b240aa600 con 0x7f4b341593d0 2024-03-29T08:24:36.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.225+0000 7f4b32000640 1 -- 172.21.15.46:0/87801934 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b240aa990 con 0x7f4b341593d0 2024-03-29T08:24:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.225+0000 7f4b39600640 1 -- 172.21.15.46:0/87801934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b341593d0 msgr2=0x7f4b3406cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.225+0000 7f4b39600640 1 --2- 172.21.15.46:0/87801934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b341593d0 0x7f4b3406cbf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4b240097c0 tx=0x7f4b2401c4a0 comp rx=0 tx=0).stop 2024-03-29T08:24:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.225+0000 7f4b39600640 1 -- 172.21.15.46:0/87801934 shutdown_connections 2024-03-29T08:24:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.225+0000 7f4b39600640 1 --2- 172.21.15.46:0/87801934 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b3415cd90 0x7f4b341597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.225+0000 7f4b39600640 1 --2- 172.21.15.46:0/87801934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b341593d0 0x7f4b3406cbf0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.225+0000 7f4b39600640 1 -- 172.21.15.46:0/87801934 >> 172.21.15.46:0/87801934 conn(0x7f4b34066110 msgr2=0x7f4b34069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.225+0000 7f4b39600640 1 -- 172.21.15.46:0/87801934 shutdown_connections 2024-03-29T08:24:36.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.225+0000 7f4b39600640 1 -- 172.21.15.46:0/87801934 wait complete. 2024-03-29T08:24:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.225+0000 7f4b39600640 1 Processor -- start 2024-03-29T08:24:36.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b39600640 1 -- start start 2024-03-29T08:24:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b39600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b341593d0 0x7f4b34150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b39600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b3415cd90 0x7f4b34153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:36.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b39600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3415d230 0x7f4b340aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b39600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b34156b00 con 0x7f4b3415d230 2024-03-29T08:24:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b39600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b34154590 con 0x7f4b3415cd90 2024-03-29T08:24:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b39600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b34154870 con 0x7f4b341593d0 2024-03-29T08:24:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b33400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3415d230 0x7f4b340aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b33400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3415d230 0x7f4b340aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49108/0 (socket says 172.21.15.46:49108) 2024-03-29T08:24:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b33400640 1 -- 172.21.15.46:0/4023980030 learned_addr learned my addr 172.21.15.46:0/4023980030 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b32a00640 1 --2- 172.21.15.46:0/4023980030 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b341593d0 0x7f4b34150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:36.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b2be00640 1 --2- 172.21.15.46:0/4023980030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b3415cd90 0x7f4b34153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:36.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b33400640 1 -- 172.21.15.46:0/4023980030 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b341593d0 msgr2=0x7f4b34150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:36.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b33400640 1 --2- 172.21.15.46:0/4023980030 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b341593d0 0x7f4b34150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:36.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b33400640 1 -- 172.21.15.46:0/4023980030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b3415cd90 msgr2=0x7f4b34153d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:36.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b33400640 1 --2- 172.21.15.46:0/4023980030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b3415cd90 0x7f4b34153d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:36.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b33400640 1 -- 172.21.15.46:0/4023980030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b24009420 con 0x7f4b3415d230 2024-03-29T08:24:36.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b33400640 1 --2- 172.21.15.46:0/4023980030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3415d230 0x7f4b340aea60 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00efc0 tx=0x7f4b2c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:36.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b2aa00640 1 -- 172.21.15.46:0/4023980030 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c009b20 con 0x7f4b3415d230 2024-03-29T08:24:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b2aa00640 1 -- 172.21.15.46:0/4023980030 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2c00f040 con 0x7f4b3415d230 2024-03-29T08:24:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b2aa00640 1 -- 172.21.15.46:0/4023980030 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c0146c0 con 0x7f4b3415d230 2024-03-29T08:24:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b39600640 1 -- 172.21.15.46:0/4023980030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b34154b50 con 0x7f4b3415d230 2024-03-29T08:24:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b39600640 1 -- 172.21.15.46:0/4023980030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b34154e30 con 0x7f4b3415d230 2024-03-29T08:24:36.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.229+0000 7f4b39600640 1 -- 172.21.15.46:0/4023980030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b34156960 con 0x7f4b3415d230 2024-03-29T08:24:36.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.233+0000 7f4b2aa00640 1 -- 172.21.15.46:0/4023980030 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4b2c014820 con 0x7f4b3415d230 2024-03-29T08:24:36.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.233+0000 7f4b2aa00640 1 --2- 172.21.15.46:0/4023980030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4b10066690 0x7f4b10068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:36.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.233+0000 7f4b2aa00640 1 -- 172.21.15.46:0/4023980030 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f4b2c084c60 con 0x7f4b3415d230 2024-03-29T08:24:36.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.233+0000 7f4b32a00640 1 --2- 172.21.15.46:0/4023980030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4b10066690 0x7f4b10068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:36.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.233+0000 7f4b32a00640 1 --2- 172.21.15.46:0/4023980030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4b10066690 0x7f4b10068b50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4b240ae000 tx=0x7f4b240b5040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:36.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.237+0000 7f4b2aa00640 1 -- 172.21.15.46:0/4023980030 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4b34156960 con 0x7f4b3415d230 2024-03-29T08:24:36.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:36.453+0000 7f4b39600640 1 -- 172.21.15.46:0/4023980030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7f4b34156d00 con 0x7f4b3415d230 2024-03-29T08:24:37.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.057+0000 7f4b2aa00640 1 -- 172.21.15.46:0/4023980030 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v311) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4b34156d00 con 0x7f4b3415d230 2024-03-29T08:24:37.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.057+0000 7f4b39600640 1 -- 172.21.15.46:0/4023980030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4b10066690 msgr2=0x7f4b10068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:37.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.057+0000 7f4b39600640 1 --2- 172.21.15.46:0/4023980030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4b10066690 0x7f4b10068b50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4b240ae000 tx=0x7f4b240b5040 comp rx=0 tx=0).stop 2024-03-29T08:24:37.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.057+0000 7f4b39600640 1 -- 172.21.15.46:0/4023980030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3415d230 msgr2=0x7f4b340aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:37.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.057+0000 7f4b39600640 1 --2- 172.21.15.46:0/4023980030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3415d230 0x7f4b340aea60 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00efc0 tx=0x7f4b2c00c490 comp rx=0 tx=0).stop 2024-03-29T08:24:37.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.061+0000 7f4b39600640 1 -- 172.21.15.46:0/4023980030 shutdown_connections 2024-03-29T08:24:37.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.061+0000 7f4b39600640 1 --2- 172.21.15.46:0/4023980030 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4b10066690 0x7f4b10068b50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:37.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.061+0000 7f4b39600640 1 --2- 172.21.15.46:0/4023980030 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b3415cd90 0x7f4b34153d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:37.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.061+0000 7f4b39600640 1 --2- 172.21.15.46:0/4023980030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b3415d230 0x7f4b340aea60 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:37.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.061+0000 7f4b39600640 1 --2- 172.21.15.46:0/4023980030 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b341593d0 0x7f4b34150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:37.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.061+0000 7f4b39600640 1 -- 172.21.15.46:0/4023980030 >> 172.21.15.46:0/4023980030 conn(0x7f4b34066110 msgr2=0x7f4b34070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:37.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.061+0000 7f4b39600640 1 -- 172.21.15.46:0/4023980030 shutdown_connections 2024-03-29T08:24:37.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.061+0000 7f4b39600640 1 -- 172.21.15.46:0/4023980030 wait complete. 2024-03-29T08:24:37.083 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-29T08:24:37.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.229+0000 7f7169800640 1 Processor -- start 2024-03-29T08:24:37.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.229+0000 7f7169800640 1 -- start start 2024-03-29T08:24:37.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.233+0000 7f7169800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f716415efe0 0x7f71640b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:37.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.233+0000 7f7169800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71640b2360 con 0x7f7164143eb0 2024-03-29T08:24:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.233+0000 7f7169800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f716415e500 con 0x7f71640cbea0 2024-03-29T08:24:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.233+0000 7f7169800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f716415e7e0 con 0x7f716415efe0 2024-03-29T08:24:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.233+0000 7f7162a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7164143eb0 0x7f71640cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36950/0 (socket says 172.21.15.46:36950) 2024-03-29T08:24:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.233+0000 7f7162a00640 1 -- 172.21.15.46:0/2698490525 learned_addr learned my addr 172.21.15.46:0/2698490525 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.233+0000 7f7163400640 1 --2- 172.21.15.46:0/2698490525 >> v2:172.21.15.73:3300/0 conn(0x7f716415efe0 0x7f71640b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.233+0000 7f7162000640 1 -- 172.21.15.46:0/2698490525 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2254744480 0 0) 0x7f71640b2360 con 0x7f7164143eb0 2024-03-29T08:24:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.233+0000 7f7162000640 1 -- 172.21.15.46:0/2698490525 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7148003530 con 0x7f7164143eb0 2024-03-29T08:24:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.233+0000 7f7162000640 1 -- 172.21.15.46:0/2698490525 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2669846508 0 0) 0x7f716415e500 con 0x7f71640cbea0 2024-03-29T08:24:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.233+0000 7f7162000640 1 -- 172.21.15.46:0/2698490525 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71640b2360 con 0x7f71640cbea0 2024-03-29T08:24:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.233+0000 7f7163400640 1 -- 172.21.15.46:0/2698490525 >> v1:172.21.15.73:6789/0 conn(0x7f71640cbea0 legacy=0x7f71640b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.233+0000 7f7163400640 1 -- 172.21.15.46:0/2698490525 >> v1:172.21.15.46:6789/0 conn(0x7f7164143eb0 legacy=0x7f71640cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.233+0000 7f7162000640 1 -- 172.21.15.46:0/2698490525 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3690157617 0 0) 0x7f7148003530 con 0x7f7164143eb0 2024-03-29T08:24:37.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.233+0000 7f7163400640 1 -- 172.21.15.46:0/2698490525 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716415eac0 con 0x7f716415efe0 2024-03-29T08:24:37.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.237+0000 7f7163400640 1 --2- 172.21.15.46:0/2698490525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f716415efe0 0x7f71640b1d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f715c009d90 tx=0x7f715c01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce39ca5a2f43f00a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.237+0000 7f7162000640 1 -- 172.21.15.46:0/2698490525 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154001680 con 0x7f716415efe0 2024-03-29T08:24:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.237+0000 7f7162000640 1 -- 172.21.15.46:0/2698490525 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c0a56f0 con 0x7f716415efe0 2024-03-29T08:24:37.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.237+0000 7f7162000640 1 -- 172.21.15.46:0/2698490525 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c0a5a80 con 0x7f716415efe0 2024-03-29T08:24:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.237+0000 7f7169800640 1 -- 172.21.15.46:0/2698490525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f716415efe0 msgr2=0x7f71640b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.237+0000 7f7169800640 1 --2- 172.21.15.46:0/2698490525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f716415efe0 0x7f71640b1d40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f715c009d90 tx=0x7f715c01c890 comp rx=0 tx=0).stop 2024-03-29T08:24:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.237+0000 7f7169800640 1 -- 172.21.15.46:0/2698490525 shutdown_connections 2024-03-29T08:24:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.237+0000 7f7169800640 1 --2- 172.21.15.46:0/2698490525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f716415efe0 0x7f71640b1d40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.237+0000 7f7169800640 1 -- 172.21.15.46:0/2698490525 >> 172.21.15.46:0/2698490525 conn(0x7f716406a390 msgr2=0x7f71640cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:37.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.237+0000 7f7169800640 1 -- 172.21.15.46:0/2698490525 shutdown_connections 2024-03-29T08:24:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.237+0000 7f7169800640 1 -- 172.21.15.46:0/2698490525 wait complete. 2024-03-29T08:24:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.237+0000 7f7169800640 1 Processor -- start 2024-03-29T08:24:37.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.237+0000 7f7169800640 1 -- start start 2024-03-29T08:24:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f7169800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f71640cbea0 0x7f7164156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f7169800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7164143eb0 0x7f716413f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f7169800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f716415efe0 0x7f716413f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:37.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f7169800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7164156e90 con 0x7f7164143eb0 2024-03-29T08:24:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f7169800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7164157170 con 0x7f716415efe0 2024-03-29T08:24:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f7169800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7164157450 con 0x7f71640cbea0 2024-03-29T08:24:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f715be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7164143eb0 0x7f716413f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f715be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7164143eb0 0x7f716413f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50640/0 (socket says 172.21.15.46:50640) 2024-03-29T08:24:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f715be00640 1 -- 172.21.15.46:0/57587232 learned_addr learned my addr 172.21.15.46:0/57587232 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f7162a00640 1 --2- 172.21.15.46:0/57587232 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f71640cbea0 0x7f7164156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f7163400640 1 --2- 172.21.15.46:0/57587232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f716415efe0 0x7f716413f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f715be00640 1 -- 172.21.15.46:0/57587232 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f71640cbea0 msgr2=0x7f7164156790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f715be00640 1 --2- 172.21.15.46:0/57587232 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f71640cbea0 0x7f7164156790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f715be00640 1 -- 172.21.15.46:0/57587232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f716415efe0 msgr2=0x7f716413f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f715be00640 1 --2- 172.21.15.46:0/57587232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f716415efe0 0x7f716413f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f715be00640 1 -- 172.21.15.46:0/57587232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c009a70 con 0x7f7164143eb0 2024-03-29T08:24:37.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f715be00640 1 --2- 172.21.15.46:0/57587232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7164143eb0 0x7f716413f020 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f714c00c490 tx=0x7f714c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:37.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f715aa00640 1 -- 172.21.15.46:0/57587232 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c0052d0 con 0x7f7164143eb0 2024-03-29T08:24:37.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f715aa00640 1 -- 172.21.15.46:0/57587232 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714c005430 con 0x7f7164143eb0 2024-03-29T08:24:37.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f7169800640 1 -- 172.21.15.46:0/57587232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7164157730 con 0x7f7164143eb0 2024-03-29T08:24:37.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f715aa00640 1 -- 172.21.15.46:0/57587232 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c012870 con 0x7f7164143eb0 2024-03-29T08:24:37.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f7169800640 1 -- 172.21.15.46:0/57587232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716413a490 con 0x7f7164143eb0 2024-03-29T08:24:37.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f7169800640 1 -- 172.21.15.46:0/57587232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7164071a70 con 0x7f7164143eb0 2024-03-29T08:24:37.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.241+0000 7f715aa00640 1 -- 172.21.15.46:0/57587232 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f714c0129d0 con 0x7f7164143eb0 2024-03-29T08:24:37.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.245+0000 7f715aa00640 1 --2- 172.21.15.46:0/57587232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7140066690 0x7f7140068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:37.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.245+0000 7f715aa00640 1 -- 172.21.15.46:0/57587232 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f714c083a80 con 0x7f7164143eb0 2024-03-29T08:24:37.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.245+0000 7f7162a00640 1 --2- 172.21.15.46:0/57587232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7140066690 0x7f7140068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:37.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.245+0000 7f7162a00640 1 --2- 172.21.15.46:0/57587232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7140066690 0x7f7140068b50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7154001ec0 tx=0x7f7154001a60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:37.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.249+0000 7f715aa00640 1 -- 172.21.15.46:0/57587232 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f714c051000 con 0x7f7164143eb0 2024-03-29T08:24:37.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.409+0000 7f7169800640 1 -- 172.21.15.46:0/57587232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7164157be0 con 0x7f7164143eb0 2024-03-29T08:24:37.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.413+0000 7f715aa00640 1 -- 172.21.15.46:0/57587232 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f714c054620 con 0x7f7164143eb0 2024-03-29T08:24:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.413+0000 7f7169800640 1 -- 172.21.15.46:0/57587232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7140066690 msgr2=0x7f7140068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.413+0000 7f7169800640 1 --2- 172.21.15.46:0/57587232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7140066690 0x7f7140068b50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7154001ec0 tx=0x7f7154001a60 comp rx=0 tx=0).stop 2024-03-29T08:24:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.413+0000 7f7169800640 1 -- 172.21.15.46:0/57587232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7164143eb0 msgr2=0x7f716413f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.413+0000 7f7169800640 1 --2- 172.21.15.46:0/57587232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7164143eb0 0x7f716413f020 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f714c00c490 tx=0x7f714c00c950 comp rx=0 tx=0).stop 2024-03-29T08:24:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.413+0000 7f7169800640 1 -- 172.21.15.46:0/57587232 shutdown_connections 2024-03-29T08:24:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.413+0000 7f7169800640 1 --2- 172.21.15.46:0/57587232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7140066690 0x7f7140068b50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.413+0000 7f7169800640 1 --2- 172.21.15.46:0/57587232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f716415efe0 0x7f716413f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:37.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.413+0000 7f7169800640 1 --2- 172.21.15.46:0/57587232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7164143eb0 0x7f716413f020 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.413+0000 7f7169800640 1 --2- 172.21.15.46:0/57587232 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f71640cbea0 0x7f7164156790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.413+0000 7f7169800640 1 -- 172.21.15.46:0/57587232 >> 172.21.15.46:0/57587232 conn(0x7f716406a390 msgr2=0x7f716406db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.413+0000 7f7169800640 1 -- 172.21.15.46:0/57587232 shutdown_connections 2024-03-29T08:24:37.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.413+0000 7f7169800640 1 -- 172.21.15.46:0/57587232 wait complete. 2024-03-29T08:24:37.434 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T08:24:37.435 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:24:37.546 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:24:37.616 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T08:24:37.616 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:24:37.616 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:24:37.625 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:24:37.625 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:24:37.625 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:24:37.634 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:24:37.635 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:24:37.635 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:24:37.675 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:24:37.675 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:24:37.675 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:24:37.685 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:24:37.685 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-29T08:24:37.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:24:37.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.857+0000 7fc6dee00640 1 Processor -- start 2024-03-29T08:24:37.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.857+0000 7fc6dee00640 1 -- start start 2024-03-29T08:24:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.861+0000 7fc6dee00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc6d815feb0 0x7fc6d8071370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.861+0000 7fc6dee00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc6d806a4a0 0x7fc6d806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.861+0000 7fc6dee00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6d8161020 con 0x7fc6d81605f0 2024-03-29T08:24:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.861+0000 7fc6dee00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6d8161300 con 0x7fc6d806a4a0 2024-03-29T08:24:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.861+0000 7fc6dee00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6d8070b70 con 0x7fc6d815feb0 2024-03-29T08:24:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.861+0000 7fc6cf400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc6d81605f0 0x7fc6d8069d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36960/0 (socket says 172.21.15.46:36960) 2024-03-29T08:24:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.861+0000 7fc6dce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc6d806a4a0 0x7fc6d806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:37.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.861+0000 7fc6cf400640 1 -- 172.21.15.46:0/2814589056 learned_addr learned my addr 172.21.15.46:0/2814589056 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:37.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.861+0000 7fc6d7e00640 1 --2- 172.21.15.46:0/2814589056 >> v2:172.21.15.73:3300/0 conn(0x7fc6d815feb0 0x7fc6d8071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:37.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.861+0000 7fc6dce00640 1 -- 172.21.15.46:0/2814589056 >> v2:172.21.15.73:3300/0 conn(0x7fc6d815feb0 msgr2=0x7fc6d8071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:37.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.861+0000 7fc6dce00640 1 --2- 172.21.15.46:0/2814589056 >> v2:172.21.15.73:3300/0 conn(0x7fc6d815feb0 0x7fc6d8071370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:37.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.861+0000 7fc6dce00640 1 -- 172.21.15.46:0/2814589056 >> v1:172.21.15.46:6789/0 conn(0x7fc6d81605f0 legacy=0x7fc6d8069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:37.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.861+0000 7fc6dce00640 1 -- 172.21.15.46:0/2814589056 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d8070e50 con 0x7fc6d806a4a0 2024-03-29T08:24:37.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.861+0000 7fc6d7400640 1 -- 172.21.15.46:0/2814589056 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 963112441 0 0) 0x7fc6d8161020 con 0x7fc6d81605f0 2024-03-29T08:24:37.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6dce00640 1 --2- 172.21.15.46:0/2814589056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6d806a4a0 0x7fc6d806d8a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0009d90 tx=0x7fc6d001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49dcaa5444f38aa4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:37.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6d7400640 1 -- 172.21.15.46:0/2814589056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d00a5840 con 0x7fc6d806a4a0 2024-03-29T08:24:37.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6d7400640 1 -- 172.21.15.46:0/2814589056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d00a59a0 con 0x7fc6d806a4a0 2024-03-29T08:24:37.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6d7400640 1 -- 172.21.15.46:0/2814589056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d00a5d30 con 0x7fc6d806a4a0 2024-03-29T08:24:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6dee00640 1 -- 172.21.15.46:0/2814589056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6d806a4a0 msgr2=0x7fc6d806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6dee00640 1 --2- 172.21.15.46:0/2814589056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6d806a4a0 0x7fc6d806d8a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc6d0009d90 tx=0x7fc6d001c8a0 comp rx=0 tx=0).stop 2024-03-29T08:24:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6dee00640 1 -- 172.21.15.46:0/2814589056 shutdown_connections 2024-03-29T08:24:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6dee00640 1 --2- 172.21.15.46:0/2814589056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6d806a4a0 0x7fc6d806d8a0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6dee00640 1 --2- 172.21.15.46:0/2814589056 >> v2:172.21.15.73:3300/0 conn(0x7fc6d815feb0 0x7fc6d8071370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6dee00640 1 -- 172.21.15.46:0/2814589056 >> 172.21.15.46:0/2814589056 conn(0x7fc6d80ced50 msgr2=0x7fc6d80cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6dee00640 1 -- 172.21.15.46:0/2814589056 shutdown_connections 2024-03-29T08:24:37.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6dee00640 1 -- 172.21.15.46:0/2814589056 wait complete. 2024-03-29T08:24:37.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6dee00640 1 Processor -- start 2024-03-29T08:24:37.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6dee00640 1 -- start start 2024-03-29T08:24:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6dee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6d806a4a0 0x7fc6d8151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6dee00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc6d815feb0 0x7fc6d814f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6dee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6d81605f0 0x7fc6d815d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6dee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d814fe30 con 0x7fc6d81605f0 2024-03-29T08:24:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6dee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d80b23b0 con 0x7fc6d806a4a0 2024-03-29T08:24:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.865+0000 7fc6dee00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d80b2690 con 0x7fc6d815feb0 2024-03-29T08:24:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6dce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6d81605f0 0x7fc6d815d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6dce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6d81605f0 0x7fc6d815d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50652/0 (socket says 172.21.15.46:50652) 2024-03-29T08:24:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6dce00640 1 -- 172.21.15.46:0/892943595 learned_addr learned my addr 172.21.15.46:0/892943595 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6cf400640 1 --2- 172.21.15.46:0/892943595 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc6d815feb0 0x7fc6d814f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:37.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6d7e00640 1 --2- 172.21.15.46:0/892943595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6d806a4a0 0x7fc6d8151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6dce00640 1 -- 172.21.15.46:0/892943595 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc6d815feb0 msgr2=0x7fc6d814f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6dce00640 1 --2- 172.21.15.46:0/892943595 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc6d815feb0 0x7fc6d814f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6dce00640 1 -- 172.21.15.46:0/892943595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6d806a4a0 msgr2=0x7fc6d8151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6dce00640 1 --2- 172.21.15.46:0/892943595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6d806a4a0 0x7fc6d8151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6dce00640 1 -- 172.21.15.46:0/892943595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d0009a70 con 0x7fc6d81605f0 2024-03-29T08:24:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6dce00640 1 --2- 172.21.15.46:0/892943595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6d81605f0 0x7fc6d815d4b0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc6d00b0040 tx=0x7fc6d0019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6d4c00640 1 -- 172.21.15.46:0/892943595 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d00b9860 con 0x7fc6d81605f0 2024-03-29T08:24:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6d4c00640 1 -- 172.21.15.46:0/892943595 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d00b99c0 con 0x7fc6d81605f0 2024-03-29T08:24:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6d4c00640 1 -- 172.21.15.46:0/892943595 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d00b9d70 con 0x7fc6d81605f0 2024-03-29T08:24:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6dee00640 1 -- 172.21.15.46:0/892943595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d80b2970 con 0x7fc6d81605f0 2024-03-29T08:24:37.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6dee00640 1 -- 172.21.15.46:0/892943595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d80b2f30 con 0x7fc6d81605f0 2024-03-29T08:24:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6dee00640 1 -- 172.21.15.46:0/892943595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d8066c90 con 0x7fc6d81605f0 2024-03-29T08:24:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6d4c00640 1 -- 172.21.15.46:0/892943595 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc6d00c4460 con 0x7fc6d81605f0 2024-03-29T08:24:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6d4c00640 1 --2- 172.21.15.46:0/892943595 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc6b4066690 0x7fc6b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.869+0000 7fc6d4c00640 1 -- 172.21.15.46:0/892943595 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fc6d011f180 con 0x7fc6d81605f0 2024-03-29T08:24:37.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.873+0000 7fc6d7e00640 1 --2- 172.21.15.46:0/892943595 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc6b4066690 0x7fc6b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:37.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.873+0000 7fc6d7e00640 1 --2- 172.21.15.46:0/892943595 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc6b4066690 0x7fc6b4068b50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8007a60 tx=0x7fc6c8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:37.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:37.873+0000 7fc6d4c00640 1 -- 172.21.15.46:0/892943595 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc6d00ec820 con 0x7fc6d81605f0 2024-03-29T08:24:38.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.069+0000 7fc6dee00640 1 -- 172.21.15.46:0/892943595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6d80cb540 con 0x7fc6d81605f0 2024-03-29T08:24:38.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.069+0000 7fc6d4c00640 1 -- 172.21.15.46:0/892943595 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fc6d00b2de0 con 0x7fc6d81605f0 2024-03-29T08:24:38.071 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:38.071 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":51,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5987,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3456771376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3456771376},{"type":"v1","addr":"172.21.15.73:6836","nonce":3456771376}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6508,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2671538025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2671538025},{"type":"v1","addr":"172.21.15.73:6837","nonce":2671538025}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":6633,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3196015693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3196015693},{"type":"v1","addr":"172.21.15.46:6833","nonce":3196015693}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":7326,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4244130035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4244130035},{"type":"v1","addr":"172.21.15.46:6835","nonce":4244130035}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[]} 2024-03-29T08:24:38.071 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 51 2024-03-29T08:24:38.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.073+0000 7fc6dee00640 1 -- 172.21.15.46:0/892943595 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc6b4066690 msgr2=0x7fc6b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:38.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.073+0000 7fc6dee00640 1 --2- 172.21.15.46:0/892943595 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc6b4066690 0x7fc6b4068b50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8007a60 tx=0x7fc6c8008040 comp rx=0 tx=0).stop 2024-03-29T08:24:38.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.073+0000 7fc6dee00640 1 -- 172.21.15.46:0/892943595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6d81605f0 msgr2=0x7fc6d815d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:38.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.073+0000 7fc6dee00640 1 --2- 172.21.15.46:0/892943595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6d81605f0 0x7fc6d815d4b0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc6d00b0040 tx=0x7fc6d0019640 comp rx=0 tx=0).stop 2024-03-29T08:24:38.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.073+0000 7fc6dee00640 1 -- 172.21.15.46:0/892943595 shutdown_connections 2024-03-29T08:24:38.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.073+0000 7fc6dee00640 1 --2- 172.21.15.46:0/892943595 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc6b4066690 0x7fc6b4068b50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:38.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.073+0000 7fc6dee00640 1 --2- 172.21.15.46:0/892943595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6d81605f0 0x7fc6d815d4b0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:38.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.073+0000 7fc6dee00640 1 --2- 172.21.15.46:0/892943595 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc6d815feb0 0x7fc6d814f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:38.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.073+0000 7fc6dee00640 1 --2- 172.21.15.46:0/892943595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6d806a4a0 0x7fc6d8151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:38.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.073+0000 7fc6dee00640 1 -- 172.21.15.46:0/892943595 >> 172.21.15.46:0/892943595 conn(0x7fc6d80ced50 msgr2=0x7fc6d80ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:38.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.073+0000 7fc6dee00640 1 -- 172.21.15.46:0/892943595 shutdown_connections 2024-03-29T08:24:38.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.073+0000 7fc6dee00640 1 -- 172.21.15.46:0/892943595 wait complete. 2024-03-29T08:24:38.092 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T08:24:38.092 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-29T08:24:38.092 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T08:24:38.092 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:24:38.092 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T08:24:38.092 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-29T08:24:38.092 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T08:24:38.093 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:24:38.093 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T08:24:38.093 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-29T08:24:38.093 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T08:24:38.093 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:24:38.093 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T08:24:38.093 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-29T08:24:38.094 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T08:24:38.094 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:24:38.106 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T08:24:38.106 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T08:24:38.107 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T08:24:38.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T08:24:38.108 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T08:24:38.109 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T08:24:38.109 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T08:24:38.109 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T08:24:38.150 INFO:tasks.ceph.mds.a:Started 2024-03-29T08:24:38.150 DEBUG:teuthology.parallel:result is None 2024-03-29T08:24:38.151 INFO:tasks.ceph.mds.c:Started 2024-03-29T08:24:38.151 DEBUG:teuthology.parallel:result is None 2024-03-29T08:24:38.156 INFO:tasks.ceph.mds.d:Started 2024-03-29T08:24:38.156 DEBUG:teuthology.parallel:result is None 2024-03-29T08:24:38.157 INFO:tasks.ceph.mds.b:Started 2024-03-29T08:24:38.157 DEBUG:teuthology.parallel:result is None 2024-03-29T08:24:38.158 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-29T08:24:38.236 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-03-29T08:24:38.237 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-03-29T08:24:38.252 INFO:tasks.ceph.mds.b.smithi073.stdout:starting mds.b at 2024-03-29T08:24:38.252 INFO:tasks.ceph.mds.d.smithi073.stdout:starting mds.d at 2024-03-29T08:24:38.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.317+0000 7f0aa5800640 1 Processor -- start 2024-03-29T08:24:38.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.317+0000 7f0aa5800640 1 -- start start 2024-03-29T08:24:38.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.317+0000 7f0aa5800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0aa006b4b0 0x7f0aa00715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:38.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.317+0000 7f0aa5800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa006da20 con 0x7f0aa015ca10 2024-03-29T08:24:38.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.317+0000 7f0aa5800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa0066060 con 0x7f0aa006c830 2024-03-29T08:24:38.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.317+0000 7f0aa5800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa00ac490 con 0x7f0aa006b4b0 2024-03-29T08:24:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.317+0000 7f0a9ea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0aa015ca10 0x7f0aa00abd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36982/0 (socket says 172.21.15.46:36982) 2024-03-29T08:24:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.317+0000 7f0a9ea00640 1 -- 172.21.15.46:0/1240870775 learned_addr learned my addr 172.21.15.46:0/1240870775 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.317+0000 7f0a9f400640 1 --2- 172.21.15.46:0/1240870775 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0aa006b4b0 0x7f0aa00715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.317+0000 7f0a9e000640 1 -- 172.21.15.46:0/1240870775 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431005097 0 0) 0x7f0aa006da20 con 0x7f0aa015ca10 2024-03-29T08:24:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.317+0000 7f0a9e000640 1 -- 172.21.15.46:0/1240870775 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a84003530 con 0x7f0aa015ca10 2024-03-29T08:24:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.317+0000 7f0a9f400640 1 -- 172.21.15.46:0/1240870775 >> v1:172.21.15.73:6789/0 conn(0x7f0aa006c830 legacy=0x7f0aa006ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.317+0000 7f0a9e000640 1 -- 172.21.15.46:0/1240870775 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38991495 0 0) 0x7f0aa0066060 con 0x7f0aa006c830 2024-03-29T08:24:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.317+0000 7f0a9f400640 1 -- 172.21.15.46:0/1240870775 >> v1:172.21.15.46:6789/0 conn(0x7f0aa015ca10 legacy=0x7f0aa00abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:38.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.317+0000 7f0a9f400640 1 -- 172.21.15.46:0/1240870775 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa006ce10 con 0x7f0aa006b4b0 2024-03-29T08:24:38.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.321+0000 7f0a9f400640 1 --2- 172.21.15.46:0/1240870775 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0aa006b4b0 0x7f0aa00715d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009dc0 tx=0x7f0a9801c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c69043c8b8668d9f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:38.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.321+0000 7f0a9e000640 1 -- 172.21.15.46:0/1240870775 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a980a5a40 con 0x7f0aa006b4b0 2024-03-29T08:24:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.321+0000 7f0a9e000640 1 -- 172.21.15.46:0/1240870775 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a980a5ba0 con 0x7f0aa006b4b0 2024-03-29T08:24:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.321+0000 7f0a9e000640 1 -- 172.21.15.46:0/1240870775 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a980ad440 con 0x7f0aa006b4b0 2024-03-29T08:24:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.321+0000 7f0aa5800640 1 -- 172.21.15.46:0/1240870775 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0aa006b4b0 msgr2=0x7f0aa00715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.321+0000 7f0aa5800640 1 --2- 172.21.15.46:0/1240870775 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0aa006b4b0 0x7f0aa00715d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009dc0 tx=0x7f0a9801c960 comp rx=0 tx=0).stop 2024-03-29T08:24:38.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.321+0000 7f0aa5800640 1 -- 172.21.15.46:0/1240870775 shutdown_connections 2024-03-29T08:24:38.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.321+0000 7f0aa5800640 1 --2- 172.21.15.46:0/1240870775 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0aa006b4b0 0x7f0aa00715d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:38.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.321+0000 7f0aa5800640 1 -- 172.21.15.46:0/1240870775 >> 172.21.15.46:0/1240870775 conn(0x7f0aa0070ab0 msgr2=0x7f0aa0070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:38.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.321+0000 7f0aa5800640 1 -- 172.21.15.46:0/1240870775 shutdown_connections 2024-03-29T08:24:38.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.321+0000 7f0aa5800640 1 -- 172.21.15.46:0/1240870775 wait complete. 2024-03-29T08:24:38.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.321+0000 7f0aa5800640 1 Processor -- start 2024-03-29T08:24:38.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.321+0000 7f0aa5800640 1 -- start start 2024-03-29T08:24:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0aa5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aa006b4b0 0x7f0aa0136da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0aa5800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aa006c830 0x7f0aa014b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0aa5800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0aa015ca10 0x7f0aa0147bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0aa5800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa014b810 con 0x7f0aa006b4b0 2024-03-29T08:24:38.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0aa5800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa01480f0 con 0x7f0aa006c830 2024-03-29T08:24:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0aa5800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa014ba90 con 0x7f0aa015ca10 2024-03-29T08:24:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0a9ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aa006b4b0 0x7f0aa0136da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0a9ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aa006b4b0 0x7f0aa0136da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50692/0 (socket says 172.21.15.46:50692) 2024-03-29T08:24:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0a9ea00640 1 -- 172.21.15.46:0/408499409 learned_addr learned my addr 172.21.15.46:0/408499409 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0a9f400640 1 --2- 172.21.15.46:0/408499409 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0aa015ca10 0x7f0aa0147bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0a96000640 1 --2- 172.21.15.46:0/408499409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aa006c830 0x7f0aa014b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0a9ea00640 1 -- 172.21.15.46:0/408499409 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0aa015ca10 msgr2=0x7f0aa0147bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0a9ea00640 1 --2- 172.21.15.46:0/408499409 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0aa015ca10 0x7f0aa0147bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0a9ea00640 1 -- 172.21.15.46:0/408499409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aa006c830 msgr2=0x7f0aa014b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0a9ea00640 1 --2- 172.21.15.46:0/408499409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aa006c830 0x7f0aa014b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0a9ea00640 1 -- 172.21.15.46:0/408499409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a98009a70 con 0x7f0aa006b4b0 2024-03-29T08:24:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0a9ea00640 1 --2- 172.21.15.46:0/408499409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aa006b4b0 0x7f0aa0136da0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000c3c0 tx=0x7f0a9000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0a97400640 1 -- 172.21.15.46:0/408499409 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90005cf0 con 0x7f0aa006b4b0 2024-03-29T08:24:38.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0a97400640 1 -- 172.21.15.46:0/408499409 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a90005e50 con 0x7f0aa006b4b0 2024-03-29T08:24:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0a97400640 1 -- 172.21.15.46:0/408499409 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a900072f0 con 0x7f0aa006b4b0 2024-03-29T08:24:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0aa5800640 1 -- 172.21.15.46:0/408499409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa014bd70 con 0x7f0aa006b4b0 2024-03-29T08:24:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0aa5800640 1 -- 172.21.15.46:0/408499409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa014c330 con 0x7f0aa006b4b0 2024-03-29T08:24:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0aa5800640 1 -- 172.21.15.46:0/408499409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa009d100 con 0x7f0aa006b4b0 2024-03-29T08:24:38.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.325+0000 7f0a97400640 1 -- 172.21.15.46:0/408499409 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0a90007450 con 0x7f0aa006b4b0 2024-03-29T08:24:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.329+0000 7f0a97400640 1 --2- 172.21.15.46:0/408499409 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0a7c066690 0x7f0a7c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:38.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.329+0000 7f0a97400640 1 -- 172.21.15.46:0/408499409 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f0a90083440 con 0x7f0aa006b4b0 2024-03-29T08:24:38.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.329+0000 7f0a96000640 1 --2- 172.21.15.46:0/408499409 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0a7c066690 0x7f0a7c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:38.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.329+0000 7f0a96000640 1 --2- 172.21.15.46:0/408499409 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0a7c066690 0x7f0a7c068b50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0aa013a8f0 tx=0x7f0a88009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:38.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.333+0000 7f0a97400640 1 -- 172.21.15.46:0/408499409 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0a90050af0 con 0x7f0aa006b4b0 2024-03-29T08:24:38.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:38.505+0000 7f0aa5800640 1 -- 172.21.15.46:0/408499409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0aa015bc20 con 0x7f0aa006b4b0 2024-03-29T08:24:39.009 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:24:39.012 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:24:39.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.293+0000 7f0a97400640 1 -- 172.21.15.46:0/408499409 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v88) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0a90017080 con 0x7f0aa006b4b0 2024-03-29T08:24:39.293 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-03-29T08:24:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.297+0000 7f0aa5800640 1 -- 172.21.15.46:0/408499409 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0a7c066690 msgr2=0x7f0a7c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.297+0000 7f0aa5800640 1 --2- 172.21.15.46:0/408499409 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0a7c066690 0x7f0a7c068b50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0aa013a8f0 tx=0x7f0a88009040 comp rx=0 tx=0).stop 2024-03-29T08:24:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.297+0000 7f0aa5800640 1 -- 172.21.15.46:0/408499409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aa006b4b0 msgr2=0x7f0aa0136da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.297+0000 7f0aa5800640 1 --2- 172.21.15.46:0/408499409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aa006b4b0 0x7f0aa0136da0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000c3c0 tx=0x7f0a9000c880 comp rx=0 tx=0).stop 2024-03-29T08:24:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.297+0000 7f0aa5800640 1 -- 172.21.15.46:0/408499409 shutdown_connections 2024-03-29T08:24:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.297+0000 7f0aa5800640 1 --2- 172.21.15.46:0/408499409 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0a7c066690 0x7f0a7c068b50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.297+0000 7f0aa5800640 1 --2- 172.21.15.46:0/408499409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0aa006c830 0x7f0aa014b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.297+0000 7f0aa5800640 1 --2- 172.21.15.46:0/408499409 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0aa015ca10 0x7f0aa0147bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:39.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.297+0000 7f0aa5800640 1 --2- 172.21.15.46:0/408499409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0aa006b4b0 0x7f0aa0136da0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:39.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.297+0000 7f0aa5800640 1 -- 172.21.15.46:0/408499409 >> 172.21.15.46:0/408499409 conn(0x7f0aa0070ab0 msgr2=0x7f0aa01587e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:39.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.297+0000 7f0aa5800640 1 -- 172.21.15.46:0/408499409 shutdown_connections 2024-03-29T08:24:39.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.297+0000 7f0aa5800640 1 -- 172.21.15.46:0/408499409 wait complete. 2024-03-29T08:24:39.319 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-29T08:24:39.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.465+0000 7fbd6d200640 1 Processor -- start 2024-03-29T08:24:39.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.465+0000 7fbd6d200640 1 -- start start 2024-03-29T08:24:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.465+0000 7fbd6d200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbd680d0660 0x7fbd680d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.465+0000 7fbd6d200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd6806b8b0 0x7fbd680cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.465+0000 7fbd6d200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd6815cfb0 con 0x7fbd6815dc00 2024-03-29T08:24:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.465+0000 7fbd6d200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6806c100 con 0x7fbd680d0660 2024-03-29T08:24:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.465+0000 7fbd6d200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd680d0f80 con 0x7fbd6806b8b0 2024-03-29T08:24:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.465+0000 7fbd65e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbd680d0660 0x7fbd680d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.465+0000 7fbd66800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbd6815dc00 0x7fbd680cf7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36988/0 (socket says 172.21.15.46:36988) 2024-03-29T08:24:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.465+0000 7fbd65e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbd680d0660 0x7fbd680d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50704/0 (socket says 172.21.15.46:50704) 2024-03-29T08:24:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.465+0000 7fbd66800640 1 -- 172.21.15.46:0/2666720087 learned_addr learned my addr 172.21.15.46:0/2666720087 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:39.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.465+0000 7fbd65400640 1 --2- 172.21.15.46:0/2666720087 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd6806b8b0 0x7fbd680cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:39.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.465+0000 7fbd65e00640 1 -- 172.21.15.46:0/2666720087 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd6806b8b0 msgr2=0x7fbd680cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:39.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.465+0000 7fbd65e00640 1 --2- 172.21.15.46:0/2666720087 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd6806b8b0 0x7fbd680cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:39.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.465+0000 7fbd65e00640 1 -- 172.21.15.46:0/2666720087 >> v1:172.21.15.46:6789/0 conn(0x7fbd6815dc00 legacy=0x7fbd680cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:39.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.465+0000 7fbd65e00640 1 -- 172.21.15.46:0/2666720087 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6806be20 con 0x7fbd680d0660 2024-03-29T08:24:39.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.465+0000 7fbd64a00640 1 -- 172.21.15.46:0/2666720087 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454984790 0 0) 0x7fbd6815cfb0 con 0x7fbd6815dc00 2024-03-29T08:24:39.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.469+0000 7fbd65e00640 1 --2- 172.21.15.46:0/2666720087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd680d0660 0x7fbd680d0a40 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fbd580097c0 tx=0x7fbd5801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947561f97b136272 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:39.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.469+0000 7fbd64a00640 1 -- 172.21.15.46:0/2666720087 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd580ab670 con 0x7fbd680d0660 2024-03-29T08:24:39.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.469+0000 7fbd64a00640 1 -- 172.21.15.46:0/2666720087 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd580ab7d0 con 0x7fbd680d0660 2024-03-29T08:24:39.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.469+0000 7fbd64a00640 1 -- 172.21.15.46:0/2666720087 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd580abb60 con 0x7fbd680d0660 2024-03-29T08:24:39.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.469+0000 7fbd6d200640 1 -- 172.21.15.46:0/2666720087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd680d0660 msgr2=0x7fbd680d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.469+0000 7fbd6d200640 1 --2- 172.21.15.46:0/2666720087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd680d0660 0x7fbd680d0a40 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fbd580097c0 tx=0x7fbd5801c470 comp rx=0 tx=0).stop 2024-03-29T08:24:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.469+0000 7fbd6d200640 1 -- 172.21.15.46:0/2666720087 shutdown_connections 2024-03-29T08:24:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.469+0000 7fbd6d200640 1 --2- 172.21.15.46:0/2666720087 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd6806b8b0 0x7fbd680cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.469+0000 7fbd6d200640 1 --2- 172.21.15.46:0/2666720087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd680d0660 0x7fbd680d0a40 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.469+0000 7fbd6d200640 1 -- 172.21.15.46:0/2666720087 >> 172.21.15.46:0/2666720087 conn(0x7fbd68089fd0 msgr2=0x7fbd6808a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.469+0000 7fbd6d200640 1 -- 172.21.15.46:0/2666720087 shutdown_connections 2024-03-29T08:24:39.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.469+0000 7fbd6d200640 1 -- 172.21.15.46:0/2666720087 wait complete. 2024-03-29T08:24:39.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.469+0000 7fbd6d200640 1 Processor -- start 2024-03-29T08:24:39.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.469+0000 7fbd6d200640 1 -- start start 2024-03-29T08:24:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd6d200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd6806b8b0 0x7fbd6809d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd6d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd680d0660 0x7fbd680b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:39.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd6d200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd6815dc00 0x7fbd6809a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd6d200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd680b5d30 con 0x7fbd680d0660 2024-03-29T08:24:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd6d200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6809e330 con 0x7fbd6806b8b0 2024-03-29T08:24:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd6d200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6809d7b0 con 0x7fbd6815dc00 2024-03-29T08:24:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd65400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd680d0660 0x7fbd680b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd65400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd680d0660 0x7fbd680b57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50712/0 (socket says 172.21.15.46:50712) 2024-03-29T08:24:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd65400640 1 -- 172.21.15.46:0/2689432899 learned_addr learned my addr 172.21.15.46:0/2689432899 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd66800640 1 --2- 172.21.15.46:0/2689432899 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd6815dc00 0x7fbd6809a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd65e00640 1 --2- 172.21.15.46:0/2689432899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd6806b8b0 0x7fbd6809d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd65400640 1 -- 172.21.15.46:0/2689432899 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd6815dc00 msgr2=0x7fbd6809a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd65400640 1 --2- 172.21.15.46:0/2689432899 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd6815dc00 0x7fbd6809a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd65400640 1 -- 172.21.15.46:0/2689432899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd6806b8b0 msgr2=0x7fbd6809d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd65400640 1 --2- 172.21.15.46:0/2689432899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd6806b8b0 0x7fbd6809d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:39.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd65400640 1 -- 172.21.15.46:0/2689432899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd58009420 con 0x7fbd680d0660 2024-03-29T08:24:39.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd65e00640 1 --2- 172.21.15.46:0/2689432899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd6806b8b0 0x7fbd6809d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:24:39.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd66800640 1 --2- 172.21.15.46:0/2689432899 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd6815dc00 0x7fbd6809a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:24:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd65400640 1 --2- 172.21.15.46:0/2689432899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd680d0660 0x7fbd680b57f0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400c7f0 tx=0x7fbd5400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd5e000640 1 -- 172.21.15.46:0/2689432899 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54004220 con 0x7fbd680d0660 2024-03-29T08:24:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd5e000640 1 -- 172.21.15.46:0/2689432899 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd540043a0 con 0x7fbd680d0660 2024-03-29T08:24:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd5e000640 1 -- 172.21.15.46:0/2689432899 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54004510 con 0x7fbd680d0660 2024-03-29T08:24:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd6d200640 1 -- 172.21.15.46:0/2689432899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6809da90 con 0x7fbd680d0660 2024-03-29T08:24:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd6d200640 1 -- 172.21.15.46:0/2689432899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6809e050 con 0x7fbd680d0660 2024-03-29T08:24:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.473+0000 7fbd6d200640 1 -- 172.21.15.46:0/2689432899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd680c5030 con 0x7fbd680d0660 2024-03-29T08:24:39.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.477+0000 7fbd5e000640 1 -- 172.21.15.46:0/2689432899 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbd54003a70 con 0x7fbd680d0660 2024-03-29T08:24:39.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.477+0000 7fbd5e000640 1 --2- 172.21.15.46:0/2689432899 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbd48066690 0x7fbd48068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:39.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.477+0000 7fbd5e000640 1 -- 172.21.15.46:0/2689432899 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbd54083ec0 con 0x7fbd680d0660 2024-03-29T08:24:39.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.477+0000 7fbd65e00640 1 --2- 172.21.15.46:0/2689432899 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbd48066690 0x7fbd48068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:39.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.477+0000 7fbd65e00640 1 --2- 172.21.15.46:0/2689432899 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbd48066690 0x7fbd48068b50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fbd580af000 tx=0x7fbd58008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:39.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.481+0000 7fbd5e000640 1 -- 172.21.15.46:0/2689432899 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbd680c5030 con 0x7fbd680d0660 2024-03-29T08:24:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.693+0000 7fbd6d200640 1 -- 172.21.15.46:0/2689432899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbd680c5030 con 0x7fbd680d0660 2024-03-29T08:24:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.693+0000 7fbd5e000640 1 -- 172.21.15.46:0/2689432899 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fbd680c5030 con 0x7fbd680d0660 2024-03-29T08:24:39.695 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:39.695 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:24:39.695 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-03-29T08:24:39.695 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.695 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-03-29T08:24:39.695 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmlnXNJBAA7lI2lCBd0PeZudPsQyMOUQ==", 2024-03-29T08:24:39.695 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.695 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:24:39.695 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:24:39.695 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:24:39.695 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:24:39.695 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.696 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-03-29T08:24:39.696 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmbPkcMRAADnwJE17uILjJo14jghJuzg==", 2024-03-29T08:24:39.696 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.696 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:24:39.696 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:24:39.696 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:24:39.696 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:24:39.696 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.696 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-03-29T08:24:39.696 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmkCUVLBAAxcAOWGBDUCpEHeg/1OeQzA==", 2024-03-29T08:24:39.697 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.697 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:24:39.697 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:24:39.697 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:24:39.697 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:24:39.697 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.697 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-03-29T08:24:39.697 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZm7L11OBAAZ3fpXtEIYG1tGm9ZFswZ5Q==", 2024-03-29T08:24:39.697 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.697 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:24:39.697 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:24:39.697 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:24:39.698 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:24:39.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.698 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-03-29T08:24:39.698 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAIeQZmTHV8HhAAODJ+RP1Bx6k0D1153XgI0Q==", 2024-03-29T08:24:39.698 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.698 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:24:39.698 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:24:39.698 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:24:39.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.698 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-03-29T08:24:39.699 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQALeQZmZRD0JBAAjU6W/21CeqOMdIP5969iNA==", 2024-03-29T08:24:39.699 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.699 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:24:39.699 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:24:39.699 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:24:39.699 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.699 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-03-29T08:24:39.699 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAOeQZmSSP+KhAA26V8CByUJNfkVcG37AdlZQ==", 2024-03-29T08:24:39.699 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.699 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:24:39.699 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:24:39.700 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:24:39.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.700 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-03-29T08:24:39.700 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAReQZmb/e8LxAAV4yC1xlLwh8OAqOSPmcyyA==", 2024-03-29T08:24:39.700 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.700 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:24:39.700 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:24:39.700 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:24:39.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.700 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-03-29T08:24:39.701 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAXeQZmrLnzNRAAFPYQRs9sBNNKeyvcoI9rSg==", 2024-03-29T08:24:39.701 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.701 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:24:39.701 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:24:39.701 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:24:39.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.701 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-03-29T08:24:39.701 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAbeQZmjTPbABAAi4qTU5CBy9LtSSRf7cmfwg==", 2024-03-29T08:24:39.701 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.701 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:24:39.701 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:24:39.701 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:24:39.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.702 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-03-29T08:24:39.702 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAeeQZmo13eBRAAWY2VVBakLze9oA4ynp1rNA==", 2024-03-29T08:24:39.702 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.702 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:24:39.702 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:24:39.702 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:24:39.702 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.702 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-03-29T08:24:39.702 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAheQZmghpZCxAAdy5xI7qPwZb8LN8Rn6Ndaw==", 2024-03-29T08:24:39.702 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.702 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:24:39.703 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:24:39.703 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:24:39.703 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.703 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-03-29T08:24:39.703 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmyk22ABAA8Pn3OaIvxMQSLrKIM8FP4w==", 2024-03-29T08:24:39.703 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.703 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:24:39.703 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-03-29T08:24:39.703 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-29T08:24:39.703 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.704 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-03-29T08:24:39.704 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmG/E8ChAAJTcJ2j5xn0oUsmlV2trWdw==", 2024-03-29T08:24:39.704 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.704 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:24:39.704 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:24:39.704 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:24:39.704 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:24:39.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.704 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-03-29T08:24:39.704 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmhsqgBhAApqnO5qMG8VP27j9TfbZ8uw==", 2024-03-29T08:24:39.704 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.704 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:24:39.704 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:24:39.704 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:24:39.705 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:24:39.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.705 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-03-29T08:24:39.705 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmatlQEBAAAGKuGnAh4YXSk91TAzomXA==", 2024-03-29T08:24:39.705 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:24:39.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:24:39.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:24:39.705 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:24:39.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.705 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-03-29T08:24:39.705 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmp2B3CBAA4cfe/hUJeOg0SIr1KbWyUg==", 2024-03-29T08:24:39.706 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:24:39.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:24:39.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:24:39.706 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:24:39.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.706 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-03-29T08:24:39.706 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmyrJXIBAAv535Rz99UI3V7JvvzQ8oSw==", 2024-03-29T08:24:39.706 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:24:39.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-03-29T08:24:39.706 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:24:39.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:39.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:39.707 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-03-29T08:24:39.707 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZm72tiHBAA2N7qQnR5Ehu/7nP2WpklGg==", 2024-03-29T08:24:39.707 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:24:39.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:24:39.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-03-29T08:24:39.707 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:24:39.707 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.707 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:39.707 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-29T08:24:39.707 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:24:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.697+0000 7fbd6d200640 1 -- 172.21.15.46:0/2689432899 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbd48066690 msgr2=0x7fbd48068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.697+0000 7fbd6d200640 1 --2- 172.21.15.46:0/2689432899 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbd48066690 0x7fbd48068b50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fbd580af000 tx=0x7fbd58008040 comp rx=0 tx=0).stop 2024-03-29T08:24:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.697+0000 7fbd6d200640 1 -- 172.21.15.46:0/2689432899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd680d0660 msgr2=0x7fbd680b57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:39.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.697+0000 7fbd6d200640 1 --2- 172.21.15.46:0/2689432899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd680d0660 0x7fbd680b57f0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400c7f0 tx=0x7fbd5400ccb0 comp rx=0 tx=0).stop 2024-03-29T08:24:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.697+0000 7fbd6d200640 1 -- 172.21.15.46:0/2689432899 shutdown_connections 2024-03-29T08:24:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.697+0000 7fbd6d200640 1 --2- 172.21.15.46:0/2689432899 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbd48066690 0x7fbd48068b50 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.697+0000 7fbd6d200640 1 --2- 172.21.15.46:0/2689432899 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbd6815dc00 0x7fbd6809a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.697+0000 7fbd6d200640 1 --2- 172.21.15.46:0/2689432899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd680d0660 0x7fbd680b57f0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.697+0000 7fbd6d200640 1 --2- 172.21.15.46:0/2689432899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd6806b8b0 0x7fbd6809d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.697+0000 7fbd6d200640 1 -- 172.21.15.46:0/2689432899 >> 172.21.15.46:0/2689432899 conn(0x7fbd68089fd0 msgr2=0x7fbd6806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.697+0000 7fbd6d200640 1 -- 172.21.15.46:0/2689432899 shutdown_connections 2024-03-29T08:24:39.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.697+0000 7fbd6d200640 1 -- 172.21.15.46:0/2689432899 wait complete. 2024-03-29T08:24:39.716 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:24:39.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.857+0000 7f3e5cc00640 1 Processor -- start 2024-03-29T08:24:39.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.857+0000 7f3e5cc00640 1 -- start start 2024-03-29T08:24:39.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.857+0000 7f3e5cc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e5815ada0 0x7f3e580ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:39.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.857+0000 7f3e5cc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e580cb510 con 0x7f3e58144210 2024-03-29T08:24:39.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.857+0000 7f3e5cc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e58070580 con 0x7f3e580ce950 2024-03-29T08:24:39.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.857+0000 7f3e5cc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e580cede0 con 0x7f3e5815ada0 2024-03-29T08:24:39.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.857+0000 7f3e4d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e5815ada0 0x7f3e580ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:39.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.857+0000 7f3e55e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3e58144210 0x7f3e580cb8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37000/0 (socket says 172.21.15.46:37000) 2024-03-29T08:24:39.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.857+0000 7f3e55e00640 1 -- 172.21.15.46:0/4260391659 learned_addr learned my addr 172.21.15.46:0/4260391659 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:39.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.857+0000 7f3e4d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3e5815ada0 0x7f3e580ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50728/0 (socket says 172.21.15.46:50728) 2024-03-29T08:24:39.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.857+0000 7f3e4d400640 1 -- 172.21.15.46:0/4260391659 >> v1:172.21.15.73:6789/0 conn(0x7f3e580ce950 legacy=0x7f3e5806c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:39.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.857+0000 7f3e4d400640 1 -- 172.21.15.46:0/4260391659 >> v1:172.21.15.46:6789/0 conn(0x7f3e58144210 legacy=0x7f3e580cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:39.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.857+0000 7f3e4d400640 1 -- 172.21.15.46:0/4260391659 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e58070860 con 0x7f3e5815ada0 2024-03-29T08:24:39.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.857+0000 7f3e55400640 1 -- 172.21.15.46:0/4260391659 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1770542154 0 0) 0x7f3e580cb510 con 0x7f3e58144210 2024-03-29T08:24:39.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.861+0000 7f3e4d400640 1 --2- 172.21.15.46:0/4260391659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e5815ada0 0x7f3e580ce410 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3e400097c0 tx=0x7f3e4001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e54fdffa296c34 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:39.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.861+0000 7f3e55400640 1 -- 172.21.15.46:0/4260391659 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40004230 con 0x7f3e5815ada0 2024-03-29T08:24:39.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.861+0000 7f3e55400640 1 -- 172.21.15.46:0/4260391659 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e400a7050 con 0x7f3e5815ada0 2024-03-29T08:24:39.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.861+0000 7f3e55400640 1 -- 172.21.15.46:0/4260391659 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40003620 con 0x7f3e5815ada0 2024-03-29T08:24:39.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.861+0000 7f3e5cc00640 1 -- 172.21.15.46:0/4260391659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e5815ada0 msgr2=0x7f3e580ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:39.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.861+0000 7f3e5cc00640 1 --2- 172.21.15.46:0/4260391659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e5815ada0 0x7f3e580ce410 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3e400097c0 tx=0x7f3e4001c460 comp rx=0 tx=0).stop 2024-03-29T08:24:39.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.861+0000 7f3e5cc00640 1 -- 172.21.15.46:0/4260391659 shutdown_connections 2024-03-29T08:24:39.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.861+0000 7f3e5cc00640 1 --2- 172.21.15.46:0/4260391659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e5815ada0 0x7f3e580ce410 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:39.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.861+0000 7f3e5cc00640 1 -- 172.21.15.46:0/4260391659 >> 172.21.15.46:0/4260391659 conn(0x7f3e58066110 msgr2=0x7f3e58069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:39.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.861+0000 7f3e5cc00640 1 -- 172.21.15.46:0/4260391659 shutdown_connections 2024-03-29T08:24:39.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.861+0000 7f3e5cc00640 1 -- 172.21.15.46:0/4260391659 wait complete. 2024-03-29T08:24:39.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e5cc00640 1 Processor -- start 2024-03-29T08:24:39.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e5cc00640 1 -- start start 2024-03-29T08:24:39.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e5cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e580ce950 0x7f3e58137bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:39.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e5cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e58144210 0x7f3e5814ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:39.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e5cc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3e5815ada0 0x7f3e581503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:39.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e5cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e58150180 con 0x7f3e580ce950 2024-03-29T08:24:39.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e5cc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5814f600 con 0x7f3e58144210 2024-03-29T08:24:39.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e5cc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5814f8e0 con 0x7f3e5815ada0 2024-03-29T08:24:39.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e55e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e580ce950 0x7f3e58137bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:39.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e55e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e580ce950 0x7f3e58137bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50736/0 (socket says 172.21.15.46:50736) 2024-03-29T08:24:39.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e55e00640 1 -- 172.21.15.46:0/3152011928 learned_addr learned my addr 172.21.15.46:0/3152011928 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:39.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e4d400640 1 --2- 172.21.15.46:0/3152011928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e58144210 0x7f3e5814ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e56800640 1 --2- 172.21.15.46:0/3152011928 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3e5815ada0 0x7f3e581503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e55e00640 1 -- 172.21.15.46:0/3152011928 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3e5815ada0 msgr2=0x7f3e581503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e55e00640 1 --2- 172.21.15.46:0/3152011928 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3e5815ada0 0x7f3e581503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e55e00640 1 -- 172.21.15.46:0/3152011928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e58144210 msgr2=0x7f3e5814ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e55e00640 1 --2- 172.21.15.46:0/3152011928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e58144210 0x7f3e5814ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e55e00640 1 -- 172.21.15.46:0/3152011928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e40009420 con 0x7f3e580ce950 2024-03-29T08:24:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e56800640 1 --2- 172.21.15.46:0/3152011928 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3e5815ada0 0x7f3e581503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e55e00640 1 --2- 172.21.15.46:0/3152011928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e580ce950 0x7f3e58137bf0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f3e480011f0 tx=0x7f3e4800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e4ea00640 1 -- 172.21.15.46:0/3152011928 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e480074e0 con 0x7f3e580ce950 2024-03-29T08:24:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e4ea00640 1 -- 172.21.15.46:0/3152011928 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e48007640 con 0x7f3e580ce950 2024-03-29T08:24:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e4ea00640 1 -- 172.21.15.46:0/3152011928 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48004a10 con 0x7f3e580ce950 2024-03-29T08:24:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e5cc00640 1 -- 172.21.15.46:0/3152011928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5814fbc0 con 0x7f3e580ce950 2024-03-29T08:24:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.865+0000 7f3e5cc00640 1 -- 172.21.15.46:0/3152011928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5814f250 con 0x7f3e580ce950 2024-03-29T08:24:39.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.869+0000 7f3e5cc00640 1 -- 172.21.15.46:0/3152011928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e58066c90 con 0x7f3e580ce950 2024-03-29T08:24:39.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.869+0000 7f3e4ea00640 1 -- 172.21.15.46:0/3152011928 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3e48004b70 con 0x7f3e580ce950 2024-03-29T08:24:39.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.869+0000 7f3e4ea00640 1 --2- 172.21.15.46:0/3152011928 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3e340666c0 0x7f3e34068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:39.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.869+0000 7f3e4ea00640 1 -- 172.21.15.46:0/3152011928 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3e48082f20 con 0x7f3e580ce950 2024-03-29T08:24:39.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.869+0000 7f3e4d400640 1 --2- 172.21.15.46:0/3152011928 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3e340666c0 0x7f3e34068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:39.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.869+0000 7f3e4d400640 1 --2- 172.21.15.46:0/3152011928 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3e340666c0 0x7f3e34068b80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3e400ad000 tx=0x7f3e400a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:39.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:39.873+0000 7f3e4ea00640 1 -- 172.21.15.46:0/3152011928 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3e48050680 con 0x7f3e580ce950 2024-03-29T08:24:40.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.033+0000 7f3e5cc00640 1 -- 172.21.15.46:0/3152011928 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e58158740 con 0x7f3e580ce950 2024-03-29T08:24:40.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.033+0000 7f3e4ea00640 1 -- 172.21.15.46:0/3152011928 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+12342 (secure 0 0 0) 0x7f3e48018090 con 0x7f3e580ce950 2024-03-29T08:24:40.035 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:40.035 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":88,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:24:39.290780+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:24:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.037+0000 7f3e5cc00640 1 -- 172.21.15.46:0/3152011928 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3e340666c0 msgr2=0x7f3e34068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.037+0000 7f3e5cc00640 1 --2- 172.21.15.46:0/3152011928 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3e340666c0 0x7f3e34068b80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3e400ad000 tx=0x7f3e400a6000 comp rx=0 tx=0).stop 2024-03-29T08:24:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.037+0000 7f3e5cc00640 1 -- 172.21.15.46:0/3152011928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e580ce950 msgr2=0x7f3e58137bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.037+0000 7f3e5cc00640 1 --2- 172.21.15.46:0/3152011928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e580ce950 0x7f3e58137bf0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f3e480011f0 tx=0x7f3e4800c9b0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.037+0000 7f3e5cc00640 1 -- 172.21.15.46:0/3152011928 shutdown_connections 2024-03-29T08:24:40.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.037+0000 7f3e5cc00640 1 --2- 172.21.15.46:0/3152011928 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3e340666c0 0x7f3e34068b80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.037+0000 7f3e5cc00640 1 --2- 172.21.15.46:0/3152011928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e58144210 0x7f3e5814ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.037+0000 7f3e5cc00640 1 --2- 172.21.15.46:0/3152011928 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3e5815ada0 0x7f3e581503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.037+0000 7f3e5cc00640 1 --2- 172.21.15.46:0/3152011928 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e580ce950 0x7f3e58137bf0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.037+0000 7f3e5cc00640 1 -- 172.21.15.46:0/3152011928 >> 172.21.15.46:0/3152011928 conn(0x7f3e58066110 msgr2=0x7f3e58141810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:40.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.037+0000 7f3e5cc00640 1 -- 172.21.15.46:0/3152011928 shutdown_connections 2024-03-29T08:24:40.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.037+0000 7f3e5cc00640 1 -- 172.21.15.46:0/3152011928 wait complete. 2024-03-29T08:24:40.057 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:24:40.057 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:24:40.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.201+0000 7f8311a00640 1 Processor -- start 2024-03-29T08:24:40.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.201+0000 7f8311a00640 1 -- start start 2024-03-29T08:24:40.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.201+0000 7f8311a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f830c160100 0x7f830c0cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.201+0000 7f8311a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f830c06bac0 0x7f830c0b1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.201+0000 7f8311a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f830c098a40 0x7f830c098eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.201+0000 7f8311a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f830c0993f0 con 0x7f830c06bac0 2024-03-29T08:24:40.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.201+0000 7f8311a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f830c0cb230 con 0x7f830c098a40 2024-03-29T08:24:40.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.201+0000 7f8311a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830c06c030 con 0x7f830c160100 2024-03-29T08:24:40.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.201+0000 7f8303e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f830c06bac0 0x7f830c0b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.205+0000 7f8303e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f830c06bac0 0x7f830c0b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50748/0 (socket says 172.21.15.46:50748) 2024-03-29T08:24:40.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.205+0000 7f8303e00640 1 -- 172.21.15.46:0/2486495830 learned_addr learned my addr 172.21.15.46:0/2486495830 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:40.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.205+0000 7f830b400640 1 --2- 172.21.15.46:0/2486495830 >> v2:172.21.15.73:3300/0 conn(0x7f830c098a40 0x7f830c098eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.205+0000 7f830aa00640 1 --2- 172.21.15.46:0/2486495830 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f830c160100 0x7f830c0cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.205+0000 7f8303e00640 1 -- 172.21.15.46:0/2486495830 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f830c160100 msgr2=0x7f830c0cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:40.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.205+0000 7f8303e00640 1 --2- 172.21.15.46:0/2486495830 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f830c160100 0x7f830c0cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.205+0000 7f8303e00640 1 -- 172.21.15.46:0/2486495830 >> v2:172.21.15.73:3300/0 conn(0x7f830c098a40 msgr2=0x7f830c098eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:40.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.205+0000 7f8303e00640 1 --2- 172.21.15.46:0/2486495830 >> v2:172.21.15.73:3300/0 conn(0x7f830c098a40 0x7f830c098eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.205+0000 7f8303e00640 1 -- 172.21.15.46:0/2486495830 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830c06c310 con 0x7f830c06bac0 2024-03-29T08:24:40.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.205+0000 7f830b400640 1 --2- 172.21.15.46:0/2486495830 >> v2:172.21.15.73:3300/0 conn(0x7f830c098a40 0x7f830c098eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:40.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.205+0000 7f8303e00640 1 --2- 172.21.15.46:0/2486495830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830c06bac0 0x7f830c0b1810 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f830c0709e0 tx=0x7f82f401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4810a766da7b27a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:40.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.205+0000 7f830a000640 1 -- 172.21.15.46:0/2486495830 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40ac630 con 0x7f830c06bac0 2024-03-29T08:24:40.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.205+0000 7f830a000640 1 -- 172.21.15.46:0/2486495830 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f40ac790 con 0x7f830c06bac0 2024-03-29T08:24:40.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.205+0000 7f830a000640 1 -- 172.21.15.46:0/2486495830 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40acb20 con 0x7f830c06bac0 2024-03-29T08:24:40.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8311a00640 1 -- 172.21.15.46:0/2486495830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830c06bac0 msgr2=0x7f830c0b1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:40.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8311a00640 1 --2- 172.21.15.46:0/2486495830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830c06bac0 0x7f830c0b1810 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f830c0709e0 tx=0x7f82f401c520 comp rx=0 tx=0).stop 2024-03-29T08:24:40.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8311a00640 1 -- 172.21.15.46:0/2486495830 shutdown_connections 2024-03-29T08:24:40.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8311a00640 1 --2- 172.21.15.46:0/2486495830 >> v2:172.21.15.73:3300/0 conn(0x7f830c098a40 0x7f830c098eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8311a00640 1 --2- 172.21.15.46:0/2486495830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830c06bac0 0x7f830c0b1810 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8311a00640 1 --2- 172.21.15.46:0/2486495830 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f830c160100 0x7f830c0cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8311a00640 1 -- 172.21.15.46:0/2486495830 >> 172.21.15.46:0/2486495830 conn(0x7f830c0b2190 msgr2=0x7f830c0b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:40.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8311a00640 1 -- 172.21.15.46:0/2486495830 shutdown_connections 2024-03-29T08:24:40.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8311a00640 1 -- 172.21.15.46:0/2486495830 wait complete. 2024-03-29T08:24:40.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8311a00640 1 Processor -- start 2024-03-29T08:24:40.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8311a00640 1 -- start start 2024-03-29T08:24:40.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8311a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f830c06bac0 0x7f830c0b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8311a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830c098a40 0x7f830c158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8311a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f830c160100 0x7f830c159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8311a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c146ef0 con 0x7f830c098a40 2024-03-29T08:24:40.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8311a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830c0ca2c0 con 0x7f830c160100 2024-03-29T08:24:40.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8311a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830c0c9fe0 con 0x7f830c06bac0 2024-03-29T08:24:40.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8303e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830c098a40 0x7f830c158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8303e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830c098a40 0x7f830c158ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50750/0 (socket says 172.21.15.46:50750) 2024-03-29T08:24:40.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8303e00640 1 -- 172.21.15.46:0/3166506109 learned_addr learned my addr 172.21.15.46:0/3166506109 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:40.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f830aa00640 1 --2- 172.21.15.46:0/3166506109 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f830c06bac0 0x7f830c0b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f830b400640 1 --2- 172.21.15.46:0/3166506109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f830c160100 0x7f830c159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8303e00640 1 -- 172.21.15.46:0/3166506109 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f830c06bac0 msgr2=0x7f830c0b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:40.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8303e00640 1 --2- 172.21.15.46:0/3166506109 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f830c06bac0 0x7f830c0b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8303e00640 1 -- 172.21.15.46:0/3166506109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f830c160100 msgr2=0x7f830c159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:40.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8303e00640 1 --2- 172.21.15.46:0/3166506109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f830c160100 0x7f830c159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.209+0000 7f8303e00640 1 -- 172.21.15.46:0/3166506109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f40094d0 con 0x7f830c098a40 2024-03-29T08:24:40.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.213+0000 7f8303e00640 1 --2- 172.21.15.46:0/3166506109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830c098a40 0x7f830c158ac0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f82f401abd0 tx=0x7f82f40192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:40.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.213+0000 7f8302a00640 1 -- 172.21.15.46:0/3166506109 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40b5640 con 0x7f830c098a40 2024-03-29T08:24:40.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.213+0000 7f8302a00640 1 -- 172.21.15.46:0/3166506109 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f40b57a0 con 0x7f830c098a40 2024-03-29T08:24:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.213+0000 7f8302a00640 1 -- 172.21.15.46:0/3166506109 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f40b5b30 con 0x7f830c098a40 2024-03-29T08:24:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.213+0000 7f8311a00640 1 -- 172.21.15.46:0/3166506109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f830c146830 con 0x7f830c098a40 2024-03-29T08:24:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.213+0000 7f8311a00640 1 -- 172.21.15.46:0/3166506109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f830c0b60b0 con 0x7f830c098a40 2024-03-29T08:24:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.213+0000 7f8311a00640 1 -- 172.21.15.46:0/3166506109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830c147030 con 0x7f830c098a40 2024-03-29T08:24:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.213+0000 7f8302a00640 1 -- 172.21.15.46:0/3166506109 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f82f40b3070 con 0x7f830c098a40 2024-03-29T08:24:40.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.213+0000 7f8302a00640 1 --2- 172.21.15.46:0/3166506109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f82e8066690 0x7f82e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.213+0000 7f8302a00640 1 -- 172.21.15.46:0/3166506109 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f82f411dd30 con 0x7f830c098a40 2024-03-29T08:24:40.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.213+0000 7f830aa00640 1 --2- 172.21.15.46:0/3166506109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f82e8066690 0x7f82e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.217+0000 7f830aa00640 1 --2- 172.21.15.46:0/3166506109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f82e8066690 0x7f82e8068b50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009ae0 tx=0x7f82fc009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:40.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.217+0000 7f8302a00640 1 -- 172.21.15.46:0/3166506109 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f830c147030 con 0x7f830c098a40 2024-03-29T08:24:40.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.377+0000 7f8311a00640 1 -- 172.21.15.46:0/3166506109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f830c147030 con 0x7f830c098a40 2024-03-29T08:24:40.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.377+0000 7f8302a00640 1 -- 172.21.15.46:0/3166506109 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f830c147030 con 0x7f830c098a40 2024-03-29T08:24:40.379 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:24:40.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.381+0000 7f8311a00640 1 -- 172.21.15.46:0/3166506109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f82e8066690 msgr2=0x7f82e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:40.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.381+0000 7f8311a00640 1 --2- 172.21.15.46:0/3166506109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f82e8066690 0x7f82e8068b50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009ae0 tx=0x7f82fc009190 comp rx=0 tx=0).stop 2024-03-29T08:24:40.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.381+0000 7f8311a00640 1 -- 172.21.15.46:0/3166506109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830c098a40 msgr2=0x7f830c158ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:40.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.381+0000 7f8311a00640 1 --2- 172.21.15.46:0/3166506109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830c098a40 0x7f830c158ac0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f82f401abd0 tx=0x7f82f40192c0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.381+0000 7f8311a00640 1 -- 172.21.15.46:0/3166506109 shutdown_connections 2024-03-29T08:24:40.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.381+0000 7f8311a00640 1 --2- 172.21.15.46:0/3166506109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f82e8066690 0x7f82e8068b50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.381+0000 7f8311a00640 1 --2- 172.21.15.46:0/3166506109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f830c160100 0x7f830c159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.381+0000 7f8311a00640 1 --2- 172.21.15.46:0/3166506109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f830c098a40 0x7f830c158ac0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.381+0000 7f8311a00640 1 --2- 172.21.15.46:0/3166506109 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f830c06bac0 0x7f830c0b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.381+0000 7f8311a00640 1 -- 172.21.15.46:0/3166506109 >> 172.21.15.46:0/3166506109 conn(0x7f830c0b2190 msgr2=0x7f830c06e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:40.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.381+0000 7f8311a00640 1 -- 172.21.15.46:0/3166506109 shutdown_connections 2024-03-29T08:24:40.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.381+0000 7f8311a00640 1 -- 172.21.15.46:0/3166506109 wait complete. 2024-03-29T08:24:40.401 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T08:24:40.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.541+0000 7fa3bd000640 1 Processor -- start 2024-03-29T08:24:40.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.541+0000 7fa3bd000640 1 -- start start 2024-03-29T08:24:40.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.545+0000 7fa3bd000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa3b815c960 0x7fa3b815cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.545+0000 7fa3bd000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3b815d280 0x7fa3b80cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.545+0000 7fa3bd000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3b80cd0e0 0x7fa3b80cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.545+0000 7fa3bd000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b815e0c0 con 0x7fa3b815d280 2024-03-29T08:24:40.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.545+0000 7fa3bd000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b815d7c0 con 0x7fa3b815c960 2024-03-29T08:24:40.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.545+0000 7fa3bd000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b815daa0 con 0x7fa3b80cd0e0 2024-03-29T08:24:40.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.545+0000 7fa3afe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3b815d280 0x7fa3b80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.545+0000 7fa3afe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3b815d280 0x7fa3b80cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50762/0 (socket says 172.21.15.46:50762) 2024-03-29T08:24:40.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.545+0000 7fa3afe00640 1 -- 172.21.15.46:0/3519503846 learned_addr learned my addr 172.21.15.46:0/3519503846 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:40.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.545+0000 7fa3b6800640 1 --2- 172.21.15.46:0/3519503846 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3b80cd0e0 0x7fa3b80cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.545+0000 7fa3b5e00640 1 --2- 172.21.15.46:0/3519503846 >> v2:172.21.15.73:3300/0 conn(0x7fa3b815c960 0x7fa3b815cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.545+0000 7fa3afe00640 1 -- 172.21.15.46:0/3519503846 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3b80cd0e0 msgr2=0x7fa3b80cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:40.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.545+0000 7fa3afe00640 1 --2- 172.21.15.46:0/3519503846 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3b80cd0e0 0x7fa3b80cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.545+0000 7fa3afe00640 1 -- 172.21.15.46:0/3519503846 >> v2:172.21.15.73:3300/0 conn(0x7fa3b815c960 msgr2=0x7fa3b815cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:40.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.545+0000 7fa3afe00640 1 --2- 172.21.15.46:0/3519503846 >> v2:172.21.15.73:3300/0 conn(0x7fa3b815c960 0x7fa3b815cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.545+0000 7fa3afe00640 1 -- 172.21.15.46:0/3519503846 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b815dd80 con 0x7fa3b815d280 2024-03-29T08:24:40.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.545+0000 7fa3afe00640 1 --2- 172.21.15.46:0/3519503846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b815d280 0x7fa3b80cf7b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8072450 tx=0x7fa3a001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508b6c1a209fe6bb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:40.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3b5400640 1 -- 172.21.15.46:0/3519503846 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00aa430 con 0x7fa3b815d280 2024-03-29T08:24:40.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3b5400640 1 -- 172.21.15.46:0/3519503846 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a00aa590 con 0x7fa3b815d280 2024-03-29T08:24:40.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3b5400640 1 -- 172.21.15.46:0/3519503846 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00aa920 con 0x7fa3b815d280 2024-03-29T08:24:40.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3bd000640 1 -- 172.21.15.46:0/3519503846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b815d280 msgr2=0x7fa3b80cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:40.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3bd000640 1 --2- 172.21.15.46:0/3519503846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b815d280 0x7fa3b80cf7b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8072450 tx=0x7fa3a001c4f0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3bd000640 1 -- 172.21.15.46:0/3519503846 shutdown_connections 2024-03-29T08:24:40.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3bd000640 1 --2- 172.21.15.46:0/3519503846 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3b80cd0e0 0x7fa3b80cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3bd000640 1 --2- 172.21.15.46:0/3519503846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b815d280 0x7fa3b80cf7b0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3bd000640 1 --2- 172.21.15.46:0/3519503846 >> v2:172.21.15.73:3300/0 conn(0x7fa3b815c960 0x7fa3b815cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3bd000640 1 -- 172.21.15.46:0/3519503846 >> 172.21.15.46:0/3519503846 conn(0x7fa3b80718c0 msgr2=0x7fa3b8071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:40.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3bd000640 1 -- 172.21.15.46:0/3519503846 shutdown_connections 2024-03-29T08:24:40.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3bd000640 1 -- 172.21.15.46:0/3519503846 wait complete. 2024-03-29T08:24:40.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3bd000640 1 Processor -- start 2024-03-29T08:24:40.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3bd000640 1 -- start start 2024-03-29T08:24:40.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3bd000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3b80cd0e0 0x7fa3b815b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3bd000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b815c960 0x7fa3b80c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3bd000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3b815d280 0x7fa3b80d0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3bd000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b81542c0 con 0x7fa3b815c960 2024-03-29T08:24:40.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3bd000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80d2820 con 0x7fa3b80cd0e0 2024-03-29T08:24:40.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3bd000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b80d2b00 con 0x7fa3b815d280 2024-03-29T08:24:40.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b815c960 0x7fa3b80c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b815c960 0x7fa3b80c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50772/0 (socket says 172.21.15.46:50772) 2024-03-29T08:24:40.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3afe00640 1 -- 172.21.15.46:0/63114945 learned_addr learned my addr 172.21.15.46:0/63114945 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:40.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3b5e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3b80cd0e0 0x7fa3b815b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3b6800640 1 --2- 172.21.15.46:0/63114945 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3b815d280 0x7fa3b80d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3afe00640 1 -- 172.21.15.46:0/63114945 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3b815d280 msgr2=0x7fa3b80d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:40.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3afe00640 1 --2- 172.21.15.46:0/63114945 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3b815d280 0x7fa3b80d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3afe00640 1 -- 172.21.15.46:0/63114945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3b80cd0e0 msgr2=0x7fa3b815b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:40.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3afe00640 1 --2- 172.21.15.46:0/63114945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3b80cd0e0 0x7fa3b815b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.549+0000 7fa3afe00640 1 -- 172.21.15.46:0/63114945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a00094d0 con 0x7fa3b815c960 2024-03-29T08:24:40.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.553+0000 7fa3afe00640 1 --2- 172.21.15.46:0/63114945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b815c960 0x7fa3b80c5940 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00ac040 tx=0x7fa3a0019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.553+0000 7fa3ae000640 1 -- 172.21.15.46:0/63114945 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00b48d0 con 0x7fa3b815c960 2024-03-29T08:24:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.553+0000 7fa3ae000640 1 -- 172.21.15.46:0/63114945 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a00b4a50 con 0x7fa3b815c960 2024-03-29T08:24:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.553+0000 7fa3ae000640 1 -- 172.21.15.46:0/63114945 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a00b4e00 con 0x7fa3b815c960 2024-03-29T08:24:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.553+0000 7fa3bd000640 1 -- 172.21.15.46:0/63114945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b80d2de0 con 0x7fa3b815c960 2024-03-29T08:24:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.553+0000 7fa3bd000640 1 -- 172.21.15.46:0/63114945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b815b790 con 0x7fa3b815c960 2024-03-29T08:24:40.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.553+0000 7fa3bd000640 1 -- 172.21.15.46:0/63114945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b8154430 con 0x7fa3b815c960 2024-03-29T08:24:40.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.553+0000 7fa3ae000640 1 -- 172.21.15.46:0/63114945 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa3a00c7440 con 0x7fa3b815c960 2024-03-29T08:24:40.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.553+0000 7fa3ae000640 1 --2- 172.21.15.46:0/63114945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa394066690 0x7fa394068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.553+0000 7fa3ae000640 1 -- 172.21.15.46:0/63114945 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa3a011eb50 con 0x7fa3b815c960 2024-03-29T08:24:40.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.553+0000 7fa3b5e00640 1 --2- 172.21.15.46:0/63114945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa394066690 0x7fa394068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.557+0000 7fa3b5e00640 1 --2- 172.21.15.46:0/63114945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa394066690 0x7fa394068b50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8007a60 tx=0x7fa3a8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:40.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.557+0000 7fa3ae000640 1 -- 172.21.15.46:0/63114945 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa3a00ec2b0 con 0x7fa3b815c960 2024-03-29T08:24:40.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.717+0000 7fa3bd000640 1 -- 172.21.15.46:0/63114945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa3b8066c90 con 0x7fa3b815c960 2024-03-29T08:24:40.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.721+0000 7fa3ae000640 1 -- 172.21.15.46:0/63114945 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v88) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa3a00ef8d0 con 0x7fa3b815c960 2024-03-29T08:24:40.719 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:40.720 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-03-29T08:24:40.720 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:24:40.720 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-03-29T08:24:40.720 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-03-29T08:24:40.720 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:40.720 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-03-29T08:24:40.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.721+0000 7fa3bd000640 1 -- 172.21.15.46:0/63114945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa394066690 msgr2=0x7fa394068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:40.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.721+0000 7fa3bd000640 1 --2- 172.21.15.46:0/63114945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa394066690 0x7fa394068b50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8007a60 tx=0x7fa3a8008040 comp rx=0 tx=0).stop 2024-03-29T08:24:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.721+0000 7fa3bd000640 1 -- 172.21.15.46:0/63114945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b815c960 msgr2=0x7fa3b80c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.721+0000 7fa3bd000640 1 --2- 172.21.15.46:0/63114945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b815c960 0x7fa3b80c5940 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa3a00ac040 tx=0x7fa3a0019290 comp rx=0 tx=0).stop 2024-03-29T08:24:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.721+0000 7fa3bd000640 1 -- 172.21.15.46:0/63114945 shutdown_connections 2024-03-29T08:24:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.721+0000 7fa3bd000640 1 --2- 172.21.15.46:0/63114945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa394066690 0x7fa394068b50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.721+0000 7fa3bd000640 1 --2- 172.21.15.46:0/63114945 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa3b815d280 0x7fa3b80d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.721+0000 7fa3bd000640 1 --2- 172.21.15.46:0/63114945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b815c960 0x7fa3b80c5940 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.721+0000 7fa3bd000640 1 --2- 172.21.15.46:0/63114945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3b80cd0e0 0x7fa3b815b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.721+0000 7fa3bd000640 1 -- 172.21.15.46:0/63114945 >> 172.21.15.46:0/63114945 conn(0x7fa3b80718c0 msgr2=0x7fa3b80cfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.721+0000 7fa3bd000640 1 -- 172.21.15.46:0/63114945 shutdown_connections 2024-03-29T08:24:40.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.721+0000 7fa3bd000640 1 -- 172.21.15.46:0/63114945 wait complete. 2024-03-29T08:24:40.740 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T08:24:40.740 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T08:24:40.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.885+0000 7f5066a00640 1 Processor -- start 2024-03-29T08:24:40.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.885+0000 7f5066a00640 1 -- start start 2024-03-29T08:24:40.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.889+0000 7f5066a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f506015a9b0 0x7f506015bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.889+0000 7f5066a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50600cbfc0 con 0x7f50600d00c0 2024-03-29T08:24:40.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.889+0000 7f5066a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50600cb130 con 0x7f50600b0220 2024-03-29T08:24:40.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.889+0000 7f5066a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50600cb3b0 con 0x7f506015a9b0 2024-03-29T08:24:40.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.889+0000 7f505f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f50600d00c0 0x7f50600b0600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37004/0 (socket says 172.21.15.46:37004) 2024-03-29T08:24:40.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.889+0000 7f505f400640 1 -- 172.21.15.46:0/3218429574 learned_addr learned my addr 172.21.15.46:0/3218429574 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:40.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.889+0000 7f5064a00640 1 --2- 172.21.15.46:0/3218429574 >> v2:172.21.15.73:3300/0 conn(0x7f506015a9b0 0x7f506015bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.889+0000 7f505ea00640 1 -- 172.21.15.46:0/3218429574 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679724221 0 0) 0x7f50600cbfc0 con 0x7f50600d00c0 2024-03-29T08:24:40.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.889+0000 7f505ea00640 1 -- 172.21.15.46:0/3218429574 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5044003530 con 0x7f50600d00c0 2024-03-29T08:24:40.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.889+0000 7f505ea00640 1 -- 172.21.15.46:0/3218429574 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947223606 0 0) 0x7f50600cb130 con 0x7f50600b0220 2024-03-29T08:24:40.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.889+0000 7f505ea00640 1 -- 172.21.15.46:0/3218429574 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50600cbfc0 con 0x7f50600b0220 2024-03-29T08:24:40.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.889+0000 7f5064a00640 1 -- 172.21.15.46:0/3218429574 >> v1:172.21.15.73:6789/0 conn(0x7f50600b0220 legacy=0x7f506015b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:40.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.889+0000 7f5064a00640 1 -- 172.21.15.46:0/3218429574 >> v1:172.21.15.46:6789/0 conn(0x7f50600d00c0 legacy=0x7f50600b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:40.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.889+0000 7f5064a00640 1 -- 172.21.15.46:0/3218429574 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50600cb690 con 0x7f506015a9b0 2024-03-29T08:24:40.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.889+0000 7f5064a00640 1 --2- 172.21.15.46:0/3218429574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f506015a9b0 0x7f506015bf40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5058009d90 tx=0x7f505801c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac1e9170e5459b68 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:40.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f505ea00640 1 -- 172.21.15.46:0/3218429574 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50580a5a00 con 0x7f506015a9b0 2024-03-29T08:24:40.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f505ea00640 1 -- 172.21.15.46:0/3218429574 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50580a5b60 con 0x7f506015a9b0 2024-03-29T08:24:40.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f505ea00640 1 -- 172.21.15.46:0/3218429574 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50580ad440 con 0x7f506015a9b0 2024-03-29T08:24:40.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f5066a00640 1 -- 172.21.15.46:0/3218429574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f506015a9b0 msgr2=0x7f506015bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:40.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f5066a00640 1 --2- 172.21.15.46:0/3218429574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f506015a9b0 0x7f506015bf40 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5058009d90 tx=0x7f505801c940 comp rx=0 tx=0).stop 2024-03-29T08:24:40.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f5066a00640 1 -- 172.21.15.46:0/3218429574 shutdown_connections 2024-03-29T08:24:40.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f5066a00640 1 --2- 172.21.15.46:0/3218429574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f506015a9b0 0x7f506015bf40 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f5066a00640 1 -- 172.21.15.46:0/3218429574 >> 172.21.15.46:0/3218429574 conn(0x7f5060098a40 msgr2=0x7f50600cf5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:40.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f5066a00640 1 -- 172.21.15.46:0/3218429574 shutdown_connections 2024-03-29T08:24:40.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f5066a00640 1 -- 172.21.15.46:0/3218429574 wait complete. 2024-03-29T08:24:40.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f5066a00640 1 Processor -- start 2024-03-29T08:24:40.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f5066a00640 1 -- start start 2024-03-29T08:24:40.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f5066a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50600b0220 0x7f506006dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f5066a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50600d00c0 0x7f506006e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f5066a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f506015a9b0 0x7f5060147110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f5066a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50600b5460 con 0x7f50600b0220 2024-03-29T08:24:40.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f5066a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5060158340 con 0x7f50600d00c0 2024-03-29T08:24:40.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f5066a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50600b5180 con 0x7f506015a9b0 2024-03-29T08:24:40.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f505fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50600b0220 0x7f506006dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f505fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50600b0220 0x7f506006dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50788/0 (socket says 172.21.15.46:50788) 2024-03-29T08:24:40.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.893+0000 7f505fe00640 1 -- 172.21.15.46:0/2330374170 learned_addr learned my addr 172.21.15.46:0/2330374170 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:40.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f5064a00640 1 --2- 172.21.15.46:0/2330374170 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f506015a9b0 0x7f5060147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f505f400640 1 --2- 172.21.15.46:0/2330374170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50600d00c0 0x7f506006e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f505fe00640 1 -- 172.21.15.46:0/2330374170 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f506015a9b0 msgr2=0x7f5060147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:40.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f505fe00640 1 --2- 172.21.15.46:0/2330374170 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f506015a9b0 0x7f5060147110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f505fe00640 1 -- 172.21.15.46:0/2330374170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50600d00c0 msgr2=0x7f506006e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:40.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f505fe00640 1 --2- 172.21.15.46:0/2330374170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50600d00c0 0x7f506006e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:40.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f505fe00640 1 -- 172.21.15.46:0/2330374170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5058009a70 con 0x7f50600b0220 2024-03-29T08:24:40.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f5064a00640 1 --2- 172.21.15.46:0/2330374170 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f506015a9b0 0x7f5060147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:24:40.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f505f400640 1 --2- 172.21.15.46:0/2330374170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50600d00c0 0x7f506006e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:24:40.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f505fe00640 1 --2- 172.21.15.46:0/2330374170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50600b0220 0x7f506006dce0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f505000c430 tx=0x7f505000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:40.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f5057e00640 1 -- 172.21.15.46:0/2330374170 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50500052d0 con 0x7f50600b0220 2024-03-29T08:24:40.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f5057e00640 1 -- 172.21.15.46:0/2330374170 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5050005450 con 0x7f50600b0220 2024-03-29T08:24:40.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f5066a00640 1 -- 172.21.15.46:0/2330374170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5060157cf0 con 0x7f50600b0220 2024-03-29T08:24:40.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f5057e00640 1 -- 172.21.15.46:0/2330374170 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050003840 con 0x7f50600b0220 2024-03-29T08:24:40.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f5066a00640 1 -- 172.21.15.46:0/2330374170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50601587d0 con 0x7f50600b0220 2024-03-29T08:24:40.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f5066a00640 1 -- 172.21.15.46:0/2330374170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5060157b50 con 0x7f50600b0220 2024-03-29T08:24:40.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f5057e00640 1 -- 172.21.15.46:0/2330374170 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f50500039a0 con 0x7f50600b0220 2024-03-29T08:24:40.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f5057e00640 1 --2- 172.21.15.46:0/2330374170 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5040066690 0x7f5040068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:40.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.897+0000 7f5057e00640 1 -- 172.21.15.46:0/2330374170 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5050083540 con 0x7f50600b0220 2024-03-29T08:24:40.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.901+0000 7f505f400640 1 --2- 172.21.15.46:0/2330374170 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5040066690 0x7f5040068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:40.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.901+0000 7f505f400640 1 --2- 172.21.15.46:0/2330374170 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5040066690 0x7f5040068b50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f50600c9700 tx=0x7f504c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:40.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:40.901+0000 7f5057e00640 1 -- 172.21.15.46:0/2330374170 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5060157b50 con 0x7f50600b0220 2024-03-29T08:24:41.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:41.065+0000 7f5066a00640 1 -- 172.21.15.46:0/2330374170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5060157b50 con 0x7f50600b0220 2024-03-29T08:24:42.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.069+0000 7f5057e00640 1 -- 172.21.15.46:0/2330374170 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v89) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5060157b50 con 0x7f50600b0220 2024-03-29T08:24:42.070 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-03-29T08:24:42.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.073+0000 7f5066a00640 1 -- 172.21.15.46:0/2330374170 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5040066690 msgr2=0x7f5040068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:42.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.073+0000 7f5066a00640 1 --2- 172.21.15.46:0/2330374170 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5040066690 0x7f5040068b50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f50600c9700 tx=0x7f504c009040 comp rx=0 tx=0).stop 2024-03-29T08:24:42.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.073+0000 7f5066a00640 1 -- 172.21.15.46:0/2330374170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50600b0220 msgr2=0x7f506006dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:42.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.073+0000 7f5066a00640 1 --2- 172.21.15.46:0/2330374170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50600b0220 0x7f506006dce0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f505000c430 tx=0x7f505000c8f0 comp rx=0 tx=0).stop 2024-03-29T08:24:42.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.073+0000 7f5066a00640 1 -- 172.21.15.46:0/2330374170 shutdown_connections 2024-03-29T08:24:42.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.073+0000 7f5066a00640 1 --2- 172.21.15.46:0/2330374170 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5040066690 0x7f5040068b50 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:42.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.073+0000 7f5066a00640 1 --2- 172.21.15.46:0/2330374170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50600d00c0 0x7f506006e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:42.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.073+0000 7f5066a00640 1 --2- 172.21.15.46:0/2330374170 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f506015a9b0 0x7f5060147110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:42.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.073+0000 7f5066a00640 1 --2- 172.21.15.46:0/2330374170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50600b0220 0x7f506006dce0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:42.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.073+0000 7f5066a00640 1 -- 172.21.15.46:0/2330374170 >> 172.21.15.46:0/2330374170 conn(0x7f5060098a40 msgr2=0x7f5060069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:42.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.077+0000 7f5066a00640 1 -- 172.21.15.46:0/2330374170 shutdown_connections 2024-03-29T08:24:42.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.077+0000 7f5066a00640 1 -- 172.21.15.46:0/2330374170 wait complete. 2024-03-29T08:24:42.102 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T08:24:42.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.257+0000 7fbc85000640 1 Processor -- start 2024-03-29T08:24:42.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.257+0000 7fbc85000640 1 -- start start 2024-03-29T08:24:42.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.257+0000 7fbc85000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc8015d480 0x7fbc801594b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:42.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.257+0000 7fbc85000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbc801599f0 0x7fbc800702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:42.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.257+0000 7fbc85000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8015ca30 con 0x7fbc800b3850 2024-03-29T08:24:42.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.257+0000 7fbc85000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc800b20f0 con 0x7fbc8015d480 2024-03-29T08:24:42.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.257+0000 7fbc85000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc800b23d0 con 0x7fbc801599f0 2024-03-29T08:24:42.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.257+0000 7fbc7d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc8015d480 0x7fbc801594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:42.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.257+0000 7fbc7d400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc8015d480 0x7fbc801594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50804/0 (socket says 172.21.15.46:50804) 2024-03-29T08:24:42.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.257+0000 7fbc7de00640 1 --1- 172.21.15.46:0/2452192255 >> v1:172.21.15.46:6789/0 conn(0x7fbc800b3850 0x7fbc800b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37018/0 (socket says 172.21.15.46:37018) 2024-03-29T08:24:42.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.257+0000 7fbc7d400640 1 -- 172.21.15.46:0/2452192255 learned_addr learned my addr 172.21.15.46:0/2452192255 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:42.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.261+0000 7fbc7e800640 1 --2- 172.21.15.46:0/2452192255 >> v2:172.21.15.73:3300/0 conn(0x7fbc801599f0 0x7fbc800702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:42.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.261+0000 7fbc7d400640 1 -- 172.21.15.46:0/2452192255 >> v2:172.21.15.73:3300/0 conn(0x7fbc801599f0 msgr2=0x7fbc800702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:42.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.261+0000 7fbc7d400640 1 --2- 172.21.15.46:0/2452192255 >> v2:172.21.15.73:3300/0 conn(0x7fbc801599f0 0x7fbc800702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:42.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.261+0000 7fbc7d400640 1 -- 172.21.15.46:0/2452192255 >> v1:172.21.15.46:6789/0 conn(0x7fbc800b3850 legacy=0x7fbc800b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:42.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.261+0000 7fbc7d400640 1 -- 172.21.15.46:0/2452192255 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8015cc80 con 0x7fbc8015d480 2024-03-29T08:24:42.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.261+0000 7fbc7d400640 1 --2- 172.21.15.46:0/2452192255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8015d480 0x7fbc801594b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fbc800b2dd0 tx=0x7fbc6c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c6d6df01710015 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:42.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.261+0000 7fbc7ca00640 1 -- 172.21.15.46:0/2452192255 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0ab680 con 0x7fbc8015d480 2024-03-29T08:24:42.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.261+0000 7fbc7ca00640 1 -- 172.21.15.46:0/2452192255 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6c0ab7e0 con 0x7fbc8015d480 2024-03-29T08:24:42.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.261+0000 7fbc7ca00640 1 -- 172.21.15.46:0/2452192255 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0abb70 con 0x7fbc8015d480 2024-03-29T08:24:42.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc85000640 1 -- 172.21.15.46:0/2452192255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8015d480 msgr2=0x7fbc801594b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:42.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc85000640 1 --2- 172.21.15.46:0/2452192255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8015d480 0x7fbc801594b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fbc800b2dd0 tx=0x7fbc6c01c460 comp rx=0 tx=0).stop 2024-03-29T08:24:42.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc85000640 1 -- 172.21.15.46:0/2452192255 shutdown_connections 2024-03-29T08:24:42.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc85000640 1 --2- 172.21.15.46:0/2452192255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8015d480 0x7fbc801594b0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:42.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc85000640 1 --2- 172.21.15.46:0/2452192255 >> v2:172.21.15.73:3300/0 conn(0x7fbc801599f0 0x7fbc800702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:42.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc85000640 1 -- 172.21.15.46:0/2452192255 >> 172.21.15.46:0/2452192255 conn(0x7fbc80066080 msgr2=0x7fbc80069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:42.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc85000640 1 -- 172.21.15.46:0/2452192255 shutdown_connections 2024-03-29T08:24:42.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc85000640 1 -- 172.21.15.46:0/2452192255 wait complete. 2024-03-29T08:24:42.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc85000640 1 Processor -- start 2024-03-29T08:24:42.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc85000640 1 -- start start 2024-03-29T08:24:42.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc85000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc800b3850 0x7fbc800a9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:42.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc85000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc801599f0 0x7fbc80150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:42.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc85000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8015d480 0x7fbc80150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:42.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc85000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8015c530 con 0x7fbc8015d480 2024-03-29T08:24:42.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc85000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc801f7710 con 0x7fbc800b3850 2024-03-29T08:24:42.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc85000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc801f6e20 con 0x7fbc801599f0 2024-03-29T08:24:42.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc7e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8015d480 0x7fbc80150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:42.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc7e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8015d480 0x7fbc80150c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50816/0 (socket says 172.21.15.46:50816) 2024-03-29T08:24:42.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc7e800640 1 -- 172.21.15.46:0/3032739629 learned_addr learned my addr 172.21.15.46:0/3032739629 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:42.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc7de00640 1 --2- 172.21.15.46:0/3032739629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc800b3850 0x7fbc800a9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:42.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc7d400640 1 --2- 172.21.15.46:0/3032739629 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc801599f0 0x7fbc80150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:42.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc7e800640 1 -- 172.21.15.46:0/3032739629 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc801599f0 msgr2=0x7fbc80150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:42.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc7e800640 1 --2- 172.21.15.46:0/3032739629 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc801599f0 0x7fbc80150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:42.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc7e800640 1 -- 172.21.15.46:0/3032739629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc800b3850 msgr2=0x7fbc800a9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:42.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc7e800640 1 --2- 172.21.15.46:0/3032739629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc800b3850 0x7fbc800a9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:42.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc7e800640 1 -- 172.21.15.46:0/3032739629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c0094d0 con 0x7fbc8015d480 2024-03-29T08:24:42.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.265+0000 7fbc7de00640 1 --2- 172.21.15.46:0/3032739629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc800b3850 0x7fbc800a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:24:42.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.269+0000 7fbc7e800640 1 --2- 172.21.15.46:0/3032739629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8015d480 0x7fbc80150c70 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fbc78007e40 tx=0x7fbc7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:42.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.269+0000 7fbc76000640 1 -- 172.21.15.46:0/3032739629 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78014620 con 0x7fbc8015d480 2024-03-29T08:24:42.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.269+0000 7fbc76000640 1 -- 172.21.15.46:0/3032739629 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7800ee90 con 0x7fbc8015d480 2024-03-29T08:24:42.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.269+0000 7fbc76000640 1 -- 172.21.15.46:0/3032739629 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc780149e0 con 0x7fbc8015d480 2024-03-29T08:24:42.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.269+0000 7fbc85000640 1 -- 172.21.15.46:0/3032739629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc800b1ec0 con 0x7fbc8015d480 2024-03-29T08:24:42.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.269+0000 7fbc85000640 1 -- 172.21.15.46:0/3032739629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc801f70c0 con 0x7fbc8015d480 2024-03-29T08:24:42.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.269+0000 7fbc85000640 1 -- 172.21.15.46:0/3032739629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc801f7880 con 0x7fbc8015d480 2024-03-29T08:24:42.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.269+0000 7fbc76000640 1 -- 172.21.15.46:0/3032739629 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbc78014b40 con 0x7fbc8015d480 2024-03-29T08:24:42.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.269+0000 7fbc76000640 1 --2- 172.21.15.46:0/3032739629 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc60066690 0x7fbc60068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:42.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.269+0000 7fbc76000640 1 -- 172.21.15.46:0/3032739629 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fbc78084820 con 0x7fbc8015d480 2024-03-29T08:24:42.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.269+0000 7fbc7de00640 1 --2- 172.21.15.46:0/3032739629 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc60066690 0x7fbc60068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:42.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.273+0000 7fbc7de00640 1 --2- 172.21.15.46:0/3032739629 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc60066690 0x7fbc60068b50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbc70000f70 tx=0x7fbc70007e50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:42.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.273+0000 7fbc76000640 1 -- 172.21.15.46:0/3032739629 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbc78051e10 con 0x7fbc8015d480 2024-03-29T08:24:42.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:42.437+0000 7fbc85000640 1 -- 172.21.15.46:0/3032739629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbc80071a90 con 0x7fbc8015d480 2024-03-29T08:24:43.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.073+0000 7fbc76000640 1 -- 172.21.15.46:0/3032739629 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v90) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbc78055430 con 0x7fbc8015d480 2024-03-29T08:24:43.076 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-03-29T08:24:43.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.081+0000 7fbc85000640 1 -- 172.21.15.46:0/3032739629 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc60066690 msgr2=0x7fbc60068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:43.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.081+0000 7fbc85000640 1 --2- 172.21.15.46:0/3032739629 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc60066690 0x7fbc60068b50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbc70000f70 tx=0x7fbc70007e50 comp rx=0 tx=0).stop 2024-03-29T08:24:43.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.081+0000 7fbc85000640 1 -- 172.21.15.46:0/3032739629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8015d480 msgr2=0x7fbc80150c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:43.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.081+0000 7fbc85000640 1 --2- 172.21.15.46:0/3032739629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8015d480 0x7fbc80150c70 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fbc78007e40 tx=0x7fbc7800e4b0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.085+0000 7fbc85000640 1 -- 172.21.15.46:0/3032739629 shutdown_connections 2024-03-29T08:24:43.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.085+0000 7fbc85000640 1 --2- 172.21.15.46:0/3032739629 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc60066690 0x7fbc60068b50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.085+0000 7fbc85000640 1 --2- 172.21.15.46:0/3032739629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8015d480 0x7fbc80150c70 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.085+0000 7fbc85000640 1 --2- 172.21.15.46:0/3032739629 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc801599f0 0x7fbc80150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.085+0000 7fbc85000640 1 --2- 172.21.15.46:0/3032739629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc800b3850 0x7fbc800a9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.085+0000 7fbc85000640 1 -- 172.21.15.46:0/3032739629 >> 172.21.15.46:0/3032739629 conn(0x7fbc80066080 msgr2=0x7fbc80152db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.085+0000 7fbc85000640 1 -- 172.21.15.46:0/3032739629 shutdown_connections 2024-03-29T08:24:43.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.085+0000 7fbc85000640 1 -- 172.21.15.46:0/3032739629 wait complete. 2024-03-29T08:24:43.116 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T08:24:43.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.261+0000 7f6f4d000640 1 Processor -- start 2024-03-29T08:24:43.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.261+0000 7f6f4d000640 1 -- start start 2024-03-29T08:24:43.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.261+0000 7f6f4d000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f480b1570 0x7f6f480b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:43.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.261+0000 7f6f4d000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6f48098a40 0x7f6f48098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:43.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.261+0000 7f6f4d000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f480cb3f0 con 0x7f6f4815dfb0 2024-03-29T08:24:43.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.261+0000 7f6f4d000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f48099410 con 0x7f6f48098a40 2024-03-29T08:24:43.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.261+0000 7f6f4d000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480b1e00 con 0x7f6f480b1570 2024-03-29T08:24:43.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.261+0000 7f6f46800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6f48098a40 0x7f6f48098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:43.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.261+0000 7f6f45e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6f4815dfb0 0x7f6f480ced10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37020/0 (socket says 172.21.15.46:37020) 2024-03-29T08:24:43.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.261+0000 7f6f45e00640 1 -- 172.21.15.46:0/2033492959 learned_addr learned my addr 172.21.15.46:0/2033492959 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:43.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.265+0000 7f6f3fe00640 1 --2- 172.21.15.46:0/2033492959 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f480b1570 0x7f6f480b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:43.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.265+0000 7f6f46800640 1 -- 172.21.15.46:0/2033492959 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f480b1570 msgr2=0x7f6f480b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:43.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.265+0000 7f6f46800640 1 --2- 172.21.15.46:0/2033492959 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f480b1570 0x7f6f480b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.265+0000 7f6f46800640 1 -- 172.21.15.46:0/2033492959 >> v1:172.21.15.46:6789/0 conn(0x7f6f4815dfb0 legacy=0x7f6f480ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:43.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.265+0000 7f6f46800640 1 -- 172.21.15.46:0/2033492959 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f480cb620 con 0x7f6f48098a40 2024-03-29T08:24:43.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.265+0000 7f6f45400640 1 -- 172.21.15.46:0/2033492959 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537021056 0 0) 0x7f6f480cb3f0 con 0x7f6f4815dfb0 2024-03-29T08:24:43.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.265+0000 7f6f46800640 1 --2- 172.21.15.46:0/2033492959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f48098a40 0x7f6f48098ed0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009d90 tx=0x7f6f4001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27f3a26938c3d18 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:43.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.265+0000 7f6f45400640 1 -- 172.21.15.46:0/2033492959 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f400a5910 con 0x7f6f48098a40 2024-03-29T08:24:43.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.265+0000 7f6f45400640 1 -- 172.21.15.46:0/2033492959 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f400a5a70 con 0x7f6f48098a40 2024-03-29T08:24:43.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.265+0000 7f6f45400640 1 -- 172.21.15.46:0/2033492959 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f400a5e00 con 0x7f6f48098a40 2024-03-29T08:24:43.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.265+0000 7f6f4d000640 1 -- 172.21.15.46:0/2033492959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f48098a40 msgr2=0x7f6f48098ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:43.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.265+0000 7f6f4d000640 1 --2- 172.21.15.46:0/2033492959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f48098a40 0x7f6f48098ed0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f6f40009d90 tx=0x7f6f4001c940 comp rx=0 tx=0).stop 2024-03-29T08:24:43.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f4d000640 1 -- 172.21.15.46:0/2033492959 shutdown_connections 2024-03-29T08:24:43.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f4d000640 1 --2- 172.21.15.46:0/2033492959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f48098a40 0x7f6f48098ed0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f4d000640 1 --2- 172.21.15.46:0/2033492959 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f480b1570 0x7f6f480b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f4d000640 1 -- 172.21.15.46:0/2033492959 >> 172.21.15.46:0/2033492959 conn(0x7f6f480b2220 msgr2=0x7f6f480b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:43.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f4d000640 1 -- 172.21.15.46:0/2033492959 shutdown_connections 2024-03-29T08:24:43.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f4d000640 1 -- 172.21.15.46:0/2033492959 wait complete. 2024-03-29T08:24:43.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f4d000640 1 Processor -- start 2024-03-29T08:24:43.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f4d000640 1 -- start start 2024-03-29T08:24:43.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f4d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f48098a40 0x7f6f48153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:43.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f4d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f480b1570 0x7f6f481636b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:43.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f4d000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f4815dfb0 0x7f6f480cdd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f4d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4806b440 con 0x7f6f48098a40 2024-03-29T08:24:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f4d000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4806b720 con 0x7f6f480b1570 2024-03-29T08:24:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f4d000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4806ba00 con 0x7f6f4815dfb0 2024-03-29T08:24:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f45e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f48098a40 0x7f6f48153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f45e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f48098a40 0x7f6f48153830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50838/0 (socket says 172.21.15.46:50838) 2024-03-29T08:24:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f45e00640 1 -- 172.21.15.46:0/332576395 learned_addr learned my addr 172.21.15.46:0/332576395 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f3fe00640 1 --2- 172.21.15.46:0/332576395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f480b1570 0x7f6f481636b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f46800640 1 --2- 172.21.15.46:0/332576395 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f4815dfb0 0x7f6f480cdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f45e00640 1 -- 172.21.15.46:0/332576395 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f4815dfb0 msgr2=0x7f6f480cdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f45e00640 1 --2- 172.21.15.46:0/332576395 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f4815dfb0 0x7f6f480cdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f45e00640 1 -- 172.21.15.46:0/332576395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f480b1570 msgr2=0x7f6f481636b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f45e00640 1 --2- 172.21.15.46:0/332576395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f480b1570 0x7f6f481636b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f45e00640 1 -- 172.21.15.46:0/332576395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f40009a70 con 0x7f6f48098a40 2024-03-29T08:24:43.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f45e00640 1 --2- 172.21.15.46:0/332576395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f48098a40 0x7f6f48153830 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800c480 tx=0x7f6f3800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:43.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.269+0000 7f6f3e000640 1 -- 172.21.15.46:0/332576395 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f380074e0 con 0x7f6f48098a40 2024-03-29T08:24:43.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.273+0000 7f6f3e000640 1 -- 172.21.15.46:0/332576395 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f38007660 con 0x7f6f48098a40 2024-03-29T08:24:43.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.273+0000 7f6f3e000640 1 -- 172.21.15.46:0/332576395 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38004a30 con 0x7f6f48098a40 2024-03-29T08:24:43.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.273+0000 7f6f4d000640 1 -- 172.21.15.46:0/332576395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f4806bce0 con 0x7f6f48098a40 2024-03-29T08:24:43.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.273+0000 7f6f4d000640 1 -- 172.21.15.46:0/332576395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f48153d70 con 0x7f6f48098a40 2024-03-29T08:24:43.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.273+0000 7f6f4d000640 1 -- 172.21.15.46:0/332576395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f4809e700 con 0x7f6f48098a40 2024-03-29T08:24:43.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.273+0000 7f6f3e000640 1 -- 172.21.15.46:0/332576395 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6f3801b360 con 0x7f6f48098a40 2024-03-29T08:24:43.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.273+0000 7f6f3e000640 1 --2- 172.21.15.46:0/332576395 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f24066690 0x7f6f24068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:43.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.277+0000 7f6f3e000640 1 -- 172.21.15.46:0/332576395 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f6f38083520 con 0x7f6f48098a40 2024-03-29T08:24:43.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.277+0000 7f6f3fe00640 1 --2- 172.21.15.46:0/332576395 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f24066690 0x7f6f24068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:43.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.277+0000 7f6f3fe00640 1 --2- 172.21.15.46:0/332576395 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f24066690 0x7f6f24068b50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6f4806d080 tx=0x7f6f30009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:43.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.281+0000 7f6f3e000640 1 -- 172.21.15.46:0/332576395 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6f38050a10 con 0x7f6f48098a40 2024-03-29T08:24:43.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.473+0000 7f6f4d000640 1 -- 172.21.15.46:0/332576395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6f4804e120 con 0x7f6f48098a40 2024-03-29T08:24:43.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.481+0000 7f6f3e000640 1 -- 172.21.15.46:0/332576395 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v53) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6f38054030 con 0x7f6f48098a40 2024-03-29T08:24:43.481 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 12 and data pool 13 2024-03-29T08:24:43.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.485+0000 7f6f4d000640 1 -- 172.21.15.46:0/332576395 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f24066690 msgr2=0x7f6f24068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:43.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.485+0000 7f6f4d000640 1 --2- 172.21.15.46:0/332576395 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f24066690 0x7f6f24068b50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6f4806d080 tx=0x7f6f30009190 comp rx=0 tx=0).stop 2024-03-29T08:24:43.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.485+0000 7f6f4d000640 1 -- 172.21.15.46:0/332576395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f48098a40 msgr2=0x7f6f48153830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:43.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.485+0000 7f6f4d000640 1 --2- 172.21.15.46:0/332576395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f48098a40 0x7f6f48153830 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6f3800c480 tx=0x7f6f3800c940 comp rx=0 tx=0).stop 2024-03-29T08:24:43.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.485+0000 7f6f4d000640 1 -- 172.21.15.46:0/332576395 shutdown_connections 2024-03-29T08:24:43.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.485+0000 7f6f4d000640 1 --2- 172.21.15.46:0/332576395 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f24066690 0x7f6f24068b50 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.485+0000 7f6f4d000640 1 --2- 172.21.15.46:0/332576395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f480b1570 0x7f6f481636b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.485+0000 7f6f4d000640 1 --2- 172.21.15.46:0/332576395 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f4815dfb0 0x7f6f480cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.485+0000 7f6f4d000640 1 --2- 172.21.15.46:0/332576395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f48098a40 0x7f6f48153830 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.485+0000 7f6f4d000640 1 -- 172.21.15.46:0/332576395 >> 172.21.15.46:0/332576395 conn(0x7f6f480b2220 msgr2=0x7f6f480cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:43.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.485+0000 7f6f4d000640 1 -- 172.21.15.46:0/332576395 shutdown_connections 2024-03-29T08:24:43.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.485+0000 7f6f4d000640 1 -- 172.21.15.46:0/332576395 wait complete. 2024-03-29T08:24:43.525 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:24:43.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.665+0000 7f80b2c00640 1 Processor -- start 2024-03-29T08:24:43.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.665+0000 7f80b2c00640 1 -- start start 2024-03-29T08:24:43.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.665+0000 7f80b2c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f80ac15cd90 0x7f80ac06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:43.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.665+0000 7f80b2c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80ac06d130 con 0x7f80ac15d940 2024-03-29T08:24:43.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.665+0000 7f80b2c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80ac15b8f0 con 0x7f80ac1593d0 2024-03-29T08:24:43.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.665+0000 7f80b2c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac15ad50 con 0x7f80ac15cd90 2024-03-29T08:24:43.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.665+0000 7f80b0c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f80ac15d940 0x7f80ac1597b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37032/0 (socket says 172.21.15.46:37032) 2024-03-29T08:24:43.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.665+0000 7f80b0c00640 1 -- 172.21.15.46:0/2350023136 learned_addr learned my addr 172.21.15.46:0/2350023136 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:43.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.665+0000 7f80a3400640 1 --2- 172.21.15.46:0/2350023136 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f80ac15cd90 0x7f80ac06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:43.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.665+0000 7f80ab400640 1 -- 172.21.15.46:0/2350023136 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138122428 0 0) 0x7f80ac06d130 con 0x7f80ac15d940 2024-03-29T08:24:43.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.665+0000 7f80ab400640 1 -- 172.21.15.46:0/2350023136 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8090003530 con 0x7f80ac15d940 2024-03-29T08:24:43.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.669+0000 7f80ab400640 1 -- 172.21.15.46:0/2350023136 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 533606917 0 0) 0x7f80ac15b8f0 con 0x7f80ac1593d0 2024-03-29T08:24:43.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.669+0000 7f80ab400640 1 -- 172.21.15.46:0/2350023136 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80ac06d130 con 0x7f80ac1593d0 2024-03-29T08:24:43.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.669+0000 7f80a3400640 1 -- 172.21.15.46:0/2350023136 >> v1:172.21.15.73:6789/0 conn(0x7f80ac1593d0 legacy=0x7f80ac15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:43.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.669+0000 7f80a3400640 1 -- 172.21.15.46:0/2350023136 >> v1:172.21.15.46:6789/0 conn(0x7f80ac15d940 legacy=0x7f80ac1597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:43.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.669+0000 7f80a3400640 1 -- 172.21.15.46:0/2350023136 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ac15afc0 con 0x7f80ac15cd90 2024-03-29T08:24:43.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.669+0000 7f80a3400640 1 --2- 172.21.15.46:0/2350023136 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f80ac15cd90 0x7f80ac06cbf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f80940097f0 tx=0x7f809401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6141dce6d1157a22 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:43.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.669+0000 7f80ab400640 1 -- 172.21.15.46:0/2350023136 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80940aa640 con 0x7f80ac15cd90 2024-03-29T08:24:43.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.669+0000 7f80ab400640 1 -- 172.21.15.46:0/2350023136 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80940aa7a0 con 0x7f80ac15cd90 2024-03-29T08:24:43.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.669+0000 7f80ab400640 1 -- 172.21.15.46:0/2350023136 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80940aab30 con 0x7f80ac15cd90 2024-03-29T08:24:43.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.669+0000 7f80b2c00640 1 -- 172.21.15.46:0/2350023136 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f80ac15cd90 msgr2=0x7f80ac06cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:43.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.669+0000 7f80b2c00640 1 --2- 172.21.15.46:0/2350023136 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f80ac15cd90 0x7f80ac06cbf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f80940097f0 tx=0x7f809401c4e0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.669+0000 7f80b2c00640 1 -- 172.21.15.46:0/2350023136 shutdown_connections 2024-03-29T08:24:43.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.669+0000 7f80b2c00640 1 --2- 172.21.15.46:0/2350023136 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f80ac15cd90 0x7f80ac06cbf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.669+0000 7f80b2c00640 1 -- 172.21.15.46:0/2350023136 >> 172.21.15.46:0/2350023136 conn(0x7f80ac066110 msgr2=0x7f80ac069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:43.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.669+0000 7f80b2c00640 1 -- 172.21.15.46:0/2350023136 shutdown_connections 2024-03-29T08:24:43.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.669+0000 7f80b2c00640 1 -- 172.21.15.46:0/2350023136 wait complete. 2024-03-29T08:24:43.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b2c00640 1 Processor -- start 2024-03-29T08:24:43.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b2c00640 1 -- start start 2024-03-29T08:24:43.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b2c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f80ac1593d0 0x7f80ac151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:43.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b2c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80ac15cd90 0x7f80ac151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:43.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b2c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac15d940 0x7f80ac14ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:43.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b2c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac156060 con 0x7f80ac15d940 2024-03-29T08:24:43.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b2c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac156340 con 0x7f80ac15cd90 2024-03-29T08:24:43.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b2c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac156620 con 0x7f80ac1593d0 2024-03-29T08:24:43.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac15d940 0x7f80ac14ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:43.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac15d940 0x7f80ac14ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50840/0 (socket says 172.21.15.46:50840) 2024-03-29T08:24:43.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b0c00640 1 -- 172.21.15.46:0/3336814069 learned_addr learned my addr 172.21.15.46:0/3336814069 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:43.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80a3400640 1 --2- 172.21.15.46:0/3336814069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80ac15cd90 0x7f80ac151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:43.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80abe00640 1 --2- 172.21.15.46:0/3336814069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f80ac1593d0 0x7f80ac151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:43.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b0c00640 1 -- 172.21.15.46:0/3336814069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f80ac1593d0 msgr2=0x7f80ac151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:43.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b0c00640 1 --2- 172.21.15.46:0/3336814069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f80ac1593d0 0x7f80ac151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b0c00640 1 -- 172.21.15.46:0/3336814069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80ac15cd90 msgr2=0x7f80ac151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:43.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b0c00640 1 --2- 172.21.15.46:0/3336814069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80ac15cd90 0x7f80ac151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b0c00640 1 -- 172.21.15.46:0/3336814069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8094009420 con 0x7f80ac15d940 2024-03-29T08:24:43.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b0c00640 1 --2- 172.21.15.46:0/3336814069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac15d940 0x7f80ac14ab80 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f80a400caa0 tx=0x7f80a400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:43.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80a8c00640 1 -- 172.21.15.46:0/3336814069 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a400ac00 con 0x7f80ac15d940 2024-03-29T08:24:43.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80a8c00640 1 -- 172.21.15.46:0/3336814069 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a400ad80 con 0x7f80ac15d940 2024-03-29T08:24:43.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80a8c00640 1 -- 172.21.15.46:0/3336814069 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4012640 con 0x7f80ac15d940 2024-03-29T08:24:43.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b2c00640 1 -- 172.21.15.46:0/3336814069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ac14f170 con 0x7f80ac15d940 2024-03-29T08:24:43.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.673+0000 7f80b2c00640 1 -- 172.21.15.46:0/3336814069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ac14bae0 con 0x7f80ac15d940 2024-03-29T08:24:43.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.677+0000 7f80b2c00640 1 -- 172.21.15.46:0/3336814069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ac066c90 con 0x7f80ac15d940 2024-03-29T08:24:43.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.677+0000 7f80a8c00640 1 -- 172.21.15.46:0/3336814069 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f80a4007070 con 0x7f80ac15d940 2024-03-29T08:24:43.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.677+0000 7f80a8c00640 1 --2- 172.21.15.46:0/3336814069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8088066690 0x7f8088068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:43.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.677+0000 7f80a8c00640 1 -- 172.21.15.46:0/3336814069 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80a4084080 con 0x7f80ac15d940 2024-03-29T08:24:43.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.677+0000 7f80abe00640 1 --2- 172.21.15.46:0/3336814069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8088066690 0x7f8088068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:43.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.677+0000 7f80abe00640 1 --2- 172.21.15.46:0/3336814069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8088066690 0x7f8088068b50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f809c0011f0 tx=0x7f809c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:43.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.681+0000 7f80a8c00640 1 -- 172.21.15.46:0/3336814069 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f80a40b5980 con 0x7f80ac15d940 2024-03-29T08:24:43.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.845+0000 7f80b2c00640 1 -- 172.21.15.46:0/3336814069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80ac1653b0 con 0x7f80ac15d940 2024-03-29T08:24:43.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.845+0000 7f80a8c00640 1 -- 172.21.15.46:0/3336814069 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16069 (secure 0 0 0) 0x7f80a4051490 con 0x7f80ac15d940 2024-03-29T08:24:43.845 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:43.845 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":91,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:24:43.476025+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:24:41.113618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-29T08:24:42.487380+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:24:43.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.845+0000 7f80b2c00640 1 -- 172.21.15.46:0/3336814069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8088066690 msgr2=0x7f8088068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:43.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.845+0000 7f80b2c00640 1 --2- 172.21.15.46:0/3336814069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8088066690 0x7f8088068b50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f809c0011f0 tx=0x7f809c009040 comp rx=0 tx=0).stop 2024-03-29T08:24:43.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.845+0000 7f80b2c00640 1 -- 172.21.15.46:0/3336814069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac15d940 msgr2=0x7f80ac14ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:43.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.845+0000 7f80b2c00640 1 --2- 172.21.15.46:0/3336814069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac15d940 0x7f80ac14ab80 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f80a400caa0 tx=0x7f80a400cf60 comp rx=0 tx=0).stop 2024-03-29T08:24:43.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.845+0000 7f80b2c00640 1 -- 172.21.15.46:0/3336814069 shutdown_connections 2024-03-29T08:24:43.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.845+0000 7f80b2c00640 1 --2- 172.21.15.46:0/3336814069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8088066690 0x7f8088068b50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.845+0000 7f80b2c00640 1 --2- 172.21.15.46:0/3336814069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80ac15cd90 0x7f80ac151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.845+0000 7f80b2c00640 1 --2- 172.21.15.46:0/3336814069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f80ac15d940 0x7f80ac14ab80 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.845+0000 7f80b2c00640 1 --2- 172.21.15.46:0/3336814069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f80ac1593d0 0x7f80ac151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:43.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.845+0000 7f80b2c00640 1 -- 172.21.15.46:0/3336814069 >> 172.21.15.46:0/3336814069 conn(0x7f80ac066110 msgr2=0x7f80ac0af830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:43.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.845+0000 7f80b2c00640 1 -- 172.21.15.46:0/3336814069 shutdown_connections 2024-03-29T08:24:43.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:43.845+0000 7f80b2c00640 1 -- 172.21.15.46:0/3336814069 wait complete. 2024-03-29T08:24:43.866 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:24:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.005+0000 7f14f3c00640 1 Processor -- start 2024-03-29T08:24:44.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.005+0000 7f14f3c00640 1 -- start start 2024-03-29T08:24:44.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.005+0000 7f14f3c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f14ec0caf50 0x7f14ec0cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:44.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.005+0000 7f14f3c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ec0cb3e0 con 0x7f14ec0ae810 2024-03-29T08:24:44.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.005+0000 7f14f3c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ec0c9bd0 con 0x7f14ec0ce8a0 2024-03-29T08:24:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.005+0000 7f14f3c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ec0cbcc0 con 0x7f14ec0caf50 2024-03-29T08:24:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.009+0000 7f14e3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f14ec0ae810 0x7f14ec0aecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37034/0 (socket says 172.21.15.46:37034) 2024-03-29T08:24:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.009+0000 7f14e3e00640 1 -- 172.21.15.46:0/1446792864 learned_addr learned my addr 172.21.15.46:0/1446792864 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.009+0000 7f14f1c00640 1 --2- 172.21.15.46:0/1446792864 >> v2:172.21.15.46:3300/0 conn(0x7f14ec0caf50 0x7f14ec0cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.009+0000 7f14f1c00640 1 -- 172.21.15.46:0/1446792864 >> v1:172.21.15.73:6789/0 conn(0x7f14ec0ce8a0 legacy=0x7f14ec0cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.009+0000 7f14f1c00640 1 -- 172.21.15.46:0/1446792864 >> v1:172.21.15.46:6789/0 conn(0x7f14ec0ae810 legacy=0x7f14ec0aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:44.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.009+0000 7f14f1c00640 1 -- 172.21.15.46:0/1446792864 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec0c9eb0 con 0x7f14ec0caf50 2024-03-29T08:24:44.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.009+0000 7f14f1c00640 1 --2- 172.21.15.46:0/1446792864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec0caf50 0x7f14ec0cb780 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009d90 tx=0x7f14e401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9402608e001af669 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.009+0000 7f14ebe00640 1 -- 172.21.15.46:0/1446792864 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e40a59c0 con 0x7f14ec0caf50 2024-03-29T08:24:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.009+0000 7f14ebe00640 1 -- 172.21.15.46:0/1446792864 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e40a5b20 con 0x7f14ec0caf50 2024-03-29T08:24:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.009+0000 7f14ebe00640 1 -- 172.21.15.46:0/1446792864 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e40ad440 con 0x7f14ec0caf50 2024-03-29T08:24:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f3c00640 1 -- 172.21.15.46:0/1446792864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec0caf50 msgr2=0x7f14ec0cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:44.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f3c00640 1 --2- 172.21.15.46:0/1446792864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec0caf50 0x7f14ec0cb780 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009d90 tx=0x7f14e401c900 comp rx=0 tx=0).stop 2024-03-29T08:24:44.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f3c00640 1 -- 172.21.15.46:0/1446792864 shutdown_connections 2024-03-29T08:24:44.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f3c00640 1 --2- 172.21.15.46:0/1446792864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec0caf50 0x7f14ec0cb780 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:44.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f3c00640 1 -- 172.21.15.46:0/1446792864 >> 172.21.15.46:0/1446792864 conn(0x7f14ec098a40 msgr2=0x7f14ec0cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:44.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f3c00640 1 -- 172.21.15.46:0/1446792864 shutdown_connections 2024-03-29T08:24:44.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f3c00640 1 -- 172.21.15.46:0/1446792864 wait complete. 2024-03-29T08:24:44.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f3c00640 1 Processor -- start 2024-03-29T08:24:44.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f3c00640 1 -- start start 2024-03-29T08:24:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f3c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec0ae810 0x7f14ec0c8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f3c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14ec0caf50 0x7f14ec1411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f3c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f14ec0ce8a0 0x7f14ec141700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f3c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec15e620 con 0x7f14ec0ae810 2024-03-29T08:24:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f3c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec0b3250 con 0x7f14ec0caf50 2024-03-29T08:24:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f3c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec0b3530 con 0x7f14ec0ce8a0 2024-03-29T08:24:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec0ae810 0x7f14ec0c8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:44.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec0ae810 0x7f14ec0c8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50846/0 (socket says 172.21.15.46:50846) 2024-03-29T08:24:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f1200640 1 -- 172.21.15.46:0/3870149800 learned_addr learned my addr 172.21.15.46:0/3870149800 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14e3e00640 1 --2- 172.21.15.46:0/3870149800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14ec0caf50 0x7f14ec1411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f1200640 1 -- 172.21.15.46:0/3870149800 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f14ec0ce8a0 msgr2=0x7f14ec141700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f1200640 1 --2- 172.21.15.46:0/3870149800 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f14ec0ce8a0 0x7f14ec141700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f1200640 1 -- 172.21.15.46:0/3870149800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14ec0caf50 msgr2=0x7f14ec1411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f1200640 1 --2- 172.21.15.46:0/3870149800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14ec0caf50 0x7f14ec1411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f1200640 1 -- 172.21.15.46:0/3870149800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e4009a70 con 0x7f14ec0ae810 2024-03-29T08:24:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14e3e00640 1 --2- 172.21.15.46:0/3870149800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14ec0caf50 0x7f14ec1411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:24:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14f1200640 1 --2- 172.21.15.46:0/3870149800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec0ae810 0x7f14ec0c8e10 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00b410 tx=0x7f14dc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.013+0000 7f14e9600640 1 -- 172.21.15.46:0/3870149800 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc004020 con 0x7f14ec0ae810 2024-03-29T08:24:44.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.017+0000 7f14e9600640 1 -- 172.21.15.46:0/3870149800 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc004180 con 0x7f14ec0ae810 2024-03-29T08:24:44.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.017+0000 7f14f3c00640 1 -- 172.21.15.46:0/3870149800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ec0b3810 con 0x7f14ec0ae810 2024-03-29T08:24:44.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.017+0000 7f14e9600640 1 -- 172.21.15.46:0/3870149800 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc011630 con 0x7f14ec0ae810 2024-03-29T08:24:44.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.017+0000 7f14f3c00640 1 -- 172.21.15.46:0/3870149800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ec0c9350 con 0x7f14ec0ae810 2024-03-29T08:24:44.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.017+0000 7f14f3c00640 1 -- 172.21.15.46:0/3870149800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ec0b1f70 con 0x7f14ec0ae810 2024-03-29T08:24:44.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.017+0000 7f14e9600640 1 -- 172.21.15.46:0/3870149800 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f14dc011790 con 0x7f14ec0ae810 2024-03-29T08:24:44.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.017+0000 7f14e9600640 1 --2- 172.21.15.46:0/3870149800 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f14c8066690 0x7f14c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:44.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.017+0000 7f14e9600640 1 -- 172.21.15.46:0/3870149800 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14dc083fa0 con 0x7f14ec0ae810 2024-03-29T08:24:44.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.017+0000 7f14e3e00640 1 --2- 172.21.15.46:0/3870149800 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f14c8066690 0x7f14c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:44.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.021+0000 7f14e3e00640 1 --2- 172.21.15.46:0/3870149800 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f14c8066690 0x7f14c8068b50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0cd5f0 tx=0x7f14d4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:44.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.021+0000 7f14e9600640 1 -- 172.21.15.46:0/3870149800 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f14dc051270 con 0x7f14ec0ae810 2024-03-29T08:24:44.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.185+0000 7f14f3c00640 1 -- 172.21.15.46:0/3870149800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14ec15e760 con 0x7f14ec0ae810 2024-03-29T08:24:44.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.189+0000 7f14e9600640 1 -- 172.21.15.46:0/3870149800 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16069 (secure 0 0 0) 0x7f14dc015090 con 0x7f14ec0ae810 2024-03-29T08:24:44.187 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:44.188 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":91,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:24:43.476025+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:24:41.113618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-29T08:24:42.487380+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:24:44.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.189+0000 7f14f3c00640 1 -- 172.21.15.46:0/3870149800 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f14c8066690 msgr2=0x7f14c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:44.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.189+0000 7f14f3c00640 1 --2- 172.21.15.46:0/3870149800 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f14c8066690 0x7f14c8068b50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0cd5f0 tx=0x7f14d4009040 comp rx=0 tx=0).stop 2024-03-29T08:24:44.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.189+0000 7f14f3c00640 1 -- 172.21.15.46:0/3870149800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec0ae810 msgr2=0x7f14ec0c8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.189+0000 7f14f3c00640 1 --2- 172.21.15.46:0/3870149800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec0ae810 0x7f14ec0c8e10 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00b410 tx=0x7f14dc00b8d0 comp rx=0 tx=0).stop 2024-03-29T08:24:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.189+0000 7f14f3c00640 1 -- 172.21.15.46:0/3870149800 shutdown_connections 2024-03-29T08:24:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.189+0000 7f14f3c00640 1 --2- 172.21.15.46:0/3870149800 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f14c8066690 0x7f14c8068b50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.189+0000 7f14f3c00640 1 --2- 172.21.15.46:0/3870149800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14ec0caf50 0x7f14ec1411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.189+0000 7f14f3c00640 1 --2- 172.21.15.46:0/3870149800 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f14ec0ce8a0 0x7f14ec141700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.189+0000 7f14f3c00640 1 --2- 172.21.15.46:0/3870149800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14ec0ae810 0x7f14ec0c8e10 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.189+0000 7f14f3c00640 1 -- 172.21.15.46:0/3870149800 >> 172.21.15.46:0/3870149800 conn(0x7f14ec098a40 msgr2=0x7f14ec0b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.189+0000 7f14f3c00640 1 -- 172.21.15.46:0/3870149800 shutdown_connections 2024-03-29T08:24:44.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.189+0000 7f14f3c00640 1 -- 172.21.15.46:0/3870149800 wait complete. 2024-03-29T08:24:44.209 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T08:24:44.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.353+0000 7f0e20c00640 1 Processor -- start 2024-03-29T08:24:44.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.353+0000 7f0e20c00640 1 -- start start 2024-03-29T08:24:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.353+0000 7f0e20c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e1c142a50 0x7f0e1c0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.353+0000 7f0e20c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e1c0ae990 0x7f0e1c0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.353+0000 7f0e20c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1c15d260 con 0x7f0e1c15df90 2024-03-29T08:24:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.353+0000 7f0e20c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e1c0aefd0 con 0x7f0e1c142a50 2024-03-29T08:24:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.353+0000 7f0e20c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c15d4b0 con 0x7f0e1c0ae990 2024-03-29T08:24:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.353+0000 7f0e19400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0e1c15df90 0x7f0e1c0ae280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37036/0 (socket says 172.21.15.46:37036) 2024-03-29T08:24:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.353+0000 7f0e19400640 1 -- 172.21.15.46:0/4070647222 learned_addr learned my addr 172.21.15.46:0/4070647222 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:44.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.353+0000 7f0e19e00640 1 --2- 172.21.15.46:0/4070647222 >> v2:172.21.15.73:3300/0 conn(0x7f0e1c142a50 0x7f0e1c0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.353+0000 7f0e1a800640 1 --2- 172.21.15.46:0/4070647222 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e1c0ae990 0x7f0e1c0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.353+0000 7f0e18a00640 1 -- 172.21.15.46:0/4070647222 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900716544 0 0) 0x7f0e1c15d260 con 0x7f0e1c15df90 2024-03-29T08:24:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.353+0000 7f0e18a00640 1 -- 172.21.15.46:0/4070647222 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e00003530 con 0x7f0e1c15df90 2024-03-29T08:24:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.353+0000 7f0e19e00640 1 -- 172.21.15.46:0/4070647222 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e1c0ae990 msgr2=0x7f0e1c0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.353+0000 7f0e19e00640 1 --2- 172.21.15.46:0/4070647222 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e1c0ae990 0x7f0e1c0cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.353+0000 7f0e19e00640 1 -- 172.21.15.46:0/4070647222 >> v1:172.21.15.46:6789/0 conn(0x7f0e1c15df90 legacy=0x7f0e1c0ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:44.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.353+0000 7f0e19e00640 1 -- 172.21.15.46:0/4070647222 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1c15d790 con 0x7f0e1c142a50 2024-03-29T08:24:44.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.357+0000 7f0e19e00640 1 --2- 172.21.15.46:0/4070647222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1c142a50 0x7f0e1c0cb790 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0097c0 tx=0x7f0e0c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc69842a952555f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:44.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.357+0000 7f0e18a00640 1 -- 172.21.15.46:0/4070647222 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c0ab600 con 0x7f0e1c142a50 2024-03-29T08:24:44.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.357+0000 7f0e18a00640 1 -- 172.21.15.46:0/4070647222 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0c0a7050 con 0x7f0e1c142a50 2024-03-29T08:24:44.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.357+0000 7f0e18a00640 1 -- 172.21.15.46:0/4070647222 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c0ab980 con 0x7f0e1c142a50 2024-03-29T08:24:44.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.357+0000 7f0e20c00640 1 -- 172.21.15.46:0/4070647222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1c142a50 msgr2=0x7f0e1c0cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:44.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.357+0000 7f0e20c00640 1 --2- 172.21.15.46:0/4070647222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1c142a50 0x7f0e1c0cb790 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0097c0 tx=0x7f0e0c01c410 comp rx=0 tx=0).stop 2024-03-29T08:24:44.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.357+0000 7f0e20c00640 1 -- 172.21.15.46:0/4070647222 shutdown_connections 2024-03-29T08:24:44.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.357+0000 7f0e20c00640 1 --2- 172.21.15.46:0/4070647222 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e1c0ae990 0x7f0e1c0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:44.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.357+0000 7f0e20c00640 1 --2- 172.21.15.46:0/4070647222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1c142a50 0x7f0e1c0cb790 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:44.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.357+0000 7f0e20c00640 1 -- 172.21.15.46:0/4070647222 >> 172.21.15.46:0/4070647222 conn(0x7f0e1c06a390 msgr2=0x7f0e1c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:44.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.357+0000 7f0e20c00640 1 -- 172.21.15.46:0/4070647222 shutdown_connections 2024-03-29T08:24:44.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.357+0000 7f0e20c00640 1 -- 172.21.15.46:0/4070647222 wait complete. 2024-03-29T08:24:44.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e20c00640 1 Processor -- start 2024-03-29T08:24:44.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e20c00640 1 -- start start 2024-03-29T08:24:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e20c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c0ae990 0x7f0e1c0cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e20c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1c142a50 0x7f0e1c13c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e20c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e1c15df90 0x7f0e1c13cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e20c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c1f8790 con 0x7f0e1c0ae990 2024-03-29T08:24:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e20c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c1f95b0 con 0x7f0e1c142a50 2024-03-29T08:24:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e20c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c1f89a0 con 0x7f0e1c15df90 2024-03-29T08:24:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e19e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c0ae990 0x7f0e1c0cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e19e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c0ae990 0x7f0e1c0cb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50858/0 (socket says 172.21.15.46:50858) 2024-03-29T08:24:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e19e00640 1 -- 172.21.15.46:0/3076033444 learned_addr learned my addr 172.21.15.46:0/3076033444 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e19400640 1 --2- 172.21.15.46:0/3076033444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1c142a50 0x7f0e1c13c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:44.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e1a800640 1 --2- 172.21.15.46:0/3076033444 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e1c15df90 0x7f0e1c13cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e19e00640 1 -- 172.21.15.46:0/3076033444 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e1c15df90 msgr2=0x7f0e1c13cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e19e00640 1 --2- 172.21.15.46:0/3076033444 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e1c15df90 0x7f0e1c13cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e19e00640 1 -- 172.21.15.46:0/3076033444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1c142a50 msgr2=0x7f0e1c13c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e19e00640 1 --2- 172.21.15.46:0/3076033444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1c142a50 0x7f0e1c13c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e19e00640 1 -- 172.21.15.46:0/3076033444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c009420 con 0x7f0e1c0ae990 2024-03-29T08:24:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e19e00640 1 --2- 172.21.15.46:0/3076033444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c0ae990 0x7f0e1c0cb2e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0097c0 tx=0x7f0e0c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e12000640 1 -- 172.21.15.46:0/3076033444 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c0b4840 con 0x7f0e1c0ae990 2024-03-29T08:24:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e12000640 1 -- 172.21.15.46:0/3076033444 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0c0b49c0 con 0x7f0e1c0ae990 2024-03-29T08:24:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e12000640 1 -- 172.21.15.46:0/3076033444 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c0b4d70 con 0x7f0e1c0ae990 2024-03-29T08:24:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e20c00640 1 -- 172.21.15.46:0/3076033444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1c1f8c80 con 0x7f0e1c0ae990 2024-03-29T08:24:44.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.361+0000 7f0e20c00640 1 -- 172.21.15.46:0/3076033444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1c1f9240 con 0x7f0e1c0ae990 2024-03-29T08:24:44.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.365+0000 7f0e20c00640 1 -- 172.21.15.46:0/3076033444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1c071a70 con 0x7f0e1c0ae990 2024-03-29T08:24:44.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.365+0000 7f0e12000640 1 -- 172.21.15.46:0/3076033444 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0e0c0b2040 con 0x7f0e1c0ae990 2024-03-29T08:24:44.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.365+0000 7f0e12000640 1 --2- 172.21.15.46:0/3076033444 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0dfc066690 0x7f0dfc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:44.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.365+0000 7f0e12000640 1 -- 172.21.15.46:0/3076033444 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e0c11e4e0 con 0x7f0e1c0ae990 2024-03-29T08:24:44.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.365+0000 7f0e19400640 1 --2- 172.21.15.46:0/3076033444 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0dfc066690 0x7f0dfc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:44.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.365+0000 7f0e19400640 1 --2- 172.21.15.46:0/3076033444 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0dfc066690 0x7f0dfc068b50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c13bf00 tx=0x7f0e08007d40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.369+0000 7f0e12000640 1 -- 172.21.15.46:0/3076033444 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0e0c0eb7d0 con 0x7f0e1c0ae990 2024-03-29T08:24:44.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:44.565+0000 7f0e20c00640 1 -- 172.21.15.46:0/3076033444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0e1c04e120 con 0x7f0e1c0ae990 2024-03-29T08:24:45.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.493+0000 7f0e12000640 1 -- 172.21.15.46:0/3076033444 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v56) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0e0c0b0020 con 0x7f0e1c0ae990 2024-03-29T08:24:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.493+0000 7f0e20c00640 1 -- 172.21.15.46:0/3076033444 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0dfc066690 msgr2=0x7f0dfc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.493+0000 7f0e20c00640 1 --2- 172.21.15.46:0/3076033444 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0dfc066690 0x7f0dfc068b50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c13bf00 tx=0x7f0e08007d40 comp rx=0 tx=0).stop 2024-03-29T08:24:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.493+0000 7f0e20c00640 1 -- 172.21.15.46:0/3076033444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c0ae990 msgr2=0x7f0e1c0cb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.493+0000 7f0e20c00640 1 --2- 172.21.15.46:0/3076033444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c0ae990 0x7f0e1c0cb2e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0097c0 tx=0x7f0e0c0191b0 comp rx=0 tx=0).stop 2024-03-29T08:24:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.493+0000 7f0e20c00640 1 -- 172.21.15.46:0/3076033444 shutdown_connections 2024-03-29T08:24:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.493+0000 7f0e20c00640 1 --2- 172.21.15.46:0/3076033444 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0dfc066690 0x7f0dfc068b50 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.493+0000 7f0e20c00640 1 --2- 172.21.15.46:0/3076033444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1c142a50 0x7f0e1c13c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.493+0000 7f0e20c00640 1 --2- 172.21.15.46:0/3076033444 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e1c15df90 0x7f0e1c13cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.493+0000 7f0e20c00640 1 --2- 172.21.15.46:0/3076033444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e1c0ae990 0x7f0e1c0cb2e0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.493+0000 7f0e20c00640 1 -- 172.21.15.46:0/3076033444 >> 172.21.15.46:0/3076033444 conn(0x7f0e1c06a390 msgr2=0x7f0e1c0cde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.497+0000 7f0e20c00640 1 -- 172.21.15.46:0/3076033444 shutdown_connections 2024-03-29T08:24:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.497+0000 7f0e20c00640 1 -- 172.21.15.46:0/3076033444 wait complete. 2024-03-29T08:24:45.520 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-29T08:24:45.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T08:24:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.673+0000 7fc2cd000640 1 Processor -- start 2024-03-29T08:24:45.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.673+0000 7fc2cd000640 1 -- start start 2024-03-29T08:24:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.673+0000 7fc2cd000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc2c815cad0 0x7fc2c806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.673+0000 7fc2cd000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2c806a4a0 0x7fc2c806d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.673+0000 7fc2cd000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c815d080 con 0x7fc2c806eff0 2024-03-29T08:24:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.673+0000 7fc2cd000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c815dd00 con 0x7fc2c815cad0 2024-03-29T08:24:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.673+0000 7fc2cd000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c806f480 con 0x7fc2c806a4a0 2024-03-29T08:24:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.673+0000 7fc2c5e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc2c815cad0 0x7fc2c806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.673+0000 7fc2c5e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc2c815cad0 0x7fc2c806eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50866/0 (socket says 172.21.15.46:50866) 2024-03-29T08:24:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.673+0000 7fc2c5e00640 1 -- 172.21.15.46:0/3867109287 learned_addr learned my addr 172.21.15.46:0/3867109287 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.673+0000 7fc2c6800640 1 --2- 172.21.15.46:0/3867109287 >> v2:172.21.15.73:3300/0 conn(0x7fc2c806a4a0 0x7fc2c806d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.673+0000 7fc2c5e00640 1 -- 172.21.15.46:0/3867109287 >> v2:172.21.15.73:3300/0 conn(0x7fc2c806a4a0 msgr2=0x7fc2c806d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.673+0000 7fc2c5e00640 1 --2- 172.21.15.46:0/3867109287 >> v2:172.21.15.73:3300/0 conn(0x7fc2c806a4a0 0x7fc2c806d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.673+0000 7fc2c5e00640 1 -- 172.21.15.46:0/3867109287 >> v1:172.21.15.73:6789/0 conn(0x7fc2c806eff0 legacy=0x7fc2c8069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.673+0000 7fc2c5e00640 1 -- 172.21.15.46:0/3867109287 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c815d360 con 0x7fc2c815cad0 2024-03-29T08:24:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.673+0000 7fc2c6800640 1 --2- 172.21.15.46:0/3867109287 >> v2:172.21.15.73:3300/0 conn(0x7fc2c806a4a0 0x7fc2c806d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.677+0000 7fc2c5e00640 1 --2- 172.21.15.46:0/3867109287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c815cad0 0x7fc2c806eab0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8098fe0 tx=0x7fc2b801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deca3c57e359ee74 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.677+0000 7fc2c5400640 1 -- 172.21.15.46:0/3867109287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b80ad660 con 0x7fc2c815cad0 2024-03-29T08:24:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.677+0000 7fc2c5400640 1 -- 172.21.15.46:0/3867109287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b80ad7c0 con 0x7fc2c815cad0 2024-03-29T08:24:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.677+0000 7fc2c5400640 1 -- 172.21.15.46:0/3867109287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b80adb50 con 0x7fc2c815cad0 2024-03-29T08:24:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.677+0000 7fc2cd000640 1 -- 172.21.15.46:0/3867109287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c815cad0 msgr2=0x7fc2c806eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.677+0000 7fc2cd000640 1 --2- 172.21.15.46:0/3867109287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c815cad0 0x7fc2c806eab0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8098fe0 tx=0x7fc2b801c450 comp rx=0 tx=0).stop 2024-03-29T08:24:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.677+0000 7fc2cd000640 1 -- 172.21.15.46:0/3867109287 shutdown_connections 2024-03-29T08:24:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.677+0000 7fc2cd000640 1 --2- 172.21.15.46:0/3867109287 >> v2:172.21.15.73:3300/0 conn(0x7fc2c806a4a0 0x7fc2c806d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.677+0000 7fc2cd000640 1 --2- 172.21.15.46:0/3867109287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c815cad0 0x7fc2c806eab0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.677+0000 7fc2cd000640 1 -- 172.21.15.46:0/3867109287 >> 172.21.15.46:0/3867109287 conn(0x7fc2c8098a40 msgr2=0x7fc2c80b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.677+0000 7fc2cd000640 1 -- 172.21.15.46:0/3867109287 shutdown_connections 2024-03-29T08:24:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.677+0000 7fc2cd000640 1 -- 172.21.15.46:0/3867109287 wait complete. 2024-03-29T08:24:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.677+0000 7fc2cd000640 1 Processor -- start 2024-03-29T08:24:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.677+0000 7fc2cd000640 1 -- start start 2024-03-29T08:24:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2cd000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c806a4a0 0x7fc2c8151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2cd000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2c806eff0 0x7fc2c80ac380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2cd000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2c815cad0 0x7fc2c80ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2cd000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c8151ad0 con 0x7fc2c806a4a0 2024-03-29T08:24:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2cd000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c81575f0 con 0x7fc2c815cad0 2024-03-29T08:24:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2cd000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c81578d0 con 0x7fc2c806eff0 2024-03-29T08:24:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2c5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c806a4a0 0x7fc2c8151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2c5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c806a4a0 0x7fc2c8151590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50878/0 (socket says 172.21.15.46:50878) 2024-03-29T08:24:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2c5e00640 1 -- 172.21.15.46:0/977924703 learned_addr learned my addr 172.21.15.46:0/977924703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2c5e00640 1 -- 172.21.15.46:0/977924703 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2c806eff0 msgr2=0x7fc2c80ac380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2c5e00640 1 --2- 172.21.15.46:0/977924703 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2c806eff0 0x7fc2c80ac380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2c5e00640 1 -- 172.21.15.46:0/977924703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2c815cad0 msgr2=0x7fc2c80ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2c6800640 1 --2- 172.21.15.46:0/977924703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2c815cad0 0x7fc2c80ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2c5e00640 1 --2- 172.21.15.46:0/977924703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2c815cad0 0x7fc2c80ad1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2c5e00640 1 -- 172.21.15.46:0/977924703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b80094d0 con 0x7fc2c806a4a0 2024-03-29T08:24:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2c5e00640 1 --2- 172.21.15.46:0/977924703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c806a4a0 0x7fc2c8151590 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc2b801a080 tx=0x7fc2b80191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2be000640 1 -- 172.21.15.46:0/977924703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b80b6640 con 0x7fc2c806a4a0 2024-03-29T08:24:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2be000640 1 -- 172.21.15.46:0/977924703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b80b67a0 con 0x7fc2c806a4a0 2024-03-29T08:24:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2be000640 1 -- 172.21.15.46:0/977924703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b80b6b50 con 0x7fc2c806a4a0 2024-03-29T08:24:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2cd000640 1 -- 172.21.15.46:0/977924703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c8157bb0 con 0x7fc2c806a4a0 2024-03-29T08:24:45.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2cd000640 1 -- 172.21.15.46:0/977924703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c8158170 con 0x7fc2c806a4a0 2024-03-29T08:24:45.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.681+0000 7fc2cd000640 1 -- 172.21.15.46:0/977924703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c80cd8f0 con 0x7fc2c806a4a0 2024-03-29T08:24:45.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.685+0000 7fc2be000640 1 -- 172.21.15.46:0/977924703 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc2b80b4070 con 0x7fc2c806a4a0 2024-03-29T08:24:45.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.685+0000 7fc2be000640 1 --2- 172.21.15.46:0/977924703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc2a4066690 0x7fc2a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:45.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.685+0000 7fc2be000640 1 -- 172.21.15.46:0/977924703 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2b811e2d0 con 0x7fc2c806a4a0 2024-03-29T08:24:45.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.685+0000 7fc2bfe00640 1 --2- 172.21.15.46:0/977924703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc2a4066690 0x7fc2a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:45.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.685+0000 7fc2bfe00640 1 --2- 172.21.15.46:0/977924703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc2a4066690 0x7fc2a4068b50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc2c815e9e0 tx=0x7fc2b00079c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:45.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.689+0000 7fc2be000640 1 -- 172.21.15.46:0/977924703 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc2b80eb750 con 0x7fc2c806a4a0 2024-03-29T08:24:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:45.885+0000 7fc2cd000640 1 -- 172.21.15.46:0/977924703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc2c80b1b50 con 0x7fc2c806a4a0 2024-03-29T08:24:46.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.493+0000 7fc2be000640 1 -- 172.21.15.46:0/977924703 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v57) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc2b80eed70 con 0x7fc2c806a4a0 2024-03-29T08:24:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.497+0000 7fc2cd000640 1 -- 172.21.15.46:0/977924703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc2a4066690 msgr2=0x7fc2a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.497+0000 7fc2cd000640 1 --2- 172.21.15.46:0/977924703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc2a4066690 0x7fc2a4068b50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc2c815e9e0 tx=0x7fc2b00079c0 comp rx=0 tx=0).stop 2024-03-29T08:24:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.497+0000 7fc2cd000640 1 -- 172.21.15.46:0/977924703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c806a4a0 msgr2=0x7fc2c8151590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.497+0000 7fc2cd000640 1 --2- 172.21.15.46:0/977924703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c806a4a0 0x7fc2c8151590 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc2b801a080 tx=0x7fc2b80191f0 comp rx=0 tx=0).stop 2024-03-29T08:24:46.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.497+0000 7fc2cd000640 1 -- 172.21.15.46:0/977924703 shutdown_connections 2024-03-29T08:24:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.497+0000 7fc2cd000640 1 --2- 172.21.15.46:0/977924703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc2a4066690 0x7fc2a4068b50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.497+0000 7fc2cd000640 1 --2- 172.21.15.46:0/977924703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2c815cad0 0x7fc2c80ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.497+0000 7fc2cd000640 1 --2- 172.21.15.46:0/977924703 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc2c806eff0 0x7fc2c80ac380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.497+0000 7fc2cd000640 1 --2- 172.21.15.46:0/977924703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2c806a4a0 0x7fc2c8151590 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.497+0000 7fc2cd000640 1 -- 172.21.15.46:0/977924703 >> 172.21.15.46:0/977924703 conn(0x7fc2c8098a40 msgr2=0x7fc2c81606a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.497+0000 7fc2cd000640 1 -- 172.21.15.46:0/977924703 shutdown_connections 2024-03-29T08:24:46.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.497+0000 7fc2cd000640 1 -- 172.21.15.46:0/977924703 wait complete. 2024-03-29T08:24:46.518 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T08:24:46.518 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:24:46.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.661+0000 7f5c85200640 1 Processor -- start 2024-03-29T08:24:46.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.661+0000 7f5c85200640 1 -- start start 2024-03-29T08:24:46.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.661+0000 7f5c85200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c80153d50 0x7f5c801611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:46.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.661+0000 7f5c85200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5c800ad3e0 0x7f5c800ad850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:46.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.661+0000 7f5c85200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8015d870 con 0x7f5c80153970 2024-03-29T08:24:46.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.661+0000 7f5c85200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c800add90 con 0x7f5c80153d50 2024-03-29T08:24:46.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.661+0000 7f5c85200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c8015db50 con 0x7f5c800ad3e0 2024-03-29T08:24:46.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.661+0000 7f5c7de00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5c80153970 0x7f5c8015cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42056/0 (socket says 172.21.15.46:42056) 2024-03-29T08:24:46.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.665+0000 7f5c7de00640 1 -- 172.21.15.46:0/2584707862 learned_addr learned my addr 172.21.15.46:0/2584707862 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:46.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.665+0000 7f5c77e00640 1 --2- 172.21.15.46:0/2584707862 >> v2:172.21.15.46:3300/0 conn(0x7f5c80153d50 0x7f5c801611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:46.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.665+0000 7f5c7e800640 1 --2- 172.21.15.46:0/2584707862 >> v2:172.21.15.73:3300/0 conn(0x7f5c800ad3e0 0x7f5c800ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:46.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.665+0000 7f5c7d400640 1 -- 172.21.15.46:0/2584707862 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518659044 0 0) 0x7f5c8015d870 con 0x7f5c80153970 2024-03-29T08:24:46.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.665+0000 7f5c7d400640 1 -- 172.21.15.46:0/2584707862 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c64003530 con 0x7f5c80153970 2024-03-29T08:24:46.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.665+0000 7f5c77e00640 1 -- 172.21.15.46:0/2584707862 >> v2:172.21.15.73:3300/0 conn(0x7f5c800ad3e0 msgr2=0x7f5c800ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:46.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.665+0000 7f5c77e00640 1 --2- 172.21.15.46:0/2584707862 >> v2:172.21.15.73:3300/0 conn(0x7f5c800ad3e0 0x7f5c800ad850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:46.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.665+0000 7f5c77e00640 1 -- 172.21.15.46:0/2584707862 >> v1:172.21.15.46:6789/0 conn(0x7f5c80153970 legacy=0x7f5c8015cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:46.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.665+0000 7f5c77e00640 1 -- 172.21.15.46:0/2584707862 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8015de30 con 0x7f5c80153d50 2024-03-29T08:24:46.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.665+0000 7f5c77e00640 1 --2- 172.21.15.46:0/2584707862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80153d50 0x7f5c801611e0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f5c800b3720 tx=0x7f5c6801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafc63d8541006c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:46.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.665+0000 7f5c7d400640 1 -- 172.21.15.46:0/2584707862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c680ab5f0 con 0x7f5c80153d50 2024-03-29T08:24:46.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.665+0000 7f5c7d400640 1 -- 172.21.15.46:0/2584707862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c680ab750 con 0x7f5c80153d50 2024-03-29T08:24:46.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.665+0000 7f5c7d400640 1 -- 172.21.15.46:0/2584707862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c680abae0 con 0x7f5c80153d50 2024-03-29T08:24:46.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c85200640 1 -- 172.21.15.46:0/2584707862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80153d50 msgr2=0x7f5c801611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c85200640 1 --2- 172.21.15.46:0/2584707862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80153d50 0x7f5c801611e0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f5c800b3720 tx=0x7f5c6801c450 comp rx=0 tx=0).stop 2024-03-29T08:24:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c85200640 1 -- 172.21.15.46:0/2584707862 shutdown_connections 2024-03-29T08:24:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c85200640 1 --2- 172.21.15.46:0/2584707862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c80153d50 0x7f5c801611e0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c85200640 1 --2- 172.21.15.46:0/2584707862 >> v2:172.21.15.73:3300/0 conn(0x7f5c800ad3e0 0x7f5c800ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c85200640 1 -- 172.21.15.46:0/2584707862 >> 172.21.15.46:0/2584707862 conn(0x7f5c80066110 msgr2=0x7f5c80069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c85200640 1 -- 172.21.15.46:0/2584707862 shutdown_connections 2024-03-29T08:24:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c85200640 1 -- 172.21.15.46:0/2584707862 wait complete. 2024-03-29T08:24:46.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c85200640 1 Processor -- start 2024-03-29T08:24:46.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c85200640 1 -- start start 2024-03-29T08:24:46.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c85200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c800ad3e0 0x7f5c80152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:46.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c85200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c80153970 0x7f5c80145270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:46.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c85200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8014fb90 0x7f5c8014a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:46.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c85200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c80155f00 con 0x7f5c8014fb90 2024-03-29T08:24:46.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c85200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c801457b0 con 0x7f5c80153970 2024-03-29T08:24:46.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c85200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c80155330 con 0x7f5c800ad3e0 2024-03-29T08:24:46.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c7e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8014fb90 0x7f5c8014a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:46.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c7e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8014fb90 0x7f5c8014a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34644/0 (socket says 172.21.15.46:34644) 2024-03-29T08:24:46.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c7e800640 1 -- 172.21.15.46:0/3643233930 learned_addr learned my addr 172.21.15.46:0/3643233930 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:46.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c77e00640 1 --2- 172.21.15.46:0/3643233930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c80153970 0x7f5c80145270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:46.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c7de00640 1 --2- 172.21.15.46:0/3643233930 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c800ad3e0 0x7f5c80152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:46.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c7e800640 1 -- 172.21.15.46:0/3643233930 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c800ad3e0 msgr2=0x7f5c80152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:46.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c7e800640 1 --2- 172.21.15.46:0/3643233930 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c800ad3e0 0x7f5c80152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:46.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c7e800640 1 -- 172.21.15.46:0/3643233930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c80153970 msgr2=0x7f5c80145270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:46.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c7e800640 1 --2- 172.21.15.46:0/3643233930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c80153970 0x7f5c80145270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:46.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c7e800640 1 -- 172.21.15.46:0/3643233930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68009420 con 0x7f5c8014fb90 2024-03-29T08:24:46.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c7e800640 1 --2- 172.21.15.46:0/3643233930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8014fb90 0x7f5c8014a0e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f5c78007e40 tx=0x7f5c7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:46.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c76000640 1 -- 172.21.15.46:0/3643233930 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c78014620 con 0x7f5c8014fb90 2024-03-29T08:24:46.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c76000640 1 -- 172.21.15.46:0/3643233930 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7800ee90 con 0x7f5c8014fb90 2024-03-29T08:24:46.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.669+0000 7f5c76000640 1 -- 172.21.15.46:0/3643233930 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c780149e0 con 0x7f5c8014fb90 2024-03-29T08:24:46.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.673+0000 7f5c85200640 1 -- 172.21.15.46:0/3643233930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8014b9f0 con 0x7f5c8014fb90 2024-03-29T08:24:46.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.673+0000 7f5c85200640 1 -- 172.21.15.46:0/3643233930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c801558b0 con 0x7f5c8014fb90 2024-03-29T08:24:46.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.673+0000 7f5c85200640 1 -- 172.21.15.46:0/3643233930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c80066c90 con 0x7f5c8014fb90 2024-03-29T08:24:46.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.673+0000 7f5c76000640 1 -- 172.21.15.46:0/3643233930 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5c78014b40 con 0x7f5c8014fb90 2024-03-29T08:24:46.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.673+0000 7f5c76000640 1 --2- 172.21.15.46:0/3643233930 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c5c066690 0x7f5c5c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:46.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.673+0000 7f5c76000640 1 -- 172.21.15.46:0/3643233930 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c78084a10 con 0x7f5c8014fb90 2024-03-29T08:24:46.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.673+0000 7f5c7de00640 1 --2- 172.21.15.46:0/3643233930 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c5c066690 0x7f5c5c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:46.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.673+0000 7f5c7de00640 1 --2- 172.21.15.46:0/3643233930 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c5c066690 0x7f5c5c068b50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5c700011f0 tx=0x7f5c70009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:46.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.677+0000 7f5c76000640 1 -- 172.21.15.46:0/3643233930 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5c78051e10 con 0x7f5c8014fb90 2024-03-29T08:24:46.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.877+0000 7f5c85200640 1 -- 172.21.15.46:0/3643233930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c800ad8e0 con 0x7f5c8014fb90 2024-03-29T08:24:46.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.877+0000 7f5c76000640 1 -- 172.21.15.46:0/3643233930 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f5c78055430 con 0x7f5c8014fb90 2024-03-29T08:24:46.878 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:46.878 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7386,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2051429508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2051429508},{"type":"v1","addr":"172.21.15.46:6835","nonce":2051429508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:24:43.476334+0000","modified":"2024-03-29T08:24:46.492649+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7395":{"gid":7395,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/4166845858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4166845858},{"type":"v1","addr":"172.21.15.73:6836","nonce":4166845858}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-29T08:24:46.878 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 57 2024-03-29T08:24:46.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.881+0000 7f5c85200640 1 -- 172.21.15.46:0/3643233930 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c5c066690 msgr2=0x7f5c5c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:46.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.881+0000 7f5c85200640 1 --2- 172.21.15.46:0/3643233930 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c5c066690 0x7f5c5c068b50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5c700011f0 tx=0x7f5c70009040 comp rx=0 tx=0).stop 2024-03-29T08:24:46.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.881+0000 7f5c85200640 1 -- 172.21.15.46:0/3643233930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8014fb90 msgr2=0x7f5c8014a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:46.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.881+0000 7f5c85200640 1 --2- 172.21.15.46:0/3643233930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8014fb90 0x7f5c8014a0e0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f5c78007e40 tx=0x7f5c7800e4b0 comp rx=0 tx=0).stop 2024-03-29T08:24:46.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.881+0000 7f5c85200640 1 -- 172.21.15.46:0/3643233930 shutdown_connections 2024-03-29T08:24:46.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.881+0000 7f5c85200640 1 --2- 172.21.15.46:0/3643233930 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c5c066690 0x7f5c5c068b50 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:46.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.881+0000 7f5c85200640 1 --2- 172.21.15.46:0/3643233930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c80153970 0x7f5c80145270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:46.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.881+0000 7f5c85200640 1 --2- 172.21.15.46:0/3643233930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c8014fb90 0x7f5c8014a0e0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:46.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.881+0000 7f5c85200640 1 --2- 172.21.15.46:0/3643233930 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c800ad3e0 0x7f5c80152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:46.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.881+0000 7f5c85200640 1 -- 172.21.15.46:0/3643233930 >> 172.21.15.46:0/3643233930 conn(0x7f5c80066110 msgr2=0x7f5c8006d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:46.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.881+0000 7f5c85200640 1 -- 172.21.15.46:0/3643233930 shutdown_connections 2024-03-29T08:24:46.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:46.881+0000 7f5c85200640 1 -- 172.21.15.46:0/3643233930 wait complete. 2024-03-29T08:24:46.899 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:24:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.041+0000 7ff40d000640 1 Processor -- start 2024-03-29T08:24:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.041+0000 7ff40d000640 1 -- start start 2024-03-29T08:24:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.041+0000 7ff40d000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff4080d0660 0x7ff4080d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.041+0000 7ff40d000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff40815cfb0 con 0x7ff40815dc00 2024-03-29T08:24:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.041+0000 7ff40d000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4080d0f80 con 0x7ff40806b8b0 2024-03-29T08:24:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.041+0000 7ff40d000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff40806c100 con 0x7ff4080d0660 2024-03-29T08:24:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.041+0000 7ff406800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff40815dc00 0x7ff4080cf7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42068/0 (socket says 172.21.15.46:42068) 2024-03-29T08:24:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.041+0000 7ff406800640 1 -- 172.21.15.46:0/1312401077 learned_addr learned my addr 172.21.15.46:0/1312401077 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.041+0000 7ff405e00640 1 --2- 172.21.15.46:0/1312401077 >> v2:172.21.15.73:3300/0 conn(0x7ff4080d0660 0x7ff4080d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.041+0000 7ff405400640 1 -- 172.21.15.46:0/1312401077 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122695155 0 0) 0x7ff40815cfb0 con 0x7ff40815dc00 2024-03-29T08:24:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.041+0000 7ff405400640 1 -- 172.21.15.46:0/1312401077 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3ec003530 con 0x7ff40815dc00 2024-03-29T08:24:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.041+0000 7ff405e00640 1 -- 172.21.15.46:0/1312401077 >> v1:172.21.15.73:6789/0 conn(0x7ff40806b8b0 legacy=0x7ff4080cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.041+0000 7ff405400640 1 -- 172.21.15.46:0/1312401077 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1784659175 0 0) 0x7ff4080d0f80 con 0x7ff40806b8b0 2024-03-29T08:24:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.041+0000 7ff405e00640 1 -- 172.21.15.46:0/1312401077 >> v1:172.21.15.46:6789/0 conn(0x7ff40815dc00 legacy=0x7ff4080cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.041+0000 7ff405e00640 1 -- 172.21.15.46:0/1312401077 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40806be20 con 0x7ff4080d0660 2024-03-29T08:24:47.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.045+0000 7ff405e00640 1 --2- 172.21.15.46:0/1312401077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4080d0660 0x7ff4080d0a40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80097c0 tx=0x7ff3f801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c8dd1e9536988f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:47.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.045+0000 7ff405400640 1 -- 172.21.15.46:0/1312401077 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f80ab670 con 0x7ff4080d0660 2024-03-29T08:24:47.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.045+0000 7ff405400640 1 -- 172.21.15.46:0/1312401077 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f80ab7d0 con 0x7ff4080d0660 2024-03-29T08:24:47.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.045+0000 7ff405400640 1 -- 172.21.15.46:0/1312401077 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f80abb40 con 0x7ff4080d0660 2024-03-29T08:24:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.045+0000 7ff40d000640 1 -- 172.21.15.46:0/1312401077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4080d0660 msgr2=0x7ff4080d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.045+0000 7ff40d000640 1 --2- 172.21.15.46:0/1312401077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4080d0660 0x7ff4080d0a40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80097c0 tx=0x7ff3f801c470 comp rx=0 tx=0).stop 2024-03-29T08:24:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.045+0000 7ff40d000640 1 -- 172.21.15.46:0/1312401077 shutdown_connections 2024-03-29T08:24:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.045+0000 7ff40d000640 1 --2- 172.21.15.46:0/1312401077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4080d0660 0x7ff4080d0a40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.045+0000 7ff40d000640 1 -- 172.21.15.46:0/1312401077 >> 172.21.15.46:0/1312401077 conn(0x7ff408089fd0 msgr2=0x7ff40808a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.045+0000 7ff40d000640 1 -- 172.21.15.46:0/1312401077 shutdown_connections 2024-03-29T08:24:47.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.045+0000 7ff40d000640 1 -- 172.21.15.46:0/1312401077 wait complete. 2024-03-29T08:24:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff40d000640 1 Processor -- start 2024-03-29T08:24:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff40d000640 1 -- start start 2024-03-29T08:24:47.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff40d000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff40806b8b0 0x7ff4081517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff40d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4080d0660 0x7ff4080b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff40d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff40815dc00 0x7ff4080ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff40d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff408151cf0 con 0x7ff4080d0660 2024-03-29T08:24:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff40d000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4081528b0 con 0x7ff40815dc00 2024-03-29T08:24:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff40d000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff408152b00 con 0x7ff40806b8b0 2024-03-29T08:24:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff3fd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4080d0660 0x7ff4080b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff3fd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4080d0660 0x7ff4080b0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34646/0 (socket says 172.21.15.46:34646) 2024-03-29T08:24:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff3fd400640 1 -- 172.21.15.46:0/1808982695 learned_addr learned my addr 172.21.15.46:0/1808982695 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff406800640 1 --2- 172.21.15.46:0/1808982695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff40815dc00 0x7ff4080ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff405e00640 1 --2- 172.21.15.46:0/1808982695 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff40806b8b0 0x7ff4081517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff3fd400640 1 -- 172.21.15.46:0/1808982695 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff40806b8b0 msgr2=0x7ff4081517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff3fd400640 1 --2- 172.21.15.46:0/1808982695 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff40806b8b0 0x7ff4081517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff3fd400640 1 -- 172.21.15.46:0/1808982695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff40815dc00 msgr2=0x7ff4080ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff3fd400640 1 --2- 172.21.15.46:0/1808982695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff40815dc00 0x7ff4080ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff3fd400640 1 -- 172.21.15.46:0/1808982695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f8009420 con 0x7ff4080d0660 2024-03-29T08:24:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff3fd400640 1 --2- 172.21.15.46:0/1808982695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4080d0660 0x7ff4080b0a40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000c500 tx=0x7ff3f000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff3fea00640 1 -- 172.21.15.46:0/1808982695 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0003600 con 0x7ff4080d0660 2024-03-29T08:24:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff3fea00640 1 -- 172.21.15.46:0/1808982695 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f0003760 con 0x7ff4080d0660 2024-03-29T08:24:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff3fea00640 1 -- 172.21.15.46:0/1808982695 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f0003b10 con 0x7ff4080d0660 2024-03-29T08:24:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff40d000640 1 -- 172.21.15.46:0/1808982695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff408152de0 con 0x7ff4080d0660 2024-03-29T08:24:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff40d000640 1 -- 172.21.15.46:0/1808982695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4081533a0 con 0x7ff4080d0660 2024-03-29T08:24:47.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.049+0000 7ff40d000640 1 -- 172.21.15.46:0/1808982695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4081535c0 con 0x7ff4080d0660 2024-03-29T08:24:47.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.053+0000 7ff3fea00640 1 -- 172.21.15.46:0/1808982695 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff3f0003c70 con 0x7ff4080d0660 2024-03-29T08:24:47.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.053+0000 7ff3fea00640 1 --2- 172.21.15.46:0/1808982695 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3e4066690 0x7ff3e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.053+0000 7ff3fea00640 1 -- 172.21.15.46:0/1808982695 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff3f00837a0 con 0x7ff4080d0660 2024-03-29T08:24:47.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.053+0000 7ff405e00640 1 --2- 172.21.15.46:0/1808982695 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3e4066690 0x7ff3e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.053+0000 7ff405e00640 1 --2- 172.21.15.46:0/1808982695 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3e4066690 0x7ff3e4068b50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80af000 tx=0x7ff3f80a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:47.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.057+0000 7ff3fea00640 1 -- 172.21.15.46:0/1808982695 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff3f0050bf0 con 0x7ff4080d0660 2024-03-29T08:24:47.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.221+0000 7ff40d000640 1 -- 172.21.15.46:0/1808982695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff408066c90 con 0x7ff4080d0660 2024-03-29T08:24:47.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.221+0000 7ff3fea00640 1 -- 172.21.15.46:0/1808982695 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16056 (secure 0 0 0) 0x7ff3f0017090 con 0x7ff4080d0660 2024-03-29T08:24:47.221 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:47.221 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":92,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:24:44.478513+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:24:41.113618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-29T08:24:42.487380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:24:47.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.221+0000 7ff40d000640 1 -- 172.21.15.46:0/1808982695 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3e4066690 msgr2=0x7ff3e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.221+0000 7ff40d000640 1 --2- 172.21.15.46:0/1808982695 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3e4066690 0x7ff3e4068b50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80af000 tx=0x7ff3f80a7000 comp rx=0 tx=0).stop 2024-03-29T08:24:47.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.221+0000 7ff40d000640 1 -- 172.21.15.46:0/1808982695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4080d0660 msgr2=0x7ff4080b0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.221+0000 7ff40d000640 1 --2- 172.21.15.46:0/1808982695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4080d0660 0x7ff4080b0a40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ff3f000c500 tx=0x7ff3f000c9c0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.221+0000 7ff40d000640 1 -- 172.21.15.46:0/1808982695 shutdown_connections 2024-03-29T08:24:47.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.221+0000 7ff40d000640 1 --2- 172.21.15.46:0/1808982695 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3e4066690 0x7ff3e4068b50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.221+0000 7ff40d000640 1 --2- 172.21.15.46:0/1808982695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff40815dc00 0x7ff4080ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.221+0000 7ff40d000640 1 --2- 172.21.15.46:0/1808982695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4080d0660 0x7ff4080b0a40 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.221+0000 7ff40d000640 1 --2- 172.21.15.46:0/1808982695 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff40806b8b0 0x7ff4081517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.221+0000 7ff40d000640 1 -- 172.21.15.46:0/1808982695 >> 172.21.15.46:0/1808982695 conn(0x7ff408089fd0 msgr2=0x7ff408155b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:47.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.225+0000 7ff40d000640 1 -- 172.21.15.46:0/1808982695 shutdown_connections 2024-03-29T08:24:47.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.225+0000 7ff40d000640 1 -- 172.21.15.46:0/1808982695 wait complete. 2024-03-29T08:24:47.242 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T08:24:47.243 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:24:47.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.381+0000 7fa4f7200640 1 Processor -- start 2024-03-29T08:24:47.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.381+0000 7fa4f7200640 1 -- start start 2024-03-29T08:24:47.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.385+0000 7fa4f7200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4f015c9d0 0x7fa4f00cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.385+0000 7fa4f7200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa4f0099150 0x7fa4f0166a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.385+0000 7fa4f7200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f00d0030 con 0x7fa4f00cef60 2024-03-29T08:24:47.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.385+0000 7fa4f7200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f00cf420 con 0x7fa4f015c9d0 2024-03-29T08:24:47.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.385+0000 7fa4f7200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f00cf700 con 0x7fa4f0099150 2024-03-29T08:24:47.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.385+0000 7fa4efe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4f015c9d0 0x7fa4f00cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.385+0000 7fa4efe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4f015c9d0 0x7fa4f00cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34656/0 (socket says 172.21.15.46:34656) 2024-03-29T08:24:47.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.385+0000 7fa4efe00640 1 -- 172.21.15.46:0/1919163116 learned_addr learned my addr 172.21.15.46:0/1919163116 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.385+0000 7fa4f5200640 1 --2- 172.21.15.46:0/1919163116 >> v2:172.21.15.73:3300/0 conn(0x7fa4f0099150 0x7fa4f0166a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.385+0000 7fa4efe00640 1 -- 172.21.15.46:0/1919163116 >> v2:172.21.15.73:3300/0 conn(0x7fa4f0099150 msgr2=0x7fa4f0166a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.385+0000 7fa4efe00640 1 --2- 172.21.15.46:0/1919163116 >> v2:172.21.15.73:3300/0 conn(0x7fa4f0099150 0x7fa4f0166a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.385+0000 7fa4efe00640 1 -- 172.21.15.46:0/1919163116 >> v1:172.21.15.73:6789/0 conn(0x7fa4f00cef60 legacy=0x7fa4f0098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.385+0000 7fa4efe00640 1 -- 172.21.15.46:0/1919163116 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f00cf9e0 con 0x7fa4f015c9d0 2024-03-29T08:24:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.385+0000 7fa4f5200640 1 --2- 172.21.15.46:0/1919163116 >> v2:172.21.15.73:3300/0 conn(0x7fa4f0099150 0x7fa4f0166a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:24:47.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.389+0000 7fa4efe00640 1 --2- 172.21.15.46:0/1919163116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f015c9d0 0x7fa4f00cea20 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0071920 tx=0x7fa4e001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0f0fd2af76bfb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:47.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.389+0000 7fa4ef400640 1 -- 172.21.15.46:0/1919163116 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00a5860 con 0x7fa4f015c9d0 2024-03-29T08:24:47.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.389+0000 7fa4ef400640 1 -- 172.21.15.46:0/1919163116 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e00a59c0 con 0x7fa4f015c9d0 2024-03-29T08:24:47.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.389+0000 7fa4ef400640 1 -- 172.21.15.46:0/1919163116 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00a5d50 con 0x7fa4f015c9d0 2024-03-29T08:24:47.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.389+0000 7fa4f7200640 1 -- 172.21.15.46:0/1919163116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f015c9d0 msgr2=0x7fa4f00cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.389+0000 7fa4f7200640 1 --2- 172.21.15.46:0/1919163116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f015c9d0 0x7fa4f00cea20 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0071920 tx=0x7fa4e001c980 comp rx=0 tx=0).stop 2024-03-29T08:24:47.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.389+0000 7fa4f7200640 1 -- 172.21.15.46:0/1919163116 shutdown_connections 2024-03-29T08:24:47.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.389+0000 7fa4f7200640 1 --2- 172.21.15.46:0/1919163116 >> v2:172.21.15.73:3300/0 conn(0x7fa4f0099150 0x7fa4f0166a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.389+0000 7fa4f7200640 1 --2- 172.21.15.46:0/1919163116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f015c9d0 0x7fa4f00cea20 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.389+0000 7fa4f7200640 1 -- 172.21.15.46:0/1919163116 >> 172.21.15.46:0/1919163116 conn(0x7fa4f00b21a0 msgr2=0x7fa4f00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:47.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.389+0000 7fa4f7200640 1 -- 172.21.15.46:0/1919163116 shutdown_connections 2024-03-29T08:24:47.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.389+0000 7fa4f7200640 1 -- 172.21.15.46:0/1919163116 wait complete. 2024-03-29T08:24:47.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.389+0000 7fa4f7200640 1 Processor -- start 2024-03-29T08:24:47.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.389+0000 7fa4f7200640 1 -- start start 2024-03-29T08:24:47.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.389+0000 7fa4f7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f0099150 0x7fa4f00c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4f7200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa4f00cef60 0x7fa4f014fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4f7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4f015c9d0 0x7fa4f00c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4f7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00c70a0 con 0x7fa4f0099150 2024-03-29T08:24:47.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4f7200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00ca620 con 0x7fa4f015c9d0 2024-03-29T08:24:47.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4f7200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00c7d00 con 0x7fa4f00cef60 2024-03-29T08:24:47.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f0099150 0x7fa4f00c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4efe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f0099150 0x7fa4f00c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34670/0 (socket says 172.21.15.46:34670) 2024-03-29T08:24:47.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4efe00640 1 -- 172.21.15.46:0/2215192558 learned_addr learned my addr 172.21.15.46:0/2215192558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4f5200640 1 --2- 172.21.15.46:0/2215192558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4f015c9d0 0x7fa4f00c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4e7400640 1 --2- 172.21.15.46:0/2215192558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa4f00cef60 0x7fa4f014fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4efe00640 1 -- 172.21.15.46:0/2215192558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa4f00cef60 msgr2=0x7fa4f014fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4efe00640 1 --2- 172.21.15.46:0/2215192558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa4f00cef60 0x7fa4f014fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4efe00640 1 -- 172.21.15.46:0/2215192558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4f015c9d0 msgr2=0x7fa4f00c41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4efe00640 1 --2- 172.21.15.46:0/2215192558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4f015c9d0 0x7fa4f00c41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4efe00640 1 -- 172.21.15.46:0/2215192558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e0009b30 con 0x7fa4f0099150 2024-03-29T08:24:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4e7400640 1 --2- 172.21.15.46:0/2215192558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa4f00cef60 0x7fa4f014fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:47.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4efe00640 1 --2- 172.21.15.46:0/2215192558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f0099150 0x7fa4f00c6b60 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fa4e001a5b0 tx=0x7fa4e0019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4ecc00640 1 -- 172.21.15.46:0/2215192558 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00b8670 con 0x7fa4f0099150 2024-03-29T08:24:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4ecc00640 1 -- 172.21.15.46:0/2215192558 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e00b87f0 con 0x7fa4f0099150 2024-03-29T08:24:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4f7200640 1 -- 172.21.15.46:0/2215192558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f00c7780 con 0x7fa4f0099150 2024-03-29T08:24:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4ecc00640 1 -- 172.21.15.46:0/2215192558 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00b8ba0 con 0x7fa4f0099150 2024-03-29T08:24:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4f7200640 1 -- 172.21.15.46:0/2215192558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f00c9f60 con 0x7fa4f0099150 2024-03-29T08:24:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4f7200640 1 -- 172.21.15.46:0/2215192558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f0066c90 con 0x7fa4f0099150 2024-03-29T08:24:47.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.393+0000 7fa4ecc00640 1 -- 172.21.15.46:0/2215192558 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa4e00b8d00 con 0x7fa4f0099150 2024-03-29T08:24:47.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.397+0000 7fa4ecc00640 1 --2- 172.21.15.46:0/2215192558 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa4cc066690 0x7fa4cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.397+0000 7fa4ecc00640 1 -- 172.21.15.46:0/2215192558 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4e011f6e0 con 0x7fa4f0099150 2024-03-29T08:24:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.397+0000 7fa4e7400640 1 --2- 172.21.15.46:0/2215192558 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa4cc066690 0x7fa4cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.397+0000 7fa4e7400640 1 --2- 172.21.15.46:0/2215192558 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa4cc066690 0x7fa4cc068b50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00cdda0 tx=0x7fa4d8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.401+0000 7fa4ecc00640 1 -- 172.21.15.46:0/2215192558 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa4e00ecc00 con 0x7fa4f0099150 2024-03-29T08:24:47.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.557+0000 7fa4f7200640 1 -- 172.21.15.46:0/2215192558 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa4f006b330 con 0x7fa4cc066690 2024-03-29T08:24:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.565+0000 7fa4ecc00640 1 -- 172.21.15.46:0/2215192558 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289414 (secure 0 0 0) 0x7fa4d8003ad0 con 0x7fa4cc066690 2024-03-29T08:24:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:47.571 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:24:47.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.569+0000 7fa4f7200640 1 -- 172.21.15.46:0/2215192558 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa4cc066690 msgr2=0x7fa4cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.569+0000 7fa4f7200640 1 --2- 172.21.15.46:0/2215192558 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa4cc066690 0x7fa4cc068b50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00cdda0 tx=0x7fa4d8009040 comp rx=0 tx=0).stop 2024-03-29T08:24:47.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.569+0000 7fa4f7200640 1 -- 172.21.15.46:0/2215192558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f0099150 msgr2=0x7fa4f00c6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.569+0000 7fa4f7200640 1 --2- 172.21.15.46:0/2215192558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f0099150 0x7fa4f00c6b60 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fa4e001a5b0 tx=0x7fa4e0019720 comp rx=0 tx=0).stop 2024-03-29T08:24:47.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.569+0000 7fa4f7200640 1 -- 172.21.15.46:0/2215192558 shutdown_connections 2024-03-29T08:24:47.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.569+0000 7fa4f7200640 1 --2- 172.21.15.46:0/2215192558 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa4cc066690 0x7fa4cc068b50 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.569+0000 7fa4f7200640 1 --2- 172.21.15.46:0/2215192558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4f015c9d0 0x7fa4f00c41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.569+0000 7fa4f7200640 1 --2- 172.21.15.46:0/2215192558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa4f00cef60 0x7fa4f014fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.569+0000 7fa4f7200640 1 --2- 172.21.15.46:0/2215192558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4f0099150 0x7fa4f00c6b60 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.569+0000 7fa4f7200640 1 -- 172.21.15.46:0/2215192558 >> 172.21.15.46:0/2215192558 conn(0x7fa4f00b21a0 msgr2=0x7fa4f00cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:47.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.569+0000 7fa4f7200640 1 -- 172.21.15.46:0/2215192558 shutdown_connections 2024-03-29T08:24:47.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.569+0000 7fa4f7200640 1 -- 172.21.15.46:0/2215192558 wait complete. 2024-03-29T08:24:47.589 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-03-29T08:24:46.140418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258896,"kb_used_data":9920,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754715824,"statfs":{"total":773094113280,"available":772829003776,"internally_reserved":0,"allocated":10158080,"data_stored":4996428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.065682"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.508588+0000","last_change":"2024-03-29T08:24:43.500863+0000","last_active":"2024-03-29T08:24:43.508588+0000","last_peered":"2024-03-29T08:24:43.508588+0000","last_clean":"2024-03-29T08:24:43.508588+0000","last_became_active":"2024-03-29T08:24:43.500282+0000","last_became_peered":"2024-03-29T08:24:43.500282+0000","last_unstale":"2024-03-29T08:24:43.508588+0000","last_undegraded":"2024-03-29T08:24:43.508588+0000","last_fullsized":"2024-03-29T08:24:43.508588+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:57:37.160390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488930+0000","last_change":"2024-03-29T08:24:43.507649+0000","last_active":"2024-03-29T08:24:44.488930+0000","last_peered":"2024-03-29T08:24:44.488930+0000","last_clean":"2024-03-29T08:24:44.488930+0000","last_became_active":"2024-03-29T08:24:43.507394+0000","last_became_peered":"2024-03-29T08:24:43.507394+0000","last_unstale":"2024-03-29T08:24:44.488930+0000","last_undegraded":"2024-03-29T08:24:44.488930+0000","last_fullsized":"2024-03-29T08:24:44.488930+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:13:35.362060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483415+0000","last_change":"2024-03-29T08:24:43.498577+0000","last_active":"2024-03-29T08:24:44.483415+0000","last_peered":"2024-03-29T08:24:44.483415+0000","last_clean":"2024-03-29T08:24:44.483415+0000","last_became_active":"2024-03-29T08:24:43.498317+0000","last_became_peered":"2024-03-29T08:24:43.498317+0000","last_unstale":"2024-03-29T08:24:44.483415+0000","last_undegraded":"2024-03-29T08:24:44.483415+0000","last_fullsized":"2024-03-29T08:24:44.483415+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:59:42.492346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.517817+0000","last_change":"2024-03-29T08:24:43.515036+0000","last_active":"2024-03-29T08:24:43.517817+0000","last_peered":"2024-03-29T08:24:43.517817+0000","last_clean":"2024-03-29T08:24:43.517817+0000","last_became_active":"2024-03-29T08:24:43.512474+0000","last_became_peered":"2024-03-29T08:24:43.512474+0000","last_unstale":"2024-03-29T08:24:43.517817+0000","last_undegraded":"2024-03-29T08:24:43.517817+0000","last_fullsized":"2024-03-29T08:24:43.517817+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:26.673863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861448+0000","last_change":"2024-03-29T08:24:43.504186+0000","last_active":"2024-03-29T08:24:44.861448+0000","last_peered":"2024-03-29T08:24:44.861448+0000","last_clean":"2024-03-29T08:24:44.861448+0000","last_became_active":"2024-03-29T08:24:43.503943+0000","last_became_peered":"2024-03-29T08:24:43.503943+0000","last_unstale":"2024-03-29T08:24:44.861448+0000","last_undegraded":"2024-03-29T08:24:44.861448+0000","last_fullsized":"2024-03-29T08:24:44.861448+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:04:07.876026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483168+0000","last_change":"2024-03-29T08:24:43.511703+0000","last_active":"2024-03-29T08:24:44.483168+0000","last_peered":"2024-03-29T08:24:44.483168+0000","last_clean":"2024-03-29T08:24:44.483168+0000","last_became_active":"2024-03-29T08:24:43.511475+0000","last_became_peered":"2024-03-29T08:24:43.511475+0000","last_unstale":"2024-03-29T08:24:44.483168+0000","last_undegraded":"2024-03-29T08:24:44.483168+0000","last_fullsized":"2024-03-29T08:24:44.483168+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:14:35.102272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.487594+0000","last_change":"2024-03-29T08:24:43.497300+0000","last_active":"2024-03-29T08:24:44.487594+0000","last_peered":"2024-03-29T08:24:44.487594+0000","last_clean":"2024-03-29T08:24:44.487594+0000","last_became_active":"2024-03-29T08:24:43.497060+0000","last_became_peered":"2024-03-29T08:24:43.497060+0000","last_unstale":"2024-03-29T08:24:44.487594+0000","last_undegraded":"2024-03-29T08:24:44.487594+0000","last_fullsized":"2024-03-29T08:24:44.487594+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:14.480052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988075+0000","last_change":"2024-03-29T08:24:43.504898+0000","last_active":"2024-03-29T08:24:44.988075+0000","last_peered":"2024-03-29T08:24:44.988075+0000","last_clean":"2024-03-29T08:24:44.988075+0000","last_became_active":"2024-03-29T08:24:43.504614+0000","last_became_peered":"2024-03-29T08:24:43.504614+0000","last_unstale":"2024-03-29T08:24:44.988075+0000","last_undegraded":"2024-03-29T08:24:44.988075+0000","last_fullsized":"2024-03-29T08:24:44.988075+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:03.589841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.509629+0000","last_change":"2024-03-29T08:24:43.503376+0000","last_active":"2024-03-29T08:24:43.509629+0000","last_peered":"2024-03-29T08:24:43.509629+0000","last_clean":"2024-03-29T08:24:43.509629+0000","last_became_active":"2024-03-29T08:24:43.503166+0000","last_became_peered":"2024-03-29T08:24:43.503166+0000","last_unstale":"2024-03-29T08:24:43.509629+0000","last_undegraded":"2024-03-29T08:24:43.509629+0000","last_fullsized":"2024-03-29T08:24:43.509629+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:53:40.790853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483504+0000","last_change":"2024-03-29T08:24:43.504360+0000","last_active":"2024-03-29T08:24:44.483504+0000","last_peered":"2024-03-29T08:24:44.483504+0000","last_clean":"2024-03-29T08:24:44.483504+0000","last_became_active":"2024-03-29T08:24:43.504125+0000","last_became_peered":"2024-03-29T08:24:43.504125+0000","last_unstale":"2024-03-29T08:24:44.483504+0000","last_undegraded":"2024-03-29T08:24:44.483504+0000","last_fullsized":"2024-03-29T08:24:44.483504+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:43:36.123959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.510666+0000","last_change":"2024-03-29T08:24:43.503954+0000","last_active":"2024-03-29T08:24:43.510666+0000","last_peered":"2024-03-29T08:24:43.510666+0000","last_clean":"2024-03-29T08:24:43.510666+0000","last_became_active":"2024-03-29T08:24:43.503649+0000","last_became_peered":"2024-03-29T08:24:43.503649+0000","last_unstale":"2024-03-29T08:24:43.510666+0000","last_undegraded":"2024-03-29T08:24:43.510666+0000","last_fullsized":"2024-03-29T08:24:43.510666+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:23.633523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988128+0000","last_change":"2024-03-29T08:24:43.500906+0000","last_active":"2024-03-29T08:24:44.988128+0000","last_peered":"2024-03-29T08:24:44.988128+0000","last_clean":"2024-03-29T08:24:44.988128+0000","last_became_active":"2024-03-29T08:24:43.500643+0000","last_became_peered":"2024-03-29T08:24:43.500643+0000","last_unstale":"2024-03-29T08:24:44.988128+0000","last_undegraded":"2024-03-29T08:24:44.988128+0000","last_fullsized":"2024-03-29T08:24:44.988128+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:23:51.374527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861312+0000","last_change":"2024-03-29T08:24:43.503715+0000","last_active":"2024-03-29T08:24:44.861312+0000","last_peered":"2024-03-29T08:24:44.861312+0000","last_clean":"2024-03-29T08:24:44.861312+0000","last_became_active":"2024-03-29T08:24:43.501398+0000","last_became_peered":"2024-03-29T08:24:43.501398+0000","last_unstale":"2024-03-29T08:24:44.861312+0000","last_undegraded":"2024-03-29T08:24:44.861312+0000","last_fullsized":"2024-03-29T08:24:44.861312+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:21:03.953033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488496+0000","last_change":"2024-03-29T08:24:43.514487+0000","last_active":"2024-03-29T08:24:44.488496+0000","last_peered":"2024-03-29T08:24:44.488496+0000","last_clean":"2024-03-29T08:24:44.488496+0000","last_became_active":"2024-03-29T08:24:43.513873+0000","last_became_peered":"2024-03-29T08:24:43.513873+0000","last_unstale":"2024-03-29T08:24:44.488496+0000","last_undegraded":"2024-03-29T08:24:44.488496+0000","last_fullsized":"2024-03-29T08:24:44.488496+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:05:19.517087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.487693+0000","last_change":"2024-03-29T08:24:43.498257+0000","last_active":"2024-03-29T08:24:44.487693+0000","last_peered":"2024-03-29T08:24:44.487693+0000","last_clean":"2024-03-29T08:24:44.487693+0000","last_became_active":"2024-03-29T08:24:43.497953+0000","last_became_peered":"2024-03-29T08:24:43.497953+0000","last_unstale":"2024-03-29T08:24:44.487693+0000","last_undegraded":"2024-03-29T08:24:44.487693+0000","last_fullsized":"2024-03-29T08:24:44.487693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:03.589841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483073+0000","last_change":"2024-03-29T08:24:43.497050+0000","last_active":"2024-03-29T08:24:44.483073+0000","last_peered":"2024-03-29T08:24:44.483073+0000","last_clean":"2024-03-29T08:24:44.483073+0000","last_became_active":"2024-03-29T08:24:43.496801+0000","last_became_peered":"2024-03-29T08:24:43.496801+0000","last_unstale":"2024-03-29T08:24:44.483073+0000","last_undegraded":"2024-03-29T08:24:44.483073+0000","last_fullsized":"2024-03-29T08:24:44.483073+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:04:35.962563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988434+0000","last_change":"2024-03-29T08:24:43.502011+0000","last_active":"2024-03-29T08:24:44.988434+0000","last_peered":"2024-03-29T08:24:44.988434+0000","last_clean":"2024-03-29T08:24:44.988434+0000","last_became_active":"2024-03-29T08:24:43.501778+0000","last_became_peered":"2024-03-29T08:24:43.501778+0000","last_unstale":"2024-03-29T08:24:44.988434+0000","last_undegraded":"2024-03-29T08:24:44.988434+0000","last_fullsized":"2024-03-29T08:24:44.988434+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:45:47.180483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.508562+0000","last_change":"2024-03-29T08:24:43.497136+0000","last_active":"2024-03-29T08:24:43.508562+0000","last_peered":"2024-03-29T08:24:43.508562+0000","last_clean":"2024-03-29T08:24:43.508562+0000","last_became_active":"2024-03-29T08:24:43.496857+0000","last_became_peered":"2024-03-29T08:24:43.496857+0000","last_unstale":"2024-03-29T08:24:43.508562+0000","last_undegraded":"2024-03-29T08:24:43.508562+0000","last_fullsized":"2024-03-29T08:24:43.508562+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:45:47.180483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488674+0000","last_change":"2024-03-29T08:24:43.508237+0000","last_active":"2024-03-29T08:24:44.488674+0000","last_peered":"2024-03-29T08:24:44.488674+0000","last_clean":"2024-03-29T08:24:44.488674+0000","last_became_active":"2024-03-29T08:24:43.507524+0000","last_became_peered":"2024-03-29T08:24:43.507524+0000","last_unstale":"2024-03-29T08:24:44.488674+0000","last_undegraded":"2024-03-29T08:24:44.488674+0000","last_fullsized":"2024-03-29T08:24:44.488674+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:02:34.591004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488314+0000","last_change":"2024-03-29T08:24:43.510515+0000","last_active":"2024-03-29T08:24:44.488314+0000","last_peered":"2024-03-29T08:24:44.488314+0000","last_clean":"2024-03-29T08:24:44.488314+0000","last_became_active":"2024-03-29T08:24:43.510164+0000","last_became_peered":"2024-03-29T08:24:43.510164+0000","last_unstale":"2024-03-29T08:24:44.488314+0000","last_undegraded":"2024-03-29T08:24:44.488314+0000","last_fullsized":"2024-03-29T08:24:44.488314+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:05:13.449547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.514804+0000","last_change":"2024-03-29T08:24:43.505783+0000","last_active":"2024-03-29T08:24:43.514804+0000","last_peered":"2024-03-29T08:24:43.514804+0000","last_clean":"2024-03-29T08:24:43.514804+0000","last_became_active":"2024-03-29T08:24:43.505339+0000","last_became_peered":"2024-03-29T08:24:43.505339+0000","last_unstale":"2024-03-29T08:24:43.514804+0000","last_undegraded":"2024-03-29T08:24:43.514804+0000","last_fullsized":"2024-03-29T08:24:43.514804+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:21:13.208809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488510+0000","last_change":"2024-03-29T08:24:43.507284+0000","last_active":"2024-03-29T08:24:44.488510+0000","last_peered":"2024-03-29T08:24:44.488510+0000","last_clean":"2024-03-29T08:24:44.488510+0000","last_became_active":"2024-03-29T08:24:43.507015+0000","last_became_peered":"2024-03-29T08:24:43.507015+0000","last_unstale":"2024-03-29T08:24:44.488510+0000","last_undegraded":"2024-03-29T08:24:44.488510+0000","last_fullsized":"2024-03-29T08:24:44.488510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:40:45.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988049+0000","last_change":"2024-03-29T08:24:43.501305+0000","last_active":"2024-03-29T08:24:44.988049+0000","last_peered":"2024-03-29T08:24:44.988049+0000","last_clean":"2024-03-29T08:24:44.988049+0000","last_became_active":"2024-03-29T08:24:43.501046+0000","last_became_peered":"2024-03-29T08:24:43.501046+0000","last_unstale":"2024-03-29T08:24:44.988049+0000","last_undegraded":"2024-03-29T08:24:44.988049+0000","last_fullsized":"2024-03-29T08:24:44.988049+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:09.731188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.510631+0000","last_change":"2024-03-29T08:24:43.505985+0000","last_active":"2024-03-29T08:24:43.510631+0000","last_peered":"2024-03-29T08:24:43.510631+0000","last_clean":"2024-03-29T08:24:43.510631+0000","last_became_active":"2024-03-29T08:24:43.505702+0000","last_became_peered":"2024-03-29T08:24:43.505702+0000","last_unstale":"2024-03-29T08:24:43.510631+0000","last_undegraded":"2024-03-29T08:24:43.510631+0000","last_fullsized":"2024-03-29T08:24:43.510631+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:36.275403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.515669+0000","last_change":"2024-03-29T08:24:43.512131+0000","last_active":"2024-03-29T08:24:43.515669+0000","last_peered":"2024-03-29T08:24:43.515669+0000","last_clean":"2024-03-29T08:24:43.515669+0000","last_became_active":"2024-03-29T08:24:43.511842+0000","last_became_peered":"2024-03-29T08:24:43.511842+0000","last_unstale":"2024-03-29T08:24:43.515669+0000","last_undegraded":"2024-03-29T08:24:43.515669+0000","last_fullsized":"2024-03-29T08:24:43.515669+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:03:15.119626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861367+0000","last_change":"2024-03-29T08:24:43.505584+0000","last_active":"2024-03-29T08:24:44.861367+0000","last_peered":"2024-03-29T08:24:44.861367+0000","last_clean":"2024-03-29T08:24:44.861367+0000","last_became_active":"2024-03-29T08:24:43.505284+0000","last_became_peered":"2024-03-29T08:24:43.505284+0000","last_unstale":"2024-03-29T08:24:44.861367+0000","last_undegraded":"2024-03-29T08:24:44.861367+0000","last_fullsized":"2024-03-29T08:24:44.861367+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:44:56.220658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861491+0000","last_change":"2024-03-29T08:24:43.500283+0000","last_active":"2024-03-29T08:24:44.861491+0000","last_peered":"2024-03-29T08:24:44.861491+0000","last_clean":"2024-03-29T08:24:44.861491+0000","last_became_active":"2024-03-29T08:24:43.499781+0000","last_became_peered":"2024-03-29T08:24:43.499781+0000","last_unstale":"2024-03-29T08:24:44.861491+0000","last_undegraded":"2024-03-29T08:24:44.861491+0000","last_fullsized":"2024-03-29T08:24:44.861491+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:29:00.089772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.498131+0000","last_change":"2024-03-29T08:24:43.495197+0000","last_active":"2024-03-29T08:24:43.498131+0000","last_peered":"2024-03-29T08:24:43.498131+0000","last_clean":"2024-03-29T08:24:43.498131+0000","last_became_active":"2024-03-29T08:24:43.494913+0000","last_became_peered":"2024-03-29T08:24:43.494913+0000","last_unstale":"2024-03-29T08:24:43.498131+0000","last_undegraded":"2024-03-29T08:24:43.498131+0000","last_fullsized":"2024-03-29T08:24:43.498131+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:21:55.614273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.514787+0000","last_change":"2024-03-29T08:24:43.505601+0000","last_active":"2024-03-29T08:24:43.514787+0000","last_peered":"2024-03-29T08:24:43.514787+0000","last_clean":"2024-03-29T08:24:43.514787+0000","last_became_active":"2024-03-29T08:24:43.505013+0000","last_became_peered":"2024-03-29T08:24:43.505013+0000","last_unstale":"2024-03-29T08:24:43.514787+0000","last_undegraded":"2024-03-29T08:24:43.514787+0000","last_fullsized":"2024-03-29T08:24:43.514787+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:08:13.184351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488593+0000","last_change":"2024-03-29T08:24:43.514589+0000","last_active":"2024-03-29T08:24:44.488593+0000","last_peered":"2024-03-29T08:24:44.488593+0000","last_clean":"2024-03-29T08:24:44.488593+0000","last_became_active":"2024-03-29T08:24:43.514253+0000","last_became_peered":"2024-03-29T08:24:43.514253+0000","last_unstale":"2024-03-29T08:24:44.488593+0000","last_undegraded":"2024-03-29T08:24:44.488593+0000","last_fullsized":"2024-03-29T08:24:44.488593+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:49:59.075543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483103+0000","last_change":"2024-03-29T08:24:43.511667+0000","last_active":"2024-03-29T08:24:44.483103+0000","last_peered":"2024-03-29T08:24:44.483103+0000","last_clean":"2024-03-29T08:24:44.483103+0000","last_became_active":"2024-03-29T08:24:43.511381+0000","last_became_peered":"2024-03-29T08:24:43.511381+0000","last_unstale":"2024-03-29T08:24:44.483103+0000","last_undegraded":"2024-03-29T08:24:44.483103+0000","last_fullsized":"2024-03-29T08:24:44.483103+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:02:29.946962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483096+0000","last_change":"2024-03-29T08:24:43.500942+0000","last_active":"2024-03-29T08:24:44.483096+0000","last_peered":"2024-03-29T08:24:44.483096+0000","last_clean":"2024-03-29T08:24:44.483096+0000","last_became_active":"2024-03-29T08:24:43.500668+0000","last_became_peered":"2024-03-29T08:24:43.500668+0000","last_unstale":"2024-03-29T08:24:44.483096+0000","last_undegraded":"2024-03-29T08:24:44.483096+0000","last_fullsized":"2024-03-29T08:24:44.483096+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:38.958688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988322+0000","last_change":"2024-03-29T08:24:43.496778+0000","last_active":"2024-03-29T08:24:44.988322+0000","last_peered":"2024-03-29T08:24:44.988322+0000","last_clean":"2024-03-29T08:24:44.988322+0000","last_became_active":"2024-03-29T08:24:43.496496+0000","last_became_peered":"2024-03-29T08:24:43.496496+0000","last_unstale":"2024-03-29T08:24:44.988322+0000","last_undegraded":"2024-03-29T08:24:44.988322+0000","last_fullsized":"2024-03-29T08:24:44.988322+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:14.480052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"91'4","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861578+0000","last_change":"2024-03-29T08:24:43.099994+0000","last_active":"2024-03-29T08:24:44.861578+0000","last_peered":"2024-03-29T08:24:44.861578+0000","last_clean":"2024-03-29T08:24:44.861578+0000","last_became_active":"2024-03-29T08:24:43.099724+0000","last_became_peered":"2024-03-29T08:24:43.099724+0000","last_unstale":"2024-03-29T08:24:44.861578+0000","last_undegraded":"2024-03-29T08:24:44.861578+0000","last_fullsized":"2024-03-29T08:24:44.861578+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:53:40.344288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.508604+0000","last_change":"2024-03-29T08:24:43.501012+0000","last_active":"2024-03-29T08:24:43.508604+0000","last_peered":"2024-03-29T08:24:43.508604+0000","last_clean":"2024-03-29T08:24:43.508604+0000","last_became_active":"2024-03-29T08:24:43.500631+0000","last_became_peered":"2024-03-29T08:24:43.500631+0000","last_unstale":"2024-03-29T08:24:43.508604+0000","last_undegraded":"2024-03-29T08:24:43.508604+0000","last_fullsized":"2024-03-29T08:24:43.508604+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:55:06.990851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"91'1","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483524+0000","last_change":"2024-03-29T08:24:43.089840+0000","last_active":"2024-03-29T08:24:44.483524+0000","last_peered":"2024-03-29T08:24:44.483524+0000","last_clean":"2024-03-29T08:24:44.483524+0000","last_became_active":"2024-03-29T08:24:43.089218+0000","last_became_peered":"2024-03-29T08:24:43.089218+0000","last_unstale":"2024-03-29T08:24:44.483524+0000","last_undegraded":"2024-03-29T08:24:44.483524+0000","last_fullsized":"2024-03-29T08:24:44.483524+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:10.817335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988163+0000","last_change":"2024-03-29T08:24:43.503203+0000","last_active":"2024-03-29T08:24:44.988163+0000","last_peered":"2024-03-29T08:24:44.988163+0000","last_clean":"2024-03-29T08:24:44.988163+0000","last_became_active":"2024-03-29T08:24:43.502676+0000","last_became_peered":"2024-03-29T08:24:43.502676+0000","last_unstale":"2024-03-29T08:24:44.988163+0000","last_undegraded":"2024-03-29T08:24:44.988163+0000","last_fullsized":"2024-03-29T08:24:44.988163+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:52.482922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.513474+0000","last_change":"2024-03-29T08:24:43.098772+0000","last_active":"2024-03-29T08:24:43.513474+0000","last_peered":"2024-03-29T08:24:43.513474+0000","last_clean":"2024-03-29T08:24:43.513474+0000","last_became_active":"2024-03-29T08:24:43.098489+0000","last_became_peered":"2024-03-29T08:24:43.098489+0000","last_unstale":"2024-03-29T08:24:43.513474+0000","last_undegraded":"2024-03-29T08:24:43.513474+0000","last_fullsized":"2024-03-29T08:24:43.513474+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:32:13.524931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.508968+0000","last_change":"2024-03-29T08:24:43.502146+0000","last_active":"2024-03-29T08:24:43.508968+0000","last_peered":"2024-03-29T08:24:43.508968+0000","last_clean":"2024-03-29T08:24:43.508968+0000","last_became_active":"2024-03-29T08:24:43.501888+0000","last_became_peered":"2024-03-29T08:24:43.501888+0000","last_unstale":"2024-03-29T08:24:43.508968+0000","last_undegraded":"2024-03-29T08:24:43.508968+0000","last_fullsized":"2024-03-29T08:24:43.508968+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:42.208598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.515459+0000","last_change":"2024-03-29T08:24:43.091946+0000","last_active":"2024-03-29T08:24:43.515459+0000","last_peered":"2024-03-29T08:24:43.515459+0000","last_clean":"2024-03-29T08:24:43.515459+0000","last_became_active":"2024-03-29T08:24:43.091643+0000","last_became_peered":"2024-03-29T08:24:43.091643+0000","last_unstale":"2024-03-29T08:24:43.515459+0000","last_undegraded":"2024-03-29T08:24:43.515459+0000","last_fullsized":"2024-03-29T08:24:43.515459+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:58:04.981235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.516336+0000","last_change":"2024-03-29T08:24:43.513319+0000","last_active":"2024-03-29T08:24:43.516336+0000","last_peered":"2024-03-29T08:24:43.516336+0000","last_clean":"2024-03-29T08:24:43.516336+0000","last_became_active":"2024-03-29T08:24:43.513073+0000","last_became_peered":"2024-03-29T08:24:43.513073+0000","last_unstale":"2024-03-29T08:24:43.516336+0000","last_undegraded":"2024-03-29T08:24:43.516336+0000","last_fullsized":"2024-03-29T08:24:43.516336+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:32:22.633370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"91'1","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988088+0000","last_change":"2024-03-29T08:24:43.089850+0000","last_active":"2024-03-29T08:24:44.988088+0000","last_peered":"2024-03-29T08:24:44.988088+0000","last_clean":"2024-03-29T08:24:44.988088+0000","last_became_active":"2024-03-29T08:24:43.089537+0000","last_became_peered":"2024-03-29T08:24:43.089537+0000","last_unstale":"2024-03-29T08:24:44.988088+0000","last_undegraded":"2024-03-29T08:24:44.988088+0000","last_fullsized":"2024-03-29T08:24:44.988088+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:21:54.611561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483353+0000","last_change":"2024-03-29T08:24:43.503817+0000","last_active":"2024-03-29T08:24:44.483353+0000","last_peered":"2024-03-29T08:24:44.483353+0000","last_clean":"2024-03-29T08:24:44.483353+0000","last_became_active":"2024-03-29T08:24:43.503454+0000","last_became_peered":"2024-03-29T08:24:43.503454+0000","last_unstale":"2024-03-29T08:24:44.483353+0000","last_undegraded":"2024-03-29T08:24:44.483353+0000","last_fullsized":"2024-03-29T08:24:44.483353+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:57:04.503788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"91'1","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483197+0000","last_change":"2024-03-29T08:24:43.091662+0000","last_active":"2024-03-29T08:24:44.483197+0000","last_peered":"2024-03-29T08:24:44.483197+0000","last_clean":"2024-03-29T08:24:44.483197+0000","last_became_active":"2024-03-29T08:24:43.091078+0000","last_became_peered":"2024-03-29T08:24:43.091078+0000","last_unstale":"2024-03-29T08:24:44.483197+0000","last_undegraded":"2024-03-29T08:24:44.483197+0000","last_fullsized":"2024-03-29T08:24:44.483197+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:08:47.116697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.509571+0000","last_change":"2024-03-29T08:24:43.502707+0000","last_active":"2024-03-29T08:24:43.509571+0000","last_peered":"2024-03-29T08:24:43.509571+0000","last_clean":"2024-03-29T08:24:43.509571+0000","last_became_active":"2024-03-29T08:24:43.502410+0000","last_became_peered":"2024-03-29T08:24:43.502410+0000","last_unstale":"2024-03-29T08:24:43.509571+0000","last_undegraded":"2024-03-29T08:24:43.509571+0000","last_fullsized":"2024-03-29T08:24:43.509571+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:32:14.527644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488245+0000","last_change":"2024-03-29T08:24:43.101557+0000","last_active":"2024-03-29T08:24:44.488245+0000","last_peered":"2024-03-29T08:24:44.488245+0000","last_clean":"2024-03-29T08:24:44.488245+0000","last_became_active":"2024-03-29T08:24:43.101270+0000","last_became_peered":"2024-03-29T08:24:43.101270+0000","last_unstale":"2024-03-29T08:24:44.488245+0000","last_undegraded":"2024-03-29T08:24:44.488245+0000","last_fullsized":"2024-03-29T08:24:44.488245+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:20:38.882583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488029+0000","last_change":"2024-03-29T08:24:43.509577+0000","last_active":"2024-03-29T08:24:44.488029+0000","last_peered":"2024-03-29T08:24:44.488029+0000","last_clean":"2024-03-29T08:24:44.488029+0000","last_became_active":"2024-03-29T08:24:43.509227+0000","last_became_peered":"2024-03-29T08:24:43.509227+0000","last_unstale":"2024-03-29T08:24:44.488029+0000","last_undegraded":"2024-03-29T08:24:44.488029+0000","last_fullsized":"2024-03-29T08:24:44.488029+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:28:22.979416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488008+0000","last_change":"2024-03-29T08:24:43.094283+0000","last_active":"2024-03-29T08:24:44.488008+0000","last_peered":"2024-03-29T08:24:44.488008+0000","last_clean":"2024-03-29T08:24:44.488008+0000","last_became_active":"2024-03-29T08:24:43.093930+0000","last_became_peered":"2024-03-29T08:24:43.093930+0000","last_unstale":"2024-03-29T08:24:44.488008+0000","last_undegraded":"2024-03-29T08:24:44.488008+0000","last_fullsized":"2024-03-29T08:24:44.488008+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:38:22.729031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488669+0000","last_change":"2024-03-29T08:24:43.502127+0000","last_active":"2024-03-29T08:24:44.488669+0000","last_peered":"2024-03-29T08:24:44.488669+0000","last_clean":"2024-03-29T08:24:44.488669+0000","last_became_active":"2024-03-29T08:24:43.501769+0000","last_became_peered":"2024-03-29T08:24:43.501769+0000","last_unstale":"2024-03-29T08:24:44.488669+0000","last_undegraded":"2024-03-29T08:24:44.488669+0000","last_fullsized":"2024-03-29T08:24:44.488669+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:42.208598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.487030+0000","last_change":"2024-03-29T08:24:43.088073+0000","last_active":"2024-03-29T08:24:43.487030+0000","last_peered":"2024-03-29T08:24:43.487030+0000","last_clean":"2024-03-29T08:24:43.487030+0000","last_became_active":"2024-03-29T08:24:43.086519+0000","last_became_peered":"2024-03-29T08:24:43.086519+0000","last_unstale":"2024-03-29T08:24:43.487030+0000","last_undegraded":"2024-03-29T08:24:43.487030+0000","last_fullsized":"2024-03-29T08:24:43.487030+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:23:21.392519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.513690+0000","last_change":"2024-03-29T08:24:43.512338+0000","last_active":"2024-03-29T08:24:43.513690+0000","last_peered":"2024-03-29T08:24:43.513690+0000","last_clean":"2024-03-29T08:24:43.513690+0000","last_became_active":"2024-03-29T08:24:43.512018+0000","last_became_peered":"2024-03-29T08:24:43.512018+0000","last_unstale":"2024-03-29T08:24:43.513690+0000","last_undegraded":"2024-03-29T08:24:43.513690+0000","last_fullsized":"2024-03-29T08:24:43.513690+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:40:45.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483576+0000","last_change":"2024-03-29T08:24:43.090188+0000","last_active":"2024-03-29T08:24:44.483576+0000","last_peered":"2024-03-29T08:24:44.483576+0000","last_clean":"2024-03-29T08:24:44.483576+0000","last_became_active":"2024-03-29T08:24:43.089629+0000","last_became_peered":"2024-03-29T08:24:43.089629+0000","last_unstale":"2024-03-29T08:24:44.483576+0000","last_undegraded":"2024-03-29T08:24:44.483576+0000","last_fullsized":"2024-03-29T08:24:44.483576+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:08.728476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.518090+0000","last_change":"2024-03-29T08:24:43.512173+0000","last_active":"2024-03-29T08:24:43.518090+0000","last_peered":"2024-03-29T08:24:43.518090+0000","last_clean":"2024-03-29T08:24:43.518090+0000","last_became_active":"2024-03-29T08:24:43.511864+0000","last_became_peered":"2024-03-29T08:24:43.511864+0000","last_unstale":"2024-03-29T08:24:43.518090+0000","last_undegraded":"2024-03-29T08:24:43.518090+0000","last_fullsized":"2024-03-29T08:24:43.518090+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:19:48.190913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"91'3","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.512755+0000","last_change":"2024-03-29T08:24:43.097101+0000","last_active":"2024-03-29T08:24:43.512755+0000","last_peered":"2024-03-29T08:24:43.512755+0000","last_clean":"2024-03-29T08:24:43.512755+0000","last_became_active":"2024-03-29T08:24:43.096811+0000","last_became_peered":"2024-03-29T08:24:43.096811+0000","last_unstale":"2024-03-29T08:24:43.512755+0000","last_undegraded":"2024-03-29T08:24:43.512755+0000","last_fullsized":"2024-03-29T08:24:43.512755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:56:44.209659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861406+0000","last_change":"2024-03-29T08:24:43.507514+0000","last_active":"2024-03-29T08:24:44.861406+0000","last_peered":"2024-03-29T08:24:44.861406+0000","last_clean":"2024-03-29T08:24:44.861406+0000","last_became_active":"2024-03-29T08:24:43.506436+0000","last_became_peered":"2024-03-29T08:24:43.506436+0000","last_unstale":"2024-03-29T08:24:44.861406+0000","last_undegraded":"2024-03-29T08:24:44.861406+0000","last_fullsized":"2024-03-29T08:24:44.861406+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:18:10.946658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"91'6","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988260+0000","last_change":"2024-03-29T08:24:43.094920+0000","last_active":"2024-03-29T08:24:44.988260+0000","last_peered":"2024-03-29T08:24:44.988260+0000","last_clean":"2024-03-29T08:24:44.988260+0000","last_became_active":"2024-03-29T08:24:43.094629+0000","last_became_peered":"2024-03-29T08:24:43.094629+0000","last_unstale":"2024-03-29T08:24:44.988260+0000","last_undegraded":"2024-03-29T08:24:44.988260+0000","last_fullsized":"2024-03-29T08:24:44.988260+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:41.205886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.498821+0000","last_change":"2024-03-29T08:24:43.492204+0000","last_active":"2024-03-29T08:24:43.498821+0000","last_peered":"2024-03-29T08:24:43.498821+0000","last_clean":"2024-03-29T08:24:43.498821+0000","last_became_active":"2024-03-29T08:24:43.491943+0000","last_became_peered":"2024-03-29T08:24:43.491943+0000","last_unstale":"2024-03-29T08:24:43.498821+0000","last_undegraded":"2024-03-29T08:24:43.498821+0000","last_fullsized":"2024-03-29T08:24:43.498821+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:23:51.374527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.516202+0000","last_change":"2024-03-29T08:24:43.095534+0000","last_active":"2024-03-29T08:24:43.516202+0000","last_peered":"2024-03-29T08:24:43.516202+0000","last_clean":"2024-03-29T08:24:43.516202+0000","last_became_active":"2024-03-29T08:24:43.095179+0000","last_became_peered":"2024-03-29T08:24:43.095179+0000","last_unstale":"2024-03-29T08:24:43.516202+0000","last_undegraded":"2024-03-29T08:24:43.516202+0000","last_fullsized":"2024-03-29T08:24:43.516202+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:20:33.235596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.510260+0000","last_change":"2024-03-29T08:24:43.505245+0000","last_active":"2024-03-29T08:24:43.510260+0000","last_peered":"2024-03-29T08:24:43.510260+0000","last_clean":"2024-03-29T08:24:43.510260+0000","last_became_active":"2024-03-29T08:24:43.502538+0000","last_became_peered":"2024-03-29T08:24:43.502538+0000","last_unstale":"2024-03-29T08:24:43.510260+0000","last_undegraded":"2024-03-29T08:24:43.510260+0000","last_fullsized":"2024-03-29T08:24:43.510260+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:55.009436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.487732+0000","last_change":"2024-03-29T08:24:43.097110+0000","last_active":"2024-03-29T08:24:44.487732+0000","last_peered":"2024-03-29T08:24:44.487732+0000","last_clean":"2024-03-29T08:24:44.487732+0000","last_became_active":"2024-03-29T08:24:43.096780+0000","last_became_peered":"2024-03-29T08:24:43.096780+0000","last_unstale":"2024-03-29T08:24:44.487732+0000","last_undegraded":"2024-03-29T08:24:44.487732+0000","last_fullsized":"2024-03-29T08:24:44.487732+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:10.817335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488027+0000","last_change":"2024-03-29T08:24:43.514216+0000","last_active":"2024-03-29T08:24:44.488027+0000","last_peered":"2024-03-29T08:24:44.488027+0000","last_clean":"2024-03-29T08:24:44.488027+0000","last_became_active":"2024-03-29T08:24:43.513784+0000","last_became_peered":"2024-03-29T08:24:43.513784+0000","last_unstale":"2024-03-29T08:24:44.488027+0000","last_undegraded":"2024-03-29T08:24:44.488027+0000","last_fullsized":"2024-03-29T08:24:44.488027+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:21:55.614273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483109+0000","last_change":"2024-03-29T08:24:43.092297+0000","last_active":"2024-03-29T08:24:44.483109+0000","last_peered":"2024-03-29T08:24:44.483109+0000","last_clean":"2024-03-29T08:24:44.483109+0000","last_became_active":"2024-03-29T08:24:43.092039+0000","last_became_peered":"2024-03-29T08:24:43.092039+0000","last_unstale":"2024-03-29T08:24:44.483109+0000","last_undegraded":"2024-03-29T08:24:44.483109+0000","last_fullsized":"2024-03-29T08:24:44.483109+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:35.272690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.511087+0000","last_change":"2024-03-29T08:24:43.506382+0000","last_active":"2024-03-29T08:24:43.511087+0000","last_peered":"2024-03-29T08:24:43.511087+0000","last_clean":"2024-03-29T08:24:43.511087+0000","last_became_active":"2024-03-29T08:24:43.506065+0000","last_became_peered":"2024-03-29T08:24:43.506065+0000","last_unstale":"2024-03-29T08:24:43.511087+0000","last_undegraded":"2024-03-29T08:24:43.511087+0000","last_fullsized":"2024-03-29T08:24:43.511087+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:09.731188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"91'8","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.512972+0000","last_change":"2024-03-29T08:24:43.100298+0000","last_active":"2024-03-29T08:24:43.512972+0000","last_peered":"2024-03-29T08:24:43.512972+0000","last_clean":"2024-03-29T08:24:43.512972+0000","last_became_active":"2024-03-29T08:24:43.099448+0000","last_became_peered":"2024-03-29T08:24:43.099448+0000","last_unstale":"2024-03-29T08:24:43.512972+0000","last_undegraded":"2024-03-29T08:24:43.512972+0000","last_fullsized":"2024-03-29T08:24:43.512972+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:53:39.788140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988365+0000","last_change":"2024-03-29T08:24:43.495633+0000","last_active":"2024-03-29T08:24:44.988365+0000","last_peered":"2024-03-29T08:24:44.988365+0000","last_clean":"2024-03-29T08:24:44.988365+0000","last_became_active":"2024-03-29T08:24:43.495358+0000","last_became_peered":"2024-03-29T08:24:43.495358+0000","last_unstale":"2024-03-29T08:24:44.988365+0000","last_undegraded":"2024-03-29T08:24:44.988365+0000","last_fullsized":"2024-03-29T08:24:44.988365+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:03:15.119626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"91'1","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861666+0000","last_change":"2024-03-29T08:24:43.099420+0000","last_active":"2024-03-29T08:24:44.861666+0000","last_peered":"2024-03-29T08:24:44.861666+0000","last_clean":"2024-03-29T08:24:44.861666+0000","last_became_active":"2024-03-29T08:24:43.099146+0000","last_became_peered":"2024-03-29T08:24:43.099146+0000","last_unstale":"2024-03-29T08:24:44.861666+0000","last_undegraded":"2024-03-29T08:24:44.861666+0000","last_fullsized":"2024-03-29T08:24:44.861666+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:05:34.165676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483642+0000","last_change":"2024-03-29T08:24:43.508543+0000","last_active":"2024-03-29T08:24:44.483642+0000","last_peered":"2024-03-29T08:24:44.483642+0000","last_clean":"2024-03-29T08:24:44.483642+0000","last_became_active":"2024-03-29T08:24:43.508332+0000","last_became_peered":"2024-03-29T08:24:43.508332+0000","last_unstale":"2024-03-29T08:24:44.483642+0000","last_undegraded":"2024-03-29T08:24:44.483642+0000","last_fullsized":"2024-03-29T08:24:44.483642+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:24:54.965342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"91'2","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488799+0000","last_change":"2024-03-29T08:24:43.100835+0000","last_active":"2024-03-29T08:24:44.488799+0000","last_peered":"2024-03-29T08:24:44.488799+0000","last_clean":"2024-03-29T08:24:44.488799+0000","last_became_active":"2024-03-29T08:24:43.100101+0000","last_became_peered":"2024-03-29T08:24:43.100101+0000","last_unstale":"2024-03-29T08:24:44.488799+0000","last_undegraded":"2024-03-29T08:24:44.488799+0000","last_fullsized":"2024-03-29T08:24:44.488799+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:50:52.971257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988196+0000","last_change":"2024-03-29T08:24:43.495626+0000","last_active":"2024-03-29T08:24:44.988196+0000","last_peered":"2024-03-29T08:24:44.988196+0000","last_clean":"2024-03-29T08:24:44.988196+0000","last_became_active":"2024-03-29T08:24:43.495385+0000","last_became_peered":"2024-03-29T08:24:43.495385+0000","last_unstale":"2024-03-29T08:24:44.988196+0000","last_undegraded":"2024-03-29T08:24:44.988196+0000","last_fullsized":"2024-03-29T08:24:44.988196+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:19:48.190913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488423+0000","last_change":"2024-03-29T08:24:43.095264+0000","last_active":"2024-03-29T08:24:44.488423+0000","last_peered":"2024-03-29T08:24:44.488423+0000","last_clean":"2024-03-29T08:24:44.488423+0000","last_became_active":"2024-03-29T08:24:43.094903+0000","last_became_peered":"2024-03-29T08:24:43.094903+0000","last_unstale":"2024-03-29T08:24:44.488423+0000","last_undegraded":"2024-03-29T08:24:44.488423+0000","last_fullsized":"2024-03-29T08:24:44.488423+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:23:07.204168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.487629+0000","last_change":"2024-03-29T08:24:43.499714+0000","last_active":"2024-03-29T08:24:44.487629+0000","last_peered":"2024-03-29T08:24:44.487629+0000","last_clean":"2024-03-29T08:24:44.487629+0000","last_became_active":"2024-03-29T08:24:43.499471+0000","last_became_peered":"2024-03-29T08:24:43.499471+0000","last_unstale":"2024-03-29T08:24:44.487629+0000","last_undegraded":"2024-03-29T08:24:44.487629+0000","last_fullsized":"2024-03-29T08:24:44.487629+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:23.914934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"91'2","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488424+0000","last_change":"2024-03-29T08:24:43.092008+0000","last_active":"2024-03-29T08:24:44.488424+0000","last_peered":"2024-03-29T08:24:44.488424+0000","last_clean":"2024-03-29T08:24:44.488424+0000","last_became_active":"2024-03-29T08:24:43.090131+0000","last_became_peered":"2024-03-29T08:24:43.090131+0000","last_unstale":"2024-03-29T08:24:44.488424+0000","last_undegraded":"2024-03-29T08:24:44.488424+0000","last_fullsized":"2024-03-29T08:24:44.488424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:42:10.005340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861415+0000","last_change":"2024-03-29T08:24:43.511254+0000","last_active":"2024-03-29T08:24:44.861415+0000","last_peered":"2024-03-29T08:24:44.861415+0000","last_clean":"2024-03-29T08:24:44.861415+0000","last_became_active":"2024-03-29T08:24:43.510955+0000","last_became_peered":"2024-03-29T08:24:43.510955+0000","last_unstale":"2024-03-29T08:24:44.861415+0000","last_undegraded":"2024-03-29T08:24:44.861415+0000","last_fullsized":"2024-03-29T08:24:44.861415+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:03:55.979572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.487797+0000","last_change":"2024-03-29T08:24:43.094275+0000","last_active":"2024-03-29T08:24:44.487797+0000","last_peered":"2024-03-29T08:24:44.487797+0000","last_clean":"2024-03-29T08:24:44.487797+0000","last_became_active":"2024-03-29T08:24:43.094022+0000","last_became_peered":"2024-03-29T08:24:43.094022+0000","last_unstale":"2024-03-29T08:24:44.487797+0000","last_undegraded":"2024-03-29T08:24:44.487797+0000","last_fullsized":"2024-03-29T08:24:44.487797+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:08.728476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.509680+0000","last_change":"2024-03-29T08:24:43.501484+0000","last_active":"2024-03-29T08:24:43.509680+0000","last_peered":"2024-03-29T08:24:43.509680+0000","last_clean":"2024-03-29T08:24:43.509680+0000","last_became_active":"2024-03-29T08:24:43.501244+0000","last_became_peered":"2024-03-29T08:24:43.501244+0000","last_unstale":"2024-03-29T08:24:43.509680+0000","last_undegraded":"2024-03-29T08:24:43.509680+0000","last_fullsized":"2024-03-29T08:24:43.509680+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:11:36.608174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.491682+0000","last_change":"2024-03-29T08:24:43.096278+0000","last_active":"2024-03-29T08:24:43.491682+0000","last_peered":"2024-03-29T08:24:43.491682+0000","last_clean":"2024-03-29T08:24:43.491682+0000","last_became_active":"2024-03-29T08:24:43.096002+0000","last_became_peered":"2024-03-29T08:24:43.096002+0000","last_unstale":"2024-03-29T08:24:43.491682+0000","last_undegraded":"2024-03-29T08:24:43.491682+0000","last_fullsized":"2024-03-29T08:24:43.491682+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:40:44.900886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988209+0000","last_change":"2024-03-29T08:24:43.496951+0000","last_active":"2024-03-29T08:24:44.988209+0000","last_peered":"2024-03-29T08:24:44.988209+0000","last_clean":"2024-03-29T08:24:44.988209+0000","last_became_active":"2024-03-29T08:24:43.496692+0000","last_became_peered":"2024-03-29T08:24:43.496692+0000","last_unstale":"2024-03-29T08:24:44.988209+0000","last_undegraded":"2024-03-29T08:24:44.988209+0000","last_fullsized":"2024-03-29T08:24:44.988209+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:38:17.620893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488103+0000","last_change":"2024-03-29T08:24:43.093087+0000","last_active":"2024-03-29T08:24:44.488103+0000","last_peered":"2024-03-29T08:24:44.488103+0000","last_clean":"2024-03-29T08:24:44.488103+0000","last_became_active":"2024-03-29T08:24:43.092804+0000","last_became_peered":"2024-03-29T08:24:43.092804+0000","last_unstale":"2024-03-29T08:24:44.488103+0000","last_undegraded":"2024-03-29T08:24:44.488103+0000","last_fullsized":"2024-03-29T08:24:44.488103+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:48:30.536411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861524+0000","last_change":"2024-03-29T08:24:43.497747+0000","last_active":"2024-03-29T08:24:44.861524+0000","last_peered":"2024-03-29T08:24:44.861524+0000","last_clean":"2024-03-29T08:24:44.861524+0000","last_became_active":"2024-03-29T08:24:43.497267+0000","last_became_peered":"2024-03-29T08:24:43.497267+0000","last_unstale":"2024-03-29T08:24:44.861524+0000","last_undegraded":"2024-03-29T08:24:44.861524+0000","last_fullsized":"2024-03-29T08:24:44.861524+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:24:39.002767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"91'2","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488841+0000","last_change":"2024-03-29T08:24:43.096961+0000","last_active":"2024-03-29T08:24:44.488841+0000","last_peered":"2024-03-29T08:24:44.488841+0000","last_clean":"2024-03-29T08:24:44.488841+0000","last_became_active":"2024-03-29T08:24:43.096657+0000","last_became_peered":"2024-03-29T08:24:43.096657+0000","last_unstale":"2024-03-29T08:24:44.488841+0000","last_undegraded":"2024-03-29T08:24:44.488841+0000","last_fullsized":"2024-03-29T08:24:44.488841+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:23:23.709515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861617+0000","last_change":"2024-03-29T08:24:43.491598+0000","last_active":"2024-03-29T08:24:44.861617+0000","last_peered":"2024-03-29T08:24:44.861617+0000","last_clean":"2024-03-29T08:24:44.861617+0000","last_became_active":"2024-03-29T08:24:43.490511+0000","last_became_peered":"2024-03-29T08:24:43.490511+0000","last_unstale":"2024-03-29T08:24:44.861617+0000","last_undegraded":"2024-03-29T08:24:44.861617+0000","last_fullsized":"2024-03-29T08:24:44.861617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:36:33.757108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.490786+0000","last_change":"2024-03-29T08:24:43.097839+0000","last_active":"2024-03-29T08:24:43.490786+0000","last_peered":"2024-03-29T08:24:43.490786+0000","last_clean":"2024-03-29T08:24:43.490786+0000","last_became_active":"2024-03-29T08:24:43.097511+0000","last_became_peered":"2024-03-29T08:24:43.097511+0000","last_unstale":"2024-03-29T08:24:43.490786+0000","last_undegraded":"2024-03-29T08:24:43.490786+0000","last_fullsized":"2024-03-29T08:24:43.490786+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:54.006723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861352+0000","last_change":"2024-03-29T08:24:43.505982+0000","last_active":"2024-03-29T08:24:44.861352+0000","last_peered":"2024-03-29T08:24:44.861352+0000","last_clean":"2024-03-29T08:24:44.861352+0000","last_became_active":"2024-03-29T08:24:43.505727+0000","last_became_peered":"2024-03-29T08:24:43.505727+0000","last_unstale":"2024-03-29T08:24:44.861352+0000","last_undegraded":"2024-03-29T08:24:44.861352+0000","last_fullsized":"2024-03-29T08:24:44.861352+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:15:42.713997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988265+0000","last_change":"2024-03-29T08:24:43.094281+0000","last_active":"2024-03-29T08:24:44.988265+0000","last_peered":"2024-03-29T08:24:44.988265+0000","last_clean":"2024-03-29T08:24:44.988265+0000","last_became_active":"2024-03-29T08:24:43.093927+0000","last_became_peered":"2024-03-29T08:24:43.093927+0000","last_unstale":"2024-03-29T08:24:44.988265+0000","last_undegraded":"2024-03-29T08:24:44.988265+0000","last_fullsized":"2024-03-29T08:24:44.988265+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:49:58.072830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483512+0000","last_change":"2024-03-29T08:24:43.506350+0000","last_active":"2024-03-29T08:24:44.483512+0000","last_peered":"2024-03-29T08:24:44.483512+0000","last_clean":"2024-03-29T08:24:44.483512+0000","last_became_active":"2024-03-29T08:24:43.506096+0000","last_became_peered":"2024-03-29T08:24:43.506096+0000","last_unstale":"2024-03-29T08:24:44.483512+0000","last_undegraded":"2024-03-29T08:24:44.483512+0000","last_fullsized":"2024-03-29T08:24:44.483512+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:18.006415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483314+0000","last_change":"2024-03-29T08:24:43.084143+0000","last_active":"2024-03-29T08:24:44.483314+0000","last_peered":"2024-03-29T08:24:44.483314+0000","last_clean":"2024-03-29T08:24:44.483314+0000","last_became_active":"2024-03-29T08:24:43.083892+0000","last_became_peered":"2024-03-29T08:24:43.083892+0000","last_unstale":"2024-03-29T08:24:44.483314+0000","last_undegraded":"2024-03-29T08:24:44.483314+0000","last_fullsized":"2024-03-29T08:24:44.483314+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:45:46.177771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.508509+0000","last_change":"2024-03-29T08:24:43.500600+0000","last_active":"2024-03-29T08:24:43.508509+0000","last_peered":"2024-03-29T08:24:43.508509+0000","last_clean":"2024-03-29T08:24:43.508509+0000","last_became_active":"2024-03-29T08:24:43.500338+0000","last_became_peered":"2024-03-29T08:24:43.500338+0000","last_unstale":"2024-03-29T08:24:43.508509+0000","last_undegraded":"2024-03-29T08:24:43.508509+0000","last_fullsized":"2024-03-29T08:24:43.508509+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:08:12.210637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.487947+0000","last_change":"2024-03-29T08:24:43.094324+0000","last_active":"2024-03-29T08:24:44.487947+0000","last_peered":"2024-03-29T08:24:44.487947+0000","last_clean":"2024-03-29T08:24:44.487947+0000","last_became_active":"2024-03-29T08:24:43.093514+0000","last_became_peered":"2024-03-29T08:24:43.093514+0000","last_unstale":"2024-03-29T08:24:44.487947+0000","last_undegraded":"2024-03-29T08:24:44.487947+0000","last_fullsized":"2024-03-29T08:24:44.487947+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:35.272690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.508825+0000","last_change":"2024-03-29T08:24:43.502050+0000","last_active":"2024-03-29T08:24:43.508825+0000","last_peered":"2024-03-29T08:24:43.508825+0000","last_clean":"2024-03-29T08:24:43.508825+0000","last_became_active":"2024-03-29T08:24:43.501547+0000","last_became_peered":"2024-03-29T08:24:43.501547+0000","last_unstale":"2024-03-29T08:24:43.508825+0000","last_undegraded":"2024-03-29T08:24:43.508825+0000","last_fullsized":"2024-03-29T08:24:43.508825+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:11.820047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483237+0000","last_change":"2024-03-29T08:24:43.091822+0000","last_active":"2024-03-29T08:24:44.483237+0000","last_peered":"2024-03-29T08:24:44.483237+0000","last_clean":"2024-03-29T08:24:44.483237+0000","last_became_active":"2024-03-29T08:24:43.091415+0000","last_became_peered":"2024-03-29T08:24:43.091415+0000","last_unstale":"2024-03-29T08:24:44.483237+0000","last_undegraded":"2024-03-29T08:24:44.483237+0000","last_fullsized":"2024-03-29T08:24:44.483237+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:35.731754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.510354+0000","last_change":"2024-03-29T08:24:43.506043+0000","last_active":"2024-03-29T08:24:43.510354+0000","last_peered":"2024-03-29T08:24:43.510354+0000","last_clean":"2024-03-29T08:24:43.510354+0000","last_became_active":"2024-03-29T08:24:43.505810+0000","last_became_peered":"2024-03-29T08:24:43.505810+0000","last_unstale":"2024-03-29T08:24:43.510354+0000","last_undegraded":"2024-03-29T08:24:43.510354+0000","last_fullsized":"2024-03-29T08:24:43.510354+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:42:03.393961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.485889+0000","last_change":"2024-03-29T08:24:43.097191+0000","last_active":"2024-03-29T08:24:43.485889+0000","last_peered":"2024-03-29T08:24:43.485889+0000","last_clean":"2024-03-29T08:24:43.485889+0000","last_became_active":"2024-03-29T08:24:43.096911+0000","last_became_peered":"2024-03-29T08:24:43.096911+0000","last_unstale":"2024-03-29T08:24:43.485889+0000","last_undegraded":"2024-03-29T08:24:43.485889+0000","last_fullsized":"2024-03-29T08:24:43.485889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:37:46.182612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.509326+0000","last_change":"2024-03-29T08:24:43.502744+0000","last_active":"2024-03-29T08:24:43.509326+0000","last_peered":"2024-03-29T08:24:43.509326+0000","last_clean":"2024-03-29T08:24:43.509326+0000","last_became_active":"2024-03-29T08:24:43.502448+0000","last_became_peered":"2024-03-29T08:24:43.502448+0000","last_unstale":"2024-03-29T08:24:43.509326+0000","last_undegraded":"2024-03-29T08:24:43.509326+0000","last_fullsized":"2024-03-29T08:24:43.509326+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:20:39.885295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988106+0000","last_change":"2024-03-29T08:24:43.094957+0000","last_active":"2024-03-29T08:24:44.988106+0000","last_peered":"2024-03-29T08:24:44.988106+0000","last_clean":"2024-03-29T08:24:44.988106+0000","last_became_active":"2024-03-29T08:24:43.094627+0000","last_became_peered":"2024-03-29T08:24:43.094627+0000","last_unstale":"2024-03-29T08:24:44.988106+0000","last_undegraded":"2024-03-29T08:24:44.988106+0000","last_fullsized":"2024-03-29T08:24:44.988106+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:40:44.900886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.515559+0000","last_change":"2024-03-29T08:24:43.511382+0000","last_active":"2024-03-29T08:24:43.515559+0000","last_peered":"2024-03-29T08:24:43.515559+0000","last_clean":"2024-03-29T08:24:43.515559+0000","last_became_active":"2024-03-29T08:24:43.511081+0000","last_became_peered":"2024-03-29T08:24:43.511081+0000","last_unstale":"2024-03-29T08:24:43.515559+0000","last_undegraded":"2024-03-29T08:24:43.515559+0000","last_fullsized":"2024-03-29T08:24:43.515559+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:52.482922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"91'1","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861498+0000","last_change":"2024-03-29T08:24:43.097252+0000","last_active":"2024-03-29T08:24:44.861498+0000","last_peered":"2024-03-29T08:24:44.861498+0000","last_clean":"2024-03-29T08:24:44.861498+0000","last_became_active":"2024-03-29T08:24:43.096977+0000","last_became_peered":"2024-03-29T08:24:43.096977+0000","last_unstale":"2024-03-29T08:24:44.861498+0000","last_undegraded":"2024-03-29T08:24:44.861498+0000","last_fullsized":"2024-03-29T08:24:44.861498+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:20:04.792923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":224,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.083443+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:24:43.083443+0000","last_peered":"2024-03-29T08:24:43.083443+0000","last_clean":"2024-03-29T08:24:43.083443+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:24:43.083443+0000","last_undegraded":"2024-03-29T08:24:43.083443+0000","last_fullsized":"2024-03-29T08:24:43.083443+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509524,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1114112,"data_stored":474866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72899999999999998,"15min":0.66900000000000004},"min":{"1min":0.504,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.97999999999999998,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75800000000000001,"15min":0.69499999999999995},"min":{"1min":0.58299999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70899999999999996,"15min":0.66100000000000003},"min":{"1min":0.55700000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88800000000000001,"5min":1.202,"15min":1.202},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72299999999999998,"15min":0.65200000000000002},"min":{"1min":0.48799999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.93400000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.72199999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71399999999999997,"15min":0.65000000000000002},"min":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.503},"max":{"1min":0.93600000000000005,"5min":1.085,"15min":1.085},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74399999999999999,"15min":0.68200000000000005},"min":{"1min":0.56899999999999995,"5min":0.53200000000000003,"15min":0.48699999999999999},"max":{"1min":0.87,"5min":1.288,"15min":1.288},"last":0.68300000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72899999999999998,"15min":0.70399999999999996},"min":{"1min":0.55900000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98099999999999998,"5min":1.018,"15min":1.018},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.754,"15min":0.73399999999999999},"min":{"1min":0.57499999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94599999999999995,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.76900000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59299999999999997,"15min":0.621},"min":{"1min":0.45000000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81699999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.625,"15min":0.69399999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.755,"5min":1.0009999999999999,"15min":1.123},"last":0.52800000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63900000000000001,"15min":0.69099999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.76100000000000001,"5min":1.056,"15min":1.056},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66100000000000003,"15min":0.68999999999999995},"min":{"1min":0.46999999999999997,"5min":0.44800000000000001,"15min":0.39500000000000002},"max":{"1min":0.81599999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.752}]},{"osd":6,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.70799999999999996,"15min":0.72099999999999997},"min":{"1min":0.49199999999999999,"5min":0.48999999999999999,"15min":0.46600000000000003},"max":{"1min":0.94099999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67600000000000005,"15min":0.73199999999999998},"min":{"1min":0.47499999999999998,"5min":0.46999999999999997,"15min":0.44900000000000001},"max":{"1min":0.89200000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.70399999999999996}]}]},{"osd":6,"up_from":15,"seq":64424509524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1118208,"data_stored":475468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.70899999999999996,"15min":0.64900000000000002},"min":{"1min":0.59799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86499999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.747,"15min":0.69899999999999995},"min":{"1min":0.59399999999999997,"5min":0.52900000000000003,"15min":0.44500000000000001},"max":{"1min":0.95599999999999996,"5min":1.079,"15min":1.079},"last":0.75900000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67300000000000004,"15min":0.625},"min":{"1min":0.45300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.97999999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69099999999999995,"15min":0.64100000000000001},"min":{"1min":0.442,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":0.88700000000000001,"5min":1.079,"15min":1.079},"last":0.89100000000000001}]},{"osd":2,"last update":"Fri Mar 29 08:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75700000000000001,"15min":0.69299999999999995},"min":{"1min":0.51800000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73999999999999999,"15min":0.71199999999999997},"min":{"1min":0.52900000000000003,"5min":0.435,"15min":0.435},"max":{"1min":0.90700000000000003,"5min":0.93200000000000005,"15min":0.95899999999999996},"last":0.91600000000000004}]},{"osd":3,"last update":"Fri Mar 29 08:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71599999999999997,"15min":0.70099999999999996},"min":{"1min":0.503,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.99399999999999999,"5min":1.109,"15min":1.109},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70999999999999996,"15min":0.69699999999999995},"min":{"1min":0.53600000000000003,"5min":0.50700000000000001,"15min":0.47799999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61399999999999999,"15min":0.65900000000000003},"min":{"1min":0.38400000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.98199999999999998,"5min":1.0620000000000001,"15min":1.556},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63,"15min":0.71199999999999997},"min":{"1min":0.44700000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.84299999999999997,"5min":1.0429999999999999,"15min":2.0539999999999998},"last":0.996}]},{"osd":5,"last update":"Fri Mar 29 08:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.626,"15min":0.68600000000000005},"min":{"1min":0.46400000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.872,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.71799999999999997},"min":{"1min":0.47099999999999997,"5min":0.37,"15min":0.37},"max":{"1min":0.93799999999999994,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.0189999999999999}]},{"osd":7,"last update":"Fri Mar 29 08:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.747},"min":{"1min":0.502,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68700000000000006,"15min":0.75700000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1,"5min":1.109,"15min":1.109},"last":0.79600000000000004}]}]},{"osd":4,"up_from":15,"seq":64424509523,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1126400,"data_stored":481130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69599999999999995,"15min":0.70599999999999996},"min":{"1min":0.47799999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92500000000000004,"5min":1.0289999999999999,"15min":2.1709999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70199999999999996,"15min":0.73599999999999999},"min":{"1min":0.56499999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95699999999999996,"5min":1.0489999999999999,"15min":2.2869999999999999},"last":0.63700000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.745,"15min":0.76100000000000001},"min":{"1min":0.53700000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90800000000000003,"5min":1.087,"15min":2.524},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.69299999999999995,"15min":0.71299999999999997},"min":{"1min":0.53100000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":2.2679999999999998},"last":0.69799999999999995}]},{"osd":2,"last update":"Fri Mar 29 08:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71399999999999997,"15min":0.72999999999999998},"min":{"1min":0.50900000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93799999999999994,"5min":1.1060000000000001,"15min":2.363},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.753,"15min":0.75800000000000001},"min":{"1min":0.44400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.97599999999999998,"5min":1.0029999999999999,"15min":2.1880000000000002},"last":0.78600000000000003}]},{"osd":3,"last update":"Fri Mar 29 08:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73899999999999999,"15min":0.754},"min":{"1min":0.57499999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.004,"5min":1.004,"15min":2.3029999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78500000000000003,"15min":0.80900000000000005},"min":{"1min":0.66600000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93899999999999995,"5min":1.0700000000000001,"15min":2.2549999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.63900000000000001,"15min":0.68400000000000005},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83599999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.63600000000000001,"15min":0.64800000000000002},"min":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":1.0149999999999999},"last":0.49199999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64500000000000002,"15min":0.66900000000000004},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93100000000000005,"5min":1.016,"15min":1.016},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.70099999999999996,"15min":0.73599999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.5},"max":{"1min":0.82299999999999995,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.58399999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68500000000000005,"15min":0.72699999999999998},"min":{"1min":0.55600000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.874,"5min":0.96399999999999997,"15min":1.1000000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73399999999999999,"15min":0.76100000000000001},"min":{"1min":0.56200000000000006,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96999999999999997,"5min":1.0760000000000001,"15min":1.4430000000000001},"last":0.61599999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509526,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1122304,"data_stored":475470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:24:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.44600000000000001,"15min":0.46000000000000002},"min":{"1min":0.29299999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68300000000000005,"5min":0.84799999999999998,"15min":1.123},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.52400000000000002,"15min":0.52500000000000002},"min":{"1min":0.438,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88100000000000001,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.36199999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:24:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44,"15min":0.45800000000000002},"min":{"1min":0.33300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71899999999999997,"5min":0.78900000000000003,"15min":1.1599999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.499,"15min":0.52900000000000003},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.73999999999999999,"5min":1.004,"15min":1.4790000000000001},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Mar 29 08:24:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.52700000000000002,"15min":0.56299999999999994},"min":{"1min":0.45400000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.73399999999999999,"5min":0.90000000000000002,"15min":1.2529999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60199999999999998,"15min":0.64800000000000002},"min":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.86599999999999999,"5min":0.97699999999999998,"15min":1.8580000000000001},"last":0.57599999999999996}]},{"osd":4,"last update":"Fri Mar 29 08:24:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.74399999999999999,"15min":0.78600000000000003},"min":{"1min":0.57899999999999996,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0149999999999999,"5min":1.1950000000000001,"15min":1.6419999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.746,"15min":0.77500000000000002},"min":{"1min":0.56999999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.391},"last":0.86799999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:24:15 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.78200000000000003,"15min":0.80400000000000005},"min":{"1min":0.59799999999999998,"5min":0.56999999999999995,"15min":0.52100000000000002},"max":{"1min":1.1619999999999999,"5min":1.671,"15min":1.671},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.875,"5min":0.80100000000000005,"15min":0.85299999999999998},"min":{"1min":0.68600000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2050000000000001,"5min":1.6599999999999999,"15min":1.7110000000000001},"last":0.71399999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:24:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.746,"15min":0.81899999999999995},"min":{"1min":0.56200000000000006,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.669},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.75700000000000001,"15min":0.81699999999999995},"min":{"1min":0.624,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.073,"5min":1.212,"15min":2.0150000000000001},"last":0.66600000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:24:15 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.77900000000000003,"15min":0.84699999999999998},"min":{"1min":0.63100000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3169999999999999,"5min":1.4279999999999999,"15min":1.887},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.76400000000000001,"15min":0.84199999999999997},"min":{"1min":0.59599999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":2.044},"last":0.82499999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509525,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1110016,"data_stored":474796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.46300000000000002,"15min":0.49299999999999999},"min":{"1min":0.33400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66200000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55000000000000004,"15min":0.51500000000000001},"min":{"1min":0.41699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.874,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.44500000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.504,"15min":0.5},"min":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73499999999999999,"5min":1.399,"15min":1.399},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.503,"15min":0.51200000000000001},"min":{"1min":0.32300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77300000000000002,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.379}]},{"osd":3,"last update":"Fri Mar 29 08:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.50700000000000001},"min":{"1min":0.38,"5min":0.35099999999999998,"15min":0.34000000000000002},"max":{"1min":0.93300000000000005,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.623,"15min":0.59099999999999997},"min":{"1min":0.48599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85099999999999998,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.496}]},{"osd":4,"last update":"Fri Mar 29 08:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.78600000000000003,"15min":0.77000000000000002},"min":{"1min":0.53000000000000003,"5min":0.52600000000000002,"15min":0.49099999999999999},"max":{"1min":1.0660000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.52800000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.0840000000000001,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.57199999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.82599999999999996,"15min":0.84099999999999997},"min":{"1min":0.66900000000000004,"5min":0.59399999999999997,"15min":0.55500000000000005},"max":{"1min":1.236,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.80100000000000005,"15min":0.84699999999999998},"min":{"1min":0.57899999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.194,"5min":1.28,"15min":1.526},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58799999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1080000000000001,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.77900000000000003,"15min":0.79100000000000004},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.52500000000000002},"max":{"1min":1.157,"5min":1.524,"15min":1.524},"last":0.69699999999999995}]},{"osd":7,"last update":"Fri Mar 29 08:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.78100000000000003,"15min":0.80200000000000005},"min":{"1min":0.66200000000000003,"5min":0.51000000000000001,"15min":0.505},"max":{"1min":1.296,"5min":1.319,"15min":1.319},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.64800000000000002,"5min":0.55600000000000005,"15min":0.54900000000000004},"max":{"1min":1.1719999999999999,"5min":2,"15min":2},"last":0.58699999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509525,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1708032,"data_stored":1065254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.438,"15min":0.59199999999999997},"min":{"1min":0.32900000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":4.2539999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.49399999999999999,"15min":0.61799999999999999},"min":{"1min":0.376,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69999999999999996,"5min":0.84299999999999997,"15min":4.1909999999999998},"last":0.34899999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48899999999999999,"15min":0.505},"min":{"1min":0.42699999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70699999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53100000000000003,"15min":0.53600000000000003},"min":{"1min":0.34599999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":0.878,"15min":0.88200000000000001},"last":0.40999999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59399999999999997,"15min":0.58499999999999996},"min":{"1min":0.48199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83299999999999996,"5min":1.52,"15min":1.52},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.42499999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73399999999999999,"5min":1.331,"15min":1.331},"last":0.42099999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.53800000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0580000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.875,"5min":0.751,"15min":0.78500000000000003},"min":{"1min":0.69399999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.175,"5min":1.175,"15min":1.23},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Mar 29 08:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.76000000000000001,"15min":0.77800000000000002},"min":{"1min":0.55800000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0449999999999999,"5min":1.093,"15min":1.1499999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.76200000000000001,"15min":0.76400000000000001},"min":{"1min":0.64600000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.99099999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Mar 29 08:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.77100000000000002,"15min":0.77200000000000002},"min":{"1min":0.64100000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.145,"5min":1.212,"15min":1.212},"last":0.624},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.78000000000000003,"15min":0.84399999999999997},"min":{"1min":0.66000000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.115,"5min":1.1459999999999999,"15min":1.196},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.65900000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1200000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.77500000000000002,"15min":0.79000000000000004},"min":{"1min":0.625,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.071,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.59699999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509524,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1138688,"data_stored":482682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42899999999999999,"15min":0.57199999999999995},"min":{"1min":0.316,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.57299999999999995,"5min":0.71099999999999997,"15min":3.9359999999999999},"last":0.314},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.78500000000000003},"min":{"1min":0.33100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.69699999999999995,"5min":0.73799999999999999,"15min":8.0419999999999998},"last":0.45700000000000002}]},{"osd":2,"last update":"Fri Mar 29 08:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.501,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":0.73399999999999999,"15min":1.258},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.52800000000000002,"15min":0.69799999999999995},"min":{"1min":0.48399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.75700000000000001,"5min":0.78700000000000003,"15min":4.907},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Mar 29 08:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.49299999999999999,"15min":0.78500000000000003},"min":{"1min":0.39000000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68400000000000005,"5min":0.91000000000000003,"15min":7.9569999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55400000000000005,"15min":0.64200000000000002},"min":{"1min":0.48099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77500000000000002,"5min":0.78300000000000003,"15min":1.397},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.77700000000000002,"15min":0.95399999999999996},"min":{"1min":0.63800000000000001,"5min":0.55100000000000005,"15min":0.53700000000000003},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":4.4720000000000004},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.754,"15min":0.92700000000000005},"min":{"1min":0.59899999999999998,"5min":0.498,"15min":0.498},"max":{"1min":1.115,"5min":1.115,"15min":4.3920000000000003},"last":0.83099999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.77500000000000002,"15min":0.90500000000000003},"min":{"1min":0.61599999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.349,"5min":1.349,"15min":4.3200000000000003},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.77400000000000002,"15min":0.85099999999999998},"min":{"1min":0.63900000000000001,"5min":0.55700000000000005,"15min":0.54600000000000004},"max":{"1min":1.26,"5min":1.26,"15min":1.6759999999999999},"last":0.65200000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.78400000000000003,"15min":0.85199999999999998},"min":{"1min":0.59399999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.5920000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.79200000000000004,"15min":0.95399999999999996},"min":{"1min":0.67800000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":4.4100000000000001},"last":0.71099999999999997}]},{"osd":7,"last update":"Fri Mar 29 08:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.79600000000000004,"15min":0.97099999999999997},"min":{"1min":0.70799999999999996,"5min":0.56999999999999995,"15min":0.496},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":4.5209999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.78500000000000003,"15min":0.95499999999999996},"min":{"1min":0.65500000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":4.4420000000000002},"last":0.69299999999999995}]}]},{"osd":5,"up_from":14,"seq":60129542229,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1720320,"data_stored":1066762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.68200000000000005,"15min":0.64500000000000002},"min":{"1min":0.49099999999999999,"5min":0.45000000000000001,"15min":0.42899999999999999},"max":{"1min":0.879,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.68799999999999994,"15min":0.66400000000000003},"min":{"1min":0.622,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67400000000000004,"15min":0.72399999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":2.5129999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.70199999999999996,"15min":0.68500000000000005},"min":{"1min":0.53400000000000003,"5min":0.44900000000000001,"15min":0.40500000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.78200000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67900000000000005,"15min":0.67500000000000004},"min":{"1min":0.51700000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70999999999999996,"15min":0.70599999999999996},"min":{"1min":0.58099999999999996,"5min":0.44900000000000001,"15min":0.39200000000000002},"max":{"1min":0.86499999999999999,"5min":0.91400000000000003,"15min":1.0209999999999999},"last":0.59899999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.71899999999999997,"15min":0.71699999999999997},"min":{"1min":0.54600000000000004,"5min":0.49399999999999999,"15min":0.47099999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.99399999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71099999999999997,"15min":0.73899999999999999},"min":{"1min":0.51900000000000002,"5min":0.47799999999999998,"15min":0.435},"max":{"1min":0.86899999999999999,"5min":0.90500000000000003,"15min":1.954},"last":0.66300000000000003}]},{"osd":4,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63200000000000001,"15min":0.64700000000000002},"min":{"1min":0.46899999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.71999999999999997,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.65500000000000003},"min":{"1min":0.45600000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":1.0820000000000001},"last":0.74099999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.628,"15min":0.66900000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83399999999999996,"5min":0.86899999999999999,"15min":1.071},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67200000000000004,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.878,"15min":1.0609999999999999},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67600000000000005,"15min":0.70899999999999996},"min":{"1min":0.51200000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90100000000000002,"5min":0.92900000000000005,"15min":1.1200000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71299999999999997,"15min":0.749},"min":{"1min":0.497,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93100000000000005,"5min":0.95199999999999996,"15min":1.04},"last":0.85199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:24:47.595 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:24:47.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.769+0000 7f6909400640 1 Processor -- start 2024-03-29T08:24:47.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.769+0000 7f6909400640 1 -- start start 2024-03-29T08:24:47.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6909400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f690415c9d0 0x7f69040cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6909400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69040d0030 con 0x7f6904099150 2024-03-29T08:24:47.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6909400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69040cf420 con 0x7f69040cef60 2024-03-29T08:24:47.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6909400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69040cf700 con 0x7f690415c9d0 2024-03-29T08:24:47.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6903200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6904099150 0x7f6904166a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42080/0 (socket says 172.21.15.46:42080) 2024-03-29T08:24:47.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6903200640 1 -- 172.21.15.46:0/3122542394 learned_addr learned my addr 172.21.15.46:0/3122542394 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:47.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6902800640 1 --2- 172.21.15.46:0/3122542394 >> v2:172.21.15.73:3300/0 conn(0x7f690415c9d0 0x7f69040cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6901e00640 1 -- 172.21.15.46:0/3122542394 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 837782163 0 0) 0x7f69040d0030 con 0x7f6904099150 2024-03-29T08:24:47.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6901e00640 1 -- 172.21.15.46:0/3122542394 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68e8003530 con 0x7f6904099150 2024-03-29T08:24:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6901e00640 1 -- 172.21.15.46:0/3122542394 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3692668623 0 0) 0x7f69040cf420 con 0x7f69040cef60 2024-03-29T08:24:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6901e00640 1 -- 172.21.15.46:0/3122542394 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69040d0030 con 0x7f69040cef60 2024-03-29T08:24:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6901e00640 1 -- 172.21.15.46:0/3122542394 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1101494523 0 0) 0x7f68e8003530 con 0x7f6904099150 2024-03-29T08:24:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6901e00640 1 -- 172.21.15.46:0/3122542394 >> v2:172.21.15.73:3300/0 conn(0x7f690415c9d0 msgr2=0x7f69040cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6901e00640 1 --2- 172.21.15.46:0/3122542394 >> v2:172.21.15.73:3300/0 conn(0x7f690415c9d0 0x7f69040cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6901e00640 1 -- 172.21.15.46:0/3122542394 >> v1:172.21.15.73:6789/0 conn(0x7f69040cef60 legacy=0x7f6904098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6901e00640 1 -- 172.21.15.46:0/3122542394 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69040cf9e0 con 0x7f6904099150 2024-03-29T08:24:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6901e00640 1 -- 172.21.15.46:0/3122542394 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f68fc001980 con 0x7f6904099150 2024-03-29T08:24:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6901e00640 1 -- 172.21.15.46:0/3122542394 >> v1:172.21.15.46:6789/0 conn(0x7f6904099150 legacy=0x7f6904166a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6901e00640 1 --2- 172.21.15.46:0/3122542394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e8003b60 0x7f68e8003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6901e00640 1 --2- 172.21.15.46:0/3122542394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68e80047d0 0x7f68e8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6901e00640 1 --2- 172.21.15.46:0/3122542394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e8005190 0x7f68e800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6901e00640 1 -- 172.21.15.46:0/3122542394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69040cf700 con 0x7f68e8003b60 2024-03-29T08:24:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6901e00640 1 -- 172.21.15.46:0/3122542394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e800c060 con 0x7f68e80047d0 2024-03-29T08:24:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6901e00640 1 -- 172.21.15.46:0/3122542394 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e800c220 con 0x7f68e8005190 2024-03-29T08:24:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6902800640 1 --2- 172.21.15.46:0/3122542394 >> v2:172.21.15.73:3300/0 conn(0x7f690415c9d0 0x7f69040cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:24:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6902800640 1 --2- 172.21.15.46:0/3122542394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e8003b60 0x7f68e8003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f68f9e00640 1 --2- 172.21.15.46:0/3122542394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68e80047d0 0x7f68e8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6903200640 1 --2- 172.21.15.46:0/3122542394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e8005190 0x7f68e800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6902800640 1 -- 172.21.15.46:0/3122542394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e8005190 msgr2=0x7f68e800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6902800640 1 --2- 172.21.15.46:0/3122542394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e8005190 0x7f68e800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6902800640 1 -- 172.21.15.46:0/3122542394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68e80047d0 msgr2=0x7f68e8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6902800640 1 --2- 172.21.15.46:0/3122542394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68e80047d0 0x7f68e8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6902800640 1 -- 172.21.15.46:0/3122542394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e8003530 con 0x7f68e8003b60 2024-03-29T08:24:47.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.773+0000 7f6903200640 1 -- 172.21.15.46:0/3122542394 reap_dead start 2024-03-29T08:24:47.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.777+0000 7f6902800640 1 --2- 172.21.15.46:0/3122542394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e8003b60 0x7f68e8003f60 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009c90 tx=0x7f68f401cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a96f3000e175d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:47.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.777+0000 7f6901e00640 1 -- 172.21.15.46:0/3122542394 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4004020 con 0x7f68e8003b60 2024-03-29T08:24:47.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.777+0000 7f6901e00640 1 -- 172.21.15.46:0/3122542394 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f4004180 con 0x7f68e8003b60 2024-03-29T08:24:47.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.777+0000 7f6901e00640 1 -- 172.21.15.46:0/3122542394 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f40a5880 con 0x7f68e8003b60 2024-03-29T08:24:47.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.777+0000 7f6909400640 1 -- 172.21.15.46:0/3122542394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e8003b60 msgr2=0x7f68e8003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.777+0000 7f6909400640 1 --2- 172.21.15.46:0/3122542394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e8003b60 0x7f68e8003f60 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009c90 tx=0x7f68f401cbc0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.777+0000 7f6909400640 1 -- 172.21.15.46:0/3122542394 shutdown_connections 2024-03-29T08:24:47.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.777+0000 7f6909400640 1 --2- 172.21.15.46:0/3122542394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e8003b60 0x7f68e8003f60 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.777+0000 7f6909400640 1 -- 172.21.15.46:0/3122542394 >> 172.21.15.46:0/3122542394 conn(0x7f69040b21a0 msgr2=0x7f69040b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:47.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.777+0000 7f6909400640 1 -- 172.21.15.46:0/3122542394 shutdown_connections 2024-03-29T08:24:47.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.777+0000 7f6909400640 1 -- 172.21.15.46:0/3122542394 wait complete. 2024-03-29T08:24:47.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.777+0000 7f6909400640 1 Processor -- start 2024-03-29T08:24:47.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.777+0000 7f6909400640 1 -- start start 2024-03-29T08:24:47.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6909400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e8003b60 0x7f690414f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6909400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69040b48b0 0x7f69040c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6909400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69040c6d40 0x7f69040b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6909400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69040b4420 con 0x7f69040c6d40 2024-03-29T08:24:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6909400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69040b3550 con 0x7f69040b48b0 2024-03-29T08:24:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6909400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69040b3810 con 0x7f68e8003b60 2024-03-29T08:24:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6902800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e8003b60 0x7f690414f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6903200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69040c6d40 0x7f69040b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6902800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e8003b60 0x7f690414f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:35728/0 (socket says 172.21.15.46:35728) 2024-03-29T08:24:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6902800640 1 -- 172.21.15.46:0/803935948 learned_addr learned my addr 172.21.15.46:0/803935948 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f68f9e00640 1 --2- 172.21.15.46:0/803935948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69040b48b0 0x7f69040c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6903200640 1 -- 172.21.15.46:0/803935948 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e8003b60 msgr2=0x7f690414f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6903200640 1 --2- 172.21.15.46:0/803935948 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e8003b60 0x7f690414f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6903200640 1 -- 172.21.15.46:0/803935948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69040b48b0 msgr2=0x7f69040c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6903200640 1 --2- 172.21.15.46:0/803935948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69040b48b0 0x7f69040c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6903200640 1 -- 172.21.15.46:0/803935948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69040b3af0 con 0x7f69040c6d40 2024-03-29T08:24:47.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6903200640 1 --2- 172.21.15.46:0/803935948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69040c6d40 0x7f69040b5610 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6904071690 tx=0x7f68fc00f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:47.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f68fb400640 1 -- 172.21.15.46:0/803935948 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc007af0 con 0x7f69040c6d40 2024-03-29T08:24:47.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f68fb400640 1 -- 172.21.15.46:0/803935948 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68fc00fd30 con 0x7f69040c6d40 2024-03-29T08:24:47.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f68fb400640 1 -- 172.21.15.46:0/803935948 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc013630 con 0x7f69040c6d40 2024-03-29T08:24:47.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6909400640 1 -- 172.21.15.46:0/803935948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69040b3dd0 con 0x7f69040c6d40 2024-03-29T08:24:47.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6909400640 1 -- 172.21.15.46:0/803935948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690414f960 con 0x7f69040c6d40 2024-03-29T08:24:47.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.781+0000 7f6909400640 1 -- 172.21.15.46:0/803935948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6904066c90 con 0x7f69040c6d40 2024-03-29T08:24:47.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.785+0000 7f68fb400640 1 -- 172.21.15.46:0/803935948 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f68fc013790 con 0x7f69040c6d40 2024-03-29T08:24:47.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.785+0000 7f68fb400640 1 --2- 172.21.15.46:0/803935948 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68e00666c0 0x7f68e0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:47.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.785+0000 7f68fb400640 1 -- 172.21.15.46:0/803935948 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68fc084360 con 0x7f69040c6d40 2024-03-29T08:24:47.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.785+0000 7f6902800640 1 --2- 172.21.15.46:0/803935948 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68e00666c0 0x7f68e0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:47.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.785+0000 7f6902800640 1 --2- 172.21.15.46:0/803935948 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68e00666c0 0x7f68e0068b80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f68f401cc00 tx=0x7f68f40046d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:47.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.789+0000 7f68fb400640 1 -- 172.21.15.46:0/803935948 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f68fc051770 con 0x7f69040c6d40 2024-03-29T08:24:47.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.945+0000 7f6909400640 1 -- 172.21.15.46:0/803935948 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f690414fb50 con 0x7f68e00666c0 2024-03-29T08:24:47.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.953+0000 7f68fb400640 1 -- 172.21.15.46:0/803935948 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289414 (secure 0 0 0) 0x7f68f40044b0 con 0x7f68e00666c0 2024-03-29T08:24:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:47.956 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:24:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.953+0000 7f6909400640 1 -- 172.21.15.46:0/803935948 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68e00666c0 msgr2=0x7f68e0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.953+0000 7f6909400640 1 --2- 172.21.15.46:0/803935948 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68e00666c0 0x7f68e0068b80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f68f401cc00 tx=0x7f68f40046d0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.953+0000 7f6909400640 1 -- 172.21.15.46:0/803935948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69040c6d40 msgr2=0x7f69040b5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.953+0000 7f6909400640 1 --2- 172.21.15.46:0/803935948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69040c6d40 0x7f69040b5610 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f6904071690 tx=0x7f68fc00f5a0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.957+0000 7f6909400640 1 -- 172.21.15.46:0/803935948 shutdown_connections 2024-03-29T08:24:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.957+0000 7f6909400640 1 --2- 172.21.15.46:0/803935948 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68e00666c0 0x7f68e0068b80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.957+0000 7f6909400640 1 --2- 172.21.15.46:0/803935948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69040b48b0 0x7f69040c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.957+0000 7f6909400640 1 --2- 172.21.15.46:0/803935948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69040c6d40 0x7f69040b5610 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.957+0000 7f6909400640 1 --2- 172.21.15.46:0/803935948 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e8003b60 0x7f690414f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.957+0000 7f6909400640 1 -- 172.21.15.46:0/803935948 >> 172.21.15.46:0/803935948 conn(0x7f69040b21a0 msgr2=0x7f690415bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.957+0000 7f6909400640 1 -- 172.21.15.46:0/803935948 shutdown_connections 2024-03-29T08:24:47.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:47.957+0000 7f6909400640 1 -- 172.21.15.46:0/803935948 wait complete. 2024-03-29T08:24:47.975 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-03-29T08:24:46.140418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258896,"kb_used_data":9920,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754715824,"statfs":{"total":773094113280,"available":772829003776,"internally_reserved":0,"allocated":10158080,"data_stored":4996428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.065682"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.508588+0000","last_change":"2024-03-29T08:24:43.500863+0000","last_active":"2024-03-29T08:24:43.508588+0000","last_peered":"2024-03-29T08:24:43.508588+0000","last_clean":"2024-03-29T08:24:43.508588+0000","last_became_active":"2024-03-29T08:24:43.500282+0000","last_became_peered":"2024-03-29T08:24:43.500282+0000","last_unstale":"2024-03-29T08:24:43.508588+0000","last_undegraded":"2024-03-29T08:24:43.508588+0000","last_fullsized":"2024-03-29T08:24:43.508588+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:57:37.160390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488930+0000","last_change":"2024-03-29T08:24:43.507649+0000","last_active":"2024-03-29T08:24:44.488930+0000","last_peered":"2024-03-29T08:24:44.488930+0000","last_clean":"2024-03-29T08:24:44.488930+0000","last_became_active":"2024-03-29T08:24:43.507394+0000","last_became_peered":"2024-03-29T08:24:43.507394+0000","last_unstale":"2024-03-29T08:24:44.488930+0000","last_undegraded":"2024-03-29T08:24:44.488930+0000","last_fullsized":"2024-03-29T08:24:44.488930+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:13:35.362060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483415+0000","last_change":"2024-03-29T08:24:43.498577+0000","last_active":"2024-03-29T08:24:44.483415+0000","last_peered":"2024-03-29T08:24:44.483415+0000","last_clean":"2024-03-29T08:24:44.483415+0000","last_became_active":"2024-03-29T08:24:43.498317+0000","last_became_peered":"2024-03-29T08:24:43.498317+0000","last_unstale":"2024-03-29T08:24:44.483415+0000","last_undegraded":"2024-03-29T08:24:44.483415+0000","last_fullsized":"2024-03-29T08:24:44.483415+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:59:42.492346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.517817+0000","last_change":"2024-03-29T08:24:43.515036+0000","last_active":"2024-03-29T08:24:43.517817+0000","last_peered":"2024-03-29T08:24:43.517817+0000","last_clean":"2024-03-29T08:24:43.517817+0000","last_became_active":"2024-03-29T08:24:43.512474+0000","last_became_peered":"2024-03-29T08:24:43.512474+0000","last_unstale":"2024-03-29T08:24:43.517817+0000","last_undegraded":"2024-03-29T08:24:43.517817+0000","last_fullsized":"2024-03-29T08:24:43.517817+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:07:26.673863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861448+0000","last_change":"2024-03-29T08:24:43.504186+0000","last_active":"2024-03-29T08:24:44.861448+0000","last_peered":"2024-03-29T08:24:44.861448+0000","last_clean":"2024-03-29T08:24:44.861448+0000","last_became_active":"2024-03-29T08:24:43.503943+0000","last_became_peered":"2024-03-29T08:24:43.503943+0000","last_unstale":"2024-03-29T08:24:44.861448+0000","last_undegraded":"2024-03-29T08:24:44.861448+0000","last_fullsized":"2024-03-29T08:24:44.861448+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:04:07.876026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483168+0000","last_change":"2024-03-29T08:24:43.511703+0000","last_active":"2024-03-29T08:24:44.483168+0000","last_peered":"2024-03-29T08:24:44.483168+0000","last_clean":"2024-03-29T08:24:44.483168+0000","last_became_active":"2024-03-29T08:24:43.511475+0000","last_became_peered":"2024-03-29T08:24:43.511475+0000","last_unstale":"2024-03-29T08:24:44.483168+0000","last_undegraded":"2024-03-29T08:24:44.483168+0000","last_fullsized":"2024-03-29T08:24:44.483168+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:14:35.102272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.487594+0000","last_change":"2024-03-29T08:24:43.497300+0000","last_active":"2024-03-29T08:24:44.487594+0000","last_peered":"2024-03-29T08:24:44.487594+0000","last_clean":"2024-03-29T08:24:44.487594+0000","last_became_active":"2024-03-29T08:24:43.497060+0000","last_became_peered":"2024-03-29T08:24:43.497060+0000","last_unstale":"2024-03-29T08:24:44.487594+0000","last_undegraded":"2024-03-29T08:24:44.487594+0000","last_fullsized":"2024-03-29T08:24:44.487594+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:14.480052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988075+0000","last_change":"2024-03-29T08:24:43.504898+0000","last_active":"2024-03-29T08:24:44.988075+0000","last_peered":"2024-03-29T08:24:44.988075+0000","last_clean":"2024-03-29T08:24:44.988075+0000","last_became_active":"2024-03-29T08:24:43.504614+0000","last_became_peered":"2024-03-29T08:24:43.504614+0000","last_unstale":"2024-03-29T08:24:44.988075+0000","last_undegraded":"2024-03-29T08:24:44.988075+0000","last_fullsized":"2024-03-29T08:24:44.988075+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:03.589841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.509629+0000","last_change":"2024-03-29T08:24:43.503376+0000","last_active":"2024-03-29T08:24:43.509629+0000","last_peered":"2024-03-29T08:24:43.509629+0000","last_clean":"2024-03-29T08:24:43.509629+0000","last_became_active":"2024-03-29T08:24:43.503166+0000","last_became_peered":"2024-03-29T08:24:43.503166+0000","last_unstale":"2024-03-29T08:24:43.509629+0000","last_undegraded":"2024-03-29T08:24:43.509629+0000","last_fullsized":"2024-03-29T08:24:43.509629+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:53:40.790853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483504+0000","last_change":"2024-03-29T08:24:43.504360+0000","last_active":"2024-03-29T08:24:44.483504+0000","last_peered":"2024-03-29T08:24:44.483504+0000","last_clean":"2024-03-29T08:24:44.483504+0000","last_became_active":"2024-03-29T08:24:43.504125+0000","last_became_peered":"2024-03-29T08:24:43.504125+0000","last_unstale":"2024-03-29T08:24:44.483504+0000","last_undegraded":"2024-03-29T08:24:44.483504+0000","last_fullsized":"2024-03-29T08:24:44.483504+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:43:36.123959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.510666+0000","last_change":"2024-03-29T08:24:43.503954+0000","last_active":"2024-03-29T08:24:43.510666+0000","last_peered":"2024-03-29T08:24:43.510666+0000","last_clean":"2024-03-29T08:24:43.510666+0000","last_became_active":"2024-03-29T08:24:43.503649+0000","last_became_peered":"2024-03-29T08:24:43.503649+0000","last_unstale":"2024-03-29T08:24:43.510666+0000","last_undegraded":"2024-03-29T08:24:43.510666+0000","last_fullsized":"2024-03-29T08:24:43.510666+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:23.633523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988128+0000","last_change":"2024-03-29T08:24:43.500906+0000","last_active":"2024-03-29T08:24:44.988128+0000","last_peered":"2024-03-29T08:24:44.988128+0000","last_clean":"2024-03-29T08:24:44.988128+0000","last_became_active":"2024-03-29T08:24:43.500643+0000","last_became_peered":"2024-03-29T08:24:43.500643+0000","last_unstale":"2024-03-29T08:24:44.988128+0000","last_undegraded":"2024-03-29T08:24:44.988128+0000","last_fullsized":"2024-03-29T08:24:44.988128+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:23:51.374527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861312+0000","last_change":"2024-03-29T08:24:43.503715+0000","last_active":"2024-03-29T08:24:44.861312+0000","last_peered":"2024-03-29T08:24:44.861312+0000","last_clean":"2024-03-29T08:24:44.861312+0000","last_became_active":"2024-03-29T08:24:43.501398+0000","last_became_peered":"2024-03-29T08:24:43.501398+0000","last_unstale":"2024-03-29T08:24:44.861312+0000","last_undegraded":"2024-03-29T08:24:44.861312+0000","last_fullsized":"2024-03-29T08:24:44.861312+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:21:03.953033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488496+0000","last_change":"2024-03-29T08:24:43.514487+0000","last_active":"2024-03-29T08:24:44.488496+0000","last_peered":"2024-03-29T08:24:44.488496+0000","last_clean":"2024-03-29T08:24:44.488496+0000","last_became_active":"2024-03-29T08:24:43.513873+0000","last_became_peered":"2024-03-29T08:24:43.513873+0000","last_unstale":"2024-03-29T08:24:44.488496+0000","last_undegraded":"2024-03-29T08:24:44.488496+0000","last_fullsized":"2024-03-29T08:24:44.488496+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:05:19.517087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.487693+0000","last_change":"2024-03-29T08:24:43.498257+0000","last_active":"2024-03-29T08:24:44.487693+0000","last_peered":"2024-03-29T08:24:44.487693+0000","last_clean":"2024-03-29T08:24:44.487693+0000","last_became_active":"2024-03-29T08:24:43.497953+0000","last_became_peered":"2024-03-29T08:24:43.497953+0000","last_unstale":"2024-03-29T08:24:44.487693+0000","last_undegraded":"2024-03-29T08:24:44.487693+0000","last_fullsized":"2024-03-29T08:24:44.487693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:27:03.589841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483073+0000","last_change":"2024-03-29T08:24:43.497050+0000","last_active":"2024-03-29T08:24:44.483073+0000","last_peered":"2024-03-29T08:24:44.483073+0000","last_clean":"2024-03-29T08:24:44.483073+0000","last_became_active":"2024-03-29T08:24:43.496801+0000","last_became_peered":"2024-03-29T08:24:43.496801+0000","last_unstale":"2024-03-29T08:24:44.483073+0000","last_undegraded":"2024-03-29T08:24:44.483073+0000","last_fullsized":"2024-03-29T08:24:44.483073+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:04:35.962563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988434+0000","last_change":"2024-03-29T08:24:43.502011+0000","last_active":"2024-03-29T08:24:44.988434+0000","last_peered":"2024-03-29T08:24:44.988434+0000","last_clean":"2024-03-29T08:24:44.988434+0000","last_became_active":"2024-03-29T08:24:43.501778+0000","last_became_peered":"2024-03-29T08:24:43.501778+0000","last_unstale":"2024-03-29T08:24:44.988434+0000","last_undegraded":"2024-03-29T08:24:44.988434+0000","last_fullsized":"2024-03-29T08:24:44.988434+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:45:47.180483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.508562+0000","last_change":"2024-03-29T08:24:43.497136+0000","last_active":"2024-03-29T08:24:43.508562+0000","last_peered":"2024-03-29T08:24:43.508562+0000","last_clean":"2024-03-29T08:24:43.508562+0000","last_became_active":"2024-03-29T08:24:43.496857+0000","last_became_peered":"2024-03-29T08:24:43.496857+0000","last_unstale":"2024-03-29T08:24:43.508562+0000","last_undegraded":"2024-03-29T08:24:43.508562+0000","last_fullsized":"2024-03-29T08:24:43.508562+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:45:47.180483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488674+0000","last_change":"2024-03-29T08:24:43.508237+0000","last_active":"2024-03-29T08:24:44.488674+0000","last_peered":"2024-03-29T08:24:44.488674+0000","last_clean":"2024-03-29T08:24:44.488674+0000","last_became_active":"2024-03-29T08:24:43.507524+0000","last_became_peered":"2024-03-29T08:24:43.507524+0000","last_unstale":"2024-03-29T08:24:44.488674+0000","last_undegraded":"2024-03-29T08:24:44.488674+0000","last_fullsized":"2024-03-29T08:24:44.488674+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:02:34.591004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488314+0000","last_change":"2024-03-29T08:24:43.510515+0000","last_active":"2024-03-29T08:24:44.488314+0000","last_peered":"2024-03-29T08:24:44.488314+0000","last_clean":"2024-03-29T08:24:44.488314+0000","last_became_active":"2024-03-29T08:24:43.510164+0000","last_became_peered":"2024-03-29T08:24:43.510164+0000","last_unstale":"2024-03-29T08:24:44.488314+0000","last_undegraded":"2024-03-29T08:24:44.488314+0000","last_fullsized":"2024-03-29T08:24:44.488314+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:05:13.449547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.514804+0000","last_change":"2024-03-29T08:24:43.505783+0000","last_active":"2024-03-29T08:24:43.514804+0000","last_peered":"2024-03-29T08:24:43.514804+0000","last_clean":"2024-03-29T08:24:43.514804+0000","last_became_active":"2024-03-29T08:24:43.505339+0000","last_became_peered":"2024-03-29T08:24:43.505339+0000","last_unstale":"2024-03-29T08:24:43.514804+0000","last_undegraded":"2024-03-29T08:24:43.514804+0000","last_fullsized":"2024-03-29T08:24:43.514804+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:21:13.208809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488510+0000","last_change":"2024-03-29T08:24:43.507284+0000","last_active":"2024-03-29T08:24:44.488510+0000","last_peered":"2024-03-29T08:24:44.488510+0000","last_clean":"2024-03-29T08:24:44.488510+0000","last_became_active":"2024-03-29T08:24:43.507015+0000","last_became_peered":"2024-03-29T08:24:43.507015+0000","last_unstale":"2024-03-29T08:24:44.488510+0000","last_undegraded":"2024-03-29T08:24:44.488510+0000","last_fullsized":"2024-03-29T08:24:44.488510+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:40:45.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988049+0000","last_change":"2024-03-29T08:24:43.501305+0000","last_active":"2024-03-29T08:24:44.988049+0000","last_peered":"2024-03-29T08:24:44.988049+0000","last_clean":"2024-03-29T08:24:44.988049+0000","last_became_active":"2024-03-29T08:24:43.501046+0000","last_became_peered":"2024-03-29T08:24:43.501046+0000","last_unstale":"2024-03-29T08:24:44.988049+0000","last_undegraded":"2024-03-29T08:24:44.988049+0000","last_fullsized":"2024-03-29T08:24:44.988049+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:09.731188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.510631+0000","last_change":"2024-03-29T08:24:43.505985+0000","last_active":"2024-03-29T08:24:43.510631+0000","last_peered":"2024-03-29T08:24:43.510631+0000","last_clean":"2024-03-29T08:24:43.510631+0000","last_became_active":"2024-03-29T08:24:43.505702+0000","last_became_peered":"2024-03-29T08:24:43.505702+0000","last_unstale":"2024-03-29T08:24:43.510631+0000","last_undegraded":"2024-03-29T08:24:43.510631+0000","last_fullsized":"2024-03-29T08:24:43.510631+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:36.275403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.515669+0000","last_change":"2024-03-29T08:24:43.512131+0000","last_active":"2024-03-29T08:24:43.515669+0000","last_peered":"2024-03-29T08:24:43.515669+0000","last_clean":"2024-03-29T08:24:43.515669+0000","last_became_active":"2024-03-29T08:24:43.511842+0000","last_became_peered":"2024-03-29T08:24:43.511842+0000","last_unstale":"2024-03-29T08:24:43.515669+0000","last_undegraded":"2024-03-29T08:24:43.515669+0000","last_fullsized":"2024-03-29T08:24:43.515669+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:03:15.119626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861367+0000","last_change":"2024-03-29T08:24:43.505584+0000","last_active":"2024-03-29T08:24:44.861367+0000","last_peered":"2024-03-29T08:24:44.861367+0000","last_clean":"2024-03-29T08:24:44.861367+0000","last_became_active":"2024-03-29T08:24:43.505284+0000","last_became_peered":"2024-03-29T08:24:43.505284+0000","last_unstale":"2024-03-29T08:24:44.861367+0000","last_undegraded":"2024-03-29T08:24:44.861367+0000","last_fullsized":"2024-03-29T08:24:44.861367+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:44:56.220658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861491+0000","last_change":"2024-03-29T08:24:43.500283+0000","last_active":"2024-03-29T08:24:44.861491+0000","last_peered":"2024-03-29T08:24:44.861491+0000","last_clean":"2024-03-29T08:24:44.861491+0000","last_became_active":"2024-03-29T08:24:43.499781+0000","last_became_peered":"2024-03-29T08:24:43.499781+0000","last_unstale":"2024-03-29T08:24:44.861491+0000","last_undegraded":"2024-03-29T08:24:44.861491+0000","last_fullsized":"2024-03-29T08:24:44.861491+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:29:00.089772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.498131+0000","last_change":"2024-03-29T08:24:43.495197+0000","last_active":"2024-03-29T08:24:43.498131+0000","last_peered":"2024-03-29T08:24:43.498131+0000","last_clean":"2024-03-29T08:24:43.498131+0000","last_became_active":"2024-03-29T08:24:43.494913+0000","last_became_peered":"2024-03-29T08:24:43.494913+0000","last_unstale":"2024-03-29T08:24:43.498131+0000","last_undegraded":"2024-03-29T08:24:43.498131+0000","last_fullsized":"2024-03-29T08:24:43.498131+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:21:55.614273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.514787+0000","last_change":"2024-03-29T08:24:43.505601+0000","last_active":"2024-03-29T08:24:43.514787+0000","last_peered":"2024-03-29T08:24:43.514787+0000","last_clean":"2024-03-29T08:24:43.514787+0000","last_became_active":"2024-03-29T08:24:43.505013+0000","last_became_peered":"2024-03-29T08:24:43.505013+0000","last_unstale":"2024-03-29T08:24:43.514787+0000","last_undegraded":"2024-03-29T08:24:43.514787+0000","last_fullsized":"2024-03-29T08:24:43.514787+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:08:13.184351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488593+0000","last_change":"2024-03-29T08:24:43.514589+0000","last_active":"2024-03-29T08:24:44.488593+0000","last_peered":"2024-03-29T08:24:44.488593+0000","last_clean":"2024-03-29T08:24:44.488593+0000","last_became_active":"2024-03-29T08:24:43.514253+0000","last_became_peered":"2024-03-29T08:24:43.514253+0000","last_unstale":"2024-03-29T08:24:44.488593+0000","last_undegraded":"2024-03-29T08:24:44.488593+0000","last_fullsized":"2024-03-29T08:24:44.488593+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:49:59.075543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483103+0000","last_change":"2024-03-29T08:24:43.511667+0000","last_active":"2024-03-29T08:24:44.483103+0000","last_peered":"2024-03-29T08:24:44.483103+0000","last_clean":"2024-03-29T08:24:44.483103+0000","last_became_active":"2024-03-29T08:24:43.511381+0000","last_became_peered":"2024-03-29T08:24:43.511381+0000","last_unstale":"2024-03-29T08:24:44.483103+0000","last_undegraded":"2024-03-29T08:24:44.483103+0000","last_fullsized":"2024-03-29T08:24:44.483103+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:02:29.946962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483096+0000","last_change":"2024-03-29T08:24:43.500942+0000","last_active":"2024-03-29T08:24:44.483096+0000","last_peered":"2024-03-29T08:24:44.483096+0000","last_clean":"2024-03-29T08:24:44.483096+0000","last_became_active":"2024-03-29T08:24:43.500668+0000","last_became_peered":"2024-03-29T08:24:43.500668+0000","last_unstale":"2024-03-29T08:24:44.483096+0000","last_undegraded":"2024-03-29T08:24:44.483096+0000","last_fullsized":"2024-03-29T08:24:44.483096+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:59:38.958688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988322+0000","last_change":"2024-03-29T08:24:43.496778+0000","last_active":"2024-03-29T08:24:44.988322+0000","last_peered":"2024-03-29T08:24:44.988322+0000","last_clean":"2024-03-29T08:24:44.988322+0000","last_became_active":"2024-03-29T08:24:43.496496+0000","last_became_peered":"2024-03-29T08:24:43.496496+0000","last_unstale":"2024-03-29T08:24:44.988322+0000","last_undegraded":"2024-03-29T08:24:44.988322+0000","last_fullsized":"2024-03-29T08:24:44.988322+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:44:14.480052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"91'4","reported_seq":15,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861578+0000","last_change":"2024-03-29T08:24:43.099994+0000","last_active":"2024-03-29T08:24:44.861578+0000","last_peered":"2024-03-29T08:24:44.861578+0000","last_clean":"2024-03-29T08:24:44.861578+0000","last_became_active":"2024-03-29T08:24:43.099724+0000","last_became_peered":"2024-03-29T08:24:43.099724+0000","last_unstale":"2024-03-29T08:24:44.861578+0000","last_undegraded":"2024-03-29T08:24:44.861578+0000","last_fullsized":"2024-03-29T08:24:44.861578+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:53:40.344288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.508604+0000","last_change":"2024-03-29T08:24:43.501012+0000","last_active":"2024-03-29T08:24:43.508604+0000","last_peered":"2024-03-29T08:24:43.508604+0000","last_clean":"2024-03-29T08:24:43.508604+0000","last_became_active":"2024-03-29T08:24:43.500631+0000","last_became_peered":"2024-03-29T08:24:43.500631+0000","last_unstale":"2024-03-29T08:24:43.508604+0000","last_undegraded":"2024-03-29T08:24:43.508604+0000","last_fullsized":"2024-03-29T08:24:43.508604+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:55:06.990851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"91'1","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483524+0000","last_change":"2024-03-29T08:24:43.089840+0000","last_active":"2024-03-29T08:24:44.483524+0000","last_peered":"2024-03-29T08:24:44.483524+0000","last_clean":"2024-03-29T08:24:44.483524+0000","last_became_active":"2024-03-29T08:24:43.089218+0000","last_became_peered":"2024-03-29T08:24:43.089218+0000","last_unstale":"2024-03-29T08:24:44.483524+0000","last_undegraded":"2024-03-29T08:24:44.483524+0000","last_fullsized":"2024-03-29T08:24:44.483524+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:10.817335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988163+0000","last_change":"2024-03-29T08:24:43.503203+0000","last_active":"2024-03-29T08:24:44.988163+0000","last_peered":"2024-03-29T08:24:44.988163+0000","last_clean":"2024-03-29T08:24:44.988163+0000","last_became_active":"2024-03-29T08:24:43.502676+0000","last_became_peered":"2024-03-29T08:24:43.502676+0000","last_unstale":"2024-03-29T08:24:44.988163+0000","last_undegraded":"2024-03-29T08:24:44.988163+0000","last_fullsized":"2024-03-29T08:24:44.988163+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:52.482922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.513474+0000","last_change":"2024-03-29T08:24:43.098772+0000","last_active":"2024-03-29T08:24:43.513474+0000","last_peered":"2024-03-29T08:24:43.513474+0000","last_clean":"2024-03-29T08:24:43.513474+0000","last_became_active":"2024-03-29T08:24:43.098489+0000","last_became_peered":"2024-03-29T08:24:43.098489+0000","last_unstale":"2024-03-29T08:24:43.513474+0000","last_undegraded":"2024-03-29T08:24:43.513474+0000","last_fullsized":"2024-03-29T08:24:43.513474+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:32:13.524931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.508968+0000","last_change":"2024-03-29T08:24:43.502146+0000","last_active":"2024-03-29T08:24:43.508968+0000","last_peered":"2024-03-29T08:24:43.508968+0000","last_clean":"2024-03-29T08:24:43.508968+0000","last_became_active":"2024-03-29T08:24:43.501888+0000","last_became_peered":"2024-03-29T08:24:43.501888+0000","last_unstale":"2024-03-29T08:24:43.508968+0000","last_undegraded":"2024-03-29T08:24:43.508968+0000","last_fullsized":"2024-03-29T08:24:43.508968+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:42.208598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.515459+0000","last_change":"2024-03-29T08:24:43.091946+0000","last_active":"2024-03-29T08:24:43.515459+0000","last_peered":"2024-03-29T08:24:43.515459+0000","last_clean":"2024-03-29T08:24:43.515459+0000","last_became_active":"2024-03-29T08:24:43.091643+0000","last_became_peered":"2024-03-29T08:24:43.091643+0000","last_unstale":"2024-03-29T08:24:43.515459+0000","last_undegraded":"2024-03-29T08:24:43.515459+0000","last_fullsized":"2024-03-29T08:24:43.515459+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:58:04.981235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.516336+0000","last_change":"2024-03-29T08:24:43.513319+0000","last_active":"2024-03-29T08:24:43.516336+0000","last_peered":"2024-03-29T08:24:43.516336+0000","last_clean":"2024-03-29T08:24:43.516336+0000","last_became_active":"2024-03-29T08:24:43.513073+0000","last_became_peered":"2024-03-29T08:24:43.513073+0000","last_unstale":"2024-03-29T08:24:43.516336+0000","last_undegraded":"2024-03-29T08:24:43.516336+0000","last_fullsized":"2024-03-29T08:24:43.516336+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:32:22.633370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"91'1","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988088+0000","last_change":"2024-03-29T08:24:43.089850+0000","last_active":"2024-03-29T08:24:44.988088+0000","last_peered":"2024-03-29T08:24:44.988088+0000","last_clean":"2024-03-29T08:24:44.988088+0000","last_became_active":"2024-03-29T08:24:43.089537+0000","last_became_peered":"2024-03-29T08:24:43.089537+0000","last_unstale":"2024-03-29T08:24:44.988088+0000","last_undegraded":"2024-03-29T08:24:44.988088+0000","last_fullsized":"2024-03-29T08:24:44.988088+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:21:54.611561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483353+0000","last_change":"2024-03-29T08:24:43.503817+0000","last_active":"2024-03-29T08:24:44.483353+0000","last_peered":"2024-03-29T08:24:44.483353+0000","last_clean":"2024-03-29T08:24:44.483353+0000","last_became_active":"2024-03-29T08:24:43.503454+0000","last_became_peered":"2024-03-29T08:24:43.503454+0000","last_unstale":"2024-03-29T08:24:44.483353+0000","last_undegraded":"2024-03-29T08:24:44.483353+0000","last_fullsized":"2024-03-29T08:24:44.483353+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:57:04.503788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"91'1","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483197+0000","last_change":"2024-03-29T08:24:43.091662+0000","last_active":"2024-03-29T08:24:44.483197+0000","last_peered":"2024-03-29T08:24:44.483197+0000","last_clean":"2024-03-29T08:24:44.483197+0000","last_became_active":"2024-03-29T08:24:43.091078+0000","last_became_peered":"2024-03-29T08:24:43.091078+0000","last_unstale":"2024-03-29T08:24:44.483197+0000","last_undegraded":"2024-03-29T08:24:44.483197+0000","last_fullsized":"2024-03-29T08:24:44.483197+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:08:47.116697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.509571+0000","last_change":"2024-03-29T08:24:43.502707+0000","last_active":"2024-03-29T08:24:43.509571+0000","last_peered":"2024-03-29T08:24:43.509571+0000","last_clean":"2024-03-29T08:24:43.509571+0000","last_became_active":"2024-03-29T08:24:43.502410+0000","last_became_peered":"2024-03-29T08:24:43.502410+0000","last_unstale":"2024-03-29T08:24:43.509571+0000","last_undegraded":"2024-03-29T08:24:43.509571+0000","last_fullsized":"2024-03-29T08:24:43.509571+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:32:14.527644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488245+0000","last_change":"2024-03-29T08:24:43.101557+0000","last_active":"2024-03-29T08:24:44.488245+0000","last_peered":"2024-03-29T08:24:44.488245+0000","last_clean":"2024-03-29T08:24:44.488245+0000","last_became_active":"2024-03-29T08:24:43.101270+0000","last_became_peered":"2024-03-29T08:24:43.101270+0000","last_unstale":"2024-03-29T08:24:44.488245+0000","last_undegraded":"2024-03-29T08:24:44.488245+0000","last_fullsized":"2024-03-29T08:24:44.488245+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:20:38.882583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488029+0000","last_change":"2024-03-29T08:24:43.509577+0000","last_active":"2024-03-29T08:24:44.488029+0000","last_peered":"2024-03-29T08:24:44.488029+0000","last_clean":"2024-03-29T08:24:44.488029+0000","last_became_active":"2024-03-29T08:24:43.509227+0000","last_became_peered":"2024-03-29T08:24:43.509227+0000","last_unstale":"2024-03-29T08:24:44.488029+0000","last_undegraded":"2024-03-29T08:24:44.488029+0000","last_fullsized":"2024-03-29T08:24:44.488029+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:28:22.979416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488008+0000","last_change":"2024-03-29T08:24:43.094283+0000","last_active":"2024-03-29T08:24:44.488008+0000","last_peered":"2024-03-29T08:24:44.488008+0000","last_clean":"2024-03-29T08:24:44.488008+0000","last_became_active":"2024-03-29T08:24:43.093930+0000","last_became_peered":"2024-03-29T08:24:43.093930+0000","last_unstale":"2024-03-29T08:24:44.488008+0000","last_undegraded":"2024-03-29T08:24:44.488008+0000","last_fullsized":"2024-03-29T08:24:44.488008+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:38:22.729031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488669+0000","last_change":"2024-03-29T08:24:43.502127+0000","last_active":"2024-03-29T08:24:44.488669+0000","last_peered":"2024-03-29T08:24:44.488669+0000","last_clean":"2024-03-29T08:24:44.488669+0000","last_became_active":"2024-03-29T08:24:43.501769+0000","last_became_peered":"2024-03-29T08:24:43.501769+0000","last_unstale":"2024-03-29T08:24:44.488669+0000","last_undegraded":"2024-03-29T08:24:44.488669+0000","last_fullsized":"2024-03-29T08:24:44.488669+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:42.208598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.487030+0000","last_change":"2024-03-29T08:24:43.088073+0000","last_active":"2024-03-29T08:24:43.487030+0000","last_peered":"2024-03-29T08:24:43.487030+0000","last_clean":"2024-03-29T08:24:43.487030+0000","last_became_active":"2024-03-29T08:24:43.086519+0000","last_became_peered":"2024-03-29T08:24:43.086519+0000","last_unstale":"2024-03-29T08:24:43.487030+0000","last_undegraded":"2024-03-29T08:24:43.487030+0000","last_fullsized":"2024-03-29T08:24:43.487030+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:23:21.392519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.513690+0000","last_change":"2024-03-29T08:24:43.512338+0000","last_active":"2024-03-29T08:24:43.513690+0000","last_peered":"2024-03-29T08:24:43.513690+0000","last_clean":"2024-03-29T08:24:43.513690+0000","last_became_active":"2024-03-29T08:24:43.512018+0000","last_became_peered":"2024-03-29T08:24:43.512018+0000","last_unstale":"2024-03-29T08:24:43.513690+0000","last_undegraded":"2024-03-29T08:24:43.513690+0000","last_fullsized":"2024-03-29T08:24:43.513690+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:40:45.903598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483576+0000","last_change":"2024-03-29T08:24:43.090188+0000","last_active":"2024-03-29T08:24:44.483576+0000","last_peered":"2024-03-29T08:24:44.483576+0000","last_clean":"2024-03-29T08:24:44.483576+0000","last_became_active":"2024-03-29T08:24:43.089629+0000","last_became_peered":"2024-03-29T08:24:43.089629+0000","last_unstale":"2024-03-29T08:24:44.483576+0000","last_undegraded":"2024-03-29T08:24:44.483576+0000","last_fullsized":"2024-03-29T08:24:44.483576+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:08.728476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.518090+0000","last_change":"2024-03-29T08:24:43.512173+0000","last_active":"2024-03-29T08:24:43.518090+0000","last_peered":"2024-03-29T08:24:43.518090+0000","last_clean":"2024-03-29T08:24:43.518090+0000","last_became_active":"2024-03-29T08:24:43.511864+0000","last_became_peered":"2024-03-29T08:24:43.511864+0000","last_unstale":"2024-03-29T08:24:43.518090+0000","last_undegraded":"2024-03-29T08:24:43.518090+0000","last_fullsized":"2024-03-29T08:24:43.518090+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:19:48.190913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"91'3","reported_seq":13,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.512755+0000","last_change":"2024-03-29T08:24:43.097101+0000","last_active":"2024-03-29T08:24:43.512755+0000","last_peered":"2024-03-29T08:24:43.512755+0000","last_clean":"2024-03-29T08:24:43.512755+0000","last_became_active":"2024-03-29T08:24:43.096811+0000","last_became_peered":"2024-03-29T08:24:43.096811+0000","last_unstale":"2024-03-29T08:24:43.512755+0000","last_undegraded":"2024-03-29T08:24:43.512755+0000","last_fullsized":"2024-03-29T08:24:43.512755+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:56:44.209659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861406+0000","last_change":"2024-03-29T08:24:43.507514+0000","last_active":"2024-03-29T08:24:44.861406+0000","last_peered":"2024-03-29T08:24:44.861406+0000","last_clean":"2024-03-29T08:24:44.861406+0000","last_became_active":"2024-03-29T08:24:43.506436+0000","last_became_peered":"2024-03-29T08:24:43.506436+0000","last_unstale":"2024-03-29T08:24:44.861406+0000","last_undegraded":"2024-03-29T08:24:44.861406+0000","last_fullsized":"2024-03-29T08:24:44.861406+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:18:10.946658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"91'6","reported_seq":17,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988260+0000","last_change":"2024-03-29T08:24:43.094920+0000","last_active":"2024-03-29T08:24:44.988260+0000","last_peered":"2024-03-29T08:24:44.988260+0000","last_clean":"2024-03-29T08:24:44.988260+0000","last_became_active":"2024-03-29T08:24:43.094629+0000","last_became_peered":"2024-03-29T08:24:43.094629+0000","last_unstale":"2024-03-29T08:24:44.988260+0000","last_undegraded":"2024-03-29T08:24:44.988260+0000","last_fullsized":"2024-03-29T08:24:44.988260+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:34:41.205886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.498821+0000","last_change":"2024-03-29T08:24:43.492204+0000","last_active":"2024-03-29T08:24:43.498821+0000","last_peered":"2024-03-29T08:24:43.498821+0000","last_clean":"2024-03-29T08:24:43.498821+0000","last_became_active":"2024-03-29T08:24:43.491943+0000","last_became_peered":"2024-03-29T08:24:43.491943+0000","last_unstale":"2024-03-29T08:24:43.498821+0000","last_undegraded":"2024-03-29T08:24:43.498821+0000","last_fullsized":"2024-03-29T08:24:43.498821+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:23:51.374527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"91'2","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.516202+0000","last_change":"2024-03-29T08:24:43.095534+0000","last_active":"2024-03-29T08:24:43.516202+0000","last_peered":"2024-03-29T08:24:43.516202+0000","last_clean":"2024-03-29T08:24:43.516202+0000","last_became_active":"2024-03-29T08:24:43.095179+0000","last_became_peered":"2024-03-29T08:24:43.095179+0000","last_unstale":"2024-03-29T08:24:43.516202+0000","last_undegraded":"2024-03-29T08:24:43.516202+0000","last_fullsized":"2024-03-29T08:24:43.516202+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:20:33.235596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.510260+0000","last_change":"2024-03-29T08:24:43.505245+0000","last_active":"2024-03-29T08:24:43.510260+0000","last_peered":"2024-03-29T08:24:43.510260+0000","last_clean":"2024-03-29T08:24:43.510260+0000","last_became_active":"2024-03-29T08:24:43.502538+0000","last_became_peered":"2024-03-29T08:24:43.502538+0000","last_unstale":"2024-03-29T08:24:43.510260+0000","last_undegraded":"2024-03-29T08:24:43.510260+0000","last_fullsized":"2024-03-29T08:24:43.510260+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:55.009436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.487732+0000","last_change":"2024-03-29T08:24:43.097110+0000","last_active":"2024-03-29T08:24:44.487732+0000","last_peered":"2024-03-29T08:24:44.487732+0000","last_clean":"2024-03-29T08:24:44.487732+0000","last_became_active":"2024-03-29T08:24:43.096780+0000","last_became_peered":"2024-03-29T08:24:43.096780+0000","last_unstale":"2024-03-29T08:24:44.487732+0000","last_undegraded":"2024-03-29T08:24:44.487732+0000","last_fullsized":"2024-03-29T08:24:44.487732+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:10.817335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488027+0000","last_change":"2024-03-29T08:24:43.514216+0000","last_active":"2024-03-29T08:24:44.488027+0000","last_peered":"2024-03-29T08:24:44.488027+0000","last_clean":"2024-03-29T08:24:44.488027+0000","last_became_active":"2024-03-29T08:24:43.513784+0000","last_became_peered":"2024-03-29T08:24:43.513784+0000","last_unstale":"2024-03-29T08:24:44.488027+0000","last_undegraded":"2024-03-29T08:24:44.488027+0000","last_fullsized":"2024-03-29T08:24:44.488027+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:21:55.614273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483109+0000","last_change":"2024-03-29T08:24:43.092297+0000","last_active":"2024-03-29T08:24:44.483109+0000","last_peered":"2024-03-29T08:24:44.483109+0000","last_clean":"2024-03-29T08:24:44.483109+0000","last_became_active":"2024-03-29T08:24:43.092039+0000","last_became_peered":"2024-03-29T08:24:43.092039+0000","last_unstale":"2024-03-29T08:24:44.483109+0000","last_undegraded":"2024-03-29T08:24:44.483109+0000","last_fullsized":"2024-03-29T08:24:44.483109+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:35.272690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.511087+0000","last_change":"2024-03-29T08:24:43.506382+0000","last_active":"2024-03-29T08:24:43.511087+0000","last_peered":"2024-03-29T08:24:43.511087+0000","last_clean":"2024-03-29T08:24:43.511087+0000","last_became_active":"2024-03-29T08:24:43.506065+0000","last_became_peered":"2024-03-29T08:24:43.506065+0000","last_unstale":"2024-03-29T08:24:43.511087+0000","last_undegraded":"2024-03-29T08:24:43.511087+0000","last_fullsized":"2024-03-29T08:24:43.511087+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:09.731188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"91'8","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.512972+0000","last_change":"2024-03-29T08:24:43.100298+0000","last_active":"2024-03-29T08:24:43.512972+0000","last_peered":"2024-03-29T08:24:43.512972+0000","last_clean":"2024-03-29T08:24:43.512972+0000","last_became_active":"2024-03-29T08:24:43.099448+0000","last_became_peered":"2024-03-29T08:24:43.099448+0000","last_unstale":"2024-03-29T08:24:43.512972+0000","last_undegraded":"2024-03-29T08:24:43.512972+0000","last_fullsized":"2024-03-29T08:24:43.512972+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:53:39.788140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988365+0000","last_change":"2024-03-29T08:24:43.495633+0000","last_active":"2024-03-29T08:24:44.988365+0000","last_peered":"2024-03-29T08:24:44.988365+0000","last_clean":"2024-03-29T08:24:44.988365+0000","last_became_active":"2024-03-29T08:24:43.495358+0000","last_became_peered":"2024-03-29T08:24:43.495358+0000","last_unstale":"2024-03-29T08:24:44.988365+0000","last_undegraded":"2024-03-29T08:24:44.988365+0000","last_fullsized":"2024-03-29T08:24:44.988365+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:03:15.119626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"91'1","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861666+0000","last_change":"2024-03-29T08:24:43.099420+0000","last_active":"2024-03-29T08:24:44.861666+0000","last_peered":"2024-03-29T08:24:44.861666+0000","last_clean":"2024-03-29T08:24:44.861666+0000","last_became_active":"2024-03-29T08:24:43.099146+0000","last_became_peered":"2024-03-29T08:24:43.099146+0000","last_unstale":"2024-03-29T08:24:44.861666+0000","last_undegraded":"2024-03-29T08:24:44.861666+0000","last_fullsized":"2024-03-29T08:24:44.861666+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:05:34.165676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483642+0000","last_change":"2024-03-29T08:24:43.508543+0000","last_active":"2024-03-29T08:24:44.483642+0000","last_peered":"2024-03-29T08:24:44.483642+0000","last_clean":"2024-03-29T08:24:44.483642+0000","last_became_active":"2024-03-29T08:24:43.508332+0000","last_became_peered":"2024-03-29T08:24:43.508332+0000","last_unstale":"2024-03-29T08:24:44.483642+0000","last_undegraded":"2024-03-29T08:24:44.483642+0000","last_fullsized":"2024-03-29T08:24:44.483642+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:24:54.965342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"91'2","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488799+0000","last_change":"2024-03-29T08:24:43.100835+0000","last_active":"2024-03-29T08:24:44.488799+0000","last_peered":"2024-03-29T08:24:44.488799+0000","last_clean":"2024-03-29T08:24:44.488799+0000","last_became_active":"2024-03-29T08:24:43.100101+0000","last_became_peered":"2024-03-29T08:24:43.100101+0000","last_unstale":"2024-03-29T08:24:44.488799+0000","last_undegraded":"2024-03-29T08:24:44.488799+0000","last_fullsized":"2024-03-29T08:24:44.488799+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:50:52.971257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988196+0000","last_change":"2024-03-29T08:24:43.495626+0000","last_active":"2024-03-29T08:24:44.988196+0000","last_peered":"2024-03-29T08:24:44.988196+0000","last_clean":"2024-03-29T08:24:44.988196+0000","last_became_active":"2024-03-29T08:24:43.495385+0000","last_became_peered":"2024-03-29T08:24:43.495385+0000","last_unstale":"2024-03-29T08:24:44.988196+0000","last_undegraded":"2024-03-29T08:24:44.988196+0000","last_fullsized":"2024-03-29T08:24:44.988196+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:19:48.190913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488423+0000","last_change":"2024-03-29T08:24:43.095264+0000","last_active":"2024-03-29T08:24:44.488423+0000","last_peered":"2024-03-29T08:24:44.488423+0000","last_clean":"2024-03-29T08:24:44.488423+0000","last_became_active":"2024-03-29T08:24:43.094903+0000","last_became_peered":"2024-03-29T08:24:43.094903+0000","last_unstale":"2024-03-29T08:24:44.488423+0000","last_undegraded":"2024-03-29T08:24:44.488423+0000","last_fullsized":"2024-03-29T08:24:44.488423+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:23:07.204168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.487629+0000","last_change":"2024-03-29T08:24:43.499714+0000","last_active":"2024-03-29T08:24:44.487629+0000","last_peered":"2024-03-29T08:24:44.487629+0000","last_clean":"2024-03-29T08:24:44.487629+0000","last_became_active":"2024-03-29T08:24:43.499471+0000","last_became_peered":"2024-03-29T08:24:43.499471+0000","last_unstale":"2024-03-29T08:24:44.487629+0000","last_undegraded":"2024-03-29T08:24:44.487629+0000","last_fullsized":"2024-03-29T08:24:44.487629+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:44:23.914934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"91'2","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488424+0000","last_change":"2024-03-29T08:24:43.092008+0000","last_active":"2024-03-29T08:24:44.488424+0000","last_peered":"2024-03-29T08:24:44.488424+0000","last_clean":"2024-03-29T08:24:44.488424+0000","last_became_active":"2024-03-29T08:24:43.090131+0000","last_became_peered":"2024-03-29T08:24:43.090131+0000","last_unstale":"2024-03-29T08:24:44.488424+0000","last_undegraded":"2024-03-29T08:24:44.488424+0000","last_fullsized":"2024-03-29T08:24:44.488424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:42:10.005340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861415+0000","last_change":"2024-03-29T08:24:43.511254+0000","last_active":"2024-03-29T08:24:44.861415+0000","last_peered":"2024-03-29T08:24:44.861415+0000","last_clean":"2024-03-29T08:24:44.861415+0000","last_became_active":"2024-03-29T08:24:43.510955+0000","last_became_peered":"2024-03-29T08:24:43.510955+0000","last_unstale":"2024-03-29T08:24:44.861415+0000","last_undegraded":"2024-03-29T08:24:44.861415+0000","last_fullsized":"2024-03-29T08:24:44.861415+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:03:55.979572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.487797+0000","last_change":"2024-03-29T08:24:43.094275+0000","last_active":"2024-03-29T08:24:44.487797+0000","last_peered":"2024-03-29T08:24:44.487797+0000","last_clean":"2024-03-29T08:24:44.487797+0000","last_became_active":"2024-03-29T08:24:43.094022+0000","last_became_peered":"2024-03-29T08:24:43.094022+0000","last_unstale":"2024-03-29T08:24:44.487797+0000","last_undegraded":"2024-03-29T08:24:44.487797+0000","last_fullsized":"2024-03-29T08:24:44.487797+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:25:08.728476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.509680+0000","last_change":"2024-03-29T08:24:43.501484+0000","last_active":"2024-03-29T08:24:43.509680+0000","last_peered":"2024-03-29T08:24:43.509680+0000","last_clean":"2024-03-29T08:24:43.509680+0000","last_became_active":"2024-03-29T08:24:43.501244+0000","last_became_peered":"2024-03-29T08:24:43.501244+0000","last_unstale":"2024-03-29T08:24:43.509680+0000","last_undegraded":"2024-03-29T08:24:43.509680+0000","last_fullsized":"2024-03-29T08:24:43.509680+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:11:36.608174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.491682+0000","last_change":"2024-03-29T08:24:43.096278+0000","last_active":"2024-03-29T08:24:43.491682+0000","last_peered":"2024-03-29T08:24:43.491682+0000","last_clean":"2024-03-29T08:24:43.491682+0000","last_became_active":"2024-03-29T08:24:43.096002+0000","last_became_peered":"2024-03-29T08:24:43.096002+0000","last_unstale":"2024-03-29T08:24:43.491682+0000","last_undegraded":"2024-03-29T08:24:43.491682+0000","last_fullsized":"2024-03-29T08:24:43.491682+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:40:44.900886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988209+0000","last_change":"2024-03-29T08:24:43.496951+0000","last_active":"2024-03-29T08:24:44.988209+0000","last_peered":"2024-03-29T08:24:44.988209+0000","last_clean":"2024-03-29T08:24:44.988209+0000","last_became_active":"2024-03-29T08:24:43.496692+0000","last_became_peered":"2024-03-29T08:24:43.496692+0000","last_unstale":"2024-03-29T08:24:44.988209+0000","last_undegraded":"2024-03-29T08:24:44.988209+0000","last_fullsized":"2024-03-29T08:24:44.988209+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:38:17.620893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488103+0000","last_change":"2024-03-29T08:24:43.093087+0000","last_active":"2024-03-29T08:24:44.488103+0000","last_peered":"2024-03-29T08:24:44.488103+0000","last_clean":"2024-03-29T08:24:44.488103+0000","last_became_active":"2024-03-29T08:24:43.092804+0000","last_became_peered":"2024-03-29T08:24:43.092804+0000","last_unstale":"2024-03-29T08:24:44.488103+0000","last_undegraded":"2024-03-29T08:24:44.488103+0000","last_fullsized":"2024-03-29T08:24:44.488103+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:48:30.536411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861524+0000","last_change":"2024-03-29T08:24:43.497747+0000","last_active":"2024-03-29T08:24:44.861524+0000","last_peered":"2024-03-29T08:24:44.861524+0000","last_clean":"2024-03-29T08:24:44.861524+0000","last_became_active":"2024-03-29T08:24:43.497267+0000","last_became_peered":"2024-03-29T08:24:43.497267+0000","last_unstale":"2024-03-29T08:24:44.861524+0000","last_undegraded":"2024-03-29T08:24:44.861524+0000","last_fullsized":"2024-03-29T08:24:44.861524+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:24:39.002767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"91'2","reported_seq":13,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.488841+0000","last_change":"2024-03-29T08:24:43.096961+0000","last_active":"2024-03-29T08:24:44.488841+0000","last_peered":"2024-03-29T08:24:44.488841+0000","last_clean":"2024-03-29T08:24:44.488841+0000","last_became_active":"2024-03-29T08:24:43.096657+0000","last_became_peered":"2024-03-29T08:24:43.096657+0000","last_unstale":"2024-03-29T08:24:44.488841+0000","last_undegraded":"2024-03-29T08:24:44.488841+0000","last_fullsized":"2024-03-29T08:24:44.488841+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:23:23.709515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861617+0000","last_change":"2024-03-29T08:24:43.491598+0000","last_active":"2024-03-29T08:24:44.861617+0000","last_peered":"2024-03-29T08:24:44.861617+0000","last_clean":"2024-03-29T08:24:44.861617+0000","last_became_active":"2024-03-29T08:24:43.490511+0000","last_became_peered":"2024-03-29T08:24:43.490511+0000","last_unstale":"2024-03-29T08:24:44.861617+0000","last_undegraded":"2024-03-29T08:24:44.861617+0000","last_fullsized":"2024-03-29T08:24:44.861617+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:36:33.757108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.490786+0000","last_change":"2024-03-29T08:24:43.097839+0000","last_active":"2024-03-29T08:24:43.490786+0000","last_peered":"2024-03-29T08:24:43.490786+0000","last_clean":"2024-03-29T08:24:43.490786+0000","last_became_active":"2024-03-29T08:24:43.097511+0000","last_became_peered":"2024-03-29T08:24:43.097511+0000","last_unstale":"2024-03-29T08:24:43.490786+0000","last_undegraded":"2024-03-29T08:24:43.490786+0000","last_fullsized":"2024-03-29T08:24:43.490786+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:54.006723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861352+0000","last_change":"2024-03-29T08:24:43.505982+0000","last_active":"2024-03-29T08:24:44.861352+0000","last_peered":"2024-03-29T08:24:44.861352+0000","last_clean":"2024-03-29T08:24:44.861352+0000","last_became_active":"2024-03-29T08:24:43.505727+0000","last_became_peered":"2024-03-29T08:24:43.505727+0000","last_unstale":"2024-03-29T08:24:44.861352+0000","last_undegraded":"2024-03-29T08:24:44.861352+0000","last_fullsized":"2024-03-29T08:24:44.861352+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:15:42.713997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988265+0000","last_change":"2024-03-29T08:24:43.094281+0000","last_active":"2024-03-29T08:24:44.988265+0000","last_peered":"2024-03-29T08:24:44.988265+0000","last_clean":"2024-03-29T08:24:44.988265+0000","last_became_active":"2024-03-29T08:24:43.093927+0000","last_became_peered":"2024-03-29T08:24:43.093927+0000","last_unstale":"2024-03-29T08:24:44.988265+0000","last_undegraded":"2024-03-29T08:24:44.988265+0000","last_fullsized":"2024-03-29T08:24:44.988265+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:49:58.072830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483512+0000","last_change":"2024-03-29T08:24:43.506350+0000","last_active":"2024-03-29T08:24:44.483512+0000","last_peered":"2024-03-29T08:24:44.483512+0000","last_clean":"2024-03-29T08:24:44.483512+0000","last_became_active":"2024-03-29T08:24:43.506096+0000","last_became_peered":"2024-03-29T08:24:43.506096+0000","last_unstale":"2024-03-29T08:24:44.483512+0000","last_undegraded":"2024-03-29T08:24:44.483512+0000","last_fullsized":"2024-03-29T08:24:44.483512+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:17:18.006415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483314+0000","last_change":"2024-03-29T08:24:43.084143+0000","last_active":"2024-03-29T08:24:44.483314+0000","last_peered":"2024-03-29T08:24:44.483314+0000","last_clean":"2024-03-29T08:24:44.483314+0000","last_became_active":"2024-03-29T08:24:43.083892+0000","last_became_peered":"2024-03-29T08:24:43.083892+0000","last_unstale":"2024-03-29T08:24:44.483314+0000","last_undegraded":"2024-03-29T08:24:44.483314+0000","last_fullsized":"2024-03-29T08:24:44.483314+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:45:46.177771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.508509+0000","last_change":"2024-03-29T08:24:43.500600+0000","last_active":"2024-03-29T08:24:43.508509+0000","last_peered":"2024-03-29T08:24:43.508509+0000","last_clean":"2024-03-29T08:24:43.508509+0000","last_became_active":"2024-03-29T08:24:43.500338+0000","last_became_peered":"2024-03-29T08:24:43.500338+0000","last_unstale":"2024-03-29T08:24:43.508509+0000","last_undegraded":"2024-03-29T08:24:43.508509+0000","last_fullsized":"2024-03-29T08:24:43.508509+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:08:12.210637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.487947+0000","last_change":"2024-03-29T08:24:43.094324+0000","last_active":"2024-03-29T08:24:44.487947+0000","last_peered":"2024-03-29T08:24:44.487947+0000","last_clean":"2024-03-29T08:24:44.487947+0000","last_became_active":"2024-03-29T08:24:43.093514+0000","last_became_peered":"2024-03-29T08:24:43.093514+0000","last_unstale":"2024-03-29T08:24:44.487947+0000","last_undegraded":"2024-03-29T08:24:44.487947+0000","last_fullsized":"2024-03-29T08:24:44.487947+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:19:35.272690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.508825+0000","last_change":"2024-03-29T08:24:43.502050+0000","last_active":"2024-03-29T08:24:43.508825+0000","last_peered":"2024-03-29T08:24:43.508825+0000","last_clean":"2024-03-29T08:24:43.508825+0000","last_became_active":"2024-03-29T08:24:43.501547+0000","last_became_peered":"2024-03-29T08:24:43.501547+0000","last_unstale":"2024-03-29T08:24:43.508825+0000","last_undegraded":"2024-03-29T08:24:43.508825+0000","last_fullsized":"2024-03-29T08:24:43.508825+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:26:11.820047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.483237+0000","last_change":"2024-03-29T08:24:43.091822+0000","last_active":"2024-03-29T08:24:44.483237+0000","last_peered":"2024-03-29T08:24:44.483237+0000","last_clean":"2024-03-29T08:24:44.483237+0000","last_became_active":"2024-03-29T08:24:43.091415+0000","last_became_peered":"2024-03-29T08:24:43.091415+0000","last_unstale":"2024-03-29T08:24:44.483237+0000","last_undegraded":"2024-03-29T08:24:44.483237+0000","last_fullsized":"2024-03-29T08:24:44.483237+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:34:35.731754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.510354+0000","last_change":"2024-03-29T08:24:43.506043+0000","last_active":"2024-03-29T08:24:43.510354+0000","last_peered":"2024-03-29T08:24:43.510354+0000","last_clean":"2024-03-29T08:24:43.510354+0000","last_became_active":"2024-03-29T08:24:43.505810+0000","last_became_peered":"2024-03-29T08:24:43.505810+0000","last_unstale":"2024-03-29T08:24:43.510354+0000","last_undegraded":"2024-03-29T08:24:43.510354+0000","last_fullsized":"2024-03-29T08:24:43.510354+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:42:03.393961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.485889+0000","last_change":"2024-03-29T08:24:43.097191+0000","last_active":"2024-03-29T08:24:43.485889+0000","last_peered":"2024-03-29T08:24:43.485889+0000","last_clean":"2024-03-29T08:24:43.485889+0000","last_became_active":"2024-03-29T08:24:43.096911+0000","last_became_peered":"2024-03-29T08:24:43.096911+0000","last_unstale":"2024-03-29T08:24:43.485889+0000","last_undegraded":"2024-03-29T08:24:43.485889+0000","last_fullsized":"2024-03-29T08:24:43.485889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:37:46.182612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.509326+0000","last_change":"2024-03-29T08:24:43.502744+0000","last_active":"2024-03-29T08:24:43.509326+0000","last_peered":"2024-03-29T08:24:43.509326+0000","last_clean":"2024-03-29T08:24:43.509326+0000","last_became_active":"2024-03-29T08:24:43.502448+0000","last_became_peered":"2024-03-29T08:24:43.502448+0000","last_unstale":"2024-03-29T08:24:43.509326+0000","last_undegraded":"2024-03-29T08:24:43.509326+0000","last_fullsized":"2024-03-29T08:24:43.509326+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:20:39.885295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.988106+0000","last_change":"2024-03-29T08:24:43.094957+0000","last_active":"2024-03-29T08:24:44.988106+0000","last_peered":"2024-03-29T08:24:44.988106+0000","last_clean":"2024-03-29T08:24:44.988106+0000","last_became_active":"2024-03-29T08:24:43.094627+0000","last_became_peered":"2024-03-29T08:24:43.094627+0000","last_unstale":"2024-03-29T08:24:44.988106+0000","last_undegraded":"2024-03-29T08:24:44.988106+0000","last_fullsized":"2024-03-29T08:24:44.988106+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:40:44.900886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.515559+0000","last_change":"2024-03-29T08:24:43.511382+0000","last_active":"2024-03-29T08:24:43.515559+0000","last_peered":"2024-03-29T08:24:43.515559+0000","last_clean":"2024-03-29T08:24:43.515559+0000","last_became_active":"2024-03-29T08:24:43.511081+0000","last_became_peered":"2024-03-29T08:24:43.511081+0000","last_unstale":"2024-03-29T08:24:43.515559+0000","last_undegraded":"2024-03-29T08:24:43.515559+0000","last_fullsized":"2024-03-29T08:24:43.515559+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:43.068889+0000","last_clean_scrub_stamp":"2024-03-29T08:24:43.068889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:39:52.482922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"91'1","reported_seq":12,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-03-29T08:24:44.861498+0000","last_change":"2024-03-29T08:24:43.097252+0000","last_active":"2024-03-29T08:24:44.861498+0000","last_peered":"2024-03-29T08:24:44.861498+0000","last_clean":"2024-03-29T08:24:44.861498+0000","last_became_active":"2024-03-29T08:24:43.096977+0000","last_became_peered":"2024-03-29T08:24:43.096977+0000","last_unstale":"2024-03-29T08:24:44.861498+0000","last_undegraded":"2024-03-29T08:24:44.861498+0000","last_fullsized":"2024-03-29T08:24:44.861498+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:24:42.066176+0000","last_clean_scrub_stamp":"2024-03-29T08:24:42.066176+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:20:04.792923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":224,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-03-29T08:24:43.083443+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:24:43.083443+0000","last_peered":"2024-03-29T08:24:43.083443+0000","last_clean":"2024-03-29T08:24:43.083443+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:24:43.083443+0000","last_undegraded":"2024-03-29T08:24:43.083443+0000","last_fullsized":"2024-03-29T08:24:43.083443+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509524,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1114112,"data_stored":474866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72899999999999998,"15min":0.66900000000000004},"min":{"1min":0.504,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.97999999999999998,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.75800000000000001,"15min":0.69499999999999995},"min":{"1min":0.58299999999999996,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.998,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.70899999999999996,"15min":0.66100000000000003},"min":{"1min":0.55700000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88800000000000001,"5min":1.202,"15min":1.202},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72299999999999998,"15min":0.65200000000000002},"min":{"1min":0.48799999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.93400000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.72199999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71399999999999997,"15min":0.65000000000000002},"min":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.503},"max":{"1min":0.93600000000000005,"5min":1.085,"15min":1.085},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.74399999999999999,"15min":0.68200000000000005},"min":{"1min":0.56899999999999995,"5min":0.53200000000000003,"15min":0.48699999999999999},"max":{"1min":0.87,"5min":1.288,"15min":1.288},"last":0.68300000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.72899999999999998,"15min":0.70399999999999996},"min":{"1min":0.55900000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98099999999999998,"5min":1.018,"15min":1.018},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.754,"15min":0.73399999999999999},"min":{"1min":0.57499999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.94599999999999995,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.76900000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59299999999999997,"15min":0.621},"min":{"1min":0.45000000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81699999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.625,"15min":0.69399999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.755,"5min":1.0009999999999999,"15min":1.123},"last":0.52800000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.63900000000000001,"15min":0.69099999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.76100000000000001,"5min":1.056,"15min":1.056},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66100000000000003,"15min":0.68999999999999995},"min":{"1min":0.46999999999999997,"5min":0.44800000000000001,"15min":0.39500000000000002},"max":{"1min":0.81599999999999995,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.752}]},{"osd":6,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.70799999999999996,"15min":0.72099999999999997},"min":{"1min":0.49199999999999999,"5min":0.48999999999999999,"15min":0.46600000000000003},"max":{"1min":0.94099999999999995,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.67600000000000005,"15min":0.73199999999999998},"min":{"1min":0.47499999999999998,"5min":0.46999999999999997,"15min":0.44900000000000001},"max":{"1min":0.89200000000000002,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.70399999999999996}]}]},{"osd":6,"up_from":15,"seq":64424509524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1118208,"data_stored":475468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.70899999999999996,"15min":0.64900000000000002},"min":{"1min":0.59799999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.86499999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.747,"15min":0.69899999999999995},"min":{"1min":0.59399999999999997,"5min":0.52900000000000003,"15min":0.44500000000000001},"max":{"1min":0.95599999999999996,"5min":1.079,"15min":1.079},"last":0.75900000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67300000000000004,"15min":0.625},"min":{"1min":0.45300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":0.97999999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.69099999999999995,"15min":0.64100000000000001},"min":{"1min":0.442,"5min":0.436,"15min":0.42899999999999999},"max":{"1min":0.88700000000000001,"5min":1.079,"15min":1.079},"last":0.89100000000000001}]},{"osd":2,"last update":"Fri Mar 29 08:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75700000000000001,"15min":0.69299999999999995},"min":{"1min":0.51800000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73999999999999999,"15min":0.71199999999999997},"min":{"1min":0.52900000000000003,"5min":0.435,"15min":0.435},"max":{"1min":0.90700000000000003,"5min":0.93200000000000005,"15min":0.95899999999999996},"last":0.91600000000000004}]},{"osd":3,"last update":"Fri Mar 29 08:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71599999999999997,"15min":0.70099999999999996},"min":{"1min":0.503,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.99399999999999999,"5min":1.109,"15min":1.109},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70999999999999996,"15min":0.69699999999999995},"min":{"1min":0.53600000000000003,"5min":0.50700000000000001,"15min":0.47799999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61399999999999999,"15min":0.65900000000000003},"min":{"1min":0.38400000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.98199999999999998,"5min":1.0620000000000001,"15min":1.556},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63,"15min":0.71199999999999997},"min":{"1min":0.44700000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.84299999999999997,"5min":1.0429999999999999,"15min":2.0539999999999998},"last":0.996}]},{"osd":5,"last update":"Fri Mar 29 08:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.626,"15min":0.68600000000000005},"min":{"1min":0.46400000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.872,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.71799999999999997},"min":{"1min":0.47099999999999997,"5min":0.37,"15min":0.37},"max":{"1min":0.93799999999999994,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":1.0189999999999999}]},{"osd":7,"last update":"Fri Mar 29 08:24:13 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.747},"min":{"1min":0.502,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.68700000000000006,"15min":0.75700000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1,"5min":1.109,"15min":1.109},"last":0.79600000000000004}]}]},{"osd":4,"up_from":15,"seq":64424509523,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1126400,"data_stored":481130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69599999999999995,"15min":0.70599999999999996},"min":{"1min":0.47799999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.92500000000000004,"5min":1.0289999999999999,"15min":2.1709999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.70199999999999996,"15min":0.73599999999999999},"min":{"1min":0.56499999999999995,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.95699999999999996,"5min":1.0489999999999999,"15min":2.2869999999999999},"last":0.63700000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.745,"15min":0.76100000000000001},"min":{"1min":0.53700000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90800000000000003,"5min":1.087,"15min":2.524},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.69299999999999995,"15min":0.71299999999999997},"min":{"1min":0.53100000000000003,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":2.2679999999999998},"last":0.69799999999999995}]},{"osd":2,"last update":"Fri Mar 29 08:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71399999999999997,"15min":0.72999999999999998},"min":{"1min":0.50900000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.93799999999999994,"5min":1.1060000000000001,"15min":2.363},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.755,"5min":0.753,"15min":0.75800000000000001},"min":{"1min":0.44400000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.97599999999999998,"5min":1.0029999999999999,"15min":2.1880000000000002},"last":0.78600000000000003}]},{"osd":3,"last update":"Fri Mar 29 08:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73899999999999999,"15min":0.754},"min":{"1min":0.57499999999999996,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.004,"5min":1.004,"15min":2.3029999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.78500000000000003,"15min":0.80900000000000005},"min":{"1min":0.66600000000000004,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.93899999999999995,"5min":1.0700000000000001,"15min":2.2549999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.63900000000000001,"15min":0.68400000000000005},"min":{"1min":0.433,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83599999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.63600000000000001,"15min":0.64800000000000002},"min":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":1.0149999999999999},"last":0.49199999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64500000000000002,"15min":0.66900000000000004},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.93100000000000005,"5min":1.016,"15min":1.016},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.70099999999999996,"15min":0.73599999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.5},"max":{"1min":0.82299999999999995,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.58399999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:24:09 2024","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68500000000000005,"15min":0.72699999999999998},"min":{"1min":0.55600000000000005,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.874,"5min":0.96399999999999997,"15min":1.1000000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73399999999999999,"15min":0.76100000000000001},"min":{"1min":0.56200000000000006,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.96999999999999997,"5min":1.0760000000000001,"15min":1.4430000000000001},"last":0.61599999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509526,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1122304,"data_stored":475470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:24:15 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.44600000000000001,"15min":0.46000000000000002},"min":{"1min":0.29299999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68300000000000005,"5min":0.84799999999999998,"15min":1.123},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.52400000000000002,"15min":0.52500000000000002},"min":{"1min":0.438,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.88100000000000001,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.36199999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:24:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44,"15min":0.45800000000000002},"min":{"1min":0.33300000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.71899999999999997,"5min":0.78900000000000003,"15min":1.1599999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.499,"15min":0.52900000000000003},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.73999999999999999,"5min":1.004,"15min":1.4790000000000001},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Mar 29 08:24:15 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.52700000000000002,"15min":0.56299999999999994},"min":{"1min":0.45400000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.73399999999999999,"5min":0.90000000000000002,"15min":1.2529999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60199999999999998,"15min":0.64800000000000002},"min":{"1min":0.46899999999999997,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.86599999999999999,"5min":0.97699999999999998,"15min":1.8580000000000001},"last":0.57599999999999996}]},{"osd":4,"last update":"Fri Mar 29 08:24:15 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.74399999999999999,"15min":0.78600000000000003},"min":{"1min":0.57899999999999996,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0149999999999999,"5min":1.1950000000000001,"15min":1.6419999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.746,"15min":0.77500000000000002},"min":{"1min":0.56999999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.391},"last":0.86799999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:24:15 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.78200000000000003,"15min":0.80400000000000005},"min":{"1min":0.59799999999999998,"5min":0.56999999999999995,"15min":0.52100000000000002},"max":{"1min":1.1619999999999999,"5min":1.671,"15min":1.671},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.875,"5min":0.80100000000000005,"15min":0.85299999999999998},"min":{"1min":0.68600000000000005,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.2050000000000001,"5min":1.6599999999999999,"15min":1.7110000000000001},"last":0.71399999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:24:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.746,"15min":0.81899999999999995},"min":{"1min":0.56200000000000006,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.669},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.75700000000000001,"15min":0.81699999999999995},"min":{"1min":0.624,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.073,"5min":1.212,"15min":2.0150000000000001},"last":0.66600000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:24:15 2024","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.77900000000000003,"15min":0.84699999999999998},"min":{"1min":0.63100000000000001,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.3169999999999999,"5min":1.4279999999999999,"15min":1.887},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.76400000000000001,"15min":0.84199999999999997},"min":{"1min":0.59599999999999997,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":2.044},"last":0.82499999999999996}]}]},{"osd":2,"up_from":15,"seq":64424509525,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1110016,"data_stored":474796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.46300000000000002,"15min":0.49299999999999999},"min":{"1min":0.33400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.66200000000000003,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55000000000000004,"15min":0.51500000000000001},"min":{"1min":0.41699999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.874,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.44500000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.504,"15min":0.5},"min":{"1min":0.36699999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.73499999999999999,"5min":1.399,"15min":1.399},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.503,"15min":0.51200000000000001},"min":{"1min":0.32300000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.77300000000000002,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.379}]},{"osd":3,"last update":"Fri Mar 29 08:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55000000000000004,"15min":0.50700000000000001},"min":{"1min":0.38,"5min":0.35099999999999998,"15min":0.34000000000000002},"max":{"1min":0.93300000000000005,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.623,"15min":0.59099999999999997},"min":{"1min":0.48599999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85099999999999998,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.496}]},{"osd":4,"last update":"Fri Mar 29 08:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.78600000000000003,"15min":0.77000000000000002},"min":{"1min":0.53000000000000003,"5min":0.52600000000000002,"15min":0.49099999999999999},"max":{"1min":1.0660000000000001,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.52800000000000002,"5min":0.435,"15min":0.435},"max":{"1min":1.0840000000000001,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.57199999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.82599999999999996,"15min":0.84099999999999997},"min":{"1min":0.66900000000000004,"5min":0.59399999999999997,"15min":0.55500000000000005},"max":{"1min":1.236,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.80100000000000005,"15min":0.84699999999999998},"min":{"1min":0.57899999999999996,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.194,"5min":1.28,"15min":1.526},"last":0.61799999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.58799999999999997,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.1080000000000001,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.77900000000000003,"15min":0.79100000000000004},"min":{"1min":0.61899999999999999,"5min":0.53400000000000003,"15min":0.52500000000000002},"max":{"1min":1.157,"5min":1.524,"15min":1.524},"last":0.69699999999999995}]},{"osd":7,"last update":"Fri Mar 29 08:24:08 2024","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.78100000000000003,"15min":0.80200000000000005},"min":{"1min":0.66200000000000003,"5min":0.51000000000000001,"15min":0.505},"max":{"1min":1.296,"5min":1.319,"15min":1.319},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.64800000000000002,"5min":0.55600000000000005,"15min":0.54900000000000004},"max":{"1min":1.1719999999999999,"5min":2,"15min":2},"last":0.58699999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509525,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1708032,"data_stored":1065254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.438,"15min":0.59199999999999997},"min":{"1min":0.32900000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":4.2539999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.49399999999999999,"15min":0.61799999999999999},"min":{"1min":0.376,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.69999999999999996,"5min":0.84299999999999997,"15min":4.1909999999999998},"last":0.34899999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.48899999999999999,"15min":0.505},"min":{"1min":0.42699999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.70699999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.53100000000000003,"15min":0.53600000000000003},"min":{"1min":0.34599999999999997,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.77700000000000002,"5min":0.878,"15min":0.88200000000000001},"last":0.40999999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59399999999999997,"15min":0.58499999999999996},"min":{"1min":0.48199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.83299999999999996,"5min":1.52,"15min":1.52},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53600000000000003,"15min":0.52800000000000002},"min":{"1min":0.42499999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.73399999999999999,"5min":1.331,"15min":1.331},"last":0.42099999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.752,"15min":0.74399999999999999},"min":{"1min":0.53800000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0580000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.875,"5min":0.751,"15min":0.78500000000000003},"min":{"1min":0.69399999999999995,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.175,"5min":1.175,"15min":1.23},"last":0.71799999999999997}]},{"osd":5,"last update":"Fri Mar 29 08:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.76000000000000001,"15min":0.77800000000000002},"min":{"1min":0.55800000000000005,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.0449999999999999,"5min":1.093,"15min":1.1499999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.76200000000000001,"15min":0.76400000000000001},"min":{"1min":0.64600000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.99099999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.68799999999999994}]},{"osd":6,"last update":"Fri Mar 29 08:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.77100000000000002,"15min":0.77200000000000002},"min":{"1min":0.64100000000000001,"5min":0.5,"15min":0.5},"max":{"1min":1.145,"5min":1.212,"15min":1.212},"last":0.624},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.78000000000000003,"15min":0.84399999999999997},"min":{"1min":0.66000000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.115,"5min":1.1459999999999999,"15min":1.196},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:24:11 2024","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.79500000000000004,"15min":0.80300000000000005},"min":{"1min":0.65900000000000003,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.1200000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.77500000000000002,"15min":0.79000000000000004},"min":{"1min":0.625,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.071,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.59699999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509524,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1138688,"data_stored":482682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.42899999999999999,"15min":0.57199999999999995},"min":{"1min":0.316,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.57299999999999995,"5min":0.71099999999999997,"15min":3.9359999999999999},"last":0.314},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.78500000000000003},"min":{"1min":0.33100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.69699999999999995,"5min":0.73799999999999999,"15min":8.0419999999999998},"last":0.45700000000000002}]},{"osd":2,"last update":"Fri Mar 29 08:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.501,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.70399999999999996,"5min":0.73399999999999999,"15min":1.258},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.52800000000000002,"15min":0.69799999999999995},"min":{"1min":0.48399999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.75700000000000001,"5min":0.78700000000000003,"15min":4.907},"last":0.53300000000000003}]},{"osd":3,"last update":"Fri Mar 29 08:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.49299999999999999,"15min":0.78500000000000003},"min":{"1min":0.39000000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68400000000000005,"5min":0.91000000000000003,"15min":7.9569999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.55400000000000005,"15min":0.64200000000000002},"min":{"1min":0.48099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77500000000000002,"5min":0.78300000000000003,"15min":1.397},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.77700000000000002,"15min":0.95399999999999996},"min":{"1min":0.63800000000000001,"5min":0.55100000000000005,"15min":0.53700000000000003},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":4.4720000000000004},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.754,"15min":0.92700000000000005},"min":{"1min":0.59899999999999998,"5min":0.498,"15min":0.498},"max":{"1min":1.115,"5min":1.115,"15min":4.3920000000000003},"last":0.83099999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.77500000000000002,"15min":0.90500000000000003},"min":{"1min":0.61599999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.349,"5min":1.349,"15min":4.3200000000000003},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.77400000000000002,"15min":0.85099999999999998},"min":{"1min":0.63900000000000001,"5min":0.55700000000000005,"15min":0.54600000000000004},"max":{"1min":1.26,"5min":1.26,"15min":1.6759999999999999},"last":0.65200000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.78400000000000003,"15min":0.85199999999999998},"min":{"1min":0.59399999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.5920000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.79200000000000004,"15min":0.95399999999999996},"min":{"1min":0.67800000000000005,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":4.4100000000000001},"last":0.71099999999999997}]},{"osd":7,"last update":"Fri Mar 29 08:24:10 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.79600000000000004,"15min":0.97099999999999997},"min":{"1min":0.70799999999999996,"5min":0.56999999999999995,"15min":0.496},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":4.5209999999999999},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.78500000000000003,"15min":0.95499999999999996},"min":{"1min":0.65500000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":4.4420000000000002},"last":0.69299999999999995}]}]},{"osd":5,"up_from":14,"seq":60129542229,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1720320,"data_stored":1066762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.68200000000000005,"15min":0.64500000000000002},"min":{"1min":0.49099999999999999,"5min":0.45000000000000001,"15min":0.42899999999999999},"max":{"1min":0.879,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.68799999999999994,"15min":0.66400000000000003},"min":{"1min":0.622,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67400000000000004,"15min":0.72399999999999998},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":2.5129999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.70199999999999996,"15min":0.68500000000000005},"min":{"1min":0.53400000000000003,"5min":0.44900000000000001,"15min":0.40500000000000003},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.78200000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.67900000000000005,"15min":0.67500000000000004},"min":{"1min":0.51700000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.91500000000000004,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.70999999999999996,"15min":0.70599999999999996},"min":{"1min":0.58099999999999996,"5min":0.44900000000000001,"15min":0.39200000000000002},"max":{"1min":0.86499999999999999,"5min":0.91400000000000003,"15min":1.0209999999999999},"last":0.59899999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.71899999999999997,"15min":0.71699999999999997},"min":{"1min":0.54600000000000004,"5min":0.49399999999999999,"15min":0.47099999999999997},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.99399999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71099999999999997,"15min":0.73899999999999999},"min":{"1min":0.51900000000000002,"5min":0.47799999999999998,"15min":0.435},"max":{"1min":0.86899999999999999,"5min":0.90500000000000003,"15min":1.954},"last":0.66300000000000003}]},{"osd":4,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63200000000000001,"15min":0.64700000000000002},"min":{"1min":0.46899999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.71999999999999997,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.628,"15min":0.65500000000000003},"min":{"1min":0.45600000000000002,"5min":0.373,"15min":0.373},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":1.0820000000000001},"last":0.74099999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.628,"15min":0.66900000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.83399999999999996,"5min":0.86899999999999999,"15min":1.071},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67200000000000004,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.81499999999999995,"5min":0.878,"15min":1.0609999999999999},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri Mar 29 08:24:12 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.67600000000000005,"15min":0.70899999999999996},"min":{"1min":0.51200000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.90100000000000002,"5min":0.92900000000000005,"15min":1.1200000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71299999999999997,"15min":0.749},"min":{"1min":0.497,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93100000000000005,"5min":0.95199999999999996,"15min":1.04},"last":0.85199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:24:47.979 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T08:24:47.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-29T08:24:48.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.165+0000 7fcd18200640 1 Processor -- start 2024-03-29T08:24:48.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.165+0000 7fcd18200640 1 -- start start 2024-03-29T08:24:48.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.165+0000 7fcd18200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcd10142fa0 0x7fcd100b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:48.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.165+0000 7fcd18200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcd1015dee0 0x7fcd100b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:48.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.165+0000 7fcd18200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcd1006d560 0x7fcd1006d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:48.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.165+0000 7fcd18200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd100b47c0 con 0x7fcd10142fa0 2024-03-29T08:24:48.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.165+0000 7fcd18200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1006def0 con 0x7fcd1006d560 2024-03-29T08:24:48.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.165+0000 7fcd18200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1015da50 con 0x7fcd1015dee0 2024-03-29T08:24:48.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.165+0000 7fcd15800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcd10142fa0 0x7fcd100b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:48.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.169+0000 7fcd15800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcd10142fa0 0x7fcd100b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34708/0 (socket says 172.21.15.46:34708) 2024-03-29T08:24:48.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.169+0000 7fcd15800640 1 -- 172.21.15.46:0/2636334550 learned_addr learned my addr 172.21.15.46:0/2636334550 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:48.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.169+0000 7fcd14e00640 1 --2- 172.21.15.46:0/2636334550 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcd1015dee0 0x7fcd100b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:48.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.169+0000 7fcd16200640 1 --2- 172.21.15.46:0/2636334550 >> v2:172.21.15.73:3300/0 conn(0x7fcd1006d560 0x7fcd1006d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:48.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.169+0000 7fcd15800640 1 -- 172.21.15.46:0/2636334550 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcd1015dee0 msgr2=0x7fcd100b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:48.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.169+0000 7fcd15800640 1 --2- 172.21.15.46:0/2636334550 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcd1015dee0 0x7fcd100b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.169+0000 7fcd15800640 1 -- 172.21.15.46:0/2636334550 >> v2:172.21.15.73:3300/0 conn(0x7fcd1006d560 msgr2=0x7fcd1006d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:48.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.169+0000 7fcd15800640 1 --2- 172.21.15.46:0/2636334550 >> v2:172.21.15.73:3300/0 conn(0x7fcd1006d560 0x7fcd1006d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.169+0000 7fcd15800640 1 -- 172.21.15.46:0/2636334550 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd100b44e0 con 0x7fcd10142fa0 2024-03-29T08:24:48.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.169+0000 7fcd16200640 1 --2- 172.21.15.46:0/2636334550 >> v2:172.21.15.73:3300/0 conn(0x7fcd1006d560 0x7fcd1006d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:24:48.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.169+0000 7fcd15800640 1 --2- 172.21.15.46:0/2636334550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd10142fa0 0x7fcd100b39b0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fcd1006e3b0 tx=0x7fcd0401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e501b052118a91 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:48.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.169+0000 7fcd0fe00640 1 -- 172.21.15.46:0/2636334550 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd040ac620 con 0x7fcd10142fa0 2024-03-29T08:24:48.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.169+0000 7fcd0fe00640 1 -- 172.21.15.46:0/2636334550 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd040ac780 con 0x7fcd10142fa0 2024-03-29T08:24:48.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.169+0000 7fcd0fe00640 1 -- 172.21.15.46:0/2636334550 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd040acb10 con 0x7fcd10142fa0 2024-03-29T08:24:48.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd18200640 1 -- 172.21.15.46:0/2636334550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd10142fa0 msgr2=0x7fcd100b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:48.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd18200640 1 --2- 172.21.15.46:0/2636334550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd10142fa0 0x7fcd100b39b0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fcd1006e3b0 tx=0x7fcd0401c4f0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd18200640 1 -- 172.21.15.46:0/2636334550 shutdown_connections 2024-03-29T08:24:48.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd18200640 1 --2- 172.21.15.46:0/2636334550 >> v2:172.21.15.73:3300/0 conn(0x7fcd1006d560 0x7fcd1006d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd18200640 1 --2- 172.21.15.46:0/2636334550 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcd1015dee0 0x7fcd100b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd18200640 1 --2- 172.21.15.46:0/2636334550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd10142fa0 0x7fcd100b39b0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd18200640 1 -- 172.21.15.46:0/2636334550 >> 172.21.15.46:0/2636334550 conn(0x7fcd1006a1c0 msgr2=0x7fcd1006a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:48.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd18200640 1 -- 172.21.15.46:0/2636334550 shutdown_connections 2024-03-29T08:24:48.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd18200640 1 -- 172.21.15.46:0/2636334550 wait complete. 2024-03-29T08:24:48.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd18200640 1 Processor -- start 2024-03-29T08:24:48.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd18200640 1 -- start start 2024-03-29T08:24:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd18200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcd1006d560 0x7fcd1013cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd18200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd10142fa0 0x7fcd101402d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd18200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1015dee0 0x7fcd1013f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd18200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd101418a0 con 0x7fcd1015dee0 2024-03-29T08:24:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd18200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd10140c90 con 0x7fcd10142fa0 2024-03-29T08:24:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd18200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd10140f70 con 0x7fcd1006d560 2024-03-29T08:24:48.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd16200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1015dee0 0x7fcd1013f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:48.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd15800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcd1006d560 0x7fcd1013cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:48.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd16200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1015dee0 0x7fcd1013f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34720/0 (socket says 172.21.15.46:34720) 2024-03-29T08:24:48.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd16200640 1 -- 172.21.15.46:0/1192792506 learned_addr learned my addr 172.21.15.46:0/1192792506 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:48.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd14e00640 1 --2- 172.21.15.46:0/1192792506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd10142fa0 0x7fcd101402d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:48.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd16200640 1 -- 172.21.15.46:0/1192792506 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcd1006d560 msgr2=0x7fcd1013cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:48.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd16200640 1 --2- 172.21.15.46:0/1192792506 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcd1006d560 0x7fcd1013cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd16200640 1 -- 172.21.15.46:0/1192792506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd10142fa0 msgr2=0x7fcd101402d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:48.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd16200640 1 --2- 172.21.15.46:0/1192792506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd10142fa0 0x7fcd101402d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd16200640 1 -- 172.21.15.46:0/1192792506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd040094d0 con 0x7fcd1015dee0 2024-03-29T08:24:48.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd15800640 1 --2- 172.21.15.46:0/1192792506 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcd1006d560 0x7fcd1013cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:24:48.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.173+0000 7fcd14e00640 1 --2- 172.21.15.46:0/1192792506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd10142fa0 0x7fcd101402d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:24:48.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.177+0000 7fcd16200640 1 --2- 172.21.15.46:0/1192792506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1015dee0 0x7fcd1013f040 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800cef0 tx=0x7fcd0800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:48.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.177+0000 7fcd0d600640 1 -- 172.21.15.46:0/1192792506 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0800d840 con 0x7fcd1015dee0 2024-03-29T08:24:48.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.177+0000 7fcd0d600640 1 -- 172.21.15.46:0/1192792506 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0800d9c0 con 0x7fcd1015dee0 2024-03-29T08:24:48.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.177+0000 7fcd18200640 1 -- 172.21.15.46:0/1192792506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd10141250 con 0x7fcd1015dee0 2024-03-29T08:24:48.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.177+0000 7fcd0d600640 1 -- 172.21.15.46:0/1192792506 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0800dd70 con 0x7fcd1015dee0 2024-03-29T08:24:48.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.177+0000 7fcd18200640 1 -- 172.21.15.46:0/1192792506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd10158ef0 con 0x7fcd1015dee0 2024-03-29T08:24:48.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.177+0000 7fcd18200640 1 -- 172.21.15.46:0/1192792506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd10141a50 con 0x7fcd1015dee0 2024-03-29T08:24:48.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.177+0000 7fcd0d600640 1 -- 172.21.15.46:0/1192792506 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcd0801e3b0 con 0x7fcd1015dee0 2024-03-29T08:24:48.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.177+0000 7fcd0d600640 1 --2- 172.21.15.46:0/1192792506 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fccf0066690 0x7fccf0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:48.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.177+0000 7fcd0d600640 1 -- 172.21.15.46:0/1192792506 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd08084ba0 con 0x7fcd1015dee0 2024-03-29T08:24:48.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.177+0000 7fcd15800640 1 --2- 172.21.15.46:0/1192792506 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fccf0066690 0x7fccf0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:48.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.181+0000 7fcd15800640 1 --2- 172.21.15.46:0/1192792506 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fccf0066690 0x7fccf0068b50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcd040ae000 tx=0x7fcd04008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.181+0000 7fcd0d600640 1 -- 172.21.15.46:0/1192792506 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcd10141a50 con 0x7fcd1015dee0 2024-03-29T08:24:48.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.397+0000 7fcd18200640 1 -- 172.21.15.46:0/1192792506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fcd10141a50 con 0x7fcd1015dee0 2024-03-29T08:24:48.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.405+0000 7fcd0d600640 1 -- 172.21.15.46:0/1192792506 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7fcd10141a50 con 0x7fcd1015dee0 2024-03-29T08:24:48.404 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-03-29T08:24:48.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.405+0000 7fcd18200640 1 -- 172.21.15.46:0/1192792506 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fccf0066690 msgr2=0x7fccf0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.405+0000 7fcd18200640 1 --2- 172.21.15.46:0/1192792506 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fccf0066690 0x7fccf0068b50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcd040ae000 tx=0x7fcd04008040 comp rx=0 tx=0).stop 2024-03-29T08:24:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.405+0000 7fcd18200640 1 -- 172.21.15.46:0/1192792506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1015dee0 msgr2=0x7fcd1013f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.405+0000 7fcd18200640 1 --2- 172.21.15.46:0/1192792506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1015dee0 0x7fcd1013f040 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800cef0 tx=0x7fcd0800a710 comp rx=0 tx=0).stop 2024-03-29T08:24:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.405+0000 7fcd18200640 1 -- 172.21.15.46:0/1192792506 shutdown_connections 2024-03-29T08:24:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.405+0000 7fcd18200640 1 --2- 172.21.15.46:0/1192792506 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fccf0066690 0x7fccf0068b50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.405+0000 7fcd18200640 1 --2- 172.21.15.46:0/1192792506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd10142fa0 0x7fcd101402d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.405+0000 7fcd18200640 1 --2- 172.21.15.46:0/1192792506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcd1015dee0 0x7fcd1013f040 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.405+0000 7fcd18200640 1 --2- 172.21.15.46:0/1192792506 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcd1006d560 0x7fcd1013cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.405+0000 7fcd18200640 1 -- 172.21.15.46:0/1192792506 >> 172.21.15.46:0/1192792506 conn(0x7fcd1006a1c0 msgr2=0x7fcd1013e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.405+0000 7fcd18200640 1 -- 172.21.15.46:0/1192792506 shutdown_connections 2024-03-29T08:24:48.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.405+0000 7fcd18200640 1 -- 172.21.15.46:0/1192792506 wait complete. 2024-03-29T08:24:48.424 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:24:48.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.565+0000 7f6abf600640 1 Processor -- start 2024-03-29T08:24:48.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.565+0000 7f6abf600640 1 -- start start 2024-03-29T08:24:48.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.565+0000 7f6abf600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ab8144210 0x7f6ab80cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:48.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.565+0000 7f6abf600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6ab815ada0 0x7f6ab80ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:48.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.565+0000 7f6abf600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ab80cb510 con 0x7f6ab80ce950 2024-03-29T08:24:48.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.565+0000 7f6abf600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ab80cede0 con 0x7f6ab815ada0 2024-03-29T08:24:48.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.565+0000 7f6abf600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab806d960 con 0x7f6ab8144210 2024-03-29T08:24:48.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.565+0000 7f6abd600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6ab80ce950 0x7f6ab815e360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42082/0 (socket says 172.21.15.46:42082) 2024-03-29T08:24:48.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.565+0000 7f6abd600640 1 -- 172.21.15.46:0/3207502726 learned_addr learned my addr 172.21.15.46:0/3207502726 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:48.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.565+0000 7f6ab7e00640 1 --2- 172.21.15.46:0/3207502726 >> v2:172.21.15.73:3300/0 conn(0x7f6ab815ada0 0x7f6ab80ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:48.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.565+0000 7f6abcc00640 1 --2- 172.21.15.46:0/3207502726 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ab8144210 0x7f6ab80cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:48.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.565+0000 7f6ab7400640 1 -- 172.21.15.46:0/3207502726 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2762317713 0 0) 0x7f6ab80cb510 con 0x7f6ab80ce950 2024-03-29T08:24:48.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.565+0000 7f6ab7400640 1 -- 172.21.15.46:0/3207502726 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6aa0003530 con 0x7f6ab80ce950 2024-03-29T08:24:48.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.565+0000 7f6ab7e00640 1 -- 172.21.15.46:0/3207502726 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ab8144210 msgr2=0x7f6ab80cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:48.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.565+0000 7f6ab7e00640 1 --2- 172.21.15.46:0/3207502726 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ab8144210 0x7f6ab80cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.565+0000 7f6ab7e00640 1 -- 172.21.15.46:0/3207502726 >> v1:172.21.15.46:6789/0 conn(0x7f6ab80ce950 legacy=0x7f6ab815e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:48.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.565+0000 7f6ab7e00640 1 -- 172.21.15.46:0/3207502726 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab8070580 con 0x7f6ab815ada0 2024-03-29T08:24:48.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.569+0000 7f6ab7e00640 1 --2- 172.21.15.46:0/3207502726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ab815ada0 0x7f6ab80ce410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40097c0 tx=0x7f6aa401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7993684412f8b49b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:48.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.569+0000 7f6ab7400640 1 -- 172.21.15.46:0/3207502726 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40ab600 con 0x7f6ab815ada0 2024-03-29T08:24:48.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.569+0000 7f6ab7400640 1 -- 172.21.15.46:0/3207502726 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa40a7050 con 0x7f6ab815ada0 2024-03-29T08:24:48.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.569+0000 7f6ab7400640 1 -- 172.21.15.46:0/3207502726 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40ab980 con 0x7f6ab815ada0 2024-03-29T08:24:48.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.569+0000 7f6abf600640 1 -- 172.21.15.46:0/3207502726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ab815ada0 msgr2=0x7f6ab80ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:48.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.569+0000 7f6abf600640 1 --2- 172.21.15.46:0/3207502726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ab815ada0 0x7f6ab80ce410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40097c0 tx=0x7f6aa401c4a0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.569+0000 7f6abf600640 1 -- 172.21.15.46:0/3207502726 shutdown_connections 2024-03-29T08:24:48.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.569+0000 7f6abf600640 1 --2- 172.21.15.46:0/3207502726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ab815ada0 0x7f6ab80ce410 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.569+0000 7f6abf600640 1 --2- 172.21.15.46:0/3207502726 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ab8144210 0x7f6ab80cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.569+0000 7f6abf600640 1 -- 172.21.15.46:0/3207502726 >> 172.21.15.46:0/3207502726 conn(0x7f6ab8066110 msgr2=0x7f6ab8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:48.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.569+0000 7f6abf600640 1 -- 172.21.15.46:0/3207502726 shutdown_connections 2024-03-29T08:24:48.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.569+0000 7f6abf600640 1 -- 172.21.15.46:0/3207502726 wait complete. 2024-03-29T08:24:48.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abf600640 1 Processor -- start 2024-03-29T08:24:48.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abf600640 1 -- start start 2024-03-29T08:24:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abf600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ab80ce950 0x7f6ab815a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abf600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ab8144210 0x7f6ab8142860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abf600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ab815ada0 0x7f6ab8142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abf600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab80b3590 con 0x7f6ab815ada0 2024-03-29T08:24:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abf600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab815b330 con 0x7f6ab80ce950 2024-03-29T08:24:48.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abf600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab80b32b0 con 0x7f6ab8144210 2024-03-29T08:24:48.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ab815ada0 0x7f6ab8142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:48.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ab815ada0 0x7f6ab8142da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34728/0 (socket says 172.21.15.46:34728) 2024-03-29T08:24:48.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abd600640 1 -- 172.21.15.46:0/3703808532 learned_addr learned my addr 172.21.15.46:0/3703808532 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:48.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6ab7e00640 1 --2- 172.21.15.46:0/3703808532 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ab8144210 0x7f6ab8142860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:48.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abcc00640 1 --2- 172.21.15.46:0/3703808532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ab80ce950 0x7f6ab815a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:48.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abd600640 1 -- 172.21.15.46:0/3703808532 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ab8144210 msgr2=0x7f6ab8142860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:48.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abd600640 1 --2- 172.21.15.46:0/3703808532 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ab8144210 0x7f6ab8142860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abd600640 1 -- 172.21.15.46:0/3703808532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ab80ce950 msgr2=0x7f6ab815a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:48.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abd600640 1 --2- 172.21.15.46:0/3703808532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ab80ce950 0x7f6ab815a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abd600640 1 -- 172.21.15.46:0/3703808532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa4009420 con 0x7f6ab815ada0 2024-03-29T08:24:48.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abd600640 1 --2- 172.21.15.46:0/3703808532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ab815ada0 0x7f6ab8142da0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000cb40 tx=0x7f6ab000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:48.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6ab4c00640 1 -- 172.21.15.46:0/3703808532 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab000aca0 con 0x7f6ab815ada0 2024-03-29T08:24:48.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6ab4c00640 1 -- 172.21.15.46:0/3703808532 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab000ae00 con 0x7f6ab815ada0 2024-03-29T08:24:48.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abf600640 1 -- 172.21.15.46:0/3703808532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab815b5d0 con 0x7f6ab815ada0 2024-03-29T08:24:48.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6ab4c00640 1 -- 172.21.15.46:0/3703808532 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab0012620 con 0x7f6ab815ada0 2024-03-29T08:24:48.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abf600640 1 -- 172.21.15.46:0/3703808532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab815a700 con 0x7f6ab815ada0 2024-03-29T08:24:48.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.573+0000 7f6abf600640 1 -- 172.21.15.46:0/3703808532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab80ce280 con 0x7f6ab815ada0 2024-03-29T08:24:48.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.577+0000 7f6ab4c00640 1 -- 172.21.15.46:0/3703808532 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6ab0007070 con 0x7f6ab815ada0 2024-03-29T08:24:48.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.577+0000 7f6ab4c00640 1 --2- 172.21.15.46:0/3703808532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a98066690 0x7f6a98068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:48.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.577+0000 7f6ab4c00640 1 -- 172.21.15.46:0/3703808532 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ab0084130 con 0x7f6ab815ada0 2024-03-29T08:24:48.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.577+0000 7f6abcc00640 1 --2- 172.21.15.46:0/3703808532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a98066690 0x7f6a98068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:48.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.577+0000 7f6abcc00640 1 --2- 172.21.15.46:0/3703808532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a98066690 0x7f6a98068b50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009b60 tx=0x7f6aac009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:48.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.581+0000 7f6ab4c00640 1 -- 172.21.15.46:0/3703808532 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6ab0051540 con 0x7f6ab815ada0 2024-03-29T08:24:48.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.777+0000 7f6abf600640 1 -- 172.21.15.46:0/3703808532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ab8066c90 con 0x7f6ab815ada0 2024-03-29T08:24:48.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.777+0000 7f6ab4c00640 1 -- 172.21.15.46:0/3703808532 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f6ab0054b60 con 0x7f6ab815ada0 2024-03-29T08:24:48.778 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:48.778 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7386,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2051429508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2051429508},{"type":"v1","addr":"172.21.15.46:6835","nonce":2051429508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:24:43.476334+0000","modified":"2024-03-29T08:24:46.492649+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7395":{"gid":7395,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/4166845858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4166845858},{"type":"v1","addr":"172.21.15.73:6836","nonce":4166845858}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-29T08:24:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 57 2024-03-29T08:24:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.781+0000 7f6abf600640 1 -- 172.21.15.46:0/3703808532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a98066690 msgr2=0x7f6a98068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.781+0000 7f6abf600640 1 --2- 172.21.15.46:0/3703808532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a98066690 0x7f6a98068b50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6aac009b60 tx=0x7f6aac009190 comp rx=0 tx=0).stop 2024-03-29T08:24:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.781+0000 7f6abf600640 1 -- 172.21.15.46:0/3703808532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ab815ada0 msgr2=0x7f6ab8142da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.781+0000 7f6abf600640 1 --2- 172.21.15.46:0/3703808532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ab815ada0 0x7f6ab8142da0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f6ab000cb40 tx=0x7f6ab000a060 comp rx=0 tx=0).stop 2024-03-29T08:24:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.781+0000 7f6abf600640 1 -- 172.21.15.46:0/3703808532 shutdown_connections 2024-03-29T08:24:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.781+0000 7f6abf600640 1 --2- 172.21.15.46:0/3703808532 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6a98066690 0x7f6a98068b50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.781+0000 7f6abf600640 1 --2- 172.21.15.46:0/3703808532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ab815ada0 0x7f6ab8142da0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.781+0000 7f6abf600640 1 --2- 172.21.15.46:0/3703808532 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6ab8144210 0x7f6ab8142860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.781+0000 7f6abf600640 1 --2- 172.21.15.46:0/3703808532 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ab80ce950 0x7f6ab815a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.781+0000 7f6abf600640 1 -- 172.21.15.46:0/3703808532 >> 172.21.15.46:0/3703808532 conn(0x7f6ab8066110 msgr2=0x7f6ab8140670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:48.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.781+0000 7f6abf600640 1 -- 172.21.15.46:0/3703808532 shutdown_connections 2024-03-29T08:24:48.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:48.781+0000 7f6abf600640 1 -- 172.21.15.46:0/3703808532 wait complete. 2024-03-29T08:24:49.068 INFO:tasks.ceph.mds.b.smithi073.stderr:2024-03-29T08:24:49.067+0000 7f7366400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:24:49.201 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:24:43.476334+0000', 'modified': '2024-03-29T08:24:46.492649+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7395}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7395': {'gid': 7395, 'name': 'b', 'rank': 0, 'incarnation': 53, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6836/4166845858', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 4166845858}, {'type': 'v1', 'addr': '172.21.15.73:6836', 'nonce': 4166845858}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:24:49.202 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T08:24:49.202 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-29T08:24:49.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.345+0000 7f6ce2e00640 1 Processor -- start 2024-03-29T08:24:49.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.345+0000 7f6ce2e00640 1 -- start start 2024-03-29T08:24:49.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.349+0000 7f6ce2e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6cdc0af530 0x7f6cdc0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.349+0000 7f6ce2e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6cdc0702a0 0x7f6cdc070730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.349+0000 7f6ce2e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cdc06c390 con 0x7f6cdc158300 2024-03-29T08:24:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.349+0000 7f6ce2e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cdc070c70 con 0x7f6cdc0af530 2024-03-29T08:24:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.349+0000 7f6ce2e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc0afff0 con 0x7f6cdc0702a0 2024-03-29T08:24:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.349+0000 7f6cdbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6cdc158300 0x7f6cdc06d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42084/0 (socket says 172.21.15.46:42084) 2024-03-29T08:24:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.349+0000 7f6cdbe00640 1 -- 172.21.15.46:0/1010258964 learned_addr learned my addr 172.21.15.46:0/1010258964 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.349+0000 7f6cd3400640 1 --2- 172.21.15.46:0/1010258964 >> v2:172.21.15.46:3300/0 conn(0x7f6cdc0af530 0x7f6cdc0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.349+0000 7f6ce0e00640 1 --2- 172.21.15.46:0/1010258964 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6cdc0702a0 0x7f6cdc070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.349+0000 7f6cdb400640 1 -- 172.21.15.46:0/1010258964 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80837153 0 0) 0x7f6cdc06c390 con 0x7f6cdc158300 2024-03-29T08:24:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.349+0000 7f6cdb400640 1 -- 172.21.15.46:0/1010258964 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cc0003530 con 0x7f6cdc158300 2024-03-29T08:24:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.349+0000 7f6cd3400640 1 -- 172.21.15.46:0/1010258964 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6cdc0702a0 msgr2=0x7f6cdc070730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.349+0000 7f6cd3400640 1 --2- 172.21.15.46:0/1010258964 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6cdc0702a0 0x7f6cdc070730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.349+0000 7f6cd3400640 1 -- 172.21.15.46:0/1010258964 >> v1:172.21.15.46:6789/0 conn(0x7f6cdc158300 legacy=0x7f6cdc06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:49.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.349+0000 7f6cd3400640 1 -- 172.21.15.46:0/1010258964 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc06c5e0 con 0x7f6cdc0af530 2024-03-29T08:24:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6cd3400640 1 --2- 172.21.15.46:0/1010258964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cdc0af530 0x7f6cdc0af9a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40097c0 tx=0x7f6cc401c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1aafbb73a68ac9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:49.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6cdb400640 1 -- 172.21.15.46:0/1010258964 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4004230 con 0x7f6cdc0af530 2024-03-29T08:24:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6cdb400640 1 -- 172.21.15.46:0/1010258964 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc40a7050 con 0x7f6cdc0af530 2024-03-29T08:24:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6cdb400640 1 -- 172.21.15.46:0/1010258964 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4003620 con 0x7f6cdc0af530 2024-03-29T08:24:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6ce2e00640 1 -- 172.21.15.46:0/1010258964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cdc0af530 msgr2=0x7f6cdc0af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:49.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6ce2e00640 1 --2- 172.21.15.46:0/1010258964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cdc0af530 0x7f6cdc0af9a0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40097c0 tx=0x7f6cc401c350 comp rx=0 tx=0).stop 2024-03-29T08:24:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6ce2e00640 1 -- 172.21.15.46:0/1010258964 shutdown_connections 2024-03-29T08:24:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6ce2e00640 1 --2- 172.21.15.46:0/1010258964 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6cdc0702a0 0x7f6cdc070730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6ce2e00640 1 --2- 172.21.15.46:0/1010258964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cdc0af530 0x7f6cdc0af9a0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6ce2e00640 1 -- 172.21.15.46:0/1010258964 >> 172.21.15.46:0/1010258964 conn(0x7f6cdc066080 msgr2=0x7f6cdc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6ce2e00640 1 -- 172.21.15.46:0/1010258964 shutdown_connections 2024-03-29T08:24:49.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6ce2e00640 1 -- 172.21.15.46:0/1010258964 wait complete. 2024-03-29T08:24:49.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6ce2e00640 1 Processor -- start 2024-03-29T08:24:49.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6ce2e00640 1 -- start start 2024-03-29T08:24:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6ce2e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6cdc0702a0 0x7f6cdc156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6ce2e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cdc0af530 0x7f6cdc14dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6ce2e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cdc158300 0x7f6cdc1553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6ce2e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc156bd0 con 0x7f6cdc158300 2024-03-29T08:24:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6ce2e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc14e4e0 con 0x7f6cdc0af530 2024-03-29T08:24:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.353+0000 7f6ce2e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cdc155f50 con 0x7f6cdc0702a0 2024-03-29T08:24:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.357+0000 7f6ce0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cdc158300 0x7f6cdc1553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.357+0000 7f6ce0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cdc158300 0x7f6cdc1553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34738/0 (socket says 172.21.15.46:34738) 2024-03-29T08:24:49.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.357+0000 7f6ce0e00640 1 -- 172.21.15.46:0/3087195123 learned_addr learned my addr 172.21.15.46:0/3087195123 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:49.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.357+0000 7f6cdbe00640 1 --2- 172.21.15.46:0/3087195123 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6cdc0702a0 0x7f6cdc156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.357+0000 7f6cd3400640 1 --2- 172.21.15.46:0/3087195123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cdc0af530 0x7f6cdc14dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.357+0000 7f6ce0e00640 1 -- 172.21.15.46:0/3087195123 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6cdc0702a0 msgr2=0x7f6cdc156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:49.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.357+0000 7f6ce0e00640 1 --2- 172.21.15.46:0/3087195123 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6cdc0702a0 0x7f6cdc156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.357+0000 7f6ce0e00640 1 -- 172.21.15.46:0/3087195123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cdc0af530 msgr2=0x7f6cdc14dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:49.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.357+0000 7f6ce0e00640 1 --2- 172.21.15.46:0/3087195123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cdc0af530 0x7f6cdc14dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.357+0000 7f6ce0e00640 1 -- 172.21.15.46:0/3087195123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc4009420 con 0x7f6cdc158300 2024-03-29T08:24:49.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.357+0000 7f6ce0e00640 1 --2- 172.21.15.46:0/3087195123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cdc158300 0x7f6cdc1553e0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400efc0 tx=0x7f6cd400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.357+0000 7f6cd8c00640 1 -- 172.21.15.46:0/3087195123 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4014480 con 0x7f6cdc158300 2024-03-29T08:24:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.357+0000 7f6cd8c00640 1 -- 172.21.15.46:0/3087195123 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd400f040 con 0x7f6cdc158300 2024-03-29T08:24:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.357+0000 7f6cd8c00640 1 -- 172.21.15.46:0/3087195123 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4014820 con 0x7f6cdc158300 2024-03-29T08:24:49.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.357+0000 7f6ce2e00640 1 -- 172.21.15.46:0/3087195123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cdc156230 con 0x7f6cdc158300 2024-03-29T08:24:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.357+0000 7f6ce2e00640 1 -- 172.21.15.46:0/3087195123 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cdc1567f0 con 0x7f6cdc158300 2024-03-29T08:24:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.357+0000 7f6cd8c00640 1 -- 172.21.15.46:0/3087195123 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6cd4014980 con 0x7f6cdc158300 2024-03-29T08:24:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cd8c00640 1 --2- 172.21.15.46:0/3087195123 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6cb8066690 0x7f6cb8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cd8c00640 1 -- 172.21.15.46:0/3087195123 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6cd4084cb0 con 0x7f6cdc158300 2024-03-29T08:24:49.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cdbe00640 1 --2- 172.21.15.46:0/3087195123 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6cb8066690 0x7f6cb8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce634e000 1 Processor -- start 2024-03-29T08:24:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce634e000 1 -- start start 2024-03-29T08:24:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cdbe00640 1 --2- 172.21.15.46:0/3087195123 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6cb8066690 0x7f6cb8068b50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0011f0 tx=0x7f6ccc007d40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce634e000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa0e7db0 0x5621aa0e8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce634e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa0e86d0 0x5621aa09df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce634e000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa09e4c0 0x5621aa09e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce634e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621a9f665a0 con 0x5621aa0e86d0 2024-03-29T08:24:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce634e000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621a9ef3af0 con 0x5621aa09e4c0 2024-03-29T08:24:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce634e000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621a9ef10c0 con 0x5621aa0e7db0 2024-03-29T08:24:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cd3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa0e86d0 0x5621aa09df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cd3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa0e86d0 0x5621aa09df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34742/0 (socket says 172.21.15.46:34742) 2024-03-29T08:24:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cd3400640 1 -- 172.21.15.46:0/226853563 learned_addr learned my addr 172.21.15.46:0/226853563 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cdbe00640 1 --2- 172.21.15.46:0/226853563 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa0e7db0 0x5621aa0e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce0e00640 1 --2- 172.21.15.46:0/226853563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa09e4c0 0x5621aa09e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cd3400640 1 -- 172.21.15.46:0/226853563 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa0e7db0 msgr2=0x5621aa0e8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cd3400640 1 --2- 172.21.15.46:0/226853563 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa0e7db0 0x5621aa0e8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cd3400640 1 -- 172.21.15.46:0/226853563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa09e4c0 msgr2=0x5621aa09e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cd3400640 1 --2- 172.21.15.46:0/226853563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa09e4c0 0x5621aa09e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cd3400640 1 -- 172.21.15.46:0/226853563 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621a9be8890 con 0x5621aa0e86d0 2024-03-29T08:24:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cdbe00640 1 --2- 172.21.15.46:0/226853563 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa0e7db0 0x5621aa0e8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce0e00640 1 --2- 172.21.15.46:0/226853563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa09e4c0 0x5621aa09e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:49.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cd3400640 1 --2- 172.21.15.46:0/226853563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa0e86d0 0x5621aa09df80 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4000c10 tx=0x7f6cc4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1861f67d2dac3e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cd0c00640 1 -- 172.21.15.46:0/226853563 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4004370 con 0x5621aa0e86d0 2024-03-29T08:24:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cd0c00640 1 -- 172.21.15.46:0/226853563 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc40044d0 con 0x5621aa0e86d0 2024-03-29T08:24:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6cd0c00640 1 -- 172.21.15.46:0/226853563 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4003860 con 0x5621aa0e86d0 2024-03-29T08:24:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce634e000 1 -- 172.21.15.46:0/226853563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa0e86d0 msgr2=0x5621aa09df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce634e000 1 --2- 172.21.15.46:0/226853563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa0e86d0 0x5621aa09df80 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4000c10 tx=0x7f6cc4004740 comp rx=0 tx=0).stop 2024-03-29T08:24:49.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce634e000 1 -- 172.21.15.46:0/226853563 shutdown_connections 2024-03-29T08:24:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce634e000 1 --2- 172.21.15.46:0/226853563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa09e4c0 0x5621aa09e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce634e000 1 --2- 172.21.15.46:0/226853563 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa0e86d0 0x5621aa09df80 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce634e000 1 --2- 172.21.15.46:0/226853563 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa0e7db0 0x5621aa0e8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce634e000 1 -- 172.21.15.46:0/226853563 >> 172.21.15.46:0/226853563 conn(0x5621aa0a4830 msgr2=0x5621aa09c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce634e000 1 -- 172.21.15.46:0/226853563 shutdown_connections 2024-03-29T08:24:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.361+0000 7f6ce634e000 1 -- 172.21.15.46:0/226853563 wait complete. 2024-03-29T08:24:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 1 -- start start 2024-03-29T08:24:49.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 1 Processor -- start 2024-03-29T08:24:49.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa09e4c0 0x5621aa0e75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa0e7db0 0x5621aa0a2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa0e86d0 0x5621aa0a2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621a9eec9b0 con 0x5621aa0e86d0 2024-03-29T08:24:49.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621a9e8bc00 con 0x5621aa0e7db0 2024-03-29T08:24:49.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621a9eaa070 con 0x5621aa09e4c0 2024-03-29T08:24:49.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 10 client.0 _ll_register_callbacks cb 0x5621aa08a0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:24:49.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa0e86d0 0x5621aa0a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa0e86d0 0x5621aa0a2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34754/0 (socket says 172.21.15.46:34754) 2024-03-29T08:24:49.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce0e00640 1 -- 172.21.15.46:0/2990975077 learned_addr learned my addr 172.21.15.46:0/2990975077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6cd3400640 1 --2- 172.21.15.46:0/2990975077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa0e7db0 0x5621aa0a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6cdbe00640 1 --2- 172.21.15.46:0/2990975077 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa09e4c0 0x5621aa0e75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce0e00640 1 -- 172.21.15.46:0/2990975077 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa09e4c0 msgr2=0x5621aa0e75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce0e00640 1 --2- 172.21.15.46:0/2990975077 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa09e4c0 0x5621aa0e75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce0e00640 1 -- 172.21.15.46:0/2990975077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa0e7db0 msgr2=0x5621aa0a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce0e00640 1 --2- 172.21.15.46:0/2990975077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa0e7db0 0x5621aa0a2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce0e00640 1 -- 172.21.15.46:0/2990975077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cdc06c5e0 con 0x5621aa0e86d0 2024-03-29T08:24:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce0e00640 1 --2- 172.21.15.46:0/2990975077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa0e86d0 0x5621aa0a2770 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009a40 tx=0x7f6cd4059b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b0c96efc2ec175 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6cc9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:24:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6cc9600640 1 -- 172.21.15.46:0/2990975077 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4054150 con 0x5621aa0e86d0 2024-03-29T08:24:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 1 -- 172.21.15.46:0/2990975077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621a9ea95f0 con 0x5621aa0e86d0 2024-03-29T08:24:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6cc9600640 1 -- 172.21.15.46:0/2990975077 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6cd40551e0 con 0x5621aa0e86d0 2024-03-29T08:24:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6cc9600640 1 -- 172.21.15.46:0/2990975077 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cd40553a0 con 0x5621aa0e86d0 2024-03-29T08:24:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 10 client.7539 fetch_fsmap learned FSMap version 57 2024-03-29T08:24:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 1 -- 172.21.15.46:0/2990975077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5621a9eb1ab0 con 0x5621aa0e86d0 2024-03-29T08:24:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6cc9600640 1 -- 172.21.15.46:0/2990975077 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5621a9ea95f0 con 0x5621aa0e86d0 2024-03-29T08:24:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6cc9600640 1 -- 172.21.15.46:0/2990975077 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 57) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6cd4005d20 con 0x5621aa0e86d0 2024-03-29T08:24:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 10 client.7539 fetch_fsmap finished waiting for FSMap version 57 2024-03-29T08:24:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 10 client.7539 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-29T08:24:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 20 client.7539 populate_metadata read hostname 'smithi046' 2024-03-29T08:24:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 1 --2- 172.21.15.46:0/2990975077 >> [v2:172.21.15.73:6834/4166845858,v1:172.21.15.73:6836/4166845858] conn(0x5621aa0a3180 0x5621aa0a3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 4 client.7539 mds_command: new command op to 7395 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:24:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6ce634e000 1 -- 172.21.15.46:0/2990975077 --> [v2:172.21.15.73:6834/4166845858,v1:172.21.15.73:6836/4166845858] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621a9ee9790 con 0x5621aa0a3180 2024-03-29T08:24:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.365+0000 7f6cdbe00640 1 --2- 172.21.15.46:0/2990975077 >> [v2:172.21.15.73:6834/4166845858,v1:172.21.15.73:6836/4166845858] conn(0x5621aa0a3180 0x5621aa0a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.369+0000 7f6cdbe00640 1 --2- 172.21.15.46:0/2990975077 >> [v2:172.21.15.73:6834/4166845858,v1:172.21.15.73:6836/4166845858] conn(0x5621aa0a3180 0x5621aa0a3560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8e841404c8aaf99 server_cookie=a3a3cdd89fb1ff20 in_seq=0 out_seq=0 2024-03-29T08:24:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.369+0000 7f6cc9600640 10 client.7539 ms_handle_connect on v2:172.21.15.73:6834/4166845858 2024-03-29T08:24:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.369+0000 7f6cc9600640 1 -- 172.21.15.46:0/2990975077 <== mds.0 v2:172.21.15.73:6834/4166845858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5621a9ee9790 con 0x5621aa0a3180 2024-03-29T08:24:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.369+0000 7f6cc9600640 10 client.7539 handle_command_reply: tid=0 2024-03-29T08:24:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.369+0000 7f6ce634e000 1 client.7539 shutdown 2024-03-29T08:24:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.369+0000 7f6ce634e000 1 -- 172.21.15.46:0/2990975077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa0e86d0 msgr2=0x5621aa0a2770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.369+0000 7f6ce634e000 1 --2- 172.21.15.46:0/2990975077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa0e86d0 0x5621aa0a2770 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4009a40 tx=0x7f6cd4059b30 comp rx=0 tx=0).stop 2024-03-29T08:24:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.369+0000 7f6ce634e000 1 -- 172.21.15.46:0/2990975077 shutdown_connections 2024-03-29T08:24:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.369+0000 7f6ce634e000 1 --2- 172.21.15.46:0/2990975077 >> [v2:172.21.15.73:6834/4166845858,v1:172.21.15.73:6836/4166845858] conn(0x5621aa0a3180 0x5621aa0a3560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.369+0000 7f6ce634e000 1 --2- 172.21.15.46:0/2990975077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa0e7db0 0x5621aa0a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.369+0000 7f6ce634e000 1 --2- 172.21.15.46:0/2990975077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa0e86d0 0x5621aa0a2770 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.369+0000 7f6ce634e000 1 --2- 172.21.15.46:0/2990975077 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa09e4c0 0x5621aa0e75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.369+0000 7f6ce634e000 1 -- 172.21.15.46:0/2990975077 >> 172.21.15.46:0/2990975077 conn(0x5621aa0a4830 msgr2=0x5621aa09cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.369+0000 7f6cc9600640 0 client.7539 ms_handle_reset on v2:172.21.15.73:6834/4166845858 2024-03-29T08:24:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.369+0000 7f6ce634e000 1 -- 172.21.15.46:0/2990975077 shutdown_connections 2024-03-29T08:24:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.369+0000 7f6ce634e000 1 -- 172.21.15.46:0/2990975077 wait complete. 2024-03-29T08:24:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.369+0000 7f6ce634e000 20 client.7539 trim_cache size 0 max 16384 2024-03-29T08:24:49.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6ce634e000 1 Processor -- start 2024-03-29T08:24:49.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6ce634e000 1 -- start start 2024-03-29T08:24:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6ce634e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa09e4c0 0x5621aa141860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6ce634e000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa0e7db0 0x5621aa141da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6ce634e000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa0e86d0 0x5621aa1422e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6ce634e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621a9ba9940 con 0x5621aa09e4c0 2024-03-29T08:24:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6ce634e000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621a9ee6bb0 con 0x5621aa0e86d0 2024-03-29T08:24:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6ce634e000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621a9eea1c0 con 0x5621aa0e7db0 2024-03-29T08:24:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6cdbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa09e4c0 0x5621aa141860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6cdbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa09e4c0 0x5621aa141860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34764/0 (socket says 172.21.15.46:34764) 2024-03-29T08:24:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6cdbe00640 1 -- 172.21.15.46:0/3054238813 learned_addr learned my addr 172.21.15.46:0/3054238813 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:49.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6ce0e00640 1 --2- 172.21.15.46:0/3054238813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa0e86d0 0x5621aa1422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6cd3400640 1 --2- 172.21.15.46:0/3054238813 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa0e7db0 0x5621aa141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6cdbe00640 1 -- 172.21.15.46:0/3054238813 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa0e7db0 msgr2=0x5621aa141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6cdbe00640 1 --2- 172.21.15.46:0/3054238813 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa0e7db0 0x5621aa141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6cdbe00640 1 -- 172.21.15.46:0/3054238813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa0e86d0 msgr2=0x5621aa1422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6cdbe00640 1 --2- 172.21.15.46:0/3054238813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa0e86d0 0x5621aa1422e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6cdbe00640 1 -- 172.21.15.46:0/3054238813 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621a9e97c00 con 0x5621aa09e4c0 2024-03-29T08:24:49.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6cdbe00640 1 --2- 172.21.15.46:0/3054238813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa09e4c0 0x5621aa141860 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc005cd0 tx=0x7f6ccc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232cbd4a0fcc3cc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:49.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6cd1600640 1 -- 172.21.15.46:0/3054238813 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc001450 con 0x5621aa09e4c0 2024-03-29T08:24:49.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6cd1600640 1 -- 172.21.15.46:0/3054238813 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc0015b0 con 0x5621aa09e4c0 2024-03-29T08:24:49.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6cd1600640 1 -- 172.21.15.46:0/3054238813 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc001720 con 0x5621aa09e4c0 2024-03-29T08:24:49.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6ce634e000 1 -- 172.21.15.46:0/3054238813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa09e4c0 msgr2=0x5621aa141860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:49.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6ce634e000 1 --2- 172.21.15.46:0/3054238813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa09e4c0 0x5621aa141860 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc005cd0 tx=0x7f6ccc012000 comp rx=0 tx=0).stop 2024-03-29T08:24:49.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6ce634e000 1 -- 172.21.15.46:0/3054238813 shutdown_connections 2024-03-29T08:24:49.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6ce634e000 1 --2- 172.21.15.46:0/3054238813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa0e86d0 0x5621aa1422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6ce634e000 1 --2- 172.21.15.46:0/3054238813 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa0e7db0 0x5621aa141da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6ce634e000 1 --2- 172.21.15.46:0/3054238813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa09e4c0 0x5621aa141860 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.385+0000 7f6ce634e000 1 -- 172.21.15.46:0/3054238813 >> 172.21.15.46:0/3054238813 conn(0x5621aa0a4830 msgr2=0x5621aa0eb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:49.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 1 -- 172.21.15.46:0/3054238813 shutdown_connections 2024-03-29T08:24:49.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 1 -- 172.21.15.46:0/3054238813 wait complete. 2024-03-29T08:24:49.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 1 -- start start 2024-03-29T08:24:49.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 1 Processor -- start 2024-03-29T08:24:49.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa09e4c0 0x5621aa146750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa0e7db0 0x5621aa146c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa0e86d0 0x5621aa0e5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621a9f2fac0 con 0x5621aa09e4c0 2024-03-29T08:24:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621a9ee9440 con 0x5621aa0e86d0 2024-03-29T08:24:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621a9ee9790 con 0x5621aa0e7db0 2024-03-29T08:24:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 10 client.0 _ll_register_callbacks cb 0x5621a9eb1590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:24:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6cdbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa09e4c0 0x5621aa146750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6cdbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa09e4c0 0x5621aa146750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34780/0 (socket says 172.21.15.46:34780) 2024-03-29T08:24:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6cdbe00640 1 -- 172.21.15.46:0/3820301107 learned_addr learned my addr 172.21.15.46:0/3820301107 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6cd3400640 1 --2- 172.21.15.46:0/3820301107 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa0e7db0 0x5621aa146c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce0e00640 1 --2- 172.21.15.46:0/3820301107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa0e86d0 0x5621aa0e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6cdbe00640 1 -- 172.21.15.46:0/3820301107 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa0e7db0 msgr2=0x5621aa146c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6cdbe00640 1 --2- 172.21.15.46:0/3820301107 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa0e7db0 0x5621aa146c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6cdbe00640 1 -- 172.21.15.46:0/3820301107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa0e86d0 msgr2=0x5621aa0e5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6cdbe00640 1 --2- 172.21.15.46:0/3820301107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa0e86d0 0x5621aa0e5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6cdbe00640 1 -- 172.21.15.46:0/3820301107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc0003530 con 0x5621aa09e4c0 2024-03-29T08:24:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce0e00640 1 --2- 172.21.15.46:0/3820301107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa0e86d0 0x5621aa0e5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6cdbe00640 1 --2- 172.21.15.46:0/3820301107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa09e4c0 0x5621aa146750 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc006ce0 tx=0x7f6ccc000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9b6c34eb4e3175 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6cc9600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:24:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6cc9600640 1 -- 172.21.15.46:0/3820301107 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc028070 con 0x5621aa09e4c0 2024-03-29T08:24:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6cc9600640 1 -- 172.21.15.46:0/3820301107 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ccc01b040 con 0x5621aa09e4c0 2024-03-29T08:24:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 1 -- 172.21.15.46:0/3820301107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621a9ebc460 con 0x5621aa09e4c0 2024-03-29T08:24:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6cc9600640 1 -- 172.21.15.46:0/3820301107 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ccc005e00 con 0x5621aa09e4c0 2024-03-29T08:24:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 10 client.7545 fetch_fsmap learned FSMap version 57 2024-03-29T08:24:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 1 -- 172.21.15.46:0/3820301107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5621a9be8cc0 con 0x5621aa09e4c0 2024-03-29T08:24:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6cc9600640 1 -- 172.21.15.46:0/3820301107 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5621a9ebc460 con 0x5621aa09e4c0 2024-03-29T08:24:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6cc9600640 1 -- 172.21.15.46:0/3820301107 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 57) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6ccc039070 con 0x5621aa09e4c0 2024-03-29T08:24:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 10 client.7545 fetch_fsmap finished waiting for FSMap version 57 2024-03-29T08:24:49.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 10 client.7545 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-29T08:24:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 20 client.7545 populate_metadata read hostname 'smithi046' 2024-03-29T08:24:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 1 --2- 172.21.15.46:0/3820301107 >> [v2:172.21.15.73:6834/4166845858,v1:172.21.15.73:6836/4166845858] conn(0x5621aa0e5a80 0x5621aa0e5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 4 client.7545 mds_command: new command op to 7395 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T08:24:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.389+0000 7f6ce634e000 1 -- 172.21.15.46:0/3820301107 --> [v2:172.21.15.73:6834/4166845858,v1:172.21.15.73:6836/4166845858] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5621a9eb59c0 con 0x5621aa0e5a80 2024-03-29T08:24:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6cd3400640 1 --2- 172.21.15.46:0/3820301107 >> [v2:172.21.15.73:6834/4166845858,v1:172.21.15.73:6836/4166845858] conn(0x5621aa0e5a80 0x5621aa0e5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6cd3400640 1 --2- 172.21.15.46:0/3820301107 >> [v2:172.21.15.73:6834/4166845858,v1:172.21.15.73:6836/4166845858] conn(0x5621aa0e5a80 0x5621aa0e5e60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7188d186d4c662ba server_cookie=986ed173c5394f6f in_seq=0 out_seq=0 2024-03-29T08:24:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6cc9600640 10 client.7545 ms_handle_connect on v2:172.21.15.73:6834/4166845858 2024-03-29T08:24:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6cc9600640 1 -- 172.21.15.46:0/3820301107 <== mds.0 v2:172.21.15.73:6834/4166845858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5621a9eb59c0 con 0x5621aa0e5a80 2024-03-29T08:24:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6cc9600640 10 client.7545 handle_command_reply: tid=0 2024-03-29T08:24:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce634e000 1 client.7545 shutdown 2024-03-29T08:24:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce634e000 1 -- 172.21.15.46:0/3820301107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa09e4c0 msgr2=0x5621aa146750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce634e000 1 --2- 172.21.15.46:0/3820301107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa09e4c0 0x5621aa146750 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc006ce0 tx=0x7f6ccc000ee0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce634e000 1 -- 172.21.15.46:0/3820301107 shutdown_connections 2024-03-29T08:24:49.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce634e000 1 --2- 172.21.15.46:0/3820301107 >> [v2:172.21.15.73:6834/4166845858,v1:172.21.15.73:6836/4166845858] conn(0x5621aa0e5a80 0x5621aa0e5e60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce634e000 1 --2- 172.21.15.46:0/3820301107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621aa0e86d0 0x5621aa0e5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce634e000 1 --2- 172.21.15.46:0/3820301107 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621aa0e7db0 0x5621aa146c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce634e000 1 --2- 172.21.15.46:0/3820301107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621aa09e4c0 0x5621aa146750 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce634e000 1 -- 172.21.15.46:0/3820301107 >> 172.21.15.46:0/3820301107 conn(0x5621aa0a4830 msgr2=0x5621aa0ec0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:49.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6cc9600640 0 client.7545 ms_handle_reset on v2:172.21.15.73:6834/4166845858 2024-03-29T08:24:49.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce634e000 1 -- 172.21.15.46:0/3820301107 shutdown_connections 2024-03-29T08:24:49.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce634e000 1 -- 172.21.15.46:0/3820301107 wait complete. 2024-03-29T08:24:49.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce634e000 20 client.7545 trim_cache size 0 max 16384 2024-03-29T08:24:49.396 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:24:49.396 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "e9b7960f-ec1f-4787-95c3-c806094d37f0", 2024-03-29T08:24:49.396 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-03-29T08:24:49.397 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 7395, 2024-03-29T08:24:49.397 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-29T08:24:49.397 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-29T08:24:49.397 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-29T08:24:49.397 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 5.9149974370000002, 2024-03-29T08:24:49.397 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 57, 2024-03-29T08:24:49.397 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 92, 2024-03-29T08:24:49.397 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 92, 2024-03-29T08:24:49.397 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 11.136303196 2024-03-29T08:24:49.397 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:24:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce2e00640 1 -- 172.21.15.46:0/3087195123 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6cb8066690 msgr2=0x7f6cb8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce2e00640 1 --2- 172.21.15.46:0/3087195123 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6cb8066690 0x7f6cb8068b50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc0011f0 tx=0x7f6ccc007d40 comp rx=0 tx=0).stop 2024-03-29T08:24:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce2e00640 1 -- 172.21.15.46:0/3087195123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cdc158300 msgr2=0x7f6cdc1553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce2e00640 1 --2- 172.21.15.46:0/3087195123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cdc158300 0x7f6cdc1553e0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400efc0 tx=0x7f6cd400c490 comp rx=0 tx=0).stop 2024-03-29T08:24:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce2e00640 1 -- 172.21.15.46:0/3087195123 shutdown_connections 2024-03-29T08:24:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce2e00640 1 --2- 172.21.15.46:0/3087195123 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6cb8066690 0x7f6cb8068b50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce2e00640 1 --2- 172.21.15.46:0/3087195123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cdc0af530 0x7f6cdc14dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce2e00640 1 --2- 172.21.15.46:0/3087195123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6cdc158300 0x7f6cdc1553e0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce2e00640 1 --2- 172.21.15.46:0/3087195123 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6cdc0702a0 0x7f6cdc156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce2e00640 1 -- 172.21.15.46:0/3087195123 >> 172.21.15.46:0/3087195123 conn(0x7f6cdc066080 msgr2=0x7f6cdc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:49.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce2e00640 1 -- 172.21.15.46:0/3087195123 shutdown_connections 2024-03-29T08:24:49.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.393+0000 7f6ce2e00640 1 -- 172.21.15.46:0/3087195123 wait complete. 2024-03-29T08:24:49.412 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:24:49.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.553+0000 7f6540c00640 1 Processor -- start 2024-03-29T08:24:49.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.553+0000 7f6540c00640 1 -- start start 2024-03-29T08:24:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.553+0000 7f6540c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f653c1593d0 0x7f653c06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.553+0000 7f6540c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f653c15d230 0x7f653c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.553+0000 7f6540c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f653c15dbd0 0x7f653c0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.553+0000 7f6540c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653c06d130 con 0x7f653c15dbd0 2024-03-29T08:24:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.553+0000 7f6540c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f653c15b8f0 con 0x7f653c15d230 2024-03-29T08:24:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.553+0000 7f6540c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653c15ad50 con 0x7f653c1593d0 2024-03-29T08:24:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.553+0000 7f653a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f653c15dbd0 0x7f653c0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.553+0000 7f653a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f653c15dbd0 0x7f653c0701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34786/0 (socket says 172.21.15.46:34786) 2024-03-29T08:24:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.553+0000 7f653a800640 1 -- 172.21.15.46:0/2937333022 learned_addr learned my addr 172.21.15.46:0/2937333022 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.553+0000 7f6531400640 1 --2- 172.21.15.46:0/2937333022 >> v2:172.21.15.73:3300/0 conn(0x7f653c15d230 0x7f653c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.553+0000 7f6539e00640 1 --2- 172.21.15.46:0/2937333022 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f653c1593d0 0x7f653c06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.553+0000 7f653a800640 1 -- 172.21.15.46:0/2937333022 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f653c1593d0 msgr2=0x7f653c06cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:49.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.557+0000 7f653a800640 1 --2- 172.21.15.46:0/2937333022 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f653c1593d0 0x7f653c06cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.557+0000 7f653a800640 1 -- 172.21.15.46:0/2937333022 >> v2:172.21.15.73:3300/0 conn(0x7f653c15d230 msgr2=0x7f653c15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:49.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.557+0000 7f653a800640 1 --2- 172.21.15.46:0/2937333022 >> v2:172.21.15.73:3300/0 conn(0x7f653c15d230 0x7f653c15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.557+0000 7f653a800640 1 -- 172.21.15.46:0/2937333022 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c15e060 con 0x7f653c15dbd0 2024-03-29T08:24:49.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.557+0000 7f653a800640 1 --2- 172.21.15.46:0/2937333022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653c15dbd0 0x7f653c0701c0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f653c071520 tx=0x7f653401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9203114ac6bec1ff server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:49.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.557+0000 7f6539400640 1 -- 172.21.15.46:0/2937333022 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65340ab5f0 con 0x7f653c15dbd0 2024-03-29T08:24:49.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.557+0000 7f6539400640 1 -- 172.21.15.46:0/2937333022 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65340ab750 con 0x7f653c15dbd0 2024-03-29T08:24:49.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.557+0000 7f6539400640 1 -- 172.21.15.46:0/2937333022 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65340abae0 con 0x7f653c15dbd0 2024-03-29T08:24:49.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.557+0000 7f6540c00640 1 -- 172.21.15.46:0/2937333022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653c15dbd0 msgr2=0x7f653c0701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:49.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.557+0000 7f6540c00640 1 --2- 172.21.15.46:0/2937333022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653c15dbd0 0x7f653c0701c0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f653c071520 tx=0x7f653401c520 comp rx=0 tx=0).stop 2024-03-29T08:24:49.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.557+0000 7f6540c00640 1 -- 172.21.15.46:0/2937333022 shutdown_connections 2024-03-29T08:24:49.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.557+0000 7f6540c00640 1 --2- 172.21.15.46:0/2937333022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653c15dbd0 0x7f653c0701c0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.557+0000 7f6540c00640 1 --2- 172.21.15.46:0/2937333022 >> v2:172.21.15.73:3300/0 conn(0x7f653c15d230 0x7f653c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.557+0000 7f6540c00640 1 --2- 172.21.15.46:0/2937333022 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f653c1593d0 0x7f653c06cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.557+0000 7f6540c00640 1 -- 172.21.15.46:0/2937333022 >> 172.21.15.46:0/2937333022 conn(0x7f653c066110 msgr2=0x7f653c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:49.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.557+0000 7f6540c00640 1 -- 172.21.15.46:0/2937333022 shutdown_connections 2024-03-29T08:24:49.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.557+0000 7f6540c00640 1 -- 172.21.15.46:0/2937333022 wait complete. 2024-03-29T08:24:49.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f6540c00640 1 Processor -- start 2024-03-29T08:24:49.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f6540c00640 1 -- start start 2024-03-29T08:24:49.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f6540c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f653c1593d0 0x7f653c0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f6540c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f653c15d230 0x7f653c14c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f6540c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653c15dbd0 0x7f653c0aa300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f6540c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c14cd10 con 0x7f653c15dbd0 2024-03-29T08:24:49.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f6540c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653c0cd2d0 con 0x7f653c15d230 2024-03-29T08:24:49.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f6540c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f653c0cd5b0 con 0x7f653c1593d0 2024-03-29T08:24:49.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f653a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653c15dbd0 0x7f653c0aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f653a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653c15dbd0 0x7f653c0aa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34800/0 (socket says 172.21.15.46:34800) 2024-03-29T08:24:49.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f653a800640 1 -- 172.21.15.46:0/2466043069 learned_addr learned my addr 172.21.15.46:0/2466043069 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:49.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f6531400640 1 --2- 172.21.15.46:0/2466043069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f653c15d230 0x7f653c14c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f6539e00640 1 --2- 172.21.15.46:0/2466043069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f653c1593d0 0x7f653c0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f653a800640 1 -- 172.21.15.46:0/2466043069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f653c1593d0 msgr2=0x7f653c0cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:49.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f653a800640 1 --2- 172.21.15.46:0/2466043069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f653c1593d0 0x7f653c0cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f653a800640 1 -- 172.21.15.46:0/2466043069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f653c15d230 msgr2=0x7f653c14c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:49.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f653a800640 1 --2- 172.21.15.46:0/2466043069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f653c15d230 0x7f653c14c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f653a800640 1 -- 172.21.15.46:0/2466043069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65340094d0 con 0x7f653c15dbd0 2024-03-29T08:24:49.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f6531400640 1 --2- 172.21.15.46:0/2466043069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f653c15d230 0x7f653c14c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:24:49.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f6539e00640 1 --2- 172.21.15.46:0/2466043069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f653c1593d0 0x7f653c0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:24:49.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f653a800640 1 --2- 172.21.15.46:0/2466043069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653c15dbd0 0x7f653c0aa300 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f65340ad040 tx=0x7f65340192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:49.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f6532a00640 1 -- 172.21.15.46:0/2466043069 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65340bf610 con 0x7f653c15dbd0 2024-03-29T08:24:49.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f6532a00640 1 -- 172.21.15.46:0/2466043069 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65340bf790 con 0x7f653c15dbd0 2024-03-29T08:24:49.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f6532a00640 1 -- 172.21.15.46:0/2466043069 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65340bfb40 con 0x7f653c15dbd0 2024-03-29T08:24:49.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f6540c00640 1 -- 172.21.15.46:0/2466043069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653c0cd890 con 0x7f653c15dbd0 2024-03-29T08:24:49.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.561+0000 7f6540c00640 1 -- 172.21.15.46:0/2466043069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653c1509e0 con 0x7f653c15dbd0 2024-03-29T08:24:49.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.565+0000 7f6540c00640 1 -- 172.21.15.46:0/2466043069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f653c15ab80 con 0x7f653c15dbd0 2024-03-29T08:24:49.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.565+0000 7f6532a00640 1 -- 172.21.15.46:0/2466043069 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f65340bfca0 con 0x7f653c15dbd0 2024-03-29T08:24:49.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.565+0000 7f6532a00640 1 --2- 172.21.15.46:0/2466043069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6518066690 0x7f6518068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:49.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.565+0000 7f6532a00640 1 -- 172.21.15.46:0/2466043069 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65340a8050 con 0x7f653c15dbd0 2024-03-29T08:24:49.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.565+0000 7f6539e00640 1 --2- 172.21.15.46:0/2466043069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6518066690 0x7f6518068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:49.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.565+0000 7f6539e00640 1 --2- 172.21.15.46:0/2466043069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6518066690 0x7f6518068b50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f653c135730 tx=0x7f652c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:49.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.569+0000 7f6532a00640 1 -- 172.21.15.46:0/2466043069 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f65340ebae0 con 0x7f653c15dbd0 2024-03-29T08:24:49.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.769+0000 7f6540c00640 1 -- 172.21.15.46:0/2466043069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f653c066c90 con 0x7f653c15dbd0 2024-03-29T08:24:49.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.769+0000 7f6532a00640 1 -- 172.21.15.46:0/2466043069 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v57) v1 ==== 53+0+73 (secure 0 0 0) 0x7f65340ef100 con 0x7f653c15dbd0 2024-03-29T08:24:49.771 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:24:49.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.769+0000 7f6540c00640 1 -- 172.21.15.46:0/2466043069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6518066690 msgr2=0x7f6518068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:49.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.769+0000 7f6540c00640 1 --2- 172.21.15.46:0/2466043069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6518066690 0x7f6518068b50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f653c135730 tx=0x7f652c008040 comp rx=0 tx=0).stop 2024-03-29T08:24:49.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.769+0000 7f6540c00640 1 -- 172.21.15.46:0/2466043069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653c15dbd0 msgr2=0x7f653c0aa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:49.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.769+0000 7f6540c00640 1 --2- 172.21.15.46:0/2466043069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653c15dbd0 0x7f653c0aa300 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f65340ad040 tx=0x7f65340192c0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.769+0000 7f6540c00640 1 -- 172.21.15.46:0/2466043069 shutdown_connections 2024-03-29T08:24:49.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.769+0000 7f6540c00640 1 --2- 172.21.15.46:0/2466043069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6518066690 0x7f6518068b50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.769+0000 7f6540c00640 1 --2- 172.21.15.46:0/2466043069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f653c15d230 0x7f653c14c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.769+0000 7f6540c00640 1 --2- 172.21.15.46:0/2466043069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f653c15dbd0 0x7f653c0aa300 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.769+0000 7f6540c00640 1 --2- 172.21.15.46:0/2466043069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f653c1593d0 0x7f653c0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:49.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.769+0000 7f6540c00640 1 -- 172.21.15.46:0/2466043069 >> 172.21.15.46:0/2466043069 conn(0x7f653c066110 msgr2=0x7f653c0adc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:49.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.769+0000 7f6540c00640 1 -- 172.21.15.46:0/2466043069 shutdown_connections 2024-03-29T08:24:49.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:49.769+0000 7f6540c00640 1 -- 172.21.15.46:0/2466043069 wait complete. 2024-03-29T08:24:49.788 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:24:49.789 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:24:49.789 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-29T08:24:49.789 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T08:24:49.789 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:24:49.789 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:24:49.789 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:24:49.789 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:24:49.789 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:24:49.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:49.789 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-29T08:24:49.798 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:24:49.798 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:24:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:24:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:24:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:24:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:24:49.799 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:24:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:24:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-29T08:24:49.799 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:24:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:24:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-29T08:24:49.799 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:24:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:24:49.799 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-29T08:24:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T08:24:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42047sec preferred_lft 42047sec 2024-03-29T08:24:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-29T08:24:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:24:49.800 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:24:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:24:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-29T08:24:49.800 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:24:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:d9:58:09:4c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:24:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:24:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:24:49.800 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:24:49.800 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T08:24:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T08:24:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:24:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T08:24:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:24:49.801 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:24:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:24:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T08:24:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:24:49.801 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:24:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T08:24:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-29T08:24:49.801 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:24:49.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:49.802 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:24:49.860 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:24:49.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:49.863 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-29T08:24:49.920 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-29T08:24:49.920 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:24:49.920 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T08:24:49.920 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:24:49.920 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T08:24:49.920 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:24:49.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:49.921 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:24:49.921 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:24:49.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:24:49.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:24:49.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:24:49.921 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:24:50.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:50.014 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:24:50.022 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T08:24:50.022 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T08:24:50.023 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:24:50.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:50.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-03-29T08:24:50.092 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T08:24:50.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:24:50.093 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T08:24:50.147 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:24:50.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:24:50.147 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:24:50.198 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:24:50.260 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:24:50.260 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:24:50.325 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:24:50.326 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:24:50.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:50.327 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:24:50.382 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:24:50.383 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T08:24:50.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:24:50.447 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:24:50.447 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:24:50.510 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:24:50.512 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:24:50.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:50.513 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:24:50.523 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-03-29T08:24:50.521+0000 7fb80f24a580 -1 init, newargv = 0x5654f8b814e0 newargc=15 2024-03-29T08:24:50.523 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[44662]: starting ceph client 2024-03-29T08:24:50.530 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[44662]: starting fuse 2024-03-29T08:24:50.533 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:24:50.533 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-29T08:24:50.533 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:24:50.533 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-03-29T08:24:50.533 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-03-29T08:24:50.533 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:24:50.534 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-03-29T08:24:50.534 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:24:50.534 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-03-29T08:24:50.534 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T08:24:50.534 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-03-29T08:24:50.534 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:24:50.534 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:24:50.534 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-03-29T08:24:50.534 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-03-29T08:24:50.534 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-03-29T08:24:50.534 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-03-29T08:24:50.534 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:24:50.534 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-03-29T08:24:50.534 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:24:50.535 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:24:50.535 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:24:50.535 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-03-29T08:24:50.535 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-03-29T08:24:50.535 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-03-29T08:24:50.535 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:24:50.535 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:24:50.535 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-03-29T08:24:50.535 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:24:50.656 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.44662.asok 2024-03-29T08:24:50.840 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44662.asok 2024-03-29T08:24:50.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:50.840 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44662.asok status 2024-03-29T08:24:51.021 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:24:51.021 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-03-29T08:24:51.021 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:24:51.021 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:24:51.021 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-03-29T08:24:51.021 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-03-29T08:24:51.021 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T08:24:51.021 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "44662", 2024-03-29T08:24:51.021 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-03-29T08:24:51.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:51.022 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-03-29T08:24:51.022 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-03-29T08:24:51.022 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 7309, 2024-03-29T08:24:51.022 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-03-29T08:24:51.022 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-03-29T08:24:51.022 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-03-29T08:24:51.022 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 7309 2024-03-29T08:24:51.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:51.022 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:24:51.022 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:24:51.022 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-03-29T08:24:51.022 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3534327319 2024-03-29T08:24:51.022 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:24:51.022 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:51.023 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:24:51.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:24:51.023 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-03-29T08:24:51.023 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3534327319 2024-03-29T08:24:51.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:24:51.023 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.7309 172.21.15.46:0/3534327319", 2024-03-29T08:24:51.023 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "172.21.15.46:0/3534327319", 2024-03-29T08:24:51.023 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-03-29T08:24:51.023 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 57, 2024-03-29T08:24:51.023 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 92, 2024-03-29T08:24:51.023 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:24:51.023 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-03-29T08:24:51.023 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-03-29T08:24:51.023 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:24:51.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:51.036 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-03-29T08:24:51.036 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:24:51.046 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-03-29T08:24:51.046 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T08:24:51.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:51.046 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:24:51.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:51.109 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-03-29T08:24:51.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:51.172 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-03-29T08:24:51.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:51.228 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-03-29T08:24:51.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:51.282 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-03-29T08:24:51.339 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:24:51.339 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:24:51.339 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:24:51.339 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:24:51.339 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-03-29T08:24:51.339 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:24:51.446 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627776 2024-03-29T08:24:51.636 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:24:51.636 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:24:51.688 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:24:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.873+0000 7f5d9f200640 1 Processor -- start 2024-03-29T08:24:51.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.873+0000 7f5d9f200640 1 -- start start 2024-03-29T08:24:51.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.877+0000 7f5d9f200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5da015cb00 0x7f5da0098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:51.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.877+0000 7f5d9f200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5da0099810 0x7f5da0098f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:51.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.877+0000 7f5d9f200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da00cb5d0 con 0x7f5da00b2b30 2024-03-29T08:24:51.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.877+0000 7f5d9f200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da0099d10 con 0x7f5da0099810 2024-03-29T08:24:51.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.877+0000 7f5d9f200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00caca0 con 0x7f5da015cb00 2024-03-29T08:24:51.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.877+0000 7f5d95400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5da0099810 0x7f5da0098f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:51.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.877+0000 7f5d95400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5da0099810 0x7f5da0098f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34816/0 (socket says 172.21.15.46:34816) 2024-03-29T08:24:51.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.877+0000 7f5d95400640 1 -- 172.21.15.46:0/2100865681 learned_addr learned my addr 172.21.15.46:0/2100865681 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:51.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.877+0000 7f5d95400640 1 -- 172.21.15.46:0/2100865681 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5da015cb00 msgr2=0x7f5da0098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:51.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.877+0000 7f5d95400640 1 --2- 172.21.15.46:0/2100865681 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5da015cb00 0x7f5da0098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:51.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.877+0000 7f5d95400640 1 -- 172.21.15.46:0/2100865681 >> v1:172.21.15.73:6789/0 conn(0x7f5da00b2b30 legacy=0x7f5da006b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:51.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.877+0000 7f5d95400640 1 -- 172.21.15.46:0/2100865681 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da00caf80 con 0x7f5da0099810 2024-03-29T08:24:51.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.877+0000 7f5d95400640 1 --2- 172.21.15.46:0/2100865681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da0099810 0x7f5da0098f80 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009e50 tx=0x7f5d8801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc368678ee962bf0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:51.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9d400640 1 -- 172.21.15.46:0/2100865681 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d880a5780 con 0x7f5da0099810 2024-03-29T08:24:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9d400640 1 -- 172.21.15.46:0/2100865681 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d880a58e0 con 0x7f5da0099810 2024-03-29T08:24:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9d400640 1 -- 172.21.15.46:0/2100865681 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d880a5c70 con 0x7f5da0099810 2024-03-29T08:24:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9f200640 1 -- 172.21.15.46:0/2100865681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da0099810 msgr2=0x7f5da0098f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9f200640 1 --2- 172.21.15.46:0/2100865681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da0099810 0x7f5da0098f80 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009e50 tx=0x7f5d8801c960 comp rx=0 tx=0).stop 2024-03-29T08:24:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9f200640 1 -- 172.21.15.46:0/2100865681 shutdown_connections 2024-03-29T08:24:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9f200640 1 --2- 172.21.15.46:0/2100865681 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da0099810 0x7f5da0098f80 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9f200640 1 --2- 172.21.15.46:0/2100865681 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5da015cb00 0x7f5da0098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9f200640 1 -- 172.21.15.46:0/2100865681 >> 172.21.15.46:0/2100865681 conn(0x7f5da00b2010 msgr2=0x7f5da00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9f200640 1 -- 172.21.15.46:0/2100865681 shutdown_connections 2024-03-29T08:24:51.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9f200640 1 -- 172.21.15.46:0/2100865681 wait complete. 2024-03-29T08:24:51.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9f200640 1 Processor -- start 2024-03-29T08:24:51.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9f200640 1 -- start start 2024-03-29T08:24:51.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5da0099810 0x7f5da014f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:51.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9f200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5da00b2b30 0x7f5da00c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:51.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da015cb00 0x7f5da00c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:51.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da00c9230 con 0x7f5da015cb00 2024-03-29T08:24:51.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da015a2f0 con 0x7f5da0099810 2024-03-29T08:24:51.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9f200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da0150d70 con 0x7f5da00b2b30 2024-03-29T08:24:51.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da015cb00 0x7f5da00c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:51.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da015cb00 0x7f5da00c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34822/0 (socket says 172.21.15.46:34822) 2024-03-29T08:24:51.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9e800640 1 -- 172.21.15.46:0/2725762881 learned_addr learned my addr 172.21.15.46:0/2725762881 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:51.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d95400640 1 --2- 172.21.15.46:0/2725762881 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5da00b2b30 0x7f5da00c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:51.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.881+0000 7f5d9de00640 1 --2- 172.21.15.46:0/2725762881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5da0099810 0x7f5da014f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:51.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.885+0000 7f5d9e800640 1 -- 172.21.15.46:0/2725762881 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5da00b2b30 msgr2=0x7f5da00c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.885+0000 7f5d9e800640 1 --2- 172.21.15.46:0/2725762881 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5da00b2b30 0x7f5da00c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.885+0000 7f5d9e800640 1 -- 172.21.15.46:0/2725762881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5da0099810 msgr2=0x7f5da014f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.885+0000 7f5d9e800640 1 --2- 172.21.15.46:0/2725762881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5da0099810 0x7f5da014f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.885+0000 7f5d9e800640 1 -- 172.21.15.46:0/2725762881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d88009b30 con 0x7f5da015cb00 2024-03-29T08:24:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.885+0000 7f5d9e800640 1 --2- 172.21.15.46:0/2725762881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da015cb00 0x7f5da00c7b30 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800b410 tx=0x7f5d9800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.885+0000 7f5d96a00640 1 -- 172.21.15.46:0/2725762881 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98011830 con 0x7f5da015cb00 2024-03-29T08:24:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.885+0000 7f5d96a00640 1 -- 172.21.15.46:0/2725762881 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d98011990 con 0x7f5da015cb00 2024-03-29T08:24:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.885+0000 7f5d96a00640 1 -- 172.21.15.46:0/2725762881 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d98011d40 con 0x7f5da015cb00 2024-03-29T08:24:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.885+0000 7f5d9f200640 1 -- 172.21.15.46:0/2725762881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da014f600 con 0x7f5da015cb00 2024-03-29T08:24:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.885+0000 7f5d9f200640 1 -- 172.21.15.46:0/2725762881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da0150fe0 con 0x7f5da015cb00 2024-03-29T08:24:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.885+0000 7f5d9f200640 1 -- 172.21.15.46:0/2725762881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da004e040 con 0x7f5da015cb00 2024-03-29T08:24:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.885+0000 7f5d96a00640 1 -- 172.21.15.46:0/2725762881 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5d9801a400 con 0x7f5da015cb00 2024-03-29T08:24:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.885+0000 7f5d96a00640 1 --2- 172.21.15.46:0/2725762881 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d7c066690 0x7f5d7c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.885+0000 7f5d96a00640 1 -- 172.21.15.46:0/2725762881 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d98084130 con 0x7f5da015cb00 2024-03-29T08:24:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.889+0000 7f5d9de00640 1 --2- 172.21.15.46:0/2725762881 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d7c066690 0x7f5d7c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.889+0000 7f5d9de00640 1 --2- 172.21.15.46:0/2725762881 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d7c066690 0x7f5d7c068b50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5da0153250 tx=0x7f5d90006f90 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:51.889+0000 7f5d96a00640 1 -- 172.21.15.46:0/2725762881 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5d98051550 con 0x7f5da015cb00 2024-03-29T08:24:52.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.053+0000 7f5d9f200640 1 -- 172.21.15.46:0/2725762881 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5da006df40 con 0x7f5da015cb00 2024-03-29T08:24:52.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.057+0000 7f5d96a00640 1 -- 172.21.15.46:0/2725762881 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16056 (secure 0 0 0) 0x7f5d98054b70 con 0x7f5da015cb00 2024-03-29T08:24:52.055 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:52.055 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":92,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:24:44.478513+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:24:41.113618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-29T08:24:42.487380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:24:52.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.057+0000 7f5d9f200640 1 -- 172.21.15.46:0/2725762881 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d7c066690 msgr2=0x7f5d7c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.057+0000 7f5d9f200640 1 --2- 172.21.15.46:0/2725762881 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d7c066690 0x7f5d7c068b50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5da0153250 tx=0x7f5d90006f90 comp rx=0 tx=0).stop 2024-03-29T08:24:52.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.057+0000 7f5d9f200640 1 -- 172.21.15.46:0/2725762881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da015cb00 msgr2=0x7f5da00c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.057+0000 7f5d9f200640 1 --2- 172.21.15.46:0/2725762881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da015cb00 0x7f5da00c7b30 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800b410 tx=0x7f5d9800b8d0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.057+0000 7f5d9f200640 1 -- 172.21.15.46:0/2725762881 shutdown_connections 2024-03-29T08:24:52.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.057+0000 7f5d9f200640 1 --2- 172.21.15.46:0/2725762881 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d7c066690 0x7f5d7c068b50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.057+0000 7f5d9f200640 1 --2- 172.21.15.46:0/2725762881 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da015cb00 0x7f5da00c7b30 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.057+0000 7f5d9f200640 1 --2- 172.21.15.46:0/2725762881 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5da00b2b30 0x7f5da00c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.057+0000 7f5d9f200640 1 --2- 172.21.15.46:0/2725762881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5da0099810 0x7f5da014f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.057+0000 7f5d9f200640 1 -- 172.21.15.46:0/2725762881 >> 172.21.15.46:0/2725762881 conn(0x7f5da00b2010 msgr2=0x7f5da0156060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:52.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.057+0000 7f5d9f200640 1 -- 172.21.15.46:0/2725762881 shutdown_connections 2024-03-29T08:24:52.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.057+0000 7f5d9f200640 1 -- 172.21.15.46:0/2725762881 wait complete. 2024-03-29T08:24:52.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:24:41.113618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:24:42.487380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:24:52.078 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:24:52.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.221+0000 7fe419e00640 1 Processor -- start 2024-03-29T08:24:52.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.221+0000 7fe419e00640 1 -- start start 2024-03-29T08:24:52.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.225+0000 7fe419e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe4140cc000 0x7fe41406b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.225+0000 7fe419e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe41415d040 con 0x7fe4140af710 2024-03-29T08:24:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.225+0000 7fe419e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4140cf800 con 0x7fe4140cd190 2024-03-29T08:24:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.225+0000 7fe419e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41406c550 con 0x7fe4140cc000 2024-03-29T08:24:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.225+0000 7fe419400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe4140af710 0x7fe414167280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42100/0 (socket says 172.21.15.46:42100) 2024-03-29T08:24:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.225+0000 7fe419400640 1 -- 172.21.15.46:0/3242101994 learned_addr learned my addr 172.21.15.46:0/3242101994 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.225+0000 7fe40be00640 1 --2- 172.21.15.46:0/3242101994 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe4140cc000 0x7fe41406b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.225+0000 7fe413e00640 1 -- 172.21.15.46:0/3242101994 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3255935789 0 0) 0x7fe41415d040 con 0x7fe4140af710 2024-03-29T08:24:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.225+0000 7fe413e00640 1 -- 172.21.15.46:0/3242101994 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3f8003530 con 0x7fe4140af710 2024-03-29T08:24:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.225+0000 7fe413e00640 1 -- 172.21.15.46:0/3242101994 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545186157 0 0) 0x7fe4140cf800 con 0x7fe4140cd190 2024-03-29T08:24:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.225+0000 7fe413e00640 1 -- 172.21.15.46:0/3242101994 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe41415d040 con 0x7fe4140cd190 2024-03-29T08:24:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.225+0000 7fe40be00640 1 -- 172.21.15.46:0/3242101994 >> v1:172.21.15.73:6789/0 conn(0x7fe4140cd190 legacy=0x7fe41406b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.225+0000 7fe40be00640 1 -- 172.21.15.46:0/3242101994 >> v1:172.21.15.46:6789/0 conn(0x7fe4140af710 legacy=0x7fe414167280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.225+0000 7fe40be00640 1 -- 172.21.15.46:0/3242101994 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4140cfae0 con 0x7fe4140cc000 2024-03-29T08:24:52.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.229+0000 7fe40be00640 1 --2- 172.21.15.46:0/3242101994 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe4140cc000 0x7fe41406b950 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0097f0 tx=0x7fe3fc01ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75bfb89624e1de62 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:52.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.229+0000 7fe413e00640 1 -- 172.21.15.46:0/3242101994 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0a89d0 con 0x7fe4140cc000 2024-03-29T08:24:52.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.229+0000 7fe413e00640 1 -- 172.21.15.46:0/3242101994 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3fc0a8b30 con 0x7fe4140cc000 2024-03-29T08:24:52.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.229+0000 7fe413e00640 1 -- 172.21.15.46:0/3242101994 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0b0440 con 0x7fe4140cc000 2024-03-29T08:24:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.229+0000 7fe419e00640 1 -- 172.21.15.46:0/3242101994 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe4140cc000 msgr2=0x7fe41406b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.229+0000 7fe419e00640 1 --2- 172.21.15.46:0/3242101994 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe4140cc000 0x7fe41406b950 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0097f0 tx=0x7fe3fc01ecd0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.229+0000 7fe419e00640 1 -- 172.21.15.46:0/3242101994 shutdown_connections 2024-03-29T08:24:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.229+0000 7fe419e00640 1 --2- 172.21.15.46:0/3242101994 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe4140cc000 0x7fe41406b950 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.229+0000 7fe419e00640 1 -- 172.21.15.46:0/3242101994 >> 172.21.15.46:0/3242101994 conn(0x7fe4140d0550 msgr2=0x7fe4140d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.229+0000 7fe419e00640 1 -- 172.21.15.46:0/3242101994 shutdown_connections 2024-03-29T08:24:52.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.229+0000 7fe419e00640 1 -- 172.21.15.46:0/3242101994 wait complete. 2024-03-29T08:24:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.229+0000 7fe419e00640 1 Processor -- start 2024-03-29T08:24:52.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.229+0000 7fe419e00640 1 -- start start 2024-03-29T08:24:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe419e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4140af710 0x7fe4140c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe419e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4140cc000 0x7fe4141f1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe419e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe4140cd190 0x7fe4141f1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe419e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4141f1ca0 con 0x7fe4140cc000 2024-03-29T08:24:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe419e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41409e120 con 0x7fe4140af710 2024-03-29T08:24:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe419e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41409d0c0 con 0x7fe4140cd190 2024-03-29T08:24:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe40be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4140cc000 0x7fe4141f1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe40be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4140cc000 0x7fe4141f1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34828/0 (socket says 172.21.15.46:34828) 2024-03-29T08:24:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe40be00640 1 -- 172.21.15.46:0/3743296778 learned_addr learned my addr 172.21.15.46:0/3743296778 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:52.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe418a00640 1 --2- 172.21.15.46:0/3743296778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4140af710 0x7fe4140c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe419400640 1 --2- 172.21.15.46:0/3743296778 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe4140cd190 0x7fe4141f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe40be00640 1 -- 172.21.15.46:0/3743296778 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe4140cd190 msgr2=0x7fe4141f1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe40be00640 1 --2- 172.21.15.46:0/3743296778 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe4140cd190 0x7fe4141f1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe40be00640 1 -- 172.21.15.46:0/3743296778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4140af710 msgr2=0x7fe4140c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe40be00640 1 --2- 172.21.15.46:0/3743296778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4140af710 0x7fe4140c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe40be00640 1 -- 172.21.15.46:0/3743296778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc009420 con 0x7fe4140cc000 2024-03-29T08:24:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe40be00640 1 --2- 172.21.15.46:0/3743296778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4140cc000 0x7fe4141f1220 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc01c900 tx=0x7fe3fc01ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe411600640 1 -- 172.21.15.46:0/3743296778 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0b1cd0 con 0x7fe4140cc000 2024-03-29T08:24:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe411600640 1 -- 172.21.15.46:0/3743296778 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3fc0b1e50 con 0x7fe4140cc000 2024-03-29T08:24:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe411600640 1 -- 172.21.15.46:0/3743296778 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0b9650 con 0x7fe4140cc000 2024-03-29T08:24:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe419e00640 1 -- 172.21.15.46:0/3743296778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe41409bc40 con 0x7fe4140cc000 2024-03-29T08:24:52.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe419e00640 1 -- 172.21.15.46:0/3743296778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe41409d640 con 0x7fe4140cc000 2024-03-29T08:24:52.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe419e00640 1 -- 172.21.15.46:0/3743296778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe41404e040 con 0x7fe4140cc000 2024-03-29T08:24:52.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.233+0000 7fe411600640 1 -- 172.21.15.46:0/3743296778 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe3fc0b7040 con 0x7fe4140cc000 2024-03-29T08:24:52.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.237+0000 7fe411600640 1 --2- 172.21.15.46:0/3743296778 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe3f0066690 0x7fe3f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:52.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.237+0000 7fe411600640 1 -- 172.21.15.46:0/3743296778 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3fc121970 con 0x7fe4140cc000 2024-03-29T08:24:52.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.237+0000 7fe418a00640 1 --2- 172.21.15.46:0/3743296778 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe3f0066690 0x7fe3f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:52.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.237+0000 7fe418a00640 1 --2- 172.21.15.46:0/3743296778 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe3f0066690 0x7fe3f0068b50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe4040011f0 tx=0x7fe404009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:52.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.241+0000 7fe411600640 1 -- 172.21.15.46:0/3743296778 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe3fc0eecf0 con 0x7fe4140cc000 2024-03-29T08:24:52.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.397+0000 7fe419e00640 1 -- 172.21.15.46:0/3743296778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4140d13a0 con 0x7fe4140cc000 2024-03-29T08:24:52.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.401+0000 7fe411600640 1 -- 172.21.15.46:0/3743296778 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe3fc0b5020 con 0x7fe4140cc000 2024-03-29T08:24:52.399 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:24:52.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.401+0000 7fe419e00640 1 -- 172.21.15.46:0/3743296778 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe3f0066690 msgr2=0x7fe3f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.401+0000 7fe419e00640 1 --2- 172.21.15.46:0/3743296778 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe3f0066690 0x7fe3f0068b50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe4040011f0 tx=0x7fe404009040 comp rx=0 tx=0).stop 2024-03-29T08:24:52.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.401+0000 7fe419e00640 1 -- 172.21.15.46:0/3743296778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4140cc000 msgr2=0x7fe4141f1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.401+0000 7fe419e00640 1 --2- 172.21.15.46:0/3743296778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4140cc000 0x7fe4141f1220 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc01c900 tx=0x7fe3fc01ba70 comp rx=0 tx=0).stop 2024-03-29T08:24:52.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.401+0000 7fe419e00640 1 -- 172.21.15.46:0/3743296778 shutdown_connections 2024-03-29T08:24:52.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.401+0000 7fe419e00640 1 --2- 172.21.15.46:0/3743296778 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe3f0066690 0x7fe3f0068b50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.401+0000 7fe419e00640 1 --2- 172.21.15.46:0/3743296778 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe4140cd190 0x7fe4141f1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.401+0000 7fe419e00640 1 --2- 172.21.15.46:0/3743296778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4140cc000 0x7fe4141f1220 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.401+0000 7fe419e00640 1 --2- 172.21.15.46:0/3743296778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4140af710 0x7fe4140c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.401+0000 7fe419e00640 1 -- 172.21.15.46:0/3743296778 >> 172.21.15.46:0/3743296778 conn(0x7fe4140d0550 msgr2=0x7fe4140cb780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:52.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.401+0000 7fe419e00640 1 -- 172.21.15.46:0/3743296778 shutdown_connections 2024-03-29T08:24:52.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.401+0000 7fe419e00640 1 -- 172.21.15.46:0/3743296778 wait complete. 2024-03-29T08:24:52.422 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:24:52.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.565+0000 7f3b61a00640 1 Processor -- start 2024-03-29T08:24:52.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.565+0000 7f3b61a00640 1 -- start start 2024-03-29T08:24:52.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.565+0000 7f3b61a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3b5c0d0dc0 0x7f3b5c0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:52.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.565+0000 7f3b61a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3b5c0cf820 0x7f3b5c0b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:52.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.565+0000 7f3b61a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b5c15cf40 con 0x7f3b5c15cb60 2024-03-29T08:24:52.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.565+0000 7f3b61a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b5c0b0250 con 0x7f3b5c0cf820 2024-03-29T08:24:52.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.565+0000 7f3b61a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c0b0810 con 0x7f3b5c0d0dc0 2024-03-29T08:24:52.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.565+0000 7f3b5b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3b5c0cf820 0x7f3b5c0b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:52.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.565+0000 7f3b5b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3b5c0cf820 0x7f3b5c0b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34844/0 (socket says 172.21.15.46:34844) 2024-03-29T08:24:52.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.565+0000 7f3b5b400640 1 -- 172.21.15.46:0/946102052 learned_addr learned my addr 172.21.15.46:0/946102052 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:52.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.565+0000 7f3b52000640 1 --2- 172.21.15.46:0/946102052 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3b5c0d0dc0 0x7f3b5c0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:52.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.565+0000 7f3b5b400640 1 -- 172.21.15.46:0/946102052 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3b5c0d0dc0 msgr2=0x7f3b5c0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:52.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.565+0000 7f3b5b400640 1 --2- 172.21.15.46:0/946102052 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3b5c0d0dc0 0x7f3b5c0cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.565+0000 7f3b5b400640 1 -- 172.21.15.46:0/946102052 >> v1:172.21.15.73:6789/0 conn(0x7f3b5c15cb60 legacy=0x7f3b5c0d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.565+0000 7f3b5b400640 1 -- 172.21.15.46:0/946102052 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c0d0010 con 0x7f3b5c0cf820 2024-03-29T08:24:52.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.569+0000 7f3b5b400640 1 --2- 172.21.15.46:0/946102052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c0cf820 0x7f3b5c0b0ca0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f3b54009870 tx=0x7f3b5401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b4980ffe07edaa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:52.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.569+0000 7f3b5a000640 1 -- 172.21.15.46:0/946102052 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b540a95f0 con 0x7f3b5c0cf820 2024-03-29T08:24:52.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.569+0000 7f3b5a000640 1 -- 172.21.15.46:0/946102052 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b540a9750 con 0x7f3b5c0cf820 2024-03-29T08:24:52.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.569+0000 7f3b5a000640 1 -- 172.21.15.46:0/946102052 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b540a9ae0 con 0x7f3b5c0cf820 2024-03-29T08:24:52.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.569+0000 7f3b61a00640 1 -- 172.21.15.46:0/946102052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c0cf820 msgr2=0x7f3b5c0b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.569+0000 7f3b61a00640 1 --2- 172.21.15.46:0/946102052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c0cf820 0x7f3b5c0b0ca0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f3b54009870 tx=0x7f3b5401c530 comp rx=0 tx=0).stop 2024-03-29T08:24:52.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.569+0000 7f3b61a00640 1 -- 172.21.15.46:0/946102052 shutdown_connections 2024-03-29T08:24:52.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.569+0000 7f3b61a00640 1 --2- 172.21.15.46:0/946102052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c0cf820 0x7f3b5c0b0ca0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.569+0000 7f3b61a00640 1 --2- 172.21.15.46:0/946102052 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3b5c0d0dc0 0x7f3b5c0cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.569+0000 7f3b61a00640 1 -- 172.21.15.46:0/946102052 >> 172.21.15.46:0/946102052 conn(0x7f3b5c099a10 msgr2=0x7f3b5c089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:52.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.569+0000 7f3b61a00640 1 -- 172.21.15.46:0/946102052 shutdown_connections 2024-03-29T08:24:52.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.569+0000 7f3b61a00640 1 -- 172.21.15.46:0/946102052 wait complete. 2024-03-29T08:24:52.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b61a00640 1 Processor -- start 2024-03-29T08:24:52.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b61a00640 1 -- start start 2024-03-29T08:24:52.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b61a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3b5c0cf820 0x7f3b5c153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:52.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b61a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c0d0dc0 0x7f3b5c14b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:52.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b61a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b5c15cb60 0x7f3b5c14bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:52.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b61a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c153580 con 0x7f3b5c0d0dc0 2024-03-29T08:24:52.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b61a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c1543c0 con 0x7f3b5c15cb60 2024-03-29T08:24:52.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b61a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c1537b0 con 0x7f3b5c0cf820 2024-03-29T08:24:52.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b52000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c0d0dc0 0x7f3b5c14b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:52.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b52000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c0d0dc0 0x7f3b5c14b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34858/0 (socket says 172.21.15.46:34858) 2024-03-29T08:24:52.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b52000640 1 -- 172.21.15.46:0/3214272458 learned_addr learned my addr 172.21.15.46:0/3214272458 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:52.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b5aa00640 1 --2- 172.21.15.46:0/3214272458 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3b5c0cf820 0x7f3b5c153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:52.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b5b400640 1 --2- 172.21.15.46:0/3214272458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b5c15cb60 0x7f3b5c14bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:52.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b52000640 1 -- 172.21.15.46:0/3214272458 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3b5c0cf820 msgr2=0x7f3b5c153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b52000640 1 --2- 172.21.15.46:0/3214272458 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3b5c0cf820 0x7f3b5c153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b52000640 1 -- 172.21.15.46:0/3214272458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b5c15cb60 msgr2=0x7f3b5c14bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b52000640 1 --2- 172.21.15.46:0/3214272458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b5c15cb60 0x7f3b5c14bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b52000640 1 -- 172.21.15.46:0/3214272458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b540094d0 con 0x7f3b5c0d0dc0 2024-03-29T08:24:52.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b52000640 1 --2- 172.21.15.46:0/3214272458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c0d0dc0 0x7f3b5c14b7f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400d5f0 tx=0x7f3b4400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:52.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b53400640 1 -- 172.21.15.46:0/3214272458 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b440125f0 con 0x7f3b5c0d0dc0 2024-03-29T08:24:52.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b53400640 1 -- 172.21.15.46:0/3214272458 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b44012750 con 0x7f3b5c0d0dc0 2024-03-29T08:24:52.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b53400640 1 -- 172.21.15.46:0/3214272458 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b44012b00 con 0x7f3b5c0d0dc0 2024-03-29T08:24:52.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b61a00640 1 -- 172.21.15.46:0/3214272458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5c153a90 con 0x7f3b5c0d0dc0 2024-03-29T08:24:52.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.573+0000 7f3b61a00640 1 -- 172.21.15.46:0/3214272458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5c154050 con 0x7f3b5c0d0dc0 2024-03-29T08:24:52.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.577+0000 7f3b61a00640 1 -- 172.21.15.46:0/3214272458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b5c154530 con 0x7f3b5c0d0dc0 2024-03-29T08:24:52.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.577+0000 7f3b53400640 1 -- 172.21.15.46:0/3214272458 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3b44012c60 con 0x7f3b5c0d0dc0 2024-03-29T08:24:52.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.577+0000 7f3b53400640 1 --2- 172.21.15.46:0/3214272458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3b380666c0 0x7f3b38068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:52.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.577+0000 7f3b53400640 1 -- 172.21.15.46:0/3214272458 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b44084620 con 0x7f3b5c0d0dc0 2024-03-29T08:24:52.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.577+0000 7f3b5aa00640 1 --2- 172.21.15.46:0/3214272458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3b380666c0 0x7f3b38068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:52.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.577+0000 7f3b5aa00640 1 --2- 172.21.15.46:0/3214272458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3b380666c0 0x7f3b38068b80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0012f0 tx=0x7f3b4c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:52.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.581+0000 7f3b53400640 1 -- 172.21.15.46:0/3214272458 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3b440519a0 con 0x7f3b5c0d0dc0 2024-03-29T08:24:52.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.741+0000 7f3b61a00640 1 -- 172.21.15.46:0/3214272458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3b5c09e7d0 con 0x7f3b5c0d0dc0 2024-03-29T08:24:52.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.741+0000 7f3b53400640 1 -- 172.21.15.46:0/3214272458 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3b44054fc0 con 0x7f3b5c0d0dc0 2024-03-29T08:24:52.742 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:24:52.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.745+0000 7f3b61a00640 1 -- 172.21.15.46:0/3214272458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3b380666c0 msgr2=0x7f3b38068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.745+0000 7f3b61a00640 1 --2- 172.21.15.46:0/3214272458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3b380666c0 0x7f3b38068b80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c0012f0 tx=0x7f3b4c009040 comp rx=0 tx=0).stop 2024-03-29T08:24:52.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.745+0000 7f3b61a00640 1 -- 172.21.15.46:0/3214272458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c0d0dc0 msgr2=0x7f3b5c14b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.745+0000 7f3b61a00640 1 --2- 172.21.15.46:0/3214272458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c0d0dc0 0x7f3b5c14b7f0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400d5f0 tx=0x7f3b4400dab0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.745+0000 7f3b61a00640 1 -- 172.21.15.46:0/3214272458 shutdown_connections 2024-03-29T08:24:52.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.745+0000 7f3b61a00640 1 --2- 172.21.15.46:0/3214272458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3b380666c0 0x7f3b38068b80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.745+0000 7f3b61a00640 1 --2- 172.21.15.46:0/3214272458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3b5c15cb60 0x7f3b5c14bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.745+0000 7f3b61a00640 1 --2- 172.21.15.46:0/3214272458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b5c0d0dc0 0x7f3b5c14b7f0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.745+0000 7f3b61a00640 1 --2- 172.21.15.46:0/3214272458 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3b5c0cf820 0x7f3b5c153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.745+0000 7f3b61a00640 1 -- 172.21.15.46:0/3214272458 >> 172.21.15.46:0/3214272458 conn(0x7f3b5c099a10 msgr2=0x7f3b5c0b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:52.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.745+0000 7f3b61a00640 1 -- 172.21.15.46:0/3214272458 shutdown_connections 2024-03-29T08:24:52.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.745+0000 7f3b61a00640 1 -- 172.21.15.46:0/3214272458 wait complete. 2024-03-29T08:24:52.764 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:24:52.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.905+0000 7f2e21400640 1 Processor -- start 2024-03-29T08:24:52.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.905+0000 7f2e21400640 1 -- start start 2024-03-29T08:24:52.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.909+0000 7f2e21400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e1c0cbea0 0x7f2e1c0ae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:52.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.909+0000 7f2e21400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1c15d260 con 0x7f2e1c15df90 2024-03-29T08:24:52.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.909+0000 7f2e21400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e1c0aefd0 con 0x7f2e1c142a50 2024-03-29T08:24:52.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.909+0000 7f2e21400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e1c15d4b0 con 0x7f2e1c0cbea0 2024-03-29T08:24:52.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.909+0000 7f2e1b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e1c0cbea0 0x7f2e1c0ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:52.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.909+0000 7f2e1b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e1c0cbea0 0x7f2e1c0ae990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34868/0 (socket says 172.21.15.46:34868) 2024-03-29T08:24:52.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.909+0000 7f2e1b200640 1 -- 172.21.15.46:0/3259414257 learned_addr learned my addr 172.21.15.46:0/3259414257 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:52.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.909+0000 7f2e11e00640 1 --1- 172.21.15.46:0/3259414257 >> v1:172.21.15.46:6789/0 conn(0x7f2e1c15df90 0x7f2e1c0ae280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42104/0 (socket says 172.21.15.46:42104) 2024-03-29T08:24:52.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.909+0000 7f2e1b200640 1 -- 172.21.15.46:0/3259414257 >> v1:172.21.15.73:6789/0 conn(0x7f2e1c142a50 legacy=0x7f2e1c0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.909+0000 7f2e1b200640 1 -- 172.21.15.46:0/3259414257 >> v1:172.21.15.46:6789/0 conn(0x7f2e1c15df90 legacy=0x7f2e1c0ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.909+0000 7f2e1b200640 1 -- 172.21.15.46:0/3259414257 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e1c15d790 con 0x7f2e1c0cbea0 2024-03-29T08:24:52.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.913+0000 7f2e1b200640 1 --2- 172.21.15.46:0/3259414257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e1c0cbea0 0x7f2e1c0ae990 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009e50 tx=0x7f2e1401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f011d40e88124b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.913+0000 7f2e19e00640 1 -- 172.21.15.46:0/3259414257 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e140a5560 con 0x7f2e1c0cbea0 2024-03-29T08:24:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.913+0000 7f2e19e00640 1 -- 172.21.15.46:0/3259414257 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e140a56c0 con 0x7f2e1c0cbea0 2024-03-29T08:24:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.913+0000 7f2e19e00640 1 -- 172.21.15.46:0/3259414257 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e140a5a50 con 0x7f2e1c0cbea0 2024-03-29T08:24:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.913+0000 7f2e21400640 1 -- 172.21.15.46:0/3259414257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e1c0cbea0 msgr2=0x7f2e1c0ae990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.913+0000 7f2e21400640 1 --2- 172.21.15.46:0/3259414257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e1c0cbea0 0x7f2e1c0ae990 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009e50 tx=0x7f2e1401c9a0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.913+0000 7f2e21400640 1 -- 172.21.15.46:0/3259414257 shutdown_connections 2024-03-29T08:24:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.913+0000 7f2e21400640 1 --2- 172.21.15.46:0/3259414257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e1c0cbea0 0x7f2e1c0ae990 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.913+0000 7f2e21400640 1 -- 172.21.15.46:0/3259414257 >> 172.21.15.46:0/3259414257 conn(0x7f2e1c06a390 msgr2=0x7f2e1c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.913+0000 7f2e21400640 1 -- 172.21.15.46:0/3259414257 shutdown_connections 2024-03-29T08:24:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.913+0000 7f2e21400640 1 -- 172.21.15.46:0/3259414257 wait complete. 2024-03-29T08:24:52.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.913+0000 7f2e21400640 1 Processor -- start 2024-03-29T08:24:52.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.913+0000 7f2e21400640 1 -- start start 2024-03-29T08:24:52.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e21400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e1c0cbea0 0x7f2e1c0cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:52.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e21400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e1c142a50 0x7f2e1c156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:52.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e21400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e1c15df90 0x7f2e1c15bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:52.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e21400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c15a650 con 0x7f2e1c0cbea0 2024-03-29T08:24:52.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e21400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c0ce270 con 0x7f2e1c142a50 2024-03-29T08:24:52.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e21400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1c15b140 con 0x7f2e1c15df90 2024-03-29T08:24:52.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e1a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e1c0cbea0 0x7f2e1c0cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:52.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e1a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e1c0cbea0 0x7f2e1c0cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34870/0 (socket says 172.21.15.46:34870) 2024-03-29T08:24:52.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e1a800640 1 -- 172.21.15.46:0/1680893989 learned_addr learned my addr 172.21.15.46:0/1680893989 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:52.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e11e00640 1 --2- 172.21.15.46:0/1680893989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e1c142a50 0x7f2e1c156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:52.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e1b200640 1 --2- 172.21.15.46:0/1680893989 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e1c15df90 0x7f2e1c15bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:52.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e1a800640 1 -- 172.21.15.46:0/1680893989 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e1c15df90 msgr2=0x7f2e1c15bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e1a800640 1 --2- 172.21.15.46:0/1680893989 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e1c15df90 0x7f2e1c15bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e1a800640 1 -- 172.21.15.46:0/1680893989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e1c142a50 msgr2=0x7f2e1c156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:52.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e1a800640 1 --2- 172.21.15.46:0/1680893989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e1c142a50 0x7f2e1c156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:52.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e1a800640 1 -- 172.21.15.46:0/1680893989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e14009b30 con 0x7f2e1c0cbea0 2024-03-29T08:24:52.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e11e00640 1 --2- 172.21.15.46:0/1680893989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e1c142a50 0x7f2e1c156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:52.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e1a800640 1 --2- 172.21.15.46:0/1680893989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e1c0cbea0 0x7f2e1c0cdd30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00c3c0 tx=0x7f2e0c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:52.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e13400640 1 -- 172.21.15.46:0/1680893989 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c0052d0 con 0x7f2e1c0cbea0 2024-03-29T08:24:52.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e13400640 1 -- 172.21.15.46:0/1680893989 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c005430 con 0x7f2e1c0cbea0 2024-03-29T08:24:52.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e13400640 1 -- 172.21.15.46:0/1680893989 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c003840 con 0x7f2e1c0cbea0 2024-03-29T08:24:52.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e21400640 1 -- 172.21.15.46:0/1680893989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e1c15a8a0 con 0x7f2e1c0cbea0 2024-03-29T08:24:52.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e21400640 1 -- 172.21.15.46:0/1680893989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e1c15ae60 con 0x7f2e1c0cbea0 2024-03-29T08:24:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e21400640 1 -- 172.21.15.46:0/1680893989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1c06ad60 con 0x7f2e1c0cbea0 2024-03-29T08:24:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.917+0000 7f2e13400640 1 -- 172.21.15.46:0/1680893989 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2e0c0039a0 con 0x7f2e1c0cbea0 2024-03-29T08:24:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.921+0000 7f2e13400640 1 --2- 172.21.15.46:0/1680893989 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2df8066690 0x7f2df8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.921+0000 7f2e13400640 1 -- 172.21.15.46:0/1680893989 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e0c083770 con 0x7f2e1c0cbea0 2024-03-29T08:24:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.921+0000 7f2e11e00640 1 --2- 172.21.15.46:0/1680893989 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2df8066690 0x7f2df8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.921+0000 7f2e11e00640 1 --2- 172.21.15.46:0/1680893989 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2df8066690 0x7f2df8068b50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c15b540 tx=0x7f2e04009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:52.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:52.925+0000 7f2e13400640 1 -- 172.21.15.46:0/1680893989 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2e0c050a40 con 0x7f2e1c0cbea0 2024-03-29T08:24:53.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.085+0000 7f2e21400640 1 -- 172.21.15.46:0/1680893989 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2e1c071a70 con 0x7f2e1c0cbea0 2024-03-29T08:24:53.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.085+0000 7f2e13400640 1 -- 172.21.15.46:0/1680893989 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2e0c017020 con 0x7f2e1c0cbea0 2024-03-29T08:24:53.087 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:24:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.089+0000 7f2e21400640 1 -- 172.21.15.46:0/1680893989 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2df8066690 msgr2=0x7f2df8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.089+0000 7f2e21400640 1 --2- 172.21.15.46:0/1680893989 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2df8066690 0x7f2df8068b50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c15b540 tx=0x7f2e04009040 comp rx=0 tx=0).stop 2024-03-29T08:24:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.089+0000 7f2e21400640 1 -- 172.21.15.46:0/1680893989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e1c0cbea0 msgr2=0x7f2e1c0cdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.089+0000 7f2e21400640 1 --2- 172.21.15.46:0/1680893989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e1c0cbea0 0x7f2e1c0cdd30 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c00c3c0 tx=0x7f2e0c00c880 comp rx=0 tx=0).stop 2024-03-29T08:24:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.089+0000 7f2e21400640 1 -- 172.21.15.46:0/1680893989 shutdown_connections 2024-03-29T08:24:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.089+0000 7f2e21400640 1 --2- 172.21.15.46:0/1680893989 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2df8066690 0x7f2df8068b50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.089+0000 7f2e21400640 1 --2- 172.21.15.46:0/1680893989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e1c142a50 0x7f2e1c156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.089+0000 7f2e21400640 1 --2- 172.21.15.46:0/1680893989 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e1c15df90 0x7f2e1c15bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.089+0000 7f2e21400640 1 --2- 172.21.15.46:0/1680893989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e1c0cbea0 0x7f2e1c0cdd30 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.089+0000 7f2e21400640 1 -- 172.21.15.46:0/1680893989 >> 172.21.15.46:0/1680893989 conn(0x7f2e1c06a390 msgr2=0x7f2e1c162ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.089+0000 7f2e21400640 1 -- 172.21.15.46:0/1680893989 shutdown_connections 2024-03-29T08:24:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.089+0000 7f2e21400640 1 -- 172.21.15.46:0/1680893989 wait complete. 2024-03-29T08:24:53.109 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:24:53.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.249+0000 7fafd9600640 1 Processor -- start 2024-03-29T08:24:53.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.249+0000 7fafd9600640 1 -- start start 2024-03-29T08:24:53.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.253+0000 7fafd9600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fafd4158300 0x7fafd406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:53.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.253+0000 7fafd9600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fafd40af530 0x7fafd40af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:53.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.253+0000 7fafd9600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd406c390 con 0x7fafd40702a0 2024-03-29T08:24:53.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.253+0000 7fafd9600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafd406d600 con 0x7fafd4158300 2024-03-29T08:24:53.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.253+0000 7fafd9600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd40afff0 con 0x7fafd40af530 2024-03-29T08:24:53.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.253+0000 7fafcbe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fafd40af530 0x7fafd40af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:53.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.253+0000 7fafd2a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fafd4158300 0x7fafd406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:53.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.253+0000 7fafcbe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fafd40af530 0x7fafd40af990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:35878/0 (socket says 172.21.15.46:35878) 2024-03-29T08:24:53.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.253+0000 7fafcbe00640 1 -- 172.21.15.46:0/1497071127 learned_addr learned my addr 172.21.15.46:0/1497071127 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:53.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.253+0000 7fafd2000640 1 -- 172.21.15.46:0/1497071127 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3640267969 0 0) 0x7fafd406c390 con 0x7fafd40702a0 2024-03-29T08:24:53.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.253+0000 7fafd2a00640 1 -- 172.21.15.46:0/1497071127 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fafd40af530 msgr2=0x7fafd40af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:53.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.253+0000 7fafd2a00640 1 --2- 172.21.15.46:0/1497071127 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fafd40af530 0x7fafd40af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.253+0000 7fafd2a00640 1 -- 172.21.15.46:0/1497071127 >> v1:172.21.15.73:6789/0 conn(0x7fafd40702a0 legacy=0x7fafd415e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:53.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.253+0000 7fafd2a00640 1 -- 172.21.15.46:0/1497071127 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd406c5e0 con 0x7fafd4158300 2024-03-29T08:24:53.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.253+0000 7fafcbe00640 1 --2- 172.21.15.46:0/1497071127 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fafd40af530 0x7fafd40af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:24:53.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.257+0000 7fafd2a00640 1 --2- 172.21.15.46:0/1497071127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafd4158300 0x7fafd406d0c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fafc40097c0 tx=0x7fafc401c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1f2bab485b708fc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:53.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.257+0000 7fafd2000640 1 -- 172.21.15.46:0/1497071127 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4004230 con 0x7fafd4158300 2024-03-29T08:24:53.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.257+0000 7fafd2000640 1 -- 172.21.15.46:0/1497071127 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc40a7050 con 0x7fafd4158300 2024-03-29T08:24:53.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.257+0000 7fafd2000640 1 -- 172.21.15.46:0/1497071127 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc4003620 con 0x7fafd4158300 2024-03-29T08:24:53.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.257+0000 7fafd9600640 1 -- 172.21.15.46:0/1497071127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafd4158300 msgr2=0x7fafd406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:53.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.257+0000 7fafd9600640 1 --2- 172.21.15.46:0/1497071127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafd4158300 0x7fafd406d0c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fafc40097c0 tx=0x7fafc401c350 comp rx=0 tx=0).stop 2024-03-29T08:24:53.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.257+0000 7fafd9600640 1 -- 172.21.15.46:0/1497071127 shutdown_connections 2024-03-29T08:24:53.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.257+0000 7fafd9600640 1 --2- 172.21.15.46:0/1497071127 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fafd40af530 0x7fafd40af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.257+0000 7fafd9600640 1 --2- 172.21.15.46:0/1497071127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafd4158300 0x7fafd406d0c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.257+0000 7fafd9600640 1 -- 172.21.15.46:0/1497071127 >> 172.21.15.46:0/1497071127 conn(0x7fafd4066080 msgr2=0x7fafd4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:53.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.257+0000 7fafd9600640 1 -- 172.21.15.46:0/1497071127 shutdown_connections 2024-03-29T08:24:53.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.257+0000 7fafd9600640 1 -- 172.21.15.46:0/1497071127 wait complete. 2024-03-29T08:24:53.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.257+0000 7fafd9600640 1 Processor -- start 2024-03-29T08:24:53.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.257+0000 7fafd9600640 1 -- start start 2024-03-29T08:24:53.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafd9600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafd40702a0 0x7fafd4153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:53.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafd9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafd40af530 0x7fafd41574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:53.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafd9600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fafd4158300 0x7fafd4149eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:53.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafd9600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd4156400 con 0x7fafd40af530 2024-03-29T08:24:53.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafd9600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd41566e0 con 0x7fafd40702a0 2024-03-29T08:24:53.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafd9600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd41569c0 con 0x7fafd4158300 2024-03-29T08:24:53.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafcbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafd40af530 0x7fafd41574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:53.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafcbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafd40af530 0x7fafd41574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34882/0 (socket says 172.21.15.46:34882) 2024-03-29T08:24:53.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafcbe00640 1 -- 172.21.15.46:0/3609647560 learned_addr learned my addr 172.21.15.46:0/3609647560 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:53.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafd2a00640 1 --2- 172.21.15.46:0/3609647560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafd40702a0 0x7fafd4153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:53.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafd3400640 1 --2- 172.21.15.46:0/3609647560 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fafd4158300 0x7fafd4149eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:53.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafcbe00640 1 -- 172.21.15.46:0/3609647560 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fafd4158300 msgr2=0x7fafd4149eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:53.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafcbe00640 1 --2- 172.21.15.46:0/3609647560 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fafd4158300 0x7fafd4149eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafcbe00640 1 -- 172.21.15.46:0/3609647560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafd40702a0 msgr2=0x7fafd4153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:53.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafcbe00640 1 --2- 172.21.15.46:0/3609647560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafd40702a0 0x7fafd4153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafcbe00640 1 -- 172.21.15.46:0/3609647560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc0094d0 con 0x7fafd40af530 2024-03-29T08:24:53.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafcbe00640 1 --2- 172.21.15.46:0/3609647560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafd40af530 0x7fafd41574a0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00e9d0 tx=0x7fafbc00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:53.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafcaa00640 1 -- 172.21.15.46:0/3609647560 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc00ca60 con 0x7fafd40af530 2024-03-29T08:24:53.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafcaa00640 1 -- 172.21.15.46:0/3609647560 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafbc00cbc0 con 0x7fafd40af530 2024-03-29T08:24:53.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafd9600640 1 -- 172.21.15.46:0/3609647560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc4009420 con 0x7fafd40af530 2024-03-29T08:24:53.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafcaa00640 1 -- 172.21.15.46:0/3609647560 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc013470 con 0x7fafd40af530 2024-03-29T08:24:53.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafd9600640 1 -- 172.21.15.46:0/3609647560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd4156f80 con 0x7fafd40af530 2024-03-29T08:24:53.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.261+0000 7fafd9600640 1 -- 172.21.15.46:0/3609647560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd4071a90 con 0x7fafd40af530 2024-03-29T08:24:53.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.265+0000 7fafcaa00640 1 -- 172.21.15.46:0/3609647560 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fafbc0135d0 con 0x7fafd40af530 2024-03-29T08:24:53.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.265+0000 7fafcaa00640 1 --2- 172.21.15.46:0/3609647560 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fafb0066690 0x7fafb0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:53.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.265+0000 7fafcaa00640 1 -- 172.21.15.46:0/3609647560 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fafbc0844e0 con 0x7fafd40af530 2024-03-29T08:24:53.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.265+0000 7fafd2a00640 1 --2- 172.21.15.46:0/3609647560 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fafb0066690 0x7fafb0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:53.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.265+0000 7fafd2a00640 1 --2- 172.21.15.46:0/3609647560 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fafb0066690 0x7fafb0068b50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fafc40ad000 tx=0x7fafc40a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:53.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.269+0000 7fafcaa00640 1 -- 172.21.15.46:0/3609647560 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fafbc051880 con 0x7fafd40af530 2024-03-29T08:24:53.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.441+0000 7fafd9600640 1 -- 172.21.15.46:0/3609647560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fafd404e120 con 0x7fafd40af530 2024-03-29T08:24:53.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.441+0000 7fafcaa00640 1 -- 172.21.15.46:0/3609647560 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v92) v1 ==== 80+0+0 (secure 0 0 0) 0x7fafbc054ea0 con 0x7fafd40af530 2024-03-29T08:24:53.441 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:24:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.441+0000 7fafd9600640 1 -- 172.21.15.46:0/3609647560 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fafb0066690 msgr2=0x7fafb0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.441+0000 7fafd9600640 1 --2- 172.21.15.46:0/3609647560 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fafb0066690 0x7fafb0068b50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fafc40ad000 tx=0x7fafc40a6000 comp rx=0 tx=0).stop 2024-03-29T08:24:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.441+0000 7fafd9600640 1 -- 172.21.15.46:0/3609647560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafd40af530 msgr2=0x7fafd41574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.441+0000 7fafd9600640 1 --2- 172.21.15.46:0/3609647560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafd40af530 0x7fafd41574a0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00e9d0 tx=0x7fafbc00ee90 comp rx=0 tx=0).stop 2024-03-29T08:24:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.441+0000 7fafd9600640 1 -- 172.21.15.46:0/3609647560 shutdown_connections 2024-03-29T08:24:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.441+0000 7fafd9600640 1 --2- 172.21.15.46:0/3609647560 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fafb0066690 0x7fafb0068b50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.441+0000 7fafd9600640 1 --2- 172.21.15.46:0/3609647560 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fafd4158300 0x7fafd4149eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.441+0000 7fafd9600640 1 --2- 172.21.15.46:0/3609647560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fafd40af530 0x7fafd41574a0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.441+0000 7fafd9600640 1 --2- 172.21.15.46:0/3609647560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafd40702a0 0x7fafd4153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.441+0000 7fafd9600640 1 -- 172.21.15.46:0/3609647560 >> 172.21.15.46:0/3609647560 conn(0x7fafd4066080 msgr2=0x7fafd4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.441+0000 7fafd9600640 1 -- 172.21.15.46:0/3609647560 shutdown_connections 2024-03-29T08:24:53.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.441+0000 7fafd9600640 1 -- 172.21.15.46:0/3609647560 wait complete. 2024-03-29T08:24:53.463 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:24:53.463 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:24:53.472 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:24:53.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.657+0000 7f07f5800640 1 Processor -- start 2024-03-29T08:24:53.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.657+0000 7f07f5800640 1 -- start start 2024-03-29T08:24:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.661+0000 7f07f5800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07f00ccd10 0x7f07f0098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.661+0000 7f07f5800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f07f015cdb0 0x7f07f0098f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.661+0000 7f07f5800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f00d0030 con 0x7f07f015c9d0 2024-03-29T08:24:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.661+0000 7f07f5800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07f00cf420 con 0x7f07f00ccd10 2024-03-29T08:24:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.661+0000 7f07f5800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f00cf700 con 0x7f07f015cdb0 2024-03-29T08:24:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.661+0000 7f07e7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07f00ccd10 0x7f07f0098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.661+0000 7f07e7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07f00ccd10 0x7f07f0098a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34890/0 (socket says 172.21.15.46:34890) 2024-03-29T08:24:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.661+0000 7f07e7e00640 1 -- 172.21.15.46:0/399899696 learned_addr learned my addr 172.21.15.46:0/399899696 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.661+0000 7f07ef400640 1 --2- 172.21.15.46:0/399899696 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f07f015cdb0 0x7f07f0098f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.661+0000 7f07e7e00640 1 -- 172.21.15.46:0/399899696 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f07f015cdb0 msgr2=0x7f07f0098f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.661+0000 7f07e7e00640 1 --2- 172.21.15.46:0/399899696 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f07f015cdb0 0x7f07f0098f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.661+0000 7f07e7e00640 1 -- 172.21.15.46:0/399899696 >> v1:172.21.15.73:6789/0 conn(0x7f07f015c9d0 legacy=0x7f07f00cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:53.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.661+0000 7f07e7e00640 1 -- 172.21.15.46:0/399899696 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f00cf9e0 con 0x7f07f00ccd10 2024-03-29T08:24:53.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.661+0000 7f07e7e00640 1 --2- 172.21.15.46:0/399899696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f00ccd10 0x7f07f0098a40 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009870 tx=0x7f07d801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c375b00cb91ea218 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:53.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.661+0000 7f07ee000640 1 -- 172.21.15.46:0/399899696 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d80aa670 con 0x7f07f00ccd10 2024-03-29T08:24:53.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.661+0000 7f07ee000640 1 -- 172.21.15.46:0/399899696 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d80aa7d0 con 0x7f07f00ccd10 2024-03-29T08:24:53.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.661+0000 7f07ee000640 1 -- 172.21.15.46:0/399899696 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d80aab60 con 0x7f07f00ccd10 2024-03-29T08:24:53.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07f5800640 1 -- 172.21.15.46:0/399899696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f00ccd10 msgr2=0x7f07f0098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:53.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07f5800640 1 --2- 172.21.15.46:0/399899696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f00ccd10 0x7f07f0098a40 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f07d8009870 tx=0x7f07d801c530 comp rx=0 tx=0).stop 2024-03-29T08:24:53.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07f5800640 1 -- 172.21.15.46:0/399899696 shutdown_connections 2024-03-29T08:24:53.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07f5800640 1 --2- 172.21.15.46:0/399899696 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f07f015cdb0 0x7f07f0098f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07f5800640 1 --2- 172.21.15.46:0/399899696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f00ccd10 0x7f07f0098a40 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07f5800640 1 -- 172.21.15.46:0/399899696 >> 172.21.15.46:0/399899696 conn(0x7f07f00b21a0 msgr2=0x7f07f00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:53.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07f5800640 1 -- 172.21.15.46:0/399899696 shutdown_connections 2024-03-29T08:24:53.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07f5800640 1 -- 172.21.15.46:0/399899696 wait complete. 2024-03-29T08:24:53.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07f5800640 1 Processor -- start 2024-03-29T08:24:53.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07f5800640 1 -- start start 2024-03-29T08:24:53.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07f5800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f07f00ccd10 0x7f07f0152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:53.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07f5800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f015c9d0 0x7f07f00c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:53.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07f5800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f015cdb0 0x7f07f00c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:53.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07f5800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f0154ee0 con 0x7f07f015c9d0 2024-03-29T08:24:53.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07f5800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f0154710 con 0x7f07f015cdb0 2024-03-29T08:24:53.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07f5800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f01549b0 con 0x7f07f00ccd10 2024-03-29T08:24:53.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f015c9d0 0x7f07f00c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:53.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f015c9d0 0x7f07f00c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34896/0 (socket says 172.21.15.46:34896) 2024-03-29T08:24:53.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07e7e00640 1 -- 172.21.15.46:0/2283422220 learned_addr learned my addr 172.21.15.46:0/2283422220 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:53.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07ef400640 1 --2- 172.21.15.46:0/2283422220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f015cdb0 0x7f07f00c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:53.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07eea00640 1 --2- 172.21.15.46:0/2283422220 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f07f00ccd10 0x7f07f0152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:53.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07e7e00640 1 -- 172.21.15.46:0/2283422220 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f07f00ccd10 msgr2=0x7f07f0152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:53.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07e7e00640 1 --2- 172.21.15.46:0/2283422220 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f07f00ccd10 0x7f07f0152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07e7e00640 1 -- 172.21.15.46:0/2283422220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f015cdb0 msgr2=0x7f07f00c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:53.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07e7e00640 1 --2- 172.21.15.46:0/2283422220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f015cdb0 0x7f07f00c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07e7e00640 1 -- 172.21.15.46:0/2283422220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d80094d0 con 0x7f07f015c9d0 2024-03-29T08:24:53.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.665+0000 7f07ef400640 1 --2- 172.21.15.46:0/2283422220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f015cdb0 0x7f07f00c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:24:53.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.669+0000 7f07e7e00640 1 --2- 172.21.15.46:0/2283422220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f015c9d0 0x7f07f00c99b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f07d80ac040 tx=0x7f07d801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:53.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.669+0000 7f07e6a00640 1 -- 172.21.15.46:0/2283422220 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d80b3800 con 0x7f07f015c9d0 2024-03-29T08:24:53.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.669+0000 7f07e6a00640 1 -- 172.21.15.46:0/2283422220 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d80b3980 con 0x7f07f015c9d0 2024-03-29T08:24:53.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.669+0000 7f07e6a00640 1 -- 172.21.15.46:0/2283422220 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d80b3d30 con 0x7f07f015c9d0 2024-03-29T08:24:53.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.669+0000 7f07f5800640 1 -- 172.21.15.46:0/2283422220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f0155f60 con 0x7f07f015c9d0 2024-03-29T08:24:53.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.669+0000 7f07f5800640 1 -- 172.21.15.46:0/2283422220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f0158330 con 0x7f07f015c9d0 2024-03-29T08:24:53.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.669+0000 7f07f5800640 1 -- 172.21.15.46:0/2283422220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f0066c90 con 0x7f07f015c9d0 2024-03-29T08:24:53.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.669+0000 7f07e6a00640 1 -- 172.21.15.46:0/2283422220 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f07d80b2380 con 0x7f07f015c9d0 2024-03-29T08:24:53.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.669+0000 7f07e6a00640 1 --2- 172.21.15.46:0/2283422220 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07cc066690 0x7f07cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:53.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.669+0000 7f07e6a00640 1 -- 172.21.15.46:0/2283422220 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07d811ea20 con 0x7f07f015c9d0 2024-03-29T08:24:53.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.669+0000 7f07eea00640 1 --2- 172.21.15.46:0/2283422220 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07cc066690 0x7f07cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.673+0000 7f07eea00640 1 --2- 172.21.15.46:0/2283422220 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07cc066690 0x7f07cc068b50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f07f00ccb00 tx=0x7f07e0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:53.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.673+0000 7f07e6a00640 1 -- 172.21.15.46:0/2283422220 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f07d80ebda0 con 0x7f07f015c9d0 2024-03-29T08:24:53.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.837+0000 7f07f5800640 1 -- 172.21.15.46:0/2283422220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07f00b2ec0 con 0x7f07f015c9d0 2024-03-29T08:24:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.837+0000 7f07e6a00640 1 -- 172.21.15.46:0/2283422220 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16056 (secure 0 0 0) 0x7f07d80c3090 con 0x7f07f015c9d0 2024-03-29T08:24:53.838 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:24:53.838 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":92,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:24:44.478513+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:24:41.113618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-29T08:24:42.487380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:24:53.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.841+0000 7f07f5800640 1 -- 172.21.15.46:0/2283422220 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07cc066690 msgr2=0x7f07cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:53.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.841+0000 7f07f5800640 1 --2- 172.21.15.46:0/2283422220 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07cc066690 0x7f07cc068b50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f07f00ccb00 tx=0x7f07e0009040 comp rx=0 tx=0).stop 2024-03-29T08:24:53.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.841+0000 7f07f5800640 1 -- 172.21.15.46:0/2283422220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f015c9d0 msgr2=0x7f07f00c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.841+0000 7f07f5800640 1 --2- 172.21.15.46:0/2283422220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f015c9d0 0x7f07f00c99b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f07d80ac040 tx=0x7f07d801ad70 comp rx=0 tx=0).stop 2024-03-29T08:24:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.841+0000 7f07f5800640 1 -- 172.21.15.46:0/2283422220 shutdown_connections 2024-03-29T08:24:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.841+0000 7f07f5800640 1 --2- 172.21.15.46:0/2283422220 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07cc066690 0x7f07cc068b50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.841+0000 7f07f5800640 1 --2- 172.21.15.46:0/2283422220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07f015cdb0 0x7f07f00c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.841+0000 7f07f5800640 1 --2- 172.21.15.46:0/2283422220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07f015c9d0 0x7f07f00c99b0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.841+0000 7f07f5800640 1 --2- 172.21.15.46:0/2283422220 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f07f00ccd10 0x7f07f0152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.841+0000 7f07f5800640 1 -- 172.21.15.46:0/2283422220 >> 172.21.15.46:0/2283422220 conn(0x7f07f00b21a0 msgr2=0x7f07f006f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.841+0000 7f07f5800640 1 -- 172.21.15.46:0/2283422220 shutdown_connections 2024-03-29T08:24:53.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:53.841+0000 7f07f5800640 1 -- 172.21.15.46:0/2283422220 wait complete. 2024-03-29T08:24:53.860 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:24:41.113618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:24:42.487380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:24:53.860 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:24:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.005+0000 7fa565800640 1 Processor -- start 2024-03-29T08:24:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.005+0000 7fa565800640 1 -- start start 2024-03-29T08:24:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.005+0000 7fa565800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa5600cea30 0x7fa5601606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.005+0000 7fa565800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa56015cd30 0x7fa560160be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.005+0000 7fa565800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5600cfb20 con 0x7fa56015c950 2024-03-29T08:24:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.005+0000 7fa565800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5600cef10 con 0x7fa56015cd30 2024-03-29T08:24:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.005+0000 7fa565800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5600cf1f0 con 0x7fa5600cea30 2024-03-29T08:24:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.005+0000 7fa55f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa56015cd30 0x7fa560160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.005+0000 7fa55f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa56015cd30 0x7fa560160be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34902/0 (socket says 172.21.15.46:34902) 2024-03-29T08:24:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.005+0000 7fa55f400640 1 -- 172.21.15.46:0/499469001 learned_addr learned my addr 172.21.15.46:0/499469001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.005+0000 7fa557e00640 1 --2- 172.21.15.46:0/499469001 >> v2:172.21.15.73:3300/0 conn(0x7fa5600cea30 0x7fa5601606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.005+0000 7fa55f400640 1 -- 172.21.15.46:0/499469001 >> v2:172.21.15.73:3300/0 conn(0x7fa5600cea30 msgr2=0x7fa5601606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.005+0000 7fa55f400640 1 --2- 172.21.15.46:0/499469001 >> v2:172.21.15.73:3300/0 conn(0x7fa5600cea30 0x7fa5601606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.005+0000 7fa55f400640 1 -- 172.21.15.46:0/499469001 >> v1:172.21.15.73:6789/0 conn(0x7fa56015c950 legacy=0x7fa56006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.005+0000 7fa55f400640 1 -- 172.21.15.46:0/499469001 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5600cf4d0 con 0x7fa56015cd30 2024-03-29T08:24:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.009+0000 7fa55f400640 1 --2- 172.21.15.46:0/499469001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56015cd30 0x7fa560160be0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fa5600c41c0 tx=0x7fa55801f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8504b927de1f0fe7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.009+0000 7fa55e000640 1 -- 172.21.15.46:0/499469001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5580af400 con 0x7fa56015cd30 2024-03-29T08:24:54.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.009+0000 7fa55e000640 1 -- 172.21.15.46:0/499469001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5580ab050 con 0x7fa56015cd30 2024-03-29T08:24:54.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.009+0000 7fa55e000640 1 -- 172.21.15.46:0/499469001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5580af780 con 0x7fa56015cd30 2024-03-29T08:24:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.009+0000 7fa565800640 1 -- 172.21.15.46:0/499469001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56015cd30 msgr2=0x7fa560160be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.009+0000 7fa565800640 1 --2- 172.21.15.46:0/499469001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56015cd30 0x7fa560160be0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fa5600c41c0 tx=0x7fa55801f1e0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.009+0000 7fa565800640 1 -- 172.21.15.46:0/499469001 shutdown_connections 2024-03-29T08:24:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.009+0000 7fa565800640 1 --2- 172.21.15.46:0/499469001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa56015cd30 0x7fa560160be0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.009+0000 7fa565800640 1 --2- 172.21.15.46:0/499469001 >> v2:172.21.15.73:3300/0 conn(0x7fa5600cea30 0x7fa5601606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.009+0000 7fa565800640 1 -- 172.21.15.46:0/499469001 >> 172.21.15.46:0/499469001 conn(0x7fa5600b21b0 msgr2=0x7fa5600b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.009+0000 7fa565800640 1 -- 172.21.15.46:0/499469001 shutdown_connections 2024-03-29T08:24:54.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.009+0000 7fa565800640 1 -- 172.21.15.46:0/499469001 wait complete. 2024-03-29T08:24:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa565800640 1 Processor -- start 2024-03-29T08:24:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa565800640 1 -- start start 2024-03-29T08:24:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa565800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5600cea30 0x7fa5601e8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa565800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa56015c950 0x7fa560154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa565800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa56015cd30 0x7fa5600b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa565800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5600c4bb0 con 0x7fa5600cea30 2024-03-29T08:24:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa565800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5600b4460 con 0x7fa56015cd30 2024-03-29T08:24:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa565800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5600b4740 con 0x7fa56015c950 2024-03-29T08:24:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa55ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5600cea30 0x7fa5601e8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa55ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5600cea30 0x7fa5601e8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34910/0 (socket says 172.21.15.46:34910) 2024-03-29T08:24:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa55ea00640 1 -- 172.21.15.46:0/4066055080 learned_addr learned my addr 172.21.15.46:0/4066055080 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa557e00640 1 --2- 172.21.15.46:0/4066055080 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa56015c950 0x7fa560154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa55f400640 1 --2- 172.21.15.46:0/4066055080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa56015cd30 0x7fa5600b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa55ea00640 1 -- 172.21.15.46:0/4066055080 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa56015c950 msgr2=0x7fa560154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa55ea00640 1 --2- 172.21.15.46:0/4066055080 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa56015c950 0x7fa560154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa55ea00640 1 -- 172.21.15.46:0/4066055080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa56015cd30 msgr2=0x7fa5600b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa55ea00640 1 --2- 172.21.15.46:0/4066055080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa56015cd30 0x7fa5600b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa55ea00640 1 -- 172.21.15.46:0/4066055080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa558009b30 con 0x7fa5600cea30 2024-03-29T08:24:54.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa55ea00640 1 --2- 172.21.15.46:0/4066055080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5600cea30 0x7fa5601e8900 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa55000c500 tx=0x7fa55000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa556a00640 1 -- 172.21.15.46:0/4066055080 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550005220 con 0x7fa5600cea30 2024-03-29T08:24:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa556a00640 1 -- 172.21.15.46:0/4066055080 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa550005380 con 0x7fa5600cea30 2024-03-29T08:24:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa556a00640 1 -- 172.21.15.46:0/4066055080 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550007270 con 0x7fa5600cea30 2024-03-29T08:24:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa565800640 1 -- 172.21.15.46:0/4066055080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56006c470 con 0x7fa5600cea30 2024-03-29T08:24:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.013+0000 7fa565800640 1 -- 172.21.15.46:0/4066055080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5600b4d00 con 0x7fa5600cea30 2024-03-29T08:24:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.017+0000 7fa565800640 1 -- 172.21.15.46:0/4066055080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56006c270 con 0x7fa5600cea30 2024-03-29T08:24:54.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.017+0000 7fa556a00640 1 -- 172.21.15.46:0/4066055080 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa5500073d0 con 0x7fa5600cea30 2024-03-29T08:24:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.017+0000 7fa556a00640 1 --2- 172.21.15.46:0/4066055080 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa53c066690 0x7fa53c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.017+0000 7fa556a00640 1 -- 172.21.15.46:0/4066055080 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa550083a20 con 0x7fa5600cea30 2024-03-29T08:24:54.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.017+0000 7fa557e00640 1 --2- 172.21.15.46:0/4066055080 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa53c066690 0x7fa53c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.017+0000 7fa557e00640 1 --2- 172.21.15.46:0/4066055080 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa53c066690 0x7fa53c068b50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa560158570 tx=0x7fa548008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:54.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.021+0000 7fa556a00640 1 -- 172.21.15.46:0/4066055080 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa550050db0 con 0x7fa5600cea30 2024-03-29T08:24:54.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.181+0000 7fa565800640 1 -- 172.21.15.46:0/4066055080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa56009e690 con 0x7fa5600cea30 2024-03-29T08:24:54.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.181+0000 7fa556a00640 1 -- 172.21.15.46:0/4066055080 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5500543d0 con 0x7fa5600cea30 2024-03-29T08:24:54.183 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:24:54.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.185+0000 7fa565800640 1 -- 172.21.15.46:0/4066055080 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa53c066690 msgr2=0x7fa53c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.185+0000 7fa565800640 1 --2- 172.21.15.46:0/4066055080 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa53c066690 0x7fa53c068b50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa560158570 tx=0x7fa548008040 comp rx=0 tx=0).stop 2024-03-29T08:24:54.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.185+0000 7fa565800640 1 -- 172.21.15.46:0/4066055080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5600cea30 msgr2=0x7fa5601e8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.185+0000 7fa565800640 1 --2- 172.21.15.46:0/4066055080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5600cea30 0x7fa5601e8900 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa55000c500 tx=0x7fa55000c9c0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.185+0000 7fa565800640 1 -- 172.21.15.46:0/4066055080 shutdown_connections 2024-03-29T08:24:54.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.185+0000 7fa565800640 1 --2- 172.21.15.46:0/4066055080 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa53c066690 0x7fa53c068b50 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.185+0000 7fa565800640 1 --2- 172.21.15.46:0/4066055080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa56015cd30 0x7fa5600b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.185+0000 7fa565800640 1 --2- 172.21.15.46:0/4066055080 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa56015c950 0x7fa560154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.185+0000 7fa565800640 1 --2- 172.21.15.46:0/4066055080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5600cea30 0x7fa5601e8900 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.185+0000 7fa565800640 1 -- 172.21.15.46:0/4066055080 >> 172.21.15.46:0/4066055080 conn(0x7fa5600b21b0 msgr2=0x7fa56015db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:54.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.185+0000 7fa565800640 1 -- 172.21.15.46:0/4066055080 shutdown_connections 2024-03-29T08:24:54.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.185+0000 7fa565800640 1 -- 172.21.15.46:0/4066055080 wait complete. 2024-03-29T08:24:54.204 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:24:54.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.349+0000 7fc101800640 1 Processor -- start 2024-03-29T08:24:54.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.349+0000 7fc101800640 1 -- start start 2024-03-29T08:24:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.349+0000 7fc101800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc0fc0d0dc0 0x7fc0fc0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.349+0000 7fc101800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0fc0cf820 0x7fc0fc0b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.349+0000 7fc101800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0fc15cf40 con 0x7fc0fc15cb60 2024-03-29T08:24:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.349+0000 7fc101800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0fc0b0250 con 0x7fc0fc0cf820 2024-03-29T08:24:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.349+0000 7fc101800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0fc0b0810 con 0x7fc0fc0d0dc0 2024-03-29T08:24:54.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.349+0000 7fc0faa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc0fc15cb60 0x7fc0fc0d06b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42116/0 (socket says 172.21.15.46:42116) 2024-03-29T08:24:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.349+0000 7fc0faa00640 1 -- 172.21.15.46:0/2536777959 learned_addr learned my addr 172.21.15.46:0/2536777959 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.349+0000 7fc0fb400640 1 --2- 172.21.15.46:0/2536777959 >> v2:172.21.15.46:3300/0 conn(0x7fc0fc0cf820 0x7fc0fc0b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.349+0000 7fc0f2000640 1 --2- 172.21.15.46:0/2536777959 >> v2:172.21.15.73:3300/0 conn(0x7fc0fc0d0dc0 0x7fc0fc0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.349+0000 7fc0fa000640 1 -- 172.21.15.46:0/2536777959 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3799443696 0 0) 0x7fc0fc15cf40 con 0x7fc0fc15cb60 2024-03-29T08:24:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.349+0000 7fc0fa000640 1 -- 172.21.15.46:0/2536777959 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0e0003530 con 0x7fc0fc15cb60 2024-03-29T08:24:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.349+0000 7fc0fb400640 1 -- 172.21.15.46:0/2536777959 >> v2:172.21.15.73:3300/0 conn(0x7fc0fc0d0dc0 msgr2=0x7fc0fc0cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.349+0000 7fc0fb400640 1 --2- 172.21.15.46:0/2536777959 >> v2:172.21.15.73:3300/0 conn(0x7fc0fc0d0dc0 0x7fc0fc0cf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.349+0000 7fc0fb400640 1 -- 172.21.15.46:0/2536777959 >> v1:172.21.15.46:6789/0 conn(0x7fc0fc15cb60 legacy=0x7fc0fc0d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.349+0000 7fc0fb400640 1 -- 172.21.15.46:0/2536777959 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc0d0010 con 0x7fc0fc0cf820 2024-03-29T08:24:54.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.353+0000 7fc0fb400640 1 --2- 172.21.15.46:0/2536777959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0fc0cf820 0x7fc0fc0b0ca0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc160530 tx=0x7fc0f401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb69c503ec6bfe0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:54.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.353+0000 7fc0fa000640 1 -- 172.21.15.46:0/2536777959 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40a59c0 con 0x7fc0fc0cf820 2024-03-29T08:24:54.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.353+0000 7fc0fa000640 1 -- 172.21.15.46:0/2536777959 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f40a5b20 con 0x7fc0fc0cf820 2024-03-29T08:24:54.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.353+0000 7fc0fa000640 1 -- 172.21.15.46:0/2536777959 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f40ad440 con 0x7fc0fc0cf820 2024-03-29T08:24:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.353+0000 7fc101800640 1 -- 172.21.15.46:0/2536777959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0fc0cf820 msgr2=0x7fc0fc0b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.353+0000 7fc101800640 1 --2- 172.21.15.46:0/2536777959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0fc0cf820 0x7fc0fc0b0ca0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc160530 tx=0x7fc0f401c900 comp rx=0 tx=0).stop 2024-03-29T08:24:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.353+0000 7fc101800640 1 -- 172.21.15.46:0/2536777959 shutdown_connections 2024-03-29T08:24:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.353+0000 7fc101800640 1 --2- 172.21.15.46:0/2536777959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0fc0cf820 0x7fc0fc0b0ca0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.353+0000 7fc101800640 1 --2- 172.21.15.46:0/2536777959 >> v2:172.21.15.73:3300/0 conn(0x7fc0fc0d0dc0 0x7fc0fc0cf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.353+0000 7fc101800640 1 -- 172.21.15.46:0/2536777959 >> 172.21.15.46:0/2536777959 conn(0x7fc0fc099a10 msgr2=0x7fc0fc089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.353+0000 7fc101800640 1 -- 172.21.15.46:0/2536777959 shutdown_connections 2024-03-29T08:24:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.353+0000 7fc101800640 1 -- 172.21.15.46:0/2536777959 wait complete. 2024-03-29T08:24:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.353+0000 7fc101800640 1 Processor -- start 2024-03-29T08:24:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc101800640 1 -- start start 2024-03-29T08:24:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc101800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0fc0cf820 0x7fc0fc0cb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc101800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0fc0d0dc0 0x7fc0fc0ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc101800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0fc15cb60 0x7fc0fc155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc101800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0c4c60 con 0x7fc0fc0d0dc0 2024-03-29T08:24:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc101800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0c5a80 con 0x7fc0fc15cb60 2024-03-29T08:24:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc101800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc155fb0 con 0x7fc0fc0cf820 2024-03-29T08:24:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc0f2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0fc0d0dc0 0x7fc0fc0ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc0f2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0fc0d0dc0 0x7fc0fc0ae150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34932/0 (socket says 172.21.15.46:34932) 2024-03-29T08:24:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc0f2000640 1 -- 172.21.15.46:0/1531958913 learned_addr learned my addr 172.21.15.46:0/1531958913 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc0faa00640 1 --2- 172.21.15.46:0/1531958913 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0fc0cf820 0x7fc0fc0cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc0fb400640 1 --2- 172.21.15.46:0/1531958913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0fc15cb60 0x7fc0fc155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc0f2000640 1 -- 172.21.15.46:0/1531958913 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0fc0cf820 msgr2=0x7fc0fc0cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc0f2000640 1 --2- 172.21.15.46:0/1531958913 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0fc0cf820 0x7fc0fc0cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc0f2000640 1 -- 172.21.15.46:0/1531958913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0fc15cb60 msgr2=0x7fc0fc155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc0f2000640 1 --2- 172.21.15.46:0/1531958913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0fc15cb60 0x7fc0fc155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc0f2000640 1 -- 172.21.15.46:0/1531958913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f4009a70 con 0x7fc0fc0d0dc0 2024-03-29T08:24:54.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc0faa00640 1 --2- 172.21.15.46:0/1531958913 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0fc0cf820 0x7fc0fc0cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:24:54.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc0fb400640 1 --2- 172.21.15.46:0/1531958913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0fc15cb60 0x7fc0fc155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:24:54.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc0f2000640 1 --2- 172.21.15.46:0/1531958913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0fc0d0dc0 0x7fc0fc0ae150 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400e930 tx=0x7fc0e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:54.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc0f3400640 1 -- 172.21.15.46:0/1531958913 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4009690 con 0x7fc0fc0d0dc0 2024-03-29T08:24:54.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc0f3400640 1 -- 172.21.15.46:0/1531958913 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e400ca00 con 0x7fc0fc0d0dc0 2024-03-29T08:24:54.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc0f3400640 1 -- 172.21.15.46:0/1531958913 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e400cd70 con 0x7fc0fc0d0dc0 2024-03-29T08:24:54.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc101800640 1 -- 172.21.15.46:0/1531958913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0fc0c4e70 con 0x7fc0fc0d0dc0 2024-03-29T08:24:54.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc101800640 1 -- 172.21.15.46:0/1531958913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0fc0c5430 con 0x7fc0fc0d0dc0 2024-03-29T08:24:54.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.357+0000 7fc101800640 1 -- 172.21.15.46:0/1531958913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0fc0c5bf0 con 0x7fc0fc0d0dc0 2024-03-29T08:24:54.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.361+0000 7fc0f3400640 1 -- 172.21.15.46:0/1531958913 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc0e4013470 con 0x7fc0fc0d0dc0 2024-03-29T08:24:54.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.361+0000 7fc0f3400640 1 --2- 172.21.15.46:0/1531958913 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0d8066690 0x7fc0d8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.361+0000 7fc0f3400640 1 -- 172.21.15.46:0/1531958913 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0e4084460 con 0x7fc0fc0d0dc0 2024-03-29T08:24:54.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.361+0000 7fc0faa00640 1 --2- 172.21.15.46:0/1531958913 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0d8066690 0x7fc0d8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.361+0000 7fc0faa00640 1 --2- 172.21.15.46:0/1531958913 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0d8066690 0x7fc0d8068b50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec0011f0 tx=0x7fc0ec009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:54.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.365+0000 7fc0f3400640 1 -- 172.21.15.46:0/1531958913 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc0e4051870 con 0x7fc0fc0d0dc0 2024-03-29T08:24:54.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.525+0000 7fc101800640 1 -- 172.21.15.46:0/1531958913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc0fc09e7d0 con 0x7fc0fc0d0dc0 2024-03-29T08:24:54.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.525+0000 7fc0f3400640 1 -- 172.21.15.46:0/1531958913 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc0e4054e90 con 0x7fc0fc0d0dc0 2024-03-29T08:24:54.526 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:24:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.529+0000 7fc101800640 1 -- 172.21.15.46:0/1531958913 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0d8066690 msgr2=0x7fc0d8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.529+0000 7fc101800640 1 --2- 172.21.15.46:0/1531958913 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0d8066690 0x7fc0d8068b50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec0011f0 tx=0x7fc0ec009040 comp rx=0 tx=0).stop 2024-03-29T08:24:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.529+0000 7fc101800640 1 -- 172.21.15.46:0/1531958913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0fc0d0dc0 msgr2=0x7fc0fc0ae150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.529+0000 7fc101800640 1 --2- 172.21.15.46:0/1531958913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0fc0d0dc0 0x7fc0fc0ae150 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400e930 tx=0x7fc0e400edf0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.529+0000 7fc101800640 1 -- 172.21.15.46:0/1531958913 shutdown_connections 2024-03-29T08:24:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.529+0000 7fc101800640 1 --2- 172.21.15.46:0/1531958913 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0d8066690 0x7fc0d8068b50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.529+0000 7fc101800640 1 --2- 172.21.15.46:0/1531958913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0fc15cb60 0x7fc0fc155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.529+0000 7fc101800640 1 --2- 172.21.15.46:0/1531958913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0fc0d0dc0 0x7fc0fc0ae150 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.529+0000 7fc101800640 1 --2- 172.21.15.46:0/1531958913 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0fc0cf820 0x7fc0fc0cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.529+0000 7fc101800640 1 -- 172.21.15.46:0/1531958913 >> 172.21.15.46:0/1531958913 conn(0x7fc0fc099a10 msgr2=0x7fc0fc0cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.529+0000 7fc101800640 1 -- 172.21.15.46:0/1531958913 shutdown_connections 2024-03-29T08:24:54.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.529+0000 7fc101800640 1 -- 172.21.15.46:0/1531958913 wait complete. 2024-03-29T08:24:54.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:24:54.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.689+0000 7fa5b0800640 1 Processor -- start 2024-03-29T08:24:54.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.689+0000 7fa5b0800640 1 -- start start 2024-03-29T08:24:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.689+0000 7fa5b0800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5a815c9c0 0x7fa5a80afde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.689+0000 7fa5b0800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa5a80ce8b0 0x7fa5a80ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.689+0000 7fa5b0800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a80af950 con 0x7fa5a8069d90 2024-03-29T08:24:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.689+0000 7fa5b0800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a80cf390 con 0x7fa5a80ce8b0 2024-03-29T08:24:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.689+0000 7fa5b0800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a80af020 con 0x7fa5a815c9c0 2024-03-29T08:24:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.689+0000 7fa5ae800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa5a8069d90 0x7fa5a80ac310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42120/0 (socket says 172.21.15.46:42120) 2024-03-29T08:24:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.689+0000 7fa5ae800640 1 -- 172.21.15.46:0/1773804689 learned_addr learned my addr 172.21.15.46:0/1773804689 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:24:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.689+0000 7fa5ade00640 1 --2- 172.21.15.46:0/1773804689 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5a815c9c0 0x7fa5a80afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.689+0000 7fa5ad400640 1 --2- 172.21.15.46:0/1773804689 >> v2:172.21.15.73:3300/0 conn(0x7fa5a80ce8b0 0x7fa5a80ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.689+0000 7fa5aca00640 1 -- 172.21.15.46:0/1773804689 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816739296 0 0) 0x7fa5a80af950 con 0x7fa5a8069d90 2024-03-29T08:24:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.693+0000 7fa5aca00640 1 -- 172.21.15.46:0/1773804689 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa590003530 con 0x7fa5a8069d90 2024-03-29T08:24:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.693+0000 7fa5ade00640 1 -- 172.21.15.46:0/1773804689 >> v2:172.21.15.73:3300/0 conn(0x7fa5a80ce8b0 msgr2=0x7fa5a80ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.693+0000 7fa5ade00640 1 --2- 172.21.15.46:0/1773804689 >> v2:172.21.15.73:3300/0 conn(0x7fa5a80ce8b0 0x7fa5a80ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.693+0000 7fa5ade00640 1 -- 172.21.15.46:0/1773804689 >> v1:172.21.15.46:6789/0 conn(0x7fa5a8069d90 legacy=0x7fa5a80ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.693+0000 7fa5ade00640 1 -- 172.21.15.46:0/1773804689 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a80af300 con 0x7fa5a815c9c0 2024-03-29T08:24:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.693+0000 7fa5ad400640 1 --2- 172.21.15.46:0/1773804689 >> v2:172.21.15.73:3300/0 conn(0x7fa5a80ce8b0 0x7fa5a80ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:24:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.693+0000 7fa5ade00640 1 --2- 172.21.15.46:0/1773804689 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5a815c9c0 0x7fa5a80afde0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0097c0 tx=0x7fa59c01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e486d1e18c62cc3b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.693+0000 7fa5aca00640 1 -- 172.21.15.46:0/1773804689 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0ac630 con 0x7fa5a815c9c0 2024-03-29T08:24:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.693+0000 7fa5aca00640 1 -- 172.21.15.46:0/1773804689 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59c0ac790 con 0x7fa5a815c9c0 2024-03-29T08:24:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.693+0000 7fa5aca00640 1 -- 172.21.15.46:0/1773804689 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c0acb00 con 0x7fa5a815c9c0 2024-03-29T08:24:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5b0800640 1 -- 172.21.15.46:0/1773804689 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5a815c9c0 msgr2=0x7fa5a80afde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5b0800640 1 --2- 172.21.15.46:0/1773804689 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5a815c9c0 0x7fa5a80afde0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0097c0 tx=0x7fa59c01c470 comp rx=0 tx=0).stop 2024-03-29T08:24:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5b0800640 1 -- 172.21.15.46:0/1773804689 shutdown_connections 2024-03-29T08:24:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5b0800640 1 --2- 172.21.15.46:0/1773804689 >> v2:172.21.15.73:3300/0 conn(0x7fa5a80ce8b0 0x7fa5a80ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5b0800640 1 --2- 172.21.15.46:0/1773804689 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5a815c9c0 0x7fa5a80afde0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5b0800640 1 -- 172.21.15.46:0/1773804689 >> 172.21.15.46:0/1773804689 conn(0x7fa5a8098a40 msgr2=0x7fa5a80cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5b0800640 1 -- 172.21.15.46:0/1773804689 shutdown_connections 2024-03-29T08:24:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5b0800640 1 -- 172.21.15.46:0/1773804689 wait complete. 2024-03-29T08:24:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5b0800640 1 Processor -- start 2024-03-29T08:24:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5b0800640 1 -- start start 2024-03-29T08:24:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5b0800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5a8069d90 0x7fa5a8153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5b0800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5a80ce8b0 0x7fa5a815b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5b0800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a815c9c0 0x7fa5a815b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5b0800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a809d100 con 0x7fa5a815c9c0 2024-03-29T08:24:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5b0800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a81544d0 con 0x7fa5a80ce8b0 2024-03-29T08:24:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5b0800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a81536b0 con 0x7fa5a8069d90 2024-03-29T08:24:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5ade00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5a8069d90 0x7fa5a8153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5ae800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a815c9c0 0x7fa5a815b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5ade00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5a8069d90 0x7fa5a8153170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:35932/0 (socket says 172.21.15.46:35932) 2024-03-29T08:24:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5ade00640 1 -- 172.21.15.46:0/68064647 learned_addr learned my addr 172.21.15.46:0/68064647 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5ad400640 1 --2- 172.21.15.46:0/68064647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5a80ce8b0 0x7fa5a815b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5ae800640 1 -- 172.21.15.46:0/68064647 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5a8069d90 msgr2=0x7fa5a8153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5ae800640 1 --2- 172.21.15.46:0/68064647 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5a8069d90 0x7fa5a8153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5ae800640 1 -- 172.21.15.46:0/68064647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5a80ce8b0 msgr2=0x7fa5a815b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5ae800640 1 --2- 172.21.15.46:0/68064647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5a80ce8b0 0x7fa5a815b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.697+0000 7fa5ae800640 1 -- 172.21.15.46:0/68064647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa598009420 con 0x7fa5a815c9c0 2024-03-29T08:24:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.701+0000 7fa5ae800640 1 --2- 172.21.15.46:0/68064647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a815c9c0 0x7fa5a815b860 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400caa0 tx=0x7fa5a400cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.701+0000 7fa5a2000640 1 -- 172.21.15.46:0/68064647 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a400ab40 con 0x7fa5a815c9c0 2024-03-29T08:24:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.701+0000 7fa5a2000640 1 -- 172.21.15.46:0/68064647 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a400aca0 con 0x7fa5a815c9c0 2024-03-29T08:24:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.701+0000 7fa5b0800640 1 -- 172.21.15.46:0/68064647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59c009420 con 0x7fa5a815c9c0 2024-03-29T08:24:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.701+0000 7fa5a2000640 1 -- 172.21.15.46:0/68064647 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4012600 con 0x7fa5a815c9c0 2024-03-29T08:24:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.701+0000 7fa5b0800640 1 -- 172.21.15.46:0/68064647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a8153c30 con 0x7fa5a815c9c0 2024-03-29T08:24:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.701+0000 7fa5b0800640 1 -- 172.21.15.46:0/68064647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a809d690 con 0x7fa5a815c9c0 2024-03-29T08:24:54.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.701+0000 7fa5a2000640 1 -- 172.21.15.46:0/68064647 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa5a4007070 con 0x7fa5a815c9c0 2024-03-29T08:24:54.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.701+0000 7fa5a2000640 1 --2- 172.21.15.46:0/68064647 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa58c066690 0x7fa58c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:54.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.701+0000 7fa5a2000640 1 -- 172.21.15.46:0/68064647 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5a4083fc0 con 0x7fa5a815c9c0 2024-03-29T08:24:54.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.701+0000 7fa5ade00640 1 --2- 172.21.15.46:0/68064647 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa58c066690 0x7fa58c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:54.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.705+0000 7fa5ade00640 1 --2- 172.21.15.46:0/68064647 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa58c066690 0x7fa58c068b50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0097c0 tx=0x7fa59c0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:54.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.705+0000 7fa5a2000640 1 -- 172.21.15.46:0/68064647 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa5a4051340 con 0x7fa5a815c9c0 2024-03-29T08:24:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.865+0000 7fa5b0800640 1 -- 172.21.15.46:0/68064647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa5a80c44a0 con 0x7fa5a815c9c0 2024-03-29T08:24:54.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.865+0000 7fa5a2000640 1 -- 172.21.15.46:0/68064647 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa5a40170a0 con 0x7fa5a815c9c0 2024-03-29T08:24:54.866 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:24:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.865+0000 7fa5b0800640 1 -- 172.21.15.46:0/68064647 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa58c066690 msgr2=0x7fa58c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.865+0000 7fa5b0800640 1 --2- 172.21.15.46:0/68064647 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa58c066690 0x7fa58c068b50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0097c0 tx=0x7fa59c0a7000 comp rx=0 tx=0).stop 2024-03-29T08:24:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.865+0000 7fa5b0800640 1 -- 172.21.15.46:0/68064647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a815c9c0 msgr2=0x7fa5a815b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.865+0000 7fa5b0800640 1 --2- 172.21.15.46:0/68064647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a815c9c0 0x7fa5a815b860 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fa5a400caa0 tx=0x7fa5a400cea0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.865+0000 7fa5b0800640 1 -- 172.21.15.46:0/68064647 shutdown_connections 2024-03-29T08:24:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.865+0000 7fa5b0800640 1 --2- 172.21.15.46:0/68064647 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa58c066690 0x7fa58c068b50 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.865+0000 7fa5b0800640 1 --2- 172.21.15.46:0/68064647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5a80ce8b0 0x7fa5a815b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.865+0000 7fa5b0800640 1 --2- 172.21.15.46:0/68064647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5a815c9c0 0x7fa5a815b860 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.865+0000 7fa5b0800640 1 --2- 172.21.15.46:0/68064647 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5a8069d90 0x7fa5a8153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.865+0000 7fa5b0800640 1 -- 172.21.15.46:0/68064647 >> 172.21.15.46:0/68064647 conn(0x7fa5a8098a40 msgr2=0x7fa5a80ad130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.865+0000 7fa5b0800640 1 -- 172.21.15.46:0/68064647 shutdown_connections 2024-03-29T08:24:54.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:54.865+0000 7fa5b0800640 1 -- 172.21.15.46:0/68064647 wait complete. 2024-03-29T08:24:54.887 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:24:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.029+0000 7f9227000640 1 Processor -- start 2024-03-29T08:24:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.029+0000 7f9227000640 1 -- start start 2024-03-29T08:24:55.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.029+0000 7f9227000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9220142ab0 0x7f922006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:55.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.029+0000 7f9227000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f922015db10 0x7f92200b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:55.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.029+0000 7f9227000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922006de40 con 0x7f922006d9e0 2024-03-29T08:24:55.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.029+0000 7f9227000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9220070c70 con 0x7f9220142ab0 2024-03-29T08:24:55.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.029+0000 7f9227000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9220070340 con 0x7f922015db10 2024-03-29T08:24:55.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.029+0000 7f921fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9220142ab0 0x7f922006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:55.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.033+0000 7f921fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9220142ab0 0x7f922006d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34956/0 (socket says 172.21.15.46:34956) 2024-03-29T08:24:55.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.033+0000 7f921fe00640 1 -- 172.21.15.46:0/100665491 learned_addr learned my addr 172.21.15.46:0/100665491 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:55.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.033+0000 7f9225000640 1 --2- 172.21.15.46:0/100665491 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f922015db10 0x7f92200b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:55.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.033+0000 7f921fe00640 1 -- 172.21.15.46:0/100665491 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f922015db10 msgr2=0x7f92200b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:24:55.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.033+0000 7f921fe00640 1 --2- 172.21.15.46:0/100665491 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f922015db10 0x7f92200b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:55.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.033+0000 7f921fe00640 1 -- 172.21.15.46:0/100665491 >> v1:172.21.15.73:6789/0 conn(0x7f922006d9e0 legacy=0x7f922015d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:55.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.033+0000 7f921fe00640 1 -- 172.21.15.46:0/100665491 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9220070620 con 0x7f9220142ab0 2024-03-29T08:24:55.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.033+0000 7f921fe00640 1 --2- 172.21.15.46:0/100665491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220142ab0 0x7f922006d4a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f9210009870 tx=0x7f921001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d1b09c2fff8a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:55.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.033+0000 7f921f400640 1 -- 172.21.15.46:0/100665491 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92100ab600 con 0x7f9220142ab0 2024-03-29T08:24:55.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.033+0000 7f921f400640 1 -- 172.21.15.46:0/100665491 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92100a7050 con 0x7f9220142ab0 2024-03-29T08:24:55.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.033+0000 7f921f400640 1 -- 172.21.15.46:0/100665491 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92100ab980 con 0x7f9220142ab0 2024-03-29T08:24:55.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9227000640 1 -- 172.21.15.46:0/100665491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220142ab0 msgr2=0x7f922006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:55.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9227000640 1 --2- 172.21.15.46:0/100665491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220142ab0 0x7f922006d4a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f9210009870 tx=0x7f921001c560 comp rx=0 tx=0).stop 2024-03-29T08:24:55.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9227000640 1 -- 172.21.15.46:0/100665491 shutdown_connections 2024-03-29T08:24:55.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9227000640 1 --2- 172.21.15.46:0/100665491 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f922015db10 0x7f92200b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:55.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9227000640 1 --2- 172.21.15.46:0/100665491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220142ab0 0x7f922006d4a0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:55.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9227000640 1 -- 172.21.15.46:0/100665491 >> 172.21.15.46:0/100665491 conn(0x7f9220066080 msgr2=0x7f9220069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:55.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9227000640 1 -- 172.21.15.46:0/100665491 shutdown_connections 2024-03-29T08:24:55.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9227000640 1 -- 172.21.15.46:0/100665491 wait complete. 2024-03-29T08:24:55.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9227000640 1 Processor -- start 2024-03-29T08:24:55.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9227000640 1 -- start start 2024-03-29T08:24:55.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9227000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f922006d9e0 0x7f9220159d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9227000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220142ab0 0x7f92201f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9227000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f922015db10 0x7f922013d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9227000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922015b070 con 0x7f9220142ab0 2024-03-29T08:24:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9227000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92200ad490 con 0x7f922006d9e0 2024-03-29T08:24:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9227000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92201f6220 con 0x7f922015db10 2024-03-29T08:24:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9217e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220142ab0 0x7f92201f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9217e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220142ab0 0x7f92201f5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34966/0 (socket says 172.21.15.46:34966) 2024-03-29T08:24:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9217e00640 1 -- 172.21.15.46:0/462003653 learned_addr learned my addr 172.21.15.46:0/462003653 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:24:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f921fe00640 1 --2- 172.21.15.46:0/462003653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f922006d9e0 0x7f9220159d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9225000640 1 --2- 172.21.15.46:0/462003653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f922015db10 0x7f922013d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9217e00640 1 -- 172.21.15.46:0/462003653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f922015db10 msgr2=0x7f922013d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9217e00640 1 --2- 172.21.15.46:0/462003653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f922015db10 0x7f922013d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9217e00640 1 -- 172.21.15.46:0/462003653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f922006d9e0 msgr2=0x7f9220159d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9217e00640 1 --2- 172.21.15.46:0/462003653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f922006d9e0 0x7f9220159d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.037+0000 7f9217e00640 1 -- 172.21.15.46:0/462003653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92100094d0 con 0x7f9220142ab0 2024-03-29T08:24:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.041+0000 7f9217e00640 1 --2- 172.21.15.46:0/462003653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220142ab0 0x7f92201f5ce0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f920800c3c0 tx=0x7f920800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.041+0000 7f921cc00640 1 -- 172.21.15.46:0/462003653 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92080052d0 con 0x7f9220142ab0 2024-03-29T08:24:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.041+0000 7f921cc00640 1 -- 172.21.15.46:0/462003653 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9208005430 con 0x7f9220142ab0 2024-03-29T08:24:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.041+0000 7f9227000640 1 -- 172.21.15.46:0/462003653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922015a3f0 con 0x7f9220142ab0 2024-03-29T08:24:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.041+0000 7f921cc00640 1 -- 172.21.15.46:0/462003653 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9208003840 con 0x7f9220142ab0 2024-03-29T08:24:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.041+0000 7f9227000640 1 -- 172.21.15.46:0/462003653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922015a9b0 con 0x7f9220142ab0 2024-03-29T08:24:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.041+0000 7f9227000640 1 -- 172.21.15.46:0/462003653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922015b1b0 con 0x7f9220142ab0 2024-03-29T08:24:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.041+0000 7f921cc00640 1 -- 172.21.15.46:0/462003653 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f92080039a0 con 0x7f9220142ab0 2024-03-29T08:24:55.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.041+0000 7f921cc00640 1 --2- 172.21.15.46:0/462003653 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f91fc066690 0x7f91fc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:24:55.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.041+0000 7f921cc00640 1 -- 172.21.15.46:0/462003653 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92080836b0 con 0x7f9220142ab0 2024-03-29T08:24:55.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.041+0000 7f921fe00640 1 --2- 172.21.15.46:0/462003653 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f91fc066690 0x7f91fc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:24:55.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.045+0000 7f921fe00640 1 --2- 172.21.15.46:0/462003653 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f91fc066690 0x7f91fc068b50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f92100ad000 tx=0x7f92100a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:24:55.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.045+0000 7f921cc00640 1 -- 172.21.15.46:0/462003653 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f922015b1b0 con 0x7f9220142ab0 2024-03-29T08:24:55.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.217+0000 7f9227000640 1 -- 172.21.15.46:0/462003653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f922015b1b0 con 0x7f9220142ab0 2024-03-29T08:24:55.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.217+0000 7f921cc00640 1 -- 172.21.15.46:0/462003653 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v92) v1 ==== 80+0+0 (secure 0 0 0) 0x7f922015b1b0 con 0x7f9220142ab0 2024-03-29T08:24:55.219 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:24:55.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.221+0000 7f9227000640 1 -- 172.21.15.46:0/462003653 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f91fc066690 msgr2=0x7f91fc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:55.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.221+0000 7f9227000640 1 --2- 172.21.15.46:0/462003653 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f91fc066690 0x7f91fc068b50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f92100ad000 tx=0x7f92100a6000 comp rx=0 tx=0).stop 2024-03-29T08:24:55.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.221+0000 7f9227000640 1 -- 172.21.15.46:0/462003653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220142ab0 msgr2=0x7f92201f5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:24:55.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.221+0000 7f9227000640 1 --2- 172.21.15.46:0/462003653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220142ab0 0x7f92201f5ce0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f920800c3c0 tx=0x7f920800c880 comp rx=0 tx=0).stop 2024-03-29T08:24:55.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.221+0000 7f9227000640 1 -- 172.21.15.46:0/462003653 shutdown_connections 2024-03-29T08:24:55.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.221+0000 7f9227000640 1 --2- 172.21.15.46:0/462003653 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f91fc066690 0x7f91fc068b50 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.221+0000 7f9227000640 1 --2- 172.21.15.46:0/462003653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f922015db10 0x7f922013d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.221+0000 7f9227000640 1 --2- 172.21.15.46:0/462003653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9220142ab0 0x7f92201f5ce0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.221+0000 7f9227000640 1 --2- 172.21.15.46:0/462003653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f922006d9e0 0x7f9220159d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:24:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.221+0000 7f9227000640 1 -- 172.21.15.46:0/462003653 >> 172.21.15.46:0/462003653 conn(0x7f9220066080 msgr2=0x7f9220069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:24:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.221+0000 7f9227000640 1 -- 172.21.15.46:0/462003653 shutdown_connections 2024-03-29T08:24:55.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:24:55.221+0000 7f9227000640 1 -- 172.21.15.46:0/462003653 wait complete. 2024-03-29T08:24:55.240 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-03-29T08:24:55.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:24:55.241 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T08:24:55.259 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[44662]: fuse finished with error 0 and tester_r 0 2024-03-29T08:24:55.261 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:25:01.363 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:25:01.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:01.363 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:25:01.374 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:25:01.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.565+0000 7f200a800640 1 Processor -- start 2024-03-29T08:25:01.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.565+0000 7f200a800640 1 -- start start 2024-03-29T08:25:01.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.569+0000 7f200a800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f20041593d0 0x7f200406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:01.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.569+0000 7f200a800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f200415cd90 0x7f20041597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:01.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.569+0000 7f200a800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f200406d130 con 0x7f200415d230 2024-03-29T08:25:01.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.569+0000 7f200a800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f200415b8f0 con 0x7f20041593d0 2024-03-29T08:25:01.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.569+0000 7f200a800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f200415ad50 con 0x7f200415cd90 2024-03-29T08:25:01.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.569+0000 7f2003400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f20041593d0 0x7f200406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:01.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.569+0000 7f2003400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f20041593d0 0x7f200406cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44996/0 (socket says 172.21.15.46:44996) 2024-03-29T08:25:01.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.569+0000 7f2003400640 1 -- 172.21.15.46:0/3618514788 learned_addr learned my addr 172.21.15.46:0/3618514788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:01.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.569+0000 7f2003e00640 1 --2- 172.21.15.46:0/3618514788 >> v2:172.21.15.73:3300/0 conn(0x7f200415cd90 0x7f20041597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:01.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.569+0000 7f2003400640 1 -- 172.21.15.46:0/3618514788 >> v2:172.21.15.73:3300/0 conn(0x7f200415cd90 msgr2=0x7f20041597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:01.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.569+0000 7f2003400640 1 --2- 172.21.15.46:0/3618514788 >> v2:172.21.15.73:3300/0 conn(0x7f200415cd90 0x7f20041597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:01.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.569+0000 7f2003400640 1 -- 172.21.15.46:0/3618514788 >> v1:172.21.15.73:6789/0 conn(0x7f200415d230 legacy=0x7f200415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:01.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.569+0000 7f2003400640 1 -- 172.21.15.46:0/3618514788 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200415afc0 con 0x7f20041593d0 2024-03-29T08:25:01.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f2003400640 1 --2- 172.21.15.46:0/3618514788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20041593d0 0x7f200406cbf0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f200406d360 tx=0x7f1ff401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a1df71a025bc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:01.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f2002a00640 1 -- 172.21.15.46:0/3618514788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff40a5870 con 0x7f20041593d0 2024-03-29T08:25:01.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f2002a00640 1 -- 172.21.15.46:0/3618514788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff40a59d0 con 0x7f20041593d0 2024-03-29T08:25:01.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f2002a00640 1 -- 172.21.15.46:0/3618514788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff40a5d60 con 0x7f20041593d0 2024-03-29T08:25:01.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f200a800640 1 -- 172.21.15.46:0/3618514788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20041593d0 msgr2=0x7f200406cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:01.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f200a800640 1 --2- 172.21.15.46:0/3618514788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20041593d0 0x7f200406cbf0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f200406d360 tx=0x7f1ff401c9b0 comp rx=0 tx=0).stop 2024-03-29T08:25:01.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f200a800640 1 -- 172.21.15.46:0/3618514788 shutdown_connections 2024-03-29T08:25:01.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f200a800640 1 --2- 172.21.15.46:0/3618514788 >> v2:172.21.15.73:3300/0 conn(0x7f200415cd90 0x7f20041597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:01.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f200a800640 1 --2- 172.21.15.46:0/3618514788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20041593d0 0x7f200406cbf0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:01.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f200a800640 1 -- 172.21.15.46:0/3618514788 >> 172.21.15.46:0/3618514788 conn(0x7f2004066110 msgr2=0x7f2004069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:01.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f200a800640 1 -- 172.21.15.46:0/3618514788 shutdown_connections 2024-03-29T08:25:01.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f200a800640 1 -- 172.21.15.46:0/3618514788 wait complete. 2024-03-29T08:25:01.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f200a800640 1 Processor -- start 2024-03-29T08:25:01.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f200a800640 1 -- start start 2024-03-29T08:25:01.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f200a800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f20041593d0 0x7f2004147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:01.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f200a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f200415cd90 0x7f20040a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:01.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f200a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f200415d230 0x7f20040a79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:01.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f200a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2004161e10 con 0x7f200415d230 2024-03-29T08:25:01.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f200a800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20041620f0 con 0x7f200415cd90 2024-03-29T08:25:01.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.573+0000 7f200a800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20041623d0 con 0x7f20041593d0 2024-03-29T08:25:01.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f2003e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f200415d230 0x7f20040a79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:01.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f2003e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f200415d230 0x7f20040a79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45010/0 (socket says 172.21.15.46:45010) 2024-03-29T08:25:01.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f2003e00640 1 -- 172.21.15.46:0/449661120 learned_addr learned my addr 172.21.15.46:0/449661120 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:01.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f2003400640 1 --2- 172.21.15.46:0/449661120 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f20041593d0 0x7f2004147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:01.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f1ffaa00640 1 --2- 172.21.15.46:0/449661120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f200415cd90 0x7f20040a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:01.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f2003e00640 1 -- 172.21.15.46:0/449661120 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f20041593d0 msgr2=0x7f2004147f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:01.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f2003e00640 1 --2- 172.21.15.46:0/449661120 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f20041593d0 0x7f2004147f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:01.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f2003e00640 1 -- 172.21.15.46:0/449661120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f200415cd90 msgr2=0x7f20040a74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:01.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f2003e00640 1 --2- 172.21.15.46:0/449661120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f200415cd90 0x7f20040a74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:01.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f2003e00640 1 -- 172.21.15.46:0/449661120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff4009b30 con 0x7f200415d230 2024-03-29T08:25:01.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f2003e00640 1 --2- 172.21.15.46:0/449661120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f200415d230 0x7f20040a79e0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc006e90 tx=0x7f1ffc00d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:01.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f1ffbe00640 1 -- 172.21.15.46:0/449661120 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc0125f0 con 0x7f200415d230 2024-03-29T08:25:01.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f1ffbe00640 1 -- 172.21.15.46:0/449661120 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ffc012750 con 0x7f200415d230 2024-03-29T08:25:01.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f200a800640 1 -- 172.21.15.46:0/449661120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20041626b0 con 0x7f200415d230 2024-03-29T08:25:01.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f1ffbe00640 1 -- 172.21.15.46:0/449661120 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ffc012b00 con 0x7f200415d230 2024-03-29T08:25:01.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f200a800640 1 -- 172.21.15.46:0/449661120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20041484c0 con 0x7f200415d230 2024-03-29T08:25:01.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f200a800640 1 -- 172.21.15.46:0/449661120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2004066c90 con 0x7f200415d230 2024-03-29T08:25:01.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f1ffbe00640 1 -- 172.21.15.46:0/449661120 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1ffc012c60 con 0x7f200415d230 2024-03-29T08:25:01.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.577+0000 7f1ffbe00640 1 --2- 172.21.15.46:0/449661120 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1fe0066690 0x7f1fe0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:01.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.581+0000 7f1ffbe00640 1 -- 172.21.15.46:0/449661120 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ffc084510 con 0x7f200415d230 2024-03-29T08:25:01.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.581+0000 7f2003400640 1 --2- 172.21.15.46:0/449661120 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1fe0066690 0x7f1fe0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:01.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.581+0000 7f2003400640 1 --2- 172.21.15.46:0/449661120 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1fe0066690 0x7f1fe0068b50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40ae000 tx=0x7f1ff40a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:01.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.585+0000 7f1ffbe00640 1 -- 172.21.15.46:0/449661120 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1ffc051870 con 0x7f200415d230 2024-03-29T08:25:01.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.781+0000 7f200a800640 1 -- 172.21.15.46:0/449661120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2004162b60 con 0x7f200415d230 2024-03-29T08:25:01.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.781+0000 7f1ffbe00640 1 -- 172.21.15.46:0/449661120 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4736 (secure 0 0 0) 0x7f1ffc054e90 con 0x7f200415d230 2024-03-29T08:25:01.783 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:25:01.783 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7386,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/2051429508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2051429508},{"type":"v1","addr":"172.21.15.46:6835","nonce":2051429508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:24:43.476334+0000","modified":"2024-03-29T08:24:46.492649+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7395},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7395":{"gid":7395,"name":"b","rank":0,"incarnation":53,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/4166845858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4166845858},{"type":"v1","addr":"172.21.15.73:6836","nonce":4166845858}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-29T08:25:01.784 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 57 2024-03-29T08:25:01.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.785+0000 7f200a800640 1 -- 172.21.15.46:0/449661120 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1fe0066690 msgr2=0x7f1fe0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:01.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.785+0000 7f200a800640 1 --2- 172.21.15.46:0/449661120 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1fe0066690 0x7f1fe0068b50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40ae000 tx=0x7f1ff40a7070 comp rx=0 tx=0).stop 2024-03-29T08:25:01.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.785+0000 7f200a800640 1 -- 172.21.15.46:0/449661120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f200415d230 msgr2=0x7f20040a79e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:01.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.785+0000 7f200a800640 1 --2- 172.21.15.46:0/449661120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f200415d230 0x7f20040a79e0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc006e90 tx=0x7f1ffc00d9e0 comp rx=0 tx=0).stop 2024-03-29T08:25:01.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.785+0000 7f200a800640 1 -- 172.21.15.46:0/449661120 shutdown_connections 2024-03-29T08:25:01.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.785+0000 7f200a800640 1 --2- 172.21.15.46:0/449661120 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1fe0066690 0x7f1fe0068b50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:01.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.785+0000 7f200a800640 1 --2- 172.21.15.46:0/449661120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f200415cd90 0x7f20040a74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:01.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.785+0000 7f200a800640 1 --2- 172.21.15.46:0/449661120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f200415d230 0x7f20040a79e0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:01.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.785+0000 7f200a800640 1 --2- 172.21.15.46:0/449661120 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f20041593d0 0x7f2004147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:01.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.785+0000 7f200a800640 1 -- 172.21.15.46:0/449661120 >> 172.21.15.46:0/449661120 conn(0x7f2004066110 msgr2=0x7f20040b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:01.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.785+0000 7f200a800640 1 -- 172.21.15.46:0/449661120 shutdown_connections 2024-03-29T08:25:01.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:01.785+0000 7f200a800640 1 -- 172.21.15.46:0/449661120 wait complete. 2024-03-29T08:25:01.805 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-03-29T08:25:01.977 INFO:teuthology.orchestra.run.smithi073.stdout:{"message":"","return_code":0} 2024-03-29T08:25:01.978 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-29T08:25:01.978 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T08:25:02.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.121+0000 7f07bb800640 1 Processor -- start 2024-03-29T08:25:02.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.121+0000 7f07bb800640 1 -- start start 2024-03-29T08:25:02.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.121+0000 7f07bb800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07b415aab0 0x7f07b40ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.121+0000 7f07bb800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f07b40adb60 0x7f07b40cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.121+0000 7f07bb800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07b40ae120 con 0x7f07b40cb120 2024-03-29T08:25:02.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.121+0000 7f07bb800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b40d1500 con 0x7f07b415aab0 2024-03-29T08:25:02.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.121+0000 7f07bb800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b40b01b0 con 0x7f07b40adb60 2024-03-29T08:25:02.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.121+0000 7f07b8e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f07b415aab0 0x7f07b40ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.121+0000 7f07b3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f07b40cb120 0x7f07b40ad450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58384/0 (socket says 172.21.15.46:58384) 2024-03-29T08:25:02.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.121+0000 7f07b3e00640 1 -- 172.21.15.46:0/2955790177 learned_addr learned my addr 172.21.15.46:0/2955790177 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:25:02.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.121+0000 7f07b9800640 1 --2- 172.21.15.46:0/2955790177 >> v2:172.21.15.73:3300/0 conn(0x7f07b40adb60 0x7f07b40cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.121+0000 7f07b8e00640 1 -- 172.21.15.46:0/2955790177 >> v2:172.21.15.73:3300/0 conn(0x7f07b40adb60 msgr2=0x7f07b40cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:02.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.121+0000 7f07b8e00640 1 --2- 172.21.15.46:0/2955790177 >> v2:172.21.15.73:3300/0 conn(0x7f07b40adb60 0x7f07b40cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.121+0000 7f07b8e00640 1 -- 172.21.15.46:0/2955790177 >> v1:172.21.15.46:6789/0 conn(0x7f07b40cb120 legacy=0x7f07b40ad450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.121+0000 7f07b8e00640 1 -- 172.21.15.46:0/2955790177 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b40ae3e0 con 0x7f07b415aab0 2024-03-29T08:25:02.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.121+0000 7f07b3400640 1 -- 172.21.15.46:0/2955790177 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192814840 0 0) 0x7f07b40ae120 con 0x7f07b40cb120 2024-03-29T08:25:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.125+0000 7f07b8e00640 1 --2- 172.21.15.46:0/2955790177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b415aab0 0x7f07b40ceb90 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f07b408a730 tx=0x7f07a801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28208af24082ac69 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.125+0000 7f07b3400640 1 -- 172.21.15.46:0/2955790177 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a80a8050 con 0x7f07b415aab0 2024-03-29T08:25:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.125+0000 7f07b3400640 1 -- 172.21.15.46:0/2955790177 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a80ac5f0 con 0x7f07b415aab0 2024-03-29T08:25:02.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.125+0000 7f07b3400640 1 -- 172.21.15.46:0/2955790177 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a80ac980 con 0x7f07b415aab0 2024-03-29T08:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.125+0000 7f07bb800640 1 -- 172.21.15.46:0/2955790177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b415aab0 msgr2=0x7f07b40ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.125+0000 7f07bb800640 1 --2- 172.21.15.46:0/2955790177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b415aab0 0x7f07b40ceb90 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f07b408a730 tx=0x7f07a801c480 comp rx=0 tx=0).stop 2024-03-29T08:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.125+0000 7f07bb800640 1 -- 172.21.15.46:0/2955790177 shutdown_connections 2024-03-29T08:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.125+0000 7f07bb800640 1 --2- 172.21.15.46:0/2955790177 >> v2:172.21.15.73:3300/0 conn(0x7f07b40adb60 0x7f07b40cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.125+0000 7f07bb800640 1 --2- 172.21.15.46:0/2955790177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b415aab0 0x7f07b40ceb90 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.125+0000 7f07bb800640 1 -- 172.21.15.46:0/2955790177 >> 172.21.15.46:0/2955790177 conn(0x7f07b4099a10 msgr2=0x7f07b4089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.125+0000 7f07bb800640 1 -- 172.21.15.46:0/2955790177 shutdown_connections 2024-03-29T08:25:02.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.125+0000 7f07bb800640 1 -- 172.21.15.46:0/2955790177 wait complete. 2024-03-29T08:25:02.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07bb800640 1 Processor -- start 2024-03-29T08:25:02.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07bb800640 1 -- start start 2024-03-29T08:25:02.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07bb800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f07b40adb60 0x7f07b40cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07bb800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07b40cb120 0x7f07b41569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07bb800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b415aab0 0x7f07b4156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07bb800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b41f6060 con 0x7f07b415aab0 2024-03-29T08:25:02.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07bb800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b40cb6f0 con 0x7f07b40cb120 2024-03-29T08:25:02.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07bb800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b41f5d80 con 0x7f07b40adb60 2024-03-29T08:25:02.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07b9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b415aab0 0x7f07b4156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07b9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b415aab0 0x7f07b4156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45036/0 (socket says 172.21.15.46:45036) 2024-03-29T08:25:02.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07b9800640 1 -- 172.21.15.46:0/4269534038 learned_addr learned my addr 172.21.15.46:0/4269534038 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07b3e00640 1 --2- 172.21.15.46:0/4269534038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07b40cb120 0x7f07b41569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07b9800640 1 -- 172.21.15.46:0/4269534038 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f07b40adb60 msgr2=0x7f07b40cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07b8e00640 1 --2- 172.21.15.46:0/4269534038 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f07b40adb60 0x7f07b40cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07b9800640 1 --2- 172.21.15.46:0/4269534038 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f07b40adb60 0x7f07b40cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07b9800640 1 -- 172.21.15.46:0/4269534038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07b40cb120 msgr2=0x7f07b41569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07b9800640 1 --2- 172.21.15.46:0/4269534038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07b40cb120 0x7f07b41569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07b9800640 1 -- 172.21.15.46:0/4269534038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a8009420 con 0x7f07b415aab0 2024-03-29T08:25:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07b3e00640 1 --2- 172.21.15.46:0/4269534038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07b40cb120 0x7f07b41569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:25:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07b8e00640 1 --2- 172.21.15.46:0/4269534038 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f07b40adb60 0x7f07b40cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:25:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07b9800640 1 --2- 172.21.15.46:0/4269534038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b415aab0 0x7f07b4156f10 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00efc0 tx=0x7f07ac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07b0c00640 1 -- 172.21.15.46:0/4269534038 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac014480 con 0x7f07b415aab0 2024-03-29T08:25:02.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07b0c00640 1 -- 172.21.15.46:0/4269534038 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ac00f040 con 0x7f07b415aab0 2024-03-29T08:25:02.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07b0c00640 1 -- 172.21.15.46:0/4269534038 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07ac014820 con 0x7f07b415aab0 2024-03-29T08:25:02.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07bb800640 1 -- 172.21.15.46:0/4269534038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b40cb990 con 0x7f07b415aab0 2024-03-29T08:25:02.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07bb800640 1 -- 172.21.15.46:0/4269534038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b40cc6d0 con 0x7f07b415aab0 2024-03-29T08:25:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.129+0000 7f07bb800640 1 -- 172.21.15.46:0/4269534038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b40cb550 con 0x7f07b415aab0 2024-03-29T08:25:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.133+0000 7f07b0c00640 1 -- 172.21.15.46:0/4269534038 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f07ac014980 con 0x7f07b415aab0 2024-03-29T08:25:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.133+0000 7f07b0c00640 1 --2- 172.21.15.46:0/4269534038 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0794066690 0x7f0794068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.133+0000 7f07b0c00640 1 -- 172.21.15.46:0/4269534038 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07ac085090 con 0x7f07b415aab0 2024-03-29T08:25:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.133+0000 7f07b8e00640 1 --2- 172.21.15.46:0/4269534038 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0794066690 0x7f0794068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.133+0000 7f07b8e00640 1 --2- 172.21.15.46:0/4269534038 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0794066690 0x7f0794068b50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f07a80ae000 tx=0x7f07a80a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.137+0000 7f07b0c00640 1 -- 172.21.15.46:0/4269534038 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f07b40cb550 con 0x7f07b415aab0 2024-03-29T08:25:02.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.333+0000 7f07bb800640 1 -- 172.21.15.46:0/4269534038 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f07b40cb550 con 0x7f07b415aab0 2024-03-29T08:25:02.332 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-03-29T08:25:02.333+0000 7fc16ae00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:25:02.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.341+0000 7f07b0c00640 1 -- 172.21.15.46:0/4269534038 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7f07b40cb550 con 0x7f07b415aab0 2024-03-29T08:25:02.340 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T08:25:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.341+0000 7f07bb800640 1 -- 172.21.15.46:0/4269534038 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0794066690 msgr2=0x7f0794068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.341+0000 7f07bb800640 1 --2- 172.21.15.46:0/4269534038 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0794066690 0x7f0794068b50 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f07a80ae000 tx=0x7f07a80a7000 comp rx=0 tx=0).stop 2024-03-29T08:25:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.341+0000 7f07bb800640 1 -- 172.21.15.46:0/4269534038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b415aab0 msgr2=0x7f07b4156f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.341+0000 7f07bb800640 1 --2- 172.21.15.46:0/4269534038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b415aab0 0x7f07b4156f10 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00efc0 tx=0x7f07ac00c490 comp rx=0 tx=0).stop 2024-03-29T08:25:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.341+0000 7f07bb800640 1 -- 172.21.15.46:0/4269534038 shutdown_connections 2024-03-29T08:25:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.341+0000 7f07bb800640 1 --2- 172.21.15.46:0/4269534038 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0794066690 0x7f0794068b50 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.341+0000 7f07bb800640 1 --2- 172.21.15.46:0/4269534038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07b40cb120 0x7f07b41569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.341+0000 7f07bb800640 1 --2- 172.21.15.46:0/4269534038 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07b415aab0 0x7f07b4156f10 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.341+0000 7f07bb800640 1 --2- 172.21.15.46:0/4269534038 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f07b40adb60 0x7f07b40cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.341+0000 7f07bb800640 1 -- 172.21.15.46:0/4269534038 >> 172.21.15.46:0/4269534038 conn(0x7f07b4099a10 msgr2=0x7f07b406e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.341+0000 7f07bb800640 1 -- 172.21.15.46:0/4269534038 shutdown_connections 2024-03-29T08:25:02.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.341+0000 7f07bb800640 1 -- 172.21.15.46:0/4269534038 wait complete. 2024-03-29T08:25:02.348 INFO:tasks.ceph.mds.b.smithi073.stderr: -3780> 2024-03-29T08:24:49.067+0000 7f7366400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:25:02.361 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-03-29T08:25:02.395 INFO:tasks.ceph.mds.b.smithi073.stdout:starting mds.b at 2024-03-29T08:25:02.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.405+0000 7fd54682b080 1 Processor -- start 2024-03-29T08:25:02.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.405+0000 7fd54682b080 1 -- start start 2024-03-29T08:25:02.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.405+0000 7fd54682b080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5635f4946090 0x5635f4a21c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.405+0000 7fd54682b080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5635f4946470 0x5635f4977b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635f4a018d0 0x5635f4ab36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f488e600 con 0x5635f4a018d0 2024-03-29T08:25:02.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f4885690 con 0x5635f4946470 2024-03-29T08:25:02.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635f498de20 con 0x5635f4946090 2024-03-29T08:25:02.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd543400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635f4a018d0 0x5635f4ab36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd542a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5635f4946090 0x5635f4a21c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd543400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635f4a018d0 0x5635f4ab36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45042/0 (socket says 172.21.15.46:45042) 2024-03-29T08:25:02.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd543400640 1 -- 172.21.15.46:0/4290026398 learned_addr learned my addr 172.21.15.46:0/4290026398 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:02.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd542000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5635f4946470 0x5635f4977b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd542a00640 1 -- 172.21.15.46:0/4290026398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5635f4946470 msgr2=0x5635f4977b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:02.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd542a00640 1 --2- 172.21.15.46:0/4290026398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5635f4946470 0x5635f4977b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd542a00640 1 -- 172.21.15.46:0/4290026398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635f4a018d0 msgr2=0x5635f4ab36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:02.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd542a00640 1 --2- 172.21.15.46:0/4290026398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635f4a018d0 0x5635f4ab36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd542a00640 1 -- 172.21.15.46:0/4290026398 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd534005df0 con 0x5635f4946090 2024-03-29T08:25:02.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd542000640 1 --2- 172.21.15.46:0/4290026398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5635f4946470 0x5635f4977b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:25:02.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd543400640 1 --2- 172.21.15.46:0/4290026398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635f4a018d0 0x5635f4ab36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:25:02.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd542a00640 1 --2- 172.21.15.46:0/4290026398 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5635f4946090 0x5635f4a21c20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5635f494bb50 tx=0x7fd534003030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=397465942ba118a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd540c00640 1 -- 172.21.15.46:0/4290026398 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5340a5bb0 con 0x5635f4946090 2024-03-29T08:25:02.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd540c00640 1 -- 172.21.15.46:0/4290026398 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5340a5d30 con 0x5635f4946090 2024-03-29T08:25:02.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd540c00640 1 -- 172.21.15.46:0/4290026398 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5340a5ea0 con 0x5635f4946090 2024-03-29T08:25:02.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 -- 172.21.15.46:0/4290026398 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5635f4946090 msgr2=0x5635f4a21c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 --2- 172.21.15.46:0/4290026398 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5635f4946090 0x5635f4a21c20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5635f494bb50 tx=0x7fd534003030 comp rx=0 tx=0).stop 2024-03-29T08:25:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 -- 172.21.15.46:0/4290026398 shutdown_connections 2024-03-29T08:25:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 --2- 172.21.15.46:0/4290026398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5635f4946470 0x5635f4977b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 --2- 172.21.15.46:0/4290026398 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635f4a018d0 0x5635f4ab36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 --2- 172.21.15.46:0/4290026398 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5635f4946090 0x5635f4a21c20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 -- 172.21.15.46:0/4290026398 >> 172.21.15.46:0/4290026398 conn(0x5635f4947cf0 msgr2=0x5635f498e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 -- 172.21.15.46:0/4290026398 shutdown_connections 2024-03-29T08:25:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 -- 172.21.15.46:0/4290026398 wait complete. 2024-03-29T08:25:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 Processor -- start 2024-03-29T08:25:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 -- start start 2024-03-29T08:25:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5635f4946090 0x5635f4ab6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5635f4946470 0x5635f4a05bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635f4a018d0 0x5635f4a052a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f4ab7560 con 0x5635f4a018d0 2024-03-29T08:25:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635f4ab7800 con 0x5635f4946470 2024-03-29T08:25:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.409+0000 7fd54682b080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635f4ab7aa0 con 0x5635f4946090 2024-03-29T08:25:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd543400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635f4a018d0 0x5635f4a052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd542000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5635f4946470 0x5635f4a05bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd543400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635f4a018d0 0x5635f4a052a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45056/0 (socket says 172.21.15.46:45056) 2024-03-29T08:25:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd543400640 1 -- 172.21.15.46:0/2951349718 learned_addr learned my addr 172.21.15.46:0/2951349718 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd542a00640 1 --2- 172.21.15.46:0/2951349718 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5635f4946090 0x5635f4ab6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd543400640 1 -- 172.21.15.46:0/2951349718 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5635f4946090 msgr2=0x5635f4ab6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd543400640 1 --2- 172.21.15.46:0/2951349718 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5635f4946090 0x5635f4ab6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd543400640 1 -- 172.21.15.46:0/2951349718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5635f4946470 msgr2=0x5635f4a05bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd543400640 1 --2- 172.21.15.46:0/2951349718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5635f4946470 0x5635f4a05bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd543400640 1 -- 172.21.15.46:0/2951349718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd530009420 con 0x5635f4a018d0 2024-03-29T08:25:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd542000640 1 --2- 172.21.15.46:0/2951349718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5635f4946470 0x5635f4a05bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:25:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd543400640 1 --2- 172.21.15.46:0/2951349718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635f4a018d0 0x5635f4a052a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0108b0 tx=0x7fd53c010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd53b400640 1 -- 172.21.15.46:0/2951349718 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c00e9b0 con 0x5635f4a018d0 2024-03-29T08:25:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd53b400640 1 -- 172.21.15.46:0/2951349718 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c00eb30 con 0x5635f4a018d0 2024-03-29T08:25:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd54682b080 1 -- 172.21.15.46:0/2951349718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53c00b3e0 con 0x5635f4a018d0 2024-03-29T08:25:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd53b400640 1 -- 172.21.15.46:0/2951349718 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c00eee0 con 0x5635f4a018d0 2024-03-29T08:25:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd54682b080 1 -- 172.21.15.46:0/2951349718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635f4890970 con 0x5635f4a018d0 2024-03-29T08:25:02.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd53b400640 1 -- 172.21.15.46:0/2951349718 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd53c013440 con 0x5635f4a018d0 2024-03-29T08:25:02.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd53b400640 1 --2- 172.21.15.46:0/2951349718 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd524065070 0x7fd524067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.413+0000 7fd53b400640 1 -- 172.21.15.46:0/2951349718 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd53c083680 con 0x5635f4a018d0 2024-03-29T08:25:02.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.417+0000 7fd54682b080 1 --2- 172.21.15.46:0/2951349718 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5635f4a001e0 0x5635f4a005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.417+0000 7fd54682b080 1 -- 172.21.15.46:0/2951349718 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e93) v8 -- 0x5635f4a00b30 con 0x5635f4a001e0 2024-03-29T08:25:02.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.417+0000 7fd542a00640 1 --2- 172.21.15.46:0/2951349718 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd524065070 0x7fd524067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.417+0000 7fd542000640 1 --2- 172.21.15.46:0/2951349718 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5635f4a001e0 0x5635f4a005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.417+0000 7fd542000640 1 --2- 172.21.15.46:0/2951349718 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5635f4a001e0 0x5635f4a005f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.417+0000 7fd542a00640 1 --2- 172.21.15.46:0/2951349718 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd524065070 0x7fd524067530 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd5340ab000 tx=0x7fd5340c3040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.421+0000 7fd542000640 1 -- 172.21.15.46:0/2951349718 <== osd.6 v2:172.21.15.73:6824/27691 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v93'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd530012040 con 0x5635f4a001e0 2024-03-29T08:25:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.421+0000 7fd54682b080 1 -- 172.21.15.46:0/2951349718 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5635f4a001e0 msgr2=0x5635f4a005f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.421+0000 7fd54682b080 1 --2- 172.21.15.46:0/2951349718 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5635f4a001e0 0x5635f4a005f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.421+0000 7fd54682b080 1 -- 172.21.15.46:0/2951349718 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd524065070 msgr2=0x7fd524067530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.421+0000 7fd54682b080 1 --2- 172.21.15.46:0/2951349718 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd524065070 0x7fd524067530 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd5340ab000 tx=0x7fd5340c3040 comp rx=0 tx=0).stop 2024-03-29T08:25:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.421+0000 7fd54682b080 1 -- 172.21.15.46:0/2951349718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635f4a018d0 msgr2=0x5635f4a052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.421+0000 7fd54682b080 1 --2- 172.21.15.46:0/2951349718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5635f4a018d0 0x5635f4a052a0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fd53c0108b0 tx=0x7fd53c010d70 comp rx=0 tx=0).stop 2024-03-29T08:25:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.421+0000 7fd543400640 1 -- 172.21.15.46:0/2951349718 reap_dead start 2024-03-29T08:25:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.421+0000 7fd54682b080 1 -- 172.21.15.46:0/2951349718 shutdown_connections 2024-03-29T08:25:02.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.421+0000 7fd54682b080 1 -- 172.21.15.46:0/2951349718 >> 172.21.15.46:0/2951349718 conn(0x5635f4947cf0 msgr2=0x5635f498e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:02.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.421+0000 7fd54682b080 1 -- 172.21.15.46:0/2951349718 shutdown_connections 2024-03-29T08:25:02.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.421+0000 7fd54682b080 1 -- 172.21.15.46:0/2951349718 wait complete. 2024-03-29T08:25:02.426 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-03-29T08:25:02.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb90274400 1 -- start start 2024-03-29T08:25:02.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb90274400 1 Processor -- start 2024-03-29T08:25:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb90274400 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec15ee9800 0x55ec15f74580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb90274400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec15ee9400 0x55ec15f74b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb90274400 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec15ee9000 0x55ec15f75080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb90274400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec15fb6a00 con 0x55ec15ee9400 2024-03-29T08:25:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb90274400 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec15fb6780 con 0x55ec15ee9000 2024-03-29T08:25:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb90274400 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec15fb6640 con 0x55ec15ee9800 2024-03-29T08:25:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb8da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec15ee9400 0x55ec15f74b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb8da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec15ee9400 0x55ec15f74b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45072/0 (socket says 172.21.15.46:45072) 2024-03-29T08:25:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb8da00640 1 -- 172.21.15.46:0/3163523946 learned_addr learned my addr 172.21.15.46:0/3163523946 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb8da00640 1 -- 172.21.15.46:0/3163523946 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec15ee9800 msgr2=0x55ec15f74580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb8da00640 1 --2- 172.21.15.46:0/3163523946 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec15ee9800 0x55ec15f74580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb8da00640 1 -- 172.21.15.46:0/3163523946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec15ee9000 msgr2=0x55ec15f75080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb8da00640 1 --2- 172.21.15.46:0/3163523946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec15ee9000 0x55ec15f75080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb8da00640 1 -- 172.21.15.46:0/3163523946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec15f1e9c0 con 0x55ec15ee9400 2024-03-29T08:25:02.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb8da00640 1 --2- 172.21.15.46:0/3163523946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec15ee9400 0x55ec15f74b00 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x55ec16d5ec90 tx=0x55ec16df6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b241e0bc3d0e3007 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb8b200640 1 -- 172.21.15.46:0/3163523946 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ec15f3d600 con 0x55ec15ee9400 2024-03-29T08:25:02.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb8b200640 1 -- 172.21.15.46:0/3163523946 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec15f46480 con 0x55ec15ee9400 2024-03-29T08:25:02.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb8b200640 1 -- 172.21.15.46:0/3163523946 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ec15f3d1e0 con 0x55ec15ee9400 2024-03-29T08:25:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb90274400 1 -- 172.21.15.46:0/3163523946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec15f1eb60 con 0x55ec15ee9400 2024-03-29T08:25:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb90274400 1 -- 172.21.15.46:0/3163523946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55ec15f1e340 con 0x55ec15ee9400 2024-03-29T08:25:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb90274400 4 waiting for MDS map... 2024-03-29T08:25:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb8b200640 1 -- 172.21.15.46:0/3163523946 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x55ec15f28c40 con 0x55ec15ee9400 2024-03-29T08:25:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb8b200640 1 -- 172.21.15.46:0/3163523946 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 58) v1 ==== 2573+0+0 (secure 0 0 0) 0x55ec15f36f00 con 0x55ec15ee9400 2024-03-29T08:25:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb90274400 4 Got MDS map 58 2024-03-29T08:25:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.513+0000 7ffb90274400 10 main: JournalTool::main 2024-03-29T08:25:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 4 main: JournalTool: connecting to RADOS... 2024-03-29T08:25:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 1 Processor -- start 2024-03-29T08:25:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 1 -- start start 2024-03-29T08:25:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec16df8c00 0x55ec15f75600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec16df9000 0x55ec15f75b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec16df9400 0x55ec15f76100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec15f2fe00 con 0x55ec16df9400 2024-03-29T08:25:02.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec15fb6780 con 0x55ec16df8c00 2024-03-29T08:25:02.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec15fb6500 con 0x55ec16df9000 2024-03-29T08:25:02.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb8da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec16df9400 0x55ec15f76100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb8da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec16df9400 0x55ec15f76100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45084/0 (socket says 172.21.15.46:45084) 2024-03-29T08:25:02.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb8da00640 1 -- 172.21.15.46:0/1002040799 learned_addr learned my addr 172.21.15.46:0/1002040799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:02.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb8e400640 1 --2- 172.21.15.46:0/1002040799 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec16df9000 0x55ec15f75b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb8ee00640 1 --2- 172.21.15.46:0/1002040799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec16df8c00 0x55ec15f75600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb8da00640 1 -- 172.21.15.46:0/1002040799 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec16df9000 msgr2=0x55ec15f75b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:02.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb8da00640 1 --2- 172.21.15.46:0/1002040799 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec16df9000 0x55ec15f75b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb8da00640 1 -- 172.21.15.46:0/1002040799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec16df8c00 msgr2=0x55ec15f75600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:02.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb8da00640 1 --2- 172.21.15.46:0/1002040799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec16df8c00 0x55ec15f75600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb8da00640 1 -- 172.21.15.46:0/1002040799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec15f1e000 con 0x55ec16df9400 2024-03-29T08:25:02.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb8da00640 1 --2- 172.21.15.46:0/1002040799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec16df9400 0x55ec15f76100 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x55ec16e5a0c0 tx=0x55ec15fc32c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e797e42fabd3677 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb88a00640 1 -- 172.21.15.46:0/1002040799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ec15f3cf20 con 0x55ec16df9400 2024-03-29T08:25:02.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb88a00640 1 -- 172.21.15.46:0/1002040799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec16cc8d80 con 0x55ec16df9400 2024-03-29T08:25:02.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb88a00640 1 -- 172.21.15.46:0/1002040799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ec15f3d600 con 0x55ec16df9400 2024-03-29T08:25:02.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 1 -- 172.21.15.46:0/1002040799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec16df9400 msgr2=0x55ec15f76100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 1 --2- 172.21.15.46:0/1002040799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec16df9400 0x55ec15f76100 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x55ec16e5a0c0 tx=0x55ec15fc32c0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 1 -- 172.21.15.46:0/1002040799 shutdown_connections 2024-03-29T08:25:02.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 1 --2- 172.21.15.46:0/1002040799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec16df9400 0x55ec15f76100 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 1 --2- 172.21.15.46:0/1002040799 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec16df9000 0x55ec15f75b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 1 --2- 172.21.15.46:0/1002040799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec16df8c00 0x55ec15f75600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 1 -- 172.21.15.46:0/1002040799 >> 172.21.15.46:0/1002040799 conn(0x55ec15ee8c00 msgr2=0x55ec15f17800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:02.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 1 -- 172.21.15.46:0/1002040799 shutdown_connections 2024-03-29T08:25:02.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.517+0000 7ffb90274400 1 -- 172.21.15.46:0/1002040799 wait complete. 2024-03-29T08:25:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb90274400 1 Processor -- start 2024-03-29T08:25:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb90274400 1 -- start start 2024-03-29T08:25:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb90274400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec16df8c00 0x55ec15f77180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb90274400 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec16df9000 0x55ec15f76c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb90274400 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec16df9400 0x55ec15f76680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb90274400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec15fb63c0 con 0x55ec16df8c00 2024-03-29T08:25:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb90274400 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec15fb6280 con 0x55ec16df9000 2024-03-29T08:25:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb90274400 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec15fb6140 con 0x55ec16df9400 2024-03-29T08:25:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb8ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec16df8c00 0x55ec15f77180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb8ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec16df8c00 0x55ec15f77180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45094/0 (socket says 172.21.15.46:45094) 2024-03-29T08:25:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb8ee00640 1 -- 172.21.15.46:0/579289909 learned_addr learned my addr 172.21.15.46:0/579289909 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb8e400640 1 --2- 172.21.15.46:0/579289909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec16df9000 0x55ec15f76c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb8da00640 1 --2- 172.21.15.46:0/579289909 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec16df9400 0x55ec15f76680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb8ee00640 1 -- 172.21.15.46:0/579289909 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec16df9400 msgr2=0x55ec15f76680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb8ee00640 1 --2- 172.21.15.46:0/579289909 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec16df9400 0x55ec15f76680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb8ee00640 1 -- 172.21.15.46:0/579289909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec16df9000 msgr2=0x55ec15f76c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb8ee00640 1 --2- 172.21.15.46:0/579289909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec16df9000 0x55ec15f76c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb8ee00640 1 -- 172.21.15.46:0/579289909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec15f1e9c0 con 0x55ec16df8c00 2024-03-29T08:25:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb8e400640 1 --2- 172.21.15.46:0/579289909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec16df9000 0x55ec15f76c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:25:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb8ee00640 1 --2- 172.21.15.46:0/579289909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec16df8c00 0x55ec15f77180 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x55ec16e5ae10 tx=0x55ec16e5e360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb86c00640 1 -- 172.21.15.46:0/579289909 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ec15f3cf20 con 0x55ec16df8c00 2024-03-29T08:25:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb86c00640 1 -- 172.21.15.46:0/579289909 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec15f46180 con 0x55ec16df8c00 2024-03-29T08:25:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb86c00640 1 -- 172.21.15.46:0/579289909 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ec15f3d600 con 0x55ec16df8c00 2024-03-29T08:25:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb90274400 1 -- 172.21.15.46:0/579289909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ec15f1eb60 con 0x55ec16df8c00 2024-03-29T08:25:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb90274400 1 -- 172.21.15.46:0/579289909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec15f1eea0 con 0x55ec16df8c00 2024-03-29T08:25:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb90274400 4 main: JournalTool: resolving pool 12 2024-03-29T08:25:02.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.521+0000 7ffb86c00640 1 -- 172.21.15.46:0/579289909 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x55ec15fb4840 con 0x55ec16df8c00 2024-03-29T08:25:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb86c00640 1 --2- 172.21.15.46:0/579289909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x55ec161cec00 0x55ec15f77700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb86c00640 1 -- 172.21.15.46:0/579289909 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x55ec15f28a80 con 0x55ec16df8c00 2024-03-29T08:25:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb90274400 4 main: JournalTool: creating IoCtx.. 2024-03-29T08:25:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb90274400 4 main: Executing for rank 0 2024-03-29T08:25:02.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb8e400640 1 --2- 172.21.15.46:0/579289909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x55ec161cec00 0x55ec15f77700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb8e400640 1 --2- 172.21.15.46:0/579289909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x55ec161cec00 0x55ec15f77700 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55ec15f218c0 tx=0x55ec16df7f20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb90274400 1 -- start start 2024-03-29T08:25:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb90274400 1 Processor -- start 2024-03-29T08:25:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb90274400 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec16df9c00 0x55ec15f76100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb90274400 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec161cf000 0x55ec15f75b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb90274400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec161cf400 0x55ec15f75600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb90274400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec15fb6280 con 0x55ec161cf400 2024-03-29T08:25:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb90274400 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec15fb6a00 con 0x55ec16df9c00 2024-03-29T08:25:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb90274400 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ec15fb6500 con 0x55ec161cf000 2024-03-29T08:25:02.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb8da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec161cf400 0x55ec15f75600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb8ee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec16df9c00 0x55ec15f76100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb8ee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec16df9c00 0x55ec15f76100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56310/0 (socket says 172.21.15.46:56310) 2024-03-29T08:25:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb8ee00640 1 -- 172.21.15.46:0/1242686852 learned_addr learned my addr 172.21.15.46:0/1242686852 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb8da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec161cf400 0x55ec15f75600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45098/0 (socket says 172.21.15.46:45098) 2024-03-29T08:25:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb8e400640 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec161cf000 0x55ec15f75b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb8ee00640 1 -- 172.21.15.46:0/1242686852 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec161cf000 msgr2=0x55ec15f75b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb8ee00640 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec161cf000 0x55ec15f75b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb8ee00640 1 -- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec161cf400 msgr2=0x55ec15f75600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb8ee00640 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec161cf400 0x55ec15f75600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb8ee00640 1 -- 172.21.15.46:0/1242686852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec161ff6c0 con 0x55ec16df9c00 2024-03-29T08:25:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb8da00640 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec161cf400 0x55ec15f75600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:25:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb8e400640 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec161cf000 0x55ec15f75b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:25:02.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb8ee00640 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec16df9c00 0x55ec15f76100 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55ec16288930 tx=0x55ec16e5ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d10b2fef26c68f00 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb83000640 1 -- 172.21.15.46:0/1242686852 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ec15f3c420 con 0x55ec16df9c00 2024-03-29T08:25:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb83000640 1 -- 172.21.15.46:0/1242686852 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ec16cc8c00 con 0x55ec16df9c00 2024-03-29T08:25:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ec161ff520 con 0x55ec16df9c00 2024-03-29T08:25:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55ec161ff380 con 0x55ec16df9c00 2024-03-29T08:25:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb90274400 4 waiting for MDS map... 2024-03-29T08:25:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb83000640 1 -- 172.21.15.46:0/1242686852 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ec15f3c6e0 con 0x55ec16df9c00 2024-03-29T08:25:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb83000640 1 -- 172.21.15.46:0/1242686852 <== mon.1 v2:172.21.15.73:3300/0 4 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x55ec15f288c0 con 0x55ec16df9c00 2024-03-29T08:25:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb83000640 1 -- 172.21.15.46:0/1242686852 <== mon.1 v2:172.21.15.73:3300/0 5 ==== fsmap(e 58) v1 ==== 2573+0+0 (secure 0 0 0) 0x55ec16326000 con 0x55ec16df9c00 2024-03-29T08:25:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.525+0000 7ffb90274400 4 Got MDS map 58 2024-03-29T08:25:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.529+0000 7ffb90274400 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x55ec161cf800 0x55ec16330000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.529+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55ec161cfc00 con 0x55ec161cf800 2024-03-29T08:25:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.529+0000 7ffb8e400640 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x55ec161cf800 0x55ec16330000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.529+0000 7ffb8e400640 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x55ec161cf800 0x55ec16330000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=866fdeb31b4e4e2f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.529+0000 7ffb8e400640 1 -- 172.21.15.46:0/1242686852 <== osd.3 v2:172.21.15.46:6808/27426 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x55ec16c48b40 con 0x55ec161cf800 2024-03-29T08:25:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.529+0000 7ffb90274400 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x55ec16315000 0x55ec16330580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.529+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:2 12.14 12:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e93) v8 -- 0x55ec16315400 con 0x55ec16315000 2024-03-29T08:25:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.529+0000 7ffb8da00640 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x55ec16315000 0x55ec16330580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.529+0000 7ffb8da00640 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x55ec16315000 0x55ec16330580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=7f8d33f120676364 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb8da00640 1 -- 172.21.15.46:0/1242686852 <== osd.0 v2:172.21.15.46:6800/27424 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv19 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x55ec16c48d80 con 0x55ec16315000 2024-03-29T08:25:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb81200640 1 -- 172.21.15.46:0/1242686852 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:3 12.14 12:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e93) v8 -- 0x55ec16338000 con 0x55ec16315000 2024-03-29T08:25:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb81200640 1 -- 172.21.15.46:0/1242686852 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:4 12.1b 12:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e93) v8 -- 0x55ec16338400 con 0x55ec16315000 2024-03-29T08:25:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb8da00640 1 -- 172.21.15.46:0/1242686852 <== osd.0 v2:172.21.15.46:6800/27424 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x55ec16c48d80 con 0x55ec16315000 2024-03-29T08:25:02.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb8da00640 1 -- 172.21.15.46:0/1242686852 <== osd.0 v2:172.21.15.46:6800/27424 3 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55ec16c48d80 con 0x55ec16315000 2024-03-29T08:25:02.533 INFO:teuthology.orchestra.run.smithi046.stdout:old journal was 4209788~892 2024-03-29T08:25:02.533 INFO:teuthology.orchestra.run.smithi046.stdout:new journal start will be 8388608 (4177928 bytes past old end) 2024-03-29T08:25:02.533 INFO:teuthology.orchestra.run.smithi046.stdout:writing journal head 2024-03-29T08:25:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:5 12.14 12:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55ec16315800 con 0x55ec16315000 2024-03-29T08:25:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb8da00640 1 -- 172.21.15.46:0/1242686852 <== osd.0 v2:172.21.15.46:6800/27424 4 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v93'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55ec16c48d80 con 0x55ec16315000 2024-03-29T08:25:02.534 INFO:teuthology.orchestra.run.smithi046.stdout:writing EResetJournal entry 2024-03-29T08:25:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:6 12.1b 12:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55ec16315c00 con 0x55ec16315000 2024-03-29T08:25:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb90274400 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x55ec16338000 0x55ec16330b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:7 12.b 12:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55ec16338400 con 0x55ec16338000 2024-03-29T08:25:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:8 12.1f 12:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55ec16338c00 con 0x55ec161cf800 2024-03-29T08:25:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb90274400 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x55ec16339000 0x55ec16331080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:9 12.1a 12:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55ec16339400 con 0x55ec16339000 2024-03-29T08:25:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:10 12.1e 12:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55ec16339800 con 0x55ec16339000 2024-03-29T08:25:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb90274400 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x55ec16315400 0x55ec16331600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:11 12.1d 12:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55ec16315800 con 0x55ec16315400 2024-03-29T08:25:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb8e400640 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x55ec16339000 0x55ec16331080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb8ee00640 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x55ec16338000 0x55ec16330b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb8da00640 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x55ec16315400 0x55ec16331600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb8e400640 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x55ec16339000 0x55ec16331080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=6880b73392f602d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb8da00640 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x55ec16315400 0x55ec16331600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=4165e8e78da51319 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb8ee00640 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x55ec16338000 0x55ec16330b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=1c13f21352a45297 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.533+0000 7ffb8da00640 1 -- 172.21.15.46:0/1242686852 <== osd.0 v2:172.21.15.46:6800/27424 5 ==== osd_op_reply(6 200.00000002 [delete] v93'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55ec16c48d80 con 0x55ec16315000 2024-03-29T08:25:02.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.537+0000 7ffb8e400640 1 -- 172.21.15.46:0/1242686852 <== osd.3 v2:172.21.15.46:6808/27426 2 ==== osd_op_reply(8 200.00000004 [delete] v93'7 uv6 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55ec16c48b40 con 0x55ec161cf800 2024-03-29T08:25:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.537+0000 7ffb8da00640 1 -- 172.21.15.46:0/1242686852 <== osd.6 v2:172.21.15.73:6824/27691 1 ==== osd_op_reply(11 200.00000007 [delete] v93'3 uv2 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55ec16c48d80 con 0x55ec16315400 2024-03-29T08:25:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.537+0000 7ffb8e400640 1 -- 172.21.15.46:0/1242686852 <== osd.2 v2:172.21.15.46:6809/27428 1 ==== osd_op_reply(9 200.00000005 [delete] v93'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55ec16c48b40 con 0x55ec16339000 2024-03-29T08:25:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.537+0000 7ffb8ee00640 1 -- 172.21.15.46:0/1242686852 <== osd.7 v2:172.21.15.73:6816/27700 1 ==== osd_op_reply(7 200.00000003 [delete] v93'4 uv3 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55ec16c498c0 con 0x55ec16338000 2024-03-29T08:25:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.537+0000 7ffb81200640 1 -- 172.21.15.46:0/1242686852 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:12 12.1b 12:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) v8 -- 0x55ec16338800 con 0x55ec16315000 2024-03-29T08:25:02.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.537+0000 7ffb8e400640 1 -- 172.21.15.46:0/1242686852 <== osd.2 v2:172.21.15.46:6809/27428 2 ==== osd_op_reply(10 200.00000006 [delete] v93'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55ec16c48b40 con 0x55ec16339000 2024-03-29T08:25:02.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb8da00640 1 -- 172.21.15.46:0/1242686852 <== osd.0 v2:172.21.15.46:6800/27424 6 ==== osd_op_reply(12 200.00000002 [write 0~48] v93'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55ec16c48d80 con 0x55ec16315000 2024-03-29T08:25:02.540 INFO:teuthology.orchestra.run.smithi046.stdout:done 2024-03-29T08:25:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x55ec16315000 msgr2=0x55ec16330580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x55ec16315000 0x55ec16330580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x55ec16339000 msgr2=0x55ec16331080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x55ec16339000 0x55ec16331080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x55ec161cf800 msgr2=0x55ec16330000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x55ec161cf800 0x55ec16330000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x55ec16315400 msgr2=0x55ec16331600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x55ec16315400 0x55ec16331600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x55ec16338000 msgr2=0x55ec16330b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x55ec16338000 0x55ec16330b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb8da00640 1 -- 172.21.15.46:0/1242686852 reap_dead start 2024-03-29T08:25:02.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb8da00640 1 -- 172.21.15.46:0/1242686852 reap_dead start 2024-03-29T08:25:02.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec16df9c00 msgr2=0x55ec15f76100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 --2- 172.21.15.46:0/1242686852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec16df9c00 0x55ec15f76100 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55ec16288930 tx=0x55ec16e5ec60 comp rx=0 tx=0).stop 2024-03-29T08:25:02.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb8da00640 1 -- 172.21.15.46:0/1242686852 reap_dead start 2024-03-29T08:25:02.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 shutdown_connections 2024-03-29T08:25:02.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 >> 172.21.15.46:0/1242686852 conn(0x55ec16df9800 msgr2=0x55ec162fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:02.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 shutdown_connections 2024-03-29T08:25:02.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/1242686852 wait complete. 2024-03-29T08:25:02.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/579289909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x55ec161cec00 msgr2=0x55ec15f77700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 --2- 172.21.15.46:0/579289909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x55ec161cec00 0x55ec15f77700 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x55ec15f218c0 tx=0x55ec16df7f20 comp rx=0 tx=0).stop 2024-03-29T08:25:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/579289909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec16df8c00 msgr2=0x55ec15f77180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 --2- 172.21.15.46:0/579289909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec16df8c00 0x55ec15f77180 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x55ec16e5ae10 tx=0x55ec16e5e360 comp rx=0 tx=0).stop 2024-03-29T08:25:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/579289909 shutdown_connections 2024-03-29T08:25:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 --2- 172.21.15.46:0/579289909 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x55ec161cec00 0x55ec15f77700 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 --2- 172.21.15.46:0/579289909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec16df9000 0x55ec15f76c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 --2- 172.21.15.46:0/579289909 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec16df9400 0x55ec15f76680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 --2- 172.21.15.46:0/579289909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec16df8c00 0x55ec15f77180 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/579289909 >> 172.21.15.46:0/579289909 conn(0x55ec15ee8c00 msgr2=0x55ec15f17800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/579289909 shutdown_connections 2024-03-29T08:25:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/579289909 wait complete. 2024-03-29T08:25:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/3163523946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec15ee9400 msgr2=0x55ec15f74b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 --2- 172.21.15.46:0/3163523946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec15ee9400 0x55ec15f74b00 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x55ec16d5ec90 tx=0x55ec16df6000 comp rx=0 tx=0).stop 2024-03-29T08:25:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/3163523946 shutdown_connections 2024-03-29T08:25:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 --2- 172.21.15.46:0/3163523946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec15ee9000 0x55ec15f75080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 --2- 172.21.15.46:0/3163523946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec15ee9400 0x55ec15f74b00 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 --2- 172.21.15.46:0/3163523946 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55ec15ee9800 0x55ec15f74580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/3163523946 >> 172.21.15.46:0/3163523946 conn(0x55ec15ee9c00 msgr2=0x55ec15f17000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:02.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/3163523946 shutdown_connections 2024-03-29T08:25:02.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.541+0000 7ffb90274400 1 -- 172.21.15.46:0/3163523946 wait complete. 2024-03-29T08:25:02.546 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.120263, result: old journal was 4209788~892 new journal start will be 8388608 (4177928 bytes past old end) writing journal head writing EResetJournal entry done 2024-03-29T08:25:02.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-29T08:25:02.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.729+0000 7fd530c00640 1 Processor -- start 2024-03-29T08:25:02.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.729+0000 7fd530c00640 1 -- start start 2024-03-29T08:25:02.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.733+0000 7fd530c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd52c0cc000 0x7fd52c06b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.733+0000 7fd530c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd52c0af710 0x7fd52c167280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.733+0000 7fd530c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd52c15d040 con 0x7fd52c0cd190 2024-03-29T08:25:02.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.733+0000 7fd530c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd52c06c550 con 0x7fd52c0af710 2024-03-29T08:25:02.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.733+0000 7fd530c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd52c0caba0 con 0x7fd52c0cc000 2024-03-29T08:25:02.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.733+0000 7fd52be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd52c0af710 0x7fd52c167280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.733+0000 7fd52be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd52c0af710 0x7fd52c167280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45112/0 (socket says 172.21.15.46:45112) 2024-03-29T08:25:02.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.733+0000 7fd52be00640 1 -- 172.21.15.46:0/1780688326 learned_addr learned my addr 172.21.15.46:0/1780688326 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:02.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.733+0000 7fd522a00640 1 --2- 172.21.15.46:0/1780688326 >> v2:172.21.15.73:3300/0 conn(0x7fd52c0cc000 0x7fd52c06b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.733+0000 7fd52be00640 1 -- 172.21.15.46:0/1780688326 >> v2:172.21.15.73:3300/0 conn(0x7fd52c0cc000 msgr2=0x7fd52c06b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:02.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.733+0000 7fd52be00640 1 --2- 172.21.15.46:0/1780688326 >> v2:172.21.15.73:3300/0 conn(0x7fd52c0cc000 0x7fd52c06b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.733+0000 7fd52be00640 1 -- 172.21.15.46:0/1780688326 >> v1:172.21.15.73:6789/0 conn(0x7fd52c0cd190 legacy=0x7fd52c06b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.733+0000 7fd52be00640 1 -- 172.21.15.46:0/1780688326 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c0cf800 con 0x7fd52c0af710 2024-03-29T08:25:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.733+0000 7fd52be00640 1 --2- 172.21.15.46:0/1780688326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd52c0af710 0x7fd52c167280 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fd524009870 tx=0x7fd52401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6b56f68e88ff3c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd52aa00640 1 -- 172.21.15.46:0/1780688326 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5240a8a10 con 0x7fd52c0af710 2024-03-29T08:25:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd52aa00640 1 -- 172.21.15.46:0/1780688326 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5240a8b70 con 0x7fd52c0af710 2024-03-29T08:25:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd52aa00640 1 -- 172.21.15.46:0/1780688326 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5240b0440 con 0x7fd52c0af710 2024-03-29T08:25:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd530c00640 1 -- 172.21.15.46:0/1780688326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd52c0af710 msgr2=0x7fd52c167280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd530c00640 1 --2- 172.21.15.46:0/1780688326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd52c0af710 0x7fd52c167280 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fd524009870 tx=0x7fd52401ed10 comp rx=0 tx=0).stop 2024-03-29T08:25:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd530c00640 1 -- 172.21.15.46:0/1780688326 shutdown_connections 2024-03-29T08:25:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd530c00640 1 --2- 172.21.15.46:0/1780688326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd52c0af710 0x7fd52c167280 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd530c00640 1 --2- 172.21.15.46:0/1780688326 >> v2:172.21.15.73:3300/0 conn(0x7fd52c0cc000 0x7fd52c06b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd530c00640 1 -- 172.21.15.46:0/1780688326 >> 172.21.15.46:0/1780688326 conn(0x7fd52c0d0550 msgr2=0x7fd52c0d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd530c00640 1 -- 172.21.15.46:0/1780688326 shutdown_connections 2024-03-29T08:25:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd530c00640 1 -- 172.21.15.46:0/1780688326 wait complete. 2024-03-29T08:25:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd530c00640 1 Processor -- start 2024-03-29T08:25:02.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd530c00640 1 -- start start 2024-03-29T08:25:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd530c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd52c0af710 0x7fd52c0c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd530c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd52c0cc000 0x7fd52c0ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd530c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd52c0cd190 0x7fd52c14a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd530c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c155380 con 0x7fd52c0cd190 2024-03-29T08:25:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd530c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c155660 con 0x7fd52c0cc000 2024-03-29T08:25:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd530c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c155940 con 0x7fd52c0af710 2024-03-29T08:25:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd52be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd52c0cd190 0x7fd52c14a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd52be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd52c0cd190 0x7fd52c14a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45128/0 (socket says 172.21.15.46:45128) 2024-03-29T08:25:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd52be00640 1 -- 172.21.15.46:0/3736860230 learned_addr learned my addr 172.21.15.46:0/3736860230 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd522a00640 1 --2- 172.21.15.46:0/3736860230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd52c0cc000 0x7fd52c0ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd52b400640 1 --2- 172.21.15.46:0/3736860230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd52c0af710 0x7fd52c0c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd52be00640 1 -- 172.21.15.46:0/3736860230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd52c0af710 msgr2=0x7fd52c0c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd52be00640 1 --2- 172.21.15.46:0/3736860230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd52c0af710 0x7fd52c0c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd52be00640 1 -- 172.21.15.46:0/3736860230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd52c0cc000 msgr2=0x7fd52c0ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd52be00640 1 --2- 172.21.15.46:0/3736860230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd52c0cc000 0x7fd52c0ae320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.737+0000 7fd52be00640 1 -- 172.21.15.46:0/3736860230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5240094d0 con 0x7fd52c0cd190 2024-03-29T08:25:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.741+0000 7fd52be00640 1 --2- 172.21.15.46:0/3736860230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd52c0cd190 0x7fd52c14a8b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fd5240b3040 tx=0x7fd52401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.741+0000 7fd523e00640 1 -- 172.21.15.46:0/3736860230 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5240b0d70 con 0x7fd52c0cd190 2024-03-29T08:25:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.741+0000 7fd523e00640 1 -- 172.21.15.46:0/3736860230 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5240bb460 con 0x7fd52c0cd190 2024-03-29T08:25:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.741+0000 7fd523e00640 1 -- 172.21.15.46:0/3736860230 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5240bb7f0 con 0x7fd52c0cd190 2024-03-29T08:25:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.741+0000 7fd530c00640 1 -- 172.21.15.46:0/3736860230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd52c0adb20 con 0x7fd52c0cd190 2024-03-29T08:25:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.741+0000 7fd530c00640 1 -- 172.21.15.46:0/3736860230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd52c0ca430 con 0x7fd52c0cd190 2024-03-29T08:25:02.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.741+0000 7fd530c00640 1 -- 172.21.15.46:0/3736860230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd52c0cfda0 con 0x7fd52c0cd190 2024-03-29T08:25:02.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.741+0000 7fd523e00640 1 -- 172.21.15.46:0/3736860230 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd5240b83f0 con 0x7fd52c0cd190 2024-03-29T08:25:02.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.741+0000 7fd523e00640 1 --2- 172.21.15.46:0/3736860230 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd508066690 0x7fd508068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:02.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.741+0000 7fd523e00640 1 -- 172.21.15.46:0/3736860230 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd5241219a0 con 0x7fd52c0cd190 2024-03-29T08:25:02.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.741+0000 7fd52b400640 1 --2- 172.21.15.46:0/3736860230 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd508066690 0x7fd508068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:02.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.745+0000 7fd52b400640 1 --2- 172.21.15.46:0/3736860230 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd508066690 0x7fd508068b50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0012f0 tx=0x7fd51c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:02.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.745+0000 7fd523e00640 1 -- 172.21.15.46:0/3736860230 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd52c0cfda0 con 0x7fd52c0cd190 2024-03-29T08:25:02.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.941+0000 7fd530c00640 1 -- 172.21.15.46:0/3736860230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fd52c0cfda0 con 0x7fd52c0cd190 2024-03-29T08:25:02.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.977+0000 7fd523e00640 1 -- 172.21.15.46:0/3736860230 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v59) v1 ==== 162+0+0 (secure 0 0 0) 0x7fd52c0cfda0 con 0x7fd52c0cd190 2024-03-29T08:25:02.977 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-29T08:25:02.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.981+0000 7fd530c00640 1 -- 172.21.15.46:0/3736860230 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd508066690 msgr2=0x7fd508068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.981+0000 7fd530c00640 1 --2- 172.21.15.46:0/3736860230 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd508066690 0x7fd508068b50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd51c0012f0 tx=0x7fd51c009040 comp rx=0 tx=0).stop 2024-03-29T08:25:02.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.981+0000 7fd530c00640 1 -- 172.21.15.46:0/3736860230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd52c0cd190 msgr2=0x7fd52c14a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:02.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.981+0000 7fd530c00640 1 --2- 172.21.15.46:0/3736860230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd52c0cd190 0x7fd52c14a8b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fd5240b3040 tx=0x7fd52401bab0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.981+0000 7fd530c00640 1 -- 172.21.15.46:0/3736860230 shutdown_connections 2024-03-29T08:25:02.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.981+0000 7fd530c00640 1 --2- 172.21.15.46:0/3736860230 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd508066690 0x7fd508068b50 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.981+0000 7fd530c00640 1 --2- 172.21.15.46:0/3736860230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd52c0cc000 0x7fd52c0ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.981+0000 7fd530c00640 1 --2- 172.21.15.46:0/3736860230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd52c0cd190 0x7fd52c14a8b0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.981+0000 7fd530c00640 1 --2- 172.21.15.46:0/3736860230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd52c0af710 0x7fd52c0c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:02.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.981+0000 7fd530c00640 1 -- 172.21.15.46:0/3736860230 >> 172.21.15.46:0/3736860230 conn(0x7fd52c0d0550 msgr2=0x7fd52c0cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:02.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.981+0000 7fd530c00640 1 -- 172.21.15.46:0/3736860230 shutdown_connections 2024-03-29T08:25:02.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:02.981+0000 7fd530c00640 1 -- 172.21.15.46:0/3736860230 wait complete. 2024-03-29T08:25:03.002 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:25:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.141+0000 7f0cbee00640 1 Processor -- start 2024-03-29T08:25:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.145+0000 7f0cbee00640 1 -- start start 2024-03-29T08:25:03.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.145+0000 7f0cbee00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cb806f090 0x7f0cb80abd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:03.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.145+0000 7f0cbee00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cb80ce870 0x7f0cb80cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:03.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.145+0000 7f0cbee00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb806f470 con 0x7f0cb8069d90 2024-03-29T08:25:03.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.145+0000 7f0cbee00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb815bc70 con 0x7f0cb80ce870 2024-03-29T08:25:03.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.145+0000 7f0cbee00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb8155240 con 0x7f0cb806f090 2024-03-29T08:25:03.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.145+0000 7f0cbe400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cb80ce870 0x7f0cb80cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:03.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.145+0000 7f0cbe400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cb80ce870 0x7f0cb80cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45142/0 (socket says 172.21.15.46:45142) 2024-03-29T08:25:03.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.145+0000 7f0cbe400640 1 -- 172.21.15.46:0/2643478083 learned_addr learned my addr 172.21.15.46:0/2643478083 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:03.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.145+0000 7f0cbda00640 1 --2- 172.21.15.46:0/2643478083 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cb806f090 0x7f0cb80abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:03.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.145+0000 7f0cbe400640 1 -- 172.21.15.46:0/2643478083 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cb806f090 msgr2=0x7f0cb80abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:03.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.145+0000 7f0cbe400640 1 --2- 172.21.15.46:0/2643478083 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cb806f090 0x7f0cb80abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.145+0000 7f0cbe400640 1 -- 172.21.15.46:0/2643478083 >> v1:172.21.15.46:6789/0 conn(0x7f0cb8069d90 legacy=0x7f0cb80ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:03.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.145+0000 7f0cbe400640 1 -- 172.21.15.46:0/2643478083 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb8066110 con 0x7f0cb80ce870 2024-03-29T08:25:03.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.145+0000 7f0cb7e00640 1 -- 172.21.15.46:0/2643478083 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39161279 0 0) 0x7f0ca4001120 con 0x7f0cb8069d90 2024-03-29T08:25:03.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.149+0000 7f0cbe400640 1 --2- 172.21.15.46:0/2643478083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb80ce870 0x7f0cb80cced0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009e50 tx=0x7f0cb001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d104148f01b68539 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:03.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.149+0000 7f0cb7e00640 1 -- 172.21.15.46:0/2643478083 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb00a5900 con 0x7f0cb80ce870 2024-03-29T08:25:03.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.149+0000 7f0cb7e00640 1 -- 172.21.15.46:0/2643478083 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb00a5a60 con 0x7f0cb80ce870 2024-03-29T08:25:03.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.149+0000 7f0cb7e00640 1 -- 172.21.15.46:0/2643478083 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb00a5df0 con 0x7f0cb80ce870 2024-03-29T08:25:03.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.149+0000 7f0cbee00640 1 -- 172.21.15.46:0/2643478083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb80ce870 msgr2=0x7f0cb80cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:03.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.149+0000 7f0cbee00640 1 --2- 172.21.15.46:0/2643478083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb80ce870 0x7f0cb80cced0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009e50 tx=0x7f0cb001ca30 comp rx=0 tx=0).stop 2024-03-29T08:25:03.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.149+0000 7f0cbee00640 1 -- 172.21.15.46:0/2643478083 shutdown_connections 2024-03-29T08:25:03.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.149+0000 7f0cbee00640 1 --2- 172.21.15.46:0/2643478083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb80ce870 0x7f0cb80cced0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.149+0000 7f0cbee00640 1 --2- 172.21.15.46:0/2643478083 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cb806f090 0x7f0cb80abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.149+0000 7f0cbee00640 1 -- 172.21.15.46:0/2643478083 >> 172.21.15.46:0/2643478083 conn(0x7f0cb80cb8a0 msgr2=0x7f0cb80cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:03.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.149+0000 7f0cbee00640 1 -- 172.21.15.46:0/2643478083 shutdown_connections 2024-03-29T08:25:03.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.149+0000 7f0cbee00640 1 -- 172.21.15.46:0/2643478083 wait complete. 2024-03-29T08:25:03.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.149+0000 7f0cbee00640 1 Processor -- start 2024-03-29T08:25:03.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.149+0000 7f0cbee00640 1 -- start start 2024-03-29T08:25:03.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbee00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cb8069d90 0x7f0cb8144b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:03.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb806f090 0x7f0cb806cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:03.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cb80ce870 0x7f0cb8151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:03.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb806e510 con 0x7f0cb806f090 2024-03-29T08:25:03.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb806d890 con 0x7f0cb80ce870 2024-03-29T08:25:03.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbee00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb806db70 con 0x7f0cb8069d90 2024-03-29T08:25:03.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbd000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb806f090 0x7f0cb806cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:03.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbd000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb806f090 0x7f0cb806cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45154/0 (socket says 172.21.15.46:45154) 2024-03-29T08:25:03.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbd000640 1 -- 172.21.15.46:0/2763046864 learned_addr learned my addr 172.21.15.46:0/2763046864 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:03.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbda00640 1 --2- 172.21.15.46:0/2763046864 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cb8069d90 0x7f0cb8144b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:03.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbd000640 1 -- 172.21.15.46:0/2763046864 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cb8069d90 msgr2=0x7f0cb8144b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:03.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbe400640 1 --2- 172.21.15.46:0/2763046864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cb80ce870 0x7f0cb8151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:03.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbd000640 1 --2- 172.21.15.46:0/2763046864 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cb8069d90 0x7f0cb8144b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbd000640 1 -- 172.21.15.46:0/2763046864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cb80ce870 msgr2=0x7f0cb8151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:03.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbd000640 1 --2- 172.21.15.46:0/2763046864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cb80ce870 0x7f0cb8151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbd000640 1 -- 172.21.15.46:0/2763046864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb0009b30 con 0x7f0cb806f090 2024-03-29T08:25:03.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbda00640 1 --2- 172.21.15.46:0/2763046864 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cb8069d90 0x7f0cb8144b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:25:03.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbe400640 1 --2- 172.21.15.46:0/2763046864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cb80ce870 0x7f0cb8151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:25:03.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbd000640 1 --2- 172.21.15.46:0/2763046864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb806f090 0x7f0cb806cdf0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400c3f0 tx=0x7f0ca400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:03.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cb5600640 1 -- 172.21.15.46:0/2763046864 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca40054e0 con 0x7f0cb806f090 2024-03-29T08:25:03.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cb5600640 1 -- 172.21.15.46:0/2763046864 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca4003600 con 0x7f0cb806f090 2024-03-29T08:25:03.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cb5600640 1 -- 172.21.15.46:0/2763046864 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4003990 con 0x7f0cb806f090 2024-03-29T08:25:03.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbee00640 1 -- 172.21.15.46:0/2763046864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb806de50 con 0x7f0cb806f090 2024-03-29T08:25:03.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbee00640 1 -- 172.21.15.46:0/2763046864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb806c9b0 con 0x7f0cb806f090 2024-03-29T08:25:03.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cbee00640 1 -- 172.21.15.46:0/2763046864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb806e650 con 0x7f0cb806f090 2024-03-29T08:25:03.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.153+0000 7f0cb5600640 1 -- 172.21.15.46:0/2763046864 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0ca4003af0 con 0x7f0cb806f090 2024-03-29T08:25:03.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.157+0000 7f0cb5600640 1 --2- 172.21.15.46:0/2763046864 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0c980666e0 0x7f0c98068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:03.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.157+0000 7f0cb5600640 1 -- 172.21.15.46:0/2763046864 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0ca4083640 con 0x7f0cb806f090 2024-03-29T08:25:03.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.157+0000 7f0cbda00640 1 --2- 172.21.15.46:0/2763046864 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0c980666e0 0x7f0c98068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:03.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.157+0000 7f0cbda00640 1 --2- 172.21.15.46:0/2763046864 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0c980666e0 0x7f0c98068ba0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0097a0 tx=0x7f0cac008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:03.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.161+0000 7f0cb5600640 1 -- 172.21.15.46:0/2763046864 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0cb806e650 con 0x7f0cb806f090 2024-03-29T08:25:03.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.361+0000 7f0cbee00640 1 -- 172.21.15.46:0/2763046864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cb806f470 con 0x7f0cb806f090 2024-03-29T08:25:03.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.361+0000 7f0cb5600640 1 -- 172.21.15.46:0/2763046864 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f0cb806f470 con 0x7f0cb806f090 2024-03-29T08:25:03.362 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:25:03.362 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/1862652020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1862652020},{"type":"v1","addr":"172.21.15.73:6836","nonce":1862652020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:24:43.476334+0000","modified":"2024-03-29T08:25:02.975446+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7386},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7386":{"gid":7386,"name":"c","rank":0,"incarnation":59,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6835/2051429508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2051429508},{"type":"v1","addr":"172.21.15.46:6835","nonce":2051429508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-29T08:25:03.362 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 59 2024-03-29T08:25:03.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.361+0000 7f0cbee00640 1 -- 172.21.15.46:0/2763046864 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0c980666e0 msgr2=0x7f0c98068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:03.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.361+0000 7f0cbee00640 1 --2- 172.21.15.46:0/2763046864 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0c980666e0 0x7f0c98068ba0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0cac0097a0 tx=0x7f0cac008040 comp rx=0 tx=0).stop 2024-03-29T08:25:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.361+0000 7f0cbee00640 1 -- 172.21.15.46:0/2763046864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb806f090 msgr2=0x7f0cb806cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.361+0000 7f0cbee00640 1 --2- 172.21.15.46:0/2763046864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb806f090 0x7f0cb806cdf0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400c3f0 tx=0x7f0ca400c8b0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.365+0000 7f0cbee00640 1 -- 172.21.15.46:0/2763046864 shutdown_connections 2024-03-29T08:25:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.365+0000 7f0cbee00640 1 --2- 172.21.15.46:0/2763046864 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0c980666e0 0x7f0c98068ba0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.365+0000 7f0cbee00640 1 --2- 172.21.15.46:0/2763046864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cb80ce870 0x7f0cb8151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.365+0000 7f0cbee00640 1 --2- 172.21.15.46:0/2763046864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb806f090 0x7f0cb806cdf0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.365+0000 7f0cbee00640 1 --2- 172.21.15.46:0/2763046864 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cb8069d90 0x7f0cb8144b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.365+0000 7f0cbee00640 1 -- 172.21.15.46:0/2763046864 >> 172.21.15.46:0/2763046864 conn(0x7f0cb80cb8a0 msgr2=0x7f0cb80aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.365+0000 7f0cbee00640 1 -- 172.21.15.46:0/2763046864 shutdown_connections 2024-03-29T08:25:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:03.365+0000 7f0cbee00640 1 -- 172.21.15.46:0/2763046864 wait complete. 2024-03-29T08:25:03.784 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:24:43.476334+0000', 'modified': '2024-03-29T08:25:02.975446+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7386}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7386': {'gid': 7386, 'name': 'c', 'rank': 0, 'incarnation': 59, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.46:6835/2051429508', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 2051429508}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2051429508}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:25:03.784 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7386:up:replay 2024-03-29T08:25:04.785 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:25:04.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.937+0000 7f1a17400640 1 Processor -- start 2024-03-29T08:25:04.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.937+0000 7f1a17400640 1 -- start start 2024-03-29T08:25:04.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.937+0000 7f1a17400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1a1015cd90 0x7f1a1006cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:04.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.937+0000 7f1a17400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1a1015d940 0x7f1a101597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:04.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.937+0000 7f1a17400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1006d130 con 0x7f1a101593d0 2024-03-29T08:25:04.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.937+0000 7f1a17400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a1015b8f0 con 0x7f1a1015cd90 2024-03-29T08:25:04.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.937+0000 7f1a17400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1015ad50 con 0x7f1a1015d940 2024-03-29T08:25:04.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.941+0000 7f1a15400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1a1015d940 0x7f1a101597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:04.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.941+0000 7f1a15400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1a1015d940 0x7f1a101597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:42958/0 (socket says 172.21.15.46:42958) 2024-03-29T08:25:04.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.941+0000 7f1a15400640 1 -- 172.21.15.46:0/2916106176 learned_addr learned my addr 172.21.15.46:0/2916106176 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:04.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.941+0000 7f1a0fe00640 1 --2- 172.21.15.46:0/2916106176 >> v2:172.21.15.73:3300/0 conn(0x7f1a1015cd90 0x7f1a1006cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:04.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.941+0000 7f1a15400640 1 -- 172.21.15.46:0/2916106176 >> v2:172.21.15.73:3300/0 conn(0x7f1a1015cd90 msgr2=0x7f1a1006cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:04.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.941+0000 7f1a15400640 1 --2- 172.21.15.46:0/2916106176 >> v2:172.21.15.73:3300/0 conn(0x7f1a1015cd90 0x7f1a1006cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.941+0000 7f1a15400640 1 -- 172.21.15.46:0/2916106176 >> v1:172.21.15.73:6789/0 conn(0x7f1a101593d0 legacy=0x7f1a1015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:04.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.941+0000 7f1a15400640 1 -- 172.21.15.46:0/2916106176 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1015afc0 con 0x7f1a1015d940 2024-03-29T08:25:04.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.941+0000 7f1a15400640 1 --2- 172.21.15.46:0/2916106176 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1a1015d940 0x7f1a101597b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800d150 tx=0x7f1a0801ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=432efd6a77440bcc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a0f400640 1 -- 172.21.15.46:0/2916106176 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a080a7d00 con 0x7f1a1015d940 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a0f400640 1 -- 172.21.15.46:0/2916106176 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a080a7e60 con 0x7f1a1015d940 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a0f400640 1 -- 172.21.15.46:0/2916106176 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0800b2f0 con 0x7f1a1015d940 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a17400640 1 -- 172.21.15.46:0/2916106176 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1a1015d940 msgr2=0x7f1a101597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a17400640 1 --2- 172.21.15.46:0/2916106176 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1a1015d940 0x7f1a101597b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800d150 tx=0x7f1a0801ec30 comp rx=0 tx=0).stop 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a17400640 1 -- 172.21.15.46:0/2916106176 shutdown_connections 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a17400640 1 --2- 172.21.15.46:0/2916106176 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1a1015d940 0x7f1a101597b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a17400640 1 --2- 172.21.15.46:0/2916106176 >> v2:172.21.15.73:3300/0 conn(0x7f1a1015cd90 0x7f1a1006cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a17400640 1 -- 172.21.15.46:0/2916106176 >> 172.21.15.46:0/2916106176 conn(0x7f1a10066110 msgr2=0x7f1a10069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a17400640 1 -- 172.21.15.46:0/2916106176 shutdown_connections 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a17400640 1 -- 172.21.15.46:0/2916106176 wait complete. 2024-03-29T08:25:04.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a17400640 1 Processor -- start 2024-03-29T08:25:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a17400640 1 -- start start 2024-03-29T08:25:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a17400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a101593d0 0x7f1a1014fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a17400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1a1015cd90 0x7f1a100ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a17400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a1015d940 0x7f1a10150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a17400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a101526d0 con 0x7f1a1015d940 2024-03-29T08:25:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a17400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a100ce6d0 con 0x7f1a101593d0 2024-03-29T08:25:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a17400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a100ce110 con 0x7f1a1015cd90 2024-03-29T08:25:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a15400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a1015d940 0x7f1a10150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a15400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a1015d940 0x7f1a10150ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45166/0 (socket says 172.21.15.46:45166) 2024-03-29T08:25:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.945+0000 7f1a15400640 1 -- 172.21.15.46:0/3493018934 learned_addr learned my addr 172.21.15.46:0/3493018934 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.949+0000 7f1a0fe00640 1 --2- 172.21.15.46:0/3493018934 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1a1015cd90 0x7f1a100ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.949+0000 7f1a14a00640 1 --2- 172.21.15.46:0/3493018934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a101593d0 0x7f1a1014fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.949+0000 7f1a15400640 1 -- 172.21.15.46:0/3493018934 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1a1015cd90 msgr2=0x7f1a100ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:04.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.949+0000 7f1a15400640 1 --2- 172.21.15.46:0/3493018934 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1a1015cd90 0x7f1a100ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.949+0000 7f1a15400640 1 -- 172.21.15.46:0/3493018934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a101593d0 msgr2=0x7f1a1014fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.949+0000 7f1a15400640 1 --2- 172.21.15.46:0/3493018934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a101593d0 0x7f1a1014fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.949+0000 7f1a15400640 1 -- 172.21.15.46:0/3493018934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0800bd40 con 0x7f1a1015d940 2024-03-29T08:25:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.949+0000 7f1a15400640 1 --2- 172.21.15.46:0/3493018934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a1015d940 0x7f1a10150ba0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f1a0801d2e0 tx=0x7f1a0801b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.949+0000 7f1a0cc00640 1 -- 172.21.15.46:0/3493018934 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a080be070 con 0x7f1a1015d940 2024-03-29T08:25:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.949+0000 7f1a0cc00640 1 -- 172.21.15.46:0/3493018934 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a08009600 con 0x7f1a1015d940 2024-03-29T08:25:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.949+0000 7f1a0cc00640 1 -- 172.21.15.46:0/3493018934 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a080099b0 con 0x7f1a1015d940 2024-03-29T08:25:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.949+0000 7f1a17400640 1 -- 172.21.15.46:0/3493018934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a100ce3f0 con 0x7f1a1015d940 2024-03-29T08:25:04.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.949+0000 7f1a17400640 1 -- 172.21.15.46:0/3493018934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a100cd300 con 0x7f1a1015d940 2024-03-29T08:25:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.949+0000 7f1a17400640 1 -- 172.21.15.46:0/3493018934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a10152150 con 0x7f1a1015d940 2024-03-29T08:25:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.949+0000 7f1a0cc00640 1 -- 172.21.15.46:0/3493018934 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1a08009b10 con 0x7f1a1015d940 2024-03-29T08:25:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.949+0000 7f1a0cc00640 1 --2- 172.21.15.46:0/3493018934 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f19f0066690 0x7f19f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.949+0000 7f1a0cc00640 1 -- 172.21.15.46:0/3493018934 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1a0811f1c0 con 0x7f1a1015d940 2024-03-29T08:25:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.953+0000 7f1a14a00640 1 --2- 172.21.15.46:0/3493018934 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f19f0066690 0x7f19f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:04.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.953+0000 7f1a14a00640 1 --2- 172.21.15.46:0/3493018934 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f19f0066690 0x7f19f0068b50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1a04000f70 tx=0x7f1a04009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:04.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:04.957+0000 7f1a0cc00640 1 -- 172.21.15.46:0/3493018934 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1a10152150 con 0x7f1a1015d940 2024-03-29T08:25:05.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:05.149+0000 7f1a17400640 1 -- 172.21.15.46:0/3493018934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a10152150 con 0x7f1a1015d940 2024-03-29T08:25:05.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:05.153+0000 7f1a0cc00640 1 -- 172.21.15.46:0/3493018934 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f1a10152150 con 0x7f1a1015d940 2024-03-29T08:25:05.151 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:25:05.152 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/1862652020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1862652020},{"type":"v1","addr":"172.21.15.73:6836","nonce":1862652020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:24:43.476334+0000","modified":"2024-03-29T08:25:04.982922+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7386},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7386":{"gid":7386,"name":"c","rank":0,"incarnation":59,"state":"up:rejoin","state_seq":9,"addr":"172.21.15.46:6835/2051429508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2051429508},{"type":"v1","addr":"172.21.15.46:6835","nonce":2051429508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-29T08:25:05.152 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 61 2024-03-29T08:25:05.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:05.153+0000 7f1a17400640 1 -- 172.21.15.46:0/3493018934 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f19f0066690 msgr2=0x7f19f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:05.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:05.153+0000 7f1a17400640 1 --2- 172.21.15.46:0/3493018934 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f19f0066690 0x7f19f0068b50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1a04000f70 tx=0x7f1a04009040 comp rx=0 tx=0).stop 2024-03-29T08:25:05.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:05.153+0000 7f1a17400640 1 -- 172.21.15.46:0/3493018934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a1015d940 msgr2=0x7f1a10150ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:05.153+0000 7f1a17400640 1 --2- 172.21.15.46:0/3493018934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a1015d940 0x7f1a10150ba0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f1a0801d2e0 tx=0x7f1a0801b9d0 comp rx=0 tx=0).stop 2024-03-29T08:25:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:05.153+0000 7f1a17400640 1 -- 172.21.15.46:0/3493018934 shutdown_connections 2024-03-29T08:25:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:05.153+0000 7f1a17400640 1 --2- 172.21.15.46:0/3493018934 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f19f0066690 0x7f19f0068b50 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:05.153+0000 7f1a17400640 1 --2- 172.21.15.46:0/3493018934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a1015d940 0x7f1a10150ba0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:05.153+0000 7f1a17400640 1 --2- 172.21.15.46:0/3493018934 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1a1015cd90 0x7f1a100ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:05.153+0000 7f1a17400640 1 --2- 172.21.15.46:0/3493018934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a101593d0 0x7f1a1014fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:05.153+0000 7f1a17400640 1 -- 172.21.15.46:0/3493018934 >> 172.21.15.46:0/3493018934 conn(0x7f1a10066110 msgr2=0x7f1a10163900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:05.153+0000 7f1a17400640 1 -- 172.21.15.46:0/3493018934 shutdown_connections 2024-03-29T08:25:05.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:05.153+0000 7f1a17400640 1 -- 172.21.15.46:0/3493018934 wait complete. 2024-03-29T08:25:05.575 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:24:43.476334+0000', 'modified': '2024-03-29T08:25:04.982922+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7386}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7386': {'gid': 7386, 'name': 'c', 'rank': 0, 'incarnation': 59, 'state': 'up:rejoin', 'state_seq': 9, 'addr': '172.21.15.46:6835/2051429508', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 2051429508}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2051429508}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:25:05.576 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7386:up:rejoin 2024-03-29T08:25:06.576 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:25:06.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.725+0000 7f8363200640 1 Processor -- start 2024-03-29T08:25:06.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.725+0000 7f8363200640 1 -- start start 2024-03-29T08:25:06.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.725+0000 7f8363200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f836415e160 0x7f83640caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:06.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.725+0000 7f8363200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f83640b0220 0x7f83640b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:06.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.725+0000 7f8363200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83640b0bb0 con 0x7f836415a6b0 2024-03-29T08:25:06.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.725+0000 7f8363200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f836415e5f0 con 0x7f836415e160 2024-03-29T08:25:06.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.725+0000 7f8363200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f836415aa90 con 0x7f83640b0220 2024-03-29T08:25:06.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.725+0000 7f8361e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f836415a6b0 0x7f836415b5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35378/0 (socket says 172.21.15.46:35378) 2024-03-29T08:25:06.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.725+0000 7f8361e00640 1 -- 172.21.15.46:0/1876276274 learned_addr learned my addr 172.21.15.46:0/1876276274 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:25:06.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.725+0000 7f8361400640 1 --2- 172.21.15.46:0/1876276274 >> v2:172.21.15.46:3300/0 conn(0x7f836415e160 0x7f83640caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:06.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.725+0000 7f8362800640 1 --2- 172.21.15.46:0/1876276274 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f83640b0220 0x7f83640b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:06.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.725+0000 7f8361400640 1 -- 172.21.15.46:0/1876276274 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f83640b0220 msgr2=0x7f83640b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:06.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.725+0000 7f8361400640 1 --2- 172.21.15.46:0/1876276274 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f83640b0220 0x7f83640b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:06.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.725+0000 7f8361400640 1 -- 172.21.15.46:0/1876276274 >> v1:172.21.15.46:6789/0 conn(0x7f836415a6b0 legacy=0x7f836415b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:06.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.725+0000 7f8361400640 1 -- 172.21.15.46:0/1876276274 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83640aef50 con 0x7f836415e160 2024-03-29T08:25:06.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.729+0000 7f8361400640 1 --2- 172.21.15.46:0/1876276274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415e160 0x7f83640caf30 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f83500097c0 tx=0x7f835001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93caed5b10209072 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:06.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.729+0000 7f8360a00640 1 -- 172.21.15.46:0/1876276274 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83500a6040 con 0x7f836415e160 2024-03-29T08:25:06.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.729+0000 7f8360a00640 1 -- 172.21.15.46:0/1876276274 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83500aa600 con 0x7f836415e160 2024-03-29T08:25:06.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.729+0000 7f8360a00640 1 -- 172.21.15.46:0/1876276274 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83500aa990 con 0x7f836415e160 2024-03-29T08:25:06.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.729+0000 7f8363200640 1 -- 172.21.15.46:0/1876276274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415e160 msgr2=0x7f83640caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:06.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.729+0000 7f8363200640 1 --2- 172.21.15.46:0/1876276274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415e160 0x7f83640caf30 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f83500097c0 tx=0x7f835001c4a0 comp rx=0 tx=0).stop 2024-03-29T08:25:06.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.729+0000 7f8363200640 1 -- 172.21.15.46:0/1876276274 shutdown_connections 2024-03-29T08:25:06.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.729+0000 7f8363200640 1 --2- 172.21.15.46:0/1876276274 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f83640b0220 0x7f83640b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:06.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.729+0000 7f8363200640 1 --2- 172.21.15.46:0/1876276274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415e160 0x7f83640caf30 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:06.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.729+0000 7f8363200640 1 -- 172.21.15.46:0/1876276274 >> 172.21.15.46:0/1876276274 conn(0x7f8364098a40 msgr2=0x7f83640cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:06.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.729+0000 7f8363200640 1 -- 172.21.15.46:0/1876276274 shutdown_connections 2024-03-29T08:25:06.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.729+0000 7f8363200640 1 -- 172.21.15.46:0/1876276274 wait complete. 2024-03-29T08:25:06.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.729+0000 7f8363200640 1 Processor -- start 2024-03-29T08:25:06.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.729+0000 7f8363200640 1 -- start start 2024-03-29T08:25:06.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8363200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83640b0220 0x7f83640c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:06.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8363200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f836415a6b0 0x7f83640add70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:06.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8363200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415e160 0x7f83641f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:06.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8363200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83640c8f90 con 0x7f836415e160 2024-03-29T08:25:06.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8363200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83640b4c80 con 0x7f83640b0220 2024-03-29T08:25:06.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8363200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83640c8380 con 0x7f836415a6b0 2024-03-29T08:25:06.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8362800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415e160 0x7f83641f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:06.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8362800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415e160 0x7f83641f4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38338/0 (socket says 172.21.15.46:38338) 2024-03-29T08:25:06.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8362800640 1 -- 172.21.15.46:0/592384401 learned_addr learned my addr 172.21.15.46:0/592384401 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:06.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8361e00640 1 --2- 172.21.15.46:0/592384401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83640b0220 0x7f83640c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:06.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8361400640 1 --2- 172.21.15.46:0/592384401 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f836415a6b0 0x7f83640add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:06.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8362800640 1 -- 172.21.15.46:0/592384401 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f836415a6b0 msgr2=0x7f83640add70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:06.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8362800640 1 --2- 172.21.15.46:0/592384401 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f836415a6b0 0x7f83640add70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:06.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8362800640 1 -- 172.21.15.46:0/592384401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83640b0220 msgr2=0x7f83640c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8362800640 1 --2- 172.21.15.46:0/592384401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83640b0220 0x7f83640c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8362800640 1 -- 172.21.15.46:0/592384401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8350009420 con 0x7f836415e160 2024-03-29T08:25:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8361400640 1 --2- 172.21.15.46:0/592384401 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f836415a6b0 0x7f83640add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8361e00640 1 --2- 172.21.15.46:0/592384401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83640b0220 0x7f83640c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:06.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8362800640 1 --2- 172.21.15.46:0/592384401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415e160 0x7f83641f4720 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f835c007e40 tx=0x7f835c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f835a000640 1 -- 172.21.15.46:0/592384401 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c014620 con 0x7f836415e160 2024-03-29T08:25:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f835a000640 1 -- 172.21.15.46:0/592384401 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835c00ee90 con 0x7f836415e160 2024-03-29T08:25:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8363200640 1 -- 172.21.15.46:0/592384401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83640c8660 con 0x7f836415e160 2024-03-29T08:25:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f835a000640 1 -- 172.21.15.46:0/592384401 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c0149e0 con 0x7f836415e160 2024-03-29T08:25:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8363200640 1 -- 172.21.15.46:0/592384401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83640c8c20 con 0x7f836415e160 2024-03-29T08:25:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.733+0000 7f8363200640 1 -- 172.21.15.46:0/592384401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83640c9100 con 0x7f836415e160 2024-03-29T08:25:06.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.737+0000 7f835a000640 1 -- 172.21.15.46:0/592384401 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f835c014b40 con 0x7f836415e160 2024-03-29T08:25:06.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.737+0000 7f835a000640 1 --2- 172.21.15.46:0/592384401 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f83440666c0 0x7f8344068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:06.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.737+0000 7f835a000640 1 -- 172.21.15.46:0/592384401 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f835c084bb0 con 0x7f836415e160 2024-03-29T08:25:06.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.737+0000 7f8361e00640 1 --2- 172.21.15.46:0/592384401 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f83440666c0 0x7f8344068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:06.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.737+0000 7f8361e00640 1 --2- 172.21.15.46:0/592384401 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f83440666c0 0x7f8344068b80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8354000f70 tx=0x7f8354009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:06.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.741+0000 7f835a000640 1 -- 172.21.15.46:0/592384401 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f835c051eb0 con 0x7f836415e160 2024-03-29T08:25:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.937+0000 7f8363200640 1 -- 172.21.15.46:0/592384401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f836404e040 con 0x7f836415e160 2024-03-29T08:25:06.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.941+0000 7f835a000640 1 -- 172.21.15.46:0/592384401 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f835c0554d0 con 0x7f836415e160 2024-03-29T08:25:06.939 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:25:06.939 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/1862652020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1862652020},{"type":"v1","addr":"172.21.15.73:6836","nonce":1862652020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:24:43.476334+0000","modified":"2024-03-29T08:25:05.985073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7386},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7386":{"gid":7386,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":10,"addr":"172.21.15.46:6835/2051429508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2051429508},{"type":"v1","addr":"172.21.15.46:6835","nonce":2051429508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-29T08:25:06.940 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 62 2024-03-29T08:25:06.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.941+0000 7f8363200640 1 -- 172.21.15.46:0/592384401 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f83440666c0 msgr2=0x7f8344068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:06.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.941+0000 7f8363200640 1 --2- 172.21.15.46:0/592384401 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f83440666c0 0x7f8344068b80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8354000f70 tx=0x7f8354009040 comp rx=0 tx=0).stop 2024-03-29T08:25:06.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.941+0000 7f8363200640 1 -- 172.21.15.46:0/592384401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415e160 msgr2=0x7f83641f4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:06.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.941+0000 7f8363200640 1 --2- 172.21.15.46:0/592384401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415e160 0x7f83641f4720 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f835c007e40 tx=0x7f835c00e4b0 comp rx=0 tx=0).stop 2024-03-29T08:25:06.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.941+0000 7f8363200640 1 -- 172.21.15.46:0/592384401 shutdown_connections 2024-03-29T08:25:06.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.941+0000 7f8363200640 1 --2- 172.21.15.46:0/592384401 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f83440666c0 0x7f8344068b80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:06.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.941+0000 7f8363200640 1 --2- 172.21.15.46:0/592384401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f836415e160 0x7f83641f4720 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:06.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.941+0000 7f8363200640 1 --2- 172.21.15.46:0/592384401 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f836415a6b0 0x7f83640add70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:06.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.941+0000 7f8363200640 1 --2- 172.21.15.46:0/592384401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83640b0220 0x7f83640c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:06.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.941+0000 7f8363200640 1 -- 172.21.15.46:0/592384401 >> 172.21.15.46:0/592384401 conn(0x7f8364098a40 msgr2=0x7f836406d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:06.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.941+0000 7f8363200640 1 -- 172.21.15.46:0/592384401 shutdown_connections 2024-03-29T08:25:06.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:06.941+0000 7f8363200640 1 -- 172.21.15.46:0/592384401 wait complete. 2024-03-29T08:25:07.364 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:24:43.476334+0000', 'modified': '2024-03-29T08:25:05.985073+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 93, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7386}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7386': {'gid': 7386, 'name': 'c', 'rank': 0, 'incarnation': 59, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.46:6835/2051429508', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 2051429508}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 2051429508}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:25:07.364 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T08:25:07.365 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-29T08:25:07.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.513+0000 7f63bd800640 1 Processor -- start 2024-03-29T08:25:07.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.513+0000 7f63bd800640 1 -- start start 2024-03-29T08:25:07.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.513+0000 7f63bd800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f63b8099910 0x7f63b80cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.513+0000 7f63bd800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f63b8098a40 0x7f63b8098ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.513+0000 7f63bd800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b80ce8f0 con 0x7f63b815dc70 2024-03-29T08:25:07.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.513+0000 7f63bd800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63b806c3a0 con 0x7f63b8098a40 2024-03-29T08:25:07.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.513+0000 7f63bd800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b8099410 con 0x7f63b8099910 2024-03-29T08:25:07.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.513+0000 7f63bce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f63b8098a40 0x7f63b8098ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.513+0000 7f63bce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f63b8098a40 0x7f63b8098ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38354/0 (socket says 172.21.15.46:38354) 2024-03-29T08:25:07.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.513+0000 7f63bce00640 1 -- 172.21.15.46:0/1948426868 learned_addr learned my addr 172.21.15.46:0/1948426868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:07.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.513+0000 7f63af400640 1 --2- 172.21.15.46:0/1948426868 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f63b8099910 0x7f63b80cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.513+0000 7f63bce00640 1 -- 172.21.15.46:0/1948426868 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f63b8099910 msgr2=0x7f63b80cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.513+0000 7f63bce00640 1 --2- 172.21.15.46:0/1948426868 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f63b8099910 0x7f63b80cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.513+0000 7f63bce00640 1 -- 172.21.15.46:0/1948426868 >> v1:172.21.15.73:6789/0 conn(0x7f63b815dc70 legacy=0x7f63b806bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.513+0000 7f63bce00640 1 -- 172.21.15.46:0/1948426868 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b80cf450 con 0x7f63b8098a40 2024-03-29T08:25:07.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.517+0000 7f63bce00640 1 --2- 172.21.15.46:0/1948426868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8098a40 0x7f63b8098ed0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009e50 tx=0x7f63b001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9d5da0606aad6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.517+0000 7f63b7400640 1 -- 172.21.15.46:0/1948426868 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b00a5700 con 0x7f63b8098a40 2024-03-29T08:25:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.517+0000 7f63b7400640 1 -- 172.21.15.46:0/1948426868 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b00a5860 con 0x7f63b8098a40 2024-03-29T08:25:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.517+0000 7f63b7400640 1 -- 172.21.15.46:0/1948426868 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b00a5bf0 con 0x7f63b8098a40 2024-03-29T08:25:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.517+0000 7f63bd800640 1 -- 172.21.15.46:0/1948426868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8098a40 msgr2=0x7f63b8098ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.517+0000 7f63bd800640 1 --2- 172.21.15.46:0/1948426868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8098a40 0x7f63b8098ed0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009e50 tx=0x7f63b001ca30 comp rx=0 tx=0).stop 2024-03-29T08:25:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.517+0000 7f63bd800640 1 -- 172.21.15.46:0/1948426868 shutdown_connections 2024-03-29T08:25:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.517+0000 7f63bd800640 1 --2- 172.21.15.46:0/1948426868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8098a40 0x7f63b8098ed0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.517+0000 7f63bd800640 1 --2- 172.21.15.46:0/1948426868 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f63b8099910 0x7f63b80cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.517+0000 7f63bd800640 1 -- 172.21.15.46:0/1948426868 >> 172.21.15.46:0/1948426868 conn(0x7f63b80b1fe0 msgr2=0x7f63b80b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.517+0000 7f63bd800640 1 -- 172.21.15.46:0/1948426868 shutdown_connections 2024-03-29T08:25:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.517+0000 7f63bd800640 1 -- 172.21.15.46:0/1948426868 wait complete. 2024-03-29T08:25:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63bd800640 1 Processor -- start 2024-03-29T08:25:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63bd800640 1 -- start start 2024-03-29T08:25:07.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63bd800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63b8098a40 0x7f63b815a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63bd800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8099910 0x7f63b80b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63bd800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f63b815dc70 0x7f63b815c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63bd800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b80c66c0 con 0x7f63b8099910 2024-03-29T08:25:07.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63bd800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b80c5ab0 con 0x7f63b8098a40 2024-03-29T08:25:07.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63bd800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b80c5d90 con 0x7f63b815dc70 2024-03-29T08:25:07.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63af400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8099910 0x7f63b80b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63af400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8099910 0x7f63b80b59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38356/0 (socket says 172.21.15.46:38356) 2024-03-29T08:25:07.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63af400640 1 -- 172.21.15.46:0/2366062076 learned_addr learned my addr 172.21.15.46:0/2366062076 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:07.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63bce00640 1 --2- 172.21.15.46:0/2366062076 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f63b815dc70 0x7f63b815c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63b7e00640 1 --2- 172.21.15.46:0/2366062076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63b8098a40 0x7f63b815a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63af400640 1 -- 172.21.15.46:0/2366062076 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f63b815dc70 msgr2=0x7f63b815c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63af400640 1 --2- 172.21.15.46:0/2366062076 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f63b815dc70 0x7f63b815c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63af400640 1 -- 172.21.15.46:0/2366062076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63b8098a40 msgr2=0x7f63b815a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63af400640 1 --2- 172.21.15.46:0/2366062076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63b8098a40 0x7f63b815a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63af400640 1 -- 172.21.15.46:0/2366062076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b0009b30 con 0x7f63b8099910 2024-03-29T08:25:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63af400640 1 --2- 172.21.15.46:0/2366062076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8099910 0x7f63b80b59b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f63a000e930 tx=0x7f63a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63b4c00640 1 -- 172.21.15.46:0/2366062076 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0009690 con 0x7f63b8099910 2024-03-29T08:25:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63b4c00640 1 -- 172.21.15.46:0/2366062076 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a000ca00 con 0x7f63b8099910 2024-03-29T08:25:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63b4c00640 1 -- 172.21.15.46:0/2366062076 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a000cd70 con 0x7f63b8099910 2024-03-29T08:25:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63bd800640 1 -- 172.21.15.46:0/2366062076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b80c6070 con 0x7f63b8099910 2024-03-29T08:25:07.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.521+0000 7f63bd800640 1 -- 172.21.15.46:0/2366062076 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b815ac80 con 0x7f63b8099910 2024-03-29T08:25:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63b4c00640 1 -- 172.21.15.46:0/2366062076 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f63a0013470 con 0x7f63b8099910 2024-03-29T08:25:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63b4c00640 1 --2- 172.21.15.46:0/2366062076 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6394066690 0x7f6394068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63b4c00640 1 -- 172.21.15.46:0/2366062076 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f63a00844b0 con 0x7f63b8099910 2024-03-29T08:25:07.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63b7e00640 1 --2- 172.21.15.46:0/2366062076 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6394066690 0x7f6394068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63c2304000 1 Processor -- start 2024-03-29T08:25:07.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63c2304000 1 -- start start 2024-03-29T08:25:07.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63b7e00640 1 --2- 172.21.15.46:0/2366062076 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6394066690 0x7f6394068b50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f63a8000f70 tx=0x7f63a8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:07.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63c2304000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409b9db0 0x5641409ba190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63c2304000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409ba6d0 0x56414096ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63c2304000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5641409704c0 0x5641409708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63c2304000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641408385a0 con 0x5641409b9db0 2024-03-29T08:25:07.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63c2304000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641407c5af0 con 0x5641409ba6d0 2024-03-29T08:25:07.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63c2304000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5641407c30c0 con 0x5641409704c0 2024-03-29T08:25:07.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409b9db0 0x5641409ba190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409b9db0 0x5641409ba190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38366/0 (socket says 172.21.15.46:38366) 2024-03-29T08:25:07.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63b7e00640 1 -- 172.21.15.46:0/1896079339 learned_addr learned my addr 172.21.15.46:0/1896079339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:07.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63af400640 1 --2- 172.21.15.46:0/1896079339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409ba6d0 0x56414096ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.525+0000 7f63bce00640 1 --2- 172.21.15.46:0/1896079339 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5641409704c0 0x5641409708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63b7e00640 1 -- 172.21.15.46:0/1896079339 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5641409704c0 msgr2=0x5641409708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:07.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63b7e00640 1 --2- 172.21.15.46:0/1896079339 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5641409704c0 0x5641409708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63b7e00640 1 -- 172.21.15.46:0/1896079339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409ba6d0 msgr2=0x56414096ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63b7e00640 1 --2- 172.21.15.46:0/1896079339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409ba6d0 0x56414096ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63b7e00640 1 -- 172.21.15.46:0/1896079339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641404ba890 con 0x5641409b9db0 2024-03-29T08:25:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63b7e00640 1 --2- 172.21.15.46:0/1896079339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409b9db0 0x5641409ba190 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f63a80052a0 tx=0x7f63a8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af921d588543865d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63acc00640 1 -- 172.21.15.46:0/1896079339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a800e6e0 con 0x5641409b9db0 2024-03-29T08:25:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63acc00640 1 -- 172.21.15.46:0/1896079339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a800e840 con 0x5641409b9db0 2024-03-29T08:25:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63acc00640 1 -- 172.21.15.46:0/1896079339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a800ebb0 con 0x5641409b9db0 2024-03-29T08:25:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 1 -- 172.21.15.46:0/1896079339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409b9db0 msgr2=0x5641409ba190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 1 --2- 172.21.15.46:0/1896079339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409b9db0 0x5641409ba190 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f63a80052a0 tx=0x7f63a8011000 comp rx=0 tx=0).stop 2024-03-29T08:25:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 1 -- 172.21.15.46:0/1896079339 shutdown_connections 2024-03-29T08:25:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 1 --2- 172.21.15.46:0/1896079339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409ba6d0 0x56414096ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 1 --2- 172.21.15.46:0/1896079339 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5641409704c0 0x5641409708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 1 --2- 172.21.15.46:0/1896079339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409b9db0 0x5641409ba190 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 1 -- 172.21.15.46:0/1896079339 >> 172.21.15.46:0/1896079339 conn(0x564140976830 msgr2=0x56414096e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:07.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 1 -- 172.21.15.46:0/1896079339 shutdown_connections 2024-03-29T08:25:07.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 1 -- 172.21.15.46:0/1896079339 wait complete. 2024-03-29T08:25:07.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 1 -- start start 2024-03-29T08:25:07.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 1 Processor -- start 2024-03-29T08:25:07.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409704c0 0x564140974cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5641409b9db0 0x564140975210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409ba6d0 0x5641409baab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641407be9b0 con 0x5641409ba6d0 2024-03-29T08:25:07.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414075dc00 con 0x5641409704c0 2024-03-29T08:25:07.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x56414077c070 con 0x5641409b9db0 2024-03-29T08:25:07.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63c2304000 10 client.0 _ll_register_callbacks cb 0x56414095c0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:25:07.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63bce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409ba6d0 0x5641409baab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63bce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409ba6d0 0x5641409baab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38382/0 (socket says 172.21.15.46:38382) 2024-03-29T08:25:07.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63bce00640 1 -- 172.21.15.46:0/2259137193 learned_addr learned my addr 172.21.15.46:0/2259137193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:07.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63af400640 1 --2- 172.21.15.46:0/2259137193 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5641409b9db0 0x564140975210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.529+0000 7f63b7e00640 1 --2- 172.21.15.46:0/2259137193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409704c0 0x564140974cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63bce00640 1 -- 172.21.15.46:0/2259137193 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5641409b9db0 msgr2=0x564140975210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:07.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63bce00640 1 --2- 172.21.15.46:0/2259137193 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5641409b9db0 0x564140975210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63bce00640 1 -- 172.21.15.46:0/2259137193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409704c0 msgr2=0x564140974cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:07.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63bce00640 1 --2- 172.21.15.46:0/2259137193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409704c0 0x564140974cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63bce00640 1 -- 172.21.15.46:0/2259137193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a800f040 con 0x5641409ba6d0 2024-03-29T08:25:07.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63af400640 1 --2- 172.21.15.46:0/2259137193 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5641409b9db0 0x564140975210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:25:07.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63bce00640 1 --2- 172.21.15.46:0/2259137193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409ba6d0 0x5641409baab0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009e50 tx=0x7f63b001b270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87aaeb9fae3b21 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:07.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63a5600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:25:07.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63a5600640 1 -- 172.21.15.46:0/2259137193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b00a5cd0 con 0x5641409ba6d0 2024-03-29T08:25:07.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63a5600640 1 -- 172.21.15.46:0/2259137193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f63b00a5e30 con 0x5641409ba6d0 2024-03-29T08:25:07.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 1 -- 172.21.15.46:0/2259137193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56414077b5f0 con 0x5641409ba6d0 2024-03-29T08:25:07.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63a5600640 1 -- 172.21.15.46:0/2259137193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63b00c0400 con 0x5641409ba6d0 2024-03-29T08:25:07.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 10 client.7707 fetch_fsmap learned FSMap version 62 2024-03-29T08:25:07.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 1 -- 172.21.15.46:0/2259137193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x564140783ab0 con 0x5641409ba6d0 2024-03-29T08:25:07.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63a5600640 1 -- 172.21.15.46:0/2259137193 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56414077b5f0 con 0x5641409ba6d0 2024-03-29T08:25:07.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63a5600640 1 -- 172.21.15.46:0/2259137193 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f63b0005d20 con 0x5641409ba6d0 2024-03-29T08:25:07.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 10 client.7707 fetch_fsmap finished waiting for FSMap version 62 2024-03-29T08:25:07.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 10 client.7707 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:25:07.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 20 client.7707 populate_metadata read hostname 'smithi046' 2024-03-29T08:25:07.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 1 --2- 172.21.15.46:0/2259137193 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5641409bb4c0 0x5641409cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 4 client.7707 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:25:07.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 1 -- 172.21.15.46:0/2259137193 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641407bb790 con 0x5641409bb4c0 2024-03-29T08:25:07.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63b7e00640 1 --2- 172.21.15.46:0/2259137193 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5641409bb4c0 0x5641409cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63a5600640 10 client.7707 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:07.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63b7e00640 1 --2- 172.21.15.46:0/2259137193 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5641409bb4c0 0x5641409cae50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbc4129abc35d1a9 server_cookie=19f7a28939c4c86 in_seq=0 out_seq=0 2024-03-29T08:25:07.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63a5600640 1 -- 172.21.15.46:0/2259137193 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5641407bb790 con 0x5641409bb4c0 2024-03-29T08:25:07.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63a5600640 10 client.7707 handle_command_reply: tid=0 2024-03-29T08:25:07.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 1 client.7707 shutdown 2024-03-29T08:25:07.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 1 -- 172.21.15.46:0/2259137193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409ba6d0 msgr2=0x5641409baab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:07.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 1 --2- 172.21.15.46:0/2259137193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409ba6d0 0x5641409baab0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f63b0009e50 tx=0x7f63b001b270 comp rx=0 tx=0).stop 2024-03-29T08:25:07.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 1 -- 172.21.15.46:0/2259137193 shutdown_connections 2024-03-29T08:25:07.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 1 --2- 172.21.15.46:0/2259137193 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5641409bb4c0 0x5641409cae50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 1 --2- 172.21.15.46:0/2259137193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409ba6d0 0x5641409baab0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 1 --2- 172.21.15.46:0/2259137193 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5641409b9db0 0x564140975210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63a5600640 0 client.7707 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:07.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 1 --2- 172.21.15.46:0/2259137193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409704c0 0x564140974cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 1 -- 172.21.15.46:0/2259137193 >> 172.21.15.46:0/2259137193 conn(0x564140976830 msgr2=0x56414096ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:07.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 1 -- 172.21.15.46:0/2259137193 shutdown_connections 2024-03-29T08:25:07.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 1 -- 172.21.15.46:0/2259137193 wait complete. 2024-03-29T08:25:07.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.533+0000 7f63c2304000 20 client.7707 trim_cache size 0 max 16384 2024-03-29T08:25:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.549+0000 7f63c2304000 1 Processor -- start 2024-03-29T08:25:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.549+0000 7f63c2304000 1 -- start start 2024-03-29T08:25:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.549+0000 7f63c2304000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x564140976830 0x5641409b87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.549+0000 7f63c2304000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409704c0 0x5641409b8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.549+0000 7f63c2304000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409b9db0 0x5641409c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.549+0000 7f63c2304000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414047b940 con 0x5641409b9db0 2024-03-29T08:25:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.549+0000 7f63c2304000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641409c6620 con 0x5641409704c0 2024-03-29T08:25:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.549+0000 7f63c2304000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x56414075bc10 con 0x564140976830 2024-03-29T08:25:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63bce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409b9db0 0x5641409c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63bce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409b9db0 0x5641409c8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38398/0 (socket says 172.21.15.46:38398) 2024-03-29T08:25:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63bce00640 1 -- 172.21.15.46:0/1117382776 learned_addr learned my addr 172.21.15.46:0/1117382776 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:07.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63b7e00640 1 --2- 172.21.15.46:0/1117382776 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x564140976830 0x5641409b87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63af400640 1 --2- 172.21.15.46:0/1117382776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409704c0 0x5641409b8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63bce00640 1 -- 172.21.15.46:0/1117382776 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x564140976830 msgr2=0x5641409b87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63bce00640 1 --2- 172.21.15.46:0/1117382776 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x564140976830 0x5641409b87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63bce00640 1 -- 172.21.15.46:0/1117382776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409704c0 msgr2=0x5641409b8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63bce00640 1 --2- 172.21.15.46:0/1117382776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409704c0 0x5641409b8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63bce00640 1 -- 172.21.15.46:0/1117382776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564140769c00 con 0x5641409b9db0 2024-03-29T08:25:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63bce00640 1 --2- 172.21.15.46:0/1117382776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409b9db0 0x5641409c8e40 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f63b001c4a0 tx=0x7f63b00b8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3640f4f3ec20657b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63ad600640 1 -- 172.21.15.46:0/1117382776 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b00a53e0 con 0x5641409b9db0 2024-03-29T08:25:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63ad600640 1 -- 172.21.15.46:0/1117382776 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b00c0a60 con 0x5641409b9db0 2024-03-29T08:25:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63ad600640 1 -- 172.21.15.46:0/1117382776 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b00c06f0 con 0x5641409b9db0 2024-03-29T08:25:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 1 -- 172.21.15.46:0/1117382776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409b9db0 msgr2=0x5641409c8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 1 --2- 172.21.15.46:0/1117382776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409b9db0 0x5641409c8e40 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f63b001c4a0 tx=0x7f63b00b8040 comp rx=0 tx=0).stop 2024-03-29T08:25:07.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 1 -- 172.21.15.46:0/1117382776 shutdown_connections 2024-03-29T08:25:07.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 1 --2- 172.21.15.46:0/1117382776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409704c0 0x5641409b8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 1 --2- 172.21.15.46:0/1117382776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409b9db0 0x5641409c8e40 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 1 --2- 172.21.15.46:0/1117382776 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x564140976830 0x5641409b87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 1 -- 172.21.15.46:0/1117382776 >> 172.21.15.46:0/1117382776 conn(0x5641409c6760 msgr2=0x5641409749f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:07.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 1 -- 172.21.15.46:0/1117382776 shutdown_connections 2024-03-29T08:25:07.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 1 -- 172.21.15.46:0/1117382776 wait complete. 2024-03-29T08:25:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 1 -- start start 2024-03-29T08:25:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 1 Processor -- start 2024-03-29T08:25:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409704c0 0x5641409cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x564140976830 0x564140a139d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409b9db0 0x564140a13f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641407c29b0 con 0x5641409704c0 2024-03-29T08:25:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56414085b470 con 0x5641409b9db0 2024-03-29T08:25:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5641407c2220 con 0x564140976830 2024-03-29T08:25:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63c2304000 10 client.0 _ll_register_callbacks cb 0x564140783590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:25:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409704c0 0x5641409cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409704c0 0x5641409cbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38414/0 (socket says 172.21.15.46:38414) 2024-03-29T08:25:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63b7e00640 1 -- 172.21.15.46:0/301894992 learned_addr learned my addr 172.21.15.46:0/301894992 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63af400640 1 --2- 172.21.15.46:0/301894992 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x564140976830 0x564140a139d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63bce00640 1 --2- 172.21.15.46:0/301894992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409b9db0 0x564140a13f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63b7e00640 1 -- 172.21.15.46:0/301894992 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x564140976830 msgr2=0x564140a139d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:07.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63b7e00640 1 --2- 172.21.15.46:0/301894992 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x564140976830 0x564140a139d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.553+0000 7f63b7e00640 1 -- 172.21.15.46:0/301894992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409b9db0 msgr2=0x564140a13f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:07.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63b7e00640 1 --2- 172.21.15.46:0/301894992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409b9db0 0x564140a13f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63b7e00640 1 -- 172.21.15.46:0/301894992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564140783ab0 con 0x5641409704c0 2024-03-29T08:25:07.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63bce00640 1 --2- 172.21.15.46:0/301894992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409b9db0 0x564140a13f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:07.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63b7e00640 1 --2- 172.21.15.46:0/301894992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409704c0 0x5641409cbeb0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009010 tx=0x7f63a8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d3ca6cf9d1b75b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:07.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63a5600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:25:07.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63a5600640 1 -- 172.21.15.46:0/301894992 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a802e5d0 con 0x5641409704c0 2024-03-29T08:25:07.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63a5600640 1 -- 172.21.15.46:0/301894992 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f63a802e910 con 0x5641409704c0 2024-03-29T08:25:07.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 1 -- 172.21.15.46:0/301894992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56414078e460 con 0x5641409704c0 2024-03-29T08:25:07.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63a5600640 1 -- 172.21.15.46:0/301894992 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63a802ead0 con 0x5641409704c0 2024-03-29T08:25:07.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 10 client.7713 fetch_fsmap learned FSMap version 62 2024-03-29T08:25:07.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 1 -- 172.21.15.46:0/301894992 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x5641404bacc0 con 0x5641409704c0 2024-03-29T08:25:07.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63a5600640 1 -- 172.21.15.46:0/301894992 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56414078e460 con 0x5641409704c0 2024-03-29T08:25:07.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63a5600640 1 -- 172.21.15.46:0/301894992 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f63a802ec30 con 0x5641409704c0 2024-03-29T08:25:07.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 10 client.7713 fetch_fsmap finished waiting for FSMap version 62 2024-03-29T08:25:07.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 10 client.7713 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:25:07.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 20 client.7713 populate_metadata read hostname 'smithi046' 2024-03-29T08:25:07.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 1 --2- 172.21.15.46:0/301894992 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5641409ba6d0 0x564140a20bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 4 client.7713 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T08:25:07.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 1 -- 172.21.15.46:0/301894992 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5641407879c0 con 0x5641409ba6d0 2024-03-29T08:25:07.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63af400640 1 --2- 172.21.15.46:0/301894992 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5641409ba6d0 0x564140a20bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63a5600640 10 client.7713 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:07.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63af400640 1 --2- 172.21.15.46:0/301894992 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5641409ba6d0 0x564140a20bd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=117f66ba40657503 server_cookie=7acb088de412e93f in_seq=0 out_seq=0 2024-03-29T08:25:07.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63a5600640 1 -- 172.21.15.46:0/301894992 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5641407879c0 con 0x5641409ba6d0 2024-03-29T08:25:07.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63a5600640 10 client.7713 handle_command_reply: tid=0 2024-03-29T08:25:07.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 1 client.7713 shutdown 2024-03-29T08:25:07.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 1 -- 172.21.15.46:0/301894992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409704c0 msgr2=0x5641409cbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:07.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 1 --2- 172.21.15.46:0/301894992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409704c0 0x5641409cbeb0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009010 tx=0x7f63a8013000 comp rx=0 tx=0).stop 2024-03-29T08:25:07.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 1 -- 172.21.15.46:0/301894992 shutdown_connections 2024-03-29T08:25:07.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 1 --2- 172.21.15.46:0/301894992 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5641409ba6d0 0x564140a20bd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 1 --2- 172.21.15.46:0/301894992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5641409b9db0 0x564140a13f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 1 --2- 172.21.15.46:0/301894992 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x564140976830 0x564140a139d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 1 --2- 172.21.15.46:0/301894992 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5641409704c0 0x5641409cbeb0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63a5600640 0 client.7713 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:07.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 1 -- 172.21.15.46:0/301894992 >> 172.21.15.46:0/301894992 conn(0x5641409c6760 msgr2=0x5641409b7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:07.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 1 -- 172.21.15.46:0/301894992 shutdown_connections 2024-03-29T08:25:07.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 1 -- 172.21.15.46:0/301894992 wait complete. 2024-03-29T08:25:07.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63c2304000 20 client.7713 trim_cache size 0 max 16384 2024-03-29T08:25:07.564 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:25:07.564 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "e9b7960f-ec1f-4787-95c3-c806094d37f0", 2024-03-29T08:25:07.564 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-03-29T08:25:07.564 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 7386, 2024-03-29T08:25:07.564 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-29T08:25:07.564 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-29T08:25:07.564 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-29T08:25:07.564 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 4.5821335870000004, 2024-03-29T08:25:07.564 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 62, 2024-03-29T08:25:07.565 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 93, 2024-03-29T08:25:07.565 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 93, 2024-03-29T08:25:07.565 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 29.316422396 2024-03-29T08:25:07.565 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:25:07.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63bd800640 1 -- 172.21.15.46:0/2366062076 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6394066690 msgr2=0x7f6394068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:07.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63bd800640 1 --2- 172.21.15.46:0/2366062076 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6394066690 0x7f6394068b50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f63a8000f70 tx=0x7f63a8009040 comp rx=0 tx=0).stop 2024-03-29T08:25:07.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63bd800640 1 -- 172.21.15.46:0/2366062076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8099910 msgr2=0x7f63b80b59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:07.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63bd800640 1 --2- 172.21.15.46:0/2366062076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8099910 0x7f63b80b59b0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f63a000e930 tx=0x7f63a000edf0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63bd800640 1 -- 172.21.15.46:0/2366062076 shutdown_connections 2024-03-29T08:25:07.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63bd800640 1 --2- 172.21.15.46:0/2366062076 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6394066690 0x7f6394068b50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63bd800640 1 --2- 172.21.15.46:0/2366062076 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f63b815dc70 0x7f63b815c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63bd800640 1 --2- 172.21.15.46:0/2366062076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f63b8099910 0x7f63b80b59b0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63bd800640 1 --2- 172.21.15.46:0/2366062076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63b8098a40 0x7f63b815a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63bd800640 1 -- 172.21.15.46:0/2366062076 >> 172.21.15.46:0/2366062076 conn(0x7f63b80b1fe0 msgr2=0x7f63b80cc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:07.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63bd800640 1 -- 172.21.15.46:0/2366062076 shutdown_connections 2024-03-29T08:25:07.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.557+0000 7f63bd800640 1 -- 172.21.15.46:0/2366062076 wait complete. 2024-03-29T08:25:07.577 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:25:07.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.717+0000 7f4b87600640 1 Processor -- start 2024-03-29T08:25:07.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.717+0000 7f4b87600640 1 -- start start 2024-03-29T08:25:07.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.717+0000 7f4b87600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b800ccd10 0x7f4b80098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.717+0000 7f4b87600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4b8015cdb0 0x7f4b80098f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.717+0000 7f4b87600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b800d0030 con 0x7f4b8015c9d0 2024-03-29T08:25:07.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.717+0000 7f4b87600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b800cf420 con 0x7f4b8015cdb0 2024-03-29T08:25:07.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.717+0000 7f4b87600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b800cf700 con 0x7f4b800ccd10 2024-03-29T08:25:07.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.717+0000 7f4b84c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4b8015c9d0 0x7f4b800cea20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35394/0 (socket says 172.21.15.46:35394) 2024-03-29T08:25:07.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.717+0000 7f4b84c00640 1 -- 172.21.15.46:0/2990821253 learned_addr learned my addr 172.21.15.46:0/2990821253 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:25:07.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.717+0000 7f4b85600640 1 --2- 172.21.15.46:0/2990821253 >> v2:172.21.15.46:3300/0 conn(0x7f4b8015cdb0 0x7f4b80098f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.721+0000 7f4b77e00640 1 --2- 172.21.15.46:0/2990821253 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b800ccd10 0x7f4b80098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.721+0000 7f4b7fe00640 1 -- 172.21.15.46:0/2990821253 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145286840 0 0) 0x7f4b800d0030 con 0x7f4b8015c9d0 2024-03-29T08:25:07.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.721+0000 7f4b7fe00640 1 -- 172.21.15.46:0/2990821253 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b64003530 con 0x7f4b8015c9d0 2024-03-29T08:25:07.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.721+0000 7f4b85600640 1 -- 172.21.15.46:0/2990821253 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b800ccd10 msgr2=0x7f4b80098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:07.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.721+0000 7f4b85600640 1 --2- 172.21.15.46:0/2990821253 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b800ccd10 0x7f4b80098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.721+0000 7f4b85600640 1 -- 172.21.15.46:0/2990821253 >> v1:172.21.15.46:6789/0 conn(0x7f4b8015c9d0 legacy=0x7f4b800cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:07.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.721+0000 7f4b85600640 1 -- 172.21.15.46:0/2990821253 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b800cf9e0 con 0x7f4b8015cdb0 2024-03-29T08:25:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.721+0000 7f4b85600640 1 --2- 172.21.15.46:0/2990821253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b8015cdb0 0x7f4b80098f80 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009d90 tx=0x7f4b7801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32354297c4ae926 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.721+0000 7f4b7fe00640 1 -- 172.21.15.46:0/2990821253 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b780a59e0 con 0x7f4b8015cdb0 2024-03-29T08:25:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.721+0000 7f4b7fe00640 1 -- 172.21.15.46:0/2990821253 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b780a5b40 con 0x7f4b8015cdb0 2024-03-29T08:25:07.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.721+0000 7f4b7fe00640 1 -- 172.21.15.46:0/2990821253 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b780ad440 con 0x7f4b8015cdb0 2024-03-29T08:25:07.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b87600640 1 -- 172.21.15.46:0/2990821253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b8015cdb0 msgr2=0x7f4b80098f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:07.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b87600640 1 --2- 172.21.15.46:0/2990821253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b8015cdb0 0x7f4b80098f80 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f4b78009d90 tx=0x7f4b7801c940 comp rx=0 tx=0).stop 2024-03-29T08:25:07.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b87600640 1 -- 172.21.15.46:0/2990821253 shutdown_connections 2024-03-29T08:25:07.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b87600640 1 --2- 172.21.15.46:0/2990821253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b8015cdb0 0x7f4b80098f80 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b87600640 1 --2- 172.21.15.46:0/2990821253 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b800ccd10 0x7f4b80098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b87600640 1 -- 172.21.15.46:0/2990821253 >> 172.21.15.46:0/2990821253 conn(0x7f4b800b21a0 msgr2=0x7f4b800b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:07.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b87600640 1 -- 172.21.15.46:0/2990821253 shutdown_connections 2024-03-29T08:25:07.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b87600640 1 -- 172.21.15.46:0/2990821253 wait complete. 2024-03-29T08:25:07.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b87600640 1 Processor -- start 2024-03-29T08:25:07.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b87600640 1 -- start start 2024-03-29T08:25:07.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b87600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b800ccd10 0x7f4b8014e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b87600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b8015c9d0 0x7f4b800c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b87600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b8015cdb0 0x7f4b8014dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b87600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b800b4c60 con 0x7f4b8015c9d0 2024-03-29T08:25:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b87600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b800b4f40 con 0x7f4b8015cdb0 2024-03-29T08:25:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b87600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b800b5220 con 0x7f4b800ccd10 2024-03-29T08:25:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b8015c9d0 0x7f4b800c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b77e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b8015c9d0 0x7f4b800c4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38432/0 (socket says 172.21.15.46:38432) 2024-03-29T08:25:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b77e00640 1 -- 172.21.15.46:0/2553700471 learned_addr learned my addr 172.21.15.46:0/2553700471 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b84c00640 1 --2- 172.21.15.46:0/2553700471 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b800ccd10 0x7f4b8014e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b85600640 1 --2- 172.21.15.46:0/2553700471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b8015cdb0 0x7f4b8014dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b77e00640 1 -- 172.21.15.46:0/2553700471 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b800ccd10 msgr2=0x7f4b8014e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b77e00640 1 --2- 172.21.15.46:0/2553700471 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b800ccd10 0x7f4b8014e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b77e00640 1 -- 172.21.15.46:0/2553700471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b8015cdb0 msgr2=0x7f4b8014dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b77e00640 1 --2- 172.21.15.46:0/2553700471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b8015cdb0 0x7f4b8014dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b77e00640 1 -- 172.21.15.46:0/2553700471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b78009a70 con 0x7f4b8015c9d0 2024-03-29T08:25:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b85600640 1 --2- 172.21.15.46:0/2553700471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b8015cdb0 0x7f4b8014dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.725+0000 7f4b77e00640 1 --2- 172.21.15.46:0/2553700471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b8015c9d0 0x7f4b800c4190 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800d800 tx=0x7f4b6800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.729+0000 7f4b7d600640 1 -- 172.21.15.46:0/2553700471 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6800b6c0 con 0x7f4b8015c9d0 2024-03-29T08:25:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.729+0000 7f4b7d600640 1 -- 172.21.15.46:0/2553700471 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6800b820 con 0x7f4b8015c9d0 2024-03-29T08:25:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.729+0000 7f4b7d600640 1 -- 172.21.15.46:0/2553700471 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6800bb90 con 0x7f4b8015c9d0 2024-03-29T08:25:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.729+0000 7f4b87600640 1 -- 172.21.15.46:0/2553700471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b8015dd90 con 0x7f4b8015c9d0 2024-03-29T08:25:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.729+0000 7f4b87600640 1 -- 172.21.15.46:0/2553700471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b800c46d0 con 0x7f4b8015c9d0 2024-03-29T08:25:07.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.729+0000 7f4b87600640 1 -- 172.21.15.46:0/2553700471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b8015dba0 con 0x7f4b8015c9d0 2024-03-29T08:25:07.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.729+0000 7f4b7d600640 1 -- 172.21.15.46:0/2553700471 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4b6800bcf0 con 0x7f4b8015c9d0 2024-03-29T08:25:07.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.729+0000 7f4b7d600640 1 --2- 172.21.15.46:0/2553700471 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4b5c066690 0x7f4b5c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:07.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.729+0000 7f4b7d600640 1 -- 172.21.15.46:0/2553700471 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4b68084700 con 0x7f4b8015c9d0 2024-03-29T08:25:07.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.729+0000 7f4b84c00640 1 --2- 172.21.15.46:0/2553700471 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4b5c066690 0x7f4b5c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:07.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.733+0000 7f4b84c00640 1 --2- 172.21.15.46:0/2553700471 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4b5c066690 0x7f4b5c068b50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4b70007fd0 tx=0x7f4b70009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:07.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.733+0000 7f4b7d600640 1 -- 172.21.15.46:0/2553700471 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4b68051a00 con 0x7f4b8015c9d0 2024-03-29T08:25:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.933+0000 7f4b87600640 1 -- 172.21.15.46:0/2553700471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4b80066c90 con 0x7f4b8015c9d0 2024-03-29T08:25:07.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.933+0000 7f4b7d600640 1 -- 172.21.15.46:0/2553700471 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4b68055020 con 0x7f4b8015c9d0 2024-03-29T08:25:07.932 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:25:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.933+0000 7f4b87600640 1 -- 172.21.15.46:0/2553700471 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4b5c066690 msgr2=0x7f4b5c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.933+0000 7f4b87600640 1 --2- 172.21.15.46:0/2553700471 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4b5c066690 0x7f4b5c068b50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4b70007fd0 tx=0x7f4b70009040 comp rx=0 tx=0).stop 2024-03-29T08:25:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.933+0000 7f4b87600640 1 -- 172.21.15.46:0/2553700471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b8015c9d0 msgr2=0x7f4b800c4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.933+0000 7f4b87600640 1 --2- 172.21.15.46:0/2553700471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b8015c9d0 0x7f4b800c4190 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800d800 tx=0x7f4b6800dcc0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.933+0000 7f4b87600640 1 -- 172.21.15.46:0/2553700471 shutdown_connections 2024-03-29T08:25:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.933+0000 7f4b87600640 1 --2- 172.21.15.46:0/2553700471 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4b5c066690 0x7f4b5c068b50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.933+0000 7f4b87600640 1 --2- 172.21.15.46:0/2553700471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b8015cdb0 0x7f4b8014dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.933+0000 7f4b87600640 1 --2- 172.21.15.46:0/2553700471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4b8015c9d0 0x7f4b800c4190 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.933+0000 7f4b87600640 1 --2- 172.21.15.46:0/2553700471 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4b800ccd10 0x7f4b8014e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.933+0000 7f4b87600640 1 -- 172.21.15.46:0/2553700471 >> 172.21.15.46:0/2553700471 conn(0x7f4b800b21a0 msgr2=0x7f4b8015bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.933+0000 7f4b87600640 1 -- 172.21.15.46:0/2553700471 shutdown_connections 2024-03-29T08:25:07.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:07.933+0000 7f4b87600640 1 -- 172.21.15.46:0/2553700471 wait complete. 2024-03-29T08:25:07.953 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:25:07.953 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:25:07.953 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-29T08:25:07.953 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T08:25:07.953 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:25:07.953 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:25:07.953 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:25:07.953 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:25:07.953 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:25:07.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:07.954 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-29T08:25:07.963 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:25:07.963 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:25:07.963 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:25:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:25:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:25:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:25:07.964 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:25:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:25:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-29T08:25:07.964 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:25:07.964 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:25:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-29T08:25:07.965 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:25:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:25:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-29T08:25:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T08:25:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42029sec preferred_lft 42029sec 2024-03-29T08:25:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-29T08:25:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:25:07.965 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:25:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:25:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-29T08:25:07.965 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:25:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:d9:58:09:4c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:25:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:25:07.965 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:25:07.966 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:25:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T08:25:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T08:25:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:25:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T08:25:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:25:07.966 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:25:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:25:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T08:25:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:25:07.966 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:25:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T08:25:07.966 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-29T08:25:07.967 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:25:07.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:07.967 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:25:08.029 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:25:08.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:08.032 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-29T08:25:08.088 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-29T08:25:08.088 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:25:08.088 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T08:25:08.089 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:25:08.089 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T08:25:08.089 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:25:08.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:08.089 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:25:08.089 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:25:08.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:25:08.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:25:08.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:25:08.089 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:25:08.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:08.181 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:25:08.190 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T08:25:08.190 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T08:25:08.191 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:25:08.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:08.191 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-03-29T08:25:08.255 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T08:25:08.255 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:25:08.255 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T08:25:08.307 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:25:08.307 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:25:08.307 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:25:08.358 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:25:08.419 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:25:08.420 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:25:08.481 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:25:08.483 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:25:08.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:08.483 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:25:08.539 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:25:08.539 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T08:25:08.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:25:08.607 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:25:08.607 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:25:08.669 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:25:08.671 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:25:08.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:08.671 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:25:08.680 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-03-29T08:25:08.681+0000 7efd5e89b580 -1 init, newargv = 0x55b64ccb8110 newargc=15 2024-03-29T08:25:08.681 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[45319]: starting ceph client 2024-03-29T08:25:08.687 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[45319]: starting fuse 2024-03-29T08:25:08.689 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:25:08.690 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-29T08:25:08.690 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:25:08.690 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-03-29T08:25:08.690 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-03-29T08:25:08.690 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:25:08.690 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-03-29T08:25:08.690 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:25:08.690 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-03-29T08:25:08.690 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T08:25:08.690 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-03-29T08:25:08.691 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:25:08.691 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:25:08.691 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-03-29T08:25:08.691 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-03-29T08:25:08.691 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-03-29T08:25:08.691 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-03-29T08:25:08.691 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:25:08.691 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-03-29T08:25:08.691 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:25:08.691 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:25:08.691 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:25:08.691 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-03-29T08:25:08.691 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-03-29T08:25:08.691 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-03-29T08:25:08.691 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:25:08.691 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:25:08.691 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-03-29T08:25:08.692 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:25:08.812 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.45319.asok 2024-03-29T08:25:08.996 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45319.asok 2024-03-29T08:25:08.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:08.996 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45319.asok status 2024-03-29T08:25:09.069 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:25:09.070 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-03-29T08:25:09.069+0000 7f279e400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:25:09.072 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:25:09.171 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:25:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-03-29T08:25:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:25:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:25:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-03-29T08:25:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-03-29T08:25:09.171 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T08:25:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "45319", 2024-03-29T08:25:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-03-29T08:25:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:25:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-03-29T08:25:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-03-29T08:25:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 7728, 2024-03-29T08:25:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-03-29T08:25:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-03-29T08:25:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-03-29T08:25:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 7728 2024-03-29T08:25:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:25:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:25:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:25:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:25:09.172 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1524863510 2024-03-29T08:25:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:25:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:25:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:25:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:25:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:25:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1524863510 2024-03-29T08:25:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:25:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.7728 192.168.0.1:0/1524863510", 2024-03-29T08:25:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/1524863510", 2024-03-29T08:25:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-03-29T08:25:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 62, 2024-03-29T08:25:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 93, 2024-03-29T08:25:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:25:09.173 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-03-29T08:25:09.174 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-03-29T08:25:09.174 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:25:09.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:09.187 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-03-29T08:25:09.187 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:25:09.196 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-03-29T08:25:09.196 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T08:25:09.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:09.196 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:25:09.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:09.306 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-03-29T08:25:09.364 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-03-29T08:25:09.365+0000 7f2799e00640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-03-29T08:25:09.365 INFO:teuthology.orchestra.run.smithi046.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-03-29T08:25:09.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:25:09.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:09.366 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-03-29T08:25:09.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.549+0000 7fb18e600640 1 Processor -- start 2024-03-29T08:25:09.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.549+0000 7fb18e600640 1 -- start start 2024-03-29T08:25:09.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.553+0000 7fb18e600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb1880b1780 0x7fb1880cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.553+0000 7fb18e600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb1880cfc10 0x7fb188098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.553+0000 7fb18e600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1880cea60 con 0x7fb188160060 2024-03-29T08:25:09.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.553+0000 7fb18e600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18806bea0 con 0x7fb1880cfc10 2024-03-29T08:25:09.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.553+0000 7fb18e600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1880ced00 con 0x7fb1880b1780 2024-03-29T08:25:09.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.553+0000 7fb187400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb188160060 0x7fb1880cf500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35404/0 (socket says 172.21.15.46:35404) 2024-03-29T08:25:09.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.553+0000 7fb187400640 1 -- 172.21.15.46:0/347511695 learned_addr learned my addr 172.21.15.46:0/347511695 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:25:09.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.553+0000 7fb187e00640 1 --2- 172.21.15.46:0/347511695 >> v2:172.21.15.46:3300/0 conn(0x7fb1880cfc10 0x7fb188098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.553+0000 7fb17fe00640 1 --2- 172.21.15.46:0/347511695 >> v2:172.21.15.73:3300/0 conn(0x7fb1880b1780 0x7fb1880cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.553+0000 7fb187e00640 1 -- 172.21.15.46:0/347511695 >> v2:172.21.15.73:3300/0 conn(0x7fb1880b1780 msgr2=0x7fb1880cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.553+0000 7fb187e00640 1 --2- 172.21.15.46:0/347511695 >> v2:172.21.15.73:3300/0 conn(0x7fb1880b1780 0x7fb1880cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.553+0000 7fb187e00640 1 -- 172.21.15.46:0/347511695 >> v1:172.21.15.46:6789/0 conn(0x7fb188160060 legacy=0x7fb1880cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.553+0000 7fb187e00640 1 -- 172.21.15.46:0/347511695 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1880cefe0 con 0x7fb1880cfc10 2024-03-29T08:25:09.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.553+0000 7fb186a00640 1 -- 172.21.15.46:0/347511695 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 272728395 0 0) 0x7fb1880cea60 con 0x7fb188160060 2024-03-29T08:25:09.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb187e00640 1 --2- 172.21.15.46:0/347511695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1880cfc10 0x7fb188098a40 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb188071a50 tx=0x7fb18001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff26e5d0a90919c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb186a00640 1 -- 172.21.15.46:0/347511695 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1800a5790 con 0x7fb1880cfc10 2024-03-29T08:25:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb186a00640 1 -- 172.21.15.46:0/347511695 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1800a58f0 con 0x7fb1880cfc10 2024-03-29T08:25:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb186a00640 1 -- 172.21.15.46:0/347511695 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1800a5c80 con 0x7fb1880cfc10 2024-03-29T08:25:09.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb18e600640 1 -- 172.21.15.46:0/347511695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1880cfc10 msgr2=0x7fb188098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb18e600640 1 --2- 172.21.15.46:0/347511695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1880cfc10 0x7fb188098a40 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb188071a50 tx=0x7fb18001c8a0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb18e600640 1 -- 172.21.15.46:0/347511695 shutdown_connections 2024-03-29T08:25:09.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb18e600640 1 --2- 172.21.15.46:0/347511695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1880cfc10 0x7fb188098a40 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb18e600640 1 --2- 172.21.15.46:0/347511695 >> v2:172.21.15.73:3300/0 conn(0x7fb1880b1780 0x7fb1880cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb18e600640 1 -- 172.21.15.46:0/347511695 >> 172.21.15.46:0/347511695 conn(0x7fb1880b21b0 msgr2=0x7fb1880b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:09.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb18e600640 1 -- 172.21.15.46:0/347511695 shutdown_connections 2024-03-29T08:25:09.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb18e600640 1 -- 172.21.15.46:0/347511695 wait complete. 2024-03-29T08:25:09.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb18e600640 1 Processor -- start 2024-03-29T08:25:09.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb18e600640 1 -- start start 2024-03-29T08:25:09.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb18e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1880b1780 0x7fb18815b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb18e600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1880cfc10 0x7fb188154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb18e600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb188160060 0x7fb188154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb18e600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1881551f0 con 0x7fb1880b1780 2024-03-29T08:25:09.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb18e600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1880c4290 con 0x7fb1880cfc10 2024-03-29T08:25:09.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb18e600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1880c4500 con 0x7fb188160060 2024-03-29T08:25:09.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.557+0000 7fb187400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1880b1780 0x7fb18815b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb187400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1880b1780 0x7fb18815b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38460/0 (socket says 172.21.15.46:38460) 2024-03-29T08:25:09.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb187400640 1 -- 172.21.15.46:0/2976193219 learned_addr learned my addr 172.21.15.46:0/2976193219 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:09.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb17fe00640 1 --2- 172.21.15.46:0/2976193219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1880cfc10 0x7fb188154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb187e00640 1 --2- 172.21.15.46:0/2976193219 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb188160060 0x7fb188154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb187400640 1 -- 172.21.15.46:0/2976193219 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb188160060 msgr2=0x7fb188154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb187400640 1 --2- 172.21.15.46:0/2976193219 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb188160060 0x7fb188154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb187400640 1 -- 172.21.15.46:0/2976193219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1880cfc10 msgr2=0x7fb188154770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb187400640 1 --2- 172.21.15.46:0/2976193219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1880cfc10 0x7fb188154770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb187400640 1 -- 172.21.15.46:0/2976193219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb180009a70 con 0x7fb1880b1780 2024-03-29T08:25:09.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb17fe00640 1 --2- 172.21.15.46:0/2976193219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1880cfc10 0x7fb188154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:09.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb187400640 1 --2- 172.21.15.46:0/2976193219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1880b1780 0x7fb18815b940 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fb17800c4b0 tx=0x7fb17800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:09.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb17f400640 1 -- 172.21.15.46:0/2976193219 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1780074e0 con 0x7fb1880b1780 2024-03-29T08:25:09.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb17f400640 1 -- 172.21.15.46:0/2976193219 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb178007660 con 0x7fb1880b1780 2024-03-29T08:25:09.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb17f400640 1 -- 172.21.15.46:0/2976193219 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb178004a30 con 0x7fb1880b1780 2024-03-29T08:25:09.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb18e600640 1 -- 172.21.15.46:0/2976193219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1880c47e0 con 0x7fb1880b1780 2024-03-29T08:25:09.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb18e600640 1 -- 172.21.15.46:0/2976193219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18815be80 con 0x7fb1880b1780 2024-03-29T08:25:09.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb17f400640 1 -- 172.21.15.46:0/2976193219 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb178004b90 con 0x7fb1880b1780 2024-03-29T08:25:09.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb17f400640 1 --2- 172.21.15.46:0/2976193219 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb164066690 0x7fb164068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.561+0000 7fb17f400640 1 -- 172.21.15.46:0/2976193219 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb178083330 con 0x7fb1880b1780 2024-03-29T08:25:09.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb17fe00640 1 --2- 172.21.15.46:0/2976193219 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb164066690 0x7fb164068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 Processor -- start 2024-03-29T08:25:09.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 -- start start 2024-03-29T08:25:09.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb17fe00640 1 --2- 172.21.15.46:0/2976193219 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb164066690 0x7fb164068b50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb18806e3e0 tx=0x7fb170008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:09.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf013820 0x5624cf013c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf014140 0x5624cefca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cefcae50 0x5624cefcb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624cee93100 con 0x5624cf013820 2024-03-29T08:25:09.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624cee20d10 con 0x5624cefcae50 2024-03-29T08:25:09.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624cee1e2e0 con 0x5624cf014140 2024-03-29T08:25:09.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb187400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf013820 0x5624cf013c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb187400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf013820 0x5624cf013c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38474/0 (socket says 172.21.15.46:38474) 2024-03-29T08:25:09.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb187400640 1 -- 172.21.15.46:0/3596602573 learned_addr learned my addr 172.21.15.46:0/3596602573 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:09.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb17fe00640 1 --2- 172.21.15.46:0/3596602573 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf014140 0x5624cefca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb187e00640 1 --2- 172.21.15.46:0/3596602573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cefcae50 0x5624cefcb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb187400640 1 -- 172.21.15.46:0/3596602573 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf014140 msgr2=0x5624cefca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb187400640 1 --2- 172.21.15.46:0/3596602573 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf014140 0x5624cefca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb187400640 1 -- 172.21.15.46:0/3596602573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cefcae50 msgr2=0x5624cefcb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb187400640 1 --2- 172.21.15.46:0/3596602573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cefcae50 0x5624cefcb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb187400640 1 -- 172.21.15.46:0/3596602573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624ceb15fb0 con 0x5624cf013820 2024-03-29T08:25:09.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb187e00640 1 --2- 172.21.15.46:0/3596602573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cefcae50 0x5624cefcb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:25:09.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb187400640 1 --2- 172.21.15.46:0/3596602573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf013820 0x5624cf013c00 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb17805c1b0 tx=0x7fb17800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3311ed566fa29630 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:09.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb177e00640 1 -- 172.21.15.46:0/3596602573 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb178069020 con 0x5624cf013820 2024-03-29T08:25:09.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb177e00640 1 -- 172.21.15.46:0/3596602573 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17805c6e0 con 0x5624cf013820 2024-03-29T08:25:09.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb177e00640 1 -- 172.21.15.46:0/3596602573 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb178001070 con 0x5624cf013820 2024-03-29T08:25:09.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 -- 172.21.15.46:0/3596602573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf013820 msgr2=0x5624cf013c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 --2- 172.21.15.46:0/3596602573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf013820 0x5624cf013c00 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb17805c1b0 tx=0x7fb17800f000 comp rx=0 tx=0).stop 2024-03-29T08:25:09.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 -- 172.21.15.46:0/3596602573 shutdown_connections 2024-03-29T08:25:09.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 --2- 172.21.15.46:0/3596602573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cefcae50 0x5624cefcb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 --2- 172.21.15.46:0/3596602573 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf014140 0x5624cefca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 --2- 172.21.15.46:0/3596602573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf013820 0x5624cf013c00 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 -- 172.21.15.46:0/3596602573 >> 172.21.15.46:0/3596602573 conn(0x5624cedc9280 msgr2=0x5624cefc91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:09.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 -- 172.21.15.46:0/3596602573 shutdown_connections 2024-03-29T08:25:09.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 -- 172.21.15.46:0/3596602573 wait complete. 2024-03-29T08:25:09.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.565+0000 7fb191a72000 1 -- start start 2024-03-29T08:25:09.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb191a72000 1 Processor -- start 2024-03-29T08:25:09.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb191a72000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cefcae50 0x5624cefd59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb191a72000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cf013820 0x5624cf01bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb191a72000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf014140 0x5624cf01c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb191a72000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624cee19bd0 con 0x5624cefcae50 2024-03-29T08:25:09.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb191a72000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624cedb8f60 con 0x5624cf013820 2024-03-29T08:25:09.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb191a72000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624cedd7180 con 0x5624cf014140 2024-03-29T08:25:09.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb191a72000 10 client.0 _ll_register_callbacks cb 0x5624cee13240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:25:09.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb187400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cefcae50 0x5624cefd59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb187400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cefcae50 0x5624cefd59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38476/0 (socket says 172.21.15.46:38476) 2024-03-29T08:25:09.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb187400640 1 -- 172.21.15.46:0/4153894675 learned_addr learned my addr 172.21.15.46:0/4153894675 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:09.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb17fe00640 1 --2- 172.21.15.46:0/4153894675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cf013820 0x5624cf01bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb187e00640 1 --2- 172.21.15.46:0/4153894675 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf014140 0x5624cf01c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb187400640 1 -- 172.21.15.46:0/4153894675 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf014140 msgr2=0x5624cf01c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb187400640 1 --2- 172.21.15.46:0/4153894675 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf014140 0x5624cf01c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb187400640 1 -- 172.21.15.46:0/4153894675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cf013820 msgr2=0x5624cf01bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb187400640 1 --2- 172.21.15.46:0/4153894675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cf013820 0x5624cf01bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb187400640 1 -- 172.21.15.46:0/4153894675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18815be80 con 0x5624cefcae50 2024-03-29T08:25:09.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb17fe00640 1 --2- 172.21.15.46:0/4153894675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cf013820 0x5624cf01bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:09.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb187e00640 1 --2- 172.21.15.46:0/4153894675 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf014140 0x5624cf01c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:09.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb187400640 1 --2- 172.21.15.46:0/4153894675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cefcae50 0x5624cefd59d0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb178053dc0 tx=0x7fb178000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17599768186055aa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:09.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb174c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:25:09.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb174c00640 1 -- 172.21.15.46:0/4153894675 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb178069070 con 0x5624cefcae50 2024-03-29T08:25:09.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb191a72000 1 -- 172.21.15.46:0/4153894675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624cede9850 con 0x5624cefcae50 2024-03-29T08:25:09.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb174c00640 1 -- 172.21.15.46:0/4153894675 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb17805c6e0 con 0x5624cefcae50 2024-03-29T08:25:09.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb174c00640 1 -- 172.21.15.46:0/4153894675 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb178001050 con 0x5624cefcae50 2024-03-29T08:25:09.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb191a72000 10 client.7743 fetch_fsmap learned FSMap version 62 2024-03-29T08:25:09.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb191a72000 1 -- 172.21.15.46:0/4153894675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x5624ceddebc0 con 0x5624cefcae50 2024-03-29T08:25:09.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb174c00640 1 -- 172.21.15.46:0/4153894675 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb178076080 con 0x5624cefcae50 2024-03-29T08:25:09.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb174c00640 1 -- 172.21.15.46:0/4153894675 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb1780720d0 con 0x5624cefcae50 2024-03-29T08:25:09.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb191a72000 10 client.7743 fetch_fsmap finished waiting for FSMap version 62 2024-03-29T08:25:09.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb191a72000 10 client.7743 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:25:09.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb191a72000 20 client.7743 populate_metadata read hostname 'smithi046' 2024-03-29T08:25:09.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb191a72000 1 --2- 172.21.15.46:0/4153894675 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5624cefd0ae0 0x5624cefd0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb191a72000 4 client.7743 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:25:09.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb191a72000 1 -- 172.21.15.46:0/4153894675 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624cee169b0 con 0x5624cefd0ae0 2024-03-29T08:25:09.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb17fe00640 1 --2- 172.21.15.46:0/4153894675 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5624cefd0ae0 0x5624cefd0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb17fe00640 1 --2- 172.21.15.46:0/4153894675 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5624cefd0ae0 0x5624cefd0ef0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5e9b0850ae3ccd2 server_cookie=4ccb4fa23c884d7 in_seq=0 out_seq=0 2024-03-29T08:25:09.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb174c00640 10 client.7743 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:09.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb174c00640 1 -- 172.21.15.46:0/4153894675 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5624cee169b0 con 0x5624cefd0ae0 2024-03-29T08:25:09.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.569+0000 7fb174c00640 10 client.7743 handle_command_reply: tid=0 2024-03-29T08:25:09.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.573+0000 7fb191a72000 1 client.7743 shutdown 2024-03-29T08:25:09.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.573+0000 7fb191a72000 1 -- 172.21.15.46:0/4153894675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cefcae50 msgr2=0x5624cefd59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.573+0000 7fb191a72000 1 --2- 172.21.15.46:0/4153894675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cefcae50 0x5624cefd59d0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb178053dc0 tx=0x7fb178000ee0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.573+0000 7fb191a72000 1 -- 172.21.15.46:0/4153894675 shutdown_connections 2024-03-29T08:25:09.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.573+0000 7fb191a72000 1 --2- 172.21.15.46:0/4153894675 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5624cefd0ae0 0x5624cefd0ef0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.573+0000 7fb191a72000 1 --2- 172.21.15.46:0/4153894675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cf013820 0x5624cf01bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.573+0000 7fb191a72000 1 --2- 172.21.15.46:0/4153894675 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf014140 0x5624cf01c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.573+0000 7fb191a72000 1 --2- 172.21.15.46:0/4153894675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cefcae50 0x5624cefd59d0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.573+0000 7fb174c00640 0 client.7743 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:09.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.573+0000 7fb191a72000 1 -- 172.21.15.46:0/4153894675 >> 172.21.15.46:0/4153894675 conn(0x5624cedc9280 msgr2=0x5624cefc98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:09.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.573+0000 7fb191a72000 1 -- 172.21.15.46:0/4153894675 shutdown_connections 2024-03-29T08:25:09.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.573+0000 7fb191a72000 1 -- 172.21.15.46:0/4153894675 wait complete. 2024-03-29T08:25:09.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.573+0000 7fb191a72000 20 client.7743 trim_cache size 0 max 16384 2024-03-29T08:25:09.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.585+0000 7fb191a72000 1 Processor -- start 2024-03-29T08:25:09.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.585+0000 7fb191a72000 1 -- start start 2024-03-29T08:25:09.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.585+0000 7fb191a72000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cefcae50 0x5624cf06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf013820 0x5624cf06df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf014140 0x5624cf06e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624cead6c30 con 0x5624cf014140 2024-03-29T08:25:09.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624cee5d2b0 con 0x5624cefcae50 2024-03-29T08:25:09.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624cede9fb0 con 0x5624cf013820 2024-03-29T08:25:09.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb187e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf014140 0x5624cf06e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb187e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf014140 0x5624cf06e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38478/0 (socket says 172.21.15.46:38478) 2024-03-29T08:25:09.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb187e00640 1 -- 172.21.15.46:0/2097221170 learned_addr learned my addr 172.21.15.46:0/2097221170 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:09.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb187400640 1 --2- 172.21.15.46:0/2097221170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cefcae50 0x5624cf06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb17fe00640 1 --2- 172.21.15.46:0/2097221170 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf013820 0x5624cf06df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb187e00640 1 -- 172.21.15.46:0/2097221170 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf013820 msgr2=0x5624cf06df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb187e00640 1 --2- 172.21.15.46:0/2097221170 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf013820 0x5624cf06df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb187e00640 1 -- 172.21.15.46:0/2097221170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cefcae50 msgr2=0x5624cf06d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb187e00640 1 --2- 172.21.15.46:0/2097221170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cefcae50 0x5624cf06d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb187e00640 1 -- 172.21.15.46:0/2097221170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624cedc4f10 con 0x5624cf014140 2024-03-29T08:25:09.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb187400640 1 --2- 172.21.15.46:0/2097221170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cefcae50 0x5624cf06d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:25:09.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb17fe00640 1 --2- 172.21.15.46:0/2097221170 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf013820 0x5624cf06df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:25:09.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb187e00640 1 --2- 172.21.15.46:0/2097221170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf014140 0x5624cf06e440 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fb1800194b0 tx=0x7fb180004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b592bd0896893b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:09.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb17cc00640 1 -- 172.21.15.46:0/2097221170 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1800a5c60 con 0x5624cf014140 2024-03-29T08:25:09.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb17cc00640 1 -- 172.21.15.46:0/2097221170 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1800a5dc0 con 0x5624cf014140 2024-03-29T08:25:09.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb17cc00640 1 -- 172.21.15.46:0/2097221170 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb180003ba0 con 0x5624cf014140 2024-03-29T08:25:09.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 -- 172.21.15.46:0/2097221170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf014140 msgr2=0x5624cf06e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 --2- 172.21.15.46:0/2097221170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf014140 0x5624cf06e440 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fb1800194b0 tx=0x7fb180004ac0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 -- 172.21.15.46:0/2097221170 shutdown_connections 2024-03-29T08:25:09.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 --2- 172.21.15.46:0/2097221170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf014140 0x5624cf06e440 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 --2- 172.21.15.46:0/2097221170 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf013820 0x5624cf06df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 --2- 172.21.15.46:0/2097221170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cefcae50 0x5624cf06d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 -- 172.21.15.46:0/2097221170 >> 172.21.15.46:0/2097221170 conn(0x5624cedc9280 msgr2=0x5624cf000270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:09.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 -- 172.21.15.46:0/2097221170 shutdown_connections 2024-03-29T08:25:09.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 -- 172.21.15.46:0/2097221170 wait complete. 2024-03-29T08:25:09.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 -- start start 2024-03-29T08:25:09.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 Processor -- start 2024-03-29T08:25:09.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cefcae50 0x5624cf020950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf013820 0x5624cf020e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf014140 0x5624cf0213d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624ceda3e80 con 0x5624cf014140 2024-03-29T08:25:09.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624cee1ae60 con 0x5624cefcae50 2024-03-29T08:25:09.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5624cedc8cc0 con 0x5624cf013820 2024-03-29T08:25:09.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.589+0000 7fb191a72000 10 client.0 _ll_register_callbacks cb 0x5624cedca7a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:25:09.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb187e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf014140 0x5624cf0213d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb187e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf014140 0x5624cf0213d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38484/0 (socket says 172.21.15.46:38484) 2024-03-29T08:25:09.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb187e00640 1 -- 172.21.15.46:0/1251676524 learned_addr learned my addr 172.21.15.46:0/1251676524 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:09.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb187400640 1 --2- 172.21.15.46:0/1251676524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cefcae50 0x5624cf020950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb17fe00640 1 --2- 172.21.15.46:0/1251676524 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf013820 0x5624cf020e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb187e00640 1 -- 172.21.15.46:0/1251676524 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf013820 msgr2=0x5624cf020e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb187e00640 1 --2- 172.21.15.46:0/1251676524 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf013820 0x5624cf020e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb187e00640 1 -- 172.21.15.46:0/1251676524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cefcae50 msgr2=0x5624cf020950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb187e00640 1 --2- 172.21.15.46:0/1251676524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cefcae50 0x5624cf020950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb187e00640 1 -- 172.21.15.46:0/1251676524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1880cefe0 con 0x5624cf014140 2024-03-29T08:25:09.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb187e00640 1 --2- 172.21.15.46:0/1251676524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf014140 0x5624cf0213d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fb1800194b0 tx=0x7fb18001b0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b637ac49294431d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:09.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb174c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:25:09.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb174c00640 1 -- 172.21.15.46:0/1251676524 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1800039a0 con 0x5624cf014140 2024-03-29T08:25:09.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 1 -- 172.21.15.46:0/1251676524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624cefcdaf0 con 0x5624cf014140 2024-03-29T08:25:09.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb174c00640 1 -- 172.21.15.46:0/1251676524 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb180003b00 con 0x5624cf014140 2024-03-29T08:25:09.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb174c00640 1 -- 172.21.15.46:0/1251676524 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb180003cc0 con 0x5624cf014140 2024-03-29T08:25:09.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 10 client.7749 fetch_fsmap learned FSMap version 62 2024-03-29T08:25:09.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 1 -- 172.21.15.46:0/1251676524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x5624cedb8a80 con 0x5624cf014140 2024-03-29T08:25:09.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb174c00640 1 -- 172.21.15.46:0/1251676524 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5624cefcdaf0 con 0x5624cf014140 2024-03-29T08:25:09.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb174c00640 1 -- 172.21.15.46:0/1251676524 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb180004040 con 0x5624cf014140 2024-03-29T08:25:09.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 10 client.7749 fetch_fsmap finished waiting for FSMap version 62 2024-03-29T08:25:09.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 10 client.7749 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:25:09.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 20 client.7749 populate_metadata read hostname 'smithi046' 2024-03-29T08:25:09.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 1 --2- 172.21.15.46:0/1251676524 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5624cf021910 0x5624cf021d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 4 client.7749 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-03-29T08:25:09.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 1 -- 172.21.15.46:0/1251676524 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) v1 -- 0x5624cede9570 con 0x5624cf021910 2024-03-29T08:25:09.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb187400640 1 --2- 172.21.15.46:0/1251676524 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5624cf021910 0x5624cf021d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb187400640 1 --2- 172.21.15.46:0/1251676524 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5624cf021910 0x5624cf021d20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c87d1fd87b7a5c39 server_cookie=6dcfa134fd847134 in_seq=0 out_seq=0 2024-03-29T08:25:09.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb174c00640 10 client.7749 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:09.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb174c00640 1 -- 172.21.15.46:0/1251676524 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5624cede9570 con 0x5624cf021910 2024-03-29T08:25:09.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb174c00640 10 client.7749 handle_command_reply: tid=0 2024-03-29T08:25:09.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 1 client.7749 shutdown 2024-03-29T08:25:09.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 1 -- 172.21.15.46:0/1251676524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf014140 msgr2=0x5624cf0213d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 1 --2- 172.21.15.46:0/1251676524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf014140 0x5624cf0213d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fb1800194b0 tx=0x7fb18001b0e0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 1 -- 172.21.15.46:0/1251676524 shutdown_connections 2024-03-29T08:25:09.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 1 --2- 172.21.15.46:0/1251676524 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5624cf021910 0x5624cf021d20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 1 --2- 172.21.15.46:0/1251676524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624cf014140 0x5624cf0213d0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 1 --2- 172.21.15.46:0/1251676524 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5624cf013820 0x5624cf020e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb174c00640 0 client.7749 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:09.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 1 --2- 172.21.15.46:0/1251676524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5624cefcae50 0x5624cf020950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 1 -- 172.21.15.46:0/1251676524 >> 172.21.15.46:0/1251676524 conn(0x5624cedc9280 msgr2=0x5624cf000b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:09.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 1 -- 172.21.15.46:0/1251676524 shutdown_connections 2024-03-29T08:25:09.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 1 -- 172.21.15.46:0/1251676524 wait complete. 2024-03-29T08:25:09.600 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-03-29T08:25:09.593+0000 7f2799e00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-03-29T08:24:51.226697+0000 1=1+0) n(v0 rc2024-03-29T08:24:51.337857+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x563bf804a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-29T08:24:51.226697+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-29T08:24:51.337857+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-03-29T08:25:09.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.593+0000 7fb191a72000 20 client.7749 trim_cache size 0 max 16384 2024-03-29T08:25:09.601 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:25:09.601 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-29T08:25:09.601 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "bbe297e5-75eb-4da6-b33c-e15e126044d2", 2024-03-29T08:25:09.601 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-29T08:25:09.601 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:25:09.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.597+0000 7fb18e600640 1 -- 172.21.15.46:0/2976193219 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb164066690 msgr2=0x7fb164068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.597+0000 7fb18e600640 1 --2- 172.21.15.46:0/2976193219 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb164066690 0x7fb164068b50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb18806e3e0 tx=0x7fb170008040 comp rx=0 tx=0).stop 2024-03-29T08:25:09.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.597+0000 7fb18e600640 1 -- 172.21.15.46:0/2976193219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1880b1780 msgr2=0x7fb18815b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.597+0000 7fb18e600640 1 --2- 172.21.15.46:0/2976193219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1880b1780 0x7fb18815b940 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fb17800c4b0 tx=0x7fb17800c970 comp rx=0 tx=0).stop 2024-03-29T08:25:09.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.597+0000 7fb18e600640 1 -- 172.21.15.46:0/2976193219 shutdown_connections 2024-03-29T08:25:09.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.597+0000 7fb18e600640 1 --2- 172.21.15.46:0/2976193219 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb164066690 0x7fb164068b50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.597+0000 7fb18e600640 1 --2- 172.21.15.46:0/2976193219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1880cfc10 0x7fb188154770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.597+0000 7fb18e600640 1 --2- 172.21.15.46:0/2976193219 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb188160060 0x7fb188154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.597+0000 7fb18e600640 1 --2- 172.21.15.46:0/2976193219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1880b1780 0x7fb18815b940 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.597+0000 7fb18e600640 1 -- 172.21.15.46:0/2976193219 >> 172.21.15.46:0/2976193219 conn(0x7fb1880b21b0 msgr2=0x7fb1880cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:09.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.597+0000 7fb18e600640 1 -- 172.21.15.46:0/2976193219 shutdown_connections 2024-03-29T08:25:09.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.597+0000 7fb18e600640 1 -- 172.21.15.46:0/2976193219 wait complete. 2024-03-29T08:25:09.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:09.614 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-29T08:25:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.757+0000 7f6d90400640 1 Processor -- start 2024-03-29T08:25:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.757+0000 7f6d90400640 1 -- start start 2024-03-29T08:25:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.757+0000 7f6d90400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6d8815df90 0x7f6d880ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.757+0000 7f6d90400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d880cbea0 0x7f6d880ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.757+0000 7f6d90400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8815d260 con 0x7f6d88142a50 2024-03-29T08:25:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.757+0000 7f6d90400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d880aed00 con 0x7f6d880cbea0 2024-03-29T08:25:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.757+0000 7f6d90400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d880aefd0 con 0x7f6d8815df90 2024-03-29T08:25:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.761+0000 7f6d8e400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d880cbea0 0x7f6d880ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.761+0000 7f6d8e400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6d880cbea0 0x7f6d880ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38500/0 (socket says 172.21.15.46:38500) 2024-03-29T08:25:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.761+0000 7f6d8e400640 1 -- 172.21.15.46:0/1242708283 learned_addr learned my addr 172.21.15.46:0/1242708283 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.761+0000 7f6d8d000640 1 --2- 172.21.15.46:0/1242708283 >> v2:172.21.15.73:3300/0 conn(0x7f6d8815df90 0x7f6d880ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.761+0000 7f6d8e400640 1 -- 172.21.15.46:0/1242708283 >> v2:172.21.15.73:3300/0 conn(0x7f6d8815df90 msgr2=0x7f6d880ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.761+0000 7f6d8e400640 1 --2- 172.21.15.46:0/1242708283 >> v2:172.21.15.73:3300/0 conn(0x7f6d8815df90 0x7f6d880ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.761+0000 7f6d8e400640 1 -- 172.21.15.46:0/1242708283 >> v1:172.21.15.73:6789/0 conn(0x7f6d88142a50 legacy=0x7f6d880cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.761+0000 7f6d8e400640 1 -- 172.21.15.46:0/1242708283 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8815d4b0 con 0x7f6d880cbea0 2024-03-29T08:25:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.761+0000 7f6d8e400640 1 --2- 172.21.15.46:0/1242708283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880cbea0 0x7f6d880ae7c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f6d8806e480 tx=0x7f6d8001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148287abb9289269 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.761+0000 7f6d87e00640 1 -- 172.21.15.46:0/1242708283 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d800a5600 con 0x7f6d880cbea0 2024-03-29T08:25:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.761+0000 7f6d87e00640 1 -- 172.21.15.46:0/1242708283 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d800a5760 con 0x7f6d880cbea0 2024-03-29T08:25:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.761+0000 7f6d87e00640 1 -- 172.21.15.46:0/1242708283 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d800a5af0 con 0x7f6d880cbea0 2024-03-29T08:25:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d90400640 1 -- 172.21.15.46:0/1242708283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880cbea0 msgr2=0x7f6d880ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d90400640 1 --2- 172.21.15.46:0/1242708283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880cbea0 0x7f6d880ae7c0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f6d8806e480 tx=0x7f6d8001c960 comp rx=0 tx=0).stop 2024-03-29T08:25:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d90400640 1 -- 172.21.15.46:0/1242708283 shutdown_connections 2024-03-29T08:25:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d90400640 1 --2- 172.21.15.46:0/1242708283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880cbea0 0x7f6d880ae7c0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d90400640 1 --2- 172.21.15.46:0/1242708283 >> v2:172.21.15.73:3300/0 conn(0x7f6d8815df90 0x7f6d880ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d90400640 1 -- 172.21.15.46:0/1242708283 >> 172.21.15.46:0/1242708283 conn(0x7f6d8806a390 msgr2=0x7f6d880cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d90400640 1 -- 172.21.15.46:0/1242708283 shutdown_connections 2024-03-29T08:25:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d90400640 1 -- 172.21.15.46:0/1242708283 wait complete. 2024-03-29T08:25:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d90400640 1 Processor -- start 2024-03-29T08:25:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d90400640 1 -- start start 2024-03-29T08:25:09.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d90400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880cbea0 0x7f6d8815f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d90400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d88142a50 0x7f6d88154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d90400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6d8815df90 0x7f6d8813fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d90400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8815e410 con 0x7f6d880cbea0 2024-03-29T08:25:09.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d90400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8815e6b0 con 0x7f6d88142a50 2024-03-29T08:25:09.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d90400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8815e990 con 0x7f6d8815df90 2024-03-29T08:25:09.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d8da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880cbea0 0x7f6d8815f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d8da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880cbea0 0x7f6d8815f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38512/0 (socket says 172.21.15.46:38512) 2024-03-29T08:25:09.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d8da00640 1 -- 172.21.15.46:0/79498999 learned_addr learned my addr 172.21.15.46:0/79498999 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:09.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d8e400640 1 --2- 172.21.15.46:0/79498999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6d8815df90 0x7f6d8813fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d8d000640 1 --2- 172.21.15.46:0/79498999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d88142a50 0x7f6d88154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d8da00640 1 -- 172.21.15.46:0/79498999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6d8815df90 msgr2=0x7f6d8813fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d8da00640 1 --2- 172.21.15.46:0/79498999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6d8815df90 0x7f6d8813fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d8da00640 1 -- 172.21.15.46:0/79498999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d88142a50 msgr2=0x7f6d88154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d8da00640 1 --2- 172.21.15.46:0/79498999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d88142a50 0x7f6d88154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d8da00640 1 -- 172.21.15.46:0/79498999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d80009b30 con 0x7f6d880cbea0 2024-03-29T08:25:09.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d8d000640 1 --2- 172.21.15.46:0/79498999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d88142a50 0x7f6d88154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:09.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.765+0000 7f6d8da00640 1 --2- 172.21.15.46:0/79498999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880cbea0 0x7f6d8815f190 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00c500 tx=0x7f6d7c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:09.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.769+0000 7f6d85600640 1 -- 172.21.15.46:0/79498999 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c005220 con 0x7f6d880cbea0 2024-03-29T08:25:09.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.769+0000 7f6d85600640 1 -- 172.21.15.46:0/79498999 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7c005380 con 0x7f6d880cbea0 2024-03-29T08:25:09.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.769+0000 7f6d85600640 1 -- 172.21.15.46:0/79498999 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c007270 con 0x7f6d880cbea0 2024-03-29T08:25:09.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.769+0000 7f6d90400640 1 -- 172.21.15.46:0/79498999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8815ec70 con 0x7f6d880cbea0 2024-03-29T08:25:09.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.769+0000 7f6d90400640 1 -- 172.21.15.46:0/79498999 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d88155450 con 0x7f6d880cbea0 2024-03-29T08:25:09.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.769+0000 7f6d85600640 1 -- 172.21.15.46:0/79498999 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6d7c0073d0 con 0x7f6d880cbea0 2024-03-29T08:25:09.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.769+0000 7f6d85600640 1 --2- 172.21.15.46:0/79498999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6d680666c0 0x7f6d68068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.769+0000 7f6d85600640 1 -- 172.21.15.46:0/79498999 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6d7c083a90 con 0x7f6d880cbea0 2024-03-29T08:25:09.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.769+0000 7f6d8d000640 1 --2- 172.21.15.46:0/79498999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6d680666c0 0x7f6d68068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.769+0000 7f6d937ac000 1 Processor -- start 2024-03-29T08:25:09.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.769+0000 7f6d937ac000 1 -- start start 2024-03-29T08:25:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d8d000640 1 --2- 172.21.15.46:0/79498999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6d680666c0 0x7f6d68068b80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6d881595b0 tx=0x7f6d74008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d937ac000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f257e60 0x55b95f258240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d937ac000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f258780 0x55b95f20b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d937ac000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f20be10 0x55b95f20c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d937ac000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95f0d2630 con 0x55b95f257e60 2024-03-29T08:25:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d937ac000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95f05fbd0 con 0x55b95f20be10 2024-03-29T08:25:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d937ac000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b95f05d1a0 con 0x55b95f258780 2024-03-29T08:25:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d8da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f257e60 0x55b95f258240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d8da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f257e60 0x55b95f258240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38528/0 (socket says 172.21.15.46:38528) 2024-03-29T08:25:09.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d8da00640 1 -- 172.21.15.46:0/198725224 learned_addr learned my addr 172.21.15.46:0/198725224 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d8d000640 1 --2- 172.21.15.46:0/198725224 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f258780 0x55b95f20b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d8e400640 1 --2- 172.21.15.46:0/198725224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f20be10 0x55b95f20c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d8da00640 1 -- 172.21.15.46:0/198725224 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f258780 msgr2=0x55b95f20b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d8da00640 1 --2- 172.21.15.46:0/198725224 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f258780 0x55b95f20b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d8da00640 1 -- 172.21.15.46:0/198725224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f20be10 msgr2=0x55b95f20c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d8da00640 1 --2- 172.21.15.46:0/198725224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f20be10 0x55b95f20c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d8da00640 1 -- 172.21.15.46:0/198725224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b95ed54e20 con 0x55b95f257e60 2024-03-29T08:25:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d8da00640 1 --2- 172.21.15.46:0/198725224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f257e60 0x55b95f258240 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c000fc0 tx=0x7f6d7c007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21fed3f8ba221580 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d7a000640 1 -- 172.21.15.46:0/198725224 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c058800 con 0x55b95f257e60 2024-03-29T08:25:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d7a000640 1 -- 172.21.15.46:0/198725224 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7c05cdc0 con 0x55b95f257e60 2024-03-29T08:25:09.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d7a000640 1 -- 172.21.15.46:0/198725224 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c0510e0 con 0x55b95f257e60 2024-03-29T08:25:09.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d937ac000 1 -- 172.21.15.46:0/198725224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f257e60 msgr2=0x55b95f258240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d937ac000 1 --2- 172.21.15.46:0/198725224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f257e60 0x55b95f258240 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c000fc0 tx=0x7f6d7c007680 comp rx=0 tx=0).stop 2024-03-29T08:25:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d937ac000 1 -- 172.21.15.46:0/198725224 shutdown_connections 2024-03-29T08:25:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d937ac000 1 --2- 172.21.15.46:0/198725224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f20be10 0x55b95f20c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d937ac000 1 --2- 172.21.15.46:0/198725224 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f258780 0x55b95f20b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d937ac000 1 --2- 172.21.15.46:0/198725224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f257e60 0x55b95f258240 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d937ac000 1 -- 172.21.15.46:0/198725224 >> 172.21.15.46:0/198725224 conn(0x55b95f008160 msgr2=0x55b95f236820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d937ac000 1 -- 172.21.15.46:0/198725224 shutdown_connections 2024-03-29T08:25:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d937ac000 1 -- 172.21.15.46:0/198725224 wait complete. 2024-03-29T08:25:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d937ac000 1 -- start start 2024-03-29T08:25:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.773+0000 7f6d937ac000 1 Processor -- start 2024-03-29T08:25:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f20be10 0x55b95f25f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f257e60 0x55b95f25f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f258780 0x55b95f23ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95f058a90 con 0x55b95f20be10 2024-03-29T08:25:09.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95eff7d40 con 0x55b95f258780 2024-03-29T08:25:09.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b95f015f30 con 0x55b95f257e60 2024-03-29T08:25:09.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 10 client.0 _ll_register_callbacks cb 0x55b95f0ba810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:25:09.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d8da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f20be10 0x55b95f25f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d8da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f20be10 0x55b95f25f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38538/0 (socket says 172.21.15.46:38538) 2024-03-29T08:25:09.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d8da00640 1 -- 172.21.15.46:0/709368913 learned_addr learned my addr 172.21.15.46:0/709368913 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:09.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d8d000640 1 --2- 172.21.15.46:0/709368913 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f257e60 0x55b95f25f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d8e400640 1 --2- 172.21.15.46:0/709368913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f258780 0x55b95f23ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d8da00640 1 -- 172.21.15.46:0/709368913 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f257e60 msgr2=0x55b95f25f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d8da00640 1 --2- 172.21.15.46:0/709368913 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f257e60 0x55b95f25f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d8da00640 1 -- 172.21.15.46:0/709368913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f258780 msgr2=0x55b95f23ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d8da00640 1 --2- 172.21.15.46:0/709368913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f258780 0x55b95f23ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d8da00640 1 -- 172.21.15.46:0/709368913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d88155450 con 0x55b95f20be10 2024-03-29T08:25:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d8da00640 1 --2- 172.21.15.46:0/709368913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f20be10 0x55b95f25f1c0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c000fc0 tx=0x7f6d7c000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e2a254aa95cd58 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:25:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d6ea00640 1 -- 172.21.15.46:0/709368913 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c058800 con 0x55b95f20be10 2024-03-29T08:25:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d6ea00640 1 -- 172.21.15.46:0/709368913 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6d7c071170 con 0x55b95f20be10 2024-03-29T08:25:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 -- 172.21.15.46:0/709368913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b95f0573e0 con 0x55b95f20be10 2024-03-29T08:25:09.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d6ea00640 1 -- 172.21.15.46:0/709368913 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d7c071330 con 0x55b95f20be10 2024-03-29T08:25:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 10 client.7761 fetch_fsmap learned FSMap version 62 2024-03-29T08:25:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 -- 172.21.15.46:0/709368913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55b95f01daa0 con 0x55b95f20be10 2024-03-29T08:25:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d6ea00640 1 -- 172.21.15.46:0/709368913 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b95f0573e0 con 0x55b95f20be10 2024-03-29T08:25:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d6ea00640 1 -- 172.21.15.46:0/709368913 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6d7c019030 con 0x55b95f20be10 2024-03-29T08:25:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 10 client.7761 fetch_fsmap finished waiting for FSMap version 62 2024-03-29T08:25:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 10 client.7761 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:25:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 20 client.7761 populate_metadata read hostname 'smithi046' 2024-03-29T08:25:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 --2- 172.21.15.46:0/709368913 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55b95f23b9d0 0x55b95f23bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 4 client.7761 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:25:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 -- 172.21.15.46:0/709368913 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b95f028d00 con 0x55b95f23b9d0 2024-03-29T08:25:09.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d8d000640 1 --2- 172.21.15.46:0/709368913 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55b95f23b9d0 0x55b95f23bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d8d000640 1 --2- 172.21.15.46:0/709368913 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55b95f23b9d0 0x55b95f23bde0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20212b38c2b50c16 server_cookie=3ceff3e86730920 in_seq=0 out_seq=0 2024-03-29T08:25:09.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d6ea00640 10 client.7761 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:09.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d6ea00640 1 -- 172.21.15.46:0/709368913 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b95f028d00 con 0x55b95f23b9d0 2024-03-29T08:25:09.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d6ea00640 10 client.7761 handle_command_reply: tid=0 2024-03-29T08:25:09.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 client.7761 shutdown 2024-03-29T08:25:09.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 -- 172.21.15.46:0/709368913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f20be10 msgr2=0x55b95f25f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 --2- 172.21.15.46:0/709368913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f20be10 0x55b95f25f1c0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c000fc0 tx=0x7f6d7c000ee0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 -- 172.21.15.46:0/709368913 shutdown_connections 2024-03-29T08:25:09.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 --2- 172.21.15.46:0/709368913 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55b95f23b9d0 0x55b95f23bde0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 --2- 172.21.15.46:0/709368913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f258780 0x55b95f23ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 --2- 172.21.15.46:0/709368913 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f257e60 0x55b95f25f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 --2- 172.21.15.46:0/709368913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f20be10 0x55b95f25f1c0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d6ea00640 0 client.7761 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 -- 172.21.15.46:0/709368913 >> 172.21.15.46:0/709368913 conn(0x55b95f008160 msgr2=0x55b95f206bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 -- 172.21.15.46:0/709368913 shutdown_connections 2024-03-29T08:25:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 1 -- 172.21.15.46:0/709368913 wait complete. 2024-03-29T08:25:09.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.777+0000 7f6d937ac000 20 client.7761 trim_cache size 0 max 16384 2024-03-29T08:25:09.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d937ac000 1 Processor -- start 2024-03-29T08:25:09.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d937ac000 1 -- start start 2024-03-29T08:25:09.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d937ac000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f20be10 0x55b95f2b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d937ac000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f257e60 0x55b95f2b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d937ac000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f258780 0x55b95f2b2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d937ac000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95ed15aa0 con 0x55b95f258780 2024-03-29T08:25:09.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d937ac000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95f060e60 con 0x55b95f20be10 2024-03-29T08:25:09.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d937ac000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b95f05a020 con 0x55b95f257e60 2024-03-29T08:25:09.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d8e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f258780 0x55b95f2b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d8e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f258780 0x55b95f2b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38550/0 (socket says 172.21.15.46:38550) 2024-03-29T08:25:09.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d8e400640 1 -- 172.21.15.46:0/2758397407 learned_addr learned my addr 172.21.15.46:0/2758397407 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:09.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d8da00640 1 --2- 172.21.15.46:0/2758397407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f20be10 0x55b95f2b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d8d000640 1 --2- 172.21.15.46:0/2758397407 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f257e60 0x55b95f2b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d8e400640 1 -- 172.21.15.46:0/2758397407 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f257e60 msgr2=0x55b95f2b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d8e400640 1 --2- 172.21.15.46:0/2758397407 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f257e60 0x55b95f2b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d8e400640 1 -- 172.21.15.46:0/2758397407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f20be10 msgr2=0x55b95f2b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d8e400640 1 --2- 172.21.15.46:0/2758397407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f20be10 0x55b95f2b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.793+0000 7f6d8e400640 1 -- 172.21.15.46:0/2758397407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b95f003df0 con 0x55b95f258780 2024-03-29T08:25:09.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d8e400640 1 --2- 172.21.15.46:0/2758397407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f258780 0x55b95f2b2640 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f6d8001a590 tx=0x7f6d800a7070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebe582aca24e536 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d7aa00640 1 -- 172.21.15.46:0/2758397407 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d800a58d0 con 0x55b95f258780 2024-03-29T08:25:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d7aa00640 1 -- 172.21.15.46:0/2758397407 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d800a5a30 con 0x55b95f258780 2024-03-29T08:25:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d7aa00640 1 -- 172.21.15.46:0/2758397407 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d800a5dc0 con 0x55b95f258780 2024-03-29T08:25:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 -- 172.21.15.46:0/2758397407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f258780 msgr2=0x55b95f2b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 --2- 172.21.15.46:0/2758397407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f258780 0x55b95f2b2640 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f6d8001a590 tx=0x7f6d800a7070 comp rx=0 tx=0).stop 2024-03-29T08:25:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 -- 172.21.15.46:0/2758397407 shutdown_connections 2024-03-29T08:25:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 --2- 172.21.15.46:0/2758397407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f258780 0x55b95f2b2640 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 --2- 172.21.15.46:0/2758397407 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f257e60 0x55b95f2b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 --2- 172.21.15.46:0/2758397407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f20be10 0x55b95f2b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 -- 172.21.15.46:0/2758397407 >> 172.21.15.46:0/2758397407 conn(0x55b95f008160 msgr2=0x55b95f239820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 -- 172.21.15.46:0/2758397407 shutdown_connections 2024-03-29T08:25:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 -- 172.21.15.46:0/2758397407 wait complete. 2024-03-29T08:25:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 -- start start 2024-03-29T08:25:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 Processor -- start 2024-03-29T08:25:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f20be10 0x55b95f2b3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f257e60 0x55b95f20e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f258780 0x55b95f20e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95f201930 con 0x55b95f257e60 2024-03-29T08:25:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b95f111ec0 con 0x55b95f20be10 2024-03-29T08:25:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b95effe070 con 0x55b95f258780 2024-03-29T08:25:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 10 client.0 _ll_register_callbacks cb 0x7f6d7c050d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:25:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d8d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f257e60 0x55b95f20e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d8d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f257e60 0x55b95f20e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38564/0 (socket says 172.21.15.46:38564) 2024-03-29T08:25:09.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d8d000640 1 -- 172.21.15.46:0/1937300796 learned_addr learned my addr 172.21.15.46:0/1937300796 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d8e400640 1 --2- 172.21.15.46:0/1937300796 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f258780 0x55b95f20e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d8da00640 1 --2- 172.21.15.46:0/1937300796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f20be10 0x55b95f2b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d8d000640 1 -- 172.21.15.46:0/1937300796 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f258780 msgr2=0x55b95f20e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d8d000640 1 --2- 172.21.15.46:0/1937300796 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f258780 0x55b95f20e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d8d000640 1 -- 172.21.15.46:0/1937300796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f20be10 msgr2=0x55b95f2b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d8d000640 1 --2- 172.21.15.46:0/1937300796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f20be10 0x55b95f2b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d8d000640 1 -- 172.21.15.46:0/1937300796 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8815d4b0 con 0x55b95f257e60 2024-03-29T08:25:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d8da00640 1 --2- 172.21.15.46:0/1937300796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f20be10 0x55b95f2b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d8d000640 1 --2- 172.21.15.46:0/1937300796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f257e60 0x55b95f20e150 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f6d880ad870 tx=0x7f6d74004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1ab5d434c6ca08 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:25:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d6ea00640 1 -- 172.21.15.46:0/1937300796 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74011b80 con 0x55b95f257e60 2024-03-29T08:25:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d6ea00640 1 -- 172.21.15.46:0/1937300796 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6d7401e040 con 0x55b95f257e60 2024-03-29T08:25:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.797+0000 7f6d937ac000 1 -- 172.21.15.46:0/1937300796 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b95f015970 con 0x55b95f257e60 2024-03-29T08:25:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d6ea00640 1 -- 172.21.15.46:0/1937300796 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d74011ce0 con 0x55b95f257e60 2024-03-29T08:25:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 10 client.7767 fetch_fsmap learned FSMap version 62 2024-03-29T08:25:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 1 -- 172.21.15.46:0/1937300796 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55b95eff7960 con 0x55b95f257e60 2024-03-29T08:25:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d6ea00640 1 -- 172.21.15.46:0/1937300796 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b95f015970 con 0x55b95f257e60 2024-03-29T08:25:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d6ea00640 1 -- 172.21.15.46:0/1937300796 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6d74003620 con 0x55b95f257e60 2024-03-29T08:25:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 10 client.7767 fetch_fsmap finished waiting for FSMap version 62 2024-03-29T08:25:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 10 client.7767 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:25:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 20 client.7767 populate_metadata read hostname 'smithi046' 2024-03-29T08:25:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 1 --2- 172.21.15.46:0/1937300796 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55b95f20ebd0 0x55b95f20efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 4 client.7767 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:25:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 1 -- 172.21.15.46:0/1937300796 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b95f087a20 con 0x55b95f20ebd0 2024-03-29T08:25:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d8da00640 1 --2- 172.21.15.46:0/1937300796 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55b95f20ebd0 0x55b95f20efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d8da00640 1 --2- 172.21.15.46:0/1937300796 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55b95f20ebd0 0x55b95f20efb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83248d989d0849a5 server_cookie=26de66ea8af46a74 in_seq=0 out_seq=0 2024-03-29T08:25:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d6ea00640 10 client.7767 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d6ea00640 1 -- 172.21.15.46:0/1937300796 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+256 (crc 0 0 0) 0x55b95f087a20 con 0x55b95f20ebd0 2024-03-29T08:25:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d6ea00640 10 client.7767 handle_command_reply: tid=0 2024-03-29T08:25:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 1 client.7767 shutdown 2024-03-29T08:25:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 1 -- 172.21.15.46:0/1937300796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f257e60 msgr2=0x55b95f20e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 1 --2- 172.21.15.46:0/1937300796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f257e60 0x55b95f20e150 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f6d880ad870 tx=0x7f6d74004740 comp rx=0 tx=0).stop 2024-03-29T08:25:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 1 -- 172.21.15.46:0/1937300796 shutdown_connections 2024-03-29T08:25:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 1 --2- 172.21.15.46:0/1937300796 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55b95f20ebd0 0x55b95f20efb0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 1 --2- 172.21.15.46:0/1937300796 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b95f258780 0x55b95f20e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 1 --2- 172.21.15.46:0/1937300796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b95f257e60 0x55b95f20e150 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 1 --2- 172.21.15.46:0/1937300796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b95f20be10 0x55b95f2b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 1 -- 172.21.15.46:0/1937300796 >> 172.21.15.46:0/1937300796 conn(0x55b95f008160 msgr2=0x55b95f206bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:09.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d6ea00640 0 client.7767 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:09.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 1 -- 172.21.15.46:0/1937300796 shutdown_connections 2024-03-29T08:25:09.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 1 -- 172.21.15.46:0/1937300796 wait complete. 2024-03-29T08:25:09.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d937ac000 20 client.7767 trim_cache size 0 max 16384 2024-03-29T08:25:09.807 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:25:09.807 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-29T08:25:09.808 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-29T08:25:09.808 INFO:teuthology.orchestra.run.smithi046.stdout: "bbe297e5-75eb-4da6-b33c-e15e126044d2": { 2024-03-29T08:25:09.808 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/dir", 2024-03-29T08:25:09.808 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "bbe297e5-75eb-4da6-b33c-e15e126044d2", 2024-03-29T08:25:09.808 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive" 2024-03-29T08:25:09.808 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:25:09.808 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:25:09.808 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:25:09.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d90400640 1 -- 172.21.15.46:0/79498999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6d680666c0 msgr2=0x7f6d68068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d90400640 1 --2- 172.21.15.46:0/79498999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6d680666c0 0x7f6d68068b80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6d881595b0 tx=0x7f6d74008040 comp rx=0 tx=0).stop 2024-03-29T08:25:09.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d90400640 1 -- 172.21.15.46:0/79498999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880cbea0 msgr2=0x7f6d8815f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:09.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d90400640 1 --2- 172.21.15.46:0/79498999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880cbea0 0x7f6d8815f190 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00c500 tx=0x7f6d7c00c9c0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d90400640 1 -- 172.21.15.46:0/79498999 shutdown_connections 2024-03-29T08:25:09.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d90400640 1 --2- 172.21.15.46:0/79498999 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6d680666c0 0x7f6d68068b80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d90400640 1 --2- 172.21.15.46:0/79498999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d88142a50 0x7f6d88154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d90400640 1 --2- 172.21.15.46:0/79498999 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6d8815df90 0x7f6d8813fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d90400640 1 --2- 172.21.15.46:0/79498999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d880cbea0 0x7f6d8815f190 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:09.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d90400640 1 -- 172.21.15.46:0/79498999 >> 172.21.15.46:0/79498999 conn(0x7f6d8806a390 msgr2=0x7f6d880b1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:09.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d90400640 1 -- 172.21.15.46:0/79498999 shutdown_connections 2024-03-29T08:25:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:09.801+0000 7f6d90400640 1 -- 172.21.15.46:0/79498999 wait complete. 2024-03-29T08:25:09.821 INFO:tasks.cephfs.filesystem:scrub status for tag:bbe297e5-75eb-4da6-b33c-e15e126044d2 - {'path': '/dir', 'tag': 'bbe297e5-75eb-4da6-b33c-e15e126044d2', 'options': 'recursive'} 2024-03-29T08:25:17.013 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-03-29T08:25:17.012+0000 7fc16e000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-03-29T08:25:39.243 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:25:39.247 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:25:39.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:39.822 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-29T08:25:39.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.976+0000 7f446c600640 1 Processor -- start 2024-03-29T08:25:39.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.976+0000 7f446c600640 1 -- start start 2024-03-29T08:25:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.976+0000 7f446c600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f446406b4b0 0x7f44640b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.976+0000 7f446c600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44640b3f20 0x7f44640b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.976+0000 7f446c600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4464157490 con 0x7f446415dbb0 2024-03-29T08:25:39.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.976+0000 7f446c600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44640b27b0 con 0x7f446406b4b0 2024-03-29T08:25:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.976+0000 7f446c600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44640b2c90 con 0x7f44640b3f20 2024-03-29T08:25:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.976+0000 7f4469c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f446415dbb0 0x7f446406ada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52204/0 (socket says 172.21.15.46:52204) 2024-03-29T08:25:39.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.976+0000 7f4469c00640 1 -- 172.21.15.46:0/3997087714 learned_addr learned my addr 172.21.15.46:0/3997087714 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:25:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.976+0000 7f446a600640 1 --2- 172.21.15.46:0/3997087714 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44640b3f20 0x7f44640b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.976+0000 7f4469200640 1 --2- 172.21.15.46:0/3997087714 >> v2:172.21.15.73:3300/0 conn(0x7f446406b4b0 0x7f44640b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.976+0000 7f4463e00640 1 -- 172.21.15.46:0/3997087714 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878946260 0 0) 0x7f4464157490 con 0x7f446415dbb0 2024-03-29T08:25:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.976+0000 7f4463e00640 1 -- 172.21.15.46:0/3997087714 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f444c003530 con 0x7f446415dbb0 2024-03-29T08:25:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.976+0000 7f446a600640 1 -- 172.21.15.46:0/3997087714 >> v2:172.21.15.73:3300/0 conn(0x7f446406b4b0 msgr2=0x7f44640b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.976+0000 7f446a600640 1 --2- 172.21.15.46:0/3997087714 >> v2:172.21.15.73:3300/0 conn(0x7f446406b4b0 0x7f44640b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.976+0000 7f446a600640 1 -- 172.21.15.46:0/3997087714 >> v1:172.21.15.46:6789/0 conn(0x7f446415dbb0 legacy=0x7f446406ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:39.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.976+0000 7f446a600640 1 -- 172.21.15.46:0/3997087714 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44640b29b0 con 0x7f44640b3f20 2024-03-29T08:25:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.980+0000 7f446a600640 1 --2- 172.21.15.46:0/3997087714 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44640b3f20 0x7f44640b2270 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f445c009d90 tx=0x7f445c01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d35bbb01094028c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:39.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.980+0000 7f4463e00640 1 -- 172.21.15.46:0/3997087714 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c0a59a0 con 0x7f44640b3f20 2024-03-29T08:25:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.980+0000 7f4463e00640 1 -- 172.21.15.46:0/3997087714 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c0a5b00 con 0x7f44640b3f20 2024-03-29T08:25:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.980+0000 7f4463e00640 1 -- 172.21.15.46:0/3997087714 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c0a5e90 con 0x7f44640b3f20 2024-03-29T08:25:39.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.980+0000 7f446c600640 1 -- 172.21.15.46:0/3997087714 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44640b3f20 msgr2=0x7f44640b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:39.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.980+0000 7f446c600640 1 --2- 172.21.15.46:0/3997087714 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44640b3f20 0x7f44640b2270 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f445c009d90 tx=0x7f445c01c8e0 comp rx=0 tx=0).stop 2024-03-29T08:25:39.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.980+0000 7f446c600640 1 -- 172.21.15.46:0/3997087714 shutdown_connections 2024-03-29T08:25:39.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.980+0000 7f446c600640 1 --2- 172.21.15.46:0/3997087714 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44640b3f20 0x7f44640b2270 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:39.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.980+0000 7f446c600640 1 --2- 172.21.15.46:0/3997087714 >> v2:172.21.15.73:3300/0 conn(0x7f446406b4b0 0x7f44640b1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:39.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.980+0000 7f446c600640 1 -- 172.21.15.46:0/3997087714 >> 172.21.15.46:0/3997087714 conn(0x7f4464070ad0 msgr2=0x7f4464070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:39.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.980+0000 7f446c600640 1 -- 172.21.15.46:0/3997087714 shutdown_connections 2024-03-29T08:25:39.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.980+0000 7f446c600640 1 -- 172.21.15.46:0/3997087714 wait complete. 2024-03-29T08:25:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f446c600640 1 Processor -- start 2024-03-29T08:25:39.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f446c600640 1 -- start start 2024-03-29T08:25:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f446c600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f446406b4b0 0x7f4464156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f446c600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44640b3f20 0x7f4464154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f446c600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f446415dbb0 0x7f446414f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f446c600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4464156200 con 0x7f44640b3f20 2024-03-29T08:25:39.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f446c600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446414f670 con 0x7f446415dbb0 2024-03-29T08:25:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f446c600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44641555a0 con 0x7f446406b4b0 2024-03-29T08:25:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f4469200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44640b3f20 0x7f4464154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f4469200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44640b3f20 0x7f4464154ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56398/0 (socket says 172.21.15.46:56398) 2024-03-29T08:25:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f4469200640 1 -- 172.21.15.46:0/1432629897 learned_addr learned my addr 172.21.15.46:0/1432629897 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f446a600640 1 --2- 172.21.15.46:0/1432629897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f446415dbb0 0x7f446414f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f4469c00640 1 --2- 172.21.15.46:0/1432629897 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f446406b4b0 0x7f4464156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f4469200640 1 -- 172.21.15.46:0/1432629897 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f446406b4b0 msgr2=0x7f4464156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f4469200640 1 --2- 172.21.15.46:0/1432629897 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f446406b4b0 0x7f4464156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f4469200640 1 -- 172.21.15.46:0/1432629897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f446415dbb0 msgr2=0x7f446414f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f4469200640 1 --2- 172.21.15.46:0/1432629897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f446415dbb0 0x7f446414f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f4469200640 1 -- 172.21.15.46:0/1432629897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c009a70 con 0x7f44640b3f20 2024-03-29T08:25:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f4469200640 1 --2- 172.21.15.46:0/1432629897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44640b3f20 0x7f4464154ec0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f445000e8d0 tx=0x7f445000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f4461600640 1 -- 172.21.15.46:0/1432629897 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445000c940 con 0x7f44640b3f20 2024-03-29T08:25:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f4461600640 1 -- 172.21.15.46:0/1432629897 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445000caa0 con 0x7f44640b3f20 2024-03-29T08:25:39.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f446c600640 1 -- 172.21.15.46:0/1432629897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44641558d0 con 0x7f44640b3f20 2024-03-29T08:25:39.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f4461600640 1 -- 172.21.15.46:0/1432629897 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445000ce10 con 0x7f44640b3f20 2024-03-29T08:25:39.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.984+0000 7f446c600640 1 -- 172.21.15.46:0/1432629897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4464155e90 con 0x7f44640b3f20 2024-03-29T08:25:39.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.988+0000 7f4461600640 1 -- 172.21.15.46:0/1432629897 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4450013470 con 0x7f44640b3f20 2024-03-29T08:25:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.988+0000 7f4461600640 1 --2- 172.21.15.46:0/1432629897 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4444066690 0x7f4444068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:39.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.988+0000 7f4461600640 1 -- 172.21.15.46:0/1432629897 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f44500843d0 con 0x7f44640b3f20 2024-03-29T08:25:39.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.988+0000 7f4469c00640 1 --2- 172.21.15.46:0/1432629897 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4444066690 0x7f4444068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:39.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.988+0000 7f446f964000 1 Processor -- start 2024-03-29T08:25:39.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.988+0000 7f446f964000 1 -- start start 2024-03-29T08:25:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.988+0000 7f4469c00640 1 --2- 172.21.15.46:0/1432629897 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4444066690 0x7f4444068b50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4458005890 tx=0x7f4458009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446f964000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b99a70 0x561629b99e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446f964000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b9a390 0x561629b4d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446f964000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b4da00 0x561629b4dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446f964000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561629a18630 con 0x561629b99a70 2024-03-29T08:25:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446f964000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616299a5bd0 con 0x561629b9a390 2024-03-29T08:25:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446f964000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616299a31a0 con 0x561629b4da00 2024-03-29T08:25:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f4469c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b99a70 0x561629b99e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f4469c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b99a70 0x561629b99e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56412/0 (socket says 172.21.15.46:56412) 2024-03-29T08:25:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f4469c00640 1 -- 172.21.15.46:0/2324761576 learned_addr learned my addr 172.21.15.46:0/2324761576 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:39.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f4469200640 1 --2- 172.21.15.46:0/2324761576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b9a390 0x561629b4d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:39.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446a600640 1 --2- 172.21.15.46:0/2324761576 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b4da00 0x561629b4dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:39.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f4469c00640 1 -- 172.21.15.46:0/2324761576 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b4da00 msgr2=0x561629b4dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:39.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f4469c00640 1 --2- 172.21.15.46:0/2324761576 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b4da00 0x561629b4dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:39.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f4469c00640 1 -- 172.21.15.46:0/2324761576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b9a390 msgr2=0x561629b4d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:39.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f4469c00640 1 --2- 172.21.15.46:0/2324761576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b9a390 0x561629b4d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:39.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f4469c00640 1 -- 172.21.15.46:0/2324761576 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56162969ae20 con 0x561629b99a70 2024-03-29T08:25:39.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446a600640 1 --2- 172.21.15.46:0/2324761576 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b4da00 0x561629b4dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:39.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f4469c00640 1 --2- 172.21.15.46:0/2324761576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b99a70 0x561629b99e50 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f4458001540 tx=0x7f4458011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ad6f70b82d8e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:39.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f4456000640 1 -- 172.21.15.46:0/2324761576 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4458005ea0 con 0x561629b99a70 2024-03-29T08:25:39.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f4456000640 1 -- 172.21.15.46:0/2324761576 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4458007710 con 0x561629b99a70 2024-03-29T08:25:39.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f4456000640 1 -- 172.21.15.46:0/2324761576 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445800e970 con 0x561629b99a70 2024-03-29T08:25:39.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446f964000 1 -- 172.21.15.46:0/2324761576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b99a70 msgr2=0x561629b99e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:39.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446f964000 1 --2- 172.21.15.46:0/2324761576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b99a70 0x561629b99e50 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f4458001540 tx=0x7f4458011000 comp rx=0 tx=0).stop 2024-03-29T08:25:39.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446f964000 1 -- 172.21.15.46:0/2324761576 shutdown_connections 2024-03-29T08:25:39.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446f964000 1 --2- 172.21.15.46:0/2324761576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b9a390 0x561629b4d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:39.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446f964000 1 --2- 172.21.15.46:0/2324761576 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b4da00 0x561629b4dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:39.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446f964000 1 --2- 172.21.15.46:0/2324761576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b99a70 0x561629b99e50 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:39.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446f964000 1 -- 172.21.15.46:0/2324761576 >> 172.21.15.46:0/2324761576 conn(0x56162994e160 msgr2=0x561629b7c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:39.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446f964000 1 -- 172.21.15.46:0/2324761576 shutdown_connections 2024-03-29T08:25:39.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446f964000 1 -- 172.21.15.46:0/2324761576 wait complete. 2024-03-29T08:25:39.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446f964000 1 -- start start 2024-03-29T08:25:39.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.992+0000 7f446f964000 1 Processor -- start 2024-03-29T08:25:39.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b4da00 0x561629b567e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:39.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b99a70 0x561629b7da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b9a390 0x561629b7df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162999ea90 con 0x561629b4da00 2024-03-29T08:25:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162993dd40 con 0x561629b9a390 2024-03-29T08:25:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x56162995bf30 con 0x561629b99a70 2024-03-29T08:25:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 10 client.0 _ll_register_callbacks cb 0x561629a00810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:25:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f4469c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b4da00 0x561629b567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f4469c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b4da00 0x561629b567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56416/0 (socket says 172.21.15.46:56416) 2024-03-29T08:25:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f4469c00640 1 -- 172.21.15.46:0/2490669382 learned_addr learned my addr 172.21.15.46:0/2490669382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446a600640 1 --2- 172.21.15.46:0/2490669382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b9a390 0x561629b7df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f4469200640 1 --2- 172.21.15.46:0/2490669382 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b99a70 0x561629b7da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f4469c00640 1 -- 172.21.15.46:0/2490669382 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b99a70 msgr2=0x561629b7da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f4469c00640 1 --2- 172.21.15.46:0/2490669382 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b99a70 0x561629b7da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f4469c00640 1 -- 172.21.15.46:0/2490669382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b9a390 msgr2=0x561629b7df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:39.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f4469c00640 1 --2- 172.21.15.46:0/2490669382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b9a390 0x561629b7df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f4469c00640 1 -- 172.21.15.46:0/2490669382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445800f040 con 0x561629b4da00 2024-03-29T08:25:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f4469c00640 1 --2- 172.21.15.46:0/2490669382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b4da00 0x561629b567e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f44580079e0 tx=0x7f4458013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ff59e18ecfb22f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f444aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:25:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f444aa00640 1 -- 172.21.15.46:0/2490669382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4458028040 con 0x561629b4da00 2024-03-29T08:25:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 -- 172.21.15.46:0/2490669382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56162999d3e0 con 0x561629b4da00 2024-03-29T08:25:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f444aa00640 1 -- 172.21.15.46:0/2490669382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4458007710 con 0x561629b4da00 2024-03-29T08:25:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f444aa00640 1 -- 172.21.15.46:0/2490669382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4458005ea0 con 0x561629b4da00 2024-03-29T08:25:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 10 client.7779 fetch_fsmap learned FSMap version 63 2024-03-29T08:25:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 -- 172.21.15.46:0/2490669382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x561629963aa0 con 0x561629b4da00 2024-03-29T08:25:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f444aa00640 1 -- 172.21.15.46:0/2490669382 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56162999d3e0 con 0x561629b4da00 2024-03-29T08:25:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f444aa00640 1 -- 172.21.15.46:0/2490669382 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 63) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f445800ed10 con 0x561629b4da00 2024-03-29T08:25:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 10 client.7779 fetch_fsmap finished waiting for FSMap version 63 2024-03-29T08:25:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 10 client.7779 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:25:40.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 20 client.7779 populate_metadata read hostname 'smithi046' 2024-03-29T08:25:40.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 --2- 172.21.15.46:0/2490669382 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x561629ba12d0 0x561629ba16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 4 client.7779 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:25:40.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 -- 172.21.15.46:0/2490669382 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56162996ed00 con 0x561629ba12d0 2024-03-29T08:25:40.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f4469200640 1 --2- 172.21.15.46:0/2490669382 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x561629ba12d0 0x561629ba16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f4469200640 1 --2- 172.21.15.46:0/2490669382 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x561629ba12d0 0x561629ba16e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b04ca88e306c2513 server_cookie=1f628b437fc02ddb in_seq=0 out_seq=0 2024-03-29T08:25:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f444aa00640 10 client.7779 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f444aa00640 1 -- 172.21.15.46:0/2490669382 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56162996ed00 con 0x561629ba12d0 2024-03-29T08:25:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f444aa00640 10 client.7779 handle_command_reply: tid=0 2024-03-29T08:25:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 client.7779 shutdown 2024-03-29T08:25:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 -- 172.21.15.46:0/2490669382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b4da00 msgr2=0x561629b567e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 --2- 172.21.15.46:0/2490669382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b4da00 0x561629b567e0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f44580079e0 tx=0x7f4458013000 comp rx=0 tx=0).stop 2024-03-29T08:25:40.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 -- 172.21.15.46:0/2490669382 shutdown_connections 2024-03-29T08:25:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 --2- 172.21.15.46:0/2490669382 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x561629ba12d0 0x561629ba16e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 --2- 172.21.15.46:0/2490669382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b9a390 0x561629b7df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 --2- 172.21.15.46:0/2490669382 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b99a70 0x561629b7da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 --2- 172.21.15.46:0/2490669382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b4da00 0x561629b567e0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f444aa00640 0 client.7779 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:39.996+0000 7f446f964000 1 -- 172.21.15.46:0/2490669382 >> 172.21.15.46:0/2490669382 conn(0x56162994e160 msgr2=0x561629b4cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.000+0000 7f446f964000 1 -- 172.21.15.46:0/2490669382 shutdown_connections 2024-03-29T08:25:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.000+0000 7f446f964000 1 -- 172.21.15.46:0/2490669382 wait complete. 2024-03-29T08:25:40.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.000+0000 7f446f964000 20 client.7779 trim_cache size 0 max 16384 2024-03-29T08:25:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.012+0000 7f446f964000 1 Processor -- start 2024-03-29T08:25:40.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.012+0000 7f446f964000 1 -- start start 2024-03-29T08:25:40.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.012+0000 7f446f964000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b4da00 0x561629b556c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.012+0000 7f446f964000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b99a70 0x561629b55c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.012+0000 7f446f964000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b9a390 0x561629b4dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.012+0000 7f446f964000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162965baa0 con 0x561629b9a390 2024-03-29T08:25:40.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.012+0000 7f446f964000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561629a57ec0 con 0x561629b4da00 2024-03-29T08:25:40.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.012+0000 7f446f964000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616299e2170 con 0x561629b99a70 2024-03-29T08:25:40.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.012+0000 7f446a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b9a390 0x561629b4dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b9a390 0x561629b4dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56418/0 (socket says 172.21.15.46:56418) 2024-03-29T08:25:40.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446a600640 1 -- 172.21.15.46:0/620493467 learned_addr learned my addr 172.21.15.46:0/620493467 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f4469c00640 1 --2- 172.21.15.46:0/620493467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b4da00 0x561629b556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f4469200640 1 --2- 172.21.15.46:0/620493467 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b99a70 0x561629b55c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446a600640 1 -- 172.21.15.46:0/620493467 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b99a70 msgr2=0x561629b55c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446a600640 1 --2- 172.21.15.46:0/620493467 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b99a70 0x561629b55c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446a600640 1 -- 172.21.15.46:0/620493467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b4da00 msgr2=0x561629b556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446a600640 1 --2- 172.21.15.46:0/620493467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b4da00 0x561629b556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446a600640 1 -- 172.21.15.46:0/620493467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561629949df0 con 0x561629b9a390 2024-03-29T08:25:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446a600640 1 --2- 172.21.15.46:0/620493467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b9a390 0x561629b4dde0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f445c0194f0 tx=0x7f445c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953c3e1af734296d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:40.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f4456a00640 1 -- 172.21.15.46:0/620493467 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c0bd070 con 0x561629b9a390 2024-03-29T08:25:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f4456a00640 1 -- 172.21.15.46:0/620493467 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445c0ad440 con 0x561629b9a390 2024-03-29T08:25:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f4456a00640 1 -- 172.21.15.46:0/620493467 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445c0ad7b0 con 0x561629b9a390 2024-03-29T08:25:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 1 -- 172.21.15.46:0/620493467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b9a390 msgr2=0x561629b4dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 1 --2- 172.21.15.46:0/620493467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b9a390 0x561629b4dde0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f445c0194f0 tx=0x7f445c004ac0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 1 -- 172.21.15.46:0/620493467 shutdown_connections 2024-03-29T08:25:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 1 --2- 172.21.15.46:0/620493467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b9a390 0x561629b4dde0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 1 --2- 172.21.15.46:0/620493467 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b99a70 0x561629b55c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 1 --2- 172.21.15.46:0/620493467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b4da00 0x561629b556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 1 -- 172.21.15.46:0/620493467 >> 172.21.15.46:0/620493467 conn(0x56162994e160 msgr2=0x561629b54fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 1 -- 172.21.15.46:0/620493467 shutdown_connections 2024-03-29T08:25:40.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 1 -- 172.21.15.46:0/620493467 wait complete. 2024-03-29T08:25:40.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 1 -- start start 2024-03-29T08:25:40.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 1 Processor -- start 2024-03-29T08:25:40.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b4da00 0x561629bf99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b99a70 0x561629bf9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b9a390 0x561629b50600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56162999c2a0 con 0x561629b4da00 2024-03-29T08:25:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616299f6920 con 0x561629b9a390 2024-03-29T08:25:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616299a15f0 con 0x561629b99a70 2024-03-29T08:25:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446f964000 10 client.0 _ll_register_callbacks cb 0x7f4458001d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:25:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f4469c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b4da00 0x561629bf99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f4469c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b4da00 0x561629bf99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56422/0 (socket says 172.21.15.46:56422) 2024-03-29T08:25:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f4469c00640 1 -- 172.21.15.46:0/2640500868 learned_addr learned my addr 172.21.15.46:0/2640500868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f446a600640 1 --2- 172.21.15.46:0/2640500868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b9a390 0x561629b50600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.016+0000 7f4469200640 1 --2- 172.21.15.46:0/2640500868 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b99a70 0x561629bf9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f4469c00640 1 -- 172.21.15.46:0/2640500868 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b99a70 msgr2=0x561629bf9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:40.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f4469c00640 1 --2- 172.21.15.46:0/2640500868 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b99a70 0x561629bf9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f4469c00640 1 -- 172.21.15.46:0/2640500868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b9a390 msgr2=0x561629b50600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f4469c00640 1 --2- 172.21.15.46:0/2640500868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b9a390 0x561629b50600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f4469c00640 1 -- 172.21.15.46:0/2640500868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44640b29b0 con 0x561629b4da00 2024-03-29T08:25:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f446a600640 1 --2- 172.21.15.46:0/2640500868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b9a390 0x561629b50600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f4469c00640 1 --2- 172.21.15.46:0/2640500868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b4da00 0x561629bf99c0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f44580070a0 tx=0x7f44580010b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf92c6d21c2e541c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f444aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:25:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f444aa00640 1 -- 172.21.15.46:0/2640500868 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4458028070 con 0x561629b4da00 2024-03-29T08:25:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f444aa00640 1 -- 172.21.15.46:0/2640500868 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f4458007710 con 0x561629b4da00 2024-03-29T08:25:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f446f964000 1 -- 172.21.15.46:0/2640500868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56162995b970 con 0x561629b4da00 2024-03-29T08:25:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f444aa00640 1 -- 172.21.15.46:0/2640500868 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44580074a0 con 0x561629b4da00 2024-03-29T08:25:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f446f964000 10 client.7785 fetch_fsmap learned FSMap version 63 2024-03-29T08:25:40.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f446f964000 1 -- 172.21.15.46:0/2640500868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x56162993d960 con 0x561629b4da00 2024-03-29T08:25:40.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f444aa00640 1 -- 172.21.15.46:0/2640500868 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56162995b970 con 0x561629b4da00 2024-03-29T08:25:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f444aa00640 1 -- 172.21.15.46:0/2640500868 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 63) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f445803e070 con 0x561629b4da00 2024-03-29T08:25:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f446f964000 10 client.7785 fetch_fsmap finished waiting for FSMap version 63 2024-03-29T08:25:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f446f964000 10 client.7785 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:25:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f446f964000 20 client.7785 populate_metadata read hostname 'smithi046' 2024-03-29T08:25:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f446f964000 1 --2- 172.21.15.46:0/2640500868 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x561629b50b40 0x561629b50f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f446f964000 4 client.7785 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:25:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f446f964000 1 -- 172.21.15.46:0/2640500868 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5616299cda20 con 0x561629b50b40 2024-03-29T08:25:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.020+0000 7f4469200640 1 --2- 172.21.15.46:0/2640500868 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x561629b50b40 0x561629b50f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f4469200640 1 --2- 172.21.15.46:0/2640500868 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x561629b50b40 0x561629b50f20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d4d2bc7b5b7c896 server_cookie=756cae588ea36fc6 in_seq=0 out_seq=0 2024-03-29T08:25:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f444aa00640 10 client.7785 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f444aa00640 1 -- 172.21.15.46:0/2640500868 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5616299cda20 con 0x561629b50b40 2024-03-29T08:25:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f444aa00640 10 client.7785 handle_command_reply: tid=0 2024-03-29T08:25:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446f964000 1 client.7785 shutdown 2024-03-29T08:25:40.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446f964000 1 -- 172.21.15.46:0/2640500868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b4da00 msgr2=0x561629bf99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446f964000 1 --2- 172.21.15.46:0/2640500868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b4da00 0x561629bf99c0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f44580070a0 tx=0x7f44580010b0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446f964000 1 -- 172.21.15.46:0/2640500868 shutdown_connections 2024-03-29T08:25:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446f964000 1 --2- 172.21.15.46:0/2640500868 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x561629b50b40 0x561629b50f20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446f964000 1 --2- 172.21.15.46:0/2640500868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561629b9a390 0x561629b50600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446f964000 1 --2- 172.21.15.46:0/2640500868 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561629b99a70 0x561629bf9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446f964000 1 --2- 172.21.15.46:0/2640500868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561629b4da00 0x561629bf99c0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f444aa00640 0 client.7785 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446f964000 1 -- 172.21.15.46:0/2640500868 >> 172.21.15.46:0/2640500868 conn(0x56162994e160 msgr2=0x561629b55840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446f964000 1 -- 172.21.15.46:0/2640500868 shutdown_connections 2024-03-29T08:25:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446f964000 1 -- 172.21.15.46:0/2640500868 wait complete. 2024-03-29T08:25:40.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446f964000 20 client.7785 trim_cache size 0 max 16384 2024-03-29T08:25:40.027 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:25:40.027 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-29T08:25:40.027 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-29T08:25:40.027 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:25:40.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446c600640 1 -- 172.21.15.46:0/1432629897 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4444066690 msgr2=0x7f4444068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446c600640 1 --2- 172.21.15.46:0/1432629897 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4444066690 0x7f4444068b50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4458005890 tx=0x7f4458009040 comp rx=0 tx=0).stop 2024-03-29T08:25:40.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446c600640 1 -- 172.21.15.46:0/1432629897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44640b3f20 msgr2=0x7f4464154ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446c600640 1 --2- 172.21.15.46:0/1432629897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44640b3f20 0x7f4464154ec0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f445000e8d0 tx=0x7f445000ed90 comp rx=0 tx=0).stop 2024-03-29T08:25:40.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446c600640 1 -- 172.21.15.46:0/1432629897 shutdown_connections 2024-03-29T08:25:40.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446c600640 1 --2- 172.21.15.46:0/1432629897 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4444066690 0x7f4444068b50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446c600640 1 --2- 172.21.15.46:0/1432629897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f446415dbb0 0x7f446414f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446c600640 1 --2- 172.21.15.46:0/1432629897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44640b3f20 0x7f4464154ec0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446c600640 1 --2- 172.21.15.46:0/1432629897 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f446406b4b0 0x7f4464156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446c600640 1 -- 172.21.15.46:0/1432629897 >> 172.21.15.46:0/1432629897 conn(0x7f4464070ad0 msgr2=0x7f44640b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:40.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446c600640 1 -- 172.21.15.46:0/1432629897 shutdown_connections 2024-03-29T08:25:40.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.024+0000 7f446c600640 1 -- 172.21.15.46:0/1432629897 wait complete. 2024-03-29T08:25:40.044 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-29T08:25:40.044 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:25:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.184+0000 7f15fd400640 1 Processor -- start 2024-03-29T08:25:40.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.184+0000 7f15fd400640 1 -- start start 2024-03-29T08:25:40.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.184+0000 7f15fd400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15f80d0150 0x7f15f80cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.184+0000 7f15fd400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f15f80cd9d0 0x7f15f80cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.184+0000 7f15fd400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f80b0ed0 con 0x7f15f80b01c0 2024-03-29T08:25:40.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.184+0000 7f15fd400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f80cff90 con 0x7f15f80cd9d0 2024-03-29T08:25:40.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.184+0000 7f15fd400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f80cbd60 con 0x7f15f80d0150 2024-03-29T08:25:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.184+0000 7f15ede00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15f80d0150 0x7f15f80cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.184+0000 7f15f7200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f15f80cd9d0 0x7f15f80cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.184+0000 7f15ede00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15f80d0150 0x7f15f80cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:35130/0 (socket says 172.21.15.46:35130) 2024-03-29T08:25:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.184+0000 7f15ede00640 1 -- 172.21.15.46:0/1846995472 learned_addr learned my addr 172.21.15.46:0/1846995472 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:40.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.184+0000 7f15ede00640 1 -- 172.21.15.46:0/1846995472 >> v2:172.21.15.73:3300/0 conn(0x7f15f80cd9d0 msgr2=0x7f15f80cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:40.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.184+0000 7f15ede00640 1 --2- 172.21.15.46:0/1846995472 >> v2:172.21.15.73:3300/0 conn(0x7f15f80cd9d0 0x7f15f80cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.184+0000 7f15ede00640 1 -- 172.21.15.46:0/1846995472 >> v1:172.21.15.73:6789/0 conn(0x7f15f80b01c0 legacy=0x7f15f80aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.184+0000 7f15ede00640 1 -- 172.21.15.46:0/1846995472 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f80ca5c0 con 0x7f15f80d0150 2024-03-29T08:25:40.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.184+0000 7f15f7200640 1 --2- 172.21.15.46:0/1846995472 >> v2:172.21.15.73:3300/0 conn(0x7f15f80cd9d0 0x7f15f80cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:25:40.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.188+0000 7f15ede00640 1 --2- 172.21.15.46:0/1846995472 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15f80d0150 0x7f15f80cb820 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009870 tx=0x7f15e801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e960f3ce92cce6b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:40.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.188+0000 7f15f5e00640 1 -- 172.21.15.46:0/1846995472 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e80ab600 con 0x7f15f80d0150 2024-03-29T08:25:40.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.188+0000 7f15f5e00640 1 -- 172.21.15.46:0/1846995472 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e80a7050 con 0x7f15f80d0150 2024-03-29T08:25:40.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.188+0000 7f15f5e00640 1 -- 172.21.15.46:0/1846995472 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e80ab980 con 0x7f15f80d0150 2024-03-29T08:25:40.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.188+0000 7f15fd400640 1 -- 172.21.15.46:0/1846995472 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15f80d0150 msgr2=0x7f15f80cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.188+0000 7f15fd400640 1 --2- 172.21.15.46:0/1846995472 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15f80d0150 0x7f15f80cb820 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f15e8009870 tx=0x7f15e801c560 comp rx=0 tx=0).stop 2024-03-29T08:25:40.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.188+0000 7f15fd400640 1 -- 172.21.15.46:0/1846995472 shutdown_connections 2024-03-29T08:25:40.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.188+0000 7f15fd400640 1 --2- 172.21.15.46:0/1846995472 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15f80d0150 0x7f15f80cb820 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.188+0000 7f15fd400640 1 --2- 172.21.15.46:0/1846995472 >> v2:172.21.15.73:3300/0 conn(0x7f15f80cd9d0 0x7f15f80cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.188+0000 7f15fd400640 1 -- 172.21.15.46:0/1846995472 >> 172.21.15.46:0/1846995472 conn(0x7f15f8098a40 msgr2=0x7f15f80cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:40.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.188+0000 7f15fd400640 1 -- 172.21.15.46:0/1846995472 shutdown_connections 2024-03-29T08:25:40.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.188+0000 7f15fd400640 1 -- 172.21.15.46:0/1846995472 wait complete. 2024-03-29T08:25:40.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15fd400640 1 Processor -- start 2024-03-29T08:25:40.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15fd400640 1 -- start start 2024-03-29T08:25:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15fd400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15f80b01c0 0x7f15f8155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15fd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f80cd9d0 0x7f15f809a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15fd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15f80d0150 0x7f15f8154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15fd400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f809ac60 con 0x7f15f80d0150 2024-03-29T08:25:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15fd400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f809c630 con 0x7f15f80cd9d0 2024-03-29T08:25:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15fd400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f809c910 con 0x7f15f80b01c0 2024-03-29T08:25:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15f7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15f80d0150 0x7f15f8154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15f7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15f80d0150 0x7f15f8154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56424/0 (socket says 172.21.15.46:56424) 2024-03-29T08:25:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15f7200640 1 -- 172.21.15.46:0/2485227538 learned_addr learned my addr 172.21.15.46:0/2485227538 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15ede00640 1 --2- 172.21.15.46:0/2485227538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f80cd9d0 0x7f15f809a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15f6800640 1 --2- 172.21.15.46:0/2485227538 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15f80b01c0 0x7f15f8155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15f7200640 1 -- 172.21.15.46:0/2485227538 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15f80b01c0 msgr2=0x7f15f8155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15f7200640 1 --2- 172.21.15.46:0/2485227538 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15f80b01c0 0x7f15f8155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15f7200640 1 -- 172.21.15.46:0/2485227538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f80cd9d0 msgr2=0x7f15f809a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15f7200640 1 --2- 172.21.15.46:0/2485227538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f80cd9d0 0x7f15f809a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15f7200640 1 -- 172.21.15.46:0/2485227538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f0009a70 con 0x7f15f80d0150 2024-03-29T08:25:40.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15ede00640 1 --2- 172.21.15.46:0/2485227538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f80cd9d0 0x7f15f809a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:40.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15f7200640 1 --2- 172.21.15.46:0/2485227538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15f80d0150 0x7f15f8154af0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f15f000efc0 tx=0x7f15f0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:40.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15ef400640 1 -- 172.21.15.46:0/2485227538 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0009d70 con 0x7f15f80d0150 2024-03-29T08:25:40.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15ef400640 1 -- 172.21.15.46:0/2485227538 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f000f040 con 0x7f15f80d0150 2024-03-29T08:25:40.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15ef400640 1 -- 172.21.15.46:0/2485227538 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f00039a0 con 0x7f15f80d0150 2024-03-29T08:25:40.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15fd400640 1 -- 172.21.15.46:0/2485227538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e80094d0 con 0x7f15f80d0150 2024-03-29T08:25:40.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15fd400640 1 -- 172.21.15.46:0/2485227538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f809ced0 con 0x7f15f80d0150 2024-03-29T08:25:40.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.192+0000 7f15fd400640 1 -- 172.21.15.46:0/2485227538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f809d0c0 con 0x7f15f80d0150 2024-03-29T08:25:40.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.196+0000 7f15ef400640 1 -- 172.21.15.46:0/2485227538 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f15f00144a0 con 0x7f15f80d0150 2024-03-29T08:25:40.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.196+0000 7f15ef400640 1 --2- 172.21.15.46:0/2485227538 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15d4066690 0x7f15d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.196+0000 7f15ef400640 1 -- 172.21.15.46:0/2485227538 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f15f0084f40 con 0x7f15f80d0150 2024-03-29T08:25:40.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.196+0000 7f15f6800640 1 --2- 172.21.15.46:0/2485227538 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15d4066690 0x7f15d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.196+0000 7f15f6800640 1 --2- 172.21.15.46:0/2485227538 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15d4066690 0x7f15d4068b50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f15f81f6b10 tx=0x7f15e0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:40.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.200+0000 7f15ef400640 1 -- 172.21.15.46:0/2485227538 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f15f00521b0 con 0x7f15f80d0150 2024-03-29T08:25:40.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.392+0000 7f15fd400640 1 -- 172.21.15.46:0/2485227538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15f8066c90 con 0x7f15f80d0150 2024-03-29T08:25:40.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.392+0000 7f15ef400640 1 -- 172.21.15.46:0/2485227538 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f15f00557d0 con 0x7f15f80d0150 2024-03-29T08:25:40.395 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:25:40.395 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/1862652020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1862652020},{"type":"v1","addr":"172.21.15.73:6836","nonce":1862652020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:24:43.476334+0000","modified":"2024-03-29T08:25:17.015744+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7386},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7386":{"gid":7386,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":13,"addr":"172.21.15.46:6835/2051429508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2051429508},{"type":"v1","addr":"172.21.15.46:6835","nonce":2051429508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-29T08:25:40.395 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 63 2024-03-29T08:25:40.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.396+0000 7f15fd400640 1 -- 172.21.15.46:0/2485227538 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15d4066690 msgr2=0x7f15d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.396+0000 7f15fd400640 1 --2- 172.21.15.46:0/2485227538 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15d4066690 0x7f15d4068b50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f15f81f6b10 tx=0x7f15e0009040 comp rx=0 tx=0).stop 2024-03-29T08:25:40.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.396+0000 7f15fd400640 1 -- 172.21.15.46:0/2485227538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15f80d0150 msgr2=0x7f15f8154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.396+0000 7f15fd400640 1 --2- 172.21.15.46:0/2485227538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15f80d0150 0x7f15f8154af0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f15f000efc0 tx=0x7f15f0009f90 comp rx=0 tx=0).stop 2024-03-29T08:25:40.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.396+0000 7f15fd400640 1 -- 172.21.15.46:0/2485227538 shutdown_connections 2024-03-29T08:25:40.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.396+0000 7f15fd400640 1 --2- 172.21.15.46:0/2485227538 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15d4066690 0x7f15d4068b50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.396+0000 7f15fd400640 1 --2- 172.21.15.46:0/2485227538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f80cd9d0 0x7f15f809a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.396+0000 7f15fd400640 1 --2- 172.21.15.46:0/2485227538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15f80d0150 0x7f15f8154af0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.396+0000 7f15fd400640 1 --2- 172.21.15.46:0/2485227538 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15f80b01c0 0x7f15f8155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.396+0000 7f15fd400640 1 -- 172.21.15.46:0/2485227538 >> 172.21.15.46:0/2485227538 conn(0x7f15f8098a40 msgr2=0x7f15f80c8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.396+0000 7f15fd400640 1 -- 172.21.15.46:0/2485227538 shutdown_connections 2024-03-29T08:25:40.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.396+0000 7f15fd400640 1 -- 172.21.15.46:0/2485227538 wait complete. 2024-03-29T08:25:40.416 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-03-29T08:25:40.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.556+0000 7f2e5e200640 1 Processor -- start 2024-03-29T08:25:40.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.556+0000 7f2e5e200640 1 -- start start 2024-03-29T08:25:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.560+0000 7f2e5e200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2e580cbea0 0x7f2e580ae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.560+0000 7f2e5e200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e5815d260 con 0x7f2e5815df90 2024-03-29T08:25:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.560+0000 7f2e5e200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e580aefd0 con 0x7f2e58142a50 2024-03-29T08:25:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.560+0000 7f2e5e200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e5815d4b0 con 0x7f2e580cbea0 2024-03-29T08:25:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.560+0000 7f2e4fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2e5815df90 0x7f2e580ae280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52214/0 (socket says 172.21.15.46:52214) 2024-03-29T08:25:40.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.560+0000 7f2e4fe00640 1 -- 172.21.15.46:0/1165192132 learned_addr learned my addr 172.21.15.46:0/1165192132 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:25:40.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.560+0000 7f2e57e00640 1 --2- 172.21.15.46:0/1165192132 >> v2:172.21.15.73:3300/0 conn(0x7f2e580cbea0 0x7f2e580ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.560+0000 7f2e56a00640 1 -- 172.21.15.46:0/1165192132 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359396844 0 0) 0x7f2e5815d260 con 0x7f2e5815df90 2024-03-29T08:25:40.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.560+0000 7f2e56a00640 1 -- 172.21.15.46:0/1165192132 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e3c003530 con 0x7f2e5815df90 2024-03-29T08:25:40.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.560+0000 7f2e56a00640 1 -- 172.21.15.46:0/1165192132 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148205419 0 0) 0x7f2e580aefd0 con 0x7f2e58142a50 2024-03-29T08:25:40.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.560+0000 7f2e56a00640 1 -- 172.21.15.46:0/1165192132 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e5815d260 con 0x7f2e58142a50 2024-03-29T08:25:40.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.560+0000 7f2e56a00640 1 -- 172.21.15.46:0/1165192132 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4198691012 0 0) 0x7f2e3c003530 con 0x7f2e5815df90 2024-03-29T08:25:40.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.560+0000 7f2e57e00640 1 -- 172.21.15.46:0/1165192132 >> v1:172.21.15.73:6789/0 conn(0x7f2e58142a50 legacy=0x7f2e580cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.560+0000 7f2e57e00640 1 -- 172.21.15.46:0/1165192132 >> v1:172.21.15.46:6789/0 conn(0x7f2e5815df90 legacy=0x7f2e580ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.560+0000 7f2e57e00640 1 -- 172.21.15.46:0/1165192132 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5815d790 con 0x7f2e580cbea0 2024-03-29T08:25:40.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.564+0000 7f2e57e00640 1 --2- 172.21.15.46:0/1165192132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e580cbea0 0x7f2e580ae990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009d90 tx=0x7f2e5001c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3175da279ab3fbf3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:40.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.564+0000 7f2e56a00640 1 -- 172.21.15.46:0/1165192132 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e40001680 con 0x7f2e580cbea0 2024-03-29T08:25:40.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.564+0000 7f2e56a00640 1 -- 172.21.15.46:0/1165192132 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e500a9450 con 0x7f2e580cbea0 2024-03-29T08:25:40.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.564+0000 7f2e56a00640 1 -- 172.21.15.46:0/1165192132 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e500a97c0 con 0x7f2e580cbea0 2024-03-29T08:25:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.564+0000 7f2e5e200640 1 -- 172.21.15.46:0/1165192132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e580cbea0 msgr2=0x7f2e580ae990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.564+0000 7f2e5e200640 1 --2- 172.21.15.46:0/1165192132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e580cbea0 0x7f2e580ae990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2e50009d90 tx=0x7f2e5001c800 comp rx=0 tx=0).stop 2024-03-29T08:25:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.564+0000 7f2e5e200640 1 -- 172.21.15.46:0/1165192132 shutdown_connections 2024-03-29T08:25:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.564+0000 7f2e5e200640 1 --2- 172.21.15.46:0/1165192132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e580cbea0 0x7f2e580ae990 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.564+0000 7f2e5e200640 1 -- 172.21.15.46:0/1165192132 >> 172.21.15.46:0/1165192132 conn(0x7f2e5806a390 msgr2=0x7f2e580cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:40.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.564+0000 7f2e5e200640 1 -- 172.21.15.46:0/1165192132 shutdown_connections 2024-03-29T08:25:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.564+0000 7f2e5e200640 1 -- 172.21.15.46:0/1165192132 wait complete. 2024-03-29T08:25:40.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.564+0000 7f2e5e200640 1 Processor -- start 2024-03-29T08:25:40.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.564+0000 7f2e5e200640 1 -- start start 2024-03-29T08:25:40.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e5e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e580cbea0 0x7f2e5815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e5e200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e58142a50 0x7f2e580b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e5e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e5815df90 0x7f2e580b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e5e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e58138cc0 con 0x7f2e580cbea0 2024-03-29T08:25:40.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e5e200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e58159ca0 con 0x7f2e5815df90 2024-03-29T08:25:40.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e5e200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e58159eb0 con 0x7f2e58142a50 2024-03-29T08:25:40.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e57400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e580cbea0 0x7f2e5815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e57400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e580cbea0 0x7f2e5815ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56436/0 (socket says 172.21.15.46:56436) 2024-03-29T08:25:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e57400640 1 -- 172.21.15.46:0/50320252 learned_addr learned my addr 172.21.15.46:0/50320252 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e57e00640 1 --2- 172.21.15.46:0/50320252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e5815df90 0x7f2e580b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e4fe00640 1 --2- 172.21.15.46:0/50320252 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e58142a50 0x7f2e580b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e57400640 1 -- 172.21.15.46:0/50320252 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e58142a50 msgr2=0x7f2e580b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e57400640 1 --2- 172.21.15.46:0/50320252 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e58142a50 0x7f2e580b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e57400640 1 -- 172.21.15.46:0/50320252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e5815df90 msgr2=0x7f2e580b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e57400640 1 --2- 172.21.15.46:0/50320252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e5815df90 0x7f2e580b2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e57400640 1 -- 172.21.15.46:0/50320252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e50009a70 con 0x7f2e580cbea0 2024-03-29T08:25:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e57e00640 1 --2- 172.21.15.46:0/50320252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e5815df90 0x7f2e580b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e57400640 1 --2- 172.21.15.46:0/50320252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e580cbea0 0x7f2e5815ac70 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800c490 tx=0x7f2e4800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e4f400640 1 -- 172.21.15.46:0/50320252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e480052d0 con 0x7f2e580cbea0 2024-03-29T08:25:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e4f400640 1 -- 172.21.15.46:0/50320252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e48005430 con 0x7f2e580cbea0 2024-03-29T08:25:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e5e200640 1 -- 172.21.15.46:0/50320252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5815a190 con 0x7f2e580cbea0 2024-03-29T08:25:40.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e4f400640 1 -- 172.21.15.46:0/50320252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e48012870 con 0x7f2e580cbea0 2024-03-29T08:25:40.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e5e200640 1 -- 172.21.15.46:0/50320252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5815a750 con 0x7f2e580cbea0 2024-03-29T08:25:40.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.568+0000 7f2e4f400640 1 -- 172.21.15.46:0/50320252 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2e480129d0 con 0x7f2e580cbea0 2024-03-29T08:25:40.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e4f400640 1 --2- 172.21.15.46:0/50320252 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e34066690 0x7f2e34068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e4f400640 1 -- 172.21.15.46:0/50320252 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2e48083e40 con 0x7f2e580cbea0 2024-03-29T08:25:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e61655000 1 Processor -- start 2024-03-29T08:25:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e4fe00640 1 --2- 172.21.15.46:0/50320252 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e34066690 0x7f2e34068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e61655000 1 -- start start 2024-03-29T08:25:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e4fe00640 1 --2- 172.21.15.46:0/50320252 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e34066690 0x7f2e34068b50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2e581f8490 tx=0x7f2e40009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:40.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e61655000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386290 0x55c8ee386670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e61655000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386bb0 0x55c8ee33cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e61655000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee33d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e61655000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ee205340 con 0x55c8ee386290 2024-03-29T08:25:40.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e61655000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ee191cd0 con 0x55c8ee33d1b0 2024-03-29T08:25:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e61655000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8ee18f2a0 con 0x55c8ee386bb0 2024-03-29T08:25:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e57400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386290 0x55c8ee386670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e57400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386290 0x55c8ee386670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56440/0 (socket says 172.21.15.46:56440) 2024-03-29T08:25:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e57400640 1 -- 172.21.15.46:0/1312562920 learned_addr learned my addr 172.21.15.46:0/1312562920 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e4fe00640 1 --2- 172.21.15.46:0/1312562920 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386bb0 0x55c8ee33cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e57e00640 1 --2- 172.21.15.46:0/1312562920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee33d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e57400640 1 -- 172.21.15.46:0/1312562920 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386bb0 msgr2=0x55c8ee33cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e57400640 1 --2- 172.21.15.46:0/1312562920 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386bb0 0x55c8ee33cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e57400640 1 -- 172.21.15.46:0/1312562920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 msgr2=0x55c8ee33d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e57400640 1 --2- 172.21.15.46:0/1312562920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee33d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e57400640 1 -- 172.21.15.46:0/1312562920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8ede7d4b0 con 0x55c8ee386290 2024-03-29T08:25:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e57400640 1 --2- 172.21.15.46:0/1312562920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386290 0x55c8ee386670 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f2e48051270 tx=0x7f2e48054800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961fc8cf28f4a004 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.572+0000 7f2e47e00640 1 -- 172.21.15.46:0/1312562920 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e48053110 con 0x55c8ee386290 2024-03-29T08:25:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e47e00640 1 -- 172.21.15.46:0/1312562920 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e48051480 con 0x55c8ee386290 2024-03-29T08:25:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e47e00640 1 -- 172.21.15.46:0/1312562920 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e480515f0 con 0x55c8ee386290 2024-03-29T08:25:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 -- 172.21.15.46:0/1312562920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386290 msgr2=0x55c8ee386670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 --2- 172.21.15.46:0/1312562920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386290 0x55c8ee386670 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f2e48051270 tx=0x7f2e48054800 comp rx=0 tx=0).stop 2024-03-29T08:25:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 -- 172.21.15.46:0/1312562920 shutdown_connections 2024-03-29T08:25:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 --2- 172.21.15.46:0/1312562920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee33d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 --2- 172.21.15.46:0/1312562920 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386bb0 0x55c8ee33cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 --2- 172.21.15.46:0/1312562920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386290 0x55c8ee386670 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 -- 172.21.15.46:0/1312562920 >> 172.21.15.46:0/1312562920 conn(0x55c8ee339160 msgr2=0x55c8ee33b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 -- 172.21.15.46:0/1312562920 shutdown_connections 2024-03-29T08:25:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 -- 172.21.15.46:0/1312562920 wait complete. 2024-03-29T08:25:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 -- start start 2024-03-29T08:25:40.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 Processor -- start 2024-03-29T08:25:40.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee3426e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386290 0x55c8ee342c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386bb0 0x55c8ee347e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ee18ab90 con 0x55c8ee386290 2024-03-29T08:25:40.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ee1480a0 con 0x55c8ee33d1b0 2024-03-29T08:25:40.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8ee147d40 con 0x55c8ee386bb0 2024-03-29T08:25:40.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 10 client.0 _ll_register_callbacks cb 0x55c8ee23fc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:25:40.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e4fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386290 0x55c8ee342c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e4fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386290 0x55c8ee342c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56454/0 (socket says 172.21.15.46:56454) 2024-03-29T08:25:40.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e4fe00640 1 -- 172.21.15.46:0/3723250868 learned_addr learned my addr 172.21.15.46:0/3723250868 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:40.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e57e00640 1 --2- 172.21.15.46:0/3723250868 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386bb0 0x55c8ee347e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e57400640 1 --2- 172.21.15.46:0/3723250868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee3426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e4fe00640 1 -- 172.21.15.46:0/3723250868 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386bb0 msgr2=0x55c8ee347e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:40.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e4fe00640 1 --2- 172.21.15.46:0/3723250868 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386bb0 0x55c8ee347e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e4fe00640 1 -- 172.21.15.46:0/3723250868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 msgr2=0x55c8ee3426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:40.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e4fe00640 1 --2- 172.21.15.46:0/3723250868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee3426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e4fe00640 1 -- 172.21.15.46:0/3723250868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5815a750 con 0x55c8ee386290 2024-03-29T08:25:40.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e57e00640 1 --2- 172.21.15.46:0/3723250868 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386bb0 0x55c8ee347e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:25:40.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e57400640 1 --2- 172.21.15.46:0/3723250868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee3426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:25:40.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e4fe00640 1 --2- 172.21.15.46:0/3723250868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386290 0x55c8ee342c20 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f2e40007100 tx=0x7f2e40001520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b81ae66efbcd54b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:40.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e44c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:25:40.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e44c00640 1 -- 172.21.15.46:0/3723250868 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e400058e0 con 0x55c8ee386290 2024-03-29T08:25:40.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e44c00640 1 -- 172.21.15.46:0/3723250868 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2e40021af0 con 0x55c8ee386290 2024-03-29T08:25:40.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e61655000 1 -- 172.21.15.46:0/3723250868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8ee15a550 con 0x55c8ee386290 2024-03-29T08:25:40.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.576+0000 7f2e44c00640 1 -- 172.21.15.46:0/3723250868 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e40021cb0 con 0x55c8ee386290 2024-03-29T08:25:40.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 10 client.7800 fetch_fsmap learned FSMap version 63 2024-03-29T08:25:40.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 1 -- 172.21.15.46:0/3723250868 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55c8ee14fba0 con 0x55c8ee386290 2024-03-29T08:25:40.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e44c00640 1 -- 172.21.15.46:0/3723250868 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c8ee15a550 con 0x55c8ee386290 2024-03-29T08:25:40.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e44c00640 1 -- 172.21.15.46:0/3723250868 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 63) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2e40029070 con 0x55c8ee386290 2024-03-29T08:25:40.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 10 client.7800 fetch_fsmap finished waiting for FSMap version 63 2024-03-29T08:25:40.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 10 client.7800 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:25:40.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 20 client.7800 populate_metadata read hostname 'smithi046' 2024-03-29T08:25:40.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 1 --2- 172.21.15.46:0/3723250868 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55c8ee395320 0x55c8ee397710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 4 client.7800 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:25:40.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 1 -- 172.21.15.46:0/3723250868 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8ee187970 con 0x55c8ee395320 2024-03-29T08:25:40.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e57400640 1 --2- 172.21.15.46:0/3723250868 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55c8ee395320 0x55c8ee397710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e57400640 1 --2- 172.21.15.46:0/3723250868 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55c8ee395320 0x55c8ee397710 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc23419cecc82326 server_cookie=ea0bf1d34436a233 in_seq=0 out_seq=0 2024-03-29T08:25:40.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e44c00640 10 client.7800 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:40.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e44c00640 1 -- 172.21.15.46:0/3723250868 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55c8ee187970 con 0x55c8ee395320 2024-03-29T08:25:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e44c00640 10 client.7800 handle_command_reply: tid=0 2024-03-29T08:25:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 1 client.7800 shutdown 2024-03-29T08:25:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 1 -- 172.21.15.46:0/3723250868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386290 msgr2=0x55c8ee342c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 1 --2- 172.21.15.46:0/3723250868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386290 0x55c8ee342c20 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f2e40007100 tx=0x7f2e40001520 comp rx=0 tx=0).stop 2024-03-29T08:25:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 1 -- 172.21.15.46:0/3723250868 shutdown_connections 2024-03-29T08:25:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 1 --2- 172.21.15.46:0/3723250868 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55c8ee395320 0x55c8ee397710 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 1 --2- 172.21.15.46:0/3723250868 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386bb0 0x55c8ee347e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 1 --2- 172.21.15.46:0/3723250868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386290 0x55c8ee342c20 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e44c00640 0 client.7800 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 1 --2- 172.21.15.46:0/3723250868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee3426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 1 -- 172.21.15.46:0/3723250868 >> 172.21.15.46:0/3723250868 conn(0x55c8ee339160 msgr2=0x55c8ee33bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 1 -- 172.21.15.46:0/3723250868 shutdown_connections 2024-03-29T08:25:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 1 -- 172.21.15.46:0/3723250868 wait complete. 2024-03-29T08:25:40.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.580+0000 7f2e61655000 20 client.7800 trim_cache size 0 max 16384 2024-03-29T08:25:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e61655000 1 Processor -- start 2024-03-29T08:25:40.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e61655000 1 -- start start 2024-03-29T08:25:40.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e61655000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee399260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e61655000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386290 0x55c8ee3997a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e61655000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386bb0 0x55c8ee395700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e61655000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ede47c40 con 0x55c8ee386bb0 2024-03-29T08:25:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e61655000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ee338b00 con 0x55c8ee33d1b0 2024-03-29T08:25:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e61655000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8ee186380 con 0x55c8ee386290 2024-03-29T08:25:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e57e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386bb0 0x55c8ee395700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e57e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386bb0 0x55c8ee395700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56470/0 (socket says 172.21.15.46:56470) 2024-03-29T08:25:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e57e00640 1 -- 172.21.15.46:0/4025851577 learned_addr learned my addr 172.21.15.46:0/4025851577 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e57400640 1 --2- 172.21.15.46:0/4025851577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee399260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e4fe00640 1 --2- 172.21.15.46:0/4025851577 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386290 0x55c8ee3997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e57e00640 1 -- 172.21.15.46:0/4025851577 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386290 msgr2=0x55c8ee3997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e57e00640 1 --2- 172.21.15.46:0/4025851577 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386290 0x55c8ee3997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e57e00640 1 -- 172.21.15.46:0/4025851577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 msgr2=0x55c8ee399260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:40.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e57e00640 1 --2- 172.21.15.46:0/4025851577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee399260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e57e00640 1 -- 172.21.15.46:0/4025851577 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8ee135ef0 con 0x55c8ee386bb0 2024-03-29T08:25:40.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.592+0000 7f2e57400640 1 --2- 172.21.15.46:0/4025851577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee399260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:40.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e57e00640 1 --2- 172.21.15.46:0/4025851577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386bb0 0x55c8ee395700 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2e50019410 tx=0x7f2e500a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a680ff462401d82 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:40.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e4cc00640 1 -- 172.21.15.46:0/4025851577 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e50003d30 con 0x55c8ee386bb0 2024-03-29T08:25:40.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e4cc00640 1 -- 172.21.15.46:0/4025851577 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e50003e90 con 0x55c8ee386bb0 2024-03-29T08:25:40.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e4cc00640 1 -- 172.21.15.46:0/4025851577 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e50004260 con 0x55c8ee386bb0 2024-03-29T08:25:40.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 1 -- 172.21.15.46:0/4025851577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386bb0 msgr2=0x55c8ee395700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 1 --2- 172.21.15.46:0/4025851577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386bb0 0x55c8ee395700 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2e50019410 tx=0x7f2e500a7000 comp rx=0 tx=0).stop 2024-03-29T08:25:40.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 1 -- 172.21.15.46:0/4025851577 shutdown_connections 2024-03-29T08:25:40.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 1 --2- 172.21.15.46:0/4025851577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386bb0 0x55c8ee395700 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 1 --2- 172.21.15.46:0/4025851577 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386290 0x55c8ee3997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 1 --2- 172.21.15.46:0/4025851577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee399260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 1 -- 172.21.15.46:0/4025851577 >> 172.21.15.46:0/4025851577 conn(0x55c8ee339160 msgr2=0x55c8ee33c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:40.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 1 -- 172.21.15.46:0/4025851577 shutdown_connections 2024-03-29T08:25:40.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 1 -- 172.21.15.46:0/4025851577 wait complete. 2024-03-29T08:25:40.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 1 -- start start 2024-03-29T08:25:40.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 1 Processor -- start 2024-03-29T08:25:40.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee3992b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386290 0x55c8ee3e0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386bb0 0x55c8ee3e0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ee3345e0 con 0x55c8ee386bb0 2024-03-29T08:25:40.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8ee187f90 con 0x55c8ee33d1b0 2024-03-29T08:25:40.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8ee133520 con 0x55c8ee386290 2024-03-29T08:25:40.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e61655000 10 client.0 _ll_register_callbacks cb 0x55c8ee228870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:25:40.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e57e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386bb0 0x55c8ee3e0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e57e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386bb0 0x55c8ee3e0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56482/0 (socket says 172.21.15.46:56482) 2024-03-29T08:25:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e57e00640 1 -- 172.21.15.46:0/4088003659 learned_addr learned my addr 172.21.15.46:0/4088003659 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e57400640 1 --2- 172.21.15.46:0/4088003659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee3992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e4fe00640 1 --2- 172.21.15.46:0/4088003659 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386290 0x55c8ee3e0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e57e00640 1 -- 172.21.15.46:0/4088003659 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386290 msgr2=0x55c8ee3e0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e57e00640 1 --2- 172.21.15.46:0/4088003659 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386290 0x55c8ee3e0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e57e00640 1 -- 172.21.15.46:0/4088003659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 msgr2=0x55c8ee3992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e57e00640 1 --2- 172.21.15.46:0/4088003659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee3992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e57e00640 1 -- 172.21.15.46:0/4088003659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5815d790 con 0x55c8ee386bb0 2024-03-29T08:25:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e57400640 1 --2- 172.21.15.46:0/4088003659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee3992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:25:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e57e00640 1 --2- 172.21.15.46:0/4088003659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386bb0 0x55c8ee3e0bc0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f2e50019410 tx=0x7f2e5001b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afafb10330607544 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.596+0000 7f2e44c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:25:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e44c00640 1 -- 172.21.15.46:0/4088003659 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e50004710 con 0x55c8ee386bb0 2024-03-29T08:25:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e44c00640 1 -- 172.21.15.46:0/4088003659 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2e500a95c0 con 0x55c8ee386bb0 2024-03-29T08:25:40.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 1 -- 172.21.15.46:0/4088003659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8ee13a620 con 0x55c8ee386bb0 2024-03-29T08:25:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e44c00640 1 -- 172.21.15.46:0/4088003659 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e50004870 con 0x55c8ee386bb0 2024-03-29T08:25:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 10 client.7806 fetch_fsmap learned FSMap version 63 2024-03-29T08:25:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 1 -- 172.21.15.46:0/4088003659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55c8ede86fc0 con 0x55c8ee386bb0 2024-03-29T08:25:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e44c00640 1 -- 172.21.15.46:0/4088003659 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c8ee13a620 con 0x55c8ee386bb0 2024-03-29T08:25:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e44c00640 1 -- 172.21.15.46:0/4088003659 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 63) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f2e500a9780 con 0x55c8ee386bb0 2024-03-29T08:25:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 10 client.7806 fetch_fsmap finished waiting for FSMap version 63 2024-03-29T08:25:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 10 client.7806 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:25:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 20 client.7806 populate_metadata read hostname 'smithi046' 2024-03-29T08:25:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 1 --2- 172.21.15.46:0/4088003659 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55c8ee3e1100 0x55c8ee3e14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 4 client.7806 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-29T08:25:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 1 -- 172.21.15.46:0/4088003659 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55c8ee1894e0 con 0x55c8ee3e1100 2024-03-29T08:25:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e57400640 1 --2- 172.21.15.46:0/4088003659 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55c8ee3e1100 0x55c8ee3e14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e57400640 1 --2- 172.21.15.46:0/4088003659 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55c8ee3e1100 0x55c8ee3e14e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0d130908855b403 server_cookie=ad39c503a2867c63 in_seq=0 out_seq=0 2024-03-29T08:25:40.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e44c00640 10 client.7806 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e44c00640 1 -- 172.21.15.46:0/4088003659 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+419 (crc 0 0 0) 0x55c8ee1894e0 con 0x55c8ee3e1100 2024-03-29T08:25:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e44c00640 10 client.7806 handle_command_reply: tid=0 2024-03-29T08:25:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 1 client.7806 shutdown 2024-03-29T08:25:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 1 -- 172.21.15.46:0/4088003659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386bb0 msgr2=0x55c8ee3e0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 1 --2- 172.21.15.46:0/4088003659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386bb0 0x55c8ee3e0bc0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f2e50019410 tx=0x7f2e5001b040 comp rx=0 tx=0).stop 2024-03-29T08:25:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 1 -- 172.21.15.46:0/4088003659 shutdown_connections 2024-03-29T08:25:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 1 --2- 172.21.15.46:0/4088003659 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55c8ee3e1100 0x55c8ee3e14e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 1 --2- 172.21.15.46:0/4088003659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8ee386bb0 0x55c8ee3e0bc0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 1 --2- 172.21.15.46:0/4088003659 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55c8ee386290 0x55c8ee3e0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 1 --2- 172.21.15.46:0/4088003659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c8ee33d1b0 0x55c8ee3992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e44c00640 0 client.7806 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 1 -- 172.21.15.46:0/4088003659 >> 172.21.15.46:0/4088003659 conn(0x55c8ee339160 msgr2=0x55c8ee341690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 1 -- 172.21.15.46:0/4088003659 shutdown_connections 2024-03-29T08:25:40.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 1 -- 172.21.15.46:0/4088003659 wait complete. 2024-03-29T08:25:40.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e61655000 20 client.7806 trim_cache size 0 max 16384 2024-03-29T08:25:40.607 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:25:40.607 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-03-29T08:25:40.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:25:40.607 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "dir_frag", 2024-03-29T08:25:40.607 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 106641998, 2024-03-29T08:25:40.607 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627776, 2024-03-29T08:25:40.607 INFO:teuthology.orchestra.run.smithi046.stdout: "frag": "*", 2024-03-29T08:25:40.607 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/dir" 2024-03-29T08:25:40.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:25:40.607 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:25:40.608 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-03-29T08:25:40.608 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 1954060539, 2024-03-29T08:25:40.608 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627776, 2024-03-29T08:25:40.608 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/dir" 2024-03-29T08:25:40.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:25:40.608 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:25:40.608 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-03-29T08:25:40.608 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 3589788952, 2024-03-29T08:25:40.608 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627777, 2024-03-29T08:25:40.608 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/testdir/hardlink" 2024-03-29T08:25:40.608 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:25:40.608 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-03-29T08:25:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e5e200640 1 -- 172.21.15.46:0/50320252 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e34066690 msgr2=0x7f2e34068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e5e200640 1 --2- 172.21.15.46:0/50320252 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e34066690 0x7f2e34068b50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2e581f8490 tx=0x7f2e40009040 comp rx=0 tx=0).stop 2024-03-29T08:25:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e5e200640 1 -- 172.21.15.46:0/50320252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e580cbea0 msgr2=0x7f2e5815ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e5e200640 1 --2- 172.21.15.46:0/50320252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e580cbea0 0x7f2e5815ac70 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800c490 tx=0x7f2e4800c950 comp rx=0 tx=0).stop 2024-03-29T08:25:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e5e200640 1 -- 172.21.15.46:0/50320252 shutdown_connections 2024-03-29T08:25:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e5e200640 1 --2- 172.21.15.46:0/50320252 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e34066690 0x7f2e34068b50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e5e200640 1 --2- 172.21.15.46:0/50320252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e5815df90 0x7f2e580b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e5e200640 1 --2- 172.21.15.46:0/50320252 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2e58142a50 0x7f2e580b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e5e200640 1 --2- 172.21.15.46:0/50320252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e580cbea0 0x7f2e5815ac70 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e5e200640 1 -- 172.21.15.46:0/50320252 >> 172.21.15.46:0/50320252 conn(0x7f2e5806a390 msgr2=0x7f2e580cb660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:40.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e5e200640 1 -- 172.21.15.46:0/50320252 shutdown_connections 2024-03-29T08:25:40.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.600+0000 7f2e5e200640 1 -- 172.21.15.46:0/50320252 wait complete. 2024-03-29T08:25:40.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-29T08:25:40.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.760+0000 7f118b200640 1 Processor -- start 2024-03-29T08:25:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.760+0000 7f118b200640 1 -- start start 2024-03-29T08:25:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.760+0000 7f118b200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11840d0660 0x7f11840d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.760+0000 7f118b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f118415dc00 0x7f11840cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.760+0000 7f118b200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118415cfb0 con 0x7f118406b8b0 2024-03-29T08:25:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.760+0000 7f118b200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f118406c100 con 0x7f118415dc00 2024-03-29T08:25:40.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.760+0000 7f118b200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11840d0f80 con 0x7f11840d0660 2024-03-29T08:25:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.760+0000 7f1189200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f118415dc00 0x7f11840cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.760+0000 7f1189200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f118415dc00 0x7f11840cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56490/0 (socket says 172.21.15.46:56490) 2024-03-29T08:25:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.760+0000 7f1189200640 1 -- 172.21.15.46:0/4054510886 learned_addr learned my addr 172.21.15.46:0/4054510886 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.760+0000 7f1183e00640 1 --2- 172.21.15.46:0/4054510886 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11840d0660 0x7f11840d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.760+0000 7f1189200640 1 -- 172.21.15.46:0/4054510886 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11840d0660 msgr2=0x7f11840d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.760+0000 7f1189200640 1 --2- 172.21.15.46:0/4054510886 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11840d0660 0x7f11840d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.760+0000 7f1189200640 1 -- 172.21.15.46:0/4054510886 >> v1:172.21.15.73:6789/0 conn(0x7f118406b8b0 legacy=0x7f11840cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:40.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.760+0000 7f1189200640 1 -- 172.21.15.46:0/4054510886 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f118406be20 con 0x7f118415dc00 2024-03-29T08:25:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.764+0000 7f1189200640 1 --2- 172.21.15.46:0/4054510886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f118415dc00 0x7f11840cf7d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f117c009e50 tx=0x7f117c01c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e5350f8a6869ccd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.764+0000 7f1183400640 1 -- 172.21.15.46:0/4054510886 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c0a5780 con 0x7f118415dc00 2024-03-29T08:25:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.764+0000 7f1183400640 1 -- 172.21.15.46:0/4054510886 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117c0a58e0 con 0x7f118415dc00 2024-03-29T08:25:40.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.764+0000 7f1183400640 1 -- 172.21.15.46:0/4054510886 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c0a5c70 con 0x7f118415dc00 2024-03-29T08:25:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.764+0000 7f118b200640 1 -- 172.21.15.46:0/4054510886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f118415dc00 msgr2=0x7f11840cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.764+0000 7f118b200640 1 --2- 172.21.15.46:0/4054510886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f118415dc00 0x7f11840cf7d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f117c009e50 tx=0x7f117c01c990 comp rx=0 tx=0).stop 2024-03-29T08:25:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.764+0000 7f118b200640 1 -- 172.21.15.46:0/4054510886 shutdown_connections 2024-03-29T08:25:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.764+0000 7f118b200640 1 --2- 172.21.15.46:0/4054510886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f118415dc00 0x7f11840cf7d0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.764+0000 7f118b200640 1 --2- 172.21.15.46:0/4054510886 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11840d0660 0x7f11840d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.764+0000 7f118b200640 1 -- 172.21.15.46:0/4054510886 >> 172.21.15.46:0/4054510886 conn(0x7f1184089fd0 msgr2=0x7f118408a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.764+0000 7f118b200640 1 -- 172.21.15.46:0/4054510886 shutdown_connections 2024-03-29T08:25:40.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.764+0000 7f118b200640 1 -- 172.21.15.46:0/4054510886 wait complete. 2024-03-29T08:25:40.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f118b200640 1 Processor -- start 2024-03-29T08:25:40.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f118b200640 1 -- start start 2024-03-29T08:25:40.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f118b200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f118406b8b0 0x7f11840c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f118b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11840d0660 0x7f11841527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f118b200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f118415dc00 0x7f118413f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f118b200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11840b5f30 con 0x7f11840d0660 2024-03-29T08:25:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f118b200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11840c7480 con 0x7f118406b8b0 2024-03-29T08:25:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f118b200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11840b5080 con 0x7f118415dc00 2024-03-29T08:25:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f117b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11840d0660 0x7f11841527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f1189200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f118415dc00 0x7f118413f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f117b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11840d0660 0x7f11841527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56502/0 (socket says 172.21.15.46:56502) 2024-03-29T08:25:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f117b400640 1 -- 172.21.15.46:0/578992692 learned_addr learned my addr 172.21.15.46:0/578992692 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f1183e00640 1 --2- 172.21.15.46:0/578992692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f118406b8b0 0x7f11840c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f117b400640 1 -- 172.21.15.46:0/578992692 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f118415dc00 msgr2=0x7f118413f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f117b400640 1 --2- 172.21.15.46:0/578992692 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f118415dc00 0x7f118413f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f117b400640 1 -- 172.21.15.46:0/578992692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f118406b8b0 msgr2=0x7f11840c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f117b400640 1 --2- 172.21.15.46:0/578992692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f118406b8b0 0x7f11840c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f117b400640 1 -- 172.21.15.46:0/578992692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c009b30 con 0x7f11840d0660 2024-03-29T08:25:40.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f117b400640 1 --2- 172.21.15.46:0/578992692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11840d0660 0x7f11841527c0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f116c00b410 tx=0x7f116c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:40.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f1180c00640 1 -- 172.21.15.46:0/578992692 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c004020 con 0x7f11840d0660 2024-03-29T08:25:40.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f1180c00640 1 -- 172.21.15.46:0/578992692 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116c004180 con 0x7f11840d0660 2024-03-29T08:25:40.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f1180c00640 1 -- 172.21.15.46:0/578992692 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c011630 con 0x7f11840d0660 2024-03-29T08:25:40.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f118b200640 1 -- 172.21.15.46:0/578992692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11840b5320 con 0x7f11840d0660 2024-03-29T08:25:40.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f118b200640 1 -- 172.21.15.46:0/578992692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11840b58e0 con 0x7f11840d0660 2024-03-29T08:25:40.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.768+0000 7f118b200640 1 -- 172.21.15.46:0/578992692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1184066c90 con 0x7f11840d0660 2024-03-29T08:25:40.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.772+0000 7f1180c00640 1 -- 172.21.15.46:0/578992692 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f116c011790 con 0x7f11840d0660 2024-03-29T08:25:40.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.772+0000 7f1180c00640 1 --2- 172.21.15.46:0/578992692 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11600666c0 0x7f1160068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:40.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.772+0000 7f1180c00640 1 -- 172.21.15.46:0/578992692 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f116c083e80 con 0x7f11840d0660 2024-03-29T08:25:40.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.772+0000 7f1183e00640 1 --2- 172.21.15.46:0/578992692 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11600666c0 0x7f1160068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:40.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.772+0000 7f1183e00640 1 --2- 172.21.15.46:0/578992692 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11600666c0 0x7f1160068b80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1174007a60 tx=0x7f1174008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:40.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.776+0000 7f1180c00640 1 -- 172.21.15.46:0/578992692 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f116c0511a0 con 0x7f11840d0660 2024-03-29T08:25:40.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.968+0000 7f118b200640 1 -- 172.21.15.46:0/578992692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f118415c100 con 0x7f11840d0660 2024-03-29T08:25:40.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.968+0000 7f1180c00640 1 -- 172.21.15.46:0/578992692 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f116c0150c0 con 0x7f11840d0660 2024-03-29T08:25:40.971 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:25:40.972 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-03-29T08:25:40.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.972+0000 7f118b200640 1 -- 172.21.15.46:0/578992692 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11600666c0 msgr2=0x7f1160068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.972+0000 7f118b200640 1 --2- 172.21.15.46:0/578992692 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11600666c0 0x7f1160068b80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1174007a60 tx=0x7f1174008040 comp rx=0 tx=0).stop 2024-03-29T08:25:40.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.972+0000 7f118b200640 1 -- 172.21.15.46:0/578992692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11840d0660 msgr2=0x7f11841527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:40.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.972+0000 7f118b200640 1 --2- 172.21.15.46:0/578992692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11840d0660 0x7f11841527c0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f116c00b410 tx=0x7f116c00b8d0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.972+0000 7f118b200640 1 -- 172.21.15.46:0/578992692 shutdown_connections 2024-03-29T08:25:40.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.972+0000 7f118b200640 1 --2- 172.21.15.46:0/578992692 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11600666c0 0x7f1160068b80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.972+0000 7f118b200640 1 --2- 172.21.15.46:0/578992692 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f118415dc00 0x7f118413f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.972+0000 7f118b200640 1 --2- 172.21.15.46:0/578992692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11840d0660 0x7f11841527c0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.972+0000 7f118b200640 1 --2- 172.21.15.46:0/578992692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f118406b8b0 0x7f11840c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:40.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.972+0000 7f118b200640 1 -- 172.21.15.46:0/578992692 >> 172.21.15.46:0/578992692 conn(0x7f1184089fd0 msgr2=0x7f11840cad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:40.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.972+0000 7f118b200640 1 -- 172.21.15.46:0/578992692 shutdown_connections 2024-03-29T08:25:40.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:40.972+0000 7f118b200640 1 -- 172.21.15.46:0/578992692 wait complete. 2024-03-29T08:25:40.992 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-29T08:25:40.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:40.992 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-03-29T08:25:41.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.136+0000 7f70cde00640 1 Processor -- start 2024-03-29T08:25:41.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.136+0000 7f70cde00640 1 -- start start 2024-03-29T08:25:41.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.136+0000 7f70cde00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f70c80cec80 0x7f70c8141770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.136+0000 7f70cde00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c806cd80 con 0x7f70c8140f40 2024-03-29T08:25:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.136+0000 7f70cde00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c806d060 con 0x7f70c8141cb0 2024-03-29T08:25:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.136+0000 7f70cde00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c80cbff0 con 0x7f70c80cec80 2024-03-29T08:25:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.136+0000 7f70c6a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f70c8140f40 0x7f70c80cb8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52224/0 (socket says 172.21.15.46:52224) 2024-03-29T08:25:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.136+0000 7f70c6a00640 1 -- 172.21.15.46:0/1148050743 learned_addr learned my addr 172.21.15.46:0/1148050743 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:25:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.136+0000 7f70be000640 1 --2- 172.21.15.46:0/1148050743 >> v2:172.21.15.73:3300/0 conn(0x7f70c80cec80 0x7f70c8141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6000640 1 -- 172.21.15.46:0/1148050743 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228285093 0 0) 0x7f70c806cd80 con 0x7f70c8140f40 2024-03-29T08:25:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6000640 1 -- 172.21.15.46:0/1148050743 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70ac003530 con 0x7f70c8140f40 2024-03-29T08:25:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6000640 1 -- 172.21.15.46:0/1148050743 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1024204596 0 0) 0x7f70c806d060 con 0x7f70c8141cb0 2024-03-29T08:25:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6000640 1 -- 172.21.15.46:0/1148050743 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70c806cd80 con 0x7f70c8141cb0 2024-03-29T08:25:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6000640 1 -- 172.21.15.46:0/1148050743 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1578346499 0 0) 0x7f70ac003530 con 0x7f70c8140f40 2024-03-29T08:25:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6000640 1 -- 172.21.15.46:0/1148050743 >> v2:172.21.15.73:3300/0 conn(0x7f70c80cec80 msgr2=0x7f70c8141770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6000640 1 --2- 172.21.15.46:0/1148050743 >> v2:172.21.15.73:3300/0 conn(0x7f70c80cec80 0x7f70c8141770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6000640 1 -- 172.21.15.46:0/1148050743 >> v1:172.21.15.73:6789/0 conn(0x7f70c8141cb0 legacy=0x7f70c806db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6000640 1 -- 172.21.15.46:0/1148050743 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c806d340 con 0x7f70c8140f40 2024-03-29T08:25:41.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6000640 1 -- 172.21.15.46:0/1148050743 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f70b8001a00 con 0x7f70c8140f40 2024-03-29T08:25:41.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6000640 1 -- 172.21.15.46:0/1148050743 >> v1:172.21.15.46:6789/0 conn(0x7f70c8140f40 legacy=0x7f70c80cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6000640 1 --2- 172.21.15.46:0/1148050743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70ac003b10 0x7f70ac003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6000640 1 --2- 172.21.15.46:0/1148050743 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70ac0047d0 0x7f70ac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6000640 1 --2- 172.21.15.46:0/1148050743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac005190 0x7f70ac00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6000640 1 -- 172.21.15.46:0/1148050743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac00bfb0 con 0x7f70ac005190 2024-03-29T08:25:41.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6000640 1 -- 172.21.15.46:0/1148050743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac00c120 con 0x7f70ac003b10 2024-03-29T08:25:41.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6000640 1 -- 172.21.15.46:0/1148050743 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ac00c2e0 con 0x7f70ac0047d0 2024-03-29T08:25:41.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c6a00640 1 --2- 172.21.15.46:0/1148050743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70ac003b10 0x7f70ac003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c7400640 1 --2- 172.21.15.46:0/1148050743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac005190 0x7f70ac00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70be000640 1 --2- 172.21.15.46:0/1148050743 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70ac0047d0 0x7f70ac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c7400640 1 -- 172.21.15.46:0/1148050743 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70ac0047d0 msgr2=0x7f70ac004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c7400640 1 --2- 172.21.15.46:0/1148050743 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70ac0047d0 0x7f70ac004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c7400640 1 -- 172.21.15.46:0/1148050743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70ac003b10 msgr2=0x7f70ac003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c7400640 1 --2- 172.21.15.46:0/1148050743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70ac003b10 0x7f70ac003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.140+0000 7f70c7400640 1 -- 172.21.15.46:0/1148050743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ac003530 con 0x7f70ac005190 2024-03-29T08:25:41.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70c7400640 1 -- 172.21.15.46:0/1148050743 reap_dead start 2024-03-29T08:25:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70c7400640 1 --2- 172.21.15.46:0/1148050743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac005190 0x7f70ac00b990 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f70c81420b0 tx=0x7f70c001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36ae98b126c6868 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70c6000640 1 -- 172.21.15.46:0/1148050743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00a6040 con 0x7f70ac005190 2024-03-29T08:25:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70c6000640 1 -- 172.21.15.46:0/1148050743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c00052d0 con 0x7f70ac005190 2024-03-29T08:25:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70c6000640 1 -- 172.21.15.46:0/1148050743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c0003600 con 0x7f70ac005190 2024-03-29T08:25:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70cde00640 1 -- 172.21.15.46:0/1148050743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac005190 msgr2=0x7f70ac00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70cde00640 1 --2- 172.21.15.46:0/1148050743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac005190 0x7f70ac00b990 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f70c81420b0 tx=0x7f70c001c560 comp rx=0 tx=0).stop 2024-03-29T08:25:41.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70cde00640 1 -- 172.21.15.46:0/1148050743 shutdown_connections 2024-03-29T08:25:41.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70cde00640 1 --2- 172.21.15.46:0/1148050743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70ac005190 0x7f70ac00b990 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70cde00640 1 -- 172.21.15.46:0/1148050743 >> 172.21.15.46:0/1148050743 conn(0x7f70c80b42d0 msgr2=0x7f70c8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:41.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70cde00640 1 -- 172.21.15.46:0/1148050743 shutdown_connections 2024-03-29T08:25:41.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70cde00640 1 -- 172.21.15.46:0/1148050743 wait complete. 2024-03-29T08:25:41.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70cde00640 1 Processor -- start 2024-03-29T08:25:41.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70cde00640 1 -- start start 2024-03-29T08:25:41.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70cde00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70ac005190 0x7f70c81e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70cde00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c81e8000 0x7f70c8143340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70cde00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c813fcd0 0x7f70c8140130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70cde00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c8140b60 con 0x7f70c813fcd0 2024-03-29T08:25:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70cde00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c81e6d20 con 0x7f70c81e8000 2024-03-29T08:25:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70cde00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c81e7b60 con 0x7f70ac005190 2024-03-29T08:25:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c813fcd0 0x7f70c8140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c813fcd0 0x7f70c8140130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56512/0 (socket says 172.21.15.46:56512) 2024-03-29T08:25:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70c7400640 1 -- 172.21.15.46:0/106411345 learned_addr learned my addr 172.21.15.46:0/106411345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70be000640 1 --2- 172.21.15.46:0/106411345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c81e8000 0x7f70c8143340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.144+0000 7f70c6a00640 1 --2- 172.21.15.46:0/106411345 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70ac005190 0x7f70c81e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.148+0000 7f70c7400640 1 -- 172.21.15.46:0/106411345 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70ac005190 msgr2=0x7f70c81e67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.148+0000 7f70c7400640 1 --2- 172.21.15.46:0/106411345 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70ac005190 0x7f70c81e67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.148+0000 7f70c7400640 1 -- 172.21.15.46:0/106411345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c81e8000 msgr2=0x7f70c8143340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.148+0000 7f70c7400640 1 --2- 172.21.15.46:0/106411345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c81e8000 0x7f70c8143340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.148+0000 7f70c7400640 1 -- 172.21.15.46:0/106411345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c8140670 con 0x7f70c813fcd0 2024-03-29T08:25:41.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.148+0000 7f70c7400640 1 --2- 172.21.15.46:0/106411345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c813fcd0 0x7f70c8140130 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f70c001ce00 tx=0x7f70ac005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:41.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.148+0000 7f70bfe00640 1 -- 172.21.15.46:0/106411345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c0005440 con 0x7f70c813fcd0 2024-03-29T08:25:41.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.148+0000 7f70bfe00640 1 -- 172.21.15.46:0/106411345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c0007030 con 0x7f70c813fcd0 2024-03-29T08:25:41.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.148+0000 7f70bfe00640 1 -- 172.21.15.46:0/106411345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00073e0 con 0x7f70c813fcd0 2024-03-29T08:25:41.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.148+0000 7f70cde00640 1 -- 172.21.15.46:0/106411345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c81e6f50 con 0x7f70c813fcd0 2024-03-29T08:25:41.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.148+0000 7f70cde00640 1 -- 172.21.15.46:0/106411345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c81e7510 con 0x7f70c813fcd0 2024-03-29T08:25:41.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.148+0000 7f70bfe00640 1 -- 172.21.15.46:0/106411345 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f70c0007540 con 0x7f70c813fcd0 2024-03-29T08:25:41.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.148+0000 7f70bfe00640 1 --2- 172.21.15.46:0/106411345 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f70a40666c0 0x7f70a4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.148+0000 7f70bfe00640 1 -- 172.21.15.46:0/106411345 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f70c011c8c0 con 0x7f70c813fcd0 2024-03-29T08:25:41.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70c6a00640 1 --2- 172.21.15.46:0/106411345 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f70a40666c0 0x7f70a4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70d135a000 1 Processor -- start 2024-03-29T08:25:41.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70d135a000 1 -- start start 2024-03-29T08:25:41.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70c6a00640 1 --2- 172.21.15.46:0/106411345 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f70a40666c0 0x7f70a4068b80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f70c815cbf0 tx=0x7f70b80050d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:41.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70d135a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbed30 0x56369cfbf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70d135a000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cfbf650 0x56369cf75a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70d135a000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cf75f60 0x56369cf76340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70d135a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369ce3d170 con 0x56369cfbed30 2024-03-29T08:25:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70d135a000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369cdcad80 con 0x56369cf75f60 2024-03-29T08:25:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70d135a000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x56369cdc8350 con 0x56369cfbf650 2024-03-29T08:25:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70c6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbed30 0x56369cfbf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70c6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbed30 0x56369cfbf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56528/0 (socket says 172.21.15.46:56528) 2024-03-29T08:25:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70c6a00640 1 -- 172.21.15.46:0/2263862975 learned_addr learned my addr 172.21.15.46:0/2263862975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70be000640 1 --2- 172.21.15.46:0/2263862975 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cfbf650 0x56369cf75a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70c7400640 1 --2- 172.21.15.46:0/2263862975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cf75f60 0x56369cf76340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70c6a00640 1 -- 172.21.15.46:0/2263862975 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cfbf650 msgr2=0x56369cf75a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70c6a00640 1 --2- 172.21.15.46:0/2263862975 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cfbf650 0x56369cf75a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70c6a00640 1 -- 172.21.15.46:0/2263862975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cf75f60 msgr2=0x56369cf76340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70c6a00640 1 --2- 172.21.15.46:0/2263862975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cf75f60 0x56369cf76340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70c6a00640 1 -- 172.21.15.46:0/2263862975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56369cac0070 con 0x56369cfbed30 2024-03-29T08:25:41.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70c7400640 1 --2- 172.21.15.46:0/2263862975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cf75f60 0x56369cf76340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:25:41.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70c6a00640 1 --2- 172.21.15.46:0/2263862975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbed30 0x56369cfbf110 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f70b80175b0 tx=0x7f70b8002100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde44583f391950 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:41.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70b7e00640 1 -- 172.21.15.46:0/2263862975 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b80083a0 con 0x56369cfbed30 2024-03-29T08:25:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70b7e00640 1 -- 172.21.15.46:0/2263862975 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b8008500 con 0x56369cfbed30 2024-03-29T08:25:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70b7e00640 1 -- 172.21.15.46:0/2263862975 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b8008670 con 0x56369cfbed30 2024-03-29T08:25:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70d135a000 1 -- 172.21.15.46:0/2263862975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbed30 msgr2=0x56369cfbf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70d135a000 1 --2- 172.21.15.46:0/2263862975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbed30 0x56369cfbf110 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f70b80175b0 tx=0x7f70b8002100 comp rx=0 tx=0).stop 2024-03-29T08:25:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70d135a000 1 -- 172.21.15.46:0/2263862975 shutdown_connections 2024-03-29T08:25:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70d135a000 1 --2- 172.21.15.46:0/2263862975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cf75f60 0x56369cf76340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70d135a000 1 --2- 172.21.15.46:0/2263862975 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cfbf650 0x56369cf75a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70d135a000 1 --2- 172.21.15.46:0/2263862975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbed30 0x56369cfbf110 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70d135a000 1 -- 172.21.15.46:0/2263862975 >> 172.21.15.46:0/2263862975 conn(0x56369cf71f10 msgr2=0x56369cf74300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70d135a000 1 -- 172.21.15.46:0/2263862975 shutdown_connections 2024-03-29T08:25:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.152+0000 7f70d135a000 1 -- 172.21.15.46:0/2263862975 wait complete. 2024-03-29T08:25:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 1 -- start start 2024-03-29T08:25:41.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 1 Processor -- start 2024-03-29T08:25:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cf75f60 0x56369cf7b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbed30 0x56369cfbfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cfbf650 0x56369cfc0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369cdc3c40 con 0x56369cfbed30 2024-03-29T08:25:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369cd62fa0 con 0x56369cfbf650 2024-03-29T08:25:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x56369cd811c0 con 0x56369cf75f60 2024-03-29T08:25:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 10 client.0 _ll_register_callbacks cb 0x56369cf61060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:25:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70be000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbed30 0x56369cfbfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70be000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbed30 0x56369cfbfc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56536/0 (socket says 172.21.15.46:56536) 2024-03-29T08:25:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70be000640 1 -- 172.21.15.46:0/2928019835 learned_addr learned my addr 172.21.15.46:0/2928019835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70c6a00640 1 --2- 172.21.15.46:0/2928019835 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cf75f60 0x56369cf7b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70c7400640 1 --2- 172.21.15.46:0/2928019835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cfbf650 0x56369cfc0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70be000640 1 -- 172.21.15.46:0/2928019835 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cf75f60 msgr2=0x56369cf7b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70be000640 1 --2- 172.21.15.46:0/2928019835 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cf75f60 0x56369cf7b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70be000640 1 -- 172.21.15.46:0/2928019835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cfbf650 msgr2=0x56369cfc0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70be000640 1 --2- 172.21.15.46:0/2928019835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cfbf650 0x56369cfc0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70be000640 1 -- 172.21.15.46:0/2928019835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c806d340 con 0x56369cfbed30 2024-03-29T08:25:41.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70be000640 1 --2- 172.21.15.46:0/2928019835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbed30 0x56369cfbfc40 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f70b00166b0 tx=0x7f70b0016ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2791219b3426b3b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:41.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:25:41.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70b4c00640 1 -- 172.21.15.46:0/2928019835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b0004240 con 0x56369cfbed30 2024-03-29T08:25:41.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70b4c00640 1 -- 172.21.15.46:0/2928019835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f70b00043a0 con 0x56369cfbed30 2024-03-29T08:25:41.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 1 -- 172.21.15.46:0/2928019835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56369cdc0a20 con 0x56369cfbed30 2024-03-29T08:25:41.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70b4c00640 1 -- 172.21.15.46:0/2928019835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70b00059a0 con 0x56369cfbed30 2024-03-29T08:25:41.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 10 client.7824 fetch_fsmap learned FSMap version 63 2024-03-29T08:25:41.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 1 -- 172.21.15.46:0/2928019835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x56369cd88ba0 con 0x56369cfbed30 2024-03-29T08:25:41.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70b4c00640 1 -- 172.21.15.46:0/2928019835 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56369cdc0a20 con 0x56369cfbed30 2024-03-29T08:25:41.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70b4c00640 1 -- 172.21.15.46:0/2928019835 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 63) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f70b0011040 con 0x56369cfbed30 2024-03-29T08:25:41.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 10 client.7824 fetch_fsmap finished waiting for FSMap version 63 2024-03-29T08:25:41.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 10 client.7824 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:25:41.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 20 client.7824 populate_metadata read hostname 'smithi046' 2024-03-29T08:25:41.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 1 --2- 172.21.15.46:0/2928019835 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x56369cfc6250 0x56369cfc6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 4 client.7824 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:25:41.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70d135a000 1 -- 172.21.15.46:0/2928019835 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56369cdc2590 con 0x56369cfc6250 2024-03-29T08:25:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70c6a00640 1 --2- 172.21.15.46:0/2928019835 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x56369cfc6250 0x56369cfc6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70c6a00640 1 --2- 172.21.15.46:0/2928019835 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x56369cfc6250 0x56369cfc6660 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26f51f35b1d41139 server_cookie=d199739e9c88eecd in_seq=0 out_seq=0 2024-03-29T08:25:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.156+0000 7f70b4c00640 10 client.7824 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.160+0000 7f70b4c00640 1 -- 172.21.15.46:0/2928019835 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56369cdc2590 con 0x56369cfc6250 2024-03-29T08:25:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.160+0000 7f70b4c00640 10 client.7824 handle_command_reply: tid=0 2024-03-29T08:25:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.160+0000 7f70d135a000 1 client.7824 shutdown 2024-03-29T08:25:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.160+0000 7f70d135a000 1 -- 172.21.15.46:0/2928019835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbed30 msgr2=0x56369cfbfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.160+0000 7f70d135a000 1 --2- 172.21.15.46:0/2928019835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbed30 0x56369cfbfc40 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f70b00166b0 tx=0x7f70b0016ab0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.160+0000 7f70d135a000 1 -- 172.21.15.46:0/2928019835 shutdown_connections 2024-03-29T08:25:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.160+0000 7f70d135a000 1 --2- 172.21.15.46:0/2928019835 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x56369cfc6250 0x56369cfc6660 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.160+0000 7f70d135a000 1 --2- 172.21.15.46:0/2928019835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cfbf650 0x56369cfc0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.160+0000 7f70d135a000 1 --2- 172.21.15.46:0/2928019835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbed30 0x56369cfbfc40 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.160+0000 7f70d135a000 1 --2- 172.21.15.46:0/2928019835 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cf75f60 0x56369cf7b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.160+0000 7f70b4c00640 0 client.7824 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.160+0000 7f70d135a000 1 -- 172.21.15.46:0/2928019835 >> 172.21.15.46:0/2928019835 conn(0x56369cf71f10 msgr2=0x56369cf74b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.160+0000 7f70d135a000 1 -- 172.21.15.46:0/2928019835 shutdown_connections 2024-03-29T08:25:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.160+0000 7f70d135a000 1 -- 172.21.15.46:0/2928019835 wait complete. 2024-03-29T08:25:41.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.160+0000 7f70d135a000 20 client.7824 trim_cache size 0 max 16384 2024-03-29T08:25:41.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.172+0000 7f70d135a000 1 Processor -- start 2024-03-29T08:25:41.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.172+0000 7f70d135a000 1 -- start start 2024-03-29T08:25:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cf75f60 0x56369cfc0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cfbed30 0x56369cfc0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbf650 0x56369cfc14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369ca80cf0 con 0x56369cfbf650 2024-03-29T08:25:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369cf78b50 con 0x56369cf75f60 2024-03-29T08:25:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x56369cdbf430 con 0x56369cfbed30 2024-03-29T08:25:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbf650 0x56369cfc14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbf650 0x56369cfc14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56542/0 (socket says 172.21.15.46:56542) 2024-03-29T08:25:41.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70c7400640 1 -- 172.21.15.46:0/3254406402 learned_addr learned my addr 172.21.15.46:0/3254406402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:41.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70c7400640 1 -- 172.21.15.46:0/3254406402 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cfbed30 msgr2=0x56369cfc0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70c7400640 1 --2- 172.21.15.46:0/3254406402 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cfbed30 0x56369cfc0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70c6a00640 1 --2- 172.21.15.46:0/3254406402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cf75f60 0x56369cfc0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70c7400640 1 -- 172.21.15.46:0/3254406402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cf75f60 msgr2=0x56369cfc0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70c7400640 1 --2- 172.21.15.46:0/3254406402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cf75f60 0x56369cfc0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70c7400640 1 -- 172.21.15.46:0/3254406402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56369cd6eef0 con 0x56369cfbf650 2024-03-29T08:25:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70c7400640 1 --2- 172.21.15.46:0/3254406402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbf650 0x56369cfc14d0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f70c00ed810 tx=0x7f70c00ed750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b772f3fcdabf7e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70bcc00640 1 -- 172.21.15.46:0/3254406402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00ed880 con 0x56369cfbf650 2024-03-29T08:25:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70bcc00640 1 -- 172.21.15.46:0/3254406402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c00f16b0 con 0x56369cfbf650 2024-03-29T08:25:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70bcc00640 1 -- 172.21.15.46:0/3254406402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00f5c70 con 0x56369cfbf650 2024-03-29T08:25:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 -- 172.21.15.46:0/3254406402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbf650 msgr2=0x56369cfc14d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 --2- 172.21.15.46:0/3254406402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbf650 0x56369cfc14d0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f70c00ed810 tx=0x7f70c00ed750 comp rx=0 tx=0).stop 2024-03-29T08:25:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 -- 172.21.15.46:0/3254406402 shutdown_connections 2024-03-29T08:25:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 --2- 172.21.15.46:0/3254406402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cfbf650 0x56369cfc14d0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 --2- 172.21.15.46:0/3254406402 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cfbed30 0x56369cfc0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 --2- 172.21.15.46:0/3254406402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cf75f60 0x56369cfc0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 -- 172.21.15.46:0/3254406402 >> 172.21.15.46:0/3254406402 conn(0x56369cf71f10 msgr2=0x56369cf7a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 -- 172.21.15.46:0/3254406402 shutdown_connections 2024-03-29T08:25:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 -- 172.21.15.46:0/3254406402 wait complete. 2024-03-29T08:25:41.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 -- start start 2024-03-29T08:25:41.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 Processor -- start 2024-03-29T08:25:41.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cf75f60 0x56369d019e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cfbed30 0x56369cfc3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cfbf650 0x56369cfc42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369cdc1450 con 0x56369cf75f60 2024-03-29T08:25:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56369cd84090 con 0x56369cfbed30 2024-03-29T08:25:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x56369cdc67a0 con 0x56369cfbf650 2024-03-29T08:25:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70d135a000 10 client.0 _ll_register_callbacks cb 0x56369ce3d0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:25:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70c6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cf75f60 0x56369d019e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70c6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cf75f60 0x56369d019e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56558/0 (socket says 172.21.15.46:56558) 2024-03-29T08:25:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.176+0000 7f70c6a00640 1 -- 172.21.15.46:0/1963589343 learned_addr learned my addr 172.21.15.46:0/1963589343 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70c7400640 1 --2- 172.21.15.46:0/1963589343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cfbf650 0x56369cfc42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70be000640 1 --2- 172.21.15.46:0/1963589343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cfbed30 0x56369cfc3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70c6a00640 1 -- 172.21.15.46:0/1963589343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cfbf650 msgr2=0x56369cfc42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70c6a00640 1 --2- 172.21.15.46:0/1963589343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cfbf650 0x56369cfc42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70c6a00640 1 -- 172.21.15.46:0/1963589343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cfbed30 msgr2=0x56369cfc3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70c6a00640 1 --2- 172.21.15.46:0/1963589343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cfbed30 0x56369cfc3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70c6a00640 1 -- 172.21.15.46:0/1963589343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c81e7510 con 0x56369cf75f60 2024-03-29T08:25:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70c6a00640 1 --2- 172.21.15.46:0/1963589343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cf75f60 0x56369d019e80 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f70b8004ef0 tx=0x7f70b8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625657009b7c14c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70b4c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:25:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70b4c00640 1 -- 172.21.15.46:0/1963589343 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b8035070 con 0x56369cf75f60 2024-03-29T08:25:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70b4c00640 1 -- 172.21.15.46:0/1963589343 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f70b803e020 con 0x56369cf75f60 2024-03-29T08:25:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 1 -- 172.21.15.46:0/1963589343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56369cd805f0 con 0x56369cf75f60 2024-03-29T08:25:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70b4c00640 1 -- 172.21.15.46:0/1963589343 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70b8039730 con 0x56369cf75f60 2024-03-29T08:25:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 10 client.7830 fetch_fsmap learned FSMap version 63 2024-03-29T08:25:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 1 -- 172.21.15.46:0/1963589343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x56369cd62a60 con 0x56369cf75f60 2024-03-29T08:25:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70b4c00640 1 -- 172.21.15.46:0/1963589343 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56369cd805f0 con 0x56369cf75f60 2024-03-29T08:25:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70b4c00640 1 -- 172.21.15.46:0/1963589343 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 63) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f70b8039450 con 0x56369cf75f60 2024-03-29T08:25:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 10 client.7830 fetch_fsmap finished waiting for FSMap version 63 2024-03-29T08:25:41.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 10 client.7830 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:25:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 20 client.7830 populate_metadata read hostname 'smithi046' 2024-03-29T08:25:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 1 --2- 172.21.15.46:0/1963589343 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x56369cfc4830 0x56369cfc4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 4 client.7830 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-03-29T08:25:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 1 -- 172.21.15.46:0/1963589343 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x56369cd82410 con 0x56369cfc4830 2024-03-29T08:25:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70be000640 1 --2- 172.21.15.46:0/1963589343 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x56369cfc4830 0x56369cfc4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70b4c00640 10 client.7830 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70be000640 1 --2- 172.21.15.46:0/1963589343 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x56369cfc4830 0x56369cfc4c40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc192c6c01058d79 server_cookie=5d053abd1093594b in_seq=0 out_seq=0 2024-03-29T08:25:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70b4c00640 1 -- 172.21.15.46:0/1963589343 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56369cd82410 con 0x56369cfc4830 2024-03-29T08:25:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70b4c00640 10 client.7830 handle_command_reply: tid=0 2024-03-29T08:25:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 1 client.7830 shutdown 2024-03-29T08:25:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 1 -- 172.21.15.46:0/1963589343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cf75f60 msgr2=0x56369d019e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 1 --2- 172.21.15.46:0/1963589343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cf75f60 0x56369d019e80 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f70b8004ef0 tx=0x7f70b8018000 comp rx=0 tx=0).stop 2024-03-29T08:25:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 1 -- 172.21.15.46:0/1963589343 shutdown_connections 2024-03-29T08:25:41.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 1 --2- 172.21.15.46:0/1963589343 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x56369cfc4830 0x56369cfc4c40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 1 --2- 172.21.15.46:0/1963589343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56369cfbed30 0x56369cfc3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 1 --2- 172.21.15.46:0/1963589343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56369cfbf650 0x56369cfc42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 1 --2- 172.21.15.46:0/1963589343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369cf75f60 0x56369d019e80 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70b4c00640 0 client.7830 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:41.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 1 -- 172.21.15.46:0/1963589343 >> 172.21.15.46:0/1963589343 conn(0x56369cf71f10 msgr2=0x56369cf74300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:41.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 1 -- 172.21.15.46:0/1963589343 shutdown_connections 2024-03-29T08:25:41.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 1 -- 172.21.15.46:0/1963589343 wait complete. 2024-03-29T08:25:41.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f70d135a000 20 client.7830 trim_cache size 0 max 16384 2024-03-29T08:25:41.188 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-03-29T08:25:41.180+0000 7f2799e00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-03-29T08:24:51.226697+0000 1=1+0)->f(v1 m2024-03-29T08:24:51.226697+0000) n(v0 rc2024-03-29T08:24:51.337857+0000 2=1+1)->n(v0 rc2024-03-29T08:24:51.337857+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x563bf804a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-29T08:24:51.226697+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-29T08:24:51.337857+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-03-29T08:25:41.188 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:25:41.188 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-29T08:25:41.188 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "164e6815-ed4f-4b27-bc0e-df097fc34e78", 2024-03-29T08:25:41.188 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-29T08:25:41.188 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:25:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.184+0000 7f70cde00640 1 -- 172.21.15.46:0/106411345 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f70a40666c0 msgr2=0x7f70a4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.184+0000 7f70cde00640 1 --2- 172.21.15.46:0/106411345 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f70a40666c0 0x7f70a4068b80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f70c815cbf0 tx=0x7f70b80050d0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.184+0000 7f70cde00640 1 -- 172.21.15.46:0/106411345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c813fcd0 msgr2=0x7f70c8140130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.184+0000 7f70cde00640 1 --2- 172.21.15.46:0/106411345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c813fcd0 0x7f70c8140130 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f70c001ce00 tx=0x7f70ac005800 comp rx=0 tx=0).stop 2024-03-29T08:25:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.184+0000 7f70cde00640 1 -- 172.21.15.46:0/106411345 shutdown_connections 2024-03-29T08:25:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.184+0000 7f70cde00640 1 --2- 172.21.15.46:0/106411345 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f70a40666c0 0x7f70a4068b80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.184+0000 7f70cde00640 1 --2- 172.21.15.46:0/106411345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c81e8000 0x7f70c8143340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.184+0000 7f70cde00640 1 --2- 172.21.15.46:0/106411345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70c813fcd0 0x7f70c8140130 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.184+0000 7f70cde00640 1 --2- 172.21.15.46:0/106411345 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f70ac005190 0x7f70c81e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.184+0000 7f70cde00640 1 -- 172.21.15.46:0/106411345 >> 172.21.15.46:0/106411345 conn(0x7f70c80b42d0 msgr2=0x7f70c80b2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:41.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.184+0000 7f70cde00640 1 -- 172.21.15.46:0/106411345 shutdown_connections 2024-03-29T08:25:41.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.184+0000 7f70cde00640 1 -- 172.21.15.46:0/106411345 wait complete. 2024-03-29T08:25:41.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:25:41.203 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-29T08:25:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.344+0000 7fc07d800640 1 Processor -- start 2024-03-29T08:25:41.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.344+0000 7fc07d800640 1 -- start start 2024-03-29T08:25:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.348+0000 7fc07d800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc07815df90 0x7fc0780ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.348+0000 7fc07d800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc07815d260 con 0x7fc0780cbea0 2024-03-29T08:25:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.348+0000 7fc07d800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0780aefd0 con 0x7fc078142a50 2024-03-29T08:25:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.348+0000 7fc07d800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07815d4b0 con 0x7fc07815df90 2024-03-29T08:25:41.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.348+0000 7fc077400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc0780cbea0 0x7fc0780ae7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:52230/0 (socket says 172.21.15.46:52230) 2024-03-29T08:25:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.348+0000 7fc077400640 1 -- 172.21.15.46:0/1590457053 learned_addr learned my addr 172.21.15.46:0/1590457053 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:25:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.348+0000 7fc06fe00640 1 --2- 172.21.15.46:0/1590457053 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc07815df90 0x7fc0780ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.348+0000 7fc076000640 1 -- 172.21.15.46:0/1590457053 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395949960 0 0) 0x7fc07815d260 con 0x7fc0780cbea0 2024-03-29T08:25:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.348+0000 7fc076000640 1 -- 172.21.15.46:0/1590457053 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc05c003530 con 0x7fc0780cbea0 2024-03-29T08:25:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.348+0000 7fc076000640 1 -- 172.21.15.46:0/1590457053 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449621971 0 0) 0x7fc0780aefd0 con 0x7fc078142a50 2024-03-29T08:25:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.348+0000 7fc076000640 1 -- 172.21.15.46:0/1590457053 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc07815d260 con 0x7fc078142a50 2024-03-29T08:25:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.348+0000 7fc06fe00640 1 -- 172.21.15.46:0/1590457053 >> v1:172.21.15.73:6789/0 conn(0x7fc078142a50 legacy=0x7fc0780cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.348+0000 7fc06fe00640 1 -- 172.21.15.46:0/1590457053 >> v1:172.21.15.46:6789/0 conn(0x7fc0780cbea0 legacy=0x7fc0780ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.348+0000 7fc06fe00640 1 -- 172.21.15.46:0/1590457053 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07815d790 con 0x7fc07815df90 2024-03-29T08:25:41.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.352+0000 7fc06fe00640 1 --2- 172.21.15.46:0/1590457053 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc07815df90 0x7fc0780ae280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc0600097f0 tx=0x7fc06001c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=442ebb9c9f849d46 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:41.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.352+0000 7fc076000640 1 -- 172.21.15.46:0/1590457053 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0600ab600 con 0x7fc07815df90 2024-03-29T08:25:41.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.352+0000 7fc076000640 1 -- 172.21.15.46:0/1590457053 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0600a7050 con 0x7fc07815df90 2024-03-29T08:25:41.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.352+0000 7fc076000640 1 -- 172.21.15.46:0/1590457053 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0600ab980 con 0x7fc07815df90 2024-03-29T08:25:41.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.352+0000 7fc07d800640 1 -- 172.21.15.46:0/1590457053 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc07815df90 msgr2=0x7fc0780ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.352+0000 7fc07d800640 1 --2- 172.21.15.46:0/1590457053 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc07815df90 0x7fc0780ae280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc0600097f0 tx=0x7fc06001c450 comp rx=0 tx=0).stop 2024-03-29T08:25:41.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.352+0000 7fc07d800640 1 -- 172.21.15.46:0/1590457053 shutdown_connections 2024-03-29T08:25:41.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.352+0000 7fc07d800640 1 --2- 172.21.15.46:0/1590457053 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc07815df90 0x7fc0780ae280 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.352+0000 7fc07d800640 1 -- 172.21.15.46:0/1590457053 >> 172.21.15.46:0/1590457053 conn(0x7fc07806a390 msgr2=0x7fc0780cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:41.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.352+0000 7fc07d800640 1 -- 172.21.15.46:0/1590457053 shutdown_connections 2024-03-29T08:25:41.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.352+0000 7fc07d800640 1 -- 172.21.15.46:0/1590457053 wait complete. 2024-03-29T08:25:41.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.352+0000 7fc07d800640 1 Processor -- start 2024-03-29T08:25:41.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.352+0000 7fc07d800640 1 -- start start 2024-03-29T08:25:41.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc07d800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0780cbea0 0x7fc0781571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc07d800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc078142a50 0x7fc078156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc07d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07815df90 0x7fc078156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc07d800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0781376a0 con 0x7fc07815df90 2024-03-29T08:25:41.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc07d800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc078152980 con 0x7fc0780cbea0 2024-03-29T08:25:41.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc07d800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07815c710 con 0x7fc078142a50 2024-03-29T08:25:41.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc077400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07815df90 0x7fc078156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc077400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07815df90 0x7fc078156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56568/0 (socket says 172.21.15.46:56568) 2024-03-29T08:25:41.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc077400640 1 -- 172.21.15.46:0/2058016006 learned_addr learned my addr 172.21.15.46:0/2058016006 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:41.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc076a00640 1 --2- 172.21.15.46:0/2058016006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0780cbea0 0x7fc0781571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc06fe00640 1 --2- 172.21.15.46:0/2058016006 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc078142a50 0x7fc078156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc077400640 1 -- 172.21.15.46:0/2058016006 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc078142a50 msgr2=0x7fc078156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc077400640 1 --2- 172.21.15.46:0/2058016006 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc078142a50 0x7fc078156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc077400640 1 -- 172.21.15.46:0/2058016006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0780cbea0 msgr2=0x7fc0781571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc077400640 1 --2- 172.21.15.46:0/2058016006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0780cbea0 0x7fc0781571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc077400640 1 -- 172.21.15.46:0/2058016006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc060009420 con 0x7fc07815df90 2024-03-29T08:25:41.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc077400640 1 --2- 172.21.15.46:0/2058016006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07815df90 0x7fc078156b50 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc07000cb10 tx=0x7fc07000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:41.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc06ea00640 1 -- 172.21.15.46:0/2058016006 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07000ac70 con 0x7fc07815df90 2024-03-29T08:25:41.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc06ea00640 1 -- 172.21.15.46:0/2058016006 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07000adf0 con 0x7fc07815df90 2024-03-29T08:25:41.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc07d800640 1 -- 172.21.15.46:0/2058016006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07815c9f0 con 0x7fc07815df90 2024-03-29T08:25:41.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc06ea00640 1 -- 172.21.15.46:0/2058016006 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc070012640 con 0x7fc07815df90 2024-03-29T08:25:41.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc07d800640 1 -- 172.21.15.46:0/2058016006 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07813a080 con 0x7fc07815df90 2024-03-29T08:25:41.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.356+0000 7fc06ea00640 1 -- 172.21.15.46:0/2058016006 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc070007070 con 0x7fc07815df90 2024-03-29T08:25:41.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc06ea00640 1 --2- 172.21.15.46:0/2058016006 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0540666c0 0x7fc054068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc06ea00640 1 -- 172.21.15.46:0/2058016006 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc070084190 con 0x7fc07815df90 2024-03-29T08:25:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc076a00640 1 --2- 172.21.15.46:0/2058016006 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0540666c0 0x7fc054068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc080c64000 1 Processor -- start 2024-03-29T08:25:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc080c64000 1 -- start start 2024-03-29T08:25:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc076a00640 1 --2- 172.21.15.46:0/2058016006 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0540666c0 0x7fc054068b80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc0680011f0 tx=0x7fc068009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc080c64000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c4114e60 0x5573c4115240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc080c64000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4115780 0x5573c40c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc080c64000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c40c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc080c64000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c3f8f630 con 0x5573c40c8e10 2024-03-29T08:25:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc080c64000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c3f1cbd0 con 0x5573c4114e60 2024-03-29T08:25:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc080c64000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573c3f1a1a0 con 0x5573c4115780 2024-03-29T08:25:41.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc077400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c40c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc076a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c4114e60 0x5573c4115240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc077400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c40c91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56576/0 (socket says 172.21.15.46:56576) 2024-03-29T08:25:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc077400640 1 -- 172.21.15.46:0/579373306 learned_addr learned my addr 172.21.15.46:0/579373306 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc06fe00640 1 --2- 172.21.15.46:0/579373306 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4115780 0x5573c40c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc077400640 1 -- 172.21.15.46:0/579373306 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4115780 msgr2=0x5573c40c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc077400640 1 --2- 172.21.15.46:0/579373306 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4115780 0x5573c40c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc077400640 1 -- 172.21.15.46:0/579373306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c4114e60 msgr2=0x5573c4115240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc077400640 1 --2- 172.21.15.46:0/579373306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c4114e60 0x5573c4115240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc077400640 1 -- 172.21.15.46:0/579373306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573c3c11e20 con 0x5573c40c8e10 2024-03-29T08:25:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc077400640 1 --2- 172.21.15.46:0/579373306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c40c91f0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc07005cf80 tx=0x7fc070055070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef43426494a2b67 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:41.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc067400640 1 -- 172.21.15.46:0/579373306 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc070053510 con 0x5573c40c8e10 2024-03-29T08:25:41.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc067400640 1 -- 172.21.15.46:0/579373306 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07005b070 con 0x5573c40c8e10 2024-03-29T08:25:41.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc067400640 1 -- 172.21.15.46:0/579373306 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07005b220 con 0x5573c40c8e10 2024-03-29T08:25:41.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc080c64000 1 -- 172.21.15.46:0/579373306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 msgr2=0x5573c40c91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.360+0000 7fc080c64000 1 --2- 172.21.15.46:0/579373306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c40c91f0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc07005cf80 tx=0x7fc070055070 comp rx=0 tx=0).stop 2024-03-29T08:25:41.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 -- 172.21.15.46:0/579373306 shutdown_connections 2024-03-29T08:25:41.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 --2- 172.21.15.46:0/579373306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c40c91f0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 --2- 172.21.15.46:0/579373306 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4115780 0x5573c40c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 --2- 172.21.15.46:0/579373306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c4114e60 0x5573c4115240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 -- 172.21.15.46:0/579373306 >> 172.21.15.46:0/579373306 conn(0x5573c3ec5160 msgr2=0x5573c40f3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:41.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 -- 172.21.15.46:0/579373306 shutdown_connections 2024-03-29T08:25:41.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 -- 172.21.15.46:0/579373306 wait complete. 2024-03-29T08:25:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 -- start start 2024-03-29T08:25:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 Processor -- start 2024-03-29T08:25:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c411c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c4114e60 0x5573c411c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4115780 0x5573c4115b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c3f15a90 con 0x5573c40c8e10 2024-03-29T08:25:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c3eb4d40 con 0x5573c4114e60 2024-03-29T08:25:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573c3ed2f30 con 0x5573c4115780 2024-03-29T08:25:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 10 client.0 _ll_register_callbacks cb 0x5573c3f77810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:25:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc076a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c411c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc076a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c411c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56592/0 (socket says 172.21.15.46:56592) 2024-03-29T08:25:41.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc076a00640 1 -- 172.21.15.46:0/4236678305 learned_addr learned my addr 172.21.15.46:0/4236678305 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc06fe00640 1 --2- 172.21.15.46:0/4236678305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c4114e60 0x5573c411c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc077400640 1 --2- 172.21.15.46:0/4236678305 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4115780 0x5573c4115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc076a00640 1 -- 172.21.15.46:0/4236678305 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4115780 msgr2=0x5573c4115b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc076a00640 1 --2- 172.21.15.46:0/4236678305 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4115780 0x5573c4115b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc076a00640 1 -- 172.21.15.46:0/4236678305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c4114e60 msgr2=0x5573c411c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc076a00640 1 --2- 172.21.15.46:0/4236678305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c4114e60 0x5573c411c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc076a00640 1 -- 172.21.15.46:0/4236678305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07813a080 con 0x5573c40c8e10 2024-03-29T08:25:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc06fe00640 1 --2- 172.21.15.46:0/4236678305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c4114e60 0x5573c411c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc076a00640 1 --2- 172.21.15.46:0/4236678305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c411c100 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fc068014700 tx=0x7fc068001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34eb95f359d5ac78 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc05be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:25:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc05be00640 1 -- 172.21.15.46:0/4236678305 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068013680 con 0x5573c40c8e10 2024-03-29T08:25:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc05be00640 1 -- 172.21.15.46:0/4236678305 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc06801c040 con 0x5573c40c8e10 2024-03-29T08:25:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 -- 172.21.15.46:0/4236678305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573c3f143e0 con 0x5573c40c8e10 2024-03-29T08:25:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc05be00640 1 -- 172.21.15.46:0/4236678305 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0680137e0 con 0x5573c40c8e10 2024-03-29T08:25:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 10 client.7842 fetch_fsmap learned FSMap version 63 2024-03-29T08:25:41.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 -- 172.21.15.46:0/4236678305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x5573c3edaaa0 con 0x5573c40c8e10 2024-03-29T08:25:41.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc05be00640 1 -- 172.21.15.46:0/4236678305 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5573c3f143e0 con 0x5573c40c8e10 2024-03-29T08:25:41.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc05be00640 1 -- 172.21.15.46:0/4236678305 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 63) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc068012070 con 0x5573c40c8e10 2024-03-29T08:25:41.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 10 client.7842 fetch_fsmap finished waiting for FSMap version 63 2024-03-29T08:25:41.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 10 client.7842 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:25:41.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 20 client.7842 populate_metadata read hostname 'smithi046' 2024-03-29T08:25:41.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 --2- 172.21.15.46:0/4236678305 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5573c4116900 0x5573c4116ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 4 client.7842 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:25:41.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.364+0000 7fc080c64000 1 -- 172.21.15.46:0/4236678305 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573c3ee5d00 con 0x5573c4116900 2024-03-29T08:25:41.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc06fe00640 1 --2- 172.21.15.46:0/4236678305 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5573c4116900 0x5573c4116ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc05be00640 10 client.7842 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:41.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc06fe00640 1 --2- 172.21.15.46:0/4236678305 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5573c4116900 0x5573c4116ce0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7171a3ebf1bce2d4 server_cookie=b2155d74e902a6ae in_seq=0 out_seq=0 2024-03-29T08:25:41.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc05be00640 1 -- 172.21.15.46:0/4236678305 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5573c3ee5d00 con 0x5573c4116900 2024-03-29T08:25:41.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc05be00640 10 client.7842 handle_command_reply: tid=0 2024-03-29T08:25:41.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc080c64000 1 client.7842 shutdown 2024-03-29T08:25:41.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc080c64000 1 -- 172.21.15.46:0/4236678305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 msgr2=0x5573c411c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc080c64000 1 --2- 172.21.15.46:0/4236678305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c411c100 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fc068014700 tx=0x7fc068001160 comp rx=0 tx=0).stop 2024-03-29T08:25:41.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc080c64000 1 -- 172.21.15.46:0/4236678305 shutdown_connections 2024-03-29T08:25:41.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc080c64000 1 --2- 172.21.15.46:0/4236678305 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5573c4116900 0x5573c4116ce0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc080c64000 1 --2- 172.21.15.46:0/4236678305 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c4114e60 0x5573c411c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc080c64000 1 --2- 172.21.15.46:0/4236678305 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4115780 0x5573c4115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc05be00640 0 client.7842 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:41.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc080c64000 1 --2- 172.21.15.46:0/4236678305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c411c100 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc080c64000 1 -- 172.21.15.46:0/4236678305 >> 172.21.15.46:0/4236678305 conn(0x5573c3ec5160 msgr2=0x5573c40c3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:41.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc080c64000 1 -- 172.21.15.46:0/4236678305 shutdown_connections 2024-03-29T08:25:41.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc080c64000 1 -- 172.21.15.46:0/4236678305 wait complete. 2024-03-29T08:25:41.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.368+0000 7fc080c64000 20 client.7842 trim_cache size 0 max 16384 2024-03-29T08:25:41.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc080c64000 1 Processor -- start 2024-03-29T08:25:41.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc080c64000 1 -- start start 2024-03-29T08:25:41.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc080c64000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c416ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc080c64000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c4114e60 0x5573c416f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc080c64000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4115780 0x5573c416f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc080c64000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c3bd2aa0 con 0x5573c40c8e10 2024-03-29T08:25:41.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc080c64000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c3f19a90 con 0x5573c4114e60 2024-03-29T08:25:41.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc080c64000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573c3f185f0 con 0x5573c4115780 2024-03-29T08:25:41.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc076a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c416ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc076a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c416ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56606/0 (socket says 172.21.15.46:56606) 2024-03-29T08:25:41.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc076a00640 1 -- 172.21.15.46:0/1281075145 learned_addr learned my addr 172.21.15.46:0/1281075145 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:41.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc077400640 1 --2- 172.21.15.46:0/1281075145 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4115780 0x5573c416f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc06fe00640 1 --2- 172.21.15.46:0/1281075145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c4114e60 0x5573c416f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc076a00640 1 -- 172.21.15.46:0/1281075145 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4115780 msgr2=0x5573c416f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc076a00640 1 --2- 172.21.15.46:0/1281075145 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4115780 0x5573c416f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc076a00640 1 -- 172.21.15.46:0/1281075145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c4114e60 msgr2=0x5573c416f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc076a00640 1 --2- 172.21.15.46:0/1281075145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c4114e60 0x5573c416f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc076a00640 1 -- 172.21.15.46:0/1281075145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573c3ec0df0 con 0x5573c40c8e10 2024-03-29T08:25:41.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc076a00640 1 --2- 172.21.15.46:0/1281075145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c416ed80 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fc0680016c0 tx=0x7fc068021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310ba50722d0feb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:41.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc067e00640 1 -- 172.21.15.46:0/1281075145 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06800fe80 con 0x5573c40c8e10 2024-03-29T08:25:41.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc067e00640 1 -- 172.21.15.46:0/1281075145 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc068013a80 con 0x5573c40c8e10 2024-03-29T08:25:41.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc067e00640 1 -- 172.21.15.46:0/1281075145 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068013e90 con 0x5573c40c8e10 2024-03-29T08:25:41.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc080c64000 1 -- 172.21.15.46:0/1281075145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 msgr2=0x5573c416ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc080c64000 1 --2- 172.21.15.46:0/1281075145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c416ed80 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fc0680016c0 tx=0x7fc068021000 comp rx=0 tx=0).stop 2024-03-29T08:25:41.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc080c64000 1 -- 172.21.15.46:0/1281075145 shutdown_connections 2024-03-29T08:25:41.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc080c64000 1 --2- 172.21.15.46:0/1281075145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c4114e60 0x5573c416f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc080c64000 1 --2- 172.21.15.46:0/1281075145 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4115780 0x5573c416f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc080c64000 1 --2- 172.21.15.46:0/1281075145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c40c8e10 0x5573c416ed80 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc080c64000 1 -- 172.21.15.46:0/1281075145 >> 172.21.15.46:0/1281075145 conn(0x5573c3ec5160 msgr2=0x5573c4122820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:41.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc080c64000 1 -- 172.21.15.46:0/1281075145 shutdown_connections 2024-03-29T08:25:41.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.384+0000 7fc080c64000 1 -- 172.21.15.46:0/1281075145 wait complete. 2024-03-29T08:25:41.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 1 -- start start 2024-03-29T08:25:41.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 1 Processor -- start 2024-03-29T08:25:41.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c40c8e10 0x5573c4157de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4114e60 0x5573c4118c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c4115780 0x5573c4119180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c3f11ff0 con 0x5573c4115780 2024-03-29T08:25:41.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573c3f132a0 con 0x5573c40c8e10 2024-03-29T08:25:41.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573c3eb9aa0 con 0x5573c4114e60 2024-03-29T08:25:41.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 10 client.0 _ll_register_callbacks cb 0x7fc0600ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:25:41.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc077400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c4115780 0x5573c4119180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc077400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c4115780 0x5573c4119180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56620/0 (socket says 172.21.15.46:56620) 2024-03-29T08:25:41.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc077400640 1 -- 172.21.15.46:0/4065886660 learned_addr learned my addr 172.21.15.46:0/4065886660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:25:41.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc076a00640 1 --2- 172.21.15.46:0/4065886660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c40c8e10 0x5573c4157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc06fe00640 1 --2- 172.21.15.46:0/4065886660 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4114e60 0x5573c4118c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc077400640 1 -- 172.21.15.46:0/4065886660 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4114e60 msgr2=0x5573c4118c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc077400640 1 --2- 172.21.15.46:0/4065886660 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4114e60 0x5573c4118c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc077400640 1 -- 172.21.15.46:0/4065886660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c40c8e10 msgr2=0x5573c4157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:25:41.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc077400640 1 --2- 172.21.15.46:0/4065886660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c40c8e10 0x5573c4157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc077400640 1 -- 172.21.15.46:0/4065886660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573c3edaaa0 con 0x5573c4115780 2024-03-29T08:25:41.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc06fe00640 1 --2- 172.21.15.46:0/4065886660 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4114e60 0x5573c4118c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:41.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc076a00640 1 --2- 172.21.15.46:0/4065886660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c40c8e10 0x5573c4157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:25:41.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc077400640 1 --2- 172.21.15.46:0/4065886660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c4115780 0x5573c4119180 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc070005c30 tx=0x7fc070014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8b4d8f1ef6f304 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:25:41.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc05be00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:25:41.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc05be00640 1 -- 172.21.15.46:0/4065886660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0700514b0 con 0x5573c4115780 2024-03-29T08:25:41.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc05be00640 1 -- 172.21.15.46:0/4065886660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc0700822a0 con 0x5573c4115780 2024-03-29T08:25:41.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 1 -- 172.21.15.46:0/4065886660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573c3ed2970 con 0x5573c4115780 2024-03-29T08:25:41.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc05be00640 1 -- 172.21.15.46:0/4065886660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc070054660 con 0x5573c4115780 2024-03-29T08:25:41.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 10 client.7848 fetch_fsmap learned FSMap version 63 2024-03-29T08:25:41.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 1 -- 172.21.15.46:0/4065886660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x5573c3eb4960 con 0x5573c4115780 2024-03-29T08:25:41.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc05be00640 1 -- 172.21.15.46:0/4065886660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5573c3ed2970 con 0x5573c4115780 2024-03-29T08:25:41.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc05be00640 1 -- 172.21.15.46:0/4065886660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 63) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc07005d020 con 0x5573c4115780 2024-03-29T08:25:41.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 10 client.7848 fetch_fsmap finished waiting for FSMap version 63 2024-03-29T08:25:41.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 10 client.7848 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:25:41.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 20 client.7848 populate_metadata read hostname 'smithi046' 2024-03-29T08:25:41.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 1 --2- 172.21.15.46:0/4065886660 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5573c41196c0 0x5573c4119ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:25:41.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 4 client.7848 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:25:41.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 1 -- 172.21.15.46:0/4065886660 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5573c3f44a20 con 0x5573c41196c0 2024-03-29T08:25:41.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc076a00640 1 --2- 172.21.15.46:0/4065886660 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5573c41196c0 0x5573c4119ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:25:41.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc076a00640 1 --2- 172.21.15.46:0/4065886660 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5573c41196c0 0x5573c4119ad0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10b8bc275daeecb9 server_cookie=6f818b2785ac3e6 in_seq=0 out_seq=0 2024-03-29T08:25:41.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc05be00640 10 client.7848 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:41.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc05be00640 1 -- 172.21.15.46:0/4065886660 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x5573c3f44a20 con 0x5573c41196c0 2024-03-29T08:25:41.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc05be00640 10 client.7848 handle_command_reply: tid=0 2024-03-29T08:25:41.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.388+0000 7fc080c64000 1 client.7848 shutdown 2024-03-29T08:25:41.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc080c64000 1 -- 172.21.15.46:0/4065886660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c4115780 msgr2=0x5573c4119180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc080c64000 1 --2- 172.21.15.46:0/4065886660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c4115780 0x5573c4119180 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc070005c30 tx=0x7fc070014070 comp rx=0 tx=0).stop 2024-03-29T08:25:41.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc080c64000 1 -- 172.21.15.46:0/4065886660 shutdown_connections 2024-03-29T08:25:41.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc080c64000 1 --2- 172.21.15.46:0/4065886660 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x5573c41196c0 0x5573c4119ad0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc080c64000 1 --2- 172.21.15.46:0/4065886660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573c4115780 0x5573c4119180 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc080c64000 1 --2- 172.21.15.46:0/4065886660 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5573c4114e60 0x5573c4118c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc080c64000 1 --2- 172.21.15.46:0/4065886660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573c40c8e10 0x5573c4157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc05be00640 0 client.7848 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:25:41.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc080c64000 1 -- 172.21.15.46:0/4065886660 >> 172.21.15.46:0/4065886660 conn(0x5573c3ec5160 msgr2=0x5573c40c34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:41.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc080c64000 1 -- 172.21.15.46:0/4065886660 shutdown_connections 2024-03-29T08:25:41.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc080c64000 1 -- 172.21.15.46:0/4065886660 wait complete. 2024-03-29T08:25:41.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc080c64000 20 client.7848 trim_cache size 0 max 16384 2024-03-29T08:25:41.408 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:25:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-29T08:25:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-29T08:25:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: "164e6815-ed4f-4b27-bc0e-df097fc34e78": { 2024-03-29T08:25:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/dir", 2024-03-29T08:25:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "164e6815-ed4f-4b27-bc0e-df097fc34e78", 2024-03-29T08:25:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,repair" 2024-03-29T08:25:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:25:41.409 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:25:41.409 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:25:41.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc07d800640 1 -- 172.21.15.46:0/2058016006 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0540666c0 msgr2=0x7fc054068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc07d800640 1 --2- 172.21.15.46:0/2058016006 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0540666c0 0x7fc054068b80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc0680011f0 tx=0x7fc068009040 comp rx=0 tx=0).stop 2024-03-29T08:25:41.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc07d800640 1 -- 172.21.15.46:0/2058016006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07815df90 msgr2=0x7fc078156b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:25:41.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc07d800640 1 --2- 172.21.15.46:0/2058016006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07815df90 0x7fc078156b50 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc07000cb10 tx=0x7fc07000a060 comp rx=0 tx=0).stop 2024-03-29T08:25:41.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc07d800640 1 -- 172.21.15.46:0/2058016006 shutdown_connections 2024-03-29T08:25:41.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc07d800640 1 --2- 172.21.15.46:0/2058016006 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc0540666c0 0x7fc054068b80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc07d800640 1 --2- 172.21.15.46:0/2058016006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07815df90 0x7fc078156b50 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc07d800640 1 --2- 172.21.15.46:0/2058016006 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc078142a50 0x7fc078156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc07d800640 1 --2- 172.21.15.46:0/2058016006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0780cbea0 0x7fc0781571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:25:41.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc07d800640 1 -- 172.21.15.46:0/2058016006 >> 172.21.15.46:0/2058016006 conn(0x7fc07806a390 msgr2=0x7fc0780cd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:25:41.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc07d800640 1 -- 172.21.15.46:0/2058016006 shutdown_connections 2024-03-29T08:25:41.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:25:41.392+0000 7fc07d800640 1 -- 172.21.15.46:0/2058016006 wait complete. 2024-03-29T08:25:41.412 INFO:tasks.cephfs.filesystem:scrub status for tag:164e6815-ed4f-4b27-bc0e-df097fc34e78 - {'path': '/dir', 'tag': '164e6815-ed4f-4b27-bc0e-df097fc34e78', 'options': 'recursive,repair'} 2024-03-29T08:26:09.334 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:26:09.338 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:26:11.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:26:11.413 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-29T08:26:11.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.572+0000 7f8889c00640 1 Processor -- start 2024-03-29T08:26:11.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.572+0000 7f8889c00640 1 -- start start 2024-03-29T08:26:11.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.572+0000 7f8889c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f888415dbb0 0x7f888406ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.572+0000 7f8889c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f88840b3f20 0x7f888406b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.572+0000 7f8889c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8884157490 con 0x7f88840b1d30 2024-03-29T08:26:11.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.572+0000 7f8889c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88840b2c90 con 0x7f88840b3f20 2024-03-29T08:26:11.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.572+0000 7f8889c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88840b29b0 con 0x7f888415dbb0 2024-03-29T08:26:11.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.572+0000 7f887be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f88840b3f20 0x7f888406b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.572+0000 7f8883400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f88840b1d30 0x7f88840b2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46972/0 (socket says 172.21.15.46:46972) 2024-03-29T08:26:11.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.572+0000 7f8883400640 1 -- 172.21.15.46:0/3058543346 learned_addr learned my addr 172.21.15.46:0/3058543346 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:26:11.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.572+0000 7f8882a00640 1 --2- 172.21.15.46:0/3058543346 >> v2:172.21.15.73:3300/0 conn(0x7f888415dbb0 0x7f888406ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.572+0000 7f887be00640 1 -- 172.21.15.46:0/3058543346 >> v2:172.21.15.73:3300/0 conn(0x7f888415dbb0 msgr2=0x7f888406ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:11.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.572+0000 7f887be00640 1 --2- 172.21.15.46:0/3058543346 >> v2:172.21.15.73:3300/0 conn(0x7f888415dbb0 0x7f888406ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.572+0000 7f887be00640 1 -- 172.21.15.46:0/3058543346 >> v1:172.21.15.46:6789/0 conn(0x7f88840b1d30 legacy=0x7f88840b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:11.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.572+0000 7f887be00640 1 -- 172.21.15.46:0/3058543346 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88840b3440 con 0x7f88840b3f20 2024-03-29T08:26:11.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.572+0000 7f8882000640 1 -- 172.21.15.46:0/3058543346 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436289308 0 0) 0x7f8884157490 con 0x7f88840b1d30 2024-03-29T08:26:11.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.576+0000 7f887be00640 1 --2- 172.21.15.46:0/3058543346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88840b3f20 0x7f888406b2e0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8884066210 tx=0x7f886c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a90b2db9988bcd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:11.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.576+0000 7f8882000640 1 -- 172.21.15.46:0/3058543346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c0ab5f0 con 0x7f88840b3f20 2024-03-29T08:26:11.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.576+0000 7f8882000640 1 -- 172.21.15.46:0/3058543346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c0ab750 con 0x7f88840b3f20 2024-03-29T08:26:11.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.576+0000 7f8882000640 1 -- 172.21.15.46:0/3058543346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c0abae0 con 0x7f88840b3f20 2024-03-29T08:26:11.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.576+0000 7f8889c00640 1 -- 172.21.15.46:0/3058543346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88840b3f20 msgr2=0x7f888406b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:11.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.576+0000 7f8889c00640 1 --2- 172.21.15.46:0/3058543346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88840b3f20 0x7f888406b2e0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8884066210 tx=0x7f886c01c3f0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.576+0000 7f8889c00640 1 -- 172.21.15.46:0/3058543346 shutdown_connections 2024-03-29T08:26:11.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.576+0000 7f8889c00640 1 --2- 172.21.15.46:0/3058543346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88840b3f20 0x7f888406b2e0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.576+0000 7f8889c00640 1 --2- 172.21.15.46:0/3058543346 >> v2:172.21.15.73:3300/0 conn(0x7f888415dbb0 0x7f888406ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.576+0000 7f8889c00640 1 -- 172.21.15.46:0/3058543346 >> 172.21.15.46:0/3058543346 conn(0x7f8884070ad0 msgr2=0x7f8884070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:11.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.576+0000 7f8889c00640 1 -- 172.21.15.46:0/3058543346 shutdown_connections 2024-03-29T08:26:11.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.576+0000 7f8889c00640 1 -- 172.21.15.46:0/3058543346 wait complete. 2024-03-29T08:26:11.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8889c00640 1 Processor -- start 2024-03-29T08:26:11.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8889c00640 1 -- start start 2024-03-29T08:26:11.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8889c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88840b1d30 0x7f88841503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8889c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f88840b3f20 0x7f888406ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8889c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f888415dbb0 0x7f888415f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8889c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888406d9c0 con 0x7f88840b1d30 2024-03-29T08:26:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8889c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88841578e0 con 0x7f888415dbb0 2024-03-29T08:26:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8889c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f888406cdb0 con 0x7f88840b3f20 2024-03-29T08:26:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8882a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88840b1d30 0x7f88841503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8882a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88840b1d30 0x7f88841503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42286/0 (socket says 172.21.15.46:42286) 2024-03-29T08:26:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8882a00640 1 -- 172.21.15.46:0/2341071670 learned_addr learned my addr 172.21.15.46:0/2341071670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8882a00640 1 -- 172.21.15.46:0/2341071670 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f88840b3f20 msgr2=0x7f888406ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8882a00640 1 --2- 172.21.15.46:0/2341071670 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f88840b3f20 0x7f888406ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8882a00640 1 -- 172.21.15.46:0/2341071670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f888415dbb0 msgr2=0x7f888415f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8882a00640 1 --2- 172.21.15.46:0/2341071670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f888415dbb0 0x7f888415f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8882a00640 1 -- 172.21.15.46:0/2341071670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886c009420 con 0x7f88840b1d30 2024-03-29T08:26:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8882a00640 1 --2- 172.21.15.46:0/2341071670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88840b1d30 0x7f88841503d0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f887400d5f0 tx=0x7f887400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f887aa00640 1 -- 172.21.15.46:0/2341071670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88740125f0 con 0x7f88840b1d30 2024-03-29T08:26:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f887aa00640 1 -- 172.21.15.46:0/2341071670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8874012750 con 0x7f88840b1d30 2024-03-29T08:26:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f887aa00640 1 -- 172.21.15.46:0/2341071670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874012ae0 con 0x7f88840b1d30 2024-03-29T08:26:11.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8889c00640 1 -- 172.21.15.46:0/2341071670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f888406d090 con 0x7f88840b1d30 2024-03-29T08:26:11.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.580+0000 7f8889c00640 1 -- 172.21.15.46:0/2341071670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888406d650 con 0x7f88840b1d30 2024-03-29T08:26:11.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f887aa00640 1 -- 172.21.15.46:0/2341071670 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8874012c40 con 0x7f88840b1d30 2024-03-29T08:26:11.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f887aa00640 1 --2- 172.21.15.46:0/2341071670 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8860066690 0x7f8860068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f887aa00640 1 -- 172.21.15.46:0/2341071670 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8874084830 con 0x7f88840b1d30 2024-03-29T08:26:11.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f887be00640 1 --2- 172.21.15.46:0/2341071670 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8860066690 0x7f8860068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f888cfa1000 1 Processor -- start 2024-03-29T08:26:11.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f888cfa1000 1 -- start start 2024-03-29T08:26:11.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f887be00640 1 --2- 172.21.15.46:0/2341071670 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8860066690 0x7f8860068b50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f886c0af000 tx=0x7f886c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:11.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f888cfa1000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68ba0a70 0x55bb68ba0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f888cfa1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba1390 0x55bb68b544c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f888cfa1000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68b54a00 0x55bb68b54de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f888cfa1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb68a1f630 con 0x55bb68ba1390 2024-03-29T08:26:11.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f888cfa1000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb689acbd0 con 0x55bb68b54a00 2024-03-29T08:26:11.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f888cfa1000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb689aa1a0 con 0x55bb68ba0a70 2024-03-29T08:26:11.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f887be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba1390 0x55bb68b544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f887be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba1390 0x55bb68b544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42298/0 (socket says 172.21.15.46:42298) 2024-03-29T08:26:11.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f887be00640 1 -- 172.21.15.46:0/334100584 learned_addr learned my addr 172.21.15.46:0/334100584 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:11.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f8882a00640 1 --2- 172.21.15.46:0/334100584 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68ba0a70 0x55bb68ba0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f8883400640 1 --2- 172.21.15.46:0/334100584 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68b54a00 0x55bb68b54de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f887be00640 1 -- 172.21.15.46:0/334100584 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68ba0a70 msgr2=0x55bb68ba0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:11.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f887be00640 1 --2- 172.21.15.46:0/334100584 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68ba0a70 0x55bb68ba0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f887be00640 1 -- 172.21.15.46:0/334100584 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68b54a00 msgr2=0x55bb68b54de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:11.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f887be00640 1 --2- 172.21.15.46:0/334100584 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68b54a00 0x55bb68b54de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f887be00640 1 -- 172.21.15.46:0/334100584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb686a1e20 con 0x55bb68ba1390 2024-03-29T08:26:11.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f8882a00640 1 --2- 172.21.15.46:0/334100584 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68ba0a70 0x55bb68ba0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f887be00640 1 --2- 172.21.15.46:0/334100584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba1390 0x55bb68b544c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f886c0ad040 tx=0x7f886c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7cb5a58302ffd8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f8873400640 1 -- 172.21.15.46:0/334100584 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c006e20 con 0x55bb68ba1390 2024-03-29T08:26:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f8873400640 1 -- 172.21.15.46:0/334100584 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c004030 con 0x55bb68ba1390 2024-03-29T08:26:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.584+0000 7f8873400640 1 -- 172.21.15.46:0/334100584 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c0043c0 con 0x55bb68ba1390 2024-03-29T08:26:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 -- 172.21.15.46:0/334100584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba1390 msgr2=0x55bb68b544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 --2- 172.21.15.46:0/334100584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba1390 0x55bb68b544c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f886c0ad040 tx=0x7f886c004740 comp rx=0 tx=0).stop 2024-03-29T08:26:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 -- 172.21.15.46:0/334100584 shutdown_connections 2024-03-29T08:26:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 --2- 172.21.15.46:0/334100584 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68b54a00 0x55bb68b54de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 --2- 172.21.15.46:0/334100584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba1390 0x55bb68b544c0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 --2- 172.21.15.46:0/334100584 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68ba0a70 0x55bb68ba0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 -- 172.21.15.46:0/334100584 >> 172.21.15.46:0/334100584 conn(0x55bb68955160 msgr2=0x55bb68b83820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:11.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 -- 172.21.15.46:0/334100584 shutdown_connections 2024-03-29T08:26:11.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 -- 172.21.15.46:0/334100584 wait complete. 2024-03-29T08:26:11.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 -- start start 2024-03-29T08:26:11.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 Processor -- start 2024-03-29T08:26:11.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68b54a00 0x55bb68b5d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68ba0a70 0x55bb68b84a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba1390 0x55bb68b84f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb689a5a90 con 0x55bb68ba1390 2024-03-29T08:26:11.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb68944d40 con 0x55bb68b54a00 2024-03-29T08:26:11.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb68962f30 con 0x55bb68ba0a70 2024-03-29T08:26:11.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 10 client.0 _ll_register_callbacks cb 0x55bb68a07810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:26:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f8883400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba1390 0x55bb68b84f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f8883400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba1390 0x55bb68b84f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42310/0 (socket says 172.21.15.46:42310) 2024-03-29T08:26:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f8883400640 1 -- 172.21.15.46:0/3571831047 learned_addr learned my addr 172.21.15.46:0/3571831047 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f887be00640 1 --2- 172.21.15.46:0/3571831047 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68ba0a70 0x55bb68b84a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f8882a00640 1 --2- 172.21.15.46:0/3571831047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68b54a00 0x55bb68b5d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f8883400640 1 -- 172.21.15.46:0/3571831047 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68ba0a70 msgr2=0x55bb68b84a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f8883400640 1 --2- 172.21.15.46:0/3571831047 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68ba0a70 0x55bb68b84a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f8883400640 1 -- 172.21.15.46:0/3571831047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68b54a00 msgr2=0x55bb68b5d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f8883400640 1 --2- 172.21.15.46:0/3571831047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68b54a00 0x55bb68b5d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f8883400640 1 -- 172.21.15.46:0/3571831047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88840b3440 con 0x55bb68ba1390 2024-03-29T08:26:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f8883400640 1 --2- 172.21.15.46:0/3571831047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba1390 0x55bb68b84f90 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f887c001640 tx=0x7f887c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b4fa7cee7fcc01 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:11.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f8867e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:26:11.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f8867e00640 1 -- 172.21.15.46:0/3571831047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c004d80 con 0x55bb68ba1390 2024-03-29T08:26:11.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 -- 172.21.15.46:0/3571831047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb689a43e0 con 0x55bb68ba1390 2024-03-29T08:26:11.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f8867e00640 1 -- 172.21.15.46:0/3571831047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f887c014af0 con 0x55bb68ba1390 2024-03-29T08:26:11.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f8867e00640 1 -- 172.21.15.46:0/3571831047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f887c014cb0 con 0x55bb68ba1390 2024-03-29T08:26:11.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 10 client.7863 fetch_fsmap learned FSMap version 63 2024-03-29T08:26:11.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.588+0000 7f888cfa1000 1 -- 172.21.15.46:0/3571831047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55bb6896aaa0 con 0x55bb68ba1390 2024-03-29T08:26:11.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f8867e00640 1 -- 172.21.15.46:0/3571831047 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bb689a43e0 con 0x55bb68ba1390 2024-03-29T08:26:11.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f8867e00640 1 -- 172.21.15.46:0/3571831047 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 63) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f887c005050 con 0x55bb68ba1390 2024-03-29T08:26:11.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 10 client.7863 fetch_fsmap finished waiting for FSMap version 63 2024-03-29T08:26:11.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 10 client.7863 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:26:11.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 20 client.7863 populate_metadata read hostname 'smithi046' 2024-03-29T08:26:11.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 1 --2- 172.21.15.46:0/3571831047 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55bb68ba82d0 0x55bb68ba86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 4 client.7863 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:26:11.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 1 -- 172.21.15.46:0/3571831047 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb68975d00 con 0x55bb68ba82d0 2024-03-29T08:26:11.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f8882a00640 1 --2- 172.21.15.46:0/3571831047 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55bb68ba82d0 0x55bb68ba86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f8882a00640 1 --2- 172.21.15.46:0/3571831047 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55bb68ba82d0 0x55bb68ba86b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e84e6391c39ffcfb server_cookie=a61e465882c256ab in_seq=0 out_seq=0 2024-03-29T08:26:11.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f8867e00640 10 client.7863 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:26:11.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f8867e00640 1 -- 172.21.15.46:0/3571831047 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bb68975d00 con 0x55bb68ba82d0 2024-03-29T08:26:11.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f8867e00640 10 client.7863 handle_command_reply: tid=0 2024-03-29T08:26:11.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 1 client.7863 shutdown 2024-03-29T08:26:11.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 1 -- 172.21.15.46:0/3571831047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba1390 msgr2=0x55bb68b84f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:11.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 1 --2- 172.21.15.46:0/3571831047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba1390 0x55bb68b84f90 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f887c001640 tx=0x7f887c001a40 comp rx=0 tx=0).stop 2024-03-29T08:26:11.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 1 -- 172.21.15.46:0/3571831047 shutdown_connections 2024-03-29T08:26:11.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 1 --2- 172.21.15.46:0/3571831047 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55bb68ba82d0 0x55bb68ba86b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 1 --2- 172.21.15.46:0/3571831047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba1390 0x55bb68b84f90 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f8867e00640 0 client.7863 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:26:11.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 1 --2- 172.21.15.46:0/3571831047 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68ba0a70 0x55bb68b84a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 1 --2- 172.21.15.46:0/3571831047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68b54a00 0x55bb68b5d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 1 -- 172.21.15.46:0/3571831047 >> 172.21.15.46:0/3571831047 conn(0x55bb68955160 msgr2=0x55bb68b53bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:11.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 1 -- 172.21.15.46:0/3571831047 shutdown_connections 2024-03-29T08:26:11.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 1 -- 172.21.15.46:0/3571831047 wait complete. 2024-03-29T08:26:11.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.592+0000 7f888cfa1000 20 client.7863 trim_cache size 0 max 16384 2024-03-29T08:26:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f888cfa1000 1 Processor -- start 2024-03-29T08:26:11.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f888cfa1000 1 -- start start 2024-03-29T08:26:11.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f888cfa1000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68b54a00 0x55bb68b5c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f888cfa1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba0a70 0x55bb68b5cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f888cfa1000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68ba1390 0x55bb68b54de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f888cfa1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb68662aa0 con 0x55bb68ba0a70 2024-03-29T08:26:11.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f888cfa1000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb689e9170 con 0x55bb68ba1390 2024-03-29T08:26:11.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f888cfa1000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb68b5bb90 con 0x55bb68b54a00 2024-03-29T08:26:11.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f887be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba0a70 0x55bb68b5cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f887be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba0a70 0x55bb68b5cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42314/0 (socket says 172.21.15.46:42314) 2024-03-29T08:26:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f887be00640 1 -- 172.21.15.46:0/3616008774 learned_addr learned my addr 172.21.15.46:0/3616008774 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f8883400640 1 --2- 172.21.15.46:0/3616008774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68ba1390 0x55bb68b54de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f8882a00640 1 --2- 172.21.15.46:0/3616008774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68b54a00 0x55bb68b5c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f887be00640 1 -- 172.21.15.46:0/3616008774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68b54a00 msgr2=0x55bb68b5c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f887be00640 1 --2- 172.21.15.46:0/3616008774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68b54a00 0x55bb68b5c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f887be00640 1 -- 172.21.15.46:0/3616008774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68ba1390 msgr2=0x55bb68b54de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f887be00640 1 --2- 172.21.15.46:0/3616008774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68ba1390 0x55bb68b54de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f887be00640 1 -- 172.21.15.46:0/3616008774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb68950df0 con 0x55bb68ba0a70 2024-03-29T08:26:11.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f8882a00640 1 --2- 172.21.15.46:0/3616008774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68b54a00 0x55bb68b5c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:11.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f8883400640 1 --2- 172.21.15.46:0/3616008774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68ba1390 0x55bb68b54de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:11.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f887be00640 1 --2- 172.21.15.46:0/3616008774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba0a70 0x55bb68b5cc30 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f886c01c8d0 tx=0x7f886c0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573e27e386625259 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:11.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f8873e00640 1 -- 172.21.15.46:0/3616008774 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c0cf070 con 0x55bb68ba0a70 2024-03-29T08:26:11.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f8873e00640 1 -- 172.21.15.46:0/3616008774 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c0d0040 con 0x55bb68ba0a70 2024-03-29T08:26:11.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f8873e00640 1 -- 172.21.15.46:0/3616008774 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c004430 con 0x55bb68ba0a70 2024-03-29T08:26:11.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f888cfa1000 1 -- 172.21.15.46:0/3616008774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba0a70 msgr2=0x55bb68b5cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:11.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f888cfa1000 1 --2- 172.21.15.46:0/3616008774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba0a70 0x55bb68b5cc30 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f886c01c8d0 tx=0x7f886c0a7000 comp rx=0 tx=0).stop 2024-03-29T08:26:11.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f888cfa1000 1 -- 172.21.15.46:0/3616008774 shutdown_connections 2024-03-29T08:26:11.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f888cfa1000 1 --2- 172.21.15.46:0/3616008774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68ba1390 0x55bb68b54de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f888cfa1000 1 --2- 172.21.15.46:0/3616008774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68ba0a70 0x55bb68b5cc30 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f888cfa1000 1 --2- 172.21.15.46:0/3616008774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68b54a00 0x55bb68b5c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f888cfa1000 1 -- 172.21.15.46:0/3616008774 >> 172.21.15.46:0/3616008774 conn(0x55bb68955160 msgr2=0x55bb68b5bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:11.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f888cfa1000 1 -- 172.21.15.46:0/3616008774 shutdown_connections 2024-03-29T08:26:11.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.608+0000 7f888cfa1000 1 -- 172.21.15.46:0/3616008774 wait complete. 2024-03-29T08:26:11.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 1 -- start start 2024-03-29T08:26:11.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 1 Processor -- start 2024-03-29T08:26:11.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68b54a00 0x55bb68be4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68ba0a70 0x55bb68b57600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68ba1390 0x55bb68b57b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6896f640 con 0x55bb68b54a00 2024-03-29T08:26:11.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb689fd920 con 0x55bb68ba0a70 2024-03-29T08:26:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bb6899f860 con 0x55bb68ba1390 2024-03-29T08:26:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 10 client.0 _ll_register_callbacks cb 0x7f886c0b9050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:26:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8882a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68b54a00 0x55bb68be4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8882a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68b54a00 0x55bb68be4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42326/0 (socket says 172.21.15.46:42326) 2024-03-29T08:26:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8882a00640 1 -- 172.21.15.46:0/1441118773 learned_addr learned my addr 172.21.15.46:0/1441118773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f887be00640 1 --2- 172.21.15.46:0/1441118773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68ba0a70 0x55bb68b57600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8883400640 1 --2- 172.21.15.46:0/1441118773 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68ba1390 0x55bb68b57b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8882a00640 1 -- 172.21.15.46:0/1441118773 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68ba1390 msgr2=0x55bb68b57b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8882a00640 1 --2- 172.21.15.46:0/1441118773 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68ba1390 0x55bb68b57b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8882a00640 1 -- 172.21.15.46:0/1441118773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68ba0a70 msgr2=0x55bb68b57600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8882a00640 1 --2- 172.21.15.46:0/1441118773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68ba0a70 0x55bb68b57600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8882a00640 1 -- 172.21.15.46:0/1441118773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bb686a1e20 con 0x55bb68b54a00 2024-03-29T08:26:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f887be00640 1 --2- 172.21.15.46:0/1441118773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68ba0a70 0x55bb68b57600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:26:11.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8883400640 1 --2- 172.21.15.46:0/1441118773 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68ba1390 0x55bb68b57b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:26:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8882a00640 1 --2- 172.21.15.46:0/1441118773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68b54a00 0x55bb68be4130 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f88740556f0 tx=0x7f887405dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca73fa77deb52ed8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8867e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:26:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8867e00640 1 -- 172.21.15.46:0/1441118773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8874082e30 con 0x55bb68b54a00 2024-03-29T08:26:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8867e00640 1 -- 172.21.15.46:0/1441118773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8874054c50 con 0x55bb68b54a00 2024-03-29T08:26:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 1 -- 172.21.15.46:0/1441118773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb68962970 con 0x55bb68b54a00 2024-03-29T08:26:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8867e00640 1 -- 172.21.15.46:0/1441118773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8874054e10 con 0x55bb68b54a00 2024-03-29T08:26:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 10 client.7869 fetch_fsmap learned FSMap version 63 2024-03-29T08:26:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 1 -- 172.21.15.46:0/1441118773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55bb68944960 con 0x55bb68b54a00 2024-03-29T08:26:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8867e00640 1 -- 172.21.15.46:0/1441118773 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bb68962970 con 0x55bb68b54a00 2024-03-29T08:26:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8867e00640 1 -- 172.21.15.46:0/1441118773 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 63) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f887405d6c0 con 0x55bb68b54a00 2024-03-29T08:26:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 10 client.7869 fetch_fsmap finished waiting for FSMap version 63 2024-03-29T08:26:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 10 client.7869 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:26:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 20 client.7869 populate_metadata read hostname 'smithi046' 2024-03-29T08:26:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 1 --2- 172.21.15.46:0/1441118773 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55bb68b58080 0x55bb68b58490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 4 client.7869 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:26:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f888cfa1000 1 -- 172.21.15.46:0/1441118773 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bb689d4a20 con 0x55bb68b58080 2024-03-29T08:26:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f887be00640 1 --2- 172.21.15.46:0/1441118773 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55bb68b58080 0x55bb68b58490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8867e00640 10 client.7869 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:26:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f887be00640 1 --2- 172.21.15.46:0/1441118773 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55bb68b58080 0x55bb68b58490 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=437db919d810d9b2 server_cookie=e0945ec2d764a438 in_seq=0 out_seq=0 2024-03-29T08:26:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8867e00640 1 -- 172.21.15.46:0/1441118773 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bb689d4a20 con 0x55bb68b58080 2024-03-29T08:26:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.612+0000 7f8867e00640 10 client.7869 handle_command_reply: tid=0 2024-03-29T08:26:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f888cfa1000 1 client.7869 shutdown 2024-03-29T08:26:11.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f888cfa1000 1 -- 172.21.15.46:0/1441118773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68b54a00 msgr2=0x55bb68be4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:11.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f888cfa1000 1 --2- 172.21.15.46:0/1441118773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68b54a00 0x55bb68be4130 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f88740556f0 tx=0x7f887405dc10 comp rx=0 tx=0).stop 2024-03-29T08:26:11.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f888cfa1000 1 -- 172.21.15.46:0/1441118773 shutdown_connections 2024-03-29T08:26:11.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f888cfa1000 1 --2- 172.21.15.46:0/1441118773 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55bb68b58080 0x55bb68b58490 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f888cfa1000 1 --2- 172.21.15.46:0/1441118773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bb68ba0a70 0x55bb68b57600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f888cfa1000 1 --2- 172.21.15.46:0/1441118773 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55bb68ba1390 0x55bb68b57b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f888cfa1000 1 --2- 172.21.15.46:0/1441118773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bb68b54a00 0x55bb68be4130 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f8867e00640 0 client.7869 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:26:11.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f888cfa1000 1 -- 172.21.15.46:0/1441118773 >> 172.21.15.46:0/1441118773 conn(0x55bb68955160 msgr2=0x55bb68b5c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:11.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f888cfa1000 1 -- 172.21.15.46:0/1441118773 shutdown_connections 2024-03-29T08:26:11.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f888cfa1000 1 -- 172.21.15.46:0/1441118773 wait complete. 2024-03-29T08:26:11.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f888cfa1000 20 client.7869 trim_cache size 0 max 16384 2024-03-29T08:26:11.622 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:26:11.622 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-29T08:26:11.622 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-29T08:26:11.622 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:26:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f8889c00640 1 -- 172.21.15.46:0/2341071670 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8860066690 msgr2=0x7f8860068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f8889c00640 1 --2- 172.21.15.46:0/2341071670 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8860066690 0x7f8860068b50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f886c0af000 tx=0x7f886c008040 comp rx=0 tx=0).stop 2024-03-29T08:26:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f8889c00640 1 -- 172.21.15.46:0/2341071670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88840b1d30 msgr2=0x7f88841503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f8889c00640 1 --2- 172.21.15.46:0/2341071670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88840b1d30 0x7f88841503d0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f887400d5f0 tx=0x7f887400dab0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f8889c00640 1 -- 172.21.15.46:0/2341071670 shutdown_connections 2024-03-29T08:26:11.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f8889c00640 1 --2- 172.21.15.46:0/2341071670 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8860066690 0x7f8860068b50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f8889c00640 1 --2- 172.21.15.46:0/2341071670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f888415dbb0 0x7f888415f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f8889c00640 1 --2- 172.21.15.46:0/2341071670 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f88840b3f20 0x7f888406ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f8889c00640 1 --2- 172.21.15.46:0/2341071670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88840b1d30 0x7f88841503d0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f8889c00640 1 -- 172.21.15.46:0/2341071670 >> 172.21.15.46:0/2341071670 conn(0x7f8884070ad0 msgr2=0x7f888415ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:11.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f8889c00640 1 -- 172.21.15.46:0/2341071670 shutdown_connections 2024-03-29T08:26:11.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.616+0000 7f8889c00640 1 -- 172.21.15.46:0/2341071670 wait complete. 2024-03-29T08:26:11.636 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-29T08:26:11.636 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:26:11.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.776+0000 7fe05a200640 1 Processor -- start 2024-03-29T08:26:11.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.776+0000 7fe05a200640 1 -- start start 2024-03-29T08:26:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.776+0000 7fe05a200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe05415abd0 0x7fe0540aee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.776+0000 7fe05a200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0540ac2a0 0x7fe0540ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.776+0000 7fe05a200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe05415afb0 con 0x7fe054069d90 2024-03-29T08:26:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.776+0000 7fe05a200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0540acc10 con 0x7fe0540ac2a0 2024-03-29T08:26:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.776+0000 7fe05a200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0540af690 con 0x7fe05415abd0 2024-03-29T08:26:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.776+0000 7fe053e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0540ac2a0 0x7fe0540ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.776+0000 7fe053e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe0540ac2a0 0x7fe0540ac6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42342/0 (socket says 172.21.15.46:42342) 2024-03-29T08:26:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.776+0000 7fe053e00640 1 -- 172.21.15.46:0/3487589128 learned_addr learned my addr 172.21.15.46:0/3487589128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.776+0000 7fe053400640 1 --2- 172.21.15.46:0/3487589128 >> v2:172.21.15.73:3300/0 conn(0x7fe05415abd0 0x7fe0540aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.776+0000 7fe053e00640 1 -- 172.21.15.46:0/3487589128 >> v2:172.21.15.73:3300/0 conn(0x7fe05415abd0 msgr2=0x7fe0540aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.776+0000 7fe053e00640 1 --2- 172.21.15.46:0/3487589128 >> v2:172.21.15.73:3300/0 conn(0x7fe05415abd0 0x7fe0540aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.776+0000 7fe053e00640 1 -- 172.21.15.46:0/3487589128 >> v1:172.21.15.73:6789/0 conn(0x7fe054069d90 legacy=0x7fe0540b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:11.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.776+0000 7fe053e00640 1 -- 172.21.15.46:0/3487589128 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0540af970 con 0x7fe0540ac2a0 2024-03-29T08:26:11.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.780+0000 7fe053e00640 1 --2- 172.21.15.46:0/3487589128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0540ac2a0 0x7fe0540ac6d0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe054071130 tx=0x7fe04c01c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37c215626f40e47 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:11.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.780+0000 7fe052a00640 1 -- 172.21.15.46:0/3487589128 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0a57b0 con 0x7fe0540ac2a0 2024-03-29T08:26:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.780+0000 7fe052a00640 1 -- 172.21.15.46:0/3487589128 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04c0a5910 con 0x7fe0540ac2a0 2024-03-29T08:26:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.780+0000 7fe052a00640 1 -- 172.21.15.46:0/3487589128 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0a5ca0 con 0x7fe0540ac2a0 2024-03-29T08:26:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.780+0000 7fe05a200640 1 -- 172.21.15.46:0/3487589128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0540ac2a0 msgr2=0x7fe0540ac6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.780+0000 7fe05a200640 1 --2- 172.21.15.46:0/3487589128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0540ac2a0 0x7fe0540ac6d0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fe054071130 tx=0x7fe04c01c8f0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.780+0000 7fe05a200640 1 -- 172.21.15.46:0/3487589128 shutdown_connections 2024-03-29T08:26:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.780+0000 7fe05a200640 1 --2- 172.21.15.46:0/3487589128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0540ac2a0 0x7fe0540ac6d0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.780+0000 7fe05a200640 1 --2- 172.21.15.46:0/3487589128 >> v2:172.21.15.73:3300/0 conn(0x7fe05415abd0 0x7fe0540aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.780+0000 7fe05a200640 1 -- 172.21.15.46:0/3487589128 >> 172.21.15.46:0/3487589128 conn(0x7fe0540cf590 msgr2=0x7fe0540cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:11.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.780+0000 7fe05a200640 1 -- 172.21.15.46:0/3487589128 shutdown_connections 2024-03-29T08:26:11.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.780+0000 7fe05a200640 1 -- 172.21.15.46:0/3487589128 wait complete. 2024-03-29T08:26:11.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe05a200640 1 Processor -- start 2024-03-29T08:26:11.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe05a200640 1 -- start start 2024-03-29T08:26:11.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe05a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe054069d90 0x7fe054155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe05a200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0540ac2a0 0x7fe0540c5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe05a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe05415abd0 0x7fe05415c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe05a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05415d5f0 con 0x7fe05415abd0 2024-03-29T08:26:11.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe05a200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0541564b0 con 0x7fe054069d90 2024-03-29T08:26:11.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe05a200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0540c5dc0 con 0x7fe0540ac2a0 2024-03-29T08:26:11.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe053e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe05415abd0 0x7fe05415c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe053e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe05415abd0 0x7fe05415c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42348/0 (socket says 172.21.15.46:42348) 2024-03-29T08:26:11.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe053e00640 1 -- 172.21.15.46:0/3653927114 learned_addr learned my addr 172.21.15.46:0/3653927114 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:11.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe053400640 1 --2- 172.21.15.46:0/3653927114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe054069d90 0x7fe054155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe04aa00640 1 --2- 172.21.15.46:0/3653927114 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0540ac2a0 0x7fe0540c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe053e00640 1 -- 172.21.15.46:0/3653927114 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0540ac2a0 msgr2=0x7fe0540c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:11.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe053e00640 1 --2- 172.21.15.46:0/3653927114 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0540ac2a0 0x7fe0540c5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe053e00640 1 -- 172.21.15.46:0/3653927114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe054069d90 msgr2=0x7fe054155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:11.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe053e00640 1 --2- 172.21.15.46:0/3653927114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe054069d90 0x7fe054155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe053e00640 1 -- 172.21.15.46:0/3653927114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c009b30 con 0x7fe05415abd0 2024-03-29T08:26:11.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe04aa00640 1 --2- 172.21.15.46:0/3653927114 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0540ac2a0 0x7fe0540c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:11.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe053e00640 1 --2- 172.21.15.46:0/3653927114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe05415abd0 0x7fe05415c2c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe04c0af040 tx=0x7fe04c019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:11.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe04be00640 1 -- 172.21.15.46:0/3653927114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0b8670 con 0x7fe05415abd0 2024-03-29T08:26:11.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe04be00640 1 -- 172.21.15.46:0/3653927114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04c0b87d0 con 0x7fe05415abd0 2024-03-29T08:26:11.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe05a200640 1 -- 172.21.15.46:0/3653927114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05415d310 con 0x7fe05415abd0 2024-03-29T08:26:11.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe04be00640 1 -- 172.21.15.46:0/3653927114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0b8b60 con 0x7fe05415abd0 2024-03-29T08:26:11.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.784+0000 7fe05a200640 1 -- 172.21.15.46:0/3653927114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0540c6a70 con 0x7fe05415abd0 2024-03-29T08:26:11.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.788+0000 7fe05a200640 1 -- 172.21.15.46:0/3653927114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0540cd9a0 con 0x7fe05415abd0 2024-03-29T08:26:11.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.788+0000 7fe04be00640 1 -- 172.21.15.46:0/3653927114 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe04c0b8cc0 con 0x7fe05415abd0 2024-03-29T08:26:11.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.788+0000 7fe04be00640 1 --2- 172.21.15.46:0/3653927114 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe0300666c0 0x7fe030068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:11.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.788+0000 7fe04be00640 1 -- 172.21.15.46:0/3653927114 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe04c11f790 con 0x7fe05415abd0 2024-03-29T08:26:11.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.788+0000 7fe053400640 1 --2- 172.21.15.46:0/3653927114 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe0300666c0 0x7fe030068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:11.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.788+0000 7fe053400640 1 --2- 172.21.15.46:0/3653927114 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe0300666c0 0x7fe030068b80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe0440079c0 tx=0x7fe044008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:11.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.792+0000 7fe04be00640 1 -- 172.21.15.46:0/3653927114 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe04c0ecb50 con 0x7fe05415abd0 2024-03-29T08:26:11.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.984+0000 7fe05a200640 1 -- 172.21.15.46:0/3653927114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe054066c90 con 0x7fe05415abd0 2024-03-29T08:26:11.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.984+0000 7fe04be00640 1 -- 172.21.15.46:0/3653927114 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4738 (secure 0 0 0) 0x7fe04c0f0170 con 0x7fe05415abd0 2024-03-29T08:26:11.987 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:26:11.987 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/1862652020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1862652020},{"type":"v1","addr":"172.21.15.73:6836","nonce":1862652020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:24:43.476334+0000","modified":"2024-03-29T08:25:17.015744+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7386},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7386":{"gid":7386,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":13,"addr":"172.21.15.46:6835/2051429508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2051429508},{"type":"v1","addr":"172.21.15.46:6835","nonce":2051429508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-29T08:26:11.987 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 63 2024-03-29T08:26:11.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.988+0000 7fe05a200640 1 -- 172.21.15.46:0/3653927114 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe0300666c0 msgr2=0x7fe030068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:11.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.988+0000 7fe05a200640 1 --2- 172.21.15.46:0/3653927114 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe0300666c0 0x7fe030068b80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe0440079c0 tx=0x7fe044008040 comp rx=0 tx=0).stop 2024-03-29T08:26:11.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.988+0000 7fe05a200640 1 -- 172.21.15.46:0/3653927114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe05415abd0 msgr2=0x7fe05415c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:11.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.988+0000 7fe05a200640 1 --2- 172.21.15.46:0/3653927114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe05415abd0 0x7fe05415c2c0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fe04c0af040 tx=0x7fe04c019690 comp rx=0 tx=0).stop 2024-03-29T08:26:11.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.988+0000 7fe05a200640 1 -- 172.21.15.46:0/3653927114 shutdown_connections 2024-03-29T08:26:11.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.988+0000 7fe05a200640 1 --2- 172.21.15.46:0/3653927114 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe0300666c0 0x7fe030068b80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.988+0000 7fe05a200640 1 --2- 172.21.15.46:0/3653927114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe05415abd0 0x7fe05415c2c0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.988+0000 7fe05a200640 1 --2- 172.21.15.46:0/3653927114 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0540ac2a0 0x7fe0540c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.988+0000 7fe05a200640 1 --2- 172.21.15.46:0/3653927114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe054069d90 0x7fe054155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:11.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.988+0000 7fe05a200640 1 -- 172.21.15.46:0/3653927114 >> 172.21.15.46:0/3653927114 conn(0x7fe0540cf590 msgr2=0x7fe05406a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:11.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.988+0000 7fe05a200640 1 -- 172.21.15.46:0/3653927114 shutdown_connections 2024-03-29T08:26:11.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:11.988+0000 7fe05a200640 1 -- 172.21.15.46:0/3653927114 wait complete. 2024-03-29T08:26:12.008 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-03-29T08:26:12.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.148+0000 7f2459400640 1 Processor -- start 2024-03-29T08:26:12.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.148+0000 7f2459400640 1 -- start start 2024-03-29T08:26:12.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.152+0000 7f2459400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2454142a50 0x7f24540cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.152+0000 7f2459400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f245415df90 0x7f24540cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.152+0000 7f2459400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f24540ae280 0x7f24540ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.152+0000 7f2459400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f245415d260 con 0x7f2454142a50 2024-03-29T08:26:12.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.152+0000 7f2459400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24540aefd0 con 0x7f24540ae280 2024-03-29T08:26:12.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.152+0000 7f2459400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f245415d4b0 con 0x7f245415df90 2024-03-29T08:26:12.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.152+0000 7f2452800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2454142a50 0x7f24540cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.152+0000 7f2452800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2454142a50 0x7f24540cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42350/0 (socket says 172.21.15.46:42350) 2024-03-29T08:26:12.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.152+0000 7f2452800640 1 -- 172.21.15.46:0/1389512963 learned_addr learned my addr 172.21.15.46:0/1389512963 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:12.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.152+0000 7f2449e00640 1 --2- 172.21.15.46:0/1389512963 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f245415df90 0x7f24540cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.152+0000 7f2453200640 1 --2- 172.21.15.46:0/1389512963 >> v2:172.21.15.73:3300/0 conn(0x7f24540ae280 0x7f24540ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.152+0000 7f2452800640 1 -- 172.21.15.46:0/1389512963 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f245415df90 msgr2=0x7f24540cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:12.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.152+0000 7f2452800640 1 --2- 172.21.15.46:0/1389512963 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f245415df90 0x7f24540cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.152+0000 7f2452800640 1 -- 172.21.15.46:0/1389512963 >> v2:172.21.15.73:3300/0 conn(0x7f24540ae280 msgr2=0x7f24540ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:12.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.152+0000 7f2452800640 1 --2- 172.21.15.46:0/1389512963 >> v2:172.21.15.73:3300/0 conn(0x7f24540ae280 0x7f24540ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.152+0000 7f2452800640 1 -- 172.21.15.46:0/1389512963 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245415d790 con 0x7f2454142a50 2024-03-29T08:26:12.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.156+0000 7f2452800640 1 --2- 172.21.15.46:0/1389512963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2454142a50 0x7f24540cb790 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f245406e580 tx=0x7f244401eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d2c9b9ce2be682 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:12.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.156+0000 7f2451e00640 1 -- 172.21.15.46:0/1389512963 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440a7b00 con 0x7f2454142a50 2024-03-29T08:26:12.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.156+0000 7f2451e00640 1 -- 172.21.15.46:0/1389512963 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24440a7c60 con 0x7f2454142a50 2024-03-29T08:26:12.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.156+0000 7f2451e00640 1 -- 172.21.15.46:0/1389512963 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440af440 con 0x7f2454142a50 2024-03-29T08:26:12.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.156+0000 7f2459400640 1 -- 172.21.15.46:0/1389512963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2454142a50 msgr2=0x7f24540cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.156+0000 7f2459400640 1 --2- 172.21.15.46:0/1389512963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2454142a50 0x7f24540cb790 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f245406e580 tx=0x7f244401eb60 comp rx=0 tx=0).stop 2024-03-29T08:26:12.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.156+0000 7f2459400640 1 -- 172.21.15.46:0/1389512963 shutdown_connections 2024-03-29T08:26:12.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.156+0000 7f2459400640 1 --2- 172.21.15.46:0/1389512963 >> v2:172.21.15.73:3300/0 conn(0x7f24540ae280 0x7f24540ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.156+0000 7f2459400640 1 --2- 172.21.15.46:0/1389512963 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f245415df90 0x7f24540cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.156+0000 7f2459400640 1 --2- 172.21.15.46:0/1389512963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2454142a50 0x7f24540cb790 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.156+0000 7f2459400640 1 -- 172.21.15.46:0/1389512963 >> 172.21.15.46:0/1389512963 conn(0x7f245406a390 msgr2=0x7f24540cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:12.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.156+0000 7f2459400640 1 -- 172.21.15.46:0/1389512963 shutdown_connections 2024-03-29T08:26:12.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.156+0000 7f2459400640 1 -- 172.21.15.46:0/1389512963 wait complete. 2024-03-29T08:26:12.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.156+0000 7f2459400640 1 Processor -- start 2024-03-29T08:26:12.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.156+0000 7f2459400640 1 -- start start 2024-03-29T08:26:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f2459400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24540ae280 0x7f2454137ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f2459400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2454142a50 0x7f2454133850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f2459400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f245415df90 0x7f2454158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f2459400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24541396d0 con 0x7f24540ae280 2024-03-29T08:26:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f2459400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24541399b0 con 0x7f2454142a50 2024-03-29T08:26:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f2459400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2454139c90 con 0x7f245415df90 2024-03-29T08:26:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f2452800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24540ae280 0x7f2454137ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f2452800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24540ae280 0x7f2454137ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42362/0 (socket says 172.21.15.46:42362) 2024-03-29T08:26:12.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f2452800640 1 -- 172.21.15.46:0/3552795112 learned_addr learned my addr 172.21.15.46:0/3552795112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:12.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f2452800640 1 -- 172.21.15.46:0/3552795112 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f245415df90 msgr2=0x7f2454158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f2452800640 1 --2- 172.21.15.46:0/3552795112 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f245415df90 0x7f2454158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f2452800640 1 -- 172.21.15.46:0/3552795112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2454142a50 msgr2=0x7f2454133850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f2452800640 1 --2- 172.21.15.46:0/3552795112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2454142a50 0x7f2454133850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f2452800640 1 -- 172.21.15.46:0/3552795112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244400b770 con 0x7f24540ae280 2024-03-29T08:26:12.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f2452800640 1 --2- 172.21.15.46:0/3552795112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24540ae280 0x7f2454137ae0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f244400b2c0 tx=0x7f244401b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:12.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f244b400640 1 -- 172.21.15.46:0/3552795112 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2444006ee0 con 0x7f24540ae280 2024-03-29T08:26:12.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f244b400640 1 -- 172.21.15.46:0/3552795112 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24440afcc0 con 0x7f24540ae280 2024-03-29T08:26:12.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f2459400640 1 -- 172.21.15.46:0/3552795112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2454139f70 con 0x7f24540ae280 2024-03-29T08:26:12.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f244b400640 1 -- 172.21.15.46:0/3552795112 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440b8640 con 0x7f24540ae280 2024-03-29T08:26:12.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f2459400640 1 -- 172.21.15.46:0/3552795112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2454131fb0 con 0x7f24540ae280 2024-03-29T08:26:12.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.160+0000 7f244b400640 1 -- 172.21.15.46:0/3552795112 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f24440b6040 con 0x7f24540ae280 2024-03-29T08:26:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f244b400640 1 --2- 172.21.15.46:0/3552795112 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2430066690 0x7f2430068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f244b400640 1 -- 172.21.15.46:0/3552795112 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2444120fe0 con 0x7f24540ae280 2024-03-29T08:26:12.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f2449e00640 1 --2- 172.21.15.46:0/3552795112 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2430066690 0x7f2430068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f245c8df000 1 Processor -- start 2024-03-29T08:26:12.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f245c8df000 1 -- start start 2024-03-29T08:26:12.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f2449e00640 1 --2- 172.21.15.46:0/3552795112 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2430066690 0x7f2430068b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f245413bab0 tx=0x7f243c009520 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:12.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f245c8df000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d94092cdd0 0x55d94092d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f245c8df000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092b080 0x55d94092b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f245c8df000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 0x55d940974aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f245c8df000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9407ef340 con 0x55d94092b080 2024-03-29T08:26:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f245c8df000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d94077bcd0 con 0x55d94092cdd0 2024-03-29T08:26:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f245c8df000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9407792a0 con 0x55d94092b9a0 2024-03-29T08:26:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f2449e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092b080 0x55d94092b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f2449e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092b080 0x55d94092b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42372/0 (socket says 172.21.15.46:42372) 2024-03-29T08:26:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f2449e00640 1 -- 172.21.15.46:0/4154494862 learned_addr learned my addr 172.21.15.46:0/4154494862 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f2453200640 1 --2- 172.21.15.46:0/4154494862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 0x55d940974aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f2452800640 1 --2- 172.21.15.46:0/4154494862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d94092cdd0 0x55d94092d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f2449e00640 1 -- 172.21.15.46:0/4154494862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 msgr2=0x55d940974aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f2449e00640 1 --2- 172.21.15.46:0/4154494862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 0x55d940974aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f2449e00640 1 -- 172.21.15.46:0/4154494862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d94092cdd0 msgr2=0x55d94092d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f2449e00640 1 --2- 172.21.15.46:0/4154494862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d94092cdd0 0x55d94092d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f2449e00640 1 -- 172.21.15.46:0/4154494862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9404674b0 con 0x55d94092b080 2024-03-29T08:26:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f2453200640 1 --2- 172.21.15.46:0/4154494862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 0x55d940974aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f2449e00640 1 --2- 172.21.15.46:0/4154494862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092b080 0x55d94092b460 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f243c009840 tx=0x7f243c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa17bd7d50eb800 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:12.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f2443400640 1 -- 172.21.15.46:0/4154494862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c015ac0 con 0x55d94092b080 2024-03-29T08:26:12.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f2443400640 1 -- 172.21.15.46:0/4154494862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c015c20 con 0x55d94092b080 2024-03-29T08:26:12.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.164+0000 7f2443400640 1 -- 172.21.15.46:0/4154494862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c013450 con 0x55d94092b080 2024-03-29T08:26:12.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 -- 172.21.15.46:0/4154494862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092b080 msgr2=0x55d94092b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 --2- 172.21.15.46:0/4154494862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092b080 0x55d94092b460 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f243c009840 tx=0x7f243c011000 comp rx=0 tx=0).stop 2024-03-29T08:26:12.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 -- 172.21.15.46:0/4154494862 shutdown_connections 2024-03-29T08:26:12.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 --2- 172.21.15.46:0/4154494862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 0x55d940974aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 --2- 172.21.15.46:0/4154494862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092b080 0x55d94092b460 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 --2- 172.21.15.46:0/4154494862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d94092cdd0 0x55d94092d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 -- 172.21.15.46:0/4154494862 >> 172.21.15.46:0/4154494862 conn(0x55d940923160 msgr2=0x55d940925550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:12.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 -- 172.21.15.46:0/4154494862 shutdown_connections 2024-03-29T08:26:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 -- 172.21.15.46:0/4154494862 wait complete. 2024-03-29T08:26:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 -- start start 2024-03-29T08:26:12.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 Processor -- start 2024-03-29T08:26:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d94092b080 0x55d940977500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 0x55d940977a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092cdd0 0x55d940977f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d940774b90 con 0x55d94092cdd0 2024-03-29T08:26:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9407320a0 con 0x55d94092b080 2024-03-29T08:26:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d940731d40 con 0x55d94092b9a0 2024-03-29T08:26:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 10 client.0 _ll_register_callbacks cb 0x55d940829c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:26:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f2453200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092cdd0 0x55d940977f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f2453200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092cdd0 0x55d940977f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42380/0 (socket says 172.21.15.46:42380) 2024-03-29T08:26:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f2453200640 1 -- 172.21.15.46:0/3204698735 learned_addr learned my addr 172.21.15.46:0/3204698735 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f2452800640 1 --2- 172.21.15.46:0/3204698735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d94092b080 0x55d940977500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f2449e00640 1 --2- 172.21.15.46:0/3204698735 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 0x55d940977a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f2453200640 1 -- 172.21.15.46:0/3204698735 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 msgr2=0x55d940977a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f2453200640 1 --2- 172.21.15.46:0/3204698735 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 0x55d940977a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f2453200640 1 -- 172.21.15.46:0/3204698735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d94092b080 msgr2=0x55d940977500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f2453200640 1 --2- 172.21.15.46:0/3204698735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d94092b080 0x55d940977500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f2453200640 1 -- 172.21.15.46:0/3204698735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243c00e7e0 con 0x55d94092cdd0 2024-03-29T08:26:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f2452800640 1 --2- 172.21.15.46:0/3204698735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d94092b080 0x55d940977500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:26:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f2453200640 1 --2- 172.21.15.46:0/3204698735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092cdd0 0x55d940977f80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f244c007e10 tx=0x7f244c010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af192ee28054434 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f2437e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:26:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f2437e00640 1 -- 172.21.15.46:0/3204698735 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c00e050 con 0x55d94092cdd0 2024-03-29T08:26:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f2437e00640 1 -- 172.21.15.46:0/3204698735 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f244c009e30 con 0x55d94092cdd0 2024-03-29T08:26:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 -- 172.21.15.46:0/3204698735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d940744550 con 0x55d94092cdd0 2024-03-29T08:26:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f2437e00640 1 -- 172.21.15.46:0/3204698735 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f244c010e20 con 0x55d94092cdd0 2024-03-29T08:26:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 10 client.7887 fetch_fsmap learned FSMap version 63 2024-03-29T08:26:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.168+0000 7f245c8df000 1 -- 172.21.15.46:0/3204698735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55d940739ba0 con 0x55d94092cdd0 2024-03-29T08:26:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f2437e00640 1 -- 172.21.15.46:0/3204698735 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d940744550 con 0x55d94092cdd0 2024-03-29T08:26:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f2437e00640 1 -- 172.21.15.46:0/3204698735 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 63) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f244c015440 con 0x55d94092cdd0 2024-03-29T08:26:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 10 client.7887 fetch_fsmap finished waiting for FSMap version 63 2024-03-29T08:26:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 10 client.7887 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:26:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 20 client.7887 populate_metadata read hostname 'smithi046' 2024-03-29T08:26:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 1 --2- 172.21.15.46:0/3204698735 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55d9409831b0 0x55d9409855a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 4 client.7887 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:26:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 1 -- 172.21.15.46:0/3204698735 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d940771970 con 0x55d9409831b0 2024-03-29T08:26:12.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f2452800640 1 --2- 172.21.15.46:0/3204698735 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55d9409831b0 0x55d9409855a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f2452800640 1 --2- 172.21.15.46:0/3204698735 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55d9409831b0 0x55d9409855a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4b10042ce4fa64 server_cookie=b81c41fd2df295d2 in_seq=0 out_seq=0 2024-03-29T08:26:12.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f2437e00640 10 client.7887 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:26:12.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f2437e00640 1 -- 172.21.15.46:0/3204698735 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55d940771970 con 0x55d9409831b0 2024-03-29T08:26:12.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f2437e00640 10 client.7887 handle_command_reply: tid=0 2024-03-29T08:26:12.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 1 client.7887 shutdown 2024-03-29T08:26:12.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 1 -- 172.21.15.46:0/3204698735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092cdd0 msgr2=0x55d940977f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 1 --2- 172.21.15.46:0/3204698735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092cdd0 0x55d940977f80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f244c007e10 tx=0x7f244c010590 comp rx=0 tx=0).stop 2024-03-29T08:26:12.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 1 -- 172.21.15.46:0/3204698735 shutdown_connections 2024-03-29T08:26:12.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 1 --2- 172.21.15.46:0/3204698735 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55d9409831b0 0x55d9409855a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 1 --2- 172.21.15.46:0/3204698735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092cdd0 0x55d940977f80 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 1 --2- 172.21.15.46:0/3204698735 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 0x55d940977a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 1 --2- 172.21.15.46:0/3204698735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d94092b080 0x55d940977500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f2437e00640 0 client.7887 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:26:12.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 1 -- 172.21.15.46:0/3204698735 >> 172.21.15.46:0/3204698735 conn(0x55d940923160 msgr2=0x55d94092bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:12.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 1 -- 172.21.15.46:0/3204698735 shutdown_connections 2024-03-29T08:26:12.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 1 -- 172.21.15.46:0/3204698735 wait complete. 2024-03-29T08:26:12.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.172+0000 7f245c8df000 20 client.7887 trim_cache size 0 max 16384 2024-03-29T08:26:12.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.184+0000 7f245c8df000 1 Processor -- start 2024-03-29T08:26:12.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.184+0000 7f245c8df000 1 -- start start 2024-03-29T08:26:12.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.184+0000 7f245c8df000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d940923160 0x55d9409cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.184+0000 7f245c8df000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d94092b080 0x55d9409cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.184+0000 7f245c8df000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 0x55d9409cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.184+0000 7f245c8df000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d940923020 con 0x55d940923160 2024-03-29T08:26:12.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.184+0000 7f245c8df000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d940431c40 con 0x55d94092b080 2024-03-29T08:26:12.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.184+0000 7f245c8df000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d94071d520 con 0x55d94092b9a0 2024-03-29T08:26:12.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.184+0000 7f2452800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d940923160 0x55d9409cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.184+0000 7f2452800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d940923160 0x55d9409cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42384/0 (socket says 172.21.15.46:42384) 2024-03-29T08:26:12.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.184+0000 7f2452800640 1 -- 172.21.15.46:0/399263732 learned_addr learned my addr 172.21.15.46:0/399263732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:12.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2452800640 1 -- 172.21.15.46:0/399263732 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 msgr2=0x55d9409cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:12.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2452800640 1 --2- 172.21.15.46:0/399263732 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 0x55d9409cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2452800640 1 -- 172.21.15.46:0/399263732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d94092b080 msgr2=0x55d9409cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:12.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2452800640 1 --2- 172.21.15.46:0/399263732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d94092b080 0x55d9409cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2452800640 1 -- 172.21.15.46:0/399263732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d94071fef0 con 0x55d940923160 2024-03-29T08:26:12.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2452800640 1 --2- 172.21.15.46:0/399263732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d940923160 0x55d9409cf180 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f24440ee970 tx=0x7f244401eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedc5ed0303b8edc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:12.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2443e00640 1 -- 172.21.15.46:0/399263732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440f1f50 con 0x55d940923160 2024-03-29T08:26:12.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2443e00640 1 -- 172.21.15.46:0/399263732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24440f5d80 con 0x55d940923160 2024-03-29T08:26:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2443e00640 1 -- 172.21.15.46:0/399263732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24440fa340 con 0x55d940923160 2024-03-29T08:26:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 1 -- 172.21.15.46:0/399263732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d940923160 msgr2=0x55d9409cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 1 --2- 172.21.15.46:0/399263732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d940923160 0x55d9409cf180 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f24440ee970 tx=0x7f244401eb60 comp rx=0 tx=0).stop 2024-03-29T08:26:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 1 -- 172.21.15.46:0/399263732 shutdown_connections 2024-03-29T08:26:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 1 --2- 172.21.15.46:0/399263732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d94092b080 0x55d9409cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 1 --2- 172.21.15.46:0/399263732 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 0x55d9409cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 1 --2- 172.21.15.46:0/399263732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d940923160 0x55d9409cf180 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 1 -- 172.21.15.46:0/399263732 >> 172.21.15.46:0/399263732 conn(0x55d940926510 msgr2=0x55d940932d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 1 -- 172.21.15.46:0/399263732 shutdown_connections 2024-03-29T08:26:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 1 -- 172.21.15.46:0/399263732 wait complete. 2024-03-29T08:26:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 1 -- start start 2024-03-29T08:26:12.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 1 Processor -- start 2024-03-29T08:26:12.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d940923160 0x55d9409d3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092b080 0x55d9409d4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 0x55d94098e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d940770380 con 0x55d94092b080 2024-03-29T08:26:12.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d940925be0 con 0x55d940923160 2024-03-29T08:26:12.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9407776f0 con 0x55d94092b9a0 2024-03-29T08:26:12.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f245c8df000 10 client.0 _ll_register_callbacks cb 0x55d940812870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:26:12.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2449e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092b080 0x55d9409d4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2449e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092b080 0x55d9409d4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42390/0 (socket says 172.21.15.46:42390) 2024-03-29T08:26:12.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2449e00640 1 -- 172.21.15.46:0/746123343 learned_addr learned my addr 172.21.15.46:0/746123343 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:12.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2453200640 1 --2- 172.21.15.46:0/746123343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 0x55d94098e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2452800640 1 --2- 172.21.15.46:0/746123343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d940923160 0x55d9409d3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2449e00640 1 -- 172.21.15.46:0/746123343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 msgr2=0x55d94098e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2449e00640 1 --2- 172.21.15.46:0/746123343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 0x55d94098e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2449e00640 1 -- 172.21.15.46:0/746123343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d940923160 msgr2=0x55d9409d3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2449e00640 1 --2- 172.21.15.46:0/746123343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d940923160 0x55d9409d3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2449e00640 1 -- 172.21.15.46:0/746123343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2454131fb0 con 0x55d94092b080 2024-03-29T08:26:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2453200640 1 --2- 172.21.15.46:0/746123343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 0x55d94098e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2452800640 1 --2- 172.21.15.46:0/746123343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d940923160 0x55d9409d3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2449e00640 1 --2- 172.21.15.46:0/746123343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092b080 0x55d9409d4020 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f243c020000 tx=0x7f243c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77f1f4777c35a2b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.188+0000 7f2437e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:26:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2437e00640 1 -- 172.21.15.46:0/746123343 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c013b90 con 0x55d94092b080 2024-03-29T08:26:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2437e00640 1 -- 172.21.15.46:0/746123343 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f243c013cf0 con 0x55d94092b080 2024-03-29T08:26:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 1 -- 172.21.15.46:0/746123343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d940724620 con 0x55d94092b080 2024-03-29T08:26:12.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2437e00640 1 -- 172.21.15.46:0/746123343 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f243c013eb0 con 0x55d94092b080 2024-03-29T08:26:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 10 client.7893 fetch_fsmap learned FSMap version 63 2024-03-29T08:26:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 1 -- 172.21.15.46:0/746123343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55d940470fc0 con 0x55d94092b080 2024-03-29T08:26:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2437e00640 1 -- 172.21.15.46:0/746123343 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d940724620 con 0x55d94092b080 2024-03-29T08:26:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2437e00640 1 -- 172.21.15.46:0/746123343 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 63) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f243c031030 con 0x55d94092b080 2024-03-29T08:26:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 10 client.7893 fetch_fsmap finished waiting for FSMap version 63 2024-03-29T08:26:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 10 client.7893 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:26:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 20 client.7893 populate_metadata read hostname 'smithi046' 2024-03-29T08:26:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 1 --2- 172.21.15.46:0/746123343 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55d94092cdd0 0x55d9409dc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 4 client.7893 mds_command: new command op to 7386 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-29T08:26:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 1 -- 172.21.15.46:0/746123343 --> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55d9407734e0 con 0x55d94092cdd0 2024-03-29T08:26:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2452800640 1 --2- 172.21.15.46:0/746123343 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55d94092cdd0 0x55d9409dc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2437e00640 10 client.7893 ms_handle_connect on v2:172.21.15.46:6833/2051429508 2024-03-29T08:26:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2452800640 1 --2- 172.21.15.46:0/746123343 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55d94092cdd0 0x55d9409dc9c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c31f2e4d5bc15d4 server_cookie=567cd7c96283dc0f in_seq=0 out_seq=0 2024-03-29T08:26:12.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2437e00640 1 -- 172.21.15.46:0/746123343 <== mds.0 v2:172.21.15.46:6833/2051429508 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+144 (crc 0 0 0) 0x55d9407734e0 con 0x55d94092cdd0 2024-03-29T08:26:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2437e00640 10 client.7893 handle_command_reply: tid=0 2024-03-29T08:26:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 1 client.7893 shutdown 2024-03-29T08:26:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 1 -- 172.21.15.46:0/746123343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092b080 msgr2=0x55d9409d4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 1 --2- 172.21.15.46:0/746123343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092b080 0x55d9409d4020 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f243c020000 tx=0x7f243c0046d0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 1 -- 172.21.15.46:0/746123343 shutdown_connections 2024-03-29T08:26:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 1 --2- 172.21.15.46:0/746123343 >> [v2:172.21.15.46:6833/2051429508,v1:172.21.15.46:6835/2051429508] conn(0x55d94092cdd0 0x55d9409dc9c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 1 --2- 172.21.15.46:0/746123343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55d94092b9a0 0x55d94098e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 1 --2- 172.21.15.46:0/746123343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d94092b080 0x55d9409d4020 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 1 --2- 172.21.15.46:0/746123343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d940923160 0x55d9409d3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2437e00640 0 client.7893 ms_handle_reset on v2:172.21.15.46:6833/2051429508 2024-03-29T08:26:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 1 -- 172.21.15.46:0/746123343 >> 172.21.15.46:0/746123343 conn(0x55d940926510 msgr2=0x55d940975fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 1 -- 172.21.15.46:0/746123343 shutdown_connections 2024-03-29T08:26:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 1 -- 172.21.15.46:0/746123343 wait complete. 2024-03-29T08:26:12.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f245c8df000 20 client.7893 trim_cache size 0 max 16384 2024-03-29T08:26:12.198 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:26:12.198 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-03-29T08:26:12.198 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:26:12.198 INFO:teuthology.orchestra.run.smithi046.stdout: "damage_type": "backtrace", 2024-03-29T08:26:12.198 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 3589788952, 2024-03-29T08:26:12.198 INFO:teuthology.orchestra.run.smithi046.stdout: "ino": 1099511627777, 2024-03-29T08:26:12.198 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/testdir/hardlink" 2024-03-29T08:26:12.198 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:26:12.198 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-03-29T08:26:12.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2459400640 1 -- 172.21.15.46:0/3552795112 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2430066690 msgr2=0x7f2430068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2459400640 1 --2- 172.21.15.46:0/3552795112 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2430066690 0x7f2430068b50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f245413bab0 tx=0x7f243c009520 comp rx=0 tx=0).stop 2024-03-29T08:26:12.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2459400640 1 -- 172.21.15.46:0/3552795112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24540ae280 msgr2=0x7f2454137ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2459400640 1 --2- 172.21.15.46:0/3552795112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24540ae280 0x7f2454137ae0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f244400b2c0 tx=0x7f244401b900 comp rx=0 tx=0).stop 2024-03-29T08:26:12.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2459400640 1 -- 172.21.15.46:0/3552795112 shutdown_connections 2024-03-29T08:26:12.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2459400640 1 --2- 172.21.15.46:0/3552795112 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2430066690 0x7f2430068b50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2459400640 1 --2- 172.21.15.46:0/3552795112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2454142a50 0x7f2454133850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2459400640 1 --2- 172.21.15.46:0/3552795112 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f245415df90 0x7f2454158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2459400640 1 --2- 172.21.15.46:0/3552795112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24540ae280 0x7f2454137ae0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2459400640 1 -- 172.21.15.46:0/3552795112 >> 172.21.15.46:0/3552795112 conn(0x7f245406a390 msgr2=0x7f24540cd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:12.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2459400640 1 -- 172.21.15.46:0/3552795112 shutdown_connections 2024-03-29T08:26:12.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.192+0000 7f2459400640 1 -- 172.21.15.46:0/3552795112 wait complete. 2024-03-29T08:26:12.213 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:26:12.213 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:26:12.222 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:26:12.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.400+0000 7fbb07000640 1 Processor -- start 2024-03-29T08:26:12.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.400+0000 7fbb07000640 1 -- start start 2024-03-29T08:26:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.404+0000 7fbb07000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbb000ae810 0x7fbb000caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.404+0000 7fbb07000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb000cbc10 con 0x7fbb000cb780 2024-03-29T08:26:12.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.404+0000 7fbb07000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb000c9bd0 con 0x7fbb000ce8a0 2024-03-29T08:26:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.404+0000 7fbb07000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb000c9eb0 con 0x7fbb000ae810 2024-03-29T08:26:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.404+0000 7fbaff400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbb000ae810 0x7fbb000caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.404+0000 7fbaff400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbb000ae810 0x7fbb000caf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42394/0 (socket says 172.21.15.46:42394) 2024-03-29T08:26:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.404+0000 7fbaff400640 1 -- 172.21.15.46:0/703477612 learned_addr learned my addr 172.21.15.46:0/703477612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.404+0000 7fbb05000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbb000cb780 0x7fbb000aecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46976/0 (socket says 172.21.15.46:46976) 2024-03-29T08:26:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.404+0000 7fbaff400640 1 -- 172.21.15.46:0/703477612 >> v1:172.21.15.73:6789/0 conn(0x7fbb000ce8a0 legacy=0x7fbb000cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.404+0000 7fbaff400640 1 -- 172.21.15.46:0/703477612 >> v1:172.21.15.46:6789/0 conn(0x7fbb000cb780 legacy=0x7fbb000aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.404+0000 7fbaff400640 1 -- 172.21.15.46:0/703477612 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb000ca190 con 0x7fbb000ae810 2024-03-29T08:26:12.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.404+0000 7fbafea00640 1 -- 172.21.15.46:0/703477612 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2994733091 0 0) 0x7fbb000cbc10 con 0x7fbb000cb780 2024-03-29T08:26:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbaff400640 1 --2- 172.21.15.46:0/703477612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb000ae810 0x7fbb000caf50 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009870 tx=0x7fbaec01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5af018af284e25 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbafea00640 1 -- 172.21.15.46:0/703477612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0a95f0 con 0x7fbb000ae810 2024-03-29T08:26:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbafea00640 1 -- 172.21.15.46:0/703477612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaec0a9750 con 0x7fbb000ae810 2024-03-29T08:26:12.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbafea00640 1 -- 172.21.15.46:0/703477612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0a9ae0 con 0x7fbb000ae810 2024-03-29T08:26:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbb07000640 1 -- 172.21.15.46:0/703477612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb000ae810 msgr2=0x7fbb000caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbb07000640 1 --2- 172.21.15.46:0/703477612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb000ae810 0x7fbb000caf50 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fbaec009870 tx=0x7fbaec01c530 comp rx=0 tx=0).stop 2024-03-29T08:26:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbb07000640 1 -- 172.21.15.46:0/703477612 shutdown_connections 2024-03-29T08:26:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbb07000640 1 --2- 172.21.15.46:0/703477612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb000ae810 0x7fbb000caf50 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbb07000640 1 -- 172.21.15.46:0/703477612 >> 172.21.15.46:0/703477612 conn(0x7fbb00098a40 msgr2=0x7fbb000cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbb07000640 1 -- 172.21.15.46:0/703477612 shutdown_connections 2024-03-29T08:26:12.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbb07000640 1 -- 172.21.15.46:0/703477612 wait complete. 2024-03-29T08:26:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbb07000640 1 Processor -- start 2024-03-29T08:26:12.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbb07000640 1 -- start start 2024-03-29T08:26:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbb07000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb000ae810 0x7fbb0015f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbb07000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb000cb780 0x7fbb00155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbb07000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb000ce8a0 0x7fbb0015b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbb07000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb00157080 con 0x7fbb000cb780 2024-03-29T08:26:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbb07000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb000c6b80 con 0x7fbb000ce8a0 2024-03-29T08:26:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.408+0000 7fbb07000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb000c6e60 con 0x7fbb000ae810 2024-03-29T08:26:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbaff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb000cb780 0x7fbb00155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbaff400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb000cb780 0x7fbb00155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42410/0 (socket says 172.21.15.46:42410) 2024-03-29T08:26:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbaff400640 1 -- 172.21.15.46:0/1872186286 learned_addr learned my addr 172.21.15.46:0/1872186286 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbb05000640 1 --2- 172.21.15.46:0/1872186286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb000ce8a0 0x7fbb0015b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbaffe00640 1 --2- 172.21.15.46:0/1872186286 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb000ae810 0x7fbb0015f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbaff400640 1 -- 172.21.15.46:0/1872186286 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb000ae810 msgr2=0x7fbb0015f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbaff400640 1 --2- 172.21.15.46:0/1872186286 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb000ae810 0x7fbb0015f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbaff400640 1 -- 172.21.15.46:0/1872186286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb000ce8a0 msgr2=0x7fbb0015b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbaff400640 1 --2- 172.21.15.46:0/1872186286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb000ce8a0 0x7fbb0015b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbaff400640 1 -- 172.21.15.46:0/1872186286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaec0094d0 con 0x7fbb000cb780 2024-03-29T08:26:12.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbaff400640 1 --2- 172.21.15.46:0/1872186286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb000cb780 0x7fbb00155ae0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0ab040 tx=0x7fbaec0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbaf7e00640 1 -- 172.21.15.46:0/1872186286 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0a95b0 con 0x7fbb000cb780 2024-03-29T08:26:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbaf7e00640 1 -- 172.21.15.46:0/1872186286 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaec004030 con 0x7fbb000cb780 2024-03-29T08:26:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbaf7e00640 1 -- 172.21.15.46:0/1872186286 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec0043c0 con 0x7fbb000cb780 2024-03-29T08:26:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbb07000640 1 -- 172.21.15.46:0/1872186286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb000c7140 con 0x7fbb000cb780 2024-03-29T08:26:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbb07000640 1 -- 172.21.15.46:0/1872186286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0015fd60 con 0x7fbb000cb780 2024-03-29T08:26:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbb07000640 1 -- 172.21.15.46:0/1872186286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb00066c90 con 0x7fbb000cb780 2024-03-29T08:26:12.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.412+0000 7fbaf7e00640 1 -- 172.21.15.46:0/1872186286 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbaec003620 con 0x7fbb000cb780 2024-03-29T08:26:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.416+0000 7fbaf7e00640 1 --2- 172.21.15.46:0/1872186286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbae0066690 0x7fbae0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.416+0000 7fbaf7e00640 1 -- 172.21.15.46:0/1872186286 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbaec11e980 con 0x7fbb000cb780 2024-03-29T08:26:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.416+0000 7fbaffe00640 1 --2- 172.21.15.46:0/1872186286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbae0066690 0x7fbae0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.416+0000 7fbaffe00640 1 --2- 172.21.15.46:0/1872186286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbae0066690 0x7fbae0068b50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0007fd0 tx=0x7fbaf0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:12.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.420+0000 7fbaf7e00640 1 -- 172.21.15.46:0/1872186286 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbaec0ebd30 con 0x7fbb000cb780 2024-03-29T08:26:12.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.580+0000 7fbb07000640 1 -- 172.21.15.46:0/1872186286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb000cf9a0 con 0x7fbb000cb780 2024-03-29T08:26:12.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.580+0000 7fbaf7e00640 1 -- 172.21.15.46:0/1872186286 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+16185 (secure 0 0 0) 0x7fbaec0c0070 con 0x7fbb000cb780 2024-03-29T08:26:12.582 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:26:12.582 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:25:02.334373+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:24:41.113618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-29T08:24:42.487380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6836/4166845858":"2024-03-30T08:25:02.334350+0000","172.21.15.73:6834/4166845858":"2024-03-30T08:25:02.334350+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:26:12.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.584+0000 7fbb07000640 1 -- 172.21.15.46:0/1872186286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbae0066690 msgr2=0x7fbae0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.584+0000 7fbb07000640 1 --2- 172.21.15.46:0/1872186286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbae0066690 0x7fbae0068b50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0007fd0 tx=0x7fbaf0009040 comp rx=0 tx=0).stop 2024-03-29T08:26:12.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.584+0000 7fbb07000640 1 -- 172.21.15.46:0/1872186286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb000cb780 msgr2=0x7fbb00155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.584+0000 7fbb07000640 1 --2- 172.21.15.46:0/1872186286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb000cb780 0x7fbb00155ae0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0ab040 tx=0x7fbaec0046d0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.584+0000 7fbb07000640 1 -- 172.21.15.46:0/1872186286 shutdown_connections 2024-03-29T08:26:12.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.584+0000 7fbb07000640 1 --2- 172.21.15.46:0/1872186286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbae0066690 0x7fbae0068b50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.584+0000 7fbb07000640 1 --2- 172.21.15.46:0/1872186286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb000ce8a0 0x7fbb0015b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.584+0000 7fbb07000640 1 --2- 172.21.15.46:0/1872186286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb000cb780 0x7fbb00155ae0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.584+0000 7fbb07000640 1 --2- 172.21.15.46:0/1872186286 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb000ae810 0x7fbb0015f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.584+0000 7fbb07000640 1 -- 172.21.15.46:0/1872186286 >> 172.21.15.46:0/1872186286 conn(0x7fbb00098a40 msgr2=0x7fbb000cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:12.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.584+0000 7fbb07000640 1 -- 172.21.15.46:0/1872186286 shutdown_connections 2024-03-29T08:26:12.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.584+0000 7fbb07000640 1 -- 172.21.15.46:0/1872186286 wait complete. 2024-03-29T08:26:12.604 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:24:41.113618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:24:42.487380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:26:12.604 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:26:12.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.744+0000 7f5683200640 1 Processor -- start 2024-03-29T08:26:12.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.744+0000 7f5683200640 1 -- start start 2024-03-29T08:26:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.748+0000 7f5683200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f567c0cbea0 0x7f567c0b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.748+0000 7f5683200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f567c15efe0 0x7f567c0b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.748+0000 7f5683200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f567c0b20b0 con 0x7f567c143eb0 2024-03-29T08:26:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.748+0000 7f5683200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f567c0b2360 con 0x7f567c0cbea0 2024-03-29T08:26:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.748+0000 7f5683200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567c15e500 con 0x7f567c15efe0 2024-03-29T08:26:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.748+0000 7f567be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f567c143eb0 0x7f567c0cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46992/0 (socket says 172.21.15.46:46992) 2024-03-29T08:26:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.748+0000 7f567be00640 1 -- 172.21.15.46:0/3675369542 learned_addr learned my addr 172.21.15.46:0/3675369542 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:26:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.748+0000 7f5673e00640 1 --2- 172.21.15.46:0/3675369542 >> v2:172.21.15.73:3300/0 conn(0x7f567c0cbea0 0x7f567c0b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.748+0000 7f5681200640 1 --2- 172.21.15.46:0/3675369542 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f567c15efe0 0x7f567c0b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.748+0000 7f567b400640 1 -- 172.21.15.46:0/3675369542 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 597402414 0 0) 0x7f567c0b20b0 con 0x7f567c143eb0 2024-03-29T08:26:12.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.748+0000 7f567b400640 1 -- 172.21.15.46:0/3675369542 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5660003530 con 0x7f567c143eb0 2024-03-29T08:26:12.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.748+0000 7f5673e00640 1 -- 172.21.15.46:0/3675369542 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f567c15efe0 msgr2=0x7f567c0b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:12.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.748+0000 7f5673e00640 1 --2- 172.21.15.46:0/3675369542 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f567c15efe0 0x7f567c0b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.748+0000 7f5673e00640 1 -- 172.21.15.46:0/3675369542 >> v1:172.21.15.46:6789/0 conn(0x7f567c143eb0 legacy=0x7f567c0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.748+0000 7f5673e00640 1 -- 172.21.15.46:0/3675369542 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567c15e7e0 con 0x7f567c0cbea0 2024-03-29T08:26:12.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.752+0000 7f5673e00640 1 --2- 172.21.15.46:0/3675369542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f567c0cbea0 0x7f567c0b1630 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f56640097c0 tx=0x7f566401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58caf889881757 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:12.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.752+0000 7f567b400640 1 -- 172.21.15.46:0/3675369542 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56640ab610 con 0x7f567c0cbea0 2024-03-29T08:26:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.752+0000 7f567b400640 1 -- 172.21.15.46:0/3675369542 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56640ab770 con 0x7f567c0cbea0 2024-03-29T08:26:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.752+0000 7f567b400640 1 -- 172.21.15.46:0/3675369542 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56640abb00 con 0x7f567c0cbea0 2024-03-29T08:26:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.752+0000 7f5683200640 1 -- 172.21.15.46:0/3675369542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f567c0cbea0 msgr2=0x7f567c0b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.752+0000 7f5683200640 1 --2- 172.21.15.46:0/3675369542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f567c0cbea0 0x7f567c0b1630 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f56640097c0 tx=0x7f566401c470 comp rx=0 tx=0).stop 2024-03-29T08:26:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.752+0000 7f5683200640 1 -- 172.21.15.46:0/3675369542 shutdown_connections 2024-03-29T08:26:12.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.752+0000 7f5683200640 1 --2- 172.21.15.46:0/3675369542 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f567c15efe0 0x7f567c0b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.752+0000 7f5683200640 1 --2- 172.21.15.46:0/3675369542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f567c0cbea0 0x7f567c0b1630 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.752+0000 7f5683200640 1 -- 172.21.15.46:0/3675369542 >> 172.21.15.46:0/3675369542 conn(0x7f567c06a390 msgr2=0x7f567c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:12.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.752+0000 7f5683200640 1 -- 172.21.15.46:0/3675369542 shutdown_connections 2024-03-29T08:26:12.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.752+0000 7f5683200640 1 -- 172.21.15.46:0/3675369542 wait complete. 2024-03-29T08:26:12.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.752+0000 7f5683200640 1 Processor -- start 2024-03-29T08:26:12.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.752+0000 7f5683200640 1 -- start start 2024-03-29T08:26:12.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f5683200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c0cbea0 0x7f567c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f5683200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f567c143eb0 0x7f567c137b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f5683200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f567c15efe0 0x7f567c138040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f5683200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c142430 con 0x7f567c0cbea0 2024-03-29T08:26:12.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f5683200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c13efe0 con 0x7f567c143eb0 2024-03-29T08:26:12.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f5683200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567c141b00 con 0x7f567c15efe0 2024-03-29T08:26:12.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f567be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c0cbea0 0x7f567c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f567be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c0cbea0 0x7f567c06c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42420/0 (socket says 172.21.15.46:42420) 2024-03-29T08:26:12.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f567be00640 1 -- 172.21.15.46:0/4224020469 learned_addr learned my addr 172.21.15.46:0/4224020469 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:12.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f5673e00640 1 --2- 172.21.15.46:0/4224020469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f567c143eb0 0x7f567c137b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f5681200640 1 --2- 172.21.15.46:0/4224020469 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f567c15efe0 0x7f567c138040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f567be00640 1 -- 172.21.15.46:0/4224020469 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f567c15efe0 msgr2=0x7f567c138040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f567be00640 1 --2- 172.21.15.46:0/4224020469 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f567c15efe0 0x7f567c138040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f567be00640 1 -- 172.21.15.46:0/4224020469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f567c143eb0 msgr2=0x7f567c137b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f567be00640 1 --2- 172.21.15.46:0/4224020469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f567c143eb0 0x7f567c137b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f567be00640 1 -- 172.21.15.46:0/4224020469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5664009420 con 0x7f567c0cbea0 2024-03-29T08:26:12.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f567be00640 1 --2- 172.21.15.46:0/4224020469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c0cbea0 0x7f567c06c5d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f566c00c4d0 tx=0x7f566c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:12.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f5678c00640 1 -- 172.21.15.46:0/4224020469 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c005cf0 con 0x7f567c0cbea0 2024-03-29T08:26:12.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f5678c00640 1 -- 172.21.15.46:0/4224020469 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c005e50 con 0x7f567c0cbea0 2024-03-29T08:26:12.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f5678c00640 1 -- 172.21.15.46:0/4224020469 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c007270 con 0x7f567c0cbea0 2024-03-29T08:26:12.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f5683200640 1 -- 172.21.15.46:0/4224020469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f567c141de0 con 0x7f567c0cbea0 2024-03-29T08:26:12.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f5683200640 1 -- 172.21.15.46:0/4224020469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f567c13ec70 con 0x7f567c0cbea0 2024-03-29T08:26:12.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f5683200640 1 -- 172.21.15.46:0/4224020469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f567c071a70 con 0x7f567c0cbea0 2024-03-29T08:26:12.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.756+0000 7f5678c00640 1 -- 172.21.15.46:0/4224020469 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f566c0073d0 con 0x7f567c0cbea0 2024-03-29T08:26:12.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.760+0000 7f5678c00640 1 --2- 172.21.15.46:0/4224020469 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f56580666c0 0x7f5658068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:12.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.760+0000 7f5678c00640 1 -- 172.21.15.46:0/4224020469 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f566c083790 con 0x7f567c0cbea0 2024-03-29T08:26:12.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.760+0000 7f5673e00640 1 --2- 172.21.15.46:0/4224020469 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f56580666c0 0x7f5658068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:12.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.760+0000 7f5673e00640 1 --2- 172.21.15.46:0/4224020469 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f56580666c0 0x7f5658068b80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f56640af000 tx=0x7f56640a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:12.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.764+0000 7f5678c00640 1 -- 172.21.15.46:0/4224020469 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f566c050ab0 con 0x7f567c0cbea0 2024-03-29T08:26:12.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.920+0000 7f5683200640 1 -- 172.21.15.46:0/4224020469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f567c0cadd0 con 0x7f567c0cbea0 2024-03-29T08:26:12.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.920+0000 7f5678c00640 1 -- 172.21.15.46:0/4224020469 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f566c0540d0 con 0x7f567c0cbea0 2024-03-29T08:26:12.923 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:26:12.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.924+0000 7f5683200640 1 -- 172.21.15.46:0/4224020469 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f56580666c0 msgr2=0x7f5658068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.924+0000 7f5683200640 1 --2- 172.21.15.46:0/4224020469 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f56580666c0 0x7f5658068b80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f56640af000 tx=0x7f56640a7000 comp rx=0 tx=0).stop 2024-03-29T08:26:12.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.924+0000 7f5683200640 1 -- 172.21.15.46:0/4224020469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c0cbea0 msgr2=0x7f567c06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:12.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.924+0000 7f5683200640 1 --2- 172.21.15.46:0/4224020469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c0cbea0 0x7f567c06c5d0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f566c00c4d0 tx=0x7f566c00c990 comp rx=0 tx=0).stop 2024-03-29T08:26:12.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.924+0000 7f5683200640 1 -- 172.21.15.46:0/4224020469 shutdown_connections 2024-03-29T08:26:12.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.924+0000 7f5683200640 1 --2- 172.21.15.46:0/4224020469 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f56580666c0 0x7f5658068b80 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.924+0000 7f5683200640 1 --2- 172.21.15.46:0/4224020469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f567c143eb0 0x7f567c137b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.924+0000 7f5683200640 1 --2- 172.21.15.46:0/4224020469 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f567c15efe0 0x7f567c138040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.924+0000 7f5683200640 1 --2- 172.21.15.46:0/4224020469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f567c0cbea0 0x7f567c06c5d0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:12.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.924+0000 7f5683200640 1 -- 172.21.15.46:0/4224020469 >> 172.21.15.46:0/4224020469 conn(0x7f567c06a390 msgr2=0x7f567c0cd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:12.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.924+0000 7f5683200640 1 -- 172.21.15.46:0/4224020469 shutdown_connections 2024-03-29T08:26:12.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:12.924+0000 7f5683200640 1 -- 172.21.15.46:0/4224020469 wait complete. 2024-03-29T08:26:12.944 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:26:13.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.088+0000 7fb35ac00640 1 Processor -- start 2024-03-29T08:26:13.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.088+0000 7fb35ac00640 1 -- start start 2024-03-29T08:26:13.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.088+0000 7fb35ac00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb354143eb0 0x7fb3540cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.088+0000 7fb35ac00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb3540b1d40 0x7fb3540cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.088+0000 7fb35ac00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3540b2360 con 0x7fb35415efe0 2024-03-29T08:26:13.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.088+0000 7fb35ac00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb35415e500 con 0x7fb3540b1d40 2024-03-29T08:26:13.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.088+0000 7fb35ac00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35415e7e0 con 0x7fb354143eb0 2024-03-29T08:26:13.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.092+0000 7fb358c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb3540b1d40 0x7fb3540cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.092+0000 7fb358c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb3540b1d40 0x7fb3540cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42422/0 (socket says 172.21.15.46:42422) 2024-03-29T08:26:13.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.092+0000 7fb358c00640 1 -- 172.21.15.46:0/562884636 learned_addr learned my addr 172.21.15.46:0/562884636 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:13.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.092+0000 7fb358c00640 1 -- 172.21.15.46:0/562884636 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb354143eb0 msgr2=0x7fb3540cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.092+0000 7fb353e00640 1 --2- 172.21.15.46:0/562884636 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb354143eb0 0x7fb3540cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.092+0000 7fb358c00640 1 --2- 172.21.15.46:0/562884636 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb354143eb0 0x7fb3540cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.092+0000 7fb358c00640 1 -- 172.21.15.46:0/562884636 >> v1:172.21.15.73:6789/0 conn(0x7fb35415efe0 legacy=0x7fb3540b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.092+0000 7fb358c00640 1 -- 172.21.15.46:0/562884636 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35415eac0 con 0x7fb3540b1d40 2024-03-29T08:26:13.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.092+0000 7fb353e00640 1 --2- 172.21.15.46:0/562884636 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb354143eb0 0x7fb3540cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:26:13.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.092+0000 7fb358c00640 1 --2- 172.21.15.46:0/562884636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3540b1d40 0x7fb3540cbcd0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009870 tx=0x7fb34c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db024c927bdc8afc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.092+0000 7fb353400640 1 -- 172.21.15.46:0/562884636 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c0ab600 con 0x7fb3540b1d40 2024-03-29T08:26:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.092+0000 7fb353400640 1 -- 172.21.15.46:0/562884636 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34c0a7050 con 0x7fb3540b1d40 2024-03-29T08:26:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.092+0000 7fb353400640 1 -- 172.21.15.46:0/562884636 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c0ab980 con 0x7fb3540b1d40 2024-03-29T08:26:13.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb35ac00640 1 -- 172.21.15.46:0/562884636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3540b1d40 msgr2=0x7fb3540cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb35ac00640 1 --2- 172.21.15.46:0/562884636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3540b1d40 0x7fb3540cbcd0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009870 tx=0x7fb34c01c500 comp rx=0 tx=0).stop 2024-03-29T08:26:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb35ac00640 1 -- 172.21.15.46:0/562884636 shutdown_connections 2024-03-29T08:26:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb35ac00640 1 --2- 172.21.15.46:0/562884636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3540b1d40 0x7fb3540cbcd0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb35ac00640 1 --2- 172.21.15.46:0/562884636 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb354143eb0 0x7fb3540cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb35ac00640 1 -- 172.21.15.46:0/562884636 >> 172.21.15.46:0/562884636 conn(0x7fb35406a390 msgr2=0x7fb3540cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb35ac00640 1 -- 172.21.15.46:0/562884636 shutdown_connections 2024-03-29T08:26:13.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb35ac00640 1 -- 172.21.15.46:0/562884636 wait complete. 2024-03-29T08:26:13.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb35ac00640 1 Processor -- start 2024-03-29T08:26:13.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb35ac00640 1 -- start start 2024-03-29T08:26:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb35ac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3540b1d40 0x7fb35415bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb35ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb354143eb0 0x7fb35413e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb35ac00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb35415efe0 0x7fb354140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb35ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35415c7a0 con 0x7fb354143eb0 2024-03-29T08:26:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb35ac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35415ca80 con 0x7fb3540b1d40 2024-03-29T08:26:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb35ac00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35415cd60 con 0x7fb35415efe0 2024-03-29T08:26:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb34b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb354143eb0 0x7fb35413e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb34b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb354143eb0 0x7fb35413e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42428/0 (socket says 172.21.15.46:42428) 2024-03-29T08:26:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb34b400640 1 -- 172.21.15.46:0/2549837863 learned_addr learned my addr 172.21.15.46:0/2549837863 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb353e00640 1 --2- 172.21.15.46:0/2549837863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3540b1d40 0x7fb35415bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb358c00640 1 --2- 172.21.15.46:0/2549837863 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb35415efe0 0x7fb354140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb34b400640 1 -- 172.21.15.46:0/2549837863 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb35415efe0 msgr2=0x7fb354140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb34b400640 1 --2- 172.21.15.46:0/2549837863 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb35415efe0 0x7fb354140270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb34b400640 1 -- 172.21.15.46:0/2549837863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3540b1d40 msgr2=0x7fb35415bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb34b400640 1 --2- 172.21.15.46:0/2549837863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3540b1d40 0x7fb35415bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb34b400640 1 -- 172.21.15.46:0/2549837863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c0094d0 con 0x7fb354143eb0 2024-03-29T08:26:13.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb34b400640 1 --2- 172.21.15.46:0/2549837863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb354143eb0 0x7fb35413e780 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00b410 tx=0x7fb33c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb350c00640 1 -- 172.21.15.46:0/2549837863 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c004020 con 0x7fb354143eb0 2024-03-29T08:26:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb350c00640 1 -- 172.21.15.46:0/2549837863 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c004180 con 0x7fb354143eb0 2024-03-29T08:26:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb350c00640 1 -- 172.21.15.46:0/2549837863 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c011630 con 0x7fb354143eb0 2024-03-29T08:26:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.096+0000 7fb35ac00640 1 -- 172.21.15.46:0/2549837863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35415d040 con 0x7fb354143eb0 2024-03-29T08:26:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.100+0000 7fb35ac00640 1 -- 172.21.15.46:0/2549837863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35415c450 con 0x7fb354143eb0 2024-03-29T08:26:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.100+0000 7fb35ac00640 1 -- 172.21.15.46:0/2549837863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb354071a70 con 0x7fb354143eb0 2024-03-29T08:26:13.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.100+0000 7fb350c00640 1 -- 172.21.15.46:0/2549837863 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb33c011790 con 0x7fb354143eb0 2024-03-29T08:26:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.100+0000 7fb350c00640 1 --2- 172.21.15.46:0/2549837863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb330066690 0x7fb330068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.100+0000 7fb350c00640 1 -- 172.21.15.46:0/2549837863 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb33c083ec0 con 0x7fb354143eb0 2024-03-29T08:26:13.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.100+0000 7fb353e00640 1 --2- 172.21.15.46:0/2549837863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb330066690 0x7fb330068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.100+0000 7fb353e00640 1 --2- 172.21.15.46:0/2549837863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb330066690 0x7fb330068b50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb35406d300 tx=0x7fb344007810 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:13.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.104+0000 7fb350c00640 1 -- 172.21.15.46:0/2549837863 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb33c0511e0 con 0x7fb354143eb0 2024-03-29T08:26:13.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.264+0000 7fb35ac00640 1 -- 172.21.15.46:0/2549837863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb35415d4f0 con 0x7fb354143eb0 2024-03-29T08:26:13.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.264+0000 7fb350c00640 1 -- 172.21.15.46:0/2549837863 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb33c015020 con 0x7fb354143eb0 2024-03-29T08:26:13.266 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:26:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.268+0000 7fb35ac00640 1 -- 172.21.15.46:0/2549837863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb330066690 msgr2=0x7fb330068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.268+0000 7fb35ac00640 1 --2- 172.21.15.46:0/2549837863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb330066690 0x7fb330068b50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fb35406d300 tx=0x7fb344007810 comp rx=0 tx=0).stop 2024-03-29T08:26:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.268+0000 7fb35ac00640 1 -- 172.21.15.46:0/2549837863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb354143eb0 msgr2=0x7fb35413e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.268+0000 7fb35ac00640 1 --2- 172.21.15.46:0/2549837863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb354143eb0 0x7fb35413e780 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00b410 tx=0x7fb33c00b8d0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.268+0000 7fb35ac00640 1 -- 172.21.15.46:0/2549837863 shutdown_connections 2024-03-29T08:26:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.268+0000 7fb35ac00640 1 --2- 172.21.15.46:0/2549837863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb330066690 0x7fb330068b50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.268+0000 7fb35ac00640 1 --2- 172.21.15.46:0/2549837863 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb35415efe0 0x7fb354140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.268+0000 7fb35ac00640 1 --2- 172.21.15.46:0/2549837863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb354143eb0 0x7fb35413e780 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.268+0000 7fb35ac00640 1 --2- 172.21.15.46:0/2549837863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3540b1d40 0x7fb35415bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.268+0000 7fb35ac00640 1 -- 172.21.15.46:0/2549837863 >> 172.21.15.46:0/2549837863 conn(0x7fb35406a390 msgr2=0x7fb35415a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.268+0000 7fb35ac00640 1 -- 172.21.15.46:0/2549837863 shutdown_connections 2024-03-29T08:26:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.268+0000 7fb35ac00640 1 -- 172.21.15.46:0/2549837863 wait complete. 2024-03-29T08:26:13.287 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:26:13.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.428+0000 7f9ea7a00640 1 Processor -- start 2024-03-29T08:26:13.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.428+0000 7f9ea7a00640 1 -- start start 2024-03-29T08:26:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.428+0000 7f9ea7a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ea0143eb0 0x7f9ea00cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.428+0000 7f9ea7a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ea015efe0 0x7f9ea00cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.428+0000 7f9ea7a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea00b2360 con 0x7f9ea00b1630 2024-03-29T08:26:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.428+0000 7f9ea7a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea015e500 con 0x7f9ea0143eb0 2024-03-29T08:26:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.428+0000 7f9ea7a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea015e7e0 con 0x7f9ea015efe0 2024-03-29T08:26:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.428+0000 7f9ea5000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ea0143eb0 0x7f9ea00cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.428+0000 7f9ea5000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ea0143eb0 0x7f9ea00cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42436/0 (socket says 172.21.15.46:42436) 2024-03-29T08:26:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.428+0000 7f9ea5000640 1 -- 172.21.15.46:0/2628732063 learned_addr learned my addr 172.21.15.46:0/2628732063 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:13.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.428+0000 7f9e97e00640 1 --2- 172.21.15.46:0/2628732063 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ea015efe0 0x7f9ea00cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.428+0000 7f9ea5000640 1 -- 172.21.15.46:0/2628732063 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ea015efe0 msgr2=0x7f9ea00cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.428+0000 7f9ea5000640 1 --2- 172.21.15.46:0/2628732063 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ea015efe0 0x7f9ea00cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.428+0000 7f9ea5000640 1 -- 172.21.15.46:0/2628732063 >> v1:172.21.15.73:6789/0 conn(0x7f9ea00b1630 legacy=0x7f9ea00b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.428+0000 7f9ea5000640 1 -- 172.21.15.46:0/2628732063 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea015eac0 con 0x7f9ea0143eb0 2024-03-29T08:26:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.428+0000 7f9e97e00640 1 --2- 172.21.15.46:0/2628732063 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ea015efe0 0x7f9ea00cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.432+0000 7f9ea5000640 1 --2- 172.21.15.46:0/2628732063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea0143eb0 0x7f9ea00cb790 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009870 tx=0x7f9e9001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759c9d8023dfc7e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.432+0000 7f9e9fe00640 1 -- 172.21.15.46:0/2628732063 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e900ad660 con 0x7f9ea0143eb0 2024-03-29T08:26:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.432+0000 7f9e9fe00640 1 -- 172.21.15.46:0/2628732063 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e900ad7c0 con 0x7f9ea0143eb0 2024-03-29T08:26:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.432+0000 7f9e9fe00640 1 -- 172.21.15.46:0/2628732063 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e900adb50 con 0x7f9ea0143eb0 2024-03-29T08:26:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.432+0000 7f9ea7a00640 1 -- 172.21.15.46:0/2628732063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea0143eb0 msgr2=0x7f9ea00cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.432+0000 7f9ea7a00640 1 --2- 172.21.15.46:0/2628732063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea0143eb0 0x7f9ea00cb790 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009870 tx=0x7f9e9001c560 comp rx=0 tx=0).stop 2024-03-29T08:26:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.432+0000 7f9ea7a00640 1 -- 172.21.15.46:0/2628732063 shutdown_connections 2024-03-29T08:26:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.432+0000 7f9ea7a00640 1 --2- 172.21.15.46:0/2628732063 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ea015efe0 0x7f9ea00cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.432+0000 7f9ea7a00640 1 --2- 172.21.15.46:0/2628732063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea0143eb0 0x7f9ea00cb790 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.432+0000 7f9ea7a00640 1 -- 172.21.15.46:0/2628732063 >> 172.21.15.46:0/2628732063 conn(0x7f9ea006a390 msgr2=0x7f9ea00cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:13.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.432+0000 7f9ea7a00640 1 -- 172.21.15.46:0/2628732063 shutdown_connections 2024-03-29T08:26:13.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.432+0000 7f9ea7a00640 1 -- 172.21.15.46:0/2628732063 wait complete. 2024-03-29T08:26:13.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea7a00640 1 Processor -- start 2024-03-29T08:26:13.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea7a00640 1 -- start start 2024-03-29T08:26:13.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea7a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea00b1630 0x7f9ea013d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea7a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ea0143eb0 0x7f9ea0141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea7a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ea015efe0 0x7f9ea01423c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea7a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea0143610 con 0x7f9ea00b1630 2024-03-29T08:26:13.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea7a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea0142a00 con 0x7f9ea015efe0 2024-03-29T08:26:13.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea7a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea0142ce0 con 0x7f9ea0143eb0 2024-03-29T08:26:13.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea00b1630 0x7f9ea013d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea00b1630 0x7f9ea013d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42446/0 (socket says 172.21.15.46:42446) 2024-03-29T08:26:13.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea5000640 1 -- 172.21.15.46:0/248830746 learned_addr learned my addr 172.21.15.46:0/248830746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:13.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea5000640 1 -- 172.21.15.46:0/248830746 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ea0143eb0 msgr2=0x7f9ea0141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea5000640 1 --2- 172.21.15.46:0/248830746 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ea0143eb0 0x7f9ea0141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea5000640 1 -- 172.21.15.46:0/248830746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ea015efe0 msgr2=0x7f9ea01423c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea5000640 1 --2- 172.21.15.46:0/248830746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ea015efe0 0x7f9ea01423c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea5000640 1 -- 172.21.15.46:0/248830746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e900094d0 con 0x7f9ea00b1630 2024-03-29T08:26:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea5000640 1 --2- 172.21.15.46:0/248830746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea00b1630 0x7f9ea013d550 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f9e900b0040 tx=0x7f9e90019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9e9d600640 1 -- 172.21.15.46:0/248830746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e900bb070 con 0x7f9ea00b1630 2024-03-29T08:26:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9e9d600640 1 -- 172.21.15.46:0/248830746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e900b6650 con 0x7f9ea00b1630 2024-03-29T08:26:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9e9d600640 1 -- 172.21.15.46:0/248830746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e900b6a00 con 0x7f9ea00b1630 2024-03-29T08:26:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea7a00640 1 -- 172.21.15.46:0/248830746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea0142fc0 con 0x7f9ea00b1630 2024-03-29T08:26:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.436+0000 7f9ea7a00640 1 -- 172.21.15.46:0/248830746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea015a500 con 0x7f9ea00b1630 2024-03-29T08:26:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.440+0000 7f9ea7a00640 1 -- 172.21.15.46:0/248830746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea0071a70 con 0x7f9ea00b1630 2024-03-29T08:26:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.440+0000 7f9e9d600640 1 -- 172.21.15.46:0/248830746 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9e900b4070 con 0x7f9ea00b1630 2024-03-29T08:26:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.440+0000 7f9e9d600640 1 --2- 172.21.15.46:0/248830746 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9e7c066690 0x7f9e7c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.440+0000 7f9e9d600640 1 -- 172.21.15.46:0/248830746 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9e9011dff0 con 0x7f9ea00b1630 2024-03-29T08:26:13.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.440+0000 7f9e97e00640 1 --2- 172.21.15.46:0/248830746 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9e7c066690 0x7f9e7c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.440+0000 7f9e97e00640 1 --2- 172.21.15.46:0/248830746 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9e7c066690 0x7f9e7c068b50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9ea01438d0 tx=0x7f9e88008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:13.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.444+0000 7f9e9d600640 1 -- 172.21.15.46:0/248830746 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9e900eb2c0 con 0x7f9ea00b1630 2024-03-29T08:26:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.608+0000 7f9ea7a00640 1 -- 172.21.15.46:0/248830746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ea00cadd0 con 0x7f9ea00b1630 2024-03-29T08:26:13.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.608+0000 7f9e9d600640 1 -- 172.21.15.46:0/248830746 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9e900ee8e0 con 0x7f9ea00b1630 2024-03-29T08:26:13.611 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:26:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.612+0000 7f9ea7a00640 1 -- 172.21.15.46:0/248830746 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9e7c066690 msgr2=0x7f9e7c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.612+0000 7f9ea7a00640 1 --2- 172.21.15.46:0/248830746 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9e7c066690 0x7f9e7c068b50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9ea01438d0 tx=0x7f9e88008040 comp rx=0 tx=0).stop 2024-03-29T08:26:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.612+0000 7f9ea7a00640 1 -- 172.21.15.46:0/248830746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea00b1630 msgr2=0x7f9ea013d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.612+0000 7f9ea7a00640 1 --2- 172.21.15.46:0/248830746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea00b1630 0x7f9ea013d550 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f9e900b0040 tx=0x7f9e90019300 comp rx=0 tx=0).stop 2024-03-29T08:26:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.612+0000 7f9ea7a00640 1 -- 172.21.15.46:0/248830746 shutdown_connections 2024-03-29T08:26:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.612+0000 7f9ea7a00640 1 --2- 172.21.15.46:0/248830746 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9e7c066690 0x7f9e7c068b50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.612+0000 7f9ea7a00640 1 --2- 172.21.15.46:0/248830746 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ea015efe0 0x7f9ea01423c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.612+0000 7f9ea7a00640 1 --2- 172.21.15.46:0/248830746 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ea0143eb0 0x7f9ea0141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.612+0000 7f9ea7a00640 1 --2- 172.21.15.46:0/248830746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ea00b1630 0x7f9ea013d550 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.612+0000 7f9ea7a00640 1 -- 172.21.15.46:0/248830746 >> 172.21.15.46:0/248830746 conn(0x7f9ea006a390 msgr2=0x7f9ea015bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:13.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.612+0000 7f9ea7a00640 1 -- 172.21.15.46:0/248830746 shutdown_connections 2024-03-29T08:26:13.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.612+0000 7f9ea7a00640 1 -- 172.21.15.46:0/248830746 wait complete. 2024-03-29T08:26:13.633 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:26:13.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.772+0000 7fb8e6600640 1 Processor -- start 2024-03-29T08:26:13.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.772+0000 7fb8e6600640 1 -- start start 2024-03-29T08:26:13.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.772+0000 7fb8e6600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb8e00af530 0x7fb8e00af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.772+0000 7fb8e6600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e006c390 con 0x7fb8e00702a0 2024-03-29T08:26:13.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.772+0000 7fb8e6600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e00afff0 con 0x7fb8e0158300 2024-03-29T08:26:13.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.772+0000 7fb8e6600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e006c5e0 con 0x7fb8e00af530 2024-03-29T08:26:13.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dfe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb8e00702a0 0x7fb8e015bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47006/0 (socket says 172.21.15.46:47006) 2024-03-29T08:26:13.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dfe00640 1 -- 172.21.15.46:0/912264287 learned_addr learned my addr 172.21.15.46:0/912264287 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:26:13.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8d6a00640 1 --2- 172.21.15.46:0/912264287 >> v2:172.21.15.73:3300/0 conn(0x7fb8e00af530 0x7fb8e00af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dea00640 1 -- 172.21.15.46:0/912264287 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318744822 0 0) 0x7fb8e006c390 con 0x7fb8e00702a0 2024-03-29T08:26:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dea00640 1 -- 172.21.15.46:0/912264287 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8c4003530 con 0x7fb8e00702a0 2024-03-29T08:26:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dea00640 1 -- 172.21.15.46:0/912264287 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1067202017 0 0) 0x7fb8e00afff0 con 0x7fb8e0158300 2024-03-29T08:26:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dea00640 1 -- 172.21.15.46:0/912264287 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8e006c390 con 0x7fb8e0158300 2024-03-29T08:26:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dea00640 1 -- 172.21.15.46:0/912264287 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3951461118 0 0) 0x7fb8c4003530 con 0x7fb8e00702a0 2024-03-29T08:26:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dea00640 1 -- 172.21.15.46:0/912264287 >> v2:172.21.15.73:3300/0 conn(0x7fb8e00af530 msgr2=0x7fb8e00af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dea00640 1 --2- 172.21.15.46:0/912264287 >> v2:172.21.15.73:3300/0 conn(0x7fb8e00af530 0x7fb8e00af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dea00640 1 -- 172.21.15.46:0/912264287 >> v1:172.21.15.73:6789/0 conn(0x7fb8e0158300 legacy=0x7fb8e006d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dea00640 1 -- 172.21.15.46:0/912264287 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e006c8c0 con 0x7fb8e00702a0 2024-03-29T08:26:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dea00640 1 -- 172.21.15.46:0/912264287 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fb8d8001a00 con 0x7fb8e00702a0 2024-03-29T08:26:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8d6a00640 1 --2- 172.21.15.46:0/912264287 >> v2:172.21.15.73:3300/0 conn(0x7fb8e00af530 0x7fb8e00af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:26:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dea00640 1 -- 172.21.15.46:0/912264287 >> v1:172.21.15.46:6789/0 conn(0x7fb8e00702a0 legacy=0x7fb8e015bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dea00640 1 --2- 172.21.15.46:0/912264287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4003b10 0x7fb8c4003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dea00640 1 --2- 172.21.15.46:0/912264287 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb8c40047d0 0x7fb8c4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dea00640 1 --2- 172.21.15.46:0/912264287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8c4005200 0x7fb8c400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dea00640 1 -- 172.21.15.46:0/912264287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e006c5e0 con 0x7fb8c4003b10 2024-03-29T08:26:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dea00640 1 -- 172.21.15.46:0/912264287 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c400c160 con 0x7fb8c4005200 2024-03-29T08:26:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dea00640 1 -- 172.21.15.46:0/912264287 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c400c320 con 0x7fb8c40047d0 2024-03-29T08:26:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8df400640 1 --2- 172.21.15.46:0/912264287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4003b10 0x7fb8c4003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8d6a00640 1 --2- 172.21.15.46:0/912264287 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb8c40047d0 0x7fb8c4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dfe00640 1 --2- 172.21.15.46:0/912264287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8c4005200 0x7fb8c400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8df400640 1 -- 172.21.15.46:0/912264287 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb8c40047d0 msgr2=0x7fb8c4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8df400640 1 --2- 172.21.15.46:0/912264287 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb8c40047d0 0x7fb8c4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8df400640 1 -- 172.21.15.46:0/912264287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8c4005200 msgr2=0x7fb8c400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8df400640 1 --2- 172.21.15.46:0/912264287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8c4005200 0x7fb8c400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8df400640 1 -- 172.21.15.46:0/912264287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c4003530 con 0x7fb8c4003b10 2024-03-29T08:26:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8dfe00640 1 -- 172.21.15.46:0/912264287 reap_dead start 2024-03-29T08:26:13.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.776+0000 7fb8d6a00640 1 --2- 172.21.15.46:0/912264287 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb8c40047d0 0x7fb8c4004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:13.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8df400640 1 --2- 172.21.15.46:0/912264287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4003b10 0x7fb8c4003f10 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00018d0 tx=0x7fb8d001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3eaadb7d8bec90d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:13.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8dea00640 1 -- 172.21.15.46:0/912264287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00052d0 con 0x7fb8c4003b10 2024-03-29T08:26:13.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8dea00640 1 -- 172.21.15.46:0/912264287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d0005450 con 0x7fb8c4003b10 2024-03-29T08:26:13.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8dea00640 1 -- 172.21.15.46:0/912264287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0003840 con 0x7fb8c4003b10 2024-03-29T08:26:13.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8e6600640 1 -- 172.21.15.46:0/912264287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4003b10 msgr2=0x7fb8c4003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8e6600640 1 --2- 172.21.15.46:0/912264287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4003b10 0x7fb8c4003f10 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00018d0 tx=0x7fb8d001c530 comp rx=0 tx=0).stop 2024-03-29T08:26:13.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8e6600640 1 -- 172.21.15.46:0/912264287 shutdown_connections 2024-03-29T08:26:13.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8e6600640 1 --2- 172.21.15.46:0/912264287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4003b10 0x7fb8c4003f10 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8e6600640 1 -- 172.21.15.46:0/912264287 >> 172.21.15.46:0/912264287 conn(0x7fb8e0066080 msgr2=0x7fb8e0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:13.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8e6600640 1 -- 172.21.15.46:0/912264287 shutdown_connections 2024-03-29T08:26:13.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8e6600640 1 -- 172.21.15.46:0/912264287 wait complete. 2024-03-29T08:26:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8e6600640 1 Processor -- start 2024-03-29T08:26:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8e6600640 1 -- start start 2024-03-29T08:26:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8e6600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4003b10 0x7fb8e01f9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8e6600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8e00cd060 0x7fb8e014c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8e6600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb8e0151340 0x7fb8e015c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8e6600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e014c9c0 con 0x7fb8c4003b10 2024-03-29T08:26:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8e6600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e01fa1f0 con 0x7fb8e00cd060 2024-03-29T08:26:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.780+0000 7fb8e6600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e00adaa0 con 0x7fb8e0151340 2024-03-29T08:26:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8df400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4003b10 0x7fb8e01f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8df400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4003b10 0x7fb8e01f9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42456/0 (socket says 172.21.15.46:42456) 2024-03-29T08:26:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8df400640 1 -- 172.21.15.46:0/4138593458 learned_addr learned my addr 172.21.15.46:0/4138593458 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8d6a00640 1 --2- 172.21.15.46:0/4138593458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8e00cd060 0x7fb8e014c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8dfe00640 1 --2- 172.21.15.46:0/4138593458 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb8e0151340 0x7fb8e015c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8df400640 1 -- 172.21.15.46:0/4138593458 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb8e0151340 msgr2=0x7fb8e015c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8df400640 1 --2- 172.21.15.46:0/4138593458 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb8e0151340 0x7fb8e015c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8df400640 1 -- 172.21.15.46:0/4138593458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8e00cd060 msgr2=0x7fb8e014c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8df400640 1 --2- 172.21.15.46:0/4138593458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8e00cd060 0x7fb8e014c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8df400640 1 -- 172.21.15.46:0/4138593458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e01f9950 con 0x7fb8c4003b10 2024-03-29T08:26:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8df400640 1 --2- 172.21.15.46:0/4138593458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4003b10 0x7fb8e01f9270 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00ae040 tx=0x7fb8c4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8d7e00640 1 -- 172.21.15.46:0/4138593458 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d00052d0 con 0x7fb8c4003b10 2024-03-29T08:26:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8d7e00640 1 -- 172.21.15.46:0/4138593458 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d0003c80 con 0x7fb8c4003b10 2024-03-29T08:26:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8e6600640 1 -- 172.21.15.46:0/4138593458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e01f9c30 con 0x7fb8c4003b10 2024-03-29T08:26:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8d7e00640 1 -- 172.21.15.46:0/4138593458 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0007020 con 0x7fb8c4003b10 2024-03-29T08:26:13.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8e6600640 1 -- 172.21.15.46:0/4138593458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e00abda0 con 0x7fb8c4003b10 2024-03-29T08:26:13.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8e6600640 1 -- 172.21.15.46:0/4138593458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e0071a90 con 0x7fb8c4003b10 2024-03-29T08:26:13.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8d7e00640 1 -- 172.21.15.46:0/4138593458 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb8d00b2040 con 0x7fb8c4003b10 2024-03-29T08:26:13.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.784+0000 7fb8d7e00640 1 --2- 172.21.15.46:0/4138593458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb8bc066690 0x7fb8bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:13.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.788+0000 7fb8d7e00640 1 -- 172.21.15.46:0/4138593458 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb8d00a8050 con 0x7fb8c4003b10 2024-03-29T08:26:13.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.788+0000 7fb8d6a00640 1 --2- 172.21.15.46:0/4138593458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb8bc066690 0x7fb8bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:13.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.788+0000 7fb8d6a00640 1 --2- 172.21.15.46:0/4138593458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb8bc066690 0x7fb8bc068b50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb8e00b0860 tx=0x7fb8c8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:13.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.792+0000 7fb8d7e00640 1 -- 172.21.15.46:0/4138593458 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb8d00eb560 con 0x7fb8c4003b10 2024-03-29T08:26:13.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.964+0000 7fb8e6600640 1 -- 172.21.15.46:0/4138593458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb8e004e120 con 0x7fb8c4003b10 2024-03-29T08:26:13.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.964+0000 7fb8d7e00640 1 -- 172.21.15.46:0/4138593458 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v93) v1 ==== 80+0+122 (secure 0 0 0) 0x7fb8d00eeb80 con 0x7fb8c4003b10 2024-03-29T08:26:13.966 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.73:6836/4166845858 2024-03-30T08:25:02.334350+0000 2024-03-29T08:26:13.966 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.73:6834/4166845858 2024-03-30T08:25:02.334350+0000 2024-03-29T08:26:13.966 INFO:teuthology.orchestra.run.smithi046.stderr:listed 2 entries 2024-03-29T08:26:13.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.968+0000 7fb8e6600640 1 -- 172.21.15.46:0/4138593458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb8bc066690 msgr2=0x7fb8bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.968+0000 7fb8e6600640 1 --2- 172.21.15.46:0/4138593458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb8bc066690 0x7fb8bc068b50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb8e00b0860 tx=0x7fb8c8008040 comp rx=0 tx=0).stop 2024-03-29T08:26:13.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.968+0000 7fb8e6600640 1 -- 172.21.15.46:0/4138593458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4003b10 msgr2=0x7fb8e01f9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:13.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.968+0000 7fb8e6600640 1 --2- 172.21.15.46:0/4138593458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4003b10 0x7fb8e01f9270 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00ae040 tx=0x7fb8c4005800 comp rx=0 tx=0).stop 2024-03-29T08:26:13.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.968+0000 7fb8e6600640 1 -- 172.21.15.46:0/4138593458 shutdown_connections 2024-03-29T08:26:13.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.968+0000 7fb8e6600640 1 --2- 172.21.15.46:0/4138593458 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb8bc066690 0x7fb8bc068b50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.968+0000 7fb8e6600640 1 --2- 172.21.15.46:0/4138593458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8e00cd060 0x7fb8e014c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.968+0000 7fb8e6600640 1 --2- 172.21.15.46:0/4138593458 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb8e0151340 0x7fb8e015c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.968+0000 7fb8e6600640 1 --2- 172.21.15.46:0/4138593458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8c4003b10 0x7fb8e01f9270 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:13.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.968+0000 7fb8e6600640 1 -- 172.21.15.46:0/4138593458 >> 172.21.15.46:0/4138593458 conn(0x7fb8e0066080 msgr2=0x7fb8e00ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.968+0000 7fb8e6600640 1 -- 172.21.15.46:0/4138593458 shutdown_connections 2024-03-29T08:26:13.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:13.968+0000 7fb8e6600640 1 -- 172.21.15.46:0/4138593458 wait complete. 2024-03-29T08:26:13.986 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:26:13.986 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:26:13.995 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:26:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.176+0000 7f38ea200640 1 Processor -- start 2024-03-29T08:26:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.176+0000 7f38ea200640 1 -- start start 2024-03-29T08:26:14.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.180+0000 7f38ea200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f38e406d910 0x7f38e40701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.180+0000 7f38ea200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38e406dda0 0x7f38e4070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.180+0000 7f38ea200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e415d7c0 con 0x7f38e41569a0 2024-03-29T08:26:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.180+0000 7f38ea200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e40b21d0 con 0x7f38e406dda0 2024-03-29T08:26:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.180+0000 7f38ea200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e415cc40 con 0x7f38e406d910 2024-03-29T08:26:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.180+0000 7f38e3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f38e41569a0 0x7f38e406c1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47010/0 (socket says 172.21.15.46:47010) 2024-03-29T08:26:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.180+0000 7f38e3400640 1 -- 172.21.15.46:0/2357177331 learned_addr learned my addr 172.21.15.46:0/2357177331 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:26:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.180+0000 7f38e3e00640 1 --2- 172.21.15.46:0/2357177331 >> v2:172.21.15.46:3300/0 conn(0x7f38e406dda0 0x7f38e4070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.180+0000 7f38daa00640 1 --2- 172.21.15.46:0/2357177331 >> v2:172.21.15.73:3300/0 conn(0x7f38e406d910 0x7f38e40701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.180+0000 7f38e2a00640 1 -- 172.21.15.46:0/2357177331 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7518834 0 0) 0x7f38e415d7c0 con 0x7f38e41569a0 2024-03-29T08:26:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.180+0000 7f38e3e00640 1 -- 172.21.15.46:0/2357177331 >> v2:172.21.15.73:3300/0 conn(0x7f38e406d910 msgr2=0x7f38e40701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.180+0000 7f38e3e00640 1 --2- 172.21.15.46:0/2357177331 >> v2:172.21.15.73:3300/0 conn(0x7f38e406d910 0x7f38e40701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.180+0000 7f38e3e00640 1 -- 172.21.15.46:0/2357177331 >> v1:172.21.15.46:6789/0 conn(0x7f38e41569a0 legacy=0x7f38e406c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.180+0000 7f38e3e00640 1 -- 172.21.15.46:0/2357177331 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e415cf20 con 0x7f38e406dda0 2024-03-29T08:26:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38e3e00640 1 --2- 172.21.15.46:0/2357177331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e406dda0 0x7f38e4070700 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009d90 tx=0x7f38dc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32696ad284259e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38e2a00640 1 -- 172.21.15.46:0/2357177331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0a57f0 con 0x7f38e406dda0 2024-03-29T08:26:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38e2a00640 1 -- 172.21.15.46:0/2357177331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc0a5950 con 0x7f38e406dda0 2024-03-29T08:26:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38e2a00640 1 -- 172.21.15.46:0/2357177331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0a5ce0 con 0x7f38e406dda0 2024-03-29T08:26:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38ea200640 1 -- 172.21.15.46:0/2357177331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e406dda0 msgr2=0x7f38e4070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38ea200640 1 --2- 172.21.15.46:0/2357177331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e406dda0 0x7f38e4070700 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009d90 tx=0x7f38dc01c930 comp rx=0 tx=0).stop 2024-03-29T08:26:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38ea200640 1 -- 172.21.15.46:0/2357177331 shutdown_connections 2024-03-29T08:26:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38ea200640 1 --2- 172.21.15.46:0/2357177331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e406dda0 0x7f38e4070700 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38ea200640 1 --2- 172.21.15.46:0/2357177331 >> v2:172.21.15.73:3300/0 conn(0x7f38e406d910 0x7f38e40701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38ea200640 1 -- 172.21.15.46:0/2357177331 >> 172.21.15.46:0/2357177331 conn(0x7f38e4066110 msgr2=0x7f38e4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38ea200640 1 -- 172.21.15.46:0/2357177331 shutdown_connections 2024-03-29T08:26:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38ea200640 1 -- 172.21.15.46:0/2357177331 wait complete. 2024-03-29T08:26:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38ea200640 1 Processor -- start 2024-03-29T08:26:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38ea200640 1 -- start start 2024-03-29T08:26:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38ea200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38e406d910 0x7f38e40cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38ea200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38e406dda0 0x7f38e406c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38ea200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e41569a0 0x7f38e41f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38ea200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e40ccf10 con 0x7f38e41569a0 2024-03-29T08:26:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38ea200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e40cc300 con 0x7f38e406d910 2024-03-29T08:26:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38ea200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e40cc5e0 con 0x7f38e406dda0 2024-03-29T08:26:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e41569a0 0x7f38e41f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e41569a0 0x7f38e41f7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42486/0 (socket says 172.21.15.46:42486) 2024-03-29T08:26:14.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.184+0000 7f38e3e00640 1 -- 172.21.15.46:0/1297325007 learned_addr learned my addr 172.21.15.46:0/1297325007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.188+0000 7f38e3400640 1 --2- 172.21.15.46:0/1297325007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38e406d910 0x7f38e40cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.188+0000 7f38daa00640 1 --2- 172.21.15.46:0/1297325007 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38e406dda0 0x7f38e406c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.188+0000 7f38e3e00640 1 -- 172.21.15.46:0/1297325007 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38e406dda0 msgr2=0x7f38e406c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.188+0000 7f38e3e00640 1 --2- 172.21.15.46:0/1297325007 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38e406dda0 0x7f38e406c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.188+0000 7f38e3e00640 1 -- 172.21.15.46:0/1297325007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38e406d910 msgr2=0x7f38e40cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.188+0000 7f38e3e00640 1 --2- 172.21.15.46:0/1297325007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38e406d910 0x7f38e40cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.188+0000 7f38e3e00640 1 -- 172.21.15.46:0/1297325007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc009a70 con 0x7f38e41569a0 2024-03-29T08:26:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.188+0000 7f38e3e00640 1 --2- 172.21.15.46:0/1297325007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e41569a0 0x7f38e41f7840 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f38dc01a560 tx=0x7f38dc0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.188+0000 7f38dbe00640 1 -- 172.21.15.46:0/1297325007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0b9650 con 0x7f38e41569a0 2024-03-29T08:26:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.188+0000 7f38dbe00640 1 -- 172.21.15.46:0/1297325007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc0b97b0 con 0x7f38e41569a0 2024-03-29T08:26:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.188+0000 7f38dbe00640 1 -- 172.21.15.46:0/1297325007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc0b9b20 con 0x7f38e41569a0 2024-03-29T08:26:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.188+0000 7f38ea200640 1 -- 172.21.15.46:0/1297325007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e40cc8c0 con 0x7f38e41569a0 2024-03-29T08:26:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.188+0000 7f38ea200640 1 -- 172.21.15.46:0/1297325007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e40cbed0 con 0x7f38e41569a0 2024-03-29T08:26:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.188+0000 7f38ea200640 1 -- 172.21.15.46:0/1297325007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e40cc0c0 con 0x7f38e41569a0 2024-03-29T08:26:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.188+0000 7f38dbe00640 1 -- 172.21.15.46:0/1297325007 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f38dc0b9c80 con 0x7f38e41569a0 2024-03-29T08:26:14.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.188+0000 7f38dbe00640 1 --2- 172.21.15.46:0/1297325007 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38c0066690 0x7f38c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.188+0000 7f38dbe00640 1 -- 172.21.15.46:0/1297325007 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f38dc11f4d0 con 0x7f38e41569a0 2024-03-29T08:26:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.192+0000 7f38e3400640 1 --2- 172.21.15.46:0/1297325007 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38c0066690 0x7f38c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.192+0000 7f38e3400640 1 --2- 172.21.15.46:0/1297325007 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38c0066690 0x7f38c0068b50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f38d4007fd0 tx=0x7f38d4007d60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.192+0000 7f38dbe00640 1 -- 172.21.15.46:0/1297325007 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f38e40cc0c0 con 0x7f38e41569a0 2024-03-29T08:26:14.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.360+0000 7f38ea200640 1 -- 172.21.15.46:0/1297325007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38e40cc0c0 con 0x7f38e41569a0 2024-03-29T08:26:14.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.360+0000 7f38dbe00640 1 -- 172.21.15.46:0/1297325007 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+16185 (secure 0 0 0) 0x7f38e40cc0c0 con 0x7f38e41569a0 2024-03-29T08:26:14.361 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:26:14.362 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:25:02.334373+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:24:41.113618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-29T08:24:42.487380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6836/4166845858":"2024-03-30T08:25:02.334350+0000","172.21.15.73:6834/4166845858":"2024-03-30T08:25:02.334350+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:26:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.360+0000 7f38ea200640 1 -- 172.21.15.46:0/1297325007 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38c0066690 msgr2=0x7f38c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.360+0000 7f38ea200640 1 --2- 172.21.15.46:0/1297325007 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38c0066690 0x7f38c0068b50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f38d4007fd0 tx=0x7f38d4007d60 comp rx=0 tx=0).stop 2024-03-29T08:26:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.360+0000 7f38ea200640 1 -- 172.21.15.46:0/1297325007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e41569a0 msgr2=0x7f38e41f7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:14.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.360+0000 7f38ea200640 1 --2- 172.21.15.46:0/1297325007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e41569a0 0x7f38e41f7840 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f38dc01a560 tx=0x7f38dc0196d0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.360+0000 7f38ea200640 1 -- 172.21.15.46:0/1297325007 shutdown_connections 2024-03-29T08:26:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.360+0000 7f38ea200640 1 --2- 172.21.15.46:0/1297325007 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38c0066690 0x7f38c0068b50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.360+0000 7f38ea200640 1 --2- 172.21.15.46:0/1297325007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38e41569a0 0x7f38e41f7840 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.360+0000 7f38ea200640 1 --2- 172.21.15.46:0/1297325007 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38e406dda0 0x7f38e406c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.360+0000 7f38ea200640 1 --2- 172.21.15.46:0/1297325007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38e406d910 0x7f38e40cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.360+0000 7f38ea200640 1 -- 172.21.15.46:0/1297325007 >> 172.21.15.46:0/1297325007 conn(0x7f38e4066110 msgr2=0x7f38e41631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.364+0000 7f38ea200640 1 -- 172.21.15.46:0/1297325007 shutdown_connections 2024-03-29T08:26:14.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.364+0000 7f38ea200640 1 -- 172.21.15.46:0/1297325007 wait complete. 2024-03-29T08:26:14.383 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:24:41.113618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:24:42.487380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:26:14.383 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:26:14.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.524+0000 7f0bede00640 1 Processor -- start 2024-03-29T08:26:14.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.524+0000 7f0bede00640 1 -- start start 2024-03-29T08:26:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.528+0000 7f0bede00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0be8143eb0 0x7f0be80cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.528+0000 7f0bede00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0be80b1d40 0x7f0be80cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.528+0000 7f0bede00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be80b2360 con 0x7f0be815efe0 2024-03-29T08:26:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.528+0000 7f0bede00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be815e500 con 0x7f0be8143eb0 2024-03-29T08:26:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.528+0000 7f0bede00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be815e7e0 con 0x7f0be80b1d40 2024-03-29T08:26:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.528+0000 7f0be6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0be8143eb0 0x7f0be80cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.528+0000 7f0bdfe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0be815efe0 0x7f0be80b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47024/0 (socket says 172.21.15.46:47024) 2024-03-29T08:26:14.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.528+0000 7f0bdfe00640 1 -- 172.21.15.46:0/3823925985 learned_addr learned my addr 172.21.15.46:0/3823925985 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:26:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.528+0000 7f0be7400640 1 --2- 172.21.15.46:0/3823925985 >> v2:172.21.15.73:3300/0 conn(0x7f0be80b1d40 0x7f0be80cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.528+0000 7f0be6a00640 1 -- 172.21.15.46:0/3823925985 >> v2:172.21.15.73:3300/0 conn(0x7f0be80b1d40 msgr2=0x7f0be80cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.528+0000 7f0be6a00640 1 --2- 172.21.15.46:0/3823925985 >> v2:172.21.15.73:3300/0 conn(0x7f0be80b1d40 0x7f0be80cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.528+0000 7f0be6a00640 1 -- 172.21.15.46:0/3823925985 >> v1:172.21.15.46:6789/0 conn(0x7f0be815efe0 legacy=0x7f0be80b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.528+0000 7f0be6a00640 1 -- 172.21.15.46:0/3823925985 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be815eac0 con 0x7f0be8143eb0 2024-03-29T08:26:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.528+0000 7f0be7400640 1 --2- 172.21.15.46:0/3823925985 >> v2:172.21.15.73:3300/0 conn(0x7f0be80b1d40 0x7f0be80cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:14.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.528+0000 7f0be6000640 1 -- 172.21.15.46:0/3823925985 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901288649 0 0) 0x7f0be80b2360 con 0x7f0be815efe0 2024-03-29T08:26:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0be6a00640 1 --2- 172.21.15.46:0/3823925985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0be8143eb0 0x7f0be80cb790 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0be806ad40 tx=0x7f0bd801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91134d8b50452d2e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:14.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0be6000640 1 -- 172.21.15.46:0/3823925985 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80a8050 con 0x7f0be8143eb0 2024-03-29T08:26:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0be6000640 1 -- 172.21.15.46:0/3823925985 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd80ac5f0 con 0x7f0be8143eb0 2024-03-29T08:26:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0be6000640 1 -- 172.21.15.46:0/3823925985 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80ac980 con 0x7f0be8143eb0 2024-03-29T08:26:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bede00640 1 -- 172.21.15.46:0/3823925985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0be8143eb0 msgr2=0x7f0be80cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:14.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bede00640 1 --2- 172.21.15.46:0/3823925985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0be8143eb0 0x7f0be80cb790 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0be806ad40 tx=0x7f0bd801c480 comp rx=0 tx=0).stop 2024-03-29T08:26:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bede00640 1 -- 172.21.15.46:0/3823925985 shutdown_connections 2024-03-29T08:26:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bede00640 1 --2- 172.21.15.46:0/3823925985 >> v2:172.21.15.73:3300/0 conn(0x7f0be80b1d40 0x7f0be80cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bede00640 1 --2- 172.21.15.46:0/3823925985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0be8143eb0 0x7f0be80cb790 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bede00640 1 -- 172.21.15.46:0/3823925985 >> 172.21.15.46:0/3823925985 conn(0x7f0be806a390 msgr2=0x7f0be80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bede00640 1 -- 172.21.15.46:0/3823925985 shutdown_connections 2024-03-29T08:26:14.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bede00640 1 -- 172.21.15.46:0/3823925985 wait complete. 2024-03-29T08:26:14.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bede00640 1 Processor -- start 2024-03-29T08:26:14.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bede00640 1 -- start start 2024-03-29T08:26:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bede00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0be80b1d40 0x7f0be81639f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bede00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0be8143eb0 0x7f0be813cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bede00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0be815efe0 0x7f0be813d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bede00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be8164380 con 0x7f0be8143eb0 2024-03-29T08:26:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bede00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be8164f00 con 0x7f0be815efe0 2024-03-29T08:26:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bede00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be8165320 con 0x7f0be80b1d40 2024-03-29T08:26:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bdfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0be8143eb0 0x7f0be813cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bdfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0be8143eb0 0x7f0be813cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42504/0 (socket says 172.21.15.46:42504) 2024-03-29T08:26:14.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bdfe00640 1 -- 172.21.15.46:0/3509474154 learned_addr learned my addr 172.21.15.46:0/3509474154 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:14.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0bdfe00640 1 -- 172.21.15.46:0/3509474154 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0be80b1d40 msgr2=0x7f0be81639f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:14.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.532+0000 7f0be6a00640 1 --2- 172.21.15.46:0/3509474154 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0be80b1d40 0x7f0be81639f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0be7400640 1 --2- 172.21.15.46:0/3509474154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0be815efe0 0x7f0be813d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0bdfe00640 1 --2- 172.21.15.46:0/3509474154 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0be80b1d40 0x7f0be81639f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0bdfe00640 1 -- 172.21.15.46:0/3509474154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0be815efe0 msgr2=0x7f0be813d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:14.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0bdfe00640 1 --2- 172.21.15.46:0/3509474154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0be815efe0 0x7f0be813d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0bdfe00640 1 -- 172.21.15.46:0/3509474154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd8009420 con 0x7f0be8143eb0 2024-03-29T08:26:14.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0be6a00640 1 --2- 172.21.15.46:0/3509474154 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0be80b1d40 0x7f0be81639f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:26:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0be7400640 1 --2- 172.21.15.46:0/3509474154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0be815efe0 0x7f0be813d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:26:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0bdfe00640 1 --2- 172.21.15.46:0/3509474154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0be8143eb0 0x7f0be813cea0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000c480 tx=0x7f0bd000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0bdf400640 1 -- 172.21.15.46:0/3509474154 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00074e0 con 0x7f0be8143eb0 2024-03-29T08:26:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0bdf400640 1 -- 172.21.15.46:0/3509474154 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd0007660 con 0x7f0be8143eb0 2024-03-29T08:26:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0bdf400640 1 -- 172.21.15.46:0/3509474154 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0004a30 con 0x7f0be8143eb0 2024-03-29T08:26:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0bede00640 1 -- 172.21.15.46:0/3509474154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be8163f30 con 0x7f0be8143eb0 2024-03-29T08:26:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0bede00640 1 -- 172.21.15.46:0/3509474154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be8164560 con 0x7f0be8143eb0 2024-03-29T08:26:14.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0bede00640 1 -- 172.21.15.46:0/3509474154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be8071a70 con 0x7f0be8143eb0 2024-03-29T08:26:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0bdf400640 1 -- 172.21.15.46:0/3509474154 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0bd0004b90 con 0x7f0be8143eb0 2024-03-29T08:26:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0bdf400640 1 --2- 172.21.15.46:0/3509474154 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0bc4066690 0x7f0bc4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0bdf400640 1 -- 172.21.15.46:0/3509474154 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f0bd0083410 con 0x7f0be8143eb0 2024-03-29T08:26:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.536+0000 7f0be6a00640 1 --2- 172.21.15.46:0/3509474154 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0bc4066690 0x7f0bc4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.540+0000 7f0be6a00640 1 --2- 172.21.15.46:0/3509474154 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0bc4066690 0x7f0bc4068b50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80ae000 tx=0x7f0bd80a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.540+0000 7f0bdf400640 1 -- 172.21.15.46:0/3509474154 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0bd00541a0 con 0x7f0be8143eb0 2024-03-29T08:26:14.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.700+0000 7f0bede00640 1 -- 172.21.15.46:0/3509474154 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0be804e120 con 0x7f0be8143eb0 2024-03-29T08:26:14.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.700+0000 7f0bdf400640 1 -- 172.21.15.46:0/3509474154 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0bd0018020 con 0x7f0be8143eb0 2024-03-29T08:26:14.703 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:26:14.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.704+0000 7f0bede00640 1 -- 172.21.15.46:0/3509474154 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0bc4066690 msgr2=0x7f0bc4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:14.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.704+0000 7f0bede00640 1 --2- 172.21.15.46:0/3509474154 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0bc4066690 0x7f0bc4068b50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0bd80ae000 tx=0x7f0bd80a7000 comp rx=0 tx=0).stop 2024-03-29T08:26:14.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.704+0000 7f0bede00640 1 -- 172.21.15.46:0/3509474154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0be8143eb0 msgr2=0x7f0be813cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:14.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.704+0000 7f0bede00640 1 --2- 172.21.15.46:0/3509474154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0be8143eb0 0x7f0be813cea0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000c480 tx=0x7f0bd000c940 comp rx=0 tx=0).stop 2024-03-29T08:26:14.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.704+0000 7f0bede00640 1 -- 172.21.15.46:0/3509474154 shutdown_connections 2024-03-29T08:26:14.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.704+0000 7f0bede00640 1 --2- 172.21.15.46:0/3509474154 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0bc4066690 0x7f0bc4068b50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.704+0000 7f0bede00640 1 --2- 172.21.15.46:0/3509474154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0be815efe0 0x7f0be813d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.704+0000 7f0bede00640 1 --2- 172.21.15.46:0/3509474154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0be8143eb0 0x7f0be813cea0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.704+0000 7f0bede00640 1 --2- 172.21.15.46:0/3509474154 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0be80b1d40 0x7f0be81639f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.704+0000 7f0bede00640 1 -- 172.21.15.46:0/3509474154 >> 172.21.15.46:0/3509474154 conn(0x7f0be806a390 msgr2=0x7f0be80cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:14.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.704+0000 7f0bede00640 1 -- 172.21.15.46:0/3509474154 shutdown_connections 2024-03-29T08:26:14.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.704+0000 7f0bede00640 1 -- 172.21.15.46:0/3509474154 wait complete. 2024-03-29T08:26:14.724 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:26:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7aa5a00640 1 Processor -- start 2024-03-29T08:26:14.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7aa5a00640 1 -- start start 2024-03-29T08:26:14.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7aa5a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7aa0142a50 0x7f7aa00cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7aa5a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa015d260 con 0x7f7aa015df90 2024-03-29T08:26:14.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7aa5a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa00aefd0 con 0x7f7aa00ae990 2024-03-29T08:26:14.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7aa5a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa015d4b0 con 0x7f7aa0142a50 2024-03-29T08:26:14.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a97e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7aa015df90 0x7f7aa00ae280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47028/0 (socket says 172.21.15.46:47028) 2024-03-29T08:26:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a97e00640 1 -- 172.21.15.46:0/3161147067 learned_addr learned my addr 172.21.15.46:0/3161147067 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:26:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9ea00640 1 --2- 172.21.15.46:0/3161147067 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7aa0142a50 0x7f7aa00cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9e000640 1 -- 172.21.15.46:0/3161147067 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082066003 0 0) 0x7f7aa015d260 con 0x7f7aa015df90 2024-03-29T08:26:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9e000640 1 -- 172.21.15.46:0/3161147067 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a84003530 con 0x7f7aa015df90 2024-03-29T08:26:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9e000640 1 -- 172.21.15.46:0/3161147067 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3503498461 0 0) 0x7f7aa00aefd0 con 0x7f7aa00ae990 2024-03-29T08:26:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9e000640 1 -- 172.21.15.46:0/3161147067 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7aa015d260 con 0x7f7aa00ae990 2024-03-29T08:26:14.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9e000640 1 -- 172.21.15.46:0/3161147067 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1963136930 0 0) 0x7f7a84003530 con 0x7f7aa015df90 2024-03-29T08:26:14.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9e000640 1 -- 172.21.15.46:0/3161147067 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7aa0142a50 msgr2=0x7f7aa00cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:14.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9e000640 1 --2- 172.21.15.46:0/3161147067 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7aa0142a50 0x7f7aa00cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9e000640 1 -- 172.21.15.46:0/3161147067 >> v1:172.21.15.73:6789/0 conn(0x7f7aa00ae990 legacy=0x7f7aa00af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:14.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9e000640 1 -- 172.21.15.46:0/3161147067 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa015d790 con 0x7f7aa015df90 2024-03-29T08:26:14.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9e000640 1 -- 172.21.15.46:0/3161147067 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f7a88001680 con 0x7f7aa015df90 2024-03-29T08:26:14.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9ea00640 1 --2- 172.21.15.46:0/3161147067 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7aa0142a50 0x7f7aa00cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:26:14.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9e000640 1 -- 172.21.15.46:0/3161147067 >> v1:172.21.15.46:6789/0 conn(0x7f7aa015df90 legacy=0x7f7aa00ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:14.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9e000640 1 --2- 172.21.15.46:0/3161147067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a84003b40 0x7f7a84003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9e000640 1 --2- 172.21.15.46:0/3161147067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a840047d0 0x7f7a84004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9e000640 1 --2- 172.21.15.46:0/3161147067 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7aa0142a50 0x7f7a84005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9e000640 1 -- 172.21.15.46:0/3161147067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa015d4b0 con 0x7f7a84003b40 2024-03-29T08:26:14.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9e000640 1 -- 172.21.15.46:0/3161147067 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8400dc80 con 0x7f7a840047d0 2024-03-29T08:26:14.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9e000640 1 -- 172.21.15.46:0/3161147067 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8400de40 con 0x7f7aa0142a50 2024-03-29T08:26:14.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9ea00640 1 --2- 172.21.15.46:0/3161147067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a84003b40 0x7f7a84003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a97e00640 1 --2- 172.21.15.46:0/3161147067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a840047d0 0x7f7a84004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9f400640 1 --2- 172.21.15.46:0/3161147067 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7aa0142a50 0x7f7a84005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9ea00640 1 -- 172.21.15.46:0/3161147067 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7aa0142a50 msgr2=0x7f7a84005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:14.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9ea00640 1 --2- 172.21.15.46:0/3161147067 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7aa0142a50 0x7f7a84005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9ea00640 1 -- 172.21.15.46:0/3161147067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a840047d0 msgr2=0x7f7a84004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:14.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9ea00640 1 --2- 172.21.15.46:0/3161147067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a840047d0 0x7f7a84004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.868+0000 7f7a9ea00640 1 -- 172.21.15.46:0/3161147067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a90009420 con 0x7f7a84003b40 2024-03-29T08:26:14.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.872+0000 7f7a9ea00640 1 --2- 172.21.15.46:0/3161147067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a84003b40 0x7f7a84003f40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009c80 tx=0x7f7a9001cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb598ccedafc578 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:14.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.872+0000 7f7a9e000640 1 -- 172.21.15.46:0/3161147067 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a900a5650 con 0x7f7a84003b40 2024-03-29T08:26:14.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.872+0000 7f7a9e000640 1 -- 172.21.15.46:0/3161147067 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a900a57b0 con 0x7f7a84003b40 2024-03-29T08:26:14.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.872+0000 7f7a9e000640 1 -- 172.21.15.46:0/3161147067 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a900a5960 con 0x7f7a84003b40 2024-03-29T08:26:14.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.872+0000 7f7aa5a00640 1 -- 172.21.15.46:0/3161147067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a84003b40 msgr2=0x7f7a84003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.872+0000 7f7aa5a00640 1 --2- 172.21.15.46:0/3161147067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a84003b40 0x7f7a84003f40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009c80 tx=0x7f7a9001cbb0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.872+0000 7f7a9f400640 1 -- 172.21.15.46:0/3161147067 reap_dead start 2024-03-29T08:26:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.872+0000 7f7aa5a00640 1 -- 172.21.15.46:0/3161147067 shutdown_connections 2024-03-29T08:26:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.872+0000 7f7aa5a00640 1 -- 172.21.15.46:0/3161147067 >> 172.21.15.46:0/3161147067 conn(0x7f7aa006a390 msgr2=0x7f7aa00cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.872+0000 7f7aa5a00640 1 -- 172.21.15.46:0/3161147067 shutdown_connections 2024-03-29T08:26:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.872+0000 7f7aa5a00640 1 -- 172.21.15.46:0/3161147067 wait complete. 2024-03-29T08:26:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7aa5a00640 1 Processor -- start 2024-03-29T08:26:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7aa5a00640 1 -- start start 2024-03-29T08:26:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7aa5a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a84003b40 0x7f7aa0158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7aa5a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa015b640 0x7f7aa013d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7aa5a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7aa015bac0 0x7f7aa013d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7aa5a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa01593b0 con 0x7f7a84003b40 2024-03-29T08:26:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7aa5a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa0138d30 con 0x7f7aa015b640 2024-03-29T08:26:14.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7aa5a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa0139010 con 0x7f7aa015bac0 2024-03-29T08:26:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7a9ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a84003b40 0x7f7aa0158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7a9ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a84003b40 0x7f7aa0158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42532/0 (socket says 172.21.15.46:42532) 2024-03-29T08:26:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7a9ea00640 1 -- 172.21.15.46:0/3816469368 learned_addr learned my addr 172.21.15.46:0/3816469368 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7a97e00640 1 --2- 172.21.15.46:0/3816469368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa015b640 0x7f7aa013d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7a9f400640 1 --2- 172.21.15.46:0/3816469368 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7aa015bac0 0x7f7aa013d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7a9ea00640 1 -- 172.21.15.46:0/3816469368 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7aa015bac0 msgr2=0x7f7aa013d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7a9ea00640 1 --2- 172.21.15.46:0/3816469368 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7aa015bac0 0x7f7aa013d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7a9ea00640 1 -- 172.21.15.46:0/3816469368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa015b640 msgr2=0x7f7aa013d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7a9ea00640 1 --2- 172.21.15.46:0/3816469368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa015b640 0x7f7aa013d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7a9ea00640 1 -- 172.21.15.46:0/3816469368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa01392f0 con 0x7f7a84003b40 2024-03-29T08:26:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7a9f400640 1 --2- 172.21.15.46:0/3816469368 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7aa015bac0 0x7f7aa013d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7a9ea00640 1 --2- 172.21.15.46:0/3816469368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a84003b40 0x7f7aa0158e70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009bd0 tx=0x7f7a84005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7a96a00640 1 -- 172.21.15.46:0/3816469368 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a900be070 con 0x7f7a84003b40 2024-03-29T08:26:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7a96a00640 1 -- 172.21.15.46:0/3816469368 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a90006e40 con 0x7f7a84003b40 2024-03-29T08:26:14.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7a96a00640 1 -- 172.21.15.46:0/3816469368 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a900a5650 con 0x7f7a84003b40 2024-03-29T08:26:14.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7aa5a00640 1 -- 172.21.15.46:0/3816469368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa01395d0 con 0x7f7a84003b40 2024-03-29T08:26:14.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.876+0000 7f7aa5a00640 1 -- 172.21.15.46:0/3816469368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa01388e0 con 0x7f7a84003b40 2024-03-29T08:26:14.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.880+0000 7f7aa5a00640 1 -- 172.21.15.46:0/3816469368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa0138ad0 con 0x7f7a84003b40 2024-03-29T08:26:14.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.880+0000 7f7a96a00640 1 -- 172.21.15.46:0/3816469368 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7a900b4040 con 0x7f7a84003b40 2024-03-29T08:26:14.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.880+0000 7f7a96a00640 1 --2- 172.21.15.46:0/3816469368 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7a7c0666c0 0x7f7a7c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:14.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.880+0000 7f7a96a00640 1 -- 172.21.15.46:0/3816469368 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7a9011e160 con 0x7f7a84003b40 2024-03-29T08:26:14.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.880+0000 7f7a97e00640 1 --2- 172.21.15.46:0/3816469368 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7a7c0666c0 0x7f7a7c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:14.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.880+0000 7f7a97e00640 1 --2- 172.21.15.46:0/3816469368 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7a7c0666c0 0x7f7a7c068b80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7aa006d3e0 tx=0x7f7a88007040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:14.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:14.884+0000 7f7a96a00640 1 -- 172.21.15.46:0/3816469368 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7a900eb5a0 con 0x7f7a84003b40 2024-03-29T08:26:15.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.044+0000 7f7aa5a00640 1 -- 172.21.15.46:0/3816469368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7aa0071a70 con 0x7f7a84003b40 2024-03-29T08:26:15.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.044+0000 7f7a96a00640 1 -- 172.21.15.46:0/3816469368 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a900eebc0 con 0x7f7a84003b40 2024-03-29T08:26:15.045 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:26:15.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.044+0000 7f7aa5a00640 1 -- 172.21.15.46:0/3816469368 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7a7c0666c0 msgr2=0x7f7a7c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:15.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.044+0000 7f7aa5a00640 1 --2- 172.21.15.46:0/3816469368 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7a7c0666c0 0x7f7a7c068b80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7aa006d3e0 tx=0x7f7a88007040 comp rx=0 tx=0).stop 2024-03-29T08:26:15.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.044+0000 7f7aa5a00640 1 -- 172.21.15.46:0/3816469368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a84003b40 msgr2=0x7f7aa0158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.044+0000 7f7aa5a00640 1 --2- 172.21.15.46:0/3816469368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a84003b40 0x7f7aa0158e70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f7a90009bd0 tx=0x7f7a84005800 comp rx=0 tx=0).stop 2024-03-29T08:26:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.044+0000 7f7aa5a00640 1 -- 172.21.15.46:0/3816469368 shutdown_connections 2024-03-29T08:26:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.044+0000 7f7aa5a00640 1 --2- 172.21.15.46:0/3816469368 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7a7c0666c0 0x7f7a7c068b80 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.044+0000 7f7aa5a00640 1 --2- 172.21.15.46:0/3816469368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa015b640 0x7f7aa013d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.044+0000 7f7aa5a00640 1 --2- 172.21.15.46:0/3816469368 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7aa015bac0 0x7f7aa013d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.044+0000 7f7aa5a00640 1 --2- 172.21.15.46:0/3816469368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a84003b40 0x7f7aa0158e70 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.044+0000 7f7aa5a00640 1 -- 172.21.15.46:0/3816469368 >> 172.21.15.46:0/3816469368 conn(0x7f7aa006a390 msgr2=0x7f7aa00af530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.044+0000 7f7aa5a00640 1 -- 172.21.15.46:0/3816469368 shutdown_connections 2024-03-29T08:26:15.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.044+0000 7f7aa5a00640 1 -- 172.21.15.46:0/3816469368 wait complete. 2024-03-29T08:26:15.066 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:26:15.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.208+0000 7f246ac00640 1 Processor -- start 2024-03-29T08:26:15.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.208+0000 7f246ac00640 1 -- start start 2024-03-29T08:26:15.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.212+0000 7f246ac00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f246415a9f0 0x7f24640ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:15.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.212+0000 7f246ac00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24640cf590 con 0x7f24640cc390 2024-03-29T08:26:15.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.212+0000 7f246ac00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f246406e130 con 0x7f24640ccf60 2024-03-29T08:26:15.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.212+0000 7f246ac00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246406e410 con 0x7f246415a9f0 2024-03-29T08:26:15.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.212+0000 7f246a200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f24640cc390 0x7f246406f600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47030/0 (socket says 172.21.15.46:47030) 2024-03-29T08:26:15.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.212+0000 7f246a200640 1 -- 172.21.15.46:0/2303698327 learned_addr learned my addr 172.21.15.46:0/2303698327 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:26:15.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.212+0000 7f2469800640 1 --2- 172.21.15.46:0/2303698327 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f246415a9f0 0x7f24640ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:15.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.212+0000 7f2463e00640 1 -- 172.21.15.46:0/2303698327 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968554370 0 0) 0x7f24640cf590 con 0x7f24640cc390 2024-03-29T08:26:15.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.212+0000 7f2463e00640 1 -- 172.21.15.46:0/2303698327 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f244c003530 con 0x7f24640cc390 2024-03-29T08:26:15.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.212+0000 7f2463e00640 1 -- 172.21.15.46:0/2303698327 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645788188 0 0) 0x7f246406e130 con 0x7f24640ccf60 2024-03-29T08:26:15.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.212+0000 7f2463e00640 1 -- 172.21.15.46:0/2303698327 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24640cf590 con 0x7f24640ccf60 2024-03-29T08:26:15.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.212+0000 7f2469800640 1 -- 172.21.15.46:0/2303698327 >> v1:172.21.15.73:6789/0 conn(0x7f24640ccf60 legacy=0x7f246406eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:15.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.212+0000 7f2463e00640 1 -- 172.21.15.46:0/2303698327 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3569437287 0 0) 0x7f244c003530 con 0x7f24640cc390 2024-03-29T08:26:15.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.212+0000 7f2469800640 1 -- 172.21.15.46:0/2303698327 >> v1:172.21.15.46:6789/0 conn(0x7f24640cc390 legacy=0x7f246406f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:15.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.212+0000 7f2469800640 1 -- 172.21.15.46:0/2303698327 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246406e6f0 con 0x7f246415a9f0 2024-03-29T08:26:15.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f2469800640 1 --2- 172.21.15.46:0/2303698327 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f246415a9f0 0x7f24640ced40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f24580097f0 tx=0x7f245801ebf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ba9c797003b02d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:15.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f2463e00640 1 -- 172.21.15.46:0/2303698327 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c001a00 con 0x7f246415a9f0 2024-03-29T08:26:15.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f2463e00640 1 -- 172.21.15.46:0/2303698327 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24580a8670 con 0x7f246415a9f0 2024-03-29T08:26:15.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f2463e00640 1 -- 172.21.15.46:0/2303698327 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24580a89e0 con 0x7f246415a9f0 2024-03-29T08:26:15.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f246ac00640 1 -- 172.21.15.46:0/2303698327 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f246415a9f0 msgr2=0x7f24640ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:15.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f246ac00640 1 --2- 172.21.15.46:0/2303698327 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f246415a9f0 0x7f24640ced40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f24580097f0 tx=0x7f245801ebf0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f246ac00640 1 -- 172.21.15.46:0/2303698327 shutdown_connections 2024-03-29T08:26:15.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f246ac00640 1 --2- 172.21.15.46:0/2303698327 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f246415a9f0 0x7f24640ced40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f246ac00640 1 -- 172.21.15.46:0/2303698327 >> 172.21.15.46:0/2303698327 conn(0x7f24640d0550 msgr2=0x7f24640d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:15.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f246ac00640 1 -- 172.21.15.46:0/2303698327 shutdown_connections 2024-03-29T08:26:15.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f246ac00640 1 -- 172.21.15.46:0/2303698327 wait complete. 2024-03-29T08:26:15.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f246ac00640 1 Processor -- start 2024-03-29T08:26:15.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f246ac00640 1 -- start start 2024-03-29T08:26:15.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f246ac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24640cc390 0x7f2464151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:15.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f246ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24640ccf60 0x7f2464152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:15.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f246ac00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f246415a9f0 0x7f24640ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:15.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f246ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246415bef0 con 0x7f24640ccf60 2024-03-29T08:26:15.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f246ac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246415c080 con 0x7f24640cc390 2024-03-29T08:26:15.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f246ac00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24640c7620 con 0x7f246415a9f0 2024-03-29T08:26:15.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.216+0000 7f2468e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24640ccf60 0x7f2464152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:15.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f2469800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24640cc390 0x7f2464151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:15.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f2468e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24640ccf60 0x7f2464152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42538/0 (socket says 172.21.15.46:42538) 2024-03-29T08:26:15.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f2469800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24640cc390 0x7f2464151630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47088/0 (socket says 172.21.15.46:47088) 2024-03-29T08:26:15.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f2468e00640 1 -- 172.21.15.46:0/2842745996 learned_addr learned my addr 172.21.15.46:0/2842745996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:15.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f246a200640 1 --2- 172.21.15.46:0/2842745996 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f246415a9f0 0x7f24640ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f2468e00640 1 -- 172.21.15.46:0/2842745996 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f246415a9f0 msgr2=0x7f24640ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f2468e00640 1 --2- 172.21.15.46:0/2842745996 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f246415a9f0 0x7f24640ae530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f2468e00640 1 -- 172.21.15.46:0/2842745996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24640cc390 msgr2=0x7f2464151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f2468e00640 1 --2- 172.21.15.46:0/2842745996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24640cc390 0x7f2464151630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f2468e00640 1 -- 172.21.15.46:0/2842745996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2458009420 con 0x7f24640ccf60 2024-03-29T08:26:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f2468e00640 1 --2- 172.21.15.46:0/2842745996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24640ccf60 0x7f2464152e40 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f245000c490 tx=0x7f245000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:15.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f2461600640 1 -- 172.21.15.46:0/2842745996 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24500052d0 con 0x7f24640ccf60 2024-03-29T08:26:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f2461600640 1 -- 172.21.15.46:0/2842745996 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2450005430 con 0x7f24640ccf60 2024-03-29T08:26:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f2461600640 1 -- 172.21.15.46:0/2842745996 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2450012870 con 0x7f24640ccf60 2024-03-29T08:26:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f246ac00640 1 -- 172.21.15.46:0/2842745996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2464151c10 con 0x7f24640ccf60 2024-03-29T08:26:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f246ac00640 1 -- 172.21.15.46:0/2842745996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24641521d0 con 0x7f24640ccf60 2024-03-29T08:26:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f246ac00640 1 -- 172.21.15.46:0/2842745996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f246404e040 con 0x7f24640ccf60 2024-03-29T08:26:15.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f2461600640 1 -- 172.21.15.46:0/2842745996 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f24500129d0 con 0x7f24640ccf60 2024-03-29T08:26:15.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f2461600640 1 --2- 172.21.15.46:0/2842745996 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2444066690 0x7f2444068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:15.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.220+0000 7f2461600640 1 -- 172.21.15.46:0/2842745996 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2450083cf0 con 0x7f24640ccf60 2024-03-29T08:26:15.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.224+0000 7f2469800640 1 --2- 172.21.15.46:0/2842745996 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2444066690 0x7f2444068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:15.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.224+0000 7f2469800640 1 --2- 172.21.15.46:0/2842745996 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2444066690 0x7f2444068b50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f24580b2000 tx=0x7f24580aa070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:15.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.224+0000 7f2461600640 1 -- 172.21.15.46:0/2842745996 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2450051010 con 0x7f24640ccf60 2024-03-29T08:26:15.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.388+0000 7f246ac00640 1 -- 172.21.15.46:0/2842745996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f24640b2e30 con 0x7f24640ccf60 2024-03-29T08:26:15.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.388+0000 7f2461600640 1 -- 172.21.15.46:0/2842745996 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2450054630 con 0x7f24640ccf60 2024-03-29T08:26:15.389 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:26:15.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.388+0000 7f246ac00640 1 -- 172.21.15.46:0/2842745996 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2444066690 msgr2=0x7f2444068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:15.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.388+0000 7f246ac00640 1 --2- 172.21.15.46:0/2842745996 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2444066690 0x7f2444068b50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f24580b2000 tx=0x7f24580aa070 comp rx=0 tx=0).stop 2024-03-29T08:26:15.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.388+0000 7f246ac00640 1 -- 172.21.15.46:0/2842745996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24640ccf60 msgr2=0x7f2464152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:15.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.388+0000 7f246ac00640 1 --2- 172.21.15.46:0/2842745996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24640ccf60 0x7f2464152e40 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f245000c490 tx=0x7f245000c950 comp rx=0 tx=0).stop 2024-03-29T08:26:15.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.388+0000 7f246ac00640 1 -- 172.21.15.46:0/2842745996 shutdown_connections 2024-03-29T08:26:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.388+0000 7f246ac00640 1 --2- 172.21.15.46:0/2842745996 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2444066690 0x7f2444068b50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.388+0000 7f246ac00640 1 --2- 172.21.15.46:0/2842745996 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f246415a9f0 0x7f24640ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.388+0000 7f246ac00640 1 --2- 172.21.15.46:0/2842745996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24640ccf60 0x7f2464152e40 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.388+0000 7f246ac00640 1 --2- 172.21.15.46:0/2842745996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24640cc390 0x7f2464151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.388+0000 7f246ac00640 1 -- 172.21.15.46:0/2842745996 >> 172.21.15.46:0/2842745996 conn(0x7f24640d0550 msgr2=0x7f24640d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.388+0000 7f246ac00640 1 -- 172.21.15.46:0/2842745996 shutdown_connections 2024-03-29T08:26:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.388+0000 7f246ac00640 1 -- 172.21.15.46:0/2842745996 wait complete. 2024-03-29T08:26:15.409 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:26:15.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.548+0000 7f1423a00640 1 Processor -- start 2024-03-29T08:26:15.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.548+0000 7f1423a00640 1 -- start start 2024-03-29T08:26:15.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.548+0000 7f1423a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f141c0b17b0 0x7f141c06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:15.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.548+0000 7f1423a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c06e310 con 0x7f141c0cdd10 2024-03-29T08:26:15.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.548+0000 7f1423a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c06e590 con 0x7f141c0aeb50 2024-03-29T08:26:15.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.548+0000 7f1423a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0b1c40 con 0x7f141c0b17b0 2024-03-29T08:26:15.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.548+0000 7f1413e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f141c0cdd10 0x7f141c15b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:47046/0 (socket says 172.21.15.46:47046) 2024-03-29T08:26:15.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.548+0000 7f1413e00640 1 -- 172.21.15.46:0/4050017837 learned_addr learned my addr 172.21.15.46:0/4050017837 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:26:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.552+0000 7f1421a00640 1 --2- 172.21.15.46:0/4050017837 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f141c0b17b0 0x7f141c06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.552+0000 7f141be00640 1 -- 172.21.15.46:0/4050017837 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2118074730 0 0) 0x7f141c06e310 con 0x7f141c0cdd10 2024-03-29T08:26:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.552+0000 7f141be00640 1 -- 172.21.15.46:0/4050017837 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1400003530 con 0x7f141c0cdd10 2024-03-29T08:26:15.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.552+0000 7f141be00640 1 -- 172.21.15.46:0/4050017837 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223466086 0 0) 0x7f141c06e590 con 0x7f141c0aeb50 2024-03-29T08:26:15.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.552+0000 7f141be00640 1 -- 172.21.15.46:0/4050017837 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f141c06e310 con 0x7f141c0aeb50 2024-03-29T08:26:15.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.552+0000 7f1421a00640 1 -- 172.21.15.46:0/4050017837 >> v1:172.21.15.73:6789/0 conn(0x7f141c0aeb50 legacy=0x7f141c0cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:15.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.552+0000 7f1421a00640 1 -- 172.21.15.46:0/4050017837 >> v1:172.21.15.46:6789/0 conn(0x7f141c0cdd10 legacy=0x7f141c15b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:15.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.552+0000 7f1421a00640 1 -- 172.21.15.46:0/4050017837 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c06e830 con 0x7f141c0b17b0 2024-03-29T08:26:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.552+0000 7f1421a00640 1 --2- 172.21.15.46:0/4050017837 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f141c0b17b0 0x7f141c06ddd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1414009dc0 tx=0x7f141401c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1945f9e01d1877f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.552+0000 7f141be00640 1 -- 172.21.15.46:0/4050017837 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14140a5a20 con 0x7f141c0b17b0 2024-03-29T08:26:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.552+0000 7f141be00640 1 -- 172.21.15.46:0/4050017837 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14140a5b80 con 0x7f141c0b17b0 2024-03-29T08:26:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.552+0000 7f141be00640 1 -- 172.21.15.46:0/4050017837 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14140ad440 con 0x7f141c0b17b0 2024-03-29T08:26:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1423a00640 1 -- 172.21.15.46:0/4050017837 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f141c0b17b0 msgr2=0x7f141c06ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1423a00640 1 --2- 172.21.15.46:0/4050017837 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f141c0b17b0 0x7f141c06ddd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1414009dc0 tx=0x7f141401c960 comp rx=0 tx=0).stop 2024-03-29T08:26:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1423a00640 1 -- 172.21.15.46:0/4050017837 shutdown_connections 2024-03-29T08:26:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1423a00640 1 --2- 172.21.15.46:0/4050017837 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f141c0b17b0 0x7f141c06ddd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1423a00640 1 -- 172.21.15.46:0/4050017837 >> 172.21.15.46:0/4050017837 conn(0x7f141c098a40 msgr2=0x7f141c0b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1423a00640 1 -- 172.21.15.46:0/4050017837 shutdown_connections 2024-03-29T08:26:15.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1423a00640 1 -- 172.21.15.46:0/4050017837 wait complete. 2024-03-29T08:26:15.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1423a00640 1 Processor -- start 2024-03-29T08:26:15.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1423a00640 1 -- start start 2024-03-29T08:26:15.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1423a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f141c0aeb50 0x7f141c165090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:15.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1423a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f141c0b17b0 0x7f141c0c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:15.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1423a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f141c0cdd10 0x7f141c0cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:15.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1423a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c1515b0 con 0x7f141c0b17b0 2024-03-29T08:26:15.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1423a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c1655d0 con 0x7f141c0aeb50 2024-03-29T08:26:15.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1423a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0ca780 con 0x7f141c0cdd10 2024-03-29T08:26:15.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1413e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f141c0b17b0 0x7f141c0c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:15.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1413e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f141c0b17b0 0x7f141c0c9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42540/0 (socket says 172.21.15.46:42540) 2024-03-29T08:26:15.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1413e00640 1 -- 172.21.15.46:0/2862121774 learned_addr learned my addr 172.21.15.46:0/2862121774 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:15.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1421a00640 1 --2- 172.21.15.46:0/2862121774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f141c0cdd10 0x7f141c0cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:15.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.556+0000 7f1421000640 1 --2- 172.21.15.46:0/2862121774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f141c0aeb50 0x7f141c165090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.560+0000 7f1413e00640 1 -- 172.21.15.46:0/2862121774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f141c0cdd10 msgr2=0x7f141c0cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.560+0000 7f1413e00640 1 --2- 172.21.15.46:0/2862121774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f141c0cdd10 0x7f141c0cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.560+0000 7f1413e00640 1 -- 172.21.15.46:0/2862121774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f141c0aeb50 msgr2=0x7f141c165090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.560+0000 7f1413e00640 1 --2- 172.21.15.46:0/2862121774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f141c0aeb50 0x7f141c165090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.560+0000 7f1413e00640 1 -- 172.21.15.46:0/2862121774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1414009a70 con 0x7f141c0b17b0 2024-03-29T08:26:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.560+0000 7f1413e00640 1 --2- 172.21.15.46:0/2862121774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f141c0b17b0 0x7f141c0c9f30 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f140400c4f0 tx=0x7f140400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.560+0000 7f1419600640 1 -- 172.21.15.46:0/2862121774 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404005cf0 con 0x7f141c0b17b0 2024-03-29T08:26:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.560+0000 7f1419600640 1 -- 172.21.15.46:0/2862121774 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1404005e50 con 0x7f141c0b17b0 2024-03-29T08:26:15.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.560+0000 7f1419600640 1 -- 172.21.15.46:0/2862121774 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404007270 con 0x7f141c0b17b0 2024-03-29T08:26:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.560+0000 7f1423a00640 1 -- 172.21.15.46:0/2862121774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141c0caa60 con 0x7f141c0b17b0 2024-03-29T08:26:15.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.560+0000 7f1423a00640 1 -- 172.21.15.46:0/2862121774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141c0cb020 con 0x7f141c0b17b0 2024-03-29T08:26:15.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.560+0000 7f1423a00640 1 -- 172.21.15.46:0/2862121774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141c0b1590 con 0x7f141c0b17b0 2024-03-29T08:26:15.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.560+0000 7f1419600640 1 -- 172.21.15.46:0/2862121774 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f14040073d0 con 0x7f141c0b17b0 2024-03-29T08:26:15.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.560+0000 7f1419600640 1 --2- 172.21.15.46:0/2862121774 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f13f80666c0 0x7f13f8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:15.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.560+0000 7f1419600640 1 -- 172.21.15.46:0/2862121774 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1404083690 con 0x7f141c0b17b0 2024-03-29T08:26:15.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.560+0000 7f1421000640 1 --2- 172.21.15.46:0/2862121774 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f13f80666c0 0x7f13f8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:15.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.564+0000 7f1421000640 1 --2- 172.21.15.46:0/2862121774 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f13f80666c0 0x7f13f8068b80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f140c0015e0 tx=0x7f140c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:15.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.564+0000 7f1419600640 1 -- 172.21.15.46:0/2862121774 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1404050a50 con 0x7f141c0b17b0 2024-03-29T08:26:15.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.736+0000 7f1423a00640 1 -- 172.21.15.46:0/2862121774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f141c04e120 con 0x7f141c0b17b0 2024-03-29T08:26:15.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.736+0000 7f1419600640 1 -- 172.21.15.46:0/2862121774 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v93) v1 ==== 80+0+122 (secure 0 0 0) 0x7f1404017090 con 0x7f141c0b17b0 2024-03-29T08:26:15.738 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.73:6836/4166845858 2024-03-30T08:25:02.334350+0000 2024-03-29T08:26:15.738 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.73:6834/4166845858 2024-03-30T08:25:02.334350+0000 2024-03-29T08:26:15.738 INFO:teuthology.orchestra.run.smithi046.stderr:listed 2 entries 2024-03-29T08:26:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.740+0000 7f1423a00640 1 -- 172.21.15.46:0/2862121774 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f13f80666c0 msgr2=0x7f13f8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.740+0000 7f1423a00640 1 --2- 172.21.15.46:0/2862121774 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f13f80666c0 0x7f13f8068b80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f140c0015e0 tx=0x7f140c009040 comp rx=0 tx=0).stop 2024-03-29T08:26:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.740+0000 7f1423a00640 1 -- 172.21.15.46:0/2862121774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f141c0b17b0 msgr2=0x7f141c0c9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.740+0000 7f1423a00640 1 --2- 172.21.15.46:0/2862121774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f141c0b17b0 0x7f141c0c9f30 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f140400c4f0 tx=0x7f140400c9b0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.740+0000 7f1423a00640 1 -- 172.21.15.46:0/2862121774 shutdown_connections 2024-03-29T08:26:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.740+0000 7f1423a00640 1 --2- 172.21.15.46:0/2862121774 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f13f80666c0 0x7f13f8068b80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.740+0000 7f1423a00640 1 --2- 172.21.15.46:0/2862121774 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f141c0cdd10 0x7f141c0cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.740+0000 7f1423a00640 1 --2- 172.21.15.46:0/2862121774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f141c0b17b0 0x7f141c0c9f30 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.740+0000 7f1423a00640 1 --2- 172.21.15.46:0/2862121774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f141c0aeb50 0x7f141c165090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.740+0000 7f1423a00640 1 -- 172.21.15.46:0/2862121774 >> 172.21.15.46:0/2862121774 conn(0x7f141c098a40 msgr2=0x7f141c0af850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.740+0000 7f1423a00640 1 -- 172.21.15.46:0/2862121774 shutdown_connections 2024-03-29T08:26:15.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:15.740+0000 7f1423a00640 1 -- 172.21.15.46:0/2862121774 wait complete. 2024-03-29T08:26:15.758 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-03-29T08:26:15.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:26:15.758 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T08:26:15.776 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[45319]: fuse finished with error 0 and tester_r 0 2024-03-29T08:26:15.779 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:26:21.881 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:26:21.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:26:21.882 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:26:21.893 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:26:22.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.084+0000 7f827cc00640 1 Processor -- start 2024-03-29T08:26:22.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.084+0000 7f827cc00640 1 -- start start 2024-03-29T08:26:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.088+0000 7f827cc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f827815a7c0 0x7f827806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.088+0000 7f827cc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827806d130 con 0x7f827815cd90 2024-03-29T08:26:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.088+0000 7f827cc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f827815b8f0 con 0x7f82781593d0 2024-03-29T08:26:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.088+0000 7f827cc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827815ad50 con 0x7f827815a7c0 2024-03-29T08:26:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.088+0000 7f8276800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f827815a7c0 0x7f827806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.088+0000 7f826fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f827815cd90 0x7f827815d940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39026/0 (socket says 172.21.15.46:39026) 2024-03-29T08:26:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.088+0000 7f826fe00640 1 -- 172.21.15.46:0/3343086154 learned_addr learned my addr 172.21.15.46:0/3343086154 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:26:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.088+0000 7f8276800640 1 -- 172.21.15.46:0/3343086154 >> v1:172.21.15.73:6789/0 conn(0x7f82781593d0 legacy=0x7f827815d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.088+0000 7f8276800640 1 -- 172.21.15.46:0/3343086154 >> v1:172.21.15.46:6789/0 conn(0x7f827815cd90 legacy=0x7f827815d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.088+0000 7f8276800640 1 -- 172.21.15.46:0/3343086154 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f827815e050 con 0x7f827815a7c0 2024-03-29T08:26:22.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.088+0000 7f8275400640 1 -- 172.21.15.46:0/3343086154 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459701245 0 0) 0x7f827806d130 con 0x7f827815cd90 2024-03-29T08:26:22.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f8276800640 1 --2- 172.21.15.46:0/3343086154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a7c0 0x7f827806cbf0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f8270009d90 tx=0x7f827001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd07e89bb079d43 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f8275400640 1 -- 172.21.15.46:0/3343086154 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82700a59f0 con 0x7f827815a7c0 2024-03-29T08:26:22.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f8275400640 1 -- 172.21.15.46:0/3343086154 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82700a5b50 con 0x7f827815a7c0 2024-03-29T08:26:22.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f8275400640 1 -- 172.21.15.46:0/3343086154 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82700ad440 con 0x7f827815a7c0 2024-03-29T08:26:22.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f827cc00640 1 -- 172.21.15.46:0/3343086154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a7c0 msgr2=0x7f827806cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f827cc00640 1 --2- 172.21.15.46:0/3343086154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a7c0 0x7f827806cbf0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f8270009d90 tx=0x7f827001c930 comp rx=0 tx=0).stop 2024-03-29T08:26:22.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f827cc00640 1 -- 172.21.15.46:0/3343086154 shutdown_connections 2024-03-29T08:26:22.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f827cc00640 1 --2- 172.21.15.46:0/3343086154 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a7c0 0x7f827806cbf0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f827cc00640 1 -- 172.21.15.46:0/3343086154 >> 172.21.15.46:0/3343086154 conn(0x7f8278066110 msgr2=0x7f8278069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:22.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f827cc00640 1 -- 172.21.15.46:0/3343086154 shutdown_connections 2024-03-29T08:26:22.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f827cc00640 1 -- 172.21.15.46:0/3343086154 wait complete. 2024-03-29T08:26:22.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f827cc00640 1 Processor -- start 2024-03-29T08:26:22.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f827cc00640 1 -- start start 2024-03-29T08:26:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f827cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82781593d0 0x7f82781570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f827cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a7c0 0x7f82780aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f827cc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f827815cd90 0x7f8278154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f827cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8278154950 con 0x7f827815a7c0 2024-03-29T08:26:22.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f827cc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82781579a0 con 0x7f82781593d0 2024-03-29T08:26:22.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.092+0000 7f827cc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8278154300 con 0x7f827815cd90 2024-03-29T08:26:22.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f826fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a7c0 0x7f82780aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f826fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a7c0 0x7f82780aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58482/0 (socket says 172.21.15.46:58482) 2024-03-29T08:26:22.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f826fe00640 1 -- 172.21.15.46:0/1018083124 learned_addr learned my addr 172.21.15.46:0/1018083124 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:22.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f8275e00640 1 --2- 172.21.15.46:0/1018083124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82781593d0 0x7f82781570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f8276800640 1 --2- 172.21.15.46:0/1018083124 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f827815cd90 0x7f8278154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f826fe00640 1 -- 172.21.15.46:0/1018083124 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f827815cd90 msgr2=0x7f8278154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f826fe00640 1 --2- 172.21.15.46:0/1018083124 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f827815cd90 0x7f8278154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f826fe00640 1 -- 172.21.15.46:0/1018083124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82781593d0 msgr2=0x7f82781570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f826fe00640 1 --2- 172.21.15.46:0/1018083124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82781593d0 0x7f82781570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f826fe00640 1 -- 172.21.15.46:0/1018083124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8270009a70 con 0x7f827815a7c0 2024-03-29T08:26:22.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f826fe00640 1 --2- 172.21.15.46:0/1018083124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a7c0 0x7f82780aea60 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f826000c480 tx=0x7f826000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f826e000640 1 -- 172.21.15.46:0/1018083124 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82600074e0 con 0x7f827815a7c0 2024-03-29T08:26:22.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f826e000640 1 -- 172.21.15.46:0/1018083124 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8260007660 con 0x7f827815a7c0 2024-03-29T08:26:22.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f827cc00640 1 -- 172.21.15.46:0/1018083124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82781545e0 con 0x7f827815a7c0 2024-03-29T08:26:22.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f826e000640 1 -- 172.21.15.46:0/1018083124 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260004a30 con 0x7f827815a7c0 2024-03-29T08:26:22.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f827cc00640 1 -- 172.21.15.46:0/1018083124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82781575e0 con 0x7f827815a7c0 2024-03-29T08:26:22.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f827cc00640 1 -- 172.21.15.46:0/1018083124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8278154ac0 con 0x7f827815a7c0 2024-03-29T08:26:22.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.096+0000 7f826e000640 1 -- 172.21.15.46:0/1018083124 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8260004b90 con 0x7f827815a7c0 2024-03-29T08:26:22.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.100+0000 7f826e000640 1 --2- 172.21.15.46:0/1018083124 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8254066690 0x7f8254068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.100+0000 7f826e000640 1 -- 172.21.15.46:0/1018083124 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f82600836f0 con 0x7f827815a7c0 2024-03-29T08:26:22.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.100+0000 7f8275e00640 1 --2- 172.21.15.46:0/1018083124 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8254066690 0x7f8254068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.100+0000 7f8275e00640 1 --2- 172.21.15.46:0/1018083124 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8254066690 0x7f8254068b50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8268004e30 tx=0x7f8268009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.104+0000 7f826e000640 1 -- 172.21.15.46:0/1018083124 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8260054480 con 0x7f827815a7c0 2024-03-29T08:26:22.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.296+0000 7f827cc00640 1 -- 172.21.15.46:0/1018083124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8278066c90 con 0x7f827815a7c0 2024-03-29T08:26:22.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.300+0000 7f826e000640 1 -- 172.21.15.46:0/1018083124 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4738 (secure 0 0 0) 0x7f8260054010 con 0x7f827815a7c0 2024-03-29T08:26:22.300 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:26:22.300 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/1862652020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1862652020},{"type":"v1","addr":"172.21.15.73:6836","nonce":1862652020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:24:43.476334+0000","modified":"2024-03-29T08:25:17.015744+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":93,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7386},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7386":{"gid":7386,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":13,"addr":"172.21.15.46:6835/2051429508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":2051429508},{"type":"v1","addr":"172.21.15.46:6835","nonce":2051429508}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-29T08:26:22.301 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 63 2024-03-29T08:26:22.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.300+0000 7f827cc00640 1 -- 172.21.15.46:0/1018083124 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8254066690 msgr2=0x7f8254068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.300+0000 7f827cc00640 1 --2- 172.21.15.46:0/1018083124 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8254066690 0x7f8254068b50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8268004e30 tx=0x7f8268009040 comp rx=0 tx=0).stop 2024-03-29T08:26:22.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.300+0000 7f827cc00640 1 -- 172.21.15.46:0/1018083124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a7c0 msgr2=0x7f82780aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.300+0000 7f827cc00640 1 --2- 172.21.15.46:0/1018083124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a7c0 0x7f82780aea60 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f826000c480 tx=0x7f826000c940 comp rx=0 tx=0).stop 2024-03-29T08:26:22.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.300+0000 7f827cc00640 1 -- 172.21.15.46:0/1018083124 shutdown_connections 2024-03-29T08:26:22.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.300+0000 7f827cc00640 1 --2- 172.21.15.46:0/1018083124 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8254066690 0x7f8254068b50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.300+0000 7f827cc00640 1 --2- 172.21.15.46:0/1018083124 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f827815cd90 0x7f8278154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.300+0000 7f827cc00640 1 --2- 172.21.15.46:0/1018083124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f827815a7c0 0x7f82780aea60 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.300+0000 7f827cc00640 1 --2- 172.21.15.46:0/1018083124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82781593d0 0x7f82781570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.300+0000 7f827cc00640 1 -- 172.21.15.46:0/1018083124 >> 172.21.15.46:0/1018083124 conn(0x7f8278066110 msgr2=0x7f82780ada50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:22.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.300+0000 7f827cc00640 1 -- 172.21.15.46:0/1018083124 shutdown_connections 2024-03-29T08:26:22.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.300+0000 7f827cc00640 1 -- 172.21.15.46:0/1018083124 wait complete. 2024-03-29T08:26:22.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-03-29T08:26:22.494 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-03-29T08:26:22.495 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-29T08:26:22.495 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T08:26:22.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.636+0000 7f2686800640 1 Processor -- start 2024-03-29T08:26:22.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.636+0000 7f2686800640 1 -- start start 2024-03-29T08:26:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.636+0000 7f2686800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2680142ab0 0x7f268006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.636+0000 7f2686800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f268006de40 con 0x7f268015db10 2024-03-29T08:26:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.636+0000 7f2686800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2680070c70 con 0x7f268006d9e0 2024-03-29T08:26:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.636+0000 7f2686800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2680070340 con 0x7f2680142ab0 2024-03-29T08:26:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.636+0000 7f267fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f268015db10 0x7f26800b1e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39034/0 (socket says 172.21.15.46:39034) 2024-03-29T08:26:22.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.636+0000 7f267fe00640 1 -- 172.21.15.46:0/2116500798 learned_addr learned my addr 172.21.15.46:0/2116500798 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:26:22.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.636+0000 7f267f400640 1 --2- 172.21.15.46:0/2116500798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2680142ab0 0x7f268006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.636+0000 7f267ea00640 1 -- 172.21.15.46:0/2116500798 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 619896456 0 0) 0x7f268006de40 con 0x7f268015db10 2024-03-29T08:26:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.636+0000 7f267ea00640 1 -- 172.21.15.46:0/2116500798 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2664003530 con 0x7f268015db10 2024-03-29T08:26:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.636+0000 7f267ea00640 1 -- 172.21.15.46:0/2116500798 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690440327 0 0) 0x7f2680070c70 con 0x7f268006d9e0 2024-03-29T08:26:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.636+0000 7f267ea00640 1 -- 172.21.15.46:0/2116500798 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f268006de40 con 0x7f268006d9e0 2024-03-29T08:26:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.636+0000 7f267f400640 1 -- 172.21.15.46:0/2116500798 >> v1:172.21.15.73:6789/0 conn(0x7f268006d9e0 legacy=0x7f268015d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.636+0000 7f267f400640 1 -- 172.21.15.46:0/2116500798 >> v1:172.21.15.46:6789/0 conn(0x7f268015db10 legacy=0x7f26800b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.636+0000 7f267f400640 1 -- 172.21.15.46:0/2116500798 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2680070620 con 0x7f2680142ab0 2024-03-29T08:26:22.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.640+0000 7f267f400640 1 --2- 172.21.15.46:0/2116500798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2680142ab0 0x7f268006d4a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2670009dc0 tx=0x7f267001c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9553e88c1a43412f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.640+0000 7f267ea00640 1 -- 172.21.15.46:0/2116500798 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26700a5650 con 0x7f2680142ab0 2024-03-29T08:26:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.640+0000 7f267ea00640 1 -- 172.21.15.46:0/2116500798 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26700a57b0 con 0x7f2680142ab0 2024-03-29T08:26:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.640+0000 7f267ea00640 1 -- 172.21.15.46:0/2116500798 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26700a5b40 con 0x7f2680142ab0 2024-03-29T08:26:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.640+0000 7f2686800640 1 -- 172.21.15.46:0/2116500798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2680142ab0 msgr2=0x7f268006d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.640+0000 7f2686800640 1 --2- 172.21.15.46:0/2116500798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2680142ab0 0x7f268006d4a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2670009dc0 tx=0x7f267001c990 comp rx=0 tx=0).stop 2024-03-29T08:26:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.640+0000 7f2686800640 1 -- 172.21.15.46:0/2116500798 shutdown_connections 2024-03-29T08:26:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.640+0000 7f2686800640 1 --2- 172.21.15.46:0/2116500798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2680142ab0 0x7f268006d4a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.640+0000 7f2686800640 1 -- 172.21.15.46:0/2116500798 >> 172.21.15.46:0/2116500798 conn(0x7f2680066080 msgr2=0x7f2680069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:22.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.640+0000 7f2686800640 1 -- 172.21.15.46:0/2116500798 shutdown_connections 2024-03-29T08:26:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.640+0000 7f2686800640 1 -- 172.21.15.46:0/2116500798 wait complete. 2024-03-29T08:26:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f2686800640 1 Processor -- start 2024-03-29T08:26:22.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f2686800640 1 -- start start 2024-03-29T08:26:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f2686800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f268006d9e0 0x7f26801591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f2686800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2680142ab0 0x7f26801418d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f2686800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268015db10 0x7f2680140f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f2686800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26801414a0 con 0x7f268015db10 2024-03-29T08:26:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f2686800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268015efd0 con 0x7f268006d9e0 2024-03-29T08:26:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f2686800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268015e3c0 con 0x7f2680142ab0 2024-03-29T08:26:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f267fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268015db10 0x7f2680140f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f2676a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2680142ab0 0x7f26801418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f267f400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f268006d9e0 0x7f26801591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f267fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268015db10 0x7f2680140f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58492/0 (socket says 172.21.15.46:58492) 2024-03-29T08:26:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f267fe00640 1 -- 172.21.15.46:0/1906158663 learned_addr learned my addr 172.21.15.46:0/1906158663 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f267f400640 1 -- 172.21.15.46:0/1906158663 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2680142ab0 msgr2=0x7f26801418d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f267f400640 1 --2- 172.21.15.46:0/1906158663 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2680142ab0 0x7f26801418d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f267f400640 1 -- 172.21.15.46:0/1906158663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268015db10 msgr2=0x7f2680140f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f267f400640 1 --2- 172.21.15.46:0/1906158663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268015db10 0x7f2680140f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f267f400640 1 -- 172.21.15.46:0/1906158663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2670009a70 con 0x7f268006d9e0 2024-03-29T08:26:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f267fe00640 1 --2- 172.21.15.46:0/1906158663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268015db10 0x7f2680140f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:26:22.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f267f400640 1 --2- 172.21.15.46:0/1906158663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f268006d9e0 0x7f26801591d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f26700ac000 tx=0x7f2670019730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f2677e00640 1 -- 172.21.15.46:0/1906158663 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26700a5650 con 0x7f268006d9e0 2024-03-29T08:26:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f2677e00640 1 -- 172.21.15.46:0/1906158663 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26700b9860 con 0x7f268006d9e0 2024-03-29T08:26:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f2677e00640 1 -- 172.21.15.46:0/1906158663 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26700b9bf0 con 0x7f268006d9e0 2024-03-29T08:26:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f2686800640 1 -- 172.21.15.46:0/1906158663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268015e6a0 con 0x7f268006d9e0 2024-03-29T08:26:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.644+0000 7f2686800640 1 -- 172.21.15.46:0/1906158663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268015ec60 con 0x7f268006d9e0 2024-03-29T08:26:22.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.648+0000 7f2686800640 1 -- 172.21.15.46:0/1906158663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2680071a90 con 0x7f268006d9e0 2024-03-29T08:26:22.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.648+0000 7f2677e00640 1 -- 172.21.15.46:0/1906158663 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f26700b9d50 con 0x7f268006d9e0 2024-03-29T08:26:22.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.648+0000 7f2677e00640 1 --2- 172.21.15.46:0/1906158663 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f265c066690 0x7f265c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.648+0000 7f2677e00640 1 -- 172.21.15.46:0/1906158663 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f267011fd80 con 0x7f268006d9e0 2024-03-29T08:26:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.648+0000 7f2676a00640 1 --2- 172.21.15.46:0/1906158663 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f265c066690 0x7f265c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.648+0000 7f2676a00640 1 --2- 172.21.15.46:0/1906158663 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f265c066690 0x7f265c068b50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2680140ef0 tx=0x7f2668009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.652+0000 7f2677e00640 1 -- 172.21.15.46:0/1906158663 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f26700ed110 con 0x7f268006d9e0 2024-03-29T08:26:22.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.848+0000 7f2686800640 1 -- 172.21.15.46:0/1906158663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f26801416b0 con 0x7f268006d9e0 2024-03-29T08:26:22.850 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-03-29T08:26:22.848+0000 7fc16ae00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:26:22.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.860+0000 7f2677e00640 1 -- 172.21.15.46:0/1906158663 <== mon.1 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) v1 ==== 161+0+0 (secure 0 0 0) 0x7f26700b2020 con 0x7f268006d9e0 2024-03-29T08:26:22.862 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T08:26:22.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.860+0000 7f2686800640 1 -- 172.21.15.46:0/1906158663 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f265c066690 msgr2=0x7f265c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.860+0000 7f2686800640 1 --2- 172.21.15.46:0/1906158663 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f265c066690 0x7f265c068b50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2680140ef0 tx=0x7f2668009040 comp rx=0 tx=0).stop 2024-03-29T08:26:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.860+0000 7f2686800640 1 -- 172.21.15.46:0/1906158663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f268006d9e0 msgr2=0x7f26801591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.860+0000 7f2686800640 1 --2- 172.21.15.46:0/1906158663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f268006d9e0 0x7f26801591d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f26700ac000 tx=0x7f2670019730 comp rx=0 tx=0).stop 2024-03-29T08:26:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.860+0000 7f2686800640 1 -- 172.21.15.46:0/1906158663 shutdown_connections 2024-03-29T08:26:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.860+0000 7f2686800640 1 --2- 172.21.15.46:0/1906158663 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f265c066690 0x7f265c068b50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.860+0000 7f2686800640 1 --2- 172.21.15.46:0/1906158663 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f268015db10 0x7f2680140f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.860+0000 7f2686800640 1 --2- 172.21.15.46:0/1906158663 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2680142ab0 0x7f26801418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.860+0000 7f2686800640 1 --2- 172.21.15.46:0/1906158663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f268006d9e0 0x7f26801591d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.860+0000 7f2686800640 1 -- 172.21.15.46:0/1906158663 >> 172.21.15.46:0/1906158663 conn(0x7f2680066080 msgr2=0x7f26800cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.864+0000 7f2686800640 1 -- 172.21.15.46:0/1906158663 shutdown_connections 2024-03-29T08:26:22.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.864+0000 7f2686800640 1 -- 172.21.15.46:0/1906158663 wait complete. 2024-03-29T08:26:22.871 INFO:tasks.ceph.mds.c.smithi046.stderr: -4250> 2024-03-29T08:25:09.069+0000 7f279e400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:26:22.871 INFO:tasks.ceph.mds.c.smithi046.stderr: -4072> 2024-03-29T08:25:09.365+0000 7f2799e00640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-03-29T08:26:22.871 INFO:tasks.ceph.mds.c.smithi046.stderr: -3906> 2024-03-29T08:25:09.593+0000 7f2799e00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-03-29T08:24:51.226697+0000 1=1+0) n(v0 rc2024-03-29T08:24:51.337857+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x563bf804a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-29T08:24:51.226697+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-29T08:24:51.337857+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-03-29T08:26:22.875 INFO:tasks.ceph.mds.c.smithi046.stderr: -2310> 2024-03-29T08:25:41.180+0000 7f2799e00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-03-29T08:24:51.226697+0000 1=1+0)->f(v1 m2024-03-29T08:24:51.226697+0000) n(v0 rc2024-03-29T08:24:51.337857+0000 2=1+1)->n(v0 rc2024-03-29T08:24:51.337857+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x563bf804a100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-03-29T08:24:51.226697+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-03-29T08:24:51.337857+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-03-29T08:26:22.885 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-03-29T08:26:22.920 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-03-29T08:26:22.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa739f848c0 1 -- start start 2024-03-29T08:26:22.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa739f848c0 1 Processor -- start 2024-03-29T08:26:22.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa739f848c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567a1179800 0x5567a1204580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa739f848c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567a1179400 0x5567a1204b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa739f848c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a1179000 0x5567a1205080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa739f848c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a1246a00 con 0x5567a1179000 2024-03-29T08:26:22.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa739f848c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a1246780 con 0x5567a1179800 2024-03-29T08:26:22.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa739f848c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567a1246640 con 0x5567a1179400 2024-03-29T08:26:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa738a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a1179000 0x5567a1205080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa738a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a1179000 0x5567a1205080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58520/0 (socket says 172.21.15.46:58520) 2024-03-29T08:26:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa738a00640 1 -- 172.21.15.46:0/4112159344 learned_addr learned my addr 172.21.15.46:0/4112159344 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa737600640 1 --2- 172.21.15.46:0/4112159344 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567a1179400 0x5567a1204b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa738000640 1 --2- 172.21.15.46:0/4112159344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567a1179800 0x5567a1204580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa738a00640 1 -- 172.21.15.46:0/4112159344 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567a1179400 msgr2=0x5567a1204b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa738a00640 1 --2- 172.21.15.46:0/4112159344 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567a1179400 0x5567a1204b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa738a00640 1 -- 172.21.15.46:0/4112159344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567a1179800 msgr2=0x5567a1204580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa738a00640 1 --2- 172.21.15.46:0/4112159344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567a1179800 0x5567a1204580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa738a00640 1 -- 172.21.15.46:0/4112159344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567a11ac9c0 con 0x5567a1179000 2024-03-29T08:26:22.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa738a00640 1 --2- 172.21.15.46:0/4112159344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a1179000 0x5567a1205080 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x5567a1fe3350 tx=0x5567a1f3eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a27abcf7c638705 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa734e00640 1 -- 172.21.15.46:0/4112159344 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567a11cd600 con 0x5567a1179000 2024-03-29T08:26:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa734e00640 1 -- 172.21.15.46:0/4112159344 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5567a11d6480 con 0x5567a1179000 2024-03-29T08:26:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa734e00640 1 -- 172.21.15.46:0/4112159344 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567a11cd1e0 con 0x5567a1179000 2024-03-29T08:26:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa739f848c0 1 -- 172.21.15.46:0/4112159344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567a11acb60 con 0x5567a1179000 2024-03-29T08:26:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa739f848c0 1 -- 172.21.15.46:0/4112159344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5567a11ac340 con 0x5567a1179000 2024-03-29T08:26:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa739f848c0 4 waiting for MDS map... 2024-03-29T08:26:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa734e00640 1 -- 172.21.15.46:0/4112159344 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x5567a11b8c40 con 0x5567a1179000 2024-03-29T08:26:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa734e00640 1 -- 172.21.15.46:0/4112159344 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 64) v1 ==== 2573+0+0 (secure 0 0 0) 0x5567a11c6f00 con 0x5567a1179000 2024-03-29T08:26:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa739f848c0 4 Got MDS map 64 2024-03-29T08:26:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa739f848c0 4 datascan.main: Using metadata pool output 2024-03-29T08:26:22.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.932+0000 7fa739f848c0 4 datascan.main: connecting to RADOS... 2024-03-29T08:26:22.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 Processor -- start 2024-03-29T08:26:22.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 -- start start 2024-03-29T08:26:22.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567a2088c00 0x5567a1205600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a2089000 0x5567a1205b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567a2089400 0x5567a1206100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a11bfe00 con 0x5567a2089000 2024-03-29T08:26:22.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a1246780 con 0x5567a2089400 2024-03-29T08:26:22.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567a1246500 con 0x5567a2088c00 2024-03-29T08:26:22.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa738000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a2089000 0x5567a1205b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa737600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567a2089400 0x5567a1206100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa738000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a2089000 0x5567a1205b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58530/0 (socket says 172.21.15.46:58530) 2024-03-29T08:26:22.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa737600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567a2089400 0x5567a1206100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:34546/0 (socket says 172.21.15.46:34546) 2024-03-29T08:26:22.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa738000640 1 -- 172.21.15.46:0/4285505274 learned_addr learned my addr 172.21.15.46:0/4285505274 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:22.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa737600640 1 --2- 172.21.15.46:0/4285505274 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567a2088c00 0x5567a1205600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa738000640 1 -- 172.21.15.46:0/4285505274 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567a2088c00 msgr2=0x5567a1205600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:22.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa738000640 1 --2- 172.21.15.46:0/4285505274 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567a2088c00 0x5567a1205600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa738000640 1 -- 172.21.15.46:0/4285505274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567a2089400 msgr2=0x5567a1206100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:22.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa738000640 1 --2- 172.21.15.46:0/4285505274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567a2089400 0x5567a1206100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa738000640 1 -- 172.21.15.46:0/4285505274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567a11ac000 con 0x5567a2089000 2024-03-29T08:26:22.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa738000640 1 --2- 172.21.15.46:0/4285505274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a2089000 0x5567a1205b80 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x5567a1fe29f0 tx=0x5567a1460000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed58cf6b0dd85ae server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa732600640 1 -- 172.21.15.46:0/4285505274 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567a11cd080 con 0x5567a2089000 2024-03-29T08:26:22.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa732600640 1 -- 172.21.15.46:0/4285505274 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5567a1f52900 con 0x5567a2089000 2024-03-29T08:26:22.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa732600640 1 -- 172.21.15.46:0/4285505274 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567a11cd600 con 0x5567a2089000 2024-03-29T08:26:22.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 -- 172.21.15.46:0/4285505274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a2089000 msgr2=0x5567a1205b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 --2- 172.21.15.46:0/4285505274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a2089000 0x5567a1205b80 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x5567a1fe29f0 tx=0x5567a1460000 comp rx=0 tx=0).stop 2024-03-29T08:26:22.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 -- 172.21.15.46:0/4285505274 shutdown_connections 2024-03-29T08:26:22.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 --2- 172.21.15.46:0/4285505274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567a2089400 0x5567a1206100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 --2- 172.21.15.46:0/4285505274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a2089000 0x5567a1205b80 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 --2- 172.21.15.46:0/4285505274 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567a2088c00 0x5567a1205600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 -- 172.21.15.46:0/4285505274 >> 172.21.15.46:0/4285505274 conn(0x5567a1178c00 msgr2=0x5567a11a5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 -- 172.21.15.46:0/4285505274 shutdown_connections 2024-03-29T08:26:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 -- 172.21.15.46:0/4285505274 wait complete. 2024-03-29T08:26:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.936+0000 7fa739f848c0 1 Processor -- start 2024-03-29T08:26:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa739f848c0 1 -- start start 2024-03-29T08:26:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa739f848c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a2088c00 0x5567a1207180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa739f848c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567a2089000 0x5567a1206c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa739f848c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567a2089400 0x5567a1206680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa739f848c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a12463c0 con 0x5567a2088c00 2024-03-29T08:26:22.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa739f848c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567a1246280 con 0x5567a2089400 2024-03-29T08:26:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa739f848c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567a1246140 con 0x5567a2089000 2024-03-29T08:26:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa737600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a2088c00 0x5567a1207180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa737600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a2088c00 0x5567a1207180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58540/0 (socket says 172.21.15.46:58540) 2024-03-29T08:26:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 learned_addr learned my addr 172.21.15.46:0/562684292 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:22.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa738000640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567a2089000 0x5567a1206c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa737600640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567a2089400 0x5567a1206680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567a2089400 msgr2=0x5567a1206680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa738000640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567a2089400 0x5567a1206680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a2088c00 msgr2=0x5567a1207180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa738000640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a2088c00 0x5567a1207180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567a11ac1a0 con 0x5567a2089000 2024-03-29T08:26:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa737600640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a2088c00 0x5567a1207180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:26:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa738000640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567a2089000 0x5567a1206c00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5567a209e750 tx=0x5567a1460180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa730800640 1 -- 172.21.15.46:0/562684292 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567a11cd1e0 con 0x5567a2089000 2024-03-29T08:26:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa730800640 1 -- 172.21.15.46:0/562684292 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5567a1f52d80 con 0x5567a2089000 2024-03-29T08:26:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5567a11ac000 con 0x5567a2089000 2024-03-29T08:26:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa730800640 1 -- 172.21.15.46:0/562684292 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567a11ccdc0 con 0x5567a2089000 2024-03-29T08:26:22.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567a11ac9c0 con 0x5567a2089000 2024-03-29T08:26:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa739f848c0 4 datascan.init: resolving metadata pool 12 2024-03-29T08:26:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.940+0000 7fa730800640 1 -- 172.21.15.46:0/562684292 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x5567a11b1080 con 0x5567a2089000 2024-03-29T08:26:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa730800640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5567a1470400 0x5567a1207700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa730800640 1 -- 172.21.15.46:0/562684292 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x5567a11b8a80 con 0x5567a2089000 2024-03-29T08:26:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa739f848c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-29T08:26:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa739f848c0 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-03-29T08:26:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa739f848c0 4 datascan.main: opening data pool 'cephfs_data' 2024-03-29T08:26:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa739f848c0 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5567a2089800 0x5567a1206100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a2089c00 con 0x5567a2089800 2024-03-29T08:26:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa737600640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5567a1470400 0x5567a1207700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa737600640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5567a2089800 0x5567a1206100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa737600640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5567a2089800 0x5567a1206100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa737600640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5567a1470400 0x5567a1207700 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5567a209f3e0 tx=0x5567a1461da0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.1 v2:172.21.15.46:6824/27427 1 ==== osd_op_reply(1 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a2089800 2024-03-29T08:26:22.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa731c00640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5567a1471000 0x5567a155e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:2 13.20 13:04000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a1471400 con 0x5567a1471000 2024-03-29T08:26:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa738a00640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5567a1471000 0x5567a155e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa738a00640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5567a1471000 0x5567a155e000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.2 v2:172.21.15.46:6809/27428 1 ==== osd_op_reply(2 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a1471000 2024-03-29T08:26:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa732600640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5567a1471c00 0x5567a155e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:3 13.10 13:08000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a156c000 con 0x5567a1471c00 2024-03-29T08:26:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa738000640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5567a1471c00 0x5567a155e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.944+0000 7fa738000640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5567a1471c00 0x5567a155e580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 <== osd.7 v2:172.21.15.73:6816/27700 1 ==== osd_op_reply(3 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cfc0 con 0x5567a1471c00 2024-03-29T08:26:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:4 13.30 13:0c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a1471800 con 0x5567a1471000 2024-03-29T08:26:22.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.2 v2:172.21.15.46:6809/27428 2 ==== osd_op_reply(4 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a1471000 2024-03-29T08:26:22.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa732600640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x5567a156c400 0x5567a155eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:5 13.8 13:10000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a156c800 con 0x5567a156c400 2024-03-29T08:26:22.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa737600640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x5567a156c400 0x5567a155eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa737600640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x5567a156c400 0x5567a155eb00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.5 v2:172.21.15.73:6808/27690 1 ==== osd_op_reply(5 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a156c400 2024-03-29T08:26:22.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa731c00640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5567a156d400 0x5567a155f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:6 13.28 13:14000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a156d800 con 0x5567a156d400 2024-03-29T08:26:22.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa738a00640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5567a156d400 0x5567a155f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa738a00640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5567a156d400 0x5567a155f600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.6 v2:172.21.15.73:6824/27691 1 ==== osd_op_reply(6 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a156d400 2024-03-29T08:26:22.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:7 13.18 13:18000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a156cc00 con 0x5567a2089800 2024-03-29T08:26:22.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.1 v2:172.21.15.46:6824/27427 2 ==== osd_op_reply(7 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a2089800 2024-03-29T08:26:22.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa731c00640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5567a156dc00 0x5567a155fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:8 13.38 13:1c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a1471400 con 0x5567a156dc00 2024-03-29T08:26:22.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.948+0000 7fa738000640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5567a156dc00 0x5567a155fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa738000640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5567a156dc00 0x5567a155fb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 <== osd.0 v2:172.21.15.46:6800/27424 1 ==== osd_op_reply(8 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cfc0 con 0x5567a156dc00 2024-03-29T08:26:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa732600640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5567a156d000 0x5567a155f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:9 13.4 13:20000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a156d800 con 0x5567a156d000 2024-03-29T08:26:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa737600640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5567a156d000 0x5567a155f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa737600640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5567a156d000 0x5567a155f080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.3 v2:172.21.15.46:6808/27426 1 ==== osd_op_reply(9 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a156d000 2024-03-29T08:26:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:10 13.24 13:24000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a1471800 con 0x5567a2089800 2024-03-29T08:26:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.1 v2:172.21.15.46:6824/27427 3 ==== osd_op_reply(10 [pgnls start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a2089800 2024-03-29T08:26:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:11 13.14 13:28000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a156c800 con 0x5567a156d400 2024-03-29T08:26:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.6 v2:172.21.15.73:6824/27691 2 ==== osd_op_reply(11 [pgnls start_epoch 94 out=49b] v92'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a156d400 2024-03-29T08:26:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:12 13.34 13:2c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a2088800 con 0x5567a156dc00 2024-03-29T08:26:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 <== osd.0 v2:172.21.15.46:6800/27424 2 ==== osd_op_reply(12 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cfc0 con 0x5567a156dc00 2024-03-29T08:26:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:13 13.c 13:30000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a156cc00 con 0x5567a156d400 2024-03-29T08:26:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.6 v2:172.21.15.73:6824/27691 3 ==== osd_op_reply(13 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a156d400 2024-03-29T08:26:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:14 13.2c 13:34000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a2088400 con 0x5567a1471c00 2024-03-29T08:26:22.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 <== osd.7 v2:172.21.15.73:6816/27700 2 ==== osd_op_reply(14 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cfc0 con 0x5567a1471c00 2024-03-29T08:26:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:15 13.1c 13:38000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a156d800 con 0x5567a156dc00 2024-03-29T08:26:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 <== osd.0 v2:172.21.15.46:6800/27424 3 ==== osd_op_reply(15 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cfc0 con 0x5567a156dc00 2024-03-29T08:26:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.952+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:16 13.3c 13:3c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a2088800 con 0x5567a156c400 2024-03-29T08:26:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.5 v2:172.21.15.73:6808/27690 2 ==== osd_op_reply(16 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a156c400 2024-03-29T08:26:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:17 13.2 13:40000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a2089c00 con 0x5567a2089800 2024-03-29T08:26:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.1 v2:172.21.15.46:6824/27427 4 ==== osd_op_reply(17 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a2089800 2024-03-29T08:26:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:18 13.22 13:44000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a2088400 con 0x5567a1471c00 2024-03-29T08:26:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 <== osd.7 v2:172.21.15.73:6816/27700 3 ==== osd_op_reply(18 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cfc0 con 0x5567a1471c00 2024-03-29T08:26:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:19 13.12 13:48000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157e000 con 0x5567a156d400 2024-03-29T08:26:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.6 v2:172.21.15.73:6824/27691 4 ==== osd_op_reply(19 [pgnls start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a156d400 2024-03-29T08:26:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:20 13.32 13:4c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a1471400 con 0x5567a1471c00 2024-03-29T08:26:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 <== osd.7 v2:172.21.15.73:6816/27700 4 ==== osd_op_reply(20 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cfc0 con 0x5567a1471c00 2024-03-29T08:26:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa732600640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x5567a157e400 0x5567a1560100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:22.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:21 13.a 13:50000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157e800 con 0x5567a157e400 2024-03-29T08:26:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa738a00640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x5567a157e400 0x5567a1560100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa738a00640 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x5567a157e400 0x5567a1560100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.4 v2:172.21.15.73:6800/27689 1 ==== osd_op_reply(21 [pgnls start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a157e400 2024-03-29T08:26:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:22 13.2a 13:54000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a156c000 con 0x5567a1471c00 2024-03-29T08:26:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 <== osd.7 v2:172.21.15.73:6816/27700 5 ==== osd_op_reply(22 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cfc0 con 0x5567a1471c00 2024-03-29T08:26:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:23 13.1a 13:58000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157ec00 con 0x5567a156c400 2024-03-29T08:26:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.5 v2:172.21.15.73:6808/27690 3 ==== osd_op_reply(23 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a156c400 2024-03-29T08:26:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:24 13.3a 13:5c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a1f3b400 con 0x5567a1471000 2024-03-29T08:26:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.2 v2:172.21.15.46:6809/27428 3 ==== osd_op_reply(24 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a1471000 2024-03-29T08:26:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.956+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:25 13.6 13:60000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157f000 con 0x5567a156d000 2024-03-29T08:26:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.3 v2:172.21.15.46:6808/27426 2 ==== osd_op_reply(25 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a156d000 2024-03-29T08:26:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:26 13.26 13:64000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157ec00 con 0x5567a156d000 2024-03-29T08:26:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.3 v2:172.21.15.46:6808/27426 3 ==== osd_op_reply(26 [pgnls start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a156d000 2024-03-29T08:26:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:27 13.16 13:68000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157f400 con 0x5567a1471c00 2024-03-29T08:26:22.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 <== osd.7 v2:172.21.15.73:6816/27700 6 ==== osd_op_reply(27 [pgnls start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cfc0 con 0x5567a1471c00 2024-03-29T08:26:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:28 13.36 13:6c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157f000 con 0x5567a1471000 2024-03-29T08:26:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.2 v2:172.21.15.46:6809/27428 4 ==== osd_op_reply(28 [pgnls start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a1471000 2024-03-29T08:26:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:29 13.e 13:70000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157e000 con 0x5567a156dc00 2024-03-29T08:26:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 <== osd.0 v2:172.21.15.46:6800/27424 4 ==== osd_op_reply(29 [pgnls start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cfc0 con 0x5567a156dc00 2024-03-29T08:26:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:30 13.2e 13:74000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157f800 con 0x5567a156d400 2024-03-29T08:26:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.6 v2:172.21.15.73:6824/27691 5 ==== osd_op_reply(30 [pgnls start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a156d400 2024-03-29T08:26:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:31 13.1e 13:78000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157e800 con 0x5567a156dc00 2024-03-29T08:26:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 <== osd.0 v2:172.21.15.46:6800/27424 5 ==== osd_op_reply(31 [pgnls start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cfc0 con 0x5567a156dc00 2024-03-29T08:26:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:32 13.3e 13:7c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a2088800 con 0x5567a157e400 2024-03-29T08:26:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.4 v2:172.21.15.73:6800/27689 2 ==== osd_op_reply(32 [pgnls start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a157e400 2024-03-29T08:26:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:33 13.1 13:80000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157f000 con 0x5567a2089800 2024-03-29T08:26:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.1 v2:172.21.15.46:6824/27427 5 ==== osd_op_reply(33 [pgnls start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a2089800 2024-03-29T08:26:22.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:34 13.21 13:84000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a2089c00 con 0x5567a1471000 2024-03-29T08:26:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.2 v2:172.21.15.46:6809/27428 5 ==== osd_op_reply(34 [pgnls start_epoch 94 out=49b] v92'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a1471000 2024-03-29T08:26:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:35 13.11 13:88000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157fc00 con 0x5567a156d400 2024-03-29T08:26:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.6 v2:172.21.15.73:6824/27691 6 ==== osd_op_reply(35 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a156d400 2024-03-29T08:26:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.960+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:36 13.31 13:8c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a1471800 con 0x5567a157e400 2024-03-29T08:26:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.4 v2:172.21.15.73:6800/27689 3 ==== osd_op_reply(36 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a157e400 2024-03-29T08:26:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:37 13.9 13:90000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a1f3b400 con 0x5567a156dc00 2024-03-29T08:26:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 <== osd.0 v2:172.21.15.46:6800/27424 6 ==== osd_op_reply(37 [pgnls start_epoch 94 out=49b] v92'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cfc0 con 0x5567a156dc00 2024-03-29T08:26:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:38 13.29 13:94000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157f400 con 0x5567a156dc00 2024-03-29T08:26:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 <== osd.0 v2:172.21.15.46:6800/27424 7 ==== osd_op_reply(38 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cfc0 con 0x5567a156dc00 2024-03-29T08:26:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:39 13.19 13:98000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a156c800 con 0x5567a1471c00 2024-03-29T08:26:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 <== osd.7 v2:172.21.15.73:6816/27700 7 ==== osd_op_reply(39 [pgnls start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cfc0 con 0x5567a1471c00 2024-03-29T08:26:22.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:40 13.39 13:9c000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157e000 con 0x5567a157e400 2024-03-29T08:26:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.4 v2:172.21.15.73:6800/27689 4 ==== osd_op_reply(40 [pgnls start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a157e400 2024-03-29T08:26:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:41 13.5 13:a0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a156cc00 con 0x5567a1471000 2024-03-29T08:26:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.2 v2:172.21.15.46:6809/27428 6 ==== osd_op_reply(41 [pgnls start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a1471000 2024-03-29T08:26:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:42 13.25 13:a4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a156d800 con 0x5567a156d000 2024-03-29T08:26:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.3 v2:172.21.15.46:6808/27426 4 ==== osd_op_reply(42 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a156d000 2024-03-29T08:26:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:43 13.15 13:a8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a156cc00 con 0x5567a156d000 2024-03-29T08:26:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.3 v2:172.21.15.46:6808/27426 5 ==== osd_op_reply(43 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a156d000 2024-03-29T08:26:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:44 13.35 13:ac000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a156c000 con 0x5567a156d400 2024-03-29T08:26:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.6 v2:172.21.15.73:6824/27691 7 ==== osd_op_reply(44 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a156d400 2024-03-29T08:26:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:45 13.d 13:b0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157f800 con 0x5567a156dc00 2024-03-29T08:26:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 <== osd.0 v2:172.21.15.46:6800/27424 8 ==== osd_op_reply(45 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cfc0 con 0x5567a156dc00 2024-03-29T08:26:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:46 13.2d 13:b4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a1471400 con 0x5567a1471c00 2024-03-29T08:26:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 <== osd.7 v2:172.21.15.73:6816/27700 8 ==== osd_op_reply(46 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cfc0 con 0x5567a1471c00 2024-03-29T08:26:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:47 13.1d 13:b8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157fc00 con 0x5567a2089800 2024-03-29T08:26:22.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.1 v2:172.21.15.46:6824/27427 6 ==== osd_op_reply(47 [pgnls start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a2089800 2024-03-29T08:26:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.964+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:48 13.3d 13:bc000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a2088400 con 0x5567a1471000 2024-03-29T08:26:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.2 v2:172.21.15.46:6809/27428 7 ==== osd_op_reply(48 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a1471000 2024-03-29T08:26:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:49 13.3 13:c0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157e000 con 0x5567a156d400 2024-03-29T08:26:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.6 v2:172.21.15.73:6824/27691 8 ==== osd_op_reply(49 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a156d400 2024-03-29T08:26:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:50 13.23 13:c4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a158c000 con 0x5567a156c400 2024-03-29T08:26:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.5 v2:172.21.15.73:6808/27690 4 ==== osd_op_reply(50 [pgnls start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a156c400 2024-03-29T08:26:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:51 13.13 13:c8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a1471800 con 0x5567a2089800 2024-03-29T08:26:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.1 v2:172.21.15.46:6824/27427 7 ==== osd_op_reply(51 [pgnls start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a2089800 2024-03-29T08:26:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:52 13.33 13:cc000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a158c400 con 0x5567a156d000 2024-03-29T08:26:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.3 v2:172.21.15.46:6808/27426 6 ==== osd_op_reply(52 [pgnls start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a156d000 2024-03-29T08:26:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:53 13.b 13:d0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a2088800 con 0x5567a156d000 2024-03-29T08:26:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.3 v2:172.21.15.46:6808/27426 7 ==== osd_op_reply(53 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a156d000 2024-03-29T08:26:22.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:54 13.2b 13:d4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a158c800 con 0x5567a156c400 2024-03-29T08:26:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.5 v2:172.21.15.73:6808/27690 5 ==== osd_op_reply(54 [pgnls start_epoch 94 out=49b] v92'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a156c400 2024-03-29T08:26:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:55 13.1b 13:d8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a2089c00 con 0x5567a1471000 2024-03-29T08:26:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.2 v2:172.21.15.46:6809/27428 8 ==== osd_op_reply(55 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a1471000 2024-03-29T08:26:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:56 13.3b 13:dc000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a158cc00 con 0x5567a156d000 2024-03-29T08:26:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.3 v2:172.21.15.46:6808/27426 8 ==== osd_op_reply(56 [pgnls start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a156d000 2024-03-29T08:26:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:57 13.7 13:e0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a158c000 con 0x5567a156d000 2024-03-29T08:26:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.3 v2:172.21.15.46:6808/27426 9 ==== osd_op_reply(57 [pgnls start_epoch 94 out=81b] v92'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5567a207cb40 con 0x5567a156d000 2024-03-29T08:26:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:58 13.7 13:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a158e000 con 0x5567a156d000 2024-03-29T08:26:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.3 v2:172.21.15.46:6808/27426 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5567a207cb40 con 0x5567a156d000 2024-03-29T08:26:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa739f848c0 10 datascan.operator(): handling object 1099511627777.0 2024-03-29T08:26:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.968+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:59 13.7 13:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a158e400 con 0x5567a156d000 2024-03-29T08:26:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.3 v2:172.21.15.46:6808/27426 11 ==== osd_op_reply(59 10000000001.00000000 [call] v94'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5567a207cb40 con 0x5567a156d000 2024-03-29T08:26:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:60 13.27 13:e4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a158e800 con 0x5567a156d400 2024-03-29T08:26:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.6 v2:172.21.15.73:6824/27691 9 ==== osd_op_reply(60 [pgnls start_epoch 94 out=49b] v92'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a156d400 2024-03-29T08:26:22.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:61 13.17 13:e8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157ec00 con 0x5567a2089800 2024-03-29T08:26:22.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.1 v2:172.21.15.46:6824/27427 8 ==== osd_op_reply(61 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a2089800 2024-03-29T08:26:22.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:62 13.37 13:ec000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a158d000 con 0x5567a2089800 2024-03-29T08:26:22.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.1 v2:172.21.15.46:6824/27427 9 ==== osd_op_reply(62 [pgnls start_epoch 94 out=49b] v92'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a2089800 2024-03-29T08:26:22.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:63 13.f 13:f0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157f000 con 0x5567a1471000 2024-03-29T08:26:22.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa738a00640 1 -- 172.21.15.46:0/562684292 <== osd.2 v2:172.21.15.46:6809/27428 9 ==== osd_op_reply(63 [pgnls start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cd80 con 0x5567a1471000 2024-03-29T08:26:22.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:64 13.2f 13:f4000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a158d400 con 0x5567a156dc00 2024-03-29T08:26:22.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 <== osd.0 v2:172.21.15.46:6800/27424 9 ==== osd_op_reply(64 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cfc0 con 0x5567a156dc00 2024-03-29T08:26:22.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa732600640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:65 13.1f 13:f8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a157fc00 con 0x5567a2089800 2024-03-29T08:26:22.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.1 v2:172.21.15.46:6824/27427 10 ==== osd_op_reply(65 [pgnls start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a2089800 2024-03-29T08:26:22.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa731c00640 1 -- 172.21.15.46:0/562684292 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:66 13.3f 13:fc000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567a158d800 con 0x5567a2089800 2024-03-29T08:26:22.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa737600640 1 -- 172.21.15.46:0/562684292 <== osd.1 v2:172.21.15.46:6824/27427 11 ==== osd_op_reply(66 [pgnls start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567a207cb40 con 0x5567a2089800 2024-03-29T08:26:22.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5567a156dc00 msgr2=0x5567a155fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5567a156dc00 0x5567a155fb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5567a2089800 msgr2=0x5567a1206100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5567a2089800 0x5567a1206100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5567a1471000 msgr2=0x5567a155e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5567a1471000 0x5567a155e000 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5567a156d000 msgr2=0x5567a155f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5567a156d000 0x5567a155f080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x5567a157e400 msgr2=0x5567a1560100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x5567a157e400 0x5567a1560100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x5567a156c400 msgr2=0x5567a155eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x5567a156c400 0x5567a155eb00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 reap_dead start 2024-03-29T08:26:22.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5567a156d400 msgr2=0x5567a155f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5567a156d400 0x5567a155f600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5567a1471c00 msgr2=0x5567a155e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5567a1471c00 0x5567a155e580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 reap_dead start 2024-03-29T08:26:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa738000640 1 -- 172.21.15.46:0/562684292 reap_dead start 2024-03-29T08:26:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5567a1470400 msgr2=0x5567a1207700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5567a1470400 0x5567a1207700 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5567a209f3e0 tx=0x5567a1461da0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567a2089000 msgr2=0x5567a1206c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567a2089000 0x5567a1206c00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5567a209e750 tx=0x5567a1460180 comp rx=0 tx=0).stop 2024-03-29T08:26:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 shutdown_connections 2024-03-29T08:26:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5567a1470400 0x5567a1207700 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 --2- 172.21.15.46:0/562684292 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567a2089000 0x5567a1206c00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 >> 172.21.15.46:0/562684292 conn(0x5567a1178c00 msgr2=0x5567a11a5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 shutdown_connections 2024-03-29T08:26:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/562684292 wait complete. 2024-03-29T08:26:22.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/4112159344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a1179000 msgr2=0x5567a1205080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:22.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 --2- 172.21.15.46:0/4112159344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a1179000 0x5567a1205080 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x5567a1fe3350 tx=0x5567a1f3eba0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/4112159344 shutdown_connections 2024-03-29T08:26:22.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 --2- 172.21.15.46:0/4112159344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567a1179000 0x5567a1205080 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 --2- 172.21.15.46:0/4112159344 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567a1179400 0x5567a1204b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 --2- 172.21.15.46:0/4112159344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567a1179800 0x5567a1204580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:22.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.972+0000 7fa739f848c0 1 -- 172.21.15.46:0/4112159344 >> 172.21.15.46:0/4112159344 conn(0x5567a1179c00 msgr2=0x5567a11a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:22.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.976+0000 7fa739f848c0 1 -- 172.21.15.46:0/4112159344 shutdown_connections 2024-03-29T08:26:22.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:22.976+0000 7fa739f848c0 1 -- 172.21.15.46:0/4112159344 wait complete. 2024-03-29T08:26:22.993 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.108393, result: 2024-03-29T08:26:22.993 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-03-29T08:26:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.064+0000 7f951ce498c0 1 -- start start 2024-03-29T08:26:23.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.064+0000 7f951ce498c0 1 Processor -- start 2024-03-29T08:26:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56219367d800 0x562193708580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56219367d400 0x562193708b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56219367d000 0x562193709080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56219374a780 con 0x56219367d800 2024-03-29T08:26:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56219374aa00 con 0x56219367d400 2024-03-29T08:26:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x56219374a640 con 0x56219367d000 2024-03-29T08:26:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56219367d800 0x562193708580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56219367d800 0x562193708580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58554/0 (socket says 172.21.15.46:58554) 2024-03-29T08:26:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951a400640 1 -- 172.21.15.46:0/1069585056 learned_addr learned my addr 172.21.15.46:0/1069585056 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f9519a00640 1 --2- 172.21.15.46:0/1069585056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56219367d400 0x562193708b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ae00640 1 --2- 172.21.15.46:0/1069585056 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56219367d000 0x562193709080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951a400640 1 -- 172.21.15.46:0/1069585056 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56219367d000 msgr2=0x562193709080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951a400640 1 --2- 172.21.15.46:0/1069585056 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56219367d000 0x562193709080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951a400640 1 -- 172.21.15.46:0/1069585056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56219367d400 msgr2=0x562193708b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951a400640 1 --2- 172.21.15.46:0/1069585056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56219367d400 0x562193708b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951a400640 1 -- 172.21.15.46:0/1069585056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621936b09c0 con 0x56219367d800 2024-03-29T08:26:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f9519a00640 1 --2- 172.21.15.46:0/1069585056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56219367d400 0x562193708b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:23.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ae00640 1 --2- 172.21.15.46:0/1069585056 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56219367d000 0x562193709080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:23.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951a400640 1 --2- 172.21.15.46:0/1069585056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56219367d800 0x562193708580 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x5621944f09f0 tx=0x56219444cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e6fb26a749f6ce server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f9517200640 1 -- 172.21.15.46:0/1069585056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5621936d1080 con 0x56219367d800 2024-03-29T08:26:23.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f9517200640 1 -- 172.21.15.46:0/1069585056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56219445c300 con 0x56219367d800 2024-03-29T08:26:23.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f9517200640 1 -- 172.21.15.46:0/1069585056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5621936d1600 con 0x56219367d800 2024-03-29T08:26:23.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 1 -- 172.21.15.46:0/1069585056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621936b01a0 con 0x56219367d800 2024-03-29T08:26:23.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 1 -- 172.21.15.46:0/1069585056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5621936b09c0 con 0x56219367d800 2024-03-29T08:26:23.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 4 waiting for MDS map... 2024-03-29T08:26:23.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f9517200640 1 -- 172.21.15.46:0/1069585056 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x5621936bcc40 con 0x56219367d800 2024-03-29T08:26:23.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f9517200640 1 -- 172.21.15.46:0/1069585056 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 64) v1 ==== 2573+0+0 (secure 0 0 0) 0x5621936caf00 con 0x56219367d800 2024-03-29T08:26:23.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 4 Got MDS map 64 2024-03-29T08:26:23.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 4 datascan.main: Using metadata pool output 2024-03-29T08:26:23.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 4 datascan.main: connecting to RADOS... 2024-03-29T08:26:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 1 Processor -- start 2024-03-29T08:26:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 1 -- start start 2024-03-29T08:26:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562194463800 0x562193709600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562194463c00 0x562193709b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621945e8000 0x56219370a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621936c3e00 con 0x562194463800 2024-03-29T08:26:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56219374aa00 con 0x562194463c00 2024-03-29T08:26:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ce498c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x56219374a780 con 0x5621945e8000 2024-03-29T08:26:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562194463800 0x562193709600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562194463800 0x562193709600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58570/0 (socket says 172.21.15.46:58570) 2024-03-29T08:26:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951ae00640 1 -- 172.21.15.46:0/114728795 learned_addr learned my addr 172.21.15.46:0/114728795 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f951a400640 1 --2- 172.21.15.46:0/114728795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562194463c00 0x562193709b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.068+0000 7f9519a00640 1 --2- 172.21.15.46:0/114728795 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621945e8000 0x56219370a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951a400640 1 -- 172.21.15.46:0/114728795 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621945e8000 msgr2=0x56219370a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951a400640 1 --2- 172.21.15.46:0/114728795 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621945e8000 0x56219370a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951a400640 1 -- 172.21.15.46:0/114728795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562194463800 msgr2=0x562193709600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951a400640 1 --2- 172.21.15.46:0/114728795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562194463800 0x562193709600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951a400640 1 -- 172.21.15.46:0/114728795 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621936b0b60 con 0x562194463c00 2024-03-29T08:26:23.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ae00640 1 --2- 172.21.15.46:0/114728795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562194463800 0x562193709600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:26:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951a400640 1 --2- 172.21.15.46:0/114728795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562194463c00 0x562193709b80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x5621944f12c0 tx=0x56219444cd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c312179c2552ffad server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f9514a00640 1 -- 172.21.15.46:0/114728795 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5621936d11e0 con 0x562194463c00 2024-03-29T08:26:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f9514a00640 1 -- 172.21.15.46:0/114728795 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56219445c300 con 0x562194463c00 2024-03-29T08:26:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f9514a00640 1 -- 172.21.15.46:0/114728795 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5621936d0f20 con 0x562194463c00 2024-03-29T08:26:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 -- 172.21.15.46:0/114728795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562194463c00 msgr2=0x562193709b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 --2- 172.21.15.46:0/114728795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562194463c00 0x562193709b80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x5621944f12c0 tx=0x56219444cd80 comp rx=0 tx=0).stop 2024-03-29T08:26:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 -- 172.21.15.46:0/114728795 shutdown_connections 2024-03-29T08:26:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 --2- 172.21.15.46:0/114728795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562194463c00 0x562193709b80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 --2- 172.21.15.46:0/114728795 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5621945e8000 0x56219370a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 --2- 172.21.15.46:0/114728795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562194463800 0x562193709600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 -- 172.21.15.46:0/114728795 >> 172.21.15.46:0/114728795 conn(0x56219367cc00 msgr2=0x5621936a9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 -- 172.21.15.46:0/114728795 shutdown_connections 2024-03-29T08:26:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 -- 172.21.15.46:0/114728795 wait complete. 2024-03-29T08:26:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 Processor -- start 2024-03-29T08:26:23.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 -- start start 2024-03-29T08:26:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562194463c00 0x56219370b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562194463800 0x56219370ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621945e9800 0x56219370a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x56219374a500 con 0x5621945e9800 2024-03-29T08:26:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56219374a3c0 con 0x562194463800 2024-03-29T08:26:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x56219374a280 con 0x562194463c00 2024-03-29T08:26:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f9519a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621945e9800 0x56219370a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f9519a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621945e9800 0x56219370a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58574/0 (socket says 172.21.15.46:58574) 2024-03-29T08:26:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 learned_addr learned my addr 172.21.15.46:0/3435570698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ae00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562194463c00 0x56219370b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951a400640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562194463800 0x56219370ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562194463c00 msgr2=0x56219370b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f9519a00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562194463c00 0x56219370b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562194463800 msgr2=0x56219370ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f9519a00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562194463800 0x56219370ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621936b01a0 con 0x5621945e9800 2024-03-29T08:26:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f9519a00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621945e9800 0x56219370a680 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x562194451350 tx=0x562194598c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f9512c00640 1 -- 172.21.15.46:0/3435570698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5621936d11e0 con 0x5621945e9800 2024-03-29T08:26:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f9512c00640 1 -- 172.21.15.46:0/3435570698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56219445db00 con 0x5621945e9800 2024-03-29T08:26:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f9512c00640 1 -- 172.21.15.46:0/3435570698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5621936d0f20 con 0x5621945e9800 2024-03-29T08:26:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5621936b09c0 con 0x5621945e9800 2024-03-29T08:26:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621936b0b60 con 0x5621945e9800 2024-03-29T08:26:23.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.072+0000 7f951ce498c0 4 datascan.init: resolving metadata pool 12 2024-03-29T08:26:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f9512c00640 1 -- 172.21.15.46:0/3435570698 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x5621936b4840 con 0x5621945e9800 2024-03-29T08:26:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f9512c00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x562193960c00 0x56219370b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f9512c00640 1 -- 172.21.15.46:0/3435570698 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x5621936bca80 con 0x5621945e9800 2024-03-29T08:26:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f951ce498c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-29T08:26:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f951ce498c0 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-03-29T08:26:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f951ce498c0 4 datascan.main: opening data pool 'cephfs_data' 2024-03-29T08:26:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f951ce498c0 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5621945e9400 0x56219370a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e9000 con 0x5621945e9400 2024-03-29T08:26:23.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f951ae00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x562193960c00 0x56219370b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f951a400640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5621945e9400 0x56219370a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f951a400640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5621945e9400 0x56219370a100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f951ae00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x562193960c00 0x56219370b700 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x5621944508a0 tx=0x562194599da0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv24 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f951ce498c0 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5621945e8c00 0x562193709b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:2 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e8800 con 0x5621945e8c00 2024-03-29T08:26:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f951ae00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5621945e8c00 0x562193709b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f951ae00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5621945e8c00 0x562193709b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f951ce498c0 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5621945e8400 0x562193709600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.076+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:3 13.0 13:00000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e8000 con 0x5621945e8400 2024-03-29T08:26:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.080+0000 7f951a400640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5621945e8400 0x562193709600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.080+0000 7f951a400640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5621945e8400 0x562193709600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.080+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.080+0000 7f9514000640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x562193961c00 0x562193a5a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.080+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:4 13.20 13:04000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64000 con 0x562193961c00 2024-03-29T08:26:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.080+0000 7f9519a00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x562193961c00 0x562193a5a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.080+0000 7f9519a00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x562193961c00 0x562193a5a000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.080+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.2 v2:172.21.15.46:6809/27428 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193961c00 2024-03-29T08:26:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.080+0000 7f9514a00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x562193a64800 0x562193a5a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.080+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:5 13.10 13:08000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64c00 con 0x562193a64800 2024-03-29T08:26:23.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.080+0000 7f951ae00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x562193a64800 0x562193a5a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.080+0000 7f951ae00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x562193a64800 0x562193a5a580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.080+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.7 v2:172.21.15.73:6816/27700 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a64800 2024-03-29T08:26:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.080+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:6 13.30 13:0c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64400 con 0x562193961c00 2024-03-29T08:26:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.080+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.2 v2:172.21.15.46:6809/27428 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193961c00 2024-03-29T08:26:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.080+0000 7f9514a00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x562193a65000 0x562193a5ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.080+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:7 13.8 13:10000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a65400 con 0x562193a65000 2024-03-29T08:26:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.084+0000 7f951a400640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x562193a65000 0x562193a5ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.084+0000 7f951a400640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x562193a65000 0x562193a5ab00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.084+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.5 v2:172.21.15.73:6808/27690 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x562193a65000 2024-03-29T08:26:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.084+0000 7f9514000640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x562193a76000 0x562193a5b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.084+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:8 13.28 13:14000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a76400 con 0x562193a76000 2024-03-29T08:26:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.084+0000 7f9519a00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x562193a76000 0x562193a5b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.084+0000 7f9519a00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x562193a76000 0x562193a5b600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.084+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.084+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:9 13.18 13:18000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a65800 con 0x5621945e8400 2024-03-29T08:26:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.084+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.084+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:10 13.38 13:1c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a76800 con 0x5621945e9400 2024-03-29T08:26:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.084+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.084+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:11 13.4 13:20000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a65c00 con 0x5621945e8c00 2024-03-29T08:26:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.084+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.084+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:12 13.24 13:24000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e8800 con 0x5621945e8400 2024-03-29T08:26:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.084+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.084+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:13 13.14 13:28000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64c00 con 0x562193a76000 2024-03-29T08:26:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 94 out=49b] v92'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:14 13.34 13:2c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e9c00 con 0x5621945e9400 2024-03-29T08:26:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 3 ==== osd_op_reply(14 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:15 13.c 13:30000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a65c00 con 0x562193a76000 2024-03-29T08:26:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:16 13.2c 13:34000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562194463400 con 0x562193a64800 2024-03-29T08:26:23.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.7 v2:172.21.15.73:6816/27700 2 ==== osd_op_reply(16 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a64800 2024-03-29T08:26:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:17 13.1c 13:38000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e9000 con 0x5621945e9400 2024-03-29T08:26:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 4 ==== osd_op_reply(17 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:18 13.3c 13:3c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193961800 con 0x562193a65000 2024-03-29T08:26:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.5 v2:172.21.15.73:6808/27690 2 ==== osd_op_reply(18 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x562193a65000 2024-03-29T08:26:23.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:19 13.2 13:40000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e8000 con 0x5621945e8400 2024-03-29T08:26:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 4 ==== osd_op_reply(19 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:20 13.22 13:44000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64000 con 0x562193a64800 2024-03-29T08:26:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.7 v2:172.21.15.73:6816/27700 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a64800 2024-03-29T08:26:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:21 13.12 13:48000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a76400 con 0x562193a76000 2024-03-29T08:26:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 4 ==== osd_op_reply(21 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:22 13.32 13:4c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64400 con 0x562193a64800 2024-03-29T08:26:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.7 v2:172.21.15.73:6816/27700 4 ==== osd_op_reply(22 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a64800 2024-03-29T08:26:23.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f9514a00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x562193a76c00 0x562193a5b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:23 13.a 13:50000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a77000 con 0x562193a76c00 2024-03-29T08:26:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f951ae00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x562193a76c00 0x562193a5b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.088+0000 7f951ae00640 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x562193a76c00 0x562193a5b080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.4 v2:172.21.15.73:6800/27689 1 ==== osd_op_reply(23 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a76c00 2024-03-29T08:26:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:24 13.2a 13:54000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64c00 con 0x562193a64800 2024-03-29T08:26:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.7 v2:172.21.15.73:6816/27700 5 ==== osd_op_reply(24 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a64800 2024-03-29T08:26:23.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:25 13.1a 13:58000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a77400 con 0x562193a65000 2024-03-29T08:26:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.5 v2:172.21.15.73:6808/27690 3 ==== osd_op_reply(25 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x562193a65000 2024-03-29T08:26:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:26 13.3a 13:5c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a65400 con 0x562193961c00 2024-03-29T08:26:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.2 v2:172.21.15.46:6809/27428 3 ==== osd_op_reply(26 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193961c00 2024-03-29T08:26:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:27 13.6 13:60000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a77800 con 0x5621945e8c00 2024-03-29T08:26:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 3 ==== osd_op_reply(27 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:28 13.26 13:64000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a65800 con 0x5621945e8c00 2024-03-29T08:26:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 4 ==== osd_op_reply(28 [pgnls-filter start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:29 13.16 13:68000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562194463400 con 0x562193a64800 2024-03-29T08:26:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.7 v2:172.21.15.73:6816/27700 6 ==== osd_op_reply(29 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a64800 2024-03-29T08:26:23.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:30 13.36 13:6c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a76800 con 0x562193961c00 2024-03-29T08:26:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.2 v2:172.21.15.46:6809/27428 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193961c00 2024-03-29T08:26:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:31 13.e 13:70000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e8800 con 0x5621945e9400 2024-03-29T08:26:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 5 ==== osd_op_reply(31 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:32 13.2e 13:74000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a77400 con 0x562193a76000 2024-03-29T08:26:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:33 13.1e 13:78000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e9c00 con 0x5621945e9400 2024-03-29T08:26:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 6 ==== osd_op_reply(33 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.092+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:34 13.3e 13:7c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a77000 con 0x562193a76c00 2024-03-29T08:26:23.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.4 v2:172.21.15.73:6800/27689 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a76c00 2024-03-29T08:26:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:35 13.1 13:80000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e9000 con 0x5621945e8400 2024-03-29T08:26:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 5 ==== osd_op_reply(35 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:36 13.21 13:84000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64000 con 0x562193961c00 2024-03-29T08:26:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.2 v2:172.21.15.46:6809/27428 5 ==== osd_op_reply(36 [pgnls-filter start_epoch 94 out=49b] v92'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193961c00 2024-03-29T08:26:23.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:37 13.11 13:88000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e8000 con 0x562193a76000 2024-03-29T08:26:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 6 ==== osd_op_reply(37 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:38 13.31 13:8c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64400 con 0x562193a76c00 2024-03-29T08:26:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.4 v2:172.21.15.73:6800/27689 3 ==== osd_op_reply(38 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a76c00 2024-03-29T08:26:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:39 13.9 13:90000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193961800 con 0x5621945e9400 2024-03-29T08:26:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 7 ==== osd_op_reply(39 [pgnls-filter start_epoch 94 out=49b] v92'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:40 13.29 13:94000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64c00 con 0x5621945e9400 2024-03-29T08:26:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 8 ==== osd_op_reply(40 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:41 13.19 13:98000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a76400 con 0x562193a64800 2024-03-29T08:26:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.7 v2:172.21.15.73:6816/27700 7 ==== osd_op_reply(41 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a64800 2024-03-29T08:26:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:42 13.39 13:9c000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a77800 con 0x562193a76c00 2024-03-29T08:26:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.4 v2:172.21.15.73:6800/27689 4 ==== osd_op_reply(42 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a76c00 2024-03-29T08:26:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:43 13.5 13:a0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a76800 con 0x562193961c00 2024-03-29T08:26:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.2 v2:172.21.15.46:6809/27428 6 ==== osd_op_reply(43 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193961c00 2024-03-29T08:26:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:44 13.25 13:a4000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a77000 con 0x5621945e8c00 2024-03-29T08:26:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.096+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:45 13.15 13:a8000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a77400 con 0x5621945e8c00 2024-03-29T08:26:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 6 ==== osd_op_reply(45 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:46 13.35 13:ac000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a76400 con 0x562193a76000 2024-03-29T08:26:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 7 ==== osd_op_reply(46 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:47 13.d 13:b0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a77c00 con 0x5621945e9400 2024-03-29T08:26:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 9 ==== osd_op_reply(47 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:48 13.2d 13:b4000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a65800 con 0x562193a64800 2024-03-29T08:26:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.7 v2:172.21.15.73:6816/27700 8 ==== osd_op_reply(48 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a64800 2024-03-29T08:26:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:49 13.1d 13:b8000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a65c00 con 0x5621945e8400 2024-03-29T08:26:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 6 ==== osd_op_reply(49 [pgnls-filter start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:50 13.3d 13:bc000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64400 con 0x562193961c00 2024-03-29T08:26:23.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.2 v2:172.21.15.46:6809/27428 7 ==== osd_op_reply(50 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193961c00 2024-03-29T08:26:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:51 13.3 13:c0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a65400 con 0x562193a76000 2024-03-29T08:26:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 8 ==== osd_op_reply(51 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:52 13.23 13:c4000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562194463400 con 0x562193a65000 2024-03-29T08:26:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.5 v2:172.21.15.73:6808/27690 4 ==== osd_op_reply(52 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x562193a65000 2024-03-29T08:26:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:53 13.13 13:c8000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a77c00 con 0x5621945e8400 2024-03-29T08:26:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:54 13.33 13:cc000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64c00 con 0x5621945e8c00 2024-03-29T08:26:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 7 ==== osd_op_reply(54 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:55 13.b 13:d0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a76800 con 0x5621945e8c00 2024-03-29T08:26:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 8 ==== osd_op_reply(55 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.100+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:56 13.2b 13:d4000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a65c00 con 0x562193a65000 2024-03-29T08:26:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.5 v2:172.21.15.73:6808/27690 5 ==== osd_op_reply(56 [pgnls-filter start_epoch 94 out=49b] v92'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x562193a65000 2024-03-29T08:26:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:57 13.1b 13:d8000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e8800 con 0x562193961c00 2024-03-29T08:26:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.2 v2:172.21.15.46:6809/27428 8 ==== osd_op_reply(57 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193961c00 2024-03-29T08:26:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:58 13.3b 13:dc000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64000 con 0x5621945e8c00 2024-03-29T08:26:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 9 ==== osd_op_reply(58 [pgnls-filter start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:59 13.7 13:e0000000::::head [pgnls-filter start_epoch 94 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e9c00 con 0x5621945e8c00 2024-03-29T08:26:23.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 10 ==== osd_op_reply(59 [pgnls-filter start_epoch 94 out=81b] v94'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:60 13.0 13:00000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a65800 con 0x5621945e8400 2024-03-29T08:26:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 8 ==== osd_op_reply(60 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:61 13.20 13:04000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e9000 con 0x562193961c00 2024-03-29T08:26:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.2 v2:172.21.15.46:6809/27428 9 ==== osd_op_reply(61 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193961c00 2024-03-29T08:26:23.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:62 13.10 13:08000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64400 con 0x562193a64800 2024-03-29T08:26:23.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.7 v2:172.21.15.73:6816/27700 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a64800 2024-03-29T08:26:23.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:63 13.30 13:0c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e8000 con 0x562193961c00 2024-03-29T08:26:23.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.2 v2:172.21.15.46:6809/27428 10 ==== osd_op_reply(63 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193961c00 2024-03-29T08:26:23.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:64 13.8 13:10000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a65400 con 0x562193a65000 2024-03-29T08:26:23.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.5 v2:172.21.15.73:6808/27690 6 ==== osd_op_reply(64 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x562193a65000 2024-03-29T08:26:23.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:65 13.28 13:14000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562194463400 con 0x562193a76000 2024-03-29T08:26:23.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 9 ==== osd_op_reply(65 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:66 13.18 13:18000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a76400 con 0x5621945e8400 2024-03-29T08:26:23.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 9 ==== osd_op_reply(66 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.104+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:67 13.38 13:1c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193961800 con 0x5621945e9400 2024-03-29T08:26:23.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 10 ==== osd_op_reply(67 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:68 13.4 13:20000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a8e000 con 0x5621945e8c00 2024-03-29T08:26:23.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 11 ==== osd_op_reply(68 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:69 13.24 13:24000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a90000 con 0x5621945e8400 2024-03-29T08:26:23.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 10 ==== osd_op_reply(69 [pgnls-filter start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:70 13.14 13:28000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a8e400 con 0x562193a76000 2024-03-29T08:26:23.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 10 ==== osd_op_reply(70 [pgnls-filter start_epoch 94 out=49b] v92'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:71 13.34 13:2c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a90400 con 0x5621945e9400 2024-03-29T08:26:23.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 11 ==== osd_op_reply(71 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:72 13.c 13:30000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a8e800 con 0x562193a76000 2024-03-29T08:26:23.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:73 13.2c 13:34000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a90800 con 0x562193a64800 2024-03-29T08:26:23.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.7 v2:172.21.15.73:6816/27700 10 ==== osd_op_reply(73 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a64800 2024-03-29T08:26:23.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:74 13.1c 13:38000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a8ec00 con 0x5621945e9400 2024-03-29T08:26:23.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 12 ==== osd_op_reply(74 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:75 13.3c 13:3c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a90c00 con 0x562193a65000 2024-03-29T08:26:23.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.5 v2:172.21.15.73:6808/27690 7 ==== osd_op_reply(75 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x562193a65000 2024-03-29T08:26:23.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:76 13.2 13:40000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a8f000 con 0x5621945e8400 2024-03-29T08:26:23.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 11 ==== osd_op_reply(76 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.108+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:77 13.22 13:44000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a91000 con 0x562193a64800 2024-03-29T08:26:23.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.7 v2:172.21.15.73:6816/27700 11 ==== osd_op_reply(77 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a64800 2024-03-29T08:26:23.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:78 13.12 13:48000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a8f400 con 0x562193a76000 2024-03-29T08:26:23.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 12 ==== osd_op_reply(78 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:79 13.32 13:4c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a91400 con 0x562193a64800 2024-03-29T08:26:23.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.7 v2:172.21.15.73:6816/27700 12 ==== osd_op_reply(79 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a64800 2024-03-29T08:26:23.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:80 13.a 13:50000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a8f800 con 0x562193a76c00 2024-03-29T08:26:23.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.4 v2:172.21.15.73:6800/27689 5 ==== osd_op_reply(80 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a76c00 2024-03-29T08:26:23.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:81 13.2a 13:54000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a91800 con 0x562193a64800 2024-03-29T08:26:23.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.7 v2:172.21.15.73:6816/27700 13 ==== osd_op_reply(81 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a64800 2024-03-29T08:26:23.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:82 13.1a 13:58000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a8fc00 con 0x562193a65000 2024-03-29T08:26:23.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.5 v2:172.21.15.73:6808/27690 8 ==== osd_op_reply(82 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x562193a65000 2024-03-29T08:26:23.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:83 13.3a 13:5c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a91c00 con 0x562193961c00 2024-03-29T08:26:23.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.2 v2:172.21.15.46:6809/27428 11 ==== osd_op_reply(83 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193961c00 2024-03-29T08:26:23.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:84 13.6 13:60000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a77c00 con 0x5621945e8c00 2024-03-29T08:26:23.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 12 ==== osd_op_reply(84 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:85 13.26 13:64000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a8e000 con 0x5621945e8c00 2024-03-29T08:26:23.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 13 ==== osd_op_reply(85 [pgnls-filter start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:86 13.16 13:68000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64400 con 0x562193a64800 2024-03-29T08:26:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.7 v2:172.21.15.73:6816/27700 14 ==== osd_op_reply(86 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a64800 2024-03-29T08:26:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:87 13.36 13:6c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a8f800 con 0x562193961c00 2024-03-29T08:26:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.2 v2:172.21.15.46:6809/27428 12 ==== osd_op_reply(87 [pgnls-filter start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193961c00 2024-03-29T08:26:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:88 13.e 13:70000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a65c00 con 0x5621945e9400 2024-03-29T08:26:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.112+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:89 13.2e 13:74000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a90800 con 0x562193a76000 2024-03-29T08:26:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 13 ==== osd_op_reply(89 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:90 13.1e 13:78000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a65800 con 0x5621945e9400 2024-03-29T08:26:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 14 ==== osd_op_reply(90 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:91 13.3e 13:7c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a91000 con 0x562193a76c00 2024-03-29T08:26:23.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.4 v2:172.21.15.73:6800/27689 6 ==== osd_op_reply(91 [pgnls-filter start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a76c00 2024-03-29T08:26:23.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:92 13.1 13:80000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a65400 con 0x5621945e8400 2024-03-29T08:26:23.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 12 ==== osd_op_reply(92 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:93 13.21 13:84000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a91400 con 0x562193961c00 2024-03-29T08:26:23.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.2 v2:172.21.15.46:6809/27428 13 ==== osd_op_reply(93 [pgnls-filter start_epoch 94 out=49b] v92'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193961c00 2024-03-29T08:26:23.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:94 13.11 13:88000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562194463400 con 0x562193a76000 2024-03-29T08:26:23.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 14 ==== osd_op_reply(94 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:95 13.31 13:8c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a91800 con 0x562193a76c00 2024-03-29T08:26:23.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.4 v2:172.21.15.73:6800/27689 7 ==== osd_op_reply(95 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a76c00 2024-03-29T08:26:23.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:96 13.9 13:90000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e9c00 con 0x5621945e9400 2024-03-29T08:26:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 15 ==== osd_op_reply(96 [pgnls-filter start_epoch 94 out=49b] v92'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:97 13.29 13:94000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a90000 con 0x5621945e9400 2024-03-29T08:26:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 16 ==== osd_op_reply(97 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:98 13.19 13:98000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e8800 con 0x562193a64800 2024-03-29T08:26:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.7 v2:172.21.15.73:6816/27700 15 ==== osd_op_reply(98 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a64800 2024-03-29T08:26:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.116+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:99 13.39 13:9c000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a90400 con 0x562193a76c00 2024-03-29T08:26:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.4 v2:172.21.15.73:6800/27689 8 ==== osd_op_reply(99 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a76c00 2024-03-29T08:26:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:100 13.5 13:a0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e9000 con 0x562193961c00 2024-03-29T08:26:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.2 v2:172.21.15.46:6809/27428 14 ==== osd_op_reply(100 [pgnls-filter start_epoch 94 out=49b] v92'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193961c00 2024-03-29T08:26:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:101 13.25 13:a4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193961800 con 0x5621945e8c00 2024-03-29T08:26:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 14 ==== osd_op_reply(101 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:102 13.15 13:a8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e8000 con 0x5621945e8c00 2024-03-29T08:26:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 15 ==== osd_op_reply(102 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:103 13.35 13:ac000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a76400 con 0x562193a76000 2024-03-29T08:26:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 15 ==== osd_op_reply(103 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:104 13.d 13:b0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a8e000 con 0x5621945e9400 2024-03-29T08:26:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 17 ==== osd_op_reply(104 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:105 13.2d 13:b4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a8f400 con 0x562193a64800 2024-03-29T08:26:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.7 v2:172.21.15.73:6816/27700 16 ==== osd_op_reply(105 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x562193a64800 2024-03-29T08:26:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:106 13.1d 13:b8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a8e400 con 0x5621945e8400 2024-03-29T08:26:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 13 ==== osd_op_reply(106 [pgnls-filter start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:107 13.3d 13:bc000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a8ec00 con 0x562193961c00 2024-03-29T08:26:23.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.2 v2:172.21.15.46:6809/27428 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193961c00 2024-03-29T08:26:23.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:108 13.3 13:c0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a8e800 con 0x562193a76000 2024-03-29T08:26:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 16 ==== osd_op_reply(108 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:109 13.23 13:c4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a91c00 con 0x562193a65000 2024-03-29T08:26:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.5 v2:172.21.15.73:6808/27690 9 ==== osd_op_reply(109 [pgnls-filter start_epoch 94 out=49b] v92'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x562193a65000 2024-03-29T08:26:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.120+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:110 13.13 13:c8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e8800 con 0x5621945e8400 2024-03-29T08:26:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 14 ==== osd_op_reply(110 [pgnls-filter start_epoch 94 out=49b] v92'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:111 13.33 13:cc000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a90800 con 0x5621945e8c00 2024-03-29T08:26:23.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 16 ==== osd_op_reply(111 [pgnls-filter start_epoch 94 out=49b] v92'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:112 13.b 13:d0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e8000 con 0x5621945e8c00 2024-03-29T08:26:23.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 17 ==== osd_op_reply(112 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:113 13.2b 13:d4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a91400 con 0x562193a65000 2024-03-29T08:26:23.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.5 v2:172.21.15.73:6808/27690 10 ==== osd_op_reply(113 [pgnls-filter start_epoch 94 out=49b] v92'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x562193a65000 2024-03-29T08:26:23.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:114 13.1b 13:d8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193961800 con 0x562193961c00 2024-03-29T08:26:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.2 v2:172.21.15.46:6809/27428 16 ==== osd_op_reply(114 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193961c00 2024-03-29T08:26:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:115 13.3b 13:dc000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a90c00 con 0x5621945e8c00 2024-03-29T08:26:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 18 ==== osd_op_reply(115 [pgnls-filter start_epoch 94 out=49b] v92'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:116 13.7 13:e0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a91000 con 0x5621945e8c00 2024-03-29T08:26:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 19 ==== osd_op_reply(116 [pgnls-filter start_epoch 94 out=81b] v94'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ce498c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-03-29T08:26:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ce498c0 10 datascan.operator(): handling object 10000000001.0 2024-03-29T08:26:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:117 13.7 13:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64c00 con 0x5621945e8c00 2024-03-29T08:26:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 20 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ce498c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-03-29T08:26:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ce498c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-03-29T08:26:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ce498c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-03-29T08:26:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:118 12.10 12:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64000 con 0x562193a76000 2024-03-29T08:26:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 17 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+59 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ce498c0 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-03-29T08:26:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a77800 con 0x5621945e8c00 2024-03-29T08:26:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ae00640 1 -- 172.21.15.46:0/3435570698 <== osd.3 v2:172.21.15.46:6808/27426 21 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=520b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+520 (crc 0 0 0) 0x5621943dad80 con 0x5621945e8c00 2024-03-29T08:26:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ce498c0 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-03-29T08:26:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ce498c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-03-29T08:26:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ce498c0 20 datascan.get_frag_of: frag is * 2024-03-29T08:26:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:120 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a77000 con 0x562193a76000 2024-03-29T08:26:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 18 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ce498c0 10 datascan.find_or_create_dirfrag: read_version = 17 2024-03-29T08:26:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ce498c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-03-29T08:26:23.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.124+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:121 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a77400 con 0x562193a76000 2024-03-29T08:26:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 19 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-03-29T08:26:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 20 datascan.read_dentry: file_head not found in result 2024-03-29T08:26:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-03-29T08:26:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:122 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a76800 con 0x562193a76000 2024-03-29T08:26:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 20 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v94'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-03-29T08:26:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:123 13.27 13:e4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a8f800 con 0x562193a76000 2024-03-29T08:26:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.6 v2:172.21.15.73:6824/27691 21 ==== osd_op_reply(123 [pgnls-filter start_epoch 94 out=49b] v92'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193a76000 2024-03-29T08:26:23.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:124 13.17 13:e8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a91800 con 0x5621945e8400 2024-03-29T08:26:23.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 15 ==== osd_op_reply(124 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:125 13.37 13:ec000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a90000 con 0x5621945e8400 2024-03-29T08:26:23.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 16 ==== osd_op_reply(125 [pgnls-filter start_epoch 94 out=49b] v92'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:126 13.f 13:f0000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a90400 con 0x562193961c00 2024-03-29T08:26:23.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 <== osd.2 v2:172.21.15.46:6809/27428 17 ==== osd_op_reply(126 [pgnls-filter start_epoch 94 out=49b] v92'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dafc0 con 0x562193961c00 2024-03-29T08:26:23.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:127 13.2f 13:f4000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e9000 con 0x5621945e9400 2024-03-29T08:26:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.0 v2:172.21.15.46:6800/27424 18 ==== osd_op_reply(127 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e9400 2024-03-29T08:26:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f9514000640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:128 13.1f 13:f8000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x562193a64400 con 0x5621945e8400 2024-03-29T08:26:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 17 ==== osd_op_reply(128 [pgnls-filter start_epoch 94 out=49b] v92'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f9514a00640 1 -- 172.21.15.46:0/3435570698 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:129 13.3f 13:fc000000::::head [pgnls-filter start_epoch 94 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5621945e9c00 con 0x5621945e8400 2024-03-29T08:26:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951a400640 1 -- 172.21.15.46:0/3435570698 <== osd.1 v2:172.21.15.46:6824/27427 18 ==== osd_op_reply(129 [pgnls-filter start_epoch 94 out=49b] v92'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5621943dab40 con 0x5621945e8400 2024-03-29T08:26:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5621945e9400 msgr2=0x56219370a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5621945e9400 0x56219370a100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5621945e8400 msgr2=0x562193709600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5621945e8400 0x562193709600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x562193961c00 msgr2=0x562193a5a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x562193961c00 0x562193a5a000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5621945e8c00 msgr2=0x562193709b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5621945e8c00 0x562193709b80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x562193a76c00 msgr2=0x562193a5b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x562193a76c00 0x562193a5b080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 reap_dead start 2024-03-29T08:26:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x562193a65000 msgr2=0x562193a5ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x562193a65000 0x562193a5ab00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x562193a76000 msgr2=0x562193a5b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x562193a76000 0x562193a5b600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x562193a64800 msgr2=0x562193a5a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x562193a64800 0x562193a5a580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 reap_dead start 2024-03-29T08:26:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f9519a00640 1 -- 172.21.15.46:0/3435570698 reap_dead start 2024-03-29T08:26:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x562193960c00 msgr2=0x56219370b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.128+0000 7f951ce498c0 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x562193960c00 0x56219370b700 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x5621944508a0 tx=0x562194599da0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.132+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621945e9800 msgr2=0x56219370a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.132+0000 7f951ce498c0 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621945e9800 0x56219370a680 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x562194451350 tx=0x562194598c00 comp rx=0 tx=0).stop 2024-03-29T08:26:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.132+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 shutdown_connections 2024-03-29T08:26:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.132+0000 7f951ce498c0 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x562193960c00 0x56219370b700 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.132+0000 7f951ce498c0 1 --2- 172.21.15.46:0/3435570698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5621945e9800 0x56219370a680 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.132+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 >> 172.21.15.46:0/3435570698 conn(0x56219367cc00 msgr2=0x5621936a9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.132+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 shutdown_connections 2024-03-29T08:26:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.132+0000 7f951ce498c0 1 -- 172.21.15.46:0/3435570698 wait complete. 2024-03-29T08:26:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.132+0000 7f951ce498c0 1 -- 172.21.15.46:0/1069585056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56219367d800 msgr2=0x562193708580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.132+0000 7f951ce498c0 1 --2- 172.21.15.46:0/1069585056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56219367d800 0x562193708580 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x5621944f09f0 tx=0x56219444cba0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.132+0000 7f951ce498c0 1 -- 172.21.15.46:0/1069585056 shutdown_connections 2024-03-29T08:26:23.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.132+0000 7f951ce498c0 1 --2- 172.21.15.46:0/1069585056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56219367d400 0x562193708b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.132+0000 7f951ce498c0 1 --2- 172.21.15.46:0/1069585056 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56219367d000 0x562193709080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.132+0000 7f951ce498c0 1 --2- 172.21.15.46:0/1069585056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56219367d800 0x562193708580 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.132+0000 7f951ce498c0 1 -- 172.21.15.46:0/1069585056 >> 172.21.15.46:0/1069585056 conn(0x56219367dc00 msgr2=0x5621936a9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:23.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.132+0000 7f951ce498c0 1 -- 172.21.15.46:0/1069585056 shutdown_connections 2024-03-29T08:26:23.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.132+0000 7f951ce498c0 1 -- 172.21.15.46:0/1069585056 wait complete. 2024-03-29T08:26:23.138 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.144227, result: 2024-03-29T08:26:23.138 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-03-29T08:26:23.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.220+0000 7f6c4ea488c0 1 -- start start 2024-03-29T08:26:23.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.220+0000 7f6c4ea488c0 1 Processor -- start 2024-03-29T08:26:23.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ea488c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ab50d800 0x5567ab598580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ea488c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567ab50d400 0x5567ab598b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ea488c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567ab50d000 0x5567ab599080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ea488c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ab5da780 con 0x5567ab50d800 2024-03-29T08:26:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ea488c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ab5daa00 con 0x5567ab50d000 2024-03-29T08:26:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ea488c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567ab5da640 con 0x5567ab50d400 2024-03-29T08:26:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4c000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ab50d800 0x5567ab598580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ca00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567ab50d000 0x5567ab599080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4c000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ab50d800 0x5567ab598580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58578/0 (socket says 172.21.15.46:58578) 2024-03-29T08:26:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4c000640 1 -- 172.21.15.46:0/3566571608 learned_addr learned my addr 172.21.15.46:0/3566571608 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4b600640 1 --2- 172.21.15.46:0/3566571608 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567ab50d400 0x5567ab598b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ca00640 1 -- 172.21.15.46:0/3566571608 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567ab50d400 msgr2=0x5567ab598b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ca00640 1 --2- 172.21.15.46:0/3566571608 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567ab50d400 0x5567ab598b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ca00640 1 -- 172.21.15.46:0/3566571608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ab50d800 msgr2=0x5567ab598580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ca00640 1 --2- 172.21.15.46:0/3566571608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ab50d800 0x5567ab598580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ca00640 1 -- 172.21.15.46:0/3566571608 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567ab5409c0 con 0x5567ab50d000 2024-03-29T08:26:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4c000640 1 --2- 172.21.15.46:0/3566571608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ab50d800 0x5567ab598580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:26:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ca00640 1 --2- 172.21.15.46:0/3566571608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567ab50d000 0x5567ab599080 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x5567ac381290 tx=0x5567ac416360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f07ed4cd6b9274ba server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c48e00640 1 -- 172.21.15.46:0/3566571608 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567ab561600 con 0x5567ab50d000 2024-03-29T08:26:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c48e00640 1 -- 172.21.15.46:0/3566571608 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5567ab56a480 con 0x5567ab50d000 2024-03-29T08:26:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c48e00640 1 -- 172.21.15.46:0/3566571608 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567ab5611e0 con 0x5567ab50d000 2024-03-29T08:26:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/3566571608 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567ab540b60 con 0x5567ab50d000 2024-03-29T08:26:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/3566571608 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5567ab540340 con 0x5567ab50d000 2024-03-29T08:26:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ea488c0 4 waiting for MDS map... 2024-03-29T08:26:23.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c48e00640 1 -- 172.21.15.46:0/3566571608 <== mon.1 v2:172.21.15.73:3300/0 4 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x5567ab54cc40 con 0x5567ab50d000 2024-03-29T08:26:23.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c48e00640 1 -- 172.21.15.46:0/3566571608 <== mon.1 v2:172.21.15.73:3300/0 5 ==== fsmap(e 64) v1 ==== 2573+0+0 (secure 0 0 0) 0x5567ab55b200 con 0x5567ab50d000 2024-03-29T08:26:23.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ea488c0 4 Got MDS map 64 2024-03-29T08:26:23.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ea488c0 4 datascan.main: Using metadata pool output 2024-03-29T08:26:23.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ea488c0 4 datascan.main: connecting to RADOS... 2024-03-29T08:26:23.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ea488c0 1 Processor -- start 2024-03-29T08:26:23.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.224+0000 7f6c4ea488c0 1 -- start start 2024-03-29T08:26:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567ac42ac00 0x5567ab599600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567ac42b000 0x5567ab599b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ac42b400 0x5567ab59a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ab553e00 con 0x5567ac42b400 2024-03-29T08:26:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ab5da780 con 0x5567ac42b000 2024-03-29T08:26:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567ab5da500 con 0x5567ac42ac00 2024-03-29T08:26:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4b600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567ac42ac00 0x5567ab599600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4c000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567ac42b000 0x5567ab599b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4c000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567ac42b000 0x5567ab599b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:34618/0 (socket says 172.21.15.46:34618) 2024-03-29T08:26:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4c000640 1 -- 172.21.15.46:0/1037851155 learned_addr learned my addr 172.21.15.46:0/1037851155 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4b600640 1 --2- 172.21.15.46:0/1037851155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ac42b400 0x5567ab59a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4c000640 1 -- 172.21.15.46:0/1037851155 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567ac42ac00 msgr2=0x5567ab599600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4c000640 1 --2- 172.21.15.46:0/1037851155 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567ac42ac00 0x5567ab599600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4c000640 1 -- 172.21.15.46:0/1037851155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ac42b400 msgr2=0x5567ab59a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4c000640 1 --2- 172.21.15.46:0/1037851155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ac42b400 0x5567ab59a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4c000640 1 -- 172.21.15.46:0/1037851155 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567ab540000 con 0x5567ac42b000 2024-03-29T08:26:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4c000640 1 --2- 172.21.15.46:0/1037851155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567ac42b000 0x5567ab599b80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x5567ac380a80 tx=0x5567ac416120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba5c736cadccf0dc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c46600640 1 -- 172.21.15.46:0/1037851155 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567ab561080 con 0x5567ac42b000 2024-03-29T08:26:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c46600640 1 -- 172.21.15.46:0/1037851155 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5567ac2f0900 con 0x5567ac42b000 2024-03-29T08:26:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c46600640 1 -- 172.21.15.46:0/1037851155 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567ab561600 con 0x5567ac42b000 2024-03-29T08:26:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1037851155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567ac42b000 msgr2=0x5567ab599b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/1037851155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567ac42b000 0x5567ab599b80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x5567ac380a80 tx=0x5567ac416120 comp rx=0 tx=0).stop 2024-03-29T08:26:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1037851155 shutdown_connections 2024-03-29T08:26:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/1037851155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567ac42b000 0x5567ab599b80 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/1037851155 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ac42b400 0x5567ab59a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/1037851155 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567ac42ac00 0x5567ab599600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1037851155 >> 172.21.15.46:0/1037851155 conn(0x5567ab50cc00 msgr2=0x5567ab539800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1037851155 shutdown_connections 2024-03-29T08:26:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1037851155 wait complete. 2024-03-29T08:26:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 Processor -- start 2024-03-29T08:26:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 -- start start 2024-03-29T08:26:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ac42ac00 0x5567ab59b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567ac42b000 0x5567ab59ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567ac42b400 0x5567ab59a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ab5da3c0 con 0x5567ac42ac00 2024-03-29T08:26:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ab5da280 con 0x5567ac42b400 2024-03-29T08:26:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4ea488c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567ab5da140 con 0x5567ac42b000 2024-03-29T08:26:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4b600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ac42ac00 0x5567ab59b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4b600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ac42ac00 0x5567ab59b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58608/0 (socket says 172.21.15.46:58608) 2024-03-29T08:26:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.228+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 learned_addr learned my addr 172.21.15.46:0/1783765290 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c4c000640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567ac42b000 0x5567ab59ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567ac42b000 msgr2=0x5567ab59ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c4b600640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567ac42b000 0x5567ab59ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567ac42b400 msgr2=0x5567ab59a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c4b600640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567ac42b400 0x5567ab59a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567ab5401a0 con 0x5567ac42ac00 2024-03-29T08:26:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c4c000640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567ac42b000 0x5567ab59ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c4b600640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ac42ac00 0x5567ab59b180 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x5567ac47f4a0 tx=0x5567ac416300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c44800640 1 -- 172.21.15.46:0/1783765290 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567ab561080 con 0x5567ac42ac00 2024-03-29T08:26:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c44800640 1 -- 172.21.15.46:0/1783765290 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5567ab56a180 con 0x5567ac42ac00 2024-03-29T08:26:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c44800640 1 -- 172.21.15.46:0/1783765290 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5567ab561600 con 0x5567ac42ac00 2024-03-29T08:26:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5567ab540000 con 0x5567ac42ac00 2024-03-29T08:26:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567ab5409c0 con 0x5567ac42ac00 2024-03-29T08:26:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c4ea488c0 4 datascan.init: resolving metadata pool 12 2024-03-29T08:26:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c44800640 1 -- 172.21.15.46:0/1783765290 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x5567ab544840 con 0x5567ac42ac00 2024-03-29T08:26:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c44800640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5567ab7fac00 0x5567ab59b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c44800640 1 -- 172.21.15.46:0/1783765290 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x5567ab54ca80 con 0x5567ac42ac00 2024-03-29T08:26:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c4ea488c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-29T08:26:23.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c4ea488c0 4 datascan.main: resolving metadata pool 12 2024-03-29T08:26:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5567ac42b800 0x5567ab59a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ac42bc00 con 0x5567ac42b800 2024-03-29T08:26:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c4c000640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5567ab7fac00 0x5567ab59b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.232+0000 7f6c4b600640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5567ac42b800 0x5567ab59a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4b600640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5567ac42b800 0x5567ab59a100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4c000640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5567ab7fac00 0x5567ab59b700 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x5567ac380a80 tx=0x5567ac4160c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.0 v2:172.21.15.46:6800/27424 1 ==== osd_op_reply(1 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42b800 2024-03-29T08:26:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4b600640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5567ac42bc00 0x5567ab8e0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:2 12.10 12:08000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab7fb000 con 0x5567ac42bc00 2024-03-29T08:26:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4ca00640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5567ac42bc00 0x5567ab8e0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4ca00640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5567ac42bc00 0x5567ab8e0000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 1 ==== osd_op_reply(2 [pgnls start_epoch 94 out=177b] v94'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4ca00640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5567ab7fb000 0x5567ab8e0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:3 12.8 12:10000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ac42a400 con 0x5567ab7fb000 2024-03-29T08:26:23.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4c000640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5567ab7fb000 0x5567ab8e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4c000640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5567ab7fb000 0x5567ab8e0580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 1 ==== osd_op_reply(3 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:4 12.18 12:18000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ac42a400 con 0x5567ab7fb000 2024-03-29T08:26:23.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 2 ==== osd_op_reply(4 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4c000640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5567ac42a400 0x5567ab8e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:5 12.4 12:20000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ac2f3400 con 0x5567ac42a400 2024-03-29T08:26:23.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4b600640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5567ac42a400 0x5567ab8e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.236+0000 7f6c4b600640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5567ac42a400 0x5567ab8e0b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.2 v2:172.21.15.46:6809/27428 1 ==== osd_op_reply(5 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42a400 2024-03-29T08:26:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:6 12.14 12:28000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ac2f3400 con 0x5567ac42b800 2024-03-29T08:26:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.0 v2:172.21.15.46:6800/27424 2 ==== osd_op_reply(6 [pgnls start_epoch 94 out=149b] v93'27 uv27 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42b800 2024-03-29T08:26:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:7 12.c 12:30000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ac2f3400 con 0x5567ab7fb000 2024-03-29T08:26:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 3 ==== osd_op_reply(7 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4c000640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x5567ac2f3400 0x5567ab8e1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:8 12.1c 12:38000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab7fbc00 con 0x5567ac2f3400 2024-03-29T08:26:23.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4ca00640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x5567ac2f3400 0x5567ab8e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4ca00640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x5567ac2f3400 0x5567ab8e1080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.5 v2:172.21.15.73:6808/27690 1 ==== osd_op_reply(8 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac2f3400 2024-03-29T08:26:23.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:9 12.2 12:40000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab7fbc00 con 0x5567ac42a400 2024-03-29T08:26:23.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.2 v2:172.21.15.46:6809/27428 2 ==== osd_op_reply(9 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42a400 2024-03-29T08:26:23.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4b600640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x5567ab7fbc00 0x5567ab8e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:10 12.12 12:48000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab7fb400 con 0x5567ab7fbc00 2024-03-29T08:26:23.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4c000640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x5567ab7fbc00 0x5567ab8e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4c000640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x5567ab7fbc00 0x5567ab8e1b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.4 v2:172.21.15.73:6800/27689 1 ==== osd_op_reply(10 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fbc00 2024-03-29T08:26:23.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:11 12.a 12:50000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab7fb400 con 0x5567ab7fbc00 2024-03-29T08:26:23.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.4 v2:172.21.15.73:6800/27689 2 ==== osd_op_reply(11 [pgnls start_epoch 94 out=81b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fbc00 2024-03-29T08:26:23.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.240+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:12 12.1a 12:58000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab7fb400 con 0x5567ac42a400 2024-03-29T08:26:23.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.2 v2:172.21.15.46:6809/27428 3 ==== osd_op_reply(12 [pgnls start_epoch 94 out=49b] v93'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42a400 2024-03-29T08:26:23.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:13 12.6 12:60000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab7fb400 con 0x5567ac42bc00 2024-03-29T08:26:23.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 2 ==== osd_op_reply(13 [pgnls start_epoch 94 out=77b] v93'8 uv8 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:14 12.16 12:68000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab7fb400 con 0x5567ac42a400 2024-03-29T08:26:23.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.2 v2:172.21.15.46:6809/27428 4 ==== osd_op_reply(14 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42a400 2024-03-29T08:26:23.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:15 12.e 12:70000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab7fb400 con 0x5567ab7fb000 2024-03-29T08:26:23.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 4 ==== osd_op_reply(15 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:16 12.1e 12:78000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab7fb400 con 0x5567ac42a400 2024-03-29T08:26:23.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.2 v2:172.21.15.46:6809/27428 5 ==== osd_op_reply(16 [pgnls start_epoch 94 out=49b] v93'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42a400 2024-03-29T08:26:23.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4b600640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5567ab7fb400 0x5567ab8e2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:17 12.1 12:80000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab7fb800 con 0x5567ab7fb400 2024-03-29T08:26:23.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4b600640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5567ab7fb400 0x5567ab8e2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4b600640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5567ab7fb400 0x5567ab8e2100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.1 v2:172.21.15.46:6824/27427 1 ==== osd_op_reply(17 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ab7fb400 2024-03-29T08:26:23.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:18 12.11 12:88000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab7fb800 con 0x5567ac42a400 2024-03-29T08:26:23.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.2 v2:172.21.15.46:6809/27428 6 ==== osd_op_reply(18 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42a400 2024-03-29T08:26:23.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:19 12.9 12:90000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab7fb800 con 0x5567ac42bc00 2024-03-29T08:26:23.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 3 ==== osd_op_reply(19 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:20 12.19 12:98000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab7fb800 con 0x5567ab7fb000 2024-03-29T08:26:23.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 5 ==== osd_op_reply(20 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:21 12.5 12:a0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab7fb800 con 0x5567ac42b800 2024-03-29T08:26:23.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.0 v2:172.21.15.46:6800/27424 3 ==== osd_op_reply(21 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42b800 2024-03-29T08:26:23.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.244+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:22 12.15 12:a8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab7fb800 con 0x5567ab7fb400 2024-03-29T08:26:23.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.1 v2:172.21.15.46:6824/27427 2 ==== osd_op_reply(22 [pgnls start_epoch 94 out=97b] v93'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ab7fb400 2024-03-29T08:26:23.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4b600640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5567ab7fb800 0x5567ab8e2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:23 12.d 12:b0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908000 con 0x5567ab7fb800 2024-03-29T08:26:23.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4ca00640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5567ab7fb800 0x5567ab8e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4ca00640 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5567ab7fb800 0x5567ab8e2680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.3 v2:172.21.15.46:6808/27426 1 ==== osd_op_reply(23 [pgnls start_epoch 94 out=74b] v91'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ab7fb800 2024-03-29T08:26:23.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:24 12.1d 12:b8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908000 con 0x5567ac42bc00 2024-03-29T08:26:23.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 4 ==== osd_op_reply(24 [pgnls start_epoch 94 out=73b] v93'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:25 12.3 12:c0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908000 con 0x5567ab7fbc00 2024-03-29T08:26:23.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.4 v2:172.21.15.73:6800/27689 3 ==== osd_op_reply(25 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fbc00 2024-03-29T08:26:23.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:26 12.13 12:c8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908000 con 0x5567ac2f3400 2024-03-29T08:26:23.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.5 v2:172.21.15.73:6808/27690 2 ==== osd_op_reply(26 [pgnls start_epoch 94 out=98b] v93'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac2f3400 2024-03-29T08:26:23.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:27 12.b 12:d0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908000 con 0x5567ab7fb000 2024-03-29T08:26:23.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 6 ==== osd_op_reply(27 [pgnls start_epoch 94 out=76b] v93'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:28 12.1b 12:d8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908000 con 0x5567ac42b800 2024-03-29T08:26:23.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.0 v2:172.21.15.46:6800/27424 4 ==== osd_op_reply(28 [pgnls start_epoch 94 out=73b] v93'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42b800 2024-03-29T08:26:23.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:29 12.7 12:e0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908000 con 0x5567ab7fb000 2024-03-29T08:26:23.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 7 ==== osd_op_reply(29 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:30 12.17 12:e8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908000 con 0x5567ab7fb400 2024-03-29T08:26:23.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.1 v2:172.21.15.46:6824/27427 3 ==== osd_op_reply(30 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ab7fb400 2024-03-29T08:26:23.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:31 12.f 12:f0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908000 con 0x5567ab7fb800 2024-03-29T08:26:23.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.3 v2:172.21.15.46:6808/27426 2 ==== osd_op_reply(31 [pgnls start_epoch 94 out=73b] v91'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ab7fb800 2024-03-29T08:26:23.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.248+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:32 12.1f 12:f8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908000 con 0x5567ab7fb800 2024-03-29T08:26:23.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.3 v2:172.21.15.46:6808/27426 3 ==== osd_op_reply(32 [pgnls start_epoch 94 out=101b] v93'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ab7fb800 2024-03-29T08:26:23.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-03-29T08:26:23.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:33 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab909000 con 0x5567ac42bc00 2024-03-29T08:26:23.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-03-29T08:26:23.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:34 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab909400 con 0x5567ac42bc00 2024-03-29T08:26:23.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-03-29T08:26:23.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:35 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab909800 con 0x5567ac42bc00 2024-03-29T08:26:23.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 20 datascan.scan_links: injected ino detected 2024-03-29T08:26:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-03-29T08:26:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:36 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab909c00 con 0x5567ac42bc00 2024-03-29T08:26:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-03-29T08:26:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:37 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908000 con 0x5567ac42bc00 2024-03-29T08:26:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-03-29T08:26:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-29T08:26:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-03-29T08:26:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-29T08:26:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-03-29T08:26:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-29T08:26:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-03-29T08:26:23.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:38 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab909000 con 0x5567ac42b800 2024-03-29T08:26:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.0 v2:172.21.15.46:6800/27424 5 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv25 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42b800 2024-03-29T08:26:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-03-29T08:26:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:39 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab909400 con 0x5567ac2f3400 2024-03-29T08:26:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.5 v2:172.21.15.73:6808/27690 3 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac2f3400 2024-03-29T08:26:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-03-29T08:26:23.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:40 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab909800 con 0x5567ab7fbc00 2024-03-29T08:26:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.4 v2:172.21.15.73:6800/27689 4 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fbc00 2024-03-29T08:26:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-03-29T08:26:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-29T08:26:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-03-29T08:26:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:41 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ac42a800 con 0x5567ab7fb000 2024-03-29T08:26:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 8 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-03-29T08:26:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:42 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90c000 con 0x5567ab7fb400 2024-03-29T08:26:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.1 v2:172.21.15.46:6824/27427 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ab7fb400 2024-03-29T08:26:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-03-29T08:26:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-29T08:26:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-03-29T08:26:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-29T08:26:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-03-29T08:26:23.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.252+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:43 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90c400 con 0x5567ac42bc00 2024-03-29T08:26:23.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 10 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-03-29T08:26:23.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-29T08:26:23.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-03-29T08:26:23.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:44 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90c800 con 0x5567ac2f3400 2024-03-29T08:26:23.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.5 v2:172.21.15.73:6808/27690 4 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac2f3400 2024-03-29T08:26:23.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-03-29T08:26:23.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-29T08:26:23.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 200.00000002 2024-03-29T08:26:23.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-03-29T08:26:23.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-03-29T08:26:23.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:45 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90cc00 con 0x5567ab7fb000 2024-03-29T08:26:23.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 9 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-03-29T08:26:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-29T08:26:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-03-29T08:26:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-29T08:26:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ea488c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-03-29T08:26:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:46 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90d000 con 0x5567ab7fb800 2024-03-29T08:26:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.3 v2:172.21.15.46:6808/27426 4 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1041 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ab7fb800 2024-03-29T08:26:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:47 12.0 12:00000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90d400 con 0x5567ac42b800 2024-03-29T08:26:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.0 v2:172.21.15.46:6800/27424 6 ==== osd_op_reply(47 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42b800 2024-03-29T08:26:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:48 12.10 12:08000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ac42bc00 2024-03-29T08:26:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 11 ==== osd_op_reply(48 [pgnls start_epoch 94 out=177b] v94'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:49 12.8 12:10000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ab7fb000 2024-03-29T08:26:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 10 ==== osd_op_reply(49 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:50 12.18 12:18000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ab7fb000 2024-03-29T08:26:23.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 11 ==== osd_op_reply(50 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:51 12.4 12:20000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ac42a400 2024-03-29T08:26:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.2 v2:172.21.15.46:6809/27428 7 ==== osd_op_reply(51 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42a400 2024-03-29T08:26:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:52 12.14 12:28000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ac42b800 2024-03-29T08:26:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.0 v2:172.21.15.46:6800/27424 7 ==== osd_op_reply(52 [pgnls start_epoch 94 out=149b] v93'27 uv27 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42b800 2024-03-29T08:26:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.256+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:53 12.c 12:30000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ab7fb000 2024-03-29T08:26:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 12 ==== osd_op_reply(53 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:54 12.1c 12:38000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ac2f3400 2024-03-29T08:26:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.5 v2:172.21.15.73:6808/27690 5 ==== osd_op_reply(54 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac2f3400 2024-03-29T08:26:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:55 12.2 12:40000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ac42a400 2024-03-29T08:26:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.2 v2:172.21.15.46:6809/27428 8 ==== osd_op_reply(55 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42a400 2024-03-29T08:26:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:56 12.12 12:48000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ab7fbc00 2024-03-29T08:26:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.4 v2:172.21.15.73:6800/27689 5 ==== osd_op_reply(56 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fbc00 2024-03-29T08:26:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:57 12.a 12:50000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ab7fbc00 2024-03-29T08:26:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.4 v2:172.21.15.73:6800/27689 6 ==== osd_op_reply(57 [pgnls start_epoch 94 out=81b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fbc00 2024-03-29T08:26:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:58 12.1a 12:58000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ac42a400 2024-03-29T08:26:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.2 v2:172.21.15.46:6809/27428 9 ==== osd_op_reply(58 [pgnls start_epoch 94 out=49b] v93'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42a400 2024-03-29T08:26:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:59 12.6 12:60000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ac42bc00 2024-03-29T08:26:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 12 ==== osd_op_reply(59 [pgnls start_epoch 94 out=77b] v93'8 uv8 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:60 12.16 12:68000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ac42a400 2024-03-29T08:26:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.2 v2:172.21.15.46:6809/27428 10 ==== osd_op_reply(60 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42a400 2024-03-29T08:26:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:61 12.e 12:70000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ab7fb000 2024-03-29T08:26:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 13 ==== osd_op_reply(61 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:62 12.1e 12:78000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ac42a400 2024-03-29T08:26:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.2 v2:172.21.15.46:6809/27428 11 ==== osd_op_reply(62 [pgnls start_epoch 94 out=49b] v93'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42a400 2024-03-29T08:26:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:63 12.1 12:80000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ab7fb400 2024-03-29T08:26:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.1 v2:172.21.15.46:6824/27427 5 ==== osd_op_reply(63 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ab7fb400 2024-03-29T08:26:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:64 12.11 12:88000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ac42a400 2024-03-29T08:26:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.2 v2:172.21.15.46:6809/27428 12 ==== osd_op_reply(64 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42a400 2024-03-29T08:26:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.260+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:65 12.9 12:90000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ac42bc00 2024-03-29T08:26:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 13 ==== osd_op_reply(65 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:66 12.19 12:98000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ab7fb000 2024-03-29T08:26:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 14 ==== osd_op_reply(66 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:67 12.5 12:a0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ac42b800 2024-03-29T08:26:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.0 v2:172.21.15.46:6800/27424 8 ==== osd_op_reply(67 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42b800 2024-03-29T08:26:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:68 12.15 12:a8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ab7fb400 2024-03-29T08:26:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.1 v2:172.21.15.46:6824/27427 6 ==== osd_op_reply(68 [pgnls start_epoch 94 out=97b] v93'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ab7fb400 2024-03-29T08:26:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:69 12.d 12:b0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ab7fb800 2024-03-29T08:26:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.3 v2:172.21.15.46:6808/27426 5 ==== osd_op_reply(69 [pgnls start_epoch 94 out=74b] v91'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ab7fb800 2024-03-29T08:26:23.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:70 12.1d 12:b8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ac42bc00 2024-03-29T08:26:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 14 ==== osd_op_reply(70 [pgnls start_epoch 94 out=73b] v93'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:71 12.3 12:c0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ab7fbc00 2024-03-29T08:26:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.4 v2:172.21.15.73:6800/27689 7 ==== osd_op_reply(71 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fbc00 2024-03-29T08:26:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:72 12.13 12:c8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ac2f3400 2024-03-29T08:26:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.5 v2:172.21.15.73:6808/27690 6 ==== osd_op_reply(72 [pgnls start_epoch 94 out=98b] v93'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac2f3400 2024-03-29T08:26:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:73 12.b 12:d0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ab7fb000 2024-03-29T08:26:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 15 ==== osd_op_reply(73 [pgnls start_epoch 94 out=76b] v93'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:74 12.1b 12:d8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ac42b800 2024-03-29T08:26:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.0 v2:172.21.15.46:6800/27424 9 ==== osd_op_reply(74 [pgnls start_epoch 94 out=73b] v93'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42b800 2024-03-29T08:26:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:75 12.7 12:e0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ab7fb000 2024-03-29T08:26:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 16 ==== osd_op_reply(75 [pgnls start_epoch 94 out=73b] v93'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:76 12.17 12:e8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ab7fb400 2024-03-29T08:26:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.1 v2:172.21.15.46:6824/27427 7 ==== osd_op_reply(76 [pgnls start_epoch 94 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ab7fb400 2024-03-29T08:26:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:77 12.f 12:f0000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ab7fb800 2024-03-29T08:26:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.3 v2:172.21.15.46:6808/27426 6 ==== osd_op_reply(77 [pgnls start_epoch 94 out=73b] v91'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ab7fb800 2024-03-29T08:26:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.264+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:78 12.1f 12:f8000000::::head [pgnls start_epoch 94 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ab7fb800 2024-03-29T08:26:23.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.3 v2:172.21.15.46:6808/27426 7 ==== osd_op_reply(78 [pgnls start_epoch 94 out=101b] v93'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ab7fb800 2024-03-29T08:26:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-03-29T08:26:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:79 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90d800 con 0x5567ac42bc00 2024-03-29T08:26:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 15 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-03-29T08:26:23.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:80 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90dc00 con 0x5567ac42bc00 2024-03-29T08:26:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 16 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-03-29T08:26:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:81 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90c000 con 0x5567ac42bc00 2024-03-29T08:26:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 17 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 20 datascan.scan_links: injected ino detected 2024-03-29T08:26:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-03-29T08:26:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-03-29T08:26:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-03-29T08:26:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:82 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90d400 con 0x5567ac42bc00 2024-03-29T08:26:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 18 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-03-29T08:26:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:83 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90c400 con 0x5567ac42bc00 2024-03-29T08:26:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 19 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-03-29T08:26:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-29T08:26:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-03-29T08:26:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-29T08:26:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-03-29T08:26:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-29T08:26:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-03-29T08:26:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:84 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90c800 con 0x5567ac42b800 2024-03-29T08:26:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.0 v2:172.21.15.46:6800/27424 10 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv25 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ac42b800 2024-03-29T08:26:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-03-29T08:26:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:85 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90d000 con 0x5567ac2f3400 2024-03-29T08:26:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.5 v2:172.21.15.73:6808/27690 7 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac2f3400 2024-03-29T08:26:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-03-29T08:26:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:86 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab909400 con 0x5567ab7fbc00 2024-03-29T08:26:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.4 v2:172.21.15.73:6800/27689 8 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fbc00 2024-03-29T08:26:23.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-03-29T08:26:23.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-29T08:26:23.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-03-29T08:26:23.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:87 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90d800 con 0x5567ab7fb000 2024-03-29T08:26:23.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 17 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-03-29T08:26:23.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:88 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90dc00 con 0x5567ab7fb400 2024-03-29T08:26:23.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4b600640 1 -- 172.21.15.46:0/1783765290 <== osd.1 v2:172.21.15.46:6824/27427 8 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26ab40 con 0x5567ab7fb400 2024-03-29T08:26:23.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-03-29T08:26:23.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-29T08:26:23.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-03-29T08:26:23.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-29T08:26:23.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-03-29T08:26:23.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90c000 con 0x5567ac42bc00 2024-03-29T08:26:23.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 20 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-03-29T08:26:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-29T08:26:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-03-29T08:26:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.268+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:90 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90d400 con 0x5567ac2f3400 2024-03-29T08:26:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.5 v2:172.21.15.73:6808/27690 8 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac2f3400 2024-03-29T08:26:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-03-29T08:26:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-29T08:26:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 200.00000002 2024-03-29T08:26:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-03-29T08:26:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-03-29T08:26:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:91 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90c400 con 0x5567ab7fb000 2024-03-29T08:26:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 <== osd.7 v2:172.21.15.73:6816/27700 18 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5567ac26afc0 con 0x5567ab7fb000 2024-03-29T08:26:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-03-29T08:26:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-29T08:26:23.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-03-29T08:26:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-29T08:26:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-03-29T08:26:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:92 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90d000 con 0x5567ab7fb800 2024-03-29T08:26:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.3 v2:172.21.15.46:6808/27426 8 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1041 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ab7fb800 2024-03-29T08:26:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-03-29T08:26:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-03-29T08:26:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-03-29T08:26:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-03-29T08:26:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:93 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab908c00 con 0x5567ac42bc00 2024-03-29T08:26:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 21 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-03-29T08:26:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:94 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90cc00 con 0x5567ac42bc00 2024-03-29T08:26:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 22 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v94'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-03-29T08:26:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 10 datascan.scan_links: processing 1 injected_inos 2024-03-29T08:26:23.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-03-29T08:26:23.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-03-29T08:26:23.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:95 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90d800 con 0x5567ac42bc00 2024-03-29T08:26:23.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 23 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-03-29T08:26:23.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 20 datascan.scan_links: first is now 2 2024-03-29T08:26:23.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.272+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:96 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab90c400 con 0x5567ac42bc00 2024-03-29T08:26:23.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.6 v2:172.21.15.73:6824/27691 24 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v94'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac42bc00 2024-03-29T08:26:23.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-03-29T08:26:23.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 10 datascan.scan_links: updating inotable 2024-03-29T08:26:23.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:97 12.13 12:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab909800 con 0x5567ac2f3400 2024-03-29T08:26:23.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.5 v2:172.21.15.73:6808/27690 9 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac2f3400 2024-03-29T08:26:23.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-03-29T08:26:23.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-03-29T08:26:23.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:98 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ab909400 con 0x5567ac2f3400 2024-03-29T08:26:23.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.5 v2:172.21.15.73:6808/27690 10 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v94'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ac2f3400 2024-03-29T08:26:23.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 10 datascan.scan_links: updating snaptable 2024-03-29T08:26:23.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:99 12.d 12:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e94) v8 -- 0x5567ac42a800 con 0x5567ab7fb800 2024-03-29T08:26:23.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ca00640 1 -- 172.21.15.46:0/1783765290 <== osd.3 v2:172.21.15.46:6808/27426 9 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x5567ac26ad80 con 0x5567ab7fb800 2024-03-29T08:26:23.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5567ac42b800 msgr2=0x5567ab59a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5567ac42b800 0x5567ab59a100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5567ab7fb400 msgr2=0x5567ab8e2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5567ab7fb400 0x5567ab8e2100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5567ac42a400 msgr2=0x5567ab8e0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5567ac42a400 0x5567ab8e0b00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5567ab7fb800 msgr2=0x5567ab8e2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 reap_dead start 2024-03-29T08:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5567ab7fb800 0x5567ab8e2680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x5567ab7fbc00 msgr2=0x5567ab8e1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x5567ab7fbc00 0x5567ab8e1b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x5567ac2f3400 msgr2=0x5567ab8e1080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x5567ac2f3400 0x5567ab8e1080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5567ac42bc00 msgr2=0x5567ab8e0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5567ac42bc00 0x5567ab8e0000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5567ab7fb000 msgr2=0x5567ab8e0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5567ab7fb000 0x5567ab8e0580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4c000640 1 -- 172.21.15.46:0/1783765290 reap_dead start 2024-03-29T08:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5567ab7fac00 msgr2=0x5567ab59b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5567ab7fac00 0x5567ab59b700 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x5567ac380a80 tx=0x5567ac4160c0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ac42ac00 msgr2=0x5567ab59b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ac42ac00 0x5567ab59b180 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x5567ac47f4a0 tx=0x5567ac416300 comp rx=0 tx=0).stop 2024-03-29T08:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 shutdown_connections 2024-03-29T08:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5567ab7fac00 0x5567ab59b700 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/1783765290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ac42ac00 0x5567ab59b180 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 >> 172.21.15.46:0/1783765290 conn(0x5567ab50cc00 msgr2=0x5567ab539800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 shutdown_connections 2024-03-29T08:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/1783765290 wait complete. 2024-03-29T08:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/3566571608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567ab50d000 msgr2=0x5567ab599080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/3566571608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567ab50d000 0x5567ab599080 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x5567ac381290 tx=0x5567ac416360 comp rx=0 tx=0).stop 2024-03-29T08:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/3566571608 shutdown_connections 2024-03-29T08:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/3566571608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5567ab50d000 0x5567ab599080 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/3566571608 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5567ab50d400 0x5567ab598b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 --2- 172.21.15.46:0/3566571608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5567ab50d800 0x5567ab598580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/3566571608 >> 172.21.15.46:0/3566571608 conn(0x5567ab50dc00 msgr2=0x5567ab539000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:23.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/3566571608 shutdown_connections 2024-03-29T08:26:23.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.276+0000 7f6c4ea488c0 1 -- 172.21.15.46:0/3566571608 wait complete. 2024-03-29T08:26:23.287 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.148734, result: 2024-03-29T08:26:23.287 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-29T08:26:23.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.468+0000 7fa282400640 1 Processor -- start 2024-03-29T08:26:23.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.468+0000 7fa282400640 1 -- start start 2024-03-29T08:26:23.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.468+0000 7fa282400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa27c142a50 0x7fa27c0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.468+0000 7fa282400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa27c0ae990 0x7fa27c0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.468+0000 7fa282400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa27c15d260 con 0x7fa27c15df90 2024-03-29T08:26:23.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.468+0000 7fa282400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa27c0aefd0 con 0x7fa27c0ae990 2024-03-29T08:26:23.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.468+0000 7fa282400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c15d4b0 con 0x7fa27c142a50 2024-03-29T08:26:23.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.468+0000 7fa27be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa27c0ae990 0x7fa27c0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.468+0000 7fa27be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa27c0ae990 0x7fa27c0cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58618/0 (socket says 172.21.15.46:58618) 2024-03-29T08:26:23.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.468+0000 7fa27be00640 1 -- 172.21.15.46:0/140959628 learned_addr learned my addr 172.21.15.46:0/140959628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:23.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.468+0000 7fa27b400640 1 --2- 172.21.15.46:0/140959628 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa27c142a50 0x7fa27c0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.468+0000 7fa27be00640 1 -- 172.21.15.46:0/140959628 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa27c142a50 msgr2=0x7fa27c0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:23.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.468+0000 7fa27be00640 1 --2- 172.21.15.46:0/140959628 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa27c142a50 0x7fa27c0cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.468+0000 7fa27be00640 1 -- 172.21.15.46:0/140959628 >> v1:172.21.15.46:6789/0 conn(0x7fa27c15df90 legacy=0x7fa27c0ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.468+0000 7fa27be00640 1 -- 172.21.15.46:0/140959628 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c15d790 con 0x7fa27c0ae990 2024-03-29T08:26:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.472+0000 7fa27be00640 1 --2- 172.21.15.46:0/140959628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c0ae990 0x7fa27c0cbcd0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fa274009870 tx=0x7fa27401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6be1f0f7cf5812 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.472+0000 7fa27aa00640 1 -- 172.21.15.46:0/140959628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2740ab600 con 0x7fa27c0ae990 2024-03-29T08:26:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.472+0000 7fa27aa00640 1 -- 172.21.15.46:0/140959628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2740a7050 con 0x7fa27c0ae990 2024-03-29T08:26:23.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.472+0000 7fa27aa00640 1 -- 172.21.15.46:0/140959628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2740ab980 con 0x7fa27c0ae990 2024-03-29T08:26:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.472+0000 7fa282400640 1 -- 172.21.15.46:0/140959628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c0ae990 msgr2=0x7fa27c0cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.472+0000 7fa282400640 1 --2- 172.21.15.46:0/140959628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c0ae990 0x7fa27c0cbcd0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fa274009870 tx=0x7fa27401c4d0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.472+0000 7fa282400640 1 -- 172.21.15.46:0/140959628 shutdown_connections 2024-03-29T08:26:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.472+0000 7fa282400640 1 --2- 172.21.15.46:0/140959628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c0ae990 0x7fa27c0cbcd0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.472+0000 7fa282400640 1 --2- 172.21.15.46:0/140959628 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa27c142a50 0x7fa27c0cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.472+0000 7fa282400640 1 -- 172.21.15.46:0/140959628 >> 172.21.15.46:0/140959628 conn(0x7fa27c06a390 msgr2=0x7fa27c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.472+0000 7fa282400640 1 -- 172.21.15.46:0/140959628 shutdown_connections 2024-03-29T08:26:23.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.472+0000 7fa282400640 1 -- 172.21.15.46:0/140959628 wait complete. 2024-03-29T08:26:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa282400640 1 Processor -- start 2024-03-29T08:26:23.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa282400640 1 -- start start 2024-03-29T08:26:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa282400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa27c0ae990 0x7fa27c13bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa282400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa27c142a50 0x7fa27c15e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa282400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c15df90 0x7fa27c0ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa282400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c0abd80 con 0x7fa27c15df90 2024-03-29T08:26:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa282400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c0a89f0 con 0x7fa27c0ae990 2024-03-29T08:26:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa282400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c0a8cd0 con 0x7fa27c142a50 2024-03-29T08:26:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa27be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c15df90 0x7fa27c0ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa27be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c15df90 0x7fa27c0ab2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58630/0 (socket says 172.21.15.46:58630) 2024-03-29T08:26:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa27be00640 1 -- 172.21.15.46:0/2810117307 learned_addr learned my addr 172.21.15.46:0/2810117307 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa27b400640 1 --2- 172.21.15.46:0/2810117307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa27c0ae990 0x7fa27c13bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa272a00640 1 --2- 172.21.15.46:0/2810117307 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa27c142a50 0x7fa27c15e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa27be00640 1 -- 172.21.15.46:0/2810117307 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa27c142a50 msgr2=0x7fa27c15e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa27be00640 1 --2- 172.21.15.46:0/2810117307 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa27c142a50 0x7fa27c15e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa27be00640 1 -- 172.21.15.46:0/2810117307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa27c0ae990 msgr2=0x7fa27c13bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa27be00640 1 --2- 172.21.15.46:0/2810117307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa27c0ae990 0x7fa27c13bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa27be00640 1 -- 172.21.15.46:0/2810117307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2740094d0 con 0x7fa27c15df90 2024-03-29T08:26:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa27be00640 1 --2- 172.21.15.46:0/2810117307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c15df90 0x7fa27c0ab2c0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa27401a100 tx=0x7fa274019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa273e00640 1 -- 172.21.15.46:0/2810117307 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2740b5600 con 0x7fa27c15df90 2024-03-29T08:26:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa273e00640 1 -- 172.21.15.46:0/2810117307 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2740b5760 con 0x7fa27c15df90 2024-03-29T08:26:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa273e00640 1 -- 172.21.15.46:0/2810117307 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2740b5b10 con 0x7fa27c15df90 2024-03-29T08:26:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa282400640 1 -- 172.21.15.46:0/2810117307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27c0a8fb0 con 0x7fa27c15df90 2024-03-29T08:26:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.476+0000 7fa282400640 1 -- 172.21.15.46:0/2810117307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27c159140 con 0x7fa27c15df90 2024-03-29T08:26:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.480+0000 7fa282400640 1 -- 172.21.15.46:0/2810117307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27c071a70 con 0x7fa27c15df90 2024-03-29T08:26:23.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.480+0000 7fa273e00640 1 -- 172.21.15.46:0/2810117307 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa2740b5c70 con 0x7fa27c15df90 2024-03-29T08:26:23.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.480+0000 7fa273e00640 1 --2- 172.21.15.46:0/2810117307 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa258066690 0x7fa258068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:23.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.480+0000 7fa273e00640 1 -- 172.21.15.46:0/2810117307 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fa27411e8f0 con 0x7fa27c15df90 2024-03-29T08:26:23.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.480+0000 7fa27b400640 1 --2- 172.21.15.46:0/2810117307 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa258066690 0x7fa258068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:23.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.480+0000 7fa27b400640 1 --2- 172.21.15.46:0/2810117307 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa258066690 0x7fa258068b50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009b60 tx=0x7fa26c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:23.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.484+0000 7fa273e00640 1 -- 172.21.15.46:0/2810117307 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa2740ebc50 con 0x7fa27c15df90 2024-03-29T08:26:23.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.680+0000 7fa282400640 1 -- 172.21.15.46:0/2810117307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fa27c0a9460 con 0x7fa27c15df90 2024-03-29T08:26:23.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.860+0000 7fa273e00640 1 -- 172.21.15.46:0/2810117307 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v65) v1 ==== 162+0+0 (secure 0 0 0) 0x7fa2740b90b0 con 0x7fa27c15df90 2024-03-29T08:26:23.863 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-29T08:26:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.864+0000 7fa282400640 1 -- 172.21.15.46:0/2810117307 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa258066690 msgr2=0x7fa258068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.864+0000 7fa282400640 1 --2- 172.21.15.46:0/2810117307 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa258066690 0x7fa258068b50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fa26c009b60 tx=0x7fa26c009190 comp rx=0 tx=0).stop 2024-03-29T08:26:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.864+0000 7fa282400640 1 -- 172.21.15.46:0/2810117307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c15df90 msgr2=0x7fa27c0ab2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:23.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.864+0000 7fa282400640 1 --2- 172.21.15.46:0/2810117307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c15df90 0x7fa27c0ab2c0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fa27401a100 tx=0x7fa274019270 comp rx=0 tx=0).stop 2024-03-29T08:26:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.864+0000 7fa282400640 1 -- 172.21.15.46:0/2810117307 shutdown_connections 2024-03-29T08:26:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.864+0000 7fa282400640 1 --2- 172.21.15.46:0/2810117307 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa258066690 0x7fa258068b50 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.864+0000 7fa282400640 1 --2- 172.21.15.46:0/2810117307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa27c15df90 0x7fa27c0ab2c0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.864+0000 7fa282400640 1 --2- 172.21.15.46:0/2810117307 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa27c142a50 0x7fa27c15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.864+0000 7fa282400640 1 --2- 172.21.15.46:0/2810117307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa27c0ae990 0x7fa27c13bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:23.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.864+0000 7fa282400640 1 -- 172.21.15.46:0/2810117307 >> 172.21.15.46:0/2810117307 conn(0x7fa27c06a390 msgr2=0x7fa27c0b0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:23.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.864+0000 7fa282400640 1 -- 172.21.15.46:0/2810117307 shutdown_connections 2024-03-29T08:26:23.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:23.864+0000 7fa282400640 1 -- 172.21.15.46:0/2810117307 wait complete. 2024-03-29T08:26:23.889 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:26:24.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2de1000640 1 Processor -- start 2024-03-29T08:26:24.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2de1000640 1 -- start start 2024-03-29T08:26:24.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2de1000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2ddc15feb0 0x7f2ddc071370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:24.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2de1000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2ddc1605f0 0x7f2ddc069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:24.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2de1000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2ddc160a80 0x7f2ddc06a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:24.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2de1000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ddc161020 con 0x7f2ddc1605f0 2024-03-29T08:26:24.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2de1000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ddc161300 con 0x7f2ddc15feb0 2024-03-29T08:26:24.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2de1000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc070b70 con 0x7f2ddc160a80 2024-03-29T08:26:24.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2dd1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2ddc1605f0 0x7f2ddc069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:24.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2dd1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2ddc1605f0 0x7f2ddc069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58634/0 (socket says 172.21.15.46:58634) 2024-03-29T08:26:24.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2dd1400640 1 -- 172.21.15.46:0/2117588944 learned_addr learned my addr 172.21.15.46:0/2117588944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:24.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2dd9e00640 1 --2- 172.21.15.46:0/2117588944 >> v2:172.21.15.73:3300/0 conn(0x7f2ddc15feb0 0x7f2ddc071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:24.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2dda800640 1 --2- 172.21.15.46:0/2117588944 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2ddc160a80 0x7f2ddc06a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:24.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2dd1400640 1 -- 172.21.15.46:0/2117588944 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2ddc160a80 msgr2=0x7f2ddc06a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:24.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2dd1400640 1 --2- 172.21.15.46:0/2117588944 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2ddc160a80 0x7f2ddc06a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:24.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2dd1400640 1 -- 172.21.15.46:0/2117588944 >> v2:172.21.15.73:3300/0 conn(0x7f2ddc15feb0 msgr2=0x7f2ddc071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:24.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2dd1400640 1 --2- 172.21.15.46:0/2117588944 >> v2:172.21.15.73:3300/0 conn(0x7f2ddc15feb0 0x7f2ddc071370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:24.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.028+0000 7f2dd1400640 1 -- 172.21.15.46:0/2117588944 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc070e50 con 0x7f2ddc1605f0 2024-03-29T08:26:24.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.032+0000 7f2dd1400640 1 --2- 172.21.15.46:0/2117588944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc1605f0 0x7f2ddc069d90 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0cca40 tx=0x7f2dc401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4f170a1da9d5dc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:24.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.032+0000 7f2dd9400640 1 -- 172.21.15.46:0/2117588944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc40a5800 con 0x7f2ddc1605f0 2024-03-29T08:26:24.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.032+0000 7f2dd9400640 1 -- 172.21.15.46:0/2117588944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc40a5960 con 0x7f2ddc1605f0 2024-03-29T08:26:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.032+0000 7f2dd9400640 1 -- 172.21.15.46:0/2117588944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc40a5cf0 con 0x7f2ddc1605f0 2024-03-29T08:26:24.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.032+0000 7f2de1000640 1 -- 172.21.15.46:0/2117588944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc1605f0 msgr2=0x7f2ddc069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.032+0000 7f2de1000640 1 --2- 172.21.15.46:0/2117588944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc1605f0 0x7f2ddc069d90 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc0cca40 tx=0x7f2dc401c960 comp rx=0 tx=0).stop 2024-03-29T08:26:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.032+0000 7f2de1000640 1 -- 172.21.15.46:0/2117588944 shutdown_connections 2024-03-29T08:26:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.032+0000 7f2de1000640 1 --2- 172.21.15.46:0/2117588944 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2ddc160a80 0x7f2ddc06a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.032+0000 7f2de1000640 1 --2- 172.21.15.46:0/2117588944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc1605f0 0x7f2ddc069d90 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.032+0000 7f2de1000640 1 --2- 172.21.15.46:0/2117588944 >> v2:172.21.15.73:3300/0 conn(0x7f2ddc15feb0 0x7f2ddc071370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.032+0000 7f2de1000640 1 -- 172.21.15.46:0/2117588944 >> 172.21.15.46:0/2117588944 conn(0x7f2ddc0ced50 msgr2=0x7f2ddc0cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.032+0000 7f2de1000640 1 -- 172.21.15.46:0/2117588944 shutdown_connections 2024-03-29T08:26:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.032+0000 7f2de1000640 1 -- 172.21.15.46:0/2117588944 wait complete. 2024-03-29T08:26:24.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2de1000640 1 Processor -- start 2024-03-29T08:26:24.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2de1000640 1 -- start start 2024-03-29T08:26:24.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2de1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc15feb0 0x7f2ddc15f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2de1000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2ddc1605f0 0x7f2ddc14e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2de1000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ddc160a80 0x7f2ddc153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2de1000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc15f790 con 0x7f2ddc15feb0 2024-03-29T08:26:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2de1000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc0aea60 con 0x7f2ddc160a80 2024-03-29T08:26:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2de1000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc154de0 con 0x7f2ddc1605f0 2024-03-29T08:26:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2dd9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc15feb0 0x7f2ddc15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2dd9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc15feb0 0x7f2ddc15f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58642/0 (socket says 172.21.15.46:58642) 2024-03-29T08:26:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2dd9e00640 1 -- 172.21.15.46:0/3713612684 learned_addr learned my addr 172.21.15.46:0/3713612684 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2dda800640 1 --2- 172.21.15.46:0/3713612684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ddc160a80 0x7f2ddc153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2dd1400640 1 --2- 172.21.15.46:0/3713612684 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2ddc1605f0 0x7f2ddc14e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2dd9e00640 1 -- 172.21.15.46:0/3713612684 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2ddc1605f0 msgr2=0x7f2ddc14e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2dd9e00640 1 --2- 172.21.15.46:0/3713612684 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2ddc1605f0 0x7f2ddc14e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:24.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2dd9e00640 1 -- 172.21.15.46:0/3713612684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ddc160a80 msgr2=0x7f2ddc153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:24.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2dd9e00640 1 --2- 172.21.15.46:0/3713612684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ddc160a80 0x7f2ddc153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:24.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2dd9e00640 1 -- 172.21.15.46:0/3713612684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc4009b30 con 0x7f2ddc15feb0 2024-03-29T08:26:24.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2dd1400640 1 --2- 172.21.15.46:0/3713612684 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2ddc1605f0 0x7f2ddc14e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:24.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2dda800640 1 --2- 172.21.15.46:0/3713612684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ddc160a80 0x7f2ddc153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:24.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2dd9e00640 1 --2- 172.21.15.46:0/3713612684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc15feb0 0x7f2ddc15f250 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00d830 tx=0x7f2dcc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:24.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2dd2a00640 1 -- 172.21.15.46:0/3713612684 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc00b6c0 con 0x7f2ddc15feb0 2024-03-29T08:26:24.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2dd2a00640 1 -- 172.21.15.46:0/3713612684 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dcc00b820 con 0x7f2ddc15feb0 2024-03-29T08:26:24.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2dd2a00640 1 -- 172.21.15.46:0/3713612684 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc00bb90 con 0x7f2ddc15feb0 2024-03-29T08:26:24.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2de1000640 1 -- 172.21.15.46:0/3713612684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ddc14da80 con 0x7f2ddc15feb0 2024-03-29T08:26:24.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.036+0000 7f2de1000640 1 -- 172.21.15.46:0/3713612684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ddc14dd20 con 0x7f2ddc15feb0 2024-03-29T08:26:24.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.040+0000 7f2de1000640 1 -- 172.21.15.46:0/3713612684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ddc065cc0 con 0x7f2ddc15feb0 2024-03-29T08:26:24.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.040+0000 7f2dd2a00640 1 -- 172.21.15.46:0/3713612684 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2dcc01e460 con 0x7f2ddc15feb0 2024-03-29T08:26:24.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.040+0000 7f2dd2a00640 1 --2- 172.21.15.46:0/3713612684 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2db8066690 0x7f2db8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:24.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.040+0000 7f2dd2a00640 1 -- 172.21.15.46:0/3713612684 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f2dcc084ab0 con 0x7f2ddc15feb0 2024-03-29T08:26:24.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.040+0000 7f2dd1400640 1 --2- 172.21.15.46:0/3713612684 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2db8066690 0x7f2db8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:24.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.040+0000 7f2dd1400640 1 --2- 172.21.15.46:0/3713612684 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2db8066690 0x7f2db8068b50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40ae000 tx=0x7f2dc40a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:24.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.044+0000 7f2dd2a00640 1 -- 172.21.15.46:0/3713612684 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2dcc051cd0 con 0x7f2ddc15feb0 2024-03-29T08:26:24.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.240+0000 7f2de1000640 1 -- 172.21.15.46:0/3713612684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ddc066c90 con 0x7f2ddc15feb0 2024-03-29T08:26:24.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.240+0000 7f2dd2a00640 1 -- 172.21.15.46:0/3713612684 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f2dcc0552f0 con 0x7f2ddc15feb0 2024-03-29T08:26:24.243 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:26:24.243 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/1862652020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1862652020},{"type":"v1","addr":"172.21.15.73:6836","nonce":1862652020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:24:43.476334+0000","modified":"2024-03-29T08:26:23.864491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7986":{"gid":7986,"name":"c","rank":0,"incarnation":66,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6835/1886680593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1886680593},{"type":"v1","addr":"172.21.15.46:6835","nonce":1886680593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-29T08:26:24.243 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 66 2024-03-29T08:26:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.244+0000 7f2de1000640 1 -- 172.21.15.46:0/3713612684 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2db8066690 msgr2=0x7f2db8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.244+0000 7f2de1000640 1 --2- 172.21.15.46:0/3713612684 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2db8066690 0x7f2db8068b50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2dc40ae000 tx=0x7f2dc40a7070 comp rx=0 tx=0).stop 2024-03-29T08:26:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.244+0000 7f2de1000640 1 -- 172.21.15.46:0/3713612684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc15feb0 msgr2=0x7f2ddc15f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.244+0000 7f2de1000640 1 --2- 172.21.15.46:0/3713612684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc15feb0 0x7f2ddc15f250 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00d830 tx=0x7f2dcc00dcf0 comp rx=0 tx=0).stop 2024-03-29T08:26:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.244+0000 7f2de1000640 1 -- 172.21.15.46:0/3713612684 shutdown_connections 2024-03-29T08:26:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.244+0000 7f2de1000640 1 --2- 172.21.15.46:0/3713612684 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2db8066690 0x7f2db8068b50 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.244+0000 7f2de1000640 1 --2- 172.21.15.46:0/3713612684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ddc160a80 0x7f2ddc153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.244+0000 7f2de1000640 1 --2- 172.21.15.46:0/3713612684 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2ddc1605f0 0x7f2ddc14e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.244+0000 7f2de1000640 1 --2- 172.21.15.46:0/3713612684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ddc15feb0 0x7f2ddc15f250 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:24.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.244+0000 7f2de1000640 1 -- 172.21.15.46:0/3713612684 >> 172.21.15.46:0/3713612684 conn(0x7f2ddc0ced50 msgr2=0x7f2ddc0b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:24.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.244+0000 7f2de1000640 1 -- 172.21.15.46:0/3713612684 shutdown_connections 2024-03-29T08:26:24.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:24.244+0000 7f2de1000640 1 -- 172.21.15.46:0/3713612684 wait complete. 2024-03-29T08:26:24.666 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:24:43.476334+0000', 'modified': '2024-03-29T08:26:23.864491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 94, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7986}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7986': {'gid': 7986, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.46:6835/1886680593', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 1886680593}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1886680593}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:26:24.666 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7986:up:replay 2024-03-29T08:26:25.668 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:26:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.816+0000 7f0819e00640 1 Processor -- start 2024-03-29T08:26:25.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.816+0000 7f0819e00640 1 -- start start 2024-03-29T08:26:25.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.820+0000 7f0819e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0814144210 0x7f08140cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:25.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.820+0000 7f0819e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f081415ada0 0x7f08140ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:25.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.820+0000 7f0819e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08140cb510 con 0x7f08140ce950 2024-03-29T08:26:25.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.820+0000 7f0819e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08140cede0 con 0x7f0814144210 2024-03-29T08:26:25.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.820+0000 7f0819e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081406d960 con 0x7f081415ada0 2024-03-29T08:26:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.820+0000 7f0812a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0814144210 0x7f08140cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.820+0000 7f0812a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0814144210 0x7f08140cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58656/0 (socket says 172.21.15.46:58656) 2024-03-29T08:26:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.820+0000 7f0812a00640 1 -- 172.21.15.46:0/1119787133 learned_addr learned my addr 172.21.15.46:0/1119787133 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.820+0000 7f080be00640 1 --2- 172.21.15.46:0/1119787133 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f081415ada0 0x7f08140ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.820+0000 7f0812a00640 1 -- 172.21.15.46:0/1119787133 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f081415ada0 msgr2=0x7f08140ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.820+0000 7f0812a00640 1 --2- 172.21.15.46:0/1119787133 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f081415ada0 0x7f08140ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.820+0000 7f0812a00640 1 -- 172.21.15.46:0/1119787133 >> v1:172.21.15.46:6789/0 conn(0x7f08140ce950 legacy=0x7f081415e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.820+0000 7f0812a00640 1 -- 172.21.15.46:0/1119787133 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0814070580 con 0x7f0814144210 2024-03-29T08:26:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.820+0000 7f080be00640 1 --2- 172.21.15.46:0/1119787133 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f081415ada0 0x7f08140ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:26:25.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.820+0000 7f0812000640 1 -- 172.21.15.46:0/1119787133 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667075274 0 0) 0x7f08140cb510 con 0x7f08140ce950 2024-03-29T08:26:25.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.824+0000 7f0812a00640 1 --2- 172.21.15.46:0/1119787133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0814144210 0x7f08140cb8b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f0804009870 tx=0x7f080401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfdce691bfdad252 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:25.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.824+0000 7f0812000640 1 -- 172.21.15.46:0/1119787133 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08040ab600 con 0x7f0814144210 2024-03-29T08:26:25.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.824+0000 7f0812000640 1 -- 172.21.15.46:0/1119787133 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08040a7050 con 0x7f0814144210 2024-03-29T08:26:25.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.824+0000 7f0812000640 1 -- 172.21.15.46:0/1119787133 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08040ab980 con 0x7f0814144210 2024-03-29T08:26:25.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.824+0000 7f0819e00640 1 -- 172.21.15.46:0/1119787133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0814144210 msgr2=0x7f08140cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:25.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.824+0000 7f0819e00640 1 --2- 172.21.15.46:0/1119787133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0814144210 0x7f08140cb8b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f0804009870 tx=0x7f080401c560 comp rx=0 tx=0).stop 2024-03-29T08:26:25.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.824+0000 7f0819e00640 1 -- 172.21.15.46:0/1119787133 shutdown_connections 2024-03-29T08:26:25.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.824+0000 7f0819e00640 1 --2- 172.21.15.46:0/1119787133 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f081415ada0 0x7f08140ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:25.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.824+0000 7f0819e00640 1 --2- 172.21.15.46:0/1119787133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0814144210 0x7f08140cb8b0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:25.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.824+0000 7f0819e00640 1 -- 172.21.15.46:0/1119787133 >> 172.21.15.46:0/1119787133 conn(0x7f0814066110 msgr2=0x7f0814069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:25.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.824+0000 7f0819e00640 1 -- 172.21.15.46:0/1119787133 shutdown_connections 2024-03-29T08:26:25.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.824+0000 7f0819e00640 1 -- 172.21.15.46:0/1119787133 wait complete. 2024-03-29T08:26:25.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.824+0000 7f0819e00640 1 Processor -- start 2024-03-29T08:26:25.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.824+0000 7f0819e00640 1 -- start start 2024-03-29T08:26:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f0819e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08140ce950 0x7f08140b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f0819e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0814144210 0x7f081415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f0819e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f081415ada0 0x7f081415c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f0819e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814165090 con 0x7f0814144210 2024-03-29T08:26:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f0819e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814164220 con 0x7f08140ce950 2024-03-29T08:26:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f0819e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081415a800 con 0x7f081415ada0 2024-03-29T08:26:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f080be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0814144210 0x7f081415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f080be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0814144210 0x7f081415a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58668/0 (socket says 172.21.15.46:58668) 2024-03-29T08:26:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f080be00640 1 -- 172.21.15.46:0/4003477111 learned_addr learned my addr 172.21.15.46:0/4003477111 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f0813400640 1 --2- 172.21.15.46:0/4003477111 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f081415ada0 0x7f081415c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f0812a00640 1 --2- 172.21.15.46:0/4003477111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08140ce950 0x7f08140b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:25.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f080be00640 1 -- 172.21.15.46:0/4003477111 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f081415ada0 msgr2=0x7f081415c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:25.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f080be00640 1 --2- 172.21.15.46:0/4003477111 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f081415ada0 0x7f081415c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:25.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f080be00640 1 -- 172.21.15.46:0/4003477111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08140ce950 msgr2=0x7f08140b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:25.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f080be00640 1 --2- 172.21.15.46:0/4003477111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08140ce950 0x7f08140b2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:25.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f080be00640 1 -- 172.21.15.46:0/4003477111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08040094d0 con 0x7f0814144210 2024-03-29T08:26:25.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f0812a00640 1 --2- 172.21.15.46:0/4003477111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08140ce950 0x7f08140b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:26:25.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f0813400640 1 --2- 172.21.15.46:0/4003477111 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f081415ada0 0x7f081415c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:26:25.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f080be00640 1 --2- 172.21.15.46:0/4003477111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0814144210 0x7f081415a2c0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00c820 tx=0x7f07fc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:25.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f080b400640 1 -- 172.21.15.46:0/4003477111 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc007a30 con 0x7f0814144210 2024-03-29T08:26:25.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f080b400640 1 -- 172.21.15.46:0/4003477111 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07fc007b90 con 0x7f0814144210 2024-03-29T08:26:25.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f080b400640 1 -- 172.21.15.46:0/4003477111 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07fc00ce10 con 0x7f0814144210 2024-03-29T08:26:25.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f0819e00640 1 -- 172.21.15.46:0/4003477111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0814164410 con 0x7f0814144210 2024-03-29T08:26:25.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f0819e00640 1 -- 172.21.15.46:0/4003477111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08141649d0 con 0x7f0814144210 2024-03-29T08:26:25.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f0819e00640 1 -- 172.21.15.46:0/4003477111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08140ce280 con 0x7f0814144210 2024-03-29T08:26:25.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.828+0000 7f080b400640 1 -- 172.21.15.46:0/4003477111 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f07fc0113f0 con 0x7f0814144210 2024-03-29T08:26:25.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.832+0000 7f080b400640 1 --2- 172.21.15.46:0/4003477111 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07f0066690 0x7f07f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:25.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.832+0000 7f080b400640 1 -- 172.21.15.46:0/4003477111 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f07fc0843e0 con 0x7f0814144210 2024-03-29T08:26:25.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.832+0000 7f0812a00640 1 --2- 172.21.15.46:0/4003477111 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07f0066690 0x7f07f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:25.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.832+0000 7f0812a00640 1 --2- 172.21.15.46:0/4003477111 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07f0066690 0x7f07f0068b50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f08040ad000 tx=0x7f08040a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:25.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:25.836+0000 7f080b400640 1 -- 172.21.15.46:0/4003477111 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f07fc051690 con 0x7f0814144210 2024-03-29T08:26:26.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:26.032+0000 7f0819e00640 1 -- 172.21.15.46:0/4003477111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08141651d0 con 0x7f0814144210 2024-03-29T08:26:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:26.032+0000 7f080b400640 1 -- 172.21.15.46:0/4003477111 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f07fc054cb0 con 0x7f0814144210 2024-03-29T08:26:26.032 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:26:26.032 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/1862652020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1862652020},{"type":"v1","addr":"172.21.15.73:6836","nonce":1862652020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:24:43.476334+0000","modified":"2024-03-29T08:26:25.871495+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7986":{"gid":7986,"name":"c","rank":0,"incarnation":66,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.46:6835/1886680593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1886680593},{"type":"v1","addr":"172.21.15.46:6835","nonce":1886680593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-29T08:26:26.032 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 68 2024-03-29T08:26:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:26.032+0000 7f0819e00640 1 -- 172.21.15.46:0/4003477111 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07f0066690 msgr2=0x7f07f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:26.032+0000 7f0819e00640 1 --2- 172.21.15.46:0/4003477111 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07f0066690 0x7f07f0068b50 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f08040ad000 tx=0x7f08040a6000 comp rx=0 tx=0).stop 2024-03-29T08:26:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:26.032+0000 7f0819e00640 1 -- 172.21.15.46:0/4003477111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0814144210 msgr2=0x7f081415a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:26.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:26.032+0000 7f0819e00640 1 --2- 172.21.15.46:0/4003477111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0814144210 0x7f081415a2c0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00c820 tx=0x7f07fc00cce0 comp rx=0 tx=0).stop 2024-03-29T08:26:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:26.032+0000 7f0819e00640 1 -- 172.21.15.46:0/4003477111 shutdown_connections 2024-03-29T08:26:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:26.032+0000 7f0819e00640 1 --2- 172.21.15.46:0/4003477111 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f07f0066690 0x7f07f0068b50 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:26.032+0000 7f0819e00640 1 --2- 172.21.15.46:0/4003477111 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f081415ada0 0x7f081415c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:26.032+0000 7f0819e00640 1 --2- 172.21.15.46:0/4003477111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0814144210 0x7f081415a2c0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:26.032+0000 7f0819e00640 1 --2- 172.21.15.46:0/4003477111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08140ce950 0x7f08140b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:26.032+0000 7f0819e00640 1 -- 172.21.15.46:0/4003477111 >> 172.21.15.46:0/4003477111 conn(0x7f0814066110 msgr2=0x7f08140af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:26.032+0000 7f0819e00640 1 -- 172.21.15.46:0/4003477111 shutdown_connections 2024-03-29T08:26:26.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:26.032+0000 7f0819e00640 1 -- 172.21.15.46:0/4003477111 wait complete. 2024-03-29T08:26:26.457 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:24:43.476334+0000', 'modified': '2024-03-29T08:26:25.871495+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 94, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7986}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7986': {'gid': 7986, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.46:6835/1886680593', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 1886680593}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1886680593}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:26:26.457 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7986:up:rejoin 2024-03-29T08:26:27.457 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:26:27.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f069da00640 1 Processor -- start 2024-03-29T08:26:27.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f069da00640 1 -- start start 2024-03-29T08:26:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f069da00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f069806b8b0 0x7f06980cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f069da00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f069815dc00 0x7f06980cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f069da00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f069815cfb0 con 0x7f06980d0660 2024-03-29T08:26:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f069da00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f069806c100 con 0x7f069815dc00 2024-03-29T08:26:27.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f069da00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069806be20 con 0x7f069806b8b0 2024-03-29T08:26:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f06980d0660 0x7f06980d0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48714/0 (socket says 172.21.15.46:48714) 2024-03-29T08:26:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696a00640 1 -- 172.21.15.46:0/261519342 learned_addr learned my addr 172.21.15.46:0/261519342 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:26:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696000640 1 -- 172.21.15.46:0/261519342 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244354026 0 0) 0x7f069815cfb0 con 0x7f06980d0660 2024-03-29T08:26:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696000640 1 -- 172.21.15.46:0/261519342 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f067c003530 con 0x7f06980d0660 2024-03-29T08:26:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f068e000640 1 --2- 172.21.15.46:0/261519342 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f069806b8b0 0x7f06980cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0697400640 1 --2- 172.21.15.46:0/261519342 >> v2:172.21.15.73:3300/0 conn(0x7f069815dc00 0x7f06980cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696000640 1 -- 172.21.15.46:0/261519342 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3789751267 0 0) 0x7f067c003530 con 0x7f06980d0660 2024-03-29T08:26:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696000640 1 -- 172.21.15.46:0/261519342 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f069806b8b0 msgr2=0x7f06980cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696000640 1 --2- 172.21.15.46:0/261519342 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f069806b8b0 0x7f06980cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696000640 1 -- 172.21.15.46:0/261519342 >> v2:172.21.15.73:3300/0 conn(0x7f069815dc00 msgr2=0x7f06980cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696000640 1 --2- 172.21.15.46:0/261519342 >> v2:172.21.15.73:3300/0 conn(0x7f069815dc00 0x7f06980cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:27.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696000640 1 -- 172.21.15.46:0/261519342 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06980cecf0 con 0x7f06980d0660 2024-03-29T08:26:27.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696000640 1 -- 172.21.15.46:0/261519342 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f0688001680 con 0x7f06980d0660 2024-03-29T08:26:27.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696000640 1 -- 172.21.15.46:0/261519342 >> v1:172.21.15.46:6789/0 conn(0x7f06980d0660 legacy=0x7f06980d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:27.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696000640 1 --2- 172.21.15.46:0/261519342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f067c003980 0x7f067c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:27.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696000640 1 --2- 172.21.15.46:0/261519342 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f069806b8b0 0x7f067c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:27.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696000640 1 --2- 172.21.15.46:0/261519342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f067c004d10 0x7f067c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:27.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696000640 1 -- 172.21.15.46:0/261519342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069806c100 con 0x7f067c003980 2024-03-29T08:26:27.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696000640 1 -- 172.21.15.46:0/261519342 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069806be20 con 0x7f067c004d10 2024-03-29T08:26:27.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696000640 1 -- 172.21.15.46:0/261519342 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067c00bac0 con 0x7f069806b8b0 2024-03-29T08:26:27.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0696a00640 1 --2- 172.21.15.46:0/261519342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f067c003980 0x7f067c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:27.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f068e000640 1 --2- 172.21.15.46:0/261519342 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f069806b8b0 0x7f067c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:27.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.611+0000 7f0697400640 1 --2- 172.21.15.46:0/261519342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f067c004d10 0x7f067c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:27.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.615+0000 7f0696a00640 1 -- 172.21.15.46:0/261519342 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f069806b8b0 msgr2=0x7f067c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:27.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.615+0000 7f0696a00640 1 --2- 172.21.15.46:0/261519342 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f069806b8b0 0x7f067c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:27.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.615+0000 7f0696a00640 1 -- 172.21.15.46:0/261519342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f067c004d10 msgr2=0x7f067c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.615+0000 7f0696a00640 1 --2- 172.21.15.46:0/261519342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f067c004d10 0x7f067c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.615+0000 7f0696a00640 1 -- 172.21.15.46:0/261519342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c003530 con 0x7f067c003980 2024-03-29T08:26:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.615+0000 7f0697400640 1 --2- 172.21.15.46:0/261519342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f067c004d10 0x7f067c005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.615+0000 7f0696a00640 1 --2- 172.21.15.46:0/261519342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f067c003980 0x7f067c003da0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f068800c110 tx=0x7f068801cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb158000b4ee2dd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.615+0000 7f0696000640 1 -- 172.21.15.46:0/261519342 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06880058e0 con 0x7f067c003980 2024-03-29T08:26:27.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.615+0000 7f0696000640 1 -- 172.21.15.46:0/261519342 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0688005a60 con 0x7f067c003980 2024-03-29T08:26:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.615+0000 7f0696000640 1 -- 172.21.15.46:0/261519342 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0688005e10 con 0x7f067c003980 2024-03-29T08:26:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.615+0000 7f069da00640 1 -- 172.21.15.46:0/261519342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f067c003980 msgr2=0x7f067c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.615+0000 7f069da00640 1 --2- 172.21.15.46:0/261519342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f067c003980 0x7f067c003da0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f068800c110 tx=0x7f068801cb90 comp rx=0 tx=0).stop 2024-03-29T08:26:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.615+0000 7f0697400640 1 -- 172.21.15.46:0/261519342 reap_dead start 2024-03-29T08:26:27.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.615+0000 7f069da00640 1 -- 172.21.15.46:0/261519342 shutdown_connections 2024-03-29T08:26:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.615+0000 7f069da00640 1 -- 172.21.15.46:0/261519342 >> 172.21.15.46:0/261519342 conn(0x7f0698089fd0 msgr2=0x7f069808a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.615+0000 7f069da00640 1 -- 172.21.15.46:0/261519342 shutdown_connections 2024-03-29T08:26:27.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.615+0000 7f069da00640 1 -- 172.21.15.46:0/261519342 wait complete. 2024-03-29T08:26:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f069da00640 1 Processor -- start 2024-03-29T08:26:27.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f069da00640 1 -- start start 2024-03-29T08:26:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f069da00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f067c003980 0x7f0698159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f069da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06980b5840 0x7f0698156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f069da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0698156ed0 0x7f0698144950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f069da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0698157360 con 0x7f06980b5840 2024-03-29T08:26:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f069da00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06981456f0 con 0x7f0698156ed0 2024-03-29T08:26:27.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f069da00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0698158fc0 con 0x7f067c003980 2024-03-29T08:26:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f068e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06980b5840 0x7f0698156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f0696a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f067c003980 0x7f0698159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f068e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06980b5840 0x7f0698156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47704/0 (socket says 172.21.15.46:47704) 2024-03-29T08:26:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f0696a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f067c003980 0x7f0698159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:52286/0 (socket says 172.21.15.46:52286) 2024-03-29T08:26:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f068e000640 1 -- 172.21.15.46:0/3387319 learned_addr learned my addr 172.21.15.46:0/3387319 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:27.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f0697400640 1 --2- 172.21.15.46:0/3387319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0698156ed0 0x7f0698144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:27.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f068e000640 1 -- 172.21.15.46:0/3387319 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f067c003980 msgr2=0x7f0698159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:27.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f068e000640 1 --2- 172.21.15.46:0/3387319 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f067c003980 0x7f0698159aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:27.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f068e000640 1 -- 172.21.15.46:0/3387319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0698156ed0 msgr2=0x7f0698144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:27.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f068e000640 1 --2- 172.21.15.46:0/3387319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0698156ed0 0x7f0698144950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:27.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f068e000640 1 -- 172.21.15.46:0/3387319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06981592a0 con 0x7f06980b5840 2024-03-29T08:26:27.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f068e000640 1 --2- 172.21.15.46:0/3387319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06980b5840 0x7f0698156990 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f0680003fd0 tx=0x7f0680007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:27.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f068f400640 1 -- 172.21.15.46:0/3387319 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680009c30 con 0x7f06980b5840 2024-03-29T08:26:27.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f068f400640 1 -- 172.21.15.46:0/3387319 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0680015420 con 0x7f06980b5840 2024-03-29T08:26:27.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f068f400640 1 -- 172.21.15.46:0/3387319 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06800157d0 con 0x7f06980b5840 2024-03-29T08:26:27.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.619+0000 7f069da00640 1 -- 172.21.15.46:0/3387319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0698159580 con 0x7f06980b5840 2024-03-29T08:26:27.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.623+0000 7f069da00640 1 -- 172.21.15.46:0/3387319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0698158c90 con 0x7f06980b5840 2024-03-29T08:26:27.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.623+0000 7f069da00640 1 -- 172.21.15.46:0/3387319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0698066c90 con 0x7f06980b5840 2024-03-29T08:26:27.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.623+0000 7f068f400640 1 -- 172.21.15.46:0/3387319 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0680015930 con 0x7f06980b5840 2024-03-29T08:26:27.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.623+0000 7f068f400640 1 --2- 172.21.15.46:0/3387319 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f06740666c0 0x7f0674068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:27.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.623+0000 7f068f400640 1 -- 172.21.15.46:0/3387319 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f0680084cd0 con 0x7f06980b5840 2024-03-29T08:26:27.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.623+0000 7f0696a00640 1 --2- 172.21.15.46:0/3387319 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f06740666c0 0x7f0674068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:27.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.623+0000 7f0696a00640 1 --2- 172.21.15.46:0/3387319 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f06740666c0 0x7f0674068b80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f06880aa000 tx=0x7f06880019b0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:27.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.627+0000 7f068f400640 1 -- 172.21.15.46:0/3387319 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0680052040 con 0x7f06980b5840 2024-03-29T08:26:27.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.823+0000 7f069da00640 1 -- 172.21.15.46:0/3387319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f069806ffa0 con 0x7f06980b5840 2024-03-29T08:26:27.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.827+0000 7f068f400640 1 -- 172.21.15.46:0/3387319 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f0680055660 con 0x7f06980b5840 2024-03-29T08:26:27.828 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:26:27.828 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/1862652020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1862652020},{"type":"v1","addr":"172.21.15.73:6836","nonce":1862652020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:24:43.476334+0000","modified":"2024-03-29T08:26:26.873752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7986":{"gid":7986,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1886680593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1886680593},{"type":"v1","addr":"172.21.15.46:6835","nonce":1886680593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-29T08:26:27.828 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 69 2024-03-29T08:26:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.827+0000 7f069da00640 1 -- 172.21.15.46:0/3387319 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f06740666c0 msgr2=0x7f0674068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.827+0000 7f069da00640 1 --2- 172.21.15.46:0/3387319 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f06740666c0 0x7f0674068b80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f06880aa000 tx=0x7f06880019b0 comp rx=0 tx=0).stop 2024-03-29T08:26:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.827+0000 7f069da00640 1 -- 172.21.15.46:0/3387319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06980b5840 msgr2=0x7f0698156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.827+0000 7f069da00640 1 --2- 172.21.15.46:0/3387319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06980b5840 0x7f0698156990 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f0680003fd0 tx=0x7f0680007700 comp rx=0 tx=0).stop 2024-03-29T08:26:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.827+0000 7f069da00640 1 -- 172.21.15.46:0/3387319 shutdown_connections 2024-03-29T08:26:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.827+0000 7f069da00640 1 --2- 172.21.15.46:0/3387319 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f06740666c0 0x7f0674068b80 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.827+0000 7f069da00640 1 --2- 172.21.15.46:0/3387319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0698156ed0 0x7f0698144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.827+0000 7f069da00640 1 --2- 172.21.15.46:0/3387319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f06980b5840 0x7f0698156990 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.827+0000 7f069da00640 1 --2- 172.21.15.46:0/3387319 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f067c003980 0x7f0698159aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.827+0000 7f069da00640 1 -- 172.21.15.46:0/3387319 >> 172.21.15.46:0/3387319 conn(0x7f0698089fd0 msgr2=0x7f06980ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:27.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.827+0000 7f069da00640 1 -- 172.21.15.46:0/3387319 shutdown_connections 2024-03-29T08:26:27.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:27.827+0000 7f069da00640 1 -- 172.21.15.46:0/3387319 wait complete. 2024-03-29T08:26:28.253 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:24:43.476334+0000', 'modified': '2024-03-29T08:26:26.873752+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 94, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7986}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7986': {'gid': 7986, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6835/1886680593', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6833', 'nonce': 1886680593}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1886680593}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:26:28.253 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T08:26:28.253 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-29T08:26:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f539cc00640 1 Processor -- start 2024-03-29T08:26:28.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f539cc00640 1 -- start start 2024-03-29T08:26:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f539cc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5398142ab0 0x7f539806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f539cc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f539806d9e0 0x7f539815d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f539cc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f539806de40 con 0x7f539815d940 2024-03-29T08:26:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f539cc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5398070c70 con 0x7f5398142ab0 2024-03-29T08:26:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f539cc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5398070340 con 0x7f539806d9e0 2024-03-29T08:26:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f5396800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f539815d940 0x7f53980ae210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48718/0 (socket says 172.21.15.46:48718) 2024-03-29T08:26:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f5396800640 1 -- 172.21.15.46:0/2802692976 learned_addr learned my addr 172.21.15.46:0/2802692976 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:26:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f538d400640 1 --2- 172.21.15.46:0/2802692976 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f539806d9e0 0x7f539815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f5395e00640 1 --2- 172.21.15.46:0/2802692976 >> v2:172.21.15.73:3300/0 conn(0x7f5398142ab0 0x7f539806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f5395400640 1 -- 172.21.15.46:0/2802692976 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1933039829 0 0) 0x7f539806de40 con 0x7f539815d940 2024-03-29T08:26:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f5395400640 1 -- 172.21.15.46:0/2802692976 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f537c005740 con 0x7f539815d940 2024-03-29T08:26:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f538d400640 1 -- 172.21.15.46:0/2802692976 >> v2:172.21.15.73:3300/0 conn(0x7f5398142ab0 msgr2=0x7f539806d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f538d400640 1 --2- 172.21.15.46:0/2802692976 >> v2:172.21.15.73:3300/0 conn(0x7f5398142ab0 0x7f539806d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f538d400640 1 -- 172.21.15.46:0/2802692976 >> v1:172.21.15.46:6789/0 conn(0x7f539815d940 legacy=0x7f53980ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f5395400640 1 -- 172.21.15.46:0/2802692976 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1378851019 0 0) 0x7f537c005740 con 0x7f539815d940 2024-03-29T08:26:28.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.395+0000 7f538d400640 1 -- 172.21.15.46:0/2802692976 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5398070620 con 0x7f539806d9e0 2024-03-29T08:26:28.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.399+0000 7f538d400640 1 --2- 172.21.15.46:0/2802692976 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f539806d9e0 0x7f539815d400 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5380009d90 tx=0x7f538001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bc8ca68679fb283 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.399+0000 7f5395400640 1 -- 172.21.15.46:0/2802692976 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53800a5750 con 0x7f539806d9e0 2024-03-29T08:26:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.399+0000 7f5395400640 1 -- 172.21.15.46:0/2802692976 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53800a58b0 con 0x7f539806d9e0 2024-03-29T08:26:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.399+0000 7f5395400640 1 -- 172.21.15.46:0/2802692976 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53800a5c20 con 0x7f539806d9e0 2024-03-29T08:26:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.399+0000 7f539cc00640 1 -- 172.21.15.46:0/2802692976 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f539806d9e0 msgr2=0x7f539815d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.399+0000 7f539cc00640 1 --2- 172.21.15.46:0/2802692976 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f539806d9e0 0x7f539815d400 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5380009d90 tx=0x7f538001c970 comp rx=0 tx=0).stop 2024-03-29T08:26:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.399+0000 7f539cc00640 1 -- 172.21.15.46:0/2802692976 shutdown_connections 2024-03-29T08:26:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.399+0000 7f539cc00640 1 --2- 172.21.15.46:0/2802692976 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f539806d9e0 0x7f539815d400 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.399+0000 7f539cc00640 1 --2- 172.21.15.46:0/2802692976 >> v2:172.21.15.73:3300/0 conn(0x7f5398142ab0 0x7f539806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.399+0000 7f539cc00640 1 -- 172.21.15.46:0/2802692976 >> 172.21.15.46:0/2802692976 conn(0x7f5398066080 msgr2=0x7f5398069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:28.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.399+0000 7f539cc00640 1 -- 172.21.15.46:0/2802692976 shutdown_connections 2024-03-29T08:26:28.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.399+0000 7f539cc00640 1 -- 172.21.15.46:0/2802692976 wait complete. 2024-03-29T08:26:28.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f539cc00640 1 Processor -- start 2024-03-29T08:26:28.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f539cc00640 1 -- start start 2024-03-29T08:26:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f539cc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f539806d9e0 0x7f53980ad810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f539cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5398142ab0 0x7f53980cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f539cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f539815d940 0x7f53980ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f539cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539813e460 con 0x7f539815d940 2024-03-29T08:26:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f539cc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539813e740 con 0x7f5398142ab0 2024-03-29T08:26:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f539cc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539813ea20 con 0x7f539806d9e0 2024-03-29T08:26:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f5396800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f539815d940 0x7f53980ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f5396800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f539815d940 0x7f53980ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47714/0 (socket says 172.21.15.46:47714) 2024-03-29T08:26:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f5396800640 1 -- 172.21.15.46:0/3306500715 learned_addr learned my addr 172.21.15.46:0/3306500715 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f5395e00640 1 --2- 172.21.15.46:0/3306500715 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f539806d9e0 0x7f53980ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f538d400640 1 --2- 172.21.15.46:0/3306500715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5398142ab0 0x7f53980cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f5396800640 1 -- 172.21.15.46:0/3306500715 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f539806d9e0 msgr2=0x7f53980ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f5396800640 1 --2- 172.21.15.46:0/3306500715 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f539806d9e0 0x7f53980ad810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f5396800640 1 -- 172.21.15.46:0/3306500715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5398142ab0 msgr2=0x7f53980cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f5396800640 1 --2- 172.21.15.46:0/3306500715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5398142ab0 0x7f53980cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f5396800640 1 -- 172.21.15.46:0/3306500715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5380009a70 con 0x7f539815d940 2024-03-29T08:26:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f538d400640 1 --2- 172.21.15.46:0/3306500715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5398142ab0 0x7f53980cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:28.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f5396800640 1 --2- 172.21.15.46:0/3306500715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f539815d940 0x7f53980ccf60 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f539000c3e0 tx=0x7f539000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:28.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f538ea00640 1 -- 172.21.15.46:0/3306500715 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390005cf0 con 0x7f539815d940 2024-03-29T08:26:28.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f538ea00640 1 -- 172.21.15.46:0/3306500715 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5390005e50 con 0x7f539815d940 2024-03-29T08:26:28.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f538ea00640 1 -- 172.21.15.46:0/3306500715 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390007310 con 0x7f539815d940 2024-03-29T08:26:28.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f539cc00640 1 -- 172.21.15.46:0/3306500715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539813ed00 con 0x7f539815d940 2024-03-29T08:26:28.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.403+0000 7f539cc00640 1 -- 172.21.15.46:0/3306500715 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5398158b20 con 0x7f539815d940 2024-03-29T08:26:28.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.407+0000 7f538ea00640 1 -- 172.21.15.46:0/3306500715 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5390007470 con 0x7f539815d940 2024-03-29T08:26:28.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.407+0000 7f538ea00640 1 --2- 172.21.15.46:0/3306500715 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5374066690 0x7f5374068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.407+0000 7f538ea00640 1 -- 172.21.15.46:0/3306500715 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f53900838a0 con 0x7f539815d940 2024-03-29T08:26:28.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.407+0000 7f5395e00640 1 --2- 172.21.15.46:0/3306500715 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5374066690 0x7f5374068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.407+0000 7f53a00cd000 1 Processor -- start 2024-03-29T08:26:28.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.407+0000 7f53a00cd000 1 -- start start 2024-03-29T08:26:28.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.407+0000 7f5395e00640 1 --2- 172.21.15.46:0/3306500715 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5374066690 0x7f5374068b50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f53981641c0 tx=0x7f5388008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:28.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f79cdb0 0x555b9f79d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f79d6d0 0x555b9f752f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f7534c0 0x555b9f7538a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b9f61b5a0 con 0x555b9f79d6d0 2024-03-29T08:26:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b9f5a8af0 con 0x555b9f7534c0 2024-03-29T08:26:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b9f5a60c0 con 0x555b9f79cdb0 2024-03-29T08:26:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f538d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f79d6d0 0x555b9f752f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f538d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f79d6d0 0x555b9f752f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47730/0 (socket says 172.21.15.46:47730) 2024-03-29T08:26:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f538d400640 1 -- 172.21.15.46:0/99565412 learned_addr learned my addr 172.21.15.46:0/99565412 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f5395e00640 1 --2- 172.21.15.46:0/99565412 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f79cdb0 0x555b9f79d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f5396800640 1 --2- 172.21.15.46:0/99565412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f7534c0 0x555b9f7538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f538d400640 1 -- 172.21.15.46:0/99565412 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f79cdb0 msgr2=0x555b9f79d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f538d400640 1 --2- 172.21.15.46:0/99565412 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f79cdb0 0x555b9f79d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f538d400640 1 -- 172.21.15.46:0/99565412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f7534c0 msgr2=0x555b9f7538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:28.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f538d400640 1 --2- 172.21.15.46:0/99565412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f7534c0 0x555b9f7538a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f538d400640 1 -- 172.21.15.46:0/99565412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b9f29d890 con 0x555b9f79d6d0 2024-03-29T08:26:28.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f538d400640 1 --2- 172.21.15.46:0/99565412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f79d6d0 0x555b9f752f80 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f538001c9b0 tx=0x7f5380004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb58f37bac52a9cd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:28.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f5386a00640 1 -- 172.21.15.46:0/99565412 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53800a5a20 con 0x555b9f79d6d0 2024-03-29T08:26:28.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f5386a00640 1 -- 172.21.15.46:0/99565412 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53800a5b80 con 0x555b9f79d6d0 2024-03-29T08:26:28.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f5386a00640 1 -- 172.21.15.46:0/99565412 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53800039a0 con 0x555b9f79d6d0 2024-03-29T08:26:28.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 -- 172.21.15.46:0/99565412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f79d6d0 msgr2=0x555b9f752f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:28.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 --2- 172.21.15.46:0/99565412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f79d6d0 0x555b9f752f80 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f538001c9b0 tx=0x7f5380004ac0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 -- 172.21.15.46:0/99565412 shutdown_connections 2024-03-29T08:26:28.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 --2- 172.21.15.46:0/99565412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f7534c0 0x555b9f7538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 --2- 172.21.15.46:0/99565412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f79d6d0 0x555b9f752f80 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 --2- 172.21.15.46:0/99565412 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f79cdb0 0x555b9f79d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 -- 172.21.15.46:0/99565412 >> 172.21.15.46:0/99565412 conn(0x555b9f759830 msgr2=0x555b9f751860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:28.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 -- 172.21.15.46:0/99565412 shutdown_connections 2024-03-29T08:26:28.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 -- 172.21.15.46:0/99565412 wait complete. 2024-03-29T08:26:28.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 -- start start 2024-03-29T08:26:28.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 Processor -- start 2024-03-29T08:26:28.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f7534c0 0x555b9f757230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f79cdb0 0x555b9f757770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f79d6d0 0x555b9f757cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b9f5a19b0 con 0x555b9f7534c0 2024-03-29T08:26:28.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b9f540c00 con 0x555b9f79d6d0 2024-03-29T08:26:28.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b9f55f070 con 0x555b9f79cdb0 2024-03-29T08:26:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.411+0000 7f53a00cd000 10 client.0 _ll_register_callbacks cb 0x555b9f73f0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:26:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f5395e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f7534c0 0x555b9f757230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f5395e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f7534c0 0x555b9f757230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47740/0 (socket says 172.21.15.46:47740) 2024-03-29T08:26:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f5395e00640 1 -- 172.21.15.46:0/983839379 learned_addr learned my addr 172.21.15.46:0/983839379 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f538d400640 1 --2- 172.21.15.46:0/983839379 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f79cdb0 0x555b9f757770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f5396800640 1 --2- 172.21.15.46:0/983839379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f79d6d0 0x555b9f757cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f5395e00640 1 -- 172.21.15.46:0/983839379 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f79cdb0 msgr2=0x555b9f757770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f5395e00640 1 --2- 172.21.15.46:0/983839379 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f79cdb0 0x555b9f757770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f5395e00640 1 -- 172.21.15.46:0/983839379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f79d6d0 msgr2=0x555b9f757cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f5395e00640 1 --2- 172.21.15.46:0/983839379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f79d6d0 0x555b9f757cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f5395e00640 1 -- 172.21.15.46:0/983839379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5398070620 con 0x555b9f7534c0 2024-03-29T08:26:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f5395e00640 1 --2- 172.21.15.46:0/983839379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f7534c0 0x555b9f757230 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f53880156c0 tx=0x7f53880046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77c511c1770ee988 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f537b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:26:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f537b400640 1 -- 172.21.15.46:0/983839379 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5388014730 con 0x555b9f7534c0 2024-03-29T08:26:28.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f537b400640 1 -- 172.21.15.46:0/983839379 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f5388014890 con 0x555b9f7534c0 2024-03-29T08:26:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 1 -- 172.21.15.46:0/983839379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b9f55e5f0 con 0x555b9f7534c0 2024-03-29T08:26:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f537b400640 1 -- 172.21.15.46:0/983839379 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5388014a90 con 0x555b9f7534c0 2024-03-29T08:26:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 10 client.8061 fetch_fsmap learned FSMap version 69 2024-03-29T08:26:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 1 -- 172.21.15.46:0/983839379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x555b9f566ab0 con 0x555b9f7534c0 2024-03-29T08:26:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f537b400640 1 -- 172.21.15.46:0/983839379 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555b9f55e5f0 con 0x555b9f7534c0 2024-03-29T08:26:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f537b400640 1 -- 172.21.15.46:0/983839379 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f5388014bf0 con 0x555b9f7534c0 2024-03-29T08:26:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 10 client.8061 fetch_fsmap finished waiting for FSMap version 69 2024-03-29T08:26:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 10 client.8061 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:26:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 20 client.8061 populate_metadata read hostname 'smithi046' 2024-03-29T08:26:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 1 --2- 172.21.15.46:0/983839379 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x555b9f7586c0 0x555b9f7ade50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 4 client.8061 mds_command: new command op to 7986 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:26:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 1 -- 172.21.15.46:0/983839379 --> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555b9f59e790 con 0x555b9f7586c0 2024-03-29T08:26:28.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f538d400640 1 --2- 172.21.15.46:0/983839379 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x555b9f7586c0 0x555b9f7ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f538d400640 1 --2- 172.21.15.46:0/983839379 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x555b9f7586c0 0x555b9f7ade50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32ec6f36b542026f server_cookie=b0ab2f4faab41de3 in_seq=0 out_seq=0 2024-03-29T08:26:28.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f537b400640 10 client.8061 ms_handle_connect on v2:172.21.15.46:6833/1886680593 2024-03-29T08:26:28.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f537b400640 1 -- 172.21.15.46:0/983839379 <== mds.0 v2:172.21.15.46:6833/1886680593 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x555b9f59e790 con 0x555b9f7586c0 2024-03-29T08:26:28.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f537b400640 10 client.8061 handle_command_reply: tid=0 2024-03-29T08:26:28.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 1 client.8061 shutdown 2024-03-29T08:26:28.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 1 -- 172.21.15.46:0/983839379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f7534c0 msgr2=0x555b9f757230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:28.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 1 --2- 172.21.15.46:0/983839379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f7534c0 0x555b9f757230 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f53880156c0 tx=0x7f53880046d0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 1 -- 172.21.15.46:0/983839379 shutdown_connections 2024-03-29T08:26:28.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 1 --2- 172.21.15.46:0/983839379 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x555b9f7586c0 0x555b9f7ade50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 1 --2- 172.21.15.46:0/983839379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f79d6d0 0x555b9f757cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 1 --2- 172.21.15.46:0/983839379 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f79cdb0 0x555b9f757770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 1 --2- 172.21.15.46:0/983839379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f7534c0 0x555b9f757230 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f537b400640 0 client.8061 ms_handle_reset on v2:172.21.15.46:6833/1886680593 2024-03-29T08:26:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 1 -- 172.21.15.46:0/983839379 >> 172.21.15.46:0/983839379 conn(0x555b9f759830 msgr2=0x555b9f751f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 1 -- 172.21.15.46:0/983839379 shutdown_connections 2024-03-29T08:26:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 1 -- 172.21.15.46:0/983839379 wait complete. 2024-03-29T08:26:28.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.415+0000 7f53a00cd000 20 client.8061 trim_cache size 0 max 16384 2024-03-29T08:26:28.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.431+0000 7f53a00cd000 1 Processor -- start 2024-03-29T08:26:28.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.431+0000 7f53a00cd000 1 -- start start 2024-03-29T08:26:28.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.431+0000 7f53a00cd000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f7534c0 0x555b9f79ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.431+0000 7f53a00cd000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f79cdb0 0x555b9f79b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.431+0000 7f53a00cd000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f79d6d0 0x555b9f7b7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.431+0000 7f53a00cd000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b9f25e940 con 0x555b9f79d6d0 2024-03-29T08:26:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.431+0000 7f53a00cd000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b9f5e4ac0 con 0x555b9f79cdb0 2024-03-29T08:26:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.431+0000 7f53a00cd000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b9f59bbb0 con 0x555b9f7534c0 2024-03-29T08:26:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f5396800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f79d6d0 0x555b9f7b7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f5396800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f79d6d0 0x555b9f7b7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47748/0 (socket says 172.21.15.46:47748) 2024-03-29T08:26:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f5396800640 1 -- 172.21.15.46:0/2961341079 learned_addr learned my addr 172.21.15.46:0/2961341079 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f5395e00640 1 --2- 172.21.15.46:0/2961341079 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f7534c0 0x555b9f79ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f538d400640 1 --2- 172.21.15.46:0/2961341079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f79cdb0 0x555b9f79b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f5396800640 1 -- 172.21.15.46:0/2961341079 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f7534c0 msgr2=0x555b9f79ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f5396800640 1 --2- 172.21.15.46:0/2961341079 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f7534c0 0x555b9f79ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f5396800640 1 -- 172.21.15.46:0/2961341079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f79cdb0 msgr2=0x555b9f79b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f5396800640 1 --2- 172.21.15.46:0/2961341079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f79cdb0 0x555b9f79b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f5396800640 1 -- 172.21.15.46:0/2961341079 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b9f54cc00 con 0x555b9f79d6d0 2024-03-29T08:26:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f5395e00640 1 --2- 172.21.15.46:0/2961341079 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f7534c0 0x555b9f79ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f538d400640 1 --2- 172.21.15.46:0/2961341079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f79cdb0 0x555b9f79b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:28.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f5396800640 1 --2- 172.21.15.46:0/2961341079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f79d6d0 0x555b9f7b7210 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f539000cda0 tx=0x7f53900547d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce9ca7fbe37de90 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:28.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f5387400640 1 -- 172.21.15.46:0/2961341079 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539005ccc0 con 0x555b9f79d6d0 2024-03-29T08:26:28.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f5387400640 1 -- 172.21.15.46:0/2961341079 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5390052c70 con 0x555b9f79d6d0 2024-03-29T08:26:28.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f5387400640 1 -- 172.21.15.46:0/2961341079 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539005c830 con 0x555b9f79d6d0 2024-03-29T08:26:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 1 -- 172.21.15.46:0/2961341079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f79d6d0 msgr2=0x555b9f7b7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 1 --2- 172.21.15.46:0/2961341079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f79d6d0 0x555b9f7b7210 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f539000cda0 tx=0x7f53900547d0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 1 -- 172.21.15.46:0/2961341079 shutdown_connections 2024-03-29T08:26:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 1 --2- 172.21.15.46:0/2961341079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f79cdb0 0x555b9f79b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 1 --2- 172.21.15.46:0/2961341079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f79d6d0 0x555b9f7b7210 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 1 --2- 172.21.15.46:0/2961341079 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f7534c0 0x555b9f79ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 1 -- 172.21.15.46:0/2961341079 >> 172.21.15.46:0/2961341079 conn(0x555b9f759830 msgr2=0x555b9f756a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 1 -- 172.21.15.46:0/2961341079 shutdown_connections 2024-03-29T08:26:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 1 -- 172.21.15.46:0/2961341079 wait complete. 2024-03-29T08:26:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 1 -- start start 2024-03-29T08:26:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 1 Processor -- start 2024-03-29T08:26:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f7534c0 0x555b9f7fff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f79cdb0 0x555b9f802480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f79d6d0 0x555b9f8049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b9f5a4510 con 0x555b9f7534c0 2024-03-29T08:26:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b9f642290 con 0x555b9f79cdb0 2024-03-29T08:26:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b9f5427f0 con 0x555b9f79d6d0 2024-03-29T08:26:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f53a00cd000 10 client.0 _ll_register_callbacks cb 0x555b9f566590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:26:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f5395e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f7534c0 0x555b9f7fff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f5395e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f7534c0 0x555b9f7fff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47754/0 (socket says 172.21.15.46:47754) 2024-03-29T08:26:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.435+0000 7f5395e00640 1 -- 172.21.15.46:0/1799473393 learned_addr learned my addr 172.21.15.46:0/1799473393 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f5395e00640 1 -- 172.21.15.46:0/1799473393 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f79d6d0 msgr2=0x555b9f8049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f5395e00640 1 --2- 172.21.15.46:0/1799473393 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f79d6d0 0x555b9f8049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f5395e00640 1 -- 172.21.15.46:0/1799473393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f79cdb0 msgr2=0x555b9f802480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f5395e00640 1 --2- 172.21.15.46:0/1799473393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f79cdb0 0x555b9f802480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f5395e00640 1 -- 172.21.15.46:0/1799473393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5398158b20 con 0x555b9f7534c0 2024-03-29T08:26:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f5395e00640 1 --2- 172.21.15.46:0/1799473393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f7534c0 0x555b9f7fff30 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f5388009280 tx=0x7f5388016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e84eefc98f0f55 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f537b400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:26:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f537b400640 1 -- 172.21.15.46:0/1799473393 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5388009840 con 0x555b9f7534c0 2024-03-29T08:26:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f537b400640 1 -- 172.21.15.46:0/1799473393 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f5388004020 con 0x555b9f7534c0 2024-03-29T08:26:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 1 -- 172.21.15.46:0/1799473393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555b9f758e10 con 0x555b9f7534c0 2024-03-29T08:26:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f537b400640 1 -- 172.21.15.46:0/1799473393 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f538802bba0 con 0x555b9f7534c0 2024-03-29T08:26:28.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 10 client.8067 fetch_fsmap learned FSMap version 69 2024-03-29T08:26:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 1 -- 172.21.15.46:0/1799473393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x555b9f29dcc0 con 0x555b9f7534c0 2024-03-29T08:26:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f537b400640 1 -- 172.21.15.46:0/1799473393 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555b9f758e10 con 0x555b9f7534c0 2024-03-29T08:26:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f537b400640 1 -- 172.21.15.46:0/1799473393 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f538802bd00 con 0x555b9f7534c0 2024-03-29T08:26:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 10 client.8067 fetch_fsmap finished waiting for FSMap version 69 2024-03-29T08:26:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 10 client.8067 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:26:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 20 client.8067 populate_metadata read hostname 'smithi046' 2024-03-29T08:26:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 1 --2- 172.21.15.46:0/1799473393 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x555b9f804f10 0x555b9f807300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 4 client.8067 mds_command: new command op to 7986 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T08:26:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 1 -- 172.21.15.46:0/1799473393 --> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555b9f571460 con 0x555b9f804f10 2024-03-29T08:26:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f538d400640 1 --2- 172.21.15.46:0/1799473393 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x555b9f804f10 0x555b9f807300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f538d400640 1 --2- 172.21.15.46:0/1799473393 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x555b9f804f10 0x555b9f807300 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ca64cac014eded1 server_cookie=eea1717c97c14a66 in_seq=0 out_seq=0 2024-03-29T08:26:28.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f537b400640 10 client.8067 ms_handle_connect on v2:172.21.15.46:6833/1886680593 2024-03-29T08:26:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f537b400640 1 -- 172.21.15.46:0/1799473393 <== mds.0 v2:172.21.15.46:6833/1886680593 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x555b9f571460 con 0x555b9f804f10 2024-03-29T08:26:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f537b400640 10 client.8067 handle_command_reply: tid=0 2024-03-29T08:26:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 1 client.8067 shutdown 2024-03-29T08:26:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 1 -- 172.21.15.46:0/1799473393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f7534c0 msgr2=0x555b9f7fff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 1 --2- 172.21.15.46:0/1799473393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f7534c0 0x555b9f7fff30 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f5388009280 tx=0x7f5388016000 comp rx=0 tx=0).stop 2024-03-29T08:26:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 1 -- 172.21.15.46:0/1799473393 shutdown_connections 2024-03-29T08:26:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 1 --2- 172.21.15.46:0/1799473393 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x555b9f804f10 0x555b9f807300 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 1 --2- 172.21.15.46:0/1799473393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555b9f79cdb0 0x555b9f802480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 1 --2- 172.21.15.46:0/1799473393 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x555b9f79d6d0 0x555b9f8049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 1 --2- 172.21.15.46:0/1799473393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b9f7534c0 0x555b9f7fff30 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f537b400640 0 client.8067 ms_handle_reset on v2:172.21.15.46:6833/1886680593 2024-03-29T08:26:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 1 -- 172.21.15.46:0/1799473393 >> 172.21.15.46:0/1799473393 conn(0x555b9f759830 msgr2=0x555b9f79bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 1 -- 172.21.15.46:0/1799473393 shutdown_connections 2024-03-29T08:26:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 1 -- 172.21.15.46:0/1799473393 wait complete. 2024-03-29T08:26:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f53a00cd000 20 client.8067 trim_cache size 0 max 16384 2024-03-29T08:26:28.447 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:26:28.447 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "e9b7960f-ec1f-4787-95c3-c806094d37f0", 2024-03-29T08:26:28.447 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-03-29T08:26:28.447 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 7986, 2024-03-29T08:26:28.447 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-29T08:26:28.447 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-29T08:26:28.447 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-29T08:26:28.447 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 4.5738969469999997, 2024-03-29T08:26:28.448 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 69, 2024-03-29T08:26:28.448 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 94, 2024-03-29T08:26:28.448 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 94, 2024-03-29T08:26:28.448 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 5.5170086129999998 2024-03-29T08:26:28.448 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:26:28.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f539cc00640 1 -- 172.21.15.46:0/3306500715 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5374066690 msgr2=0x7f5374068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:28.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f539cc00640 1 --2- 172.21.15.46:0/3306500715 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5374066690 0x7f5374068b50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f53981641c0 tx=0x7f5388008040 comp rx=0 tx=0).stop 2024-03-29T08:26:28.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f539cc00640 1 -- 172.21.15.46:0/3306500715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f539815d940 msgr2=0x7f53980ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:28.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f539cc00640 1 --2- 172.21.15.46:0/3306500715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f539815d940 0x7f53980ccf60 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f539000c3e0 tx=0x7f539000c8a0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f539cc00640 1 -- 172.21.15.46:0/3306500715 shutdown_connections 2024-03-29T08:26:28.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f539cc00640 1 --2- 172.21.15.46:0/3306500715 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5374066690 0x7f5374068b50 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f539cc00640 1 --2- 172.21.15.46:0/3306500715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5398142ab0 0x7f53980cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f539cc00640 1 --2- 172.21.15.46:0/3306500715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f539815d940 0x7f53980ccf60 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f539cc00640 1 --2- 172.21.15.46:0/3306500715 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f539806d9e0 0x7f53980ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f539cc00640 1 -- 172.21.15.46:0/3306500715 >> 172.21.15.46:0/3306500715 conn(0x7f5398066080 msgr2=0x7f5398159e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:28.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f539cc00640 1 -- 172.21.15.46:0/3306500715 shutdown_connections 2024-03-29T08:26:28.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.439+0000 7f539cc00640 1 -- 172.21.15.46:0/3306500715 wait complete. 2024-03-29T08:26:28.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:26:28.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.599+0000 7f78db800640 1 Processor -- start 2024-03-29T08:26:28.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.599+0000 7f78db800640 1 -- start start 2024-03-29T08:26:28.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.599+0000 7f78db800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f78d415c9d0 0x7f78d40cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.599+0000 7f78db800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d40d0030 con 0x7f78d4099150 2024-03-29T08:26:28.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.599+0000 7f78db800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d40cf420 con 0x7f78d40cef60 2024-03-29T08:26:28.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.599+0000 7f78db800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d40cf700 con 0x7f78d415c9d0 2024-03-29T08:26:28.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.603+0000 7f78d9800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f78d4099150 0x7f78d4166a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48720/0 (socket says 172.21.15.46:48720) 2024-03-29T08:26:28.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.603+0000 7f78d9800640 1 -- 172.21.15.46:0/4185108607 learned_addr learned my addr 172.21.15.46:0/4185108607 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:26:28.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.603+0000 7f78d8e00640 1 --2- 172.21.15.46:0/4185108607 >> v2:172.21.15.73:3300/0 conn(0x7f78d415c9d0 0x7f78d40cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.603+0000 7f78d3400640 1 -- 172.21.15.46:0/4185108607 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246234681 0 0) 0x7f78d40d0030 con 0x7f78d4099150 2024-03-29T08:26:28.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.603+0000 7f78d3400640 1 -- 172.21.15.46:0/4185108607 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78bc003530 con 0x7f78d4099150 2024-03-29T08:26:28.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.603+0000 7f78d8e00640 1 -- 172.21.15.46:0/4185108607 >> v1:172.21.15.73:6789/0 conn(0x7f78d40cef60 legacy=0x7f78d4098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:28.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.603+0000 7f78d8e00640 1 -- 172.21.15.46:0/4185108607 >> v1:172.21.15.46:6789/0 conn(0x7f78d4099150 legacy=0x7f78d4166a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:28.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.603+0000 7f78d8e00640 1 -- 172.21.15.46:0/4185108607 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d40cf9e0 con 0x7f78d415c9d0 2024-03-29T08:26:28.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.603+0000 7f78d8e00640 1 --2- 172.21.15.46:0/4185108607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d415c9d0 0x7f78d40cea20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f78c80097c0 tx=0x7f78c801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfa874b03a6b305e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:28.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78d3400640 1 -- 172.21.15.46:0/4185108607 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c80aa5f0 con 0x7f78d415c9d0 2024-03-29T08:26:28.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78d3400640 1 -- 172.21.15.46:0/4185108607 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c80aa750 con 0x7f78d415c9d0 2024-03-29T08:26:28.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78d3400640 1 -- 172.21.15.46:0/4185108607 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c80aaae0 con 0x7f78d415c9d0 2024-03-29T08:26:28.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78db800640 1 -- 172.21.15.46:0/4185108607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d415c9d0 msgr2=0x7f78d40cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:28.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78db800640 1 --2- 172.21.15.46:0/4185108607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d415c9d0 0x7f78d40cea20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f78c80097c0 tx=0x7f78c801c470 comp rx=0 tx=0).stop 2024-03-29T08:26:28.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78db800640 1 -- 172.21.15.46:0/4185108607 shutdown_connections 2024-03-29T08:26:28.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78db800640 1 --2- 172.21.15.46:0/4185108607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d415c9d0 0x7f78d40cea20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78db800640 1 -- 172.21.15.46:0/4185108607 >> 172.21.15.46:0/4185108607 conn(0x7f78d40b21a0 msgr2=0x7f78d40b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:28.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78db800640 1 -- 172.21.15.46:0/4185108607 shutdown_connections 2024-03-29T08:26:28.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78db800640 1 -- 172.21.15.46:0/4185108607 wait complete. 2024-03-29T08:26:28.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78db800640 1 Processor -- start 2024-03-29T08:26:28.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78db800640 1 -- start start 2024-03-29T08:26:28.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78db800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78d4099150 0x7f78d40b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78db800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d40cef60 0x7f78d415b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78db800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d415c9d0 0x7f78d4154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78db800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d40b4860 con 0x7f78d40cef60 2024-03-29T08:26:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78db800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d40b6090 con 0x7f78d415c9d0 2024-03-29T08:26:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78db800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d40b4ab0 con 0x7f78d4099150 2024-03-29T08:26:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d40cef60 0x7f78d415b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d40cef60 0x7f78d415b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47766/0 (socket says 172.21.15.46:47766) 2024-03-29T08:26:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78d3e00640 1 -- 172.21.15.46:0/4210876574 learned_addr learned my addr 172.21.15.46:0/4210876574 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78d8e00640 1 --2- 172.21.15.46:0/4210876574 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78d4099150 0x7f78d40b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.607+0000 7f78d9800640 1 --2- 172.21.15.46:0/4210876574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d415c9d0 0x7f78d4154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.611+0000 7f78d3e00640 1 -- 172.21.15.46:0/4210876574 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78d4099150 msgr2=0x7f78d40b5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.611+0000 7f78d3e00640 1 --2- 172.21.15.46:0/4210876574 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78d4099150 0x7f78d40b5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.611+0000 7f78d3e00640 1 -- 172.21.15.46:0/4210876574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d415c9d0 msgr2=0x7f78d4154bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:28.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.611+0000 7f78d3e00640 1 --2- 172.21.15.46:0/4210876574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d415c9d0 0x7f78d4154bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.611+0000 7f78d3e00640 1 -- 172.21.15.46:0/4210876574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c8009420 con 0x7f78d40cef60 2024-03-29T08:26:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.611+0000 7f78d9800640 1 --2- 172.21.15.46:0/4210876574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d415c9d0 0x7f78d4154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:26:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.611+0000 7f78d3e00640 1 --2- 172.21.15.46:0/4210876574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d40cef60 0x7f78d415b0c0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f78c000c500 tx=0x7f78c000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.611+0000 7f78d0c00640 1 -- 172.21.15.46:0/4210876574 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c0005220 con 0x7f78d40cef60 2024-03-29T08:26:28.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.611+0000 7f78d0c00640 1 -- 172.21.15.46:0/4210876574 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c0005380 con 0x7f78d40cef60 2024-03-29T08:26:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.611+0000 7f78d0c00640 1 -- 172.21.15.46:0/4210876574 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c0007270 con 0x7f78d40cef60 2024-03-29T08:26:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.611+0000 7f78db800640 1 -- 172.21.15.46:0/4210876574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d40b4d90 con 0x7f78d40cef60 2024-03-29T08:26:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.611+0000 7f78db800640 1 -- 172.21.15.46:0/4210876574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d40b5350 con 0x7f78d40cef60 2024-03-29T08:26:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.611+0000 7f78db800640 1 -- 172.21.15.46:0/4210876574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d41585b0 con 0x7f78d40cef60 2024-03-29T08:26:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.611+0000 7f78d0c00640 1 -- 172.21.15.46:0/4210876574 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f78c00073d0 con 0x7f78d40cef60 2024-03-29T08:26:28.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.611+0000 7f78d0c00640 1 --2- 172.21.15.46:0/4210876574 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78b4066690 0x7f78b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:28.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.611+0000 7f78d0c00640 1 -- 172.21.15.46:0/4210876574 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f78c0083a80 con 0x7f78d40cef60 2024-03-29T08:26:28.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.611+0000 7f78d8e00640 1 --2- 172.21.15.46:0/4210876574 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78b4066690 0x7f78b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:28.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.615+0000 7f78d8e00640 1 --2- 172.21.15.46:0/4210876574 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78b4066690 0x7f78b4068b50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f78c80ae000 tx=0x7f78c80a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:28.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.615+0000 7f78d0c00640 1 -- 172.21.15.46:0/4210876574 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f78c0050d60 con 0x7f78d40cef60 2024-03-29T08:26:28.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.811+0000 7f78db800640 1 -- 172.21.15.46:0/4210876574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f78d4066c90 con 0x7f78d40cef60 2024-03-29T08:26:28.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.811+0000 7f78d0c00640 1 -- 172.21.15.46:0/4210876574 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v69) v1 ==== 53+0+73 (secure 0 0 0) 0x7f78c0054380 con 0x7f78d40cef60 2024-03-29T08:26:28.819 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:26:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.815+0000 7f78db800640 1 -- 172.21.15.46:0/4210876574 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78b4066690 msgr2=0x7f78b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.815+0000 7f78db800640 1 --2- 172.21.15.46:0/4210876574 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78b4066690 0x7f78b4068b50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f78c80ae000 tx=0x7f78c80a7000 comp rx=0 tx=0).stop 2024-03-29T08:26:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.815+0000 7f78db800640 1 -- 172.21.15.46:0/4210876574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d40cef60 msgr2=0x7f78d415b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.815+0000 7f78db800640 1 --2- 172.21.15.46:0/4210876574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d40cef60 0x7f78d415b0c0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f78c000c500 tx=0x7f78c000c9c0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.815+0000 7f78db800640 1 -- 172.21.15.46:0/4210876574 shutdown_connections 2024-03-29T08:26:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.815+0000 7f78db800640 1 --2- 172.21.15.46:0/4210876574 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78b4066690 0x7f78b4068b50 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.815+0000 7f78db800640 1 --2- 172.21.15.46:0/4210876574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d415c9d0 0x7f78d4154bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.815+0000 7f78db800640 1 --2- 172.21.15.46:0/4210876574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78d40cef60 0x7f78d415b0c0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.815+0000 7f78db800640 1 --2- 172.21.15.46:0/4210876574 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78d4099150 0x7f78d40b5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:28.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.815+0000 7f78db800640 1 -- 172.21.15.46:0/4210876574 >> 172.21.15.46:0/4210876574 conn(0x7f78d40b21a0 msgr2=0x7f78d40cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:28.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.815+0000 7f78db800640 1 -- 172.21.15.46:0/4210876574 shutdown_connections 2024-03-29T08:26:28.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:28.815+0000 7f78db800640 1 -- 172.21.15.46:0/4210876574 wait complete. 2024-03-29T08:26:28.834 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:26:28.835 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:26:28.835 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-29T08:26:28.835 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T08:26:28.835 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:26:28.835 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:26:28.835 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:26:28.835 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:26:28.835 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:26:28.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:26:28.835 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-29T08:26:28.845 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:26:28.845 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:26:28.845 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:26:28.845 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:26:28.845 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:26:28.845 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:26:28.845 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:26:28.846 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:26:28.846 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-29T08:26:28.846 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:26:28.846 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:26:28.846 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-29T08:26:28.846 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:26:28.846 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:26:28.846 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-29T08:26:28.846 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T08:26:28.846 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41948sec preferred_lft 41948sec 2024-03-29T08:26:28.846 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-29T08:26:28.846 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:26:28.846 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:26:28.847 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:26:28.847 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-29T08:26:28.847 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:26:28.847 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:d9:58:09:4c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:26:28.847 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:26:28.847 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:26:28.847 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:26:28.847 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T08:26:28.847 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T08:26:28.847 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:26:28.847 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T08:26:28.847 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:26:28.847 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:26:28.848 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:26:28.848 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T08:26:28.848 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:26:28.848 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:26:28.848 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T08:26:28.848 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-29T08:26:28.848 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:26:28.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:26:28.848 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:26:28.870 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-03-29T08:26:28.867+0000 7f5018800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:26:28.888 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:26:28.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:26:28.890 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-29T08:26:28.941 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-29T08:26:28.941 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:26:28.941 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T08:26:28.941 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:26:28.941 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T08:26:28.941 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:26:28.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:26:28.942 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:26:28.942 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:26:28.942 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:26:28.942 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:26:28.942 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:26:28.942 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:26:29.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:26:29.034 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:26:29.043 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T08:26:29.043 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T08:26:29.043 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:26:29.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:26:29.043 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-03-29T08:26:29.107 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T08:26:29.107 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:26:29.108 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T08:26:29.163 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:26:29.163 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:26:29.163 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:26:29.215 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:26:29.280 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:26:29.280 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:26:29.341 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:26:29.343 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:26:29.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:26:29.344 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:26:29.395 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:26:29.395 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T08:26:29.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:26:29.463 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:26:29.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:26:29.525 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:26:29.527 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:26:29.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:26:29.527 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:26:29.538 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-03-29T08:26:29.535+0000 7f43e8231580 -1 init, newargv = 0x55981259b0c0 newargc=15 2024-03-29T08:26:29.538 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[46501]: starting ceph client 2024-03-29T08:26:29.545 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[46501]: starting fuse 2024-03-29T08:26:29.547 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:26:29.547 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-29T08:26:29.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:26:29.547 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-03-29T08:26:29.548 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-03-29T08:26:29.548 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:26:29.548 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-03-29T08:26:29.548 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:26:29.548 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-03-29T08:26:29.548 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T08:26:29.548 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-03-29T08:26:29.548 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:26:29.548 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:26:29.548 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-03-29T08:26:29.548 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-03-29T08:26:29.548 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-03-29T08:26:29.548 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-03-29T08:26:29.548 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:26:29.548 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-03-29T08:26:29.549 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:26:29.549 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:26:29.549 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:26:29.549 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-03-29T08:26:29.549 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-03-29T08:26:29.549 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-03-29T08:26:29.549 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:26:29.549 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:26:29.549 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-03-29T08:26:29.549 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:26:29.672 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.46501.asok 2024-03-29T08:26:29.855 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46501.asok 2024-03-29T08:26:29.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:26:29.856 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46501.asok status 2024-03-29T08:26:30.033 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:26:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-03-29T08:26:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:26:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:26:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-03-29T08:26:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-03-29T08:26:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T08:26:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "46501", 2024-03-29T08:26:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-03-29T08:26:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:26:30.033 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-03-29T08:26:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-03-29T08:26:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 8076, 2024-03-29T08:26:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-03-29T08:26:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-03-29T08:26:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-03-29T08:26:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 8076 2024-03-29T08:26:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:26:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:26:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:26:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:26:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2618262115 2024-03-29T08:26:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:26:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:26:30.034 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:26:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:26:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:26:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2618262115 2024-03-29T08:26:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:26:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.8076 192.168.0.1:0/2618262115", 2024-03-29T08:26:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/2618262115", 2024-03-29T08:26:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-03-29T08:26:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 69, 2024-03-29T08:26:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 94, 2024-03-29T08:26:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:26:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-03-29T08:26:30.035 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-03-29T08:26:30.035 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:26:30.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:26:30.048 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-03-29T08:26:30.048 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:26:30.057 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-03-29T08:26:30.057 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T08:26:30.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:26:30.058 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:26:30.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:26:30.119 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-03-29T08:26:30.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.303+0000 7f4a85800640 1 Processor -- start 2024-03-29T08:26:30.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.303+0000 7f4a85800640 1 -- start start 2024-03-29T08:26:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.307+0000 7f4a85800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a800cec80 0x7f4a80141770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.307+0000 7f4a85800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8006cd80 con 0x7f4a80140f40 2024-03-29T08:26:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.307+0000 7f4a85800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a8006d060 con 0x7f4a80141cb0 2024-03-29T08:26:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.307+0000 7f4a85800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a800cbff0 con 0x7f4a800cec80 2024-03-29T08:26:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.307+0000 7f4a76000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a800cec80 0x7f4a80141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.307+0000 7f4a7ea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4a80140f40 0x7f4a800cb8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48728/0 (socket says 172.21.15.46:48728) 2024-03-29T08:26:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.307+0000 7f4a7ea00640 1 -- 172.21.15.46:0/576822888 learned_addr learned my addr 172.21.15.46:0/576822888 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:26:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.307+0000 7f4a76000640 1 -- 172.21.15.46:0/576822888 >> v1:172.21.15.73:6789/0 conn(0x7f4a80141cb0 legacy=0x7f4a8006db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.307+0000 7f4a76000640 1 -- 172.21.15.46:0/576822888 >> v1:172.21.15.46:6789/0 conn(0x7f4a80140f40 legacy=0x7f4a800cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.307+0000 7f4a7e000640 1 -- 172.21.15.46:0/576822888 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812855162 0 0) 0x7f4a8006cd80 con 0x7f4a80140f40 2024-03-29T08:26:30.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.307+0000 7f4a76000640 1 -- 172.21.15.46:0/576822888 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8006d340 con 0x7f4a800cec80 2024-03-29T08:26:30.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a76000640 1 --2- 172.21.15.46:0/576822888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a800cec80 0x7f4a80141770 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f4a680097c0 tx=0x7f4a6801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6117898d1c799f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a7e000640 1 -- 172.21.15.46:0/576822888 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a680aa5f0 con 0x7f4a800cec80 2024-03-29T08:26:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a7e000640 1 -- 172.21.15.46:0/576822888 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a680aa750 con 0x7f4a800cec80 2024-03-29T08:26:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a7e000640 1 -- 172.21.15.46:0/576822888 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a680aaae0 con 0x7f4a800cec80 2024-03-29T08:26:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a85800640 1 -- 172.21.15.46:0/576822888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a800cec80 msgr2=0x7f4a80141770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a85800640 1 --2- 172.21.15.46:0/576822888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a800cec80 0x7f4a80141770 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f4a680097c0 tx=0x7f4a6801c470 comp rx=0 tx=0).stop 2024-03-29T08:26:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a85800640 1 -- 172.21.15.46:0/576822888 shutdown_connections 2024-03-29T08:26:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a85800640 1 --2- 172.21.15.46:0/576822888 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a800cec80 0x7f4a80141770 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a85800640 1 -- 172.21.15.46:0/576822888 >> 172.21.15.46:0/576822888 conn(0x7f4a800b42d0 msgr2=0x7f4a80069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a85800640 1 -- 172.21.15.46:0/576822888 shutdown_connections 2024-03-29T08:26:30.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a85800640 1 -- 172.21.15.46:0/576822888 wait complete. 2024-03-29T08:26:30.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a85800640 1 Processor -- start 2024-03-29T08:26:30.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a85800640 1 -- start start 2024-03-29T08:26:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a85800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a800cec80 0x7f4a80157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a85800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a80140f40 0x7f4a8013d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a85800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4a80141cb0 0x7f4a800a88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a85800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8013da30 con 0x7f4a800cec80 2024-03-29T08:26:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a85800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8013dd10 con 0x7f4a80140f40 2024-03-29T08:26:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.311+0000 7f4a85800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8013dff0 con 0x7f4a80141cb0 2024-03-29T08:26:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a7ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a800cec80 0x7f4a80157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a7ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a800cec80 0x7f4a80157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47786/0 (socket says 172.21.15.46:47786) 2024-03-29T08:26:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a7ea00640 1 -- 172.21.15.46:0/2965898779 learned_addr learned my addr 172.21.15.46:0/2965898779 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a76000640 1 --2- 172.21.15.46:0/2965898779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a80140f40 0x7f4a8013d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a7f400640 1 --2- 172.21.15.46:0/2965898779 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4a80141cb0 0x7f4a800a88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a7ea00640 1 -- 172.21.15.46:0/2965898779 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4a80141cb0 msgr2=0x7f4a800a88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a7ea00640 1 --2- 172.21.15.46:0/2965898779 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4a80141cb0 0x7f4a800a88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a7ea00640 1 -- 172.21.15.46:0/2965898779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a80140f40 msgr2=0x7f4a8013d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a7ea00640 1 --2- 172.21.15.46:0/2965898779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a80140f40 0x7f4a8013d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a7ea00640 1 -- 172.21.15.46:0/2965898779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a68009420 con 0x7f4a800cec80 2024-03-29T08:26:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a7f400640 1 --2- 172.21.15.46:0/2965898779 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4a80141cb0 0x7f4a800a88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a7ea00640 1 --2- 172.21.15.46:0/2965898779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a800cec80 0x7f4a80157a40 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000c4a0 tx=0x7f4a7000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a77400640 1 -- 172.21.15.46:0/2965898779 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a700074e0 con 0x7f4a800cec80 2024-03-29T08:26:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a77400640 1 -- 172.21.15.46:0/2965898779 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a70007640 con 0x7f4a800cec80 2024-03-29T08:26:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a77400640 1 -- 172.21.15.46:0/2965898779 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a70004a30 con 0x7f4a800cec80 2024-03-29T08:26:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a85800640 1 -- 172.21.15.46:0/2965898779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a8013e2d0 con 0x7f4a800cec80 2024-03-29T08:26:30.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a85800640 1 -- 172.21.15.46:0/2965898779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8015a6d0 con 0x7f4a800cec80 2024-03-29T08:26:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.315+0000 7f4a77400640 1 -- 172.21.15.46:0/2965898779 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4a70004b90 con 0x7f4a800cec80 2024-03-29T08:26:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a77400640 1 --2- 172.21.15.46:0/2965898779 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4a5c066690 0x7f4a5c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a77400640 1 -- 172.21.15.46:0/2965898779 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f4a700834a0 con 0x7f4a800cec80 2024-03-29T08:26:30.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a76000640 1 --2- 172.21.15.46:0/2965898779 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4a5c066690 0x7f4a5c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a88d4e000 1 Processor -- start 2024-03-29T08:26:30.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a88d4e000 1 -- start start 2024-03-29T08:26:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a76000640 1 --2- 172.21.15.46:0/2965898779 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4a5c066690 0x7f4a5c068b50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4a680ae000 tx=0x7f4a680a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:30.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a88d4e000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d4d9d30 0x55619d4da110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a88d4e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4da650 0x55619d490a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a88d4e000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d490f60 0x55619d491340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a88d4e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55619d358170 con 0x55619d4da650 2024-03-29T08:26:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a88d4e000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55619d2e5d80 con 0x55619d490f60 2024-03-29T08:26:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a88d4e000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55619d2e3350 con 0x55619d4d9d30 2024-03-29T08:26:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a76000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4da650 0x55619d490a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a76000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4da650 0x55619d490a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47788/0 (socket says 172.21.15.46:47788) 2024-03-29T08:26:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a76000640 1 -- 172.21.15.46:0/2735931948 learned_addr learned my addr 172.21.15.46:0/2735931948 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a7ea00640 1 --2- 172.21.15.46:0/2735931948 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d4d9d30 0x55619d4da110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a7f400640 1 --2- 172.21.15.46:0/2735931948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d490f60 0x55619d491340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a76000640 1 -- 172.21.15.46:0/2735931948 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d4d9d30 msgr2=0x55619d4da110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a76000640 1 --2- 172.21.15.46:0/2735931948 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d4d9d30 0x55619d4da110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a76000640 1 -- 172.21.15.46:0/2735931948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d490f60 msgr2=0x55619d491340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a76000640 1 --2- 172.21.15.46:0/2735931948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d490f60 0x55619d491340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a76000640 1 -- 172.21.15.46:0/2735931948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55619cfdb070 con 0x55619d4da650 2024-03-29T08:26:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a7ea00640 1 --2- 172.21.15.46:0/2735931948 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d4d9d30 0x55619d4da110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:26:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a76000640 1 --2- 172.21.15.46:0/2735931948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4da650 0x55619d490a20 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f4a68097f60 tx=0x7f4a68004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d0262d5fb6b1bd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a6f400640 1 -- 172.21.15.46:0/2735931948 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a680b9070 con 0x55619d4da650 2024-03-29T08:26:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a6f400640 1 -- 172.21.15.46:0/2735931948 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a68003620 con 0x55619d4da650 2024-03-29T08:26:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a6f400640 1 -- 172.21.15.46:0/2735931948 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68003990 con 0x55619d4da650 2024-03-29T08:26:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a88d4e000 1 -- 172.21.15.46:0/2735931948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4da650 msgr2=0x55619d490a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/2735931948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4da650 0x55619d490a20 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f4a68097f60 tx=0x7f4a68004740 comp rx=0 tx=0).stop 2024-03-29T08:26:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a88d4e000 1 -- 172.21.15.46:0/2735931948 shutdown_connections 2024-03-29T08:26:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/2735931948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d490f60 0x55619d491340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/2735931948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4da650 0x55619d490a20 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/2735931948 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d4d9d30 0x55619d4da110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a88d4e000 1 -- 172.21.15.46:0/2735931948 >> 172.21.15.46:0/2735931948 conn(0x55619d48cf10 msgr2=0x55619d48f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:30.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a88d4e000 1 -- 172.21.15.46:0/2735931948 shutdown_connections 2024-03-29T08:26:30.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.319+0000 7f4a88d4e000 1 -- 172.21.15.46:0/2735931948 wait complete. 2024-03-29T08:26:30.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 1 -- start start 2024-03-29T08:26:30.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 1 Processor -- start 2024-03-29T08:26:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d490f60 0x55619d496930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4d9d30 0x55619d4dac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d4da650 0x55619d4db180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55619d2dec40 con 0x55619d4d9d30 2024-03-29T08:26:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55619d27dfa0 con 0x55619d4da650 2024-03-29T08:26:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55619d29c1c0 con 0x55619d490f60 2024-03-29T08:26:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 10 client.0 _ll_register_callbacks cb 0x55619d47c060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:26:30.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a76000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4d9d30 0x55619d4dac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a76000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4d9d30 0x55619d4dac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47794/0 (socket says 172.21.15.46:47794) 2024-03-29T08:26:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a76000640 1 -- 172.21.15.46:0/3732638101 learned_addr learned my addr 172.21.15.46:0/3732638101 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a7f400640 1 --2- 172.21.15.46:0/3732638101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d4da650 0x55619d4db180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a7ea00640 1 --2- 172.21.15.46:0/3732638101 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d490f60 0x55619d496930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a76000640 1 -- 172.21.15.46:0/3732638101 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d490f60 msgr2=0x55619d496930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a76000640 1 --2- 172.21.15.46:0/3732638101 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d490f60 0x55619d496930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a76000640 1 -- 172.21.15.46:0/3732638101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d4da650 msgr2=0x55619d4db180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a76000640 1 --2- 172.21.15.46:0/3732638101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d4da650 0x55619d4db180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a76000640 1 -- 172.21.15.46:0/3732638101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8015a6d0 con 0x55619d4d9d30 2024-03-29T08:26:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a76000640 1 --2- 172.21.15.46:0/3732638101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4d9d30 0x55619d4dac40 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f4a68097f60 tx=0x7f4a6801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420196f7e4b79458 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a63e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:26:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a63e00640 1 -- 172.21.15.46:0/3732638101 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a680aaed0 con 0x55619d4d9d30 2024-03-29T08:26:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 1 -- 172.21.15.46:0/3732638101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55619d2dba20 con 0x55619d4d9d30 2024-03-29T08:26:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a63e00640 1 -- 172.21.15.46:0/3732638101 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f4a680a7070 con 0x55619d4d9d30 2024-03-29T08:26:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a63e00640 1 -- 172.21.15.46:0/3732638101 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a680aa5d0 con 0x55619d4d9d30 2024-03-29T08:26:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 10 client.8091 fetch_fsmap learned FSMap version 69 2024-03-29T08:26:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 1 -- 172.21.15.46:0/3732638101 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x7f4a8006d340 con 0x55619d4d9d30 2024-03-29T08:26:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a63e00640 1 -- 172.21.15.46:0/3732638101 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55619d2dba20 con 0x55619d4d9d30 2024-03-29T08:26:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a63e00640 1 -- 172.21.15.46:0/3732638101 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4a680a9070 con 0x55619d4d9d30 2024-03-29T08:26:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 10 client.8091 fetch_fsmap finished waiting for FSMap version 69 2024-03-29T08:26:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 10 client.8091 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:26:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 20 client.8091 populate_metadata read hostname 'smithi046' 2024-03-29T08:26:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/3732638101 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x55619d4e1250 0x55619d4e1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 4 client.8091 mds_command: new command op to 7986 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:26:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a88d4e000 1 -- 172.21.15.46:0/3732638101 --> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55619d2dd590 con 0x55619d4e1250 2024-03-29T08:26:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a7ea00640 1 --2- 172.21.15.46:0/3732638101 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x55619d4e1250 0x55619d4e1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a7ea00640 1 --2- 172.21.15.46:0/3732638101 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x55619d4e1250 0x55619d4e1660 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e1d8800313d93d server_cookie=1cff2d6ef6034e7b in_seq=0 out_seq=0 2024-03-29T08:26:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.323+0000 7f4a63e00640 10 client.8091 ms_handle_connect on v2:172.21.15.46:6833/1886680593 2024-03-29T08:26:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.327+0000 7f4a63e00640 1 -- 172.21.15.46:0/3732638101 <== mds.0 v2:172.21.15.46:6833/1886680593 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55619d2dd590 con 0x55619d4e1250 2024-03-29T08:26:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.327+0000 7f4a63e00640 10 client.8091 handle_command_reply: tid=0 2024-03-29T08:26:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.327+0000 7f4a88d4e000 1 client.8091 shutdown 2024-03-29T08:26:30.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.327+0000 7f4a88d4e000 1 -- 172.21.15.46:0/3732638101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4d9d30 msgr2=0x55619d4dac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.327+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/3732638101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4d9d30 0x55619d4dac40 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f4a68097f60 tx=0x7f4a6801acb0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.327+0000 7f4a88d4e000 1 -- 172.21.15.46:0/3732638101 shutdown_connections 2024-03-29T08:26:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.327+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/3732638101 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x55619d4e1250 0x55619d4e1660 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.327+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/3732638101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d4da650 0x55619d4db180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.327+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/3732638101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4d9d30 0x55619d4dac40 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.327+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/3732638101 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d490f60 0x55619d496930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.327+0000 7f4a63e00640 0 client.8091 ms_handle_reset on v2:172.21.15.46:6833/1886680593 2024-03-29T08:26:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.327+0000 7f4a88d4e000 1 -- 172.21.15.46:0/3732638101 >> 172.21.15.46:0/3732638101 conn(0x55619d48cf10 msgr2=0x55619d48fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.327+0000 7f4a88d4e000 1 -- 172.21.15.46:0/3732638101 shutdown_connections 2024-03-29T08:26:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.327+0000 7f4a88d4e000 1 -- 172.21.15.46:0/3732638101 wait complete. 2024-03-29T08:26:30.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.327+0000 7f4a88d4e000 20 client.8091 trim_cache size 0 max 16384 2024-03-29T08:26:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.339+0000 7f4a88d4e000 1 Processor -- start 2024-03-29T08:26:30.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.339+0000 7f4a88d4e000 1 -- start start 2024-03-29T08:26:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a88d4e000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d490f60 0x55619d490310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a88d4e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4d9d30 0x55619d490850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a88d4e000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d4da650 0x55619d4dba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a88d4e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55619cf9bcf0 con 0x55619d4d9d30 2024-03-29T08:26:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a88d4e000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55619d493b50 con 0x55619d4da650 2024-03-29T08:26:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a88d4e000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55619d2da430 con 0x55619d490f60 2024-03-29T08:26:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a76000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4d9d30 0x55619d490850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a76000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4d9d30 0x55619d490850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47810/0 (socket says 172.21.15.46:47810) 2024-03-29T08:26:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a76000640 1 -- 172.21.15.46:0/4077468872 learned_addr learned my addr 172.21.15.46:0/4077468872 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a7ea00640 1 --2- 172.21.15.46:0/4077468872 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d490f60 0x55619d490310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a7f400640 1 --2- 172.21.15.46:0/4077468872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d4da650 0x55619d4dba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a76000640 1 -- 172.21.15.46:0/4077468872 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d490f60 msgr2=0x55619d490310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a76000640 1 --2- 172.21.15.46:0/4077468872 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d490f60 0x55619d490310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a76000640 1 -- 172.21.15.46:0/4077468872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d4da650 msgr2=0x55619d4dba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a76000640 1 --2- 172.21.15.46:0/4077468872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d4da650 0x55619d4dba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a76000640 1 -- 172.21.15.46:0/4077468872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55619d2a3ba0 con 0x55619d4d9d30 2024-03-29T08:26:30.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a7f400640 1 --2- 172.21.15.46:0/4077468872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d4da650 0x55619d4dba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a76000640 1 --2- 172.21.15.46:0/4077468872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4d9d30 0x55619d490850 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f4a68004660 tx=0x7f4a680d0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e13e7760159b5a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a6fe00640 1 -- 172.21.15.46:0/4077468872 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a680aaed0 con 0x55619d4d9d30 2024-03-29T08:26:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a6fe00640 1 -- 172.21.15.46:0/4077468872 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a680aa5d0 con 0x55619d4d9d30 2024-03-29T08:26:30.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a6fe00640 1 -- 172.21.15.46:0/4077468872 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a68003b30 con 0x55619d4d9d30 2024-03-29T08:26:30.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a88d4e000 1 -- 172.21.15.46:0/4077468872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4d9d30 msgr2=0x55619d490850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/4077468872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4d9d30 0x55619d490850 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f4a68004660 tx=0x7f4a680d0000 comp rx=0 tx=0).stop 2024-03-29T08:26:30.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a88d4e000 1 -- 172.21.15.46:0/4077468872 shutdown_connections 2024-03-29T08:26:30.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/4077468872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d4da650 0x55619d4dba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/4077468872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4d9d30 0x55619d490850 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/4077468872 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d490f60 0x55619d490310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a88d4e000 1 -- 172.21.15.46:0/4077468872 >> 172.21.15.46:0/4077468872 conn(0x55619d48cf10 msgr2=0x55619d495990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a88d4e000 1 -- 172.21.15.46:0/4077468872 shutdown_connections 2024-03-29T08:26:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a88d4e000 1 -- 172.21.15.46:0/4077468872 wait complete. 2024-03-29T08:26:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a88d4e000 1 -- start start 2024-03-29T08:26:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.343+0000 7f4a88d4e000 1 Processor -- start 2024-03-29T08:26:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d490f60 0x55619d538e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d4d9d30 0x55619d4de870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4da650 0x55619d4dedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55619d2ddad0 con 0x55619d4da650 2024-03-29T08:26:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55619d2dc040 con 0x55619d4d9d30 2024-03-29T08:26:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55619d2a5420 con 0x55619d490f60 2024-03-29T08:26:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 10 client.0 _ll_register_callbacks cb 0x55619d3580c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:26:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a7f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4da650 0x55619d4dedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a7f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4da650 0x55619d4dedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47814/0 (socket says 172.21.15.46:47814) 2024-03-29T08:26:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a7f400640 1 -- 172.21.15.46:0/3651827800 learned_addr learned my addr 172.21.15.46:0/3651827800 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:30.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a76000640 1 --2- 172.21.15.46:0/3651827800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d4d9d30 0x55619d4de870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a7ea00640 1 --2- 172.21.15.46:0/3651827800 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d490f60 0x55619d538e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a7f400640 1 -- 172.21.15.46:0/3651827800 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d490f60 msgr2=0x55619d538e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a7f400640 1 --2- 172.21.15.46:0/3651827800 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d490f60 0x55619d538e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a7f400640 1 -- 172.21.15.46:0/3651827800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d4d9d30 msgr2=0x55619d4de870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a7f400640 1 --2- 172.21.15.46:0/3651827800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d4d9d30 0x55619d4de870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a7f400640 1 -- 172.21.15.46:0/3651827800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8006d340 con 0x55619d4da650 2024-03-29T08:26:30.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a7f400640 1 --2- 172.21.15.46:0/3651827800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4da650 0x55619d4dedb0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f4a78015750 tx=0x7f4a78015b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3829c6131c6fb766 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:30.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a63e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:26:30.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a63e00640 1 -- 172.21.15.46:0/3651827800 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a78015e60 con 0x55619d4da650 2024-03-29T08:26:30.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a63e00640 1 -- 172.21.15.46:0/3651827800 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f4a78006d70 con 0x55619d4da650 2024-03-29T08:26:30.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 -- 172.21.15.46:0/3651827800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55619d29d410 con 0x55619d4da650 2024-03-29T08:26:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a63e00640 1 -- 172.21.15.46:0/3651827800 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a78003980 con 0x55619d4da650 2024-03-29T08:26:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 10 client.8097 fetch_fsmap learned FSMap version 69 2024-03-29T08:26:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 -- 172.21.15.46:0/3651827800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55619d289ef0 con 0x55619d4da650 2024-03-29T08:26:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a63e00640 1 -- 172.21.15.46:0/3651827800 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55619d29d410 con 0x55619d4da650 2024-03-29T08:26:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a63e00640 1 -- 172.21.15.46:0/3651827800 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f4a78003ae0 con 0x55619d4da650 2024-03-29T08:26:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 10 client.8097 fetch_fsmap finished waiting for FSMap version 69 2024-03-29T08:26:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 10 client.8097 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:26:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 20 client.8097 populate_metadata read hostname 'smithi046' 2024-03-29T08:26:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/3651827800 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x55619d5393c0 0x55619d4df2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 4 client.8097 mds_command: new command op to 7986 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-03-29T08:26:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 -- 172.21.15.46:0/3651827800 --> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x55619d2d7700 con 0x55619d5393c0 2024-03-29T08:26:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a7ea00640 1 --2- 172.21.15.46:0/3651827800 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x55619d5393c0 0x55619d4df2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a7ea00640 1 --2- 172.21.15.46:0/3651827800 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x55619d5393c0 0x55619d4df2f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2b8d5441e932c5a server_cookie=6771eb8f3279f6b0 in_seq=0 out_seq=0 2024-03-29T08:26:30.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a63e00640 10 client.8097 ms_handle_connect on v2:172.21.15.46:6833/1886680593 2024-03-29T08:26:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a63e00640 1 -- 172.21.15.46:0/3651827800 <== mds.0 v2:172.21.15.46:6833/1886680593 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55619d2d7700 con 0x55619d5393c0 2024-03-29T08:26:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a63e00640 10 client.8097 handle_command_reply: tid=0 2024-03-29T08:26:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 client.8097 shutdown 2024-03-29T08:26:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 -- 172.21.15.46:0/3651827800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4da650 msgr2=0x55619d4dedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/3651827800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4da650 0x55619d4dedb0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f4a78015750 tx=0x7f4a78015b50 comp rx=0 tx=0).stop 2024-03-29T08:26:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 -- 172.21.15.46:0/3651827800 shutdown_connections 2024-03-29T08:26:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/3651827800 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x55619d5393c0 0x55619d4df2f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/3651827800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55619d4d9d30 0x55619d4de870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/3651827800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55619d4da650 0x55619d4dedb0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 --2- 172.21.15.46:0/3651827800 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55619d490f60 0x55619d538e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a63e00640 0 client.8097 ms_handle_reset on v2:172.21.15.46:6833/1886680593 2024-03-29T08:26:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 -- 172.21.15.46:0/3651827800 >> 172.21.15.46:0/3651827800 conn(0x55619d48cf10 msgr2=0x55619d48f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 -- 172.21.15.46:0/3651827800 shutdown_connections 2024-03-29T08:26:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.347+0000 7f4a88d4e000 1 -- 172.21.15.46:0/3651827800 wait complete. 2024-03-29T08:26:30.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.351+0000 7f4a88d4e000 20 client.8097 trim_cache size 0 max 16384 2024-03-29T08:26:30.357 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:26:30.357 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-29T08:26:30.357 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "a505da77-7361-44dd-9f30-f91f749b15b3", 2024-03-29T08:26:30.357 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-29T08:26:30.357 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:26:30.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.351+0000 7f4a85800640 1 -- 172.21.15.46:0/2965898779 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4a5c066690 msgr2=0x7f4a5c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.351+0000 7f4a85800640 1 --2- 172.21.15.46:0/2965898779 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4a5c066690 0x7f4a5c068b50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f4a680ae000 tx=0x7f4a680a7000 comp rx=0 tx=0).stop 2024-03-29T08:26:30.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.351+0000 7f4a85800640 1 -- 172.21.15.46:0/2965898779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a800cec80 msgr2=0x7f4a80157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.351+0000 7f4a85800640 1 --2- 172.21.15.46:0/2965898779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a800cec80 0x7f4a80157a40 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f4a7000c4a0 tx=0x7f4a7000c960 comp rx=0 tx=0).stop 2024-03-29T08:26:30.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.351+0000 7f4a85800640 1 -- 172.21.15.46:0/2965898779 shutdown_connections 2024-03-29T08:26:30.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.351+0000 7f4a85800640 1 --2- 172.21.15.46:0/2965898779 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4a5c066690 0x7f4a5c068b50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.351+0000 7f4a85800640 1 --2- 172.21.15.46:0/2965898779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a80140f40 0x7f4a8013d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.351+0000 7f4a85800640 1 --2- 172.21.15.46:0/2965898779 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4a80141cb0 0x7f4a800a88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.351+0000 7f4a85800640 1 --2- 172.21.15.46:0/2965898779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a800cec80 0x7f4a80157a40 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.351+0000 7f4a85800640 1 -- 172.21.15.46:0/2965898779 >> 172.21.15.46:0/2965898779 conn(0x7f4a800b42d0 msgr2=0x7f4a80138980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:30.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.351+0000 7f4a85800640 1 -- 172.21.15.46:0/2965898779 shutdown_connections 2024-03-29T08:26:30.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.351+0000 7f4a85800640 1 -- 172.21.15.46:0/2965898779 wait complete. 2024-03-29T08:26:30.359 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-03-29T08:26:30.351+0000 7f5014200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55bb7d366680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-29T08:26:30.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:26:30.372 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-29T08:26:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.511+0000 7f0fb3c00640 1 Processor -- start 2024-03-29T08:26:30.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.511+0000 7f0fb3c00640 1 -- start start 2024-03-29T08:26:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.515+0000 7f0fb3c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0fac143eb0 0x7f0fac0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.515+0000 7f0fb3c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0fac0b1d40 0x7f0fac0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.515+0000 7f0fb3c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fac0b2360 con 0x7f0fac15efe0 2024-03-29T08:26:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.515+0000 7f0fb3c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fac15e500 con 0x7f0fac143eb0 2024-03-29T08:26:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.515+0000 7f0fb3c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fac15e7e0 con 0x7f0fac0b1d40 2024-03-29T08:26:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.515+0000 7f0fb1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0fac143eb0 0x7f0fac0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.515+0000 7f0fb1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0fac143eb0 0x7f0fac0cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47818/0 (socket says 172.21.15.46:47818) 2024-03-29T08:26:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.515+0000 7f0fb1200640 1 -- 172.21.15.46:0/1440753684 learned_addr learned my addr 172.21.15.46:0/1440753684 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:30.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.515+0000 7f0fb1c00640 1 --2- 172.21.15.46:0/1440753684 >> v2:172.21.15.73:3300/0 conn(0x7f0fac0b1d40 0x7f0fac0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.515+0000 7f0fb1200640 1 -- 172.21.15.46:0/1440753684 >> v2:172.21.15.73:3300/0 conn(0x7f0fac0b1d40 msgr2=0x7f0fac0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.515+0000 7f0fb1200640 1 --2- 172.21.15.46:0/1440753684 >> v2:172.21.15.73:3300/0 conn(0x7f0fac0b1d40 0x7f0fac0cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.515+0000 7f0fb1200640 1 -- 172.21.15.46:0/1440753684 >> v1:172.21.15.73:6789/0 conn(0x7f0fac15efe0 legacy=0x7f0fac0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.515+0000 7f0fb1200640 1 -- 172.21.15.46:0/1440753684 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fac15eac0 con 0x7f0fac143eb0 2024-03-29T08:26:30.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.515+0000 7f0fab400640 1 -- 172.21.15.46:0/1440753684 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892712441 0 0) 0x7f0fac0b2360 con 0x7f0fac15efe0 2024-03-29T08:26:30.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.515+0000 7f0fb1200640 1 --2- 172.21.15.46:0/1440753684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fac143eb0 0x7f0fac0cb790 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f0fac06ad40 tx=0x7f0fa001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8130ceca51ef7473 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fab400640 1 -- 172.21.15.46:0/1440753684 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa00a7070 con 0x7f0fac143eb0 2024-03-29T08:26:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fab400640 1 -- 172.21.15.46:0/1440753684 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa00ab430 con 0x7f0fac143eb0 2024-03-29T08:26:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fab400640 1 -- 172.21.15.46:0/1440753684 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa00ab7c0 con 0x7f0fac143eb0 2024-03-29T08:26:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb3c00640 1 -- 172.21.15.46:0/1440753684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fac143eb0 msgr2=0x7f0fac0cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb3c00640 1 --2- 172.21.15.46:0/1440753684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fac143eb0 0x7f0fac0cb790 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f0fac06ad40 tx=0x7f0fa001c520 comp rx=0 tx=0).stop 2024-03-29T08:26:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb3c00640 1 -- 172.21.15.46:0/1440753684 shutdown_connections 2024-03-29T08:26:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb3c00640 1 --2- 172.21.15.46:0/1440753684 >> v2:172.21.15.73:3300/0 conn(0x7f0fac0b1d40 0x7f0fac0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb3c00640 1 --2- 172.21.15.46:0/1440753684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fac143eb0 0x7f0fac0cb790 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb3c00640 1 -- 172.21.15.46:0/1440753684 >> 172.21.15.46:0/1440753684 conn(0x7f0fac06a390 msgr2=0x7f0fac0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:30.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb3c00640 1 -- 172.21.15.46:0/1440753684 shutdown_connections 2024-03-29T08:26:30.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb3c00640 1 -- 172.21.15.46:0/1440753684 wait complete. 2024-03-29T08:26:30.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb3c00640 1 Processor -- start 2024-03-29T08:26:30.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb3c00640 1 -- start start 2024-03-29T08:26:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb3c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0fac0b1d40 0x7f0fac13cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb3c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fac143eb0 0x7f0fac13bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb3c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fac15efe0 0x7f0fac13c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb3c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac13c7e0 con 0x7f0fac143eb0 2024-03-29T08:26:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb3c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac1f3390 con 0x7f0fac15efe0 2024-03-29T08:26:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb3c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac142950 con 0x7f0fac0b1d40 2024-03-29T08:26:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fabe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fac143eb0 0x7f0fac13bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fabe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fac143eb0 0x7f0fac13bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47828/0 (socket says 172.21.15.46:47828) 2024-03-29T08:26:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fabe00640 1 -- 172.21.15.46:0/147895391 learned_addr learned my addr 172.21.15.46:0/147895391 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb1200640 1 --2- 172.21.15.46:0/147895391 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0fac0b1d40 0x7f0fac13cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fb1c00640 1 --2- 172.21.15.46:0/147895391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fac15efe0 0x7f0fac13c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fabe00640 1 -- 172.21.15.46:0/147895391 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0fac0b1d40 msgr2=0x7f0fac13cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fabe00640 1 --2- 172.21.15.46:0/147895391 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0fac0b1d40 0x7f0fac13cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fabe00640 1 -- 172.21.15.46:0/147895391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fac15efe0 msgr2=0x7f0fac13c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fabe00640 1 --2- 172.21.15.46:0/147895391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fac15efe0 0x7f0fac13c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.519+0000 7f0fabe00640 1 -- 172.21.15.46:0/147895391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa00094d0 con 0x7f0fac143eb0 2024-03-29T08:26:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.523+0000 7f0fabe00640 1 --2- 172.21.15.46:0/147895391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fac143eb0 0x7f0fac13bd60 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f0f98001ec0 tx=0x7f0f9800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.523+0000 7f0fa8c00640 1 -- 172.21.15.46:0/147895391 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f980054e0 con 0x7f0fac143eb0 2024-03-29T08:26:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.523+0000 7f0fa8c00640 1 -- 172.21.15.46:0/147895391 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f98003600 con 0x7f0fac143eb0 2024-03-29T08:26:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.523+0000 7f0fa8c00640 1 -- 172.21.15.46:0/147895391 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f980039b0 con 0x7f0fac143eb0 2024-03-29T08:26:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.523+0000 7f0fb3c00640 1 -- 172.21.15.46:0/147895391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fac142c30 con 0x7f0fac143eb0 2024-03-29T08:26:30.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.523+0000 7f0fb3c00640 1 -- 172.21.15.46:0/147895391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fac1431f0 con 0x7f0fac143eb0 2024-03-29T08:26:30.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.523+0000 7f0fa8c00640 1 -- 172.21.15.46:0/147895391 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0f98003b10 con 0x7f0fac143eb0 2024-03-29T08:26:30.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fa8c00640 1 --2- 172.21.15.46:0/147895391 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f8c0666c0 0x7f0f8c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fa8c00640 1 -- 172.21.15.46:0/147895391 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f0f980837f0 con 0x7f0fac143eb0 2024-03-29T08:26:30.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb6f79000 1 Processor -- start 2024-03-29T08:26:30.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb1200640 1 --2- 172.21.15.46:0/147895391 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f8c0666c0 0x7f0f8c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb6f79000 1 -- start start 2024-03-29T08:26:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb1200640 1 --2- 172.21.15.46:0/147895391 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f8c0666c0 0x7f0f8c068b80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00af000 tx=0x7f0fa00b6040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb6f79000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a76173e60 0x560a76174240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb6f79000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76174780 0x560a761278d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb6f79000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a76127e10 0x560a761281f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb6f79000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a75fee630 con 0x560a76174780 2024-03-29T08:26:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb6f79000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a75f7bbd0 con 0x560a76173e60 2024-03-29T08:26:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb6f79000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a75f791a0 con 0x560a76127e10 2024-03-29T08:26:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fabe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76174780 0x560a761278d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fabe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76174780 0x560a761278d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47838/0 (socket says 172.21.15.46:47838) 2024-03-29T08:26:30.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fabe00640 1 -- 172.21.15.46:0/1158550119 learned_addr learned my addr 172.21.15.46:0/1158550119 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb1c00640 1 --2- 172.21.15.46:0/1158550119 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a76127e10 0x560a761281f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb1200640 1 --2- 172.21.15.46:0/1158550119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a76173e60 0x560a76174240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fabe00640 1 -- 172.21.15.46:0/1158550119 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a76127e10 msgr2=0x560a761281f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fabe00640 1 --2- 172.21.15.46:0/1158550119 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a76127e10 0x560a761281f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fabe00640 1 -- 172.21.15.46:0/1158550119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a76173e60 msgr2=0x560a76174240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fabe00640 1 --2- 172.21.15.46:0/1158550119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a76173e60 0x560a76174240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fabe00640 1 -- 172.21.15.46:0/1158550119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a75c70e20 con 0x560a76174780 2024-03-29T08:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fabe00640 1 --2- 172.21.15.46:0/1158550119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76174780 0x560a761278d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f0f9805c590 tx=0x7f0f98054210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82dd2869ff9e3e6b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0f9d600640 1 -- 172.21.15.46:0/1158550119 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98003dc0 con 0x560a76174780 2024-03-29T08:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0f9d600640 1 -- 172.21.15.46:0/1158550119 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f98052b20 con 0x560a76174780 2024-03-29T08:26:30.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0f9d600640 1 -- 172.21.15.46:0/1158550119 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98050e90 con 0x560a76174780 2024-03-29T08:26:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb6f79000 1 -- 172.21.15.46:0/1158550119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76174780 msgr2=0x560a761278d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/1158550119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76174780 0x560a761278d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f0f9805c590 tx=0x7f0f98054210 comp rx=0 tx=0).stop 2024-03-29T08:26:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb6f79000 1 -- 172.21.15.46:0/1158550119 shutdown_connections 2024-03-29T08:26:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/1158550119 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a76127e10 0x560a761281f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/1158550119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76174780 0x560a761278d0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/1158550119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a76173e60 0x560a76174240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb6f79000 1 -- 172.21.15.46:0/1158550119 >> 172.21.15.46:0/1158550119 conn(0x560a75f24160 msgr2=0x560a76152820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb6f79000 1 -- 172.21.15.46:0/1158550119 shutdown_connections 2024-03-29T08:26:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.527+0000 7f0fb6f79000 1 -- 172.21.15.46:0/1158550119 wait complete. 2024-03-29T08:26:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 1 -- start start 2024-03-29T08:26:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 1 Processor -- start 2024-03-29T08:26:30.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a76127e10 0x560a7617b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 0x560a7617b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a76174780 0x560a76156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a75f74a90 con 0x560a76173e60 2024-03-29T08:26:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a75f13d40 con 0x560a76127e10 2024-03-29T08:26:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a75f31f30 con 0x560a76174780 2024-03-29T08:26:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 10 client.0 _ll_register_callbacks cb 0x560a75fd6810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:26:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fabe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 0x560a7617b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fabe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 0x560a7617b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47854/0 (socket says 172.21.15.46:47854) 2024-03-29T08:26:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fabe00640 1 -- 172.21.15.46:0/3661176575 learned_addr learned my addr 172.21.15.46:0/3661176575 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb1c00640 1 --2- 172.21.15.46:0/3661176575 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a76174780 0x560a76156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb1200640 1 --2- 172.21.15.46:0/3661176575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a76127e10 0x560a7617b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fabe00640 1 -- 172.21.15.46:0/3661176575 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a76174780 msgr2=0x560a76156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fabe00640 1 --2- 172.21.15.46:0/3661176575 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a76174780 0x560a76156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fabe00640 1 -- 172.21.15.46:0/3661176575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a76127e10 msgr2=0x560a7617b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fabe00640 1 --2- 172.21.15.46:0/3661176575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a76127e10 0x560a7617b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fabe00640 1 -- 172.21.15.46:0/3661176575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fac1431f0 con 0x560a76173e60 2024-03-29T08:26:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb1200640 1 --2- 172.21.15.46:0/3661176575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a76127e10 0x560a7617b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:26:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fabe00640 1 --2- 172.21.15.46:0/3661176575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 0x560a7617b700 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f0f9805c590 tx=0x7f0f9800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6d376a86efee3b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0f92000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:26:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0f92000640 1 -- 172.21.15.46:0/3661176575 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98050e90 con 0x560a76173e60 2024-03-29T08:26:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0f92000640 1 -- 172.21.15.46:0/3661176575 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f0f98050ff0 con 0x560a76173e60 2024-03-29T08:26:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 1 -- 172.21.15.46:0/3661176575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a75f733e0 con 0x560a76173e60 2024-03-29T08:26:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0f92000640 1 -- 172.21.15.46:0/3661176575 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f9805c630 con 0x560a76173e60 2024-03-29T08:26:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 10 client.8109 fetch_fsmap learned FSMap version 69 2024-03-29T08:26:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 1 -- 172.21.15.46:0/3661176575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x560a75f39aa0 con 0x560a76173e60 2024-03-29T08:26:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0f92000640 1 -- 172.21.15.46:0/3661176575 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560a75f733e0 con 0x560a76173e60 2024-03-29T08:26:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0f92000640 1 -- 172.21.15.46:0/3661176575 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0f98050570 con 0x560a76173e60 2024-03-29T08:26:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 10 client.8109 fetch_fsmap finished waiting for FSMap version 69 2024-03-29T08:26:30.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 10 client.8109 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:26:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 20 client.8109 populate_metadata read hostname 'smithi046' 2024-03-29T08:26:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/3661176575 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x560a76157a60 0x560a76157e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 4 client.8109 mds_command: new command op to 7986 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:26:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb6f79000 1 -- 172.21.15.46:0/3661176575 --> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a75f44d00 con 0x560a76157a60 2024-03-29T08:26:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb1200640 1 --2- 172.21.15.46:0/3661176575 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x560a76157a60 0x560a76157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0f92000640 10 client.8109 ms_handle_connect on v2:172.21.15.46:6833/1886680593 2024-03-29T08:26:30.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.531+0000 7f0fb1200640 1 --2- 172.21.15.46:0/3661176575 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x560a76157a60 0x560a76157e70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ca7457680e6aa9 server_cookie=3d02d190940e88cc in_seq=0 out_seq=0 2024-03-29T08:26:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.535+0000 7f0f92000640 1 -- 172.21.15.46:0/3661176575 <== mds.0 v2:172.21.15.46:6833/1886680593 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560a75f44d00 con 0x560a76157a60 2024-03-29T08:26:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.535+0000 7f0f92000640 10 client.8109 handle_command_reply: tid=0 2024-03-29T08:26:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.535+0000 7f0fb6f79000 1 client.8109 shutdown 2024-03-29T08:26:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.535+0000 7f0fb6f79000 1 -- 172.21.15.46:0/3661176575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 msgr2=0x560a7617b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.535+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/3661176575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 0x560a7617b700 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f0f9805c590 tx=0x7f0f9800cf90 comp rx=0 tx=0).stop 2024-03-29T08:26:30.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.535+0000 7f0fb6f79000 1 -- 172.21.15.46:0/3661176575 shutdown_connections 2024-03-29T08:26:30.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.535+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/3661176575 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x560a76157a60 0x560a76157e70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.535+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/3661176575 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a76174780 0x560a76156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.535+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/3661176575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 0x560a7617b700 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.535+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/3661176575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a76127e10 0x560a7617b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.535+0000 7f0f92000640 0 client.8109 ms_handle_reset on v2:172.21.15.46:6833/1886680593 2024-03-29T08:26:30.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.535+0000 7f0fb6f79000 1 -- 172.21.15.46:0/3661176575 >> 172.21.15.46:0/3661176575 conn(0x560a75f24160 msgr2=0x560a76122bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:30.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.535+0000 7f0fb6f79000 1 -- 172.21.15.46:0/3661176575 shutdown_connections 2024-03-29T08:26:30.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.535+0000 7f0fb6f79000 1 -- 172.21.15.46:0/3661176575 wait complete. 2024-03-29T08:26:30.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.535+0000 7f0fb6f79000 20 client.8109 trim_cache size 0 max 16384 2024-03-29T08:26:30.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.547+0000 7f0fb6f79000 1 Processor -- start 2024-03-29T08:26:30.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.547+0000 7f0fb6f79000 1 -- start start 2024-03-29T08:26:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb6f79000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a75f24160 0x560a761cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb6f79000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a76127e10 0x560a761ce100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb6f79000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 0x560a761ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb6f79000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a75c31aa0 con 0x560a76173e60 2024-03-29T08:26:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb6f79000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a75f71e90 con 0x560a76127e10 2024-03-29T08:26:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb6f79000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a75f76020 con 0x560a75f24160 2024-03-29T08:26:30.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 0x560a761ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 0x560a761ce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47862/0 (socket says 172.21.15.46:47862) 2024-03-29T08:26:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb1c00640 1 -- 172.21.15.46:0/3169896499 learned_addr learned my addr 172.21.15.46:0/3169896499 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb1200640 1 --2- 172.21.15.46:0/3169896499 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a75f24160 0x560a761cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fabe00640 1 --2- 172.21.15.46:0/3169896499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a76127e10 0x560a761ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb1c00640 1 -- 172.21.15.46:0/3169896499 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a75f24160 msgr2=0x560a761cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb1c00640 1 --2- 172.21.15.46:0/3169896499 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a75f24160 0x560a761cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb1c00640 1 -- 172.21.15.46:0/3169896499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a76127e10 msgr2=0x560a761ce100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb1c00640 1 --2- 172.21.15.46:0/3169896499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a76127e10 0x560a761ce100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb1c00640 1 -- 172.21.15.46:0/3169896499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a75f1fdf0 con 0x560a76173e60 2024-03-29T08:26:30.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb1c00640 1 --2- 172.21.15.46:0/3169896499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 0x560a761ce640 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4017760 tx=0x7f0fa4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e57f7742947e6d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0f9e000640 1 -- 172.21.15.46:0/3169896499 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa400e040 con 0x560a76173e60 2024-03-29T08:26:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0f9e000640 1 -- 172.21.15.46:0/3169896499 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa400f040 con 0x560a76173e60 2024-03-29T08:26:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0f9e000640 1 -- 172.21.15.46:0/3169896499 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4015d20 con 0x560a76173e60 2024-03-29T08:26:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb6f79000 1 -- 172.21.15.46:0/3169896499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 msgr2=0x560a761ce640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/3169896499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 0x560a761ce640 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4017760 tx=0x7f0fa4009750 comp rx=0 tx=0).stop 2024-03-29T08:26:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb6f79000 1 -- 172.21.15.46:0/3169896499 shutdown_connections 2024-03-29T08:26:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/3169896499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a76127e10 0x560a761ce100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/3169896499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 0x560a761ce640 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/3169896499 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a75f24160 0x560a761cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb6f79000 1 -- 172.21.15.46:0/3169896499 >> 172.21.15.46:0/3169896499 conn(0x560a76181c40 msgr2=0x560a76122b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb6f79000 1 -- 172.21.15.46:0/3169896499 shutdown_connections 2024-03-29T08:26:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb6f79000 1 -- 172.21.15.46:0/3169896499 wait complete. 2024-03-29T08:26:30.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb6f79000 1 -- start start 2024-03-29T08:26:30.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.551+0000 7f0fb6f79000 1 Processor -- start 2024-03-29T08:26:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a75f24160 0x560a761cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a76127e10 0x560a76129cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 0x560a7612a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a75fb8170 con 0x560a76173e60 2024-03-29T08:26:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a7611d930 con 0x560a75f24160 2024-03-29T08:26:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a7602dec0 con 0x560a76127e10 2024-03-29T08:26:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 10 client.0 _ll_register_callbacks cb 0x7f0fa40095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:26:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a75f24160 0x560a761cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 0x560a7612a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb1200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a75f24160 0x560a761cf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47054/0 (socket says 172.21.15.46:47054) 2024-03-29T08:26:30.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb1200640 1 -- 172.21.15.46:0/2925125426 learned_addr learned my addr 172.21.15.46:0/2925125426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:26:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fabe00640 1 --2- 172.21.15.46:0/2925125426 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a76127e10 0x560a76129cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb1200640 1 -- 172.21.15.46:0/2925125426 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a76127e10 msgr2=0x560a76129cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb1200640 1 --2- 172.21.15.46:0/2925125426 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a76127e10 0x560a76129cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb1200640 1 -- 172.21.15.46:0/2925125426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 msgr2=0x560a7612a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:26:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb1200640 1 --2- 172.21.15.46:0/2925125426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 0x560a7612a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb1200640 1 -- 172.21.15.46:0/2925125426 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa4017410 con 0x560a75f24160 2024-03-29T08:26:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb1c00640 1 --2- 172.21.15.46:0/2925125426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 0x560a7612a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:26:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb1200640 1 --2- 172.21.15.46:0/2925125426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a75f24160 0x560a761cf1c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0019130 tx=0x7f0fa001ad60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e8a8150343800f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:26:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0f92000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-29T08:26:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0f92000640 1 -- 172.21.15.46:0/2925125426 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa00c5040 con 0x560a75f24160 2024-03-29T08:26:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0f92000640 1 -- 172.21.15.46:0/2925125426 <== mon.1 v2:172.21.15.73:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f0fa00cc4b0 con 0x560a75f24160 2024-03-29T08:26:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 -- 172.21.15.46:0/2925125426 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a75f31970 con 0x560a75f24160 2024-03-29T08:26:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0f92000640 1 -- 172.21.15.46:0/2925125426 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fa0003620 con 0x560a75f24160 2024-03-29T08:26:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 10 client.7777 fetch_fsmap learned FSMap version 69 2024-03-29T08:26:30.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 -- 172.21.15.46:0/2925125426 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x560a75f13960 con 0x560a75f24160 2024-03-29T08:26:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0f92000640 1 -- 172.21.15.46:0/2925125426 <== mon.1 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560a75f31970 con 0x560a75f24160 2024-03-29T08:26:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0f92000640 1 -- 172.21.15.46:0/2925125426 <== mon.1 v2:172.21.15.73:3300/0 5 ==== fsmap(e 69) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0fa0003780 con 0x560a75f24160 2024-03-29T08:26:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 10 client.7777 fetch_fsmap finished waiting for FSMap version 69 2024-03-29T08:26:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 10 client.7777 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:26:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 20 client.7777 populate_metadata read hostname 'smithi046' 2024-03-29T08:26:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/2925125426 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x560a76174780 0x560a7612a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:26:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 4 client.7777 mds_command: new command op to 7986 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:26:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 -- 172.21.15.46:0/2925125426 --> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560a75fa3a20 con 0x560a76174780 2024-03-29T08:26:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fabe00640 1 --2- 172.21.15.46:0/2925125426 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x560a76174780 0x560a7612a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:26:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0f92000640 10 client.7777 ms_handle_connect on v2:172.21.15.46:6833/1886680593 2024-03-29T08:26:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fabe00640 1 --2- 172.21.15.46:0/2925125426 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x560a76174780 0x560a7612a770 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3213796370bfb7ed server_cookie=d9a966539a58000a in_seq=0 out_seq=0 2024-03-29T08:26:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0f92000640 1 -- 172.21.15.46:0/2925125426 <== mds.0 v2:172.21.15.46:6833/1886680593 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x560a75fa3a20 con 0x560a76174780 2024-03-29T08:26:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0f92000640 10 client.7777 handle_command_reply: tid=0 2024-03-29T08:26:30.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 client.7777 shutdown 2024-03-29T08:26:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 -- 172.21.15.46:0/2925125426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a75f24160 msgr2=0x560a761cf1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/2925125426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a75f24160 0x560a761cf1c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0019130 tx=0x7f0fa001ad60 comp rx=0 tx=0).stop 2024-03-29T08:26:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 -- 172.21.15.46:0/2925125426 shutdown_connections 2024-03-29T08:26:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/2925125426 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x560a76174780 0x560a7612a770 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/2925125426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560a76173e60 0x560a7612a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0f92000640 0 client.7777 ms_handle_reset on v2:172.21.15.46:6833/1886680593 2024-03-29T08:26:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/2925125426 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560a76127e10 0x560a76129cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 --2- 172.21.15.46:0/2925125426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560a75f24160 0x560a761cf1c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 -- 172.21.15.46:0/2925125426 >> 172.21.15.46:0/2925125426 conn(0x560a76181c40 msgr2=0x560a76155880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 -- 172.21.15.46:0/2925125426 shutdown_connections 2024-03-29T08:26:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 1 -- 172.21.15.46:0/2925125426 wait complete. 2024-03-29T08:26:30.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.555+0000 7f0fb6f79000 20 client.7777 trim_cache size 0 max 16384 2024-03-29T08:26:30.563 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:26:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-29T08:26:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-29T08:26:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: "a505da77-7361-44dd-9f30-f91f749b15b3": { 2024-03-29T08:26:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/dir", 2024-03-29T08:26:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "a505da77-7361-44dd-9f30-f91f749b15b3", 2024-03-29T08:26:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,repair" 2024-03-29T08:26:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:26:30.563 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:26:30.563 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:26:30.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.559+0000 7f0fb3c00640 1 -- 172.21.15.46:0/147895391 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f8c0666c0 msgr2=0x7f0f8c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.559+0000 7f0fb3c00640 1 --2- 172.21.15.46:0/147895391 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f8c0666c0 0x7f0f8c068b80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00af000 tx=0x7f0fa00b6040 comp rx=0 tx=0).stop 2024-03-29T08:26:30.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.559+0000 7f0fb3c00640 1 -- 172.21.15.46:0/147895391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fac143eb0 msgr2=0x7f0fac13bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:26:30.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.559+0000 7f0fb3c00640 1 --2- 172.21.15.46:0/147895391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fac143eb0 0x7f0fac13bd60 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f0f98001ec0 tx=0x7f0f9800c8e0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.559+0000 7f0fb3c00640 1 -- 172.21.15.46:0/147895391 shutdown_connections 2024-03-29T08:26:30.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.559+0000 7f0fb3c00640 1 --2- 172.21.15.46:0/147895391 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0f8c0666c0 0x7f0f8c068b80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.559+0000 7f0fb3c00640 1 --2- 172.21.15.46:0/147895391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fac15efe0 0x7f0fac13c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.559+0000 7f0fb3c00640 1 --2- 172.21.15.46:0/147895391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0fac143eb0 0x7f0fac13bd60 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.559+0000 7f0fb3c00640 1 --2- 172.21.15.46:0/147895391 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0fac0b1d40 0x7f0fac13cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:26:30.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.559+0000 7f0fb3c00640 1 -- 172.21.15.46:0/147895391 >> 172.21.15.46:0/147895391 conn(0x7f0fac06a390 msgr2=0x7f0fac0cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:26:30.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.559+0000 7f0fb3c00640 1 -- 172.21.15.46:0/147895391 shutdown_connections 2024-03-29T08:26:30.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:26:30.559+0000 7f0fb3c00640 1 -- 172.21.15.46:0/147895391 wait complete. 2024-03-29T08:26:30.578 INFO:tasks.cephfs.filesystem:scrub status for tag:a505da77-7361-44dd-9f30-f91f749b15b3 - {'path': '/dir', 'tag': 'a505da77-7361-44dd-9f30-f91f749b15b3', 'options': 'recursive,repair'} 2024-03-29T08:26:39.407 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:26:39.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:27:00.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:00.579 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-03-29T08:27:00.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.731+0000 7f1b86c00640 1 Processor -- start 2024-03-29T08:27:00.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.731+0000 7f1b86c00640 1 -- start start 2024-03-29T08:27:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.735+0000 7f1b86c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1b80143eb0 0x7f1b800cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.735+0000 7f1b86c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b8015efe0 0x7f1b800cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.735+0000 7f1b86c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b800b2360 con 0x7f1b800b1630 2024-03-29T08:27:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.735+0000 7f1b86c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b8015e500 con 0x7f1b80143eb0 2024-03-29T08:27:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.735+0000 7f1b86c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8015e7e0 con 0x7f1b8015efe0 2024-03-29T08:27:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.735+0000 7f1b84c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1b800b1630 0x7f1b800b1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37842/0 (socket says 172.21.15.46:37842) 2024-03-29T08:27:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.735+0000 7f1b84c00640 1 -- 172.21.15.46:0/4178858804 learned_addr learned my addr 172.21.15.46:0/4178858804 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:00.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.735+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/4178858804 >> v2:172.21.15.73:3300/0 conn(0x7f1b80143eb0 0x7f1b800cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.735+0000 7f1b77400640 1 --2- 172.21.15.46:0/4178858804 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b8015efe0 0x7f1b800cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.735+0000 7f1b7f400640 1 -- 172.21.15.46:0/4178858804 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196451846 0 0) 0x7f1b800b2360 con 0x7f1b800b1630 2024-03-29T08:27:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.735+0000 7f1b7f400640 1 -- 172.21.15.46:0/4178858804 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b64003530 con 0x7f1b800b1630 2024-03-29T08:27:00.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.735+0000 7f1b7fe00640 1 -- 172.21.15.46:0/4178858804 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b8015efe0 msgr2=0x7f1b800cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.735+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/4178858804 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b8015efe0 0x7f1b800cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.735+0000 7f1b7fe00640 1 -- 172.21.15.46:0/4178858804 >> v1:172.21.15.46:6789/0 conn(0x7f1b800b1630 legacy=0x7f1b800b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:00.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.735+0000 7f1b7fe00640 1 -- 172.21.15.46:0/4178858804 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8015eac0 con 0x7f1b80143eb0 2024-03-29T08:27:00.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.739+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/4178858804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b80143eb0 0x7f1b800cb790 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1b700097c0 tx=0x7f1b7001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efea835d5b0699c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.739+0000 7f1b7f400640 1 -- 172.21.15.46:0/4178858804 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b700ab600 con 0x7f1b80143eb0 2024-03-29T08:27:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.739+0000 7f1b7f400640 1 -- 172.21.15.46:0/4178858804 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b700a7050 con 0x7f1b80143eb0 2024-03-29T08:27:00.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.739+0000 7f1b7f400640 1 -- 172.21.15.46:0/4178858804 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b700ab980 con 0x7f1b80143eb0 2024-03-29T08:27:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.739+0000 7f1b86c00640 1 -- 172.21.15.46:0/4178858804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b80143eb0 msgr2=0x7f1b800cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.739+0000 7f1b86c00640 1 --2- 172.21.15.46:0/4178858804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b80143eb0 0x7f1b800cb790 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1b700097c0 tx=0x7f1b7001c4a0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.739+0000 7f1b86c00640 1 -- 172.21.15.46:0/4178858804 shutdown_connections 2024-03-29T08:27:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.739+0000 7f1b86c00640 1 --2- 172.21.15.46:0/4178858804 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b8015efe0 0x7f1b800cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.739+0000 7f1b86c00640 1 --2- 172.21.15.46:0/4178858804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b80143eb0 0x7f1b800cb790 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.739+0000 7f1b86c00640 1 -- 172.21.15.46:0/4178858804 >> 172.21.15.46:0/4178858804 conn(0x7f1b8006a390 msgr2=0x7f1b800cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.739+0000 7f1b86c00640 1 -- 172.21.15.46:0/4178858804 shutdown_connections 2024-03-29T08:27:00.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.739+0000 7f1b86c00640 1 -- 172.21.15.46:0/4178858804 wait complete. 2024-03-29T08:27:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.739+0000 7f1b86c00640 1 Processor -- start 2024-03-29T08:27:00.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.739+0000 7f1b86c00640 1 -- start start 2024-03-29T08:27:00.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b86c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b800b1630 0x7f1b801f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b86c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b80143eb0 0x7f1b8013f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b86c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b8015efe0 0x7f1b800c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b86c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b800c6a50 con 0x7f1b80143eb0 2024-03-29T08:27:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b86c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b801f5da0 con 0x7f1b8015efe0 2024-03-29T08:27:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b86c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b800c8cb0 con 0x7f1b800b1630 2024-03-29T08:27:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b77400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b80143eb0 0x7f1b8013f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b77400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b80143eb0 0x7f1b8013f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41098/0 (socket says 172.21.15.46:41098) 2024-03-29T08:27:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b77400640 1 -- 172.21.15.46:0/602291054 learned_addr learned my addr 172.21.15.46:0/602291054 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/602291054 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b800b1630 0x7f1b801f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b84c00640 1 --2- 172.21.15.46:0/602291054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b8015efe0 0x7f1b800c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b77400640 1 -- 172.21.15.46:0/602291054 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b800b1630 msgr2=0x7f1b801f5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b77400640 1 --2- 172.21.15.46:0/602291054 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b800b1630 0x7f1b801f5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b77400640 1 -- 172.21.15.46:0/602291054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b8015efe0 msgr2=0x7f1b800c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b77400640 1 --2- 172.21.15.46:0/602291054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b8015efe0 0x7f1b800c9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b77400640 1 -- 172.21.15.46:0/602291054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b70009420 con 0x7f1b80143eb0 2024-03-29T08:27:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/602291054 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b800b1630 0x7f1b801f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b84c00640 1 --2- 172.21.15.46:0/602291054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b8015efe0 0x7f1b800c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:00.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b77400640 1 --2- 172.21.15.46:0/602291054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b80143eb0 0x7f1b8013f760 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800e930 tx=0x7f1b6800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:00.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b7cc00640 1 -- 172.21.15.46:0/602291054 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68009690 con 0x7f1b80143eb0 2024-03-29T08:27:00.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b7cc00640 1 -- 172.21.15.46:0/602291054 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6800ca00 con 0x7f1b80143eb0 2024-03-29T08:27:00.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b86c00640 1 -- 172.21.15.46:0/602291054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b800c8f90 con 0x7f1b80143eb0 2024-03-29T08:27:00.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b7cc00640 1 -- 172.21.15.46:0/602291054 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6800cd70 con 0x7f1b80143eb0 2024-03-29T08:27:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b86c00640 1 -- 172.21.15.46:0/602291054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b800c9550 con 0x7f1b80143eb0 2024-03-29T08:27:00.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.743+0000 7f1b7cc00640 1 -- 172.21.15.46:0/602291054 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1b68013470 con 0x7f1b80143eb0 2024-03-29T08:27:00.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b7cc00640 1 --2- 172.21.15.46:0/602291054 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b5c066690 0x7f1b5c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b7cc00640 1 -- 172.21.15.46:0/602291054 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f1b68084540 con 0x7f1b80143eb0 2024-03-29T08:27:00.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/602291054 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b5c066690 0x7f1b5c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b89feb000 1 Processor -- start 2024-03-29T08:27:00.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b89feb000 1 -- start start 2024-03-29T08:27:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/602291054 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b5c066690 0x7f1b5c068b50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1b700ad000 tx=0x7f1b70008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b89feb000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e846a70 0x55fc0e846e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b89feb000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e847390 0x55fc0e7fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b89feb000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e7faa00 0x55fc0e7fade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b89feb000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc0e6c5630 con 0x55fc0e847390 2024-03-29T08:27:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b89feb000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc0e652bd0 con 0x55fc0e7faa00 2024-03-29T08:27:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b89feb000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc0e6501a0 con 0x55fc0e846a70 2024-03-29T08:27:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b7fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e846a70 0x55fc0e846e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b77400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e847390 0x55fc0e7fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b7fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e846a70 0x55fc0e846e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:35010/0 (socket says 172.21.15.46:35010) 2024-03-29T08:27:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b7fe00640 1 -- 172.21.15.46:0/2055109579 learned_addr learned my addr 172.21.15.46:0/2055109579 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b84c00640 1 --2- 172.21.15.46:0/2055109579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e7faa00 0x55fc0e7fade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b77400640 1 -- 172.21.15.46:0/2055109579 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e846a70 msgr2=0x55fc0e846e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b77400640 1 --2- 172.21.15.46:0/2055109579 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e846a70 0x55fc0e846e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b77400640 1 -- 172.21.15.46:0/2055109579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e7faa00 msgr2=0x55fc0e7fade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b77400640 1 --2- 172.21.15.46:0/2055109579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e7faa00 0x55fc0e7fade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b77400640 1 -- 172.21.15.46:0/2055109579 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc0e347e20 con 0x55fc0e847390 2024-03-29T08:27:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/2055109579 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e846a70 0x55fc0e846e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:27:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b77400640 1 --2- 172.21.15.46:0/2055109579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e847390 0x55fc0e7fa4c0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f1b68082920 tx=0x7f1b680554e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ea75cbdc10eb3e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b74c00640 1 -- 172.21.15.46:0/2055109579 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68053880 con 0x55fc0e847390 2024-03-29T08:27:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b74c00640 1 -- 172.21.15.46:0/2055109579 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b68059020 con 0x55fc0e847390 2024-03-29T08:27:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.747+0000 7f1b74c00640 1 -- 172.21.15.46:0/2055109579 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6805d440 con 0x55fc0e847390 2024-03-29T08:27:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 -- 172.21.15.46:0/2055109579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e847390 msgr2=0x55fc0e7fa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 --2- 172.21.15.46:0/2055109579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e847390 0x55fc0e7fa4c0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f1b68082920 tx=0x7f1b680554e0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 -- 172.21.15.46:0/2055109579 shutdown_connections 2024-03-29T08:27:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 --2- 172.21.15.46:0/2055109579 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e7faa00 0x55fc0e7fade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 --2- 172.21.15.46:0/2055109579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e847390 0x55fc0e7fa4c0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 --2- 172.21.15.46:0/2055109579 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e846a70 0x55fc0e846e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 -- 172.21.15.46:0/2055109579 >> 172.21.15.46:0/2055109579 conn(0x55fc0e5fb160 msgr2=0x55fc0e829820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 -- 172.21.15.46:0/2055109579 shutdown_connections 2024-03-29T08:27:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 -- 172.21.15.46:0/2055109579 wait complete. 2024-03-29T08:27:00.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 -- start start 2024-03-29T08:27:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 Processor -- start 2024-03-29T08:27:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e7faa00 0x55fc0e8037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e846a70 0x55fc0e82aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e847390 0x55fc0e82af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc0e64ba90 con 0x55fc0e846a70 2024-03-29T08:27:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc0e5ead40 con 0x55fc0e847390 2024-03-29T08:27:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc0e608f30 con 0x55fc0e7faa00 2024-03-29T08:27:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 10 client.0 _ll_register_callbacks cb 0x55fc0e6ad810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:27:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b77400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e846a70 0x55fc0e82aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b77400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e846a70 0x55fc0e82aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41114/0 (socket says 172.21.15.46:41114) 2024-03-29T08:27:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b77400640 1 -- 172.21.15.46:0/113305877 learned_addr learned my addr 172.21.15.46:0/113305877 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b84c00640 1 --2- 172.21.15.46:0/113305877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e847390 0x55fc0e82af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/113305877 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e7faa00 0x55fc0e8037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b77400640 1 -- 172.21.15.46:0/113305877 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e7faa00 msgr2=0x55fc0e8037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b77400640 1 --2- 172.21.15.46:0/113305877 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e7faa00 0x55fc0e8037e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b77400640 1 -- 172.21.15.46:0/113305877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e847390 msgr2=0x55fc0e82af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b77400640 1 --2- 172.21.15.46:0/113305877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e847390 0x55fc0e82af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b77400640 1 -- 172.21.15.46:0/113305877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8015eac0 con 0x55fc0e846a70 2024-03-29T08:27:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/113305877 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e7faa00 0x55fc0e8037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b77400640 1 --2- 172.21.15.46:0/113305877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e846a70 0x55fc0e82aa50 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f1b68055470 tx=0x7f1b680092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884d839690c0645c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b6d600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:27:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b6d600640 1 -- 172.21.15.46:0/113305877 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68070030 con 0x55fc0e846a70 2024-03-29T08:27:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b6d600640 1 -- 172.21.15.46:0/113305877 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f1b68071070 con 0x55fc0e846a70 2024-03-29T08:27:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 -- 172.21.15.46:0/113305877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc0e64a3e0 con 0x55fc0e846a70 2024-03-29T08:27:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b6d600640 1 -- 172.21.15.46:0/113305877 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b68073310 con 0x55fc0e846a70 2024-03-29T08:27:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 10 client.8127 fetch_fsmap learned FSMap version 69 2024-03-29T08:27:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b89feb000 1 -- 172.21.15.46:0/113305877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x7f1b800c9550 con 0x55fc0e846a70 2024-03-29T08:27:00.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.751+0000 7f1b6d600640 1 -- 172.21.15.46:0/113305877 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fc0e64a3e0 con 0x55fc0e846a70 2024-03-29T08:27:00.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b6d600640 1 -- 172.21.15.46:0/113305877 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1b68011070 con 0x55fc0e846a70 2024-03-29T08:27:00.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 10 client.8127 fetch_fsmap finished waiting for FSMap version 69 2024-03-29T08:27:00.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 10 client.8127 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:27:00.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 20 client.8127 populate_metadata read hostname 'smithi046' 2024-03-29T08:27:00.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 1 --2- 172.21.15.46:0/113305877 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x55fc0e84e2d0 0x55fc0e84e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 4 client.8127 mds_command: new command op to 7986 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:27:00.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 1 -- 172.21.15.46:0/113305877 --> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc0e61bd00 con 0x55fc0e84e2d0 2024-03-29T08:27:00.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/113305877 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x55fc0e84e2d0 0x55fc0e84e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/113305877 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x55fc0e84e2d0 0x55fc0e84e6b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fff3191ee223f8a3 server_cookie=3f71f05024e96d26 in_seq=0 out_seq=0 2024-03-29T08:27:00.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b6d600640 10 client.8127 ms_handle_connect on v2:172.21.15.46:6833/1886680593 2024-03-29T08:27:00.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b6d600640 1 -- 172.21.15.46:0/113305877 <== mds.0 v2:172.21.15.46:6833/1886680593 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fc0e61bd00 con 0x55fc0e84e2d0 2024-03-29T08:27:00.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b6d600640 10 client.8127 handle_command_reply: tid=0 2024-03-29T08:27:00.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 1 client.8127 shutdown 2024-03-29T08:27:00.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 1 -- 172.21.15.46:0/113305877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e846a70 msgr2=0x55fc0e82aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:00.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 1 --2- 172.21.15.46:0/113305877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e846a70 0x55fc0e82aa50 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f1b68055470 tx=0x7f1b680092a0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 1 -- 172.21.15.46:0/113305877 shutdown_connections 2024-03-29T08:27:00.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 1 --2- 172.21.15.46:0/113305877 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x55fc0e84e2d0 0x55fc0e84e6b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 1 --2- 172.21.15.46:0/113305877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e847390 0x55fc0e82af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 1 --2- 172.21.15.46:0/113305877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e846a70 0x55fc0e82aa50 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 1 --2- 172.21.15.46:0/113305877 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e7faa00 0x55fc0e8037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b6d600640 0 client.8127 ms_handle_reset on v2:172.21.15.46:6833/1886680593 2024-03-29T08:27:00.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 1 -- 172.21.15.46:0/113305877 >> 172.21.15.46:0/113305877 conn(0x55fc0e5fb160 msgr2=0x55fc0e7f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:00.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 1 -- 172.21.15.46:0/113305877 shutdown_connections 2024-03-29T08:27:00.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 1 -- 172.21.15.46:0/113305877 wait complete. 2024-03-29T08:27:00.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.755+0000 7f1b89feb000 20 client.8127 trim_cache size 0 max 16384 2024-03-29T08:27:00.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 Processor -- start 2024-03-29T08:27:00.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 -- start start 2024-03-29T08:27:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e7faa00 0x55fc0e7f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e846a70 0x55fc0e7fa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e847390 0x55fc0e7fade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc0e308aa0 con 0x55fc0e7faa00 2024-03-29T08:27:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc0e64fa90 con 0x55fc0e846a70 2024-03-29T08:27:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc0e617390 con 0x55fc0e847390 2024-03-29T08:27:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e7faa00 0x55fc0e7f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b7fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e7faa00 0x55fc0e7f9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41122/0 (socket says 172.21.15.46:41122) 2024-03-29T08:27:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b7fe00640 1 -- 172.21.15.46:0/1973857866 learned_addr learned my addr 172.21.15.46:0/1973857866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b84c00640 1 --2- 172.21.15.46:0/1973857866 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e847390 0x55fc0e7fade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b77400640 1 --2- 172.21.15.46:0/1973857866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e846a70 0x55fc0e7fa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b7fe00640 1 -- 172.21.15.46:0/1973857866 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e847390 msgr2=0x55fc0e7fade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/1973857866 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e847390 0x55fc0e7fade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b7fe00640 1 -- 172.21.15.46:0/1973857866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e846a70 msgr2=0x55fc0e7fa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/1973857866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e846a70 0x55fc0e7fa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b7fe00640 1 -- 172.21.15.46:0/1973857866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b800c47d0 con 0x55fc0e7faa00 2024-03-29T08:27:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b77400640 1 --2- 172.21.15.46:0/1973857866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e846a70 0x55fc0e7fa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/1973857866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e7faa00 0x55fc0e7f9c30 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f1b7001c980 tx=0x7f1b700a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed7d0c7ba38c93f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b75600640 1 -- 172.21.15.46:0/1973857866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b700bd040 con 0x55fc0e7faa00 2024-03-29T08:27:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b75600640 1 -- 172.21.15.46:0/1973857866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b700abe80 con 0x55fc0e7faa00 2024-03-29T08:27:00.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b75600640 1 -- 172.21.15.46:0/1973857866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b700c6bf0 con 0x55fc0e7faa00 2024-03-29T08:27:00.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 -- 172.21.15.46:0/1973857866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e7faa00 msgr2=0x55fc0e7f9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:00.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 --2- 172.21.15.46:0/1973857866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e7faa00 0x55fc0e7f9c30 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f1b7001c980 tx=0x7f1b700a6000 comp rx=0 tx=0).stop 2024-03-29T08:27:00.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 -- 172.21.15.46:0/1973857866 shutdown_connections 2024-03-29T08:27:00.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 --2- 172.21.15.46:0/1973857866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e846a70 0x55fc0e7fa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 --2- 172.21.15.46:0/1973857866 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e847390 0x55fc0e7fade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 --2- 172.21.15.46:0/1973857866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e7faa00 0x55fc0e7f9c30 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 -- 172.21.15.46:0/1973857866 >> 172.21.15.46:0/1973857866 conn(0x55fc0e5fb160 msgr2=0x55fc0e802840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:00.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 -- 172.21.15.46:0/1973857866 shutdown_connections 2024-03-29T08:27:00.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 -- 172.21.15.46:0/1973857866 wait complete. 2024-03-29T08:27:00.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.771+0000 7f1b89feb000 1 -- start start 2024-03-29T08:27:00.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 Processor -- start 2024-03-29T08:27:00.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e7faa00 0x55fc0e8a6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e846a70 0x55fc0e8a6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e847390 0x55fc0e7fdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc0e64e5f0 con 0x55fc0e846a70 2024-03-29T08:27:00.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc0e82b0e0 con 0x55fc0e7faa00 2024-03-29T08:27:00.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc0e615ee0 con 0x55fc0e847390 2024-03-29T08:27:00.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 10 client.0 _ll_register_callbacks cb 0x7f1b680518a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:27:00.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b77400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e846a70 0x55fc0e8a6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b77400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e846a70 0x55fc0e8a6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41128/0 (socket says 172.21.15.46:41128) 2024-03-29T08:27:00.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b77400640 1 -- 172.21.15.46:0/3305653279 learned_addr learned my addr 172.21.15.46:0/3305653279 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:00.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/3305653279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e7faa00 0x55fc0e8a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b84c00640 1 --2- 172.21.15.46:0/3305653279 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e847390 0x55fc0e7fdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b77400640 1 -- 172.21.15.46:0/3305653279 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e847390 msgr2=0x55fc0e7fdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:00.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b77400640 1 --2- 172.21.15.46:0/3305653279 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e847390 0x55fc0e7fdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b77400640 1 -- 172.21.15.46:0/3305653279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e7faa00 msgr2=0x55fc0e8a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:00.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b77400640 1 --2- 172.21.15.46:0/3305653279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e7faa00 0x55fc0e8a6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b77400640 1 -- 172.21.15.46:0/3305653279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b700aba60 con 0x55fc0e846a70 2024-03-29T08:27:00.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b77400640 1 --2- 172.21.15.46:0/3305653279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e846a70 0x55fc0e8a6d40 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f1b680519e0 tx=0x7f1b68051950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1435b4c1532faa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:00.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b6d600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:27:00.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b6d600640 1 -- 172.21.15.46:0/3305653279 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68070070 con 0x55fc0e846a70 2024-03-29T08:27:00.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b6d600640 1 -- 172.21.15.46:0/3305653279 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f1b68074e40 con 0x55fc0e846a70 2024-03-29T08:27:00.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 -- 172.21.15.46:0/3305653279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc0e67aa20 con 0x55fc0e846a70 2024-03-29T08:27:00.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b6d600640 1 -- 172.21.15.46:0/3305653279 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b68055020 con 0x55fc0e846a70 2024-03-29T08:27:00.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 10 client.8133 fetch_fsmap learned FSMap version 69 2024-03-29T08:27:00.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 -- 172.21.15.46:0/3305653279 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55fc0e610aa0 con 0x55fc0e846a70 2024-03-29T08:27:00.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b6d600640 1 -- 172.21.15.46:0/3305653279 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b68049020 con 0x55fc0e846a70 2024-03-29T08:27:00.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b6d600640 1 -- 172.21.15.46:0/3305653279 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f1b68011040 con 0x55fc0e846a70 2024-03-29T08:27:00.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 10 client.8133 fetch_fsmap finished waiting for FSMap version 69 2024-03-29T08:27:00.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 10 client.8133 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-03-29T08:27:00.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 20 client.8133 populate_metadata read hostname 'smithi046' 2024-03-29T08:27:00.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 --2- 172.21.15.46:0/3305653279 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x55fc0e7fe0e0 0x55fc0e7fe4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 4 client.8133 mds_command: new command op to 7986 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:27:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 -- 172.21.15.46:0/3305653279 --> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fc0e644550 con 0x55fc0e7fe0e0 2024-03-29T08:27:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/3305653279 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x55fc0e7fe0e0 0x55fc0e7fe4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b7fe00640 1 --2- 172.21.15.46:0/3305653279 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x55fc0e7fe0e0 0x55fc0e7fe4c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a15db14816cfc986 server_cookie=5f3e75e27e626344 in_seq=0 out_seq=0 2024-03-29T08:27:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b6d600640 10 client.8133 ms_handle_connect on v2:172.21.15.46:6833/1886680593 2024-03-29T08:27:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b6d600640 1 -- 172.21.15.46:0/3305653279 <== mds.0 v2:172.21.15.46:6833/1886680593 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fc0e644550 con 0x55fc0e7fe0e0 2024-03-29T08:27:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b6d600640 10 client.8133 handle_command_reply: tid=0 2024-03-29T08:27:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 client.8133 shutdown 2024-03-29T08:27:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 -- 172.21.15.46:0/3305653279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e846a70 msgr2=0x55fc0e8a6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 --2- 172.21.15.46:0/3305653279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e846a70 0x55fc0e8a6d40 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f1b680519e0 tx=0x7f1b68051950 comp rx=0 tx=0).stop 2024-03-29T08:27:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 -- 172.21.15.46:0/3305653279 shutdown_connections 2024-03-29T08:27:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 --2- 172.21.15.46:0/3305653279 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x55fc0e7fe0e0 0x55fc0e7fe4c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 --2- 172.21.15.46:0/3305653279 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc0e847390 0x55fc0e7fdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 --2- 172.21.15.46:0/3305653279 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc0e846a70 0x55fc0e8a6d40 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b6d600640 0 client.8133 ms_handle_reset on v2:172.21.15.46:6833/1886680593 2024-03-29T08:27:00.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 --2- 172.21.15.46:0/3305653279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc0e7faa00 0x55fc0e8a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 -- 172.21.15.46:0/3305653279 >> 172.21.15.46:0/3305653279 conn(0x55fc0e5fb160 msgr2=0x55fc0e7f8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:00.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 -- 172.21.15.46:0/3305653279 shutdown_connections 2024-03-29T08:27:00.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.775+0000 7f1b89feb000 1 -- 172.21.15.46:0/3305653279 wait complete. 2024-03-29T08:27:00.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.779+0000 7f1b89feb000 20 client.8133 trim_cache size 0 max 16384 2024-03-29T08:27:00.788 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:27:00.788 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-29T08:27:00.788 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-29T08:27:00.788 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:27:00.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.779+0000 7f1b86c00640 1 -- 172.21.15.46:0/602291054 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b5c066690 msgr2=0x7f1b5c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:00.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.779+0000 7f1b86c00640 1 --2- 172.21.15.46:0/602291054 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b5c066690 0x7f1b5c068b50 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1b700ad000 tx=0x7f1b70008040 comp rx=0 tx=0).stop 2024-03-29T08:27:00.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.779+0000 7f1b86c00640 1 -- 172.21.15.46:0/602291054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b80143eb0 msgr2=0x7f1b8013f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:00.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.779+0000 7f1b86c00640 1 --2- 172.21.15.46:0/602291054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b80143eb0 0x7f1b8013f760 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f1b6800e930 tx=0x7f1b6800edf0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.779+0000 7f1b86c00640 1 -- 172.21.15.46:0/602291054 shutdown_connections 2024-03-29T08:27:00.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.779+0000 7f1b86c00640 1 --2- 172.21.15.46:0/602291054 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b5c066690 0x7f1b5c068b50 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.779+0000 7f1b86c00640 1 --2- 172.21.15.46:0/602291054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b8015efe0 0x7f1b800c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.779+0000 7f1b86c00640 1 --2- 172.21.15.46:0/602291054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b80143eb0 0x7f1b8013f760 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.779+0000 7f1b86c00640 1 --2- 172.21.15.46:0/602291054 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b800b1630 0x7f1b801f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.779+0000 7f1b86c00640 1 -- 172.21.15.46:0/602291054 >> 172.21.15.46:0/602291054 conn(0x7f1b8006a390 msgr2=0x7f1b800cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:00.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.779+0000 7f1b86c00640 1 -- 172.21.15.46:0/602291054 shutdown_connections 2024-03-29T08:27:00.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.779+0000 7f1b86c00640 1 -- 172.21.15.46:0/602291054 wait complete. 2024-03-29T08:27:00.799 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-29T08:27:00.799 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:27:00.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.935+0000 7fd3f5000640 1 Processor -- start 2024-03-29T08:27:00.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.935+0000 7fd3f5000640 1 -- start start 2024-03-29T08:27:00.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.939+0000 7fd3f5000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3f00ceb20 0x7fd3f01656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.939+0000 7fd3f5000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f00cb510 con 0x7fd3f015ada0 2024-03-29T08:27:00.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.939+0000 7fd3f5000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f0070580 con 0x7fd3f0144210 2024-03-29T08:27:00.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.939+0000 7fd3f5000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f006d960 con 0x7fd3f00ceb20 2024-03-29T08:27:00.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.939+0000 7fd3e7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd3f015ada0 0x7fd3f00ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37858/0 (socket says 172.21.15.46:37858) 2024-03-29T08:27:00.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.939+0000 7fd3e7e00640 1 -- 172.21.15.46:0/1961428509 learned_addr learned my addr 172.21.15.46:0/1961428509 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:00.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.939+0000 7fd3ee800640 1 --2- 172.21.15.46:0/1961428509 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3f00ceb20 0x7fd3f01656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.939+0000 7fd3ed400640 1 -- 172.21.15.46:0/1961428509 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579431013 0 0) 0x7fd3f0070580 con 0x7fd3f0144210 2024-03-29T08:27:00.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.939+0000 7fd3ed400640 1 -- 172.21.15.46:0/1961428509 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3d4003530 con 0x7fd3f0144210 2024-03-29T08:27:00.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.939+0000 7fd3ed400640 1 -- 172.21.15.46:0/1961428509 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2121424318 0 0) 0x7fd3f00cb510 con 0x7fd3f015ada0 2024-03-29T08:27:00.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.939+0000 7fd3ed400640 1 -- 172.21.15.46:0/1961428509 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3f0070580 con 0x7fd3f015ada0 2024-03-29T08:27:00.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.939+0000 7fd3ee800640 1 -- 172.21.15.46:0/1961428509 >> v1:172.21.15.73:6789/0 conn(0x7fd3f0144210 legacy=0x7fd3f00cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:00.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.939+0000 7fd3ee800640 1 -- 172.21.15.46:0/1961428509 >> v1:172.21.15.46:6789/0 conn(0x7fd3f015ada0 legacy=0x7fd3f00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:00.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.939+0000 7fd3ee800640 1 -- 172.21.15.46:0/1961428509 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f0070860 con 0x7fd3f00ceb20 2024-03-29T08:27:00.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.943+0000 7fd3ee800640 1 --2- 172.21.15.46:0/1961428509 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3f00ceb20 0x7fd3f01656d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009dc0 tx=0x7fd3e801c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2de2909f337fdd53 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:00.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.943+0000 7fd3ed400640 1 -- 172.21.15.46:0/1961428509 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80a5940 con 0x7fd3f00ceb20 2024-03-29T08:27:00.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.943+0000 7fd3ed400640 1 -- 172.21.15.46:0/1961428509 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e80a5aa0 con 0x7fd3f00ceb20 2024-03-29T08:27:00.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.943+0000 7fd3ed400640 1 -- 172.21.15.46:0/1961428509 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80a5e10 con 0x7fd3f00ceb20 2024-03-29T08:27:00.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.943+0000 7fd3f5000640 1 -- 172.21.15.46:0/1961428509 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3f00ceb20 msgr2=0x7fd3f01656d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:00.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.943+0000 7fd3f5000640 1 --2- 172.21.15.46:0/1961428509 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3f00ceb20 0x7fd3f01656d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8009dc0 tx=0x7fd3e801c990 comp rx=0 tx=0).stop 2024-03-29T08:27:00.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.943+0000 7fd3f5000640 1 -- 172.21.15.46:0/1961428509 shutdown_connections 2024-03-29T08:27:00.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.943+0000 7fd3f5000640 1 --2- 172.21.15.46:0/1961428509 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3f00ceb20 0x7fd3f01656d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.943+0000 7fd3f5000640 1 -- 172.21.15.46:0/1961428509 >> 172.21.15.46:0/1961428509 conn(0x7fd3f0066110 msgr2=0x7fd3f0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:00.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.943+0000 7fd3f5000640 1 -- 172.21.15.46:0/1961428509 shutdown_connections 2024-03-29T08:27:00.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.943+0000 7fd3f5000640 1 -- 172.21.15.46:0/1961428509 wait complete. 2024-03-29T08:27:00.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.943+0000 7fd3f5000640 1 Processor -- start 2024-03-29T08:27:00.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.943+0000 7fd3f5000640 1 -- start start 2024-03-29T08:27:00.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3f5000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3f00ceb20 0x7fd3f0132d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3f5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3f0144210 0x7fd3f0159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3f5000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3f015ada0 0x7fd3f0130190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3f5000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f0130f00 con 0x7fd3f0144210 2024-03-29T08:27:00.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3f5000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f0130bc0 con 0x7fd3f00ceb20 2024-03-29T08:27:00.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3f5000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f0133240 con 0x7fd3f015ada0 2024-03-29T08:27:00.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3f0144210 0x7fd3f0159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3f0144210 0x7fd3f0159770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41142/0 (socket says 172.21.15.46:41142) 2024-03-29T08:27:00.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3e7e00640 1 -- 172.21.15.46:0/4234487069 learned_addr learned my addr 172.21.15.46:0/4234487069 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:00.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3ede00640 1 --2- 172.21.15.46:0/4234487069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3f00ceb20 0x7fd3f0132d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3ee800640 1 --2- 172.21.15.46:0/4234487069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3f015ada0 0x7fd3f0130190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3e7e00640 1 -- 172.21.15.46:0/4234487069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3f015ada0 msgr2=0x7fd3f0130190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3e7e00640 1 --2- 172.21.15.46:0/4234487069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3f015ada0 0x7fd3f0130190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3e7e00640 1 -- 172.21.15.46:0/4234487069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3f00ceb20 msgr2=0x7fd3f0132d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3e7e00640 1 --2- 172.21.15.46:0/4234487069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3f00ceb20 0x7fd3f0132d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3e7e00640 1 -- 172.21.15.46:0/4234487069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e8009a70 con 0x7fd3f0144210 2024-03-29T08:27:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3e7e00640 1 --2- 172.21.15.46:0/4234487069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3f0144210 0x7fd3f0159770 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800c410 tx=0x7fd3d800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3e6000640 1 -- 172.21.15.46:0/4234487069 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8005cf0 con 0x7fd3f0144210 2024-03-29T08:27:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3e6000640 1 -- 172.21.15.46:0/4234487069 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d8005e50 con 0x7fd3f0144210 2024-03-29T08:27:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3f5000640 1 -- 172.21.15.46:0/4234487069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f01310e0 con 0x7fd3f0144210 2024-03-29T08:27:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3e6000640 1 -- 172.21.15.46:0/4234487069 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8007310 con 0x7fd3f0144210 2024-03-29T08:27:00.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3f5000640 1 -- 172.21.15.46:0/4234487069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f01316a0 con 0x7fd3f0144210 2024-03-29T08:27:00.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3f5000640 1 -- 172.21.15.46:0/4234487069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f00ce280 con 0x7fd3f0144210 2024-03-29T08:27:00.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.947+0000 7fd3e6000640 1 -- 172.21.15.46:0/4234487069 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd3d8007470 con 0x7fd3f0144210 2024-03-29T08:27:00.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.951+0000 7fd3e6000640 1 --2- 172.21.15.46:0/4234487069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3cc066690 0x7fd3cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:00.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.951+0000 7fd3e6000640 1 -- 172.21.15.46:0/4234487069 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fd3d8083890 con 0x7fd3f0144210 2024-03-29T08:27:00.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.951+0000 7fd3ede00640 1 --2- 172.21.15.46:0/4234487069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3cc066690 0x7fd3cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:00.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.951+0000 7fd3ede00640 1 --2- 172.21.15.46:0/4234487069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3cc066690 0x7fd3cc068b50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0007fd0 tx=0x7fd3e0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:00.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:00.955+0000 7fd3e6000640 1 -- 172.21.15.46:0/4234487069 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd3d8050b40 con 0x7fd3f0144210 2024-03-29T08:27:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.147+0000 7fd3f5000640 1 -- 172.21.15.46:0/4234487069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3f0066c90 con 0x7fd3f0144210 2024-03-29T08:27:01.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.147+0000 7fd3e6000640 1 -- 172.21.15.46:0/4234487069 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4737 (secure 0 0 0) 0x7fd3d8017090 con 0x7fd3f0144210 2024-03-29T08:27:01.152 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:01.152 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/1862652020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1862652020},{"type":"v1","addr":"172.21.15.73:6836","nonce":1862652020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:24:43.476334+0000","modified":"2024-03-29T08:26:26.873752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7986":{"gid":7986,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1886680593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1886680593},{"type":"v1","addr":"172.21.15.46:6835","nonce":1886680593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-29T08:27:01.154 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 69 2024-03-29T08:27:01.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.151+0000 7fd3f5000640 1 -- 172.21.15.46:0/4234487069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3cc066690 msgr2=0x7fd3cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.151+0000 7fd3f5000640 1 --2- 172.21.15.46:0/4234487069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3cc066690 0x7fd3cc068b50 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd3e0007fd0 tx=0x7fd3e0009040 comp rx=0 tx=0).stop 2024-03-29T08:27:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.151+0000 7fd3f5000640 1 -- 172.21.15.46:0/4234487069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3f0144210 msgr2=0x7fd3f0159770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.151+0000 7fd3f5000640 1 --2- 172.21.15.46:0/4234487069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3f0144210 0x7fd3f0159770 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800c410 tx=0x7fd3d800c8d0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.151+0000 7fd3f5000640 1 -- 172.21.15.46:0/4234487069 shutdown_connections 2024-03-29T08:27:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.151+0000 7fd3f5000640 1 --2- 172.21.15.46:0/4234487069 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3cc066690 0x7fd3cc068b50 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.151+0000 7fd3f5000640 1 --2- 172.21.15.46:0/4234487069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3f015ada0 0x7fd3f0130190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.151+0000 7fd3f5000640 1 --2- 172.21.15.46:0/4234487069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3f0144210 0x7fd3f0159770 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.151+0000 7fd3f5000640 1 --2- 172.21.15.46:0/4234487069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3f00ceb20 0x7fd3f0132d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.151+0000 7fd3f5000640 1 -- 172.21.15.46:0/4234487069 >> 172.21.15.46:0/4234487069 conn(0x7fd3f0066110 msgr2=0x7fd3f01386d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:01.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.151+0000 7fd3f5000640 1 -- 172.21.15.46:0/4234487069 shutdown_connections 2024-03-29T08:27:01.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.151+0000 7fd3f5000640 1 -- 172.21.15.46:0/4234487069 wait complete. 2024-03-29T08:27:01.172 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-03-29T08:27:01.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.311+0000 7fe5a3200640 1 Processor -- start 2024-03-29T08:27:01.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.311+0000 7fe5a3200640 1 -- start start 2024-03-29T08:27:01.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.315+0000 7fe5a3200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe59c099150 0x7fe59c0cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.315+0000 7fe5a3200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59c0cbd00 con 0x7fe59c15abd0 2024-03-29T08:27:01.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.315+0000 7fe5a3200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59c0cbfe0 con 0x7fe59c15a7f0 2024-03-29T08:27:01.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.315+0000 7fe5a3200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe59c0b4500 con 0x7fe59c099150 2024-03-29T08:27:01.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.315+0000 7fe5a1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe59c099150 0x7fe59c0cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.315+0000 7fe5a1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe59c099150 0x7fe59c0cf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41152/0 (socket says 172.21.15.46:41152) 2024-03-29T08:27:01.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.315+0000 7fe5a1200640 1 -- 172.21.15.46:0/3926096505 learned_addr learned my addr 172.21.15.46:0/3926096505 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:01.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.315+0000 7fe5a1200640 1 -- 172.21.15.46:0/3926096505 >> v1:172.21.15.73:6789/0 conn(0x7fe59c15a7f0 legacy=0x7fe59c0cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.315+0000 7fe5a1200640 1 -- 172.21.15.46:0/3926096505 >> v1:172.21.15.46:6789/0 conn(0x7fe59c15abd0 legacy=0x7fe59c098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.315+0000 7fe5a1200640 1 -- 172.21.15.46:0/3926096505 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c0cc2c0 con 0x7fe59c099150 2024-03-29T08:27:01.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe5a1200640 1 --2- 172.21.15.46:0/3926096505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe59c099150 0x7fe59c0cf430 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe594009870 tx=0x7fe59401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31c7986b8707b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:01.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe59b400640 1 -- 172.21.15.46:0/3926096505 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5940ac620 con 0x7fe59c099150 2024-03-29T08:27:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe59b400640 1 -- 172.21.15.46:0/3926096505 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5940ac780 con 0x7fe59c099150 2024-03-29T08:27:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe59b400640 1 -- 172.21.15.46:0/3926096505 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5940acb10 con 0x7fe59c099150 2024-03-29T08:27:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe5a3200640 1 -- 172.21.15.46:0/3926096505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe59c099150 msgr2=0x7fe59c0cf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe5a3200640 1 --2- 172.21.15.46:0/3926096505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe59c099150 0x7fe59c0cf430 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fe594009870 tx=0x7fe59401c4d0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe5a3200640 1 -- 172.21.15.46:0/3926096505 shutdown_connections 2024-03-29T08:27:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe5a3200640 1 --2- 172.21.15.46:0/3926096505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe59c099150 0x7fe59c0cf430 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe5a3200640 1 -- 172.21.15.46:0/3926096505 >> 172.21.15.46:0/3926096505 conn(0x7fe59c0b2190 msgr2=0x7fe59c0b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe5a3200640 1 -- 172.21.15.46:0/3926096505 shutdown_connections 2024-03-29T08:27:01.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe5a3200640 1 -- 172.21.15.46:0/3926096505 wait complete. 2024-03-29T08:27:01.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe5a3200640 1 Processor -- start 2024-03-29T08:27:01.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe5a3200640 1 -- start start 2024-03-29T08:27:01.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe5a3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59c099150 0x7fe59c155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe5a3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe59c15a7f0 0x7fe59c14fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe5a3200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe59c15abd0 0x7fe59c15c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe5a3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c155750 con 0x7fe59c15a7f0 2024-03-29T08:27:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe5a3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c154d80 con 0x7fe59c099150 2024-03-29T08:27:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe5a3200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c150410 con 0x7fe59c15abd0 2024-03-29T08:27:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe593400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe59c15a7f0 0x7fe59c14fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe593400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe59c15a7f0 0x7fe59c14fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41158/0 (socket says 172.21.15.46:41158) 2024-03-29T08:27:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe593400640 1 -- 172.21.15.46:0/730319745 learned_addr learned my addr 172.21.15.46:0/730319745 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:01.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe59be00640 1 --2- 172.21.15.46:0/730319745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59c099150 0x7fe59c155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.319+0000 7fe5a1200640 1 --2- 172.21.15.46:0/730319745 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe59c15abd0 0x7fe59c15c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.323+0000 7fe593400640 1 -- 172.21.15.46:0/730319745 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe59c15abd0 msgr2=0x7fe59c15c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.323+0000 7fe593400640 1 --2- 172.21.15.46:0/730319745 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe59c15abd0 0x7fe59c15c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.323+0000 7fe593400640 1 -- 172.21.15.46:0/730319745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59c099150 msgr2=0x7fe59c155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.323+0000 7fe593400640 1 --2- 172.21.15.46:0/730319745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59c099150 0x7fe59c155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.323+0000 7fe593400640 1 -- 172.21.15.46:0/730319745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5940094d0 con 0x7fe59c15a7f0 2024-03-29T08:27:01.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.323+0000 7fe593400640 1 --2- 172.21.15.46:0/730319745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe59c15a7f0 0x7fe59c14fed0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe58400c3c0 tx=0x7fe58400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:01.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.323+0000 7fe598c00640 1 -- 172.21.15.46:0/730319745 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5840052d0 con 0x7fe59c15a7f0 2024-03-29T08:27:01.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.323+0000 7fe598c00640 1 -- 172.21.15.46:0/730319745 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe584005430 con 0x7fe59c15a7f0 2024-03-29T08:27:01.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.323+0000 7fe5a3200640 1 -- 172.21.15.46:0/730319745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59c15ce50 con 0x7fe59c15a7f0 2024-03-29T08:27:01.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.323+0000 7fe598c00640 1 -- 172.21.15.46:0/730319745 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584003840 con 0x7fe59c15a7f0 2024-03-29T08:27:01.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.323+0000 7fe5a3200640 1 -- 172.21.15.46:0/730319745 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59c154a10 con 0x7fe59c15a7f0 2024-03-29T08:27:01.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.323+0000 7fe598c00640 1 -- 172.21.15.46:0/730319745 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe5840039a0 con 0x7fe59c15a7f0 2024-03-29T08:27:01.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.323+0000 7fe598c00640 1 --2- 172.21.15.46:0/730319745 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe5780666c0 0x7fe578068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.323+0000 7fe598c00640 1 -- 172.21.15.46:0/730319745 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fe584083730 con 0x7fe59c15a7f0 2024-03-29T08:27:01.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.323+0000 7fe59be00640 1 --2- 172.21.15.46:0/730319745 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe5780666c0 0x7fe578068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 Processor -- start 2024-03-29T08:27:01.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 -- start start 2024-03-29T08:27:01.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe59be00640 1 --2- 172.21.15.46:0/730319745 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe5780666c0 0x7fe578068b80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe59c066760 tx=0x7fe58c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:01.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd2137670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd2137bb0 0x560fd20edc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd20ee1b0 0x560fd20ee590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd1fb6340 con 0x560fd2137290 2024-03-29T08:27:01.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd1f42cd0 con 0x560fd20ee1b0 2024-03-29T08:27:01.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560fd1f402a0 con 0x560fd2137bb0 2024-03-29T08:27:01.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe59be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd2137670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe59be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd2137670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41168/0 (socket says 172.21.15.46:41168) 2024-03-29T08:27:01.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe59be00640 1 -- 172.21.15.46:0/3325344067 learned_addr learned my addr 172.21.15.46:0/3325344067 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:01.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe593400640 1 --2- 172.21.15.46:0/3325344067 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd2137bb0 0x560fd20edc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a1200640 1 --2- 172.21.15.46:0/3325344067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd20ee1b0 0x560fd20ee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe59be00640 1 -- 172.21.15.46:0/3325344067 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd2137bb0 msgr2=0x560fd20edc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:01.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe59be00640 1 --2- 172.21.15.46:0/3325344067 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd2137bb0 0x560fd20edc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe59be00640 1 -- 172.21.15.46:0/3325344067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd20ee1b0 msgr2=0x560fd20ee590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:01.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe59be00640 1 --2- 172.21.15.46:0/3325344067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd20ee1b0 0x560fd20ee590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe59be00640 1 -- 172.21.15.46:0/3325344067 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fd1c2e4b0 con 0x560fd2137290 2024-03-29T08:27:01.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe593400640 1 --2- 172.21.15.46:0/3325344067 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd2137bb0 0x560fd20edc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:01.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a1200640 1 --2- 172.21.15.46:0/3325344067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd20ee1b0 0x560fd20ee590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:01.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe59be00640 1 --2- 172.21.15.46:0/3325344067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd2137670 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fe58c002200 tx=0x7fe58c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7950d4656e6859 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:01.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe590c00640 1 -- 172.21.15.46:0/3325344067 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c00fec0 con 0x560fd2137290 2024-03-29T08:27:01.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe590c00640 1 -- 172.21.15.46:0/3325344067 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58c001be0 con 0x560fd2137290 2024-03-29T08:27:01.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe590c00640 1 -- 172.21.15.46:0/3325344067 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c020610 con 0x560fd2137290 2024-03-29T08:27:01.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 -- 172.21.15.46:0/3325344067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 msgr2=0x560fd2137670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 --2- 172.21.15.46:0/3325344067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd2137670 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fe58c002200 tx=0x7fe58c012000 comp rx=0 tx=0).stop 2024-03-29T08:27:01.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 -- 172.21.15.46:0/3325344067 shutdown_connections 2024-03-29T08:27:01.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 --2- 172.21.15.46:0/3325344067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd20ee1b0 0x560fd20ee590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 --2- 172.21.15.46:0/3325344067 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd2137bb0 0x560fd20edc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 --2- 172.21.15.46:0/3325344067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd2137670 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 -- 172.21.15.46:0/3325344067 >> 172.21.15.46:0/3325344067 conn(0x560fd20ea160 msgr2=0x560fd20ec550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 -- 172.21.15.46:0/3325344067 shutdown_connections 2024-03-29T08:27:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 -- 172.21.15.46:0/3325344067 wait complete. 2024-03-29T08:27:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 -- start start 2024-03-29T08:27:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.327+0000 7fe5a6722000 1 Processor -- start 2024-03-29T08:27:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a6722000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd20ee1b0 0x560fd20f3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a6722000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd20f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a6722000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd2137bb0 0x560fd20f8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a6722000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd1f3bb90 con 0x560fd2137290 2024-03-29T08:27:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a6722000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd1ef90a0 con 0x560fd20ee1b0 2024-03-29T08:27:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a6722000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560fd1ef8d40 con 0x560fd2137bb0 2024-03-29T08:27:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a6722000 10 client.0 _ll_register_callbacks cb 0x560fd1ff0c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:27:01.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe593400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd20f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe593400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd20f3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41182/0 (socket says 172.21.15.46:41182) 2024-03-29T08:27:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe593400640 1 -- 172.21.15.46:0/1858947552 learned_addr learned my addr 172.21.15.46:0/1858947552 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe59be00640 1 --2- 172.21.15.46:0/1858947552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd20ee1b0 0x560fd20f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a1200640 1 --2- 172.21.15.46:0/1858947552 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd2137bb0 0x560fd20f8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe593400640 1 -- 172.21.15.46:0/1858947552 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd2137bb0 msgr2=0x560fd20f8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe593400640 1 --2- 172.21.15.46:0/1858947552 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd2137bb0 0x560fd20f8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe593400640 1 -- 172.21.15.46:0/1858947552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd20ee1b0 msgr2=0x560fd20f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe593400640 1 --2- 172.21.15.46:0/1858947552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd20ee1b0 0x560fd20f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe593400640 1 -- 172.21.15.46:0/1858947552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59c0cbfe0 con 0x560fd2137290 2024-03-29T08:27:01.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a1200640 1 --2- 172.21.15.46:0/1858947552 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd2137bb0 0x560fd20f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:01.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe593400640 1 --2- 172.21.15.46:0/1858947552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd20f3bc0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fe584081ab0 tx=0x7fe584054780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb53906a69121238 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:01.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe589600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:27:01.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe589600640 1 -- 172.21.15.46:0/1858947552 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584050dd0 con 0x560fd2137290 2024-03-29T08:27:01.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe589600640 1 -- 172.21.15.46:0/1858947552 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fe584050f30 con 0x560fd2137290 2024-03-29T08:27:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a6722000 1 -- 172.21.15.46:0/1858947552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fd1f0b550 con 0x560fd2137290 2024-03-29T08:27:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe589600640 1 -- 172.21.15.46:0/1858947552 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe58405c570 con 0x560fd2137290 2024-03-29T08:27:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a6722000 10 client.8154 fetch_fsmap learned FSMap version 69 2024-03-29T08:27:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a6722000 1 -- 172.21.15.46:0/1858947552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x560fd1f00ba0 con 0x560fd2137290 2024-03-29T08:27:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe589600640 1 -- 172.21.15.46:0/1858947552 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560fd1f0b550 con 0x560fd2137290 2024-03-29T08:27:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe589600640 1 -- 172.21.15.46:0/1858947552 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe58405a3e0 con 0x560fd2137290 2024-03-29T08:27:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a6722000 10 client.8154 fetch_fsmap finished waiting for FSMap version 69 2024-03-29T08:27:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a6722000 10 client.8154 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:27:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a6722000 20 client.8154 populate_metadata read hostname 'smithi046' 2024-03-29T08:27:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a6722000 1 --2- 172.21.15.46:0/1858947552 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x560fd2146320 0x560fd2148710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a6722000 4 client.8154 mds_command: new command op to 7986 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:27:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe5a6722000 1 -- 172.21.15.46:0/1858947552 --> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560fd1f38970 con 0x560fd2146320 2024-03-29T08:27:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe59be00640 1 --2- 172.21.15.46:0/1858947552 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x560fd2146320 0x560fd2148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe59be00640 1 --2- 172.21.15.46:0/1858947552 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x560fd2146320 0x560fd2148710 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b91daf5d3354549a server_cookie=f17678feec2d4087 in_seq=0 out_seq=0 2024-03-29T08:27:01.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.331+0000 7fe589600640 10 client.8154 ms_handle_connect on v2:172.21.15.46:6833/1886680593 2024-03-29T08:27:01.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.335+0000 7fe589600640 1 -- 172.21.15.46:0/1858947552 <== mds.0 v2:172.21.15.46:6833/1886680593 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x560fd1f38970 con 0x560fd2146320 2024-03-29T08:27:01.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.335+0000 7fe589600640 10 client.8154 handle_command_reply: tid=0 2024-03-29T08:27:01.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.335+0000 7fe5a6722000 1 client.8154 shutdown 2024-03-29T08:27:01.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.335+0000 7fe5a6722000 1 -- 172.21.15.46:0/1858947552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 msgr2=0x560fd20f3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.335+0000 7fe5a6722000 1 --2- 172.21.15.46:0/1858947552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd20f3bc0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fe584081ab0 tx=0x7fe584054780 comp rx=0 tx=0).stop 2024-03-29T08:27:01.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.335+0000 7fe5a6722000 1 -- 172.21.15.46:0/1858947552 shutdown_connections 2024-03-29T08:27:01.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.335+0000 7fe5a6722000 1 --2- 172.21.15.46:0/1858947552 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x560fd2146320 0x560fd2148710 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.335+0000 7fe5a6722000 1 --2- 172.21.15.46:0/1858947552 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd2137bb0 0x560fd20f8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.335+0000 7fe5a6722000 1 --2- 172.21.15.46:0/1858947552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd20f3bc0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.335+0000 7fe5a6722000 1 --2- 172.21.15.46:0/1858947552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd20ee1b0 0x560fd20f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.335+0000 7fe589600640 0 client.8154 ms_handle_reset on v2:172.21.15.46:6833/1886680593 2024-03-29T08:27:01.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.335+0000 7fe5a6722000 1 -- 172.21.15.46:0/1858947552 >> 172.21.15.46:0/1858947552 conn(0x560fd20ea160 msgr2=0x560fd20ecde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:01.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.335+0000 7fe5a6722000 1 -- 172.21.15.46:0/1858947552 shutdown_connections 2024-03-29T08:27:01.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.335+0000 7fe5a6722000 1 -- 172.21.15.46:0/1858947552 wait complete. 2024-03-29T08:27:01.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.335+0000 7fe5a6722000 20 client.8154 trim_cache size 0 max 16384 2024-03-29T08:27:01.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 Processor -- start 2024-03-29T08:27:01.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 -- start start 2024-03-29T08:27:01.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd20ea160 0x560fd20ed560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd20ee1b0 0x560fd20edaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd2137290 0x560fd2146700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd1bf8c40 con 0x560fd20ee1b0 2024-03-29T08:27:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd1f06ad0 con 0x560fd2137290 2024-03-29T08:27:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560fd1f3fb90 con 0x560fd20ea160 2024-03-29T08:27:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe593400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd20ee1b0 0x560fd20edaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe593400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd20ee1b0 0x560fd20edaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41198/0 (socket says 172.21.15.46:41198) 2024-03-29T08:27:01.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe593400640 1 -- 172.21.15.46:0/3518756012 learned_addr learned my addr 172.21.15.46:0/3518756012 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe593400640 1 -- 172.21.15.46:0/3518756012 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd20ea160 msgr2=0x560fd20ed560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe593400640 1 --2- 172.21.15.46:0/3518756012 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd20ea160 0x560fd20ed560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe593400640 1 -- 172.21.15.46:0/3518756012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd2137290 msgr2=0x560fd2146700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe593400640 1 --2- 172.21.15.46:0/3518756012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd2137290 0x560fd2146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe593400640 1 -- 172.21.15.46:0/3518756012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fd1ee6ef0 con 0x560fd20ee1b0 2024-03-29T08:27:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe593400640 1 --2- 172.21.15.46:0/3518756012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd20ee1b0 0x560fd20edaa0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe584054650 tx=0x7fe58401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1beff495dd85d3ab server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe591600640 1 -- 172.21.15.46:0/3518756012 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58400e040 con 0x560fd20ee1b0 2024-03-29T08:27:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe591600640 1 -- 172.21.15.46:0/3518756012 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58406c020 con 0x560fd20ee1b0 2024-03-29T08:27:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe591600640 1 -- 172.21.15.46:0/3518756012 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584054200 con 0x560fd20ee1b0 2024-03-29T08:27:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 -- 172.21.15.46:0/3518756012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd20ee1b0 msgr2=0x560fd20edaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 --2- 172.21.15.46:0/3518756012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd20ee1b0 0x560fd20edaa0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fe584054650 tx=0x7fe58401e000 comp rx=0 tx=0).stop 2024-03-29T08:27:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 -- 172.21.15.46:0/3518756012 shutdown_connections 2024-03-29T08:27:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 --2- 172.21.15.46:0/3518756012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd2137290 0x560fd2146700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 --2- 172.21.15.46:0/3518756012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd20ee1b0 0x560fd20edaa0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 --2- 172.21.15.46:0/3518756012 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd20ea160 0x560fd20ed560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 -- 172.21.15.46:0/3518756012 >> 172.21.15.46:0/3518756012 conn(0x560fd21367b0 msgr2=0x560fd20f3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 -- 172.21.15.46:0/3518756012 shutdown_connections 2024-03-29T08:27:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 -- 172.21.15.46:0/3518756012 wait complete. 2024-03-29T08:27:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 -- start start 2024-03-29T08:27:01.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.347+0000 7fe5a6722000 1 Processor -- start 2024-03-29T08:27:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd20ea160 0x560fd2149940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd20ee1b0 0x560fd21916f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd2191c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd1f43f60 con 0x560fd2137290 2024-03-29T08:27:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fd20e55e0 con 0x560fd20ee1b0 2024-03-29T08:27:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560fd1f393a0 con 0x560fd20ea160 2024-03-29T08:27:01.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 10 client.0 _ll_register_callbacks cb 0x560fd1fd9870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:27:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd2191c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd2191c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41208/0 (socket says 172.21.15.46:41208) 2024-03-29T08:27:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a1200640 1 -- 172.21.15.46:0/275151997 learned_addr learned my addr 172.21.15.46:0/275151997 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe593400640 1 --2- 172.21.15.46:0/275151997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd20ee1b0 0x560fd21916f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe59be00640 1 --2- 172.21.15.46:0/275151997 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd20ea160 0x560fd2149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a1200640 1 -- 172.21.15.46:0/275151997 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd20ea160 msgr2=0x560fd2149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a1200640 1 --2- 172.21.15.46:0/275151997 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd20ea160 0x560fd2149940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a1200640 1 -- 172.21.15.46:0/275151997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd20ee1b0 msgr2=0x560fd21916f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a1200640 1 --2- 172.21.15.46:0/275151997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd20ee1b0 0x560fd21916f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a1200640 1 -- 172.21.15.46:0/275151997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560fd1f00ba0 con 0x560fd2137290 2024-03-29T08:27:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe59be00640 1 --2- 172.21.15.46:0/275151997 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd20ea160 0x560fd2149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe593400640 1 --2- 172.21.15.46:0/275151997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd20ee1b0 0x560fd21916f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a1200640 1 --2- 172.21.15.46:0/275151997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd2191c30 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe5940190e0 tx=0x7fe594019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d42466f5f4f7c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:01.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe589600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:27:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe589600640 1 -- 172.21.15.46:0/275151997 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5940b6470 con 0x560fd2137290 2024-03-29T08:27:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe589600640 1 -- 172.21.15.46:0/275151997 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fe5940b65d0 con 0x560fd2137290 2024-03-29T08:27:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 -- 172.21.15.46:0/275151997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fd1eeb620 con 0x560fd2137290 2024-03-29T08:27:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe589600640 1 -- 172.21.15.46:0/275151997 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5940b6790 con 0x560fd2137290 2024-03-29T08:27:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 10 client.8160 fetch_fsmap learned FSMap version 69 2024-03-29T08:27:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 -- 172.21.15.46:0/275151997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x560fd1c37fc0 con 0x560fd2137290 2024-03-29T08:27:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe589600640 1 -- 172.21.15.46:0/275151997 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560fd1eeb620 con 0x560fd2137290 2024-03-29T08:27:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe589600640 1 -- 172.21.15.46:0/275151997 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 69) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe5940b68f0 con 0x560fd2137290 2024-03-29T08:27:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 10 client.8160 fetch_fsmap finished waiting for FSMap version 69 2024-03-29T08:27:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 10 client.8160 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:27:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 20 client.8160 populate_metadata read hostname 'smithi046' 2024-03-29T08:27:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 --2- 172.21.15.46:0/275151997 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x560fd2137bb0 0x560fd2192170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 4 client.8160 mds_command: new command op to 7986 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-03-29T08:27:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 -- 172.21.15.46:0/275151997 --> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x560fd1f3a4e0 con 0x560fd2137bb0 2024-03-29T08:27:01.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe59be00640 1 --2- 172.21.15.46:0/275151997 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x560fd2137bb0 0x560fd2192170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe59be00640 1 --2- 172.21.15.46:0/275151997 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x560fd2137bb0 0x560fd2192170 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=169191998395fbac server_cookie=eb2ce593f478dc26 in_seq=0 out_seq=0 2024-03-29T08:27:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe589600640 10 client.8160 ms_handle_connect on v2:172.21.15.46:6833/1886680593 2024-03-29T08:27:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe589600640 1 -- 172.21.15.46:0/275151997 <== mds.0 v2:172.21.15.46:6833/1886680593 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560fd1f3a4e0 con 0x560fd2137bb0 2024-03-29T08:27:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe589600640 10 client.8160 handle_command_reply: tid=0 2024-03-29T08:27:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 client.8160 shutdown 2024-03-29T08:27:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 -- 172.21.15.46:0/275151997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 msgr2=0x560fd2191c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 --2- 172.21.15.46:0/275151997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd2191c30 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fe5940190e0 tx=0x7fe594019270 comp rx=0 tx=0).stop 2024-03-29T08:27:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 -- 172.21.15.46:0/275151997 shutdown_connections 2024-03-29T08:27:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 --2- 172.21.15.46:0/275151997 >> [v2:172.21.15.46:6833/1886680593,v1:172.21.15.46:6835/1886680593] conn(0x560fd2137bb0 0x560fd2192170 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 --2- 172.21.15.46:0/275151997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560fd20ee1b0 0x560fd21916f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 --2- 172.21.15.46:0/275151997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560fd2137290 0x560fd2191c30 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 --2- 172.21.15.46:0/275151997 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560fd20ea160 0x560fd2149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe589600640 0 client.8160 ms_handle_reset on v2:172.21.15.46:6833/1886680593 2024-03-29T08:27:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.351+0000 7fe5a6722000 1 -- 172.21.15.46:0/275151997 >> 172.21.15.46:0/275151997 conn(0x560fd21367b0 msgr2=0x560fd20ec550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.355+0000 7fe5a6722000 1 -- 172.21.15.46:0/275151997 shutdown_connections 2024-03-29T08:27:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.355+0000 7fe5a6722000 1 -- 172.21.15.46:0/275151997 wait complete. 2024-03-29T08:27:01.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.355+0000 7fe5a6722000 20 client.8160 trim_cache size 0 max 16384 2024-03-29T08:27:01.362 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:01.362 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-03-29T08:27:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.355+0000 7fe5a3200640 1 -- 172.21.15.46:0/730319745 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe5780666c0 msgr2=0x7fe578068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.355+0000 7fe5a3200640 1 --2- 172.21.15.46:0/730319745 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe5780666c0 0x7fe578068b80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe59c066760 tx=0x7fe58c009040 comp rx=0 tx=0).stop 2024-03-29T08:27:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.355+0000 7fe5a3200640 1 -- 172.21.15.46:0/730319745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe59c15a7f0 msgr2=0x7fe59c14fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.355+0000 7fe5a3200640 1 --2- 172.21.15.46:0/730319745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe59c15a7f0 0x7fe59c14fed0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fe58400c3c0 tx=0x7fe58400c880 comp rx=0 tx=0).stop 2024-03-29T08:27:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.355+0000 7fe5a3200640 1 -- 172.21.15.46:0/730319745 shutdown_connections 2024-03-29T08:27:01.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.355+0000 7fe5a3200640 1 --2- 172.21.15.46:0/730319745 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe5780666c0 0x7fe578068b80 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.355+0000 7fe5a3200640 1 --2- 172.21.15.46:0/730319745 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe59c15abd0 0x7fe59c15c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.355+0000 7fe5a3200640 1 --2- 172.21.15.46:0/730319745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe59c15a7f0 0x7fe59c14fed0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.355+0000 7fe5a3200640 1 --2- 172.21.15.46:0/730319745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe59c099150 0x7fe59c155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.355+0000 7fe5a3200640 1 -- 172.21.15.46:0/730319745 >> 172.21.15.46:0/730319745 conn(0x7fe59c0b2190 msgr2=0x7fe59c06e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.355+0000 7fe5a3200640 1 -- 172.21.15.46:0/730319745 shutdown_connections 2024-03-29T08:27:01.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.355+0000 7fe5a3200640 1 -- 172.21.15.46:0/730319745 wait complete. 2024-03-29T08:27:01.375 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-29T08:27:01.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.511+0000 7fe018000640 1 Processor -- start 2024-03-29T08:27:01.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.511+0000 7fe018000640 1 -- start start 2024-03-29T08:27:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.515+0000 7fe018000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe0100d0150 0x7fe0100cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.515+0000 7fe018000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0100cd9d0 0x7fe0100cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.515+0000 7fe018000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0100b0ed0 con 0x7fe0100b01c0 2024-03-29T08:27:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.515+0000 7fe018000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0100cff90 con 0x7fe0100d0150 2024-03-29T08:27:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.515+0000 7fe018000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100cbd60 con 0x7fe0100cd9d0 2024-03-29T08:27:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.515+0000 7fe014c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe0100d0150 0x7fe0100cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.515+0000 7fe014c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe0100d0150 0x7fe0100cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:43122/0 (socket says 172.21.15.46:43122) 2024-03-29T08:27:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.515+0000 7fe016000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0100cd9d0 0x7fe0100cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.515+0000 7fe014c00640 1 -- 172.21.15.46:0/4149080754 learned_addr learned my addr 172.21.15.46:0/4149080754 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.515+0000 7fe014c00640 1 -- 172.21.15.46:0/4149080754 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0100cd9d0 msgr2=0x7fe0100cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.515+0000 7fe014c00640 1 --2- 172.21.15.46:0/4149080754 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0100cd9d0 0x7fe0100cde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.515+0000 7fe014c00640 1 -- 172.21.15.46:0/4149080754 >> v1:172.21.15.73:6789/0 conn(0x7fe0100b01c0 legacy=0x7fe0100aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.515+0000 7fe014c00640 1 -- 172.21.15.46:0/4149080754 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0100ca5c0 con 0x7fe0100d0150 2024-03-29T08:27:01.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.519+0000 7fe014c00640 1 --2- 172.21.15.46:0/4149080754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0100d0150 0x7fe0100cb820 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00bb10 tx=0x7fdffc01ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d22b2ac0d780608 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:01.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.519+0000 7fe00fe00640 1 -- 172.21.15.46:0/4149080754 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc0a7730 con 0x7fe0100d0150 2024-03-29T08:27:01.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.519+0000 7fe00fe00640 1 -- 172.21.15.46:0/4149080754 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdffc0a7890 con 0x7fe0100d0150 2024-03-29T08:27:01.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.519+0000 7fe00fe00640 1 -- 172.21.15.46:0/4149080754 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdffc0a7c20 con 0x7fe0100d0150 2024-03-29T08:27:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.519+0000 7fe018000640 1 -- 172.21.15.46:0/4149080754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0100d0150 msgr2=0x7fe0100cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.519+0000 7fe018000640 1 --2- 172.21.15.46:0/4149080754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0100d0150 0x7fe0100cb820 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdffc00bb10 tx=0x7fdffc01ec30 comp rx=0 tx=0).stop 2024-03-29T08:27:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.519+0000 7fe018000640 1 -- 172.21.15.46:0/4149080754 shutdown_connections 2024-03-29T08:27:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.519+0000 7fe018000640 1 --2- 172.21.15.46:0/4149080754 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0100cd9d0 0x7fe0100cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.519+0000 7fe018000640 1 --2- 172.21.15.46:0/4149080754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0100d0150 0x7fe0100cb820 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.519+0000 7fe018000640 1 -- 172.21.15.46:0/4149080754 >> 172.21.15.46:0/4149080754 conn(0x7fe010098a40 msgr2=0x7fe0100cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.519+0000 7fe018000640 1 -- 172.21.15.46:0/4149080754 shutdown_connections 2024-03-29T08:27:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.519+0000 7fe018000640 1 -- 172.21.15.46:0/4149080754 wait complete. 2024-03-29T08:27:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.519+0000 7fe018000640 1 Processor -- start 2024-03-29T08:27:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.519+0000 7fe018000640 1 -- start start 2024-03-29T08:27:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe018000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0100b01c0 0x7fe010163ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe018000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0100cd9d0 0x7fe0100b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe018000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0100d0150 0x7fe0101f8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe018000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100c4720 con 0x7fe0100b01c0 2024-03-29T08:27:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe018000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe010163410 con 0x7fe0100d0150 2024-03-29T08:27:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe018000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0101641e0 con 0x7fe0100cd9d0 2024-03-29T08:27:01.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe015600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0100b01c0 0x7fe010163ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe015600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0100b01c0 0x7fe010163ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41214/0 (socket says 172.21.15.46:41214) 2024-03-29T08:27:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe015600640 1 -- 172.21.15.46:0/3243298096 learned_addr learned my addr 172.21.15.46:0/3243298096 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe014c00640 1 --2- 172.21.15.46:0/3243298096 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0100cd9d0 0x7fe0100b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe016000640 1 --2- 172.21.15.46:0/3243298096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0100d0150 0x7fe0101f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe016000640 1 -- 172.21.15.46:0/3243298096 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0100cd9d0 msgr2=0x7fe0100b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe016000640 1 --2- 172.21.15.46:0/3243298096 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0100cd9d0 0x7fe0100b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe016000640 1 -- 172.21.15.46:0/3243298096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0100b01c0 msgr2=0x7fe010163ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe016000640 1 --2- 172.21.15.46:0/3243298096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0100b01c0 0x7fe010163ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe016000640 1 -- 172.21.15.46:0/3243298096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdffc00b770 con 0x7fe0100d0150 2024-03-29T08:27:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe015600640 1 --2- 172.21.15.46:0/3243298096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0100b01c0 0x7fe010163ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:27:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe016000640 1 --2- 172.21.15.46:0/3243298096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0100d0150 0x7fe0101f8950 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe00800efc0 tx=0x7fe00800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe00d600640 1 -- 172.21.15.46:0/3243298096 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008009c00 con 0x7fe0100d0150 2024-03-29T08:27:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe00d600640 1 -- 172.21.15.46:0/3243298096 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00800f040 con 0x7fe0100d0150 2024-03-29T08:27:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe018000640 1 -- 172.21.15.46:0/3243298096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0100b38b0 con 0x7fe0100d0150 2024-03-29T08:27:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe00d600640 1 -- 172.21.15.46:0/3243298096 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0080146c0 con 0x7fe0100d0150 2024-03-29T08:27:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe018000640 1 -- 172.21.15.46:0/3243298096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0100b3e70 con 0x7fe0100d0150 2024-03-29T08:27:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.523+0000 7fe018000640 1 -- 172.21.15.46:0/3243298096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe010066c90 con 0x7fe0100d0150 2024-03-29T08:27:01.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.527+0000 7fe00d600640 1 -- 172.21.15.46:0/3243298096 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe008014820 con 0x7fe0100d0150 2024-03-29T08:27:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.527+0000 7fe00d600640 1 --2- 172.21.15.46:0/3243298096 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdff0066690 0x7fdff0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.527+0000 7fe00d600640 1 -- 172.21.15.46:0/3243298096 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fe0080851b0 con 0x7fe0100d0150 2024-03-29T08:27:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.527+0000 7fe015600640 1 --2- 172.21.15.46:0/3243298096 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdff0066690 0x7fdff0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.527+0000 7fe015600640 1 --2- 172.21.15.46:0/3243298096 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdff0066690 0x7fdff0068b50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe004001ca0 tx=0x7fe004009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:01.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.531+0000 7fe00d600640 1 -- 172.21.15.46:0/3243298096 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe008052590 con 0x7fe0100d0150 2024-03-29T08:27:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.723+0000 7fe018000640 1 -- 172.21.15.46:0/3243298096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe010071560 con 0x7fe0100d0150 2024-03-29T08:27:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.727+0000 7fe00d600640 1 -- 172.21.15.46:0/3243298096 <== mon.1 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fe008055bb0 con 0x7fe0100d0150 2024-03-29T08:27:01.728 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:01.728 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-29T08:27:01.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.727+0000 7fe018000640 1 -- 172.21.15.46:0/3243298096 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdff0066690 msgr2=0x7fdff0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.727+0000 7fe018000640 1 --2- 172.21.15.46:0/3243298096 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdff0066690 0x7fdff0068b50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe004001ca0 tx=0x7fe004009040 comp rx=0 tx=0).stop 2024-03-29T08:27:01.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.727+0000 7fe018000640 1 -- 172.21.15.46:0/3243298096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0100d0150 msgr2=0x7fe0101f8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:01.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.727+0000 7fe018000640 1 --2- 172.21.15.46:0/3243298096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0100d0150 0x7fe0101f8950 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe00800efc0 tx=0x7fe00800c490 comp rx=0 tx=0).stop 2024-03-29T08:27:01.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.727+0000 7fe018000640 1 -- 172.21.15.46:0/3243298096 shutdown_connections 2024-03-29T08:27:01.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.727+0000 7fe018000640 1 --2- 172.21.15.46:0/3243298096 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdff0066690 0x7fdff0068b50 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.727+0000 7fe018000640 1 --2- 172.21.15.46:0/3243298096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0100d0150 0x7fe0101f8950 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.727+0000 7fe018000640 1 --2- 172.21.15.46:0/3243298096 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe0100cd9d0 0x7fe0100b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.727+0000 7fe018000640 1 --2- 172.21.15.46:0/3243298096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe0100b01c0 0x7fe010163ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:01.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.727+0000 7fe018000640 1 -- 172.21.15.46:0/3243298096 >> 172.21.15.46:0/3243298096 conn(0x7fe010098a40 msgr2=0x7fe0100ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:01.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.727+0000 7fe018000640 1 -- 172.21.15.46:0/3243298096 shutdown_connections 2024-03-29T08:27:01.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:01.727+0000 7fe018000640 1 -- 172.21.15.46:0/3243298096 wait complete. 2024-03-29T08:27:01.749 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-03-29T08:27:01.750 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T08:27:01.750 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:27:01.813 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:27:01.866 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T08:27:01.867 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:27:01.867 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:27:01.876 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:27:02.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd9ab800640 1 Processor -- start 2024-03-29T08:27:02.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd9ab800640 1 -- start start 2024-03-29T08:27:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd9ab800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd9a415c960 0x7fd9a415cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd9ab800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd9a415d280 0x7fd9a40cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd9ab800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd9a40cd0e0 0x7fd9a40cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd9ab800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a415e0c0 con 0x7fd9a415d280 2024-03-29T08:27:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd9ab800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a415d7c0 con 0x7fd9a40cd0e0 2024-03-29T08:27:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd9ab800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a415daa0 con 0x7fd9a415c960 2024-03-29T08:27:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd99be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd9a415d280 0x7fd9a40cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd99be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd9a415d280 0x7fd9a40cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41230/0 (socket says 172.21.15.46:41230) 2024-03-29T08:27:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd99be00640 1 -- 172.21.15.46:0/1403549232 learned_addr learned my addr 172.21.15.46:0/1403549232 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd9a8e00640 1 --2- 172.21.15.46:0/1403549232 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd9a415c960 0x7fd9a415cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd9a9800640 1 --2- 172.21.15.46:0/1403549232 >> v2:172.21.15.73:3300/0 conn(0x7fd9a40cd0e0 0x7fd9a40cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd99be00640 1 -- 172.21.15.46:0/1403549232 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd9a415c960 msgr2=0x7fd9a415cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd99be00640 1 --2- 172.21.15.46:0/1403549232 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd9a415c960 0x7fd9a415cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd99be00640 1 -- 172.21.15.46:0/1403549232 >> v2:172.21.15.73:3300/0 conn(0x7fd9a40cd0e0 msgr2=0x7fd9a40cfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:02.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd99be00640 1 --2- 172.21.15.46:0/1403549232 >> v2:172.21.15.73:3300/0 conn(0x7fd9a40cd0e0 0x7fd9a40cfcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd99be00640 1 -- 172.21.15.46:0/1403549232 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a415dd80 con 0x7fd9a415d280 2024-03-29T08:27:02.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd9a8e00640 1 --2- 172.21.15.46:0/1403549232 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd9a415c960 0x7fd9a415cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:02.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.059+0000 7fd9a9800640 1 --2- 172.21.15.46:0/1403549232 >> v2:172.21.15.73:3300/0 conn(0x7fd9a40cd0e0 0x7fd9a40cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:02.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.063+0000 7fd99be00640 1 --2- 172.21.15.46:0/1403549232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a415d280 0x7fd9a40cf7b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4160430 tx=0x7fd98c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f144abb9e1d98f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.063+0000 7fd9a3e00640 1 -- 172.21.15.46:0/1403549232 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0aa430 con 0x7fd9a415d280 2024-03-29T08:27:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.063+0000 7fd9a3e00640 1 -- 172.21.15.46:0/1403549232 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c0aa590 con 0x7fd9a415d280 2024-03-29T08:27:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.063+0000 7fd9a3e00640 1 -- 172.21.15.46:0/1403549232 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0aa920 con 0x7fd9a415d280 2024-03-29T08:27:02.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.063+0000 7fd9ab800640 1 -- 172.21.15.46:0/1403549232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a415d280 msgr2=0x7fd9a40cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.063+0000 7fd9ab800640 1 --2- 172.21.15.46:0/1403549232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a415d280 0x7fd9a40cf7b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4160430 tx=0x7fd98c01c4f0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.063+0000 7fd9ab800640 1 -- 172.21.15.46:0/1403549232 shutdown_connections 2024-03-29T08:27:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.063+0000 7fd9ab800640 1 --2- 172.21.15.46:0/1403549232 >> v2:172.21.15.73:3300/0 conn(0x7fd9a40cd0e0 0x7fd9a40cfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.063+0000 7fd9ab800640 1 --2- 172.21.15.46:0/1403549232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a415d280 0x7fd9a40cf7b0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.063+0000 7fd9ab800640 1 --2- 172.21.15.46:0/1403549232 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd9a415c960 0x7fd9a415cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.063+0000 7fd9ab800640 1 -- 172.21.15.46:0/1403549232 >> 172.21.15.46:0/1403549232 conn(0x7fd9a40718c0 msgr2=0x7fd9a4071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.063+0000 7fd9ab800640 1 -- 172.21.15.46:0/1403549232 shutdown_connections 2024-03-29T08:27:02.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.063+0000 7fd9ab800640 1 -- 172.21.15.46:0/1403549232 wait complete. 2024-03-29T08:27:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9ab800640 1 Processor -- start 2024-03-29T08:27:02.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9ab800640 1 -- start start 2024-03-29T08:27:02.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9ab800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a40cd0e0 0x7fd9a40d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9ab800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd9a415c960 0x7fd9a41f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9ab800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9a415d280 0x7fd9a40cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9ab800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a40cbd30 con 0x7fd9a40cd0e0 2024-03-29T08:27:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9ab800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a40d31b0 con 0x7fd9a415d280 2024-03-29T08:27:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9ab800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a40caa30 con 0x7fd9a415c960 2024-03-29T08:27:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9a8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a40cd0e0 0x7fd9a40d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9a8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a40cd0e0 0x7fd9a40d2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41234/0 (socket says 172.21.15.46:41234) 2024-03-29T08:27:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9a8e00640 1 -- 172.21.15.46:0/800397620 learned_addr learned my addr 172.21.15.46:0/800397620 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9a9800640 1 --2- 172.21.15.46:0/800397620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9a415d280 0x7fd9a40cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd99be00640 1 --2- 172.21.15.46:0/800397620 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd9a415c960 0x7fd9a41f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9a8e00640 1 -- 172.21.15.46:0/800397620 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd9a415c960 msgr2=0x7fd9a41f5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9a8e00640 1 --2- 172.21.15.46:0/800397620 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd9a415c960 0x7fd9a41f5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9a8e00640 1 -- 172.21.15.46:0/800397620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9a415d280 msgr2=0x7fd9a40cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:02.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9a8e00640 1 --2- 172.21.15.46:0/800397620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9a415d280 0x7fd9a40cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9a8e00640 1 -- 172.21.15.46:0/800397620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c0094d0 con 0x7fd9a40cd0e0 2024-03-29T08:27:02.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9a8e00640 1 --2- 172.21.15.46:0/800397620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a40cd0e0 0x7fd9a40d2b20 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fd99400e810 tx=0x7fd99400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:02.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9a1600640 1 -- 172.21.15.46:0/800397620 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99400c8d0 con 0x7fd9a40cd0e0 2024-03-29T08:27:02.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9a1600640 1 -- 172.21.15.46:0/800397620 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99400ca30 con 0x7fd9a40cd0e0 2024-03-29T08:27:02.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9ab800640 1 -- 172.21.15.46:0/800397620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a40cad10 con 0x7fd9a40cd0e0 2024-03-29T08:27:02.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9a1600640 1 -- 172.21.15.46:0/800397620 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99400cdc0 con 0x7fd9a40cd0e0 2024-03-29T08:27:02.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9ab800640 1 -- 172.21.15.46:0/800397620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a40cb2d0 con 0x7fd9a40cd0e0 2024-03-29T08:27:02.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.067+0000 7fd9ab800640 1 -- 172.21.15.46:0/800397620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a4066c90 con 0x7fd9a40cd0e0 2024-03-29T08:27:02.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.071+0000 7fd9a1600640 1 -- 172.21.15.46:0/800397620 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd994013410 con 0x7fd9a40cd0e0 2024-03-29T08:27:02.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.071+0000 7fd9a1600640 1 --2- 172.21.15.46:0/800397620 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd980066690 0x7fd980068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.071+0000 7fd9a1600640 1 -- 172.21.15.46:0/800397620 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fd9940840c0 con 0x7fd9a40cd0e0 2024-03-29T08:27:02.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.071+0000 7fd99be00640 1 --2- 172.21.15.46:0/800397620 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd980066690 0x7fd980068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.071+0000 7fd99be00640 1 --2- 172.21.15.46:0/800397620 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd980066690 0x7fd980068b50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0ae000 tx=0x7fd98c0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:02.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.075+0000 7fd9a1600640 1 -- 172.21.15.46:0/800397620 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd994051370 con 0x7fd9a40cd0e0 2024-03-29T08:27:02.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.239+0000 7fd9ab800640 1 -- 172.21.15.46:0/800397620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9a40b16a0 con 0x7fd9a40cd0e0 2024-03-29T08:27:02.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.239+0000 7fd9a1600640 1 -- 172.21.15.46:0/800397620 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16315 (secure 0 0 0) 0x7fd994009480 con 0x7fd9a40cd0e0 2024-03-29T08:27:02.242 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:02.242 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:26:22.851956+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:24:41.113618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-29T08:24:42.487380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/2051429508":"2024-03-30T08:26:22.851926+0000","172.21.15.46:6833/2051429508":"2024-03-30T08:26:22.851926+0000","172.21.15.73:6836/4166845858":"2024-03-30T08:25:02.334350+0000","172.21.15.73:6834/4166845858":"2024-03-30T08:25:02.334350+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:27:02.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.239+0000 7fd9ab800640 1 -- 172.21.15.46:0/800397620 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd980066690 msgr2=0x7fd980068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:02.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.239+0000 7fd9ab800640 1 --2- 172.21.15.46:0/800397620 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd980066690 0x7fd980068b50 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0ae000 tx=0x7fd98c0a7000 comp rx=0 tx=0).stop 2024-03-29T08:27:02.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.239+0000 7fd9ab800640 1 -- 172.21.15.46:0/800397620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a40cd0e0 msgr2=0x7fd9a40d2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:02.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.239+0000 7fd9ab800640 1 --2- 172.21.15.46:0/800397620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a40cd0e0 0x7fd9a40d2b20 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fd99400e810 tx=0x7fd99400ecd0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.239+0000 7fd9ab800640 1 -- 172.21.15.46:0/800397620 shutdown_connections 2024-03-29T08:27:02.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.239+0000 7fd9ab800640 1 --2- 172.21.15.46:0/800397620 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd980066690 0x7fd980068b50 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.239+0000 7fd9ab800640 1 --2- 172.21.15.46:0/800397620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9a415d280 0x7fd9a40cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.239+0000 7fd9ab800640 1 --2- 172.21.15.46:0/800397620 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd9a415c960 0x7fd9a41f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.239+0000 7fd9ab800640 1 --2- 172.21.15.46:0/800397620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9a40cd0e0 0x7fd9a40d2b20 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.239+0000 7fd9ab800640 1 -- 172.21.15.46:0/800397620 >> 172.21.15.46:0/800397620 conn(0x7fd9a40718c0 msgr2=0x7fd9a415ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:02.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.243+0000 7fd9ab800640 1 -- 172.21.15.46:0/800397620 shutdown_connections 2024-03-29T08:27:02.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.243+0000 7fd9ab800640 1 -- 172.21.15.46:0/800397620 wait complete. 2024-03-29T08:27:02.263 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:24:41.113618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:24:42.487380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:27:02.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:27:02.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b1d800640 1 Processor -- start 2024-03-29T08:27:02.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b1d800640 1 -- start start 2024-03-29T08:27:02.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b1d800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2b1815c950 0x7f2b1806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b1d800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b180cea30 0x7f2b181606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b1d800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2b1815cd30 0x7f2b18160be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b1d800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b180cfb20 con 0x7f2b1815c950 2024-03-29T08:27:02.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b1d800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b180cef10 con 0x7f2b1815cd30 2024-03-29T08:27:02.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b1d800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b180cf1f0 con 0x7f2b180cea30 2024-03-29T08:27:02.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b16a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2b1815c950 0x7f2b1806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b16a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2b1815c950 0x7f2b1806bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41246/0 (socket says 172.21.15.46:41246) 2024-03-29T08:27:02.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b16a00640 1 -- 172.21.15.46:0/605074419 learned_addr learned my addr 172.21.15.46:0/605074419 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:02.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b17400640 1 --2- 172.21.15.46:0/605074419 >> v2:172.21.15.73:3300/0 conn(0x7f2b1815cd30 0x7f2b18160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b0fe00640 1 --2- 172.21.15.46:0/605074419 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b180cea30 0x7f2b181606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b16a00640 1 -- 172.21.15.46:0/605074419 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b180cea30 msgr2=0x7f2b181606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:02.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b16a00640 1 --2- 172.21.15.46:0/605074419 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b180cea30 0x7f2b181606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b16a00640 1 -- 172.21.15.46:0/605074419 >> v2:172.21.15.73:3300/0 conn(0x7f2b1815cd30 msgr2=0x7f2b18160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:02.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b16a00640 1 --2- 172.21.15.46:0/605074419 >> v2:172.21.15.73:3300/0 conn(0x7f2b1815cd30 0x7f2b18160be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b16a00640 1 -- 172.21.15.46:0/605074419 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b180cf4d0 con 0x7f2b1815c950 2024-03-29T08:27:02.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.407+0000 7f2b17400640 1 --2- 172.21.15.46:0/605074419 >> v2:172.21.15.73:3300/0 conn(0x7f2b1815cd30 0x7f2b18160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:02.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.411+0000 7f2b16a00640 1 --2- 172.21.15.46:0/605074419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1815c950 0x7f2b1806bbe0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f2b18071700 tx=0x7f2b0801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d604e84061c14d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.411+0000 7f2b16000640 1 -- 172.21.15.46:0/605074419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b080a89d0 con 0x7f2b1815c950 2024-03-29T08:27:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.411+0000 7f2b16000640 1 -- 172.21.15.46:0/605074419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b080a8b30 con 0x7f2b1815c950 2024-03-29T08:27:02.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.411+0000 7f2b16000640 1 -- 172.21.15.46:0/605074419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b080b0440 con 0x7f2b1815c950 2024-03-29T08:27:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.411+0000 7f2b1d800640 1 -- 172.21.15.46:0/605074419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1815c950 msgr2=0x7f2b1806bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.411+0000 7f2b1d800640 1 --2- 172.21.15.46:0/605074419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1815c950 0x7f2b1806bbe0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f2b18071700 tx=0x7f2b0801ed10 comp rx=0 tx=0).stop 2024-03-29T08:27:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.411+0000 7f2b1d800640 1 -- 172.21.15.46:0/605074419 shutdown_connections 2024-03-29T08:27:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.411+0000 7f2b1d800640 1 --2- 172.21.15.46:0/605074419 >> v2:172.21.15.73:3300/0 conn(0x7f2b1815cd30 0x7f2b18160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.411+0000 7f2b1d800640 1 --2- 172.21.15.46:0/605074419 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b180cea30 0x7f2b181606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.411+0000 7f2b1d800640 1 --2- 172.21.15.46:0/605074419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1815c950 0x7f2b1806bbe0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.411+0000 7f2b1d800640 1 -- 172.21.15.46:0/605074419 >> 172.21.15.46:0/605074419 conn(0x7f2b180b21b0 msgr2=0x7f2b180b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.411+0000 7f2b1d800640 1 -- 172.21.15.46:0/605074419 shutdown_connections 2024-03-29T08:27:02.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.411+0000 7f2b1d800640 1 -- 172.21.15.46:0/605074419 wait complete. 2024-03-29T08:27:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.411+0000 7f2b1d800640 1 Processor -- start 2024-03-29T08:27:02.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.411+0000 7f2b1d800640 1 -- start start 2024-03-29T08:27:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b1d800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b180cea30 0x7f2b18157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b1d800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b1815c950 0x7f2b180b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b1d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1815cd30 0x7f2b180b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b1d800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b180b58c0 con 0x7f2b1815cd30 2024-03-29T08:27:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b1d800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b181f35d0 con 0x7f2b1815c950 2024-03-29T08:27:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b1d800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b181f38b0 con 0x7f2b180cea30 2024-03-29T08:27:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b17400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1815cd30 0x7f2b180b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b17400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1815cd30 0x7f2b180b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41248/0 (socket says 172.21.15.46:41248) 2024-03-29T08:27:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b17400640 1 -- 172.21.15.46:0/1737865945 learned_addr learned my addr 172.21.15.46:0/1737865945 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b0fe00640 1 --2- 172.21.15.46:0/1737865945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b1815c950 0x7f2b180b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b16a00640 1 --2- 172.21.15.46:0/1737865945 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b180cea30 0x7f2b18157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b17400640 1 -- 172.21.15.46:0/1737865945 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b180cea30 msgr2=0x7f2b18157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b17400640 1 --2- 172.21.15.46:0/1737865945 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b180cea30 0x7f2b18157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b17400640 1 -- 172.21.15.46:0/1737865945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b1815c950 msgr2=0x7f2b180b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b17400640 1 --2- 172.21.15.46:0/1737865945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b1815c950 0x7f2b180b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b17400640 1 -- 172.21.15.46:0/1737865945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b080094d0 con 0x7f2b1815cd30 2024-03-29T08:27:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b17400640 1 --2- 172.21.15.46:0/1737865945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1815cd30 0x7f2b180b31e0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f2b10007e70 tx=0x7f2b1000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b0ea00640 1 -- 172.21.15.46:0/1737865945 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b10003ba0 con 0x7f2b1815cd30 2024-03-29T08:27:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b0ea00640 1 -- 172.21.15.46:0/1737865945 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1000f040 con 0x7f2b1815cd30 2024-03-29T08:27:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b0ea00640 1 -- 172.21.15.46:0/1737865945 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1000de80 con 0x7f2b1815cd30 2024-03-29T08:27:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b1d800640 1 -- 172.21.15.46:0/1737865945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b181f3b90 con 0x7f2b1815cd30 2024-03-29T08:27:02.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b1d800640 1 -- 172.21.15.46:0/1737865945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b180b5310 con 0x7f2b1815cd30 2024-03-29T08:27:02.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.415+0000 7f2b1d800640 1 -- 172.21.15.46:0/1737865945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b181f40b0 con 0x7f2b1815cd30 2024-03-29T08:27:02.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.419+0000 7f2b0ea00640 1 -- 172.21.15.46:0/1737865945 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2b10004030 con 0x7f2b1815cd30 2024-03-29T08:27:02.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.419+0000 7f2b0ea00640 1 --2- 172.21.15.46:0/1737865945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2af4066690 0x7f2af4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.419+0000 7f2b0ea00640 1 -- 172.21.15.46:0/1737865945 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f2b10084570 con 0x7f2b1815cd30 2024-03-29T08:27:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.419+0000 7f2b16a00640 1 --2- 172.21.15.46:0/1737865945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2af4066690 0x7f2af4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.419+0000 7f2b16a00640 1 --2- 172.21.15.46:0/1737865945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2af4066690 0x7f2af4068b50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2b080b2000 tx=0x7f2b08008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:02.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.423+0000 7f2b0ea00640 1 -- 172.21.15.46:0/1737865945 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2b181f40b0 con 0x7f2b1815cd30 2024-03-29T08:27:02.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.583+0000 7f2b1d800640 1 -- 172.21.15.46:0/1737865945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2b181f40b0 con 0x7f2b1815cd30 2024-03-29T08:27:02.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.587+0000 7f2b0ea00640 1 -- 172.21.15.46:0/1737865945 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2b181f40b0 con 0x7f2b1815cd30 2024-03-29T08:27:02.588 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:27:02.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.587+0000 7f2b1d800640 1 -- 172.21.15.46:0/1737865945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2af4066690 msgr2=0x7f2af4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:02.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.587+0000 7f2b1d800640 1 --2- 172.21.15.46:0/1737865945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2af4066690 0x7f2af4068b50 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2b080b2000 tx=0x7f2b08008040 comp rx=0 tx=0).stop 2024-03-29T08:27:02.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.587+0000 7f2b1d800640 1 -- 172.21.15.46:0/1737865945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1815cd30 msgr2=0x7f2b180b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:02.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.587+0000 7f2b1d800640 1 --2- 172.21.15.46:0/1737865945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1815cd30 0x7f2b180b31e0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f2b10007e70 tx=0x7f2b1000d4a0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.587+0000 7f2b1d800640 1 -- 172.21.15.46:0/1737865945 shutdown_connections 2024-03-29T08:27:02.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.587+0000 7f2b1d800640 1 --2- 172.21.15.46:0/1737865945 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2af4066690 0x7f2af4068b50 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.587+0000 7f2b1d800640 1 --2- 172.21.15.46:0/1737865945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b1815c950 0x7f2b180b4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.587+0000 7f2b1d800640 1 --2- 172.21.15.46:0/1737865945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b1815cd30 0x7f2b180b31e0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.587+0000 7f2b1d800640 1 --2- 172.21.15.46:0/1737865945 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2b180cea30 0x7f2b18157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.587+0000 7f2b1d800640 1 -- 172.21.15.46:0/1737865945 >> 172.21.15.46:0/1737865945 conn(0x7f2b180b21b0 msgr2=0x7f2b1815d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:02.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.587+0000 7f2b1d800640 1 -- 172.21.15.46:0/1737865945 shutdown_connections 2024-03-29T08:27:02.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.587+0000 7f2b1d800640 1 -- 172.21.15.46:0/1737865945 wait complete. 2024-03-29T08:27:02.609 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:27:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7feff3200640 1 Processor -- start 2024-03-29T08:27:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7feff3200640 1 -- start start 2024-03-29T08:27:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7feff3200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7feff415cb40 0x7feff40cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7feff3200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feff406c990 0x7feff40c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7feff3200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff40ca340 con 0x7feff40ccb50 2024-03-29T08:27:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7feff3200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff415cf20 con 0x7feff406c990 2024-03-29T08:27:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7feff3200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff40ca5e0 con 0x7feff415cb40 2024-03-29T08:27:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7feff1e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7feff415cb40 0x7feff40cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7fefe9400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feff406c990 0x7feff40c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7feff1e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7feff415cb40 0x7feff40cb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:35168/0 (socket says 172.21.15.46:35168) 2024-03-29T08:27:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7feff1e00640 1 -- 172.21.15.46:0/207225965 learned_addr learned my addr 172.21.15.46:0/207225965 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7feff2800640 1 --1- 172.21.15.46:0/207225965 >> v1:172.21.15.73:6789/0 conn(0x7feff40ccb50 0x7feff406a910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:57088/0 (socket says 172.21.15.46:57088) 2024-03-29T08:27:02.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7feff1400640 1 -- 172.21.15.46:0/207225965 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803319867 0 0) 0x7feff40ca340 con 0x7feff40ccb50 2024-03-29T08:27:02.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7feff1400640 1 -- 172.21.15.46:0/207225965 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefd8003530 con 0x7feff40ccb50 2024-03-29T08:27:02.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7feff1e00640 1 -- 172.21.15.46:0/207225965 >> v2:172.21.15.73:3300/0 conn(0x7feff406c990 msgr2=0x7feff40c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:02.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7feff1e00640 1 --2- 172.21.15.46:0/207225965 >> v2:172.21.15.73:3300/0 conn(0x7feff406c990 0x7feff40c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7feff1e00640 1 -- 172.21.15.46:0/207225965 >> v1:172.21.15.73:6789/0 conn(0x7feff40ccb50 legacy=0x7feff406a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:02.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7feff1e00640 1 -- 172.21.15.46:0/207225965 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff4069e30 con 0x7feff415cb40 2024-03-29T08:27:02.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.751+0000 7fefe9400640 1 --2- 172.21.15.46:0/207225965 >> v2:172.21.15.73:3300/0 conn(0x7feff406c990 0x7feff40c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:27:02.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.755+0000 7feff1e00640 1 --2- 172.21.15.46:0/207225965 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7feff415cb40 0x7feff40cb0d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009e50 tx=0x7fefe401ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39379ed9005db3b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:02.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.755+0000 7feff1400640 1 -- 172.21.15.46:0/207225965 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe40a57f0 con 0x7feff415cb40 2024-03-29T08:27:02.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.755+0000 7feff1400640 1 -- 172.21.15.46:0/207225965 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe40a5950 con 0x7feff415cb40 2024-03-29T08:27:02.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.755+0000 7feff1400640 1 -- 172.21.15.46:0/207225965 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe40a5ce0 con 0x7feff415cb40 2024-03-29T08:27:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.755+0000 7feff3200640 1 -- 172.21.15.46:0/207225965 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7feff415cb40 msgr2=0x7feff40cb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.755+0000 7feff3200640 1 --2- 172.21.15.46:0/207225965 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7feff415cb40 0x7feff40cb0d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fefe4009e50 tx=0x7fefe401ca30 comp rx=0 tx=0).stop 2024-03-29T08:27:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.755+0000 7feff3200640 1 -- 172.21.15.46:0/207225965 shutdown_connections 2024-03-29T08:27:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.755+0000 7feff3200640 1 --2- 172.21.15.46:0/207225965 >> v2:172.21.15.73:3300/0 conn(0x7feff406c990 0x7feff40c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.755+0000 7feff3200640 1 --2- 172.21.15.46:0/207225965 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7feff415cb40 0x7feff40cb0d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.755+0000 7feff3200640 1 -- 172.21.15.46:0/207225965 >> 172.21.15.46:0/207225965 conn(0x7feff40cb6d0 msgr2=0x7feff40cbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:02.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.755+0000 7feff3200640 1 -- 172.21.15.46:0/207225965 shutdown_connections 2024-03-29T08:27:02.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.755+0000 7feff3200640 1 -- 172.21.15.46:0/207225965 wait complete. 2024-03-29T08:27:02.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff3200640 1 Processor -- start 2024-03-29T08:27:02.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff3200640 1 -- start start 2024-03-29T08:27:02.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff3200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7feff406c990 0x7feff40c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feff40ccb50 0x7feff40c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff415cb40 0x7feff4145240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff40c4f50 con 0x7feff415cb40 2024-03-29T08:27:02.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff40b23c0 con 0x7feff40ccb50 2024-03-29T08:27:02.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff3200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff40b26a0 con 0x7feff406c990 2024-03-29T08:27:02.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff415cb40 0x7feff4145240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff415cb40 0x7feff4145240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41264/0 (socket says 172.21.15.46:41264) 2024-03-29T08:27:02.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff2800640 1 -- 172.21.15.46:0/1111043078 learned_addr learned my addr 172.21.15.46:0/1111043078 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:02.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7fefe9400640 1 --2- 172.21.15.46:0/1111043078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feff40ccb50 0x7feff40c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff1e00640 1 --2- 172.21.15.46:0/1111043078 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7feff406c990 0x7feff40c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff2800640 1 -- 172.21.15.46:0/1111043078 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7feff406c990 msgr2=0x7feff40c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:02.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff2800640 1 --2- 172.21.15.46:0/1111043078 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7feff406c990 0x7feff40c44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff2800640 1 -- 172.21.15.46:0/1111043078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feff40ccb50 msgr2=0x7feff40c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:02.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff2800640 1 --2- 172.21.15.46:0/1111043078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feff40ccb50 0x7feff40c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff2800640 1 -- 172.21.15.46:0/1111043078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc009420 con 0x7feff415cb40 2024-03-29T08:27:02.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff2800640 1 --2- 172.21.15.46:0/1111043078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff415cb40 0x7feff4145240 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fefec00c450 tx=0x7fefec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:02.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7fefeaa00640 1 -- 172.21.15.46:0/1111043078 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec007480 con 0x7feff415cb40 2024-03-29T08:27:02.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7fefeaa00640 1 -- 172.21.15.46:0/1111043078 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefec0075e0 con 0x7feff415cb40 2024-03-29T08:27:02.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7fefeaa00640 1 -- 172.21.15.46:0/1111043078 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec004a30 con 0x7feff415cb40 2024-03-29T08:27:02.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff3200640 1 -- 172.21.15.46:0/1111043078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe4009b30 con 0x7feff415cb40 2024-03-29T08:27:02.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.759+0000 7feff3200640 1 -- 172.21.15.46:0/1111043078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff40b2c60 con 0x7feff415cb40 2024-03-29T08:27:02.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.763+0000 7feff3200640 1 -- 172.21.15.46:0/1111043078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff40b3180 con 0x7feff415cb40 2024-03-29T08:27:02.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.763+0000 7fefeaa00640 1 -- 172.21.15.46:0/1111043078 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fefec004b90 con 0x7feff415cb40 2024-03-29T08:27:02.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.763+0000 7fefeaa00640 1 --2- 172.21.15.46:0/1111043078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fefd0066690 0x7fefd0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:02.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.763+0000 7fefeaa00640 1 -- 172.21.15.46:0/1111043078 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fefec0833f0 con 0x7feff415cb40 2024-03-29T08:27:02.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.763+0000 7feff1e00640 1 --2- 172.21.15.46:0/1111043078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fefd0066690 0x7fefd0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:02.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.763+0000 7feff1e00640 1 --2- 172.21.15.46:0/1111043078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fefd0066690 0x7fefd0068b50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fefe40ae000 tx=0x7fefe40a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:02.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.767+0000 7fefeaa00640 1 -- 172.21.15.46:0/1111043078 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fefec0506c0 con 0x7feff415cb40 2024-03-29T08:27:02.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.927+0000 7feff3200640 1 -- 172.21.15.46:0/1111043078 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feff404e040 con 0x7feff415cb40 2024-03-29T08:27:02.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.927+0000 7fefeaa00640 1 -- 172.21.15.46:0/1111043078 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7fefec053ce0 con 0x7feff415cb40 2024-03-29T08:27:02.929 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:27:02.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.927+0000 7feff3200640 1 -- 172.21.15.46:0/1111043078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fefd0066690 msgr2=0x7fefd0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:02.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.927+0000 7feff3200640 1 --2- 172.21.15.46:0/1111043078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fefd0066690 0x7fefd0068b50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fefe40ae000 tx=0x7fefe40a7070 comp rx=0 tx=0).stop 2024-03-29T08:27:02.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.927+0000 7feff3200640 1 -- 172.21.15.46:0/1111043078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff415cb40 msgr2=0x7feff4145240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:02.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.927+0000 7feff3200640 1 --2- 172.21.15.46:0/1111043078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff415cb40 0x7feff4145240 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fefec00c450 tx=0x7fefec00c910 comp rx=0 tx=0).stop 2024-03-29T08:27:02.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.927+0000 7feff3200640 1 -- 172.21.15.46:0/1111043078 shutdown_connections 2024-03-29T08:27:02.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.927+0000 7feff3200640 1 --2- 172.21.15.46:0/1111043078 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fefd0066690 0x7fefd0068b50 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.927+0000 7feff3200640 1 --2- 172.21.15.46:0/1111043078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feff40ccb50 0x7feff40c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.927+0000 7feff3200640 1 --2- 172.21.15.46:0/1111043078 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feff415cb40 0x7feff4145240 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.927+0000 7feff3200640 1 --2- 172.21.15.46:0/1111043078 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7feff406c990 0x7feff40c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:02.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.927+0000 7feff3200640 1 -- 172.21.15.46:0/1111043078 >> 172.21.15.46:0/1111043078 conn(0x7feff40cb6d0 msgr2=0x7feff40af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:02.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.927+0000 7feff3200640 1 -- 172.21.15.46:0/1111043078 shutdown_connections 2024-03-29T08:27:02.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:02.927+0000 7feff3200640 1 -- 172.21.15.46:0/1111043078 wait complete. 2024-03-29T08:27:02.950 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:27:03.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.091+0000 7f11fd400640 1 Processor -- start 2024-03-29T08:27:03.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.091+0000 7f11fd400640 1 -- start start 2024-03-29T08:27:03.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.091+0000 7f11fd400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11f815c950 0x7f11f806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:03.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.091+0000 7f11fd400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11f80cea30 0x7f11f81606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:03.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.091+0000 7f11fd400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f11f815cd30 0x7f11f8160be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:03.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.091+0000 7f11fd400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f80cfb20 con 0x7f11f80cea30 2024-03-29T08:27:03.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.091+0000 7f11fd400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f80cef10 con 0x7f11f815cd30 2024-03-29T08:27:03.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.091+0000 7f11fd400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80cf1f0 con 0x7f11f815c950 2024-03-29T08:27:03.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.095+0000 7f11efe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11f80cea30 0x7f11f81606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.095+0000 7f11efe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f11f80cea30 0x7f11f81606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41274/0 (socket says 172.21.15.46:41274) 2024-03-29T08:27:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.095+0000 7f11efe00640 1 -- 172.21.15.46:0/2872938572 learned_addr learned my addr 172.21.15.46:0/2872938572 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.095+0000 7f11f6800640 1 --2- 172.21.15.46:0/2872938572 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11f815c950 0x7f11f806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.095+0000 7f11f7200640 1 --2- 172.21.15.46:0/2872938572 >> v2:172.21.15.73:3300/0 conn(0x7f11f815cd30 0x7f11f8160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.095+0000 7f11efe00640 1 -- 172.21.15.46:0/2872938572 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11f815c950 msgr2=0x7f11f806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.095+0000 7f11efe00640 1 --2- 172.21.15.46:0/2872938572 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11f815c950 0x7f11f806bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.095+0000 7f11efe00640 1 -- 172.21.15.46:0/2872938572 >> v2:172.21.15.73:3300/0 conn(0x7f11f815cd30 msgr2=0x7f11f8160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.095+0000 7f11efe00640 1 --2- 172.21.15.46:0/2872938572 >> v2:172.21.15.73:3300/0 conn(0x7f11f815cd30 0x7f11f8160be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.095+0000 7f11efe00640 1 -- 172.21.15.46:0/2872938572 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f80cf4d0 con 0x7f11f80cea30 2024-03-29T08:27:03.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.095+0000 7f11f7200640 1 --2- 172.21.15.46:0/2872938572 >> v2:172.21.15.73:3300/0 conn(0x7f11f815cd30 0x7f11f8160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:03.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.095+0000 7f11efe00640 1 --2- 172.21.15.46:0/2872938572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f80cea30 0x7f11f81606a0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f11f8071700 tx=0x7f11e001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67b300529312867 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:03.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.095+0000 7f11f5e00640 1 -- 172.21.15.46:0/2872938572 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e00a89d0 con 0x7f11f80cea30 2024-03-29T08:27:03.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.095+0000 7f11f5e00640 1 -- 172.21.15.46:0/2872938572 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e00a8b30 con 0x7f11f80cea30 2024-03-29T08:27:03.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.095+0000 7f11f5e00640 1 -- 172.21.15.46:0/2872938572 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e00b0440 con 0x7f11f80cea30 2024-03-29T08:27:03.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11fd400640 1 -- 172.21.15.46:0/2872938572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f80cea30 msgr2=0x7f11f81606a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:03.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11fd400640 1 --2- 172.21.15.46:0/2872938572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f80cea30 0x7f11f81606a0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f11f8071700 tx=0x7f11e001ed10 comp rx=0 tx=0).stop 2024-03-29T08:27:03.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11fd400640 1 -- 172.21.15.46:0/2872938572 shutdown_connections 2024-03-29T08:27:03.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11fd400640 1 --2- 172.21.15.46:0/2872938572 >> v2:172.21.15.73:3300/0 conn(0x7f11f815cd30 0x7f11f8160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11fd400640 1 --2- 172.21.15.46:0/2872938572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f80cea30 0x7f11f81606a0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11fd400640 1 --2- 172.21.15.46:0/2872938572 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11f815c950 0x7f11f806bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11fd400640 1 -- 172.21.15.46:0/2872938572 >> 172.21.15.46:0/2872938572 conn(0x7f11f80b21b0 msgr2=0x7f11f80b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11fd400640 1 -- 172.21.15.46:0/2872938572 shutdown_connections 2024-03-29T08:27:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11fd400640 1 -- 172.21.15.46:0/2872938572 wait complete. 2024-03-29T08:27:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11fd400640 1 Processor -- start 2024-03-29T08:27:03.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11fd400640 1 -- start start 2024-03-29T08:27:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11fd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f80cea30 0x7f11f8159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11fd400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11f815c950 0x7f11f809dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11fd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f815cd30 0x7f11f815a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11fd400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80b4c40 con 0x7f11f80cea30 2024-03-29T08:27:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11fd400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80b5aa0 con 0x7f11f815cd30 2024-03-29T08:27:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11fd400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80b4e90 con 0x7f11f815c950 2024-03-29T08:27:03.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11f6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f80cea30 0x7f11f8159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11f6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f80cea30 0x7f11f8159190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41288/0 (socket says 172.21.15.46:41288) 2024-03-29T08:27:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11f6800640 1 -- 172.21.15.46:0/2262076408 learned_addr learned my addr 172.21.15.46:0/2262076408 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11efe00640 1 --2- 172.21.15.46:0/2262076408 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11f815c950 0x7f11f809dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11f7200640 1 --2- 172.21.15.46:0/2262076408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f815cd30 0x7f11f815a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11f6800640 1 -- 172.21.15.46:0/2262076408 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11f815c950 msgr2=0x7f11f809dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11f6800640 1 --2- 172.21.15.46:0/2262076408 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11f815c950 0x7f11f809dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11f6800640 1 -- 172.21.15.46:0/2262076408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f815cd30 msgr2=0x7f11f815a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11f6800640 1 --2- 172.21.15.46:0/2262076408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f815cd30 0x7f11f815a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.099+0000 7f11f6800640 1 -- 172.21.15.46:0/2262076408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e00094d0 con 0x7f11f80cea30 2024-03-29T08:27:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.103+0000 7f11f6800640 1 --2- 172.21.15.46:0/2262076408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f80cea30 0x7f11f8159190 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f11e800d800 tx=0x7f11e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.103+0000 7f11eea00640 1 -- 172.21.15.46:0/2262076408 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e800b6c0 con 0x7f11f80cea30 2024-03-29T08:27:03.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.103+0000 7f11eea00640 1 -- 172.21.15.46:0/2262076408 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e800b820 con 0x7f11f80cea30 2024-03-29T08:27:03.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.103+0000 7f11fd400640 1 -- 172.21.15.46:0/2262076408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f80b5170 con 0x7f11f80cea30 2024-03-29T08:27:03.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.103+0000 7f11eea00640 1 -- 172.21.15.46:0/2262076408 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e800bb90 con 0x7f11f80cea30 2024-03-29T08:27:03.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.103+0000 7f11fd400640 1 -- 172.21.15.46:0/2262076408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f80b5730 con 0x7f11f80cea30 2024-03-29T08:27:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.103+0000 7f11fd400640 1 -- 172.21.15.46:0/2262076408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f809e690 con 0x7f11f80cea30 2024-03-29T08:27:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.103+0000 7f11eea00640 1 -- 172.21.15.46:0/2262076408 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f11e800bcf0 con 0x7f11f80cea30 2024-03-29T08:27:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.103+0000 7f11eea00640 1 --2- 172.21.15.46:0/2262076408 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11d4066690 0x7f11d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.103+0000 7f11eea00640 1 -- 172.21.15.46:0/2262076408 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f11e8084870 con 0x7f11f80cea30 2024-03-29T08:27:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.103+0000 7f11efe00640 1 --2- 172.21.15.46:0/2262076408 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11d4066690 0x7f11d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:03.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.107+0000 7f11efe00640 1 --2- 172.21.15.46:0/2262076408 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11d4066690 0x7f11d4068b50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f11e00b2000 tx=0x7f11e0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:03.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.107+0000 7f11eea00640 1 -- 172.21.15.46:0/2262076408 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f11e8051a90 con 0x7f11f80cea30 2024-03-29T08:27:03.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.267+0000 7f11fd400640 1 -- 172.21.15.46:0/2262076408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11f80b1980 con 0x7f11f80cea30 2024-03-29T08:27:03.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.271+0000 7f11eea00640 1 -- 172.21.15.46:0/2262076408 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f11e8011340 con 0x7f11f80cea30 2024-03-29T08:27:03.272 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:27:03.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.271+0000 7f11fd400640 1 -- 172.21.15.46:0/2262076408 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11d4066690 msgr2=0x7f11d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.271+0000 7f11fd400640 1 --2- 172.21.15.46:0/2262076408 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11d4066690 0x7f11d4068b50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f11e00b2000 tx=0x7f11e0008040 comp rx=0 tx=0).stop 2024-03-29T08:27:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.271+0000 7f11fd400640 1 -- 172.21.15.46:0/2262076408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f80cea30 msgr2=0x7f11f8159190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.271+0000 7f11fd400640 1 --2- 172.21.15.46:0/2262076408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f80cea30 0x7f11f8159190 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f11e800d800 tx=0x7f11e800dcc0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.271+0000 7f11fd400640 1 -- 172.21.15.46:0/2262076408 shutdown_connections 2024-03-29T08:27:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.271+0000 7f11fd400640 1 --2- 172.21.15.46:0/2262076408 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11d4066690 0x7f11d4068b50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.271+0000 7f11fd400640 1 --2- 172.21.15.46:0/2262076408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f815cd30 0x7f11f815a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.271+0000 7f11fd400640 1 --2- 172.21.15.46:0/2262076408 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f11f815c950 0x7f11f809dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.271+0000 7f11fd400640 1 --2- 172.21.15.46:0/2262076408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11f80cea30 0x7f11f8159190 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.271+0000 7f11fd400640 1 -- 172.21.15.46:0/2262076408 >> 172.21.15.46:0/2262076408 conn(0x7f11f80b21b0 msgr2=0x7f11f80cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:03.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.271+0000 7f11fd400640 1 -- 172.21.15.46:0/2262076408 shutdown_connections 2024-03-29T08:27:03.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.271+0000 7f11fd400640 1 -- 172.21.15.46:0/2262076408 wait complete. 2024-03-29T08:27:03.293 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:27:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.431+0000 7f3086400640 1 Processor -- start 2024-03-29T08:27:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.431+0000 7f3086400640 1 -- start start 2024-03-29T08:27:03.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.431+0000 7f3086400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f30800af530 0x7f30800af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:03.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.431+0000 7f3086400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308006c390 con 0x7f3080158300 2024-03-29T08:27:03.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.431+0000 7f3086400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30800afff0 con 0x7f30800702a0 2024-03-29T08:27:03.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.431+0000 7f3086400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308006c5e0 con 0x7f30800af530 2024-03-29T08:27:03.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.435+0000 7f3077e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f30800af530 0x7f30800af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:03.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.435+0000 7f307f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3080158300 0x7f308006d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37862/0 (socket says 172.21.15.46:37862) 2024-03-29T08:27:03.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.435+0000 7f307f400640 1 -- 172.21.15.46:0/1951578894 learned_addr learned my addr 172.21.15.46:0/1951578894 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:03.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.435+0000 7f3077e00640 1 -- 172.21.15.46:0/1951578894 >> v1:172.21.15.73:6789/0 conn(0x7f30800702a0 legacy=0x7f308015bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:03.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.435+0000 7f3077e00640 1 -- 172.21.15.46:0/1951578894 >> v1:172.21.15.46:6789/0 conn(0x7f3080158300 legacy=0x7f308006d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:03.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.435+0000 7f3077e00640 1 -- 172.21.15.46:0/1951578894 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308006c8c0 con 0x7f30800af530 2024-03-29T08:27:03.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.435+0000 7f307ea00640 1 -- 172.21.15.46:0/1951578894 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1933199301 0 0) 0x7f308006c390 con 0x7f3080158300 2024-03-29T08:27:03.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.435+0000 7f3077e00640 1 --2- 172.21.15.46:0/1951578894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800af530 0x7f30800af9a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f30680097c0 tx=0x7f306801c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f97ae69804e827 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:03.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.435+0000 7f307ea00640 1 -- 172.21.15.46:0/1951578894 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30680ab600 con 0x7f30800af530 2024-03-29T08:27:03.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.435+0000 7f307ea00640 1 -- 172.21.15.46:0/1951578894 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30680a7050 con 0x7f30800af530 2024-03-29T08:27:03.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.435+0000 7f307ea00640 1 -- 172.21.15.46:0/1951578894 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30680ab980 con 0x7f30800af530 2024-03-29T08:27:03.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f3086400640 1 -- 172.21.15.46:0/1951578894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800af530 msgr2=0x7f30800af9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:03.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f3086400640 1 --2- 172.21.15.46:0/1951578894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800af530 0x7f30800af9a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f30680097c0 tx=0x7f306801c350 comp rx=0 tx=0).stop 2024-03-29T08:27:03.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f3086400640 1 -- 172.21.15.46:0/1951578894 shutdown_connections 2024-03-29T08:27:03.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f3086400640 1 --2- 172.21.15.46:0/1951578894 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800af530 0x7f30800af9a0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f3086400640 1 -- 172.21.15.46:0/1951578894 >> 172.21.15.46:0/1951578894 conn(0x7f3080066080 msgr2=0x7f3080069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:03.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f3086400640 1 -- 172.21.15.46:0/1951578894 shutdown_connections 2024-03-29T08:27:03.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f3086400640 1 -- 172.21.15.46:0/1951578894 wait complete. 2024-03-29T08:27:03.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f3086400640 1 Processor -- start 2024-03-29T08:27:03.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f3086400640 1 -- start start 2024-03-29T08:27:03.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f3086400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800702a0 0x7f30800a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:03.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f3086400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30800af530 0x7f30801570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:03.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f3086400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3080158300 0x7f3080164900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:03.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f3086400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30800a9e30 con 0x7f30800702a0 2024-03-29T08:27:03.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f3086400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3080157600 con 0x7f3080158300 2024-03-29T08:27:03.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f3086400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30800acbe0 con 0x7f30800af530 2024-03-29T08:27:03.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f307f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800702a0 0x7f30800a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:03.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f307f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800702a0 0x7f30800a8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41300/0 (socket says 172.21.15.46:41300) 2024-03-29T08:27:03.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f307f400640 1 -- 172.21.15.46:0/883522218 learned_addr learned my addr 172.21.15.46:0/883522218 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:03.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f307fe00640 1 --2- 172.21.15.46:0/883522218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3080158300 0x7f3080164900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:03.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f3077e00640 1 --2- 172.21.15.46:0/883522218 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30800af530 0x7f30801570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:03.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f307f400640 1 -- 172.21.15.46:0/883522218 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30800af530 msgr2=0x7f30801570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:03.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f307f400640 1 --2- 172.21.15.46:0/883522218 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30800af530 0x7f30801570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f307f400640 1 -- 172.21.15.46:0/883522218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3080158300 msgr2=0x7f3080164900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:03.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f307f400640 1 --2- 172.21.15.46:0/883522218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3080158300 0x7f3080164900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.439+0000 7f307f400640 1 -- 172.21.15.46:0/883522218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3068009420 con 0x7f30800702a0 2024-03-29T08:27:03.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.443+0000 7f307f400640 1 --2- 172.21.15.46:0/883522218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800702a0 0x7f30800a8c00 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f307000c440 tx=0x7f307000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:03.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.443+0000 7f3077400640 1 -- 172.21.15.46:0/883522218 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30700074e0 con 0x7f30800702a0 2024-03-29T08:27:03.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.443+0000 7f3077400640 1 -- 172.21.15.46:0/883522218 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3070007640 con 0x7f30800702a0 2024-03-29T08:27:03.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.443+0000 7f3086400640 1 -- 172.21.15.46:0/883522218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3080165da0 con 0x7f30800702a0 2024-03-29T08:27:03.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.443+0000 7f3077400640 1 -- 172.21.15.46:0/883522218 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070004a10 con 0x7f30800702a0 2024-03-29T08:27:03.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.443+0000 7f3086400640 1 -- 172.21.15.46:0/883522218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30800a9590 con 0x7f30800702a0 2024-03-29T08:27:03.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.443+0000 7f3086400640 1 -- 172.21.15.46:0/883522218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3080071a90 con 0x7f30800702a0 2024-03-29T08:27:03.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.443+0000 7f3077400640 1 -- 172.21.15.46:0/883522218 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3070004b70 con 0x7f30800702a0 2024-03-29T08:27:03.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.443+0000 7f3077400640 1 --2- 172.21.15.46:0/883522218 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f305c066690 0x7f305c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:03.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.443+0000 7f3077400640 1 -- 172.21.15.46:0/883522218 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f3070083570 con 0x7f30800702a0 2024-03-29T08:27:03.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.443+0000 7f3077e00640 1 --2- 172.21.15.46:0/883522218 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f305c066690 0x7f305c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:03.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.447+0000 7f3077e00640 1 --2- 172.21.15.46:0/883522218 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f305c066690 0x7f305c068b50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f30680ad000 tx=0x7f30680a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:03.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.447+0000 7f3077400640 1 -- 172.21.15.46:0/883522218 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3070050970 con 0x7f30800702a0 2024-03-29T08:27:03.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.619+0000 7f3086400640 1 -- 172.21.15.46:0/883522218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f308004e120 con 0x7f30800702a0 2024-03-29T08:27:03.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.619+0000 7f3077400640 1 -- 172.21.15.46:0/883522218 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v94) v1 ==== 80+0+244 (secure 0 0 0) 0x7f3070018110 con 0x7f30800702a0 2024-03-29T08:27:03.622 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6835/2051429508 2024-03-30T08:26:22.851926+0000 2024-03-29T08:27:03.622 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6833/2051429508 2024-03-30T08:26:22.851926+0000 2024-03-29T08:27:03.622 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.73:6836/4166845858 2024-03-30T08:25:02.334350+0000 2024-03-29T08:27:03.622 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.73:6834/4166845858 2024-03-30T08:25:02.334350+0000 2024-03-29T08:27:03.623 INFO:teuthology.orchestra.run.smithi046.stderr:listed 4 entries 2024-03-29T08:27:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.623+0000 7f3086400640 1 -- 172.21.15.46:0/883522218 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f305c066690 msgr2=0x7f305c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.623+0000 7f3086400640 1 --2- 172.21.15.46:0/883522218 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f305c066690 0x7f305c068b50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f30680ad000 tx=0x7f30680a6000 comp rx=0 tx=0).stop 2024-03-29T08:27:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.623+0000 7f3086400640 1 -- 172.21.15.46:0/883522218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800702a0 msgr2=0x7f30800a8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.623+0000 7f3086400640 1 --2- 172.21.15.46:0/883522218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800702a0 0x7f30800a8c00 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f307000c440 tx=0x7f307000c900 comp rx=0 tx=0).stop 2024-03-29T08:27:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.623+0000 7f3086400640 1 -- 172.21.15.46:0/883522218 shutdown_connections 2024-03-29T08:27:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.623+0000 7f3086400640 1 --2- 172.21.15.46:0/883522218 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f305c066690 0x7f305c068b50 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.623+0000 7f3086400640 1 --2- 172.21.15.46:0/883522218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3080158300 0x7f3080164900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.623+0000 7f3086400640 1 --2- 172.21.15.46:0/883522218 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30800af530 0x7f30801570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.623+0000 7f3086400640 1 --2- 172.21.15.46:0/883522218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30800702a0 0x7f30800a8c00 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:03.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.623+0000 7f3086400640 1 -- 172.21.15.46:0/883522218 >> 172.21.15.46:0/883522218 conn(0x7f3080066080 msgr2=0x7f3080069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:03.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.623+0000 7f3086400640 1 -- 172.21.15.46:0/883522218 shutdown_connections 2024-03-29T08:27:03.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:03.623+0000 7f3086400640 1 -- 172.21.15.46:0/883522218 wait complete. 2024-03-29T08:27:03.643 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-03-29T08:27:03.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:03.643 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T08:27:03.661 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[46501]: fuse finished with error 0 and tester_r 0 2024-03-29T08:27:03.664 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:27:03.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:03.664 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:27:03.768 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:27:03.768 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:27:03.778 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:27:03.779 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:27:03.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:03.779 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T08:27:03.831 INFO:teuthology.orchestra.run.smithi073.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-29T08:27:03.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:27:03.832 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:27:03.832 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:27:03.841 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:27:03.841 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:27:03.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:03.841 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-29T08:27:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-29T08:27:03.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:27:03.892 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:27:03.892 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:27:03.901 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:27:03.901 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:27:03.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:03.901 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-29T08:27:03.951 INFO:teuthology.orchestra.run.smithi073.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-29T08:27:03.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:27:03.952 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:27:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.091+0000 7f12a0800640 1 Processor -- start 2024-03-29T08:27:04.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.091+0000 7f12a0800640 1 -- start start 2024-03-29T08:27:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.091+0000 7f12a0800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f129815c9d0 0x7f12980cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.091+0000 7f12a0800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12980cef60 0x7f1298098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.091+0000 7f12a0800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12980d0030 con 0x7f12980ccd10 2024-03-29T08:27:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.091+0000 7f12a0800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12980cf420 con 0x7f129815c9d0 2024-03-29T08:27:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.091+0000 7f12a0800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12980cf700 con 0x7f12980cef60 2024-03-29T08:27:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.091+0000 7f129de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f129815c9d0 0x7f12980cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.095+0000 7f129de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f129815c9d0 0x7f12980cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41314/0 (socket says 172.21.15.46:41314) 2024-03-29T08:27:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.095+0000 7f129de00640 1 -- 172.21.15.46:0/30513189 learned_addr learned my addr 172.21.15.46:0/30513189 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.095+0000 7f129d400640 1 --2- 172.21.15.46:0/30513189 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12980cef60 0x7f1298098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.095+0000 7f129de00640 1 -- 172.21.15.46:0/30513189 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12980cef60 msgr2=0x7f1298098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.095+0000 7f129de00640 1 --2- 172.21.15.46:0/30513189 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12980cef60 0x7f1298098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.095+0000 7f129de00640 1 -- 172.21.15.46:0/30513189 >> v1:172.21.15.73:6789/0 conn(0x7f12980ccd10 legacy=0x7f1298160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.095+0000 7f129de00640 1 -- 172.21.15.46:0/30513189 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12980cf9e0 con 0x7f129815c9d0 2024-03-29T08:27:04.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.095+0000 7f129de00640 1 --2- 172.21.15.46:0/30513189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129815c9d0 0x7f12980cea20 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f128c009870 tx=0x7f128c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1d7a53e2212461 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:04.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.095+0000 7f129ca00640 1 -- 172.21.15.46:0/30513189 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c0a9610 con 0x7f129815c9d0 2024-03-29T08:27:04.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.095+0000 7f129ca00640 1 -- 172.21.15.46:0/30513189 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128c0a9770 con 0x7f129815c9d0 2024-03-29T08:27:04.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.095+0000 7f129ca00640 1 -- 172.21.15.46:0/30513189 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c0a9b00 con 0x7f129815c9d0 2024-03-29T08:27:04.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.095+0000 7f12a0800640 1 -- 172.21.15.46:0/30513189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129815c9d0 msgr2=0x7f12980cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.095+0000 7f12a0800640 1 --2- 172.21.15.46:0/30513189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129815c9d0 0x7f12980cea20 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f128c009870 tx=0x7f128c01c530 comp rx=0 tx=0).stop 2024-03-29T08:27:04.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f12a0800640 1 -- 172.21.15.46:0/30513189 shutdown_connections 2024-03-29T08:27:04.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f12a0800640 1 --2- 172.21.15.46:0/30513189 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12980cef60 0x7f1298098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f12a0800640 1 --2- 172.21.15.46:0/30513189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129815c9d0 0x7f12980cea20 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f12a0800640 1 -- 172.21.15.46:0/30513189 >> 172.21.15.46:0/30513189 conn(0x7f12980b21a0 msgr2=0x7f12980b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:04.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f12a0800640 1 -- 172.21.15.46:0/30513189 shutdown_connections 2024-03-29T08:27:04.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f12a0800640 1 -- 172.21.15.46:0/30513189 wait complete. 2024-03-29T08:27:04.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f12a0800640 1 Processor -- start 2024-03-29T08:27:04.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f12a0800640 1 -- start start 2024-03-29T08:27:04.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f12a0800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12980ccd10 0x7f1298145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f12a0800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12980cef60 0x7f12980c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f12a0800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129815c9d0 0x7f12980c5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f12a0800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129806fe40 con 0x7f129815c9d0 2024-03-29T08:27:04.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f12a0800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12980c8140 con 0x7f12980cef60 2024-03-29T08:27:04.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f12a0800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12980c83e0 con 0x7f12980ccd10 2024-03-29T08:27:04.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f129de00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12980ccd10 0x7f1298145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f129e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129815c9d0 0x7f12980c5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f129de00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12980ccd10 0x7f1298145b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:35224/0 (socket says 172.21.15.46:35224) 2024-03-29T08:27:04.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f129de00640 1 -- 172.21.15.46:0/3064606134 learned_addr learned my addr 172.21.15.46:0/3064606134 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:04.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f129d400640 1 --2- 172.21.15.46:0/3064606134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12980cef60 0x7f12980c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f129e800640 1 -- 172.21.15.46:0/3064606134 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12980ccd10 msgr2=0x7f1298145b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f129e800640 1 --2- 172.21.15.46:0/3064606134 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12980ccd10 0x7f1298145b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f129e800640 1 -- 172.21.15.46:0/3064606134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12980cef60 msgr2=0x7f12980c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f129e800640 1 --2- 172.21.15.46:0/3064606134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12980cef60 0x7f12980c4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f129e800640 1 -- 172.21.15.46:0/3064606134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c0094d0 con 0x7f129815c9d0 2024-03-29T08:27:04.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.099+0000 7f129e800640 1 --2- 172.21.15.46:0/3064606134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129815c9d0 0x7f12980c5190 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f129400c700 tx=0x7f129400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:04.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.103+0000 7f1292000640 1 -- 172.21.15.46:0/3064606134 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294003b80 con 0x7f129815c9d0 2024-03-29T08:27:04.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.103+0000 7f1292000640 1 -- 172.21.15.46:0/3064606134 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1294003ce0 con 0x7f129815c9d0 2024-03-29T08:27:04.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.103+0000 7f12a0800640 1 -- 172.21.15.46:0/3064606134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12980c86c0 con 0x7f129815c9d0 2024-03-29T08:27:04.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.103+0000 7f1292000640 1 -- 172.21.15.46:0/3064606134 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1294005310 con 0x7f129815c9d0 2024-03-29T08:27:04.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.103+0000 7f12a0800640 1 -- 172.21.15.46:0/3064606134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12980c7170 con 0x7f129815c9d0 2024-03-29T08:27:04.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.103+0000 7f12a0800640 1 -- 172.21.15.46:0/3064606134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1298066c90 con 0x7f129815c9d0 2024-03-29T08:27:04.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.103+0000 7f1292000640 1 -- 172.21.15.46:0/3064606134 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1294005470 con 0x7f129815c9d0 2024-03-29T08:27:04.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.103+0000 7f1292000640 1 --2- 172.21.15.46:0/3064606134 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f127c066690 0x7f127c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.103+0000 7f1292000640 1 -- 172.21.15.46:0/3064606134 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f1294083e00 con 0x7f129815c9d0 2024-03-29T08:27:04.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.103+0000 7f129de00640 1 --2- 172.21.15.46:0/3064606134 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f127c066690 0x7f127c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.107+0000 7f129de00640 1 --2- 172.21.15.46:0/3064606134 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f127c066690 0x7f127c068b50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f128c009870 tx=0x7f128c0b3040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:04.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.107+0000 7f1292000640 1 -- 172.21.15.46:0/3064606134 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1294051160 con 0x7f129815c9d0 2024-03-29T08:27:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.303+0000 7f12a0800640 1 -- 172.21.15.46:0/3064606134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f12980c8b70 con 0x7f129815c9d0 2024-03-29T08:27:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.303+0000 7f1292000640 1 -- 172.21.15.46:0/3064606134 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f1294054780 con 0x7f129815c9d0 2024-03-29T08:27:04.306 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:04.306 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/1862652020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1862652020},{"type":"v1","addr":"172.21.15.73:6836","nonce":1862652020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:24:43.476334+0000","modified":"2024-03-29T08:26:26.873752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7986":{"gid":7986,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1886680593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1886680593},{"type":"v1","addr":"172.21.15.46:6835","nonce":1886680593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-29T08:27:04.306 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 69 2024-03-29T08:27:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.303+0000 7f12a0800640 1 -- 172.21.15.46:0/3064606134 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f127c066690 msgr2=0x7f127c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.303+0000 7f12a0800640 1 --2- 172.21.15.46:0/3064606134 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f127c066690 0x7f127c068b50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f128c009870 tx=0x7f128c0b3040 comp rx=0 tx=0).stop 2024-03-29T08:27:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.303+0000 7f12a0800640 1 -- 172.21.15.46:0/3064606134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129815c9d0 msgr2=0x7f12980c5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.303+0000 7f12a0800640 1 --2- 172.21.15.46:0/3064606134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129815c9d0 0x7f12980c5190 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f129400c700 tx=0x7f129400cbc0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.303+0000 7f12a0800640 1 -- 172.21.15.46:0/3064606134 shutdown_connections 2024-03-29T08:27:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.303+0000 7f12a0800640 1 --2- 172.21.15.46:0/3064606134 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f127c066690 0x7f127c068b50 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.303+0000 7f12a0800640 1 --2- 172.21.15.46:0/3064606134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12980cef60 0x7f12980c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.303+0000 7f12a0800640 1 --2- 172.21.15.46:0/3064606134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f129815c9d0 0x7f12980c5190 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.303+0000 7f12a0800640 1 --2- 172.21.15.46:0/3064606134 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12980ccd10 0x7f1298145b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.303+0000 7f12a0800640 1 -- 172.21.15.46:0/3064606134 >> 172.21.15.46:0/3064606134 conn(0x7f12980b21a0 msgr2=0x7f12980cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.307+0000 7f12a0800640 1 -- 172.21.15.46:0/3064606134 shutdown_connections 2024-03-29T08:27:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.307+0000 7f12a0800640 1 -- 172.21.15.46:0/3064606134 wait complete. 2024-03-29T08:27:04.327 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:27:04.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.463+0000 7f2281400640 1 Processor -- start 2024-03-29T08:27:04.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.463+0000 7f2281400640 1 -- start start 2024-03-29T08:27:04.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.467+0000 7f2281400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f227c1569a0 0x7f227c06c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.467+0000 7f2281400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f227c0708d0 0x7f227c1596c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.467+0000 7f2281400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f227c15d7c0 con 0x7f227c06d910 2024-03-29T08:27:04.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.467+0000 7f2281400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f227c0b21d0 con 0x7f227c0708d0 2024-03-29T08:27:04.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.467+0000 7f2281400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227c15cc40 con 0x7f227c1569a0 2024-03-29T08:27:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.467+0000 7f227a800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f227c1569a0 0x7f227c06c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.467+0000 7f227b200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f227c0708d0 0x7f227c1596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.467+0000 7f2271e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f227c06d910 0x7f227c0701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:57128/0 (socket says 172.21.15.46:57128) 2024-03-29T08:27:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.467+0000 7f2271e00640 1 -- 172.21.15.46:0/3048269998 learned_addr learned my addr 172.21.15.46:0/3048269998 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.467+0000 7f227a800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f227c1569a0 0x7f227c06c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:35238/0 (socket says 172.21.15.46:35238) 2024-03-29T08:27:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.467+0000 7f227b200640 1 -- 172.21.15.46:0/3048269998 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f227c1569a0 msgr2=0x7f227c06c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.467+0000 7f227b200640 1 --2- 172.21.15.46:0/3048269998 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f227c1569a0 0x7f227c06c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.467+0000 7f227b200640 1 -- 172.21.15.46:0/3048269998 >> v1:172.21.15.73:6789/0 conn(0x7f227c06d910 legacy=0x7f227c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.467+0000 7f227b200640 1 -- 172.21.15.46:0/3048269998 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227c15cf20 con 0x7f227c0708d0 2024-03-29T08:27:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.467+0000 7f2279e00640 1 -- 172.21.15.46:0/3048269998 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3273098428 0 0) 0x7f227c15d7c0 con 0x7f227c06d910 2024-03-29T08:27:04.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.471+0000 7f227b200640 1 --2- 172.21.15.46:0/3048269998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f227c0708d0 0x7f227c1596c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2274009d90 tx=0x7f227401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1176c77cd46c3d0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:04.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.471+0000 7f2279e00640 1 -- 172.21.15.46:0/3048269998 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22740a5810 con 0x7f227c0708d0 2024-03-29T08:27:04.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.471+0000 7f2279e00640 1 -- 172.21.15.46:0/3048269998 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22740a5970 con 0x7f227c0708d0 2024-03-29T08:27:04.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.471+0000 7f2279e00640 1 -- 172.21.15.46:0/3048269998 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22740a5d00 con 0x7f227c0708d0 2024-03-29T08:27:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.471+0000 7f2281400640 1 -- 172.21.15.46:0/3048269998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f227c0708d0 msgr2=0x7f227c1596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.471+0000 7f2281400640 1 --2- 172.21.15.46:0/3048269998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f227c0708d0 0x7f227c1596c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2274009d90 tx=0x7f227401c970 comp rx=0 tx=0).stop 2024-03-29T08:27:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.471+0000 7f2281400640 1 -- 172.21.15.46:0/3048269998 shutdown_connections 2024-03-29T08:27:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.471+0000 7f2281400640 1 --2- 172.21.15.46:0/3048269998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f227c0708d0 0x7f227c1596c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.471+0000 7f2281400640 1 --2- 172.21.15.46:0/3048269998 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f227c1569a0 0x7f227c06c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.471+0000 7f2281400640 1 -- 172.21.15.46:0/3048269998 >> 172.21.15.46:0/3048269998 conn(0x7f227c066110 msgr2=0x7f227c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.471+0000 7f2281400640 1 -- 172.21.15.46:0/3048269998 shutdown_connections 2024-03-29T08:27:04.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.471+0000 7f2281400640 1 -- 172.21.15.46:0/3048269998 wait complete. 2024-03-29T08:27:04.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.471+0000 7f2281400640 1 Processor -- start 2024-03-29T08:27:04.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.471+0000 7f2281400640 1 -- start start 2024-03-29T08:27:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f2281400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f227c06d910 0x7f227c13cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f2281400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f227c0708d0 0x7f227c14eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f2281400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227c1569a0 0x7f227c14f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f2281400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c150c00 con 0x7f227c1569a0 2024-03-29T08:27:04.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f2281400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227c13d4c0 con 0x7f227c06d910 2024-03-29T08:27:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f2281400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227c150770 con 0x7f227c0708d0 2024-03-29T08:27:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f227b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227c1569a0 0x7f227c14f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f227b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227c1569a0 0x7f227c14f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41328/0 (socket says 172.21.15.46:41328) 2024-03-29T08:27:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f227b200640 1 -- 172.21.15.46:0/950500658 learned_addr learned my addr 172.21.15.46:0/950500658 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f2271e00640 1 --2- 172.21.15.46:0/950500658 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f227c0708d0 0x7f227c14eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f227a800640 1 --2- 172.21.15.46:0/950500658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f227c06d910 0x7f227c13cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f227b200640 1 -- 172.21.15.46:0/950500658 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f227c0708d0 msgr2=0x7f227c14eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f227b200640 1 --2- 172.21.15.46:0/950500658 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f227c0708d0 0x7f227c14eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f227b200640 1 -- 172.21.15.46:0/950500658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f227c06d910 msgr2=0x7f227c13cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f227b200640 1 --2- 172.21.15.46:0/950500658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f227c06d910 0x7f227c13cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f227b200640 1 -- 172.21.15.46:0/950500658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2274009a70 con 0x7f227c1569a0 2024-03-29T08:27:04.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f227b200640 1 --2- 172.21.15.46:0/950500658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227c1569a0 0x7f227c14f530 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f22740b0040 tx=0x7f2274019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:04.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f2273400640 1 -- 172.21.15.46:0/950500658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22740b9860 con 0x7f227c1569a0 2024-03-29T08:27:04.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f2273400640 1 -- 172.21.15.46:0/950500658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22740b99c0 con 0x7f227c1569a0 2024-03-29T08:27:04.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f2281400640 1 -- 172.21.15.46:0/950500658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227c14fb60 con 0x7f227c1569a0 2024-03-29T08:27:04.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f2273400640 1 -- 172.21.15.46:0/950500658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22740b9d70 con 0x7f227c1569a0 2024-03-29T08:27:04.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f2281400640 1 -- 172.21.15.46:0/950500658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227c150120 con 0x7f227c1569a0 2024-03-29T08:27:04.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f2281400640 1 -- 172.21.15.46:0/950500658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227c150900 con 0x7f227c1569a0 2024-03-29T08:27:04.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.475+0000 7f2273400640 1 -- 172.21.15.46:0/950500658 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f22740c4460 con 0x7f227c1569a0 2024-03-29T08:27:04.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.479+0000 7f2273400640 1 --2- 172.21.15.46:0/950500658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2258066690 0x7f2258068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.479+0000 7f2273400640 1 -- 172.21.15.46:0/950500658 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f227411f4c0 con 0x7f227c1569a0 2024-03-29T08:27:04.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.479+0000 7f227a800640 1 --2- 172.21.15.46:0/950500658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2258066690 0x7f2258068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.479+0000 7f227a800640 1 --2- 172.21.15.46:0/950500658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2258066690 0x7f2258068b50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f226c009a50 tx=0x7f226c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:04.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.483+0000 7f2273400640 1 -- 172.21.15.46:0/950500658 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f22740ec800 con 0x7f227c1569a0 2024-03-29T08:27:04.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.643+0000 7f2281400640 1 -- 172.21.15.46:0/950500658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f227c066c90 con 0x7f227c1569a0 2024-03-29T08:27:04.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.647+0000 7f2273400640 1 -- 172.21.15.46:0/950500658 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16315 (secure 0 0 0) 0x7f22740efe20 con 0x7f227c1569a0 2024-03-29T08:27:04.649 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:04.649 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:26:22.851956+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:24:41.113618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-29T08:24:42.487380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/2051429508":"2024-03-30T08:26:22.851926+0000","172.21.15.46:6833/2051429508":"2024-03-30T08:26:22.851926+0000","172.21.15.73:6836/4166845858":"2024-03-30T08:25:02.334350+0000","172.21.15.73:6834/4166845858":"2024-03-30T08:25:02.334350+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:27:04.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.647+0000 7f2281400640 1 -- 172.21.15.46:0/950500658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2258066690 msgr2=0x7f2258068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.647+0000 7f2281400640 1 --2- 172.21.15.46:0/950500658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2258066690 0x7f2258068b50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f226c009a50 tx=0x7f226c008040 comp rx=0 tx=0).stop 2024-03-29T08:27:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.647+0000 7f2281400640 1 -- 172.21.15.46:0/950500658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227c1569a0 msgr2=0x7f227c14f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.647+0000 7f2281400640 1 --2- 172.21.15.46:0/950500658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227c1569a0 0x7f227c14f530 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f22740b0040 tx=0x7f2274019710 comp rx=0 tx=0).stop 2024-03-29T08:27:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.647+0000 7f2281400640 1 -- 172.21.15.46:0/950500658 shutdown_connections 2024-03-29T08:27:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.647+0000 7f2281400640 1 --2- 172.21.15.46:0/950500658 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2258066690 0x7f2258068b50 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.647+0000 7f2281400640 1 --2- 172.21.15.46:0/950500658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f227c1569a0 0x7f227c14f530 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.647+0000 7f2281400640 1 --2- 172.21.15.46:0/950500658 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f227c0708d0 0x7f227c14eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.647+0000 7f2281400640 1 --2- 172.21.15.46:0/950500658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f227c06d910 0x7f227c13cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.647+0000 7f2281400640 1 -- 172.21.15.46:0/950500658 >> 172.21.15.46:0/950500658 conn(0x7f227c066110 msgr2=0x7f227c0701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.647+0000 7f2281400640 1 -- 172.21.15.46:0/950500658 shutdown_connections 2024-03-29T08:27:04.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.647+0000 7f2281400640 1 -- 172.21.15.46:0/950500658 wait complete. 2024-03-29T08:27:04.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:24:41.113618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:24:42.487380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:27:04.670 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:27:04.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.811+0000 7fbc8bc00640 1 Processor -- start 2024-03-29T08:27:04.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.811+0000 7fbc8bc00640 1 -- start start 2024-03-29T08:27:04.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.811+0000 7fbc8bc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc8415df90 0x7fbc840ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.811+0000 7fbc8bc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbc840cbea0 0x7fbc840ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.811+0000 7fbc8bc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc8415d260 con 0x7fbc84142a50 2024-03-29T08:27:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.811+0000 7fbc8bc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc840aed00 con 0x7fbc8415df90 2024-03-29T08:27:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.811+0000 7fbc8bc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc840aefd0 con 0x7fbc840cbea0 2024-03-29T08:27:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.811+0000 7fbc7be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc8415df90 0x7fbc840ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.811+0000 7fbc89200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbc84142a50 0x7fbc840cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37874/0 (socket says 172.21.15.46:37874) 2024-03-29T08:27:04.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.811+0000 7fbc89200640 1 -- 172.21.15.46:0/3274085744 learned_addr learned my addr 172.21.15.46:0/3274085744 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.811+0000 7fbc89c00640 1 --2- 172.21.15.46:0/3274085744 >> v2:172.21.15.73:3300/0 conn(0x7fbc840cbea0 0x7fbc840ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.811+0000 7fbc7be00640 1 -- 172.21.15.46:0/3274085744 >> v2:172.21.15.73:3300/0 conn(0x7fbc840cbea0 msgr2=0x7fbc840ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.811+0000 7fbc7be00640 1 --2- 172.21.15.46:0/3274085744 >> v2:172.21.15.73:3300/0 conn(0x7fbc840cbea0 0x7fbc840ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.811+0000 7fbc7be00640 1 -- 172.21.15.46:0/3274085744 >> v1:172.21.15.46:6789/0 conn(0x7fbc84142a50 legacy=0x7fbc840cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.811+0000 7fbc7be00640 1 -- 172.21.15.46:0/3274085744 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8415d4b0 con 0x7fbc8415df90 2024-03-29T08:27:04.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.811+0000 7fbc83e00640 1 -- 172.21.15.46:0/3274085744 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1512056318 0 0) 0x7fbc8415d260 con 0x7fbc84142a50 2024-03-29T08:27:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.815+0000 7fbc7be00640 1 --2- 172.21.15.46:0/3274085744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8415df90 0x7fbc840ae280 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fbc8406e4e0 tx=0x7fbc6c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce70bc595d225b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.815+0000 7fbc83e00640 1 -- 172.21.15.46:0/3274085744 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0ab600 con 0x7fbc8415df90 2024-03-29T08:27:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.815+0000 7fbc83e00640 1 -- 172.21.15.46:0/3274085744 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6c0a7050 con 0x7fbc8415df90 2024-03-29T08:27:04.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.815+0000 7fbc83e00640 1 -- 172.21.15.46:0/3274085744 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6c0ab980 con 0x7fbc8415df90 2024-03-29T08:27:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.815+0000 7fbc8bc00640 1 -- 172.21.15.46:0/3274085744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8415df90 msgr2=0x7fbc840ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.815+0000 7fbc8bc00640 1 --2- 172.21.15.46:0/3274085744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8415df90 0x7fbc840ae280 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fbc8406e4e0 tx=0x7fbc6c01c3f0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.815+0000 7fbc8bc00640 1 -- 172.21.15.46:0/3274085744 shutdown_connections 2024-03-29T08:27:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.815+0000 7fbc8bc00640 1 --2- 172.21.15.46:0/3274085744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc8415df90 0x7fbc840ae280 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.815+0000 7fbc8bc00640 1 --2- 172.21.15.46:0/3274085744 >> v2:172.21.15.73:3300/0 conn(0x7fbc840cbea0 0x7fbc840ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.815+0000 7fbc8bc00640 1 -- 172.21.15.46:0/3274085744 >> 172.21.15.46:0/3274085744 conn(0x7fbc8406a390 msgr2=0x7fbc840cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:04.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.815+0000 7fbc8bc00640 1 -- 172.21.15.46:0/3274085744 shutdown_connections 2024-03-29T08:27:04.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.815+0000 7fbc8bc00640 1 -- 172.21.15.46:0/3274085744 wait complete. 2024-03-29T08:27:04.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc8bc00640 1 Processor -- start 2024-03-29T08:27:04.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc8bc00640 1 -- start start 2024-03-29T08:27:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc8bc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc840cbea0 0x7fbc8413f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc8bc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc84142a50 0x7fbc841412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc8bc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc8415df90 0x7fbc840a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc8bc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc84141820 con 0x7fbc840cbea0 2024-03-29T08:27:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc8bc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8413fb00 con 0x7fbc84142a50 2024-03-29T08:27:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc8bc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8413fde0 con 0x7fbc8415df90 2024-03-29T08:27:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc89200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc840cbea0 0x7fbc8413f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc89200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc840cbea0 0x7fbc8413f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41352/0 (socket says 172.21.15.46:41352) 2024-03-29T08:27:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc89200640 1 -- 172.21.15.46:0/2625232426 learned_addr learned my addr 172.21.15.46:0/2625232426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc7be00640 1 --2- 172.21.15.46:0/2625232426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc84142a50 0x7fbc841412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc89c00640 1 --2- 172.21.15.46:0/2625232426 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc8415df90 0x7fbc840a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc89200640 1 -- 172.21.15.46:0/2625232426 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc8415df90 msgr2=0x7fbc840a8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc89200640 1 --2- 172.21.15.46:0/2625232426 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc8415df90 0x7fbc840a8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc89200640 1 -- 172.21.15.46:0/2625232426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc84142a50 msgr2=0x7fbc841412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc89200640 1 --2- 172.21.15.46:0/2625232426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc84142a50 0x7fbc841412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc89200640 1 -- 172.21.15.46:0/2625232426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c009420 con 0x7fbc840cbea0 2024-03-29T08:27:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc7be00640 1 --2- 172.21.15.46:0/2625232426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc84142a50 0x7fbc841412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc89c00640 1 --2- 172.21.15.46:0/2625232426 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc8415df90 0x7fbc840a8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc89200640 1 --2- 172.21.15.46:0/2625232426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc840cbea0 0x7fbc8413f4e0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fbc740011f0 tx=0x7fbc7400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc81600640 1 -- 172.21.15.46:0/2625232426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc740074e0 con 0x7fbc840cbea0 2024-03-29T08:27:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc81600640 1 -- 172.21.15.46:0/2625232426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc74007640 con 0x7fbc840cbea0 2024-03-29T08:27:04.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc81600640 1 -- 172.21.15.46:0/2625232426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc74004a10 con 0x7fbc840cbea0 2024-03-29T08:27:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc8bc00640 1 -- 172.21.15.46:0/2625232426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc841400c0 con 0x7fbc840cbea0 2024-03-29T08:27:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.819+0000 7fbc8bc00640 1 -- 172.21.15.46:0/2625232426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc840aac60 con 0x7fbc840cbea0 2024-03-29T08:27:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.823+0000 7fbc8bc00640 1 -- 172.21.15.46:0/2625232426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8406ad60 con 0x7fbc840cbea0 2024-03-29T08:27:04.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.823+0000 7fbc81600640 1 -- 172.21.15.46:0/2625232426 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbc74004b70 con 0x7fbc840cbea0 2024-03-29T08:27:04.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.823+0000 7fbc81600640 1 --2- 172.21.15.46:0/2625232426 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc60066690 0x7fbc60068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:04.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.823+0000 7fbc81600640 1 -- 172.21.15.46:0/2625232426 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fbc74083530 con 0x7fbc840cbea0 2024-03-29T08:27:04.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.823+0000 7fbc7be00640 1 --2- 172.21.15.46:0/2625232426 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc60066690 0x7fbc60068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:04.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.823+0000 7fbc7be00640 1 --2- 172.21.15.46:0/2625232426 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc60066690 0x7fbc60068b50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0ad000 tx=0x7fbc6c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:04.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.827+0000 7fbc81600640 1 -- 172.21.15.46:0/2625232426 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbc740507e0 con 0x7fbc840cbea0 2024-03-29T08:27:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.987+0000 7fbc8bc00640 1 -- 172.21.15.46:0/2625232426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbc84071a70 con 0x7fbc840cbea0 2024-03-29T08:27:04.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.987+0000 7fbc81600640 1 -- 172.21.15.46:0/2625232426 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbc740180a0 con 0x7fbc840cbea0 2024-03-29T08:27:04.990 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:27:04.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.991+0000 7fbc8bc00640 1 -- 172.21.15.46:0/2625232426 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc60066690 msgr2=0x7fbc60068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.991+0000 7fbc8bc00640 1 --2- 172.21.15.46:0/2625232426 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc60066690 0x7fbc60068b50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0ad000 tx=0x7fbc6c008040 comp rx=0 tx=0).stop 2024-03-29T08:27:04.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.991+0000 7fbc8bc00640 1 -- 172.21.15.46:0/2625232426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc840cbea0 msgr2=0x7fbc8413f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.991+0000 7fbc8bc00640 1 --2- 172.21.15.46:0/2625232426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc840cbea0 0x7fbc8413f4e0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fbc740011f0 tx=0x7fbc7400c9b0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.991+0000 7fbc8bc00640 1 -- 172.21.15.46:0/2625232426 shutdown_connections 2024-03-29T08:27:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.991+0000 7fbc8bc00640 1 --2- 172.21.15.46:0/2625232426 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbc60066690 0x7fbc60068b50 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.991+0000 7fbc8bc00640 1 --2- 172.21.15.46:0/2625232426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbc84142a50 0x7fbc841412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.991+0000 7fbc8bc00640 1 --2- 172.21.15.46:0/2625232426 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbc8415df90 0x7fbc840a8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.991+0000 7fbc8bc00640 1 --2- 172.21.15.46:0/2625232426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc840cbea0 0x7fbc8413f4e0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.991+0000 7fbc8bc00640 1 -- 172.21.15.46:0/2625232426 >> 172.21.15.46:0/2625232426 conn(0x7fbc8406a390 msgr2=0x7fbc8406c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.991+0000 7fbc8bc00640 1 -- 172.21.15.46:0/2625232426 shutdown_connections 2024-03-29T08:27:04.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:04.991+0000 7fbc8bc00640 1 -- 172.21.15.46:0/2625232426 wait complete. 2024-03-29T08:27:05.011 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:27:05.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.151+0000 7f68cbe00640 1 Processor -- start 2024-03-29T08:27:05.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.151+0000 7f68cbe00640 1 -- start start 2024-03-29T08:27:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.155+0000 7f68cbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68c4143eb0 0x7f68c40cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.155+0000 7f68cbe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f68c415efe0 0x7f68c40cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.155+0000 7f68cbe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c40b2360 con 0x7f68c40b1630 2024-03-29T08:27:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.155+0000 7f68cbe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c415e500 con 0x7f68c4143eb0 2024-03-29T08:27:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.155+0000 7f68cbe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c415e7e0 con 0x7f68c415efe0 2024-03-29T08:27:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.155+0000 7f68c9400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68c4143eb0 0x7f68c40cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.155+0000 7f68c9e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f68c40b1630 0x7f68c40b1a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37884/0 (socket says 172.21.15.46:37884) 2024-03-29T08:27:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.155+0000 7f68c9e00640 1 -- 172.21.15.46:0/4081984684 learned_addr learned my addr 172.21.15.46:0/4081984684 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.155+0000 7f68c8a00640 1 --2- 172.21.15.46:0/4081984684 >> v2:172.21.15.73:3300/0 conn(0x7f68c415efe0 0x7f68c40cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:05.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.155+0000 7f68c9400640 1 -- 172.21.15.46:0/4081984684 >> v2:172.21.15.73:3300/0 conn(0x7f68c415efe0 msgr2=0x7f68c40cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.155+0000 7f68c9400640 1 --2- 172.21.15.46:0/4081984684 >> v2:172.21.15.73:3300/0 conn(0x7f68c415efe0 0x7f68c40cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.155+0000 7f68c9400640 1 -- 172.21.15.46:0/4081984684 >> v1:172.21.15.46:6789/0 conn(0x7f68c40b1630 legacy=0x7f68c40b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:05.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.155+0000 7f68c9400640 1 -- 172.21.15.46:0/4081984684 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c415eac0 con 0x7f68c4143eb0 2024-03-29T08:27:05.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68c9400640 1 --2- 172.21.15.46:0/4081984684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c4143eb0 0x7f68c40cb790 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f68c4158930 tx=0x7f68b801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d01eaa19dd347c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:05.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68c3e00640 1 -- 172.21.15.46:0/4081984684 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b80a8050 con 0x7f68c4143eb0 2024-03-29T08:27:05.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68c3e00640 1 -- 172.21.15.46:0/4081984684 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b80ac5f0 con 0x7f68c4143eb0 2024-03-29T08:27:05.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68c3e00640 1 -- 172.21.15.46:0/4081984684 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b80ac980 con 0x7f68c4143eb0 2024-03-29T08:27:05.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68cbe00640 1 -- 172.21.15.46:0/4081984684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c4143eb0 msgr2=0x7f68c40cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:05.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68cbe00640 1 --2- 172.21.15.46:0/4081984684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c4143eb0 0x7f68c40cb790 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f68c4158930 tx=0x7f68b801c480 comp rx=0 tx=0).stop 2024-03-29T08:27:05.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68cbe00640 1 -- 172.21.15.46:0/4081984684 shutdown_connections 2024-03-29T08:27:05.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68cbe00640 1 --2- 172.21.15.46:0/4081984684 >> v2:172.21.15.73:3300/0 conn(0x7f68c415efe0 0x7f68c40cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68cbe00640 1 --2- 172.21.15.46:0/4081984684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c4143eb0 0x7f68c40cb790 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68cbe00640 1 -- 172.21.15.46:0/4081984684 >> 172.21.15.46:0/4081984684 conn(0x7f68c406a390 msgr2=0x7f68c40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:05.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68cbe00640 1 -- 172.21.15.46:0/4081984684 shutdown_connections 2024-03-29T08:27:05.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68cbe00640 1 -- 172.21.15.46:0/4081984684 wait complete. 2024-03-29T08:27:05.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68cbe00640 1 Processor -- start 2024-03-29T08:27:05.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68cbe00640 1 -- start start 2024-03-29T08:27:05.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68cbe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68c40b1630 0x7f68c415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68cbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68c4143eb0 0x7f68c415d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68cbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c415efe0 0x7f68c41620b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68cbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c41fa950 con 0x7f68c415efe0 2024-03-29T08:27:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68cbe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c41fabf0 con 0x7f68c4143eb0 2024-03-29T08:27:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.159+0000 7f68cbe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c41faed0 con 0x7f68c40b1630 2024-03-29T08:27:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68c9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c415efe0 0x7f68c41620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68c9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c415efe0 0x7f68c41620b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41368/0 (socket says 172.21.15.46:41368) 2024-03-29T08:27:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68c9e00640 1 -- 172.21.15.46:0/3270355847 learned_addr learned my addr 172.21.15.46:0/3270355847 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68c8a00640 1 --2- 172.21.15.46:0/3270355847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68c4143eb0 0x7f68c415d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68c9400640 1 --2- 172.21.15.46:0/3270355847 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68c40b1630 0x7f68c415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68c9e00640 1 -- 172.21.15.46:0/3270355847 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68c40b1630 msgr2=0x7f68c415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:05.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68c9e00640 1 --2- 172.21.15.46:0/3270355847 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68c40b1630 0x7f68c415a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68c9e00640 1 -- 172.21.15.46:0/3270355847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68c4143eb0 msgr2=0x7f68c415d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:05.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68c9e00640 1 --2- 172.21.15.46:0/3270355847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68c4143eb0 0x7f68c415d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68c9e00640 1 -- 172.21.15.46:0/3270355847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b8009420 con 0x7f68c415efe0 2024-03-29T08:27:05.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68c9e00640 1 --2- 172.21.15.46:0/3270355847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c415efe0 0x7f68c41620b0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f68bc00ca80 tx=0x7f68bc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:05.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68c1600640 1 -- 172.21.15.46:0/3270355847 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc00ac30 con 0x7f68c415efe0 2024-03-29T08:27:05.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68c1600640 1 -- 172.21.15.46:0/3270355847 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc00ad90 con 0x7f68c415efe0 2024-03-29T08:27:05.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68c1600640 1 -- 172.21.15.46:0/3270355847 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc012600 con 0x7f68c415efe0 2024-03-29T08:27:05.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68cbe00640 1 -- 172.21.15.46:0/3270355847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c41fb1b0 con 0x7f68c415efe0 2024-03-29T08:27:05.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68cbe00640 1 -- 172.21.15.46:0/3270355847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c415f450 con 0x7f68c415efe0 2024-03-29T08:27:05.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68cbe00640 1 -- 172.21.15.46:0/3270355847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c41fb3d0 con 0x7f68c415efe0 2024-03-29T08:27:05.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68c1600640 1 -- 172.21.15.46:0/3270355847 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f68bc007070 con 0x7f68c415efe0 2024-03-29T08:27:05.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68c1600640 1 --2- 172.21.15.46:0/3270355847 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68a4066690 0x7f68a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:05.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.163+0000 7f68c1600640 1 -- 172.21.15.46:0/3270355847 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f68bc084080 con 0x7f68c415efe0 2024-03-29T08:27:05.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.167+0000 7f68c9400640 1 --2- 172.21.15.46:0/3270355847 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68a4066690 0x7f68a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:05.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.167+0000 7f68c9400640 1 --2- 172.21.15.46:0/3270355847 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68a4066690 0x7f68a4068b50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f68b80ae000 tx=0x7f68b8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:05.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.171+0000 7f68c1600640 1 -- 172.21.15.46:0/3270355847 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f68bc0b5980 con 0x7f68c415efe0 2024-03-29T08:27:05.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.331+0000 7f68cbe00640 1 -- 172.21.15.46:0/3270355847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68c4071a70 con 0x7f68c415efe0 2024-03-29T08:27:05.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.331+0000 7f68c1600640 1 -- 172.21.15.46:0/3270355847 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f68bc051350 con 0x7f68c415efe0 2024-03-29T08:27:05.332 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:27:05.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.331+0000 7f68cbe00640 1 -- 172.21.15.46:0/3270355847 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68a4066690 msgr2=0x7f68a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.331+0000 7f68cbe00640 1 --2- 172.21.15.46:0/3270355847 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68a4066690 0x7f68a4068b50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f68b80ae000 tx=0x7f68b8008040 comp rx=0 tx=0).stop 2024-03-29T08:27:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.331+0000 7f68cbe00640 1 -- 172.21.15.46:0/3270355847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c415efe0 msgr2=0x7f68c41620b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.331+0000 7f68cbe00640 1 --2- 172.21.15.46:0/3270355847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c415efe0 0x7f68c41620b0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f68bc00ca80 tx=0x7f68bc00cf40 comp rx=0 tx=0).stop 2024-03-29T08:27:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.331+0000 7f68cbe00640 1 -- 172.21.15.46:0/3270355847 shutdown_connections 2024-03-29T08:27:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.331+0000 7f68cbe00640 1 --2- 172.21.15.46:0/3270355847 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68a4066690 0x7f68a4068b50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.331+0000 7f68cbe00640 1 --2- 172.21.15.46:0/3270355847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68c4143eb0 0x7f68c415d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.331+0000 7f68cbe00640 1 --2- 172.21.15.46:0/3270355847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68c415efe0 0x7f68c41620b0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.331+0000 7f68cbe00640 1 --2- 172.21.15.46:0/3270355847 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68c40b1630 0x7f68c415a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.331+0000 7f68cbe00640 1 -- 172.21.15.46:0/3270355847 >> 172.21.15.46:0/3270355847 conn(0x7f68c406a390 msgr2=0x7f68c40cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.331+0000 7f68cbe00640 1 -- 172.21.15.46:0/3270355847 shutdown_connections 2024-03-29T08:27:05.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.331+0000 7f68cbe00640 1 -- 172.21.15.46:0/3270355847 wait complete. 2024-03-29T08:27:05.354 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:27:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.495+0000 7f443ce00640 1 Processor -- start 2024-03-29T08:27:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.495+0000 7f443ce00640 1 -- start start 2024-03-29T08:27:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.495+0000 7f443ce00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f443806f1c0 0x7f4438069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.495+0000 7f443ce00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f443806a720 con 0x7f443806a2d0 2024-03-29T08:27:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.495+0000 7f443ce00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f443815d080 con 0x7f443815cad0 2024-03-29T08:27:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.495+0000 7f443ce00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443815dd00 con 0x7f443806f1c0 2024-03-29T08:27:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.495+0000 7f4436800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f443806a2d0 0x7f443806d470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:37886/0 (socket says 172.21.15.46:37886) 2024-03-29T08:27:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.495+0000 7f4436800640 1 -- 172.21.15.46:0/74315380 learned_addr learned my addr 172.21.15.46:0/74315380 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.495+0000 7f4435400640 1 --2- 172.21.15.46:0/74315380 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f443806f1c0 0x7f4438069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.495+0000 7f4434a00640 1 -- 172.21.15.46:0/74315380 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283117575 0 0) 0x7f443806a720 con 0x7f443806a2d0 2024-03-29T08:27:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.499+0000 7f4434a00640 1 -- 172.21.15.46:0/74315380 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f441c003530 con 0x7f443806a2d0 2024-03-29T08:27:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.499+0000 7f4434a00640 1 -- 172.21.15.46:0/74315380 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260737850 0 0) 0x7f443815d080 con 0x7f443815cad0 2024-03-29T08:27:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.499+0000 7f4434a00640 1 -- 172.21.15.46:0/74315380 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f443806a720 con 0x7f443815cad0 2024-03-29T08:27:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.499+0000 7f4435400640 1 -- 172.21.15.46:0/74315380 >> v1:172.21.15.73:6789/0 conn(0x7f443815cad0 legacy=0x7f443806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.499+0000 7f4435400640 1 -- 172.21.15.46:0/74315380 >> v1:172.21.15.46:6789/0 conn(0x7f443806a2d0 legacy=0x7f443806d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.499+0000 7f4434a00640 1 -- 172.21.15.46:0/74315380 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 166657623 0 0) 0x7f441c003530 con 0x7f443806a2d0 2024-03-29T08:27:05.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.499+0000 7f4435400640 1 -- 172.21.15.46:0/74315380 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443815d360 con 0x7f443806f1c0 2024-03-29T08:27:05.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.499+0000 7f4435400640 1 --2- 172.21.15.46:0/74315380 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f443806f1c0 0x7f4438069d90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f44240097f0 tx=0x7f442401c370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa23dda3a8ccd020 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.499+0000 7f4434a00640 1 -- 172.21.15.46:0/74315380 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430001a00 con 0x7f443806f1c0 2024-03-29T08:27:05.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.499+0000 7f4434a00640 1 -- 172.21.15.46:0/74315380 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44240a7050 con 0x7f443806f1c0 2024-03-29T08:27:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.499+0000 7f4434a00640 1 -- 172.21.15.46:0/74315380 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44240ab620 con 0x7f443806f1c0 2024-03-29T08:27:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.499+0000 7f443ce00640 1 -- 172.21.15.46:0/74315380 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f443806f1c0 msgr2=0x7f4438069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.499+0000 7f443ce00640 1 --2- 172.21.15.46:0/74315380 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f443806f1c0 0x7f4438069d90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f44240097f0 tx=0x7f442401c370 comp rx=0 tx=0).stop 2024-03-29T08:27:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.499+0000 7f443ce00640 1 -- 172.21.15.46:0/74315380 shutdown_connections 2024-03-29T08:27:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.499+0000 7f443ce00640 1 --2- 172.21.15.46:0/74315380 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f443806f1c0 0x7f4438069d90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f443ce00640 1 -- 172.21.15.46:0/74315380 >> 172.21.15.46:0/74315380 conn(0x7f4438098a40 msgr2=0x7f44380b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f443ce00640 1 -- 172.21.15.46:0/74315380 shutdown_connections 2024-03-29T08:27:05.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f443ce00640 1 -- 172.21.15.46:0/74315380 wait complete. 2024-03-29T08:27:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f443ce00640 1 Processor -- start 2024-03-29T08:27:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f443ce00640 1 -- start start 2024-03-29T08:27:05.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f443ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f443806a2d0 0x7f44380c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f443ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f443806f1c0 0x7f4438160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f443ce00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f443815cad0 0x7f443815f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f443ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443815f760 con 0x7f443806f1c0 2024-03-29T08:27:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f443ce00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4438152060 con 0x7f443806a2d0 2024-03-29T08:27:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f443ce00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443815f970 con 0x7f443815cad0 2024-03-29T08:27:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f4435400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f443806f1c0 0x7f4438160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f4435400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f443806f1c0 0x7f4438160a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41378/0 (socket says 172.21.15.46:41378) 2024-03-29T08:27:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f4435400640 1 -- 172.21.15.46:0/3921470808 learned_addr learned my addr 172.21.15.46:0/3921470808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f4435e00640 1 --2- 172.21.15.46:0/3921470808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f443806a2d0 0x7f44380c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:05.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f4436800640 1 --2- 172.21.15.46:0/3921470808 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f443815cad0 0x7f443815f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f4435400640 1 -- 172.21.15.46:0/3921470808 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f443815cad0 msgr2=0x7f443815f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f4435400640 1 --2- 172.21.15.46:0/3921470808 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f443815cad0 0x7f443815f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f4435400640 1 -- 172.21.15.46:0/3921470808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f443806a2d0 msgr2=0x7f44380c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f4435400640 1 --2- 172.21.15.46:0/3921470808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f443806a2d0 0x7f44380c5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f4435400640 1 -- 172.21.15.46:0/3921470808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4424009420 con 0x7f443806f1c0 2024-03-29T08:27:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f4435400640 1 --2- 172.21.15.46:0/3921470808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f443806f1c0 0x7f4438160a80 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f4424009740 tx=0x7f4424019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f442e000640 1 -- 172.21.15.46:0/3921470808 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424004020 con 0x7f443806f1c0 2024-03-29T08:27:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f442e000640 1 -- 172.21.15.46:0/3921470808 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44240a4e90 con 0x7f443806f1c0 2024-03-29T08:27:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f443ce00640 1 -- 172.21.15.46:0/3921470808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443815fcc0 con 0x7f443806f1c0 2024-03-29T08:27:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.503+0000 7f442e000640 1 -- 172.21.15.46:0/3921470808 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424004380 con 0x7f443806f1c0 2024-03-29T08:27:05.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.507+0000 7f443ce00640 1 -- 172.21.15.46:0/3921470808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4438160280 con 0x7f443806f1c0 2024-03-29T08:27:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.507+0000 7f443ce00640 1 -- 172.21.15.46:0/3921470808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44380cd8f0 con 0x7f443806f1c0 2024-03-29T08:27:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.507+0000 7f442e000640 1 -- 172.21.15.46:0/3921470808 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f44240b2040 con 0x7f443806f1c0 2024-03-29T08:27:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.507+0000 7f442e000640 1 --2- 172.21.15.46:0/3921470808 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4418066690 0x7f4418068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:05.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.507+0000 7f442e000640 1 -- 172.21.15.46:0/3921470808 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f442411de70 con 0x7f443806f1c0 2024-03-29T08:27:05.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.507+0000 7f4435e00640 1 --2- 172.21.15.46:0/3921470808 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4418066690 0x7f4418068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:05.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.507+0000 7f4435e00640 1 --2- 172.21.15.46:0/3921470808 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4418066690 0x7f4418068b50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f44280011f0 tx=0x7f4428009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:05.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.511+0000 7f442e000640 1 -- 172.21.15.46:0/3921470808 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f44240eb120 con 0x7f443806f1c0 2024-03-29T08:27:05.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.671+0000 7f443ce00640 1 -- 172.21.15.46:0/3921470808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f443809d700 con 0x7f443806f1c0 2024-03-29T08:27:05.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.671+0000 7f442e000640 1 -- 172.21.15.46:0/3921470808 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f44240ee740 con 0x7f443806f1c0 2024-03-29T08:27:05.674 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:27:05.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.675+0000 7f443ce00640 1 -- 172.21.15.46:0/3921470808 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4418066690 msgr2=0x7f4418068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:05.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.675+0000 7f443ce00640 1 --2- 172.21.15.46:0/3921470808 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4418066690 0x7f4418068b50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f44280011f0 tx=0x7f4428009040 comp rx=0 tx=0).stop 2024-03-29T08:27:05.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.675+0000 7f443ce00640 1 -- 172.21.15.46:0/3921470808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f443806f1c0 msgr2=0x7f4438160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:05.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.675+0000 7f443ce00640 1 --2- 172.21.15.46:0/3921470808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f443806f1c0 0x7f4438160a80 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f4424009740 tx=0x7f4424019110 comp rx=0 tx=0).stop 2024-03-29T08:27:05.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.675+0000 7f443ce00640 1 -- 172.21.15.46:0/3921470808 shutdown_connections 2024-03-29T08:27:05.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.675+0000 7f443ce00640 1 --2- 172.21.15.46:0/3921470808 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4418066690 0x7f4418068b50 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.675+0000 7f443ce00640 1 --2- 172.21.15.46:0/3921470808 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f443815cad0 0x7f443815f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.675+0000 7f443ce00640 1 --2- 172.21.15.46:0/3921470808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f443806f1c0 0x7f4438160a80 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.675+0000 7f443ce00640 1 --2- 172.21.15.46:0/3921470808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f443806a2d0 0x7f44380c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.675+0000 7f443ce00640 1 -- 172.21.15.46:0/3921470808 >> 172.21.15.46:0/3921470808 conn(0x7f4438098a40 msgr2=0x7f44380b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:05.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.675+0000 7f443ce00640 1 -- 172.21.15.46:0/3921470808 shutdown_connections 2024-03-29T08:27:05.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.675+0000 7f443ce00640 1 -- 172.21.15.46:0/3921470808 wait complete. 2024-03-29T08:27:05.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:27:05.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.835+0000 7f8a1fa00640 1 Processor -- start 2024-03-29T08:27:05.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.835+0000 7f8a1fa00640 1 -- start start 2024-03-29T08:27:05.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.835+0000 7f8a1fa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8a181593d0 0x7f8a1806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:05.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.835+0000 7f8a1fa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a1815d230 0x7f8a1815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:05.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.835+0000 7f8a1fa00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a1806d130 con 0x7f8a1815dbd0 2024-03-29T08:27:05.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.835+0000 7f8a1fa00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a1815b8f0 con 0x7f8a181593d0 2024-03-29T08:27:05.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.835+0000 7f8a1fa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1815ad50 con 0x7f8a1815d230 2024-03-29T08:27:05.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.835+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a1815d230 0x7f8a1815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:05.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.835+0000 7f8a1d000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8a181593d0 0x7f8a1806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:05.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.835+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a1815d230 0x7f8a1815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:35302/0 (socket says 172.21.15.46:35302) 2024-03-29T08:27:05.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.835+0000 7f8a17e00640 1 -- 172.21.15.46:0/177516294 learned_addr learned my addr 172.21.15.46:0/177516294 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:05.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.835+0000 7f8a17e00640 1 -- 172.21.15.46:0/177516294 >> v2:172.21.15.73:3300/0 conn(0x7f8a181593d0 msgr2=0x7f8a1806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:05.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.835+0000 7f8a17e00640 1 --2- 172.21.15.46:0/177516294 >> v2:172.21.15.73:3300/0 conn(0x7f8a181593d0 0x7f8a1806cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.835+0000 7f8a17e00640 1 -- 172.21.15.46:0/177516294 >> v1:172.21.15.73:6789/0 conn(0x7f8a1815dbd0 legacy=0x7f8a1815bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:05.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.835+0000 7f8a17e00640 1 -- 172.21.15.46:0/177516294 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1815e060 con 0x7f8a1815d230 2024-03-29T08:27:05.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.839+0000 7f8a17e00640 1 --2- 172.21.15.46:0/177516294 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a1815d230 0x7f8a1815d690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009870 tx=0x7f8a0401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f4aae8302e58754 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:05.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.839+0000 7f8a17400640 1 -- 172.21.15.46:0/177516294 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a040a9620 con 0x7f8a1815d230 2024-03-29T08:27:05.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.839+0000 7f8a17400640 1 -- 172.21.15.46:0/177516294 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a040a9780 con 0x7f8a1815d230 2024-03-29T08:27:05.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.839+0000 7f8a17400640 1 -- 172.21.15.46:0/177516294 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a040a9b10 con 0x7f8a1815d230 2024-03-29T08:27:05.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.839+0000 7f8a1fa00640 1 -- 172.21.15.46:0/177516294 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a1815d230 msgr2=0x7f8a1815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:05.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.839+0000 7f8a1fa00640 1 --2- 172.21.15.46:0/177516294 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a1815d230 0x7f8a1815d690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009870 tx=0x7f8a0401c560 comp rx=0 tx=0).stop 2024-03-29T08:27:05.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.839+0000 7f8a1fa00640 1 -- 172.21.15.46:0/177516294 shutdown_connections 2024-03-29T08:27:05.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.839+0000 7f8a1fa00640 1 --2- 172.21.15.46:0/177516294 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a1815d230 0x7f8a1815d690 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.839+0000 7f8a1fa00640 1 --2- 172.21.15.46:0/177516294 >> v2:172.21.15.73:3300/0 conn(0x7f8a181593d0 0x7f8a1806cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.839+0000 7f8a1fa00640 1 -- 172.21.15.46:0/177516294 >> 172.21.15.46:0/177516294 conn(0x7f8a18066110 msgr2=0x7f8a18069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:05.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.839+0000 7f8a1fa00640 1 -- 172.21.15.46:0/177516294 shutdown_connections 2024-03-29T08:27:05.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.839+0000 7f8a1fa00640 1 -- 172.21.15.46:0/177516294 wait complete. 2024-03-29T08:27:05.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a1fa00640 1 Processor -- start 2024-03-29T08:27:05.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a1fa00640 1 -- start start 2024-03-29T08:27:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a1fa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a181593d0 0x7f8a1814c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a1fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a1815d230 0x7f8a181525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a1fa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a1815dbd0 0x7f8a180b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a1fa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18152130 con 0x7f8a1815d230 2024-03-29T08:27:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a1fa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1814c840 con 0x7f8a181593d0 2024-03-29T08:27:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a1fa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a18151800 con 0x7f8a1815dbd0 2024-03-29T08:27:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a1815d230 0x7f8a181525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a17e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a1815d230 0x7f8a181525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41380/0 (socket says 172.21.15.46:41380) 2024-03-29T08:27:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a17e00640 1 -- 172.21.15.46:0/2075940286 learned_addr learned my addr 172.21.15.46:0/2075940286 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a1d000640 1 --2- 172.21.15.46:0/2075940286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a181593d0 0x7f8a1814c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a1da00640 1 --2- 172.21.15.46:0/2075940286 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a1815dbd0 0x7f8a180b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a17e00640 1 -- 172.21.15.46:0/2075940286 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a1815dbd0 msgr2=0x7f8a180b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a17e00640 1 --2- 172.21.15.46:0/2075940286 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a1815dbd0 0x7f8a180b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a17e00640 1 -- 172.21.15.46:0/2075940286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a181593d0 msgr2=0x7f8a1814c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a17e00640 1 --2- 172.21.15.46:0/2075940286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a181593d0 0x7f8a1814c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:05.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a17e00640 1 -- 172.21.15.46:0/2075940286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a040094d0 con 0x7f8a1815d230 2024-03-29T08:27:05.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a17e00640 1 --2- 172.21.15.46:0/2075940286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a1815d230 0x7f8a181525c0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f8a0401ac10 tx=0x7f8a04019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:05.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a14c00640 1 -- 172.21.15.46:0/2075940286 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a040a9e80 con 0x7f8a1815d230 2024-03-29T08:27:05.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a14c00640 1 -- 172.21.15.46:0/2075940286 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a040bb3f0 con 0x7f8a1815d230 2024-03-29T08:27:05.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a1fa00640 1 -- 172.21.15.46:0/2075940286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a18151ae0 con 0x7f8a1815d230 2024-03-29T08:27:05.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a14c00640 1 -- 172.21.15.46:0/2075940286 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a040bb780 con 0x7f8a1815d230 2024-03-29T08:27:05.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.843+0000 7f8a1fa00640 1 -- 172.21.15.46:0/2075940286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a18152b00 con 0x7f8a1815d230 2024-03-29T08:27:05.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.847+0000 7f8a1fa00640 1 -- 172.21.15.46:0/2075940286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a181522a0 con 0x7f8a1815d230 2024-03-29T08:27:05.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.847+0000 7f8a14c00640 1 -- 172.21.15.46:0/2075940286 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8a040bb8e0 con 0x7f8a1815d230 2024-03-29T08:27:05.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.847+0000 7f8a14c00640 1 --2- 172.21.15.46:0/2075940286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f89f8066690 0x7f89f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:05.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.847+0000 7f8a14c00640 1 -- 172.21.15.46:0/2075940286 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f8a0411e480 con 0x7f8a1815d230 2024-03-29T08:27:05.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.847+0000 7f8a1d000640 1 --2- 172.21.15.46:0/2075940286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f89f8066690 0x7f89f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:05.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.847+0000 7f8a1d000640 1 --2- 172.21.15.46:0/2075940286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f89f8066690 0x7f89f8068b50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009ac0 tx=0x7f8a0c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:05.851+0000 7f8a14c00640 1 -- 172.21.15.46:0/2075940286 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8a040eb6a0 con 0x7f8a1815d230 2024-03-29T08:27:06.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.051+0000 7f8a1fa00640 1 -- 172.21.15.46:0/2075940286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a18066c90 con 0x7f8a1815d230 2024-03-29T08:27:06.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.051+0000 7f8a14c00640 1 -- 172.21.15.46:0/2075940286 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f8a040eecc0 con 0x7f8a1815d230 2024-03-29T08:27:06.053 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:06.053 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/1862652020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1862652020},{"type":"v1","addr":"172.21.15.73:6836","nonce":1862652020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:24:43.476334+0000","modified":"2024-03-29T08:26:26.873752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7986":{"gid":7986,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1886680593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1886680593},{"type":"v1","addr":"172.21.15.46:6835","nonce":1886680593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-29T08:27:06.054 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 69 2024-03-29T08:27:06.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.051+0000 7f8a1fa00640 1 -- 172.21.15.46:0/2075940286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f89f8066690 msgr2=0x7f89f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:06.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.051+0000 7f8a1fa00640 1 --2- 172.21.15.46:0/2075940286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f89f8066690 0x7f89f8068b50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009ac0 tx=0x7f8a0c009190 comp rx=0 tx=0).stop 2024-03-29T08:27:06.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.051+0000 7f8a1fa00640 1 -- 172.21.15.46:0/2075940286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a1815d230 msgr2=0x7f8a181525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.051+0000 7f8a1fa00640 1 --2- 172.21.15.46:0/2075940286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a1815d230 0x7f8a181525c0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f8a0401ac10 tx=0x7f8a04019300 comp rx=0 tx=0).stop 2024-03-29T08:27:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.051+0000 7f8a1fa00640 1 -- 172.21.15.46:0/2075940286 shutdown_connections 2024-03-29T08:27:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.051+0000 7f8a1fa00640 1 --2- 172.21.15.46:0/2075940286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f89f8066690 0x7f89f8068b50 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.051+0000 7f8a1fa00640 1 --2- 172.21.15.46:0/2075940286 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8a1815dbd0 0x7f8a180b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.051+0000 7f8a1fa00640 1 --2- 172.21.15.46:0/2075940286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a1815d230 0x7f8a181525c0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.051+0000 7f8a1fa00640 1 --2- 172.21.15.46:0/2075940286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a181593d0 0x7f8a1814c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.051+0000 7f8a1fa00640 1 -- 172.21.15.46:0/2075940286 >> 172.21.15.46:0/2075940286 conn(0x7f8a18066110 msgr2=0x7f8a1815bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.055+0000 7f8a1fa00640 1 -- 172.21.15.46:0/2075940286 shutdown_connections 2024-03-29T08:27:06.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.055+0000 7f8a1fa00640 1 -- 172.21.15.46:0/2075940286 wait complete. 2024-03-29T08:27:06.075 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:27:06.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f4000640 1 Processor -- start 2024-03-29T08:27:06.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f4000640 1 -- start start 2024-03-29T08:27:06.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f4000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53ec1593d0 0x7f53ec06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f4000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53ec15d230 0x7f53ec15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f4000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f53ec15dbd0 0x7f53ec15bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f4000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ec06d130 con 0x7f53ec1593d0 2024-03-29T08:27:06.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f4000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ec15b8f0 con 0x7f53ec15dbd0 2024-03-29T08:27:06.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f4000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec15ad50 con 0x7f53ec15d230 2024-03-29T08:27:06.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f1600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53ec1593d0 0x7f53ec06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f1600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f53ec1593d0 0x7f53ec06cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41396/0 (socket says 172.21.15.46:41396) 2024-03-29T08:27:06.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f1600640 1 -- 172.21.15.46:0/794543808 learned_addr learned my addr 172.21.15.46:0/794543808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:06.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f2000640 1 --2- 172.21.15.46:0/794543808 >> v2:172.21.15.73:3300/0 conn(0x7f53ec15dbd0 0x7f53ec15bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f0c00640 1 --2- 172.21.15.46:0/794543808 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53ec15d230 0x7f53ec15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f1600640 1 -- 172.21.15.46:0/794543808 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53ec15d230 msgr2=0x7f53ec15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:06.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f1600640 1 --2- 172.21.15.46:0/794543808 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53ec15d230 0x7f53ec15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f1600640 1 -- 172.21.15.46:0/794543808 >> v2:172.21.15.73:3300/0 conn(0x7f53ec15dbd0 msgr2=0x7f53ec15bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:06.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f1600640 1 --2- 172.21.15.46:0/794543808 >> v2:172.21.15.73:3300/0 conn(0x7f53ec15dbd0 0x7f53ec15bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.215+0000 7f53f1600640 1 -- 172.21.15.46:0/794543808 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec15e060 con 0x7f53ec1593d0 2024-03-29T08:27:06.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.219+0000 7f53f1600640 1 --2- 172.21.15.46:0/794543808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53ec1593d0 0x7f53ec06cbf0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f53ec15b860 tx=0x7f53e001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1693c158a2a685e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:06.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.219+0000 7f53ebe00640 1 -- 172.21.15.46:0/794543808 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e00aa640 con 0x7f53ec1593d0 2024-03-29T08:27:06.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.219+0000 7f53ebe00640 1 -- 172.21.15.46:0/794543808 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e00aa7a0 con 0x7f53ec1593d0 2024-03-29T08:27:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.219+0000 7f53ebe00640 1 -- 172.21.15.46:0/794543808 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e00aab30 con 0x7f53ec1593d0 2024-03-29T08:27:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.219+0000 7f53f4000640 1 -- 172.21.15.46:0/794543808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53ec1593d0 msgr2=0x7f53ec06cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.219+0000 7f53f4000640 1 --2- 172.21.15.46:0/794543808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53ec1593d0 0x7f53ec06cbf0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f53ec15b860 tx=0x7f53e001c520 comp rx=0 tx=0).stop 2024-03-29T08:27:06.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.219+0000 7f53f4000640 1 -- 172.21.15.46:0/794543808 shutdown_connections 2024-03-29T08:27:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.219+0000 7f53f4000640 1 --2- 172.21.15.46:0/794543808 >> v2:172.21.15.73:3300/0 conn(0x7f53ec15dbd0 0x7f53ec15bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.219+0000 7f53f4000640 1 --2- 172.21.15.46:0/794543808 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53ec15d230 0x7f53ec15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.219+0000 7f53f4000640 1 --2- 172.21.15.46:0/794543808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53ec1593d0 0x7f53ec06cbf0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.219+0000 7f53f4000640 1 -- 172.21.15.46:0/794543808 >> 172.21.15.46:0/794543808 conn(0x7f53ec066110 msgr2=0x7f53ec069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.219+0000 7f53f4000640 1 -- 172.21.15.46:0/794543808 shutdown_connections 2024-03-29T08:27:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.219+0000 7f53f4000640 1 -- 172.21.15.46:0/794543808 wait complete. 2024-03-29T08:27:06.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f4000640 1 Processor -- start 2024-03-29T08:27:06.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f4000640 1 -- start start 2024-03-29T08:27:06.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f4000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53ec1593d0 0x7f53ec0ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f4000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53ec15d230 0x7f53ec0ce610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f4000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53ec15dbd0 0x7f53ec1541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f4000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec13a9d0 con 0x7f53ec1593d0 2024-03-29T08:27:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f4000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec157730 con 0x7f53ec15dbd0 2024-03-29T08:27:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f4000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec157a10 con 0x7f53ec15d230 2024-03-29T08:27:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53ec1593d0 0x7f53ec0ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53ec1593d0 0x7f53ec0ce0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41410/0 (socket says 172.21.15.46:41410) 2024-03-29T08:27:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f1600640 1 -- 172.21.15.46:0/1139665236 learned_addr learned my addr 172.21.15.46:0/1139665236 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f2000640 1 --2- 172.21.15.46:0/1139665236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53ec15dbd0 0x7f53ec1541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f0c00640 1 --2- 172.21.15.46:0/1139665236 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53ec15d230 0x7f53ec0ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f1600640 1 -- 172.21.15.46:0/1139665236 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53ec15d230 msgr2=0x7f53ec0ce610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f1600640 1 --2- 172.21.15.46:0/1139665236 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53ec15d230 0x7f53ec0ce610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f1600640 1 -- 172.21.15.46:0/1139665236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53ec15dbd0 msgr2=0x7f53ec1541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f1600640 1 --2- 172.21.15.46:0/1139665236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53ec15dbd0 0x7f53ec1541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f1600640 1 -- 172.21.15.46:0/1139665236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e00094d0 con 0x7f53ec1593d0 2024-03-29T08:27:06.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f2000640 1 --2- 172.21.15.46:0/1139665236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53ec15dbd0 0x7f53ec1541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:06.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f0c00640 1 --2- 172.21.15.46:0/1139665236 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53ec15d230 0x7f53ec0ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:06.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f1600640 1 --2- 172.21.15.46:0/1139665236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53ec1593d0 0x7f53ec0ce0d0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f53e001abd0 tx=0x7f53e001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:06.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53e9600640 1 -- 172.21.15.46:0/1139665236 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e00bd3f0 con 0x7f53ec1593d0 2024-03-29T08:27:06.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53e9600640 1 -- 172.21.15.46:0/1139665236 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e00bd550 con 0x7f53ec1593d0 2024-03-29T08:27:06.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f4000640 1 -- 172.21.15.46:0/1139665236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec157cf0 con 0x7f53ec1593d0 2024-03-29T08:27:06.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53e9600640 1 -- 172.21.15.46:0/1139665236 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e00bd900 con 0x7f53ec1593d0 2024-03-29T08:27:06.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f4000640 1 -- 172.21.15.46:0/1139665236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec14eb00 con 0x7f53ec1593d0 2024-03-29T08:27:06.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.223+0000 7f53f4000640 1 -- 172.21.15.46:0/1139665236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ec066c90 con 0x7f53ec1593d0 2024-03-29T08:27:06.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.227+0000 7f53e9600640 1 -- 172.21.15.46:0/1139665236 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f53e00a8050 con 0x7f53ec1593d0 2024-03-29T08:27:06.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.227+0000 7f53e9600640 1 --2- 172.21.15.46:0/1139665236 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53cc066690 0x7f53cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.227+0000 7f53e9600640 1 -- 172.21.15.46:0/1139665236 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f53e011e0e0 con 0x7f53ec1593d0 2024-03-29T08:27:06.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.227+0000 7f53f0c00640 1 --2- 172.21.15.46:0/1139665236 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53cc066690 0x7f53cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.227+0000 7f53f0c00640 1 --2- 172.21.15.46:0/1139665236 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53cc066690 0x7f53cc068b50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0ad7d0 tx=0x7f53d8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:06.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.231+0000 7f53e9600640 1 -- 172.21.15.46:0/1139665236 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f53e00eb4d0 con 0x7f53ec1593d0 2024-03-29T08:27:06.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.399+0000 7f53f4000640 1 -- 172.21.15.46:0/1139665236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53ec0add70 con 0x7f53ec1593d0 2024-03-29T08:27:06.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.399+0000 7f53e9600640 1 -- 172.21.15.46:0/1139665236 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16315 (secure 0 0 0) 0x7f53e00eeaf0 con 0x7f53ec1593d0 2024-03-29T08:27:06.403 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:06.403 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:26:22.851956+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:24:41.113618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-29T08:24:42.487380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/2051429508":"2024-03-30T08:26:22.851926+0000","172.21.15.46:6833/2051429508":"2024-03-30T08:26:22.851926+0000","172.21.15.73:6836/4166845858":"2024-03-30T08:25:02.334350+0000","172.21.15.73:6834/4166845858":"2024-03-30T08:25:02.334350+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:27:06.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.403+0000 7f53f4000640 1 -- 172.21.15.46:0/1139665236 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53cc066690 msgr2=0x7f53cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:06.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.403+0000 7f53f4000640 1 --2- 172.21.15.46:0/1139665236 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53cc066690 0x7f53cc068b50 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f53ec0ad7d0 tx=0x7f53d8008040 comp rx=0 tx=0).stop 2024-03-29T08:27:06.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.403+0000 7f53f4000640 1 -- 172.21.15.46:0/1139665236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53ec1593d0 msgr2=0x7f53ec0ce0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:06.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.403+0000 7f53f4000640 1 --2- 172.21.15.46:0/1139665236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53ec1593d0 0x7f53ec0ce0d0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f53e001abd0 tx=0x7f53e001ad60 comp rx=0 tx=0).stop 2024-03-29T08:27:06.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.403+0000 7f53f4000640 1 -- 172.21.15.46:0/1139665236 shutdown_connections 2024-03-29T08:27:06.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.403+0000 7f53f4000640 1 --2- 172.21.15.46:0/1139665236 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f53cc066690 0x7f53cc068b50 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.403+0000 7f53f4000640 1 --2- 172.21.15.46:0/1139665236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53ec15dbd0 0x7f53ec1541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.403+0000 7f53f4000640 1 --2- 172.21.15.46:0/1139665236 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f53ec15d230 0x7f53ec0ce610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.403+0000 7f53f4000640 1 --2- 172.21.15.46:0/1139665236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53ec1593d0 0x7f53ec0ce0d0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.403+0000 7f53f4000640 1 -- 172.21.15.46:0/1139665236 >> 172.21.15.46:0/1139665236 conn(0x7f53ec066110 msgr2=0x7f53ec0ae870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:06.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.403+0000 7f53f4000640 1 -- 172.21.15.46:0/1139665236 shutdown_connections 2024-03-29T08:27:06.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.403+0000 7f53f4000640 1 -- 172.21.15.46:0/1139665236 wait complete. 2024-03-29T08:27:06.425 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T08:27:06.425 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:27:06.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.563+0000 7f42af600640 1 Processor -- start 2024-03-29T08:27:06.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.563+0000 7f42af600640 1 -- start start 2024-03-29T08:27:06.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.563+0000 7f42af600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f42a81593d0 0x7f42a806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.563+0000 7f42af600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f42a815d230 0x7f42a815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.563+0000 7f42af600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a806d130 con 0x7f42a815dbd0 2024-03-29T08:27:06.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.563+0000 7f42af600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a815b8f0 con 0x7f42a815d230 2024-03-29T08:27:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.563+0000 7f42af600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a815ad50 con 0x7f42a81593d0 2024-03-29T08:27:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.563+0000 7f42a7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f42a815d230 0x7f42a815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.563+0000 7f42a7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f42a815d230 0x7f42a815d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57830/0 (socket says 172.21.15.46:57830) 2024-03-29T08:27:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.563+0000 7f42a7e00640 1 -- 172.21.15.46:0/3011339511 learned_addr learned my addr 172.21.15.46:0/3011339511 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.563+0000 7f42acc00640 1 --2- 172.21.15.46:0/3011339511 >> v2:172.21.15.73:3300/0 conn(0x7f42a81593d0 0x7f42a806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.563+0000 7f42a7e00640 1 -- 172.21.15.46:0/3011339511 >> v2:172.21.15.73:3300/0 conn(0x7f42a81593d0 msgr2=0x7f42a806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:06.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.563+0000 7f42a7e00640 1 --2- 172.21.15.46:0/3011339511 >> v2:172.21.15.73:3300/0 conn(0x7f42a81593d0 0x7f42a806cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.563+0000 7f42a7e00640 1 -- 172.21.15.46:0/3011339511 >> v1:172.21.15.73:6789/0 conn(0x7f42a815dbd0 legacy=0x7f42a815bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:06.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.563+0000 7f42a7e00640 1 -- 172.21.15.46:0/3011339511 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a815e060 con 0x7f42a815d230 2024-03-29T08:27:06.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.563+0000 7f42acc00640 1 --2- 172.21.15.46:0/3011339511 >> v2:172.21.15.73:3300/0 conn(0x7f42a81593d0 0x7f42a806cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.567+0000 7f42a7e00640 1 --2- 172.21.15.46:0/3011339511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a815d230 0x7f42a815d690 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f42a806e460 tx=0x7f429401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7350a8639b7e5c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.567+0000 7f42a7400640 1 -- 172.21.15.46:0/3011339511 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42940aa680 con 0x7f42a815d230 2024-03-29T08:27:06.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.567+0000 7f42a7400640 1 -- 172.21.15.46:0/3011339511 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42940aa7e0 con 0x7f42a815d230 2024-03-29T08:27:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.567+0000 7f42a7400640 1 -- 172.21.15.46:0/3011339511 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42940aab70 con 0x7f42a815d230 2024-03-29T08:27:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.567+0000 7f42af600640 1 -- 172.21.15.46:0/3011339511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a815d230 msgr2=0x7f42a815d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.567+0000 7f42af600640 1 --2- 172.21.15.46:0/3011339511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a815d230 0x7f42a815d690 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f42a806e460 tx=0x7f429401c520 comp rx=0 tx=0).stop 2024-03-29T08:27:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.567+0000 7f42af600640 1 -- 172.21.15.46:0/3011339511 shutdown_connections 2024-03-29T08:27:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.567+0000 7f42af600640 1 --2- 172.21.15.46:0/3011339511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a815d230 0x7f42a815d690 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.567+0000 7f42af600640 1 --2- 172.21.15.46:0/3011339511 >> v2:172.21.15.73:3300/0 conn(0x7f42a81593d0 0x7f42a806cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.567+0000 7f42af600640 1 -- 172.21.15.46:0/3011339511 >> 172.21.15.46:0/3011339511 conn(0x7f42a8066110 msgr2=0x7f42a8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.567+0000 7f42af600640 1 -- 172.21.15.46:0/3011339511 shutdown_connections 2024-03-29T08:27:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.567+0000 7f42af600640 1 -- 172.21.15.46:0/3011339511 wait complete. 2024-03-29T08:27:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42af600640 1 Processor -- start 2024-03-29T08:27:06.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42af600640 1 -- start start 2024-03-29T08:27:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42af600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f42a81593d0 0x7f42a814ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42af600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a815d230 0x7f42a8146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42af600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42a815dbd0 0x7f42a80abf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42af600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a814fcf0 con 0x7f42a815d230 2024-03-29T08:27:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42af600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80abae0 con 0x7f42a815dbd0 2024-03-29T08:27:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42af600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a814f450 con 0x7f42a81593d0 2024-03-29T08:27:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a815d230 0x7f42a8146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42a7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a815d230 0x7f42a8146010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57846/0 (socket says 172.21.15.46:57846) 2024-03-29T08:27:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42a7e00640 1 -- 172.21.15.46:0/2391671816 learned_addr learned my addr 172.21.15.46:0/2391671816 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42ad600640 1 --2- 172.21.15.46:0/2391671816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42a815dbd0 0x7f42a80abf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42acc00640 1 --2- 172.21.15.46:0/2391671816 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f42a81593d0 0x7f42a814ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42a7e00640 1 -- 172.21.15.46:0/2391671816 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f42a81593d0 msgr2=0x7f42a814ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42a7e00640 1 --2- 172.21.15.46:0/2391671816 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f42a81593d0 0x7f42a814ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42a7e00640 1 -- 172.21.15.46:0/2391671816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42a815dbd0 msgr2=0x7f42a80abf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42a7e00640 1 --2- 172.21.15.46:0/2391671816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42a815dbd0 0x7f42a80abf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42a7e00640 1 -- 172.21.15.46:0/2391671816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42940094d0 con 0x7f42a815d230 2024-03-29T08:27:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42acc00640 1 --2- 172.21.15.46:0/2391671816 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f42a81593d0 0x7f42a814ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42ad600640 1 --2- 172.21.15.46:0/2391671816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42a815dbd0 0x7f42a80abf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42a7e00640 1 --2- 172.21.15.46:0/2391671816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a815d230 0x7f42a8146010 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f42940ac040 tx=0x7f429401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42a4c00640 1 -- 172.21.15.46:0/2391671816 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42940b3800 con 0x7f42a815d230 2024-03-29T08:27:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42a4c00640 1 -- 172.21.15.46:0/2391671816 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42940b3980 con 0x7f42a815d230 2024-03-29T08:27:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42a4c00640 1 -- 172.21.15.46:0/2391671816 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42940b3d30 con 0x7f42a815d230 2024-03-29T08:27:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42af600640 1 -- 172.21.15.46:0/2391671816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a814f730 con 0x7f42a815d230 2024-03-29T08:27:06.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42af600640 1 -- 172.21.15.46:0/2391671816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a8146550 con 0x7f42a815d230 2024-03-29T08:27:06.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.571+0000 7f42af600640 1 -- 172.21.15.46:0/2391671816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a80ab8b0 con 0x7f42a815d230 2024-03-29T08:27:06.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.575+0000 7f42a4c00640 1 -- 172.21.15.46:0/2391671816 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f42940c6440 con 0x7f42a815d230 2024-03-29T08:27:06.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.575+0000 7f42a4c00640 1 --2- 172.21.15.46:0/2391671816 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4288066690 0x7f4288068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.575+0000 7f42a4c00640 1 -- 172.21.15.46:0/2391671816 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f429411efd0 con 0x7f42a815d230 2024-03-29T08:27:06.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.575+0000 7f42acc00640 1 --2- 172.21.15.46:0/2391671816 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4288066690 0x7f4288068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.575+0000 7f42acc00640 1 --2- 172.21.15.46:0/2391671816 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4288066690 0x7f4288068b50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f42a8158c70 tx=0x7f429c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.579+0000 7f42a4c00640 1 -- 172.21.15.46:0/2391671816 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f42a80ab8b0 con 0x7f42a815d230 2024-03-29T08:27:06.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.779+0000 7f42af600640 1 -- 172.21.15.46:0/2391671816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42a80ab8b0 con 0x7f42a815d230 2024-03-29T08:27:06.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.779+0000 7f42a4c00640 1 -- 172.21.15.46:0/2391671816 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f42a80ab8b0 con 0x7f42a815d230 2024-03-29T08:27:06.781 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:06.781 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/1862652020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1862652020},{"type":"v1","addr":"172.21.15.73:6836","nonce":1862652020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:24:43.476334+0000","modified":"2024-03-29T08:26:26.873752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7986":{"gid":7986,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1886680593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1886680593},{"type":"v1","addr":"172.21.15.46:6835","nonce":1886680593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-29T08:27:06.781 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 69 2024-03-29T08:27:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.779+0000 7f42af600640 1 -- 172.21.15.46:0/2391671816 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4288066690 msgr2=0x7f4288068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.779+0000 7f42af600640 1 --2- 172.21.15.46:0/2391671816 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4288066690 0x7f4288068b50 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f42a8158c70 tx=0x7f429c008040 comp rx=0 tx=0).stop 2024-03-29T08:27:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.779+0000 7f42af600640 1 -- 172.21.15.46:0/2391671816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a815d230 msgr2=0x7f42a8146010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.779+0000 7f42af600640 1 --2- 172.21.15.46:0/2391671816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a815d230 0x7f42a8146010 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f42940ac040 tx=0x7f429401ad60 comp rx=0 tx=0).stop 2024-03-29T08:27:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.779+0000 7f42af600640 1 -- 172.21.15.46:0/2391671816 shutdown_connections 2024-03-29T08:27:06.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.779+0000 7f42af600640 1 --2- 172.21.15.46:0/2391671816 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4288066690 0x7f4288068b50 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.779+0000 7f42af600640 1 --2- 172.21.15.46:0/2391671816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42a815dbd0 0x7f42a80abf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.779+0000 7f42af600640 1 --2- 172.21.15.46:0/2391671816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a815d230 0x7f42a8146010 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.779+0000 7f42af600640 1 --2- 172.21.15.46:0/2391671816 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f42a81593d0 0x7f42a814ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.779+0000 7f42af600640 1 -- 172.21.15.46:0/2391671816 >> 172.21.15.46:0/2391671816 conn(0x7f42a8066110 msgr2=0x7f42a806c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:06.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.779+0000 7f42af600640 1 -- 172.21.15.46:0/2391671816 shutdown_connections 2024-03-29T08:27:06.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.779+0000 7f42af600640 1 -- 172.21.15.46:0/2391671816 wait complete. 2024-03-29T08:27:06.803 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:27:06.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.943+0000 7f2f18e00640 1 Processor -- start 2024-03-29T08:27:06.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.943+0000 7f2f18e00640 1 -- start start 2024-03-29T08:27:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.943+0000 7f2f18e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f140cf3c0 0x7f2f14165660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.943+0000 7f2f18e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f140b0ab0 con 0x7f2f1415ad20 2024-03-29T08:27:06.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.943+0000 7f2f18e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f140b0d90 con 0x7f2f1415a940 2024-03-29T08:27:06.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.943+0000 7f2f18e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f140b1070 con 0x7f2f140cf3c0 2024-03-29T08:27:06.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.943+0000 7f2f12800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f140cf3c0 0x7f2f14165660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.943+0000 7f2f09400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2f1415ad20 0x7f2f140cecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33694/0 (socket says 172.21.15.46:33694) 2024-03-29T08:27:06.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.943+0000 7f2f09400640 1 -- 172.21.15.46:0/2675877466 learned_addr learned my addr 172.21.15.46:0/2675877466 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:06.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.943+0000 7f2f12800640 1 -- 172.21.15.46:0/2675877466 >> v1:172.21.15.73:6789/0 conn(0x7f2f1415a940 legacy=0x7f2f140afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:06.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.943+0000 7f2f12800640 1 -- 172.21.15.46:0/2675877466 >> v1:172.21.15.46:6789/0 conn(0x7f2f1415ad20 legacy=0x7f2f140cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:06.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.943+0000 7f2f12800640 1 -- 172.21.15.46:0/2675877466 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f140b1350 con 0x7f2f140cf3c0 2024-03-29T08:27:06.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.943+0000 7f2f11400640 1 -- 172.21.15.46:0/2675877466 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750821827 0 0) 0x7f2f140b0ab0 con 0x7f2f1415ad20 2024-03-29T08:27:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.947+0000 7f2f12800640 1 --2- 172.21.15.46:0/2675877466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f140cf3c0 0x7f2f14165660 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c0097c0 tx=0x7f2f0c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae0a1015752fe37 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.947+0000 7f2f11400640 1 -- 172.21.15.46:0/2675877466 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c0ab600 con 0x7f2f140cf3c0 2024-03-29T08:27:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.947+0000 7f2f11400640 1 -- 172.21.15.46:0/2675877466 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0c0ab760 con 0x7f2f140cf3c0 2024-03-29T08:27:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.947+0000 7f2f11400640 1 -- 172.21.15.46:0/2675877466 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c0abaf0 con 0x7f2f140cf3c0 2024-03-29T08:27:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.947+0000 7f2f18e00640 1 -- 172.21.15.46:0/2675877466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f140cf3c0 msgr2=0x7f2f14165660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.947+0000 7f2f18e00640 1 --2- 172.21.15.46:0/2675877466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f140cf3c0 0x7f2f14165660 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c0097c0 tx=0x7f2f0c01c3e0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.947+0000 7f2f18e00640 1 -- 172.21.15.46:0/2675877466 shutdown_connections 2024-03-29T08:27:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.947+0000 7f2f18e00640 1 --2- 172.21.15.46:0/2675877466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f140cf3c0 0x7f2f14165660 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.947+0000 7f2f18e00640 1 -- 172.21.15.46:0/2675877466 >> 172.21.15.46:0/2675877466 conn(0x7f2f14089fd0 msgr2=0x7f2f1408a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:06.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.947+0000 7f2f18e00640 1 -- 172.21.15.46:0/2675877466 shutdown_connections 2024-03-29T08:27:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.947+0000 7f2f18e00640 1 -- 172.21.15.46:0/2675877466 wait complete. 2024-03-29T08:27:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f18e00640 1 Processor -- start 2024-03-29T08:27:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f18e00640 1 -- start start 2024-03-29T08:27:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f18e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f140cf3c0 0x7f2f1414bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f18e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f1415a940 0x7f2f14150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f18e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f1415ad20 0x7f2f140ae350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f18e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1414c470 con 0x7f2f1415a940 2024-03-29T08:27:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f18e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1414fcf0 con 0x7f2f1415ad20 2024-03-29T08:27:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f18e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1414ffd0 con 0x7f2f140cf3c0 2024-03-29T08:27:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f09400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f1415a940 0x7f2f14150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f09400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f1415a940 0x7f2f14150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57858/0 (socket says 172.21.15.46:57858) 2024-03-29T08:27:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f09400640 1 -- 172.21.15.46:0/3831315286 learned_addr learned my addr 172.21.15.46:0/3831315286 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f12800640 1 --2- 172.21.15.46:0/3831315286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f1415ad20 0x7f2f140ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f11e00640 1 --2- 172.21.15.46:0/3831315286 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f140cf3c0 0x7f2f1414bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f09400640 1 -- 172.21.15.46:0/3831315286 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f140cf3c0 msgr2=0x7f2f1414bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:06.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f09400640 1 --2- 172.21.15.46:0/3831315286 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f140cf3c0 0x7f2f1414bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f09400640 1 -- 172.21.15.46:0/3831315286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f1415ad20 msgr2=0x7f2f140ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:06.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f09400640 1 --2- 172.21.15.46:0/3831315286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f1415ad20 0x7f2f140ae350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:06.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f09400640 1 -- 172.21.15.46:0/3831315286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c009420 con 0x7f2f1415a940 2024-03-29T08:27:06.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f09400640 1 --2- 172.21.15.46:0/3831315286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f1415a940 0x7f2f14150d90 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00c410 tx=0x7f2efc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:06.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f0aa00640 1 -- 172.21.15.46:0/3831315286 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc0074e0 con 0x7f2f1415a940 2024-03-29T08:27:06.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f0aa00640 1 -- 172.21.15.46:0/3831315286 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2efc007640 con 0x7f2f1415a940 2024-03-29T08:27:06.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f0aa00640 1 -- 172.21.15.46:0/3831315286 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc004a30 con 0x7f2f1415a940 2024-03-29T08:27:06.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f18e00640 1 -- 172.21.15.46:0/3831315286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f141502b0 con 0x7f2f1415a940 2024-03-29T08:27:06.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f18e00640 1 -- 172.21.15.46:0/3831315286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f14150870 con 0x7f2f1415a940 2024-03-29T08:27:06.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.951+0000 7f2f18e00640 1 -- 172.21.15.46:0/3831315286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f14066c90 con 0x7f2f1415a940 2024-03-29T08:27:06.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.955+0000 7f2f0aa00640 1 -- 172.21.15.46:0/3831315286 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2efc004b90 con 0x7f2f1415a940 2024-03-29T08:27:06.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.955+0000 7f2f0aa00640 1 --2- 172.21.15.46:0/3831315286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2ef0066690 0x7f2ef0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:06.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.955+0000 7f2f0aa00640 1 -- 172.21.15.46:0/3831315286 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f2efc083f30 con 0x7f2f1415a940 2024-03-29T08:27:06.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.955+0000 7f2f11e00640 1 --2- 172.21.15.46:0/3831315286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2ef0066690 0x7f2ef0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:06.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.955+0000 7f2f11e00640 1 --2- 172.21.15.46:0/3831315286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2ef0066690 0x7f2ef0068b50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2f040012f0 tx=0x7f2f04009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:06.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:06.959+0000 7f2f0aa00640 1 -- 172.21.15.46:0/3831315286 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2efc051290 con 0x7f2f1415a940 2024-03-29T08:27:07.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.159+0000 7f2f18e00640 1 -- 172.21.15.46:0/3831315286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f140d0580 con 0x7f2f1415a940 2024-03-29T08:27:07.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.159+0000 7f2f0aa00640 1 -- 172.21.15.46:0/3831315286 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4737 (secure 0 0 0) 0x7f2efc0548b0 con 0x7f2f1415a940 2024-03-29T08:27:07.161 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:07.161 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/1862652020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1862652020},{"type":"v1","addr":"172.21.15.73:6836","nonce":1862652020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:24:43.476334+0000","modified":"2024-03-29T08:26:26.873752+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":94,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7986},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7986":{"gid":7986,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1886680593","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1886680593},{"type":"v1","addr":"172.21.15.46:6835","nonce":1886680593}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-03-29T08:27:07.161 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 69 2024-03-29T08:27:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.159+0000 7f2f18e00640 1 -- 172.21.15.46:0/3831315286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2ef0066690 msgr2=0x7f2ef0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.159+0000 7f2f18e00640 1 --2- 172.21.15.46:0/3831315286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2ef0066690 0x7f2ef0068b50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2f040012f0 tx=0x7f2f04009040 comp rx=0 tx=0).stop 2024-03-29T08:27:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.159+0000 7f2f18e00640 1 -- 172.21.15.46:0/3831315286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f1415a940 msgr2=0x7f2f14150d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.159+0000 7f2f18e00640 1 --2- 172.21.15.46:0/3831315286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f1415a940 0x7f2f14150d90 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00c410 tx=0x7f2efc00c8d0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.159+0000 7f2f18e00640 1 -- 172.21.15.46:0/3831315286 shutdown_connections 2024-03-29T08:27:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.159+0000 7f2f18e00640 1 --2- 172.21.15.46:0/3831315286 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2ef0066690 0x7f2ef0068b50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.159+0000 7f2f18e00640 1 --2- 172.21.15.46:0/3831315286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f1415ad20 0x7f2f140ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.159+0000 7f2f18e00640 1 --2- 172.21.15.46:0/3831315286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f1415a940 0x7f2f14150d90 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.159+0000 7f2f18e00640 1 --2- 172.21.15.46:0/3831315286 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f140cf3c0 0x7f2f1414bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.159+0000 7f2f18e00640 1 -- 172.21.15.46:0/3831315286 >> 172.21.15.46:0/3831315286 conn(0x7f2f14089fd0 msgr2=0x7f2f14152460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.159+0000 7f2f18e00640 1 -- 172.21.15.46:0/3831315286 shutdown_connections 2024-03-29T08:27:07.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.159+0000 7f2f18e00640 1 -- 172.21.15.46:0/3831315286 wait complete. 2024-03-29T08:27:07.182 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:27:07.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.323+0000 7fc9a8a00640 1 Processor -- start 2024-03-29T08:27:07.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.323+0000 7fc9a8a00640 1 -- start start 2024-03-29T08:27:07.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.323+0000 7fc9a8a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9a40b0310 0x7fc9a40ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:07.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.323+0000 7fc9a8a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc9a40aeee0 0x7fc9a40af370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:07.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.323+0000 7fc9a8a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a406a5e0 con 0x7fc9a40b06f0 2024-03-29T08:27:07.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.323+0000 7fc9a8a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a406aba0 con 0x7fc9a40aeee0 2024-03-29T08:27:07.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.323+0000 7fc9a8a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a406a8c0 con 0x7fc9a40b0310 2024-03-29T08:27:07.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.323+0000 7fc9a3400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9a40b0310 0x7fc9a40ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:07.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.323+0000 7fc9a3400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9a40b0310 0x7fc9a40ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:60552/0 (socket says 172.21.15.46:60552) 2024-03-29T08:27:07.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.323+0000 7fc9a3400640 1 -- 172.21.15.46:0/2697421586 learned_addr learned my addr 172.21.15.46:0/2697421586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:07.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.323+0000 7fc9a3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc9a40aeee0 0x7fc9a40af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:07.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.323+0000 7fc9a3400640 1 -- 172.21.15.46:0/2697421586 >> v2:172.21.15.73:3300/0 conn(0x7fc9a40aeee0 msgr2=0x7fc9a40af370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:07.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.323+0000 7fc9a3400640 1 --2- 172.21.15.46:0/2697421586 >> v2:172.21.15.73:3300/0 conn(0x7fc9a40aeee0 0x7fc9a40af370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.323+0000 7fc9a3400640 1 -- 172.21.15.46:0/2697421586 >> v1:172.21.15.73:6789/0 conn(0x7fc9a40b06f0 legacy=0x7fc9a4069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:07.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.323+0000 7fc9a3400640 1 -- 172.21.15.46:0/2697421586 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a40ae6e0 con 0x7fc9a40b0310 2024-03-29T08:27:07.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.327+0000 7fc9a3400640 1 --2- 172.21.15.46:0/2697421586 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9a40b0310 0x7fc9a40ce590 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc994009870 tx=0x7fc99401c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faee7f851435bef9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.327+0000 7fc9a2a00640 1 -- 172.21.15.46:0/2697421586 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40b0ce0 con 0x7fc9a40b0310 2024-03-29T08:27:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.327+0000 7fc9a2a00640 1 -- 172.21.15.46:0/2697421586 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9940a9610 con 0x7fc9a40b0310 2024-03-29T08:27:07.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.327+0000 7fc9a2a00640 1 -- 172.21.15.46:0/2697421586 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9940a99a0 con 0x7fc9a40b0310 2024-03-29T08:27:07.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.327+0000 7fc9a8a00640 1 -- 172.21.15.46:0/2697421586 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9a40b0310 msgr2=0x7fc9a40ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:07.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.327+0000 7fc9a8a00640 1 --2- 172.21.15.46:0/2697421586 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9a40b0310 0x7fc9a40ce590 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc994009870 tx=0x7fc99401c530 comp rx=0 tx=0).stop 2024-03-29T08:27:07.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.327+0000 7fc9a8a00640 1 -- 172.21.15.46:0/2697421586 shutdown_connections 2024-03-29T08:27:07.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.327+0000 7fc9a8a00640 1 --2- 172.21.15.46:0/2697421586 >> v2:172.21.15.73:3300/0 conn(0x7fc9a40aeee0 0x7fc9a40af370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.327+0000 7fc9a8a00640 1 --2- 172.21.15.46:0/2697421586 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9a40b0310 0x7fc9a40ce590 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.327+0000 7fc9a8a00640 1 -- 172.21.15.46:0/2697421586 >> 172.21.15.46:0/2697421586 conn(0x7fc9a4098a40 msgr2=0x7fc9a40cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:07.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.327+0000 7fc9a8a00640 1 -- 172.21.15.46:0/2697421586 shutdown_connections 2024-03-29T08:27:07.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.327+0000 7fc9a8a00640 1 -- 172.21.15.46:0/2697421586 wait complete. 2024-03-29T08:27:07.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.327+0000 7fc9a8a00640 1 Processor -- start 2024-03-29T08:27:07.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.327+0000 7fc9a8a00640 1 -- start start 2024-03-29T08:27:07.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9a40aeee0 0x7fc9a40cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:07.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9a40b0310 0x7fc9a4155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:07.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a8a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9a40b06f0 0x7fc9a40ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:07.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a8a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a4155ec0 con 0x7fc9a40aeee0 2024-03-29T08:27:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a8a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a40caa40 con 0x7fc9a40b0310 2024-03-29T08:27:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a8a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a40cac70 con 0x7fc9a40b06f0 2024-03-29T08:27:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9a40aeee0 0x7fc9a40cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9a40aeee0 0x7fc9a40cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57860/0 (socket says 172.21.15.46:57860) 2024-03-29T08:27:07.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a3400640 1 -- 172.21.15.46:0/2831567924 learned_addr learned my addr 172.21.15.46:0/2831567924 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:07.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc99aa00640 1 --2- 172.21.15.46:0/2831567924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9a40b0310 0x7fc9a4155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:07.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a3400640 1 -- 172.21.15.46:0/2831567924 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9a40b06f0 msgr2=0x7fc9a40ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:07.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a3e00640 1 --2- 172.21.15.46:0/2831567924 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9a40b06f0 0x7fc9a40ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a3400640 1 --2- 172.21.15.46:0/2831567924 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9a40b06f0 0x7fc9a40ca500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a3400640 1 -- 172.21.15.46:0/2831567924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9a40b0310 msgr2=0x7fc9a4155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a3400640 1 --2- 172.21.15.46:0/2831567924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9a40b0310 0x7fc9a4155980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a3400640 1 -- 172.21.15.46:0/2831567924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9940094d0 con 0x7fc9a40aeee0 2024-03-29T08:27:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc99aa00640 1 --2- 172.21.15.46:0/2831567924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9a40b0310 0x7fc9a4155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a3e00640 1 --2- 172.21.15.46:0/2831567924 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9a40b06f0 0x7fc9a40ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a3400640 1 --2- 172.21.15.46:0/2831567924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9a40aeee0 0x7fc9a40cc4d0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc99401abe0 tx=0x7fc9940192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc99be00640 1 -- 172.21.15.46:0/2831567924 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9940bb3f0 con 0x7fc9a40aeee0 2024-03-29T08:27:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc99be00640 1 -- 172.21.15.46:0/2831567924 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9940bb550 con 0x7fc9a40aeee0 2024-03-29T08:27:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a8a00640 1 -- 172.21.15.46:0/2831567924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a40caf50 con 0x7fc9a40aeee0 2024-03-29T08:27:07.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc99be00640 1 -- 172.21.15.46:0/2831567924 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9940bb8e0 con 0x7fc9a40aeee0 2024-03-29T08:27:07.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a8a00640 1 -- 172.21.15.46:0/2831567924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a40cb510 con 0x7fc9a40aeee0 2024-03-29T08:27:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.331+0000 7fc9a8a00640 1 -- 172.21.15.46:0/2831567924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a404e040 con 0x7fc9a40aeee0 2024-03-29T08:27:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.335+0000 7fc99be00640 1 -- 172.21.15.46:0/2831567924 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc9940bba40 con 0x7fc9a40aeee0 2024-03-29T08:27:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.335+0000 7fc99be00640 1 --2- 172.21.15.46:0/2831567924 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc980066690 0x7fc980068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.335+0000 7fc99be00640 1 -- 172.21.15.46:0/2831567924 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fc99411e650 con 0x7fc9a40aeee0 2024-03-29T08:27:07.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.335+0000 7fc99aa00640 1 --2- 172.21.15.46:0/2831567924 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc980066690 0x7fc980068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:07.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.335+0000 7fc99aa00640 1 --2- 172.21.15.46:0/2831567924 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc980066690 0x7fc980068b50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40b22a0 tx=0x7fc98c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:07.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.339+0000 7fc99be00640 1 -- 172.21.15.46:0/2831567924 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc9940eb910 con 0x7fc9a40aeee0 2024-03-29T08:27:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.503+0000 7fc9a8a00640 1 -- 172.21.15.46:0/2831567924 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9a41589b0 con 0x7fc9a40aeee0 2024-03-29T08:27:07.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.503+0000 7fc99be00640 1 -- 172.21.15.46:0/2831567924 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16315 (secure 0 0 0) 0x7fc9940eef30 con 0x7fc9a40aeee0 2024-03-29T08:27:07.506 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:07.506 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:26:22.851956+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:24:41.113618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-03-29T08:24:42.487380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/2051429508":"2024-03-30T08:26:22.851926+0000","172.21.15.46:6833/2051429508":"2024-03-30T08:26:22.851926+0000","172.21.15.73:6836/4166845858":"2024-03-30T08:25:02.334350+0000","172.21.15.73:6834/4166845858":"2024-03-30T08:25:02.334350+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:27:07.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.507+0000 7fc9a8a00640 1 -- 172.21.15.46:0/2831567924 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc980066690 msgr2=0x7fc980068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:07.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.507+0000 7fc9a8a00640 1 --2- 172.21.15.46:0/2831567924 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc980066690 0x7fc980068b50 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40b22a0 tx=0x7fc98c009040 comp rx=0 tx=0).stop 2024-03-29T08:27:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.507+0000 7fc9a8a00640 1 -- 172.21.15.46:0/2831567924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9a40aeee0 msgr2=0x7fc9a40cc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.507+0000 7fc9a8a00640 1 --2- 172.21.15.46:0/2831567924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9a40aeee0 0x7fc9a40cc4d0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc99401abe0 tx=0x7fc9940192d0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.507+0000 7fc9a8a00640 1 -- 172.21.15.46:0/2831567924 shutdown_connections 2024-03-29T08:27:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.507+0000 7fc9a8a00640 1 --2- 172.21.15.46:0/2831567924 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc980066690 0x7fc980068b50 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.507+0000 7fc9a8a00640 1 --2- 172.21.15.46:0/2831567924 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9a40b0310 0x7fc9a4155980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.507+0000 7fc9a8a00640 1 --2- 172.21.15.46:0/2831567924 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc9a40b06f0 0x7fc9a40ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.507+0000 7fc9a8a00640 1 --2- 172.21.15.46:0/2831567924 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9a40aeee0 0x7fc9a40cc4d0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.507+0000 7fc9a8a00640 1 -- 172.21.15.46:0/2831567924 >> 172.21.15.46:0/2831567924 conn(0x7fc9a4098a40 msgr2=0x7fc9a4158ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.507+0000 7fc9a8a00640 1 -- 172.21.15.46:0/2831567924 shutdown_connections 2024-03-29T08:27:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.507+0000 7fc9a8a00640 1 -- 172.21.15.46:0/2831567924 wait complete. 2024-03-29T08:27:07.530 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T08:27:07.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.667+0000 7fb2ed000640 1 Processor -- start 2024-03-29T08:27:07.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.667+0000 7fb2ed000640 1 -- start start 2024-03-29T08:27:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.671+0000 7fb2ed000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2e8158300 0x7fb2e806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.671+0000 7fb2ed000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb2e80709b0 0x7fb2e80af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.671+0000 7fb2ed000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e806c390 con 0x7fb2e80af530 2024-03-29T08:27:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.671+0000 7fb2ed000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e806d600 con 0x7fb2e8158300 2024-03-29T08:27:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.671+0000 7fb2ed000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e80afff0 con 0x7fb2e80709b0 2024-03-29T08:27:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.671+0000 7fb2e5e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2e8158300 0x7fb2e806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.671+0000 7fb2e5e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2e8158300 0x7fb2e806d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57872/0 (socket says 172.21.15.46:57872) 2024-03-29T08:27:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.671+0000 7fb2e5e00640 1 -- 172.21.15.46:0/1230151378 learned_addr learned my addr 172.21.15.46:0/1230151378 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.671+0000 7fb2e6800640 1 --2- 172.21.15.46:0/1230151378 >> v2:172.21.15.73:3300/0 conn(0x7fb2e80709b0 0x7fb2e80af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:07.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.671+0000 7fb2e5e00640 1 -- 172.21.15.46:0/1230151378 >> v2:172.21.15.73:3300/0 conn(0x7fb2e80709b0 msgr2=0x7fb2e80af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:07.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.671+0000 7fb2e5e00640 1 --2- 172.21.15.46:0/1230151378 >> v2:172.21.15.73:3300/0 conn(0x7fb2e80709b0 0x7fb2e80af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.671+0000 7fb2e5e00640 1 -- 172.21.15.46:0/1230151378 >> v1:172.21.15.73:6789/0 conn(0x7fb2e80af530 legacy=0x7fb2e80702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:07.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.671+0000 7fb2e5e00640 1 -- 172.21.15.46:0/1230151378 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e806c5e0 con 0x7fb2e8158300 2024-03-29T08:27:07.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.671+0000 7fb2e6800640 1 --2- 172.21.15.46:0/1230151378 >> v2:172.21.15.73:3300/0 conn(0x7fb2e80709b0 0x7fb2e80af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.675+0000 7fb2e5e00640 1 --2- 172.21.15.46:0/1230151378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e8158300 0x7fb2e806d0c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80b13b0 tx=0x7fb2d801c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f01f4fd77172e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.675+0000 7fb2e5400640 1 -- 172.21.15.46:0/1230151378 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d80a5720 con 0x7fb2e8158300 2024-03-29T08:27:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.675+0000 7fb2e5400640 1 -- 172.21.15.46:0/1230151378 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d80a5880 con 0x7fb2e8158300 2024-03-29T08:27:07.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.675+0000 7fb2e5400640 1 -- 172.21.15.46:0/1230151378 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d80a5c10 con 0x7fb2e8158300 2024-03-29T08:27:07.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.675+0000 7fb2ed000640 1 -- 172.21.15.46:0/1230151378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e8158300 msgr2=0x7fb2e806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:07.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.675+0000 7fb2ed000640 1 --2- 172.21.15.46:0/1230151378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e8158300 0x7fb2e806d0c0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80b13b0 tx=0x7fb2d801c860 comp rx=0 tx=0).stop 2024-03-29T08:27:07.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.675+0000 7fb2ed000640 1 -- 172.21.15.46:0/1230151378 shutdown_connections 2024-03-29T08:27:07.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.675+0000 7fb2ed000640 1 --2- 172.21.15.46:0/1230151378 >> v2:172.21.15.73:3300/0 conn(0x7fb2e80709b0 0x7fb2e80af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.675+0000 7fb2ed000640 1 --2- 172.21.15.46:0/1230151378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e8158300 0x7fb2e806d0c0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.675+0000 7fb2ed000640 1 -- 172.21.15.46:0/1230151378 >> 172.21.15.46:0/1230151378 conn(0x7fb2e8066080 msgr2=0x7fb2e8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:07.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.675+0000 7fb2ed000640 1 -- 172.21.15.46:0/1230151378 shutdown_connections 2024-03-29T08:27:07.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.675+0000 7fb2ed000640 1 -- 172.21.15.46:0/1230151378 wait complete. 2024-03-29T08:27:07.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.675+0000 7fb2ed000640 1 Processor -- start 2024-03-29T08:27:07.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.675+0000 7fb2ed000640 1 -- start start 2024-03-29T08:27:07.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2ed000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e80709b0 0x7fb2e80b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:07.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2ed000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2e80af530 0x7fb2e8151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:07.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2ed000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb2e8158300 0x7fb2e8164a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:07.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2ed000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e81531f0 con 0x7fb2e80709b0 2024-03-29T08:27:07.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2ed000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e81534d0 con 0x7fb2e80af530 2024-03-29T08:27:07.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2ed000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e81537b0 con 0x7fb2e8158300 2024-03-29T08:27:07.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2e5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e80709b0 0x7fb2e80b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:07.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2e5e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e80709b0 0x7fb2e80b2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57886/0 (socket says 172.21.15.46:57886) 2024-03-29T08:27:07.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2e5e00640 1 -- 172.21.15.46:0/2170452212 learned_addr learned my addr 172.21.15.46:0/2170452212 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:07.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2e6800640 1 --2- 172.21.15.46:0/2170452212 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb2e8158300 0x7fb2e8164a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:07.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2dd400640 1 --2- 172.21.15.46:0/2170452212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2e80af530 0x7fb2e8151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:07.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2e5e00640 1 -- 172.21.15.46:0/2170452212 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb2e8158300 msgr2=0x7fb2e8164a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:07.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2e5e00640 1 --2- 172.21.15.46:0/2170452212 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb2e8158300 0x7fb2e8164a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2e5e00640 1 -- 172.21.15.46:0/2170452212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2e80af530 msgr2=0x7fb2e8151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2e5e00640 1 --2- 172.21.15.46:0/2170452212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2e80af530 0x7fb2e8151230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2e5e00640 1 -- 172.21.15.46:0/2170452212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d8009b30 con 0x7fb2e80709b0 2024-03-29T08:27:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2dd400640 1 --2- 172.21.15.46:0/2170452212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2e80af530 0x7fb2e8151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2e5e00640 1 --2- 172.21.15.46:0/2170452212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e80709b0 0x7fb2e80b2510 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80af040 tx=0x7fb2d8019600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2dea00640 1 -- 172.21.15.46:0/2170452212 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d80b8670 con 0x7fb2e80709b0 2024-03-29T08:27:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2dea00640 1 -- 172.21.15.46:0/2170452212 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d80b87d0 con 0x7fb2e80709b0 2024-03-29T08:27:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2dea00640 1 -- 172.21.15.46:0/2170452212 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d80b8b40 con 0x7fb2e80709b0 2024-03-29T08:27:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2ed000640 1 -- 172.21.15.46:0/2170452212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e8153a90 con 0x7fb2e80709b0 2024-03-29T08:27:07.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2ed000640 1 -- 172.21.15.46:0/2170452212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e80b2a50 con 0x7fb2e80709b0 2024-03-29T08:27:07.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2ed000640 1 -- 172.21.15.46:0/2170452212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e8071a90 con 0x7fb2e80709b0 2024-03-29T08:27:07.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.679+0000 7fb2dea00640 1 -- 172.21.15.46:0/2170452212 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb2d80b8ca0 con 0x7fb2e80709b0 2024-03-29T08:27:07.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.683+0000 7fb2dea00640 1 --2- 172.21.15.46:0/2170452212 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb2c40666c0 0x7fb2c4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:07.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.683+0000 7fb2dea00640 1 -- 172.21.15.46:0/2170452212 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fb2d811f920 con 0x7fb2e80709b0 2024-03-29T08:27:07.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.683+0000 7fb2dd400640 1 --2- 172.21.15.46:0/2170452212 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb2c40666c0 0x7fb2c4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:07.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.683+0000 7fb2dd400640 1 --2- 172.21.15.46:0/2170452212 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb2c40666c0 0x7fb2c4068b80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80b0e60 tx=0x7fb2d0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.687+0000 7fb2dea00640 1 -- 172.21.15.46:0/2170452212 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb2d80ecb40 con 0x7fb2e80709b0 2024-03-29T08:27:07.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.879+0000 7fb2ed000640 1 -- 172.21.15.46:0/2170452212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb2e804e120 con 0x7fb2e80709b0 2024-03-29T08:27:07.883 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-03-29T08:27:07.879+0000 7fc16ae00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:27:07.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.891+0000 7fb2dea00640 1 -- 172.21.15.46:0/2170452212 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb2d80f0160 con 0x7fb2e80709b0 2024-03-29T08:27:07.893 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T08:27:07.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.891+0000 7fb2ed000640 1 -- 172.21.15.46:0/2170452212 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb2c40666c0 msgr2=0x7fb2c4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:07.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.891+0000 7fb2ed000640 1 --2- 172.21.15.46:0/2170452212 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb2c40666c0 0x7fb2c4068b80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80b0e60 tx=0x7fb2d0009040 comp rx=0 tx=0).stop 2024-03-29T08:27:07.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.891+0000 7fb2ed000640 1 -- 172.21.15.46:0/2170452212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e80709b0 msgr2=0x7fb2e80b2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:07.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.891+0000 7fb2ed000640 1 --2- 172.21.15.46:0/2170452212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e80709b0 0x7fb2e80b2510 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80af040 tx=0x7fb2d8019600 comp rx=0 tx=0).stop 2024-03-29T08:27:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.891+0000 7fb2ed000640 1 -- 172.21.15.46:0/2170452212 shutdown_connections 2024-03-29T08:27:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.891+0000 7fb2ed000640 1 --2- 172.21.15.46:0/2170452212 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb2c40666c0 0x7fb2c4068b80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.891+0000 7fb2ed000640 1 --2- 172.21.15.46:0/2170452212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2e80af530 0x7fb2e8151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.891+0000 7fb2ed000640 1 --2- 172.21.15.46:0/2170452212 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb2e8158300 0x7fb2e8164a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.891+0000 7fb2ed000640 1 --2- 172.21.15.46:0/2170452212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2e80709b0 0x7fb2e80b2510 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.891+0000 7fb2ed000640 1 -- 172.21.15.46:0/2170452212 >> 172.21.15.46:0/2170452212 conn(0x7fb2e8066080 msgr2=0x7fb2e80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.891+0000 7fb2ed000640 1 -- 172.21.15.46:0/2170452212 shutdown_connections 2024-03-29T08:27:07.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:07.891+0000 7fb2ed000640 1 -- 172.21.15.46:0/2170452212 wait complete. 2024-03-29T08:27:07.900 INFO:tasks.ceph.mds.c.smithi046.stderr: -2130> 2024-03-29T08:26:28.867+0000 7f5018800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:27:07.901 INFO:tasks.ceph.mds.c.smithi046.stderr: -1765> 2024-03-29T08:26:30.351+0000 7f5014200640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55bb7d366680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-03-29T08:27:07.915 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T08:27:07.945 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-03-29T08:27:08.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.055+0000 7fdd6f800640 1 Processor -- start 2024-03-29T08:27:08.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.059+0000 7fdd6f800640 1 -- start start 2024-03-29T08:27:08.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.059+0000 7fdd6f800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd6815a8a0 0x7fdd680cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:08.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.059+0000 7fdd6f800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd6815b390 0x7fdd680b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:08.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.059+0000 7fdd6f800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd680afdf0 con 0x7fdd68099770 2024-03-29T08:27:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.059+0000 7fdd6f800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6815b7a0 con 0x7fdd6815a8a0 2024-03-29T08:27:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.059+0000 7fdd6f800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd680b0090 con 0x7fdd6815b390 2024-03-29T08:27:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.059+0000 7fdd6ce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd6815a8a0 0x7fdd680cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.059+0000 7fdd6ce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd6815a8a0 0x7fdd680cee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57910/0 (socket says 172.21.15.46:57910) 2024-03-29T08:27:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.059+0000 7fdd6ce00640 1 -- 172.21.15.46:0/4171548859 learned_addr learned my addr 172.21.15.46:0/4171548859 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.059+0000 7fdd67e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdd68099770 0x7fdd6815ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33722/0 (socket says 172.21.15.46:33722) 2024-03-29T08:27:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.059+0000 7fdd6d800640 1 --2- 172.21.15.46:0/4171548859 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd6815b390 0x7fdd680b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.059+0000 7fdd6ce00640 1 -- 172.21.15.46:0/4171548859 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd6815b390 msgr2=0x7fdd680b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.059+0000 7fdd6ce00640 1 --2- 172.21.15.46:0/4171548859 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd6815b390 0x7fdd680b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.059+0000 7fdd6ce00640 1 -- 172.21.15.46:0/4171548859 >> v1:172.21.15.46:6789/0 conn(0x7fdd68099770 legacy=0x7fdd6815ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.059+0000 7fdd6ce00640 1 -- 172.21.15.46:0/4171548859 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd680b0370 con 0x7fdd6815a8a0 2024-03-29T08:27:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.059+0000 7fdd67400640 1 -- 172.21.15.46:0/4171548859 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438318586 0 0) 0x7fdd680afdf0 con 0x7fdd68099770 2024-03-29T08:27:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.063+0000 7fdd6ce00640 1 --2- 172.21.15.46:0/4171548859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd6815a8a0 0x7fdd680cee30 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009870 tx=0x7fdd5c01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd8479983289e1c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.063+0000 7fdd67400640 1 -- 172.21.15.46:0/4171548859 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c0a95f0 con 0x7fdd6815a8a0 2024-03-29T08:27:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.063+0000 7fdd67400640 1 -- 172.21.15.46:0/4171548859 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5c0a9750 con 0x7fdd6815a8a0 2024-03-29T08:27:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.063+0000 7fdd67400640 1 -- 172.21.15.46:0/4171548859 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c0a9ae0 con 0x7fdd6815a8a0 2024-03-29T08:27:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.063+0000 7fdd6f800640 1 -- 172.21.15.46:0/4171548859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd6815a8a0 msgr2=0x7fdd680cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.063+0000 7fdd6f800640 1 --2- 172.21.15.46:0/4171548859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd6815a8a0 0x7fdd680cee30 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c009870 tx=0x7fdd5c01c530 comp rx=0 tx=0).stop 2024-03-29T08:27:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.063+0000 7fdd6f800640 1 -- 172.21.15.46:0/4171548859 shutdown_connections 2024-03-29T08:27:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.063+0000 7fdd6f800640 1 --2- 172.21.15.46:0/4171548859 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd6815b390 0x7fdd680b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.063+0000 7fdd6f800640 1 --2- 172.21.15.46:0/4171548859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd6815a8a0 0x7fdd680cee30 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.063+0000 7fdd6f800640 1 -- 172.21.15.46:0/4171548859 >> 172.21.15.46:0/4171548859 conn(0x7fdd680d0730 msgr2=0x7fdd680d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.063+0000 7fdd6f800640 1 -- 172.21.15.46:0/4171548859 shutdown_connections 2024-03-29T08:27:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.063+0000 7fdd6f800640 1 -- 172.21.15.46:0/4171548859 wait complete. 2024-03-29T08:27:08.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.063+0000 7fdd6f800640 1 Processor -- start 2024-03-29T08:27:08.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.063+0000 7fdd6f800640 1 -- start start 2024-03-29T08:27:08.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd6f800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd68099770 0x7fdd680ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:08.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd6f800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd6815a8a0 0x7fdd681633d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd6f800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6815b390 0x7fdd680ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd6f800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd68164710 con 0x7fdd6815a8a0 2024-03-29T08:27:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd6f800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd68163e70 con 0x7fdd6815b390 2024-03-29T08:27:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd6f800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd68164150 con 0x7fdd68099770 2024-03-29T08:27:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd67e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd6815a8a0 0x7fdd681633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd67e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd6815a8a0 0x7fdd681633d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57912/0 (socket says 172.21.15.46:57912) 2024-03-29T08:27:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd67e00640 1 -- 172.21.15.46:0/1362786578 learned_addr learned my addr 172.21.15.46:0/1362786578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd6d800640 1 --2- 172.21.15.46:0/1362786578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6815b390 0x7fdd680ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd6ce00640 1 --2- 172.21.15.46:0/1362786578 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd68099770 0x7fdd680ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd67e00640 1 -- 172.21.15.46:0/1362786578 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd68099770 msgr2=0x7fdd680ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd67e00640 1 --2- 172.21.15.46:0/1362786578 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd68099770 0x7fdd680ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd67e00640 1 -- 172.21.15.46:0/1362786578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6815b390 msgr2=0x7fdd680ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd67e00640 1 --2- 172.21.15.46:0/1362786578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6815b390 0x7fdd680ae4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd67e00640 1 -- 172.21.15.46:0/1362786578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c0094d0 con 0x7fdd6815a8a0 2024-03-29T08:27:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd67e00640 1 --2- 172.21.15.46:0/1362786578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd6815a8a0 0x7fdd681633d0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400c4b0 tx=0x7fdd5400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd64c00640 1 -- 172.21.15.46:0/1362786578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54007480 con 0x7fdd6815a8a0 2024-03-29T08:27:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd64c00640 1 -- 172.21.15.46:0/1362786578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd540075e0 con 0x7fdd6815a8a0 2024-03-29T08:27:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd6f800640 1 -- 172.21.15.46:0/1362786578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd68164430 con 0x7fdd6815a8a0 2024-03-29T08:27:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd64c00640 1 -- 172.21.15.46:0/1362786578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54004a30 con 0x7fdd6815a8a0 2024-03-29T08:27:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd6f800640 1 -- 172.21.15.46:0/1362786578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6814c450 con 0x7fdd6815a8a0 2024-03-29T08:27:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd6f800640 1 -- 172.21.15.46:0/1362786578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6809e6b0 con 0x7fdd6815a8a0 2024-03-29T08:27:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.067+0000 7fdd64c00640 1 -- 172.21.15.46:0/1362786578 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdd54004b90 con 0x7fdd6815a8a0 2024-03-29T08:27:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.071+0000 7fdd64c00640 1 --2- 172.21.15.46:0/1362786578 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd48066690 0x7fdd48068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.071+0000 7fdd64c00640 1 -- 172.21.15.46:0/1362786578 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fdd54083640 con 0x7fdd6815a8a0 2024-03-29T08:27:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.071+0000 7fdd6ce00640 1 --2- 172.21.15.46:0/1362786578 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd48066690 0x7fdd48068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:08.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.071+0000 7fdd6ce00640 1 --2- 172.21.15.46:0/1362786578 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd48066690 0x7fdd48068b50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0ae000 tx=0x7fdd5c0b3040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:08.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.075+0000 7fdd64c00640 1 -- 172.21.15.46:0/1362786578 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdd54050ab0 con 0x7fdd6815a8a0 2024-03-29T08:27:08.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.271+0000 7fdd6f800640 1 -- 172.21.15.46:0/1362786578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdd6804e120 con 0x7fdd6815a8a0 2024-03-29T08:27:08.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.895+0000 7fdd64c00640 1 -- 172.21.15.46:0/1362786578 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v71) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdd540540d0 con 0x7fdd6815a8a0 2024-03-29T08:27:08.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.895+0000 7fdd6f800640 1 -- 172.21.15.46:0/1362786578 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd48066690 msgr2=0x7fdd48068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:08.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.895+0000 7fdd6f800640 1 --2- 172.21.15.46:0/1362786578 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd48066690 0x7fdd48068b50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c0ae000 tx=0x7fdd5c0b3040 comp rx=0 tx=0).stop 2024-03-29T08:27:08.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.895+0000 7fdd6f800640 1 -- 172.21.15.46:0/1362786578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd6815a8a0 msgr2=0x7fdd681633d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:08.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.895+0000 7fdd6f800640 1 --2- 172.21.15.46:0/1362786578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd6815a8a0 0x7fdd681633d0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400c4b0 tx=0x7fdd5400c970 comp rx=0 tx=0).stop 2024-03-29T08:27:08.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.895+0000 7fdd6f800640 1 -- 172.21.15.46:0/1362786578 shutdown_connections 2024-03-29T08:27:08.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.895+0000 7fdd6f800640 1 --2- 172.21.15.46:0/1362786578 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd48066690 0x7fdd48068b50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:08.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.895+0000 7fdd6f800640 1 --2- 172.21.15.46:0/1362786578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6815b390 0x7fdd680ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:08.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.895+0000 7fdd6f800640 1 --2- 172.21.15.46:0/1362786578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd6815a8a0 0x7fdd681633d0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:08.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.895+0000 7fdd6f800640 1 --2- 172.21.15.46:0/1362786578 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd68099770 0x7fdd680ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:08.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.895+0000 7fdd6f800640 1 -- 172.21.15.46:0/1362786578 >> 172.21.15.46:0/1362786578 conn(0x7fdd680d0730 msgr2=0x7fdd680ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:08.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.895+0000 7fdd6f800640 1 -- 172.21.15.46:0/1362786578 shutdown_connections 2024-03-29T08:27:08.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:08.895+0000 7fdd6f800640 1 -- 172.21.15.46:0/1362786578 wait complete. 2024-03-29T08:27:08.924 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T08:27:09.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6db800640 1 Processor -- start 2024-03-29T08:27:09.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6db800640 1 -- start start 2024-03-29T08:27:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6db800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6d41598a0 0x7fb6d40cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:09.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6db800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb6d40b1470 0x7fb6d415d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:09.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6db800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d415da60 0x7fb6d4159c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:09.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6db800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d415de80 con 0x7fb6d41598a0 2024-03-29T08:27:09.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6db800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d40b1bc0 con 0x7fb6d40b1470 2024-03-29T08:27:09.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6db800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d40b1ea0 con 0x7fb6d415da60 2024-03-29T08:27:09.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6d8e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6d41598a0 0x7fb6d40cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:09.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6d8e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6d41598a0 0x7fb6d40cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57916/0 (socket says 172.21.15.46:57916) 2024-03-29T08:27:09.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6d8e00640 1 -- 172.21.15.46:0/4183334525 learned_addr learned my addr 172.21.15.46:0/4183334525 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:09.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6cbe00640 1 --2- 172.21.15.46:0/4183334525 >> v2:172.21.15.73:3300/0 conn(0x7fb6d40b1470 0x7fb6d415d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:09.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6d9800640 1 --2- 172.21.15.46:0/4183334525 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d415da60 0x7fb6d4159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:09.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6d8e00640 1 -- 172.21.15.46:0/4183334525 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d415da60 msgr2=0x7fb6d4159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:09.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6d8e00640 1 --2- 172.21.15.46:0/4183334525 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d415da60 0x7fb6d4159c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:09.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6d8e00640 1 -- 172.21.15.46:0/4183334525 >> v2:172.21.15.73:3300/0 conn(0x7fb6d40b1470 msgr2=0x7fb6d415d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:09.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6d8e00640 1 --2- 172.21.15.46:0/4183334525 >> v2:172.21.15.73:3300/0 conn(0x7fb6d40b1470 0x7fb6d415d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:09.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.119+0000 7fb6d8e00640 1 -- 172.21.15.46:0/4183334525 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d40b2180 con 0x7fb6d41598a0 2024-03-29T08:27:09.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.123+0000 7fb6d8e00640 1 --2- 172.21.15.46:0/4183334525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d41598a0 0x7fb6d40cadb0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fb6d415e020 tx=0x7fb6c401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177f752983cc7440 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:09.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.123+0000 7fb6d3e00640 1 -- 172.21.15.46:0/4183334525 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c40aa610 con 0x7fb6d41598a0 2024-03-29T08:27:09.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.123+0000 7fb6d3e00640 1 -- 172.21.15.46:0/4183334525 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c40a6040 con 0x7fb6d41598a0 2024-03-29T08:27:09.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.123+0000 7fb6d3e00640 1 -- 172.21.15.46:0/4183334525 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c40aa990 con 0x7fb6d41598a0 2024-03-29T08:27:09.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.123+0000 7fb6db800640 1 -- 172.21.15.46:0/4183334525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d41598a0 msgr2=0x7fb6d40cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:09.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.123+0000 7fb6db800640 1 --2- 172.21.15.46:0/4183334525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d41598a0 0x7fb6d40cadb0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fb6d415e020 tx=0x7fb6c401c4f0 comp rx=0 tx=0).stop 2024-03-29T08:27:09.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.123+0000 7fb6db800640 1 -- 172.21.15.46:0/4183334525 shutdown_connections 2024-03-29T08:27:09.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.123+0000 7fb6db800640 1 --2- 172.21.15.46:0/4183334525 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d415da60 0x7fb6d4159c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:09.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.123+0000 7fb6db800640 1 --2- 172.21.15.46:0/4183334525 >> v2:172.21.15.73:3300/0 conn(0x7fb6d40b1470 0x7fb6d415d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:09.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.123+0000 7fb6db800640 1 --2- 172.21.15.46:0/4183334525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d41598a0 0x7fb6d40cadb0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:09.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.123+0000 7fb6db800640 1 -- 172.21.15.46:0/4183334525 >> 172.21.15.46:0/4183334525 conn(0x7fb6d40b42d0 msgr2=0x7fb6d4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:09.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.123+0000 7fb6db800640 1 -- 172.21.15.46:0/4183334525 shutdown_connections 2024-03-29T08:27:09.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.123+0000 7fb6db800640 1 -- 172.21.15.46:0/4183334525 wait complete. 2024-03-29T08:27:09.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6db800640 1 Processor -- start 2024-03-29T08:27:09.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6db800640 1 -- start start 2024-03-29T08:27:09.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6db800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d40b1470 0x7fb6d4149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:09.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6db800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d41598a0 0x7fb6d4148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:09.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6db800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d415da60 0x7fb6d40cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:09.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6db800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d4155550 con 0x7fb6d415da60 2024-03-29T08:27:09.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6db800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d4155830 con 0x7fb6d40b1470 2024-03-29T08:27:09.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6db800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d4157220 con 0x7fb6d41598a0 2024-03-29T08:27:09.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6d9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d415da60 0x7fb6d40cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:09.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6d9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d415da60 0x7fb6d40cc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57920/0 (socket says 172.21.15.46:57920) 2024-03-29T08:27:09.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6d9800640 1 -- 172.21.15.46:0/2744809348 learned_addr learned my addr 172.21.15.46:0/2744809348 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:09.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6cbe00640 1 --2- 172.21.15.46:0/2744809348 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d41598a0 0x7fb6d4148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:09.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6d8e00640 1 --2- 172.21.15.46:0/2744809348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d40b1470 0x7fb6d4149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:09.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6d9800640 1 -- 172.21.15.46:0/2744809348 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d41598a0 msgr2=0x7fb6d4148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:09.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6d9800640 1 --2- 172.21.15.46:0/2744809348 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d41598a0 0x7fb6d4148d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:09.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6d9800640 1 -- 172.21.15.46:0/2744809348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d40b1470 msgr2=0x7fb6d4149560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:09.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6d9800640 1 --2- 172.21.15.46:0/2744809348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d40b1470 0x7fb6d4149560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:09.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6d9800640 1 -- 172.21.15.46:0/2744809348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c40094d0 con 0x7fb6d415da60 2024-03-29T08:27:09.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6d9800640 1 --2- 172.21.15.46:0/2744809348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d415da60 0x7fb6d40cc600 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc007e60 tx=0x7fb6cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:09.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6d1600640 1 -- 172.21.15.46:0/2744809348 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc014420 con 0x7fb6d415da60 2024-03-29T08:27:09.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6d1600640 1 -- 172.21.15.46:0/2744809348 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6cc0145a0 con 0x7fb6d415da60 2024-03-29T08:27:09.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6d1600640 1 -- 172.21.15.46:0/2744809348 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc014950 con 0x7fb6d415da60 2024-03-29T08:27:09.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6db800640 1 -- 172.21.15.46:0/2744809348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d4157500 con 0x7fb6d415da60 2024-03-29T08:27:09.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6db800640 1 -- 172.21.15.46:0/2744809348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d4156e40 con 0x7fb6d415da60 2024-03-29T08:27:09.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.127+0000 7fb6db800640 1 -- 172.21.15.46:0/2744809348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d41576f0 con 0x7fb6d415da60 2024-03-29T08:27:09.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.131+0000 7fb6d1600640 1 -- 172.21.15.46:0/2744809348 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb6cc014ab0 con 0x7fb6d415da60 2024-03-29T08:27:09.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.131+0000 7fb6d1600640 1 --2- 172.21.15.46:0/2744809348 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6b0066690 0x7fb6b0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:09.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.131+0000 7fb6d1600640 1 -- 172.21.15.46:0/2744809348 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fb6cc084c90 con 0x7fb6d415da60 2024-03-29T08:27:09.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.131+0000 7fb6d8e00640 1 --2- 172.21.15.46:0/2744809348 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6b0066690 0x7fb6b0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:09.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.131+0000 7fb6d8e00640 1 --2- 172.21.15.46:0/2744809348 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6b0066690 0x7fb6b0068b50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40ae000 tx=0x7fb6c40a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:09.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.135+0000 7fb6d1600640 1 -- 172.21.15.46:0/2744809348 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb6cc051e60 con 0x7fb6d415da60 2024-03-29T08:27:09.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:09.299+0000 7fb6db800640 1 -- 172.21.15.46:0/2744809348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb6d4155690 con 0x7fb6d415da60 2024-03-29T08:27:09.471 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:27:09.476 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:27:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.059+0000 7fb6d1600640 1 -- 172.21.15.46:0/2744809348 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v96) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb6d4155690 con 0x7fb6d415da60 2024-03-29T08:27:10.062 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-03-29T08:27:10.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.063+0000 7fb6db800640 1 -- 172.21.15.46:0/2744809348 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6b0066690 msgr2=0x7fb6b0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:10.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.063+0000 7fb6db800640 1 --2- 172.21.15.46:0/2744809348 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6b0066690 0x7fb6b0068b50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40ae000 tx=0x7fb6c40a7000 comp rx=0 tx=0).stop 2024-03-29T08:27:10.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.063+0000 7fb6db800640 1 -- 172.21.15.46:0/2744809348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d415da60 msgr2=0x7fb6d40cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.063+0000 7fb6db800640 1 --2- 172.21.15.46:0/2744809348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d415da60 0x7fb6d40cc600 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc007e60 tx=0x7fb6cc00e4b0 comp rx=0 tx=0).stop 2024-03-29T08:27:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.063+0000 7fb6db800640 1 -- 172.21.15.46:0/2744809348 shutdown_connections 2024-03-29T08:27:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.063+0000 7fb6db800640 1 --2- 172.21.15.46:0/2744809348 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb6b0066690 0x7fb6b0068b50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.063+0000 7fb6db800640 1 --2- 172.21.15.46:0/2744809348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6d415da60 0x7fb6d40cc600 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.063+0000 7fb6db800640 1 --2- 172.21.15.46:0/2744809348 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb6d41598a0 0x7fb6d4148d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.063+0000 7fb6db800640 1 --2- 172.21.15.46:0/2744809348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6d40b1470 0x7fb6d4149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:10.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.063+0000 7fb6db800640 1 -- 172.21.15.46:0/2744809348 >> 172.21.15.46:0/2744809348 conn(0x7fb6d40b42d0 msgr2=0x7fb6d4070180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:10.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.063+0000 7fb6db800640 1 -- 172.21.15.46:0/2744809348 shutdown_connections 2024-03-29T08:27:10.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.063+0000 7fb6db800640 1 -- 172.21.15.46:0/2744809348 wait complete. 2024-03-29T08:27:10.091 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T08:27:10.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f0361c00640 1 Processor -- start 2024-03-29T08:27:10.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f0361c00640 1 -- start start 2024-03-29T08:27:10.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f0361c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f035c15a970 0x7f035c0d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:10.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f0361c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f035c0d0b00 0x7f035c0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:10.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f0361c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035c0cba70 con 0x7f035c0b0b30 2024-03-29T08:27:10.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f0361c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f035c0d0f60 con 0x7f035c0d0b00 2024-03-29T08:27:10.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f0361c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035c0b1320 con 0x7f035c15a970 2024-03-29T08:27:10.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f035b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f035c0b0b30 0x7f035c0cd060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33738/0 (socket says 172.21.15.46:33738) 2024-03-29T08:27:10.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f035b400640 1 -- 172.21.15.46:0/4113987519 learned_addr learned my addr 172.21.15.46:0/4113987519 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f035aa00640 1 --2- 172.21.15.46:0/4113987519 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f035c15a970 0x7f035c0d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f0352000640 1 --2- 172.21.15.46:0/4113987519 >> v2:172.21.15.73:3300/0 conn(0x7f035c0d0b00 0x7f035c0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f035a000640 1 -- 172.21.15.46:0/4113987519 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1019388187 0 0) 0x7f035c0cba70 con 0x7f035c0b0b30 2024-03-29T08:27:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f035a000640 1 -- 172.21.15.46:0/4113987519 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0340003530 con 0x7f035c0b0b30 2024-03-29T08:27:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f0352000640 1 -- 172.21.15.46:0/4113987519 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f035c15a970 msgr2=0x7f035c0d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f0352000640 1 --2- 172.21.15.46:0/4113987519 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f035c15a970 0x7f035c0d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f0352000640 1 -- 172.21.15.46:0/4113987519 >> v1:172.21.15.46:6789/0 conn(0x7f035c0b0b30 legacy=0x7f035c0cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f0352000640 1 -- 172.21.15.46:0/4113987519 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c0cb420 con 0x7f035c0d0b00 2024-03-29T08:27:10.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.239+0000 7f035aa00640 1 --2- 172.21.15.46:0/4113987519 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f035c15a970 0x7f035c0d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:27:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.243+0000 7f0352000640 1 --2- 172.21.15.46:0/4113987519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f035c0d0b00 0x7f035c0b05f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0344009d90 tx=0x7f034401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deef47e762f9b495 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.243+0000 7f035a000640 1 -- 172.21.15.46:0/4113987519 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03440a5900 con 0x7f035c0d0b00 2024-03-29T08:27:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.243+0000 7f035a000640 1 -- 172.21.15.46:0/4113987519 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03440a5a60 con 0x7f035c0d0b00 2024-03-29T08:27:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.243+0000 7f035a000640 1 -- 172.21.15.46:0/4113987519 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03440a5df0 con 0x7f035c0d0b00 2024-03-29T08:27:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.243+0000 7f0361c00640 1 -- 172.21.15.46:0/4113987519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f035c0d0b00 msgr2=0x7f035c0b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.243+0000 7f0361c00640 1 --2- 172.21.15.46:0/4113987519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f035c0d0b00 0x7f035c0b05f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0344009d90 tx=0x7f034401c940 comp rx=0 tx=0).stop 2024-03-29T08:27:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.243+0000 7f0361c00640 1 -- 172.21.15.46:0/4113987519 shutdown_connections 2024-03-29T08:27:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.243+0000 7f0361c00640 1 --2- 172.21.15.46:0/4113987519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f035c0d0b00 0x7f035c0b05f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.243+0000 7f0361c00640 1 --2- 172.21.15.46:0/4113987519 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f035c15a970 0x7f035c0d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.243+0000 7f0361c00640 1 -- 172.21.15.46:0/4113987519 >> 172.21.15.46:0/4113987519 conn(0x7f035c089fd0 msgr2=0x7f035c08a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.243+0000 7f0361c00640 1 -- 172.21.15.46:0/4113987519 shutdown_connections 2024-03-29T08:27:10.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.243+0000 7f0361c00640 1 -- 172.21.15.46:0/4113987519 wait complete. 2024-03-29T08:27:10.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f0361c00640 1 Processor -- start 2024-03-29T08:27:10.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f0361c00640 1 -- start start 2024-03-29T08:27:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f0361c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c0b0b30 0x7f035c0c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f0361c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f035c0d0b00 0x7f035c153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f0361c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f035c15a970 0x7f035c0c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f0361c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c153dd0 con 0x7f035c0b0b30 2024-03-29T08:27:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f0361c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c0c5590 con 0x7f035c0d0b00 2024-03-29T08:27:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f0361c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035c0c4980 con 0x7f035c15a970 2024-03-29T08:27:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f035aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c0b0b30 0x7f035c0c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f035aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c0b0b30 0x7f035c0c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57936/0 (socket says 172.21.15.46:57936) 2024-03-29T08:27:10.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f035aa00640 1 -- 172.21.15.46:0/1001286916 learned_addr learned my addr 172.21.15.46:0/1001286916 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f0352000640 1 --2- 172.21.15.46:0/1001286916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f035c0d0b00 0x7f035c153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f035b400640 1 --2- 172.21.15.46:0/1001286916 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f035c15a970 0x7f035c0c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f035aa00640 1 -- 172.21.15.46:0/1001286916 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f035c15a970 msgr2=0x7f035c0c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f035aa00640 1 --2- 172.21.15.46:0/1001286916 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f035c15a970 0x7f035c0c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f035aa00640 1 -- 172.21.15.46:0/1001286916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f035c0d0b00 msgr2=0x7f035c153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f035aa00640 1 --2- 172.21.15.46:0/1001286916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f035c0d0b00 0x7f035c153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f035aa00640 1 -- 172.21.15.46:0/1001286916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c009420 con 0x7f035c0b0b30 2024-03-29T08:27:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f035aa00640 1 --2- 172.21.15.46:0/1001286916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c0b0b30 0x7f035c0c5a20 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f034c00e980 tx=0x7f034c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f0353400640 1 -- 172.21.15.46:0/1001286916 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c00ca00 con 0x7f035c0b0b30 2024-03-29T08:27:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f0353400640 1 -- 172.21.15.46:0/1001286916 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034c00cb60 con 0x7f035c0b0b30 2024-03-29T08:27:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f0353400640 1 -- 172.21.15.46:0/1001286916 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c013470 con 0x7f035c0b0b30 2024-03-29T08:27:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f0361c00640 1 -- 172.21.15.46:0/1001286916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0344009a70 con 0x7f035c0b0b30 2024-03-29T08:27:10.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.247+0000 7f0361c00640 1 -- 172.21.15.46:0/1001286916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035c0c4f40 con 0x7f035c0b0b30 2024-03-29T08:27:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.251+0000 7f0361c00640 1 -- 172.21.15.46:0/1001286916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f035c066c90 con 0x7f035c0b0b30 2024-03-29T08:27:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.251+0000 7f0353400640 1 -- 172.21.15.46:0/1001286916 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f034c0135d0 con 0x7f035c0b0b30 2024-03-29T08:27:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.251+0000 7f0353400640 1 --2- 172.21.15.46:0/1001286916 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0338066690 0x7f0338068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:10.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.251+0000 7f0353400640 1 -- 172.21.15.46:0/1001286916 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5529+0+0 (secure 0 0 0) 0x7f034c084350 con 0x7f035c0b0b30 2024-03-29T08:27:10.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.251+0000 7f0352000640 1 --2- 172.21.15.46:0/1001286916 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0338066690 0x7f0338068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:10.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.251+0000 7f0352000640 1 --2- 172.21.15.46:0/1001286916 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0338066690 0x7f0338068b50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f03440af000 tx=0x7f03440a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:10.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.255+0000 7f0353400640 1 -- 172.21.15.46:0/1001286916 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f034c051780 con 0x7f035c0b0b30 2024-03-29T08:27:10.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:10.415+0000 7f0361c00640 1 -- 172.21.15.46:0/1001286916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f035c070100 con 0x7f035c0b0b30 2024-03-29T08:27:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.075+0000 7f0353400640 1 -- 172.21.15.46:0/1001286916 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v97) v1 ==== 169+0+0 (secure 0 0 0) 0x7f034c054da0 con 0x7f035c0b0b30 2024-03-29T08:27:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-03-29T08:27:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.079+0000 7f0361c00640 1 -- 172.21.15.46:0/1001286916 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0338066690 msgr2=0x7f0338068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.079+0000 7f0361c00640 1 --2- 172.21.15.46:0/1001286916 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0338066690 0x7f0338068b50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f03440af000 tx=0x7f03440a7070 comp rx=0 tx=0).stop 2024-03-29T08:27:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.079+0000 7f0361c00640 1 -- 172.21.15.46:0/1001286916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c0b0b30 msgr2=0x7f035c0c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.079+0000 7f0361c00640 1 --2- 172.21.15.46:0/1001286916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c0b0b30 0x7f035c0c5a20 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f034c00e980 tx=0x7f034c00ee40 comp rx=0 tx=0).stop 2024-03-29T08:27:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.079+0000 7f0361c00640 1 -- 172.21.15.46:0/1001286916 shutdown_connections 2024-03-29T08:27:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.079+0000 7f0361c00640 1 --2- 172.21.15.46:0/1001286916 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0338066690 0x7f0338068b50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.079+0000 7f0361c00640 1 --2- 172.21.15.46:0/1001286916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f035c0d0b00 0x7f035c153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.079+0000 7f0361c00640 1 --2- 172.21.15.46:0/1001286916 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f035c15a970 0x7f035c0c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.079+0000 7f0361c00640 1 --2- 172.21.15.46:0/1001286916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f035c0b0b30 0x7f035c0c5a20 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:11.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.079+0000 7f0361c00640 1 -- 172.21.15.46:0/1001286916 >> 172.21.15.46:0/1001286916 conn(0x7f035c089fd0 msgr2=0x7f035c0b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:11.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.079+0000 7f0361c00640 1 -- 172.21.15.46:0/1001286916 shutdown_connections 2024-03-29T08:27:11.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.079+0000 7f0361c00640 1 -- 172.21.15.46:0/1001286916 wait complete. 2024-03-29T08:27:11.108 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-29T08:27:11.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.251+0000 7f0cc9000640 1 Processor -- start 2024-03-29T08:27:11.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.251+0000 7f0cc9000640 1 -- start start 2024-03-29T08:27:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.255+0000 7f0cc9000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cc4143180 0x7f0cc40adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.255+0000 7f0cc9000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cc40ae1e0 0x7f0cc40cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.255+0000 7f0cc9000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cc40afe80 con 0x7f0cc4142da0 2024-03-29T08:27:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.255+0000 7f0cc9000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc406cd80 con 0x7f0cc40ae1e0 2024-03-29T08:27:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.255+0000 7f0cc9000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc406d020 con 0x7f0cc4143180 2024-03-29T08:27:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.255+0000 7f0cc1e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0cc4142da0 0x7f0cc40abac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33746/0 (socket says 172.21.15.46:33746) 2024-03-29T08:27:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.255+0000 7f0cc1e00640 1 -- 172.21.15.46:0/1424455782 learned_addr learned my addr 172.21.15.46:0/1424455782 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:11.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.255+0000 7f0cc2800640 1 --2- 172.21.15.46:0/1424455782 >> v2:172.21.15.46:3300/0 conn(0x7f0cc40ae1e0 0x7f0cc40cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:11.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.255+0000 7f0cb9400640 1 --2- 172.21.15.46:0/1424455782 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cc4143180 0x7f0cc40adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:11.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.255+0000 7f0cc2800640 1 -- 172.21.15.46:0/1424455782 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cc4143180 msgr2=0x7f0cc40adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:11.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.255+0000 7f0cc2800640 1 --2- 172.21.15.46:0/1424455782 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cc4143180 0x7f0cc40adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:11.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.255+0000 7f0cc2800640 1 -- 172.21.15.46:0/1424455782 >> v1:172.21.15.46:6789/0 conn(0x7f0cc4142da0 legacy=0x7f0cc40abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:11.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.255+0000 7f0cc2800640 1 -- 172.21.15.46:0/1424455782 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc40b00f0 con 0x7f0cc40ae1e0 2024-03-29T08:27:11.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.255+0000 7f0cc2800640 1 --2- 172.21.15.46:0/1424455782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc40ae1e0 0x7f0cc40cb790 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009d90 tx=0x7f0cbc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ff410c3a14fafd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:11.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc1400640 1 -- 172.21.15.46:0/1424455782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc0a58c0 con 0x7f0cc40ae1e0 2024-03-29T08:27:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc1400640 1 -- 172.21.15.46:0/1424455782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cbc0a5a20 con 0x7f0cc40ae1e0 2024-03-29T08:27:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc1400640 1 -- 172.21.15.46:0/1424455782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc0a5db0 con 0x7f0cc40ae1e0 2024-03-29T08:27:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc9000640 1 -- 172.21.15.46:0/1424455782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc40ae1e0 msgr2=0x7f0cc40cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc9000640 1 --2- 172.21.15.46:0/1424455782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc40ae1e0 0x7f0cc40cb790 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc009d90 tx=0x7f0cbc01c970 comp rx=0 tx=0).stop 2024-03-29T08:27:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc9000640 1 -- 172.21.15.46:0/1424455782 shutdown_connections 2024-03-29T08:27:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc9000640 1 --2- 172.21.15.46:0/1424455782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc40ae1e0 0x7f0cc40cb790 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc9000640 1 --2- 172.21.15.46:0/1424455782 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cc4143180 0x7f0cc40adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc9000640 1 -- 172.21.15.46:0/1424455782 >> 172.21.15.46:0/1424455782 conn(0x7f0cc406a390 msgr2=0x7f0cc40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc9000640 1 -- 172.21.15.46:0/1424455782 shutdown_connections 2024-03-29T08:27:11.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc9000640 1 -- 172.21.15.46:0/1424455782 wait complete. 2024-03-29T08:27:11.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc9000640 1 Processor -- start 2024-03-29T08:27:11.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc9000640 1 -- start start 2024-03-29T08:27:11.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc9000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cc40ae1e0 0x7f0cc41421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc9000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cc4142da0 0x7f0cc4159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc9000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc4143180 0x7f0cc4159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc9000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc415d000 con 0x7f0cc4143180 2024-03-29T08:27:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc9000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4144480 con 0x7f0cc4142da0 2024-03-29T08:27:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc9000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc4144760 con 0x7f0cc40ae1e0 2024-03-29T08:27:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc4143180 0x7f0cc4159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc4143180 0x7f0cc4159850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57952/0 (socket says 172.21.15.46:57952) 2024-03-29T08:27:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc2800640 1 -- 172.21.15.46:0/1287367855 learned_addr learned my addr 172.21.15.46:0/1287367855 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cc1e00640 1 --2- 172.21.15.46:0/1287367855 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cc40ae1e0 0x7f0cc41421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.259+0000 7f0cb9400640 1 --2- 172.21.15.46:0/1287367855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cc4142da0 0x7f0cc4159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.263+0000 7f0cc2800640 1 -- 172.21.15.46:0/1287367855 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cc40ae1e0 msgr2=0x7f0cc41421d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.263+0000 7f0cc2800640 1 --2- 172.21.15.46:0/1287367855 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cc40ae1e0 0x7f0cc41421d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.263+0000 7f0cc2800640 1 -- 172.21.15.46:0/1287367855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cc4142da0 msgr2=0x7f0cc4159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:11.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.263+0000 7f0cc2800640 1 --2- 172.21.15.46:0/1287367855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cc4142da0 0x7f0cc4159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.263+0000 7f0cc2800640 1 -- 172.21.15.46:0/1287367855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc009a70 con 0x7f0cc4143180 2024-03-29T08:27:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.263+0000 7f0cc2800640 1 --2- 172.21.15.46:0/1287367855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc4143180 0x7f0cc4159850 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0b0040 tx=0x7f0cbc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.263+0000 7f0cbaa00640 1 -- 172.21.15.46:0/1287367855 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc0b9860 con 0x7f0cc4143180 2024-03-29T08:27:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.263+0000 7f0cbaa00640 1 -- 172.21.15.46:0/1287367855 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cbc0b99c0 con 0x7f0cc4143180 2024-03-29T08:27:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.263+0000 7f0cc9000640 1 -- 172.21.15.46:0/1287367855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc4144a40 con 0x7f0cc4143180 2024-03-29T08:27:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.263+0000 7f0cbaa00640 1 -- 172.21.15.46:0/1287367855 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cbc0b9d70 con 0x7f0cc4143180 2024-03-29T08:27:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.263+0000 7f0cc9000640 1 -- 172.21.15.46:0/1287367855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc4139f10 con 0x7f0cc4143180 2024-03-29T08:27:11.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.263+0000 7f0cc9000640 1 -- 172.21.15.46:0/1287367855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc413a100 con 0x7f0cc4143180 2024-03-29T08:27:11.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.263+0000 7f0cbaa00640 1 -- 172.21.15.46:0/1287367855 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0cbc0ad440 con 0x7f0cc4143180 2024-03-29T08:27:11.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.263+0000 7f0cbaa00640 1 --2- 172.21.15.46:0/1287367855 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ca0066690 0x7f0ca0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:11.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.263+0000 7f0cbaa00640 1 -- 172.21.15.46:0/1287367855 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f0cbc11f240 con 0x7f0cc4143180 2024-03-29T08:27:11.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.263+0000 7f0cc1e00640 1 --2- 172.21.15.46:0/1287367855 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ca0066690 0x7f0ca0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:11.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.267+0000 7f0cc1e00640 1 --2- 172.21.15.46:0/1287367855 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ca0066690 0x7f0ca0068b50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4005890 tx=0x7f0cb4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.267+0000 7f0cbaa00640 1 -- 172.21.15.46:0/1287367855 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0cbc0ec8a0 con 0x7f0cc4143180 2024-03-29T08:27:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.435+0000 7f0cc9000640 1 -- 172.21.15.46:0/1287367855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0cc406ad40 con 0x7f0cc4143180 2024-03-29T08:27:11.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.439+0000 7f0cbaa00640 1 -- 172.21.15.46:0/1287367855 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0cbc0efec0 con 0x7f0cc4143180 2024-03-29T08:27:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.443+0000 7f0cc9000640 1 -- 172.21.15.46:0/1287367855 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ca0066690 msgr2=0x7f0ca0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.443+0000 7f0cc9000640 1 --2- 172.21.15.46:0/1287367855 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ca0066690 0x7f0ca0068b50 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4005890 tx=0x7f0cb4009040 comp rx=0 tx=0).stop 2024-03-29T08:27:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.443+0000 7f0cc9000640 1 -- 172.21.15.46:0/1287367855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc4143180 msgr2=0x7f0cc4159850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.443+0000 7f0cc9000640 1 --2- 172.21.15.46:0/1287367855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc4143180 0x7f0cc4159850 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0b0040 tx=0x7f0cbc019710 comp rx=0 tx=0).stop 2024-03-29T08:27:11.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.443+0000 7f0cc9000640 1 -- 172.21.15.46:0/1287367855 shutdown_connections 2024-03-29T08:27:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.443+0000 7f0cc9000640 1 --2- 172.21.15.46:0/1287367855 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0ca0066690 0x7f0ca0068b50 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.443+0000 7f0cc9000640 1 --2- 172.21.15.46:0/1287367855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cc4142da0 0x7f0cc4159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.443+0000 7f0cc9000640 1 --2- 172.21.15.46:0/1287367855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cc4143180 0x7f0cc4159850 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.443+0000 7f0cc9000640 1 --2- 172.21.15.46:0/1287367855 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0cc40ae1e0 0x7f0cc41421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.443+0000 7f0cc9000640 1 -- 172.21.15.46:0/1287367855 >> 172.21.15.46:0/1287367855 conn(0x7f0cc406a390 msgr2=0x7f0cc40ccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.443+0000 7f0cc9000640 1 -- 172.21.15.46:0/1287367855 shutdown_connections 2024-03-29T08:27:11.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.443+0000 7f0cc9000640 1 -- 172.21.15.46:0/1287367855 wait complete. 2024-03-29T08:27:11.465 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-03-29T08:27:11.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.607+0000 7fd0a4c00640 1 Processor -- start 2024-03-29T08:27:11.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.607+0000 7fd0a4c00640 1 -- start start 2024-03-29T08:27:11.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.607+0000 7fd0a4c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0a00cf3c0 0x7fd0a0165660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:11.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.607+0000 7fd0a4c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a00b0ab0 con 0x7fd0a015ad20 2024-03-29T08:27:11.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.607+0000 7fd0a4c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a00b0d90 con 0x7fd0a015a940 2024-03-29T08:27:11.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.607+0000 7fd0a4c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a00b1070 con 0x7fd0a00cf3c0 2024-03-29T08:27:11.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.607+0000 7fd095400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd0a015ad20 0x7fd0a00cecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33750/0 (socket says 172.21.15.46:33750) 2024-03-29T08:27:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.607+0000 7fd095400640 1 -- 172.21.15.46:0/617003987 learned_addr learned my addr 172.21.15.46:0/617003987 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.607+0000 7fd09e800640 1 --2- 172.21.15.46:0/617003987 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0a00cf3c0 0x7fd0a0165660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.607+0000 7fd09d400640 1 -- 172.21.15.46:0/617003987 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3306995133 0 0) 0x7fd0a00b0ab0 con 0x7fd0a015ad20 2024-03-29T08:27:11.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.607+0000 7fd09d400640 1 -- 172.21.15.46:0/617003987 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd084003530 con 0x7fd0a015ad20 2024-03-29T08:27:11.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.607+0000 7fd09d400640 1 -- 172.21.15.46:0/617003987 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621678993 0 0) 0x7fd0a00b0d90 con 0x7fd0a015a940 2024-03-29T08:27:11.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.607+0000 7fd09d400640 1 -- 172.21.15.46:0/617003987 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0a00b0ab0 con 0x7fd0a015a940 2024-03-29T08:27:11.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.607+0000 7fd09e800640 1 -- 172.21.15.46:0/617003987 >> v1:172.21.15.73:6789/0 conn(0x7fd0a015a940 legacy=0x7fd0a00afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:11.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.607+0000 7fd09e800640 1 -- 172.21.15.46:0/617003987 >> v1:172.21.15.46:6789/0 conn(0x7fd0a015ad20 legacy=0x7fd0a00cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:11.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.607+0000 7fd09e800640 1 -- 172.21.15.46:0/617003987 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a00b1350 con 0x7fd0a00cf3c0 2024-03-29T08:27:11.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.611+0000 7fd09e800640 1 --2- 172.21.15.46:0/617003987 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0a00cf3c0 0x7fd0a0165660 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd0980097f0 tx=0x7fd09801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fb9bfcff8daad62 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:11.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.611+0000 7fd09d400640 1 -- 172.21.15.46:0/617003987 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0980aa430 con 0x7fd0a00cf3c0 2024-03-29T08:27:11.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.611+0000 7fd09d400640 1 -- 172.21.15.46:0/617003987 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0980aa590 con 0x7fd0a00cf3c0 2024-03-29T08:27:11.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.611+0000 7fd09d400640 1 -- 172.21.15.46:0/617003987 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0980aa920 con 0x7fd0a00cf3c0 2024-03-29T08:27:11.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.611+0000 7fd0a4c00640 1 -- 172.21.15.46:0/617003987 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0a00cf3c0 msgr2=0x7fd0a0165660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:11.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.611+0000 7fd0a4c00640 1 --2- 172.21.15.46:0/617003987 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0a00cf3c0 0x7fd0a0165660 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd0980097f0 tx=0x7fd09801c4b0 comp rx=0 tx=0).stop 2024-03-29T08:27:11.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.611+0000 7fd0a4c00640 1 -- 172.21.15.46:0/617003987 shutdown_connections 2024-03-29T08:27:11.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.611+0000 7fd0a4c00640 1 --2- 172.21.15.46:0/617003987 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0a00cf3c0 0x7fd0a0165660 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:11.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.611+0000 7fd0a4c00640 1 -- 172.21.15.46:0/617003987 >> 172.21.15.46:0/617003987 conn(0x7fd0a0089fd0 msgr2=0x7fd0a008a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:11.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.611+0000 7fd0a4c00640 1 -- 172.21.15.46:0/617003987 shutdown_connections 2024-03-29T08:27:11.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.611+0000 7fd0a4c00640 1 -- 172.21.15.46:0/617003987 wait complete. 2024-03-29T08:27:11.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd0a4c00640 1 Processor -- start 2024-03-29T08:27:11.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd0a4c00640 1 -- start start 2024-03-29T08:27:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd0a4c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0a00cf3c0 0x7fd0a00c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd0a4c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0a015a940 0x7fd0a00c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd0a4c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0a015ad20 0x7fd0a00ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd0a4c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a015ce20 con 0x7fd0a015ad20 2024-03-29T08:27:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd0a4c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a00c4c50 con 0x7fd0a00cf3c0 2024-03-29T08:27:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd0a4c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a015c210 con 0x7fd0a015a940 2024-03-29T08:27:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd09e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0a015ad20 0x7fd0a00ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd09e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0a015ad20 0x7fd0a00ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57956/0 (socket says 172.21.15.46:57956) 2024-03-29T08:27:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd09e800640 1 -- 172.21.15.46:0/3515752372 learned_addr learned my addr 172.21.15.46:0/3515752372 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:11.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd09e800640 1 -- 172.21.15.46:0/3515752372 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0a015a940 msgr2=0x7fd0a00c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd095400640 1 --2- 172.21.15.46:0/3515752372 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0a015a940 0x7fd0a00c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd09de00640 1 --2- 172.21.15.46:0/3515752372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0a00cf3c0 0x7fd0a00c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd09e800640 1 --2- 172.21.15.46:0/3515752372 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0a015a940 0x7fd0a00c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd09e800640 1 -- 172.21.15.46:0/3515752372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0a00cf3c0 msgr2=0x7fd0a00c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd09e800640 1 --2- 172.21.15.46:0/3515752372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0a00cf3c0 0x7fd0a00c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd09e800640 1 -- 172.21.15.46:0/3515752372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd098009420 con 0x7fd0a015ad20 2024-03-29T08:27:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd09de00640 1 --2- 172.21.15.46:0/3515752372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0a00cf3c0 0x7fd0a00c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd095400640 1 --2- 172.21.15.46:0/3515752372 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0a015a940 0x7fd0a00c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd09e800640 1 --2- 172.21.15.46:0/3515752372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0a015ad20 0x7fd0a00ccb00 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fd0980190c0 tx=0x7fd09801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd096a00640 1 -- 172.21.15.46:0/3515752372 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0980b4810 con 0x7fd0a015ad20 2024-03-29T08:27:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd096a00640 1 -- 172.21.15.46:0/3515752372 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0980b4970 con 0x7fd0a015ad20 2024-03-29T08:27:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd096a00640 1 -- 172.21.15.46:0/3515752372 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0980b4d00 con 0x7fd0a015ad20 2024-03-29T08:27:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd0a4c00640 1 -- 172.21.15.46:0/3515752372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a015c4f0 con 0x7fd0a015ad20 2024-03-29T08:27:11.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.615+0000 7fd0a4c00640 1 -- 172.21.15.46:0/3515752372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a015cab0 con 0x7fd0a015ad20 2024-03-29T08:27:11.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.619+0000 7fd0a4c00640 1 -- 172.21.15.46:0/3515752372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a015cc40 con 0x7fd0a015ad20 2024-03-29T08:27:11.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.619+0000 7fd096a00640 1 -- 172.21.15.46:0/3515752372 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd0980b3390 con 0x7fd0a015ad20 2024-03-29T08:27:11.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.619+0000 7fd096a00640 1 --2- 172.21.15.46:0/3515752372 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd07c066690 0x7fd07c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:11.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.619+0000 7fd096a00640 1 -- 172.21.15.46:0/3515752372 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fd09811e6f0 con 0x7fd0a015ad20 2024-03-29T08:27:11.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.619+0000 7fd09de00640 1 --2- 172.21.15.46:0/3515752372 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd07c066690 0x7fd07c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:11.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.619+0000 7fd09de00640 1 --2- 172.21.15.46:0/3515752372 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd07c066690 0x7fd07c068b50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd0900011f0 tx=0x7fd090009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:11.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.623+0000 7fd096a00640 1 -- 172.21.15.46:0/3515752372 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd0980ebd40 con 0x7fd0a015ad20 2024-03-29T08:27:11.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:11.839+0000 7fd0a4c00640 1 -- 172.21.15.46:0/3515752372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7fd0a0066c90 con 0x7fd0a015ad20 2024-03-29T08:27:12.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.075+0000 7fd096a00640 1 -- 172.21.15.46:0/3515752372 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v412) v1 ==== 167+0+0 (secure 0 0 0) 0x7fd0980b4e60 con 0x7fd0a015ad20 2024-03-29T08:27:12.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.079+0000 7fd0a4c00640 1 -- 172.21.15.46:0/3515752372 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd07c066690 msgr2=0x7fd07c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.079+0000 7fd0a4c00640 1 --2- 172.21.15.46:0/3515752372 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd07c066690 0x7fd07c068b50 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd0900011f0 tx=0x7fd090009040 comp rx=0 tx=0).stop 2024-03-29T08:27:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.079+0000 7fd0a4c00640 1 -- 172.21.15.46:0/3515752372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0a015ad20 msgr2=0x7fd0a00ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.079+0000 7fd0a4c00640 1 --2- 172.21.15.46:0/3515752372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0a015ad20 0x7fd0a00ccb00 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fd0980190c0 tx=0x7fd09801acf0 comp rx=0 tx=0).stop 2024-03-29T08:27:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.079+0000 7fd0a4c00640 1 -- 172.21.15.46:0/3515752372 shutdown_connections 2024-03-29T08:27:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.079+0000 7fd0a4c00640 1 --2- 172.21.15.46:0/3515752372 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd07c066690 0x7fd07c068b50 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.079+0000 7fd0a4c00640 1 --2- 172.21.15.46:0/3515752372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0a015ad20 0x7fd0a00ccb00 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.079+0000 7fd0a4c00640 1 --2- 172.21.15.46:0/3515752372 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0a015a940 0x7fd0a00c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.079+0000 7fd0a4c00640 1 --2- 172.21.15.46:0/3515752372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0a00cf3c0 0x7fd0a00c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.079+0000 7fd0a4c00640 1 -- 172.21.15.46:0/3515752372 >> 172.21.15.46:0/3515752372 conn(0x7fd0a0089fd0 msgr2=0x7fd0a0152310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.079+0000 7fd0a4c00640 1 -- 172.21.15.46:0/3515752372 shutdown_connections 2024-03-29T08:27:12.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.079+0000 7fd0a4c00640 1 -- 172.21.15.46:0/3515752372 wait complete. 2024-03-29T08:27:12.100 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-03-29T08:27:12.100 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-29T08:27:12.101 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-03-29T08:27:12.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.239+0000 7f2797e00640 1 Processor -- start 2024-03-29T08:27:12.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.239+0000 7f2797e00640 1 -- start start 2024-03-29T08:27:12.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2797e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f27981600f0 0x7f27980cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:12.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2797e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27980cfc20 con 0x7f2798099150 2024-03-29T08:27:12.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2797e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27980cf660 con 0x7f27980cf150 2024-03-29T08:27:12.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2797e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27980cf940 con 0x7f27981600f0 2024-03-29T08:27:12.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2797400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2798099150 0x7f27980cb510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33762/0 (socket says 172.21.15.46:33762) 2024-03-29T08:27:12.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2797400640 1 -- 172.21.15.46:0/1477058492 learned_addr learned my addr 172.21.15.46:0/1477058492 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796a00640 1 --2- 172.21.15.46:0/1477058492 >> v2:172.21.15.73:3300/0 conn(0x7f27981600f0 0x7f27980cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796000640 1 -- 172.21.15.46:0/1477058492 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537511411 0 0) 0x7f27900014a0 con 0x7f2798099150 2024-03-29T08:27:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796000640 1 -- 172.21.15.46:0/1477058492 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f277c003530 con 0x7f2798099150 2024-03-29T08:27:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796000640 1 -- 172.21.15.46:0/1477058492 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1435598443 0 0) 0x7f277c003530 con 0x7f2798099150 2024-03-29T08:27:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796000640 1 -- 172.21.15.46:0/1477058492 >> v2:172.21.15.73:3300/0 conn(0x7f27981600f0 msgr2=0x7f27980cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796000640 1 --2- 172.21.15.46:0/1477058492 >> v2:172.21.15.73:3300/0 conn(0x7f27981600f0 0x7f27980cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796000640 1 -- 172.21.15.46:0/1477058492 >> v1:172.21.15.73:6789/0 conn(0x7f27980cf150 legacy=0x7f2798098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796000640 1 -- 172.21.15.46:0/1477058492 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27980cad10 con 0x7f2798099150 2024-03-29T08:27:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796000640 1 -- 172.21.15.46:0/1477058492 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f2790001b90 con 0x7f2798099150 2024-03-29T08:27:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796000640 1 -- 172.21.15.46:0/1477058492 >> v1:172.21.15.46:6789/0 conn(0x7f2798099150 legacy=0x7f27980cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:12.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796000640 1 --2- 172.21.15.46:0/1477058492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c003980 0x7f277c003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796000640 1 --2- 172.21.15.46:0/1477058492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f277c004380 0x7f277c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796000640 1 --2- 172.21.15.46:0/1477058492 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f277c0050f0 0x7f277c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796000640 1 -- 172.21.15.46:0/1477058492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27980cf940 con 0x7f277c003980 2024-03-29T08:27:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796000640 1 -- 172.21.15.46:0/1477058492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277c00c030 con 0x7f277c004380 2024-03-29T08:27:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796000640 1 -- 172.21.15.46:0/1477058492 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f277c00c1f0 con 0x7f277c0050f0 2024-03-29T08:27:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796a00640 1 --2- 172.21.15.46:0/1477058492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c003980 0x7f277c003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2797400640 1 --2- 172.21.15.46:0/1477058492 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f277c0050f0 0x7f277c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f278fe00640 1 --2- 172.21.15.46:0/1477058492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f277c004380 0x7f277c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796a00640 1 -- 172.21.15.46:0/1477058492 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f277c0050f0 msgr2=0x7f277c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796a00640 1 --2- 172.21.15.46:0/1477058492 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f277c0050f0 0x7f277c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796a00640 1 -- 172.21.15.46:0/1477058492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f277c004380 msgr2=0x7f277c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796a00640 1 --2- 172.21.15.46:0/1477058492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f277c004380 0x7f277c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2796a00640 1 -- 172.21.15.46:0/1477058492 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c003530 con 0x7f277c003980 2024-03-29T08:27:12.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2797400640 1 --2- 172.21.15.46:0/1477058492 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f277c0050f0 0x7f277c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:12.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f278fe00640 1 --2- 172.21.15.46:0/1477058492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f277c004380 0x7f277c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:12.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.243+0000 7f2797400640 1 -- 172.21.15.46:0/1477058492 reap_dead start 2024-03-29T08:27:12.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.247+0000 7f2796a00640 1 --2- 172.21.15.46:0/1477058492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c003980 0x7f277c003d60 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f2788009ae0 tx=0x7f278801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e66da1d2b5f0ff server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:12.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.247+0000 7f2796000640 1 -- 172.21.15.46:0/1477058492 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27880a6500 con 0x7f277c003980 2024-03-29T08:27:12.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.247+0000 7f2796000640 1 -- 172.21.15.46:0/1477058492 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27880a6660 con 0x7f277c003980 2024-03-29T08:27:12.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.247+0000 7f2796000640 1 -- 172.21.15.46:0/1477058492 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27880a6a10 con 0x7f277c003980 2024-03-29T08:27:12.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.247+0000 7f2797e00640 1 -- 172.21.15.46:0/1477058492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c003980 msgr2=0x7f277c003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:12.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.247+0000 7f2797e00640 1 --2- 172.21.15.46:0/1477058492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c003980 0x7f277c003d60 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f2788009ae0 tx=0x7f278801ca40 comp rx=0 tx=0).stop 2024-03-29T08:27:12.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.247+0000 7f2797e00640 1 -- 172.21.15.46:0/1477058492 shutdown_connections 2024-03-29T08:27:12.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.247+0000 7f2797e00640 1 --2- 172.21.15.46:0/1477058492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f277c003980 0x7f277c003d60 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:12.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.247+0000 7f2797e00640 1 -- 172.21.15.46:0/1477058492 >> 172.21.15.46:0/1477058492 conn(0x7f27980b2010 msgr2=0x7f27980b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:12.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.247+0000 7f2797e00640 1 -- 172.21.15.46:0/1477058492 shutdown_connections 2024-03-29T08:27:12.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.247+0000 7f2797e00640 1 -- 172.21.15.46:0/1477058492 wait complete. 2024-03-29T08:27:12.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.247+0000 7f2797e00640 1 Processor -- start 2024-03-29T08:27:12.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.247+0000 7f2797e00640 1 -- start start 2024-03-29T08:27:12.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f2797e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f277c003980 0x7f279814faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:12.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f2797e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27980c5900 0x7f27980c4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:12.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f2797e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f279815cb20 0x7f27980c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:12.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f2797e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27980c4590 con 0x7f27980c5900 2024-03-29T08:27:12.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f2797e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27980c4860 con 0x7f277c003980 2024-03-29T08:27:12.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f2797e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27980c4b40 con 0x7f279815cb20 2024-03-29T08:27:12.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f278fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27980c5900 0x7f27980c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:12.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f278fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27980c5900 0x7f27980c4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57972/0 (socket says 172.21.15.46:57972) 2024-03-29T08:27:12.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f278fe00640 1 -- 172.21.15.46:0/2512760742 learned_addr learned my addr 172.21.15.46:0/2512760742 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:12.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f2796a00640 1 --2- 172.21.15.46:0/2512760742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f277c003980 0x7f279814faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:12.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f2797400640 1 --2- 172.21.15.46:0/2512760742 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f279815cb20 0x7f27980c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:12.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f278fe00640 1 -- 172.21.15.46:0/2512760742 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f279815cb20 msgr2=0x7f27980c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:12.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f278fe00640 1 --2- 172.21.15.46:0/2512760742 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f279815cb20 0x7f27980c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:12.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f278fe00640 1 -- 172.21.15.46:0/2512760742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f277c003980 msgr2=0x7f279814faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:12.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f278fe00640 1 --2- 172.21.15.46:0/2512760742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f277c003980 0x7f279814faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:12.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f278fe00640 1 -- 172.21.15.46:0/2512760742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27980c4e20 con 0x7f27980c5900 2024-03-29T08:27:12.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f278fe00640 1 --2- 172.21.15.46:0/2512760742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27980c5900 0x7f27980c4050 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f278000e550 tx=0x7f278000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f278ea00640 1 -- 172.21.15.46:0/2512760742 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27800052e0 con 0x7f27980c5900 2024-03-29T08:27:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f278ea00640 1 -- 172.21.15.46:0/2512760742 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2780005440 con 0x7f27980c5900 2024-03-29T08:27:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f278ea00640 1 -- 172.21.15.46:0/2512760742 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2780003840 con 0x7f27980c5900 2024-03-29T08:27:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f2797e00640 1 -- 172.21.15.46:0/2512760742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27980c5100 con 0x7f27980c5900 2024-03-29T08:27:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f2797e00640 1 -- 172.21.15.46:0/2512760742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27980c94e0 con 0x7f27980c5900 2024-03-29T08:27:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f2797e00640 1 -- 172.21.15.46:0/2512760742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27980671b0 con 0x7f27980c5900 2024-03-29T08:27:12.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.251+0000 7f278ea00640 1 -- 172.21.15.46:0/2512760742 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f27800039a0 con 0x7f27980c5900 2024-03-29T08:27:12.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.255+0000 7f278ea00640 1 --2- 172.21.15.46:0/2512760742 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27740666c0 0x7f2774068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:12.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.255+0000 7f278ea00640 1 -- 172.21.15.46:0/2512760742 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f2780083c50 con 0x7f27980c5900 2024-03-29T08:27:12.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.255+0000 7f2796a00640 1 --2- 172.21.15.46:0/2512760742 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27740666c0 0x7f2774068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:12.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.255+0000 7f2796a00640 1 --2- 172.21.15.46:0/2512760742 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27740666c0 0x7f2774068b80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f27880af000 tx=0x7f27880046d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:12.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.259+0000 7f278ea00640 1 -- 172.21.15.46:0/2512760742 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2780051220 con 0x7f27980c5900 2024-03-29T08:27:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:12.475+0000 7f2797e00640 1 -- 172.21.15.46:0/2512760742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7f279804e040 con 0x7f27980c5900 2024-03-29T08:27:13.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.083+0000 7f278ea00640 1 -- 172.21.15.46:0/2512760742 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v413) v1 ==== 149+0+0 (secure 0 0 0) 0x7f2780054840 con 0x7f27980c5900 2024-03-29T08:27:13.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.083+0000 7f2797e00640 1 -- 172.21.15.46:0/2512760742 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27740666c0 msgr2=0x7f2774068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:13.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.083+0000 7f2797e00640 1 --2- 172.21.15.46:0/2512760742 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27740666c0 0x7f2774068b80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f27880af000 tx=0x7f27880046d0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.083+0000 7f2797e00640 1 -- 172.21.15.46:0/2512760742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27980c5900 msgr2=0x7f27980c4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:13.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.083+0000 7f2797e00640 1 --2- 172.21.15.46:0/2512760742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27980c5900 0x7f27980c4050 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f278000e550 tx=0x7f278000ea10 comp rx=0 tx=0).stop 2024-03-29T08:27:13.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.083+0000 7f2797e00640 1 -- 172.21.15.46:0/2512760742 shutdown_connections 2024-03-29T08:27:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.083+0000 7f2797e00640 1 --2- 172.21.15.46:0/2512760742 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f27740666c0 0x7f2774068b80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.083+0000 7f2797e00640 1 --2- 172.21.15.46:0/2512760742 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f279815cb20 0x7f27980c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.083+0000 7f2797e00640 1 --2- 172.21.15.46:0/2512760742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27980c5900 0x7f27980c4050 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.083+0000 7f2797e00640 1 --2- 172.21.15.46:0/2512760742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f277c003980 0x7f279814faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.083+0000 7f2797e00640 1 -- 172.21.15.46:0/2512760742 >> 172.21.15.46:0/2512760742 conn(0x7f27980b2010 msgr2=0x7f27980ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.083+0000 7f2797e00640 1 -- 172.21.15.46:0/2512760742 shutdown_connections 2024-03-29T08:27:13.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.083+0000 7f2797e00640 1 -- 172.21.15.46:0/2512760742 wait complete. 2024-03-29T08:27:13.109 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-29T08:27:13.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7888800640 1 Processor -- start 2024-03-29T08:27:13.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7888800640 1 -- start start 2024-03-29T08:27:13.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7888800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f788015df90 0x7f78800ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:13.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7888800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788015d260 con 0x7f78800cbea0 2024-03-29T08:27:13.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7888800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78800aefd0 con 0x7f7880142a50 2024-03-29T08:27:13.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7888800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788015d4b0 con 0x7f788015df90 2024-03-29T08:27:13.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7886800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f78800cbea0 0x7f78800ae7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33776/0 (socket says 172.21.15.46:33776) 2024-03-29T08:27:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7886800640 1 -- 172.21.15.46:0/29462954 learned_addr learned my addr 172.21.15.46:0/29462954 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7885400640 1 --2- 172.21.15.46:0/29462954 >> v2:172.21.15.73:3300/0 conn(0x7f788015df90 0x7f78800ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7884a00640 1 -- 172.21.15.46:0/29462954 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663263089 0 0) 0x7f788015d260 con 0x7f78800cbea0 2024-03-29T08:27:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7884a00640 1 -- 172.21.15.46:0/29462954 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7868003530 con 0x7f78800cbea0 2024-03-29T08:27:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7884a00640 1 -- 172.21.15.46:0/29462954 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297843366 0 0) 0x7f78800aefd0 con 0x7f7880142a50 2024-03-29T08:27:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7884a00640 1 -- 172.21.15.46:0/29462954 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f788015d260 con 0x7f7880142a50 2024-03-29T08:27:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7884a00640 1 -- 172.21.15.46:0/29462954 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1837618212 0 0) 0x7f7868003530 con 0x7f78800cbea0 2024-03-29T08:27:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7884a00640 1 -- 172.21.15.46:0/29462954 >> v2:172.21.15.73:3300/0 conn(0x7f788015df90 msgr2=0x7f78800ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7884a00640 1 --2- 172.21.15.46:0/29462954 >> v2:172.21.15.73:3300/0 conn(0x7f788015df90 0x7f78800ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7884a00640 1 -- 172.21.15.46:0/29462954 >> v1:172.21.15.73:6789/0 conn(0x7f7880142a50 legacy=0x7f78800cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7884a00640 1 -- 172.21.15.46:0/29462954 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788015d790 con 0x7f78800cbea0 2024-03-29T08:27:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7884a00640 1 -- 172.21.15.46:0/29462954 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f787c001a00 con 0x7f78800cbea0 2024-03-29T08:27:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7885400640 1 --2- 172.21.15.46:0/29462954 >> v2:172.21.15.73:3300/0 conn(0x7f788015df90 0x7f78800ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:27:13.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7884a00640 1 -- 172.21.15.46:0/29462954 >> v1:172.21.15.46:6789/0 conn(0x7f78800cbea0 legacy=0x7f78800ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:13.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7884a00640 1 --2- 172.21.15.46:0/29462954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7868003b10 0x7f7868003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:13.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7884a00640 1 --2- 172.21.15.46:0/29462954 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78680047d0 0x7f7868004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:13.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7884a00640 1 --2- 172.21.15.46:0/29462954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7868005190 0x7f786800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:13.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7884a00640 1 -- 172.21.15.46:0/29462954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788015d4b0 con 0x7f7868003b10 2024-03-29T08:27:13.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7884a00640 1 -- 172.21.15.46:0/29462954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786800c0c0 con 0x7f7868005190 2024-03-29T08:27:13.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7884a00640 1 -- 172.21.15.46:0/29462954 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786800c280 con 0x7f78680047d0 2024-03-29T08:27:13.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7885e00640 1 --2- 172.21.15.46:0/29462954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7868003b10 0x7f7868003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:13.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7886800640 1 --2- 172.21.15.46:0/29462954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7868005190 0x7f786800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:13.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7885400640 1 --2- 172.21.15.46:0/29462954 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78680047d0 0x7f7868004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:13.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7885e00640 1 -- 172.21.15.46:0/29462954 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78680047d0 msgr2=0x7f7868004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:13.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7885e00640 1 --2- 172.21.15.46:0/29462954 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78680047d0 0x7f7868004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7885e00640 1 -- 172.21.15.46:0/29462954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7868005190 msgr2=0x7f786800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:13.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7885e00640 1 --2- 172.21.15.46:0/29462954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7868005190 0x7f786800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7885e00640 1 -- 172.21.15.46:0/29462954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7868003530 con 0x7f7868003b10 2024-03-29T08:27:13.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.255+0000 7f7886800640 1 -- 172.21.15.46:0/29462954 reap_dead start 2024-03-29T08:27:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.259+0000 7f7885e00640 1 --2- 172.21.15.46:0/29462954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7868003b10 0x7f7868003f10 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f78740018d0 tx=0x7f787401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b424bee5e051a675 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.259+0000 7f7884a00640 1 -- 172.21.15.46:0/29462954 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78740052d0 con 0x7f7868003b10 2024-03-29T08:27:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.259+0000 7f7884a00640 1 -- 172.21.15.46:0/29462954 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78740a7050 con 0x7f7868003b10 2024-03-29T08:27:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.259+0000 7f7884a00640 1 -- 172.21.15.46:0/29462954 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874003600 con 0x7f7868003b10 2024-03-29T08:27:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.259+0000 7f7888800640 1 -- 172.21.15.46:0/29462954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7868003b10 msgr2=0x7f7868003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.259+0000 7f7888800640 1 --2- 172.21.15.46:0/29462954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7868003b10 0x7f7868003f10 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f78740018d0 tx=0x7f787401c560 comp rx=0 tx=0).stop 2024-03-29T08:27:13.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.259+0000 7f7888800640 1 -- 172.21.15.46:0/29462954 shutdown_connections 2024-03-29T08:27:13.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.259+0000 7f7888800640 1 --2- 172.21.15.46:0/29462954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7868003b10 0x7f7868003f10 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.259+0000 7f7888800640 1 -- 172.21.15.46:0/29462954 >> 172.21.15.46:0/29462954 conn(0x7f788006a390 msgr2=0x7f78800cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:13.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.259+0000 7f7888800640 1 -- 172.21.15.46:0/29462954 shutdown_connections 2024-03-29T08:27:13.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.259+0000 7f7888800640 1 -- 172.21.15.46:0/29462954 wait complete. 2024-03-29T08:27:13.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7888800640 1 Processor -- start 2024-03-29T08:27:13.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7888800640 1 -- start start 2024-03-29T08:27:13.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7888800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7868003b10 0x7f78800c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:13.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7888800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7880159350 0x7f78800ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:13.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7888800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78800c8590 0x7f78800c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:13.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7888800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78800c8f50 con 0x7f7868003b10 2024-03-29T08:27:13.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7888800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78800c50f0 con 0x7f78800c8590 2024-03-29T08:27:13.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7888800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78800c4570 con 0x7f7880159350 2024-03-29T08:27:13.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7885e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7868003b10 0x7f78800c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:13.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7885e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7868003b10 0x7f78800c3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57994/0 (socket says 172.21.15.46:57994) 2024-03-29T08:27:13.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7885e00640 1 -- 172.21.15.46:0/2253869012 learned_addr learned my addr 172.21.15.46:0/2253869012 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:13.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7885400640 1 --2- 172.21.15.46:0/2253869012 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7880159350 0x7f78800ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:13.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7886800640 1 --2- 172.21.15.46:0/2253869012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78800c8590 0x7f78800c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:13.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7885e00640 1 -- 172.21.15.46:0/2253869012 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7880159350 msgr2=0x7f78800ca860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:13.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7885e00640 1 --2- 172.21.15.46:0/2253869012 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7880159350 0x7f78800ca860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7885e00640 1 -- 172.21.15.46:0/2253869012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78800c8590 msgr2=0x7f78800c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7885e00640 1 --2- 172.21.15.46:0/2253869012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78800c8590 0x7f78800c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7885e00640 1 -- 172.21.15.46:0/2253869012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78800c4850 con 0x7f7868003b10 2024-03-29T08:27:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7885e00640 1 --2- 172.21.15.46:0/2253869012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7868003b10 0x7f78800c3f80 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f78740ad040 tx=0x7f7868005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f787a000640 1 -- 172.21.15.46:0/2253869012 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78740069e0 con 0x7f7868003b10 2024-03-29T08:27:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f787a000640 1 -- 172.21.15.46:0/2253869012 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7874006b40 con 0x7f7868003b10 2024-03-29T08:27:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f787a000640 1 -- 172.21.15.46:0/2253869012 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874007030 con 0x7f7868003b10 2024-03-29T08:27:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7888800640 1 -- 172.21.15.46:0/2253869012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78800c4b30 con 0x7f7868003b10 2024-03-29T08:27:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.263+0000 7f7888800640 1 -- 172.21.15.46:0/2253869012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78800cada0 con 0x7f7868003b10 2024-03-29T08:27:13.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.267+0000 7f7888800640 1 -- 172.21.15.46:0/2253869012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7880159730 con 0x7f7868003b10 2024-03-29T08:27:13.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.267+0000 7f787a000640 1 -- 172.21.15.46:0/2253869012 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7874007190 con 0x7f7868003b10 2024-03-29T08:27:13.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.267+0000 7f787a000640 1 --2- 172.21.15.46:0/2253869012 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78640666c0 0x7f7864068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:13.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.267+0000 7f787a000640 1 -- 172.21.15.46:0/2253869012 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f787411de70 con 0x7f7868003b10 2024-03-29T08:27:13.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.267+0000 7f7885400640 1 --2- 172.21.15.46:0/2253869012 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78640666c0 0x7f7864068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:13.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.267+0000 7f7885400640 1 --2- 172.21.15.46:0/2253869012 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78640666c0 0x7f7864068b80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7880140ee0 tx=0x7f7870009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:13.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.271+0000 7f787a000640 1 -- 172.21.15.46:0/2253869012 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f78740eb4d0 con 0x7f7868003b10 2024-03-29T08:27:13.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.431+0000 7f7888800640 1 -- 172.21.15.46:0/2253869012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7880071a70 con 0x7f7868003b10 2024-03-29T08:27:13.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.435+0000 7f787a000640 1 -- 172.21.15.46:0/2253869012 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f78740eeaf0 con 0x7f7868003b10 2024-03-29T08:27:13.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.439+0000 7f7888800640 1 -- 172.21.15.46:0/2253869012 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78640666c0 msgr2=0x7f7864068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:13.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.439+0000 7f7888800640 1 --2- 172.21.15.46:0/2253869012 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78640666c0 0x7f7864068b80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7880140ee0 tx=0x7f7870009190 comp rx=0 tx=0).stop 2024-03-29T08:27:13.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.439+0000 7f7888800640 1 -- 172.21.15.46:0/2253869012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7868003b10 msgr2=0x7f78800c3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:13.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.439+0000 7f7888800640 1 --2- 172.21.15.46:0/2253869012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7868003b10 0x7f78800c3f80 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f78740ad040 tx=0x7f7868005800 comp rx=0 tx=0).stop 2024-03-29T08:27:13.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.439+0000 7f7888800640 1 -- 172.21.15.46:0/2253869012 shutdown_connections 2024-03-29T08:27:13.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.439+0000 7f7888800640 1 --2- 172.21.15.46:0/2253869012 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78640666c0 0x7f7864068b80 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.439+0000 7f7888800640 1 --2- 172.21.15.46:0/2253869012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78800c8590 0x7f78800c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.439+0000 7f7888800640 1 --2- 172.21.15.46:0/2253869012 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7880159350 0x7f78800ca860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.439+0000 7f7888800640 1 --2- 172.21.15.46:0/2253869012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7868003b10 0x7f78800c3f80 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.439+0000 7f7888800640 1 -- 172.21.15.46:0/2253869012 >> 172.21.15.46:0/2253869012 conn(0x7f788006a390 msgr2=0x7f788006c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:13.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.439+0000 7f7888800640 1 -- 172.21.15.46:0/2253869012 shutdown_connections 2024-03-29T08:27:13.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.439+0000 7f7888800640 1 -- 172.21.15.46:0/2253869012 wait complete. 2024-03-29T08:27:13.460 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T08:27:13.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:27:13.538 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:27:13.617 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T08:27:13.618 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:27:13.618 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:27:13.626 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:27:13.627 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:27:13.627 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:27:13.636 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:27:13.636 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:27:13.636 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:27:13.675 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:27:13.675 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:27:13.675 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:27:13.686 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:27:13.687 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-29T08:27:13.687 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:27:13.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.859+0000 7f3073600640 1 Processor -- start 2024-03-29T08:27:13.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.859+0000 7f3073600640 1 -- start start 2024-03-29T08:27:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.859+0000 7f3073600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f306c15a940 0x7f306c0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.859+0000 7f3073600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f306c0cf3c0 0x7f306c0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.859+0000 7f3073600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c0b00d0 con 0x7f306c15ad20 2024-03-29T08:27:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.859+0000 7f3073600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306c0b0ab0 con 0x7f306c0cf3c0 2024-03-29T08:27:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.859+0000 7f3073600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306c0b0d90 con 0x7f306c15a940 2024-03-29T08:27:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.859+0000 7f3063e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f306c15ad20 0x7f306c0cecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33782/0 (socket says 172.21.15.46:33782) 2024-03-29T08:27:13.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.859+0000 7f3063e00640 1 -- 172.21.15.46:0/980591159 learned_addr learned my addr 172.21.15.46:0/980591159 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.859+0000 7f3070c00640 1 --2- 172.21.15.46:0/980591159 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f306c15a940 0x7f306c0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.859+0000 7f3071600640 1 --2- 172.21.15.46:0/980591159 >> v2:172.21.15.73:3300/0 conn(0x7f306c0cf3c0 0x7f306c0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.859+0000 7f306be00640 1 -- 172.21.15.46:0/980591159 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2012847483 0 0) 0x7f306c0b00d0 con 0x7f306c15ad20 2024-03-29T08:27:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.859+0000 7f306be00640 1 -- 172.21.15.46:0/980591159 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3050003530 con 0x7f306c15ad20 2024-03-29T08:27:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.859+0000 7f3070c00640 1 -- 172.21.15.46:0/980591159 >> v2:172.21.15.73:3300/0 conn(0x7f306c0cf3c0 msgr2=0x7f306c0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.859+0000 7f3070c00640 1 --2- 172.21.15.46:0/980591159 >> v2:172.21.15.73:3300/0 conn(0x7f306c0cf3c0 0x7f306c0cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.859+0000 7f3070c00640 1 -- 172.21.15.46:0/980591159 >> v1:172.21.15.46:6789/0 conn(0x7f306c15ad20 legacy=0x7f306c0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:13.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.859+0000 7f3070c00640 1 -- 172.21.15.46:0/980591159 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c0b1070 con 0x7f306c15a940 2024-03-29T08:27:13.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.863+0000 7f3070c00640 1 --2- 172.21.15.46:0/980591159 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f306c15a940 0x7f306c0afb90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f305c0097c0 tx=0x7f305c01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69e8ca473cce8267 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:13.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.863+0000 7f306be00640 1 -- 172.21.15.46:0/980591159 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c0ac640 con 0x7f306c15a940 2024-03-29T08:27:13.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.863+0000 7f306be00640 1 -- 172.21.15.46:0/980591159 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c0ac7a0 con 0x7f306c15a940 2024-03-29T08:27:13.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.863+0000 7f306be00640 1 -- 172.21.15.46:0/980591159 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c0acb30 con 0x7f306c15a940 2024-03-29T08:27:13.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.863+0000 7f3073600640 1 -- 172.21.15.46:0/980591159 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f306c15a940 msgr2=0x7f306c0afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:13.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.863+0000 7f3073600640 1 --2- 172.21.15.46:0/980591159 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f306c15a940 0x7f306c0afb90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f305c0097c0 tx=0x7f305c01c3e0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.863+0000 7f3073600640 1 -- 172.21.15.46:0/980591159 shutdown_connections 2024-03-29T08:27:13.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.863+0000 7f3073600640 1 --2- 172.21.15.46:0/980591159 >> v2:172.21.15.73:3300/0 conn(0x7f306c0cf3c0 0x7f306c0cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.863+0000 7f3073600640 1 --2- 172.21.15.46:0/980591159 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f306c15a940 0x7f306c0afb90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.863+0000 7f3073600640 1 -- 172.21.15.46:0/980591159 >> 172.21.15.46:0/980591159 conn(0x7f306c089fd0 msgr2=0x7f306c08a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.863+0000 7f3073600640 1 -- 172.21.15.46:0/980591159 shutdown_connections 2024-03-29T08:27:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.863+0000 7f3073600640 1 -- 172.21.15.46:0/980591159 wait complete. 2024-03-29T08:27:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3073600640 1 Processor -- start 2024-03-29T08:27:13.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3073600640 1 -- start start 2024-03-29T08:27:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3073600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f306c0cf3c0 0x7f306c159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3073600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306c15a940 0x7f306c0ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3073600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f306c15ad20 0x7f306c1522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3073600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c159a60 con 0x7f306c15a940 2024-03-29T08:27:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3073600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c159c90 con 0x7f306c15ad20 2024-03-29T08:27:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3073600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306c1648c0 con 0x7f306c0cf3c0 2024-03-29T08:27:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3063e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306c15a940 0x7f306c0ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:13.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3063e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306c15a940 0x7f306c0ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58008/0 (socket says 172.21.15.46:58008) 2024-03-29T08:27:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3063e00640 1 -- 172.21.15.46:0/748524339 learned_addr learned my addr 172.21.15.46:0/748524339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3070c00640 1 --2- 172.21.15.46:0/748524339 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f306c0cf3c0 0x7f306c159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3071600640 1 --2- 172.21.15.46:0/748524339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f306c15ad20 0x7f306c1522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3063e00640 1 -- 172.21.15.46:0/748524339 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f306c0cf3c0 msgr2=0x7f306c159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3063e00640 1 --2- 172.21.15.46:0/748524339 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f306c0cf3c0 0x7f306c159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3063e00640 1 -- 172.21.15.46:0/748524339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f306c15ad20 msgr2=0x7f306c1522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3063e00640 1 --2- 172.21.15.46:0/748524339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f306c15ad20 0x7f306c1522a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3063e00640 1 -- 172.21.15.46:0/748524339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c009420 con 0x7f306c15a940 2024-03-29T08:27:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3063e00640 1 --2- 172.21.15.46:0/748524339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306c15a940 0x7f306c0ae830 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f305400c4d0 tx=0x7f305400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3069600640 1 -- 172.21.15.46:0/748524339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3054005cf0 con 0x7f306c15a940 2024-03-29T08:27:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3069600640 1 -- 172.21.15.46:0/748524339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3054005e50 con 0x7f306c15a940 2024-03-29T08:27:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3073600640 1 -- 172.21.15.46:0/748524339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306c164ba0 con 0x7f306c15a940 2024-03-29T08:27:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3069600640 1 -- 172.21.15.46:0/748524339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3054007270 con 0x7f306c15a940 2024-03-29T08:27:13.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.867+0000 7f3073600640 1 -- 172.21.15.46:0/748524339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c165160 con 0x7f306c15a940 2024-03-29T08:27:13.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.871+0000 7f3073600640 1 -- 172.21.15.46:0/748524339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306c165350 con 0x7f306c15a940 2024-03-29T08:27:13.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.871+0000 7f3069600640 1 -- 172.21.15.46:0/748524339 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f30540073d0 con 0x7f306c15a940 2024-03-29T08:27:13.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.871+0000 7f3069600640 1 --2- 172.21.15.46:0/748524339 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3048066690 0x7f3048068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:13.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.871+0000 7f3069600640 1 -- 172.21.15.46:0/748524339 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f30540834b0 con 0x7f306c15a940 2024-03-29T08:27:13.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.871+0000 7f3070c00640 1 --2- 172.21.15.46:0/748524339 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3048066690 0x7f3048068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:13.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.871+0000 7f3070c00640 1 --2- 172.21.15.46:0/748524339 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3048066690 0x7f3048068b50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f305c0ae000 tx=0x7f305c0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:13.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:13.875+0000 7f3069600640 1 -- 172.21.15.46:0/748524339 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3054050b20 con 0x7f306c15a940 2024-03-29T08:27:14.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.079+0000 7f3073600640 1 -- 172.21.15.46:0/748524339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f306c066c90 con 0x7f306c15a940 2024-03-29T08:27:14.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.079+0000 7f3069600640 1 -- 172.21.15.46:0/748524339 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f3054017090 con 0x7f306c15a940 2024-03-29T08:27:14.081 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:14.081 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7180,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6834/2583114362","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2583114362},{"type":"v1","addr":"172.21.15.46:6834","nonce":2583114362}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7195,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3298124730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3298124730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3298124730}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":52},{"gid":7381,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/1862652020","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1862652020},{"type":"v1","addr":"172.21.15.73:6836","nonce":1862652020}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59},{"gid":8268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/1474698202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6833","nonce":1474698202},{"type":"v1","addr":"172.21.15.46:6835","nonce":1474698202}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[]} 2024-03-29T08:27:14.081 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 71 2024-03-29T08:27:14.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.079+0000 7f3073600640 1 -- 172.21.15.46:0/748524339 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3048066690 msgr2=0x7f3048068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:14.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.079+0000 7f3073600640 1 --2- 172.21.15.46:0/748524339 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3048066690 0x7f3048068b50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f305c0ae000 tx=0x7f305c0a7000 comp rx=0 tx=0).stop 2024-03-29T08:27:14.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.079+0000 7f3073600640 1 -- 172.21.15.46:0/748524339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306c15a940 msgr2=0x7f306c0ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:14.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.079+0000 7f3073600640 1 --2- 172.21.15.46:0/748524339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306c15a940 0x7f306c0ae830 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f305400c4d0 tx=0x7f305400c990 comp rx=0 tx=0).stop 2024-03-29T08:27:14.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.079+0000 7f3073600640 1 -- 172.21.15.46:0/748524339 shutdown_connections 2024-03-29T08:27:14.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.079+0000 7f3073600640 1 --2- 172.21.15.46:0/748524339 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3048066690 0x7f3048068b50 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.079+0000 7f3073600640 1 --2- 172.21.15.46:0/748524339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f306c15ad20 0x7f306c1522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.079+0000 7f3073600640 1 --2- 172.21.15.46:0/748524339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f306c15a940 0x7f306c0ae830 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.079+0000 7f3073600640 1 --2- 172.21.15.46:0/748524339 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f306c0cf3c0 0x7f306c159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.079+0000 7f3073600640 1 -- 172.21.15.46:0/748524339 >> 172.21.15.46:0/748524339 conn(0x7f306c089fd0 msgr2=0x7f306c153cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:14.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.079+0000 7f3073600640 1 -- 172.21.15.46:0/748524339 shutdown_connections 2024-03-29T08:27:14.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.079+0000 7f3073600640 1 -- 172.21.15.46:0/748524339 wait complete. 2024-03-29T08:27:14.105 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T08:27:14.105 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-29T08:27:14.106 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T08:27:14.106 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:27:14.106 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T08:27:14.106 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-29T08:27:14.106 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T08:27:14.107 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:27:14.107 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T08:27:14.107 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-29T08:27:14.107 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T08:27:14.107 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:27:14.107 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T08:27:14.107 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-29T08:27:14.108 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T08:27:14.108 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:27:14.117 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T08:27:14.117 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T08:27:14.119 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T08:27:14.119 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T08:27:14.121 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T08:27:14.121 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T08:27:14.122 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T08:27:14.122 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T08:27:14.162 INFO:tasks.ceph.mds.a:Started 2024-03-29T08:27:14.162 DEBUG:teuthology.parallel:result is None 2024-03-29T08:27:14.164 INFO:tasks.ceph.mds.c:Started 2024-03-29T08:27:14.164 DEBUG:teuthology.parallel:result is None 2024-03-29T08:27:14.165 INFO:tasks.ceph.mds.b:Started 2024-03-29T08:27:14.166 DEBUG:teuthology.parallel:result is None 2024-03-29T08:27:14.167 INFO:tasks.ceph.mds.d:Started 2024-03-29T08:27:14.167 DEBUG:teuthology.parallel:result is None 2024-03-29T08:27:14.167 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-29T08:27:14.247 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-03-29T08:27:14.250 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-03-29T08:27:14.254 INFO:tasks.ceph.mds.d.smithi073.stdout:starting mds.d at 2024-03-29T08:27:14.254 INFO:tasks.ceph.mds.b.smithi073.stdout:starting mds.b at 2024-03-29T08:27:14.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.319+0000 7f867f000640 1 Processor -- start 2024-03-29T08:27:14.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.319+0000 7f867f000640 1 -- start start 2024-03-29T08:27:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.319+0000 7f867f000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f867815aab0 0x7f86780ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.319+0000 7f867f000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f86780cb120 0x7f86780cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:14.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.319+0000 7f867f000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86780ae120 con 0x7f86780ad450 2024-03-29T08:27:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.319+0000 7f867f000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86780d1500 con 0x7f867815aab0 2024-03-29T08:27:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.319+0000 7f867f000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86780b01b0 con 0x7f86780cb120 2024-03-29T08:27:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.319+0000 7f8677e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f867815aab0 0x7f86780ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.319+0000 7f8677e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f867815aab0 0x7f86780ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58042/0 (socket says 172.21.15.46:58042) 2024-03-29T08:27:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.319+0000 7f8677e00640 1 -- 172.21.15.46:0/1183880156 learned_addr learned my addr 172.21.15.46:0/1183880156 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.319+0000 7f866f400640 1 --2- 172.21.15.46:0/1183880156 >> v2:172.21.15.73:3300/0 conn(0x7f86780cb120 0x7f86780cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.319+0000 7f8677e00640 1 -- 172.21.15.46:0/1183880156 >> v2:172.21.15.73:3300/0 conn(0x7f86780cb120 msgr2=0x7f86780cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.319+0000 7f8677e00640 1 --2- 172.21.15.46:0/1183880156 >> v2:172.21.15.73:3300/0 conn(0x7f86780cb120 0x7f86780cf0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.319+0000 7f8677e00640 1 -- 172.21.15.46:0/1183880156 >> v1:172.21.15.73:6789/0 conn(0x7f86780ad450 legacy=0x7f86780ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:14.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.319+0000 7f8677e00640 1 -- 172.21.15.46:0/1183880156 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86780ae3e0 con 0x7f867815aab0 2024-03-29T08:27:14.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.323+0000 7f8677e00640 1 --2- 172.21.15.46:0/1183880156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f867815aab0 0x7f86780ceb90 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f867809a200 tx=0x7f866801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94e937e8633cd75 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:14.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.323+0000 7f8677400640 1 -- 172.21.15.46:0/1183880156 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86680aa680 con 0x7f867815aab0 2024-03-29T08:27:14.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.323+0000 7f8677400640 1 -- 172.21.15.46:0/1183880156 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86680aa7e0 con 0x7f867815aab0 2024-03-29T08:27:14.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.323+0000 7f8677400640 1 -- 172.21.15.46:0/1183880156 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86680aab70 con 0x7f867815aab0 2024-03-29T08:27:14.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.323+0000 7f867f000640 1 -- 172.21.15.46:0/1183880156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f867815aab0 msgr2=0x7f86780ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:14.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.323+0000 7f867f000640 1 --2- 172.21.15.46:0/1183880156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f867815aab0 0x7f86780ceb90 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f867809a200 tx=0x7f866801c520 comp rx=0 tx=0).stop 2024-03-29T08:27:14.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.323+0000 7f867f000640 1 -- 172.21.15.46:0/1183880156 shutdown_connections 2024-03-29T08:27:14.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.323+0000 7f867f000640 1 --2- 172.21.15.46:0/1183880156 >> v2:172.21.15.73:3300/0 conn(0x7f86780cb120 0x7f86780cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.323+0000 7f867f000640 1 --2- 172.21.15.46:0/1183880156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f867815aab0 0x7f86780ceb90 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.323+0000 7f867f000640 1 -- 172.21.15.46:0/1183880156 >> 172.21.15.46:0/1183880156 conn(0x7f8678099a10 msgr2=0x7f8678089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:14.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.323+0000 7f867f000640 1 -- 172.21.15.46:0/1183880156 shutdown_connections 2024-03-29T08:27:14.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.323+0000 7f867f000640 1 -- 172.21.15.46:0/1183880156 wait complete. 2024-03-29T08:27:14.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f867f000640 1 Processor -- start 2024-03-29T08:27:14.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f867f000640 1 -- start start 2024-03-29T08:27:14.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f867f000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86780ad450 0x7f867814b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:14.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f867f000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86780cb120 0x7f8678158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:14.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f867f000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f867815aab0 0x7f867814aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f867f000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86781591e0 con 0x7f867815aab0 2024-03-29T08:27:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f867f000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86781594c0 con 0x7f86780ad450 2024-03-29T08:27:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f867f000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86781597a0 con 0x7f86780cb120 2024-03-29T08:27:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f867d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f867815aab0 0x7f867814aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f8677e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86780ad450 0x7f867814b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f866f400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86780cb120 0x7f8678158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f8677e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86780ad450 0x7f867814b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:56318/0 (socket says 172.21.15.46:56318) 2024-03-29T08:27:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f866f400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86780cb120 0x7f8678158810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:60770/0 (socket says 172.21.15.46:60770) 2024-03-29T08:27:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f867d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f867815aab0 0x7f867814aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58048/0 (socket says 172.21.15.46:58048) 2024-03-29T08:27:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f8677e00640 1 -- 172.21.15.46:0/4062646098 learned_addr learned my addr 172.21.15.46:0/4062646098 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f8677e00640 1 -- 172.21.15.46:0/4062646098 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86780cb120 msgr2=0x7f8678158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f8677e00640 1 --2- 172.21.15.46:0/4062646098 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86780cb120 0x7f8678158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f8677e00640 1 -- 172.21.15.46:0/4062646098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f867815aab0 msgr2=0x7f867814aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:14.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f8677e00640 1 --2- 172.21.15.46:0/4062646098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f867815aab0 0x7f867814aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:14.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f8677e00640 1 -- 172.21.15.46:0/4062646098 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86680094d0 con 0x7f86780ad450 2024-03-29T08:27:14.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f866f400640 1 --2- 172.21.15.46:0/4062646098 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86780cb120 0x7f8678158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:27:14.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f867d000640 1 --2- 172.21.15.46:0/4062646098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f867815aab0 0x7f867814aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:27:14.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f8677e00640 1 --2- 172.21.15.46:0/4062646098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86780ad450 0x7f867814b9c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8668019130 tx=0x7f866801ad60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:14.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f8674c00640 1 -- 172.21.15.46:0/4062646098 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86680b3800 con 0x7f86780ad450 2024-03-29T08:27:14.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f867f000640 1 -- 172.21.15.46:0/4062646098 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867815bfc0 con 0x7f86780ad450 2024-03-29T08:27:14.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f8674c00640 1 -- 172.21.15.46:0/4062646098 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86680b3980 con 0x7f86780ad450 2024-03-29T08:27:14.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f867f000640 1 -- 172.21.15.46:0/4062646098 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867814bf00 con 0x7f86780ad450 2024-03-29T08:27:14.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f8674c00640 1 -- 172.21.15.46:0/4062646098 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86680b3cf0 con 0x7f86780ad450 2024-03-29T08:27:14.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.327+0000 7f867f000640 1 -- 172.21.15.46:0/4062646098 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867809e7d0 con 0x7f86780ad450 2024-03-29T08:27:14.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.331+0000 7f8674c00640 1 -- 172.21.15.46:0/4062646098 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f86680c7420 con 0x7f86780ad450 2024-03-29T08:27:14.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.331+0000 7f8674c00640 1 --2- 172.21.15.46:0/4062646098 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8654066690 0x7f8654068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:14.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.331+0000 7f8674c00640 1 -- 172.21.15.46:0/4062646098 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f866811e7d0 con 0x7f86780ad450 2024-03-29T08:27:14.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.331+0000 7f866f400640 1 --2- 172.21.15.46:0/4062646098 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8654066690 0x7f8654068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:14.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.331+0000 7f866f400640 1 --2- 172.21.15.46:0/4062646098 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8654066690 0x7f8654068b50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f86780c6910 tx=0x7f8660009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:14.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.335+0000 7f8674c00640 1 -- 172.21.15.46:0/4062646098 <== mon.1 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f86680ebec0 con 0x7f86780ad450 2024-03-29T08:27:14.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:14.507+0000 7f867f000640 1 -- 172.21.15.46:0/4062646098 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8678159600 con 0x7f86780ad450 2024-03-29T08:27:15.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.283+0000 7f8674c00640 1 -- 172.21.15.46:0/4062646098 <== mon.1 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v102) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8678159600 con 0x7f86780ad450 2024-03-29T08:27:15.286 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-03-29T08:27:15.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.287+0000 7f867f000640 1 -- 172.21.15.46:0/4062646098 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8654066690 msgr2=0x7f8654068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.287+0000 7f867f000640 1 --2- 172.21.15.46:0/4062646098 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8654066690 0x7f8654068b50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f86780c6910 tx=0x7f8660009190 comp rx=0 tx=0).stop 2024-03-29T08:27:15.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.287+0000 7f867f000640 1 -- 172.21.15.46:0/4062646098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86780ad450 msgr2=0x7f867814b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.287+0000 7f867f000640 1 --2- 172.21.15.46:0/4062646098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86780ad450 0x7f867814b9c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8668019130 tx=0x7f866801ad60 comp rx=0 tx=0).stop 2024-03-29T08:27:15.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.287+0000 7f867f000640 1 -- 172.21.15.46:0/4062646098 shutdown_connections 2024-03-29T08:27:15.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.287+0000 7f867f000640 1 --2- 172.21.15.46:0/4062646098 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8654066690 0x7f8654068b50 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.287+0000 7f867f000640 1 --2- 172.21.15.46:0/4062646098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f867815aab0 0x7f867814aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.287+0000 7f867f000640 1 --2- 172.21.15.46:0/4062646098 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86780cb120 0x7f8678158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.287+0000 7f867f000640 1 --2- 172.21.15.46:0/4062646098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86780ad450 0x7f867814b9c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.287+0000 7f867f000640 1 -- 172.21.15.46:0/4062646098 >> 172.21.15.46:0/4062646098 conn(0x7f8678099a10 msgr2=0x7f86780b0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:15.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.287+0000 7f867f000640 1 -- 172.21.15.46:0/4062646098 shutdown_connections 2024-03-29T08:27:15.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.287+0000 7f867f000640 1 -- 172.21.15.46:0/4062646098 wait complete. 2024-03-29T08:27:15.313 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-29T08:27:15.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.455+0000 7ff3fbe00640 1 Processor -- start 2024-03-29T08:27:15.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.455+0000 7ff3fbe00640 1 -- start start 2024-03-29T08:27:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.455+0000 7ff3fbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff3f41605f0 0x7ff3f4069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.455+0000 7ff3fbe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff3f4160a80 0x7ff3f406a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.455+0000 7ff3fbe00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f4161020 con 0x7ff3f415feb0 2024-03-29T08:27:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.455+0000 7ff3fbe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f4161300 con 0x7ff3f41605f0 2024-03-29T08:27:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.455+0000 7ff3fbe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f4070b70 con 0x7ff3f4160a80 2024-03-29T08:27:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.455+0000 7ff3f8a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff3f41605f0 0x7ff3f4069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.455+0000 7ff3f8a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff3f41605f0 0x7ff3f4069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58058/0 (socket says 172.21.15.46:58058) 2024-03-29T08:27:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.455+0000 7ff3f8a00640 1 -- 172.21.15.46:0/3707734350 learned_addr learned my addr 172.21.15.46:0/3707734350 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.455+0000 7ff3f9e00640 1 --2- 172.21.15.46:0/3707734350 >> v2:172.21.15.73:3300/0 conn(0x7ff3f4160a80 0x7ff3f406a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.459+0000 7ff3f8a00640 1 -- 172.21.15.46:0/3707734350 >> v2:172.21.15.73:3300/0 conn(0x7ff3f4160a80 msgr2=0x7ff3f406a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.459+0000 7ff3f8a00640 1 --2- 172.21.15.46:0/3707734350 >> v2:172.21.15.73:3300/0 conn(0x7ff3f4160a80 0x7ff3f406a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.459+0000 7ff3f8a00640 1 -- 172.21.15.46:0/3707734350 >> v1:172.21.15.73:6789/0 conn(0x7ff3f415feb0 legacy=0x7ff3f4071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.459+0000 7ff3f8a00640 1 -- 172.21.15.46:0/3707734350 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f4070e50 con 0x7ff3f41605f0 2024-03-29T08:27:15.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.459+0000 7ff3f8a00640 1 --2- 172.21.15.46:0/3707734350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f41605f0 0x7ff3f4069d90 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009870 tx=0x7ff3e001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f5250472ce23a23 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.459+0000 7ff3f3e00640 1 -- 172.21.15.46:0/3707734350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e00a9400 con 0x7ff3f41605f0 2024-03-29T08:27:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.459+0000 7ff3f3e00640 1 -- 172.21.15.46:0/3707734350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e00a9560 con 0x7ff3f41605f0 2024-03-29T08:27:15.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.459+0000 7ff3f3e00640 1 -- 172.21.15.46:0/3707734350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e00a98f0 con 0x7ff3f41605f0 2024-03-29T08:27:15.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.459+0000 7ff3fbe00640 1 -- 172.21.15.46:0/3707734350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f41605f0 msgr2=0x7ff3f4069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.459+0000 7ff3fbe00640 1 --2- 172.21.15.46:0/3707734350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f41605f0 0x7ff3f4069d90 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7ff3e0009870 tx=0x7ff3e001c520 comp rx=0 tx=0).stop 2024-03-29T08:27:15.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.459+0000 7ff3fbe00640 1 -- 172.21.15.46:0/3707734350 shutdown_connections 2024-03-29T08:27:15.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.459+0000 7ff3fbe00640 1 --2- 172.21.15.46:0/3707734350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f41605f0 0x7ff3f4069d90 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.459+0000 7ff3fbe00640 1 --2- 172.21.15.46:0/3707734350 >> v2:172.21.15.73:3300/0 conn(0x7ff3f4160a80 0x7ff3f406a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.459+0000 7ff3fbe00640 1 -- 172.21.15.46:0/3707734350 >> 172.21.15.46:0/3707734350 conn(0x7ff3f40ced50 msgr2=0x7ff3f40cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:15.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.459+0000 7ff3fbe00640 1 -- 172.21.15.46:0/3707734350 shutdown_connections 2024-03-29T08:27:15.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.459+0000 7ff3fbe00640 1 -- 172.21.15.46:0/3707734350 wait complete. 2024-03-29T08:27:15.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3fbe00640 1 Processor -- start 2024-03-29T08:27:15.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3fbe00640 1 -- start start 2024-03-29T08:27:15.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3fbe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff3f415feb0 0x7ff3f41543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3fbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3f41605f0 0x7ff3f415c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f4160a80 0x7ff3f415c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3fbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f4166d80 con 0x7ff3f4160a80 2024-03-29T08:27:15.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3fbe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f415cc30 con 0x7ff3f41605f0 2024-03-29T08:27:15.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3fbe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f415a3b0 con 0x7ff3f415feb0 2024-03-29T08:27:15.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3f9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f4160a80 0x7ff3f415c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:15.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3f8a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3f41605f0 0x7ff3f415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:15.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3f9400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff3f415feb0 0x7ff3f41543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:15.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3f9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f4160a80 0x7ff3f415c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58060/0 (socket says 172.21.15.46:58060) 2024-03-29T08:27:15.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3f9e00640 1 -- 172.21.15.46:0/3461121222 learned_addr learned my addr 172.21.15.46:0/3461121222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:15.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3f9e00640 1 -- 172.21.15.46:0/3461121222 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff3f415feb0 msgr2=0x7ff3f41543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3f9e00640 1 --2- 172.21.15.46:0/3461121222 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff3f415feb0 0x7ff3f41543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3f9e00640 1 -- 172.21.15.46:0/3461121222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3f41605f0 msgr2=0x7ff3f415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3f9e00640 1 --2- 172.21.15.46:0/3461121222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3f41605f0 0x7ff3f415c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3f9e00640 1 -- 172.21.15.46:0/3461121222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e00094d0 con 0x7ff3f4160a80 2024-03-29T08:27:15.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3f9e00640 1 --2- 172.21.15.46:0/3461121222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f4160a80 0x7ff3f415c5c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec007e60 tx=0x7ff3ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:15.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3f1600640 1 -- 172.21.15.46:0/3461121222 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec014420 con 0x7ff3f4160a80 2024-03-29T08:27:15.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3f1600640 1 -- 172.21.15.46:0/3461121222 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ec0145a0 con 0x7ff3f4160a80 2024-03-29T08:27:15.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3fbe00640 1 -- 172.21.15.46:0/3461121222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f415a690 con 0x7ff3f4160a80 2024-03-29T08:27:15.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.463+0000 7ff3f1600640 1 -- 172.21.15.46:0/3461121222 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ec014950 con 0x7ff3f4160a80 2024-03-29T08:27:15.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.467+0000 7ff3fbe00640 1 -- 172.21.15.46:0/3461121222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f41673e0 con 0x7ff3f4160a80 2024-03-29T08:27:15.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.467+0000 7ff3fbe00640 1 -- 172.21.15.46:0/3461121222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f415a210 con 0x7ff3f4160a80 2024-03-29T08:27:15.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.467+0000 7ff3f1600640 1 -- 172.21.15.46:0/3461121222 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff3ec014ab0 con 0x7ff3f4160a80 2024-03-29T08:27:15.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.467+0000 7ff3f1600640 1 --2- 172.21.15.46:0/3461121222 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3d4066690 0x7ff3d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.467+0000 7ff3f1600640 1 -- 172.21.15.46:0/3461121222 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff3ec084490 con 0x7ff3f4160a80 2024-03-29T08:27:15.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.467+0000 7ff3f9400640 1 --2- 172.21.15.46:0/3461121222 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3d4066690 0x7ff3d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:15.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.467+0000 7ff3f9400640 1 --2- 172.21.15.46:0/3461121222 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3d4066690 0x7ff3d4068b50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff3f406a590 tx=0x7ff3e8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:15.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.471+0000 7ff3f1600640 1 -- 172.21.15.46:0/3461121222 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff3f415a210 con 0x7ff3f4160a80 2024-03-29T08:27:15.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.691+0000 7ff3fbe00640 1 -- 172.21.15.46:0/3461121222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff3f41673e0 con 0x7ff3f4160a80 2024-03-29T08:27:15.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.691+0000 7ff3f1600640 1 -- 172.21.15.46:0/3461121222 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff3f41673e0 con 0x7ff3f4160a80 2024-03-29T08:27:15.695 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:15.695 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:27:15.695 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-03-29T08:27:15.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.696 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-03-29T08:27:15.696 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmlnXNJBAA7lI2lCBd0PeZudPsQyMOUQ==", 2024-03-29T08:27:15.696 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.696 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:27:15.696 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:27:15.696 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:27:15.696 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:27:15.696 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.696 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.696 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-03-29T08:27:15.696 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmbPkcMRAADnwJE17uILjJo14jghJuzg==", 2024-03-29T08:27:15.696 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.697 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:27:15.697 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:27:15.697 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:27:15.697 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:27:15.697 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.697 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.697 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-03-29T08:27:15.697 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmkCUVLBAAxcAOWGBDUCpEHeg/1OeQzA==", 2024-03-29T08:27:15.697 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.697 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:27:15.697 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:27:15.697 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:27:15.697 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:27:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-03-29T08:27:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZm7L11OBAAZ3fpXtEIYG1tGm9ZFswZ5Q==", 2024-03-29T08:27:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:27:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:27:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:27:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:27:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.698 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-03-29T08:27:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAIeQZmTHV8HhAAODJ+RP1Bx6k0D1153XgI0Q==", 2024-03-29T08:27:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:27:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:27:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:27:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-03-29T08:27:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQALeQZmZRD0JBAAjU6W/21CeqOMdIP5969iNA==", 2024-03-29T08:27:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:27:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:27:15.699 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:27:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-03-29T08:27:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAOeQZmSSP+KhAA26V8CByUJNfkVcG37AdlZQ==", 2024-03-29T08:27:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:27:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:27:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:27:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-03-29T08:27:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAReQZmb/e8LxAAV4yC1xlLwh8OAqOSPmcyyA==", 2024-03-29T08:27:15.700 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:27:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:27:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:27:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-03-29T08:27:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAXeQZmrLnzNRAAFPYQRs9sBNNKeyvcoI9rSg==", 2024-03-29T08:27:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:27:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:27:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:27:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-03-29T08:27:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAbeQZmjTPbABAAi4qTU5CBy9LtSSRf7cmfwg==", 2024-03-29T08:27:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:27:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:27:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:27:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-03-29T08:27:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAeeQZmo13eBRAAWY2VVBakLze9oA4ynp1rNA==", 2024-03-29T08:27:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.702 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:27:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:27:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:27:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-03-29T08:27:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAheQZmghpZCxAAdy5xI7qPwZb8LN8Rn6Ndaw==", 2024-03-29T08:27:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:27:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:27:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:27:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.703 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-03-29T08:27:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmyk22ABAA8Pn3OaIvxMQSLrKIM8FP4w==", 2024-03-29T08:27:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:27:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-03-29T08:27:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-29T08:27:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-03-29T08:27:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmG/E8ChAAJTcJ2j5xn0oUsmlV2trWdw==", 2024-03-29T08:27:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:27:15.704 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:27:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:27:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:27:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-03-29T08:27:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmhsqgBhAApqnO5qMG8VP27j9TfbZ8uw==", 2024-03-29T08:27:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:27:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:27:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:27:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:27:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-03-29T08:27:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmatlQEBAAAGKuGnAh4YXSk91TAzomXA==", 2024-03-29T08:27:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:27:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:27:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:27:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:27:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-03-29T08:27:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmp2B3CBAA4cfe/hUJeOg0SIr1KbWyUg==", 2024-03-29T08:27:15.706 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:27:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:27:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:27:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:27:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-03-29T08:27:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmyrJXIBAAv535Rz99UI3V7JvvzQ8oSw==", 2024-03-29T08:27:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:27:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-03-29T08:27:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:27:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-03-29T08:27:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZm72tiHBAA2N7qQnR5Ehu/7nP2WpklGg==", 2024-03-29T08:27:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:27:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:27:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-03-29T08:27:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:27:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:15.708 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-29T08:27:15.708 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:27:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.695+0000 7ff3fbe00640 1 -- 172.21.15.46:0/3461121222 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3d4066690 msgr2=0x7ff3d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.695+0000 7ff3fbe00640 1 --2- 172.21.15.46:0/3461121222 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3d4066690 0x7ff3d4068b50 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff3f406a590 tx=0x7ff3e8009040 comp rx=0 tx=0).stop 2024-03-29T08:27:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.695+0000 7ff3fbe00640 1 -- 172.21.15.46:0/3461121222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f4160a80 msgr2=0x7ff3f415c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.695+0000 7ff3fbe00640 1 --2- 172.21.15.46:0/3461121222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f4160a80 0x7ff3f415c5c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec007e60 tx=0x7ff3ec00e4b0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.695+0000 7ff3fbe00640 1 -- 172.21.15.46:0/3461121222 shutdown_connections 2024-03-29T08:27:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.695+0000 7ff3fbe00640 1 --2- 172.21.15.46:0/3461121222 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff3d4066690 0x7ff3d4068b50 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.695+0000 7ff3fbe00640 1 --2- 172.21.15.46:0/3461121222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3f41605f0 0x7ff3f415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.695+0000 7ff3fbe00640 1 --2- 172.21.15.46:0/3461121222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3f4160a80 0x7ff3f415c5c0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.695+0000 7ff3fbe00640 1 --2- 172.21.15.46:0/3461121222 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff3f415feb0 0x7ff3f41543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.695+0000 7ff3fbe00640 1 -- 172.21.15.46:0/3461121222 >> 172.21.15.46:0/3461121222 conn(0x7ff3f40ced50 msgr2=0x7ff3f40ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.695+0000 7ff3fbe00640 1 -- 172.21.15.46:0/3461121222 shutdown_connections 2024-03-29T08:27:15.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.695+0000 7ff3fbe00640 1 -- 172.21.15.46:0/3461121222 wait complete. 2024-03-29T08:27:15.717 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:27:15.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.855+0000 7f514ec00640 1 Processor -- start 2024-03-29T08:27:15.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.855+0000 7f514ec00640 1 -- start start 2024-03-29T08:27:15.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.855+0000 7f514ec00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f51480ccd10 0x7f5148098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.855+0000 7f514ec00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f514815cdb0 0x7f5148098f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.855+0000 7f514ec00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51480d0030 con 0x7f514815c9d0 2024-03-29T08:27:15.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.855+0000 7f514ec00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51480cf420 con 0x7f51480ccd10 2024-03-29T08:27:15.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.855+0000 7f514ec00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51480cf700 con 0x7f514815cdb0 2024-03-29T08:27:15.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.855+0000 7f513f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f51480ccd10 0x7f5148098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:15.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.855+0000 7f513f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f51480ccd10 0x7f5148098a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58070/0 (socket says 172.21.15.46:58070) 2024-03-29T08:27:15.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.855+0000 7f513f400640 1 -- 172.21.15.46:0/3199376739 learned_addr learned my addr 172.21.15.46:0/3199376739 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:15.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.855+0000 7f513f400640 1 -- 172.21.15.46:0/3199376739 >> v2:172.21.15.73:3300/0 conn(0x7f514815cdb0 msgr2=0x7f5148098f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:15.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.855+0000 7f513f400640 1 --2- 172.21.15.46:0/3199376739 >> v2:172.21.15.73:3300/0 conn(0x7f514815cdb0 0x7f5148098f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.855+0000 7f513f400640 1 -- 172.21.15.46:0/3199376739 >> v1:172.21.15.73:6789/0 conn(0x7f514815c9d0 legacy=0x7f51480cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:15.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.855+0000 7f513f400640 1 -- 172.21.15.46:0/3199376739 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51480cf9e0 con 0x7f51480ccd10 2024-03-29T08:27:15.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.859+0000 7f513f400640 1 --2- 172.21.15.46:0/3199376739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51480ccd10 0x7f5148098a40 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5148071920 tx=0x7f513001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=786e9dc16ab56236 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:15.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.859+0000 7f5147400640 1 -- 172.21.15.46:0/3199376739 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51300aa5f0 con 0x7f51480ccd10 2024-03-29T08:27:15.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.859+0000 7f5147400640 1 -- 172.21.15.46:0/3199376739 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51300aa750 con 0x7f51480ccd10 2024-03-29T08:27:15.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.859+0000 7f5147400640 1 -- 172.21.15.46:0/3199376739 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51300aaae0 con 0x7f51480ccd10 2024-03-29T08:27:15.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.859+0000 7f514ec00640 1 -- 172.21.15.46:0/3199376739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51480ccd10 msgr2=0x7f5148098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.859+0000 7f514ec00640 1 --2- 172.21.15.46:0/3199376739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51480ccd10 0x7f5148098a40 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5148071920 tx=0x7f513001c4b0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.859+0000 7f514ec00640 1 -- 172.21.15.46:0/3199376739 shutdown_connections 2024-03-29T08:27:15.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.859+0000 7f514ec00640 1 --2- 172.21.15.46:0/3199376739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f51480ccd10 0x7f5148098a40 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.859+0000 7f514ec00640 1 --2- 172.21.15.46:0/3199376739 >> v2:172.21.15.73:3300/0 conn(0x7f514815cdb0 0x7f5148098f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.859+0000 7f514ec00640 1 -- 172.21.15.46:0/3199376739 >> 172.21.15.46:0/3199376739 conn(0x7f51480b21a0 msgr2=0x7f51480b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:15.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.859+0000 7f514ec00640 1 -- 172.21.15.46:0/3199376739 shutdown_connections 2024-03-29T08:27:15.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.859+0000 7f514ec00640 1 -- 172.21.15.46:0/3199376739 wait complete. 2024-03-29T08:27:15.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f514ec00640 1 Processor -- start 2024-03-29T08:27:15.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f514ec00640 1 -- start start 2024-03-29T08:27:15.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f514ec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51480ccd10 0x7f51480b5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f514ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f514815c9d0 0x7f514815d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f514ec00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f514815cdb0 0x7f5148151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f514ec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5148152d20 con 0x7f514815c9d0 2024-03-29T08:27:15.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f514ec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51480b5ca0 con 0x7f51480ccd10 2024-03-29T08:27:15.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f514ec00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514815ebb0 con 0x7f514815cdb0 2024-03-29T08:27:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f513f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f514815c9d0 0x7f514815d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f513f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f514815c9d0 0x7f514815d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58074/0 (socket says 172.21.15.46:58074) 2024-03-29T08:27:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f513f400640 1 -- 172.21.15.46:0/3592672374 learned_addr learned my addr 172.21.15.46:0/3592672374 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f5147e00640 1 --2- 172.21.15.46:0/3592672374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51480ccd10 0x7f51480b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f514cc00640 1 --2- 172.21.15.46:0/3592672374 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f514815cdb0 0x7f5148151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f513f400640 1 -- 172.21.15.46:0/3592672374 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f514815cdb0 msgr2=0x7f5148151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f513f400640 1 --2- 172.21.15.46:0/3592672374 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f514815cdb0 0x7f5148151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f513f400640 1 -- 172.21.15.46:0/3592672374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51480ccd10 msgr2=0x7f51480b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f513f400640 1 --2- 172.21.15.46:0/3592672374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51480ccd10 0x7f51480b5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f513f400640 1 -- 172.21.15.46:0/3592672374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51300094d0 con 0x7f514815c9d0 2024-03-29T08:27:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f5147e00640 1 --2- 172.21.15.46:0/3592672374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51480ccd10 0x7f51480b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f513f400640 1 --2- 172.21.15.46:0/3592672374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f514815c9d0 0x7f514815d1e0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f513001a0e0 tx=0x7f513001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f5144c00640 1 -- 172.21.15.46:0/3592672374 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51300b3800 con 0x7f514815c9d0 2024-03-29T08:27:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f5144c00640 1 -- 172.21.15.46:0/3592672374 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51300b3960 con 0x7f514815c9d0 2024-03-29T08:27:15.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f514ec00640 1 -- 172.21.15.46:0/3592672374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514815d720 con 0x7f514815c9d0 2024-03-29T08:27:15.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f5144c00640 1 -- 172.21.15.46:0/3592672374 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51300b3d10 con 0x7f514815c9d0 2024-03-29T08:27:15.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.863+0000 7f514ec00640 1 -- 172.21.15.46:0/3592672374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5148152760 con 0x7f514815c9d0 2024-03-29T08:27:15.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.867+0000 7f514ec00640 1 -- 172.21.15.46:0/3592672374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5148066c90 con 0x7f514815c9d0 2024-03-29T08:27:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.867+0000 7f5144c00640 1 -- 172.21.15.46:0/3592672374 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f51300c6390 con 0x7f514815c9d0 2024-03-29T08:27:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.867+0000 7f5144c00640 1 --2- 172.21.15.46:0/3592672374 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5124066690 0x7f5124068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.867+0000 7f5144c00640 1 -- 172.21.15.46:0/3592672374 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f51300a7070 con 0x7f514815c9d0 2024-03-29T08:27:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.867+0000 7f5147e00640 1 --2- 172.21.15.46:0/3592672374 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5124066690 0x7f5124068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.867+0000 7f5147e00640 1 --2- 172.21.15.46:0/3592672374 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5124066690 0x7f5124068b50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f51480c76f0 tx=0x7f5138009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:15.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:15.871+0000 7f5144c00640 1 -- 172.21.15.46:0/3592672374 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f51300ec030 con 0x7f514815c9d0 2024-03-29T08:27:16.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.035+0000 7f514ec00640 1 -- 172.21.15.46:0/3592672374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51480b1a00 con 0x7f514815c9d0 2024-03-29T08:27:16.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.035+0000 7f5144c00640 1 -- 172.21.15.46:0/3592672374 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+12343 (secure 0 0 0) 0x7f51300ef650 con 0x7f514815c9d0 2024-03-29T08:27:16.037 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:16.037 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":102,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:27:15.278487+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:27:16.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.035+0000 7f514ec00640 1 -- 172.21.15.46:0/3592672374 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5124066690 msgr2=0x7f5124068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.035+0000 7f514ec00640 1 --2- 172.21.15.46:0/3592672374 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5124066690 0x7f5124068b50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f51480c76f0 tx=0x7f5138009040 comp rx=0 tx=0).stop 2024-03-29T08:27:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.035+0000 7f514ec00640 1 -- 172.21.15.46:0/3592672374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f514815c9d0 msgr2=0x7f514815d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.035+0000 7f514ec00640 1 --2- 172.21.15.46:0/3592672374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f514815c9d0 0x7f514815d1e0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f513001a0e0 tx=0x7f513001acf0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.035+0000 7f514ec00640 1 -- 172.21.15.46:0/3592672374 shutdown_connections 2024-03-29T08:27:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.035+0000 7f514ec00640 1 --2- 172.21.15.46:0/3592672374 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5124066690 0x7f5124068b50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.035+0000 7f514ec00640 1 --2- 172.21.15.46:0/3592672374 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f514815cdb0 0x7f5148151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.035+0000 7f514ec00640 1 --2- 172.21.15.46:0/3592672374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f514815c9d0 0x7f514815d1e0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.035+0000 7f514ec00640 1 --2- 172.21.15.46:0/3592672374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51480ccd10 0x7f51480b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.035+0000 7f514ec00640 1 -- 172.21.15.46:0/3592672374 >> 172.21.15.46:0/3592672374 conn(0x7f51480b21a0 msgr2=0x7f514806f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.035+0000 7f514ec00640 1 -- 172.21.15.46:0/3592672374 shutdown_connections 2024-03-29T08:27:16.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.035+0000 7f514ec00640 1 -- 172.21.15.46:0/3592672374 wait complete. 2024-03-29T08:27:16.059 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:27:16.059 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:27:16.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.203+0000 7f4714e00640 1 Processor -- start 2024-03-29T08:27:16.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.203+0000 7f4714e00640 1 -- start start 2024-03-29T08:27:16.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.203+0000 7f4714e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f47100cbea0 0x7f47100b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.203+0000 7f4714e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47100b2360 con 0x7f471015efe0 2024-03-29T08:27:16.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.203+0000 7f4714e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471015e500 con 0x7f4710143eb0 2024-03-29T08:27:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.203+0000 7f4714e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f471015e7e0 con 0x7f47100cbea0 2024-03-29T08:27:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.203+0000 7f4707e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f47100cbea0 0x7f47100b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.203+0000 7f470e800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f471015efe0 0x7f47100b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33798/0 (socket says 172.21.15.46:33798) 2024-03-29T08:27:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.203+0000 7f470e800640 1 -- 172.21.15.46:0/546320656 learned_addr learned my addr 172.21.15.46:0/546320656 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.203+0000 7f4707e00640 1 -- 172.21.15.46:0/546320656 >> v1:172.21.15.73:6789/0 conn(0x7f4710143eb0 legacy=0x7f47100cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.203+0000 7f4707e00640 1 -- 172.21.15.46:0/546320656 >> v1:172.21.15.46:6789/0 conn(0x7f471015efe0 legacy=0x7f47100b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.203+0000 7f4707e00640 1 -- 172.21.15.46:0/546320656 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471015eac0 con 0x7f47100cbea0 2024-03-29T08:27:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.207+0000 7f4707e00640 1 --2- 172.21.15.46:0/546320656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47100cbea0 0x7f47100b1630 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f46f80097c0 tx=0x7f46f801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182f76a5c76f9872 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.207+0000 7f470d400640 1 -- 172.21.15.46:0/546320656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f80aa620 con 0x7f47100cbea0 2024-03-29T08:27:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.207+0000 7f470d400640 1 -- 172.21.15.46:0/546320656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f80aa780 con 0x7f47100cbea0 2024-03-29T08:27:16.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.207+0000 7f470d400640 1 -- 172.21.15.46:0/546320656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f80aab10 con 0x7f47100cbea0 2024-03-29T08:27:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.207+0000 7f4714e00640 1 -- 172.21.15.46:0/546320656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47100cbea0 msgr2=0x7f47100b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.207+0000 7f4714e00640 1 --2- 172.21.15.46:0/546320656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47100cbea0 0x7f47100b1630 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f46f80097c0 tx=0x7f46f801c4a0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.207+0000 7f4714e00640 1 -- 172.21.15.46:0/546320656 shutdown_connections 2024-03-29T08:27:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.207+0000 7f4714e00640 1 --2- 172.21.15.46:0/546320656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f47100cbea0 0x7f47100b1630 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.207+0000 7f4714e00640 1 -- 172.21.15.46:0/546320656 >> 172.21.15.46:0/546320656 conn(0x7f471006a390 msgr2=0x7f47100cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.207+0000 7f4714e00640 1 -- 172.21.15.46:0/546320656 shutdown_connections 2024-03-29T08:27:16.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.207+0000 7f4714e00640 1 -- 172.21.15.46:0/546320656 wait complete. 2024-03-29T08:27:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.207+0000 7f4714e00640 1 Processor -- start 2024-03-29T08:27:16.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.207+0000 7f4714e00640 1 -- start start 2024-03-29T08:27:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4714e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47100cbea0 0x7f471013c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4714e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4710143eb0 0x7f471013d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4714e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f471015efe0 0x7f471013bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4714e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471013d850 con 0x7f4710143eb0 2024-03-29T08:27:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4714e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471013ecc0 con 0x7f47100cbea0 2024-03-29T08:27:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4714e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471013d5b0 con 0x7f471015efe0 2024-03-29T08:27:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4707e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4710143eb0 0x7f471013d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4707e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4710143eb0 0x7f471013d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58088/0 (socket says 172.21.15.46:58088) 2024-03-29T08:27:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4707e00640 1 -- 172.21.15.46:0/1691081673 learned_addr learned my addr 172.21.15.46:0/1691081673 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f470e800640 1 --2- 172.21.15.46:0/1691081673 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f471015efe0 0x7f471013bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f470de00640 1 --2- 172.21.15.46:0/1691081673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47100cbea0 0x7f471013c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4707e00640 1 -- 172.21.15.46:0/1691081673 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f471015efe0 msgr2=0x7f471013bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4707e00640 1 --2- 172.21.15.46:0/1691081673 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f471015efe0 0x7f471013bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4707e00640 1 -- 172.21.15.46:0/1691081673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47100cbea0 msgr2=0x7f471013c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4707e00640 1 --2- 172.21.15.46:0/1691081673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47100cbea0 0x7f471013c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4707e00640 1 -- 172.21.15.46:0/1691081673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f8009420 con 0x7f4710143eb0 2024-03-29T08:27:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f470e800640 1 --2- 172.21.15.46:0/1691081673 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f471015efe0 0x7f471013bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f470de00640 1 --2- 172.21.15.46:0/1691081673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47100cbea0 0x7f471013c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4707e00640 1 --2- 172.21.15.46:0/1691081673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4710143eb0 0x7f471013d070 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f46f80ac040 tx=0x7f46f801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4706000640 1 -- 172.21.15.46:0/1691081673 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f80be610 con 0x7f4710143eb0 2024-03-29T08:27:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4706000640 1 -- 172.21.15.46:0/1691081673 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f80be7b0 con 0x7f4710143eb0 2024-03-29T08:27:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4706000640 1 -- 172.21.15.46:0/1691081673 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f80beb60 con 0x7f4710143eb0 2024-03-29T08:27:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4714e00640 1 -- 172.21.15.46:0/1691081673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471013ef60 con 0x7f4710143eb0 2024-03-29T08:27:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4714e00640 1 -- 172.21.15.46:0/1691081673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471013f520 con 0x7f4710143eb0 2024-03-29T08:27:16.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4714e00640 1 -- 172.21.15.46:0/1691081673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4710071a70 con 0x7f4710143eb0 2024-03-29T08:27:16.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.211+0000 7f4706000640 1 -- 172.21.15.46:0/1691081673 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f46f80b4400 con 0x7f4710143eb0 2024-03-29T08:27:16.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.215+0000 7f4706000640 1 --2- 172.21.15.46:0/1691081673 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f46ec066690 0x7f46ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.215+0000 7f4706000640 1 -- 172.21.15.46:0/1691081673 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f46f811eb40 con 0x7f4710143eb0 2024-03-29T08:27:16.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.215+0000 7f470de00640 1 --2- 172.21.15.46:0/1691081673 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f46ec066690 0x7f46ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.215+0000 7f470de00640 1 --2- 172.21.15.46:0/1691081673 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f46ec066690 0x7f46ec068b50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f47101f86f0 tx=0x7f4700009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:16.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.219+0000 7f4706000640 1 -- 172.21.15.46:0/1691081673 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f46f80ec1f0 con 0x7f4710143eb0 2024-03-29T08:27:16.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.379+0000 7f4714e00640 1 -- 172.21.15.46:0/1691081673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f471004e120 con 0x7f4710143eb0 2024-03-29T08:27:16.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.383+0000 7f4706000640 1 -- 172.21.15.46:0/1691081673 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7f46f80ef810 con 0x7f4710143eb0 2024-03-29T08:27:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:27:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.383+0000 7f4714e00640 1 -- 172.21.15.46:0/1691081673 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f46ec066690 msgr2=0x7f46ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.383+0000 7f4714e00640 1 --2- 172.21.15.46:0/1691081673 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f46ec066690 0x7f46ec068b50 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f47101f86f0 tx=0x7f4700009040 comp rx=0 tx=0).stop 2024-03-29T08:27:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.383+0000 7f4714e00640 1 -- 172.21.15.46:0/1691081673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4710143eb0 msgr2=0x7f471013d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.383+0000 7f4714e00640 1 --2- 172.21.15.46:0/1691081673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4710143eb0 0x7f471013d070 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f46f80ac040 tx=0x7f46f801ace0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.383+0000 7f4714e00640 1 -- 172.21.15.46:0/1691081673 shutdown_connections 2024-03-29T08:27:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.383+0000 7f4714e00640 1 --2- 172.21.15.46:0/1691081673 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f46ec066690 0x7f46ec068b50 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.383+0000 7f4714e00640 1 --2- 172.21.15.46:0/1691081673 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f471015efe0 0x7f471013bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.383+0000 7f4714e00640 1 --2- 172.21.15.46:0/1691081673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4710143eb0 0x7f471013d070 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.383+0000 7f4714e00640 1 --2- 172.21.15.46:0/1691081673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47100cbea0 0x7f471013c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.383+0000 7f4714e00640 1 -- 172.21.15.46:0/1691081673 >> 172.21.15.46:0/1691081673 conn(0x7f471006a390 msgr2=0x7f47100cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.383+0000 7f4714e00640 1 -- 172.21.15.46:0/1691081673 shutdown_connections 2024-03-29T08:27:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.383+0000 7f4714e00640 1 -- 172.21.15.46:0/1691081673 wait complete. 2024-03-29T08:27:16.405 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T08:27:16.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.543+0000 7fedcbe00640 1 Processor -- start 2024-03-29T08:27:16.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.543+0000 7fedcbe00640 1 -- start start 2024-03-29T08:27:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.543+0000 7fedcbe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fedc415ada0 0x7fedc40ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.543+0000 7fedcbe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fedc40ce950 0x7fedc406c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.543+0000 7fedcbe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc40cb510 con 0x7fedc4144210 2024-03-29T08:27:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.543+0000 7fedcbe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc40cede0 con 0x7fedc40ce950 2024-03-29T08:27:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.543+0000 7fedcbe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc4070580 con 0x7fedc415ada0 2024-03-29T08:27:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.543+0000 7fedc9400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fedc4144210 0x7fedc40cb8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38802/0 (socket says 172.21.15.46:38802) 2024-03-29T08:27:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.543+0000 7fedc9400640 1 -- 172.21.15.46:0/1386586036 learned_addr learned my addr 172.21.15.46:0/1386586036 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.543+0000 7fedc8a00640 1 --2- 172.21.15.46:0/1386586036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fedc415ada0 0x7fedc40ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.543+0000 7fedc9e00640 1 --2- 172.21.15.46:0/1386586036 >> v2:172.21.15.73:3300/0 conn(0x7fedc40ce950 0x7fedc406c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.543+0000 7fedc3e00640 1 -- 172.21.15.46:0/1386586036 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222687366 0 0) 0x7fedc40cb510 con 0x7fedc4144210 2024-03-29T08:27:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.543+0000 7fedc3e00640 1 -- 172.21.15.46:0/1386586036 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedac003530 con 0x7fedc4144210 2024-03-29T08:27:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.543+0000 7fedc8a00640 1 -- 172.21.15.46:0/1386586036 >> v2:172.21.15.73:3300/0 conn(0x7fedc40ce950 msgr2=0x7fedc406c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.543+0000 7fedc8a00640 1 --2- 172.21.15.46:0/1386586036 >> v2:172.21.15.73:3300/0 conn(0x7fedc40ce950 0x7fedc406c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.543+0000 7fedc8a00640 1 -- 172.21.15.46:0/1386586036 >> v1:172.21.15.46:6789/0 conn(0x7fedc4144210 legacy=0x7fedc40cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.543+0000 7fedc8a00640 1 -- 172.21.15.46:0/1386586036 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc4070860 con 0x7fedc415ada0 2024-03-29T08:27:16.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.547+0000 7fedc8a00640 1 --2- 172.21.15.46:0/1386586036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fedc415ada0 0x7fedc40ce410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fedb00097c0 tx=0x7fedb001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c849c67325e3942d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:16.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.547+0000 7fedc3e00640 1 -- 172.21.15.46:0/1386586036 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb00ad640 con 0x7fedc415ada0 2024-03-29T08:27:16.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.547+0000 7fedc3e00640 1 -- 172.21.15.46:0/1386586036 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb00ad7a0 con 0x7fedc415ada0 2024-03-29T08:27:16.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.547+0000 7fedc3e00640 1 -- 172.21.15.46:0/1386586036 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb00adb30 con 0x7fedc415ada0 2024-03-29T08:27:16.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.547+0000 7fedcbe00640 1 -- 172.21.15.46:0/1386586036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fedc415ada0 msgr2=0x7fedc40ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.547+0000 7fedcbe00640 1 --2- 172.21.15.46:0/1386586036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fedc415ada0 0x7fedc40ce410 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fedb00097c0 tx=0x7fedb001c4a0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.547+0000 7fedcbe00640 1 -- 172.21.15.46:0/1386586036 shutdown_connections 2024-03-29T08:27:16.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.547+0000 7fedcbe00640 1 --2- 172.21.15.46:0/1386586036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fedc415ada0 0x7fedc40ce410 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.547+0000 7fedcbe00640 1 --2- 172.21.15.46:0/1386586036 >> v2:172.21.15.73:3300/0 conn(0x7fedc40ce950 0x7fedc406c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.547+0000 7fedcbe00640 1 -- 172.21.15.46:0/1386586036 >> 172.21.15.46:0/1386586036 conn(0x7fedc4066110 msgr2=0x7fedc4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:16.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.547+0000 7fedcbe00640 1 -- 172.21.15.46:0/1386586036 shutdown_connections 2024-03-29T08:27:16.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.547+0000 7fedcbe00640 1 -- 172.21.15.46:0/1386586036 wait complete. 2024-03-29T08:27:16.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedcbe00640 1 Processor -- start 2024-03-29T08:27:16.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedcbe00640 1 -- start start 2024-03-29T08:27:16.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedcbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedc40ce950 0x7fedc415c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedcbe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fedc4144210 0x7fedc415e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedcbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedc415ada0 0x7fedc413c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedcbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc413dc40 con 0x7fedc40ce950 2024-03-29T08:27:16.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedcbe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc415ee00 con 0x7fedc415ada0 2024-03-29T08:27:16.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedcbe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc413de50 con 0x7fedc4144210 2024-03-29T08:27:16.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedc9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedc40ce950 0x7fedc415c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedc9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedc40ce950 0x7fedc415c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33470/0 (socket says 172.21.15.46:33470) 2024-03-29T08:27:16.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedc9400640 1 -- 172.21.15.46:0/4234200598 learned_addr learned my addr 172.21.15.46:0/4234200598 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:16.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedc9e00640 1 --2- 172.21.15.46:0/4234200598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedc415ada0 0x7fedc413c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedc8a00640 1 --2- 172.21.15.46:0/4234200598 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fedc4144210 0x7fedc415e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedc9400640 1 -- 172.21.15.46:0/4234200598 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fedc4144210 msgr2=0x7fedc415e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedc9400640 1 --2- 172.21.15.46:0/4234200598 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fedc4144210 0x7fedc415e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedc9400640 1 -- 172.21.15.46:0/4234200598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedc415ada0 msgr2=0x7fedc413c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedc9400640 1 --2- 172.21.15.46:0/4234200598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedc415ada0 0x7fedc413c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedc9400640 1 -- 172.21.15.46:0/4234200598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb0009420 con 0x7fedc40ce950 2024-03-29T08:27:16.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedc9400640 1 --2- 172.21.15.46:0/4234200598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedc40ce950 0x7fedc415c210 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fedb800c450 tx=0x7fedb800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:16.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedc1600640 1 -- 172.21.15.46:0/4234200598 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8005cf0 con 0x7fedc40ce950 2024-03-29T08:27:16.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedc1600640 1 -- 172.21.15.46:0/4234200598 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb8005e50 con 0x7fedc40ce950 2024-03-29T08:27:16.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedcbe00640 1 -- 172.21.15.46:0/4234200598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc413e130 con 0x7fedc40ce950 2024-03-29T08:27:16.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedc1600640 1 -- 172.21.15.46:0/4234200598 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb8007290 con 0x7fedc40ce950 2024-03-29T08:27:16.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.551+0000 7fedcbe00640 1 -- 172.21.15.46:0/4234200598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc415eb10 con 0x7fedc40ce950 2024-03-29T08:27:16.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.555+0000 7fedcbe00640 1 -- 172.21.15.46:0/4234200598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc40ce280 con 0x7fedc40ce950 2024-03-29T08:27:16.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.555+0000 7fedc1600640 1 -- 172.21.15.46:0/4234200598 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fedb80073f0 con 0x7fedc40ce950 2024-03-29T08:27:16.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.555+0000 7fedc1600640 1 --2- 172.21.15.46:0/4234200598 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7feda4066690 0x7feda4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.555+0000 7fedc1600640 1 -- 172.21.15.46:0/4234200598 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fedb8083400 con 0x7fedc40ce950 2024-03-29T08:27:16.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.555+0000 7fedc8a00640 1 --2- 172.21.15.46:0/4234200598 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7feda4066690 0x7feda4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.555+0000 7fedc8a00640 1 --2- 172.21.15.46:0/4234200598 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7feda4066690 0x7feda4068b50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fedb00af000 tx=0x7fedb00a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:16.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.559+0000 7fedc1600640 1 -- 172.21.15.46:0/4234200598 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fedb8050ab0 con 0x7fedc40ce950 2024-03-29T08:27:16.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.723+0000 7fedcbe00640 1 -- 172.21.15.46:0/4234200598 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fedc4066c90 con 0x7fedc40ce950 2024-03-29T08:27:16.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.727+0000 7fedc1600640 1 -- 172.21.15.46:0/4234200598 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v102) v1 ==== 84+0+65 (secure 0 0 0) 0x7fedb8017090 con 0x7fedc40ce950 2024-03-29T08:27:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:16.728 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-03-29T08:27:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:27:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-03-29T08:27:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-03-29T08:27:16.728 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:16.728 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-03-29T08:27:16.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.727+0000 7fedcbe00640 1 -- 172.21.15.46:0/4234200598 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7feda4066690 msgr2=0x7feda4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.727+0000 7fedcbe00640 1 --2- 172.21.15.46:0/4234200598 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7feda4066690 0x7feda4068b50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fedb00af000 tx=0x7fedb00a7070 comp rx=0 tx=0).stop 2024-03-29T08:27:16.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.727+0000 7fedcbe00640 1 -- 172.21.15.46:0/4234200598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedc40ce950 msgr2=0x7fedc415c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.727+0000 7fedcbe00640 1 --2- 172.21.15.46:0/4234200598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedc40ce950 0x7fedc415c210 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fedb800c450 tx=0x7fedb800c910 comp rx=0 tx=0).stop 2024-03-29T08:27:16.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.727+0000 7fedcbe00640 1 -- 172.21.15.46:0/4234200598 shutdown_connections 2024-03-29T08:27:16.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.727+0000 7fedcbe00640 1 --2- 172.21.15.46:0/4234200598 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7feda4066690 0x7feda4068b50 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.727+0000 7fedcbe00640 1 --2- 172.21.15.46:0/4234200598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedc415ada0 0x7fedc413c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.727+0000 7fedcbe00640 1 --2- 172.21.15.46:0/4234200598 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fedc4144210 0x7fedc415e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.727+0000 7fedcbe00640 1 --2- 172.21.15.46:0/4234200598 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fedc40ce950 0x7fedc415c210 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.727+0000 7fedcbe00640 1 -- 172.21.15.46:0/4234200598 >> 172.21.15.46:0/4234200598 conn(0x7fedc4066110 msgr2=0x7fedc40cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:16.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.727+0000 7fedcbe00640 1 -- 172.21.15.46:0/4234200598 shutdown_connections 2024-03-29T08:27:16.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.727+0000 7fedcbe00640 1 -- 172.21.15.46:0/4234200598 wait complete. 2024-03-29T08:27:16.749 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T08:27:16.749 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T08:27:16.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.891+0000 7fd805200640 1 Processor -- start 2024-03-29T08:27:16.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.891+0000 7fd805200640 1 -- start start 2024-03-29T08:27:16.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.891+0000 7fd805200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd8000cec80 0x7fd800141770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.891+0000 7fd805200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd800141cb0 0x7fd80006db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.891+0000 7fd805200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd80006cd80 con 0x7fd800140f40 2024-03-29T08:27:16.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.891+0000 7fd805200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8000cbff0 con 0x7fd800141cb0 2024-03-29T08:27:16.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.891+0000 7fd805200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd80006d060 con 0x7fd8000cec80 2024-03-29T08:27:16.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.891+0000 7fd7fde00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd800140f40 0x7fd8000cb8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38808/0 (socket says 172.21.15.46:38808) 2024-03-29T08:27:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.891+0000 7fd7fde00640 1 -- 172.21.15.46:0/2669506455 learned_addr learned my addr 172.21.15.46:0/2669506455 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.891+0000 7fd7fe800640 1 --2- 172.21.15.46:0/2669506455 >> v2:172.21.15.46:3300/0 conn(0x7fd800141cb0 0x7fd80006db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.891+0000 7fd7fd400640 1 -- 172.21.15.46:0/2669506455 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859373496 0 0) 0x7fd80006cd80 con 0x7fd800140f40 2024-03-29T08:27:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.891+0000 7fd7f5400640 1 --2- 172.21.15.46:0/2669506455 >> v2:172.21.15.73:3300/0 conn(0x7fd8000cec80 0x7fd800141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.891+0000 7fd7fd400640 1 -- 172.21.15.46:0/2669506455 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7e4003530 con 0x7fd800140f40 2024-03-29T08:27:16.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.891+0000 7fd7fe800640 1 -- 172.21.15.46:0/2669506455 >> v2:172.21.15.73:3300/0 conn(0x7fd8000cec80 msgr2=0x7fd800141770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:16.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.891+0000 7fd7fe800640 1 --2- 172.21.15.46:0/2669506455 >> v2:172.21.15.73:3300/0 conn(0x7fd8000cec80 0x7fd800141770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.891+0000 7fd7fe800640 1 -- 172.21.15.46:0/2669506455 >> v1:172.21.15.46:6789/0 conn(0x7fd800140f40 legacy=0x7fd8000cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.891+0000 7fd7fe800640 1 -- 172.21.15.46:0/2669506455 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80006d340 con 0x7fd800141cb0 2024-03-29T08:27:16.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.895+0000 7fd7fe800640 1 --2- 172.21.15.46:0/2669506455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd800141cb0 0x7fd80006db40 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fd800071560 tx=0x7fd7f801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85ee49b3a6c1fb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:16.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.895+0000 7fd7fd400640 1 -- 172.21.15.46:0/2669506455 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80ad660 con 0x7fd800141cb0 2024-03-29T08:27:16.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.895+0000 7fd7fd400640 1 -- 172.21.15.46:0/2669506455 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f80ad7c0 con 0x7fd800141cb0 2024-03-29T08:27:16.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.895+0000 7fd7fd400640 1 -- 172.21.15.46:0/2669506455 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80adb30 con 0x7fd800141cb0 2024-03-29T08:27:16.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.895+0000 7fd805200640 1 -- 172.21.15.46:0/2669506455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd800141cb0 msgr2=0x7fd80006db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.895+0000 7fd805200640 1 --2- 172.21.15.46:0/2669506455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd800141cb0 0x7fd80006db40 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fd800071560 tx=0x7fd7f801c450 comp rx=0 tx=0).stop 2024-03-29T08:27:16.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.895+0000 7fd805200640 1 -- 172.21.15.46:0/2669506455 shutdown_connections 2024-03-29T08:27:16.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.895+0000 7fd805200640 1 --2- 172.21.15.46:0/2669506455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd800141cb0 0x7fd80006db40 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.895+0000 7fd805200640 1 --2- 172.21.15.46:0/2669506455 >> v2:172.21.15.73:3300/0 conn(0x7fd8000cec80 0x7fd800141770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.895+0000 7fd805200640 1 -- 172.21.15.46:0/2669506455 >> 172.21.15.46:0/2669506455 conn(0x7fd8000b42d0 msgr2=0x7fd800069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:16.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.895+0000 7fd805200640 1 -- 172.21.15.46:0/2669506455 shutdown_connections 2024-03-29T08:27:16.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.895+0000 7fd805200640 1 -- 172.21.15.46:0/2669506455 wait complete. 2024-03-29T08:27:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd805200640 1 Processor -- start 2024-03-29T08:27:16.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd805200640 1 -- start start 2024-03-29T08:27:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd805200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8000cec80 0x7fd80013afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd805200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd800140f40 0x7fd800137a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd805200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd800141cb0 0x7fd800137f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd805200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8001535f0 con 0x7fd800140f40 2024-03-29T08:27:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd805200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80013c980 con 0x7fd8000cec80 2024-03-29T08:27:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd805200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd80013b8e0 con 0x7fd800141cb0 2024-03-29T08:27:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd7f5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd800140f40 0x7fd800137a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd7f5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd800140f40 0x7fd800137a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33490/0 (socket says 172.21.15.46:33490) 2024-03-29T08:27:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd7f5400640 1 -- 172.21.15.46:0/3646247150 learned_addr learned my addr 172.21.15.46:0/3646247150 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd7fde00640 1 --2- 172.21.15.46:0/3646247150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8000cec80 0x7fd80013afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd7fe800640 1 --2- 172.21.15.46:0/3646247150 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd800141cb0 0x7fd800137f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd7f5400640 1 -- 172.21.15.46:0/3646247150 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd800141cb0 msgr2=0x7fd800137f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd7f5400640 1 --2- 172.21.15.46:0/3646247150 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd800141cb0 0x7fd800137f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd7f5400640 1 -- 172.21.15.46:0/3646247150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8000cec80 msgr2=0x7fd80013afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd7f5400640 1 --2- 172.21.15.46:0/3646247150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8000cec80 0x7fd80013afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd7f5400640 1 -- 172.21.15.46:0/3646247150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f8009420 con 0x7fd800140f40 2024-03-29T08:27:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd7f5400640 1 --2- 172.21.15.46:0/3646247150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd800140f40 0x7fd800137a00 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800d830 tx=0x7fd7e800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd7f6a00640 1 -- 172.21.15.46:0/3646247150 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e800b6c0 con 0x7fd800140f40 2024-03-29T08:27:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd7f6a00640 1 -- 172.21.15.46:0/3646247150 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e800b820 con 0x7fd800140f40 2024-03-29T08:27:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd7f6a00640 1 -- 172.21.15.46:0/3646247150 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e800bb90 con 0x7fd800140f40 2024-03-29T08:27:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd805200640 1 -- 172.21.15.46:0/3646247150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80013bbc0 con 0x7fd800140f40 2024-03-29T08:27:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd805200640 1 -- 172.21.15.46:0/3646247150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80013c180 con 0x7fd800140f40 2024-03-29T08:27:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.899+0000 7fd805200640 1 -- 172.21.15.46:0/3646247150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd800066c90 con 0x7fd800140f40 2024-03-29T08:27:16.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.903+0000 7fd7f6a00640 1 -- 172.21.15.46:0/3646247150 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd7e801e460 con 0x7fd800140f40 2024-03-29T08:27:16.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.903+0000 7fd7f6a00640 1 --2- 172.21.15.46:0/3646247150 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7dc066690 0x7fd7dc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:16.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.903+0000 7fd7f6a00640 1 -- 172.21.15.46:0/3646247150 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd7e8084590 con 0x7fd800140f40 2024-03-29T08:27:16.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.903+0000 7fd7fde00640 1 --2- 172.21.15.46:0/3646247150 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7dc066690 0x7fd7dc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.903+0000 7fd7fde00640 1 --2- 172.21.15.46:0/3646247150 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7dc066690 0x7fd7dc068b50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0005c10 tx=0x7fd7f0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:16.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:16.907+0000 7fd7f6a00640 1 -- 172.21.15.46:0/3646247150 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd7e8051bd0 con 0x7fd800140f40 2024-03-29T08:27:17.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.067+0000 7fd805200640 1 -- 172.21.15.46:0/3646247150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd80006a070 con 0x7fd800140f40 2024-03-29T08:27:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.167+0000 7fd7f6a00640 1 -- 172.21.15.46:0/3646247150 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v103) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd7e80551f0 con 0x7fd800140f40 2024-03-29T08:27:17.171 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-03-29T08:27:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.171+0000 7fd805200640 1 -- 172.21.15.46:0/3646247150 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7dc066690 msgr2=0x7fd7dc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.171+0000 7fd805200640 1 --2- 172.21.15.46:0/3646247150 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7dc066690 0x7fd7dc068b50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0005c10 tx=0x7fd7f0009040 comp rx=0 tx=0).stop 2024-03-29T08:27:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.171+0000 7fd805200640 1 -- 172.21.15.46:0/3646247150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd800140f40 msgr2=0x7fd800137a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.171+0000 7fd805200640 1 --2- 172.21.15.46:0/3646247150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd800140f40 0x7fd800137a00 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800d830 tx=0x7fd7e800dcf0 comp rx=0 tx=0).stop 2024-03-29T08:27:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.171+0000 7fd805200640 1 -- 172.21.15.46:0/3646247150 shutdown_connections 2024-03-29T08:27:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.171+0000 7fd805200640 1 --2- 172.21.15.46:0/3646247150 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7dc066690 0x7fd7dc068b50 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.171+0000 7fd805200640 1 --2- 172.21.15.46:0/3646247150 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd800141cb0 0x7fd800137f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.171+0000 7fd805200640 1 --2- 172.21.15.46:0/3646247150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd800140f40 0x7fd800137a00 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.171+0000 7fd805200640 1 --2- 172.21.15.46:0/3646247150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8000cec80 0x7fd80013afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.171+0000 7fd805200640 1 -- 172.21.15.46:0/3646247150 >> 172.21.15.46:0/3646247150 conn(0x7fd8000b42d0 msgr2=0x7fd8000af630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.171+0000 7fd805200640 1 -- 172.21.15.46:0/3646247150 shutdown_connections 2024-03-29T08:27:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.171+0000 7fd805200640 1 -- 172.21.15.46:0/3646247150 wait complete. 2024-03-29T08:27:17.197 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T08:27:17.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.343+0000 7f6137600640 1 Processor -- start 2024-03-29T08:27:17.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.343+0000 7f6137600640 1 -- start start 2024-03-29T08:27:17.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.343+0000 7f6137600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f613015c950 0x7f613006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:17.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.343+0000 7f6137600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61300cea30 0x7f61301606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:17.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.343+0000 7f6137600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61300cfb20 con 0x7f613015cd30 2024-03-29T08:27:17.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.343+0000 7f6137600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61300cef10 con 0x7f613015c950 2024-03-29T08:27:17.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.343+0000 7f6137600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61300cf1f0 con 0x7f61300cea30 2024-03-29T08:27:17.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.343+0000 7f6134c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f613015c950 0x7f613006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:17.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.347+0000 7f6134c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f613015c950 0x7f613006bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33504/0 (socket says 172.21.15.46:33504) 2024-03-29T08:27:17.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.347+0000 7f6134c00640 1 -- 172.21.15.46:0/1610447366 learned_addr learned my addr 172.21.15.46:0/1610447366 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:17.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.347+0000 7f6127e00640 1 --2- 172.21.15.46:0/1610447366 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61300cea30 0x7f61301606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:17.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.347+0000 7f6134c00640 1 -- 172.21.15.46:0/1610447366 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61300cea30 msgr2=0x7f61301606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:17.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.347+0000 7f6134c00640 1 --2- 172.21.15.46:0/1610447366 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61300cea30 0x7f61301606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:17.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.347+0000 7f6134c00640 1 -- 172.21.15.46:0/1610447366 >> v1:172.21.15.73:6789/0 conn(0x7f613015cd30 legacy=0x7f61300cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:17.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.347+0000 7f6134c00640 1 -- 172.21.15.46:0/1610447366 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61300cf4d0 con 0x7f613015c950 2024-03-29T08:27:17.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.347+0000 7f6134c00640 1 --2- 172.21.15.46:0/1610447366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613015c950 0x7f613006bbe0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f6120009870 tx=0x7f612001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f624fd941477d766 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:17.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.347+0000 7f612fe00640 1 -- 172.21.15.46:0/1610447366 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61200a8a50 con 0x7f613015c950 2024-03-29T08:27:17.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.347+0000 7f612fe00640 1 -- 172.21.15.46:0/1610447366 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61200a8bb0 con 0x7f613015c950 2024-03-29T08:27:17.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.347+0000 7f612fe00640 1 -- 172.21.15.46:0/1610447366 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61200b0440 con 0x7f613015c950 2024-03-29T08:27:17.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6137600640 1 -- 172.21.15.46:0/1610447366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613015c950 msgr2=0x7f613006bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:17.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6137600640 1 --2- 172.21.15.46:0/1610447366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613015c950 0x7f613006bbe0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f6120009870 tx=0x7f612001ed50 comp rx=0 tx=0).stop 2024-03-29T08:27:17.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6137600640 1 -- 172.21.15.46:0/1610447366 shutdown_connections 2024-03-29T08:27:17.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6137600640 1 --2- 172.21.15.46:0/1610447366 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61300cea30 0x7f61301606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:17.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6137600640 1 --2- 172.21.15.46:0/1610447366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613015c950 0x7f613006bbe0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:17.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6137600640 1 -- 172.21.15.46:0/1610447366 >> 172.21.15.46:0/1610447366 conn(0x7f61300b21b0 msgr2=0x7f61300b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:17.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6137600640 1 -- 172.21.15.46:0/1610447366 shutdown_connections 2024-03-29T08:27:17.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6137600640 1 -- 172.21.15.46:0/1610447366 wait complete. 2024-03-29T08:27:17.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6137600640 1 Processor -- start 2024-03-29T08:27:17.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6137600640 1 -- start start 2024-03-29T08:27:17.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6137600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61300cea30 0x7f61301f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:17.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6137600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f613015c950 0x7f61301f6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:17.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6137600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613015cd30 0x7f6130157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:17.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6137600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61300b31e0 con 0x7f613015cd30 2024-03-29T08:27:17.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6137600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61301f6b90 con 0x7f613015c950 2024-03-29T08:27:17.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6137600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61301f6e70 con 0x7f61300cea30 2024-03-29T08:27:17.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6135600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613015cd30 0x7f6130157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:17.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6135600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613015cd30 0x7f6130157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33516/0 (socket says 172.21.15.46:33516) 2024-03-29T08:27:17.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6135600640 1 -- 172.21.15.46:0/614390971 learned_addr learned my addr 172.21.15.46:0/614390971 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:17.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6127e00640 1 --2- 172.21.15.46:0/614390971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f613015c950 0x7f61301f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:17.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6134c00640 1 --2- 172.21.15.46:0/614390971 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61300cea30 0x7f61301f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:17.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6135600640 1 -- 172.21.15.46:0/614390971 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61300cea30 msgr2=0x7f61301f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:17.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6135600640 1 --2- 172.21.15.46:0/614390971 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61300cea30 0x7f61301f3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:17.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6135600640 1 -- 172.21.15.46:0/614390971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f613015c950 msgr2=0x7f61301f6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:17.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6135600640 1 --2- 172.21.15.46:0/614390971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f613015c950 0x7f61301f6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:17.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6135600640 1 -- 172.21.15.46:0/614390971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61200094d0 con 0x7f613015cd30 2024-03-29T08:27:17.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.351+0000 7f6135600640 1 --2- 172.21.15.46:0/614390971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613015cd30 0x7f6130157540 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f612800c730 tx=0x7f612800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:17.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.355+0000 7f612d600640 1 -- 172.21.15.46:0/614390971 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128003b80 con 0x7f613015cd30 2024-03-29T08:27:17.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.355+0000 7f612d600640 1 -- 172.21.15.46:0/614390971 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6128003ce0 con 0x7f613015cd30 2024-03-29T08:27:17.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.355+0000 7f6137600640 1 -- 172.21.15.46:0/614390971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61301f7150 con 0x7f613015cd30 2024-03-29T08:27:17.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.355+0000 7f612d600640 1 -- 172.21.15.46:0/614390971 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612800aa30 con 0x7f613015cd30 2024-03-29T08:27:17.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.355+0000 7f6137600640 1 -- 172.21.15.46:0/614390971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61301f6880 con 0x7f613015cd30 2024-03-29T08:27:17.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.355+0000 7f6137600640 1 -- 172.21.15.46:0/614390971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613009e690 con 0x7f613015cd30 2024-03-29T08:27:17.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.355+0000 7f612d600640 1 -- 172.21.15.46:0/614390971 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f612800ab90 con 0x7f613015cd30 2024-03-29T08:27:17.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.355+0000 7f612d600640 1 --2- 172.21.15.46:0/614390971 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f610c066690 0x7f610c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:17.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.355+0000 7f612d600640 1 -- 172.21.15.46:0/614390971 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6128083ce0 con 0x7f613015cd30 2024-03-29T08:27:17.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.355+0000 7f6134c00640 1 --2- 172.21.15.46:0/614390971 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f610c066690 0x7f610c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:17.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.359+0000 7f6134c00640 1 --2- 172.21.15.46:0/614390971 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f610c066690 0x7f610c068b50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f61200b2000 tx=0x7f6120009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:17.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.359+0000 7f612d600640 1 -- 172.21.15.46:0/614390971 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6128051240 con 0x7f613015cd30 2024-03-29T08:27:17.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:17.523+0000 7f6137600640 1 -- 172.21.15.46:0/614390971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f61300b3650 con 0x7f613015cd30 2024-03-29T08:27:18.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.171+0000 7f612d600640 1 -- 172.21.15.46:0/614390971 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v104) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6128054860 con 0x7f613015cd30 2024-03-29T08:27:18.175 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-03-29T08:27:18.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.179+0000 7f6137600640 1 -- 172.21.15.46:0/614390971 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f610c066690 msgr2=0x7f610c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:18.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.179+0000 7f6137600640 1 --2- 172.21.15.46:0/614390971 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f610c066690 0x7f610c068b50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f61200b2000 tx=0x7f6120009040 comp rx=0 tx=0).stop 2024-03-29T08:27:18.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.179+0000 7f6137600640 1 -- 172.21.15.46:0/614390971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613015cd30 msgr2=0x7f6130157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:18.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.179+0000 7f6137600640 1 --2- 172.21.15.46:0/614390971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613015cd30 0x7f6130157540 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f612800c730 tx=0x7f612800cbf0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.179+0000 7f6137600640 1 -- 172.21.15.46:0/614390971 shutdown_connections 2024-03-29T08:27:18.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.179+0000 7f6137600640 1 --2- 172.21.15.46:0/614390971 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f610c066690 0x7f610c068b50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.179+0000 7f6137600640 1 --2- 172.21.15.46:0/614390971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f613015c950 0x7f61301f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.179+0000 7f6137600640 1 --2- 172.21.15.46:0/614390971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f613015cd30 0x7f6130157540 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.179+0000 7f6137600640 1 --2- 172.21.15.46:0/614390971 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61300cea30 0x7f61301f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.179+0000 7f6137600640 1 -- 172.21.15.46:0/614390971 >> 172.21.15.46:0/614390971 conn(0x7f61300b21b0 msgr2=0x7f613015d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:18.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.183+0000 7f6137600640 1 -- 172.21.15.46:0/614390971 shutdown_connections 2024-03-29T08:27:18.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.183+0000 7f6137600640 1 -- 172.21.15.46:0/614390971 wait complete. 2024-03-29T08:27:18.217 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T08:27:18.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.359+0000 7f5e43c00640 1 Processor -- start 2024-03-29T08:27:18.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.359+0000 7f5e43c00640 1 -- start start 2024-03-29T08:27:18.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e43c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5e3c070650 0x7f5e3c06c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:18.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e43c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5e3c0b2450 0x7f5e3c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:18.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e43c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5e3c06dbe0 0x7f5e3c0abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:18.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e43c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c0b15b0 con 0x7f5e3c06dbe0 2024-03-29T08:27:18.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e43c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c06c910 con 0x7f5e3c070650 2024-03-29T08:27:18.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e43c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c070f70 con 0x7f5e3c0b2450 2024-03-29T08:27:18.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e41c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5e3c06dbe0 0x7f5e3c0abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:18.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e3be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5e3c0b2450 0x7f5e3c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:18.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e41200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5e3c070650 0x7f5e3c06c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:18.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e41c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5e3c06dbe0 0x7f5e3c0abcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33530/0 (socket says 172.21.15.46:33530) 2024-03-29T08:27:18.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e41c00640 1 -- 172.21.15.46:0/413810952 learned_addr learned my addr 172.21.15.46:0/413810952 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:18.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e3be00640 1 -- 172.21.15.46:0/413810952 >> v2:172.21.15.73:3300/0 conn(0x7f5e3c070650 msgr2=0x7f5e3c06c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:18.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e3be00640 1 --2- 172.21.15.46:0/413810952 >> v2:172.21.15.73:3300/0 conn(0x7f5e3c070650 0x7f5e3c06c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e3be00640 1 -- 172.21.15.46:0/413810952 >> v2:172.21.15.46:3300/0 conn(0x7f5e3c06dbe0 msgr2=0x7f5e3c0abcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:18.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e3be00640 1 --2- 172.21.15.46:0/413810952 >> v2:172.21.15.46:3300/0 conn(0x7f5e3c06dbe0 0x7f5e3c0abcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e3be00640 1 -- 172.21.15.46:0/413810952 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c06cb40 con 0x7f5e3c0b2450 2024-03-29T08:27:18.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e41200640 1 --2- 172.21.15.46:0/413810952 >> v2:172.21.15.73:3300/0 conn(0x7f5e3c070650 0x7f5e3c06c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:27:18.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.363+0000 7f5e41c00640 1 --2- 172.21.15.46:0/413810952 >> v2:172.21.15.46:3300/0 conn(0x7f5e3c06dbe0 0x7f5e3c0abcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:27:18.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.367+0000 7f5e3be00640 1 --2- 172.21.15.46:0/413810952 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5e3c0b2450 0x7f5e3c070a30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0cf9a0 tx=0x7f5e2801c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23fa1d47e348b55e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:18.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.367+0000 7f5e3b400640 1 -- 172.21.15.46:0/413810952 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e280ad660 con 0x7f5e3c0b2450 2024-03-29T08:27:18.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.367+0000 7f5e3b400640 1 -- 172.21.15.46:0/413810952 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e280ad7c0 con 0x7f5e3c0b2450 2024-03-29T08:27:18.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.367+0000 7f5e3b400640 1 -- 172.21.15.46:0/413810952 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e280adb50 con 0x7f5e3c0b2450 2024-03-29T08:27:18.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.367+0000 7f5e43c00640 1 -- 172.21.15.46:0/413810952 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5e3c0b2450 msgr2=0x7f5e3c070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:18.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.367+0000 7f5e43c00640 1 --2- 172.21.15.46:0/413810952 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5e3c0b2450 0x7f5e3c070a30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c0cf9a0 tx=0x7f5e2801c420 comp rx=0 tx=0).stop 2024-03-29T08:27:18.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.367+0000 7f5e43c00640 1 -- 172.21.15.46:0/413810952 shutdown_connections 2024-03-29T08:27:18.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.367+0000 7f5e43c00640 1 --2- 172.21.15.46:0/413810952 >> v2:172.21.15.46:3300/0 conn(0x7f5e3c06dbe0 0x7f5e3c0abcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.367+0000 7f5e43c00640 1 --2- 172.21.15.46:0/413810952 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5e3c0b2450 0x7f5e3c070a30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.367+0000 7f5e43c00640 1 --2- 172.21.15.46:0/413810952 >> v2:172.21.15.73:3300/0 conn(0x7f5e3c070650 0x7f5e3c06c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.367+0000 7f5e43c00640 1 -- 172.21.15.46:0/413810952 >> 172.21.15.46:0/413810952 conn(0x7f5e3c0b0640 msgr2=0x7f5e3c0b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:18.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.367+0000 7f5e43c00640 1 -- 172.21.15.46:0/413810952 shutdown_connections 2024-03-29T08:27:18.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.367+0000 7f5e43c00640 1 -- 172.21.15.46:0/413810952 wait complete. 2024-03-29T08:27:18.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.367+0000 7f5e43c00640 1 Processor -- start 2024-03-29T08:27:18.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.367+0000 7f5e43c00640 1 -- start start 2024-03-29T08:27:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e43c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5e3c06dbe0 0x7f5e3c15f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e43c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c070650 0x7f5e3c13e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e43c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e3c0b2450 0x7f5e3c13eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e43c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c167000 con 0x7f5e3c0b2450 2024-03-29T08:27:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e43c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c144fa0 con 0x7f5e3c070650 2024-03-29T08:27:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e43c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c145280 con 0x7f5e3c06dbe0 2024-03-29T08:27:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e41c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e3c0b2450 0x7f5e3c13eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e41c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e3c0b2450 0x7f5e3c13eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33532/0 (socket says 172.21.15.46:33532) 2024-03-29T08:27:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e41c00640 1 -- 172.21.15.46:0/2486228165 learned_addr learned my addr 172.21.15.46:0/2486228165 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e3be00640 1 --2- 172.21.15.46:0/2486228165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c070650 0x7f5e3c13e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e41200640 1 --2- 172.21.15.46:0/2486228165 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5e3c06dbe0 0x7f5e3c15f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e41c00640 1 -- 172.21.15.46:0/2486228165 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5e3c06dbe0 msgr2=0x7f5e3c15f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e41c00640 1 --2- 172.21.15.46:0/2486228165 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5e3c06dbe0 0x7f5e3c15f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e41c00640 1 -- 172.21.15.46:0/2486228165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c070650 msgr2=0x7f5e3c13e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:18.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e41c00640 1 --2- 172.21.15.46:0/2486228165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c070650 0x7f5e3c13e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e41c00640 1 -- 172.21.15.46:0/2486228165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e34009b30 con 0x7f5e3c0b2450 2024-03-29T08:27:18.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e41200640 1 --2- 172.21.15.46:0/2486228165 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5e3c06dbe0 0x7f5e3c15f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:18.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e41c00640 1 --2- 172.21.15.46:0/2486228165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e3c0b2450 0x7f5e3c13eee0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400eed0 tx=0x7f5e3400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:18.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e38c00640 1 -- 172.21.15.46:0/2486228165 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34018070 con 0x7f5e3c0b2450 2024-03-29T08:27:18.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e38c00640 1 -- 172.21.15.46:0/2486228165 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e340092e0 con 0x7f5e3c0b2450 2024-03-29T08:27:18.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e43c00640 1 -- 172.21.15.46:0/2486228165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e30009420 con 0x7f5e3c0b2450 2024-03-29T08:27:18.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e38c00640 1 -- 172.21.15.46:0/2486228165 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34010040 con 0x7f5e3c0b2450 2024-03-29T08:27:18.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e43c00640 1 -- 172.21.15.46:0/2486228165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e28009420 con 0x7f5e3c0b2450 2024-03-29T08:27:18.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e43c00640 1 -- 172.21.15.46:0/2486228165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c0667c0 con 0x7f5e3c0b2450 2024-03-29T08:27:18.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.371+0000 7f5e38c00640 1 -- 172.21.15.46:0/2486228165 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5e3400bd10 con 0x7f5e3c0b2450 2024-03-29T08:27:18.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.375+0000 7f5e38c00640 1 --2- 172.21.15.46:0/2486228165 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5e1c0666c0 0x7f5e1c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:18.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.375+0000 7f5e38c00640 1 -- 172.21.15.46:0/2486228165 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f5e340841a0 con 0x7f5e3c0b2450 2024-03-29T08:27:18.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.375+0000 7f5e41200640 1 --2- 172.21.15.46:0/2486228165 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5e1c0666c0 0x7f5e1c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:18.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.375+0000 7f5e41200640 1 --2- 172.21.15.46:0/2486228165 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5e1c0666c0 0x7f5e1c068b80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009d40 tx=0x7f5e30008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:18.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.379+0000 7f5e38c00640 1 -- 172.21.15.46:0/2486228165 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5e340514d0 con 0x7f5e3c0b2450 2024-03-29T08:27:18.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.579+0000 7f5e43c00640 1 -- 172.21.15.46:0/2486228165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5e3c0ae390 con 0x7f5e3c0b2450 2024-03-29T08:27:18.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.583+0000 7f5e38c00640 1 -- 172.21.15.46:0/2486228165 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v73) v1 ==== 174+0+0 (secure 0 0 0) 0x7f5e34054af0 con 0x7f5e3c0b2450 2024-03-29T08:27:18.587 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 14 and data pool 15 2024-03-29T08:27:18.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.587+0000 7f5e43c00640 1 -- 172.21.15.46:0/2486228165 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5e1c0666c0 msgr2=0x7f5e1c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:18.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.587+0000 7f5e43c00640 1 --2- 172.21.15.46:0/2486228165 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5e1c0666c0 0x7f5e1c068b80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009d40 tx=0x7f5e30008040 comp rx=0 tx=0).stop 2024-03-29T08:27:18.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.587+0000 7f5e43c00640 1 -- 172.21.15.46:0/2486228165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e3c0b2450 msgr2=0x7f5e3c13eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:18.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.587+0000 7f5e43c00640 1 --2- 172.21.15.46:0/2486228165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e3c0b2450 0x7f5e3c13eee0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400eed0 tx=0x7f5e3400b480 comp rx=0 tx=0).stop 2024-03-29T08:27:18.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.587+0000 7f5e43c00640 1 -- 172.21.15.46:0/2486228165 shutdown_connections 2024-03-29T08:27:18.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.587+0000 7f5e43c00640 1 --2- 172.21.15.46:0/2486228165 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5e1c0666c0 0x7f5e1c068b80 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.587+0000 7f5e43c00640 1 --2- 172.21.15.46:0/2486228165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c070650 0x7f5e3c13e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.587+0000 7f5e43c00640 1 --2- 172.21.15.46:0/2486228165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e3c0b2450 0x7f5e3c13eee0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.587+0000 7f5e43c00640 1 --2- 172.21.15.46:0/2486228165 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5e3c06dbe0 0x7f5e3c15f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.587+0000 7f5e43c00640 1 -- 172.21.15.46:0/2486228165 >> 172.21.15.46:0/2486228165 conn(0x7f5e3c0b0640 msgr2=0x7f5e3c0aca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:18.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.587+0000 7f5e43c00640 1 -- 172.21.15.46:0/2486228165 shutdown_connections 2024-03-29T08:27:18.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.587+0000 7f5e43c00640 1 -- 172.21.15.46:0/2486228165 wait complete. 2024-03-29T08:27:18.627 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:27:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7df1400640 1 Processor -- start 2024-03-29T08:27:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7df1400640 1 -- start start 2024-03-29T08:27:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7df1400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7dec06b8b0 0x7f7dec0cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7df1400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7dec15dc00 0x7f7dec0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7df1400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dec15cfb0 con 0x7f7dec0d0660 2024-03-29T08:27:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7df1400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dec06c100 con 0x7f7dec15dc00 2024-03-29T08:27:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7df1400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec06be20 con 0x7f7dec06b8b0 2024-03-29T08:27:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7deb200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7dec15dc00 0x7f7dec0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7de1e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7dec06b8b0 0x7f7dec0cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7deb200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7dec15dc00 0x7f7dec0cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:60458/0 (socket says 172.21.15.46:60458) 2024-03-29T08:27:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7dea800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7dec0d0660 0x7f7dec0d0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:56644/0 (socket says 172.21.15.46:56644) 2024-03-29T08:27:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7deb200640 1 -- 172.21.15.46:0/2163483679 learned_addr learned my addr 172.21.15.46:0/2163483679 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7de9e00640 1 -- 172.21.15.46:0/2163483679 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982844090 0 0) 0x7f7dec15cfb0 con 0x7f7dec0d0660 2024-03-29T08:27:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7de9e00640 1 -- 172.21.15.46:0/2163483679 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dd0003530 con 0x7f7dec0d0660 2024-03-29T08:27:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7deb200640 1 -- 172.21.15.46:0/2163483679 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7dec06b8b0 msgr2=0x7f7dec0cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7deb200640 1 --2- 172.21.15.46:0/2163483679 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7dec06b8b0 0x7f7dec0cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7deb200640 1 -- 172.21.15.46:0/2163483679 >> v1:172.21.15.73:6789/0 conn(0x7f7dec0d0660 legacy=0x7f7dec0d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7deb200640 1 -- 172.21.15.46:0/2163483679 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dec0cecf0 con 0x7f7dec15dc00 2024-03-29T08:27:18.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.767+0000 7f7de1e00640 1 --2- 172.21.15.46:0/2163483679 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7dec06b8b0 0x7f7dec0cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:27:18.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.771+0000 7f7deb200640 1 --2- 172.21.15.46:0/2163483679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dec15dc00 0x7f7dec0cf7d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009870 tx=0x7f7de401c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4d8756d2aac12e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:18.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.771+0000 7f7de9e00640 1 -- 172.21.15.46:0/2163483679 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de40a7070 con 0x7f7dec15dc00 2024-03-29T08:27:18.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.771+0000 7f7de9e00640 1 -- 172.21.15.46:0/2163483679 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de40ab430 con 0x7f7dec15dc00 2024-03-29T08:27:18.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.771+0000 7f7de9e00640 1 -- 172.21.15.46:0/2163483679 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de40ab7c0 con 0x7f7dec15dc00 2024-03-29T08:27:18.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.771+0000 7f7df1400640 1 -- 172.21.15.46:0/2163483679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dec15dc00 msgr2=0x7f7dec0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:18.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.771+0000 7f7df1400640 1 --2- 172.21.15.46:0/2163483679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dec15dc00 0x7f7dec0cf7d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009870 tx=0x7f7de401c500 comp rx=0 tx=0).stop 2024-03-29T08:27:18.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.771+0000 7f7df1400640 1 -- 172.21.15.46:0/2163483679 shutdown_connections 2024-03-29T08:27:18.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.771+0000 7f7df1400640 1 --2- 172.21.15.46:0/2163483679 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7dec06b8b0 0x7f7dec0cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.771+0000 7f7df1400640 1 --2- 172.21.15.46:0/2163483679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dec15dc00 0x7f7dec0cf7d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.771+0000 7f7df1400640 1 -- 172.21.15.46:0/2163483679 >> 172.21.15.46:0/2163483679 conn(0x7f7dec089fd0 msgr2=0x7f7dec08a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:18.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.771+0000 7f7df1400640 1 -- 172.21.15.46:0/2163483679 shutdown_connections 2024-03-29T08:27:18.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.771+0000 7f7df1400640 1 -- 172.21.15.46:0/2163483679 wait complete. 2024-03-29T08:27:18.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7df1400640 1 Processor -- start 2024-03-29T08:27:18.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7df1400640 1 -- start start 2024-03-29T08:27:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7df1400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7dec06b8b0 0x7f7dec0b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7df1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec0d0660 0x7f7dec0b5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7df1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dec15dc00 0x7f7dec154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7df1400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec0b4600 con 0x7f7dec0d0660 2024-03-29T08:27:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7df1400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec155650 con 0x7f7dec15dc00 2024-03-29T08:27:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7df1400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec0b3cd0 con 0x7f7dec06b8b0 2024-03-29T08:27:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7de1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec0d0660 0x7f7dec0b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7de1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec0d0660 0x7f7dec0b5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33544/0 (socket says 172.21.15.46:33544) 2024-03-29T08:27:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7de1e00640 1 -- 172.21.15.46:0/2148366288 learned_addr learned my addr 172.21.15.46:0/2148366288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7dea800640 1 --2- 172.21.15.46:0/2148366288 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7dec06b8b0 0x7f7dec0b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:18.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7deb200640 1 --2- 172.21.15.46:0/2148366288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dec15dc00 0x7f7dec154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7de1e00640 1 -- 172.21.15.46:0/2148366288 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7dec06b8b0 msgr2=0x7f7dec0b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7de1e00640 1 --2- 172.21.15.46:0/2148366288 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7dec06b8b0 0x7f7dec0b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7de1e00640 1 -- 172.21.15.46:0/2148366288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dec15dc00 msgr2=0x7f7dec154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7de1e00640 1 --2- 172.21.15.46:0/2148366288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dec15dc00 0x7f7dec154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7de1e00640 1 -- 172.21.15.46:0/2148366288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd4009a70 con 0x7f7dec0d0660 2024-03-29T08:27:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7dea800640 1 --2- 172.21.15.46:0/2148366288 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7dec06b8b0 0x7f7dec0b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7deb200640 1 --2- 172.21.15.46:0/2148366288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dec15dc00 0x7f7dec154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7de1e00640 1 --2- 172.21.15.46:0/2148366288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec0d0660 0x7f7dec0b5820 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400efc0 tx=0x7f7dd400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7de3400640 1 -- 172.21.15.46:0/2148366288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4014680 con 0x7f7dec0d0660 2024-03-29T08:27:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7de3400640 1 -- 172.21.15.46:0/2148366288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd400f040 con 0x7f7dec0d0660 2024-03-29T08:27:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7de3400640 1 -- 172.21.15.46:0/2148366288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4014a00 con 0x7f7dec0d0660 2024-03-29T08:27:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7df1400640 1 -- 172.21.15.46:0/2148366288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de40094d0 con 0x7f7dec0d0660 2024-03-29T08:27:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7df1400640 1 -- 172.21.15.46:0/2148366288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dec0b4290 con 0x7f7dec0d0660 2024-03-29T08:27:18.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.775+0000 7f7df1400640 1 -- 172.21.15.46:0/2148366288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dec0b47b0 con 0x7f7dec0d0660 2024-03-29T08:27:18.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.779+0000 7f7de3400640 1 -- 172.21.15.46:0/2148366288 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7dd4014b60 con 0x7f7dec0d0660 2024-03-29T08:27:18.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.779+0000 7f7de3400640 1 --2- 172.21.15.46:0/2148366288 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7dc80666c0 0x7f7dc8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:18.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.779+0000 7f7de3400640 1 -- 172.21.15.46:0/2148366288 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7dd4085240 con 0x7f7dec0d0660 2024-03-29T08:27:18.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.779+0000 7f7dea800640 1 --2- 172.21.15.46:0/2148366288 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7dc80666c0 0x7f7dc8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:18.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.779+0000 7f7dea800640 1 --2- 172.21.15.46:0/2148366288 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7dc80666c0 0x7f7dc8068b80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0011f0 tx=0x7f7ddc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:18.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.783+0000 7f7de3400640 1 -- 172.21.15.46:0/2148366288 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7dec0b47b0 con 0x7f7dec0d0660 2024-03-29T08:27:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.943+0000 7f7df1400640 1 -- 172.21.15.46:0/2148366288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7dec066c90 con 0x7f7dec0d0660 2024-03-29T08:27:18.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.947+0000 7f7de3400640 1 -- 172.21.15.46:0/2148366288 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16016 (secure 0 0 0) 0x7f7dd4052510 con 0x7f7dec0d0660 2024-03-29T08:27:18.948 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:18.948 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":105,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:27:18.582894+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:27:17.118875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-29T08:27:17.573928+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:27:18.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.947+0000 7f7df1400640 1 -- 172.21.15.46:0/2148366288 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7dc80666c0 msgr2=0x7f7dc8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:18.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.947+0000 7f7df1400640 1 --2- 172.21.15.46:0/2148366288 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7dc80666c0 0x7f7dc8068b80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0011f0 tx=0x7f7ddc009040 comp rx=0 tx=0).stop 2024-03-29T08:27:18.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.947+0000 7f7df1400640 1 -- 172.21.15.46:0/2148366288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec0d0660 msgr2=0x7f7dec0b5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:18.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.947+0000 7f7df1400640 1 --2- 172.21.15.46:0/2148366288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec0d0660 0x7f7dec0b5820 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400efc0 tx=0x7f7dd400c490 comp rx=0 tx=0).stop 2024-03-29T08:27:18.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.947+0000 7f7df1400640 1 -- 172.21.15.46:0/2148366288 shutdown_connections 2024-03-29T08:27:18.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.947+0000 7f7df1400640 1 --2- 172.21.15.46:0/2148366288 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7dc80666c0 0x7f7dc8068b80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.947+0000 7f7df1400640 1 --2- 172.21.15.46:0/2148366288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7dec15dc00 0x7f7dec154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.947+0000 7f7df1400640 1 --2- 172.21.15.46:0/2148366288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7dec0d0660 0x7f7dec0b5820 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.947+0000 7f7df1400640 1 --2- 172.21.15.46:0/2148366288 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7dec06b8b0 0x7f7dec0b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:18.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.947+0000 7f7df1400640 1 -- 172.21.15.46:0/2148366288 >> 172.21.15.46:0/2148366288 conn(0x7f7dec089fd0 msgr2=0x7f7dec0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:18.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.947+0000 7f7df1400640 1 -- 172.21.15.46:0/2148366288 shutdown_connections 2024-03-29T08:27:18.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:18.947+0000 7f7df1400640 1 -- 172.21.15.46:0/2148366288 wait complete. 2024-03-29T08:27:18.969 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:27:19.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.107+0000 7f30ad200640 1 Processor -- start 2024-03-29T08:27:19.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.107+0000 7f30ad200640 1 -- start start 2024-03-29T08:27:19.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.111+0000 7f30ad200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f30a81593d0 0x7f30a806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:19.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.111+0000 7f30ad200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30a815d230 0x7f30a815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.111+0000 7f30ad200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a806d130 con 0x7f30a815dbd0 2024-03-29T08:27:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.111+0000 7f30ad200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a815b8f0 con 0x7f30a81593d0 2024-03-29T08:27:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.111+0000 7f30ad200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a815ad50 con 0x7f30a815d230 2024-03-29T08:27:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.111+0000 7f30a5e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f30a81593d0 0x7f30a806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.111+0000 7f30a6800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f30a815dbd0 0x7f30a815bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38820/0 (socket says 172.21.15.46:38820) 2024-03-29T08:27:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.111+0000 7f30a6800640 1 -- 172.21.15.46:0/1172994394 learned_addr learned my addr 172.21.15.46:0/1172994394 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.111+0000 7f309fe00640 1 --2- 172.21.15.46:0/1172994394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30a815d230 0x7f30a815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.111+0000 7f30a5e00640 1 -- 172.21.15.46:0/1172994394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30a815d230 msgr2=0x7f30a815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.111+0000 7f30a5e00640 1 --2- 172.21.15.46:0/1172994394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30a815d230 0x7f30a815d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.111+0000 7f30a5e00640 1 -- 172.21.15.46:0/1172994394 >> v1:172.21.15.46:6789/0 conn(0x7f30a815dbd0 legacy=0x7f30a815bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.111+0000 7f30a5e00640 1 -- 172.21.15.46:0/1172994394 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a815e060 con 0x7f30a81593d0 2024-03-29T08:27:19.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.111+0000 7f30a5e00640 1 --2- 172.21.15.46:0/1172994394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a81593d0 0x7f30a806cbf0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f30980097c0 tx=0x7f309801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6873a7be6046abf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:19.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30a5400640 1 -- 172.21.15.46:0/1172994394 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30980ab600 con 0x7f30a81593d0 2024-03-29T08:27:19.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30a5400640 1 -- 172.21.15.46:0/1172994394 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30980a7050 con 0x7f30a81593d0 2024-03-29T08:27:19.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30a5400640 1 -- 172.21.15.46:0/1172994394 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30980ab980 con 0x7f30a81593d0 2024-03-29T08:27:19.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30ad200640 1 -- 172.21.15.46:0/1172994394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a81593d0 msgr2=0x7f30a806cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:19.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30ad200640 1 --2- 172.21.15.46:0/1172994394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a81593d0 0x7f30a806cbf0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f30980097c0 tx=0x7f309801c330 comp rx=0 tx=0).stop 2024-03-29T08:27:19.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30ad200640 1 -- 172.21.15.46:0/1172994394 shutdown_connections 2024-03-29T08:27:19.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30ad200640 1 --2- 172.21.15.46:0/1172994394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30a815d230 0x7f30a815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:19.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30ad200640 1 --2- 172.21.15.46:0/1172994394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a81593d0 0x7f30a806cbf0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:19.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30ad200640 1 -- 172.21.15.46:0/1172994394 >> 172.21.15.46:0/1172994394 conn(0x7f30a8066110 msgr2=0x7f30a8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:19.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30ad200640 1 -- 172.21.15.46:0/1172994394 shutdown_connections 2024-03-29T08:27:19.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30ad200640 1 -- 172.21.15.46:0/1172994394 wait complete. 2024-03-29T08:27:19.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30ad200640 1 Processor -- start 2024-03-29T08:27:19.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30ad200640 1 -- start start 2024-03-29T08:27:19.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30ad200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30a81593d0 0x7f30a80ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:19.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30ad200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a815d230 0x7f30a8156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:19.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30ad200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a815dbd0 0x7f30a8156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:19.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30ad200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a8154520 con 0x7f30a815dbd0 2024-03-29T08:27:19.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30ad200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a8158530 con 0x7f30a815d230 2024-03-29T08:27:19.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30ad200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a8157490 con 0x7f30a81593d0 2024-03-29T08:27:19.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30a6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a815dbd0 0x7f30a8156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:19.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30a6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a815dbd0 0x7f30a8156dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33564/0 (socket says 172.21.15.46:33564) 2024-03-29T08:27:19.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30a6800640 1 -- 172.21.15.46:0/1074014787 learned_addr learned my addr 172.21.15.46:0/1074014787 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:19.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f309fe00640 1 --2- 172.21.15.46:0/1074014787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a815d230 0x7f30a8156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:19.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.115+0000 7f30a5e00640 1 --2- 172.21.15.46:0/1074014787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30a81593d0 0x7f30a80ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:19.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.119+0000 7f30a6800640 1 -- 172.21.15.46:0/1074014787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30a81593d0 msgr2=0x7f30a80ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:19.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.119+0000 7f30a6800640 1 --2- 172.21.15.46:0/1074014787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30a81593d0 0x7f30a80ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:19.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.119+0000 7f30a6800640 1 -- 172.21.15.46:0/1074014787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a815d230 msgr2=0x7f30a8156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:19.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.119+0000 7f30a6800640 1 --2- 172.21.15.46:0/1074014787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a815d230 0x7f30a8156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:19.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.119+0000 7f30a6800640 1 -- 172.21.15.46:0/1074014787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3098009420 con 0x7f30a815dbd0 2024-03-29T08:27:19.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.119+0000 7f30a6800640 1 --2- 172.21.15.46:0/1074014787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a815dbd0 0x7f30a8156dc0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f30a000e550 tx=0x7f30a000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:19.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.119+0000 7f309e000640 1 -- 172.21.15.46:0/1074014787 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0012810 con 0x7f30a815dbd0 2024-03-29T08:27:19.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.119+0000 7f309e000640 1 -- 172.21.15.46:0/1074014787 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a0012970 con 0x7f30a815dbd0 2024-03-29T08:27:19.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.119+0000 7f309e000640 1 -- 172.21.15.46:0/1074014787 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a0012d20 con 0x7f30a815dbd0 2024-03-29T08:27:19.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.119+0000 7f30ad200640 1 -- 172.21.15.46:0/1074014787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a8157770 con 0x7f30a815dbd0 2024-03-29T08:27:19.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.119+0000 7f30ad200640 1 -- 172.21.15.46:0/1074014787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a8157d30 con 0x7f30a815dbd0 2024-03-29T08:27:19.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.119+0000 7f30ad200640 1 -- 172.21.15.46:0/1074014787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a8066c90 con 0x7f30a815dbd0 2024-03-29T08:27:19.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.119+0000 7f309e000640 1 -- 172.21.15.46:0/1074014787 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f30a0009070 con 0x7f30a815dbd0 2024-03-29T08:27:19.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.119+0000 7f309e000640 1 --2- 172.21.15.46:0/1074014787 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3084066690 0x7f3084068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:19.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.119+0000 7f309e000640 1 -- 172.21.15.46:0/1074014787 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30a0086ff0 con 0x7f30a815dbd0 2024-03-29T08:27:19.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.123+0000 7f30a5e00640 1 --2- 172.21.15.46:0/1074014787 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3084066690 0x7f3084068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:19.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.123+0000 7f30a5e00640 1 --2- 172.21.15.46:0/1074014787 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3084066690 0x7f3084068b50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f30980ad000 tx=0x7f3098008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:19.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.127+0000 7f309e000640 1 -- 172.21.15.46:0/1074014787 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f30a00b8980 con 0x7f30a815dbd0 2024-03-29T08:27:19.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.287+0000 7f30ad200640 1 -- 172.21.15.46:0/1074014787 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30a8155610 con 0x7f30a815dbd0 2024-03-29T08:27:19.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.287+0000 7f309e000640 1 -- 172.21.15.46:0/1074014787 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16016 (secure 0 0 0) 0x7f30a0016070 con 0x7f30a815dbd0 2024-03-29T08:27:19.291 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:19.291 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":105,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:27:18.582894+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:27:17.118875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-29T08:27:17.573928+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:27:19.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.291+0000 7f30ad200640 1 -- 172.21.15.46:0/1074014787 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3084066690 msgr2=0x7f3084068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:19.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.291+0000 7f30ad200640 1 --2- 172.21.15.46:0/1074014787 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3084066690 0x7f3084068b50 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f30980ad000 tx=0x7f3098008040 comp rx=0 tx=0).stop 2024-03-29T08:27:19.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.291+0000 7f30ad200640 1 -- 172.21.15.46:0/1074014787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a815dbd0 msgr2=0x7f30a8156dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:19.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.291+0000 7f30ad200640 1 --2- 172.21.15.46:0/1074014787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a815dbd0 0x7f30a8156dc0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f30a000e550 tx=0x7f30a000ea10 comp rx=0 tx=0).stop 2024-03-29T08:27:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.291+0000 7f30ad200640 1 -- 172.21.15.46:0/1074014787 shutdown_connections 2024-03-29T08:27:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.291+0000 7f30ad200640 1 --2- 172.21.15.46:0/1074014787 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3084066690 0x7f3084068b50 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.291+0000 7f30ad200640 1 --2- 172.21.15.46:0/1074014787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a815d230 0x7f30a8156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.291+0000 7f30ad200640 1 --2- 172.21.15.46:0/1074014787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f30a815dbd0 0x7f30a8156dc0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.291+0000 7f30ad200640 1 --2- 172.21.15.46:0/1074014787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f30a81593d0 0x7f30a80ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.291+0000 7f30ad200640 1 -- 172.21.15.46:0/1074014787 >> 172.21.15.46:0/1074014787 conn(0x7f30a8066110 msgr2=0x7f30a80b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.291+0000 7f30ad200640 1 -- 172.21.15.46:0/1074014787 shutdown_connections 2024-03-29T08:27:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.291+0000 7f30ad200640 1 -- 172.21.15.46:0/1074014787 wait complete. 2024-03-29T08:27:19.313 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T08:27:19.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.455+0000 7fdd7a200640 1 Processor -- start 2024-03-29T08:27:19.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.455+0000 7fdd7a200640 1 -- start start 2024-03-29T08:27:19.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.455+0000 7fdd7a200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd7415df90 0x7fdd740ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:19.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.455+0000 7fdd7a200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd740cbea0 0x7fdd740ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:19.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.455+0000 7fdd7a200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7415d260 con 0x7fdd74142a50 2024-03-29T08:27:19.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.455+0000 7fdd7a200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd740aed00 con 0x7fdd740cbea0 2024-03-29T08:27:19.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.455+0000 7fdd7a200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740aefd0 con 0x7fdd7415df90 2024-03-29T08:27:19.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.455+0000 7fdd73e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd740cbea0 0x7fdd740ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:19.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.455+0000 7fdd73e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd740cbea0 0x7fdd740ae7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33574/0 (socket says 172.21.15.46:33574) 2024-03-29T08:27:19.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.455+0000 7fdd73e00640 1 -- 172.21.15.46:0/286552378 learned_addr learned my addr 172.21.15.46:0/286552378 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:19.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.455+0000 7fdd6aa00640 1 --2- 172.21.15.46:0/286552378 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd7415df90 0x7fdd740ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:19.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.455+0000 7fdd73e00640 1 -- 172.21.15.46:0/286552378 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd7415df90 msgr2=0x7fdd740ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:19.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.455+0000 7fdd73e00640 1 --2- 172.21.15.46:0/286552378 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd7415df90 0x7fdd740ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:19.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.455+0000 7fdd73e00640 1 -- 172.21.15.46:0/286552378 >> v1:172.21.15.73:6789/0 conn(0x7fdd74142a50 legacy=0x7fdd740cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:19.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.455+0000 7fdd73e00640 1 -- 172.21.15.46:0/286552378 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7415d4b0 con 0x7fdd740cbea0 2024-03-29T08:27:19.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.455+0000 7fdd6aa00640 1 --2- 172.21.15.46:0/286552378 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd7415df90 0x7fdd740ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:19.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.459+0000 7fdd73e00640 1 --2- 172.21.15.46:0/286552378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740cbea0 0x7fdd740ae7c0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009e50 tx=0x7fdd6c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf707bdd464475 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:19.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.459+0000 7fdd72a00640 1 -- 172.21.15.46:0/286552378 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c0a5640 con 0x7fdd740cbea0 2024-03-29T08:27:19.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.459+0000 7fdd72a00640 1 -- 172.21.15.46:0/286552378 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6c0a57a0 con 0x7fdd740cbea0 2024-03-29T08:27:19.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.459+0000 7fdd72a00640 1 -- 172.21.15.46:0/286552378 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c0a5b30 con 0x7fdd740cbea0 2024-03-29T08:27:19.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.459+0000 7fdd7a200640 1 -- 172.21.15.46:0/286552378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740cbea0 msgr2=0x7fdd740ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:19.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.459+0000 7fdd7a200640 1 --2- 172.21.15.46:0/286552378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740cbea0 0x7fdd740ae7c0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009e50 tx=0x7fdd6c01c9a0 comp rx=0 tx=0).stop 2024-03-29T08:27:19.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.459+0000 7fdd7a200640 1 -- 172.21.15.46:0/286552378 shutdown_connections 2024-03-29T08:27:19.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.459+0000 7fdd7a200640 1 --2- 172.21.15.46:0/286552378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740cbea0 0x7fdd740ae7c0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:19.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.459+0000 7fdd7a200640 1 --2- 172.21.15.46:0/286552378 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd7415df90 0x7fdd740ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:19.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.459+0000 7fdd7a200640 1 -- 172.21.15.46:0/286552378 >> 172.21.15.46:0/286552378 conn(0x7fdd7406a390 msgr2=0x7fdd740cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:19.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.459+0000 7fdd7a200640 1 -- 172.21.15.46:0/286552378 shutdown_connections 2024-03-29T08:27:19.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.459+0000 7fdd7a200640 1 -- 172.21.15.46:0/286552378 wait complete. 2024-03-29T08:27:19.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd7a200640 1 Processor -- start 2024-03-29T08:27:19.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd7a200640 1 -- start start 2024-03-29T08:27:19.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd7a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740cbea0 0x7fdd7413c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:19.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd7a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd74142a50 0x7fdd74140310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:19.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd7a200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd7415df90 0x7fdd741f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:19.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd7a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740a8cc0 con 0x7fdd740cbea0 2024-03-29T08:27:19.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd7a200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7413ef90 con 0x7fdd74142a50 2024-03-29T08:27:19.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd7a200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7413f270 con 0x7fdd7415df90 2024-03-29T08:27:19.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd73400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740cbea0 0x7fdd7413c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:19.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd73400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740cbea0 0x7fdd7413c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33584/0 (socket says 172.21.15.46:33584) 2024-03-29T08:27:19.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd73400640 1 -- 172.21.15.46:0/319626376 learned_addr learned my addr 172.21.15.46:0/319626376 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:19.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd6aa00640 1 --2- 172.21.15.46:0/319626376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd74142a50 0x7fdd74140310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:19.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd73e00640 1 --2- 172.21.15.46:0/319626376 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd7415df90 0x7fdd741f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:19.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd73400640 1 -- 172.21.15.46:0/319626376 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd7415df90 msgr2=0x7fdd741f54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:19.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd73400640 1 --2- 172.21.15.46:0/319626376 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd7415df90 0x7fdd741f54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:19.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd73400640 1 -- 172.21.15.46:0/319626376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd74142a50 msgr2=0x7fdd74140310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:19.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd73400640 1 --2- 172.21.15.46:0/319626376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd74142a50 0x7fdd74140310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:19.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd73400640 1 -- 172.21.15.46:0/319626376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c009b30 con 0x7fdd740cbea0 2024-03-29T08:27:19.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd6aa00640 1 --2- 172.21.15.46:0/319626376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd74142a50 0x7fdd74140310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:19.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd73400640 1 --2- 172.21.15.46:0/319626376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740cbea0 0x7fdd7413c250 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c500 tx=0x7fdd6400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:19.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd6be00640 1 -- 172.21.15.46:0/319626376 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64005220 con 0x7fdd740cbea0 2024-03-29T08:27:19.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd6be00640 1 -- 172.21.15.46:0/319626376 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd64005380 con 0x7fdd740cbea0 2024-03-29T08:27:19.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd7a200640 1 -- 172.21.15.46:0/319626376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7413f550 con 0x7fdd740cbea0 2024-03-29T08:27:19.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd6be00640 1 -- 172.21.15.46:0/319626376 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64007270 con 0x7fdd740cbea0 2024-03-29T08:27:19.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd7a200640 1 -- 172.21.15.46:0/319626376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7413fb10 con 0x7fdd740cbea0 2024-03-29T08:27:19.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.463+0000 7fdd7a200640 1 -- 172.21.15.46:0/319626376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd74071a70 con 0x7fdd740cbea0 2024-03-29T08:27:19.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.467+0000 7fdd6be00640 1 -- 172.21.15.46:0/319626376 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdd640073d0 con 0x7fdd740cbea0 2024-03-29T08:27:19.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.467+0000 7fdd6be00640 1 --2- 172.21.15.46:0/319626376 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd50066690 0x7fdd50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:19.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.467+0000 7fdd6be00640 1 -- 172.21.15.46:0/319626376 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd64083b20 con 0x7fdd740cbea0 2024-03-29T08:27:19.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.467+0000 7fdd6aa00640 1 --2- 172.21.15.46:0/319626376 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd50066690 0x7fdd50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:19.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.467+0000 7fdd6aa00640 1 --2- 172.21.15.46:0/319626376 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd50066690 0x7fdd50068b50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdd740a7c40 tx=0x7fdd5c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:19.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.471+0000 7fdd6be00640 1 -- 172.21.15.46:0/319626376 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdd64050e00 con 0x7fdd740cbea0 2024-03-29T08:27:19.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:19.671+0000 7fdd7a200640 1 -- 172.21.15.46:0/319626376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdd7404e120 con 0x7fdd740cbea0 2024-03-29T08:27:20.105 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-03-29T08:27:20.103+0000 7fe23a400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:27:20.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.591+0000 7fdd6be00640 1 -- 172.21.15.46:0/319626376 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v76) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdd64054420 con 0x7fdd740cbea0 2024-03-29T08:27:20.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.591+0000 7fdd7a200640 1 -- 172.21.15.46:0/319626376 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd50066690 msgr2=0x7fdd50068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:20.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.591+0000 7fdd7a200640 1 --2- 172.21.15.46:0/319626376 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd50066690 0x7fdd50068b50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fdd740a7c40 tx=0x7fdd5c008040 comp rx=0 tx=0).stop 2024-03-29T08:27:20.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.591+0000 7fdd7a200640 1 -- 172.21.15.46:0/319626376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740cbea0 msgr2=0x7fdd7413c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:20.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.591+0000 7fdd7a200640 1 --2- 172.21.15.46:0/319626376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740cbea0 0x7fdd7413c250 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c500 tx=0x7fdd6400c9c0 comp rx=0 tx=0).stop 2024-03-29T08:27:20.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.591+0000 7fdd7a200640 1 -- 172.21.15.46:0/319626376 shutdown_connections 2024-03-29T08:27:20.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.591+0000 7fdd7a200640 1 --2- 172.21.15.46:0/319626376 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd50066690 0x7fdd50068b50 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:20.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.591+0000 7fdd7a200640 1 --2- 172.21.15.46:0/319626376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd74142a50 0x7fdd74140310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:20.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.591+0000 7fdd7a200640 1 --2- 172.21.15.46:0/319626376 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd7415df90 0x7fdd741f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:20.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.591+0000 7fdd7a200640 1 --2- 172.21.15.46:0/319626376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd740cbea0 0x7fdd7413c250 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:20.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.591+0000 7fdd7a200640 1 -- 172.21.15.46:0/319626376 >> 172.21.15.46:0/319626376 conn(0x7fdd7406a390 msgr2=0x7fdd740cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:20.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.591+0000 7fdd7a200640 1 -- 172.21.15.46:0/319626376 shutdown_connections 2024-03-29T08:27:20.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.591+0000 7fdd7a200640 1 -- 172.21.15.46:0/319626376 wait complete. 2024-03-29T08:27:20.618 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-29T08:27:20.618 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T08:27:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.763+0000 7fa658e00640 1 Processor -- start 2024-03-29T08:27:20.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.763+0000 7fa658e00640 1 -- start start 2024-03-29T08:27:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.763+0000 7fa658e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa654142a50 0x7fa6540cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.763+0000 7fa658e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa65415df90 0x7fa6540cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.763+0000 7fa658e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65415d260 con 0x7fa6540ae280 2024-03-29T08:27:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.763+0000 7fa658e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6540aefd0 con 0x7fa654142a50 2024-03-29T08:27:20.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.763+0000 7fa658e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65415d4b0 con 0x7fa65415df90 2024-03-29T08:27:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.763+0000 7fa651e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa654142a50 0x7fa6540cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.763+0000 7fa652800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa6540ae280 0x7fa6540ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38836/0 (socket says 172.21.15.46:38836) 2024-03-29T08:27:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.763+0000 7fa652800640 1 -- 172.21.15.46:0/436695911 learned_addr learned my addr 172.21.15.46:0/436695911 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.767+0000 7fa64be00640 1 --2- 172.21.15.46:0/436695911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa65415df90 0x7fa6540cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.767+0000 7fa651e00640 1 -- 172.21.15.46:0/436695911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa65415df90 msgr2=0x7fa6540cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.767+0000 7fa651e00640 1 --2- 172.21.15.46:0/436695911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa65415df90 0x7fa6540cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:20.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.767+0000 7fa651400640 1 -- 172.21.15.46:0/436695911 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2550083536 0 0) 0x7fa65415d260 con 0x7fa6540ae280 2024-03-29T08:27:20.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.767+0000 7fa651e00640 1 -- 172.21.15.46:0/436695911 >> v1:172.21.15.46:6789/0 conn(0x7fa6540ae280 legacy=0x7fa6540ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:20.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.767+0000 7fa651e00640 1 -- 172.21.15.46:0/436695911 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65415d790 con 0x7fa654142a50 2024-03-29T08:27:20.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.767+0000 7fa64be00640 1 --2- 172.21.15.46:0/436695911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa65415df90 0x7fa6540cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:20.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.767+0000 7fa651e00640 1 --2- 172.21.15.46:0/436695911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa654142a50 0x7fa6540cb790 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa6440097c0 tx=0x7fa64401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1be7ded1bae238 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.767+0000 7fa651400640 1 -- 172.21.15.46:0/436695911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6440a4ea0 con 0x7fa654142a50 2024-03-29T08:27:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.767+0000 7fa651400640 1 -- 172.21.15.46:0/436695911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6440a7050 con 0x7fa654142a50 2024-03-29T08:27:20.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.767+0000 7fa651400640 1 -- 172.21.15.46:0/436695911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6440ab820 con 0x7fa654142a50 2024-03-29T08:27:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa658e00640 1 -- 172.21.15.46:0/436695911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa654142a50 msgr2=0x7fa6540cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa658e00640 1 --2- 172.21.15.46:0/436695911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa654142a50 0x7fa6540cb790 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa6440097c0 tx=0x7fa64401c410 comp rx=0 tx=0).stop 2024-03-29T08:27:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa658e00640 1 -- 172.21.15.46:0/436695911 shutdown_connections 2024-03-29T08:27:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa658e00640 1 --2- 172.21.15.46:0/436695911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa65415df90 0x7fa6540cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa658e00640 1 --2- 172.21.15.46:0/436695911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa654142a50 0x7fa6540cb790 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa658e00640 1 -- 172.21.15.46:0/436695911 >> 172.21.15.46:0/436695911 conn(0x7fa65406a390 msgr2=0x7fa6540cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa658e00640 1 -- 172.21.15.46:0/436695911 shutdown_connections 2024-03-29T08:27:20.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa658e00640 1 -- 172.21.15.46:0/436695911 wait complete. 2024-03-29T08:27:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa658e00640 1 Processor -- start 2024-03-29T08:27:20.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa658e00640 1 -- start start 2024-03-29T08:27:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa658e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6540ae280 0x7fa65413bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa658e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa654142a50 0x7fa6541612c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa658e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa65415df90 0x7fa6541f3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa658e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6540ca8d0 con 0x7fa6540ae280 2024-03-29T08:27:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa658e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa654161800 con 0x7fa654142a50 2024-03-29T08:27:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa658e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa654161a10 con 0x7fa65415df90 2024-03-29T08:27:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa651e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6540ae280 0x7fa65413bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:20.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa651e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6540ae280 0x7fa65413bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33602/0 (socket says 172.21.15.46:33602) 2024-03-29T08:27:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa651e00640 1 -- 172.21.15.46:0/4196041423 learned_addr learned my addr 172.21.15.46:0/4196041423 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa652800640 1 --2- 172.21.15.46:0/4196041423 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa65415df90 0x7fa6541f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa64be00640 1 --2- 172.21.15.46:0/4196041423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa654142a50 0x7fa6541612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa651e00640 1 -- 172.21.15.46:0/4196041423 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa65415df90 msgr2=0x7fa6541f3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa651e00640 1 --2- 172.21.15.46:0/4196041423 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa65415df90 0x7fa6541f3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa651e00640 1 -- 172.21.15.46:0/4196041423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa654142a50 msgr2=0x7fa6541612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa651e00640 1 --2- 172.21.15.46:0/4196041423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa654142a50 0x7fa6541612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa651e00640 1 -- 172.21.15.46:0/4196041423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa644009420 con 0x7fa6540ae280 2024-03-29T08:27:20.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.771+0000 7fa651e00640 1 --2- 172.21.15.46:0/4196041423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6540ae280 0x7fa65413bfa0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa6440ae040 tx=0x7fa64401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:20.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.775+0000 7fa64a000640 1 -- 172.21.15.46:0/4196041423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6440b7860 con 0x7fa6540ae280 2024-03-29T08:27:20.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.775+0000 7fa64a000640 1 -- 172.21.15.46:0/4196041423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6440abe90 con 0x7fa6540ae280 2024-03-29T08:27:20.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.775+0000 7fa64a000640 1 -- 172.21.15.46:0/4196041423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6440b7c20 con 0x7fa6540ae280 2024-03-29T08:27:20.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.775+0000 7fa658e00640 1 -- 172.21.15.46:0/4196041423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa654161cf0 con 0x7fa6540ae280 2024-03-29T08:27:20.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.775+0000 7fa658e00640 1 -- 172.21.15.46:0/4196041423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65415f980 con 0x7fa6540ae280 2024-03-29T08:27:20.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.775+0000 7fa658e00640 1 -- 172.21.15.46:0/4196041423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65415fb70 con 0x7fa6540ae280 2024-03-29T08:27:20.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.775+0000 7fa64a000640 1 -- 172.21.15.46:0/4196041423 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa6440b5400 con 0x7fa6540ae280 2024-03-29T08:27:20.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.775+0000 7fa64a000640 1 --2- 172.21.15.46:0/4196041423 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa630066690 0x7fa630068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:20.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.775+0000 7fa64a000640 1 -- 172.21.15.46:0/4196041423 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa64411ec70 con 0x7fa6540ae280 2024-03-29T08:27:20.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.775+0000 7fa64be00640 1 --2- 172.21.15.46:0/4196041423 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa630066690 0x7fa630068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:20.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.779+0000 7fa64be00640 1 --2- 172.21.15.46:0/4196041423 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa630066690 0x7fa630068b50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa63c006fd0 tx=0x7fa63c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:20.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.779+0000 7fa64a000640 1 -- 172.21.15.46:0/4196041423 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa6440ebfd0 con 0x7fa6540ae280 2024-03-29T08:27:20.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:20.971+0000 7fa658e00640 1 -- 172.21.15.46:0/4196041423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa654071a70 con 0x7fa6540ae280 2024-03-29T08:27:21.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.595+0000 7fa64a000640 1 -- 172.21.15.46:0/4196041423 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v77) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa644006e20 con 0x7fa6540ae280 2024-03-29T08:27:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.599+0000 7fa658e00640 1 -- 172.21.15.46:0/4196041423 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa630066690 msgr2=0x7fa630068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.599+0000 7fa658e00640 1 --2- 172.21.15.46:0/4196041423 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa630066690 0x7fa630068b50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa63c006fd0 tx=0x7fa63c008040 comp rx=0 tx=0).stop 2024-03-29T08:27:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.599+0000 7fa658e00640 1 -- 172.21.15.46:0/4196041423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6540ae280 msgr2=0x7fa65413bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:21.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.599+0000 7fa658e00640 1 --2- 172.21.15.46:0/4196041423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6540ae280 0x7fa65413bfa0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa6440ae040 tx=0x7fa64401ac50 comp rx=0 tx=0).stop 2024-03-29T08:27:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.599+0000 7fa658e00640 1 -- 172.21.15.46:0/4196041423 shutdown_connections 2024-03-29T08:27:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.599+0000 7fa658e00640 1 --2- 172.21.15.46:0/4196041423 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa630066690 0x7fa630068b50 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.599+0000 7fa658e00640 1 --2- 172.21.15.46:0/4196041423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa654142a50 0x7fa6541612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.599+0000 7fa658e00640 1 --2- 172.21.15.46:0/4196041423 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa65415df90 0x7fa6541f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.599+0000 7fa658e00640 1 --2- 172.21.15.46:0/4196041423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6540ae280 0x7fa65413bfa0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.599+0000 7fa658e00640 1 -- 172.21.15.46:0/4196041423 >> 172.21.15.46:0/4196041423 conn(0x7fa65406a390 msgr2=0x7fa65413c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.599+0000 7fa658e00640 1 -- 172.21.15.46:0/4196041423 shutdown_connections 2024-03-29T08:27:21.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.599+0000 7fa658e00640 1 -- 172.21.15.46:0/4196041423 wait complete. 2024-03-29T08:27:21.621 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T08:27:21.622 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:27:21.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.763+0000 7f2147200640 1 Processor -- start 2024-03-29T08:27:21.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.763+0000 7f2147200640 1 -- start start 2024-03-29T08:27:21.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.763+0000 7f2147200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f214015ada0 0x7f21400ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.763+0000 7f2147200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21400cb510 con 0x7f21400ce950 2024-03-29T08:27:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.763+0000 7f2147200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2140070580 con 0x7f2140144210 2024-03-29T08:27:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.763+0000 7f2147200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21400cede0 con 0x7f214015ada0 2024-03-29T08:27:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.763+0000 7f2137400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f214015ada0 0x7f21400ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.763+0000 7f2145200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f21400ce950 0x7f214006c950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38846/0 (socket says 172.21.15.46:38846) 2024-03-29T08:27:21.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.763+0000 7f2145200640 1 -- 172.21.15.46:0/2208529768 learned_addr learned my addr 172.21.15.46:0/2208529768 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:21.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.763+0000 7f2137400640 1 -- 172.21.15.46:0/2208529768 >> v1:172.21.15.73:6789/0 conn(0x7f2140144210 legacy=0x7f21400cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:21.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.763+0000 7f2137400640 1 -- 172.21.15.46:0/2208529768 >> v1:172.21.15.46:6789/0 conn(0x7f21400ce950 legacy=0x7f214006c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:21.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.763+0000 7f2137400640 1 -- 172.21.15.46:0/2208529768 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2140070860 con 0x7f214015ada0 2024-03-29T08:27:21.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.763+0000 7f213f400640 1 -- 172.21.15.46:0/2208529768 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384882151 0 0) 0x7f21400cb510 con 0x7f21400ce950 2024-03-29T08:27:21.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.767+0000 7f2137400640 1 --2- 172.21.15.46:0/2208529768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f214015ada0 0x7f21400ce410 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f21280097c0 tx=0x7f212801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d26e7efd625987 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:21.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.767+0000 7f213f400640 1 -- 172.21.15.46:0/2208529768 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21280a8050 con 0x7f214015ada0 2024-03-29T08:27:21.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.767+0000 7f213f400640 1 -- 172.21.15.46:0/2208529768 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21280ac5f0 con 0x7f214015ada0 2024-03-29T08:27:21.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.767+0000 7f213f400640 1 -- 172.21.15.46:0/2208529768 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21280ac980 con 0x7f214015ada0 2024-03-29T08:27:21.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.767+0000 7f2147200640 1 -- 172.21.15.46:0/2208529768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f214015ada0 msgr2=0x7f21400ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:21.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.767+0000 7f2147200640 1 --2- 172.21.15.46:0/2208529768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f214015ada0 0x7f21400ce410 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f21280097c0 tx=0x7f212801c4a0 comp rx=0 tx=0).stop 2024-03-29T08:27:21.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.767+0000 7f2147200640 1 -- 172.21.15.46:0/2208529768 shutdown_connections 2024-03-29T08:27:21.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.767+0000 7f2147200640 1 --2- 172.21.15.46:0/2208529768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f214015ada0 0x7f21400ce410 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:21.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.767+0000 7f2147200640 1 -- 172.21.15.46:0/2208529768 >> 172.21.15.46:0/2208529768 conn(0x7f2140066110 msgr2=0x7f2140069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:21.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.767+0000 7f2147200640 1 -- 172.21.15.46:0/2208529768 shutdown_connections 2024-03-29T08:27:21.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.767+0000 7f2147200640 1 -- 172.21.15.46:0/2208529768 wait complete. 2024-03-29T08:27:21.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f2147200640 1 Processor -- start 2024-03-29T08:27:21.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f2147200640 1 -- start start 2024-03-29T08:27:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f2147200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21400ce950 0x7f2140137dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f2147200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2140144210 0x7f2140151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f2147200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f214015ada0 0x7f21400b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f2147200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21401421b0 con 0x7f21400ce950 2024-03-29T08:27:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f2147200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21400ac570 con 0x7f2140144210 2024-03-29T08:27:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f2147200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21400ac850 con 0x7f214015ada0 2024-03-29T08:27:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f213fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21400ce950 0x7f2140137dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f213fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21400ce950 0x7f2140137dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33626/0 (socket says 172.21.15.46:33626) 2024-03-29T08:27:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f213fe00640 1 -- 172.21.15.46:0/2895854801 learned_addr learned my addr 172.21.15.46:0/2895854801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:21.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f2137400640 1 --2- 172.21.15.46:0/2895854801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2140144210 0x7f2140151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f213fe00640 1 -- 172.21.15.46:0/2895854801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f214015ada0 msgr2=0x7f21400b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f2145200640 1 --2- 172.21.15.46:0/2895854801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f214015ada0 0x7f21400b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f213fe00640 1 --2- 172.21.15.46:0/2895854801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f214015ada0 0x7f21400b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f213fe00640 1 -- 172.21.15.46:0/2895854801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2140144210 msgr2=0x7f2140151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f213fe00640 1 --2- 172.21.15.46:0/2895854801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2140144210 0x7f2140151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f213fe00640 1 -- 172.21.15.46:0/2895854801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2128009420 con 0x7f21400ce950 2024-03-29T08:27:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f2137400640 1 --2- 172.21.15.46:0/2895854801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2140144210 0x7f2140151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f2145200640 1 --2- 172.21.15.46:0/2895854801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f214015ada0 0x7f21400b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f213fe00640 1 --2- 172.21.15.46:0/2895854801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21400ce950 0x7f2140137dc0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f213000c500 tx=0x7f213000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f213cc00640 1 -- 172.21.15.46:0/2895854801 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130005220 con 0x7f21400ce950 2024-03-29T08:27:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f213cc00640 1 -- 172.21.15.46:0/2895854801 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2130005380 con 0x7f21400ce950 2024-03-29T08:27:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f213cc00640 1 -- 172.21.15.46:0/2895854801 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130007270 con 0x7f21400ce950 2024-03-29T08:27:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f2147200640 1 -- 172.21.15.46:0/2895854801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21400acb30 con 0x7f21400ce950 2024-03-29T08:27:21.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f2147200640 1 -- 172.21.15.46:0/2895854801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2140138300 con 0x7f21400ce950 2024-03-29T08:27:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.771+0000 7f2147200640 1 -- 172.21.15.46:0/2895854801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21400ce280 con 0x7f21400ce950 2024-03-29T08:27:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.775+0000 7f213cc00640 1 -- 172.21.15.46:0/2895854801 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f21300073d0 con 0x7f21400ce950 2024-03-29T08:27:21.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.775+0000 7f213cc00640 1 --2- 172.21.15.46:0/2895854801 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f211c066690 0x7f211c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.775+0000 7f213cc00640 1 -- 172.21.15.46:0/2895854801 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2130083a80 con 0x7f21400ce950 2024-03-29T08:27:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.775+0000 7f2137400640 1 --2- 172.21.15.46:0/2895854801 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f211c066690 0x7f211c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:21.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.775+0000 7f2137400640 1 --2- 172.21.15.46:0/2895854801 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f211c066690 0x7f211c068b50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f21280ae000 tx=0x7f2128008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:21.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.779+0000 7f213cc00640 1 -- 172.21.15.46:0/2895854801 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2130050e00 con 0x7f21400ce950 2024-03-29T08:27:21.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.971+0000 7f2147200640 1 -- 172.21.15.46:0/2895854801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2140066c90 con 0x7f21400ce950 2024-03-29T08:27:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.971+0000 7f213cc00640 1 -- 172.21.15.46:0/2895854801 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4730 (secure 0 0 0) 0x7f2130054420 con 0x7f21400ce950 2024-03-29T08:27:21.974 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:21.974 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4230030957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4230030957},{"type":"v1","addr":"172.21.15.73:6836","nonce":4230030957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/297138217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":297138217},{"type":"v1","addr":"172.21.15.73:6837","nonce":297138217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3856305332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3856305332},{"type":"v1","addr":"172.21.15.46:6833","nonce":3856305332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:27:18.583223+0000","modified":"2024-03-29T08:27:21.592418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7966},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7966":{"gid":7966,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/824184385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":824184385},{"type":"v1","addr":"172.21.15.46:6835","nonce":824184385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-29T08:27:21.974 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 77 2024-03-29T08:27:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.975+0000 7f2147200640 1 -- 172.21.15.46:0/2895854801 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f211c066690 msgr2=0x7f211c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.975+0000 7f2147200640 1 --2- 172.21.15.46:0/2895854801 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f211c066690 0x7f211c068b50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f21280ae000 tx=0x7f2128008040 comp rx=0 tx=0).stop 2024-03-29T08:27:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.975+0000 7f2147200640 1 -- 172.21.15.46:0/2895854801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21400ce950 msgr2=0x7f2140137dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.975+0000 7f2147200640 1 --2- 172.21.15.46:0/2895854801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21400ce950 0x7f2140137dc0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f213000c500 tx=0x7f213000c9c0 comp rx=0 tx=0).stop 2024-03-29T08:27:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.975+0000 7f2147200640 1 -- 172.21.15.46:0/2895854801 shutdown_connections 2024-03-29T08:27:21.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.975+0000 7f2147200640 1 --2- 172.21.15.46:0/2895854801 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f211c066690 0x7f211c068b50 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.975+0000 7f2147200640 1 --2- 172.21.15.46:0/2895854801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2140144210 0x7f2140151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.975+0000 7f2147200640 1 --2- 172.21.15.46:0/2895854801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f214015ada0 0x7f21400b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.975+0000 7f2147200640 1 --2- 172.21.15.46:0/2895854801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21400ce950 0x7f2140137dc0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.975+0000 7f2147200640 1 -- 172.21.15.46:0/2895854801 >> 172.21.15.46:0/2895854801 conn(0x7f2140066110 msgr2=0x7f21400af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.975+0000 7f2147200640 1 -- 172.21.15.46:0/2895854801 shutdown_connections 2024-03-29T08:27:21.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:21.975+0000 7f2147200640 1 -- 172.21.15.46:0/2895854801 wait complete. 2024-03-29T08:27:21.994 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:27:22.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.131+0000 7f68e9c00640 1 Processor -- start 2024-03-29T08:27:22.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.131+0000 7f68e9c00640 1 -- start start 2024-03-29T08:27:22.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.131+0000 7f68e9c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68e41593d0 0x7f68e406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.135+0000 7f68e9c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e415cd90 0x7f68e41597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.135+0000 7f68e9c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e406d130 con 0x7f68e415d230 2024-03-29T08:27:22.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.135+0000 7f68e9c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68e415b8f0 con 0x7f68e41593d0 2024-03-29T08:27:22.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.135+0000 7f68e9c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e415ad50 con 0x7f68e415cd90 2024-03-29T08:27:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.135+0000 7f68dbe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f68e415d230 0x7f68e415d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38850/0 (socket says 172.21.15.46:38850) 2024-03-29T08:27:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.135+0000 7f68e2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f68e41593d0 0x7f68e406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.135+0000 7f68dbe00640 1 -- 172.21.15.46:0/3723320787 learned_addr learned my addr 172.21.15.46:0/3723320787 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.135+0000 7f68e3400640 1 --2- 172.21.15.46:0/3723320787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e415cd90 0x7f68e41597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.135+0000 7f68e2a00640 1 -- 172.21.15.46:0/3723320787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e415cd90 msgr2=0x7f68e41597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.135+0000 7f68e2a00640 1 --2- 172.21.15.46:0/3723320787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e415cd90 0x7f68e41597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.135+0000 7f68e2a00640 1 -- 172.21.15.46:0/3723320787 >> v1:172.21.15.46:6789/0 conn(0x7f68e415d230 legacy=0x7f68e415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.135+0000 7f68e2a00640 1 -- 172.21.15.46:0/3723320787 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e415afc0 con 0x7f68e41593d0 2024-03-29T08:27:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.135+0000 7f68e2000640 1 -- 172.21.15.46:0/3723320787 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 592898372 0 0) 0x7f68e406d130 con 0x7f68e415d230 2024-03-29T08:27:22.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.135+0000 7f68e2a00640 1 --2- 172.21.15.46:0/3723320787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e41593d0 0x7f68e406cbf0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f68d40097c0 tx=0x7f68d401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94de784ee00c6b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:22.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.135+0000 7f68e2000640 1 -- 172.21.15.46:0/3723320787 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d40a6040 con 0x7f68e41593d0 2024-03-29T08:27:22.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.135+0000 7f68e2000640 1 -- 172.21.15.46:0/3723320787 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d40aa600 con 0x7f68e41593d0 2024-03-29T08:27:22.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.135+0000 7f68e2000640 1 -- 172.21.15.46:0/3723320787 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d40aa990 con 0x7f68e41593d0 2024-03-29T08:27:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e9c00640 1 -- 172.21.15.46:0/3723320787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e41593d0 msgr2=0x7f68e406cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e9c00640 1 --2- 172.21.15.46:0/3723320787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e41593d0 0x7f68e406cbf0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f68d40097c0 tx=0x7f68d401c4a0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e9c00640 1 -- 172.21.15.46:0/3723320787 shutdown_connections 2024-03-29T08:27:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e9c00640 1 --2- 172.21.15.46:0/3723320787 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e415cd90 0x7f68e41597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e9c00640 1 --2- 172.21.15.46:0/3723320787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e41593d0 0x7f68e406cbf0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e9c00640 1 -- 172.21.15.46:0/3723320787 >> 172.21.15.46:0/3723320787 conn(0x7f68e4066110 msgr2=0x7f68e4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:22.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e9c00640 1 -- 172.21.15.46:0/3723320787 shutdown_connections 2024-03-29T08:27:22.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e9c00640 1 -- 172.21.15.46:0/3723320787 wait complete. 2024-03-29T08:27:22.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e9c00640 1 Processor -- start 2024-03-29T08:27:22.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e9c00640 1 -- start start 2024-03-29T08:27:22.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e9c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e41593d0 0x7f68e41471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e9c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68e415cd90 0x7f68e41501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e415d230 0x7f68e4145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e9c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e4150b10 con 0x7f68e415d230 2024-03-29T08:27:22.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e9c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e4146da0 con 0x7f68e415cd90 2024-03-29T08:27:22.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e9c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e4146400 con 0x7f68e41593d0 2024-03-29T08:27:22.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e415d230 0x7f68e4145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e415d230 0x7f68e4145860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33650/0 (socket says 172.21.15.46:33650) 2024-03-29T08:27:22.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e3400640 1 -- 172.21.15.46:0/2061419132 learned_addr learned my addr 172.21.15.46:0/2061419132 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:22.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e2a00640 1 --2- 172.21.15.46:0/2061419132 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e41593d0 0x7f68e41471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68dbe00640 1 --2- 172.21.15.46:0/2061419132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68e415cd90 0x7f68e41501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e3400640 1 -- 172.21.15.46:0/2061419132 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e41593d0 msgr2=0x7f68e41471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:22.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e3400640 1 --2- 172.21.15.46:0/2061419132 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e41593d0 0x7f68e41471c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e3400640 1 -- 172.21.15.46:0/2061419132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68e415cd90 msgr2=0x7f68e41501a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:22.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e3400640 1 --2- 172.21.15.46:0/2061419132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68e415cd90 0x7f68e41501a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68e3400640 1 -- 172.21.15.46:0/2061419132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d4009420 con 0x7f68e415d230 2024-03-29T08:27:22.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.139+0000 7f68dbe00640 1 --2- 172.21.15.46:0/2061419132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68e415cd90 0x7f68e41501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.143+0000 7f68e3400640 1 --2- 172.21.15.46:0/2061419132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e415d230 0x7f68e4145860 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009fd0 tx=0x7f68dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.143+0000 7f68daa00640 1 -- 172.21.15.46:0/2061419132 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc014420 con 0x7f68e415d230 2024-03-29T08:27:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.143+0000 7f68daa00640 1 -- 172.21.15.46:0/2061419132 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68dc014580 con 0x7f68e415d230 2024-03-29T08:27:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.143+0000 7f68daa00640 1 -- 172.21.15.46:0/2061419132 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc014930 con 0x7f68e415d230 2024-03-29T08:27:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.143+0000 7f68e9c00640 1 -- 172.21.15.46:0/2061419132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e41466e0 con 0x7f68e415d230 2024-03-29T08:27:22.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.143+0000 7f68e9c00640 1 -- 172.21.15.46:0/2061419132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e41506e0 con 0x7f68e415d230 2024-03-29T08:27:22.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.143+0000 7f68e9c00640 1 -- 172.21.15.46:0/2061419132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e4146ee0 con 0x7f68e415d230 2024-03-29T08:27:22.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.143+0000 7f68daa00640 1 -- 172.21.15.46:0/2061419132 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f68dc014a90 con 0x7f68e415d230 2024-03-29T08:27:22.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.143+0000 7f68daa00640 1 --2- 172.21.15.46:0/2061419132 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68c0066690 0x7f68c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.143+0000 7f68daa00640 1 -- 172.21.15.46:0/2061419132 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68dc084ab0 con 0x7f68e415d230 2024-03-29T08:27:22.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.143+0000 7f68e2a00640 1 --2- 172.21.15.46:0/2061419132 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68c0066690 0x7f68c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.147+0000 7f68e2a00640 1 --2- 172.21.15.46:0/2061419132 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68c0066690 0x7f68c0068b50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f68d40ae000 tx=0x7f68d40b5040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:22.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.147+0000 7f68daa00640 1 -- 172.21.15.46:0/2061419132 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f68e4146ee0 con 0x7f68e415d230 2024-03-29T08:27:22.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.311+0000 7f68e9c00640 1 -- 172.21.15.46:0/2061419132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68e4146ee0 con 0x7f68e415d230 2024-03-29T08:27:22.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.311+0000 7f68daa00640 1 -- 172.21.15.46:0/2061419132 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16003 (secure 0 0 0) 0x7f68e4146ee0 con 0x7f68e415d230 2024-03-29T08:27:22.314 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:22.314 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":106,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:27:19.585517+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:27:17.118875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-29T08:27:17.573928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:27:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.311+0000 7f68e9c00640 1 -- 172.21.15.46:0/2061419132 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68c0066690 msgr2=0x7f68c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.311+0000 7f68e9c00640 1 --2- 172.21.15.46:0/2061419132 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68c0066690 0x7f68c0068b50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f68d40ae000 tx=0x7f68d40b5040 comp rx=0 tx=0).stop 2024-03-29T08:27:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.315+0000 7f68e9c00640 1 -- 172.21.15.46:0/2061419132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e415d230 msgr2=0x7f68e4145860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.315+0000 7f68e9c00640 1 --2- 172.21.15.46:0/2061419132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e415d230 0x7f68e4145860 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f68dc009fd0 tx=0x7f68dc00e4b0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.315+0000 7f68e9c00640 1 -- 172.21.15.46:0/2061419132 shutdown_connections 2024-03-29T08:27:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.315+0000 7f68e9c00640 1 --2- 172.21.15.46:0/2061419132 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f68c0066690 0x7f68c0068b50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.315+0000 7f68e9c00640 1 --2- 172.21.15.46:0/2061419132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68e415cd90 0x7f68e41501a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.315+0000 7f68e9c00640 1 --2- 172.21.15.46:0/2061419132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68e415d230 0x7f68e4145860 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.315+0000 7f68e9c00640 1 --2- 172.21.15.46:0/2061419132 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68e41593d0 0x7f68e41471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.315+0000 7f68e9c00640 1 -- 172.21.15.46:0/2061419132 >> 172.21.15.46:0/2061419132 conn(0x7f68e4066110 msgr2=0x7f68e4164da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.315+0000 7f68e9c00640 1 -- 172.21.15.46:0/2061419132 shutdown_connections 2024-03-29T08:27:22.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.315+0000 7f68e9c00640 1 -- 172.21.15.46:0/2061419132 wait complete. 2024-03-29T08:27:22.334 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T08:27:22.335 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:27:22.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.471+0000 7fc43a400640 1 Processor -- start 2024-03-29T08:27:22.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.471+0000 7fc43a400640 1 -- start start 2024-03-29T08:27:22.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.475+0000 7fc43a400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc434144210 0x7fc4340cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.475+0000 7fc43a400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc43415ada0 0x7fc4340ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.475+0000 7fc43a400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4340cb510 con 0x7fc4340ce950 2024-03-29T08:27:22.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.475+0000 7fc43a400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4340cede0 con 0x7fc43415ada0 2024-03-29T08:27:22.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.475+0000 7fc43a400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43406d960 con 0x7fc434144210 2024-03-29T08:27:22.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.475+0000 7fc42be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc43415ada0 0x7fc4340ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.475+0000 7fc42be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc43415ada0 0x7fc4340ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33660/0 (socket says 172.21.15.46:33660) 2024-03-29T08:27:22.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.475+0000 7fc42be00640 1 -- 172.21.15.46:0/3298155534 learned_addr learned my addr 172.21.15.46:0/3298155534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:22.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.475+0000 7fc433400640 1 --2- 172.21.15.46:0/3298155534 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc434144210 0x7fc4340cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.475+0000 7fc42be00640 1 -- 172.21.15.46:0/3298155534 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc434144210 msgr2=0x7fc4340cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:22.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.475+0000 7fc42be00640 1 --2- 172.21.15.46:0/3298155534 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc434144210 0x7fc4340cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.475+0000 7fc42be00640 1 -- 172.21.15.46:0/3298155534 >> v1:172.21.15.73:6789/0 conn(0x7fc4340ce950 legacy=0x7fc43415e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:22.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.475+0000 7fc42be00640 1 -- 172.21.15.46:0/3298155534 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc434070580 con 0x7fc43415ada0 2024-03-29T08:27:22.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.475+0000 7fc42be00640 1 --2- 172.21.15.46:0/3298155534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc43415ada0 0x7fc4340ce410 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fc41c009870 tx=0x7fc41c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0b652e26147586 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:22.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc432a00640 1 -- 172.21.15.46:0/3298155534 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c0ab600 con 0x7fc43415ada0 2024-03-29T08:27:22.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc432a00640 1 -- 172.21.15.46:0/3298155534 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41c0a7050 con 0x7fc43415ada0 2024-03-29T08:27:22.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc432a00640 1 -- 172.21.15.46:0/3298155534 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c0ab980 con 0x7fc43415ada0 2024-03-29T08:27:22.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc43a400640 1 -- 172.21.15.46:0/3298155534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc43415ada0 msgr2=0x7fc4340ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:22.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc43a400640 1 --2- 172.21.15.46:0/3298155534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc43415ada0 0x7fc4340ce410 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fc41c009870 tx=0x7fc41c01c560 comp rx=0 tx=0).stop 2024-03-29T08:27:22.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc43a400640 1 -- 172.21.15.46:0/3298155534 shutdown_connections 2024-03-29T08:27:22.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc43a400640 1 --2- 172.21.15.46:0/3298155534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc43415ada0 0x7fc4340ce410 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc43a400640 1 --2- 172.21.15.46:0/3298155534 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc434144210 0x7fc4340cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc43a400640 1 -- 172.21.15.46:0/3298155534 >> 172.21.15.46:0/3298155534 conn(0x7fc434066110 msgr2=0x7fc434069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:22.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc43a400640 1 -- 172.21.15.46:0/3298155534 shutdown_connections 2024-03-29T08:27:22.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc43a400640 1 -- 172.21.15.46:0/3298155534 wait complete. 2024-03-29T08:27:22.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc43a400640 1 Processor -- start 2024-03-29T08:27:22.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc43a400640 1 -- start start 2024-03-29T08:27:22.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc43a400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4340ce950 0x7fc43413fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc43a400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc434144210 0x7fc43415b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc43a400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc43415ada0 0x7fc434142c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc43a400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43413f060 con 0x7fc4340ce950 2024-03-29T08:27:22.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc43a400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc43413f5e0 con 0x7fc43415ada0 2024-03-29T08:27:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc43a400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc43413ffb0 con 0x7fc434144210 2024-03-29T08:27:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc433400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4340ce950 0x7fc43413fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc433400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4340ce950 0x7fc43413fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33672/0 (socket says 172.21.15.46:33672) 2024-03-29T08:27:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc433400640 1 -- 172.21.15.46:0/587914031 learned_addr learned my addr 172.21.15.46:0/587914031 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc42be00640 1 --2- 172.21.15.46:0/587914031 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc434144210 0x7fc43415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.479+0000 7fc433e00640 1 --2- 172.21.15.46:0/587914031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc43415ada0 0x7fc434142c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.483+0000 7fc433400640 1 -- 172.21.15.46:0/587914031 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc434144210 msgr2=0x7fc43415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.483+0000 7fc433400640 1 --2- 172.21.15.46:0/587914031 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc434144210 0x7fc43415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.483+0000 7fc433400640 1 -- 172.21.15.46:0/587914031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc43415ada0 msgr2=0x7fc434142c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.483+0000 7fc433400640 1 --2- 172.21.15.46:0/587914031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc43415ada0 0x7fc434142c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.483+0000 7fc433400640 1 -- 172.21.15.46:0/587914031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41c0094d0 con 0x7fc4340ce950 2024-03-29T08:27:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.483+0000 7fc433e00640 1 --2- 172.21.15.46:0/587914031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc43415ada0 0x7fc434142c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.483+0000 7fc433400640 1 --2- 172.21.15.46:0/587914031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4340ce950 0x7fc43413fa70 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fc42400d830 tx=0x7fc42400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.483+0000 7fc42b400640 1 -- 172.21.15.46:0/587914031 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42400b6c0 con 0x7fc4340ce950 2024-03-29T08:27:22.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.483+0000 7fc42b400640 1 -- 172.21.15.46:0/587914031 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42400b860 con 0x7fc4340ce950 2024-03-29T08:27:22.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.483+0000 7fc42b400640 1 -- 172.21.15.46:0/587914031 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42400bc10 con 0x7fc4340ce950 2024-03-29T08:27:22.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.483+0000 7fc43a400640 1 -- 172.21.15.46:0/587914031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc43413d5e0 con 0x7fc4340ce950 2024-03-29T08:27:22.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.483+0000 7fc43a400640 1 -- 172.21.15.46:0/587914031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43413dba0 con 0x7fc4340ce950 2024-03-29T08:27:22.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.483+0000 7fc43a400640 1 -- 172.21.15.46:0/587914031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4340ce280 con 0x7fc4340ce950 2024-03-29T08:27:22.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.483+0000 7fc42b400640 1 -- 172.21.15.46:0/587914031 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc42401e460 con 0x7fc4340ce950 2024-03-29T08:27:22.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.483+0000 7fc42b400640 1 --2- 172.21.15.46:0/587914031 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc410066690 0x7fc410068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.483+0000 7fc42b400640 1 -- 172.21.15.46:0/587914031 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc424084950 con 0x7fc4340ce950 2024-03-29T08:27:22.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.487+0000 7fc42be00640 1 --2- 172.21.15.46:0/587914031 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc410066690 0x7fc410068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.487+0000 7fc42be00640 1 --2- 172.21.15.46:0/587914031 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc410066690 0x7fc410068b50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0ad000 tx=0x7fc41c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.487+0000 7fc42b400640 1 -- 172.21.15.46:0/587914031 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc424051c20 con 0x7fc4340ce950 2024-03-29T08:27:22.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.651+0000 7fc43a400640 1 -- 172.21.15.46:0/587914031 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc4340cbdf0 con 0x7fc410066690 2024-03-29T08:27:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.659+0000 7fc42b400640 1 -- 172.21.15.46:0/587914031 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289770 (secure 0 0 0) 0x7fc41c0ab620 con 0x7fc410066690 2024-03-29T08:27:22.665 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:22.668 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:27:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.663+0000 7fc43a400640 1 -- 172.21.15.46:0/587914031 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc410066690 msgr2=0x7fc410068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.663+0000 7fc43a400640 1 --2- 172.21.15.46:0/587914031 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc410066690 0x7fc410068b50 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0ad000 tx=0x7fc41c008040 comp rx=0 tx=0).stop 2024-03-29T08:27:22.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.663+0000 7fc43a400640 1 -- 172.21.15.46:0/587914031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4340ce950 msgr2=0x7fc43413fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.663+0000 7fc43a400640 1 --2- 172.21.15.46:0/587914031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4340ce950 0x7fc43413fa70 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fc42400d830 tx=0x7fc42400dcf0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.663+0000 7fc43a400640 1 -- 172.21.15.46:0/587914031 shutdown_connections 2024-03-29T08:27:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.663+0000 7fc43a400640 1 --2- 172.21.15.46:0/587914031 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc410066690 0x7fc410068b50 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.663+0000 7fc43a400640 1 --2- 172.21.15.46:0/587914031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc43415ada0 0x7fc434142c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.663+0000 7fc43a400640 1 --2- 172.21.15.46:0/587914031 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc434144210 0x7fc43415b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.663+0000 7fc43a400640 1 --2- 172.21.15.46:0/587914031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4340ce950 0x7fc43413fa70 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.663+0000 7fc43a400640 1 -- 172.21.15.46:0/587914031 >> 172.21.15.46:0/587914031 conn(0x7fc434066110 msgr2=0x7fc43406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.663+0000 7fc43a400640 1 -- 172.21.15.46:0/587914031 shutdown_connections 2024-03-29T08:27:22.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.663+0000 7fc43a400640 1 -- 172.21.15.46:0/587914031 wait complete. 2024-03-29T08:27:22.685 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-03-29T08:27:22.182703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260352,"kb_used_data":11364,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754714368,"statfs":{"total":773094113280,"available":772827512832,"internally_reserved":0,"allocated":11636736,"data_stored":5672184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.005890"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952281+0000","last_change":"2024-03-29T08:27:18.623414+0000","last_active":"2024-03-29T08:27:19.952281+0000","last_peered":"2024-03-29T08:27:19.952281+0000","last_clean":"2024-03-29T08:27:19.952281+0000","last_became_active":"2024-03-29T08:27:18.622981+0000","last_became_peered":"2024-03-29T08:27:18.622981+0000","last_unstale":"2024-03-29T08:27:19.952281+0000","last_undegraded":"2024-03-29T08:27:19.952281+0000","last_fullsized":"2024-03-29T08:27:19.952281+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:36.631872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951993+0000","last_change":"2024-03-29T08:27:18.625651+0000","last_active":"2024-03-29T08:27:19.951993+0000","last_peered":"2024-03-29T08:27:19.951993+0000","last_clean":"2024-03-29T08:27:19.951993+0000","last_became_active":"2024-03-29T08:27:18.624134+0000","last_became_peered":"2024-03-29T08:27:18.624134+0000","last_unstale":"2024-03-29T08:27:19.951993+0000","last_undegraded":"2024-03-29T08:27:19.951993+0000","last_fullsized":"2024-03-29T08:27:19.951993+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:17:30.862649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598970+0000","last_change":"2024-03-29T08:27:18.621776+0000","last_active":"2024-03-29T08:27:19.598970+0000","last_peered":"2024-03-29T08:27:19.598970+0000","last_clean":"2024-03-29T08:27:19.598970+0000","last_became_active":"2024-03-29T08:27:18.621465+0000","last_became_peered":"2024-03-29T08:27:18.621465+0000","last_unstale":"2024-03-29T08:27:19.598970+0000","last_undegraded":"2024-03-29T08:27:19.598970+0000","last_fullsized":"2024-03-29T08:27:19.598970+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:23.870278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951605+0000","last_change":"2024-03-29T08:27:18.630007+0000","last_active":"2024-03-29T08:27:19.951605+0000","last_peered":"2024-03-29T08:27:19.951605+0000","last_clean":"2024-03-29T08:27:19.951605+0000","last_became_active":"2024-03-29T08:27:18.628978+0000","last_became_peered":"2024-03-29T08:27:18.628978+0000","last_unstale":"2024-03-29T08:27:19.951605+0000","last_undegraded":"2024-03-29T08:27:19.951605+0000","last_fullsized":"2024-03-29T08:27:19.951605+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:23.386306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593370+0000","last_change":"2024-03-29T08:27:18.605886+0000","last_active":"2024-03-29T08:27:19.593370+0000","last_peered":"2024-03-29T08:27:19.593370+0000","last_clean":"2024-03-29T08:27:19.593370+0000","last_became_active":"2024-03-29T08:27:18.605245+0000","last_became_peered":"2024-03-29T08:27:18.605245+0000","last_unstale":"2024-03-29T08:27:19.593370+0000","last_undegraded":"2024-03-29T08:27:19.593370+0000","last_fullsized":"2024-03-29T08:27:19.593370+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:11.165228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593629+0000","last_change":"2024-03-29T08:27:18.612974+0000","last_active":"2024-03-29T08:27:19.593629+0000","last_peered":"2024-03-29T08:27:19.593629+0000","last_clean":"2024-03-29T08:27:19.593629+0000","last_became_active":"2024-03-29T08:27:18.612652+0000","last_became_peered":"2024-03-29T08:27:18.612652+0000","last_unstale":"2024-03-29T08:27:19.593629+0000","last_undegraded":"2024-03-29T08:27:19.593629+0000","last_fullsized":"2024-03-29T08:27:19.593629+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:07:06.162438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593231+0000","last_change":"2024-03-29T08:27:18.606026+0000","last_active":"2024-03-29T08:27:19.593231+0000","last_peered":"2024-03-29T08:27:19.593231+0000","last_clean":"2024-03-29T08:27:19.593231+0000","last_became_active":"2024-03-29T08:27:18.605734+0000","last_became_peered":"2024-03-29T08:27:18.605734+0000","last_unstale":"2024-03-29T08:27:19.593231+0000","last_undegraded":"2024-03-29T08:27:19.593231+0000","last_fullsized":"2024-03-29T08:27:19.593231+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:57:59.583230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598435+0000","last_change":"2024-03-29T08:27:18.626903+0000","last_active":"2024-03-29T08:27:19.598435+0000","last_peered":"2024-03-29T08:27:19.598435+0000","last_clean":"2024-03-29T08:27:19.598435+0000","last_became_active":"2024-03-29T08:27:18.626589+0000","last_became_peered":"2024-03-29T08:27:18.626589+0000","last_unstale":"2024-03-29T08:27:19.598435+0000","last_undegraded":"2024-03-29T08:27:19.598435+0000","last_fullsized":"2024-03-29T08:27:19.598435+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:38:15.048270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593760+0000","last_change":"2024-03-29T08:27:18.617848+0000","last_active":"2024-03-29T08:27:19.593760+0000","last_peered":"2024-03-29T08:27:19.593760+0000","last_clean":"2024-03-29T08:27:19.593760+0000","last_became_active":"2024-03-29T08:27:18.617592+0000","last_became_peered":"2024-03-29T08:27:18.617592+0000","last_unstale":"2024-03-29T08:27:19.593760+0000","last_undegraded":"2024-03-29T08:27:19.593760+0000","last_fullsized":"2024-03-29T08:27:19.593760+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:44:13.762743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.592376+0000","last_change":"2024-03-29T08:27:18.620874+0000","last_active":"2024-03-29T08:27:19.592376+0000","last_peered":"2024-03-29T08:27:19.592376+0000","last_clean":"2024-03-29T08:27:19.592376+0000","last_became_active":"2024-03-29T08:27:18.619900+0000","last_became_peered":"2024-03-29T08:27:18.619900+0000","last_unstale":"2024-03-29T08:27:19.592376+0000","last_undegraded":"2024-03-29T08:27:19.592376+0000","last_fullsized":"2024-03-29T08:27:19.592376+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:35:54.064179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593396+0000","last_change":"2024-03-29T08:27:18.622044+0000","last_active":"2024-03-29T08:27:19.593396+0000","last_peered":"2024-03-29T08:27:19.593396+0000","last_clean":"2024-03-29T08:27:19.593396+0000","last_became_active":"2024-03-29T08:27:18.620960+0000","last_became_peered":"2024-03-29T08:27:18.620960+0000","last_unstale":"2024-03-29T08:27:19.593396+0000","last_undegraded":"2024-03-29T08:27:19.593396+0000","last_fullsized":"2024-03-29T08:27:19.593396+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:05:15.943323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946504+0000","last_change":"2024-03-29T08:27:18.620770+0000","last_active":"2024-03-29T08:27:19.946504+0000","last_peered":"2024-03-29T08:27:19.946504+0000","last_clean":"2024-03-29T08:27:19.946504+0000","last_became_active":"2024-03-29T08:27:18.620514+0000","last_became_peered":"2024-03-29T08:27:18.620514+0000","last_unstale":"2024-03-29T08:27:19.946504+0000","last_undegraded":"2024-03-29T08:27:19.946504+0000","last_fullsized":"2024-03-29T08:27:19.946504+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:30:27.814197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952253+0000","last_change":"2024-03-29T08:27:18.618647+0000","last_active":"2024-03-29T08:27:19.952253+0000","last_peered":"2024-03-29T08:27:19.952253+0000","last_clean":"2024-03-29T08:27:19.952253+0000","last_became_active":"2024-03-29T08:27:18.618391+0000","last_became_peered":"2024-03-29T08:27:18.618391+0000","last_unstale":"2024-03-29T08:27:19.952253+0000","last_undegraded":"2024-03-29T08:27:19.952253+0000","last_fullsized":"2024-03-29T08:27:19.952253+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:27.887597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952337+0000","last_change":"2024-03-29T08:27:18.623490+0000","last_active":"2024-03-29T08:27:19.952337+0000","last_peered":"2024-03-29T08:27:19.952337+0000","last_clean":"2024-03-29T08:27:19.952337+0000","last_became_active":"2024-03-29T08:27:18.623126+0000","last_became_peered":"2024-03-29T08:27:18.623126+0000","last_unstale":"2024-03-29T08:27:19.952337+0000","last_undegraded":"2024-03-29T08:27:19.952337+0000","last_fullsized":"2024-03-29T08:27:19.952337+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:23:47.930784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951385+0000","last_change":"2024-03-29T08:27:18.614993+0000","last_active":"2024-03-29T08:27:19.951385+0000","last_peered":"2024-03-29T08:27:19.951385+0000","last_clean":"2024-03-29T08:27:19.951385+0000","last_became_active":"2024-03-29T08:27:18.614734+0000","last_became_peered":"2024-03-29T08:27:18.614734+0000","last_unstale":"2024-03-29T08:27:19.951385+0000","last_undegraded":"2024-03-29T08:27:19.951385+0000","last_fullsized":"2024-03-29T08:27:19.951385+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:38:15.048270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593235+0000","last_change":"2024-03-29T08:27:18.607097+0000","last_active":"2024-03-29T08:27:19.593235+0000","last_peered":"2024-03-29T08:27:19.593235+0000","last_clean":"2024-03-29T08:27:19.593235+0000","last_became_active":"2024-03-29T08:27:18.606339+0000","last_became_peered":"2024-03-29T08:27:18.606339+0000","last_unstale":"2024-03-29T08:27:19.593235+0000","last_undegraded":"2024-03-29T08:27:19.593235+0000","last_fullsized":"2024-03-29T08:27:19.593235+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:33:34.629131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.613850+0000","last_change":"2024-03-29T08:27:18.609933+0000","last_active":"2024-03-29T08:27:18.613850+0000","last_peered":"2024-03-29T08:27:18.613850+0000","last_clean":"2024-03-29T08:27:18.613850+0000","last_became_active":"2024-03-29T08:27:18.609689+0000","last_became_peered":"2024-03-29T08:27:18.609689+0000","last_unstale":"2024-03-29T08:27:18.613850+0000","last_undegraded":"2024-03-29T08:27:18.613850+0000","last_fullsized":"2024-03-29T08:27:18.613850+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:26:04.517467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952043+0000","last_change":"2024-03-29T08:27:18.627100+0000","last_active":"2024-03-29T08:27:19.952043+0000","last_peered":"2024-03-29T08:27:19.952043+0000","last_clean":"2024-03-29T08:27:19.952043+0000","last_became_active":"2024-03-29T08:27:18.624833+0000","last_became_peered":"2024-03-29T08:27:18.624833+0000","last_unstale":"2024-03-29T08:27:19.952043+0000","last_undegraded":"2024-03-29T08:27:19.952043+0000","last_fullsized":"2024-03-29T08:27:19.952043+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:20:57.177828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.630341+0000","last_change":"2024-03-29T08:27:18.629841+0000","last_active":"2024-03-29T08:27:18.630341+0000","last_peered":"2024-03-29T08:27:18.630341+0000","last_clean":"2024-03-29T08:27:18.630341+0000","last_became_active":"2024-03-29T08:27:18.628874+0000","last_became_peered":"2024-03-29T08:27:18.628874+0000","last_unstale":"2024-03-29T08:27:18.630341+0000","last_undegraded":"2024-03-29T08:27:18.630341+0000","last_fullsized":"2024-03-29T08:27:18.630341+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:55:26.401848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946512+0000","last_change":"2024-03-29T08:27:18.600427+0000","last_active":"2024-03-29T08:27:19.946512+0000","last_peered":"2024-03-29T08:27:19.946512+0000","last_clean":"2024-03-29T08:27:19.946512+0000","last_became_active":"2024-03-29T08:27:18.599117+0000","last_became_peered":"2024-03-29T08:27:18.599117+0000","last_unstale":"2024-03-29T08:27:19.946512+0000","last_undegraded":"2024-03-29T08:27:19.946512+0000","last_fullsized":"2024-03-29T08:27:19.946512+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:29:48.275525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598878+0000","last_change":"2024-03-29T08:27:18.620169+0000","last_active":"2024-03-29T08:27:19.598878+0000","last_peered":"2024-03-29T08:27:19.598878+0000","last_clean":"2024-03-29T08:27:19.598878+0000","last_became_active":"2024-03-29T08:27:18.619676+0000","last_became_peered":"2024-03-29T08:27:18.619676+0000","last_unstale":"2024-03-29T08:27:19.598878+0000","last_undegraded":"2024-03-29T08:27:19.598878+0000","last_fullsized":"2024-03-29T08:27:19.598878+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:17:30.862649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598671+0000","last_change":"2024-03-29T08:27:18.620995+0000","last_active":"2024-03-29T08:27:19.598671+0000","last_peered":"2024-03-29T08:27:19.598671+0000","last_clean":"2024-03-29T08:27:19.598671+0000","last_became_active":"2024-03-29T08:27:18.620642+0000","last_became_peered":"2024-03-29T08:27:18.620642+0000","last_unstale":"2024-03-29T08:27:19.598671+0000","last_undegraded":"2024-03-29T08:27:19.598671+0000","last_fullsized":"2024-03-29T08:27:19.598671+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:03:53.966917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951503+0000","last_change":"2024-03-29T08:27:18.611914+0000","last_active":"2024-03-29T08:27:19.951503+0000","last_peered":"2024-03-29T08:27:19.951503+0000","last_clean":"2024-03-29T08:27:19.951503+0000","last_became_active":"2024-03-29T08:27:18.611591+0000","last_became_peered":"2024-03-29T08:27:18.611591+0000","last_unstale":"2024-03-29T08:27:19.951503+0000","last_undegraded":"2024-03-29T08:27:19.951503+0000","last_fullsized":"2024-03-29T08:27:19.951503+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:55.220264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946006+0000","last_change":"2024-03-29T08:27:18.619621+0000","last_active":"2024-03-29T08:27:19.946006+0000","last_peered":"2024-03-29T08:27:19.946006+0000","last_clean":"2024-03-29T08:27:19.946006+0000","last_became_active":"2024-03-29T08:27:18.616379+0000","last_became_peered":"2024-03-29T08:27:18.616379+0000","last_unstale":"2024-03-29T08:27:19.946006+0000","last_undegraded":"2024-03-29T08:27:19.946006+0000","last_fullsized":"2024-03-29T08:27:19.946006+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:23.870278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.592459+0000","last_change":"2024-03-29T08:27:18.599989+0000","last_active":"2024-03-29T08:27:19.592459+0000","last_peered":"2024-03-29T08:27:19.592459+0000","last_clean":"2024-03-29T08:27:19.592459+0000","last_became_active":"2024-03-29T08:27:18.599695+0000","last_became_peered":"2024-03-29T08:27:18.599695+0000","last_unstale":"2024-03-29T08:27:19.592459+0000","last_undegraded":"2024-03-29T08:27:19.592459+0000","last_fullsized":"2024-03-29T08:27:19.592459+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:33.137545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951757+0000","last_change":"2024-03-29T08:27:18.618132+0000","last_active":"2024-03-29T08:27:19.951757+0000","last_peered":"2024-03-29T08:27:19.951757+0000","last_clean":"2024-03-29T08:27:19.951757+0000","last_became_active":"2024-03-29T08:27:18.617846+0000","last_became_peered":"2024-03-29T08:27:18.617846+0000","last_unstale":"2024-03-29T08:27:19.951757+0000","last_undegraded":"2024-03-29T08:27:19.951757+0000","last_fullsized":"2024-03-29T08:27:19.951757+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:56:58.908625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946848+0000","last_change":"2024-03-29T08:27:18.605001+0000","last_active":"2024-03-29T08:27:19.946848+0000","last_peered":"2024-03-29T08:27:19.946848+0000","last_clean":"2024-03-29T08:27:19.946848+0000","last_became_active":"2024-03-29T08:27:18.604709+0000","last_became_peered":"2024-03-29T08:27:18.604709+0000","last_unstale":"2024-03-29T08:27:19.946848+0000","last_undegraded":"2024-03-29T08:27:19.946848+0000","last_fullsized":"2024-03-29T08:27:19.946848+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:38:15.048270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946175+0000","last_change":"2024-03-29T08:27:18.610209+0000","last_active":"2024-03-29T08:27:19.946175+0000","last_peered":"2024-03-29T08:27:19.946175+0000","last_clean":"2024-03-29T08:27:19.946175+0000","last_became_active":"2024-03-29T08:27:18.609954+0000","last_became_peered":"2024-03-29T08:27:18.609954+0000","last_unstale":"2024-03-29T08:27:19.946175+0000","last_undegraded":"2024-03-29T08:27:19.946175+0000","last_fullsized":"2024-03-29T08:27:19.946175+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:08:06.894379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952350+0000","last_change":"2024-03-29T08:27:18.610360+0000","last_active":"2024-03-29T08:27:19.952350+0000","last_peered":"2024-03-29T08:27:19.952350+0000","last_clean":"2024-03-29T08:27:19.952350+0000","last_became_active":"2024-03-29T08:27:18.610070+0000","last_became_peered":"2024-03-29T08:27:18.610070+0000","last_unstale":"2024-03-29T08:27:19.952350+0000","last_undegraded":"2024-03-29T08:27:19.952350+0000","last_fullsized":"2024-03-29T08:27:19.952350+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:57:51.478616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952572+0000","last_change":"2024-03-29T08:27:18.613026+0000","last_active":"2024-03-29T08:27:19.952572+0000","last_peered":"2024-03-29T08:27:19.952572+0000","last_clean":"2024-03-29T08:27:19.952572+0000","last_became_active":"2024-03-29T08:27:18.612775+0000","last_became_peered":"2024-03-29T08:27:18.612775+0000","last_unstale":"2024-03-29T08:27:19.952572+0000","last_undegraded":"2024-03-29T08:27:19.952572+0000","last_fullsized":"2024-03-29T08:27:19.952572+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:55:56.587423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598565+0000","last_change":"2024-03-29T08:27:18.612729+0000","last_active":"2024-03-29T08:27:19.598565+0000","last_peered":"2024-03-29T08:27:19.598565+0000","last_clean":"2024-03-29T08:27:19.598565+0000","last_became_active":"2024-03-29T08:27:18.612483+0000","last_became_peered":"2024-03-29T08:27:18.612483+0000","last_unstale":"2024-03-29T08:27:19.598565+0000","last_undegraded":"2024-03-29T08:27:19.598565+0000","last_fullsized":"2024-03-29T08:27:19.598565+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:25.324724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.950794+0000","last_change":"2024-03-29T08:27:18.613817+0000","last_active":"2024-03-29T08:27:19.950794+0000","last_peered":"2024-03-29T08:27:19.950794+0000","last_clean":"2024-03-29T08:27:19.950794+0000","last_became_active":"2024-03-29T08:27:18.610960+0000","last_became_peered":"2024-03-29T08:27:18.610960+0000","last_unstale":"2024-03-29T08:27:19.950794+0000","last_undegraded":"2024-03-29T08:27:19.950794+0000","last_fullsized":"2024-03-29T08:27:19.950794+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:23.870278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.608524+0000","last_change":"2024-03-29T08:27:18.602948+0000","last_active":"2024-03-29T08:27:18.608524+0000","last_peered":"2024-03-29T08:27:18.608524+0000","last_clean":"2024-03-29T08:27:18.608524+0000","last_became_active":"2024-03-29T08:27:18.602691+0000","last_became_peered":"2024-03-29T08:27:18.602691+0000","last_unstale":"2024-03-29T08:27:18.608524+0000","last_undegraded":"2024-03-29T08:27:18.608524+0000","last_fullsized":"2024-03-29T08:27:18.608524+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:01:12.985977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"105'4","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952072+0000","last_change":"2024-03-29T08:27:18.208888+0000","last_active":"2024-03-29T08:27:19.952072+0000","last_peered":"2024-03-29T08:27:19.952072+0000","last_clean":"2024-03-29T08:27:19.952072+0000","last_became_active":"2024-03-29T08:27:18.208588+0000","last_became_peered":"2024-03-29T08:27:18.208588+0000","last_unstale":"2024-03-29T08:27:19.952072+0000","last_undegraded":"2024-03-29T08:27:19.952072+0000","last_fullsized":"2024-03-29T08:27:19.952072+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:09:14.747589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951987+0000","last_change":"2024-03-29T08:27:18.613680+0000","last_active":"2024-03-29T08:27:19.951987+0000","last_peered":"2024-03-29T08:27:19.951987+0000","last_clean":"2024-03-29T08:27:19.951987+0000","last_became_active":"2024-03-29T08:27:18.613225+0000","last_became_peered":"2024-03-29T08:27:18.613225+0000","last_unstale":"2024-03-29T08:27:19.951987+0000","last_undegraded":"2024-03-29T08:27:19.951987+0000","last_fullsized":"2024-03-29T08:27:19.951987+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:06.295589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"105'1","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593637+0000","last_change":"2024-03-29T08:27:18.191059+0000","last_active":"2024-03-29T08:27:19.593637+0000","last_peered":"2024-03-29T08:27:19.593637+0000","last_clean":"2024-03-29T08:27:19.593637+0000","last_became_active":"2024-03-29T08:27:18.190819+0000","last_became_peered":"2024-03-29T08:27:18.190819+0000","last_unstale":"2024-03-29T08:27:19.593637+0000","last_undegraded":"2024-03-29T08:27:19.593637+0000","last_fullsized":"2024-03-29T08:27:19.593637+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:56:25.523454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951711+0000","last_change":"2024-03-29T08:27:18.623453+0000","last_active":"2024-03-29T08:27:19.951711+0000","last_peered":"2024-03-29T08:27:19.951711+0000","last_clean":"2024-03-29T08:27:19.951711+0000","last_became_active":"2024-03-29T08:27:18.623150+0000","last_became_peered":"2024-03-29T08:27:18.623150+0000","last_unstale":"2024-03-29T08:27:19.951711+0000","last_undegraded":"2024-03-29T08:27:19.951711+0000","last_fullsized":"2024-03-29T08:27:19.951711+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:03:53.966917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"105'2","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.599149+0000","last_change":"2024-03-29T08:27:18.209722+0000","last_active":"2024-03-29T08:27:19.599149+0000","last_peered":"2024-03-29T08:27:19.599149+0000","last_clean":"2024-03-29T08:27:19.599149+0000","last_became_active":"2024-03-29T08:27:18.209395+0000","last_became_peered":"2024-03-29T08:27:18.209395+0000","last_unstale":"2024-03-29T08:27:19.599149+0000","last_undegraded":"2024-03-29T08:27:19.599149+0000","last_fullsized":"2024-03-29T08:27:19.599149+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:45:54.654095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.592576+0000","last_change":"2024-03-29T08:27:18.610025+0000","last_active":"2024-03-29T08:27:19.592576+0000","last_peered":"2024-03-29T08:27:19.592576+0000","last_clean":"2024-03-29T08:27:19.592576+0000","last_became_active":"2024-03-29T08:27:18.609771+0000","last_became_peered":"2024-03-29T08:27:18.609771+0000","last_unstale":"2024-03-29T08:27:19.592576+0000","last_undegraded":"2024-03-29T08:27:19.592576+0000","last_fullsized":"2024-03-29T08:27:19.592576+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:11:19.080978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"105'2","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951827+0000","last_change":"2024-03-29T08:27:18.207299+0000","last_active":"2024-03-29T08:27:19.951827+0000","last_peered":"2024-03-29T08:27:19.951827+0000","last_clean":"2024-03-29T08:27:19.951827+0000","last_became_active":"2024-03-29T08:27:18.206874+0000","last_became_peered":"2024-03-29T08:27:18.206874+0000","last_unstale":"2024-03-29T08:27:19.951827+0000","last_undegraded":"2024-03-29T08:27:19.951827+0000","last_fullsized":"2024-03-29T08:27:19.951827+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:54:28.856275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946281+0000","last_change":"2024-03-29T08:27:18.604980+0000","last_active":"2024-03-29T08:27:19.946281+0000","last_peered":"2024-03-29T08:27:19.946281+0000","last_clean":"2024-03-29T08:27:19.946281+0000","last_became_active":"2024-03-29T08:27:18.604675+0000","last_became_peered":"2024-03-29T08:27:18.604675+0000","last_unstale":"2024-03-29T08:27:19.946281+0000","last_undegraded":"2024-03-29T08:27:19.946281+0000","last_fullsized":"2024-03-29T08:27:19.946281+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:40:03.566438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"105'1","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951662+0000","last_change":"2024-03-29T08:27:18.196647+0000","last_active":"2024-03-29T08:27:19.951662+0000","last_peered":"2024-03-29T08:27:19.951662+0000","last_clean":"2024-03-29T08:27:19.951662+0000","last_became_active":"2024-03-29T08:27:18.196399+0000","last_became_peered":"2024-03-29T08:27:18.196399+0000","last_unstale":"2024-03-29T08:27:19.951662+0000","last_undegraded":"2024-03-29T08:27:19.951662+0000","last_fullsized":"2024-03-29T08:27:19.951662+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:26.885570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593373+0000","last_change":"2024-03-29T08:27:18.606031+0000","last_active":"2024-03-29T08:27:19.593373+0000","last_peered":"2024-03-29T08:27:19.593373+0000","last_clean":"2024-03-29T08:27:19.593373+0000","last_became_active":"2024-03-29T08:27:18.605630+0000","last_became_peered":"2024-03-29T08:27:18.605630+0000","last_unstale":"2024-03-29T08:27:19.593373+0000","last_undegraded":"2024-03-29T08:27:19.593373+0000","last_fullsized":"2024-03-29T08:27:19.593373+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:36:28.363643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"105'1","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598822+0000","last_change":"2024-03-29T08:27:18.202384+0000","last_active":"2024-03-29T08:27:19.598822+0000","last_peered":"2024-03-29T08:27:19.598822+0000","last_clean":"2024-03-29T08:27:19.598822+0000","last_became_active":"2024-03-29T08:27:18.202088+0000","last_became_peered":"2024-03-29T08:27:18.202088+0000","last_unstale":"2024-03-29T08:27:19.598822+0000","last_undegraded":"2024-03-29T08:27:19.598822+0000","last_fullsized":"2024-03-29T08:27:19.598822+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:40:57.395634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951360+0000","last_change":"2024-03-29T08:27:18.622412+0000","last_active":"2024-03-29T08:27:19.951360+0000","last_peered":"2024-03-29T08:27:19.951360+0000","last_clean":"2024-03-29T08:27:19.951360+0000","last_became_active":"2024-03-29T08:27:18.622123+0000","last_became_peered":"2024-03-29T08:27:18.622123+0000","last_unstale":"2024-03-29T08:27:19.951360+0000","last_undegraded":"2024-03-29T08:27:19.951360+0000","last_fullsized":"2024-03-29T08:27:19.951360+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:54:29.858302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593352+0000","last_change":"2024-03-29T08:27:18.195629+0000","last_active":"2024-03-29T08:27:19.593352+0000","last_peered":"2024-03-29T08:27:19.593352+0000","last_clean":"2024-03-29T08:27:19.593352+0000","last_became_active":"2024-03-29T08:27:18.195315+0000","last_became_peered":"2024-03-29T08:27:18.195315+0000","last_unstale":"2024-03-29T08:27:19.593352+0000","last_undegraded":"2024-03-29T08:27:19.593352+0000","last_fullsized":"2024-03-29T08:27:19.593352+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:55.763695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946540+0000","last_change":"2024-03-29T08:27:18.609143+0000","last_active":"2024-03-29T08:27:19.946540+0000","last_peered":"2024-03-29T08:27:19.946540+0000","last_clean":"2024-03-29T08:27:19.946540+0000","last_became_active":"2024-03-29T08:27:18.608712+0000","last_became_peered":"2024-03-29T08:27:18.608712+0000","last_unstale":"2024-03-29T08:27:19.946540+0000","last_undegraded":"2024-03-29T08:27:19.946540+0000","last_fullsized":"2024-03-29T08:27:19.946540+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:53.626567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.599424+0000","last_change":"2024-03-29T08:27:18.200264+0000","last_active":"2024-03-29T08:27:18.599424+0000","last_peered":"2024-03-29T08:27:18.599424+0000","last_clean":"2024-03-29T08:27:18.599424+0000","last_became_active":"2024-03-29T08:27:18.199981+0000","last_became_peered":"2024-03-29T08:27:18.199981+0000","last_unstale":"2024-03-29T08:27:18.599424+0000","last_undegraded":"2024-03-29T08:27:18.599424+0000","last_fullsized":"2024-03-29T08:27:18.599424+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:11:24.362800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952092+0000","last_change":"2024-03-29T08:27:18.625409+0000","last_active":"2024-03-29T08:27:19.952092+0000","last_peered":"2024-03-29T08:27:19.952092+0000","last_clean":"2024-03-29T08:27:19.952092+0000","last_became_active":"2024-03-29T08:27:18.623585+0000","last_became_peered":"2024-03-29T08:27:18.623585+0000","last_unstale":"2024-03-29T08:27:19.952092+0000","last_undegraded":"2024-03-29T08:27:19.952092+0000","last_fullsized":"2024-03-29T08:27:19.952092+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:25.324724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.599102+0000","last_change":"2024-03-29T08:27:18.206460+0000","last_active":"2024-03-29T08:27:19.599102+0000","last_peered":"2024-03-29T08:27:19.599102+0000","last_clean":"2024-03-29T08:27:19.599102+0000","last_became_active":"2024-03-29T08:27:18.206081+0000","last_became_peered":"2024-03-29T08:27:18.206081+0000","last_unstale":"2024-03-29T08:27:19.599102+0000","last_undegraded":"2024-03-29T08:27:19.599102+0000","last_fullsized":"2024-03-29T08:27:19.599102+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:09:24.332081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946913+0000","last_change":"2024-03-29T08:27:18.593221+0000","last_active":"2024-03-29T08:27:19.946913+0000","last_peered":"2024-03-29T08:27:19.946913+0000","last_clean":"2024-03-29T08:27:19.946913+0000","last_became_active":"2024-03-29T08:27:18.592927+0000","last_became_peered":"2024-03-29T08:27:18.592927+0000","last_unstale":"2024-03-29T08:27:19.946913+0000","last_undegraded":"2024-03-29T08:27:19.946913+0000","last_fullsized":"2024-03-29T08:27:19.946913+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:23.386306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952173+0000","last_change":"2024-03-29T08:27:18.209092+0000","last_active":"2024-03-29T08:27:19.952173+0000","last_peered":"2024-03-29T08:27:19.952173+0000","last_clean":"2024-03-29T08:27:19.952173+0000","last_became_active":"2024-03-29T08:27:18.208292+0000","last_became_peered":"2024-03-29T08:27:18.208292+0000","last_unstale":"2024-03-29T08:27:19.952173+0000","last_undegraded":"2024-03-29T08:27:19.952173+0000","last_fullsized":"2024-03-29T08:27:19.952173+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:35:53.062152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593694+0000","last_change":"2024-03-29T08:27:18.622138+0000","last_active":"2024-03-29T08:27:19.593694+0000","last_peered":"2024-03-29T08:27:19.593694+0000","last_clean":"2024-03-29T08:27:19.593694+0000","last_became_active":"2024-03-29T08:27:18.621894+0000","last_became_peered":"2024-03-29T08:27:18.621894+0000","last_unstale":"2024-03-29T08:27:19.593694+0000","last_undegraded":"2024-03-29T08:27:19.593694+0000","last_fullsized":"2024-03-29T08:27:19.593694+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:33:39.623305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"105'3","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593610+0000","last_change":"2024-03-29T08:27:18.193031+0000","last_active":"2024-03-29T08:27:19.593610+0000","last_peered":"2024-03-29T08:27:19.593610+0000","last_clean":"2024-03-29T08:27:19.593610+0000","last_became_active":"2024-03-29T08:27:18.192645+0000","last_became_peered":"2024-03-29T08:27:18.192645+0000","last_unstale":"2024-03-29T08:27:19.593610+0000","last_undegraded":"2024-03-29T08:27:19.593610+0000","last_fullsized":"2024-03-29T08:27:19.593610+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:47:20.012694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946746+0000","last_change":"2024-03-29T08:27:18.603075+0000","last_active":"2024-03-29T08:27:19.946746+0000","last_peered":"2024-03-29T08:27:19.946746+0000","last_clean":"2024-03-29T08:27:19.946746+0000","last_became_active":"2024-03-29T08:27:18.601081+0000","last_became_peered":"2024-03-29T08:27:18.601081+0000","last_unstale":"2024-03-29T08:27:19.946746+0000","last_undegraded":"2024-03-29T08:27:19.946746+0000","last_fullsized":"2024-03-29T08:27:19.946746+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:20:57.177828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"105'6","reported_seq":17,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946591+0000","last_change":"2024-03-29T08:27:18.194476+0000","last_active":"2024-03-29T08:27:19.946591+0000","last_peered":"2024-03-29T08:27:19.946591+0000","last_clean":"2024-03-29T08:27:19.946591+0000","last_became_active":"2024-03-29T08:27:18.194201+0000","last_became_peered":"2024-03-29T08:27:18.194201+0000","last_unstale":"2024-03-29T08:27:19.946591+0000","last_undegraded":"2024-03-29T08:27:19.946591+0000","last_fullsized":"2024-03-29T08:27:19.946591+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:23:07.623906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951597+0000","last_change":"2024-03-29T08:27:18.619655+0000","last_active":"2024-03-29T08:27:19.951597+0000","last_peered":"2024-03-29T08:27:19.951597+0000","last_clean":"2024-03-29T08:27:19.951597+0000","last_became_active":"2024-03-29T08:27:18.619405+0000","last_became_peered":"2024-03-29T08:27:18.619405+0000","last_unstale":"2024-03-29T08:27:19.951597+0000","last_undegraded":"2024-03-29T08:27:19.951597+0000","last_fullsized":"2024-03-29T08:27:19.951597+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:12:08.460617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"105'2","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.628692+0000","last_change":"2024-03-29T08:27:18.199098+0000","last_active":"2024-03-29T08:27:18.628692+0000","last_peered":"2024-03-29T08:27:18.628692+0000","last_clean":"2024-03-29T08:27:18.628692+0000","last_became_active":"2024-03-29T08:27:18.198849+0000","last_became_peered":"2024-03-29T08:27:18.198849+0000","last_unstale":"2024-03-29T08:27:18.628692+0000","last_undegraded":"2024-03-29T08:27:18.628692+0000","last_fullsized":"2024-03-29T08:27:18.628692+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:37:40.981819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598872+0000","last_change":"2024-03-29T08:27:18.614573+0000","last_active":"2024-03-29T08:27:19.598872+0000","last_peered":"2024-03-29T08:27:19.598872+0000","last_clean":"2024-03-29T08:27:19.598872+0000","last_became_active":"2024-03-29T08:27:18.614306+0000","last_became_peered":"2024-03-29T08:27:18.614306+0000","last_unstale":"2024-03-29T08:27:19.598872+0000","last_undegraded":"2024-03-29T08:27:19.598872+0000","last_fullsized":"2024-03-29T08:27:19.598872+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:20:57.177828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951700+0000","last_change":"2024-03-29T08:27:18.201545+0000","last_active":"2024-03-29T08:27:19.951700+0000","last_peered":"2024-03-29T08:27:19.951700+0000","last_clean":"2024-03-29T08:27:19.951700+0000","last_became_active":"2024-03-29T08:27:18.201287+0000","last_became_peered":"2024-03-29T08:27:18.201287+0000","last_unstale":"2024-03-29T08:27:19.951700+0000","last_undegraded":"2024-03-29T08:27:19.951700+0000","last_fullsized":"2024-03-29T08:27:19.951700+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:55:55.585396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593296+0000","last_change":"2024-03-29T08:27:18.613019+0000","last_active":"2024-03-29T08:27:19.593296+0000","last_peered":"2024-03-29T08:27:19.593296+0000","last_clean":"2024-03-29T08:27:19.593296+0000","last_became_active":"2024-03-29T08:27:18.612574+0000","last_became_peered":"2024-03-29T08:27:18.612574+0000","last_unstale":"2024-03-29T08:27:19.593296+0000","last_undegraded":"2024-03-29T08:27:19.593296+0000","last_fullsized":"2024-03-29T08:27:19.593296+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:07:55.309563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946363+0000","last_change":"2024-03-29T08:27:18.189310+0000","last_active":"2024-03-29T08:27:19.946363+0000","last_peered":"2024-03-29T08:27:19.946363+0000","last_clean":"2024-03-29T08:27:19.946363+0000","last_became_active":"2024-03-29T08:27:18.188803+0000","last_became_peered":"2024-03-29T08:27:18.188803+0000","last_unstale":"2024-03-29T08:27:19.946363+0000","last_undegraded":"2024-03-29T08:27:19.946363+0000","last_fullsized":"2024-03-29T08:27:19.946363+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:32:21.502500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598640+0000","last_change":"2024-03-29T08:27:18.624216+0000","last_active":"2024-03-29T08:27:19.598640+0000","last_peered":"2024-03-29T08:27:19.598640+0000","last_clean":"2024-03-29T08:27:19.598640+0000","last_became_active":"2024-03-29T08:27:18.623842+0000","last_became_peered":"2024-03-29T08:27:18.623842+0000","last_unstale":"2024-03-29T08:27:19.598640+0000","last_undegraded":"2024-03-29T08:27:19.598640+0000","last_fullsized":"2024-03-29T08:27:19.598640+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:12:08.460617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"105'8","reported_seq":19,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951105+0000","last_change":"2024-03-29T08:27:18.197306+0000","last_active":"2024-03-29T08:27:19.951105+0000","last_peered":"2024-03-29T08:27:19.951105+0000","last_clean":"2024-03-29T08:27:19.951105+0000","last_became_active":"2024-03-29T08:27:18.197015+0000","last_became_peered":"2024-03-29T08:27:18.197015+0000","last_unstale":"2024-03-29T08:27:19.951105+0000","last_undegraded":"2024-03-29T08:27:19.951105+0000","last_fullsized":"2024-03-29T08:27:19.951105+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:56:57.906598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946932+0000","last_change":"2024-03-29T08:27:18.596781+0000","last_active":"2024-03-29T08:27:19.946932+0000","last_peered":"2024-03-29T08:27:19.946932+0000","last_clean":"2024-03-29T08:27:19.946932+0000","last_became_active":"2024-03-29T08:27:18.596528+0000","last_became_peered":"2024-03-29T08:27:18.596528+0000","last_unstale":"2024-03-29T08:27:19.946932+0000","last_undegraded":"2024-03-29T08:27:19.946932+0000","last_fullsized":"2024-03-29T08:27:19.946932+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:36.631872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"105'2","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946452+0000","last_change":"2024-03-29T08:27:18.188319+0000","last_active":"2024-03-29T08:27:19.946452+0000","last_peered":"2024-03-29T08:27:19.946452+0000","last_clean":"2024-03-29T08:27:19.946452+0000","last_became_active":"2024-03-29T08:27:18.188073+0000","last_became_peered":"2024-03-29T08:27:18.188073+0000","last_unstale":"2024-03-29T08:27:19.946452+0000","last_undegraded":"2024-03-29T08:27:19.946452+0000","last_fullsized":"2024-03-29T08:27:19.946452+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:12.994086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946536+0000","last_change":"2024-03-29T08:27:18.596679+0000","last_active":"2024-03-29T08:27:19.946536+0000","last_peered":"2024-03-29T08:27:19.946536+0000","last_clean":"2024-03-29T08:27:19.946536+0000","last_became_active":"2024-03-29T08:27:18.595841+0000","last_became_peered":"2024-03-29T08:27:18.595841+0000","last_unstale":"2024-03-29T08:27:19.946536+0000","last_undegraded":"2024-03-29T08:27:19.946536+0000","last_fullsized":"2024-03-29T08:27:19.946536+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:17:30.862649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"105'1","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593529+0000","last_change":"2024-03-29T08:27:18.192026+0000","last_active":"2024-03-29T08:27:19.593529+0000","last_peered":"2024-03-29T08:27:19.593529+0000","last_clean":"2024-03-29T08:27:19.593529+0000","last_became_active":"2024-03-29T08:27:18.191739+0000","last_became_peered":"2024-03-29T08:27:18.191739+0000","last_unstale":"2024-03-29T08:27:19.593529+0000","last_undegraded":"2024-03-29T08:27:19.593529+0000","last_fullsized":"2024-03-29T08:27:19.593529+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:08:26.898804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946574+0000","last_change":"2024-03-29T08:27:18.599059+0000","last_active":"2024-03-29T08:27:19.946574+0000","last_peered":"2024-03-29T08:27:19.946574+0000","last_clean":"2024-03-29T08:27:19.946574+0000","last_became_active":"2024-03-29T08:27:18.598785+0000","last_became_peered":"2024-03-29T08:27:18.598785+0000","last_unstale":"2024-03-29T08:27:19.946574+0000","last_undegraded":"2024-03-29T08:27:19.946574+0000","last_fullsized":"2024-03-29T08:27:19.946574+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:47:12.134933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.598976+0000","last_change":"2024-03-29T08:27:18.197415+0000","last_active":"2024-03-29T08:27:18.598976+0000","last_peered":"2024-03-29T08:27:18.598976+0000","last_clean":"2024-03-29T08:27:18.598976+0000","last_became_active":"2024-03-29T08:27:18.197122+0000","last_became_peered":"2024-03-29T08:27:18.197122+0000","last_unstale":"2024-03-29T08:27:18.598976+0000","last_undegraded":"2024-03-29T08:27:18.598976+0000","last_fullsized":"2024-03-29T08:27:18.598976+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:16:46.630797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.618348+0000","last_change":"2024-03-29T08:27:18.615797+0000","last_active":"2024-03-29T08:27:18.618348+0000","last_peered":"2024-03-29T08:27:18.618348+0000","last_clean":"2024-03-29T08:27:18.618348+0000","last_became_active":"2024-03-29T08:27:18.615532+0000","last_became_peered":"2024-03-29T08:27:18.615532+0000","last_unstale":"2024-03-29T08:27:18.618348+0000","last_undegraded":"2024-03-29T08:27:18.618348+0000","last_fullsized":"2024-03-29T08:27:18.618348+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:22.690924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"105'2","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951767+0000","last_change":"2024-03-29T08:27:18.210866+0000","last_active":"2024-03-29T08:27:19.951767+0000","last_peered":"2024-03-29T08:27:19.951767+0000","last_clean":"2024-03-29T08:27:19.951767+0000","last_became_active":"2024-03-29T08:27:18.210247+0000","last_became_peered":"2024-03-29T08:27:18.210247+0000","last_unstale":"2024-03-29T08:27:19.951767+0000","last_undegraded":"2024-03-29T08:27:19.951767+0000","last_fullsized":"2024-03-29T08:27:19.951767+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:32.135518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.619054+0000","last_change":"2024-03-29T08:27:18.615978+0000","last_active":"2024-03-29T08:27:18.619054+0000","last_peered":"2024-03-29T08:27:18.619054+0000","last_clean":"2024-03-29T08:27:18.619054+0000","last_became_active":"2024-03-29T08:27:18.615670+0000","last_became_peered":"2024-03-29T08:27:18.615670+0000","last_unstale":"2024-03-29T08:27:18.619054+0000","last_undegraded":"2024-03-29T08:27:18.619054+0000","last_fullsized":"2024-03-29T08:27:18.619054+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:49:52.815184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946040+0000","last_change":"2024-03-29T08:27:18.195955+0000","last_active":"2024-03-29T08:27:19.946040+0000","last_peered":"2024-03-29T08:27:19.946040+0000","last_clean":"2024-03-29T08:27:19.946040+0000","last_became_active":"2024-03-29T08:27:18.195631+0000","last_became_peered":"2024-03-29T08:27:18.195631+0000","last_unstale":"2024-03-29T08:27:19.946040+0000","last_undegraded":"2024-03-29T08:27:19.946040+0000","last_fullsized":"2024-03-29T08:27:19.946040+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:30:17.738008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.950892+0000","last_change":"2024-03-29T08:27:18.613281+0000","last_active":"2024-03-29T08:27:19.950892+0000","last_peered":"2024-03-29T08:27:19.950892+0000","last_clean":"2024-03-29T08:27:19.950892+0000","last_became_active":"2024-03-29T08:27:18.611492+0000","last_became_peered":"2024-03-29T08:27:18.611492+0000","last_unstale":"2024-03-29T08:27:19.950892+0000","last_undegraded":"2024-03-29T08:27:19.950892+0000","last_fullsized":"2024-03-29T08:27:19.950892+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:45.577113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.592336+0000","last_change":"2024-03-29T08:27:18.193706+0000","last_active":"2024-03-29T08:27:19.592336+0000","last_peered":"2024-03-29T08:27:19.592336+0000","last_clean":"2024-03-29T08:27:19.592336+0000","last_became_active":"2024-03-29T08:27:18.193394+0000","last_became_peered":"2024-03-29T08:27:18.193394+0000","last_unstale":"2024-03-29T08:27:19.592336+0000","last_undegraded":"2024-03-29T08:27:19.592336+0000","last_fullsized":"2024-03-29T08:27:19.592336+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:57:50.476589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946155+0000","last_change":"2024-03-29T08:27:18.610911+0000","last_active":"2024-03-29T08:27:19.946155+0000","last_peered":"2024-03-29T08:27:19.946155+0000","last_clean":"2024-03-29T08:27:19.946155+0000","last_became_active":"2024-03-29T08:27:18.610667+0000","last_became_peered":"2024-03-29T08:27:18.610667+0000","last_unstale":"2024-03-29T08:27:19.946155+0000","last_undegraded":"2024-03-29T08:27:19.946155+0000","last_fullsized":"2024-03-29T08:27:19.946155+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:50:18.987192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598543+0000","last_change":"2024-03-29T08:27:18.209289+0000","last_active":"2024-03-29T08:27:19.598543+0000","last_peered":"2024-03-29T08:27:19.598543+0000","last_clean":"2024-03-29T08:27:19.598543+0000","last_became_active":"2024-03-29T08:27:18.209033+0000","last_became_peered":"2024-03-29T08:27:18.209033+0000","last_unstale":"2024-03-29T08:27:19.598543+0000","last_undegraded":"2024-03-29T08:27:19.598543+0000","last_fullsized":"2024-03-29T08:27:19.598543+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:52:10.210324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.608499+0000","last_change":"2024-03-29T08:27:18.602427+0000","last_active":"2024-03-29T08:27:18.608499+0000","last_peered":"2024-03-29T08:27:18.608499+0000","last_clean":"2024-03-29T08:27:18.608499+0000","last_became_active":"2024-03-29T08:27:18.602157+0000","last_became_peered":"2024-03-29T08:27:18.602157+0000","last_unstale":"2024-03-29T08:27:18.608499+0000","last_undegraded":"2024-03-29T08:27:18.608499+0000","last_fullsized":"2024-03-29T08:27:18.608499+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:54:49.572315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"105'2","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946720+0000","last_change":"2024-03-29T08:27:18.189214+0000","last_active":"2024-03-29T08:27:19.946720+0000","last_peered":"2024-03-29T08:27:19.946720+0000","last_clean":"2024-03-29T08:27:19.946720+0000","last_became_active":"2024-03-29T08:27:18.188866+0000","last_became_peered":"2024-03-29T08:27:18.188866+0000","last_unstale":"2024-03-29T08:27:19.946720+0000","last_undegraded":"2024-03-29T08:27:19.946720+0000","last_fullsized":"2024-03-29T08:27:19.946720+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:09.365581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.628508+0000","last_change":"2024-03-29T08:27:18.626711+0000","last_active":"2024-03-29T08:27:18.628508+0000","last_peered":"2024-03-29T08:27:18.628508+0000","last_clean":"2024-03-29T08:27:18.628508+0000","last_became_active":"2024-03-29T08:27:18.626076+0000","last_became_peered":"2024-03-29T08:27:18.626076+0000","last_unstale":"2024-03-29T08:27:18.628508+0000","last_undegraded":"2024-03-29T08:27:18.628508+0000","last_fullsized":"2024-03-29T08:27:18.628508+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:04:06.478927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.599630+0000","last_change":"2024-03-29T08:27:18.209259+0000","last_active":"2024-03-29T08:27:18.599630+0000","last_peered":"2024-03-29T08:27:18.599630+0000","last_clean":"2024-03-29T08:27:18.599630+0000","last_became_active":"2024-03-29T08:27:18.208959+0000","last_became_peered":"2024-03-29T08:27:18.208959+0000","last_unstale":"2024-03-29T08:27:18.599630+0000","last_undegraded":"2024-03-29T08:27:18.599630+0000","last_fullsized":"2024-03-29T08:27:18.599630+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:00:59.709598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598809+0000","last_change":"2024-03-29T08:27:18.626266+0000","last_active":"2024-03-29T08:27:19.598809+0000","last_peered":"2024-03-29T08:27:19.598809+0000","last_clean":"2024-03-29T08:27:19.598809+0000","last_became_active":"2024-03-29T08:27:18.626013+0000","last_became_peered":"2024-03-29T08:27:18.626013+0000","last_unstale":"2024-03-29T08:27:19.598809+0000","last_undegraded":"2024-03-29T08:27:19.598809+0000","last_fullsized":"2024-03-29T08:27:19.598809+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:36.631872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.599425+0000","last_change":"2024-03-29T08:27:18.195634+0000","last_active":"2024-03-29T08:27:18.599425+0000","last_peered":"2024-03-29T08:27:18.599425+0000","last_clean":"2024-03-29T08:27:18.599425+0000","last_became_active":"2024-03-29T08:27:18.195388+0000","last_became_peered":"2024-03-29T08:27:18.195388+0000","last_unstale":"2024-03-29T08:27:18.599425+0000","last_undegraded":"2024-03-29T08:27:18.599425+0000","last_fullsized":"2024-03-29T08:27:18.599425+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:16:54.769206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593403+0000","last_change":"2024-03-29T08:27:18.613296+0000","last_active":"2024-03-29T08:27:19.593403+0000","last_peered":"2024-03-29T08:27:19.593403+0000","last_clean":"2024-03-29T08:27:19.593403+0000","last_became_active":"2024-03-29T08:27:18.612843+0000","last_became_peered":"2024-03-29T08:27:18.612843+0000","last_unstale":"2024-03-29T08:27:19.593403+0000","last_undegraded":"2024-03-29T08:27:19.593403+0000","last_fullsized":"2024-03-29T08:27:19.593403+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:52:20.003422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946568+0000","last_change":"2024-03-29T08:27:18.191694+0000","last_active":"2024-03-29T08:27:19.946568+0000","last_peered":"2024-03-29T08:27:19.946568+0000","last_clean":"2024-03-29T08:27:19.946568+0000","last_became_active":"2024-03-29T08:27:18.191378+0000","last_became_peered":"2024-03-29T08:27:18.191378+0000","last_unstale":"2024-03-29T08:27:19.946568+0000","last_undegraded":"2024-03-29T08:27:19.946568+0000","last_fullsized":"2024-03-29T08:27:19.946568+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:24:43.469430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.612053+0000","last_change":"2024-03-29T08:27:18.604095+0000","last_active":"2024-03-29T08:27:18.612053+0000","last_peered":"2024-03-29T08:27:18.612053+0000","last_clean":"2024-03-29T08:27:18.612053+0000","last_became_active":"2024-03-29T08:27:18.603458+0000","last_became_peered":"2024-03-29T08:27:18.603458+0000","last_unstale":"2024-03-29T08:27:18.612053+0000","last_undegraded":"2024-03-29T08:27:18.612053+0000","last_fullsized":"2024-03-29T08:27:18.612053+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:20:23.286313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.599013+0000","last_change":"2024-03-29T08:27:18.209215+0000","last_active":"2024-03-29T08:27:19.599013+0000","last_peered":"2024-03-29T08:27:19.599013+0000","last_clean":"2024-03-29T08:27:19.599013+0000","last_became_active":"2024-03-29T08:27:18.208953+0000","last_became_peered":"2024-03-29T08:27:18.208953+0000","last_unstale":"2024-03-29T08:27:19.599013+0000","last_undegraded":"2024-03-29T08:27:19.599013+0000","last_fullsized":"2024-03-29T08:27:19.599013+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:13:20.157128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593492+0000","last_change":"2024-03-29T08:27:18.618710+0000","last_active":"2024-03-29T08:27:19.593492+0000","last_peered":"2024-03-29T08:27:19.593492+0000","last_clean":"2024-03-29T08:27:19.593492+0000","last_became_active":"2024-03-29T08:27:18.618465+0000","last_became_peered":"2024-03-29T08:27:18.618465+0000","last_unstale":"2024-03-29T08:27:19.593492+0000","last_undegraded":"2024-03-29T08:27:19.593492+0000","last_fullsized":"2024-03-29T08:27:19.593492+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:16:57.103209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946290+0000","last_change":"2024-03-29T08:27:18.188109+0000","last_active":"2024-03-29T08:27:19.946290+0000","last_peered":"2024-03-29T08:27:19.946290+0000","last_clean":"2024-03-29T08:27:19.946290+0000","last_became_active":"2024-03-29T08:27:18.187843+0000","last_became_peered":"2024-03-29T08:27:18.187843+0000","last_unstale":"2024-03-29T08:27:19.946290+0000","last_undegraded":"2024-03-29T08:27:19.946290+0000","last_fullsized":"2024-03-29T08:27:19.946290+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:23:22.947938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.620121+0000","last_change":"2024-03-29T08:27:18.613882+0000","last_active":"2024-03-29T08:27:18.620121+0000","last_peered":"2024-03-29T08:27:18.620121+0000","last_clean":"2024-03-29T08:27:18.620121+0000","last_became_active":"2024-03-29T08:27:18.610873+0000","last_became_peered":"2024-03-29T08:27:18.610873+0000","last_unstale":"2024-03-29T08:27:18.620121+0000","last_undegraded":"2024-03-29T08:27:18.620121+0000","last_fullsized":"2024-03-29T08:27:18.620121+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:27:04.811546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951693+0000","last_change":"2024-03-29T08:27:18.202858+0000","last_active":"2024-03-29T08:27:19.951693+0000","last_peered":"2024-03-29T08:27:19.951693+0000","last_clean":"2024-03-29T08:27:19.951693+0000","last_became_active":"2024-03-29T08:27:18.202610+0000","last_became_peered":"2024-03-29T08:27:18.202610+0000","last_unstale":"2024-03-29T08:27:19.951693+0000","last_undegraded":"2024-03-29T08:27:19.951693+0000","last_fullsized":"2024-03-29T08:27:19.951693+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:49:09.785927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946300+0000","last_change":"2024-03-29T08:27:18.608958+0000","last_active":"2024-03-29T08:27:19.946300+0000","last_peered":"2024-03-29T08:27:19.946300+0000","last_clean":"2024-03-29T08:27:19.946300+0000","last_became_active":"2024-03-29T08:27:18.605287+0000","last_became_peered":"2024-03-29T08:27:18.605287+0000","last_unstale":"2024-03-29T08:27:19.946300+0000","last_undegraded":"2024-03-29T08:27:19.946300+0000","last_fullsized":"2024-03-29T08:27:19.946300+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:09:15.749616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598538+0000","last_change":"2024-03-29T08:27:18.209857+0000","last_active":"2024-03-29T08:27:19.598538+0000","last_peered":"2024-03-29T08:27:19.598538+0000","last_clean":"2024-03-29T08:27:19.598538+0000","last_became_active":"2024-03-29T08:27:18.208138+0000","last_became_peered":"2024-03-29T08:27:18.208138+0000","last_unstale":"2024-03-29T08:27:19.598538+0000","last_undegraded":"2024-03-29T08:27:19.598538+0000","last_fullsized":"2024-03-29T08:27:19.598538+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:07:08.533818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.599077+0000","last_change":"2024-03-29T08:27:18.625750+0000","last_active":"2024-03-29T08:27:19.599077+0000","last_peered":"2024-03-29T08:27:19.599077+0000","last_clean":"2024-03-29T08:27:19.599077+0000","last_became_active":"2024-03-29T08:27:18.625507+0000","last_became_peered":"2024-03-29T08:27:18.625507+0000","last_unstale":"2024-03-29T08:27:19.599077+0000","last_undegraded":"2024-03-29T08:27:19.599077+0000","last_fullsized":"2024-03-29T08:27:19.599077+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:50:18.987192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"105'1","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946646+0000","last_change":"2024-03-29T08:27:18.194454+0000","last_active":"2024-03-29T08:27:19.946646+0000","last_peered":"2024-03-29T08:27:19.946646+0000","last_clean":"2024-03-29T08:27:19.946646+0000","last_became_active":"2024-03-29T08:27:18.194117+0000","last_became_peered":"2024-03-29T08:27:18.194117+0000","last_unstale":"2024-03-29T08:27:19.946646+0000","last_undegraded":"2024-03-29T08:27:19.946646+0000","last_fullsized":"2024-03-29T08:27:19.946646+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:32:14.427500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":236,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.591533+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:27:18.591533+0000","last_peered":"2024-03-29T08:27:18.591533+0000","last_clean":"2024-03-29T08:27:18.591533+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:27:18.591533+0000","last_undegraded":"2024-03-29T08:27:18.591533+0000","last_fullsized":"2024-03-29T08:27:18.591533+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509555,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1306624,"data_stored":563785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.72799999999999998,"15min":0.68999999999999995},"min":{"1min":0.52600000000000002,"5min":0.39200000000000002,"15min":0.34999999999999998},"max":{"1min":0.93300000000000005,"5min":0.97999999999999998,"15min":1.1639999999999999},"last":0.626},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.753,"15min":0.71299999999999997},"min":{"1min":0.624,"5min":0.55100000000000005,"15min":0.45800000000000002},"max":{"1min":0.92900000000000005,"5min":0.998,"15min":1.2410000000000001},"last":0.72399999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69599999999999995,"15min":0.67300000000000004},"min":{"1min":0.499,"5min":0.443,"15min":0.39600000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":1.202},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69599999999999995,"15min":0.66900000000000004},"min":{"1min":0.55300000000000005,"5min":0.44,"15min":0.44},"max":{"1min":0.89000000000000001,"5min":0.93400000000000005,"15min":1.1339999999999999},"last":0.67000000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69399999999999995,"15min":0.66400000000000003},"min":{"1min":0.50700000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86499999999999999,"5min":0.93999999999999995,"15min":1.085},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73199999999999998,"15min":0.69899999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.288},"last":0.65500000000000003}]},{"osd":3,"last update":"Fri Mar 29 08:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.71799999999999997,"15min":0.70899999999999996},"min":{"1min":0.51100000000000001,"5min":0.496,"15min":0.46000000000000002},"max":{"1min":0.90500000000000003,"5min":0.98099999999999998,"15min":1.018},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.73399999999999999,"15min":0.73499999999999999},"min":{"1min":0.59499999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90300000000000002,"5min":0.94599999999999995,"15min":1.2609999999999999},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Mar 29 08:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57699999999999996,"15min":0.60999999999999999},"min":{"1min":0.41499999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84799999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.67100000000000004},"min":{"1min":0.434,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79900000000000004,"5min":0.89600000000000002,"15min":1.123},"last":0.49199999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61299999999999999,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":1.056},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63400000000000001,"15min":0.67500000000000004},"min":{"1min":0.48799999999999999,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.83899999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.60999999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.68999999999999995,"15min":0.71499999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46600000000000003},"max":{"1min":0.96399999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.71199999999999997},"min":{"1min":0.47899999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.98299999999999998},"last":0.57899999999999996}]}]},{"osd":6,"up_from":15,"seq":64424509555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1302528,"data_stored":563203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71999999999999997,"15min":0.67200000000000004},"min":{"1min":0.51800000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.90700000000000003,"5min":0.93100000000000005,"15min":1.0189999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.751,"15min":0.71599999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.44500000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.079},"last":0.89800000000000002}]},{"osd":1,"last update":"Fri Mar 29 08:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.67900000000000005,"15min":0.64400000000000002},"min":{"1min":0.52800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86199999999999999,"5min":0.97999999999999998,"15min":0.99299999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.70199999999999996,"15min":0.66100000000000003},"min":{"1min":0.54900000000000004,"5min":0.442,"15min":0.42899999999999999},"max":{"1min":0.92800000000000005,"5min":0.95599999999999996,"15min":1.079},"last":0.92000000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76800000000000002,"15min":0.71799999999999997},"min":{"1min":0.56999999999999995,"5min":0.51800000000000002,"15min":0.44800000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.75,"15min":0.72399999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.435},"max":{"1min":0.94899999999999995,"5min":0.95099999999999996,"15min":0.95899999999999996},"last":0.80500000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72099999999999997,"15min":0.70799999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.51600000000000001,"5min":0.51000000000000001,"15min":0.47799999999999998},"max":{"1min":0.94199999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.94199999999999995}]},{"osd":4,"last update":"Fri Mar 29 08:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61899999999999999,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.556},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.63800000000000001,"15min":0.68799999999999994},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.371},"max":{"1min":0.96299999999999997,"5min":0.996,"15min":2.0539999999999998},"last":0.71699999999999997}]},{"osd":5,"last update":"Fri Mar 29 08:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63,"15min":0.66700000000000004},"min":{"1min":0.39000000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":1.0660000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68899999999999995,"15min":0.70799999999999996},"min":{"1min":0.50800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.042,"5min":1.042,"15min":1.1419999999999999},"last":1.042}]},{"osd":7,"last update":"Fri Mar 29 08:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69999999999999996,"15min":0.73199999999999998},"min":{"1min":0.46800000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":1.03},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69199999999999995,"15min":0.73699999999999999},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.1160000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509554,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1306624,"data_stored":563785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70999999999999996,"15min":0.70599999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.94799999999999995,"5min":1.0289999999999999,"15min":2.1709999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68700000000000006,"15min":0.71999999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.93000000000000005,"5min":0.95699999999999996,"15min":2.2869999999999999},"last":0.997}]},{"osd":1,"last update":"Fri Mar 29 08:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73899999999999999,"15min":0.753},"min":{"1min":0.48699999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.871,"5min":1.042,"15min":2.524},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69399999999999995,"15min":0.70899999999999996},"min":{"1min":0.441,"5min":0.41099999999999998,"15min":0.40699999999999997},"max":{"1min":0.90000000000000002,"5min":0.95599999999999996,"15min":2.2679999999999998},"last":1.3149999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70499999999999996,"15min":0.72199999999999998},"min":{"1min":0.45700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91700000000000004,"5min":0.97799999999999998,"15min":2.363},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73499999999999999,"15min":0.751},"min":{"1min":0.45600000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.94499999999999995,"5min":0.97799999999999998,"15min":2.1880000000000002},"last":0.88700000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.748,"15min":0.751},"min":{"1min":0.53400000000000003,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":0.96499999999999997,"5min":1.004,"15min":2.3029999999999999},"last":1.3759999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77700000000000002,"15min":0.79700000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96799999999999997,"5min":1.0549999999999999,"15min":2.2549999999999999},"last":1.016}]},{"osd":5,"last update":"Fri Mar 29 08:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.624,"15min":0.66600000000000004},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.437,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85499999999999998,"5min":0.88700000000000001,"15min":1.0149999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Mar 29 08:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65300000000000002,"15min":0.66200000000000003},"min":{"1min":0.42599999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85299999999999998,"5min":1.091,"15min":1.091},"last":0.629},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70399999999999996,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.499,"15min":0.499},"max":{"1min":0.879,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.66500000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69299999999999995,"15min":0.71599999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91800000000000004,"5min":0.95899999999999996,"15min":1.1000000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.751},"min":{"1min":0.47999999999999998,"5min":0.44400000000000001,"15min":0.42899999999999999},"max":{"1min":1.036,"5min":1.0760000000000001,"15min":1.4430000000000001},"last":0.83799999999999997}]}]},{"osd":3,"up_from":15,"seq":64424509557,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1314816,"data_stored":563397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.44500000000000001,"15min":0.45600000000000002},"min":{"1min":0.316,"5min":0.26600000000000001,"15min":0.26200000000000001},"max":{"1min":0.61299999999999999,"5min":0.84799999999999998,"15min":1.123},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.38400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.751,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri Mar 29 08:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45000000000000001,"15min":0.45500000000000002},"min":{"1min":0.32000000000000001,"5min":0.30199999999999999,"15min":0.26500000000000001},"max":{"1min":0.63,"5min":0.78900000000000003,"15min":1.1599999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.52400000000000002},"min":{"1min":0.41299999999999998,"5min":0.32100000000000001,"15min":0.28599999999999998},"max":{"1min":0.78500000000000003,"5min":1.004,"15min":1.4790000000000001},"last":0.41299999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53600000000000003,"15min":0.55500000000000005},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.309},"max":{"1min":0.80300000000000005,"5min":0.90000000000000002,"15min":1.2529999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60099999999999998,"15min":0.63600000000000001},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.41799999999999998},"max":{"1min":0.83599999999999997,"5min":0.97699999999999998,"15min":1.8580000000000001},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76500000000000001,"15min":0.78200000000000003},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.42199999999999999},"max":{"1min":0.93999999999999995,"5min":1.0449999999999999,"15min":1.6419999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76800000000000002,"15min":0.77200000000000002},"min":{"1min":0.56599999999999995,"5min":0.46999999999999997,"15min":0.45400000000000001},"max":{"1min":0.96699999999999997,"5min":1.3340000000000001,"15min":1.391},"last":0.67600000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80700000000000005,"15min":0.80500000000000005},"min":{"1min":0.66400000000000003,"5min":0.56999999999999995,"15min":0.52100000000000002},"max":{"1min":0.97399999999999998,"5min":1.1850000000000001,"15min":1.671},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51400000000000001},"max":{"1min":1.016,"5min":1.2050000000000001,"15min":1.7110000000000001},"last":0.73499999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78100000000000003,"15min":0.81100000000000005},"min":{"1min":0.60699999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96099999999999997,"5min":1.2609999999999999,"15min":1.669},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.80700000000000005},"min":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0509999999999999,"5min":1.119,"15min":2.0150000000000001},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79900000000000004,"15min":0.83699999999999997},"min":{"1min":0.55900000000000005,"5min":0.55000000000000004,"15min":0.48799999999999999},"max":{"1min":1.073,"5min":1.3169999999999999,"15min":1.887},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.79500000000000004,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.52200000000000002,"15min":0.47999999999999998},"max":{"1min":0.999,"5min":1.1259999999999999,"15min":2.044},"last":0.78900000000000003}]}]},{"osd":2,"up_from":15,"seq":64424509556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1294336,"data_stored":556973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46200000000000002,"15min":0.48199999999999998},"min":{"1min":0.25800000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59599999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55000000000000004,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.34699999999999998,"15min":0.32300000000000001},"max":{"1min":0.72599999999999998,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.46600000000000003}]},{"osd":1,"last update":"Fri Mar 29 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.65000000000000002,"5min":1.399,"15min":1.399},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.51900000000000002,"15min":0.50800000000000001},"min":{"1min":0.32700000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70399999999999996,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.58499999999999996}]},{"osd":3,"last update":"Fri Mar 29 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53900000000000003,"15min":0.51300000000000001},"min":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.34000000000000002},"max":{"1min":0.76000000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.625,"15min":0.59499999999999997},"min":{"1min":0.47299999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81000000000000005,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.54000000000000004}]},{"osd":4,"last update":"Fri Mar 29 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79700000000000004,"15min":0.77700000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.49099999999999999},"max":{"1min":1.109,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.77100000000000002,"15min":0.76500000000000001},"min":{"1min":0.59199999999999997,"5min":0.51700000000000002,"15min":0.435},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.76600000000000001}]},{"osd":5,"last update":"Fri Mar 29 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.84999999999999998,"15min":0.84199999999999997},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.55500000000000005},"max":{"1min":1.0149999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.628},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.81499999999999995,"15min":0.83599999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.48799999999999999},"max":{"1min":1.1759999999999999,"5min":1.194,"15min":1.526},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.78800000000000003,"15min":0.78100000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.48599999999999999},"max":{"1min":1.131,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.82799999999999996,"15min":0.79900000000000004},"min":{"1min":0.64100000000000001,"5min":0.57799999999999996,"15min":0.52500000000000002},"max":{"1min":1.0640000000000001,"5min":1.524,"15min":1.524},"last":0.69999999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.79800000000000004,"15min":0.80200000000000005},"min":{"1min":0.56799999999999995,"5min":0.54900000000000004,"15min":0.505},"max":{"1min":1.1259999999999999,"5min":1.296,"15min":1.319},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81499999999999995,"15min":0.79900000000000004},"min":{"1min":0.58999999999999997,"5min":0.56299999999999994,"15min":0.54900000000000004},"max":{"1min":1.0860000000000001,"5min":2,"15min":2},"last":0.57399999999999995}]}]},{"osd":1,"up_from":15,"seq":64424509556,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32980,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338860,"statfs":{"total":96636764160,"available":96602992640,"internally_reserved":0,"allocated":1904640,"data_stored":1148947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44900000000000001,"15min":0.54700000000000004},"min":{"1min":0.31900000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.61499999999999999,"5min":0.75900000000000001,"15min":4.2539999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48399999999999999,"15min":0.57799999999999996},"min":{"1min":0.317,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67900000000000005,"5min":0.73299999999999998,"15min":4.1909999999999998},"last":0.317}]},{"osd":2,"last update":"Fri Mar 29 08:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.501,"15min":0.502},"min":{"1min":0.33400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67900000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53000000000000003,"15min":0.53200000000000003},"min":{"1min":0.42699999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67800000000000005,"5min":0.79000000000000004,"15min":0.88200000000000001},"last":0.54800000000000004}]},{"osd":3,"last update":"Fri Mar 29 08:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88300000000000001,"5min":1.52,"15min":1.52},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.374,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.64900000000000002,"5min":1.331,"15min":1.331},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78000000000000003,"15min":0.753},"min":{"1min":0.52400000000000002,"5min":0.51800000000000002,"15min":0.46200000000000002},"max":{"1min":0.86599999999999999,"5min":1.117,"15min":1.117},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.77400000000000002,"15min":0.77700000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.47999999999999998},"max":{"1min":1,"5min":1.175,"15min":1.23},"last":0.58099999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78800000000000003,"15min":0.77900000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.94799999999999995,"5min":1.1339999999999999,"15min":1.1499999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78700000000000003,"15min":0.76700000000000002},"min":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.46500000000000002},"max":{"1min":0.94599999999999995,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.65600000000000003}]},{"osd":6,"last update":"Fri Mar 29 08:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77800000000000002,"15min":0.77000000000000002},"min":{"1min":0.54900000000000004,"5min":0.505,"15min":0.5},"max":{"1min":0.90100000000000002,"5min":1.212,"15min":1.212},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80800000000000005,"15min":0.82999999999999996},"min":{"1min":0.56399999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91900000000000004,"5min":1.22,"15min":1.22},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80300000000000005,"15min":0.80000000000000004},"min":{"1min":0.60299999999999998,"5min":0.56100000000000005,"15min":0.52100000000000002},"max":{"1min":0.93000000000000005,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79900000000000004,"15min":0.78900000000000003},"min":{"1min":0.59499999999999997,"5min":0.503,"15min":0.503},"max":{"1min":0.97699999999999998,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.74099999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509555,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1302528,"data_stored":563203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:27:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.44800000000000001,"15min":0.53300000000000003},"min":{"1min":0.30299999999999999,"5min":0.28499999999999998,"15min":0.22800000000000001},"max":{"1min":0.70499999999999996,"5min":0.75800000000000001,"15min":3.9359999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.49299999999999999,"15min":0.69699999999999995},"min":{"1min":0.34799999999999998,"5min":0.28499999999999998,"15min":0.253},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":8.0419999999999998},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:27:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51400000000000001,"15min":0.53200000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34599999999999997},"max":{"1min":0.746,"5min":0.78000000000000003,"15min":1.258},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.65100000000000002},"min":{"1min":0.41399999999999998,"5min":0.32800000000000001,"15min":0.312},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":4.907},"last":0.40300000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:27:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.504,"15min":0.69999999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.28499999999999998},"max":{"1min":0.71899999999999997,"5min":0.83399999999999996,"15min":7.9569999999999999},"last":0.5},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57499999999999996,"15min":0.621},"min":{"1min":0.41899999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.87,"5min":0.87,"15min":1.397},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Mar 29 08:27:13 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.78800000000000003,"15min":0.90100000000000002},"min":{"1min":0.64700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.98299999999999998,"5min":1.4399999999999999,"15min":4.4720000000000004},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78200000000000003,"15min":0.88400000000000001},"min":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.153,"5min":1.153,"15min":4.3920000000000003},"last":0.65000000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:27:13 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.78500000000000003,"15min":0.86699999999999999},"min":{"1min":0.61299999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.038,"5min":1.349,"15min":4.3200000000000003},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79600000000000004,"15min":0.83099999999999996},"min":{"1min":0.58799999999999997,"5min":0.55700000000000005,"15min":0.54600000000000004},"max":{"1min":1.079,"5min":1.26,"15min":1.6759999999999999},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Mar 29 08:27:13 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78900000000000003,"15min":0.83199999999999996},"min":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.54400000000000004},"max":{"1min":1.006,"5min":1.1579999999999999,"15min":1.5920000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.80900000000000005,"15min":0.90800000000000003},"min":{"1min":0.60099999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0569999999999999,"5min":1.1990000000000001,"15min":4.4100000000000001},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Mar 29 08:27:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81100000000000005,"15min":0.92200000000000004},"min":{"1min":0.67000000000000004,"5min":0.54100000000000004,"15min":0.496},"max":{"1min":1.119,"5min":1.3959999999999999,"15min":4.5209999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.80100000000000005,"15min":0.90400000000000003},"min":{"1min":0.55400000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1000000000000001,"5min":1.2390000000000001,"15min":4.4420000000000002},"last":0.79700000000000004}]}]},{"osd":5,"up_from":14,"seq":60129542260,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32980,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338860,"statfs":{"total":96636764160,"available":96602992640,"internally_reserved":0,"allocated":1904640,"data_stored":1148891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69399999999999995,"15min":0.65300000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.88100000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.67400000000000004},"min":{"1min":0.56699999999999995,"5min":0.46200000000000002,"15min":0.39800000000000002},"max":{"1min":0.91500000000000004,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.79400000000000004}]},{"osd":1,"last update":"Fri Mar 29 08:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68400000000000005,"15min":0.71599999999999997},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88,"5min":0.90200000000000002,"15min":2.5129999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72099999999999997,"15min":0.69199999999999995},"min":{"1min":0.51300000000000001,"5min":0.47199999999999998,"15min":0.40500000000000003},"max":{"1min":0.89600000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.875}]},{"osd":2,"last update":"Fri Mar 29 08:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.42499999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71599999999999997,"15min":0.70799999999999996},"min":{"1min":0.47499999999999998,"5min":0.44900000000000001,"15min":0.39200000000000002},"max":{"1min":0.92900000000000005,"5min":1.002,"15min":1.0209999999999999},"last":0.90300000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.63300000000000001,"5min":0.49399999999999999,"15min":0.47099999999999997},"max":{"1min":0.874,"5min":1.038,"15min":1.038},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72099999999999997,"15min":0.73899999999999999},"min":{"1min":0.60299999999999998,"5min":0.47799999999999998,"15min":0.435},"max":{"1min":0.95099999999999996,"5min":1.0209999999999999,"15min":1.954},"last":0.83399999999999996}]},{"osd":4,"last update":"Fri Mar 29 08:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.61899999999999999,"15min":0.64200000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.93500000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61899999999999999,"15min":0.64700000000000002},"min":{"1min":0.41799999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.89200000000000002,"5min":0.94699999999999995,"15min":1.0820000000000001},"last":0.65500000000000003}]},{"osd":6,"last update":"Fri Mar 29 08:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.623,"15min":0.66000000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":1.071},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66600000000000004,"15min":0.68799999999999994},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.432},"max":{"1min":0.90800000000000003,"5min":0.91100000000000003,"15min":1.0609999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66800000000000004,"15min":0.69999999999999996},"min":{"1min":0.46000000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.1200000000000001},"last":0.754},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70999999999999996,"15min":0.73899999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.04},"last":0.72099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:27:22.691 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:27:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.863+0000 7febfbe00640 1 Processor -- start 2024-03-29T08:27:22.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.863+0000 7febfbe00640 1 -- start start 2024-03-29T08:27:22.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.867+0000 7febfbe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7febf41593d0 0x7febf406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.867+0000 7febfbe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7febf415d230 0x7febf415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.867+0000 7febfbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febf415dbd0 0x7febf415bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.867+0000 7febfbe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf406d130 con 0x7febf415dbd0 2024-03-29T08:27:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.867+0000 7febfbe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf415b8f0 con 0x7febf415d230 2024-03-29T08:27:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.867+0000 7febfbe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf415ad50 con 0x7febf41593d0 2024-03-29T08:27:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.867+0000 7febf9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febf415dbd0 0x7febf415bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.867+0000 7febf9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febf415dbd0 0x7febf415bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33676/0 (socket says 172.21.15.46:33676) 2024-03-29T08:27:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.867+0000 7febf9e00640 1 -- 172.21.15.46:0/47334599 learned_addr learned my addr 172.21.15.46:0/47334599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.867+0000 7febf8a00640 1 --2- 172.21.15.46:0/47334599 >> v2:172.21.15.73:3300/0 conn(0x7febf415d230 0x7febf415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.867+0000 7febf9400640 1 --2- 172.21.15.46:0/47334599 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7febf41593d0 0x7febf406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.867+0000 7febf9e00640 1 -- 172.21.15.46:0/47334599 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7febf41593d0 msgr2=0x7febf406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.867+0000 7febf9e00640 1 --2- 172.21.15.46:0/47334599 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7febf41593d0 0x7febf406cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.867+0000 7febf9e00640 1 -- 172.21.15.46:0/47334599 >> v2:172.21.15.73:3300/0 conn(0x7febf415d230 msgr2=0x7febf415d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.867+0000 7febf9e00640 1 --2- 172.21.15.46:0/47334599 >> v2:172.21.15.73:3300/0 conn(0x7febf415d230 0x7febf415d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.867+0000 7febf9e00640 1 -- 172.21.15.46:0/47334599 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf415e060 con 0x7febf415dbd0 2024-03-29T08:27:22.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.867+0000 7febf9400640 1 --2- 172.21.15.46:0/47334599 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7febf41593d0 0x7febf406cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:22.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.871+0000 7febf9e00640 1 --2- 172.21.15.46:0/47334599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf415dbd0 0x7febf415bd80 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7febf4071520 tx=0x7febec01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5513174f5faab4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:22.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.871+0000 7febf3e00640 1 -- 172.21.15.46:0/47334599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec0aa640 con 0x7febf415dbd0 2024-03-29T08:27:22.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.871+0000 7febf3e00640 1 -- 172.21.15.46:0/47334599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec0aa7a0 con 0x7febf415dbd0 2024-03-29T08:27:22.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.871+0000 7febf3e00640 1 -- 172.21.15.46:0/47334599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec0aab30 con 0x7febf415dbd0 2024-03-29T08:27:22.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.871+0000 7febfbe00640 1 -- 172.21.15.46:0/47334599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf415dbd0 msgr2=0x7febf415bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:22.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.871+0000 7febfbe00640 1 --2- 172.21.15.46:0/47334599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf415dbd0 0x7febf415bd80 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7febf4071520 tx=0x7febec01c520 comp rx=0 tx=0).stop 2024-03-29T08:27:22.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.871+0000 7febfbe00640 1 -- 172.21.15.46:0/47334599 shutdown_connections 2024-03-29T08:27:22.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.871+0000 7febfbe00640 1 --2- 172.21.15.46:0/47334599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf415dbd0 0x7febf415bd80 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.871+0000 7febfbe00640 1 --2- 172.21.15.46:0/47334599 >> v2:172.21.15.73:3300/0 conn(0x7febf415d230 0x7febf415d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.871+0000 7febfbe00640 1 --2- 172.21.15.46:0/47334599 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7febf41593d0 0x7febf406cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.871+0000 7febfbe00640 1 -- 172.21.15.46:0/47334599 >> 172.21.15.46:0/47334599 conn(0x7febf4066110 msgr2=0x7febf4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.871+0000 7febfbe00640 1 -- 172.21.15.46:0/47334599 shutdown_connections 2024-03-29T08:27:22.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.871+0000 7febfbe00640 1 -- 172.21.15.46:0/47334599 wait complete. 2024-03-29T08:27:22.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.871+0000 7febfbe00640 1 Processor -- start 2024-03-29T08:27:22.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.871+0000 7febfbe00640 1 -- start start 2024-03-29T08:27:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febfbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf41593d0 0x7febf4163bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febfbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febf415d230 0x7febf4151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febfbe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7febf415dbd0 0x7febf41520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febfbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf41f57c0 con 0x7febf41593d0 2024-03-29T08:27:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febfbe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf4152610 con 0x7febf415d230 2024-03-29T08:27:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febfbe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf41f4e90 con 0x7febf415dbd0 2024-03-29T08:27:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febf9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf41593d0 0x7febf4163bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febf9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf41593d0 0x7febf4163bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33684/0 (socket says 172.21.15.46:33684) 2024-03-29T08:27:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febf9400640 1 -- 172.21.15.46:0/2995172260 learned_addr learned my addr 172.21.15.46:0/2995172260 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febf9e00640 1 --2- 172.21.15.46:0/2995172260 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7febf415dbd0 0x7febf41520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febf8a00640 1 --2- 172.21.15.46:0/2995172260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febf415d230 0x7febf4151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febf9400640 1 -- 172.21.15.46:0/2995172260 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7febf415dbd0 msgr2=0x7febf41520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:22.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febf9400640 1 --2- 172.21.15.46:0/2995172260 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7febf415dbd0 0x7febf41520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febf9400640 1 -- 172.21.15.46:0/2995172260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febf415d230 msgr2=0x7febf4151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:22.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febf9400640 1 --2- 172.21.15.46:0/2995172260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febf415d230 0x7febf4151b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:22.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febf9400640 1 -- 172.21.15.46:0/2995172260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febec0094d0 con 0x7febf41593d0 2024-03-29T08:27:22.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febf9400640 1 --2- 172.21.15.46:0/2995172260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf41593d0 0x7febf4163bf0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7febe8007e40 tx=0x7febe800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:22.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febf1600640 1 -- 172.21.15.46:0/2995172260 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8003ba0 con 0x7febf41593d0 2024-03-29T08:27:22.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febf1600640 1 -- 172.21.15.46:0/2995172260 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe800f040 con 0x7febf41593d0 2024-03-29T08:27:22.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febfbe00640 1 -- 172.21.15.46:0/2995172260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf41f5100 con 0x7febf41593d0 2024-03-29T08:27:22.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febf1600640 1 -- 172.21.15.46:0/2995172260 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe800de80 con 0x7febf41593d0 2024-03-29T08:27:22.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febfbe00640 1 -- 172.21.15.46:0/2995172260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf41f5490 con 0x7febf41593d0 2024-03-29T08:27:22.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febfbe00640 1 -- 172.21.15.46:0/2995172260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf41f5a30 con 0x7febf41593d0 2024-03-29T08:27:22.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.875+0000 7febf1600640 1 -- 172.21.15.46:0/2995172260 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7febe8004030 con 0x7febf41593d0 2024-03-29T08:27:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.879+0000 7febf1600640 1 --2- 172.21.15.46:0/2995172260 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7febd4066690 0x7febd4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.879+0000 7febf1600640 1 -- 172.21.15.46:0/2995172260 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febe80843a0 con 0x7febf41593d0 2024-03-29T08:27:22.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.879+0000 7febf8a00640 1 --2- 172.21.15.46:0/2995172260 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7febd4066690 0x7febd4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:22.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.879+0000 7febf8a00640 1 --2- 172.21.15.46:0/2995172260 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7febd4066690 0x7febd4068b50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7febf4155150 tx=0x7febe000b3a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:22.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:22.883+0000 7febf1600640 1 -- 172.21.15.46:0/2995172260 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7febe80517b0 con 0x7febf41593d0 2024-03-29T08:27:23.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.039+0000 7febfbe00640 1 -- 172.21.15.46:0/2995172260 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7febf4066c90 con 0x7febd4066690 2024-03-29T08:27:23.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.047+0000 7febf1600640 1 -- 172.21.15.46:0/2995172260 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289770 (secure 0 0 0) 0x7febf4066c90 con 0x7febd4066690 2024-03-29T08:27:23.053 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:23.055 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:27:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.051+0000 7febfbe00640 1 -- 172.21.15.46:0/2995172260 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7febd4066690 msgr2=0x7febd4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.051+0000 7febfbe00640 1 --2- 172.21.15.46:0/2995172260 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7febd4066690 0x7febd4068b50 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7febf4155150 tx=0x7febe000b3a0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.051+0000 7febfbe00640 1 -- 172.21.15.46:0/2995172260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf41593d0 msgr2=0x7febf4163bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:23.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.051+0000 7febfbe00640 1 --2- 172.21.15.46:0/2995172260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf41593d0 0x7febf4163bf0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7febe8007e40 tx=0x7febe800d4a0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.051+0000 7febfbe00640 1 -- 172.21.15.46:0/2995172260 shutdown_connections 2024-03-29T08:27:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.051+0000 7febfbe00640 1 --2- 172.21.15.46:0/2995172260 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7febd4066690 0x7febd4068b50 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.051+0000 7febfbe00640 1 --2- 172.21.15.46:0/2995172260 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febf415d230 0x7febf4151b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.051+0000 7febfbe00640 1 --2- 172.21.15.46:0/2995172260 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7febf415dbd0 0x7febf41520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.051+0000 7febfbe00640 1 --2- 172.21.15.46:0/2995172260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febf41593d0 0x7febf4163bf0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.051+0000 7febfbe00640 1 -- 172.21.15.46:0/2995172260 >> 172.21.15.46:0/2995172260 conn(0x7febf4066110 msgr2=0x7febf40b1730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.051+0000 7febfbe00640 1 -- 172.21.15.46:0/2995172260 shutdown_connections 2024-03-29T08:27:23.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.051+0000 7febfbe00640 1 -- 172.21.15.46:0/2995172260 wait complete. 2024-03-29T08:27:23.074 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-03-29T08:27:22.182703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260352,"kb_used_data":11364,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754714368,"statfs":{"total":773094113280,"available":772827512832,"internally_reserved":0,"allocated":11636736,"data_stored":5672184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.005890"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952281+0000","last_change":"2024-03-29T08:27:18.623414+0000","last_active":"2024-03-29T08:27:19.952281+0000","last_peered":"2024-03-29T08:27:19.952281+0000","last_clean":"2024-03-29T08:27:19.952281+0000","last_became_active":"2024-03-29T08:27:18.622981+0000","last_became_peered":"2024-03-29T08:27:18.622981+0000","last_unstale":"2024-03-29T08:27:19.952281+0000","last_undegraded":"2024-03-29T08:27:19.952281+0000","last_fullsized":"2024-03-29T08:27:19.952281+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:36.631872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951993+0000","last_change":"2024-03-29T08:27:18.625651+0000","last_active":"2024-03-29T08:27:19.951993+0000","last_peered":"2024-03-29T08:27:19.951993+0000","last_clean":"2024-03-29T08:27:19.951993+0000","last_became_active":"2024-03-29T08:27:18.624134+0000","last_became_peered":"2024-03-29T08:27:18.624134+0000","last_unstale":"2024-03-29T08:27:19.951993+0000","last_undegraded":"2024-03-29T08:27:19.951993+0000","last_fullsized":"2024-03-29T08:27:19.951993+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:17:30.862649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598970+0000","last_change":"2024-03-29T08:27:18.621776+0000","last_active":"2024-03-29T08:27:19.598970+0000","last_peered":"2024-03-29T08:27:19.598970+0000","last_clean":"2024-03-29T08:27:19.598970+0000","last_became_active":"2024-03-29T08:27:18.621465+0000","last_became_peered":"2024-03-29T08:27:18.621465+0000","last_unstale":"2024-03-29T08:27:19.598970+0000","last_undegraded":"2024-03-29T08:27:19.598970+0000","last_fullsized":"2024-03-29T08:27:19.598970+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:23.870278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951605+0000","last_change":"2024-03-29T08:27:18.630007+0000","last_active":"2024-03-29T08:27:19.951605+0000","last_peered":"2024-03-29T08:27:19.951605+0000","last_clean":"2024-03-29T08:27:19.951605+0000","last_became_active":"2024-03-29T08:27:18.628978+0000","last_became_peered":"2024-03-29T08:27:18.628978+0000","last_unstale":"2024-03-29T08:27:19.951605+0000","last_undegraded":"2024-03-29T08:27:19.951605+0000","last_fullsized":"2024-03-29T08:27:19.951605+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:23.386306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593370+0000","last_change":"2024-03-29T08:27:18.605886+0000","last_active":"2024-03-29T08:27:19.593370+0000","last_peered":"2024-03-29T08:27:19.593370+0000","last_clean":"2024-03-29T08:27:19.593370+0000","last_became_active":"2024-03-29T08:27:18.605245+0000","last_became_peered":"2024-03-29T08:27:18.605245+0000","last_unstale":"2024-03-29T08:27:19.593370+0000","last_undegraded":"2024-03-29T08:27:19.593370+0000","last_fullsized":"2024-03-29T08:27:19.593370+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:51:11.165228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593629+0000","last_change":"2024-03-29T08:27:18.612974+0000","last_active":"2024-03-29T08:27:19.593629+0000","last_peered":"2024-03-29T08:27:19.593629+0000","last_clean":"2024-03-29T08:27:19.593629+0000","last_became_active":"2024-03-29T08:27:18.612652+0000","last_became_peered":"2024-03-29T08:27:18.612652+0000","last_unstale":"2024-03-29T08:27:19.593629+0000","last_undegraded":"2024-03-29T08:27:19.593629+0000","last_fullsized":"2024-03-29T08:27:19.593629+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:07:06.162438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593231+0000","last_change":"2024-03-29T08:27:18.606026+0000","last_active":"2024-03-29T08:27:19.593231+0000","last_peered":"2024-03-29T08:27:19.593231+0000","last_clean":"2024-03-29T08:27:19.593231+0000","last_became_active":"2024-03-29T08:27:18.605734+0000","last_became_peered":"2024-03-29T08:27:18.605734+0000","last_unstale":"2024-03-29T08:27:19.593231+0000","last_undegraded":"2024-03-29T08:27:19.593231+0000","last_fullsized":"2024-03-29T08:27:19.593231+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:57:59.583230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598435+0000","last_change":"2024-03-29T08:27:18.626903+0000","last_active":"2024-03-29T08:27:19.598435+0000","last_peered":"2024-03-29T08:27:19.598435+0000","last_clean":"2024-03-29T08:27:19.598435+0000","last_became_active":"2024-03-29T08:27:18.626589+0000","last_became_peered":"2024-03-29T08:27:18.626589+0000","last_unstale":"2024-03-29T08:27:19.598435+0000","last_undegraded":"2024-03-29T08:27:19.598435+0000","last_fullsized":"2024-03-29T08:27:19.598435+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:38:15.048270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593760+0000","last_change":"2024-03-29T08:27:18.617848+0000","last_active":"2024-03-29T08:27:19.593760+0000","last_peered":"2024-03-29T08:27:19.593760+0000","last_clean":"2024-03-29T08:27:19.593760+0000","last_became_active":"2024-03-29T08:27:18.617592+0000","last_became_peered":"2024-03-29T08:27:18.617592+0000","last_unstale":"2024-03-29T08:27:19.593760+0000","last_undegraded":"2024-03-29T08:27:19.593760+0000","last_fullsized":"2024-03-29T08:27:19.593760+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:44:13.762743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.592376+0000","last_change":"2024-03-29T08:27:18.620874+0000","last_active":"2024-03-29T08:27:19.592376+0000","last_peered":"2024-03-29T08:27:19.592376+0000","last_clean":"2024-03-29T08:27:19.592376+0000","last_became_active":"2024-03-29T08:27:18.619900+0000","last_became_peered":"2024-03-29T08:27:18.619900+0000","last_unstale":"2024-03-29T08:27:19.592376+0000","last_undegraded":"2024-03-29T08:27:19.592376+0000","last_fullsized":"2024-03-29T08:27:19.592376+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:35:54.064179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593396+0000","last_change":"2024-03-29T08:27:18.622044+0000","last_active":"2024-03-29T08:27:19.593396+0000","last_peered":"2024-03-29T08:27:19.593396+0000","last_clean":"2024-03-29T08:27:19.593396+0000","last_became_active":"2024-03-29T08:27:18.620960+0000","last_became_peered":"2024-03-29T08:27:18.620960+0000","last_unstale":"2024-03-29T08:27:19.593396+0000","last_undegraded":"2024-03-29T08:27:19.593396+0000","last_fullsized":"2024-03-29T08:27:19.593396+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:05:15.943323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946504+0000","last_change":"2024-03-29T08:27:18.620770+0000","last_active":"2024-03-29T08:27:19.946504+0000","last_peered":"2024-03-29T08:27:19.946504+0000","last_clean":"2024-03-29T08:27:19.946504+0000","last_became_active":"2024-03-29T08:27:18.620514+0000","last_became_peered":"2024-03-29T08:27:18.620514+0000","last_unstale":"2024-03-29T08:27:19.946504+0000","last_undegraded":"2024-03-29T08:27:19.946504+0000","last_fullsized":"2024-03-29T08:27:19.946504+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:30:27.814197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952253+0000","last_change":"2024-03-29T08:27:18.618647+0000","last_active":"2024-03-29T08:27:19.952253+0000","last_peered":"2024-03-29T08:27:19.952253+0000","last_clean":"2024-03-29T08:27:19.952253+0000","last_became_active":"2024-03-29T08:27:18.618391+0000","last_became_peered":"2024-03-29T08:27:18.618391+0000","last_unstale":"2024-03-29T08:27:19.952253+0000","last_undegraded":"2024-03-29T08:27:19.952253+0000","last_fullsized":"2024-03-29T08:27:19.952253+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:27.887597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952337+0000","last_change":"2024-03-29T08:27:18.623490+0000","last_active":"2024-03-29T08:27:19.952337+0000","last_peered":"2024-03-29T08:27:19.952337+0000","last_clean":"2024-03-29T08:27:19.952337+0000","last_became_active":"2024-03-29T08:27:18.623126+0000","last_became_peered":"2024-03-29T08:27:18.623126+0000","last_unstale":"2024-03-29T08:27:19.952337+0000","last_undegraded":"2024-03-29T08:27:19.952337+0000","last_fullsized":"2024-03-29T08:27:19.952337+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:23:47.930784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951385+0000","last_change":"2024-03-29T08:27:18.614993+0000","last_active":"2024-03-29T08:27:19.951385+0000","last_peered":"2024-03-29T08:27:19.951385+0000","last_clean":"2024-03-29T08:27:19.951385+0000","last_became_active":"2024-03-29T08:27:18.614734+0000","last_became_peered":"2024-03-29T08:27:18.614734+0000","last_unstale":"2024-03-29T08:27:19.951385+0000","last_undegraded":"2024-03-29T08:27:19.951385+0000","last_fullsized":"2024-03-29T08:27:19.951385+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:38:15.048270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593235+0000","last_change":"2024-03-29T08:27:18.607097+0000","last_active":"2024-03-29T08:27:19.593235+0000","last_peered":"2024-03-29T08:27:19.593235+0000","last_clean":"2024-03-29T08:27:19.593235+0000","last_became_active":"2024-03-29T08:27:18.606339+0000","last_became_peered":"2024-03-29T08:27:18.606339+0000","last_unstale":"2024-03-29T08:27:19.593235+0000","last_undegraded":"2024-03-29T08:27:19.593235+0000","last_fullsized":"2024-03-29T08:27:19.593235+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:33:34.629131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.613850+0000","last_change":"2024-03-29T08:27:18.609933+0000","last_active":"2024-03-29T08:27:18.613850+0000","last_peered":"2024-03-29T08:27:18.613850+0000","last_clean":"2024-03-29T08:27:18.613850+0000","last_became_active":"2024-03-29T08:27:18.609689+0000","last_became_peered":"2024-03-29T08:27:18.609689+0000","last_unstale":"2024-03-29T08:27:18.613850+0000","last_undegraded":"2024-03-29T08:27:18.613850+0000","last_fullsized":"2024-03-29T08:27:18.613850+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:26:04.517467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952043+0000","last_change":"2024-03-29T08:27:18.627100+0000","last_active":"2024-03-29T08:27:19.952043+0000","last_peered":"2024-03-29T08:27:19.952043+0000","last_clean":"2024-03-29T08:27:19.952043+0000","last_became_active":"2024-03-29T08:27:18.624833+0000","last_became_peered":"2024-03-29T08:27:18.624833+0000","last_unstale":"2024-03-29T08:27:19.952043+0000","last_undegraded":"2024-03-29T08:27:19.952043+0000","last_fullsized":"2024-03-29T08:27:19.952043+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:20:57.177828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.630341+0000","last_change":"2024-03-29T08:27:18.629841+0000","last_active":"2024-03-29T08:27:18.630341+0000","last_peered":"2024-03-29T08:27:18.630341+0000","last_clean":"2024-03-29T08:27:18.630341+0000","last_became_active":"2024-03-29T08:27:18.628874+0000","last_became_peered":"2024-03-29T08:27:18.628874+0000","last_unstale":"2024-03-29T08:27:18.630341+0000","last_undegraded":"2024-03-29T08:27:18.630341+0000","last_fullsized":"2024-03-29T08:27:18.630341+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:55:26.401848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946512+0000","last_change":"2024-03-29T08:27:18.600427+0000","last_active":"2024-03-29T08:27:19.946512+0000","last_peered":"2024-03-29T08:27:19.946512+0000","last_clean":"2024-03-29T08:27:19.946512+0000","last_became_active":"2024-03-29T08:27:18.599117+0000","last_became_peered":"2024-03-29T08:27:18.599117+0000","last_unstale":"2024-03-29T08:27:19.946512+0000","last_undegraded":"2024-03-29T08:27:19.946512+0000","last_fullsized":"2024-03-29T08:27:19.946512+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:29:48.275525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598878+0000","last_change":"2024-03-29T08:27:18.620169+0000","last_active":"2024-03-29T08:27:19.598878+0000","last_peered":"2024-03-29T08:27:19.598878+0000","last_clean":"2024-03-29T08:27:19.598878+0000","last_became_active":"2024-03-29T08:27:18.619676+0000","last_became_peered":"2024-03-29T08:27:18.619676+0000","last_unstale":"2024-03-29T08:27:19.598878+0000","last_undegraded":"2024-03-29T08:27:19.598878+0000","last_fullsized":"2024-03-29T08:27:19.598878+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:17:30.862649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598671+0000","last_change":"2024-03-29T08:27:18.620995+0000","last_active":"2024-03-29T08:27:19.598671+0000","last_peered":"2024-03-29T08:27:19.598671+0000","last_clean":"2024-03-29T08:27:19.598671+0000","last_became_active":"2024-03-29T08:27:18.620642+0000","last_became_peered":"2024-03-29T08:27:18.620642+0000","last_unstale":"2024-03-29T08:27:19.598671+0000","last_undegraded":"2024-03-29T08:27:19.598671+0000","last_fullsized":"2024-03-29T08:27:19.598671+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:03:53.966917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951503+0000","last_change":"2024-03-29T08:27:18.611914+0000","last_active":"2024-03-29T08:27:19.951503+0000","last_peered":"2024-03-29T08:27:19.951503+0000","last_clean":"2024-03-29T08:27:19.951503+0000","last_became_active":"2024-03-29T08:27:18.611591+0000","last_became_peered":"2024-03-29T08:27:18.611591+0000","last_unstale":"2024-03-29T08:27:19.951503+0000","last_undegraded":"2024-03-29T08:27:19.951503+0000","last_fullsized":"2024-03-29T08:27:19.951503+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:55.220264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946006+0000","last_change":"2024-03-29T08:27:18.619621+0000","last_active":"2024-03-29T08:27:19.946006+0000","last_peered":"2024-03-29T08:27:19.946006+0000","last_clean":"2024-03-29T08:27:19.946006+0000","last_became_active":"2024-03-29T08:27:18.616379+0000","last_became_peered":"2024-03-29T08:27:18.616379+0000","last_unstale":"2024-03-29T08:27:19.946006+0000","last_undegraded":"2024-03-29T08:27:19.946006+0000","last_fullsized":"2024-03-29T08:27:19.946006+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:23.870278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.592459+0000","last_change":"2024-03-29T08:27:18.599989+0000","last_active":"2024-03-29T08:27:19.592459+0000","last_peered":"2024-03-29T08:27:19.592459+0000","last_clean":"2024-03-29T08:27:19.592459+0000","last_became_active":"2024-03-29T08:27:18.599695+0000","last_became_peered":"2024-03-29T08:27:18.599695+0000","last_unstale":"2024-03-29T08:27:19.592459+0000","last_undegraded":"2024-03-29T08:27:19.592459+0000","last_fullsized":"2024-03-29T08:27:19.592459+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:33.137545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951757+0000","last_change":"2024-03-29T08:27:18.618132+0000","last_active":"2024-03-29T08:27:19.951757+0000","last_peered":"2024-03-29T08:27:19.951757+0000","last_clean":"2024-03-29T08:27:19.951757+0000","last_became_active":"2024-03-29T08:27:18.617846+0000","last_became_peered":"2024-03-29T08:27:18.617846+0000","last_unstale":"2024-03-29T08:27:19.951757+0000","last_undegraded":"2024-03-29T08:27:19.951757+0000","last_fullsized":"2024-03-29T08:27:19.951757+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:56:58.908625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946848+0000","last_change":"2024-03-29T08:27:18.605001+0000","last_active":"2024-03-29T08:27:19.946848+0000","last_peered":"2024-03-29T08:27:19.946848+0000","last_clean":"2024-03-29T08:27:19.946848+0000","last_became_active":"2024-03-29T08:27:18.604709+0000","last_became_peered":"2024-03-29T08:27:18.604709+0000","last_unstale":"2024-03-29T08:27:19.946848+0000","last_undegraded":"2024-03-29T08:27:19.946848+0000","last_fullsized":"2024-03-29T08:27:19.946848+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:38:15.048270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946175+0000","last_change":"2024-03-29T08:27:18.610209+0000","last_active":"2024-03-29T08:27:19.946175+0000","last_peered":"2024-03-29T08:27:19.946175+0000","last_clean":"2024-03-29T08:27:19.946175+0000","last_became_active":"2024-03-29T08:27:18.609954+0000","last_became_peered":"2024-03-29T08:27:18.609954+0000","last_unstale":"2024-03-29T08:27:19.946175+0000","last_undegraded":"2024-03-29T08:27:19.946175+0000","last_fullsized":"2024-03-29T08:27:19.946175+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:08:06.894379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952350+0000","last_change":"2024-03-29T08:27:18.610360+0000","last_active":"2024-03-29T08:27:19.952350+0000","last_peered":"2024-03-29T08:27:19.952350+0000","last_clean":"2024-03-29T08:27:19.952350+0000","last_became_active":"2024-03-29T08:27:18.610070+0000","last_became_peered":"2024-03-29T08:27:18.610070+0000","last_unstale":"2024-03-29T08:27:19.952350+0000","last_undegraded":"2024-03-29T08:27:19.952350+0000","last_fullsized":"2024-03-29T08:27:19.952350+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:57:51.478616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952572+0000","last_change":"2024-03-29T08:27:18.613026+0000","last_active":"2024-03-29T08:27:19.952572+0000","last_peered":"2024-03-29T08:27:19.952572+0000","last_clean":"2024-03-29T08:27:19.952572+0000","last_became_active":"2024-03-29T08:27:18.612775+0000","last_became_peered":"2024-03-29T08:27:18.612775+0000","last_unstale":"2024-03-29T08:27:19.952572+0000","last_undegraded":"2024-03-29T08:27:19.952572+0000","last_fullsized":"2024-03-29T08:27:19.952572+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:55:56.587423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598565+0000","last_change":"2024-03-29T08:27:18.612729+0000","last_active":"2024-03-29T08:27:19.598565+0000","last_peered":"2024-03-29T08:27:19.598565+0000","last_clean":"2024-03-29T08:27:19.598565+0000","last_became_active":"2024-03-29T08:27:18.612483+0000","last_became_peered":"2024-03-29T08:27:18.612483+0000","last_unstale":"2024-03-29T08:27:19.598565+0000","last_undegraded":"2024-03-29T08:27:19.598565+0000","last_fullsized":"2024-03-29T08:27:19.598565+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:25.324724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.950794+0000","last_change":"2024-03-29T08:27:18.613817+0000","last_active":"2024-03-29T08:27:19.950794+0000","last_peered":"2024-03-29T08:27:19.950794+0000","last_clean":"2024-03-29T08:27:19.950794+0000","last_became_active":"2024-03-29T08:27:18.610960+0000","last_became_peered":"2024-03-29T08:27:18.610960+0000","last_unstale":"2024-03-29T08:27:19.950794+0000","last_undegraded":"2024-03-29T08:27:19.950794+0000","last_fullsized":"2024-03-29T08:27:19.950794+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:50:23.870278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.608524+0000","last_change":"2024-03-29T08:27:18.602948+0000","last_active":"2024-03-29T08:27:18.608524+0000","last_peered":"2024-03-29T08:27:18.608524+0000","last_clean":"2024-03-29T08:27:18.608524+0000","last_became_active":"2024-03-29T08:27:18.602691+0000","last_became_peered":"2024-03-29T08:27:18.602691+0000","last_unstale":"2024-03-29T08:27:18.608524+0000","last_undegraded":"2024-03-29T08:27:18.608524+0000","last_fullsized":"2024-03-29T08:27:18.608524+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:01:12.985977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"105'4","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952072+0000","last_change":"2024-03-29T08:27:18.208888+0000","last_active":"2024-03-29T08:27:19.952072+0000","last_peered":"2024-03-29T08:27:19.952072+0000","last_clean":"2024-03-29T08:27:19.952072+0000","last_became_active":"2024-03-29T08:27:18.208588+0000","last_became_peered":"2024-03-29T08:27:18.208588+0000","last_unstale":"2024-03-29T08:27:19.952072+0000","last_undegraded":"2024-03-29T08:27:19.952072+0000","last_fullsized":"2024-03-29T08:27:19.952072+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:09:14.747589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951987+0000","last_change":"2024-03-29T08:27:18.613680+0000","last_active":"2024-03-29T08:27:19.951987+0000","last_peered":"2024-03-29T08:27:19.951987+0000","last_clean":"2024-03-29T08:27:19.951987+0000","last_became_active":"2024-03-29T08:27:18.613225+0000","last_became_peered":"2024-03-29T08:27:18.613225+0000","last_unstale":"2024-03-29T08:27:19.951987+0000","last_undegraded":"2024-03-29T08:27:19.951987+0000","last_fullsized":"2024-03-29T08:27:19.951987+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:00:06.295589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"105'1","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593637+0000","last_change":"2024-03-29T08:27:18.191059+0000","last_active":"2024-03-29T08:27:19.593637+0000","last_peered":"2024-03-29T08:27:19.593637+0000","last_clean":"2024-03-29T08:27:19.593637+0000","last_became_active":"2024-03-29T08:27:18.190819+0000","last_became_peered":"2024-03-29T08:27:18.190819+0000","last_unstale":"2024-03-29T08:27:19.593637+0000","last_undegraded":"2024-03-29T08:27:19.593637+0000","last_fullsized":"2024-03-29T08:27:19.593637+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:56:25.523454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951711+0000","last_change":"2024-03-29T08:27:18.623453+0000","last_active":"2024-03-29T08:27:19.951711+0000","last_peered":"2024-03-29T08:27:19.951711+0000","last_clean":"2024-03-29T08:27:19.951711+0000","last_became_active":"2024-03-29T08:27:18.623150+0000","last_became_peered":"2024-03-29T08:27:18.623150+0000","last_unstale":"2024-03-29T08:27:19.951711+0000","last_undegraded":"2024-03-29T08:27:19.951711+0000","last_fullsized":"2024-03-29T08:27:19.951711+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:03:53.966917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"105'2","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.599149+0000","last_change":"2024-03-29T08:27:18.209722+0000","last_active":"2024-03-29T08:27:19.599149+0000","last_peered":"2024-03-29T08:27:19.599149+0000","last_clean":"2024-03-29T08:27:19.599149+0000","last_became_active":"2024-03-29T08:27:18.209395+0000","last_became_peered":"2024-03-29T08:27:18.209395+0000","last_unstale":"2024-03-29T08:27:19.599149+0000","last_undegraded":"2024-03-29T08:27:19.599149+0000","last_fullsized":"2024-03-29T08:27:19.599149+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:45:54.654095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.592576+0000","last_change":"2024-03-29T08:27:18.610025+0000","last_active":"2024-03-29T08:27:19.592576+0000","last_peered":"2024-03-29T08:27:19.592576+0000","last_clean":"2024-03-29T08:27:19.592576+0000","last_became_active":"2024-03-29T08:27:18.609771+0000","last_became_peered":"2024-03-29T08:27:18.609771+0000","last_unstale":"2024-03-29T08:27:19.592576+0000","last_undegraded":"2024-03-29T08:27:19.592576+0000","last_fullsized":"2024-03-29T08:27:19.592576+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:11:19.080978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"105'2","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951827+0000","last_change":"2024-03-29T08:27:18.207299+0000","last_active":"2024-03-29T08:27:19.951827+0000","last_peered":"2024-03-29T08:27:19.951827+0000","last_clean":"2024-03-29T08:27:19.951827+0000","last_became_active":"2024-03-29T08:27:18.206874+0000","last_became_peered":"2024-03-29T08:27:18.206874+0000","last_unstale":"2024-03-29T08:27:19.951827+0000","last_undegraded":"2024-03-29T08:27:19.951827+0000","last_fullsized":"2024-03-29T08:27:19.951827+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:54:28.856275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946281+0000","last_change":"2024-03-29T08:27:18.604980+0000","last_active":"2024-03-29T08:27:19.946281+0000","last_peered":"2024-03-29T08:27:19.946281+0000","last_clean":"2024-03-29T08:27:19.946281+0000","last_became_active":"2024-03-29T08:27:18.604675+0000","last_became_peered":"2024-03-29T08:27:18.604675+0000","last_unstale":"2024-03-29T08:27:19.946281+0000","last_undegraded":"2024-03-29T08:27:19.946281+0000","last_fullsized":"2024-03-29T08:27:19.946281+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:40:03.566438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"105'1","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951662+0000","last_change":"2024-03-29T08:27:18.196647+0000","last_active":"2024-03-29T08:27:19.951662+0000","last_peered":"2024-03-29T08:27:19.951662+0000","last_clean":"2024-03-29T08:27:19.951662+0000","last_became_active":"2024-03-29T08:27:18.196399+0000","last_became_peered":"2024-03-29T08:27:18.196399+0000","last_unstale":"2024-03-29T08:27:19.951662+0000","last_undegraded":"2024-03-29T08:27:19.951662+0000","last_fullsized":"2024-03-29T08:27:19.951662+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:14:26.885570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593373+0000","last_change":"2024-03-29T08:27:18.606031+0000","last_active":"2024-03-29T08:27:19.593373+0000","last_peered":"2024-03-29T08:27:19.593373+0000","last_clean":"2024-03-29T08:27:19.593373+0000","last_became_active":"2024-03-29T08:27:18.605630+0000","last_became_peered":"2024-03-29T08:27:18.605630+0000","last_unstale":"2024-03-29T08:27:19.593373+0000","last_undegraded":"2024-03-29T08:27:19.593373+0000","last_fullsized":"2024-03-29T08:27:19.593373+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:36:28.363643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"105'1","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598822+0000","last_change":"2024-03-29T08:27:18.202384+0000","last_active":"2024-03-29T08:27:19.598822+0000","last_peered":"2024-03-29T08:27:19.598822+0000","last_clean":"2024-03-29T08:27:19.598822+0000","last_became_active":"2024-03-29T08:27:18.202088+0000","last_became_peered":"2024-03-29T08:27:18.202088+0000","last_unstale":"2024-03-29T08:27:19.598822+0000","last_undegraded":"2024-03-29T08:27:19.598822+0000","last_fullsized":"2024-03-29T08:27:19.598822+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:40:57.395634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951360+0000","last_change":"2024-03-29T08:27:18.622412+0000","last_active":"2024-03-29T08:27:19.951360+0000","last_peered":"2024-03-29T08:27:19.951360+0000","last_clean":"2024-03-29T08:27:19.951360+0000","last_became_active":"2024-03-29T08:27:18.622123+0000","last_became_peered":"2024-03-29T08:27:18.622123+0000","last_unstale":"2024-03-29T08:27:19.951360+0000","last_undegraded":"2024-03-29T08:27:19.951360+0000","last_fullsized":"2024-03-29T08:27:19.951360+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:54:29.858302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593352+0000","last_change":"2024-03-29T08:27:18.195629+0000","last_active":"2024-03-29T08:27:19.593352+0000","last_peered":"2024-03-29T08:27:19.593352+0000","last_clean":"2024-03-29T08:27:19.593352+0000","last_became_active":"2024-03-29T08:27:18.195315+0000","last_became_peered":"2024-03-29T08:27:18.195315+0000","last_unstale":"2024-03-29T08:27:19.593352+0000","last_undegraded":"2024-03-29T08:27:19.593352+0000","last_fullsized":"2024-03-29T08:27:19.593352+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:59:55.763695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946540+0000","last_change":"2024-03-29T08:27:18.609143+0000","last_active":"2024-03-29T08:27:19.946540+0000","last_peered":"2024-03-29T08:27:19.946540+0000","last_clean":"2024-03-29T08:27:19.946540+0000","last_became_active":"2024-03-29T08:27:18.608712+0000","last_became_peered":"2024-03-29T08:27:18.608712+0000","last_unstale":"2024-03-29T08:27:19.946540+0000","last_undegraded":"2024-03-29T08:27:19.946540+0000","last_fullsized":"2024-03-29T08:27:19.946540+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:53.626567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.599424+0000","last_change":"2024-03-29T08:27:18.200264+0000","last_active":"2024-03-29T08:27:18.599424+0000","last_peered":"2024-03-29T08:27:18.599424+0000","last_clean":"2024-03-29T08:27:18.599424+0000","last_became_active":"2024-03-29T08:27:18.199981+0000","last_became_peered":"2024-03-29T08:27:18.199981+0000","last_unstale":"2024-03-29T08:27:18.599424+0000","last_undegraded":"2024-03-29T08:27:18.599424+0000","last_fullsized":"2024-03-29T08:27:18.599424+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:11:24.362800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952092+0000","last_change":"2024-03-29T08:27:18.625409+0000","last_active":"2024-03-29T08:27:19.952092+0000","last_peered":"2024-03-29T08:27:19.952092+0000","last_clean":"2024-03-29T08:27:19.952092+0000","last_became_active":"2024-03-29T08:27:18.623585+0000","last_became_peered":"2024-03-29T08:27:18.623585+0000","last_unstale":"2024-03-29T08:27:19.952092+0000","last_undegraded":"2024-03-29T08:27:19.952092+0000","last_fullsized":"2024-03-29T08:27:19.952092+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:25.324724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.599102+0000","last_change":"2024-03-29T08:27:18.206460+0000","last_active":"2024-03-29T08:27:19.599102+0000","last_peered":"2024-03-29T08:27:19.599102+0000","last_clean":"2024-03-29T08:27:19.599102+0000","last_became_active":"2024-03-29T08:27:18.206081+0000","last_became_peered":"2024-03-29T08:27:18.206081+0000","last_unstale":"2024-03-29T08:27:19.599102+0000","last_undegraded":"2024-03-29T08:27:19.599102+0000","last_fullsized":"2024-03-29T08:27:19.599102+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:09:24.332081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946913+0000","last_change":"2024-03-29T08:27:18.593221+0000","last_active":"2024-03-29T08:27:19.946913+0000","last_peered":"2024-03-29T08:27:19.946913+0000","last_clean":"2024-03-29T08:27:19.946913+0000","last_became_active":"2024-03-29T08:27:18.592927+0000","last_became_peered":"2024-03-29T08:27:18.592927+0000","last_unstale":"2024-03-29T08:27:19.946913+0000","last_undegraded":"2024-03-29T08:27:19.946913+0000","last_fullsized":"2024-03-29T08:27:19.946913+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:23.386306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.952173+0000","last_change":"2024-03-29T08:27:18.209092+0000","last_active":"2024-03-29T08:27:19.952173+0000","last_peered":"2024-03-29T08:27:19.952173+0000","last_clean":"2024-03-29T08:27:19.952173+0000","last_became_active":"2024-03-29T08:27:18.208292+0000","last_became_peered":"2024-03-29T08:27:18.208292+0000","last_unstale":"2024-03-29T08:27:19.952173+0000","last_undegraded":"2024-03-29T08:27:19.952173+0000","last_fullsized":"2024-03-29T08:27:19.952173+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:35:53.062152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593694+0000","last_change":"2024-03-29T08:27:18.622138+0000","last_active":"2024-03-29T08:27:19.593694+0000","last_peered":"2024-03-29T08:27:19.593694+0000","last_clean":"2024-03-29T08:27:19.593694+0000","last_became_active":"2024-03-29T08:27:18.621894+0000","last_became_peered":"2024-03-29T08:27:18.621894+0000","last_unstale":"2024-03-29T08:27:19.593694+0000","last_undegraded":"2024-03-29T08:27:19.593694+0000","last_fullsized":"2024-03-29T08:27:19.593694+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:33:39.623305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"105'3","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593610+0000","last_change":"2024-03-29T08:27:18.193031+0000","last_active":"2024-03-29T08:27:19.593610+0000","last_peered":"2024-03-29T08:27:19.593610+0000","last_clean":"2024-03-29T08:27:19.593610+0000","last_became_active":"2024-03-29T08:27:18.192645+0000","last_became_peered":"2024-03-29T08:27:18.192645+0000","last_unstale":"2024-03-29T08:27:19.593610+0000","last_undegraded":"2024-03-29T08:27:19.593610+0000","last_fullsized":"2024-03-29T08:27:19.593610+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:47:20.012694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946746+0000","last_change":"2024-03-29T08:27:18.603075+0000","last_active":"2024-03-29T08:27:19.946746+0000","last_peered":"2024-03-29T08:27:19.946746+0000","last_clean":"2024-03-29T08:27:19.946746+0000","last_became_active":"2024-03-29T08:27:18.601081+0000","last_became_peered":"2024-03-29T08:27:18.601081+0000","last_unstale":"2024-03-29T08:27:19.946746+0000","last_undegraded":"2024-03-29T08:27:19.946746+0000","last_fullsized":"2024-03-29T08:27:19.946746+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:20:57.177828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"105'6","reported_seq":17,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946591+0000","last_change":"2024-03-29T08:27:18.194476+0000","last_active":"2024-03-29T08:27:19.946591+0000","last_peered":"2024-03-29T08:27:19.946591+0000","last_clean":"2024-03-29T08:27:19.946591+0000","last_became_active":"2024-03-29T08:27:18.194201+0000","last_became_peered":"2024-03-29T08:27:18.194201+0000","last_unstale":"2024-03-29T08:27:19.946591+0000","last_undegraded":"2024-03-29T08:27:19.946591+0000","last_fullsized":"2024-03-29T08:27:19.946591+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:23:07.623906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951597+0000","last_change":"2024-03-29T08:27:18.619655+0000","last_active":"2024-03-29T08:27:19.951597+0000","last_peered":"2024-03-29T08:27:19.951597+0000","last_clean":"2024-03-29T08:27:19.951597+0000","last_became_active":"2024-03-29T08:27:18.619405+0000","last_became_peered":"2024-03-29T08:27:18.619405+0000","last_unstale":"2024-03-29T08:27:19.951597+0000","last_undegraded":"2024-03-29T08:27:19.951597+0000","last_fullsized":"2024-03-29T08:27:19.951597+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:12:08.460617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"105'2","reported_seq":12,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.628692+0000","last_change":"2024-03-29T08:27:18.199098+0000","last_active":"2024-03-29T08:27:18.628692+0000","last_peered":"2024-03-29T08:27:18.628692+0000","last_clean":"2024-03-29T08:27:18.628692+0000","last_became_active":"2024-03-29T08:27:18.198849+0000","last_became_peered":"2024-03-29T08:27:18.198849+0000","last_unstale":"2024-03-29T08:27:18.628692+0000","last_undegraded":"2024-03-29T08:27:18.628692+0000","last_fullsized":"2024-03-29T08:27:18.628692+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:37:40.981819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598872+0000","last_change":"2024-03-29T08:27:18.614573+0000","last_active":"2024-03-29T08:27:19.598872+0000","last_peered":"2024-03-29T08:27:19.598872+0000","last_clean":"2024-03-29T08:27:19.598872+0000","last_became_active":"2024-03-29T08:27:18.614306+0000","last_became_peered":"2024-03-29T08:27:18.614306+0000","last_unstale":"2024-03-29T08:27:19.598872+0000","last_undegraded":"2024-03-29T08:27:19.598872+0000","last_fullsized":"2024-03-29T08:27:19.598872+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:20:57.177828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951700+0000","last_change":"2024-03-29T08:27:18.201545+0000","last_active":"2024-03-29T08:27:19.951700+0000","last_peered":"2024-03-29T08:27:19.951700+0000","last_clean":"2024-03-29T08:27:19.951700+0000","last_became_active":"2024-03-29T08:27:18.201287+0000","last_became_peered":"2024-03-29T08:27:18.201287+0000","last_unstale":"2024-03-29T08:27:19.951700+0000","last_undegraded":"2024-03-29T08:27:19.951700+0000","last_fullsized":"2024-03-29T08:27:19.951700+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:55:55.585396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593296+0000","last_change":"2024-03-29T08:27:18.613019+0000","last_active":"2024-03-29T08:27:19.593296+0000","last_peered":"2024-03-29T08:27:19.593296+0000","last_clean":"2024-03-29T08:27:19.593296+0000","last_became_active":"2024-03-29T08:27:18.612574+0000","last_became_peered":"2024-03-29T08:27:18.612574+0000","last_unstale":"2024-03-29T08:27:19.593296+0000","last_undegraded":"2024-03-29T08:27:19.593296+0000","last_fullsized":"2024-03-29T08:27:19.593296+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:07:55.309563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946363+0000","last_change":"2024-03-29T08:27:18.189310+0000","last_active":"2024-03-29T08:27:19.946363+0000","last_peered":"2024-03-29T08:27:19.946363+0000","last_clean":"2024-03-29T08:27:19.946363+0000","last_became_active":"2024-03-29T08:27:18.188803+0000","last_became_peered":"2024-03-29T08:27:18.188803+0000","last_unstale":"2024-03-29T08:27:19.946363+0000","last_undegraded":"2024-03-29T08:27:19.946363+0000","last_fullsized":"2024-03-29T08:27:19.946363+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:32:21.502500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598640+0000","last_change":"2024-03-29T08:27:18.624216+0000","last_active":"2024-03-29T08:27:19.598640+0000","last_peered":"2024-03-29T08:27:19.598640+0000","last_clean":"2024-03-29T08:27:19.598640+0000","last_became_active":"2024-03-29T08:27:18.623842+0000","last_became_peered":"2024-03-29T08:27:18.623842+0000","last_unstale":"2024-03-29T08:27:19.598640+0000","last_undegraded":"2024-03-29T08:27:19.598640+0000","last_fullsized":"2024-03-29T08:27:19.598640+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:12:08.460617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"105'8","reported_seq":19,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951105+0000","last_change":"2024-03-29T08:27:18.197306+0000","last_active":"2024-03-29T08:27:19.951105+0000","last_peered":"2024-03-29T08:27:19.951105+0000","last_clean":"2024-03-29T08:27:19.951105+0000","last_became_active":"2024-03-29T08:27:18.197015+0000","last_became_peered":"2024-03-29T08:27:18.197015+0000","last_unstale":"2024-03-29T08:27:19.951105+0000","last_undegraded":"2024-03-29T08:27:19.951105+0000","last_fullsized":"2024-03-29T08:27:19.951105+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:56:57.906598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946932+0000","last_change":"2024-03-29T08:27:18.596781+0000","last_active":"2024-03-29T08:27:19.946932+0000","last_peered":"2024-03-29T08:27:19.946932+0000","last_clean":"2024-03-29T08:27:19.946932+0000","last_became_active":"2024-03-29T08:27:18.596528+0000","last_became_peered":"2024-03-29T08:27:18.596528+0000","last_unstale":"2024-03-29T08:27:19.946932+0000","last_undegraded":"2024-03-29T08:27:19.946932+0000","last_fullsized":"2024-03-29T08:27:19.946932+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:36.631872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"105'2","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946452+0000","last_change":"2024-03-29T08:27:18.188319+0000","last_active":"2024-03-29T08:27:19.946452+0000","last_peered":"2024-03-29T08:27:19.946452+0000","last_clean":"2024-03-29T08:27:19.946452+0000","last_became_active":"2024-03-29T08:27:18.188073+0000","last_became_peered":"2024-03-29T08:27:18.188073+0000","last_unstale":"2024-03-29T08:27:19.946452+0000","last_undegraded":"2024-03-29T08:27:19.946452+0000","last_fullsized":"2024-03-29T08:27:19.946452+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:31:12.994086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946536+0000","last_change":"2024-03-29T08:27:18.596679+0000","last_active":"2024-03-29T08:27:19.946536+0000","last_peered":"2024-03-29T08:27:19.946536+0000","last_clean":"2024-03-29T08:27:19.946536+0000","last_became_active":"2024-03-29T08:27:18.595841+0000","last_became_peered":"2024-03-29T08:27:18.595841+0000","last_unstale":"2024-03-29T08:27:19.946536+0000","last_undegraded":"2024-03-29T08:27:19.946536+0000","last_fullsized":"2024-03-29T08:27:19.946536+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:17:30.862649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"105'1","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593529+0000","last_change":"2024-03-29T08:27:18.192026+0000","last_active":"2024-03-29T08:27:19.593529+0000","last_peered":"2024-03-29T08:27:19.593529+0000","last_clean":"2024-03-29T08:27:19.593529+0000","last_became_active":"2024-03-29T08:27:18.191739+0000","last_became_peered":"2024-03-29T08:27:18.191739+0000","last_unstale":"2024-03-29T08:27:19.593529+0000","last_undegraded":"2024-03-29T08:27:19.593529+0000","last_fullsized":"2024-03-29T08:27:19.593529+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:08:26.898804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946574+0000","last_change":"2024-03-29T08:27:18.599059+0000","last_active":"2024-03-29T08:27:19.946574+0000","last_peered":"2024-03-29T08:27:19.946574+0000","last_clean":"2024-03-29T08:27:19.946574+0000","last_became_active":"2024-03-29T08:27:18.598785+0000","last_became_peered":"2024-03-29T08:27:18.598785+0000","last_unstale":"2024-03-29T08:27:19.946574+0000","last_undegraded":"2024-03-29T08:27:19.946574+0000","last_fullsized":"2024-03-29T08:27:19.946574+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:47:12.134933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.598976+0000","last_change":"2024-03-29T08:27:18.197415+0000","last_active":"2024-03-29T08:27:18.598976+0000","last_peered":"2024-03-29T08:27:18.598976+0000","last_clean":"2024-03-29T08:27:18.598976+0000","last_became_active":"2024-03-29T08:27:18.197122+0000","last_became_peered":"2024-03-29T08:27:18.197122+0000","last_unstale":"2024-03-29T08:27:18.598976+0000","last_undegraded":"2024-03-29T08:27:18.598976+0000","last_fullsized":"2024-03-29T08:27:18.598976+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:16:46.630797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.618348+0000","last_change":"2024-03-29T08:27:18.615797+0000","last_active":"2024-03-29T08:27:18.618348+0000","last_peered":"2024-03-29T08:27:18.618348+0000","last_clean":"2024-03-29T08:27:18.618348+0000","last_became_active":"2024-03-29T08:27:18.615532+0000","last_became_peered":"2024-03-29T08:27:18.615532+0000","last_unstale":"2024-03-29T08:27:18.618348+0000","last_undegraded":"2024-03-29T08:27:18.618348+0000","last_fullsized":"2024-03-29T08:27:18.618348+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:58:22.690924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"105'2","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951767+0000","last_change":"2024-03-29T08:27:18.210866+0000","last_active":"2024-03-29T08:27:19.951767+0000","last_peered":"2024-03-29T08:27:19.951767+0000","last_clean":"2024-03-29T08:27:19.951767+0000","last_became_active":"2024-03-29T08:27:18.210247+0000","last_became_peered":"2024-03-29T08:27:18.210247+0000","last_unstale":"2024-03-29T08:27:19.951767+0000","last_undegraded":"2024-03-29T08:27:19.951767+0000","last_fullsized":"2024-03-29T08:27:19.951767+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:03:32.135518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.619054+0000","last_change":"2024-03-29T08:27:18.615978+0000","last_active":"2024-03-29T08:27:18.619054+0000","last_peered":"2024-03-29T08:27:18.619054+0000","last_clean":"2024-03-29T08:27:18.619054+0000","last_became_active":"2024-03-29T08:27:18.615670+0000","last_became_peered":"2024-03-29T08:27:18.615670+0000","last_unstale":"2024-03-29T08:27:18.619054+0000","last_undegraded":"2024-03-29T08:27:18.619054+0000","last_fullsized":"2024-03-29T08:27:18.619054+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:49:52.815184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946040+0000","last_change":"2024-03-29T08:27:18.195955+0000","last_active":"2024-03-29T08:27:19.946040+0000","last_peered":"2024-03-29T08:27:19.946040+0000","last_clean":"2024-03-29T08:27:19.946040+0000","last_became_active":"2024-03-29T08:27:18.195631+0000","last_became_peered":"2024-03-29T08:27:18.195631+0000","last_unstale":"2024-03-29T08:27:19.946040+0000","last_undegraded":"2024-03-29T08:27:19.946040+0000","last_fullsized":"2024-03-29T08:27:19.946040+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:30:17.738008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.950892+0000","last_change":"2024-03-29T08:27:18.613281+0000","last_active":"2024-03-29T08:27:19.950892+0000","last_peered":"2024-03-29T08:27:19.950892+0000","last_clean":"2024-03-29T08:27:19.950892+0000","last_became_active":"2024-03-29T08:27:18.611492+0000","last_became_peered":"2024-03-29T08:27:18.611492+0000","last_unstale":"2024-03-29T08:27:19.950892+0000","last_undegraded":"2024-03-29T08:27:19.950892+0000","last_fullsized":"2024-03-29T08:27:19.950892+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:44:45.577113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.592336+0000","last_change":"2024-03-29T08:27:18.193706+0000","last_active":"2024-03-29T08:27:19.592336+0000","last_peered":"2024-03-29T08:27:19.592336+0000","last_clean":"2024-03-29T08:27:19.592336+0000","last_became_active":"2024-03-29T08:27:18.193394+0000","last_became_peered":"2024-03-29T08:27:18.193394+0000","last_unstale":"2024-03-29T08:27:19.592336+0000","last_undegraded":"2024-03-29T08:27:19.592336+0000","last_fullsized":"2024-03-29T08:27:19.592336+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:57:50.476589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946155+0000","last_change":"2024-03-29T08:27:18.610911+0000","last_active":"2024-03-29T08:27:19.946155+0000","last_peered":"2024-03-29T08:27:19.946155+0000","last_clean":"2024-03-29T08:27:19.946155+0000","last_became_active":"2024-03-29T08:27:18.610667+0000","last_became_peered":"2024-03-29T08:27:18.610667+0000","last_unstale":"2024-03-29T08:27:19.946155+0000","last_undegraded":"2024-03-29T08:27:19.946155+0000","last_fullsized":"2024-03-29T08:27:19.946155+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:50:18.987192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598543+0000","last_change":"2024-03-29T08:27:18.209289+0000","last_active":"2024-03-29T08:27:19.598543+0000","last_peered":"2024-03-29T08:27:19.598543+0000","last_clean":"2024-03-29T08:27:19.598543+0000","last_became_active":"2024-03-29T08:27:18.209033+0000","last_became_peered":"2024-03-29T08:27:18.209033+0000","last_unstale":"2024-03-29T08:27:19.598543+0000","last_undegraded":"2024-03-29T08:27:19.598543+0000","last_fullsized":"2024-03-29T08:27:19.598543+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:52:10.210324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.608499+0000","last_change":"2024-03-29T08:27:18.602427+0000","last_active":"2024-03-29T08:27:18.608499+0000","last_peered":"2024-03-29T08:27:18.608499+0000","last_clean":"2024-03-29T08:27:18.608499+0000","last_became_active":"2024-03-29T08:27:18.602157+0000","last_became_peered":"2024-03-29T08:27:18.602157+0000","last_unstale":"2024-03-29T08:27:18.608499+0000","last_undegraded":"2024-03-29T08:27:18.608499+0000","last_fullsized":"2024-03-29T08:27:18.608499+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:54:49.572315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"105'2","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946720+0000","last_change":"2024-03-29T08:27:18.189214+0000","last_active":"2024-03-29T08:27:19.946720+0000","last_peered":"2024-03-29T08:27:19.946720+0000","last_clean":"2024-03-29T08:27:19.946720+0000","last_became_active":"2024-03-29T08:27:18.188866+0000","last_became_peered":"2024-03-29T08:27:18.188866+0000","last_unstale":"2024-03-29T08:27:19.946720+0000","last_undegraded":"2024-03-29T08:27:19.946720+0000","last_fullsized":"2024-03-29T08:27:19.946720+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:33:09.365581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.628508+0000","last_change":"2024-03-29T08:27:18.626711+0000","last_active":"2024-03-29T08:27:18.628508+0000","last_peered":"2024-03-29T08:27:18.628508+0000","last_clean":"2024-03-29T08:27:18.628508+0000","last_became_active":"2024-03-29T08:27:18.626076+0000","last_became_peered":"2024-03-29T08:27:18.626076+0000","last_unstale":"2024-03-29T08:27:18.628508+0000","last_undegraded":"2024-03-29T08:27:18.628508+0000","last_fullsized":"2024-03-29T08:27:18.628508+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:04:06.478927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.599630+0000","last_change":"2024-03-29T08:27:18.209259+0000","last_active":"2024-03-29T08:27:18.599630+0000","last_peered":"2024-03-29T08:27:18.599630+0000","last_clean":"2024-03-29T08:27:18.599630+0000","last_became_active":"2024-03-29T08:27:18.208959+0000","last_became_peered":"2024-03-29T08:27:18.208959+0000","last_unstale":"2024-03-29T08:27:18.599630+0000","last_undegraded":"2024-03-29T08:27:18.599630+0000","last_fullsized":"2024-03-29T08:27:18.599630+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:00:59.709598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598809+0000","last_change":"2024-03-29T08:27:18.626266+0000","last_active":"2024-03-29T08:27:19.598809+0000","last_peered":"2024-03-29T08:27:19.598809+0000","last_clean":"2024-03-29T08:27:19.598809+0000","last_became_active":"2024-03-29T08:27:18.626013+0000","last_became_peered":"2024-03-29T08:27:18.626013+0000","last_unstale":"2024-03-29T08:27:19.598809+0000","last_undegraded":"2024-03-29T08:27:19.598809+0000","last_fullsized":"2024-03-29T08:27:19.598809+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:09:36.631872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.599425+0000","last_change":"2024-03-29T08:27:18.195634+0000","last_active":"2024-03-29T08:27:18.599425+0000","last_peered":"2024-03-29T08:27:18.599425+0000","last_clean":"2024-03-29T08:27:18.599425+0000","last_became_active":"2024-03-29T08:27:18.195388+0000","last_became_peered":"2024-03-29T08:27:18.195388+0000","last_unstale":"2024-03-29T08:27:18.599425+0000","last_undegraded":"2024-03-29T08:27:18.599425+0000","last_fullsized":"2024-03-29T08:27:18.599425+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:16:54.769206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593403+0000","last_change":"2024-03-29T08:27:18.613296+0000","last_active":"2024-03-29T08:27:19.593403+0000","last_peered":"2024-03-29T08:27:19.593403+0000","last_clean":"2024-03-29T08:27:19.593403+0000","last_became_active":"2024-03-29T08:27:18.612843+0000","last_became_peered":"2024-03-29T08:27:18.612843+0000","last_unstale":"2024-03-29T08:27:19.593403+0000","last_undegraded":"2024-03-29T08:27:19.593403+0000","last_fullsized":"2024-03-29T08:27:19.593403+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:52:20.003422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946568+0000","last_change":"2024-03-29T08:27:18.191694+0000","last_active":"2024-03-29T08:27:19.946568+0000","last_peered":"2024-03-29T08:27:19.946568+0000","last_clean":"2024-03-29T08:27:19.946568+0000","last_became_active":"2024-03-29T08:27:18.191378+0000","last_became_peered":"2024-03-29T08:27:18.191378+0000","last_unstale":"2024-03-29T08:27:19.946568+0000","last_undegraded":"2024-03-29T08:27:19.946568+0000","last_fullsized":"2024-03-29T08:27:19.946568+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:24:43.469430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.612053+0000","last_change":"2024-03-29T08:27:18.604095+0000","last_active":"2024-03-29T08:27:18.612053+0000","last_peered":"2024-03-29T08:27:18.612053+0000","last_clean":"2024-03-29T08:27:18.612053+0000","last_became_active":"2024-03-29T08:27:18.603458+0000","last_became_peered":"2024-03-29T08:27:18.603458+0000","last_unstale":"2024-03-29T08:27:18.612053+0000","last_undegraded":"2024-03-29T08:27:18.612053+0000","last_fullsized":"2024-03-29T08:27:18.612053+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:20:23.286313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.599013+0000","last_change":"2024-03-29T08:27:18.209215+0000","last_active":"2024-03-29T08:27:19.599013+0000","last_peered":"2024-03-29T08:27:19.599013+0000","last_clean":"2024-03-29T08:27:19.599013+0000","last_became_active":"2024-03-29T08:27:18.208953+0000","last_became_peered":"2024-03-29T08:27:18.208953+0000","last_unstale":"2024-03-29T08:27:19.599013+0000","last_undegraded":"2024-03-29T08:27:19.599013+0000","last_fullsized":"2024-03-29T08:27:19.599013+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:13:20.157128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.593492+0000","last_change":"2024-03-29T08:27:18.618710+0000","last_active":"2024-03-29T08:27:19.593492+0000","last_peered":"2024-03-29T08:27:19.593492+0000","last_clean":"2024-03-29T08:27:19.593492+0000","last_became_active":"2024-03-29T08:27:18.618465+0000","last_became_peered":"2024-03-29T08:27:18.618465+0000","last_unstale":"2024-03-29T08:27:19.593492+0000","last_undegraded":"2024-03-29T08:27:19.593492+0000","last_fullsized":"2024-03-29T08:27:19.593492+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:16:57.103209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946290+0000","last_change":"2024-03-29T08:27:18.188109+0000","last_active":"2024-03-29T08:27:19.946290+0000","last_peered":"2024-03-29T08:27:19.946290+0000","last_clean":"2024-03-29T08:27:19.946290+0000","last_became_active":"2024-03-29T08:27:18.187843+0000","last_became_peered":"2024-03-29T08:27:18.187843+0000","last_unstale":"2024-03-29T08:27:19.946290+0000","last_undegraded":"2024-03-29T08:27:19.946290+0000","last_fullsized":"2024-03-29T08:27:19.946290+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:23:22.947938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.620121+0000","last_change":"2024-03-29T08:27:18.613882+0000","last_active":"2024-03-29T08:27:18.620121+0000","last_peered":"2024-03-29T08:27:18.620121+0000","last_clean":"2024-03-29T08:27:18.620121+0000","last_became_active":"2024-03-29T08:27:18.610873+0000","last_became_peered":"2024-03-29T08:27:18.610873+0000","last_unstale":"2024-03-29T08:27:18.620121+0000","last_undegraded":"2024-03-29T08:27:18.620121+0000","last_fullsized":"2024-03-29T08:27:18.620121+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:27:04.811546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.951693+0000","last_change":"2024-03-29T08:27:18.202858+0000","last_active":"2024-03-29T08:27:19.951693+0000","last_peered":"2024-03-29T08:27:19.951693+0000","last_clean":"2024-03-29T08:27:19.951693+0000","last_became_active":"2024-03-29T08:27:18.202610+0000","last_became_peered":"2024-03-29T08:27:18.202610+0000","last_unstale":"2024-03-29T08:27:19.951693+0000","last_undegraded":"2024-03-29T08:27:19.951693+0000","last_fullsized":"2024-03-29T08:27:19.951693+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:49:09.785927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946300+0000","last_change":"2024-03-29T08:27:18.608958+0000","last_active":"2024-03-29T08:27:19.946300+0000","last_peered":"2024-03-29T08:27:19.946300+0000","last_clean":"2024-03-29T08:27:19.946300+0000","last_became_active":"2024-03-29T08:27:18.605287+0000","last_became_peered":"2024-03-29T08:27:18.605287+0000","last_unstale":"2024-03-29T08:27:19.946300+0000","last_undegraded":"2024-03-29T08:27:19.946300+0000","last_fullsized":"2024-03-29T08:27:19.946300+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:09:15.749616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.598538+0000","last_change":"2024-03-29T08:27:18.209857+0000","last_active":"2024-03-29T08:27:19.598538+0000","last_peered":"2024-03-29T08:27:19.598538+0000","last_clean":"2024-03-29T08:27:19.598538+0000","last_became_active":"2024-03-29T08:27:18.208138+0000","last_became_peered":"2024-03-29T08:27:18.208138+0000","last_unstale":"2024-03-29T08:27:19.598538+0000","last_undegraded":"2024-03-29T08:27:19.598538+0000","last_fullsized":"2024-03-29T08:27:19.598538+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:07:08.533818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.599077+0000","last_change":"2024-03-29T08:27:18.625750+0000","last_active":"2024-03-29T08:27:19.599077+0000","last_peered":"2024-03-29T08:27:19.599077+0000","last_clean":"2024-03-29T08:27:19.599077+0000","last_became_active":"2024-03-29T08:27:18.625507+0000","last_became_peered":"2024-03-29T08:27:18.625507+0000","last_unstale":"2024-03-29T08:27:19.599077+0000","last_undegraded":"2024-03-29T08:27:19.599077+0000","last_fullsized":"2024-03-29T08:27:19.599077+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:18.171046+0000","last_clean_scrub_stamp":"2024-03-29T08:27:18.171046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:50:18.987192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"105'1","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-03-29T08:27:19.946646+0000","last_change":"2024-03-29T08:27:18.194454+0000","last_active":"2024-03-29T08:27:19.946646+0000","last_peered":"2024-03-29T08:27:19.946646+0000","last_clean":"2024-03-29T08:27:19.946646+0000","last_became_active":"2024-03-29T08:27:18.194117+0000","last_became_peered":"2024-03-29T08:27:18.194117+0000","last_unstale":"2024-03-29T08:27:19.946646+0000","last_undegraded":"2024-03-29T08:27:19.946646+0000","last_fullsized":"2024-03-29T08:27:19.946646+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:27:17.169019+0000","last_clean_scrub_stamp":"2024-03-29T08:27:17.169019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:32:14.427500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":236,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-03-29T08:27:18.591533+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:27:18.591533+0000","last_peered":"2024-03-29T08:27:18.591533+0000","last_clean":"2024-03-29T08:27:18.591533+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:27:18.591533+0000","last_undegraded":"2024-03-29T08:27:18.591533+0000","last_fullsized":"2024-03-29T08:27:18.591533+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509555,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1306624,"data_stored":563785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.72799999999999998,"15min":0.68999999999999995},"min":{"1min":0.52600000000000002,"5min":0.39200000000000002,"15min":0.34999999999999998},"max":{"1min":0.93300000000000005,"5min":0.97999999999999998,"15min":1.1639999999999999},"last":0.626},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.753,"15min":0.71299999999999997},"min":{"1min":0.624,"5min":0.55100000000000005,"15min":0.45800000000000002},"max":{"1min":0.92900000000000005,"5min":0.998,"15min":1.2410000000000001},"last":0.72399999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69599999999999995,"15min":0.67300000000000004},"min":{"1min":0.499,"5min":0.443,"15min":0.39600000000000002},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":1.202},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69599999999999995,"15min":0.66900000000000004},"min":{"1min":0.55300000000000005,"5min":0.44,"15min":0.44},"max":{"1min":0.89000000000000001,"5min":0.93400000000000005,"15min":1.1339999999999999},"last":0.67000000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.69399999999999995,"15min":0.66400000000000003},"min":{"1min":0.50700000000000001,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.86499999999999999,"5min":0.93999999999999995,"15min":1.085},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.73199999999999998,"15min":0.69899999999999995},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.288},"last":0.65500000000000003}]},{"osd":3,"last update":"Fri Mar 29 08:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.71799999999999997,"15min":0.70899999999999996},"min":{"1min":0.51100000000000001,"5min":0.496,"15min":0.46000000000000002},"max":{"1min":0.90500000000000003,"5min":0.98099999999999998,"15min":1.018},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.752,"5min":0.73399999999999999,"15min":0.73499999999999999},"min":{"1min":0.59499999999999997,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90300000000000002,"5min":0.94599999999999995,"15min":1.2609999999999999},"last":0.59499999999999997}]},{"osd":4,"last update":"Fri Mar 29 08:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.57699999999999996,"15min":0.60999999999999999},"min":{"1min":0.41499999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84799999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60799999999999998,"15min":0.67100000000000004},"min":{"1min":0.434,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.79900000000000004,"5min":0.89600000000000002,"15min":1.123},"last":0.49199999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61299999999999999,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":1.056},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.63400000000000001,"15min":0.67500000000000004},"min":{"1min":0.48799999999999999,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.83899999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.60999999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:27:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.68999999999999995,"15min":0.71499999999999997},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46600000000000003},"max":{"1min":0.96399999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.71199999999999997},"min":{"1min":0.47899999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.98299999999999998},"last":0.57899999999999996}]}]},{"osd":6,"up_from":15,"seq":64424509555,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32396,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339444,"statfs":{"total":96636764160,"available":96603590656,"internally_reserved":0,"allocated":1302528,"data_stored":563203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71999999999999997,"15min":0.67200000000000004},"min":{"1min":0.51800000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.90700000000000003,"5min":0.93100000000000005,"15min":1.0189999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.751,"15min":0.71599999999999997},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.44500000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.079},"last":0.89800000000000002}]},{"osd":1,"last update":"Fri Mar 29 08:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.67900000000000005,"15min":0.64400000000000002},"min":{"1min":0.52800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86199999999999999,"5min":0.97999999999999998,"15min":0.99299999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.70199999999999996,"15min":0.66100000000000003},"min":{"1min":0.54900000000000004,"5min":0.442,"15min":0.42899999999999999},"max":{"1min":0.92800000000000005,"5min":0.95599999999999996,"15min":1.079},"last":0.92000000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76800000000000002,"15min":0.71799999999999997},"min":{"1min":0.56999999999999995,"5min":0.51800000000000002,"15min":0.44800000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.75,"15min":0.72399999999999998},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.435},"max":{"1min":0.94899999999999995,"5min":0.95099999999999996,"15min":0.95899999999999996},"last":0.80500000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72099999999999997,"15min":0.70799999999999996},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71199999999999997,"15min":0.70199999999999996},"min":{"1min":0.51600000000000001,"5min":0.51000000000000001,"15min":0.47799999999999998},"max":{"1min":0.94199999999999995,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.94199999999999995}]},{"osd":4,"last update":"Fri Mar 29 08:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.61899999999999999,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.38100000000000001,"15min":0.36499999999999999},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.556},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.63800000000000001,"15min":0.68799999999999994},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.371},"max":{"1min":0.96299999999999997,"5min":0.996,"15min":2.0539999999999998},"last":0.71699999999999997}]},{"osd":5,"last update":"Fri Mar 29 08:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.63,"15min":0.66700000000000004},"min":{"1min":0.39000000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":1.0660000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.68899999999999995,"15min":0.70799999999999996},"min":{"1min":0.50800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":1.042,"5min":1.042,"15min":1.1419999999999999},"last":1.042}]},{"osd":7,"last update":"Fri Mar 29 08:27:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.69999999999999996,"15min":0.73199999999999998},"min":{"1min":0.46800000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":1.03},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69199999999999995,"15min":0.73699999999999999},"min":{"1min":0.46400000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":1.1160000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509554,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1306624,"data_stored":563785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.70999999999999996,"15min":0.70599999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.94799999999999995,"5min":1.0289999999999999,"15min":2.1709999999999998},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.68700000000000006,"15min":0.71999999999999997},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.93000000000000005,"5min":0.95699999999999996,"15min":2.2869999999999999},"last":0.997}]},{"osd":1,"last update":"Fri Mar 29 08:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.73899999999999999,"15min":0.753},"min":{"1min":0.48699999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.871,"5min":1.042,"15min":2.524},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.69399999999999995,"15min":0.70899999999999996},"min":{"1min":0.441,"5min":0.41099999999999998,"15min":0.40699999999999997},"max":{"1min":0.90000000000000002,"5min":0.95599999999999996,"15min":2.2679999999999998},"last":1.3149999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.70499999999999996,"15min":0.72199999999999998},"min":{"1min":0.45700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.91700000000000004,"5min":0.97799999999999998,"15min":2.363},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.73499999999999999,"15min":0.751},"min":{"1min":0.45600000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.94499999999999995,"5min":0.97799999999999998,"15min":2.1880000000000002},"last":0.88700000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.748,"15min":0.751},"min":{"1min":0.53400000000000003,"5min":0.52600000000000002,"15min":0.46300000000000002},"max":{"1min":0.96499999999999997,"5min":1.004,"15min":2.3029999999999999},"last":1.3759999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.77700000000000002,"15min":0.79700000000000004},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.96799999999999997,"5min":1.0549999999999999,"15min":2.2549999999999999},"last":1.016}]},{"osd":5,"last update":"Fri Mar 29 08:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.624,"15min":0.66600000000000004},"min":{"1min":0.41099999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.998,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.437,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.85499999999999998,"5min":0.88700000000000001,"15min":1.0149999999999999},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Mar 29 08:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65300000000000002,"15min":0.66200000000000003},"min":{"1min":0.42599999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85299999999999998,"5min":1.091,"15min":1.091},"last":0.629},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70399999999999996,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.499,"15min":0.499},"max":{"1min":0.879,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.66500000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:27:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69299999999999995,"15min":0.71599999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91800000000000004,"5min":0.95899999999999996,"15min":1.1000000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.73699999999999999,"15min":0.751},"min":{"1min":0.47999999999999998,"5min":0.44400000000000001,"15min":0.42899999999999999},"max":{"1min":1.036,"5min":1.0760000000000001,"15min":1.4430000000000001},"last":0.83799999999999997}]}]},{"osd":3,"up_from":15,"seq":64424509557,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1314816,"data_stored":563397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.44500000000000001,"15min":0.45600000000000002},"min":{"1min":0.316,"5min":0.26600000000000001,"15min":0.26200000000000001},"max":{"1min":0.61299999999999999,"5min":0.84799999999999998,"15min":1.123},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52900000000000003,"15min":0.52500000000000002},"min":{"1min":0.38400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.751,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri Mar 29 08:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45000000000000001,"15min":0.45500000000000002},"min":{"1min":0.32000000000000001,"5min":0.30199999999999999,"15min":0.26500000000000001},"max":{"1min":0.63,"5min":0.78900000000000003,"15min":1.1599999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.51400000000000001,"15min":0.52400000000000002},"min":{"1min":0.41299999999999998,"5min":0.32100000000000001,"15min":0.28599999999999998},"max":{"1min":0.78500000000000003,"5min":1.004,"15min":1.4790000000000001},"last":0.41299999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.53600000000000003,"15min":0.55500000000000005},"min":{"1min":0.34000000000000002,"5min":0.33200000000000002,"15min":0.309},"max":{"1min":0.80300000000000005,"5min":0.90000000000000002,"15min":1.2529999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60099999999999998,"15min":0.63600000000000001},"min":{"1min":0.48099999999999998,"5min":0.44700000000000001,"15min":0.41799999999999998},"max":{"1min":0.83599999999999997,"5min":0.97699999999999998,"15min":1.8580000000000001},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.76500000000000001,"15min":0.78200000000000003},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.42199999999999999},"max":{"1min":0.93999999999999995,"5min":1.0449999999999999,"15min":1.6419999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.76800000000000002,"15min":0.77200000000000002},"min":{"1min":0.56599999999999995,"5min":0.46999999999999997,"15min":0.45400000000000001},"max":{"1min":0.96699999999999997,"5min":1.3340000000000001,"15min":1.391},"last":0.67600000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80700000000000005,"15min":0.80500000000000005},"min":{"1min":0.66400000000000003,"5min":0.56999999999999995,"15min":0.52100000000000002},"max":{"1min":0.97399999999999998,"5min":1.1850000000000001,"15min":1.671},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81599999999999995,"15min":0.84599999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51400000000000001},"max":{"1min":1.016,"5min":1.2050000000000001,"15min":1.7110000000000001},"last":0.73499999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78100000000000003,"15min":0.81100000000000005},"min":{"1min":0.60699999999999998,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.96099999999999997,"5min":1.2609999999999999,"15min":1.669},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.80700000000000005},"min":{"1min":0.55000000000000004,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0509999999999999,"5min":1.119,"15min":2.0150000000000001},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:26:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.79900000000000004,"15min":0.83699999999999997},"min":{"1min":0.55900000000000005,"5min":0.55000000000000004,"15min":0.48799999999999999},"max":{"1min":1.073,"5min":1.3169999999999999,"15min":1.887},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.79500000000000004,"15min":0.83199999999999996},"min":{"1min":0.58299999999999996,"5min":0.52200000000000002,"15min":0.47999999999999998},"max":{"1min":0.999,"5min":1.1259999999999999,"15min":2.044},"last":0.78900000000000003}]}]},{"osd":2,"up_from":15,"seq":64424509556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1294336,"data_stored":556973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46200000000000002,"15min":0.48199999999999998},"min":{"1min":0.25800000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.59599999999999997,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.55000000000000004,"15min":0.52000000000000002},"min":{"1min":0.38700000000000001,"5min":0.34699999999999998,"15min":0.32300000000000001},"max":{"1min":0.72599999999999998,"5min":2.0880000000000001,"15min":2.0880000000000001},"last":0.46600000000000003}]},{"osd":1,"last update":"Fri Mar 29 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.49299999999999999,"15min":0.495},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.65000000000000002,"5min":1.399,"15min":1.399},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.51900000000000002,"15min":0.50800000000000001},"min":{"1min":0.32700000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70399999999999996,"5min":3.0459999999999998,"15min":3.0459999999999998},"last":0.58499999999999996}]},{"osd":3,"last update":"Fri Mar 29 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53900000000000003,"15min":0.51300000000000001},"min":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.34000000000000002},"max":{"1min":0.76000000000000001,"5min":2.1280000000000001,"15min":2.1280000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.625,"15min":0.59499999999999997},"min":{"1min":0.47299999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81000000000000005,"5min":2.9950000000000001,"15min":2.9950000000000001},"last":0.54000000000000004}]},{"osd":4,"last update":"Fri Mar 29 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79700000000000004,"15min":0.77700000000000002},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.49099999999999999},"max":{"1min":1.109,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.77100000000000002,"15min":0.76500000000000001},"min":{"1min":0.59199999999999997,"5min":0.51700000000000002,"15min":0.435},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.76600000000000001}]},{"osd":5,"last update":"Fri Mar 29 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.83499999999999996,"5min":0.84999999999999998,"15min":0.84199999999999997},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.55500000000000005},"max":{"1min":1.0149999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.628},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.81499999999999995,"15min":0.83599999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.48799999999999999},"max":{"1min":1.1759999999999999,"5min":1.194,"15min":1.526},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.78800000000000003,"15min":0.78100000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.48599999999999999},"max":{"1min":1.131,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.82799999999999996,"15min":0.79900000000000004},"min":{"1min":0.64100000000000001,"5min":0.57799999999999996,"15min":0.52500000000000002},"max":{"1min":1.0640000000000001,"5min":1.524,"15min":1.524},"last":0.69999999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:27:12 2024","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.79800000000000004,"15min":0.80200000000000005},"min":{"1min":0.56799999999999995,"5min":0.54900000000000004,"15min":0.505},"max":{"1min":1.1259999999999999,"5min":1.296,"15min":1.319},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81499999999999995,"15min":0.79900000000000004},"min":{"1min":0.58999999999999997,"5min":0.56299999999999994,"15min":0.54900000000000004},"max":{"1min":1.0860000000000001,"5min":2,"15min":2},"last":0.57399999999999995}]}]},{"osd":1,"up_from":15,"seq":64424509556,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32980,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338860,"statfs":{"total":96636764160,"available":96602992640,"internally_reserved":0,"allocated":1904640,"data_stored":1148947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44900000000000001,"15min":0.54700000000000004},"min":{"1min":0.31900000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.61499999999999999,"5min":0.75900000000000001,"15min":4.2539999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.48399999999999999,"15min":0.57799999999999996},"min":{"1min":0.317,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67900000000000005,"5min":0.73299999999999998,"15min":4.1909999999999998},"last":0.317}]},{"osd":2,"last update":"Fri Mar 29 08:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.501,"15min":0.502},"min":{"1min":0.33400000000000002,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.67900000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.53000000000000003,"15min":0.53200000000000003},"min":{"1min":0.42699999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67800000000000005,"5min":0.79000000000000004,"15min":0.88200000000000001},"last":0.54800000000000004}]},{"osd":3,"last update":"Fri Mar 29 08:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59199999999999997,"15min":0.58499999999999996},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88300000000000001,"5min":1.52,"15min":1.52},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.53000000000000003,"15min":0.52600000000000002},"min":{"1min":0.374,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.64900000000000002,"5min":1.331,"15min":1.331},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.78000000000000003,"15min":0.753},"min":{"1min":0.52400000000000002,"5min":0.51800000000000002,"15min":0.46200000000000002},"max":{"1min":0.86599999999999999,"5min":1.117,"15min":1.117},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.77400000000000002,"15min":0.77700000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.47999999999999998},"max":{"1min":1,"5min":1.175,"15min":1.23},"last":0.58099999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78800000000000003,"15min":0.77900000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":0.94799999999999995,"5min":1.1339999999999999,"15min":1.1499999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.78700000000000003,"15min":0.76700000000000002},"min":{"1min":0.55100000000000005,"5min":0.53700000000000003,"15min":0.46500000000000002},"max":{"1min":0.94599999999999995,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.65600000000000003}]},{"osd":6,"last update":"Fri Mar 29 08:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.77800000000000002,"15min":0.77000000000000002},"min":{"1min":0.54900000000000004,"5min":0.505,"15min":0.5},"max":{"1min":0.90100000000000002,"5min":1.212,"15min":1.212},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80800000000000005,"15min":0.82999999999999996},"min":{"1min":0.56399999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.91900000000000004,"5min":1.22,"15min":1.22},"last":0.63300000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:27:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.80300000000000005,"15min":0.80000000000000004},"min":{"1min":0.60299999999999998,"5min":0.56100000000000005,"15min":0.52100000000000002},"max":{"1min":0.93000000000000005,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.79900000000000004,"15min":0.78900000000000003},"min":{"1min":0.59499999999999997,"5min":0.503,"15min":0.503},"max":{"1min":0.97699999999999998,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.74099999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509555,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1302528,"data_stored":563203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:27:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.44800000000000001,"15min":0.53300000000000003},"min":{"1min":0.30299999999999999,"5min":0.28499999999999998,"15min":0.22800000000000001},"max":{"1min":0.70499999999999996,"5min":0.75800000000000001,"15min":3.9359999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.49299999999999999,"15min":0.69699999999999995},"min":{"1min":0.34799999999999998,"5min":0.28499999999999998,"15min":0.253},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":8.0419999999999998},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:27:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.51400000000000001,"15min":0.53200000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34599999999999997},"max":{"1min":0.746,"5min":0.78000000000000003,"15min":1.258},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.65100000000000002},"min":{"1min":0.41399999999999998,"5min":0.32800000000000001,"15min":0.312},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":4.907},"last":0.40300000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:27:13 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.504,"15min":0.69999999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.28499999999999998},"max":{"1min":0.71899999999999997,"5min":0.83399999999999996,"15min":7.9569999999999999},"last":0.5},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57499999999999996,"15min":0.621},"min":{"1min":0.41899999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.87,"5min":0.87,"15min":1.397},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Mar 29 08:27:13 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.78800000000000003,"15min":0.90100000000000002},"min":{"1min":0.64700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.98299999999999998,"5min":1.4399999999999999,"15min":4.4720000000000004},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78200000000000003,"15min":0.88400000000000001},"min":{"1min":0.496,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.153,"5min":1.153,"15min":4.3920000000000003},"last":0.65000000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:27:13 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.78500000000000003,"15min":0.86699999999999999},"min":{"1min":0.61299999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.038,"5min":1.349,"15min":4.3200000000000003},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79600000000000004,"15min":0.83099999999999996},"min":{"1min":0.58799999999999997,"5min":0.55700000000000005,"15min":0.54600000000000004},"max":{"1min":1.079,"5min":1.26,"15min":1.6759999999999999},"last":0.78700000000000003}]},{"osd":6,"last update":"Fri Mar 29 08:27:13 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.78900000000000003,"15min":0.83199999999999996},"min":{"1min":0.57699999999999996,"5min":0.54900000000000004,"15min":0.54400000000000004},"max":{"1min":1.006,"5min":1.1579999999999999,"15min":1.5920000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.80900000000000005,"15min":0.90800000000000003},"min":{"1min":0.60099999999999998,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0569999999999999,"5min":1.1990000000000001,"15min":4.4100000000000001},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Mar 29 08:27:13 2024","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81100000000000005,"15min":0.92200000000000004},"min":{"1min":0.67000000000000004,"5min":0.54100000000000004,"15min":0.496},"max":{"1min":1.119,"5min":1.3959999999999999,"15min":4.5209999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.80100000000000005,"15min":0.90400000000000003},"min":{"1min":0.55400000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1000000000000001,"5min":1.2390000000000001,"15min":4.4420000000000002},"last":0.79700000000000004}]}]},{"osd":5,"up_from":14,"seq":60129542260,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32980,"kb_used_data":1860,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338860,"statfs":{"total":96636764160,"available":96602992640,"internally_reserved":0,"allocated":1904640,"data_stored":1148891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69399999999999995,"15min":0.65300000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.88100000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.71399999999999997,"15min":0.67400000000000004},"min":{"1min":0.56699999999999995,"5min":0.46200000000000002,"15min":0.39800000000000002},"max":{"1min":0.91500000000000004,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.79400000000000004}]},{"osd":1,"last update":"Fri Mar 29 08:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68400000000000005,"15min":0.71599999999999997},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88,"5min":0.90200000000000002,"15min":2.5129999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.72099999999999997,"15min":0.69199999999999995},"min":{"1min":0.51300000000000001,"5min":0.47199999999999998,"15min":0.40500000000000003},"max":{"1min":0.89600000000000002,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.875}]},{"osd":2,"last update":"Fri Mar 29 08:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.68799999999999994,"15min":0.67800000000000005},"min":{"1min":0.53600000000000003,"5min":0.46700000000000003,"15min":0.42499999999999999},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71599999999999997,"15min":0.70799999999999996},"min":{"1min":0.47499999999999998,"5min":0.44900000000000001,"15min":0.39200000000000002},"max":{"1min":0.92900000000000005,"5min":1.002,"15min":1.0209999999999999},"last":0.90300000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.63300000000000001,"5min":0.49399999999999999,"15min":0.47099999999999997},"max":{"1min":0.874,"5min":1.038,"15min":1.038},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.72099999999999997,"15min":0.73899999999999999},"min":{"1min":0.60299999999999998,"5min":0.47799999999999998,"15min":0.435},"max":{"1min":0.95099999999999996,"5min":1.0209999999999999,"15min":1.954},"last":0.83399999999999996}]},{"osd":4,"last update":"Fri Mar 29 08:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.61899999999999999,"15min":0.64200000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.93500000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61899999999999999,"15min":0.64700000000000002},"min":{"1min":0.41799999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.89200000000000002,"5min":0.94699999999999995,"15min":1.0820000000000001},"last":0.65500000000000003}]},{"osd":6,"last update":"Fri Mar 29 08:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.623,"15min":0.66000000000000003},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":1.071},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.66600000000000004,"15min":0.68799999999999994},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.432},"max":{"1min":0.90800000000000003,"5min":0.91100000000000003,"15min":1.0609999999999999},"last":0.64100000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.66800000000000004,"15min":0.69999999999999996},"min":{"1min":0.46000000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.1200000000000001},"last":0.754},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70999999999999996,"15min":0.73899999999999999},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.04},"last":0.72099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:27:23.079 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T08:27:23.079 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-29T08:27:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.259+0000 7f5c48000640 1 Processor -- start 2024-03-29T08:27:23.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.259+0000 7f5c48000640 1 -- start start 2024-03-29T08:27:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c48000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c40142ab0 0x7f5c4006d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c48000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c4015db10 0x7f5c400b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:23.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c48000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4006de40 con 0x7f5c4006d9e0 2024-03-29T08:27:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c48000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c40070c70 con 0x7f5c4015db10 2024-03-29T08:27:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c48000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c40070340 con 0x7f5c40142ab0 2024-03-29T08:27:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c46000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c4015db10 0x7f5c400b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c46000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5c4015db10 0x7f5c400b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33694/0 (socket says 172.21.15.46:33694) 2024-03-29T08:27:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c46000640 1 -- 172.21.15.46:0/4226494532 learned_addr learned my addr 172.21.15.46:0/4226494532 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c45600640 1 --2- 172.21.15.46:0/4226494532 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c40142ab0 0x7f5c4006d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c46000640 1 -- 172.21.15.46:0/4226494532 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c40142ab0 msgr2=0x7f5c4006d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c46000640 1 --2- 172.21.15.46:0/4226494532 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c40142ab0 0x7f5c4006d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c46000640 1 -- 172.21.15.46:0/4226494532 >> v1:172.21.15.73:6789/0 conn(0x7f5c4006d9e0 legacy=0x7f5c4015d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c46000640 1 -- 172.21.15.46:0/4226494532 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c40070620 con 0x7f5c4015db10 2024-03-29T08:27:23.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c45600640 1 --2- 172.21.15.46:0/4226494532 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c40142ab0 0x7f5c4006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c46000640 1 --2- 172.21.15.46:0/4226494532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4015db10 0x7f5c400b1e60 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f5c38009e50 tx=0x7f5c3801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=160cd10eeeafb55 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c3fe00640 1 -- 172.21.15.46:0/4226494532 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c380a9440 con 0x7f5c4015db10 2024-03-29T08:27:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c3fe00640 1 -- 172.21.15.46:0/4226494532 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c380a95a0 con 0x7f5c4015db10 2024-03-29T08:27:23.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.263+0000 7f5c3fe00640 1 -- 172.21.15.46:0/4226494532 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c380a9930 con 0x7f5c4015db10 2024-03-29T08:27:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c48000640 1 -- 172.21.15.46:0/4226494532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4015db10 msgr2=0x7f5c400b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c48000640 1 --2- 172.21.15.46:0/4226494532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4015db10 0x7f5c400b1e60 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f5c38009e50 tx=0x7f5c3801c9f0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c48000640 1 -- 172.21.15.46:0/4226494532 shutdown_connections 2024-03-29T08:27:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c48000640 1 --2- 172.21.15.46:0/4226494532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4015db10 0x7f5c400b1e60 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c48000640 1 --2- 172.21.15.46:0/4226494532 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c40142ab0 0x7f5c4006d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c48000640 1 -- 172.21.15.46:0/4226494532 >> 172.21.15.46:0/4226494532 conn(0x7f5c40066080 msgr2=0x7f5c40069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c48000640 1 -- 172.21.15.46:0/4226494532 shutdown_connections 2024-03-29T08:27:23.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c48000640 1 -- 172.21.15.46:0/4226494532 wait complete. 2024-03-29T08:27:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c48000640 1 Processor -- start 2024-03-29T08:27:23.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c48000640 1 -- start start 2024-03-29T08:27:23.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c48000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4006d9e0 0x7f5c40158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:23.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c48000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c40142ab0 0x7f5c4013d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:23.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c48000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c4015db10 0x7f5c4013e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:23.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c48000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c400a74a0 con 0x7f5c4006d9e0 2024-03-29T08:27:23.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c48000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4013cab0 con 0x7f5c40142ab0 2024-03-29T08:27:23.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c48000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c400a76b0 con 0x7f5c4015db10 2024-03-29T08:27:23.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c45600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4006d9e0 0x7f5c40158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:23.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c45600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4006d9e0 0x7f5c40158280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33696/0 (socket says 172.21.15.46:33696) 2024-03-29T08:27:23.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c45600640 1 -- 172.21.15.46:0/110279622 learned_addr learned my addr 172.21.15.46:0/110279622 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:23.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c46000640 1 --2- 172.21.15.46:0/110279622 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c4015db10 0x7f5c4013e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:23.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c44c00640 1 --2- 172.21.15.46:0/110279622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c40142ab0 0x7f5c4013d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:23.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c45600640 1 -- 172.21.15.46:0/110279622 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c4015db10 msgr2=0x7f5c4013e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:23.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c45600640 1 --2- 172.21.15.46:0/110279622 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c4015db10 0x7f5c4013e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c45600640 1 -- 172.21.15.46:0/110279622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c40142ab0 msgr2=0x7f5c4013d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:23.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c45600640 1 --2- 172.21.15.46:0/110279622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c40142ab0 0x7f5c4013d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.267+0000 7f5c45600640 1 -- 172.21.15.46:0/110279622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c38009b30 con 0x7f5c4006d9e0 2024-03-29T08:27:23.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.271+0000 7f5c45600640 1 --2- 172.21.15.46:0/110279622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4006d9e0 0x7f5c40158280 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400c7c0 tx=0x7f5c3400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:23.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.271+0000 7f5c3d600640 1 -- 172.21.15.46:0/110279622 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34007a30 con 0x7f5c4006d9e0 2024-03-29T08:27:23.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.271+0000 7f5c3d600640 1 -- 172.21.15.46:0/110279622 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c34007b90 con 0x7f5c4006d9e0 2024-03-29T08:27:23.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.271+0000 7f5c3d600640 1 -- 172.21.15.46:0/110279622 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3400ce70 con 0x7f5c4006d9e0 2024-03-29T08:27:23.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.271+0000 7f5c48000640 1 -- 172.21.15.46:0/110279622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c400a7990 con 0x7f5c4006d9e0 2024-03-29T08:27:23.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.271+0000 7f5c48000640 1 -- 172.21.15.46:0/110279622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4013cfe0 con 0x7f5c4006d9e0 2024-03-29T08:27:23.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.271+0000 7f5c48000640 1 -- 172.21.15.46:0/110279622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c400671b0 con 0x7f5c4006d9e0 2024-03-29T08:27:23.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.271+0000 7f5c3d600640 1 -- 172.21.15.46:0/110279622 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5c340113f0 con 0x7f5c4006d9e0 2024-03-29T08:27:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.271+0000 7f5c3d600640 1 --2- 172.21.15.46:0/110279622 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c20066690 0x7f5c20068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.271+0000 7f5c3d600640 1 -- 172.21.15.46:0/110279622 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c340846b0 con 0x7f5c4006d9e0 2024-03-29T08:27:23.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.275+0000 7f5c44c00640 1 --2- 172.21.15.46:0/110279622 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c20066690 0x7f5c20068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:23.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.275+0000 7f5c44c00640 1 --2- 172.21.15.46:0/110279622 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c20066690 0x7f5c20068b50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5c4015bed0 tx=0x7f5c2c007e40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:23.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.279+0000 7f5c3d600640 1 -- 172.21.15.46:0/110279622 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5c340519a0 con 0x7f5c4006d9e0 2024-03-29T08:27:23.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.495+0000 7f5c48000640 1 -- 172.21.15.46:0/110279622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5c400a7b80 con 0x7f5c4006d9e0 2024-03-29T08:27:23.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.499+0000 7f5c3d600640 1 -- 172.21.15.46:0/110279622 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f5c34054fc0 con 0x7f5c4006d9e0 2024-03-29T08:27:23.500 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-03-29T08:27:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.499+0000 7f5c48000640 1 -- 172.21.15.46:0/110279622 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c20066690 msgr2=0x7f5c20068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.499+0000 7f5c48000640 1 --2- 172.21.15.46:0/110279622 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c20066690 0x7f5c20068b50 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5c4015bed0 tx=0x7f5c2c007e40 comp rx=0 tx=0).stop 2024-03-29T08:27:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.499+0000 7f5c48000640 1 -- 172.21.15.46:0/110279622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4006d9e0 msgr2=0x7f5c40158280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.499+0000 7f5c48000640 1 --2- 172.21.15.46:0/110279622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4006d9e0 0x7f5c40158280 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400c7c0 tx=0x7f5c3400cc80 comp rx=0 tx=0).stop 2024-03-29T08:27:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.499+0000 7f5c48000640 1 -- 172.21.15.46:0/110279622 shutdown_connections 2024-03-29T08:27:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.499+0000 7f5c48000640 1 --2- 172.21.15.46:0/110279622 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5c20066690 0x7f5c20068b50 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.499+0000 7f5c48000640 1 --2- 172.21.15.46:0/110279622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c40142ab0 0x7f5c4013d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.499+0000 7f5c48000640 1 --2- 172.21.15.46:0/110279622 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5c4015db10 0x7f5c4013e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.499+0000 7f5c48000640 1 --2- 172.21.15.46:0/110279622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5c4006d9e0 0x7f5c40158280 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.499+0000 7f5c48000640 1 -- 172.21.15.46:0/110279622 >> 172.21.15.46:0/110279622 conn(0x7f5c40066080 msgr2=0x7f5c400cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.499+0000 7f5c48000640 1 -- 172.21.15.46:0/110279622 shutdown_connections 2024-03-29T08:27:23.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.499+0000 7f5c48000640 1 -- 172.21.15.46:0/110279622 wait complete. 2024-03-29T08:27:23.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:27:23.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.663+0000 7f4406a00640 1 Processor -- start 2024-03-29T08:27:23.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.663+0000 7f4406a00640 1 -- start start 2024-03-29T08:27:23.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.663+0000 7f4406a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f44001605f0 0x7f4400069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:23.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.663+0000 7f4406a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4400160a80 0x7f440006a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:23.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.663+0000 7f4406a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4400161020 con 0x7f440015feb0 2024-03-29T08:27:23.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.663+0000 7f4406a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4400161300 con 0x7f44001605f0 2024-03-29T08:27:23.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.663+0000 7f4406a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4400070b70 con 0x7f4400160a80 2024-03-29T08:27:23.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.663+0000 7f43f7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f44001605f0 0x7f4400069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:23.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.663+0000 7f43f7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f44001605f0 0x7f4400069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33698/0 (socket says 172.21.15.46:33698) 2024-03-29T08:27:23.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.663+0000 7f43f7400640 1 -- 172.21.15.46:0/3137363138 learned_addr learned my addr 172.21.15.46:0/3137363138 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:23.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.663+0000 7f4404a00640 1 --2- 172.21.15.46:0/3137363138 >> v2:172.21.15.73:3300/0 conn(0x7f4400160a80 0x7f440006a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:23.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.663+0000 7f43f7400640 1 -- 172.21.15.46:0/3137363138 >> v2:172.21.15.73:3300/0 conn(0x7f4400160a80 msgr2=0x7f440006a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:23.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.663+0000 7f43f7400640 1 --2- 172.21.15.46:0/3137363138 >> v2:172.21.15.73:3300/0 conn(0x7f4400160a80 0x7f440006a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.663+0000 7f43f7400640 1 -- 172.21.15.46:0/3137363138 >> v1:172.21.15.73:6789/0 conn(0x7f440015feb0 legacy=0x7f4400071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:23.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.663+0000 7f43f7400640 1 -- 172.21.15.46:0/3137363138 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4400070e50 con 0x7f44001605f0 2024-03-29T08:27:23.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.663+0000 7f4404a00640 1 --2- 172.21.15.46:0/3137363138 >> v2:172.21.15.73:3300/0 conn(0x7f4400160a80 0x7f440006a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:23.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.667+0000 7f43f7400640 1 --2- 172.21.15.46:0/3137363138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44001605f0 0x7f4400069d90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009e50 tx=0x7f43e801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d887fa9fac91c0d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:23.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.667+0000 7f43ff400640 1 -- 172.21.15.46:0/3137363138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e80a5890 con 0x7f44001605f0 2024-03-29T08:27:23.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.667+0000 7f43ff400640 1 -- 172.21.15.46:0/3137363138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e80a59f0 con 0x7f44001605f0 2024-03-29T08:27:23.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.667+0000 7f43ff400640 1 -- 172.21.15.46:0/3137363138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e80a5d80 con 0x7f44001605f0 2024-03-29T08:27:23.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.667+0000 7f4406a00640 1 -- 172.21.15.46:0/3137363138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44001605f0 msgr2=0x7f4400069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:23.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.667+0000 7f4406a00640 1 --2- 172.21.15.46:0/3137363138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44001605f0 0x7f4400069d90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009e50 tx=0x7f43e801c9b0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.667+0000 7f4406a00640 1 -- 172.21.15.46:0/3137363138 shutdown_connections 2024-03-29T08:27:23.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.667+0000 7f4406a00640 1 --2- 172.21.15.46:0/3137363138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44001605f0 0x7f4400069d90 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.667+0000 7f4406a00640 1 --2- 172.21.15.46:0/3137363138 >> v2:172.21.15.73:3300/0 conn(0x7f4400160a80 0x7f440006a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.667+0000 7f4406a00640 1 -- 172.21.15.46:0/3137363138 >> 172.21.15.46:0/3137363138 conn(0x7f44000ced50 msgr2=0x7f44000cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:23.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.667+0000 7f4406a00640 1 -- 172.21.15.46:0/3137363138 shutdown_connections 2024-03-29T08:27:23.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.667+0000 7f4406a00640 1 -- 172.21.15.46:0/3137363138 wait complete. 2024-03-29T08:27:23.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f4406a00640 1 Processor -- start 2024-03-29T08:27:23.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f4406a00640 1 -- start start 2024-03-29T08:27:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f4406a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f440015feb0 0x7f440014c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:23.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f4406a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44001605f0 0x7f4400151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f4406a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4400160a80 0x7f4400152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f4406a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440014e7f0 con 0x7f440015feb0 2024-03-29T08:27:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f4406a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440014cf40 con 0x7f4400160a80 2024-03-29T08:27:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f4406a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440014cd00 con 0x7f44001605f0 2024-03-29T08:27:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f43ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f440015feb0 0x7f440014c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f43ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f440015feb0 0x7f440014c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33714/0 (socket says 172.21.15.46:33714) 2024-03-29T08:27:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f43ffe00640 1 -- 172.21.15.46:0/132421504 learned_addr learned my addr 172.21.15.46:0/132421504 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f4404a00640 1 --2- 172.21.15.46:0/132421504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4400160a80 0x7f4400152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f43f7400640 1 --2- 172.21.15.46:0/132421504 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44001605f0 0x7f4400151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f43ffe00640 1 -- 172.21.15.46:0/132421504 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44001605f0 msgr2=0x7f4400151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:23.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f43ffe00640 1 --2- 172.21.15.46:0/132421504 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44001605f0 0x7f4400151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f43ffe00640 1 -- 172.21.15.46:0/132421504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4400160a80 msgr2=0x7f4400152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:23.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f43ffe00640 1 --2- 172.21.15.46:0/132421504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4400160a80 0x7f4400152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f43ffe00640 1 -- 172.21.15.46:0/132421504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e8009b30 con 0x7f440015feb0 2024-03-29T08:27:23.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f43ffe00640 1 --2- 172.21.15.46:0/132421504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f440015feb0 0x7f440014c7c0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f43f000c450 tx=0x7f43f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:23.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f43fcc00640 1 -- 172.21.15.46:0/132421504 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0012830 con 0x7f440015feb0 2024-03-29T08:27:23.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f43fcc00640 1 -- 172.21.15.46:0/132421504 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f00129b0 con 0x7f440015feb0 2024-03-29T08:27:23.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f43fcc00640 1 -- 172.21.15.46:0/132421504 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f0012d60 con 0x7f440015feb0 2024-03-29T08:27:23.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f4406a00640 1 -- 172.21.15.46:0/132421504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440014d190 con 0x7f440015feb0 2024-03-29T08:27:23.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.671+0000 7f4406a00640 1 -- 172.21.15.46:0/132421504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440014d750 con 0x7f440015feb0 2024-03-29T08:27:23.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.675+0000 7f4406a00640 1 -- 172.21.15.46:0/132421504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4400066c90 con 0x7f440015feb0 2024-03-29T08:27:23.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.675+0000 7f43fcc00640 1 -- 172.21.15.46:0/132421504 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f43f001c460 con 0x7f440015feb0 2024-03-29T08:27:23.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.675+0000 7f43fcc00640 1 --2- 172.21.15.46:0/132421504 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f43dc066690 0x7f43dc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:23.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.675+0000 7f43fcc00640 1 -- 172.21.15.46:0/132421504 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43f0083f90 con 0x7f440015feb0 2024-03-29T08:27:23.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.675+0000 7f43f7400640 1 --2- 172.21.15.46:0/132421504 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f43dc066690 0x7f43dc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:23.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.675+0000 7f43f7400640 1 --2- 172.21.15.46:0/132421504 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f43dc066690 0x7f43dc068b50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f43e80ae000 tx=0x7f43e80a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:23.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.679+0000 7f43fcc00640 1 -- 172.21.15.46:0/132421504 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f43f0051290 con 0x7f440015feb0 2024-03-29T08:27:23.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.871+0000 7f4406a00640 1 -- 172.21.15.46:0/132421504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44000cb540 con 0x7f440015feb0 2024-03-29T08:27:23.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.871+0000 7f43fcc00640 1 -- 172.21.15.46:0/132421504 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4730 (secure 0 0 0) 0x7f43f00548b0 con 0x7f440015feb0 2024-03-29T08:27:23.875 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:23.875 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4230030957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4230030957},{"type":"v1","addr":"172.21.15.73:6836","nonce":4230030957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/297138217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":297138217},{"type":"v1","addr":"172.21.15.73:6837","nonce":297138217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3856305332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3856305332},{"type":"v1","addr":"172.21.15.46:6833","nonce":3856305332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:27:18.583223+0000","modified":"2024-03-29T08:27:21.592418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7966},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7966":{"gid":7966,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/824184385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":824184385},{"type":"v1","addr":"172.21.15.46:6835","nonce":824184385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-29T08:27:23.876 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 77 2024-03-29T08:27:23.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.875+0000 7f4406a00640 1 -- 172.21.15.46:0/132421504 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f43dc066690 msgr2=0x7f43dc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:23.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.875+0000 7f4406a00640 1 --2- 172.21.15.46:0/132421504 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f43dc066690 0x7f43dc068b50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f43e80ae000 tx=0x7f43e80a7070 comp rx=0 tx=0).stop 2024-03-29T08:27:23.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.875+0000 7f4406a00640 1 -- 172.21.15.46:0/132421504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f440015feb0 msgr2=0x7f440014c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:23.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.875+0000 7f4406a00640 1 --2- 172.21.15.46:0/132421504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f440015feb0 0x7f440014c7c0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f43f000c450 tx=0x7f43f000c910 comp rx=0 tx=0).stop 2024-03-29T08:27:23.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.875+0000 7f4406a00640 1 -- 172.21.15.46:0/132421504 shutdown_connections 2024-03-29T08:27:23.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.875+0000 7f4406a00640 1 --2- 172.21.15.46:0/132421504 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f43dc066690 0x7f43dc068b50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.875+0000 7f4406a00640 1 --2- 172.21.15.46:0/132421504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4400160a80 0x7f4400152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.875+0000 7f4406a00640 1 --2- 172.21.15.46:0/132421504 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f44001605f0 0x7f4400151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.875+0000 7f4406a00640 1 --2- 172.21.15.46:0/132421504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f440015feb0 0x7f440014c7c0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:23.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.875+0000 7f4406a00640 1 -- 172.21.15.46:0/132421504 >> 172.21.15.46:0/132421504 conn(0x7f44000ced50 msgr2=0x7f44000cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:23.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.875+0000 7f4406a00640 1 -- 172.21.15.46:0/132421504 shutdown_connections 2024-03-29T08:27:23.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:23.875+0000 7f4406a00640 1 -- 172.21.15.46:0/132421504 wait complete. 2024-03-29T08:27:24.299 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:27:18.583223+0000', 'modified': '2024-03-29T08:27:21.592418+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7966}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7966': {'gid': 7966, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/824184385', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 824184385}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 824184385}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:27:24.299 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T08:27:24.299 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-29T08:27:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8223c00640 1 Processor -- start 2024-03-29T08:27:24.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8223c00640 1 -- start start 2024-03-29T08:27:24.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8223c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f821c098e20 0x7f821c0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8223c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f821c0b1b50 0x7f821c0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8223c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821c0cd160 con 0x7f821c15aa20 2024-03-29T08:27:24.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8223c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f821c0cd3b0 con 0x7f821c0b1b50 2024-03-29T08:27:24.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8223c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0cd690 con 0x7f821c098e20 2024-03-29T08:27:24.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8221200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f821c15aa20 0x7f821c0cb000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38852/0 (socket says 172.21.15.46:38852) 2024-03-29T08:27:24.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8221200640 1 -- 172.21.15.46:0/869262230 learned_addr learned my addr 172.21.15.46:0/869262230 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:24.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f821be00640 1 -- 172.21.15.46:0/869262230 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554406477 0 0) 0x7f821c0cd160 con 0x7f821c15aa20 2024-03-29T08:27:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f821be00640 1 -- 172.21.15.46:0/869262230 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8200003530 con 0x7f821c15aa20 2024-03-29T08:27:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8213e00640 1 --2- 172.21.15.46:0/869262230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f821c098e20 0x7f821c0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8221c00640 1 --2- 172.21.15.46:0/869262230 >> v2:172.21.15.73:3300/0 conn(0x7f821c0b1b50 0x7f821c0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f821be00640 1 -- 172.21.15.46:0/869262230 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 596646568 0 0) 0x7f8200003530 con 0x7f821c15aa20 2024-03-29T08:27:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f821be00640 1 -- 172.21.15.46:0/869262230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f821c098e20 msgr2=0x7f821c0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f821be00640 1 --2- 172.21.15.46:0/869262230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f821c098e20 0x7f821c0ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f821be00640 1 -- 172.21.15.46:0/869262230 >> v2:172.21.15.73:3300/0 conn(0x7f821c0b1b50 msgr2=0x7f821c0ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f821be00640 1 --2- 172.21.15.46:0/869262230 >> v2:172.21.15.73:3300/0 conn(0x7f821c0b1b50 0x7f821c0ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f821be00640 1 -- 172.21.15.46:0/869262230 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c0cd970 con 0x7f821c15aa20 2024-03-29T08:27:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f821be00640 1 -- 172.21.15.46:0/869262230 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f820c001680 con 0x7f821c15aa20 2024-03-29T08:27:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f821be00640 1 -- 172.21.15.46:0/869262230 >> v1:172.21.15.46:6789/0 conn(0x7f821c15aa20 legacy=0x7f821c0cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f821be00640 1 --2- 172.21.15.46:0/869262230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8200003980 0x7f8200003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f821be00640 1 --2- 172.21.15.46:0/869262230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000047d0 0x7f8200004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f821be00640 1 --2- 172.21.15.46:0/869262230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f821c098e20 0x7f8200005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f821be00640 1 -- 172.21.15.46:0/869262230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0cd3b0 con 0x7f82000047d0 2024-03-29T08:27:24.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f821be00640 1 -- 172.21.15.46:0/869262230 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0cd690 con 0x7f8200003980 2024-03-29T08:27:24.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f821be00640 1 -- 172.21.15.46:0/869262230 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820000ba40 con 0x7f821c098e20 2024-03-29T08:27:24.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8221c00640 1 --2- 172.21.15.46:0/869262230 >> v2:172.21.15.73:3300/0 conn(0x7f821c0b1b50 0x7f821c0ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:24.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8213e00640 1 --2- 172.21.15.46:0/869262230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000047d0 0x7f8200004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8221200640 1 --2- 172.21.15.46:0/869262230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8200003980 0x7f8200003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8221c00640 1 --2- 172.21.15.46:0/869262230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f821c098e20 0x7f8200005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8213e00640 1 -- 172.21.15.46:0/869262230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f821c098e20 msgr2=0x7f8200005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:24.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8213e00640 1 --2- 172.21.15.46:0/869262230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f821c098e20 0x7f8200005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8213e00640 1 -- 172.21.15.46:0/869262230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8200003980 msgr2=0x7f8200003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:24.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8213e00640 1 --2- 172.21.15.46:0/869262230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8200003980 0x7f8200003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.443+0000 7f8213e00640 1 -- 172.21.15.46:0/869262230 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8200003530 con 0x7f82000047d0 2024-03-29T08:27:24.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.447+0000 7f8221c00640 1 --2- 172.21.15.46:0/869262230 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f821c098e20 0x7f8200005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:24.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.447+0000 7f8213e00640 1 --2- 172.21.15.46:0/869262230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000047d0 0x7f8200004c50 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f821c099220 tx=0x7f820401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb338d4b7e4d13b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:24.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.447+0000 7f821be00640 1 -- 172.21.15.46:0/869262230 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040a8450 con 0x7f82000047d0 2024-03-29T08:27:24.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.447+0000 7f821be00640 1 -- 172.21.15.46:0/869262230 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82040a85b0 con 0x7f82000047d0 2024-03-29T08:27:24.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.447+0000 7f821be00640 1 -- 172.21.15.46:0/869262230 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040a8960 con 0x7f82000047d0 2024-03-29T08:27:24.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.447+0000 7f8223c00640 1 -- 172.21.15.46:0/869262230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000047d0 msgr2=0x7f8200004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:24.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.447+0000 7f8223c00640 1 --2- 172.21.15.46:0/869262230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000047d0 0x7f8200004c50 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f821c099220 tx=0x7f820401c900 comp rx=0 tx=0).stop 2024-03-29T08:27:24.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.447+0000 7f8221c00640 1 -- 172.21.15.46:0/869262230 reap_dead start 2024-03-29T08:27:24.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.447+0000 7f8223c00640 1 -- 172.21.15.46:0/869262230 shutdown_connections 2024-03-29T08:27:24.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.447+0000 7f8223c00640 1 -- 172.21.15.46:0/869262230 >> 172.21.15.46:0/869262230 conn(0x7f821c098a40 msgr2=0x7f821c0b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:24.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.447+0000 7f8223c00640 1 -- 172.21.15.46:0/869262230 shutdown_connections 2024-03-29T08:27:24.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.447+0000 7f8223c00640 1 -- 172.21.15.46:0/869262230 wait complete. 2024-03-29T08:27:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8223c00640 1 Processor -- start 2024-03-29T08:27:24.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8223c00640 1 -- start start 2024-03-29T08:27:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8223c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000047d0 0x7f821c1519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8223c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f821c0c56a0 0x7f821c1598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8223c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f821c0ad500 0x7f821c0ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8223c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0adef0 con 0x7f82000047d0 2024-03-29T08:27:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8223c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c15ae90 con 0x7f821c0ad500 2024-03-29T08:27:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8223c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0af4d0 con 0x7f821c0c56a0 2024-03-29T08:27:24.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8221200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000047d0 0x7f821c1519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8221200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000047d0 0x7f821c1519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33740/0 (socket says 172.21.15.46:33740) 2024-03-29T08:27:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8221200640 1 -- 172.21.15.46:0/1177366264 learned_addr learned my addr 172.21.15.46:0/1177366264 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8221200640 1 -- 172.21.15.46:0/1177366264 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f821c0c56a0 msgr2=0x7f821c1598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8221200640 1 --2- 172.21.15.46:0/1177366264 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f821c0c56a0 0x7f821c1598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8221200640 1 -- 172.21.15.46:0/1177366264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f821c0ad500 msgr2=0x7f821c0ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8221200640 1 --2- 172.21.15.46:0/1177366264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f821c0ad500 0x7f821c0ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8221200640 1 -- 172.21.15.46:0/1177366264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c0cf400 con 0x7f82000047d0 2024-03-29T08:27:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8221200640 1 --2- 172.21.15.46:0/1177366264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000047d0 0x7f821c1519d0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f820c007310 tx=0x7f820c0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8219600640 1 -- 172.21.15.46:0/1177366264 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c00f8e0 con 0x7f82000047d0 2024-03-29T08:27:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8219600640 1 -- 172.21.15.46:0/1177366264 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820c00fa40 con 0x7f82000047d0 2024-03-29T08:27:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8219600640 1 -- 172.21.15.46:0/1177366264 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c00fdd0 con 0x7f82000047d0 2024-03-29T08:27:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8223c00640 1 -- 172.21.15.46:0/1177366264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821c0af7b0 con 0x7f82000047d0 2024-03-29T08:27:24.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.451+0000 7f8223c00640 1 -- 172.21.15.46:0/1177366264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821c0afd70 con 0x7f82000047d0 2024-03-29T08:27:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.455+0000 7f8219600640 1 -- 172.21.15.46:0/1177366264 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f820c01d460 con 0x7f82000047d0 2024-03-29T08:27:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.455+0000 7f8219600640 1 --2- 172.21.15.46:0/1177366264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f81f8066690 0x7f81f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.455+0000 7f8219600640 1 -- 172.21.15.46:0/1177366264 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f820c084230 con 0x7f82000047d0 2024-03-29T08:27:24.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.455+0000 7f8213e00640 1 --2- 172.21.15.46:0/1177366264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f81f8066690 0x7f81f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.455+0000 7f8227019000 1 Processor -- start 2024-03-29T08:27:24.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.455+0000 7f8227019000 1 -- start start 2024-03-29T08:27:24.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8213e00640 1 --2- 172.21.15.46:0/1177366264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f81f8066690 0x7f81f8068b50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f82040af000 tx=0x7f82040aa040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8227019000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a798db0 0x55834a799190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8227019000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a7996d0 0x55834a74ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8227019000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a74f4c0 0x55834a74f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8227019000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55834a6175a0 con 0x55834a798db0 2024-03-29T08:27:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8227019000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55834a5a4af0 con 0x55834a74f4c0 2024-03-29T08:27:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8227019000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55834a5a20c0 con 0x55834a7996d0 2024-03-29T08:27:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8221200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a798db0 0x55834a799190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8221200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a798db0 0x55834a799190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33746/0 (socket says 172.21.15.46:33746) 2024-03-29T08:27:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8221200640 1 -- 172.21.15.46:0/613275381 learned_addr learned my addr 172.21.15.46:0/613275381 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8221c00640 1 --2- 172.21.15.46:0/613275381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a74f4c0 0x55834a74f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8213e00640 1 --2- 172.21.15.46:0/613275381 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a7996d0 0x55834a74ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8221200640 1 -- 172.21.15.46:0/613275381 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a7996d0 msgr2=0x55834a74ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8221200640 1 --2- 172.21.15.46:0/613275381 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a7996d0 0x55834a74ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8221200640 1 -- 172.21.15.46:0/613275381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a74f4c0 msgr2=0x55834a74f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8221200640 1 --2- 172.21.15.46:0/613275381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a74f4c0 0x55834a74f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8221200640 1 -- 172.21.15.46:0/613275381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55834a299890 con 0x55834a798db0 2024-03-29T08:27:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8221c00640 1 --2- 172.21.15.46:0/613275381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a74f4c0 0x55834a74f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:24.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8221200640 1 --2- 172.21.15.46:0/613275381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a798db0 0x55834a799190 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f820c082790 tx=0x7f820c054d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff23c33a8d4177e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8211600640 1 -- 172.21.15.46:0/613275381 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c06a070 con 0x55834a798db0 2024-03-29T08:27:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8211600640 1 -- 172.21.15.46:0/613275381 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820c0840c0 con 0x55834a798db0 2024-03-29T08:27:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8211600640 1 -- 172.21.15.46:0/613275381 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c0552f0 con 0x55834a798db0 2024-03-29T08:27:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8227019000 1 -- 172.21.15.46:0/613275381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a798db0 msgr2=0x55834a799190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8227019000 1 --2- 172.21.15.46:0/613275381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a798db0 0x55834a799190 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f820c082790 tx=0x7f820c054d10 comp rx=0 tx=0).stop 2024-03-29T08:27:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8227019000 1 -- 172.21.15.46:0/613275381 shutdown_connections 2024-03-29T08:27:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8227019000 1 --2- 172.21.15.46:0/613275381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a74f4c0 0x55834a74f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8227019000 1 --2- 172.21.15.46:0/613275381 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a7996d0 0x55834a74ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8227019000 1 --2- 172.21.15.46:0/613275381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a798db0 0x55834a799190 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8227019000 1 -- 172.21.15.46:0/613275381 >> 172.21.15.46:0/613275381 conn(0x55834a755830 msgr2=0x55834a74d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8227019000 1 -- 172.21.15.46:0/613275381 shutdown_connections 2024-03-29T08:27:24.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8227019000 1 -- 172.21.15.46:0/613275381 wait complete. 2024-03-29T08:27:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8227019000 1 -- start start 2024-03-29T08:27:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.459+0000 7f8227019000 1 Processor -- start 2024-03-29T08:27:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8227019000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a74f4c0 0x55834a753230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8227019000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a798db0 0x55834a753770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8227019000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a7996d0 0x55834a753cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8227019000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55834a59d9b0 con 0x55834a7996d0 2024-03-29T08:27:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8227019000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55834a53cc00 con 0x55834a798db0 2024-03-29T08:27:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8227019000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55834a55b070 con 0x55834a74f4c0 2024-03-29T08:27:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8227019000 10 client.0 _ll_register_callbacks cb 0x55834a73b0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:27:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8221c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a7996d0 0x55834a753cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8221c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a7996d0 0x55834a753cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33762/0 (socket says 172.21.15.46:33762) 2024-03-29T08:27:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8221c00640 1 -- 172.21.15.46:0/641525390 learned_addr learned my addr 172.21.15.46:0/641525390 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8221200640 1 --2- 172.21.15.46:0/641525390 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a74f4c0 0x55834a753230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8213e00640 1 --2- 172.21.15.46:0/641525390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a798db0 0x55834a753770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8221c00640 1 -- 172.21.15.46:0/641525390 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a74f4c0 msgr2=0x55834a753230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8221c00640 1 --2- 172.21.15.46:0/641525390 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a74f4c0 0x55834a753230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8221c00640 1 -- 172.21.15.46:0/641525390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a798db0 msgr2=0x55834a753770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8221c00640 1 --2- 172.21.15.46:0/641525390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a798db0 0x55834a753770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8221c00640 1 -- 172.21.15.46:0/641525390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821c0afd70 con 0x55834a7996d0 2024-03-29T08:27:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8213e00640 1 --2- 172.21.15.46:0/641525390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a798db0 0x55834a753770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8221c00640 1 --2- 172.21.15.46:0/641525390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a7996d0 0x55834a753cb0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f821c0b1fc0 tx=0x7f8214004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a889508cfc07589 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f820a000640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:27:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f820a000640 1 -- 172.21.15.46:0/641525390 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214007860 con 0x55834a7996d0 2024-03-29T08:27:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8227019000 1 -- 172.21.15.46:0/641525390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55834a55a5f0 con 0x55834a7996d0 2024-03-29T08:27:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f820a000640 1 -- 172.21.15.46:0/641525390 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82140079c0 con 0x55834a7996d0 2024-03-29T08:27:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f820a000640 1 -- 172.21.15.46:0/641525390 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8214007b80 con 0x55834a7996d0 2024-03-29T08:27:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8227019000 10 client.8484 fetch_fsmap learned FSMap version 77 2024-03-29T08:27:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8227019000 1 -- 172.21.15.46:0/641525390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55834a562ab0 con 0x55834a7996d0 2024-03-29T08:27:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f820a000640 1 -- 172.21.15.46:0/641525390 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55834a55a5f0 con 0x55834a7996d0 2024-03-29T08:27:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f820a000640 1 -- 172.21.15.46:0/641525390 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 77) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f8214014070 con 0x55834a7996d0 2024-03-29T08:27:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8227019000 10 client.8484 fetch_fsmap finished waiting for FSMap version 77 2024-03-29T08:27:24.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8227019000 10 client.8484 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:27:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8227019000 20 client.8484 populate_metadata read hostname 'smithi046' 2024-03-29T08:27:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8227019000 1 --2- 172.21.15.46:0/641525390 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55834a7546c0 0x55834a7a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8227019000 4 client.8484 mds_command: new command op to 7966 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:27:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8227019000 1 -- 172.21.15.46:0/641525390 --> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55834a59a790 con 0x55834a7546c0 2024-03-29T08:27:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8221200640 1 --2- 172.21.15.46:0/641525390 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55834a7546c0 0x55834a7a9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f8221200640 1 --2- 172.21.15.46:0/641525390 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55834a7546c0 0x55834a7a9ec0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e195f3d87123bdf7 server_cookie=6033f41de4f153f3 in_seq=0 out_seq=0 2024-03-29T08:27:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.463+0000 7f820a000640 10 client.8484 ms_handle_connect on v2:172.21.15.46:6834/824184385 2024-03-29T08:27:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.467+0000 7f820a000640 1 -- 172.21.15.46:0/641525390 <== mds.0 v2:172.21.15.46:6834/824184385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55834a59a790 con 0x55834a7546c0 2024-03-29T08:27:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.467+0000 7f820a000640 10 client.8484 handle_command_reply: tid=0 2024-03-29T08:27:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.467+0000 7f8227019000 1 client.8484 shutdown 2024-03-29T08:27:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.467+0000 7f8227019000 1 -- 172.21.15.46:0/641525390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a7996d0 msgr2=0x55834a753cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.467+0000 7f8227019000 1 --2- 172.21.15.46:0/641525390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a7996d0 0x55834a753cb0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f821c0b1fc0 tx=0x7f8214004640 comp rx=0 tx=0).stop 2024-03-29T08:27:24.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.467+0000 7f8227019000 1 -- 172.21.15.46:0/641525390 shutdown_connections 2024-03-29T08:27:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.467+0000 7f8227019000 1 --2- 172.21.15.46:0/641525390 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55834a7546c0 0x55834a7a9ec0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.467+0000 7f8227019000 1 --2- 172.21.15.46:0/641525390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a798db0 0x55834a753770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.467+0000 7f8227019000 1 --2- 172.21.15.46:0/641525390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a7996d0 0x55834a753cb0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.467+0000 7f8227019000 1 --2- 172.21.15.46:0/641525390 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a74f4c0 0x55834a753230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.467+0000 7f8227019000 1 -- 172.21.15.46:0/641525390 >> 172.21.15.46:0/641525390 conn(0x55834a755830 msgr2=0x55834a74df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.467+0000 7f820a000640 0 client.8484 ms_handle_reset on v2:172.21.15.46:6834/824184385 2024-03-29T08:27:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.467+0000 7f8227019000 1 -- 172.21.15.46:0/641525390 shutdown_connections 2024-03-29T08:27:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.467+0000 7f8227019000 1 -- 172.21.15.46:0/641525390 wait complete. 2024-03-29T08:27:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.467+0000 7f8227019000 20 client.8484 trim_cache size 0 max 16384 2024-03-29T08:27:24.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 Processor -- start 2024-03-29T08:27:24.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 -- start start 2024-03-29T08:27:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a74f4c0 0x55834a7f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a798db0 0x55834a7f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a7996d0 0x55834a7f3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55834a25a940 con 0x55834a74f4c0 2024-03-29T08:27:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55834a556f80 con 0x55834a7996d0 2024-03-29T08:27:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55834a79a8c0 con 0x55834a798db0 2024-03-29T08:27:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8221200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a74f4c0 0x55834a7f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8221200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a74f4c0 0x55834a7f26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33770/0 (socket says 172.21.15.46:33770) 2024-03-29T08:27:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8221200640 1 -- 172.21.15.46:0/492241354 learned_addr learned my addr 172.21.15.46:0/492241354 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8213e00640 1 --2- 172.21.15.46:0/492241354 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a798db0 0x55834a7f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8221c00640 1 --2- 172.21.15.46:0/492241354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a7996d0 0x55834a7f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8221200640 1 -- 172.21.15.46:0/492241354 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a798db0 msgr2=0x55834a7f2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8221200640 1 --2- 172.21.15.46:0/492241354 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a798db0 0x55834a7f2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8221200640 1 -- 172.21.15.46:0/492241354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a7996d0 msgr2=0x55834a7f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8221200640 1 --2- 172.21.15.46:0/492241354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a7996d0 0x55834a7f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8221200640 1 -- 172.21.15.46:0/492241354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55834a548c00 con 0x55834a74f4c0 2024-03-29T08:27:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8213e00640 1 --2- 172.21.15.46:0/492241354 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a798db0 0x55834a7f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:24.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8221c00640 1 --2- 172.21.15.46:0/492241354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a7996d0 0x55834a7f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8221200640 1 --2- 172.21.15.46:0/492241354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a74f4c0 0x55834a7f26e0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f820c082840 tx=0x7f820c084050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa2a6ffa07a0a93 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8212000640 1 -- 172.21.15.46:0/492241354 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c06a040 con 0x55834a74f4c0 2024-03-29T08:27:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8212000640 1 -- 172.21.15.46:0/492241354 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820c06b040 con 0x55834a74f4c0 2024-03-29T08:27:24.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8212000640 1 -- 172.21.15.46:0/492241354 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c0840c0 con 0x55834a74f4c0 2024-03-29T08:27:24.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 -- 172.21.15.46:0/492241354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a74f4c0 msgr2=0x55834a7f26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:24.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 --2- 172.21.15.46:0/492241354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a74f4c0 0x55834a7f26e0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f820c082840 tx=0x7f820c084050 comp rx=0 tx=0).stop 2024-03-29T08:27:24.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 -- 172.21.15.46:0/492241354 shutdown_connections 2024-03-29T08:27:24.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 --2- 172.21.15.46:0/492241354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a7996d0 0x55834a7f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 --2- 172.21.15.46:0/492241354 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a798db0 0x55834a7f2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 --2- 172.21.15.46:0/492241354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a74f4c0 0x55834a7f26e0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 -- 172.21.15.46:0/492241354 >> 172.21.15.46:0/492241354 conn(0x55834a755830 msgr2=0x55834a7527c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 -- 172.21.15.46:0/492241354 shutdown_connections 2024-03-29T08:27:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 -- 172.21.15.46:0/492241354 wait complete. 2024-03-29T08:27:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 -- start start 2024-03-29T08:27:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.483+0000 7f8227019000 1 Processor -- start 2024-03-29T08:27:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a74f4c0 0x55834a7ddf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a798db0 0x55834a796400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a7996d0 0x55834a796940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55834a5e0ac0 con 0x55834a798db0 2024-03-29T08:27:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55834a59adb0 con 0x55834a74f4c0 2024-03-29T08:27:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55834a59c840 con 0x55834a7996d0 2024-03-29T08:27:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 10 client.0 _ll_register_callbacks cb 0x55834a562590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:27:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8213e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a798db0 0x55834a796400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8213e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a798db0 0x55834a796400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33772/0 (socket says 172.21.15.46:33772) 2024-03-29T08:27:24.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8213e00640 1 -- 172.21.15.46:0/662315962 learned_addr learned my addr 172.21.15.46:0/662315962 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:24.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8221200640 1 --2- 172.21.15.46:0/662315962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a74f4c0 0x55834a7ddf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8221c00640 1 --2- 172.21.15.46:0/662315962 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a7996d0 0x55834a796940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8221200640 1 -- 172.21.15.46:0/662315962 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a7996d0 msgr2=0x55834a796940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:24.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8221200640 1 --2- 172.21.15.46:0/662315962 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a7996d0 0x55834a796940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8221200640 1 -- 172.21.15.46:0/662315962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a798db0 msgr2=0x55834a796400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:24.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8221200640 1 --2- 172.21.15.46:0/662315962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a798db0 0x55834a796400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8221200640 1 -- 172.21.15.46:0/662315962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55834a299890 con 0x55834a74f4c0 2024-03-29T08:27:24.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8213e00640 1 --2- 172.21.15.46:0/662315962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a798db0 0x55834a796400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:27:24.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8221200640 1 --2- 172.21.15.46:0/662315962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a74f4c0 0x55834a7ddf60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f820c059140 tx=0x7f820c004f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1da7011907d67c4b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:24.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f820a000640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-03-29T08:27:24.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f820a000640 1 -- 172.21.15.46:0/662315962 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c06a040 con 0x55834a74f4c0 2024-03-29T08:27:24.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f820a000640 1 -- 172.21.15.46:0/662315962 <== mon.1 v2:172.21.15.73:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f820c05d6e0 con 0x55834a74f4c0 2024-03-29T08:27:24.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 -- 172.21.15.46:0/662315962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55834a79b3a0 con 0x55834a74f4c0 2024-03-29T08:27:24.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f820a000640 1 -- 172.21.15.46:0/662315962 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f820c051630 con 0x55834a74f4c0 2024-03-29T08:27:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 10 client.8080 fetch_fsmap learned FSMap version 77 2024-03-29T08:27:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 -- 172.21.15.46:0/662315962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55834a299cc0 con 0x55834a74f4c0 2024-03-29T08:27:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f820a000640 1 -- 172.21.15.46:0/662315962 <== mon.1 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55834a79b3a0 con 0x55834a74f4c0 2024-03-29T08:27:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f820a000640 1 -- 172.21.15.46:0/662315962 <== mon.1 v2:172.21.15.73:3300/0 5 ==== fsmap(e 77) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f820c012070 con 0x55834a74f4c0 2024-03-29T08:27:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 10 client.8080 fetch_fsmap finished waiting for FSMap version 77 2024-03-29T08:27:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 10 client.8080 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:27:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 20 client.8080 populate_metadata read hostname 'smithi046' 2024-03-29T08:27:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 --2- 172.21.15.46:0/662315962 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55834a796e80 0x55834a797260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 4 client.8080 mds_command: new command op to 7966 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T08:27:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 -- 172.21.15.46:0/662315962 --> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55834a56d460 con 0x55834a796e80 2024-03-29T08:27:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8213e00640 1 --2- 172.21.15.46:0/662315962 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55834a796e80 0x55834a797260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8213e00640 1 --2- 172.21.15.46:0/662315962 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55834a796e80 0x55834a797260 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af899b85285ec4b1 server_cookie=f32472856f2736a1 in_seq=0 out_seq=0 2024-03-29T08:27:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f820a000640 10 client.8080 ms_handle_connect on v2:172.21.15.46:6834/824184385 2024-03-29T08:27:24.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f820a000640 1 -- 172.21.15.46:0/662315962 <== mds.0 v2:172.21.15.46:6834/824184385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55834a56d460 con 0x55834a796e80 2024-03-29T08:27:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f820a000640 10 client.8080 handle_command_reply: tid=0 2024-03-29T08:27:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 client.8080 shutdown 2024-03-29T08:27:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 -- 172.21.15.46:0/662315962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a74f4c0 msgr2=0x55834a7ddf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 --2- 172.21.15.46:0/662315962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a74f4c0 0x55834a7ddf60 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f820c059140 tx=0x7f820c004f90 comp rx=0 tx=0).stop 2024-03-29T08:27:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 -- 172.21.15.46:0/662315962 shutdown_connections 2024-03-29T08:27:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 --2- 172.21.15.46:0/662315962 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55834a796e80 0x55834a797260 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 --2- 172.21.15.46:0/662315962 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55834a7996d0 0x55834a796940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 --2- 172.21.15.46:0/662315962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55834a798db0 0x55834a796400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 --2- 172.21.15.46:0/662315962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55834a74f4c0 0x55834a7ddf60 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f8227019000 1 -- 172.21.15.46:0/662315962 >> 172.21.15.46:0/662315962 conn(0x55834a755830 msgr2=0x55834a752ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:24.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.487+0000 7f820a000640 0 client.8080 ms_handle_reset on v2:172.21.15.46:6834/824184385 2024-03-29T08:27:24.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.491+0000 7f8227019000 1 -- 172.21.15.46:0/662315962 shutdown_connections 2024-03-29T08:27:24.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.491+0000 7f8227019000 1 -- 172.21.15.46:0/662315962 wait complete. 2024-03-29T08:27:24.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.491+0000 7f8227019000 20 client.8080 trim_cache size 0 max 16384 2024-03-29T08:27:24.496 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:27:24.496 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "e9b7960f-ec1f-4787-95c3-c806094d37f0", 2024-03-29T08:27:24.496 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-03-29T08:27:24.497 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 7966, 2024-03-29T08:27:24.497 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-29T08:27:24.497 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-29T08:27:24.497 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-29T08:27:24.497 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 5.9034034240000004, 2024-03-29T08:27:24.497 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 77, 2024-03-29T08:27:24.497 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 106, 2024-03-29T08:27:24.497 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 106, 2024-03-29T08:27:24.497 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 10.236419095 2024-03-29T08:27:24.497 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:27:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.491+0000 7f8223c00640 1 -- 172.21.15.46:0/1177366264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f81f8066690 msgr2=0x7f81f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.491+0000 7f8223c00640 1 --2- 172.21.15.46:0/1177366264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f81f8066690 0x7f81f8068b50 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f82040af000 tx=0x7f82040aa040 comp rx=0 tx=0).stop 2024-03-29T08:27:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.491+0000 7f8223c00640 1 -- 172.21.15.46:0/1177366264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000047d0 msgr2=0x7f821c1519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.491+0000 7f8223c00640 1 --2- 172.21.15.46:0/1177366264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000047d0 0x7f821c1519d0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f820c007310 tx=0x7f820c0077d0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.491+0000 7f8223c00640 1 -- 172.21.15.46:0/1177366264 shutdown_connections 2024-03-29T08:27:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.491+0000 7f8223c00640 1 --2- 172.21.15.46:0/1177366264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f81f8066690 0x7f81f8068b50 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.491+0000 7f8223c00640 1 --2- 172.21.15.46:0/1177366264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f821c0ad500 0x7f821c0ad9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.491+0000 7f8223c00640 1 --2- 172.21.15.46:0/1177366264 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f821c0c56a0 0x7f821c1598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.491+0000 7f8223c00640 1 --2- 172.21.15.46:0/1177366264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82000047d0 0x7f821c1519d0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.491+0000 7f8223c00640 1 -- 172.21.15.46:0/1177366264 >> 172.21.15.46:0/1177366264 conn(0x7f821c098a40 msgr2=0x7f821c06a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:24.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.491+0000 7f8223c00640 1 -- 172.21.15.46:0/1177366264 shutdown_connections 2024-03-29T08:27:24.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.491+0000 7f8223c00640 1 -- 172.21.15.46:0/1177366264 wait complete. 2024-03-29T08:27:24.511 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:27:24.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.651+0000 7f5d89000640 1 Processor -- start 2024-03-29T08:27:24.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.651+0000 7f5d89000640 1 -- start start 2024-03-29T08:27:24.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.651+0000 7f5d89000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5d840b1510 0x7f5d84165440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.651+0000 7f5d89000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d840cc2e0 con 0x7f5d8415a890 2024-03-29T08:27:24.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.651+0000 7f5d89000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d840cc580 con 0x7f5d840b2b30 2024-03-29T08:27:24.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.651+0000 7f5d89000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d840cce20 con 0x7f5d840b1510 2024-03-29T08:27:24.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.651+0000 7f5d83400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5d8415a890 0x7f5d84098a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:38866/0 (socket says 172.21.15.46:38866) 2024-03-29T08:27:24.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.651+0000 7f5d83400640 1 -- 172.21.15.46:0/2130306402 learned_addr learned my addr 172.21.15.46:0/2130306402 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:24.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.651+0000 7f5d83e00640 1 --2- 172.21.15.46:0/2130306402 >> v2:172.21.15.73:3300/0 conn(0x7f5d840b1510 0x7f5d84165440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.651+0000 7f5d82a00640 1 -- 172.21.15.46:0/2130306402 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702297632 0 0) 0x7f5d840cc2e0 con 0x7f5d8415a890 2024-03-29T08:27:24.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.651+0000 7f5d82a00640 1 -- 172.21.15.46:0/2130306402 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d68003530 con 0x7f5d8415a890 2024-03-29T08:27:24.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.651+0000 7f5d83e00640 1 -- 172.21.15.46:0/2130306402 >> v1:172.21.15.73:6789/0 conn(0x7f5d840b2b30 legacy=0x7f5d84099150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:24.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.651+0000 7f5d83e00640 1 -- 172.21.15.46:0/2130306402 >> v1:172.21.15.46:6789/0 conn(0x7f5d8415a890 legacy=0x7f5d84098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:24.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.651+0000 7f5d83e00640 1 -- 172.21.15.46:0/2130306402 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d840cc860 con 0x7f5d840b1510 2024-03-29T08:27:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.655+0000 7f5d83e00640 1 --2- 172.21.15.46:0/2130306402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d840b1510 0x7f5d84165440 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009d90 tx=0x7f5d7c01c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9fcffd21a0d2442 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.655+0000 7f5d82a00640 1 -- 172.21.15.46:0/2130306402 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0a5910 con 0x7f5d840b1510 2024-03-29T08:27:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.655+0000 7f5d82a00640 1 -- 172.21.15.46:0/2130306402 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c0a5a70 con 0x7f5d840b1510 2024-03-29T08:27:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.655+0000 7f5d82a00640 1 -- 172.21.15.46:0/2130306402 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0a5e00 con 0x7f5d840b1510 2024-03-29T08:27:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.655+0000 7f5d89000640 1 -- 172.21.15.46:0/2130306402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d840b1510 msgr2=0x7f5d84165440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.655+0000 7f5d89000640 1 --2- 172.21.15.46:0/2130306402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d840b1510 0x7f5d84165440 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c009d90 tx=0x7f5d7c01c8b0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.655+0000 7f5d89000640 1 -- 172.21.15.46:0/2130306402 shutdown_connections 2024-03-29T08:27:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.655+0000 7f5d89000640 1 --2- 172.21.15.46:0/2130306402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d840b1510 0x7f5d84165440 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.655+0000 7f5d89000640 1 -- 172.21.15.46:0/2130306402 >> 172.21.15.46:0/2130306402 conn(0x7f5d840b2010 msgr2=0x7f5d840b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.655+0000 7f5d89000640 1 -- 172.21.15.46:0/2130306402 shutdown_connections 2024-03-29T08:27:24.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.655+0000 7f5d89000640 1 -- 172.21.15.46:0/2130306402 wait complete. 2024-03-29T08:27:24.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.655+0000 7f5d89000640 1 Processor -- start 2024-03-29T08:27:24.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.655+0000 7f5d89000640 1 -- start start 2024-03-29T08:27:24.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d89000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d840b1510 0x7f5d840c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d89000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d840b2b30 0x7f5d84159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d89000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8415a890 0x7f5d84153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d89000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d84158980 con 0x7f5d8415a890 2024-03-29T08:27:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d89000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d840c6ae0 con 0x7f5d840b1510 2024-03-29T08:27:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d89000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d841583c0 con 0x7f5d840b2b30 2024-03-29T08:27:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8415a890 0x7f5d84153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d83e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8415a890 0x7f5d84153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33788/0 (socket says 172.21.15.46:33788) 2024-03-29T08:27:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d83e00640 1 -- 172.21.15.46:0/4130195493 learned_addr learned my addr 172.21.15.46:0/4130195493 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d83400640 1 --2- 172.21.15.46:0/4130195493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d840b1510 0x7f5d840c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d7be00640 1 --2- 172.21.15.46:0/4130195493 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d840b2b30 0x7f5d84159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d83e00640 1 -- 172.21.15.46:0/4130195493 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d840b2b30 msgr2=0x7f5d84159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d83e00640 1 --2- 172.21.15.46:0/4130195493 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d840b2b30 0x7f5d84159110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d83e00640 1 -- 172.21.15.46:0/4130195493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d840b1510 msgr2=0x7f5d840c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d83e00640 1 --2- 172.21.15.46:0/4130195493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d840b1510 0x7f5d840c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d83e00640 1 -- 172.21.15.46:0/4130195493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c009a70 con 0x7f5d8415a890 2024-03-29T08:27:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d7be00640 1 --2- 172.21.15.46:0/4130195493 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d840b2b30 0x7f5d84159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d83e00640 1 --2- 172.21.15.46:0/4130195493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8415a890 0x7f5d84153550 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c01a4e0 tx=0x7f5d7c019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d7b400640 1 -- 172.21.15.46:0/4130195493 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0b9860 con 0x7f5d8415a890 2024-03-29T08:27:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d7b400640 1 -- 172.21.15.46:0/4130195493 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d7c0a5900 con 0x7f5d8415a890 2024-03-29T08:27:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d7b400640 1 -- 172.21.15.46:0/4130195493 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0b9bc0 con 0x7f5d8415a890 2024-03-29T08:27:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d89000640 1 -- 172.21.15.46:0/4130195493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d841586a0 con 0x7f5d8415a890 2024-03-29T08:27:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d89000640 1 -- 172.21.15.46:0/4130195493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d840c4550 con 0x7f5d8415a890 2024-03-29T08:27:24.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.659+0000 7f5d89000640 1 -- 172.21.15.46:0/4130195493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d840b6470 con 0x7f5d8415a890 2024-03-29T08:27:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.663+0000 7f5d7b400640 1 -- 172.21.15.46:0/4130195493 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5d7c0b9d20 con 0x7f5d8415a890 2024-03-29T08:27:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.663+0000 7f5d7b400640 1 --2- 172.21.15.46:0/4130195493 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d60066690 0x7f5d60068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.663+0000 7f5d7b400640 1 -- 172.21.15.46:0/4130195493 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d7c11f340 con 0x7f5d8415a890 2024-03-29T08:27:24.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.663+0000 7f5d83400640 1 --2- 172.21.15.46:0/4130195493 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d60066690 0x7f5d60068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:24.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.663+0000 7f5d83400640 1 --2- 172.21.15.46:0/4130195493 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d60066690 0x7f5d60068b50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5d740011f0 tx=0x7f5d74007d40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:24.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.667+0000 7f5d7b400640 1 -- 172.21.15.46:0/4130195493 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5d7c0ec740 con 0x7f5d8415a890 2024-03-29T08:27:24.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.859+0000 7f5d89000640 1 -- 172.21.15.46:0/4130195493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5d840671b0 con 0x7f5d8415a890 2024-03-29T08:27:24.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.863+0000 7f5d7b400640 1 -- 172.21.15.46:0/4130195493 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v77) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5d7c0efd60 con 0x7f5d8415a890 2024-03-29T08:27:24.864 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:27:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.863+0000 7f5d89000640 1 -- 172.21.15.46:0/4130195493 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d60066690 msgr2=0x7f5d60068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.863+0000 7f5d89000640 1 --2- 172.21.15.46:0/4130195493 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d60066690 0x7f5d60068b50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5d740011f0 tx=0x7f5d74007d40 comp rx=0 tx=0).stop 2024-03-29T08:27:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.863+0000 7f5d89000640 1 -- 172.21.15.46:0/4130195493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8415a890 msgr2=0x7f5d84153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.863+0000 7f5d89000640 1 --2- 172.21.15.46:0/4130195493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8415a890 0x7f5d84153550 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c01a4e0 tx=0x7f5d7c019650 comp rx=0 tx=0).stop 2024-03-29T08:27:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.863+0000 7f5d89000640 1 -- 172.21.15.46:0/4130195493 shutdown_connections 2024-03-29T08:27:24.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.863+0000 7f5d89000640 1 --2- 172.21.15.46:0/4130195493 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d60066690 0x7f5d60068b50 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.863+0000 7f5d89000640 1 --2- 172.21.15.46:0/4130195493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d8415a890 0x7f5d84153550 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.863+0000 7f5d89000640 1 --2- 172.21.15.46:0/4130195493 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5d840b2b30 0x7f5d84159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.863+0000 7f5d89000640 1 --2- 172.21.15.46:0/4130195493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d840b1510 0x7f5d840c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.863+0000 7f5d89000640 1 -- 172.21.15.46:0/4130195493 >> 172.21.15.46:0/4130195493 conn(0x7f5d840b2010 msgr2=0x7f5d840cfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.863+0000 7f5d89000640 1 -- 172.21.15.46:0/4130195493 shutdown_connections 2024-03-29T08:27:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:24.863+0000 7f5d89000640 1 -- 172.21.15.46:0/4130195493 wait complete. 2024-03-29T08:27:24.884 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:27:24.884 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:27:24.884 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-29T08:27:24.884 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T08:27:24.884 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:27:24.885 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:27:24.885 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:27:24.885 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:27:24.885 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:27:24.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:24.885 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-29T08:27:24.895 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:27:24.895 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:27:24.895 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:27:24.895 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:27:24.895 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:27:24.895 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:27:24.895 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:27:24.896 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:27:24.896 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-29T08:27:24.896 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:27:24.896 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:27:24.896 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-29T08:27:24.896 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:27:24.896 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:27:24.896 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-29T08:27:24.896 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T08:27:24.896 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41892sec preferred_lft 41892sec 2024-03-29T08:27:24.896 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-29T08:27:24.896 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:27:24.896 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:27:24.897 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:27:24.897 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-29T08:27:24.897 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:27:24.897 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:d9:58:09:4c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:27:24.897 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:27:24.897 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:27:24.897 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:27:24.897 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T08:27:24.897 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T08:27:24.897 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:27:24.897 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T08:27:24.897 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:27:24.897 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:27:24.897 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:27:24.897 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T08:27:24.898 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:27:24.898 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:27:24.898 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T08:27:24.898 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-29T08:27:24.898 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:27:24.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:24.898 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:27:24.957 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:27:24.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:24.960 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-29T08:27:25.013 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-29T08:27:25.013 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:27:25.013 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T08:27:25.014 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:27:25.014 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T08:27:25.014 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:27:25.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:25.014 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:27:25.014 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:27:25.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:27:25.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:27:25.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:27:25.015 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:27:25.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:25.107 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:27:25.116 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T08:27:25.116 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T08:27:25.117 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:27:25.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:25.117 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-03-29T08:27:25.184 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T08:27:25.184 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:27:25.184 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T08:27:25.235 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:27:25.236 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:27:25.236 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:27:25.287 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:27:25.348 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:27:25.348 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:27:25.413 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:27:25.415 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:27:25.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:25.415 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:27:25.467 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:27:25.467 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T08:27:25.514 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:27:25.535 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:27:25.535 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:27:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:27:25.601 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:27:25.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:25.601 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:27:25.609 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-03-29T08:27:25.607+0000 7f35cc348580 -1 init, newargv = 0x55c90a7e9080 newargc=15 2024-03-29T08:27:25.609 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[48103]: starting ceph client 2024-03-29T08:27:25.616 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[48103]: starting fuse 2024-03-29T08:27:25.618 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:27:25.618 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-29T08:27:25.618 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:27:25.618 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-03-29T08:27:25.619 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-03-29T08:27:25.619 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:27:25.619 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-03-29T08:27:25.619 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:27:25.619 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-03-29T08:27:25.619 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T08:27:25.619 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-03-29T08:27:25.619 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:27:25.619 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:27:25.619 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-03-29T08:27:25.619 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-03-29T08:27:25.619 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-03-29T08:27:25.619 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-03-29T08:27:25.619 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:27:25.619 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-03-29T08:27:25.620 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:27:25.620 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:27:25.620 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:27:25.620 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-03-29T08:27:25.620 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-03-29T08:27:25.620 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-03-29T08:27:25.620 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:27:25.620 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:27:25.620 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-03-29T08:27:25.620 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:27:25.739 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.48103.asok 2024-03-29T08:27:25.924 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48103.asok 2024-03-29T08:27:25.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:25.924 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48103.asok status 2024-03-29T08:27:26.100 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:27:26.100 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-03-29T08:27:26.100 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:27:26.100 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:27:26.100 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-03-29T08:27:26.100 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-03-29T08:27:26.100 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T08:27:26.100 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "48103", 2024-03-29T08:27:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-03-29T08:27:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-03-29T08:27:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-03-29T08:27:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 8502, 2024-03-29T08:27:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-03-29T08:27:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-03-29T08:27:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-03-29T08:27:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 8502 2024-03-29T08:27:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:27:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:27:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:27:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1785090426 2024-03-29T08:27:26.101 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:27:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:27:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:27:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1785090426 2024-03-29T08:27:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.8502 192.168.0.1:0/1785090426", 2024-03-29T08:27:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/1785090426", 2024-03-29T08:27:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-03-29T08:27:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 77, 2024-03-29T08:27:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 106, 2024-03-29T08:27:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:27:26.102 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-03-29T08:27:26.103 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-03-29T08:27:26.103 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:27:26.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:26.115 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-03-29T08:27:26.115 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:27:26.124 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-03-29T08:27:26.124 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T08:27:26.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:26.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:27:26.187 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:27:26.188 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:27:26.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:26.240 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-03-29T08:27:26.378 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records in 2024-03-29T08:27:26.378 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records out 2024-03-29T08:27:26.378 INFO:teuthology.orchestra.run.smithi046.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0766629 s, 82.1 MB/s 2024-03-29T08:27:26.379 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:27:26.379 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:27:26.389 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:27:26.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.567+0000 7f78f6800640 1 Processor -- start 2024-03-29T08:27:26.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.567+0000 7f78f6800640 1 -- start start 2024-03-29T08:27:26.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.571+0000 7f78f6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f78f015a940 0x7f78f00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:26.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.571+0000 7f78f6800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f78f00cf3c0 0x7f78f00cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:26.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.571+0000 7f78f6800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78f00b00d0 con 0x7f78f015ad20 2024-03-29T08:27:26.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.571+0000 7f78f6800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f00b0ab0 con 0x7f78f015a940 2024-03-29T08:27:26.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.571+0000 7f78f6800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78f00b0d90 con 0x7f78f00cf3c0 2024-03-29T08:27:26.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.571+0000 7f78ef400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f78f015a940 0x7f78f00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:26.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.571+0000 7f78ef400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f78f015a940 0x7f78f00afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35840/0 (socket says 172.21.15.46:35840) 2024-03-29T08:27:26.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.571+0000 7f78ef400640 1 -- 172.21.15.46:0/1219087990 learned_addr learned my addr 172.21.15.46:0/1219087990 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:26.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.571+0000 7f78efe00640 1 --2- 172.21.15.46:0/1219087990 >> v2:172.21.15.73:3300/0 conn(0x7f78f00cf3c0 0x7f78f00cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:26.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.571+0000 7f78ef400640 1 -- 172.21.15.46:0/1219087990 >> v2:172.21.15.73:3300/0 conn(0x7f78f00cf3c0 msgr2=0x7f78f00cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:26.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.571+0000 7f78ef400640 1 --2- 172.21.15.46:0/1219087990 >> v2:172.21.15.73:3300/0 conn(0x7f78f00cf3c0 0x7f78f00cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:26.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.571+0000 7f78ef400640 1 -- 172.21.15.46:0/1219087990 >> v1:172.21.15.73:6789/0 conn(0x7f78f015ad20 legacy=0x7f78f00cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:26.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.571+0000 7f78ef400640 1 -- 172.21.15.46:0/1219087990 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f00b1070 con 0x7f78f015a940 2024-03-29T08:27:26.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.575+0000 7f78ef400640 1 --2- 172.21.15.46:0/1219087990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f015a940 0x7f78f00afb90 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f78f00d1320 tx=0x7f78e001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8addee4eb51b682d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:26.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.575+0000 7f78eea00640 1 -- 172.21.15.46:0/1219087990 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e00ab620 con 0x7f78f015a940 2024-03-29T08:27:26.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.575+0000 7f78eea00640 1 -- 172.21.15.46:0/1219087990 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e00ab780 con 0x7f78f015a940 2024-03-29T08:27:26.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.575+0000 7f78eea00640 1 -- 172.21.15.46:0/1219087990 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e00abb10 con 0x7f78f015a940 2024-03-29T08:27:26.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.575+0000 7f78f6800640 1 -- 172.21.15.46:0/1219087990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f015a940 msgr2=0x7f78f00afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:26.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.575+0000 7f78f6800640 1 --2- 172.21.15.46:0/1219087990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f015a940 0x7f78f00afb90 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f78f00d1320 tx=0x7f78e001c420 comp rx=0 tx=0).stop 2024-03-29T08:27:26.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.575+0000 7f78f6800640 1 -- 172.21.15.46:0/1219087990 shutdown_connections 2024-03-29T08:27:26.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.575+0000 7f78f6800640 1 --2- 172.21.15.46:0/1219087990 >> v2:172.21.15.73:3300/0 conn(0x7f78f00cf3c0 0x7f78f00cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:26.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.575+0000 7f78f6800640 1 --2- 172.21.15.46:0/1219087990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f015a940 0x7f78f00afb90 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:26.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.575+0000 7f78f6800640 1 -- 172.21.15.46:0/1219087990 >> 172.21.15.46:0/1219087990 conn(0x7f78f0089fd0 msgr2=0x7f78f008a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:26.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.575+0000 7f78f6800640 1 -- 172.21.15.46:0/1219087990 shutdown_connections 2024-03-29T08:27:26.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.575+0000 7f78f6800640 1 -- 172.21.15.46:0/1219087990 wait complete. 2024-03-29T08:27:26.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.575+0000 7f78f6800640 1 Processor -- start 2024-03-29T08:27:26.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.575+0000 7f78f6800640 1 -- start start 2024-03-29T08:27:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78f6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f00cf3c0 0x7f78f00aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78f6800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f015a940 0x7f78f01559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78f6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f015ad20 0x7f78f015d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78f6800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f01569f0 con 0x7f78f00cf3c0 2024-03-29T08:27:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78f6800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f0156cd0 con 0x7f78f015ad20 2024-03-29T08:27:26.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78f6800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f0163220 con 0x7f78f015a940 2024-03-29T08:27:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f00cf3c0 0x7f78f00aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78ef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f00cf3c0 0x7f78f00aed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35844/0 (socket says 172.21.15.46:35844) 2024-03-29T08:27:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78ef400640 1 -- 172.21.15.46:0/4202270749 learned_addr learned my addr 172.21.15.46:0/4202270749 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78e7e00640 1 --2- 172.21.15.46:0/4202270749 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f015a940 0x7f78f01559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78efe00640 1 --2- 172.21.15.46:0/4202270749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f015ad20 0x7f78f015d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78ef400640 1 -- 172.21.15.46:0/4202270749 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f015a940 msgr2=0x7f78f01559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78ef400640 1 --2- 172.21.15.46:0/4202270749 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f015a940 0x7f78f01559f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78ef400640 1 -- 172.21.15.46:0/4202270749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f015ad20 msgr2=0x7f78f015d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78ef400640 1 --2- 172.21.15.46:0/4202270749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f015ad20 0x7f78f015d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78ef400640 1 -- 172.21.15.46:0/4202270749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e00094d0 con 0x7f78f00cf3c0 2024-03-29T08:27:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78ef400640 1 --2- 172.21.15.46:0/4202270749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f00cf3c0 0x7f78f00aed50 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f78e001a050 tx=0x7f78e001ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:26.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78e7400640 1 -- 172.21.15.46:0/4202270749 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e00be3f0 con 0x7f78f00cf3c0 2024-03-29T08:27:26.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78e7400640 1 -- 172.21.15.46:0/4202270749 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e00be550 con 0x7f78f00cf3c0 2024-03-29T08:27:26.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78e7400640 1 -- 172.21.15.46:0/4202270749 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e00be900 con 0x7f78f00cf3c0 2024-03-29T08:27:26.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78f6800640 1 -- 172.21.15.46:0/4202270749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f0163500 con 0x7f78f00cf3c0 2024-03-29T08:27:26.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78f6800640 1 -- 172.21.15.46:0/4202270749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f0156fb0 con 0x7f78f00cf3c0 2024-03-29T08:27:26.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78f6800640 1 -- 172.21.15.46:0/4202270749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f0066c90 con 0x7f78f00cf3c0 2024-03-29T08:27:26.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.579+0000 7f78e7400640 1 -- 172.21.15.46:0/4202270749 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f78e00c7440 con 0x7f78f00cf3c0 2024-03-29T08:27:26.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.583+0000 7f78e7400640 1 --2- 172.21.15.46:0/4202270749 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78cc066690 0x7f78cc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:26.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.583+0000 7f78e7400640 1 -- 172.21.15.46:0/4202270749 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78e011e9f0 con 0x7f78f00cf3c0 2024-03-29T08:27:26.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.583+0000 7f78e7e00640 1 --2- 172.21.15.46:0/4202270749 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78cc066690 0x7f78cc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:26.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.583+0000 7f78e7e00640 1 --2- 172.21.15.46:0/4202270749 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78cc066690 0x7f78cc068b50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f78f01f85f0 tx=0x7f78d8009060 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:26.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.587+0000 7f78e7400640 1 -- 172.21.15.46:0/4202270749 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f78e00ebe00 con 0x7f78f00cf3c0 2024-03-29T08:27:26.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.751+0000 7f78f6800640 1 -- 172.21.15.46:0/4202270749 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78f00c3fb0 con 0x7f78f00cf3c0 2024-03-29T08:27:26.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.755+0000 7f78e7400640 1 -- 172.21.15.46:0/4202270749 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16003 (secure 0 0 0) 0x7f78e00ef420 con 0x7f78f00cf3c0 2024-03-29T08:27:26.756 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:26.756 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":106,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:27:19.585517+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:27:17.118875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-29T08:27:17.573928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:27:26.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.755+0000 7f78f6800640 1 -- 172.21.15.46:0/4202270749 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78cc066690 msgr2=0x7f78cc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:26.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.755+0000 7f78f6800640 1 --2- 172.21.15.46:0/4202270749 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78cc066690 0x7f78cc068b50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f78f01f85f0 tx=0x7f78d8009060 comp rx=0 tx=0).stop 2024-03-29T08:27:26.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.755+0000 7f78f6800640 1 -- 172.21.15.46:0/4202270749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f00cf3c0 msgr2=0x7f78f00aed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:26.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.755+0000 7f78f6800640 1 --2- 172.21.15.46:0/4202270749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f00cf3c0 0x7f78f00aed50 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f78e001a050 tx=0x7f78e001ac60 comp rx=0 tx=0).stop 2024-03-29T08:27:26.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.755+0000 7f78f6800640 1 -- 172.21.15.46:0/4202270749 shutdown_connections 2024-03-29T08:27:26.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.755+0000 7f78f6800640 1 --2- 172.21.15.46:0/4202270749 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f78cc066690 0x7f78cc068b50 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:26.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.755+0000 7f78f6800640 1 --2- 172.21.15.46:0/4202270749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78f015ad20 0x7f78f015d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:26.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.755+0000 7f78f6800640 1 --2- 172.21.15.46:0/4202270749 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f78f015a940 0x7f78f01559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:26.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.755+0000 7f78f6800640 1 --2- 172.21.15.46:0/4202270749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78f00cf3c0 0x7f78f00aed50 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:26.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.755+0000 7f78f6800640 1 -- 172.21.15.46:0/4202270749 >> 172.21.15.46:0/4202270749 conn(0x7f78f0089fd0 msgr2=0x7f78f00cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:26.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.755+0000 7f78f6800640 1 -- 172.21.15.46:0/4202270749 shutdown_connections 2024-03-29T08:27:26.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.755+0000 7f78f6800640 1 -- 172.21.15.46:0/4202270749 wait complete. 2024-03-29T08:27:26.777 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:27:17.118875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:27:17.573928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:27:26.777 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:27:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa313e00640 1 Processor -- start 2024-03-29T08:27:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa313e00640 1 -- start start 2024-03-29T08:27:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa313e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa30c06b4b0 0x7fa30c0b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa313e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa30c0b3f20 0x7fa30c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa313e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa30c157490 con 0x7fa30c15dbb0 2024-03-29T08:27:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa313e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa30c0b27b0 con 0x7fa30c0b3f20 2024-03-29T08:27:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa313e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0b2c90 con 0x7fa30c06b4b0 2024-03-29T08:27:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa311e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa30c0b3f20 0x7fa30c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa311400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa30c15dbb0 0x7fa30c06ada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:37746/0 (socket says 172.21.15.46:37746) 2024-03-29T08:27:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa311400640 1 -- 172.21.15.46:0/3828984488 learned_addr learned my addr 172.21.15.46:0/3828984488 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa310a00640 1 --2- 172.21.15.46:0/3828984488 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa30c06b4b0 0x7fa30c0b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa30be00640 1 -- 172.21.15.46:0/3828984488 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2803978400 0 0) 0x7fa30c157490 con 0x7fa30c15dbb0 2024-03-29T08:27:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa30be00640 1 -- 172.21.15.46:0/3828984488 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2f4003530 con 0x7fa30c15dbb0 2024-03-29T08:27:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa310a00640 1 -- 172.21.15.46:0/3828984488 >> v2:172.21.15.73:3300/0 conn(0x7fa30c0b3f20 msgr2=0x7fa30c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa310a00640 1 --2- 172.21.15.46:0/3828984488 >> v2:172.21.15.73:3300/0 conn(0x7fa30c0b3f20 0x7fa30c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa310a00640 1 -- 172.21.15.46:0/3828984488 >> v1:172.21.15.73:6789/0 conn(0x7fa30c15dbb0 legacy=0x7fa30c06ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa310a00640 1 -- 172.21.15.46:0/3828984488 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c0b29b0 con 0x7fa30c06b4b0 2024-03-29T08:27:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.919+0000 7fa311e00640 1 --2- 172.21.15.46:0/3828984488 >> v2:172.21.15.73:3300/0 conn(0x7fa30c0b3f20 0x7fa30c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:27:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.923+0000 7fa310a00640 1 --2- 172.21.15.46:0/3828984488 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa30c06b4b0 0x7fa30c0b1d30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa2f80097c0 tx=0x7fa2f801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a20475fb1dac3eef server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:26.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.923+0000 7fa30be00640 1 -- 172.21.15.46:0/3828984488 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f80a7070 con 0x7fa30c06b4b0 2024-03-29T08:27:26.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.923+0000 7fa30be00640 1 -- 172.21.15.46:0/3828984488 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f80ab430 con 0x7fa30c06b4b0 2024-03-29T08:27:26.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.923+0000 7fa30be00640 1 -- 172.21.15.46:0/3828984488 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f80ab7a0 con 0x7fa30c06b4b0 2024-03-29T08:27:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.923+0000 7fa313e00640 1 -- 172.21.15.46:0/3828984488 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa30c06b4b0 msgr2=0x7fa30c0b1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.923+0000 7fa313e00640 1 --2- 172.21.15.46:0/3828984488 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa30c06b4b0 0x7fa30c0b1d30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa2f80097c0 tx=0x7fa2f801c410 comp rx=0 tx=0).stop 2024-03-29T08:27:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.923+0000 7fa313e00640 1 -- 172.21.15.46:0/3828984488 shutdown_connections 2024-03-29T08:27:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.923+0000 7fa313e00640 1 --2- 172.21.15.46:0/3828984488 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa30c06b4b0 0x7fa30c0b1d30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.923+0000 7fa313e00640 1 --2- 172.21.15.46:0/3828984488 >> v2:172.21.15.73:3300/0 conn(0x7fa30c0b3f20 0x7fa30c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.923+0000 7fa313e00640 1 -- 172.21.15.46:0/3828984488 >> 172.21.15.46:0/3828984488 conn(0x7fa30c070ad0 msgr2=0x7fa30c070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.923+0000 7fa313e00640 1 -- 172.21.15.46:0/3828984488 shutdown_connections 2024-03-29T08:27:26.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.923+0000 7fa313e00640 1 -- 172.21.15.46:0/3828984488 wait complete. 2024-03-29T08:27:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa313e00640 1 Processor -- start 2024-03-29T08:27:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa313e00640 1 -- start start 2024-03-29T08:27:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa313e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa30c06b4b0 0x7fa30c1534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa313e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa30c0b3f20 0x7fa30c155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa313e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa30c15dbb0 0x7fa30c14d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa313e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c1539e0 con 0x7fa30c15dbb0 2024-03-29T08:27:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa313e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c152930 con 0x7fa30c0b3f20 2024-03-29T08:27:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa313e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c14ec80 con 0x7fa30c06b4b0 2024-03-29T08:27:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa311e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa30c15dbb0 0x7fa30c14d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa311e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa30c15dbb0 0x7fa30c14d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35850/0 (socket says 172.21.15.46:35850) 2024-03-29T08:27:26.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa311e00640 1 -- 172.21.15.46:0/1385855356 learned_addr learned my addr 172.21.15.46:0/1385855356 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:26.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa311400640 1 --2- 172.21.15.46:0/1385855356 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa30c06b4b0 0x7fa30c1534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:26.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa310a00640 1 --2- 172.21.15.46:0/1385855356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa30c0b3f20 0x7fa30c155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:26.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa311e00640 1 -- 172.21.15.46:0/1385855356 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa30c06b4b0 msgr2=0x7fa30c1534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:26.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa311e00640 1 --2- 172.21.15.46:0/1385855356 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa30c06b4b0 0x7fa30c1534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:26.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa311e00640 1 -- 172.21.15.46:0/1385855356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa30c0b3f20 msgr2=0x7fa30c155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:26.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa311e00640 1 --2- 172.21.15.46:0/1385855356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa30c0b3f20 0x7fa30c155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:26.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa311e00640 1 -- 172.21.15.46:0/1385855356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa304009a70 con 0x7fa30c15dbb0 2024-03-29T08:27:26.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa311e00640 1 --2- 172.21.15.46:0/1385855356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa30c15dbb0 0x7fa30c14d510 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fa30400efc0 tx=0x7fa304009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa309600640 1 -- 172.21.15.46:0/1385855356 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa304009d70 con 0x7fa30c15dbb0 2024-03-29T08:27:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa309600640 1 -- 172.21.15.46:0/1385855356 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30400f040 con 0x7fa30c15dbb0 2024-03-29T08:27:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa309600640 1 -- 172.21.15.46:0/1385855356 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3040039a0 con 0x7fa30c15dbb0 2024-03-29T08:27:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa313e00640 1 -- 172.21.15.46:0/1385855356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f8009420 con 0x7fa30c15dbb0 2024-03-29T08:27:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa313e00640 1 -- 172.21.15.46:0/1385855356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30c152650 con 0x7fa30c15dbb0 2024-03-29T08:27:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.927+0000 7fa313e00640 1 -- 172.21.15.46:0/1385855356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30c09d100 con 0x7fa30c15dbb0 2024-03-29T08:27:26.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.931+0000 7fa309600640 1 -- 172.21.15.46:0/1385855356 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa3040144a0 con 0x7fa30c15dbb0 2024-03-29T08:27:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.931+0000 7fa309600640 1 --2- 172.21.15.46:0/1385855356 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa2ec0666e0 0x7fa2ec068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.931+0000 7fa309600640 1 -- 172.21.15.46:0/1385855356 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa304084cb0 con 0x7fa30c15dbb0 2024-03-29T08:27:26.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.931+0000 7fa311400640 1 --2- 172.21.15.46:0/1385855356 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa2ec0666e0 0x7fa2ec068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:26.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.931+0000 7fa311400640 1 --2- 172.21.15.46:0/1385855356 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa2ec0666e0 0x7fa2ec068ba0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa3000011f0 tx=0x7fa300007d40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:26.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:26.935+0000 7fa309600640 1 -- 172.21.15.46:0/1385855356 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa304052140 con 0x7fa30c15dbb0 2024-03-29T08:27:27.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.095+0000 7fa313e00640 1 -- 172.21.15.46:0/1385855356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa30c04e120 con 0x7fa30c15dbb0 2024-03-29T08:27:27.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.095+0000 7fa309600640 1 -- 172.21.15.46:0/1385855356 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa3040180a0 con 0x7fa30c15dbb0 2024-03-29T08:27:27.098 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:27:27.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.099+0000 7fa313e00640 1 -- 172.21.15.46:0/1385855356 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa2ec0666e0 msgr2=0x7fa2ec068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.099+0000 7fa313e00640 1 --2- 172.21.15.46:0/1385855356 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa2ec0666e0 0x7fa2ec068ba0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa3000011f0 tx=0x7fa300007d40 comp rx=0 tx=0).stop 2024-03-29T08:27:27.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.099+0000 7fa313e00640 1 -- 172.21.15.46:0/1385855356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa30c15dbb0 msgr2=0x7fa30c14d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.099+0000 7fa313e00640 1 --2- 172.21.15.46:0/1385855356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa30c15dbb0 0x7fa30c14d510 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fa30400efc0 tx=0x7fa304009f90 comp rx=0 tx=0).stop 2024-03-29T08:27:27.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.099+0000 7fa313e00640 1 -- 172.21.15.46:0/1385855356 shutdown_connections 2024-03-29T08:27:27.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.099+0000 7fa313e00640 1 --2- 172.21.15.46:0/1385855356 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa2ec0666e0 0x7fa2ec068ba0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.099+0000 7fa313e00640 1 --2- 172.21.15.46:0/1385855356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa30c0b3f20 0x7fa30c155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.099+0000 7fa313e00640 1 --2- 172.21.15.46:0/1385855356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa30c15dbb0 0x7fa30c14d510 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.099+0000 7fa313e00640 1 --2- 172.21.15.46:0/1385855356 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa30c06b4b0 0x7fa30c1534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.099+0000 7fa313e00640 1 -- 172.21.15.46:0/1385855356 >> 172.21.15.46:0/1385855356 conn(0x7fa30c070ad0 msgr2=0x7fa30c154190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:27.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.099+0000 7fa313e00640 1 -- 172.21.15.46:0/1385855356 shutdown_connections 2024-03-29T08:27:27.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.099+0000 7fa313e00640 1 -- 172.21.15.46:0/1385855356 wait complete. 2024-03-29T08:27:27.119 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:27:27.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.259+0000 7f68c3000640 1 Processor -- start 2024-03-29T08:27:27.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.259+0000 7f68c3000640 1 -- start start 2024-03-29T08:27:27.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.263+0000 7f68c3000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f68bc143eb0 0x7f68bc0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:27.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.263+0000 7f68c3000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68bc0b2360 con 0x7f68bc0b1d40 2024-03-29T08:27:27.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.263+0000 7f68c3000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68bc15e500 con 0x7f68bc15efe0 2024-03-29T08:27:27.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.263+0000 7f68c3000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68bc15e7e0 con 0x7f68bc143eb0 2024-03-29T08:27:27.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.263+0000 7f68c1000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f68bc0b1d40 0x7f68bc15ad70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59594/0 (socket says 172.21.15.46:59594) 2024-03-29T08:27:27.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.263+0000 7f68c1000640 1 -- 172.21.15.46:0/2585930392 learned_addr learned my addr 172.21.15.46:0/2585930392 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:27.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.263+0000 7f68bbe00640 1 --2- 172.21.15.46:0/2585930392 >> v2:172.21.15.73:3300/0 conn(0x7f68bc143eb0 0x7f68bc0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:27.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.263+0000 7f68bb400640 1 -- 172.21.15.46:0/2585930392 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1226446660 0 0) 0x7f68bc0b2360 con 0x7f68bc0b1d40 2024-03-29T08:27:27.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.263+0000 7f68bb400640 1 -- 172.21.15.46:0/2585930392 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68a0003530 con 0x7f68bc0b1d40 2024-03-29T08:27:27.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.263+0000 7f68bb400640 1 -- 172.21.15.46:0/2585930392 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170952642 0 0) 0x7f68bc15e500 con 0x7f68bc15efe0 2024-03-29T08:27:27.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.263+0000 7f68bb400640 1 -- 172.21.15.46:0/2585930392 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68bc0b2360 con 0x7f68bc15efe0 2024-03-29T08:27:27.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.263+0000 7f68bbe00640 1 -- 172.21.15.46:0/2585930392 >> v1:172.21.15.73:6789/0 conn(0x7f68bc15efe0 legacy=0x7f68bc0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.263+0000 7f68bbe00640 1 -- 172.21.15.46:0/2585930392 >> v1:172.21.15.46:6789/0 conn(0x7f68bc0b1d40 legacy=0x7f68bc15ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.263+0000 7f68bbe00640 1 -- 172.21.15.46:0/2585930392 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68bc15eac0 con 0x7f68bc143eb0 2024-03-29T08:27:27.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.267+0000 7f68bbe00640 1 --2- 172.21.15.46:0/2585930392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68bc143eb0 0x7f68bc0cb790 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0097c0 tx=0x7f68ac01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b68a871fa7364b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:27.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.267+0000 7f68bb400640 1 -- 172.21.15.46:0/2585930392 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0ac410 con 0x7f68bc143eb0 2024-03-29T08:27:27.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.267+0000 7f68bb400640 1 -- 172.21.15.46:0/2585930392 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc15bc90 con 0x7f68bc143eb0 2024-03-29T08:27:27.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.267+0000 7f68bb400640 1 -- 172.21.15.46:0/2585930392 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ac0ac790 con 0x7f68bc143eb0 2024-03-29T08:27:27.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.267+0000 7f68c3000640 1 -- 172.21.15.46:0/2585930392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68bc143eb0 msgr2=0x7f68bc0cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.267+0000 7f68c3000640 1 --2- 172.21.15.46:0/2585930392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68bc143eb0 0x7f68bc0cb790 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0097c0 tx=0x7f68ac01c4a0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.267+0000 7f68c3000640 1 -- 172.21.15.46:0/2585930392 shutdown_connections 2024-03-29T08:27:27.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.267+0000 7f68c3000640 1 --2- 172.21.15.46:0/2585930392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68bc143eb0 0x7f68bc0cb790 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.267+0000 7f68c3000640 1 -- 172.21.15.46:0/2585930392 >> 172.21.15.46:0/2585930392 conn(0x7f68bc06a390 msgr2=0x7f68bc0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:27.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.267+0000 7f68c3000640 1 -- 172.21.15.46:0/2585930392 shutdown_connections 2024-03-29T08:27:27.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.267+0000 7f68c3000640 1 -- 172.21.15.46:0/2585930392 wait complete. 2024-03-29T08:27:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.267+0000 7f68c3000640 1 Processor -- start 2024-03-29T08:27:27.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.267+0000 7f68c3000640 1 -- start start 2024-03-29T08:27:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c3000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68bc0b1d40 0x7f68bc0cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c3000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68bc143eb0 0x7f68bc132f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c3000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68bc15efe0 0x7f68bc13a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c3000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc0cc480 con 0x7f68bc15efe0 2024-03-29T08:27:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c3000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc0af090 con 0x7f68bc143eb0 2024-03-29T08:27:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c3000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc0af370 con 0x7f68bc0b1d40 2024-03-29T08:27:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68bc15efe0 0x7f68bc13a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68bc15efe0 0x7f68bc13a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35866/0 (socket says 172.21.15.46:35866) 2024-03-29T08:27:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c1000640 1 -- 172.21.15.46:0/1619788100 learned_addr learned my addr 172.21.15.46:0/1619788100 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:27.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68bbe00640 1 --2- 172.21.15.46:0/1619788100 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68bc0b1d40 0x7f68bc0cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68b3e00640 1 --2- 172.21.15.46:0/1619788100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68bc143eb0 0x7f68bc132f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c1000640 1 -- 172.21.15.46:0/1619788100 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68bc0b1d40 msgr2=0x7f68bc0cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c1000640 1 --2- 172.21.15.46:0/1619788100 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68bc0b1d40 0x7f68bc0cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c1000640 1 -- 172.21.15.46:0/1619788100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68bc143eb0 msgr2=0x7f68bc132f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c1000640 1 --2- 172.21.15.46:0/1619788100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68bc143eb0 0x7f68bc132f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c1000640 1 -- 172.21.15.46:0/1619788100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac009420 con 0x7f68bc15efe0 2024-03-29T08:27:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c1000640 1 --2- 172.21.15.46:0/1619788100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68bc15efe0 0x7f68bc13a850 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f68b400cba0 tx=0x7f68b400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68b8c00640 1 -- 172.21.15.46:0/1619788100 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b400ace0 con 0x7f68bc15efe0 2024-03-29T08:27:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68b8c00640 1 -- 172.21.15.46:0/1619788100 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b400ae40 con 0x7f68bc15efe0 2024-03-29T08:27:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c3000640 1 -- 172.21.15.46:0/1619788100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68bc0cc6d0 con 0x7f68bc15efe0 2024-03-29T08:27:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68b8c00640 1 -- 172.21.15.46:0/1619788100 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b4012620 con 0x7f68bc15efe0 2024-03-29T08:27:27.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c3000640 1 -- 172.21.15.46:0/1619788100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68bc0aed20 con 0x7f68bc15efe0 2024-03-29T08:27:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.271+0000 7f68c3000640 1 -- 172.21.15.46:0/1619788100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68bc0ccbf0 con 0x7f68bc15efe0 2024-03-29T08:27:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.275+0000 7f68b8c00640 1 -- 172.21.15.46:0/1619788100 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f68b4007070 con 0x7f68bc15efe0 2024-03-29T08:27:27.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.275+0000 7f68b8c00640 1 --2- 172.21.15.46:0/1619788100 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6898066690 0x7f6898068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.275+0000 7f68b8c00640 1 -- 172.21.15.46:0/1619788100 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68b40841a0 con 0x7f68bc15efe0 2024-03-29T08:27:27.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.275+0000 7f68bbe00640 1 --2- 172.21.15.46:0/1619788100 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6898066690 0x7f6898068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.275+0000 7f68bbe00640 1 --2- 172.21.15.46:0/1619788100 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6898066690 0x7f6898068b50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0ae000 tx=0x7f68ac008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.279+0000 7f68b8c00640 1 -- 172.21.15.46:0/1619788100 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f68bc0ccbf0 con 0x7f68bc15efe0 2024-03-29T08:27:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.439+0000 7f68c3000640 1 -- 172.21.15.46:0/1619788100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68bc0ccbf0 con 0x7f68bc15efe0 2024-03-29T08:27:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.439+0000 7f68b8c00640 1 -- 172.21.15.46:0/1619788100 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f68bc0ccbf0 con 0x7f68bc15efe0 2024-03-29T08:27:27.441 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:27:27.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.439+0000 7f68c3000640 1 -- 172.21.15.46:0/1619788100 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6898066690 msgr2=0x7f6898068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.439+0000 7f68c3000640 1 --2- 172.21.15.46:0/1619788100 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6898066690 0x7f6898068b50 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0ae000 tx=0x7f68ac008040 comp rx=0 tx=0).stop 2024-03-29T08:27:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.439+0000 7f68c3000640 1 -- 172.21.15.46:0/1619788100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68bc15efe0 msgr2=0x7f68bc13a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.439+0000 7f68c3000640 1 --2- 172.21.15.46:0/1619788100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68bc15efe0 0x7f68bc13a850 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f68b400cba0 tx=0x7f68b400a430 comp rx=0 tx=0).stop 2024-03-29T08:27:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.439+0000 7f68c3000640 1 -- 172.21.15.46:0/1619788100 shutdown_connections 2024-03-29T08:27:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.439+0000 7f68c3000640 1 --2- 172.21.15.46:0/1619788100 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6898066690 0x7f6898068b50 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.439+0000 7f68c3000640 1 --2- 172.21.15.46:0/1619788100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68bc143eb0 0x7f68bc132f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.439+0000 7f68c3000640 1 --2- 172.21.15.46:0/1619788100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f68bc15efe0 0x7f68bc13a850 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.439+0000 7f68c3000640 1 --2- 172.21.15.46:0/1619788100 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f68bc0b1d40 0x7f68bc0cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.439+0000 7f68c3000640 1 -- 172.21.15.46:0/1619788100 >> 172.21.15.46:0/1619788100 conn(0x7f68bc06a390 msgr2=0x7f68bc0b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.439+0000 7f68c3000640 1 -- 172.21.15.46:0/1619788100 shutdown_connections 2024-03-29T08:27:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.439+0000 7f68c3000640 1 -- 172.21.15.46:0/1619788100 wait complete. 2024-03-29T08:27:27.461 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:27:27.604 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.603+0000 7f6f95400640 1 Processor -- start 2024-03-29T08:27:27.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.603+0000 7f6f95400640 1 -- start start 2024-03-29T08:27:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.603+0000 7f6f95400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f9015efe0 0x7f6f900b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.603+0000 7f6f95400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f900b2360 con 0x7f6f90143eb0 2024-03-29T08:27:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.603+0000 7f6f95400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f9015e500 con 0x7f6f900cbea0 2024-03-29T08:27:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.603+0000 7f6f95400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9015e7e0 con 0x7f6f9015efe0 2024-03-29T08:27:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.603+0000 7f6f8e800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6f90143eb0 0x7f6f900cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59610/0 (socket says 172.21.15.46:59610) 2024-03-29T08:27:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.603+0000 7f6f8e800640 1 -- 172.21.15.46:0/3104573119 learned_addr learned my addr 172.21.15.46:0/3104573119 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:27.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.603+0000 7f6f8f200640 1 --2- 172.21.15.46:0/3104573119 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f9015efe0 0x7f6f900b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.603+0000 7f6f8de00640 1 -- 172.21.15.46:0/3104573119 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455045510 0 0) 0x7f6f900b2360 con 0x7f6f90143eb0 2024-03-29T08:27:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.603+0000 7f6f8de00640 1 -- 172.21.15.46:0/3104573119 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f74003530 con 0x7f6f90143eb0 2024-03-29T08:27:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.603+0000 7f6f8de00640 1 -- 172.21.15.46:0/3104573119 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 686835220 0 0) 0x7f6f9015e500 con 0x7f6f900cbea0 2024-03-29T08:27:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.603+0000 7f6f8de00640 1 -- 172.21.15.46:0/3104573119 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f900b2360 con 0x7f6f900cbea0 2024-03-29T08:27:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.603+0000 7f6f8f200640 1 -- 172.21.15.46:0/3104573119 >> v1:172.21.15.73:6789/0 conn(0x7f6f900cbea0 legacy=0x7f6f900b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.603+0000 7f6f8f200640 1 -- 172.21.15.46:0/3104573119 >> v1:172.21.15.46:6789/0 conn(0x7f6f90143eb0 legacy=0x7f6f900cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.603+0000 7f6f8f200640 1 -- 172.21.15.46:0/3104573119 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9015eac0 con 0x7f6f9015efe0 2024-03-29T08:27:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.607+0000 7f6f8f200640 1 --2- 172.21.15.46:0/3104573119 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f9015efe0 0x7f6f900b1d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009dc0 tx=0x7f6f8801c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=527af4d042d2d0f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.607+0000 7f6f8de00640 1 -- 172.21.15.46:0/3104573119 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f880a5a70 con 0x7f6f9015efe0 2024-03-29T08:27:27.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.607+0000 7f6f8de00640 1 -- 172.21.15.46:0/3104573119 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f880a5bd0 con 0x7f6f9015efe0 2024-03-29T08:27:27.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.607+0000 7f6f8de00640 1 -- 172.21.15.46:0/3104573119 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f880ad440 con 0x7f6f9015efe0 2024-03-29T08:27:27.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.607+0000 7f6f95400640 1 -- 172.21.15.46:0/3104573119 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f9015efe0 msgr2=0x7f6f900b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.607+0000 7f6f95400640 1 --2- 172.21.15.46:0/3104573119 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f9015efe0 0x7f6f900b1d40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009dc0 tx=0x7f6f8801c990 comp rx=0 tx=0).stop 2024-03-29T08:27:27.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.607+0000 7f6f95400640 1 -- 172.21.15.46:0/3104573119 shutdown_connections 2024-03-29T08:27:27.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.607+0000 7f6f95400640 1 --2- 172.21.15.46:0/3104573119 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f9015efe0 0x7f6f900b1d40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.607+0000 7f6f95400640 1 -- 172.21.15.46:0/3104573119 >> 172.21.15.46:0/3104573119 conn(0x7f6f9006a390 msgr2=0x7f6f900cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:27.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.607+0000 7f6f95400640 1 -- 172.21.15.46:0/3104573119 shutdown_connections 2024-03-29T08:27:27.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.607+0000 7f6f95400640 1 -- 172.21.15.46:0/3104573119 wait complete. 2024-03-29T08:27:27.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f95400640 1 Processor -- start 2024-03-29T08:27:27.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f95400640 1 -- start start 2024-03-29T08:27:27.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f95400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f900cbea0 0x7f6f9013b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:27.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f95400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f90143eb0 0x7f6f90155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:27.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f95400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f9015efe0 0x7f6f90138670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:27.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f95400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f90155c40 con 0x7f6f90143eb0 2024-03-29T08:27:27.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f95400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f90155f20 con 0x7f6f9015efe0 2024-03-29T08:27:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f95400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f90156200 con 0x7f6f900cbea0 2024-03-29T08:27:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f90143eb0 0x7f6f90155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f90143eb0 0x7f6f90155010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35880/0 (socket says 172.21.15.46:35880) 2024-03-29T08:27:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f87e00640 1 -- 172.21.15.46:0/3898094402 learned_addr learned my addr 172.21.15.46:0/3898094402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f8e800640 1 --2- 172.21.15.46:0/3898094402 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f900cbea0 0x7f6f9013b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f8f200640 1 --2- 172.21.15.46:0/3898094402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f9015efe0 0x7f6f90138670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f87e00640 1 -- 172.21.15.46:0/3898094402 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f900cbea0 msgr2=0x7f6f9013b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f87e00640 1 --2- 172.21.15.46:0/3898094402 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f900cbea0 0x7f6f9013b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f87e00640 1 -- 172.21.15.46:0/3898094402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f9015efe0 msgr2=0x7f6f90138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f87e00640 1 --2- 172.21.15.46:0/3898094402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f9015efe0 0x7f6f90138670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f87e00640 1 -- 172.21.15.46:0/3898094402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f88009a70 con 0x7f6f90143eb0 2024-03-29T08:27:27.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f87e00640 1 --2- 172.21.15.46:0/3898094402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f90143eb0 0x7f6f90155010 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f6f7800c490 tx=0x7f6f7800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f86a00640 1 -- 172.21.15.46:0/3898094402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f780052d0 con 0x7f6f90143eb0 2024-03-29T08:27:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f86a00640 1 -- 172.21.15.46:0/3898094402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f78005430 con 0x7f6f90143eb0 2024-03-29T08:27:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f95400640 1 -- 172.21.15.46:0/3898094402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f901564e0 con 0x7f6f90143eb0 2024-03-29T08:27:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f86a00640 1 -- 172.21.15.46:0/3898094402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f78012870 con 0x7f6f90143eb0 2024-03-29T08:27:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f95400640 1 -- 172.21.15.46:0/3898094402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f90155550 con 0x7f6f90143eb0 2024-03-29T08:27:27.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.611+0000 7f6f95400640 1 -- 172.21.15.46:0/3898094402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f90071a70 con 0x7f6f90143eb0 2024-03-29T08:27:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.615+0000 7f6f86a00640 1 -- 172.21.15.46:0/3898094402 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6f780129d0 con 0x7f6f90143eb0 2024-03-29T08:27:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.615+0000 7f6f86a00640 1 --2- 172.21.15.46:0/3898094402 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f6c066690 0x7f6f6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:27.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.615+0000 7f6f86a00640 1 -- 172.21.15.46:0/3898094402 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f78083c90 con 0x7f6f90143eb0 2024-03-29T08:27:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.615+0000 7f6f8e800640 1 --2- 172.21.15.46:0/3898094402 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f6c066690 0x7f6f6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:27.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.615+0000 7f6f8e800640 1 --2- 172.21.15.46:0/3898094402 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f6c066690 0x7f6f6c068b50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6f80007fd0 tx=0x7f6f80007d60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:27.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.619+0000 7f6f86a00640 1 -- 172.21.15.46:0/3898094402 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6f78051010 con 0x7f6f90143eb0 2024-03-29T08:27:27.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.779+0000 7f6f95400640 1 -- 172.21.15.46:0/3898094402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f90156990 con 0x7f6f90143eb0 2024-03-29T08:27:27.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.779+0000 7f6f86a00640 1 -- 172.21.15.46:0/3898094402 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6f78054630 con 0x7f6f90143eb0 2024-03-29T08:27:27.780 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:27:27.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.779+0000 7f6f95400640 1 -- 172.21.15.46:0/3898094402 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f6c066690 msgr2=0x7f6f6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.779+0000 7f6f95400640 1 --2- 172.21.15.46:0/3898094402 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f6c066690 0x7f6f6c068b50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6f80007fd0 tx=0x7f6f80007d60 comp rx=0 tx=0).stop 2024-03-29T08:27:27.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.779+0000 7f6f95400640 1 -- 172.21.15.46:0/3898094402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f90143eb0 msgr2=0x7f6f90155010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.779+0000 7f6f95400640 1 --2- 172.21.15.46:0/3898094402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f90143eb0 0x7f6f90155010 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f6f7800c490 tx=0x7f6f7800c950 comp rx=0 tx=0).stop 2024-03-29T08:27:27.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.779+0000 7f6f95400640 1 -- 172.21.15.46:0/3898094402 shutdown_connections 2024-03-29T08:27:27.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.779+0000 7f6f95400640 1 --2- 172.21.15.46:0/3898094402 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6f6c066690 0x7f6f6c068b50 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.779+0000 7f6f95400640 1 --2- 172.21.15.46:0/3898094402 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f9015efe0 0x7f6f90138670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.779+0000 7f6f95400640 1 --2- 172.21.15.46:0/3898094402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f90143eb0 0x7f6f90155010 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.779+0000 7f6f95400640 1 --2- 172.21.15.46:0/3898094402 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6f900cbea0 0x7f6f9013b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.779+0000 7f6f95400640 1 -- 172.21.15.46:0/3898094402 >> 172.21.15.46:0/3898094402 conn(0x7f6f9006a390 msgr2=0x7f6f900cb790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:27.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.779+0000 7f6f95400640 1 -- 172.21.15.46:0/3898094402 shutdown_connections 2024-03-29T08:27:27.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.779+0000 7f6f95400640 1 -- 172.21.15.46:0/3898094402 wait complete. 2024-03-29T08:27:27.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:27:27.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.939+0000 7fce18c00640 1 Processor -- start 2024-03-29T08:27:27.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.939+0000 7fce18c00640 1 -- start start 2024-03-29T08:27:27.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.943+0000 7fce18c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce1415de20 0x7fce140b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:27.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.943+0000 7fce18c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fce140b3ac0 0x7fce140cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:27.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.943+0000 7fce18c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce14099950 con 0x7fce14099450 2024-03-29T08:27:27.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.943+0000 7fce18c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce14099c30 con 0x7fce140b3ac0 2024-03-29T08:27:27.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.943+0000 7fce18c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce140cb590 con 0x7fce1415de20 2024-03-29T08:27:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.943+0000 7fce09400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fce140b3ac0 0x7fce140cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.943+0000 7fce09400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fce140b3ac0 0x7fce140cbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35884/0 (socket says 172.21.15.46:35884) 2024-03-29T08:27:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.943+0000 7fce09400640 1 -- 172.21.15.46:0/3573745734 learned_addr learned my addr 172.21.15.46:0/3573745734 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.943+0000 7fce12800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fce14099450 0x7fce140ce300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59616/0 (socket says 172.21.15.46:59616) 2024-03-29T08:27:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.943+0000 7fce11e00640 1 --2- 172.21.15.46:0/3573745734 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce1415de20 0x7fce140b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.943+0000 7fce09400640 1 -- 172.21.15.46:0/3573745734 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce1415de20 msgr2=0x7fce140b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.943+0000 7fce09400640 1 --2- 172.21.15.46:0/3573745734 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce1415de20 0x7fce140b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.943+0000 7fce09400640 1 -- 172.21.15.46:0/3573745734 >> v1:172.21.15.46:6789/0 conn(0x7fce14099450 legacy=0x7fce140ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.943+0000 7fce09400640 1 -- 172.21.15.46:0/3573745734 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce140cb870 con 0x7fce140b3ac0 2024-03-29T08:27:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce09400640 1 --2- 172.21.15.46:0/3573745734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce140b3ac0 0x7fce140cbd90 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009870 tx=0x7fcdfc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f1eebddb1b591 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce11400640 1 -- 172.21.15.46:0/3573745734 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0ac410 con 0x7fce140b3ac0 2024-03-29T08:27:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce11400640 1 -- 172.21.15.46:0/3573745734 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc0ac570 con 0x7fce140b3ac0 2024-03-29T08:27:27.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce11400640 1 -- 172.21.15.46:0/3573745734 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0ac900 con 0x7fce140b3ac0 2024-03-29T08:27:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce18c00640 1 -- 172.21.15.46:0/3573745734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce140b3ac0 msgr2=0x7fce140cbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce18c00640 1 --2- 172.21.15.46:0/3573745734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce140b3ac0 0x7fce140cbd90 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc009870 tx=0x7fcdfc01c560 comp rx=0 tx=0).stop 2024-03-29T08:27:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce18c00640 1 -- 172.21.15.46:0/3573745734 shutdown_connections 2024-03-29T08:27:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce18c00640 1 --2- 172.21.15.46:0/3573745734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce140b3ac0 0x7fce140cbd90 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce18c00640 1 --2- 172.21.15.46:0/3573745734 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce1415de20 0x7fce140b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce18c00640 1 -- 172.21.15.46:0/3573745734 >> 172.21.15.46:0/3573745734 conn(0x7fce140b2220 msgr2=0x7fce140b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce18c00640 1 -- 172.21.15.46:0/3573745734 shutdown_connections 2024-03-29T08:27:27.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce18c00640 1 -- 172.21.15.46:0/3573745734 wait complete. 2024-03-29T08:27:27.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce18c00640 1 Processor -- start 2024-03-29T08:27:27.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce18c00640 1 -- start start 2024-03-29T08:27:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce18c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce14099450 0x7fce140c3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce18c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce1415de20 0x7fce1414fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce18c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce14151900 0x7fce140c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce18c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce14153da0 con 0x7fce14099450 2024-03-29T08:27:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce18c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce14151670 con 0x7fce1415de20 2024-03-29T08:27:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce18c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce14153ff0 con 0x7fce14151900 2024-03-29T08:27:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.947+0000 7fce11e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce14099450 0x7fce140c3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce11e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce14099450 0x7fce140c3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35896/0 (socket says 172.21.15.46:35896) 2024-03-29T08:27:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce11e00640 1 -- 172.21.15.46:0/3682787676 learned_addr learned my addr 172.21.15.46:0/3682787676 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce09400640 1 --2- 172.21.15.46:0/3682787676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce1415de20 0x7fce1414fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce12800640 1 --2- 172.21.15.46:0/3682787676 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce14151900 0x7fce140c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce11e00640 1 -- 172.21.15.46:0/3682787676 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce14151900 msgr2=0x7fce140c6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce11e00640 1 --2- 172.21.15.46:0/3682787676 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce14151900 0x7fce140c6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce11e00640 1 -- 172.21.15.46:0/3682787676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce1415de20 msgr2=0x7fce1414fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce11e00640 1 --2- 172.21.15.46:0/3682787676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce1415de20 0x7fce1414fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce11e00640 1 -- 172.21.15.46:0/3682787676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc0094d0 con 0x7fce14099450 2024-03-29T08:27:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce11e00640 1 --2- 172.21.15.46:0/3682787676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce14099450 0x7fce140c3bb0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fce04007e40 tx=0x7fce0400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce0aa00640 1 -- 172.21.15.46:0/3682787676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce04014620 con 0x7fce14099450 2024-03-29T08:27:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce0aa00640 1 -- 172.21.15.46:0/3682787676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce0400ee90 con 0x7fce14099450 2024-03-29T08:27:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce0aa00640 1 -- 172.21.15.46:0/3682787676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce040149c0 con 0x7fce14099450 2024-03-29T08:27:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce18c00640 1 -- 172.21.15.46:0/3682787676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce141542d0 con 0x7fce14099450 2024-03-29T08:27:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce18c00640 1 -- 172.21.15.46:0/3682787676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce14151150 con 0x7fce14099450 2024-03-29T08:27:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce18c00640 1 -- 172.21.15.46:0/3682787676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce14150ca0 con 0x7fce14099450 2024-03-29T08:27:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce0aa00640 1 -- 172.21.15.46:0/3682787676 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fce04014b20 con 0x7fce14099450 2024-03-29T08:27:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce0aa00640 1 --2- 172.21.15.46:0/3682787676 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcdf0066690 0x7fcdf0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.951+0000 7fce0aa00640 1 -- 172.21.15.46:0/3682787676 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce04084ab0 con 0x7fce14099450 2024-03-29T08:27:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.955+0000 7fce09400640 1 --2- 172.21.15.46:0/3682787676 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcdf0066690 0x7fcdf0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:27.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.955+0000 7fce09400640 1 --2- 172.21.15.46:0/3682787676 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcdf0066690 0x7fcdf0068b50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0ae000 tx=0x7fcdfc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:27.959+0000 7fce0aa00640 1 -- 172.21.15.46:0/3682787676 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fce04051e10 con 0x7fce14099450 2024-03-29T08:27:28.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.135+0000 7fce18c00640 1 -- 172.21.15.46:0/3682787676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fce141544c0 con 0x7fce14099450 2024-03-29T08:27:28.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.135+0000 7fce0aa00640 1 -- 172.21.15.46:0/3682787676 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) v1 ==== 80+0+0 (secure 0 0 0) 0x7fce04055430 con 0x7fce14099450 2024-03-29T08:27:28.138 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:27:28.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.139+0000 7fce18c00640 1 -- 172.21.15.46:0/3682787676 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcdf0066690 msgr2=0x7fcdf0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:28.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.139+0000 7fce18c00640 1 --2- 172.21.15.46:0/3682787676 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcdf0066690 0x7fcdf0068b50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0ae000 tx=0x7fcdfc009040 comp rx=0 tx=0).stop 2024-03-29T08:27:28.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.139+0000 7fce18c00640 1 -- 172.21.15.46:0/3682787676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce14099450 msgr2=0x7fce140c3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:28.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.139+0000 7fce18c00640 1 --2- 172.21.15.46:0/3682787676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce14099450 0x7fce140c3bb0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fce04007e40 tx=0x7fce0400e4b0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.139+0000 7fce18c00640 1 -- 172.21.15.46:0/3682787676 shutdown_connections 2024-03-29T08:27:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.139+0000 7fce18c00640 1 --2- 172.21.15.46:0/3682787676 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcdf0066690 0x7fcdf0068b50 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.139+0000 7fce18c00640 1 --2- 172.21.15.46:0/3682787676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce1415de20 0x7fce1414fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.139+0000 7fce18c00640 1 --2- 172.21.15.46:0/3682787676 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce14151900 0x7fce140c6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.139+0000 7fce18c00640 1 --2- 172.21.15.46:0/3682787676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce14099450 0x7fce140c3bb0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.139+0000 7fce18c00640 1 -- 172.21.15.46:0/3682787676 >> 172.21.15.46:0/3682787676 conn(0x7fce140b2220 msgr2=0x7fce1406ea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.139+0000 7fce18c00640 1 -- 172.21.15.46:0/3682787676 shutdown_connections 2024-03-29T08:27:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.139+0000 7fce18c00640 1 -- 172.21.15.46:0/3682787676 wait complete. 2024-03-29T08:27:28.159 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:27:28.159 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:27:28.168 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:27:28.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.347+0000 7f97e9c00640 1 Processor -- start 2024-03-29T08:27:28.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.347+0000 7f97e9c00640 1 -- start start 2024-03-29T08:27:28.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.351+0000 7f97e9c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f97e415feb0 0x7f97e4071370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:28.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.351+0000 7f97e9c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f97e41605f0 0x7f97e4069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:28.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.351+0000 7f97e9c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e4161020 con 0x7f97e4160a80 2024-03-29T08:27:28.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.351+0000 7f97e9c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e4161300 con 0x7f97e41605f0 2024-03-29T08:27:28.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.351+0000 7f97e9c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e4070b70 con 0x7f97e415feb0 2024-03-29T08:27:28.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.351+0000 7f97da000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f97e41605f0 0x7f97e4069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:28.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.351+0000 7f97da000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f97e41605f0 0x7f97e4069d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35906/0 (socket says 172.21.15.46:35906) 2024-03-29T08:27:28.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.351+0000 7f97da000640 1 -- 172.21.15.46:0/1585218726 learned_addr learned my addr 172.21.15.46:0/1585218726 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:28.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.351+0000 7f97e2a00640 1 --2- 172.21.15.46:0/1585218726 >> v2:172.21.15.73:3300/0 conn(0x7f97e415feb0 0x7f97e4071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:28.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.351+0000 7f97da000640 1 -- 172.21.15.46:0/1585218726 >> v2:172.21.15.73:3300/0 conn(0x7f97e415feb0 msgr2=0x7f97e4071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:28.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.351+0000 7f97da000640 1 --2- 172.21.15.46:0/1585218726 >> v2:172.21.15.73:3300/0 conn(0x7f97e415feb0 0x7f97e4071370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.351+0000 7f97da000640 1 -- 172.21.15.46:0/1585218726 >> v1:172.21.15.73:6789/0 conn(0x7f97e4160a80 legacy=0x7f97e406d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:28.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.351+0000 7f97da000640 1 -- 172.21.15.46:0/1585218726 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e4070e50 con 0x7f97e41605f0 2024-03-29T08:27:28.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.355+0000 7f97da000640 1 --2- 172.21.15.46:0/1585218726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e41605f0 0x7f97e4069d90 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009870 tx=0x7f97cc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53c18dd6c18961a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:28.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.355+0000 7f97e2000640 1 -- 172.21.15.46:0/1585218726 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0ac600 con 0x7f97e41605f0 2024-03-29T08:27:28.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.355+0000 7f97e2000640 1 -- 172.21.15.46:0/1585218726 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97cc0a8050 con 0x7f97e41605f0 2024-03-29T08:27:28.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.355+0000 7f97e2000640 1 -- 172.21.15.46:0/1585218726 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97cc0ac980 con 0x7f97e41605f0 2024-03-29T08:27:28.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.355+0000 7f97e9c00640 1 -- 172.21.15.46:0/1585218726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e41605f0 msgr2=0x7f97e4069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:28.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.355+0000 7f97e9c00640 1 --2- 172.21.15.46:0/1585218726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e41605f0 0x7f97e4069d90 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009870 tx=0x7f97cc01c490 comp rx=0 tx=0).stop 2024-03-29T08:27:28.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.355+0000 7f97e9c00640 1 -- 172.21.15.46:0/1585218726 shutdown_connections 2024-03-29T08:27:28.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.355+0000 7f97e9c00640 1 --2- 172.21.15.46:0/1585218726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e41605f0 0x7f97e4069d90 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.355+0000 7f97e9c00640 1 --2- 172.21.15.46:0/1585218726 >> v2:172.21.15.73:3300/0 conn(0x7f97e415feb0 0x7f97e4071370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.355+0000 7f97e9c00640 1 -- 172.21.15.46:0/1585218726 >> 172.21.15.46:0/1585218726 conn(0x7f97e40ced50 msgr2=0x7f97e40cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:28.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.355+0000 7f97e9c00640 1 -- 172.21.15.46:0/1585218726 shutdown_connections 2024-03-29T08:27:28.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.355+0000 7f97e9c00640 1 -- 172.21.15.46:0/1585218726 wait complete. 2024-03-29T08:27:28.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.355+0000 7f97e9c00640 1 Processor -- start 2024-03-29T08:27:28.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.355+0000 7f97e9c00640 1 -- start start 2024-03-29T08:27:28.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e9c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e415feb0 0x7f97e413ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:28.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e9c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f97e41605f0 0x7f97e415dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:28.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e9c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e4160a80 0x7f97e4159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:28.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e9c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e413ef40 con 0x7f97e415feb0 2024-03-29T08:27:28.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e9c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e415ef40 con 0x7f97e4160a80 2024-03-29T08:27:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e9c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e415e400 con 0x7f97e41605f0 2024-03-29T08:27:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e415feb0 0x7f97e413ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e415feb0 0x7f97e413ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35918/0 (socket says 172.21.15.46:35918) 2024-03-29T08:27:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e2a00640 1 -- 172.21.15.46:0/3597460675 learned_addr learned my addr 172.21.15.46:0/3597460675 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e3400640 1 --2- 172.21.15.46:0/3597460675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e4160a80 0x7f97e4159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97da000640 1 --2- 172.21.15.46:0/3597460675 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f97e41605f0 0x7f97e415dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e2a00640 1 -- 172.21.15.46:0/3597460675 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f97e41605f0 msgr2=0x7f97e415dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e2a00640 1 --2- 172.21.15.46:0/3597460675 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f97e41605f0 0x7f97e415dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e2a00640 1 -- 172.21.15.46:0/3597460675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e4160a80 msgr2=0x7f97e4159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e2a00640 1 --2- 172.21.15.46:0/3597460675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e4160a80 0x7f97e4159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e2a00640 1 -- 172.21.15.46:0/3597460675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97cc0094d0 con 0x7f97e415feb0 2024-03-29T08:27:28.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e2a00640 1 --2- 172.21.15.46:0/3597460675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e415feb0 0x7f97e413ea00 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f97d400e930 tx=0x7f97d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97db400640 1 -- 172.21.15.46:0/3597460675 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d4009690 con 0x7f97e415feb0 2024-03-29T08:27:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97db400640 1 -- 172.21.15.46:0/3597460675 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d400ca00 con 0x7f97e415feb0 2024-03-29T08:27:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97db400640 1 -- 172.21.15.46:0/3597460675 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d400cd70 con 0x7f97e415feb0 2024-03-29T08:27:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e9c00640 1 -- 172.21.15.46:0/3597460675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e415e6a0 con 0x7f97e415feb0 2024-03-29T08:27:28.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e9c00640 1 -- 172.21.15.46:0/3597460675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e415ec60 con 0x7f97e415feb0 2024-03-29T08:27:28.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97e9c00640 1 -- 172.21.15.46:0/3597460675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e4066c90 con 0x7f97e415feb0 2024-03-29T08:27:28.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.359+0000 7f97db400640 1 -- 172.21.15.46:0/3597460675 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f97d4013470 con 0x7f97e415feb0 2024-03-29T08:27:28.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.363+0000 7f97db400640 1 --2- 172.21.15.46:0/3597460675 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97c00666c0 0x7f97c0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:28.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.363+0000 7f97db400640 1 -- 172.21.15.46:0/3597460675 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97d4084350 con 0x7f97e415feb0 2024-03-29T08:27:28.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.363+0000 7f97da000640 1 --2- 172.21.15.46:0/3597460675 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97c00666c0 0x7f97c0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:28.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.363+0000 7f97da000640 1 --2- 172.21.15.46:0/3597460675 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97c00666c0 0x7f97c0068b80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0ae000 tx=0x7f97cc0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:28.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.367+0000 7f97db400640 1 -- 172.21.15.46:0/3597460675 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f97d4051830 con 0x7f97e415feb0 2024-03-29T08:27:28.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.535+0000 7f97e9c00640 1 -- 172.21.15.46:0/3597460675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97e40abee0 con 0x7f97e415feb0 2024-03-29T08:27:28.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.535+0000 7f97db400640 1 -- 172.21.15.46:0/3597460675 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16003 (secure 0 0 0) 0x7f97d4054e50 con 0x7f97e415feb0 2024-03-29T08:27:28.537 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:28.537 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":106,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:27:19.585517+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:27:17.118875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-29T08:27:17.573928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:27:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.535+0000 7f97e9c00640 1 -- 172.21.15.46:0/3597460675 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97c00666c0 msgr2=0x7f97c0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.535+0000 7f97e9c00640 1 --2- 172.21.15.46:0/3597460675 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97c00666c0 0x7f97c0068b80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0ae000 tx=0x7f97cc0a7000 comp rx=0 tx=0).stop 2024-03-29T08:27:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.535+0000 7f97e9c00640 1 -- 172.21.15.46:0/3597460675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e415feb0 msgr2=0x7f97e413ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.535+0000 7f97e9c00640 1 --2- 172.21.15.46:0/3597460675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e415feb0 0x7f97e413ea00 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f97d400e930 tx=0x7f97d400edf0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.535+0000 7f97e9c00640 1 -- 172.21.15.46:0/3597460675 shutdown_connections 2024-03-29T08:27:28.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.535+0000 7f97e9c00640 1 --2- 172.21.15.46:0/3597460675 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f97c00666c0 0x7f97c0068b80 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.535+0000 7f97e9c00640 1 --2- 172.21.15.46:0/3597460675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97e4160a80 0x7f97e4159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.535+0000 7f97e9c00640 1 --2- 172.21.15.46:0/3597460675 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f97e41605f0 0x7f97e415dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.535+0000 7f97e9c00640 1 --2- 172.21.15.46:0/3597460675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97e415feb0 0x7f97e413ea00 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.535+0000 7f97e9c00640 1 -- 172.21.15.46:0/3597460675 >> 172.21.15.46:0/3597460675 conn(0x7f97e40ced50 msgr2=0x7f97e4158370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.535+0000 7f97e9c00640 1 -- 172.21.15.46:0/3597460675 shutdown_connections 2024-03-29T08:27:28.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.535+0000 7f97e9c00640 1 -- 172.21.15.46:0/3597460675 wait complete. 2024-03-29T08:27:28.559 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:27:17.118875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:27:17.573928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:27:28.560 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:27:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.699+0000 7efe44a00640 1 Processor -- start 2024-03-29T08:27:28.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.699+0000 7efe44a00640 1 -- start start 2024-03-29T08:27:28.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.703+0000 7efe44a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efe4015c950 0x7efe4006bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:28.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.703+0000 7efe44a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe400cea30 0x7efe401606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.703+0000 7efe44a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe4015cd30 0x7efe40160be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.703+0000 7efe44a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe400cfb20 con 0x7efe4015c950 2024-03-29T08:27:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.703+0000 7efe44a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe400cef10 con 0x7efe4015cd30 2024-03-29T08:27:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.703+0000 7efe44a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe400cf1f0 con 0x7efe400cea30 2024-03-29T08:27:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.703+0000 7efe3de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efe4015c950 0x7efe4006bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.703+0000 7efe3de00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efe4015c950 0x7efe4006bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35932/0 (socket says 172.21.15.46:35932) 2024-03-29T08:27:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.703+0000 7efe3de00640 1 -- 172.21.15.46:0/180961008 learned_addr learned my addr 172.21.15.46:0/180961008 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.703+0000 7efe3e800640 1 --2- 172.21.15.46:0/180961008 >> v2:172.21.15.73:3300/0 conn(0x7efe4015cd30 0x7efe40160be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.703+0000 7efe37e00640 1 --2- 172.21.15.46:0/180961008 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe400cea30 0x7efe401606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.703+0000 7efe3de00640 1 -- 172.21.15.46:0/180961008 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe400cea30 msgr2=0x7efe401606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.703+0000 7efe3de00640 1 --2- 172.21.15.46:0/180961008 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe400cea30 0x7efe401606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.703+0000 7efe3de00640 1 -- 172.21.15.46:0/180961008 >> v2:172.21.15.73:3300/0 conn(0x7efe4015cd30 msgr2=0x7efe40160be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.703+0000 7efe3de00640 1 --2- 172.21.15.46:0/180961008 >> v2:172.21.15.73:3300/0 conn(0x7efe4015cd30 0x7efe40160be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.703+0000 7efe3de00640 1 -- 172.21.15.46:0/180961008 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe400cf4d0 con 0x7efe4015c950 2024-03-29T08:27:28.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.703+0000 7efe3de00640 1 --2- 172.21.15.46:0/180961008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe4015c950 0x7efe4006bbe0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7efe40071700 tx=0x7efe3001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1802b3c656595c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:28.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.707+0000 7efe3d400640 1 -- 172.21.15.46:0/180961008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe300a89d0 con 0x7efe4015c950 2024-03-29T08:27:28.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.707+0000 7efe3d400640 1 -- 172.21.15.46:0/180961008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe300a8b30 con 0x7efe4015c950 2024-03-29T08:27:28.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.707+0000 7efe3d400640 1 -- 172.21.15.46:0/180961008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe300b0440 con 0x7efe4015c950 2024-03-29T08:27:28.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.707+0000 7efe44a00640 1 -- 172.21.15.46:0/180961008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe4015c950 msgr2=0x7efe4006bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.707+0000 7efe44a00640 1 --2- 172.21.15.46:0/180961008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe4015c950 0x7efe4006bbe0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7efe40071700 tx=0x7efe3001ed10 comp rx=0 tx=0).stop 2024-03-29T08:27:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.707+0000 7efe44a00640 1 -- 172.21.15.46:0/180961008 shutdown_connections 2024-03-29T08:27:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.707+0000 7efe44a00640 1 --2- 172.21.15.46:0/180961008 >> v2:172.21.15.73:3300/0 conn(0x7efe4015cd30 0x7efe40160be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.707+0000 7efe44a00640 1 --2- 172.21.15.46:0/180961008 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe400cea30 0x7efe401606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.707+0000 7efe44a00640 1 --2- 172.21.15.46:0/180961008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe4015c950 0x7efe4006bbe0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.707+0000 7efe44a00640 1 -- 172.21.15.46:0/180961008 >> 172.21.15.46:0/180961008 conn(0x7efe400b21b0 msgr2=0x7efe400b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.707+0000 7efe44a00640 1 -- 172.21.15.46:0/180961008 shutdown_connections 2024-03-29T08:27:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.707+0000 7efe44a00640 1 -- 172.21.15.46:0/180961008 wait complete. 2024-03-29T08:27:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.707+0000 7efe44a00640 1 Processor -- start 2024-03-29T08:27:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.707+0000 7efe44a00640 1 -- start start 2024-03-29T08:27:28.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe44a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe400cea30 0x7efe40157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:28.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe44a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe4015c950 0x7efe4009b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:28.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe44a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe4015cd30 0x7efe400b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe44a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe400b39d0 con 0x7efe4015cd30 2024-03-29T08:27:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe44a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe40157cb0 con 0x7efe400cea30 2024-03-29T08:27:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe44a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4009d510 con 0x7efe4015c950 2024-03-29T08:27:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe3e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe4015cd30 0x7efe400b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe3e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe4015cd30 0x7efe400b50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35942/0 (socket says 172.21.15.46:35942) 2024-03-29T08:27:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe3e800640 1 -- 172.21.15.46:0/2525662993 learned_addr learned my addr 172.21.15.46:0/2525662993 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe37e00640 1 --2- 172.21.15.46:0/2525662993 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe4015c950 0x7efe4009b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe3de00640 1 --2- 172.21.15.46:0/2525662993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe400cea30 0x7efe40157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe3e800640 1 -- 172.21.15.46:0/2525662993 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe4015c950 msgr2=0x7efe4009b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe3e800640 1 --2- 172.21.15.46:0/2525662993 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe4015c950 0x7efe4009b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe3e800640 1 -- 172.21.15.46:0/2525662993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe400cea30 msgr2=0x7efe40157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe3e800640 1 --2- 172.21.15.46:0/2525662993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe400cea30 0x7efe40157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe3e800640 1 -- 172.21.15.46:0/2525662993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe300094d0 con 0x7efe4015cd30 2024-03-29T08:27:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe3e800640 1 --2- 172.21.15.46:0/2525662993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe4015cd30 0x7efe400b50c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7efe38007e60 tx=0x7efe3800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:28.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe36000640 1 -- 172.21.15.46:0/2525662993 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38014420 con 0x7efe4015cd30 2024-03-29T08:27:28.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe36000640 1 -- 172.21.15.46:0/2525662993 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe380145a0 con 0x7efe4015cd30 2024-03-29T08:27:28.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe36000640 1 -- 172.21.15.46:0/2525662993 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38014950 con 0x7efe4015cd30 2024-03-29T08:27:28.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe44a00640 1 -- 172.21.15.46:0/2525662993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe400b5b10 con 0x7efe4015cd30 2024-03-29T08:27:28.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe44a00640 1 -- 172.21.15.46:0/2525662993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe400b35f0 con 0x7efe4015cd30 2024-03-29T08:27:28.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe44a00640 1 -- 172.21.15.46:0/2525662993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe400b3b10 con 0x7efe4015cd30 2024-03-29T08:27:28.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.711+0000 7efe36000640 1 -- 172.21.15.46:0/2525662993 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7efe38014ab0 con 0x7efe4015cd30 2024-03-29T08:27:28.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.715+0000 7efe36000640 1 --2- 172.21.15.46:0/2525662993 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe1c066690 0x7efe1c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:28.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.715+0000 7efe36000640 1 -- 172.21.15.46:0/2525662993 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe38084840 con 0x7efe4015cd30 2024-03-29T08:27:28.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.715+0000 7efe3de00640 1 --2- 172.21.15.46:0/2525662993 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe1c066690 0x7efe1c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:28.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.715+0000 7efe3de00640 1 --2- 172.21.15.46:0/2525662993 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe1c066690 0x7efe1c068b50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7efe300b2000 tx=0x7efe30009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:28.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.719+0000 7efe36000640 1 -- 172.21.15.46:0/2525662993 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7efe400b3b10 con 0x7efe4015cd30 2024-03-29T08:27:28.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.879+0000 7efe44a00640 1 -- 172.21.15.46:0/2525662993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe400b3b10 con 0x7efe4015cd30 2024-03-29T08:27:28.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.879+0000 7efe36000640 1 -- 172.21.15.46:0/2525662993 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe400b3b10 con 0x7efe4015cd30 2024-03-29T08:27:28.882 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:27:28.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.883+0000 7efe44a00640 1 -- 172.21.15.46:0/2525662993 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe1c066690 msgr2=0x7efe1c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:28.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.883+0000 7efe44a00640 1 --2- 172.21.15.46:0/2525662993 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe1c066690 0x7efe1c068b50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7efe300b2000 tx=0x7efe30009040 comp rx=0 tx=0).stop 2024-03-29T08:27:28.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.883+0000 7efe44a00640 1 -- 172.21.15.46:0/2525662993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe4015cd30 msgr2=0x7efe400b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:28.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.883+0000 7efe44a00640 1 --2- 172.21.15.46:0/2525662993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe4015cd30 0x7efe400b50c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7efe38007e60 tx=0x7efe3800e4b0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.883+0000 7efe44a00640 1 -- 172.21.15.46:0/2525662993 shutdown_connections 2024-03-29T08:27:28.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.883+0000 7efe44a00640 1 --2- 172.21.15.46:0/2525662993 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7efe1c066690 0x7efe1c068b50 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.883+0000 7efe44a00640 1 --2- 172.21.15.46:0/2525662993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efe4015cd30 0x7efe400b50c0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.883+0000 7efe44a00640 1 --2- 172.21.15.46:0/2525662993 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7efe4015c950 0x7efe4009b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.883+0000 7efe44a00640 1 --2- 172.21.15.46:0/2525662993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe400cea30 0x7efe40157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:28.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.883+0000 7efe44a00640 1 -- 172.21.15.46:0/2525662993 >> 172.21.15.46:0/2525662993 conn(0x7efe400b21b0 msgr2=0x7efe4015b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:28.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.883+0000 7efe44a00640 1 -- 172.21.15.46:0/2525662993 shutdown_connections 2024-03-29T08:27:28.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:28.883+0000 7efe44a00640 1 -- 172.21.15.46:0/2525662993 wait complete. 2024-03-29T08:27:28.903 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:27:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.047+0000 7faf96000640 1 Processor -- start 2024-03-29T08:27:29.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.047+0000 7faf96000640 1 -- start start 2024-03-29T08:27:29.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.047+0000 7faf96000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faf9015df90 0x7faf900ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.047+0000 7faf96000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf900cbea0 0x7faf900ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.047+0000 7faf96000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9015d260 con 0x7faf90142a50 2024-03-29T08:27:29.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.047+0000 7faf96000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf900aed00 con 0x7faf9015df90 2024-03-29T08:27:29.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.047+0000 7faf96000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf900aefd0 con 0x7faf900cbea0 2024-03-29T08:27:29.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.047+0000 7faf87e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faf9015df90 0x7faf900ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.047+0000 7faf8fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf900cbea0 0x7faf900ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.047+0000 7faf87e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faf9015df90 0x7faf900ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33308/0 (socket says 172.21.15.46:33308) 2024-03-29T08:27:29.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.047+0000 7faf87e00640 1 -- 172.21.15.46:0/4205380125 learned_addr learned my addr 172.21.15.46:0/4205380125 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:29.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.047+0000 7faf87e00640 1 -- 172.21.15.46:0/4205380125 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf900cbea0 msgr2=0x7faf900ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:29.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.047+0000 7faf87e00640 1 --2- 172.21.15.46:0/4205380125 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf900cbea0 0x7faf900ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.047+0000 7faf87e00640 1 -- 172.21.15.46:0/4205380125 >> v1:172.21.15.73:6789/0 conn(0x7faf90142a50 legacy=0x7faf900cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.047+0000 7faf87e00640 1 -- 172.21.15.46:0/4205380125 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9015d4b0 con 0x7faf9015df90 2024-03-29T08:27:29.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.051+0000 7faf87e00640 1 --2- 172.21.15.46:0/4205380125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf9015df90 0x7faf900ae280 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7faf78009870 tx=0x7faf7801c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5d221f976243c7f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:29.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.051+0000 7faf8ea00640 1 -- 172.21.15.46:0/4205380125 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf780ab600 con 0x7faf9015df90 2024-03-29T08:27:29.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.051+0000 7faf8ea00640 1 -- 172.21.15.46:0/4205380125 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf780a7050 con 0x7faf9015df90 2024-03-29T08:27:29.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.051+0000 7faf8ea00640 1 -- 172.21.15.46:0/4205380125 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf780ab980 con 0x7faf9015df90 2024-03-29T08:27:29.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.051+0000 7faf96000640 1 -- 172.21.15.46:0/4205380125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf9015df90 msgr2=0x7faf900ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.051+0000 7faf96000640 1 --2- 172.21.15.46:0/4205380125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf9015df90 0x7faf900ae280 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7faf78009870 tx=0x7faf7801c4d0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.051+0000 7faf96000640 1 -- 172.21.15.46:0/4205380125 shutdown_connections 2024-03-29T08:27:29.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.051+0000 7faf96000640 1 --2- 172.21.15.46:0/4205380125 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf900cbea0 0x7faf900ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.051+0000 7faf96000640 1 --2- 172.21.15.46:0/4205380125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf9015df90 0x7faf900ae280 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.051+0000 7faf96000640 1 -- 172.21.15.46:0/4205380125 >> 172.21.15.46:0/4205380125 conn(0x7faf9006a390 msgr2=0x7faf900cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:29.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.051+0000 7faf96000640 1 -- 172.21.15.46:0/4205380125 shutdown_connections 2024-03-29T08:27:29.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.051+0000 7faf96000640 1 -- 172.21.15.46:0/4205380125 wait complete. 2024-03-29T08:27:29.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf96000640 1 Processor -- start 2024-03-29T08:27:29.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf96000640 1 -- start start 2024-03-29T08:27:29.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf96000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf900cbea0 0x7faf9013b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf96000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90142a50 0x7faf900a74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf96000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf9015df90 0x7faf90159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf96000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9013e340 con 0x7faf90142a50 2024-03-29T08:27:29.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf96000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf90129930 con 0x7faf9015df90 2024-03-29T08:27:29.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf96000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf90129c10 con 0x7faf900cbea0 2024-03-29T08:27:29.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90142a50 0x7faf900a74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf87e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90142a50 0x7faf900a74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35944/0 (socket says 172.21.15.46:35944) 2024-03-29T08:27:29.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf87e00640 1 -- 172.21.15.46:0/3444248660 learned_addr learned my addr 172.21.15.46:0/3444248660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:29.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf8fe00640 1 --2- 172.21.15.46:0/3444248660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf9015df90 0x7faf90159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf8f400640 1 --2- 172.21.15.46:0/3444248660 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf900cbea0 0x7faf9013b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf87e00640 1 -- 172.21.15.46:0/3444248660 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf900cbea0 msgr2=0x7faf9013b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf87e00640 1 --2- 172.21.15.46:0/3444248660 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf900cbea0 0x7faf9013b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf87e00640 1 -- 172.21.15.46:0/3444248660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf9015df90 msgr2=0x7faf90159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf87e00640 1 --2- 172.21.15.46:0/3444248660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf9015df90 0x7faf90159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf87e00640 1 -- 172.21.15.46:0/3444248660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf780094d0 con 0x7faf90142a50 2024-03-29T08:27:29.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf8fe00640 1 --2- 172.21.15.46:0/3444248660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf9015df90 0x7faf90159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:29.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf87e00640 1 --2- 172.21.15.46:0/3444248660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90142a50 0x7faf900a74a0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7faf780ae040 tx=0x7faf78019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:29.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf87400640 1 -- 172.21.15.46:0/3444248660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf780b6420 con 0x7faf90142a50 2024-03-29T08:27:29.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf87400640 1 -- 172.21.15.46:0/3444248660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf780b65a0 con 0x7faf90142a50 2024-03-29T08:27:29.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf87400640 1 -- 172.21.15.46:0/3444248660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf780b6950 con 0x7faf90142a50 2024-03-29T08:27:29.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf96000640 1 -- 172.21.15.46:0/3444248660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf90129ef0 con 0x7faf90142a50 2024-03-29T08:27:29.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf96000640 1 -- 172.21.15.46:0/3444248660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9012a4b0 con 0x7faf90142a50 2024-03-29T08:27:29.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.055+0000 7faf96000640 1 -- 172.21.15.46:0/3444248660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf90071a70 con 0x7faf90142a50 2024-03-29T08:27:29.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.059+0000 7faf87400640 1 -- 172.21.15.46:0/3444248660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7faf780b2040 con 0x7faf90142a50 2024-03-29T08:27:29.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.059+0000 7faf87400640 1 --2- 172.21.15.46:0/3444248660 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf6c066690 0x7faf6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.059+0000 7faf87400640 1 -- 172.21.15.46:0/3444248660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf7811e380 con 0x7faf90142a50 2024-03-29T08:27:29.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.059+0000 7faf8f400640 1 --2- 172.21.15.46:0/3444248660 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf6c066690 0x7faf6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.059+0000 7faf8f400640 1 --2- 172.21.15.46:0/3444248660 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf6c066690 0x7faf6c068b50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7faf80000f70 tx=0x7faf80009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:29.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.063+0000 7faf87400640 1 -- 172.21.15.46:0/3444248660 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7faf780eb6e0 con 0x7faf90142a50 2024-03-29T08:27:29.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.223+0000 7faf96000640 1 -- 172.21.15.46:0/3444248660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faf9004e120 con 0x7faf90142a50 2024-03-29T08:27:29.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.223+0000 7faf87400640 1 -- 172.21.15.46:0/3444248660 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7faf780eed00 con 0x7faf90142a50 2024-03-29T08:27:29.225 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:27:29.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.223+0000 7faf96000640 1 -- 172.21.15.46:0/3444248660 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf6c066690 msgr2=0x7faf6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.223+0000 7faf96000640 1 --2- 172.21.15.46:0/3444248660 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf6c066690 0x7faf6c068b50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7faf80000f70 tx=0x7faf80009040 comp rx=0 tx=0).stop 2024-03-29T08:27:29.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.223+0000 7faf96000640 1 -- 172.21.15.46:0/3444248660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90142a50 msgr2=0x7faf900a74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.223+0000 7faf96000640 1 --2- 172.21.15.46:0/3444248660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90142a50 0x7faf900a74a0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7faf780ae040 tx=0x7faf78019270 comp rx=0 tx=0).stop 2024-03-29T08:27:29.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.223+0000 7faf96000640 1 -- 172.21.15.46:0/3444248660 shutdown_connections 2024-03-29T08:27:29.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.223+0000 7faf96000640 1 --2- 172.21.15.46:0/3444248660 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faf6c066690 0x7faf6c068b50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.223+0000 7faf96000640 1 --2- 172.21.15.46:0/3444248660 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf9015df90 0x7faf90159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.223+0000 7faf96000640 1 --2- 172.21.15.46:0/3444248660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90142a50 0x7faf900a74a0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.223+0000 7faf96000640 1 --2- 172.21.15.46:0/3444248660 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faf900cbea0 0x7faf9013b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.223+0000 7faf96000640 1 -- 172.21.15.46:0/3444248660 >> 172.21.15.46:0/3444248660 conn(0x7faf9006a390 msgr2=0x7faf900cb290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:29.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.223+0000 7faf96000640 1 -- 172.21.15.46:0/3444248660 shutdown_connections 2024-03-29T08:27:29.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.223+0000 7faf96000640 1 -- 172.21.15.46:0/3444248660 wait complete. 2024-03-29T08:27:29.246 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:27:29.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.387+0000 7fcdaa200640 1 Processor -- start 2024-03-29T08:27:29.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.387+0000 7fcdaa200640 1 -- start start 2024-03-29T08:27:29.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.387+0000 7fcdaa200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcda4143eb0 0x7fcda40cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.387+0000 7fcdaa200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcda415efe0 0x7fcda40cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.387+0000 7fcdaa200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda40b2360 con 0x7fcda40b1630 2024-03-29T08:27:29.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.387+0000 7fcdaa200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda415e500 con 0x7fcda4143eb0 2024-03-29T08:27:29.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.387+0000 7fcdaa200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda415e7e0 con 0x7fcda415efe0 2024-03-29T08:27:29.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.387+0000 7fcda3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcda40b1630 0x7fcda40b1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59618/0 (socket says 172.21.15.46:59618) 2024-03-29T08:27:29.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.387+0000 7fcda3e00640 1 -- 172.21.15.46:0/282875940 learned_addr learned my addr 172.21.15.46:0/282875940 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:29.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.387+0000 7fcda3400640 1 --2- 172.21.15.46:0/282875940 >> v2:172.21.15.73:3300/0 conn(0x7fcda4143eb0 0x7fcda40cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.387+0000 7fcd9be00640 1 --2- 172.21.15.46:0/282875940 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcda415efe0 0x7fcda40cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.391+0000 7fcda2a00640 1 -- 172.21.15.46:0/282875940 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644341038 0 0) 0x7fcda40b2360 con 0x7fcda40b1630 2024-03-29T08:27:29.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.391+0000 7fcda2a00640 1 -- 172.21.15.46:0/282875940 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd88003530 con 0x7fcda40b1630 2024-03-29T08:27:29.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.391+0000 7fcda3400640 1 -- 172.21.15.46:0/282875940 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcda415efe0 msgr2=0x7fcda40cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:29.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.391+0000 7fcda3400640 1 --2- 172.21.15.46:0/282875940 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcda415efe0 0x7fcda40cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.391+0000 7fcda3400640 1 -- 172.21.15.46:0/282875940 >> v1:172.21.15.46:6789/0 conn(0x7fcda40b1630 legacy=0x7fcda40b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.391+0000 7fcda3400640 1 -- 172.21.15.46:0/282875940 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda415eac0 con 0x7fcda4143eb0 2024-03-29T08:27:29.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.391+0000 7fcd9be00640 1 --2- 172.21.15.46:0/282875940 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcda415efe0 0x7fcda40cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:27:29.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.391+0000 7fcda3400640 1 --2- 172.21.15.46:0/282875940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcda4143eb0 0x7fcda40cb790 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fcd940097c0 tx=0x7fcd9401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cb1798d9c833952 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:29.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.391+0000 7fcda2a00640 1 -- 172.21.15.46:0/282875940 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd940ab600 con 0x7fcda4143eb0 2024-03-29T08:27:29.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.391+0000 7fcda2a00640 1 -- 172.21.15.46:0/282875940 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd940a7050 con 0x7fcda4143eb0 2024-03-29T08:27:29.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.391+0000 7fcda2a00640 1 -- 172.21.15.46:0/282875940 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd940ab980 con 0x7fcda4143eb0 2024-03-29T08:27:29.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcdaa200640 1 -- 172.21.15.46:0/282875940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcda4143eb0 msgr2=0x7fcda40cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcdaa200640 1 --2- 172.21.15.46:0/282875940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcda4143eb0 0x7fcda40cb790 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fcd940097c0 tx=0x7fcd9401c4a0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcdaa200640 1 -- 172.21.15.46:0/282875940 shutdown_connections 2024-03-29T08:27:29.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcdaa200640 1 --2- 172.21.15.46:0/282875940 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcda415efe0 0x7fcda40cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcdaa200640 1 --2- 172.21.15.46:0/282875940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcda4143eb0 0x7fcda40cb790 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcdaa200640 1 -- 172.21.15.46:0/282875940 >> 172.21.15.46:0/282875940 conn(0x7fcda406a390 msgr2=0x7fcda40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:29.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcdaa200640 1 -- 172.21.15.46:0/282875940 shutdown_connections 2024-03-29T08:27:29.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcdaa200640 1 -- 172.21.15.46:0/282875940 wait complete. 2024-03-29T08:27:29.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcdaa200640 1 Processor -- start 2024-03-29T08:27:29.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcdaa200640 1 -- start start 2024-03-29T08:27:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcdaa200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40b1630 0x7fcda41f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcdaa200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcda4143eb0 0x7fcda40c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcdaa200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcda415efe0 0x7fcda40c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcdaa200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda41335b0 con 0x7fcda40b1630 2024-03-29T08:27:29.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcdaa200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda41f4e00 con 0x7fcda415efe0 2024-03-29T08:27:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcdaa200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda40c4b90 con 0x7fcda4143eb0 2024-03-29T08:27:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcda3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40b1630 0x7fcda41f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcda3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40b1630 0x7fcda41f48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35954/0 (socket says 172.21.15.46:35954) 2024-03-29T08:27:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcda3400640 1 -- 172.21.15.46:0/3053790459 learned_addr learned my addr 172.21.15.46:0/3053790459 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcd9be00640 1 --2- 172.21.15.46:0/3053790459 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcda4143eb0 0x7fcda40c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcda3e00640 1 --2- 172.21.15.46:0/3053790459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcda415efe0 0x7fcda40c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcda3400640 1 -- 172.21.15.46:0/3053790459 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcda4143eb0 msgr2=0x7fcda40c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcda3400640 1 --2- 172.21.15.46:0/3053790459 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcda4143eb0 0x7fcda40c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcda3400640 1 -- 172.21.15.46:0/3053790459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcda415efe0 msgr2=0x7fcda40c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcda3400640 1 --2- 172.21.15.46:0/3053790459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcda415efe0 0x7fcda40c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcda3400640 1 -- 172.21.15.46:0/3053790459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c009420 con 0x7fcda40b1630 2024-03-29T08:27:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcda3e00640 1 --2- 172.21.15.46:0/3053790459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcda415efe0 0x7fcda40c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.395+0000 7fcda3400640 1 --2- 172.21.15.46:0/3053790459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40b1630 0x7fcda41f48c0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fcd9401a0d0 tx=0x7fcd94019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:29.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.399+0000 7fcd9b400640 1 -- 172.21.15.46:0/3053790459 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd940ba070 con 0x7fcda40b1630 2024-03-29T08:27:29.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.399+0000 7fcd9b400640 1 -- 172.21.15.46:0/3053790459 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd940b4400 con 0x7fcda40b1630 2024-03-29T08:27:29.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.399+0000 7fcdaa200640 1 -- 172.21.15.46:0/3053790459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd94009420 con 0x7fcda40b1630 2024-03-29T08:27:29.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.399+0000 7fcd9b400640 1 -- 172.21.15.46:0/3053790459 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd940b47d0 con 0x7fcda40b1630 2024-03-29T08:27:29.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.399+0000 7fcdaa200640 1 -- 172.21.15.46:0/3053790459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda40c60b0 con 0x7fcda40b1630 2024-03-29T08:27:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.399+0000 7fcdaa200640 1 -- 172.21.15.46:0/3053790459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda4071a70 con 0x7fcda40b1630 2024-03-29T08:27:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.399+0000 7fcd9b400640 1 -- 172.21.15.46:0/3053790459 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcd940b2040 con 0x7fcda40b1630 2024-03-29T08:27:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.399+0000 7fcd9b400640 1 --2- 172.21.15.46:0/3053790459 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcd80066690 0x7fcd80068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.399+0000 7fcd9b400640 1 -- 172.21.15.46:0/3053790459 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd9411d8e0 con 0x7fcda40b1630 2024-03-29T08:27:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.399+0000 7fcd9be00640 1 --2- 172.21.15.46:0/3053790459 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcd80066690 0x7fcd80068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.403+0000 7fcd9be00640 1 --2- 172.21.15.46:0/3053790459 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcd80066690 0x7fcd80068b50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fcda40c8820 tx=0x7fcd8c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.403+0000 7fcd9b400640 1 -- 172.21.15.46:0/3053790459 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcd940eac40 con 0x7fcda40b1630 2024-03-29T08:27:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.563+0000 7fcdaa200640 1 -- 172.21.15.46:0/3053790459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcda40cadd0 con 0x7fcda40b1630 2024-03-29T08:27:29.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.563+0000 7fcd9b400640 1 -- 172.21.15.46:0/3053790459 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd940ee260 con 0x7fcda40b1630 2024-03-29T08:27:29.570 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:27:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.567+0000 7fcdaa200640 1 -- 172.21.15.46:0/3053790459 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcd80066690 msgr2=0x7fcd80068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.567+0000 7fcdaa200640 1 --2- 172.21.15.46:0/3053790459 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcd80066690 0x7fcd80068b50 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fcda40c8820 tx=0x7fcd8c009190 comp rx=0 tx=0).stop 2024-03-29T08:27:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.567+0000 7fcdaa200640 1 -- 172.21.15.46:0/3053790459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40b1630 msgr2=0x7fcda41f48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.567+0000 7fcdaa200640 1 --2- 172.21.15.46:0/3053790459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40b1630 0x7fcda41f48c0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fcd9401a0d0 tx=0x7fcd94019240 comp rx=0 tx=0).stop 2024-03-29T08:27:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.567+0000 7fcdaa200640 1 -- 172.21.15.46:0/3053790459 shutdown_connections 2024-03-29T08:27:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.567+0000 7fcdaa200640 1 --2- 172.21.15.46:0/3053790459 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcd80066690 0x7fcd80068b50 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.567+0000 7fcdaa200640 1 --2- 172.21.15.46:0/3053790459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcda415efe0 0x7fcda40c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.567+0000 7fcdaa200640 1 --2- 172.21.15.46:0/3053790459 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcda4143eb0 0x7fcda40c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.567+0000 7fcdaa200640 1 --2- 172.21.15.46:0/3053790459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcda40b1630 0x7fcda41f48c0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.567+0000 7fcdaa200640 1 -- 172.21.15.46:0/3053790459 >> 172.21.15.46:0/3053790459 conn(0x7fcda406a390 msgr2=0x7fcda40b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.567+0000 7fcdaa200640 1 -- 172.21.15.46:0/3053790459 shutdown_connections 2024-03-29T08:27:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.567+0000 7fcdaa200640 1 -- 172.21.15.46:0/3053790459 wait complete. 2024-03-29T08:27:29.589 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:27:29.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.727+0000 7f5990e00640 1 Processor -- start 2024-03-29T08:27:29.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.727+0000 7f5990e00640 1 -- start start 2024-03-29T08:27:29.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.731+0000 7f5990e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f598c15c950 0x7f598c06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.731+0000 7f5990e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f598c160db0 0x7f598c0cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.731+0000 7f5990e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f598c0cfb20 con 0x7f598c0cea30 2024-03-29T08:27:29.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.731+0000 7f5990e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f598c0cef10 con 0x7f598c15c950 2024-03-29T08:27:29.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.731+0000 7f5990e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f598c0cf1f0 con 0x7f598c160db0 2024-03-29T08:27:29.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.731+0000 7f5989e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f598c15c950 0x7f598c06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.731+0000 7f5989e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f598c15c950 0x7f598c06bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35966/0 (socket says 172.21.15.46:35966) 2024-03-29T08:27:29.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.731+0000 7f5989e00640 1 -- 172.21.15.46:0/461904345 learned_addr learned my addr 172.21.15.46:0/461904345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:29.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.731+0000 7f598a800640 1 --2- 172.21.15.46:0/461904345 >> v2:172.21.15.73:3300/0 conn(0x7f598c160db0 0x7f598c0cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.731+0000 7f5989e00640 1 -- 172.21.15.46:0/461904345 >> v2:172.21.15.73:3300/0 conn(0x7f598c160db0 msgr2=0x7f598c0cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:29.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.731+0000 7f5989e00640 1 --2- 172.21.15.46:0/461904345 >> v2:172.21.15.73:3300/0 conn(0x7f598c160db0 0x7f598c0cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.731+0000 7f5989e00640 1 -- 172.21.15.46:0/461904345 >> v1:172.21.15.73:6789/0 conn(0x7f598c0cea30 legacy=0x7f598c1606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.731+0000 7f5989e00640 1 -- 172.21.15.46:0/461904345 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598c0cf4d0 con 0x7f598c15c950 2024-03-29T08:27:29.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5989e00640 1 --2- 172.21.15.46:0/461904345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15c950 0x7f598c06bbe0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f598c0723a0 tx=0x7f597c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f6ca59eee49d46 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:29.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5989400640 1 -- 172.21.15.46:0/461904345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c0a8a10 con 0x7f598c15c950 2024-03-29T08:27:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5989400640 1 -- 172.21.15.46:0/461904345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597c0a8b70 con 0x7f598c15c950 2024-03-29T08:27:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5989400640 1 -- 172.21.15.46:0/461904345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c0b0440 con 0x7f598c15c950 2024-03-29T08:27:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5990e00640 1 -- 172.21.15.46:0/461904345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15c950 msgr2=0x7f598c06bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5990e00640 1 --2- 172.21.15.46:0/461904345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15c950 0x7f598c06bbe0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f598c0723a0 tx=0x7f597c01ed10 comp rx=0 tx=0).stop 2024-03-29T08:27:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5990e00640 1 -- 172.21.15.46:0/461904345 shutdown_connections 2024-03-29T08:27:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5990e00640 1 --2- 172.21.15.46:0/461904345 >> v2:172.21.15.73:3300/0 conn(0x7f598c160db0 0x7f598c0cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5990e00640 1 --2- 172.21.15.46:0/461904345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15c950 0x7f598c06bbe0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5990e00640 1 -- 172.21.15.46:0/461904345 >> 172.21.15.46:0/461904345 conn(0x7f598c0b21b0 msgr2=0x7f598c0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5990e00640 1 -- 172.21.15.46:0/461904345 shutdown_connections 2024-03-29T08:27:29.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5990e00640 1 -- 172.21.15.46:0/461904345 wait complete. 2024-03-29T08:27:29.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5990e00640 1 Processor -- start 2024-03-29T08:27:29.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5990e00640 1 -- start start 2024-03-29T08:27:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5990e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598c0cea30 0x7f598c158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5990e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15c950 0x7f598c154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5990e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f598c160db0 0x7f598c159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5990e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c155580 con 0x7f598c15c950 2024-03-29T08:27:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5990e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598c1586b0 con 0x7f598c0cea30 2024-03-29T08:27:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.735+0000 7f5990e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598c0b4da0 con 0x7f598c160db0 2024-03-29T08:27:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f5983e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15c950 0x7f598c154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f5983e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15c950 0x7f598c154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35978/0 (socket says 172.21.15.46:35978) 2024-03-29T08:27:29.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f5983e00640 1 -- 172.21.15.46:0/4017850786 learned_addr learned my addr 172.21.15.46:0/4017850786 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:29.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f5989e00640 1 --2- 172.21.15.46:0/4017850786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598c0cea30 0x7f598c158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f598a800640 1 --2- 172.21.15.46:0/4017850786 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f598c160db0 0x7f598c159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f5983e00640 1 -- 172.21.15.46:0/4017850786 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f598c160db0 msgr2=0x7f598c159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f5983e00640 1 --2- 172.21.15.46:0/4017850786 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f598c160db0 0x7f598c159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f5983e00640 1 -- 172.21.15.46:0/4017850786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598c0cea30 msgr2=0x7f598c158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f5983e00640 1 --2- 172.21.15.46:0/4017850786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598c0cea30 0x7f598c158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f5983e00640 1 -- 172.21.15.46:0/4017850786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597c0094d0 con 0x7f598c15c950 2024-03-29T08:27:29.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f5983e00640 1 --2- 172.21.15.46:0/4017850786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15c950 0x7f598c154950 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f597400c4d0 tx=0x7f597400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:29.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f5982000640 1 -- 172.21.15.46:0/4017850786 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59740052d0 con 0x7f598c15c950 2024-03-29T08:27:29.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f5982000640 1 -- 172.21.15.46:0/4017850786 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5974005430 con 0x7f598c15c950 2024-03-29T08:27:29.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f5982000640 1 -- 172.21.15.46:0/4017850786 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5974012830 con 0x7f598c15c950 2024-03-29T08:27:29.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f5990e00640 1 -- 172.21.15.46:0/4017850786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598c1557f0 con 0x7f598c15c950 2024-03-29T08:27:29.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f5990e00640 1 -- 172.21.15.46:0/4017850786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598c154e90 con 0x7f598c15c950 2024-03-29T08:27:29.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f5990e00640 1 -- 172.21.15.46:0/4017850786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f598c0ccba0 con 0x7f598c15c950 2024-03-29T08:27:29.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.739+0000 7f5982000640 1 -- 172.21.15.46:0/4017850786 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5974012990 con 0x7f598c15c950 2024-03-29T08:27:29.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.743+0000 7f5982000640 1 --2- 172.21.15.46:0/4017850786 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5968066690 0x7f5968068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.743+0000 7f5982000640 1 -- 172.21.15.46:0/4017850786 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5974083b50 con 0x7f598c15c950 2024-03-29T08:27:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.743+0000 7f5989e00640 1 --2- 172.21.15.46:0/4017850786 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5968066690 0x7f5968068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.743+0000 7f5989e00640 1 --2- 172.21.15.46:0/4017850786 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5968066690 0x7f5968068b50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f597c0b2000 tx=0x7f597c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.743+0000 7f5982000640 1 -- 172.21.15.46:0/4017850786 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5974050f60 con 0x7f598c15c950 2024-03-29T08:27:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.915+0000 7f5990e00640 1 -- 172.21.15.46:0/4017850786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f598c06c0a0 con 0x7f598c15c950 2024-03-29T08:27:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.915+0000 7f5982000640 1 -- 172.21.15.46:0/4017850786 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5974054580 con 0x7f598c15c950 2024-03-29T08:27:29.919 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:27:29.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.919+0000 7f5990e00640 1 -- 172.21.15.46:0/4017850786 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5968066690 msgr2=0x7f5968068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.919+0000 7f5990e00640 1 --2- 172.21.15.46:0/4017850786 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5968066690 0x7f5968068b50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f597c0b2000 tx=0x7f597c009040 comp rx=0 tx=0).stop 2024-03-29T08:27:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.919+0000 7f5990e00640 1 -- 172.21.15.46:0/4017850786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15c950 msgr2=0x7f598c154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.919+0000 7f5990e00640 1 --2- 172.21.15.46:0/4017850786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15c950 0x7f598c154950 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f597400c4d0 tx=0x7f597400c990 comp rx=0 tx=0).stop 2024-03-29T08:27:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.919+0000 7f5990e00640 1 -- 172.21.15.46:0/4017850786 shutdown_connections 2024-03-29T08:27:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.919+0000 7f5990e00640 1 --2- 172.21.15.46:0/4017850786 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5968066690 0x7f5968068b50 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.919+0000 7f5990e00640 1 --2- 172.21.15.46:0/4017850786 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f598c160db0 0x7f598c159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.919+0000 7f5990e00640 1 --2- 172.21.15.46:0/4017850786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598c15c950 0x7f598c154950 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.919+0000 7f5990e00640 1 --2- 172.21.15.46:0/4017850786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598c0cea30 0x7f598c158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.919+0000 7f5990e00640 1 -- 172.21.15.46:0/4017850786 >> 172.21.15.46:0/4017850786 conn(0x7f598c0b21b0 msgr2=0x7f598c15d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.919+0000 7f5990e00640 1 -- 172.21.15.46:0/4017850786 shutdown_connections 2024-03-29T08:27:29.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:29.919+0000 7f5990e00640 1 -- 172.21.15.46:0/4017850786 wait complete. 2024-03-29T08:27:29.941 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-03-29T08:27:29.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:29.941 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T08:27:29.959 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[48103]: fuse finished with error 0 and tester_r 0 2024-03-29T08:27:29.963 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:27:29.996 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:27:29.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:29.996 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:27:30.053 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:27:30.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.231+0000 7fcdc8400640 1 Processor -- start 2024-03-29T08:27:30.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.231+0000 7fcdc8400640 1 -- start start 2024-03-29T08:27:30.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.235+0000 7fcdc8400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcdc015c9d0 0x7fcdc00cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:30.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.235+0000 7fcdc8400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdc00cef60 0x7fcdc0098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:30.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.235+0000 7fcdc8400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc00d0030 con 0x7fcdc00ccd10 2024-03-29T08:27:30.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.235+0000 7fcdc8400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc00cf420 con 0x7fcdc015c9d0 2024-03-29T08:27:30.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.235+0000 7fcdc8400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc00cf700 con 0x7fcdc00cef60 2024-03-29T08:27:30.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.235+0000 7fcdc5000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdc00cef60 0x7fcdc0098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:30.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.235+0000 7fcdc5a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcdc015c9d0 0x7fcdc00cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:30.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.235+0000 7fcdc5000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdc00cef60 0x7fcdc0098a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:60078/0 (socket says 172.21.15.46:60078) 2024-03-29T08:27:30.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.235+0000 7fcdc6400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fcdc00ccd10 0x7fcdc0160700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:37798/0 (socket says 172.21.15.46:37798) 2024-03-29T08:27:30.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.235+0000 7fcdc5000640 1 -- 172.21.15.46:0/3969243267 learned_addr learned my addr 172.21.15.46:0/3969243267 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:30.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.235+0000 7fcdc5a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcdc015c9d0 0x7fcdc00cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:33366/0 (socket says 172.21.15.46:33366) 2024-03-29T08:27:30.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.235+0000 7fcdc5a00640 1 -- 172.21.15.46:0/3969243267 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdc00cef60 msgr2=0x7fcdc0098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:30.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.235+0000 7fcdc5a00640 1 --2- 172.21.15.46:0/3969243267 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdc00cef60 0x7fcdc0098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.235+0000 7fcdc5a00640 1 -- 172.21.15.46:0/3969243267 >> v1:172.21.15.73:6789/0 conn(0x7fcdc00ccd10 legacy=0x7fcdc0160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:30.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.235+0000 7fcdc5a00640 1 -- 172.21.15.46:0/3969243267 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc00cf9e0 con 0x7fcdc015c9d0 2024-03-29T08:27:30.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.235+0000 7fcdc5000640 1 --2- 172.21.15.46:0/3969243267 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdc00cef60 0x7fcdc0098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:27:30.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.235+0000 7fcdbfe00640 1 -- 172.21.15.46:0/3969243267 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473019609 0 0) 0x7fcdc00d0030 con 0x7fcdc00ccd10 2024-03-29T08:27:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.239+0000 7fcdc5a00640 1 --2- 172.21.15.46:0/3969243267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdc015c9d0 0x7fcdc00cea20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40097c0 tx=0x7fcdb401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff5e1af67903c6d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.239+0000 7fcdbfe00640 1 -- 172.21.15.46:0/3969243267 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb40aa5f0 con 0x7fcdc015c9d0 2024-03-29T08:27:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.239+0000 7fcdbfe00640 1 -- 172.21.15.46:0/3969243267 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb40aa750 con 0x7fcdc015c9d0 2024-03-29T08:27:30.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.239+0000 7fcdbfe00640 1 -- 172.21.15.46:0/3969243267 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb40aaae0 con 0x7fcdc015c9d0 2024-03-29T08:27:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.239+0000 7fcdc8400640 1 -- 172.21.15.46:0/3969243267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdc015c9d0 msgr2=0x7fcdc00cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.239+0000 7fcdc8400640 1 --2- 172.21.15.46:0/3969243267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdc015c9d0 0x7fcdc00cea20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40097c0 tx=0x7fcdb401c470 comp rx=0 tx=0).stop 2024-03-29T08:27:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.239+0000 7fcdc8400640 1 -- 172.21.15.46:0/3969243267 shutdown_connections 2024-03-29T08:27:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.239+0000 7fcdc8400640 1 --2- 172.21.15.46:0/3969243267 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdc00cef60 0x7fcdc0098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.239+0000 7fcdc8400640 1 --2- 172.21.15.46:0/3969243267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdc015c9d0 0x7fcdc00cea20 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.239+0000 7fcdc8400640 1 -- 172.21.15.46:0/3969243267 >> 172.21.15.46:0/3969243267 conn(0x7fcdc00b21a0 msgr2=0x7fcdc00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.239+0000 7fcdc8400640 1 -- 172.21.15.46:0/3969243267 shutdown_connections 2024-03-29T08:27:30.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.239+0000 7fcdc8400640 1 -- 172.21.15.46:0/3969243267 wait complete. 2024-03-29T08:27:30.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.239+0000 7fcdc8400640 1 Processor -- start 2024-03-29T08:27:30.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.239+0000 7fcdc8400640 1 -- start start 2024-03-29T08:27:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.239+0000 7fcdc8400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdc00ccd10 0x7fcdc014a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc8400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdc00cef60 0x7fcdc015fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc8400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdc015c9d0 0x7fcdc0149c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc8400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc014bd40 con 0x7fcdc00cef60 2024-03-29T08:27:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc8400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc014aae0 con 0x7fcdc00ccd10 2024-03-29T08:27:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc8400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc014adc0 con 0x7fcdc015c9d0 2024-03-29T08:27:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdc00cef60 0x7fcdc015fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdc00cef60 0x7fcdc015fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35992/0 (socket says 172.21.15.46:35992) 2024-03-29T08:27:30.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc5000640 1 -- 172.21.15.46:0/2678067852 learned_addr learned my addr 172.21.15.46:0/2678067852 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc5a00640 1 --2- 172.21.15.46:0/2678067852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdc00ccd10 0x7fcdc014a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc6400640 1 --2- 172.21.15.46:0/2678067852 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdc015c9d0 0x7fcdc0149c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc5000640 1 -- 172.21.15.46:0/2678067852 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdc015c9d0 msgr2=0x7fcdc0149c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc5000640 1 --2- 172.21.15.46:0/2678067852 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdc015c9d0 0x7fcdc0149c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc5000640 1 -- 172.21.15.46:0/2678067852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdc00ccd10 msgr2=0x7fcdc014a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc5000640 1 --2- 172.21.15.46:0/2678067852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdc00ccd10 0x7fcdc014a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc5000640 1 -- 172.21.15.46:0/2678067852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac0094d0 con 0x7fcdc00cef60 2024-03-29T08:27:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc5000640 1 --2- 172.21.15.46:0/2678067852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdc00cef60 0x7fcdc015fc70 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00e9d0 tx=0x7fcdac00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdbd600640 1 -- 172.21.15.46:0/2678067852 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac00ca60 con 0x7fcdc00cef60 2024-03-29T08:27:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdbd600640 1 -- 172.21.15.46:0/2678067852 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdac00cbc0 con 0x7fcdc00cef60 2024-03-29T08:27:30.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdbd600640 1 -- 172.21.15.46:0/2678067852 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdac013470 con 0x7fcdc00cef60 2024-03-29T08:27:30.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc8400640 1 -- 172.21.15.46:0/2678067852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb4009420 con 0x7fcdc00cef60 2024-03-29T08:27:30.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc8400640 1 -- 172.21.15.46:0/2678067852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc014b380 con 0x7fcdc00cef60 2024-03-29T08:27:30.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdc8400640 1 -- 172.21.15.46:0/2678067852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc0066c90 con 0x7fcdc00cef60 2024-03-29T08:27:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.243+0000 7fcdbd600640 1 -- 172.21.15.46:0/2678067852 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcdac0135d0 con 0x7fcdc00cef60 2024-03-29T08:27:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.247+0000 7fcdbd600640 1 --2- 172.21.15.46:0/2678067852 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcda0066690 0x7fcda0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.247+0000 7fcdbd600640 1 -- 172.21.15.46:0/2678067852 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdac0847c0 con 0x7fcdc00cef60 2024-03-29T08:27:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.247+0000 7fcdc5a00640 1 --2- 172.21.15.46:0/2678067852 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcda0066690 0x7fcda0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:30.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.247+0000 7fcdc5a00640 1 --2- 172.21.15.46:0/2678067852 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcda0066690 0x7fcda0068b50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40ae000 tx=0x7fcdb40a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:30.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.251+0000 7fcdbd600640 1 -- 172.21.15.46:0/2678067852 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcdac051bd0 con 0x7fcdc00cef60 2024-03-29T08:27:30.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.443+0000 7fcdc8400640 1 -- 172.21.15.46:0/2678067852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcdc00b2ec0 con 0x7fcdc00cef60 2024-03-29T08:27:30.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.443+0000 7fcdbd600640 1 -- 172.21.15.46:0/2678067852 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4730 (secure 0 0 0) 0x7fcdac0551f0 con 0x7fcdc00cef60 2024-03-29T08:27:30.445 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:30.445 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4230030957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4230030957},{"type":"v1","addr":"172.21.15.73:6836","nonce":4230030957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/297138217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":297138217},{"type":"v1","addr":"172.21.15.73:6837","nonce":297138217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3856305332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3856305332},{"type":"v1","addr":"172.21.15.46:6833","nonce":3856305332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:27:18.583223+0000","modified":"2024-03-29T08:27:21.592418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7966},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7966":{"gid":7966,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/824184385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":824184385},{"type":"v1","addr":"172.21.15.46:6835","nonce":824184385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-29T08:27:30.446 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 77 2024-03-29T08:27:30.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.443+0000 7fcdc8400640 1 -- 172.21.15.46:0/2678067852 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcda0066690 msgr2=0x7fcda0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:30.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.443+0000 7fcdc8400640 1 --2- 172.21.15.46:0/2678067852 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcda0066690 0x7fcda0068b50 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40ae000 tx=0x7fcdb40a7000 comp rx=0 tx=0).stop 2024-03-29T08:27:30.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.443+0000 7fcdc8400640 1 -- 172.21.15.46:0/2678067852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdc00cef60 msgr2=0x7fcdc015fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:30.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.443+0000 7fcdc8400640 1 --2- 172.21.15.46:0/2678067852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdc00cef60 0x7fcdc015fc70 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00e9d0 tx=0x7fcdac00ee90 comp rx=0 tx=0).stop 2024-03-29T08:27:30.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.443+0000 7fcdc8400640 1 -- 172.21.15.46:0/2678067852 shutdown_connections 2024-03-29T08:27:30.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.443+0000 7fcdc8400640 1 --2- 172.21.15.46:0/2678067852 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcda0066690 0x7fcda0068b50 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.443+0000 7fcdc8400640 1 --2- 172.21.15.46:0/2678067852 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdc015c9d0 0x7fcdc0149c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.443+0000 7fcdc8400640 1 --2- 172.21.15.46:0/2678067852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdc00cef60 0x7fcdc015fc70 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.443+0000 7fcdc8400640 1 --2- 172.21.15.46:0/2678067852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdc00ccd10 0x7fcdc014a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.443+0000 7fcdc8400640 1 -- 172.21.15.46:0/2678067852 >> 172.21.15.46:0/2678067852 conn(0x7fcdc00b21a0 msgr2=0x7fcdc01557c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:30.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.443+0000 7fcdc8400640 1 -- 172.21.15.46:0/2678067852 shutdown_connections 2024-03-29T08:27:30.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.443+0000 7fcdc8400640 1 -- 172.21.15.46:0/2678067852 wait complete. 2024-03-29T08:27:30.467 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-03-29T08:27:30.638 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-03-29T08:27:30.639 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-29T08:27:30.639 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:27:30.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.775+0000 7fec90000640 1 Processor -- start 2024-03-29T08:27:30.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.775+0000 7fec90000640 1 -- start start 2024-03-29T08:27:30.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.779+0000 7fec90000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fec881593d0 0x7fec8806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:30.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.779+0000 7fec90000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fec8815cd90 0x7fec881597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:30.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.779+0000 7fec90000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8806d130 con 0x7fec8815d230 2024-03-29T08:27:30.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.779+0000 7fec90000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec8815b8f0 con 0x7fec8815cd90 2024-03-29T08:27:30.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.779+0000 7fec90000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec8815ad50 con 0x7fec881593d0 2024-03-29T08:27:30.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.779+0000 7fec8cc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fec8815d230 0x7fec8815d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59624/0 (socket says 172.21.15.46:59624) 2024-03-29T08:27:30.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.779+0000 7fec8cc00640 1 -- 172.21.15.46:0/2245342486 learned_addr learned my addr 172.21.15.46:0/2245342486 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:30.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.779+0000 7fec8e000640 1 --2- 172.21.15.46:0/2245342486 >> v2:172.21.15.46:3300/0 conn(0x7fec8815cd90 0x7fec881597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:30.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.779+0000 7fec8d600640 1 --2- 172.21.15.46:0/2245342486 >> v2:172.21.15.73:3300/0 conn(0x7fec881593d0 0x7fec8806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:30.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.779+0000 7fec87e00640 1 -- 172.21.15.46:0/2245342486 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1554971053 0 0) 0x7fec8806d130 con 0x7fec8815d230 2024-03-29T08:27:30.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.779+0000 7fec87e00640 1 -- 172.21.15.46:0/2245342486 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec70003530 con 0x7fec8815d230 2024-03-29T08:27:30.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.779+0000 7fec8e000640 1 -- 172.21.15.46:0/2245342486 >> v2:172.21.15.73:3300/0 conn(0x7fec881593d0 msgr2=0x7fec8806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:30.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.779+0000 7fec8e000640 1 --2- 172.21.15.46:0/2245342486 >> v2:172.21.15.73:3300/0 conn(0x7fec881593d0 0x7fec8806cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.779+0000 7fec8e000640 1 -- 172.21.15.46:0/2245342486 >> v1:172.21.15.46:6789/0 conn(0x7fec8815d230 legacy=0x7fec8815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:30.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.779+0000 7fec8e000640 1 -- 172.21.15.46:0/2245342486 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8815afc0 con 0x7fec8815cd90 2024-03-29T08:27:30.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.779+0000 7fec8d600640 1 --2- 172.21.15.46:0/2245342486 >> v2:172.21.15.73:3300/0 conn(0x7fec881593d0 0x7fec8806cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.779+0000 7fec8e000640 1 --2- 172.21.15.46:0/2245342486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec8815cd90 0x7fec881597b0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fec8806e460 tx=0x7fec8001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c78b5dbbff5ed84 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec87e00640 1 -- 172.21.15.46:0/2245342486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec800a59f0 con 0x7fec8815cd90 2024-03-29T08:27:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec87e00640 1 -- 172.21.15.46:0/2245342486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec800a5b50 con 0x7fec8815cd90 2024-03-29T08:27:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec87e00640 1 -- 172.21.15.46:0/2245342486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec800ad440 con 0x7fec8815cd90 2024-03-29T08:27:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec90000640 1 -- 172.21.15.46:0/2245342486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec8815cd90 msgr2=0x7fec881597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec90000640 1 --2- 172.21.15.46:0/2245342486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec8815cd90 0x7fec881597b0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fec8806e460 tx=0x7fec8001c930 comp rx=0 tx=0).stop 2024-03-29T08:27:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec90000640 1 -- 172.21.15.46:0/2245342486 shutdown_connections 2024-03-29T08:27:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec90000640 1 --2- 172.21.15.46:0/2245342486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec8815cd90 0x7fec881597b0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec90000640 1 --2- 172.21.15.46:0/2245342486 >> v2:172.21.15.73:3300/0 conn(0x7fec881593d0 0x7fec8806cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec90000640 1 -- 172.21.15.46:0/2245342486 >> 172.21.15.46:0/2245342486 conn(0x7fec88066110 msgr2=0x7fec88069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec90000640 1 -- 172.21.15.46:0/2245342486 shutdown_connections 2024-03-29T08:27:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec90000640 1 -- 172.21.15.46:0/2245342486 wait complete. 2024-03-29T08:27:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec90000640 1 Processor -- start 2024-03-29T08:27:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec90000640 1 -- start start 2024-03-29T08:27:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec90000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fec881593d0 0x7fec881f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec90000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec8815cd90 0x7fec880ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec90000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec8815d230 0x7fec880b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec90000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec881f96e0 con 0x7fec8815cd90 2024-03-29T08:27:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec90000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec880ce3a0 con 0x7fec8815d230 2024-03-29T08:27:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec90000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8815fce0 con 0x7fec881593d0 2024-03-29T08:27:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec8cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec8815cd90 0x7fec880ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec8cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec8815cd90 0x7fec880ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36016/0 (socket says 172.21.15.46:36016) 2024-03-29T08:27:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec8cc00640 1 -- 172.21.15.46:0/1529202779 learned_addr learned my addr 172.21.15.46:0/1529202779 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec8d600640 1 --2- 172.21.15.46:0/1529202779 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fec881593d0 0x7fec881f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.783+0000 7fec8e000640 1 --2- 172.21.15.46:0/1529202779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec8815d230 0x7fec880b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:30.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.787+0000 7fec8cc00640 1 -- 172.21.15.46:0/1529202779 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fec881593d0 msgr2=0x7fec881f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:30.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.787+0000 7fec8cc00640 1 --2- 172.21.15.46:0/1529202779 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fec881593d0 0x7fec881f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.787+0000 7fec8cc00640 1 -- 172.21.15.46:0/1529202779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec8815d230 msgr2=0x7fec880b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:30.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.787+0000 7fec8cc00640 1 --2- 172.21.15.46:0/1529202779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec8815d230 0x7fec880b1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.787+0000 7fec8cc00640 1 -- 172.21.15.46:0/1529202779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec80009a70 con 0x7fec8815cd90 2024-03-29T08:27:30.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.787+0000 7fec8cc00640 1 --2- 172.21.15.46:0/1529202779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec8815cd90 0x7fec880ccc50 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fec7400c450 tx=0x7fec7400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:30.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.787+0000 7fec85600640 1 -- 172.21.15.46:0/1529202779 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74005cf0 con 0x7fec8815cd90 2024-03-29T08:27:30.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.787+0000 7fec85600640 1 -- 172.21.15.46:0/1529202779 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec74005e50 con 0x7fec8815cd90 2024-03-29T08:27:30.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.787+0000 7fec85600640 1 -- 172.21.15.46:0/1529202779 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74007290 con 0x7fec8815cd90 2024-03-29T08:27:30.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.787+0000 7fec90000640 1 -- 172.21.15.46:0/1529202779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8815ffc0 con 0x7fec8815cd90 2024-03-29T08:27:30.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.787+0000 7fec90000640 1 -- 172.21.15.46:0/1529202779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec880cc450 con 0x7fec8815cd90 2024-03-29T08:27:30.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.787+0000 7fec90000640 1 -- 172.21.15.46:0/1529202779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec88066c90 con 0x7fec8815cd90 2024-03-29T08:27:30.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.787+0000 7fec85600640 1 -- 172.21.15.46:0/1529202779 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fec740073f0 con 0x7fec8815cd90 2024-03-29T08:27:30.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.787+0000 7fec85600640 1 --2- 172.21.15.46:0/1529202779 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fec68066690 0x7fec68068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:30.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.787+0000 7fec85600640 1 -- 172.21.15.46:0/1529202779 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec74083730 con 0x7fec8815cd90 2024-03-29T08:27:30.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.791+0000 7fec8d600640 1 --2- 172.21.15.46:0/1529202779 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fec68066690 0x7fec68068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.791+0000 7fec8d600640 1 --2- 172.21.15.46:0/1529202779 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fec68066690 0x7fec68068b50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fec88152260 tx=0x7fec7c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:30.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.795+0000 7fec85600640 1 -- 172.21.15.46:0/1529202779 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fec74050ab0 con 0x7fec8815cd90 2024-03-29T08:27:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.987+0000 7fec90000640 1 -- 172.21.15.46:0/1529202779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec88159990 con 0x7fec8815cd90 2024-03-29T08:27:30.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.987+0000 7fec85600640 1 -- 172.21.15.46:0/1529202779 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4730 (secure 0 0 0) 0x7fec74017090 con 0x7fec8815cd90 2024-03-29T08:27:30.990 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:30.990 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4230030957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4230030957},{"type":"v1","addr":"172.21.15.73:6836","nonce":4230030957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/297138217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":297138217},{"type":"v1","addr":"172.21.15.73:6837","nonce":297138217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3856305332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3856305332},{"type":"v1","addr":"172.21.15.46:6833","nonce":3856305332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:27:18.583223+0000","modified":"2024-03-29T08:27:21.592418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7966},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7966":{"gid":7966,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/824184385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":824184385},{"type":"v1","addr":"172.21.15.46:6835","nonce":824184385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-29T08:27:30.990 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 77 2024-03-29T08:27:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.987+0000 7fec90000640 1 -- 172.21.15.46:0/1529202779 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fec68066690 msgr2=0x7fec68068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.987+0000 7fec90000640 1 --2- 172.21.15.46:0/1529202779 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fec68066690 0x7fec68068b50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fec88152260 tx=0x7fec7c008040 comp rx=0 tx=0).stop 2024-03-29T08:27:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.987+0000 7fec90000640 1 -- 172.21.15.46:0/1529202779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec8815cd90 msgr2=0x7fec880ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.987+0000 7fec90000640 1 --2- 172.21.15.46:0/1529202779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec8815cd90 0x7fec880ccc50 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fec7400c450 tx=0x7fec7400c910 comp rx=0 tx=0).stop 2024-03-29T08:27:30.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.987+0000 7fec90000640 1 -- 172.21.15.46:0/1529202779 shutdown_connections 2024-03-29T08:27:30.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.987+0000 7fec90000640 1 --2- 172.21.15.46:0/1529202779 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fec68066690 0x7fec68068b50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.987+0000 7fec90000640 1 --2- 172.21.15.46:0/1529202779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec8815d230 0x7fec880b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.987+0000 7fec90000640 1 --2- 172.21.15.46:0/1529202779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec8815cd90 0x7fec880ccc50 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.987+0000 7fec90000640 1 --2- 172.21.15.46:0/1529202779 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fec881593d0 0x7fec881f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:30.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.987+0000 7fec90000640 1 -- 172.21.15.46:0/1529202779 >> 172.21.15.46:0/1529202779 conn(0x7fec88066110 msgr2=0x7fec88152da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:30.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.987+0000 7fec90000640 1 -- 172.21.15.46:0/1529202779 shutdown_connections 2024-03-29T08:27:30.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:30.987+0000 7fec90000640 1 -- 172.21.15.46:0/1529202779 wait complete. 2024-03-29T08:27:31.012 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-03-29T08:27:31.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.055+0000 7f18248d3080 1 Processor -- start 2024-03-29T08:27:31.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.055+0000 7f18248d3080 1 -- start start 2024-03-29T08:27:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.055+0000 7f18248d3080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8c1b81270 0x55f8c1c0c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.055+0000 7f18248d3080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f8c1b83eb0 0x55f8c1c0c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.055+0000 7f18248d3080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c1b9e0d0 0x55f8c1c24e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.055+0000 7f18248d3080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8c1a9c960 con 0x55f8c1b9e0d0 2024-03-29T08:27:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.055+0000 7f18248d3080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8c1a9c600 con 0x55f8c1b81270 2024-03-29T08:27:31.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.055+0000 7f18248d3080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8c1a93690 con 0x55f8c1b83eb0 2024-03-29T08:27:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.055+0000 7f1822400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c1b9e0d0 0x55f8c1c24e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1822400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c1b9e0d0 0x55f8c1c24e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36022/0 (socket says 172.21.15.46:36022) 2024-03-29T08:27:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1822400640 1 -- 172.21.15.46:0/76480214 learned_addr learned my addr 172.21.15.46:0/76480214 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1821a00640 1 --2- 172.21.15.46:0/76480214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8c1b81270 0x55f8c1c0c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1821000640 1 --2- 172.21.15.46:0/76480214 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f8c1b83eb0 0x55f8c1c0c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1822400640 1 -- 172.21.15.46:0/76480214 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f8c1b83eb0 msgr2=0x55f8c1c0c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1822400640 1 --2- 172.21.15.46:0/76480214 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f8c1b83eb0 0x55f8c1c0c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1822400640 1 -- 172.21.15.46:0/76480214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8c1b81270 msgr2=0x55f8c1c0c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1822400640 1 --2- 172.21.15.46:0/76480214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8c1b81270 0x55f8c1c0c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1822400640 1 -- 172.21.15.46:0/76480214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1810005df0 con 0x55f8c1b9e0d0 2024-03-29T08:27:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1822400640 1 --2- 172.21.15.46:0/76480214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c1b9e0d0 0x55f8c1c24e20 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f181c001550 tx=0x7f181c001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33858a10ba415410 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f181b400640 1 -- 172.21.15.46:0/76480214 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c013620 con 0x55f8c1b9e0d0 2024-03-29T08:27:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f181b400640 1 -- 172.21.15.46:0/76480214 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c013780 con 0x55f8c1b9e0d0 2024-03-29T08:27:31.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f181b400640 1 -- 172.21.15.46:0/76480214 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c013b10 con 0x55f8c1b9e0d0 2024-03-29T08:27:31.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f18248d3080 1 -- 172.21.15.46:0/76480214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c1b9e0d0 msgr2=0x55f8c1c24e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f18248d3080 1 --2- 172.21.15.46:0/76480214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c1b9e0d0 0x55f8c1c24e20 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f181c001550 tx=0x7f181c001a10 comp rx=0 tx=0).stop 2024-03-29T08:27:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f18248d3080 1 -- 172.21.15.46:0/76480214 shutdown_connections 2024-03-29T08:27:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f18248d3080 1 --2- 172.21.15.46:0/76480214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c1b9e0d0 0x55f8c1c24e20 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f18248d3080 1 --2- 172.21.15.46:0/76480214 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f8c1b83eb0 0x55f8c1c0c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f18248d3080 1 --2- 172.21.15.46:0/76480214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8c1b81270 0x55f8c1c0c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f18248d3080 1 -- 172.21.15.46:0/76480214 >> 172.21.15.46:0/76480214 conn(0x55f8c1b7f910 msgr2=0x55f8c1b84b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f18248d3080 1 -- 172.21.15.46:0/76480214 shutdown_connections 2024-03-29T08:27:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f18248d3080 1 -- 172.21.15.46:0/76480214 wait complete. 2024-03-29T08:27:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f18248d3080 1 Processor -- start 2024-03-29T08:27:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f18248d3080 1 -- start start 2024-03-29T08:27:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f18248d3080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f8c1b81270 0x55f8c1c24f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f18248d3080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c1b83eb0 0x55f8c1b9b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:31.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f18248d3080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8c1b9e0d0 0x55f8c1b849a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f18248d3080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8c1b582d0 con 0x55f8c1b83eb0 2024-03-29T08:27:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f18248d3080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8c1c24650 con 0x55f8c1b9e0d0 2024-03-29T08:27:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f18248d3080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8c1b991e0 con 0x55f8c1b81270 2024-03-29T08:27:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1821000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c1b83eb0 0x55f8c1b9b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1821000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c1b83eb0 0x55f8c1b9b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36030/0 (socket says 172.21.15.46:36030) 2024-03-29T08:27:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1821000640 1 -- 172.21.15.46:0/4017362491 learned_addr learned my addr 172.21.15.46:0/4017362491 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1821a00640 1 --2- 172.21.15.46:0/4017362491 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f8c1b81270 0x55f8c1c24f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1822400640 1 --2- 172.21.15.46:0/4017362491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8c1b9e0d0 0x55f8c1b849a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1821000640 1 -- 172.21.15.46:0/4017362491 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f8c1b81270 msgr2=0x55f8c1c24f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1821000640 1 --2- 172.21.15.46:0/4017362491 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f8c1b81270 0x55f8c1c24f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1821000640 1 -- 172.21.15.46:0/4017362491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8c1b9e0d0 msgr2=0x55f8c1b849a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1821000640 1 --2- 172.21.15.46:0/4017362491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f8c1b9e0d0 0x55f8c1b849a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.059+0000 7f1821000640 1 -- 172.21.15.46:0/4017362491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8c1a9e970 con 0x55f8c1b83eb0 2024-03-29T08:27:31.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.063+0000 7f1821000640 1 --2- 172.21.15.46:0/4017362491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c1b83eb0 0x55f8c1b9b500 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f181000a590 tx=0x7f18100184f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:31.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.063+0000 7f181a000640 1 -- 172.21.15.46:0/4017362491 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18100bb850 con 0x55f8c1b83eb0 2024-03-29T08:27:31.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.063+0000 7f181a000640 1 -- 172.21.15.46:0/4017362491 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18100b5040 con 0x55f8c1b83eb0 2024-03-29T08:27:31.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.063+0000 7f181a000640 1 -- 172.21.15.46:0/4017362491 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18100bbbb0 con 0x55f8c1b83eb0 2024-03-29T08:27:31.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.063+0000 7f18248d3080 1 -- 172.21.15.46:0/4017362491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f8c1aab8c0 con 0x55f8c1b83eb0 2024-03-29T08:27:31.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.063+0000 7f18248d3080 1 -- 172.21.15.46:0/4017362491 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f8c1a9ed00 con 0x55f8c1b83eb0 2024-03-29T08:27:31.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.063+0000 7f181a000640 1 -- 172.21.15.46:0/4017362491 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f18100bbd10 con 0x55f8c1b83eb0 2024-03-29T08:27:31.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.063+0000 7f181a000640 1 --2- 172.21.15.46:0/4017362491 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18040650a0 0x7f1804067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:31.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.063+0000 7f181a000640 1 -- 172.21.15.46:0/4017362491 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f181011e780 con 0x55f8c1b83eb0 2024-03-29T08:27:31.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.063+0000 7f18248d3080 1 --2- 172.21.15.46:0/4017362491 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x55f8c1b57ab0 0x55f8c1c26d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:31.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.063+0000 7f1821a00640 1 --2- 172.21.15.46:0/4017362491 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18040650a0 0x7f1804067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.063+0000 7f18248d3080 1 -- 172.21.15.46:0/4017362491 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e106) v8 -- 0x55f8c1c272c0 con 0x55f8c1b57ab0 2024-03-29T08:27:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.063+0000 7f1822400640 1 --2- 172.21.15.46:0/4017362491 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x55f8c1b57ab0 0x55f8c1c26d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.067+0000 7f1821a00640 1 --2- 172.21.15.46:0/4017362491 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18040650a0 0x7f1804067560 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f18140098d0 tx=0x7f1814005af0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.067+0000 7f1822400640 1 --2- 172.21.15.46:0/4017362491 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x55f8c1b57ab0 0x55f8c1c26d80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:31.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.067+0000 7f1822400640 1 -- 172.21.15.46:0/4017362491 <== osd.5 v2:172.21.15.73:6808/27690 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x7f181c0134a0 con 0x55f8c1b57ab0 2024-03-29T08:27:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.067+0000 7f18248d3080 1 -- 172.21.15.46:0/4017362491 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x55f8c1b57ab0 msgr2=0x55f8c1c26d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.067+0000 7f18248d3080 1 --2- 172.21.15.46:0/4017362491 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x55f8c1b57ab0 0x55f8c1c26d80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.067+0000 7f18248d3080 1 -- 172.21.15.46:0/4017362491 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18040650a0 msgr2=0x7f1804067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.067+0000 7f18248d3080 1 --2- 172.21.15.46:0/4017362491 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18040650a0 0x7f1804067560 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f18140098d0 tx=0x7f1814005af0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.067+0000 7f18248d3080 1 -- 172.21.15.46:0/4017362491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c1b83eb0 msgr2=0x55f8c1b9b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.067+0000 7f18248d3080 1 --2- 172.21.15.46:0/4017362491 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f8c1b83eb0 0x55f8c1b9b500 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f181000a590 tx=0x7f18100184f0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.067+0000 7f1822400640 1 -- 172.21.15.46:0/4017362491 reap_dead start 2024-03-29T08:27:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.067+0000 7f18248d3080 1 -- 172.21.15.46:0/4017362491 shutdown_connections 2024-03-29T08:27:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.067+0000 7f18248d3080 1 -- 172.21.15.46:0/4017362491 >> 172.21.15.46:0/4017362491 conn(0x55f8c1b7f910 msgr2=0x55f8c1b84290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.067+0000 7f18248d3080 1 -- 172.21.15.46:0/4017362491 shutdown_connections 2024-03-29T08:27:31.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.067+0000 7f18248d3080 1 -- 172.21.15.46:0/4017362491 wait complete. 2024-03-29T08:27:31.073 INFO:teuthology.orchestra.run.smithi046.stdout:����� 2024-03-29T08:27:31.074 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:27:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.251+0000 7fca40200640 1 Processor -- start 2024-03-29T08:27:31.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.251+0000 7fca40200640 1 -- start start 2024-03-29T08:27:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.255+0000 7fca40200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fca3815cd90 0x7fca3806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.255+0000 7fca40200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3806d130 con 0x7fca3815d940 2024-03-29T08:27:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.255+0000 7fca40200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3815b8f0 con 0x7fca381593d0 2024-03-29T08:27:31.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.255+0000 7fca40200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3815ad50 con 0x7fca3815cd90 2024-03-29T08:27:31.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.255+0000 7fca3e200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fca3815d940 0x7fca381597b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59628/0 (socket says 172.21.15.46:59628) 2024-03-29T08:27:31.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.255+0000 7fca3e200640 1 -- 172.21.15.46:0/3013296613 learned_addr learned my addr 172.21.15.46:0/3013296613 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:31.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.255+0000 7fca3ce00640 1 --2- 172.21.15.46:0/3013296613 >> v2:172.21.15.73:3300/0 conn(0x7fca3815cd90 0x7fca3806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:31.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.255+0000 7fca37e00640 1 -- 172.21.15.46:0/3013296613 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264561465 0 0) 0x7fca3806d130 con 0x7fca3815d940 2024-03-29T08:27:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.255+0000 7fca37e00640 1 -- 172.21.15.46:0/3013296613 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca20003530 con 0x7fca3815d940 2024-03-29T08:27:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.255+0000 7fca37e00640 1 -- 172.21.15.46:0/3013296613 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2224024903 0 0) 0x7fca3815b8f0 con 0x7fca381593d0 2024-03-29T08:27:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.255+0000 7fca37e00640 1 -- 172.21.15.46:0/3013296613 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca3806d130 con 0x7fca381593d0 2024-03-29T08:27:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.255+0000 7fca3ce00640 1 -- 172.21.15.46:0/3013296613 >> v1:172.21.15.73:6789/0 conn(0x7fca381593d0 legacy=0x7fca3815d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.255+0000 7fca3ce00640 1 -- 172.21.15.46:0/3013296613 >> v1:172.21.15.46:6789/0 conn(0x7fca3815d940 legacy=0x7fca381597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:31.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.255+0000 7fca3ce00640 1 -- 172.21.15.46:0/3013296613 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3815afc0 con 0x7fca3815cd90 2024-03-29T08:27:31.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.259+0000 7fca3ce00640 1 --2- 172.21.15.46:0/3013296613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca3815cd90 0x7fca3806cbf0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fca240097c0 tx=0x7fca2401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3513ec02898839f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:31.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.259+0000 7fca37e00640 1 -- 172.21.15.46:0/3013296613 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca240a6040 con 0x7fca3815cd90 2024-03-29T08:27:31.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.259+0000 7fca37e00640 1 -- 172.21.15.46:0/3013296613 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca240aa600 con 0x7fca3815cd90 2024-03-29T08:27:31.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.259+0000 7fca37e00640 1 -- 172.21.15.46:0/3013296613 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca240aa990 con 0x7fca3815cd90 2024-03-29T08:27:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.259+0000 7fca40200640 1 -- 172.21.15.46:0/3013296613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca3815cd90 msgr2=0x7fca3806cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.259+0000 7fca40200640 1 --2- 172.21.15.46:0/3013296613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca3815cd90 0x7fca3806cbf0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fca240097c0 tx=0x7fca2401c4a0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.259+0000 7fca40200640 1 -- 172.21.15.46:0/3013296613 shutdown_connections 2024-03-29T08:27:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.259+0000 7fca40200640 1 --2- 172.21.15.46:0/3013296613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca3815cd90 0x7fca3806cbf0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.259+0000 7fca40200640 1 -- 172.21.15.46:0/3013296613 >> 172.21.15.46:0/3013296613 conn(0x7fca38066110 msgr2=0x7fca38069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:31.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.259+0000 7fca40200640 1 -- 172.21.15.46:0/3013296613 shutdown_connections 2024-03-29T08:27:31.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.259+0000 7fca40200640 1 -- 172.21.15.46:0/3013296613 wait complete. 2024-03-29T08:27:31.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.259+0000 7fca40200640 1 Processor -- start 2024-03-29T08:27:31.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.259+0000 7fca40200640 1 -- start start 2024-03-29T08:27:31.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca40200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca381593d0 0x7fca3814f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:31.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca40200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca3815cd90 0x7fca3814e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:31.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca40200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca3815d940 0x7fca38155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:31.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca40200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca38154000 con 0x7fca3815d940 2024-03-29T08:27:31.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca40200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3814fd30 con 0x7fca381593d0 2024-03-29T08:27:31.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca40200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca38153380 con 0x7fca3815cd90 2024-03-29T08:27:31.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca3e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca3815d940 0x7fca38155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:31.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca3e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca3815d940 0x7fca38155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36034/0 (socket says 172.21.15.46:36034) 2024-03-29T08:27:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca3e200640 1 -- 172.21.15.46:0/2471789727 learned_addr learned my addr 172.21.15.46:0/2471789727 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca3d800640 1 --2- 172.21.15.46:0/2471789727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca381593d0 0x7fca3814f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca3ce00640 1 --2- 172.21.15.46:0/2471789727 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca3815cd90 0x7fca3814e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca3e200640 1 -- 172.21.15.46:0/2471789727 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca3815cd90 msgr2=0x7fca3814e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca3e200640 1 --2- 172.21.15.46:0/2471789727 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca3815cd90 0x7fca3814e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca3e200640 1 -- 172.21.15.46:0/2471789727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca381593d0 msgr2=0x7fca3814f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca3e200640 1 --2- 172.21.15.46:0/2471789727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca381593d0 0x7fca3814f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca3e200640 1 -- 172.21.15.46:0/2471789727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca24009420 con 0x7fca3815d940 2024-03-29T08:27:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca3ce00640 1 --2- 172.21.15.46:0/2471789727 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca3815cd90 0x7fca3814e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca3e200640 1 --2- 172.21.15.46:0/2471789727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca3815d940 0x7fca38155d10 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fca3000caf0 tx=0x7fca3000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca35600640 1 -- 172.21.15.46:0/2471789727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3000ac40 con 0x7fca3815d940 2024-03-29T08:27:31.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca35600640 1 -- 172.21.15.46:0/2471789727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3000ada0 con 0x7fca3815d940 2024-03-29T08:27:31.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca40200640 1 -- 172.21.15.46:0/2471789727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca38153660 con 0x7fca3815d940 2024-03-29T08:27:31.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca35600640 1 -- 172.21.15.46:0/2471789727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30012620 con 0x7fca3815d940 2024-03-29T08:27:31.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca40200640 1 -- 172.21.15.46:0/2471789727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca38153c20 con 0x7fca3815d940 2024-03-29T08:27:31.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca40200640 1 -- 172.21.15.46:0/2471789727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca38154140 con 0x7fca3815d940 2024-03-29T08:27:31.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.263+0000 7fca35600640 1 -- 172.21.15.46:0/2471789727 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fca30007070 con 0x7fca3815d940 2024-03-29T08:27:31.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.267+0000 7fca35600640 1 --2- 172.21.15.46:0/2471789727 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca18066690 0x7fca18068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:31.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.267+0000 7fca35600640 1 -- 172.21.15.46:0/2471789727 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca30084040 con 0x7fca3815d940 2024-03-29T08:27:31.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.267+0000 7fca3d800640 1 --2- 172.21.15.46:0/2471789727 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca18066690 0x7fca18068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:31.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.267+0000 7fca3d800640 1 --2- 172.21.15.46:0/2471789727 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca18066690 0x7fca18068b50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0011f0 tx=0x7fca2c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:31.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.271+0000 7fca35600640 1 -- 172.21.15.46:0/2471789727 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fca38154140 con 0x7fca3815d940 2024-03-29T08:27:31.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.467+0000 7fca40200640 1 -- 172.21.15.46:0/2471789727 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fca38154140 con 0x7fca3815d940 2024-03-29T08:27:31.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.467+0000 7fca35600640 1 -- 172.21.15.46:0/2471789727 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v77) v1 ==== 53+0+73 (secure 0 0 0) 0x7fca38154140 con 0x7fca3815d940 2024-03-29T08:27:31.470 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:27:31.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.471+0000 7fca40200640 1 -- 172.21.15.46:0/2471789727 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca18066690 msgr2=0x7fca18068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:31.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.471+0000 7fca40200640 1 --2- 172.21.15.46:0/2471789727 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca18066690 0x7fca18068b50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0011f0 tx=0x7fca2c009040 comp rx=0 tx=0).stop 2024-03-29T08:27:31.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.471+0000 7fca40200640 1 -- 172.21.15.46:0/2471789727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca3815d940 msgr2=0x7fca38155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:31.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.471+0000 7fca40200640 1 --2- 172.21.15.46:0/2471789727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca3815d940 0x7fca38155d10 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fca3000caf0 tx=0x7fca3000a060 comp rx=0 tx=0).stop 2024-03-29T08:27:31.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.471+0000 7fca40200640 1 -- 172.21.15.46:0/2471789727 shutdown_connections 2024-03-29T08:27:31.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.471+0000 7fca40200640 1 --2- 172.21.15.46:0/2471789727 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fca18066690 0x7fca18068b50 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.471+0000 7fca40200640 1 --2- 172.21.15.46:0/2471789727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca3815d940 0x7fca38155d10 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.471+0000 7fca40200640 1 --2- 172.21.15.46:0/2471789727 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fca3815cd90 0x7fca3814e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.471+0000 7fca40200640 1 --2- 172.21.15.46:0/2471789727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fca381593d0 0x7fca3814f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:31.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.471+0000 7fca40200640 1 -- 172.21.15.46:0/2471789727 >> 172.21.15.46:0/2471789727 conn(0x7fca38066110 msgr2=0x7fca380b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:31.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.471+0000 7fca40200640 1 -- 172.21.15.46:0/2471789727 shutdown_connections 2024-03-29T08:27:31.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:31.471+0000 7fca40200640 1 -- 172.21.15.46:0/2471789727 wait complete. 2024-03-29T08:27:31.491 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:27:31.491 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:27:31.491 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-29T08:27:31.491 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T08:27:31.491 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:27:31.491 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:27:31.492 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:27:31.492 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:27:31.492 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:27:31.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:31.492 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-29T08:27:31.502 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:27:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:27:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:27:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:27:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:27:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:27:31.502 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:27:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:27:31.502 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-29T08:27:31.502 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:27:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:27:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-29T08:27:31.503 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:27:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:27:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-29T08:27:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T08:27:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41886sec preferred_lft 41886sec 2024-03-29T08:27:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-29T08:27:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:27:31.503 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:27:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:27:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-29T08:27:31.503 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:27:31.503 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:d9:58:09:4c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:27:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:27:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:27:31.504 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:27:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T08:27:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T08:27:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:27:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T08:27:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:27:31.504 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:27:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:27:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T08:27:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:27:31.504 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:27:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T08:27:31.504 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-29T08:27:31.505 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:27:31.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:31.505 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:27:31.565 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:27:31.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:31.567 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-29T08:27:31.617 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-29T08:27:31.617 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:27:31.617 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T08:27:31.617 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:27:31.617 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T08:27:31.617 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:27:31.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:31.618 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:27:31.618 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:27:31.618 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:27:31.618 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:27:31.618 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:27:31.618 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:27:31.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:31.715 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:27:31.724 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T08:27:31.724 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T08:27:31.725 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:27:31.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:31.725 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-03-29T08:27:31.787 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T08:27:31.787 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:27:31.788 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T08:27:31.839 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:27:31.840 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:27:31.840 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:27:31.894 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:27:31.956 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:27:31.956 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:27:32.018 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:27:32.019 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:27:32.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:32.020 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:27:32.071 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:27:32.072 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T08:27:32.118 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:27:32.138 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:27:32.138 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:27:32.197 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:27:32.199 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:27:32.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:32.200 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:27:32.213 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-03-29T08:27:32.211+0000 7fd8f7b8c580 -1 init, newargv = 0x5589e607bdd0 newargc=15 2024-03-29T08:27:32.213 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[48553]: starting ceph client 2024-03-29T08:27:32.219 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[48553]: starting fuse 2024-03-29T08:27:32.222 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:27:32.222 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-29T08:27:32.222 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:27:32.222 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-03-29T08:27:32.222 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-03-29T08:27:32.223 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:27:32.223 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-03-29T08:27:32.223 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:27:32.223 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-03-29T08:27:32.223 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T08:27:32.223 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-03-29T08:27:32.223 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:27:32.223 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:27:32.223 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-03-29T08:27:32.223 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-03-29T08:27:32.223 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-03-29T08:27:32.223 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-03-29T08:27:32.223 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:27:32.223 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-03-29T08:27:32.223 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:27:32.223 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:27:32.224 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:27:32.224 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-03-29T08:27:32.224 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-03-29T08:27:32.224 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-03-29T08:27:32.224 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:27:32.224 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:27:32.224 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-03-29T08:27:32.224 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:27:32.348 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.48553.asok 2024-03-29T08:27:32.534 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48553.asok 2024-03-29T08:27:32.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:32.534 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48553.asok status 2024-03-29T08:27:32.707 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:27:32.707 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-03-29T08:27:32.707 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:27:32.707 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:27:32.707 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-03-29T08:27:32.707 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-03-29T08:27:32.708 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T08:27:32.708 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "48553", 2024-03-29T08:27:32.708 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-03-29T08:27:32.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:32.708 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-03-29T08:27:32.708 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-03-29T08:27:32.708 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 8595, 2024-03-29T08:27:32.708 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-03-29T08:27:32.708 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-03-29T08:27:32.708 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-03-29T08:27:32.708 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 8595 2024-03-29T08:27:32.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:32.708 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:27:32.708 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:27:32.709 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:27:32.709 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1015089811 2024-03-29T08:27:32.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:32.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:32.709 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:27:32.709 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:27:32.709 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:27:32.709 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1015089811 2024-03-29T08:27:32.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:27:32.709 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.8595 192.168.0.1:0/1015089811", 2024-03-29T08:27:32.709 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/1015089811", 2024-03-29T08:27:32.709 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-03-29T08:27:32.709 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 77, 2024-03-29T08:27:32.710 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 106, 2024-03-29T08:27:32.710 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:27:32.710 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-03-29T08:27:32.710 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-03-29T08:27:32.710 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:27:32.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:32.722 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-03-29T08:27:32.722 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:27:32.735 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-03-29T08:27:32.735 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T08:27:32.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:32.735 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:27:32.795 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:27:32.795 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:27:32.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:32.848 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-03-29T08:27:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records in 2024-03-29T08:27:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records out 2024-03-29T08:27:32.990 INFO:teuthology.orchestra.run.smithi046.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0854031 s, 73.7 MB/s 2024-03-29T08:27:32.991 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:27:32.991 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:27:33.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:33.000 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-03-29T08:27:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records in 2024-03-29T08:27:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:6+0 records out 2024-03-29T08:27:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0726042 s, 86.7 MB/s 2024-03-29T08:27:33.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:33.128 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-29T08:27:33.141 INFO:teuthology.orchestra.run.smithi046.stdout:./ 2024-03-29T08:27:33.141 INFO:teuthology.orchestra.run.smithi046.stdout:./file2_sixmegs 2024-03-29T08:27:33.141 INFO:teuthology.orchestra.run.smithi046.stdout:./file3_sixmegs 2024-03-29T08:27:33.141 INFO:teuthology.orchestra.run.smithi046.stdout:./file1_sixmegs 2024-03-29T08:27:33.145 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:27:33.145 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:27:33.145 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:27:33.145 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:27:33.146 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-29T08:27:33.146 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:27:33.247 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:27:33.437 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:27:33.437 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:27:33.437 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:27:33.437 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:27:33.437 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-03-29T08:27:33.437 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:27:33.546 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627777 2024-03-29T08:27:33.735 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:27:33.735 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:27:33.735 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:27:33.735 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:27:33.735 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-03-29T08:27:33.735 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:27:33.844 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627778 2024-03-29T08:27:34.033 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:27:34.034 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:27:34.034 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:27:34.034 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:27:34.034 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-03-29T08:27:34.034 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:27:34.144 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627776 2024-03-29T08:27:34.333 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:27:34.334 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:27:34.388 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:27:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.567+0000 7f236da00640 1 Processor -- start 2024-03-29T08:27:34.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.567+0000 7f236da00640 1 -- start start 2024-03-29T08:27:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.571+0000 7f236da00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f236815ada0 0x7f23680ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:34.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.571+0000 7f236da00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f23680ce950 0x7f236806c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.571+0000 7f236da00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23680cb510 con 0x7f2368144210 2024-03-29T08:27:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.571+0000 7f236da00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23680cede0 con 0x7f23680ce950 2024-03-29T08:27:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.571+0000 7f236da00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2368070580 con 0x7f236815ada0 2024-03-29T08:27:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.571+0000 7f2367400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f23680ce950 0x7f236806c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.571+0000 7f2367400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f23680ce950 0x7f236806c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36036/0 (socket says 172.21.15.46:36036) 2024-03-29T08:27:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.571+0000 7f2367400640 1 -- 172.21.15.46:0/2988928446 learned_addr learned my addr 172.21.15.46:0/2988928446 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.571+0000 7f235e000640 1 --2- 172.21.15.46:0/2988928446 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f236815ada0 0x7f23680ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.571+0000 7f2367400640 1 -- 172.21.15.46:0/2988928446 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f236815ada0 msgr2=0x7f23680ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:34.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.571+0000 7f2367400640 1 --2- 172.21.15.46:0/2988928446 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f236815ada0 0x7f23680ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:34.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.571+0000 7f2367400640 1 -- 172.21.15.46:0/2988928446 >> v1:172.21.15.73:6789/0 conn(0x7f2368144210 legacy=0x7f23680cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:34.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.571+0000 7f2367400640 1 -- 172.21.15.46:0/2988928446 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2368070860 con 0x7f23680ce950 2024-03-29T08:27:34.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f2367400640 1 --2- 172.21.15.46:0/2988928446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23680ce950 0x7f236806c950 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2360009e50 tx=0x7f236001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772b392190bc6aaa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:34.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f2366000640 1 -- 172.21.15.46:0/2988928446 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600a5760 con 0x7f23680ce950 2024-03-29T08:27:34.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f2366000640 1 -- 172.21.15.46:0/2988928446 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23600a58c0 con 0x7f23680ce950 2024-03-29T08:27:34.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f2366000640 1 -- 172.21.15.46:0/2988928446 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23600a5c50 con 0x7f23680ce950 2024-03-29T08:27:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f236da00640 1 -- 172.21.15.46:0/2988928446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23680ce950 msgr2=0x7f236806c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f236da00640 1 --2- 172.21.15.46:0/2988928446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23680ce950 0x7f236806c950 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2360009e50 tx=0x7f236001c9f0 comp rx=0 tx=0).stop 2024-03-29T08:27:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f236da00640 1 -- 172.21.15.46:0/2988928446 shutdown_connections 2024-03-29T08:27:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f236da00640 1 --2- 172.21.15.46:0/2988928446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23680ce950 0x7f236806c950 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f236da00640 1 --2- 172.21.15.46:0/2988928446 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f236815ada0 0x7f23680ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f236da00640 1 -- 172.21.15.46:0/2988928446 >> 172.21.15.46:0/2988928446 conn(0x7f2368066110 msgr2=0x7f2368069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f236da00640 1 -- 172.21.15.46:0/2988928446 shutdown_connections 2024-03-29T08:27:34.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f236da00640 1 -- 172.21.15.46:0/2988928446 wait complete. 2024-03-29T08:27:34.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f236da00640 1 Processor -- start 2024-03-29T08:27:34.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f236da00640 1 -- start start 2024-03-29T08:27:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f236da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23680ce950 0x7f236813d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f236da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2368144210 0x7f23680b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f236da00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f236815ada0 0x7f23680b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f236da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23680b2ad0 con 0x7f2368144210 2024-03-29T08:27:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f236da00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236813da30 con 0x7f23680ce950 2024-03-29T08:27:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f236da00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23680b1ec0 con 0x7f236815ada0 2024-03-29T08:27:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.575+0000 7f235e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2368144210 0x7f23680b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.579+0000 7f235e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2368144210 0x7f23680b2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36044/0 (socket says 172.21.15.46:36044) 2024-03-29T08:27:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.579+0000 7f235e000640 1 -- 172.21.15.46:0/2557468363 learned_addr learned my addr 172.21.15.46:0/2557468363 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.579+0000 7f2366a00640 1 --2- 172.21.15.46:0/2557468363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23680ce950 0x7f236813d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:34.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.579+0000 7f2367400640 1 --2- 172.21.15.46:0/2557468363 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f236815ada0 0x7f23680b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.579+0000 7f235e000640 1 -- 172.21.15.46:0/2557468363 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f236815ada0 msgr2=0x7f23680b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.579+0000 7f235e000640 1 --2- 172.21.15.46:0/2557468363 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f236815ada0 0x7f23680b1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.579+0000 7f235e000640 1 -- 172.21.15.46:0/2557468363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23680ce950 msgr2=0x7f236813d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.579+0000 7f235e000640 1 --2- 172.21.15.46:0/2557468363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23680ce950 0x7f236813d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.579+0000 7f235e000640 1 -- 172.21.15.46:0/2557468363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2360009b30 con 0x7f2368144210 2024-03-29T08:27:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.579+0000 7f235e000640 1 --2- 172.21.15.46:0/2557468363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2368144210 0x7f23680b2f60 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f235000d800 tx=0x7f235000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.579+0000 7f235f400640 1 -- 172.21.15.46:0/2557468363 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235000b6c0 con 0x7f2368144210 2024-03-29T08:27:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.579+0000 7f235f400640 1 -- 172.21.15.46:0/2557468363 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235000b820 con 0x7f2368144210 2024-03-29T08:27:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.579+0000 7f235f400640 1 -- 172.21.15.46:0/2557468363 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235000bb90 con 0x7f2368144210 2024-03-29T08:27:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.579+0000 7f236da00640 1 -- 172.21.15.46:0/2557468363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23680b21a0 con 0x7f2368144210 2024-03-29T08:27:34.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.579+0000 7f236da00640 1 -- 172.21.15.46:0/2557468363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23680b2760 con 0x7f2368144210 2024-03-29T08:27:34.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.579+0000 7f236da00640 1 -- 172.21.15.46:0/2557468363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23680b2920 con 0x7f2368144210 2024-03-29T08:27:34.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.579+0000 7f235f400640 1 -- 172.21.15.46:0/2557468363 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f235000bcf0 con 0x7f2368144210 2024-03-29T08:27:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.583+0000 7f235f400640 1 --2- 172.21.15.46:0/2557468363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2344066690 0x7f2344068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.583+0000 7f235f400640 1 -- 172.21.15.46:0/2557468363 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23500846e0 con 0x7f2368144210 2024-03-29T08:27:34.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.583+0000 7f2366a00640 1 --2- 172.21.15.46:0/2557468363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2344066690 0x7f2344068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:34.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.583+0000 7f2366a00640 1 --2- 172.21.15.46:0/2557468363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2344066690 0x7f2344068b50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f23580098e0 tx=0x7f2358009060 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:34.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.587+0000 7f235f400640 1 -- 172.21.15.46:0/2557468363 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f23500519d0 con 0x7f2368144210 2024-03-29T08:27:34.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.751+0000 7f236da00640 1 -- 172.21.15.46:0/2557468363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23680ce280 con 0x7f2368144210 2024-03-29T08:27:34.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.755+0000 7f235f400640 1 -- 172.21.15.46:0/2557468363 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16003 (secure 0 0 0) 0x7f2350054ff0 con 0x7f2368144210 2024-03-29T08:27:34.756 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:34.756 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":106,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:27:19.585517+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:27:17.118875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-29T08:27:17.573928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:27:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.755+0000 7f236da00640 1 -- 172.21.15.46:0/2557468363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2344066690 msgr2=0x7f2344068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.755+0000 7f236da00640 1 --2- 172.21.15.46:0/2557468363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2344066690 0x7f2344068b50 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f23580098e0 tx=0x7f2358009060 comp rx=0 tx=0).stop 2024-03-29T08:27:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.755+0000 7f236da00640 1 -- 172.21.15.46:0/2557468363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2368144210 msgr2=0x7f23680b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.755+0000 7f236da00640 1 --2- 172.21.15.46:0/2557468363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2368144210 0x7f23680b2f60 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f235000d800 tx=0x7f235000dcc0 comp rx=0 tx=0).stop 2024-03-29T08:27:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.755+0000 7f236da00640 1 -- 172.21.15.46:0/2557468363 shutdown_connections 2024-03-29T08:27:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.755+0000 7f236da00640 1 --2- 172.21.15.46:0/2557468363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2344066690 0x7f2344068b50 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.755+0000 7f236da00640 1 --2- 172.21.15.46:0/2557468363 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f236815ada0 0x7f23680b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.755+0000 7f236da00640 1 --2- 172.21.15.46:0/2557468363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2368144210 0x7f23680b2f60 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:34.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.755+0000 7f236da00640 1 --2- 172.21.15.46:0/2557468363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23680ce950 0x7f236813d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:34.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.755+0000 7f236da00640 1 -- 172.21.15.46:0/2557468363 >> 172.21.15.46:0/2557468363 conn(0x7f2368066110 msgr2=0x7f23680af570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:34.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.755+0000 7f236da00640 1 -- 172.21.15.46:0/2557468363 shutdown_connections 2024-03-29T08:27:34.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.755+0000 7f236da00640 1 -- 172.21.15.46:0/2557468363 wait complete. 2024-03-29T08:27:34.779 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:27:17.118875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:27:17.573928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:27:34.780 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:27:34.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.919+0000 7fc5a1c00640 1 Processor -- start 2024-03-29T08:27:34.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.923+0000 7fc5a1c00640 1 -- start start 2024-03-29T08:27:34.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.923+0000 7fc5a1c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc59c0cbea0 0x7fc59c0b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:34.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.923+0000 7fc5a1c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc59c15efe0 0x7fc59c0b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:34.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.923+0000 7fc5a1c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc59c0b20b0 con 0x7fc59c143eb0 2024-03-29T08:27:34.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.923+0000 7fc5a1c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59c0b2360 con 0x7fc59c0cbea0 2024-03-29T08:27:34.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.923+0000 7fc5a1c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c15e500 con 0x7fc59c15efe0 2024-03-29T08:27:34.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.923+0000 7fc592000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc59c0cbea0 0x7fc59c0b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:34.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.923+0000 7fc592000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc59c0cbea0 0x7fc59c0b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36050/0 (socket says 172.21.15.46:36050) 2024-03-29T08:27:34.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.923+0000 7fc592000640 1 -- 172.21.15.46:0/1939936345 learned_addr learned my addr 172.21.15.46:0/1939936345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:34.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.923+0000 7fc59b400640 1 --2- 172.21.15.46:0/1939936345 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc59c15efe0 0x7fc59c0b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:34.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.923+0000 7fc592000640 1 -- 172.21.15.46:0/1939936345 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc59c15efe0 msgr2=0x7fc59c0b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:34.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.923+0000 7fc592000640 1 --2- 172.21.15.46:0/1939936345 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc59c15efe0 0x7fc59c0b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:34.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.923+0000 7fc592000640 1 -- 172.21.15.46:0/1939936345 >> v1:172.21.15.73:6789/0 conn(0x7fc59c143eb0 legacy=0x7fc59c0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:34.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.923+0000 7fc592000640 1 -- 172.21.15.46:0/1939936345 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c15e7e0 con 0x7fc59c0cbea0 2024-03-29T08:27:34.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.927+0000 7fc592000640 1 --2- 172.21.15.46:0/1939936345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59c0cbea0 0x7fc59c0b1630 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fc584009870 tx=0x7fc58401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd48163a9a4a5a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:34.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.927+0000 7fc59a000640 1 -- 172.21.15.46:0/1939936345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5840aa430 con 0x7fc59c0cbea0 2024-03-29T08:27:34.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.927+0000 7fc59a000640 1 -- 172.21.15.46:0/1939936345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5840aa590 con 0x7fc59c0cbea0 2024-03-29T08:27:34.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.927+0000 7fc59a000640 1 -- 172.21.15.46:0/1939936345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5840aa920 con 0x7fc59c0cbea0 2024-03-29T08:27:34.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.927+0000 7fc5a1c00640 1 -- 172.21.15.46:0/1939936345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59c0cbea0 msgr2=0x7fc59c0b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:34.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.927+0000 7fc5a1c00640 1 --2- 172.21.15.46:0/1939936345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59c0cbea0 0x7fc59c0b1630 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fc584009870 tx=0x7fc58401c530 comp rx=0 tx=0).stop 2024-03-29T08:27:34.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.927+0000 7fc5a1c00640 1 -- 172.21.15.46:0/1939936345 shutdown_connections 2024-03-29T08:27:34.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.927+0000 7fc5a1c00640 1 --2- 172.21.15.46:0/1939936345 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc59c15efe0 0x7fc59c0b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:34.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.927+0000 7fc5a1c00640 1 --2- 172.21.15.46:0/1939936345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59c0cbea0 0x7fc59c0b1630 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:34.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.927+0000 7fc5a1c00640 1 -- 172.21.15.46:0/1939936345 >> 172.21.15.46:0/1939936345 conn(0x7fc59c06a390 msgr2=0x7fc59c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:34.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.927+0000 7fc5a1c00640 1 -- 172.21.15.46:0/1939936345 shutdown_connections 2024-03-29T08:27:34.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.927+0000 7fc5a1c00640 1 -- 172.21.15.46:0/1939936345 wait complete. 2024-03-29T08:27:34.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.927+0000 7fc5a1c00640 1 Processor -- start 2024-03-29T08:27:34.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.927+0000 7fc5a1c00640 1 -- start start 2024-03-29T08:27:34.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc5a1c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc59c0cbea0 0x7fc59c140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:34.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc5a1c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc59c143eb0 0x7fc59c13d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:34.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc5a1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59c15efe0 0x7fc59c1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:34.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc5a1c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c13c770 con 0x7fc59c15efe0 2024-03-29T08:27:34.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc5a1c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c13ca50 con 0x7fc59c0cbea0 2024-03-29T08:27:34.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc5a1c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c13cd30 con 0x7fc59c143eb0 2024-03-29T08:27:34.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc59b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59c15efe0 0x7fc59c1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:34.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc59b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59c15efe0 0x7fc59c1599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36058/0 (socket says 172.21.15.46:36058) 2024-03-29T08:27:34.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc59b400640 1 -- 172.21.15.46:0/1466321392 learned_addr learned my addr 172.21.15.46:0/1466321392 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:34.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc592000640 1 --2- 172.21.15.46:0/1466321392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc59c143eb0 0x7fc59c13d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:34.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc59aa00640 1 --2- 172.21.15.46:0/1466321392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc59c0cbea0 0x7fc59c140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:34.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc59b400640 1 -- 172.21.15.46:0/1466321392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc59c143eb0 msgr2=0x7fc59c13d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:34.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc59b400640 1 --2- 172.21.15.46:0/1466321392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc59c143eb0 0x7fc59c13d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:34.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc59b400640 1 -- 172.21.15.46:0/1466321392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc59c0cbea0 msgr2=0x7fc59c140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc59b400640 1 --2- 172.21.15.46:0/1466321392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc59c0cbea0 0x7fc59c140fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc59b400640 1 -- 172.21.15.46:0/1466321392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5840094d0 con 0x7fc59c15efe0 2024-03-29T08:27:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc592000640 1 --2- 172.21.15.46:0/1466321392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc59c143eb0 0x7fc59c13d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc59aa00640 1 --2- 172.21.15.46:0/1466321392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc59c0cbea0 0x7fc59c140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc59b400640 1 --2- 172.21.15.46:0/1466321392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59c15efe0 0x7fc59c1599b0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fc594006e90 tx=0x7fc59400d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc593400640 1 -- 172.21.15.46:0/1466321392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5940125f0 con 0x7fc59c15efe0 2024-03-29T08:27:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc593400640 1 -- 172.21.15.46:0/1466321392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc594012750 con 0x7fc59c15efe0 2024-03-29T08:27:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc593400640 1 -- 172.21.15.46:0/1466321392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc594012b00 con 0x7fc59c15efe0 2024-03-29T08:27:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc5a1c00640 1 -- 172.21.15.46:0/1466321392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc59c13d010 con 0x7fc59c15efe0 2024-03-29T08:27:34.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc5a1c00640 1 -- 172.21.15.46:0/1466321392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc59c159ef0 con 0x7fc59c15efe0 2024-03-29T08:27:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc5a1c00640 1 -- 172.21.15.46:0/1466321392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59c071a70 con 0x7fc59c15efe0 2024-03-29T08:27:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.931+0000 7fc593400640 1 -- 172.21.15.46:0/1466321392 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc594012c60 con 0x7fc59c15efe0 2024-03-29T08:27:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.935+0000 7fc593400640 1 --2- 172.21.15.46:0/1466321392 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc578066690 0x7fc578068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:34.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.935+0000 7fc593400640 1 -- 172.21.15.46:0/1466321392 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc594084660 con 0x7fc59c15efe0 2024-03-29T08:27:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.935+0000 7fc59aa00640 1 --2- 172.21.15.46:0/1466321392 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc578066690 0x7fc578068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:34.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.935+0000 7fc59aa00640 1 --2- 172.21.15.46:0/1466321392 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc578066690 0x7fc578068b50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc58c007fd0 tx=0x7fc58c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:34.939+0000 7fc593400640 1 -- 172.21.15.46:0/1466321392 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc5940519c0 con 0x7fc59c15efe0 2024-03-29T08:27:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.099+0000 7fc5a1c00640 1 -- 172.21.15.46:0/1466321392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc59c13c8b0 con 0x7fc59c15efe0 2024-03-29T08:27:35.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.099+0000 7fc593400640 1 -- 172.21.15.46:0/1466321392 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc59c13c8b0 con 0x7fc59c15efe0 2024-03-29T08:27:35.101 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:27:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.099+0000 7fc5a1c00640 1 -- 172.21.15.46:0/1466321392 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc578066690 msgr2=0x7fc578068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.099+0000 7fc5a1c00640 1 --2- 172.21.15.46:0/1466321392 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc578066690 0x7fc578068b50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fc58c007fd0 tx=0x7fc58c009040 comp rx=0 tx=0).stop 2024-03-29T08:27:35.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.099+0000 7fc5a1c00640 1 -- 172.21.15.46:0/1466321392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59c15efe0 msgr2=0x7fc59c1599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.099+0000 7fc5a1c00640 1 --2- 172.21.15.46:0/1466321392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59c15efe0 0x7fc59c1599b0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fc594006e90 tx=0x7fc59400d9e0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.099+0000 7fc5a1c00640 1 -- 172.21.15.46:0/1466321392 shutdown_connections 2024-03-29T08:27:35.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.099+0000 7fc5a1c00640 1 --2- 172.21.15.46:0/1466321392 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc578066690 0x7fc578068b50 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.099+0000 7fc5a1c00640 1 --2- 172.21.15.46:0/1466321392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc59c15efe0 0x7fc59c1599b0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.099+0000 7fc5a1c00640 1 --2- 172.21.15.46:0/1466321392 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc59c143eb0 0x7fc59c13d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.099+0000 7fc5a1c00640 1 --2- 172.21.15.46:0/1466321392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc59c0cbea0 0x7fc59c140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.099+0000 7fc5a1c00640 1 -- 172.21.15.46:0/1466321392 >> 172.21.15.46:0/1466321392 conn(0x7fc59c06a390 msgr2=0x7fc59c158990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:35.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.099+0000 7fc5a1c00640 1 -- 172.21.15.46:0/1466321392 shutdown_connections 2024-03-29T08:27:35.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.099+0000 7fc5a1c00640 1 -- 172.21.15.46:0/1466321392 wait complete. 2024-03-29T08:27:35.122 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:27:35.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.263+0000 7fb5de600640 1 Processor -- start 2024-03-29T08:27:35.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.263+0000 7fb5de600640 1 -- start start 2024-03-29T08:27:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.263+0000 7fb5de600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb5d815df90 0x7fb5d80ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.263+0000 7fb5de600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d80cbea0 0x7fb5d80ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.263+0000 7fb5de600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d815d260 con 0x7fb5d8142a50 2024-03-29T08:27:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.263+0000 7fb5de600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d80aed00 con 0x7fb5d815df90 2024-03-29T08:27:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.263+0000 7fb5de600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d80aefd0 con 0x7fb5d80cbea0 2024-03-29T08:27:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.263+0000 7fb5d7400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb5d8142a50 0x7fb5d80cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59638/0 (socket says 172.21.15.46:59638) 2024-03-29T08:27:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.263+0000 7fb5d7400640 1 -- 172.21.15.46:0/1504764626 learned_addr learned my addr 172.21.15.46:0/1504764626 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.263+0000 7fb5cea00640 1 --2- 172.21.15.46:0/1504764626 >> v2:172.21.15.46:3300/0 conn(0x7fb5d815df90 0x7fb5d80ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.263+0000 7fb5d7e00640 1 --2- 172.21.15.46:0/1504764626 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d80cbea0 0x7fb5d80ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.263+0000 7fb5cea00640 1 -- 172.21.15.46:0/1504764626 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d80cbea0 msgr2=0x7fb5d80ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.263+0000 7fb5cea00640 1 --2- 172.21.15.46:0/1504764626 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d80cbea0 0x7fb5d80ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.263+0000 7fb5cea00640 1 -- 172.21.15.46:0/1504764626 >> v1:172.21.15.46:6789/0 conn(0x7fb5d8142a50 legacy=0x7fb5d80cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.263+0000 7fb5cea00640 1 -- 172.21.15.46:0/1504764626 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d815d4b0 con 0x7fb5d815df90 2024-03-29T08:27:35.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.267+0000 7fb5cea00640 1 --2- 172.21.15.46:0/1504764626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d815df90 0x7fb5d80ae280 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009d90 tx=0x7fb5c001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d77d50d56c6d89 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:35.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.267+0000 7fb5d6a00640 1 -- 172.21.15.46:0/1504764626 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00a5780 con 0x7fb5d815df90 2024-03-29T08:27:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.267+0000 7fb5d6a00640 1 -- 172.21.15.46:0/1504764626 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c00a58e0 con 0x7fb5d815df90 2024-03-29T08:27:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.267+0000 7fb5d6a00640 1 -- 172.21.15.46:0/1504764626 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00a5c70 con 0x7fb5d815df90 2024-03-29T08:27:35.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.267+0000 7fb5de600640 1 -- 172.21.15.46:0/1504764626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d815df90 msgr2=0x7fb5d80ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.267+0000 7fb5de600640 1 --2- 172.21.15.46:0/1504764626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d815df90 0x7fb5d80ae280 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009d90 tx=0x7fb5c001c8e0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.267+0000 7fb5de600640 1 -- 172.21.15.46:0/1504764626 shutdown_connections 2024-03-29T08:27:35.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.267+0000 7fb5de600640 1 --2- 172.21.15.46:0/1504764626 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d80cbea0 0x7fb5d80ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.267+0000 7fb5de600640 1 --2- 172.21.15.46:0/1504764626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d815df90 0x7fb5d80ae280 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.267+0000 7fb5de600640 1 -- 172.21.15.46:0/1504764626 >> 172.21.15.46:0/1504764626 conn(0x7fb5d806a390 msgr2=0x7fb5d80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:35.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.267+0000 7fb5de600640 1 -- 172.21.15.46:0/1504764626 shutdown_connections 2024-03-29T08:27:35.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.267+0000 7fb5de600640 1 -- 172.21.15.46:0/1504764626 wait complete. 2024-03-29T08:27:35.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5de600640 1 Processor -- start 2024-03-29T08:27:35.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5de600640 1 -- start start 2024-03-29T08:27:35.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5de600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d80cbea0 0x7fb5d8158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5de600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d8142a50 0x7fb5d81406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5de600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d815df90 0x7fb5d815b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5de600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d815a1a0 con 0x7fb5d80cbea0 2024-03-29T08:27:35.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5de600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d815a480 con 0x7fb5d8142a50 2024-03-29T08:27:35.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5de600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d815a760 con 0x7fb5d815df90 2024-03-29T08:27:35.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5d7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d80cbea0 0x7fb5d8158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5d7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d80cbea0 0x7fb5d8158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36062/0 (socket says 172.21.15.46:36062) 2024-03-29T08:27:35.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5d7400640 1 -- 172.21.15.46:0/1599360200 learned_addr learned my addr 172.21.15.46:0/1599360200 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:35.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5d7e00640 1 --2- 172.21.15.46:0/1599360200 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d815df90 0x7fb5d815b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5cea00640 1 --2- 172.21.15.46:0/1599360200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d8142a50 0x7fb5d81406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5d7400640 1 -- 172.21.15.46:0/1599360200 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d815df90 msgr2=0x7fb5d815b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5d7400640 1 --2- 172.21.15.46:0/1599360200 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d815df90 0x7fb5d815b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5d7400640 1 -- 172.21.15.46:0/1599360200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d8142a50 msgr2=0x7fb5d81406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5d7400640 1 --2- 172.21.15.46:0/1599360200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d8142a50 0x7fb5d81406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5d7400640 1 -- 172.21.15.46:0/1599360200 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c0009a70 con 0x7fb5d80cbea0 2024-03-29T08:27:35.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5d7400640 1 --2- 172.21.15.46:0/1599360200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d80cbea0 0x7fb5d8158a20 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800c500 tx=0x7fb5c800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:35.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5cfe00640 1 -- 172.21.15.46:0/1599360200 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8005cf0 con 0x7fb5d80cbea0 2024-03-29T08:27:35.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5cfe00640 1 -- 172.21.15.46:0/1599360200 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c8005e50 con 0x7fb5d80cbea0 2024-03-29T08:27:35.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5cfe00640 1 -- 172.21.15.46:0/1599360200 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c80072f0 con 0x7fb5d80cbea0 2024-03-29T08:27:35.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5de600640 1 -- 172.21.15.46:0/1599360200 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d815aa40 con 0x7fb5d80cbea0 2024-03-29T08:27:35.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5de600640 1 -- 172.21.15.46:0/1599360200 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d8158f60 con 0x7fb5d80cbea0 2024-03-29T08:27:35.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.271+0000 7fb5de600640 1 -- 172.21.15.46:0/1599360200 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d815af60 con 0x7fb5d80cbea0 2024-03-29T08:27:35.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.275+0000 7fb5cfe00640 1 -- 172.21.15.46:0/1599360200 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb5c8007450 con 0x7fb5d80cbea0 2024-03-29T08:27:35.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.275+0000 7fb5cfe00640 1 --2- 172.21.15.46:0/1599360200 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb5b4066690 0x7fb5b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.275+0000 7fb5cfe00640 1 -- 172.21.15.46:0/1599360200 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5c8083650 con 0x7fb5d80cbea0 2024-03-29T08:27:35.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.275+0000 7fb5cea00640 1 --2- 172.21.15.46:0/1599360200 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb5b4066690 0x7fb5b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.275+0000 7fb5cea00640 1 --2- 172.21.15.46:0/1599360200 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb5b4066690 0x7fb5b4068b50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00af000 tx=0x7fb5c00a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:35.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.279+0000 7fb5cfe00640 1 -- 172.21.15.46:0/1599360200 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb5c8050a60 con 0x7fb5d80cbea0 2024-03-29T08:27:35.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.439+0000 7fb5de600640 1 -- 172.21.15.46:0/1599360200 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb5d806ad20 con 0x7fb5d80cbea0 2024-03-29T08:27:35.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.439+0000 7fb5cfe00640 1 -- 172.21.15.46:0/1599360200 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb5c8054080 con 0x7fb5d80cbea0 2024-03-29T08:27:35.441 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:27:35.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.439+0000 7fb5de600640 1 -- 172.21.15.46:0/1599360200 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb5b4066690 msgr2=0x7fb5b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.439+0000 7fb5de600640 1 --2- 172.21.15.46:0/1599360200 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb5b4066690 0x7fb5b4068b50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00af000 tx=0x7fb5c00a7070 comp rx=0 tx=0).stop 2024-03-29T08:27:35.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.439+0000 7fb5de600640 1 -- 172.21.15.46:0/1599360200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d80cbea0 msgr2=0x7fb5d8158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.439+0000 7fb5de600640 1 --2- 172.21.15.46:0/1599360200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d80cbea0 0x7fb5d8158a20 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800c500 tx=0x7fb5c800c9c0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.439+0000 7fb5de600640 1 -- 172.21.15.46:0/1599360200 shutdown_connections 2024-03-29T08:27:35.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.439+0000 7fb5de600640 1 --2- 172.21.15.46:0/1599360200 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb5b4066690 0x7fb5b4068b50 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.439+0000 7fb5de600640 1 --2- 172.21.15.46:0/1599360200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d8142a50 0x7fb5d81406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.439+0000 7fb5de600640 1 --2- 172.21.15.46:0/1599360200 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5d815df90 0x7fb5d815b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.439+0000 7fb5de600640 1 --2- 172.21.15.46:0/1599360200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5d80cbea0 0x7fb5d8158a20 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.439+0000 7fb5de600640 1 -- 172.21.15.46:0/1599360200 >> 172.21.15.46:0/1599360200 conn(0x7fb5d806a390 msgr2=0x7fb5d80cb290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:35.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.439+0000 7fb5de600640 1 -- 172.21.15.46:0/1599360200 shutdown_connections 2024-03-29T08:27:35.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.439+0000 7fb5de600640 1 -- 172.21.15.46:0/1599360200 wait complete. 2024-03-29T08:27:35.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:27:35.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.603+0000 7fd0d1600640 1 Processor -- start 2024-03-29T08:27:35.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.603+0000 7fd0d1600640 1 -- start start 2024-03-29T08:27:35.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.603+0000 7fd0d1600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd0cc15a9c0 0x7fd0cc0cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.603+0000 7fd0d1600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0cc0ce290 con 0x7fd0cc0cc370 2024-03-29T08:27:35.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.603+0000 7fd0d1600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0cc0ce570 con 0x7fd0cc099150 2024-03-29T08:27:35.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.603+0000 7fd0d1600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0cc0ce850 con 0x7fd0cc15a9c0 2024-03-29T08:27:35.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.603+0000 7fd0c2000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd0cc0cc370 0x7fd0cc098a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59640/0 (socket says 172.21.15.46:59640) 2024-03-29T08:27:35.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.603+0000 7fd0c2000640 1 -- 172.21.15.46:0/1163703798 learned_addr learned my addr 172.21.15.46:0/1163703798 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:35.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.603+0000 7fd0caa00640 1 --2- 172.21.15.46:0/1163703798 >> v2:172.21.15.73:3300/0 conn(0x7fd0cc15a9c0 0x7fd0cc0cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.603+0000 7fd0ca000640 1 -- 172.21.15.46:0/1163703798 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983668982 0 0) 0x7fd0cc0ce290 con 0x7fd0cc0cc370 2024-03-29T08:27:35.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0ca000640 1 -- 172.21.15.46:0/1163703798 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0b0003530 con 0x7fd0cc0cc370 2024-03-29T08:27:35.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0ca000640 1 -- 172.21.15.46:0/1163703798 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010728621 0 0) 0x7fd0cc0ce570 con 0x7fd0cc099150 2024-03-29T08:27:35.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0ca000640 1 -- 172.21.15.46:0/1163703798 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0cc0ce290 con 0x7fd0cc099150 2024-03-29T08:27:35.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0ca000640 1 -- 172.21.15.46:0/1163703798 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1616551878 0 0) 0x7fd0b0003530 con 0x7fd0cc0cc370 2024-03-29T08:27:35.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0ca000640 1 -- 172.21.15.46:0/1163703798 >> v2:172.21.15.73:3300/0 conn(0x7fd0cc15a9c0 msgr2=0x7fd0cc0cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:35.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0ca000640 1 --2- 172.21.15.46:0/1163703798 >> v2:172.21.15.73:3300/0 conn(0x7fd0cc15a9c0 0x7fd0cc0cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0ca000640 1 -- 172.21.15.46:0/1163703798 >> v1:172.21.15.73:6789/0 conn(0x7fd0cc099150 legacy=0x7fd0cc0b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0ca000640 1 -- 172.21.15.46:0/1163703798 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc0ceb30 con 0x7fd0cc0cc370 2024-03-29T08:27:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0ca000640 1 -- 172.21.15.46:0/1163703798 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fd0b4001680 con 0x7fd0cc0cc370 2024-03-29T08:27:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0caa00640 1 --2- 172.21.15.46:0/1163703798 >> v2:172.21.15.73:3300/0 conn(0x7fd0cc15a9c0 0x7fd0cc0cbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:27:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0ca000640 1 -- 172.21.15.46:0/1163703798 >> v1:172.21.15.46:6789/0 conn(0x7fd0cc0cc370 legacy=0x7fd0cc098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0ca000640 1 --2- 172.21.15.46:0/1163703798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0b0003b10 0x7fd0b0003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0ca000640 1 --2- 172.21.15.46:0/1163703798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0b00047d0 0x7fd0b0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0ca000640 1 --2- 172.21.15.46:0/1163703798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0b0005190 0x7fd0b000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0ca000640 1 -- 172.21.15.46:0/1163703798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0ce850 con 0x7fd0b00047d0 2024-03-29T08:27:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0ca000640 1 -- 172.21.15.46:0/1163703798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b000c0c0 con 0x7fd0b0003b10 2024-03-29T08:27:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0ca000640 1 -- 172.21.15.46:0/1163703798 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b000c280 con 0x7fd0b0005190 2024-03-29T08:27:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0c2000640 1 --2- 172.21.15.46:0/1163703798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0b00047d0 0x7fd0b0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0caa00640 1 --2- 172.21.15.46:0/1163703798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0b0003b10 0x7fd0b0003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0cb400640 1 --2- 172.21.15.46:0/1163703798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0b0005190 0x7fd0b000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0c2000640 1 -- 172.21.15.46:0/1163703798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0b0005190 msgr2=0x7fd0b000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:35.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0c2000640 1 --2- 172.21.15.46:0/1163703798 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0b0005190 0x7fd0b000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0c2000640 1 -- 172.21.15.46:0/1163703798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0b0003b10 msgr2=0x7fd0b0003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0c2000640 1 --2- 172.21.15.46:0/1163703798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0b0003b10 0x7fd0b0003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0c2000640 1 -- 172.21.15.46:0/1163703798 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b0003530 con 0x7fd0b00047d0 2024-03-29T08:27:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0cb400640 1 -- 172.21.15.46:0/1163703798 reap_dead start 2024-03-29T08:27:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.607+0000 7fd0c2000640 1 --2- 172.21.15.46:0/1163703798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0b00047d0 0x7fd0b0004c50 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400c160 tx=0x7fd0b401cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1612b6baf4eadb59 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0ca000640 1 -- 172.21.15.46:0/1163703798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b40058e0 con 0x7fd0b00047d0 2024-03-29T08:27:35.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0ca000640 1 -- 172.21.15.46:0/1163703798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b4005a60 con 0x7fd0b00047d0 2024-03-29T08:27:35.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0ca000640 1 -- 172.21.15.46:0/1163703798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4005e50 con 0x7fd0b00047d0 2024-03-29T08:27:35.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0d1600640 1 -- 172.21.15.46:0/1163703798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0b00047d0 msgr2=0x7fd0b0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0d1600640 1 --2- 172.21.15.46:0/1163703798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0b00047d0 0x7fd0b0004c50 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fd0b400c160 tx=0x7fd0b401cbc0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0d1600640 1 -- 172.21.15.46:0/1163703798 shutdown_connections 2024-03-29T08:27:35.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0d1600640 1 --2- 172.21.15.46:0/1163703798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0b00047d0 0x7fd0b0004c50 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0d1600640 1 -- 172.21.15.46:0/1163703798 >> 172.21.15.46:0/1163703798 conn(0x7fd0cc0b21b0 msgr2=0x7fd0cc0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:35.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0d1600640 1 -- 172.21.15.46:0/1163703798 shutdown_connections 2024-03-29T08:27:35.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0d1600640 1 -- 172.21.15.46:0/1163703798 wait complete. 2024-03-29T08:27:35.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0d1600640 1 Processor -- start 2024-03-29T08:27:35.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0d1600640 1 -- start start 2024-03-29T08:27:35.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0d1600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0b00047d0 0x7fd0cc0b4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0d1600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0cc153c80 0x7fd0cc0c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0d1600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0cc153730 0x7fd0cc152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0d1600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0b4030 con 0x7fd0cc153c80 2024-03-29T08:27:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0d1600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0b3c90 con 0x7fd0cc153730 2024-03-29T08:27:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0d1600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc0b5b80 con 0x7fd0b00047d0 2024-03-29T08:27:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0c2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0cc153c80 0x7fd0cc0c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0c2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0cc153c80 0x7fd0cc0c9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36074/0 (socket says 172.21.15.46:36074) 2024-03-29T08:27:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0c2000640 1 -- 172.21.15.46:0/1645121523 learned_addr learned my addr 172.21.15.46:0/1645121523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0cb400640 1 --2- 172.21.15.46:0/1645121523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0cc153730 0x7fd0cc152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.611+0000 7fd0caa00640 1 --2- 172.21.15.46:0/1645121523 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0b00047d0 0x7fd0cc0b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.615+0000 7fd0c2000640 1 -- 172.21.15.46:0/1645121523 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0b00047d0 msgr2=0x7fd0cc0b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.615+0000 7fd0c2000640 1 --2- 172.21.15.46:0/1645121523 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0b00047d0 0x7fd0cc0b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.615+0000 7fd0c2000640 1 -- 172.21.15.46:0/1645121523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0cc153730 msgr2=0x7fd0cc152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.615+0000 7fd0c2000640 1 --2- 172.21.15.46:0/1645121523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0cc153730 0x7fd0cc152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.615+0000 7fd0c2000640 1 -- 172.21.15.46:0/1645121523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc0b5e60 con 0x7fd0cc153c80 2024-03-29T08:27:35.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.615+0000 7fd0c2000640 1 --2- 172.21.15.46:0/1645121523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0cc153c80 0x7fd0cc0c9890 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40018c0 tx=0x7fd0b0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:35.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.615+0000 7fd0c3400640 1 -- 172.21.15.46:0/1645121523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b40047f0 con 0x7fd0cc153c80 2024-03-29T08:27:35.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.615+0000 7fd0c3400640 1 -- 172.21.15.46:0/1645121523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b4004950 con 0x7fd0cc153c80 2024-03-29T08:27:35.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.615+0000 7fd0c3400640 1 -- 172.21.15.46:0/1645121523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b4004cc0 con 0x7fd0cc153c80 2024-03-29T08:27:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.615+0000 7fd0d1600640 1 -- 172.21.15.46:0/1645121523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0cc0b6140 con 0x7fd0cc153c80 2024-03-29T08:27:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.615+0000 7fd0d1600640 1 -- 172.21.15.46:0/1645121523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc0b4990 con 0x7fd0cc153c80 2024-03-29T08:27:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.615+0000 7fd0d1600640 1 -- 172.21.15.46:0/1645121523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0cc0d0040 con 0x7fd0cc153c80 2024-03-29T08:27:35.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.615+0000 7fd0c3400640 1 -- 172.21.15.46:0/1645121523 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd0b4007040 con 0x7fd0cc153c80 2024-03-29T08:27:35.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.615+0000 7fd0c3400640 1 --2- 172.21.15.46:0/1645121523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd0a80666c0 0x7fd0a8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.615+0000 7fd0c3400640 1 -- 172.21.15.46:0/1645121523 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0b411da30 con 0x7fd0cc153c80 2024-03-29T08:27:35.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.615+0000 7fd0caa00640 1 --2- 172.21.15.46:0/1645121523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd0a80666c0 0x7fd0a8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.619+0000 7fd0caa00640 1 --2- 172.21.15.46:0/1645121523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd0a80666c0 0x7fd0a8068b80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc014590 tx=0x7fd0bc00b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:35.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.619+0000 7fd0c3400640 1 -- 172.21.15.46:0/1645121523 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd0b40eae40 con 0x7fd0cc153c80 2024-03-29T08:27:35.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.779+0000 7fd0d1600640 1 -- 172.21.15.46:0/1645121523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0cc0b4b80 con 0x7fd0cc153c80 2024-03-29T08:27:35.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.783+0000 7fd0c3400640 1 -- 172.21.15.46:0/1645121523 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd0cc0b4b80 con 0x7fd0cc153c80 2024-03-29T08:27:35.784 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:27:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.783+0000 7fd0d1600640 1 -- 172.21.15.46:0/1645121523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd0a80666c0 msgr2=0x7fd0a8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.783+0000 7fd0d1600640 1 --2- 172.21.15.46:0/1645121523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd0a80666c0 0x7fd0a8068b80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc014590 tx=0x7fd0bc00b040 comp rx=0 tx=0).stop 2024-03-29T08:27:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.783+0000 7fd0d1600640 1 -- 172.21.15.46:0/1645121523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0cc153c80 msgr2=0x7fd0cc0c9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.783+0000 7fd0d1600640 1 --2- 172.21.15.46:0/1645121523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0cc153c80 0x7fd0cc0c9890 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fd0b40018c0 tx=0x7fd0b0005800 comp rx=0 tx=0).stop 2024-03-29T08:27:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.783+0000 7fd0d1600640 1 -- 172.21.15.46:0/1645121523 shutdown_connections 2024-03-29T08:27:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.783+0000 7fd0d1600640 1 --2- 172.21.15.46:0/1645121523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd0a80666c0 0x7fd0a8068b80 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.783+0000 7fd0d1600640 1 --2- 172.21.15.46:0/1645121523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0cc153730 0x7fd0cc152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.783+0000 7fd0d1600640 1 --2- 172.21.15.46:0/1645121523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0cc153c80 0x7fd0cc0c9890 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.783+0000 7fd0d1600640 1 --2- 172.21.15.46:0/1645121523 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0b00047d0 0x7fd0cc0b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.783+0000 7fd0d1600640 1 -- 172.21.15.46:0/1645121523 >> 172.21.15.46:0/1645121523 conn(0x7fd0cc0b21b0 msgr2=0x7fd0cc06e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.783+0000 7fd0d1600640 1 -- 172.21.15.46:0/1645121523 shutdown_connections 2024-03-29T08:27:35.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.783+0000 7fd0d1600640 1 -- 172.21.15.46:0/1645121523 wait complete. 2024-03-29T08:27:35.805 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:27:35.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.943+0000 7f22e3c00640 1 Processor -- start 2024-03-29T08:27:35.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.943+0000 7f22e3c00640 1 -- start start 2024-03-29T08:27:35.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.943+0000 7f22e3c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22dc0af530 0x7f22dc0af970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.943+0000 7f22e3c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22dc0702a0 0x7f22dc070730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.943+0000 7f22e3c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22dc06c390 con 0x7f22dc158300 2024-03-29T08:27:35.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.943+0000 7f22e3c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22dc070c70 con 0x7f22dc0702a0 2024-03-29T08:27:35.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.943+0000 7f22e3c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc0afff0 con 0x7f22dc0af530 2024-03-29T08:27:35.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.943+0000 7f22e1c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22dc0702a0 0x7f22dc070730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.947+0000 7f22e1c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22dc0702a0 0x7f22dc070730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36078/0 (socket says 172.21.15.46:36078) 2024-03-29T08:27:35.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.947+0000 7f22e1c00640 1 -- 172.21.15.46:0/3112139176 learned_addr learned my addr 172.21.15.46:0/3112139176 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:35.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.947+0000 7f22d3e00640 1 --2- 172.21.15.46:0/3112139176 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22dc0af530 0x7f22dc0af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.947+0000 7f22e1c00640 1 -- 172.21.15.46:0/3112139176 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22dc0af530 msgr2=0x7f22dc0af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:35.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.947+0000 7f22e1c00640 1 --2- 172.21.15.46:0/3112139176 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22dc0af530 0x7f22dc0af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.947+0000 7f22e1c00640 1 -- 172.21.15.46:0/3112139176 >> v1:172.21.15.73:6789/0 conn(0x7f22dc158300 legacy=0x7f22dc06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.947+0000 7f22e1c00640 1 -- 172.21.15.46:0/3112139176 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc06c5e0 con 0x7f22dc0702a0 2024-03-29T08:27:35.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.947+0000 7f22e1c00640 1 --2- 172.21.15.46:0/3112139176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0702a0 0x7f22dc070730 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009e50 tx=0x7f22d401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41681f45b01ae447 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:35.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.947+0000 7f22dbe00640 1 -- 172.21.15.46:0/3112139176 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40a57a0 con 0x7f22dc0702a0 2024-03-29T08:27:35.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.947+0000 7f22dbe00640 1 -- 172.21.15.46:0/3112139176 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d40a5900 con 0x7f22dc0702a0 2024-03-29T08:27:35.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.947+0000 7f22dbe00640 1 -- 172.21.15.46:0/3112139176 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d40a5c90 con 0x7f22dc0702a0 2024-03-29T08:27:35.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e3c00640 1 -- 172.21.15.46:0/3112139176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0702a0 msgr2=0x7f22dc070730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e3c00640 1 --2- 172.21.15.46:0/3112139176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0702a0 0x7f22dc070730 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f22d4009e50 tx=0x7f22d401c8e0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e3c00640 1 -- 172.21.15.46:0/3112139176 shutdown_connections 2024-03-29T08:27:35.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e3c00640 1 --2- 172.21.15.46:0/3112139176 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0702a0 0x7f22dc070730 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e3c00640 1 --2- 172.21.15.46:0/3112139176 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22dc0af530 0x7f22dc0af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e3c00640 1 -- 172.21.15.46:0/3112139176 >> 172.21.15.46:0/3112139176 conn(0x7f22dc066080 msgr2=0x7f22dc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:35.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e3c00640 1 -- 172.21.15.46:0/3112139176 shutdown_connections 2024-03-29T08:27:35.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e3c00640 1 -- 172.21.15.46:0/3112139176 wait complete. 2024-03-29T08:27:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e3c00640 1 Processor -- start 2024-03-29T08:27:35.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e3c00640 1 -- start start 2024-03-29T08:27:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e3c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0702a0 0x7f22dc0b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e3c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22dc0af530 0x7f22dc159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e3c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22dc158300 0x7f22dc1f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e3c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc152d90 con 0x7f22dc0702a0 2024-03-29T08:27:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e3c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc0b1da0 con 0x7f22dc0af530 2024-03-29T08:27:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e3c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22dc0b2080 con 0x7f22dc158300 2024-03-29T08:27:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0702a0 0x7f22dc0b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0702a0 0x7f22dc0b1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36094/0 (socket says 172.21.15.46:36094) 2024-03-29T08:27:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e1200640 1 -- 172.21.15.46:0/1663964683 learned_addr learned my addr 172.21.15.46:0/1663964683 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22d3e00640 1 --2- 172.21.15.46:0/1663964683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22dc0af530 0x7f22dc159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e1c00640 1 --2- 172.21.15.46:0/1663964683 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22dc158300 0x7f22dc1f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e1200640 1 -- 172.21.15.46:0/1663964683 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22dc158300 msgr2=0x7f22dc1f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e1200640 1 --2- 172.21.15.46:0/1663964683 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22dc158300 0x7f22dc1f54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e1200640 1 -- 172.21.15.46:0/1663964683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22dc0af530 msgr2=0x7f22dc159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:35.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e1200640 1 --2- 172.21.15.46:0/1663964683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22dc0af530 0x7f22dc159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e1200640 1 -- 172.21.15.46:0/1663964683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d4009b30 con 0x7f22dc0702a0 2024-03-29T08:27:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22e1200640 1 --2- 172.21.15.46:0/1663964683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0702a0 0x7f22dc0b1730 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00c3c0 tx=0x7f22cc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22d9600640 1 -- 172.21.15.46:0/1663964683 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc0052d0 con 0x7f22dc0702a0 2024-03-29T08:27:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22d9600640 1 -- 172.21.15.46:0/1663964683 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22cc005430 con 0x7f22dc0702a0 2024-03-29T08:27:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.951+0000 7f22d9600640 1 -- 172.21.15.46:0/1663964683 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22cc003840 con 0x7f22dc0702a0 2024-03-29T08:27:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.955+0000 7f22e3c00640 1 -- 172.21.15.46:0/1663964683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22dc0b2360 con 0x7f22dc0702a0 2024-03-29T08:27:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.955+0000 7f22e3c00640 1 -- 172.21.15.46:0/1663964683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22dc0b2920 con 0x7f22dc0702a0 2024-03-29T08:27:35.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.955+0000 7f22e3c00640 1 -- 172.21.15.46:0/1663964683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22dc0671b0 con 0x7f22dc0702a0 2024-03-29T08:27:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.955+0000 7f22d9600640 1 -- 172.21.15.46:0/1663964683 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f22cc0039a0 con 0x7f22dc0702a0 2024-03-29T08:27:35.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.955+0000 7f22d9600640 1 --2- 172.21.15.46:0/1663964683 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22b8066690 0x7f22b8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:35.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.955+0000 7f22d9600640 1 -- 172.21.15.46:0/1663964683 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22cc0836d0 con 0x7f22dc0702a0 2024-03-29T08:27:35.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.955+0000 7f22d3e00640 1 --2- 172.21.15.46:0/1663964683 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22b8066690 0x7f22b8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:35.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.959+0000 7f22d3e00640 1 --2- 172.21.15.46:0/1663964683 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22b8066690 0x7f22b8068b50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f22dc1644b0 tx=0x7f22c4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:35.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:35.959+0000 7f22d9600640 1 -- 172.21.15.46:0/1663964683 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f22cc050a40 con 0x7f22dc0702a0 2024-03-29T08:27:36.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.135+0000 7f22e3c00640 1 -- 172.21.15.46:0/1663964683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f22dc071a90 con 0x7f22dc0702a0 2024-03-29T08:27:36.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.135+0000 7f22d9600640 1 -- 172.21.15.46:0/1663964683 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) v1 ==== 80+0+0 (secure 0 0 0) 0x7f22cc054060 con 0x7f22dc0702a0 2024-03-29T08:27:36.136 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:27:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.135+0000 7f22e3c00640 1 -- 172.21.15.46:0/1663964683 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22b8066690 msgr2=0x7f22b8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.135+0000 7f22e3c00640 1 --2- 172.21.15.46:0/1663964683 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22b8066690 0x7f22b8068b50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f22dc1644b0 tx=0x7f22c4008040 comp rx=0 tx=0).stop 2024-03-29T08:27:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.135+0000 7f22e3c00640 1 -- 172.21.15.46:0/1663964683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0702a0 msgr2=0x7f22dc0b1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.135+0000 7f22e3c00640 1 --2- 172.21.15.46:0/1663964683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0702a0 0x7f22dc0b1730 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00c3c0 tx=0x7f22cc00c880 comp rx=0 tx=0).stop 2024-03-29T08:27:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.135+0000 7f22e3c00640 1 -- 172.21.15.46:0/1663964683 shutdown_connections 2024-03-29T08:27:36.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.135+0000 7f22e3c00640 1 --2- 172.21.15.46:0/1663964683 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22b8066690 0x7f22b8068b50 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.135+0000 7f22e3c00640 1 --2- 172.21.15.46:0/1663964683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22dc0af530 0x7f22dc159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.135+0000 7f22e3c00640 1 --2- 172.21.15.46:0/1663964683 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22dc158300 0x7f22dc1f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.135+0000 7f22e3c00640 1 --2- 172.21.15.46:0/1663964683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc0702a0 0x7f22dc0b1730 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.135+0000 7f22e3c00640 1 -- 172.21.15.46:0/1663964683 >> 172.21.15.46:0/1663964683 conn(0x7f22dc066080 msgr2=0x7f22dc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:36.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.135+0000 7f22e3c00640 1 -- 172.21.15.46:0/1663964683 shutdown_connections 2024-03-29T08:27:36.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.135+0000 7f22e3c00640 1 -- 172.21.15.46:0/1663964683 wait complete. 2024-03-29T08:27:36.157 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:27:36.157 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:27:36.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:27:36.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.347+0000 7fdce2600640 1 Processor -- start 2024-03-29T08:27:36.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.347+0000 7fdce2600640 1 -- start start 2024-03-29T08:27:36.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.347+0000 7fdce2600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdcdc069d90 0x7fdcdc06a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:36.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.347+0000 7fdce2600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdcdc161170 0x7fdcdc0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:36.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.347+0000 7fdce2600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcdc160440 con 0x7fdcdc0cb550 2024-03-29T08:27:36.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.347+0000 7fdce2600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcdc160ce0 con 0x7fdcdc069d90 2024-03-29T08:27:36.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.347+0000 7fdce2600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcdc06a6b0 con 0x7fdcdc161170 2024-03-29T08:27:36.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.347+0000 7fdce1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdcdc069d90 0x7fdcdc06a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:36.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.347+0000 7fdce1c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdcdc0cb550 0x7fdcdc0cbb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59650/0 (socket says 172.21.15.46:59650) 2024-03-29T08:27:36.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.347+0000 7fdce1200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdcdc069d90 0x7fdcdc06a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36104/0 (socket says 172.21.15.46:36104) 2024-03-29T08:27:36.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.347+0000 7fdce1c00640 1 -- 172.21.15.46:0/543758199 learned_addr learned my addr 172.21.15.46:0/543758199 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:36.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.347+0000 7fdcd3e00640 1 --2- 172.21.15.46:0/543758199 >> v2:172.21.15.73:3300/0 conn(0x7fdcdc161170 0x7fdcdc0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:36.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.347+0000 7fdce1200640 1 -- 172.21.15.46:0/543758199 >> v2:172.21.15.73:3300/0 conn(0x7fdcdc161170 msgr2=0x7fdcdc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:36.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.347+0000 7fdce1200640 1 --2- 172.21.15.46:0/543758199 >> v2:172.21.15.73:3300/0 conn(0x7fdcdc161170 0x7fdcdc0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.347+0000 7fdce1200640 1 -- 172.21.15.46:0/543758199 >> v1:172.21.15.46:6789/0 conn(0x7fdcdc0cb550 legacy=0x7fdcdc0cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:36.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.347+0000 7fdce1200640 1 -- 172.21.15.46:0/543758199 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc0cefa0 con 0x7fdcdc069d90 2024-03-29T08:27:36.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.351+0000 7fdce1200640 1 --2- 172.21.15.46:0/543758199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc069d90 0x7fdcdc06a170 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0cdb40 tx=0x7fdccc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7f61349f93bd73 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:36.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.351+0000 7fdcdbe00640 1 -- 172.21.15.46:0/543758199 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc0ac620 con 0x7fdcdc069d90 2024-03-29T08:27:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.351+0000 7fdcdbe00640 1 -- 172.21.15.46:0/543758199 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdccc0ac780 con 0x7fdcdc069d90 2024-03-29T08:27:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.351+0000 7fdcdbe00640 1 -- 172.21.15.46:0/543758199 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc0acb10 con 0x7fdcdc069d90 2024-03-29T08:27:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.351+0000 7fdce2600640 1 -- 172.21.15.46:0/543758199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc069d90 msgr2=0x7fdcdc06a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.351+0000 7fdce2600640 1 --2- 172.21.15.46:0/543758199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc069d90 0x7fdcdc06a170 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0cdb40 tx=0x7fdccc01c3c0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.351+0000 7fdce2600640 1 -- 172.21.15.46:0/543758199 shutdown_connections 2024-03-29T08:27:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.351+0000 7fdce2600640 1 --2- 172.21.15.46:0/543758199 >> v2:172.21.15.73:3300/0 conn(0x7fdcdc161170 0x7fdcdc0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.351+0000 7fdce2600640 1 --2- 172.21.15.46:0/543758199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc069d90 0x7fdcdc06a170 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.351+0000 7fdce2600640 1 -- 172.21.15.46:0/543758199 >> 172.21.15.46:0/543758199 conn(0x7fdcdc098a40 msgr2=0x7fdcdc0cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.351+0000 7fdce2600640 1 -- 172.21.15.46:0/543758199 shutdown_connections 2024-03-29T08:27:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.351+0000 7fdce2600640 1 -- 172.21.15.46:0/543758199 wait complete. 2024-03-29T08:27:36.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.351+0000 7fdce2600640 1 Processor -- start 2024-03-29T08:27:36.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.351+0000 7fdce2600640 1 -- start start 2024-03-29T08:27:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdce2600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcdc069d90 0x7fdcdc149cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdce2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc0cb550 0x7fdcdc14b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdce2600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdcdc161170 0x7fdcdc0b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdce2600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc0b51d0 con 0x7fdcdc0cb550 2024-03-29T08:27:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdce2600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc136f40 con 0x7fdcdc069d90 2024-03-29T08:27:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdce2600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc137220 con 0x7fdcdc161170 2024-03-29T08:27:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdcd3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc0cb550 0x7fdcdc14b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdcd3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc0cb550 0x7fdcdc14b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36108/0 (socket says 172.21.15.46:36108) 2024-03-29T08:27:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdcd3e00640 1 -- 172.21.15.46:0/1375195602 learned_addr learned my addr 172.21.15.46:0/1375195602 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdce1c00640 1 --2- 172.21.15.46:0/1375195602 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdcdc161170 0x7fdcdc0b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdce1200640 1 --2- 172.21.15.46:0/1375195602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcdc069d90 0x7fdcdc149cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdcd3e00640 1 -- 172.21.15.46:0/1375195602 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdcdc161170 msgr2=0x7fdcdc0b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdcd3e00640 1 --2- 172.21.15.46:0/1375195602 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdcdc161170 0x7fdcdc0b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdcd3e00640 1 -- 172.21.15.46:0/1375195602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcdc069d90 msgr2=0x7fdcdc149cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdcd3e00640 1 --2- 172.21.15.46:0/1375195602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcdc069d90 0x7fdcdc149cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdcd3e00640 1 -- 172.21.15.46:0/1375195602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc009420 con 0x7fdcdc0cb550 2024-03-29T08:27:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdcd3e00640 1 --2- 172.21.15.46:0/1375195602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc0cb550 0x7fdcdc14b870 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400d5f0 tx=0x7fdcc400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdcd9600640 1 -- 172.21.15.46:0/1375195602 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40125f0 con 0x7fdcdc0cb550 2024-03-29T08:27:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdcd9600640 1 -- 172.21.15.46:0/1375195602 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc4012750 con 0x7fdcdc0cb550 2024-03-29T08:27:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdcd9600640 1 -- 172.21.15.46:0/1375195602 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4012ae0 con 0x7fdcdc0cb550 2024-03-29T08:27:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdce2600640 1 -- 172.21.15.46:0/1375195602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcdc137500 con 0x7fdcdc0cb550 2024-03-29T08:27:36.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdce2600640 1 -- 172.21.15.46:0/1375195602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcdc137ac0 con 0x7fdcdc0cb550 2024-03-29T08:27:36.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.355+0000 7fdce2600640 1 -- 172.21.15.46:0/1375195602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcdc137c50 con 0x7fdcdc0cb550 2024-03-29T08:27:36.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.359+0000 7fdcd9600640 1 -- 172.21.15.46:0/1375195602 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdcc4012c40 con 0x7fdcdc0cb550 2024-03-29T08:27:36.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.359+0000 7fdcd9600640 1 --2- 172.21.15.46:0/1375195602 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdcb8066690 0x7fdcb8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:36.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.359+0000 7fdcd9600640 1 -- 172.21.15.46:0/1375195602 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdcc4084560 con 0x7fdcdc0cb550 2024-03-29T08:27:36.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.359+0000 7fdce1200640 1 --2- 172.21.15.46:0/1375195602 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdcb8066690 0x7fdcb8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:36.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.359+0000 7fdce1200640 1 --2- 172.21.15.46:0/1375195602 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdcb8066690 0x7fdcb8068b50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0ae000 tx=0x7fdccc0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:36.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.363+0000 7fdcd9600640 1 -- 172.21.15.46:0/1375195602 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdcc4051970 con 0x7fdcdc0cb550 2024-03-29T08:27:36.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.523+0000 7fdce2600640 1 -- 172.21.15.46:0/1375195602 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcdc04e040 con 0x7fdcdc0cb550 2024-03-29T08:27:36.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.527+0000 7fdcd9600640 1 -- 172.21.15.46:0/1375195602 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16003 (secure 0 0 0) 0x7fdcc4007380 con 0x7fdcdc0cb550 2024-03-29T08:27:36.529 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:36.529 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":106,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:27:19.585517+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:27:17.118875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-29T08:27:17.573928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:27:36.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.527+0000 7fdce2600640 1 -- 172.21.15.46:0/1375195602 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdcb8066690 msgr2=0x7fdcb8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:36.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.527+0000 7fdce2600640 1 --2- 172.21.15.46:0/1375195602 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdcb8066690 0x7fdcb8068b50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0ae000 tx=0x7fdccc0a7000 comp rx=0 tx=0).stop 2024-03-29T08:27:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.527+0000 7fdce2600640 1 -- 172.21.15.46:0/1375195602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc0cb550 msgr2=0x7fdcdc14b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.527+0000 7fdce2600640 1 --2- 172.21.15.46:0/1375195602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc0cb550 0x7fdcdc14b870 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400d5f0 tx=0x7fdcc400dab0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.527+0000 7fdce2600640 1 -- 172.21.15.46:0/1375195602 shutdown_connections 2024-03-29T08:27:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.527+0000 7fdce2600640 1 --2- 172.21.15.46:0/1375195602 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdcb8066690 0x7fdcb8068b50 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.527+0000 7fdce2600640 1 --2- 172.21.15.46:0/1375195602 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdcdc161170 0x7fdcdc0b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.527+0000 7fdce2600640 1 --2- 172.21.15.46:0/1375195602 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdcdc0cb550 0x7fdcdc14b870 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.527+0000 7fdce2600640 1 --2- 172.21.15.46:0/1375195602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcdc069d90 0x7fdcdc149cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.527+0000 7fdce2600640 1 -- 172.21.15.46:0/1375195602 >> 172.21.15.46:0/1375195602 conn(0x7fdcdc098a40 msgr2=0x7fdcdc06e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.527+0000 7fdce2600640 1 -- 172.21.15.46:0/1375195602 shutdown_connections 2024-03-29T08:27:36.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.527+0000 7fdce2600640 1 -- 172.21.15.46:0/1375195602 wait complete. 2024-03-29T08:27:36.549 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:27:17.118875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:27:17.573928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:27:36.550 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:27:36.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ffbe00640 1 Processor -- start 2024-03-29T08:27:36.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ffbe00640 1 -- start start 2024-03-29T08:27:36.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ffbe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5ff4142a50 0x7f5ff40cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:36.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ffbe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5ff415df90 0x7f5ff40cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:36.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ffbe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff415d260 con 0x7f5ff40ae280 2024-03-29T08:27:36.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ffbe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff40aefd0 con 0x7f5ff4142a50 2024-03-29T08:27:36.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ffbe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff415d4b0 con 0x7f5ff415df90 2024-03-29T08:27:36.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff9e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5ff40ae280 0x7f5ff40ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45006/0 (socket says 172.21.15.46:45006) 2024-03-29T08:27:36.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff9e00640 1 -- 172.21.15.46:0/3822144098 learned_addr learned my addr 172.21.15.46:0/3822144098 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:36.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff8a00640 1 --2- 172.21.15.46:0/3822144098 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5ff415df90 0x7f5ff40cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:36.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff9400640 1 --2- 172.21.15.46:0/3822144098 >> v2:172.21.15.73:3300/0 conn(0x7f5ff4142a50 0x7f5ff40cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:36.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff3e00640 1 -- 172.21.15.46:0/3822144098 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2275395114 0 0) 0x7f5ff415d260 con 0x7f5ff40ae280 2024-03-29T08:27:36.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff3e00640 1 -- 172.21.15.46:0/3822144098 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fdc003530 con 0x7f5ff40ae280 2024-03-29T08:27:36.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff3e00640 1 -- 172.21.15.46:0/3822144098 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 836105972 0 0) 0x7f5fdc003530 con 0x7f5ff40ae280 2024-03-29T08:27:36.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff3e00640 1 -- 172.21.15.46:0/3822144098 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5ff415df90 msgr2=0x7f5ff40cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:36.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff3e00640 1 --2- 172.21.15.46:0/3822144098 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5ff415df90 0x7f5ff40cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff3e00640 1 -- 172.21.15.46:0/3822144098 >> v2:172.21.15.73:3300/0 conn(0x7f5ff4142a50 msgr2=0x7f5ff40cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:36.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff3e00640 1 --2- 172.21.15.46:0/3822144098 >> v2:172.21.15.73:3300/0 conn(0x7f5ff4142a50 0x7f5ff40cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff3e00640 1 -- 172.21.15.46:0/3822144098 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff415d790 con 0x7f5ff40ae280 2024-03-29T08:27:36.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff3e00640 1 -- 172.21.15.46:0/3822144098 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f5fec001a00 con 0x7f5ff40ae280 2024-03-29T08:27:36.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff3e00640 1 -- 172.21.15.46:0/3822144098 >> v1:172.21.15.46:6789/0 conn(0x7f5ff40ae280 legacy=0x7f5ff40ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:36.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff3e00640 1 --2- 172.21.15.46:0/3822144098 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5ff415df90 0x7f5fdc003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:36.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff3e00640 1 --2- 172.21.15.46:0/3822144098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fdc003f50 0x7f5fdc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:36.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff3e00640 1 --2- 172.21.15.46:0/3822144098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fdc004d10 0x7f5fdc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:36.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff3e00640 1 -- 172.21.15.46:0/3822144098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff40aefd0 con 0x7f5fdc004d10 2024-03-29T08:27:36.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff3e00640 1 -- 172.21.15.46:0/3822144098 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff415d4b0 con 0x7f5fdc003f50 2024-03-29T08:27:36.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff3e00640 1 -- 172.21.15.46:0/3822144098 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc00ba40 con 0x7f5ff415df90 2024-03-29T08:27:36.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff9e00640 1 --2- 172.21.15.46:0/3822144098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fdc004d10 0x7f5fdc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:36.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff8a00640 1 --2- 172.21.15.46:0/3822144098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fdc003f50 0x7f5fdc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:36.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.691+0000 7f5ff9400640 1 --2- 172.21.15.46:0/3822144098 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5ff415df90 0x7f5fdc003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:36.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.695+0000 7f5ff9e00640 1 -- 172.21.15.46:0/3822144098 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5ff415df90 msgr2=0x7f5fdc003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:36.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.695+0000 7f5ff9e00640 1 --2- 172.21.15.46:0/3822144098 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5ff415df90 0x7f5fdc003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.695+0000 7f5ff9e00640 1 -- 172.21.15.46:0/3822144098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fdc003f50 msgr2=0x7f5fdc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:36.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.695+0000 7f5ff9e00640 1 --2- 172.21.15.46:0/3822144098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fdc003f50 0x7f5fdc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.695+0000 7f5ff9e00640 1 -- 172.21.15.46:0/3822144098 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fdc003530 con 0x7f5fdc004d10 2024-03-29T08:27:36.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.695+0000 7f5ff9e00640 1 --2- 172.21.15.46:0/3822144098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fdc004d10 0x7f5fdc005190 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00a790 tx=0x7f5fec01fa20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da207e712e1c118b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:36.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.695+0000 7f5ff3e00640 1 -- 172.21.15.46:0/3822144098 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0aa650 con 0x7f5fdc004d10 2024-03-29T08:27:36.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.695+0000 7f5ff3e00640 1 -- 172.21.15.46:0/3822144098 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec0aa7d0 con 0x7f5fdc004d10 2024-03-29T08:27:36.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.695+0000 7f5ff3e00640 1 -- 172.21.15.46:0/3822144098 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0aabc0 con 0x7f5fdc004d10 2024-03-29T08:27:36.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.695+0000 7f5ffbe00640 1 -- 172.21.15.46:0/3822144098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fdc004d10 msgr2=0x7f5fdc005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:36.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.695+0000 7f5ffbe00640 1 --2- 172.21.15.46:0/3822144098 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fdc004d10 0x7f5fdc005190 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00a790 tx=0x7f5fec01fa20 comp rx=0 tx=0).stop 2024-03-29T08:27:36.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.695+0000 7f5ff9e00640 1 -- 172.21.15.46:0/3822144098 reap_dead start 2024-03-29T08:27:36.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.695+0000 7f5ffbe00640 1 -- 172.21.15.46:0/3822144098 shutdown_connections 2024-03-29T08:27:36.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.695+0000 7f5ffbe00640 1 -- 172.21.15.46:0/3822144098 >> 172.21.15.46:0/3822144098 conn(0x7f5ff406a390 msgr2=0x7f5ff40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:36.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.695+0000 7f5ffbe00640 1 -- 172.21.15.46:0/3822144098 shutdown_connections 2024-03-29T08:27:36.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.695+0000 7f5ffbe00640 1 -- 172.21.15.46:0/3822144098 wait complete. 2024-03-29T08:27:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ffbe00640 1 Processor -- start 2024-03-29T08:27:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ffbe00640 1 -- start start 2024-03-29T08:27:36.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ffbe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fdc004d10 0x7f5ff4158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ffbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ff41f36d0 0x7f5ff40ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ffbe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5ff40ac8f0 0x7f5ff40cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ffbe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff41594b0 con 0x7f5ff41f36d0 2024-03-29T08:27:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ffbe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff4159d40 con 0x7f5fdc004d10 2024-03-29T08:27:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ffbe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff415a020 con 0x7f5ff40ac8f0 2024-03-29T08:27:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ff8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ff41f36d0 0x7f5ff40ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ff8a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ff41f36d0 0x7f5ff40ac3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51462/0 (socket says 172.21.15.46:51462) 2024-03-29T08:27:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ff8a00640 1 -- 172.21.15.46:0/1337266771 learned_addr learned my addr 172.21.15.46:0/1337266771 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ff9e00640 1 --2- 172.21.15.46:0/1337266771 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5ff40ac8f0 0x7f5ff40cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ff9400640 1 --2- 172.21.15.46:0/1337266771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fdc004d10 0x7f5ff4158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ff8a00640 1 -- 172.21.15.46:0/1337266771 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5ff40ac8f0 msgr2=0x7f5ff40cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ff8a00640 1 --2- 172.21.15.46:0/1337266771 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5ff40ac8f0 0x7f5ff40cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ff8a00640 1 -- 172.21.15.46:0/1337266771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fdc004d10 msgr2=0x7f5ff4158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ff8a00640 1 --2- 172.21.15.46:0/1337266771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fdc004d10 0x7f5ff4158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ff8a00640 1 -- 172.21.15.46:0/1337266771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff415a300 con 0x7f5ff41f36d0 2024-03-29T08:27:36.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ff9400640 1 --2- 172.21.15.46:0/1337266771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fdc004d10 0x7f5ff4158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:36.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ff8a00640 1 --2- 172.21.15.46:0/1337266771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ff41f36d0 0x7f5ff40ac3b0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0011960 tx=0x7f5fe0011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:36.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ff1600640 1 -- 172.21.15.46:0/1337266771 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe000faf0 con 0x7f5ff41f36d0 2024-03-29T08:27:36.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ff1600640 1 -- 172.21.15.46:0/1337266771 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe000fc50 con 0x7f5ff41f36d0 2024-03-29T08:27:36.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ffbe00640 1 -- 172.21.15.46:0/1337266771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff415a5e0 con 0x7f5ff41f36d0 2024-03-29T08:27:36.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ff1600640 1 -- 172.21.15.46:0/1337266771 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0014440 con 0x7f5ff41f36d0 2024-03-29T08:27:36.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.699+0000 7f5ffbe00640 1 -- 172.21.15.46:0/1337266771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff41f3b30 con 0x7f5ff41f36d0 2024-03-29T08:27:36.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.703+0000 7f5ffbe00640 1 -- 172.21.15.46:0/1337266771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff4071a70 con 0x7f5ff41f36d0 2024-03-29T08:27:36.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.703+0000 7f5ff1600640 1 -- 172.21.15.46:0/1337266771 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5fe00145a0 con 0x7f5ff41f36d0 2024-03-29T08:27:36.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.703+0000 7f5ff1600640 1 --2- 172.21.15.46:0/1337266771 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5fd4066690 0x7f5fd4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:36.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.703+0000 7f5ff1600640 1 -- 172.21.15.46:0/1337266771 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5fe0084a50 con 0x7f5ff41f36d0 2024-03-29T08:27:36.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.703+0000 7f5ff9400640 1 --2- 172.21.15.46:0/1337266771 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5fd4066690 0x7f5fd4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:36.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.703+0000 7f5ff9400640 1 --2- 172.21.15.46:0/1337266771 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5fd4066690 0x7f5fd4068b50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009c60 tx=0x7f5fe8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:36.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.707+0000 7f5ff1600640 1 -- 172.21.15.46:0/1337266771 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5fe0051d20 con 0x7f5ff41f36d0 2024-03-29T08:27:36.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.871+0000 7f5ffbe00640 1 -- 172.21.15.46:0/1337266771 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5ff40aedb0 con 0x7f5ff41f36d0 2024-03-29T08:27:36.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.871+0000 7f5ff1600640 1 -- 172.21.15.46:0/1337266771 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5fe0055340 con 0x7f5ff41f36d0 2024-03-29T08:27:36.873 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:27:36.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.871+0000 7f5ffbe00640 1 -- 172.21.15.46:0/1337266771 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5fd4066690 msgr2=0x7f5fd4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:36.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.871+0000 7f5ffbe00640 1 --2- 172.21.15.46:0/1337266771 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5fd4066690 0x7f5fd4068b50 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009c60 tx=0x7f5fe8008040 comp rx=0 tx=0).stop 2024-03-29T08:27:36.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.871+0000 7f5ffbe00640 1 -- 172.21.15.46:0/1337266771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ff41f36d0 msgr2=0x7f5ff40ac3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:36.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.871+0000 7f5ffbe00640 1 --2- 172.21.15.46:0/1337266771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ff41f36d0 0x7f5ff40ac3b0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0011960 tx=0x7f5fe0011e20 comp rx=0 tx=0).stop 2024-03-29T08:27:36.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.871+0000 7f5ffbe00640 1 -- 172.21.15.46:0/1337266771 shutdown_connections 2024-03-29T08:27:36.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.871+0000 7f5ffbe00640 1 --2- 172.21.15.46:0/1337266771 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5fd4066690 0x7f5fd4068b50 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.871+0000 7f5ffbe00640 1 --2- 172.21.15.46:0/1337266771 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5ff40ac8f0 0x7f5ff40cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.871+0000 7f5ffbe00640 1 --2- 172.21.15.46:0/1337266771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ff41f36d0 0x7f5ff40ac3b0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.871+0000 7f5ffbe00640 1 --2- 172.21.15.46:0/1337266771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fdc004d10 0x7f5ff4158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:36.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.871+0000 7f5ffbe00640 1 -- 172.21.15.46:0/1337266771 >> 172.21.15.46:0/1337266771 conn(0x7f5ff406a390 msgr2=0x7f5ff415beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:36.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.871+0000 7f5ffbe00640 1 -- 172.21.15.46:0/1337266771 shutdown_connections 2024-03-29T08:27:36.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:36.871+0000 7f5ffbe00640 1 -- 172.21.15.46:0/1337266771 wait complete. 2024-03-29T08:27:36.894 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:27:37.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.035+0000 7f2f8be00640 1 Processor -- start 2024-03-29T08:27:37.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.035+0000 7f2f8be00640 1 -- start start 2024-03-29T08:27:37.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.039+0000 7f2f8be00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2f8c0ccf60 0x7f2f8c0cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.039+0000 7f2f8be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f8c0cb380 0x7f2f8c0cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.039+0000 7f2f8be00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f8c06ff00 con 0x7f2f8c15a9f0 2024-03-29T08:27:37.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.039+0000 7f2f8be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f8c0cd910 con 0x7f2f8c0cb380 2024-03-29T08:27:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.039+0000 7f2f8be00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f8c0cf590 con 0x7f2f8c0ccf60 2024-03-29T08:27:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.039+0000 7f2f8b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f8c0cb380 0x7f2f8c0cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.039+0000 7f2f8b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2f8c0cb380 0x7f2f8c0cb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51476/0 (socket says 172.21.15.46:51476) 2024-03-29T08:27:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.039+0000 7f2f8b400640 1 -- 172.21.15.46:0/639986453 learned_addr learned my addr 172.21.15.46:0/639986453 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.039+0000 7f2f8b400640 1 -- 172.21.15.46:0/639986453 >> v2:172.21.15.73:3300/0 conn(0x7f2f8c0ccf60 msgr2=0x7f2f8c0cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.039+0000 7f2f82000640 1 --2- 172.21.15.46:0/639986453 >> v2:172.21.15.73:3300/0 conn(0x7f2f8c0ccf60 0x7f2f8c0cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.039+0000 7f2f8b400640 1 --2- 172.21.15.46:0/639986453 >> v2:172.21.15.73:3300/0 conn(0x7f2f8c0ccf60 0x7f2f8c0cd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.039+0000 7f2f8b400640 1 -- 172.21.15.46:0/639986453 >> v1:172.21.15.73:6789/0 conn(0x7f2f8c15a9f0 legacy=0x7f2f8c0ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.039+0000 7f2f8b400640 1 -- 172.21.15.46:0/639986453 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8c06e130 con 0x7f2f8c0cb380 2024-03-29T08:27:37.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.039+0000 7f2f82000640 1 --2- 172.21.15.46:0/639986453 >> v2:172.21.15.73:3300/0 conn(0x7f2f8c0ccf60 0x7f2f8c0cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:37.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.039+0000 7f2f8b400640 1 --2- 172.21.15.46:0/639986453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c0cb380 0x7f2f8c0cb7b0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c066a10 tx=0x7f2f8401f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3057b060d8d6d26 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:37.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8a000640 1 -- 172.21.15.46:0/639986453 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f840af400 con 0x7f2f8c0cb380 2024-03-29T08:27:37.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8a000640 1 -- 172.21.15.46:0/639986453 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f840ab050 con 0x7f2f8c0cb380 2024-03-29T08:27:37.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8a000640 1 -- 172.21.15.46:0/639986453 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f840af780 con 0x7f2f8c0cb380 2024-03-29T08:27:37.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8be00640 1 -- 172.21.15.46:0/639986453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c0cb380 msgr2=0x7f2f8c0cb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8be00640 1 --2- 172.21.15.46:0/639986453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c0cb380 0x7f2f8c0cb7b0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c066a10 tx=0x7f2f8401f1a0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8be00640 1 -- 172.21.15.46:0/639986453 shutdown_connections 2024-03-29T08:27:37.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8be00640 1 --2- 172.21.15.46:0/639986453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c0cb380 0x7f2f8c0cb7b0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8be00640 1 --2- 172.21.15.46:0/639986453 >> v2:172.21.15.73:3300/0 conn(0x7f2f8c0ccf60 0x7f2f8c0cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8be00640 1 -- 172.21.15.46:0/639986453 >> 172.21.15.46:0/639986453 conn(0x7f2f8c0d0550 msgr2=0x7f2f8c0d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:37.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8be00640 1 -- 172.21.15.46:0/639986453 shutdown_connections 2024-03-29T08:27:37.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8be00640 1 -- 172.21.15.46:0/639986453 wait complete. 2024-03-29T08:27:37.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8be00640 1 Processor -- start 2024-03-29T08:27:37.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8be00640 1 -- start start 2024-03-29T08:27:37.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f8c0cb380 0x7f2f8c14f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f8c0ccf60 0x7f2f8c14f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c15a9f0 0x7f2f8c1f2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c150320 con 0x7f2f8c15a9f0 2024-03-29T08:27:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c0c69f0 con 0x7f2f8c0cb380 2024-03-29T08:27:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8be00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c0c6cd0 con 0x7f2f8c0ccf60 2024-03-29T08:27:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c15a9f0 0x7f2f8c1f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c15a9f0 0x7f2f8c1f2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51478/0 (socket says 172.21.15.46:51478) 2024-03-29T08:27:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8b400640 1 -- 172.21.15.46:0/2666089961 learned_addr learned my addr 172.21.15.46:0/2666089961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f8aa00640 1 --2- 172.21.15.46:0/2666089961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f8c0cb380 0x7f2f8c14f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.043+0000 7f2f82000640 1 --2- 172.21.15.46:0/2666089961 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f8c0ccf60 0x7f2f8c14f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.047+0000 7f2f8b400640 1 -- 172.21.15.46:0/2666089961 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f8c0ccf60 msgr2=0x7f2f8c14f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.047+0000 7f2f8b400640 1 --2- 172.21.15.46:0/2666089961 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f8c0ccf60 0x7f2f8c14f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.047+0000 7f2f8b400640 1 -- 172.21.15.46:0/2666089961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f8c0cb380 msgr2=0x7f2f8c14f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.047+0000 7f2f8b400640 1 --2- 172.21.15.46:0/2666089961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f8c0cb380 0x7f2f8c14f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.047+0000 7f2f8b400640 1 -- 172.21.15.46:0/2666089961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f84009b30 con 0x7f2f8c15a9f0 2024-03-29T08:27:37.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.047+0000 7f2f8b400640 1 --2- 172.21.15.46:0/2666089961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c15a9f0 0x7f2f8c1f2790 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f2f8401d850 tx=0x7f2f8401bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.047+0000 7f2f83e00640 1 -- 172.21.15.46:0/2666089961 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f840afe20 con 0x7f2f8c15a9f0 2024-03-29T08:27:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.047+0000 7f2f83e00640 1 -- 172.21.15.46:0/2666089961 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f840b9410 con 0x7f2f8c15a9f0 2024-03-29T08:27:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.047+0000 7f2f8be00640 1 -- 172.21.15.46:0/2666089961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8c1f3c00 con 0x7f2f8c15a9f0 2024-03-29T08:27:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.047+0000 7f2f83e00640 1 -- 172.21.15.46:0/2666089961 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f840b97c0 con 0x7f2f8c15a9f0 2024-03-29T08:27:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.047+0000 7f2f8be00640 1 -- 172.21.15.46:0/2666089961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8c1f41c0 con 0x7f2f8c15a9f0 2024-03-29T08:27:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.047+0000 7f2f8be00640 1 -- 172.21.15.46:0/2666089961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8c1f43b0 con 0x7f2f8c15a9f0 2024-03-29T08:27:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.047+0000 7f2f83e00640 1 -- 172.21.15.46:0/2666089961 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2f840b6050 con 0x7f2f8c15a9f0 2024-03-29T08:27:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.047+0000 7f2f83e00640 1 --2- 172.21.15.46:0/2666089961 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f68066690 0x7f2f68068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.047+0000 7f2f83e00640 1 -- 172.21.15.46:0/2666089961 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f84122310 con 0x7f2f8c15a9f0 2024-03-29T08:27:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.047+0000 7f2f8aa00640 1 --2- 172.21.15.46:0/2666089961 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f68066690 0x7f2f68068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.051+0000 7f2f8aa00640 1 --2- 172.21.15.46:0/2666089961 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f68066690 0x7f2f68068b50 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0098e0 tx=0x7f2f7c009060 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:37.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.051+0000 7f2f83e00640 1 -- 172.21.15.46:0/2666089961 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2f840ef840 con 0x7f2f8c15a9f0 2024-03-29T08:27:37.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.215+0000 7f2f8be00640 1 -- 172.21.15.46:0/2666089961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2f8c04e040 con 0x7f2f8c15a9f0 2024-03-29T08:27:37.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.215+0000 7f2f83e00640 1 -- 172.21.15.46:0/2666089961 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2f840f2e60 con 0x7f2f8c15a9f0 2024-03-29T08:27:37.216 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:27:37.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.215+0000 7f2f8be00640 1 -- 172.21.15.46:0/2666089961 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f68066690 msgr2=0x7f2f68068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.215+0000 7f2f8be00640 1 --2- 172.21.15.46:0/2666089961 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f68066690 0x7f2f68068b50 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c0098e0 tx=0x7f2f7c009060 comp rx=0 tx=0).stop 2024-03-29T08:27:37.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.215+0000 7f2f8be00640 1 -- 172.21.15.46:0/2666089961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c15a9f0 msgr2=0x7f2f8c1f2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.215+0000 7f2f8be00640 1 --2- 172.21.15.46:0/2666089961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c15a9f0 0x7f2f8c1f2790 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f2f8401d850 tx=0x7f2f8401bf40 comp rx=0 tx=0).stop 2024-03-29T08:27:37.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.215+0000 7f2f8be00640 1 -- 172.21.15.46:0/2666089961 shutdown_connections 2024-03-29T08:27:37.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.215+0000 7f2f8be00640 1 --2- 172.21.15.46:0/2666089961 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2f68066690 0x7f2f68068b50 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.215+0000 7f2f8be00640 1 --2- 172.21.15.46:0/2666089961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f8c15a9f0 0x7f2f8c1f2790 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.215+0000 7f2f8be00640 1 --2- 172.21.15.46:0/2666089961 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2f8c0ccf60 0x7f2f8c14f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.215+0000 7f2f8be00640 1 --2- 172.21.15.46:0/2666089961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f8c0cb380 0x7f2f8c14f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.215+0000 7f2f8be00640 1 -- 172.21.15.46:0/2666089961 >> 172.21.15.46:0/2666089961 conn(0x7f2f8c0d0550 msgr2=0x7f2f8c0ad530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.215+0000 7f2f8be00640 1 -- 172.21.15.46:0/2666089961 shutdown_connections 2024-03-29T08:27:37.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.215+0000 7f2f8be00640 1 -- 172.21.15.46:0/2666089961 wait complete. 2024-03-29T08:27:37.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:27:37.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.375+0000 7faea6800640 1 Processor -- start 2024-03-29T08:27:37.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.375+0000 7faea6800640 1 -- start start 2024-03-29T08:27:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7faea6800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faea00cee30 0x7faea00b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7faea6800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea00cc250 con 0x7faea015c9d0 2024-03-29T08:27:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7faea6800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea015cdb0 con 0x7faea00b0440 2024-03-29T08:27:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7faea6800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea00cb640 con 0x7faea00cee30 2024-03-29T08:27:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7faea5400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7faea015c9d0 0x7faea00cc6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45008/0 (socket says 172.21.15.46:45008) 2024-03-29T08:27:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7faea5400640 1 -- 172.21.15.46:0/2992259784 learned_addr learned my addr 172.21.15.46:0/2992259784 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7faea5e00640 1 --2- 172.21.15.46:0/2992259784 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faea00cee30 0x7faea00b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7fae9fe00640 1 -- 172.21.15.46:0/2992259784 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923333751 0 0) 0x7faea00cc250 con 0x7faea015c9d0 2024-03-29T08:27:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7fae9fe00640 1 -- 172.21.15.46:0/2992259784 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae88003530 con 0x7faea015c9d0 2024-03-29T08:27:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7fae9fe00640 1 -- 172.21.15.46:0/2992259784 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1186122008 0 0) 0x7fae88003530 con 0x7faea015c9d0 2024-03-29T08:27:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7fae9fe00640 1 -- 172.21.15.46:0/2992259784 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faea00cee30 msgr2=0x7faea00b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:37.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7fae9fe00640 1 --2- 172.21.15.46:0/2992259784 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faea00cee30 0x7faea00b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7fae9fe00640 1 -- 172.21.15.46:0/2992259784 >> v1:172.21.15.73:6789/0 conn(0x7faea00b0440 legacy=0x7faea00ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7fae9fe00640 1 -- 172.21.15.46:0/2992259784 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea00cb920 con 0x7faea015c9d0 2024-03-29T08:27:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7fae9fe00640 1 -- 172.21.15.46:0/2992259784 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fae94001680 con 0x7faea015c9d0 2024-03-29T08:27:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7fae9fe00640 1 -- 172.21.15.46:0/2992259784 >> v1:172.21.15.46:6789/0 conn(0x7faea015c9d0 legacy=0x7faea00cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7fae9fe00640 1 --2- 172.21.15.46:0/2992259784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae880039b0 0x7fae88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7fae9fe00640 1 --2- 172.21.15.46:0/2992259784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae880047d0 0x7fae88004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7fae9fe00640 1 --2- 172.21.15.46:0/2992259784 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faea00cee30 0x7fae880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7fae9fe00640 1 -- 172.21.15.46:0/2992259784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea00cb640 con 0x7fae880039b0 2024-03-29T08:27:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7fae9fe00640 1 -- 172.21.15.46:0/2992259784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8800bac0 con 0x7fae880047d0 2024-03-29T08:27:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7fae9fe00640 1 -- 172.21.15.46:0/2992259784 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8800bc80 con 0x7faea00cee30 2024-03-29T08:27:37.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7faea5400640 1 --2- 172.21.15.46:0/2992259784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae880039b0 0x7fae88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7faea4a00640 1 --2- 172.21.15.46:0/2992259784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae880047d0 0x7fae88004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7faea5e00640 1 --2- 172.21.15.46:0/2992259784 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faea00cee30 0x7fae880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7faea5400640 1 -- 172.21.15.46:0/2992259784 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faea00cee30 msgr2=0x7fae880051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7faea5400640 1 --2- 172.21.15.46:0/2992259784 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faea00cee30 0x7fae880051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7faea5400640 1 -- 172.21.15.46:0/2992259784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae880047d0 msgr2=0x7fae88004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7faea5400640 1 --2- 172.21.15.46:0/2992259784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae880047d0 0x7fae88004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.379+0000 7faea5400640 1 -- 172.21.15.46:0/2992259784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae88003530 con 0x7fae880039b0 2024-03-29T08:27:37.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.383+0000 7faea5400640 1 --2- 172.21.15.46:0/2992259784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae880039b0 0x7fae88003dd0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fae9400c240 tx=0x7fae9401cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34235b7b2a3329b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:37.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.383+0000 7fae9fe00640 1 -- 172.21.15.46:0/2992259784 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae940a5b10 con 0x7fae880039b0 2024-03-29T08:27:37.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.383+0000 7fae9fe00640 1 -- 172.21.15.46:0/2992259784 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae940a5c70 con 0x7fae880039b0 2024-03-29T08:27:37.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.383+0000 7fae9fe00640 1 -- 172.21.15.46:0/2992259784 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94005b20 con 0x7fae880039b0 2024-03-29T08:27:37.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.383+0000 7faea6800640 1 -- 172.21.15.46:0/2992259784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae880039b0 msgr2=0x7fae88003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.383+0000 7faea6800640 1 --2- 172.21.15.46:0/2992259784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae880039b0 0x7fae88003dd0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fae9400c240 tx=0x7fae9401cc30 comp rx=0 tx=0).stop 2024-03-29T08:27:37.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.383+0000 7faea5e00640 1 -- 172.21.15.46:0/2992259784 reap_dead start 2024-03-29T08:27:37.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.383+0000 7faea6800640 1 -- 172.21.15.46:0/2992259784 shutdown_connections 2024-03-29T08:27:37.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.383+0000 7faea6800640 1 -- 172.21.15.46:0/2992259784 >> 172.21.15.46:0/2992259784 conn(0x7faea0098a40 msgr2=0x7faea00cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:37.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.383+0000 7faea6800640 1 -- 172.21.15.46:0/2992259784 shutdown_connections 2024-03-29T08:27:37.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.383+0000 7faea6800640 1 -- 172.21.15.46:0/2992259784 wait complete. 2024-03-29T08:27:37.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.383+0000 7faea6800640 1 Processor -- start 2024-03-29T08:27:37.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.383+0000 7faea6800640 1 -- start start 2024-03-29T08:27:37.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae880039b0 0x7faea01526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea6800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faea0153450 0x7faea00c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faea00b2230 0x7faea00b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea6800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea0156400 con 0x7fae880039b0 2024-03-29T08:27:37.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea6800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea00b2bf0 con 0x7faea00b2230 2024-03-29T08:27:37.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea6800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea0153200 con 0x7faea0153450 2024-03-29T08:27:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae880039b0 0x7faea01526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae880039b0 0x7faea01526c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51502/0 (socket says 172.21.15.46:51502) 2024-03-29T08:27:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea5400640 1 -- 172.21.15.46:0/2342637032 learned_addr learned my addr 172.21.15.46:0/2342637032 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea4a00640 1 --2- 172.21.15.46:0/2342637032 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faea0153450 0x7faea00c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea5e00640 1 --2- 172.21.15.46:0/2342637032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faea00b2230 0x7faea00b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea5400640 1 -- 172.21.15.46:0/2342637032 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faea0153450 msgr2=0x7faea00c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea5400640 1 --2- 172.21.15.46:0/2342637032 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faea0153450 0x7faea00c6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea5400640 1 -- 172.21.15.46:0/2342637032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faea00b2230 msgr2=0x7faea00b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea5400640 1 --2- 172.21.15.46:0/2342637032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faea00b2230 0x7faea00b26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea5400640 1 -- 172.21.15.46:0/2342637032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea0152f20 con 0x7fae880039b0 2024-03-29T08:27:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea5400640 1 --2- 172.21.15.46:0/2342637032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae880039b0 0x7faea01526c0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fae9400c240 tx=0x7fae88005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7fae9d600640 1 -- 172.21.15.46:0/2342637032 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae940b4070 con 0x7fae880039b0 2024-03-29T08:27:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7fae9d600640 1 -- 172.21.15.46:0/2342637032 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae94000ee0 con 0x7fae880039b0 2024-03-29T08:27:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7fae9d600640 1 -- 172.21.15.46:0/2342637032 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae940a5b10 con 0x7fae880039b0 2024-03-29T08:27:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea6800640 1 -- 172.21.15.46:0/2342637032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea0155e40 con 0x7fae880039b0 2024-03-29T08:27:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea6800640 1 -- 172.21.15.46:0/2342637032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea00c3ac0 con 0x7fae880039b0 2024-03-29T08:27:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7faea6800640 1 -- 172.21.15.46:0/2342637032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea00c3920 con 0x7fae880039b0 2024-03-29T08:27:37.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.387+0000 7fae9d600640 1 -- 172.21.15.46:0/2342637032 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fae94001440 con 0x7fae880039b0 2024-03-29T08:27:37.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.391+0000 7fae9d600640 1 --2- 172.21.15.46:0/2342637032 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fae80066690 0x7fae80068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.391+0000 7fae9d600640 1 -- 172.21.15.46:0/2342637032 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae9411ddd0 con 0x7fae880039b0 2024-03-29T08:27:37.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.391+0000 7faea4a00640 1 --2- 172.21.15.46:0/2342637032 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fae80066690 0x7fae80068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.391+0000 7faea4a00640 1 --2- 172.21.15.46:0/2342637032 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fae80066690 0x7fae80068b50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7faea00b52b0 tx=0x7fae8c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:37.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.395+0000 7fae9d600640 1 -- 172.21.15.46:0/2342637032 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7faea00c3920 con 0x7fae880039b0 2024-03-29T08:27:37.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.551+0000 7faea6800640 1 -- 172.21.15.46:0/2342637032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faea00c3920 con 0x7fae880039b0 2024-03-29T08:27:37.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.555+0000 7fae9d600640 1 -- 172.21.15.46:0/2342637032 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7faea00c3920 con 0x7fae880039b0 2024-03-29T08:27:37.556 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:27:37.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.555+0000 7faea6800640 1 -- 172.21.15.46:0/2342637032 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fae80066690 msgr2=0x7fae80068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.555+0000 7faea6800640 1 --2- 172.21.15.46:0/2342637032 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fae80066690 0x7fae80068b50 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7faea00b52b0 tx=0x7fae8c009040 comp rx=0 tx=0).stop 2024-03-29T08:27:37.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.555+0000 7faea6800640 1 -- 172.21.15.46:0/2342637032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae880039b0 msgr2=0x7faea01526c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.555+0000 7faea6800640 1 --2- 172.21.15.46:0/2342637032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae880039b0 0x7faea01526c0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fae9400c240 tx=0x7fae88005800 comp rx=0 tx=0).stop 2024-03-29T08:27:37.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.555+0000 7faea6800640 1 -- 172.21.15.46:0/2342637032 shutdown_connections 2024-03-29T08:27:37.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.555+0000 7faea6800640 1 --2- 172.21.15.46:0/2342637032 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fae80066690 0x7fae80068b50 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.555+0000 7faea6800640 1 --2- 172.21.15.46:0/2342637032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faea00b2230 0x7faea00b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.555+0000 7faea6800640 1 --2- 172.21.15.46:0/2342637032 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faea0153450 0x7faea00c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.555+0000 7faea6800640 1 --2- 172.21.15.46:0/2342637032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae880039b0 0x7faea01526c0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.555+0000 7faea6800640 1 -- 172.21.15.46:0/2342637032 >> 172.21.15.46:0/2342637032 conn(0x7faea0098a40 msgr2=0x7faea0156ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:37.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.555+0000 7faea6800640 1 -- 172.21.15.46:0/2342637032 shutdown_connections 2024-03-29T08:27:37.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.555+0000 7faea6800640 1 -- 172.21.15.46:0/2342637032 wait complete. 2024-03-29T08:27:37.577 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:27:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.715+0000 7f2729a00640 1 Processor -- start 2024-03-29T08:27:37.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.715+0000 7f2729a00640 1 -- start start 2024-03-29T08:27:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.715+0000 7f2729a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f272406dbb0 0x7f272415d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.715+0000 7f2729a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f272415d940 0x7f27240b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.715+0000 7f2729a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2724070c70 con 0x7f2724142ab0 2024-03-29T08:27:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.715+0000 7f2729a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2724070340 con 0x7f272415d940 2024-03-29T08:27:37.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.715+0000 7f2729a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2724070620 con 0x7f272406dbb0 2024-03-29T08:27:37.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.719+0000 7f271be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f272406dbb0 0x7f272415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.719+0000 7f2723400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f272415d940 0x7f27240b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.719+0000 7f271be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f272406dbb0 0x7f272415d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:50106/0 (socket says 172.21.15.46:50106) 2024-03-29T08:27:37.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.719+0000 7f271be00640 1 -- 172.21.15.46:0/957384211 learned_addr learned my addr 172.21.15.46:0/957384211 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:37.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.719+0000 7f271be00640 1 -- 172.21.15.46:0/957384211 >> v2:172.21.15.73:3300/0 conn(0x7f272415d940 msgr2=0x7f27240b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:37.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.719+0000 7f271be00640 1 --2- 172.21.15.46:0/957384211 >> v2:172.21.15.73:3300/0 conn(0x7f272415d940 0x7f27240b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.719+0000 7f271be00640 1 -- 172.21.15.46:0/957384211 >> v1:172.21.15.73:6789/0 conn(0x7f2724142ab0 legacy=0x7f272406d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.719+0000 7f271be00640 1 -- 172.21.15.46:0/957384211 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2724070900 con 0x7f272406dbb0 2024-03-29T08:27:37.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.719+0000 7f271be00640 1 --2- 172.21.15.46:0/957384211 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f272406dbb0 0x7f272415d400 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f270c009870 tx=0x7f270c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9feeaad15c6904a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:37.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.719+0000 7f2722000640 1 -- 172.21.15.46:0/957384211 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c0ab600 con 0x7f272406dbb0 2024-03-29T08:27:37.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.719+0000 7f2722000640 1 -- 172.21.15.46:0/957384211 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270c0a7050 con 0x7f272406dbb0 2024-03-29T08:27:37.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.719+0000 7f2722000640 1 -- 172.21.15.46:0/957384211 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c0ab980 con 0x7f272406dbb0 2024-03-29T08:27:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2729a00640 1 -- 172.21.15.46:0/957384211 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f272406dbb0 msgr2=0x7f272415d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2729a00640 1 --2- 172.21.15.46:0/957384211 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f272406dbb0 0x7f272415d400 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f270c009870 tx=0x7f270c01c560 comp rx=0 tx=0).stop 2024-03-29T08:27:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2729a00640 1 -- 172.21.15.46:0/957384211 shutdown_connections 2024-03-29T08:27:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2729a00640 1 --2- 172.21.15.46:0/957384211 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f272406dbb0 0x7f272415d400 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2729a00640 1 --2- 172.21.15.46:0/957384211 >> v2:172.21.15.73:3300/0 conn(0x7f272415d940 0x7f27240b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2729a00640 1 -- 172.21.15.46:0/957384211 >> 172.21.15.46:0/957384211 conn(0x7f2724066080 msgr2=0x7f2724069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2729a00640 1 -- 172.21.15.46:0/957384211 shutdown_connections 2024-03-29T08:27:37.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2729a00640 1 -- 172.21.15.46:0/957384211 wait complete. 2024-03-29T08:27:37.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2729a00640 1 Processor -- start 2024-03-29T08:27:37.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2729a00640 1 -- start start 2024-03-29T08:27:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2729a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f272406dbb0 0x7f2724138f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2729a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2724142ab0 0x7f2724159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2729a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f272415d940 0x7f272413ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2729a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2724164de0 con 0x7f2724142ab0 2024-03-29T08:27:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2729a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2724166020 con 0x7f272415d940 2024-03-29T08:27:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2729a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2724158b00 con 0x7f272406dbb0 2024-03-29T08:27:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f271be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2724142ab0 0x7f2724159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f271be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2724142ab0 0x7f2724159d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51518/0 (socket says 172.21.15.46:51518) 2024-03-29T08:27:37.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2723400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f272415d940 0x7f272413ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f271be00640 1 -- 172.21.15.46:0/2514627853 learned_addr learned my addr 172.21.15.46:0/2514627853 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:37.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2722a00640 1 --2- 172.21.15.46:0/2514627853 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f272406dbb0 0x7f2724138f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f271be00640 1 -- 172.21.15.46:0/2514627853 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f272406dbb0 msgr2=0x7f2724138f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f271be00640 1 --2- 172.21.15.46:0/2514627853 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f272406dbb0 0x7f2724138f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f271be00640 1 -- 172.21.15.46:0/2514627853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f272415d940 msgr2=0x7f272413ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f271be00640 1 --2- 172.21.15.46:0/2514627853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f272415d940 0x7f272413ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f271be00640 1 -- 172.21.15.46:0/2514627853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270c0094d0 con 0x7f2724142ab0 2024-03-29T08:27:37.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.723+0000 7f2723400640 1 --2- 172.21.15.46:0/2514627853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f272415d940 0x7f272413ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:27:37.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.727+0000 7f271be00640 1 --2- 172.21.15.46:0/2514627853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2724142ab0 0x7f2724159d10 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f270c01a190 tx=0x7f270c019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:37.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.727+0000 7f271aa00640 1 -- 172.21.15.46:0/2514627853 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c0b4420 con 0x7f2724142ab0 2024-03-29T08:27:37.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.727+0000 7f271aa00640 1 -- 172.21.15.46:0/2514627853 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270c0b4580 con 0x7f2724142ab0 2024-03-29T08:27:37.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.727+0000 7f2729a00640 1 -- 172.21.15.46:0/2514627853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2724158de0 con 0x7f2724142ab0 2024-03-29T08:27:37.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.727+0000 7f271aa00640 1 -- 172.21.15.46:0/2514627853 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c0b48f0 con 0x7f2724142ab0 2024-03-29T08:27:37.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.727+0000 7f2729a00640 1 -- 172.21.15.46:0/2514627853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27241662e0 con 0x7f2724142ab0 2024-03-29T08:27:37.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.727+0000 7f2729a00640 1 -- 172.21.15.46:0/2514627853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27240ce3b0 con 0x7f2724142ab0 2024-03-29T08:27:37.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.727+0000 7f271aa00640 1 -- 172.21.15.46:0/2514627853 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f270c0b2070 con 0x7f2724142ab0 2024-03-29T08:27:37.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.727+0000 7f271aa00640 1 --2- 172.21.15.46:0/2514627853 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2700066690 0x7f2700068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:37.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.727+0000 7f271aa00640 1 -- 172.21.15.46:0/2514627853 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f270c11d6b0 con 0x7f2724142ab0 2024-03-29T08:27:37.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.727+0000 7f2722a00640 1 --2- 172.21.15.46:0/2514627853 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2700066690 0x7f2700068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:37.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.731+0000 7f2722a00640 1 --2- 172.21.15.46:0/2514627853 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2700066690 0x7f2700068b50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2714007fd0 tx=0x7f2714009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:37.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.731+0000 7f271aa00640 1 -- 172.21.15.46:0/2514627853 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f270c0eaa30 con 0x7f2724142ab0 2024-03-29T08:27:37.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.907+0000 7f2729a00640 1 -- 172.21.15.46:0/2514627853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f27241664d0 con 0x7f2724142ab0 2024-03-29T08:27:37.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.907+0000 7f271aa00640 1 -- 172.21.15.46:0/2514627853 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) v1 ==== 80+0+0 (secure 0 0 0) 0x7f270c0ee050 con 0x7f2724142ab0 2024-03-29T08:27:37.909 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:27:37.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.907+0000 7f2729a00640 1 -- 172.21.15.46:0/2514627853 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2700066690 msgr2=0x7f2700068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.907+0000 7f2729a00640 1 --2- 172.21.15.46:0/2514627853 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2700066690 0x7f2700068b50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2714007fd0 tx=0x7f2714009040 comp rx=0 tx=0).stop 2024-03-29T08:27:37.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.907+0000 7f2729a00640 1 -- 172.21.15.46:0/2514627853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2724142ab0 msgr2=0x7f2724159d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:37.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.907+0000 7f2729a00640 1 --2- 172.21.15.46:0/2514627853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2724142ab0 0x7f2724159d10 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f270c01a190 tx=0x7f270c019300 comp rx=0 tx=0).stop 2024-03-29T08:27:37.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.907+0000 7f2729a00640 1 -- 172.21.15.46:0/2514627853 shutdown_connections 2024-03-29T08:27:37.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.907+0000 7f2729a00640 1 --2- 172.21.15.46:0/2514627853 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2700066690 0x7f2700068b50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.907+0000 7f2729a00640 1 --2- 172.21.15.46:0/2514627853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f272415d940 0x7f272413ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.907+0000 7f2729a00640 1 --2- 172.21.15.46:0/2514627853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2724142ab0 0x7f2724159d10 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.907+0000 7f2729a00640 1 --2- 172.21.15.46:0/2514627853 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f272406dbb0 0x7f2724138f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:37.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.907+0000 7f2729a00640 1 -- 172.21.15.46:0/2514627853 >> 172.21.15.46:0/2514627853 conn(0x7f2724066080 msgr2=0x7f27240b0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:37.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.907+0000 7f2729a00640 1 -- 172.21.15.46:0/2514627853 shutdown_connections 2024-03-29T08:27:37.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:37.907+0000 7f2729a00640 1 -- 172.21.15.46:0/2514627853 wait complete. 2024-03-29T08:27:37.930 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-03-29T08:27:37.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:37.930 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T08:27:37.949 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[48553]: fuse finished with error 0 and tester_r 0 2024-03-29T08:27:37.955 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:27:39.534 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:27:39.539 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:27:44.056 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:27:44.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:44.057 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:27:44.067 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:27:44.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.259+0000 7f4058400640 1 Processor -- start 2024-03-29T08:27:44.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.259+0000 7f4058400640 1 -- start start 2024-03-29T08:27:44.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.263+0000 7f4058400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f405015ada0 0x7f40500ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.263+0000 7f4058400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f40500ce950 0x7f405006c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.263+0000 7f4058400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40500cb510 con 0x7f4050144210 2024-03-29T08:27:44.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.263+0000 7f4058400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40500cede0 con 0x7f40500ce950 2024-03-29T08:27:44.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.263+0000 7f4058400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4050070580 con 0x7f405015ada0 2024-03-29T08:27:44.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.263+0000 7f4056400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f40500ce950 0x7f405006c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.263+0000 7f4056400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f40500ce950 0x7f405006c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51534/0 (socket says 172.21.15.46:51534) 2024-03-29T08:27:44.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.263+0000 7f4056400640 1 -- 172.21.15.46:0/375941021 learned_addr learned my addr 172.21.15.46:0/375941021 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:44.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.263+0000 7f4055000640 1 --2- 172.21.15.46:0/375941021 >> v2:172.21.15.73:3300/0 conn(0x7f405015ada0 0x7f40500ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.263+0000 7f4056400640 1 -- 172.21.15.46:0/375941021 >> v2:172.21.15.73:3300/0 conn(0x7f405015ada0 msgr2=0x7f40500ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:44.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.263+0000 7f4056400640 1 --2- 172.21.15.46:0/375941021 >> v2:172.21.15.73:3300/0 conn(0x7f405015ada0 0x7f40500ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.263+0000 7f4056400640 1 -- 172.21.15.46:0/375941021 >> v1:172.21.15.73:6789/0 conn(0x7f4050144210 legacy=0x7f40500cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.263+0000 7f4056400640 1 -- 172.21.15.46:0/375941021 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4050070860 con 0x7f40500ce950 2024-03-29T08:27:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f4056400640 1 --2- 172.21.15.46:0/375941021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40500ce950 0x7f405006c950 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f405006e410 tx=0x7f404801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1a533016305cad server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f404fe00640 1 -- 172.21.15.46:0/375941021 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40480a57a0 con 0x7f40500ce950 2024-03-29T08:27:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f404fe00640 1 -- 172.21.15.46:0/375941021 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40480a5900 con 0x7f40500ce950 2024-03-29T08:27:44.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f404fe00640 1 -- 172.21.15.46:0/375941021 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40480a5c90 con 0x7f40500ce950 2024-03-29T08:27:44.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f4058400640 1 -- 172.21.15.46:0/375941021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40500ce950 msgr2=0x7f405006c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f4058400640 1 --2- 172.21.15.46:0/375941021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40500ce950 0x7f405006c950 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f405006e410 tx=0x7f404801c9f0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f4058400640 1 -- 172.21.15.46:0/375941021 shutdown_connections 2024-03-29T08:27:44.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f4058400640 1 --2- 172.21.15.46:0/375941021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40500ce950 0x7f405006c950 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f4058400640 1 --2- 172.21.15.46:0/375941021 >> v2:172.21.15.73:3300/0 conn(0x7f405015ada0 0x7f40500ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f4058400640 1 -- 172.21.15.46:0/375941021 >> 172.21.15.46:0/375941021 conn(0x7f4050066110 msgr2=0x7f4050069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:44.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f4058400640 1 -- 172.21.15.46:0/375941021 shutdown_connections 2024-03-29T08:27:44.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f4058400640 1 -- 172.21.15.46:0/375941021 wait complete. 2024-03-29T08:27:44.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f4058400640 1 Processor -- start 2024-03-29T08:27:44.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f4058400640 1 -- start start 2024-03-29T08:27:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f4058400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f40500ce950 0x7f4050157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f4058400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4050144210 0x7f4050139650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f4058400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f405015ada0 0x7f4050139b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f4058400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4050158000 con 0x7f4050144210 2024-03-29T08:27:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f4058400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405013f1c0 con 0x7f405015ada0 2024-03-29T08:27:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.267+0000 7f4058400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405013f4a0 con 0x7f40500ce950 2024-03-29T08:27:44.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f4055000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4050144210 0x7f4050139650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f4055000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4050144210 0x7f4050139650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51540/0 (socket says 172.21.15.46:51540) 2024-03-29T08:27:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f4055000640 1 -- 172.21.15.46:0/3396848646 learned_addr learned my addr 172.21.15.46:0/3396848646 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f4056400640 1 --2- 172.21.15.46:0/3396848646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f405015ada0 0x7f4050139b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f4055a00640 1 --2- 172.21.15.46:0/3396848646 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f40500ce950 0x7f4050157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f4055000640 1 -- 172.21.15.46:0/3396848646 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f40500ce950 msgr2=0x7f4050157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f4055000640 1 --2- 172.21.15.46:0/3396848646 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f40500ce950 0x7f4050157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f4055000640 1 -- 172.21.15.46:0/3396848646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f405015ada0 msgr2=0x7f4050139b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f4055000640 1 --2- 172.21.15.46:0/3396848646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f405015ada0 0x7f4050139b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f4055000640 1 -- 172.21.15.46:0/3396848646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4048009b30 con 0x7f4050144210 2024-03-29T08:27:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f4056400640 1 --2- 172.21.15.46:0/3396848646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f405015ada0 0x7f4050139b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f4055a00640 1 --2- 172.21.15.46:0/3396848646 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f40500ce950 0x7f4050157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f4055000640 1 --2- 172.21.15.46:0/3396848646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4050144210 0x7f4050139650 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f403c00d5f0 tx=0x7f403c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:44.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f404d600640 1 -- 172.21.15.46:0/3396848646 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c0125f0 con 0x7f4050144210 2024-03-29T08:27:44.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f404d600640 1 -- 172.21.15.46:0/3396848646 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403c012750 con 0x7f4050144210 2024-03-29T08:27:44.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f404d600640 1 -- 172.21.15.46:0/3396848646 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f403c012ae0 con 0x7f4050144210 2024-03-29T08:27:44.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f4058400640 1 -- 172.21.15.46:0/3396848646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405013f780 con 0x7f4050144210 2024-03-29T08:27:44.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f4058400640 1 -- 172.21.15.46:0/3396848646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4050154900 con 0x7f4050144210 2024-03-29T08:27:44.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f4058400640 1 -- 172.21.15.46:0/3396848646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40500ce280 con 0x7f4050144210 2024-03-29T08:27:44.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f404d600640 1 -- 172.21.15.46:0/3396848646 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f403c012c40 con 0x7f4050144210 2024-03-29T08:27:44.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f404d600640 1 --2- 172.21.15.46:0/3396848646 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4030066690 0x7f4030068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.271+0000 7f404d600640 1 -- 172.21.15.46:0/3396848646 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f403c0847f0 con 0x7f4050144210 2024-03-29T08:27:44.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.275+0000 7f4055a00640 1 --2- 172.21.15.46:0/3396848646 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4030066690 0x7f4030068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.275+0000 7f4055a00640 1 --2- 172.21.15.46:0/3396848646 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4030066690 0x7f4030068b50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4044000f70 tx=0x7f4044009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:44.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.279+0000 7f404d600640 1 -- 172.21.15.46:0/3396848646 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f403c051a30 con 0x7f4050144210 2024-03-29T08:27:44.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.471+0000 7f4058400640 1 -- 172.21.15.46:0/3396848646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4050066c90 con 0x7f4050144210 2024-03-29T08:27:44.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.471+0000 7f404d600640 1 -- 172.21.15.46:0/3396848646 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4730 (secure 0 0 0) 0x7f403c055050 con 0x7f4050144210 2024-03-29T08:27:44.476 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:44.476 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":77,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4230030957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4230030957},{"type":"v1","addr":"172.21.15.73:6836","nonce":4230030957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/297138217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":297138217},{"type":"v1","addr":"172.21.15.73:6837","nonce":297138217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3856305332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3856305332},{"type":"v1","addr":"172.21.15.46:6833","nonce":3856305332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:27:18.583223+0000","modified":"2024-03-29T08:27:21.592418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7966},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7966":{"gid":7966,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/824184385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":824184385},{"type":"v1","addr":"172.21.15.46:6835","nonce":824184385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-29T08:27:44.476 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 77 2024-03-29T08:27:44.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.475+0000 7f4058400640 1 -- 172.21.15.46:0/3396848646 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4030066690 msgr2=0x7f4030068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.475+0000 7f4058400640 1 --2- 172.21.15.46:0/3396848646 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4030066690 0x7f4030068b50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4044000f70 tx=0x7f4044009040 comp rx=0 tx=0).stop 2024-03-29T08:27:44.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.475+0000 7f4058400640 1 -- 172.21.15.46:0/3396848646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4050144210 msgr2=0x7f4050139650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.475+0000 7f4058400640 1 --2- 172.21.15.46:0/3396848646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4050144210 0x7f4050139650 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f403c00d5f0 tx=0x7f403c00dab0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.475+0000 7f4058400640 1 -- 172.21.15.46:0/3396848646 shutdown_connections 2024-03-29T08:27:44.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.475+0000 7f4058400640 1 --2- 172.21.15.46:0/3396848646 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4030066690 0x7f4030068b50 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.475+0000 7f4058400640 1 --2- 172.21.15.46:0/3396848646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f405015ada0 0x7f4050139b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.475+0000 7f4058400640 1 --2- 172.21.15.46:0/3396848646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4050144210 0x7f4050139650 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.475+0000 7f4058400640 1 --2- 172.21.15.46:0/3396848646 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f40500ce950 0x7f4050157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.475+0000 7f4058400640 1 -- 172.21.15.46:0/3396848646 >> 172.21.15.46:0/3396848646 conn(0x7f4050066110 msgr2=0x7f4050069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:44.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.475+0000 7f4058400640 1 -- 172.21.15.46:0/3396848646 shutdown_connections 2024-03-29T08:27:44.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.475+0000 7f4058400640 1 -- 172.21.15.46:0/3396848646 wait complete. 2024-03-29T08:27:44.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-03-29T08:27:44.668 INFO:teuthology.orchestra.run.smithi046.stdout:{"message":"","return_code":0} 2024-03-29T08:27:44.668 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-03-29T08:27:44.668 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:27:44.668 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:27:44.678 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:27:44.678 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi046.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-03-29T08:27:44.678 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:27:44.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:44.679 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:27:44.727 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-29T08:27:44.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:27:44.728 DEBUG:teuthology.orchestra.run.smithi046:> sudo daemon-helper kill ceph --cluster ceph -w 2024-03-29T08:27:44.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:44.774 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-03-29T08:27:44.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.919+0000 7f9f3e200640 1 Processor -- start 2024-03-29T08:27:44.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.919+0000 7f9f3e200640 1 -- start start 2024-03-29T08:27:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.919+0000 7f9f3e200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f3815a970 0x7f9f380d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.919+0000 7f9f3e200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f380b0d00 0x7f9f380cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.919+0000 7f9f3e200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f380cba70 con 0x7f9f380d0b00 2024-03-29T08:27:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.919+0000 7f9f3e200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f380d0f60 con 0x7f9f3815a970 2024-03-29T08:27:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.919+0000 7f9f3e200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f380b1320 con 0x7f9f380b0d00 2024-03-29T08:27:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.919+0000 7f9f37400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f3815a970 0x7f9f380d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.919+0000 7f9f2fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9f380d0b00 0x7f9f380b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45018/0 (socket says 172.21.15.46:45018) 2024-03-29T08:27:44.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.919+0000 7f9f2fe00640 1 -- 172.21.15.46:0/978629931 learned_addr learned my addr 172.21.15.46:0/978629931 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:27:44.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.919+0000 7f9f37e00640 1 --2- 172.21.15.46:0/978629931 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f380b0d00 0x7f9f380cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.919+0000 7f9f37400640 1 -- 172.21.15.46:0/978629931 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f380b0d00 msgr2=0x7f9f380cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:44.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.919+0000 7f9f37400640 1 --2- 172.21.15.46:0/978629931 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f380b0d00 0x7f9f380cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.919+0000 7f9f37400640 1 -- 172.21.15.46:0/978629931 >> v1:172.21.15.46:6789/0 conn(0x7f9f380d0b00 legacy=0x7f9f380b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.919+0000 7f9f37400640 1 -- 172.21.15.46:0/978629931 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f380cb420 con 0x7f9f3815a970 2024-03-29T08:27:44.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.919+0000 7f9f37e00640 1 --2- 172.21.15.46:0/978629931 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f380b0d00 0x7f9f380cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.923+0000 7f9f37400640 1 --2- 172.21.15.46:0/978629931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3815a970 0x7f9f380d05c0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f9f280097c0 tx=0x7f9f2801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f423a8a4d32bb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.923+0000 7f9f36a00640 1 -- 172.21.15.46:0/978629931 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f280ad680 con 0x7f9f3815a970 2024-03-29T08:27:44.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.923+0000 7f9f36a00640 1 -- 172.21.15.46:0/978629931 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f280ad7e0 con 0x7f9f3815a970 2024-03-29T08:27:44.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.923+0000 7f9f36a00640 1 -- 172.21.15.46:0/978629931 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f280adb70 con 0x7f9f3815a970 2024-03-29T08:27:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.923+0000 7f9f3e200640 1 -- 172.21.15.46:0/978629931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3815a970 msgr2=0x7f9f380d05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.923+0000 7f9f3e200640 1 --2- 172.21.15.46:0/978629931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3815a970 0x7f9f380d05c0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f9f280097c0 tx=0x7f9f2801c470 comp rx=0 tx=0).stop 2024-03-29T08:27:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.923+0000 7f9f3e200640 1 -- 172.21.15.46:0/978629931 shutdown_connections 2024-03-29T08:27:44.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.923+0000 7f9f3e200640 1 --2- 172.21.15.46:0/978629931 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f380b0d00 0x7f9f380cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.923+0000 7f9f3e200640 1 --2- 172.21.15.46:0/978629931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f3815a970 0x7f9f380d05c0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.923+0000 7f9f3e200640 1 -- 172.21.15.46:0/978629931 >> 172.21.15.46:0/978629931 conn(0x7f9f38089fd0 msgr2=0x7f9f3808a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.923+0000 7f9f3e200640 1 -- 172.21.15.46:0/978629931 shutdown_connections 2024-03-29T08:27:44.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.923+0000 7f9f3e200640 1 -- 172.21.15.46:0/978629931 wait complete. 2024-03-29T08:27:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f3e200640 1 Processor -- start 2024-03-29T08:27:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f3e200640 1 -- start start 2024-03-29T08:27:44.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f3e200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f380b0d00 0x7f9f380ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f3e200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f380d0b00 0x7f9f380c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f3e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f3815a970 0x7f9f380c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f3e200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f38158e30 con 0x7f9f380b0d00 2024-03-29T08:27:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f3e200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f38159ba0 con 0x7f9f3815a970 2024-03-29T08:27:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f3e200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f38159e80 con 0x7f9f380d0b00 2024-03-29T08:27:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f37400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f380b0d00 0x7f9f380ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f37400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f380b0d00 0x7f9f380ceeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51552/0 (socket says 172.21.15.46:51552) 2024-03-29T08:27:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f37400640 1 -- 172.21.15.46:0/1743214470 learned_addr learned my addr 172.21.15.46:0/1743214470 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f2fe00640 1 --2- 172.21.15.46:0/1743214470 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f380d0b00 0x7f9f380c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f37e00640 1 --2- 172.21.15.46:0/1743214470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f3815a970 0x7f9f380c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f37400640 1 -- 172.21.15.46:0/1743214470 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f380d0b00 msgr2=0x7f9f380c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f37400640 1 --2- 172.21.15.46:0/1743214470 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f380d0b00 0x7f9f380c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f37400640 1 -- 172.21.15.46:0/1743214470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f3815a970 msgr2=0x7f9f380c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f37400640 1 --2- 172.21.15.46:0/1743214470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f3815a970 0x7f9f380c7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f37400640 1 -- 172.21.15.46:0/1743214470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f28009420 con 0x7f9f380b0d00 2024-03-29T08:27:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f37400640 1 --2- 172.21.15.46:0/1743214470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f380b0d00 0x7f9f380ceeb0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f9f2801ab20 tx=0x7f9f2801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f2f400640 1 -- 172.21.15.46:0/1743214470 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f280adea0 con 0x7f9f380b0d00 2024-03-29T08:27:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f2f400640 1 -- 172.21.15.46:0/1743214470 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f280b8420 con 0x7f9f380b0d00 2024-03-29T08:27:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f2f400640 1 -- 172.21.15.46:0/1743214470 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f280b87f0 con 0x7f9f380b0d00 2024-03-29T08:27:44.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f3e200640 1 -- 172.21.15.46:0/1743214470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f3815a160 con 0x7f9f380b0d00 2024-03-29T08:27:44.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.927+0000 7f9f3e200640 1 -- 172.21.15.46:0/1743214470 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f38158a50 con 0x7f9f380b0d00 2024-03-29T08:27:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f2f400640 1 -- 172.21.15.46:0/1743214470 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9f280b4040 con 0x7f9f380b0d00 2024-03-29T08:27:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f2f400640 1 --2- 172.21.15.46:0/1743214470 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f14066690 0x7f9f14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f2f400640 1 -- 172.21.15.46:0/1743214470 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f2811e720 con 0x7f9f380b0d00 2024-03-29T08:27:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f2fe00640 1 --2- 172.21.15.46:0/1743214470 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f14066690 0x7f9f14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f415b8000 1 Processor -- start 2024-03-29T08:27:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f415b8000 1 -- start start 2024-03-29T08:27:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f2fe00640 1 --2- 172.21.15.46:0/1743214470 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f14066690 0x7f9f14068b50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9f380cc410 tx=0x7f9f20009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f415b8000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc64598d30 0x55fc64599110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f415b8000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc64599650 0x55fc6454fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f415b8000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc6454ff60 0x55fc64550340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f415b8000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc64417170 con 0x55fc6454ff60 2024-03-29T08:27:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f415b8000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc643a4d80 con 0x55fc64599650 2024-03-29T08:27:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f415b8000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc643a2350 con 0x55fc64598d30 2024-03-29T08:27:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc6454ff60 0x55fc64550340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc6454ff60 0x55fc64550340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51556/0 (socket says 172.21.15.46:51556) 2024-03-29T08:27:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f37e00640 1 -- 172.21.15.46:0/567024174 learned_addr learned my addr 172.21.15.46:0/567024174 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:44.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f37400640 1 --2- 172.21.15.46:0/567024174 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc64598d30 0x55fc64599110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.931+0000 7f9f2fe00640 1 --2- 172.21.15.46:0/567024174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc64599650 0x55fc6454fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f37400640 1 -- 172.21.15.46:0/567024174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc64599650 msgr2=0x55fc6454fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f37400640 1 --2- 172.21.15.46:0/567024174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc64599650 0x55fc6454fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f37400640 1 -- 172.21.15.46:0/567024174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc6454ff60 msgr2=0x55fc64550340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f37400640 1 --2- 172.21.15.46:0/567024174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc6454ff60 0x55fc64550340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f37400640 1 -- 172.21.15.46:0/567024174 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc6409a070 con 0x55fc64598d30 2024-03-29T08:27:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f37e00640 1 --2- 172.21.15.46:0/567024174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc6454ff60 0x55fc64550340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:27:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f37400640 1 --2- 172.21.15.46:0/567024174 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc64598d30 0x55fc64599110 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9f2811cc40 tx=0x7f9f280ef270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e42fc305669daaa4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f27e00640 1 -- 172.21.15.46:0/567024174 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f280f7c50 con 0x55fc64598d30 2024-03-29T08:27:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f27e00640 1 -- 172.21.15.46:0/567024174 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f280edcb0 con 0x55fc64598d30 2024-03-29T08:27:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f27e00640 1 -- 172.21.15.46:0/567024174 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f280eee00 con 0x55fc64598d30 2024-03-29T08:27:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 1 -- 172.21.15.46:0/567024174 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc64598d30 msgr2=0x55fc64599110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 1 --2- 172.21.15.46:0/567024174 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc64598d30 0x55fc64599110 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9f2811cc40 tx=0x7f9f280ef270 comp rx=0 tx=0).stop 2024-03-29T08:27:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 1 -- 172.21.15.46:0/567024174 shutdown_connections 2024-03-29T08:27:44.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 1 --2- 172.21.15.46:0/567024174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc64599650 0x55fc6454fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 1 --2- 172.21.15.46:0/567024174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc6454ff60 0x55fc64550340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 1 --2- 172.21.15.46:0/567024174 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc64598d30 0x55fc64599110 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 1 -- 172.21.15.46:0/567024174 >> 172.21.15.46:0/567024174 conn(0x55fc6454bf10 msgr2=0x55fc6454e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 1 -- 172.21.15.46:0/567024174 shutdown_connections 2024-03-29T08:27:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 1 -- 172.21.15.46:0/567024174 wait complete. 2024-03-29T08:27:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 1 -- start start 2024-03-29T08:27:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 1 Processor -- start 2024-03-29T08:27:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc6454ff60 0x55fc645559a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc64598d30 0x55fc64586a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc64599650 0x55fc64586f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6439dc40 con 0x55fc64599650 2024-03-29T08:27:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6433cfa0 con 0x55fc6454ff60 2024-03-29T08:27:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc6435b1c0 con 0x55fc64598d30 2024-03-29T08:27:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.935+0000 7f9f415b8000 10 client.0 _ll_register_callbacks cb 0x55fc6453b060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:27:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc64599650 0x55fc64586f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc64599650 0x55fc64586f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51572/0 (socket says 172.21.15.46:51572) 2024-03-29T08:27:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f37e00640 1 -- 172.21.15.46:0/1862850885 learned_addr learned my addr 172.21.15.46:0/1862850885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f37400640 1 --2- 172.21.15.46:0/1862850885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc6454ff60 0x55fc645559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f2fe00640 1 --2- 172.21.15.46:0/1862850885 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc64598d30 0x55fc64586a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f37e00640 1 -- 172.21.15.46:0/1862850885 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc64598d30 msgr2=0x55fc64586a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f37e00640 1 --2- 172.21.15.46:0/1862850885 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc64598d30 0x55fc64586a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f37e00640 1 -- 172.21.15.46:0/1862850885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc6454ff60 msgr2=0x55fc645559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f37e00640 1 --2- 172.21.15.46:0/1862850885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc6454ff60 0x55fc645559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f37e00640 1 -- 172.21.15.46:0/1862850885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f30007dd0 con 0x55fc64599650 2024-03-29T08:27:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f37e00640 1 --2- 172.21.15.46:0/1862850885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc64599650 0x55fc64586f40 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000ef50 tx=0x7f9f3000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e363c2cdc7893348 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f24c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:27:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f24c00640 1 -- 172.21.15.46:0/1862850885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30013070 con 0x55fc64599650 2024-03-29T08:27:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f24c00640 1 -- 172.21.15.46:0/1862850885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f300039a0 con 0x55fc64599650 2024-03-29T08:27:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 1 -- 172.21.15.46:0/1862850885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc6439aa20 con 0x55fc64599650 2024-03-29T08:27:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f24c00640 1 -- 172.21.15.46:0/1862850885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f3000cea0 con 0x55fc64599650 2024-03-29T08:27:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 10 client.8679 fetch_fsmap learned FSMap version 77 2024-03-29T08:27:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 1 -- 172.21.15.46:0/1862850885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x7f9f38158a50 con 0x55fc64599650 2024-03-29T08:27:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f24c00640 1 -- 172.21.15.46:0/1862850885 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fc6439aa20 con 0x55fc64599650 2024-03-29T08:27:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f24c00640 1 -- 172.21.15.46:0/1862850885 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 77) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9f30008040 con 0x55fc64599650 2024-03-29T08:27:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 10 client.8679 fetch_fsmap finished waiting for FSMap version 77 2024-03-29T08:27:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 10 client.8679 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-29T08:27:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 20 client.8679 populate_metadata read hostname 'smithi046' 2024-03-29T08:27:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 1 --2- 172.21.15.46:0/1862850885 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55fc645a0380 0x55fc645a0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 4 client.8679 mds_command: new command op to 7966 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:27:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 1 -- 172.21.15.46:0/1862850885 --> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc6439c590 con 0x55fc645a0380 2024-03-29T08:27:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f37400640 1 --2- 172.21.15.46:0/1862850885 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55fc645a0380 0x55fc645a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f24c00640 10 client.8679 ms_handle_connect on v2:172.21.15.46:6834/824184385 2024-03-29T08:27:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f37400640 1 --2- 172.21.15.46:0/1862850885 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55fc645a0380 0x55fc645a0790 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af8bc934c6610994 server_cookie=51d76a7ee3c52042 in_seq=0 out_seq=0 2024-03-29T08:27:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f24c00640 1 -- 172.21.15.46:0/1862850885 <== mds.0 v2:172.21.15.46:6834/824184385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55fc6439c590 con 0x55fc645a0380 2024-03-29T08:27:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f24c00640 10 client.8679 handle_command_reply: tid=0 2024-03-29T08:27:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 1 client.8679 shutdown 2024-03-29T08:27:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 1 -- 172.21.15.46:0/1862850885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc64599650 msgr2=0x55fc64586f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 1 --2- 172.21.15.46:0/1862850885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc64599650 0x55fc64586f40 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000ef50 tx=0x7f9f3000c7f0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 1 -- 172.21.15.46:0/1862850885 shutdown_connections 2024-03-29T08:27:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 1 --2- 172.21.15.46:0/1862850885 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55fc645a0380 0x55fc645a0790 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 1 --2- 172.21.15.46:0/1862850885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc64599650 0x55fc64586f40 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 1 --2- 172.21.15.46:0/1862850885 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc64598d30 0x55fc64586a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 1 --2- 172.21.15.46:0/1862850885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc6454ff60 0x55fc645559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f24c00640 0 client.8679 ms_handle_reset on v2:172.21.15.46:6834/824184385 2024-03-29T08:27:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.939+0000 7f9f415b8000 1 -- 172.21.15.46:0/1862850885 >> 172.21.15.46:0/1862850885 conn(0x55fc6454bf10 msgr2=0x55fc6454eb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f9f415b8000 1 -- 172.21.15.46:0/1862850885 shutdown_connections 2024-03-29T08:27:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f9f415b8000 1 -- 172.21.15.46:0/1862850885 wait complete. 2024-03-29T08:27:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f9f415b8000 20 client.8679 trim_cache size 0 max 16384 2024-03-29T08:27:44.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5988000640 1 Processor -- start 2024-03-29T08:27:44.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5988000640 1 -- start start 2024-03-29T08:27:44.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5988000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f598015ca10 0x7f59800cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5988000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f59800cd970 0x7f59800cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5988000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59800b14e0 0x7f5980160590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5988000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59800b6ea0 con 0x7f598015ca10 2024-03-29T08:27:44.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5988000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59800cdec0 con 0x7f59800cd970 2024-03-29T08:27:44.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5988000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59800b6570 con 0x7f59800b14e0 2024-03-29T08:27:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5985600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f598015ca10 0x7f59800cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5985600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f598015ca10 0x7f59800cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51578/0 (socket says 172.21.15.46:51578) 2024-03-29T08:27:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5985600640 1 -- 172.21.15.46:0/820128581 learned_addr learned my addr 172.21.15.46:0/820128581 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5985600640 1 -- 172.21.15.46:0/820128581 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59800b14e0 msgr2=0x7f5980160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5985600640 1 --2- 172.21.15.46:0/820128581 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59800b14e0 0x7f5980160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5985600640 1 -- 172.21.15.46:0/820128581 >> v2:172.21.15.73:3300/0 conn(0x7f59800cd970 msgr2=0x7f59800cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5984c00640 1 --2- 172.21.15.46:0/820128581 >> v2:172.21.15.73:3300/0 conn(0x7f59800cd970 0x7f59800cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5985600640 1 --2- 172.21.15.46:0/820128581 >> v2:172.21.15.73:3300/0 conn(0x7f59800cd970 0x7f59800cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5985600640 1 -- 172.21.15.46:0/820128581 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59800b6850 con 0x7f598015ca10 2024-03-29T08:27:44.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.943+0000 7f5984c00640 1 --2- 172.21.15.46:0/820128581 >> v2:172.21.15.73:3300/0 conn(0x7f59800cd970 0x7f59800cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.947+0000 7f5985600640 1 --2- 172.21.15.46:0/820128581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598015ca10 0x7f59800cf880 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f59800cf310 tx=0x7f597401c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b62ab1e681776a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.947+0000 7f597fe00640 1 -- 172.21.15.46:0/820128581 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59740ab600 con 0x7f598015ca10 2024-03-29T08:27:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.947+0000 7f597fe00640 1 -- 172.21.15.46:0/820128581 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59740a7050 con 0x7f598015ca10 2024-03-29T08:27:44.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.947+0000 7f597fe00640 1 -- 172.21.15.46:0/820128581 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59740ab980 con 0x7f598015ca10 2024-03-29T08:27:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.947+0000 7f5988000640 1 -- 172.21.15.46:0/820128581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598015ca10 msgr2=0x7f59800cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.947+0000 7f5988000640 1 --2- 172.21.15.46:0/820128581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598015ca10 0x7f59800cf880 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f59800cf310 tx=0x7f597401c230 comp rx=0 tx=0).stop 2024-03-29T08:27:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.947+0000 7f5988000640 1 -- 172.21.15.46:0/820128581 shutdown_connections 2024-03-29T08:27:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.947+0000 7f5988000640 1 --2- 172.21.15.46:0/820128581 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59800b14e0 0x7f5980160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.947+0000 7f5988000640 1 --2- 172.21.15.46:0/820128581 >> v2:172.21.15.73:3300/0 conn(0x7f59800cd970 0x7f59800cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.947+0000 7f5988000640 1 --2- 172.21.15.46:0/820128581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f598015ca10 0x7f59800cf880 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.947+0000 7f5988000640 1 -- 172.21.15.46:0/820128581 >> 172.21.15.46:0/820128581 conn(0x7f59800718c0 msgr2=0x7f5980071cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.947+0000 7f5988000640 1 -- 172.21.15.46:0/820128581 shutdown_connections 2024-03-29T08:27:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.947+0000 7f5988000640 1 -- 172.21.15.46:0/820128581 wait complete. 2024-03-29T08:27:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5988000640 1 Processor -- start 2024-03-29T08:27:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5988000640 1 -- start start 2024-03-29T08:27:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5988000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59800b14e0 0x7f598015d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5988000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59800cd970 0x7f59800c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5988000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598015ca10 0x7f5980141b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5988000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598015d5d0 con 0x7f59800b14e0 2024-03-29T08:27:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5988000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598015a390 con 0x7f598015ca10 2024-03-29T08:27:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5988000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598015ff10 con 0x7f59800cd970 2024-03-29T08:27:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5985600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59800b14e0 0x7f598015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5985600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59800b14e0 0x7f598015d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51594/0 (socket says 172.21.15.46:51594) 2024-03-29T08:27:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5985600640 1 -- 172.21.15.46:0/4005038153 learned_addr learned my addr 172.21.15.46:0/4005038153 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5984c00640 1 --2- 172.21.15.46:0/4005038153 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59800cd970 0x7f59800c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5985600640 1 -- 172.21.15.46:0/4005038153 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59800cd970 msgr2=0x7f59800c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5986000640 1 --2- 172.21.15.46:0/4005038153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598015ca10 0x7f5980141b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5985600640 1 --2- 172.21.15.46:0/4005038153 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59800cd970 0x7f59800c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5985600640 1 -- 172.21.15.46:0/4005038153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598015ca10 msgr2=0x7f5980141b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5985600640 1 --2- 172.21.15.46:0/4005038153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598015ca10 0x7f5980141b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5985600640 1 -- 172.21.15.46:0/4005038153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59740094d0 con 0x7f59800b14e0 2024-03-29T08:27:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5984c00640 1 --2- 172.21.15.46:0/4005038153 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59800cd970 0x7f59800c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5986000640 1 --2- 172.21.15.46:0/4005038153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f598015ca10 0x7f5980141b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:27:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5985600640 1 --2- 172.21.15.46:0/4005038153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59800b14e0 0x7f598015d090 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f597401cad0 tx=0x7f597401aa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:44.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f597d600640 1 -- 172.21.15.46:0/4005038153 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59740b6620 con 0x7f59800b14e0 2024-03-29T08:27:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f597d600640 1 -- 172.21.15.46:0/4005038153 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59740b67a0 con 0x7f59800b14e0 2024-03-29T08:27:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f597d600640 1 -- 172.21.15.46:0/4005038153 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59740b6b50 con 0x7f59800b14e0 2024-03-29T08:27:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5988000640 1 -- 172.21.15.46:0/4005038153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598015f950 con 0x7f59800b14e0 2024-03-29T08:27:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5988000640 1 -- 172.21.15.46:0/4005038153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f598015a020 con 0x7f59800b14e0 2024-03-29T08:27:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.951+0000 7f5988000640 1 -- 172.21.15.46:0/4005038153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f598006c540 con 0x7f59800b14e0 2024-03-29T08:27:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.955+0000 7f597d600640 1 -- 172.21.15.46:0/4005038153 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f59740b2040 con 0x7f59800b14e0 2024-03-29T08:27:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.955+0000 7f597d600640 1 --2- 172.21.15.46:0/4005038153 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5960066690 0x7f5960068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.955+0000 7f597d600640 1 -- 172.21.15.46:0/4005038153 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f597411eb40 con 0x7f59800b14e0 2024-03-29T08:27:44.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.955+0000 7f597d600640 1 -- 172.21.15.46:0/4005038153 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+511 (secure 0 0 0) 0x7f597411f2b0 con 0x7f59800b14e0 2024-03-29T08:27:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.955+0000 7f5984c00640 1 --2- 172.21.15.46:0/4005038153 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5960066690 0x7f5960068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.955+0000 7f5988000640 1 -- 172.21.15.46:0/4005038153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f598004e120 con 0x7f59800b14e0 2024-03-29T08:27:44.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.955+0000 7f5984c00640 1 --2- 172.21.15.46:0/4005038153 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5960066690 0x7f5960068b50 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f598015c5d0 tx=0x7f596c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:44.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f415b8000 1 Processor -- start 2024-03-29T08:27:44.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f415b8000 1 -- start start 2024-03-29T08:27:44.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f415b8000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc6454ff60 0x55fc64597f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f415b8000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc64598d30 0x55fc645984b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f415b8000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc64599650 0x55fc645b4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f415b8000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc6405acf0 con 0x55fc64599650 2024-03-29T08:27:44.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f415b8000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc643a07a0 con 0x55fc64598d30 2024-03-29T08:27:44.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f415b8000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc6435e090 con 0x55fc6454ff60 2024-03-29T08:27:44.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc64599650 0x55fc645b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f37e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc64599650 0x55fc645b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51608/0 (socket says 172.21.15.46:51608) 2024-03-29T08:27:44.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f37e00640 1 -- 172.21.15.46:0/1332145349 learned_addr learned my addr 172.21.15.46:0/1332145349 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:44.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f37e00640 1 -- 172.21.15.46:0/1332145349 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc6454ff60 msgr2=0x55fc64597f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:44.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f37e00640 1 --2- 172.21.15.46:0/1332145349 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc6454ff60 0x55fc64597f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f37e00640 1 -- 172.21.15.46:0/1332145349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc64598d30 msgr2=0x55fc645984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:44.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f37e00640 1 --2- 172.21.15.46:0/1332145349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc64598d30 0x55fc645984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f37e00640 1 -- 172.21.15.46:0/1332145349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc64362ba0 con 0x55fc64599650 2024-03-29T08:27:44.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f37e00640 1 --2- 172.21.15.46:0/1332145349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc64599650 0x55fc645b4290 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f9f30007b40 tx=0x7f9f30014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7093e2c9b7ce6677 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:44.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f2cc00640 1 -- 172.21.15.46:0/1332145349 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30003c00 con 0x55fc64599650 2024-03-29T08:27:44.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f2cc00640 1 -- 172.21.15.46:0/1332145349 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f3000cea0 con 0x55fc64599650 2024-03-29T08:27:44.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f2cc00640 1 -- 172.21.15.46:0/1332145349 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30003fe0 con 0x55fc64599650 2024-03-29T08:27:44.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f415b8000 1 -- 172.21.15.46:0/1332145349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc64599650 msgr2=0x55fc645b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.959+0000 7f9f415b8000 1 --2- 172.21.15.46:0/1332145349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc64599650 0x55fc645b4290 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f9f30007b40 tx=0x7f9f30014000 comp rx=0 tx=0).stop 2024-03-29T08:27:44.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 -- 172.21.15.46:0/1332145349 shutdown_connections 2024-03-29T08:27:44.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 --2- 172.21.15.46:0/1332145349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc64598d30 0x55fc645984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 --2- 172.21.15.46:0/1332145349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc64599650 0x55fc645b4290 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 --2- 172.21.15.46:0/1332145349 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc6454ff60 0x55fc64597f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 -- 172.21.15.46:0/1332145349 >> 172.21.15.46:0/1332145349 conn(0x55fc6454bf10 msgr2=0x55fc64597860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:44.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 -- 172.21.15.46:0/1332145349 shutdown_connections 2024-03-29T08:27:44.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 -- 172.21.15.46:0/1332145349 wait complete. 2024-03-29T08:27:44.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 -- start start 2024-03-29T08:27:44.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 Processor -- start 2024-03-29T08:27:44.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc6454ff60 0x55fc645fcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc64598d30 0x55fc645fd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc64599650 0x55fc645e0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc64364420 con 0x55fc6454ff60 2024-03-29T08:27:44.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc64417170 con 0x55fc64598d30 2024-03-29T08:27:44.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fc64553d80 con 0x55fc64599650 2024-03-29T08:27:44.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 10 client.0 _ll_register_callbacks cb 0x55fc644170c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:27:44.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f37400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc6454ff60 0x55fc645fcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f37400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc6454ff60 0x55fc645fcbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51616/0 (socket says 172.21.15.46:51616) 2024-03-29T08:27:44.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f37400640 1 -- 172.21.15.46:0/1854853310 learned_addr learned my addr 172.21.15.46:0/1854853310 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:44.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f37e00640 1 --2- 172.21.15.46:0/1854853310 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc64599650 0x55fc645e0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f2fe00640 1 --2- 172.21.15.46:0/1854853310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc64598d30 0x55fc645fd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f37400640 1 -- 172.21.15.46:0/1854853310 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc64599650 msgr2=0x55fc645e0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:44.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f37400640 1 --2- 172.21.15.46:0/1854853310 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc64599650 0x55fc645e0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f37400640 1 -- 172.21.15.46:0/1854853310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc64598d30 msgr2=0x55fc645fd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:44.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f37400640 1 --2- 172.21.15.46:0/1854853310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc64598d30 0x55fc645fd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f37400640 1 -- 172.21.15.46:0/1854853310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f38158a50 con 0x55fc6454ff60 2024-03-29T08:27:44.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f37e00640 1 --2- 172.21.15.46:0/1854853310 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc64599650 0x55fc645e0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:44.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f37400640 1 --2- 172.21.15.46:0/1854853310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc6454ff60 0x55fc645fcbb0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f9f280097c0 tx=0x7f9f2811cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4855b92446864133 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:44.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f24c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:27:44.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f24c00640 1 -- 172.21.15.46:0/1854853310 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f281070d0 con 0x55fc6454ff60 2024-03-29T08:27:44.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f24c00640 1 -- 172.21.15.46:0/1854853310 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f28107230 con 0x55fc6454ff60 2024-03-29T08:27:44.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 -- 172.21.15.46:0/1854853310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc6435a5f0 con 0x55fc6454ff60 2024-03-29T08:27:44.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f24c00640 1 -- 172.21.15.46:0/1854853310 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f280f7ea0 con 0x55fc6454ff60 2024-03-29T08:27:44.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 10 client.8691 fetch_fsmap learned FSMap version 77 2024-03-29T08:27:44.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 -- 172.21.15.46:0/1854853310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55fc64348ef0 con 0x55fc6454ff60 2024-03-29T08:27:44.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f24c00640 1 -- 172.21.15.46:0/1854853310 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f280fb080 con 0x55fc6454ff60 2024-03-29T08:27:44.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f24c00640 1 -- 172.21.15.46:0/1854853310 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 77) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9f280eee00 con 0x55fc6454ff60 2024-03-29T08:27:44.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 10 client.8691 fetch_fsmap finished waiting for FSMap version 77 2024-03-29T08:27:44.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 10 client.8691 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-29T08:27:44.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 20 client.8691 populate_metadata read hostname 'smithi046' 2024-03-29T08:27:44.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 --2- 172.21.15.46:0/1854853310 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55fc645e0ed0 0x55fc645e12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:44.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 4 client.8691 mds_command: new command op to 7966 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-03-29T08:27:44.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f415b8000 1 -- 172.21.15.46:0/1854853310 --> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55fc6435c410 con 0x55fc645e0ed0 2024-03-29T08:27:44.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.963+0000 7f9f2fe00640 1 --2- 172.21.15.46:0/1854853310 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55fc645e0ed0 0x55fc645e12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:44.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f2fe00640 1 --2- 172.21.15.46:0/1854853310 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55fc645e0ed0 0x55fc645e12e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5e1b49032a73d17 server_cookie=ded186dba09704a4 in_seq=0 out_seq=0 2024-03-29T08:27:44.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f24c00640 10 client.8691 ms_handle_connect on v2:172.21.15.46:6834/824184385 2024-03-29T08:27:44.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f24c00640 1 -- 172.21.15.46:0/1854853310 <== mds.0 v2:172.21.15.46:6834/824184385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fc6435c410 con 0x55fc645e0ed0 2024-03-29T08:27:44.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f24c00640 10 client.8691 handle_command_reply: tid=0 2024-03-29T08:27:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f415b8000 1 client.8691 shutdown 2024-03-29T08:27:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f415b8000 1 -- 172.21.15.46:0/1854853310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc6454ff60 msgr2=0x55fc645fcbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f415b8000 1 --2- 172.21.15.46:0/1854853310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc6454ff60 0x55fc645fcbb0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f9f280097c0 tx=0x7f9f2811cb10 comp rx=0 tx=0).stop 2024-03-29T08:27:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f415b8000 1 -- 172.21.15.46:0/1854853310 shutdown_connections 2024-03-29T08:27:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f415b8000 1 --2- 172.21.15.46:0/1854853310 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55fc645e0ed0 0x55fc645e12e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f415b8000 1 --2- 172.21.15.46:0/1854853310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc64598d30 0x55fc645fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f415b8000 1 --2- 172.21.15.46:0/1854853310 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55fc64599650 0x55fc645e0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f415b8000 1 --2- 172.21.15.46:0/1854853310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fc6454ff60 0x55fc645fcbb0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f415b8000 1 -- 172.21.15.46:0/1854853310 >> 172.21.15.46:0/1854853310 conn(0x55fc6454bf10 msgr2=0x55fc645980f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f24c00640 0 client.8691 ms_handle_reset on v2:172.21.15.46:6834/824184385 2024-03-29T08:27:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f415b8000 1 -- 172.21.15.46:0/1854853310 shutdown_connections 2024-03-29T08:27:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f415b8000 1 -- 172.21.15.46:0/1854853310 wait complete. 2024-03-29T08:27:44.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f415b8000 20 client.8691 trim_cache size 0 max 16384 2024-03-29T08:27:44.982 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:27:44.982 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-29T08:27:44.982 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "eee539f3-15e5-446e-a614-6cb0bbff026e", 2024-03-29T08:27:44.982 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-29T08:27:44.982 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:27:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f3e200640 1 -- 172.21.15.46:0/1743214470 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f14066690 msgr2=0x7f9f14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f3e200640 1 --2- 172.21.15.46:0/1743214470 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f14066690 0x7f9f14068b50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9f380cc410 tx=0x7f9f20009040 comp rx=0 tx=0).stop 2024-03-29T08:27:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f3e200640 1 -- 172.21.15.46:0/1743214470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f380b0d00 msgr2=0x7f9f380ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f3e200640 1 --2- 172.21.15.46:0/1743214470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f380b0d00 0x7f9f380ceeb0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f9f2801ab20 tx=0x7f9f2801acb0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f3e200640 1 -- 172.21.15.46:0/1743214470 shutdown_connections 2024-03-29T08:27:44.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f3e200640 1 --2- 172.21.15.46:0/1743214470 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f14066690 0x7f9f14068b50 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f3e200640 1 --2- 172.21.15.46:0/1743214470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f3815a970 0x7f9f380c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f3e200640 1 --2- 172.21.15.46:0/1743214470 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f380d0b00 0x7f9f380c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f3e200640 1 --2- 172.21.15.46:0/1743214470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f380b0d00 0x7f9f380ceeb0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f3e200640 1 -- 172.21.15.46:0/1743214470 >> 172.21.15.46:0/1743214470 conn(0x7f9f38089fd0 msgr2=0x7f9f381535f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f3e200640 1 -- 172.21.15.46:0/1743214470 shutdown_connections 2024-03-29T08:27:44.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:44.967+0000 7f9f3e200640 1 -- 172.21.15.46:0/1743214470 wait complete. 2024-03-29T08:27:44.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:27:44.989 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-29T08:27:45.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.131+0000 7fabf2000640 1 Processor -- start 2024-03-29T08:27:45.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.131+0000 7fabf2000640 1 -- start start 2024-03-29T08:27:45.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.131+0000 7fabf2000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabec15df90 0x7fabec0ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.131+0000 7fabf2000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fabec0cbea0 0x7fabec0ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.131+0000 7fabf2000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabec15d260 con 0x7fabec142a50 2024-03-29T08:27:45.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.131+0000 7fabf2000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabec0aed00 con 0x7fabec0cbea0 2024-03-29T08:27:45.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.131+0000 7fabf2000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0aefd0 con 0x7fabec15df90 2024-03-29T08:27:45.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.131+0000 7fabe3e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabec15df90 0x7fabec0ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.131+0000 7fabebe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fabec0cbea0 0x7fabec0ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.131+0000 7fabe3e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabec15df90 0x7fabec0ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:50188/0 (socket says 172.21.15.46:50188) 2024-03-29T08:27:45.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.131+0000 7fabe3e00640 1 -- 172.21.15.46:0/4289805382 learned_addr learned my addr 172.21.15.46:0/4289805382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:45.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.131+0000 7fabebe00640 1 -- 172.21.15.46:0/4289805382 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabec15df90 msgr2=0x7fabec0ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:45.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.131+0000 7fabebe00640 1 --2- 172.21.15.46:0/4289805382 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabec15df90 0x7fabec0ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.131+0000 7fabebe00640 1 -- 172.21.15.46:0/4289805382 >> v1:172.21.15.73:6789/0 conn(0x7fabec142a50 legacy=0x7fabec0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:45.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.131+0000 7fabebe00640 1 -- 172.21.15.46:0/4289805382 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec15d4b0 con 0x7fabec0cbea0 2024-03-29T08:27:45.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.131+0000 7fabeaa00640 1 -- 172.21.15.46:0/4289805382 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4102266022 0 0) 0x7fabec15d260 con 0x7fabec142a50 2024-03-29T08:27:45.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.135+0000 7fabebe00640 1 --2- 172.21.15.46:0/4289805382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabec0cbea0 0x7fabec0ae7c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009d90 tx=0x7fabe401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bb6987cd2d15f65 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:45.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.135+0000 7fabeaa00640 1 -- 172.21.15.46:0/4289805382 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe40a5790 con 0x7fabec0cbea0 2024-03-29T08:27:45.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.135+0000 7fabeaa00640 1 -- 172.21.15.46:0/4289805382 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe40a58f0 con 0x7fabec0cbea0 2024-03-29T08:27:45.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.135+0000 7fabeaa00640 1 -- 172.21.15.46:0/4289805382 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe40a5c80 con 0x7fabec0cbea0 2024-03-29T08:27:45.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.135+0000 7fabf2000640 1 -- 172.21.15.46:0/4289805382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabec0cbea0 msgr2=0x7fabec0ae7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:45.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.135+0000 7fabf2000640 1 --2- 172.21.15.46:0/4289805382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabec0cbea0 0x7fabec0ae7c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009d90 tx=0x7fabe401c8e0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.135+0000 7fabf2000640 1 -- 172.21.15.46:0/4289805382 shutdown_connections 2024-03-29T08:27:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.135+0000 7fabf2000640 1 --2- 172.21.15.46:0/4289805382 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabec15df90 0x7fabec0ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.135+0000 7fabf2000640 1 --2- 172.21.15.46:0/4289805382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabec0cbea0 0x7fabec0ae7c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.135+0000 7fabf2000640 1 -- 172.21.15.46:0/4289805382 >> 172.21.15.46:0/4289805382 conn(0x7fabec06a390 msgr2=0x7fabec0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.135+0000 7fabf2000640 1 -- 172.21.15.46:0/4289805382 shutdown_connections 2024-03-29T08:27:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.135+0000 7fabf2000640 1 -- 172.21.15.46:0/4289805382 wait complete. 2024-03-29T08:27:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.135+0000 7fabf2000640 1 Processor -- start 2024-03-29T08:27:45.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabf2000640 1 -- start start 2024-03-29T08:27:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabf2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabec0cbea0 0x7fabec13ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabf2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabec142a50 0x7fabec158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabf2000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabec15df90 0x7fabec139890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabf2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec144050 con 0x7fabec0cbea0 2024-03-29T08:27:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabf2000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec143440 con 0x7fabec142a50 2024-03-29T08:27:45.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabf2000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec143720 con 0x7fabec15df90 2024-03-29T08:27:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabeb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabec0cbea0 0x7fabec13ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabe3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabec142a50 0x7fabec158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabeb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabec0cbea0 0x7fabec13ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51626/0 (socket says 172.21.15.46:51626) 2024-03-29T08:27:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabeb400640 1 -- 172.21.15.46:0/1091864554 learned_addr learned my addr 172.21.15.46:0/1091864554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabebe00640 1 --2- 172.21.15.46:0/1091864554 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabec15df90 0x7fabec139890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabe3e00640 1 -- 172.21.15.46:0/1091864554 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabec15df90 msgr2=0x7fabec139890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabe3e00640 1 --2- 172.21.15.46:0/1091864554 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabec15df90 0x7fabec139890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabe3e00640 1 -- 172.21.15.46:0/1091864554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabec0cbea0 msgr2=0x7fabec13ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabe3e00640 1 --2- 172.21.15.46:0/1091864554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabec0cbea0 0x7fabec13ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabe3e00640 1 -- 172.21.15.46:0/1091864554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe4009a70 con 0x7fabec142a50 2024-03-29T08:27:45.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabe3e00640 1 --2- 172.21.15.46:0/1091864554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabec142a50 0x7fabec158940 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fabd4010b90 tx=0x7fabd400e730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:45.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabe3400640 1 -- 172.21.15.46:0/1091864554 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4011650 con 0x7fabec142a50 2024-03-29T08:27:45.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabe3400640 1 -- 172.21.15.46:0/1091864554 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd40117b0 con 0x7fabec142a50 2024-03-29T08:27:45.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabe3400640 1 -- 172.21.15.46:0/1091864554 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4011b40 con 0x7fabec142a50 2024-03-29T08:27:45.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabf2000640 1 -- 172.21.15.46:0/1091864554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabec143a00 con 0x7fabec142a50 2024-03-29T08:27:45.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.139+0000 7fabf2000640 1 -- 172.21.15.46:0/1091864554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabec158e80 con 0x7fabec142a50 2024-03-29T08:27:45.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabe3400640 1 -- 172.21.15.46:0/1091864554 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fabd4011ca0 con 0x7fabec142a50 2024-03-29T08:27:45.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabe3400640 1 --2- 172.21.15.46:0/1091864554 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fabc80666e0 0x7fabc8068ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabe3400640 1 -- 172.21.15.46:0/1091864554 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabd4086ce0 con 0x7fabec142a50 2024-03-29T08:27:45.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabeb400640 1 --2- 172.21.15.46:0/1091864554 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fabc80666e0 0x7fabc8068ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabf537d000 1 Processor -- start 2024-03-29T08:27:45.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabf537d000 1 -- start start 2024-03-29T08:27:45.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabeb400640 1 --2- 172.21.15.46:0/1091864554 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fabc80666e0 0x7fabc8068ba0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fabdc007fd0 tx=0x7fabdc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabf537d000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e56cc0 0x55b0f5e570a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabf537d000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e575e0 0x55b0f5e0c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabf537d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e0ce20 0x55b0f5e0d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabf537d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f5cd1630 con 0x55b0f5e0ce20 2024-03-29T08:27:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabf537d000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f5c5ebd0 con 0x55b0f5e56cc0 2024-03-29T08:27:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabf537d000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0f5c5c1a0 con 0x55b0f5e575e0 2024-03-29T08:27:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e0ce20 0x55b0f5e0d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e0ce20 0x55b0f5e0d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51632/0 (socket says 172.21.15.46:51632) 2024-03-29T08:27:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabebe00640 1 -- 172.21.15.46:0/3534805316 learned_addr learned my addr 172.21.15.46:0/3534805316 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabeb400640 1 --2- 172.21.15.46:0/3534805316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e56cc0 0x55b0f5e570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabe3e00640 1 --2- 172.21.15.46:0/3534805316 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e575e0 0x55b0f5e0c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabebe00640 1 -- 172.21.15.46:0/3534805316 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e575e0 msgr2=0x55b0f5e0c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:45.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabebe00640 1 --2- 172.21.15.46:0/3534805316 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e575e0 0x55b0f5e0c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabebe00640 1 -- 172.21.15.46:0/3534805316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e56cc0 msgr2=0x55b0f5e570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:45.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabebe00640 1 --2- 172.21.15.46:0/3534805316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e56cc0 0x55b0f5e570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabebe00640 1 -- 172.21.15.46:0/3534805316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0f5953e20 con 0x55b0f5e0ce20 2024-03-29T08:27:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabebe00640 1 --2- 172.21.15.46:0/3534805316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e0ce20 0x55b0f5e0d200 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fabe40b0040 tx=0x7fabe4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c4c1496fca3bc4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabdbe00640 1 -- 172.21.15.46:0/3534805316 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe40a5c60 con 0x55b0f5e0ce20 2024-03-29T08:27:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabdbe00640 1 -- 172.21.15.46:0/3534805316 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe40bc070 con 0x55b0f5e0ce20 2024-03-29T08:27:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.143+0000 7fabdbe00640 1 -- 172.21.15.46:0/3534805316 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe40c3480 con 0x55b0f5e0ce20 2024-03-29T08:27:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 -- 172.21.15.46:0/3534805316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e0ce20 msgr2=0x55b0f5e0d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:45.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 --2- 172.21.15.46:0/3534805316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e0ce20 0x55b0f5e0d200 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fabe40b0040 tx=0x7fabe4004ac0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 -- 172.21.15.46:0/3534805316 shutdown_connections 2024-03-29T08:27:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 --2- 172.21.15.46:0/3534805316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e0ce20 0x55b0f5e0d200 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 --2- 172.21.15.46:0/3534805316 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e575e0 0x55b0f5e0c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 --2- 172.21.15.46:0/3534805316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e56cc0 0x55b0f5e570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 -- 172.21.15.46:0/3534805316 >> 172.21.15.46:0/3534805316 conn(0x55b0f5c07160 msgr2=0x55b0f5e0f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 -- 172.21.15.46:0/3534805316 shutdown_connections 2024-03-29T08:27:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 -- 172.21.15.46:0/3534805316 wait complete. 2024-03-29T08:27:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 -- start start 2024-03-29T08:27:45.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 Processor -- start 2024-03-29T08:27:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e0ce20 0x55b0f5e56710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e56cc0 0x55b0f5e181a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e575e0 0x55b0f5e186e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f5c57a90 con 0x55b0f5e0ce20 2024-03-29T08:27:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f5bf6d40 con 0x55b0f5e575e0 2024-03-29T08:27:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0f5c14f30 con 0x55b0f5e56cc0 2024-03-29T08:27:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 10 client.0 _ll_register_callbacks cb 0x55b0f5cb9810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:27:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabeb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e0ce20 0x55b0f5e56710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabeb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e0ce20 0x55b0f5e56710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51648/0 (socket says 172.21.15.46:51648) 2024-03-29T08:27:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabeb400640 1 -- 172.21.15.46:0/915000430 learned_addr learned my addr 172.21.15.46:0/915000430 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabe3e00640 1 --2- 172.21.15.46:0/915000430 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e56cc0 0x55b0f5e181a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabebe00640 1 --2- 172.21.15.46:0/915000430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e575e0 0x55b0f5e186e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabeb400640 1 -- 172.21.15.46:0/915000430 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e56cc0 msgr2=0x55b0f5e181a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabeb400640 1 --2- 172.21.15.46:0/915000430 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e56cc0 0x55b0f5e181a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabeb400640 1 -- 172.21.15.46:0/915000430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e575e0 msgr2=0x55b0f5e186e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabeb400640 1 --2- 172.21.15.46:0/915000430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e575e0 0x55b0f5e186e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabeb400640 1 -- 172.21.15.46:0/915000430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabec15d4b0 con 0x55b0f5e0ce20 2024-03-29T08:27:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabeb400640 1 --2- 172.21.15.46:0/915000430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e0ce20 0x55b0f5e56710 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0147a0 tx=0x7fabdc001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a4e66b6195f548 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:27:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabd8c00640 1 -- 172.21.15.46:0/915000430 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc0138b0 con 0x55b0f5e0ce20 2024-03-29T08:27:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabd8c00640 1 -- 172.21.15.46:0/915000430 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabdc01c040 con 0x55b0f5e0ce20 2024-03-29T08:27:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 -- 172.21.15.46:0/915000430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0f5c563e0 con 0x55b0f5e0ce20 2024-03-29T08:27:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabd8c00640 1 -- 172.21.15.46:0/915000430 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabdc013a10 con 0x55b0f5e0ce20 2024-03-29T08:27:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 10 client.8700 fetch_fsmap learned FSMap version 77 2024-03-29T08:27:45.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.147+0000 7fabf537d000 1 -- 172.21.15.46:0/915000430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55b0f5c1caa0 con 0x55b0f5e0ce20 2024-03-29T08:27:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabd8c00640 1 -- 172.21.15.46:0/915000430 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b0f5c563e0 con 0x55b0f5e0ce20 2024-03-29T08:27:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabd8c00640 1 -- 172.21.15.46:0/915000430 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 77) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fabdc012070 con 0x55b0f5e0ce20 2024-03-29T08:27:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 10 client.8700 fetch_fsmap finished waiting for FSMap version 77 2024-03-29T08:27:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 10 client.8700 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-29T08:27:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 20 client.8700 populate_metadata read hostname 'smithi046' 2024-03-29T08:27:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 1 --2- 172.21.15.46:0/915000430 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55b0f5e19480 0x55b0f5e68d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 4 client.8700 mds_command: new command op to 7966 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:27:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 1 -- 172.21.15.46:0/915000430 --> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b0f5c27d00 con 0x55b0f5e19480 2024-03-29T08:27:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabe3e00640 1 --2- 172.21.15.46:0/915000430 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55b0f5e19480 0x55b0f5e68d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabe3e00640 1 --2- 172.21.15.46:0/915000430 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55b0f5e19480 0x55b0f5e68d70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58b87a988c10db2f server_cookie=d96e624e563a05b6 in_seq=0 out_seq=0 2024-03-29T08:27:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabd8c00640 10 client.8700 ms_handle_connect on v2:172.21.15.46:6834/824184385 2024-03-29T08:27:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabd8c00640 1 -- 172.21.15.46:0/915000430 <== mds.0 v2:172.21.15.46:6834/824184385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55b0f5c27d00 con 0x55b0f5e19480 2024-03-29T08:27:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabd8c00640 10 client.8700 handle_command_reply: tid=0 2024-03-29T08:27:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 1 client.8700 shutdown 2024-03-29T08:27:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 1 -- 172.21.15.46:0/915000430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e0ce20 msgr2=0x55b0f5e56710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:45.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 1 --2- 172.21.15.46:0/915000430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e0ce20 0x55b0f5e56710 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0147a0 tx=0x7fabdc001090 comp rx=0 tx=0).stop 2024-03-29T08:27:45.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 1 -- 172.21.15.46:0/915000430 shutdown_connections 2024-03-29T08:27:45.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 1 --2- 172.21.15.46:0/915000430 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55b0f5e19480 0x55b0f5e68d70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 1 --2- 172.21.15.46:0/915000430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e575e0 0x55b0f5e186e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 1 --2- 172.21.15.46:0/915000430 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e56cc0 0x55b0f5e181a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 1 --2- 172.21.15.46:0/915000430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e0ce20 0x55b0f5e56710 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 1 -- 172.21.15.46:0/915000430 >> 172.21.15.46:0/915000430 conn(0x55b0f5c07160 msgr2=0x55b0f5e098c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:45.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabd8c00640 0 client.8700 ms_handle_reset on v2:172.21.15.46:6834/824184385 2024-03-29T08:27:45.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 1 -- 172.21.15.46:0/915000430 shutdown_connections 2024-03-29T08:27:45.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 1 -- 172.21.15.46:0/915000430 wait complete. 2024-03-29T08:27:45.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.151+0000 7fabf537d000 20 client.8700 trim_cache size 0 max 16384 2024-03-29T08:27:45.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabf537d000 1 Processor -- start 2024-03-29T08:27:45.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabf537d000 1 -- start start 2024-03-29T08:27:45.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabf537d000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e0ce20 0x55b0f5eb0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabf537d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e56cc0 0x55b0f5eb1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabf537d000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e575e0 0x55b0f5eb15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabf537d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f5914aa0 con 0x55b0f5e56cc0 2024-03-29T08:27:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabf537d000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f5c5ba90 con 0x55b0f5e0ce20 2024-03-29T08:27:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabf537d000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0f5bfd070 con 0x55b0f5e575e0 2024-03-29T08:27:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabe3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e56cc0 0x55b0f5eb1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabe3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e56cc0 0x55b0f5eb1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51664/0 (socket says 172.21.15.46:51664) 2024-03-29T08:27:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabe3e00640 1 -- 172.21.15.46:0/2496859321 learned_addr learned my addr 172.21.15.46:0/2496859321 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabebe00640 1 --2- 172.21.15.46:0/2496859321 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e575e0 0x55b0f5eb15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabeb400640 1 --2- 172.21.15.46:0/2496859321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e0ce20 0x55b0f5eb0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabe3e00640 1 -- 172.21.15.46:0/2496859321 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e575e0 msgr2=0x55b0f5eb15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabe3e00640 1 --2- 172.21.15.46:0/2496859321 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e575e0 0x55b0f5eb15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabe3e00640 1 -- 172.21.15.46:0/2496859321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e0ce20 msgr2=0x55b0f5eb0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabe3e00640 1 --2- 172.21.15.46:0/2496859321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e0ce20 0x55b0f5eb0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabe3e00640 1 -- 172.21.15.46:0/2496859321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0f5c02df0 con 0x55b0f5e56cc0 2024-03-29T08:27:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabebe00640 1 --2- 172.21.15.46:0/2496859321 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e575e0 0x55b0f5eb15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabeb400640 1 --2- 172.21.15.46:0/2496859321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e0ce20 0x55b0f5eb0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabe3e00640 1 --2- 172.21.15.46:0/2496859321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e56cc0 0x55b0f5eb1080 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fabd4054330 tx=0x7fabd4013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819374cc6966776c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabe0c00640 1 -- 172.21.15.46:0/2496859321 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4056030 con 0x55b0f5e56cc0 2024-03-29T08:27:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabe0c00640 1 -- 172.21.15.46:0/2496859321 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd40561d0 con 0x55b0f5e56cc0 2024-03-29T08:27:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabe0c00640 1 -- 172.21.15.46:0/2496859321 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4054540 con 0x55b0f5e56cc0 2024-03-29T08:27:45.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabf537d000 1 -- 172.21.15.46:0/2496859321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e56cc0 msgr2=0x55b0f5eb1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:45.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabf537d000 1 --2- 172.21.15.46:0/2496859321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e56cc0 0x55b0f5eb1080 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fabd4054330 tx=0x7fabd4013040 comp rx=0 tx=0).stop 2024-03-29T08:27:45.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabf537d000 1 -- 172.21.15.46:0/2496859321 shutdown_connections 2024-03-29T08:27:45.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabf537d000 1 --2- 172.21.15.46:0/2496859321 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e575e0 0x55b0f5eb15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabf537d000 1 --2- 172.21.15.46:0/2496859321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e56cc0 0x55b0f5eb1080 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabf537d000 1 --2- 172.21.15.46:0/2496859321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e0ce20 0x55b0f5eb0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabf537d000 1 -- 172.21.15.46:0/2496859321 >> 172.21.15.46:0/2496859321 conn(0x55b0f5c07160 msgr2=0x55b0f5e55a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabf537d000 1 -- 172.21.15.46:0/2496859321 shutdown_connections 2024-03-29T08:27:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.167+0000 7fabf537d000 1 -- 172.21.15.46:0/2496859321 wait complete. 2024-03-29T08:27:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 1 -- start start 2024-03-29T08:27:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 1 Processor -- start 2024-03-29T08:27:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e0ce20 0x55b0f5eb57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e56cc0 0x55b0f5e68720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e575e0 0x55b0f5e68c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f5c9b170 con 0x55b0f5e56cc0 2024-03-29T08:27:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0f5d10ec0 con 0x55b0f5e575e0 2024-03-29T08:27:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0f5c21ee0 con 0x55b0f5e0ce20 2024-03-29T08:27:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 10 client.0 _ll_register_callbacks cb 0x7fabd400b2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:27:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabe3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e56cc0 0x55b0f5e68720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabe3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e56cc0 0x55b0f5e68720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51680/0 (socket says 172.21.15.46:51680) 2024-03-29T08:27:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabe3e00640 1 -- 172.21.15.46:0/972940944 learned_addr learned my addr 172.21.15.46:0/972940944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:27:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabeb400640 1 --2- 172.21.15.46:0/972940944 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e0ce20 0x55b0f5eb57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabebe00640 1 --2- 172.21.15.46:0/972940944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e575e0 0x55b0f5e68c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabe3e00640 1 -- 172.21.15.46:0/972940944 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e0ce20 msgr2=0x55b0f5eb57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabe3e00640 1 --2- 172.21.15.46:0/972940944 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e0ce20 0x55b0f5eb57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabe3e00640 1 -- 172.21.15.46:0/972940944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e575e0 msgr2=0x55b0f5e68c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:27:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabe3e00640 1 --2- 172.21.15.46:0/972940944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e575e0 0x55b0f5e68c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabe3e00640 1 -- 172.21.15.46:0/972940944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabec158e80 con 0x55b0f5e56cc0 2024-03-29T08:27:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabebe00640 1 --2- 172.21.15.46:0/972940944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e575e0 0x55b0f5e68c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:27:45.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabe3e00640 1 --2- 172.21.15.46:0/972940944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e56cc0 0x55b0f5e68720 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fabd4054330 tx=0x7fabdc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd9fc92c145700 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:27:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:27:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabd8c00640 1 -- 172.21.15.46:0/972940944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4057520 con 0x55b0f5e56cc0 2024-03-29T08:27:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabd8c00640 1 -- 172.21.15.46:0/972940944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabd405fce0 con 0x55b0f5e56cc0 2024-03-29T08:27:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 1 -- 172.21.15.46:0/972940944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b0f5c14970 con 0x55b0f5e56cc0 2024-03-29T08:27:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabd8c00640 1 -- 172.21.15.46:0/972940944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabd4056030 con 0x55b0f5e56cc0 2024-03-29T08:27:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 10 client.8706 fetch_fsmap learned FSMap version 77 2024-03-29T08:27:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 1 -- 172.21.15.46:0/972940944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55b0f5bf6960 con 0x55b0f5e56cc0 2024-03-29T08:27:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabd8c00640 1 -- 172.21.15.46:0/972940944 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b0f5c14970 con 0x55b0f5e56cc0 2024-03-29T08:27:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabd8c00640 1 -- 172.21.15.46:0/972940944 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 77) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fabd407d020 con 0x55b0f5e56cc0 2024-03-29T08:27:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 10 client.8706 fetch_fsmap finished waiting for FSMap version 77 2024-03-29T08:27:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 10 client.8706 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-29T08:27:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 20 client.8706 populate_metadata read hostname 'smithi046' 2024-03-29T08:27:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 1 --2- 172.21.15.46:0/972940944 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55b0f5e691a0 0x55b0f5e695b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:27:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 4 client.8706 mds_command: new command op to 7966 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:27:45.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabf537d000 1 -- 172.21.15.46:0/972940944 --> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b0f5c86a20 con 0x55b0f5e691a0 2024-03-29T08:27:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.171+0000 7fabeb400640 1 --2- 172.21.15.46:0/972940944 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55b0f5e691a0 0x55b0f5e695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:27:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabeb400640 1 --2- 172.21.15.46:0/972940944 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55b0f5e691a0 0x55b0f5e695b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9593f064f0c9912b server_cookie=c62a3718626890b1 in_seq=0 out_seq=0 2024-03-29T08:27:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabd8c00640 10 client.8706 ms_handle_connect on v2:172.21.15.46:6834/824184385 2024-03-29T08:27:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabd8c00640 1 -- 172.21.15.46:0/972940944 <== mds.0 v2:172.21.15.46:6834/824184385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55b0f5c86a20 con 0x55b0f5e691a0 2024-03-29T08:27:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabd8c00640 10 client.8706 handle_command_reply: tid=0 2024-03-29T08:27:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf537d000 1 client.8706 shutdown 2024-03-29T08:27:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf537d000 1 -- 172.21.15.46:0/972940944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e56cc0 msgr2=0x55b0f5e68720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf537d000 1 --2- 172.21.15.46:0/972940944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e56cc0 0x55b0f5e68720 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fabd4054330 tx=0x7fabdc012000 comp rx=0 tx=0).stop 2024-03-29T08:27:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf537d000 1 -- 172.21.15.46:0/972940944 shutdown_connections 2024-03-29T08:27:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf537d000 1 --2- 172.21.15.46:0/972940944 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55b0f5e691a0 0x55b0f5e695b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf537d000 1 --2- 172.21.15.46:0/972940944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0f5e575e0 0x55b0f5e68c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf537d000 1 --2- 172.21.15.46:0/972940944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b0f5e56cc0 0x55b0f5e68720 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf537d000 1 --2- 172.21.15.46:0/972940944 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55b0f5e0ce20 0x55b0f5eb57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabd8c00640 0 client.8706 ms_handle_reset on v2:172.21.15.46:6834/824184385 2024-03-29T08:27:45.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf537d000 1 -- 172.21.15.46:0/972940944 >> 172.21.15.46:0/972940944 conn(0x55b0f5c07160 msgr2=0x55b0f5e642b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf537d000 1 -- 172.21.15.46:0/972940944 shutdown_connections 2024-03-29T08:27:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf537d000 1 -- 172.21.15.46:0/972940944 wait complete. 2024-03-29T08:27:45.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf537d000 20 client.8706 trim_cache size 0 max 16384 2024-03-29T08:27:45.181 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:27:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-29T08:27:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-29T08:27:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "eee539f3-15e5-446e-a614-6cb0bbff026e": { 2024-03-29T08:27:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-29T08:27:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "eee539f3-15e5-446e-a614-6cb0bbff026e", 2024-03-29T08:27:45.181 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,repair" 2024-03-29T08:27:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:45.182 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:27:45.182 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:27:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf2000640 1 -- 172.21.15.46:0/1091864554 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fabc80666e0 msgr2=0x7fabc8068ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf2000640 1 --2- 172.21.15.46:0/1091864554 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fabc80666e0 0x7fabc8068ba0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fabdc007fd0 tx=0x7fabdc009040 comp rx=0 tx=0).stop 2024-03-29T08:27:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf2000640 1 -- 172.21.15.46:0/1091864554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabec142a50 msgr2=0x7fabec158940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:27:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf2000640 1 --2- 172.21.15.46:0/1091864554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabec142a50 0x7fabec158940 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fabd4010b90 tx=0x7fabd400e730 comp rx=0 tx=0).stop 2024-03-29T08:27:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf2000640 1 -- 172.21.15.46:0/1091864554 shutdown_connections 2024-03-29T08:27:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf2000640 1 --2- 172.21.15.46:0/1091864554 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fabc80666e0 0x7fabc8068ba0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf2000640 1 --2- 172.21.15.46:0/1091864554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabec142a50 0x7fabec158940 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf2000640 1 --2- 172.21.15.46:0/1091864554 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fabec15df90 0x7fabec139890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf2000640 1 --2- 172.21.15.46:0/1091864554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fabec0cbea0 0x7fabec13ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:27:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf2000640 1 -- 172.21.15.46:0/1091864554 >> 172.21.15.46:0/1091864554 conn(0x7fabec06a390 msgr2=0x7fabec0afdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:27:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf2000640 1 -- 172.21.15.46:0/1091864554 shutdown_connections 2024-03-29T08:27:45.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:45.175+0000 7fabf2000640 1 -- 172.21.15.46:0/1091864554 wait complete. 2024-03-29T08:27:45.196 INFO:tasks.cephfs.filesystem:scrub status for tag:eee539f3-15e5-446e-a614-6cb0bbff026e - {'path': '/', 'tag': 'eee539f3-15e5-446e-a614-6cb0bbff026e', 'options': 'recursive,repair'} 2024-03-29T08:27:46.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:46.023+0000 7f597d600640 1 -- 172.21.15.46:0/4005038153 <== mon.0 v2:172.21.15.46:3300/0 7 ==== log(4 entries from seq 1 at 2024-03-29T08:27:44.970080+0000) v1 ==== 695+0+0 (secure 0 0 0) 0x7f59740f0260 con 0x7f59800b14e0 2024-03-29T08:27:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: cluster: 2024-03-29T08:27:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: id: e9b7960f-ec1f-4787-95c3-c806094d37f0 2024-03-29T08:27:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: health: HEALTH_OK 2024-03-29T08:27:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: services: 2024-03-29T08:27:46.026 INFO:teuthology.orchestra.run.smithi046.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-03-29T08:27:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: mgr: x(active, since 9m), standbys: y 2024-03-29T08:27:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: mds: 1/1 daemons up, 3 standby 2024-03-29T08:27:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-03-29T08:27:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: data: 2024-03-29T08:27:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: volumes: 1/1 healthy 2024-03-29T08:27:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: pools: 3 pools, 97 pgs 2024-03-29T08:27:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: objects: 30 objects, 19 MiB 2024-03-29T08:27:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: usage: 290 MiB used, 720 GiB / 720 GiB avail 2024-03-29T08:27:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: pgs: 97 active+clean 2024-03-29T08:27:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: io: 2024-03-29T08:27:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: client: 85 B/s rd, 1.0 MiB/s wr, 0 op/s rd, 2 op/s wr 2024-03-29T08:27:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:46.027 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:27:46.028 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:27:44.970080+0000 mds.c [INF] scrub queued for path: / 2024-03-29T08:27:46.028 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:27:44.970084+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-03-29T08:27:46.028 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:27:44.970088+0000 mds.c [INF] scrub summary: active paths [/] 2024-03-29T08:27:46.028 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:27:44.971482+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-03-29T08:27:51.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:51.034+0000 7f597d600640 1 -- 172.21.15.46:0/4005038153 <== mon.0 v2:172.21.15.46:3300/0 8 ==== log(1 entries from seq 5 at 2024-03-29T08:27:49.266316+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f59740efb60 con 0x7f59800b14e0 2024-03-29T08:27:51.039 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:27:49.266316+0000 mds.c [INF] scrub summary: idle 2024-03-29T08:27:57.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:57.046+0000 7f597d600640 1 -- 172.21.15.46:0/4005038153 <== mon.0 v2:172.21.15.46:3300/0 9 ==== log(4 entries from seq 1029 at 2024-03-29T08:27:56.993685+0000) v1 ==== 1112+0+0 (secure 0 0 0) 0x7f59740f3990 con 0x7f59800b14e0 2024-03-29T08:27:58.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:27:58.050+0000 7f597d600640 1 -- 172.21.15.46:0/4005038153 <== mon.0 v2:172.21.15.46:3300/0 10 ==== log(2 entries from seq 1031 at 2024-03-29T08:27:57.056130+0000) v1 ==== 559+0+0 (secure 0 0 0) 0x7f597411f000 con 0x7f59800b14e0 2024-03-29T08:28:09.589 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:28:09.594 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:28:15.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:28:15.197 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-29T08:28:15.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.346+0000 7f0e44200640 1 Processor -- start 2024-03-29T08:28:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.346+0000 7f0e44200640 1 -- start start 2024-03-29T08:28:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.350+0000 7f0e44200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e3c143180 0x7f0e3c0adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.350+0000 7f0e44200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0e3c0ae1e0 0x7f0e3c0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.350+0000 7f0e44200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3c0afe80 con 0x7f0e3c142da0 2024-03-29T08:28:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.350+0000 7f0e44200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e3c06cd80 con 0x7f0e3c0ae1e0 2024-03-29T08:28:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.350+0000 7f0e44200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e3c06d020 con 0x7f0e3c143180 2024-03-29T08:28:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.350+0000 7f0e41800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0e3c142da0 0x7f0e3c0abac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43194/0 (socket says 172.21.15.46:43194) 2024-03-29T08:28:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.350+0000 7f0e41800640 1 -- 172.21.15.46:0/293429283 learned_addr learned my addr 172.21.15.46:0/293429283 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:28:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.350+0000 7f0e42200640 1 --2- 172.21.15.46:0/293429283 >> v2:172.21.15.46:3300/0 conn(0x7f0e3c0ae1e0 0x7f0e3c0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.350+0000 7f0e40e00640 1 --2- 172.21.15.46:0/293429283 >> v2:172.21.15.73:3300/0 conn(0x7f0e3c143180 0x7f0e3c0adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.350+0000 7f0e3be00640 1 -- 172.21.15.46:0/293429283 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3772807449 0 0) 0x7f0e3c0afe80 con 0x7f0e3c142da0 2024-03-29T08:28:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.350+0000 7f0e3be00640 1 -- 172.21.15.46:0/293429283 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e24003530 con 0x7f0e3c142da0 2024-03-29T08:28:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.350+0000 7f0e42200640 1 -- 172.21.15.46:0/293429283 >> v2:172.21.15.73:3300/0 conn(0x7f0e3c143180 msgr2=0x7f0e3c0adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.350+0000 7f0e42200640 1 --2- 172.21.15.46:0/293429283 >> v2:172.21.15.73:3300/0 conn(0x7f0e3c143180 0x7f0e3c0adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.350+0000 7f0e42200640 1 -- 172.21.15.46:0/293429283 >> v1:172.21.15.46:6789/0 conn(0x7f0e3c142da0 legacy=0x7f0e3c0abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.350+0000 7f0e42200640 1 -- 172.21.15.46:0/293429283 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c0b00f0 con 0x7f0e3c0ae1e0 2024-03-29T08:28:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.354+0000 7f0e42200640 1 --2- 172.21.15.46:0/293429283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e3c0ae1e0 0x7f0e3c0cb790 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c06d7c0 tx=0x7f0e3401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff91cf607d482bae server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.354+0000 7f0e3be00640 1 -- 172.21.15.46:0/293429283 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e340a5900 con 0x7f0e3c0ae1e0 2024-03-29T08:28:15.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.354+0000 7f0e3be00640 1 -- 172.21.15.46:0/293429283 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e340a5a60 con 0x7f0e3c0ae1e0 2024-03-29T08:28:15.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.354+0000 7f0e3be00640 1 -- 172.21.15.46:0/293429283 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e340a5df0 con 0x7f0e3c0ae1e0 2024-03-29T08:28:15.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.354+0000 7f0e44200640 1 -- 172.21.15.46:0/293429283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e3c0ae1e0 msgr2=0x7f0e3c0cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.354+0000 7f0e44200640 1 --2- 172.21.15.46:0/293429283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e3c0ae1e0 0x7f0e3c0cb790 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c06d7c0 tx=0x7f0e3401c930 comp rx=0 tx=0).stop 2024-03-29T08:28:15.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.354+0000 7f0e44200640 1 -- 172.21.15.46:0/293429283 shutdown_connections 2024-03-29T08:28:15.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.354+0000 7f0e44200640 1 --2- 172.21.15.46:0/293429283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e3c0ae1e0 0x7f0e3c0cb790 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.354+0000 7f0e44200640 1 --2- 172.21.15.46:0/293429283 >> v2:172.21.15.73:3300/0 conn(0x7f0e3c143180 0x7f0e3c0adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.354+0000 7f0e44200640 1 -- 172.21.15.46:0/293429283 >> 172.21.15.46:0/293429283 conn(0x7f0e3c06a390 msgr2=0x7f0e3c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:15.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.354+0000 7f0e44200640 1 -- 172.21.15.46:0/293429283 shutdown_connections 2024-03-29T08:28:15.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.354+0000 7f0e44200640 1 -- 172.21.15.46:0/293429283 wait complete. 2024-03-29T08:28:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.354+0000 7f0e44200640 1 Processor -- start 2024-03-29T08:28:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.354+0000 7f0e44200640 1 -- start start 2024-03-29T08:28:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e44200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e3c0ae1e0 0x7f0e3c15bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e44200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e3c142da0 0x7f0e3c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e44200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e3c143180 0x7f0e3c13a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e44200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c146060 con 0x7f0e3c143180 2024-03-29T08:28:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e44200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c145ae0 con 0x7f0e3c0ae1e0 2024-03-29T08:28:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e44200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c145d80 con 0x7f0e3c142da0 2024-03-29T08:28:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e41800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e3c0ae1e0 0x7f0e3c15bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e42200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e3c143180 0x7f0e3c13a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e41800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e3c0ae1e0 0x7f0e3c15bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:36426/0 (socket says 172.21.15.46:36426) 2024-03-29T08:28:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e41800640 1 -- 172.21.15.46:0/2982942090 learned_addr learned my addr 172.21.15.46:0/2982942090 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e40e00640 1 --2- 172.21.15.46:0/2982942090 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e3c142da0 0x7f0e3c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e41800640 1 -- 172.21.15.46:0/2982942090 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e3c142da0 msgr2=0x7f0e3c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e41800640 1 --2- 172.21.15.46:0/2982942090 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e3c142da0 0x7f0e3c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e41800640 1 -- 172.21.15.46:0/2982942090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e3c143180 msgr2=0x7f0e3c13a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e41800640 1 --2- 172.21.15.46:0/2982942090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e3c143180 0x7f0e3c13a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e41800640 1 -- 172.21.15.46:0/2982942090 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e34009a70 con 0x7f0e3c0ae1e0 2024-03-29T08:28:15.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e42200640 1 --2- 172.21.15.46:0/2982942090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e3c143180 0x7f0e3c13a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:28:15.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e41800640 1 --2- 172.21.15.46:0/2982942090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e3c0ae1e0 0x7f0e3c15bc50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000c4d0 tx=0x7f0e3000c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e39600640 1 -- 172.21.15.46:0/2982942090 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e30005cf0 con 0x7f0e3c0ae1e0 2024-03-29T08:28:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e39600640 1 -- 172.21.15.46:0/2982942090 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e30005e50 con 0x7f0e3c0ae1e0 2024-03-29T08:28:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e39600640 1 -- 172.21.15.46:0/2982942090 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e30007270 con 0x7f0e3c0ae1e0 2024-03-29T08:28:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e44200640 1 -- 172.21.15.46:0/2982942090 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3c144610 con 0x7f0e3c0ae1e0 2024-03-29T08:28:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e44200640 1 -- 172.21.15.46:0/2982942090 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c142030 con 0x7f0e3c0ae1e0 2024-03-29T08:28:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.358+0000 7f0e39600640 1 -- 172.21.15.46:0/2982942090 <== mon.1 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0e300073d0 con 0x7f0e3c0ae1e0 2024-03-29T08:28:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e39600640 1 --2- 172.21.15.46:0/2982942090 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e1c066690 0x7f0e1c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e39600640 1 -- 172.21.15.46:0/2982942090 <== mon.1 v2:172.21.15.73:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0e300836e0 con 0x7f0e3c0ae1e0 2024-03-29T08:28:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e40e00640 1 --2- 172.21.15.46:0/2982942090 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e1c066690 0x7f0e1c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e4764a000 1 Processor -- start 2024-03-29T08:28:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e4764a000 1 -- start start 2024-03-29T08:28:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e40e00640 1 --2- 172.21.15.46:0/2982942090 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e1c066690 0x7f0e1c068b50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c141920 tx=0x7f0e28008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:15.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e4764a000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc0fa70 0x55737fc0fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e4764a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc10390 0x55737fbc34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e4764a000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fbc3a00 0x55737fbc3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e4764a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737fa8e630 con 0x55737fc10390 2024-03-29T08:28:15.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e4764a000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737fa1bbd0 con 0x55737fc0fa70 2024-03-29T08:28:15.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e4764a000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55737fa191a0 con 0x55737fbc3a00 2024-03-29T08:28:15.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e41800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc0fa70 0x55737fc0fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e40e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc10390 0x55737fbc34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e41800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc0fa70 0x55737fc0fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:36438/0 (socket says 172.21.15.46:36438) 2024-03-29T08:28:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e41800640 1 -- 172.21.15.46:0/3985073017 learned_addr learned my addr 172.21.15.46:0/3985073017 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e42200640 1 --2- 172.21.15.46:0/3985073017 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fbc3a00 0x55737fbc3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e41800640 1 -- 172.21.15.46:0/3985073017 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fbc3a00 msgr2=0x55737fbc3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e41800640 1 --2- 172.21.15.46:0/3985073017 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fbc3a00 0x55737fbc3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e41800640 1 -- 172.21.15.46:0/3985073017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc10390 msgr2=0x55737fbc34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e41800640 1 --2- 172.21.15.46:0/3985073017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc10390 0x55737fbc34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e41800640 1 -- 172.21.15.46:0/3985073017 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55737f710e20 con 0x55737fc0fa70 2024-03-29T08:28:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e41800640 1 --2- 172.21.15.46:0/3985073017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc0fa70 0x55737fc0fe50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0e30054700 tx=0x7f0e30054770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a81559f68337de5b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e2e000640 1 -- 172.21.15.46:0/3985073017 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e30052b10 con 0x55737fc0fa70 2024-03-29T08:28:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e2e000640 1 -- 172.21.15.46:0/3985073017 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e30007680 con 0x55737fc0fa70 2024-03-29T08:28:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.362+0000 7f0e2e000640 1 -- 172.21.15.46:0/3985073017 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e30050e80 con 0x55737fc0fa70 2024-03-29T08:28:15.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 -- 172.21.15.46:0/3985073017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc0fa70 msgr2=0x55737fc0fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 --2- 172.21.15.46:0/3985073017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc0fa70 0x55737fc0fe50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0e30054700 tx=0x7f0e30054770 comp rx=0 tx=0).stop 2024-03-29T08:28:15.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 -- 172.21.15.46:0/3985073017 shutdown_connections 2024-03-29T08:28:15.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 --2- 172.21.15.46:0/3985073017 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fbc3a00 0x55737fbc3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 --2- 172.21.15.46:0/3985073017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc10390 0x55737fbc34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 --2- 172.21.15.46:0/3985073017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc0fa70 0x55737fc0fe50 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 -- 172.21.15.46:0/3985073017 >> 172.21.15.46:0/3985073017 conn(0x55737f9c4160 msgr2=0x55737fbf2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 -- 172.21.15.46:0/3985073017 shutdown_connections 2024-03-29T08:28:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 -- 172.21.15.46:0/3985073017 wait complete. 2024-03-29T08:28:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 -- start start 2024-03-29T08:28:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 Processor -- start 2024-03-29T08:28:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fbc3a00 0x55737fbcc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fc0fa70 0x55737fbf3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc10390 0x55737fbf3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737fa14a90 con 0x55737fc10390 2024-03-29T08:28:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737f9b3d40 con 0x55737fbc3a00 2024-03-29T08:28:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55737f9d1f30 con 0x55737fc0fa70 2024-03-29T08:28:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 10 client.0 _ll_register_callbacks cb 0x55737fa76810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:28:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e42200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc10390 0x55737fbf3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e42200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc10390 0x55737fbf3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54854/0 (socket says 172.21.15.46:54854) 2024-03-29T08:28:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e42200640 1 -- 172.21.15.46:0/1533737462 learned_addr learned my addr 172.21.15.46:0/1533737462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e41800640 1 --2- 172.21.15.46:0/1533737462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fbc3a00 0x55737fbcc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e40e00640 1 --2- 172.21.15.46:0/1533737462 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fc0fa70 0x55737fbf3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e42200640 1 -- 172.21.15.46:0/1533737462 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fc0fa70 msgr2=0x55737fbf3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e42200640 1 --2- 172.21.15.46:0/1533737462 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fc0fa70 0x55737fbf3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e42200640 1 -- 172.21.15.46:0/1533737462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fbc3a00 msgr2=0x55737fbcc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e42200640 1 --2- 172.21.15.46:0/1533737462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fbc3a00 0x55737fbcc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e42200640 1 -- 172.21.15.46:0/1533737462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c142030 con 0x55737fc10390 2024-03-29T08:28:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e41800640 1 --2- 172.21.15.46:0/1533737462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fbc3a00 0x55737fbcc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:28:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e40e00640 1 --2- 172.21.15.46:0/1533737462 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fc0fa70 0x55737fbf3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:28:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e42200640 1 --2- 172.21.15.46:0/1533737462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc10390 0x55737fbf3f90 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f0e3401ce10 tx=0x7f0e3401b170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d018620ca52ed server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e22a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:28:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e22a00640 1 -- 172.21.15.46:0/1533737462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e340ad850 con 0x55737fc10390 2024-03-29T08:28:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e22a00640 1 -- 172.21.15.46:0/1533737462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0e340ad9b0 con 0x55737fc10390 2024-03-29T08:28:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 -- 172.21.15.46:0/1533737462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55737fa133e0 con 0x55737fc10390 2024-03-29T08:28:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e22a00640 1 -- 172.21.15.46:0/1533737462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e340adb70 con 0x55737fc10390 2024-03-29T08:28:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 10 client.8721 fetch_fsmap learned FSMap version 77 2024-03-29T08:28:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.366+0000 7f0e4764a000 1 -- 172.21.15.46:0/1533737462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55737f9d9aa0 con 0x55737fc10390 2024-03-29T08:28:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e22a00640 1 -- 172.21.15.46:0/1533737462 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55737fa133e0 con 0x55737fc10390 2024-03-29T08:28:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e22a00640 1 -- 172.21.15.46:0/1533737462 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 77) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0e340adcd0 con 0x55737fc10390 2024-03-29T08:28:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 10 client.8721 fetch_fsmap finished waiting for FSMap version 77 2024-03-29T08:28:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 10 client.8721 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-29T08:28:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 20 client.8721 populate_metadata read hostname 'smithi046' 2024-03-29T08:28:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 1 --2- 172.21.15.46:0/1533737462 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55737fc172d0 0x55737fc176b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 4 client.8721 mds_command: new command op to 7966 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:28:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 1 -- 172.21.15.46:0/1533737462 --> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55737f9e4d00 con 0x55737fc172d0 2024-03-29T08:28:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e41800640 1 --2- 172.21.15.46:0/1533737462 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55737fc172d0 0x55737fc176b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e22a00640 10 client.8721 ms_handle_connect on v2:172.21.15.46:6834/824184385 2024-03-29T08:28:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e41800640 1 --2- 172.21.15.46:0/1533737462 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55737fc172d0 0x55737fc176b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13a048533157b6c6 server_cookie=b24738d9fab0c113 in_seq=0 out_seq=0 2024-03-29T08:28:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e22a00640 1 -- 172.21.15.46:0/1533737462 <== mds.0 v2:172.21.15.46:6834/824184385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55737f9e4d00 con 0x55737fc172d0 2024-03-29T08:28:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e22a00640 10 client.8721 handle_command_reply: tid=0 2024-03-29T08:28:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 1 client.8721 shutdown 2024-03-29T08:28:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 1 -- 172.21.15.46:0/1533737462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc10390 msgr2=0x55737fbf3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 1 --2- 172.21.15.46:0/1533737462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc10390 0x55737fbf3f90 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f0e3401ce10 tx=0x7f0e3401b170 comp rx=0 tx=0).stop 2024-03-29T08:28:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 1 -- 172.21.15.46:0/1533737462 shutdown_connections 2024-03-29T08:28:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 1 --2- 172.21.15.46:0/1533737462 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55737fc172d0 0x55737fc176b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 1 --2- 172.21.15.46:0/1533737462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc10390 0x55737fbf3f90 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 1 --2- 172.21.15.46:0/1533737462 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fc0fa70 0x55737fbf3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 1 --2- 172.21.15.46:0/1533737462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fbc3a00 0x55737fbcc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e22a00640 0 client.8721 ms_handle_reset on v2:172.21.15.46:6834/824184385 2024-03-29T08:28:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 1 -- 172.21.15.46:0/1533737462 >> 172.21.15.46:0/1533737462 conn(0x55737f9c4160 msgr2=0x55737fbc2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 1 -- 172.21.15.46:0/1533737462 shutdown_connections 2024-03-29T08:28:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 1 -- 172.21.15.46:0/1533737462 wait complete. 2024-03-29T08:28:15.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.370+0000 7f0e4764a000 20 client.8721 trim_cache size 0 max 16384 2024-03-29T08:28:15.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e4764a000 1 Processor -- start 2024-03-29T08:28:15.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e4764a000 1 -- start start 2024-03-29T08:28:15.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e4764a000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fbc3a00 0x55737fbc2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e4764a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc0fa70 0x55737fbc3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e4764a000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc10390 0x55737fbc3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e4764a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737f6d1aa0 con 0x55737fc0fa70 2024-03-29T08:28:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e4764a000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737fbbd930 con 0x55737fc10390 2024-03-29T08:28:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e4764a000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55737fa175f0 con 0x55737fbc3a00 2024-03-29T08:28:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e40e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc0fa70 0x55737fbc3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e40e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc0fa70 0x55737fbc3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54864/0 (socket says 172.21.15.46:54864) 2024-03-29T08:28:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e40e00640 1 -- 172.21.15.46:0/3443426901 learned_addr learned my addr 172.21.15.46:0/3443426901 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e42200640 1 --2- 172.21.15.46:0/3443426901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc10390 0x55737fbc3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e41800640 1 --2- 172.21.15.46:0/3443426901 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fbc3a00 0x55737fbc2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e40e00640 1 -- 172.21.15.46:0/3443426901 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fbc3a00 msgr2=0x55737fbc2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e40e00640 1 --2- 172.21.15.46:0/3443426901 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fbc3a00 0x55737fbc2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e40e00640 1 -- 172.21.15.46:0/3443426901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc10390 msgr2=0x55737fbc3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:15.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e40e00640 1 --2- 172.21.15.46:0/3443426901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc10390 0x55737fbc3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e40e00640 1 -- 172.21.15.46:0/3443426901 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55737f9bfdf0 con 0x55737fc0fa70 2024-03-29T08:28:15.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e42200640 1 --2- 172.21.15.46:0/3443426901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc10390 0x55737fbc3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:15.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e41800640 1 --2- 172.21.15.46:0/3443426901 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fbc3a00 0x55737fbc2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:15.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e40e00640 1 --2- 172.21.15.46:0/3443426901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc0fa70 0x55737fbc3170 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f0e28010c20 tx=0x7f0e28016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6721c4f21b65f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:15.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e2ea00640 1 -- 172.21.15.46:0/3443426901 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28016d90 con 0x55737fc0fa70 2024-03-29T08:28:15.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e2ea00640 1 -- 172.21.15.46:0/3443426901 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e28024470 con 0x55737fc0fa70 2024-03-29T08:28:15.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e2ea00640 1 -- 172.21.15.46:0/3443426901 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e280247e0 con 0x55737fc0fa70 2024-03-29T08:28:15.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e4764a000 1 -- 172.21.15.46:0/3443426901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc0fa70 msgr2=0x55737fbc3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e4764a000 1 --2- 172.21.15.46:0/3443426901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc0fa70 0x55737fbc3170 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f0e28010c20 tx=0x7f0e28016000 comp rx=0 tx=0).stop 2024-03-29T08:28:15.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e4764a000 1 -- 172.21.15.46:0/3443426901 shutdown_connections 2024-03-29T08:28:15.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e4764a000 1 --2- 172.21.15.46:0/3443426901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc10390 0x55737fbc3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e4764a000 1 --2- 172.21.15.46:0/3443426901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc0fa70 0x55737fbc3170 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e4764a000 1 --2- 172.21.15.46:0/3443426901 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fbc3a00 0x55737fbc2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.386+0000 7f0e4764a000 1 -- 172.21.15.46:0/3443426901 >> 172.21.15.46:0/3443426901 conn(0x55737f9c4160 msgr2=0x55737fbcb870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:15.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 1 -- 172.21.15.46:0/3443426901 shutdown_connections 2024-03-29T08:28:15.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 1 -- 172.21.15.46:0/3443426901 wait complete. 2024-03-29T08:28:15.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 1 -- start start 2024-03-29T08:28:15.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 1 Processor -- start 2024-03-29T08:28:15.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fbc3a00 0x55737fbc7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc0fa70 0x55737fbc75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc10390 0x55737fc6c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737fa16020 con 0x55737fc0fa70 2024-03-29T08:28:15.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55737fa1ce60 con 0x55737fc10390 2024-03-29T08:28:15.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55737f99bed0 con 0x55737fbc3a00 2024-03-29T08:28:15.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 10 client.0 _ll_register_callbacks cb 0x7f0e340b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:28:15.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e40e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc0fa70 0x55737fbc75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e40e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc0fa70 0x55737fbc75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54876/0 (socket says 172.21.15.46:54876) 2024-03-29T08:28:15.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e40e00640 1 -- 172.21.15.46:0/873328504 learned_addr learned my addr 172.21.15.46:0/873328504 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:15.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e41800640 1 --2- 172.21.15.46:0/873328504 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fbc3a00 0x55737fbc7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e42200640 1 --2- 172.21.15.46:0/873328504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc10390 0x55737fc6c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e40e00640 1 -- 172.21.15.46:0/873328504 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fbc3a00 msgr2=0x55737fbc7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:15.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e40e00640 1 --2- 172.21.15.46:0/873328504 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fbc3a00 0x55737fbc7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e40e00640 1 -- 172.21.15.46:0/873328504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc10390 msgr2=0x55737fc6c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:15.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e40e00640 1 --2- 172.21.15.46:0/873328504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc10390 0x55737fc6c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e40e00640 1 -- 172.21.15.46:0/873328504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2800e040 con 0x55737fc0fa70 2024-03-29T08:28:15.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e40e00640 1 --2- 172.21.15.46:0/873328504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc0fa70 0x55737fbc75d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x55737fa358e0 tx=0x7f0e28013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e323dd1b8286b793 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:15.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e22a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:28:15.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e22a00640 1 -- 172.21.15.46:0/873328504 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e280249f0 con 0x55737fc0fa70 2024-03-29T08:28:15.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e22a00640 1 -- 172.21.15.46:0/873328504 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0e28024b50 con 0x55737fc0fa70 2024-03-29T08:28:15.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 1 -- 172.21.15.46:0/873328504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55737f9d1970 con 0x55737fc0fa70 2024-03-29T08:28:15.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e22a00640 1 -- 172.21.15.46:0/873328504 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e28024d10 con 0x55737fc0fa70 2024-03-29T08:28:15.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 10 client.8727 fetch_fsmap learned FSMap version 77 2024-03-29T08:28:15.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 1 -- 172.21.15.46:0/873328504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x55737f9b3960 con 0x55737fc0fa70 2024-03-29T08:28:15.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e22a00640 1 -- 172.21.15.46:0/873328504 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55737f9d1970 con 0x55737fc0fa70 2024-03-29T08:28:15.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e22a00640 1 -- 172.21.15.46:0/873328504 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 77) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0e28014430 con 0x55737fc0fa70 2024-03-29T08:28:15.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 10 client.8727 fetch_fsmap finished waiting for FSMap version 77 2024-03-29T08:28:15.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 10 client.8727 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-29T08:28:15.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 20 client.8727 populate_metadata read hostname 'smithi046' 2024-03-29T08:28:15.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 1 --2- 172.21.15.46:0/873328504 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55737fc6c7c0 0x55737fc6cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 4 client.8727 mds_command: new command op to 7966 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:28:15.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e4764a000 1 -- 172.21.15.46:0/873328504 --> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55737fa43a20 con 0x55737fc6c7c0 2024-03-29T08:28:15.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.390+0000 7f0e41800640 1 --2- 172.21.15.46:0/873328504 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55737fc6c7c0 0x55737fc6cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e41800640 1 --2- 172.21.15.46:0/873328504 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55737fc6c7c0 0x55737fc6cba0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6897013781477412 server_cookie=eb4aa7c6377294b8 in_seq=0 out_seq=0 2024-03-29T08:28:15.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e22a00640 10 client.8727 ms_handle_connect on v2:172.21.15.46:6834/824184385 2024-03-29T08:28:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e22a00640 1 -- 172.21.15.46:0/873328504 <== mds.0 v2:172.21.15.46:6834/824184385 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55737fa43a20 con 0x55737fc6c7c0 2024-03-29T08:28:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e22a00640 10 client.8727 handle_command_reply: tid=0 2024-03-29T08:28:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e4764a000 1 client.8727 shutdown 2024-03-29T08:28:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e4764a000 1 -- 172.21.15.46:0/873328504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc0fa70 msgr2=0x55737fbc75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e4764a000 1 --2- 172.21.15.46:0/873328504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc0fa70 0x55737fbc75d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x55737fa358e0 tx=0x7f0e28013000 comp rx=0 tx=0).stop 2024-03-29T08:28:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e4764a000 1 -- 172.21.15.46:0/873328504 shutdown_connections 2024-03-29T08:28:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e4764a000 1 --2- 172.21.15.46:0/873328504 >> [v2:172.21.15.46:6834/824184385,v1:172.21.15.46:6835/824184385] conn(0x55737fc6c7c0 0x55737fc6cba0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e4764a000 1 --2- 172.21.15.46:0/873328504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55737fc10390 0x55737fc6c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e4764a000 1 --2- 172.21.15.46:0/873328504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55737fc0fa70 0x55737fbc75d0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e22a00640 0 client.8727 ms_handle_reset on v2:172.21.15.46:6834/824184385 2024-03-29T08:28:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e4764a000 1 --2- 172.21.15.46:0/873328504 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55737fbc3a00 0x55737fbc7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e4764a000 1 -- 172.21.15.46:0/873328504 >> 172.21.15.46:0/873328504 conn(0x55737f9c4160 msgr2=0x55737fbc1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e4764a000 1 -- 172.21.15.46:0/873328504 shutdown_connections 2024-03-29T08:28:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e4764a000 1 -- 172.21.15.46:0/873328504 wait complete. 2024-03-29T08:28:15.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e4764a000 20 client.8727 trim_cache size 0 max 16384 2024-03-29T08:28:15.401 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:28:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-29T08:28:15.401 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-29T08:28:15.401 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:28:15.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e44200640 1 -- 172.21.15.46:0/2982942090 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e1c066690 msgr2=0x7f0e1c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e44200640 1 --2- 172.21.15.46:0/2982942090 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e1c066690 0x7f0e1c068b50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c141920 tx=0x7f0e28008040 comp rx=0 tx=0).stop 2024-03-29T08:28:15.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e44200640 1 -- 172.21.15.46:0/2982942090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e3c0ae1e0 msgr2=0x7f0e3c15bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e44200640 1 --2- 172.21.15.46:0/2982942090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e3c0ae1e0 0x7f0e3c15bc50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000c4d0 tx=0x7f0e3000c990 comp rx=0 tx=0).stop 2024-03-29T08:28:15.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e44200640 1 -- 172.21.15.46:0/2982942090 shutdown_connections 2024-03-29T08:28:15.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e44200640 1 --2- 172.21.15.46:0/2982942090 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e1c066690 0x7f0e1c068b50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e44200640 1 --2- 172.21.15.46:0/2982942090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e3c143180 0x7f0e3c13a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e44200640 1 --2- 172.21.15.46:0/2982942090 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e3c142da0 0x7f0e3c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e44200640 1 --2- 172.21.15.46:0/2982942090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e3c0ae1e0 0x7f0e3c15bc50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e44200640 1 -- 172.21.15.46:0/2982942090 >> 172.21.15.46:0/2982942090 conn(0x7f0e3c06a390 msgr2=0x7f0e3c160460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:15.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e44200640 1 -- 172.21.15.46:0/2982942090 shutdown_connections 2024-03-29T08:28:15.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.394+0000 7f0e44200640 1 -- 172.21.15.46:0/2982942090 wait complete. 2024-03-29T08:28:15.416 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-29T08:28:15.517 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T08:28:15.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.658+0000 7f52fda00640 1 Processor -- start 2024-03-29T08:28:15.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.658+0000 7f52fda00640 1 -- start start 2024-03-29T08:28:15.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.658+0000 7f52fda00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f52f815e080 0x7f52f80ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.658+0000 7f52fda00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f80cb500 con 0x7f52f8099150 2024-03-29T08:28:15.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.658+0000 7f52fda00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f80cb7e0 con 0x7f52f8099950 2024-03-29T08:28:15.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.658+0000 7f52fda00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f80cbac0 con 0x7f52f815e080 2024-03-29T08:28:15.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.658+0000 7f52fd000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f52f8099150 0x7f52f815c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43208/0 (socket says 172.21.15.46:43208) 2024-03-29T08:28:15.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.658+0000 7f52fd000640 1 -- 172.21.15.46:0/2794346349 learned_addr learned my addr 172.21.15.46:0/2794346349 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:28:15.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.658+0000 7f52f7e00640 1 --2- 172.21.15.46:0/2794346349 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f52f815e080 0x7f52f80ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.658+0000 7f52f7400640 1 -- 172.21.15.46:0/2794346349 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072328601 0 0) 0x7f52f80cb500 con 0x7f52f8099150 2024-03-29T08:28:15.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.658+0000 7f52f7400640 1 -- 172.21.15.46:0/2794346349 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52dc003530 con 0x7f52f8099150 2024-03-29T08:28:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.658+0000 7f52f7400640 1 -- 172.21.15.46:0/2794346349 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286685982 0 0) 0x7f52f80cb7e0 con 0x7f52f8099950 2024-03-29T08:28:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.658+0000 7f52f7400640 1 -- 172.21.15.46:0/2794346349 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52f80cb500 con 0x7f52f8099950 2024-03-29T08:28:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.658+0000 7f52f7e00640 1 -- 172.21.15.46:0/2794346349 >> v1:172.21.15.73:6789/0 conn(0x7f52f8099950 legacy=0x7f52f8098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.658+0000 7f52f7e00640 1 -- 172.21.15.46:0/2794346349 >> v1:172.21.15.46:6789/0 conn(0x7f52f8099150 legacy=0x7f52f815c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.658+0000 7f52f7e00640 1 -- 172.21.15.46:0/2794346349 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f80cbda0 con 0x7f52f815e080 2024-03-29T08:28:15.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.662+0000 7f52f7e00640 1 --2- 172.21.15.46:0/2794346349 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f52f815e080 0x7f52f80ce2d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f52e80097f0 tx=0x7f52e801c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=483604f104261155 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:15.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.662+0000 7f52f7400640 1 -- 172.21.15.46:0/2794346349 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e80aa610 con 0x7f52f815e080 2024-03-29T08:28:15.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.662+0000 7f52f7400640 1 -- 172.21.15.46:0/2794346349 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e80a6040 con 0x7f52f815e080 2024-03-29T08:28:15.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.662+0000 7f52f7400640 1 -- 172.21.15.46:0/2794346349 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e80aa990 con 0x7f52f815e080 2024-03-29T08:28:15.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.662+0000 7f52fda00640 1 -- 172.21.15.46:0/2794346349 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f52f815e080 msgr2=0x7f52f80ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.662+0000 7f52fda00640 1 --2- 172.21.15.46:0/2794346349 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f52f815e080 0x7f52f80ce2d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f52e80097f0 tx=0x7f52e801c4b0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.662+0000 7f52fda00640 1 -- 172.21.15.46:0/2794346349 shutdown_connections 2024-03-29T08:28:15.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.662+0000 7f52fda00640 1 --2- 172.21.15.46:0/2794346349 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f52f815e080 0x7f52f80ce2d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.662+0000 7f52fda00640 1 -- 172.21.15.46:0/2794346349 >> 172.21.15.46:0/2794346349 conn(0x7f52f80b1fe0 msgr2=0x7f52f80b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:15.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.662+0000 7f52fda00640 1 -- 172.21.15.46:0/2794346349 shutdown_connections 2024-03-29T08:28:15.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.662+0000 7f52fda00640 1 -- 172.21.15.46:0/2794346349 wait complete. 2024-03-29T08:28:15.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52fda00640 1 Processor -- start 2024-03-29T08:28:15.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52fda00640 1 -- start start 2024-03-29T08:28:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52fda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8099150 0x7f52f8158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52fda00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f8099950 0x7f52f81659d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52fda00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f52f815e080 0x7f52f814e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52fda00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f814eed0 con 0x7f52f8099150 2024-03-29T08:28:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52fda00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f8166670 con 0x7f52f8099950 2024-03-29T08:28:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52fda00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f81660b0 con 0x7f52f815e080 2024-03-29T08:28:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8099150 0x7f52f8158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8099150 0x7f52f8158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54890/0 (socket says 172.21.15.46:54890) 2024-03-29T08:28:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52f7e00640 1 -- 172.21.15.46:0/3827213704 learned_addr learned my addr 172.21.15.46:0/3827213704 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52fd000640 1 --2- 172.21.15.46:0/3827213704 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f52f815e080 0x7f52f814e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52ef400640 1 --2- 172.21.15.46:0/3827213704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f8099950 0x7f52f81659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52f7e00640 1 -- 172.21.15.46:0/3827213704 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f52f815e080 msgr2=0x7f52f814e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52f7e00640 1 --2- 172.21.15.46:0/3827213704 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f52f815e080 0x7f52f814e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52f7e00640 1 -- 172.21.15.46:0/3827213704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f8099950 msgr2=0x7f52f81659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52f7e00640 1 --2- 172.21.15.46:0/3827213704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f8099950 0x7f52f81659d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52f7e00640 1 -- 172.21.15.46:0/3827213704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e8009420 con 0x7f52f8099150 2024-03-29T08:28:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52f7e00640 1 --2- 172.21.15.46:0/3827213704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8099150 0x7f52f8158c40 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f52e80190c0 tx=0x7f52e801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52f4c00640 1 -- 172.21.15.46:0/3827213704 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e80be610 con 0x7f52f8099150 2024-03-29T08:28:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52f4c00640 1 -- 172.21.15.46:0/3827213704 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e80be790 con 0x7f52f8099150 2024-03-29T08:28:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52f4c00640 1 -- 172.21.15.46:0/3827213704 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e80beb40 con 0x7f52f8099150 2024-03-29T08:28:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52fda00640 1 -- 172.21.15.46:0/3827213704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f8166390 con 0x7f52f8099150 2024-03-29T08:28:15.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.666+0000 7f52fda00640 1 -- 172.21.15.46:0/3827213704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f8152d20 con 0x7f52f8099150 2024-03-29T08:28:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.670+0000 7f52fda00640 1 -- 172.21.15.46:0/3827213704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f80671b0 con 0x7f52f8099150 2024-03-29T08:28:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.670+0000 7f52f4c00640 1 -- 172.21.15.46:0/3827213704 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f52e80beca0 con 0x7f52f8099150 2024-03-29T08:28:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.670+0000 7f52f4c00640 1 --2- 172.21.15.46:0/3827213704 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52d4066690 0x7f52d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.670+0000 7f52f4c00640 1 -- 172.21.15.46:0/3827213704 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f52e811e6c0 con 0x7f52f8099150 2024-03-29T08:28:15.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.670+0000 7f52ef400640 1 --2- 172.21.15.46:0/3827213704 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52d4066690 0x7f52d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.670+0000 7f52ef400640 1 --2- 172.21.15.46:0/3827213704 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52d4066690 0x7f52d4068b50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f52f806d020 tx=0x7f52e0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:15.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.674+0000 7f52f4c00640 1 -- 172.21.15.46:0/3827213704 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f52e80eb9e0 con 0x7f52f8099150 2024-03-29T08:28:15.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.870+0000 7f52fda00640 1 -- 172.21.15.46:0/3827213704 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f52f81528a0 con 0x7f52f8099150 2024-03-29T08:28:15.873 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-03-29T08:28:15.870+0000 7fc16ae00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:28:15.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.878+0000 7f52f4c00640 1 -- 172.21.15.46:0/3827213704 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) v1 ==== 161+0+0 (secure 0 0 0) 0x7f52f81528a0 con 0x7f52f8099150 2024-03-29T08:28:15.882 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T08:28:15.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.882+0000 7f52fda00640 1 -- 172.21.15.46:0/3827213704 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52d4066690 msgr2=0x7f52d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.882+0000 7f52fda00640 1 --2- 172.21.15.46:0/3827213704 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52d4066690 0x7f52d4068b50 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f52f806d020 tx=0x7f52e0009040 comp rx=0 tx=0).stop 2024-03-29T08:28:15.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.882+0000 7f52fda00640 1 -- 172.21.15.46:0/3827213704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8099150 msgr2=0x7f52f8158c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.882+0000 7f52fda00640 1 --2- 172.21.15.46:0/3827213704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8099150 0x7f52f8158c40 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f52e80190c0 tx=0x7f52e801acf0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.882+0000 7f52fda00640 1 -- 172.21.15.46:0/3827213704 shutdown_connections 2024-03-29T08:28:15.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.882+0000 7f52fda00640 1 --2- 172.21.15.46:0/3827213704 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f52d4066690 0x7f52d4068b50 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.882+0000 7f52fda00640 1 --2- 172.21.15.46:0/3827213704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f8099950 0x7f52f81659d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.882+0000 7f52fda00640 1 --2- 172.21.15.46:0/3827213704 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f52f815e080 0x7f52f814e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.882+0000 7f52fda00640 1 --2- 172.21.15.46:0/3827213704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52f8099150 0x7f52f8158c40 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.882+0000 7f52fda00640 1 -- 172.21.15.46:0/3827213704 >> 172.21.15.46:0/3827213704 conn(0x7f52f80b1fe0 msgr2=0x7f52f8155550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:15.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.882+0000 7f52fda00640 1 -- 172.21.15.46:0/3827213704 shutdown_connections 2024-03-29T08:28:15.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.882+0000 7f52fda00640 1 -- 172.21.15.46:0/3827213704 wait complete. 2024-03-29T08:28:15.892 INFO:tasks.ceph.mds.c.smithi046.stderr: -6014> 2024-03-29T08:27:20.103+0000 7fe23a400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:28:15.905 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-03-29T08:28:15.939 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-03-29T08:28:15.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff91584a400 1 -- start start 2024-03-29T08:28:15.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff91584a400 1 Processor -- start 2024-03-29T08:28:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff91584a400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3aad800 0x5564e3b38580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff91584a400 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5564e3aad400 0x5564e3b38b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff91584a400 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564e3aad000 0x5564e3b39080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff91584a400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e3af3e00 con 0x5564e3aad800 2024-03-29T08:28:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff91584a400 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e3b7aa00 con 0x5564e3aad000 2024-03-29T08:28:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff91584a400 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564e3b7a640 con 0x5564e3aad400 2024-03-29T08:28:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff912e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3aad800 0x5564e3b38580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff912e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3aad800 0x5564e3b38580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54916/0 (socket says 172.21.15.46:54916) 2024-03-29T08:28:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff913800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564e3aad000 0x5564e3b39080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff912e00640 1 -- 172.21.15.46:0/4126705653 learned_addr learned my addr 172.21.15.46:0/4126705653 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:15.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff912400640 1 --2- 172.21.15.46:0/4126705653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5564e3aad400 0x5564e3b38b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff912e00640 1 -- 172.21.15.46:0/4126705653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5564e3aad400 msgr2=0x5564e3b38b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:15.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff912e00640 1 --2- 172.21.15.46:0/4126705653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5564e3aad400 0x5564e3b38b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff912e00640 1 -- 172.21.15.46:0/4126705653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564e3aad000 msgr2=0x5564e3b39080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:15.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff912e00640 1 --2- 172.21.15.46:0/4126705653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564e3aad000 0x5564e3b39080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff912e00640 1 -- 172.21.15.46:0/4126705653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564e3ae29c0 con 0x5564e3aad800 2024-03-29T08:28:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.950+0000 7ff913800640 1 --2- 172.21.15.46:0/4126705653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564e3aad000 0x5564e3b39080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:28:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff912e00640 1 --2- 172.21.15.46:0/4126705653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3aad800 0x5564e3b38580 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x5564e491cc90 tx=0x5564e49c4540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eae7f68c9a44289 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff90fc00640 1 -- 172.21.15.46:0/4126705653 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5564e3b01600 con 0x5564e3aad800 2024-03-29T08:28:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff90fc00640 1 -- 172.21.15.46:0/4126705653 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5564e3b0a900 con 0x5564e3aad800 2024-03-29T08:28:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff90fc00640 1 -- 172.21.15.46:0/4126705653 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5564e3b011e0 con 0x5564e3aad800 2024-03-29T08:28:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff91584a400 1 -- 172.21.15.46:0/4126705653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564e3ae2b60 con 0x5564e3aad800 2024-03-29T08:28:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff91584a400 1 -- 172.21.15.46:0/4126705653 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5564e3ae2340 con 0x5564e3aad800 2024-03-29T08:28:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff91584a400 4 waiting for MDS map... 2024-03-29T08:28:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff90fc00640 1 -- 172.21.15.46:0/4126705653 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x5564e3aecc40 con 0x5564e3aad800 2024-03-29T08:28:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff90fc00640 1 -- 172.21.15.46:0/4126705653 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 78) v1 ==== 2573+0+0 (secure 0 0 0) 0x5564e3afaf00 con 0x5564e3aad800 2024-03-29T08:28:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff91584a400 4 Got MDS map 78 2024-03-29T08:28:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff91584a400 10 main: JournalTool::main 2024-03-29T08:28:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff91584a400 4 main: JournalTool: connecting to RADOS... 2024-03-29T08:28:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff91584a400 1 Processor -- start 2024-03-29T08:28:15.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff91584a400 1 -- start start 2024-03-29T08:28:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff91584a400 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564e49de800 0x5564e3b39600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff91584a400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e49dec00 0x5564e3b39b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff91584a400 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5564e49df000 0x5564e3b3a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff91584a400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e3b7a500 con 0x5564e49dec00 2024-03-29T08:28:15.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff91584a400 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e3b7a3c0 con 0x5564e49de800 2024-03-29T08:28:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff91584a400 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564e3b7aa00 con 0x5564e49df000 2024-03-29T08:28:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff912e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e49dec00 0x5564e3b39b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff912e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e49dec00 0x5564e3b39b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54922/0 (socket says 172.21.15.46:54922) 2024-03-29T08:28:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff912e00640 1 -- 172.21.15.46:0/2732273559 learned_addr learned my addr 172.21.15.46:0/2732273559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff912400640 1 --2- 172.21.15.46:0/2732273559 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5564e49df000 0x5564e3b3a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff913800640 1 --2- 172.21.15.46:0/2732273559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564e49de800 0x5564e3b39600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff912e00640 1 -- 172.21.15.46:0/2732273559 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5564e49df000 msgr2=0x5564e3b3a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff912e00640 1 --2- 172.21.15.46:0/2732273559 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5564e49df000 0x5564e3b3a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff912e00640 1 -- 172.21.15.46:0/2732273559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564e49de800 msgr2=0x5564e3b39600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff912e00640 1 --2- 172.21.15.46:0/2732273559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564e49de800 0x5564e3b39600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff912e00640 1 -- 172.21.15.46:0/2732273559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564e3ae2000 con 0x5564e49dec00 2024-03-29T08:28:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff913800640 1 --2- 172.21.15.46:0/2732273559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564e49de800 0x5564e3b39600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff912400640 1 --2- 172.21.15.46:0/2732273559 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5564e49df000 0x5564e3b3a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:15.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff912e00640 1 --2- 172.21.15.46:0/2732273559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e49dec00 0x5564e3b39b80 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x5564e491d080 tx=0x5564e49c40c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f887aa414371a0e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff90d400640 1 -- 172.21.15.46:0/2732273559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5564e3b00f20 con 0x5564e49dec00 2024-03-29T08:28:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff90d400640 1 -- 172.21.15.46:0/2732273559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5564e4878300 con 0x5564e49dec00 2024-03-29T08:28:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff90d400640 1 -- 172.21.15.46:0/2732273559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5564e3b01600 con 0x5564e49dec00 2024-03-29T08:28:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff91584a400 1 -- 172.21.15.46:0/2732273559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e49dec00 msgr2=0x5564e3b39b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.954+0000 7ff91584a400 1 --2- 172.21.15.46:0/2732273559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e49dec00 0x5564e3b39b80 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x5564e491d080 tx=0x5564e49c40c0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 1 -- 172.21.15.46:0/2732273559 shutdown_connections 2024-03-29T08:28:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 1 --2- 172.21.15.46:0/2732273559 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5564e49df000 0x5564e3b3a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 1 --2- 172.21.15.46:0/2732273559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e49dec00 0x5564e3b39b80 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 1 --2- 172.21.15.46:0/2732273559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564e49de800 0x5564e3b39600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 1 -- 172.21.15.46:0/2732273559 >> 172.21.15.46:0/2732273559 conn(0x5564e3aacc00 msgr2=0x5564e3ad9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 1 -- 172.21.15.46:0/2732273559 shutdown_connections 2024-03-29T08:28:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 1 -- 172.21.15.46:0/2732273559 wait complete. 2024-03-29T08:28:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 1 Processor -- start 2024-03-29T08:28:15.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 1 -- start start 2024-03-29T08:28:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564e49de800 0x5564e3b3b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e49dec00 0x5564e3b3ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5564e49df000 0x5564e3b3a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e3b7a280 con 0x5564e49dec00 2024-03-29T08:28:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564e3b7a140 con 0x5564e49de800 2024-03-29T08:28:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564e3b7a000 con 0x5564e49df000 2024-03-29T08:28:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff912e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e49dec00 0x5564e3b3ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff912e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e49dec00 0x5564e3b3ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54938/0 (socket says 172.21.15.46:54938) 2024-03-29T08:28:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff912e00640 1 -- 172.21.15.46:0/3620962639 learned_addr learned my addr 172.21.15.46:0/3620962639 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff912e00640 1 -- 172.21.15.46:0/3620962639 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5564e49df000 msgr2=0x5564e3b3a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff912e00640 1 --2- 172.21.15.46:0/3620962639 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5564e49df000 0x5564e3b3a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff912e00640 1 -- 172.21.15.46:0/3620962639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564e49de800 msgr2=0x5564e3b3b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff912e00640 1 --2- 172.21.15.46:0/3620962639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564e49de800 0x5564e3b3b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff912e00640 1 -- 172.21.15.46:0/3620962639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564e3ae29c0 con 0x5564e49dec00 2024-03-29T08:28:15.965 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff912e00640 1 --2- 172.21.15.46:0/3620962639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e49dec00 0x5564e3b3ac00 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x5564e3d8c270 tx=0x5564e4880c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff90b600640 1 -- 172.21.15.46:0/3620962639 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5564e3b011e0 con 0x5564e49dec00 2024-03-29T08:28:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff90b600640 1 -- 172.21.15.46:0/3620962639 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5564e4878900 con 0x5564e49dec00 2024-03-29T08:28:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 1 -- 172.21.15.46:0/3620962639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5564e3ae2b60 con 0x5564e49dec00 2024-03-29T08:28:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff90b600640 1 -- 172.21.15.46:0/3620962639 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5564e3b00dc0 con 0x5564e49dec00 2024-03-29T08:28:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 1 -- 172.21.15.46:0/3620962639 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564e3ae2340 con 0x5564e49dec00 2024-03-29T08:28:15.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.958+0000 7ff91584a400 4 main: JournalTool: resolving pool 14 2024-03-29T08:28:15.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff90b600640 1 -- 172.21.15.46:0/3620962639 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x5564e3b78840 con 0x5564e49dec00 2024-03-29T08:28:15.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff90b600640 1 --2- 172.21.15.46:0/3620962639 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5564e4a36c00 0x5564e3b3b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff90b600640 1 -- 172.21.15.46:0/3620962639 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x5564e3aeca80 con 0x5564e49dec00 2024-03-29T08:28:15.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 4 main: JournalTool: creating IoCtx.. 2024-03-29T08:28:15.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 4 main: Executing for rank 0 2024-03-29T08:28:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 4 Filtering by inode '1099511627777' 2024-03-29T08:28:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 1 --2- 172.21.15.46:0/3620962639 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5564e49df400 0x5564e3b3a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 1 -- 172.21.15.46:0/3620962639 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v8 -- 0x5564e49df800 con 0x5564e49df400 2024-03-29T08:28:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff913800640 1 --2- 172.21.15.46:0/3620962639 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5564e4a36c00 0x5564e3b3b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff912400640 1 --2- 172.21.15.46:0/3620962639 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5564e49df400 0x5564e3b3a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff912400640 1 --2- 172.21.15.46:0/3620962639 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5564e49df400 0x5564e3b3a100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff913800640 1 --2- 172.21.15.46:0/3620962639 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5564e4a36c00 0x5564e3b3b700 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x5564e3b84270 tx=0x5564e4880cc0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff912400640 1 -- 172.21.15.46:0/3620962639 <== osd.1 v2:172.21.15.46:6824/27427 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x5564e4876b40 con 0x5564e49df400 2024-03-29T08:28:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 4 Pointer 400.00000000 is readable 2024-03-29T08:28:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 4 JournalScanner::scan: reading header object '200.00000000' 2024-03-29T08:28:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 1 -- 172.21.15.46:0/3620962639 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v8 -- 0x5564e49dfc00 con 0x5564e49df400 2024-03-29T08:28:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff912400640 1 -- 172.21.15.46:0/3620962639 <== osd.1 v2:172.21.15.46:6824/27427 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv29 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x5564e4876b40 con 0x5564e49df400 2024-03-29T08:28:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 10 Header 0x400000 0x405ee2 0x40625e 2024-03-29T08:28:15.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 10 Starting journal scan from offset 0x405ee2 2024-03-29T08:28:15.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 1 -- 172.21.15.46:0/3620962639 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 24290~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v8 -- 0x5564e4a37800 con 0x5564e49df400 2024-03-29T08:28:15.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff912400640 1 -- 172.21.15.46:0/3620962639 <== osd.1 v2:172.21.15.46:6824/27427 3 ==== osd_op_reply(3 200.00000001 [read 24290~892 out=892b] v0'0 uv26 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x5564e4876b40 con 0x5564e49df400 2024-03-29T08:28:15.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-03-29T08:28:15.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 10 Parsing data, 0x37c bytes available 2024-03-29T08:28:15.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 10 Attempting decode at 0x405ee2 2024-03-29T08:28:15.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 10 Consumed 0x37c bytes 2024-03-29T08:28:15.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 10 Valid entry at 0x405ee2 2024-03-29T08:28:15.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 1 --2- 172.21.15.46:0/3620962639 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5564e4a37c00 0x5564e3b39b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:15.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff91584a400 1 -- 172.21.15.46:0/3620962639 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v8 -- 0x5564e49df800 con 0x5564e4a37c00 2024-03-29T08:28:15.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.962+0000 7ff913800640 1 --2- 172.21.15.46:0/3620962639 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5564e4a37c00 0x5564e3b39b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:15.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff913800640 1 --2- 172.21.15.46:0/3620962639 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5564e4a37c00 0x5564e3b39b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:15.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff913800640 1 -- 172.21.15.46:0/3620962639 <== osd.6 v2:172.21.15.73:6824/27691 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5564e4876d80 con 0x5564e4a37c00 2024-03-29T08:28:15.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 4 Reached end of journal objects 2024-03-29T08:28:15.971 INFO:teuthology.orchestra.run.smithi046.stdout:Events by type: 2024-03-29T08:28:15.971 INFO:teuthology.orchestra.run.smithi046.stdout:Errors: 0 2024-03-29T08:28:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 4 Scanned objects, 0 missing, 1 valid 2024-03-29T08:28:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 4 Events scanned, 0 gaps 2024-03-29T08:28:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 4 Found 1 valid events 2024-03-29T08:28:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 4 Selected 0 events events for processing 2024-03-29T08:28:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 4 0 events 2024-03-29T08:28:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 1 -- 172.21.15.46:0/3620962639 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5564e49df400 msgr2=0x5564e3b3a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 1 --2- 172.21.15.46:0/3620962639 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5564e49df400 0x5564e3b3a100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 1 -- 172.21.15.46:0/3620962639 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5564e4a37c00 msgr2=0x5564e3b39b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 1 --2- 172.21.15.46:0/3620962639 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5564e4a37c00 0x5564e3b39b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 1 -- 172.21.15.46:0/3620962639 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5564e4a36c00 msgr2=0x5564e3b3b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 1 --2- 172.21.15.46:0/3620962639 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5564e4a36c00 0x5564e3b3b700 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x5564e3b84270 tx=0x5564e4880cc0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 1 -- 172.21.15.46:0/3620962639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e49dec00 msgr2=0x5564e3b3ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 1 --2- 172.21.15.46:0/3620962639 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e49dec00 0x5564e3b3ac00 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x5564e3d8c270 tx=0x5564e4880c00 comp rx=0 tx=0).stop 2024-03-29T08:28:15.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff912400640 1 -- 172.21.15.46:0/3620962639 reap_dead start 2024-03-29T08:28:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 1 -- 172.21.15.46:0/3620962639 shutdown_connections 2024-03-29T08:28:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 1 -- 172.21.15.46:0/3620962639 >> 172.21.15.46:0/3620962639 conn(0x5564e3aacc00 msgr2=0x5564e3ad9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 1 -- 172.21.15.46:0/3620962639 shutdown_connections 2024-03-29T08:28:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.966+0000 7ff91584a400 1 -- 172.21.15.46:0/3620962639 wait complete. 2024-03-29T08:28:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.970+0000 7ff91584a400 1 -- 172.21.15.46:0/4126705653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3aad800 msgr2=0x5564e3b38580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.970+0000 7ff91584a400 1 --2- 172.21.15.46:0/4126705653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3aad800 0x5564e3b38580 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x5564e491cc90 tx=0x5564e49c4540 comp rx=0 tx=0).stop 2024-03-29T08:28:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.970+0000 7ff91584a400 1 -- 172.21.15.46:0/4126705653 shutdown_connections 2024-03-29T08:28:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.970+0000 7ff91584a400 1 --2- 172.21.15.46:0/4126705653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5564e3aad000 0x5564e3b39080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.970+0000 7ff91584a400 1 --2- 172.21.15.46:0/4126705653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5564e3aad400 0x5564e3b38b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.970+0000 7ff91584a400 1 --2- 172.21.15.46:0/4126705653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564e3aad800 0x5564e3b38580 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.970+0000 7ff91584a400 1 -- 172.21.15.46:0/4126705653 >> 172.21.15.46:0/4126705653 conn(0x5564e3aadc00 msgr2=0x5564e3ad9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.970+0000 7ff91584a400 1 -- 172.21.15.46:0/4126705653 shutdown_connections 2024-03-29T08:28:15.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:15.970+0000 7ff91584a400 1 -- 172.21.15.46:0/4126705653 wait complete. 2024-03-29T08:28:15.977 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.071952, result: Events by type: Errors: 0 2024-03-29T08:28:15.977 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-03-29T08:28:16.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.062+0000 7f5cfcc48400 1 -- start start 2024-03-29T08:28:16.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.062+0000 7f5cfcc48400 1 Processor -- start 2024-03-29T08:28:16.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f2fc69800 0x556f2fcf4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556f2fc69400 0x556f2fcf4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f2fc69000 0x556f2fcf5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2fd36780 con 0x556f2fc69800 2024-03-29T08:28:16.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2fd36a00 con 0x556f2fc69000 2024-03-29T08:28:16.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f2fd36640 con 0x556f2fc69400 2024-03-29T08:28:16.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfa200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f2fc69800 0x556f2fcf4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfa200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f2fc69800 0x556f2fcf4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54944/0 (socket says 172.21.15.46:54944) 2024-03-29T08:28:16.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfa200640 1 -- 172.21.15.46:0/117620494 learned_addr learned my addr 172.21.15.46:0/117620494 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:16.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfac00640 1 --2- 172.21.15.46:0/117620494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f2fc69000 0x556f2fcf5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cf9800640 1 --2- 172.21.15.46:0/117620494 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556f2fc69400 0x556f2fcf4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfa200640 1 -- 172.21.15.46:0/117620494 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556f2fc69400 msgr2=0x556f2fcf4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:16.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfa200640 1 --2- 172.21.15.46:0/117620494 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556f2fc69400 0x556f2fcf4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfa200640 1 -- 172.21.15.46:0/117620494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f2fc69000 msgr2=0x556f2fcf5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:16.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfa200640 1 --2- 172.21.15.46:0/117620494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f2fc69000 0x556f2fcf5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfa200640 1 -- 172.21.15.46:0/117620494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f2fc9e9c0 con 0x556f2fc69800 2024-03-29T08:28:16.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfa200640 1 --2- 172.21.15.46:0/117620494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f2fc69800 0x556f2fcf4580 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x556f30ae6c00 tx=0x556f309c8120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1840f319bf2cee0c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:16.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cf7000640 1 -- 172.21.15.46:0/117620494 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556f2fcbd080 con 0x556f2fc69800 2024-03-29T08:28:16.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cf7000640 1 -- 172.21.15.46:0/117620494 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556f30a4a300 con 0x556f2fc69800 2024-03-29T08:28:16.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cf7000640 1 -- 172.21.15.46:0/117620494 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556f2fcbd600 con 0x556f2fc69800 2024-03-29T08:28:16.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 1 -- 172.21.15.46:0/117620494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f2fc9e1a0 con 0x556f2fc69800 2024-03-29T08:28:16.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 1 -- 172.21.15.46:0/117620494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x556f2fc9e9c0 con 0x556f2fc69800 2024-03-29T08:28:16.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 4 waiting for MDS map... 2024-03-29T08:28:16.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cf7000640 1 -- 172.21.15.46:0/117620494 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x556f2fca8c40 con 0x556f2fc69800 2024-03-29T08:28:16.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cf7000640 1 -- 172.21.15.46:0/117620494 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 78) v1 ==== 2573+0+0 (secure 0 0 0) 0x556f2fcb6f00 con 0x556f2fc69800 2024-03-29T08:28:16.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 4 Got MDS map 78 2024-03-29T08:28:16.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 10 main: JournalTool::main 2024-03-29T08:28:16.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 4 main: JournalTool: connecting to RADOS... 2024-03-29T08:28:16.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 1 Processor -- start 2024-03-29T08:28:16.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 1 -- start start 2024-03-29T08:28:16.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f30a4d800 0x556f2fcf5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f30a4dc00 0x556f2fcf5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556f30bd4000 0x556f2fcf6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2fcafe00 con 0x556f30a4dc00 2024-03-29T08:28:16.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2fd36a00 con 0x556f30a4d800 2024-03-29T08:28:16.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.066+0000 7f5cfcc48400 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f2fd36780 con 0x556f30bd4000 2024-03-29T08:28:16.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfa200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f30a4dc00 0x556f2fcf5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfa200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f30a4dc00 0x556f2fcf5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54958/0 (socket says 172.21.15.46:54958) 2024-03-29T08:28:16.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfa200640 1 -- 172.21.15.46:0/950776447 learned_addr learned my addr 172.21.15.46:0/950776447 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:16.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfa200640 1 -- 172.21.15.46:0/950776447 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556f30bd4000 msgr2=0x556f2fcf6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:16.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cf9800640 1 --2- 172.21.15.46:0/950776447 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556f30bd4000 0x556f2fcf6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfa200640 1 --2- 172.21.15.46:0/950776447 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556f30bd4000 0x556f2fcf6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfa200640 1 -- 172.21.15.46:0/950776447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f30a4d800 msgr2=0x556f2fcf5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:16.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfac00640 1 --2- 172.21.15.46:0/950776447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f30a4d800 0x556f2fcf5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfa200640 1 --2- 172.21.15.46:0/950776447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f30a4d800 0x556f2fcf5600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfa200640 1 -- 172.21.15.46:0/950776447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f2fc9eb60 con 0x556f30a4dc00 2024-03-29T08:28:16.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cf9800640 1 --2- 172.21.15.46:0/950776447 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556f30bd4000 0x556f2fcf6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:16.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfac00640 1 --2- 172.21.15.46:0/950776447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f30a4d800 0x556f2fcf5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:16.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfa200640 1 --2- 172.21.15.46:0/950776447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f30a4dc00 0x556f2fcf5b80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x556f30ae7020 tx=0x556f309c81e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb0ff5cbbd8d73b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:16.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cf4800640 1 -- 172.21.15.46:0/950776447 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556f2fcbd1e0 con 0x556f30a4dc00 2024-03-29T08:28:16.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cf4800640 1 -- 172.21.15.46:0/950776447 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556f30a4a300 con 0x556f30a4dc00 2024-03-29T08:28:16.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cf4800640 1 -- 172.21.15.46:0/950776447 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556f2fcbcf20 con 0x556f30a4dc00 2024-03-29T08:28:16.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfcc48400 1 -- 172.21.15.46:0/950776447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f30a4dc00 msgr2=0x556f2fcf5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfcc48400 1 --2- 172.21.15.46:0/950776447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f30a4dc00 0x556f2fcf5b80 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x556f30ae7020 tx=0x556f309c81e0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfcc48400 1 -- 172.21.15.46:0/950776447 shutdown_connections 2024-03-29T08:28:16.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfcc48400 1 --2- 172.21.15.46:0/950776447 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556f30bd4000 0x556f2fcf6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfcc48400 1 --2- 172.21.15.46:0/950776447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f30a4dc00 0x556f2fcf5b80 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfcc48400 1 --2- 172.21.15.46:0/950776447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f30a4d800 0x556f2fcf5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfcc48400 1 -- 172.21.15.46:0/950776447 >> 172.21.15.46:0/950776447 conn(0x556f2fc68c00 msgr2=0x556f2fc97800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:16.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfcc48400 1 -- 172.21.15.46:0/950776447 shutdown_connections 2024-03-29T08:28:16.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfcc48400 1 -- 172.21.15.46:0/950776447 wait complete. 2024-03-29T08:28:16.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfcc48400 1 Processor -- start 2024-03-29T08:28:16.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfcc48400 1 -- start start 2024-03-29T08:28:16.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfcc48400 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556f30a4dc00 0x556f2fcf7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfcc48400 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f30a4d800 0x556f2fcf6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfcc48400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f30bd5800 0x556f2fcf6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfcc48400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2fd36500 con 0x556f30bd5800 2024-03-29T08:28:16.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfcc48400 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f2fd363c0 con 0x556f30a4d800 2024-03-29T08:28:16.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfcc48400 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f2fd36280 con 0x556f30a4dc00 2024-03-29T08:28:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cf9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f30bd5800 0x556f2fcf6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cf9800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f30bd5800 0x556f2fcf6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54966/0 (socket says 172.21.15.46:54966) 2024-03-29T08:28:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cf9800640 1 -- 172.21.15.46:0/1750797495 learned_addr learned my addr 172.21.15.46:0/1750797495 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfac00640 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556f30a4dc00 0x556f2fcf7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfa200640 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f30a4d800 0x556f2fcf6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cf9800640 1 -- 172.21.15.46:0/1750797495 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556f30a4dc00 msgr2=0x556f2fcf7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cf9800640 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556f30a4dc00 0x556f2fcf7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cf9800640 1 -- 172.21.15.46:0/1750797495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f30a4d800 msgr2=0x556f2fcf6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cf9800640 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f30a4d800 0x556f2fcf6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cf9800640 1 -- 172.21.15.46:0/1750797495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f2fc9e1a0 con 0x556f30bd5800 2024-03-29T08:28:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.070+0000 7f5cfac00640 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556f30a4dc00 0x556f2fcf7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:28:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cf9800640 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f30bd5800 0x556f2fcf6680 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x556f30beacc0 tx=0x556f30a48ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cf2a00640 1 -- 172.21.15.46:0/1750797495 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556f2fcbd1e0 con 0x556f30bd5800 2024-03-29T08:28:16.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cf2a00640 1 -- 172.21.15.46:0/1750797495 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556f30a4b800 con 0x556f30bd5800 2024-03-29T08:28:16.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cf2a00640 1 -- 172.21.15.46:0/1750797495 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556f2fcbcf20 con 0x556f30bd5800 2024-03-29T08:28:16.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cfcc48400 1 -- 172.21.15.46:0/1750797495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556f2fc9e9c0 con 0x556f30bd5800 2024-03-29T08:28:16.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cfcc48400 1 -- 172.21.15.46:0/1750797495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f2fc9eb60 con 0x556f30bd5800 2024-03-29T08:28:16.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cfcc48400 4 main: JournalTool: resolving pool 14 2024-03-29T08:28:16.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cf2a00640 1 -- 172.21.15.46:0/1750797495 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x556f2fd34840 con 0x556f30bd5800 2024-03-29T08:28:16.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cf2a00640 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x556f2ff4f000 0x556f2fcf7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cf2a00640 1 -- 172.21.15.46:0/1750797495 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x556f2fca8a80 con 0x556f30bd5800 2024-03-29T08:28:16.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cfcc48400 4 main: JournalTool: creating IoCtx.. 2024-03-29T08:28:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cfcc48400 4 main: Executing for rank 0 2024-03-29T08:28:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cfcc48400 4 Filtering by inode '1099511627778' 2024-03-29T08:28:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cfcc48400 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x556f30bd5400 0x556f2fcf6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cfcc48400 1 -- 172.21.15.46:0/1750797495 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v8 -- 0x556f30bd5000 con 0x556f30bd5400 2024-03-29T08:28:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cfac00640 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x556f2ff4f000 0x556f2fcf7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cfa200640 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x556f30bd5400 0x556f2fcf6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cfa200640 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x556f30bd5400 0x556f2fcf6100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cfac00640 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x556f2ff4f000 0x556f2fcf7700 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x556f30a2e210 tx=0x556f3003e000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cfa200640 1 -- 172.21.15.46:0/1750797495 <== osd.1 v2:172.21.15.46:6824/27427 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x556f309c0b40 con 0x556f30bd5400 2024-03-29T08:28:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cfcc48400 4 Pointer 400.00000000 is readable 2024-03-29T08:28:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cfcc48400 4 JournalScanner::scan: reading header object '200.00000000' 2024-03-29T08:28:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.074+0000 7f5cfcc48400 1 -- 172.21.15.46:0/1750797495 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v8 -- 0x556f30bd4c00 con 0x556f30bd5400 2024-03-29T08:28:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfa200640 1 -- 172.21.15.46:0/1750797495 <== osd.1 v2:172.21.15.46:6824/27427 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv29 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x556f309c0b40 con 0x556f30bd5400 2024-03-29T08:28:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 10 Header 0x400000 0x405ee2 0x40625e 2024-03-29T08:28:16.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 10 Starting journal scan from offset 0x405ee2 2024-03-29T08:28:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 -- 172.21.15.46:0/1750797495 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 24290~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v8 -- 0x556f30bd4800 con 0x556f30bd5400 2024-03-29T08:28:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfa200640 1 -- 172.21.15.46:0/1750797495 <== osd.1 v2:172.21.15.46:6824/27427 3 ==== osd_op_reply(3 200.00000001 [read 24290~892 out=892b] v0'0 uv26 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x556f309c0b40 con 0x556f30bd5400 2024-03-29T08:28:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-03-29T08:28:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 10 Parsing data, 0x37c bytes available 2024-03-29T08:28:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 10 Attempting decode at 0x405ee2 2024-03-29T08:28:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 10 Consumed 0x37c bytes 2024-03-29T08:28:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 10 Valid entry at 0x405ee2 2024-03-29T08:28:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x556f30bd4400 0x556f2fcf5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 -- 172.21.15.46:0/1750797495 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e109) v8 -- 0x556f30bd4000 con 0x556f30bd4400 2024-03-29T08:28:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfac00640 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x556f30bd4400 0x556f2fcf5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfac00640 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x556f30bd4400 0x556f2fcf5b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfac00640 1 -- 172.21.15.46:0/1750797495 <== osd.6 v2:172.21.15.73:6824/27691 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x556f309c0d80 con 0x556f30bd4400 2024-03-29T08:28:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 4 Reached end of journal objects 2024-03-29T08:28:16.086 INFO:teuthology.orchestra.run.smithi046.stdout:Events by type: 2024-03-29T08:28:16.086 INFO:teuthology.orchestra.run.smithi046.stdout:Errors: 0 2024-03-29T08:28:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 4 Scanned objects, 0 missing, 1 valid 2024-03-29T08:28:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 4 Events scanned, 0 gaps 2024-03-29T08:28:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 4 Found 1 valid events 2024-03-29T08:28:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 4 Selected 0 events events for processing 2024-03-29T08:28:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 4 0 events 2024-03-29T08:28:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 -- 172.21.15.46:0/1750797495 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x556f30bd5400 msgr2=0x556f2fcf6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x556f30bd5400 0x556f2fcf6100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 -- 172.21.15.46:0/1750797495 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x556f30bd4400 msgr2=0x556f2fcf5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x556f30bd4400 0x556f2fcf5b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 -- 172.21.15.46:0/1750797495 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x556f2ff4f000 msgr2=0x556f2fcf7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x556f2ff4f000 0x556f2fcf7700 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x556f30a2e210 tx=0x556f3003e000 comp rx=0 tx=0).stop 2024-03-29T08:28:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 -- 172.21.15.46:0/1750797495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f30bd5800 msgr2=0x556f2fcf6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 --2- 172.21.15.46:0/1750797495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f30bd5800 0x556f2fcf6680 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x556f30beacc0 tx=0x556f30a48ea0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cf9800640 1 -- 172.21.15.46:0/1750797495 reap_dead start 2024-03-29T08:28:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cf9800640 1 -- 172.21.15.46:0/1750797495 reap_dead start 2024-03-29T08:28:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 -- 172.21.15.46:0/1750797495 shutdown_connections 2024-03-29T08:28:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 -- 172.21.15.46:0/1750797495 >> 172.21.15.46:0/1750797495 conn(0x556f2fc68c00 msgr2=0x556f2fc97800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 -- 172.21.15.46:0/1750797495 shutdown_connections 2024-03-29T08:28:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 -- 172.21.15.46:0/1750797495 wait complete. 2024-03-29T08:28:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 -- 172.21.15.46:0/117620494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f2fc69800 msgr2=0x556f2fcf4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 --2- 172.21.15.46:0/117620494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f2fc69800 0x556f2fcf4580 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x556f30ae6c00 tx=0x556f309c8120 comp rx=0 tx=0).stop 2024-03-29T08:28:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 -- 172.21.15.46:0/117620494 shutdown_connections 2024-03-29T08:28:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 --2- 172.21.15.46:0/117620494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f2fc69000 0x556f2fcf5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 --2- 172.21.15.46:0/117620494 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x556f2fc69400 0x556f2fcf4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 --2- 172.21.15.46:0/117620494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f2fc69800 0x556f2fcf4580 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 -- 172.21.15.46:0/117620494 >> 172.21.15.46:0/117620494 conn(0x556f2fc69c00 msgr2=0x556f2fc97000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 -- 172.21.15.46:0/117620494 shutdown_connections 2024-03-29T08:28:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.078+0000 7f5cfcc48400 1 -- 172.21.15.46:0/117620494 wait complete. 2024-03-29T08:28:16.090 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.112671, result: Events by type: Errors: 0 2024-03-29T08:28:16.090 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-03-29T08:28:16.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 Processor -- start 2024-03-29T08:28:16.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 -- start start 2024-03-29T08:28:16.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f079e420b0 0x55f079e2a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f079e439a0 0x55f079ecc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f079ecd800 0x55f079f645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f079da41e0 con 0x55f079e420b0 2024-03-29T08:28:16.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f079d3f960 con 0x55f079ecd800 2024-03-29T08:28:16.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f079d3f600 con 0x55f079e439a0 2024-03-29T08:28:16.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f079e420b0 0x55f079e2a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f079e420b0 0x55f079e2a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54968/0 (socket says 172.21.15.46:54968) 2024-03-29T08:28:16.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403b400640 1 -- 172.21.15.46:0/1460429560 learned_addr learned my addr 172.21.15.46:0/1460429560 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:16.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403aa00640 1 --2- 172.21.15.46:0/1460429560 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f079e439a0 0x55f079ecc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403be00640 1 --2- 172.21.15.46:0/1460429560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f079ecd800 0x55f079f645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403b400640 1 -- 172.21.15.46:0/1460429560 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f079e439a0 msgr2=0x55f079ecc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:16.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403b400640 1 --2- 172.21.15.46:0/1460429560 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f079e439a0 0x55f079ecc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403b400640 1 -- 172.21.15.46:0/1460429560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f079ecd800 msgr2=0x55f079f645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:16.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403b400640 1 --2- 172.21.15.46:0/1460429560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f079ecd800 0x55f079f645d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403b400640 1 -- 172.21.15.46:0/1460429560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c005d30 con 0x55f079e420b0 2024-03-29T08:28:16.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403b400640 1 --2- 172.21.15.46:0/1460429560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f079e420b0 0x55f079e2a1d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f402c00a530 tx=0x7f402c003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79ad73aaee8e61d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:16.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f4039600640 1 -- 172.21.15.46:0/1460429560 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c0b4410 con 0x55f079e420b0 2024-03-29T08:28:16.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f4039600640 1 -- 172.21.15.46:0/1460429560 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c0b4570 con 0x55f079e420b0 2024-03-29T08:28:16.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f4039600640 1 -- 172.21.15.46:0/1460429560 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c0b48e0 con 0x55f079e420b0 2024-03-29T08:28:16.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 -- 172.21.15.46:0/1460429560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f079e420b0 msgr2=0x55f079e2a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 --2- 172.21.15.46:0/1460429560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f079e420b0 0x55f079e2a1d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f402c00a530 tx=0x7f402c003030 comp rx=0 tx=0).stop 2024-03-29T08:28:16.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 -- 172.21.15.46:0/1460429560 shutdown_connections 2024-03-29T08:28:16.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 --2- 172.21.15.46:0/1460429560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f079ecd800 0x55f079f645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 --2- 172.21.15.46:0/1460429560 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f079e439a0 0x55f079ecc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 --2- 172.21.15.46:0/1460429560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f079e420b0 0x55f079e2a1d0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 -- 172.21.15.46:0/1460429560 >> 172.21.15.46:0/1460429560 conn(0x55f079e24fa0 msgr2=0x55f079e3eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:16.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 -- 172.21.15.46:0/1460429560 shutdown_connections 2024-03-29T08:28:16.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 -- 172.21.15.46:0/1460429560 wait complete. 2024-03-29T08:28:16.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 Processor -- start 2024-03-29T08:28:16.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.134+0000 7f403e2a8080 1 -- start start 2024-03-29T08:28:16.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403e2a8080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f079e420b0 0x55f079eccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403e2a8080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f079e439a0 0x55f079ec7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403e2a8080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f079ecd800 0x55f079ec2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403e2a8080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f079d36690 con 0x55f079e439a0 2024-03-29T08:28:16.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403e2a8080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f079ec6b20 con 0x55f079ecd800 2024-03-29T08:28:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403e2a8080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f079ec6dc0 con 0x55f079e420b0 2024-03-29T08:28:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f079e439a0 0x55f079ec7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f079e439a0 0x55f079ec7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54980/0 (socket says 172.21.15.46:54980) 2024-03-29T08:28:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403aa00640 1 -- 172.21.15.46:0/1440741185 learned_addr learned my addr 172.21.15.46:0/1440741185 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403b400640 1 --2- 172.21.15.46:0/1440741185 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f079e420b0 0x55f079eccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403be00640 1 --2- 172.21.15.46:0/1440741185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f079ecd800 0x55f079ec2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403aa00640 1 -- 172.21.15.46:0/1440741185 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f079e420b0 msgr2=0x55f079eccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403aa00640 1 --2- 172.21.15.46:0/1440741185 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55f079e420b0 0x55f079eccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403aa00640 1 -- 172.21.15.46:0/1440741185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f079ecd800 msgr2=0x55f079ec2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403aa00640 1 --2- 172.21.15.46:0/1440741185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f079ecd800 0x55f079ec2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403aa00640 1 -- 172.21.15.46:0/1440741185 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f079d4e8c0 con 0x55f079e439a0 2024-03-29T08:28:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403be00640 1 --2- 172.21.15.46:0/1440741185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f079ecd800 0x55f079ec2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403aa00640 1 --2- 172.21.15.46:0/1440741185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f079e439a0 0x55f079ec7a40 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f402800c620 tx=0x7f402800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f4033e00640 1 -- 172.21.15.46:0/1440741185 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402800b740 con 0x55f079e439a0 2024-03-29T08:28:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f4033e00640 1 -- 172.21.15.46:0/1440741185 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402800b8a0 con 0x55f079e439a0 2024-03-29T08:28:16.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f4033e00640 1 -- 172.21.15.46:0/1440741185 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402800bc10 con 0x55f079e439a0 2024-03-29T08:28:16.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403e2a8080 1 -- 172.21.15.46:0/1440741185 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f079d41d00 con 0x55f079e439a0 2024-03-29T08:28:16.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f403e2a8080 1 -- 172.21.15.46:0/1440741185 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f079d41760 con 0x55f079e439a0 2024-03-29T08:28:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.138+0000 7f4033e00640 1 -- 172.21.15.46:0/1440741185 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f402800a040 con 0x55f079e439a0 2024-03-29T08:28:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.142+0000 7f4033e00640 1 --2- 172.21.15.46:0/1440741185 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f401c0650a0 0x7f401c067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.142+0000 7f4033e00640 1 -- 172.21.15.46:0/1440741185 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4028083710 con 0x55f079e439a0 2024-03-29T08:28:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.142+0000 7f403b400640 1 --2- 172.21.15.46:0/1440741185 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f401c0650a0 0x7f401c067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.142+0000 7f403e2a8080 1 --2- 172.21.15.46:0/1440741185 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x55f079f6b430 0x55f079f6b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.142+0000 7f403e2a8080 1 -- 172.21.15.46:0/1440741185 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109) v8 -- 0x55f079ec2fa0 con 0x55f079f6b430 2024-03-29T08:28:16.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.142+0000 7f403be00640 1 --2- 172.21.15.46:0/1440741185 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x55f079f6b430 0x55f079f6b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.142+0000 7f403b400640 1 --2- 172.21.15.46:0/1440741185 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f401c0650a0 0x7f401c067560 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f402c0b9000 tx=0x7f402c0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:16.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.142+0000 7f403be00640 1 --2- 172.21.15.46:0/1440741185 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x55f079f6b430 0x55f079f6b840 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:16.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.146+0000 7f403be00640 1 -- 172.21.15.46:0/1440741185 <== osd.5 v2:172.21.15.73:6808/27690 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v109'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f403400e1d0 con 0x55f079f6b430 2024-03-29T08:28:16.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.146+0000 7f403e2a8080 1 -- 172.21.15.46:0/1440741185 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x55f079f6b430 msgr2=0x55f079f6b840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.146+0000 7f403e2a8080 1 --2- 172.21.15.46:0/1440741185 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x55f079f6b430 0x55f079f6b840 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.146+0000 7f403e2a8080 1 -- 172.21.15.46:0/1440741185 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f401c0650a0 msgr2=0x7f401c067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.146+0000 7f403e2a8080 1 --2- 172.21.15.46:0/1440741185 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f401c0650a0 0x7f401c067560 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f402c0b9000 tx=0x7f402c0a7000 comp rx=0 tx=0).stop 2024-03-29T08:28:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.146+0000 7f403e2a8080 1 -- 172.21.15.46:0/1440741185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f079e439a0 msgr2=0x55f079ec7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.146+0000 7f403e2a8080 1 --2- 172.21.15.46:0/1440741185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f079e439a0 0x55f079ec7a40 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f402800c620 tx=0x7f402800cae0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.146+0000 7f403be00640 1 -- 172.21.15.46:0/1440741185 reap_dead start 2024-03-29T08:28:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.146+0000 7f403e2a8080 1 -- 172.21.15.46:0/1440741185 shutdown_connections 2024-03-29T08:28:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.146+0000 7f403e2a8080 1 -- 172.21.15.46:0/1440741185 >> 172.21.15.46:0/1440741185 conn(0x55f079e24fa0 msgr2=0x55f079e297c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.146+0000 7f403e2a8080 1 -- 172.21.15.46:0/1440741185 shutdown_connections 2024-03-29T08:28:16.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.146+0000 7f403e2a8080 1 -- 172.21.15.46:0/1440741185 wait complete. 2024-03-29T08:28:16.155 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-29T08:28:16.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.334+0000 7fb083400640 1 Processor -- start 2024-03-29T08:28:16.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.334+0000 7fb083400640 1 -- start start 2024-03-29T08:28:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.338+0000 7fb083400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb07c15cad0 0x7fb07c06eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.338+0000 7fb083400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb07c06a4a0 0x7fb07c06d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.338+0000 7fb083400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb07c15d080 con 0x7fb07c06eff0 2024-03-29T08:28:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.338+0000 7fb083400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb07c15dd00 con 0x7fb07c15cad0 2024-03-29T08:28:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.338+0000 7fb083400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c06f480 con 0x7fb07c06a4a0 2024-03-29T08:28:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.338+0000 7fb073e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb07c06eff0 0x7fb07c069d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43240/0 (socket says 172.21.15.46:43240) 2024-03-29T08:28:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.338+0000 7fb073e00640 1 -- 172.21.15.46:0/2219713069 learned_addr learned my addr 172.21.15.46:0/2219713069 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:28:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.338+0000 7fb080a00640 1 --2- 172.21.15.46:0/2219713069 >> v2:172.21.15.73:3300/0 conn(0x7fb07c15cad0 0x7fb07c06eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.338+0000 7fb081400640 1 --2- 172.21.15.46:0/2219713069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb07c06a4a0 0x7fb07c06d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.338+0000 7fb07be00640 1 -- 172.21.15.46:0/2219713069 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 110184873 0 0) 0x7fb07c15d080 con 0x7fb07c06eff0 2024-03-29T08:28:16.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.338+0000 7fb07be00640 1 -- 172.21.15.46:0/2219713069 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb060003530 con 0x7fb07c06eff0 2024-03-29T08:28:16.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.338+0000 7fb07be00640 1 -- 172.21.15.46:0/2219713069 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2199527930 0 0) 0x7fb060003530 con 0x7fb07c06eff0 2024-03-29T08:28:16.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.338+0000 7fb081400640 1 -- 172.21.15.46:0/2219713069 >> v2:172.21.15.73:3300/0 conn(0x7fb07c15cad0 msgr2=0x7fb07c06eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:16.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.338+0000 7fb081400640 1 --2- 172.21.15.46:0/2219713069 >> v2:172.21.15.73:3300/0 conn(0x7fb07c15cad0 0x7fb07c06eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.338+0000 7fb081400640 1 -- 172.21.15.46:0/2219713069 >> v1:172.21.15.46:6789/0 conn(0x7fb07c06eff0 legacy=0x7fb07c069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.338+0000 7fb081400640 1 -- 172.21.15.46:0/2219713069 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c15d360 con 0x7fb07c06a4a0 2024-03-29T08:28:16.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.338+0000 7fb080a00640 1 --2- 172.21.15.46:0/2219713069 >> v2:172.21.15.73:3300/0 conn(0x7fb07c15cad0 0x7fb07c06eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:28:16.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.342+0000 7fb081400640 1 --2- 172.21.15.46:0/2219713069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb07c06a4a0 0x7fb07c06d470 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb074009d90 tx=0x7fb07401c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adc093e0c5595fcf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:16.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.342+0000 7fb07be00640 1 -- 172.21.15.46:0/2219713069 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0640016a0 con 0x7fb07c06a4a0 2024-03-29T08:28:16.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.342+0000 7fb07be00640 1 -- 172.21.15.46:0/2219713069 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0740a5510 con 0x7fb07c06a4a0 2024-03-29T08:28:16.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.342+0000 7fb07be00640 1 -- 172.21.15.46:0/2219713069 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0740a5880 con 0x7fb07c06a4a0 2024-03-29T08:28:16.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.342+0000 7fb083400640 1 -- 172.21.15.46:0/2219713069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb07c06a4a0 msgr2=0x7fb07c06d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.342+0000 7fb083400640 1 --2- 172.21.15.46:0/2219713069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb07c06a4a0 0x7fb07c06d470 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb074009d90 tx=0x7fb07401c800 comp rx=0 tx=0).stop 2024-03-29T08:28:16.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.342+0000 7fb083400640 1 -- 172.21.15.46:0/2219713069 shutdown_connections 2024-03-29T08:28:16.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.342+0000 7fb083400640 1 --2- 172.21.15.46:0/2219713069 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb07c06a4a0 0x7fb07c06d470 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.342+0000 7fb083400640 1 --2- 172.21.15.46:0/2219713069 >> v2:172.21.15.73:3300/0 conn(0x7fb07c15cad0 0x7fb07c06eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.342+0000 7fb083400640 1 -- 172.21.15.46:0/2219713069 >> 172.21.15.46:0/2219713069 conn(0x7fb07c098a40 msgr2=0x7fb07c0b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.342+0000 7fb083400640 1 -- 172.21.15.46:0/2219713069 shutdown_connections 2024-03-29T08:28:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.342+0000 7fb083400640 1 -- 172.21.15.46:0/2219713069 wait complete. 2024-03-29T08:28:16.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.342+0000 7fb083400640 1 Processor -- start 2024-03-29T08:28:16.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.342+0000 7fb083400640 1 -- start start 2024-03-29T08:28:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb083400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb07c06a4a0 0x7fb07c150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb083400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb07c06eff0 0x7fb07c14efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb083400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb07c15cad0 0x7fb07c0af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb083400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c0b08a0 con 0x7fb07c06eff0 2024-03-29T08:28:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb083400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c150910 con 0x7fb07c06a4a0 2024-03-29T08:28:16.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb083400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c15a7c0 con 0x7fb07c15cad0 2024-03-29T08:28:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb073e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb07c06eff0 0x7fb07c14efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb073e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb07c06eff0 0x7fb07c14efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54996/0 (socket says 172.21.15.46:54996) 2024-03-29T08:28:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb073e00640 1 -- 172.21.15.46:0/367349371 learned_addr learned my addr 172.21.15.46:0/367349371 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb080a00640 1 --2- 172.21.15.46:0/367349371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb07c06a4a0 0x7fb07c150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb081400640 1 --2- 172.21.15.46:0/367349371 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb07c15cad0 0x7fb07c0af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb073e00640 1 -- 172.21.15.46:0/367349371 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb07c15cad0 msgr2=0x7fb07c0af900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb073e00640 1 --2- 172.21.15.46:0/367349371 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb07c15cad0 0x7fb07c0af900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb073e00640 1 -- 172.21.15.46:0/367349371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb07c06a4a0 msgr2=0x7fb07c150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb073e00640 1 --2- 172.21.15.46:0/367349371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb07c06a4a0 0x7fb07c150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb073e00640 1 -- 172.21.15.46:0/367349371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06c009420 con 0x7fb07c06eff0 2024-03-29T08:28:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb080a00640 1 --2- 172.21.15.46:0/367349371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb07c06a4a0 0x7fb07c150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:28:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb073e00640 1 --2- 172.21.15.46:0/367349371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb07c06eff0 0x7fb07c14efd0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fb064007fd0 tx=0x7fb06400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:16.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb079600640 1 -- 172.21.15.46:0/367349371 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0640072d0 con 0x7fb07c06eff0 2024-03-29T08:28:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb079600640 1 -- 172.21.15.46:0/367349371 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb064007450 con 0x7fb07c06eff0 2024-03-29T08:28:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb083400640 1 -- 172.21.15.46:0/367349371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb074009a70 con 0x7fb07c06eff0 2024-03-29T08:28:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb079600640 1 -- 172.21.15.46:0/367349371 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0640047f0 con 0x7fb07c06eff0 2024-03-29T08:28:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb083400640 1 -- 172.21.15.46:0/367349371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb07c0b02e0 con 0x7fb07c06eff0 2024-03-29T08:28:16.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb083400640 1 -- 172.21.15.46:0/367349371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb07c0cd8f0 con 0x7fb07c06eff0 2024-03-29T08:28:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.346+0000 7fb079600640 1 -- 172.21.15.46:0/367349371 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb064004950 con 0x7fb07c06eff0 2024-03-29T08:28:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.350+0000 7fb079600640 1 --2- 172.21.15.46:0/367349371 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb058066690 0x7fb058068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.350+0000 7fb079600640 1 -- 172.21.15.46:0/367349371 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb064083900 con 0x7fb07c06eff0 2024-03-29T08:28:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.350+0000 7fb080a00640 1 --2- 172.21.15.46:0/367349371 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb058066690 0x7fb058068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:16.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.350+0000 7fb080a00640 1 --2- 172.21.15.46:0/367349371 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb058066690 0x7fb058068b50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb07c152c10 tx=0x7fb06c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:16.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.354+0000 7fb079600640 1 -- 172.21.15.46:0/367349371 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb064050b40 con 0x7fb07c06eff0 2024-03-29T08:28:16.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.550+0000 7fb083400640 1 -- 172.21.15.46:0/367349371 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fb07c09d700 con 0x7fb07c06eff0 2024-03-29T08:28:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.882+0000 7fb079600640 1 -- 172.21.15.46:0/367349371 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v79) v1 ==== 162+0+0 (secure 0 0 0) 0x7fb064018020 con 0x7fb07c06eff0 2024-03-29T08:28:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-29T08:28:16.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.882+0000 7fb083400640 1 -- 172.21.15.46:0/367349371 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb058066690 msgr2=0x7fb058068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.882+0000 7fb083400640 1 --2- 172.21.15.46:0/367349371 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb058066690 0x7fb058068b50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb07c152c10 tx=0x7fb06c008040 comp rx=0 tx=0).stop 2024-03-29T08:28:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.882+0000 7fb083400640 1 -- 172.21.15.46:0/367349371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb07c06eff0 msgr2=0x7fb07c14efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.882+0000 7fb083400640 1 --2- 172.21.15.46:0/367349371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb07c06eff0 0x7fb07c14efd0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fb064007fd0 tx=0x7fb06400cbe0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.882+0000 7fb083400640 1 -- 172.21.15.46:0/367349371 shutdown_connections 2024-03-29T08:28:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.882+0000 7fb083400640 1 --2- 172.21.15.46:0/367349371 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb058066690 0x7fb058068b50 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.882+0000 7fb083400640 1 --2- 172.21.15.46:0/367349371 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb07c15cad0 0x7fb07c0af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.882+0000 7fb083400640 1 --2- 172.21.15.46:0/367349371 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb07c06eff0 0x7fb07c14efd0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.882+0000 7fb083400640 1 --2- 172.21.15.46:0/367349371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb07c06a4a0 0x7fb07c150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.882+0000 7fb083400640 1 -- 172.21.15.46:0/367349371 >> 172.21.15.46:0/367349371 conn(0x7fb07c098a40 msgr2=0x7fb07c0b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.882+0000 7fb083400640 1 -- 172.21.15.46:0/367349371 shutdown_connections 2024-03-29T08:28:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:16.882+0000 7fb083400640 1 -- 172.21.15.46:0/367349371 wait complete. 2024-03-29T08:28:16.910 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:28:17.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.046+0000 7fa717200640 1 Processor -- start 2024-03-29T08:28:17.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.046+0000 7fa717200640 1 -- start start 2024-03-29T08:28:17.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.046+0000 7fa717200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7100cf3c0 0x7fa710165660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.046+0000 7fa717200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7100b0ab0 con 0x7fa71015ad20 2024-03-29T08:28:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.046+0000 7fa717200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7100b0d90 con 0x7fa71015a940 2024-03-29T08:28:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.046+0000 7fa717200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7100b1070 con 0x7fa7100cf3c0 2024-03-29T08:28:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.050+0000 7fa715200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7100cf3c0 0x7fa710165660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.050+0000 7fa715200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7100cf3c0 0x7fa710165660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34600/0 (socket says 172.21.15.46:34600) 2024-03-29T08:28:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.050+0000 7fa70f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa71015ad20 0x7fa7100cecb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36088/0 (socket says 172.21.15.46:36088) 2024-03-29T08:28:17.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.050+0000 7fa715200640 1 -- 172.21.15.46:0/1407955202 learned_addr learned my addr 172.21.15.46:0/1407955202 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.050+0000 7fa70ea00640 1 -- 172.21.15.46:0/1407955202 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1336333980 0 0) 0x7fa7100b0d90 con 0x7fa71015a940 2024-03-29T08:28:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.050+0000 7fa70ea00640 1 -- 172.21.15.46:0/1407955202 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f4003530 con 0x7fa71015a940 2024-03-29T08:28:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.050+0000 7fa715200640 1 -- 172.21.15.46:0/1407955202 >> v1:172.21.15.73:6789/0 conn(0x7fa71015a940 legacy=0x7fa7100afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.050+0000 7fa715200640 1 -- 172.21.15.46:0/1407955202 >> v1:172.21.15.46:6789/0 conn(0x7fa71015ad20 legacy=0x7fa7100cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.050+0000 7fa70ea00640 1 -- 172.21.15.46:0/1407955202 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217128895 0 0) 0x7fa7100b0ab0 con 0x7fa71015ad20 2024-03-29T08:28:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.050+0000 7fa715200640 1 -- 172.21.15.46:0/1407955202 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7100b1350 con 0x7fa7100cf3c0 2024-03-29T08:28:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.050+0000 7fa715200640 1 --2- 172.21.15.46:0/1407955202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7100cf3c0 0x7fa710165660 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fa708009e50 tx=0x7fa70801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf98f9a949fd4cee server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.050+0000 7fa70ea00640 1 -- 172.21.15.46:0/1407955202 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080a5850 con 0x7fa7100cf3c0 2024-03-29T08:28:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.050+0000 7fa70ea00640 1 -- 172.21.15.46:0/1407955202 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7080a59b0 con 0x7fa7100cf3c0 2024-03-29T08:28:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.050+0000 7fa70ea00640 1 -- 172.21.15.46:0/1407955202 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080a5d40 con 0x7fa7100cf3c0 2024-03-29T08:28:17.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa717200640 1 -- 172.21.15.46:0/1407955202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7100cf3c0 msgr2=0x7fa710165660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:17.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa717200640 1 --2- 172.21.15.46:0/1407955202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7100cf3c0 0x7fa710165660 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fa708009e50 tx=0x7fa70801c970 comp rx=0 tx=0).stop 2024-03-29T08:28:17.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa717200640 1 -- 172.21.15.46:0/1407955202 shutdown_connections 2024-03-29T08:28:17.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa717200640 1 --2- 172.21.15.46:0/1407955202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7100cf3c0 0x7fa710165660 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:17.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa717200640 1 -- 172.21.15.46:0/1407955202 >> 172.21.15.46:0/1407955202 conn(0x7fa710089fd0 msgr2=0x7fa71008a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:17.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa717200640 1 -- 172.21.15.46:0/1407955202 shutdown_connections 2024-03-29T08:28:17.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa717200640 1 -- 172.21.15.46:0/1407955202 wait complete. 2024-03-29T08:28:17.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa717200640 1 Processor -- start 2024-03-29T08:28:17.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa717200640 1 -- start start 2024-03-29T08:28:17.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa717200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7100cf3c0 0x7fa71014dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:17.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa717200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa71015a940 0x7fa71006b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:17.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa717200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71015ad20 0x7fa71006c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa717200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa710150140 con 0x7fa71015ad20 2024-03-29T08:28:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa717200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71014fe60 con 0x7fa7100cf3c0 2024-03-29T08:28:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa717200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71006bb70 con 0x7fa71015a940 2024-03-29T08:28:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa715200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71015ad20 0x7fa71006c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa70f400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa71015a940 0x7fa71006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa70fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7100cf3c0 0x7fa71014dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa715200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71015ad20 0x7fa71006c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34602/0 (socket says 172.21.15.46:34602) 2024-03-29T08:28:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa70fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7100cf3c0 0x7fa71014dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47846/0 (socket says 172.21.15.46:47846) 2024-03-29T08:28:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.054+0000 7fa715200640 1 -- 172.21.15.46:0/3145867223 learned_addr learned my addr 172.21.15.46:0/3145867223 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.058+0000 7fa715200640 1 -- 172.21.15.46:0/3145867223 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa71015a940 msgr2=0x7fa71006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.058+0000 7fa715200640 1 --2- 172.21.15.46:0/3145867223 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa71015a940 0x7fa71006b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.058+0000 7fa715200640 1 -- 172.21.15.46:0/3145867223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7100cf3c0 msgr2=0x7fa71014dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.058+0000 7fa715200640 1 --2- 172.21.15.46:0/3145867223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7100cf3c0 0x7fa71014dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.058+0000 7fa715200640 1 -- 172.21.15.46:0/3145867223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa708009b30 con 0x7fa71015ad20 2024-03-29T08:28:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.058+0000 7fa715200640 1 --2- 172.21.15.46:0/3145867223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71015ad20 0x7fa71006c650 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa708009e50 tx=0x7fa708019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.058+0000 7fa707e00640 1 -- 172.21.15.46:0/3145867223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080b8670 con 0x7fa71015ad20 2024-03-29T08:28:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.058+0000 7fa707e00640 1 -- 172.21.15.46:0/3145867223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7080b87d0 con 0x7fa71015ad20 2024-03-29T08:28:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.058+0000 7fa717200640 1 -- 172.21.15.46:0/3145867223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa71006be50 con 0x7fa71015ad20 2024-03-29T08:28:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.058+0000 7fa707e00640 1 -- 172.21.15.46:0/3145867223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7080b8b40 con 0x7fa71015ad20 2024-03-29T08:28:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.058+0000 7fa717200640 1 -- 172.21.15.46:0/3145867223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71006b660 con 0x7fa71015ad20 2024-03-29T08:28:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.058+0000 7fa717200640 1 -- 172.21.15.46:0/3145867223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa71006b8b0 con 0x7fa71015ad20 2024-03-29T08:28:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.058+0000 7fa707e00640 1 -- 172.21.15.46:0/3145867223 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa7080b8ca0 con 0x7fa71015ad20 2024-03-29T08:28:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.058+0000 7fa707e00640 1 --2- 172.21.15.46:0/3145867223 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6f0066690 0x7fa6f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.058+0000 7fa707e00640 1 -- 172.21.15.46:0/3145867223 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa70811f740 con 0x7fa71015ad20 2024-03-29T08:28:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.058+0000 7fa70fe00640 1 --2- 172.21.15.46:0/3145867223 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6f0066690 0x7fa6f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:17.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.062+0000 7fa70fe00640 1 --2- 172.21.15.46:0/3145867223 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6f0066690 0x7fa6f0068b50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa7000017c0 tx=0x7fa700009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:17.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.062+0000 7fa707e00640 1 -- 172.21.15.46:0/3145867223 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa7080ecac0 con 0x7fa71015ad20 2024-03-29T08:28:17.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.258+0000 7fa717200640 1 -- 172.21.15.46:0/3145867223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7100cf190 con 0x7fa71015ad20 2024-03-29T08:28:17.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.258+0000 7fa707e00640 1 -- 172.21.15.46:0/3145867223 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4735 (secure 0 0 0) 0x7fa7080f00e0 con 0x7fa71015ad20 2024-03-29T08:28:17.263 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:28:17.263 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4230030957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4230030957},{"type":"v1","addr":"172.21.15.73:6836","nonce":4230030957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/297138217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":297138217},{"type":"v1","addr":"172.21.15.73:6837","nonce":297138217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3856305332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3856305332},{"type":"v1","addr":"172.21.15.46:6833","nonce":3856305332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:27:18.583223+0000","modified":"2024-03-29T08:28:16.885668+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8739":{"gid":8739,"name":"c","rank":0,"incarnation":80,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6835/1455003836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1455003836},{"type":"v1","addr":"172.21.15.46:6835","nonce":1455003836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-29T08:28:17.263 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 80 2024-03-29T08:28:17.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.262+0000 7fa717200640 1 -- 172.21.15.46:0/3145867223 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6f0066690 msgr2=0x7fa6f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:17.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.262+0000 7fa717200640 1 --2- 172.21.15.46:0/3145867223 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6f0066690 0x7fa6f0068b50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa7000017c0 tx=0x7fa700009040 comp rx=0 tx=0).stop 2024-03-29T08:28:17.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.262+0000 7fa717200640 1 -- 172.21.15.46:0/3145867223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71015ad20 msgr2=0x7fa71006c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:17.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.262+0000 7fa717200640 1 --2- 172.21.15.46:0/3145867223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71015ad20 0x7fa71006c650 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa708009e50 tx=0x7fa708019710 comp rx=0 tx=0).stop 2024-03-29T08:28:17.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.262+0000 7fa717200640 1 -- 172.21.15.46:0/3145867223 shutdown_connections 2024-03-29T08:28:17.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.262+0000 7fa717200640 1 --2- 172.21.15.46:0/3145867223 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa6f0066690 0x7fa6f0068b50 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:17.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.262+0000 7fa717200640 1 --2- 172.21.15.46:0/3145867223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa71015ad20 0x7fa71006c650 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:17.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.262+0000 7fa717200640 1 --2- 172.21.15.46:0/3145867223 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa71015a940 0x7fa71006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:17.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.262+0000 7fa717200640 1 --2- 172.21.15.46:0/3145867223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7100cf3c0 0x7fa71014dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:17.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.262+0000 7fa717200640 1 -- 172.21.15.46:0/3145867223 >> 172.21.15.46:0/3145867223 conn(0x7fa710089fd0 msgr2=0x7fa7101539b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:17.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.262+0000 7fa717200640 1 -- 172.21.15.46:0/3145867223 shutdown_connections 2024-03-29T08:28:17.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:17.262+0000 7fa717200640 1 -- 172.21.15.46:0/3145867223 wait complete. 2024-03-29T08:28:17.686 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:27:18.583223+0000', 'modified': '2024-03-29T08:28:16.885668+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 109, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8739}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8739': {'gid': 8739, 'name': 'c', 'rank': 0, 'incarnation': 80, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.46:6835/1455003836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 1455003836}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1455003836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:28:17.686 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8739:up:replay 2024-03-29T08:28:18.687 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:28:18.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.834+0000 7f3824200640 1 Processor -- start 2024-03-29T08:28:18.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.838+0000 7f3824200640 1 -- start start 2024-03-29T08:28:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.838+0000 7f3824200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f381c0ce8a0 0x7f381c0caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.838+0000 7f3824200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f381c0ae810 0x7f381c0cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.838+0000 7f3824200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f381c0c9bd0 con 0x7f381c0cec80 2024-03-29T08:28:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.838+0000 7f3824200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381c0cbcc0 con 0x7f381c0ae810 2024-03-29T08:28:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.838+0000 7f3824200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381c0c9eb0 con 0x7f381c0ce8a0 2024-03-29T08:28:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.838+0000 7f3820e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f381c0ae810 0x7f381c0cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:18.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.838+0000 7f3820e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f381c0ae810 0x7f381c0cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34616/0 (socket says 172.21.15.46:34616) 2024-03-29T08:28:18.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.838+0000 7f3820e00640 1 -- 172.21.15.46:0/3713720071 learned_addr learned my addr 172.21.15.46:0/3713720071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:18.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.838+0000 7f3821800640 1 --2- 172.21.15.46:0/3713720071 >> v2:172.21.15.73:3300/0 conn(0x7f381c0ce8a0 0x7f381c0caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.838+0000 7f3820e00640 1 -- 172.21.15.46:0/3713720071 >> v2:172.21.15.73:3300/0 conn(0x7f381c0ce8a0 msgr2=0x7f381c0caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.838+0000 7f3820e00640 1 --2- 172.21.15.46:0/3713720071 >> v2:172.21.15.73:3300/0 conn(0x7f381c0ce8a0 0x7f381c0caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.838+0000 7f3820e00640 1 -- 172.21.15.46:0/3713720071 >> v1:172.21.15.73:6789/0 conn(0x7f381c0cec80 legacy=0x7f381c0aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.838+0000 7f3820e00640 1 -- 172.21.15.46:0/3713720071 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c0ca190 con 0x7f381c0ae810 2024-03-29T08:28:18.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.838+0000 7f3821800640 1 --2- 172.21.15.46:0/3713720071 >> v2:172.21.15.73:3300/0 conn(0x7f381c0ce8a0 0x7f381c0caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:18.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.842+0000 7f3820e00640 1 --2- 172.21.15.46:0/3713720071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f381c0ae810 0x7f381c0cb780 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f381c0cf4b0 tx=0x7f380801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802ffc799c1376a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:18.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.842+0000 7f381be00640 1 -- 172.21.15.46:0/3713720071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38080aa610 con 0x7f381c0ae810 2024-03-29T08:28:18.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.842+0000 7f381be00640 1 -- 172.21.15.46:0/3713720071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38080a6040 con 0x7f381c0ae810 2024-03-29T08:28:18.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.842+0000 7f381be00640 1 -- 172.21.15.46:0/3713720071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38080aa990 con 0x7f381c0ae810 2024-03-29T08:28:18.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.842+0000 7f3824200640 1 -- 172.21.15.46:0/3713720071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f381c0ae810 msgr2=0x7f381c0cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:18.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.842+0000 7f3824200640 1 --2- 172.21.15.46:0/3713720071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f381c0ae810 0x7f381c0cb780 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f381c0cf4b0 tx=0x7f380801c4b0 comp rx=0 tx=0).stop 2024-03-29T08:28:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.842+0000 7f3824200640 1 -- 172.21.15.46:0/3713720071 shutdown_connections 2024-03-29T08:28:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.842+0000 7f3824200640 1 --2- 172.21.15.46:0/3713720071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f381c0ae810 0x7f381c0cb780 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.842+0000 7f3824200640 1 --2- 172.21.15.46:0/3713720071 >> v2:172.21.15.73:3300/0 conn(0x7f381c0ce8a0 0x7f381c0caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.842+0000 7f3824200640 1 -- 172.21.15.46:0/3713720071 >> 172.21.15.46:0/3713720071 conn(0x7f381c098a40 msgr2=0x7f381c0cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.842+0000 7f3824200640 1 -- 172.21.15.46:0/3713720071 shutdown_connections 2024-03-29T08:28:18.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.842+0000 7f3824200640 1 -- 172.21.15.46:0/3713720071 wait complete. 2024-03-29T08:28:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.842+0000 7f3824200640 1 Processor -- start 2024-03-29T08:28:18.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.842+0000 7f3824200640 1 -- start start 2024-03-29T08:28:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3824200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f381c0ae810 0x7f381c14cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3824200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f381c0ce8a0 0x7f381c147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3824200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f381c0cec80 0x7f381c150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3824200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c149450 con 0x7f381c0ae810 2024-03-29T08:28:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3824200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c0afe10 con 0x7f381c0ce8a0 2024-03-29T08:28:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3824200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f381c0b00f0 con 0x7f381c0cec80 2024-03-29T08:28:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3821800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f381c0ae810 0x7f381c14cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:18.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3821800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f381c0ae810 0x7f381c14cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34618/0 (socket says 172.21.15.46:34618) 2024-03-29T08:28:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3821800640 1 -- 172.21.15.46:0/1185822240 learned_addr learned my addr 172.21.15.46:0/1185822240 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3820e00640 1 --2- 172.21.15.46:0/1185822240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f381c0ce8a0 0x7f381c147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3822200640 1 --2- 172.21.15.46:0/1185822240 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f381c0cec80 0x7f381c150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3821800640 1 -- 172.21.15.46:0/1185822240 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f381c0cec80 msgr2=0x7f381c150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3821800640 1 --2- 172.21.15.46:0/1185822240 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f381c0cec80 0x7f381c150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3821800640 1 -- 172.21.15.46:0/1185822240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f381c0ce8a0 msgr2=0x7f381c147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3821800640 1 --2- 172.21.15.46:0/1185822240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f381c0ce8a0 0x7f381c147860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3821800640 1 -- 172.21.15.46:0/1185822240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38080094d0 con 0x7f381c0ae810 2024-03-29T08:28:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3822200640 1 --2- 172.21.15.46:0/1185822240 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f381c0cec80 0x7f381c150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:28:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3821800640 1 --2- 172.21.15.46:0/1185822240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f381c0ae810 0x7f381c14cb30 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f381000c7c0 tx=0x7f381000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3819600640 1 -- 172.21.15.46:0/1185822240 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810007a30 con 0x7f381c0ae810 2024-03-29T08:28:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3819600640 1 -- 172.21.15.46:0/1185822240 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3810007b90 con 0x7f381c0ae810 2024-03-29T08:28:18.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3819600640 1 -- 172.21.15.46:0/1185822240 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381000ce70 con 0x7f381c0ae810 2024-03-29T08:28:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3824200640 1 -- 172.21.15.46:0/1185822240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381c148c50 con 0x7f381c0ae810 2024-03-29T08:28:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3824200640 1 -- 172.21.15.46:0/1185822240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c147da0 con 0x7f381c0ae810 2024-03-29T08:28:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3824200640 1 -- 172.21.15.46:0/1185822240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381c148ab0 con 0x7f381c0ae810 2024-03-29T08:28:18.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.846+0000 7f3819600640 1 -- 172.21.15.46:0/1185822240 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f38100113f0 con 0x7f381c0ae810 2024-03-29T08:28:18.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.850+0000 7f3819600640 1 --2- 172.21.15.46:0/1185822240 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f37fc066690 0x7f37fc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:18.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.850+0000 7f3819600640 1 -- 172.21.15.46:0/1185822240 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f38100846b0 con 0x7f381c0ae810 2024-03-29T08:28:18.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.850+0000 7f3820e00640 1 --2- 172.21.15.46:0/1185822240 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f37fc066690 0x7f37fc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:18.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.850+0000 7f3820e00640 1 --2- 172.21.15.46:0/1185822240 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f37fc066690 0x7f37fc068b50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f38080ae000 tx=0x7f38080b5040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:18.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:18.854+0000 7f3819600640 1 -- 172.21.15.46:0/1185822240 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f381c148ab0 con 0x7f381c0ae810 2024-03-29T08:28:19.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:19.050+0000 7f3824200640 1 -- 172.21.15.46:0/1185822240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f381c148ab0 con 0x7f381c0ae810 2024-03-29T08:28:19.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:19.054+0000 7f3819600640 1 -- 172.21.15.46:0/1185822240 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f381c148ab0 con 0x7f381c0ae810 2024-03-29T08:28:19.056 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:28:19.057 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4230030957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4230030957},{"type":"v1","addr":"172.21.15.73:6836","nonce":4230030957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/297138217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":297138217},{"type":"v1","addr":"172.21.15.73:6837","nonce":297138217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3856305332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3856305332},{"type":"v1","addr":"172.21.15.46:6833","nonce":3856305332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:27:18.583223+0000","modified":"2024-03-29T08:28:18.889630+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8739":{"gid":8739,"name":"c","rank":0,"incarnation":80,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.46:6835/1455003836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1455003836},{"type":"v1","addr":"172.21.15.46:6835","nonce":1455003836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-29T08:28:19.057 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 82 2024-03-29T08:28:19.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:19.054+0000 7f3824200640 1 -- 172.21.15.46:0/1185822240 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f37fc066690 msgr2=0x7f37fc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:19.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:19.054+0000 7f3824200640 1 --2- 172.21.15.46:0/1185822240 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f37fc066690 0x7f37fc068b50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f38080ae000 tx=0x7f38080b5040 comp rx=0 tx=0).stop 2024-03-29T08:28:19.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:19.054+0000 7f3824200640 1 -- 172.21.15.46:0/1185822240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f381c0ae810 msgr2=0x7f381c14cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:19.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:19.054+0000 7f3824200640 1 --2- 172.21.15.46:0/1185822240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f381c0ae810 0x7f381c14cb30 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f381000c7c0 tx=0x7f381000cc80 comp rx=0 tx=0).stop 2024-03-29T08:28:19.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:19.054+0000 7f3824200640 1 -- 172.21.15.46:0/1185822240 shutdown_connections 2024-03-29T08:28:19.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:19.054+0000 7f3824200640 1 --2- 172.21.15.46:0/1185822240 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f37fc066690 0x7f37fc068b50 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:19.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:19.054+0000 7f3824200640 1 --2- 172.21.15.46:0/1185822240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f381c0ce8a0 0x7f381c147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:19.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:19.054+0000 7f3824200640 1 --2- 172.21.15.46:0/1185822240 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f381c0cec80 0x7f381c150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:19.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:19.054+0000 7f3824200640 1 --2- 172.21.15.46:0/1185822240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f381c0ae810 0x7f381c14cb30 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:19.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:19.054+0000 7f3824200640 1 -- 172.21.15.46:0/1185822240 >> 172.21.15.46:0/1185822240 conn(0x7f381c098a40 msgr2=0x7f381c1534a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:19.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:19.054+0000 7f3824200640 1 -- 172.21.15.46:0/1185822240 shutdown_connections 2024-03-29T08:28:19.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:19.054+0000 7f3824200640 1 -- 172.21.15.46:0/1185822240 wait complete. 2024-03-29T08:28:19.480 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:27:18.583223+0000', 'modified': '2024-03-29T08:28:18.889630+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 109, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8739}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8739': {'gid': 8739, 'name': 'c', 'rank': 0, 'incarnation': 80, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.46:6835/1455003836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 1455003836}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1455003836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:28:19.481 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8739:up:rejoin 2024-03-29T08:28:20.482 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:28:20.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.630+0000 7fb9eb600640 1 Processor -- start 2024-03-29T08:28:20.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.630+0000 7fb9eb600640 1 -- start start 2024-03-29T08:28:20.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.634+0000 7fb9eb600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb9e40708d0 0x7fb9e41596c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:20.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.634+0000 7fb9eb600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e415d7c0 con 0x7fb9e41569a0 2024-03-29T08:28:20.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.634+0000 7fb9eb600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e415cc40 con 0x7fb9e406d910 2024-03-29T08:28:20.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.634+0000 7fb9eb600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e40b21d0 con 0x7fb9e40708d0 2024-03-29T08:28:20.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.634+0000 7fb9e8c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb9e41569a0 0x7fb9e406c1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36096/0 (socket says 172.21.15.46:36096) 2024-03-29T08:28:20.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.634+0000 7fb9e8c00640 1 -- 172.21.15.46:0/301678633 learned_addr learned my addr 172.21.15.46:0/301678633 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:28:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.634+0000 7fb9e9600640 1 --2- 172.21.15.46:0/301678633 >> v2:172.21.15.73:3300/0 conn(0x7fb9e40708d0 0x7fb9e41596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.634+0000 7fb9e3e00640 1 -- 172.21.15.46:0/301678633 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703482731 0 0) 0x7fb9e415d7c0 con 0x7fb9e41569a0 2024-03-29T08:28:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.634+0000 7fb9e3e00640 1 -- 172.21.15.46:0/301678633 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9c8003530 con 0x7fb9e41569a0 2024-03-29T08:28:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.634+0000 7fb9e3e00640 1 -- 172.21.15.46:0/301678633 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679903801 0 0) 0x7fb9e415cc40 con 0x7fb9e406d910 2024-03-29T08:28:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.634+0000 7fb9e3e00640 1 -- 172.21.15.46:0/301678633 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9e415d7c0 con 0x7fb9e406d910 2024-03-29T08:28:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.634+0000 7fb9e9600640 1 -- 172.21.15.46:0/301678633 >> v1:172.21.15.73:6789/0 conn(0x7fb9e406d910 legacy=0x7fb9e40701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.634+0000 7fb9e9600640 1 -- 172.21.15.46:0/301678633 >> v1:172.21.15.46:6789/0 conn(0x7fb9e41569a0 legacy=0x7fb9e406c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.634+0000 7fb9e9600640 1 -- 172.21.15.46:0/301678633 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e415cf20 con 0x7fb9e40708d0 2024-03-29T08:28:20.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.638+0000 7fb9e9600640 1 --2- 172.21.15.46:0/301678633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9e40708d0 0x7fb9e41596c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00aed0 tx=0x7fb9dc01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bc1d4bc75e5bb2d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.638+0000 7fb9e3e00640 1 -- 172.21.15.46:0/301678633 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc00a2d0 con 0x7fb9e40708d0 2024-03-29T08:28:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.638+0000 7fb9e3e00640 1 -- 172.21.15.46:0/301678633 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9dc0a55f0 con 0x7fb9e40708d0 2024-03-29T08:28:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.638+0000 7fb9e3e00640 1 -- 172.21.15.46:0/301678633 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc0a5960 con 0x7fb9e40708d0 2024-03-29T08:28:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.638+0000 7fb9eb600640 1 -- 172.21.15.46:0/301678633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9e40708d0 msgr2=0x7fb9e41596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.638+0000 7fb9eb600640 1 --2- 172.21.15.46:0/301678633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9e40708d0 0x7fb9e41596c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc00aed0 tx=0x7fb9dc01c970 comp rx=0 tx=0).stop 2024-03-29T08:28:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.638+0000 7fb9eb600640 1 -- 172.21.15.46:0/301678633 shutdown_connections 2024-03-29T08:28:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.638+0000 7fb9eb600640 1 --2- 172.21.15.46:0/301678633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9e40708d0 0x7fb9e41596c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.638+0000 7fb9eb600640 1 -- 172.21.15.46:0/301678633 >> 172.21.15.46:0/301678633 conn(0x7fb9e4066110 msgr2=0x7fb9e4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.638+0000 7fb9eb600640 1 -- 172.21.15.46:0/301678633 shutdown_connections 2024-03-29T08:28:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.638+0000 7fb9eb600640 1 -- 172.21.15.46:0/301678633 wait complete. 2024-03-29T08:28:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.638+0000 7fb9eb600640 1 Processor -- start 2024-03-29T08:28:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.638+0000 7fb9eb600640 1 -- start start 2024-03-29T08:28:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9eb600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9e406d910 0x7fb9e40b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9eb600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb9e40708d0 0x7fb9e413aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9eb600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e41569a0 0x7fb9e41377b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9eb600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e413aff0 con 0x7fb9e41569a0 2024-03-29T08:28:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9eb600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e4137cf0 con 0x7fb9e406d910 2024-03-29T08:28:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9eb600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e4154750 con 0x7fb9e40708d0 2024-03-29T08:28:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9e9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e41569a0 0x7fb9e41377b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9e8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9e406d910 0x7fb9e40b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9dbe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb9e40708d0 0x7fb9e413aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:20.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9e8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9e406d910 0x7fb9e40b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:47892/0 (socket says 172.21.15.46:47892) 2024-03-29T08:28:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9e9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e41569a0 0x7fb9e41377b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34620/0 (socket says 172.21.15.46:34620) 2024-03-29T08:28:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9e8c00640 1 -- 172.21.15.46:0/2381232251 learned_addr learned my addr 172.21.15.46:0/2381232251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9e9600640 1 -- 172.21.15.46:0/2381232251 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb9e40708d0 msgr2=0x7fb9e413aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9e9600640 1 --2- 172.21.15.46:0/2381232251 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb9e40708d0 0x7fb9e413aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9e9600640 1 -- 172.21.15.46:0/2381232251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9e406d910 msgr2=0x7fb9e40b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9e9600640 1 --2- 172.21.15.46:0/2381232251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9e406d910 0x7fb9e40b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9e9600640 1 -- 172.21.15.46:0/2381232251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9dc00ab80 con 0x7fb9e41569a0 2024-03-29T08:28:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9e9600640 1 --2- 172.21.15.46:0/2381232251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e41569a0 0x7fb9e41377b0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0ab000 tx=0x7fb9dc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9e1600640 1 -- 172.21.15.46:0/2381232251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc0a5960 con 0x7fb9e41569a0 2024-03-29T08:28:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9e1600640 1 -- 172.21.15.46:0/2381232251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9dc0a5ac0 con 0x7fb9e41569a0 2024-03-29T08:28:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9eb600640 1 -- 172.21.15.46:0/2381232251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e4154a30 con 0x7fb9e41569a0 2024-03-29T08:28:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9e1600640 1 -- 172.21.15.46:0/2381232251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9dc0a5e30 con 0x7fb9e41569a0 2024-03-29T08:28:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9eb600640 1 -- 172.21.15.46:0/2381232251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e4154ff0 con 0x7fb9e41569a0 2024-03-29T08:28:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9eb600640 1 -- 172.21.15.46:0/2381232251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e41551e0 con 0x7fb9e41569a0 2024-03-29T08:28:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.642+0000 7fb9e1600640 1 -- 172.21.15.46:0/2381232251 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb9dc004040 con 0x7fb9e41569a0 2024-03-29T08:28:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.646+0000 7fb9e1600640 1 --2- 172.21.15.46:0/2381232251 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb9c0066690 0x7fb9c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.646+0000 7fb9e1600640 1 -- 172.21.15.46:0/2381232251 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fb9dc11efc0 con 0x7fb9e41569a0 2024-03-29T08:28:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.646+0000 7fb9e8c00640 1 --2- 172.21.15.46:0/2381232251 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb9c0066690 0x7fb9c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.646+0000 7fb9e8c00640 1 --2- 172.21.15.46:0/2381232251 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb9c0066690 0x7fb9c0068b50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4000f70 tx=0x7fb9d4007e20 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:20.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.650+0000 7fb9e1600640 1 -- 172.21.15.46:0/2381232251 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb9dc0ec460 con 0x7fb9e41569a0 2024-03-29T08:28:20.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.846+0000 7fb9eb600640 1 -- 172.21.15.46:0/2381232251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9e4066c90 con 0x7fb9e41569a0 2024-03-29T08:28:20.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.846+0000 7fb9e1600640 1 -- 172.21.15.46:0/2381232251 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4735 (secure 0 0 0) 0x7fb9dc0efa80 con 0x7fb9e41569a0 2024-03-29T08:28:20.849 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:28:20.849 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4230030957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4230030957},{"type":"v1","addr":"172.21.15.73:6836","nonce":4230030957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/297138217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":297138217},{"type":"v1","addr":"172.21.15.73:6837","nonce":297138217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3856305332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3856305332},{"type":"v1","addr":"172.21.15.46:6833","nonce":3856305332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:27:18.583223+0000","modified":"2024-03-29T08:28:19.891199+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8739":{"gid":8739,"name":"c","rank":0,"incarnation":80,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/1455003836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":1455003836},{"type":"v1","addr":"172.21.15.46:6835","nonce":1455003836}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-29T08:28:20.849 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 83 2024-03-29T08:28:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.846+0000 7fb9eb600640 1 -- 172.21.15.46:0/2381232251 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb9c0066690 msgr2=0x7fb9c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.846+0000 7fb9eb600640 1 --2- 172.21.15.46:0/2381232251 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb9c0066690 0x7fb9c0068b50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4000f70 tx=0x7fb9d4007e20 comp rx=0 tx=0).stop 2024-03-29T08:28:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.846+0000 7fb9eb600640 1 -- 172.21.15.46:0/2381232251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e41569a0 msgr2=0x7fb9e41377b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.846+0000 7fb9eb600640 1 --2- 172.21.15.46:0/2381232251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e41569a0 0x7fb9e41377b0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc0ab000 tx=0x7fb9dc019710 comp rx=0 tx=0).stop 2024-03-29T08:28:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.846+0000 7fb9eb600640 1 -- 172.21.15.46:0/2381232251 shutdown_connections 2024-03-29T08:28:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.846+0000 7fb9eb600640 1 --2- 172.21.15.46:0/2381232251 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb9c0066690 0x7fb9c0068b50 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.846+0000 7fb9eb600640 1 --2- 172.21.15.46:0/2381232251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9e41569a0 0x7fb9e41377b0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.846+0000 7fb9eb600640 1 --2- 172.21.15.46:0/2381232251 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb9e40708d0 0x7fb9e413aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.846+0000 7fb9eb600640 1 --2- 172.21.15.46:0/2381232251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9e406d910 0x7fb9e40b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.846+0000 7fb9eb600640 1 -- 172.21.15.46:0/2381232251 >> 172.21.15.46:0/2381232251 conn(0x7fb9e4066110 msgr2=0x7fb9e406c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.846+0000 7fb9eb600640 1 -- 172.21.15.46:0/2381232251 shutdown_connections 2024-03-29T08:28:20.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:20.846+0000 7fb9eb600640 1 -- 172.21.15.46:0/2381232251 wait complete. 2024-03-29T08:28:21.273 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 83, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:27:18.583223+0000', 'modified': '2024-03-29T08:28:19.891199+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 109, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8739}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8739': {'gid': 8739, 'name': 'c', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6835/1455003836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 1455003836}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 1455003836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:28:21.273 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T08:28:21.273 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-03-29T08:28:21.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.414+0000 7fa642e00640 1 Processor -- start 2024-03-29T08:28:21.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.414+0000 7fa642e00640 1 -- start start 2024-03-29T08:28:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.414+0000 7fa642e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa63c06dbb0 0x7fa63c15d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.414+0000 7fa642e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa63c15d940 0x7fa63c0b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.414+0000 7fa642e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63c070c70 con 0x7fa63c142ab0 2024-03-29T08:28:21.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.414+0000 7fa642e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63c070340 con 0x7fa63c06dbb0 2024-03-29T08:28:21.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.414+0000 7fa642e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63c070620 con 0x7fa63c15d940 2024-03-29T08:28:21.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.414+0000 7fa633e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa63c06dbb0 0x7fa63c15d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.414+0000 7fa633e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa63c06dbb0 0x7fa63c15d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34630/0 (socket says 172.21.15.46:34630) 2024-03-29T08:28:21.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.414+0000 7fa633e00640 1 -- 172.21.15.46:0/1443586578 learned_addr learned my addr 172.21.15.46:0/1443586578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.414+0000 7fa640e00640 1 --2- 172.21.15.46:0/1443586578 >> v2:172.21.15.73:3300/0 conn(0x7fa63c15d940 0x7fa63c0b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.414+0000 7fa633e00640 1 -- 172.21.15.46:0/1443586578 >> v2:172.21.15.73:3300/0 conn(0x7fa63c15d940 msgr2=0x7fa63c0b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.414+0000 7fa633e00640 1 --2- 172.21.15.46:0/1443586578 >> v2:172.21.15.73:3300/0 conn(0x7fa63c15d940 0x7fa63c0b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.414+0000 7fa633e00640 1 -- 172.21.15.46:0/1443586578 >> v1:172.21.15.73:6789/0 conn(0x7fa63c142ab0 legacy=0x7fa63c06d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.414+0000 7fa633e00640 1 -- 172.21.15.46:0/1443586578 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c070900 con 0x7fa63c06dbb0 2024-03-29T08:28:21.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.418+0000 7fa633e00640 1 --2- 172.21.15.46:0/1443586578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c06dbb0 0x7fa63c15d400 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fa63c06e740 tx=0x7fa62401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ea3d179cfa611a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.418+0000 7fa63b400640 1 -- 172.21.15.46:0/1443586578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6240ab600 con 0x7fa63c06dbb0 2024-03-29T08:28:21.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.418+0000 7fa63b400640 1 -- 172.21.15.46:0/1443586578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6240a7050 con 0x7fa63c06dbb0 2024-03-29T08:28:21.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.418+0000 7fa63b400640 1 -- 172.21.15.46:0/1443586578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6240ab980 con 0x7fa63c06dbb0 2024-03-29T08:28:21.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.418+0000 7fa642e00640 1 -- 172.21.15.46:0/1443586578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c06dbb0 msgr2=0x7fa63c15d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.418+0000 7fa642e00640 1 --2- 172.21.15.46:0/1443586578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c06dbb0 0x7fa63c15d400 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fa63c06e740 tx=0x7fa62401c520 comp rx=0 tx=0).stop 2024-03-29T08:28:21.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.418+0000 7fa642e00640 1 -- 172.21.15.46:0/1443586578 shutdown_connections 2024-03-29T08:28:21.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.418+0000 7fa642e00640 1 --2- 172.21.15.46:0/1443586578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c06dbb0 0x7fa63c15d400 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.418+0000 7fa642e00640 1 --2- 172.21.15.46:0/1443586578 >> v2:172.21.15.73:3300/0 conn(0x7fa63c15d940 0x7fa63c0b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.418+0000 7fa642e00640 1 -- 172.21.15.46:0/1443586578 >> 172.21.15.46:0/1443586578 conn(0x7fa63c066080 msgr2=0x7fa63c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.418+0000 7fa642e00640 1 -- 172.21.15.46:0/1443586578 shutdown_connections 2024-03-29T08:28:21.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.418+0000 7fa642e00640 1 -- 172.21.15.46:0/1443586578 wait complete. 2024-03-29T08:28:21.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.418+0000 7fa642e00640 1 Processor -- start 2024-03-29T08:28:21.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.418+0000 7fa642e00640 1 -- start start 2024-03-29T08:28:21.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa642e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa63c06dbb0 0x7fa63c13a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa642e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa63c142ab0 0x7fa63c15bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa642e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c15d940 0x7fa63c159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa642e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c13bb30 con 0x7fa63c15d940 2024-03-29T08:28:21.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa642e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c15c440 con 0x7fa63c142ab0 2024-03-29T08:28:21.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa642e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c13aeb0 con 0x7fa63c06dbb0 2024-03-29T08:28:21.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa640e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c15d940 0x7fa63c159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa640e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c15d940 0x7fa63c159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34638/0 (socket says 172.21.15.46:34638) 2024-03-29T08:28:21.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa640e00640 1 -- 172.21.15.46:0/256193728 learned_addr learned my addr 172.21.15.46:0/256193728 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa633e00640 1 --2- 172.21.15.46:0/256193728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa63c142ab0 0x7fa63c15bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa63be00640 1 --2- 172.21.15.46:0/256193728 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa63c06dbb0 0x7fa63c13a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa640e00640 1 -- 172.21.15.46:0/256193728 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa63c06dbb0 msgr2=0x7fa63c13a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa640e00640 1 --2- 172.21.15.46:0/256193728 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa63c06dbb0 0x7fa63c13a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa640e00640 1 -- 172.21.15.46:0/256193728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa63c142ab0 msgr2=0x7fa63c15bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa640e00640 1 --2- 172.21.15.46:0/256193728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa63c142ab0 0x7fa63c15bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa640e00640 1 -- 172.21.15.46:0/256193728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6240094d0 con 0x7fa63c15d940 2024-03-29T08:28:21.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa640e00640 1 --2- 172.21.15.46:0/256193728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c15d940 0x7fa63c159cf0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fa634007e60 tx=0x7fa63400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa638c00640 1 -- 172.21.15.46:0/256193728 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634014420 con 0x7fa63c15d940 2024-03-29T08:28:21.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa638c00640 1 -- 172.21.15.46:0/256193728 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6340145a0 con 0x7fa63c15d940 2024-03-29T08:28:21.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa642e00640 1 -- 172.21.15.46:0/256193728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63c13b190 con 0x7fa63c15d940 2024-03-29T08:28:21.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa638c00640 1 -- 172.21.15.46:0/256193728 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634014950 con 0x7fa63c15d940 2024-03-29T08:28:21.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.422+0000 7fa642e00640 1 -- 172.21.15.46:0/256193728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c13b750 con 0x7fa63c15d940 2024-03-29T08:28:21.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa638c00640 1 -- 172.21.15.46:0/256193728 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa634014ab0 con 0x7fa63c15d940 2024-03-29T08:28:21.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa638c00640 1 --2- 172.21.15.46:0/256193728 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa618066690 0x7fa618068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa638c00640 1 -- 172.21.15.46:0/256193728 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa634084a10 con 0x7fa63c15d940 2024-03-29T08:28:21.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa6461b5000 1 Processor -- start 2024-03-29T08:28:21.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa63be00640 1 --2- 172.21.15.46:0/256193728 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa618066690 0x7fa618068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa6461b5000 1 -- start start 2024-03-29T08:28:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa63be00640 1 --2- 172.21.15.46:0/256193728 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa618066690 0x7fa618068b50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa62c007fd0 tx=0x7fa62c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa6461b5000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e98db0 0x562f63e99190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa6461b5000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e996d0 0x562f63e4ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa6461b5000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e4f4c0 0x562f63e4f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa6461b5000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f63d175a0 con 0x562f63e98db0 2024-03-29T08:28:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa6461b5000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f63ca4af0 con 0x562f63e4f4c0 2024-03-29T08:28:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa6461b5000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f63ca20c0 con 0x562f63e996d0 2024-03-29T08:28:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa63be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e98db0 0x562f63e99190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa63be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e98db0 0x562f63e99190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34648/0 (socket says 172.21.15.46:34648) 2024-03-29T08:28:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa63be00640 1 -- 172.21.15.46:0/226266321 learned_addr learned my addr 172.21.15.46:0/226266321 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa633e00640 1 --2- 172.21.15.46:0/226266321 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e996d0 0x562f63e4ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa640e00640 1 --2- 172.21.15.46:0/226266321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e4f4c0 0x562f63e4f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa63be00640 1 -- 172.21.15.46:0/226266321 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e996d0 msgr2=0x562f63e4ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa63be00640 1 --2- 172.21.15.46:0/226266321 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e996d0 0x562f63e4ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa63be00640 1 -- 172.21.15.46:0/226266321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e4f4c0 msgr2=0x562f63e4f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa63be00640 1 --2- 172.21.15.46:0/226266321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e4f4c0 0x562f63e4f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa63be00640 1 -- 172.21.15.46:0/226266321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f63999890 con 0x562f63e98db0 2024-03-29T08:28:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa63be00640 1 --2- 172.21.15.46:0/226266321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e98db0 0x562f63e99190 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00f950 tx=0x7fa62c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4970ff238d3a3cb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa630c00640 1 -- 172.21.15.46:0/226266321 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c00fed0 con 0x562f63e98db0 2024-03-29T08:28:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa630c00640 1 -- 172.21.15.46:0/226266321 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62c014be0 con 0x562f63e98db0 2024-03-29T08:28:21.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.426+0000 7fa630c00640 1 -- 172.21.15.46:0/226266321 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c020410 con 0x562f63e98db0 2024-03-29T08:28:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 -- 172.21.15.46:0/226266321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e98db0 msgr2=0x562f63e99190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 --2- 172.21.15.46:0/226266321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e98db0 0x562f63e99190 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00f950 tx=0x7fa62c012000 comp rx=0 tx=0).stop 2024-03-29T08:28:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 -- 172.21.15.46:0/226266321 shutdown_connections 2024-03-29T08:28:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 --2- 172.21.15.46:0/226266321 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e4f4c0 0x562f63e4f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 --2- 172.21.15.46:0/226266321 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e996d0 0x562f63e4ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 --2- 172.21.15.46:0/226266321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e98db0 0x562f63e99190 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 -- 172.21.15.46:0/226266321 >> 172.21.15.46:0/226266321 conn(0x562f63e55830 msgr2=0x562f63e4d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 -- 172.21.15.46:0/226266321 shutdown_connections 2024-03-29T08:28:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 -- 172.21.15.46:0/226266321 wait complete. 2024-03-29T08:28:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 -- start start 2024-03-29T08:28:21.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 Processor -- start 2024-03-29T08:28:21.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e4f4c0 0x562f63e985a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e98db0 0x562f63e53230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e996d0 0x562f63e53770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f63c9d9b0 con 0x562f63e996d0 2024-03-29T08:28:21.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f63c3cc00 con 0x562f63e98db0 2024-03-29T08:28:21.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f63c5b070 con 0x562f63e4f4c0 2024-03-29T08:28:21.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 10 client.0 _ll_register_callbacks cb 0x562f63e3b0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:28:21.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa640e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e996d0 0x562f63e53770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa633e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e98db0 0x562f63e53230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa640e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e996d0 0x562f63e53770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34650/0 (socket says 172.21.15.46:34650) 2024-03-29T08:28:21.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa63be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e4f4c0 0x562f63e985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa640e00640 1 -- 172.21.15.46:0/3967385597 learned_addr learned my addr 172.21.15.46:0/3967385597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa640e00640 1 -- 172.21.15.46:0/3967385597 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e4f4c0 msgr2=0x562f63e985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa640e00640 1 --2- 172.21.15.46:0/3967385597 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e4f4c0 0x562f63e985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa640e00640 1 -- 172.21.15.46:0/3967385597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e98db0 msgr2=0x562f63e53230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa640e00640 1 --2- 172.21.15.46:0/3967385597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e98db0 0x562f63e53230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa640e00640 1 -- 172.21.15.46:0/3967385597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c070c70 con 0x562f63e996d0 2024-03-29T08:28:21.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa640e00640 1 --2- 172.21.15.46:0/3967385597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e996d0 0x562f63e53770 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fa63400efd0 tx=0x7fa63405dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83abc061f31a14ef server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa629600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:28:21.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa629600640 1 -- 172.21.15.46:0/3967385597 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63406d380 con 0x562f63e996d0 2024-03-29T08:28:21.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa629600640 1 -- 172.21.15.46:0/3967385597 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa634054f40 con 0x562f63e996d0 2024-03-29T08:28:21.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 -- 172.21.15.46:0/3967385597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f63c5a5f0 con 0x562f63e996d0 2024-03-29T08:28:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa629600640 1 -- 172.21.15.46:0/3967385597 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa634055100 con 0x562f63e996d0 2024-03-29T08:28:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 10 client.8808 fetch_fsmap learned FSMap version 83 2024-03-29T08:28:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 1 -- 172.21.15.46:0/3967385597 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x562f63c62ab0 con 0x562f63e996d0 2024-03-29T08:28:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa629600640 1 -- 172.21.15.46:0/3967385597 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa634071070 con 0x562f63e996d0 2024-03-29T08:28:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa629600640 1 -- 172.21.15.46:0/3967385597 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa634005d20 con 0x562f63e996d0 2024-03-29T08:28:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 10 client.8808 fetch_fsmap finished waiting for FSMap version 83 2024-03-29T08:28:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.430+0000 7fa6461b5000 10 client.8808 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:28:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa6461b5000 20 client.8808 populate_metadata read hostname 'smithi046' 2024-03-29T08:28:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa6461b5000 1 --2- 172.21.15.46:0/3967385597 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x562f63e54180 0x562f63e54560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa6461b5000 4 client.8808 mds_command: new command op to 8739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:28:21.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa6461b5000 1 -- 172.21.15.46:0/3967385597 --> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f63c9a790 con 0x562f63e54180 2024-03-29T08:28:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa63be00640 1 --2- 172.21.15.46:0/3967385597 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x562f63e54180 0x562f63e54560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa63be00640 1 --2- 172.21.15.46:0/3967385597 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x562f63e54180 0x562f63e54560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e82eaed90107638 server_cookie=602f1670e619b8ec in_seq=0 out_seq=0 2024-03-29T08:28:21.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa629600640 10 client.8808 ms_handle_connect on v2:172.21.15.46:6834/1455003836 2024-03-29T08:28:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa629600640 1 -- 172.21.15.46:0/3967385597 <== mds.0 v2:172.21.15.46:6834/1455003836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x562f63c9a790 con 0x562f63e54180 2024-03-29T08:28:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa629600640 10 client.8808 handle_command_reply: tid=0 2024-03-29T08:28:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa6461b5000 1 client.8808 shutdown 2024-03-29T08:28:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa6461b5000 1 -- 172.21.15.46:0/3967385597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e996d0 msgr2=0x562f63e53770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa6461b5000 1 --2- 172.21.15.46:0/3967385597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e996d0 0x562f63e53770 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fa63400efd0 tx=0x7fa63405dd90 comp rx=0 tx=0).stop 2024-03-29T08:28:21.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa6461b5000 1 -- 172.21.15.46:0/3967385597 shutdown_connections 2024-03-29T08:28:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa6461b5000 1 --2- 172.21.15.46:0/3967385597 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x562f63e54180 0x562f63e54560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa6461b5000 1 --2- 172.21.15.46:0/3967385597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e98db0 0x562f63e53230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa6461b5000 1 --2- 172.21.15.46:0/3967385597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e996d0 0x562f63e53770 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa6461b5000 1 --2- 172.21.15.46:0/3967385597 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e4f4c0 0x562f63e985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa629600640 0 client.8808 ms_handle_reset on v2:172.21.15.46:6834/1455003836 2024-03-29T08:28:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa6461b5000 1 -- 172.21.15.46:0/3967385597 >> 172.21.15.46:0/3967385597 conn(0x562f63e55830 msgr2=0x562f63e4df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa6461b5000 1 -- 172.21.15.46:0/3967385597 shutdown_connections 2024-03-29T08:28:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa6461b5000 1 -- 172.21.15.46:0/3967385597 wait complete. 2024-03-29T08:28:21.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.434+0000 7fa6461b5000 20 client.8808 trim_cache size 0 max 16384 2024-03-29T08:28:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa6461b5000 1 Processor -- start 2024-03-29T08:28:21.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa6461b5000 1 -- start start 2024-03-29T08:28:21.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa6461b5000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e4f4c0 0x562f63e9e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa6461b5000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e98db0 0x562f63e9e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa6461b5000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e996d0 0x562f63e92770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa6461b5000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f6395a940 con 0x562f63e4f4c0 2024-03-29T08:28:21.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa6461b5000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f63c64c60 con 0x562f63e98db0 2024-03-29T08:28:21.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa6461b5000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f63ca0510 con 0x562f63e996d0 2024-03-29T08:28:21.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa63be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e4f4c0 0x562f63e9e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa63be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e4f4c0 0x562f63e9e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34664/0 (socket says 172.21.15.46:34664) 2024-03-29T08:28:21.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa63be00640 1 -- 172.21.15.46:0/2632504397 learned_addr learned my addr 172.21.15.46:0/2632504397 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa63be00640 1 -- 172.21.15.46:0/2632504397 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e996d0 msgr2=0x562f63e92770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa63be00640 1 --2- 172.21.15.46:0/2632504397 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e996d0 0x562f63e92770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa63be00640 1 -- 172.21.15.46:0/2632504397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e98db0 msgr2=0x562f63e9e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa63be00640 1 --2- 172.21.15.46:0/2632504397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e98db0 0x562f63e9e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa63be00640 1 -- 172.21.15.46:0/2632504397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f63c48c00 con 0x562f63e4f4c0 2024-03-29T08:28:21.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa63be00640 1 --2- 172.21.15.46:0/2632504397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e4f4c0 0x562f63e9e340 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fa62c023040 tx=0x7fa62c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a10d118a4116c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa631600640 1 -- 172.21.15.46:0/2632504397 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c030040 con 0x562f63e4f4c0 2024-03-29T08:28:21.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa631600640 1 -- 172.21.15.46:0/2632504397 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62c0324c0 con 0x562f63e4f4c0 2024-03-29T08:28:21.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa631600640 1 -- 172.21.15.46:0/2632504397 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c032a30 con 0x562f63e4f4c0 2024-03-29T08:28:21.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa6461b5000 1 -- 172.21.15.46:0/2632504397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e4f4c0 msgr2=0x562f63e9e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa6461b5000 1 --2- 172.21.15.46:0/2632504397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e4f4c0 0x562f63e9e340 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fa62c023040 tx=0x7fa62c015000 comp rx=0 tx=0).stop 2024-03-29T08:28:21.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa6461b5000 1 -- 172.21.15.46:0/2632504397 shutdown_connections 2024-03-29T08:28:21.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa6461b5000 1 --2- 172.21.15.46:0/2632504397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e98db0 0x562f63e9e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa6461b5000 1 --2- 172.21.15.46:0/2632504397 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e996d0 0x562f63e92770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa6461b5000 1 --2- 172.21.15.46:0/2632504397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e4f4c0 0x562f63e9e340 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa6461b5000 1 -- 172.21.15.46:0/2632504397 >> 172.21.15.46:0/2632504397 conn(0x562f63e55830 msgr2=0x562f63e9cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa6461b5000 1 -- 172.21.15.46:0/2632504397 shutdown_connections 2024-03-29T08:28:21.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.450+0000 7fa6461b5000 1 -- 172.21.15.46:0/2632504397 wait complete. 2024-03-29T08:28:21.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 1 -- start start 2024-03-29T08:28:21.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 1 Processor -- start 2024-03-29T08:28:21.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e4f4c0 0x562f63ef76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e98db0 0x562f63ef7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e996d0 0x562f63e95a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f63ca5d80 con 0x562f63e98db0 2024-03-29T08:28:21.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f63c9b1c0 con 0x562f63e996d0 2024-03-29T08:28:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f63ce0ac0 con 0x562f63e4f4c0 2024-03-29T08:28:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 10 client.0 _ll_register_callbacks cb 0x562f63c62590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:28:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa633e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e98db0 0x562f63ef7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa633e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e98db0 0x562f63ef7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34668/0 (socket says 172.21.15.46:34668) 2024-03-29T08:28:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa633e00640 1 -- 172.21.15.46:0/1162636340 learned_addr learned my addr 172.21.15.46:0/1162636340 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa640e00640 1 --2- 172.21.15.46:0/1162636340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e996d0 0x562f63e95a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa63be00640 1 --2- 172.21.15.46:0/1162636340 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e4f4c0 0x562f63ef76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa633e00640 1 -- 172.21.15.46:0/1162636340 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e4f4c0 msgr2=0x562f63ef76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa633e00640 1 --2- 172.21.15.46:0/1162636340 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e4f4c0 0x562f63ef76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa633e00640 1 -- 172.21.15.46:0/1162636340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e996d0 msgr2=0x562f63e95a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa633e00640 1 --2- 172.21.15.46:0/1162636340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e996d0 0x562f63e95a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa633e00640 1 -- 172.21.15.46:0/1162636340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f63999890 con 0x562f63e98db0 2024-03-29T08:28:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa633e00640 1 --2- 172.21.15.46:0/1162636340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e98db0 0x562f63ef7c10 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fa6240ae040 tx=0x7fa6240192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453132a42d24e3a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa629600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:28:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa629600640 1 -- 172.21.15.46:0/1162636340 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6240abc10 con 0x562f63e98db0 2024-03-29T08:28:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa629600640 1 -- 172.21.15.46:0/1162636340 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa6240ba070 con 0x562f63e98db0 2024-03-29T08:28:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 1 -- 172.21.15.46:0/1162636340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f63c6d460 con 0x562f63e98db0 2024-03-29T08:28:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa629600640 1 -- 172.21.15.46:0/1162636340 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6240abd70 con 0x562f63e98db0 2024-03-29T08:28:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 10 client.8814 fetch_fsmap learned FSMap version 83 2024-03-29T08:28:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 1 -- 172.21.15.46:0/1162636340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x562f63999cc0 con 0x562f63e98db0 2024-03-29T08:28:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa629600640 1 -- 172.21.15.46:0/1162636340 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f63c6d460 con 0x562f63e98db0 2024-03-29T08:28:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa629600640 1 -- 172.21.15.46:0/1162636340 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa6240bf480 con 0x562f63e98db0 2024-03-29T08:28:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 10 client.8814 fetch_fsmap finished waiting for FSMap version 83 2024-03-29T08:28:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 10 client.8814 resolve_mds: resolved name 'c' to daemon mds.c 2024-03-29T08:28:21.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 20 client.8814 populate_metadata read hostname 'smithi046' 2024-03-29T08:28:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 1 --2- 172.21.15.46:0/1162636340 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x562f63e95f60 0x562f63e96340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 4 client.8814 mds_command: new command op to 8739 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T08:28:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa6461b5000 1 -- 172.21.15.46:0/1162636340 --> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562f63c669c0 con 0x562f63e95f60 2024-03-29T08:28:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.454+0000 7fa63be00640 1 --2- 172.21.15.46:0/1162636340 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x562f63e95f60 0x562f63e96340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa63be00640 1 --2- 172.21.15.46:0/1162636340 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x562f63e95f60 0x562f63e96340 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c98b1d2772250324 server_cookie=4c36fb3caa75e9c3 in_seq=0 out_seq=0 2024-03-29T08:28:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa629600640 10 client.8814 ms_handle_connect on v2:172.21.15.46:6834/1455003836 2024-03-29T08:28:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa629600640 1 -- 172.21.15.46:0/1162636340 <== mds.0 v2:172.21.15.46:6834/1455003836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x562f63c669c0 con 0x562f63e95f60 2024-03-29T08:28:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa629600640 10 client.8814 handle_command_reply: tid=0 2024-03-29T08:28:21.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa6461b5000 1 client.8814 shutdown 2024-03-29T08:28:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa6461b5000 1 -- 172.21.15.46:0/1162636340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e98db0 msgr2=0x562f63ef7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa6461b5000 1 --2- 172.21.15.46:0/1162636340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e98db0 0x562f63ef7c10 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fa6240ae040 tx=0x7fa6240192c0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa6461b5000 1 -- 172.21.15.46:0/1162636340 shutdown_connections 2024-03-29T08:28:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa6461b5000 1 --2- 172.21.15.46:0/1162636340 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x562f63e95f60 0x562f63e96340 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa6461b5000 1 --2- 172.21.15.46:0/1162636340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562f63e996d0 0x562f63e95a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa6461b5000 1 --2- 172.21.15.46:0/1162636340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562f63e98db0 0x562f63ef7c10 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa629600640 0 client.8814 ms_handle_reset on v2:172.21.15.46:6834/1455003836 2024-03-29T08:28:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa6461b5000 1 --2- 172.21.15.46:0/1162636340 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562f63e4f4c0 0x562f63ef76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa6461b5000 1 -- 172.21.15.46:0/1162636340 >> 172.21.15.46:0/1162636340 conn(0x562f63e55830 msgr2=0x562f63e9d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa6461b5000 1 -- 172.21.15.46:0/1162636340 shutdown_connections 2024-03-29T08:28:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa6461b5000 1 -- 172.21.15.46:0/1162636340 wait complete. 2024-03-29T08:28:21.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa6461b5000 20 client.8814 trim_cache size 0 max 16384 2024-03-29T08:28:21.467 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:28:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "e9b7960f-ec1f-4787-95c3-c806094d37f0", 2024-03-29T08:28:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-03-29T08:28:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 8739, 2024-03-29T08:28:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-29T08:28:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-29T08:28:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-29T08:28:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 4.5713139350000001, 2024-03-29T08:28:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 83, 2024-03-29T08:28:21.467 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 109, 2024-03-29T08:28:21.468 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 109, 2024-03-29T08:28:21.468 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 5.5176393859999999 2024-03-29T08:28:21.468 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:28:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa642e00640 1 -- 172.21.15.46:0/256193728 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa618066690 msgr2=0x7fa618068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa642e00640 1 --2- 172.21.15.46:0/256193728 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa618066690 0x7fa618068b50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa62c007fd0 tx=0x7fa62c009040 comp rx=0 tx=0).stop 2024-03-29T08:28:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa642e00640 1 -- 172.21.15.46:0/256193728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c15d940 msgr2=0x7fa63c159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa642e00640 1 --2- 172.21.15.46:0/256193728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c15d940 0x7fa63c159cf0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fa634007e60 tx=0x7fa63400e4b0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa642e00640 1 -- 172.21.15.46:0/256193728 shutdown_connections 2024-03-29T08:28:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa642e00640 1 --2- 172.21.15.46:0/256193728 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa618066690 0x7fa618068b50 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa642e00640 1 --2- 172.21.15.46:0/256193728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa63c142ab0 0x7fa63c15bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa642e00640 1 --2- 172.21.15.46:0/256193728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c15d940 0x7fa63c159cf0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa642e00640 1 --2- 172.21.15.46:0/256193728 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa63c06dbb0 0x7fa63c13a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa642e00640 1 -- 172.21.15.46:0/256193728 >> 172.21.15.46:0/256193728 conn(0x7fa63c066080 msgr2=0x7fa63c06c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa642e00640 1 -- 172.21.15.46:0/256193728 shutdown_connections 2024-03-29T08:28:21.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.458+0000 7fa642e00640 1 -- 172.21.15.46:0/256193728 wait complete. 2024-03-29T08:28:21.480 DEBUG:teuthology.orchestra.run.smithi046:> sudo daemon-helper kill ceph --cluster ceph -w 2024-03-29T08:28:21.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:28:21.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-03-29T08:28:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.626+0000 7f6fe7200640 1 Processor -- start 2024-03-29T08:28:21.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.626+0000 7f6fe7200640 1 -- start start 2024-03-29T08:28:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.626+0000 7f6fe7200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fe00b21a0 0x7f6fe01590c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.626+0000 7f6fe7200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6fe00ce960 0x7f6fe0070180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.626+0000 7f6fe7200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe0159600 con 0x7f6fe00ce500 2024-03-29T08:28:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.626+0000 7f6fe7200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe00b1470 con 0x7f6fe00ce960 2024-03-29T08:28:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.626+0000 7f6fe7200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe006de60 con 0x7f6fe00b21a0 2024-03-29T08:28:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.626+0000 7f6fd7400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6fe00ce500 0x7f6fe006d3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36110/0 (socket says 172.21.15.46:36110) 2024-03-29T08:28:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.626+0000 7f6fd7400640 1 -- 172.21.15.46:0/2684726115 learned_addr learned my addr 172.21.15.46:0/2684726115 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:28:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.626+0000 7f6fe5200640 1 --2- 172.21.15.46:0/2684726115 >> v2:172.21.15.46:3300/0 conn(0x7f6fe00ce960 0x7f6fe0070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.626+0000 7f6fdfe00640 1 --2- 172.21.15.46:0/2684726115 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fe00b21a0 0x7f6fe01590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.626+0000 7f6fdf400640 1 -- 172.21.15.46:0/2684726115 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2045883088 0 0) 0x7f6fe0159600 con 0x7f6fe00ce500 2024-03-29T08:28:21.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.626+0000 7f6fdf400640 1 -- 172.21.15.46:0/2684726115 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc4003530 con 0x7f6fe00ce500 2024-03-29T08:28:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.626+0000 7f6fe5200640 1 -- 172.21.15.46:0/2684726115 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fe00b21a0 msgr2=0x7f6fe01590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.626+0000 7f6fe5200640 1 --2- 172.21.15.46:0/2684726115 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fe00b21a0 0x7f6fe01590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.626+0000 7f6fe5200640 1 -- 172.21.15.46:0/2684726115 >> v1:172.21.15.46:6789/0 conn(0x7f6fe00ce500 legacy=0x7f6fe006d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.626+0000 7f6fe5200640 1 -- 172.21.15.46:0/2684726115 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe00b16c0 con 0x7f6fe00ce960 2024-03-29T08:28:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.634+0000 7f6fe5200640 1 --2- 172.21.15.46:0/2684726115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe00ce960 0x7f6fe0070180 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009d90 tx=0x7f6fd801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d699f69651b3b79f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.634+0000 7f6fdf400640 1 -- 172.21.15.46:0/2684726115 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd80a5830 con 0x7f6fe00ce960 2024-03-29T08:28:21.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.634+0000 7f6fdf400640 1 -- 172.21.15.46:0/2684726115 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd80a5990 con 0x7f6fe00ce960 2024-03-29T08:28:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.634+0000 7f6fdf400640 1 -- 172.21.15.46:0/2684726115 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd80a5d20 con 0x7f6fe00ce960 2024-03-29T08:28:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.634+0000 7f6fe7200640 1 -- 172.21.15.46:0/2684726115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe00ce960 msgr2=0x7f6fe0070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.634+0000 7f6fe7200640 1 --2- 172.21.15.46:0/2684726115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe00ce960 0x7f6fe0070180 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009d90 tx=0x7f6fd801c990 comp rx=0 tx=0).stop 2024-03-29T08:28:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.634+0000 7f6fe7200640 1 -- 172.21.15.46:0/2684726115 shutdown_connections 2024-03-29T08:28:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.634+0000 7f6fe7200640 1 --2- 172.21.15.46:0/2684726115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe00ce960 0x7f6fe0070180 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.634+0000 7f6fe7200640 1 --2- 172.21.15.46:0/2684726115 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fe00b21a0 0x7f6fe01590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.634+0000 7f6fe7200640 1 -- 172.21.15.46:0/2684726115 >> 172.21.15.46:0/2684726115 conn(0x7f6fe00b42d0 msgr2=0x7f6fe0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.634+0000 7f6fe7200640 1 -- 172.21.15.46:0/2684726115 shutdown_connections 2024-03-29T08:28:21.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.634+0000 7f6fe7200640 1 -- 172.21.15.46:0/2684726115 wait complete. 2024-03-29T08:28:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.634+0000 7f6fe7200640 1 Processor -- start 2024-03-29T08:28:21.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.634+0000 7f6fe7200640 1 -- start start 2024-03-29T08:28:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fe7200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe00b21a0 0x7f6fe0151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fe7200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fe00ce500 0x7f6fe015ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fe7200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fe00ce960 0x7f6fe0154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fe7200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe015bb00 con 0x7f6fe00b21a0 2024-03-29T08:28:21.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fe7200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe015b2c0 con 0x7f6fe00ce960 2024-03-29T08:28:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fe7200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe015b540 con 0x7f6fe00ce500 2024-03-29T08:28:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fdfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe00b21a0 0x7f6fe0151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fdfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe00b21a0 0x7f6fe0151c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34684/0 (socket says 172.21.15.46:34684) 2024-03-29T08:28:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fdfe00640 1 -- 172.21.15.46:0/2933004776 learned_addr learned my addr 172.21.15.46:0/2933004776 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fd7400640 1 --2- 172.21.15.46:0/2933004776 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fe00ce500 0x7f6fe015ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fe5200640 1 --2- 172.21.15.46:0/2933004776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fe00ce960 0x7f6fe0154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fdfe00640 1 -- 172.21.15.46:0/2933004776 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fe00ce500 msgr2=0x7f6fe015ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fdfe00640 1 --2- 172.21.15.46:0/2933004776 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fe00ce500 0x7f6fe015ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fdfe00640 1 -- 172.21.15.46:0/2933004776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fe00ce960 msgr2=0x7f6fe0154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fdfe00640 1 --2- 172.21.15.46:0/2933004776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fe00ce960 0x7f6fe0154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fdfe00640 1 -- 172.21.15.46:0/2933004776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd8009a70 con 0x7f6fe00b21a0 2024-03-29T08:28:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fe5200640 1 --2- 172.21.15.46:0/2933004776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fe00ce960 0x7f6fe0154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:28:21.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fdfe00640 1 --2- 172.21.15.46:0/2933004776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe00b21a0 0x7f6fe0151c80 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000e930 tx=0x7f6fd000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fdcc00640 1 -- 172.21.15.46:0/2933004776 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd0009690 con 0x7f6fe00b21a0 2024-03-29T08:28:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fdcc00640 1 -- 172.21.15.46:0/2933004776 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd000ca00 con 0x7f6fe00b21a0 2024-03-29T08:28:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fdcc00640 1 -- 172.21.15.46:0/2933004776 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd000cd70 con 0x7f6fe00b21a0 2024-03-29T08:28:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fe7200640 1 -- 172.21.15.46:0/2933004776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe006dc30 con 0x7f6fe00b21a0 2024-03-29T08:28:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.638+0000 7f6fe7200640 1 -- 172.21.15.46:0/2933004776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe0159240 con 0x7f6fe00b21a0 2024-03-29T08:28:21.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fdcc00640 1 -- 172.21.15.46:0/2933004776 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6fd0013470 con 0x7f6fe00b21a0 2024-03-29T08:28:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fdcc00640 1 --2- 172.21.15.46:0/2933004776 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6fbc066690 0x7f6fbc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fdcc00640 1 -- 172.21.15.46:0/2933004776 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6fd0084570 con 0x7f6fe00b21a0 2024-03-29T08:28:21.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fd7400640 1 --2- 172.21.15.46:0/2933004776 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6fbc066690 0x7f6fbc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fea74c000 1 Processor -- start 2024-03-29T08:28:21.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fea74c000 1 -- start start 2024-03-29T08:28:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fd7400640 1 --2- 172.21.15.46:0/2933004776 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6fbc066690 0x7f6fbc068b50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6fe015a5b0 tx=0x7f6fc8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fea74c000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c4025bd30 0x561c4025c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fea74c000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c4025c650 0x561c40212a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fea74c000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c40212f60 0x561c40213340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fea74c000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c400da170 con 0x561c40212f60 2024-03-29T08:28:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fea74c000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c40067d80 con 0x561c4025c650 2024-03-29T08:28:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fea74c000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c40065350 con 0x561c4025bd30 2024-03-29T08:28:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fe5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c40212f60 0x561c40213340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fe5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c40212f60 0x561c40213340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34692/0 (socket says 172.21.15.46:34692) 2024-03-29T08:28:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fe5200640 1 -- 172.21.15.46:0/2171025703 learned_addr learned my addr 172.21.15.46:0/2171025703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fd7400640 1 --2- 172.21.15.46:0/2171025703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c4025c650 0x561c40212a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fdfe00640 1 --2- 172.21.15.46:0/2171025703 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c4025bd30 0x561c4025c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fe5200640 1 -- 172.21.15.46:0/2171025703 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c4025bd30 msgr2=0x561c4025c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fe5200640 1 --2- 172.21.15.46:0/2171025703 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c4025bd30 0x561c4025c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fe5200640 1 -- 172.21.15.46:0/2171025703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c4025c650 msgr2=0x561c40212a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fe5200640 1 --2- 172.21.15.46:0/2171025703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c4025c650 0x561c40212a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fe5200640 1 -- 172.21.15.46:0/2171025703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c3fd5d070 con 0x561c40212f60 2024-03-29T08:28:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fd7400640 1 --2- 172.21.15.46:0/2171025703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c4025c650 0x561c40212a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fdfe00640 1 --2- 172.21.15.46:0/2171025703 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c4025bd30 0x561c4025c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.642+0000 7f6fe5200640 1 --2- 172.21.15.46:0/2171025703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c40212f60 0x561c40213340 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f6fd801b040 tx=0x7f6fd8004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ddba83368a3c00 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fd4c00640 1 -- 172.21.15.46:0/2171025703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd80a5d00 con 0x561c40212f60 2024-03-29T08:28:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fd4c00640 1 -- 172.21.15.46:0/2171025703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd80a5e60 con 0x561c40212f60 2024-03-29T08:28:21.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fd4c00640 1 -- 172.21.15.46:0/2171025703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd80b8680 con 0x561c40212f60 2024-03-29T08:28:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 1 -- 172.21.15.46:0/2171025703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c40212f60 msgr2=0x561c40213340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 1 --2- 172.21.15.46:0/2171025703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c40212f60 0x561c40213340 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f6fd801b040 tx=0x7f6fd8004ac0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 1 -- 172.21.15.46:0/2171025703 shutdown_connections 2024-03-29T08:28:21.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 1 --2- 172.21.15.46:0/2171025703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c4025c650 0x561c40212a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 1 --2- 172.21.15.46:0/2171025703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c40212f60 0x561c40213340 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 1 --2- 172.21.15.46:0/2171025703 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c4025bd30 0x561c4025c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 1 -- 172.21.15.46:0/2171025703 >> 172.21.15.46:0/2171025703 conn(0x561c4020ef10 msgr2=0x561c40211300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 1 -- 172.21.15.46:0/2171025703 shutdown_connections 2024-03-29T08:28:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 1 -- 172.21.15.46:0/2171025703 wait complete. 2024-03-29T08:28:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 1 -- start start 2024-03-29T08:28:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 1 Processor -- start 2024-03-29T08:28:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c40212f60 0x561c40218970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c4025bd30 0x561c40249a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c4025c650 0x561c40249f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c40060c40 con 0x561c4025c650 2024-03-29T08:28:21.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c3fffffa0 con 0x561c4025bd30 2024-03-29T08:28:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c4001e1c0 con 0x561c40212f60 2024-03-29T08:28:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fea74c000 10 client.0 _ll_register_callbacks cb 0x561c401fe060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:28:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fe5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c4025c650 0x561c40249f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fe5200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c4025c650 0x561c40249f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34706/0 (socket says 172.21.15.46:34706) 2024-03-29T08:28:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fe5200640 1 -- 172.21.15.46:0/3215545317 learned_addr learned my addr 172.21.15.46:0/3215545317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fd7400640 1 --2- 172.21.15.46:0/3215545317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c4025bd30 0x561c40249a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fdfe00640 1 --2- 172.21.15.46:0/3215545317 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c40212f60 0x561c40218970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fe5200640 1 -- 172.21.15.46:0/3215545317 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c40212f60 msgr2=0x561c40218970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fe5200640 1 --2- 172.21.15.46:0/3215545317 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c40212f60 0x561c40218970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fe5200640 1 -- 172.21.15.46:0/3215545317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c4025bd30 msgr2=0x561c40249a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fe5200640 1 --2- 172.21.15.46:0/3215545317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c4025bd30 0x561c40249a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.646+0000 7f6fe5200640 1 -- 172.21.15.46:0/3215545317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe00b16c0 con 0x561c4025c650 2024-03-29T08:28:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fe5200640 1 --2- 172.21.15.46:0/3215545317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c4025c650 0x561c40249f40 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009d90 tx=0x7f6fd801b1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b760f3f39edee82d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fcd600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:28:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fcd600640 1 -- 172.21.15.46:0/3215545317 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8003ba0 con 0x561c4025c650 2024-03-29T08:28:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fcd600640 1 -- 172.21.15.46:0/3215545317 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6fd8003d00 con 0x561c4025c650 2024-03-29T08:28:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 1 -- 172.21.15.46:0/3215545317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c4005da20 con 0x561c4025c650 2024-03-29T08:28:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fcd600640 1 -- 172.21.15.46:0/3215545317 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fd80070a0 con 0x561c4025c650 2024-03-29T08:28:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 10 client.8829 fetch_fsmap learned FSMap version 83 2024-03-29T08:28:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 1 -- 172.21.15.46:0/3215545317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x561c40025ba0 con 0x561c4025c650 2024-03-29T08:28:21.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fcd600640 1 -- 172.21.15.46:0/3215545317 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd80d2020 con 0x561c4025c650 2024-03-29T08:28:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fcd600640 1 -- 172.21.15.46:0/3215545317 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6fd8004040 con 0x561c4025c650 2024-03-29T08:28:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 10 client.8829 fetch_fsmap finished waiting for FSMap version 83 2024-03-29T08:28:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 10 client.8829 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-29T08:28:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 20 client.8829 populate_metadata read hostname 'smithi046' 2024-03-29T08:28:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 1 --2- 172.21.15.46:0/3215545317 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x561c40262eb0 0x561c402632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 4 client.8829 mds_command: new command op to 8739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:28:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 1 -- 172.21.15.46:0/3215545317 --> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c4005f590 con 0x561c40262eb0 2024-03-29T08:28:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fdfe00640 1 --2- 172.21.15.46:0/3215545317 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x561c40262eb0 0x561c402632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fcd600640 10 client.8829 ms_handle_connect on v2:172.21.15.46:6834/1455003836 2024-03-29T08:28:21.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fdfe00640 1 --2- 172.21.15.46:0/3215545317 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x561c40262eb0 0x561c402632c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f29c98804351a1a4 server_cookie=cead0fe94d2e7a7c in_seq=0 out_seq=0 2024-03-29T08:28:21.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f88e6800640 1 Processor -- start 2024-03-29T08:28:21.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f88e6800640 1 -- start start 2024-03-29T08:28:21.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fcd600640 1 -- 172.21.15.46:0/3215545317 <== mds.0 v2:172.21.15.46:6834/1455003836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x561c4005f590 con 0x561c40262eb0 2024-03-29T08:28:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fcd600640 10 client.8829 handle_command_reply: tid=0 2024-03-29T08:28:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 1 client.8829 shutdown 2024-03-29T08:28:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 1 -- 172.21.15.46:0/3215545317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c4025c650 msgr2=0x561c40249f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 1 --2- 172.21.15.46:0/3215545317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c4025c650 0x561c40249f40 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8009d90 tx=0x7f6fd801b1d0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 1 -- 172.21.15.46:0/3215545317 shutdown_connections 2024-03-29T08:28:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 1 --2- 172.21.15.46:0/3215545317 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x561c40262eb0 0x561c402632c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 1 --2- 172.21.15.46:0/3215545317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c4025bd30 0x561c40249a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 1 --2- 172.21.15.46:0/3215545317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c4025c650 0x561c40249f40 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 1 --2- 172.21.15.46:0/3215545317 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c40212f60 0x561c40218970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 1 -- 172.21.15.46:0/3215545317 >> 172.21.15.46:0/3215545317 conn(0x561c4020ef10 msgr2=0x561c40211b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fcd600640 0 client.8829 ms_handle_reset on v2:172.21.15.46:6834/1455003836 2024-03-29T08:28:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 1 -- 172.21.15.46:0/3215545317 shutdown_connections 2024-03-29T08:28:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 1 -- 172.21.15.46:0/3215545317 wait complete. 2024-03-29T08:28:21.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f6fea74c000 20 client.8829 trim_cache size 0 max 16384 2024-03-29T08:28:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f88e6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f88e00af9d0 0x7f88e00b0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f88e6800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f88e006a750 0x7f88e00b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.650+0000 7f88e6800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e015b1e0 con 0x7f88e00b1ac0 2024-03-29T08:28:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.654+0000 7f88e6800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e015b4c0 con 0x7f88e00af9d0 2024-03-29T08:28:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.654+0000 7f88e6800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e015b7a0 con 0x7f88e006a750 2024-03-29T08:28:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.654+0000 7f88df400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f88e00af9d0 0x7f88e00b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.654+0000 7f88df400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f88e00af9d0 0x7f88e00b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34716/0 (socket says 172.21.15.46:34716) 2024-03-29T08:28:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.654+0000 7f88df400640 1 -- 172.21.15.46:0/3018842441 learned_addr learned my addr 172.21.15.46:0/3018842441 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.654+0000 7f88dea00640 1 --2- 172.21.15.46:0/3018842441 >> v2:172.21.15.73:3300/0 conn(0x7f88e006a750 0x7f88e00b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.654+0000 7f88df400640 1 -- 172.21.15.46:0/3018842441 >> v2:172.21.15.73:3300/0 conn(0x7f88e006a750 msgr2=0x7f88e00b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.654+0000 7f88df400640 1 --2- 172.21.15.46:0/3018842441 >> v2:172.21.15.73:3300/0 conn(0x7f88e006a750 0x7f88e00b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.654+0000 7f88df400640 1 -- 172.21.15.46:0/3018842441 >> v1:172.21.15.73:6789/0 conn(0x7f88e00b1ac0 legacy=0x7f88e015e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.654+0000 7f88df400640 1 -- 172.21.15.46:0/3018842441 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e015ba80 con 0x7f88e00af9d0 2024-03-29T08:28:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.654+0000 7f88dea00640 1 --2- 172.21.15.46:0/3018842441 >> v2:172.21.15.73:3300/0 conn(0x7f88e006a750 0x7f88e00b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.654+0000 7f88df400640 1 --2- 172.21.15.46:0/3018842441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88e00af9d0 0x7f88e00b0330 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f88e00c3100 tx=0x7f88d001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7243a13ba3f383 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.654+0000 7f88de000640 1 -- 172.21.15.46:0/3018842441 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d00aa430 con 0x7f88e00af9d0 2024-03-29T08:28:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.654+0000 7f88de000640 1 -- 172.21.15.46:0/3018842441 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d00aa590 con 0x7f88e00af9d0 2024-03-29T08:28:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.654+0000 7f88de000640 1 -- 172.21.15.46:0/3018842441 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d00aa920 con 0x7f88e00af9d0 2024-03-29T08:28:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88e6800640 1 -- 172.21.15.46:0/3018842441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88e00af9d0 msgr2=0x7f88e00b0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88e6800640 1 --2- 172.21.15.46:0/3018842441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88e00af9d0 0x7f88e00b0330 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f88e00c3100 tx=0x7f88d001c4b0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88e6800640 1 -- 172.21.15.46:0/3018842441 shutdown_connections 2024-03-29T08:28:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88e6800640 1 --2- 172.21.15.46:0/3018842441 >> v2:172.21.15.73:3300/0 conn(0x7f88e006a750 0x7f88e00b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88e6800640 1 --2- 172.21.15.46:0/3018842441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88e00af9d0 0x7f88e00b0330 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88e6800640 1 -- 172.21.15.46:0/3018842441 >> 172.21.15.46:0/3018842441 conn(0x7f88e0098a40 msgr2=0x7f88e00b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88e6800640 1 -- 172.21.15.46:0/3018842441 shutdown_connections 2024-03-29T08:28:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88e6800640 1 -- 172.21.15.46:0/3018842441 wait complete. 2024-03-29T08:28:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88e6800640 1 Processor -- start 2024-03-29T08:28:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88e6800640 1 -- start start 2024-03-29T08:28:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88e6800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88e006a750 0x7f88e00aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88e6800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f88e00af9d0 0x7f88e0153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88e6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88e00b1ac0 0x7f88e014d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88e6800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e00ad3c0 con 0x7f88e00b1ac0 2024-03-29T08:28:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88e6800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e00c6720 con 0x7f88e006a750 2024-03-29T08:28:21.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88e6800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e00af260 con 0x7f88e00af9d0 2024-03-29T08:28:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88dfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88e00b1ac0 0x7f88e014d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88dfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88e00b1ac0 0x7f88e014d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34726/0 (socket says 172.21.15.46:34726) 2024-03-29T08:28:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88dfe00640 1 -- 172.21.15.46:0/4243829370 learned_addr learned my addr 172.21.15.46:0/4243829370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88df400640 1 --2- 172.21.15.46:0/4243829370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88e006a750 0x7f88e00aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88dea00640 1 --2- 172.21.15.46:0/4243829370 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f88e00af9d0 0x7f88e0153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88dfe00640 1 -- 172.21.15.46:0/4243829370 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f88e00af9d0 msgr2=0x7f88e0153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88dfe00640 1 --2- 172.21.15.46:0/4243829370 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f88e00af9d0 0x7f88e0153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.658+0000 7f88dfe00640 1 -- 172.21.15.46:0/4243829370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88e006a750 msgr2=0x7f88e00aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.662+0000 7f88dfe00640 1 --2- 172.21.15.46:0/4243829370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88e006a750 0x7f88e00aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.662+0000 7f88dfe00640 1 -- 172.21.15.46:0/4243829370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d00094d0 con 0x7f88e00b1ac0 2024-03-29T08:28:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.662+0000 7f88dfe00640 1 --2- 172.21.15.46:0/4243829370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88e00b1ac0 0x7f88e014d990 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f88d800b970 tx=0x7f88d800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.662+0000 7f88d7400640 1 -- 172.21.15.46:0/4243829370 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d800cb40 con 0x7f88e00b1ac0 2024-03-29T08:28:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.662+0000 7f88d7400640 1 -- 172.21.15.46:0/4243829370 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d800cca0 con 0x7f88e00b1ac0 2024-03-29T08:28:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.662+0000 7f88d7400640 1 -- 172.21.15.46:0/4243829370 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8014680 con 0x7f88e00b1ac0 2024-03-29T08:28:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.662+0000 7f88e6800640 1 -- 172.21.15.46:0/4243829370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e00ac740 con 0x7f88e00b1ac0 2024-03-29T08:28:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.662+0000 7f88e6800640 1 -- 172.21.15.46:0/4243829370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e00acd00 con 0x7f88e00b1ac0 2024-03-29T08:28:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.662+0000 7f88e6800640 1 -- 172.21.15.46:0/4243829370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f88e009d790 con 0x7f88e00b1ac0 2024-03-29T08:28:21.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.662+0000 7f88d7400640 1 -- 172.21.15.46:0/4243829370 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f88d801d3b0 con 0x7f88e00b1ac0 2024-03-29T08:28:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.662+0000 7f88d7400640 1 --2- 172.21.15.46:0/4243829370 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f88c0066690 0x7f88c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.662+0000 7f88d7400640 1 -- 172.21.15.46:0/4243829370 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f88d8084240 con 0x7f88e00b1ac0 2024-03-29T08:28:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.662+0000 7f88d7400640 1 -- 172.21.15.46:0/4243829370 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+516 (secure 0 0 0) 0x7f88d80846b0 con 0x7f88e00b1ac0 2024-03-29T08:28:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.662+0000 7f88df400640 1 --2- 172.21.15.46:0/4243829370 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f88c0066690 0x7f88c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.666+0000 7f88e6800640 1 -- 172.21.15.46:0/4243829370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f88e00b55d0 con 0x7f88e00b1ac0 2024-03-29T08:28:21.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.666+0000 7f88d7400640 1 -- 172.21.15.46:0/4243829370 <== mon.0 v2:172.21.15.46:3300/0 7 ==== log(2 entries from seq 1055 at 2024-03-29T08:28:19.891296+0000) v1 ==== 389+0+0 (secure 0 0 0) 0x7f88d80827f0 con 0x7f88e00b1ac0 2024-03-29T08:28:21.673 INFO:teuthology.orchestra.run.smithi046.stdout: cluster: 2024-03-29T08:28:21.673 INFO:teuthology.orchestra.run.smithi046.stdout: id: e9b7960f-ec1f-4787-95c3-c806094d37f0 2024-03-29T08:28:21.673 INFO:teuthology.orchestra.run.smithi046.stdout: health: HEALTH_OK 2024-03-29T08:28:21.673 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:28:21.673 INFO:teuthology.orchestra.run.smithi046.stdout: services: 2024-03-29T08:28:21.673 INFO:teuthology.orchestra.run.smithi046.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-03-29T08:28:21.673 INFO:teuthology.orchestra.run.smithi046.stdout: mgr: x(active, since 10m), standbys: y 2024-03-29T08:28:21.673 INFO:teuthology.orchestra.run.smithi046.stdout: mds: 1/1 daemons up, 3 standby 2024-03-29T08:28:21.674 INFO:teuthology.orchestra.run.smithi046.stdout: osd: 8 osds: 8 up (since 10m), 8 in (since 10m) 2024-03-29T08:28:21.674 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:28:21.674 INFO:teuthology.orchestra.run.smithi046.stdout: data: 2024-03-29T08:28:21.674 INFO:teuthology.orchestra.run.smithi046.stdout: volumes: 1/1 healthy 2024-03-29T08:28:21.674 INFO:teuthology.orchestra.run.smithi046.stdout: pools: 3 pools, 97 pgs 2024-03-29T08:28:21.674 INFO:teuthology.orchestra.run.smithi046.stdout: objects: 30 objects, 19 MiB 2024-03-29T08:28:21.674 INFO:teuthology.orchestra.run.smithi046.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-03-29T08:28:21.674 INFO:teuthology.orchestra.run.smithi046.stdout: pgs: 97 active+clean 2024-03-29T08:28:21.674 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:28:21.674 INFO:teuthology.orchestra.run.smithi046.stdout: io: 2024-03-29T08:28:21.674 INFO:teuthology.orchestra.run.smithi046.stdout: client: 8.2 KiB/s rd, 102 B/s wr, 8 op/s rd, 0 op/s wr 2024-03-29T08:28:21.674 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:28:21.674 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:28:21.675 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:28:19.891296+0000 mon.a [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-29T08:28:21.675 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:28:19.891321+0000 mon.a [INF] Cluster is now healthy 2024-03-29T08:28:21.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.666+0000 7f88df400640 1 --2- 172.21.15.46:0/4243829370 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f88c0066690 0x7f88c0068b50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f88d00ae000 tx=0x7f88d00b5040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fea74c000 1 Processor -- start 2024-03-29T08:28:21.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fea74c000 1 -- start start 2024-03-29T08:28:21.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fea74c000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c40212f60 0x561c402b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fea74c000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c4025bd30 0x561c402b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fea74c000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c4025c650 0x561c402b6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fea74c000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c3fd1dcf0 con 0x561c4025bd30 2024-03-29T08:28:21.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fea74c000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c4020eae0 con 0x561c40212f60 2024-03-29T08:28:21.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fea74c000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c40027420 con 0x561c4025c650 2024-03-29T08:28:21.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fd7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c4025bd30 0x561c402b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fd7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c4025bd30 0x561c402b5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34734/0 (socket says 172.21.15.46:34734) 2024-03-29T08:28:21.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fd7400640 1 -- 172.21.15.46:0/2614388001 learned_addr learned my addr 172.21.15.46:0/2614388001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fe5200640 1 --2- 172.21.15.46:0/2614388001 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c4025c650 0x561c402b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fdfe00640 1 --2- 172.21.15.46:0/2614388001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c40212f60 0x561c402b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fd7400640 1 -- 172.21.15.46:0/2614388001 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c4025c650 msgr2=0x561c402b6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fd7400640 1 --2- 172.21.15.46:0/2614388001 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c4025c650 0x561c402b6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fd7400640 1 -- 172.21.15.46:0/2614388001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c40212f60 msgr2=0x561c402b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fd7400640 1 --2- 172.21.15.46:0/2614388001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c40212f60 0x561c402b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fd7400640 1 -- 172.21.15.46:0/2614388001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c4000bef0 con 0x561c4025bd30 2024-03-29T08:28:21.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fd7400640 1 --2- 172.21.15.46:0/2614388001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c4025bd30 0x561c402b5e30 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8001ec0 tx=0x7f6fc8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737d9daaaa011b7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fd5600640 1 -- 172.21.15.46:0/2614388001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8007710 con 0x561c4025bd30 2024-03-29T08:28:21.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fd5600640 1 -- 172.21.15.46:0/2614388001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc801b030 con 0x561c4025bd30 2024-03-29T08:28:21.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fd5600640 1 -- 172.21.15.46:0/2614388001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc800ebc0 con 0x561c4025bd30 2024-03-29T08:28:21.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fea74c000 1 -- 172.21.15.46:0/2614388001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c4025bd30 msgr2=0x561c402b5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fea74c000 1 --2- 172.21.15.46:0/2614388001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c4025bd30 0x561c402b5e30 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8001ec0 tx=0x7f6fc8012000 comp rx=0 tx=0).stop 2024-03-29T08:28:21.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fea74c000 1 -- 172.21.15.46:0/2614388001 shutdown_connections 2024-03-29T08:28:21.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fea74c000 1 --2- 172.21.15.46:0/2614388001 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c4025c650 0x561c402b6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fea74c000 1 --2- 172.21.15.46:0/2614388001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c4025bd30 0x561c402b5e30 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fea74c000 1 --2- 172.21.15.46:0/2614388001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c40212f60 0x561c402b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fea74c000 1 -- 172.21.15.46:0/2614388001 >> 172.21.15.46:0/2614388001 conn(0x561c4020ef10 msgr2=0x561c4025b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fea74c000 1 -- 172.21.15.46:0/2614388001 shutdown_connections 2024-03-29T08:28:21.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.670+0000 7f6fea74c000 1 -- 172.21.15.46:0/2614388001 wait complete. 2024-03-29T08:28:21.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 1 -- start start 2024-03-29T08:28:21.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 1 Processor -- start 2024-03-29T08:28:21.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c40212f60 0x561c402bb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c4025bd30 0x561c402bb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c4025c650 0x561c40270790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c400637a0 con 0x561c40212f60 2024-03-29T08:28:21.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c400a40c0 con 0x561c4025c650 2024-03-29T08:28:21.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c40216d80 con 0x561c4025bd30 2024-03-29T08:28:21.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 10 client.0 _ll_register_callbacks cb 0x561c400da0c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:28:21.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fdfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c40212f60 0x561c402bb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fdfe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c40212f60 0x561c402bb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34744/0 (socket says 172.21.15.46:34744) 2024-03-29T08:28:21.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fdfe00640 1 -- 172.21.15.46:0/3375023720 learned_addr learned my addr 172.21.15.46:0/3375023720 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fd7400640 1 --2- 172.21.15.46:0/3375023720 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c4025bd30 0x561c402bb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fe5200640 1 --2- 172.21.15.46:0/3375023720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c4025c650 0x561c40270790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fdfe00640 1 -- 172.21.15.46:0/3375023720 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c4025bd30 msgr2=0x561c402bb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fdfe00640 1 --2- 172.21.15.46:0/3375023720 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c4025bd30 0x561c402bb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fdfe00640 1 -- 172.21.15.46:0/3375023720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c4025c650 msgr2=0x561c40270790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fdfe00640 1 --2- 172.21.15.46:0/3375023720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c4025c650 0x561c40270790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fdfe00640 1 -- 172.21.15.46:0/3375023720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc800f040 con 0x561c40212f60 2024-03-29T08:28:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fdfe00640 1 --2- 172.21.15.46:0/3375023720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c40212f60 0x561c402bb100 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000e900 tx=0x7f6fd0055380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47c840c2393b9ea server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fcd600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:28:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fcd600640 1 -- 172.21.15.46:0/3375023720 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd00538d0 con 0x561c40212f60 2024-03-29T08:28:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fcd600640 1 -- 172.21.15.46:0/3375023720 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6fd0073350 con 0x561c40212f60 2024-03-29T08:28:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 1 -- 172.21.15.46:0/3375023720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c4001d5f0 con 0x561c40212f60 2024-03-29T08:28:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fcd600640 1 -- 172.21.15.46:0/3375023720 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6fd0051c40 con 0x561c40212f60 2024-03-29T08:28:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 10 client.8841 fetch_fsmap learned FSMap version 83 2024-03-29T08:28:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 1 -- 172.21.15.46:0/3375023720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x561c3ffffa60 con 0x561c40212f60 2024-03-29T08:28:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fcd600640 1 -- 172.21.15.46:0/3375023720 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561c4001d5f0 con 0x561c40212f60 2024-03-29T08:28:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fcd600640 1 -- 172.21.15.46:0/3375023720 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6fd0011070 con 0x561c40212f60 2024-03-29T08:28:21.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 10 client.8841 fetch_fsmap finished waiting for FSMap version 83 2024-03-29T08:28:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 10 client.8841 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-29T08:28:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 20 client.8841 populate_metadata read hostname 'smithi046' 2024-03-29T08:28:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 1 --2- 172.21.15.46:0/3375023720 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x561c40270cd0 0x561c402710e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 4 client.8841 mds_command: new command op to 8739 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-03-29T08:28:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fea74c000 1 -- 172.21.15.46:0/3375023720 --> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x561c4001f410 con 0x561c40270cd0 2024-03-29T08:28:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fd7400640 1 --2- 172.21.15.46:0/3375023720 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x561c40270cd0 0x561c402710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fd7400640 1 --2- 172.21.15.46:0/3375023720 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x561c40270cd0 0x561c402710e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b80d9fd62cd61f server_cookie=a70e9d9d44e39eee in_seq=0 out_seq=0 2024-03-29T08:28:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.674+0000 7f6fcd600640 10 client.8841 ms_handle_connect on v2:172.21.15.46:6834/1455003836 2024-03-29T08:28:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fcd600640 1 -- 172.21.15.46:0/3375023720 <== mds.0 v2:172.21.15.46:6834/1455003836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561c4001f410 con 0x561c40270cd0 2024-03-29T08:28:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fcd600640 10 client.8841 handle_command_reply: tid=0 2024-03-29T08:28:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fea74c000 1 client.8841 shutdown 2024-03-29T08:28:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fea74c000 1 -- 172.21.15.46:0/3375023720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c40212f60 msgr2=0x561c402bb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fea74c000 1 --2- 172.21.15.46:0/3375023720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c40212f60 0x561c402bb100 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000e900 tx=0x7f6fd0055380 comp rx=0 tx=0).stop 2024-03-29T08:28:21.687 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-03-29T08:28:21.678+0000 7efff8200640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-03-29T08:28:21.687 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-03-29T08:28:21.678+0000 7efff8200640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-03-29T08:28:21.687 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-03-29T08:28:21.678+0000 7efff8200640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-03-29T08:28:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fea74c000 1 -- 172.21.15.46:0/3375023720 shutdown_connections 2024-03-29T08:28:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fea74c000 1 --2- 172.21.15.46:0/3375023720 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x561c40270cd0 0x561c402710e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fea74c000 1 --2- 172.21.15.46:0/3375023720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c4025c650 0x561c40270790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fea74c000 1 --2- 172.21.15.46:0/3375023720 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x561c4025bd30 0x561c402bb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fea74c000 1 --2- 172.21.15.46:0/3375023720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561c40212f60 0x561c402bb100 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fea74c000 1 -- 172.21.15.46:0/3375023720 >> 172.21.15.46:0/3375023720 conn(0x561c4020ef10 msgr2=0x561c40268a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fcd600640 0 client.8841 ms_handle_reset on v2:172.21.15.46:6834/1455003836 2024-03-29T08:28:21.688 INFO:tasks.ceph.mds.c.smithi046.stderr:2024-03-29T08:28:21.678+0000 7efff8200640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-03-29T08:28:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fea74c000 1 -- 172.21.15.46:0/3375023720 shutdown_connections 2024-03-29T08:28:21.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fea74c000 1 -- 172.21.15.46:0/3375023720 wait complete. 2024-03-29T08:28:21.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fea74c000 20 client.8841 trim_cache size 0 max 16384 2024-03-29T08:28:21.689 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:28:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-03-29T08:28:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "d03fec76-9d34-4065-bdc8-3a22afc995ac", 2024-03-29T08:28:21.689 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-03-29T08:28:21.689 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:28:21.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fe7200640 1 -- 172.21.15.46:0/2933004776 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6fbc066690 msgr2=0x7f6fbc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fe7200640 1 --2- 172.21.15.46:0/2933004776 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6fbc066690 0x7f6fbc068b50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6fe015a5b0 tx=0x7f6fc8009040 comp rx=0 tx=0).stop 2024-03-29T08:28:21.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fe7200640 1 -- 172.21.15.46:0/2933004776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe00b21a0 msgr2=0x7f6fe0151c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fe7200640 1 --2- 172.21.15.46:0/2933004776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe00b21a0 0x7f6fe0151c80 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f6fd000e930 tx=0x7f6fd000edf0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fe7200640 1 -- 172.21.15.46:0/2933004776 shutdown_connections 2024-03-29T08:28:21.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fe7200640 1 --2- 172.21.15.46:0/2933004776 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6fbc066690 0x7f6fbc068b50 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fe7200640 1 --2- 172.21.15.46:0/2933004776 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fe00ce960 0x7f6fe0154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fe7200640 1 --2- 172.21.15.46:0/2933004776 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6fe00ce500 0x7f6fe015ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fe7200640 1 --2- 172.21.15.46:0/2933004776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6fe00b21a0 0x7f6fe0151c80 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fe7200640 1 -- 172.21.15.46:0/2933004776 >> 172.21.15.46:0/2933004776 conn(0x7f6fe00b42d0 msgr2=0x7f6fe00b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fe7200640 1 -- 172.21.15.46:0/2933004776 shutdown_connections 2024-03-29T08:28:21.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.678+0000 7f6fe7200640 1 -- 172.21.15.46:0/2933004776 wait complete. 2024-03-29T08:28:21.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:28:21.701 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-29T08:28:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.842+0000 7f7b99600640 1 Processor -- start 2024-03-29T08:28:21.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.842+0000 7f7b99600640 1 -- start start 2024-03-29T08:28:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.842+0000 7f7b99600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7b9415df90 0x7f7b940ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.842+0000 7f7b99600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9415d260 con 0x7f7b94142a50 2024-03-29T08:28:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.842+0000 7f7b99600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b940aefd0 con 0x7f7b940cbea0 2024-03-29T08:28:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.842+0000 7f7b99600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9415d4b0 con 0x7f7b9415df90 2024-03-29T08:28:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.842+0000 7f7b92a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7b94142a50 0x7f7b940cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36116/0 (socket says 172.21.15.46:36116) 2024-03-29T08:28:21.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.842+0000 7f7b92a00640 1 -- 172.21.15.46:0/3976832297 learned_addr learned my addr 172.21.15.46:0/3976832297 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:28:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.842+0000 7f7b8a000640 1 --2- 172.21.15.46:0/3976832297 >> v2:172.21.15.73:3300/0 conn(0x7f7b9415df90 0x7f7b940ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.842+0000 7f7b92000640 1 -- 172.21.15.46:0/3976832297 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862690204 0 0) 0x7f7b9415d260 con 0x7f7b94142a50 2024-03-29T08:28:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.842+0000 7f7b92000640 1 -- 172.21.15.46:0/3976832297 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b78005740 con 0x7f7b94142a50 2024-03-29T08:28:21.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.842+0000 7f7b92000640 1 -- 172.21.15.46:0/3976832297 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137480423 0 0) 0x7f7b940aefd0 con 0x7f7b940cbea0 2024-03-29T08:28:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.842+0000 7f7b92000640 1 -- 172.21.15.46:0/3976832297 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b9415d260 con 0x7f7b940cbea0 2024-03-29T08:28:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.842+0000 7f7b92000640 1 -- 172.21.15.46:0/3976832297 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2408260644 0 0) 0x7f7b78005740 con 0x7f7b94142a50 2024-03-29T08:28:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.842+0000 7f7b8a000640 1 -- 172.21.15.46:0/3976832297 >> v1:172.21.15.73:6789/0 conn(0x7f7b940cbea0 legacy=0x7f7b940ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.842+0000 7f7b8a000640 1 -- 172.21.15.46:0/3976832297 >> v1:172.21.15.46:6789/0 conn(0x7f7b94142a50 legacy=0x7f7b940cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.842+0000 7f7b8a000640 1 -- 172.21.15.46:0/3976832297 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9415d790 con 0x7f7b9415df90 2024-03-29T08:28:21.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.846+0000 7f7b8a000640 1 --2- 172.21.15.46:0/3976832297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b9415df90 0x7f7b940ae280 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0097c0 tx=0x7f7b7c01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b35ca073bee482b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.846+0000 7f7b92000640 1 -- 172.21.15.46:0/3976832297 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84001680 con 0x7f7b9415df90 2024-03-29T08:28:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.846+0000 7f7b92000640 1 -- 172.21.15.46:0/3976832297 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7c0a7050 con 0x7f7b9415df90 2024-03-29T08:28:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.846+0000 7f7b92000640 1 -- 172.21.15.46:0/3976832297 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0ab620 con 0x7f7b9415df90 2024-03-29T08:28:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.846+0000 7f7b99600640 1 -- 172.21.15.46:0/3976832297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b9415df90 msgr2=0x7f7b940ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.846+0000 7f7b99600640 1 --2- 172.21.15.46:0/3976832297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b9415df90 0x7f7b940ae280 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0097c0 tx=0x7f7b7c01c330 comp rx=0 tx=0).stop 2024-03-29T08:28:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.846+0000 7f7b99600640 1 -- 172.21.15.46:0/3976832297 shutdown_connections 2024-03-29T08:28:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.846+0000 7f7b99600640 1 --2- 172.21.15.46:0/3976832297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b9415df90 0x7f7b940ae280 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.846+0000 7f7b99600640 1 -- 172.21.15.46:0/3976832297 >> 172.21.15.46:0/3976832297 conn(0x7f7b9406a390 msgr2=0x7f7b940cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.846+0000 7f7b99600640 1 -- 172.21.15.46:0/3976832297 shutdown_connections 2024-03-29T08:28:21.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.846+0000 7f7b99600640 1 -- 172.21.15.46:0/3976832297 wait complete. 2024-03-29T08:28:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.846+0000 7f7b99600640 1 Processor -- start 2024-03-29T08:28:21.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b99600640 1 -- start start 2024-03-29T08:28:21.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b99600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b940cbea0 0x7f7b94158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b99600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b94142a50 0x7f7b94139670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b99600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b9415df90 0x7f7b94139bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b99600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9415a870 con 0x7f7b9415df90 2024-03-29T08:28:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b99600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b94134bd0 con 0x7f7b94142a50 2024-03-29T08:28:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b99600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b94134610 con 0x7f7b940cbea0 2024-03-29T08:28:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b93400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b9415df90 0x7f7b94139bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b93400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b9415df90 0x7f7b94139bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34752/0 (socket says 172.21.15.46:34752) 2024-03-29T08:28:21.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b93400640 1 -- 172.21.15.46:0/3246618523 learned_addr learned my addr 172.21.15.46:0/3246618523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b8a000640 1 --2- 172.21.15.46:0/3246618523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b94142a50 0x7f7b94139670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b92a00640 1 --2- 172.21.15.46:0/3246618523 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b940cbea0 0x7f7b94158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b93400640 1 -- 172.21.15.46:0/3246618523 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b940cbea0 msgr2=0x7f7b94158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b93400640 1 --2- 172.21.15.46:0/3246618523 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b940cbea0 0x7f7b94158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b93400640 1 -- 172.21.15.46:0/3246618523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b94142a50 msgr2=0x7f7b94139670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b93400640 1 --2- 172.21.15.46:0/3246618523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b94142a50 0x7f7b94139670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b93400640 1 -- 172.21.15.46:0/3246618523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c009420 con 0x7f7b9415df90 2024-03-29T08:28:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b92a00640 1 --2- 172.21.15.46:0/3246618523 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b940cbea0 0x7f7b94158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b93400640 1 --2- 172.21.15.46:0/3246618523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b9415df90 0x7f7b94139bb0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00c730 tx=0x7f7b8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b8b400640 1 -- 172.21.15.46:0/3246618523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c003b80 con 0x7f7b9415df90 2024-03-29T08:28:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b8b400640 1 -- 172.21.15.46:0/3246618523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8c003ce0 con 0x7f7b9415df90 2024-03-29T08:28:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b99600640 1 -- 172.21.15.46:0/3246618523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b941348f0 con 0x7f7b9415df90 2024-03-29T08:28:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b8b400640 1 -- 172.21.15.46:0/3246618523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c005310 con 0x7f7b9415df90 2024-03-29T08:28:21.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.850+0000 7f7b99600640 1 -- 172.21.15.46:0/3246618523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b94158f60 con 0x7f7b9415df90 2024-03-29T08:28:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b8b400640 1 -- 172.21.15.46:0/3246618523 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7b8c005680 con 0x7f7b9415df90 2024-03-29T08:28:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b8b400640 1 --2- 172.21.15.46:0/3246618523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b70066690 0x7f7b70068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b8b400640 1 -- 172.21.15.46:0/3246618523 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f7b8c084250 con 0x7f7b9415df90 2024-03-29T08:28:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b92a00640 1 --2- 172.21.15.46:0/3246618523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b70066690 0x7f7b70068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b9cb39000 1 Processor -- start 2024-03-29T08:28:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b9cb39000 1 -- start start 2024-03-29T08:28:21.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b92a00640 1 --2- 172.21.15.46:0/3246618523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b70066690 0x7f7b70068b50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7b84001390 tx=0x7f7b84009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b9cb39000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacaeee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b9cb39000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaef390 0x559aacaa24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b9cb39000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaa2a00 0x559aacaa2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b9cb39000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aac96d630 con 0x559aacaeea70 2024-03-29T08:28:21.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b9cb39000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aac8fabd0 con 0x559aacaa2a00 2024-03-29T08:28:21.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b9cb39000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x559aac8f81a0 con 0x559aacaef390 2024-03-29T08:28:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b92a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacaeee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b92a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacaeee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34758/0 (socket says 172.21.15.46:34758) 2024-03-29T08:28:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b92a00640 1 -- 172.21.15.46:0/1488454174 learned_addr learned my addr 172.21.15.46:0/1488454174 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b8a000640 1 --2- 172.21.15.46:0/1488454174 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaef390 0x559aacaa24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b93400640 1 --2- 172.21.15.46:0/1488454174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaa2a00 0x559aacaa2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b92a00640 1 -- 172.21.15.46:0/1488454174 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaef390 msgr2=0x559aacaa24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b92a00640 1 --2- 172.21.15.46:0/1488454174 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaef390 0x559aacaa24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b92a00640 1 -- 172.21.15.46:0/1488454174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaa2a00 msgr2=0x559aacaa2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b92a00640 1 --2- 172.21.15.46:0/1488454174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaa2a00 0x559aacaa2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.854+0000 7f7b92a00640 1 -- 172.21.15.46:0/1488454174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559aac5efe20 con 0x559aacaeea70 2024-03-29T08:28:21.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b92a00640 1 --2- 172.21.15.46:0/1488454174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacaeee50 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f7b84005c80 tx=0x7f7b84010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bb8994ff9fcbac server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b83400640 1 -- 172.21.15.46:0/1488454174 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84007560 con 0x559aacaeea70 2024-03-29T08:28:21.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b83400640 1 -- 172.21.15.46:0/1488454174 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b840076c0 con 0x559aacaeea70 2024-03-29T08:28:21.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b83400640 1 -- 172.21.15.46:0/1488454174 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84020650 con 0x559aacaeea70 2024-03-29T08:28:21.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 1 -- 172.21.15.46:0/1488454174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 msgr2=0x559aacaeee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/1488454174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacaeee50 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f7b84005c80 tx=0x7f7b84010000 comp rx=0 tx=0).stop 2024-03-29T08:28:21.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 1 -- 172.21.15.46:0/1488454174 shutdown_connections 2024-03-29T08:28:21.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/1488454174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaa2a00 0x559aacaa2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/1488454174 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaef390 0x559aacaa24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/1488454174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacaeee50 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 1 -- 172.21.15.46:0/1488454174 >> 172.21.15.46:0/1488454174 conn(0x559aac8a3160 msgr2=0x559aacad1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 1 -- 172.21.15.46:0/1488454174 shutdown_connections 2024-03-29T08:28:21.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 1 -- 172.21.15.46:0/1488454174 wait complete. 2024-03-29T08:28:21.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 1 -- start start 2024-03-29T08:28:21.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 1 Processor -- start 2024-03-29T08:28:21.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaa2a00 0x559aacaab810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaeea70 0x559aacad2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaef390 0x559aacad2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aac8f3a90 con 0x559aacaa2a00 2024-03-29T08:28:21.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aac892d40 con 0x559aacaef390 2024-03-29T08:28:21.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x559aac8b0f30 con 0x559aacaeea70 2024-03-29T08:28:21.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b9cb39000 10 client.0 _ll_register_callbacks cb 0x559aac955810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:28:21.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b92a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaa2a00 0x559aacaab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b92a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaa2a00 0x559aacaab810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34764/0 (socket says 172.21.15.46:34764) 2024-03-29T08:28:21.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b92a00640 1 -- 172.21.15.46:0/2591156788 learned_addr learned my addr 172.21.15.46:0/2591156788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b8a000640 1 --2- 172.21.15.46:0/2591156788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaeea70 0x559aacad2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b93400640 1 --2- 172.21.15.46:0/2591156788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaef390 0x559aacad2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b92a00640 1 -- 172.21.15.46:0/2591156788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaeea70 msgr2=0x559aacad2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b92a00640 1 --2- 172.21.15.46:0/2591156788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaeea70 0x559aacad2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b92a00640 1 -- 172.21.15.46:0/2591156788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaef390 msgr2=0x559aacad2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b92a00640 1 --2- 172.21.15.46:0/2591156788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaef390 0x559aacad2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b92a00640 1 -- 172.21.15.46:0/2591156788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b840059c0 con 0x559aacaa2a00 2024-03-29T08:28:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b92a00640 1 --2- 172.21.15.46:0/2591156788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaa2a00 0x559aacaab810 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400e010 tx=0x7f7b84012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2516eefb786377 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.858+0000 7f7b77e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:28:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b77e00640 1 -- 172.21.15.46:0/2591156788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84027070 con 0x559aacaa2a00 2024-03-29T08:28:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b77e00640 1 -- 172.21.15.46:0/2591156788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f7b84020660 con 0x559aacaa2a00 2024-03-29T08:28:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2591156788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559aac8f23e0 con 0x559aacaa2a00 2024-03-29T08:28:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b77e00640 1 -- 172.21.15.46:0/2591156788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b84020820 con 0x559aacaa2a00 2024-03-29T08:28:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 10 client.8853 fetch_fsmap learned FSMap version 83 2024-03-29T08:28:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2591156788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x559aac8b8aa0 con 0x559aacaa2a00 2024-03-29T08:28:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b77e00640 1 -- 172.21.15.46:0/2591156788 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559aac8f23e0 con 0x559aacaa2a00 2024-03-29T08:28:21.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b77e00640 1 -- 172.21.15.46:0/2591156788 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7b84020980 con 0x559aacaa2a00 2024-03-29T08:28:21.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 10 client.8853 fetch_fsmap finished waiting for FSMap version 83 2024-03-29T08:28:21.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 10 client.8853 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-29T08:28:21.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 20 client.8853 populate_metadata read hostname 'smithi046' 2024-03-29T08:28:21.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/2591156788 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x559aacaf62d0 0x559aacaf66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 4 client.8853 mds_command: new command op to 8739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:28:21.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2591156788 --> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559aac8c3d00 con 0x559aacaf62d0 2024-03-29T08:28:21.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b8a000640 1 --2- 172.21.15.46:0/2591156788 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x559aacaf62d0 0x559aacaf66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b77e00640 10 client.8853 ms_handle_connect on v2:172.21.15.46:6834/1455003836 2024-03-29T08:28:21.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b8a000640 1 --2- 172.21.15.46:0/2591156788 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x559aacaf62d0 0x559aacaf66b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e73074b89fc0ecc server_cookie=50ebc9968c51a29b in_seq=0 out_seq=0 2024-03-29T08:28:21.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b77e00640 1 -- 172.21.15.46:0/2591156788 <== mds.0 v2:172.21.15.46:6834/1455003836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x559aac8c3d00 con 0x559aacaf62d0 2024-03-29T08:28:21.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b77e00640 10 client.8853 handle_command_reply: tid=0 2024-03-29T08:28:21.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 1 client.8853 shutdown 2024-03-29T08:28:21.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2591156788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaa2a00 msgr2=0x559aacaab810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/2591156788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaa2a00 0x559aacaab810 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400e010 tx=0x7f7b84012000 comp rx=0 tx=0).stop 2024-03-29T08:28:21.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2591156788 shutdown_connections 2024-03-29T08:28:21.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/2591156788 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x559aacaf62d0 0x559aacaf66b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/2591156788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaef390 0x559aacad2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/2591156788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaeea70 0x559aacad2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b77e00640 0 client.8853 ms_handle_reset on v2:172.21.15.46:6834/1455003836 2024-03-29T08:28:21.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/2591156788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaa2a00 0x559aacaab810 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2591156788 >> 172.21.15.46:0/2591156788 conn(0x559aac8a3160 msgr2=0x559aacaa1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2591156788 shutdown_connections 2024-03-29T08:28:21.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2591156788 wait complete. 2024-03-29T08:28:21.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.862+0000 7f7b9cb39000 20 client.8853 trim_cache size 0 max 16384 2024-03-29T08:28:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b9cb39000 1 Processor -- start 2024-03-29T08:28:21.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b9cb39000 1 -- start start 2024-03-29T08:28:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b9cb39000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaa2a00 0x559aacaa1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b9cb39000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacaa2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b9cb39000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaef390 0x559aacb0a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b9cb39000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aac5b0aa0 con 0x559aacaeea70 2024-03-29T08:28:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b9cb39000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aac8f2920 con 0x559aacaa2a00 2024-03-29T08:28:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b9cb39000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x559aac8bdee0 con 0x559aacaef390 2024-03-29T08:28:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b8a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacaa2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b8a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacaa2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34778/0 (socket says 172.21.15.46:34778) 2024-03-29T08:28:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b8a000640 1 -- 172.21.15.46:0/2980216479 learned_addr learned my addr 172.21.15.46:0/2980216479 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b93400640 1 --2- 172.21.15.46:0/2980216479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaef390 0x559aacb0a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b92a00640 1 --2- 172.21.15.46:0/2980216479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaa2a00 0x559aacaa1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b8a000640 1 -- 172.21.15.46:0/2980216479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaef390 msgr2=0x559aacb0a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b8a000640 1 --2- 172.21.15.46:0/2980216479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaef390 0x559aacb0a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b8a000640 1 -- 172.21.15.46:0/2980216479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaa2a00 msgr2=0x559aacaa1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b8a000640 1 --2- 172.21.15.46:0/2980216479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaa2a00 0x559aacaa1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b8a000640 1 -- 172.21.15.46:0/2980216479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559aac89edf0 con 0x559aacaeea70 2024-03-29T08:28:21.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b92a00640 1 --2- 172.21.15.46:0/2980216479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaa2a00 0x559aacaa1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:21.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b93400640 1 --2- 172.21.15.46:0/2980216479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaef390 0x559aacb0a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:21.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b8a000640 1 --2- 172.21.15.46:0/2980216479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacaa2170 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c019f60 tx=0x7f7b7c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd065b07e46f3dfd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b83e00640 1 -- 172.21.15.46:0/2980216479 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0bb040 con 0x559aacaeea70 2024-03-29T08:28:21.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b83e00640 1 -- 172.21.15.46:0/2980216479 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7c004280 con 0x559aacaeea70 2024-03-29T08:28:21.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b83e00640 1 -- 172.21.15.46:0/2980216479 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0c1970 con 0x559aacaeea70 2024-03-29T08:28:21.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2980216479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 msgr2=0x559aacaa2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.878+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/2980216479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacaa2170 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c019f60 tx=0x7f7b7c004740 comp rx=0 tx=0).stop 2024-03-29T08:28:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2980216479 shutdown_connections 2024-03-29T08:28:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/2980216479 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaef390 0x559aacb0a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/2980216479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacaa2170 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/2980216479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaa2a00 0x559aacaa1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2980216479 >> 172.21.15.46:0/2980216479 conn(0x559aac8a3160 msgr2=0x559aacaaa870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2980216479 shutdown_connections 2024-03-29T08:28:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2980216479 wait complete. 2024-03-29T08:28:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 -- start start 2024-03-29T08:28:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 Processor -- start 2024-03-29T08:28:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaa2a00 0x559aacb53400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacb53940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaef390 0x559aacb37390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aacaabcc0 con 0x559aacaeea70 2024-03-29T08:28:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559aac8fbe60 con 0x559aacaa2a00 2024-03-29T08:28:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x559aacad2c80 con 0x559aacaef390 2024-03-29T08:28:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 10 client.0 _ll_register_callbacks cb 0x7f7b84001cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:28:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b8a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacb53940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b8a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacb53940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34782/0 (socket says 172.21.15.46:34782) 2024-03-29T08:28:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b8a000640 1 -- 172.21.15.46:0/2774786262 learned_addr learned my addr 172.21.15.46:0/2774786262 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b93400640 1 --2- 172.21.15.46:0/2774786262 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaef390 0x559aacb37390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b92a00640 1 --2- 172.21.15.46:0/2774786262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaa2a00 0x559aacb53400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b8a000640 1 -- 172.21.15.46:0/2774786262 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaef390 msgr2=0x559aacb37390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b8a000640 1 --2- 172.21.15.46:0/2774786262 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaef390 0x559aacb37390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b8a000640 1 -- 172.21.15.46:0/2774786262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaa2a00 msgr2=0x559aacb53400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:21.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b8a000640 1 --2- 172.21.15.46:0/2774786262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaa2a00 0x559aacb53400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b8a000640 1 -- 172.21.15.46:0/2774786262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9415d790 con 0x559aacaeea70 2024-03-29T08:28:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b8a000640 1 --2- 172.21.15.46:0/2774786262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacb53940 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c019f60 tx=0x7f7b7c01ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5a572b56bc3aad server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b77e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:28:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b77e00640 1 -- 172.21.15.46:0/2774786262 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c0bb040 con 0x559aacaeea70 2024-03-29T08:28:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b77e00640 1 -- 172.21.15.46:0/2774786262 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f7b7c0ccb40 con 0x559aacaeea70 2024-03-29T08:28:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2774786262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559aac8b0970 con 0x559aacaeea70 2024-03-29T08:28:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b77e00640 1 -- 172.21.15.46:0/2774786262 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b7c0ccd00 con 0x559aacaeea70 2024-03-29T08:28:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 10 client.8859 fetch_fsmap learned FSMap version 83 2024-03-29T08:28:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2774786262 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x559aac892960 con 0x559aacaeea70 2024-03-29T08:28:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b77e00640 1 -- 172.21.15.46:0/2774786262 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559aac8b0970 con 0x559aacaeea70 2024-03-29T08:28:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b77e00640 1 -- 172.21.15.46:0/2774786262 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7b7c0c9070 con 0x559aacaeea70 2024-03-29T08:28:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 10 client.8859 fetch_fsmap finished waiting for FSMap version 83 2024-03-29T08:28:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 10 client.8859 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-29T08:28:21.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 20 client.8859 populate_metadata read hostname 'smithi046' 2024-03-29T08:28:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/2774786262 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x559aacb378d0 0x559aacb37ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 4 client.8859 mds_command: new command op to 8739 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:28:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.882+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2774786262 --> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559aac922a20 con 0x559aacb378d0 2024-03-29T08:28:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b92a00640 1 --2- 172.21.15.46:0/2774786262 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x559aacb378d0 0x559aacb37ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b92a00640 1 --2- 172.21.15.46:0/2774786262 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x559aacb378d0 0x559aacb37ce0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22e46a65f8799a93 server_cookie=cc78cdd0ec40d3a4 in_seq=0 out_seq=0 2024-03-29T08:28:21.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b77e00640 10 client.8859 ms_handle_connect on v2:172.21.15.46:6834/1455003836 2024-03-29T08:28:21.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b77e00640 1 -- 172.21.15.46:0/2774786262 <== mds.0 v2:172.21.15.46:6834/1455003836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x559aac922a20 con 0x559aacb378d0 2024-03-29T08:28:21.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b77e00640 10 client.8859 handle_command_reply: tid=0 2024-03-29T08:28:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b9cb39000 1 client.8859 shutdown 2024-03-29T08:28:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2774786262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 msgr2=0x559aacb53940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/2774786262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacb53940 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c019f60 tx=0x7f7b7c01ab70 comp rx=0 tx=0).stop 2024-03-29T08:28:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2774786262 shutdown_connections 2024-03-29T08:28:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/2774786262 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x559aacb378d0 0x559aacb37ce0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/2774786262 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559aacaef390 0x559aacb37390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/2774786262 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559aacaeea70 0x559aacb53940 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b9cb39000 1 --2- 172.21.15.46:0/2774786262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559aacaa2a00 0x559aacb53400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2774786262 >> 172.21.15.46:0/2774786262 conn(0x559aac8a3160 msgr2=0x559aacaa0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b77e00640 0 client.8859 ms_handle_reset on v2:172.21.15.46:6834/1455003836 2024-03-29T08:28:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2774786262 shutdown_connections 2024-03-29T08:28:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b9cb39000 1 -- 172.21.15.46:0/2774786262 wait complete. 2024-03-29T08:28:21.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b9cb39000 20 client.8859 trim_cache size 0 max 16384 2024-03-29T08:28:21.894 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:28:21.894 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-03-29T08:28:21.894 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-03-29T08:28:21.894 INFO:teuthology.orchestra.run.smithi046.stdout: "d03fec76-9d34-4065-bdc8-3a22afc995ac": { 2024-03-29T08:28:21.894 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-03-29T08:28:21.894 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "d03fec76-9d34-4065-bdc8-3a22afc995ac", 2024-03-29T08:28:21.894 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,repair" 2024-03-29T08:28:21.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:28:21.894 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:28:21.894 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:28:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b99600640 1 -- 172.21.15.46:0/3246618523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b70066690 msgr2=0x7f7b70068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b99600640 1 --2- 172.21.15.46:0/3246618523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b70066690 0x7f7b70068b50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7b84001390 tx=0x7f7b84009040 comp rx=0 tx=0).stop 2024-03-29T08:28:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b99600640 1 -- 172.21.15.46:0/3246618523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b9415df90 msgr2=0x7f7b94139bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b99600640 1 --2- 172.21.15.46:0/3246618523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b9415df90 0x7f7b94139bb0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00c730 tx=0x7f7b8c00cbf0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b99600640 1 -- 172.21.15.46:0/3246618523 shutdown_connections 2024-03-29T08:28:21.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b99600640 1 --2- 172.21.15.46:0/3246618523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7b70066690 0x7f7b70068b50 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b99600640 1 --2- 172.21.15.46:0/3246618523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b94142a50 0x7f7b94139670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b99600640 1 --2- 172.21.15.46:0/3246618523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b9415df90 0x7f7b94139bb0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b99600640 1 --2- 172.21.15.46:0/3246618523 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7b940cbea0 0x7f7b94158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b99600640 1 -- 172.21.15.46:0/3246618523 >> 172.21.15.46:0/3246618523 conn(0x7f7b9406a390 msgr2=0x7f7b940cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b99600640 1 -- 172.21.15.46:0/3246618523 shutdown_connections 2024-03-29T08:28:21.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:21.886+0000 7f7b99600640 1 -- 172.21.15.46:0/3246618523 wait complete. 2024-03-29T08:28:21.909 INFO:tasks.cephfs.filesystem:scrub status for tag:d03fec76-9d34-4065-bdc8-3a22afc995ac - {'path': '/', 'tag': 'd03fec76-9d34-4065-bdc8-3a22afc995ac', 'options': 'recursive,repair'} 2024-03-29T08:28:22.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:22.906+0000 7f88d7400640 1 -- 172.21.15.46:0/4243829370 <== mon.0 v2:172.21.15.46:3300/0 8 ==== log(8 entries from seq 1 at 2024-03-29T08:28:21.681635+0000) v1 ==== 1407+0+0 (secure 0 0 0) 0x7f88d8053460 con 0x7f88e00b1ac0 2024-03-29T08:28:22.911 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:28:21.681635+0000 mds.c [INF] scrub queued for path: / 2024-03-29T08:28:22.912 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:28:21.681639+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-03-29T08:28:22.912 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:28:21.681643+0000 mds.c [INF] scrub summary: active paths [/] 2024-03-29T08:28:22.912 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:28:21.682421+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-03-29T08:28:22.912 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:28:21.682429+0000 mds.c [ERR] inode table repaired for inode: 0x10000000001 2024-03-29T08:28:22.912 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:28:21.682491+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-03-29T08:28:22.912 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:28:21.682497+0000 mds.c [ERR] inode table repaired for inode: 0x10000000002 2024-03-29T08:28:22.912 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:28:21.683089+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-03-29T08:28:27.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:27.030+0000 7f88d7400640 1 -- 172.21.15.46:0/4243829370 <== mon.0 v2:172.21.15.46:3300/0 9 ==== log(1 entries from seq 9 at 2024-03-29T08:28:25.947045+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f88d8053710 con 0x7f88e00b1ac0 2024-03-29T08:28:27.035 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:28:25.947045+0000 mds.c [INF] scrub summary: idle 2024-03-29T08:28:39.654 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:28:39.659 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:28:51.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:28:51.910 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-03-29T08:28:52.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.062+0000 7fa5fd400640 1 Processor -- start 2024-03-29T08:28:52.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.062+0000 7fa5fd400640 1 -- start start 2024-03-29T08:28:52.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.062+0000 7fa5fd400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa5f815e000 0x7fa5f80b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.062+0000 7fa5fd400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5f8099780 0x7fa5f8098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.066+0000 7fa5fd400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f80cf4f0 con 0x7fa5f8098f80 2024-03-29T08:28:52.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.066+0000 7fa5fd400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f8099410 con 0x7fa5f815e000 2024-03-29T08:28:52.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.066+0000 7fa5fd400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f80ce090 con 0x7fa5f8099780 2024-03-29T08:28:52.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.066+0000 7fa5f7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa5f815e000 0x7fa5f80b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.066+0000 7fa5fca00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa5f8098f80 0x7fa5f80cb180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53988/0 (socket says 172.21.15.46:53988) 2024-03-29T08:28:52.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.066+0000 7fa5fca00640 1 -- 172.21.15.46:0/670975946 learned_addr learned my addr 172.21.15.46:0/670975946 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:28:52.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.066+0000 7fa5efe00640 1 --2- 172.21.15.46:0/670975946 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5f8099780 0x7fa5f8098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.066+0000 7fa5f7e00640 1 -- 172.21.15.46:0/670975946 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5f8099780 msgr2=0x7fa5f8098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:52.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.066+0000 7fa5f7e00640 1 --2- 172.21.15.46:0/670975946 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5f8099780 0x7fa5f8098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.066+0000 7fa5f7e00640 1 -- 172.21.15.46:0/670975946 >> v1:172.21.15.46:6789/0 conn(0x7fa5f8098f80 legacy=0x7fa5f80cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.066+0000 7fa5f7e00640 1 -- 172.21.15.46:0/670975946 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f80b0730 con 0x7fa5f815e000 2024-03-29T08:28:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.066+0000 7fa5f7e00640 1 --2- 172.21.15.46:0/670975946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5f815e000 0x7fa5f80b12b0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80097c0 tx=0x7fa5e801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9452869e9ac06900 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.066+0000 7fa5f7400640 1 -- 172.21.15.46:0/670975946 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80ab670 con 0x7fa5f815e000 2024-03-29T08:28:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.066+0000 7fa5f7400640 1 -- 172.21.15.46:0/670975946 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e80ab7d0 con 0x7fa5f815e000 2024-03-29T08:28:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.066+0000 7fa5f7400640 1 -- 172.21.15.46:0/670975946 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80abb60 con 0x7fa5f815e000 2024-03-29T08:28:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fd400640 1 -- 172.21.15.46:0/670975946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5f815e000 msgr2=0x7fa5f80b12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fd400640 1 --2- 172.21.15.46:0/670975946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5f815e000 0x7fa5f80b12b0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80097c0 tx=0x7fa5e801c470 comp rx=0 tx=0).stop 2024-03-29T08:28:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fd400640 1 -- 172.21.15.46:0/670975946 shutdown_connections 2024-03-29T08:28:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fd400640 1 --2- 172.21.15.46:0/670975946 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5f8099780 0x7fa5f8098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fd400640 1 --2- 172.21.15.46:0/670975946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5f815e000 0x7fa5f80b12b0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fd400640 1 -- 172.21.15.46:0/670975946 >> 172.21.15.46:0/670975946 conn(0x7fa5f80b1fb0 msgr2=0x7fa5f80b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:52.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fd400640 1 -- 172.21.15.46:0/670975946 shutdown_connections 2024-03-29T08:28:52.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fd400640 1 -- 172.21.15.46:0/670975946 wait complete. 2024-03-29T08:28:52.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fd400640 1 Processor -- start 2024-03-29T08:28:52.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fd400640 1 -- start start 2024-03-29T08:28:52.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fd400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5f8098f80 0x7fa5f8153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fd400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5f8099780 0x7fa5f8164970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5f815e000 0x7fa5f8164eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fd400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f8163f10 con 0x7fa5f815e000 2024-03-29T08:28:52.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fd400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f80aedf0 con 0x7fa5f8099780 2024-03-29T08:28:52.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fd400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f8154050 con 0x7fa5f8098f80 2024-03-29T08:28:52.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5f815e000 0x7fa5f8164eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5f815e000 0x7fa5f8164eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45252/0 (socket says 172.21.15.46:45252) 2024-03-29T08:28:52.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fca00640 1 -- 172.21.15.46:0/1230959023 learned_addr learned my addr 172.21.15.46:0/1230959023 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:52.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5f7e00640 1 --2- 172.21.15.46:0/1230959023 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5f8098f80 0x7fa5f8153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5efe00640 1 --2- 172.21.15.46:0/1230959023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5f8099780 0x7fa5f8164970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fca00640 1 -- 172.21.15.46:0/1230959023 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5f8098f80 msgr2=0x7fa5f8153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fca00640 1 --2- 172.21.15.46:0/1230959023 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5f8098f80 0x7fa5f8153b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fca00640 1 -- 172.21.15.46:0/1230959023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5f8099780 msgr2=0x7fa5f8164970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fca00640 1 --2- 172.21.15.46:0/1230959023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5f8099780 0x7fa5f8164970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.070+0000 7fa5fca00640 1 -- 172.21.15.46:0/1230959023 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e8009420 con 0x7fa5f815e000 2024-03-29T08:28:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.074+0000 7fa5fca00640 1 --2- 172.21.15.46:0/1230959023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5f815e000 0x7fa5f8164eb0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000ca80 tx=0x7fa5f000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.074+0000 7fa5f4c00640 1 -- 172.21.15.46:0/1230959023 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f000ac30 con 0x7fa5f815e000 2024-03-29T08:28:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.074+0000 7fa5f4c00640 1 -- 172.21.15.46:0/1230959023 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f000ad90 con 0x7fa5f815e000 2024-03-29T08:28:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.074+0000 7fa5f4c00640 1 -- 172.21.15.46:0/1230959023 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0012600 con 0x7fa5f815e000 2024-03-29T08:28:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.074+0000 7fa5fd400640 1 -- 172.21.15.46:0/1230959023 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f8163570 con 0x7fa5f815e000 2024-03-29T08:28:52.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.074+0000 7fa5fd400640 1 -- 172.21.15.46:0/1230959023 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f8163b30 con 0x7fa5f815e000 2024-03-29T08:28:52.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.074+0000 7fa5f4c00640 1 -- 172.21.15.46:0/1230959023 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa5f0007070 con 0x7fa5f815e000 2024-03-29T08:28:52.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.074+0000 7fa5f4c00640 1 --2- 172.21.15.46:0/1230959023 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa5d4066690 0x7fa5d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.074+0000 7fa5f4c00640 1 -- 172.21.15.46:0/1230959023 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa5f0083f80 con 0x7fa5f815e000 2024-03-29T08:28:52.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.074+0000 7fa601eb8000 1 Processor -- start 2024-03-29T08:28:52.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.074+0000 7fa5f7e00640 1 --2- 172.21.15.46:0/1230959023 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa5d4066690 0x7fa5d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.074+0000 7fa601eb8000 1 -- start start 2024-03-29T08:28:52.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa601eb8000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d6da70 0x5561d2d6de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa601eb8000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d6e390 0x5561d2d214c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa601eb8000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d21a00 0x5561d2d21de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa601eb8000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d2bec630 con 0x5561d2d6da70 2024-03-29T08:28:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa601eb8000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d2b79bd0 con 0x5561d2d6e390 2024-03-29T08:28:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa601eb8000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561d2b771a0 con 0x5561d2d21a00 2024-03-29T08:28:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa5f7e00640 1 --2- 172.21.15.46:0/1230959023 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa5d4066690 0x7fa5d4068b50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80af000 tx=0x7fa5e80a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa5f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d6da70 0x5561d2d6de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa5f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d6da70 0x5561d2d6de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45260/0 (socket says 172.21.15.46:45260) 2024-03-29T08:28:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa5f7e00640 1 -- 172.21.15.46:0/18541308 learned_addr learned my addr 172.21.15.46:0/18541308 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa5efe00640 1 --2- 172.21.15.46:0/18541308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d6e390 0x5561d2d214c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa5fca00640 1 --2- 172.21.15.46:0/18541308 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d21a00 0x5561d2d21de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa5f7e00640 1 -- 172.21.15.46:0/18541308 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d21a00 msgr2=0x5561d2d21de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa5f7e00640 1 --2- 172.21.15.46:0/18541308 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d21a00 0x5561d2d21de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa5f7e00640 1 -- 172.21.15.46:0/18541308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d6e390 msgr2=0x5561d2d214c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa5f7e00640 1 --2- 172.21.15.46:0/18541308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d6e390 0x5561d2d214c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa5f7e00640 1 -- 172.21.15.46:0/18541308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561d286ee20 con 0x5561d2d6da70 2024-03-29T08:28:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa5efe00640 1 --2- 172.21.15.46:0/18541308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d6e390 0x5561d2d214c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:28:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa5f7e00640 1 --2- 172.21.15.46:0/18541308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d6da70 0x5561d2d6de50 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8006cf0 tx=0x7fa5e8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d83848ab4ebc15 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa5ecc00640 1 -- 172.21.15.46:0/18541308 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80b6890 con 0x5561d2d6da70 2024-03-29T08:28:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa5ecc00640 1 -- 172.21.15.46:0/18541308 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e80c6070 con 0x5561d2d6da70 2024-03-29T08:28:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa5ecc00640 1 -- 172.21.15.46:0/18541308 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80b6bf0 con 0x5561d2d6da70 2024-03-29T08:28:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa601eb8000 1 -- 172.21.15.46:0/18541308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d6da70 msgr2=0x5561d2d6de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa601eb8000 1 --2- 172.21.15.46:0/18541308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d6da70 0x5561d2d6de50 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8006cf0 tx=0x7fa5e8004740 comp rx=0 tx=0).stop 2024-03-29T08:28:52.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa601eb8000 1 -- 172.21.15.46:0/18541308 shutdown_connections 2024-03-29T08:28:52.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa601eb8000 1 --2- 172.21.15.46:0/18541308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d6e390 0x5561d2d214c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa601eb8000 1 --2- 172.21.15.46:0/18541308 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d21a00 0x5561d2d21de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa601eb8000 1 --2- 172.21.15.46:0/18541308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d6da70 0x5561d2d6de50 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa601eb8000 1 -- 172.21.15.46:0/18541308 >> 172.21.15.46:0/18541308 conn(0x5561d2b22160 msgr2=0x5561d2d50820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:52.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa601eb8000 1 -- 172.21.15.46:0/18541308 shutdown_connections 2024-03-29T08:28:52.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa601eb8000 1 -- 172.21.15.46:0/18541308 wait complete. 2024-03-29T08:28:52.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa601eb8000 1 -- start start 2024-03-29T08:28:52.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.078+0000 7fa601eb8000 1 Processor -- start 2024-03-29T08:28:52.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa601eb8000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d21a00 0x5561d2d2a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa601eb8000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d6da70 0x5561d2d51a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa601eb8000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d6e390 0x5561d2d51f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa601eb8000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d2b72a90 con 0x5561d2d21a00 2024-03-29T08:28:52.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa601eb8000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d2b11d40 con 0x5561d2d6e390 2024-03-29T08:28:52.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa601eb8000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561d2b2ff30 con 0x5561d2d6da70 2024-03-29T08:28:52.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa601eb8000 10 client.0 _ll_register_callbacks cb 0x5561d2bd4810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:28:52.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d21a00 0x5561d2d2a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d21a00 0x5561d2d2a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45266/0 (socket says 172.21.15.46:45266) 2024-03-29T08:28:52.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5f7e00640 1 -- 172.21.15.46:0/439824941 learned_addr learned my addr 172.21.15.46:0/439824941 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:52.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5fca00640 1 --2- 172.21.15.46:0/439824941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d6e390 0x5561d2d51f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5efe00640 1 --2- 172.21.15.46:0/439824941 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d6da70 0x5561d2d51a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5f7e00640 1 -- 172.21.15.46:0/439824941 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d6da70 msgr2=0x5561d2d51a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:52.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5f7e00640 1 --2- 172.21.15.46:0/439824941 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d6da70 0x5561d2d51a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5f7e00640 1 -- 172.21.15.46:0/439824941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d6e390 msgr2=0x5561d2d51f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:52.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5f7e00640 1 --2- 172.21.15.46:0/439824941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d6e390 0x5561d2d51f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5f7e00640 1 -- 172.21.15.46:0/439824941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e000e040 con 0x5561d2d21a00 2024-03-29T08:28:52.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5f7e00640 1 --2- 172.21.15.46:0/439824941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d21a00 0x5561d2d2a810 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8006cf0 tx=0x7fa5e80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32e42a6777d3c9d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:52.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5e5600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:28:52.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5e5600640 1 -- 172.21.15.46:0/439824941 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80b78e0 con 0x5561d2d21a00 2024-03-29T08:28:52.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa601eb8000 1 -- 172.21.15.46:0/439824941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561d2b713e0 con 0x5561d2d21a00 2024-03-29T08:28:52.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5e5600640 1 -- 172.21.15.46:0/439824941 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa5e80d4b40 con 0x5561d2d21a00 2024-03-29T08:28:52.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5e5600640 1 -- 172.21.15.46:0/439824941 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5e80d4d00 con 0x5561d2d21a00 2024-03-29T08:28:52.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa601eb8000 10 client.8874 fetch_fsmap learned FSMap version 83 2024-03-29T08:28:52.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa601eb8000 1 -- 172.21.15.46:0/439824941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x7fa5e80bc070 con 0x5561d2d21a00 2024-03-29T08:28:52.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5e5600640 1 -- 172.21.15.46:0/439824941 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5561d2b713e0 con 0x5561d2d21a00 2024-03-29T08:28:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5e5600640 1 -- 172.21.15.46:0/439824941 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa5e80b7a40 con 0x5561d2d21a00 2024-03-29T08:28:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa601eb8000 10 client.8874 fetch_fsmap finished waiting for FSMap version 83 2024-03-29T08:28:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa601eb8000 10 client.8874 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-29T08:28:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa601eb8000 20 client.8874 populate_metadata read hostname 'smithi046' 2024-03-29T08:28:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa601eb8000 1 --2- 172.21.15.46:0/439824941 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x5561d2d752d0 0x5561d2d756b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa601eb8000 4 client.8874 mds_command: new command op to 8739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:28:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa601eb8000 1 -- 172.21.15.46:0/439824941 --> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561d2b42d00 con 0x5561d2d752d0 2024-03-29T08:28:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5efe00640 1 --2- 172.21.15.46:0/439824941 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x5561d2d752d0 0x5561d2d756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5efe00640 1 --2- 172.21.15.46:0/439824941 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x5561d2d752d0 0x5561d2d756b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a8d2723a7fd3b5 server_cookie=fb2c4593426a6aea in_seq=0 out_seq=0 2024-03-29T08:28:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.082+0000 7fa5e5600640 10 client.8874 ms_handle_connect on v2:172.21.15.46:6834/1455003836 2024-03-29T08:28:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.086+0000 7fa5e5600640 1 -- 172.21.15.46:0/439824941 <== mds.0 v2:172.21.15.46:6834/1455003836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5561d2b42d00 con 0x5561d2d752d0 2024-03-29T08:28:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.086+0000 7fa5e5600640 10 client.8874 handle_command_reply: tid=0 2024-03-29T08:28:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.086+0000 7fa601eb8000 1 client.8874 shutdown 2024-03-29T08:28:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.086+0000 7fa601eb8000 1 -- 172.21.15.46:0/439824941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d21a00 msgr2=0x5561d2d2a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.086+0000 7fa601eb8000 1 --2- 172.21.15.46:0/439824941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d21a00 0x5561d2d2a810 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8006cf0 tx=0x7fa5e80046d0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.086+0000 7fa601eb8000 1 -- 172.21.15.46:0/439824941 shutdown_connections 2024-03-29T08:28:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.086+0000 7fa601eb8000 1 --2- 172.21.15.46:0/439824941 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x5561d2d752d0 0x5561d2d756b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.086+0000 7fa601eb8000 1 --2- 172.21.15.46:0/439824941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d6e390 0x5561d2d51f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.086+0000 7fa601eb8000 1 --2- 172.21.15.46:0/439824941 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d6da70 0x5561d2d51a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.086+0000 7fa601eb8000 1 --2- 172.21.15.46:0/439824941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d21a00 0x5561d2d2a810 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.086+0000 7fa5e5600640 0 client.8874 ms_handle_reset on v2:172.21.15.46:6834/1455003836 2024-03-29T08:28:52.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.086+0000 7fa601eb8000 1 -- 172.21.15.46:0/439824941 >> 172.21.15.46:0/439824941 conn(0x5561d2b22160 msgr2=0x5561d2d20bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:52.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.086+0000 7fa601eb8000 1 -- 172.21.15.46:0/439824941 shutdown_connections 2024-03-29T08:28:52.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.086+0000 7fa601eb8000 1 -- 172.21.15.46:0/439824941 wait complete. 2024-03-29T08:28:52.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.086+0000 7fa601eb8000 20 client.8874 trim_cache size 0 max 16384 2024-03-29T08:28:52.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.098+0000 7fa601eb8000 1 Processor -- start 2024-03-29T08:28:52.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.098+0000 7fa601eb8000 1 -- start start 2024-03-29T08:28:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d21a00 0x5561d2d29b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d6da70 0x5561d2d20c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d6e390 0x5561d2d21170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d282faa0 con 0x5561d2d6e390 2024-03-29T08:28:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d2b74020 con 0x5561d2d21a00 2024-03-29T08:28:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561d2bb6170 con 0x5561d2d6da70 2024-03-29T08:28:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa5fca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d6e390 0x5561d2d21170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa5fca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d6e390 0x5561d2d21170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45268/0 (socket says 172.21.15.46:45268) 2024-03-29T08:28:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa5fca00640 1 -- 172.21.15.46:0/3642144829 learned_addr learned my addr 172.21.15.46:0/3642144829 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa5f7e00640 1 --2- 172.21.15.46:0/3642144829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d21a00 0x5561d2d29b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa5efe00640 1 --2- 172.21.15.46:0/3642144829 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d6da70 0x5561d2d20c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa5fca00640 1 -- 172.21.15.46:0/3642144829 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d6da70 msgr2=0x5561d2d20c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa5fca00640 1 --2- 172.21.15.46:0/3642144829 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d6da70 0x5561d2d20c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa5fca00640 1 -- 172.21.15.46:0/3642144829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d21a00 msgr2=0x5561d2d29b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa5fca00640 1 --2- 172.21.15.46:0/3642144829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d21a00 0x5561d2d29b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa5fca00640 1 -- 172.21.15.46:0/3642144829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561d2b37aa0 con 0x5561d2d6e390 2024-03-29T08:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa5f7e00640 1 --2- 172.21.15.46:0/3642144829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d21a00 0x5561d2d29b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa5fca00640 1 --2- 172.21.15.46:0/3642144829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d6e390 0x5561d2d21170 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0001340 tx=0x7fa5f005b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c60baf993dfc71 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa5ed600640 1 -- 172.21.15.46:0/3642144829 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0082210 con 0x5561d2d6e390 2024-03-29T08:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa5ed600640 1 -- 172.21.15.46:0/3642144829 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f005d3a0 con 0x5561d2d6e390 2024-03-29T08:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa5ed600640 1 -- 172.21.15.46:0/3642144829 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0053400 con 0x5561d2d6e390 2024-03-29T08:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 -- 172.21.15.46:0/3642144829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d6e390 msgr2=0x5561d2d21170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 --2- 172.21.15.46:0/3642144829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d6e390 0x5561d2d21170 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0001340 tx=0x7fa5f005b040 comp rx=0 tx=0).stop 2024-03-29T08:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 -- 172.21.15.46:0/3642144829 shutdown_connections 2024-03-29T08:28:52.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 --2- 172.21.15.46:0/3642144829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d6e390 0x5561d2d21170 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 --2- 172.21.15.46:0/3642144829 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d6da70 0x5561d2d20c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 --2- 172.21.15.46:0/3642144829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d21a00 0x5561d2d29b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 -- 172.21.15.46:0/3642144829 >> 172.21.15.46:0/3642144829 conn(0x5561d2b22160 msgr2=0x5561d2d28fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 -- 172.21.15.46:0/3642144829 shutdown_connections 2024-03-29T08:28:52.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 -- 172.21.15.46:0/3642144829 wait complete. 2024-03-29T08:28:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 -- start start 2024-03-29T08:28:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 Processor -- start 2024-03-29T08:28:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d21a00 0x5561d2d24600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d6da70 0x5561d2d24b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d6e390 0x5561d2d25080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d2b420b0 con 0x5561d2d21a00 2024-03-29T08:28:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d2b08a90 con 0x5561d2d6da70 2024-03-29T08:28:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561d2b6e280 con 0x5561d2d6e390 2024-03-29T08:28:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.102+0000 7fa601eb8000 10 client.0 _ll_register_callbacks cb 0x7fa5e8006c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:28:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d21a00 0x5561d2d24600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d21a00 0x5561d2d24600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45282/0 (socket says 172.21.15.46:45282) 2024-03-29T08:28:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5f7e00640 1 -- 172.21.15.46:0/1472874548 learned_addr learned my addr 172.21.15.46:0/1472874548 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:52.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5efe00640 1 --2- 172.21.15.46:0/1472874548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d6da70 0x5561d2d24b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5fca00640 1 --2- 172.21.15.46:0/1472874548 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d6e390 0x5561d2d25080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5f7e00640 1 -- 172.21.15.46:0/1472874548 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d6e390 msgr2=0x5561d2d25080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:52.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5f7e00640 1 --2- 172.21.15.46:0/1472874548 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d6e390 0x5561d2d25080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5f7e00640 1 -- 172.21.15.46:0/1472874548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d6da70 msgr2=0x5561d2d24b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:52.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5f7e00640 1 --2- 172.21.15.46:0/1472874548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d6da70 0x5561d2d24b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5f7e00640 1 -- 172.21.15.46:0/1472874548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f8163b30 con 0x5561d2d21a00 2024-03-29T08:28:52.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5fca00640 1 --2- 172.21.15.46:0/1472874548 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d6e390 0x5561d2d25080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:28:52.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5f7e00640 1 --2- 172.21.15.46:0/1472874548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d21a00 0x5561d2d24600 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fa5e801ab20 tx=0x7fa5e801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6657302a362500 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:52.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5e5600640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:28:52.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5e5600640 1 -- 172.21.15.46:0/1472874548 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80b7450 con 0x5561d2d21a00 2024-03-29T08:28:52.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5e5600640 1 -- 172.21.15.46:0/1472874548 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fa5e80ab530 con 0x5561d2d21a00 2024-03-29T08:28:52.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 1 -- 172.21.15.46:0/1472874548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5561d2b2f970 con 0x5561d2d21a00 2024-03-29T08:28:52.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5e5600640 1 -- 172.21.15.46:0/1472874548 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=83) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5e80b75b0 con 0x5561d2d21a00 2024-03-29T08:28:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 10 client.8880 fetch_fsmap learned FSMap version 83 2024-03-29T08:28:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 1 -- 172.21.15.46:0/1472874548 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=83,monmap=2+}) v3 -- 0x5561d2b1ddf0 con 0x5561d2d21a00 2024-03-29T08:28:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5e5600640 1 -- 172.21.15.46:0/1472874548 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5561d2b2f970 con 0x5561d2d21a00 2024-03-29T08:28:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5e5600640 1 -- 172.21.15.46:0/1472874548 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 83) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fa5e80b6690 con 0x5561d2d21a00 2024-03-29T08:28:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 10 client.8880 fetch_fsmap finished waiting for FSMap version 83 2024-03-29T08:28:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 10 client.8880 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-03-29T08:28:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 20 client.8880 populate_metadata read hostname 'smithi046' 2024-03-29T08:28:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 1 --2- 172.21.15.46:0/1472874548 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x5561d2d255c0 0x5561d2d259d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 4 client.8880 mds_command: new command op to 8739 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-03-29T08:28:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 1 -- 172.21.15.46:0/1472874548 --> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5561d2ba1a20 con 0x5561d2d255c0 2024-03-29T08:28:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5efe00640 1 --2- 172.21.15.46:0/1472874548 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x5561d2d255c0 0x5561d2d259d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5efe00640 1 --2- 172.21.15.46:0/1472874548 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x5561d2d255c0 0x5561d2d259d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b715f556a8e73924 server_cookie=85726b59a00428cc in_seq=0 out_seq=0 2024-03-29T08:28:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5e5600640 10 client.8880 ms_handle_connect on v2:172.21.15.46:6834/1455003836 2024-03-29T08:28:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5e5600640 1 -- 172.21.15.46:0/1472874548 <== mds.0 v2:172.21.15.46:6834/1455003836 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5561d2ba1a20 con 0x5561d2d255c0 2024-03-29T08:28:52.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5e5600640 10 client.8880 handle_command_reply: tid=0 2024-03-29T08:28:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 1 client.8880 shutdown 2024-03-29T08:28:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 1 -- 172.21.15.46:0/1472874548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d21a00 msgr2=0x5561d2d24600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 1 --2- 172.21.15.46:0/1472874548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d21a00 0x5561d2d24600 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fa5e801ab20 tx=0x7fa5e801acb0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 1 -- 172.21.15.46:0/1472874548 shutdown_connections 2024-03-29T08:28:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 1 --2- 172.21.15.46:0/1472874548 >> [v2:172.21.15.46:6834/1455003836,v1:172.21.15.46:6835/1455003836] conn(0x5561d2d255c0 0x5561d2d259d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 1 --2- 172.21.15.46:0/1472874548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2d6da70 0x5561d2d24b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 1 --2- 172.21.15.46:0/1472874548 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5561d2d6e390 0x5561d2d25080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 1 --2- 172.21.15.46:0/1472874548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561d2d21a00 0x5561d2d24600 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 1 -- 172.21.15.46:0/1472874548 >> 172.21.15.46:0/1472874548 conn(0x5561d2b22160 msgr2=0x5561d2d29870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5e5600640 0 client.8880 ms_handle_reset on v2:172.21.15.46:6834/1455003836 2024-03-29T08:28:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 1 -- 172.21.15.46:0/1472874548 shutdown_connections 2024-03-29T08:28:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 1 -- 172.21.15.46:0/1472874548 wait complete. 2024-03-29T08:28:52.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa601eb8000 20 client.8880 trim_cache size 0 max 16384 2024-03-29T08:28:52.117 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:28:52.117 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-03-29T08:28:52.117 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-03-29T08:28:52.117 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:28:52.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5fd400640 1 -- 172.21.15.46:0/1230959023 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa5d4066690 msgr2=0x7fa5d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5fd400640 1 --2- 172.21.15.46:0/1230959023 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa5d4066690 0x7fa5d4068b50 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80af000 tx=0x7fa5e80a7000 comp rx=0 tx=0).stop 2024-03-29T08:28:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5fd400640 1 -- 172.21.15.46:0/1230959023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5f815e000 msgr2=0x7fa5f8164eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5fd400640 1 --2- 172.21.15.46:0/1230959023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5f815e000 0x7fa5f8164eb0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000ca80 tx=0x7fa5f000cf40 comp rx=0 tx=0).stop 2024-03-29T08:28:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5fd400640 1 -- 172.21.15.46:0/1230959023 shutdown_connections 2024-03-29T08:28:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5fd400640 1 --2- 172.21.15.46:0/1230959023 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa5d4066690 0x7fa5d4068b50 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5fd400640 1 --2- 172.21.15.46:0/1230959023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5f8099780 0x7fa5f8164970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5fd400640 1 --2- 172.21.15.46:0/1230959023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5f815e000 0x7fa5f8164eb0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5fd400640 1 --2- 172.21.15.46:0/1230959023 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5f8098f80 0x7fa5f8153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5fd400640 1 -- 172.21.15.46:0/1230959023 >> 172.21.15.46:0/1230959023 conn(0x7fa5f80b1fb0 msgr2=0x7fa5f815cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5fd400640 1 -- 172.21.15.46:0/1230959023 shutdown_connections 2024-03-29T08:28:52.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.106+0000 7fa5fd400640 1 -- 172.21.15.46:0/1230959023 wait complete. 2024-03-29T08:28:52.130 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-03-29T08:28:52.231 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T08:28:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.370+0000 7fe48c200640 1 Processor -- start 2024-03-29T08:28:52.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.370+0000 7fe48c200640 1 -- start start 2024-03-29T08:28:52.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.374+0000 7fe48c200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe484158300 0x7fe48406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.374+0000 7fe48c200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe4840709b0 0x7fe4840af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.374+0000 7fe48c200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe48406c390 con 0x7fe4840af530 2024-03-29T08:28:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.374+0000 7fe48c200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe48406d600 con 0x7fe4840709b0 2024-03-29T08:28:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.374+0000 7fe48c200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4840afff0 con 0x7fe484158300 2024-03-29T08:28:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.374+0000 7fe48a200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe4840709b0 0x7fe4840af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.374+0000 7fe488e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe4840af530 0x7fe4840702a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53996/0 (socket says 172.21.15.46:53996) 2024-03-29T08:28:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.374+0000 7fe488e00640 1 -- 172.21.15.46:0/3663240300 learned_addr learned my addr 172.21.15.46:0/3663240300 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:28:52.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.374+0000 7fe489800640 1 --2- 172.21.15.46:0/3663240300 >> v2:172.21.15.73:3300/0 conn(0x7fe484158300 0x7fe48406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.374+0000 7fe48a200640 1 -- 172.21.15.46:0/3663240300 >> v2:172.21.15.73:3300/0 conn(0x7fe484158300 msgr2=0x7fe48406d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:52.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.374+0000 7fe48a200640 1 --2- 172.21.15.46:0/3663240300 >> v2:172.21.15.73:3300/0 conn(0x7fe484158300 0x7fe48406d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.374+0000 7fe48a200640 1 -- 172.21.15.46:0/3663240300 >> v1:172.21.15.46:6789/0 conn(0x7fe4840af530 legacy=0x7fe4840702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.374+0000 7fe483e00640 1 -- 172.21.15.46:0/3663240300 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652190639 0 0) 0x7fe48406c390 con 0x7fe4840af530 2024-03-29T08:28:52.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.374+0000 7fe48a200640 1 -- 172.21.15.46:0/3663240300 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48406c5e0 con 0x7fe4840709b0 2024-03-29T08:28:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.378+0000 7fe48a200640 1 --2- 172.21.15.46:0/3663240300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4840709b0 0x7fe4840af9c0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fe48406aa60 tx=0x7fe47c01c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc4f3176f550215 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.378+0000 7fe483e00640 1 -- 172.21.15.46:0/3663240300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0a5880 con 0x7fe4840709b0 2024-03-29T08:28:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.378+0000 7fe483e00640 1 -- 172.21.15.46:0/3663240300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c0a59e0 con 0x7fe4840709b0 2024-03-29T08:28:52.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.378+0000 7fe483e00640 1 -- 172.21.15.46:0/3663240300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0a5d70 con 0x7fe4840709b0 2024-03-29T08:28:52.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.378+0000 7fe48c200640 1 -- 172.21.15.46:0/3663240300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4840709b0 msgr2=0x7fe4840af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.378+0000 7fe48c200640 1 --2- 172.21.15.46:0/3663240300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4840709b0 0x7fe4840af9c0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fe48406aa60 tx=0x7fe47c01c7c0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.378+0000 7fe48c200640 1 -- 172.21.15.46:0/3663240300 shutdown_connections 2024-03-29T08:28:52.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.378+0000 7fe48c200640 1 --2- 172.21.15.46:0/3663240300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4840709b0 0x7fe4840af9c0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.378+0000 7fe48c200640 1 --2- 172.21.15.46:0/3663240300 >> v2:172.21.15.73:3300/0 conn(0x7fe484158300 0x7fe48406d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.378+0000 7fe48c200640 1 -- 172.21.15.46:0/3663240300 >> 172.21.15.46:0/3663240300 conn(0x7fe484066080 msgr2=0x7fe484069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.378+0000 7fe48c200640 1 -- 172.21.15.46:0/3663240300 shutdown_connections 2024-03-29T08:28:52.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.378+0000 7fe48c200640 1 -- 172.21.15.46:0/3663240300 wait complete. 2024-03-29T08:28:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.378+0000 7fe48c200640 1 Processor -- start 2024-03-29T08:28:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.378+0000 7fe48c200640 1 -- start start 2024-03-29T08:28:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.378+0000 7fe48c200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4840709b0 0x7fe484151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.378+0000 7fe48c200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4840af530 0x7fe484152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe48c200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe484158300 0x7fe48415de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe48c200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe484164010 con 0x7fe4840af530 2024-03-29T08:28:52.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe48c200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe484152c80 con 0x7fe4840709b0 2024-03-29T08:28:52.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe48c200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe484151a90 con 0x7fe484158300 2024-03-29T08:28:52.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe488e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4840af530 0x7fe484152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe488e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4840af530 0x7fe484152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45306/0 (socket says 172.21.15.46:45306) 2024-03-29T08:28:52.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe488e00640 1 -- 172.21.15.46:0/1916659687 learned_addr learned my addr 172.21.15.46:0/1916659687 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:52.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe489800640 1 --2- 172.21.15.46:0/1916659687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4840709b0 0x7fe484151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe48a200640 1 --2- 172.21.15.46:0/1916659687 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe484158300 0x7fe48415de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe488e00640 1 -- 172.21.15.46:0/1916659687 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe484158300 msgr2=0x7fe48415de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe488e00640 1 --2- 172.21.15.46:0/1916659687 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe484158300 0x7fe48415de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe488e00640 1 -- 172.21.15.46:0/1916659687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4840709b0 msgr2=0x7fe484151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe488e00640 1 --2- 172.21.15.46:0/1916659687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4840709b0 0x7fe484151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe488e00640 1 -- 172.21.15.46:0/1916659687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47c009a70 con 0x7fe4840af530 2024-03-29T08:28:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe489800640 1 --2- 172.21.15.46:0/1916659687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4840709b0 0x7fe484151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe488e00640 1 --2- 172.21.15.46:0/1916659687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4840af530 0x7fe484152740 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe4700011f0 tx=0x7fe47000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe481600640 1 -- 172.21.15.46:0/1916659687 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4700074e0 con 0x7fe4840af530 2024-03-29T08:28:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe481600640 1 -- 172.21.15.46:0/1916659687 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe470007640 con 0x7fe4840af530 2024-03-29T08:28:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe481600640 1 -- 172.21.15.46:0/1916659687 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe470004a10 con 0x7fe4840af530 2024-03-29T08:28:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe48c200640 1 -- 172.21.15.46:0/1916659687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe484151d70 con 0x7fe4840af530 2024-03-29T08:28:52.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe48c200640 1 -- 172.21.15.46:0/1916659687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe484163c30 con 0x7fe4840af530 2024-03-29T08:28:52.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe48c200640 1 -- 172.21.15.46:0/1916659687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4840671b0 con 0x7fe4840af530 2024-03-29T08:28:52.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.382+0000 7fe481600640 1 -- 172.21.15.46:0/1916659687 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe470004b70 con 0x7fe4840af530 2024-03-29T08:28:52.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.386+0000 7fe481600640 1 --2- 172.21.15.46:0/1916659687 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe464066690 0x7fe464068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.386+0000 7fe481600640 1 -- 172.21.15.46:0/1916659687 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe4700834e0 con 0x7fe4840af530 2024-03-29T08:28:52.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.386+0000 7fe489800640 1 --2- 172.21.15.46:0/1916659687 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe464066690 0x7fe464068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.386+0000 7fe489800640 1 --2- 172.21.15.46:0/1916659687 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe464066690 0x7fe464068b50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe478009a40 tx=0x7fe478008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:52.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.390+0000 7fe481600640 1 -- 172.21.15.46:0/1916659687 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe4700507c0 con 0x7fe4840af530 2024-03-29T08:28:52.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.586+0000 7fe48c200640 1 -- 172.21.15.46:0/1916659687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe484071a90 con 0x7fe4840af530 2024-03-29T08:28:52.589 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-03-29T08:28:52.586+0000 7fc16ae00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:28:52.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.594+0000 7fe481600640 1 -- 172.21.15.46:0/1916659687 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe470018090 con 0x7fe4840af530 2024-03-29T08:28:52.597 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T08:28:52.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.594+0000 7fe48c200640 1 -- 172.21.15.46:0/1916659687 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe464066690 msgr2=0x7fe464068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.594+0000 7fe48c200640 1 --2- 172.21.15.46:0/1916659687 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe464066690 0x7fe464068b50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe478009a40 tx=0x7fe478008040 comp rx=0 tx=0).stop 2024-03-29T08:28:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.594+0000 7fe48c200640 1 -- 172.21.15.46:0/1916659687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4840af530 msgr2=0x7fe484152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.594+0000 7fe48c200640 1 --2- 172.21.15.46:0/1916659687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4840af530 0x7fe484152740 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fe4700011f0 tx=0x7fe47000c9b0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.594+0000 7fe48c200640 1 -- 172.21.15.46:0/1916659687 shutdown_connections 2024-03-29T08:28:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.594+0000 7fe48c200640 1 --2- 172.21.15.46:0/1916659687 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe464066690 0x7fe464068b50 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.594+0000 7fe48c200640 1 --2- 172.21.15.46:0/1916659687 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe484158300 0x7fe48415de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.594+0000 7fe48c200640 1 --2- 172.21.15.46:0/1916659687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4840af530 0x7fe484152740 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.594+0000 7fe48c200640 1 --2- 172.21.15.46:0/1916659687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4840709b0 0x7fe484151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.594+0000 7fe48c200640 1 -- 172.21.15.46:0/1916659687 >> 172.21.15.46:0/1916659687 conn(0x7fe484066080 msgr2=0x7fe4840cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.598+0000 7fe48c200640 1 -- 172.21.15.46:0/1916659687 shutdown_connections 2024-03-29T08:28:52.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.598+0000 7fe48c200640 1 -- 172.21.15.46:0/1916659687 wait complete. 2024-03-29T08:28:52.603 INFO:tasks.ceph.mds.c.smithi046.stderr: -994> 2024-03-29T08:28:21.678+0000 7efff8200640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-03-29T08:28:52.604 INFO:tasks.ceph.mds.c.smithi046.stderr: -990> 2024-03-29T08:28:21.678+0000 7efff8200640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-03-29T08:28:52.604 INFO:tasks.ceph.mds.c.smithi046.stderr: -974> 2024-03-29T08:28:21.678+0000 7efff8200640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-03-29T08:28:52.604 INFO:tasks.ceph.mds.c.smithi046.stderr: -971> 2024-03-29T08:28:21.678+0000 7efff8200640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-03-29T08:28:52.619 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-03-29T08:28:52.645 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-03-29T08:28:52.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.658+0000 7ff9400d5400 1 -- start start 2024-03-29T08:28:52.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.658+0000 7ff9400d5400 1 Processor -- start 2024-03-29T08:28:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff9400d5400 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55985013b800 0x5598501c6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff9400d5400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55985013b400 0x5598501c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff9400d5400 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55985013b000 0x5598501c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff9400d5400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559850208a00 con 0x55985013b400 2024-03-29T08:28:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff9400d5400 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559850208780 con 0x55985013b800 2024-03-29T08:28:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff9400d5400 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x559850208640 con 0x55985013b000 2024-03-29T08:28:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55985013b400 0x5598501c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55985013b400 0x5598501c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45344/0 (socket says 172.21.15.46:45344) 2024-03-29T08:28:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93d800640 1 -- 172.21.15.46:0/1820944039 learned_addr learned my addr 172.21.15.46:0/1820944039 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93e200640 1 --2- 172.21.15.46:0/1820944039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55985013b800 0x5598501c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93ec00640 1 --2- 172.21.15.46:0/1820944039 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55985013b000 0x5598501c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93d800640 1 -- 172.21.15.46:0/1820944039 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55985013b000 msgr2=0x5598501c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93d800640 1 --2- 172.21.15.46:0/1820944039 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55985013b000 0x5598501c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93d800640 1 -- 172.21.15.46:0/1820944039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55985013b800 msgr2=0x5598501c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93d800640 1 --2- 172.21.15.46:0/1820944039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55985013b800 0x5598501c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93d800640 1 -- 172.21.15.46:0/1820944039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598501709c0 con 0x55985013b400 2024-03-29T08:28:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93e200640 1 --2- 172.21.15.46:0/1820944039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55985013b800 0x5598501c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93ec00640 1 --2- 172.21.15.46:0/1820944039 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55985013b000 0x5598501c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:52.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93d800640 1 --2- 172.21.15.46:0/1820944039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55985013b400 0x5598501c6b00 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x5598510427b0 tx=0x55985104a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f46facbfcdb96c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93b000640 1 -- 172.21.15.46:0/1820944039 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55985018f600 con 0x55985013b400 2024-03-29T08:28:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93b000640 1 -- 172.21.15.46:0/1820944039 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559850198480 con 0x55985013b400 2024-03-29T08:28:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93b000640 1 -- 172.21.15.46:0/1820944039 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55985018f1e0 con 0x55985013b400 2024-03-29T08:28:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff9400d5400 1 -- 172.21.15.46:0/1820944039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559850170b60 con 0x55985013b400 2024-03-29T08:28:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff9400d5400 1 -- 172.21.15.46:0/1820944039 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x559850170340 con 0x55985013b400 2024-03-29T08:28:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff9400d5400 4 waiting for MDS map... 2024-03-29T08:28:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93b000640 1 -- 172.21.15.46:0/1820944039 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 5939+0+0 (secure 0 0 0) 0x55985017ac40 con 0x55985013b400 2024-03-29T08:28:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff93b000640 1 -- 172.21.15.46:0/1820944039 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 84) v1 ==== 2573+0+0 (secure 0 0 0) 0x559850188f00 con 0x55985013b400 2024-03-29T08:28:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff9400d5400 4 Got MDS map 84 2024-03-29T08:28:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff9400d5400 10 main: main 2024-03-29T08:28:52.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff9400d5400 4 main: connecting to RADOS... 2024-03-29T08:28:52.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff9400d5400 1 Processor -- start 2024-03-29T08:28:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.662+0000 7ff9400d5400 1 -- start start 2024-03-29T08:28:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559851050c00 0x5598501c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559851051000 0x5598501c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559851051400 0x5598501c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x559850181e00 con 0x559851051400 2024-03-29T08:28:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559850208780 con 0x559851051000 2024-03-29T08:28:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x559850208500 con 0x559851050c00 2024-03-29T08:28:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff93d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559851051400 0x5598501c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff93e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559851051000 0x5598501c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff93e200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559851051000 0x5598501c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:51062/0 (socket says 172.21.15.46:51062) 2024-03-29T08:28:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff93e200640 1 -- 172.21.15.46:0/1619392125 learned_addr learned my addr 172.21.15.46:0/1619392125 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff93ec00640 1 --2- 172.21.15.46:0/1619392125 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559851050c00 0x5598501c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff93e200640 1 -- 172.21.15.46:0/1619392125 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559851050c00 msgr2=0x5598501c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff93e200640 1 --2- 172.21.15.46:0/1619392125 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559851050c00 0x5598501c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff93e200640 1 -- 172.21.15.46:0/1619392125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559851051400 msgr2=0x5598501c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff93e200640 1 --2- 172.21.15.46:0/1619392125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559851051400 0x5598501c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff93e200640 1 -- 172.21.15.46:0/1619392125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559850170000 con 0x559851051000 2024-03-29T08:28:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff93e200640 1 --2- 172.21.15.46:0/1619392125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559851051000 0x5598501c7b80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x559851043bf0 tx=0x55985104a120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=547503094a621a7a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff938800640 1 -- 172.21.15.46:0/1619392125 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55985018f080 con 0x559851051000 2024-03-29T08:28:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff938800640 1 -- 172.21.15.46:0/1619392125 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559850f10a80 con 0x559851051000 2024-03-29T08:28:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff938800640 1 -- 172.21.15.46:0/1619392125 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55985018f600 con 0x559851051000 2024-03-29T08:28:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 -- 172.21.15.46:0/1619392125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559851051000 msgr2=0x5598501c7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 --2- 172.21.15.46:0/1619392125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559851051000 0x5598501c7b80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x559851043bf0 tx=0x55985104a120 comp rx=0 tx=0).stop 2024-03-29T08:28:52.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 -- 172.21.15.46:0/1619392125 shutdown_connections 2024-03-29T08:28:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 --2- 172.21.15.46:0/1619392125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559851051000 0x5598501c7b80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 --2- 172.21.15.46:0/1619392125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559851051400 0x5598501c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 --2- 172.21.15.46:0/1619392125 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559851050c00 0x5598501c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 -- 172.21.15.46:0/1619392125 >> 172.21.15.46:0/1619392125 conn(0x55985013ac00 msgr2=0x559850169800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 -- 172.21.15.46:0/1619392125 shutdown_connections 2024-03-29T08:28:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 -- 172.21.15.46:0/1619392125 wait complete. 2024-03-29T08:28:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 Processor -- start 2024-03-29T08:28:52.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 -- start start 2024-03-29T08:28:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559851050c00 0x5598501c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559851051000 0x5598501c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559851051400 0x5598501c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598502083c0 con 0x559851050c00 2024-03-29T08:28:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559850208280 con 0x559851051000 2024-03-29T08:28:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff9400d5400 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x559850208140 con 0x559851051400 2024-03-29T08:28:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff93ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559851050c00 0x5598501c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.666+0000 7ff93ec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559851050c00 0x5598501c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45358/0 (socket says 172.21.15.46:45358) 2024-03-29T08:28:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff93ec00640 1 -- 172.21.15.46:0/3896976950 learned_addr learned my addr 172.21.15.46:0/3896976950 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:52.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff93d800640 1 --2- 172.21.15.46:0/3896976950 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559851051400 0x5598501c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff93e200640 1 --2- 172.21.15.46:0/3896976950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559851051000 0x5598501c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff93ec00640 1 -- 172.21.15.46:0/3896976950 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559851051400 msgr2=0x5598501c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff93ec00640 1 --2- 172.21.15.46:0/3896976950 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x559851051400 0x5598501c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff93ec00640 1 -- 172.21.15.46:0/3896976950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559851051000 msgr2=0x5598501c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff93ec00640 1 --2- 172.21.15.46:0/3896976950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559851051000 0x5598501c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff93ec00640 1 -- 172.21.15.46:0/3896976950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598501701a0 con 0x559851050c00 2024-03-29T08:28:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff93ec00640 1 --2- 172.21.15.46:0/3896976950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559851050c00 0x5598501c9180 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x559850420c30 tx=0x559851068cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff936a00640 1 -- 172.21.15.46:0/3896976950 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55985018f080 con 0x559851050c00 2024-03-29T08:28:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff936a00640 1 -- 172.21.15.46:0/3896976950 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559850198180 con 0x559851050c00 2024-03-29T08:28:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff936a00640 1 -- 172.21.15.46:0/3896976950 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55985018f600 con 0x559851050c00 2024-03-29T08:28:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff9400d5400 1 -- 172.21.15.46:0/3896976950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559850170000 con 0x559851050c00 2024-03-29T08:28:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff9400d5400 1 -- 172.21.15.46:0/3896976950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598501709c0 con 0x559851050c00 2024-03-29T08:28:52.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff9400d5400 4 main: resolving pool 14 2024-03-29T08:28:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff936a00640 1 -- 172.21.15.46:0/3896976950 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x559850206840 con 0x559851050c00 2024-03-29T08:28:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff936a00640 1 --2- 172.21.15.46:0/3896976950 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x559850f02c00 0x5598501c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff936a00640 1 -- 172.21.15.46:0/3896976950 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5939+0+0 (secure 0 0 0) 0x55985017aa80 con 0x559851050c00 2024-03-29T08:28:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff9400d5400 4 main: creating IoCtx.. 2024-03-29T08:28:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff9400d5400 1 --2- 172.21.15.46:0/3896976950 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x559851051800 0x5598501c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff9400d5400 1 -- 172.21.15.46:0/3896976950 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x559851051c00 con 0x559851051800 2024-03-29T08:28:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff93e200640 1 --2- 172.21.15.46:0/3896976950 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x559850f02c00 0x5598501c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.670+0000 7ff93d800640 1 --2- 172.21.15.46:0/3896976950 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x559851051800 0x5598501c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.674+0000 7ff93e200640 1 --2- 172.21.15.46:0/3896976950 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x559850f02c00 0x5598501c9700 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x559851043bf0 tx=0x55985104a000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:52.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.674+0000 7ff93d800640 1 --2- 172.21.15.46:0/3896976950 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x559851051800 0x5598501c8100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:52.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.674+0000 7ff93d800640 1 -- 172.21.15.46:0/3896976950 <== osd.5 v2:172.21.15.73:6808/27690 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x55985103eb40 con 0x559851051800 2024-03-29T08:28:52.680 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:28:52.680 INFO:teuthology.orchestra.run.smithi046.stdout: "0": { 2024-03-29T08:28:52.680 INFO:teuthology.orchestra.run.smithi046.stdout: "data": { 2024-03-29T08:28:52.680 INFO:teuthology.orchestra.run.smithi046.stdout: "version": 5, 2024-03-29T08:28:52.680 INFO:teuthology.orchestra.run.smithi046.stdout: "inotable": { 2024-03-29T08:28:52.680 INFO:teuthology.orchestra.run.smithi046.stdout: "projected_free": [ 2024-03-29T08:28:52.680 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:28:52.680 INFO:teuthology.orchestra.run.smithi046.stdout: "start": 1099511627779, 2024-03-29T08:28:52.680 INFO:teuthology.orchestra.run.smithi046.stdout: "len": 1099511627773 2024-03-29T08:28:52.680 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:28:52.681 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-03-29T08:28:52.681 INFO:teuthology.orchestra.run.smithi046.stdout: "free": [ 2024-03-29T08:28:52.681 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:28:52.681 INFO:teuthology.orchestra.run.smithi046.stdout: "start": 1099511627779, 2024-03-29T08:28:52.681 INFO:teuthology.orchestra.run.smithi046.stdout: "len": 1099511627773 2024-03-29T08:28:52.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:28:52.681 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-29T08:28:52.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:28:52.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:28:52.681 INFO:teuthology.orchestra.run.smithi046.stdout: "result": 0 2024-03-29T08:28:52.681 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:28:52.681 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:28:52.681 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:28:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.674+0000 7ff9400d5400 1 -- 172.21.15.46:0/3896976950 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x559851051800 msgr2=0x5598501c8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.674+0000 7ff9400d5400 1 --2- 172.21.15.46:0/3896976950 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x559851051800 0x5598501c8100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.674+0000 7ff9400d5400 1 -- 172.21.15.46:0/3896976950 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x559850f02c00 msgr2=0x5598501c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.674+0000 7ff9400d5400 1 --2- 172.21.15.46:0/3896976950 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x559850f02c00 0x5598501c9700 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x559851043bf0 tx=0x55985104a000 comp rx=0 tx=0).stop 2024-03-29T08:28:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.674+0000 7ff9400d5400 1 -- 172.21.15.46:0/3896976950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559851050c00 msgr2=0x5598501c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.674+0000 7ff9400d5400 1 --2- 172.21.15.46:0/3896976950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559851050c00 0x5598501c9180 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x559850420c30 tx=0x559851068cc0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.674+0000 7ff93e200640 1 -- 172.21.15.46:0/3896976950 reap_dead start 2024-03-29T08:28:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.674+0000 7ff9400d5400 1 -- 172.21.15.46:0/3896976950 shutdown_connections 2024-03-29T08:28:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.674+0000 7ff9400d5400 1 -- 172.21.15.46:0/3896976950 >> 172.21.15.46:0/3896976950 conn(0x55985013ac00 msgr2=0x559850169800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.674+0000 7ff9400d5400 1 -- 172.21.15.46:0/3896976950 shutdown_connections 2024-03-29T08:28:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.674+0000 7ff9400d5400 1 -- 172.21.15.46:0/3896976950 wait complete. 2024-03-29T08:28:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.678+0000 7ff9400d5400 1 -- 172.21.15.46:0/1820944039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55985013b400 msgr2=0x5598501c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:52.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.678+0000 7ff9400d5400 1 --2- 172.21.15.46:0/1820944039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55985013b400 0x5598501c6b00 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x5598510427b0 tx=0x55985104a2a0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.678+0000 7ff9400d5400 1 -- 172.21.15.46:0/1820944039 shutdown_connections 2024-03-29T08:28:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.678+0000 7ff9400d5400 1 --2- 172.21.15.46:0/1820944039 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55985013b000 0x5598501c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.678+0000 7ff9400d5400 1 --2- 172.21.15.46:0/1820944039 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55985013b400 0x5598501c6b00 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.678+0000 7ff9400d5400 1 --2- 172.21.15.46:0/1820944039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55985013b800 0x5598501c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.678+0000 7ff9400d5400 1 -- 172.21.15.46:0/1820944039 >> 172.21.15.46:0/1820944039 conn(0x55985013bc00 msgr2=0x559850169000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.678+0000 7ff9400d5400 1 -- 172.21.15.46:0/1820944039 shutdown_connections 2024-03-29T08:28:52.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:52.678+0000 7ff9400d5400 1 -- 172.21.15.46:0/1820944039 wait complete. 2024-03-29T08:28:52.685 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.065325, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-03-29T08:28:52.685 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T08:28:52.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:28:52.810 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:28:52.889 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T08:28:52.890 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:28:52.890 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:28:52.899 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:28:52.900 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:28:52.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:28:52.900 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:28:52.951 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-29T08:28:52.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:28:52.952 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:28:52.952 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:28:52.961 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:28:52.961 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:28:52.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:28:52.962 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T08:28:53.010 INFO:teuthology.orchestra.run.smithi073.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-29T08:28:53.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:28:53.011 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:28:53.011 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:28:53.021 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:28:53.021 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:28:53.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:28:53.021 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-29T08:28:53.071 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-29T08:28:53.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:28:53.072 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:28:53.072 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:28:53.082 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:28:53.082 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:28:53.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:28:53.082 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-29T08:28:53.134 INFO:teuthology.orchestra.run.smithi073.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-29T08:28:53.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:28:53.135 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:28:53.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.270+0000 7fa9d3600640 1 Processor -- start 2024-03-29T08:28:53.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.270+0000 7fa9d3600640 1 -- start start 2024-03-29T08:28:53.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9d3600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9cc15c9d0 0x7fa9cc0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9d3600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9cc0d0030 con 0x7fa9cc099150 2024-03-29T08:28:53.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9d3600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9cc0cf420 con 0x7fa9cc0cef60 2024-03-29T08:28:53.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9d3600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc0cf700 con 0x7fa9cc15c9d0 2024-03-29T08:28:53.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9d1600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa9cc099150 0x7fa9cc166a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54018/0 (socket says 172.21.15.46:54018) 2024-03-29T08:28:53.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9d1600640 1 -- 172.21.15.46:0/1068413807 learned_addr learned my addr 172.21.15.46:0/1068413807 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:28:53.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9d0c00640 1 --2- 172.21.15.46:0/1068413807 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9cc15c9d0 0x7fa9cc0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cb400640 1 -- 172.21.15.46:0/1068413807 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3256677851 0 0) 0x7fa9cc0d0030 con 0x7fa9cc099150 2024-03-29T08:28:53.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cb400640 1 -- 172.21.15.46:0/1068413807 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9b4003530 con 0x7fa9cc099150 2024-03-29T08:28:53.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cb400640 1 -- 172.21.15.46:0/1068413807 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2835449529 0 0) 0x7fa9cc0cf420 con 0x7fa9cc0cef60 2024-03-29T08:28:53.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cb400640 1 -- 172.21.15.46:0/1068413807 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9cc0d0030 con 0x7fa9cc0cef60 2024-03-29T08:28:53.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cb400640 1 -- 172.21.15.46:0/1068413807 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1359886271 0 0) 0x7fa9b4003530 con 0x7fa9cc099150 2024-03-29T08:28:53.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cb400640 1 -- 172.21.15.46:0/1068413807 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9cc15c9d0 msgr2=0x7fa9cc0cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:53.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cb400640 1 --2- 172.21.15.46:0/1068413807 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9cc15c9d0 0x7fa9cc0cea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cb400640 1 -- 172.21.15.46:0/1068413807 >> v1:172.21.15.73:6789/0 conn(0x7fa9cc0cef60 legacy=0x7fa9cc098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cb400640 1 -- 172.21.15.46:0/1068413807 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc0cf9e0 con 0x7fa9cc099150 2024-03-29T08:28:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cb400640 1 -- 172.21.15.46:0/1068413807 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fa9c4001a00 con 0x7fa9cc099150 2024-03-29T08:28:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cb400640 1 -- 172.21.15.46:0/1068413807 >> v1:172.21.15.46:6789/0 conn(0x7fa9cc099150 legacy=0x7fa9cc166a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cb400640 1 --2- 172.21.15.46:0/1068413807 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9b4003b40 0x7fa9b4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cb400640 1 --2- 172.21.15.46:0/1068413807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9b40047d0 0x7fa9b4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cb400640 1 --2- 172.21.15.46:0/1068413807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b4005190 0x7fa9b400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cb400640 1 -- 172.21.15.46:0/1068413807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc0cf700 con 0x7fa9b4005190 2024-03-29T08:28:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cb400640 1 -- 172.21.15.46:0/1068413807 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b400bfb0 con 0x7fa9b40047d0 2024-03-29T08:28:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cb400640 1 -- 172.21.15.46:0/1068413807 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b400c170 con 0x7fa9b4003b40 2024-03-29T08:28:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9d0c00640 1 --2- 172.21.15.46:0/1068413807 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9cc15c9d0 0x7fa9cc0cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:28:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9d1600640 1 --2- 172.21.15.46:0/1068413807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b4005190 0x7fa9b400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cbe00640 1 --2- 172.21.15.46:0/1068413807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9b40047d0 0x7fa9b4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9d0c00640 1 --2- 172.21.15.46:0/1068413807 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9b4003b40 0x7fa9b4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9d1600640 1 -- 172.21.15.46:0/1068413807 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9b4003b40 msgr2=0x7fa9b4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9d1600640 1 --2- 172.21.15.46:0/1068413807 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9b4003b40 0x7fa9b4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9d1600640 1 -- 172.21.15.46:0/1068413807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9b40047d0 msgr2=0x7fa9b4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9d1600640 1 --2- 172.21.15.46:0/1068413807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9b40047d0 0x7fa9b4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9d1600640 1 -- 172.21.15.46:0/1068413807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b4003530 con 0x7fa9b4005190 2024-03-29T08:28:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.274+0000 7fa9cbe00640 1 --2- 172.21.15.46:0/1068413807 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9b40047d0 0x7fa9b4004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:28:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.278+0000 7fa9d1600640 1 --2- 172.21.15.46:0/1068413807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b4005190 0x7fa9b400b990 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400a700 tx=0x7fa9c401f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0a5550aea57bdb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:53.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.278+0000 7fa9cb400640 1 -- 172.21.15.46:0/1068413807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c40ac610 con 0x7fa9b4005190 2024-03-29T08:28:53.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.278+0000 7fa9cb400640 1 -- 172.21.15.46:0/1068413807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c40ac770 con 0x7fa9b4005190 2024-03-29T08:28:53.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.278+0000 7fa9cb400640 1 -- 172.21.15.46:0/1068413807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c40acb80 con 0x7fa9b4005190 2024-03-29T08:28:53.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.278+0000 7fa9d3600640 1 -- 172.21.15.46:0/1068413807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b4005190 msgr2=0x7fa9b400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.278+0000 7fa9d3600640 1 --2- 172.21.15.46:0/1068413807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9b4005190 0x7fa9b400b990 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fa9c400a700 tx=0x7fa9c401f960 comp rx=0 tx=0).stop 2024-03-29T08:28:53.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.278+0000 7fa9d1600640 1 -- 172.21.15.46:0/1068413807 reap_dead start 2024-03-29T08:28:53.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.278+0000 7fa9d3600640 1 -- 172.21.15.46:0/1068413807 shutdown_connections 2024-03-29T08:28:53.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.278+0000 7fa9d3600640 1 -- 172.21.15.46:0/1068413807 >> 172.21.15.46:0/1068413807 conn(0x7fa9cc0b21a0 msgr2=0x7fa9cc0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:53.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.278+0000 7fa9d3600640 1 -- 172.21.15.46:0/1068413807 shutdown_connections 2024-03-29T08:28:53.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.278+0000 7fa9d3600640 1 -- 172.21.15.46:0/1068413807 wait complete. 2024-03-29T08:28:53.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d3600640 1 Processor -- start 2024-03-29T08:28:53.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d3600640 1 -- start start 2024-03-29T08:28:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d3600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9cc0c6160 0x7fa9cc06be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d3600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9cc156ce0 0x7fa9cc0cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d3600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9cc0c6540 0x7fa9cc14c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d3600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc0c5800 con 0x7fa9cc0c6160 2024-03-29T08:28:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d3600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc0ccbe0 con 0x7fa9cc156ce0 2024-03-29T08:28:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d3600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc0c5240 con 0x7fa9cc0c6540 2024-03-29T08:28:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9cc0c6160 0x7fa9cc06be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d0c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9cc0c6160 0x7fa9cc06be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45372/0 (socket says 172.21.15.46:45372) 2024-03-29T08:28:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d0c00640 1 -- 172.21.15.46:0/783072196 learned_addr learned my addr 172.21.15.46:0/783072196 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9cbe00640 1 --2- 172.21.15.46:0/783072196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9cc156ce0 0x7fa9cc0cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d1600640 1 --2- 172.21.15.46:0/783072196 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9cc0c6540 0x7fa9cc14c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d0c00640 1 -- 172.21.15.46:0/783072196 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9cc0c6540 msgr2=0x7fa9cc14c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d0c00640 1 --2- 172.21.15.46:0/783072196 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9cc0c6540 0x7fa9cc14c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d0c00640 1 -- 172.21.15.46:0/783072196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9cc156ce0 msgr2=0x7fa9cc0cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d0c00640 1 --2- 172.21.15.46:0/783072196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9cc156ce0 0x7fa9cc0cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d0c00640 1 -- 172.21.15.46:0/783072196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc0c5520 con 0x7fa9cc0c6160 2024-03-29T08:28:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d0c00640 1 --2- 172.21.15.46:0/783072196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9cc0c6160 0x7fa9cc06be10 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0011620 tx=0x7fa9c0011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:53.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9c8c00640 1 -- 172.21.15.46:0/783072196 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c000f920 con 0x7fa9cc0c6160 2024-03-29T08:28:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9c8c00640 1 -- 172.21.15.46:0/783072196 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c000fa80 con 0x7fa9cc0c6160 2024-03-29T08:28:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9c8c00640 1 -- 172.21.15.46:0/783072196 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c000fe30 con 0x7fa9cc0c6160 2024-03-29T08:28:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d3600640 1 -- 172.21.15.46:0/783072196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9cc0cc870 con 0x7fa9cc0c6160 2024-03-29T08:28:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d3600640 1 -- 172.21.15.46:0/783072196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9cc1573c0 con 0x7fa9cc0c6160 2024-03-29T08:28:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.282+0000 7fa9d3600640 1 -- 172.21.15.46:0/783072196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9cc0cca00 con 0x7fa9cc0c6160 2024-03-29T08:28:53.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.286+0000 7fa9c8c00640 1 -- 172.21.15.46:0/783072196 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa9c0015460 con 0x7fa9cc0c6160 2024-03-29T08:28:53.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.286+0000 7fa9c8c00640 1 --2- 172.21.15.46:0/783072196 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa9ac066690 0x7fa9ac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.286+0000 7fa9c8c00640 1 -- 172.21.15.46:0/783072196 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fa9c0084a30 con 0x7fa9cc0c6160 2024-03-29T08:28:53.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.286+0000 7fa9cbe00640 1 --2- 172.21.15.46:0/783072196 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa9ac066690 0x7fa9ac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.286+0000 7fa9cbe00640 1 --2- 172.21.15.46:0/783072196 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa9ac066690 0x7fa9ac068b50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8000c10 tx=0x7fa9b8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.290+0000 7fa9c8c00640 1 -- 172.21.15.46:0/783072196 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa9c0051d50 con 0x7fa9cc0c6160 2024-03-29T08:28:53.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.482+0000 7fa9d3600640 1 -- 172.21.15.46:0/783072196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9cc066c90 con 0x7fa9cc0c6160 2024-03-29T08:28:53.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.482+0000 7fa9c8c00640 1 -- 172.21.15.46:0/783072196 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+3970 (secure 0 0 0) 0x7fa9c0055370 con 0x7fa9cc0c6160 2024-03-29T08:28:53.487 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:28:53.487 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":84,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4230030957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4230030957},{"type":"v1","addr":"172.21.15.73:6836","nonce":4230030957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/297138217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":297138217},{"type":"v1","addr":"172.21.15.73:6837","nonce":297138217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3856305332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3856305332},{"type":"v1","addr":"172.21.15.46:6833","nonce":3856305332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":84,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:27:18.583223+0000","modified":"2024-03-29T08:28:52.591232+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-29T08:28:53.487 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 84 2024-03-29T08:28:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.486+0000 7fa9d3600640 1 -- 172.21.15.46:0/783072196 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa9ac066690 msgr2=0x7fa9ac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.486+0000 7fa9d3600640 1 --2- 172.21.15.46:0/783072196 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa9ac066690 0x7fa9ac068b50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8000c10 tx=0x7fa9b8009040 comp rx=0 tx=0).stop 2024-03-29T08:28:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.486+0000 7fa9d3600640 1 -- 172.21.15.46:0/783072196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9cc0c6160 msgr2=0x7fa9cc06be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.486+0000 7fa9d3600640 1 --2- 172.21.15.46:0/783072196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9cc0c6160 0x7fa9cc06be10 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0011620 tx=0x7fa9c0011ae0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.486+0000 7fa9d3600640 1 -- 172.21.15.46:0/783072196 shutdown_connections 2024-03-29T08:28:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.486+0000 7fa9d3600640 1 --2- 172.21.15.46:0/783072196 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa9ac066690 0x7fa9ac068b50 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.486+0000 7fa9d3600640 1 --2- 172.21.15.46:0/783072196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9cc156ce0 0x7fa9cc0cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.486+0000 7fa9d3600640 1 --2- 172.21.15.46:0/783072196 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9cc0c6540 0x7fa9cc14c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.486+0000 7fa9d3600640 1 --2- 172.21.15.46:0/783072196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9cc0c6160 0x7fa9cc06be10 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.486+0000 7fa9d3600640 1 -- 172.21.15.46:0/783072196 >> 172.21.15.46:0/783072196 conn(0x7fa9cc0b21a0 msgr2=0x7fa9cc0c7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.486+0000 7fa9d3600640 1 -- 172.21.15.46:0/783072196 shutdown_connections 2024-03-29T08:28:53.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.486+0000 7fa9d3600640 1 -- 172.21.15.46:0/783072196 wait complete. 2024-03-29T08:28:53.508 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:28:53.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.646+0000 7fee40400640 1 Processor -- start 2024-03-29T08:28:53.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.646+0000 7fee40400640 1 -- start start 2024-03-29T08:28:53.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.646+0000 7fee40400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fee3815a7c0 0x7fee3806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.646+0000 7fee40400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3806d130 con 0x7fee381593d0 2024-03-29T08:28:53.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.646+0000 7fee40400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3815b8f0 con 0x7fee3815cd90 2024-03-29T08:28:53.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.646+0000 7fee40400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee3815ad50 con 0x7fee3815a7c0 2024-03-29T08:28:53.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.646+0000 7fee3da00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fee381593d0 0x7fee3815d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54020/0 (socket says 172.21.15.46:54020) 2024-03-29T08:28:53.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.646+0000 7fee3da00640 1 -- 172.21.15.46:0/2483702331 learned_addr learned my addr 172.21.15.46:0/2483702331 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:28:53.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee3e400640 1 --2- 172.21.15.46:0/2483702331 >> v2:172.21.15.73:3300/0 conn(0x7fee3815a7c0 0x7fee3806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee37e00640 1 -- 172.21.15.46:0/2483702331 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769961587 0 0) 0x7fee3806d130 con 0x7fee381593d0 2024-03-29T08:28:53.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee37e00640 1 -- 172.21.15.46:0/2483702331 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee20003530 con 0x7fee381593d0 2024-03-29T08:28:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee37e00640 1 -- 172.21.15.46:0/2483702331 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 76976157 0 0) 0x7fee20003530 con 0x7fee381593d0 2024-03-29T08:28:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee37e00640 1 -- 172.21.15.46:0/2483702331 >> v2:172.21.15.73:3300/0 conn(0x7fee3815a7c0 msgr2=0x7fee3806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee37e00640 1 --2- 172.21.15.46:0/2483702331 >> v2:172.21.15.73:3300/0 conn(0x7fee3815a7c0 0x7fee3806cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee37e00640 1 -- 172.21.15.46:0/2483702331 >> v1:172.21.15.73:6789/0 conn(0x7fee3815cd90 legacy=0x7fee3815d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee37e00640 1 -- 172.21.15.46:0/2483702331 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3815e050 con 0x7fee381593d0 2024-03-29T08:28:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee37e00640 1 -- 172.21.15.46:0/2483702331 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fee2c001680 con 0x7fee381593d0 2024-03-29T08:28:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee37e00640 1 -- 172.21.15.46:0/2483702331 >> v1:172.21.15.46:6789/0 conn(0x7fee381593d0 legacy=0x7fee3815d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee37e00640 1 --2- 172.21.15.46:0/2483702331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee20003980 0x7fee20003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee37e00640 1 --2- 172.21.15.46:0/2483702331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee200047d0 0x7fee20004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee37e00640 1 --2- 172.21.15.46:0/2483702331 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fee20005190 0x7fee2000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee37e00640 1 -- 172.21.15.46:0/2483702331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3815ad50 con 0x7fee20003980 2024-03-29T08:28:53.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee37e00640 1 -- 172.21.15.46:0/2483702331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2000c030 con 0x7fee200047d0 2024-03-29T08:28:53.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee37e00640 1 -- 172.21.15.46:0/2483702331 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2000c1f0 con 0x7fee20005190 2024-03-29T08:28:53.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee3da00640 1 --2- 172.21.15.46:0/2483702331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee20003980 0x7fee20003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee3d000640 1 --2- 172.21.15.46:0/2483702331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee200047d0 0x7fee20004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee3e400640 1 --2- 172.21.15.46:0/2483702331 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fee20005190 0x7fee2000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee3da00640 1 -- 172.21.15.46:0/2483702331 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fee20005190 msgr2=0x7fee2000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:53.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee3da00640 1 --2- 172.21.15.46:0/2483702331 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fee20005190 0x7fee2000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee3da00640 1 -- 172.21.15.46:0/2483702331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee200047d0 msgr2=0x7fee20004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:53.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee3da00640 1 --2- 172.21.15.46:0/2483702331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee200047d0 0x7fee20004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee3da00640 1 -- 172.21.15.46:0/2483702331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee20003530 con 0x7fee20003980 2024-03-29T08:28:53.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.650+0000 7fee3e400640 1 -- 172.21.15.46:0/2483702331 reap_dead start 2024-03-29T08:28:53.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee3da00640 1 --2- 172.21.15.46:0/2483702331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee20003980 0x7fee20003da0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00c240 tx=0x7fee2c01cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4570285e787b11 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:53.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee37e00640 1 -- 172.21.15.46:0/2483702331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c005ae0 con 0x7fee20003980 2024-03-29T08:28:53.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee37e00640 1 -- 172.21.15.46:0/2483702331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c005c40 con 0x7fee20003980 2024-03-29T08:28:53.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee37e00640 1 -- 172.21.15.46:0/2483702331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c0a5aa0 con 0x7fee20003980 2024-03-29T08:28:53.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee40400640 1 -- 172.21.15.46:0/2483702331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee20003980 msgr2=0x7fee20003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee40400640 1 --2- 172.21.15.46:0/2483702331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee20003980 0x7fee20003da0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fee2c00c240 tx=0x7fee2c01cbe0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee40400640 1 -- 172.21.15.46:0/2483702331 shutdown_connections 2024-03-29T08:28:53.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee40400640 1 --2- 172.21.15.46:0/2483702331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee20003980 0x7fee20003da0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee40400640 1 -- 172.21.15.46:0/2483702331 >> 172.21.15.46:0/2483702331 conn(0x7fee38066110 msgr2=0x7fee38069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:53.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee40400640 1 -- 172.21.15.46:0/2483702331 shutdown_connections 2024-03-29T08:28:53.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee40400640 1 -- 172.21.15.46:0/2483702331 wait complete. 2024-03-29T08:28:53.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee40400640 1 Processor -- start 2024-03-29T08:28:53.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee40400640 1 -- start start 2024-03-29T08:28:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee40400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee20003980 0x7fee38158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee40400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fee380a74a0 0x7fee380a9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee40400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee380a8c90 0x7fee380b1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee40400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee380a8800 con 0x7fee380a8c90 2024-03-29T08:28:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee40400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee380aa100 con 0x7fee20003980 2024-03-29T08:28:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee40400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee380a7bf0 con 0x7fee380a74a0 2024-03-29T08:28:53.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee3e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee380a8c90 0x7fee380b1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee3e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee380a8c90 0x7fee380b1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45396/0 (socket says 172.21.15.46:45396) 2024-03-29T08:28:53.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.654+0000 7fee3e400640 1 -- 172.21.15.46:0/1637612649 learned_addr learned my addr 172.21.15.46:0/1637612649 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:53.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.658+0000 7fee3da00640 1 --2- 172.21.15.46:0/1637612649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee20003980 0x7fee38158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.658+0000 7fee3d000640 1 --2- 172.21.15.46:0/1637612649 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fee380a74a0 0x7fee380a9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.658+0000 7fee3e400640 1 -- 172.21.15.46:0/1637612649 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fee380a74a0 msgr2=0x7fee380a9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.658+0000 7fee3e400640 1 --2- 172.21.15.46:0/1637612649 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fee380a74a0 0x7fee380a9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.658+0000 7fee3e400640 1 -- 172.21.15.46:0/1637612649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee20003980 msgr2=0x7fee38158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.658+0000 7fee3e400640 1 --2- 172.21.15.46:0/1637612649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee20003980 0x7fee38158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.658+0000 7fee3e400640 1 -- 172.21.15.46:0/1637612649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee380a7ed0 con 0x7fee380a8c90 2024-03-29T08:28:53.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.658+0000 7fee3d000640 1 --2- 172.21.15.46:0/1637612649 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fee380a74a0 0x7fee380a9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:53.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.658+0000 7fee3e400640 1 --2- 172.21.15.46:0/1637612649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee380a8c90 0x7fee380b1be0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fee380cb030 tx=0x7fee3000f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:53.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.658+0000 7fee35600640 1 -- 172.21.15.46:0/1637612649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3000e040 con 0x7fee380a8c90 2024-03-29T08:28:53.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.658+0000 7fee35600640 1 -- 172.21.15.46:0/1637612649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3000f040 con 0x7fee380a8c90 2024-03-29T08:28:53.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.658+0000 7fee35600640 1 -- 172.21.15.46:0/1637612649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30015430 con 0x7fee380a8c90 2024-03-29T08:28:53.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.658+0000 7fee40400640 1 -- 172.21.15.46:0/1637612649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee380a81b0 con 0x7fee380a8c90 2024-03-29T08:28:53.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.658+0000 7fee40400640 1 -- 172.21.15.46:0/1637612649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee380a7900 con 0x7fee380a8c90 2024-03-29T08:28:53.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.658+0000 7fee40400640 1 -- 172.21.15.46:0/1637612649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee380cf170 con 0x7fee380a8c90 2024-03-29T08:28:53.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.658+0000 7fee35600640 1 -- 172.21.15.46:0/1637612649 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fee30015590 con 0x7fee380a8c90 2024-03-29T08:28:53.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.662+0000 7fee35600640 1 --2- 172.21.15.46:0/1637612649 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fee18066690 0x7fee18068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:53.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.662+0000 7fee35600640 1 -- 172.21.15.46:0/1637612649 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fee300849a0 con 0x7fee380a8c90 2024-03-29T08:28:53.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.662+0000 7fee3da00640 1 --2- 172.21.15.46:0/1637612649 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fee18066690 0x7fee18068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:53.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.662+0000 7fee3da00640 1 --2- 172.21.15.46:0/1637612649 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fee18066690 0x7fee18068b50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0ab000 tx=0x7fee2c0b1040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:53.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.666+0000 7fee35600640 1 -- 172.21.15.46:0/1637612649 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fee30051b80 con 0x7fee380a8c90 2024-03-29T08:28:53.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.830+0000 7fee40400640 1 -- 172.21.15.46:0/1637612649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee38066c90 con 0x7fee380a8c90 2024-03-29T08:28:53.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.830+0000 7fee35600640 1 -- 172.21.15.46:0/1637612649 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16357 (secure 0 0 0) 0x7fee300551a0 con 0x7fee380a8c90 2024-03-29T08:28:53.836 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:28:53.836 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":110,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:28:52.590731+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:27:17.118875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-29T08:27:17.573928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.f","mappings":[{"from":5,"to":0}]},{"pgid":"14.14","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1455003836":"2024-03-30T08:28:52.590708+0000","172.21.15.46:6834/1455003836":"2024-03-30T08:28:52.590708+0000","172.21.15.46:6835/824184385":"2024-03-30T08:28:15.875168+0000","172.21.15.46:6834/824184385":"2024-03-30T08:28:15.875168+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:28:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.834+0000 7fee40400640 1 -- 172.21.15.46:0/1637612649 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fee18066690 msgr2=0x7fee18068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.834+0000 7fee40400640 1 --2- 172.21.15.46:0/1637612649 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fee18066690 0x7fee18068b50 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0ab000 tx=0x7fee2c0b1040 comp rx=0 tx=0).stop 2024-03-29T08:28:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.834+0000 7fee40400640 1 -- 172.21.15.46:0/1637612649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee380a8c90 msgr2=0x7fee380b1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.834+0000 7fee40400640 1 --2- 172.21.15.46:0/1637612649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee380a8c90 0x7fee380b1be0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7fee380cb030 tx=0x7fee3000f1b0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.834+0000 7fee40400640 1 -- 172.21.15.46:0/1637612649 shutdown_connections 2024-03-29T08:28:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.834+0000 7fee40400640 1 --2- 172.21.15.46:0/1637612649 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fee18066690 0x7fee18068b50 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.834+0000 7fee40400640 1 --2- 172.21.15.46:0/1637612649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee380a8c90 0x7fee380b1be0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.834+0000 7fee40400640 1 --2- 172.21.15.46:0/1637612649 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fee380a74a0 0x7fee380a9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.834+0000 7fee40400640 1 --2- 172.21.15.46:0/1637612649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee20003980 0x7fee38158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.834+0000 7fee40400640 1 -- 172.21.15.46:0/1637612649 >> 172.21.15.46:0/1637612649 conn(0x7fee38066110 msgr2=0x7fee380aedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.834+0000 7fee40400640 1 -- 172.21.15.46:0/1637612649 shutdown_connections 2024-03-29T08:28:53.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.834+0000 7fee40400640 1 -- 172.21.15.46:0/1637612649 wait complete. 2024-03-29T08:28:53.858 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:27:17.118875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:27:17.573928+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:28:53.859 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:28:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.998+0000 7ffbd1400640 1 Processor -- start 2024-03-29T08:28:54.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.998+0000 7ffbd1400640 1 -- start start 2024-03-29T08:28:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.998+0000 7ffbd1400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffbcc15dfb0 0x7ffbcc0b1570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.998+0000 7ffbd1400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbcc0cb3f0 con 0x7ffbcc0ced10 2024-03-29T08:28:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.998+0000 7ffbd1400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbcc0cb620 con 0x7ffbcc098a40 2024-03-29T08:28:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:53.998+0000 7ffbd1400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc0b1e00 con 0x7ffbcc15dfb0 2024-03-29T08:28:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.002+0000 7ffbc1e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffbcc0ced10 0x7ffbcc0b3580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54032/0 (socket says 172.21.15.46:54032) 2024-03-29T08:28:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.002+0000 7ffbc1e00640 1 -- 172.21.15.46:0/2187240911 learned_addr learned my addr 172.21.15.46:0/2187240911 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:28:54.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.002+0000 7ffbca800640 1 --2- 172.21.15.46:0/2187240911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffbcc15dfb0 0x7ffbcc0b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.002+0000 7ffbc9e00640 1 -- 172.21.15.46:0/2187240911 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 664989812 0 0) 0x7ffbcc0cb3f0 con 0x7ffbcc0ced10 2024-03-29T08:28:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.002+0000 7ffbc9e00640 1 -- 172.21.15.46:0/2187240911 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbb0003530 con 0x7ffbcc0ced10 2024-03-29T08:28:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.002+0000 7ffbca800640 1 -- 172.21.15.46:0/2187240911 >> v1:172.21.15.73:6789/0 conn(0x7ffbcc098a40 legacy=0x7ffbcc0cc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.002+0000 7ffbca800640 1 -- 172.21.15.46:0/2187240911 >> v1:172.21.15.46:6789/0 conn(0x7ffbcc0ced10 legacy=0x7ffbcc0b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.002+0000 7ffbc9e00640 1 -- 172.21.15.46:0/2187240911 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 888342137 0 0) 0x7ffbcc0cb620 con 0x7ffbcc098a40 2024-03-29T08:28:54.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.002+0000 7ffbca800640 1 -- 172.21.15.46:0/2187240911 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbcc0cf4f0 con 0x7ffbcc15dfb0 2024-03-29T08:28:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.002+0000 7ffbca800640 1 --2- 172.21.15.46:0/2187240911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffbcc15dfb0 0x7ffbcc0b1570 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009dc0 tx=0x7ffbbc01c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d46440d2b23568e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.002+0000 7ffbc9e00640 1 -- 172.21.15.46:0/2187240911 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0a5930 con 0x7ffbcc15dfb0 2024-03-29T08:28:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.002+0000 7ffbc9e00640 1 -- 172.21.15.46:0/2187240911 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc0a5a90 con 0x7ffbcc15dfb0 2024-03-29T08:28:54.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.002+0000 7ffbc9e00640 1 -- 172.21.15.46:0/2187240911 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0a5e20 con 0x7ffbcc15dfb0 2024-03-29T08:28:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbd1400640 1 -- 172.21.15.46:0/2187240911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffbcc15dfb0 msgr2=0x7ffbcc0b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbd1400640 1 --2- 172.21.15.46:0/2187240911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffbcc15dfb0 0x7ffbcc0b1570 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009dc0 tx=0x7ffbbc01c960 comp rx=0 tx=0).stop 2024-03-29T08:28:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbd1400640 1 -- 172.21.15.46:0/2187240911 shutdown_connections 2024-03-29T08:28:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbd1400640 1 --2- 172.21.15.46:0/2187240911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffbcc15dfb0 0x7ffbcc0b1570 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbd1400640 1 -- 172.21.15.46:0/2187240911 >> 172.21.15.46:0/2187240911 conn(0x7ffbcc0b2220 msgr2=0x7ffbcc0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbd1400640 1 -- 172.21.15.46:0/2187240911 shutdown_connections 2024-03-29T08:28:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbd1400640 1 -- 172.21.15.46:0/2187240911 wait complete. 2024-03-29T08:28:54.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbd1400640 1 Processor -- start 2024-03-29T08:28:54.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbd1400640 1 -- start start 2024-03-29T08:28:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbd1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbcc098a40 0x7ffbcc0c3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbd1400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbcc0ced10 0x7ffbcc150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbd1400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffbcc15dfb0 0x7ffbcc151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbd1400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc0c7b50 con 0x7ffbcc098a40 2024-03-29T08:28:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbd1400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc0b59c0 con 0x7ffbcc0ced10 2024-03-29T08:28:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbd1400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc0c7800 con 0x7ffbcc15dfb0 2024-03-29T08:28:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbcc098a40 0x7ffbcc0c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbcc098a40 0x7ffbcc0c3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45406/0 (socket says 172.21.15.46:45406) 2024-03-29T08:28:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbca800640 1 -- 172.21.15.46:0/2027976855 learned_addr learned my addr 172.21.15.46:0/2027976855 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:54.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbc1e00640 1 --2- 172.21.15.46:0/2027976855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbcc0ced10 0x7ffbcc150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbcb200640 1 --2- 172.21.15.46:0/2027976855 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffbcc15dfb0 0x7ffbcc151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbca800640 1 -- 172.21.15.46:0/2027976855 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffbcc15dfb0 msgr2=0x7ffbcc151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbca800640 1 --2- 172.21.15.46:0/2027976855 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffbcc15dfb0 0x7ffbcc151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbca800640 1 -- 172.21.15.46:0/2027976855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbcc0ced10 msgr2=0x7ffbcc150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbca800640 1 --2- 172.21.15.46:0/2027976855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbcc0ced10 0x7ffbcc150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbca800640 1 -- 172.21.15.46:0/2027976855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc009a70 con 0x7ffbcc098a40 2024-03-29T08:28:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbcb200640 1 --2- 172.21.15.46:0/2027976855 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffbcc15dfb0 0x7ffbcc151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:28:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.006+0000 7ffbc1e00640 1 --2- 172.21.15.46:0/2027976855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbcc0ced10 0x7ffbcc150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:28:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.010+0000 7ffbca800640 1 --2- 172.21.15.46:0/2027976855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbcc098a40 0x7ffbcc0c3200 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc01a590 tx=0x7ffbbc019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.010+0000 7ffbc3400640 1 -- 172.21.15.46:0/2027976855 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0addd0 con 0x7ffbcc098a40 2024-03-29T08:28:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.010+0000 7ffbc3400640 1 -- 172.21.15.46:0/2027976855 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc009450 con 0x7ffbcc098a40 2024-03-29T08:28:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.010+0000 7ffbc3400640 1 -- 172.21.15.46:0/2027976855 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0b7650 con 0x7ffbcc098a40 2024-03-29T08:28:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.010+0000 7ffbd1400640 1 -- 172.21.15.46:0/2027976855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbcc0b56e0 con 0x7ffbcc098a40 2024-03-29T08:28:54.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.010+0000 7ffbd1400640 1 -- 172.21.15.46:0/2027976855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbcc0c3740 con 0x7ffbcc098a40 2024-03-29T08:28:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.010+0000 7ffbd1400640 1 -- 172.21.15.46:0/2027976855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbcc0b4ff0 con 0x7ffbcc098a40 2024-03-29T08:28:54.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.010+0000 7ffbc3400640 1 -- 172.21.15.46:0/2027976855 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ffbbc0b4050 con 0x7ffbcc098a40 2024-03-29T08:28:54.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.010+0000 7ffbc3400640 1 --2- 172.21.15.46:0/2027976855 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffba8066690 0x7ffba8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.010+0000 7ffbc3400640 1 -- 172.21.15.46:0/2027976855 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ffbbc11f520 con 0x7ffbcc098a40 2024-03-29T08:28:54.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.010+0000 7ffbc1e00640 1 --2- 172.21.15.46:0/2027976855 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffba8066690 0x7ffba8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.014+0000 7ffbc1e00640 1 --2- 172.21.15.46:0/2027976855 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffba8066690 0x7ffba8068b50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc145430 tx=0x7ffbb4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:54.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.014+0000 7ffbc3400640 1 -- 172.21.15.46:0/2027976855 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ffbcc0b4ff0 con 0x7ffbcc098a40 2024-03-29T08:28:54.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.174+0000 7ffbd1400640 1 -- 172.21.15.46:0/2027976855 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffbcc0b4ff0 con 0x7ffbcc098a40 2024-03-29T08:28:54.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.174+0000 7ffbc3400640 1 -- 172.21.15.46:0/2027976855 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffbcc0b4ff0 con 0x7ffbcc098a40 2024-03-29T08:28:54.179 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:28:54.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.178+0000 7ffbd1400640 1 -- 172.21.15.46:0/2027976855 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffba8066690 msgr2=0x7ffba8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.178+0000 7ffbd1400640 1 --2- 172.21.15.46:0/2027976855 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffba8066690 0x7ffba8068b50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc145430 tx=0x7ffbb4009040 comp rx=0 tx=0).stop 2024-03-29T08:28:54.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.178+0000 7ffbd1400640 1 -- 172.21.15.46:0/2027976855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbcc098a40 msgr2=0x7ffbcc0c3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.178+0000 7ffbd1400640 1 --2- 172.21.15.46:0/2027976855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbcc098a40 0x7ffbcc0c3200 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc01a590 tx=0x7ffbbc019700 comp rx=0 tx=0).stop 2024-03-29T08:28:54.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.178+0000 7ffbd1400640 1 -- 172.21.15.46:0/2027976855 shutdown_connections 2024-03-29T08:28:54.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.178+0000 7ffbd1400640 1 --2- 172.21.15.46:0/2027976855 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffba8066690 0x7ffba8068b50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.178+0000 7ffbd1400640 1 --2- 172.21.15.46:0/2027976855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbcc0ced10 0x7ffbcc150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.178+0000 7ffbd1400640 1 --2- 172.21.15.46:0/2027976855 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffbcc15dfb0 0x7ffbcc151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.178+0000 7ffbd1400640 1 --2- 172.21.15.46:0/2027976855 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffbcc098a40 0x7ffbcc0c3200 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.178+0000 7ffbd1400640 1 -- 172.21.15.46:0/2027976855 >> 172.21.15.46:0/2027976855 conn(0x7ffbcc0b2220 msgr2=0x7ffbcc0ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:54.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.178+0000 7ffbd1400640 1 -- 172.21.15.46:0/2027976855 shutdown_connections 2024-03-29T08:28:54.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.178+0000 7ffbd1400640 1 -- 172.21.15.46:0/2027976855 wait complete. 2024-03-29T08:28:54.201 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:28:54.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.342+0000 7fe5e3e00640 1 Processor -- start 2024-03-29T08:28:54.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.342+0000 7fe5e3e00640 1 -- start start 2024-03-29T08:28:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.342+0000 7fe5e3e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5dc143eb0 0x7fe5dc0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.342+0000 7fe5e3e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe5dc0b1d40 0x7fe5dc0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.342+0000 7fe5e3e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5dc0b2360 con 0x7fe5dc15efe0 2024-03-29T08:28:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.342+0000 7fe5e3e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5dc15e500 con 0x7fe5dc143eb0 2024-03-29T08:28:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.342+0000 7fe5e3e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5dc15e7e0 con 0x7fe5dc0b1d40 2024-03-29T08:28:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.342+0000 7fe5e1400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe5dc143eb0 0x7fe5dc0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.342+0000 7fe5e0a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe5dc15efe0 0x7fe5dc0b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54044/0 (socket says 172.21.15.46:54044) 2024-03-29T08:28:54.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.342+0000 7fe5e0a00640 1 -- 172.21.15.46:0/1583164962 learned_addr learned my addr 172.21.15.46:0/1583164962 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:28:54.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.342+0000 7fe5e1e00640 1 --2- 172.21.15.46:0/1583164962 >> v2:172.21.15.73:3300/0 conn(0x7fe5dc0b1d40 0x7fe5dc0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.342+0000 7fe5e1400640 1 -- 172.21.15.46:0/1583164962 >> v2:172.21.15.73:3300/0 conn(0x7fe5dc0b1d40 msgr2=0x7fe5dc0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:54.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.342+0000 7fe5e1400640 1 --2- 172.21.15.46:0/1583164962 >> v2:172.21.15.73:3300/0 conn(0x7fe5dc0b1d40 0x7fe5dc0cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.342+0000 7fe5e1400640 1 -- 172.21.15.46:0/1583164962 >> v1:172.21.15.46:6789/0 conn(0x7fe5dc15efe0 legacy=0x7fe5dc0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.342+0000 7fe5e1400640 1 -- 172.21.15.46:0/1583164962 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc15eac0 con 0x7fe5dc143eb0 2024-03-29T08:28:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.346+0000 7fe5e1400640 1 --2- 172.21.15.46:0/1583164962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5dc143eb0 0x7fe5dc0cb790 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc06ad40 tx=0x7fe5d001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc1c42b6d68cb73 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.346+0000 7fe5dbe00640 1 -- 172.21.15.46:0/1583164962 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00a8050 con 0x7fe5dc143eb0 2024-03-29T08:28:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.346+0000 7fe5dbe00640 1 -- 172.21.15.46:0/1583164962 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d00ac5f0 con 0x7fe5dc143eb0 2024-03-29T08:28:54.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.346+0000 7fe5dbe00640 1 -- 172.21.15.46:0/1583164962 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00ac980 con 0x7fe5dc143eb0 2024-03-29T08:28:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.346+0000 7fe5e3e00640 1 -- 172.21.15.46:0/1583164962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5dc143eb0 msgr2=0x7fe5dc0cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.346+0000 7fe5e3e00640 1 --2- 172.21.15.46:0/1583164962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5dc143eb0 0x7fe5dc0cb790 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc06ad40 tx=0x7fe5d001c480 comp rx=0 tx=0).stop 2024-03-29T08:28:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.346+0000 7fe5e3e00640 1 -- 172.21.15.46:0/1583164962 shutdown_connections 2024-03-29T08:28:54.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.346+0000 7fe5e3e00640 1 --2- 172.21.15.46:0/1583164962 >> v2:172.21.15.73:3300/0 conn(0x7fe5dc0b1d40 0x7fe5dc0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.346+0000 7fe5e3e00640 1 --2- 172.21.15.46:0/1583164962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5dc143eb0 0x7fe5dc0cb790 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.346+0000 7fe5e3e00640 1 -- 172.21.15.46:0/1583164962 >> 172.21.15.46:0/1583164962 conn(0x7fe5dc06a390 msgr2=0x7fe5dc0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.346+0000 7fe5e3e00640 1 -- 172.21.15.46:0/1583164962 shutdown_connections 2024-03-29T08:28:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.346+0000 7fe5e3e00640 1 -- 172.21.15.46:0/1583164962 wait complete. 2024-03-29T08:28:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e3e00640 1 Processor -- start 2024-03-29T08:28:54.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e3e00640 1 -- start start 2024-03-29T08:28:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e3e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe5dc0b1d40 0x7fe5dc15bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5dc143eb0 0x7fe5dc13d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e3e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5dc15efe0 0x7fe5dc0c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e3e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc164190 con 0x7fe5dc143eb0 2024-03-29T08:28:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e3e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc1f9120 con 0x7fe5dc15efe0 2024-03-29T08:28:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e3e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc163860 con 0x7fe5dc0b1d40 2024-03-29T08:28:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5dc143eb0 0x7fe5dc13d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5dc143eb0 0x7fe5dc13d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45426/0 (socket says 172.21.15.46:45426) 2024-03-29T08:28:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e0a00640 1 -- 172.21.15.46:0/1907478723 learned_addr learned my addr 172.21.15.46:0/1907478723 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e1e00640 1 --2- 172.21.15.46:0/1907478723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5dc15efe0 0x7fe5dc0c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e1400640 1 --2- 172.21.15.46:0/1907478723 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe5dc0b1d40 0x7fe5dc15bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e0a00640 1 -- 172.21.15.46:0/1907478723 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe5dc0b1d40 msgr2=0x7fe5dc15bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e0a00640 1 --2- 172.21.15.46:0/1907478723 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe5dc0b1d40 0x7fe5dc15bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e0a00640 1 -- 172.21.15.46:0/1907478723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5dc15efe0 msgr2=0x7fe5dc0c35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e0a00640 1 --2- 172.21.15.46:0/1907478723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5dc15efe0 0x7fe5dc0c35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e0a00640 1 -- 172.21.15.46:0/1907478723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d0009420 con 0x7fe5dc143eb0 2024-03-29T08:28:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e1e00640 1 --2- 172.21.15.46:0/1907478723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5dc15efe0 0x7fe5dc0c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:54.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e1400640 1 --2- 172.21.15.46:0/1907478723 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe5dc0b1d40 0x7fe5dc15bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:54.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e0a00640 1 --2- 172.21.15.46:0/1907478723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5dc143eb0 0x7fe5dc13d400 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800c500 tx=0x7fe5c800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:54.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5d9600640 1 -- 172.21.15.46:0/1907478723 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8005cf0 con 0x7fe5dc143eb0 2024-03-29T08:28:54.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5d9600640 1 -- 172.21.15.46:0/1907478723 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c8005e50 con 0x7fe5dc143eb0 2024-03-29T08:28:54.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5d9600640 1 -- 172.21.15.46:0/1907478723 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80072f0 con 0x7fe5dc143eb0 2024-03-29T08:28:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e3e00640 1 -- 172.21.15.46:0/1907478723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5dc163b40 con 0x7fe5dc143eb0 2024-03-29T08:28:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.350+0000 7fe5e3e00640 1 -- 172.21.15.46:0/1907478723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5dc15c450 con 0x7fe5dc143eb0 2024-03-29T08:28:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.354+0000 7fe5e3e00640 1 -- 172.21.15.46:0/1907478723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5dc071a70 con 0x7fe5dc143eb0 2024-03-29T08:28:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.354+0000 7fe5d9600640 1 -- 172.21.15.46:0/1907478723 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe5c8007450 con 0x7fe5dc143eb0 2024-03-29T08:28:54.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.354+0000 7fe5d9600640 1 --2- 172.21.15.46:0/1907478723 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe5bc066690 0x7fe5bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.354+0000 7fe5d9600640 1 -- 172.21.15.46:0/1907478723 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fe5c80837d0 con 0x7fe5dc143eb0 2024-03-29T08:28:54.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.354+0000 7fe5e1400640 1 --2- 172.21.15.46:0/1907478723 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe5bc066690 0x7fe5bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.354+0000 7fe5e1400640 1 --2- 172.21.15.46:0/1907478723 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe5bc066690 0x7fe5bc068b50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00ae000 tx=0x7fe5d00a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:54.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.358+0000 7fe5d9600640 1 -- 172.21.15.46:0/1907478723 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe5c8050af0 con 0x7fe5dc143eb0 2024-03-29T08:28:54.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.518+0000 7fe5e3e00640 1 -- 172.21.15.46:0/1907478723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe5dc0cadd0 con 0x7fe5dc143eb0 2024-03-29T08:28:54.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.518+0000 7fe5d9600640 1 -- 172.21.15.46:0/1907478723 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe5c8054110 con 0x7fe5dc143eb0 2024-03-29T08:28:54.521 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:28:54.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.518+0000 7fe5e3e00640 1 -- 172.21.15.46:0/1907478723 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe5bc066690 msgr2=0x7fe5bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.518+0000 7fe5e3e00640 1 --2- 172.21.15.46:0/1907478723 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe5bc066690 0x7fe5bc068b50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00ae000 tx=0x7fe5d00a7000 comp rx=0 tx=0).stop 2024-03-29T08:28:54.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.518+0000 7fe5e3e00640 1 -- 172.21.15.46:0/1907478723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5dc143eb0 msgr2=0x7fe5dc13d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.518+0000 7fe5e3e00640 1 --2- 172.21.15.46:0/1907478723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5dc143eb0 0x7fe5dc13d400 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fe5c800c500 tx=0x7fe5c800c9c0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.518+0000 7fe5e3e00640 1 -- 172.21.15.46:0/1907478723 shutdown_connections 2024-03-29T08:28:54.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.518+0000 7fe5e3e00640 1 --2- 172.21.15.46:0/1907478723 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe5bc066690 0x7fe5bc068b50 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.518+0000 7fe5e3e00640 1 --2- 172.21.15.46:0/1907478723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5dc15efe0 0x7fe5dc0c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.518+0000 7fe5e3e00640 1 --2- 172.21.15.46:0/1907478723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5dc143eb0 0x7fe5dc13d400 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.518+0000 7fe5e3e00640 1 --2- 172.21.15.46:0/1907478723 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe5dc0b1d40 0x7fe5dc15bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.518+0000 7fe5e3e00640 1 -- 172.21.15.46:0/1907478723 >> 172.21.15.46:0/1907478723 conn(0x7fe5dc06a390 msgr2=0x7fe5dc0cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:54.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.518+0000 7fe5e3e00640 1 -- 172.21.15.46:0/1907478723 shutdown_connections 2024-03-29T08:28:54.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.518+0000 7fe5e3e00640 1 -- 172.21.15.46:0/1907478723 wait complete. 2024-03-29T08:28:54.542 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:28:54.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.682+0000 7ff897c00640 1 Processor -- start 2024-03-29T08:28:54.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.682+0000 7ff897c00640 1 -- start start 2024-03-29T08:28:54.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.686+0000 7ff897c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff890142a50 0x7ff8900cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.686+0000 7ff897c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff89015df90 0x7ff8900cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.686+0000 7ff897c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff89015d260 con 0x7ff8900ae280 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.686+0000 7ff897c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8900aefd0 con 0x7ff890142a50 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.686+0000 7ff897c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff89015d4b0 con 0x7ff89015df90 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.686+0000 7ff895200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff890142a50 0x7ff8900cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.686+0000 7ff895c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff8900ae280 0x7ff8900ae710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54058/0 (socket says 172.21.15.46:54058) 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.686+0000 7ff895200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff890142a50 0x7ff8900cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45430/0 (socket says 172.21.15.46:45430) 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.686+0000 7ff895c00640 1 -- 172.21.15.46:0/3480614120 learned_addr learned my addr 172.21.15.46:0/3480614120 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.686+0000 7ff88fe00640 1 --2- 172.21.15.46:0/3480614120 >> v2:172.21.15.73:3300/0 conn(0x7ff89015df90 0x7ff8900cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.686+0000 7ff895200640 1 -- 172.21.15.46:0/3480614120 >> v2:172.21.15.73:3300/0 conn(0x7ff89015df90 msgr2=0x7ff8900cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.686+0000 7ff895200640 1 --2- 172.21.15.46:0/3480614120 >> v2:172.21.15.73:3300/0 conn(0x7ff89015df90 0x7ff8900cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.686+0000 7ff895200640 1 -- 172.21.15.46:0/3480614120 >> v1:172.21.15.46:6789/0 conn(0x7ff8900ae280 legacy=0x7ff8900ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.686+0000 7ff895200640 1 -- 172.21.15.46:0/3480614120 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89015d790 con 0x7ff890142a50 2024-03-29T08:28:54.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.686+0000 7ff88f400640 1 -- 172.21.15.46:0/3480614120 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1924005217 0 0) 0x7ff89015d260 con 0x7ff8900ae280 2024-03-29T08:28:54.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.686+0000 7ff88fe00640 1 --2- 172.21.15.46:0/3480614120 >> v2:172.21.15.73:3300/0 conn(0x7ff89015df90 0x7ff8900cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:28:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.690+0000 7ff895200640 1 --2- 172.21.15.46:0/3480614120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff890142a50 0x7ff8900cb790 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7ff89006e480 tx=0x7ff88401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7a26984710fa89 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:54.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.690+0000 7ff88f400640 1 -- 172.21.15.46:0/3480614120 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8840ab600 con 0x7ff890142a50 2024-03-29T08:28:54.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.690+0000 7ff88f400640 1 -- 172.21.15.46:0/3480614120 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8840a7050 con 0x7ff890142a50 2024-03-29T08:28:54.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.690+0000 7ff88f400640 1 -- 172.21.15.46:0/3480614120 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8840ab980 con 0x7ff890142a50 2024-03-29T08:28:54.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.690+0000 7ff897c00640 1 -- 172.21.15.46:0/3480614120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff890142a50 msgr2=0x7ff8900cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.690+0000 7ff897c00640 1 --2- 172.21.15.46:0/3480614120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff890142a50 0x7ff8900cb790 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7ff89006e480 tx=0x7ff88401c3f0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.690+0000 7ff897c00640 1 -- 172.21.15.46:0/3480614120 shutdown_connections 2024-03-29T08:28:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.690+0000 7ff897c00640 1 --2- 172.21.15.46:0/3480614120 >> v2:172.21.15.73:3300/0 conn(0x7ff89015df90 0x7ff8900cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.690+0000 7ff897c00640 1 --2- 172.21.15.46:0/3480614120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff890142a50 0x7ff8900cb790 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.690+0000 7ff897c00640 1 -- 172.21.15.46:0/3480614120 >> 172.21.15.46:0/3480614120 conn(0x7ff89006a390 msgr2=0x7ff8900cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.690+0000 7ff897c00640 1 -- 172.21.15.46:0/3480614120 shutdown_connections 2024-03-29T08:28:54.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.690+0000 7ff897c00640 1 -- 172.21.15.46:0/3480614120 wait complete. 2024-03-29T08:28:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.690+0000 7ff897c00640 1 Processor -- start 2024-03-29T08:28:54.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.690+0000 7ff897c00640 1 -- start start 2024-03-29T08:28:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff897c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8900ae280 0x7ff890137660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff897c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff890142a50 0x7ff890158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff897c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff89015df90 0x7ff8900cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff897c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff890138f50 con 0x7ff8900ae280 2024-03-29T08:28:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff897c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff890137eb0 con 0x7ff890142a50 2024-03-29T08:28:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff897c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff890138190 con 0x7ff89015df90 2024-03-29T08:28:54.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff895200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8900ae280 0x7ff890137660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff895200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8900ae280 0x7ff890137660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45444/0 (socket says 172.21.15.46:45444) 2024-03-29T08:28:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff895200640 1 -- 172.21.15.46:0/2818284697 learned_addr learned my addr 172.21.15.46:0/2818284697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff895200640 1 -- 172.21.15.46:0/2818284697 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff89015df90 msgr2=0x7ff8900cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff895200640 1 --2- 172.21.15.46:0/2818284697 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff89015df90 0x7ff8900cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff895200640 1 -- 172.21.15.46:0/2818284697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff890142a50 msgr2=0x7ff890158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff895200640 1 --2- 172.21.15.46:0/2818284697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff890142a50 0x7ff890158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff895200640 1 -- 172.21.15.46:0/2818284697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff884009420 con 0x7ff8900ae280 2024-03-29T08:28:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff895200640 1 --2- 172.21.15.46:0/2818284697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8900ae280 0x7ff890137660 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7ff8840ae040 tx=0x7ff884019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff88cc00640 1 -- 172.21.15.46:0/2818284697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8840b4820 con 0x7ff8900ae280 2024-03-29T08:28:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff88cc00640 1 -- 172.21.15.46:0/2818284697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8840b4980 con 0x7ff8900ae280 2024-03-29T08:28:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff88cc00640 1 -- 172.21.15.46:0/2818284697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8840b4d10 con 0x7ff8900ae280 2024-03-29T08:28:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff897c00640 1 -- 172.21.15.46:0/2818284697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff890138470 con 0x7ff8900ae280 2024-03-29T08:28:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff897c00640 1 -- 172.21.15.46:0/2818284697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff890138a30 con 0x7ff8900ae280 2024-03-29T08:28:54.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff897c00640 1 -- 172.21.15.46:0/2818284697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff890071a70 con 0x7ff8900ae280 2024-03-29T08:28:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.694+0000 7ff88cc00640 1 -- 172.21.15.46:0/2818284697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff8840b2040 con 0x7ff8900ae280 2024-03-29T08:28:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.698+0000 7ff88cc00640 1 --2- 172.21.15.46:0/2818284697 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff870066690 0x7ff870068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:54.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.698+0000 7ff88cc00640 1 -- 172.21.15.46:0/2818284697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff88411e6e0 con 0x7ff8900ae280 2024-03-29T08:28:54.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.698+0000 7ff88fe00640 1 --2- 172.21.15.46:0/2818284697 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff870066690 0x7ff870068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:54.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.698+0000 7ff88fe00640 1 --2- 172.21.15.46:0/2818284697 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff870066690 0x7ff870068b50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff87c006e90 tx=0x7ff87c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:54.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.702+0000 7ff88cc00640 1 -- 172.21.15.46:0/2818284697 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff8840eb8c0 con 0x7ff8900ae280 2024-03-29T08:28:54.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.858+0000 7ff897c00640 1 -- 172.21.15.46:0/2818284697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff89004e120 con 0x7ff8900ae280 2024-03-29T08:28:54.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.858+0000 7ff88cc00640 1 -- 172.21.15.46:0/2818284697 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff884006e20 con 0x7ff8900ae280 2024-03-29T08:28:54.863 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:28:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.862+0000 7ff897c00640 1 -- 172.21.15.46:0/2818284697 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff870066690 msgr2=0x7ff870068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.862+0000 7ff897c00640 1 --2- 172.21.15.46:0/2818284697 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff870066690 0x7ff870068b50 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff87c006e90 tx=0x7ff87c008040 comp rx=0 tx=0).stop 2024-03-29T08:28:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.862+0000 7ff897c00640 1 -- 172.21.15.46:0/2818284697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8900ae280 msgr2=0x7ff890137660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.862+0000 7ff897c00640 1 --2- 172.21.15.46:0/2818284697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8900ae280 0x7ff890137660 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7ff8840ae040 tx=0x7ff884019190 comp rx=0 tx=0).stop 2024-03-29T08:28:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.862+0000 7ff897c00640 1 -- 172.21.15.46:0/2818284697 shutdown_connections 2024-03-29T08:28:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.862+0000 7ff897c00640 1 --2- 172.21.15.46:0/2818284697 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff870066690 0x7ff870068b50 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.862+0000 7ff897c00640 1 --2- 172.21.15.46:0/2818284697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff890142a50 0x7ff890158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.862+0000 7ff897c00640 1 --2- 172.21.15.46:0/2818284697 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff89015df90 0x7ff8900cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.862+0000 7ff897c00640 1 --2- 172.21.15.46:0/2818284697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8900ae280 0x7ff890137660 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.862+0000 7ff897c00640 1 -- 172.21.15.46:0/2818284697 >> 172.21.15.46:0/2818284697 conn(0x7ff89006a390 msgr2=0x7ff8900af530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:54.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.862+0000 7ff897c00640 1 -- 172.21.15.46:0/2818284697 shutdown_connections 2024-03-29T08:28:54.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:54.862+0000 7ff897c00640 1 -- 172.21.15.46:0/2818284697 wait complete. 2024-03-29T08:28:54.884 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:28:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4614000640 1 Processor -- start 2024-03-29T08:28:55.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4614000640 1 -- start start 2024-03-29T08:28:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4614000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f460c0d0660 0x7f460c0d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4614000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f460c06b8b0 0x7f460c0cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4614000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f460c15dc00 0x7f460c0cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4614000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460c06c100 con 0x7f460c15dc00 2024-03-29T08:28:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4614000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460c15cfb0 con 0x7f460c06b8b0 2024-03-29T08:28:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4614000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460c0d0f80 con 0x7f460c0d0660 2024-03-29T08:28:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4612000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f460c15dc00 0x7f460c0cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4612000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f460c15dc00 0x7f460c0cf7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45448/0 (socket says 172.21.15.46:45448) 2024-03-29T08:28:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4612000640 1 -- 172.21.15.46:0/918252652 learned_addr learned my addr 172.21.15.46:0/918252652 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4611600640 1 --2- 172.21.15.46:0/918252652 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f460c0d0660 0x7f460c0d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4610c00640 1 --2- 172.21.15.46:0/918252652 >> v2:172.21.15.73:3300/0 conn(0x7f460c06b8b0 0x7f460c0cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4612000640 1 -- 172.21.15.46:0/918252652 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f460c0d0660 msgr2=0x7f460c0d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:55.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4612000640 1 --2- 172.21.15.46:0/918252652 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f460c0d0660 0x7f460c0d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4612000640 1 -- 172.21.15.46:0/918252652 >> v2:172.21.15.73:3300/0 conn(0x7f460c06b8b0 msgr2=0x7f460c0cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:55.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4612000640 1 --2- 172.21.15.46:0/918252652 >> v2:172.21.15.73:3300/0 conn(0x7f460c06b8b0 0x7f460c0cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4612000640 1 -- 172.21.15.46:0/918252652 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c06be20 con 0x7f460c15dc00 2024-03-29T08:28:55.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.022+0000 7f4610c00640 1 --2- 172.21.15.46:0/918252652 >> v2:172.21.15.73:3300/0 conn(0x7f460c06b8b0 0x7f460c0cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:28:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.026+0000 7f4612000640 1 --2- 172.21.15.46:0/918252652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f460c15dc00 0x7f460c0cf7d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f460c0b32c0 tx=0x7f460401c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ae2fdb26d9adda server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.026+0000 7f460be00640 1 -- 172.21.15.46:0/918252652 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46040a5780 con 0x7f460c15dc00 2024-03-29T08:28:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.026+0000 7f460be00640 1 -- 172.21.15.46:0/918252652 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46040a58e0 con 0x7f460c15dc00 2024-03-29T08:28:55.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.026+0000 7f460be00640 1 -- 172.21.15.46:0/918252652 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46040a5c70 con 0x7f460c15dc00 2024-03-29T08:28:55.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.026+0000 7f4614000640 1 -- 172.21.15.46:0/918252652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f460c15dc00 msgr2=0x7f460c0cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.026+0000 7f4614000640 1 --2- 172.21.15.46:0/918252652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f460c15dc00 0x7f460c0cf7d0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f460c0b32c0 tx=0x7f460401c990 comp rx=0 tx=0).stop 2024-03-29T08:28:55.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.026+0000 7f4614000640 1 -- 172.21.15.46:0/918252652 shutdown_connections 2024-03-29T08:28:55.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.026+0000 7f4614000640 1 --2- 172.21.15.46:0/918252652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f460c15dc00 0x7f460c0cf7d0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.026+0000 7f4614000640 1 --2- 172.21.15.46:0/918252652 >> v2:172.21.15.73:3300/0 conn(0x7f460c06b8b0 0x7f460c0cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.026+0000 7f4614000640 1 --2- 172.21.15.46:0/918252652 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f460c0d0660 0x7f460c0d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.026+0000 7f4614000640 1 -- 172.21.15.46:0/918252652 >> 172.21.15.46:0/918252652 conn(0x7f460c089fd0 msgr2=0x7f460c08a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:55.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.026+0000 7f4614000640 1 -- 172.21.15.46:0/918252652 shutdown_connections 2024-03-29T08:28:55.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.026+0000 7f4614000640 1 -- 172.21.15.46:0/918252652 wait complete. 2024-03-29T08:28:55.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4614000640 1 Processor -- start 2024-03-29T08:28:55.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4614000640 1 -- start start 2024-03-29T08:28:55.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4614000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f460c06b8b0 0x7f460c14bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4614000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460c0d0660 0x7f460c159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4614000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f460c15dc00 0x7f460c15a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4614000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c153fc0 con 0x7f460c15dc00 2024-03-29T08:28:55.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4614000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460c1542a0 con 0x7f460c0d0660 2024-03-29T08:28:55.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4614000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460c154580 con 0x7f460c06b8b0 2024-03-29T08:28:55.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4612000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f460c15dc00 0x7f460c15a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4612000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f460c15dc00 0x7f460c15a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45452/0 (socket says 172.21.15.46:45452) 2024-03-29T08:28:55.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4612000640 1 -- 172.21.15.46:0/3659500591 learned_addr learned my addr 172.21.15.46:0/3659500591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:55.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4611600640 1 --2- 172.21.15.46:0/3659500591 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f460c06b8b0 0x7f460c14bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4610c00640 1 --2- 172.21.15.46:0/3659500591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460c0d0660 0x7f460c159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4612000640 1 -- 172.21.15.46:0/3659500591 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f460c06b8b0 msgr2=0x7f460c14bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4612000640 1 --2- 172.21.15.46:0/3659500591 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f460c06b8b0 0x7f460c14bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4612000640 1 -- 172.21.15.46:0/3659500591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460c0d0660 msgr2=0x7f460c159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4612000640 1 --2- 172.21.15.46:0/3659500591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460c0d0660 0x7f460c159ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4612000640 1 -- 172.21.15.46:0/3659500591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4604009b30 con 0x7f460c15dc00 2024-03-29T08:28:55.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4612000640 1 --2- 172.21.15.46:0/3659500591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f460c15dc00 0x7f460c15a010 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f46040af040 tx=0x7f4604019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:55.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4609600640 1 -- 172.21.15.46:0/3659500591 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46040be070 con 0x7f460c15dc00 2024-03-29T08:28:55.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4609600640 1 -- 172.21.15.46:0/3659500591 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46040b8440 con 0x7f460c15dc00 2024-03-29T08:28:55.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4609600640 1 -- 172.21.15.46:0/3659500591 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46040b87f0 con 0x7f460c15dc00 2024-03-29T08:28:55.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4614000640 1 -- 172.21.15.46:0/3659500591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460c154860 con 0x7f460c15dc00 2024-03-29T08:28:55.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.030+0000 7f4614000640 1 -- 172.21.15.46:0/3659500591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c14c270 con 0x7f460c15dc00 2024-03-29T08:28:55.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.034+0000 7f4614000640 1 -- 172.21.15.46:0/3659500591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460c066c90 con 0x7f460c15dc00 2024-03-29T08:28:55.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.034+0000 7f4609600640 1 -- 172.21.15.46:0/3659500591 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f46040b8950 con 0x7f460c15dc00 2024-03-29T08:28:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.034+0000 7f4609600640 1 --2- 172.21.15.46:0/3659500591 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f45ec066690 0x7f45ec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.034+0000 7f4609600640 1 -- 172.21.15.46:0/3659500591 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f460411e9c0 con 0x7f460c15dc00 2024-03-29T08:28:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.034+0000 7f4611600640 1 --2- 172.21.15.46:0/3659500591 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f45ec066690 0x7f45ec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.034+0000 7f4611600640 1 --2- 172.21.15.46:0/3659500591 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f45ec066690 0x7f45ec068b50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f460c08a830 tx=0x7f4600009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:55.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.038+0000 7f4609600640 1 -- 172.21.15.46:0/3659500591 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f46040ebe10 con 0x7f460c15dc00 2024-03-29T08:28:55.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.234+0000 7f4614000640 1 -- 172.21.15.46:0/3659500591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f460c0b6520 con 0x7f460c15dc00 2024-03-29T08:28:55.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.234+0000 7f4609600640 1 -- 172.21.15.46:0/3659500591 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4727 (secure 0 0 0) 0x7f46040ef430 con 0x7f460c15dc00 2024-03-29T08:28:55.239 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:28:55.239 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4230030957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4230030957},{"type":"v1","addr":"172.21.15.73:6836","nonce":4230030957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/297138217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":297138217},{"type":"v1","addr":"172.21.15.73:6837","nonce":297138217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3856305332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3856305332},{"type":"v1","addr":"172.21.15.46:6833","nonce":3856305332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":8895,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3142618727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3142618727},{"type":"v1","addr":"172.21.15.46:6835","nonce":3142618727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":84,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:27:18.583223+0000","modified":"2024-03-29T08:28:52.591232+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-29T08:28:55.240 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 85 2024-03-29T08:28:55.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.238+0000 7f4614000640 1 -- 172.21.15.46:0/3659500591 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f45ec066690 msgr2=0x7f45ec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.238+0000 7f4614000640 1 --2- 172.21.15.46:0/3659500591 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f45ec066690 0x7f45ec068b50 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f460c08a830 tx=0x7f4600009190 comp rx=0 tx=0).stop 2024-03-29T08:28:55.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.238+0000 7f4614000640 1 -- 172.21.15.46:0/3659500591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f460c15dc00 msgr2=0x7f460c15a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.238+0000 7f4614000640 1 --2- 172.21.15.46:0/3659500591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f460c15dc00 0x7f460c15a010 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f46040af040 tx=0x7f4604019730 comp rx=0 tx=0).stop 2024-03-29T08:28:55.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.238+0000 7f4614000640 1 -- 172.21.15.46:0/3659500591 shutdown_connections 2024-03-29T08:28:55.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.238+0000 7f4614000640 1 --2- 172.21.15.46:0/3659500591 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f45ec066690 0x7f45ec068b50 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.238+0000 7f4614000640 1 --2- 172.21.15.46:0/3659500591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460c0d0660 0x7f460c159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.238+0000 7f4614000640 1 --2- 172.21.15.46:0/3659500591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f460c15dc00 0x7f460c15a010 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.238+0000 7f4614000640 1 --2- 172.21.15.46:0/3659500591 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f460c06b8b0 0x7f460c14bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.238+0000 7f4614000640 1 -- 172.21.15.46:0/3659500591 >> 172.21.15.46:0/3659500591 conn(0x7f460c089fd0 msgr2=0x7f460c0cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:55.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.238+0000 7f4614000640 1 -- 172.21.15.46:0/3659500591 shutdown_connections 2024-03-29T08:28:55.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.238+0000 7f4614000640 1 -- 172.21.15.46:0/3659500591 wait complete. 2024-03-29T08:28:55.260 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:28:55.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.398+0000 7fd01da00640 1 Processor -- start 2024-03-29T08:28:55.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.398+0000 7fd01da00640 1 -- start start 2024-03-29T08:28:55.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.398+0000 7fd01da00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd0181593d0 0x7fd01806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.398+0000 7fd01da00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd01815cd90 0x7fd0181597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.398+0000 7fd01da00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd01806d130 con 0x7fd01815d230 2024-03-29T08:28:55.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.398+0000 7fd01da00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd01815b8f0 con 0x7fd0181593d0 2024-03-29T08:28:55.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.398+0000 7fd01da00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01815ad50 con 0x7fd01815cd90 2024-03-29T08:28:55.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.398+0000 7fd016a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd0181593d0 0x7fd01806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.398+0000 7fd016a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd0181593d0 0x7fd01806cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45454/0 (socket says 172.21.15.46:45454) 2024-03-29T08:28:55.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.398+0000 7fd016a00640 1 -- 172.21.15.46:0/70881373 learned_addr learned my addr 172.21.15.46:0/70881373 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:55.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.398+0000 7fd00fe00640 1 --1- 172.21.15.46:0/70881373 >> v1:172.21.15.46:6789/0 conn(0x7fd01815d230 0x7fd01815d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54068/0 (socket says 172.21.15.46:54068) 2024-03-29T08:28:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.398+0000 7fd017400640 1 --2- 172.21.15.46:0/70881373 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd01815cd90 0x7fd0181597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.398+0000 7fd016a00640 1 -- 172.21.15.46:0/70881373 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd01815cd90 msgr2=0x7fd0181597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.398+0000 7fd016a00640 1 --2- 172.21.15.46:0/70881373 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd01815cd90 0x7fd0181597b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.398+0000 7fd016a00640 1 -- 172.21.15.46:0/70881373 >> v1:172.21.15.46:6789/0 conn(0x7fd01815d230 legacy=0x7fd01815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.398+0000 7fd016a00640 1 -- 172.21.15.46:0/70881373 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01815afc0 con 0x7fd0181593d0 2024-03-29T08:28:55.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.402+0000 7fd016a00640 1 --2- 172.21.15.46:0/70881373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0181593d0 0x7fd01806cbf0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fd008009870 tx=0x7fd00801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c9c54f6e4c7578 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.402+0000 7fd016000640 1 -- 172.21.15.46:0/70881373 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0080a9620 con 0x7fd0181593d0 2024-03-29T08:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.402+0000 7fd016000640 1 -- 172.21.15.46:0/70881373 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0080a9780 con 0x7fd0181593d0 2024-03-29T08:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.402+0000 7fd016000640 1 -- 172.21.15.46:0/70881373 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0080a9b10 con 0x7fd0181593d0 2024-03-29T08:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.402+0000 7fd01da00640 1 -- 172.21.15.46:0/70881373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0181593d0 msgr2=0x7fd01806cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.402+0000 7fd01da00640 1 --2- 172.21.15.46:0/70881373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0181593d0 0x7fd01806cbf0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fd008009870 tx=0x7fd00801c560 comp rx=0 tx=0).stop 2024-03-29T08:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.402+0000 7fd01da00640 1 -- 172.21.15.46:0/70881373 shutdown_connections 2024-03-29T08:28:55.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.402+0000 7fd01da00640 1 --2- 172.21.15.46:0/70881373 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd01815cd90 0x7fd0181597b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.402+0000 7fd01da00640 1 --2- 172.21.15.46:0/70881373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0181593d0 0x7fd01806cbf0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.402+0000 7fd01da00640 1 -- 172.21.15.46:0/70881373 >> 172.21.15.46:0/70881373 conn(0x7fd018066110 msgr2=0x7fd018069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:55.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.402+0000 7fd01da00640 1 -- 172.21.15.46:0/70881373 shutdown_connections 2024-03-29T08:28:55.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.402+0000 7fd01da00640 1 -- 172.21.15.46:0/70881373 wait complete. 2024-03-29T08:28:55.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd01da00640 1 Processor -- start 2024-03-29T08:28:55.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd01da00640 1 -- start start 2024-03-29T08:28:55.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd01da00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0181593d0 0x7fd0180a8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd01da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd01815cd90 0x7fd0180cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd01da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd01815d230 0x7fd0180cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd01da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0180a7b90 con 0x7fd01815cd90 2024-03-29T08:28:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd01da00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0180a8560 con 0x7fd01815d230 2024-03-29T08:28:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd01da00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0180cd000 con 0x7fd0181593d0 2024-03-29T08:28:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd00fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd01815cd90 0x7fd0180cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd00fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd01815cd90 0x7fd0180cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45470/0 (socket says 172.21.15.46:45470) 2024-03-29T08:28:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd00fe00640 1 -- 172.21.15.46:0/225092405 learned_addr learned my addr 172.21.15.46:0/225092405 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd017400640 1 --2- 172.21.15.46:0/225092405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd01815d230 0x7fd0180cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd016a00640 1 --2- 172.21.15.46:0/225092405 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0181593d0 0x7fd0180a8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd00fe00640 1 -- 172.21.15.46:0/225092405 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0181593d0 msgr2=0x7fd0180a8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd00fe00640 1 --2- 172.21.15.46:0/225092405 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0181593d0 0x7fd0180a8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd00fe00640 1 -- 172.21.15.46:0/225092405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd01815d230 msgr2=0x7fd0180cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd00fe00640 1 --2- 172.21.15.46:0/225092405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd01815d230 0x7fd0180cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd00fe00640 1 -- 172.21.15.46:0/225092405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0080094d0 con 0x7fd01815cd90 2024-03-29T08:28:55.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd017400640 1 --2- 172.21.15.46:0/225092405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd01815d230 0x7fd0180cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:28:55.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd00fe00640 1 --2- 172.21.15.46:0/225092405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd01815cd90 0x7fd0180cc3b0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fd00000c420 tx=0x7fd00000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:55.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd00ea00640 1 -- 172.21.15.46:0/225092405 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0000074e0 con 0x7fd01815cd90 2024-03-29T08:28:55.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd00ea00640 1 -- 172.21.15.46:0/225092405 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd000007640 con 0x7fd01815cd90 2024-03-29T08:28:55.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd01da00640 1 -- 172.21.15.46:0/225092405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0180cd2e0 con 0x7fd01815cd90 2024-03-29T08:28:55.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd00ea00640 1 -- 172.21.15.46:0/225092405 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd000004a30 con 0x7fd01815cd90 2024-03-29T08:28:55.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd01da00640 1 -- 172.21.15.46:0/225092405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0180a7820 con 0x7fd01815cd90 2024-03-29T08:28:55.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.406+0000 7fd01da00640 1 -- 172.21.15.46:0/225092405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0180a79e0 con 0x7fd01815cd90 2024-03-29T08:28:55.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.410+0000 7fd00ea00640 1 -- 172.21.15.46:0/225092405 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd000004b90 con 0x7fd01815cd90 2024-03-29T08:28:55.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.410+0000 7fd00ea00640 1 --2- 172.21.15.46:0/225092405 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcff4066690 0x7fcff4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.410+0000 7fd00ea00640 1 -- 172.21.15.46:0/225092405 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fd000083c30 con 0x7fd01815cd90 2024-03-29T08:28:55.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.410+0000 7fd016a00640 1 --2- 172.21.15.46:0/225092405 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcff4066690 0x7fcff4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.410+0000 7fd016a00640 1 --2- 172.21.15.46:0/225092405 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcff4066690 0x7fcff4068b50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd0080ae000 tx=0x7fd0080b3040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:55.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.414+0000 7fd00ea00640 1 -- 172.21.15.46:0/225092405 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd000050eb0 con 0x7fd01815cd90 2024-03-29T08:28:55.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.578+0000 7fd01da00640 1 -- 172.21.15.46:0/225092405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd018066c90 con 0x7fd01815cd90 2024-03-29T08:28:55.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.578+0000 7fd00ea00640 1 -- 172.21.15.46:0/225092405 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16357 (secure 0 0 0) 0x7fd0000544d0 con 0x7fd01815cd90 2024-03-29T08:28:55.581 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:28:55.581 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":110,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:28:52.590731+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:27:17.118875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-29T08:27:17.573928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.f","mappings":[{"from":5,"to":0}]},{"pgid":"14.14","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1455003836":"2024-03-30T08:28:52.590708+0000","172.21.15.46:6834/1455003836":"2024-03-30T08:28:52.590708+0000","172.21.15.46:6835/824184385":"2024-03-30T08:28:15.875168+0000","172.21.15.46:6834/824184385":"2024-03-30T08:28:15.875168+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:28:55.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.578+0000 7fd01da00640 1 -- 172.21.15.46:0/225092405 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcff4066690 msgr2=0x7fcff4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.578+0000 7fd01da00640 1 --2- 172.21.15.46:0/225092405 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcff4066690 0x7fcff4068b50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd0080ae000 tx=0x7fd0080b3040 comp rx=0 tx=0).stop 2024-03-29T08:28:55.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.578+0000 7fd01da00640 1 -- 172.21.15.46:0/225092405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd01815cd90 msgr2=0x7fd0180cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.578+0000 7fd01da00640 1 --2- 172.21.15.46:0/225092405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd01815cd90 0x7fd0180cc3b0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fd00000c420 tx=0x7fd00000c8e0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.578+0000 7fd01da00640 1 -- 172.21.15.46:0/225092405 shutdown_connections 2024-03-29T08:28:55.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.578+0000 7fd01da00640 1 --2- 172.21.15.46:0/225092405 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcff4066690 0x7fcff4068b50 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.578+0000 7fd01da00640 1 --2- 172.21.15.46:0/225092405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd01815d230 0x7fd0180cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.578+0000 7fd01da00640 1 --2- 172.21.15.46:0/225092405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd01815cd90 0x7fd0180cc3b0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.578+0000 7fd01da00640 1 --2- 172.21.15.46:0/225092405 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd0181593d0 0x7fd0180a8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.578+0000 7fd01da00640 1 -- 172.21.15.46:0/225092405 >> 172.21.15.46:0/225092405 conn(0x7fd018066110 msgr2=0x7fd0180b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:55.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.578+0000 7fd01da00640 1 -- 172.21.15.46:0/225092405 shutdown_connections 2024-03-29T08:28:55.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.578+0000 7fd01da00640 1 -- 172.21.15.46:0/225092405 wait complete. 2024-03-29T08:28:55.603 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T08:28:55.603 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:28:55.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.738+0000 7f15ee200640 1 Processor -- start 2024-03-29T08:28:55.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.738+0000 7f15ee200640 1 -- start start 2024-03-29T08:28:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15ee200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15e815cd90 0x7f15e806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15ee200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15e806d130 con 0x7f15e815d940 2024-03-29T08:28:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15ee200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15e815b8f0 con 0x7f15e81593d0 2024-03-29T08:28:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15ee200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e815ad50 con 0x7f15e815cd90 2024-03-29T08:28:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f15e815d940 0x7f15e81597b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54076/0 (socket says 172.21.15.46:54076) 2024-03-29T08:28:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e7e00640 1 -- 172.21.15.46:0/3721838381 learned_addr learned my addr 172.21.15.46:0/3721838381 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:28:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15dfe00640 1 --2- 172.21.15.46:0/3721838381 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15e815cd90 0x7f15e806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e6a00640 1 -- 172.21.15.46:0/3721838381 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389688488 0 0) 0x7f15e806d130 con 0x7f15e815d940 2024-03-29T08:28:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e6a00640 1 -- 172.21.15.46:0/3721838381 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15cc005740 con 0x7f15e815d940 2024-03-29T08:28:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e6a00640 1 -- 172.21.15.46:0/3721838381 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165097878 0 0) 0x7f15e815b8f0 con 0x7f15e81593d0 2024-03-29T08:28:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e6a00640 1 -- 172.21.15.46:0/3721838381 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15e806d130 con 0x7f15e81593d0 2024-03-29T08:28:55.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e6a00640 1 -- 172.21.15.46:0/3721838381 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2397791645 0 0) 0x7f15cc005740 con 0x7f15e815d940 2024-03-29T08:28:55.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e6a00640 1 -- 172.21.15.46:0/3721838381 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15e815cd90 msgr2=0x7f15e806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:55.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e6a00640 1 --2- 172.21.15.46:0/3721838381 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15e815cd90 0x7f15e806cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e6a00640 1 -- 172.21.15.46:0/3721838381 >> v1:172.21.15.73:6789/0 conn(0x7f15e81593d0 legacy=0x7f15e815d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e6a00640 1 -- 172.21.15.46:0/3721838381 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e815afc0 con 0x7f15e815d940 2024-03-29T08:28:55.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e6a00640 1 -- 172.21.15.46:0/3721838381 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f15e0001a00 con 0x7f15e815d940 2024-03-29T08:28:55.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15dfe00640 1 --2- 172.21.15.46:0/3721838381 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15e815cd90 0x7f15e806cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:28:55.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e6a00640 1 -- 172.21.15.46:0/3721838381 >> v1:172.21.15.46:6789/0 conn(0x7f15e815d940 legacy=0x7f15e81597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e6a00640 1 --2- 172.21.15.46:0/3721838381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15cc005d50 0x7f15cc006150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e6a00640 1 --2- 172.21.15.46:0/3721838381 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15e815cd90 0x7f15cc0069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e6a00640 1 --2- 172.21.15.46:0/3721838381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15cc006f20 0x7f15cc0073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e6a00640 1 -- 172.21.15.46:0/3721838381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e815ad50 con 0x7f15cc006f20 2024-03-29T08:28:55.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e6a00640 1 -- 172.21.15.46:0/3721838381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc00dc50 con 0x7f15cc005d50 2024-03-29T08:28:55.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e6a00640 1 -- 172.21.15.46:0/3721838381 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc00de10 con 0x7f15e815cd90 2024-03-29T08:28:55.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e7e00640 1 --2- 172.21.15.46:0/3721838381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15cc006f20 0x7f15cc0073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15dfe00640 1 --2- 172.21.15.46:0/3721838381 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15e815cd90 0x7f15cc0069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e7400640 1 --2- 172.21.15.46:0/3721838381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15cc005d50 0x7f15cc006150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e7e00640 1 -- 172.21.15.46:0/3721838381 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15e815cd90 msgr2=0x7f15cc0069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e7e00640 1 --2- 172.21.15.46:0/3721838381 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15e815cd90 0x7f15cc0069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e7e00640 1 -- 172.21.15.46:0/3721838381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15cc005d50 msgr2=0x7f15cc006150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e7e00640 1 --2- 172.21.15.46:0/3721838381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15cc005d50 0x7f15cc006150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.742+0000 7f15e7e00640 1 -- 172.21.15.46:0/3721838381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d0009420 con 0x7f15cc006f20 2024-03-29T08:28:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.746+0000 7f15e7e00640 1 --2- 172.21.15.46:0/3721838381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15cc006f20 0x7f15cc0073a0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f15e000a710 tx=0x7f15e001f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13705b631d3e6935 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.746+0000 7f15e6a00640 1 -- 172.21.15.46:0/3721838381 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e00ac630 con 0x7f15cc006f20 2024-03-29T08:28:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.746+0000 7f15e6a00640 1 -- 172.21.15.46:0/3721838381 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e00ac790 con 0x7f15cc006f20 2024-03-29T08:28:55.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.746+0000 7f15e6a00640 1 -- 172.21.15.46:0/3721838381 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e00acb80 con 0x7f15cc006f20 2024-03-29T08:28:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.746+0000 7f15ee200640 1 -- 172.21.15.46:0/3721838381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15cc006f20 msgr2=0x7f15cc0073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.746+0000 7f15ee200640 1 --2- 172.21.15.46:0/3721838381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15cc006f20 0x7f15cc0073a0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f15e000a710 tx=0x7f15e001f980 comp rx=0 tx=0).stop 2024-03-29T08:28:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.746+0000 7f15e7e00640 1 -- 172.21.15.46:0/3721838381 reap_dead start 2024-03-29T08:28:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.746+0000 7f15ee200640 1 -- 172.21.15.46:0/3721838381 shutdown_connections 2024-03-29T08:28:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.746+0000 7f15ee200640 1 -- 172.21.15.46:0/3721838381 >> 172.21.15.46:0/3721838381 conn(0x7f15e8066110 msgr2=0x7f15e8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.746+0000 7f15ee200640 1 -- 172.21.15.46:0/3721838381 shutdown_connections 2024-03-29T08:28:55.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.746+0000 7f15ee200640 1 -- 172.21.15.46:0/3721838381 wait complete. 2024-03-29T08:28:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.746+0000 7f15ee200640 1 Processor -- start 2024-03-29T08:28:55.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.746+0000 7f15ee200640 1 -- start start 2024-03-29T08:28:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15ee200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15cc006f20 0x7f15e8149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15ee200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15e80b20b0 0x7f15e80b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15ee200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e8157c20 0x7f15e81580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15ee200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e8151930 con 0x7f15e8157c20 2024-03-29T08:28:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15ee200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e80b2a50 con 0x7f15e80b20b0 2024-03-29T08:28:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15ee200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e8158950 con 0x7f15cc006f20 2024-03-29T08:28:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e8157c20 0x7f15e81580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e8157c20 0x7f15e81580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45500/0 (socket says 172.21.15.46:45500) 2024-03-29T08:28:55.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15e7e00640 1 -- 172.21.15.46:0/3793594014 learned_addr learned my addr 172.21.15.46:0/3793594014 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:55.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15e7400640 1 --2- 172.21.15.46:0/3793594014 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15cc006f20 0x7f15e8149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15dfe00640 1 --2- 172.21.15.46:0/3793594014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15e80b20b0 0x7f15e80b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15e7e00640 1 -- 172.21.15.46:0/3793594014 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15cc006f20 msgr2=0x7f15e8149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15e7e00640 1 --2- 172.21.15.46:0/3793594014 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15cc006f20 0x7f15e8149ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15e7e00640 1 -- 172.21.15.46:0/3793594014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15e80b20b0 msgr2=0x7f15e80b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15e7e00640 1 --2- 172.21.15.46:0/3793594014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15e80b20b0 0x7f15e80b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15e7e00640 1 -- 172.21.15.46:0/3793594014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e8151b40 con 0x7f15e8157c20 2024-03-29T08:28:55.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15e7e00640 1 --2- 172.21.15.46:0/3793594014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e8157c20 0x7f15e81580a0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f15e000a710 tx=0x7f15cc007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:55.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15df400640 1 -- 172.21.15.46:0/3793594014 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e0001210 con 0x7f15e8157c20 2024-03-29T08:28:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15df400640 1 -- 172.21.15.46:0/3793594014 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e0001800 con 0x7f15e8157c20 2024-03-29T08:28:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15df400640 1 -- 172.21.15.46:0/3793594014 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e00ac630 con 0x7f15e8157c20 2024-03-29T08:28:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15ee200640 1 -- 172.21.15.46:0/3793594014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e8151e20 con 0x7f15e8157c20 2024-03-29T08:28:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15ee200640 1 -- 172.21.15.46:0/3793594014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e81585e0 con 0x7f15e8157c20 2024-03-29T08:28:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15ee200640 1 -- 172.21.15.46:0/3793594014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e8158c30 con 0x7f15e8157c20 2024-03-29T08:28:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.750+0000 7f15df400640 1 -- 172.21.15.46:0/3793594014 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f15e00be070 con 0x7f15e8157c20 2024-03-29T08:28:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.754+0000 7f15df400640 1 --2- 172.21.15.46:0/3793594014 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15c4066690 0x7f15c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.754+0000 7f15df400640 1 -- 172.21.15.46:0/3793594014 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f15e0122880 con 0x7f15e8157c20 2024-03-29T08:28:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.754+0000 7f15e7400640 1 --2- 172.21.15.46:0/3793594014 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15c4066690 0x7f15c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.754+0000 7f15e7400640 1 --2- 172.21.15.46:0/3793594014 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15c4066690 0x7f15c4068b50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f15d8007fd0 tx=0x7f15d8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:55.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.758+0000 7f15df400640 1 -- 172.21.15.46:0/3793594014 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f15e00f3510 con 0x7f15e8157c20 2024-03-29T08:28:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.950+0000 7f15ee200640 1 -- 172.21.15.46:0/3793594014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15e8066c90 con 0x7f15e8157c20 2024-03-29T08:28:55.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.950+0000 7f15df400640 1 -- 172.21.15.46:0/3793594014 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4727 (secure 0 0 0) 0x7f15e00f30a0 con 0x7f15e8157c20 2024-03-29T08:28:55.953 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:28:55.954 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4230030957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4230030957},{"type":"v1","addr":"172.21.15.73:6836","nonce":4230030957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/297138217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":297138217},{"type":"v1","addr":"172.21.15.73:6837","nonce":297138217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3856305332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3856305332},{"type":"v1","addr":"172.21.15.46:6833","nonce":3856305332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":8895,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3142618727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3142618727},{"type":"v1","addr":"172.21.15.46:6835","nonce":3142618727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":84,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:27:18.583223+0000","modified":"2024-03-29T08:28:52.591232+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-29T08:28:55.954 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 85 2024-03-29T08:28:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.950+0000 7f15ee200640 1 -- 172.21.15.46:0/3793594014 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15c4066690 msgr2=0x7f15c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.950+0000 7f15ee200640 1 --2- 172.21.15.46:0/3793594014 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15c4066690 0x7f15c4068b50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f15d8007fd0 tx=0x7f15d8009040 comp rx=0 tx=0).stop 2024-03-29T08:28:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.950+0000 7f15ee200640 1 -- 172.21.15.46:0/3793594014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e8157c20 msgr2=0x7f15e81580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.950+0000 7f15ee200640 1 --2- 172.21.15.46:0/3793594014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e8157c20 0x7f15e81580a0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f15e000a710 tx=0x7f15cc007a10 comp rx=0 tx=0).stop 2024-03-29T08:28:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.950+0000 7f15ee200640 1 -- 172.21.15.46:0/3793594014 shutdown_connections 2024-03-29T08:28:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.950+0000 7f15ee200640 1 --2- 172.21.15.46:0/3793594014 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15c4066690 0x7f15c4068b50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.950+0000 7f15ee200640 1 --2- 172.21.15.46:0/3793594014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15e80b20b0 0x7f15e80b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.950+0000 7f15ee200640 1 --2- 172.21.15.46:0/3793594014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e8157c20 0x7f15e81580a0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.950+0000 7f15ee200640 1 --2- 172.21.15.46:0/3793594014 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15cc006f20 0x7f15e8149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.950+0000 7f15ee200640 1 -- 172.21.15.46:0/3793594014 >> 172.21.15.46:0/3793594014 conn(0x7f15e8066110 msgr2=0x7f15e80af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.954+0000 7f15ee200640 1 -- 172.21.15.46:0/3793594014 shutdown_connections 2024-03-29T08:28:55.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:55.954+0000 7f15ee200640 1 -- 172.21.15.46:0/3793594014 wait complete. 2024-03-29T08:28:55.974 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:28:56.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.110+0000 7f7ea2600640 1 Processor -- start 2024-03-29T08:28:56.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.110+0000 7f7ea2600640 1 -- start start 2024-03-29T08:28:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.114+0000 7f7ea2600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7e9c15c9d0 0x7f7e9c0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.114+0000 7f7ea2600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c0d0030 con 0x7f7e9c099150 2024-03-29T08:28:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.114+0000 7f7ea2600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e9c0cf420 con 0x7f7e9c0cef60 2024-03-29T08:28:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.114+0000 7f7ea2600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e9c0cf700 con 0x7f7e9c15c9d0 2024-03-29T08:28:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.114+0000 7f7e9b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7e9c15c9d0 0x7f7e9c0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.114+0000 7f7e9be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7e9c099150 0x7f7e9c166a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:54086/0 (socket says 172.21.15.46:54086) 2024-03-29T08:28:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.114+0000 7f7e9be00640 1 -- 172.21.15.46:0/2719598649 learned_addr learned my addr 172.21.15.46:0/2719598649 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:28:56.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.114+0000 7f7e9b400640 1 -- 172.21.15.46:0/2719598649 >> v1:172.21.15.73:6789/0 conn(0x7f7e9c0cef60 legacy=0x7f7e9c098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:56.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.114+0000 7f7e9b400640 1 -- 172.21.15.46:0/2719598649 >> v1:172.21.15.46:6789/0 conn(0x7f7e9c099150 legacy=0x7f7e9c166a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:56.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.114+0000 7f7e9b400640 1 -- 172.21.15.46:0/2719598649 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e9c0cf9e0 con 0x7f7e9c15c9d0 2024-03-29T08:28:56.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.114+0000 7f7e9aa00640 1 -- 172.21.15.46:0/2719598649 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155512506 0 0) 0x7f7e9c0d0030 con 0x7f7e9c099150 2024-03-29T08:28:56.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7e9b400640 1 --2- 172.21.15.46:0/2719598649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c15c9d0 0x7f7e9c0cea20 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0097c0 tx=0x7f7e8c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b488f84a47162c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:56.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7e9aa00640 1 -- 172.21.15.46:0/2719598649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0aa5f0 con 0x7f7e9c15c9d0 2024-03-29T08:28:56.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7e9aa00640 1 -- 172.21.15.46:0/2719598649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8c0aa750 con 0x7f7e9c15c9d0 2024-03-29T08:28:56.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7e9aa00640 1 -- 172.21.15.46:0/2719598649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0aaae0 con 0x7f7e9c15c9d0 2024-03-29T08:28:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7ea2600640 1 -- 172.21.15.46:0/2719598649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c15c9d0 msgr2=0x7f7e9c0cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7ea2600640 1 --2- 172.21.15.46:0/2719598649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c15c9d0 0x7f7e9c0cea20 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0097c0 tx=0x7f7e8c01c470 comp rx=0 tx=0).stop 2024-03-29T08:28:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7ea2600640 1 -- 172.21.15.46:0/2719598649 shutdown_connections 2024-03-29T08:28:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7ea2600640 1 --2- 172.21.15.46:0/2719598649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c15c9d0 0x7f7e9c0cea20 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7ea2600640 1 -- 172.21.15.46:0/2719598649 >> 172.21.15.46:0/2719598649 conn(0x7f7e9c0b21a0 msgr2=0x7f7e9c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7ea2600640 1 -- 172.21.15.46:0/2719598649 shutdown_connections 2024-03-29T08:28:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7ea2600640 1 -- 172.21.15.46:0/2719598649 wait complete. 2024-03-29T08:28:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7ea2600640 1 Processor -- start 2024-03-29T08:28:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7ea2600640 1 -- start start 2024-03-29T08:28:56.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7ea2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c099150 0x7f7e9c15a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:56.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7ea2600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e9c0cef60 0x7f7e9c157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:56.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7ea2600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7e9c15c9d0 0x7f7e9c06ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:56.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7ea2600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c15ae30 con 0x7f7e9c099150 2024-03-29T08:28:56.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7ea2600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c15b110 con 0x7f7e9c0cef60 2024-03-29T08:28:56.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7ea2600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9c15b3f0 con 0x7f7e9c15c9d0 2024-03-29T08:28:56.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7e9b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c099150 0x7f7e9c15a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:56.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7e9b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c099150 0x7f7e9c15a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45514/0 (socket says 172.21.15.46:45514) 2024-03-29T08:28:56.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.118+0000 7f7e9b400640 1 -- 172.21.15.46:0/68855193 learned_addr learned my addr 172.21.15.46:0/68855193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:56.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.122+0000 7f7e9be00640 1 --2- 172.21.15.46:0/68855193 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7e9c15c9d0 0x7f7e9c06ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.122+0000 7f7e93e00640 1 --2- 172.21.15.46:0/68855193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e9c0cef60 0x7f7e9c157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.122+0000 7f7e9b400640 1 -- 172.21.15.46:0/68855193 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7e9c15c9d0 msgr2=0x7f7e9c06ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.122+0000 7f7e9b400640 1 --2- 172.21.15.46:0/68855193 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7e9c15c9d0 0x7f7e9c06ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.122+0000 7f7e9b400640 1 -- 172.21.15.46:0/68855193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e9c0cef60 msgr2=0x7f7e9c157800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.122+0000 7f7e9b400640 1 --2- 172.21.15.46:0/68855193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e9c0cef60 0x7f7e9c157800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.122+0000 7f7e9b400640 1 -- 172.21.15.46:0/68855193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c009420 con 0x7f7e9c099150 2024-03-29T08:28:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.122+0000 7f7e9b400640 1 --2- 172.21.15.46:0/68855193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c099150 0x7f7e9c15a850 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0ac040 tx=0x7f7e8c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.122+0000 7f7e93400640 1 -- 172.21.15.46:0/68855193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0be610 con 0x7f7e9c099150 2024-03-29T08:28:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.122+0000 7f7e93400640 1 -- 172.21.15.46:0/68855193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e8c0be7b0 con 0x7f7e9c099150 2024-03-29T08:28:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.122+0000 7f7e93400640 1 -- 172.21.15.46:0/68855193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c0beb60 con 0x7f7e9c099150 2024-03-29T08:28:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.122+0000 7f7ea2600640 1 -- 172.21.15.46:0/68855193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e9c0c8950 con 0x7f7e9c099150 2024-03-29T08:28:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.122+0000 7f7ea2600640 1 -- 172.21.15.46:0/68855193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e9c06f280 con 0x7f7e9c099150 2024-03-29T08:28:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.122+0000 7f7ea2600640 1 -- 172.21.15.46:0/68855193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e9c06f470 con 0x7f7e9c099150 2024-03-29T08:28:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.122+0000 7f7e93400640 1 -- 172.21.15.46:0/68855193 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7e8c0b4400 con 0x7f7e9c099150 2024-03-29T08:28:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.126+0000 7f7e93400640 1 --2- 172.21.15.46:0/68855193 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7e78066690 0x7f7e78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.126+0000 7f7e93400640 1 -- 172.21.15.46:0/68855193 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f7e8c0a7070 con 0x7f7e9c099150 2024-03-29T08:28:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.126+0000 7f7e93e00640 1 --2- 172.21.15.46:0/68855193 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7e78066690 0x7f7e78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.126+0000 7f7e93e00640 1 --2- 172.21.15.46:0/68855193 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7e78066690 0x7f7e78068b50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0ca9b0 tx=0x7f7e84009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:56.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.130+0000 7f7e93400640 1 -- 172.21.15.46:0/68855193 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7e9c06f470 con 0x7f7e9c099150 2024-03-29T08:28:56.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.326+0000 7f7ea2600640 1 -- 172.21.15.46:0/68855193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e9c06f470 con 0x7f7e9c099150 2024-03-29T08:28:56.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.330+0000 7f7e93400640 1 -- 172.21.15.46:0/68855193 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4727 (secure 0 0 0) 0x7f7e9c06f470 con 0x7f7e9c099150 2024-03-29T08:28:56.332 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:28:56.332 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":85,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4230030957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4230030957},{"type":"v1","addr":"172.21.15.73:6836","nonce":4230030957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/297138217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":297138217},{"type":"v1","addr":"172.21.15.73:6837","nonce":297138217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3856305332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3856305332},{"type":"v1","addr":"172.21.15.46:6833","nonce":3856305332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":8895,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3142618727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3142618727},{"type":"v1","addr":"172.21.15.46:6835","nonce":3142618727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":84,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:27:18.583223+0000","modified":"2024-03-29T08:28:52.591232+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-03-29T08:28:56.333 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 85 2024-03-29T08:28:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.330+0000 7f7ea2600640 1 -- 172.21.15.46:0/68855193 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7e78066690 msgr2=0x7f7e78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.330+0000 7f7ea2600640 1 --2- 172.21.15.46:0/68855193 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7e78066690 0x7f7e78068b50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0ca9b0 tx=0x7f7e84009040 comp rx=0 tx=0).stop 2024-03-29T08:28:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.330+0000 7f7ea2600640 1 -- 172.21.15.46:0/68855193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c099150 msgr2=0x7f7e9c15a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.330+0000 7f7ea2600640 1 --2- 172.21.15.46:0/68855193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c099150 0x7f7e9c15a850 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c0ac040 tx=0x7f7e8c01acb0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.330+0000 7f7ea2600640 1 -- 172.21.15.46:0/68855193 shutdown_connections 2024-03-29T08:28:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.330+0000 7f7ea2600640 1 --2- 172.21.15.46:0/68855193 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7e78066690 0x7f7e78068b50 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.330+0000 7f7ea2600640 1 --2- 172.21.15.46:0/68855193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e9c0cef60 0x7f7e9c157800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.330+0000 7f7ea2600640 1 --2- 172.21.15.46:0/68855193 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7e9c15c9d0 0x7f7e9c06ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.330+0000 7f7ea2600640 1 --2- 172.21.15.46:0/68855193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e9c099150 0x7f7e9c15a850 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.330+0000 7f7ea2600640 1 -- 172.21.15.46:0/68855193 >> 172.21.15.46:0/68855193 conn(0x7f7e9c0b21a0 msgr2=0x7f7e9c0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.330+0000 7f7ea2600640 1 -- 172.21.15.46:0/68855193 shutdown_connections 2024-03-29T08:28:56.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.330+0000 7f7ea2600640 1 -- 172.21.15.46:0/68855193 wait complete. 2024-03-29T08:28:56.354 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:28:56.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.490+0000 7f59dd200640 1 Processor -- start 2024-03-29T08:28:56.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.490+0000 7f59dd200640 1 -- start start 2024-03-29T08:28:56.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.494+0000 7f59dd200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f59d81593d0 0x7f59d806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:56.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.494+0000 7f59dd200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59d815d230 0x7f59d815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:56.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.494+0000 7f59dd200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f59d815dbd0 0x7f59d815bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:56.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.494+0000 7f59dd200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d806d130 con 0x7f59d815dbd0 2024-03-29T08:28:56.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.494+0000 7f59dd200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d815b8f0 con 0x7f59d81593d0 2024-03-29T08:28:56.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.494+0000 7f59dd200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d815ad50 con 0x7f59d815d230 2024-03-29T08:28:56.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.494+0000 7f59d6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f59d815dbd0 0x7f59d815bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:56.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.494+0000 7f59d6800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f59d815dbd0 0x7f59d815bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38882/0 (socket says 172.21.15.46:38882) 2024-03-29T08:28:56.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.494+0000 7f59d6800640 1 -- 172.21.15.46:0/1871876752 learned_addr learned my addr 172.21.15.46:0/1871876752 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:56.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.494+0000 7f59d5e00640 1 --2- 172.21.15.46:0/1871876752 >> v2:172.21.15.73:3300/0 conn(0x7f59d81593d0 0x7f59d806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:56.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.494+0000 7f59cfe00640 1 --2- 172.21.15.46:0/1871876752 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59d815d230 0x7f59d815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:56.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.494+0000 7f59d6800640 1 -- 172.21.15.46:0/1871876752 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59d815d230 msgr2=0x7f59d815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:56.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.494+0000 7f59d6800640 1 --2- 172.21.15.46:0/1871876752 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59d815d230 0x7f59d815d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.494+0000 7f59d6800640 1 -- 172.21.15.46:0/1871876752 >> v2:172.21.15.73:3300/0 conn(0x7f59d81593d0 msgr2=0x7f59d806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:56.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.494+0000 7f59d6800640 1 --2- 172.21.15.46:0/1871876752 >> v2:172.21.15.73:3300/0 conn(0x7f59d81593d0 0x7f59d806cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.494+0000 7f59d6800640 1 -- 172.21.15.46:0/1871876752 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d815e060 con 0x7f59d815dbd0 2024-03-29T08:28:56.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.498+0000 7f59d6800640 1 --2- 172.21.15.46:0/1871876752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59d815dbd0 0x7f59d815bd80 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f59d806e460 tx=0x7f59d001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b1e2020415b1c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:56.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.498+0000 7f59d5400640 1 -- 172.21.15.46:0/1871876752 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d00a5870 con 0x7f59d815dbd0 2024-03-29T08:28:56.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.498+0000 7f59d5400640 1 -- 172.21.15.46:0/1871876752 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d00a59d0 con 0x7f59d815dbd0 2024-03-29T08:28:56.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.498+0000 7f59d5400640 1 -- 172.21.15.46:0/1871876752 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d00a5d60 con 0x7f59d815dbd0 2024-03-29T08:28:56.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.498+0000 7f59dd200640 1 -- 172.21.15.46:0/1871876752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59d815dbd0 msgr2=0x7f59d815bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:56.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.498+0000 7f59dd200640 1 --2- 172.21.15.46:0/1871876752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59d815dbd0 0x7f59d815bd80 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f59d806e460 tx=0x7f59d001c9f0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.498+0000 7f59dd200640 1 -- 172.21.15.46:0/1871876752 shutdown_connections 2024-03-29T08:28:56.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.498+0000 7f59dd200640 1 --2- 172.21.15.46:0/1871876752 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59d815dbd0 0x7f59d815bd80 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.498+0000 7f59dd200640 1 --2- 172.21.15.46:0/1871876752 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59d815d230 0x7f59d815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.498+0000 7f59dd200640 1 --2- 172.21.15.46:0/1871876752 >> v2:172.21.15.73:3300/0 conn(0x7f59d81593d0 0x7f59d806cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.498+0000 7f59dd200640 1 -- 172.21.15.46:0/1871876752 >> 172.21.15.46:0/1871876752 conn(0x7f59d8066110 msgr2=0x7f59d8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:56.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.498+0000 7f59dd200640 1 -- 172.21.15.46:0/1871876752 shutdown_connections 2024-03-29T08:28:56.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.498+0000 7f59dd200640 1 -- 172.21.15.46:0/1871876752 wait complete. 2024-03-29T08:28:56.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.498+0000 7f59dd200640 1 Processor -- start 2024-03-29T08:28:56.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.498+0000 7f59dd200640 1 -- start start 2024-03-29T08:28:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59dd200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59d81593d0 0x7f59d80cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59dd200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59d815d230 0x7f59d814fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59dd200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59d815dbd0 0x7f59d80ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59dd200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d80ce0f0 con 0x7f59d815dbd0 2024-03-29T08:28:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59dd200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d80cdb30 con 0x7f59d81593d0 2024-03-29T08:28:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59dd200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d80cd230 con 0x7f59d815d230 2024-03-29T08:28:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59d6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59d815dbd0 0x7f59d80ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59d6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59d815dbd0 0x7f59d80ae9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38898/0 (socket says 172.21.15.46:38898) 2024-03-29T08:28:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59d6800640 1 -- 172.21.15.46:0/1600845441 learned_addr learned my addr 172.21.15.46:0/1600845441 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59d5e00640 1 --2- 172.21.15.46:0/1600845441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59d81593d0 0x7f59d80cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59cfe00640 1 --2- 172.21.15.46:0/1600845441 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59d815d230 0x7f59d814fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:56.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59d6800640 1 -- 172.21.15.46:0/1600845441 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59d815d230 msgr2=0x7f59d814fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:56.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59d6800640 1 --2- 172.21.15.46:0/1600845441 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59d815d230 0x7f59d814fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59d6800640 1 -- 172.21.15.46:0/1600845441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59d81593d0 msgr2=0x7f59d80cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:56.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59d6800640 1 --2- 172.21.15.46:0/1600845441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59d81593d0 0x7f59d80cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59d6800640 1 -- 172.21.15.46:0/1600845441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d0009b30 con 0x7f59d815dbd0 2024-03-29T08:28:56.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59d6800640 1 --2- 172.21.15.46:0/1600845441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59d815dbd0 0x7f59d80ae9d0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f59d00af040 tx=0x7f59d0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:56.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59ce000640 1 -- 172.21.15.46:0/1600845441 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d00b8640 con 0x7f59d815dbd0 2024-03-29T08:28:56.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59ce000640 1 -- 172.21.15.46:0/1600845441 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d00b87a0 con 0x7f59d815dbd0 2024-03-29T08:28:56.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59dd200640 1 -- 172.21.15.46:0/1600845441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d81559d0 con 0x7f59d815dbd0 2024-03-29T08:28:56.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59ce000640 1 -- 172.21.15.46:0/1600845441 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d00b8b30 con 0x7f59d815dbd0 2024-03-29T08:28:56.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59dd200640 1 -- 172.21.15.46:0/1600845441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d80cdd80 con 0x7f59d815dbd0 2024-03-29T08:28:56.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59dd200640 1 -- 172.21.15.46:0/1600845441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d815ab80 con 0x7f59d815dbd0 2024-03-29T08:28:56.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.502+0000 7f59ce000640 1 -- 172.21.15.46:0/1600845441 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f59d00b8c90 con 0x7f59d815dbd0 2024-03-29T08:28:56.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.506+0000 7f59ce000640 1 --2- 172.21.15.46:0/1600845441 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f59b4066690 0x7f59b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:56.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.506+0000 7f59ce000640 1 -- 172.21.15.46:0/1600845441 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f59d011ee10 con 0x7f59d815dbd0 2024-03-29T08:28:56.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.506+0000 7f59d5e00640 1 --2- 172.21.15.46:0/1600845441 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f59b4066690 0x7f59b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:56.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.506+0000 7f59d5e00640 1 --2- 172.21.15.46:0/1600845441 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f59b4066690 0x7f59b4068b50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f59c80099d0 tx=0x7f59c8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:56.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.510+0000 7f59ce000640 1 -- 172.21.15.46:0/1600845441 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f59d0122030 con 0x7f59d815dbd0 2024-03-29T08:28:56.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.674+0000 7f59dd200640 1 -- 172.21.15.46:0/1600845441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59d8066c90 con 0x7f59d815dbd0 2024-03-29T08:28:56.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.674+0000 7f59ce000640 1 -- 172.21.15.46:0/1600845441 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16357 (secure 0 0 0) 0x7f59d00ec080 con 0x7f59d815dbd0 2024-03-29T08:28:56.678 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:28:56.678 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":110,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:28:52.590731+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:27:17.118875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-03-29T08:27:17.573928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.f","mappings":[{"from":5,"to":0}]},{"pgid":"14.14","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/1455003836":"2024-03-30T08:28:52.590708+0000","172.21.15.46:6834/1455003836":"2024-03-30T08:28:52.590708+0000","172.21.15.46:6835/824184385":"2024-03-30T08:28:15.875168+0000","172.21.15.46:6834/824184385":"2024-03-30T08:28:15.875168+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:28:56.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.678+0000 7f59dd200640 1 -- 172.21.15.46:0/1600845441 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f59b4066690 msgr2=0x7f59b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:56.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.678+0000 7f59dd200640 1 --2- 172.21.15.46:0/1600845441 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f59b4066690 0x7f59b4068b50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f59c80099d0 tx=0x7f59c8008040 comp rx=0 tx=0).stop 2024-03-29T08:28:56.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.678+0000 7f59dd200640 1 -- 172.21.15.46:0/1600845441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59d815dbd0 msgr2=0x7f59d80ae9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:56.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.678+0000 7f59dd200640 1 --2- 172.21.15.46:0/1600845441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59d815dbd0 0x7f59d80ae9d0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f59d00af040 tx=0x7f59d0019790 comp rx=0 tx=0).stop 2024-03-29T08:28:56.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.678+0000 7f59dd200640 1 -- 172.21.15.46:0/1600845441 shutdown_connections 2024-03-29T08:28:56.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.678+0000 7f59dd200640 1 --2- 172.21.15.46:0/1600845441 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f59b4066690 0x7f59b4068b50 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.678+0000 7f59dd200640 1 --2- 172.21.15.46:0/1600845441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59d815dbd0 0x7f59d80ae9d0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.678+0000 7f59dd200640 1 --2- 172.21.15.46:0/1600845441 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f59d815d230 0x7f59d814fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.678+0000 7f59dd200640 1 --2- 172.21.15.46:0/1600845441 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59d81593d0 0x7f59d80cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.678+0000 7f59dd200640 1 -- 172.21.15.46:0/1600845441 >> 172.21.15.46:0/1600845441 conn(0x7f59d8066110 msgr2=0x7f59d80701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:56.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.678+0000 7f59dd200640 1 -- 172.21.15.46:0/1600845441 shutdown_connections 2024-03-29T08:28:56.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.678+0000 7f59dd200640 1 -- 172.21.15.46:0/1600845441 wait complete. 2024-03-29T08:28:56.700 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T08:28:56.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.838+0000 7f5769e00640 1 Processor -- start 2024-03-29T08:28:56.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.838+0000 7f5769e00640 1 -- start start 2024-03-29T08:28:56.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.838+0000 7f5769e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576406dbb0 0x7f576415d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:56.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.838+0000 7f5769e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5764070c70 con 0x7f5764142ab0 2024-03-29T08:28:56.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.838+0000 7f5769e00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5764070340 con 0x7f576415d940 2024-03-29T08:28:56.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.838+0000 7f5769e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5764070620 con 0x7f576406dbb0 2024-03-29T08:28:56.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.838+0000 7f5762a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5764142ab0 0x7f576406d4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55760/0 (socket says 172.21.15.46:55760) 2024-03-29T08:28:56.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.838+0000 7f5762a00640 1 -- 172.21.15.46:0/267349314 learned_addr learned my addr 172.21.15.46:0/267349314 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:28:56.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.838+0000 7f575a000640 1 --2- 172.21.15.46:0/267349314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576406dbb0 0x7f576415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:56.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.842+0000 7f575a000640 1 -- 172.21.15.46:0/267349314 >> v1:172.21.15.73:6789/0 conn(0x7f576415d940 legacy=0x7f57640b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:56.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.842+0000 7f5762000640 1 -- 172.21.15.46:0/267349314 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2350637649 0 0) 0x7f5764070c70 con 0x7f5764142ab0 2024-03-29T08:28:56.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.842+0000 7f575a000640 1 -- 172.21.15.46:0/267349314 >> v1:172.21.15.46:6789/0 conn(0x7f5764142ab0 legacy=0x7f576406d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:56.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.842+0000 7f575a000640 1 -- 172.21.15.46:0/267349314 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5764070900 con 0x7f576406dbb0 2024-03-29T08:28:56.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.842+0000 7f575a000640 1 --2- 172.21.15.46:0/267349314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576406dbb0 0x7f576415d400 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f574c0097f0 tx=0x7f574c01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a9990d305819bfe server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:56.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.842+0000 7f5762000640 1 -- 172.21.15.46:0/267349314 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c0ab3f0 con 0x7f576406dbb0 2024-03-29T08:28:56.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.842+0000 7f5762000640 1 -- 172.21.15.46:0/267349314 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574c0ab550 con 0x7f576406dbb0 2024-03-29T08:28:56.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.842+0000 7f5762000640 1 -- 172.21.15.46:0/267349314 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c0ab8e0 con 0x7f576406dbb0 2024-03-29T08:28:56.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.842+0000 7f5769e00640 1 -- 172.21.15.46:0/267349314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576406dbb0 msgr2=0x7f576415d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:56.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.842+0000 7f5769e00640 1 --2- 172.21.15.46:0/267349314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576406dbb0 0x7f576415d400 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f574c0097f0 tx=0x7f574c01c4e0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f5769e00640 1 -- 172.21.15.46:0/267349314 shutdown_connections 2024-03-29T08:28:56.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f5769e00640 1 --2- 172.21.15.46:0/267349314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576406dbb0 0x7f576415d400 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f5769e00640 1 -- 172.21.15.46:0/267349314 >> 172.21.15.46:0/267349314 conn(0x7f5764066080 msgr2=0x7f5764069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:56.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f5769e00640 1 -- 172.21.15.46:0/267349314 shutdown_connections 2024-03-29T08:28:56.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f5769e00640 1 -- 172.21.15.46:0/267349314 wait complete. 2024-03-29T08:28:56.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f5769e00640 1 Processor -- start 2024-03-29T08:28:56.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f5769e00640 1 -- start start 2024-03-29T08:28:56.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f5769e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f576406dbb0 0x7f57640acc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:56.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f5769e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5764142ab0 0x7f576413f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:56.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f5769e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576415d940 0x7f5764139b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f5769e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576413d080 con 0x7f5764142ab0 2024-03-29T08:28:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f5769e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576413cb00 con 0x7f576406dbb0 2024-03-29T08:28:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f5769e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576413c4f0 con 0x7f576415d940 2024-03-29T08:28:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f575a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5764142ab0 0x7f576413f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f575a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5764142ab0 0x7f576413f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38908/0 (socket says 172.21.15.46:38908) 2024-03-29T08:28:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f575a000640 1 -- 172.21.15.46:0/21387169 learned_addr learned my addr 172.21.15.46:0/21387169 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f5762a00640 1 --2- 172.21.15.46:0/21387169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f576406dbb0 0x7f57640acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f5763400640 1 --2- 172.21.15.46:0/21387169 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576415d940 0x7f5764139b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f575a000640 1 -- 172.21.15.46:0/21387169 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576415d940 msgr2=0x7f5764139b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f575a000640 1 --2- 172.21.15.46:0/21387169 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576415d940 0x7f5764139b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f575a000640 1 -- 172.21.15.46:0/21387169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f576406dbb0 msgr2=0x7f57640acc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f575a000640 1 --2- 172.21.15.46:0/21387169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f576406dbb0 0x7f57640acc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f575a000640 1 -- 172.21.15.46:0/21387169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c009420 con 0x7f5764142ab0 2024-03-29T08:28:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.846+0000 7f575a000640 1 --2- 172.21.15.46:0/21387169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5764142ab0 0x7f576413f6e0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f574c0ad040 tx=0x7f574c01ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:56.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.850+0000 7f575be00640 1 -- 172.21.15.46:0/21387169 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c004220 con 0x7f5764142ab0 2024-03-29T08:28:56.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.850+0000 7f575be00640 1 -- 172.21.15.46:0/21387169 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574c0b9070 con 0x7f5764142ab0 2024-03-29T08:28:56.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.850+0000 7f575be00640 1 -- 172.21.15.46:0/21387169 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574c006e20 con 0x7f5764142ab0 2024-03-29T08:28:56.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.850+0000 7f5769e00640 1 -- 172.21.15.46:0/21387169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576413c7c0 con 0x7f5764142ab0 2024-03-29T08:28:56.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.850+0000 7f5769e00640 1 -- 172.21.15.46:0/21387169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576413f1c0 con 0x7f5764142ab0 2024-03-29T08:28:56.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.850+0000 7f5769e00640 1 -- 172.21.15.46:0/21387169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576413f3b0 con 0x7f5764142ab0 2024-03-29T08:28:56.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.850+0000 7f575be00640 1 -- 172.21.15.46:0/21387169 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f574c003620 con 0x7f5764142ab0 2024-03-29T08:28:56.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.850+0000 7f575be00640 1 --2- 172.21.15.46:0/21387169 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5740066690 0x7f5740068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:56.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.850+0000 7f575be00640 1 -- 172.21.15.46:0/21387169 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f574c0bb070 con 0x7f5764142ab0 2024-03-29T08:28:56.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.850+0000 7f5762a00640 1 --2- 172.21.15.46:0/21387169 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5740066690 0x7f5740068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:56.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.850+0000 7f5762a00640 1 --2- 172.21.15.46:0/21387169 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5740066690 0x7f5740068b50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5754001950 tx=0x7f57540018e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:56.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:56.854+0000 7f575be00640 1 -- 172.21.15.46:0/21387169 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f574c0ec0c0 con 0x7f5764142ab0 2024-03-29T08:28:57.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.050+0000 7f5769e00640 1 -- 172.21.15.46:0/21387169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5764071a90 con 0x7f5764142ab0 2024-03-29T08:28:57.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.270+0000 7f575be00640 1 -- 172.21.15.46:0/21387169 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) v1 ==== 161+0+0 (secure 0 0 0) 0x7f574c0c4070 con 0x7f5764142ab0 2024-03-29T08:28:57.275 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T08:28:57.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.274+0000 7f5769e00640 1 -- 172.21.15.46:0/21387169 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5740066690 msgr2=0x7f5740068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:57.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.274+0000 7f5769e00640 1 --2- 172.21.15.46:0/21387169 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5740066690 0x7f5740068b50 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5754001950 tx=0x7f57540018e0 comp rx=0 tx=0).stop 2024-03-29T08:28:57.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.274+0000 7f5769e00640 1 -- 172.21.15.46:0/21387169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5764142ab0 msgr2=0x7f576413f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:57.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.274+0000 7f5769e00640 1 --2- 172.21.15.46:0/21387169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5764142ab0 0x7f576413f6e0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f574c0ad040 tx=0x7f574c01ad20 comp rx=0 tx=0).stop 2024-03-29T08:28:57.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.274+0000 7f5769e00640 1 -- 172.21.15.46:0/21387169 shutdown_connections 2024-03-29T08:28:57.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.274+0000 7f5769e00640 1 --2- 172.21.15.46:0/21387169 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5740066690 0x7f5740068b50 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:57.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.274+0000 7f5769e00640 1 --2- 172.21.15.46:0/21387169 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f576415d940 0x7f5764139b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:57.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.274+0000 7f5769e00640 1 --2- 172.21.15.46:0/21387169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5764142ab0 0x7f576413f6e0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:57.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.274+0000 7f5769e00640 1 --2- 172.21.15.46:0/21387169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f576406dbb0 0x7f57640acc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:57.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.274+0000 7f5769e00640 1 -- 172.21.15.46:0/21387169 >> 172.21.15.46:0/21387169 conn(0x7f5764066080 msgr2=0x7f576415a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:57.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.274+0000 7f5769e00640 1 -- 172.21.15.46:0/21387169 shutdown_connections 2024-03-29T08:28:57.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.274+0000 7f5769e00640 1 -- 172.21.15.46:0/21387169 wait complete. 2024-03-29T08:28:57.296 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T08:28:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.434+0000 7f86c2a00640 1 Processor -- start 2024-03-29T08:28:57.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.438+0000 7f86c2a00640 1 -- start start 2024-03-29T08:28:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.438+0000 7f86c2a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86bc143eb0 0x7f86bc0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.438+0000 7f86c2a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86bc0b1d40 0x7f86bc0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.438+0000 7f86c2a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86bc0b2360 con 0x7f86bc15efe0 2024-03-29T08:28:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.438+0000 7f86c2a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86bc15e500 con 0x7f86bc143eb0 2024-03-29T08:28:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.438+0000 7f86c2a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc15e7e0 con 0x7f86bc0b1d40 2024-03-29T08:28:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.438+0000 7f86bbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86bc143eb0 0x7f86bc0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.438+0000 7f86bbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f86bc143eb0 0x7f86bc0cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38920/0 (socket says 172.21.15.46:38920) 2024-03-29T08:28:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.438+0000 7f86b3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f86bc15efe0 0x7f86bc0b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55764/0 (socket says 172.21.15.46:55764) 2024-03-29T08:28:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.438+0000 7f86bbe00640 1 -- 172.21.15.46:0/38513788 learned_addr learned my addr 172.21.15.46:0/38513788 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.438+0000 7f86c0a00640 1 --2- 172.21.15.46:0/38513788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86bc0b1d40 0x7f86bc0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.438+0000 7f86bbe00640 1 -- 172.21.15.46:0/38513788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86bc0b1d40 msgr2=0x7f86bc0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:57.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.438+0000 7f86bbe00640 1 --2- 172.21.15.46:0/38513788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86bc0b1d40 0x7f86bc0cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.438+0000 7f86bbe00640 1 -- 172.21.15.46:0/38513788 >> v1:172.21.15.46:6789/0 conn(0x7f86bc15efe0 legacy=0x7f86bc0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.438+0000 7f86bbe00640 1 -- 172.21.15.46:0/38513788 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc15eac0 con 0x7f86bc143eb0 2024-03-29T08:28:57.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.438+0000 7f86bb400640 1 -- 172.21.15.46:0/38513788 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 47731592 0 0) 0x7f86bc0b2360 con 0x7f86bc15efe0 2024-03-29T08:28:57.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.442+0000 7f86bbe00640 1 --2- 172.21.15.46:0/38513788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86bc143eb0 0x7f86bc0cb790 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009870 tx=0x7f86ac01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e541df0b7a31191f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.442+0000 7f86bb400640 1 -- 172.21.15.46:0/38513788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0ab670 con 0x7f86bc143eb0 2024-03-29T08:28:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.442+0000 7f86bb400640 1 -- 172.21.15.46:0/38513788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ac0ab7d0 con 0x7f86bc143eb0 2024-03-29T08:28:57.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.442+0000 7f86bb400640 1 -- 172.21.15.46:0/38513788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0abb60 con 0x7f86bc143eb0 2024-03-29T08:28:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.442+0000 7f86c2a00640 1 -- 172.21.15.46:0/38513788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86bc143eb0 msgr2=0x7f86bc0cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.442+0000 7f86c2a00640 1 --2- 172.21.15.46:0/38513788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86bc143eb0 0x7f86bc0cb790 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f86ac009870 tx=0x7f86ac01c560 comp rx=0 tx=0).stop 2024-03-29T08:28:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.442+0000 7f86c2a00640 1 -- 172.21.15.46:0/38513788 shutdown_connections 2024-03-29T08:28:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.442+0000 7f86c2a00640 1 --2- 172.21.15.46:0/38513788 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86bc0b1d40 0x7f86bc0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.442+0000 7f86c2a00640 1 --2- 172.21.15.46:0/38513788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86bc143eb0 0x7f86bc0cb790 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.442+0000 7f86c2a00640 1 -- 172.21.15.46:0/38513788 >> 172.21.15.46:0/38513788 conn(0x7f86bc06a390 msgr2=0x7f86bc0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.442+0000 7f86c2a00640 1 -- 172.21.15.46:0/38513788 shutdown_connections 2024-03-29T08:28:57.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.442+0000 7f86c2a00640 1 -- 172.21.15.46:0/38513788 wait complete. 2024-03-29T08:28:57.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.442+0000 7f86c2a00640 1 Processor -- start 2024-03-29T08:28:57.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.442+0000 7f86c2a00640 1 -- start start 2024-03-29T08:28:57.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c2a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86bc0b1d40 0x7f86bc143020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:57.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c2a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86bc143eb0 0x7f86bc13c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:57.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c2a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86bc15efe0 0x7f86bc0afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:57.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c2a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc0ad250 con 0x7f86bc15efe0 2024-03-29T08:28:57.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c2a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc0ad530 con 0x7f86bc0b1d40 2024-03-29T08:28:57.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c2a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc0ad810 con 0x7f86bc143eb0 2024-03-29T08:28:57.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86bc15efe0 0x7f86bc0afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:57.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86bc15efe0 0x7f86bc0afba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38932/0 (socket says 172.21.15.46:38932) 2024-03-29T08:28:57.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c0a00640 1 -- 172.21.15.46:0/3510358339 learned_addr learned my addr 172.21.15.46:0/3510358339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:57.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86b3400640 1 --2- 172.21.15.46:0/3510358339 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86bc143eb0 0x7f86bc13c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:57.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86bbe00640 1 --2- 172.21.15.46:0/3510358339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86bc0b1d40 0x7f86bc143020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:57.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c0a00640 1 -- 172.21.15.46:0/3510358339 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86bc143eb0 msgr2=0x7f86bc13c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:57.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c0a00640 1 --2- 172.21.15.46:0/3510358339 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86bc143eb0 0x7f86bc13c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:57.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c0a00640 1 -- 172.21.15.46:0/3510358339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86bc0b1d40 msgr2=0x7f86bc143020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:57.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c0a00640 1 --2- 172.21.15.46:0/3510358339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86bc0b1d40 0x7f86bc143020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:57.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c0a00640 1 -- 172.21.15.46:0/3510358339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ac0094d0 con 0x7f86bc15efe0 2024-03-29T08:28:57.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86bbe00640 1 --2- 172.21.15.46:0/3510358339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86bc0b1d40 0x7f86bc143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:28:57.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c0a00640 1 --2- 172.21.15.46:0/3510358339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86bc15efe0 0x7f86bc0afba0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f86b400d800 tx=0x7f86b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:57.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86b8c00640 1 -- 172.21.15.46:0/3510358339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b400b6c0 con 0x7f86bc15efe0 2024-03-29T08:28:57.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86b8c00640 1 -- 172.21.15.46:0/3510358339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b400b820 con 0x7f86bc15efe0 2024-03-29T08:28:57.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86b8c00640 1 -- 172.21.15.46:0/3510358339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b400bbd0 con 0x7f86bc15efe0 2024-03-29T08:28:57.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c2a00640 1 -- 172.21.15.46:0/3510358339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86bc142b00 con 0x7f86bc15efe0 2024-03-29T08:28:57.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c2a00640 1 -- 172.21.15.46:0/3510358339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86bc1427b0 con 0x7f86bc15efe0 2024-03-29T08:28:57.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86c2a00640 1 -- 172.21.15.46:0/3510358339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86bc142d20 con 0x7f86bc15efe0 2024-03-29T08:28:57.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.446+0000 7f86b8c00640 1 -- 172.21.15.46:0/3510358339 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f86b400bd30 con 0x7f86bc15efe0 2024-03-29T08:28:57.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.450+0000 7f86b8c00640 1 --2- 172.21.15.46:0/3510358339 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8698066690 0x7f8698068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:57.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.450+0000 7f86b8c00640 1 -- 172.21.15.46:0/3510358339 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f86b4084740 con 0x7f86bc15efe0 2024-03-29T08:28:57.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.450+0000 7f86bbe00640 1 --2- 172.21.15.46:0/3510358339 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8698066690 0x7f8698068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:57.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.450+0000 7f86bbe00640 1 --2- 172.21.15.46:0/3510358339 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8698066690 0x7f8698068b50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0af000 tx=0x7f86ac0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:57.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.454+0000 7f86b8c00640 1 -- 172.21.15.46:0/3510358339 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f86b40519c0 con 0x7f86bc15efe0 2024-03-29T08:28:57.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:57.646+0000 7f86c2a00640 1 -- 172.21.15.46:0/3510358339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f86bc071a70 con 0x7f86bc15efe0 2024-03-29T08:28:58.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.278+0000 7f86b8c00640 1 -- 172.21.15.46:0/3510358339 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v87) v1 ==== 104+0+0 (secure 0 0 0) 0x7f86b4054fe0 con 0x7f86bc15efe0 2024-03-29T08:28:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.282+0000 7f86c2a00640 1 -- 172.21.15.46:0/3510358339 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8698066690 msgr2=0x7f8698068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.282+0000 7f86c2a00640 1 --2- 172.21.15.46:0/3510358339 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8698066690 0x7f8698068b50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f86ac0af000 tx=0x7f86ac0a7000 comp rx=0 tx=0).stop 2024-03-29T08:28:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.282+0000 7f86c2a00640 1 -- 172.21.15.46:0/3510358339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86bc15efe0 msgr2=0x7f86bc0afba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.282+0000 7f86c2a00640 1 --2- 172.21.15.46:0/3510358339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86bc15efe0 0x7f86bc0afba0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f86b400d800 tx=0x7f86b400dcc0 comp rx=0 tx=0).stop 2024-03-29T08:28:58.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.282+0000 7f86c2a00640 1 -- 172.21.15.46:0/3510358339 shutdown_connections 2024-03-29T08:28:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.282+0000 7f86c2a00640 1 --2- 172.21.15.46:0/3510358339 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8698066690 0x7f8698068b50 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.282+0000 7f86c2a00640 1 --2- 172.21.15.46:0/3510358339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86bc15efe0 0x7f86bc0afba0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.282+0000 7f86c2a00640 1 --2- 172.21.15.46:0/3510358339 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86bc143eb0 0x7f86bc13c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.282+0000 7f86c2a00640 1 --2- 172.21.15.46:0/3510358339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86bc0b1d40 0x7f86bc143020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.282+0000 7f86c2a00640 1 -- 172.21.15.46:0/3510358339 >> 172.21.15.46:0/3510358339 conn(0x7f86bc06a390 msgr2=0x7f86bc1589f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.282+0000 7f86c2a00640 1 -- 172.21.15.46:0/3510358339 shutdown_connections 2024-03-29T08:28:58.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.282+0000 7f86c2a00640 1 -- 172.21.15.46:0/3510358339 wait complete. 2024-03-29T08:28:58.309 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T08:28:58.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.454+0000 7fb315c00640 1 Processor -- start 2024-03-29T08:28:58.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.454+0000 7fb315c00640 1 -- start start 2024-03-29T08:28:58.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.454+0000 7fb315c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb310160060 0x7fb3100cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:58.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.454+0000 7fb315c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb3100cfc10 0x7fb310098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:58.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.454+0000 7fb315c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3100cea60 con 0x7fb3100b1780 2024-03-29T08:28:58.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.454+0000 7fb315c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb31006bea0 con 0x7fb3100cfc10 2024-03-29T08:28:58.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.454+0000 7fb315c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3100ced00 con 0x7fb310160060 2024-03-29T08:28:58.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.454+0000 7fb30f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb3100cfc10 0x7fb310098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:58.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.454+0000 7fb30f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb3100cfc10 0x7fb310098a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38936/0 (socket says 172.21.15.46:38936) 2024-03-29T08:28:58.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.454+0000 7fb30f400640 1 -- 172.21.15.46:0/1143460334 learned_addr learned my addr 172.21.15.46:0/1143460334 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.454+0000 7fb30ea00640 1 --2- 172.21.15.46:0/1143460334 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb310160060 0x7fb3100cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.454+0000 7fb30f400640 1 -- 172.21.15.46:0/1143460334 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb310160060 msgr2=0x7fb3100cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.454+0000 7fb30f400640 1 --2- 172.21.15.46:0/1143460334 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb310160060 0x7fb3100cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.454+0000 7fb30f400640 1 -- 172.21.15.46:0/1143460334 >> v1:172.21.15.73:6789/0 conn(0x7fb3100b1780 legacy=0x7fb3100cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.454+0000 7fb30f400640 1 -- 172.21.15.46:0/1143460334 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3100cefe0 con 0x7fb3100cfc10 2024-03-29T08:28:58.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.454+0000 7fb30ea00640 1 --2- 172.21.15.46:0/1143460334 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb310160060 0x7fb3100cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:58.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.458+0000 7fb30f400640 1 --2- 172.21.15.46:0/1143460334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3100cfc10 0x7fb310098a40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fb308009e50 tx=0x7fb30801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa54f11cb3f6e30 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:58.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.458+0000 7fb30e000640 1 -- 172.21.15.46:0/1143460334 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3080a5610 con 0x7fb3100cfc10 2024-03-29T08:28:58.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.458+0000 7fb30e000640 1 -- 172.21.15.46:0/1143460334 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3080a5770 con 0x7fb3100cfc10 2024-03-29T08:28:58.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.458+0000 7fb30e000640 1 -- 172.21.15.46:0/1143460334 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3080a5b00 con 0x7fb3100cfc10 2024-03-29T08:28:58.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.458+0000 7fb315c00640 1 -- 172.21.15.46:0/1143460334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3100cfc10 msgr2=0x7fb310098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:58.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.458+0000 7fb315c00640 1 --2- 172.21.15.46:0/1143460334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3100cfc10 0x7fb310098a40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fb308009e50 tx=0x7fb30801c960 comp rx=0 tx=0).stop 2024-03-29T08:28:58.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.458+0000 7fb315c00640 1 -- 172.21.15.46:0/1143460334 shutdown_connections 2024-03-29T08:28:58.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.458+0000 7fb315c00640 1 --2- 172.21.15.46:0/1143460334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3100cfc10 0x7fb310098a40 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:58.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.458+0000 7fb315c00640 1 --2- 172.21.15.46:0/1143460334 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb310160060 0x7fb3100cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:58.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.458+0000 7fb315c00640 1 -- 172.21.15.46:0/1143460334 >> 172.21.15.46:0/1143460334 conn(0x7fb3100b21b0 msgr2=0x7fb3100b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.458+0000 7fb315c00640 1 -- 172.21.15.46:0/1143460334 shutdown_connections 2024-03-29T08:28:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.458+0000 7fb315c00640 1 -- 172.21.15.46:0/1143460334 wait complete. 2024-03-29T08:28:58.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb315c00640 1 Processor -- start 2024-03-29T08:28:58.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb315c00640 1 -- start start 2024-03-29T08:28:58.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb315c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3100b1780 0x7fb31015adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:58.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb315c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb3100cfc10 0x7fb3100b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:58.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb315c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb310160060 0x7fb3100b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:58.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb315c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3100b3b90 con 0x7fb3100b1780 2024-03-29T08:28:58.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb315c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb310159d20 con 0x7fb310160060 2024-03-29T08:28:58.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb315c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31015a000 con 0x7fb3100cfc10 2024-03-29T08:28:58.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb30ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3100b1780 0x7fb31015adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:58.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb30ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3100b1780 0x7fb31015adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38950/0 (socket says 172.21.15.46:38950) 2024-03-29T08:28:58.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb30ea00640 1 -- 172.21.15.46:0/1766293152 learned_addr learned my addr 172.21.15.46:0/1766293152 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:58.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb30f400640 1 --2- 172.21.15.46:0/1766293152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb310160060 0x7fb3100b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:58.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb30ea00640 1 -- 172.21.15.46:0/1766293152 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb3100cfc10 msgr2=0x7fb3100b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:58.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb30ea00640 1 --2- 172.21.15.46:0/1766293152 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb3100cfc10 0x7fb3100b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:58.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb30ea00640 1 -- 172.21.15.46:0/1766293152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb310160060 msgr2=0x7fb3100b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:58.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb30ea00640 1 --2- 172.21.15.46:0/1766293152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb310160060 0x7fb3100b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:58.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb30ea00640 1 -- 172.21.15.46:0/1766293152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb308009b30 con 0x7fb3100b1780 2024-03-29T08:28:58.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb30f400640 1 --2- 172.21.15.46:0/1766293152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb310160060 0x7fb3100b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:28:58.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb30ea00640 1 --2- 172.21.15.46:0/1766293152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3100b1780 0x7fb31015adc0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fb30000c7f0 tx=0x7fb30000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:58.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb306a00640 1 -- 172.21.15.46:0/1766293152 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb300007820 con 0x7fb3100b1780 2024-03-29T08:28:58.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb306a00640 1 -- 172.21.15.46:0/1766293152 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb300007980 con 0x7fb3100b1780 2024-03-29T08:28:58.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb315c00640 1 -- 172.21.15.46:0/1766293152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31015a2e0 con 0x7fb3100b1780 2024-03-29T08:28:58.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb306a00640 1 -- 172.21.15.46:0/1766293152 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb300007d30 con 0x7fb3100b1780 2024-03-29T08:28:58.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.462+0000 7fb315c00640 1 -- 172.21.15.46:0/1766293152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31015a8a0 con 0x7fb3100b1780 2024-03-29T08:28:58.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.466+0000 7fb315c00640 1 -- 172.21.15.46:0/1766293152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb31015aac0 con 0x7fb3100b1780 2024-03-29T08:28:58.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.466+0000 7fb306a00640 1 -- 172.21.15.46:0/1766293152 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb3000113f0 con 0x7fb3100b1780 2024-03-29T08:28:58.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.466+0000 7fb306a00640 1 --2- 172.21.15.46:0/1766293152 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb2ec0666c0 0x7fb2ec068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:58.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.466+0000 7fb306a00640 1 -- 172.21.15.46:0/1766293152 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fb300084560 con 0x7fb3100b1780 2024-03-29T08:28:58.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.466+0000 7fb307e00640 1 --2- 172.21.15.46:0/1766293152 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb2ec0666c0 0x7fb2ec068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:58.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.466+0000 7fb307e00640 1 --2- 172.21.15.46:0/1766293152 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb2ec0666c0 0x7fb2ec068b80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8007fd0 tx=0x7fb2f80079a0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:58.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.470+0000 7fb306a00640 1 -- 172.21.15.46:0/1766293152 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb3000517d0 con 0x7fb3100b1780 2024-03-29T08:28:58.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:58.634+0000 7fb315c00640 1 -- 172.21.15.46:0/1766293152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb310066c90 con 0x7fb3100b1780 2024-03-29T08:28:59.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.286+0000 7fb306a00640 1 -- 172.21.15.46:0/1766293152 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb300054df0 con 0x7fb3100b1780 2024-03-29T08:28:59.288 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-03-29T08:28:59.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.286+0000 7fb315c00640 1 -- 172.21.15.46:0/1766293152 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb2ec0666c0 msgr2=0x7fb2ec068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:59.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.286+0000 7fb315c00640 1 --2- 172.21.15.46:0/1766293152 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb2ec0666c0 0x7fb2ec068b80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb2f8007fd0 tx=0x7fb2f80079a0 comp rx=0 tx=0).stop 2024-03-29T08:28:59.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.286+0000 7fb315c00640 1 -- 172.21.15.46:0/1766293152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3100b1780 msgr2=0x7fb31015adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:59.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.286+0000 7fb315c00640 1 --2- 172.21.15.46:0/1766293152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3100b1780 0x7fb31015adc0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fb30000c7f0 tx=0x7fb30000ccb0 comp rx=0 tx=0).stop 2024-03-29T08:28:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.286+0000 7fb315c00640 1 -- 172.21.15.46:0/1766293152 shutdown_connections 2024-03-29T08:28:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.286+0000 7fb315c00640 1 --2- 172.21.15.46:0/1766293152 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb2ec0666c0 0x7fb2ec068b80 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.286+0000 7fb315c00640 1 --2- 172.21.15.46:0/1766293152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb310160060 0x7fb3100b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.286+0000 7fb315c00640 1 --2- 172.21.15.46:0/1766293152 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb3100cfc10 0x7fb3100b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.286+0000 7fb315c00640 1 --2- 172.21.15.46:0/1766293152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3100b1780 0x7fb31015adc0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.286+0000 7fb315c00640 1 -- 172.21.15.46:0/1766293152 >> 172.21.15.46:0/1766293152 conn(0x7fb3100b21b0 msgr2=0x7fb310072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.286+0000 7fb315c00640 1 -- 172.21.15.46:0/1766293152 shutdown_connections 2024-03-29T08:28:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.286+0000 7fb315c00640 1 -- 172.21.15.46:0/1766293152 wait complete. 2024-03-29T08:28:59.314 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T08:28:59.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.458+0000 7f75a2600640 1 Processor -- start 2024-03-29T08:28:59.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.458+0000 7f75a2600640 1 -- start start 2024-03-29T08:28:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.462+0000 7f75a2600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f759c15cc50 0x7f759c161100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.462+0000 7f75a2600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f759c06b470 0x7f759c0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.462+0000 7f75a2600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f759c1606b0 con 0x7f759c0709f0 2024-03-29T08:28:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.462+0000 7f75a2600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f759c15d030 con 0x7f759c15cc50 2024-03-29T08:28:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.462+0000 7f75a2600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759c160c70 con 0x7f759c06b470 2024-03-29T08:28:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.462+0000 7f759aa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f759c06b470 0x7f759c0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.462+0000 7f759be00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f759c0709f0 0x7f759c06d8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:36408/0 (socket says 172.21.15.46:36408) 2024-03-29T08:28:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.462+0000 7f759be00640 1 -- 172.21.15.46:0/1888258606 learned_addr learned my addr 172.21.15.46:0/1888258606 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:28:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.462+0000 7f759b400640 1 --2- 172.21.15.46:0/1888258606 >> v2:172.21.15.73:3300/0 conn(0x7f759c15cc50 0x7f759c161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.462+0000 7f759a000640 1 -- 172.21.15.46:0/1888258606 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2901910370 0 0) 0x7f759c1606b0 con 0x7f759c0709f0 2024-03-29T08:28:59.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.462+0000 7f759a000640 1 -- 172.21.15.46:0/1888258606 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7580003530 con 0x7f759c0709f0 2024-03-29T08:28:59.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.462+0000 7f759aa00640 1 -- 172.21.15.46:0/1888258606 >> v2:172.21.15.73:3300/0 conn(0x7f759c15cc50 msgr2=0x7f759c161100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:28:59.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.462+0000 7f759aa00640 1 --2- 172.21.15.46:0/1888258606 >> v2:172.21.15.73:3300/0 conn(0x7f759c15cc50 0x7f759c161100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:59.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.462+0000 7f759aa00640 1 -- 172.21.15.46:0/1888258606 >> v1:172.21.15.73:6789/0 conn(0x7f759c0709f0 legacy=0x7f759c06d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:59.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.462+0000 7f759aa00640 1 -- 172.21.15.46:0/1888258606 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c0cd190 con 0x7f759c06b470 2024-03-29T08:28:59.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.462+0000 7f759b400640 1 --2- 172.21.15.46:0/1888258606 >> v2:172.21.15.73:3300/0 conn(0x7f759c15cc50 0x7f759c161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:28:59.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f759aa00640 1 --2- 172.21.15.46:0/1888258606 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f759c06b470 0x7f759c0ccc50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f75880097c0 tx=0x7f759c06e7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19e267d3d693db7c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f759a000640 1 -- 172.21.15.46:0/1888258606 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75880ad640 con 0x7f759c06b470 2024-03-29T08:28:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f759a000640 1 -- 172.21.15.46:0/1888258606 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75880ad7a0 con 0x7f759c06b470 2024-03-29T08:28:59.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f759a000640 1 -- 172.21.15.46:0/1888258606 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75880adb30 con 0x7f759c06b470 2024-03-29T08:28:59.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f75a2600640 1 -- 172.21.15.46:0/1888258606 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f759c06b470 msgr2=0x7f759c0ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:59.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f75a2600640 1 --2- 172.21.15.46:0/1888258606 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f759c06b470 0x7f759c0ccc50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f75880097c0 tx=0x7f759c06e7e0 comp rx=0 tx=0).stop 2024-03-29T08:28:59.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f75a2600640 1 -- 172.21.15.46:0/1888258606 shutdown_connections 2024-03-29T08:28:59.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f75a2600640 1 --2- 172.21.15.46:0/1888258606 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f759c06b470 0x7f759c0ccc50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:59.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f75a2600640 1 --2- 172.21.15.46:0/1888258606 >> v2:172.21.15.73:3300/0 conn(0x7f759c15cc50 0x7f759c161100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:59.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f75a2600640 1 -- 172.21.15.46:0/1888258606 >> 172.21.15.46:0/1888258606 conn(0x7f759c0ced50 msgr2=0x7f759c0cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:28:59.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f75a2600640 1 -- 172.21.15.46:0/1888258606 shutdown_connections 2024-03-29T08:28:59.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f75a2600640 1 -- 172.21.15.46:0/1888258606 wait complete. 2024-03-29T08:28:59.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f75a2600640 1 Processor -- start 2024-03-29T08:28:59.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f75a2600640 1 -- start start 2024-03-29T08:28:59.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f75a2600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c06b470 0x7f759c14f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:59.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f75a2600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f759c0709f0 0x7f759c147ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:59.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f75a2600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f759c15cc50 0x7f759c150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:59.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f75a2600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c148800 con 0x7f759c06b470 2024-03-29T08:28:59.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f75a2600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c15f430 con 0x7f759c15cc50 2024-03-29T08:28:59.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.466+0000 7f75a2600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759c15f710 con 0x7f759c0709f0 2024-03-29T08:28:59.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f759b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c06b470 0x7f759c14f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:59.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f759b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c06b470 0x7f759c14f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38962/0 (socket says 172.21.15.46:38962) 2024-03-29T08:28:59.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f759b400640 1 -- 172.21.15.46:0/3709351355 learned_addr learned my addr 172.21.15.46:0/3709351355 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:28:59.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f759aa00640 1 --2- 172.21.15.46:0/3709351355 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f759c0709f0 0x7f759c147ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:59.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f759be00640 1 --2- 172.21.15.46:0/3709351355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f759c15cc50 0x7f759c150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:59.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f759b400640 1 -- 172.21.15.46:0/3709351355 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f759c0709f0 msgr2=0x7f759c147ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:59.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f759b400640 1 --2- 172.21.15.46:0/3709351355 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f759c0709f0 0x7f759c147ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:59.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f759b400640 1 -- 172.21.15.46:0/3709351355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f759c15cc50 msgr2=0x7f759c150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:28:59.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f759b400640 1 --2- 172.21.15.46:0/3709351355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f759c15cc50 0x7f759c150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:28:59.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f759b400640 1 -- 172.21.15.46:0/3709351355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c009420 con 0x7f759c06b470 2024-03-29T08:28:59.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f759b400640 1 --2- 172.21.15.46:0/3709351355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c06b470 0x7f759c14f4c0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f758c00e980 tx=0x7f758c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:59.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f7593400640 1 -- 172.21.15.46:0/3709351355 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c00ca00 con 0x7f759c06b470 2024-03-29T08:28:59.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f7593400640 1 -- 172.21.15.46:0/3709351355 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758c00cb60 con 0x7f759c06b470 2024-03-29T08:28:59.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f7593400640 1 -- 172.21.15.46:0/3709351355 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c013470 con 0x7f759c06b470 2024-03-29T08:28:59.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f75a2600640 1 -- 172.21.15.46:0/3709351355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7588009420 con 0x7f759c06b470 2024-03-29T08:28:59.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f75a2600640 1 -- 172.21.15.46:0/3709351355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759c15fcd0 con 0x7f759c06b470 2024-03-29T08:28:59.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f75a2600640 1 -- 172.21.15.46:0/3709351355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f759c15fec0 con 0x7f759c06b470 2024-03-29T08:28:59.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.470+0000 7f7593400640 1 -- 172.21.15.46:0/3709351355 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f758c0135d0 con 0x7f759c06b470 2024-03-29T08:28:59.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.474+0000 7f7593400640 1 --2- 172.21.15.46:0/3709351355 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f757c066690 0x7f757c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:28:59.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.474+0000 7f7593400640 1 -- 172.21.15.46:0/3709351355 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5443+0+0 (secure 0 0 0) 0x7f758c084370 con 0x7f759c06b470 2024-03-29T08:28:59.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.474+0000 7f759aa00640 1 --2- 172.21.15.46:0/3709351355 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f757c066690 0x7f757c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:28:59.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.474+0000 7f759aa00640 1 --2- 172.21.15.46:0/3709351355 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f757c066690 0x7f757c068b50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f75880af000 tx=0x7f7588004740 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:28:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.478+0000 7f7593400640 1 -- 172.21.15.46:0/3709351355 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f758c0517d0 con 0x7f759c06b470 2024-03-29T08:28:59.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:28:59.642+0000 7f75a2600640 1 -- 172.21.15.46:0/3709351355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f759c066c90 con 0x7f759c06b470 2024-03-29T08:29:00.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.290+0000 7f7593400640 1 -- 172.21.15.46:0/3709351355 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) v1 ==== 169+0+0 (secure 0 0 0) 0x7f758c054df0 con 0x7f759c06b470 2024-03-29T08:29:00.298 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-03-29T08:29:00.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.298+0000 7f75a2600640 1 -- 172.21.15.46:0/3709351355 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f757c066690 msgr2=0x7f757c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:00.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.298+0000 7f75a2600640 1 --2- 172.21.15.46:0/3709351355 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f757c066690 0x7f757c068b50 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f75880af000 tx=0x7f7588004740 comp rx=0 tx=0).stop 2024-03-29T08:29:00.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.298+0000 7f75a2600640 1 -- 172.21.15.46:0/3709351355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c06b470 msgr2=0x7f759c14f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:00.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.298+0000 7f75a2600640 1 --2- 172.21.15.46:0/3709351355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c06b470 0x7f759c14f4c0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f758c00e980 tx=0x7f758c00ee40 comp rx=0 tx=0).stop 2024-03-29T08:29:00.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.298+0000 7f75a2600640 1 -- 172.21.15.46:0/3709351355 shutdown_connections 2024-03-29T08:29:00.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.298+0000 7f75a2600640 1 --2- 172.21.15.46:0/3709351355 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f757c066690 0x7f757c068b50 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.298+0000 7f75a2600640 1 --2- 172.21.15.46:0/3709351355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f759c15cc50 0x7f759c150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.298+0000 7f75a2600640 1 --2- 172.21.15.46:0/3709351355 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f759c0709f0 0x7f759c147ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.298+0000 7f75a2600640 1 --2- 172.21.15.46:0/3709351355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f759c06b470 0x7f759c14f4c0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.298+0000 7f75a2600640 1 -- 172.21.15.46:0/3709351355 >> 172.21.15.46:0/3709351355 conn(0x7f759c0ced50 msgr2=0x7f759c0b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:00.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.298+0000 7f75a2600640 1 -- 172.21.15.46:0/3709351355 shutdown_connections 2024-03-29T08:29:00.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.298+0000 7f75a2600640 1 -- 172.21.15.46:0/3709351355 wait complete. 2024-03-29T08:29:00.328 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-29T08:29:00.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.470+0000 7f9100400640 1 Processor -- start 2024-03-29T08:29:00.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.470+0000 7f9100400640 1 -- start start 2024-03-29T08:29:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.470+0000 7f9100400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f90f815a790 0x7f90f80cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.470+0000 7f9100400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f90f80cbbe0 0x7f90f80d0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.470+0000 7f9100400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f80cc660 con 0x7f90f80cf340 2024-03-29T08:29:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.470+0000 7f9100400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f80cc0a0 con 0x7f90f80cbbe0 2024-03-29T08:29:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.470+0000 7f9100400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f80cc380 con 0x7f90f815a790 2024-03-29T08:29:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.470+0000 7f90fd000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f90f80cbbe0 0x7f90f80d0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:00.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.470+0000 7f90fd000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f90f80cbbe0 0x7f90f80d0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38964/0 (socket says 172.21.15.46:38964) 2024-03-29T08:29:00.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.470+0000 7f90fd000640 1 -- 172.21.15.46:0/3161070741 learned_addr learned my addr 172.21.15.46:0/3161070741 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:00.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.470+0000 7f90fda00640 1 --2- 172.21.15.46:0/3161070741 >> v2:172.21.15.73:3300/0 conn(0x7f90f815a790 0x7f90f80cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:00.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.470+0000 7f90fd000640 1 -- 172.21.15.46:0/3161070741 >> v2:172.21.15.73:3300/0 conn(0x7f90f815a790 msgr2=0x7f90f80cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:00.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.470+0000 7f90fd000640 1 --2- 172.21.15.46:0/3161070741 >> v2:172.21.15.73:3300/0 conn(0x7f90f815a790 0x7f90f80cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.470+0000 7f90fd000640 1 -- 172.21.15.46:0/3161070741 >> v1:172.21.15.73:6789/0 conn(0x7f90f80cf340 legacy=0x7f90f80ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:00.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.470+0000 7f90fd000640 1 -- 172.21.15.46:0/3161070741 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f80b0380 con 0x7f90f80cbbe0 2024-03-29T08:29:00.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.470+0000 7f90fda00640 1 --2- 172.21.15.46:0/3161070741 >> v2:172.21.15.73:3300/0 conn(0x7f90f815a790 0x7f90f80cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:00.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.474+0000 7f90fd000640 1 --2- 172.21.15.46:0/3161070741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f80cbbe0 0x7f90f80d0650 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f90f80b2200 tx=0x7f90e401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4508be016dc2fe58 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:00.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.474+0000 7f90f7e00640 1 -- 172.21.15.46:0/3161070741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e40ab5f0 con 0x7f90f80cbbe0 2024-03-29T08:29:00.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.474+0000 7f90f7e00640 1 -- 172.21.15.46:0/3161070741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e40ab750 con 0x7f90f80cbbe0 2024-03-29T08:29:00.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.474+0000 7f90f7e00640 1 -- 172.21.15.46:0/3161070741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e40abae0 con 0x7f90f80cbbe0 2024-03-29T08:29:00.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.474+0000 7f9100400640 1 -- 172.21.15.46:0/3161070741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f80cbbe0 msgr2=0x7f90f80d0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:00.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.474+0000 7f9100400640 1 --2- 172.21.15.46:0/3161070741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f80cbbe0 0x7f90f80d0650 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f90f80b2200 tx=0x7f90e401c3d0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.474+0000 7f9100400640 1 -- 172.21.15.46:0/3161070741 shutdown_connections 2024-03-29T08:29:00.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.474+0000 7f9100400640 1 --2- 172.21.15.46:0/3161070741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f80cbbe0 0x7f90f80d0650 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.474+0000 7f9100400640 1 --2- 172.21.15.46:0/3161070741 >> v2:172.21.15.73:3300/0 conn(0x7f90f815a790 0x7f90f80cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.474+0000 7f9100400640 1 -- 172.21.15.46:0/3161070741 >> 172.21.15.46:0/3161070741 conn(0x7f90f8099a10 msgr2=0x7f90f8089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:00.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.474+0000 7f9100400640 1 -- 172.21.15.46:0/3161070741 shutdown_connections 2024-03-29T08:29:00.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.474+0000 7f9100400640 1 -- 172.21.15.46:0/3161070741 wait complete. 2024-03-29T08:29:00.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f9100400640 1 Processor -- start 2024-03-29T08:29:00.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f9100400640 1 -- start start 2024-03-29T08:29:00.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f9100400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f80cbbe0 0x7f90f814a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:00.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f9100400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90f80cf340 0x7f90f8152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:00.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f9100400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f90f815a790 0x7f90f8152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:00.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f9100400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f814aa30 con 0x7f90f80cbbe0 2024-03-29T08:29:00.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f9100400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f8152bc0 con 0x7f90f80cf340 2024-03-29T08:29:00.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f9100400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f81521f0 con 0x7f90f815a790 2024-03-29T08:29:00.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f90fda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f80cbbe0 0x7f90f814a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:00.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f90fda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f80cbbe0 0x7f90f814a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38968/0 (socket says 172.21.15.46:38968) 2024-03-29T08:29:00.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f90fda00640 1 -- 172.21.15.46:0/3152036700 learned_addr learned my addr 172.21.15.46:0/3152036700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:00.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f90fd000640 1 --2- 172.21.15.46:0/3152036700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90f80cf340 0x7f90f8152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:00.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f90fe400640 1 --2- 172.21.15.46:0/3152036700 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f90f815a790 0x7f90f8152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:00.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f90fda00640 1 -- 172.21.15.46:0/3152036700 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f90f815a790 msgr2=0x7f90f8152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:00.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f90fda00640 1 --2- 172.21.15.46:0/3152036700 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f90f815a790 0x7f90f8152f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f90fda00640 1 -- 172.21.15.46:0/3152036700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90f80cf340 msgr2=0x7f90f8152680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:00.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f90fda00640 1 --2- 172.21.15.46:0/3152036700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90f80cf340 0x7f90f8152680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f90fda00640 1 -- 172.21.15.46:0/3152036700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e40094d0 con 0x7f90f80cbbe0 2024-03-29T08:29:00.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f90fda00640 1 --2- 172.21.15.46:0/3152036700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f80cbbe0 0x7f90f814a4f0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00c820 tx=0x7f90ec00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:00.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f90f5600640 1 -- 172.21.15.46:0/3152036700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec0079d0 con 0x7f90f80cbbe0 2024-03-29T08:29:00.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f90f5600640 1 -- 172.21.15.46:0/3152036700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ec007b30 con 0x7f90f80cbbe0 2024-03-29T08:29:00.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f9100400640 1 -- 172.21.15.46:0/3152036700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f8153850 con 0x7f90f80cbbe0 2024-03-29T08:29:00.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f90f5600640 1 -- 172.21.15.46:0/3152036700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ec00ce10 con 0x7f90f80cbbe0 2024-03-29T08:29:00.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f9100400640 1 -- 172.21.15.46:0/3152036700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f8151ba0 con 0x7f90f80cbbe0 2024-03-29T08:29:00.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.478+0000 7f9100400640 1 -- 172.21.15.46:0/3152036700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f8151a00 con 0x7f90f80cbbe0 2024-03-29T08:29:00.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.482+0000 7f90f5600640 1 -- 172.21.15.46:0/3152036700 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f90ec0113f0 con 0x7f90f80cbbe0 2024-03-29T08:29:00.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.482+0000 7f90f5600640 1 --2- 172.21.15.46:0/3152036700 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f90d8066690 0x7f90d8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:00.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.482+0000 7f90f5600640 1 -- 172.21.15.46:0/3152036700 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f90ec083f10 con 0x7f90f80cbbe0 2024-03-29T08:29:00.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.482+0000 7f90fd000640 1 --2- 172.21.15.46:0/3152036700 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f90d8066690 0x7f90d8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:00.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.482+0000 7f90fd000640 1 --2- 172.21.15.46:0/3152036700 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f90d8066690 0x7f90d8068b50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f90e40af000 tx=0x7f90e40b6040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:00.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.486+0000 7f90f5600640 1 -- 172.21.15.46:0/3152036700 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f90f8151a00 con 0x7f90f80cbbe0 2024-03-29T08:29:00.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.646+0000 7f9100400640 1 -- 172.21.15.46:0/3152036700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f90f8151a00 con 0x7f90f80cbbe0 2024-03-29T08:29:00.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.650+0000 7f90f5600640 1 -- 172.21.15.46:0/3152036700 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f90f8151a00 con 0x7f90f80cbbe0 2024-03-29T08:29:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.654+0000 7f9100400640 1 -- 172.21.15.46:0/3152036700 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f90d8066690 msgr2=0x7f90d8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.654+0000 7f9100400640 1 --2- 172.21.15.46:0/3152036700 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f90d8066690 0x7f90d8068b50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f90e40af000 tx=0x7f90e40b6040 comp rx=0 tx=0).stop 2024-03-29T08:29:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.654+0000 7f9100400640 1 -- 172.21.15.46:0/3152036700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f80cbbe0 msgr2=0x7f90f814a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.654+0000 7f9100400640 1 --2- 172.21.15.46:0/3152036700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f80cbbe0 0x7f90f814a4f0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00c820 tx=0x7f90ec00cce0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.654+0000 7f9100400640 1 -- 172.21.15.46:0/3152036700 shutdown_connections 2024-03-29T08:29:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.654+0000 7f9100400640 1 --2- 172.21.15.46:0/3152036700 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f90d8066690 0x7f90d8068b50 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.654+0000 7f9100400640 1 --2- 172.21.15.46:0/3152036700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90f80cf340 0x7f90f8152680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.654+0000 7f9100400640 1 --2- 172.21.15.46:0/3152036700 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f90f815a790 0x7f90f8152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.654+0000 7f9100400640 1 --2- 172.21.15.46:0/3152036700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90f80cbbe0 0x7f90f814a4f0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.654+0000 7f9100400640 1 -- 172.21.15.46:0/3152036700 >> 172.21.15.46:0/3152036700 conn(0x7f90f8099a10 msgr2=0x7f90f806f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.654+0000 7f9100400640 1 -- 172.21.15.46:0/3152036700 shutdown_connections 2024-03-29T08:29:00.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.654+0000 7f9100400640 1 -- 172.21.15.46:0/3152036700 wait complete. 2024-03-29T08:29:00.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-03-29T08:29:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f988b000640 1 Processor -- start 2024-03-29T08:29:00.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f988b000640 1 -- start start 2024-03-29T08:29:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f988b000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f988415a940 0x7f98840afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f988b000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f988415ad20 0x7f98840cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f988b000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f98840b0540 0x7f98840cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:00.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f988b000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98840b00d0 con 0x7f988415a940 2024-03-29T08:29:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f988b000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98840b0ab0 con 0x7f98840b0540 2024-03-29T08:29:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f988b000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98840b0d90 con 0x7f988415ad20 2024-03-29T08:29:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f9883e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f988415a940 0x7f98840afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f9883e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f988415a940 0x7f98840afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38972/0 (socket says 172.21.15.46:38972) 2024-03-29T08:29:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f9883e00640 1 -- 172.21.15.46:0/1990013983 learned_addr learned my addr 172.21.15.46:0/1990013983 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f987b400640 1 --2- 172.21.15.46:0/1990013983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f988415ad20 0x7f98840cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f9889000640 1 --2- 172.21.15.46:0/1990013983 >> v2:172.21.15.73:3300/0 conn(0x7f98840b0540 0x7f98840cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f9883e00640 1 -- 172.21.15.46:0/1990013983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f988415ad20 msgr2=0x7f98840cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f9883e00640 1 --2- 172.21.15.46:0/1990013983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f988415ad20 0x7f98840cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f9883e00640 1 -- 172.21.15.46:0/1990013983 >> v2:172.21.15.73:3300/0 conn(0x7f98840b0540 msgr2=0x7f98840cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f9883e00640 1 --2- 172.21.15.46:0/1990013983 >> v2:172.21.15.73:3300/0 conn(0x7f98840b0540 0x7f98840cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f9883e00640 1 -- 172.21.15.46:0/1990013983 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98840b1070 con 0x7f988415a940 2024-03-29T08:29:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f987b400640 1 --2- 172.21.15.46:0/1990013983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f988415ad20 0x7f98840cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:00.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.818+0000 7f9889000640 1 --2- 172.21.15.46:0/1990013983 >> v2:172.21.15.73:3300/0 conn(0x7f98840b0540 0x7f98840cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:00.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.822+0000 7f9883e00640 1 --2- 172.21.15.46:0/1990013983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988415a940 0x7f98840afb90 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f98840c7720 tx=0x7f987401c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3ab443bd39b6f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.822+0000 7f9883400640 1 -- 172.21.15.46:0/1990013983 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98740a5780 con 0x7f988415a940 2024-03-29T08:29:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.822+0000 7f9883400640 1 -- 172.21.15.46:0/1990013983 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98740a58e0 con 0x7f988415a940 2024-03-29T08:29:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.822+0000 7f9883400640 1 -- 172.21.15.46:0/1990013983 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98740a5c70 con 0x7f988415a940 2024-03-29T08:29:00.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.822+0000 7f988b000640 1 -- 172.21.15.46:0/1990013983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988415a940 msgr2=0x7f98840afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.822+0000 7f988b000640 1 --2- 172.21.15.46:0/1990013983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988415a940 0x7f98840afb90 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f98840c7720 tx=0x7f987401c990 comp rx=0 tx=0).stop 2024-03-29T08:29:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.822+0000 7f988b000640 1 -- 172.21.15.46:0/1990013983 shutdown_connections 2024-03-29T08:29:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.822+0000 7f988b000640 1 --2- 172.21.15.46:0/1990013983 >> v2:172.21.15.73:3300/0 conn(0x7f98840b0540 0x7f98840cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.822+0000 7f988b000640 1 --2- 172.21.15.46:0/1990013983 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f988415ad20 0x7f98840cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.822+0000 7f988b000640 1 --2- 172.21.15.46:0/1990013983 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988415a940 0x7f98840afb90 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.822+0000 7f988b000640 1 -- 172.21.15.46:0/1990013983 >> 172.21.15.46:0/1990013983 conn(0x7f9884089fd0 msgr2=0x7f988408a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.822+0000 7f988b000640 1 -- 172.21.15.46:0/1990013983 shutdown_connections 2024-03-29T08:29:00.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.822+0000 7f988b000640 1 -- 172.21.15.46:0/1990013983 wait complete. 2024-03-29T08:29:00.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f988b000640 1 Processor -- start 2024-03-29T08:29:00.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f988b000640 1 -- start start 2024-03-29T08:29:00.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f988b000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98840b0540 0x7f98841583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f988b000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f988415a940 0x7f988406bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f988b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988415ad20 0x7f9884162770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f988b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988406bab0 con 0x7f988415ad20 2024-03-29T08:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f988b000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988406aea0 con 0x7f98840b0540 2024-03-29T08:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f988b000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988406b180 con 0x7f988415a940 2024-03-29T08:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f9889000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988415ad20 0x7f9884162770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f9889000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988415ad20 0x7f9884162770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38988/0 (socket says 172.21.15.46:38988) 2024-03-29T08:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f9889000640 1 -- 172.21.15.46:0/3286767552 learned_addr learned my addr 172.21.15.46:0/3286767552 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f9883e00640 1 --2- 172.21.15.46:0/3286767552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98840b0540 0x7f98841583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f987b400640 1 --2- 172.21.15.46:0/3286767552 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f988415a940 0x7f988406bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f9889000640 1 -- 172.21.15.46:0/3286767552 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f988415a940 msgr2=0x7f988406bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f9889000640 1 --2- 172.21.15.46:0/3286767552 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f988415a940 0x7f988406bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f9889000640 1 -- 172.21.15.46:0/3286767552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98840b0540 msgr2=0x7f98841583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:00.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f9889000640 1 --2- 172.21.15.46:0/3286767552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98840b0540 0x7f98841583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f9889000640 1 -- 172.21.15.46:0/3286767552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9874009b30 con 0x7f988415ad20 2024-03-29T08:29:00.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f9889000640 1 --2- 172.21.15.46:0/3286767552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988415ad20 0x7f9884162770 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f987c00c570 tx=0x7f987c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:00.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f9880c00640 1 -- 172.21.15.46:0/3286767552 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c004440 con 0x7f988415ad20 2024-03-29T08:29:00.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f9880c00640 1 -- 172.21.15.46:0/3286767552 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c003620 con 0x7f988415ad20 2024-03-29T08:29:00.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f9880c00640 1 -- 172.21.15.46:0/3286767552 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c0039d0 con 0x7f988415ad20 2024-03-29T08:29:00.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f988b000640 1 -- 172.21.15.46:0/3286767552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988406b460 con 0x7f988415ad20 2024-03-29T08:29:00.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f988b000640 1 -- 172.21.15.46:0/3286767552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988415a130 con 0x7f988415ad20 2024-03-29T08:29:00.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.826+0000 7f988b000640 1 -- 172.21.15.46:0/3286767552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988406bc20 con 0x7f988415ad20 2024-03-29T08:29:00.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.830+0000 7f9880c00640 1 -- 172.21.15.46:0/3286767552 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f987c003b30 con 0x7f988415ad20 2024-03-29T08:29:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.830+0000 7f9880c00640 1 --2- 172.21.15.46:0/3286767552 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f98600666c0 0x7f9860068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.830+0000 7f9880c00640 1 -- 172.21.15.46:0/3286767552 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f987c083ff0 con 0x7f988415ad20 2024-03-29T08:29:00.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.830+0000 7f9883e00640 1 --2- 172.21.15.46:0/3286767552 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f98600666c0 0x7f9860068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:00.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.830+0000 7f9883e00640 1 --2- 172.21.15.46:0/3286767552 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f98600666c0 0x7f9860068b80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f98740ae000 tx=0x7f98740a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:00.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:00.834+0000 7f9880c00640 1 -- 172.21.15.46:0/3286767552 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f987c0b5980 con 0x7f988415ad20 2024-03-29T08:29:01.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.050+0000 7f988b000640 1 -- 172.21.15.46:0/3286767552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7f9884066c90 con 0x7f988415ad20 2024-03-29T08:29:01.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.290+0000 7f9880c00640 1 -- 172.21.15.46:0/3286767552 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v490) v1 ==== 146+0+0 (secure 0 0 0) 0x7f987c0516e0 con 0x7f988415ad20 2024-03-29T08:29:01.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.294+0000 7f988b000640 1 -- 172.21.15.46:0/3286767552 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f98600666c0 msgr2=0x7f9860068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:01.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.294+0000 7f988b000640 1 --2- 172.21.15.46:0/3286767552 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f98600666c0 0x7f9860068b80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f98740ae000 tx=0x7f98740a7070 comp rx=0 tx=0).stop 2024-03-29T08:29:01.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.294+0000 7f988b000640 1 -- 172.21.15.46:0/3286767552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988415ad20 msgr2=0x7f9884162770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:01.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.294+0000 7f988b000640 1 --2- 172.21.15.46:0/3286767552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988415ad20 0x7f9884162770 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f987c00c570 tx=0x7f987c00ca30 comp rx=0 tx=0).stop 2024-03-29T08:29:01.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.294+0000 7f988b000640 1 -- 172.21.15.46:0/3286767552 shutdown_connections 2024-03-29T08:29:01.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.294+0000 7f988b000640 1 --2- 172.21.15.46:0/3286767552 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f98600666c0 0x7f9860068b80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:01.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.294+0000 7f988b000640 1 --2- 172.21.15.46:0/3286767552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f988415ad20 0x7f9884162770 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:01.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.294+0000 7f988b000640 1 --2- 172.21.15.46:0/3286767552 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f988415a940 0x7f988406bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:01.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.294+0000 7f988b000640 1 --2- 172.21.15.46:0/3286767552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98840b0540 0x7f98841583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:01.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.294+0000 7f988b000640 1 -- 172.21.15.46:0/3286767552 >> 172.21.15.46:0/3286767552 conn(0x7f9884089fd0 msgr2=0x7f98840ad360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:01.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.294+0000 7f988b000640 1 -- 172.21.15.46:0/3286767552 shutdown_connections 2024-03-29T08:29:01.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.294+0000 7f988b000640 1 -- 172.21.15.46:0/3286767552 wait complete. 2024-03-29T08:29:01.318 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-03-29T08:29:01.318 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-03-29T08:29:01.318 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-03-29T08:29:01.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.458+0000 7f568b000640 1 Processor -- start 2024-03-29T08:29:01.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.458+0000 7f568b000640 1 -- start start 2024-03-29T08:29:01.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.458+0000 7f568b000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f568415ada0 0x7f56840ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:01.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.458+0000 7f568b000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f56840ce950 0x7f568406c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:01.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.458+0000 7f568b000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56840cb510 con 0x7f5684144210 2024-03-29T08:29:01.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.458+0000 7f568b000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56840cede0 con 0x7f568415ada0 2024-03-29T08:29:01.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.458+0000 7f568b000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5684070580 con 0x7f56840ce950 2024-03-29T08:29:01.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.458+0000 7f567be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f568415ada0 0x7f56840ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:01.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.458+0000 7f567be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f568415ada0 0x7f56840ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39004/0 (socket says 172.21.15.46:39004) 2024-03-29T08:29:01.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.458+0000 7f567be00640 1 -- 172.21.15.46:0/2457415268 learned_addr learned my addr 172.21.15.46:0/2457415268 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:01.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.458+0000 7f5689000640 1 --2- 172.21.15.46:0/2457415268 >> v2:172.21.15.73:3300/0 conn(0x7f56840ce950 0x7f568406c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:01.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.458+0000 7f567be00640 1 -- 172.21.15.46:0/2457415268 >> v2:172.21.15.73:3300/0 conn(0x7f56840ce950 msgr2=0x7f568406c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:01.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.458+0000 7f567be00640 1 --2- 172.21.15.46:0/2457415268 >> v2:172.21.15.73:3300/0 conn(0x7f56840ce950 0x7f568406c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:01.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.458+0000 7f567be00640 1 -- 172.21.15.46:0/2457415268 >> v1:172.21.15.73:6789/0 conn(0x7f5684144210 legacy=0x7f56840cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:01.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.458+0000 7f567be00640 1 -- 172.21.15.46:0/2457415268 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5684070860 con 0x7f568415ada0 2024-03-29T08:29:01.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.462+0000 7f567be00640 1 --2- 172.21.15.46:0/2457415268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f568415ada0 0x7f56840ce410 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f56840cf170 tx=0x7f566c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c1ba686c64ba45 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:01.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.462+0000 7f5683400640 1 -- 172.21.15.46:0/2457415268 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c0ab610 con 0x7f568415ada0 2024-03-29T08:29:01.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.462+0000 7f5683400640 1 -- 172.21.15.46:0/2457415268 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566c0ab770 con 0x7f568415ada0 2024-03-29T08:29:01.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.462+0000 7f5683400640 1 -- 172.21.15.46:0/2457415268 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566c0abb00 con 0x7f568415ada0 2024-03-29T08:29:01.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.462+0000 7f568b000640 1 -- 172.21.15.46:0/2457415268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f568415ada0 msgr2=0x7f56840ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:01.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.462+0000 7f568b000640 1 --2- 172.21.15.46:0/2457415268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f568415ada0 0x7f56840ce410 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f56840cf170 tx=0x7f566c01c520 comp rx=0 tx=0).stop 2024-03-29T08:29:01.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.462+0000 7f568b000640 1 -- 172.21.15.46:0/2457415268 shutdown_connections 2024-03-29T08:29:01.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.462+0000 7f568b000640 1 --2- 172.21.15.46:0/2457415268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f568415ada0 0x7f56840ce410 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:01.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.462+0000 7f568b000640 1 --2- 172.21.15.46:0/2457415268 >> v2:172.21.15.73:3300/0 conn(0x7f56840ce950 0x7f568406c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:01.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.462+0000 7f568b000640 1 -- 172.21.15.46:0/2457415268 >> 172.21.15.46:0/2457415268 conn(0x7f5684066110 msgr2=0x7f5684069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:01.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.462+0000 7f568b000640 1 -- 172.21.15.46:0/2457415268 shutdown_connections 2024-03-29T08:29:01.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.462+0000 7f568b000640 1 -- 172.21.15.46:0/2457415268 wait complete. 2024-03-29T08:29:01.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f568b000640 1 Processor -- start 2024-03-29T08:29:01.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f568b000640 1 -- start start 2024-03-29T08:29:01.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f568b000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56840ce950 0x7f56841f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:01.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f568b000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5684144210 0x7f56841f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:01.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f568b000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f568415ada0 0x7f56841f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:01.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f568b000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568413dd20 con 0x7f568415ada0 2024-03-29T08:29:01.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f568b000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568413df70 con 0x7f56840ce950 2024-03-29T08:29:01.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f568b000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568413e250 con 0x7f5684144210 2024-03-29T08:29:01.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f5689000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f568415ada0 0x7f56841f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:01.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f5689000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f568415ada0 0x7f56841f7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39018/0 (socket says 172.21.15.46:39018) 2024-03-29T08:29:01.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f5689000640 1 -- 172.21.15.46:0/2672989124 learned_addr learned my addr 172.21.15.46:0/2672989124 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:01.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f567be00640 1 --2- 172.21.15.46:0/2672989124 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5684144210 0x7f56841f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:01.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f5683e00640 1 --2- 172.21.15.46:0/2672989124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56840ce950 0x7f56841f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:01.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f5689000640 1 -- 172.21.15.46:0/2672989124 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5684144210 msgr2=0x7f56841f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:01.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f5689000640 1 --2- 172.21.15.46:0/2672989124 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5684144210 0x7f56841f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:01.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f5689000640 1 -- 172.21.15.46:0/2672989124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56840ce950 msgr2=0x7f56841f88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:01.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f5689000640 1 --2- 172.21.15.46:0/2672989124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56840ce950 0x7f56841f88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:01.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f5689000640 1 -- 172.21.15.46:0/2672989124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566c0094d0 con 0x7f568415ada0 2024-03-29T08:29:01.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f5689000640 1 --2- 172.21.15.46:0/2672989124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f568415ada0 0x7f56841f7c50 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f567c007e40 tx=0x7f567c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:01.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f5680c00640 1 -- 172.21.15.46:0/2672989124 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c014620 con 0x7f568415ada0 2024-03-29T08:29:01.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f5680c00640 1 -- 172.21.15.46:0/2672989124 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c00ee90 con 0x7f568415ada0 2024-03-29T08:29:01.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f568b000640 1 -- 172.21.15.46:0/2672989124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568413e530 con 0x7f568415ada0 2024-03-29T08:29:01.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f5680c00640 1 -- 172.21.15.46:0/2672989124 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c0149e0 con 0x7f568415ada0 2024-03-29T08:29:01.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f568b000640 1 -- 172.21.15.46:0/2672989124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56840b3490 con 0x7f568415ada0 2024-03-29T08:29:01.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.466+0000 7f568b000640 1 -- 172.21.15.46:0/2672989124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56840ce280 con 0x7f568415ada0 2024-03-29T08:29:01.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.470+0000 7f5680c00640 1 -- 172.21.15.46:0/2672989124 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f567c014b40 con 0x7f568415ada0 2024-03-29T08:29:01.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.470+0000 7f5680c00640 1 --2- 172.21.15.46:0/2672989124 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5660066690 0x7f5660068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:01.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.470+0000 7f5680c00640 1 -- 172.21.15.46:0/2672989124 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f567c0847f0 con 0x7f568415ada0 2024-03-29T08:29:01.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.470+0000 7f5683e00640 1 --2- 172.21.15.46:0/2672989124 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5660066690 0x7f5660068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:01.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.470+0000 7f5683e00640 1 --2- 172.21.15.46:0/2672989124 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5660066690 0x7f5660068b50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5674007fd0 tx=0x7f5674009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:01.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.474+0000 7f5680c00640 1 -- 172.21.15.46:0/2672989124 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f567c051e10 con 0x7f568415ada0 2024-03-29T08:29:01.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:01.694+0000 7f568b000640 1 -- 172.21.15.46:0/2672989124 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7f5684066c90 con 0x7f568415ada0 2024-03-29T08:29:02.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.298+0000 7f5680c00640 1 -- 172.21.15.46:0/2672989124 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v491) v1 ==== 147+0+0 (secure 0 0 0) 0x7f567c055430 con 0x7f568415ada0 2024-03-29T08:29:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.298+0000 7f568b000640 1 -- 172.21.15.46:0/2672989124 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5660066690 msgr2=0x7f5660068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.298+0000 7f568b000640 1 --2- 172.21.15.46:0/2672989124 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5660066690 0x7f5660068b50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5674007fd0 tx=0x7f5674009040 comp rx=0 tx=0).stop 2024-03-29T08:29:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.298+0000 7f568b000640 1 -- 172.21.15.46:0/2672989124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f568415ada0 msgr2=0x7f56841f7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.298+0000 7f568b000640 1 --2- 172.21.15.46:0/2672989124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f568415ada0 0x7f56841f7c50 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f567c007e40 tx=0x7f567c00e4b0 comp rx=0 tx=0).stop 2024-03-29T08:29:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.298+0000 7f568b000640 1 -- 172.21.15.46:0/2672989124 shutdown_connections 2024-03-29T08:29:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.298+0000 7f568b000640 1 --2- 172.21.15.46:0/2672989124 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5660066690 0x7f5660068b50 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.298+0000 7f568b000640 1 --2- 172.21.15.46:0/2672989124 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f568415ada0 0x7f56841f7c50 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.298+0000 7f568b000640 1 --2- 172.21.15.46:0/2672989124 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5684144210 0x7f56841f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:02.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.298+0000 7f568b000640 1 --2- 172.21.15.46:0/2672989124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56840ce950 0x7f56841f88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.298+0000 7f568b000640 1 -- 172.21.15.46:0/2672989124 >> 172.21.15.46:0/2672989124 conn(0x7f5684066110 msgr2=0x7f5684140f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.298+0000 7f568b000640 1 -- 172.21.15.46:0/2672989124 shutdown_connections 2024-03-29T08:29:02.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.298+0000 7f568b000640 1 -- 172.21.15.46:0/2672989124 wait complete. 2024-03-29T08:29:02.324 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-29T08:29:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.466+0000 7f08dd600640 1 Processor -- start 2024-03-29T08:29:02.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.466+0000 7f08dd600640 1 -- start start 2024-03-29T08:29:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08dd600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08d815cb00 0x7f08d80cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08dd600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d815cee0 con 0x7f08d806bc30 2024-03-29T08:29:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08dd600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d80ce760 con 0x7f08d80b00a0 2024-03-29T08:29:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08dd600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d80b14a0 con 0x7f08d815cb00 2024-03-29T08:29:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08dcc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f08d806bc30 0x7f08d80cec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55776/0 (socket says 172.21.15.46:55776) 2024-03-29T08:29:02.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08dcc00640 1 -- 172.21.15.46:0/1411750343 learned_addr learned my addr 172.21.15.46:0/1411750343 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7e00640 1 --2- 172.21.15.46:0/1411750343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08d815cb00 0x7f08d80cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7400640 1 -- 172.21.15.46:0/1411750343 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226452396 0 0) 0x7f08d815cee0 con 0x7f08d806bc30 2024-03-29T08:29:02.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7400640 1 -- 172.21.15.46:0/1411750343 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08bc003530 con 0x7f08d806bc30 2024-03-29T08:29:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7400640 1 -- 172.21.15.46:0/1411750343 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4277596030 0 0) 0x7f08d80ce760 con 0x7f08d80b00a0 2024-03-29T08:29:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7400640 1 -- 172.21.15.46:0/1411750343 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08d815cee0 con 0x7f08d80b00a0 2024-03-29T08:29:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7400640 1 -- 172.21.15.46:0/1411750343 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 377622738 0 0) 0x7f08bc003530 con 0x7f08d806bc30 2024-03-29T08:29:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7400640 1 -- 172.21.15.46:0/1411750343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08d815cb00 msgr2=0x7f08d80cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7400640 1 --2- 172.21.15.46:0/1411750343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08d815cb00 0x7f08d80cd7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7400640 1 -- 172.21.15.46:0/1411750343 >> v1:172.21.15.73:6789/0 conn(0x7f08d80b00a0 legacy=0x7f08d806b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7400640 1 -- 172.21.15.46:0/1411750343 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d80b0920 con 0x7f08d806bc30 2024-03-29T08:29:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7400640 1 -- 172.21.15.46:0/1411750343 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f08d0001a00 con 0x7f08d806bc30 2024-03-29T08:29:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7400640 1 -- 172.21.15.46:0/1411750343 >> v1:172.21.15.46:6789/0 conn(0x7f08d806bc30 legacy=0x7f08d80cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7400640 1 --2- 172.21.15.46:0/1411750343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08bc003b40 0x7f08bc003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7400640 1 --2- 172.21.15.46:0/1411750343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08d815cb00 0x7f08bc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7400640 1 --2- 172.21.15.46:0/1411750343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08bc004d10 0x7f08bc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:02.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7400640 1 -- 172.21.15.46:0/1411750343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d80b14a0 con 0x7f08bc003b40 2024-03-29T08:29:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7400640 1 -- 172.21.15.46:0/1411750343 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc00ba40 con 0x7f08bc004d10 2024-03-29T08:29:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7400640 1 -- 172.21.15.46:0/1411750343 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc00bbb0 con 0x7f08d815cb00 2024-03-29T08:29:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7e00640 1 --2- 172.21.15.46:0/1411750343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08bc003b40 0x7f08bc003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08cf400640 1 --2- 172.21.15.46:0/1411750343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08d815cb00 0x7f08bc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08dcc00640 1 --2- 172.21.15.46:0/1411750343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08bc004d10 0x7f08bc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7e00640 1 -- 172.21.15.46:0/1411750343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08d815cb00 msgr2=0x7f08bc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7e00640 1 --2- 172.21.15.46:0/1411750343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08d815cb00 0x7f08bc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7e00640 1 -- 172.21.15.46:0/1411750343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08bc004d10 msgr2=0x7f08bc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7e00640 1 --2- 172.21.15.46:0/1411750343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08bc004d10 0x7f08bc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:02.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.470+0000 7f08d7e00640 1 -- 172.21.15.46:0/1411750343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08bc003530 con 0x7f08bc003b40 2024-03-29T08:29:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.474+0000 7f08d7e00640 1 --2- 172.21.15.46:0/1411750343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08bc003b40 0x7f08bc003f40 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009ae0 tx=0x7f08c801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceee9e3e6bf71e51 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:02.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.474+0000 7f08d7400640 1 -- 172.21.15.46:0/1411750343 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c80a5520 con 0x7f08bc003b40 2024-03-29T08:29:02.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.474+0000 7f08d7400640 1 -- 172.21.15.46:0/1411750343 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c80a5680 con 0x7f08bc003b40 2024-03-29T08:29:02.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.474+0000 7f08d7400640 1 -- 172.21.15.46:0/1411750343 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c80a5a30 con 0x7f08bc003b40 2024-03-29T08:29:02.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.474+0000 7f08dd600640 1 -- 172.21.15.46:0/1411750343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08bc003b40 msgr2=0x7f08bc003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:02.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.474+0000 7f08dd600640 1 --2- 172.21.15.46:0/1411750343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08bc003b40 0x7f08bc003f40 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009ae0 tx=0x7f08c801ca40 comp rx=0 tx=0).stop 2024-03-29T08:29:02.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.474+0000 7f08dcc00640 1 -- 172.21.15.46:0/1411750343 reap_dead start 2024-03-29T08:29:02.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.474+0000 7f08dd600640 1 -- 172.21.15.46:0/1411750343 shutdown_connections 2024-03-29T08:29:02.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.474+0000 7f08dd600640 1 -- 172.21.15.46:0/1411750343 >> 172.21.15.46:0/1411750343 conn(0x7f08d80d0520 msgr2=0x7f08d80d0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:02.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.474+0000 7f08dd600640 1 -- 172.21.15.46:0/1411750343 shutdown_connections 2024-03-29T08:29:02.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.474+0000 7f08dd600640 1 -- 172.21.15.46:0/1411750343 wait complete. 2024-03-29T08:29:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.474+0000 7f08dd600640 1 Processor -- start 2024-03-29T08:29:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.474+0000 7f08dd600640 1 -- start start 2024-03-29T08:29:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08dd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08bc003b40 0x7f08d8159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08dd600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08d80c4b50 0x7f08d8098fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08dd600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08d80c4fb0 0x7f08d8099520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08dd600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d80c7f00 con 0x7f08bc003b40 2024-03-29T08:29:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08dd600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d80c5a00 con 0x7f08d80c4fb0 2024-03-29T08:29:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08dd600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d80c5ce0 con 0x7f08d80c4b50 2024-03-29T08:29:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08d7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08bc003b40 0x7f08d8159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08d7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08bc003b40 0x7f08d8159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39032/0 (socket says 172.21.15.46:39032) 2024-03-29T08:29:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08d7e00640 1 -- 172.21.15.46:0/3204531837 learned_addr learned my addr 172.21.15.46:0/3204531837 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08cf400640 1 --2- 172.21.15.46:0/3204531837 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08d80c4b50 0x7f08d8098fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08dcc00640 1 --2- 172.21.15.46:0/3204531837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08d80c4fb0 0x7f08d8099520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08d7e00640 1 -- 172.21.15.46:0/3204531837 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08d80c4b50 msgr2=0x7f08d8098fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08d7e00640 1 --2- 172.21.15.46:0/3204531837 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08d80c4b50 0x7f08d8098fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08d7e00640 1 -- 172.21.15.46:0/3204531837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08d80c4fb0 msgr2=0x7f08d8099520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08d7e00640 1 --2- 172.21.15.46:0/3204531837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08d80c4fb0 0x7f08d8099520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08d7e00640 1 -- 172.21.15.46:0/3204531837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d80c5fc0 con 0x7f08bc003b40 2024-03-29T08:29:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08d7e00640 1 --2- 172.21.15.46:0/3204531837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08bc003b40 0x7f08d8159cf0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009ae0 tx=0x7f08bc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08d4c00640 1 -- 172.21.15.46:0/3204531837 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c80be070 con 0x7f08bc003b40 2024-03-29T08:29:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08d4c00640 1 -- 172.21.15.46:0/3204531837 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c80a5e70 con 0x7f08bc003b40 2024-03-29T08:29:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08d4c00640 1 -- 172.21.15.46:0/3204531837 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c80a5520 con 0x7f08bc003b40 2024-03-29T08:29:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08dd600640 1 -- 172.21.15.46:0/3204531837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d80c7ab0 con 0x7f08bc003b40 2024-03-29T08:29:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08dd600640 1 -- 172.21.15.46:0/3204531837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d80c5780 con 0x7f08bc003b40 2024-03-29T08:29:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08dd600640 1 -- 172.21.15.46:0/3204531837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d8066960 con 0x7f08bc003b40 2024-03-29T08:29:02.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.478+0000 7f08d4c00640 1 -- 172.21.15.46:0/3204531837 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f08c80b4040 con 0x7f08bc003b40 2024-03-29T08:29:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.482+0000 7f08d4c00640 1 --2- 172.21.15.46:0/3204531837 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f08b4066690 0x7f08b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.482+0000 7f08d4c00640 1 -- 172.21.15.46:0/3204531837 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f08c811d9f0 con 0x7f08bc003b40 2024-03-29T08:29:02.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.482+0000 7f08cf400640 1 --2- 172.21.15.46:0/3204531837 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f08b4066690 0x7f08b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:02.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.482+0000 7f08cf400640 1 --2- 172.21.15.46:0/3204531837 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f08b4066690 0x7f08b4068b50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f08d80ca920 tx=0x7f08c0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:02.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.486+0000 7f08d4c00640 1 -- 172.21.15.46:0/3204531837 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f08c80eb080 con 0x7f08bc003b40 2024-03-29T08:29:02.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.646+0000 7f08dd600640 1 -- 172.21.15.46:0/3204531837 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f08d804e040 con 0x7f08bc003b40 2024-03-29T08:29:02.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.650+0000 7f08d4c00640 1 -- 172.21.15.46:0/3204531837 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f08c80ee6a0 con 0x7f08bc003b40 2024-03-29T08:29:02.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.650+0000 7f08dd600640 1 -- 172.21.15.46:0/3204531837 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f08b4066690 msgr2=0x7f08b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:02.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.650+0000 7f08dd600640 1 --2- 172.21.15.46:0/3204531837 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f08b4066690 0x7f08b4068b50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f08d80ca920 tx=0x7f08c0009040 comp rx=0 tx=0).stop 2024-03-29T08:29:02.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.650+0000 7f08dd600640 1 -- 172.21.15.46:0/3204531837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08bc003b40 msgr2=0x7f08d8159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:02.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.650+0000 7f08dd600640 1 --2- 172.21.15.46:0/3204531837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08bc003b40 0x7f08d8159cf0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009ae0 tx=0x7f08bc005800 comp rx=0 tx=0).stop 2024-03-29T08:29:02.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.650+0000 7f08dd600640 1 -- 172.21.15.46:0/3204531837 shutdown_connections 2024-03-29T08:29:02.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.650+0000 7f08dd600640 1 --2- 172.21.15.46:0/3204531837 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f08b4066690 0x7f08b4068b50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:02.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.650+0000 7f08dd600640 1 --2- 172.21.15.46:0/3204531837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08d80c4fb0 0x7f08d8099520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:02.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.650+0000 7f08dd600640 1 --2- 172.21.15.46:0/3204531837 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f08d80c4b50 0x7f08d8098fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:02.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.650+0000 7f08dd600640 1 --2- 172.21.15.46:0/3204531837 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08bc003b40 0x7f08d8159cf0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:02.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.650+0000 7f08dd600640 1 -- 172.21.15.46:0/3204531837 >> 172.21.15.46:0/3204531837 conn(0x7f08d80d0520 msgr2=0x7f08d80adaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:02.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.650+0000 7f08dd600640 1 -- 172.21.15.46:0/3204531837 shutdown_connections 2024-03-29T08:29:02.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:02.650+0000 7f08dd600640 1 -- 172.21.15.46:0/3204531837 wait complete. 2024-03-29T08:29:02.675 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T08:29:02.675 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:29:02.766 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:29:02.811 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T08:29:02.811 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:29:02.811 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:29:02.820 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:29:02.820 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:29:02.820 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:29:02.866 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:29:02.867 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:29:02.867 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:29:02.875 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:29:02.875 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:29:02.875 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:29:02.921 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:29:02.985 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-03-29T08:29:02.986 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:29:03.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.122+0000 7f7eebe00640 1 Processor -- start 2024-03-29T08:29:03.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.126+0000 7f7eebe00640 1 -- start start 2024-03-29T08:29:03.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.126+0000 7f7eebe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7eec0cce70 0x7f7eec06b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.126+0000 7f7eebe00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7eec0cd250 0x7f7eec0cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.126+0000 7f7eebe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eec160940 con 0x7f7eec0cf3b0 2024-03-29T08:29:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.126+0000 7f7eebe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eec160c20 con 0x7f7eec0cce70 2024-03-29T08:29:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.126+0000 7f7eebe00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eec160f00 con 0x7f7eec0cd250 2024-03-29T08:29:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.126+0000 7f7eeaa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7eec0cce70 0x7f7eec06b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.126+0000 7f7eeb400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7eec0cf3b0 0x7f7eec0b0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55792/0 (socket says 172.21.15.46:55792) 2024-03-29T08:29:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.126+0000 7f7eeb400640 1 -- 172.21.15.46:0/3050902309 learned_addr learned my addr 172.21.15.46:0/3050902309 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.126+0000 7f7ee3e00640 1 --2- 172.21.15.46:0/3050902309 >> v2:172.21.15.73:3300/0 conn(0x7f7eec0cd250 0x7f7eec0cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.126+0000 7f7eea000640 1 -- 172.21.15.46:0/3050902309 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1421843083 0 0) 0x7f7eec160940 con 0x7f7eec0cf3b0 2024-03-29T08:29:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.126+0000 7f7eea000640 1 -- 172.21.15.46:0/3050902309 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ed0003530 con 0x7f7eec0cf3b0 2024-03-29T08:29:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.126+0000 7f7eeaa00640 1 -- 172.21.15.46:0/3050902309 >> v2:172.21.15.73:3300/0 conn(0x7f7eec0cd250 msgr2=0x7f7eec0cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.126+0000 7f7eeaa00640 1 --2- 172.21.15.46:0/3050902309 >> v2:172.21.15.73:3300/0 conn(0x7f7eec0cd250 0x7f7eec0cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.126+0000 7f7eeaa00640 1 -- 172.21.15.46:0/3050902309 >> v1:172.21.15.46:6789/0 conn(0x7f7eec0cf3b0 legacy=0x7f7eec0b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:03.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.126+0000 7f7eeaa00640 1 -- 172.21.15.46:0/3050902309 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec1611e0 con 0x7f7eec0cce70 2024-03-29T08:29:03.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.130+0000 7f7eeaa00640 1 --2- 172.21.15.46:0/3050902309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7eec0cce70 0x7f7eec06b3b0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f7eec06b1a0 tx=0x7f7edc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfc073da2913f85 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.130+0000 7f7eea000640 1 -- 172.21.15.46:0/3050902309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec0cd840 con 0x7f7eec0cce70 2024-03-29T08:29:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.130+0000 7f7eea000640 1 -- 172.21.15.46:0/3050902309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7edc0ab610 con 0x7f7eec0cce70 2024-03-29T08:29:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.130+0000 7f7eea000640 1 -- 172.21.15.46:0/3050902309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc0ab9a0 con 0x7f7eec0cce70 2024-03-29T08:29:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.130+0000 7f7eebe00640 1 -- 172.21.15.46:0/3050902309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7eec0cce70 msgr2=0x7f7eec06b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.130+0000 7f7eebe00640 1 --2- 172.21.15.46:0/3050902309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7eec0cce70 0x7f7eec06b3b0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f7eec06b1a0 tx=0x7f7edc01c3f0 comp rx=0 tx=0).stop 2024-03-29T08:29:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.130+0000 7f7eebe00640 1 -- 172.21.15.46:0/3050902309 shutdown_connections 2024-03-29T08:29:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.130+0000 7f7eebe00640 1 --2- 172.21.15.46:0/3050902309 >> v2:172.21.15.73:3300/0 conn(0x7f7eec0cd250 0x7f7eec0cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.130+0000 7f7eebe00640 1 --2- 172.21.15.46:0/3050902309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7eec0cce70 0x7f7eec06b3b0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.130+0000 7f7eebe00640 1 -- 172.21.15.46:0/3050902309 >> 172.21.15.46:0/3050902309 conn(0x7f7eec0d0560 msgr2=0x7f7eec0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.130+0000 7f7eebe00640 1 -- 172.21.15.46:0/3050902309 shutdown_connections 2024-03-29T08:29:03.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.130+0000 7f7eebe00640 1 -- 172.21.15.46:0/3050902309 wait complete. 2024-03-29T08:29:03.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.130+0000 7f7eebe00640 1 Processor -- start 2024-03-29T08:29:03.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.130+0000 7f7eebe00640 1 -- start start 2024-03-29T08:29:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eebe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7eec0cce70 0x7f7eec1591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eebe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eec0cd250 0x7f7eec157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eebe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7eec0cf3b0 0x7f7eec153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eebe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec15ea50 con 0x7f7eec0cf3b0 2024-03-29T08:29:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eebe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec0b5f70 con 0x7f7eec0cd250 2024-03-29T08:29:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eebe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eec0b59b0 con 0x7f7eec0cce70 2024-03-29T08:29:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eeb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7eec0cf3b0 0x7f7eec153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eeb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7eec0cf3b0 0x7f7eec153140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39060/0 (socket says 172.21.15.46:39060) 2024-03-29T08:29:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eeb400640 1 -- 172.21.15.46:0/1430879486 learned_addr learned my addr 172.21.15.46:0/1430879486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eeaa00640 1 --2- 172.21.15.46:0/1430879486 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7eec0cce70 0x7f7eec1591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7ee3e00640 1 --2- 172.21.15.46:0/1430879486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eec0cd250 0x7f7eec157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eeb400640 1 -- 172.21.15.46:0/1430879486 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7eec0cce70 msgr2=0x7f7eec1591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:03.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eeb400640 1 --2- 172.21.15.46:0/1430879486 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7eec0cce70 0x7f7eec1591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:03.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eeb400640 1 -- 172.21.15.46:0/1430879486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eec0cd250 msgr2=0x7f7eec157560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:03.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eeb400640 1 --2- 172.21.15.46:0/1430879486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eec0cd250 0x7f7eec157560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:03.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eeb400640 1 -- 172.21.15.46:0/1430879486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc009420 con 0x7f7eec0cf3b0 2024-03-29T08:29:03.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eeaa00640 1 --2- 172.21.15.46:0/1430879486 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7eec0cce70 0x7f7eec1591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:03.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7ee3e00640 1 --2- 172.21.15.46:0/1430879486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eec0cd250 0x7f7eec157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:03.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eeb400640 1 --2- 172.21.15.46:0/1430879486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7eec0cf3b0 0x7f7eec153140 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f7ee400caa0 tx=0x7f7ee400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:03.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7ee2a00640 1 -- 172.21.15.46:0/1430879486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee400ac00 con 0x7f7eec0cf3b0 2024-03-29T08:29:03.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7ee2a00640 1 -- 172.21.15.46:0/1430879486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee400ad60 con 0x7f7eec0cf3b0 2024-03-29T08:29:03.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7ee2a00640 1 -- 172.21.15.46:0/1430879486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee4012600 con 0x7f7eec0cf3b0 2024-03-29T08:29:03.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eebe00640 1 -- 172.21.15.46:0/1430879486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eec0b5c90 con 0x7f7eec0cf3b0 2024-03-29T08:29:03.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eebe00640 1 -- 172.21.15.46:0/1430879486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eec15ef40 con 0x7f7eec0cf3b0 2024-03-29T08:29:03.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7eebe00640 1 -- 172.21.15.46:0/1430879486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eec15f160 con 0x7f7eec0cf3b0 2024-03-29T08:29:03.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.134+0000 7f7ee2a00640 1 -- 172.21.15.46:0/1430879486 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7ee4007070 con 0x7f7eec0cf3b0 2024-03-29T08:29:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.138+0000 7f7ee2a00640 1 --2- 172.21.15.46:0/1430879486 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ec8066690 0x7f7ec8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.138+0000 7f7ee2a00640 1 -- 172.21.15.46:0/1430879486 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f7ee4083c70 con 0x7f7eec0cf3b0 2024-03-29T08:29:03.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.138+0000 7f7eeaa00640 1 --2- 172.21.15.46:0/1430879486 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ec8066690 0x7f7ec8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:03.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.138+0000 7f7eeaa00640 1 --2- 172.21.15.46:0/1430879486 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ec8066690 0x7f7ec8068b50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0af000 tx=0x7f7edc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:03.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.142+0000 7f7ee2a00640 1 -- 172.21.15.46:0/1430879486 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7ee40513c0 con 0x7f7eec0cf3b0 2024-03-29T08:29:03.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.338+0000 7f7eebe00640 1 -- 172.21.15.46:0/1430879486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7eec04e040 con 0x7f7eec0cf3b0 2024-03-29T08:29:03.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.338+0000 7f7ee2a00640 1 -- 172.21.15.46:0/1430879486 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f7ee4017090 con 0x7f7eec0cf3b0 2024-03-29T08:29:03.343 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:03.343 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":87,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4230030957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4230030957},{"type":"v1","addr":"172.21.15.73:6836","nonce":4230030957}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7175,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/297138217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":297138217},{"type":"v1","addr":"172.21.15.73:6837","nonce":297138217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":7963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3856305332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3856305332},{"type":"v1","addr":"172.21.15.46:6833","nonce":3856305332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":8895,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3142618727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3142618727},{"type":"v1","addr":"172.21.15.46:6835","nonce":3142618727}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":85}],"filesystems":[]} 2024-03-29T08:29:03.343 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 87 2024-03-29T08:29:03.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.342+0000 7f7eebe00640 1 -- 172.21.15.46:0/1430879486 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ec8066690 msgr2=0x7f7ec8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:03.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.342+0000 7f7eebe00640 1 --2- 172.21.15.46:0/1430879486 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ec8066690 0x7f7ec8068b50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0af000 tx=0x7f7edc008040 comp rx=0 tx=0).stop 2024-03-29T08:29:03.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.342+0000 7f7eebe00640 1 -- 172.21.15.46:0/1430879486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7eec0cf3b0 msgr2=0x7f7eec153140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:03.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.342+0000 7f7eebe00640 1 --2- 172.21.15.46:0/1430879486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7eec0cf3b0 0x7f7eec153140 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f7ee400caa0 tx=0x7f7ee400cf60 comp rx=0 tx=0).stop 2024-03-29T08:29:03.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.342+0000 7f7eebe00640 1 -- 172.21.15.46:0/1430879486 shutdown_connections 2024-03-29T08:29:03.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.342+0000 7f7eebe00640 1 --2- 172.21.15.46:0/1430879486 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7ec8066690 0x7f7ec8068b50 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:03.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.342+0000 7f7eebe00640 1 --2- 172.21.15.46:0/1430879486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eec0cd250 0x7f7eec157560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.342+0000 7f7eebe00640 1 --2- 172.21.15.46:0/1430879486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7eec0cf3b0 0x7f7eec153140 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.342+0000 7f7eebe00640 1 --2- 172.21.15.46:0/1430879486 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7eec0cce70 0x7f7eec1591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.342+0000 7f7eebe00640 1 -- 172.21.15.46:0/1430879486 >> 172.21.15.46:0/1430879486 conn(0x7f7eec0d0560 msgr2=0x7f7eec0b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.342+0000 7f7eebe00640 1 -- 172.21.15.46:0/1430879486 shutdown_connections 2024-03-29T08:29:03.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:03.342+0000 7f7eebe00640 1 -- 172.21.15.46:0/1430879486 wait complete. 2024-03-29T08:29:03.364 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-29T08:29:03.364 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-29T08:29:03.365 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T08:29:03.365 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:29:03.365 INFO:tasks.ceph.mds.c:Restarting daemon 2024-03-29T08:29:03.365 INFO:tasks.ceph.mds.c:Stopping old one... 2024-03-29T08:29:03.365 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T08:29:03.365 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:29:03.365 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-29T08:29:03.365 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-29T08:29:03.366 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T08:29:03.366 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:29:03.366 INFO:tasks.ceph.mds.d:Restarting daemon 2024-03-29T08:29:03.366 INFO:tasks.ceph.mds.d:Stopping old one... 2024-03-29T08:29:03.366 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T08:29:03.366 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:29:03.380 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T08:29:03.380 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-03-29T08:29:03.382 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T08:29:03.382 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-03-29T08:29:03.383 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T08:29:03.383 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-29T08:29:03.429 INFO:tasks.ceph.mds.c:Started 2024-03-29T08:29:03.430 DEBUG:teuthology.parallel:result is None 2024-03-29T08:29:03.433 INFO:tasks.ceph.mds.d:Started 2024-03-29T08:29:03.433 DEBUG:teuthology.parallel:result is None 2024-03-29T08:29:03.434 INFO:tasks.ceph.mds.b:Started 2024-03-29T08:29:03.435 DEBUG:teuthology.parallel:result is None 2024-03-29T08:29:03.513 INFO:tasks.ceph.mds.c.smithi046.stdout:starting mds.c at 2024-03-29T08:29:03.526 INFO:tasks.ceph.mds.b.smithi073.stdout:starting mds.b at 2024-03-29T08:29:03.526 INFO:tasks.ceph.mds.d.smithi073.stdout:starting mds.d at 2024-03-29T08:29:03.585 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-03-29T08:29:03.585 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-03-29T08:29:09.380 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T08:29:09.380 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-29T08:29:09.384 INFO:tasks.ceph.mds.a:Started 2024-03-29T08:29:09.384 DEBUG:teuthology.parallel:result is None 2024-03-29T08:29:09.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-29T08:29:09.480 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-03-29T08:29:09.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.534+0000 7f5fca800640 1 Processor -- start 2024-03-29T08:29:09.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.534+0000 7f5fca800640 1 -- start start 2024-03-29T08:29:09.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.538+0000 7f5fca800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5fc4158300 0x7f5fc406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:09.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.538+0000 7f5fca800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc406c390 con 0x7f5fc40af530 2024-03-29T08:29:09.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.538+0000 7f5fca800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc406d600 con 0x7f5fc40709b0 2024-03-29T08:29:09.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.538+0000 7f5fca800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc40afff0 con 0x7f5fc4158300 2024-03-29T08:29:09.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.538+0000 7f5fc3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5fc4158300 0x7f5fc406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:09.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.538+0000 7f5fc3400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5fc4158300 0x7f5fc406d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33508/0 (socket says 172.21.15.46:33508) 2024-03-29T08:29:09.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.538+0000 7f5fc3400640 1 -- 172.21.15.46:0/4169136462 learned_addr learned my addr 172.21.15.46:0/4169136462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:09.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.538+0000 7f5fbaa00640 1 --1- 172.21.15.46:0/4169136462 >> v1:172.21.15.46:6789/0 conn(0x7f5fc40af530 0x7f5fc40702a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59166/0 (socket says 172.21.15.46:59166) 2024-03-29T08:29:09.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.538+0000 7f5fc3400640 1 -- 172.21.15.46:0/4169136462 >> v1:172.21.15.73:6789/0 conn(0x7f5fc40709b0 legacy=0x7f5fc415bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:09.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.538+0000 7f5fc3400640 1 -- 172.21.15.46:0/4169136462 >> v1:172.21.15.46:6789/0 conn(0x7f5fc40af530 legacy=0x7f5fc40702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:09.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.538+0000 7f5fc3400640 1 -- 172.21.15.46:0/4169136462 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc406c5e0 con 0x7f5fc4158300 2024-03-29T08:29:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.542+0000 7f5fc3400640 1 --2- 172.21.15.46:0/4169136462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fc4158300 0x7f5fc406d0c0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009870 tx=0x7f5fb401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2816e3644cfa9353 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.542+0000 7f5fc2a00640 1 -- 172.21.15.46:0/4169136462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40ab600 con 0x7f5fc4158300 2024-03-29T08:29:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.542+0000 7f5fc2a00640 1 -- 172.21.15.46:0/4169136462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb40a7050 con 0x7f5fc4158300 2024-03-29T08:29:09.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.542+0000 7f5fc2a00640 1 -- 172.21.15.46:0/4169136462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40ab980 con 0x7f5fc4158300 2024-03-29T08:29:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.542+0000 7f5fca800640 1 -- 172.21.15.46:0/4169136462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fc4158300 msgr2=0x7f5fc406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.542+0000 7f5fca800640 1 --2- 172.21.15.46:0/4169136462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fc4158300 0x7f5fc406d0c0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009870 tx=0x7f5fb401c410 comp rx=0 tx=0).stop 2024-03-29T08:29:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.542+0000 7f5fca800640 1 -- 172.21.15.46:0/4169136462 shutdown_connections 2024-03-29T08:29:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.542+0000 7f5fca800640 1 --2- 172.21.15.46:0/4169136462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fc4158300 0x7f5fc406d0c0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.542+0000 7f5fca800640 1 -- 172.21.15.46:0/4169136462 >> 172.21.15.46:0/4169136462 conn(0x7f5fc4066080 msgr2=0x7f5fc4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:09.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.542+0000 7f5fca800640 1 -- 172.21.15.46:0/4169136462 shutdown_connections 2024-03-29T08:29:09.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.542+0000 7f5fca800640 1 -- 172.21.15.46:0/4169136462 wait complete. 2024-03-29T08:29:09.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.542+0000 7f5fca800640 1 Processor -- start 2024-03-29T08:29:09.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.542+0000 7f5fca800640 1 -- start start 2024-03-29T08:29:09.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fca800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fc40709b0 0x7f5fc413fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:09.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fca800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fc40af530 0x7f5fc40c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fca800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5fc4158300 0x7f5fc40c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fca800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40c4800 con 0x7f5fc40af530 2024-03-29T08:29:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fca800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40c4ae0 con 0x7f5fc40709b0 2024-03-29T08:29:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fca800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40c4dc0 con 0x7f5fc4158300 2024-03-29T08:29:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fbaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fc40af530 0x7f5fc40c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fbaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fc40af530 0x7f5fc40c3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33518/0 (socket says 172.21.15.46:33518) 2024-03-29T08:29:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fbaa00640 1 -- 172.21.15.46:0/1239757427 learned_addr learned my addr 172.21.15.46:0/1239757427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fc3400640 1 --2- 172.21.15.46:0/1239757427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fc40709b0 0x7f5fc413fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fc3e00640 1 --2- 172.21.15.46:0/1239757427 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5fc4158300 0x7f5fc40c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fbaa00640 1 -- 172.21.15.46:0/1239757427 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5fc4158300 msgr2=0x7f5fc40c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fbaa00640 1 --2- 172.21.15.46:0/1239757427 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5fc4158300 0x7f5fc40c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:09.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fbaa00640 1 -- 172.21.15.46:0/1239757427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fc40709b0 msgr2=0x7f5fc413fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:09.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fbaa00640 1 --2- 172.21.15.46:0/1239757427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fc40709b0 0x7f5fc413fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:09.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fbaa00640 1 -- 172.21.15.46:0/1239757427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb40094d0 con 0x7f5fc40af530 2024-03-29T08:29:09.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fbaa00640 1 --2- 172.21.15.46:0/1239757427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fc40af530 0x7f5fc40c3f80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00c490 tx=0x7f5fac00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:09.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fbbe00640 1 -- 172.21.15.46:0/1239757427 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac007480 con 0x7f5fc40af530 2024-03-29T08:29:09.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fbbe00640 1 -- 172.21.15.46:0/1239757427 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac0075e0 con 0x7f5fc40af530 2024-03-29T08:29:09.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fbbe00640 1 -- 172.21.15.46:0/1239757427 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac004a10 con 0x7f5fc40af530 2024-03-29T08:29:09.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fca800640 1 -- 172.21.15.46:0/1239757427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc40c50a0 con 0x7f5fc40af530 2024-03-29T08:29:09.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fca800640 1 -- 172.21.15.46:0/1239757427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc4140320 con 0x7f5fc40af530 2024-03-29T08:29:09.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fca800640 1 -- 172.21.15.46:0/1239757427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc4071a90 con 0x7f5fc40af530 2024-03-29T08:29:09.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.546+0000 7f5fbbe00640 1 -- 172.21.15.46:0/1239757427 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5fac004b70 con 0x7f5fc40af530 2024-03-29T08:29:09.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.550+0000 7f5fbbe00640 1 --2- 172.21.15.46:0/1239757427 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5fa0066690 0x7f5fa0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:09.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.550+0000 7f5fbbe00640 1 -- 172.21.15.46:0/1239757427 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f5fac083060 con 0x7f5fc40af530 2024-03-29T08:29:09.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.550+0000 7f5fc3400640 1 --2- 172.21.15.46:0/1239757427 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5fa0066690 0x7f5fa0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:09.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.550+0000 7f5fc3400640 1 --2- 172.21.15.46:0/1239757427 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5fa0066690 0x7f5fa0068b50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40ad000 tx=0x7f5fb40a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:09.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.554+0000 7f5fbbe00640 1 -- 172.21.15.46:0/1239757427 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5fac0b4980 con 0x7f5fc40af530 2024-03-29T08:29:09.709 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:29:09.713 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:29:09.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:09.726+0000 7f5fca800640 1 -- 172.21.15.46:0/1239757427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5fc40c4940 con 0x7f5fc40af530 2024-03-29T08:29:10.299 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-03-29T08:29:10.299 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-03-29T08:29:10.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.494+0000 7f5fbbe00640 1 -- 172.21.15.46:0/1239757427 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v117) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5fc40c4940 con 0x7f5fc40af530 2024-03-29T08:29:10.498 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2024-03-29T08:29:10.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.498+0000 7f5fca800640 1 -- 172.21.15.46:0/1239757427 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5fa0066690 msgr2=0x7f5fa0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:10.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.498+0000 7f5fca800640 1 --2- 172.21.15.46:0/1239757427 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5fa0066690 0x7f5fa0068b50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40ad000 tx=0x7f5fb40a6000 comp rx=0 tx=0).stop 2024-03-29T08:29:10.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.498+0000 7f5fca800640 1 -- 172.21.15.46:0/1239757427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fc40af530 msgr2=0x7f5fc40c3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:10.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.498+0000 7f5fca800640 1 --2- 172.21.15.46:0/1239757427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fc40af530 0x7f5fc40c3f80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f5fac00c490 tx=0x7f5fac00c950 comp rx=0 tx=0).stop 2024-03-29T08:29:10.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.498+0000 7f5fca800640 1 -- 172.21.15.46:0/1239757427 shutdown_connections 2024-03-29T08:29:10.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.498+0000 7f5fca800640 1 --2- 172.21.15.46:0/1239757427 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5fa0066690 0x7f5fa0068b50 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:10.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.498+0000 7f5fca800640 1 --2- 172.21.15.46:0/1239757427 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5fc4158300 0x7f5fc40c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:10.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.498+0000 7f5fca800640 1 --2- 172.21.15.46:0/1239757427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5fc40af530 0x7f5fc40c3f80 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:10.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.498+0000 7f5fca800640 1 --2- 172.21.15.46:0/1239757427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5fc40709b0 0x7f5fc413fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:10.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.498+0000 7f5fca800640 1 -- 172.21.15.46:0/1239757427 >> 172.21.15.46:0/1239757427 conn(0x7f5fc4066080 msgr2=0x7f5fc40cdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:10.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.498+0000 7f5fca800640 1 -- 172.21.15.46:0/1239757427 shutdown_connections 2024-03-29T08:29:10.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.498+0000 7f5fca800640 1 -- 172.21.15.46:0/1239757427 wait complete. 2024-03-29T08:29:10.525 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-29T08:29:10.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.666+0000 7fc1eec00640 1 Processor -- start 2024-03-29T08:29:10.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.666+0000 7fc1eec00640 1 -- start start 2024-03-29T08:29:10.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.670+0000 7fc1eec00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc1e8146250 0x7fc1e8160730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.670+0000 7fc1eec00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc1e80abc00 0x7fc1e8069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.670+0000 7fc1eec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e806e390 con 0x7fc1e8146630 2024-03-29T08:29:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.670+0000 7fc1eec00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e806c970 con 0x7fc1e80abc00 2024-03-29T08:29:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.670+0000 7fc1eec00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e81611f0 con 0x7fc1e8146250 2024-03-29T08:29:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.670+0000 7fc1dfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc1e80abc00 0x7fc1e8069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.670+0000 7fc1ecc00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc1e8146630 0x7fc1e80705b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59180/0 (socket says 172.21.15.46:59180) 2024-03-29T08:29:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.670+0000 7fc1ecc00640 1 -- 172.21.15.46:0/3448511069 learned_addr learned my addr 172.21.15.46:0/3448511069 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.670+0000 7fc1e7e00640 1 --2- 172.21.15.46:0/3448511069 >> v2:172.21.15.73:3300/0 conn(0x7fc1e8146250 0x7fc1e8160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.670+0000 7fc1dfe00640 1 -- 172.21.15.46:0/3448511069 >> v2:172.21.15.73:3300/0 conn(0x7fc1e8146250 msgr2=0x7fc1e8160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.670+0000 7fc1dfe00640 1 --2- 172.21.15.46:0/3448511069 >> v2:172.21.15.73:3300/0 conn(0x7fc1e8146250 0x7fc1e8160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.670+0000 7fc1dfe00640 1 -- 172.21.15.46:0/3448511069 >> v1:172.21.15.46:6789/0 conn(0x7fc1e8146630 legacy=0x7fc1e80705b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.670+0000 7fc1dfe00640 1 -- 172.21.15.46:0/3448511069 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e81614d0 con 0x7fc1e80abc00 2024-03-29T08:29:10.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.670+0000 7fc1e7400640 1 -- 172.21.15.46:0/3448511069 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582025797 0 0) 0x7fc1e806e390 con 0x7fc1e8146630 2024-03-29T08:29:10.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1dfe00640 1 --2- 172.21.15.46:0/3448511069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e80abc00 0x7fc1e8069d90 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8073580 tx=0x7fc1d001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca84708fcfe55833 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:10.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1e7400640 1 -- 172.21.15.46:0/3448511069 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00ab650 con 0x7fc1e80abc00 2024-03-29T08:29:10.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1e7400640 1 -- 172.21.15.46:0/3448511069 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d00ab7b0 con 0x7fc1e80abc00 2024-03-29T08:29:10.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1e7400640 1 -- 172.21.15.46:0/3448511069 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d00abb40 con 0x7fc1e80abc00 2024-03-29T08:29:10.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1eec00640 1 -- 172.21.15.46:0/3448511069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e80abc00 msgr2=0x7fc1e8069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:10.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1eec00640 1 --2- 172.21.15.46:0/3448511069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e80abc00 0x7fc1e8069d90 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fc1e8073580 tx=0x7fc1d001c450 comp rx=0 tx=0).stop 2024-03-29T08:29:10.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1eec00640 1 -- 172.21.15.46:0/3448511069 shutdown_connections 2024-03-29T08:29:10.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1eec00640 1 --2- 172.21.15.46:0/3448511069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e80abc00 0x7fc1e8069d90 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:10.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1eec00640 1 --2- 172.21.15.46:0/3448511069 >> v2:172.21.15.73:3300/0 conn(0x7fc1e8146250 0x7fc1e8160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:10.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1eec00640 1 -- 172.21.15.46:0/3448511069 >> 172.21.15.46:0/3448511069 conn(0x7fc1e80b05d0 msgr2=0x7fc1e80b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:10.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1eec00640 1 -- 172.21.15.46:0/3448511069 shutdown_connections 2024-03-29T08:29:10.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1eec00640 1 -- 172.21.15.46:0/3448511069 wait complete. 2024-03-29T08:29:10.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1eec00640 1 Processor -- start 2024-03-29T08:29:10.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1eec00640 1 -- start start 2024-03-29T08:29:10.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1eec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1e80abc00 0x7fc1e813f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:10.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1eec00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1e8146250 0x7fc1e8144b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:10.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1eec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e8146630 0x7fc1e8142ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:10.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1eec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e813f3e0 con 0x7fc1e8146630 2024-03-29T08:29:10.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1eec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e813e7d0 con 0x7fc1e80abc00 2024-03-29T08:29:10.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1eec00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e813eab0 con 0x7fc1e8146250 2024-03-29T08:29:10.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1ecc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e8146630 0x7fc1e8142ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:10.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1ecc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e8146630 0x7fc1e8142ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33534/0 (socket says 172.21.15.46:33534) 2024-03-29T08:29:10.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1ecc00640 1 -- 172.21.15.46:0/253548026 learned_addr learned my addr 172.21.15.46:0/253548026 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:10.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.674+0000 7fc1e7e00640 1 --2- 172.21.15.46:0/253548026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1e80abc00 0x7fc1e813f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:10.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.678+0000 7fc1dfe00640 1 --2- 172.21.15.46:0/253548026 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1e8146250 0x7fc1e8144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:10.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.678+0000 7fc1ecc00640 1 -- 172.21.15.46:0/253548026 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1e8146250 msgr2=0x7fc1e8144b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:10.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.678+0000 7fc1ecc00640 1 --2- 172.21.15.46:0/253548026 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1e8146250 0x7fc1e8144b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:10.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.678+0000 7fc1ecc00640 1 -- 172.21.15.46:0/253548026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1e80abc00 msgr2=0x7fc1e813f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:10.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.678+0000 7fc1ecc00640 1 --2- 172.21.15.46:0/253548026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1e80abc00 0x7fc1e813f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:10.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.678+0000 7fc1ecc00640 1 -- 172.21.15.46:0/253548026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d0009420 con 0x7fc1e8146630 2024-03-29T08:29:10.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.678+0000 7fc1ecc00640 1 --2- 172.21.15.46:0/253548026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e8146630 0x7fc1e8142ef0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000c780 tx=0x7fc1e000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:10.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.678+0000 7fc1e4c00640 1 -- 172.21.15.46:0/253548026 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e000ce90 con 0x7fc1e8146630 2024-03-29T08:29:10.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.678+0000 7fc1e4c00640 1 -- 172.21.15.46:0/253548026 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e000a640 con 0x7fc1e8146630 2024-03-29T08:29:10.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.678+0000 7fc1e4c00640 1 -- 172.21.15.46:0/253548026 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e000a9d0 con 0x7fc1e8146630 2024-03-29T08:29:10.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.678+0000 7fc1eec00640 1 -- 172.21.15.46:0/253548026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e813ed90 con 0x7fc1e8146630 2024-03-29T08:29:10.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.678+0000 7fc1eec00640 1 -- 172.21.15.46:0/253548026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e813fdb0 con 0x7fc1e8146630 2024-03-29T08:29:10.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.678+0000 7fc1eec00640 1 -- 172.21.15.46:0/253548026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e813f550 con 0x7fc1e8146630 2024-03-29T08:29:10.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.678+0000 7fc1e4c00640 1 -- 172.21.15.46:0/253548026 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc1e000ab30 con 0x7fc1e8146630 2024-03-29T08:29:10.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.678+0000 7fc1e4c00640 1 --2- 172.21.15.46:0/253548026 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc1c4066690 0x7fc1c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:10.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.678+0000 7fc1e4c00640 1 -- 172.21.15.46:0/253548026 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc1e00832b0 con 0x7fc1e8146630 2024-03-29T08:29:10.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.678+0000 7fc1e7e00640 1 --2- 172.21.15.46:0/253548026 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc1c4066690 0x7fc1c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:10.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.682+0000 7fc1e7e00640 1 --2- 172.21.15.46:0/253548026 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc1c4066690 0x7fc1c4068b50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009b60 tx=0x7fc1d8009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:10.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.682+0000 7fc1e4c00640 1 -- 172.21.15.46:0/253548026 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc1e0050a10 con 0x7fc1e8146630 2024-03-29T08:29:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.902+0000 7fc1eec00640 1 -- 172.21.15.46:0/253548026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc1e8066c90 con 0x7fc1e8146630 2024-03-29T08:29:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.906+0000 7fc1e4c00640 1 -- 172.21.15.46:0/253548026 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc1e0018090 con 0x7fc1e8146630 2024-03-29T08:29:10.908 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:10.908 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:29:10.908 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2024-03-29T08:29:10.908 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.908 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2024-03-29T08:29:10.908 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmlnXNJBAA7lI2lCBd0PeZudPsQyMOUQ==", 2024-03-29T08:29:10.908 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.908 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:29:10.909 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:29:10.909 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:29:10.909 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:29:10.909 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.909 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.909 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2024-03-29T08:29:10.909 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmbPkcMRAADnwJE17uILjJo14jghJuzg==", 2024-03-29T08:29:10.909 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.909 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:29:10.909 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:29:10.909 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:29:10.910 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:29:10.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.910 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.c", 2024-03-29T08:29:10.910 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmkCUVLBAAxcAOWGBDUCpEHeg/1OeQzA==", 2024-03-29T08:29:10.910 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.910 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:29:10.910 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:29:10.910 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:29:10.910 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:29:10.910 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.910 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.911 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.d", 2024-03-29T08:29:10.911 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZm7L11OBAAZ3fpXtEIYG1tGm9ZFswZ5Q==", 2024-03-29T08:29:10.911 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.911 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:29:10.911 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:29:10.911 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:29:10.911 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:29:10.911 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.911 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.911 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2024-03-29T08:29:10.911 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAIeQZmTHV8HhAAODJ+RP1Bx6k0D1153XgI0Q==", 2024-03-29T08:29:10.911 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.911 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:29:10.912 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:29:10.912 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:29:10.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.912 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.912 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2024-03-29T08:29:10.912 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQALeQZmZRD0JBAAjU6W/21CeqOMdIP5969iNA==", 2024-03-29T08:29:10.912 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.912 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:29:10.912 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:29:10.912 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:29:10.912 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.913 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.913 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2024-03-29T08:29:10.913 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAOeQZmSSP+KhAA26V8CByUJNfkVcG37AdlZQ==", 2024-03-29T08:29:10.913 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.913 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:29:10.913 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:29:10.913 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:29:10.913 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.913 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.914 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2024-03-29T08:29:10.914 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAReQZmb/e8LxAAV4yC1xlLwh8OAqOSPmcyyA==", 2024-03-29T08:29:10.914 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.914 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:29:10.914 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:29:10.914 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:29:10.914 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.914 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.914 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2024-03-29T08:29:10.914 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAXeQZmrLnzNRAAFPYQRs9sBNNKeyvcoI9rSg==", 2024-03-29T08:29:10.914 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.915 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:29:10.915 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:29:10.915 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:29:10.915 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.915 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.915 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2024-03-29T08:29:10.915 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAbeQZmjTPbABAAi4qTU5CBy9LtSSRf7cmfwg==", 2024-03-29T08:29:10.915 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.915 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:29:10.916 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:29:10.916 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:29:10.916 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.916 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.916 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2024-03-29T08:29:10.916 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAeeQZmo13eBRAAWY2VVBakLze9oA4ynp1rNA==", 2024-03-29T08:29:10.916 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.916 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:29:10.916 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:29:10.916 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:29:10.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.917 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2024-03-29T08:29:10.917 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAheQZmghpZCxAAdy5xI7qPwZb8LN8Rn6Ndaw==", 2024-03-29T08:29:10.917 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.917 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2024-03-29T08:29:10.917 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2024-03-29T08:29:10.917 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:29:10.917 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.917 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.917 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2024-03-29T08:29:10.917 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmyk22ABAA8Pn3OaIvxMQSLrKIM8FP4w==", 2024-03-29T08:29:10.917 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.917 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:29:10.918 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow r", 2024-03-29T08:29:10.918 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-03-29T08:29:10.918 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.918 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.918 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.1", 2024-03-29T08:29:10.918 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmG/E8ChAAJTcJ2j5xn0oUsmlV2trWdw==", 2024-03-29T08:29:10.918 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.918 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:29:10.918 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:29:10.918 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:29:10.918 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:29:10.918 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.918 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.919 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.2", 2024-03-29T08:29:10.919 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmhsqgBhAApqnO5qMG8VP27j9TfbZ8uw==", 2024-03-29T08:29:10.919 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.919 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:29:10.919 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:29:10.919 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:29:10.919 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:29:10.919 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.919 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.919 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.3", 2024-03-29T08:29:10.919 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAHeQZmatlQEBAAAGKuGnAh4YXSk91TAzomXA==", 2024-03-29T08:29:10.919 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.919 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2024-03-29T08:29:10.920 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2024-03-29T08:29:10.920 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2024-03-29T08:29:10.920 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2024-03-29T08:29:10.920 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.920 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.920 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2024-03-29T08:29:10.920 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmp2B3CBAA4cfe/hUJeOg0SIr1KbWyUg==", 2024-03-29T08:29:10.920 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.920 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:29:10.920 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2024-03-29T08:29:10.920 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2024-03-29T08:29:10.920 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:29:10.921 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.921 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2024-03-29T08:29:10.921 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZmyrJXIBAAv535Rz99UI3V7JvvzQ8oSw==", 2024-03-29T08:29:10.921 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.921 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:29:10.921 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-03-29T08:29:10.921 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:29:10.921 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:10.921 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:10.921 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2024-03-29T08:29:10.921 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQAGeQZm72tiHBAA2N7qQnR5Ehu/7nP2WpklGg==", 2024-03-29T08:29:10.921 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2024-03-29T08:29:10.921 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2024-03-29T08:29:10.922 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2024-03-29T08:29:10.922 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2024-03-29T08:29:10.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.922 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:10.922 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-03-29T08:29:10.922 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:29:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.906+0000 7fc1eec00640 1 -- 172.21.15.46:0/253548026 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc1c4066690 msgr2=0x7fc1c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.906+0000 7fc1eec00640 1 --2- 172.21.15.46:0/253548026 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc1c4066690 0x7fc1c4068b50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc1d8009b60 tx=0x7fc1d8009190 comp rx=0 tx=0).stop 2024-03-29T08:29:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.906+0000 7fc1eec00640 1 -- 172.21.15.46:0/253548026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e8146630 msgr2=0x7fc1e8142ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.906+0000 7fc1eec00640 1 --2- 172.21.15.46:0/253548026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e8146630 0x7fc1e8142ef0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fc1e000c780 tx=0x7fc1e000cc40 comp rx=0 tx=0).stop 2024-03-29T08:29:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.906+0000 7fc1eec00640 1 -- 172.21.15.46:0/253548026 shutdown_connections 2024-03-29T08:29:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.906+0000 7fc1eec00640 1 --2- 172.21.15.46:0/253548026 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc1c4066690 0x7fc1c4068b50 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.906+0000 7fc1eec00640 1 --2- 172.21.15.46:0/253548026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc1e8146630 0x7fc1e8142ef0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.906+0000 7fc1eec00640 1 --2- 172.21.15.46:0/253548026 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc1e8146250 0x7fc1e8144b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:10.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.906+0000 7fc1eec00640 1 --2- 172.21.15.46:0/253548026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1e80abc00 0x7fc1e813f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.906+0000 7fc1eec00640 1 -- 172.21.15.46:0/253548026 >> 172.21.15.46:0/253548026 conn(0x7fc1e80b05d0 msgr2=0x7fc1e806d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.906+0000 7fc1eec00640 1 -- 172.21.15.46:0/253548026 shutdown_connections 2024-03-29T08:29:10.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:10.906+0000 7fc1eec00640 1 -- 172.21.15.46:0/253548026 wait complete. 2024-03-29T08:29:10.929 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:29:11.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.070+0000 7f5daf400640 1 Processor -- start 2024-03-29T08:29:11.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.070+0000 7f5daf400640 1 -- start start 2024-03-29T08:29:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.070+0000 7f5daf400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5da80d0660 0x7f5da80d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.070+0000 7f5daf400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5da806b8b0 0x7f5da80cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.070+0000 7f5daf400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da815cfb0 con 0x7f5da815dc00 2024-03-29T08:29:11.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.070+0000 7f5daf400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da806c100 con 0x7f5da80d0660 2024-03-29T08:29:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.070+0000 7f5daf400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da80d0f80 con 0x7f5da806b8b0 2024-03-29T08:29:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.070+0000 7f5daca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5da80d0660 0x7f5da80d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.070+0000 7f5daca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5da80d0660 0x7f5da80d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33542/0 (socket says 172.21.15.46:33542) 2024-03-29T08:29:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.070+0000 7f5daca00640 1 -- 172.21.15.46:0/985736064 learned_addr learned my addr 172.21.15.46:0/985736064 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.070+0000 7f5da7e00640 1 --2- 172.21.15.46:0/985736064 >> v2:172.21.15.73:3300/0 conn(0x7f5da806b8b0 0x7f5da80cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.070+0000 7f5daca00640 1 -- 172.21.15.46:0/985736064 >> v2:172.21.15.73:3300/0 conn(0x7f5da806b8b0 msgr2=0x7f5da80cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.070+0000 7f5daca00640 1 --2- 172.21.15.46:0/985736064 >> v2:172.21.15.73:3300/0 conn(0x7f5da806b8b0 0x7f5da80cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.070+0000 7f5daca00640 1 -- 172.21.15.46:0/985736064 >> v1:172.21.15.73:6789/0 conn(0x7f5da815dc00 legacy=0x7f5da80cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.070+0000 7f5daca00640 1 -- 172.21.15.46:0/985736064 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da806be20 con 0x7f5da80d0660 2024-03-29T08:29:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.074+0000 7f5daca00640 1 --2- 172.21.15.46:0/985736064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da80d0660 0x7f5da80d0a40 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f5da806acb0 tx=0x7f5d9c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753de64090b950a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.074+0000 7f5da7400640 1 -- 172.21.15.46:0/985736064 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0aa650 con 0x7f5da80d0660 2024-03-29T08:29:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.074+0000 7f5da7400640 1 -- 172.21.15.46:0/985736064 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9c0aa7b0 con 0x7f5da80d0660 2024-03-29T08:29:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.074+0000 7f5da7400640 1 -- 172.21.15.46:0/985736064 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c0aab40 con 0x7f5da80d0660 2024-03-29T08:29:11.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.074+0000 7f5daf400640 1 -- 172.21.15.46:0/985736064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da80d0660 msgr2=0x7f5da80d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.074+0000 7f5daf400640 1 --2- 172.21.15.46:0/985736064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da80d0660 0x7f5da80d0a40 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f5da806acb0 tx=0x7f5d9c01c4f0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.074+0000 7f5daf400640 1 -- 172.21.15.46:0/985736064 shutdown_connections 2024-03-29T08:29:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.074+0000 7f5daf400640 1 --2- 172.21.15.46:0/985736064 >> v2:172.21.15.73:3300/0 conn(0x7f5da806b8b0 0x7f5da80cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.074+0000 7f5daf400640 1 --2- 172.21.15.46:0/985736064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da80d0660 0x7f5da80d0a40 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.074+0000 7f5daf400640 1 -- 172.21.15.46:0/985736064 >> 172.21.15.46:0/985736064 conn(0x7f5da8089fd0 msgr2=0x7f5da808a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.074+0000 7f5daf400640 1 -- 172.21.15.46:0/985736064 shutdown_connections 2024-03-29T08:29:11.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.074+0000 7f5daf400640 1 -- 172.21.15.46:0/985736064 wait complete. 2024-03-29T08:29:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.074+0000 7f5daf400640 1 Processor -- start 2024-03-29T08:29:11.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.074+0000 7f5daf400640 1 -- start start 2024-03-29T08:29:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5daf400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5da806b8b0 0x7f5da80ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5daf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da80d0660 0x7f5da81518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5daf400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5da815dc00 0x7f5da8151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5daf400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da80b3760 con 0x7f5da80d0660 2024-03-29T08:29:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5daf400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da80b58c0 con 0x7f5da806b8b0 2024-03-29T08:29:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5daf400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da80b5ba0 con 0x7f5da815dc00 2024-03-29T08:29:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5da7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da80d0660 0x7f5da81518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:11.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5da7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da80d0660 0x7f5da81518d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33554/0 (socket says 172.21.15.46:33554) 2024-03-29T08:29:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5da7e00640 1 -- 172.21.15.46:0/1381512047 learned_addr learned my addr 172.21.15.46:0/1381512047 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5daca00640 1 --2- 172.21.15.46:0/1381512047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5da806b8b0 0x7f5da80ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5dad400640 1 --2- 172.21.15.46:0/1381512047 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5da815dc00 0x7f5da8151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5da7e00640 1 -- 172.21.15.46:0/1381512047 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5da815dc00 msgr2=0x7f5da8151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5da7e00640 1 --2- 172.21.15.46:0/1381512047 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5da815dc00 0x7f5da8151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5da7e00640 1 -- 172.21.15.46:0/1381512047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5da806b8b0 msgr2=0x7f5da80ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5da7e00640 1 --2- 172.21.15.46:0/1381512047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5da806b8b0 0x7f5da80ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5da7e00640 1 -- 172.21.15.46:0/1381512047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c0094d0 con 0x7f5da80d0660 2024-03-29T08:29:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5da7e00640 1 --2- 172.21.15.46:0/1381512047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da80d0660 0x7f5da81518d0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400d5f0 tx=0x7f5d9400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:11.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5da4c00640 1 -- 172.21.15.46:0/1381512047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d940125f0 con 0x7f5da80d0660 2024-03-29T08:29:11.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5da4c00640 1 -- 172.21.15.46:0/1381512047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d94012750 con 0x7f5da80d0660 2024-03-29T08:29:11.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5da4c00640 1 -- 172.21.15.46:0/1381512047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d94012b00 con 0x7f5da80d0660 2024-03-29T08:29:11.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5daf400640 1 -- 172.21.15.46:0/1381512047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da80b5e80 con 0x7f5da80d0660 2024-03-29T08:29:11.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5daf400640 1 -- 172.21.15.46:0/1381512047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da80ca620 con 0x7f5da80d0660 2024-03-29T08:29:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.078+0000 7f5daf400640 1 -- 172.21.15.46:0/1381512047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da8066c90 con 0x7f5da80d0660 2024-03-29T08:29:11.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.082+0000 7f5da4c00640 1 -- 172.21.15.46:0/1381512047 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5d94012c60 con 0x7f5da80d0660 2024-03-29T08:29:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.082+0000 7f5da4c00640 1 --2- 172.21.15.46:0/1381512047 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d88066690 0x7f5d88068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.082+0000 7f5da4c00640 1 -- 172.21.15.46:0/1381512047 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5d94084240 con 0x7f5da80d0660 2024-03-29T08:29:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.082+0000 7f5daca00640 1 --2- 172.21.15.46:0/1381512047 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d88066690 0x7f5d88068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:11.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.082+0000 7f5daca00640 1 --2- 172.21.15.46:0/1381512047 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d88066690 0x7f5d88068b50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0ae000 tx=0x7f5d9c0b5040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:11.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.086+0000 7f5da4c00640 1 -- 172.21.15.46:0/1381512047 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5d940519a0 con 0x7f5da80d0660 2024-03-29T08:29:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.246+0000 7f5daf400640 1 -- 172.21.15.46:0/1381512047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5da80cb230 con 0x7f5da80d0660 2024-03-29T08:29:11.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.246+0000 7f5da4c00640 1 -- 172.21.15.46:0/1381512047 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+12351 (secure 0 0 0) 0x7f5d94054fc0 con 0x7f5da80d0660 2024-03-29T08:29:11.250 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:11.250 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":117,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:29:10.494610+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:29:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.250+0000 7f5daf400640 1 -- 172.21.15.46:0/1381512047 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d88066690 msgr2=0x7f5d88068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.250+0000 7f5daf400640 1 --2- 172.21.15.46:0/1381512047 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d88066690 0x7f5d88068b50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0ae000 tx=0x7f5d9c0b5040 comp rx=0 tx=0).stop 2024-03-29T08:29:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.250+0000 7f5daf400640 1 -- 172.21.15.46:0/1381512047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da80d0660 msgr2=0x7f5da81518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.250+0000 7f5daf400640 1 --2- 172.21.15.46:0/1381512047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da80d0660 0x7f5da81518d0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400d5f0 tx=0x7f5d9400dab0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.250+0000 7f5daf400640 1 -- 172.21.15.46:0/1381512047 shutdown_connections 2024-03-29T08:29:11.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.250+0000 7f5daf400640 1 --2- 172.21.15.46:0/1381512047 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5d88066690 0x7f5d88068b50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.250+0000 7f5daf400640 1 --2- 172.21.15.46:0/1381512047 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5da815dc00 0x7f5da8151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.250+0000 7f5daf400640 1 --2- 172.21.15.46:0/1381512047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5da80d0660 0x7f5da81518d0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.250+0000 7f5daf400640 1 --2- 172.21.15.46:0/1381512047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5da806b8b0 0x7f5da80ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.250+0000 7f5daf400640 1 -- 172.21.15.46:0/1381512047 >> 172.21.15.46:0/1381512047 conn(0x7f5da8089fd0 msgr2=0x7f5da80cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:11.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.250+0000 7f5daf400640 1 -- 172.21.15.46:0/1381512047 shutdown_connections 2024-03-29T08:29:11.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.250+0000 7f5daf400640 1 -- 172.21.15.46:0/1381512047 wait complete. 2024-03-29T08:29:11.272 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:29:11.273 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:29:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.414+0000 7ff781200640 1 Processor -- start 2024-03-29T08:29:11.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.414+0000 7ff781200640 1 -- start start 2024-03-29T08:29:11.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.414+0000 7ff781200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff77c142a50 0x7ff77c0cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:11.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.414+0000 7ff781200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff77c15df90 0x7ff77c0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:11.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.414+0000 7ff781200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77c15d260 con 0x7ff77c0ae280 2024-03-29T08:29:11.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.414+0000 7ff781200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77c0aefd0 con 0x7ff77c15df90 2024-03-29T08:29:11.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.414+0000 7ff781200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c15d4b0 con 0x7ff77c142a50 2024-03-29T08:29:11.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.414+0000 7ff771400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff77c15df90 0x7ff77c0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:11.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.414+0000 7ff771400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff77c15df90 0x7ff77c0cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33556/0 (socket says 172.21.15.46:33556) 2024-03-29T08:29:11.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.414+0000 7ff771400640 1 -- 172.21.15.46:0/3727123969 learned_addr learned my addr 172.21.15.46:0/3727123969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:11.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.414+0000 7ff779e00640 1 --2- 172.21.15.46:0/3727123969 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff77c142a50 0x7ff77c0cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:11.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.414+0000 7ff771400640 1 -- 172.21.15.46:0/3727123969 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff77c142a50 msgr2=0x7ff77c0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:11.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.414+0000 7ff771400640 1 --2- 172.21.15.46:0/3727123969 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff77c142a50 0x7ff77c0cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.414+0000 7ff771400640 1 -- 172.21.15.46:0/3727123969 >> v1:172.21.15.73:6789/0 conn(0x7ff77c0ae280 legacy=0x7ff77c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.414+0000 7ff771400640 1 -- 172.21.15.46:0/3727123969 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c15d790 con 0x7ff77c15df90 2024-03-29T08:29:11.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.414+0000 7ff779e00640 1 --2- 172.21.15.46:0/3727123969 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff77c142a50 0x7ff77c0cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.418+0000 7ff771400640 1 --2- 172.21.15.46:0/3727123969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff77c15df90 0x7ff77c0cbcd0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7ff764009870 tx=0x7ff76401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a799df863030f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.418+0000 7ff779400640 1 -- 172.21.15.46:0/3727123969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7640ab600 con 0x7ff77c15df90 2024-03-29T08:29:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.418+0000 7ff779400640 1 -- 172.21.15.46:0/3727123969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7640a7050 con 0x7ff77c15df90 2024-03-29T08:29:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.418+0000 7ff779400640 1 -- 172.21.15.46:0/3727123969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7640ab980 con 0x7ff77c15df90 2024-03-29T08:29:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.418+0000 7ff781200640 1 -- 172.21.15.46:0/3727123969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff77c15df90 msgr2=0x7ff77c0cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.418+0000 7ff781200640 1 --2- 172.21.15.46:0/3727123969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff77c15df90 0x7ff77c0cbcd0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7ff764009870 tx=0x7ff76401c4d0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.418+0000 7ff781200640 1 -- 172.21.15.46:0/3727123969 shutdown_connections 2024-03-29T08:29:11.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.418+0000 7ff781200640 1 --2- 172.21.15.46:0/3727123969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff77c15df90 0x7ff77c0cbcd0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.418+0000 7ff781200640 1 --2- 172.21.15.46:0/3727123969 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff77c142a50 0x7ff77c0cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.418+0000 7ff781200640 1 -- 172.21.15.46:0/3727123969 >> 172.21.15.46:0/3727123969 conn(0x7ff77c06a390 msgr2=0x7ff77c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:11.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.418+0000 7ff781200640 1 -- 172.21.15.46:0/3727123969 shutdown_connections 2024-03-29T08:29:11.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.418+0000 7ff781200640 1 -- 172.21.15.46:0/3727123969 wait complete. 2024-03-29T08:29:11.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff781200640 1 Processor -- start 2024-03-29T08:29:11.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff781200640 1 -- start start 2024-03-29T08:29:11.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff781200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff77c0ae280 0x7ff77c1584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:11.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff781200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff77c142a50 0x7ff77c140eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:11.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff781200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff77c15df90 0x7ff77c1413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:11.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff781200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c1f52a0 con 0x7ff77c142a50 2024-03-29T08:29:11.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff781200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c1f4690 con 0x7ff77c15df90 2024-03-29T08:29:11.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff781200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c1f4970 con 0x7ff77c0ae280 2024-03-29T08:29:11.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff771400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff77c142a50 0x7ff77c140eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff771400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff77c142a50 0x7ff77c140eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33566/0 (socket says 172.21.15.46:33566) 2024-03-29T08:29:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff771400640 1 -- 172.21.15.46:0/788965910 learned_addr learned my addr 172.21.15.46:0/788965910 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff779e00640 1 --2- 172.21.15.46:0/788965910 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff77c0ae280 0x7ff77c1584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff77a800640 1 --2- 172.21.15.46:0/788965910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff77c15df90 0x7ff77c1413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff771400640 1 -- 172.21.15.46:0/788965910 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff77c0ae280 msgr2=0x7ff77c1584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff771400640 1 --2- 172.21.15.46:0/788965910 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff77c0ae280 0x7ff77c1584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff771400640 1 -- 172.21.15.46:0/788965910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff77c15df90 msgr2=0x7ff77c1413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff771400640 1 --2- 172.21.15.46:0/788965910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff77c15df90 0x7ff77c1413f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff771400640 1 -- 172.21.15.46:0/788965910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7640094d0 con 0x7ff77c142a50 2024-03-29T08:29:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff77a800640 1 --2- 172.21.15.46:0/788965910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff77c15df90 0x7ff77c1413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff771400640 1 --2- 172.21.15.46:0/788965910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff77c142a50 0x7ff77c140eb0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7ff7640ae040 tx=0x7ff76401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff772a00640 1 -- 172.21.15.46:0/788965910 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7640b6830 con 0x7ff77c142a50 2024-03-29T08:29:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff772a00640 1 -- 172.21.15.46:0/788965910 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7640abe90 con 0x7ff77c142a50 2024-03-29T08:29:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff781200640 1 -- 172.21.15.46:0/788965910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77c1f4c50 con 0x7ff77c142a50 2024-03-29T08:29:11.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff772a00640 1 -- 172.21.15.46:0/788965910 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7640b6bf0 con 0x7ff77c142a50 2024-03-29T08:29:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff781200640 1 -- 172.21.15.46:0/788965910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77c1589f0 con 0x7ff77c142a50 2024-03-29T08:29:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.422+0000 7ff781200640 1 -- 172.21.15.46:0/788965910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77c0b45c0 con 0x7ff77c142a50 2024-03-29T08:29:11.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.426+0000 7ff772a00640 1 -- 172.21.15.46:0/788965910 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff7640b2040 con 0x7ff77c142a50 2024-03-29T08:29:11.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.426+0000 7ff772a00640 1 --2- 172.21.15.46:0/788965910 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff758066690 0x7ff758068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:11.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.426+0000 7ff772a00640 1 -- 172.21.15.46:0/788965910 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff76411e350 con 0x7ff77c142a50 2024-03-29T08:29:11.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.426+0000 7ff779e00640 1 --2- 172.21.15.46:0/788965910 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff758066690 0x7ff758068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:11.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.426+0000 7ff779e00640 1 --2- 172.21.15.46:0/788965910 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff758066690 0x7ff758068b50 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff76c006e90 tx=0x7ff76c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.430+0000 7ff772a00640 1 -- 172.21.15.46:0/788965910 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff7640eb8e0 con 0x7ff77c142a50 2024-03-29T08:29:11.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.590+0000 7ff781200640 1 -- 172.21.15.46:0/788965910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff77c071a70 con 0x7ff77c142a50 2024-03-29T08:29:11.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.590+0000 7ff772a00640 1 -- 172.21.15.46:0/788965910 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff7640a4f00 con 0x7ff77c142a50 2024-03-29T08:29:11.595 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:29:11.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.594+0000 7ff781200640 1 -- 172.21.15.46:0/788965910 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff758066690 msgr2=0x7ff758068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.594+0000 7ff781200640 1 --2- 172.21.15.46:0/788965910 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff758066690 0x7ff758068b50 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff76c006e90 tx=0x7ff76c008040 comp rx=0 tx=0).stop 2024-03-29T08:29:11.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.594+0000 7ff781200640 1 -- 172.21.15.46:0/788965910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff77c142a50 msgr2=0x7ff77c140eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.594+0000 7ff781200640 1 --2- 172.21.15.46:0/788965910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff77c142a50 0x7ff77c140eb0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7ff7640ae040 tx=0x7ff76401ad10 comp rx=0 tx=0).stop 2024-03-29T08:29:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.594+0000 7ff781200640 1 -- 172.21.15.46:0/788965910 shutdown_connections 2024-03-29T08:29:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.594+0000 7ff781200640 1 --2- 172.21.15.46:0/788965910 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff758066690 0x7ff758068b50 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.594+0000 7ff781200640 1 --2- 172.21.15.46:0/788965910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff77c15df90 0x7ff77c1413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.594+0000 7ff781200640 1 --2- 172.21.15.46:0/788965910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff77c142a50 0x7ff77c140eb0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.594+0000 7ff781200640 1 --2- 172.21.15.46:0/788965910 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff77c0ae280 0x7ff77c1584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.594+0000 7ff781200640 1 -- 172.21.15.46:0/788965910 >> 172.21.15.46:0/788965910 conn(0x7ff77c06a390 msgr2=0x7ff77c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.594+0000 7ff781200640 1 -- 172.21.15.46:0/788965910 shutdown_connections 2024-03-29T08:29:11.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.594+0000 7ff781200640 1 -- 172.21.15.46:0/788965910 wait complete. 2024-03-29T08:29:11.616 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-29T08:29:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.754+0000 7f2ebec00640 1 Processor -- start 2024-03-29T08:29:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.754+0000 7f2ebec00640 1 -- start start 2024-03-29T08:29:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.754+0000 7f2ebec00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2eb815a890 0x7f2eb8098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.754+0000 7f2ebec00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb80cc2e0 con 0x7f2eb8099690 2024-03-29T08:29:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.754+0000 7f2ebec00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb80cc580 con 0x7f2eb80b2b30 2024-03-29T08:29:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.754+0000 7f2ebec00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80cce20 con 0x7f2eb815a890 2024-03-29T08:29:11.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.754+0000 7f2ebe200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2eb8099690 0x7f2eb80b0500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59194/0 (socket says 172.21.15.46:59194) 2024-03-29T08:29:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.754+0000 7f2ebe200640 1 -- 172.21.15.46:0/81589364 learned_addr learned my addr 172.21.15.46:0/81589364 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.754+0000 7f2ebd800640 1 --2- 172.21.15.46:0/81589364 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2eb815a890 0x7f2eb8098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.754+0000 7f2eb7e00640 1 -- 172.21.15.46:0/81589364 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329868440 0 0) 0x7f2eb80cc2e0 con 0x7f2eb8099690 2024-03-29T08:29:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.754+0000 7f2eb7e00640 1 -- 172.21.15.46:0/81589364 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ea0003530 con 0x7f2eb8099690 2024-03-29T08:29:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.754+0000 7f2eb7e00640 1 -- 172.21.15.46:0/81589364 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638287505 0 0) 0x7f2eb80cc580 con 0x7f2eb80b2b30 2024-03-29T08:29:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.754+0000 7f2eb7e00640 1 -- 172.21.15.46:0/81589364 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2eb80cc2e0 con 0x7f2eb80b2b30 2024-03-29T08:29:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.754+0000 7f2ebd800640 1 -- 172.21.15.46:0/81589364 >> v1:172.21.15.73:6789/0 conn(0x7f2eb80b2b30 legacy=0x7f2eb8098f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.754+0000 7f2ebd800640 1 -- 172.21.15.46:0/81589364 >> v1:172.21.15.46:6789/0 conn(0x7f2eb8099690 legacy=0x7f2eb80b0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.754+0000 7f2ebd800640 1 -- 172.21.15.46:0/81589364 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb80cc860 con 0x7f2eb815a890 2024-03-29T08:29:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.758+0000 7f2ebd800640 1 --2- 172.21.15.46:0/81589364 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2eb815a890 0x7f2eb8098a40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0097f0 tx=0x7f2eac01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd6c77a102956b27 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.758+0000 7f2eb7e00640 1 -- 172.21.15.46:0/81589364 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0ab600 con 0x7f2eb815a890 2024-03-29T08:29:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.758+0000 7f2eb7e00640 1 -- 172.21.15.46:0/81589364 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eac0a7050 con 0x7f2eb815a890 2024-03-29T08:29:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.758+0000 7f2eb7e00640 1 -- 172.21.15.46:0/81589364 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0ab980 con 0x7f2eb815a890 2024-03-29T08:29:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.758+0000 7f2ebec00640 1 -- 172.21.15.46:0/81589364 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2eb815a890 msgr2=0x7f2eb8098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.758+0000 7f2ebec00640 1 --2- 172.21.15.46:0/81589364 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2eb815a890 0x7f2eb8098a40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0097f0 tx=0x7f2eac01c420 comp rx=0 tx=0).stop 2024-03-29T08:29:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.758+0000 7f2ebec00640 1 -- 172.21.15.46:0/81589364 shutdown_connections 2024-03-29T08:29:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.758+0000 7f2ebec00640 1 --2- 172.21.15.46:0/81589364 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2eb815a890 0x7f2eb8098a40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.758+0000 7f2ebec00640 1 -- 172.21.15.46:0/81589364 >> 172.21.15.46:0/81589364 conn(0x7f2eb80b2010 msgr2=0x7f2eb80b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:11.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.758+0000 7f2ebec00640 1 -- 172.21.15.46:0/81589364 shutdown_connections 2024-03-29T08:29:11.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.758+0000 7f2ebec00640 1 -- 172.21.15.46:0/81589364 wait complete. 2024-03-29T08:29:11.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.758+0000 7f2ebec00640 1 Processor -- start 2024-03-29T08:29:11.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.758+0000 7f2ebec00640 1 -- start start 2024-03-29T08:29:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebec00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2eb8099690 0x7f2eb80c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebec00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb80b2b30 0x7f2eb814d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebec00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2eb815a890 0x7f2eb814d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:11.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebec00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb814dd30 con 0x7f2eb80b2b30 2024-03-29T08:29:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebec00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80c7030 con 0x7f2eb8099690 2024-03-29T08:29:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebec00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80b57d0 con 0x7f2eb815a890 2024-03-29T08:29:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb80b2b30 0x7f2eb814d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb80b2b30 0x7f2eb814d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33574/0 (socket says 172.21.15.46:33574) 2024-03-29T08:29:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebce00640 1 -- 172.21.15.46:0/851820236 learned_addr learned my addr 172.21.15.46:0/851820236 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebd800640 1 --2- 172.21.15.46:0/851820236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2eb8099690 0x7f2eb80c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:11.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebce00640 1 -- 172.21.15.46:0/851820236 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2eb815a890 msgr2=0x7f2eb814d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebce00640 1 --2- 172.21.15.46:0/851820236 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2eb815a890 0x7f2eb814d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebce00640 1 -- 172.21.15.46:0/851820236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2eb8099690 msgr2=0x7f2eb80c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebce00640 1 --2- 172.21.15.46:0/851820236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2eb8099690 0x7f2eb80c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebce00640 1 -- 172.21.15.46:0/851820236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac009420 con 0x7f2eb80b2b30 2024-03-29T08:29:11.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebd800640 1 --2- 172.21.15.46:0/851820236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2eb8099690 0x7f2eb80c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:11.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebce00640 1 --2- 172.21.15.46:0/851820236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb80b2b30 0x7f2eb814d080 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400c430 tx=0x7f2ea400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:11.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2eb5600640 1 -- 172.21.15.46:0/851820236 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40052d0 con 0x7f2eb80b2b30 2024-03-29T08:29:11.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2eb5600640 1 -- 172.21.15.46:0/851820236 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea4005450 con 0x7f2eb80b2b30 2024-03-29T08:29:11.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2eb5600640 1 -- 172.21.15.46:0/851820236 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4003840 con 0x7f2eb80b2b30 2024-03-29T08:29:11.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebec00640 1 -- 172.21.15.46:0/851820236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb80b5ab0 con 0x7f2eb80b2b30 2024-03-29T08:29:11.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebec00640 1 -- 172.21.15.46:0/851820236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb814a4a0 con 0x7f2eb80b2b30 2024-03-29T08:29:11.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2ebec00640 1 -- 172.21.15.46:0/851820236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb814de70 con 0x7f2eb80b2b30 2024-03-29T08:29:11.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.762+0000 7f2eb5600640 1 -- 172.21.15.46:0/851820236 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2ea40039a0 con 0x7f2eb80b2b30 2024-03-29T08:29:11.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.766+0000 7f2eb5600640 1 --2- 172.21.15.46:0/851820236 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e98066690 0x7f2e98068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:11.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.766+0000 7f2eb5600640 1 -- 172.21.15.46:0/851820236 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2ea40833e0 con 0x7f2eb80b2b30 2024-03-29T08:29:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.766+0000 7f2ebd800640 1 --2- 172.21.15.46:0/851820236 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e98066690 0x7f2e98068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.766+0000 7f2ebd800640 1 --2- 172.21.15.46:0/851820236 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e98066690 0x7f2e98068b50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0ad000 tx=0x7f2eac0a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:11.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.770+0000 7f2eb5600640 1 -- 172.21.15.46:0/851820236 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2eb814de70 con 0x7f2eb80b2b30 2024-03-29T08:29:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.930+0000 7f2ebec00640 1 -- 172.21.15.46:0/851820236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2eb814de70 con 0x7f2eb80b2b30 2024-03-29T08:29:11.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.930+0000 7f2eb5600640 1 -- 172.21.15.46:0/851820236 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v117) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2ea4017060 con 0x7f2eb80b2b30 2024-03-29T08:29:11.935 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:11.935 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-03-29T08:29:11.935 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-03-29T08:29:11.935 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-03-29T08:29:11.935 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-03-29T08:29:11.935 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:11.935 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-03-29T08:29:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.934+0000 7f2ebec00640 1 -- 172.21.15.46:0/851820236 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e98066690 msgr2=0x7f2e98068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.934+0000 7f2ebec00640 1 --2- 172.21.15.46:0/851820236 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e98066690 0x7f2e98068b50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0ad000 tx=0x7f2eac0a6000 comp rx=0 tx=0).stop 2024-03-29T08:29:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.934+0000 7f2ebec00640 1 -- 172.21.15.46:0/851820236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb80b2b30 msgr2=0x7f2eb814d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.934+0000 7f2ebec00640 1 --2- 172.21.15.46:0/851820236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb80b2b30 0x7f2eb814d080 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400c430 tx=0x7f2ea400c8f0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.934+0000 7f2ebec00640 1 -- 172.21.15.46:0/851820236 shutdown_connections 2024-03-29T08:29:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.934+0000 7f2ebec00640 1 --2- 172.21.15.46:0/851820236 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2e98066690 0x7f2e98068b50 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.934+0000 7f2ebec00640 1 --2- 172.21.15.46:0/851820236 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2eb815a890 0x7f2eb814d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.934+0000 7f2ebec00640 1 --2- 172.21.15.46:0/851820236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb80b2b30 0x7f2eb814d080 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.934+0000 7f2ebec00640 1 --2- 172.21.15.46:0/851820236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2eb8099690 0x7f2eb80c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.934+0000 7f2ebec00640 1 -- 172.21.15.46:0/851820236 >> 172.21.15.46:0/851820236 conn(0x7f2eb80b2010 msgr2=0x7f2eb806aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.934+0000 7f2ebec00640 1 -- 172.21.15.46:0/851820236 shutdown_connections 2024-03-29T08:29:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:11.934+0000 7f2ebec00640 1 -- 172.21.15.46:0/851820236 wait complete. 2024-03-29T08:29:11.955 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-29T08:29:11.955 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-29T08:29:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f997bc00640 1 Processor -- start 2024-03-29T08:29:12.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f997bc00640 1 -- start start 2024-03-29T08:29:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f997bc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f997406bfc0 0x7f99740ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f997bc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99740cef50 con 0x7f997415c870 2024-03-29T08:29:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f997bc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99740cf230 con 0x7f997415cc50 2024-03-29T08:29:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f997bc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99740cf510 con 0x7f997406bfc0 2024-03-29T08:29:12.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9979200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f997415c870 0x7f99740997a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59200/0 (socket says 172.21.15.46:59200) 2024-03-29T08:29:12.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9979200640 1 -- 172.21.15.46:0/2444605847 learned_addr learned my addr 172.21.15.46:0/2444605847 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:12.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9973e00640 1 -- 172.21.15.46:0/2444605847 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232917963 0 0) 0x7f99740cef50 con 0x7f997415c870 2024-03-29T08:29:12.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9973e00640 1 -- 172.21.15.46:0/2444605847 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9958003530 con 0x7f997415c870 2024-03-29T08:29:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9979c00640 1 --2- 172.21.15.46:0/2444605847 >> v2:172.21.15.73:3300/0 conn(0x7f997406bfc0 0x7f99740ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9973e00640 1 -- 172.21.15.46:0/2444605847 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1211224448 0 0) 0x7f9958003530 con 0x7f997415c870 2024-03-29T08:29:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9973e00640 1 -- 172.21.15.46:0/2444605847 >> v2:172.21.15.73:3300/0 conn(0x7f997406bfc0 msgr2=0x7f99740ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9973e00640 1 --2- 172.21.15.46:0/2444605847 >> v2:172.21.15.73:3300/0 conn(0x7f997406bfc0 0x7f99740ce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9973e00640 1 -- 172.21.15.46:0/2444605847 >> v1:172.21.15.73:6789/0 conn(0x7f997415cc50 legacy=0x7f997406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9973e00640 1 -- 172.21.15.46:0/2444605847 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99740cf7f0 con 0x7f997415c870 2024-03-29T08:29:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9973e00640 1 -- 172.21.15.46:0/2444605847 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f9964001680 con 0x7f997415c870 2024-03-29T08:29:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9973e00640 1 -- 172.21.15.46:0/2444605847 >> v1:172.21.15.46:6789/0 conn(0x7f997415c870 legacy=0x7f99740997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9973e00640 1 --2- 172.21.15.46:0/2444605847 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99580039d0 0x7f9958003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9973e00640 1 --2- 172.21.15.46:0/2444605847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99580047d0 0x7f9958004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9973e00640 1 --2- 172.21.15.46:0/2444605847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99580052b0 0x7f995800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9973e00640 1 -- 172.21.15.46:0/2444605847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99740cf510 con 0x7f99580047d0 2024-03-29T08:29:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9973e00640 1 -- 172.21.15.46:0/2444605847 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995800c1b0 con 0x7f99580052b0 2024-03-29T08:29:12.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9973e00640 1 -- 172.21.15.46:0/2444605847 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995800c370 con 0x7f99580039d0 2024-03-29T08:29:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f996be00640 1 --2- 172.21.15.46:0/2444605847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99580047d0 0x7f9958004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9979c00640 1 --2- 172.21.15.46:0/2444605847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99580052b0 0x7f995800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.098+0000 7f9979200640 1 --2- 172.21.15.46:0/2444605847 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99580039d0 0x7f9958003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.102+0000 7f996be00640 1 -- 172.21.15.46:0/2444605847 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99580039d0 msgr2=0x7f9958003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.102+0000 7f996be00640 1 --2- 172.21.15.46:0/2444605847 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99580039d0 0x7f9958003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.102+0000 7f996be00640 1 -- 172.21.15.46:0/2444605847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99580052b0 msgr2=0x7f995800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.102+0000 7f996be00640 1 --2- 172.21.15.46:0/2444605847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99580052b0 0x7f995800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.102+0000 7f996be00640 1 -- 172.21.15.46:0/2444605847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9958003530 con 0x7f99580047d0 2024-03-29T08:29:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.102+0000 7f9979c00640 1 -- 172.21.15.46:0/2444605847 reap_dead start 2024-03-29T08:29:12.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.102+0000 7f996be00640 1 --2- 172.21.15.46:0/2444605847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99580047d0 0x7f9958004c50 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f995c001780 tx=0x7f995c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aece74582d3969ef server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:12.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.102+0000 7f9973e00640 1 -- 172.21.15.46:0/2444605847 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c0052d0 con 0x7f99580047d0 2024-03-29T08:29:12.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.102+0000 7f9973e00640 1 -- 172.21.15.46:0/2444605847 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c005450 con 0x7f99580047d0 2024-03-29T08:29:12.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.102+0000 7f9973e00640 1 -- 172.21.15.46:0/2444605847 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c003840 con 0x7f99580047d0 2024-03-29T08:29:12.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.102+0000 7f997bc00640 1 -- 172.21.15.46:0/2444605847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99580047d0 msgr2=0x7f9958004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:12.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.102+0000 7f997bc00640 1 --2- 172.21.15.46:0/2444605847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99580047d0 0x7f9958004c50 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f995c001780 tx=0x7f995c01c3d0 comp rx=0 tx=0).stop 2024-03-29T08:29:12.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.102+0000 7f997bc00640 1 -- 172.21.15.46:0/2444605847 shutdown_connections 2024-03-29T08:29:12.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.102+0000 7f997bc00640 1 --2- 172.21.15.46:0/2444605847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99580047d0 0x7f9958004c50 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:12.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.102+0000 7f997bc00640 1 -- 172.21.15.46:0/2444605847 >> 172.21.15.46:0/2444605847 conn(0x7f99740b21d0 msgr2=0x7f99740b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:12.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.102+0000 7f997bc00640 1 -- 172.21.15.46:0/2444605847 shutdown_connections 2024-03-29T08:29:12.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f997bc00640 1 -- 172.21.15.46:0/2444605847 wait complete. 2024-03-29T08:29:12.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f997bc00640 1 Processor -- start 2024-03-29T08:29:12.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f997bc00640 1 -- start start 2024-03-29T08:29:12.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f997bc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99580047d0 0x7f997415fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:12.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f997bc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99740ca0d0 0x7f99740c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:12.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f997bc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99740c88f0 0x7f99740ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:12.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f997bc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99740c9d30 con 0x7f99740c88f0 2024-03-29T08:29:12.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f997bc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99740ca5d0 con 0x7f99740ca0d0 2024-03-29T08:29:12.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f997bc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9974158fd0 con 0x7f99580047d0 2024-03-29T08:29:12.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f9979c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99740c88f0 0x7f99740ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:12.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f9979c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99740c88f0 0x7f99740ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33586/0 (socket says 172.21.15.46:33586) 2024-03-29T08:29:12.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f9979c00640 1 -- 172.21.15.46:0/849289283 learned_addr learned my addr 172.21.15.46:0/849289283 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:12.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f9979200640 1 --2- 172.21.15.46:0/849289283 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99580047d0 0x7f997415fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:12.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f996be00640 1 --2- 172.21.15.46:0/849289283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99740ca0d0 0x7f99740c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:12.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f9979c00640 1 -- 172.21.15.46:0/849289283 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99580047d0 msgr2=0x7f997415fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:12.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f9979c00640 1 --2- 172.21.15.46:0/849289283 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99580047d0 0x7f997415fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:12.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f9979c00640 1 -- 172.21.15.46:0/849289283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99740ca0d0 msgr2=0x7f99740c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:12.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f9979c00640 1 --2- 172.21.15.46:0/849289283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99740ca0d0 0x7f99740c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:12.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f9979c00640 1 -- 172.21.15.46:0/849289283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9974158a10 con 0x7f99740c88f0 2024-03-29T08:29:12.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f9979c00640 1 --2- 172.21.15.46:0/849289283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99740c88f0 0x7f99740ada30 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f9974073540 tx=0x7f996c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f9971600640 1 -- 172.21.15.46:0/849289283 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c00e040 con 0x7f99740c88f0 2024-03-29T08:29:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f9971600640 1 -- 172.21.15.46:0/849289283 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c00f040 con 0x7f99740c88f0 2024-03-29T08:29:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f9971600640 1 -- 172.21.15.46:0/849289283 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c015430 con 0x7f99740c88f0 2024-03-29T08:29:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.106+0000 7f997bc00640 1 -- 172.21.15.46:0/849289283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9974158cf0 con 0x7f99740c88f0 2024-03-29T08:29:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.110+0000 7f997bc00640 1 -- 172.21.15.46:0/849289283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99740c8da0 con 0x7f99740c88f0 2024-03-29T08:29:12.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.110+0000 7f997bc00640 1 -- 172.21.15.46:0/849289283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9974066c90 con 0x7f99740c88f0 2024-03-29T08:29:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.110+0000 7f9971600640 1 -- 172.21.15.46:0/849289283 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f996c015590 con 0x7f99740c88f0 2024-03-29T08:29:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.110+0000 7f9971600640 1 --2- 172.21.15.46:0/849289283 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99500666c0 0x7f9950068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.110+0000 7f9971600640 1 -- 172.21.15.46:0/849289283 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f996c084880 con 0x7f99740c88f0 2024-03-29T08:29:12.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.110+0000 7f9979200640 1 --2- 172.21.15.46:0/849289283 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99500666c0 0x7f9950068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:12.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.110+0000 7f9979200640 1 --2- 172.21.15.46:0/849289283 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99500666c0 0x7f9950068b80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9964000c10 tx=0x7f9964000ff0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:12.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.114+0000 7f9971600640 1 -- 172.21.15.46:0/849289283 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f996c051fe0 con 0x7f99740c88f0 2024-03-29T08:29:12.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:12.278+0000 7f997bc00640 1 -- 172.21.15.46:0/849289283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f997415d920 con 0x7f99740c88f0 2024-03-29T08:29:13.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.054+0000 7f9971600640 1 -- 172.21.15.46:0/849289283 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v118) v1 ==== 138+0+0 (secure 0 0 0) 0x7f996c055600 con 0x7f99740c88f0 2024-03-29T08:29:13.058 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-03-29T08:29:13.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.058+0000 7f997bc00640 1 -- 172.21.15.46:0/849289283 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99500666c0 msgr2=0x7f9950068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:13.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.058+0000 7f997bc00640 1 --2- 172.21.15.46:0/849289283 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99500666c0 0x7f9950068b80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f9964000c10 tx=0x7f9964000ff0 comp rx=0 tx=0).stop 2024-03-29T08:29:13.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.058+0000 7f997bc00640 1 -- 172.21.15.46:0/849289283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99740c88f0 msgr2=0x7f99740ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:13.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.058+0000 7f997bc00640 1 --2- 172.21.15.46:0/849289283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99740c88f0 0x7f99740ada30 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f9974073540 tx=0x7f996c00f580 comp rx=0 tx=0).stop 2024-03-29T08:29:13.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.058+0000 7f997bc00640 1 -- 172.21.15.46:0/849289283 shutdown_connections 2024-03-29T08:29:13.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.058+0000 7f997bc00640 1 --2- 172.21.15.46:0/849289283 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99500666c0 0x7f9950068b80 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:13.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.058+0000 7f997bc00640 1 --2- 172.21.15.46:0/849289283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99740ca0d0 0x7f99740c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:13.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.058+0000 7f997bc00640 1 --2- 172.21.15.46:0/849289283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99740c88f0 0x7f99740ada30 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:13.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.058+0000 7f997bc00640 1 --2- 172.21.15.46:0/849289283 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f99580047d0 0x7f997415fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:13.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.058+0000 7f997bc00640 1 -- 172.21.15.46:0/849289283 >> 172.21.15.46:0/849289283 conn(0x7f99740b21d0 msgr2=0x7f99740b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:13.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.058+0000 7f997bc00640 1 -- 172.21.15.46:0/849289283 shutdown_connections 2024-03-29T08:29:13.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.058+0000 7f997bc00640 1 -- 172.21.15.46:0/849289283 wait complete. 2024-03-29T08:29:13.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-29T08:29:13.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7e3200640 1 Processor -- start 2024-03-29T08:29:13.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7e3200640 1 -- start start 2024-03-29T08:29:13.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7e3200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd7e41433c0 0x7fd7e40ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:13.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7e3200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd7e40cec10 0x7fd7e40b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:13.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7e3200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e40cf050 con 0x7fd7e40b1cf0 2024-03-29T08:29:13.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7e3200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e4159d00 con 0x7fd7e41433c0 2024-03-29T08:29:13.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7e3200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e41437a0 con 0x7fd7e40cec10 2024-03-29T08:29:13.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7e2800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd7e40b1cf0 0x7fd7e415d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59208/0 (socket says 172.21.15.46:59208) 2024-03-29T08:29:13.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7e2800640 1 -- 172.21.15.46:0/2450096816 learned_addr learned my addr 172.21.15.46:0/2450096816 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:13.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7e1e00640 1 --2- 172.21.15.46:0/2450096816 >> v2:172.21.15.73:3300/0 conn(0x7fd7e41433c0 0x7fd7e40ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:13.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7d9400640 1 --2- 172.21.15.46:0/2450096816 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd7e40cec10 0x7fd7e40b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:13.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7e1400640 1 -- 172.21.15.46:0/2450096816 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211159101 0 0) 0x7fd7e40cf050 con 0x7fd7e40b1cf0 2024-03-29T08:29:13.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7e1400640 1 -- 172.21.15.46:0/2450096816 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7c8003530 con 0x7fd7e40b1cf0 2024-03-29T08:29:13.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7d9400640 1 -- 172.21.15.46:0/2450096816 >> v2:172.21.15.73:3300/0 conn(0x7fd7e41433c0 msgr2=0x7fd7e40ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7d9400640 1 --2- 172.21.15.46:0/2450096816 >> v2:172.21.15.73:3300/0 conn(0x7fd7e41433c0 0x7fd7e40ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7d9400640 1 -- 172.21.15.46:0/2450096816 >> v1:172.21.15.46:6789/0 conn(0x7fd7e40b1cf0 legacy=0x7fd7e415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7d9400640 1 -- 172.21.15.46:0/2450096816 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e40705e0 con 0x7fd7e40cec10 2024-03-29T08:29:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.238+0000 7fd7e1e00640 1 --2- 172.21.15.46:0/2450096816 >> v2:172.21.15.73:3300/0 conn(0x7fd7e41433c0 0x7fd7e40ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:29:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.242+0000 7fd7d9400640 1 --2- 172.21.15.46:0/2450096816 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd7e40cec10 0x7fd7e40b17b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0097c0 tx=0x7fd7cc01ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=275bb5e64e97f227 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.242+0000 7fd7e1400640 1 -- 172.21.15.46:0/2450096816 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0a8990 con 0x7fd7e40cec10 2024-03-29T08:29:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.242+0000 7fd7e1400640 1 -- 172.21.15.46:0/2450096816 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7cc0a8af0 con 0x7fd7e40cec10 2024-03-29T08:29:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.242+0000 7fd7e1400640 1 -- 172.21.15.46:0/2450096816 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0a8e80 con 0x7fd7e40cec10 2024-03-29T08:29:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.242+0000 7fd7e3200640 1 -- 172.21.15.46:0/2450096816 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd7e40cec10 msgr2=0x7fd7e40b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.242+0000 7fd7e3200640 1 --2- 172.21.15.46:0/2450096816 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd7e40cec10 0x7fd7e40b17b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0097c0 tx=0x7fd7cc01ec90 comp rx=0 tx=0).stop 2024-03-29T08:29:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.242+0000 7fd7e3200640 1 -- 172.21.15.46:0/2450096816 shutdown_connections 2024-03-29T08:29:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.242+0000 7fd7e3200640 1 --2- 172.21.15.46:0/2450096816 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd7e40cec10 0x7fd7e40b17b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.242+0000 7fd7e3200640 1 --2- 172.21.15.46:0/2450096816 >> v2:172.21.15.73:3300/0 conn(0x7fd7e41433c0 0x7fd7e40ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.242+0000 7fd7e3200640 1 -- 172.21.15.46:0/2450096816 >> 172.21.15.46:0/2450096816 conn(0x7fd7e4066080 msgr2=0x7fd7e4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.242+0000 7fd7e3200640 1 -- 172.21.15.46:0/2450096816 shutdown_connections 2024-03-29T08:29:13.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.242+0000 7fd7e3200640 1 -- 172.21.15.46:0/2450096816 wait complete. 2024-03-29T08:29:13.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.242+0000 7fd7e3200640 1 Processor -- start 2024-03-29T08:29:13.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7e3200640 1 -- start start 2024-03-29T08:29:13.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7e3200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd7e40b1cf0 0x7fd7e4163200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:13.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7e3200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7e40cec10 0x7fd7e406cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:13.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7e3200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7e41433c0 0x7fd7e406d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:13.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7e3200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e415abe0 con 0x7fd7e40cec10 2024-03-29T08:29:13.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7e3200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e4159fd0 con 0x7fd7e41433c0 2024-03-29T08:29:13.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7e3200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e415a2b0 con 0x7fd7e40b1cf0 2024-03-29T08:29:13.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7d9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7e40cec10 0x7fd7e406cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:13.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7d9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7e40cec10 0x7fd7e406cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33602/0 (socket says 172.21.15.46:33602) 2024-03-29T08:29:13.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7d9400640 1 -- 172.21.15.46:0/3856642430 learned_addr learned my addr 172.21.15.46:0/3856642430 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:13.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7e1e00640 1 --2- 172.21.15.46:0/3856642430 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd7e40b1cf0 0x7fd7e4163200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:13.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7e2800640 1 --2- 172.21.15.46:0/3856642430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7e41433c0 0x7fd7e406d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:13.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7d9400640 1 -- 172.21.15.46:0/3856642430 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd7e40b1cf0 msgr2=0x7fd7e4163200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:13.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7d9400640 1 --2- 172.21.15.46:0/3856642430 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd7e40b1cf0 0x7fd7e4163200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:13.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7d9400640 1 -- 172.21.15.46:0/3856642430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7e41433c0 msgr2=0x7fd7e406d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:13.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7d9400640 1 --2- 172.21.15.46:0/3856642430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7e41433c0 0x7fd7e406d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:13.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7d9400640 1 -- 172.21.15.46:0/3856642430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d4009a70 con 0x7fd7e40cec10 2024-03-29T08:29:13.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7d9400640 1 --2- 172.21.15.46:0/3856642430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7e40cec10 0x7fd7e406cf50 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc01c8c0 tx=0x7fd7cc01ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:13.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7daa00640 1 -- 172.21.15.46:0/3856642430 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0b0d60 con 0x7fd7e40cec10 2024-03-29T08:29:13.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7daa00640 1 -- 172.21.15.46:0/3856642430 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7cc0b9410 con 0x7fd7e40cec10 2024-03-29T08:29:13.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7daa00640 1 -- 172.21.15.46:0/3856642430 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0b97c0 con 0x7fd7e40cec10 2024-03-29T08:29:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7e3200640 1 -- 172.21.15.46:0/3856642430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7cc009420 con 0x7fd7e40cec10 2024-03-29T08:29:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7e3200640 1 -- 172.21.15.46:0/3856642430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e415a870 con 0x7fd7e40cec10 2024-03-29T08:29:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.246+0000 7fd7e3200640 1 -- 172.21.15.46:0/3856642430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e404e040 con 0x7fd7e40cec10 2024-03-29T08:29:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.250+0000 7fd7daa00640 1 -- 172.21.15.46:0/3856642430 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd7cc0b7040 con 0x7fd7e40cec10 2024-03-29T08:29:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.250+0000 7fd7daa00640 1 --2- 172.21.15.46:0/3856642430 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7c00666c0 0x7fd7c0068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.250+0000 7fd7daa00640 1 -- 172.21.15.46:0/3856642430 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fd7cc121b30 con 0x7fd7e40cec10 2024-03-29T08:29:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.250+0000 7fd7e1e00640 1 --2- 172.21.15.46:0/3856642430 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7c00666c0 0x7fd7c0068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:13.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.250+0000 7fd7e1e00640 1 --2- 172.21.15.46:0/3856642430 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7c00666c0 0x7fd7c0068b80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4066ec0 tx=0x7fd7d400b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:13.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.254+0000 7fd7daa00640 1 -- 172.21.15.46:0/3856642430 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd7cc0ef100 con 0x7fd7e40cec10 2024-03-29T08:29:13.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:13.414+0000 7fd7e3200640 1 -- 172.21.15.46:0/3856642430 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd7e406ca20 con 0x7fd7e40cec10 2024-03-29T08:29:14.066 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-03-29T08:29:14.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.062+0000 7fd7daa00640 1 -- 172.21.15.46:0/3856642430 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v119) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd7cc0b50d0 con 0x7fd7e40cec10 2024-03-29T08:29:14.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.070+0000 7fd7e3200640 1 -- 172.21.15.46:0/3856642430 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7c00666c0 msgr2=0x7fd7c0068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:14.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.070+0000 7fd7e3200640 1 --2- 172.21.15.46:0/3856642430 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7c00666c0 0x7fd7c0068b80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4066ec0 tx=0x7fd7d400b040 comp rx=0 tx=0).stop 2024-03-29T08:29:14.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.070+0000 7fd7e3200640 1 -- 172.21.15.46:0/3856642430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7e40cec10 msgr2=0x7fd7e406cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:14.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.070+0000 7fd7e3200640 1 --2- 172.21.15.46:0/3856642430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7e40cec10 0x7fd7e406cf50 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc01c8c0 tx=0x7fd7cc01ba30 comp rx=0 tx=0).stop 2024-03-29T08:29:14.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.070+0000 7fd7e3200640 1 -- 172.21.15.46:0/3856642430 shutdown_connections 2024-03-29T08:29:14.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.070+0000 7fd7e3200640 1 --2- 172.21.15.46:0/3856642430 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7c00666c0 0x7fd7c0068b80 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.070+0000 7fd7e3200640 1 --2- 172.21.15.46:0/3856642430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7e41433c0 0x7fd7e406d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.070+0000 7fd7e3200640 1 --2- 172.21.15.46:0/3856642430 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7e40cec10 0x7fd7e406cf50 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.070+0000 7fd7e3200640 1 --2- 172.21.15.46:0/3856642430 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd7e40b1cf0 0x7fd7e4163200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.070+0000 7fd7e3200640 1 -- 172.21.15.46:0/3856642430 >> 172.21.15.46:0/3856642430 conn(0x7fd7e4066080 msgr2=0x7fd7e40cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:14.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.074+0000 7fd7e3200640 1 -- 172.21.15.46:0/3856642430 shutdown_connections 2024-03-29T08:29:14.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.074+0000 7fd7e3200640 1 -- 172.21.15.46:0/3856642430 wait complete. 2024-03-29T08:29:14.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-29T08:29:14.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.250+0000 7ff2b2c00640 1 Processor -- start 2024-03-29T08:29:14.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.250+0000 7ff2b2c00640 1 -- start start 2024-03-29T08:29:14.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.254+0000 7ff2b2c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff2ac0cd3d0 0x7ff2ac0cba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:14.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.254+0000 7ff2b2c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2ac06ff00 con 0x7ff2ac0ccf60 2024-03-29T08:29:14.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.254+0000 7ff2b2c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2ac0cf590 con 0x7ff2ac15a9f0 2024-03-29T08:29:14.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.254+0000 7ff2b2c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2ac06e130 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.254+0000 7ff2b2200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff2ac0cd3d0 0x7ff2ac0cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:14.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.254+0000 7ff2b2200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff2ac0cd3d0 0x7ff2ac0cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33612/0 (socket says 172.21.15.46:33612) 2024-03-29T08:29:14.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.254+0000 7ff2b2200640 1 -- 172.21.15.46:0/2444555773 learned_addr learned my addr 172.21.15.46:0/2444555773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:14.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.254+0000 7ff2b2200640 1 -- 172.21.15.46:0/2444555773 >> v1:172.21.15.73:6789/0 conn(0x7ff2ac15a9f0 legacy=0x7ff2ac0ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:14.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.254+0000 7ff2b2200640 1 -- 172.21.15.46:0/2444555773 >> v1:172.21.15.46:6789/0 conn(0x7ff2ac0ccf60 legacy=0x7ff2ac0cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:14.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.254+0000 7ff2b2200640 1 -- 172.21.15.46:0/2444555773 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac06e410 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b2200640 1 --2- 172.21.15.46:0/2444555773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2ac0cd3d0 0x7ff2ac0cba90 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009e50 tx=0x7ff2a401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8625906a00021f1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:14.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2abe00640 1 -- 172.21.15.46:0/2444555773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40af400 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2abe00640 1 -- 172.21.15.46:0/2444555773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a40ab050 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2abe00640 1 -- 172.21.15.46:0/2444555773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40af780 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b2c00640 1 -- 172.21.15.46:0/2444555773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2ac0cd3d0 msgr2=0x7ff2ac0cba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:14.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b2c00640 1 --2- 172.21.15.46:0/2444555773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2ac0cd3d0 0x7ff2ac0cba90 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009e50 tx=0x7ff2a401f210 comp rx=0 tx=0).stop 2024-03-29T08:29:14.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b2c00640 1 -- 172.21.15.46:0/2444555773 shutdown_connections 2024-03-29T08:29:14.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b2c00640 1 --2- 172.21.15.46:0/2444555773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2ac0cd3d0 0x7ff2ac0cba90 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b2c00640 1 -- 172.21.15.46:0/2444555773 >> 172.21.15.46:0/2444555773 conn(0x7ff2ac0d0550 msgr2=0x7ff2ac0d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:14.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b2c00640 1 -- 172.21.15.46:0/2444555773 shutdown_connections 2024-03-29T08:29:14.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b2c00640 1 -- 172.21.15.46:0/2444555773 wait complete. 2024-03-29T08:29:14.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b2c00640 1 Processor -- start 2024-03-29T08:29:14.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b2c00640 1 -- start start 2024-03-29T08:29:14.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b2c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2ac0ccf60 0x7ff2ac0b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:14.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b2c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2ac0cd3d0 0x7ff2ac158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:14.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b2c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2ac15a9f0 0x7ff2ac0afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:14.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b2c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0c4560 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b2c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0b2af0 con 0x7ff2ac15a9f0 2024-03-29T08:29:14.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b2c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0b3170 con 0x7ff2ac0ccf60 2024-03-29T08:29:14.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2ac0cd3d0 0x7ff2ac158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:14.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.258+0000 7ff2b0e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2ac0cd3d0 0x7ff2ac158c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33616/0 (socket says 172.21.15.46:33616) 2024-03-29T08:29:14.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.262+0000 7ff2b0e00640 1 -- 172.21.15.46:0/4134636794 learned_addr learned my addr 172.21.15.46:0/4134636794 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:14.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.262+0000 7ff2b2200640 1 --2- 172.21.15.46:0/4134636794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2ac15a9f0 0x7ff2ac0afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:14.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.262+0000 7ff2b1800640 1 --2- 172.21.15.46:0/4134636794 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2ac0ccf60 0x7ff2ac0b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:14.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.262+0000 7ff2b0e00640 1 -- 172.21.15.46:0/4134636794 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2ac0ccf60 msgr2=0x7ff2ac0b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:14.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.262+0000 7ff2b0e00640 1 --2- 172.21.15.46:0/4134636794 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2ac0ccf60 0x7ff2ac0b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.262+0000 7ff2b0e00640 1 -- 172.21.15.46:0/4134636794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2ac15a9f0 msgr2=0x7ff2ac0afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:14.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.262+0000 7ff2b0e00640 1 --2- 172.21.15.46:0/4134636794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2ac15a9f0 0x7ff2ac0afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.262+0000 7ff2b0e00640 1 -- 172.21.15.46:0/4134636794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a4009b30 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.262+0000 7ff2b0e00640 1 --2- 172.21.15.46:0/4134636794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2ac0cd3d0 0x7ff2ac158c50 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7ff29800c3f0 tx=0x7ff29800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:14.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.262+0000 7ff2a9600640 1 -- 172.21.15.46:0/4134636794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2980052d0 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.262+0000 7ff2a9600640 1 -- 172.21.15.46:0/4134636794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff298005430 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.262+0000 7ff2a9600640 1 -- 172.21.15.46:0/4134636794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298003840 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.262+0000 7ff2b2c00640 1 -- 172.21.15.46:0/4134636794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ac0b2530 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.262+0000 7ff2b2c00640 1 -- 172.21.15.46:0/4134636794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ac0c4180 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.262+0000 7ff2b2c00640 1 -- 172.21.15.46:0/4134636794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ac0c3fe0 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.262+0000 7ff2a9600640 1 -- 172.21.15.46:0/4134636794 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff2980039a0 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.266+0000 7ff2a9600640 1 --2- 172.21.15.46:0/4134636794 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff28c066690 0x7ff28c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:14.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.266+0000 7ff2a9600640 1 -- 172.21.15.46:0/4134636794 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5609+0+0 (secure 0 0 0) 0x7ff298083840 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.266+0000 7ff2b1800640 1 --2- 172.21.15.46:0/4134636794 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff28c066690 0x7ff28c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:14.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.266+0000 7ff2b1800640 1 --2- 172.21.15.46:0/4134636794 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff28c066690 0x7ff28c068b50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0007fd0 tx=0x7ff2a0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:14.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.266+0000 7ff2a9600640 1 -- 172.21.15.46:0/4134636794 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff298050d30 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.466+0000 7ff2b2c00640 1 -- 172.21.15.46:0/4134636794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff2ac0c3fe0 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.474+0000 7ff2a9600640 1 -- 172.21.15.46:0/4134636794 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v90) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff2ac0c3fe0 con 0x7ff2ac0cd3d0 2024-03-29T08:29:14.477 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 16 and data pool 17 2024-03-29T08:29:14.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.478+0000 7ff2b2c00640 1 -- 172.21.15.46:0/4134636794 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff28c066690 msgr2=0x7ff28c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:14.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.478+0000 7ff2b2c00640 1 --2- 172.21.15.46:0/4134636794 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff28c066690 0x7ff28c068b50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0007fd0 tx=0x7ff2a0009040 comp rx=0 tx=0).stop 2024-03-29T08:29:14.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.478+0000 7ff2b2c00640 1 -- 172.21.15.46:0/4134636794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2ac0cd3d0 msgr2=0x7ff2ac158c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:14.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.478+0000 7ff2b2c00640 1 --2- 172.21.15.46:0/4134636794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2ac0cd3d0 0x7ff2ac158c50 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7ff29800c3f0 tx=0x7ff29800c8b0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.482+0000 7ff2b2c00640 1 -- 172.21.15.46:0/4134636794 shutdown_connections 2024-03-29T08:29:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.482+0000 7ff2b2c00640 1 --2- 172.21.15.46:0/4134636794 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff28c066690 0x7ff28c068b50 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.482+0000 7ff2b2c00640 1 --2- 172.21.15.46:0/4134636794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2ac15a9f0 0x7ff2ac0afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.482+0000 7ff2b2c00640 1 --2- 172.21.15.46:0/4134636794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2ac0cd3d0 0x7ff2ac158c50 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.482+0000 7ff2b2c00640 1 --2- 172.21.15.46:0/4134636794 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff2ac0ccf60 0x7ff2ac0b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.482+0000 7ff2b2c00640 1 -- 172.21.15.46:0/4134636794 >> 172.21.15.46:0/4134636794 conn(0x7ff2ac0d0550 msgr2=0x7ff2ac0ad5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:14.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.486+0000 7ff2b2c00640 1 -- 172.21.15.46:0/4134636794 shutdown_connections 2024-03-29T08:29:14.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.486+0000 7ff2b2c00640 1 -- 172.21.15.46:0/4134636794 wait complete. 2024-03-29T08:29:14.523 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:29:14.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.658+0000 7f72c1000640 1 Processor -- start 2024-03-29T08:29:14.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.658+0000 7f72c1000640 1 -- start start 2024-03-29T08:29:14.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.662+0000 7f72c1000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f72bc151880 0x7f72bc06b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:14.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.662+0000 7f72c1000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f72bc071100 0x7f72bc15e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:14.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.662+0000 7f72c1000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72bc06b760 con 0x7f72bc0ccc50 2024-03-29T08:29:14.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.662+0000 7f72c1000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72bc15bea0 con 0x7f72bc151880 2024-03-29T08:29:14.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.662+0000 7f72c1000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72bc15b290 con 0x7f72bc071100 2024-03-29T08:29:14.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.662+0000 7f72b9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f72bc151880 0x7f72bc06b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:14.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.662+0000 7f72b9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f72bc151880 0x7f72bc06b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33624/0 (socket says 172.21.15.46:33624) 2024-03-29T08:29:14.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.662+0000 7f72b9e00640 1 -- 172.21.15.46:0/2001590758 learned_addr learned my addr 172.21.15.46:0/2001590758 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:14.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.662+0000 7f72ba800640 1 --2- 172.21.15.46:0/2001590758 >> v2:172.21.15.73:3300/0 conn(0x7f72bc071100 0x7f72bc15e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:14.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.662+0000 7f72b9e00640 1 -- 172.21.15.46:0/2001590758 >> v2:172.21.15.73:3300/0 conn(0x7f72bc071100 msgr2=0x7f72bc15e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:14.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.662+0000 7f72b9e00640 1 --2- 172.21.15.46:0/2001590758 >> v2:172.21.15.73:3300/0 conn(0x7f72bc071100 0x7f72bc15e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.662+0000 7f72b9e00640 1 -- 172.21.15.46:0/2001590758 >> v1:172.21.15.73:6789/0 conn(0x7f72bc0ccc50 legacy=0x7f72bc0709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:14.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.662+0000 7f72b9e00640 1 -- 172.21.15.46:0/2001590758 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72bc15b570 con 0x7f72bc151880 2024-03-29T08:29:14.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.666+0000 7f72b9e00640 1 --2- 172.21.15.46:0/2001590758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc151880 0x7f72bc06b220 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0cfd30 tx=0x7f72ac01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77553fc51e7e9fc server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:14.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.666+0000 7f72b9400640 1 -- 172.21.15.46:0/2001590758 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0ab660 con 0x7f72bc151880 2024-03-29T08:29:14.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.666+0000 7f72b9400640 1 -- 172.21.15.46:0/2001590758 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ac0ab7c0 con 0x7f72bc151880 2024-03-29T08:29:14.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.666+0000 7f72b9400640 1 -- 172.21.15.46:0/2001590758 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0abb50 con 0x7f72bc151880 2024-03-29T08:29:14.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.666+0000 7f72c1000640 1 -- 172.21.15.46:0/2001590758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc151880 msgr2=0x7f72bc06b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:14.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.666+0000 7f72c1000640 1 --2- 172.21.15.46:0/2001590758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc151880 0x7f72bc06b220 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f72bc0cfd30 tx=0x7f72ac01c460 comp rx=0 tx=0).stop 2024-03-29T08:29:14.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.666+0000 7f72c1000640 1 -- 172.21.15.46:0/2001590758 shutdown_connections 2024-03-29T08:29:14.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.666+0000 7f72c1000640 1 --2- 172.21.15.46:0/2001590758 >> v2:172.21.15.73:3300/0 conn(0x7f72bc071100 0x7f72bc15e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.666+0000 7f72c1000640 1 --2- 172.21.15.46:0/2001590758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc151880 0x7f72bc06b220 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.666+0000 7f72c1000640 1 -- 172.21.15.46:0/2001590758 >> 172.21.15.46:0/2001590758 conn(0x7f72bc0ced50 msgr2=0x7f72bc0cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:14.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.666+0000 7f72c1000640 1 -- 172.21.15.46:0/2001590758 shutdown_connections 2024-03-29T08:29:14.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.666+0000 7f72c1000640 1 -- 172.21.15.46:0/2001590758 wait complete. 2024-03-29T08:29:14.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.666+0000 7f72c1000640 1 Processor -- start 2024-03-29T08:29:14.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.666+0000 7f72c1000640 1 -- start start 2024-03-29T08:29:14.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.666+0000 7f72c1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc071100 0x7f72bc1f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:14.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.666+0000 7f72c1000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f72bc0ccc50 0x7f72bc06aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:14.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.666+0000 7f72c1000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72bc151880 0x7f72bc06b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:14.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72c1000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc1f5260 con 0x7f72bc071100 2024-03-29T08:29:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72c1000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc14f890 con 0x7f72bc151880 2024-03-29T08:29:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72c1000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72bc14ec80 con 0x7f72bc0ccc50 2024-03-29T08:29:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc071100 0x7f72bc1f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72b9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc071100 0x7f72bc1f4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33640/0 (socket says 172.21.15.46:33640) 2024-03-29T08:29:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72b9e00640 1 -- 172.21.15.46:0/1082020966 learned_addr learned my addr 172.21.15.46:0/1082020966 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72b1400640 1 --2- 172.21.15.46:0/1082020966 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f72bc0ccc50 0x7f72bc06aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72ba800640 1 --2- 172.21.15.46:0/1082020966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72bc151880 0x7f72bc06b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72b9e00640 1 -- 172.21.15.46:0/1082020966 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f72bc0ccc50 msgr2=0x7f72bc06aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72b9e00640 1 --2- 172.21.15.46:0/1082020966 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f72bc0ccc50 0x7f72bc06aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72b9e00640 1 -- 172.21.15.46:0/1082020966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72bc151880 msgr2=0x7f72bc06b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72b9e00640 1 --2- 172.21.15.46:0/1082020966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72bc151880 0x7f72bc06b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72b9e00640 1 -- 172.21.15.46:0/1082020966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ac0094d0 con 0x7f72bc071100 2024-03-29T08:29:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72b9e00640 1 --2- 172.21.15.46:0/1082020966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc071100 0x7f72bc1f4d20 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f72ac019070 tx=0x7f72ac01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:14.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72b2a00640 1 -- 172.21.15.46:0/1082020966 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0b4800 con 0x7f72bc071100 2024-03-29T08:29:14.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72b2a00640 1 -- 172.21.15.46:0/1082020966 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ac0b4980 con 0x7f72bc071100 2024-03-29T08:29:14.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72b2a00640 1 -- 172.21.15.46:0/1082020966 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac0b4d30 con 0x7f72bc071100 2024-03-29T08:29:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72c1000640 1 -- 172.21.15.46:0/1082020966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72bc14ef60 con 0x7f72bc071100 2024-03-29T08:29:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72c1000640 1 -- 172.21.15.46:0/1082020966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72bc14f520 con 0x7f72bc071100 2024-03-29T08:29:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72c1000640 1 -- 172.21.15.46:0/1082020966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72bc0cd030 con 0x7f72bc071100 2024-03-29T08:29:14.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.670+0000 7f72b2a00640 1 -- 172.21.15.46:0/1082020966 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f72ac0a9070 con 0x7f72bc071100 2024-03-29T08:29:14.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.674+0000 7f72b2a00640 1 --2- 172.21.15.46:0/1082020966 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7298066690 0x7f7298068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:14.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.674+0000 7f72b2a00640 1 -- 172.21.15.46:0/1082020966 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72ac11e5b0 con 0x7f72bc071100 2024-03-29T08:29:14.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.674+0000 7f72b1400640 1 --2- 172.21.15.46:0/1082020966 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7298066690 0x7f7298068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:14.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.674+0000 7f72b1400640 1 --2- 172.21.15.46:0/1082020966 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7298066690 0x7f7298068b50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f72bc14aca0 tx=0x7f72a4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:14.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.678+0000 7f72b2a00640 1 -- 172.21.15.46:0/1082020966 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f72bc0cd030 con 0x7f72bc071100 2024-03-29T08:29:14.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.838+0000 7f72c1000640 1 -- 172.21.15.46:0/1082020966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72bc0cd030 con 0x7f72bc071100 2024-03-29T08:29:14.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.838+0000 7f72b2a00640 1 -- 172.21.15.46:0/1082020966 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16080 (secure 0 0 0) 0x7f72bc0cd030 con 0x7f72bc071100 2024-03-29T08:29:14.842 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:14.843 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":120,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:29:14.470547+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:29:12.329121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-29T08:29:13.465939+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:29:14.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.842+0000 7f72c1000640 1 -- 172.21.15.46:0/1082020966 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7298066690 msgr2=0x7f7298068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:14.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.842+0000 7f72c1000640 1 --2- 172.21.15.46:0/1082020966 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7298066690 0x7f7298068b50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f72bc14aca0 tx=0x7f72a4009040 comp rx=0 tx=0).stop 2024-03-29T08:29:14.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.842+0000 7f72c1000640 1 -- 172.21.15.46:0/1082020966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc071100 msgr2=0x7f72bc1f4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:14.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.842+0000 7f72c1000640 1 --2- 172.21.15.46:0/1082020966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc071100 0x7f72bc1f4d20 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f72ac019070 tx=0x7f72ac01aca0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.842+0000 7f72c1000640 1 -- 172.21.15.46:0/1082020966 shutdown_connections 2024-03-29T08:29:14.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.842+0000 7f72c1000640 1 --2- 172.21.15.46:0/1082020966 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7298066690 0x7f7298068b50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.842+0000 7f72c1000640 1 --2- 172.21.15.46:0/1082020966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72bc151880 0x7f72bc06b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.842+0000 7f72c1000640 1 --2- 172.21.15.46:0/1082020966 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f72bc0ccc50 0x7f72bc06aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.842+0000 7f72c1000640 1 --2- 172.21.15.46:0/1082020966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72bc071100 0x7f72bc1f4d20 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:14.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.842+0000 7f72c1000640 1 -- 172.21.15.46:0/1082020966 >> 172.21.15.46:0/1082020966 conn(0x7f72bc0ced50 msgr2=0x7f72bc0709f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:14.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.842+0000 7f72c1000640 1 -- 172.21.15.46:0/1082020966 shutdown_connections 2024-03-29T08:29:14.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:14.842+0000 7f72c1000640 1 -- 172.21.15.46:0/1082020966 wait complete. 2024-03-29T08:29:14.865 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:29:15.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.002+0000 7f698ce00640 1 Processor -- start 2024-03-29T08:29:15.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.002+0000 7f698ce00640 1 -- start start 2024-03-29T08:29:15.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.002+0000 7f698ce00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f69881605f0 0x7f6988069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:15.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.002+0000 7f698ce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6988160a80 0x7f698806a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:15.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.002+0000 7f698ce00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6988161020 con 0x7f698815feb0 2024-03-29T08:29:15.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.002+0000 7f698ce00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6988161300 con 0x7f6988160a80 2024-03-29T08:29:15.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.002+0000 7f698ce00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6988070b70 con 0x7f69881605f0 2024-03-29T08:29:15.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.002+0000 7f6986800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6988160a80 0x7f698806a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:15.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.002+0000 7f6986800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6988160a80 0x7f698806a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33648/0 (socket says 172.21.15.46:33648) 2024-03-29T08:29:15.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.002+0000 7f6986800640 1 -- 172.21.15.46:0/3628685020 learned_addr learned my addr 172.21.15.46:0/3628685020 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:15.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.002+0000 7f6985e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f698815feb0 0x7f6988071370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59220/0 (socket says 172.21.15.46:59220) 2024-03-29T08:29:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.002+0000 7f697d400640 1 --2- 172.21.15.46:0/3628685020 >> v2:172.21.15.73:3300/0 conn(0x7f69881605f0 0x7f6988069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.006+0000 7f6986800640 1 -- 172.21.15.46:0/3628685020 >> v2:172.21.15.73:3300/0 conn(0x7f69881605f0 msgr2=0x7f6988069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.006+0000 7f6986800640 1 --2- 172.21.15.46:0/3628685020 >> v2:172.21.15.73:3300/0 conn(0x7f69881605f0 0x7f6988069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.006+0000 7f6986800640 1 -- 172.21.15.46:0/3628685020 >> v1:172.21.15.46:6789/0 conn(0x7f698815feb0 legacy=0x7f6988071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.006+0000 7f6986800640 1 -- 172.21.15.46:0/3628685020 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6988070e50 con 0x7f6988160a80 2024-03-29T08:29:15.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.006+0000 7f6985400640 1 -- 172.21.15.46:0/3628685020 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013031168 0 0) 0x7f6988161020 con 0x7f698815feb0 2024-03-29T08:29:15.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.006+0000 7f6986800640 1 --2- 172.21.15.46:0/3628685020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6988160a80 0x7f698806a2d0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f6980009870 tx=0x7f698001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ade2064aed0fa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:15.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.006+0000 7f6985400640 1 -- 172.21.15.46:0/3628685020 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69800a9400 con 0x7f6988160a80 2024-03-29T08:29:15.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.006+0000 7f6985400640 1 -- 172.21.15.46:0/3628685020 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69800a9560 con 0x7f6988160a80 2024-03-29T08:29:15.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.006+0000 7f6985400640 1 -- 172.21.15.46:0/3628685020 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69800a98f0 con 0x7f6988160a80 2024-03-29T08:29:15.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.006+0000 7f698ce00640 1 -- 172.21.15.46:0/3628685020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6988160a80 msgr2=0x7f698806a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:15.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.006+0000 7f698ce00640 1 --2- 172.21.15.46:0/3628685020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6988160a80 0x7f698806a2d0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f6980009870 tx=0x7f698001c520 comp rx=0 tx=0).stop 2024-03-29T08:29:15.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.006+0000 7f698ce00640 1 -- 172.21.15.46:0/3628685020 shutdown_connections 2024-03-29T08:29:15.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.006+0000 7f698ce00640 1 --2- 172.21.15.46:0/3628685020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6988160a80 0x7f698806a2d0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:15.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.006+0000 7f698ce00640 1 --2- 172.21.15.46:0/3628685020 >> v2:172.21.15.73:3300/0 conn(0x7f69881605f0 0x7f6988069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:15.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.006+0000 7f698ce00640 1 -- 172.21.15.46:0/3628685020 >> 172.21.15.46:0/3628685020 conn(0x7f69880ced50 msgr2=0x7f69880cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:15.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.006+0000 7f698ce00640 1 -- 172.21.15.46:0/3628685020 shutdown_connections 2024-03-29T08:29:15.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.006+0000 7f698ce00640 1 -- 172.21.15.46:0/3628685020 wait complete. 2024-03-29T08:29:15.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f698ce00640 1 Processor -- start 2024-03-29T08:29:15.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f698ce00640 1 -- start start 2024-03-29T08:29:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f698ce00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f698815feb0 0x7f6988150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f698ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69881605f0 0x7f69880acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f698ce00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6988160a80 0x7f69880ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f698ce00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698815d2d0 con 0x7f69881605f0 2024-03-29T08:29:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f698ce00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6988157350 con 0x7f6988160a80 2024-03-29T08:29:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f698ce00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6988152790 con 0x7f698815feb0 2024-03-29T08:29:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f697d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69881605f0 0x7f69880acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f697d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69881605f0 0x7f69880acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33658/0 (socket says 172.21.15.46:33658) 2024-03-29T08:29:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f697d400640 1 -- 172.21.15.46:0/2989626523 learned_addr learned my addr 172.21.15.46:0/2989626523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f6985e00640 1 --2- 172.21.15.46:0/2989626523 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f698815feb0 0x7f6988150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f6986800640 1 --2- 172.21.15.46:0/2989626523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6988160a80 0x7f69880ad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f697d400640 1 -- 172.21.15.46:0/2989626523 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f698815feb0 msgr2=0x7f6988150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f697d400640 1 --2- 172.21.15.46:0/2989626523 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f698815feb0 0x7f6988150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:15.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f697d400640 1 -- 172.21.15.46:0/2989626523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6988160a80 msgr2=0x7f69880ad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f697d400640 1 --2- 172.21.15.46:0/2989626523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6988160a80 0x7f69880ad310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f697d400640 1 -- 172.21.15.46:0/2989626523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69800094d0 con 0x7f69881605f0 2024-03-29T08:29:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f697d400640 1 --2- 172.21.15.46:0/2989626523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69881605f0 0x7f69880acdd0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f697000efc0 tx=0x7f697000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:15.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f697ea00640 1 -- 172.21.15.46:0/2989626523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6970014480 con 0x7f69881605f0 2024-03-29T08:29:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f697ea00640 1 -- 172.21.15.46:0/2989626523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697000f040 con 0x7f69881605f0 2024-03-29T08:29:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f698ce00640 1 -- 172.21.15.46:0/2989626523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6988156d00 con 0x7f69881605f0 2024-03-29T08:29:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f697ea00640 1 -- 172.21.15.46:0/2989626523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6970014800 con 0x7f69881605f0 2024-03-29T08:29:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.010+0000 7f698ce00640 1 -- 172.21.15.46:0/2989626523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6988151390 con 0x7f69881605f0 2024-03-29T08:29:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.014+0000 7f698ce00640 1 -- 172.21.15.46:0/2989626523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6988066c90 con 0x7f69881605f0 2024-03-29T08:29:15.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.014+0000 7f697ea00640 1 -- 172.21.15.46:0/2989626523 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f6970014960 con 0x7f69881605f0 2024-03-29T08:29:15.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.014+0000 7f697ea00640 1 --2- 172.21.15.46:0/2989626523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6964066690 0x7f6964068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:15.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.014+0000 7f697ea00640 1 -- 172.21.15.46:0/2989626523 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6970084fb0 con 0x7f69881605f0 2024-03-29T08:29:15.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.014+0000 7f6985e00640 1 --2- 172.21.15.46:0/2989626523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6964066690 0x7f6964068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:15.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.014+0000 7f6985e00640 1 --2- 172.21.15.46:0/2989626523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6964066690 0x7f6964068b50 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6978007fd0 tx=0x7f6978007d60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:15.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.018+0000 7f697ea00640 1 -- 172.21.15.46:0/2989626523 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6970052330 con 0x7f69881605f0 2024-03-29T08:29:15.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.182+0000 7f698ce00640 1 -- 172.21.15.46:0/2989626523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69880cb540 con 0x7f69881605f0 2024-03-29T08:29:15.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.186+0000 7f697ea00640 1 -- 172.21.15.46:0/2989626523 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16080 (secure 0 0 0) 0x7f6970055950 con 0x7f69881605f0 2024-03-29T08:29:15.188 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:15.188 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":120,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:29:14.470547+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:29:12.329121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-29T08:29:13.465939+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:29:15.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.186+0000 7f698ce00640 1 -- 172.21.15.46:0/2989626523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6964066690 msgr2=0x7f6964068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:15.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.186+0000 7f698ce00640 1 --2- 172.21.15.46:0/2989626523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6964066690 0x7f6964068b50 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6978007fd0 tx=0x7f6978007d60 comp rx=0 tx=0).stop 2024-03-29T08:29:15.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.186+0000 7f698ce00640 1 -- 172.21.15.46:0/2989626523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69881605f0 msgr2=0x7f69880acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:15.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.186+0000 7f698ce00640 1 --2- 172.21.15.46:0/2989626523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69881605f0 0x7f69880acdd0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f697000efc0 tx=0x7f697000c490 comp rx=0 tx=0).stop 2024-03-29T08:29:15.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.186+0000 7f698ce00640 1 -- 172.21.15.46:0/2989626523 shutdown_connections 2024-03-29T08:29:15.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.186+0000 7f698ce00640 1 --2- 172.21.15.46:0/2989626523 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6964066690 0x7f6964068b50 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:15.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.186+0000 7f698ce00640 1 --2- 172.21.15.46:0/2989626523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6988160a80 0x7f69880ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:15.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.186+0000 7f698ce00640 1 --2- 172.21.15.46:0/2989626523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69881605f0 0x7f69880acdd0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:15.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.186+0000 7f698ce00640 1 --2- 172.21.15.46:0/2989626523 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f698815feb0 0x7f6988150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:15.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.186+0000 7f698ce00640 1 -- 172.21.15.46:0/2989626523 >> 172.21.15.46:0/2989626523 conn(0x7f69880ced50 msgr2=0x7f69880aeb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:15.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.186+0000 7f698ce00640 1 -- 172.21.15.46:0/2989626523 shutdown_connections 2024-03-29T08:29:15.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.186+0000 7f698ce00640 1 -- 172.21.15.46:0/2989626523 wait complete. 2024-03-29T08:29:15.209 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-29T08:29:15.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.350+0000 7fec47400640 1 Processor -- start 2024-03-29T08:29:15.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.350+0000 7fec47400640 1 -- start start 2024-03-29T08:29:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.350+0000 7fec47400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fec40160db0 0x7fec400cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.350+0000 7fec47400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec400cfb20 con 0x7fec400cea30 2024-03-29T08:29:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.350+0000 7fec47400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec400cef10 con 0x7fec4015c950 2024-03-29T08:29:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.350+0000 7fec47400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec400cf1f0 con 0x7fec40160db0 2024-03-29T08:29:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.350+0000 7fec37e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fec400cea30 0x7fec401606a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59230/0 (socket says 172.21.15.46:59230) 2024-03-29T08:29:15.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.350+0000 7fec37e00640 1 -- 172.21.15.46:0/673591673 learned_addr learned my addr 172.21.15.46:0/673591673 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.350+0000 7fec45400640 1 --2- 172.21.15.46:0/673591673 >> v2:172.21.15.73:3300/0 conn(0x7fec40160db0 0x7fec400cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.350+0000 7fec3fe00640 1 -- 172.21.15.46:0/673591673 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602218181 0 0) 0x7fec400cfb20 con 0x7fec400cea30 2024-03-29T08:29:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.350+0000 7fec3fe00640 1 -- 172.21.15.46:0/673591673 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec24003530 con 0x7fec400cea30 2024-03-29T08:29:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.350+0000 7fec3fe00640 1 -- 172.21.15.46:0/673591673 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988575032 0 0) 0x7fec400cef10 con 0x7fec4015c950 2024-03-29T08:29:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.350+0000 7fec3fe00640 1 -- 172.21.15.46:0/673591673 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec400cfb20 con 0x7fec4015c950 2024-03-29T08:29:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.350+0000 7fec45400640 1 -- 172.21.15.46:0/673591673 >> v1:172.21.15.73:6789/0 conn(0x7fec4015c950 legacy=0x7fec4006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.350+0000 7fec45400640 1 -- 172.21.15.46:0/673591673 >> v1:172.21.15.46:6789/0 conn(0x7fec400cea30 legacy=0x7fec401606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:15.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.350+0000 7fec45400640 1 -- 172.21.15.46:0/673591673 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec400cf4d0 con 0x7fec40160db0 2024-03-29T08:29:15.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.354+0000 7fec45400640 1 --2- 172.21.15.46:0/673591673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec40160db0 0x7fec400cad10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fec38009d90 tx=0x7fec3801f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bc7304e754c8056 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:15.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.354+0000 7fec3fe00640 1 -- 172.21.15.46:0/673591673 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec380af600 con 0x7fec40160db0 2024-03-29T08:29:15.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.354+0000 7fec3fe00640 1 -- 172.21.15.46:0/673591673 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec380ab050 con 0x7fec40160db0 2024-03-29T08:29:15.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.354+0000 7fec3fe00640 1 -- 172.21.15.46:0/673591673 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec380af960 con 0x7fec40160db0 2024-03-29T08:29:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.354+0000 7fec47400640 1 -- 172.21.15.46:0/673591673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec40160db0 msgr2=0x7fec400cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.354+0000 7fec47400640 1 --2- 172.21.15.46:0/673591673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec40160db0 0x7fec400cad10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fec38009d90 tx=0x7fec3801f150 comp rx=0 tx=0).stop 2024-03-29T08:29:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.354+0000 7fec47400640 1 -- 172.21.15.46:0/673591673 shutdown_connections 2024-03-29T08:29:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.354+0000 7fec47400640 1 --2- 172.21.15.46:0/673591673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec40160db0 0x7fec400cad10 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.354+0000 7fec47400640 1 -- 172.21.15.46:0/673591673 >> 172.21.15.46:0/673591673 conn(0x7fec400b21b0 msgr2=0x7fec400b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.354+0000 7fec47400640 1 -- 172.21.15.46:0/673591673 shutdown_connections 2024-03-29T08:29:15.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.354+0000 7fec47400640 1 -- 172.21.15.46:0/673591673 wait complete. 2024-03-29T08:29:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec47400640 1 Processor -- start 2024-03-29T08:29:15.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec47400640 1 -- start start 2024-03-29T08:29:15.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec47400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec400cea30 0x7fec40157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:15.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec47400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fec4015c950 0x7fec401e7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:15.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec47400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec40160db0 0x7fec400c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:15.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec47400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec400b3660 con 0x7fec40160db0 2024-03-29T08:29:15.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec47400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec401e8330 con 0x7fec400cea30 2024-03-29T08:29:15.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec47400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec401e7d70 con 0x7fec4015c950 2024-03-29T08:29:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec45400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec40160db0 0x7fec400c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec45400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec40160db0 0x7fec400c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33666/0 (socket says 172.21.15.46:33666) 2024-03-29T08:29:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec45400640 1 -- 172.21.15.46:0/3278309795 learned_addr learned my addr 172.21.15.46:0/3278309795 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec44a00640 1 --2- 172.21.15.46:0/3278309795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec400cea30 0x7fec40157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec37e00640 1 --2- 172.21.15.46:0/3278309795 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fec4015c950 0x7fec401e7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec45400640 1 -- 172.21.15.46:0/3278309795 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fec4015c950 msgr2=0x7fec401e7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec45400640 1 --2- 172.21.15.46:0/3278309795 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fec4015c950 0x7fec401e7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec45400640 1 -- 172.21.15.46:0/3278309795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec400cea30 msgr2=0x7fec40157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec45400640 1 --2- 172.21.15.46:0/3278309795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec400cea30 0x7fec40157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec45400640 1 -- 172.21.15.46:0/3278309795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec38009a70 con 0x7fec40160db0 2024-03-29T08:29:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec45400640 1 --2- 172.21.15.46:0/3278309795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec40160db0 0x7fec400c5330 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fec3801cd80 tx=0x7fec3801bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec3d600640 1 -- 172.21.15.46:0/3278309795 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec380bf070 con 0x7fec40160db0 2024-03-29T08:29:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec3d600640 1 -- 172.21.15.46:0/3278309795 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec38004250 con 0x7fec40160db0 2024-03-29T08:29:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec3d600640 1 -- 172.21.15.46:0/3278309795 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec380045e0 con 0x7fec40160db0 2024-03-29T08:29:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec47400640 1 -- 172.21.15.46:0/3278309795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4006c470 con 0x7fec40160db0 2024-03-29T08:29:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.358+0000 7fec47400640 1 -- 172.21.15.46:0/3278309795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec400b3280 con 0x7fec40160db0 2024-03-29T08:29:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.362+0000 7fec47400640 1 -- 172.21.15.46:0/3278309795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec400b37a0 con 0x7fec40160db0 2024-03-29T08:29:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.362+0000 7fec3d600640 1 -- 172.21.15.46:0/3278309795 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fec38004740 con 0x7fec40160db0 2024-03-29T08:29:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.362+0000 7fec3d600640 1 --2- 172.21.15.46:0/3278309795 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fec1c066690 0x7fec1c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.362+0000 7fec3d600640 1 -- 172.21.15.46:0/3278309795 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec381220b0 con 0x7fec40160db0 2024-03-29T08:29:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.362+0000 7fec44a00640 1 --2- 172.21.15.46:0/3278309795 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fec1c066690 0x7fec1c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:15.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.362+0000 7fec44a00640 1 --2- 172.21.15.46:0/3278309795 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fec1c066690 0x7fec1c068b50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fec300011f0 tx=0x7fec30009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:15.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.366+0000 7fec3d600640 1 -- 172.21.15.46:0/3278309795 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fec400b37a0 con 0x7fec40160db0 2024-03-29T08:29:15.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:15.562+0000 7fec47400640 1 -- 172.21.15.46:0/3278309795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fec400b37a0 con 0x7fec40160db0 2024-03-29T08:29:16.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.478+0000 7fec3d600640 1 -- 172.21.15.46:0/3278309795 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v93) v1 ==== 118+0+0 (secure 0 0 0) 0x7fec400b37a0 con 0x7fec40160db0 2024-03-29T08:29:16.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.478+0000 7fec47400640 1 -- 172.21.15.46:0/3278309795 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fec1c066690 msgr2=0x7fec1c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:16.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.478+0000 7fec47400640 1 --2- 172.21.15.46:0/3278309795 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fec1c066690 0x7fec1c068b50 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fec300011f0 tx=0x7fec30009040 comp rx=0 tx=0).stop 2024-03-29T08:29:16.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.478+0000 7fec47400640 1 -- 172.21.15.46:0/3278309795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec40160db0 msgr2=0x7fec400c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:16.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.478+0000 7fec47400640 1 --2- 172.21.15.46:0/3278309795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec40160db0 0x7fec400c5330 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fec3801cd80 tx=0x7fec3801bef0 comp rx=0 tx=0).stop 2024-03-29T08:29:16.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.478+0000 7fec47400640 1 -- 172.21.15.46:0/3278309795 shutdown_connections 2024-03-29T08:29:16.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.478+0000 7fec47400640 1 --2- 172.21.15.46:0/3278309795 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fec1c066690 0x7fec1c068b50 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:16.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.478+0000 7fec47400640 1 --2- 172.21.15.46:0/3278309795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec40160db0 0x7fec400c5330 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:16.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.478+0000 7fec47400640 1 --2- 172.21.15.46:0/3278309795 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fec4015c950 0x7fec401e7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:16.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.478+0000 7fec47400640 1 --2- 172.21.15.46:0/3278309795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec400cea30 0x7fec40157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:16.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.478+0000 7fec47400640 1 -- 172.21.15.46:0/3278309795 >> 172.21.15.46:0/3278309795 conn(0x7fec400b21b0 msgr2=0x7fec4006aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:16.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.478+0000 7fec47400640 1 -- 172.21.15.46:0/3278309795 shutdown_connections 2024-03-29T08:29:16.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.478+0000 7fec47400640 1 -- 172.21.15.46:0/3278309795 wait complete. 2024-03-29T08:29:16.508 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-29T08:29:16.508 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-29T08:29:16.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.654+0000 7f4852c00640 1 Processor -- start 2024-03-29T08:29:16.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.654+0000 7f4852c00640 1 -- start start 2024-03-29T08:29:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.654+0000 7f4852c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f484c15a9c0 0x7f484c0cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.654+0000 7f4852c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f484c0ce290 con 0x7f484c099150 2024-03-29T08:29:16.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.654+0000 7f4852c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f484c0ce570 con 0x7f484c0cc370 2024-03-29T08:29:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.654+0000 7f4852c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484c0ce850 con 0x7f484c15a9c0 2024-03-29T08:29:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.654+0000 7f484be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f484c15a9c0 0x7f484c0cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.654+0000 7f484be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f484c15a9c0 0x7f484c0cbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38976/0 (socket says 172.21.15.46:38976) 2024-03-29T08:29:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.654+0000 7f484be00640 1 -- 172.21.15.46:0/981132156 learned_addr learned my addr 172.21.15.46:0/981132156 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.654+0000 7f4850c00640 1 --1- 172.21.15.46:0/981132156 >> v1:172.21.15.46:6789/0 conn(0x7f484c099150 0x7f484c0b3510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48182/0 (socket says 172.21.15.46:48182) 2024-03-29T08:29:16.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.654+0000 7f484be00640 1 -- 172.21.15.46:0/981132156 >> v1:172.21.15.73:6789/0 conn(0x7f484c0cc370 legacy=0x7f484c098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.654+0000 7f484be00640 1 -- 172.21.15.46:0/981132156 >> v1:172.21.15.46:6789/0 conn(0x7f484c099150 legacy=0x7f484c0b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:16.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.654+0000 7f484be00640 1 -- 172.21.15.46:0/981132156 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c0ceb30 con 0x7f484c15a9c0 2024-03-29T08:29:16.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.658+0000 7f484be00640 1 --2- 172.21.15.46:0/981132156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484c15a9c0 0x7f484c0cbe30 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f483c009870 tx=0x7f483c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b92ce4137fb91c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:16.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.658+0000 7f484aa00640 1 -- 172.21.15.46:0/981132156 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c0ac620 con 0x7f484c15a9c0 2024-03-29T08:29:16.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.658+0000 7f484aa00640 1 -- 172.21.15.46:0/981132156 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c0ac780 con 0x7f484c15a9c0 2024-03-29T08:29:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.658+0000 7f484aa00640 1 -- 172.21.15.46:0/981132156 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c0acb10 con 0x7f484c15a9c0 2024-03-29T08:29:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.658+0000 7f4852c00640 1 -- 172.21.15.46:0/981132156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484c15a9c0 msgr2=0x7f484c0cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.658+0000 7f4852c00640 1 --2- 172.21.15.46:0/981132156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484c15a9c0 0x7f484c0cbe30 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f483c009870 tx=0x7f483c01c4d0 comp rx=0 tx=0).stop 2024-03-29T08:29:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.658+0000 7f4852c00640 1 -- 172.21.15.46:0/981132156 shutdown_connections 2024-03-29T08:29:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.658+0000 7f4852c00640 1 --2- 172.21.15.46:0/981132156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484c15a9c0 0x7f484c0cbe30 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.658+0000 7f4852c00640 1 -- 172.21.15.46:0/981132156 >> 172.21.15.46:0/981132156 conn(0x7f484c0b21b0 msgr2=0x7f484c0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.658+0000 7f4852c00640 1 -- 172.21.15.46:0/981132156 shutdown_connections 2024-03-29T08:29:16.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.658+0000 7f4852c00640 1 -- 172.21.15.46:0/981132156 wait complete. 2024-03-29T08:29:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.658+0000 7f4852c00640 1 Processor -- start 2024-03-29T08:29:16.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.658+0000 7f4852c00640 1 -- start start 2024-03-29T08:29:16.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f4852c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f484c099150 0x7f484c163aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:16.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f4852c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484c0cc370 0x7f484c1653e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f4852c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f484c15a9c0 0x7f484c151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f4852c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c164fc0 con 0x7f484c0cc370 2024-03-29T08:29:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f4852c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c164340 con 0x7f484c099150 2024-03-29T08:29:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f4852c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484c164620 con 0x7f484c15a9c0 2024-03-29T08:29:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f484b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484c0cc370 0x7f484c1653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f484b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484c0cc370 0x7f484c1653e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38980/0 (socket says 172.21.15.46:38980) 2024-03-29T08:29:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f484b400640 1 -- 172.21.15.46:0/2350812310 learned_addr learned my addr 172.21.15.46:0/2350812310 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f484be00640 1 --2- 172.21.15.46:0/2350812310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f484c099150 0x7f484c163aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f4850c00640 1 --2- 172.21.15.46:0/2350812310 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f484c15a9c0 0x7f484c151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f484b400640 1 -- 172.21.15.46:0/2350812310 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f484c15a9c0 msgr2=0x7f484c151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:16.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f484b400640 1 --2- 172.21.15.46:0/2350812310 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f484c15a9c0 0x7f484c151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:16.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f484b400640 1 -- 172.21.15.46:0/2350812310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f484c099150 msgr2=0x7f484c163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:16.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f484b400640 1 --2- 172.21.15.46:0/2350812310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f484c099150 0x7f484c163aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:16.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f484b400640 1 -- 172.21.15.46:0/2350812310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c0094d0 con 0x7f484c0cc370 2024-03-29T08:29:16.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f484b400640 1 --2- 172.21.15.46:0/2350812310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484c0cc370 0x7f484c1653e0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f483800c420 tx=0x7f483800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:16.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f4843e00640 1 -- 172.21.15.46:0/2350812310 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48380052d0 con 0x7f484c0cc370 2024-03-29T08:29:16.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f4843e00640 1 -- 172.21.15.46:0/2350812310 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4838005430 con 0x7f484c0cc370 2024-03-29T08:29:16.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f4843e00640 1 -- 172.21.15.46:0/2350812310 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838007260 con 0x7f484c0cc370 2024-03-29T08:29:16.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f4852c00640 1 -- 172.21.15.46:0/2350812310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484c0cf790 con 0x7f484c0cc370 2024-03-29T08:29:16.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f4852c00640 1 -- 172.21.15.46:0/2350812310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f484c164be0 con 0x7f484c0cc370 2024-03-29T08:29:16.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f4852c00640 1 -- 172.21.15.46:0/2350812310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484c0d0040 con 0x7f484c0cc370 2024-03-29T08:29:16.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.662+0000 7f4843e00640 1 -- 172.21.15.46:0/2350812310 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f48380073c0 con 0x7f484c0cc370 2024-03-29T08:29:16.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.666+0000 7f4843e00640 1 --2- 172.21.15.46:0/2350812310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f482c0666c0 0x7f482c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:16.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.666+0000 7f4843e00640 1 -- 172.21.15.46:0/2350812310 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4838083580 con 0x7f484c0cc370 2024-03-29T08:29:16.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.666+0000 7f484be00640 1 --2- 172.21.15.46:0/2350812310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f482c0666c0 0x7f482c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:16.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.666+0000 7f484be00640 1 --2- 172.21.15.46:0/2350812310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f482c0666c0 0x7f482c068b80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f483c0ae000 tx=0x7f483c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:16.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.670+0000 7f4843e00640 1 -- 172.21.15.46:0/2350812310 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f48380508e0 con 0x7f484c0cc370 2024-03-29T08:29:16.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:16.861+0000 7f4852c00640 1 -- 172.21.15.46:0/2350812310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f484c165100 con 0x7f484c0cc370 2024-03-29T08:29:17.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.481+0000 7f4843e00640 1 -- 172.21.15.46:0/2350812310 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v94) v1 ==== 122+0+0 (secure 0 0 0) 0x7f484c165100 con 0x7f484c0cc370 2024-03-29T08:29:17.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.485+0000 7f4852c00640 1 -- 172.21.15.46:0/2350812310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f482c0666c0 msgr2=0x7f482c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:17.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.485+0000 7f4852c00640 1 --2- 172.21.15.46:0/2350812310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f482c0666c0 0x7f482c068b80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f483c0ae000 tx=0x7f483c009040 comp rx=0 tx=0).stop 2024-03-29T08:29:17.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.485+0000 7f4852c00640 1 -- 172.21.15.46:0/2350812310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484c0cc370 msgr2=0x7f484c1653e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:17.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.485+0000 7f4852c00640 1 --2- 172.21.15.46:0/2350812310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484c0cc370 0x7f484c1653e0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f483800c420 tx=0x7f483800c8e0 comp rx=0 tx=0).stop 2024-03-29T08:29:17.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.485+0000 7f4852c00640 1 -- 172.21.15.46:0/2350812310 shutdown_connections 2024-03-29T08:29:17.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.485+0000 7f4852c00640 1 --2- 172.21.15.46:0/2350812310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f482c0666c0 0x7f482c068b80 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:17.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.485+0000 7f4852c00640 1 --2- 172.21.15.46:0/2350812310 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f484c15a9c0 0x7f484c151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:17.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.485+0000 7f4852c00640 1 --2- 172.21.15.46:0/2350812310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484c0cc370 0x7f484c1653e0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:17.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.485+0000 7f4852c00640 1 --2- 172.21.15.46:0/2350812310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f484c099150 0x7f484c163aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:17.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.485+0000 7f4852c00640 1 -- 172.21.15.46:0/2350812310 >> 172.21.15.46:0/2350812310 conn(0x7f484c0b21b0 msgr2=0x7f484c0b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:17.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.485+0000 7f4852c00640 1 -- 172.21.15.46:0/2350812310 shutdown_connections 2024-03-29T08:29:17.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.485+0000 7f4852c00640 1 -- 172.21.15.46:0/2350812310 wait complete. 2024-03-29T08:29:17.510 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-29T08:29:17.510 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:29:17.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.645+0000 7f9f3bc00640 1 Processor -- start 2024-03-29T08:29:17.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.645+0000 7f9f3bc00640 1 -- start start 2024-03-29T08:29:17.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.649+0000 7f9f3bc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f3415feb0 0x7f9f34071370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.649+0000 7f9f3bc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9f341605f0 0x7f9f34069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.649+0000 7f9f3bc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f34160a50 0x7f9f3406a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.649+0000 7f9f3bc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f34161020 con 0x7f9f34160a50 2024-03-29T08:29:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.649+0000 7f9f3bc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f34161300 con 0x7f9f341605f0 2024-03-29T08:29:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.649+0000 7f9f3bc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f34070b70 con 0x7f9f3415feb0 2024-03-29T08:29:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.649+0000 7f9f39c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f34160a50 0x7f9f3406a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.649+0000 7f9f39c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f34160a50 0x7f9f3406a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38992/0 (socket says 172.21.15.46:38992) 2024-03-29T08:29:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.649+0000 7f9f39c00640 1 -- 172.21.15.46:0/1336203669 learned_addr learned my addr 172.21.15.46:0/1336203669 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.649+0000 7f9f39200640 1 --2- 172.21.15.46:0/1336203669 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f3415feb0 0x7f9f34071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.649+0000 7f9f33e00640 1 --2- 172.21.15.46:0/1336203669 >> v2:172.21.15.73:3300/0 conn(0x7f9f341605f0 0x7f9f34069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.649+0000 7f9f39c00640 1 -- 172.21.15.46:0/1336203669 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f3415feb0 msgr2=0x7f9f34071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.649+0000 7f9f39c00640 1 --2- 172.21.15.46:0/1336203669 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f3415feb0 0x7f9f34071370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.649+0000 7f9f39c00640 1 -- 172.21.15.46:0/1336203669 >> v2:172.21.15.73:3300/0 conn(0x7f9f341605f0 msgr2=0x7f9f34069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.649+0000 7f9f39c00640 1 --2- 172.21.15.46:0/1336203669 >> v2:172.21.15.73:3300/0 conn(0x7f9f341605f0 0x7f9f34069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.649+0000 7f9f39c00640 1 -- 172.21.15.46:0/1336203669 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f34070e50 con 0x7f9f34160a50 2024-03-29T08:29:17.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.653+0000 7f9f39c00640 1 --2- 172.21.15.46:0/1336203669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f34160a50 0x7f9f3406a2d0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f9f340ccb20 tx=0x7f9f2c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fa1a6fe9d34e9f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:17.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.653+0000 7f9f33400640 1 -- 172.21.15.46:0/1336203669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0a5800 con 0x7f9f34160a50 2024-03-29T08:29:17.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.653+0000 7f9f33400640 1 -- 172.21.15.46:0/1336203669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c0a5960 con 0x7f9f34160a50 2024-03-29T08:29:17.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.653+0000 7f9f33400640 1 -- 172.21.15.46:0/1336203669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0a5cf0 con 0x7f9f34160a50 2024-03-29T08:29:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.653+0000 7f9f3bc00640 1 -- 172.21.15.46:0/1336203669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f34160a50 msgr2=0x7f9f3406a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.653+0000 7f9f3bc00640 1 --2- 172.21.15.46:0/1336203669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f34160a50 0x7f9f3406a2d0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f9f340ccb20 tx=0x7f9f2c01c960 comp rx=0 tx=0).stop 2024-03-29T08:29:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.653+0000 7f9f3bc00640 1 -- 172.21.15.46:0/1336203669 shutdown_connections 2024-03-29T08:29:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.653+0000 7f9f3bc00640 1 --2- 172.21.15.46:0/1336203669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f34160a50 0x7f9f3406a2d0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.653+0000 7f9f3bc00640 1 --2- 172.21.15.46:0/1336203669 >> v2:172.21.15.73:3300/0 conn(0x7f9f341605f0 0x7f9f34069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.653+0000 7f9f3bc00640 1 --2- 172.21.15.46:0/1336203669 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f3415feb0 0x7f9f34071370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.653+0000 7f9f3bc00640 1 -- 172.21.15.46:0/1336203669 >> 172.21.15.46:0/1336203669 conn(0x7f9f340ced50 msgr2=0x7f9f340cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.653+0000 7f9f3bc00640 1 -- 172.21.15.46:0/1336203669 shutdown_connections 2024-03-29T08:29:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.653+0000 7f9f3bc00640 1 -- 172.21.15.46:0/1336203669 wait complete. 2024-03-29T08:29:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.653+0000 7f9f3bc00640 1 Processor -- start 2024-03-29T08:29:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.653+0000 7f9f3bc00640 1 -- start start 2024-03-29T08:29:17.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f3bc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f3415feb0 0x7f9f341448b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:17.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f3bc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f341605f0 0x7f9f340b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:17.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f3bc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f34160a50 0x7f9f34159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:17.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f3bc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3415a6f0 con 0x7f9f341605f0 2024-03-29T08:29:17.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f3bc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3415d7c0 con 0x7f9f3415feb0 2024-03-29T08:29:17.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f3bc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3415d4b0 con 0x7f9f34160a50 2024-03-29T08:29:17.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f33e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f341605f0 0x7f9f340b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:17.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f33e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f341605f0 0x7f9f340b23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39006/0 (socket says 172.21.15.46:39006) 2024-03-29T08:29:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f33e00640 1 -- 172.21.15.46:0/2765100180 learned_addr learned my addr 172.21.15.46:0/2765100180 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f39c00640 1 --2- 172.21.15.46:0/2765100180 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f34160a50 0x7f9f34159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f39200640 1 --2- 172.21.15.46:0/2765100180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f3415feb0 0x7f9f341448b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f33e00640 1 -- 172.21.15.46:0/2765100180 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f34160a50 msgr2=0x7f9f34159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f33e00640 1 --2- 172.21.15.46:0/2765100180 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f34160a50 0x7f9f34159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f33e00640 1 -- 172.21.15.46:0/2765100180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f3415feb0 msgr2=0x7f9f341448b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f33e00640 1 --2- 172.21.15.46:0/2765100180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f3415feb0 0x7f9f341448b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f33e00640 1 -- 172.21.15.46:0/2765100180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c009b30 con 0x7f9f341605f0 2024-03-29T08:29:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f39200640 1 --2- 172.21.15.46:0/2765100180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f3415feb0 0x7f9f341448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f33e00640 1 --2- 172.21.15.46:0/2765100180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f341605f0 0x7f9f340b23b0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000e870 tx=0x7f9f2000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f30c00640 1 -- 172.21.15.46:0/2765100180 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f20009510 con 0x7f9f341605f0 2024-03-29T08:29:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f30c00640 1 -- 172.21.15.46:0/2765100180 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2000eec0 con 0x7f9f341605f0 2024-03-29T08:29:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f30c00640 1 -- 172.21.15.46:0/2765100180 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2000cb40 con 0x7f9f341605f0 2024-03-29T08:29:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f3bc00640 1 -- 172.21.15.46:0/2765100180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f340b2c00 con 0x7f9f341605f0 2024-03-29T08:29:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f3bc00640 1 -- 172.21.15.46:0/2765100180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f340b31c0 con 0x7f9f341605f0 2024-03-29T08:29:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.657+0000 7f9f3bc00640 1 -- 172.21.15.46:0/2765100180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f3415d900 con 0x7f9f341605f0 2024-03-29T08:29:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.661+0000 7f9f30c00640 1 -- 172.21.15.46:0/2765100180 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9f2000cca0 con 0x7f9f341605f0 2024-03-29T08:29:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.661+0000 7f9f30c00640 1 --2- 172.21.15.46:0/2765100180 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f14066690 0x7f9f14068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.661+0000 7f9f30c00640 1 -- 172.21.15.46:0/2765100180 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f20084160 con 0x7f9f341605f0 2024-03-29T08:29:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.661+0000 7f9f39200640 1 --2- 172.21.15.46:0/2765100180 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f14066690 0x7f9f14068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:17.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.661+0000 7f9f39200640 1 --2- 172.21.15.46:0/2765100180 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f14066690 0x7f9f14068b50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9f28009b10 tx=0x7f9f28008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.665+0000 7f9f30c00640 1 -- 172.21.15.46:0/2765100180 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9f3415d900 con 0x7f9f341605f0 2024-03-29T08:29:17.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.857+0000 7f9f3bc00640 1 -- 172.21.15.46:0/2765100180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f3415d900 con 0x7f9f341605f0 2024-03-29T08:29:17.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.857+0000 7f9f30c00640 1 -- 172.21.15.46:0/2765100180 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4733 (secure 0 0 0) 0x7f9f3415d900 con 0x7f9f341605f0 2024-03-29T08:29:17.860 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:17.861 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2506132630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2506132630},{"type":"v1","addr":"172.21.15.73:6836","nonce":2506132630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2108115501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2108115501},{"type":"v1","addr":"172.21.15.73:6837","nonce":2108115501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":9045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3210062895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3210062895},{"type":"v1","addr":"172.21.15.46:6833","nonce":3210062895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:29:14.470858+0000","modified":"2024-03-29T08:29:17.480376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/929698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":929698978},{"type":"v1","addr":"172.21.15.46:6835","nonce":929698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-29T08:29:17.861 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 94 2024-03-29T08:29:17.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.857+0000 7f9f3bc00640 1 -- 172.21.15.46:0/2765100180 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f14066690 msgr2=0x7f9f14068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:17.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.857+0000 7f9f3bc00640 1 --2- 172.21.15.46:0/2765100180 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f14066690 0x7f9f14068b50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9f28009b10 tx=0x7f9f28008040 comp rx=0 tx=0).stop 2024-03-29T08:29:17.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.857+0000 7f9f3bc00640 1 -- 172.21.15.46:0/2765100180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f341605f0 msgr2=0x7f9f340b23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:17.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.857+0000 7f9f3bc00640 1 --2- 172.21.15.46:0/2765100180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f341605f0 0x7f9f340b23b0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000e870 tx=0x7f9f2000ec70 comp rx=0 tx=0).stop 2024-03-29T08:29:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.857+0000 7f9f3bc00640 1 -- 172.21.15.46:0/2765100180 shutdown_connections 2024-03-29T08:29:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.857+0000 7f9f3bc00640 1 --2- 172.21.15.46:0/2765100180 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f14066690 0x7f9f14068b50 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.857+0000 7f9f3bc00640 1 --2- 172.21.15.46:0/2765100180 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9f34160a50 0x7f9f34159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.857+0000 7f9f3bc00640 1 --2- 172.21.15.46:0/2765100180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f341605f0 0x7f9f340b23b0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.857+0000 7f9f3bc00640 1 --2- 172.21.15.46:0/2765100180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f3415feb0 0x7f9f341448b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.857+0000 7f9f3bc00640 1 -- 172.21.15.46:0/2765100180 >> 172.21.15.46:0/2765100180 conn(0x7f9f340ced50 msgr2=0x7f9f34069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.857+0000 7f9f3bc00640 1 -- 172.21.15.46:0/2765100180 shutdown_connections 2024-03-29T08:29:17.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:17.857+0000 7f9f3bc00640 1 -- 172.21.15.46:0/2765100180 wait complete. 2024-03-29T08:29:17.882 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:29:18.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.017+0000 7f3fd9400640 1 Processor -- start 2024-03-29T08:29:18.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.017+0000 7f3fd9400640 1 -- start start 2024-03-29T08:29:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.017+0000 7f3fd9400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3fd4144210 0x7f3fd40cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.017+0000 7f3fd9400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3fd415ada0 0x7f3fd40ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.017+0000 7f3fd9400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd40cb510 con 0x7f3fd40ce950 2024-03-29T08:29:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.021+0000 7f3fd9400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd40cede0 con 0x7f3fd4144210 2024-03-29T08:29:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.021+0000 7f3fd9400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd406d960 con 0x7f3fd415ada0 2024-03-29T08:29:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.021+0000 7f3fd2800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3fd4144210 0x7f3fd40cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.021+0000 7f3fd2800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3fd4144210 0x7f3fd40cb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39014/0 (socket says 172.21.15.46:39014) 2024-03-29T08:29:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.021+0000 7f3fd2800640 1 -- 172.21.15.46:0/3621554773 learned_addr learned my addr 172.21.15.46:0/3621554773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.021+0000 7f3fc9e00640 1 --2- 172.21.15.46:0/3621554773 >> v2:172.21.15.73:3300/0 conn(0x7f3fd415ada0 0x7f3fd40ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.021+0000 7f3fd2800640 1 -- 172.21.15.46:0/3621554773 >> v2:172.21.15.73:3300/0 conn(0x7f3fd415ada0 msgr2=0x7f3fd40ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.021+0000 7f3fd2800640 1 --2- 172.21.15.46:0/3621554773 >> v2:172.21.15.73:3300/0 conn(0x7f3fd415ada0 0x7f3fd40ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.021+0000 7f3fd2800640 1 -- 172.21.15.46:0/3621554773 >> v1:172.21.15.46:6789/0 conn(0x7f3fd40ce950 legacy=0x7f3fd415e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:18.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.021+0000 7f3fd2800640 1 -- 172.21.15.46:0/3621554773 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd4070580 con 0x7f3fd4144210 2024-03-29T08:29:18.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.021+0000 7f3fd2800640 1 --2- 172.21.15.46:0/3621554773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd4144210 0x7f3fd40cb8b0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f3fd406e410 tx=0x7f3fc401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155998d1051976f7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.021+0000 7f3fd1e00640 1 -- 172.21.15.46:0/3621554773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40ab600 con 0x7f3fd4144210 2024-03-29T08:29:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.021+0000 7f3fd1e00640 1 -- 172.21.15.46:0/3621554773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc40a7050 con 0x7f3fd4144210 2024-03-29T08:29:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.021+0000 7f3fd1e00640 1 -- 172.21.15.46:0/3621554773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc40ab980 con 0x7f3fd4144210 2024-03-29T08:29:18.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd9400640 1 -- 172.21.15.46:0/3621554773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd4144210 msgr2=0x7f3fd40cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:18.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd9400640 1 --2- 172.21.15.46:0/3621554773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd4144210 0x7f3fd40cb8b0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f3fd406e410 tx=0x7f3fc401c520 comp rx=0 tx=0).stop 2024-03-29T08:29:18.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd9400640 1 -- 172.21.15.46:0/3621554773 shutdown_connections 2024-03-29T08:29:18.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd9400640 1 --2- 172.21.15.46:0/3621554773 >> v2:172.21.15.73:3300/0 conn(0x7f3fd415ada0 0x7f3fd40ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd9400640 1 --2- 172.21.15.46:0/3621554773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd4144210 0x7f3fd40cb8b0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd9400640 1 -- 172.21.15.46:0/3621554773 >> 172.21.15.46:0/3621554773 conn(0x7f3fd4066110 msgr2=0x7f3fd4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:18.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd9400640 1 -- 172.21.15.46:0/3621554773 shutdown_connections 2024-03-29T08:29:18.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd9400640 1 -- 172.21.15.46:0/3621554773 wait complete. 2024-03-29T08:29:18.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd9400640 1 Processor -- start 2024-03-29T08:29:18.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd9400640 1 -- start start 2024-03-29T08:29:18.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd9400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3fd40ce950 0x7f3fd415a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd9400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd4144210 0x7f3fd413ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd9400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fd415ada0 0x7f3fd40ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd9400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd4140a90 con 0x7f3fd4144210 2024-03-29T08:29:18.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd9400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd413e300 con 0x7f3fd415ada0 2024-03-29T08:29:18.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd9400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd4140eb0 con 0x7f3fd40ce950 2024-03-29T08:29:18.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fc9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd4144210 0x7f3fd413ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fc9e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd4144210 0x7f3fd413ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39022/0 (socket says 172.21.15.46:39022) 2024-03-29T08:29:18.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fc9e00640 1 -- 172.21.15.46:0/2265128918 learned_addr learned my addr 172.21.15.46:0/2265128918 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:18.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd2800640 1 --2- 172.21.15.46:0/2265128918 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3fd40ce950 0x7f3fd415a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd3200640 1 --2- 172.21.15.46:0/2265128918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fd415ada0 0x7f3fd40ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fc9e00640 1 -- 172.21.15.46:0/2265128918 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3fd40ce950 msgr2=0x7f3fd415a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:18.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fc9e00640 1 --2- 172.21.15.46:0/2265128918 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3fd40ce950 0x7f3fd415a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fc9e00640 1 -- 172.21.15.46:0/2265128918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fd415ada0 msgr2=0x7f3fd40ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:18.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fc9e00640 1 --2- 172.21.15.46:0/2265128918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fd415ada0 0x7f3fd40ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fc9e00640 1 -- 172.21.15.46:0/2265128918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc40094d0 con 0x7f3fd4144210 2024-03-29T08:29:18.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.025+0000 7f3fd3200640 1 --2- 172.21.15.46:0/2265128918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fd415ada0 0x7f3fd40ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:18.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.029+0000 7f3fc9e00640 1 --2- 172.21.15.46:0/2265128918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd4144210 0x7f3fd413ddc0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00d830 tx=0x7f3fbc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:18.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.029+0000 7f3fcb400640 1 -- 172.21.15.46:0/2265128918 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc00b6c0 con 0x7f3fd4144210 2024-03-29T08:29:18.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.029+0000 7f3fcb400640 1 -- 172.21.15.46:0/2265128918 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc00b820 con 0x7f3fd4144210 2024-03-29T08:29:18.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.029+0000 7f3fd9400640 1 -- 172.21.15.46:0/2265128918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd41400f0 con 0x7f3fd4144210 2024-03-29T08:29:18.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.029+0000 7f3fcb400640 1 -- 172.21.15.46:0/2265128918 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc00bb90 con 0x7f3fd4144210 2024-03-29T08:29:18.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.029+0000 7f3fd9400640 1 -- 172.21.15.46:0/2265128918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd41406b0 con 0x7f3fd4144210 2024-03-29T08:29:18.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.029+0000 7f3fd9400640 1 -- 172.21.15.46:0/2265128918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd40ce280 con 0x7f3fd4144210 2024-03-29T08:29:18.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.029+0000 7f3fcb400640 1 -- 172.21.15.46:0/2265128918 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f3fbc01e460 con 0x7f3fd4144210 2024-03-29T08:29:18.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.029+0000 7f3fcb400640 1 --2- 172.21.15.46:0/2265128918 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3fb0066690 0x7f3fb0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.029+0000 7f3fcb400640 1 -- 172.21.15.46:0/2265128918 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3fbc084870 con 0x7f3fd4144210 2024-03-29T08:29:18.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.029+0000 7f3fd2800640 1 --2- 172.21.15.46:0/2265128918 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3fb0066690 0x7f3fb0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.033+0000 7f3fd2800640 1 --2- 172.21.15.46:0/2265128918 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3fb0066690 0x7f3fb0068b50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40ad000 tx=0x7f3fc4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:18.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.037+0000 7f3fcb400640 1 -- 172.21.15.46:0/2265128918 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f3fbc051bd0 con 0x7f3fd4144210 2024-03-29T08:29:18.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.201+0000 7f3fd9400640 1 -- 172.21.15.46:0/2265128918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3fd4066c90 con 0x7f3fd4144210 2024-03-29T08:29:18.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.201+0000 7f3fcb400640 1 -- 172.21.15.46:0/2265128918 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16067 (secure 0 0 0) 0x7f3fbc0551f0 con 0x7f3fd4144210 2024-03-29T08:29:18.207 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:18.207 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":121,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:29:15.473193+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:29:12.329121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-29T08:29:13.465939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:29:18.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.205+0000 7f3fd9400640 1 -- 172.21.15.46:0/2265128918 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3fb0066690 msgr2=0x7f3fb0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:18.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.205+0000 7f3fd9400640 1 --2- 172.21.15.46:0/2265128918 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3fb0066690 0x7f3fb0068b50 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40ad000 tx=0x7f3fc4008040 comp rx=0 tx=0).stop 2024-03-29T08:29:18.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.205+0000 7f3fd9400640 1 -- 172.21.15.46:0/2265128918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd4144210 msgr2=0x7f3fd413ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:18.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.205+0000 7f3fd9400640 1 --2- 172.21.15.46:0/2265128918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd4144210 0x7f3fd413ddc0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00d830 tx=0x7f3fbc00dcf0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.205+0000 7f3fd9400640 1 -- 172.21.15.46:0/2265128918 shutdown_connections 2024-03-29T08:29:18.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.205+0000 7f3fd9400640 1 --2- 172.21.15.46:0/2265128918 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3fb0066690 0x7f3fb0068b50 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.205+0000 7f3fd9400640 1 --2- 172.21.15.46:0/2265128918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fd415ada0 0x7f3fd40ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.205+0000 7f3fd9400640 1 --2- 172.21.15.46:0/2265128918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fd4144210 0x7f3fd413ddc0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.205+0000 7f3fd9400640 1 --2- 172.21.15.46:0/2265128918 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3fd40ce950 0x7f3fd415a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.205+0000 7f3fd9400640 1 -- 172.21.15.46:0/2265128918 >> 172.21.15.46:0/2265128918 conn(0x7f3fd4066110 msgr2=0x7f3fd40ad8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:18.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.205+0000 7f3fd9400640 1 -- 172.21.15.46:0/2265128918 shutdown_connections 2024-03-29T08:29:18.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.205+0000 7f3fd9400640 1 -- 172.21.15.46:0/2265128918 wait complete. 2024-03-29T08:29:18.229 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-29T08:29:18.229 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:29:18.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22f9600640 1 Processor -- start 2024-03-29T08:29:18.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22f9600640 1 -- start start 2024-03-29T08:29:18.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22f9600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22f41593d0 0x7f22f406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22f9600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22f415d230 0x7f22f415d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22f9600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f22f415dbd0 0x7f22f415bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22f9600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f406d130 con 0x7f22f415d230 2024-03-29T08:29:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22f9600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f415b8f0 con 0x7f22f415dbd0 2024-03-29T08:29:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22f9600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f415ad50 con 0x7f22f41593d0 2024-03-29T08:29:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22ea000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22f415d230 0x7f22f415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22ea000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22f415d230 0x7f22f415d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39034/0 (socket says 172.21.15.46:39034) 2024-03-29T08:29:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22ea000640 1 -- 172.21.15.46:0/260422202 learned_addr learned my addr 172.21.15.46:0/260422202 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22f2a00640 1 --2- 172.21.15.46:0/260422202 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22f41593d0 0x7f22f406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22f3400640 1 --2- 172.21.15.46:0/260422202 >> v2:172.21.15.73:3300/0 conn(0x7f22f415dbd0 0x7f22f415bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22ea000640 1 -- 172.21.15.46:0/260422202 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22f41593d0 msgr2=0x7f22f406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22ea000640 1 --2- 172.21.15.46:0/260422202 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22f41593d0 0x7f22f406cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22ea000640 1 -- 172.21.15.46:0/260422202 >> v2:172.21.15.73:3300/0 conn(0x7f22f415dbd0 msgr2=0x7f22f415bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22ea000640 1 --2- 172.21.15.46:0/260422202 >> v2:172.21.15.73:3300/0 conn(0x7f22f415dbd0 0x7f22f415bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22ea000640 1 -- 172.21.15.46:0/260422202 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f415e060 con 0x7f22f415d230 2024-03-29T08:29:18.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.365+0000 7f22f3400640 1 --2- 172.21.15.46:0/260422202 >> v2:172.21.15.73:3300/0 conn(0x7f22f415dbd0 0x7f22f415bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:18.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.369+0000 7f22ea000640 1 --2- 172.21.15.46:0/260422202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22f415d230 0x7f22f415d690 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f22f415b860 tx=0x7f22dc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc9f665d578c1e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:18.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.369+0000 7f22f2000640 1 -- 172.21.15.46:0/260422202 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0aa640 con 0x7f22f415d230 2024-03-29T08:29:18.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.369+0000 7f22f2000640 1 -- 172.21.15.46:0/260422202 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc0aa7a0 con 0x7f22f415d230 2024-03-29T08:29:18.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.369+0000 7f22f2000640 1 -- 172.21.15.46:0/260422202 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0aab30 con 0x7f22f415d230 2024-03-29T08:29:18.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.369+0000 7f22f9600640 1 -- 172.21.15.46:0/260422202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22f415d230 msgr2=0x7f22f415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:18.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.369+0000 7f22f9600640 1 --2- 172.21.15.46:0/260422202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22f415d230 0x7f22f415d690 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f22f415b860 tx=0x7f22dc01c520 comp rx=0 tx=0).stop 2024-03-29T08:29:18.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.369+0000 7f22f9600640 1 -- 172.21.15.46:0/260422202 shutdown_connections 2024-03-29T08:29:18.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.369+0000 7f22f9600640 1 --2- 172.21.15.46:0/260422202 >> v2:172.21.15.73:3300/0 conn(0x7f22f415dbd0 0x7f22f415bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.369+0000 7f22f9600640 1 --2- 172.21.15.46:0/260422202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22f415d230 0x7f22f415d690 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.369+0000 7f22f9600640 1 --2- 172.21.15.46:0/260422202 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22f41593d0 0x7f22f406cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.369+0000 7f22f9600640 1 -- 172.21.15.46:0/260422202 >> 172.21.15.46:0/260422202 conn(0x7f22f4066110 msgr2=0x7f22f4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:18.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.369+0000 7f22f9600640 1 -- 172.21.15.46:0/260422202 shutdown_connections 2024-03-29T08:29:18.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.369+0000 7f22f9600640 1 -- 172.21.15.46:0/260422202 wait complete. 2024-03-29T08:29:18.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f9600640 1 Processor -- start 2024-03-29T08:29:18.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f9600640 1 -- start start 2024-03-29T08:29:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f9600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22f41593d0 0x7f22f4155830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f9600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22f415d230 0x7f22f40cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f9600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22f415dbd0 0x7f22f40cd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f9600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f41f6400 con 0x7f22f415dbd0 2024-03-29T08:29:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f9600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f41f66e0 con 0x7f22f415d230 2024-03-29T08:29:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f9600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f41f69c0 con 0x7f22f41593d0 2024-03-29T08:29:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22f415dbd0 0x7f22f40cd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22f415dbd0 0x7f22f40cd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39036/0 (socket says 172.21.15.46:39036) 2024-03-29T08:29:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f3400640 1 -- 172.21.15.46:0/2299790533 learned_addr learned my addr 172.21.15.46:0/2299790533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f2a00640 1 --2- 172.21.15.46:0/2299790533 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22f41593d0 0x7f22f4155830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22ea000640 1 --2- 172.21.15.46:0/2299790533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22f415d230 0x7f22f40cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f3400640 1 -- 172.21.15.46:0/2299790533 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22f41593d0 msgr2=0x7f22f4155830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f3400640 1 --2- 172.21.15.46:0/2299790533 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22f41593d0 0x7f22f4155830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f3400640 1 -- 172.21.15.46:0/2299790533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22f415d230 msgr2=0x7f22f40cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f3400640 1 --2- 172.21.15.46:0/2299790533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22f415d230 0x7f22f40cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f3400640 1 -- 172.21.15.46:0/2299790533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22dc0094d0 con 0x7f22f415dbd0 2024-03-29T08:29:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f3400640 1 --2- 172.21.15.46:0/2299790533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22f415dbd0 0x7f22f40cd870 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00cef0 tx=0x7f22ec00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22eb400640 1 -- 172.21.15.46:0/2299790533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec00d840 con 0x7f22f415dbd0 2024-03-29T08:29:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22eb400640 1 -- 172.21.15.46:0/2299790533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ec00d9c0 con 0x7f22f415dbd0 2024-03-29T08:29:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22eb400640 1 -- 172.21.15.46:0/2299790533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ec00dd70 con 0x7f22f415dbd0 2024-03-29T08:29:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f9600640 1 -- 172.21.15.46:0/2299790533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f41f6ca0 con 0x7f22f415dbd0 2024-03-29T08:29:18.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.373+0000 7f22f9600640 1 -- 172.21.15.46:0/2299790533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f4155d70 con 0x7f22f415dbd0 2024-03-29T08:29:18.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.377+0000 7f22f9600640 1 -- 172.21.15.46:0/2299790533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f4066c90 con 0x7f22f415dbd0 2024-03-29T08:29:18.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.377+0000 7f22eb400640 1 -- 172.21.15.46:0/2299790533 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f22ec01e3b0 con 0x7f22f415dbd0 2024-03-29T08:29:18.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.377+0000 7f22eb400640 1 --2- 172.21.15.46:0/2299790533 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22d0066690 0x7f22d0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.377+0000 7f22eb400640 1 -- 172.21.15.46:0/2299790533 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22ec084c40 con 0x7f22f415dbd0 2024-03-29T08:29:18.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.377+0000 7f22f2a00640 1 --2- 172.21.15.46:0/2299790533 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22d0066690 0x7f22d0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.377+0000 7f22f2a00640 1 --2- 172.21.15.46:0/2299790533 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22d0066690 0x7f22d0068b50 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f22f40ce590 tx=0x7f22e4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:18.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.381+0000 7f22eb400640 1 -- 172.21.15.46:0/2299790533 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f22ec052080 con 0x7f22f415dbd0 2024-03-29T08:29:18.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.541+0000 7f22f9600640 1 -- 172.21.15.46:0/2299790533 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22f40710b0 con 0x7f22d0066690 2024-03-29T08:29:18.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.549+0000 7f22eb400640 1 -- 172.21.15.46:0/2299790533 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289105 (secure 0 0 0) 0x7f22f40710b0 con 0x7f22d0066690 2024-03-29T08:29:18.558 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:18.560 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:29:18.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.553+0000 7f22f9600640 1 -- 172.21.15.46:0/2299790533 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22d0066690 msgr2=0x7f22d0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:18.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.553+0000 7f22f9600640 1 --2- 172.21.15.46:0/2299790533 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22d0066690 0x7f22d0068b50 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f22f40ce590 tx=0x7f22e4008040 comp rx=0 tx=0).stop 2024-03-29T08:29:18.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.553+0000 7f22f9600640 1 -- 172.21.15.46:0/2299790533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22f415dbd0 msgr2=0x7f22f40cd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:18.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.553+0000 7f22f9600640 1 --2- 172.21.15.46:0/2299790533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22f415dbd0 0x7f22f40cd870 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f22ec00cef0 tx=0x7f22ec00a710 comp rx=0 tx=0).stop 2024-03-29T08:29:18.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.553+0000 7f22f9600640 1 -- 172.21.15.46:0/2299790533 shutdown_connections 2024-03-29T08:29:18.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.553+0000 7f22f9600640 1 --2- 172.21.15.46:0/2299790533 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f22d0066690 0x7f22d0068b50 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.553+0000 7f22f9600640 1 --2- 172.21.15.46:0/2299790533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f22f415d230 0x7f22f40cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.553+0000 7f22f9600640 1 --2- 172.21.15.46:0/2299790533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22f415dbd0 0x7f22f40cd870 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.553+0000 7f22f9600640 1 --2- 172.21.15.46:0/2299790533 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f22f41593d0 0x7f22f4155830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.553+0000 7f22f9600640 1 -- 172.21.15.46:0/2299790533 >> 172.21.15.46:0/2299790533 conn(0x7f22f4066110 msgr2=0x7f22f4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:18.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.553+0000 7f22f9600640 1 -- 172.21.15.46:0/2299790533 shutdown_connections 2024-03-29T08:29:18.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.553+0000 7f22f9600640 1 -- 172.21.15.46:0/2299790533 wait complete. 2024-03-29T08:29:18.578 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-03-29T08:29:18.211688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261740,"kb_used_data":12764,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754712980,"statfs":{"total":773094113280,"available":772826091520,"internally_reserved":0,"allocated":13070336,"data_stored":6325748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.148834"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509634+0000","last_change":"2024-03-29T08:29:14.504298+0000","last_active":"2024-03-29T08:29:14.509634+0000","last_peered":"2024-03-29T08:29:14.509634+0000","last_clean":"2024-03-29T08:29:14.509634+0000","last_became_active":"2024-03-29T08:29:14.504060+0000","last_became_peered":"2024-03-29T08:29:14.504060+0000","last_unstale":"2024-03-29T08:29:14.509634+0000","last_undegraded":"2024-03-29T08:29:14.509634+0000","last_fullsized":"2024-03-29T08:29:14.509634+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:47:24.542958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.491055+0000","last_change":"2024-03-29T08:29:14.488152+0000","last_active":"2024-03-29T08:29:14.491055+0000","last_peered":"2024-03-29T08:29:14.491055+0000","last_clean":"2024-03-29T08:29:14.491055+0000","last_became_active":"2024-03-29T08:29:14.487884+0000","last_became_peered":"2024-03-29T08:29:14.487884+0000","last_unstale":"2024-03-29T08:29:14.491055+0000","last_undegraded":"2024-03-29T08:29:14.491055+0000","last_fullsized":"2024-03-29T08:29:14.491055+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:49:14.237508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.492505+0000","last_change":"2024-03-29T08:29:14.490304+0000","last_active":"2024-03-29T08:29:14.492505+0000","last_peered":"2024-03-29T08:29:14.492505+0000","last_clean":"2024-03-29T08:29:14.492505+0000","last_became_active":"2024-03-29T08:29:14.489916+0000","last_became_peered":"2024-03-29T08:29:14.489916+0000","last_unstale":"2024-03-29T08:29:14.492505+0000","last_undegraded":"2024-03-29T08:29:14.492505+0000","last_fullsized":"2024-03-29T08:29:14.492505+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:55.868596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.510337+0000","last_change":"2024-03-29T08:29:14.501687+0000","last_active":"2024-03-29T08:29:14.510337+0000","last_peered":"2024-03-29T08:29:14.510337+0000","last_clean":"2024-03-29T08:29:14.510337+0000","last_became_active":"2024-03-29T08:29:14.501377+0000","last_became_peered":"2024-03-29T08:29:14.501377+0000","last_unstale":"2024-03-29T08:29:14.510337+0000","last_undegraded":"2024-03-29T08:29:14.510337+0000","last_fullsized":"2024-03-29T08:29:14.510337+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:38:29.985399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.506541+0000","last_change":"2024-03-29T08:29:14.504615+0000","last_active":"2024-03-29T08:29:14.506541+0000","last_peered":"2024-03-29T08:29:14.506541+0000","last_clean":"2024-03-29T08:29:14.506541+0000","last_became_active":"2024-03-29T08:29:14.504347+0000","last_became_peered":"2024-03-29T08:29:14.504347+0000","last_unstale":"2024-03-29T08:29:14.506541+0000","last_undegraded":"2024-03-29T08:29:14.506541+0000","last_fullsized":"2024-03-29T08:29:14.506541+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:25:45.763218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.084660+0000","last_change":"2024-03-29T08:29:14.084660+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.084660+0000","last_undegraded":"2024-03-29T08:29:14.084660+0000","last_fullsized":"2024-03-29T08:29:14.084660+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:51.101298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.503717+0000","last_change":"2024-03-29T08:29:14.494643+0000","last_active":"2024-03-29T08:29:14.503717+0000","last_peered":"2024-03-29T08:29:14.503717+0000","last_clean":"2024-03-29T08:29:14.503717+0000","last_became_active":"2024-03-29T08:29:14.494385+0000","last_became_peered":"2024-03-29T08:29:14.494385+0000","last_unstale":"2024-03-29T08:29:14.503717+0000","last_undegraded":"2024-03-29T08:29:14.503717+0000","last_fullsized":"2024-03-29T08:29:14.503717+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:11:26.639183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.495613+0000","last_change":"2024-03-29T08:29:14.489843+0000","last_active":"2024-03-29T08:29:14.495613+0000","last_peered":"2024-03-29T08:29:14.495613+0000","last_clean":"2024-03-29T08:29:14.495613+0000","last_became_active":"2024-03-29T08:29:14.489598+0000","last_became_peered":"2024-03-29T08:29:14.489598+0000","last_unstale":"2024-03-29T08:29:14.495613+0000","last_undegraded":"2024-03-29T08:29:14.495613+0000","last_fullsized":"2024-03-29T08:29:14.495613+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:49:19.274668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.512231+0000","last_change":"2024-03-29T08:29:14.507881+0000","last_active":"2024-03-29T08:29:14.512231+0000","last_peered":"2024-03-29T08:29:14.512231+0000","last_clean":"2024-03-29T08:29:14.512231+0000","last_became_active":"2024-03-29T08:29:14.507445+0000","last_became_peered":"2024-03-29T08:29:14.507445+0000","last_unstale":"2024-03-29T08:29:14.512231+0000","last_undegraded":"2024-03-29T08:29:14.512231+0000","last_fullsized":"2024-03-29T08:29:14.512231+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:50:39.114131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504217+0000","last_change":"2024-03-29T08:29:14.500130+0000","last_active":"2024-03-29T08:29:14.504217+0000","last_peered":"2024-03-29T08:29:14.504217+0000","last_clean":"2024-03-29T08:29:14.504217+0000","last_became_active":"2024-03-29T08:29:14.499827+0000","last_became_peered":"2024-03-29T08:29:14.499827+0000","last_unstale":"2024-03-29T08:29:14.504217+0000","last_undegraded":"2024-03-29T08:29:14.504217+0000","last_fullsized":"2024-03-29T08:29:14.504217+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:01.476642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.502452+0000","last_change":"2024-03-29T08:29:14.494953+0000","last_active":"2024-03-29T08:29:14.502452+0000","last_peered":"2024-03-29T08:29:14.502452+0000","last_clean":"2024-03-29T08:29:14.502452+0000","last_became_active":"2024-03-29T08:29:14.494599+0000","last_became_peered":"2024-03-29T08:29:14.494599+0000","last_unstale":"2024-03-29T08:29:14.502452+0000","last_undegraded":"2024-03-29T08:29:14.502452+0000","last_fullsized":"2024-03-29T08:29:14.502452+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:49:02.417397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504246+0000","last_change":"2024-03-29T08:29:14.499160+0000","last_active":"2024-03-29T08:29:14.504246+0000","last_peered":"2024-03-29T08:29:14.504246+0000","last_clean":"2024-03-29T08:29:14.504246+0000","last_became_active":"2024-03-29T08:29:14.498900+0000","last_became_peered":"2024-03-29T08:29:14.498900+0000","last_unstale":"2024-03-29T08:29:14.504246+0000","last_undegraded":"2024-03-29T08:29:14.504246+0000","last_fullsized":"2024-03-29T08:29:14.504246+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:19.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.493310+0000","last_change":"2024-03-29T08:29:14.486530+0000","last_active":"2024-03-29T08:29:14.493310+0000","last_peered":"2024-03-29T08:29:14.493310+0000","last_clean":"2024-03-29T08:29:14.493310+0000","last_became_active":"2024-03-29T08:29:14.486158+0000","last_became_peered":"2024-03-29T08:29:14.486158+0000","last_unstale":"2024-03-29T08:29:14.493310+0000","last_undegraded":"2024-03-29T08:29:14.493310+0000","last_fullsized":"2024-03-29T08:29:14.493310+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:40.282997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508861+0000","last_change":"2024-03-29T08:29:14.507343+0000","last_active":"2024-03-29T08:29:14.508861+0000","last_peered":"2024-03-29T08:29:14.508861+0000","last_clean":"2024-03-29T08:29:14.508861+0000","last_became_active":"2024-03-29T08:29:14.507072+0000","last_became_peered":"2024-03-29T08:29:14.507072+0000","last_unstale":"2024-03-29T08:29:14.508861+0000","last_undegraded":"2024-03-29T08:29:14.508861+0000","last_fullsized":"2024-03-29T08:29:14.508861+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:19.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509523+0000","last_change":"2024-03-29T08:29:14.498131+0000","last_active":"2024-03-29T08:29:14.509523+0000","last_peered":"2024-03-29T08:29:14.509523+0000","last_clean":"2024-03-29T08:29:14.509523+0000","last_became_active":"2024-03-29T08:29:14.497856+0000","last_became_peered":"2024-03-29T08:29:14.497856+0000","last_unstale":"2024-03-29T08:29:14.509523+0000","last_undegraded":"2024-03-29T08:29:14.509523+0000","last_fullsized":"2024-03-29T08:29:14.509523+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:43:24.083895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504685+0000","last_change":"2024-03-29T08:29:14.499902+0000","last_active":"2024-03-29T08:29:14.504685+0000","last_peered":"2024-03-29T08:29:14.504685+0000","last_clean":"2024-03-29T08:29:14.504685+0000","last_became_active":"2024-03-29T08:29:14.499642+0000","last_became_peered":"2024-03-29T08:29:14.499642+0000","last_unstale":"2024-03-29T08:29:14.504685+0000","last_undegraded":"2024-03-29T08:29:14.504685+0000","last_fullsized":"2024-03-29T08:29:14.504685+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:06:54.374100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.502730+0000","last_change":"2024-03-29T08:29:14.496277+0000","last_active":"2024-03-29T08:29:14.502730+0000","last_peered":"2024-03-29T08:29:14.502730+0000","last_clean":"2024-03-29T08:29:14.502730+0000","last_became_active":"2024-03-29T08:29:14.496028+0000","last_became_peered":"2024-03-29T08:29:14.496028+0000","last_unstale":"2024-03-29T08:29:14.502730+0000","last_undegraded":"2024-03-29T08:29:14.502730+0000","last_fullsized":"2024-03-29T08:29:14.502730+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:08:59.334138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505390+0000","last_change":"2024-03-29T08:29:14.503103+0000","last_active":"2024-03-29T08:29:14.505390+0000","last_peered":"2024-03-29T08:29:14.505390+0000","last_clean":"2024-03-29T08:29:14.505390+0000","last_became_active":"2024-03-29T08:29:14.502841+0000","last_became_peered":"2024-03-29T08:29:14.502841+0000","last_unstale":"2024-03-29T08:29:14.505390+0000","last_undegraded":"2024-03-29T08:29:14.505390+0000","last_fullsized":"2024-03-29T08:29:14.505390+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:11:21.620572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.498334+0000","last_change":"2024-03-29T08:29:14.491176+0000","last_active":"2024-03-29T08:29:14.498334+0000","last_peered":"2024-03-29T08:29:14.498334+0000","last_clean":"2024-03-29T08:29:14.498334+0000","last_became_active":"2024-03-29T08:29:14.490845+0000","last_became_peered":"2024-03-29T08:29:14.490845+0000","last_unstale":"2024-03-29T08:29:14.498334+0000","last_undegraded":"2024-03-29T08:29:14.498334+0000","last_fullsized":"2024-03-29T08:29:14.498334+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:44:41.312640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504227+0000","last_change":"2024-03-29T08:29:14.501383+0000","last_active":"2024-03-29T08:29:14.504227+0000","last_peered":"2024-03-29T08:29:14.504227+0000","last_clean":"2024-03-29T08:29:14.504227+0000","last_became_active":"2024-03-29T08:29:14.501106+0000","last_became_peered":"2024-03-29T08:29:14.501106+0000","last_unstale":"2024-03-29T08:29:14.504227+0000","last_undegraded":"2024-03-29T08:29:14.504227+0000","last_fullsized":"2024-03-29T08:29:14.504227+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:23:10.605671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504862+0000","last_change":"2024-03-29T08:29:14.497743+0000","last_active":"2024-03-29T08:29:14.504862+0000","last_peered":"2024-03-29T08:29:14.504862+0000","last_clean":"2024-03-29T08:29:14.504862+0000","last_became_active":"2024-03-29T08:29:14.497338+0000","last_became_peered":"2024-03-29T08:29:14.497338+0000","last_unstale":"2024-03-29T08:29:14.504862+0000","last_undegraded":"2024-03-29T08:29:14.504862+0000","last_fullsized":"2024-03-29T08:29:14.504862+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:24.513091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.084054+0000","last_change":"2024-03-29T08:29:14.084054+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.084054+0000","last_undegraded":"2024-03-29T08:29:14.084054+0000","last_fullsized":"2024-03-29T08:29:14.084054+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:57:23.884319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.506608+0000","last_change":"2024-03-29T08:29:14.502935+0000","last_active":"2024-03-29T08:29:14.506608+0000","last_peered":"2024-03-29T08:29:14.506608+0000","last_clean":"2024-03-29T08:29:14.506608+0000","last_became_active":"2024-03-29T08:29:14.502701+0000","last_became_peered":"2024-03-29T08:29:14.502701+0000","last_unstale":"2024-03-29T08:29:14.506608+0000","last_undegraded":"2024-03-29T08:29:14.506608+0000","last_fullsized":"2024-03-29T08:29:14.506608+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:35.756569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.511647+0000","last_change":"2024-03-29T08:29:14.507544+0000","last_active":"2024-03-29T08:29:14.511647+0000","last_peered":"2024-03-29T08:29:14.511647+0000","last_clean":"2024-03-29T08:29:14.511647+0000","last_became_active":"2024-03-29T08:29:14.507285+0000","last_became_peered":"2024-03-29T08:29:14.507285+0000","last_unstale":"2024-03-29T08:29:14.511647+0000","last_undegraded":"2024-03-29T08:29:14.511647+0000","last_fullsized":"2024-03-29T08:29:14.511647+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:50:14.687728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.494822+0000","last_change":"2024-03-29T08:29:14.492853+0000","last_active":"2024-03-29T08:29:14.494822+0000","last_peered":"2024-03-29T08:29:14.494822+0000","last_clean":"2024-03-29T08:29:14.494822+0000","last_became_active":"2024-03-29T08:29:14.492537+0000","last_became_peered":"2024-03-29T08:29:14.492537+0000","last_unstale":"2024-03-29T08:29:14.494822+0000","last_undegraded":"2024-03-29T08:29:14.494822+0000","last_fullsized":"2024-03-29T08:29:14.494822+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:08:28.043449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.085680+0000","last_change":"2024-03-29T08:29:14.085680+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.085680+0000","last_undegraded":"2024-03-29T08:29:14.085680+0000","last_fullsized":"2024-03-29T08:29:14.085680+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:10:44.097211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.511774+0000","last_change":"2024-03-29T08:29:14.508869+0000","last_active":"2024-03-29T08:29:14.511774+0000","last_peered":"2024-03-29T08:29:14.511774+0000","last_clean":"2024-03-29T08:29:14.511774+0000","last_became_active":"2024-03-29T08:29:14.508466+0000","last_became_peered":"2024-03-29T08:29:14.508466+0000","last_unstale":"2024-03-29T08:29:14.511774+0000","last_undegraded":"2024-03-29T08:29:14.511774+0000","last_fullsized":"2024-03-29T08:29:14.511774+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:08:28.619413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.087721+0000","last_change":"2024-03-29T08:29:14.087721+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.087721+0000","last_undegraded":"2024-03-29T08:29:14.087721+0000","last_fullsized":"2024-03-29T08:29:14.087721+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:53.749284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508906+0000","last_change":"2024-03-29T08:29:14.507369+0000","last_active":"2024-03-29T08:29:14.508906+0000","last_peered":"2024-03-29T08:29:14.508906+0000","last_clean":"2024-03-29T08:29:14.508906+0000","last_became_active":"2024-03-29T08:29:14.507164+0000","last_became_peered":"2024-03-29T08:29:14.507164+0000","last_unstale":"2024-03-29T08:29:14.508906+0000","last_undegraded":"2024-03-29T08:29:14.508906+0000","last_fullsized":"2024-03-29T08:29:14.508906+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:20:13.956872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505661+0000","last_change":"2024-03-29T08:29:14.503526+0000","last_active":"2024-03-29T08:29:14.505661+0000","last_peered":"2024-03-29T08:29:14.505661+0000","last_clean":"2024-03-29T08:29:14.505661+0000","last_became_active":"2024-03-29T08:29:14.503149+0000","last_became_peered":"2024-03-29T08:29:14.503149+0000","last_unstale":"2024-03-29T08:29:14.505661+0000","last_undegraded":"2024-03-29T08:29:14.505661+0000","last_fullsized":"2024-03-29T08:29:14.505661+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:46:13.948298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509541+0000","last_change":"2024-03-29T08:29:14.500403+0000","last_active":"2024-03-29T08:29:14.509541+0000","last_peered":"2024-03-29T08:29:14.509541+0000","last_clean":"2024-03-29T08:29:14.509541+0000","last_became_active":"2024-03-29T08:29:14.500132+0000","last_became_peered":"2024-03-29T08:29:14.500132+0000","last_unstale":"2024-03-29T08:29:14.509541+0000","last_undegraded":"2024-03-29T08:29:14.509541+0000","last_fullsized":"2024-03-29T08:29:14.509541+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:09:34.129852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504668+0000","last_change":"2024-03-29T08:29:14.499479+0000","last_active":"2024-03-29T08:29:14.504668+0000","last_peered":"2024-03-29T08:29:14.504668+0000","last_clean":"2024-03-29T08:29:14.504668+0000","last_became_active":"2024-03-29T08:29:14.498994+0000","last_became_peered":"2024-03-29T08:29:14.498994+0000","last_unstale":"2024-03-29T08:29:14.504668+0000","last_undegraded":"2024-03-29T08:29:14.504668+0000","last_fullsized":"2024-03-29T08:29:14.504668+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:25:13.598107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.513004+0000","last_change":"2024-03-29T08:29:14.511693+0000","last_active":"2024-03-29T08:29:14.513004+0000","last_peered":"2024-03-29T08:29:14.513004+0000","last_clean":"2024-03-29T08:29:14.513004+0000","last_became_active":"2024-03-29T08:29:14.511041+0000","last_became_peered":"2024-03-29T08:29:14.511041+0000","last_unstale":"2024-03-29T08:29:14.513004+0000","last_undegraded":"2024-03-29T08:29:14.513004+0000","last_fullsized":"2024-03-29T08:29:14.513004+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:06.874362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"120'4","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507837+0000","last_change":"2024-03-29T08:29:14.086060+0000","last_active":"2024-03-29T08:29:14.507837+0000","last_peered":"2024-03-29T08:29:14.507837+0000","last_clean":"2024-03-29T08:29:14.507837+0000","last_became_active":"2024-03-29T08:29:14.085788+0000","last_became_peered":"2024-03-29T08:29:14.085788+0000","last_unstale":"2024-03-29T08:29:14.507837+0000","last_undegraded":"2024-03-29T08:29:14.507837+0000","last_fullsized":"2024-03-29T08:29:14.507837+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:52:04.161387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507246+0000","last_change":"2024-03-29T08:29:14.503865+0000","last_active":"2024-03-29T08:29:14.507246+0000","last_peered":"2024-03-29T08:29:14.507246+0000","last_clean":"2024-03-29T08:29:14.507246+0000","last_became_active":"2024-03-29T08:29:14.503622+0000","last_became_peered":"2024-03-29T08:29:14.503622+0000","last_unstale":"2024-03-29T08:29:14.507246+0000","last_undegraded":"2024-03-29T08:29:14.507246+0000","last_fullsized":"2024-03-29T08:29:14.507246+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:48:23.465613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.501135+0000","last_change":"2024-03-29T08:29:14.084881+0000","last_active":"2024-03-29T08:29:14.501135+0000","last_peered":"2024-03-29T08:29:14.501135+0000","last_clean":"2024-03-29T08:29:14.501135+0000","last_became_active":"2024-03-29T08:29:14.084587+0000","last_became_peered":"2024-03-29T08:29:14.084587+0000","last_unstale":"2024-03-29T08:29:14.501135+0000","last_undegraded":"2024-03-29T08:29:14.501135+0000","last_fullsized":"2024-03-29T08:29:14.501135+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:45.558553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.085712+0000","last_change":"2024-03-29T08:29:14.085712+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.085712+0000","last_undegraded":"2024-03-29T08:29:14.085712+0000","last_fullsized":"2024-03-29T08:29:14.085712+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:23:32.467222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509018+0000","last_change":"2024-03-29T08:29:14.089964+0000","last_active":"2024-03-29T08:29:14.509018+0000","last_peered":"2024-03-29T08:29:14.509018+0000","last_clean":"2024-03-29T08:29:14.509018+0000","last_became_active":"2024-03-29T08:29:14.088099+0000","last_became_peered":"2024-03-29T08:29:14.088099+0000","last_unstale":"2024-03-29T08:29:14.509018+0000","last_undegraded":"2024-03-29T08:29:14.509018+0000","last_fullsized":"2024-03-29T08:29:14.509018+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:05.868646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.498316+0000","last_change":"2024-03-29T08:29:14.491102+0000","last_active":"2024-03-29T08:29:14.498316+0000","last_peered":"2024-03-29T08:29:14.498316+0000","last_clean":"2024-03-29T08:29:14.498316+0000","last_became_active":"2024-03-29T08:29:14.490878+0000","last_became_peered":"2024-03-29T08:29:14.490878+0000","last_unstale":"2024-03-29T08:29:14.498316+0000","last_undegraded":"2024-03-29T08:29:14.498316+0000","last_fullsized":"2024-03-29T08:29:14.498316+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:06:54.374100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509548+0000","last_change":"2024-03-29T08:29:14.097684+0000","last_active":"2024-03-29T08:29:14.509548+0000","last_peered":"2024-03-29T08:29:14.509548+0000","last_clean":"2024-03-29T08:29:14.509548+0000","last_became_active":"2024-03-29T08:29:14.095879+0000","last_became_peered":"2024-03-29T08:29:14.095879+0000","last_unstale":"2024-03-29T08:29:14.509548+0000","last_undegraded":"2024-03-29T08:29:14.509548+0000","last_fullsized":"2024-03-29T08:29:14.509548+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:50:38.108416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.511687+0000","last_change":"2024-03-29T08:29:14.507697+0000","last_active":"2024-03-29T08:29:14.511687+0000","last_peered":"2024-03-29T08:29:14.511687+0000","last_clean":"2024-03-29T08:29:14.511687+0000","last_became_active":"2024-03-29T08:29:14.507356+0000","last_became_peered":"2024-03-29T08:29:14.507356+0000","last_unstale":"2024-03-29T08:29:14.511687+0000","last_undegraded":"2024-03-29T08:29:14.511687+0000","last_fullsized":"2024-03-29T08:29:14.511687+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:11.283108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.502762+0000","last_change":"2024-03-29T08:29:14.086806+0000","last_active":"2024-03-29T08:29:14.502762+0000","last_peered":"2024-03-29T08:29:14.502762+0000","last_clean":"2024-03-29T08:29:14.502762+0000","last_became_active":"2024-03-29T08:29:14.086468+0000","last_became_peered":"2024-03-29T08:29:14.086468+0000","last_unstale":"2024-03-29T08:29:14.502762+0000","last_undegraded":"2024-03-29T08:29:14.502762+0000","last_fullsized":"2024-03-29T08:29:14.502762+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:37:57.757695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504878+0000","last_change":"2024-03-29T08:29:14.498041+0000","last_active":"2024-03-29T08:29:14.504878+0000","last_peered":"2024-03-29T08:29:14.504878+0000","last_clean":"2024-03-29T08:29:14.504878+0000","last_became_active":"2024-03-29T08:29:14.497768+0000","last_became_peered":"2024-03-29T08:29:14.497768+0000","last_unstale":"2024-03-29T08:29:14.504878+0000","last_undegraded":"2024-03-29T08:29:14.504878+0000","last_fullsized":"2024-03-29T08:29:14.504878+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:55.518674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504448+0000","last_change":"2024-03-29T08:29:14.088920+0000","last_active":"2024-03-29T08:29:14.504448+0000","last_peered":"2024-03-29T08:29:14.504448+0000","last_clean":"2024-03-29T08:29:14.504448+0000","last_became_active":"2024-03-29T08:29:14.088593+0000","last_became_peered":"2024-03-29T08:29:14.088593+0000","last_unstale":"2024-03-29T08:29:14.504448+0000","last_undegraded":"2024-03-29T08:29:14.504448+0000","last_fullsized":"2024-03-29T08:29:14.504448+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:51.446071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504649+0000","last_change":"2024-03-29T08:29:14.499273+0000","last_active":"2024-03-29T08:29:14.504649+0000","last_peered":"2024-03-29T08:29:14.504649+0000","last_clean":"2024-03-29T08:29:14.504649+0000","last_became_active":"2024-03-29T08:29:14.498368+0000","last_became_peered":"2024-03-29T08:29:14.498368+0000","last_unstale":"2024-03-29T08:29:14.504649+0000","last_undegraded":"2024-03-29T08:29:14.504649+0000","last_fullsized":"2024-03-29T08:29:14.504649+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:23:10.605671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.492779+0000","last_change":"2024-03-29T08:29:14.079054+0000","last_active":"2024-03-29T08:29:14.492779+0000","last_peered":"2024-03-29T08:29:14.492779+0000","last_clean":"2024-03-29T08:29:14.492779+0000","last_became_active":"2024-03-29T08:29:14.078272+0000","last_became_peered":"2024-03-29T08:29:14.078272+0000","last_unstale":"2024-03-29T08:29:14.492779+0000","last_undegraded":"2024-03-29T08:29:14.492779+0000","last_fullsized":"2024-03-29T08:29:14.492779+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:57:54.510656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.503738+0000","last_change":"2024-03-29T08:29:14.495103+0000","last_active":"2024-03-29T08:29:14.503738+0000","last_peered":"2024-03-29T08:29:14.503738+0000","last_clean":"2024-03-29T08:29:14.503738+0000","last_became_active":"2024-03-29T08:29:14.494873+0000","last_became_peered":"2024-03-29T08:29:14.494873+0000","last_unstale":"2024-03-29T08:29:14.503738+0000","last_undegraded":"2024-03-29T08:29:14.503738+0000","last_fullsized":"2024-03-29T08:29:14.503738+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:49:15.540195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.484296+0000","last_change":"2024-03-29T08:29:14.091783+0000","last_active":"2024-03-29T08:29:14.484296+0000","last_peered":"2024-03-29T08:29:14.484296+0000","last_clean":"2024-03-29T08:29:14.484296+0000","last_became_active":"2024-03-29T08:29:14.091531+0000","last_became_peered":"2024-03-29T08:29:14.091531+0000","last_unstale":"2024-03-29T08:29:14.484296+0000","last_undegraded":"2024-03-29T08:29:14.484296+0000","last_fullsized":"2024-03-29T08:29:14.484296+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:32:23.859154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504693+0000","last_change":"2024-03-29T08:29:14.498939+0000","last_active":"2024-03-29T08:29:14.504693+0000","last_peered":"2024-03-29T08:29:14.504693+0000","last_clean":"2024-03-29T08:29:14.504693+0000","last_became_active":"2024-03-29T08:29:14.498589+0000","last_became_peered":"2024-03-29T08:29:14.498589+0000","last_unstale":"2024-03-29T08:29:14.504693+0000","last_undegraded":"2024-03-29T08:29:14.504693+0000","last_fullsized":"2024-03-29T08:29:14.504693+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:32:24.719500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.496023+0000","last_change":"2024-03-29T08:29:14.096474+0000","last_active":"2024-03-29T08:29:14.496023+0000","last_peered":"2024-03-29T08:29:14.496023+0000","last_clean":"2024-03-29T08:29:14.496023+0000","last_became_active":"2024-03-29T08:29:14.094336+0000","last_became_peered":"2024-03-29T08:29:14.094336+0000","last_unstale":"2024-03-29T08:29:14.496023+0000","last_undegraded":"2024-03-29T08:29:14.496023+0000","last_fullsized":"2024-03-29T08:29:14.496023+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:39.277282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.489947+0000","last_change":"2024-03-29T08:29:14.487254+0000","last_active":"2024-03-29T08:29:14.489947+0000","last_peered":"2024-03-29T08:29:14.489947+0000","last_clean":"2024-03-29T08:29:14.489947+0000","last_became_active":"2024-03-29T08:29:14.485472+0000","last_became_peered":"2024-03-29T08:29:14.485472+0000","last_unstale":"2024-03-29T08:29:14.489947+0000","last_undegraded":"2024-03-29T08:29:14.489947+0000","last_fullsized":"2024-03-29T08:29:14.489947+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:08:57.590159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.484431+0000","last_change":"2024-03-29T08:29:14.092400+0000","last_active":"2024-03-29T08:29:14.484431+0000","last_peered":"2024-03-29T08:29:14.484431+0000","last_clean":"2024-03-29T08:29:14.484431+0000","last_became_active":"2024-03-29T08:29:14.091857+0000","last_became_peered":"2024-03-29T08:29:14.091857+0000","last_unstale":"2024-03-29T08:29:14.484431+0000","last_undegraded":"2024-03-29T08:29:14.484431+0000","last_fullsized":"2024-03-29T08:29:14.484431+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:37:57.757695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.085301+0000","last_change":"2024-03-29T08:29:14.085301+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.085301+0000","last_undegraded":"2024-03-29T08:29:14.085301+0000","last_fullsized":"2024-03-29T08:29:14.085301+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:45.863920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"120'3","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509397+0000","last_change":"2024-03-29T08:29:14.092891+0000","last_active":"2024-03-29T08:29:14.509397+0000","last_peered":"2024-03-29T08:29:14.509397+0000","last_clean":"2024-03-29T08:29:14.509397+0000","last_became_active":"2024-03-29T08:29:14.092632+0000","last_became_peered":"2024-03-29T08:29:14.092632+0000","last_unstale":"2024-03-29T08:29:14.509397+0000","last_undegraded":"2024-03-29T08:29:14.509397+0000","last_fullsized":"2024-03-29T08:29:14.509397+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:52:04.161387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.492713+0000","last_change":"2024-03-29T08:29:14.489765+0000","last_active":"2024-03-29T08:29:14.492713+0000","last_peered":"2024-03-29T08:29:14.492713+0000","last_clean":"2024-03-29T08:29:14.492713+0000","last_became_active":"2024-03-29T08:29:14.489440+0000","last_became_peered":"2024-03-29T08:29:14.489440+0000","last_unstale":"2024-03-29T08:29:14.492713+0000","last_undegraded":"2024-03-29T08:29:14.492713+0000","last_fullsized":"2024-03-29T08:29:14.492713+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:53:42.976104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"120'6","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.512807+0000","last_change":"2024-03-29T08:29:14.096725+0000","last_active":"2024-03-29T08:29:14.512807+0000","last_peered":"2024-03-29T08:29:14.512807+0000","last_clean":"2024-03-29T08:29:14.512807+0000","last_became_active":"2024-03-29T08:29:14.094695+0000","last_became_peered":"2024-03-29T08:29:14.094695+0000","last_unstale":"2024-03-29T08:29:14.512807+0000","last_undegraded":"2024-03-29T08:29:14.512807+0000","last_fullsized":"2024-03-29T08:29:14.512807+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:15:28.507662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504528+0000","last_change":"2024-03-29T08:29:14.499972+0000","last_active":"2024-03-29T08:29:14.504528+0000","last_peered":"2024-03-29T08:29:14.504528+0000","last_clean":"2024-03-29T08:29:14.504528+0000","last_became_active":"2024-03-29T08:29:14.499698+0000","last_became_peered":"2024-03-29T08:29:14.499698+0000","last_unstale":"2024-03-29T08:29:14.504528+0000","last_undegraded":"2024-03-29T08:29:14.504528+0000","last_fullsized":"2024-03-29T08:29:14.504528+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:44:41.312640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508984+0000","last_change":"2024-03-29T08:29:14.092574+0000","last_active":"2024-03-29T08:29:14.508984+0000","last_peered":"2024-03-29T08:29:14.508984+0000","last_clean":"2024-03-29T08:29:14.508984+0000","last_became_active":"2024-03-29T08:29:14.092004+0000","last_became_peered":"2024-03-29T08:29:14.092004+0000","last_unstale":"2024-03-29T08:29:14.508984+0000","last_undegraded":"2024-03-29T08:29:14.508984+0000","last_fullsized":"2024-03-29T08:29:14.508984+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:21.448128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.510447+0000","last_change":"2024-03-29T08:29:14.502264+0000","last_active":"2024-03-29T08:29:14.510447+0000","last_peered":"2024-03-29T08:29:14.510447+0000","last_clean":"2024-03-29T08:29:14.510447+0000","last_became_active":"2024-03-29T08:29:14.501882+0000","last_became_peered":"2024-03-29T08:29:14.501882+0000","last_unstale":"2024-03-29T08:29:14.510447+0000","last_undegraded":"2024-03-29T08:29:14.510447+0000","last_fullsized":"2024-03-29T08:29:14.510447+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:31:05.279265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.480729+0000","last_change":"2024-03-29T08:29:14.079278+0000","last_active":"2024-03-29T08:29:14.480729+0000","last_peered":"2024-03-29T08:29:14.480729+0000","last_clean":"2024-03-29T08:29:14.480729+0000","last_became_active":"2024-03-29T08:29:14.078982+0000","last_became_peered":"2024-03-29T08:29:14.078982+0000","last_unstale":"2024-03-29T08:29:14.480729+0000","last_undegraded":"2024-03-29T08:29:14.480729+0000","last_fullsized":"2024-03-29T08:29:14.480729+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:06.149502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509069+0000","last_change":"2024-03-29T08:29:14.500759+0000","last_active":"2024-03-29T08:29:14.509069+0000","last_peered":"2024-03-29T08:29:14.509069+0000","last_clean":"2024-03-29T08:29:14.509069+0000","last_became_active":"2024-03-29T08:29:14.500463+0000","last_became_peered":"2024-03-29T08:29:14.500463+0000","last_unstale":"2024-03-29T08:29:14.509069+0000","last_undegraded":"2024-03-29T08:29:14.509069+0000","last_fullsized":"2024-03-29T08:29:14.509069+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:13:49.079869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"120'8","reported_seq":18,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.510685+0000","last_change":"2024-03-29T08:29:14.090542+0000","last_active":"2024-03-29T08:29:14.510685+0000","last_peered":"2024-03-29T08:29:14.510685+0000","last_clean":"2024-03-29T08:29:14.510685+0000","last_became_active":"2024-03-29T08:29:14.090235+0000","last_became_peered":"2024-03-29T08:29:14.090235+0000","last_unstale":"2024-03-29T08:29:14.510685+0000","last_undegraded":"2024-03-29T08:29:14.510685+0000","last_fullsized":"2024-03-29T08:29:14.510685+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:45.558553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508199+0000","last_change":"2024-03-29T08:29:14.497747+0000","last_active":"2024-03-29T08:29:14.508199+0000","last_peered":"2024-03-29T08:29:14.508199+0000","last_clean":"2024-03-29T08:29:14.508199+0000","last_became_active":"2024-03-29T08:29:14.497456+0000","last_became_peered":"2024-03-29T08:29:14.497456+0000","last_unstale":"2024-03-29T08:29:14.508199+0000","last_undegraded":"2024-03-29T08:29:14.508199+0000","last_fullsized":"2024-03-29T08:29:14.508199+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:49.180147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.097550+0000","last_change":"2024-03-29T08:29:14.094084+0000","last_active":"2024-03-29T08:29:14.097550+0000","last_peered":"2024-03-29T08:29:14.097550+0000","last_clean":"2024-03-29T08:29:14.097550+0000","last_became_active":"2024-03-29T08:29:14.093447+0000","last_became_peered":"2024-03-29T08:29:14.093447+0000","last_unstale":"2024-03-29T08:29:14.097550+0000","last_undegraded":"2024-03-29T08:29:14.097550+0000","last_fullsized":"2024-03-29T08:29:14.097550+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:11:26.207009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.512213+0000","last_change":"2024-03-29T08:29:14.507678+0000","last_active":"2024-03-29T08:29:14.512213+0000","last_peered":"2024-03-29T08:29:14.512213+0000","last_clean":"2024-03-29T08:29:14.512213+0000","last_became_active":"2024-03-29T08:29:14.507132+0000","last_became_peered":"2024-03-29T08:29:14.507132+0000","last_unstale":"2024-03-29T08:29:14.512213+0000","last_undegraded":"2024-03-29T08:29:14.512213+0000","last_fullsized":"2024-03-29T08:29:14.512213+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:15:29.513377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.502517+0000","last_change":"2024-03-29T08:29:14.095051+0000","last_active":"2024-03-29T08:29:14.502517+0000","last_peered":"2024-03-29T08:29:14.502517+0000","last_clean":"2024-03-29T08:29:14.502517+0000","last_became_active":"2024-03-29T08:29:14.092852+0000","last_became_peered":"2024-03-29T08:29:14.092852+0000","last_unstale":"2024-03-29T08:29:14.502517+0000","last_undegraded":"2024-03-29T08:29:14.502517+0000","last_fullsized":"2024-03-29T08:29:14.502517+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:06.149502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504967+0000","last_change":"2024-03-29T08:29:14.497468+0000","last_active":"2024-03-29T08:29:14.504967+0000","last_peered":"2024-03-29T08:29:14.504967+0000","last_clean":"2024-03-29T08:29:14.504967+0000","last_became_active":"2024-03-29T08:29:14.497061+0000","last_became_peered":"2024-03-29T08:29:14.497061+0000","last_unstale":"2024-03-29T08:29:14.504967+0000","last_undegraded":"2024-03-29T08:29:14.504967+0000","last_fullsized":"2024-03-29T08:29:14.504967+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:09:20.324353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.501012+0000","last_change":"2024-03-29T08:29:14.090528+0000","last_active":"2024-03-29T08:29:14.501012+0000","last_peered":"2024-03-29T08:29:14.501012+0000","last_clean":"2024-03-29T08:29:14.501012+0000","last_became_active":"2024-03-29T08:29:14.090055+0000","last_became_peered":"2024-03-29T08:29:14.090055+0000","last_unstale":"2024-03-29T08:29:14.501012+0000","last_undegraded":"2024-03-29T08:29:14.501012+0000","last_fullsized":"2024-03-29T08:29:14.501012+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:21.448128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505830+0000","last_change":"2024-03-29T08:29:14.502568+0000","last_active":"2024-03-29T08:29:14.505830+0000","last_peered":"2024-03-29T08:29:14.505830+0000","last_clean":"2024-03-29T08:29:14.505830+0000","last_became_active":"2024-03-29T08:29:14.500161+0000","last_became_peered":"2024-03-29T08:29:14.500161+0000","last_unstale":"2024-03-29T08:29:14.505830+0000","last_undegraded":"2024-03-29T08:29:14.505830+0000","last_fullsized":"2024-03-29T08:29:14.505830+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:25:45.763218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.487422+0000","last_change":"2024-03-29T08:29:14.078956+0000","last_active":"2024-03-29T08:29:14.487422+0000","last_peered":"2024-03-29T08:29:14.487422+0000","last_clean":"2024-03-29T08:29:14.487422+0000","last_became_active":"2024-03-29T08:29:14.078689+0000","last_became_peered":"2024-03-29T08:29:14.078689+0000","last_unstale":"2024-03-29T08:29:14.487422+0000","last_undegraded":"2024-03-29T08:29:14.487422+0000","last_fullsized":"2024-03-29T08:29:14.487422+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:42:53.153883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.489423+0000","last_change":"2024-03-29T08:29:14.486493+0000","last_active":"2024-03-29T08:29:14.489423+0000","last_peered":"2024-03-29T08:29:14.489423+0000","last_clean":"2024-03-29T08:29:14.489423+0000","last_became_active":"2024-03-29T08:29:14.486242+0000","last_became_peered":"2024-03-29T08:29:14.486242+0000","last_unstale":"2024-03-29T08:29:14.489423+0000","last_undegraded":"2024-03-29T08:29:14.489423+0000","last_fullsized":"2024-03-29T08:29:14.489423+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:30:38.933793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507685+0000","last_change":"2024-03-29T08:29:14.092768+0000","last_active":"2024-03-29T08:29:14.507685+0000","last_peered":"2024-03-29T08:29:14.507685+0000","last_clean":"2024-03-29T08:29:14.507685+0000","last_became_active":"2024-03-29T08:29:14.092290+0000","last_became_peered":"2024-03-29T08:29:14.092290+0000","last_unstale":"2024-03-29T08:29:14.507685+0000","last_undegraded":"2024-03-29T08:29:14.507685+0000","last_fullsized":"2024-03-29T08:29:14.507685+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:45.752363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509731+0000","last_change":"2024-03-29T08:29:14.502876+0000","last_active":"2024-03-29T08:29:14.509731+0000","last_peered":"2024-03-29T08:29:14.509731+0000","last_clean":"2024-03-29T08:29:14.509731+0000","last_became_active":"2024-03-29T08:29:14.502600+0000","last_became_peered":"2024-03-29T08:29:14.502600+0000","last_unstale":"2024-03-29T08:29:14.509731+0000","last_undegraded":"2024-03-29T08:29:14.509731+0000","last_fullsized":"2024-03-29T08:29:14.509731+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:49:19.406820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.486956+0000","last_change":"2024-03-29T08:29:14.090942+0000","last_active":"2024-03-29T08:29:14.486956+0000","last_peered":"2024-03-29T08:29:14.486956+0000","last_clean":"2024-03-29T08:29:14.486956+0000","last_became_active":"2024-03-29T08:29:14.090695+0000","last_became_peered":"2024-03-29T08:29:14.090695+0000","last_unstale":"2024-03-29T08:29:14.486956+0000","last_undegraded":"2024-03-29T08:29:14.486956+0000","last_fullsized":"2024-03-29T08:29:14.486956+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:18.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509268+0000","last_change":"2024-03-29T08:29:14.508582+0000","last_active":"2024-03-29T08:29:14.509268+0000","last_peered":"2024-03-29T08:29:14.509268+0000","last_clean":"2024-03-29T08:29:14.509268+0000","last_became_active":"2024-03-29T08:29:14.508201+0000","last_became_peered":"2024-03-29T08:29:14.508201+0000","last_unstale":"2024-03-29T08:29:14.509268+0000","last_undegraded":"2024-03-29T08:29:14.509268+0000","last_fullsized":"2024-03-29T08:29:14.509268+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:59:12.691184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.497159+0000","last_change":"2024-03-29T08:29:14.096474+0000","last_active":"2024-03-29T08:29:14.497159+0000","last_peered":"2024-03-29T08:29:14.497159+0000","last_clean":"2024-03-29T08:29:14.497159+0000","last_became_active":"2024-03-29T08:29:14.094758+0000","last_became_peered":"2024-03-29T08:29:14.094758+0000","last_unstale":"2024-03-29T08:29:14.497159+0000","last_undegraded":"2024-03-29T08:29:14.497159+0000","last_fullsized":"2024-03-29T08:29:14.497159+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:32:23.713785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505838+0000","last_change":"2024-03-29T08:29:14.503676+0000","last_active":"2024-03-29T08:29:14.505838+0000","last_peered":"2024-03-29T08:29:14.505838+0000","last_clean":"2024-03-29T08:29:14.505838+0000","last_became_active":"2024-03-29T08:29:14.503314+0000","last_became_peered":"2024-03-29T08:29:14.503314+0000","last_unstale":"2024-03-29T08:29:14.505838+0000","last_undegraded":"2024-03-29T08:29:14.505838+0000","last_fullsized":"2024-03-29T08:29:14.505838+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:08:59.334138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.496690+0000","last_change":"2024-03-29T08:29:14.089768+0000","last_active":"2024-03-29T08:29:14.496690+0000","last_peered":"2024-03-29T08:29:14.496690+0000","last_clean":"2024-03-29T08:29:14.496690+0000","last_became_active":"2024-03-29T08:29:14.089486+0000","last_became_peered":"2024-03-29T08:29:14.089486+0000","last_unstale":"2024-03-29T08:29:14.496690+0000","last_undegraded":"2024-03-29T08:29:14.496690+0000","last_fullsized":"2024-03-29T08:29:14.496690+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:59.174853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507881+0000","last_change":"2024-03-29T08:29:14.505681+0000","last_active":"2024-03-29T08:29:14.507881+0000","last_peered":"2024-03-29T08:29:14.507881+0000","last_clean":"2024-03-29T08:29:14.507881+0000","last_became_active":"2024-03-29T08:29:14.505394+0000","last_became_peered":"2024-03-29T08:29:14.505394+0000","last_unstale":"2024-03-29T08:29:14.507881+0000","last_undegraded":"2024-03-29T08:29:14.507881+0000","last_fullsized":"2024-03-29T08:29:14.507881+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:32:12.510083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508946+0000","last_change":"2024-03-29T08:29:14.088394+0000","last_active":"2024-03-29T08:29:14.508946+0000","last_peered":"2024-03-29T08:29:14.508946+0000","last_clean":"2024-03-29T08:29:14.508946+0000","last_became_active":"2024-03-29T08:29:14.087525+0000","last_became_peered":"2024-03-29T08:29:14.087525+0000","last_unstale":"2024-03-29T08:29:14.508946+0000","last_undegraded":"2024-03-29T08:29:14.508946+0000","last_fullsized":"2024-03-29T08:29:14.508946+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:54:27.262284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.086933+0000","last_change":"2024-03-29T08:29:14.086933+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.086933+0000","last_undegraded":"2024-03-29T08:29:14.086933+0000","last_fullsized":"2024-03-29T08:29:14.086933+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:14:06.265379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.497231+0000","last_change":"2024-03-29T08:29:14.088202+0000","last_active":"2024-03-29T08:29:14.497231+0000","last_peered":"2024-03-29T08:29:14.497231+0000","last_clean":"2024-03-29T08:29:14.497231+0000","last_became_active":"2024-03-29T08:29:14.087860+0000","last_became_peered":"2024-03-29T08:29:14.087860+0000","last_unstale":"2024-03-29T08:29:14.497231+0000","last_undegraded":"2024-03-29T08:29:14.497231+0000","last_fullsized":"2024-03-29T08:29:14.497231+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:30:51.004244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509624+0000","last_change":"2024-03-29T08:29:14.501270+0000","last_active":"2024-03-29T08:29:14.509624+0000","last_peered":"2024-03-29T08:29:14.509624+0000","last_clean":"2024-03-29T08:29:14.509624+0000","last_became_active":"2024-03-29T08:29:14.498111+0000","last_became_peered":"2024-03-29T08:29:14.498111+0000","last_unstale":"2024-03-29T08:29:14.509624+0000","last_undegraded":"2024-03-29T08:29:14.509624+0000","last_fullsized":"2024-03-29T08:29:14.509624+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:26:17.474099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.495208+0000","last_change":"2024-03-29T08:29:14.090681+0000","last_active":"2024-03-29T08:29:14.495208+0000","last_peered":"2024-03-29T08:29:14.495208+0000","last_clean":"2024-03-29T08:29:14.495208+0000","last_became_active":"2024-03-29T08:29:14.090416+0000","last_became_peered":"2024-03-29T08:29:14.090416+0000","last_unstale":"2024-03-29T08:29:14.495208+0000","last_undegraded":"2024-03-29T08:29:14.495208+0000","last_fullsized":"2024-03-29T08:29:14.495208+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:08:56.584444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507376+0000","last_change":"2024-03-29T08:29:14.505745+0000","last_active":"2024-03-29T08:29:14.507376+0000","last_peered":"2024-03-29T08:29:14.507376+0000","last_clean":"2024-03-29T08:29:14.507376+0000","last_became_active":"2024-03-29T08:29:14.504499+0000","last_became_peered":"2024-03-29T08:29:14.504499+0000","last_unstale":"2024-03-29T08:29:14.507376+0000","last_undegraded":"2024-03-29T08:29:14.507376+0000","last_fullsized":"2024-03-29T08:29:14.507376+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:23:28.769780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.495503+0000","last_change":"2024-03-29T08:29:14.096941+0000","last_active":"2024-03-29T08:29:14.495503+0000","last_peered":"2024-03-29T08:29:14.495503+0000","last_clean":"2024-03-29T08:29:14.495503+0000","last_became_active":"2024-03-29T08:29:14.096439+0000","last_became_peered":"2024-03-29T08:29:14.096439+0000","last_unstale":"2024-03-29T08:29:14.495503+0000","last_undegraded":"2024-03-29T08:29:14.495503+0000","last_fullsized":"2024-03-29T08:29:14.495503+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:49:13.231792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.503705+0000","last_change":"2024-03-29T08:29:14.496299+0000","last_active":"2024-03-29T08:29:14.503705+0000","last_peered":"2024-03-29T08:29:14.503705+0000","last_clean":"2024-03-29T08:29:14.503705+0000","last_became_active":"2024-03-29T08:29:14.496042+0000","last_became_peered":"2024-03-29T08:29:14.496042+0000","last_unstale":"2024-03-29T08:29:14.503705+0000","last_undegraded":"2024-03-29T08:29:14.503705+0000","last_fullsized":"2024-03-29T08:29:14.503705+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:23:28.769780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.497619+0000","last_change":"2024-03-29T08:29:14.081880+0000","last_active":"2024-03-29T08:29:14.497619+0000","last_peered":"2024-03-29T08:29:14.497619+0000","last_clean":"2024-03-29T08:29:14.497619+0000","last_became_active":"2024-03-29T08:29:14.079732+0000","last_became_peered":"2024-03-29T08:29:14.079732+0000","last_unstale":"2024-03-29T08:29:14.497619+0000","last_undegraded":"2024-03-29T08:29:14.497619+0000","last_fullsized":"2024-03-29T08:29:14.497619+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:01.494143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.503628+0000","last_change":"2024-03-29T08:29:14.501776+0000","last_active":"2024-03-29T08:29:14.503628+0000","last_peered":"2024-03-29T08:29:14.503628+0000","last_clean":"2024-03-29T08:29:14.503628+0000","last_became_active":"2024-03-29T08:29:14.501508+0000","last_became_peered":"2024-03-29T08:29:14.501508+0000","last_unstale":"2024-03-29T08:29:14.503628+0000","last_undegraded":"2024-03-29T08:29:14.503628+0000","last_fullsized":"2024-03-29T08:29:14.503628+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:01.476642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.487173+0000","last_change":"2024-03-29T08:29:14.091456+0000","last_active":"2024-03-29T08:29:14.487173+0000","last_peered":"2024-03-29T08:29:14.487173+0000","last_clean":"2024-03-29T08:29:14.487173+0000","last_became_active":"2024-03-29T08:29:14.091121+0000","last_became_peered":"2024-03-29T08:29:14.091121+0000","last_unstale":"2024-03-29T08:29:14.487173+0000","last_undegraded":"2024-03-29T08:29:14.487173+0000","last_fullsized":"2024-03-29T08:29:14.487173+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:30.979257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507721+0000","last_change":"2024-03-29T08:29:14.503755+0000","last_active":"2024-03-29T08:29:14.507721+0000","last_peered":"2024-03-29T08:29:14.507721+0000","last_clean":"2024-03-29T08:29:14.507721+0000","last_became_active":"2024-03-29T08:29:14.503521+0000","last_became_peered":"2024-03-29T08:29:14.503521+0000","last_unstale":"2024-03-29T08:29:14.507721+0000","last_undegraded":"2024-03-29T08:29:14.507721+0000","last_fullsized":"2024-03-29T08:29:14.507721+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:33:36.260388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.486511+0000","last_change":"2024-03-29T08:29:14.089662+0000","last_active":"2024-03-29T08:29:14.486511+0000","last_peered":"2024-03-29T08:29:14.486511+0000","last_clean":"2024-03-29T08:29:14.486511+0000","last_became_active":"2024-03-29T08:29:14.089393+0000","last_became_peered":"2024-03-29T08:29:14.089393+0000","last_unstale":"2024-03-29T08:29:14.486511+0000","last_undegraded":"2024-03-29T08:29:14.486511+0000","last_fullsized":"2024-03-29T08:29:14.486511+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:27:50.778160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.084305+0000","last_change":"2024-03-29T08:29:14.084305+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.084305+0000","last_undegraded":"2024-03-29T08:29:14.084305+0000","last_fullsized":"2024-03-29T08:29:14.084305+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:44:25.936884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.095596+0000","last_change":"2024-03-29T08:29:14.092230+0000","last_active":"2024-03-29T08:29:14.095596+0000","last_peered":"2024-03-29T08:29:14.095596+0000","last_clean":"2024-03-29T08:29:14.095596+0000","last_became_active":"2024-03-29T08:29:14.091979+0000","last_became_peered":"2024-03-29T08:29:14.091979+0000","last_unstale":"2024-03-29T08:29:14.095596+0000","last_undegraded":"2024-03-29T08:29:14.095596+0000","last_fullsized":"2024-03-29T08:29:14.095596+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:41:42.442894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.510494+0000","last_change":"2024-03-29T08:29:14.502476+0000","last_active":"2024-03-29T08:29:14.510494+0000","last_peered":"2024-03-29T08:29:14.510494+0000","last_clean":"2024-03-29T08:29:14.510494+0000","last_became_active":"2024-03-29T08:29:14.502191+0000","last_became_peered":"2024-03-29T08:29:14.502191+0000","last_unstale":"2024-03-29T08:29:14.510494+0000","last_undegraded":"2024-03-29T08:29:14.510494+0000","last_fullsized":"2024-03-29T08:29:14.510494+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:06.826332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.475058+0000","last_change":"2024-03-29T08:29:14.077404+0000","last_active":"2024-03-29T08:29:14.475058+0000","last_peered":"2024-03-29T08:29:14.475058+0000","last_clean":"2024-03-29T08:29:14.475058+0000","last_became_active":"2024-03-29T08:29:14.077141+0000","last_became_peered":"2024-03-29T08:29:14.077141+0000","last_unstale":"2024-03-29T08:29:14.475058+0000","last_undegraded":"2024-03-29T08:29:14.475058+0000","last_fullsized":"2024-03-29T08:29:14.475058+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:51.446071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":250,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.072521+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:29:14.072521+0000","last_peered":"2024-03-29T08:29:14.072521+0000","last_clean":"2024-03-29T08:29:14.072521+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:29:14.072521+0000","last_undegraded":"2024-03-29T08:29:14.072521+0000","last_fullsized":"2024-03-29T08:29:14.072521+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509578,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1503232,"data_stored":646009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.69299999999999995},"min":{"1min":0.48199999999999998,"5min":0.41899999999999998,"15min":0.34999999999999998},"max":{"1min":0.88,"5min":0.97999999999999998,"15min":1.1639999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.75600000000000001,"15min":0.71899999999999997},"min":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.45800000000000002},"max":{"1min":0.95099999999999996,"5min":0.998,"15min":1.2410000000000001},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70199999999999996,"15min":0.67600000000000005},"min":{"1min":0.51500000000000001,"5min":0.45700000000000002,"15min":0.39600000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.202},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70799999999999996,"15min":0.67600000000000005},"min":{"1min":0.56100000000000005,"5min":0.48799999999999999,"15min":0.44},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.1339999999999999},"last":0.505}]},{"osd":2,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69799999999999995,"15min":0.67000000000000004},"min":{"1min":0.502,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.90300000000000002,"5min":0.93600000000000005,"15min":1.085},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72899999999999998,"15min":0.70299999999999996},"min":{"1min":0.55500000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.288},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72299999999999998,"15min":0.70999999999999996},"min":{"1min":0.57599999999999996,"5min":0.51100000000000001,"15min":0.46000000000000002},"max":{"1min":0.97199999999999998,"5min":0.98099999999999998,"15min":1.018},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.60899999999999999,"5min":0.52300000000000002,"15min":0.48499999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.2609999999999999},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58399999999999996,"15min":0.60999999999999999},"min":{"1min":0.48099999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85299999999999998,"5min":0.87,"15min":0.95299999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60599999999999998,"15min":0.66300000000000003},"min":{"1min":0.45100000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.123},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60999999999999999,"15min":0.66200000000000003},"min":{"1min":0.48199999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.877,"5min":0.89600000000000002,"15min":1.056},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.628,"15min":0.66900000000000004},"min":{"1min":0.49199999999999999,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.84399999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.47399999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69199999999999995,"15min":0.71399999999999997},"min":{"1min":0.55900000000000005,"5min":0.46999999999999997,"15min":0.46600000000000003},"max":{"1min":0.89400000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.436},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65800000000000003,"15min":0.70599999999999996},"min":{"1min":0.52800000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.81799999999999995,"5min":0.93799999999999994,"15min":0.98299999999999998},"last":0.64400000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509578,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1482752,"data_stored":644321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72399999999999998,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.38900000000000001},"max":{"1min":0.873,"5min":0.93100000000000005,"15min":1.0189999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.751,"15min":0.71899999999999997},"min":{"1min":0.57199999999999995,"5min":0.51700000000000002,"15min":0.44500000000000001},"max":{"1min":0.97699999999999998,"5min":0.99299999999999999,"15min":1.079},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68200000000000005,"15min":0.64700000000000002},"min":{"1min":0.45400000000000001,"5min":0.45300000000000001,"15min":0.377},"max":{"1min":0.85199999999999998,"5min":0.97999999999999998,"15min":0.99299999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42899999999999999},"max":{"1min":0.78200000000000003,"5min":0.95599999999999996,"15min":1.079},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76900000000000002,"15min":0.72299999999999998},"min":{"1min":0.52100000000000002,"5min":0.51800000000000002,"15min":0.44800000000000001},"max":{"1min":0.91500000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.746,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.46400000000000002,"15min":0.435},"max":{"1min":0.93899999999999995,"5min":0.95099999999999996,"15min":0.95899999999999996},"last":0.753}]},{"osd":3,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.443,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82099999999999995,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.55700000000000005,"5min":0.51000000000000001,"15min":0.47799999999999998},"max":{"1min":0.89800000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.71299999999999997}]},{"osd":4,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.623,"15min":0.64600000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.36499999999999999},"max":{"1min":0.93100000000000005,"5min":1.0840000000000001,"15min":1.556},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63900000000000001,"15min":0.68200000000000005},"min":{"1min":0.46400000000000002,"5min":0.38200000000000001,"15min":0.371},"max":{"1min":0.872,"5min":0.996,"15min":2.0539999999999998},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.627,"15min":0.66100000000000003},"min":{"1min":0.45000000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83599999999999997,"5min":0.99399999999999999,"15min":1.0660000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68100000000000005,"15min":0.70199999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":1.042,"15min":1.1419999999999999},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69599999999999995,"15min":0.72499999999999998},"min":{"1min":0.499,"5min":0.46800000000000003,"15min":0.38500000000000001},"max":{"1min":0.89600000000000002,"5min":1.03,"15min":1.03},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.72999999999999998},"min":{"1min":0.51600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91500000000000004,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.63}]}]},{"osd":4,"up_from":15,"seq":64424509577,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1499136,"data_stored":645975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.45000000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.026,"5min":1.026,"15min":2.1709999999999998},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69699999999999995,"15min":0.71799999999999997},"min":{"1min":0.495,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.997,"5min":0.997,"15min":2.2869999999999999},"last":0.82999999999999996}]},{"osd":1,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.72499999999999998,"15min":0.748},"min":{"1min":0.436,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":2.524},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71099999999999997,"15min":0.70899999999999996},"min":{"1min":0.46400000000000002,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":2.2679999999999998},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69999999999999996,"15min":0.71799999999999997},"min":{"1min":0.45300000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":2.363},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72999999999999998,"15min":0.748},"min":{"1min":0.52500000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.88700000000000001,"5min":0.97799999999999998,"15min":2.1880000000000002},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76200000000000001,"15min":0.753},"min":{"1min":0.60899999999999999,"5min":0.53400000000000003,"15min":0.46300000000000002},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":2.3029999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.77100000000000002,"15min":0.79300000000000004},"min":{"1min":0.55600000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.016,"5min":1.016,"15min":2.2549999999999999},"last":0.81599999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60699999999999998,"15min":0.65800000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.36499999999999999},"max":{"1min":0.76400000000000001,"5min":0.998,"15min":1.0189999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.62,"15min":0.63900000000000001},"min":{"1min":0.45800000000000002,"5min":0.437,"15min":0.41799999999999998},"max":{"1min":0.82699999999999996,"5min":0.85499999999999998,"15min":1.0149999999999999},"last":0.59599999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.63600000000000001,"15min":0.65700000000000003},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.85599999999999998,"5min":1.091,"15min":1.091},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69399999999999995,"15min":0.72099999999999997},"min":{"1min":0.505,"5min":0.499,"15min":0.499},"max":{"1min":0.86399999999999999,"5min":0.91500000000000004,"15min":1.1220000000000001},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.71199999999999997},"min":{"1min":0.51400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91400000000000003,"5min":0.91800000000000004,"15min":1.1000000000000001},"last":0.754},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72599999999999998,"15min":0.747},"min":{"1min":0.434,"5min":0.434,"15min":0.42899999999999999},"max":{"1min":0.96499999999999997,"5min":1.054,"15min":1.4430000000000001},"last":0.69399999999999995}]}]},{"osd":3,"up_from":15,"seq":64424509580,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1482752,"data_stored":644321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.28100000000000003,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":0.71799999999999997,"5min":0.748,"15min":1.123},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69599999999999995,"5min":0.88100000000000001,"15min":1.0409999999999999},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.44,"15min":0.44800000000000001},"min":{"1min":0.32800000000000001,"5min":0.29599999999999999,"15min":0.26500000000000001},"max":{"1min":0.623,"5min":0.76800000000000002,"15min":1.1599999999999999},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.28599999999999998},"max":{"1min":0.82099999999999995,"5min":0.94499999999999995,"15min":1.4790000000000001},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.54400000000000004},"min":{"1min":0.35499999999999998,"5min":0.32900000000000001,"15min":0.309},"max":{"1min":0.76200000000000001,"5min":0.80300000000000005,"15min":1.2529999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.624},"min":{"1min":0.439,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78300000000000003,"5min":0.86599999999999999,"15min":1.8580000000000001},"last":0.59899999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77200000000000002,"15min":0.77500000000000002},"min":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.42199999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.6419999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.77900000000000003,"15min":0.76900000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.45400000000000001},"max":{"1min":1.083,"5min":1.3340000000000001,"15min":1.391},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82199999999999995,"15min":0.80300000000000005},"min":{"1min":0.59599999999999997,"5min":0.57099999999999995,"15min":0.52100000000000002},"max":{"1min":1.081,"5min":1.1850000000000001,"15min":1.671},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.82399999999999995,"15min":0.83499999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0249999999999999,"5min":1.2050000000000001,"15min":1.7110000000000001},"last":0.58699999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.55400000000000005,"5min":0.51700000000000002,"15min":0.48799999999999999},"max":{"1min":1.232,"5min":1.2609999999999999,"15min":1.669},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78100000000000003,"15min":0.79600000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0029999999999999,"5min":1.119,"15min":2.0150000000000001},"last":0.54600000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.82199999999999995},"min":{"1min":0.58899999999999997,"5min":0.50600000000000001,"15min":0.48799999999999999},"max":{"1min":1.0069999999999999,"5min":1.3169999999999999,"15min":1.887},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.81899999999999995},"min":{"1min":0.58999999999999997,"5min":0.49399999999999999,"15min":0.47999999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":2.044},"last":0.57199999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509579,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1470464,"data_stored":637546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45900000000000002,"15min":0.47799999999999998},"min":{"1min":0.312,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.67800000000000005,"5min":0.754,"15min":0.94299999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.378,"5min":0.34699999999999998,"15min":0.32300000000000001},"max":{"1min":0.70699999999999996,"5min":0.874,"15min":2.0880000000000001},"last":0.45200000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.49399999999999999},"min":{"1min":0.34000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70599999999999996,"5min":0.73499999999999999,"15min":1.399},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.501,"15min":0.50600000000000001},"min":{"1min":0.312,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":3.0459999999999998},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79900000000000004,"5min":0.93300000000000005,"15min":2.1280000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.35999999999999999},"max":{"1min":0.79900000000000004,"5min":0.88800000000000001,"15min":2.9950000000000001},"last":0.71399999999999997}]},{"osd":4,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79400000000000004,"15min":0.77700000000000002},"min":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.49099999999999999},"max":{"1min":1.0329999999999999,"5min":1.109,"15min":1.4239999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.76600000000000001},"min":{"1min":0.51900000000000002,"5min":0.51700000000000002,"15min":0.435},"max":{"1min":1.1479999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.92800000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.85599999999999998,"15min":0.83999999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.143,"5min":1.236,"15min":1.4890000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.48799999999999999},"max":{"1min":1.1100000000000001,"5min":1.194,"15min":1.526},"last":0.97799999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79100000000000004,"15min":0.78200000000000003},"min":{"1min":0.57899999999999996,"5min":0.54600000000000004,"15min":0.48599999999999999},"max":{"1min":1.1040000000000001,"5min":1.131,"15min":2.0470000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.82999999999999996,"15min":0.79800000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.52500000000000002},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.524},"last":1.002}]},{"osd":7,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.81399999999999995,"15min":0.80100000000000005},"min":{"1min":0.59099999999999997,"5min":0.54900000000000004,"15min":0.505},"max":{"1min":1.1850000000000001,"5min":1.296,"15min":1.319},"last":0.874},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.81000000000000005,"15min":0.79700000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.214,"5min":1.214,"15min":2},"last":0.90800000000000003}]}]},{"osd":1,"up_from":15,"seq":64424509579,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2076672,"data_stored":1234689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45800000000000002,"15min":0.53200000000000003},"min":{"1min":0.28699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.69699999999999995,"5min":0.75900000000000001,"15min":4.2539999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48299999999999998,"15min":0.56399999999999995},"min":{"1min":0.29399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":4.1909999999999998},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.31900000000000001},"max":{"1min":0.63600000000000001,"5min":0.71099999999999997,"15min":0.82899999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.34599999999999997,"15min":0.33600000000000002},"max":{"1min":0.67000000000000004,"5min":0.77700000000000002,"15min":0.88200000000000001},"last":0.51900000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.35499999999999998},"max":{"1min":0.75700000000000001,"5min":0.88300000000000001,"15min":1.52},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.496,"5min":0.51900000000000002,"15min":0.52400000000000002},"min":{"1min":0.35999999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68999999999999995,"5min":0.94499999999999995,"15min":1.331},"last":0.44}]},{"osd":4,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.75600000000000001},"min":{"1min":0.60199999999999998,"5min":0.51800000000000002,"15min":0.46200000000000002},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.77600000000000002},"min":{"1min":0.56599999999999995,"5min":0.502,"15min":0.47999999999999998},"max":{"1min":1.212,"5min":1.212,"15min":1.23},"last":0.80100000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79600000000000004,"15min":0.77900000000000003},"min":{"1min":0.59699999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.76800000000000002},"min":{"1min":0.55200000000000005,"5min":0.55100000000000005,"15min":0.46500000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.2509999999999999},"last":0.81200000000000006}]},{"osd":6,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.79300000000000004,"15min":0.77300000000000002},"min":{"1min":0.58299999999999996,"5min":0.54900000000000004,"15min":0.5},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.81499999999999995,"15min":0.82399999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51300000000000001},"max":{"1min":1.0209999999999999,"5min":1.22,"15min":1.22},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.52100000000000002},"max":{"1min":1.1499999999999999,"5min":1.2410000000000001,"15min":1.3089999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.80700000000000005,"15min":0.79000000000000004},"min":{"1min":0.622,"5min":0.503,"15min":0.503},"max":{"1min":1.1759999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.89800000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509578,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1490944,"data_stored":644973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.44400000000000001,"15min":0.51900000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.22800000000000001},"max":{"1min":0.62,"5min":0.75800000000000001,"15min":3.9359999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.49299999999999999,"15min":0.66700000000000004},"min":{"1min":0.31900000000000001,"5min":0.28499999999999998,"15min":0.253},"max":{"1min":0.72599999999999998,"5min":0.77000000000000002,"15min":8.0419999999999998},"last":0.34300000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50900000000000001,"15min":0.52700000000000002},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34599999999999997},"max":{"1min":0.68500000000000005,"5min":0.78000000000000003,"15min":1.258},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54500000000000004,"15min":0.63300000000000001},"min":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.312},"max":{"1min":0.77400000000000002,"5min":0.81399999999999995,"15min":4.907},"last":0.44}]},{"osd":3,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.501,"15min":0.66900000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.28499999999999998},"max":{"1min":0.65100000000000002,"5min":0.83399999999999996,"15min":7.9569999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.61399999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.34799999999999998},"max":{"1min":0.80100000000000005,"5min":0.87,"15min":1.397},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79500000000000004,"15min":0.88500000000000001},"min":{"1min":0.56799999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1020000000000001,"5min":1.4399999999999999,"15min":4.4720000000000004},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.79400000000000004,"15min":0.86799999999999999},"min":{"1min":0.51500000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.141,"5min":1.153,"15min":4.3920000000000003},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.85499999999999998},"min":{"1min":0.56699999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.014,"5min":1.349,"15min":4.3200000000000003},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80300000000000005,"15min":0.82499999999999996},"min":{"1min":0.59599999999999997,"5min":0.56000000000000005,"15min":0.54600000000000004},"max":{"1min":1.081,"5min":1.26,"15min":1.6759999999999999},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.79800000000000004,"15min":0.82699999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.54400000000000004},"max":{"1min":1.0640000000000001,"5min":1.1579999999999999,"15min":1.5920000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81599999999999995,"15min":0.89300000000000002},"min":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.165,"5min":1.1990000000000001,"15min":4.4100000000000001},"last":0.85499999999999998}]},{"osd":7,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82399999999999995,"15min":0.90600000000000003},"min":{"1min":0.61499999999999999,"5min":0.54100000000000004,"15min":0.496},"max":{"1min":1.141,"5min":1.3959999999999999,"15min":4.5209999999999999},"last":0.752},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80500000000000005,"15min":0.88900000000000001},"min":{"1min":0.56499999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.089,"5min":1.2390000000000001,"15min":4.4420000000000002},"last":0.69999999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542283,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":2064384,"data_stored":1227914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70899999999999996,"15min":0.66400000000000003},"min":{"1min":0.47499999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.98499999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72999999999999998,"15min":0.68400000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.39800000000000002},"max":{"1min":0.84799999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64100000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70699999999999996,"15min":0.71199999999999997},"min":{"1min":0.48499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.008,"5min":1.008,"15min":2.5129999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74299999999999999,"15min":0.70299999999999996},"min":{"1min":0.56799999999999995,"5min":0.51300000000000001,"15min":0.40500000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.85599999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71799999999999997,"15min":0.68600000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92800000000000005,"5min":0.997,"15min":0.997},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.747,"15min":0.71799999999999997},"min":{"1min":0.60899999999999999,"5min":0.47499999999999998,"15min":0.39200000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.77700000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.755,"15min":0.72599999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.47099999999999997},"max":{"1min":1.02,"5min":1.038,"15min":1.038},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.752,"15min":0.74099999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.435},"max":{"1min":0.995,"5min":1.0209999999999999,"15min":1.954},"last":0.81200000000000006}]},{"osd":4,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.624,"15min":0.64000000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.91200000000000003,"15min":0.93500000000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.373},"max":{"1min":0.83299999999999996,"5min":0.94699999999999995,"15min":1.0820000000000001},"last":0.82699999999999996}]},{"osd":6,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.621,"15min":0.65000000000000002},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85999999999999999,"5min":0.93300000000000005,"15min":1.071},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.68500000000000005},"min":{"1min":0.53200000000000003,"5min":0.45300000000000001,"15min":0.432},"max":{"1min":0.92000000000000004,"5min":0.94299999999999995,"15min":1.0609999999999999},"last":0.89800000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90100000000000002,"5min":0.92200000000000004,"15min":1.1200000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70099999999999996,"15min":0.72899999999999998},"min":{"1min":0.46800000000000003,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.94999999999999996,"5min":0.96499999999999997,"15min":1.04},"last":0.98399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:29:18.618 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:29:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.753+0000 7f9a6ee00640 1 Processor -- start 2024-03-29T08:29:18.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.753+0000 7f9a6ee00640 1 -- start start 2024-03-29T08:29:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.753+0000 7f9a6ee00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9a6815feb0 0x7f9a68071370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.753+0000 7f9a6ee00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9a681605f0 0x7f9a68069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.753+0000 7f9a6ee00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a68161020 con 0x7f9a68160a50 2024-03-29T08:29:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.753+0000 7f9a6ee00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a68161300 con 0x7f9a681605f0 2024-03-29T08:29:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.753+0000 7f9a6ee00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a68070b70 con 0x7f9a6815feb0 2024-03-29T08:29:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.753+0000 7f9a6ce00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9a68160a50 0x7f9a6806d8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48198/0 (socket says 172.21.15.46:48198) 2024-03-29T08:29:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.753+0000 7f9a6ce00640 1 -- 172.21.15.46:0/2823785208 learned_addr learned my addr 172.21.15.46:0/2823785208 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.757+0000 7f9a5fe00640 1 --2- 172.21.15.46:0/2823785208 >> v2:172.21.15.73:3300/0 conn(0x7f9a681605f0 0x7f9a68069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.757+0000 7f9a67e00640 1 --2- 172.21.15.46:0/2823785208 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9a6815feb0 0x7f9a68071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.757+0000 7f9a67400640 1 -- 172.21.15.46:0/2823785208 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2834831470 0 0) 0x7f9a68161020 con 0x7f9a68160a50 2024-03-29T08:29:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.757+0000 7f9a67400640 1 -- 172.21.15.46:0/2823785208 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a4c003530 con 0x7f9a68160a50 2024-03-29T08:29:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.757+0000 7f9a67e00640 1 -- 172.21.15.46:0/2823785208 >> v2:172.21.15.73:3300/0 conn(0x7f9a681605f0 msgr2=0x7f9a68069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.757+0000 7f9a67e00640 1 --2- 172.21.15.46:0/2823785208 >> v2:172.21.15.73:3300/0 conn(0x7f9a681605f0 0x7f9a68069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.757+0000 7f9a67e00640 1 -- 172.21.15.46:0/2823785208 >> v1:172.21.15.46:6789/0 conn(0x7f9a68160a50 legacy=0x7f9a6806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:18.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.757+0000 7f9a67e00640 1 -- 172.21.15.46:0/2823785208 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a68070e50 con 0x7f9a6815feb0 2024-03-29T08:29:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.757+0000 7f9a67e00640 1 --2- 172.21.15.46:0/2823785208 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9a6815feb0 0x7f9a68071370 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9a580097c0 tx=0x7f9a5801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe1b127fce4eafc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.757+0000 7f9a67400640 1 -- 172.21.15.46:0/2823785208 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a580a7070 con 0x7f9a6815feb0 2024-03-29T08:29:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.757+0000 7f9a67400640 1 -- 172.21.15.46:0/2823785208 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a580ab430 con 0x7f9a6815feb0 2024-03-29T08:29:18.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.757+0000 7f9a67400640 1 -- 172.21.15.46:0/2823785208 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a580ab7c0 con 0x7f9a6815feb0 2024-03-29T08:29:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ee00640 1 -- 172.21.15.46:0/2823785208 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9a6815feb0 msgr2=0x7f9a68071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ee00640 1 --2- 172.21.15.46:0/2823785208 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9a6815feb0 0x7f9a68071370 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9a580097c0 tx=0x7f9a5801c410 comp rx=0 tx=0).stop 2024-03-29T08:29:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ee00640 1 -- 172.21.15.46:0/2823785208 shutdown_connections 2024-03-29T08:29:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ee00640 1 --2- 172.21.15.46:0/2823785208 >> v2:172.21.15.73:3300/0 conn(0x7f9a681605f0 0x7f9a68069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ee00640 1 --2- 172.21.15.46:0/2823785208 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9a6815feb0 0x7f9a68071370 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ee00640 1 -- 172.21.15.46:0/2823785208 >> 172.21.15.46:0/2823785208 conn(0x7f9a680ced50 msgr2=0x7f9a680cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ee00640 1 -- 172.21.15.46:0/2823785208 shutdown_connections 2024-03-29T08:29:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ee00640 1 -- 172.21.15.46:0/2823785208 wait complete. 2024-03-29T08:29:18.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ee00640 1 Processor -- start 2024-03-29T08:29:18.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ee00640 1 -- start start 2024-03-29T08:29:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a6815feb0 0x7f9a6815ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ee00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9a681605f0 0x7f9a680af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a68160a50 0x7f9a680af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6815e310 con 0x7f9a68160a50 2024-03-29T08:29:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6815e7a0 con 0x7f9a6815feb0 2024-03-29T08:29:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ee00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6815d460 con 0x7f9a681605f0 2024-03-29T08:29:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a68160a50 0x7f9a680af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a67e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a6815feb0 0x7f9a6815ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ce00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a68160a50 0x7f9a680af810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39044/0 (socket says 172.21.15.46:39044) 2024-03-29T08:29:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ce00640 1 -- 172.21.15.46:0/2838832533 learned_addr learned my addr 172.21.15.46:0/2838832533 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a5fe00640 1 --2- 172.21.15.46:0/2838832533 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9a681605f0 0x7f9a680af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ce00640 1 -- 172.21.15.46:0/2838832533 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9a681605f0 msgr2=0x7f9a680af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ce00640 1 --2- 172.21.15.46:0/2838832533 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9a681605f0 0x7f9a680af2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ce00640 1 -- 172.21.15.46:0/2838832533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a6815feb0 msgr2=0x7f9a6815ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ce00640 1 --2- 172.21.15.46:0/2838832533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a6815feb0 0x7f9a6815ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.761+0000 7f9a6ce00640 1 -- 172.21.15.46:0/2838832533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a58009420 con 0x7f9a68160a50 2024-03-29T08:29:18.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.765+0000 7f9a6ce00640 1 --2- 172.21.15.46:0/2838832533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a68160a50 0x7f9a680af810 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000caa0 tx=0x7f9a6000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.765+0000 7f9a64c00640 1 -- 172.21.15.46:0/2838832533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6000ac00 con 0x7f9a68160a50 2024-03-29T08:29:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.765+0000 7f9a64c00640 1 -- 172.21.15.46:0/2838832533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6000ad60 con 0x7f9a68160a50 2024-03-29T08:29:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.765+0000 7f9a64c00640 1 -- 172.21.15.46:0/2838832533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60012620 con 0x7f9a68160a50 2024-03-29T08:29:18.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.765+0000 7f9a6ee00640 1 -- 172.21.15.46:0/2838832533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6815bfd0 con 0x7f9a68160a50 2024-03-29T08:29:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.765+0000 7f9a6ee00640 1 -- 172.21.15.46:0/2838832533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6815d9e0 con 0x7f9a68160a50 2024-03-29T08:29:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.765+0000 7f9a6ee00640 1 -- 172.21.15.46:0/2838832533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6815e480 con 0x7f9a68160a50 2024-03-29T08:29:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.765+0000 7f9a64c00640 1 -- 172.21.15.46:0/2838832533 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9a60007070 con 0x7f9a68160a50 2024-03-29T08:29:18.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.765+0000 7f9a64c00640 1 --2- 172.21.15.46:0/2838832533 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9a440666c0 0x7f9a44068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.765+0000 7f9a64c00640 1 -- 172.21.15.46:0/2838832533 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a60084180 con 0x7f9a68160a50 2024-03-29T08:29:18.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.765+0000 7f9a67e00640 1 --2- 172.21.15.46:0/2838832533 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9a440666c0 0x7f9a44068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:18.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.769+0000 7f9a67e00640 1 --2- 172.21.15.46:0/2838832533 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9a440666c0 0x7f9a44068b80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9a580af000 tx=0x7f9a58008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:18.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.769+0000 7f9a64c00640 1 -- 172.21.15.46:0/2838832533 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9a60051470 con 0x7f9a68160a50 2024-03-29T08:29:18.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.925+0000 7f9a6ee00640 1 -- 172.21.15.46:0/2838832533 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a6806a6e0 con 0x7f9a440666c0 2024-03-29T08:29:18.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.933+0000 7f9a64c00640 1 -- 172.21.15.46:0/2838832533 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289105 (secure 0 0 0) 0x7f9a58006e20 con 0x7f9a440666c0 2024-03-29T08:29:18.942 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:18.944 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:29:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.937+0000 7f9a6ee00640 1 -- 172.21.15.46:0/2838832533 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9a440666c0 msgr2=0x7f9a44068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.937+0000 7f9a6ee00640 1 --2- 172.21.15.46:0/2838832533 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9a440666c0 0x7f9a44068b80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9a580af000 tx=0x7f9a58008040 comp rx=0 tx=0).stop 2024-03-29T08:29:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.937+0000 7f9a6ee00640 1 -- 172.21.15.46:0/2838832533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a68160a50 msgr2=0x7f9a680af810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.937+0000 7f9a6ee00640 1 --2- 172.21.15.46:0/2838832533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a68160a50 0x7f9a680af810 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000caa0 tx=0x7f9a6000cf60 comp rx=0 tx=0).stop 2024-03-29T08:29:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.937+0000 7f9a6ee00640 1 -- 172.21.15.46:0/2838832533 shutdown_connections 2024-03-29T08:29:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.937+0000 7f9a6ee00640 1 --2- 172.21.15.46:0/2838832533 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9a440666c0 0x7f9a44068b80 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.937+0000 7f9a6ee00640 1 --2- 172.21.15.46:0/2838832533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9a68160a50 0x7f9a680af810 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.937+0000 7f9a6ee00640 1 --2- 172.21.15.46:0/2838832533 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9a681605f0 0x7f9a680af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.937+0000 7f9a6ee00640 1 --2- 172.21.15.46:0/2838832533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a6815feb0 0x7f9a6815ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:18.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.937+0000 7f9a6ee00640 1 -- 172.21.15.46:0/2838832533 >> 172.21.15.46:0/2838832533 conn(0x7f9a680ced50 msgr2=0x7f9a680b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:18.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.937+0000 7f9a6ee00640 1 -- 172.21.15.46:0/2838832533 shutdown_connections 2024-03-29T08:29:18.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:18.937+0000 7f9a6ee00640 1 -- 172.21.15.46:0/2838832533 wait complete. 2024-03-29T08:29:18.963 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-03-29T08:29:18.211688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261740,"kb_used_data":12764,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754712980,"statfs":{"total":773094113280,"available":772826091520,"internally_reserved":0,"allocated":13070336,"data_stored":6325748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.148834"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509634+0000","last_change":"2024-03-29T08:29:14.504298+0000","last_active":"2024-03-29T08:29:14.509634+0000","last_peered":"2024-03-29T08:29:14.509634+0000","last_clean":"2024-03-29T08:29:14.509634+0000","last_became_active":"2024-03-29T08:29:14.504060+0000","last_became_peered":"2024-03-29T08:29:14.504060+0000","last_unstale":"2024-03-29T08:29:14.509634+0000","last_undegraded":"2024-03-29T08:29:14.509634+0000","last_fullsized":"2024-03-29T08:29:14.509634+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:47:24.542958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.491055+0000","last_change":"2024-03-29T08:29:14.488152+0000","last_active":"2024-03-29T08:29:14.491055+0000","last_peered":"2024-03-29T08:29:14.491055+0000","last_clean":"2024-03-29T08:29:14.491055+0000","last_became_active":"2024-03-29T08:29:14.487884+0000","last_became_peered":"2024-03-29T08:29:14.487884+0000","last_unstale":"2024-03-29T08:29:14.491055+0000","last_undegraded":"2024-03-29T08:29:14.491055+0000","last_fullsized":"2024-03-29T08:29:14.491055+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:49:14.237508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.492505+0000","last_change":"2024-03-29T08:29:14.490304+0000","last_active":"2024-03-29T08:29:14.492505+0000","last_peered":"2024-03-29T08:29:14.492505+0000","last_clean":"2024-03-29T08:29:14.492505+0000","last_became_active":"2024-03-29T08:29:14.489916+0000","last_became_peered":"2024-03-29T08:29:14.489916+0000","last_unstale":"2024-03-29T08:29:14.492505+0000","last_undegraded":"2024-03-29T08:29:14.492505+0000","last_fullsized":"2024-03-29T08:29:14.492505+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:55.868596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.510337+0000","last_change":"2024-03-29T08:29:14.501687+0000","last_active":"2024-03-29T08:29:14.510337+0000","last_peered":"2024-03-29T08:29:14.510337+0000","last_clean":"2024-03-29T08:29:14.510337+0000","last_became_active":"2024-03-29T08:29:14.501377+0000","last_became_peered":"2024-03-29T08:29:14.501377+0000","last_unstale":"2024-03-29T08:29:14.510337+0000","last_undegraded":"2024-03-29T08:29:14.510337+0000","last_fullsized":"2024-03-29T08:29:14.510337+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:38:29.985399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.506541+0000","last_change":"2024-03-29T08:29:14.504615+0000","last_active":"2024-03-29T08:29:14.506541+0000","last_peered":"2024-03-29T08:29:14.506541+0000","last_clean":"2024-03-29T08:29:14.506541+0000","last_became_active":"2024-03-29T08:29:14.504347+0000","last_became_peered":"2024-03-29T08:29:14.504347+0000","last_unstale":"2024-03-29T08:29:14.506541+0000","last_undegraded":"2024-03-29T08:29:14.506541+0000","last_fullsized":"2024-03-29T08:29:14.506541+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:25:45.763218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.084660+0000","last_change":"2024-03-29T08:29:14.084660+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.084660+0000","last_undegraded":"2024-03-29T08:29:14.084660+0000","last_fullsized":"2024-03-29T08:29:14.084660+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:51.101298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.503717+0000","last_change":"2024-03-29T08:29:14.494643+0000","last_active":"2024-03-29T08:29:14.503717+0000","last_peered":"2024-03-29T08:29:14.503717+0000","last_clean":"2024-03-29T08:29:14.503717+0000","last_became_active":"2024-03-29T08:29:14.494385+0000","last_became_peered":"2024-03-29T08:29:14.494385+0000","last_unstale":"2024-03-29T08:29:14.503717+0000","last_undegraded":"2024-03-29T08:29:14.503717+0000","last_fullsized":"2024-03-29T08:29:14.503717+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:11:26.639183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.495613+0000","last_change":"2024-03-29T08:29:14.489843+0000","last_active":"2024-03-29T08:29:14.495613+0000","last_peered":"2024-03-29T08:29:14.495613+0000","last_clean":"2024-03-29T08:29:14.495613+0000","last_became_active":"2024-03-29T08:29:14.489598+0000","last_became_peered":"2024-03-29T08:29:14.489598+0000","last_unstale":"2024-03-29T08:29:14.495613+0000","last_undegraded":"2024-03-29T08:29:14.495613+0000","last_fullsized":"2024-03-29T08:29:14.495613+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:49:19.274668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.512231+0000","last_change":"2024-03-29T08:29:14.507881+0000","last_active":"2024-03-29T08:29:14.512231+0000","last_peered":"2024-03-29T08:29:14.512231+0000","last_clean":"2024-03-29T08:29:14.512231+0000","last_became_active":"2024-03-29T08:29:14.507445+0000","last_became_peered":"2024-03-29T08:29:14.507445+0000","last_unstale":"2024-03-29T08:29:14.512231+0000","last_undegraded":"2024-03-29T08:29:14.512231+0000","last_fullsized":"2024-03-29T08:29:14.512231+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:50:39.114131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504217+0000","last_change":"2024-03-29T08:29:14.500130+0000","last_active":"2024-03-29T08:29:14.504217+0000","last_peered":"2024-03-29T08:29:14.504217+0000","last_clean":"2024-03-29T08:29:14.504217+0000","last_became_active":"2024-03-29T08:29:14.499827+0000","last_became_peered":"2024-03-29T08:29:14.499827+0000","last_unstale":"2024-03-29T08:29:14.504217+0000","last_undegraded":"2024-03-29T08:29:14.504217+0000","last_fullsized":"2024-03-29T08:29:14.504217+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:01.476642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.502452+0000","last_change":"2024-03-29T08:29:14.494953+0000","last_active":"2024-03-29T08:29:14.502452+0000","last_peered":"2024-03-29T08:29:14.502452+0000","last_clean":"2024-03-29T08:29:14.502452+0000","last_became_active":"2024-03-29T08:29:14.494599+0000","last_became_peered":"2024-03-29T08:29:14.494599+0000","last_unstale":"2024-03-29T08:29:14.502452+0000","last_undegraded":"2024-03-29T08:29:14.502452+0000","last_fullsized":"2024-03-29T08:29:14.502452+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:49:02.417397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504246+0000","last_change":"2024-03-29T08:29:14.499160+0000","last_active":"2024-03-29T08:29:14.504246+0000","last_peered":"2024-03-29T08:29:14.504246+0000","last_clean":"2024-03-29T08:29:14.504246+0000","last_became_active":"2024-03-29T08:29:14.498900+0000","last_became_peered":"2024-03-29T08:29:14.498900+0000","last_unstale":"2024-03-29T08:29:14.504246+0000","last_undegraded":"2024-03-29T08:29:14.504246+0000","last_fullsized":"2024-03-29T08:29:14.504246+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:19.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.493310+0000","last_change":"2024-03-29T08:29:14.486530+0000","last_active":"2024-03-29T08:29:14.493310+0000","last_peered":"2024-03-29T08:29:14.493310+0000","last_clean":"2024-03-29T08:29:14.493310+0000","last_became_active":"2024-03-29T08:29:14.486158+0000","last_became_peered":"2024-03-29T08:29:14.486158+0000","last_unstale":"2024-03-29T08:29:14.493310+0000","last_undegraded":"2024-03-29T08:29:14.493310+0000","last_fullsized":"2024-03-29T08:29:14.493310+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:40.282997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508861+0000","last_change":"2024-03-29T08:29:14.507343+0000","last_active":"2024-03-29T08:29:14.508861+0000","last_peered":"2024-03-29T08:29:14.508861+0000","last_clean":"2024-03-29T08:29:14.508861+0000","last_became_active":"2024-03-29T08:29:14.507072+0000","last_became_peered":"2024-03-29T08:29:14.507072+0000","last_unstale":"2024-03-29T08:29:14.508861+0000","last_undegraded":"2024-03-29T08:29:14.508861+0000","last_fullsized":"2024-03-29T08:29:14.508861+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:19.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509523+0000","last_change":"2024-03-29T08:29:14.498131+0000","last_active":"2024-03-29T08:29:14.509523+0000","last_peered":"2024-03-29T08:29:14.509523+0000","last_clean":"2024-03-29T08:29:14.509523+0000","last_became_active":"2024-03-29T08:29:14.497856+0000","last_became_peered":"2024-03-29T08:29:14.497856+0000","last_unstale":"2024-03-29T08:29:14.509523+0000","last_undegraded":"2024-03-29T08:29:14.509523+0000","last_fullsized":"2024-03-29T08:29:14.509523+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:43:24.083895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504685+0000","last_change":"2024-03-29T08:29:14.499902+0000","last_active":"2024-03-29T08:29:14.504685+0000","last_peered":"2024-03-29T08:29:14.504685+0000","last_clean":"2024-03-29T08:29:14.504685+0000","last_became_active":"2024-03-29T08:29:14.499642+0000","last_became_peered":"2024-03-29T08:29:14.499642+0000","last_unstale":"2024-03-29T08:29:14.504685+0000","last_undegraded":"2024-03-29T08:29:14.504685+0000","last_fullsized":"2024-03-29T08:29:14.504685+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:06:54.374100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.502730+0000","last_change":"2024-03-29T08:29:14.496277+0000","last_active":"2024-03-29T08:29:14.502730+0000","last_peered":"2024-03-29T08:29:14.502730+0000","last_clean":"2024-03-29T08:29:14.502730+0000","last_became_active":"2024-03-29T08:29:14.496028+0000","last_became_peered":"2024-03-29T08:29:14.496028+0000","last_unstale":"2024-03-29T08:29:14.502730+0000","last_undegraded":"2024-03-29T08:29:14.502730+0000","last_fullsized":"2024-03-29T08:29:14.502730+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:08:59.334138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505390+0000","last_change":"2024-03-29T08:29:14.503103+0000","last_active":"2024-03-29T08:29:14.505390+0000","last_peered":"2024-03-29T08:29:14.505390+0000","last_clean":"2024-03-29T08:29:14.505390+0000","last_became_active":"2024-03-29T08:29:14.502841+0000","last_became_peered":"2024-03-29T08:29:14.502841+0000","last_unstale":"2024-03-29T08:29:14.505390+0000","last_undegraded":"2024-03-29T08:29:14.505390+0000","last_fullsized":"2024-03-29T08:29:14.505390+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:11:21.620572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.498334+0000","last_change":"2024-03-29T08:29:14.491176+0000","last_active":"2024-03-29T08:29:14.498334+0000","last_peered":"2024-03-29T08:29:14.498334+0000","last_clean":"2024-03-29T08:29:14.498334+0000","last_became_active":"2024-03-29T08:29:14.490845+0000","last_became_peered":"2024-03-29T08:29:14.490845+0000","last_unstale":"2024-03-29T08:29:14.498334+0000","last_undegraded":"2024-03-29T08:29:14.498334+0000","last_fullsized":"2024-03-29T08:29:14.498334+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:44:41.312640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504227+0000","last_change":"2024-03-29T08:29:14.501383+0000","last_active":"2024-03-29T08:29:14.504227+0000","last_peered":"2024-03-29T08:29:14.504227+0000","last_clean":"2024-03-29T08:29:14.504227+0000","last_became_active":"2024-03-29T08:29:14.501106+0000","last_became_peered":"2024-03-29T08:29:14.501106+0000","last_unstale":"2024-03-29T08:29:14.504227+0000","last_undegraded":"2024-03-29T08:29:14.504227+0000","last_fullsized":"2024-03-29T08:29:14.504227+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:23:10.605671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504862+0000","last_change":"2024-03-29T08:29:14.497743+0000","last_active":"2024-03-29T08:29:14.504862+0000","last_peered":"2024-03-29T08:29:14.504862+0000","last_clean":"2024-03-29T08:29:14.504862+0000","last_became_active":"2024-03-29T08:29:14.497338+0000","last_became_peered":"2024-03-29T08:29:14.497338+0000","last_unstale":"2024-03-29T08:29:14.504862+0000","last_undegraded":"2024-03-29T08:29:14.504862+0000","last_fullsized":"2024-03-29T08:29:14.504862+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:24.513091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.084054+0000","last_change":"2024-03-29T08:29:14.084054+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.084054+0000","last_undegraded":"2024-03-29T08:29:14.084054+0000","last_fullsized":"2024-03-29T08:29:14.084054+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:57:23.884319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.506608+0000","last_change":"2024-03-29T08:29:14.502935+0000","last_active":"2024-03-29T08:29:14.506608+0000","last_peered":"2024-03-29T08:29:14.506608+0000","last_clean":"2024-03-29T08:29:14.506608+0000","last_became_active":"2024-03-29T08:29:14.502701+0000","last_became_peered":"2024-03-29T08:29:14.502701+0000","last_unstale":"2024-03-29T08:29:14.506608+0000","last_undegraded":"2024-03-29T08:29:14.506608+0000","last_fullsized":"2024-03-29T08:29:14.506608+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:35.756569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.511647+0000","last_change":"2024-03-29T08:29:14.507544+0000","last_active":"2024-03-29T08:29:14.511647+0000","last_peered":"2024-03-29T08:29:14.511647+0000","last_clean":"2024-03-29T08:29:14.511647+0000","last_became_active":"2024-03-29T08:29:14.507285+0000","last_became_peered":"2024-03-29T08:29:14.507285+0000","last_unstale":"2024-03-29T08:29:14.511647+0000","last_undegraded":"2024-03-29T08:29:14.511647+0000","last_fullsized":"2024-03-29T08:29:14.511647+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:50:14.687728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.494822+0000","last_change":"2024-03-29T08:29:14.492853+0000","last_active":"2024-03-29T08:29:14.494822+0000","last_peered":"2024-03-29T08:29:14.494822+0000","last_clean":"2024-03-29T08:29:14.494822+0000","last_became_active":"2024-03-29T08:29:14.492537+0000","last_became_peered":"2024-03-29T08:29:14.492537+0000","last_unstale":"2024-03-29T08:29:14.494822+0000","last_undegraded":"2024-03-29T08:29:14.494822+0000","last_fullsized":"2024-03-29T08:29:14.494822+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:08:28.043449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.085680+0000","last_change":"2024-03-29T08:29:14.085680+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.085680+0000","last_undegraded":"2024-03-29T08:29:14.085680+0000","last_fullsized":"2024-03-29T08:29:14.085680+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:10:44.097211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.511774+0000","last_change":"2024-03-29T08:29:14.508869+0000","last_active":"2024-03-29T08:29:14.511774+0000","last_peered":"2024-03-29T08:29:14.511774+0000","last_clean":"2024-03-29T08:29:14.511774+0000","last_became_active":"2024-03-29T08:29:14.508466+0000","last_became_peered":"2024-03-29T08:29:14.508466+0000","last_unstale":"2024-03-29T08:29:14.511774+0000","last_undegraded":"2024-03-29T08:29:14.511774+0000","last_fullsized":"2024-03-29T08:29:14.511774+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:08:28.619413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.087721+0000","last_change":"2024-03-29T08:29:14.087721+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.087721+0000","last_undegraded":"2024-03-29T08:29:14.087721+0000","last_fullsized":"2024-03-29T08:29:14.087721+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:53.749284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508906+0000","last_change":"2024-03-29T08:29:14.507369+0000","last_active":"2024-03-29T08:29:14.508906+0000","last_peered":"2024-03-29T08:29:14.508906+0000","last_clean":"2024-03-29T08:29:14.508906+0000","last_became_active":"2024-03-29T08:29:14.507164+0000","last_became_peered":"2024-03-29T08:29:14.507164+0000","last_unstale":"2024-03-29T08:29:14.508906+0000","last_undegraded":"2024-03-29T08:29:14.508906+0000","last_fullsized":"2024-03-29T08:29:14.508906+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:20:13.956872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505661+0000","last_change":"2024-03-29T08:29:14.503526+0000","last_active":"2024-03-29T08:29:14.505661+0000","last_peered":"2024-03-29T08:29:14.505661+0000","last_clean":"2024-03-29T08:29:14.505661+0000","last_became_active":"2024-03-29T08:29:14.503149+0000","last_became_peered":"2024-03-29T08:29:14.503149+0000","last_unstale":"2024-03-29T08:29:14.505661+0000","last_undegraded":"2024-03-29T08:29:14.505661+0000","last_fullsized":"2024-03-29T08:29:14.505661+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:46:13.948298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509541+0000","last_change":"2024-03-29T08:29:14.500403+0000","last_active":"2024-03-29T08:29:14.509541+0000","last_peered":"2024-03-29T08:29:14.509541+0000","last_clean":"2024-03-29T08:29:14.509541+0000","last_became_active":"2024-03-29T08:29:14.500132+0000","last_became_peered":"2024-03-29T08:29:14.500132+0000","last_unstale":"2024-03-29T08:29:14.509541+0000","last_undegraded":"2024-03-29T08:29:14.509541+0000","last_fullsized":"2024-03-29T08:29:14.509541+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:09:34.129852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504668+0000","last_change":"2024-03-29T08:29:14.499479+0000","last_active":"2024-03-29T08:29:14.504668+0000","last_peered":"2024-03-29T08:29:14.504668+0000","last_clean":"2024-03-29T08:29:14.504668+0000","last_became_active":"2024-03-29T08:29:14.498994+0000","last_became_peered":"2024-03-29T08:29:14.498994+0000","last_unstale":"2024-03-29T08:29:14.504668+0000","last_undegraded":"2024-03-29T08:29:14.504668+0000","last_fullsized":"2024-03-29T08:29:14.504668+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:25:13.598107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.513004+0000","last_change":"2024-03-29T08:29:14.511693+0000","last_active":"2024-03-29T08:29:14.513004+0000","last_peered":"2024-03-29T08:29:14.513004+0000","last_clean":"2024-03-29T08:29:14.513004+0000","last_became_active":"2024-03-29T08:29:14.511041+0000","last_became_peered":"2024-03-29T08:29:14.511041+0000","last_unstale":"2024-03-29T08:29:14.513004+0000","last_undegraded":"2024-03-29T08:29:14.513004+0000","last_fullsized":"2024-03-29T08:29:14.513004+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:06.874362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"120'4","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507837+0000","last_change":"2024-03-29T08:29:14.086060+0000","last_active":"2024-03-29T08:29:14.507837+0000","last_peered":"2024-03-29T08:29:14.507837+0000","last_clean":"2024-03-29T08:29:14.507837+0000","last_became_active":"2024-03-29T08:29:14.085788+0000","last_became_peered":"2024-03-29T08:29:14.085788+0000","last_unstale":"2024-03-29T08:29:14.507837+0000","last_undegraded":"2024-03-29T08:29:14.507837+0000","last_fullsized":"2024-03-29T08:29:14.507837+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:52:04.161387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507246+0000","last_change":"2024-03-29T08:29:14.503865+0000","last_active":"2024-03-29T08:29:14.507246+0000","last_peered":"2024-03-29T08:29:14.507246+0000","last_clean":"2024-03-29T08:29:14.507246+0000","last_became_active":"2024-03-29T08:29:14.503622+0000","last_became_peered":"2024-03-29T08:29:14.503622+0000","last_unstale":"2024-03-29T08:29:14.507246+0000","last_undegraded":"2024-03-29T08:29:14.507246+0000","last_fullsized":"2024-03-29T08:29:14.507246+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:48:23.465613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.501135+0000","last_change":"2024-03-29T08:29:14.084881+0000","last_active":"2024-03-29T08:29:14.501135+0000","last_peered":"2024-03-29T08:29:14.501135+0000","last_clean":"2024-03-29T08:29:14.501135+0000","last_became_active":"2024-03-29T08:29:14.084587+0000","last_became_peered":"2024-03-29T08:29:14.084587+0000","last_unstale":"2024-03-29T08:29:14.501135+0000","last_undegraded":"2024-03-29T08:29:14.501135+0000","last_fullsized":"2024-03-29T08:29:14.501135+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:45.558553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.085712+0000","last_change":"2024-03-29T08:29:14.085712+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.085712+0000","last_undegraded":"2024-03-29T08:29:14.085712+0000","last_fullsized":"2024-03-29T08:29:14.085712+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:23:32.467222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509018+0000","last_change":"2024-03-29T08:29:14.089964+0000","last_active":"2024-03-29T08:29:14.509018+0000","last_peered":"2024-03-29T08:29:14.509018+0000","last_clean":"2024-03-29T08:29:14.509018+0000","last_became_active":"2024-03-29T08:29:14.088099+0000","last_became_peered":"2024-03-29T08:29:14.088099+0000","last_unstale":"2024-03-29T08:29:14.509018+0000","last_undegraded":"2024-03-29T08:29:14.509018+0000","last_fullsized":"2024-03-29T08:29:14.509018+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:05.868646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.498316+0000","last_change":"2024-03-29T08:29:14.491102+0000","last_active":"2024-03-29T08:29:14.498316+0000","last_peered":"2024-03-29T08:29:14.498316+0000","last_clean":"2024-03-29T08:29:14.498316+0000","last_became_active":"2024-03-29T08:29:14.490878+0000","last_became_peered":"2024-03-29T08:29:14.490878+0000","last_unstale":"2024-03-29T08:29:14.498316+0000","last_undegraded":"2024-03-29T08:29:14.498316+0000","last_fullsized":"2024-03-29T08:29:14.498316+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:06:54.374100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509548+0000","last_change":"2024-03-29T08:29:14.097684+0000","last_active":"2024-03-29T08:29:14.509548+0000","last_peered":"2024-03-29T08:29:14.509548+0000","last_clean":"2024-03-29T08:29:14.509548+0000","last_became_active":"2024-03-29T08:29:14.095879+0000","last_became_peered":"2024-03-29T08:29:14.095879+0000","last_unstale":"2024-03-29T08:29:14.509548+0000","last_undegraded":"2024-03-29T08:29:14.509548+0000","last_fullsized":"2024-03-29T08:29:14.509548+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:50:38.108416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.511687+0000","last_change":"2024-03-29T08:29:14.507697+0000","last_active":"2024-03-29T08:29:14.511687+0000","last_peered":"2024-03-29T08:29:14.511687+0000","last_clean":"2024-03-29T08:29:14.511687+0000","last_became_active":"2024-03-29T08:29:14.507356+0000","last_became_peered":"2024-03-29T08:29:14.507356+0000","last_unstale":"2024-03-29T08:29:14.511687+0000","last_undegraded":"2024-03-29T08:29:14.511687+0000","last_fullsized":"2024-03-29T08:29:14.511687+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:11.283108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.502762+0000","last_change":"2024-03-29T08:29:14.086806+0000","last_active":"2024-03-29T08:29:14.502762+0000","last_peered":"2024-03-29T08:29:14.502762+0000","last_clean":"2024-03-29T08:29:14.502762+0000","last_became_active":"2024-03-29T08:29:14.086468+0000","last_became_peered":"2024-03-29T08:29:14.086468+0000","last_unstale":"2024-03-29T08:29:14.502762+0000","last_undegraded":"2024-03-29T08:29:14.502762+0000","last_fullsized":"2024-03-29T08:29:14.502762+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:37:57.757695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504878+0000","last_change":"2024-03-29T08:29:14.498041+0000","last_active":"2024-03-29T08:29:14.504878+0000","last_peered":"2024-03-29T08:29:14.504878+0000","last_clean":"2024-03-29T08:29:14.504878+0000","last_became_active":"2024-03-29T08:29:14.497768+0000","last_became_peered":"2024-03-29T08:29:14.497768+0000","last_unstale":"2024-03-29T08:29:14.504878+0000","last_undegraded":"2024-03-29T08:29:14.504878+0000","last_fullsized":"2024-03-29T08:29:14.504878+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:55.518674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504448+0000","last_change":"2024-03-29T08:29:14.088920+0000","last_active":"2024-03-29T08:29:14.504448+0000","last_peered":"2024-03-29T08:29:14.504448+0000","last_clean":"2024-03-29T08:29:14.504448+0000","last_became_active":"2024-03-29T08:29:14.088593+0000","last_became_peered":"2024-03-29T08:29:14.088593+0000","last_unstale":"2024-03-29T08:29:14.504448+0000","last_undegraded":"2024-03-29T08:29:14.504448+0000","last_fullsized":"2024-03-29T08:29:14.504448+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:51.446071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504649+0000","last_change":"2024-03-29T08:29:14.499273+0000","last_active":"2024-03-29T08:29:14.504649+0000","last_peered":"2024-03-29T08:29:14.504649+0000","last_clean":"2024-03-29T08:29:14.504649+0000","last_became_active":"2024-03-29T08:29:14.498368+0000","last_became_peered":"2024-03-29T08:29:14.498368+0000","last_unstale":"2024-03-29T08:29:14.504649+0000","last_undegraded":"2024-03-29T08:29:14.504649+0000","last_fullsized":"2024-03-29T08:29:14.504649+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:23:10.605671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.492779+0000","last_change":"2024-03-29T08:29:14.079054+0000","last_active":"2024-03-29T08:29:14.492779+0000","last_peered":"2024-03-29T08:29:14.492779+0000","last_clean":"2024-03-29T08:29:14.492779+0000","last_became_active":"2024-03-29T08:29:14.078272+0000","last_became_peered":"2024-03-29T08:29:14.078272+0000","last_unstale":"2024-03-29T08:29:14.492779+0000","last_undegraded":"2024-03-29T08:29:14.492779+0000","last_fullsized":"2024-03-29T08:29:14.492779+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:57:54.510656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.503738+0000","last_change":"2024-03-29T08:29:14.495103+0000","last_active":"2024-03-29T08:29:14.503738+0000","last_peered":"2024-03-29T08:29:14.503738+0000","last_clean":"2024-03-29T08:29:14.503738+0000","last_became_active":"2024-03-29T08:29:14.494873+0000","last_became_peered":"2024-03-29T08:29:14.494873+0000","last_unstale":"2024-03-29T08:29:14.503738+0000","last_undegraded":"2024-03-29T08:29:14.503738+0000","last_fullsized":"2024-03-29T08:29:14.503738+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:49:15.540195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.484296+0000","last_change":"2024-03-29T08:29:14.091783+0000","last_active":"2024-03-29T08:29:14.484296+0000","last_peered":"2024-03-29T08:29:14.484296+0000","last_clean":"2024-03-29T08:29:14.484296+0000","last_became_active":"2024-03-29T08:29:14.091531+0000","last_became_peered":"2024-03-29T08:29:14.091531+0000","last_unstale":"2024-03-29T08:29:14.484296+0000","last_undegraded":"2024-03-29T08:29:14.484296+0000","last_fullsized":"2024-03-29T08:29:14.484296+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:32:23.859154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504693+0000","last_change":"2024-03-29T08:29:14.498939+0000","last_active":"2024-03-29T08:29:14.504693+0000","last_peered":"2024-03-29T08:29:14.504693+0000","last_clean":"2024-03-29T08:29:14.504693+0000","last_became_active":"2024-03-29T08:29:14.498589+0000","last_became_peered":"2024-03-29T08:29:14.498589+0000","last_unstale":"2024-03-29T08:29:14.504693+0000","last_undegraded":"2024-03-29T08:29:14.504693+0000","last_fullsized":"2024-03-29T08:29:14.504693+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:32:24.719500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.496023+0000","last_change":"2024-03-29T08:29:14.096474+0000","last_active":"2024-03-29T08:29:14.496023+0000","last_peered":"2024-03-29T08:29:14.496023+0000","last_clean":"2024-03-29T08:29:14.496023+0000","last_became_active":"2024-03-29T08:29:14.094336+0000","last_became_peered":"2024-03-29T08:29:14.094336+0000","last_unstale":"2024-03-29T08:29:14.496023+0000","last_undegraded":"2024-03-29T08:29:14.496023+0000","last_fullsized":"2024-03-29T08:29:14.496023+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:39.277282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.489947+0000","last_change":"2024-03-29T08:29:14.487254+0000","last_active":"2024-03-29T08:29:14.489947+0000","last_peered":"2024-03-29T08:29:14.489947+0000","last_clean":"2024-03-29T08:29:14.489947+0000","last_became_active":"2024-03-29T08:29:14.485472+0000","last_became_peered":"2024-03-29T08:29:14.485472+0000","last_unstale":"2024-03-29T08:29:14.489947+0000","last_undegraded":"2024-03-29T08:29:14.489947+0000","last_fullsized":"2024-03-29T08:29:14.489947+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:08:57.590159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.484431+0000","last_change":"2024-03-29T08:29:14.092400+0000","last_active":"2024-03-29T08:29:14.484431+0000","last_peered":"2024-03-29T08:29:14.484431+0000","last_clean":"2024-03-29T08:29:14.484431+0000","last_became_active":"2024-03-29T08:29:14.091857+0000","last_became_peered":"2024-03-29T08:29:14.091857+0000","last_unstale":"2024-03-29T08:29:14.484431+0000","last_undegraded":"2024-03-29T08:29:14.484431+0000","last_fullsized":"2024-03-29T08:29:14.484431+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:37:57.757695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.085301+0000","last_change":"2024-03-29T08:29:14.085301+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.085301+0000","last_undegraded":"2024-03-29T08:29:14.085301+0000","last_fullsized":"2024-03-29T08:29:14.085301+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:45.863920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"120'3","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509397+0000","last_change":"2024-03-29T08:29:14.092891+0000","last_active":"2024-03-29T08:29:14.509397+0000","last_peered":"2024-03-29T08:29:14.509397+0000","last_clean":"2024-03-29T08:29:14.509397+0000","last_became_active":"2024-03-29T08:29:14.092632+0000","last_became_peered":"2024-03-29T08:29:14.092632+0000","last_unstale":"2024-03-29T08:29:14.509397+0000","last_undegraded":"2024-03-29T08:29:14.509397+0000","last_fullsized":"2024-03-29T08:29:14.509397+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:52:04.161387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.492713+0000","last_change":"2024-03-29T08:29:14.489765+0000","last_active":"2024-03-29T08:29:14.492713+0000","last_peered":"2024-03-29T08:29:14.492713+0000","last_clean":"2024-03-29T08:29:14.492713+0000","last_became_active":"2024-03-29T08:29:14.489440+0000","last_became_peered":"2024-03-29T08:29:14.489440+0000","last_unstale":"2024-03-29T08:29:14.492713+0000","last_undegraded":"2024-03-29T08:29:14.492713+0000","last_fullsized":"2024-03-29T08:29:14.492713+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:53:42.976104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"120'6","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.512807+0000","last_change":"2024-03-29T08:29:14.096725+0000","last_active":"2024-03-29T08:29:14.512807+0000","last_peered":"2024-03-29T08:29:14.512807+0000","last_clean":"2024-03-29T08:29:14.512807+0000","last_became_active":"2024-03-29T08:29:14.094695+0000","last_became_peered":"2024-03-29T08:29:14.094695+0000","last_unstale":"2024-03-29T08:29:14.512807+0000","last_undegraded":"2024-03-29T08:29:14.512807+0000","last_fullsized":"2024-03-29T08:29:14.512807+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:15:28.507662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504528+0000","last_change":"2024-03-29T08:29:14.499972+0000","last_active":"2024-03-29T08:29:14.504528+0000","last_peered":"2024-03-29T08:29:14.504528+0000","last_clean":"2024-03-29T08:29:14.504528+0000","last_became_active":"2024-03-29T08:29:14.499698+0000","last_became_peered":"2024-03-29T08:29:14.499698+0000","last_unstale":"2024-03-29T08:29:14.504528+0000","last_undegraded":"2024-03-29T08:29:14.504528+0000","last_fullsized":"2024-03-29T08:29:14.504528+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:44:41.312640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508984+0000","last_change":"2024-03-29T08:29:14.092574+0000","last_active":"2024-03-29T08:29:14.508984+0000","last_peered":"2024-03-29T08:29:14.508984+0000","last_clean":"2024-03-29T08:29:14.508984+0000","last_became_active":"2024-03-29T08:29:14.092004+0000","last_became_peered":"2024-03-29T08:29:14.092004+0000","last_unstale":"2024-03-29T08:29:14.508984+0000","last_undegraded":"2024-03-29T08:29:14.508984+0000","last_fullsized":"2024-03-29T08:29:14.508984+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:21.448128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.510447+0000","last_change":"2024-03-29T08:29:14.502264+0000","last_active":"2024-03-29T08:29:14.510447+0000","last_peered":"2024-03-29T08:29:14.510447+0000","last_clean":"2024-03-29T08:29:14.510447+0000","last_became_active":"2024-03-29T08:29:14.501882+0000","last_became_peered":"2024-03-29T08:29:14.501882+0000","last_unstale":"2024-03-29T08:29:14.510447+0000","last_undegraded":"2024-03-29T08:29:14.510447+0000","last_fullsized":"2024-03-29T08:29:14.510447+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:31:05.279265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.480729+0000","last_change":"2024-03-29T08:29:14.079278+0000","last_active":"2024-03-29T08:29:14.480729+0000","last_peered":"2024-03-29T08:29:14.480729+0000","last_clean":"2024-03-29T08:29:14.480729+0000","last_became_active":"2024-03-29T08:29:14.078982+0000","last_became_peered":"2024-03-29T08:29:14.078982+0000","last_unstale":"2024-03-29T08:29:14.480729+0000","last_undegraded":"2024-03-29T08:29:14.480729+0000","last_fullsized":"2024-03-29T08:29:14.480729+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:06.149502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509069+0000","last_change":"2024-03-29T08:29:14.500759+0000","last_active":"2024-03-29T08:29:14.509069+0000","last_peered":"2024-03-29T08:29:14.509069+0000","last_clean":"2024-03-29T08:29:14.509069+0000","last_became_active":"2024-03-29T08:29:14.500463+0000","last_became_peered":"2024-03-29T08:29:14.500463+0000","last_unstale":"2024-03-29T08:29:14.509069+0000","last_undegraded":"2024-03-29T08:29:14.509069+0000","last_fullsized":"2024-03-29T08:29:14.509069+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:13:49.079869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"120'8","reported_seq":18,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.510685+0000","last_change":"2024-03-29T08:29:14.090542+0000","last_active":"2024-03-29T08:29:14.510685+0000","last_peered":"2024-03-29T08:29:14.510685+0000","last_clean":"2024-03-29T08:29:14.510685+0000","last_became_active":"2024-03-29T08:29:14.090235+0000","last_became_peered":"2024-03-29T08:29:14.090235+0000","last_unstale":"2024-03-29T08:29:14.510685+0000","last_undegraded":"2024-03-29T08:29:14.510685+0000","last_fullsized":"2024-03-29T08:29:14.510685+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:45.558553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508199+0000","last_change":"2024-03-29T08:29:14.497747+0000","last_active":"2024-03-29T08:29:14.508199+0000","last_peered":"2024-03-29T08:29:14.508199+0000","last_clean":"2024-03-29T08:29:14.508199+0000","last_became_active":"2024-03-29T08:29:14.497456+0000","last_became_peered":"2024-03-29T08:29:14.497456+0000","last_unstale":"2024-03-29T08:29:14.508199+0000","last_undegraded":"2024-03-29T08:29:14.508199+0000","last_fullsized":"2024-03-29T08:29:14.508199+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:49.180147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.097550+0000","last_change":"2024-03-29T08:29:14.094084+0000","last_active":"2024-03-29T08:29:14.097550+0000","last_peered":"2024-03-29T08:29:14.097550+0000","last_clean":"2024-03-29T08:29:14.097550+0000","last_became_active":"2024-03-29T08:29:14.093447+0000","last_became_peered":"2024-03-29T08:29:14.093447+0000","last_unstale":"2024-03-29T08:29:14.097550+0000","last_undegraded":"2024-03-29T08:29:14.097550+0000","last_fullsized":"2024-03-29T08:29:14.097550+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:11:26.207009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.512213+0000","last_change":"2024-03-29T08:29:14.507678+0000","last_active":"2024-03-29T08:29:14.512213+0000","last_peered":"2024-03-29T08:29:14.512213+0000","last_clean":"2024-03-29T08:29:14.512213+0000","last_became_active":"2024-03-29T08:29:14.507132+0000","last_became_peered":"2024-03-29T08:29:14.507132+0000","last_unstale":"2024-03-29T08:29:14.512213+0000","last_undegraded":"2024-03-29T08:29:14.512213+0000","last_fullsized":"2024-03-29T08:29:14.512213+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:15:29.513377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.502517+0000","last_change":"2024-03-29T08:29:14.095051+0000","last_active":"2024-03-29T08:29:14.502517+0000","last_peered":"2024-03-29T08:29:14.502517+0000","last_clean":"2024-03-29T08:29:14.502517+0000","last_became_active":"2024-03-29T08:29:14.092852+0000","last_became_peered":"2024-03-29T08:29:14.092852+0000","last_unstale":"2024-03-29T08:29:14.502517+0000","last_undegraded":"2024-03-29T08:29:14.502517+0000","last_fullsized":"2024-03-29T08:29:14.502517+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:06.149502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504967+0000","last_change":"2024-03-29T08:29:14.497468+0000","last_active":"2024-03-29T08:29:14.504967+0000","last_peered":"2024-03-29T08:29:14.504967+0000","last_clean":"2024-03-29T08:29:14.504967+0000","last_became_active":"2024-03-29T08:29:14.497061+0000","last_became_peered":"2024-03-29T08:29:14.497061+0000","last_unstale":"2024-03-29T08:29:14.504967+0000","last_undegraded":"2024-03-29T08:29:14.504967+0000","last_fullsized":"2024-03-29T08:29:14.504967+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:09:20.324353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.501012+0000","last_change":"2024-03-29T08:29:14.090528+0000","last_active":"2024-03-29T08:29:14.501012+0000","last_peered":"2024-03-29T08:29:14.501012+0000","last_clean":"2024-03-29T08:29:14.501012+0000","last_became_active":"2024-03-29T08:29:14.090055+0000","last_became_peered":"2024-03-29T08:29:14.090055+0000","last_unstale":"2024-03-29T08:29:14.501012+0000","last_undegraded":"2024-03-29T08:29:14.501012+0000","last_fullsized":"2024-03-29T08:29:14.501012+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:21.448128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505830+0000","last_change":"2024-03-29T08:29:14.502568+0000","last_active":"2024-03-29T08:29:14.505830+0000","last_peered":"2024-03-29T08:29:14.505830+0000","last_clean":"2024-03-29T08:29:14.505830+0000","last_became_active":"2024-03-29T08:29:14.500161+0000","last_became_peered":"2024-03-29T08:29:14.500161+0000","last_unstale":"2024-03-29T08:29:14.505830+0000","last_undegraded":"2024-03-29T08:29:14.505830+0000","last_fullsized":"2024-03-29T08:29:14.505830+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:25:45.763218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.487422+0000","last_change":"2024-03-29T08:29:14.078956+0000","last_active":"2024-03-29T08:29:14.487422+0000","last_peered":"2024-03-29T08:29:14.487422+0000","last_clean":"2024-03-29T08:29:14.487422+0000","last_became_active":"2024-03-29T08:29:14.078689+0000","last_became_peered":"2024-03-29T08:29:14.078689+0000","last_unstale":"2024-03-29T08:29:14.487422+0000","last_undegraded":"2024-03-29T08:29:14.487422+0000","last_fullsized":"2024-03-29T08:29:14.487422+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:42:53.153883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.489423+0000","last_change":"2024-03-29T08:29:14.486493+0000","last_active":"2024-03-29T08:29:14.489423+0000","last_peered":"2024-03-29T08:29:14.489423+0000","last_clean":"2024-03-29T08:29:14.489423+0000","last_became_active":"2024-03-29T08:29:14.486242+0000","last_became_peered":"2024-03-29T08:29:14.486242+0000","last_unstale":"2024-03-29T08:29:14.489423+0000","last_undegraded":"2024-03-29T08:29:14.489423+0000","last_fullsized":"2024-03-29T08:29:14.489423+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:30:38.933793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507685+0000","last_change":"2024-03-29T08:29:14.092768+0000","last_active":"2024-03-29T08:29:14.507685+0000","last_peered":"2024-03-29T08:29:14.507685+0000","last_clean":"2024-03-29T08:29:14.507685+0000","last_became_active":"2024-03-29T08:29:14.092290+0000","last_became_peered":"2024-03-29T08:29:14.092290+0000","last_unstale":"2024-03-29T08:29:14.507685+0000","last_undegraded":"2024-03-29T08:29:14.507685+0000","last_fullsized":"2024-03-29T08:29:14.507685+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:45.752363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509731+0000","last_change":"2024-03-29T08:29:14.502876+0000","last_active":"2024-03-29T08:29:14.509731+0000","last_peered":"2024-03-29T08:29:14.509731+0000","last_clean":"2024-03-29T08:29:14.509731+0000","last_became_active":"2024-03-29T08:29:14.502600+0000","last_became_peered":"2024-03-29T08:29:14.502600+0000","last_unstale":"2024-03-29T08:29:14.509731+0000","last_undegraded":"2024-03-29T08:29:14.509731+0000","last_fullsized":"2024-03-29T08:29:14.509731+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:49:19.406820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.486956+0000","last_change":"2024-03-29T08:29:14.090942+0000","last_active":"2024-03-29T08:29:14.486956+0000","last_peered":"2024-03-29T08:29:14.486956+0000","last_clean":"2024-03-29T08:29:14.486956+0000","last_became_active":"2024-03-29T08:29:14.090695+0000","last_became_peered":"2024-03-29T08:29:14.090695+0000","last_unstale":"2024-03-29T08:29:14.486956+0000","last_undegraded":"2024-03-29T08:29:14.486956+0000","last_fullsized":"2024-03-29T08:29:14.486956+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:18.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509268+0000","last_change":"2024-03-29T08:29:14.508582+0000","last_active":"2024-03-29T08:29:14.509268+0000","last_peered":"2024-03-29T08:29:14.509268+0000","last_clean":"2024-03-29T08:29:14.509268+0000","last_became_active":"2024-03-29T08:29:14.508201+0000","last_became_peered":"2024-03-29T08:29:14.508201+0000","last_unstale":"2024-03-29T08:29:14.509268+0000","last_undegraded":"2024-03-29T08:29:14.509268+0000","last_fullsized":"2024-03-29T08:29:14.509268+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:59:12.691184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.497159+0000","last_change":"2024-03-29T08:29:14.096474+0000","last_active":"2024-03-29T08:29:14.497159+0000","last_peered":"2024-03-29T08:29:14.497159+0000","last_clean":"2024-03-29T08:29:14.497159+0000","last_became_active":"2024-03-29T08:29:14.094758+0000","last_became_peered":"2024-03-29T08:29:14.094758+0000","last_unstale":"2024-03-29T08:29:14.497159+0000","last_undegraded":"2024-03-29T08:29:14.497159+0000","last_fullsized":"2024-03-29T08:29:14.497159+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:32:23.713785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505838+0000","last_change":"2024-03-29T08:29:14.503676+0000","last_active":"2024-03-29T08:29:14.505838+0000","last_peered":"2024-03-29T08:29:14.505838+0000","last_clean":"2024-03-29T08:29:14.505838+0000","last_became_active":"2024-03-29T08:29:14.503314+0000","last_became_peered":"2024-03-29T08:29:14.503314+0000","last_unstale":"2024-03-29T08:29:14.505838+0000","last_undegraded":"2024-03-29T08:29:14.505838+0000","last_fullsized":"2024-03-29T08:29:14.505838+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:08:59.334138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.496690+0000","last_change":"2024-03-29T08:29:14.089768+0000","last_active":"2024-03-29T08:29:14.496690+0000","last_peered":"2024-03-29T08:29:14.496690+0000","last_clean":"2024-03-29T08:29:14.496690+0000","last_became_active":"2024-03-29T08:29:14.089486+0000","last_became_peered":"2024-03-29T08:29:14.089486+0000","last_unstale":"2024-03-29T08:29:14.496690+0000","last_undegraded":"2024-03-29T08:29:14.496690+0000","last_fullsized":"2024-03-29T08:29:14.496690+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:59.174853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507881+0000","last_change":"2024-03-29T08:29:14.505681+0000","last_active":"2024-03-29T08:29:14.507881+0000","last_peered":"2024-03-29T08:29:14.507881+0000","last_clean":"2024-03-29T08:29:14.507881+0000","last_became_active":"2024-03-29T08:29:14.505394+0000","last_became_peered":"2024-03-29T08:29:14.505394+0000","last_unstale":"2024-03-29T08:29:14.507881+0000","last_undegraded":"2024-03-29T08:29:14.507881+0000","last_fullsized":"2024-03-29T08:29:14.507881+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:32:12.510083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508946+0000","last_change":"2024-03-29T08:29:14.088394+0000","last_active":"2024-03-29T08:29:14.508946+0000","last_peered":"2024-03-29T08:29:14.508946+0000","last_clean":"2024-03-29T08:29:14.508946+0000","last_became_active":"2024-03-29T08:29:14.087525+0000","last_became_peered":"2024-03-29T08:29:14.087525+0000","last_unstale":"2024-03-29T08:29:14.508946+0000","last_undegraded":"2024-03-29T08:29:14.508946+0000","last_fullsized":"2024-03-29T08:29:14.508946+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:54:27.262284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.086933+0000","last_change":"2024-03-29T08:29:14.086933+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.086933+0000","last_undegraded":"2024-03-29T08:29:14.086933+0000","last_fullsized":"2024-03-29T08:29:14.086933+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:14:06.265379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.497231+0000","last_change":"2024-03-29T08:29:14.088202+0000","last_active":"2024-03-29T08:29:14.497231+0000","last_peered":"2024-03-29T08:29:14.497231+0000","last_clean":"2024-03-29T08:29:14.497231+0000","last_became_active":"2024-03-29T08:29:14.087860+0000","last_became_peered":"2024-03-29T08:29:14.087860+0000","last_unstale":"2024-03-29T08:29:14.497231+0000","last_undegraded":"2024-03-29T08:29:14.497231+0000","last_fullsized":"2024-03-29T08:29:14.497231+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:30:51.004244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509624+0000","last_change":"2024-03-29T08:29:14.501270+0000","last_active":"2024-03-29T08:29:14.509624+0000","last_peered":"2024-03-29T08:29:14.509624+0000","last_clean":"2024-03-29T08:29:14.509624+0000","last_became_active":"2024-03-29T08:29:14.498111+0000","last_became_peered":"2024-03-29T08:29:14.498111+0000","last_unstale":"2024-03-29T08:29:14.509624+0000","last_undegraded":"2024-03-29T08:29:14.509624+0000","last_fullsized":"2024-03-29T08:29:14.509624+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:26:17.474099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.495208+0000","last_change":"2024-03-29T08:29:14.090681+0000","last_active":"2024-03-29T08:29:14.495208+0000","last_peered":"2024-03-29T08:29:14.495208+0000","last_clean":"2024-03-29T08:29:14.495208+0000","last_became_active":"2024-03-29T08:29:14.090416+0000","last_became_peered":"2024-03-29T08:29:14.090416+0000","last_unstale":"2024-03-29T08:29:14.495208+0000","last_undegraded":"2024-03-29T08:29:14.495208+0000","last_fullsized":"2024-03-29T08:29:14.495208+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:08:56.584444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507376+0000","last_change":"2024-03-29T08:29:14.505745+0000","last_active":"2024-03-29T08:29:14.507376+0000","last_peered":"2024-03-29T08:29:14.507376+0000","last_clean":"2024-03-29T08:29:14.507376+0000","last_became_active":"2024-03-29T08:29:14.504499+0000","last_became_peered":"2024-03-29T08:29:14.504499+0000","last_unstale":"2024-03-29T08:29:14.507376+0000","last_undegraded":"2024-03-29T08:29:14.507376+0000","last_fullsized":"2024-03-29T08:29:14.507376+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:23:28.769780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.495503+0000","last_change":"2024-03-29T08:29:14.096941+0000","last_active":"2024-03-29T08:29:14.495503+0000","last_peered":"2024-03-29T08:29:14.495503+0000","last_clean":"2024-03-29T08:29:14.495503+0000","last_became_active":"2024-03-29T08:29:14.096439+0000","last_became_peered":"2024-03-29T08:29:14.096439+0000","last_unstale":"2024-03-29T08:29:14.495503+0000","last_undegraded":"2024-03-29T08:29:14.495503+0000","last_fullsized":"2024-03-29T08:29:14.495503+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:49:13.231792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.503705+0000","last_change":"2024-03-29T08:29:14.496299+0000","last_active":"2024-03-29T08:29:14.503705+0000","last_peered":"2024-03-29T08:29:14.503705+0000","last_clean":"2024-03-29T08:29:14.503705+0000","last_became_active":"2024-03-29T08:29:14.496042+0000","last_became_peered":"2024-03-29T08:29:14.496042+0000","last_unstale":"2024-03-29T08:29:14.503705+0000","last_undegraded":"2024-03-29T08:29:14.503705+0000","last_fullsized":"2024-03-29T08:29:14.503705+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:23:28.769780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.497619+0000","last_change":"2024-03-29T08:29:14.081880+0000","last_active":"2024-03-29T08:29:14.497619+0000","last_peered":"2024-03-29T08:29:14.497619+0000","last_clean":"2024-03-29T08:29:14.497619+0000","last_became_active":"2024-03-29T08:29:14.079732+0000","last_became_peered":"2024-03-29T08:29:14.079732+0000","last_unstale":"2024-03-29T08:29:14.497619+0000","last_undegraded":"2024-03-29T08:29:14.497619+0000","last_fullsized":"2024-03-29T08:29:14.497619+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:01.494143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.503628+0000","last_change":"2024-03-29T08:29:14.501776+0000","last_active":"2024-03-29T08:29:14.503628+0000","last_peered":"2024-03-29T08:29:14.503628+0000","last_clean":"2024-03-29T08:29:14.503628+0000","last_became_active":"2024-03-29T08:29:14.501508+0000","last_became_peered":"2024-03-29T08:29:14.501508+0000","last_unstale":"2024-03-29T08:29:14.503628+0000","last_undegraded":"2024-03-29T08:29:14.503628+0000","last_fullsized":"2024-03-29T08:29:14.503628+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:01.476642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.487173+0000","last_change":"2024-03-29T08:29:14.091456+0000","last_active":"2024-03-29T08:29:14.487173+0000","last_peered":"2024-03-29T08:29:14.487173+0000","last_clean":"2024-03-29T08:29:14.487173+0000","last_became_active":"2024-03-29T08:29:14.091121+0000","last_became_peered":"2024-03-29T08:29:14.091121+0000","last_unstale":"2024-03-29T08:29:14.487173+0000","last_undegraded":"2024-03-29T08:29:14.487173+0000","last_fullsized":"2024-03-29T08:29:14.487173+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:30.979257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507721+0000","last_change":"2024-03-29T08:29:14.503755+0000","last_active":"2024-03-29T08:29:14.507721+0000","last_peered":"2024-03-29T08:29:14.507721+0000","last_clean":"2024-03-29T08:29:14.507721+0000","last_became_active":"2024-03-29T08:29:14.503521+0000","last_became_peered":"2024-03-29T08:29:14.503521+0000","last_unstale":"2024-03-29T08:29:14.507721+0000","last_undegraded":"2024-03-29T08:29:14.507721+0000","last_fullsized":"2024-03-29T08:29:14.507721+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:33:36.260388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.486511+0000","last_change":"2024-03-29T08:29:14.089662+0000","last_active":"2024-03-29T08:29:14.486511+0000","last_peered":"2024-03-29T08:29:14.486511+0000","last_clean":"2024-03-29T08:29:14.486511+0000","last_became_active":"2024-03-29T08:29:14.089393+0000","last_became_peered":"2024-03-29T08:29:14.089393+0000","last_unstale":"2024-03-29T08:29:14.486511+0000","last_undegraded":"2024-03-29T08:29:14.486511+0000","last_fullsized":"2024-03-29T08:29:14.486511+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:27:50.778160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.084305+0000","last_change":"2024-03-29T08:29:14.084305+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.084305+0000","last_undegraded":"2024-03-29T08:29:14.084305+0000","last_fullsized":"2024-03-29T08:29:14.084305+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:44:25.936884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.095596+0000","last_change":"2024-03-29T08:29:14.092230+0000","last_active":"2024-03-29T08:29:14.095596+0000","last_peered":"2024-03-29T08:29:14.095596+0000","last_clean":"2024-03-29T08:29:14.095596+0000","last_became_active":"2024-03-29T08:29:14.091979+0000","last_became_peered":"2024-03-29T08:29:14.091979+0000","last_unstale":"2024-03-29T08:29:14.095596+0000","last_undegraded":"2024-03-29T08:29:14.095596+0000","last_fullsized":"2024-03-29T08:29:14.095596+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:41:42.442894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.510494+0000","last_change":"2024-03-29T08:29:14.502476+0000","last_active":"2024-03-29T08:29:14.510494+0000","last_peered":"2024-03-29T08:29:14.510494+0000","last_clean":"2024-03-29T08:29:14.510494+0000","last_became_active":"2024-03-29T08:29:14.502191+0000","last_became_peered":"2024-03-29T08:29:14.502191+0000","last_unstale":"2024-03-29T08:29:14.510494+0000","last_undegraded":"2024-03-29T08:29:14.510494+0000","last_fullsized":"2024-03-29T08:29:14.510494+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:06.826332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.475058+0000","last_change":"2024-03-29T08:29:14.077404+0000","last_active":"2024-03-29T08:29:14.475058+0000","last_peered":"2024-03-29T08:29:14.475058+0000","last_clean":"2024-03-29T08:29:14.475058+0000","last_became_active":"2024-03-29T08:29:14.077141+0000","last_became_peered":"2024-03-29T08:29:14.077141+0000","last_unstale":"2024-03-29T08:29:14.475058+0000","last_undegraded":"2024-03-29T08:29:14.475058+0000","last_fullsized":"2024-03-29T08:29:14.475058+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:51.446071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":250,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.072521+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:29:14.072521+0000","last_peered":"2024-03-29T08:29:14.072521+0000","last_clean":"2024-03-29T08:29:14.072521+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:29:14.072521+0000","last_undegraded":"2024-03-29T08:29:14.072521+0000","last_fullsized":"2024-03-29T08:29:14.072521+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509578,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1503232,"data_stored":646009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.69299999999999995},"min":{"1min":0.48199999999999998,"5min":0.41899999999999998,"15min":0.34999999999999998},"max":{"1min":0.88,"5min":0.97999999999999998,"15min":1.1639999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.75600000000000001,"15min":0.71899999999999997},"min":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.45800000000000002},"max":{"1min":0.95099999999999996,"5min":0.998,"15min":1.2410000000000001},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70199999999999996,"15min":0.67600000000000005},"min":{"1min":0.51500000000000001,"5min":0.45700000000000002,"15min":0.39600000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.202},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70799999999999996,"15min":0.67600000000000005},"min":{"1min":0.56100000000000005,"5min":0.48799999999999999,"15min":0.44},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.1339999999999999},"last":0.505}]},{"osd":2,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69799999999999995,"15min":0.67000000000000004},"min":{"1min":0.502,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.90300000000000002,"5min":0.93600000000000005,"15min":1.085},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72899999999999998,"15min":0.70299999999999996},"min":{"1min":0.55500000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.288},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72299999999999998,"15min":0.70999999999999996},"min":{"1min":0.57599999999999996,"5min":0.51100000000000001,"15min":0.46000000000000002},"max":{"1min":0.97199999999999998,"5min":0.98099999999999998,"15min":1.018},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.60899999999999999,"5min":0.52300000000000002,"15min":0.48499999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.2609999999999999},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58399999999999996,"15min":0.60999999999999999},"min":{"1min":0.48099999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85299999999999998,"5min":0.87,"15min":0.95299999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60599999999999998,"15min":0.66300000000000003},"min":{"1min":0.45100000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.123},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60999999999999999,"15min":0.66200000000000003},"min":{"1min":0.48199999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.877,"5min":0.89600000000000002,"15min":1.056},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.628,"15min":0.66900000000000004},"min":{"1min":0.49199999999999999,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.84399999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.47399999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69199999999999995,"15min":0.71399999999999997},"min":{"1min":0.55900000000000005,"5min":0.46999999999999997,"15min":0.46600000000000003},"max":{"1min":0.89400000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.436},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65800000000000003,"15min":0.70599999999999996},"min":{"1min":0.52800000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.81799999999999995,"5min":0.93799999999999994,"15min":0.98299999999999998},"last":0.64400000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509578,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1482752,"data_stored":644321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72399999999999998,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.38900000000000001},"max":{"1min":0.873,"5min":0.93100000000000005,"15min":1.0189999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.751,"15min":0.71899999999999997},"min":{"1min":0.57199999999999995,"5min":0.51700000000000002,"15min":0.44500000000000001},"max":{"1min":0.97699999999999998,"5min":0.99299999999999999,"15min":1.079},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68200000000000005,"15min":0.64700000000000002},"min":{"1min":0.45400000000000001,"5min":0.45300000000000001,"15min":0.377},"max":{"1min":0.85199999999999998,"5min":0.97999999999999998,"15min":0.99299999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42899999999999999},"max":{"1min":0.78200000000000003,"5min":0.95599999999999996,"15min":1.079},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76900000000000002,"15min":0.72299999999999998},"min":{"1min":0.52100000000000002,"5min":0.51800000000000002,"15min":0.44800000000000001},"max":{"1min":0.91500000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.746,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.46400000000000002,"15min":0.435},"max":{"1min":0.93899999999999995,"5min":0.95099999999999996,"15min":0.95899999999999996},"last":0.753}]},{"osd":3,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.443,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82099999999999995,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.55700000000000005,"5min":0.51000000000000001,"15min":0.47799999999999998},"max":{"1min":0.89800000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.71299999999999997}]},{"osd":4,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.623,"15min":0.64600000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.36499999999999999},"max":{"1min":0.93100000000000005,"5min":1.0840000000000001,"15min":1.556},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63900000000000001,"15min":0.68200000000000005},"min":{"1min":0.46400000000000002,"5min":0.38200000000000001,"15min":0.371},"max":{"1min":0.872,"5min":0.996,"15min":2.0539999999999998},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.627,"15min":0.66100000000000003},"min":{"1min":0.45000000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83599999999999997,"5min":0.99399999999999999,"15min":1.0660000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68100000000000005,"15min":0.70199999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":1.042,"15min":1.1419999999999999},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69599999999999995,"15min":0.72499999999999998},"min":{"1min":0.499,"5min":0.46800000000000003,"15min":0.38500000000000001},"max":{"1min":0.89600000000000002,"5min":1.03,"15min":1.03},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.72999999999999998},"min":{"1min":0.51600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91500000000000004,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.63}]}]},{"osd":4,"up_from":15,"seq":64424509577,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1499136,"data_stored":645975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.45000000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.026,"5min":1.026,"15min":2.1709999999999998},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69699999999999995,"15min":0.71799999999999997},"min":{"1min":0.495,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.997,"5min":0.997,"15min":2.2869999999999999},"last":0.82999999999999996}]},{"osd":1,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.72499999999999998,"15min":0.748},"min":{"1min":0.436,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":2.524},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71099999999999997,"15min":0.70899999999999996},"min":{"1min":0.46400000000000002,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":2.2679999999999998},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69999999999999996,"15min":0.71799999999999997},"min":{"1min":0.45300000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":2.363},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72999999999999998,"15min":0.748},"min":{"1min":0.52500000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.88700000000000001,"5min":0.97799999999999998,"15min":2.1880000000000002},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76200000000000001,"15min":0.753},"min":{"1min":0.60899999999999999,"5min":0.53400000000000003,"15min":0.46300000000000002},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":2.3029999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.77100000000000002,"15min":0.79300000000000004},"min":{"1min":0.55600000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.016,"5min":1.016,"15min":2.2549999999999999},"last":0.81599999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60699999999999998,"15min":0.65800000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.36499999999999999},"max":{"1min":0.76400000000000001,"5min":0.998,"15min":1.0189999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.62,"15min":0.63900000000000001},"min":{"1min":0.45800000000000002,"5min":0.437,"15min":0.41799999999999998},"max":{"1min":0.82699999999999996,"5min":0.85499999999999998,"15min":1.0149999999999999},"last":0.59599999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.63600000000000001,"15min":0.65700000000000003},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.85599999999999998,"5min":1.091,"15min":1.091},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69399999999999995,"15min":0.72099999999999997},"min":{"1min":0.505,"5min":0.499,"15min":0.499},"max":{"1min":0.86399999999999999,"5min":0.91500000000000004,"15min":1.1220000000000001},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.71199999999999997},"min":{"1min":0.51400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91400000000000003,"5min":0.91800000000000004,"15min":1.1000000000000001},"last":0.754},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72599999999999998,"15min":0.747},"min":{"1min":0.434,"5min":0.434,"15min":0.42899999999999999},"max":{"1min":0.96499999999999997,"5min":1.054,"15min":1.4430000000000001},"last":0.69399999999999995}]}]},{"osd":3,"up_from":15,"seq":64424509580,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1482752,"data_stored":644321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.28100000000000003,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":0.71799999999999997,"5min":0.748,"15min":1.123},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69599999999999995,"5min":0.88100000000000001,"15min":1.0409999999999999},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.44,"15min":0.44800000000000001},"min":{"1min":0.32800000000000001,"5min":0.29599999999999999,"15min":0.26500000000000001},"max":{"1min":0.623,"5min":0.76800000000000002,"15min":1.1599999999999999},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.28599999999999998},"max":{"1min":0.82099999999999995,"5min":0.94499999999999995,"15min":1.4790000000000001},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.54400000000000004},"min":{"1min":0.35499999999999998,"5min":0.32900000000000001,"15min":0.309},"max":{"1min":0.76200000000000001,"5min":0.80300000000000005,"15min":1.2529999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.624},"min":{"1min":0.439,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78300000000000003,"5min":0.86599999999999999,"15min":1.8580000000000001},"last":0.59899999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77200000000000002,"15min":0.77500000000000002},"min":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.42199999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.6419999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.77900000000000003,"15min":0.76900000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.45400000000000001},"max":{"1min":1.083,"5min":1.3340000000000001,"15min":1.391},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82199999999999995,"15min":0.80300000000000005},"min":{"1min":0.59599999999999997,"5min":0.57099999999999995,"15min":0.52100000000000002},"max":{"1min":1.081,"5min":1.1850000000000001,"15min":1.671},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.82399999999999995,"15min":0.83499999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0249999999999999,"5min":1.2050000000000001,"15min":1.7110000000000001},"last":0.58699999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.55400000000000005,"5min":0.51700000000000002,"15min":0.48799999999999999},"max":{"1min":1.232,"5min":1.2609999999999999,"15min":1.669},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78100000000000003,"15min":0.79600000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0029999999999999,"5min":1.119,"15min":2.0150000000000001},"last":0.54600000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.82199999999999995},"min":{"1min":0.58899999999999997,"5min":0.50600000000000001,"15min":0.48799999999999999},"max":{"1min":1.0069999999999999,"5min":1.3169999999999999,"15min":1.887},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.81899999999999995},"min":{"1min":0.58999999999999997,"5min":0.49399999999999999,"15min":0.47999999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":2.044},"last":0.57199999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509579,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1470464,"data_stored":637546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45900000000000002,"15min":0.47799999999999998},"min":{"1min":0.312,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.67800000000000005,"5min":0.754,"15min":0.94299999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.378,"5min":0.34699999999999998,"15min":0.32300000000000001},"max":{"1min":0.70699999999999996,"5min":0.874,"15min":2.0880000000000001},"last":0.45200000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.49399999999999999},"min":{"1min":0.34000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70599999999999996,"5min":0.73499999999999999,"15min":1.399},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.501,"15min":0.50600000000000001},"min":{"1min":0.312,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":3.0459999999999998},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79900000000000004,"5min":0.93300000000000005,"15min":2.1280000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.35999999999999999},"max":{"1min":0.79900000000000004,"5min":0.88800000000000001,"15min":2.9950000000000001},"last":0.71399999999999997}]},{"osd":4,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79400000000000004,"15min":0.77700000000000002},"min":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.49099999999999999},"max":{"1min":1.0329999999999999,"5min":1.109,"15min":1.4239999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.76600000000000001},"min":{"1min":0.51900000000000002,"5min":0.51700000000000002,"15min":0.435},"max":{"1min":1.1479999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.92800000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.85599999999999998,"15min":0.83999999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.143,"5min":1.236,"15min":1.4890000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.48799999999999999},"max":{"1min":1.1100000000000001,"5min":1.194,"15min":1.526},"last":0.97799999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79100000000000004,"15min":0.78200000000000003},"min":{"1min":0.57899999999999996,"5min":0.54600000000000004,"15min":0.48599999999999999},"max":{"1min":1.1040000000000001,"5min":1.131,"15min":2.0470000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.82999999999999996,"15min":0.79800000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.52500000000000002},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.524},"last":1.002}]},{"osd":7,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.81399999999999995,"15min":0.80100000000000005},"min":{"1min":0.59099999999999997,"5min":0.54900000000000004,"15min":0.505},"max":{"1min":1.1850000000000001,"5min":1.296,"15min":1.319},"last":0.874},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.81000000000000005,"15min":0.79700000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.214,"5min":1.214,"15min":2},"last":0.90800000000000003}]}]},{"osd":1,"up_from":15,"seq":64424509579,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2076672,"data_stored":1234689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45800000000000002,"15min":0.53200000000000003},"min":{"1min":0.28699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.69699999999999995,"5min":0.75900000000000001,"15min":4.2539999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48299999999999998,"15min":0.56399999999999995},"min":{"1min":0.29399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":4.1909999999999998},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.31900000000000001},"max":{"1min":0.63600000000000001,"5min":0.71099999999999997,"15min":0.82899999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.34599999999999997,"15min":0.33600000000000002},"max":{"1min":0.67000000000000004,"5min":0.77700000000000002,"15min":0.88200000000000001},"last":0.51900000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.35499999999999998},"max":{"1min":0.75700000000000001,"5min":0.88300000000000001,"15min":1.52},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.496,"5min":0.51900000000000002,"15min":0.52400000000000002},"min":{"1min":0.35999999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68999999999999995,"5min":0.94499999999999995,"15min":1.331},"last":0.44}]},{"osd":4,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.75600000000000001},"min":{"1min":0.60199999999999998,"5min":0.51800000000000002,"15min":0.46200000000000002},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.77600000000000002},"min":{"1min":0.56599999999999995,"5min":0.502,"15min":0.47999999999999998},"max":{"1min":1.212,"5min":1.212,"15min":1.23},"last":0.80100000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79600000000000004,"15min":0.77900000000000003},"min":{"1min":0.59699999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.76800000000000002},"min":{"1min":0.55200000000000005,"5min":0.55100000000000005,"15min":0.46500000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.2509999999999999},"last":0.81200000000000006}]},{"osd":6,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.79300000000000004,"15min":0.77300000000000002},"min":{"1min":0.58299999999999996,"5min":0.54900000000000004,"15min":0.5},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.81499999999999995,"15min":0.82399999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51300000000000001},"max":{"1min":1.0209999999999999,"5min":1.22,"15min":1.22},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.52100000000000002},"max":{"1min":1.1499999999999999,"5min":1.2410000000000001,"15min":1.3089999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.80700000000000005,"15min":0.79000000000000004},"min":{"1min":0.622,"5min":0.503,"15min":0.503},"max":{"1min":1.1759999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.89800000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509578,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1490944,"data_stored":644973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.44400000000000001,"15min":0.51900000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.22800000000000001},"max":{"1min":0.62,"5min":0.75800000000000001,"15min":3.9359999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.49299999999999999,"15min":0.66700000000000004},"min":{"1min":0.31900000000000001,"5min":0.28499999999999998,"15min":0.253},"max":{"1min":0.72599999999999998,"5min":0.77000000000000002,"15min":8.0419999999999998},"last":0.34300000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50900000000000001,"15min":0.52700000000000002},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34599999999999997},"max":{"1min":0.68500000000000005,"5min":0.78000000000000003,"15min":1.258},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54500000000000004,"15min":0.63300000000000001},"min":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.312},"max":{"1min":0.77400000000000002,"5min":0.81399999999999995,"15min":4.907},"last":0.44}]},{"osd":3,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.501,"15min":0.66900000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.28499999999999998},"max":{"1min":0.65100000000000002,"5min":0.83399999999999996,"15min":7.9569999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.61399999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.34799999999999998},"max":{"1min":0.80100000000000005,"5min":0.87,"15min":1.397},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79500000000000004,"15min":0.88500000000000001},"min":{"1min":0.56799999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1020000000000001,"5min":1.4399999999999999,"15min":4.4720000000000004},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.79400000000000004,"15min":0.86799999999999999},"min":{"1min":0.51500000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.141,"5min":1.153,"15min":4.3920000000000003},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.85499999999999998},"min":{"1min":0.56699999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.014,"5min":1.349,"15min":4.3200000000000003},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80300000000000005,"15min":0.82499999999999996},"min":{"1min":0.59599999999999997,"5min":0.56000000000000005,"15min":0.54600000000000004},"max":{"1min":1.081,"5min":1.26,"15min":1.6759999999999999},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.79800000000000004,"15min":0.82699999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.54400000000000004},"max":{"1min":1.0640000000000001,"5min":1.1579999999999999,"15min":1.5920000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81599999999999995,"15min":0.89300000000000002},"min":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.165,"5min":1.1990000000000001,"15min":4.4100000000000001},"last":0.85499999999999998}]},{"osd":7,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82399999999999995,"15min":0.90600000000000003},"min":{"1min":0.61499999999999999,"5min":0.54100000000000004,"15min":0.496},"max":{"1min":1.141,"5min":1.3959999999999999,"15min":4.5209999999999999},"last":0.752},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80500000000000005,"15min":0.88900000000000001},"min":{"1min":0.56499999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.089,"5min":1.2390000000000001,"15min":4.4420000000000002},"last":0.69999999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542283,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":2064384,"data_stored":1227914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70899999999999996,"15min":0.66400000000000003},"min":{"1min":0.47499999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.98499999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72999999999999998,"15min":0.68400000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.39800000000000002},"max":{"1min":0.84799999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64100000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70699999999999996,"15min":0.71199999999999997},"min":{"1min":0.48499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.008,"5min":1.008,"15min":2.5129999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74299999999999999,"15min":0.70299999999999996},"min":{"1min":0.56799999999999995,"5min":0.51300000000000001,"15min":0.40500000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.85599999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71799999999999997,"15min":0.68600000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92800000000000005,"5min":0.997,"15min":0.997},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.747,"15min":0.71799999999999997},"min":{"1min":0.60899999999999999,"5min":0.47499999999999998,"15min":0.39200000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.77700000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.755,"15min":0.72599999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.47099999999999997},"max":{"1min":1.02,"5min":1.038,"15min":1.038},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.752,"15min":0.74099999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.435},"max":{"1min":0.995,"5min":1.0209999999999999,"15min":1.954},"last":0.81200000000000006}]},{"osd":4,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.624,"15min":0.64000000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.91200000000000003,"15min":0.93500000000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.373},"max":{"1min":0.83299999999999996,"5min":0.94699999999999995,"15min":1.0820000000000001},"last":0.82699999999999996}]},{"osd":6,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.621,"15min":0.65000000000000002},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85999999999999999,"5min":0.93300000000000005,"15min":1.071},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.68500000000000005},"min":{"1min":0.53200000000000003,"5min":0.45300000000000001,"15min":0.432},"max":{"1min":0.92000000000000004,"5min":0.94299999999999995,"15min":1.0609999999999999},"last":0.89800000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90100000000000002,"5min":0.92200000000000004,"15min":1.1200000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70099999999999996,"15min":0.72899999999999998},"min":{"1min":0.46800000000000003,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.94999999999999996,"5min":0.96499999999999997,"15min":1.04},"last":0.98399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:29:18.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:29:19.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.141+0000 7f999d600640 1 Processor -- start 2024-03-29T08:29:19.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.141+0000 7f999d600640 1 -- start start 2024-03-29T08:29:19.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.145+0000 7f999d600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f99981593d0 0x7f999806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.145+0000 7f999d600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f999806d130 con 0x7f999815d230 2024-03-29T08:29:19.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.145+0000 7f999d600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f999815b8f0 con 0x7f999815cd90 2024-03-29T08:29:19.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.145+0000 7f999d600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f999815ad50 con 0x7f99981593d0 2024-03-29T08:29:19.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.145+0000 7f998e000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f999815d230 0x7f999815d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48200/0 (socket says 172.21.15.46:48200) 2024-03-29T08:29:19.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.145+0000 7f998e000640 1 -- 172.21.15.46:0/1249618934 learned_addr learned my addr 172.21.15.46:0/1249618934 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:19.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.145+0000 7f9996a00640 1 --2- 172.21.15.46:0/1249618934 >> v2:172.21.15.73:3300/0 conn(0x7f99981593d0 0x7f999806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.145+0000 7f9996000640 1 -- 172.21.15.46:0/1249618934 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438199321 0 0) 0x7f999806d130 con 0x7f999815d230 2024-03-29T08:29:19.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.145+0000 7f9996000640 1 -- 172.21.15.46:0/1249618934 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f997c005740 con 0x7f999815d230 2024-03-29T08:29:19.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.145+0000 7f9996000640 1 -- 172.21.15.46:0/1249618934 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296866258 0 0) 0x7f999815b8f0 con 0x7f999815cd90 2024-03-29T08:29:19.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.145+0000 7f9996000640 1 -- 172.21.15.46:0/1249618934 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f999806d130 con 0x7f999815cd90 2024-03-29T08:29:19.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.145+0000 7f9996a00640 1 -- 172.21.15.46:0/1249618934 >> v1:172.21.15.73:6789/0 conn(0x7f999815cd90 legacy=0x7f99981597b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.145+0000 7f9996a00640 1 -- 172.21.15.46:0/1249618934 >> v1:172.21.15.46:6789/0 conn(0x7f999815d230 legacy=0x7f999815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.145+0000 7f9996a00640 1 -- 172.21.15.46:0/1249618934 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999815afc0 con 0x7f99981593d0 2024-03-29T08:29:19.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.149+0000 7f9996a00640 1 --2- 172.21.15.46:0/1249618934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99981593d0 0x7f999806cbf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f99880097c0 tx=0x7f998801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb47e66eedb63661 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:19.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.149+0000 7f9996000640 1 -- 172.21.15.46:0/1249618934 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99880a6040 con 0x7f99981593d0 2024-03-29T08:29:19.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.149+0000 7f9996000640 1 -- 172.21.15.46:0/1249618934 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99880aa600 con 0x7f99981593d0 2024-03-29T08:29:19.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.149+0000 7f9996000640 1 -- 172.21.15.46:0/1249618934 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99880aa990 con 0x7f99981593d0 2024-03-29T08:29:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.149+0000 7f999d600640 1 -- 172.21.15.46:0/1249618934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99981593d0 msgr2=0x7f999806cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.149+0000 7f999d600640 1 --2- 172.21.15.46:0/1249618934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99981593d0 0x7f999806cbf0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f99880097c0 tx=0x7f998801c4a0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.149+0000 7f999d600640 1 -- 172.21.15.46:0/1249618934 shutdown_connections 2024-03-29T08:29:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.149+0000 7f999d600640 1 --2- 172.21.15.46:0/1249618934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99981593d0 0x7f999806cbf0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.149+0000 7f999d600640 1 -- 172.21.15.46:0/1249618934 >> 172.21.15.46:0/1249618934 conn(0x7f9998066110 msgr2=0x7f9998069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.149+0000 7f999d600640 1 -- 172.21.15.46:0/1249618934 shutdown_connections 2024-03-29T08:29:19.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.149+0000 7f999d600640 1 -- 172.21.15.46:0/1249618934 wait complete. 2024-03-29T08:29:19.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.149+0000 7f999d600640 1 Processor -- start 2024-03-29T08:29:19.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.149+0000 7f999d600640 1 -- start start 2024-03-29T08:29:19.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f999d600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99981593d0 0x7f9998158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f999d600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f999815cd90 0x7f999814b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f999d600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999815d230 0x7f9998154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f999d600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9998153c40 con 0x7f99981593d0 2024-03-29T08:29:19.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f999d600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9998157720 con 0x7f999815d230 2024-03-29T08:29:19.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f999d600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9998157a00 con 0x7f999815cd90 2024-03-29T08:29:19.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f9996a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99981593d0 0x7f9998158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f9996a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99981593d0 0x7f9998158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39056/0 (socket says 172.21.15.46:39056) 2024-03-29T08:29:19.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f9996a00640 1 -- 172.21.15.46:0/1958513958 learned_addr learned my addr 172.21.15.46:0/1958513958 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:19.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f998e000640 1 --2- 172.21.15.46:0/1958513958 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f999815cd90 0x7f999814b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f9997400640 1 --2- 172.21.15.46:0/1958513958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999815d230 0x7f9998154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f9996a00640 1 -- 172.21.15.46:0/1958513958 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f999815cd90 msgr2=0x7f999814b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f9996a00640 1 --2- 172.21.15.46:0/1958513958 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f999815cd90 0x7f999814b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f9996a00640 1 -- 172.21.15.46:0/1958513958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999815d230 msgr2=0x7f9998154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f9996a00640 1 --2- 172.21.15.46:0/1958513958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999815d230 0x7f9998154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f9996a00640 1 -- 172.21.15.46:0/1958513958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9988009420 con 0x7f99981593d0 2024-03-29T08:29:19.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f9997400640 1 --2- 172.21.15.46:0/1958513958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999815d230 0x7f9998154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:19.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f9996a00640 1 --2- 172.21.15.46:0/1958513958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99981593d0 0x7f9998158670 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f99880190b0 tx=0x7f998801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:19.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f998f400640 1 -- 172.21.15.46:0/1958513958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9988006e20 con 0x7f99981593d0 2024-03-29T08:29:19.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f998f400640 1 -- 172.21.15.46:0/1958513958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99880b3600 con 0x7f99981593d0 2024-03-29T08:29:19.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f998f400640 1 -- 172.21.15.46:0/1958513958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99880b3990 con 0x7f99981593d0 2024-03-29T08:29:19.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f999d600640 1 -- 172.21.15.46:0/1958513958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9998153e90 con 0x7f99981593d0 2024-03-29T08:29:19.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f999d600640 1 -- 172.21.15.46:0/1958513958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999814b630 con 0x7f99981593d0 2024-03-29T08:29:19.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f999d600640 1 -- 172.21.15.46:0/1958513958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9998157bc0 con 0x7f99981593d0 2024-03-29T08:29:19.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.153+0000 7f998f400640 1 -- 172.21.15.46:0/1958513958 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f99880b3af0 con 0x7f99981593d0 2024-03-29T08:29:19.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.157+0000 7f998f400640 1 --2- 172.21.15.46:0/1958513958 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99740666c0 0x7f9974068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.157+0000 7f998f400640 1 -- 172.21.15.46:0/1958513958 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f998811dcc0 con 0x7f99981593d0 2024-03-29T08:29:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.157+0000 7f998e000640 1 --2- 172.21.15.46:0/1958513958 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99740666c0 0x7f9974068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.157+0000 7f998e000640 1 --2- 172.21.15.46:0/1958513958 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99740666c0 0x7f9974068b80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f99981f8850 tx=0x7f9980007d80 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:19.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.161+0000 7f998f400640 1 -- 172.21.15.46:0/1958513958 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9988124080 con 0x7f99981593d0 2024-03-29T08:29:19.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.317+0000 7f999d600640 1 -- 172.21.15.46:0/1958513958 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f99980701c0 con 0x7f99740666c0 2024-03-29T08:29:19.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.325+0000 7f998f400640 1 -- 172.21.15.46:0/1958513958 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289105 (secure 0 0 0) 0x7f9980001470 con 0x7f99740666c0 2024-03-29T08:29:19.333 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:19.335 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:29:19.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.329+0000 7f999d600640 1 -- 172.21.15.46:0/1958513958 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99740666c0 msgr2=0x7f9974068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.329+0000 7f999d600640 1 --2- 172.21.15.46:0/1958513958 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99740666c0 0x7f9974068b80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f99981f8850 tx=0x7f9980007d80 comp rx=0 tx=0).stop 2024-03-29T08:29:19.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.329+0000 7f999d600640 1 -- 172.21.15.46:0/1958513958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99981593d0 msgr2=0x7f9998158670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.329+0000 7f999d600640 1 --2- 172.21.15.46:0/1958513958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99981593d0 0x7f9998158670 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f99880190b0 tx=0x7f998801ace0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.329+0000 7f999d600640 1 -- 172.21.15.46:0/1958513958 shutdown_connections 2024-03-29T08:29:19.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.329+0000 7f999d600640 1 --2- 172.21.15.46:0/1958513958 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f99740666c0 0x7f9974068b80 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.329+0000 7f999d600640 1 --2- 172.21.15.46:0/1958513958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f999815d230 0x7f9998154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.329+0000 7f999d600640 1 --2- 172.21.15.46:0/1958513958 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f999815cd90 0x7f999814b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.329+0000 7f999d600640 1 --2- 172.21.15.46:0/1958513958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99981593d0 0x7f9998158670 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.329+0000 7f999d600640 1 -- 172.21.15.46:0/1958513958 >> 172.21.15.46:0/1958513958 conn(0x7f9998066110 msgr2=0x7f99980705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:19.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.329+0000 7f999d600640 1 -- 172.21.15.46:0/1958513958 shutdown_connections 2024-03-29T08:29:19.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.329+0000 7f999d600640 1 -- 172.21.15.46:0/1958513958 wait complete. 2024-03-29T08:29:19.353 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-03-29T08:29:18.211688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261740,"kb_used_data":12764,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754712980,"statfs":{"total":773094113280,"available":772826091520,"internally_reserved":0,"allocated":13070336,"data_stored":6325748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.148834"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509634+0000","last_change":"2024-03-29T08:29:14.504298+0000","last_active":"2024-03-29T08:29:14.509634+0000","last_peered":"2024-03-29T08:29:14.509634+0000","last_clean":"2024-03-29T08:29:14.509634+0000","last_became_active":"2024-03-29T08:29:14.504060+0000","last_became_peered":"2024-03-29T08:29:14.504060+0000","last_unstale":"2024-03-29T08:29:14.509634+0000","last_undegraded":"2024-03-29T08:29:14.509634+0000","last_fullsized":"2024-03-29T08:29:14.509634+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:47:24.542958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.491055+0000","last_change":"2024-03-29T08:29:14.488152+0000","last_active":"2024-03-29T08:29:14.491055+0000","last_peered":"2024-03-29T08:29:14.491055+0000","last_clean":"2024-03-29T08:29:14.491055+0000","last_became_active":"2024-03-29T08:29:14.487884+0000","last_became_peered":"2024-03-29T08:29:14.487884+0000","last_unstale":"2024-03-29T08:29:14.491055+0000","last_undegraded":"2024-03-29T08:29:14.491055+0000","last_fullsized":"2024-03-29T08:29:14.491055+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:49:14.237508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.492505+0000","last_change":"2024-03-29T08:29:14.490304+0000","last_active":"2024-03-29T08:29:14.492505+0000","last_peered":"2024-03-29T08:29:14.492505+0000","last_clean":"2024-03-29T08:29:14.492505+0000","last_became_active":"2024-03-29T08:29:14.489916+0000","last_became_peered":"2024-03-29T08:29:14.489916+0000","last_unstale":"2024-03-29T08:29:14.492505+0000","last_undegraded":"2024-03-29T08:29:14.492505+0000","last_fullsized":"2024-03-29T08:29:14.492505+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:55.868596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.510337+0000","last_change":"2024-03-29T08:29:14.501687+0000","last_active":"2024-03-29T08:29:14.510337+0000","last_peered":"2024-03-29T08:29:14.510337+0000","last_clean":"2024-03-29T08:29:14.510337+0000","last_became_active":"2024-03-29T08:29:14.501377+0000","last_became_peered":"2024-03-29T08:29:14.501377+0000","last_unstale":"2024-03-29T08:29:14.510337+0000","last_undegraded":"2024-03-29T08:29:14.510337+0000","last_fullsized":"2024-03-29T08:29:14.510337+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:38:29.985399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.506541+0000","last_change":"2024-03-29T08:29:14.504615+0000","last_active":"2024-03-29T08:29:14.506541+0000","last_peered":"2024-03-29T08:29:14.506541+0000","last_clean":"2024-03-29T08:29:14.506541+0000","last_became_active":"2024-03-29T08:29:14.504347+0000","last_became_peered":"2024-03-29T08:29:14.504347+0000","last_unstale":"2024-03-29T08:29:14.506541+0000","last_undegraded":"2024-03-29T08:29:14.506541+0000","last_fullsized":"2024-03-29T08:29:14.506541+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:25:45.763218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.084660+0000","last_change":"2024-03-29T08:29:14.084660+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.084660+0000","last_undegraded":"2024-03-29T08:29:14.084660+0000","last_fullsized":"2024-03-29T08:29:14.084660+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:51.101298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.503717+0000","last_change":"2024-03-29T08:29:14.494643+0000","last_active":"2024-03-29T08:29:14.503717+0000","last_peered":"2024-03-29T08:29:14.503717+0000","last_clean":"2024-03-29T08:29:14.503717+0000","last_became_active":"2024-03-29T08:29:14.494385+0000","last_became_peered":"2024-03-29T08:29:14.494385+0000","last_unstale":"2024-03-29T08:29:14.503717+0000","last_undegraded":"2024-03-29T08:29:14.503717+0000","last_fullsized":"2024-03-29T08:29:14.503717+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:11:26.639183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.495613+0000","last_change":"2024-03-29T08:29:14.489843+0000","last_active":"2024-03-29T08:29:14.495613+0000","last_peered":"2024-03-29T08:29:14.495613+0000","last_clean":"2024-03-29T08:29:14.495613+0000","last_became_active":"2024-03-29T08:29:14.489598+0000","last_became_peered":"2024-03-29T08:29:14.489598+0000","last_unstale":"2024-03-29T08:29:14.495613+0000","last_undegraded":"2024-03-29T08:29:14.495613+0000","last_fullsized":"2024-03-29T08:29:14.495613+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:49:19.274668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.512231+0000","last_change":"2024-03-29T08:29:14.507881+0000","last_active":"2024-03-29T08:29:14.512231+0000","last_peered":"2024-03-29T08:29:14.512231+0000","last_clean":"2024-03-29T08:29:14.512231+0000","last_became_active":"2024-03-29T08:29:14.507445+0000","last_became_peered":"2024-03-29T08:29:14.507445+0000","last_unstale":"2024-03-29T08:29:14.512231+0000","last_undegraded":"2024-03-29T08:29:14.512231+0000","last_fullsized":"2024-03-29T08:29:14.512231+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:50:39.114131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504217+0000","last_change":"2024-03-29T08:29:14.500130+0000","last_active":"2024-03-29T08:29:14.504217+0000","last_peered":"2024-03-29T08:29:14.504217+0000","last_clean":"2024-03-29T08:29:14.504217+0000","last_became_active":"2024-03-29T08:29:14.499827+0000","last_became_peered":"2024-03-29T08:29:14.499827+0000","last_unstale":"2024-03-29T08:29:14.504217+0000","last_undegraded":"2024-03-29T08:29:14.504217+0000","last_fullsized":"2024-03-29T08:29:14.504217+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:01.476642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.502452+0000","last_change":"2024-03-29T08:29:14.494953+0000","last_active":"2024-03-29T08:29:14.502452+0000","last_peered":"2024-03-29T08:29:14.502452+0000","last_clean":"2024-03-29T08:29:14.502452+0000","last_became_active":"2024-03-29T08:29:14.494599+0000","last_became_peered":"2024-03-29T08:29:14.494599+0000","last_unstale":"2024-03-29T08:29:14.502452+0000","last_undegraded":"2024-03-29T08:29:14.502452+0000","last_fullsized":"2024-03-29T08:29:14.502452+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:49:02.417397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504246+0000","last_change":"2024-03-29T08:29:14.499160+0000","last_active":"2024-03-29T08:29:14.504246+0000","last_peered":"2024-03-29T08:29:14.504246+0000","last_clean":"2024-03-29T08:29:14.504246+0000","last_became_active":"2024-03-29T08:29:14.498900+0000","last_became_peered":"2024-03-29T08:29:14.498900+0000","last_unstale":"2024-03-29T08:29:14.504246+0000","last_undegraded":"2024-03-29T08:29:14.504246+0000","last_fullsized":"2024-03-29T08:29:14.504246+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:19.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.493310+0000","last_change":"2024-03-29T08:29:14.486530+0000","last_active":"2024-03-29T08:29:14.493310+0000","last_peered":"2024-03-29T08:29:14.493310+0000","last_clean":"2024-03-29T08:29:14.493310+0000","last_became_active":"2024-03-29T08:29:14.486158+0000","last_became_peered":"2024-03-29T08:29:14.486158+0000","last_unstale":"2024-03-29T08:29:14.493310+0000","last_undegraded":"2024-03-29T08:29:14.493310+0000","last_fullsized":"2024-03-29T08:29:14.493310+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:40.282997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508861+0000","last_change":"2024-03-29T08:29:14.507343+0000","last_active":"2024-03-29T08:29:14.508861+0000","last_peered":"2024-03-29T08:29:14.508861+0000","last_clean":"2024-03-29T08:29:14.508861+0000","last_became_active":"2024-03-29T08:29:14.507072+0000","last_became_peered":"2024-03-29T08:29:14.507072+0000","last_unstale":"2024-03-29T08:29:14.508861+0000","last_undegraded":"2024-03-29T08:29:14.508861+0000","last_fullsized":"2024-03-29T08:29:14.508861+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:19.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509523+0000","last_change":"2024-03-29T08:29:14.498131+0000","last_active":"2024-03-29T08:29:14.509523+0000","last_peered":"2024-03-29T08:29:14.509523+0000","last_clean":"2024-03-29T08:29:14.509523+0000","last_became_active":"2024-03-29T08:29:14.497856+0000","last_became_peered":"2024-03-29T08:29:14.497856+0000","last_unstale":"2024-03-29T08:29:14.509523+0000","last_undegraded":"2024-03-29T08:29:14.509523+0000","last_fullsized":"2024-03-29T08:29:14.509523+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:43:24.083895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504685+0000","last_change":"2024-03-29T08:29:14.499902+0000","last_active":"2024-03-29T08:29:14.504685+0000","last_peered":"2024-03-29T08:29:14.504685+0000","last_clean":"2024-03-29T08:29:14.504685+0000","last_became_active":"2024-03-29T08:29:14.499642+0000","last_became_peered":"2024-03-29T08:29:14.499642+0000","last_unstale":"2024-03-29T08:29:14.504685+0000","last_undegraded":"2024-03-29T08:29:14.504685+0000","last_fullsized":"2024-03-29T08:29:14.504685+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:06:54.374100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.502730+0000","last_change":"2024-03-29T08:29:14.496277+0000","last_active":"2024-03-29T08:29:14.502730+0000","last_peered":"2024-03-29T08:29:14.502730+0000","last_clean":"2024-03-29T08:29:14.502730+0000","last_became_active":"2024-03-29T08:29:14.496028+0000","last_became_peered":"2024-03-29T08:29:14.496028+0000","last_unstale":"2024-03-29T08:29:14.502730+0000","last_undegraded":"2024-03-29T08:29:14.502730+0000","last_fullsized":"2024-03-29T08:29:14.502730+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:08:59.334138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505390+0000","last_change":"2024-03-29T08:29:14.503103+0000","last_active":"2024-03-29T08:29:14.505390+0000","last_peered":"2024-03-29T08:29:14.505390+0000","last_clean":"2024-03-29T08:29:14.505390+0000","last_became_active":"2024-03-29T08:29:14.502841+0000","last_became_peered":"2024-03-29T08:29:14.502841+0000","last_unstale":"2024-03-29T08:29:14.505390+0000","last_undegraded":"2024-03-29T08:29:14.505390+0000","last_fullsized":"2024-03-29T08:29:14.505390+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:11:21.620572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.498334+0000","last_change":"2024-03-29T08:29:14.491176+0000","last_active":"2024-03-29T08:29:14.498334+0000","last_peered":"2024-03-29T08:29:14.498334+0000","last_clean":"2024-03-29T08:29:14.498334+0000","last_became_active":"2024-03-29T08:29:14.490845+0000","last_became_peered":"2024-03-29T08:29:14.490845+0000","last_unstale":"2024-03-29T08:29:14.498334+0000","last_undegraded":"2024-03-29T08:29:14.498334+0000","last_fullsized":"2024-03-29T08:29:14.498334+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:44:41.312640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504227+0000","last_change":"2024-03-29T08:29:14.501383+0000","last_active":"2024-03-29T08:29:14.504227+0000","last_peered":"2024-03-29T08:29:14.504227+0000","last_clean":"2024-03-29T08:29:14.504227+0000","last_became_active":"2024-03-29T08:29:14.501106+0000","last_became_peered":"2024-03-29T08:29:14.501106+0000","last_unstale":"2024-03-29T08:29:14.504227+0000","last_undegraded":"2024-03-29T08:29:14.504227+0000","last_fullsized":"2024-03-29T08:29:14.504227+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:23:10.605671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504862+0000","last_change":"2024-03-29T08:29:14.497743+0000","last_active":"2024-03-29T08:29:14.504862+0000","last_peered":"2024-03-29T08:29:14.504862+0000","last_clean":"2024-03-29T08:29:14.504862+0000","last_became_active":"2024-03-29T08:29:14.497338+0000","last_became_peered":"2024-03-29T08:29:14.497338+0000","last_unstale":"2024-03-29T08:29:14.504862+0000","last_undegraded":"2024-03-29T08:29:14.504862+0000","last_fullsized":"2024-03-29T08:29:14.504862+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:24.513091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.084054+0000","last_change":"2024-03-29T08:29:14.084054+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.084054+0000","last_undegraded":"2024-03-29T08:29:14.084054+0000","last_fullsized":"2024-03-29T08:29:14.084054+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:57:23.884319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.506608+0000","last_change":"2024-03-29T08:29:14.502935+0000","last_active":"2024-03-29T08:29:14.506608+0000","last_peered":"2024-03-29T08:29:14.506608+0000","last_clean":"2024-03-29T08:29:14.506608+0000","last_became_active":"2024-03-29T08:29:14.502701+0000","last_became_peered":"2024-03-29T08:29:14.502701+0000","last_unstale":"2024-03-29T08:29:14.506608+0000","last_undegraded":"2024-03-29T08:29:14.506608+0000","last_fullsized":"2024-03-29T08:29:14.506608+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:35.756569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.511647+0000","last_change":"2024-03-29T08:29:14.507544+0000","last_active":"2024-03-29T08:29:14.511647+0000","last_peered":"2024-03-29T08:29:14.511647+0000","last_clean":"2024-03-29T08:29:14.511647+0000","last_became_active":"2024-03-29T08:29:14.507285+0000","last_became_peered":"2024-03-29T08:29:14.507285+0000","last_unstale":"2024-03-29T08:29:14.511647+0000","last_undegraded":"2024-03-29T08:29:14.511647+0000","last_fullsized":"2024-03-29T08:29:14.511647+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:50:14.687728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.494822+0000","last_change":"2024-03-29T08:29:14.492853+0000","last_active":"2024-03-29T08:29:14.494822+0000","last_peered":"2024-03-29T08:29:14.494822+0000","last_clean":"2024-03-29T08:29:14.494822+0000","last_became_active":"2024-03-29T08:29:14.492537+0000","last_became_peered":"2024-03-29T08:29:14.492537+0000","last_unstale":"2024-03-29T08:29:14.494822+0000","last_undegraded":"2024-03-29T08:29:14.494822+0000","last_fullsized":"2024-03-29T08:29:14.494822+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:08:28.043449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.085680+0000","last_change":"2024-03-29T08:29:14.085680+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.085680+0000","last_undegraded":"2024-03-29T08:29:14.085680+0000","last_fullsized":"2024-03-29T08:29:14.085680+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:10:44.097211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.511774+0000","last_change":"2024-03-29T08:29:14.508869+0000","last_active":"2024-03-29T08:29:14.511774+0000","last_peered":"2024-03-29T08:29:14.511774+0000","last_clean":"2024-03-29T08:29:14.511774+0000","last_became_active":"2024-03-29T08:29:14.508466+0000","last_became_peered":"2024-03-29T08:29:14.508466+0000","last_unstale":"2024-03-29T08:29:14.511774+0000","last_undegraded":"2024-03-29T08:29:14.511774+0000","last_fullsized":"2024-03-29T08:29:14.511774+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:08:28.619413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.087721+0000","last_change":"2024-03-29T08:29:14.087721+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.087721+0000","last_undegraded":"2024-03-29T08:29:14.087721+0000","last_fullsized":"2024-03-29T08:29:14.087721+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:53.749284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508906+0000","last_change":"2024-03-29T08:29:14.507369+0000","last_active":"2024-03-29T08:29:14.508906+0000","last_peered":"2024-03-29T08:29:14.508906+0000","last_clean":"2024-03-29T08:29:14.508906+0000","last_became_active":"2024-03-29T08:29:14.507164+0000","last_became_peered":"2024-03-29T08:29:14.507164+0000","last_unstale":"2024-03-29T08:29:14.508906+0000","last_undegraded":"2024-03-29T08:29:14.508906+0000","last_fullsized":"2024-03-29T08:29:14.508906+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:20:13.956872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505661+0000","last_change":"2024-03-29T08:29:14.503526+0000","last_active":"2024-03-29T08:29:14.505661+0000","last_peered":"2024-03-29T08:29:14.505661+0000","last_clean":"2024-03-29T08:29:14.505661+0000","last_became_active":"2024-03-29T08:29:14.503149+0000","last_became_peered":"2024-03-29T08:29:14.503149+0000","last_unstale":"2024-03-29T08:29:14.505661+0000","last_undegraded":"2024-03-29T08:29:14.505661+0000","last_fullsized":"2024-03-29T08:29:14.505661+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:46:13.948298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509541+0000","last_change":"2024-03-29T08:29:14.500403+0000","last_active":"2024-03-29T08:29:14.509541+0000","last_peered":"2024-03-29T08:29:14.509541+0000","last_clean":"2024-03-29T08:29:14.509541+0000","last_became_active":"2024-03-29T08:29:14.500132+0000","last_became_peered":"2024-03-29T08:29:14.500132+0000","last_unstale":"2024-03-29T08:29:14.509541+0000","last_undegraded":"2024-03-29T08:29:14.509541+0000","last_fullsized":"2024-03-29T08:29:14.509541+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:09:34.129852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504668+0000","last_change":"2024-03-29T08:29:14.499479+0000","last_active":"2024-03-29T08:29:14.504668+0000","last_peered":"2024-03-29T08:29:14.504668+0000","last_clean":"2024-03-29T08:29:14.504668+0000","last_became_active":"2024-03-29T08:29:14.498994+0000","last_became_peered":"2024-03-29T08:29:14.498994+0000","last_unstale":"2024-03-29T08:29:14.504668+0000","last_undegraded":"2024-03-29T08:29:14.504668+0000","last_fullsized":"2024-03-29T08:29:14.504668+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:25:13.598107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.513004+0000","last_change":"2024-03-29T08:29:14.511693+0000","last_active":"2024-03-29T08:29:14.513004+0000","last_peered":"2024-03-29T08:29:14.513004+0000","last_clean":"2024-03-29T08:29:14.513004+0000","last_became_active":"2024-03-29T08:29:14.511041+0000","last_became_peered":"2024-03-29T08:29:14.511041+0000","last_unstale":"2024-03-29T08:29:14.513004+0000","last_undegraded":"2024-03-29T08:29:14.513004+0000","last_fullsized":"2024-03-29T08:29:14.513004+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:06.874362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"120'4","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507837+0000","last_change":"2024-03-29T08:29:14.086060+0000","last_active":"2024-03-29T08:29:14.507837+0000","last_peered":"2024-03-29T08:29:14.507837+0000","last_clean":"2024-03-29T08:29:14.507837+0000","last_became_active":"2024-03-29T08:29:14.085788+0000","last_became_peered":"2024-03-29T08:29:14.085788+0000","last_unstale":"2024-03-29T08:29:14.507837+0000","last_undegraded":"2024-03-29T08:29:14.507837+0000","last_fullsized":"2024-03-29T08:29:14.507837+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:52:04.161387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507246+0000","last_change":"2024-03-29T08:29:14.503865+0000","last_active":"2024-03-29T08:29:14.507246+0000","last_peered":"2024-03-29T08:29:14.507246+0000","last_clean":"2024-03-29T08:29:14.507246+0000","last_became_active":"2024-03-29T08:29:14.503622+0000","last_became_peered":"2024-03-29T08:29:14.503622+0000","last_unstale":"2024-03-29T08:29:14.507246+0000","last_undegraded":"2024-03-29T08:29:14.507246+0000","last_fullsized":"2024-03-29T08:29:14.507246+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:48:23.465613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.501135+0000","last_change":"2024-03-29T08:29:14.084881+0000","last_active":"2024-03-29T08:29:14.501135+0000","last_peered":"2024-03-29T08:29:14.501135+0000","last_clean":"2024-03-29T08:29:14.501135+0000","last_became_active":"2024-03-29T08:29:14.084587+0000","last_became_peered":"2024-03-29T08:29:14.084587+0000","last_unstale":"2024-03-29T08:29:14.501135+0000","last_undegraded":"2024-03-29T08:29:14.501135+0000","last_fullsized":"2024-03-29T08:29:14.501135+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:45.558553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.085712+0000","last_change":"2024-03-29T08:29:14.085712+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.085712+0000","last_undegraded":"2024-03-29T08:29:14.085712+0000","last_fullsized":"2024-03-29T08:29:14.085712+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:23:32.467222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509018+0000","last_change":"2024-03-29T08:29:14.089964+0000","last_active":"2024-03-29T08:29:14.509018+0000","last_peered":"2024-03-29T08:29:14.509018+0000","last_clean":"2024-03-29T08:29:14.509018+0000","last_became_active":"2024-03-29T08:29:14.088099+0000","last_became_peered":"2024-03-29T08:29:14.088099+0000","last_unstale":"2024-03-29T08:29:14.509018+0000","last_undegraded":"2024-03-29T08:29:14.509018+0000","last_fullsized":"2024-03-29T08:29:14.509018+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:05.868646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.498316+0000","last_change":"2024-03-29T08:29:14.491102+0000","last_active":"2024-03-29T08:29:14.498316+0000","last_peered":"2024-03-29T08:29:14.498316+0000","last_clean":"2024-03-29T08:29:14.498316+0000","last_became_active":"2024-03-29T08:29:14.490878+0000","last_became_peered":"2024-03-29T08:29:14.490878+0000","last_unstale":"2024-03-29T08:29:14.498316+0000","last_undegraded":"2024-03-29T08:29:14.498316+0000","last_fullsized":"2024-03-29T08:29:14.498316+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:06:54.374100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509548+0000","last_change":"2024-03-29T08:29:14.097684+0000","last_active":"2024-03-29T08:29:14.509548+0000","last_peered":"2024-03-29T08:29:14.509548+0000","last_clean":"2024-03-29T08:29:14.509548+0000","last_became_active":"2024-03-29T08:29:14.095879+0000","last_became_peered":"2024-03-29T08:29:14.095879+0000","last_unstale":"2024-03-29T08:29:14.509548+0000","last_undegraded":"2024-03-29T08:29:14.509548+0000","last_fullsized":"2024-03-29T08:29:14.509548+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:50:38.108416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.511687+0000","last_change":"2024-03-29T08:29:14.507697+0000","last_active":"2024-03-29T08:29:14.511687+0000","last_peered":"2024-03-29T08:29:14.511687+0000","last_clean":"2024-03-29T08:29:14.511687+0000","last_became_active":"2024-03-29T08:29:14.507356+0000","last_became_peered":"2024-03-29T08:29:14.507356+0000","last_unstale":"2024-03-29T08:29:14.511687+0000","last_undegraded":"2024-03-29T08:29:14.511687+0000","last_fullsized":"2024-03-29T08:29:14.511687+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:11.283108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.502762+0000","last_change":"2024-03-29T08:29:14.086806+0000","last_active":"2024-03-29T08:29:14.502762+0000","last_peered":"2024-03-29T08:29:14.502762+0000","last_clean":"2024-03-29T08:29:14.502762+0000","last_became_active":"2024-03-29T08:29:14.086468+0000","last_became_peered":"2024-03-29T08:29:14.086468+0000","last_unstale":"2024-03-29T08:29:14.502762+0000","last_undegraded":"2024-03-29T08:29:14.502762+0000","last_fullsized":"2024-03-29T08:29:14.502762+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:37:57.757695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504878+0000","last_change":"2024-03-29T08:29:14.498041+0000","last_active":"2024-03-29T08:29:14.504878+0000","last_peered":"2024-03-29T08:29:14.504878+0000","last_clean":"2024-03-29T08:29:14.504878+0000","last_became_active":"2024-03-29T08:29:14.497768+0000","last_became_peered":"2024-03-29T08:29:14.497768+0000","last_unstale":"2024-03-29T08:29:14.504878+0000","last_undegraded":"2024-03-29T08:29:14.504878+0000","last_fullsized":"2024-03-29T08:29:14.504878+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:55.518674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504448+0000","last_change":"2024-03-29T08:29:14.088920+0000","last_active":"2024-03-29T08:29:14.504448+0000","last_peered":"2024-03-29T08:29:14.504448+0000","last_clean":"2024-03-29T08:29:14.504448+0000","last_became_active":"2024-03-29T08:29:14.088593+0000","last_became_peered":"2024-03-29T08:29:14.088593+0000","last_unstale":"2024-03-29T08:29:14.504448+0000","last_undegraded":"2024-03-29T08:29:14.504448+0000","last_fullsized":"2024-03-29T08:29:14.504448+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:51.446071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504649+0000","last_change":"2024-03-29T08:29:14.499273+0000","last_active":"2024-03-29T08:29:14.504649+0000","last_peered":"2024-03-29T08:29:14.504649+0000","last_clean":"2024-03-29T08:29:14.504649+0000","last_became_active":"2024-03-29T08:29:14.498368+0000","last_became_peered":"2024-03-29T08:29:14.498368+0000","last_unstale":"2024-03-29T08:29:14.504649+0000","last_undegraded":"2024-03-29T08:29:14.504649+0000","last_fullsized":"2024-03-29T08:29:14.504649+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:23:10.605671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.492779+0000","last_change":"2024-03-29T08:29:14.079054+0000","last_active":"2024-03-29T08:29:14.492779+0000","last_peered":"2024-03-29T08:29:14.492779+0000","last_clean":"2024-03-29T08:29:14.492779+0000","last_became_active":"2024-03-29T08:29:14.078272+0000","last_became_peered":"2024-03-29T08:29:14.078272+0000","last_unstale":"2024-03-29T08:29:14.492779+0000","last_undegraded":"2024-03-29T08:29:14.492779+0000","last_fullsized":"2024-03-29T08:29:14.492779+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:57:54.510656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.503738+0000","last_change":"2024-03-29T08:29:14.495103+0000","last_active":"2024-03-29T08:29:14.503738+0000","last_peered":"2024-03-29T08:29:14.503738+0000","last_clean":"2024-03-29T08:29:14.503738+0000","last_became_active":"2024-03-29T08:29:14.494873+0000","last_became_peered":"2024-03-29T08:29:14.494873+0000","last_unstale":"2024-03-29T08:29:14.503738+0000","last_undegraded":"2024-03-29T08:29:14.503738+0000","last_fullsized":"2024-03-29T08:29:14.503738+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:49:15.540195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.484296+0000","last_change":"2024-03-29T08:29:14.091783+0000","last_active":"2024-03-29T08:29:14.484296+0000","last_peered":"2024-03-29T08:29:14.484296+0000","last_clean":"2024-03-29T08:29:14.484296+0000","last_became_active":"2024-03-29T08:29:14.091531+0000","last_became_peered":"2024-03-29T08:29:14.091531+0000","last_unstale":"2024-03-29T08:29:14.484296+0000","last_undegraded":"2024-03-29T08:29:14.484296+0000","last_fullsized":"2024-03-29T08:29:14.484296+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:32:23.859154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504693+0000","last_change":"2024-03-29T08:29:14.498939+0000","last_active":"2024-03-29T08:29:14.504693+0000","last_peered":"2024-03-29T08:29:14.504693+0000","last_clean":"2024-03-29T08:29:14.504693+0000","last_became_active":"2024-03-29T08:29:14.498589+0000","last_became_peered":"2024-03-29T08:29:14.498589+0000","last_unstale":"2024-03-29T08:29:14.504693+0000","last_undegraded":"2024-03-29T08:29:14.504693+0000","last_fullsized":"2024-03-29T08:29:14.504693+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:32:24.719500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.496023+0000","last_change":"2024-03-29T08:29:14.096474+0000","last_active":"2024-03-29T08:29:14.496023+0000","last_peered":"2024-03-29T08:29:14.496023+0000","last_clean":"2024-03-29T08:29:14.496023+0000","last_became_active":"2024-03-29T08:29:14.094336+0000","last_became_peered":"2024-03-29T08:29:14.094336+0000","last_unstale":"2024-03-29T08:29:14.496023+0000","last_undegraded":"2024-03-29T08:29:14.496023+0000","last_fullsized":"2024-03-29T08:29:14.496023+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:39.277282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.489947+0000","last_change":"2024-03-29T08:29:14.487254+0000","last_active":"2024-03-29T08:29:14.489947+0000","last_peered":"2024-03-29T08:29:14.489947+0000","last_clean":"2024-03-29T08:29:14.489947+0000","last_became_active":"2024-03-29T08:29:14.485472+0000","last_became_peered":"2024-03-29T08:29:14.485472+0000","last_unstale":"2024-03-29T08:29:14.489947+0000","last_undegraded":"2024-03-29T08:29:14.489947+0000","last_fullsized":"2024-03-29T08:29:14.489947+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:08:57.590159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.484431+0000","last_change":"2024-03-29T08:29:14.092400+0000","last_active":"2024-03-29T08:29:14.484431+0000","last_peered":"2024-03-29T08:29:14.484431+0000","last_clean":"2024-03-29T08:29:14.484431+0000","last_became_active":"2024-03-29T08:29:14.091857+0000","last_became_peered":"2024-03-29T08:29:14.091857+0000","last_unstale":"2024-03-29T08:29:14.484431+0000","last_undegraded":"2024-03-29T08:29:14.484431+0000","last_fullsized":"2024-03-29T08:29:14.484431+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:37:57.757695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.085301+0000","last_change":"2024-03-29T08:29:14.085301+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.085301+0000","last_undegraded":"2024-03-29T08:29:14.085301+0000","last_fullsized":"2024-03-29T08:29:14.085301+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:45.863920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"120'3","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509397+0000","last_change":"2024-03-29T08:29:14.092891+0000","last_active":"2024-03-29T08:29:14.509397+0000","last_peered":"2024-03-29T08:29:14.509397+0000","last_clean":"2024-03-29T08:29:14.509397+0000","last_became_active":"2024-03-29T08:29:14.092632+0000","last_became_peered":"2024-03-29T08:29:14.092632+0000","last_unstale":"2024-03-29T08:29:14.509397+0000","last_undegraded":"2024-03-29T08:29:14.509397+0000","last_fullsized":"2024-03-29T08:29:14.509397+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:52:04.161387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.492713+0000","last_change":"2024-03-29T08:29:14.489765+0000","last_active":"2024-03-29T08:29:14.492713+0000","last_peered":"2024-03-29T08:29:14.492713+0000","last_clean":"2024-03-29T08:29:14.492713+0000","last_became_active":"2024-03-29T08:29:14.489440+0000","last_became_peered":"2024-03-29T08:29:14.489440+0000","last_unstale":"2024-03-29T08:29:14.492713+0000","last_undegraded":"2024-03-29T08:29:14.492713+0000","last_fullsized":"2024-03-29T08:29:14.492713+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:53:42.976104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"120'6","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.512807+0000","last_change":"2024-03-29T08:29:14.096725+0000","last_active":"2024-03-29T08:29:14.512807+0000","last_peered":"2024-03-29T08:29:14.512807+0000","last_clean":"2024-03-29T08:29:14.512807+0000","last_became_active":"2024-03-29T08:29:14.094695+0000","last_became_peered":"2024-03-29T08:29:14.094695+0000","last_unstale":"2024-03-29T08:29:14.512807+0000","last_undegraded":"2024-03-29T08:29:14.512807+0000","last_fullsized":"2024-03-29T08:29:14.512807+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:15:28.507662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504528+0000","last_change":"2024-03-29T08:29:14.499972+0000","last_active":"2024-03-29T08:29:14.504528+0000","last_peered":"2024-03-29T08:29:14.504528+0000","last_clean":"2024-03-29T08:29:14.504528+0000","last_became_active":"2024-03-29T08:29:14.499698+0000","last_became_peered":"2024-03-29T08:29:14.499698+0000","last_unstale":"2024-03-29T08:29:14.504528+0000","last_undegraded":"2024-03-29T08:29:14.504528+0000","last_fullsized":"2024-03-29T08:29:14.504528+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:44:41.312640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508984+0000","last_change":"2024-03-29T08:29:14.092574+0000","last_active":"2024-03-29T08:29:14.508984+0000","last_peered":"2024-03-29T08:29:14.508984+0000","last_clean":"2024-03-29T08:29:14.508984+0000","last_became_active":"2024-03-29T08:29:14.092004+0000","last_became_peered":"2024-03-29T08:29:14.092004+0000","last_unstale":"2024-03-29T08:29:14.508984+0000","last_undegraded":"2024-03-29T08:29:14.508984+0000","last_fullsized":"2024-03-29T08:29:14.508984+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:21.448128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.510447+0000","last_change":"2024-03-29T08:29:14.502264+0000","last_active":"2024-03-29T08:29:14.510447+0000","last_peered":"2024-03-29T08:29:14.510447+0000","last_clean":"2024-03-29T08:29:14.510447+0000","last_became_active":"2024-03-29T08:29:14.501882+0000","last_became_peered":"2024-03-29T08:29:14.501882+0000","last_unstale":"2024-03-29T08:29:14.510447+0000","last_undegraded":"2024-03-29T08:29:14.510447+0000","last_fullsized":"2024-03-29T08:29:14.510447+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:31:05.279265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.480729+0000","last_change":"2024-03-29T08:29:14.079278+0000","last_active":"2024-03-29T08:29:14.480729+0000","last_peered":"2024-03-29T08:29:14.480729+0000","last_clean":"2024-03-29T08:29:14.480729+0000","last_became_active":"2024-03-29T08:29:14.078982+0000","last_became_peered":"2024-03-29T08:29:14.078982+0000","last_unstale":"2024-03-29T08:29:14.480729+0000","last_undegraded":"2024-03-29T08:29:14.480729+0000","last_fullsized":"2024-03-29T08:29:14.480729+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:06.149502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509069+0000","last_change":"2024-03-29T08:29:14.500759+0000","last_active":"2024-03-29T08:29:14.509069+0000","last_peered":"2024-03-29T08:29:14.509069+0000","last_clean":"2024-03-29T08:29:14.509069+0000","last_became_active":"2024-03-29T08:29:14.500463+0000","last_became_peered":"2024-03-29T08:29:14.500463+0000","last_unstale":"2024-03-29T08:29:14.509069+0000","last_undegraded":"2024-03-29T08:29:14.509069+0000","last_fullsized":"2024-03-29T08:29:14.509069+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:13:49.079869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"120'8","reported_seq":18,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.510685+0000","last_change":"2024-03-29T08:29:14.090542+0000","last_active":"2024-03-29T08:29:14.510685+0000","last_peered":"2024-03-29T08:29:14.510685+0000","last_clean":"2024-03-29T08:29:14.510685+0000","last_became_active":"2024-03-29T08:29:14.090235+0000","last_became_peered":"2024-03-29T08:29:14.090235+0000","last_unstale":"2024-03-29T08:29:14.510685+0000","last_undegraded":"2024-03-29T08:29:14.510685+0000","last_fullsized":"2024-03-29T08:29:14.510685+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:45.558553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508199+0000","last_change":"2024-03-29T08:29:14.497747+0000","last_active":"2024-03-29T08:29:14.508199+0000","last_peered":"2024-03-29T08:29:14.508199+0000","last_clean":"2024-03-29T08:29:14.508199+0000","last_became_active":"2024-03-29T08:29:14.497456+0000","last_became_peered":"2024-03-29T08:29:14.497456+0000","last_unstale":"2024-03-29T08:29:14.508199+0000","last_undegraded":"2024-03-29T08:29:14.508199+0000","last_fullsized":"2024-03-29T08:29:14.508199+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:49.180147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.097550+0000","last_change":"2024-03-29T08:29:14.094084+0000","last_active":"2024-03-29T08:29:14.097550+0000","last_peered":"2024-03-29T08:29:14.097550+0000","last_clean":"2024-03-29T08:29:14.097550+0000","last_became_active":"2024-03-29T08:29:14.093447+0000","last_became_peered":"2024-03-29T08:29:14.093447+0000","last_unstale":"2024-03-29T08:29:14.097550+0000","last_undegraded":"2024-03-29T08:29:14.097550+0000","last_fullsized":"2024-03-29T08:29:14.097550+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:11:26.207009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.512213+0000","last_change":"2024-03-29T08:29:14.507678+0000","last_active":"2024-03-29T08:29:14.512213+0000","last_peered":"2024-03-29T08:29:14.512213+0000","last_clean":"2024-03-29T08:29:14.512213+0000","last_became_active":"2024-03-29T08:29:14.507132+0000","last_became_peered":"2024-03-29T08:29:14.507132+0000","last_unstale":"2024-03-29T08:29:14.512213+0000","last_undegraded":"2024-03-29T08:29:14.512213+0000","last_fullsized":"2024-03-29T08:29:14.512213+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:15:29.513377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.502517+0000","last_change":"2024-03-29T08:29:14.095051+0000","last_active":"2024-03-29T08:29:14.502517+0000","last_peered":"2024-03-29T08:29:14.502517+0000","last_clean":"2024-03-29T08:29:14.502517+0000","last_became_active":"2024-03-29T08:29:14.092852+0000","last_became_peered":"2024-03-29T08:29:14.092852+0000","last_unstale":"2024-03-29T08:29:14.502517+0000","last_undegraded":"2024-03-29T08:29:14.502517+0000","last_fullsized":"2024-03-29T08:29:14.502517+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:06.149502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504967+0000","last_change":"2024-03-29T08:29:14.497468+0000","last_active":"2024-03-29T08:29:14.504967+0000","last_peered":"2024-03-29T08:29:14.504967+0000","last_clean":"2024-03-29T08:29:14.504967+0000","last_became_active":"2024-03-29T08:29:14.497061+0000","last_became_peered":"2024-03-29T08:29:14.497061+0000","last_unstale":"2024-03-29T08:29:14.504967+0000","last_undegraded":"2024-03-29T08:29:14.504967+0000","last_fullsized":"2024-03-29T08:29:14.504967+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:09:20.324353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.501012+0000","last_change":"2024-03-29T08:29:14.090528+0000","last_active":"2024-03-29T08:29:14.501012+0000","last_peered":"2024-03-29T08:29:14.501012+0000","last_clean":"2024-03-29T08:29:14.501012+0000","last_became_active":"2024-03-29T08:29:14.090055+0000","last_became_peered":"2024-03-29T08:29:14.090055+0000","last_unstale":"2024-03-29T08:29:14.501012+0000","last_undegraded":"2024-03-29T08:29:14.501012+0000","last_fullsized":"2024-03-29T08:29:14.501012+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:21.448128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505830+0000","last_change":"2024-03-29T08:29:14.502568+0000","last_active":"2024-03-29T08:29:14.505830+0000","last_peered":"2024-03-29T08:29:14.505830+0000","last_clean":"2024-03-29T08:29:14.505830+0000","last_became_active":"2024-03-29T08:29:14.500161+0000","last_became_peered":"2024-03-29T08:29:14.500161+0000","last_unstale":"2024-03-29T08:29:14.505830+0000","last_undegraded":"2024-03-29T08:29:14.505830+0000","last_fullsized":"2024-03-29T08:29:14.505830+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:25:45.763218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.487422+0000","last_change":"2024-03-29T08:29:14.078956+0000","last_active":"2024-03-29T08:29:14.487422+0000","last_peered":"2024-03-29T08:29:14.487422+0000","last_clean":"2024-03-29T08:29:14.487422+0000","last_became_active":"2024-03-29T08:29:14.078689+0000","last_became_peered":"2024-03-29T08:29:14.078689+0000","last_unstale":"2024-03-29T08:29:14.487422+0000","last_undegraded":"2024-03-29T08:29:14.487422+0000","last_fullsized":"2024-03-29T08:29:14.487422+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:42:53.153883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.489423+0000","last_change":"2024-03-29T08:29:14.486493+0000","last_active":"2024-03-29T08:29:14.489423+0000","last_peered":"2024-03-29T08:29:14.489423+0000","last_clean":"2024-03-29T08:29:14.489423+0000","last_became_active":"2024-03-29T08:29:14.486242+0000","last_became_peered":"2024-03-29T08:29:14.486242+0000","last_unstale":"2024-03-29T08:29:14.489423+0000","last_undegraded":"2024-03-29T08:29:14.489423+0000","last_fullsized":"2024-03-29T08:29:14.489423+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:30:38.933793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507685+0000","last_change":"2024-03-29T08:29:14.092768+0000","last_active":"2024-03-29T08:29:14.507685+0000","last_peered":"2024-03-29T08:29:14.507685+0000","last_clean":"2024-03-29T08:29:14.507685+0000","last_became_active":"2024-03-29T08:29:14.092290+0000","last_became_peered":"2024-03-29T08:29:14.092290+0000","last_unstale":"2024-03-29T08:29:14.507685+0000","last_undegraded":"2024-03-29T08:29:14.507685+0000","last_fullsized":"2024-03-29T08:29:14.507685+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:45.752363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509731+0000","last_change":"2024-03-29T08:29:14.502876+0000","last_active":"2024-03-29T08:29:14.509731+0000","last_peered":"2024-03-29T08:29:14.509731+0000","last_clean":"2024-03-29T08:29:14.509731+0000","last_became_active":"2024-03-29T08:29:14.502600+0000","last_became_peered":"2024-03-29T08:29:14.502600+0000","last_unstale":"2024-03-29T08:29:14.509731+0000","last_undegraded":"2024-03-29T08:29:14.509731+0000","last_fullsized":"2024-03-29T08:29:14.509731+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:49:19.406820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.486956+0000","last_change":"2024-03-29T08:29:14.090942+0000","last_active":"2024-03-29T08:29:14.486956+0000","last_peered":"2024-03-29T08:29:14.486956+0000","last_clean":"2024-03-29T08:29:14.486956+0000","last_became_active":"2024-03-29T08:29:14.090695+0000","last_became_peered":"2024-03-29T08:29:14.090695+0000","last_unstale":"2024-03-29T08:29:14.486956+0000","last_undegraded":"2024-03-29T08:29:14.486956+0000","last_fullsized":"2024-03-29T08:29:14.486956+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:18.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509268+0000","last_change":"2024-03-29T08:29:14.508582+0000","last_active":"2024-03-29T08:29:14.509268+0000","last_peered":"2024-03-29T08:29:14.509268+0000","last_clean":"2024-03-29T08:29:14.509268+0000","last_became_active":"2024-03-29T08:29:14.508201+0000","last_became_peered":"2024-03-29T08:29:14.508201+0000","last_unstale":"2024-03-29T08:29:14.509268+0000","last_undegraded":"2024-03-29T08:29:14.509268+0000","last_fullsized":"2024-03-29T08:29:14.509268+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:59:12.691184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.497159+0000","last_change":"2024-03-29T08:29:14.096474+0000","last_active":"2024-03-29T08:29:14.497159+0000","last_peered":"2024-03-29T08:29:14.497159+0000","last_clean":"2024-03-29T08:29:14.497159+0000","last_became_active":"2024-03-29T08:29:14.094758+0000","last_became_peered":"2024-03-29T08:29:14.094758+0000","last_unstale":"2024-03-29T08:29:14.497159+0000","last_undegraded":"2024-03-29T08:29:14.497159+0000","last_fullsized":"2024-03-29T08:29:14.497159+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:32:23.713785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505838+0000","last_change":"2024-03-29T08:29:14.503676+0000","last_active":"2024-03-29T08:29:14.505838+0000","last_peered":"2024-03-29T08:29:14.505838+0000","last_clean":"2024-03-29T08:29:14.505838+0000","last_became_active":"2024-03-29T08:29:14.503314+0000","last_became_peered":"2024-03-29T08:29:14.503314+0000","last_unstale":"2024-03-29T08:29:14.505838+0000","last_undegraded":"2024-03-29T08:29:14.505838+0000","last_fullsized":"2024-03-29T08:29:14.505838+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:08:59.334138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.496690+0000","last_change":"2024-03-29T08:29:14.089768+0000","last_active":"2024-03-29T08:29:14.496690+0000","last_peered":"2024-03-29T08:29:14.496690+0000","last_clean":"2024-03-29T08:29:14.496690+0000","last_became_active":"2024-03-29T08:29:14.089486+0000","last_became_peered":"2024-03-29T08:29:14.089486+0000","last_unstale":"2024-03-29T08:29:14.496690+0000","last_undegraded":"2024-03-29T08:29:14.496690+0000","last_fullsized":"2024-03-29T08:29:14.496690+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:59.174853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507881+0000","last_change":"2024-03-29T08:29:14.505681+0000","last_active":"2024-03-29T08:29:14.507881+0000","last_peered":"2024-03-29T08:29:14.507881+0000","last_clean":"2024-03-29T08:29:14.507881+0000","last_became_active":"2024-03-29T08:29:14.505394+0000","last_became_peered":"2024-03-29T08:29:14.505394+0000","last_unstale":"2024-03-29T08:29:14.507881+0000","last_undegraded":"2024-03-29T08:29:14.507881+0000","last_fullsized":"2024-03-29T08:29:14.507881+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:32:12.510083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508946+0000","last_change":"2024-03-29T08:29:14.088394+0000","last_active":"2024-03-29T08:29:14.508946+0000","last_peered":"2024-03-29T08:29:14.508946+0000","last_clean":"2024-03-29T08:29:14.508946+0000","last_became_active":"2024-03-29T08:29:14.087525+0000","last_became_peered":"2024-03-29T08:29:14.087525+0000","last_unstale":"2024-03-29T08:29:14.508946+0000","last_undegraded":"2024-03-29T08:29:14.508946+0000","last_fullsized":"2024-03-29T08:29:14.508946+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:54:27.262284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.086933+0000","last_change":"2024-03-29T08:29:14.086933+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.086933+0000","last_undegraded":"2024-03-29T08:29:14.086933+0000","last_fullsized":"2024-03-29T08:29:14.086933+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:14:06.265379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.497231+0000","last_change":"2024-03-29T08:29:14.088202+0000","last_active":"2024-03-29T08:29:14.497231+0000","last_peered":"2024-03-29T08:29:14.497231+0000","last_clean":"2024-03-29T08:29:14.497231+0000","last_became_active":"2024-03-29T08:29:14.087860+0000","last_became_peered":"2024-03-29T08:29:14.087860+0000","last_unstale":"2024-03-29T08:29:14.497231+0000","last_undegraded":"2024-03-29T08:29:14.497231+0000","last_fullsized":"2024-03-29T08:29:14.497231+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:30:51.004244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509624+0000","last_change":"2024-03-29T08:29:14.501270+0000","last_active":"2024-03-29T08:29:14.509624+0000","last_peered":"2024-03-29T08:29:14.509624+0000","last_clean":"2024-03-29T08:29:14.509624+0000","last_became_active":"2024-03-29T08:29:14.498111+0000","last_became_peered":"2024-03-29T08:29:14.498111+0000","last_unstale":"2024-03-29T08:29:14.509624+0000","last_undegraded":"2024-03-29T08:29:14.509624+0000","last_fullsized":"2024-03-29T08:29:14.509624+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:26:17.474099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.495208+0000","last_change":"2024-03-29T08:29:14.090681+0000","last_active":"2024-03-29T08:29:14.495208+0000","last_peered":"2024-03-29T08:29:14.495208+0000","last_clean":"2024-03-29T08:29:14.495208+0000","last_became_active":"2024-03-29T08:29:14.090416+0000","last_became_peered":"2024-03-29T08:29:14.090416+0000","last_unstale":"2024-03-29T08:29:14.495208+0000","last_undegraded":"2024-03-29T08:29:14.495208+0000","last_fullsized":"2024-03-29T08:29:14.495208+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:08:56.584444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507376+0000","last_change":"2024-03-29T08:29:14.505745+0000","last_active":"2024-03-29T08:29:14.507376+0000","last_peered":"2024-03-29T08:29:14.507376+0000","last_clean":"2024-03-29T08:29:14.507376+0000","last_became_active":"2024-03-29T08:29:14.504499+0000","last_became_peered":"2024-03-29T08:29:14.504499+0000","last_unstale":"2024-03-29T08:29:14.507376+0000","last_undegraded":"2024-03-29T08:29:14.507376+0000","last_fullsized":"2024-03-29T08:29:14.507376+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:23:28.769780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.495503+0000","last_change":"2024-03-29T08:29:14.096941+0000","last_active":"2024-03-29T08:29:14.495503+0000","last_peered":"2024-03-29T08:29:14.495503+0000","last_clean":"2024-03-29T08:29:14.495503+0000","last_became_active":"2024-03-29T08:29:14.096439+0000","last_became_peered":"2024-03-29T08:29:14.096439+0000","last_unstale":"2024-03-29T08:29:14.495503+0000","last_undegraded":"2024-03-29T08:29:14.495503+0000","last_fullsized":"2024-03-29T08:29:14.495503+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:49:13.231792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.503705+0000","last_change":"2024-03-29T08:29:14.496299+0000","last_active":"2024-03-29T08:29:14.503705+0000","last_peered":"2024-03-29T08:29:14.503705+0000","last_clean":"2024-03-29T08:29:14.503705+0000","last_became_active":"2024-03-29T08:29:14.496042+0000","last_became_peered":"2024-03-29T08:29:14.496042+0000","last_unstale":"2024-03-29T08:29:14.503705+0000","last_undegraded":"2024-03-29T08:29:14.503705+0000","last_fullsized":"2024-03-29T08:29:14.503705+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:23:28.769780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.497619+0000","last_change":"2024-03-29T08:29:14.081880+0000","last_active":"2024-03-29T08:29:14.497619+0000","last_peered":"2024-03-29T08:29:14.497619+0000","last_clean":"2024-03-29T08:29:14.497619+0000","last_became_active":"2024-03-29T08:29:14.079732+0000","last_became_peered":"2024-03-29T08:29:14.079732+0000","last_unstale":"2024-03-29T08:29:14.497619+0000","last_undegraded":"2024-03-29T08:29:14.497619+0000","last_fullsized":"2024-03-29T08:29:14.497619+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:01.494143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.503628+0000","last_change":"2024-03-29T08:29:14.501776+0000","last_active":"2024-03-29T08:29:14.503628+0000","last_peered":"2024-03-29T08:29:14.503628+0000","last_clean":"2024-03-29T08:29:14.503628+0000","last_became_active":"2024-03-29T08:29:14.501508+0000","last_became_peered":"2024-03-29T08:29:14.501508+0000","last_unstale":"2024-03-29T08:29:14.503628+0000","last_undegraded":"2024-03-29T08:29:14.503628+0000","last_fullsized":"2024-03-29T08:29:14.503628+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:01.476642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.487173+0000","last_change":"2024-03-29T08:29:14.091456+0000","last_active":"2024-03-29T08:29:14.487173+0000","last_peered":"2024-03-29T08:29:14.487173+0000","last_clean":"2024-03-29T08:29:14.487173+0000","last_became_active":"2024-03-29T08:29:14.091121+0000","last_became_peered":"2024-03-29T08:29:14.091121+0000","last_unstale":"2024-03-29T08:29:14.487173+0000","last_undegraded":"2024-03-29T08:29:14.487173+0000","last_fullsized":"2024-03-29T08:29:14.487173+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:30.979257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507721+0000","last_change":"2024-03-29T08:29:14.503755+0000","last_active":"2024-03-29T08:29:14.507721+0000","last_peered":"2024-03-29T08:29:14.507721+0000","last_clean":"2024-03-29T08:29:14.507721+0000","last_became_active":"2024-03-29T08:29:14.503521+0000","last_became_peered":"2024-03-29T08:29:14.503521+0000","last_unstale":"2024-03-29T08:29:14.507721+0000","last_undegraded":"2024-03-29T08:29:14.507721+0000","last_fullsized":"2024-03-29T08:29:14.507721+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:33:36.260388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.486511+0000","last_change":"2024-03-29T08:29:14.089662+0000","last_active":"2024-03-29T08:29:14.486511+0000","last_peered":"2024-03-29T08:29:14.486511+0000","last_clean":"2024-03-29T08:29:14.486511+0000","last_became_active":"2024-03-29T08:29:14.089393+0000","last_became_peered":"2024-03-29T08:29:14.089393+0000","last_unstale":"2024-03-29T08:29:14.486511+0000","last_undegraded":"2024-03-29T08:29:14.486511+0000","last_fullsized":"2024-03-29T08:29:14.486511+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:27:50.778160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.084305+0000","last_change":"2024-03-29T08:29:14.084305+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.084305+0000","last_undegraded":"2024-03-29T08:29:14.084305+0000","last_fullsized":"2024-03-29T08:29:14.084305+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:44:25.936884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.095596+0000","last_change":"2024-03-29T08:29:14.092230+0000","last_active":"2024-03-29T08:29:14.095596+0000","last_peered":"2024-03-29T08:29:14.095596+0000","last_clean":"2024-03-29T08:29:14.095596+0000","last_became_active":"2024-03-29T08:29:14.091979+0000","last_became_peered":"2024-03-29T08:29:14.091979+0000","last_unstale":"2024-03-29T08:29:14.095596+0000","last_undegraded":"2024-03-29T08:29:14.095596+0000","last_fullsized":"2024-03-29T08:29:14.095596+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:41:42.442894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.510494+0000","last_change":"2024-03-29T08:29:14.502476+0000","last_active":"2024-03-29T08:29:14.510494+0000","last_peered":"2024-03-29T08:29:14.510494+0000","last_clean":"2024-03-29T08:29:14.510494+0000","last_became_active":"2024-03-29T08:29:14.502191+0000","last_became_peered":"2024-03-29T08:29:14.502191+0000","last_unstale":"2024-03-29T08:29:14.510494+0000","last_undegraded":"2024-03-29T08:29:14.510494+0000","last_fullsized":"2024-03-29T08:29:14.510494+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:06.826332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.475058+0000","last_change":"2024-03-29T08:29:14.077404+0000","last_active":"2024-03-29T08:29:14.475058+0000","last_peered":"2024-03-29T08:29:14.475058+0000","last_clean":"2024-03-29T08:29:14.475058+0000","last_became_active":"2024-03-29T08:29:14.077141+0000","last_became_peered":"2024-03-29T08:29:14.077141+0000","last_unstale":"2024-03-29T08:29:14.475058+0000","last_undegraded":"2024-03-29T08:29:14.475058+0000","last_fullsized":"2024-03-29T08:29:14.475058+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:51.446071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":250,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.072521+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:29:14.072521+0000","last_peered":"2024-03-29T08:29:14.072521+0000","last_clean":"2024-03-29T08:29:14.072521+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:29:14.072521+0000","last_undegraded":"2024-03-29T08:29:14.072521+0000","last_fullsized":"2024-03-29T08:29:14.072521+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509578,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1503232,"data_stored":646009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.69299999999999995},"min":{"1min":0.48199999999999998,"5min":0.41899999999999998,"15min":0.34999999999999998},"max":{"1min":0.88,"5min":0.97999999999999998,"15min":1.1639999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.75600000000000001,"15min":0.71899999999999997},"min":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.45800000000000002},"max":{"1min":0.95099999999999996,"5min":0.998,"15min":1.2410000000000001},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70199999999999996,"15min":0.67600000000000005},"min":{"1min":0.51500000000000001,"5min":0.45700000000000002,"15min":0.39600000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.202},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70799999999999996,"15min":0.67600000000000005},"min":{"1min":0.56100000000000005,"5min":0.48799999999999999,"15min":0.44},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.1339999999999999},"last":0.505}]},{"osd":2,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69799999999999995,"15min":0.67000000000000004},"min":{"1min":0.502,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.90300000000000002,"5min":0.93600000000000005,"15min":1.085},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72899999999999998,"15min":0.70299999999999996},"min":{"1min":0.55500000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.288},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72299999999999998,"15min":0.70999999999999996},"min":{"1min":0.57599999999999996,"5min":0.51100000000000001,"15min":0.46000000000000002},"max":{"1min":0.97199999999999998,"5min":0.98099999999999998,"15min":1.018},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.60899999999999999,"5min":0.52300000000000002,"15min":0.48499999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.2609999999999999},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58399999999999996,"15min":0.60999999999999999},"min":{"1min":0.48099999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85299999999999998,"5min":0.87,"15min":0.95299999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60599999999999998,"15min":0.66300000000000003},"min":{"1min":0.45100000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.123},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60999999999999999,"15min":0.66200000000000003},"min":{"1min":0.48199999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.877,"5min":0.89600000000000002,"15min":1.056},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.628,"15min":0.66900000000000004},"min":{"1min":0.49199999999999999,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.84399999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.47399999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69199999999999995,"15min":0.71399999999999997},"min":{"1min":0.55900000000000005,"5min":0.46999999999999997,"15min":0.46600000000000003},"max":{"1min":0.89400000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.436},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65800000000000003,"15min":0.70599999999999996},"min":{"1min":0.52800000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.81799999999999995,"5min":0.93799999999999994,"15min":0.98299999999999998},"last":0.64400000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509578,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1482752,"data_stored":644321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72399999999999998,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.38900000000000001},"max":{"1min":0.873,"5min":0.93100000000000005,"15min":1.0189999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.751,"15min":0.71899999999999997},"min":{"1min":0.57199999999999995,"5min":0.51700000000000002,"15min":0.44500000000000001},"max":{"1min":0.97699999999999998,"5min":0.99299999999999999,"15min":1.079},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68200000000000005,"15min":0.64700000000000002},"min":{"1min":0.45400000000000001,"5min":0.45300000000000001,"15min":0.377},"max":{"1min":0.85199999999999998,"5min":0.97999999999999998,"15min":0.99299999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42899999999999999},"max":{"1min":0.78200000000000003,"5min":0.95599999999999996,"15min":1.079},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76900000000000002,"15min":0.72299999999999998},"min":{"1min":0.52100000000000002,"5min":0.51800000000000002,"15min":0.44800000000000001},"max":{"1min":0.91500000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.746,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.46400000000000002,"15min":0.435},"max":{"1min":0.93899999999999995,"5min":0.95099999999999996,"15min":0.95899999999999996},"last":0.753}]},{"osd":3,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.443,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82099999999999995,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.55700000000000005,"5min":0.51000000000000001,"15min":0.47799999999999998},"max":{"1min":0.89800000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.71299999999999997}]},{"osd":4,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.623,"15min":0.64600000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.36499999999999999},"max":{"1min":0.93100000000000005,"5min":1.0840000000000001,"15min":1.556},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63900000000000001,"15min":0.68200000000000005},"min":{"1min":0.46400000000000002,"5min":0.38200000000000001,"15min":0.371},"max":{"1min":0.872,"5min":0.996,"15min":2.0539999999999998},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.627,"15min":0.66100000000000003},"min":{"1min":0.45000000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83599999999999997,"5min":0.99399999999999999,"15min":1.0660000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68100000000000005,"15min":0.70199999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":1.042,"15min":1.1419999999999999},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69599999999999995,"15min":0.72499999999999998},"min":{"1min":0.499,"5min":0.46800000000000003,"15min":0.38500000000000001},"max":{"1min":0.89600000000000002,"5min":1.03,"15min":1.03},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.72999999999999998},"min":{"1min":0.51600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91500000000000004,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.63}]}]},{"osd":4,"up_from":15,"seq":64424509577,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1499136,"data_stored":645975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.45000000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.026,"5min":1.026,"15min":2.1709999999999998},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69699999999999995,"15min":0.71799999999999997},"min":{"1min":0.495,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.997,"5min":0.997,"15min":2.2869999999999999},"last":0.82999999999999996}]},{"osd":1,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.72499999999999998,"15min":0.748},"min":{"1min":0.436,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":2.524},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71099999999999997,"15min":0.70899999999999996},"min":{"1min":0.46400000000000002,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":2.2679999999999998},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69999999999999996,"15min":0.71799999999999997},"min":{"1min":0.45300000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":2.363},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72999999999999998,"15min":0.748},"min":{"1min":0.52500000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.88700000000000001,"5min":0.97799999999999998,"15min":2.1880000000000002},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76200000000000001,"15min":0.753},"min":{"1min":0.60899999999999999,"5min":0.53400000000000003,"15min":0.46300000000000002},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":2.3029999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.77100000000000002,"15min":0.79300000000000004},"min":{"1min":0.55600000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.016,"5min":1.016,"15min":2.2549999999999999},"last":0.81599999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60699999999999998,"15min":0.65800000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.36499999999999999},"max":{"1min":0.76400000000000001,"5min":0.998,"15min":1.0189999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.62,"15min":0.63900000000000001},"min":{"1min":0.45800000000000002,"5min":0.437,"15min":0.41799999999999998},"max":{"1min":0.82699999999999996,"5min":0.85499999999999998,"15min":1.0149999999999999},"last":0.59599999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.63600000000000001,"15min":0.65700000000000003},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.85599999999999998,"5min":1.091,"15min":1.091},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69399999999999995,"15min":0.72099999999999997},"min":{"1min":0.505,"5min":0.499,"15min":0.499},"max":{"1min":0.86399999999999999,"5min":0.91500000000000004,"15min":1.1220000000000001},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.71199999999999997},"min":{"1min":0.51400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91400000000000003,"5min":0.91800000000000004,"15min":1.1000000000000001},"last":0.754},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72599999999999998,"15min":0.747},"min":{"1min":0.434,"5min":0.434,"15min":0.42899999999999999},"max":{"1min":0.96499999999999997,"5min":1.054,"15min":1.4430000000000001},"last":0.69399999999999995}]}]},{"osd":3,"up_from":15,"seq":64424509580,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1482752,"data_stored":644321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.28100000000000003,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":0.71799999999999997,"5min":0.748,"15min":1.123},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69599999999999995,"5min":0.88100000000000001,"15min":1.0409999999999999},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.44,"15min":0.44800000000000001},"min":{"1min":0.32800000000000001,"5min":0.29599999999999999,"15min":0.26500000000000001},"max":{"1min":0.623,"5min":0.76800000000000002,"15min":1.1599999999999999},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.28599999999999998},"max":{"1min":0.82099999999999995,"5min":0.94499999999999995,"15min":1.4790000000000001},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.54400000000000004},"min":{"1min":0.35499999999999998,"5min":0.32900000000000001,"15min":0.309},"max":{"1min":0.76200000000000001,"5min":0.80300000000000005,"15min":1.2529999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.624},"min":{"1min":0.439,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78300000000000003,"5min":0.86599999999999999,"15min":1.8580000000000001},"last":0.59899999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77200000000000002,"15min":0.77500000000000002},"min":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.42199999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.6419999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.77900000000000003,"15min":0.76900000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.45400000000000001},"max":{"1min":1.083,"5min":1.3340000000000001,"15min":1.391},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82199999999999995,"15min":0.80300000000000005},"min":{"1min":0.59599999999999997,"5min":0.57099999999999995,"15min":0.52100000000000002},"max":{"1min":1.081,"5min":1.1850000000000001,"15min":1.671},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.82399999999999995,"15min":0.83499999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0249999999999999,"5min":1.2050000000000001,"15min":1.7110000000000001},"last":0.58699999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.55400000000000005,"5min":0.51700000000000002,"15min":0.48799999999999999},"max":{"1min":1.232,"5min":1.2609999999999999,"15min":1.669},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78100000000000003,"15min":0.79600000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0029999999999999,"5min":1.119,"15min":2.0150000000000001},"last":0.54600000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.82199999999999995},"min":{"1min":0.58899999999999997,"5min":0.50600000000000001,"15min":0.48799999999999999},"max":{"1min":1.0069999999999999,"5min":1.3169999999999999,"15min":1.887},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.81899999999999995},"min":{"1min":0.58999999999999997,"5min":0.49399999999999999,"15min":0.47999999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":2.044},"last":0.57199999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509579,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1470464,"data_stored":637546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45900000000000002,"15min":0.47799999999999998},"min":{"1min":0.312,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.67800000000000005,"5min":0.754,"15min":0.94299999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.378,"5min":0.34699999999999998,"15min":0.32300000000000001},"max":{"1min":0.70699999999999996,"5min":0.874,"15min":2.0880000000000001},"last":0.45200000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.49399999999999999},"min":{"1min":0.34000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70599999999999996,"5min":0.73499999999999999,"15min":1.399},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.501,"15min":0.50600000000000001},"min":{"1min":0.312,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":3.0459999999999998},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79900000000000004,"5min":0.93300000000000005,"15min":2.1280000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.35999999999999999},"max":{"1min":0.79900000000000004,"5min":0.88800000000000001,"15min":2.9950000000000001},"last":0.71399999999999997}]},{"osd":4,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79400000000000004,"15min":0.77700000000000002},"min":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.49099999999999999},"max":{"1min":1.0329999999999999,"5min":1.109,"15min":1.4239999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.76600000000000001},"min":{"1min":0.51900000000000002,"5min":0.51700000000000002,"15min":0.435},"max":{"1min":1.1479999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.92800000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.85599999999999998,"15min":0.83999999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.143,"5min":1.236,"15min":1.4890000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.48799999999999999},"max":{"1min":1.1100000000000001,"5min":1.194,"15min":1.526},"last":0.97799999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79100000000000004,"15min":0.78200000000000003},"min":{"1min":0.57899999999999996,"5min":0.54600000000000004,"15min":0.48599999999999999},"max":{"1min":1.1040000000000001,"5min":1.131,"15min":2.0470000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.82999999999999996,"15min":0.79800000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.52500000000000002},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.524},"last":1.002}]},{"osd":7,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.81399999999999995,"15min":0.80100000000000005},"min":{"1min":0.59099999999999997,"5min":0.54900000000000004,"15min":0.505},"max":{"1min":1.1850000000000001,"5min":1.296,"15min":1.319},"last":0.874},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.81000000000000005,"15min":0.79700000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.214,"5min":1.214,"15min":2},"last":0.90800000000000003}]}]},{"osd":1,"up_from":15,"seq":64424509579,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2076672,"data_stored":1234689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45800000000000002,"15min":0.53200000000000003},"min":{"1min":0.28699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.69699999999999995,"5min":0.75900000000000001,"15min":4.2539999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48299999999999998,"15min":0.56399999999999995},"min":{"1min":0.29399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":4.1909999999999998},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.31900000000000001},"max":{"1min":0.63600000000000001,"5min":0.71099999999999997,"15min":0.82899999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.34599999999999997,"15min":0.33600000000000002},"max":{"1min":0.67000000000000004,"5min":0.77700000000000002,"15min":0.88200000000000001},"last":0.51900000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.35499999999999998},"max":{"1min":0.75700000000000001,"5min":0.88300000000000001,"15min":1.52},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.496,"5min":0.51900000000000002,"15min":0.52400000000000002},"min":{"1min":0.35999999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68999999999999995,"5min":0.94499999999999995,"15min":1.331},"last":0.44}]},{"osd":4,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.75600000000000001},"min":{"1min":0.60199999999999998,"5min":0.51800000000000002,"15min":0.46200000000000002},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.77600000000000002},"min":{"1min":0.56599999999999995,"5min":0.502,"15min":0.47999999999999998},"max":{"1min":1.212,"5min":1.212,"15min":1.23},"last":0.80100000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79600000000000004,"15min":0.77900000000000003},"min":{"1min":0.59699999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.76800000000000002},"min":{"1min":0.55200000000000005,"5min":0.55100000000000005,"15min":0.46500000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.2509999999999999},"last":0.81200000000000006}]},{"osd":6,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.79300000000000004,"15min":0.77300000000000002},"min":{"1min":0.58299999999999996,"5min":0.54900000000000004,"15min":0.5},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.81499999999999995,"15min":0.82399999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51300000000000001},"max":{"1min":1.0209999999999999,"5min":1.22,"15min":1.22},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.52100000000000002},"max":{"1min":1.1499999999999999,"5min":1.2410000000000001,"15min":1.3089999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.80700000000000005,"15min":0.79000000000000004},"min":{"1min":0.622,"5min":0.503,"15min":0.503},"max":{"1min":1.1759999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.89800000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509578,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1490944,"data_stored":644973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.44400000000000001,"15min":0.51900000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.22800000000000001},"max":{"1min":0.62,"5min":0.75800000000000001,"15min":3.9359999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.49299999999999999,"15min":0.66700000000000004},"min":{"1min":0.31900000000000001,"5min":0.28499999999999998,"15min":0.253},"max":{"1min":0.72599999999999998,"5min":0.77000000000000002,"15min":8.0419999999999998},"last":0.34300000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50900000000000001,"15min":0.52700000000000002},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34599999999999997},"max":{"1min":0.68500000000000005,"5min":0.78000000000000003,"15min":1.258},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54500000000000004,"15min":0.63300000000000001},"min":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.312},"max":{"1min":0.77400000000000002,"5min":0.81399999999999995,"15min":4.907},"last":0.44}]},{"osd":3,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.501,"15min":0.66900000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.28499999999999998},"max":{"1min":0.65100000000000002,"5min":0.83399999999999996,"15min":7.9569999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.61399999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.34799999999999998},"max":{"1min":0.80100000000000005,"5min":0.87,"15min":1.397},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79500000000000004,"15min":0.88500000000000001},"min":{"1min":0.56799999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1020000000000001,"5min":1.4399999999999999,"15min":4.4720000000000004},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.79400000000000004,"15min":0.86799999999999999},"min":{"1min":0.51500000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.141,"5min":1.153,"15min":4.3920000000000003},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.85499999999999998},"min":{"1min":0.56699999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.014,"5min":1.349,"15min":4.3200000000000003},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80300000000000005,"15min":0.82499999999999996},"min":{"1min":0.59599999999999997,"5min":0.56000000000000005,"15min":0.54600000000000004},"max":{"1min":1.081,"5min":1.26,"15min":1.6759999999999999},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.79800000000000004,"15min":0.82699999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.54400000000000004},"max":{"1min":1.0640000000000001,"5min":1.1579999999999999,"15min":1.5920000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81599999999999995,"15min":0.89300000000000002},"min":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.165,"5min":1.1990000000000001,"15min":4.4100000000000001},"last":0.85499999999999998}]},{"osd":7,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82399999999999995,"15min":0.90600000000000003},"min":{"1min":0.61499999999999999,"5min":0.54100000000000004,"15min":0.496},"max":{"1min":1.141,"5min":1.3959999999999999,"15min":4.5209999999999999},"last":0.752},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80500000000000005,"15min":0.88900000000000001},"min":{"1min":0.56499999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.089,"5min":1.2390000000000001,"15min":4.4420000000000002},"last":0.69999999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542283,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":2064384,"data_stored":1227914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70899999999999996,"15min":0.66400000000000003},"min":{"1min":0.47499999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.98499999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72999999999999998,"15min":0.68400000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.39800000000000002},"max":{"1min":0.84799999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64100000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70699999999999996,"15min":0.71199999999999997},"min":{"1min":0.48499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.008,"5min":1.008,"15min":2.5129999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74299999999999999,"15min":0.70299999999999996},"min":{"1min":0.56799999999999995,"5min":0.51300000000000001,"15min":0.40500000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.85599999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71799999999999997,"15min":0.68600000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92800000000000005,"5min":0.997,"15min":0.997},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.747,"15min":0.71799999999999997},"min":{"1min":0.60899999999999999,"5min":0.47499999999999998,"15min":0.39200000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.77700000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.755,"15min":0.72599999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.47099999999999997},"max":{"1min":1.02,"5min":1.038,"15min":1.038},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.752,"15min":0.74099999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.435},"max":{"1min":0.995,"5min":1.0209999999999999,"15min":1.954},"last":0.81200000000000006}]},{"osd":4,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.624,"15min":0.64000000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.91200000000000003,"15min":0.93500000000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.373},"max":{"1min":0.83299999999999996,"5min":0.94699999999999995,"15min":1.0820000000000001},"last":0.82699999999999996}]},{"osd":6,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.621,"15min":0.65000000000000002},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85999999999999999,"5min":0.93300000000000005,"15min":1.071},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.68500000000000005},"min":{"1min":0.53200000000000003,"5min":0.45300000000000001,"15min":0.432},"max":{"1min":0.92000000000000004,"5min":0.94299999999999995,"15min":1.0609999999999999},"last":0.89800000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90100000000000002,"5min":0.92200000000000004,"15min":1.1200000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70099999999999996,"15min":0.72899999999999998},"min":{"1min":0.46800000000000003,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.94999999999999996,"5min":0.96499999999999997,"15min":1.04},"last":0.98399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:29:19.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3a is not active+clean 2024-03-29T08:29:19.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:29:14.084660+0000', 'last_change': '2024-03-29T08:29:14.084660+0000', 'last_active': '2024-03-29T08:29:14.057150+0000', 'last_peered': '2024-03-29T08:29:14.057150+0000', 'last_clean': '2024-03-29T08:29:14.057150+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:29:14.084660+0000', 'last_undegraded': '2024-03-29T08:29:14.084660+0000', 'last_fullsized': '2024-03-29T08:29:14.084660+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'last_clean_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T12:35:51.101298+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:29:19.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2a is not active+clean 2024-03-29T08:29:19.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:29:14.084054+0000', 'last_change': '2024-03-29T08:29:14.084054+0000', 'last_active': '2024-03-29T08:29:14.057150+0000', 'last_peered': '2024-03-29T08:29:14.057150+0000', 'last_clean': '2024-03-29T08:29:14.057150+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:29:14.084054+0000', 'last_undegraded': '2024-03-29T08:29:14.084054+0000', 'last_fullsized': '2024-03-29T08:29:14.084054+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'last_clean_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T18:57:23.884319+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:29:19.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.26 is not active+clean 2024-03-29T08:29:19.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:29:14.085680+0000', 'last_change': '2024-03-29T08:29:14.085680+0000', 'last_active': '2024-03-29T08:29:14.057150+0000', 'last_peered': '2024-03-29T08:29:14.057150+0000', 'last_clean': '2024-03-29T08:29:14.057150+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:29:14.085680+0000', 'last_undegraded': '2024-03-29T08:29:14.085680+0000', 'last_fullsized': '2024-03-29T08:29:14.085680+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'last_clean_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T18:10:44.097211+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:29:19.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.24 is not active+clean 2024-03-29T08:29:19.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:29:14.087721+0000', 'last_change': '2024-03-29T08:29:14.087721+0000', 'last_active': '2024-03-29T08:29:14.057150+0000', 'last_peered': '2024-03-29T08:29:14.057150+0000', 'last_clean': '2024-03-29T08:29:14.057150+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:29:14.087721+0000', 'last_undegraded': '2024-03-29T08:29:14.087721+0000', 'last_fullsized': '2024-03-29T08:29:14.087721+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'last_clean_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T12:46:53.749284+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:29:19.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1c is not active+clean 2024-03-29T08:29:19.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:29:14.085712+0000', 'last_change': '2024-03-29T08:29:14.085712+0000', 'last_active': '2024-03-29T08:29:14.057150+0000', 'last_peered': '2024-03-29T08:29:14.057150+0000', 'last_clean': '2024-03-29T08:29:14.057150+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:29:14.085712+0000', 'last_undegraded': '2024-03-29T08:29:14.085712+0000', 'last_fullsized': '2024-03-29T08:29:14.085712+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'last_clean_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T19:23:32.467222+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:29:19.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.14 is not active+clean 2024-03-29T08:29:19.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:29:14.085301+0000', 'last_change': '2024-03-29T08:29:14.085301+0000', 'last_active': '2024-03-29T08:29:14.057150+0000', 'last_peered': '2024-03-29T08:29:14.057150+0000', 'last_clean': '2024-03-29T08:29:14.057150+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:29:14.085301+0000', 'last_undegraded': '2024-03-29T08:29:14.085301+0000', 'last_fullsized': '2024-03-29T08:29:14.085301+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'last_clean_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T15:28:45.863920+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:29:19.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.7 is not active+clean 2024-03-29T08:29:19.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:29:14.086933+0000', 'last_change': '2024-03-29T08:29:14.086933+0000', 'last_active': '2024-03-29T08:29:14.057150+0000', 'last_peered': '2024-03-29T08:29:14.057150+0000', 'last_clean': '2024-03-29T08:29:14.057150+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:29:14.086933+0000', 'last_undegraded': '2024-03-29T08:29:14.086933+0000', 'last_fullsized': '2024-03-29T08:29:14.086933+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'last_clean_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T16:14:06.265379+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:29:19.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1 is not active+clean 2024-03-29T08:29:19.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-03-29T08:29:14.084305+0000', 'last_change': '2024-03-29T08:29:14.084305+0000', 'last_active': '2024-03-29T08:29:14.057150+0000', 'last_peered': '2024-03-29T08:29:14.057150+0000', 'last_clean': '2024-03-29T08:29:14.057150+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-03-29T08:29:14.084305+0000', 'last_undegraded': '2024-03-29T08:29:14.084305+0000', 'last_fullsized': '2024-03-29T08:29:14.084305+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'last_clean_scrub_stamp': '2024-03-29T08:29:14.057150+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-03-30T09:44:25.936884+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-03-29T08:29:19.359 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-29T08:29:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.529+0000 7f0413000640 1 Processor -- start 2024-03-29T08:29:19.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.529+0000 7f0413000640 1 -- start start 2024-03-29T08:29:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.533+0000 7f0413000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f040c15e160 0x7f040c0caf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.533+0000 7f0413000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f040c0b0220 0x7f040c0b0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.533+0000 7f0413000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f040c0aef50 con 0x7f040c15a6b0 2024-03-29T08:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.533+0000 7f0413000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f040c15e5f0 con 0x7f040c15e160 2024-03-29T08:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.533+0000 7f0413000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f040c15aa90 con 0x7f040c0b0220 2024-03-29T08:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.533+0000 7f0411200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f040c15e160 0x7f040c0caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.533+0000 7f0411200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f040c15e160 0x7f040c0caf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39066/0 (socket says 172.21.15.46:39066) 2024-03-29T08:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.533+0000 7f0411200640 1 -- 172.21.15.46:0/608852762 learned_addr learned my addr 172.21.15.46:0/608852762 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.533+0000 7f0412600640 1 --2- 172.21.15.46:0/608852762 >> v2:172.21.15.73:3300/0 conn(0x7f040c0b0220 0x7f040c0b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.533+0000 7f0411200640 1 -- 172.21.15.46:0/608852762 >> v2:172.21.15.73:3300/0 conn(0x7f040c0b0220 msgr2=0x7f040c0b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.533+0000 7f0411200640 1 --2- 172.21.15.46:0/608852762 >> v2:172.21.15.73:3300/0 conn(0x7f040c0b0220 0x7f040c0b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.533+0000 7f0411200640 1 -- 172.21.15.46:0/608852762 >> v1:172.21.15.46:6789/0 conn(0x7f040c15a6b0 legacy=0x7f040c15b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.533+0000 7f0411200640 1 -- 172.21.15.46:0/608852762 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c0b0b60 con 0x7f040c15e160 2024-03-29T08:29:19.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.533+0000 7f0411200640 1 --2- 172.21.15.46:0/608852762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f040c15e160 0x7f040c0caf30 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f040c15bcb0 tx=0x7f03f801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0a808764d242f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f040be00640 1 -- 172.21.15.46:0/608852762 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f80aa680 con 0x7f040c15e160 2024-03-29T08:29:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f040be00640 1 -- 172.21.15.46:0/608852762 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f80aa7e0 con 0x7f040c15e160 2024-03-29T08:29:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f040be00640 1 -- 172.21.15.46:0/608852762 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f80aab70 con 0x7f040c15e160 2024-03-29T08:29:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0413000640 1 -- 172.21.15.46:0/608852762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f040c15e160 msgr2=0x7f040c0caf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0413000640 1 --2- 172.21.15.46:0/608852762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f040c15e160 0x7f040c0caf30 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f040c15bcb0 tx=0x7f03f801c520 comp rx=0 tx=0).stop 2024-03-29T08:29:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0413000640 1 -- 172.21.15.46:0/608852762 shutdown_connections 2024-03-29T08:29:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0413000640 1 --2- 172.21.15.46:0/608852762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f040c15e160 0x7f040c0caf30 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0413000640 1 --2- 172.21.15.46:0/608852762 >> v2:172.21.15.73:3300/0 conn(0x7f040c0b0220 0x7f040c0b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0413000640 1 -- 172.21.15.46:0/608852762 >> 172.21.15.46:0/608852762 conn(0x7f040c098a40 msgr2=0x7f040c0cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0413000640 1 -- 172.21.15.46:0/608852762 shutdown_connections 2024-03-29T08:29:19.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0413000640 1 -- 172.21.15.46:0/608852762 wait complete. 2024-03-29T08:29:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0413000640 1 Processor -- start 2024-03-29T08:29:19.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0413000640 1 -- start start 2024-03-29T08:29:19.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0413000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f040c0b0220 0x7f040c159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0413000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f040c15a6b0 0x7f040c15d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0413000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f040c15e160 0x7f040c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0413000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c0c4430 con 0x7f040c0b0220 2024-03-29T08:29:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0413000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c15a0a0 con 0x7f040c15e160 2024-03-29T08:29:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0413000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040c15d540 con 0x7f040c15a6b0 2024-03-29T08:29:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0411c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f040c0b0220 0x7f040c159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0411c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f040c0b0220 0x7f040c159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39068/0 (socket says 172.21.15.46:39068) 2024-03-29T08:29:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0411c00640 1 -- 172.21.15.46:0/2337938264 learned_addr learned my addr 172.21.15.46:0/2337938264 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0412600640 1 --2- 172.21.15.46:0/2337938264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f040c15e160 0x7f040c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.537+0000 7f0411200640 1 --2- 172.21.15.46:0/2337938264 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f040c15a6b0 0x7f040c15d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.541+0000 7f0411c00640 1 -- 172.21.15.46:0/2337938264 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f040c15a6b0 msgr2=0x7f040c15d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.541+0000 7f0411c00640 1 --2- 172.21.15.46:0/2337938264 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f040c15a6b0 0x7f040c15d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.541+0000 7f0411c00640 1 -- 172.21.15.46:0/2337938264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f040c15e160 msgr2=0x7f040c06e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.541+0000 7f0411c00640 1 --2- 172.21.15.46:0/2337938264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f040c15e160 0x7f040c06e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.541+0000 7f0411c00640 1 -- 172.21.15.46:0/2337938264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f80094d0 con 0x7f040c0b0220 2024-03-29T08:29:19.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.541+0000 7f0411c00640 1 --2- 172.21.15.46:0/2337938264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f040c0b0220 0x7f040c159b60 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f040000c500 tx=0x7f040000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:19.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.541+0000 7f0409600640 1 -- 172.21.15.46:0/2337938264 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0400005220 con 0x7f040c0b0220 2024-03-29T08:29:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.541+0000 7f0409600640 1 -- 172.21.15.46:0/2337938264 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0400005380 con 0x7f040c0b0220 2024-03-29T08:29:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.541+0000 7f0413000640 1 -- 172.21.15.46:0/2337938264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f040c0b4e30 con 0x7f040c0b0220 2024-03-29T08:29:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.541+0000 7f0409600640 1 -- 172.21.15.46:0/2337938264 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0400007270 con 0x7f040c0b0220 2024-03-29T08:29:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.541+0000 7f0413000640 1 -- 172.21.15.46:0/2337938264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f040c0b4500 con 0x7f040c0b0220 2024-03-29T08:29:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.541+0000 7f0413000640 1 -- 172.21.15.46:0/2337938264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040c04e040 con 0x7f040c0b0220 2024-03-29T08:29:19.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.541+0000 7f0409600640 1 -- 172.21.15.46:0/2337938264 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f04000073d0 con 0x7f040c0b0220 2024-03-29T08:29:19.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.541+0000 7f0409600640 1 --2- 172.21.15.46:0/2337938264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f03ec0666c0 0x7f03ec068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.541+0000 7f0409600640 1 -- 172.21.15.46:0/2337938264 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0400083980 con 0x7f040c0b0220 2024-03-29T08:29:19.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.545+0000 7f0411200640 1 --2- 172.21.15.46:0/2337938264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f03ec0666c0 0x7f03ec068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.545+0000 7f0411200640 1 --2- 172.21.15.46:0/2337938264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f03ec0666c0 0x7f03ec068b80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f03f80ae000 tx=0x7f03f80b5040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:19.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.545+0000 7f0409600640 1 -- 172.21.15.46:0/2337938264 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0400050d10 con 0x7f040c0b0220 2024-03-29T08:29:19.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.765+0000 7f0413000640 1 -- 172.21.15.46:0/2337938264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f040c0d00d0 con 0x7f040c0b0220 2024-03-29T08:29:19.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.765+0000 7f0409600640 1 -- 172.21.15.46:0/2337938264 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1261 (secure 0 0 0) 0x7f0400054330 con 0x7f040c0b0220 2024-03-29T08:29:19.769 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:19.769 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":691,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":121,"num_osds":8,"num_up_osds":8,"osd_up_since":1711700279,"num_in_osds":8,"osd_in_since":1711700270,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":89},{"state_name":"creating+peering","count":8}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":592708,"bytes_used":268021760,"bytes_avail":772826091520,"bytes_total":773094113280,"inactive_pgs_ratio":0.082474224269390106,"write_bytes_sec":2585,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":94,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":9066}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":31,"modified":"2024-03-29T08:28:28.199940+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-29T08:29:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.765+0000 7f0413000640 1 -- 172.21.15.46:0/2337938264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f03ec0666c0 msgr2=0x7f03ec068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.765+0000 7f0413000640 1 --2- 172.21.15.46:0/2337938264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f03ec0666c0 0x7f03ec068b80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f03f80ae000 tx=0x7f03f80b5040 comp rx=0 tx=0).stop 2024-03-29T08:29:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.765+0000 7f0413000640 1 -- 172.21.15.46:0/2337938264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f040c0b0220 msgr2=0x7f040c159b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.765+0000 7f0413000640 1 --2- 172.21.15.46:0/2337938264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f040c0b0220 0x7f040c159b60 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f040000c500 tx=0x7f040000c9c0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.765+0000 7f0413000640 1 -- 172.21.15.46:0/2337938264 shutdown_connections 2024-03-29T08:29:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.765+0000 7f0413000640 1 --2- 172.21.15.46:0/2337938264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f03ec0666c0 0x7f03ec068b80 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.765+0000 7f0413000640 1 --2- 172.21.15.46:0/2337938264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f040c15e160 0x7f040c06e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.765+0000 7f0413000640 1 --2- 172.21.15.46:0/2337938264 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f040c15a6b0 0x7f040c15d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.765+0000 7f0413000640 1 --2- 172.21.15.46:0/2337938264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f040c0b0220 0x7f040c159b60 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.765+0000 7f0413000640 1 -- 172.21.15.46:0/2337938264 >> 172.21.15.46:0/2337938264 conn(0x7f040c098a40 msgr2=0x7f040c164d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.765+0000 7f0413000640 1 -- 172.21.15.46:0/2337938264 shutdown_connections 2024-03-29T08:29:19.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.765+0000 7f0413000640 1 -- 172.21.15.46:0/2337938264 wait complete. 2024-03-29T08:29:19.789 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-29T08:29:19.790 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:29:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.925+0000 7f9c68c00640 1 Processor -- start 2024-03-29T08:29:19.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.925+0000 7f9c68c00640 1 -- start start 2024-03-29T08:29:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.925+0000 7f9c68c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c6415a7c0 0x7f9c6406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.925+0000 7f9c68c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c6406d130 con 0x7f9c641593d0 2024-03-29T08:29:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.925+0000 7f9c68c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c6415b8f0 con 0x7f9c6415cd90 2024-03-29T08:29:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.925+0000 7f9c68c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6415ad50 con 0x7f9c6415a7c0 2024-03-29T08:29:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.925+0000 7f9c61e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9c641593d0 0x7f9c6415d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48220/0 (socket says 172.21.15.46:48220) 2024-03-29T08:29:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.925+0000 7f9c61e00640 1 -- 172.21.15.46:0/579958919 learned_addr learned my addr 172.21.15.46:0/579958919 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:19.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.925+0000 7f9c62800640 1 --2- 172.21.15.46:0/579958919 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c6415a7c0 0x7f9c6406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.925+0000 7f9c61400640 1 -- 172.21.15.46:0/579958919 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3373012230 0 0) 0x7f9c6406d130 con 0x7f9c641593d0 2024-03-29T08:29:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.925+0000 7f9c61400640 1 -- 172.21.15.46:0/579958919 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c48005740 con 0x7f9c641593d0 2024-03-29T08:29:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 -- 172.21.15.46:0/579958919 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 180069223 0 0) 0x7f9c6415b8f0 con 0x7f9c6415cd90 2024-03-29T08:29:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 -- 172.21.15.46:0/579958919 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c6406d130 con 0x7f9c6415cd90 2024-03-29T08:29:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 -- 172.21.15.46:0/579958919 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3613643180 0 0) 0x7f9c48005740 con 0x7f9c641593d0 2024-03-29T08:29:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 -- 172.21.15.46:0/579958919 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c6415a7c0 msgr2=0x7f9c6406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:19.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 --2- 172.21.15.46:0/579958919 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c6415a7c0 0x7f9c6406cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 -- 172.21.15.46:0/579958919 >> v1:172.21.15.73:6789/0 conn(0x7f9c6415cd90 legacy=0x7f9c6415d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 -- 172.21.15.46:0/579958919 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6415e050 con 0x7f9c641593d0 2024-03-29T08:29:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 -- 172.21.15.46:0/579958919 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f9c54001a00 con 0x7f9c641593d0 2024-03-29T08:29:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 -- 172.21.15.46:0/579958919 >> v1:172.21.15.46:6789/0 conn(0x7f9c641593d0 legacy=0x7f9c6415d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 --2- 172.21.15.46:0/579958919 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c6415a7c0 0x7f9c48005d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 --2- 172.21.15.46:0/579958919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c48006300 0x7f9c480069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 --2- 172.21.15.46:0/579958919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c48006f20 0x7f9c480073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 -- 172.21.15.46:0/579958919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6415ad50 con 0x7f9c48006300 2024-03-29T08:29:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 -- 172.21.15.46:0/579958919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4800dce0 con 0x7f9c48006f20 2024-03-29T08:29:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 -- 172.21.15.46:0/579958919 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4800dea0 con 0x7f9c6415a7c0 2024-03-29T08:29:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c59400640 1 --2- 172.21.15.46:0/579958919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c48006300 0x7f9c480069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61e00640 1 --2- 172.21.15.46:0/579958919 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c6415a7c0 0x7f9c48005d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c62800640 1 --2- 172.21.15.46:0/579958919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c48006f20 0x7f9c480073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c59400640 1 -- 172.21.15.46:0/579958919 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c6415a7c0 msgr2=0x7f9c48005d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:19.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c59400640 1 --2- 172.21.15.46:0/579958919 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c6415a7c0 0x7f9c48005d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c59400640 1 -- 172.21.15.46:0/579958919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c48006f20 msgr2=0x7f9c480073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:19.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c59400640 1 --2- 172.21.15.46:0/579958919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c48006f20 0x7f9c480073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c59400640 1 -- 172.21.15.46:0/579958919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c48005740 con 0x7f9c48006300 2024-03-29T08:29:19.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c62800640 1 --2- 172.21.15.46:0/579958919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c48006f20 0x7f9c480073a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c59400640 1 --2- 172.21.15.46:0/579958919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c48006300 0x7f9c480069e0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0018d0 tx=0x7f9c4c01ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf155cc6caaa0b6e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 -- 172.21.15.46:0/579958919 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0052d0 con 0x7f9c48006300 2024-03-29T08:29:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 -- 172.21.15.46:0/579958919 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c005450 con 0x7f9c48006300 2024-03-29T08:29:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.929+0000 7f9c61400640 1 -- 172.21.15.46:0/579958919 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0a8890 con 0x7f9c48006300 2024-03-29T08:29:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c68c00640 1 -- 172.21.15.46:0/579958919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c48006300 msgr2=0x7f9c480069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c68c00640 1 --2- 172.21.15.46:0/579958919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c48006300 0x7f9c480069e0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0018d0 tx=0x7f9c4c01ec30 comp rx=0 tx=0).stop 2024-03-29T08:29:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c62800640 1 -- 172.21.15.46:0/579958919 reap_dead start 2024-03-29T08:29:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c68c00640 1 -- 172.21.15.46:0/579958919 shutdown_connections 2024-03-29T08:29:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c68c00640 1 -- 172.21.15.46:0/579958919 >> 172.21.15.46:0/579958919 conn(0x7f9c64066110 msgr2=0x7f9c64069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c68c00640 1 -- 172.21.15.46:0/579958919 shutdown_connections 2024-03-29T08:29:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c68c00640 1 -- 172.21.15.46:0/579958919 wait complete. 2024-03-29T08:29:19.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c68c00640 1 Processor -- start 2024-03-29T08:29:19.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c68c00640 1 -- start start 2024-03-29T08:29:19.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c68c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c48006300 0x7f9c64152060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c68c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c641570b0 0x7f9c641ee970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c68c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c641f1550 0x7f9c641529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c68c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c641edf20 con 0x7f9c641570b0 2024-03-29T08:29:19.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c68c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c641ee4e0 con 0x7f9c641f1550 2024-03-29T08:29:19.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c68c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c641525a0 con 0x7f9c48006300 2024-03-29T08:29:19.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c59400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c641570b0 0x7f9c641ee970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c59400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c641570b0 0x7f9c641ee970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39094/0 (socket says 172.21.15.46:39094) 2024-03-29T08:29:19.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c59400640 1 -- 172.21.15.46:0/60148859 learned_addr learned my addr 172.21.15.46:0/60148859 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:19.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c61e00640 1 --2- 172.21.15.46:0/60148859 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c48006300 0x7f9c64152060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c62800640 1 --2- 172.21.15.46:0/60148859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c641f1550 0x7f9c641529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c59400640 1 -- 172.21.15.46:0/60148859 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c48006300 msgr2=0x7f9c64152060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c59400640 1 --2- 172.21.15.46:0/60148859 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c48006300 0x7f9c64152060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c59400640 1 -- 172.21.15.46:0/60148859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c641f1550 msgr2=0x7f9c641529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c59400640 1 --2- 172.21.15.46:0/60148859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c641f1550 0x7f9c641529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.933+0000 7f9c59400640 1 -- 172.21.15.46:0/60148859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c641eeeb0 con 0x7f9c641570b0 2024-03-29T08:29:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.937+0000 7f9c59400640 1 --2- 172.21.15.46:0/60148859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c641570b0 0x7f9c641ee970 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0018d0 tx=0x7f9c48007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.937+0000 7f9c5aa00640 1 -- 172.21.15.46:0/60148859 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0a8d80 con 0x7f9c641570b0 2024-03-29T08:29:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.937+0000 7f9c5aa00640 1 -- 172.21.15.46:0/60148859 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c0050d0 con 0x7f9c641570b0 2024-03-29T08:29:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.937+0000 7f9c68c00640 1 -- 172.21.15.46:0/60148859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c641f1f30 con 0x7f9c641570b0 2024-03-29T08:29:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.937+0000 7f9c5aa00640 1 -- 172.21.15.46:0/60148859 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c003600 con 0x7f9c641570b0 2024-03-29T08:29:19.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.937+0000 7f9c68c00640 1 -- 172.21.15.46:0/60148859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c641ee170 con 0x7f9c641570b0 2024-03-29T08:29:19.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.937+0000 7f9c68c00640 1 -- 172.21.15.46:0/60148859 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c641ee650 con 0x7f9c641570b0 2024-03-29T08:29:19.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.937+0000 7f9c5aa00640 1 -- 172.21.15.46:0/60148859 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9c4c003760 con 0x7f9c641570b0 2024-03-29T08:29:19.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.937+0000 7f9c5aa00640 1 --2- 172.21.15.46:0/60148859 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c400666c0 0x7f9c40068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:19.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.937+0000 7f9c5aa00640 1 -- 172.21.15.46:0/60148859 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c4c1211d0 con 0x7f9c641570b0 2024-03-29T08:29:19.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.937+0000 7f9c61e00640 1 --2- 172.21.15.46:0/60148859 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c400666c0 0x7f9c40068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:19.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.941+0000 7f9c61e00640 1 --2- 172.21.15.46:0/60148859 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c400666c0 0x7f9c40068b80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9c540013e0 tx=0x7f9c54006040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:19.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:19.941+0000 7f9c5aa00640 1 -- 172.21.15.46:0/60148859 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9c4c0ee4a0 con 0x7f9c641570b0 2024-03-29T08:29:20.047 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-03-29T08:29:20.041+0000 7f7d71200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:29:20.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:20.101+0000 7f9c68c00640 1 -- 172.21.15.46:0/60148859 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c6415b580 con 0x7f9c400666c0 2024-03-29T08:29:20.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:20.109+0000 7f9c5aa00640 1 -- 172.21.15.46:0/60148859 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289105 (secure 0 0 0) 0x7f9c54001920 con 0x7f9c400666c0 2024-03-29T08:29:20.117 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:20.119 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:29:20.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:20.113+0000 7f9c68c00640 1 -- 172.21.15.46:0/60148859 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c400666c0 msgr2=0x7f9c40068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:20.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:20.113+0000 7f9c68c00640 1 --2- 172.21.15.46:0/60148859 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c400666c0 0x7f9c40068b80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9c540013e0 tx=0x7f9c54006040 comp rx=0 tx=0).stop 2024-03-29T08:29:20.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:20.113+0000 7f9c68c00640 1 -- 172.21.15.46:0/60148859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c641570b0 msgr2=0x7f9c641ee970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:20.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:20.113+0000 7f9c68c00640 1 --2- 172.21.15.46:0/60148859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c641570b0 0x7f9c641ee970 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c0018d0 tx=0x7f9c48007a10 comp rx=0 tx=0).stop 2024-03-29T08:29:20.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:20.113+0000 7f9c68c00640 1 -- 172.21.15.46:0/60148859 shutdown_connections 2024-03-29T08:29:20.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:20.113+0000 7f9c68c00640 1 --2- 172.21.15.46:0/60148859 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c400666c0 0x7f9c40068b80 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:20.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:20.113+0000 7f9c68c00640 1 --2- 172.21.15.46:0/60148859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c641f1550 0x7f9c641529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:20.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:20.113+0000 7f9c68c00640 1 --2- 172.21.15.46:0/60148859 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9c641570b0 0x7f9c641ee970 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:20.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:20.113+0000 7f9c68c00640 1 --2- 172.21.15.46:0/60148859 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9c48006300 0x7f9c64152060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:20.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:20.113+0000 7f9c68c00640 1 -- 172.21.15.46:0/60148859 >> 172.21.15.46:0/60148859 conn(0x7f9c64066110 msgr2=0x7f9c640ae6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:20.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:20.113+0000 7f9c68c00640 1 -- 172.21.15.46:0/60148859 shutdown_connections 2024-03-29T08:29:20.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:20.113+0000 7f9c68c00640 1 -- 172.21.15.46:0/60148859 wait complete. 2024-03-29T08:29:20.136 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-03-29T08:29:18.211688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":202,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154,"ondisk_log_size":154,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261740,"kb_used_data":12764,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754712980,"statfs":{"total":773094113280,"available":772826091520,"internally_reserved":0,"allocated":13070336,"data_stored":6325748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.148834"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509634+0000","last_change":"2024-03-29T08:29:14.504298+0000","last_active":"2024-03-29T08:29:14.509634+0000","last_peered":"2024-03-29T08:29:14.509634+0000","last_clean":"2024-03-29T08:29:14.509634+0000","last_became_active":"2024-03-29T08:29:14.504060+0000","last_became_peered":"2024-03-29T08:29:14.504060+0000","last_unstale":"2024-03-29T08:29:14.509634+0000","last_undegraded":"2024-03-29T08:29:14.509634+0000","last_fullsized":"2024-03-29T08:29:14.509634+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:47:24.542958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.491055+0000","last_change":"2024-03-29T08:29:14.488152+0000","last_active":"2024-03-29T08:29:14.491055+0000","last_peered":"2024-03-29T08:29:14.491055+0000","last_clean":"2024-03-29T08:29:14.491055+0000","last_became_active":"2024-03-29T08:29:14.487884+0000","last_became_peered":"2024-03-29T08:29:14.487884+0000","last_unstale":"2024-03-29T08:29:14.491055+0000","last_undegraded":"2024-03-29T08:29:14.491055+0000","last_fullsized":"2024-03-29T08:29:14.491055+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:49:14.237508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.492505+0000","last_change":"2024-03-29T08:29:14.490304+0000","last_active":"2024-03-29T08:29:14.492505+0000","last_peered":"2024-03-29T08:29:14.492505+0000","last_clean":"2024-03-29T08:29:14.492505+0000","last_became_active":"2024-03-29T08:29:14.489916+0000","last_became_peered":"2024-03-29T08:29:14.489916+0000","last_unstale":"2024-03-29T08:29:14.492505+0000","last_undegraded":"2024-03-29T08:29:14.492505+0000","last_fullsized":"2024-03-29T08:29:14.492505+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:55.868596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.510337+0000","last_change":"2024-03-29T08:29:14.501687+0000","last_active":"2024-03-29T08:29:14.510337+0000","last_peered":"2024-03-29T08:29:14.510337+0000","last_clean":"2024-03-29T08:29:14.510337+0000","last_became_active":"2024-03-29T08:29:14.501377+0000","last_became_peered":"2024-03-29T08:29:14.501377+0000","last_unstale":"2024-03-29T08:29:14.510337+0000","last_undegraded":"2024-03-29T08:29:14.510337+0000","last_fullsized":"2024-03-29T08:29:14.510337+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:38:29.985399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.506541+0000","last_change":"2024-03-29T08:29:14.504615+0000","last_active":"2024-03-29T08:29:14.506541+0000","last_peered":"2024-03-29T08:29:14.506541+0000","last_clean":"2024-03-29T08:29:14.506541+0000","last_became_active":"2024-03-29T08:29:14.504347+0000","last_became_peered":"2024-03-29T08:29:14.504347+0000","last_unstale":"2024-03-29T08:29:14.506541+0000","last_undegraded":"2024-03-29T08:29:14.506541+0000","last_fullsized":"2024-03-29T08:29:14.506541+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:25:45.763218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.084660+0000","last_change":"2024-03-29T08:29:14.084660+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.084660+0000","last_undegraded":"2024-03-29T08:29:14.084660+0000","last_fullsized":"2024-03-29T08:29:14.084660+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:51.101298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.503717+0000","last_change":"2024-03-29T08:29:14.494643+0000","last_active":"2024-03-29T08:29:14.503717+0000","last_peered":"2024-03-29T08:29:14.503717+0000","last_clean":"2024-03-29T08:29:14.503717+0000","last_became_active":"2024-03-29T08:29:14.494385+0000","last_became_peered":"2024-03-29T08:29:14.494385+0000","last_unstale":"2024-03-29T08:29:14.503717+0000","last_undegraded":"2024-03-29T08:29:14.503717+0000","last_fullsized":"2024-03-29T08:29:14.503717+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:11:26.639183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.495613+0000","last_change":"2024-03-29T08:29:14.489843+0000","last_active":"2024-03-29T08:29:14.495613+0000","last_peered":"2024-03-29T08:29:14.495613+0000","last_clean":"2024-03-29T08:29:14.495613+0000","last_became_active":"2024-03-29T08:29:14.489598+0000","last_became_peered":"2024-03-29T08:29:14.489598+0000","last_unstale":"2024-03-29T08:29:14.495613+0000","last_undegraded":"2024-03-29T08:29:14.495613+0000","last_fullsized":"2024-03-29T08:29:14.495613+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:49:19.274668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.512231+0000","last_change":"2024-03-29T08:29:14.507881+0000","last_active":"2024-03-29T08:29:14.512231+0000","last_peered":"2024-03-29T08:29:14.512231+0000","last_clean":"2024-03-29T08:29:14.512231+0000","last_became_active":"2024-03-29T08:29:14.507445+0000","last_became_peered":"2024-03-29T08:29:14.507445+0000","last_unstale":"2024-03-29T08:29:14.512231+0000","last_undegraded":"2024-03-29T08:29:14.512231+0000","last_fullsized":"2024-03-29T08:29:14.512231+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:50:39.114131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504217+0000","last_change":"2024-03-29T08:29:14.500130+0000","last_active":"2024-03-29T08:29:14.504217+0000","last_peered":"2024-03-29T08:29:14.504217+0000","last_clean":"2024-03-29T08:29:14.504217+0000","last_became_active":"2024-03-29T08:29:14.499827+0000","last_became_peered":"2024-03-29T08:29:14.499827+0000","last_unstale":"2024-03-29T08:29:14.504217+0000","last_undegraded":"2024-03-29T08:29:14.504217+0000","last_fullsized":"2024-03-29T08:29:14.504217+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:01.476642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.502452+0000","last_change":"2024-03-29T08:29:14.494953+0000","last_active":"2024-03-29T08:29:14.502452+0000","last_peered":"2024-03-29T08:29:14.502452+0000","last_clean":"2024-03-29T08:29:14.502452+0000","last_became_active":"2024-03-29T08:29:14.494599+0000","last_became_peered":"2024-03-29T08:29:14.494599+0000","last_unstale":"2024-03-29T08:29:14.502452+0000","last_undegraded":"2024-03-29T08:29:14.502452+0000","last_fullsized":"2024-03-29T08:29:14.502452+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:49:02.417397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504246+0000","last_change":"2024-03-29T08:29:14.499160+0000","last_active":"2024-03-29T08:29:14.504246+0000","last_peered":"2024-03-29T08:29:14.504246+0000","last_clean":"2024-03-29T08:29:14.504246+0000","last_became_active":"2024-03-29T08:29:14.498900+0000","last_became_peered":"2024-03-29T08:29:14.498900+0000","last_unstale":"2024-03-29T08:29:14.504246+0000","last_undegraded":"2024-03-29T08:29:14.504246+0000","last_fullsized":"2024-03-29T08:29:14.504246+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:19.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.493310+0000","last_change":"2024-03-29T08:29:14.486530+0000","last_active":"2024-03-29T08:29:14.493310+0000","last_peered":"2024-03-29T08:29:14.493310+0000","last_clean":"2024-03-29T08:29:14.493310+0000","last_became_active":"2024-03-29T08:29:14.486158+0000","last_became_peered":"2024-03-29T08:29:14.486158+0000","last_unstale":"2024-03-29T08:29:14.493310+0000","last_undegraded":"2024-03-29T08:29:14.493310+0000","last_fullsized":"2024-03-29T08:29:14.493310+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:40.282997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508861+0000","last_change":"2024-03-29T08:29:14.507343+0000","last_active":"2024-03-29T08:29:14.508861+0000","last_peered":"2024-03-29T08:29:14.508861+0000","last_clean":"2024-03-29T08:29:14.508861+0000","last_became_active":"2024-03-29T08:29:14.507072+0000","last_became_peered":"2024-03-29T08:29:14.507072+0000","last_unstale":"2024-03-29T08:29:14.508861+0000","last_undegraded":"2024-03-29T08:29:14.508861+0000","last_fullsized":"2024-03-29T08:29:14.508861+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:19.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509523+0000","last_change":"2024-03-29T08:29:14.498131+0000","last_active":"2024-03-29T08:29:14.509523+0000","last_peered":"2024-03-29T08:29:14.509523+0000","last_clean":"2024-03-29T08:29:14.509523+0000","last_became_active":"2024-03-29T08:29:14.497856+0000","last_became_peered":"2024-03-29T08:29:14.497856+0000","last_unstale":"2024-03-29T08:29:14.509523+0000","last_undegraded":"2024-03-29T08:29:14.509523+0000","last_fullsized":"2024-03-29T08:29:14.509523+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:43:24.083895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504685+0000","last_change":"2024-03-29T08:29:14.499902+0000","last_active":"2024-03-29T08:29:14.504685+0000","last_peered":"2024-03-29T08:29:14.504685+0000","last_clean":"2024-03-29T08:29:14.504685+0000","last_became_active":"2024-03-29T08:29:14.499642+0000","last_became_peered":"2024-03-29T08:29:14.499642+0000","last_unstale":"2024-03-29T08:29:14.504685+0000","last_undegraded":"2024-03-29T08:29:14.504685+0000","last_fullsized":"2024-03-29T08:29:14.504685+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:06:54.374100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.502730+0000","last_change":"2024-03-29T08:29:14.496277+0000","last_active":"2024-03-29T08:29:14.502730+0000","last_peered":"2024-03-29T08:29:14.502730+0000","last_clean":"2024-03-29T08:29:14.502730+0000","last_became_active":"2024-03-29T08:29:14.496028+0000","last_became_peered":"2024-03-29T08:29:14.496028+0000","last_unstale":"2024-03-29T08:29:14.502730+0000","last_undegraded":"2024-03-29T08:29:14.502730+0000","last_fullsized":"2024-03-29T08:29:14.502730+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:08:59.334138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505390+0000","last_change":"2024-03-29T08:29:14.503103+0000","last_active":"2024-03-29T08:29:14.505390+0000","last_peered":"2024-03-29T08:29:14.505390+0000","last_clean":"2024-03-29T08:29:14.505390+0000","last_became_active":"2024-03-29T08:29:14.502841+0000","last_became_peered":"2024-03-29T08:29:14.502841+0000","last_unstale":"2024-03-29T08:29:14.505390+0000","last_undegraded":"2024-03-29T08:29:14.505390+0000","last_fullsized":"2024-03-29T08:29:14.505390+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:11:21.620572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.498334+0000","last_change":"2024-03-29T08:29:14.491176+0000","last_active":"2024-03-29T08:29:14.498334+0000","last_peered":"2024-03-29T08:29:14.498334+0000","last_clean":"2024-03-29T08:29:14.498334+0000","last_became_active":"2024-03-29T08:29:14.490845+0000","last_became_peered":"2024-03-29T08:29:14.490845+0000","last_unstale":"2024-03-29T08:29:14.498334+0000","last_undegraded":"2024-03-29T08:29:14.498334+0000","last_fullsized":"2024-03-29T08:29:14.498334+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:44:41.312640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504227+0000","last_change":"2024-03-29T08:29:14.501383+0000","last_active":"2024-03-29T08:29:14.504227+0000","last_peered":"2024-03-29T08:29:14.504227+0000","last_clean":"2024-03-29T08:29:14.504227+0000","last_became_active":"2024-03-29T08:29:14.501106+0000","last_became_peered":"2024-03-29T08:29:14.501106+0000","last_unstale":"2024-03-29T08:29:14.504227+0000","last_undegraded":"2024-03-29T08:29:14.504227+0000","last_fullsized":"2024-03-29T08:29:14.504227+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:23:10.605671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504862+0000","last_change":"2024-03-29T08:29:14.497743+0000","last_active":"2024-03-29T08:29:14.504862+0000","last_peered":"2024-03-29T08:29:14.504862+0000","last_clean":"2024-03-29T08:29:14.504862+0000","last_became_active":"2024-03-29T08:29:14.497338+0000","last_became_peered":"2024-03-29T08:29:14.497338+0000","last_unstale":"2024-03-29T08:29:14.504862+0000","last_undegraded":"2024-03-29T08:29:14.504862+0000","last_fullsized":"2024-03-29T08:29:14.504862+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:24.513091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.084054+0000","last_change":"2024-03-29T08:29:14.084054+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.084054+0000","last_undegraded":"2024-03-29T08:29:14.084054+0000","last_fullsized":"2024-03-29T08:29:14.084054+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:57:23.884319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.506608+0000","last_change":"2024-03-29T08:29:14.502935+0000","last_active":"2024-03-29T08:29:14.506608+0000","last_peered":"2024-03-29T08:29:14.506608+0000","last_clean":"2024-03-29T08:29:14.506608+0000","last_became_active":"2024-03-29T08:29:14.502701+0000","last_became_peered":"2024-03-29T08:29:14.502701+0000","last_unstale":"2024-03-29T08:29:14.506608+0000","last_undegraded":"2024-03-29T08:29:14.506608+0000","last_fullsized":"2024-03-29T08:29:14.506608+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:35.756569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.511647+0000","last_change":"2024-03-29T08:29:14.507544+0000","last_active":"2024-03-29T08:29:14.511647+0000","last_peered":"2024-03-29T08:29:14.511647+0000","last_clean":"2024-03-29T08:29:14.511647+0000","last_became_active":"2024-03-29T08:29:14.507285+0000","last_became_peered":"2024-03-29T08:29:14.507285+0000","last_unstale":"2024-03-29T08:29:14.511647+0000","last_undegraded":"2024-03-29T08:29:14.511647+0000","last_fullsized":"2024-03-29T08:29:14.511647+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:50:14.687728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.494822+0000","last_change":"2024-03-29T08:29:14.492853+0000","last_active":"2024-03-29T08:29:14.494822+0000","last_peered":"2024-03-29T08:29:14.494822+0000","last_clean":"2024-03-29T08:29:14.494822+0000","last_became_active":"2024-03-29T08:29:14.492537+0000","last_became_peered":"2024-03-29T08:29:14.492537+0000","last_unstale":"2024-03-29T08:29:14.494822+0000","last_undegraded":"2024-03-29T08:29:14.494822+0000","last_fullsized":"2024-03-29T08:29:14.494822+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:08:28.043449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.085680+0000","last_change":"2024-03-29T08:29:14.085680+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.085680+0000","last_undegraded":"2024-03-29T08:29:14.085680+0000","last_fullsized":"2024-03-29T08:29:14.085680+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:10:44.097211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.511774+0000","last_change":"2024-03-29T08:29:14.508869+0000","last_active":"2024-03-29T08:29:14.511774+0000","last_peered":"2024-03-29T08:29:14.511774+0000","last_clean":"2024-03-29T08:29:14.511774+0000","last_became_active":"2024-03-29T08:29:14.508466+0000","last_became_peered":"2024-03-29T08:29:14.508466+0000","last_unstale":"2024-03-29T08:29:14.511774+0000","last_undegraded":"2024-03-29T08:29:14.511774+0000","last_fullsized":"2024-03-29T08:29:14.511774+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:08:28.619413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.087721+0000","last_change":"2024-03-29T08:29:14.087721+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.087721+0000","last_undegraded":"2024-03-29T08:29:14.087721+0000","last_fullsized":"2024-03-29T08:29:14.087721+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:53.749284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508906+0000","last_change":"2024-03-29T08:29:14.507369+0000","last_active":"2024-03-29T08:29:14.508906+0000","last_peered":"2024-03-29T08:29:14.508906+0000","last_clean":"2024-03-29T08:29:14.508906+0000","last_became_active":"2024-03-29T08:29:14.507164+0000","last_became_peered":"2024-03-29T08:29:14.507164+0000","last_unstale":"2024-03-29T08:29:14.508906+0000","last_undegraded":"2024-03-29T08:29:14.508906+0000","last_fullsized":"2024-03-29T08:29:14.508906+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:20:13.956872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505661+0000","last_change":"2024-03-29T08:29:14.503526+0000","last_active":"2024-03-29T08:29:14.505661+0000","last_peered":"2024-03-29T08:29:14.505661+0000","last_clean":"2024-03-29T08:29:14.505661+0000","last_became_active":"2024-03-29T08:29:14.503149+0000","last_became_peered":"2024-03-29T08:29:14.503149+0000","last_unstale":"2024-03-29T08:29:14.505661+0000","last_undegraded":"2024-03-29T08:29:14.505661+0000","last_fullsized":"2024-03-29T08:29:14.505661+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:46:13.948298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509541+0000","last_change":"2024-03-29T08:29:14.500403+0000","last_active":"2024-03-29T08:29:14.509541+0000","last_peered":"2024-03-29T08:29:14.509541+0000","last_clean":"2024-03-29T08:29:14.509541+0000","last_became_active":"2024-03-29T08:29:14.500132+0000","last_became_peered":"2024-03-29T08:29:14.500132+0000","last_unstale":"2024-03-29T08:29:14.509541+0000","last_undegraded":"2024-03-29T08:29:14.509541+0000","last_fullsized":"2024-03-29T08:29:14.509541+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:09:34.129852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504668+0000","last_change":"2024-03-29T08:29:14.499479+0000","last_active":"2024-03-29T08:29:14.504668+0000","last_peered":"2024-03-29T08:29:14.504668+0000","last_clean":"2024-03-29T08:29:14.504668+0000","last_became_active":"2024-03-29T08:29:14.498994+0000","last_became_peered":"2024-03-29T08:29:14.498994+0000","last_unstale":"2024-03-29T08:29:14.504668+0000","last_undegraded":"2024-03-29T08:29:14.504668+0000","last_fullsized":"2024-03-29T08:29:14.504668+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:25:13.598107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.513004+0000","last_change":"2024-03-29T08:29:14.511693+0000","last_active":"2024-03-29T08:29:14.513004+0000","last_peered":"2024-03-29T08:29:14.513004+0000","last_clean":"2024-03-29T08:29:14.513004+0000","last_became_active":"2024-03-29T08:29:14.511041+0000","last_became_peered":"2024-03-29T08:29:14.511041+0000","last_unstale":"2024-03-29T08:29:14.513004+0000","last_undegraded":"2024-03-29T08:29:14.513004+0000","last_fullsized":"2024-03-29T08:29:14.513004+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:06.874362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"120'4","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507837+0000","last_change":"2024-03-29T08:29:14.086060+0000","last_active":"2024-03-29T08:29:14.507837+0000","last_peered":"2024-03-29T08:29:14.507837+0000","last_clean":"2024-03-29T08:29:14.507837+0000","last_became_active":"2024-03-29T08:29:14.085788+0000","last_became_peered":"2024-03-29T08:29:14.085788+0000","last_unstale":"2024-03-29T08:29:14.507837+0000","last_undegraded":"2024-03-29T08:29:14.507837+0000","last_fullsized":"2024-03-29T08:29:14.507837+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:52:04.161387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507246+0000","last_change":"2024-03-29T08:29:14.503865+0000","last_active":"2024-03-29T08:29:14.507246+0000","last_peered":"2024-03-29T08:29:14.507246+0000","last_clean":"2024-03-29T08:29:14.507246+0000","last_became_active":"2024-03-29T08:29:14.503622+0000","last_became_peered":"2024-03-29T08:29:14.503622+0000","last_unstale":"2024-03-29T08:29:14.507246+0000","last_undegraded":"2024-03-29T08:29:14.507246+0000","last_fullsized":"2024-03-29T08:29:14.507246+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:48:23.465613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.501135+0000","last_change":"2024-03-29T08:29:14.084881+0000","last_active":"2024-03-29T08:29:14.501135+0000","last_peered":"2024-03-29T08:29:14.501135+0000","last_clean":"2024-03-29T08:29:14.501135+0000","last_became_active":"2024-03-29T08:29:14.084587+0000","last_became_peered":"2024-03-29T08:29:14.084587+0000","last_unstale":"2024-03-29T08:29:14.501135+0000","last_undegraded":"2024-03-29T08:29:14.501135+0000","last_fullsized":"2024-03-29T08:29:14.501135+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:45.558553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.085712+0000","last_change":"2024-03-29T08:29:14.085712+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.085712+0000","last_undegraded":"2024-03-29T08:29:14.085712+0000","last_fullsized":"2024-03-29T08:29:14.085712+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:23:32.467222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509018+0000","last_change":"2024-03-29T08:29:14.089964+0000","last_active":"2024-03-29T08:29:14.509018+0000","last_peered":"2024-03-29T08:29:14.509018+0000","last_clean":"2024-03-29T08:29:14.509018+0000","last_became_active":"2024-03-29T08:29:14.088099+0000","last_became_peered":"2024-03-29T08:29:14.088099+0000","last_unstale":"2024-03-29T08:29:14.509018+0000","last_undegraded":"2024-03-29T08:29:14.509018+0000","last_fullsized":"2024-03-29T08:29:14.509018+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:05.868646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.498316+0000","last_change":"2024-03-29T08:29:14.491102+0000","last_active":"2024-03-29T08:29:14.498316+0000","last_peered":"2024-03-29T08:29:14.498316+0000","last_clean":"2024-03-29T08:29:14.498316+0000","last_became_active":"2024-03-29T08:29:14.490878+0000","last_became_peered":"2024-03-29T08:29:14.490878+0000","last_unstale":"2024-03-29T08:29:14.498316+0000","last_undegraded":"2024-03-29T08:29:14.498316+0000","last_fullsized":"2024-03-29T08:29:14.498316+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:06:54.374100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509548+0000","last_change":"2024-03-29T08:29:14.097684+0000","last_active":"2024-03-29T08:29:14.509548+0000","last_peered":"2024-03-29T08:29:14.509548+0000","last_clean":"2024-03-29T08:29:14.509548+0000","last_became_active":"2024-03-29T08:29:14.095879+0000","last_became_peered":"2024-03-29T08:29:14.095879+0000","last_unstale":"2024-03-29T08:29:14.509548+0000","last_undegraded":"2024-03-29T08:29:14.509548+0000","last_fullsized":"2024-03-29T08:29:14.509548+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:50:38.108416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.511687+0000","last_change":"2024-03-29T08:29:14.507697+0000","last_active":"2024-03-29T08:29:14.511687+0000","last_peered":"2024-03-29T08:29:14.511687+0000","last_clean":"2024-03-29T08:29:14.511687+0000","last_became_active":"2024-03-29T08:29:14.507356+0000","last_became_peered":"2024-03-29T08:29:14.507356+0000","last_unstale":"2024-03-29T08:29:14.511687+0000","last_undegraded":"2024-03-29T08:29:14.511687+0000","last_fullsized":"2024-03-29T08:29:14.511687+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:11.283108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.502762+0000","last_change":"2024-03-29T08:29:14.086806+0000","last_active":"2024-03-29T08:29:14.502762+0000","last_peered":"2024-03-29T08:29:14.502762+0000","last_clean":"2024-03-29T08:29:14.502762+0000","last_became_active":"2024-03-29T08:29:14.086468+0000","last_became_peered":"2024-03-29T08:29:14.086468+0000","last_unstale":"2024-03-29T08:29:14.502762+0000","last_undegraded":"2024-03-29T08:29:14.502762+0000","last_fullsized":"2024-03-29T08:29:14.502762+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:37:57.757695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504878+0000","last_change":"2024-03-29T08:29:14.498041+0000","last_active":"2024-03-29T08:29:14.504878+0000","last_peered":"2024-03-29T08:29:14.504878+0000","last_clean":"2024-03-29T08:29:14.504878+0000","last_became_active":"2024-03-29T08:29:14.497768+0000","last_became_peered":"2024-03-29T08:29:14.497768+0000","last_unstale":"2024-03-29T08:29:14.504878+0000","last_undegraded":"2024-03-29T08:29:14.504878+0000","last_fullsized":"2024-03-29T08:29:14.504878+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:55.518674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504448+0000","last_change":"2024-03-29T08:29:14.088920+0000","last_active":"2024-03-29T08:29:14.504448+0000","last_peered":"2024-03-29T08:29:14.504448+0000","last_clean":"2024-03-29T08:29:14.504448+0000","last_became_active":"2024-03-29T08:29:14.088593+0000","last_became_peered":"2024-03-29T08:29:14.088593+0000","last_unstale":"2024-03-29T08:29:14.504448+0000","last_undegraded":"2024-03-29T08:29:14.504448+0000","last_fullsized":"2024-03-29T08:29:14.504448+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:51.446071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504649+0000","last_change":"2024-03-29T08:29:14.499273+0000","last_active":"2024-03-29T08:29:14.504649+0000","last_peered":"2024-03-29T08:29:14.504649+0000","last_clean":"2024-03-29T08:29:14.504649+0000","last_became_active":"2024-03-29T08:29:14.498368+0000","last_became_peered":"2024-03-29T08:29:14.498368+0000","last_unstale":"2024-03-29T08:29:14.504649+0000","last_undegraded":"2024-03-29T08:29:14.504649+0000","last_fullsized":"2024-03-29T08:29:14.504649+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:23:10.605671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.492779+0000","last_change":"2024-03-29T08:29:14.079054+0000","last_active":"2024-03-29T08:29:14.492779+0000","last_peered":"2024-03-29T08:29:14.492779+0000","last_clean":"2024-03-29T08:29:14.492779+0000","last_became_active":"2024-03-29T08:29:14.078272+0000","last_became_peered":"2024-03-29T08:29:14.078272+0000","last_unstale":"2024-03-29T08:29:14.492779+0000","last_undegraded":"2024-03-29T08:29:14.492779+0000","last_fullsized":"2024-03-29T08:29:14.492779+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:57:54.510656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.503738+0000","last_change":"2024-03-29T08:29:14.495103+0000","last_active":"2024-03-29T08:29:14.503738+0000","last_peered":"2024-03-29T08:29:14.503738+0000","last_clean":"2024-03-29T08:29:14.503738+0000","last_became_active":"2024-03-29T08:29:14.494873+0000","last_became_peered":"2024-03-29T08:29:14.494873+0000","last_unstale":"2024-03-29T08:29:14.503738+0000","last_undegraded":"2024-03-29T08:29:14.503738+0000","last_fullsized":"2024-03-29T08:29:14.503738+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:49:15.540195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.484296+0000","last_change":"2024-03-29T08:29:14.091783+0000","last_active":"2024-03-29T08:29:14.484296+0000","last_peered":"2024-03-29T08:29:14.484296+0000","last_clean":"2024-03-29T08:29:14.484296+0000","last_became_active":"2024-03-29T08:29:14.091531+0000","last_became_peered":"2024-03-29T08:29:14.091531+0000","last_unstale":"2024-03-29T08:29:14.484296+0000","last_undegraded":"2024-03-29T08:29:14.484296+0000","last_fullsized":"2024-03-29T08:29:14.484296+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:32:23.859154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504693+0000","last_change":"2024-03-29T08:29:14.498939+0000","last_active":"2024-03-29T08:29:14.504693+0000","last_peered":"2024-03-29T08:29:14.504693+0000","last_clean":"2024-03-29T08:29:14.504693+0000","last_became_active":"2024-03-29T08:29:14.498589+0000","last_became_peered":"2024-03-29T08:29:14.498589+0000","last_unstale":"2024-03-29T08:29:14.504693+0000","last_undegraded":"2024-03-29T08:29:14.504693+0000","last_fullsized":"2024-03-29T08:29:14.504693+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:32:24.719500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.496023+0000","last_change":"2024-03-29T08:29:14.096474+0000","last_active":"2024-03-29T08:29:14.496023+0000","last_peered":"2024-03-29T08:29:14.496023+0000","last_clean":"2024-03-29T08:29:14.496023+0000","last_became_active":"2024-03-29T08:29:14.094336+0000","last_became_peered":"2024-03-29T08:29:14.094336+0000","last_unstale":"2024-03-29T08:29:14.496023+0000","last_undegraded":"2024-03-29T08:29:14.496023+0000","last_fullsized":"2024-03-29T08:29:14.496023+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:39.277282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.489947+0000","last_change":"2024-03-29T08:29:14.487254+0000","last_active":"2024-03-29T08:29:14.489947+0000","last_peered":"2024-03-29T08:29:14.489947+0000","last_clean":"2024-03-29T08:29:14.489947+0000","last_became_active":"2024-03-29T08:29:14.485472+0000","last_became_peered":"2024-03-29T08:29:14.485472+0000","last_unstale":"2024-03-29T08:29:14.489947+0000","last_undegraded":"2024-03-29T08:29:14.489947+0000","last_fullsized":"2024-03-29T08:29:14.489947+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:08:57.590159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.484431+0000","last_change":"2024-03-29T08:29:14.092400+0000","last_active":"2024-03-29T08:29:14.484431+0000","last_peered":"2024-03-29T08:29:14.484431+0000","last_clean":"2024-03-29T08:29:14.484431+0000","last_became_active":"2024-03-29T08:29:14.091857+0000","last_became_peered":"2024-03-29T08:29:14.091857+0000","last_unstale":"2024-03-29T08:29:14.484431+0000","last_undegraded":"2024-03-29T08:29:14.484431+0000","last_fullsized":"2024-03-29T08:29:14.484431+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:37:57.757695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.085301+0000","last_change":"2024-03-29T08:29:14.085301+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.085301+0000","last_undegraded":"2024-03-29T08:29:14.085301+0000","last_fullsized":"2024-03-29T08:29:14.085301+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:45.863920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"120'3","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509397+0000","last_change":"2024-03-29T08:29:14.092891+0000","last_active":"2024-03-29T08:29:14.509397+0000","last_peered":"2024-03-29T08:29:14.509397+0000","last_clean":"2024-03-29T08:29:14.509397+0000","last_became_active":"2024-03-29T08:29:14.092632+0000","last_became_peered":"2024-03-29T08:29:14.092632+0000","last_unstale":"2024-03-29T08:29:14.509397+0000","last_undegraded":"2024-03-29T08:29:14.509397+0000","last_fullsized":"2024-03-29T08:29:14.509397+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:52:04.161387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.492713+0000","last_change":"2024-03-29T08:29:14.489765+0000","last_active":"2024-03-29T08:29:14.492713+0000","last_peered":"2024-03-29T08:29:14.492713+0000","last_clean":"2024-03-29T08:29:14.492713+0000","last_became_active":"2024-03-29T08:29:14.489440+0000","last_became_peered":"2024-03-29T08:29:14.489440+0000","last_unstale":"2024-03-29T08:29:14.492713+0000","last_undegraded":"2024-03-29T08:29:14.492713+0000","last_fullsized":"2024-03-29T08:29:14.492713+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:53:42.976104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"120'6","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.512807+0000","last_change":"2024-03-29T08:29:14.096725+0000","last_active":"2024-03-29T08:29:14.512807+0000","last_peered":"2024-03-29T08:29:14.512807+0000","last_clean":"2024-03-29T08:29:14.512807+0000","last_became_active":"2024-03-29T08:29:14.094695+0000","last_became_peered":"2024-03-29T08:29:14.094695+0000","last_unstale":"2024-03-29T08:29:14.512807+0000","last_undegraded":"2024-03-29T08:29:14.512807+0000","last_fullsized":"2024-03-29T08:29:14.512807+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:15:28.507662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504528+0000","last_change":"2024-03-29T08:29:14.499972+0000","last_active":"2024-03-29T08:29:14.504528+0000","last_peered":"2024-03-29T08:29:14.504528+0000","last_clean":"2024-03-29T08:29:14.504528+0000","last_became_active":"2024-03-29T08:29:14.499698+0000","last_became_peered":"2024-03-29T08:29:14.499698+0000","last_unstale":"2024-03-29T08:29:14.504528+0000","last_undegraded":"2024-03-29T08:29:14.504528+0000","last_fullsized":"2024-03-29T08:29:14.504528+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:44:41.312640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508984+0000","last_change":"2024-03-29T08:29:14.092574+0000","last_active":"2024-03-29T08:29:14.508984+0000","last_peered":"2024-03-29T08:29:14.508984+0000","last_clean":"2024-03-29T08:29:14.508984+0000","last_became_active":"2024-03-29T08:29:14.092004+0000","last_became_peered":"2024-03-29T08:29:14.092004+0000","last_unstale":"2024-03-29T08:29:14.508984+0000","last_undegraded":"2024-03-29T08:29:14.508984+0000","last_fullsized":"2024-03-29T08:29:14.508984+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:21.448128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.510447+0000","last_change":"2024-03-29T08:29:14.502264+0000","last_active":"2024-03-29T08:29:14.510447+0000","last_peered":"2024-03-29T08:29:14.510447+0000","last_clean":"2024-03-29T08:29:14.510447+0000","last_became_active":"2024-03-29T08:29:14.501882+0000","last_became_peered":"2024-03-29T08:29:14.501882+0000","last_unstale":"2024-03-29T08:29:14.510447+0000","last_undegraded":"2024-03-29T08:29:14.510447+0000","last_fullsized":"2024-03-29T08:29:14.510447+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:31:05.279265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.480729+0000","last_change":"2024-03-29T08:29:14.079278+0000","last_active":"2024-03-29T08:29:14.480729+0000","last_peered":"2024-03-29T08:29:14.480729+0000","last_clean":"2024-03-29T08:29:14.480729+0000","last_became_active":"2024-03-29T08:29:14.078982+0000","last_became_peered":"2024-03-29T08:29:14.078982+0000","last_unstale":"2024-03-29T08:29:14.480729+0000","last_undegraded":"2024-03-29T08:29:14.480729+0000","last_fullsized":"2024-03-29T08:29:14.480729+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:06.149502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509069+0000","last_change":"2024-03-29T08:29:14.500759+0000","last_active":"2024-03-29T08:29:14.509069+0000","last_peered":"2024-03-29T08:29:14.509069+0000","last_clean":"2024-03-29T08:29:14.509069+0000","last_became_active":"2024-03-29T08:29:14.500463+0000","last_became_peered":"2024-03-29T08:29:14.500463+0000","last_unstale":"2024-03-29T08:29:14.509069+0000","last_undegraded":"2024-03-29T08:29:14.509069+0000","last_fullsized":"2024-03-29T08:29:14.509069+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:13:49.079869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"120'8","reported_seq":18,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.510685+0000","last_change":"2024-03-29T08:29:14.090542+0000","last_active":"2024-03-29T08:29:14.510685+0000","last_peered":"2024-03-29T08:29:14.510685+0000","last_clean":"2024-03-29T08:29:14.510685+0000","last_became_active":"2024-03-29T08:29:14.090235+0000","last_became_peered":"2024-03-29T08:29:14.090235+0000","last_unstale":"2024-03-29T08:29:14.510685+0000","last_undegraded":"2024-03-29T08:29:14.510685+0000","last_fullsized":"2024-03-29T08:29:14.510685+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:45.558553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508199+0000","last_change":"2024-03-29T08:29:14.497747+0000","last_active":"2024-03-29T08:29:14.508199+0000","last_peered":"2024-03-29T08:29:14.508199+0000","last_clean":"2024-03-29T08:29:14.508199+0000","last_became_active":"2024-03-29T08:29:14.497456+0000","last_became_peered":"2024-03-29T08:29:14.497456+0000","last_unstale":"2024-03-29T08:29:14.508199+0000","last_undegraded":"2024-03-29T08:29:14.508199+0000","last_fullsized":"2024-03-29T08:29:14.508199+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:49.180147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.097550+0000","last_change":"2024-03-29T08:29:14.094084+0000","last_active":"2024-03-29T08:29:14.097550+0000","last_peered":"2024-03-29T08:29:14.097550+0000","last_clean":"2024-03-29T08:29:14.097550+0000","last_became_active":"2024-03-29T08:29:14.093447+0000","last_became_peered":"2024-03-29T08:29:14.093447+0000","last_unstale":"2024-03-29T08:29:14.097550+0000","last_undegraded":"2024-03-29T08:29:14.097550+0000","last_fullsized":"2024-03-29T08:29:14.097550+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:11:26.207009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.512213+0000","last_change":"2024-03-29T08:29:14.507678+0000","last_active":"2024-03-29T08:29:14.512213+0000","last_peered":"2024-03-29T08:29:14.512213+0000","last_clean":"2024-03-29T08:29:14.512213+0000","last_became_active":"2024-03-29T08:29:14.507132+0000","last_became_peered":"2024-03-29T08:29:14.507132+0000","last_unstale":"2024-03-29T08:29:14.512213+0000","last_undegraded":"2024-03-29T08:29:14.512213+0000","last_fullsized":"2024-03-29T08:29:14.512213+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:15:29.513377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.502517+0000","last_change":"2024-03-29T08:29:14.095051+0000","last_active":"2024-03-29T08:29:14.502517+0000","last_peered":"2024-03-29T08:29:14.502517+0000","last_clean":"2024-03-29T08:29:14.502517+0000","last_became_active":"2024-03-29T08:29:14.092852+0000","last_became_peered":"2024-03-29T08:29:14.092852+0000","last_unstale":"2024-03-29T08:29:14.502517+0000","last_undegraded":"2024-03-29T08:29:14.502517+0000","last_fullsized":"2024-03-29T08:29:14.502517+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:06.149502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.504967+0000","last_change":"2024-03-29T08:29:14.497468+0000","last_active":"2024-03-29T08:29:14.504967+0000","last_peered":"2024-03-29T08:29:14.504967+0000","last_clean":"2024-03-29T08:29:14.504967+0000","last_became_active":"2024-03-29T08:29:14.497061+0000","last_became_peered":"2024-03-29T08:29:14.497061+0000","last_unstale":"2024-03-29T08:29:14.504967+0000","last_undegraded":"2024-03-29T08:29:14.504967+0000","last_fullsized":"2024-03-29T08:29:14.504967+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:09:20.324353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"120'1","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.501012+0000","last_change":"2024-03-29T08:29:14.090528+0000","last_active":"2024-03-29T08:29:14.501012+0000","last_peered":"2024-03-29T08:29:14.501012+0000","last_clean":"2024-03-29T08:29:14.501012+0000","last_became_active":"2024-03-29T08:29:14.090055+0000","last_became_peered":"2024-03-29T08:29:14.090055+0000","last_unstale":"2024-03-29T08:29:14.501012+0000","last_undegraded":"2024-03-29T08:29:14.501012+0000","last_fullsized":"2024-03-29T08:29:14.501012+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:21.448128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505830+0000","last_change":"2024-03-29T08:29:14.502568+0000","last_active":"2024-03-29T08:29:14.505830+0000","last_peered":"2024-03-29T08:29:14.505830+0000","last_clean":"2024-03-29T08:29:14.505830+0000","last_became_active":"2024-03-29T08:29:14.500161+0000","last_became_peered":"2024-03-29T08:29:14.500161+0000","last_unstale":"2024-03-29T08:29:14.505830+0000","last_undegraded":"2024-03-29T08:29:14.505830+0000","last_fullsized":"2024-03-29T08:29:14.505830+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:25:45.763218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.487422+0000","last_change":"2024-03-29T08:29:14.078956+0000","last_active":"2024-03-29T08:29:14.487422+0000","last_peered":"2024-03-29T08:29:14.487422+0000","last_clean":"2024-03-29T08:29:14.487422+0000","last_became_active":"2024-03-29T08:29:14.078689+0000","last_became_peered":"2024-03-29T08:29:14.078689+0000","last_unstale":"2024-03-29T08:29:14.487422+0000","last_undegraded":"2024-03-29T08:29:14.487422+0000","last_fullsized":"2024-03-29T08:29:14.487422+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:42:53.153883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.489423+0000","last_change":"2024-03-29T08:29:14.486493+0000","last_active":"2024-03-29T08:29:14.489423+0000","last_peered":"2024-03-29T08:29:14.489423+0000","last_clean":"2024-03-29T08:29:14.489423+0000","last_became_active":"2024-03-29T08:29:14.486242+0000","last_became_peered":"2024-03-29T08:29:14.486242+0000","last_unstale":"2024-03-29T08:29:14.489423+0000","last_undegraded":"2024-03-29T08:29:14.489423+0000","last_fullsized":"2024-03-29T08:29:14.489423+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:30:38.933793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507685+0000","last_change":"2024-03-29T08:29:14.092768+0000","last_active":"2024-03-29T08:29:14.507685+0000","last_peered":"2024-03-29T08:29:14.507685+0000","last_clean":"2024-03-29T08:29:14.507685+0000","last_became_active":"2024-03-29T08:29:14.092290+0000","last_became_peered":"2024-03-29T08:29:14.092290+0000","last_unstale":"2024-03-29T08:29:14.507685+0000","last_undegraded":"2024-03-29T08:29:14.507685+0000","last_fullsized":"2024-03-29T08:29:14.507685+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:45.752363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509731+0000","last_change":"2024-03-29T08:29:14.502876+0000","last_active":"2024-03-29T08:29:14.509731+0000","last_peered":"2024-03-29T08:29:14.509731+0000","last_clean":"2024-03-29T08:29:14.509731+0000","last_became_active":"2024-03-29T08:29:14.502600+0000","last_became_peered":"2024-03-29T08:29:14.502600+0000","last_unstale":"2024-03-29T08:29:14.509731+0000","last_undegraded":"2024-03-29T08:29:14.509731+0000","last_fullsized":"2024-03-29T08:29:14.509731+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:49:19.406820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.486956+0000","last_change":"2024-03-29T08:29:14.090942+0000","last_active":"2024-03-29T08:29:14.486956+0000","last_peered":"2024-03-29T08:29:14.486956+0000","last_clean":"2024-03-29T08:29:14.486956+0000","last_became_active":"2024-03-29T08:29:14.090695+0000","last_became_peered":"2024-03-29T08:29:14.090695+0000","last_unstale":"2024-03-29T08:29:14.486956+0000","last_undegraded":"2024-03-29T08:29:14.486956+0000","last_fullsized":"2024-03-29T08:29:14.486956+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:18.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509268+0000","last_change":"2024-03-29T08:29:14.508582+0000","last_active":"2024-03-29T08:29:14.509268+0000","last_peered":"2024-03-29T08:29:14.509268+0000","last_clean":"2024-03-29T08:29:14.509268+0000","last_became_active":"2024-03-29T08:29:14.508201+0000","last_became_peered":"2024-03-29T08:29:14.508201+0000","last_unstale":"2024-03-29T08:29:14.509268+0000","last_undegraded":"2024-03-29T08:29:14.509268+0000","last_fullsized":"2024-03-29T08:29:14.509268+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:59:12.691184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.497159+0000","last_change":"2024-03-29T08:29:14.096474+0000","last_active":"2024-03-29T08:29:14.497159+0000","last_peered":"2024-03-29T08:29:14.497159+0000","last_clean":"2024-03-29T08:29:14.497159+0000","last_became_active":"2024-03-29T08:29:14.094758+0000","last_became_peered":"2024-03-29T08:29:14.094758+0000","last_unstale":"2024-03-29T08:29:14.497159+0000","last_undegraded":"2024-03-29T08:29:14.497159+0000","last_fullsized":"2024-03-29T08:29:14.497159+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:32:23.713785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505838+0000","last_change":"2024-03-29T08:29:14.503676+0000","last_active":"2024-03-29T08:29:14.505838+0000","last_peered":"2024-03-29T08:29:14.505838+0000","last_clean":"2024-03-29T08:29:14.505838+0000","last_became_active":"2024-03-29T08:29:14.503314+0000","last_became_peered":"2024-03-29T08:29:14.503314+0000","last_unstale":"2024-03-29T08:29:14.505838+0000","last_undegraded":"2024-03-29T08:29:14.505838+0000","last_fullsized":"2024-03-29T08:29:14.505838+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:08:59.334138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.496690+0000","last_change":"2024-03-29T08:29:14.089768+0000","last_active":"2024-03-29T08:29:14.496690+0000","last_peered":"2024-03-29T08:29:14.496690+0000","last_clean":"2024-03-29T08:29:14.496690+0000","last_became_active":"2024-03-29T08:29:14.089486+0000","last_became_peered":"2024-03-29T08:29:14.089486+0000","last_unstale":"2024-03-29T08:29:14.496690+0000","last_undegraded":"2024-03-29T08:29:14.496690+0000","last_fullsized":"2024-03-29T08:29:14.496690+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:59.174853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507881+0000","last_change":"2024-03-29T08:29:14.505681+0000","last_active":"2024-03-29T08:29:14.507881+0000","last_peered":"2024-03-29T08:29:14.507881+0000","last_clean":"2024-03-29T08:29:14.507881+0000","last_became_active":"2024-03-29T08:29:14.505394+0000","last_became_peered":"2024-03-29T08:29:14.505394+0000","last_unstale":"2024-03-29T08:29:14.507881+0000","last_undegraded":"2024-03-29T08:29:14.507881+0000","last_fullsized":"2024-03-29T08:29:14.507881+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:32:12.510083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"120'2","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.508946+0000","last_change":"2024-03-29T08:29:14.088394+0000","last_active":"2024-03-29T08:29:14.508946+0000","last_peered":"2024-03-29T08:29:14.508946+0000","last_clean":"2024-03-29T08:29:14.508946+0000","last_became_active":"2024-03-29T08:29:14.087525+0000","last_became_peered":"2024-03-29T08:29:14.087525+0000","last_unstale":"2024-03-29T08:29:14.508946+0000","last_undegraded":"2024-03-29T08:29:14.508946+0000","last_fullsized":"2024-03-29T08:29:14.508946+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:54:27.262284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.086933+0000","last_change":"2024-03-29T08:29:14.086933+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.086933+0000","last_undegraded":"2024-03-29T08:29:14.086933+0000","last_fullsized":"2024-03-29T08:29:14.086933+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:14:06.265379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.497231+0000","last_change":"2024-03-29T08:29:14.088202+0000","last_active":"2024-03-29T08:29:14.497231+0000","last_peered":"2024-03-29T08:29:14.497231+0000","last_clean":"2024-03-29T08:29:14.497231+0000","last_became_active":"2024-03-29T08:29:14.087860+0000","last_became_peered":"2024-03-29T08:29:14.087860+0000","last_unstale":"2024-03-29T08:29:14.497231+0000","last_undegraded":"2024-03-29T08:29:14.497231+0000","last_fullsized":"2024-03-29T08:29:14.497231+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:30:51.004244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.509624+0000","last_change":"2024-03-29T08:29:14.501270+0000","last_active":"2024-03-29T08:29:14.509624+0000","last_peered":"2024-03-29T08:29:14.509624+0000","last_clean":"2024-03-29T08:29:14.509624+0000","last_became_active":"2024-03-29T08:29:14.498111+0000","last_became_peered":"2024-03-29T08:29:14.498111+0000","last_unstale":"2024-03-29T08:29:14.509624+0000","last_undegraded":"2024-03-29T08:29:14.509624+0000","last_fullsized":"2024-03-29T08:29:14.509624+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:26:17.474099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.495208+0000","last_change":"2024-03-29T08:29:14.090681+0000","last_active":"2024-03-29T08:29:14.495208+0000","last_peered":"2024-03-29T08:29:14.495208+0000","last_clean":"2024-03-29T08:29:14.495208+0000","last_became_active":"2024-03-29T08:29:14.090416+0000","last_became_peered":"2024-03-29T08:29:14.090416+0000","last_unstale":"2024-03-29T08:29:14.495208+0000","last_undegraded":"2024-03-29T08:29:14.495208+0000","last_fullsized":"2024-03-29T08:29:14.495208+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:08:56.584444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507376+0000","last_change":"2024-03-29T08:29:14.505745+0000","last_active":"2024-03-29T08:29:14.507376+0000","last_peered":"2024-03-29T08:29:14.507376+0000","last_clean":"2024-03-29T08:29:14.507376+0000","last_became_active":"2024-03-29T08:29:14.504499+0000","last_became_peered":"2024-03-29T08:29:14.504499+0000","last_unstale":"2024-03-29T08:29:14.507376+0000","last_undegraded":"2024-03-29T08:29:14.507376+0000","last_fullsized":"2024-03-29T08:29:14.507376+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:23:28.769780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.495503+0000","last_change":"2024-03-29T08:29:14.096941+0000","last_active":"2024-03-29T08:29:14.495503+0000","last_peered":"2024-03-29T08:29:14.495503+0000","last_clean":"2024-03-29T08:29:14.495503+0000","last_became_active":"2024-03-29T08:29:14.096439+0000","last_became_peered":"2024-03-29T08:29:14.096439+0000","last_unstale":"2024-03-29T08:29:14.495503+0000","last_undegraded":"2024-03-29T08:29:14.495503+0000","last_fullsized":"2024-03-29T08:29:14.495503+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:49:13.231792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.503705+0000","last_change":"2024-03-29T08:29:14.496299+0000","last_active":"2024-03-29T08:29:14.503705+0000","last_peered":"2024-03-29T08:29:14.503705+0000","last_clean":"2024-03-29T08:29:14.503705+0000","last_became_active":"2024-03-29T08:29:14.496042+0000","last_became_peered":"2024-03-29T08:29:14.496042+0000","last_unstale":"2024-03-29T08:29:14.503705+0000","last_undegraded":"2024-03-29T08:29:14.503705+0000","last_fullsized":"2024-03-29T08:29:14.503705+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:23:28.769780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.497619+0000","last_change":"2024-03-29T08:29:14.081880+0000","last_active":"2024-03-29T08:29:14.497619+0000","last_peered":"2024-03-29T08:29:14.497619+0000","last_clean":"2024-03-29T08:29:14.497619+0000","last_became_active":"2024-03-29T08:29:14.079732+0000","last_became_peered":"2024-03-29T08:29:14.079732+0000","last_unstale":"2024-03-29T08:29:14.497619+0000","last_undegraded":"2024-03-29T08:29:14.497619+0000","last_fullsized":"2024-03-29T08:29:14.497619+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:01.494143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.503628+0000","last_change":"2024-03-29T08:29:14.501776+0000","last_active":"2024-03-29T08:29:14.503628+0000","last_peered":"2024-03-29T08:29:14.503628+0000","last_clean":"2024-03-29T08:29:14.503628+0000","last_became_active":"2024-03-29T08:29:14.501508+0000","last_became_peered":"2024-03-29T08:29:14.501508+0000","last_unstale":"2024-03-29T08:29:14.503628+0000","last_undegraded":"2024-03-29T08:29:14.503628+0000","last_fullsized":"2024-03-29T08:29:14.503628+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:01.476642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.487173+0000","last_change":"2024-03-29T08:29:14.091456+0000","last_active":"2024-03-29T08:29:14.487173+0000","last_peered":"2024-03-29T08:29:14.487173+0000","last_clean":"2024-03-29T08:29:14.487173+0000","last_became_active":"2024-03-29T08:29:14.091121+0000","last_became_peered":"2024-03-29T08:29:14.091121+0000","last_unstale":"2024-03-29T08:29:14.487173+0000","last_undegraded":"2024-03-29T08:29:14.487173+0000","last_fullsized":"2024-03-29T08:29:14.487173+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:30.979257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507721+0000","last_change":"2024-03-29T08:29:14.503755+0000","last_active":"2024-03-29T08:29:14.507721+0000","last_peered":"2024-03-29T08:29:14.507721+0000","last_clean":"2024-03-29T08:29:14.507721+0000","last_became_active":"2024-03-29T08:29:14.503521+0000","last_became_peered":"2024-03-29T08:29:14.503521+0000","last_unstale":"2024-03-29T08:29:14.507721+0000","last_undegraded":"2024-03-29T08:29:14.507721+0000","last_fullsized":"2024-03-29T08:29:14.507721+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:33:36.260388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.486511+0000","last_change":"2024-03-29T08:29:14.089662+0000","last_active":"2024-03-29T08:29:14.486511+0000","last_peered":"2024-03-29T08:29:14.486511+0000","last_clean":"2024-03-29T08:29:14.486511+0000","last_became_active":"2024-03-29T08:29:14.089393+0000","last_became_peered":"2024-03-29T08:29:14.089393+0000","last_unstale":"2024-03-29T08:29:14.486511+0000","last_undegraded":"2024-03-29T08:29:14.486511+0000","last_fullsized":"2024-03-29T08:29:14.486511+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:27:50.778160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-03-29T08:29:14.084305+0000","last_change":"2024-03-29T08:29:14.084305+0000","last_active":"2024-03-29T08:29:14.057150+0000","last_peered":"2024-03-29T08:29:14.057150+0000","last_clean":"2024-03-29T08:29:14.057150+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-29T08:29:14.084305+0000","last_undegraded":"2024-03-29T08:29:14.084305+0000","last_fullsized":"2024-03-29T08:29:14.084305+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:44:25.936884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.095596+0000","last_change":"2024-03-29T08:29:14.092230+0000","last_active":"2024-03-29T08:29:14.095596+0000","last_peered":"2024-03-29T08:29:14.095596+0000","last_clean":"2024-03-29T08:29:14.095596+0000","last_became_active":"2024-03-29T08:29:14.091979+0000","last_became_peered":"2024-03-29T08:29:14.091979+0000","last_unstale":"2024-03-29T08:29:14.095596+0000","last_undegraded":"2024-03-29T08:29:14.095596+0000","last_fullsized":"2024-03-29T08:29:14.095596+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:41:42.442894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.510494+0000","last_change":"2024-03-29T08:29:14.502476+0000","last_active":"2024-03-29T08:29:14.510494+0000","last_peered":"2024-03-29T08:29:14.510494+0000","last_clean":"2024-03-29T08:29:14.510494+0000","last_became_active":"2024-03-29T08:29:14.502191+0000","last_became_peered":"2024-03-29T08:29:14.502191+0000","last_unstale":"2024-03-29T08:29:14.510494+0000","last_undegraded":"2024-03-29T08:29:14.510494+0000","last_fullsized":"2024-03-29T08:29:14.510494+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:06.826332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.475058+0000","last_change":"2024-03-29T08:29:14.077404+0000","last_active":"2024-03-29T08:29:14.475058+0000","last_peered":"2024-03-29T08:29:14.475058+0000","last_clean":"2024-03-29T08:29:14.475058+0000","last_became_active":"2024-03-29T08:29:14.077141+0000","last_became_peered":"2024-03-29T08:29:14.077141+0000","last_unstale":"2024-03-29T08:29:14.475058+0000","last_undegraded":"2024-03-29T08:29:14.475058+0000","last_fullsized":"2024-03-29T08:29:14.475058+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:51.446071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":250,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.072521+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:29:14.072521+0000","last_peered":"2024-03-29T08:29:14.072521+0000","last_clean":"2024-03-29T08:29:14.072521+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:29:14.072521+0000","last_undegraded":"2024-03-29T08:29:14.072521+0000","last_fullsized":"2024-03-29T08:29:14.072521+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37,"ondisk_log_size":37,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509578,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1503232,"data_stored":646009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.69299999999999995},"min":{"1min":0.48199999999999998,"5min":0.41899999999999998,"15min":0.34999999999999998},"max":{"1min":0.88,"5min":0.97999999999999998,"15min":1.1639999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.746,"5min":0.75600000000000001,"15min":0.71899999999999997},"min":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.45800000000000002},"max":{"1min":0.95099999999999996,"5min":0.998,"15min":1.2410000000000001},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70199999999999996,"15min":0.67600000000000005},"min":{"1min":0.51500000000000001,"5min":0.45700000000000002,"15min":0.39600000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.202},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70799999999999996,"15min":0.67600000000000005},"min":{"1min":0.56100000000000005,"5min":0.48799999999999999,"15min":0.44},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.1339999999999999},"last":0.505}]},{"osd":2,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69799999999999995,"15min":0.67000000000000004},"min":{"1min":0.502,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.90300000000000002,"5min":0.93600000000000005,"15min":1.085},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72899999999999998,"15min":0.70299999999999996},"min":{"1min":0.55500000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.288},"last":0.49099999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72299999999999998,"15min":0.70999999999999996},"min":{"1min":0.57599999999999996,"5min":0.51100000000000001,"15min":0.46000000000000002},"max":{"1min":0.97199999999999998,"5min":0.98099999999999998,"15min":1.018},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.60899999999999999,"5min":0.52300000000000002,"15min":0.48499999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.2609999999999999},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58399999999999996,"15min":0.60999999999999999},"min":{"1min":0.48099999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85299999999999998,"5min":0.87,"15min":0.95299999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60599999999999998,"15min":0.66300000000000003},"min":{"1min":0.45100000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.123},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60999999999999999,"15min":0.66200000000000003},"min":{"1min":0.48199999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.877,"5min":0.89600000000000002,"15min":1.056},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.628,"15min":0.66900000000000004},"min":{"1min":0.49199999999999999,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.84399999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.47399999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69199999999999995,"15min":0.71399999999999997},"min":{"1min":0.55900000000000005,"5min":0.46999999999999997,"15min":0.46600000000000003},"max":{"1min":0.89400000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.436},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65800000000000003,"15min":0.70599999999999996},"min":{"1min":0.52800000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.81799999999999995,"5min":0.93799999999999994,"15min":0.98299999999999998},"last":0.64400000000000002}]}]},{"osd":6,"up_from":15,"seq":64424509578,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1482752,"data_stored":644321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72399999999999998,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.38900000000000001},"max":{"1min":0.873,"5min":0.93100000000000005,"15min":1.0189999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.751,"15min":0.71899999999999997},"min":{"1min":0.57199999999999995,"5min":0.51700000000000002,"15min":0.44500000000000001},"max":{"1min":0.97699999999999998,"5min":0.99299999999999999,"15min":1.079},"last":0.72299999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68200000000000005,"15min":0.64700000000000002},"min":{"1min":0.45400000000000001,"5min":0.45300000000000001,"15min":0.377},"max":{"1min":0.85199999999999998,"5min":0.97999999999999998,"15min":0.99299999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42899999999999999},"max":{"1min":0.78200000000000003,"5min":0.95599999999999996,"15min":1.079},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76900000000000002,"15min":0.72299999999999998},"min":{"1min":0.52100000000000002,"5min":0.51800000000000002,"15min":0.44800000000000001},"max":{"1min":0.91500000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.746,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.46400000000000002,"15min":0.435},"max":{"1min":0.93899999999999995,"5min":0.95099999999999996,"15min":0.95899999999999996},"last":0.753}]},{"osd":3,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.443,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82099999999999995,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.55700000000000005,"5min":0.51000000000000001,"15min":0.47799999999999998},"max":{"1min":0.89800000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.71299999999999997}]},{"osd":4,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.623,"15min":0.64600000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.36499999999999999},"max":{"1min":0.93100000000000005,"5min":1.0840000000000001,"15min":1.556},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63900000000000001,"15min":0.68200000000000005},"min":{"1min":0.46400000000000002,"5min":0.38200000000000001,"15min":0.371},"max":{"1min":0.872,"5min":0.996,"15min":2.0539999999999998},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.627,"15min":0.66100000000000003},"min":{"1min":0.45000000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83599999999999997,"5min":0.99399999999999999,"15min":1.0660000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68100000000000005,"15min":0.70199999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":1.042,"15min":1.1419999999999999},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69599999999999995,"15min":0.72499999999999998},"min":{"1min":0.499,"5min":0.46800000000000003,"15min":0.38500000000000001},"max":{"1min":0.89600000000000002,"5min":1.03,"15min":1.03},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.72999999999999998},"min":{"1min":0.51600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91500000000000004,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.63}]}]},{"osd":4,"up_from":15,"seq":64424509577,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1499136,"data_stored":645975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.45000000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.026,"5min":1.026,"15min":2.1709999999999998},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69699999999999995,"15min":0.71799999999999997},"min":{"1min":0.495,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.997,"5min":0.997,"15min":2.2869999999999999},"last":0.82999999999999996}]},{"osd":1,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.72499999999999998,"15min":0.748},"min":{"1min":0.436,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":2.524},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71099999999999997,"15min":0.70899999999999996},"min":{"1min":0.46400000000000002,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":2.2679999999999998},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.69999999999999996,"15min":0.71799999999999997},"min":{"1min":0.45300000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":2.363},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72999999999999998,"15min":0.748},"min":{"1min":0.52500000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.88700000000000001,"5min":0.97799999999999998,"15min":2.1880000000000002},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76200000000000001,"15min":0.753},"min":{"1min":0.60899999999999999,"5min":0.53400000000000003,"15min":0.46300000000000002},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":2.3029999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.77100000000000002,"15min":0.79300000000000004},"min":{"1min":0.55600000000000005,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.016,"5min":1.016,"15min":2.2549999999999999},"last":0.81599999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60699999999999998,"15min":0.65800000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.36499999999999999},"max":{"1min":0.76400000000000001,"5min":0.998,"15min":1.0189999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.62,"15min":0.63900000000000001},"min":{"1min":0.45800000000000002,"5min":0.437,"15min":0.41799999999999998},"max":{"1min":0.82699999999999996,"5min":0.85499999999999998,"15min":1.0149999999999999},"last":0.59599999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.63600000000000001,"15min":0.65700000000000003},"min":{"1min":0.44400000000000001,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.85599999999999998,"5min":1.091,"15min":1.091},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69399999999999995,"15min":0.72099999999999997},"min":{"1min":0.505,"5min":0.499,"15min":0.499},"max":{"1min":0.86399999999999999,"5min":0.91500000000000004,"15min":1.1220000000000001},"last":0.67500000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:28:18 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.68799999999999994,"15min":0.71199999999999997},"min":{"1min":0.51400000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.91400000000000003,"5min":0.91800000000000004,"15min":1.1000000000000001},"last":0.754},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72599999999999998,"15min":0.747},"min":{"1min":0.434,"5min":0.434,"15min":0.42899999999999999},"max":{"1min":0.96499999999999997,"5min":1.054,"15min":1.4430000000000001},"last":0.69399999999999995}]}]},{"osd":3,"up_from":15,"seq":64424509580,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1482752,"data_stored":644321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.28100000000000003,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":0.71799999999999997,"5min":0.748,"15min":1.123},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69599999999999995,"5min":0.88100000000000001,"15min":1.0409999999999999},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.44,"15min":0.44800000000000001},"min":{"1min":0.32800000000000001,"5min":0.29599999999999999,"15min":0.26500000000000001},"max":{"1min":0.623,"5min":0.76800000000000002,"15min":1.1599999999999999},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.28599999999999998},"max":{"1min":0.82099999999999995,"5min":0.94499999999999995,"15min":1.4790000000000001},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.54400000000000004},"min":{"1min":0.35499999999999998,"5min":0.32900000000000001,"15min":0.309},"max":{"1min":0.76200000000000001,"5min":0.80300000000000005,"15min":1.2529999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.624},"min":{"1min":0.439,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78300000000000003,"5min":0.86599999999999999,"15min":1.8580000000000001},"last":0.59899999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77200000000000002,"15min":0.77500000000000002},"min":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.42199999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.6419999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.77900000000000003,"15min":0.76900000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.45400000000000001},"max":{"1min":1.083,"5min":1.3340000000000001,"15min":1.391},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82199999999999995,"15min":0.80300000000000005},"min":{"1min":0.59599999999999997,"5min":0.57099999999999995,"15min":0.52100000000000002},"max":{"1min":1.081,"5min":1.1850000000000001,"15min":1.671},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.82399999999999995,"15min":0.83499999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0249999999999999,"5min":1.2050000000000001,"15min":1.7110000000000001},"last":0.58699999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.55400000000000005,"5min":0.51700000000000002,"15min":0.48799999999999999},"max":{"1min":1.232,"5min":1.2609999999999999,"15min":1.669},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78100000000000003,"15min":0.79600000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0029999999999999,"5min":1.119,"15min":2.0150000000000001},"last":0.54600000000000004}]},{"osd":7,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.82199999999999995},"min":{"1min":0.58899999999999997,"5min":0.50600000000000001,"15min":0.48799999999999999},"max":{"1min":1.0069999999999999,"5min":1.3169999999999999,"15min":1.887},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.81899999999999995},"min":{"1min":0.58999999999999997,"5min":0.49399999999999999,"15min":0.47999999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":2.044},"last":0.57199999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509579,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32556,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339284,"statfs":{"total":96636764160,"available":96603426816,"internally_reserved":0,"allocated":1470464,"data_stored":637546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.45900000000000002,"15min":0.47799999999999998},"min":{"1min":0.312,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.67800000000000005,"5min":0.754,"15min":0.94299999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.53200000000000003,"15min":0.52000000000000002},"min":{"1min":0.378,"5min":0.34699999999999998,"15min":0.32300000000000001},"max":{"1min":0.70699999999999996,"5min":0.874,"15min":2.0880000000000001},"last":0.45200000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.48799999999999999,"15min":0.49399999999999999},"min":{"1min":0.34000000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.70599999999999996,"5min":0.73499999999999999,"15min":1.399},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.501,"15min":0.50600000000000001},"min":{"1min":0.312,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":3.0459999999999998},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51800000000000002,"15min":0.51300000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79900000000000004,"5min":0.93300000000000005,"15min":2.1280000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60199999999999998,"15min":0.59499999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.35999999999999999},"max":{"1min":0.79900000000000004,"5min":0.88800000000000001,"15min":2.9950000000000001},"last":0.71399999999999997}]},{"osd":4,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79400000000000004,"15min":0.77700000000000002},"min":{"1min":0.51800000000000002,"5min":0.50700000000000001,"15min":0.49099999999999999},"max":{"1min":1.0329999999999999,"5min":1.109,"15min":1.4239999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.78200000000000003,"15min":0.76600000000000001},"min":{"1min":0.51900000000000002,"5min":0.51700000000000002,"15min":0.435},"max":{"1min":1.1479999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.92800000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.85599999999999998,"15min":0.83999999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.143,"5min":1.236,"15min":1.4890000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.81999999999999995,"15min":0.83099999999999996},"min":{"1min":0.53300000000000003,"5min":0.52300000000000002,"15min":0.48799999999999999},"max":{"1min":1.1100000000000001,"5min":1.194,"15min":1.526},"last":0.97799999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79100000000000004,"15min":0.78200000000000003},"min":{"1min":0.57899999999999996,"5min":0.54600000000000004,"15min":0.48599999999999999},"max":{"1min":1.1040000000000001,"5min":1.131,"15min":2.0470000000000002},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.82999999999999996,"15min":0.79800000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.52500000000000002},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.524},"last":1.002}]},{"osd":7,"last update":"Fri Mar 29 08:28:14 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.81399999999999995,"15min":0.80100000000000005},"min":{"1min":0.59099999999999997,"5min":0.54900000000000004,"15min":0.505},"max":{"1min":1.1850000000000001,"5min":1.296,"15min":1.319},"last":0.874},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.81000000000000005,"15min":0.79700000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.214,"5min":1.214,"15min":2},"last":0.90800000000000003}]}]},{"osd":1,"up_from":15,"seq":64424509579,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":2028,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2076672,"data_stored":1234689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45800000000000002,"15min":0.53200000000000003},"min":{"1min":0.28699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.69699999999999995,"5min":0.75900000000000001,"15min":4.2539999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48299999999999998,"15min":0.56399999999999995},"min":{"1min":0.29399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":4.1909999999999998},"last":0.40500000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.31900000000000001},"max":{"1min":0.63600000000000001,"5min":0.71099999999999997,"15min":0.82899999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.34599999999999997,"15min":0.33600000000000002},"max":{"1min":0.67000000000000004,"5min":0.77700000000000002,"15min":0.88200000000000001},"last":0.51900000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.35499999999999998},"max":{"1min":0.75700000000000001,"5min":0.88300000000000001,"15min":1.52},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.496,"5min":0.51900000000000002,"15min":0.52400000000000002},"min":{"1min":0.35999999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68999999999999995,"5min":0.94499999999999995,"15min":1.331},"last":0.44}]},{"osd":4,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.75600000000000001},"min":{"1min":0.60199999999999998,"5min":0.51800000000000002,"15min":0.46200000000000002},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.77600000000000002},"min":{"1min":0.56599999999999995,"5min":0.502,"15min":0.47999999999999998},"max":{"1min":1.212,"5min":1.212,"15min":1.23},"last":0.80100000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79600000000000004,"15min":0.77900000000000003},"min":{"1min":0.59699999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.76800000000000002},"min":{"1min":0.55200000000000005,"5min":0.55100000000000005,"15min":0.46500000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.2509999999999999},"last":0.81200000000000006}]},{"osd":6,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.79300000000000004,"15min":0.77300000000000002},"min":{"1min":0.58299999999999996,"5min":0.54900000000000004,"15min":0.5},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.81499999999999995,"15min":0.82399999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51300000000000001},"max":{"1min":1.0209999999999999,"5min":1.22,"15min":1.22},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.52100000000000002},"max":{"1min":1.1499999999999999,"5min":1.2410000000000001,"15min":1.3089999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.80700000000000005,"15min":0.79000000000000004},"min":{"1min":0.622,"5min":0.503,"15min":0.503},"max":{"1min":1.1759999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.89800000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509578,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1490944,"data_stored":644973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.44400000000000001,"15min":0.51900000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.22800000000000001},"max":{"1min":0.62,"5min":0.75800000000000001,"15min":3.9359999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.497,"5min":0.49299999999999999,"15min":0.66700000000000004},"min":{"1min":0.31900000000000001,"5min":0.28499999999999998,"15min":0.253},"max":{"1min":0.72599999999999998,"5min":0.77000000000000002,"15min":8.0419999999999998},"last":0.34300000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.50900000000000001,"15min":0.52700000000000002},"min":{"1min":0.39300000000000002,"5min":0.34799999999999998,"15min":0.34599999999999997},"max":{"1min":0.68500000000000005,"5min":0.78000000000000003,"15min":1.258},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54500000000000004,"15min":0.63300000000000001},"min":{"1min":0.33800000000000002,"5min":0.32800000000000001,"15min":0.312},"max":{"1min":0.77400000000000002,"5min":0.81399999999999995,"15min":4.907},"last":0.44}]},{"osd":3,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.501,"15min":0.66900000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.28499999999999998},"max":{"1min":0.65100000000000002,"5min":0.83399999999999996,"15min":7.9569999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.61399999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.34799999999999998},"max":{"1min":0.80100000000000005,"5min":0.87,"15min":1.397},"last":0.46500000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79500000000000004,"15min":0.88500000000000001},"min":{"1min":0.56799999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1020000000000001,"5min":1.4399999999999999,"15min":4.4720000000000004},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.79400000000000004,"15min":0.86799999999999999},"min":{"1min":0.51500000000000001,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.141,"5min":1.153,"15min":4.3920000000000003},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79400000000000004,"15min":0.85499999999999998},"min":{"1min":0.56699999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.014,"5min":1.349,"15min":4.3200000000000003},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80300000000000005,"15min":0.82499999999999996},"min":{"1min":0.59599999999999997,"5min":0.56000000000000005,"15min":0.54600000000000004},"max":{"1min":1.081,"5min":1.26,"15min":1.6759999999999999},"last":0.76200000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.79800000000000004,"15min":0.82699999999999996},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.54400000000000004},"max":{"1min":1.0640000000000001,"5min":1.1579999999999999,"15min":1.5920000000000001},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81599999999999995,"15min":0.89300000000000002},"min":{"1min":0.54200000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.165,"5min":1.1990000000000001,"15min":4.4100000000000001},"last":0.85499999999999998}]},{"osd":7,"last update":"Fri Mar 29 08:28:16 2024","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.82399999999999995,"15min":0.90600000000000003},"min":{"1min":0.61499999999999999,"5min":0.54100000000000004,"15min":0.496},"max":{"1min":1.141,"5min":1.3959999999999999,"15min":4.5209999999999999},"last":0.752},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.80500000000000005,"15min":0.88900000000000001},"min":{"1min":0.56499999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.089,"5min":1.2390000000000001,"15min":4.4420000000000002},"last":0.69999999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542283,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33136,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338704,"statfs":{"total":96636764160,"available":96602832896,"internally_reserved":0,"allocated":2064384,"data_stored":1227914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70899999999999996,"15min":0.66400000000000003},"min":{"1min":0.47499999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.98499999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72999999999999998,"15min":0.68400000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.39800000000000002},"max":{"1min":0.84799999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64100000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70699999999999996,"15min":0.71199999999999997},"min":{"1min":0.48499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.008,"5min":1.008,"15min":2.5129999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74299999999999999,"15min":0.70299999999999996},"min":{"1min":0.56799999999999995,"5min":0.51300000000000001,"15min":0.40500000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.85599999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71799999999999997,"15min":0.68600000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92800000000000005,"5min":0.997,"15min":0.997},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.747,"15min":0.71799999999999997},"min":{"1min":0.60899999999999999,"5min":0.47499999999999998,"15min":0.39200000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.77700000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.755,"15min":0.72599999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.47099999999999997},"max":{"1min":1.02,"5min":1.038,"15min":1.038},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.752,"15min":0.74099999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.435},"max":{"1min":0.995,"5min":1.0209999999999999,"15min":1.954},"last":0.81200000000000006}]},{"osd":4,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.624,"15min":0.64000000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.91200000000000003,"15min":0.93500000000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.373},"max":{"1min":0.83299999999999996,"5min":0.94699999999999995,"15min":1.0820000000000001},"last":0.82699999999999996}]},{"osd":6,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.621,"15min":0.65000000000000002},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85999999999999999,"5min":0.93300000000000005,"15min":1.071},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.68500000000000005},"min":{"1min":0.53200000000000003,"5min":0.45300000000000001,"15min":0.432},"max":{"1min":0.92000000000000004,"5min":0.94299999999999995,"15min":1.0609999999999999},"last":0.89800000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90100000000000002,"5min":0.92200000000000004,"15min":1.1200000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70099999999999996,"15min":0.72899999999999998},"min":{"1min":0.46800000000000003,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.94999999999999996,"5min":0.96499999999999997,"15min":1.04},"last":0.98399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:29:23.142 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:29:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.289+0000 7f149bc00640 1 Processor -- start 2024-03-29T08:29:23.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.289+0000 7f149bc00640 1 -- start start 2024-03-29T08:29:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.293+0000 7f149bc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1494153970 0x7f14941611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.293+0000 7f149bc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1494153d50 0x7f149415cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.293+0000 7f149bc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f149415d870 con 0x7f14940ad3e0 2024-03-29T08:29:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.293+0000 7f149bc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f149415db50 con 0x7f1494153970 2024-03-29T08:29:23.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.293+0000 7f149bc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149415de30 con 0x7f1494153d50 2024-03-29T08:29:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.293+0000 7f1499200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1494153970 0x7f14941611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:23.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.293+0000 7f1499200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1494153970 0x7f14941611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39106/0 (socket says 172.21.15.46:39106) 2024-03-29T08:29:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.293+0000 7f1499200640 1 -- 172.21.15.46:0/294799821 learned_addr learned my addr 172.21.15.46:0/294799821 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.293+0000 7f1499c00640 1 --1- 172.21.15.46:0/294799821 >> v1:172.21.15.46:6789/0 conn(0x7f14940ad3e0 0x7f14940ac3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48232/0 (socket says 172.21.15.46:48232) 2024-03-29T08:29:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.293+0000 7f1493e00640 1 --2- 172.21.15.46:0/294799821 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1494153d50 0x7f149415cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.293+0000 7f1499200640 1 -- 172.21.15.46:0/294799821 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1494153d50 msgr2=0x7f149415cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.293+0000 7f1499200640 1 --2- 172.21.15.46:0/294799821 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1494153d50 0x7f149415cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.293+0000 7f1499200640 1 -- 172.21.15.46:0/294799821 >> v1:172.21.15.46:6789/0 conn(0x7f14940ad3e0 legacy=0x7f14940ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.293+0000 7f1499200640 1 -- 172.21.15.46:0/294799821 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14940abea0 con 0x7f1494153970 2024-03-29T08:29:23.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.293+0000 7f1493400640 1 -- 172.21.15.46:0/294799821 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282111385 0 0) 0x7f149415d870 con 0x7f14940ad3e0 2024-03-29T08:29:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.297+0000 7f1499200640 1 --2- 172.21.15.46:0/294799821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1494153970 0x7f14941611e0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f1488009870 tx=0x7f148801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1c63742719bbaa server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.297+0000 7f1493400640 1 -- 172.21.15.46:0/294799821 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14880ab670 con 0x7f1494153970 2024-03-29T08:29:23.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.297+0000 7f1493400640 1 -- 172.21.15.46:0/294799821 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14880ab7d0 con 0x7f1494153970 2024-03-29T08:29:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.297+0000 7f1493400640 1 -- 172.21.15.46:0/294799821 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14880abb60 con 0x7f1494153970 2024-03-29T08:29:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.297+0000 7f149bc00640 1 -- 172.21.15.46:0/294799821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1494153970 msgr2=0x7f14941611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.297+0000 7f149bc00640 1 --2- 172.21.15.46:0/294799821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1494153970 0x7f14941611e0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f1488009870 tx=0x7f148801c560 comp rx=0 tx=0).stop 2024-03-29T08:29:23.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.297+0000 7f149bc00640 1 -- 172.21.15.46:0/294799821 shutdown_connections 2024-03-29T08:29:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.297+0000 7f149bc00640 1 --2- 172.21.15.46:0/294799821 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1494153d50 0x7f149415cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.297+0000 7f149bc00640 1 --2- 172.21.15.46:0/294799821 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1494153970 0x7f14941611e0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.297+0000 7f149bc00640 1 -- 172.21.15.46:0/294799821 >> 172.21.15.46:0/294799821 conn(0x7f1494066110 msgr2=0x7f1494069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.297+0000 7f149bc00640 1 -- 172.21.15.46:0/294799821 shutdown_connections 2024-03-29T08:29:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.297+0000 7f149bc00640 1 -- 172.21.15.46:0/294799821 wait complete. 2024-03-29T08:29:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.297+0000 7f149bc00640 1 Processor -- start 2024-03-29T08:29:23.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.297+0000 7f149bc00640 1 -- start start 2024-03-29T08:29:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f149bc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940ad3e0 0x7f14941443b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f149bc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1494153970 0x7f14940cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f149bc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14940cb8a0 0x7f14940701c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f149bc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14941448f0 con 0x7f14940ad3e0 2024-03-29T08:29:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f149bc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14941472f0 con 0x7f14940cb8a0 2024-03-29T08:29:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f149bc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14940cbed0 con 0x7f1494153970 2024-03-29T08:29:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f1499200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940ad3e0 0x7f14941443b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f1499200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940ad3e0 0x7f14941443b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39114/0 (socket says 172.21.15.46:39114) 2024-03-29T08:29:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f1499200640 1 -- 172.21.15.46:0/4094980594 learned_addr learned my addr 172.21.15.46:0/4094980594 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:23.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f1499c00640 1 --2- 172.21.15.46:0/4094980594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14940cb8a0 0x7f14940701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f1493e00640 1 --2- 172.21.15.46:0/4094980594 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1494153970 0x7f14940cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f1499200640 1 -- 172.21.15.46:0/4094980594 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1494153970 msgr2=0x7f14940cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f1499200640 1 --2- 172.21.15.46:0/4094980594 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1494153970 0x7f14940cb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f1499200640 1 -- 172.21.15.46:0/4094980594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14940cb8a0 msgr2=0x7f14940701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f1499200640 1 --2- 172.21.15.46:0/4094980594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14940cb8a0 0x7f14940701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f1499200640 1 -- 172.21.15.46:0/4094980594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14880094d0 con 0x7f14940ad3e0 2024-03-29T08:29:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f1499c00640 1 --2- 172.21.15.46:0/4094980594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14940cb8a0 0x7f14940701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f1499200640 1 --2- 172.21.15.46:0/4094980594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940ad3e0 0x7f14941443b0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f14880ad040 tx=0x7f148801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f1490c00640 1 -- 172.21.15.46:0/4094980594 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14880b4800 con 0x7f14940ad3e0 2024-03-29T08:29:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f1490c00640 1 -- 172.21.15.46:0/4094980594 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14880b4960 con 0x7f14940ad3e0 2024-03-29T08:29:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f1490c00640 1 -- 172.21.15.46:0/4094980594 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14880b4d10 con 0x7f14940ad3e0 2024-03-29T08:29:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f149bc00640 1 -- 172.21.15.46:0/4094980594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14940cc1b0 con 0x7f14940ad3e0 2024-03-29T08:29:23.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f149bc00640 1 -- 172.21.15.46:0/4094980594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14940cc770 con 0x7f14940ad3e0 2024-03-29T08:29:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f149bc00640 1 -- 172.21.15.46:0/4094980594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1494066c90 con 0x7f14940ad3e0 2024-03-29T08:29:23.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.301+0000 7f1490c00640 1 -- 172.21.15.46:0/4094980594 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f14880b3380 con 0x7f14940ad3e0 2024-03-29T08:29:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.305+0000 7f1490c00640 1 --2- 172.21.15.46:0/4094980594 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1474066690 0x7f1474068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.305+0000 7f1490c00640 1 -- 172.21.15.46:0/4094980594 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f148811e900 con 0x7f14940ad3e0 2024-03-29T08:29:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.305+0000 7f1493e00640 1 --2- 172.21.15.46:0/4094980594 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1474066690 0x7f1474068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:23.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.305+0000 7f1493e00640 1 --2- 172.21.15.46:0/4094980594 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1474066690 0x7f1474068b50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f1494069fb0 tx=0x7f1480008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:23.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.309+0000 7f1490c00640 1 -- 172.21.15.46:0/4094980594 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f14880ebbf0 con 0x7f14940ad3e0 2024-03-29T08:29:23.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.465+0000 7f149bc00640 1 -- 172.21.15.46:0/4094980594 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f14940adc40 con 0x7f1474066690 2024-03-29T08:29:23.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.473+0000 7f1490c00640 1 -- 172.21.15.46:0/4094980594 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289665 (secure 0 0 0) 0x7f14800095e0 con 0x7f1474066690 2024-03-29T08:29:23.482 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:23.484 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:29:23.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.477+0000 7f149bc00640 1 -- 172.21.15.46:0/4094980594 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1474066690 msgr2=0x7f1474068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:23.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.477+0000 7f149bc00640 1 --2- 172.21.15.46:0/4094980594 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1474066690 0x7f1474068b50 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f1494069fb0 tx=0x7f1480008040 comp rx=0 tx=0).stop 2024-03-29T08:29:23.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.477+0000 7f149bc00640 1 -- 172.21.15.46:0/4094980594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940ad3e0 msgr2=0x7f14941443b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:23.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.477+0000 7f149bc00640 1 --2- 172.21.15.46:0/4094980594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940ad3e0 0x7f14941443b0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f14880ad040 tx=0x7f148801ada0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.477+0000 7f149bc00640 1 -- 172.21.15.46:0/4094980594 shutdown_connections 2024-03-29T08:29:23.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.477+0000 7f149bc00640 1 --2- 172.21.15.46:0/4094980594 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1474066690 0x7f1474068b50 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.477+0000 7f149bc00640 1 --2- 172.21.15.46:0/4094980594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14940cb8a0 0x7f14940701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.477+0000 7f149bc00640 1 --2- 172.21.15.46:0/4094980594 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1494153970 0x7f14940cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.477+0000 7f149bc00640 1 --2- 172.21.15.46:0/4094980594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940ad3e0 0x7f14941443b0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.477+0000 7f149bc00640 1 -- 172.21.15.46:0/4094980594 >> 172.21.15.46:0/4094980594 conn(0x7f1494066110 msgr2=0x7f14940ac3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:23.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.477+0000 7f149bc00640 1 -- 172.21.15.46:0/4094980594 shutdown_connections 2024-03-29T08:29:23.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.477+0000 7f149bc00640 1 -- 172.21.15.46:0/4094980594 wait complete. 2024-03-29T08:29:23.502 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-03-29T08:29:22.212778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261840,"kb_used_data":12904,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754712880,"statfs":{"total":773094113280,"available":772825989120,"internally_reserved":0,"allocated":13213696,"data_stored":6390428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001956"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.179300+0000","last_change":"2024-03-29T08:29:14.504298+0000","last_active":"2024-03-29T08:29:15.179300+0000","last_peered":"2024-03-29T08:29:15.179300+0000","last_clean":"2024-03-29T08:29:15.179300+0000","last_became_active":"2024-03-29T08:29:14.504060+0000","last_became_peered":"2024-03-29T08:29:14.504060+0000","last_unstale":"2024-03-29T08:29:15.179300+0000","last_undegraded":"2024-03-29T08:29:15.179300+0000","last_fullsized":"2024-03-29T08:29:15.179300+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:47:24.542958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171755+0000","last_change":"2024-03-29T08:29:14.488152+0000","last_active":"2024-03-29T08:29:15.171755+0000","last_peered":"2024-03-29T08:29:15.171755+0000","last_clean":"2024-03-29T08:29:15.171755+0000","last_became_active":"2024-03-29T08:29:14.487884+0000","last_became_peered":"2024-03-29T08:29:14.487884+0000","last_unstale":"2024-03-29T08:29:15.171755+0000","last_undegraded":"2024-03-29T08:29:15.171755+0000","last_fullsized":"2024-03-29T08:29:15.171755+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:49:14.237508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171818+0000","last_change":"2024-03-29T08:29:14.490304+0000","last_active":"2024-03-29T08:29:15.171818+0000","last_peered":"2024-03-29T08:29:15.171818+0000","last_clean":"2024-03-29T08:29:15.171818+0000","last_became_active":"2024-03-29T08:29:14.489916+0000","last_became_peered":"2024-03-29T08:29:14.489916+0000","last_unstale":"2024-03-29T08:29:15.171818+0000","last_undegraded":"2024-03-29T08:29:15.171818+0000","last_fullsized":"2024-03-29T08:29:15.171818+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:10:55.868596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.177652+0000","last_change":"2024-03-29T08:29:14.501687+0000","last_active":"2024-03-29T08:29:15.177652+0000","last_peered":"2024-03-29T08:29:15.177652+0000","last_clean":"2024-03-29T08:29:15.177652+0000","last_became_active":"2024-03-29T08:29:14.501377+0000","last_became_peered":"2024-03-29T08:29:14.501377+0000","last_unstale":"2024-03-29T08:29:15.177652+0000","last_undegraded":"2024-03-29T08:29:15.177652+0000","last_fullsized":"2024-03-29T08:29:15.177652+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:38:29.985399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.183339+0000","last_change":"2024-03-29T08:29:14.504615+0000","last_active":"2024-03-29T08:29:15.183339+0000","last_peered":"2024-03-29T08:29:15.183339+0000","last_clean":"2024-03-29T08:29:15.183339+0000","last_became_active":"2024-03-29T08:29:14.504347+0000","last_became_peered":"2024-03-29T08:29:14.504347+0000","last_unstale":"2024-03-29T08:29:15.183339+0000","last_undegraded":"2024-03-29T08:29:15.183339+0000","last_fullsized":"2024-03-29T08:29:15.183339+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:25:45.763218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.506613+0000","last_change":"2024-03-29T08:29:14.496527+0000","last_active":"2024-03-29T08:29:14.506613+0000","last_peered":"2024-03-29T08:29:14.506613+0000","last_clean":"2024-03-29T08:29:14.506613+0000","last_became_active":"2024-03-29T08:29:14.496253+0000","last_became_peered":"2024-03-29T08:29:14.496253+0000","last_unstale":"2024-03-29T08:29:14.506613+0000","last_undegraded":"2024-03-29T08:29:14.506613+0000","last_fullsized":"2024-03-29T08:29:14.506613+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:35:51.101298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.174943+0000","last_change":"2024-03-29T08:29:14.494643+0000","last_active":"2024-03-29T08:29:15.174943+0000","last_peered":"2024-03-29T08:29:15.174943+0000","last_clean":"2024-03-29T08:29:15.174943+0000","last_became_active":"2024-03-29T08:29:14.494385+0000","last_became_peered":"2024-03-29T08:29:14.494385+0000","last_unstale":"2024-03-29T08:29:15.174943+0000","last_undegraded":"2024-03-29T08:29:15.174943+0000","last_fullsized":"2024-03-29T08:29:15.174943+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:11:26.639183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.174916+0000","last_change":"2024-03-29T08:29:14.489843+0000","last_active":"2024-03-29T08:29:15.174916+0000","last_peered":"2024-03-29T08:29:15.174916+0000","last_clean":"2024-03-29T08:29:15.174916+0000","last_became_active":"2024-03-29T08:29:14.489598+0000","last_became_peered":"2024-03-29T08:29:14.489598+0000","last_unstale":"2024-03-29T08:29:15.174916+0000","last_undegraded":"2024-03-29T08:29:15.174916+0000","last_fullsized":"2024-03-29T08:29:15.174916+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:49:19.274668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.179317+0000","last_change":"2024-03-29T08:29:14.507881+0000","last_active":"2024-03-29T08:29:15.179317+0000","last_peered":"2024-03-29T08:29:15.179317+0000","last_clean":"2024-03-29T08:29:15.179317+0000","last_became_active":"2024-03-29T08:29:14.507445+0000","last_became_peered":"2024-03-29T08:29:14.507445+0000","last_unstale":"2024-03-29T08:29:15.179317+0000","last_undegraded":"2024-03-29T08:29:15.179317+0000","last_fullsized":"2024-03-29T08:29:15.179317+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:50:39.114131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.183431+0000","last_change":"2024-03-29T08:29:14.500130+0000","last_active":"2024-03-29T08:29:15.183431+0000","last_peered":"2024-03-29T08:29:15.183431+0000","last_clean":"2024-03-29T08:29:15.183431+0000","last_became_active":"2024-03-29T08:29:14.499827+0000","last_became_peered":"2024-03-29T08:29:14.499827+0000","last_unstale":"2024-03-29T08:29:15.183431+0000","last_undegraded":"2024-03-29T08:29:15.183431+0000","last_fullsized":"2024-03-29T08:29:15.183431+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:01.476642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.175031+0000","last_change":"2024-03-29T08:29:14.494953+0000","last_active":"2024-03-29T08:29:15.175031+0000","last_peered":"2024-03-29T08:29:15.175031+0000","last_clean":"2024-03-29T08:29:15.175031+0000","last_became_active":"2024-03-29T08:29:14.494599+0000","last_became_peered":"2024-03-29T08:29:14.494599+0000","last_unstale":"2024-03-29T08:29:15.175031+0000","last_undegraded":"2024-03-29T08:29:15.175031+0000","last_fullsized":"2024-03-29T08:29:15.175031+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:49:02.417397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.183408+0000","last_change":"2024-03-29T08:29:14.499160+0000","last_active":"2024-03-29T08:29:15.183408+0000","last_peered":"2024-03-29T08:29:15.183408+0000","last_clean":"2024-03-29T08:29:15.183408+0000","last_became_active":"2024-03-29T08:29:14.498900+0000","last_became_peered":"2024-03-29T08:29:14.498900+0000","last_unstale":"2024-03-29T08:29:15.183408+0000","last_undegraded":"2024-03-29T08:29:15.183408+0000","last_fullsized":"2024-03-29T08:29:15.183408+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:19.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171728+0000","last_change":"2024-03-29T08:29:14.486530+0000","last_active":"2024-03-29T08:29:15.171728+0000","last_peered":"2024-03-29T08:29:15.171728+0000","last_clean":"2024-03-29T08:29:15.171728+0000","last_became_active":"2024-03-29T08:29:14.486158+0000","last_became_peered":"2024-03-29T08:29:14.486158+0000","last_unstale":"2024-03-29T08:29:15.171728+0000","last_undegraded":"2024-03-29T08:29:15.171728+0000","last_fullsized":"2024-03-29T08:29:15.171728+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:40.282997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171359+0000","last_change":"2024-03-29T08:29:14.507343+0000","last_active":"2024-03-29T08:29:15.171359+0000","last_peered":"2024-03-29T08:29:15.171359+0000","last_clean":"2024-03-29T08:29:15.171359+0000","last_became_active":"2024-03-29T08:29:14.507072+0000","last_became_peered":"2024-03-29T08:29:14.507072+0000","last_unstale":"2024-03-29T08:29:15.171359+0000","last_undegraded":"2024-03-29T08:29:15.171359+0000","last_fullsized":"2024-03-29T08:29:15.171359+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:24:19.958634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.179114+0000","last_change":"2024-03-29T08:29:14.498131+0000","last_active":"2024-03-29T08:29:15.179114+0000","last_peered":"2024-03-29T08:29:15.179114+0000","last_clean":"2024-03-29T08:29:15.179114+0000","last_became_active":"2024-03-29T08:29:14.497856+0000","last_became_peered":"2024-03-29T08:29:14.497856+0000","last_unstale":"2024-03-29T08:29:15.179114+0000","last_undegraded":"2024-03-29T08:29:15.179114+0000","last_fullsized":"2024-03-29T08:29:15.179114+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:43:24.083895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.183224+0000","last_change":"2024-03-29T08:29:14.499902+0000","last_active":"2024-03-29T08:29:15.183224+0000","last_peered":"2024-03-29T08:29:15.183224+0000","last_clean":"2024-03-29T08:29:15.183224+0000","last_became_active":"2024-03-29T08:29:14.499642+0000","last_became_peered":"2024-03-29T08:29:14.499642+0000","last_unstale":"2024-03-29T08:29:15.183224+0000","last_undegraded":"2024-03-29T08:29:15.183224+0000","last_fullsized":"2024-03-29T08:29:15.183224+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:06:54.374100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171844+0000","last_change":"2024-03-29T08:29:14.496277+0000","last_active":"2024-03-29T08:29:15.171844+0000","last_peered":"2024-03-29T08:29:15.171844+0000","last_clean":"2024-03-29T08:29:15.171844+0000","last_became_active":"2024-03-29T08:29:14.496028+0000","last_became_peered":"2024-03-29T08:29:14.496028+0000","last_unstale":"2024-03-29T08:29:15.171844+0000","last_undegraded":"2024-03-29T08:29:15.171844+0000","last_fullsized":"2024-03-29T08:29:15.171844+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:08:59.334138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.175048+0000","last_change":"2024-03-29T08:29:14.503103+0000","last_active":"2024-03-29T08:29:15.175048+0000","last_peered":"2024-03-29T08:29:15.175048+0000","last_clean":"2024-03-29T08:29:15.175048+0000","last_became_active":"2024-03-29T08:29:14.502841+0000","last_became_peered":"2024-03-29T08:29:14.502841+0000","last_unstale":"2024-03-29T08:29:15.175048+0000","last_undegraded":"2024-03-29T08:29:15.175048+0000","last_fullsized":"2024-03-29T08:29:15.175048+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:11:21.620572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171835+0000","last_change":"2024-03-29T08:29:14.491176+0000","last_active":"2024-03-29T08:29:15.171835+0000","last_peered":"2024-03-29T08:29:15.171835+0000","last_clean":"2024-03-29T08:29:15.171835+0000","last_became_active":"2024-03-29T08:29:14.490845+0000","last_became_peered":"2024-03-29T08:29:14.490845+0000","last_unstale":"2024-03-29T08:29:15.171835+0000","last_undegraded":"2024-03-29T08:29:15.171835+0000","last_fullsized":"2024-03-29T08:29:15.171835+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:44:41.312640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171809+0000","last_change":"2024-03-29T08:29:14.501383+0000","last_active":"2024-03-29T08:29:15.171809+0000","last_peered":"2024-03-29T08:29:15.171809+0000","last_clean":"2024-03-29T08:29:15.171809+0000","last_became_active":"2024-03-29T08:29:14.501106+0000","last_became_peered":"2024-03-29T08:29:14.501106+0000","last_unstale":"2024-03-29T08:29:15.171809+0000","last_undegraded":"2024-03-29T08:29:15.171809+0000","last_fullsized":"2024-03-29T08:29:15.171809+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:23:10.605671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.175003+0000","last_change":"2024-03-29T08:29:14.497743+0000","last_active":"2024-03-29T08:29:15.175003+0000","last_peered":"2024-03-29T08:29:15.175003+0000","last_clean":"2024-03-29T08:29:15.175003+0000","last_became_active":"2024-03-29T08:29:14.497338+0000","last_became_peered":"2024-03-29T08:29:14.497338+0000","last_unstale":"2024-03-29T08:29:15.175003+0000","last_undegraded":"2024-03-29T08:29:15.175003+0000","last_fullsized":"2024-03-29T08:29:15.175003+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:24.513091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.506632+0000","last_change":"2024-03-29T08:29:14.499378+0000","last_active":"2024-03-29T08:29:14.506632+0000","last_peered":"2024-03-29T08:29:14.506632+0000","last_clean":"2024-03-29T08:29:14.506632+0000","last_became_active":"2024-03-29T08:29:14.499000+0000","last_became_peered":"2024-03-29T08:29:14.499000+0000","last_unstale":"2024-03-29T08:29:14.506632+0000","last_undegraded":"2024-03-29T08:29:14.506632+0000","last_fullsized":"2024-03-29T08:29:14.506632+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:57:23.884319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171351+0000","last_change":"2024-03-29T08:29:14.502935+0000","last_active":"2024-03-29T08:29:15.171351+0000","last_peered":"2024-03-29T08:29:15.171351+0000","last_clean":"2024-03-29T08:29:15.171351+0000","last_became_active":"2024-03-29T08:29:14.502701+0000","last_became_peered":"2024-03-29T08:29:14.502701+0000","last_unstale":"2024-03-29T08:29:15.171351+0000","last_undegraded":"2024-03-29T08:29:15.171351+0000","last_fullsized":"2024-03-29T08:29:15.171351+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:35.756569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.177512+0000","last_change":"2024-03-29T08:29:14.507544+0000","last_active":"2024-03-29T08:29:15.177512+0000","last_peered":"2024-03-29T08:29:15.177512+0000","last_clean":"2024-03-29T08:29:15.177512+0000","last_became_active":"2024-03-29T08:29:14.507285+0000","last_became_peered":"2024-03-29T08:29:14.507285+0000","last_unstale":"2024-03-29T08:29:15.177512+0000","last_undegraded":"2024-03-29T08:29:15.177512+0000","last_fullsized":"2024-03-29T08:29:15.177512+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:50:14.687728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.175064+0000","last_change":"2024-03-29T08:29:14.492853+0000","last_active":"2024-03-29T08:29:15.175064+0000","last_peered":"2024-03-29T08:29:15.175064+0000","last_clean":"2024-03-29T08:29:15.175064+0000","last_became_active":"2024-03-29T08:29:14.492537+0000","last_became_peered":"2024-03-29T08:29:14.492537+0000","last_unstale":"2024-03-29T08:29:15.175064+0000","last_undegraded":"2024-03-29T08:29:15.175064+0000","last_fullsized":"2024-03-29T08:29:15.175064+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:08:28.043449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.495245+0000","last_change":"2024-03-29T08:29:14.491656+0000","last_active":"2024-03-29T08:29:14.495245+0000","last_peered":"2024-03-29T08:29:14.495245+0000","last_clean":"2024-03-29T08:29:14.495245+0000","last_became_active":"2024-03-29T08:29:14.490927+0000","last_became_peered":"2024-03-29T08:29:14.490927+0000","last_unstale":"2024-03-29T08:29:14.495245+0000","last_undegraded":"2024-03-29T08:29:14.495245+0000","last_fullsized":"2024-03-29T08:29:14.495245+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:10:44.097211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.177695+0000","last_change":"2024-03-29T08:29:14.508869+0000","last_active":"2024-03-29T08:29:15.177695+0000","last_peered":"2024-03-29T08:29:15.177695+0000","last_clean":"2024-03-29T08:29:15.177695+0000","last_became_active":"2024-03-29T08:29:14.508466+0000","last_became_peered":"2024-03-29T08:29:14.508466+0000","last_unstale":"2024-03-29T08:29:15.177695+0000","last_undegraded":"2024-03-29T08:29:15.177695+0000","last_fullsized":"2024-03-29T08:29:15.177695+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:08:28.619413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505602+0000","last_change":"2024-03-29T08:29:14.497215+0000","last_active":"2024-03-29T08:29:14.505602+0000","last_peered":"2024-03-29T08:29:14.505602+0000","last_clean":"2024-03-29T08:29:14.505602+0000","last_became_active":"2024-03-29T08:29:14.496926+0000","last_became_peered":"2024-03-29T08:29:14.496926+0000","last_unstale":"2024-03-29T08:29:14.505602+0000","last_undegraded":"2024-03-29T08:29:14.505602+0000","last_fullsized":"2024-03-29T08:29:14.505602+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:46:53.749284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171376+0000","last_change":"2024-03-29T08:29:14.507369+0000","last_active":"2024-03-29T08:29:15.171376+0000","last_peered":"2024-03-29T08:29:15.171376+0000","last_clean":"2024-03-29T08:29:15.171376+0000","last_became_active":"2024-03-29T08:29:14.507164+0000","last_became_peered":"2024-03-29T08:29:14.507164+0000","last_unstale":"2024-03-29T08:29:15.171376+0000","last_undegraded":"2024-03-29T08:29:15.171376+0000","last_fullsized":"2024-03-29T08:29:15.171376+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:20:13.956872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.174986+0000","last_change":"2024-03-29T08:29:14.503526+0000","last_active":"2024-03-29T08:29:15.174986+0000","last_peered":"2024-03-29T08:29:15.174986+0000","last_clean":"2024-03-29T08:29:15.174986+0000","last_became_active":"2024-03-29T08:29:14.503149+0000","last_became_peered":"2024-03-29T08:29:14.503149+0000","last_unstale":"2024-03-29T08:29:15.174986+0000","last_undegraded":"2024-03-29T08:29:15.174986+0000","last_fullsized":"2024-03-29T08:29:15.174986+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:46:13.948298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.179131+0000","last_change":"2024-03-29T08:29:14.500403+0000","last_active":"2024-03-29T08:29:15.179131+0000","last_peered":"2024-03-29T08:29:15.179131+0000","last_clean":"2024-03-29T08:29:15.179131+0000","last_became_active":"2024-03-29T08:29:14.500132+0000","last_became_peered":"2024-03-29T08:29:14.500132+0000","last_unstale":"2024-03-29T08:29:15.179131+0000","last_undegraded":"2024-03-29T08:29:15.179131+0000","last_fullsized":"2024-03-29T08:29:15.179131+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:09:34.129852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.183244+0000","last_change":"2024-03-29T08:29:14.499479+0000","last_active":"2024-03-29T08:29:15.183244+0000","last_peered":"2024-03-29T08:29:15.183244+0000","last_clean":"2024-03-29T08:29:15.183244+0000","last_became_active":"2024-03-29T08:29:14.498994+0000","last_became_peered":"2024-03-29T08:29:14.498994+0000","last_unstale":"2024-03-29T08:29:15.183244+0000","last_undegraded":"2024-03-29T08:29:15.183244+0000","last_fullsized":"2024-03-29T08:29:15.183244+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:25:13.598107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.179265+0000","last_change":"2024-03-29T08:29:14.511693+0000","last_active":"2024-03-29T08:29:15.179265+0000","last_peered":"2024-03-29T08:29:15.179265+0000","last_clean":"2024-03-29T08:29:15.179265+0000","last_became_active":"2024-03-29T08:29:14.511041+0000","last_became_peered":"2024-03-29T08:29:14.511041+0000","last_unstale":"2024-03-29T08:29:15.179265+0000","last_undegraded":"2024-03-29T08:29:15.179265+0000","last_fullsized":"2024-03-29T08:29:15.179265+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:06.874362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"120'4","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171445+0000","last_change":"2024-03-29T08:29:14.086060+0000","last_active":"2024-03-29T08:29:15.171445+0000","last_peered":"2024-03-29T08:29:15.171445+0000","last_clean":"2024-03-29T08:29:15.171445+0000","last_became_active":"2024-03-29T08:29:14.085788+0000","last_became_peered":"2024-03-29T08:29:14.085788+0000","last_unstale":"2024-03-29T08:29:15.171445+0000","last_undegraded":"2024-03-29T08:29:15.171445+0000","last_fullsized":"2024-03-29T08:29:15.171445+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:52:04.161387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171434+0000","last_change":"2024-03-29T08:29:14.503865+0000","last_active":"2024-03-29T08:29:15.171434+0000","last_peered":"2024-03-29T08:29:15.171434+0000","last_clean":"2024-03-29T08:29:15.171434+0000","last_became_active":"2024-03-29T08:29:14.503622+0000","last_became_peered":"2024-03-29T08:29:14.503622+0000","last_unstale":"2024-03-29T08:29:15.171434+0000","last_undegraded":"2024-03-29T08:29:15.171434+0000","last_fullsized":"2024-03-29T08:29:15.171434+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:48:23.465613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"120'1","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171790+0000","last_change":"2024-03-29T08:29:14.084881+0000","last_active":"2024-03-29T08:29:15.171790+0000","last_peered":"2024-03-29T08:29:15.171790+0000","last_clean":"2024-03-29T08:29:15.171790+0000","last_became_active":"2024-03-29T08:29:14.084587+0000","last_became_peered":"2024-03-29T08:29:14.084587+0000","last_unstale":"2024-03-29T08:29:15.171790+0000","last_undegraded":"2024-03-29T08:29:15.171790+0000","last_fullsized":"2024-03-29T08:29:15.171790+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:45.558553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.506005+0000","last_change":"2024-03-29T08:29:14.500004+0000","last_active":"2024-03-29T08:29:14.506005+0000","last_peered":"2024-03-29T08:29:14.506005+0000","last_clean":"2024-03-29T08:29:14.506005+0000","last_became_active":"2024-03-29T08:29:14.499637+0000","last_became_peered":"2024-03-29T08:29:14.499637+0000","last_unstale":"2024-03-29T08:29:14.506005+0000","last_undegraded":"2024-03-29T08:29:14.506005+0000","last_fullsized":"2024-03-29T08:29:14.506005+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:23:32.467222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"120'2","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.177705+0000","last_change":"2024-03-29T08:29:14.089964+0000","last_active":"2024-03-29T08:29:15.177705+0000","last_peered":"2024-03-29T08:29:15.177705+0000","last_clean":"2024-03-29T08:29:15.177705+0000","last_became_active":"2024-03-29T08:29:14.088099+0000","last_became_peered":"2024-03-29T08:29:14.088099+0000","last_unstale":"2024-03-29T08:29:15.177705+0000","last_undegraded":"2024-03-29T08:29:15.177705+0000","last_fullsized":"2024-03-29T08:29:15.177705+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:54:05.868646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171827+0000","last_change":"2024-03-29T08:29:14.491102+0000","last_active":"2024-03-29T08:29:15.171827+0000","last_peered":"2024-03-29T08:29:15.171827+0000","last_clean":"2024-03-29T08:29:15.171827+0000","last_became_active":"2024-03-29T08:29:14.490878+0000","last_became_peered":"2024-03-29T08:29:14.490878+0000","last_unstale":"2024-03-29T08:29:15.171827+0000","last_undegraded":"2024-03-29T08:29:15.171827+0000","last_fullsized":"2024-03-29T08:29:15.171827+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:06:54.374100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"120'2","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.177663+0000","last_change":"2024-03-29T08:29:14.097684+0000","last_active":"2024-03-29T08:29:15.177663+0000","last_peered":"2024-03-29T08:29:15.177663+0000","last_clean":"2024-03-29T08:29:15.177663+0000","last_became_active":"2024-03-29T08:29:14.095879+0000","last_became_peered":"2024-03-29T08:29:14.095879+0000","last_unstale":"2024-03-29T08:29:15.177663+0000","last_undegraded":"2024-03-29T08:29:15.177663+0000","last_fullsized":"2024-03-29T08:29:15.177663+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:50:38.108416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.177632+0000","last_change":"2024-03-29T08:29:14.507697+0000","last_active":"2024-03-29T08:29:15.177632+0000","last_peered":"2024-03-29T08:29:15.177632+0000","last_clean":"2024-03-29T08:29:15.177632+0000","last_became_active":"2024-03-29T08:29:14.507356+0000","last_became_peered":"2024-03-29T08:29:14.507356+0000","last_unstale":"2024-03-29T08:29:15.177632+0000","last_undegraded":"2024-03-29T08:29:15.177632+0000","last_fullsized":"2024-03-29T08:29:15.177632+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:18:11.283108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"120'1","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171800+0000","last_change":"2024-03-29T08:29:14.086806+0000","last_active":"2024-03-29T08:29:15.171800+0000","last_peered":"2024-03-29T08:29:15.171800+0000","last_clean":"2024-03-29T08:29:15.171800+0000","last_became_active":"2024-03-29T08:29:14.086468+0000","last_became_peered":"2024-03-29T08:29:14.086468+0000","last_unstale":"2024-03-29T08:29:15.171800+0000","last_undegraded":"2024-03-29T08:29:15.171800+0000","last_fullsized":"2024-03-29T08:29:15.171800+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:37:57.757695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.174995+0000","last_change":"2024-03-29T08:29:14.498041+0000","last_active":"2024-03-29T08:29:15.174995+0000","last_peered":"2024-03-29T08:29:15.174995+0000","last_clean":"2024-03-29T08:29:15.174995+0000","last_became_active":"2024-03-29T08:29:14.497768+0000","last_became_peered":"2024-03-29T08:29:14.497768+0000","last_unstale":"2024-03-29T08:29:15.174995+0000","last_undegraded":"2024-03-29T08:29:15.174995+0000","last_fullsized":"2024-03-29T08:29:15.174995+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:16:55.518674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"120'1","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.179189+0000","last_change":"2024-03-29T08:29:14.088920+0000","last_active":"2024-03-29T08:29:15.179189+0000","last_peered":"2024-03-29T08:29:15.179189+0000","last_clean":"2024-03-29T08:29:15.179189+0000","last_became_active":"2024-03-29T08:29:14.088593+0000","last_became_peered":"2024-03-29T08:29:14.088593+0000","last_unstale":"2024-03-29T08:29:15.179189+0000","last_undegraded":"2024-03-29T08:29:15.179189+0000","last_fullsized":"2024-03-29T08:29:15.179189+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:51.446071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.183257+0000","last_change":"2024-03-29T08:29:14.499273+0000","last_active":"2024-03-29T08:29:15.183257+0000","last_peered":"2024-03-29T08:29:15.183257+0000","last_clean":"2024-03-29T08:29:15.183257+0000","last_became_active":"2024-03-29T08:29:14.498368+0000","last_became_peered":"2024-03-29T08:29:14.498368+0000","last_unstale":"2024-03-29T08:29:15.183257+0000","last_undegraded":"2024-03-29T08:29:15.183257+0000","last_fullsized":"2024-03-29T08:29:15.183257+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:23:10.605671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.174977+0000","last_change":"2024-03-29T08:29:14.079054+0000","last_active":"2024-03-29T08:29:15.174977+0000","last_peered":"2024-03-29T08:29:15.174977+0000","last_clean":"2024-03-29T08:29:15.174977+0000","last_became_active":"2024-03-29T08:29:14.078272+0000","last_became_peered":"2024-03-29T08:29:14.078272+0000","last_unstale":"2024-03-29T08:29:15.174977+0000","last_undegraded":"2024-03-29T08:29:15.174977+0000","last_fullsized":"2024-03-29T08:29:15.174977+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T08:57:54.510656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.174962+0000","last_change":"2024-03-29T08:29:14.495103+0000","last_active":"2024-03-29T08:29:15.174962+0000","last_peered":"2024-03-29T08:29:15.174962+0000","last_clean":"2024-03-29T08:29:15.174962+0000","last_became_active":"2024-03-29T08:29:14.494873+0000","last_became_peered":"2024-03-29T08:29:14.494873+0000","last_unstale":"2024-03-29T08:29:15.174962+0000","last_undegraded":"2024-03-29T08:29:15.174962+0000","last_fullsized":"2024-03-29T08:29:15.174962+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:49:15.540195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.183269+0000","last_change":"2024-03-29T08:29:14.091783+0000","last_active":"2024-03-29T08:29:15.183269+0000","last_peered":"2024-03-29T08:29:15.183269+0000","last_clean":"2024-03-29T08:29:15.183269+0000","last_became_active":"2024-03-29T08:29:14.091531+0000","last_became_peered":"2024-03-29T08:29:14.091531+0000","last_unstale":"2024-03-29T08:29:15.183269+0000","last_undegraded":"2024-03-29T08:29:15.183269+0000","last_fullsized":"2024-03-29T08:29:15.183269+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:32:23.859154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171772+0000","last_change":"2024-03-29T08:29:14.498939+0000","last_active":"2024-03-29T08:29:15.171772+0000","last_peered":"2024-03-29T08:29:15.171772+0000","last_clean":"2024-03-29T08:29:15.171772+0000","last_became_active":"2024-03-29T08:29:14.498589+0000","last_became_peered":"2024-03-29T08:29:14.498589+0000","last_unstale":"2024-03-29T08:29:15.171772+0000","last_undegraded":"2024-03-29T08:29:15.171772+0000","last_fullsized":"2024-03-29T08:29:15.171772+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:32:24.719500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.177729+0000","last_change":"2024-03-29T08:29:14.096474+0000","last_active":"2024-03-29T08:29:15.177729+0000","last_peered":"2024-03-29T08:29:15.177729+0000","last_clean":"2024-03-29T08:29:15.177729+0000","last_became_active":"2024-03-29T08:29:14.094336+0000","last_became_peered":"2024-03-29T08:29:14.094336+0000","last_unstale":"2024-03-29T08:29:15.177729+0000","last_undegraded":"2024-03-29T08:29:15.177729+0000","last_fullsized":"2024-03-29T08:29:15.177729+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:09:39.277282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171807+0000","last_change":"2024-03-29T08:29:14.487254+0000","last_active":"2024-03-29T08:29:15.171807+0000","last_peered":"2024-03-29T08:29:15.171807+0000","last_clean":"2024-03-29T08:29:15.171807+0000","last_became_active":"2024-03-29T08:29:14.485472+0000","last_became_peered":"2024-03-29T08:29:14.485472+0000","last_unstale":"2024-03-29T08:29:15.171807+0000","last_undegraded":"2024-03-29T08:29:15.171807+0000","last_fullsized":"2024-03-29T08:29:15.171807+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:08:57.590159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.183441+0000","last_change":"2024-03-29T08:29:14.092400+0000","last_active":"2024-03-29T08:29:15.183441+0000","last_peered":"2024-03-29T08:29:15.183441+0000","last_clean":"2024-03-29T08:29:15.183441+0000","last_became_active":"2024-03-29T08:29:14.091857+0000","last_became_peered":"2024-03-29T08:29:14.091857+0000","last_unstale":"2024-03-29T08:29:15.183441+0000","last_undegraded":"2024-03-29T08:29:15.183441+0000","last_fullsized":"2024-03-29T08:29:15.183441+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:37:57.757695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.495807+0000","last_change":"2024-03-29T08:29:14.488940+0000","last_active":"2024-03-29T08:29:14.495807+0000","last_peered":"2024-03-29T08:29:14.495807+0000","last_clean":"2024-03-29T08:29:14.495807+0000","last_became_active":"2024-03-29T08:29:14.488529+0000","last_became_peered":"2024-03-29T08:29:14.488529+0000","last_unstale":"2024-03-29T08:29:14.495807+0000","last_undegraded":"2024-03-29T08:29:14.495807+0000","last_fullsized":"2024-03-29T08:29:14.495807+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:45.863920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"120'3","reported_seq":14,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.183420+0000","last_change":"2024-03-29T08:29:14.092891+0000","last_active":"2024-03-29T08:29:15.183420+0000","last_peered":"2024-03-29T08:29:15.183420+0000","last_clean":"2024-03-29T08:29:15.183420+0000","last_became_active":"2024-03-29T08:29:14.092632+0000","last_became_peered":"2024-03-29T08:29:14.092632+0000","last_unstale":"2024-03-29T08:29:15.183420+0000","last_undegraded":"2024-03-29T08:29:15.183420+0000","last_fullsized":"2024-03-29T08:29:15.183420+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:52:04.161387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.175040+0000","last_change":"2024-03-29T08:29:14.489765+0000","last_active":"2024-03-29T08:29:15.175040+0000","last_peered":"2024-03-29T08:29:15.175040+0000","last_clean":"2024-03-29T08:29:15.175040+0000","last_became_active":"2024-03-29T08:29:14.489440+0000","last_became_peered":"2024-03-29T08:29:14.489440+0000","last_unstale":"2024-03-29T08:29:15.175040+0000","last_undegraded":"2024-03-29T08:29:15.175040+0000","last_fullsized":"2024-03-29T08:29:15.175040+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:53:42.976104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"120'6","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.177674+0000","last_change":"2024-03-29T08:29:14.096725+0000","last_active":"2024-03-29T08:29:15.177674+0000","last_peered":"2024-03-29T08:29:15.177674+0000","last_clean":"2024-03-29T08:29:15.177674+0000","last_became_active":"2024-03-29T08:29:14.094695+0000","last_became_peered":"2024-03-29T08:29:14.094695+0000","last_unstale":"2024-03-29T08:29:15.177674+0000","last_undegraded":"2024-03-29T08:29:15.177674+0000","last_fullsized":"2024-03-29T08:29:15.177674+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:15:28.507662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.183313+0000","last_change":"2024-03-29T08:29:14.499972+0000","last_active":"2024-03-29T08:29:15.183313+0000","last_peered":"2024-03-29T08:29:15.183313+0000","last_clean":"2024-03-29T08:29:15.183313+0000","last_became_active":"2024-03-29T08:29:14.499698+0000","last_became_peered":"2024-03-29T08:29:14.499698+0000","last_unstale":"2024-03-29T08:29:15.183313+0000","last_undegraded":"2024-03-29T08:29:15.183313+0000","last_fullsized":"2024-03-29T08:29:15.183313+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:44:41.312640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"120'2","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.183359+0000","last_change":"2024-03-29T08:29:14.092574+0000","last_active":"2024-03-29T08:29:15.183359+0000","last_peered":"2024-03-29T08:29:15.183359+0000","last_clean":"2024-03-29T08:29:15.183359+0000","last_became_active":"2024-03-29T08:29:14.092004+0000","last_became_peered":"2024-03-29T08:29:14.092004+0000","last_unstale":"2024-03-29T08:29:15.183359+0000","last_undegraded":"2024-03-29T08:29:15.183359+0000","last_fullsized":"2024-03-29T08:29:15.183359+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:21.448128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.177643+0000","last_change":"2024-03-29T08:29:14.502264+0000","last_active":"2024-03-29T08:29:15.177643+0000","last_peered":"2024-03-29T08:29:15.177643+0000","last_clean":"2024-03-29T08:29:15.177643+0000","last_became_active":"2024-03-29T08:29:14.501882+0000","last_became_peered":"2024-03-29T08:29:14.501882+0000","last_unstale":"2024-03-29T08:29:15.177643+0000","last_undegraded":"2024-03-29T08:29:15.177643+0000","last_fullsized":"2024-03-29T08:29:15.177643+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:31:05.279265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171784+0000","last_change":"2024-03-29T08:29:14.079278+0000","last_active":"2024-03-29T08:29:15.171784+0000","last_peered":"2024-03-29T08:29:15.171784+0000","last_clean":"2024-03-29T08:29:15.171784+0000","last_became_active":"2024-03-29T08:29:14.078982+0000","last_became_peered":"2024-03-29T08:29:14.078982+0000","last_unstale":"2024-03-29T08:29:15.171784+0000","last_undegraded":"2024-03-29T08:29:15.171784+0000","last_fullsized":"2024-03-29T08:29:15.171784+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:06.149502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.177594+0000","last_change":"2024-03-29T08:29:14.500759+0000","last_active":"2024-03-29T08:29:15.177594+0000","last_peered":"2024-03-29T08:29:15.177594+0000","last_clean":"2024-03-29T08:29:15.177594+0000","last_became_active":"2024-03-29T08:29:14.500463+0000","last_became_peered":"2024-03-29T08:29:14.500463+0000","last_unstale":"2024-03-29T08:29:15.177594+0000","last_undegraded":"2024-03-29T08:29:15.177594+0000","last_fullsized":"2024-03-29T08:29:15.177594+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:13:49.079869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"120'8","reported_seq":19,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.179090+0000","last_change":"2024-03-29T08:29:14.090542+0000","last_active":"2024-03-29T08:29:15.179090+0000","last_peered":"2024-03-29T08:29:15.179090+0000","last_clean":"2024-03-29T08:29:15.179090+0000","last_became_active":"2024-03-29T08:29:14.090235+0000","last_became_peered":"2024-03-29T08:29:14.090235+0000","last_unstale":"2024-03-29T08:29:15.179090+0000","last_undegraded":"2024-03-29T08:29:15.179090+0000","last_fullsized":"2024-03-29T08:29:15.179090+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:50:45.558553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.179283+0000","last_change":"2024-03-29T08:29:14.497747+0000","last_active":"2024-03-29T08:29:15.179283+0000","last_peered":"2024-03-29T08:29:15.179283+0000","last_clean":"2024-03-29T08:29:15.179283+0000","last_became_active":"2024-03-29T08:29:14.497456+0000","last_became_peered":"2024-03-29T08:29:14.497456+0000","last_unstale":"2024-03-29T08:29:15.179283+0000","last_undegraded":"2024-03-29T08:29:15.179283+0000","last_fullsized":"2024-03-29T08:29:15.179283+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:22:49.180147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"120'1","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.507179+0000","last_change":"2024-03-29T08:29:14.094084+0000","last_active":"2024-03-29T08:29:14.507179+0000","last_peered":"2024-03-29T08:29:14.507179+0000","last_clean":"2024-03-29T08:29:14.507179+0000","last_became_active":"2024-03-29T08:29:14.093447+0000","last_became_peered":"2024-03-29T08:29:14.093447+0000","last_unstale":"2024-03-29T08:29:14.507179+0000","last_undegraded":"2024-03-29T08:29:14.507179+0000","last_fullsized":"2024-03-29T08:29:14.507179+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T20:11:26.207009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.179333+0000","last_change":"2024-03-29T08:29:14.507678+0000","last_active":"2024-03-29T08:29:15.179333+0000","last_peered":"2024-03-29T08:29:15.179333+0000","last_clean":"2024-03-29T08:29:15.179333+0000","last_became_active":"2024-03-29T08:29:14.507132+0000","last_became_peered":"2024-03-29T08:29:14.507132+0000","last_unstale":"2024-03-29T08:29:15.179333+0000","last_undegraded":"2024-03-29T08:29:15.179333+0000","last_fullsized":"2024-03-29T08:29:15.179333+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:15:29.513377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"120'2","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.183451+0000","last_change":"2024-03-29T08:29:14.095051+0000","last_active":"2024-03-29T08:29:15.183451+0000","last_peered":"2024-03-29T08:29:15.183451+0000","last_clean":"2024-03-29T08:29:15.183451+0000","last_became_active":"2024-03-29T08:29:14.092852+0000","last_became_peered":"2024-03-29T08:29:14.092852+0000","last_unstale":"2024-03-29T08:29:15.183451+0000","last_undegraded":"2024-03-29T08:29:15.183451+0000","last_fullsized":"2024-03-29T08:29:15.183451+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:29:06.149502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.175021+0000","last_change":"2024-03-29T08:29:14.497468+0000","last_active":"2024-03-29T08:29:15.175021+0000","last_peered":"2024-03-29T08:29:15.175021+0000","last_clean":"2024-03-29T08:29:15.175021+0000","last_became_active":"2024-03-29T08:29:14.497061+0000","last_became_peered":"2024-03-29T08:29:14.497061+0000","last_unstale":"2024-03-29T08:29:15.175021+0000","last_undegraded":"2024-03-29T08:29:15.175021+0000","last_fullsized":"2024-03-29T08:29:15.175021+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:09:20.324353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"120'1","reported_seq":12,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171417+0000","last_change":"2024-03-29T08:29:14.090528+0000","last_active":"2024-03-29T08:29:15.171417+0000","last_peered":"2024-03-29T08:29:15.171417+0000","last_clean":"2024-03-29T08:29:15.171417+0000","last_became_active":"2024-03-29T08:29:14.090055+0000","last_became_peered":"2024-03-29T08:29:14.090055+0000","last_unstale":"2024-03-29T08:29:15.171417+0000","last_undegraded":"2024-03-29T08:29:15.171417+0000","last_fullsized":"2024-03-29T08:29:15.171417+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:57:21.448128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171401+0000","last_change":"2024-03-29T08:29:14.502568+0000","last_active":"2024-03-29T08:29:15.171401+0000","last_peered":"2024-03-29T08:29:15.171401+0000","last_clean":"2024-03-29T08:29:15.171401+0000","last_became_active":"2024-03-29T08:29:14.500161+0000","last_became_peered":"2024-03-29T08:29:14.500161+0000","last_unstale":"2024-03-29T08:29:15.171401+0000","last_undegraded":"2024-03-29T08:29:15.171401+0000","last_fullsized":"2024-03-29T08:29:15.171401+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:25:45.763218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171393+0000","last_change":"2024-03-29T08:29:14.078956+0000","last_active":"2024-03-29T08:29:15.171393+0000","last_peered":"2024-03-29T08:29:15.171393+0000","last_clean":"2024-03-29T08:29:15.171393+0000","last_became_active":"2024-03-29T08:29:14.078689+0000","last_became_peered":"2024-03-29T08:29:14.078689+0000","last_unstale":"2024-03-29T08:29:15.171393+0000","last_undegraded":"2024-03-29T08:29:15.171393+0000","last_fullsized":"2024-03-29T08:29:15.171393+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:42:53.153883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171751+0000","last_change":"2024-03-29T08:29:14.486493+0000","last_active":"2024-03-29T08:29:15.171751+0000","last_peered":"2024-03-29T08:29:15.171751+0000","last_clean":"2024-03-29T08:29:15.171751+0000","last_became_active":"2024-03-29T08:29:14.486242+0000","last_became_peered":"2024-03-29T08:29:14.486242+0000","last_unstale":"2024-03-29T08:29:15.171751+0000","last_undegraded":"2024-03-29T08:29:15.171751+0000","last_fullsized":"2024-03-29T08:29:15.171751+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:30:38.933793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"120'2","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.183382+0000","last_change":"2024-03-29T08:29:14.092768+0000","last_active":"2024-03-29T08:29:15.183382+0000","last_peered":"2024-03-29T08:29:15.183382+0000","last_clean":"2024-03-29T08:29:15.183382+0000","last_became_active":"2024-03-29T08:29:14.092290+0000","last_became_peered":"2024-03-29T08:29:14.092290+0000","last_unstale":"2024-03-29T08:29:15.183382+0000","last_undegraded":"2024-03-29T08:29:15.183382+0000","last_fullsized":"2024-03-29T08:29:15.183382+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:47:45.752363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.179225+0000","last_change":"2024-03-29T08:29:14.502876+0000","last_active":"2024-03-29T08:29:15.179225+0000","last_peered":"2024-03-29T08:29:15.179225+0000","last_clean":"2024-03-29T08:29:15.179225+0000","last_became_active":"2024-03-29T08:29:14.502600+0000","last_became_peered":"2024-03-29T08:29:14.502600+0000","last_unstale":"2024-03-29T08:29:15.179225+0000","last_undegraded":"2024-03-29T08:29:15.179225+0000","last_fullsized":"2024-03-29T08:29:15.179225+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:49:19.406820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.179208+0000","last_change":"2024-03-29T08:29:14.090942+0000","last_active":"2024-03-29T08:29:15.179208+0000","last_peered":"2024-03-29T08:29:15.179208+0000","last_clean":"2024-03-29T08:29:15.179208+0000","last_became_active":"2024-03-29T08:29:14.090695+0000","last_became_peered":"2024-03-29T08:29:14.090695+0000","last_unstale":"2024-03-29T08:29:15.179208+0000","last_undegraded":"2024-03-29T08:29:15.179208+0000","last_fullsized":"2024-03-29T08:29:15.179208+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:09:18.077251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171327+0000","last_change":"2024-03-29T08:29:14.508582+0000","last_active":"2024-03-29T08:29:15.171327+0000","last_peered":"2024-03-29T08:29:15.171327+0000","last_clean":"2024-03-29T08:29:15.171327+0000","last_became_active":"2024-03-29T08:29:14.508201+0000","last_became_peered":"2024-03-29T08:29:14.508201+0000","last_unstale":"2024-03-29T08:29:15.171327+0000","last_undegraded":"2024-03-29T08:29:15.171327+0000","last_fullsized":"2024-03-29T08:29:15.171327+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:59:12.691184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.177614+0000","last_change":"2024-03-29T08:29:14.096474+0000","last_active":"2024-03-29T08:29:15.177614+0000","last_peered":"2024-03-29T08:29:15.177614+0000","last_clean":"2024-03-29T08:29:15.177614+0000","last_became_active":"2024-03-29T08:29:14.094758+0000","last_became_peered":"2024-03-29T08:29:14.094758+0000","last_unstale":"2024-03-29T08:29:15.177614+0000","last_undegraded":"2024-03-29T08:29:15.177614+0000","last_fullsized":"2024-03-29T08:29:15.177614+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:32:23.713785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.183291+0000","last_change":"2024-03-29T08:29:14.503676+0000","last_active":"2024-03-29T08:29:15.183291+0000","last_peered":"2024-03-29T08:29:15.183291+0000","last_clean":"2024-03-29T08:29:15.183291+0000","last_became_active":"2024-03-29T08:29:14.503314+0000","last_became_peered":"2024-03-29T08:29:14.503314+0000","last_unstale":"2024-03-29T08:29:15.183291+0000","last_undegraded":"2024-03-29T08:29:15.183291+0000","last_fullsized":"2024-03-29T08:29:15.183291+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:08:59.334138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.177574+0000","last_change":"2024-03-29T08:29:14.089768+0000","last_active":"2024-03-29T08:29:15.177574+0000","last_peered":"2024-03-29T08:29:15.177574+0000","last_clean":"2024-03-29T08:29:15.177574+0000","last_became_active":"2024-03-29T08:29:14.089486+0000","last_became_peered":"2024-03-29T08:29:14.089486+0000","last_unstale":"2024-03-29T08:29:15.177574+0000","last_undegraded":"2024-03-29T08:29:15.177574+0000","last_fullsized":"2024-03-29T08:29:15.177574+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:55:59.174853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171426+0000","last_change":"2024-03-29T08:29:14.505681+0000","last_active":"2024-03-29T08:29:15.171426+0000","last_peered":"2024-03-29T08:29:15.171426+0000","last_clean":"2024-03-29T08:29:15.171426+0000","last_became_active":"2024-03-29T08:29:14.505394+0000","last_became_peered":"2024-03-29T08:29:14.505394+0000","last_unstale":"2024-03-29T08:29:15.171426+0000","last_undegraded":"2024-03-29T08:29:15.171426+0000","last_fullsized":"2024-03-29T08:29:15.171426+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:32:12.510083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"120'2","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.177743+0000","last_change":"2024-03-29T08:29:14.088394+0000","last_active":"2024-03-29T08:29:15.177743+0000","last_peered":"2024-03-29T08:29:15.177743+0000","last_clean":"2024-03-29T08:29:15.177743+0000","last_became_active":"2024-03-29T08:29:14.087525+0000","last_became_peered":"2024-03-29T08:29:14.087525+0000","last_unstale":"2024-03-29T08:29:15.177743+0000","last_undegraded":"2024-03-29T08:29:15.177743+0000","last_fullsized":"2024-03-29T08:29:15.177743+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:54:27.262284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.495205+0000","last_change":"2024-03-29T08:29:14.489834+0000","last_active":"2024-03-29T08:29:14.495205+0000","last_peered":"2024-03-29T08:29:14.495205+0000","last_clean":"2024-03-29T08:29:14.495205+0000","last_became_active":"2024-03-29T08:29:14.489576+0000","last_became_peered":"2024-03-29T08:29:14.489576+0000","last_unstale":"2024-03-29T08:29:14.495205+0000","last_undegraded":"2024-03-29T08:29:14.495205+0000","last_fullsized":"2024-03-29T08:29:14.495205+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:14:06.265379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"121'1","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-03-29T08:29:17.198918+0000","last_change":"2024-03-29T08:29:14.088202+0000","last_active":"2024-03-29T08:29:17.198918+0000","last_peered":"2024-03-29T08:29:17.198918+0000","last_clean":"2024-03-29T08:29:17.198918+0000","last_became_active":"2024-03-29T08:29:14.087860+0000","last_became_peered":"2024-03-29T08:29:14.087860+0000","last_unstale":"2024-03-29T08:29:17.198918+0000","last_undegraded":"2024-03-29T08:29:17.198918+0000","last_fullsized":"2024-03-29T08:29:17.198918+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T19:30:51.004244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.179242+0000","last_change":"2024-03-29T08:29:14.501270+0000","last_active":"2024-03-29T08:29:15.179242+0000","last_peered":"2024-03-29T08:29:15.179242+0000","last_clean":"2024-03-29T08:29:15.179242+0000","last_became_active":"2024-03-29T08:29:14.498111+0000","last_became_peered":"2024-03-29T08:29:14.498111+0000","last_unstale":"2024-03-29T08:29:15.179242+0000","last_undegraded":"2024-03-29T08:29:15.179242+0000","last_fullsized":"2024-03-29T08:29:15.179242+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:26:17.474099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.177715+0000","last_change":"2024-03-29T08:29:14.090681+0000","last_active":"2024-03-29T08:29:15.177715+0000","last_peered":"2024-03-29T08:29:15.177715+0000","last_clean":"2024-03-29T08:29:15.177715+0000","last_became_active":"2024-03-29T08:29:14.090416+0000","last_became_peered":"2024-03-29T08:29:14.090416+0000","last_unstale":"2024-03-29T08:29:15.177715+0000","last_undegraded":"2024-03-29T08:29:15.177715+0000","last_fullsized":"2024-03-29T08:29:15.177715+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:08:56.584444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171409+0000","last_change":"2024-03-29T08:29:14.505745+0000","last_active":"2024-03-29T08:29:15.171409+0000","last_peered":"2024-03-29T08:29:15.171409+0000","last_clean":"2024-03-29T08:29:15.171409+0000","last_became_active":"2024-03-29T08:29:14.504499+0000","last_became_peered":"2024-03-29T08:29:14.504499+0000","last_unstale":"2024-03-29T08:29:15.171409+0000","last_undegraded":"2024-03-29T08:29:15.171409+0000","last_fullsized":"2024-03-29T08:29:15.171409+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:23:28.769780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.177685+0000","last_change":"2024-03-29T08:29:14.096941+0000","last_active":"2024-03-29T08:29:15.177685+0000","last_peered":"2024-03-29T08:29:15.177685+0000","last_clean":"2024-03-29T08:29:15.177685+0000","last_became_active":"2024-03-29T08:29:14.096439+0000","last_became_peered":"2024-03-29T08:29:14.096439+0000","last_unstale":"2024-03-29T08:29:15.177685+0000","last_undegraded":"2024-03-29T08:29:15.177685+0000","last_fullsized":"2024-03-29T08:29:15.177685+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T18:49:13.231792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171769+0000","last_change":"2024-03-29T08:29:14.496299+0000","last_active":"2024-03-29T08:29:15.171769+0000","last_peered":"2024-03-29T08:29:15.171769+0000","last_clean":"2024-03-29T08:29:15.171769+0000","last_became_active":"2024-03-29T08:29:14.496042+0000","last_became_peered":"2024-03-29T08:29:14.496042+0000","last_unstale":"2024-03-29T08:29:15.171769+0000","last_undegraded":"2024-03-29T08:29:15.171769+0000","last_fullsized":"2024-03-29T08:29:15.171769+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T17:23:28.769780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.175013+0000","last_change":"2024-03-29T08:29:14.081880+0000","last_active":"2024-03-29T08:29:15.175013+0000","last_peered":"2024-03-29T08:29:15.175013+0000","last_clean":"2024-03-29T08:29:15.175013+0000","last_became_active":"2024-03-29T08:29:14.079732+0000","last_became_peered":"2024-03-29T08:29:14.079732+0000","last_unstale":"2024-03-29T08:29:15.175013+0000","last_undegraded":"2024-03-29T08:29:15.175013+0000","last_fullsized":"2024-03-29T08:29:15.175013+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:56:01.494143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171385+0000","last_change":"2024-03-29T08:29:14.501776+0000","last_active":"2024-03-29T08:29:15.171385+0000","last_peered":"2024-03-29T08:29:15.171385+0000","last_clean":"2024-03-29T08:29:15.171385+0000","last_became_active":"2024-03-29T08:29:14.501508+0000","last_became_peered":"2024-03-29T08:29:14.501508+0000","last_unstale":"2024-03-29T08:29:15.171385+0000","last_undegraded":"2024-03-29T08:29:15.171385+0000","last_fullsized":"2024-03-29T08:29:15.171385+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T10:08:01.476642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171368+0000","last_change":"2024-03-29T08:29:14.091456+0000","last_active":"2024-03-29T08:29:15.171368+0000","last_peered":"2024-03-29T08:29:15.171368+0000","last_clean":"2024-03-29T08:29:15.171368+0000","last_became_active":"2024-03-29T08:29:14.091121+0000","last_became_peered":"2024-03-29T08:29:14.091121+0000","last_unstale":"2024-03-29T08:29:15.171368+0000","last_undegraded":"2024-03-29T08:29:15.171368+0000","last_fullsized":"2024-03-29T08:29:15.171368+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T13:47:30.979257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171341+0000","last_change":"2024-03-29T08:29:14.503755+0000","last_active":"2024-03-29T08:29:15.171341+0000","last_peered":"2024-03-29T08:29:15.171341+0000","last_clean":"2024-03-29T08:29:15.171341+0000","last_became_active":"2024-03-29T08:29:14.503521+0000","last_became_peered":"2024-03-29T08:29:14.503521+0000","last_unstale":"2024-03-29T08:29:15.171341+0000","last_undegraded":"2024-03-29T08:29:15.171341+0000","last_fullsized":"2024-03-29T08:29:15.171341+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T12:33:36.260388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.179147+0000","last_change":"2024-03-29T08:29:14.089662+0000","last_active":"2024-03-29T08:29:15.179147+0000","last_peered":"2024-03-29T08:29:15.179147+0000","last_clean":"2024-03-29T08:29:15.179147+0000","last_became_active":"2024-03-29T08:29:14.089393+0000","last_became_peered":"2024-03-29T08:29:14.089393+0000","last_unstale":"2024-03-29T08:29:15.179147+0000","last_undegraded":"2024-03-29T08:29:15.179147+0000","last_fullsized":"2024-03-29T08:29:15.179147+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T16:27:50.778160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.505327+0000","last_change":"2024-03-29T08:29:14.498733+0000","last_active":"2024-03-29T08:29:14.505327+0000","last_peered":"2024-03-29T08:29:14.505327+0000","last_clean":"2024-03-29T08:29:14.505327+0000","last_became_active":"2024-03-29T08:29:14.498474+0000","last_became_peered":"2024-03-29T08:29:14.498474+0000","last_unstale":"2024-03-29T08:29:14.505327+0000","last_undegraded":"2024-03-29T08:29:14.505327+0000","last_fullsized":"2024-03-29T08:29:14.505327+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T09:44:25.936884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.483137+0000","last_change":"2024-03-29T08:29:14.092230+0000","last_active":"2024-03-29T08:29:14.483137+0000","last_peered":"2024-03-29T08:29:14.483137+0000","last_clean":"2024-03-29T08:29:14.483137+0000","last_became_active":"2024-03-29T08:29:14.091979+0000","last_became_peered":"2024-03-29T08:29:14.091979+0000","last_unstale":"2024-03-29T08:29:14.483137+0000","last_undegraded":"2024-03-29T08:29:14.483137+0000","last_fullsized":"2024-03-29T08:29:14.483137+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:41:42.442894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.177547+0000","last_change":"2024-03-29T08:29:14.502476+0000","last_active":"2024-03-29T08:29:15.177547+0000","last_peered":"2024-03-29T08:29:15.177547+0000","last_clean":"2024-03-29T08:29:15.177547+0000","last_became_active":"2024-03-29T08:29:14.502191+0000","last_became_peered":"2024-03-29T08:29:14.502191+0000","last_unstale":"2024-03-29T08:29:15.177547+0000","last_undegraded":"2024-03-29T08:29:15.177547+0000","last_fullsized":"2024-03-29T08:29:15.177547+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:14.057150+0000","last_clean_scrub_stamp":"2024-03-29T08:29:14.057150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T15:28:06.826332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:15.171707+0000","last_change":"2024-03-29T08:29:14.077404+0000","last_active":"2024-03-29T08:29:15.171707+0000","last_peered":"2024-03-29T08:29:15.171707+0000","last_clean":"2024-03-29T08:29:15.171707+0000","last_became_active":"2024-03-29T08:29:14.077141+0000","last_became_peered":"2024-03-29T08:29:14.077141+0000","last_unstale":"2024-03-29T08:29:15.171707+0000","last_undegraded":"2024-03-29T08:29:15.171707+0000","last_fullsized":"2024-03-29T08:29:15.171707+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:29:13.051434+0000","last_clean_scrub_stamp":"2024-03-29T08:29:13.051434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:51.446071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":251,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-03-29T08:29:14.481590+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:29:14.481590+0000","last_peered":"2024-03-29T08:29:14.481590+0000","last_clean":"2024-03-29T08:29:14.481590+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:29:14.481590+0000","last_undegraded":"2024-03-29T08:29:14.481590+0000","last_fullsized":"2024-03-29T08:29:14.481590+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509579,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1515520,"data_stored":652309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.73199999999999998,"15min":0.69299999999999995},"min":{"1min":0.48199999999999998,"5min":0.41899999999999998,"15min":0.34999999999999998},"max":{"1min":0.88,"5min":0.97999999999999998,"15min":1.1639999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.746,"5min":0.75600000000000001,"15min":0.71899999999999997},"min":{"1min":0.59599999999999997,"5min":0.56699999999999995,"15min":0.45800000000000002},"max":{"1min":0.95099999999999996,"5min":0.998,"15min":1.2410000000000001},"last":0.82899999999999996}]},{"osd":1,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70199999999999996,"15min":0.67600000000000005},"min":{"1min":0.51500000000000001,"5min":0.45700000000000002,"15min":0.39600000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":1.202},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70799999999999996,"15min":0.67600000000000005},"min":{"1min":0.56100000000000005,"5min":0.48799999999999999,"15min":0.44},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.1339999999999999},"last":0.73399999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.69799999999999995,"15min":0.67000000000000004},"min":{"1min":0.502,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.90300000000000002,"5min":0.93600000000000005,"15min":1.085},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.72899999999999998,"15min":0.70299999999999996},"min":{"1min":0.55500000000000005,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.006,"5min":1.006,"15min":1.288},"last":0.92600000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72299999999999998,"15min":0.70999999999999996},"min":{"1min":0.57599999999999996,"5min":0.51100000000000001,"15min":0.46000000000000002},"max":{"1min":0.97199999999999998,"5min":0.98099999999999998,"15min":1.018},"last":0.875},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.73599999999999999},"min":{"1min":0.60899999999999999,"5min":0.52300000000000002,"15min":0.48499999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.2609999999999999},"last":0.81100000000000005}]},{"osd":4,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58399999999999996,"15min":0.60999999999999999},"min":{"1min":0.48099999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85299999999999998,"5min":0.87,"15min":0.95299999999999996},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60599999999999998,"15min":0.66300000000000003},"min":{"1min":0.45100000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":1.123},"last":0.78200000000000003}]},{"osd":5,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60999999999999999,"15min":0.66200000000000003},"min":{"1min":0.48199999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.877,"5min":0.89600000000000002,"15min":1.056},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.628,"15min":0.66900000000000004},"min":{"1min":0.49199999999999999,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.84399999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.89800000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69199999999999995,"15min":0.71399999999999997},"min":{"1min":0.55900000000000005,"5min":0.46999999999999997,"15min":0.46600000000000003},"max":{"1min":0.89400000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.65800000000000003,"15min":0.70599999999999996},"min":{"1min":0.52800000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.81799999999999995,"5min":0.93799999999999994,"15min":0.98299999999999998},"last":0.91400000000000003}]}]},{"osd":6,"up_from":15,"seq":64424509579,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1495040,"data_stored":650621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72399999999999998,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.38900000000000001},"max":{"1min":0.873,"5min":0.93100000000000005,"15min":1.0189999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.751,"15min":0.71899999999999997},"min":{"1min":0.57199999999999995,"5min":0.51700000000000002,"15min":0.44500000000000001},"max":{"1min":0.97699999999999998,"5min":0.99299999999999999,"15min":1.079},"last":0.83899999999999997}]},{"osd":1,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68200000000000005,"15min":0.64700000000000002},"min":{"1min":0.45400000000000001,"5min":0.45300000000000001,"15min":0.377},"max":{"1min":0.85199999999999998,"5min":0.97999999999999998,"15min":0.99299999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.69599999999999995,"15min":0.66200000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42899999999999999},"max":{"1min":0.78200000000000003,"5min":0.95599999999999996,"15min":1.079},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76900000000000002,"15min":0.72299999999999998},"min":{"1min":0.52100000000000002,"5min":0.51800000000000002,"15min":0.44800000000000001},"max":{"1min":0.91500000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.746,"15min":0.72399999999999998},"min":{"1min":0.46999999999999997,"5min":0.46400000000000002,"15min":0.435},"max":{"1min":0.93899999999999995,"5min":0.95099999999999996,"15min":0.95899999999999996},"last":0.68600000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.71399999999999997,"15min":0.70399999999999996},"min":{"1min":0.443,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82099999999999995,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.71399999999999997,"15min":0.70199999999999996},"min":{"1min":0.55700000000000005,"5min":0.51000000000000001,"15min":0.47799999999999998},"max":{"1min":0.89800000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.66500000000000004}]},{"osd":4,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.623,"15min":0.64600000000000002},"min":{"1min":0.378,"5min":0.378,"15min":0.36499999999999999},"max":{"1min":0.93100000000000005,"5min":1.0840000000000001,"15min":1.556},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.63900000000000001,"15min":0.68200000000000005},"min":{"1min":0.46400000000000002,"5min":0.38200000000000001,"15min":0.371},"max":{"1min":0.872,"5min":0.996,"15min":2.0539999999999998},"last":0.72799999999999998}]},{"osd":5,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.627,"15min":0.66100000000000003},"min":{"1min":0.45000000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83599999999999997,"5min":0.99399999999999999,"15min":1.0660000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.68100000000000005,"15min":0.70199999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.37},"max":{"1min":0.88600000000000001,"5min":1.042,"15min":1.1419999999999999},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri Mar 29 08:28:17 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69599999999999995,"15min":0.72499999999999998},"min":{"1min":0.499,"5min":0.46800000000000003,"15min":0.38500000000000001},"max":{"1min":0.89600000000000002,"5min":1.03,"15min":1.03},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.68899999999999995,"15min":0.72999999999999998},"min":{"1min":0.51600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.91500000000000004,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.746}]}]},{"osd":4,"up_from":15,"seq":64424509578,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1511424,"data_stored":652275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.34499999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90000000000000002,"5min":1.026,"15min":2.1709999999999998},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68400000000000005,"15min":0.71299999999999997},"min":{"1min":0.40799999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84799999999999998,"5min":0.997,"15min":2.2869999999999999},"last":0.68100000000000005}]},{"osd":1,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71499999999999997,"15min":0.74299999999999999},"min":{"1min":0.54800000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.871,"5min":1.0049999999999999,"15min":2.524},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.442,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":0.89100000000000001,"5min":1.3149999999999999,"15min":2.2679999999999998},"last":0.59499999999999997}]},{"osd":2,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69399999999999995,"15min":0.71499999999999997},"min":{"1min":0.45000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93799999999999994,"5min":1.0720000000000001,"15min":2.363},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.745},"min":{"1min":0.49099999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.91300000000000003,"5min":0.97799999999999998,"15min":2.1880000000000002},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.751,"15min":0.75},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.46300000000000002},"max":{"1min":0.93999999999999995,"5min":1.3759999999999999,"15min":2.3029999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.752,"15min":0.78600000000000003},"min":{"1min":0.54600000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98099999999999998,"5min":1.016,"15min":2.2549999999999999},"last":0.57999999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61499999999999999,"15min":0.65400000000000003},"min":{"1min":0.48199999999999998,"5min":0.38300000000000001,"15min":0.36499999999999999},"max":{"1min":0.85799999999999998,"5min":0.998,"15min":1.0189999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.63600000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":1.0149999999999999},"last":0.73199999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.626,"15min":0.65100000000000002},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.39000000000000001},"max":{"1min":0.82299999999999995,"5min":1.091,"15min":1.091},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69399999999999995,"15min":0.71599999999999997},"min":{"1min":0.54300000000000004,"5min":0.499,"15min":0.499},"max":{"1min":0.81499999999999995,"5min":0.91500000000000004,"15min":1.1220000000000001},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69199999999999995,"15min":0.70799999999999996},"min":{"1min":0.498,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":1.1000000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71899999999999997,"15min":0.74299999999999999},"min":{"1min":0.51200000000000001,"5min":0.434,"15min":0.42899999999999999},"max":{"1min":0.85999999999999999,"5min":1.054,"15min":1.4430000000000001},"last":0.69799999999999995}]}]},{"osd":3,"up_from":15,"seq":64424509581,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1495040,"data_stored":650621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.44900000000000001,"15min":0.45400000000000001},"min":{"1min":0.28100000000000003,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":0.71799999999999997,"5min":0.748,"15min":1.123},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.40500000000000003,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.69599999999999995,"5min":0.88100000000000001,"15min":1.0409999999999999},"last":0.40200000000000002}]},{"osd":1,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.44,"15min":0.44800000000000001},"min":{"1min":0.32800000000000001,"5min":0.29599999999999999,"15min":0.26500000000000001},"max":{"1min":0.623,"5min":0.76800000000000002,"15min":1.1599999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.28599999999999998},"max":{"1min":0.82099999999999995,"5min":0.94499999999999995,"15min":1.4790000000000001},"last":0.42999999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.54400000000000004},"min":{"1min":0.35499999999999998,"5min":0.32900000000000001,"15min":0.309},"max":{"1min":0.76200000000000001,"5min":0.80300000000000005,"15min":1.2529999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59799999999999998,"15min":0.624},"min":{"1min":0.439,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.78300000000000003,"5min":0.86599999999999999,"15min":1.8580000000000001},"last":0.47299999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.77200000000000002,"15min":0.77500000000000002},"min":{"1min":0.55500000000000005,"5min":0.53300000000000003,"15min":0.42199999999999999},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.6419999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.77900000000000003,"15min":0.76900000000000002},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.45400000000000001},"max":{"1min":1.083,"5min":1.3340000000000001,"15min":1.391},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82199999999999995,"15min":0.80300000000000005},"min":{"1min":0.59599999999999997,"5min":0.57099999999999995,"15min":0.52100000000000002},"max":{"1min":1.081,"5min":1.1850000000000001,"15min":1.671},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.82399999999999995,"15min":0.83499999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0249999999999999,"5min":1.2050000000000001,"15min":1.7110000000000001},"last":0.69099999999999995}]},{"osd":6,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.55400000000000005,"5min":0.51700000000000002,"15min":0.48799999999999999},"max":{"1min":1.232,"5min":1.2609999999999999,"15min":1.669},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.78100000000000003,"15min":0.79600000000000004},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.0029999999999999,"5min":1.119,"15min":2.0150000000000001},"last":0.64300000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:28:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.80500000000000005,"15min":0.82199999999999995},"min":{"1min":0.58899999999999997,"5min":0.50600000000000001,"15min":0.48799999999999999},"max":{"1min":1.0069999999999999,"5min":1.3169999999999999,"15min":1.887},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.80100000000000005,"15min":0.81899999999999995},"min":{"1min":0.58999999999999997,"5min":0.49399999999999999,"15min":0.47999999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":2.044},"last":0.67300000000000004}]}]},{"osd":2,"up_from":15,"seq":64424509580,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1507328,"data_stored":651259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45600000000000002,"15min":0.47499999999999998},"min":{"1min":0.30599999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.64000000000000001,"5min":0.754,"15min":0.94299999999999995},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51900000000000002,"15min":0.52200000000000002},"min":{"1min":0.42899999999999999,"5min":0.34699999999999998,"15min":0.32300000000000001},"max":{"1min":0.749,"5min":0.749,"15min":2.0880000000000001},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:29:15 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.47899999999999998,"15min":0.49399999999999999},"min":{"1min":0.34599999999999997,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":1.399},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.48799999999999999,"15min":0.502},"min":{"1min":0.32600000000000001,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.67100000000000004,"5min":0.83299999999999996,"15min":3.0459999999999998},"last":0.39000000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:29:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.38700000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.67800000000000005,"5min":0.79900000000000004,"15min":2.1280000000000001},"last":0.439},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.59499999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.35999999999999999},"max":{"1min":0.73199999999999998,"5min":0.88800000000000001,"15min":2.9950000000000001},"last":0.53000000000000003}]},{"osd":4,"last update":"Fri Mar 29 08:29:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.79300000000000004,"15min":0.78000000000000003},"min":{"1min":0.61799999999999999,"5min":0.50700000000000001,"15min":0.49099999999999999},"max":{"1min":0.96199999999999997,"5min":1.109,"15min":1.4239999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.78500000000000003,"15min":0.76900000000000002},"min":{"1min":0.60799999999999998,"5min":0.51700000000000002,"15min":0.435},"max":{"1min":1.0920000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.88700000000000001}]},{"osd":5,"last update":"Fri Mar 29 08:29:15 2024","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.84999999999999998,"15min":0.84099999999999997},"min":{"1min":0.70299999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.139,"5min":1.143,"15min":1.4890000000000001},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.81000000000000005,"15min":0.82699999999999996},"min":{"1min":0.61799999999999999,"5min":0.52300000000000002,"15min":0.48799999999999999},"max":{"1min":1.1379999999999999,"5min":1.1759999999999999,"15min":1.526},"last":0.85099999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:29:15 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.79000000000000004,"15min":0.78300000000000003},"min":{"1min":0.58799999999999997,"5min":0.54600000000000004,"15min":0.48599999999999999},"max":{"1min":1.167,"5min":1.167,"15min":2.0470000000000002},"last":0.871},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.82499999999999996,"15min":0.80100000000000005},"min":{"1min":0.65600000000000003,"5min":0.54800000000000004,"15min":0.52500000000000002},"max":{"1min":1.117,"5min":1.2609999999999999,"15min":1.524},"last":0.70899999999999996}]},{"osd":7,"last update":"Fri Mar 29 08:29:15 2024","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.81200000000000006,"15min":0.80200000000000005},"min":{"1min":0.55600000000000005,"5min":0.54900000000000004,"15min":0.505},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.319},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.80500000000000005,"15min":0.79800000000000004},"min":{"1min":0.627,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.127,"5min":1.214,"15min":2},"last":0.92000000000000004}]}]},{"osd":1,"up_from":15,"seq":64424509580,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":2040,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2088960,"data_stored":1240989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45800000000000002,"15min":0.53200000000000003},"min":{"1min":0.28699999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.69699999999999995,"5min":0.75900000000000001,"15min":4.2539999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48299999999999998,"15min":0.56399999999999995},"min":{"1min":0.29399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":4.1909999999999998},"last":0.48799999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.31900000000000001},"max":{"1min":0.63600000000000001,"5min":0.71099999999999997,"15min":0.82899999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.34599999999999997,"15min":0.33600000000000002},"max":{"1min":0.67000000000000004,"5min":0.77700000000000002,"15min":0.88200000000000001},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.35499999999999998},"max":{"1min":0.75700000000000001,"5min":0.88300000000000001,"15min":1.52},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.51900000000000002,"15min":0.52400000000000002},"min":{"1min":0.35999999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68999999999999995,"5min":0.94499999999999995,"15min":1.331},"last":0.51200000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.75600000000000001},"min":{"1min":0.60199999999999998,"5min":0.51800000000000002,"15min":0.46200000000000002},"max":{"1min":1.157,"5min":1.157,"15min":1.157},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.77600000000000002},"min":{"1min":0.56599999999999995,"5min":0.502,"15min":0.47999999999999998},"max":{"1min":1.212,"5min":1.212,"15min":1.23},"last":0.94599999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79600000000000004,"15min":0.77900000000000003},"min":{"1min":0.59699999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.79200000000000004,"15min":0.76800000000000002},"min":{"1min":0.55200000000000005,"5min":0.55100000000000005,"15min":0.46500000000000002},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.2509999999999999},"last":0.81499999999999995}]},{"osd":6,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.79300000000000004,"15min":0.77300000000000002},"min":{"1min":0.58299999999999996,"5min":0.54900000000000004,"15min":0.5},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.81499999999999995,"15min":0.82399999999999995},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51300000000000001},"max":{"1min":1.0209999999999999,"5min":1.22,"15min":1.22},"last":1.0129999999999999}]},{"osd":7,"last update":"Fri Mar 29 08:28:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.80300000000000005,"15min":0.79800000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.52100000000000002},"max":{"1min":1.1499999999999999,"5min":1.2410000000000001,"15min":1.3089999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.81200000000000006,"5min":0.80700000000000005,"15min":0.79000000000000004},"min":{"1min":0.622,"5min":0.503,"15min":0.503},"max":{"1min":1.1759999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.96299999999999997}]}]},{"osd":0,"up_from":15,"seq":64424509579,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1503232,"data_stored":651273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.44,"15min":0.504},"min":{"1min":0.27800000000000002,"5min":0.27000000000000002,"15min":0.22800000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":3.9359999999999999},"last":0.37},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.49199999999999999,"15min":0.63700000000000001},"min":{"1min":0.33700000000000002,"5min":0.28499999999999998,"15min":0.253},"max":{"1min":0.66900000000000004,"5min":0.77000000000000002,"15min":8.0419999999999998},"last":0.504}]},{"osd":2,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.503,"15min":0.52300000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64600000000000002,"5min":0.78000000000000003,"15min":1.258},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53400000000000003,"15min":0.61499999999999999},"min":{"1min":0.38700000000000001,"5min":0.32800000000000001,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":0.81399999999999995,"15min":4.907},"last":0.54600000000000004}]},{"osd":3,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49399999999999999,"15min":0.63900000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.28499999999999998},"max":{"1min":0.66600000000000004,"5min":0.83399999999999996,"15min":7.9569999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57599999999999996,"15min":0.60599999999999998},"min":{"1min":0.39600000000000002,"5min":0.36099999999999999,"15min":0.34799999999999998},"max":{"1min":0.72599999999999998,"5min":0.87,"15min":1.397},"last":0.55800000000000005}]},{"osd":4,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.78900000000000003,"15min":0.86899999999999999},"min":{"1min":0.54200000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.282,"5min":1.282,"15min":4.4720000000000004},"last":0.622},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78700000000000003,"15min":0.85099999999999998},"min":{"1min":0.499,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97999999999999998,"5min":1.153,"15min":4.3920000000000003},"last":0.69899999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78700000000000003,"15min":0.84399999999999997},"min":{"1min":0.52800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.24,"5min":1.24,"15min":4.3200000000000003},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78600000000000003,"15min":0.81699999999999995},"min":{"1min":0.58399999999999996,"5min":0.56000000000000005,"15min":0.54600000000000004},"max":{"1min":1.0680000000000001,"5min":1.107,"15min":1.6759999999999999},"last":0.74299999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79200000000000004,"15min":0.81999999999999995},"min":{"1min":0.57299999999999995,"5min":0.55800000000000005,"15min":0.54400000000000004},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.5920000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80900000000000005,"15min":0.879},"min":{"1min":0.63,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":4.4100000000000001},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81299999999999994,"15min":0.88900000000000001},"min":{"1min":0.57199999999999995,"5min":0.54100000000000004,"15min":0.496},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":4.5209999999999999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.79400000000000004,"15min":0.873},"min":{"1min":0.59099999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":4.4420000000000002},"last":0.76700000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542284,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":2097152,"data_stored":1241081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70899999999999996,"15min":0.66400000000000003},"min":{"1min":0.47499999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.98499999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72999999999999998,"15min":0.68400000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.39800000000000002},"max":{"1min":0.84799999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.76100000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.70699999999999996,"15min":0.71199999999999997},"min":{"1min":0.48499999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.008,"5min":1.008,"15min":2.5129999999999999},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.74299999999999999,"15min":0.70299999999999996},"min":{"1min":0.56799999999999995,"5min":0.51300000000000001,"15min":0.40500000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.70199999999999996}]},{"osd":2,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71799999999999997,"15min":0.68600000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.92800000000000005,"5min":0.997,"15min":0.997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.747,"15min":0.71799999999999997},"min":{"1min":0.60899999999999999,"5min":0.47499999999999998,"15min":0.39200000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.71099999999999997}]},{"osd":3,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.755,"15min":0.72599999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.47099999999999997},"max":{"1min":1.02,"5min":1.038,"15min":1.038},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.752,"15min":0.74099999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.435},"max":{"1min":0.995,"5min":1.0209999999999999,"15min":1.954},"last":0.628}]},{"osd":4,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.624,"15min":0.64000000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.91200000000000003,"15min":0.93500000000000005},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.623,"15min":0.64300000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.373},"max":{"1min":0.83299999999999996,"5min":0.94699999999999995,"15min":1.0820000000000001},"last":0.66900000000000004}]},{"osd":6,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.621,"15min":0.65000000000000002},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85999999999999999,"5min":0.93300000000000005,"15min":1.071},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.67400000000000004,"15min":0.68500000000000005},"min":{"1min":0.53200000000000003,"5min":0.45300000000000001,"15min":0.432},"max":{"1min":0.92000000000000004,"5min":0.94299999999999995,"15min":1.0609999999999999},"last":0.68799999999999994}]},{"osd":7,"last update":"Fri Mar 29 08:28:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.90100000000000002,"5min":0.92200000000000004,"15min":1.1200000000000001},"last":0.747},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.70099999999999996,"15min":0.72899999999999998},"min":{"1min":0.46800000000000003,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.94999999999999996,"5min":0.96499999999999997,"15min":1.04},"last":0.79100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:29:23.507 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-29T08:29:23.508 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-29T08:29:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.685+0000 7fe45ea00640 1 Processor -- start 2024-03-29T08:29:23.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.685+0000 7fe45ea00640 1 -- start start 2024-03-29T08:29:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.685+0000 7fe45ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe458158300 0x7fe45806d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.685+0000 7fe45ea00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe4580af530 0x7fe4580af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.685+0000 7fe45ea00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe45806c390 con 0x7fe4580702a0 2024-03-29T08:29:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.689+0000 7fe45ea00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe45806d600 con 0x7fe458158300 2024-03-29T08:29:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.689+0000 7fe45ea00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580afff0 con 0x7fe4580af530 2024-03-29T08:29:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.689+0000 7fe457e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe458158300 0x7fe45806d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.689+0000 7fe457e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe458158300 0x7fe45806d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39124/0 (socket says 172.21.15.46:39124) 2024-03-29T08:29:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.689+0000 7fe457e00640 1 -- 172.21.15.46:0/643135661 learned_addr learned my addr 172.21.15.46:0/643135661 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:23.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.689+0000 7fe44f400640 1 --2- 172.21.15.46:0/643135661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe4580af530 0x7fe4580af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.689+0000 7fe457e00640 1 -- 172.21.15.46:0/643135661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe4580af530 msgr2=0x7fe4580af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.689+0000 7fe457e00640 1 --2- 172.21.15.46:0/643135661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe4580af530 0x7fe4580af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.689+0000 7fe457e00640 1 -- 172.21.15.46:0/643135661 >> v1:172.21.15.46:6789/0 conn(0x7fe4580702a0 legacy=0x7fe45815e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:23.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.689+0000 7fe457e00640 1 -- 172.21.15.46:0/643135661 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45806c5e0 con 0x7fe458158300 2024-03-29T08:29:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.689+0000 7fe457e00640 1 --2- 172.21.15.46:0/643135661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe458158300 0x7fe45806d0c0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fe448009e50 tx=0x7fe44801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3723517d0326cca server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.689+0000 7fe457400640 1 -- 172.21.15.46:0/643135661 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4480a57a0 con 0x7fe458158300 2024-03-29T08:29:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.689+0000 7fe457400640 1 -- 172.21.15.46:0/643135661 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4480a5900 con 0x7fe458158300 2024-03-29T08:29:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.689+0000 7fe457400640 1 -- 172.21.15.46:0/643135661 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4480a5c90 con 0x7fe458158300 2024-03-29T08:29:23.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ea00640 1 -- 172.21.15.46:0/643135661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe458158300 msgr2=0x7fe45806d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:23.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ea00640 1 --2- 172.21.15.46:0/643135661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe458158300 0x7fe45806d0c0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fe448009e50 tx=0x7fe44801c8e0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ea00640 1 -- 172.21.15.46:0/643135661 shutdown_connections 2024-03-29T08:29:23.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ea00640 1 --2- 172.21.15.46:0/643135661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe4580af530 0x7fe4580af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ea00640 1 --2- 172.21.15.46:0/643135661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe458158300 0x7fe45806d0c0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ea00640 1 -- 172.21.15.46:0/643135661 >> 172.21.15.46:0/643135661 conn(0x7fe458066080 msgr2=0x7fe458069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ea00640 1 -- 172.21.15.46:0/643135661 shutdown_connections 2024-03-29T08:29:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ea00640 1 -- 172.21.15.46:0/643135661 wait complete. 2024-03-29T08:29:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ea00640 1 Processor -- start 2024-03-29T08:29:23.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ea00640 1 -- start start 2024-03-29T08:29:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4580702a0 0x7fe458159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4580af530 0x7fe4580abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ea00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe458158300 0x7fe458144b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe458146100 con 0x7fe4580af530 2024-03-29T08:29:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ea00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580ac4b0 con 0x7fe4580702a0 2024-03-29T08:29:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ea00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe458145a40 con 0x7fe458158300 2024-03-29T08:29:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe44f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4580af530 0x7fe4580abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe44f400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4580af530 0x7fe4580abb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39130/0 (socket says 172.21.15.46:39130) 2024-03-29T08:29:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe44f400640 1 -- 172.21.15.46:0/3316944766 learned_addr learned my addr 172.21.15.46:0/3316944766 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe457e00640 1 --2- 172.21.15.46:0/3316944766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4580702a0 0x7fe458159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ca00640 1 --2- 172.21.15.46:0/3316944766 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe458158300 0x7fe458144b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:23.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe44f400640 1 -- 172.21.15.46:0/3316944766 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe458158300 msgr2=0x7fe458144b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:23.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe44f400640 1 --2- 172.21.15.46:0/3316944766 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe458158300 0x7fe458144b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe44f400640 1 -- 172.21.15.46:0/3316944766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4580702a0 msgr2=0x7fe458159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:23.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe44f400640 1 --2- 172.21.15.46:0/3316944766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4580702a0 0x7fe458159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe44f400640 1 -- 172.21.15.46:0/3316944766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe448009b30 con 0x7fe4580af530 2024-03-29T08:29:23.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe45ca00640 1 --2- 172.21.15.46:0/3316944766 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe458158300 0x7fe458144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:23.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.693+0000 7fe44f400640 1 --2- 172.21.15.46:0/3316944766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4580af530 0x7fe4580abb00 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fe44000e930 tx=0x7fe44000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:23.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.697+0000 7fe454c00640 1 -- 172.21.15.46:0/3316944766 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe440009690 con 0x7fe4580af530 2024-03-29T08:29:23.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.697+0000 7fe454c00640 1 -- 172.21.15.46:0/3316944766 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44000ca00 con 0x7fe4580af530 2024-03-29T08:29:23.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.697+0000 7fe454c00640 1 -- 172.21.15.46:0/3316944766 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44000cd70 con 0x7fe4580af530 2024-03-29T08:29:23.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.697+0000 7fe45ea00640 1 -- 172.21.15.46:0/3316944766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe458070c40 con 0x7fe4580af530 2024-03-29T08:29:23.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.697+0000 7fe45ea00640 1 -- 172.21.15.46:0/3316944766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45815a090 con 0x7fe4580af530 2024-03-29T08:29:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.697+0000 7fe45ea00640 1 -- 172.21.15.46:0/3316944766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe458071a90 con 0x7fe4580af530 2024-03-29T08:29:23.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.697+0000 7fe454c00640 1 -- 172.21.15.46:0/3316944766 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe440013470 con 0x7fe4580af530 2024-03-29T08:29:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.697+0000 7fe454c00640 1 --2- 172.21.15.46:0/3316944766 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe434066690 0x7fe434068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.697+0000 7fe454c00640 1 -- 172.21.15.46:0/3316944766 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4400843a0 con 0x7fe4580af530 2024-03-29T08:29:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.697+0000 7fe457e00640 1 --2- 172.21.15.46:0/3316944766 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe434066690 0x7fe434068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:23.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.701+0000 7fe457e00640 1 --2- 172.21.15.46:0/3316944766 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe434066690 0x7fe434068b50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe4480ae000 tx=0x7fe4480a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:23.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.701+0000 7fe454c00640 1 -- 172.21.15.46:0/3316944766 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe440051870 con 0x7fe4580af530 2024-03-29T08:29:23.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.917+0000 7fe45ea00640 1 -- 172.21.15.46:0/3316944766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe45804e120 con 0x7fe4580af530 2024-03-29T08:29:23.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.921+0000 7fe454c00640 1 -- 172.21.15.46:0/3316944766 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe440054e90 con 0x7fe4580af530 2024-03-29T08:29:23.926 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2024-03-29T08:29:23.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.925+0000 7fe45ea00640 1 -- 172.21.15.46:0/3316944766 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe434066690 msgr2=0x7fe434068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:23.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.925+0000 7fe45ea00640 1 --2- 172.21.15.46:0/3316944766 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe434066690 0x7fe434068b50 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe4480ae000 tx=0x7fe4480a7070 comp rx=0 tx=0).stop 2024-03-29T08:29:23.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.925+0000 7fe45ea00640 1 -- 172.21.15.46:0/3316944766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4580af530 msgr2=0x7fe4580abb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:23.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.925+0000 7fe45ea00640 1 --2- 172.21.15.46:0/3316944766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4580af530 0x7fe4580abb00 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fe44000e930 tx=0x7fe44000edf0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.925+0000 7fe45ea00640 1 -- 172.21.15.46:0/3316944766 shutdown_connections 2024-03-29T08:29:23.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.925+0000 7fe45ea00640 1 --2- 172.21.15.46:0/3316944766 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe434066690 0x7fe434068b50 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.925+0000 7fe45ea00640 1 --2- 172.21.15.46:0/3316944766 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe458158300 0x7fe458144b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.925+0000 7fe45ea00640 1 --2- 172.21.15.46:0/3316944766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe4580af530 0x7fe4580abb00 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.925+0000 7fe45ea00640 1 --2- 172.21.15.46:0/3316944766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4580702a0 0x7fe458159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:23.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.925+0000 7fe45ea00640 1 -- 172.21.15.46:0/3316944766 >> 172.21.15.46:0/3316944766 conn(0x7fe458066080 msgr2=0x7fe458158930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:23.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.925+0000 7fe45ea00640 1 -- 172.21.15.46:0/3316944766 shutdown_connections 2024-03-29T08:29:23.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:23.925+0000 7fe45ea00640 1 -- 172.21.15.46:0/3316944766 wait complete. 2024-03-29T08:29:23.947 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:29:24.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.085+0000 7f6448200640 1 Processor -- start 2024-03-29T08:29:24.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.085+0000 7f6448200640 1 -- start start 2024-03-29T08:29:24.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.085+0000 7f6448200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6440144210 0x7f64400cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.085+0000 7f6448200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64400ceb20 0x7f644006c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.085+0000 7f6448200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64400cb510 con 0x7f644015ada0 2024-03-29T08:29:24.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.085+0000 7f6448200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6440070580 con 0x7f64400ceb20 2024-03-29T08:29:24.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.085+0000 7f6448200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6440070860 con 0x7f6440144210 2024-03-29T08:29:24.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.085+0000 7f6446200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64400ceb20 0x7f644006c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.085+0000 7f6446200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f64400ceb20 0x7f644006c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39140/0 (socket says 172.21.15.46:39140) 2024-03-29T08:29:24.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.085+0000 7f6446200640 1 -- 172.21.15.46:0/3512961623 learned_addr learned my addr 172.21.15.46:0/3512961623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:24.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.085+0000 7f6445800640 1 --2- 172.21.15.46:0/3512961623 >> v2:172.21.15.73:3300/0 conn(0x7f6440144210 0x7f64400cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.085+0000 7f6446200640 1 -- 172.21.15.46:0/3512961623 >> v2:172.21.15.73:3300/0 conn(0x7f6440144210 msgr2=0x7f64400cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:24.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.085+0000 7f6446200640 1 --2- 172.21.15.46:0/3512961623 >> v2:172.21.15.73:3300/0 conn(0x7f6440144210 0x7f64400cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.085+0000 7f6446200640 1 -- 172.21.15.46:0/3512961623 >> v1:172.21.15.73:6789/0 conn(0x7f644015ada0 legacy=0x7f64400ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:24.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.085+0000 7f6446200640 1 -- 172.21.15.46:0/3512961623 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6440070b40 con 0x7f64400ceb20 2024-03-29T08:29:24.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.089+0000 7f6446200640 1 --2- 172.21.15.46:0/3512961623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64400ceb20 0x7f644006c950 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f6440071360 tx=0x7f643801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c8908e21f425bf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:24.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.089+0000 7f643fe00640 1 -- 172.21.15.46:0/3512961623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64380a57d0 con 0x7f64400ceb20 2024-03-29T08:29:24.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.089+0000 7f643fe00640 1 -- 172.21.15.46:0/3512961623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64380a5930 con 0x7f64400ceb20 2024-03-29T08:29:24.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.089+0000 7f643fe00640 1 -- 172.21.15.46:0/3512961623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64380a5cc0 con 0x7f64400ceb20 2024-03-29T08:29:24.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.089+0000 7f6448200640 1 -- 172.21.15.46:0/3512961623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64400ceb20 msgr2=0x7f644006c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:24.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.089+0000 7f6448200640 1 --2- 172.21.15.46:0/3512961623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64400ceb20 0x7f644006c950 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f6440071360 tx=0x7f643801c9f0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.089+0000 7f6448200640 1 -- 172.21.15.46:0/3512961623 shutdown_connections 2024-03-29T08:29:24.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.089+0000 7f6448200640 1 --2- 172.21.15.46:0/3512961623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64400ceb20 0x7f644006c950 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.089+0000 7f6448200640 1 --2- 172.21.15.46:0/3512961623 >> v2:172.21.15.73:3300/0 conn(0x7f6440144210 0x7f64400cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.089+0000 7f6448200640 1 -- 172.21.15.46:0/3512961623 >> 172.21.15.46:0/3512961623 conn(0x7f6440066110 msgr2=0x7f6440069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:24.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.089+0000 7f6448200640 1 -- 172.21.15.46:0/3512961623 shutdown_connections 2024-03-29T08:29:24.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.089+0000 7f6448200640 1 -- 172.21.15.46:0/3512961623 wait complete. 2024-03-29T08:29:24.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.089+0000 7f6448200640 1 Processor -- start 2024-03-29T08:29:24.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6448200640 1 -- start start 2024-03-29T08:29:24.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6448200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64400ceb20 0x7f644014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6448200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6440144210 0x7f6440141810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6448200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f644015ada0 0x7f644013bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6448200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6440142260 con 0x7f6440144210 2024-03-29T08:29:24.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6448200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6440142540 con 0x7f644015ada0 2024-03-29T08:29:24.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6448200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6440142820 con 0x7f64400ceb20 2024-03-29T08:29:24.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6444e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6440144210 0x7f6440141810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6444e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6440144210 0x7f6440141810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39142/0 (socket says 172.21.15.46:39142) 2024-03-29T08:29:24.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6444e00640 1 -- 172.21.15.46:0/3589780944 learned_addr learned my addr 172.21.15.46:0/3589780944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:24.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6445800640 1 --2- 172.21.15.46:0/3589780944 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64400ceb20 0x7f644014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6446200640 1 --2- 172.21.15.46:0/3589780944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f644015ada0 0x7f644013bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6444e00640 1 -- 172.21.15.46:0/3589780944 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64400ceb20 msgr2=0x7f644014fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:24.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6444e00640 1 --2- 172.21.15.46:0/3589780944 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64400ceb20 0x7f644014fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6444e00640 1 -- 172.21.15.46:0/3589780944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f644015ada0 msgr2=0x7f644013bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:24.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6444e00640 1 --2- 172.21.15.46:0/3589780944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f644015ada0 0x7f644013bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6444e00640 1 -- 172.21.15.46:0/3589780944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6438009b30 con 0x7f6440144210 2024-03-29T08:29:24.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6445800640 1 --2- 172.21.15.46:0/3589780944 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64400ceb20 0x7f644014fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:24.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6444e00640 1 --2- 172.21.15.46:0/3589780944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6440144210 0x7f6440141810 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f642c00c3c0 tx=0x7f642c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:24.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f643d600640 1 -- 172.21.15.46:0/3589780944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c0052d0 con 0x7f6440144210 2024-03-29T08:29:24.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f643d600640 1 -- 172.21.15.46:0/3589780944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c005430 con 0x7f6440144210 2024-03-29T08:29:24.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f643d600640 1 -- 172.21.15.46:0/3589780944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c003840 con 0x7f6440144210 2024-03-29T08:29:24.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6448200640 1 -- 172.21.15.46:0/3589780944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6440142b00 con 0x7f6440144210 2024-03-29T08:29:24.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6448200640 1 -- 172.21.15.46:0/3589780944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6440150240 con 0x7f6440144210 2024-03-29T08:29:24.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.093+0000 7f6448200640 1 -- 172.21.15.46:0/3589780944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64400ce280 con 0x7f6440144210 2024-03-29T08:29:24.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.097+0000 7f643d600640 1 -- 172.21.15.46:0/3589780944 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f642c0039a0 con 0x7f6440144210 2024-03-29T08:29:24.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.097+0000 7f643d600640 1 --2- 172.21.15.46:0/3589780944 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6420066690 0x7f6420068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.097+0000 7f643d600640 1 -- 172.21.15.46:0/3589780944 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f642c083620 con 0x7f6440144210 2024-03-29T08:29:24.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.097+0000 7f6445800640 1 --2- 172.21.15.46:0/3589780944 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6420066690 0x7f6420068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.097+0000 7f6445800640 1 --2- 172.21.15.46:0/3589780944 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6420066690 0x7f6420068b50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6434007a70 tx=0x7f6434008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:24.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.101+0000 7f643d600640 1 -- 172.21.15.46:0/3589780944 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f642c050a30 con 0x7f6440144210 2024-03-29T08:29:24.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.293+0000 7f6448200640 1 -- 172.21.15.46:0/3589780944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6440066c90 con 0x7f6440144210 2024-03-29T08:29:24.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.293+0000 7f643d600640 1 -- 172.21.15.46:0/3589780944 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4733 (secure 0 0 0) 0x7f642c054050 con 0x7f6440144210 2024-03-29T08:29:24.299 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:24.299 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":94,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2506132630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2506132630},{"type":"v1","addr":"172.21.15.73:6836","nonce":2506132630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2108115501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2108115501},{"type":"v1","addr":"172.21.15.73:6837","nonce":2108115501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":9045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3210062895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3210062895},{"type":"v1","addr":"172.21.15.46:6833","nonce":3210062895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:29:14.470858+0000","modified":"2024-03-29T08:29:17.480376+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9066},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9066":{"gid":9066,"name":"a","rank":0,"incarnation":90,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6835/929698978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":929698978},{"type":"v1","addr":"172.21.15.46:6835","nonce":929698978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-29T08:29:24.300 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 94 2024-03-29T08:29:24.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.297+0000 7f6448200640 1 -- 172.21.15.46:0/3589780944 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6420066690 msgr2=0x7f6420068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:24.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.297+0000 7f6448200640 1 --2- 172.21.15.46:0/3589780944 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6420066690 0x7f6420068b50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f6434007a70 tx=0x7f6434008040 comp rx=0 tx=0).stop 2024-03-29T08:29:24.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.297+0000 7f6448200640 1 -- 172.21.15.46:0/3589780944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6440144210 msgr2=0x7f6440141810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:24.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.297+0000 7f6448200640 1 --2- 172.21.15.46:0/3589780944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6440144210 0x7f6440141810 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f642c00c3c0 tx=0x7f642c00c880 comp rx=0 tx=0).stop 2024-03-29T08:29:24.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.297+0000 7f6448200640 1 -- 172.21.15.46:0/3589780944 shutdown_connections 2024-03-29T08:29:24.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.297+0000 7f6448200640 1 --2- 172.21.15.46:0/3589780944 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6420066690 0x7f6420068b50 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.297+0000 7f6448200640 1 --2- 172.21.15.46:0/3589780944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f644015ada0 0x7f644013bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.297+0000 7f6448200640 1 --2- 172.21.15.46:0/3589780944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6440144210 0x7f6440141810 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.297+0000 7f6448200640 1 --2- 172.21.15.46:0/3589780944 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f64400ceb20 0x7f644014fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.297+0000 7f6448200640 1 -- 172.21.15.46:0/3589780944 >> 172.21.15.46:0/3589780944 conn(0x7f6440066110 msgr2=0x7f644013d560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.297+0000 7f6448200640 1 -- 172.21.15.46:0/3589780944 shutdown_connections 2024-03-29T08:29:24.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.297+0000 7f6448200640 1 -- 172.21.15.46:0/3589780944 wait complete. 2024-03-29T08:29:24.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 94, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:29:14.470858+0000', 'modified': '2024-03-29T08:29:17.480376+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9066}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9066': {'gid': 9066, 'name': 'a', 'rank': 0, 'incarnation': 90, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6835/929698978', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 929698978}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 929698978}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:29:24.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T08:29:24.723 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-29T08:29:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.861+0000 7fdbf0a00640 1 Processor -- start 2024-03-29T08:29:24.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.861+0000 7fdbf0a00640 1 -- start start 2024-03-29T08:29:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.865+0000 7fdbf0a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdbec142ab0 0x7fdbec06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.865+0000 7fdbf0a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbec06d9e0 0x7fdbec15d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.865+0000 7fdbf0a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdbec15d940 0x7fdbec0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.865+0000 7fdbf0a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbec06de40 con 0x7fdbec142ab0 2024-03-29T08:29:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.865+0000 7fdbf0a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbec070c70 con 0x7fdbec15d940 2024-03-29T08:29:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.865+0000 7fdbf0a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec070340 con 0x7fdbec06d9e0 2024-03-29T08:29:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.865+0000 7fdbe9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdbec142ab0 0x7fdbec06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.865+0000 7fdbe9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdbec142ab0 0x7fdbec06d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39154/0 (socket says 172.21.15.46:39154) 2024-03-29T08:29:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.865+0000 7fdbe9e00640 1 -- 172.21.15.46:0/1626941554 learned_addr learned my addr 172.21.15.46:0/1626941554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.865+0000 7fdbea800640 1 --2- 172.21.15.46:0/1626941554 >> v2:172.21.15.73:3300/0 conn(0x7fdbec15d940 0x7fdbec0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.865+0000 7fdbe1400640 1 --2- 172.21.15.46:0/1626941554 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbec06d9e0 0x7fdbec15d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.865+0000 7fdbe9e00640 1 -- 172.21.15.46:0/1626941554 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbec06d9e0 msgr2=0x7fdbec15d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.865+0000 7fdbe9e00640 1 --2- 172.21.15.46:0/1626941554 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbec06d9e0 0x7fdbec15d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.865+0000 7fdbe9e00640 1 -- 172.21.15.46:0/1626941554 >> v2:172.21.15.73:3300/0 conn(0x7fdbec15d940 msgr2=0x7fdbec0ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.865+0000 7fdbe9e00640 1 --2- 172.21.15.46:0/1626941554 >> v2:172.21.15.73:3300/0 conn(0x7fdbec15d940 0x7fdbec0ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.865+0000 7fdbe9e00640 1 -- 172.21.15.46:0/1626941554 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec070620 con 0x7fdbec142ab0 2024-03-29T08:29:24.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbe9e00640 1 --2- 172.21.15.46:0/1626941554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbec142ab0 0x7fdbec06d4a0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fdbec070be0 tx=0x7fdbdc01e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dc74de3f8b7539 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:24.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbe9400640 1 -- 172.21.15.46:0/1626941554 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00b2d0 con 0x7fdbec142ab0 2024-03-29T08:29:24.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbe9400640 1 -- 172.21.15.46:0/1626941554 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbdc00b430 con 0x7fdbec142ab0 2024-03-29T08:29:24.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbe9400640 1 -- 172.21.15.46:0/1626941554 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0a7870 con 0x7fdbec142ab0 2024-03-29T08:29:24.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbf0a00640 1 -- 172.21.15.46:0/1626941554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbec142ab0 msgr2=0x7fdbec06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:24.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbf0a00640 1 --2- 172.21.15.46:0/1626941554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbec142ab0 0x7fdbec06d4a0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fdbec070be0 tx=0x7fdbdc01e930 comp rx=0 tx=0).stop 2024-03-29T08:29:24.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbf0a00640 1 -- 172.21.15.46:0/1626941554 shutdown_connections 2024-03-29T08:29:24.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbf0a00640 1 --2- 172.21.15.46:0/1626941554 >> v2:172.21.15.73:3300/0 conn(0x7fdbec15d940 0x7fdbec0ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbf0a00640 1 --2- 172.21.15.46:0/1626941554 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbec06d9e0 0x7fdbec15d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbf0a00640 1 --2- 172.21.15.46:0/1626941554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbec142ab0 0x7fdbec06d4a0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbf0a00640 1 -- 172.21.15.46:0/1626941554 >> 172.21.15.46:0/1626941554 conn(0x7fdbec066080 msgr2=0x7fdbec069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:24.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbf0a00640 1 -- 172.21.15.46:0/1626941554 shutdown_connections 2024-03-29T08:29:24.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbf0a00640 1 -- 172.21.15.46:0/1626941554 wait complete. 2024-03-29T08:29:24.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbf0a00640 1 Processor -- start 2024-03-29T08:29:24.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbf0a00640 1 -- start start 2024-03-29T08:29:24.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbf0a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbec06d9e0 0x7fdbec159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbf0a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbec142ab0 0x7fdbec0cc980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbf0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbec15d940 0x7fdbec13bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbf0a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec13d3b0 con 0x7fdbec15d940 2024-03-29T08:29:24.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbf0a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec15a2e0 con 0x7fdbec06d9e0 2024-03-29T08:29:24.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbf0a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec15a5c0 con 0x7fdbec142ab0 2024-03-29T08:29:24.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.869+0000 7fdbea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbec15d940 0x7fdbec13bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.873+0000 7fdbea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbec15d940 0x7fdbec13bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39156/0 (socket says 172.21.15.46:39156) 2024-03-29T08:29:24.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.873+0000 7fdbea800640 1 -- 172.21.15.46:0/1698658517 learned_addr learned my addr 172.21.15.46:0/1698658517 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:24.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.873+0000 7fdbe9e00640 1 --2- 172.21.15.46:0/1698658517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbec06d9e0 0x7fdbec159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.873+0000 7fdbe1400640 1 --2- 172.21.15.46:0/1698658517 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbec142ab0 0x7fdbec0cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.873+0000 7fdbea800640 1 -- 172.21.15.46:0/1698658517 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbec142ab0 msgr2=0x7fdbec0cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:24.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.873+0000 7fdbea800640 1 --2- 172.21.15.46:0/1698658517 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbec142ab0 0x7fdbec0cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.873+0000 7fdbea800640 1 -- 172.21.15.46:0/1698658517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbec06d9e0 msgr2=0x7fdbec159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:24.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.873+0000 7fdbea800640 1 --2- 172.21.15.46:0/1698658517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbec06d9e0 0x7fdbec159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.873+0000 7fdbea800640 1 -- 172.21.15.46:0/1698658517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc00bd40 con 0x7fdbec15d940 2024-03-29T08:29:24.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.873+0000 7fdbea800640 1 --2- 172.21.15.46:0/1698658517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbec15d940 0x7fdbec13bf60 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400d560 tx=0x7fdbe400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:24.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.873+0000 7fdbe2a00640 1 -- 172.21.15.46:0/1698658517 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe40125f0 con 0x7fdbec15d940 2024-03-29T08:29:24.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.873+0000 7fdbe2a00640 1 -- 172.21.15.46:0/1698658517 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe4012770 con 0x7fdbec15d940 2024-03-29T08:29:24.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.873+0000 7fdbe2a00640 1 -- 172.21.15.46:0/1698658517 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4012b20 con 0x7fdbec15d940 2024-03-29T08:29:24.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.873+0000 7fdbf0a00640 1 -- 172.21.15.46:0/1698658517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbec15a8a0 con 0x7fdbec15d940 2024-03-29T08:29:24.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.873+0000 7fdbf0a00640 1 -- 172.21.15.46:0/1698658517 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbec0a7540 con 0x7fdbec15d940 2024-03-29T08:29:24.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.873+0000 7fdbe2a00640 1 -- 172.21.15.46:0/1698658517 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdbe4012c80 con 0x7fdbec15d940 2024-03-29T08:29:24.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.873+0000 7fdbe2a00640 1 --2- 172.21.15.46:0/1698658517 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdbc8066690 0x7fdbc8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbe2a00640 1 -- 172.21.15.46:0/1698658517 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbe4084480 con 0x7fdbec15d940 2024-03-29T08:29:24.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbe9e00640 1 --2- 172.21.15.46:0/1698658517 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdbc8066690 0x7fdbc8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbf3e0c000 1 Processor -- start 2024-03-29T08:29:24.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbf3e0c000 1 -- start start 2024-03-29T08:29:24.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbe9e00640 1 --2- 172.21.15.46:0/1698658517 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdbc8066690 0x7fdbc8068b50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0b0000 tx=0x7fdbdc0a9070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbf3e0c000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be5272db0 0x563be5273190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbf3e0c000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be52736d0 0x563be5228f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbf3e0c000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be52294c0 0x563be52298a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbf3e0c000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563be50f15a0 con 0x563be52736d0 2024-03-29T08:29:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbf3e0c000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563be507eaf0 con 0x563be52294c0 2024-03-29T08:29:24.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbf3e0c000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x563be507c0c0 con 0x563be5272db0 2024-03-29T08:29:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbe1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be52736d0 0x563be5228f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbe9e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be5272db0 0x563be5273190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbe1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be52736d0 0x563be5228f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39168/0 (socket says 172.21.15.46:39168) 2024-03-29T08:29:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbe1400640 1 -- 172.21.15.46:0/3504615468 learned_addr learned my addr 172.21.15.46:0/3504615468 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbea800640 1 --2- 172.21.15.46:0/3504615468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be52294c0 0x563be52298a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbe1400640 1 -- 172.21.15.46:0/3504615468 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be5272db0 msgr2=0x563be5273190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbe1400640 1 --2- 172.21.15.46:0/3504615468 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be5272db0 0x563be5273190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbe1400640 1 -- 172.21.15.46:0/3504615468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be52294c0 msgr2=0x563be52298a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbe1400640 1 --2- 172.21.15.46:0/3504615468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be52294c0 0x563be52298a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbe1400640 1 -- 172.21.15.46:0/3504615468 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563be4d73890 con 0x563be52736d0 2024-03-29T08:29:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbea800640 1 --2- 172.21.15.46:0/3504615468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be52294c0 0x563be52298a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:24.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbe1400640 1 --2- 172.21.15.46:0/3504615468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be52736d0 0x563be5228f80 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4007940 tx=0x7fdbd4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310f6fb2949f56b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:24.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbdaa00640 1 -- 172.21.15.46:0/3504615468 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4004300 con 0x563be52736d0 2024-03-29T08:29:24.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbdaa00640 1 -- 172.21.15.46:0/3504615468 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd4004460 con 0x563be52736d0 2024-03-29T08:29:24.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbdaa00640 1 -- 172.21.15.46:0/3504615468 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4010ab0 con 0x563be52736d0 2024-03-29T08:29:24.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/3504615468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be52736d0 msgr2=0x563be5228f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:24.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/3504615468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be52736d0 0x563be5228f80 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4007940 tx=0x7fdbd4007e00 comp rx=0 tx=0).stop 2024-03-29T08:29:24.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/3504615468 shutdown_connections 2024-03-29T08:29:24.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/3504615468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be52294c0 0x563be52298a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/3504615468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be52736d0 0x563be5228f80 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/3504615468 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be5272db0 0x563be5273190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/3504615468 >> 172.21.15.46:0/3504615468 conn(0x563be522f830 msgr2=0x563be5227860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:24.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/3504615468 shutdown_connections 2024-03-29T08:29:24.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.877+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/3504615468 wait complete. 2024-03-29T08:29:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 1 -- start start 2024-03-29T08:29:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 1 Processor -- start 2024-03-29T08:29:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be52294c0 0x563be522d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be5272db0 0x563be522d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be52736d0 0x563be522dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563be50779b0 con 0x563be52736d0 2024-03-29T08:29:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563be5016c00 con 0x563be5272db0 2024-03-29T08:29:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x563be5035070 con 0x563be52294c0 2024-03-29T08:29:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 10 client.0 _ll_register_callbacks cb 0x563be52150e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:29:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be52736d0 0x563be522dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbe9e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be52294c0 0x563be522d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbea800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be52736d0 0x563be522dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39184/0 (socket says 172.21.15.46:39184) 2024-03-29T08:29:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbe9e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be52294c0 0x563be522d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:51880/0 (socket says 172.21.15.46:51880) 2024-03-29T08:29:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbea800640 1 -- 172.21.15.46:0/3178307647 learned_addr learned my addr 172.21.15.46:0/3178307647 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbe1400640 1 --2- 172.21.15.46:0/3178307647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be5272db0 0x563be522d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbea800640 1 -- 172.21.15.46:0/3178307647 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be52294c0 msgr2=0x563be522d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbea800640 1 --2- 172.21.15.46:0/3178307647 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be52294c0 0x563be522d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbea800640 1 -- 172.21.15.46:0/3178307647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be5272db0 msgr2=0x563be522d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbea800640 1 --2- 172.21.15.46:0/3178307647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be5272db0 0x563be522d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbea800640 1 -- 172.21.15.46:0/3178307647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd400e040 con 0x563be52736d0 2024-03-29T08:29:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbea800640 1 --2- 172.21.15.46:0/3178307647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be52736d0 0x563be522dcb0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40554e0 tx=0x7fdbe40555d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1fc3a6204fe4ee server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbcf400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:29:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbcf400640 1 -- 172.21.15.46:0/3178307647 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe40840b0 con 0x563be52736d0 2024-03-29T08:29:24.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/3178307647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563be50345f0 con 0x563be52736d0 2024-03-29T08:29:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbcf400640 1 -- 172.21.15.46:0/3178307647 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbe4084210 con 0x563be52736d0 2024-03-29T08:29:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbcf400640 1 -- 172.21.15.46:0/3178307647 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbe405d400 con 0x563be52736d0 2024-03-29T08:29:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 10 client.9237 fetch_fsmap learned FSMap version 94 2024-03-29T08:29:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/3178307647 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x563be503cab0 con 0x563be52736d0 2024-03-29T08:29:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbcf400640 1 -- 172.21.15.46:0/3178307647 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563be50345f0 con 0x563be52736d0 2024-03-29T08:29:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbcf400640 1 -- 172.21.15.46:0/3178307647 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 94) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdbe405b260 con 0x563be52736d0 2024-03-29T08:29:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 10 client.9237 fetch_fsmap finished waiting for FSMap version 94 2024-03-29T08:29:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 10 client.9237 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T08:29:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 20 client.9237 populate_metadata read hostname 'smithi046' 2024-03-29T08:29:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/3178307647 >> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] conn(0x563be522e6c0 0x563be5284240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 4 client.9237 mds_command: new command op to 9066 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:29:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/3178307647 --> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563be5074790 con 0x563be522e6c0 2024-03-29T08:29:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.881+0000 7fdbe9e00640 1 --2- 172.21.15.46:0/3178307647 >> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] conn(0x563be522e6c0 0x563be5284240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.885+0000 7fdbe9e00640 1 --2- 172.21.15.46:0/3178307647 >> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] conn(0x563be522e6c0 0x563be5284240 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3368c670f9d5e7b2 server_cookie=6debac4edf4df4d5 in_seq=0 out_seq=0 2024-03-29T08:29:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.885+0000 7fdbcf400640 10 client.9237 ms_handle_connect on v2:172.21.15.46:6834/929698978 2024-03-29T08:29:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.885+0000 7fdbcf400640 1 -- 172.21.15.46:0/3178307647 <== mds.0 v2:172.21.15.46:6834/929698978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x563be5074790 con 0x563be522e6c0 2024-03-29T08:29:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.885+0000 7fdbcf400640 10 client.9237 handle_command_reply: tid=0 2024-03-29T08:29:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.885+0000 7fdbf3e0c000 1 client.9237 shutdown 2024-03-29T08:29:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.885+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/3178307647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be52736d0 msgr2=0x563be522dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.885+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/3178307647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be52736d0 0x563be522dcb0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fdbe40554e0 tx=0x7fdbe40555d0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.885+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/3178307647 shutdown_connections 2024-03-29T08:29:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.885+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/3178307647 >> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] conn(0x563be522e6c0 0x563be5284240 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.885+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/3178307647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be5272db0 0x563be522d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.885+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/3178307647 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be52736d0 0x563be522dcb0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.885+0000 7fdbcf400640 0 client.9237 ms_handle_reset on v2:172.21.15.46:6834/929698978 2024-03-29T08:29:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.885+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/3178307647 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be52294c0 0x563be522d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.885+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/3178307647 >> 172.21.15.46:0/3178307647 conn(0x563be522f830 msgr2=0x563be5227f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.885+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/3178307647 shutdown_connections 2024-03-29T08:29:24.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.885+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/3178307647 wait complete. 2024-03-29T08:29:24.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.885+0000 7fdbf3e0c000 20 client.9237 trim_cache size 0 max 16384 2024-03-29T08:29:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbf3e0c000 1 Processor -- start 2024-03-29T08:29:24.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbf3e0c000 1 -- start start 2024-03-29T08:29:24.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbf3e0c000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be52294c0 0x563be52cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbf3e0c000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be5272db0 0x563be52ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbf3e0c000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be52736d0 0x563be52cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbf3e0c000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563be4d34940 con 0x563be5272db0 2024-03-29T08:29:24.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbf3e0c000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563be507fd80 con 0x563be52736d0 2024-03-29T08:29:24.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbf3e0c000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x563be5288490 con 0x563be52294c0 2024-03-29T08:29:24.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbe1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be5272db0 0x563be52ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbe1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be5272db0 0x563be52ccfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39194/0 (socket says 172.21.15.46:39194) 2024-03-29T08:29:24.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbe1400640 1 -- 172.21.15.46:0/2962576028 learned_addr learned my addr 172.21.15.46:0/2962576028 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:24.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbea800640 1 --2- 172.21.15.46:0/2962576028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be52736d0 0x563be52cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbe9e00640 1 --2- 172.21.15.46:0/2962576028 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be52294c0 0x563be52cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbe1400640 1 -- 172.21.15.46:0/2962576028 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be52294c0 msgr2=0x563be52cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbe1400640 1 --2- 172.21.15.46:0/2962576028 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be52294c0 0x563be52cca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbe1400640 1 -- 172.21.15.46:0/2962576028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be52736d0 msgr2=0x563be52cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbe1400640 1 --2- 172.21.15.46:0/2962576028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be52736d0 0x563be52cd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbe1400640 1 -- 172.21.15.46:0/2962576028 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563be5022c00 con 0x563be5272db0 2024-03-29T08:29:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbea800640 1 --2- 172.21.15.46:0/2962576028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be52736d0 0x563be52cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbe9e00640 1 --2- 172.21.15.46:0/2962576028 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be52294c0 0x563be52cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:24.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbe1400640 1 --2- 172.21.15.46:0/2962576028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be5272db0 0x563be52ccfa0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4016000 tx=0x7fdbd4009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624244efd07028c3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:24.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbdb400640 1 -- 172.21.15.46:0/2962576028 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4007510 con 0x563be5272db0 2024-03-29T08:29:24.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbdb400640 1 -- 172.21.15.46:0/2962576028 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd4004300 con 0x563be5272db0 2024-03-29T08:29:24.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbdb400640 1 -- 172.21.15.46:0/2962576028 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4010890 con 0x563be5272db0 2024-03-29T08:29:24.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/2962576028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be5272db0 msgr2=0x563be52ccfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/2962576028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be5272db0 0x563be52ccfa0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4016000 tx=0x7fdbd4009000 comp rx=0 tx=0).stop 2024-03-29T08:29:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/2962576028 shutdown_connections 2024-03-29T08:29:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/2962576028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be52736d0 0x563be52cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/2962576028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be5272db0 0x563be52ccfa0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/2962576028 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be52294c0 0x563be52cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/2962576028 >> 172.21.15.46:0/2962576028 conn(0x563be522f830 msgr2=0x563be522c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/2962576028 shutdown_connections 2024-03-29T08:29:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.901+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/2962576028 wait complete. 2024-03-29T08:29:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 1 -- start start 2024-03-29T08:29:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 1 Processor -- start 2024-03-29T08:29:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be52294c0 0x563be52d1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be5272db0 0x563be52d1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be52736d0 0x563be5270400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563be5114400 con 0x563be5272db0 2024-03-29T08:29:24.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563be50751c0 con 0x563be52294c0 2024-03-29T08:29:24.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x563be5078f40 con 0x563be52736d0 2024-03-29T08:29:24.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 10 client.0 _ll_register_callbacks cb 0x563be503c590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:29:24.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbe1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be5272db0 0x563be52d1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbe1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be5272db0 0x563be52d1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39202/0 (socket says 172.21.15.46:39202) 2024-03-29T08:29:24.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbe1400640 1 -- 172.21.15.46:0/1354181390 learned_addr learned my addr 172.21.15.46:0/1354181390 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:24.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbe1400640 1 -- 172.21.15.46:0/1354181390 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be52736d0 msgr2=0x563be5270400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:24.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbe1400640 1 --2- 172.21.15.46:0/1354181390 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be52736d0 0x563be5270400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbe1400640 1 -- 172.21.15.46:0/1354181390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be52294c0 msgr2=0x563be52d1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:24.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbe1400640 1 --2- 172.21.15.46:0/1354181390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be52294c0 0x563be52d1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbe1400640 1 -- 172.21.15.46:0/1354181390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563be4d73890 con 0x563be5272db0 2024-03-29T08:29:24.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbe1400640 1 --2- 172.21.15.46:0/1354181390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be5272db0 0x563be52d1e90 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4010620 tx=0x7fdbd4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be2091439fe88de server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:24.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbcf400640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:29:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbcf400640 1 -- 172.21.15.46:0/1354181390 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd4007510 con 0x563be5272db0 2024-03-29T08:29:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbcf400640 1 -- 172.21.15.46:0/1354181390 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbd4004470 con 0x563be5272db0 2024-03-29T08:29:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/1354181390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563be5047460 con 0x563be5272db0 2024-03-29T08:29:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbcf400640 1 -- 172.21.15.46:0/1354181390 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbd4003620 con 0x563be5272db0 2024-03-29T08:29:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 10 client.9243 fetch_fsmap learned FSMap version 94 2024-03-29T08:29:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/1354181390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x563be4d73cc0 con 0x563be5272db0 2024-03-29T08:29:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbcf400640 1 -- 172.21.15.46:0/1354181390 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563be5047460 con 0x563be5272db0 2024-03-29T08:29:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbcf400640 1 -- 172.21.15.46:0/1354181390 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 94) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdbd4003780 con 0x563be5272db0 2024-03-29T08:29:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 10 client.9243 fetch_fsmap finished waiting for FSMap version 94 2024-03-29T08:29:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 10 client.9243 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T08:29:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 20 client.9243 populate_metadata read hostname 'smithi046' 2024-03-29T08:29:24.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/1354181390 >> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] conn(0x563be5270940 0x563be5270d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 4 client.9243 mds_command: new command op to 9066 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T08:29:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/1354181390 --> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563be50409c0 con 0x563be5270940 2024-03-29T08:29:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbe9e00640 1 --2- 172.21.15.46:0/1354181390 >> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] conn(0x563be5270940 0x563be5270d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbcf400640 10 client.9243 ms_handle_connect on v2:172.21.15.46:6834/929698978 2024-03-29T08:29:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.905+0000 7fdbe9e00640 1 --2- 172.21.15.46:0/1354181390 >> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] conn(0x563be5270940 0x563be5270d20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb7b28a7b76b8ca8 server_cookie=55dccfafd01c778c in_seq=0 out_seq=0 2024-03-29T08:29:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbcf400640 1 -- 172.21.15.46:0/1354181390 <== mds.0 v2:172.21.15.46:6834/929698978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x563be50409c0 con 0x563be5270940 2024-03-29T08:29:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbcf400640 10 client.9243 handle_command_reply: tid=0 2024-03-29T08:29:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf3e0c000 1 client.9243 shutdown 2024-03-29T08:29:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/1354181390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be5272db0 msgr2=0x563be52d1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/1354181390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be5272db0 0x563be52d1e90 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4010620 tx=0x7fdbd4004740 comp rx=0 tx=0).stop 2024-03-29T08:29:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/1354181390 shutdown_connections 2024-03-29T08:29:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/1354181390 >> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] conn(0x563be5270940 0x563be5270d20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/1354181390 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563be52736d0 0x563be5270400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/1354181390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563be5272db0 0x563be52d1e90 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf3e0c000 1 --2- 172.21.15.46:0/1354181390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563be52294c0 0x563be52d1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbcf400640 0 client.9243 ms_handle_reset on v2:172.21.15.46:6834/929698978 2024-03-29T08:29:24.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/1354181390 >> 172.21.15.46:0/1354181390 conn(0x563be522f830 msgr2=0x563be522cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:24.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/1354181390 shutdown_connections 2024-03-29T08:29:24.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf3e0c000 1 -- 172.21.15.46:0/1354181390 wait complete. 2024-03-29T08:29:24.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf3e0c000 20 client.9243 trim_cache size 0 max 16384 2024-03-29T08:29:24.916 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:29:24.916 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "e9b7960f-ec1f-4787-95c3-c806094d37f0", 2024-03-29T08:29:24.916 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-03-29T08:29:24.916 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 9066, 2024-03-29T08:29:24.916 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-29T08:29:24.917 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-29T08:29:24.917 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-29T08:29:24.917 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 10.439864829999999, 2024-03-29T08:29:24.917 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 94, 2024-03-29T08:29:24.917 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 121, 2024-03-29T08:29:24.917 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 121, 2024-03-29T08:29:24.917 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 15.427765587 2024-03-29T08:29:24.917 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:29:24.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf0a00640 1 -- 172.21.15.46:0/1698658517 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdbc8066690 msgr2=0x7fdbc8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:24.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf0a00640 1 --2- 172.21.15.46:0/1698658517 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdbc8066690 0x7fdbc8068b50 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0b0000 tx=0x7fdbdc0a9070 comp rx=0 tx=0).stop 2024-03-29T08:29:24.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf0a00640 1 -- 172.21.15.46:0/1698658517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbec15d940 msgr2=0x7fdbec13bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:24.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf0a00640 1 --2- 172.21.15.46:0/1698658517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbec15d940 0x7fdbec13bf60 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fdbe400d560 tx=0x7fdbe400da20 comp rx=0 tx=0).stop 2024-03-29T08:29:24.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf0a00640 1 -- 172.21.15.46:0/1698658517 shutdown_connections 2024-03-29T08:29:24.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf0a00640 1 --2- 172.21.15.46:0/1698658517 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdbc8066690 0x7fdbc8068b50 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf0a00640 1 --2- 172.21.15.46:0/1698658517 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbec15d940 0x7fdbec13bf60 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf0a00640 1 --2- 172.21.15.46:0/1698658517 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdbec142ab0 0x7fdbec0cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf0a00640 1 --2- 172.21.15.46:0/1698658517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbec06d9e0 0x7fdbec159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:24.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf0a00640 1 -- 172.21.15.46:0/1698658517 >> 172.21.15.46:0/1698658517 conn(0x7fdbec066080 msgr2=0x7fdbec0ada50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:24.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf0a00640 1 -- 172.21.15.46:0/1698658517 shutdown_connections 2024-03-29T08:29:24.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:24.909+0000 7fdbf0a00640 1 -- 172.21.15.46:0/1698658517 wait complete. 2024-03-29T08:29:24.932 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:29:25.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.069+0000 7f1396800640 1 Processor -- start 2024-03-29T08:29:25.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.069+0000 7f1396800640 1 -- start start 2024-03-29T08:29:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.069+0000 7f1396800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f139015dc00 0x7f13900cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.069+0000 7f1396800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139015cfb0 con 0x7f13900d0660 2024-03-29T08:29:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.069+0000 7f1396800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139006be20 con 0x7f139006b8b0 2024-03-29T08:29:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.069+0000 7f1396800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139006c100 con 0x7f139015dc00 2024-03-29T08:29:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.069+0000 7f138f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f13900d0660 0x7f13900d0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:48244/0 (socket says 172.21.15.46:48244) 2024-03-29T08:29:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.069+0000 7f138f400640 1 -- 172.21.15.46:0/2744335438 learned_addr learned my addr 172.21.15.46:0/2744335438 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:25.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.069+0000 7f138fe00640 1 --2- 172.21.15.46:0/2744335438 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f139015dc00 0x7f13900cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.069+0000 7f138ea00640 1 -- 172.21.15.46:0/2744335438 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009809400 0 0) 0x7f139015cfb0 con 0x7f13900d0660 2024-03-29T08:29:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.069+0000 7f138ea00640 1 -- 172.21.15.46:0/2744335438 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1374003530 con 0x7f13900d0660 2024-03-29T08:29:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.069+0000 7f138ea00640 1 -- 172.21.15.46:0/2744335438 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148650603 0 0) 0x7f139006be20 con 0x7f139006b8b0 2024-03-29T08:29:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.069+0000 7f138ea00640 1 -- 172.21.15.46:0/2744335438 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f139015cfb0 con 0x7f139006b8b0 2024-03-29T08:29:25.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.069+0000 7f138fe00640 1 -- 172.21.15.46:0/2744335438 >> v1:172.21.15.73:6789/0 conn(0x7f139006b8b0 legacy=0x7f13900cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.069+0000 7f138fe00640 1 -- 172.21.15.46:0/2744335438 >> v1:172.21.15.46:6789/0 conn(0x7f13900d0660 legacy=0x7f13900d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:25.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.069+0000 7f138fe00640 1 -- 172.21.15.46:0/2744335438 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13900cecf0 con 0x7f139015dc00 2024-03-29T08:29:25.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.073+0000 7f138fe00640 1 --2- 172.21.15.46:0/2744335438 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f139015dc00 0x7f13900cf7d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f13880097f0 tx=0x7f138801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8f1ab4de087dd91 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:25.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.073+0000 7f138ea00640 1 -- 172.21.15.46:0/2744335438 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13880ac410 con 0x7f139015dc00 2024-03-29T08:29:25.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.073+0000 7f138ea00640 1 -- 172.21.15.46:0/2744335438 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13880ac570 con 0x7f139015dc00 2024-03-29T08:29:25.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.073+0000 7f138ea00640 1 -- 172.21.15.46:0/2744335438 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13880ac900 con 0x7f139015dc00 2024-03-29T08:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.073+0000 7f1396800640 1 -- 172.21.15.46:0/2744335438 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f139015dc00 msgr2=0x7f13900cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.073+0000 7f1396800640 1 --2- 172.21.15.46:0/2744335438 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f139015dc00 0x7f13900cf7d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f13880097f0 tx=0x7f138801c480 comp rx=0 tx=0).stop 2024-03-29T08:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.073+0000 7f1396800640 1 -- 172.21.15.46:0/2744335438 shutdown_connections 2024-03-29T08:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.073+0000 7f1396800640 1 --2- 172.21.15.46:0/2744335438 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f139015dc00 0x7f13900cf7d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.073+0000 7f1396800640 1 -- 172.21.15.46:0/2744335438 >> 172.21.15.46:0/2744335438 conn(0x7f1390089fd0 msgr2=0x7f139008a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.073+0000 7f1396800640 1 -- 172.21.15.46:0/2744335438 shutdown_connections 2024-03-29T08:29:25.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.073+0000 7f1396800640 1 -- 172.21.15.46:0/2744335438 wait complete. 2024-03-29T08:29:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f1396800640 1 Processor -- start 2024-03-29T08:29:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f1396800640 1 -- start start 2024-03-29T08:29:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f1396800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f139006b8b0 0x7f13900b3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f1396800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13900d0660 0x7f13900c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f1396800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f139015dc00 0x7f1390142290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f1396800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13901541f0 con 0x7f139015dc00 2024-03-29T08:29:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f1396800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13900c9780 con 0x7f13900d0660 2024-03-29T08:29:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f1396800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13900c89c0 con 0x7f139006b8b0 2024-03-29T08:29:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f138fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f139015dc00 0x7f1390142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f138fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f139015dc00 0x7f1390142290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39204/0 (socket says 172.21.15.46:39204) 2024-03-29T08:29:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f138fe00640 1 -- 172.21.15.46:0/1185411794 learned_addr learned my addr 172.21.15.46:0/1185411794 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f138f400640 1 --2- 172.21.15.46:0/1185411794 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f139006b8b0 0x7f13900b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:25.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f1386a00640 1 --2- 172.21.15.46:0/1185411794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13900d0660 0x7f13900c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f138fe00640 1 -- 172.21.15.46:0/1185411794 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f139006b8b0 msgr2=0x7f13900b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:25.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f138fe00640 1 --2- 172.21.15.46:0/1185411794 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f139006b8b0 0x7f13900b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f138fe00640 1 -- 172.21.15.46:0/1185411794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13900d0660 msgr2=0x7f13900c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f138fe00640 1 --2- 172.21.15.46:0/1185411794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13900d0660 0x7f13900c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f138fe00640 1 -- 172.21.15.46:0/1185411794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1388009420 con 0x7f139015dc00 2024-03-29T08:29:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f138fe00640 1 --2- 172.21.15.46:0/1185411794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f139015dc00 0x7f1390142290 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f138801a0b0 tx=0x7f1388019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f1387e00640 1 -- 172.21.15.46:0/1185411794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388006e20 con 0x7f139015dc00 2024-03-29T08:29:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f1387e00640 1 -- 172.21.15.46:0/1185411794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13880b7420 con 0x7f139015dc00 2024-03-29T08:29:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f1387e00640 1 -- 172.21.15.46:0/1185411794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13880b77b0 con 0x7f139015dc00 2024-03-29T08:29:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f1396800640 1 -- 172.21.15.46:0/1185411794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13900c8ca0 con 0x7f139015dc00 2024-03-29T08:29:25.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.077+0000 7f1396800640 1 -- 172.21.15.46:0/1185411794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13900c9260 con 0x7f139015dc00 2024-03-29T08:29:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.081+0000 7f1396800640 1 -- 172.21.15.46:0/1185411794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13900c9450 con 0x7f139015dc00 2024-03-29T08:29:25.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.081+0000 7f1387e00640 1 -- 172.21.15.46:0/1185411794 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f13880b3040 con 0x7f139015dc00 2024-03-29T08:29:25.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.081+0000 7f1387e00640 1 --2- 172.21.15.46:0/1185411794 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f136c066690 0x7f136c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:25.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.081+0000 7f1387e00640 1 -- 172.21.15.46:0/1185411794 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f138811e2a0 con 0x7f139015dc00 2024-03-29T08:29:25.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.081+0000 7f138f400640 1 --2- 172.21.15.46:0/1185411794 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f136c066690 0x7f136c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:25.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.081+0000 7f138f400640 1 --2- 172.21.15.46:0/1185411794 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f136c066690 0x7f136c068b50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f13800012f0 tx=0x7f1380001ae0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:25.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.085+0000 7f1387e00640 1 -- 172.21.15.46:0/1185411794 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f13880eb590 con 0x7f139015dc00 2024-03-29T08:29:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.281+0000 7f1396800640 1 -- 172.21.15.46:0/1185411794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1390066c90 con 0x7f139015dc00 2024-03-29T08:29:25.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.281+0000 7f1387e00640 1 -- 172.21.15.46:0/1185411794 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) v1 ==== 53+0+73 (secure 0 0 0) 0x7f13880eebb0 con 0x7f139015dc00 2024-03-29T08:29:25.287 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:29:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.285+0000 7f1396800640 1 -- 172.21.15.46:0/1185411794 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f136c066690 msgr2=0x7f136c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.285+0000 7f1396800640 1 --2- 172.21.15.46:0/1185411794 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f136c066690 0x7f136c068b50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f13800012f0 tx=0x7f1380001ae0 comp rx=0 tx=0).stop 2024-03-29T08:29:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.285+0000 7f1396800640 1 -- 172.21.15.46:0/1185411794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f139015dc00 msgr2=0x7f1390142290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.285+0000 7f1396800640 1 --2- 172.21.15.46:0/1185411794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f139015dc00 0x7f1390142290 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f138801a0b0 tx=0x7f1388019220 comp rx=0 tx=0).stop 2024-03-29T08:29:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.285+0000 7f1396800640 1 -- 172.21.15.46:0/1185411794 shutdown_connections 2024-03-29T08:29:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.285+0000 7f1396800640 1 --2- 172.21.15.46:0/1185411794 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f136c066690 0x7f136c068b50 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.285+0000 7f1396800640 1 --2- 172.21.15.46:0/1185411794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13900d0660 0x7f13900c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:25.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.285+0000 7f1396800640 1 --2- 172.21.15.46:0/1185411794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f139015dc00 0x7f1390142290 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.285+0000 7f1396800640 1 --2- 172.21.15.46:0/1185411794 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f139006b8b0 0x7f13900b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.285+0000 7f1396800640 1 -- 172.21.15.46:0/1185411794 >> 172.21.15.46:0/1185411794 conn(0x7f1390089fd0 msgr2=0x7f13900d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.285+0000 7f1396800640 1 -- 172.21.15.46:0/1185411794 shutdown_connections 2024-03-29T08:29:25.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:25.285+0000 7f1396800640 1 -- 172.21.15.46:0/1185411794 wait complete. 2024-03-29T08:29:25.310 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:29:25.310 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:29:25.311 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-29T08:29:25.311 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T08:29:25.311 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:25.311 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:29:25.311 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:29:25.311 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:29:25.311 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:29:25.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:25.311 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-29T08:29:25.321 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:29:25.321 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:29:25.321 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:29:25.321 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:25.321 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:29:25.322 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:25.322 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:29:25.322 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:25.322 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-29T08:29:25.322 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:29:25.322 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:25.322 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-29T08:29:25.322 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:29:25.322 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:25.322 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-29T08:29:25.322 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T08:29:25.322 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41772sec preferred_lft 41772sec 2024-03-29T08:29:25.322 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-29T08:29:25.322 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:25.323 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:29:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-29T08:29:25.323 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:29:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:d9:58:09:4c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:29:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:25.323 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:29:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T08:29:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T08:29:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:25.323 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:29:25.323 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:29:25.324 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T08:29:25.324 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:25.324 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:29:25.324 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T08:29:25.324 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-29T08:29:25.324 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:25.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:25.324 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:29:25.381 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:29:25.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:25.383 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-29T08:29:25.437 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-29T08:29:25.437 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:29:25.437 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T08:29:25.437 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:29:25.437 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T08:29:25.438 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:29:25.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:25.438 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:29:25.438 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:29:25.438 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:29:25.438 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:29:25.438 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:29:25.438 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:29:25.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:25.529 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:29:25.538 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T08:29:25.538 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T08:29:25.539 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:29:25.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:25.539 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-03-29T08:29:25.599 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T08:29:25.599 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:29:25.599 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:25.651 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:29:25.652 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:29:25.652 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:25.703 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:29:25.768 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:29:25.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:29:25.829 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:29:25.831 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:29:25.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:25.831 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:29:25.887 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:29:25.887 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T08:29:25.938 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:29:25.958 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:29:25.958 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:29:26.017 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:29:26.019 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:29:26.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:26.020 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:29:26.033 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-03-29T08:29:26.029+0000 7fc8f8648580 -1 init, newargv = 0x561389bdcd10 newargc=15 2024-03-29T08:29:26.033 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[50887]: starting ceph client 2024-03-29T08:29:26.040 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[50887]: starting fuse 2024-03-29T08:29:26.041 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:29:26.042 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-29T08:29:26.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:26.042 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-03-29T08:29:26.042 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-03-29T08:29:26.042 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:26.042 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-03-29T08:29:26.042 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:26.042 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-03-29T08:29:26.042 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T08:29:26.042 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-03-29T08:29:26.042 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:29:26.042 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:26.043 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-03-29T08:29:26.043 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-03-29T08:29:26.043 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-03-29T08:29:26.043 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-03-29T08:29:26.043 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:26.043 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-03-29T08:29:26.043 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:29:26.043 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:29:26.043 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:29:26.043 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-03-29T08:29:26.043 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-03-29T08:29:26.043 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-03-29T08:29:26.043 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:29:26.043 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:26.044 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-03-29T08:29:26.044 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:26.169 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.50887.asok 2024-03-29T08:29:26.353 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50887.asok 2024-03-29T08:29:26.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:26.353 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50887.asok status 2024-03-29T08:29:26.527 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:29:26.528 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-03-29T08:29:26.528 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:29:26.528 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:29:26.528 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-03-29T08:29:26.528 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-03-29T08:29:26.528 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T08:29:26.528 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "50887", 2024-03-29T08:29:26.528 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-03-29T08:29:26.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:26.528 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-03-29T08:29:26.528 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-03-29T08:29:26.528 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 9258, 2024-03-29T08:29:26.528 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-03-29T08:29:26.529 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-03-29T08:29:26.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-03-29T08:29:26.529 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 9258 2024-03-29T08:29:26.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:26.529 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:29:26.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:29:26.529 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:29:26.529 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2597401938 2024-03-29T08:29:26.529 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:26.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:26.529 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:29:26.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:29:26.529 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:29:26.529 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2597401938 2024-03-29T08:29:26.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:26.530 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.9258 192.168.0.1:0/2597401938", 2024-03-29T08:29:26.530 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/2597401938", 2024-03-29T08:29:26.530 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-03-29T08:29:26.530 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 94, 2024-03-29T08:29:26.530 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 121, 2024-03-29T08:29:26.530 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:29:26.530 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-03-29T08:29:26.530 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-03-29T08:29:26.530 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:29:26.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:26.542 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-03-29T08:29:26.543 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:26.551 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-03-29T08:29:26.552 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:26.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:26.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:26.615 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:26.615 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:26.615 DEBUG:teuthology.orchestra.run.smithi046:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:29:26.615 DEBUG:teuthology.orchestra.run.smithi046:> parent = os.path.join(mount_point, "parent") 2024-03-29T08:29:26.615 DEBUG:teuthology.orchestra.run.smithi046:> os.mkdir(parent) 2024-03-29T08:29:26.615 DEBUG:teuthology.orchestra.run.smithi046:> flushed = os.path.join(parent, "flushed") 2024-03-29T08:29:26.615 DEBUG:teuthology.orchestra.run.smithi046:> os.mkdir(flushed) 2024-03-29T08:29:26.615 DEBUG:teuthology.orchestra.run.smithi046:> for f in ["alpha", "bravo", "charlie"]: 2024-03-29T08:29:26.616 DEBUG:teuthology.orchestra.run.smithi046:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-03-29T08:29:26.616 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:26.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:26.909 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-29T08:29:26.972 INFO:teuthology.orchestra.run.smithi046.stdout:./ 2024-03-29T08:29:26.973 INFO:teuthology.orchestra.run.smithi046.stdout:./parent 2024-03-29T08:29:26.973 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed 2024-03-29T08:29:26.973 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed/charlie 2024-03-29T08:29:26.973 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed/alpha 2024-03-29T08:29:26.973 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed/bravo 2024-03-29T08:29:26.974 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:26.974 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:26.974 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:29:26.974 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:26.974 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-29T08:29:26.974 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:27.083 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:29:27.273 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:27.273 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:27.273 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:29:27.273 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:27.273 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-03-29T08:29:27.273 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:27.384 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627776 2024-03-29T08:29:27.574 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:27.574 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:27.574 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:29:27.574 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:27.574 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-03-29T08:29:27.575 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:27.685 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627777 2024-03-29T08:29:27.874 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:27.874 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:27.874 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:29:27.874 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:27.874 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-03-29T08:29:27.874 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:27.980 INFO:teuthology.orchestra.run.smithi046.stdout:1099511628280 2024-03-29T08:29:28.168 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:28.168 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:28.168 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:29:28.168 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:28.169 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-03-29T08:29:28.169 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:28.276 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627778 2024-03-29T08:29:28.465 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:28.465 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:28.465 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:29:28.465 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:28.465 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-03-29T08:29:28.465 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:28.572 INFO:teuthology.orchestra.run.smithi046.stdout:1099511628279 2024-03-29T08:29:28.761 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:29:28.761 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:29:28.816 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:29:29.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.993+0000 7f41bb200640 1 Processor -- start 2024-03-29T08:29:29.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.993+0000 7f41bb200640 1 -- start start 2024-03-29T08:29:29.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.997+0000 7f41bb200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41bc0b0310 0x7f41bc0ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.997+0000 7f41bb200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f41bc0b06f0 0x7f41bc069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.997+0000 7f41bb200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41bc06a5e0 con 0x7f41bc0cbe70 2024-03-29T08:29:29.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.997+0000 7f41bb200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41bc06aba0 con 0x7f41bc0b06f0 2024-03-29T08:29:29.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.997+0000 7f41bb200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc0afef0 con 0x7f41bc0b0310 2024-03-29T08:29:29.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.997+0000 7f41b9e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41bc0b0310 0x7f41bc0ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.997+0000 7f41b9e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41bc0b0310 0x7f41bc0ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:35964/0 (socket says 172.21.15.46:35964) 2024-03-29T08:29:29.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.997+0000 7f41b9e00640 1 -- 172.21.15.46:0/859137418 learned_addr learned my addr 172.21.15.46:0/859137418 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.997+0000 7f41b3e00640 1 --2- 172.21.15.46:0/859137418 >> v2:172.21.15.73:3300/0 conn(0x7f41bc0b06f0 0x7f41bc069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.997+0000 7f41ba800640 1 --1- 172.21.15.46:0/859137418 >> v1:172.21.15.73:6789/0 conn(0x7f41bc0cbe70 0x7f41bc0cc300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:48908/0 (socket says 172.21.15.46:48908) 2024-03-29T08:29:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.997+0000 7f41b9e00640 1 -- 172.21.15.46:0/859137418 >> v2:172.21.15.73:3300/0 conn(0x7f41bc0b06f0 msgr2=0x7f41bc069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.997+0000 7f41b9e00640 1 --2- 172.21.15.46:0/859137418 >> v2:172.21.15.73:3300/0 conn(0x7f41bc0b06f0 0x7f41bc069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.997+0000 7f41b9e00640 1 -- 172.21.15.46:0/859137418 >> v1:172.21.15.73:6789/0 conn(0x7f41bc0cbe70 legacy=0x7f41bc0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.997+0000 7f41b9400640 1 -- 172.21.15.46:0/859137418 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224594830 0 0) 0x7f41bc06a5e0 con 0x7f41bc0cbe70 2024-03-29T08:29:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.997+0000 7f41b9e00640 1 -- 172.21.15.46:0/859137418 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc06a8c0 con 0x7f41bc0b0310 2024-03-29T08:29:29.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:28.997+0000 7f41b3e00640 1 --2- 172.21.15.46:0/859137418 >> v2:172.21.15.73:3300/0 conn(0x7f41bc0b06f0 0x7f41bc069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:29:29.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.001+0000 7f41b9e00640 1 --2- 172.21.15.46:0/859137418 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41bc0b0310 0x7f41bc0ce590 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009870 tx=0x7f41ac01c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=930af4091c0d87f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:29.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.001+0000 7f41b9400640 1 -- 172.21.15.46:0/859137418 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0a95f0 con 0x7f41bc0b0310 2024-03-29T08:29:29.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.001+0000 7f41b9400640 1 -- 172.21.15.46:0/859137418 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ac0a9750 con 0x7f41bc0b0310 2024-03-29T08:29:29.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.001+0000 7f41b9400640 1 -- 172.21.15.46:0/859137418 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0a9ae0 con 0x7f41bc0b0310 2024-03-29T08:29:29.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.001+0000 7f41bb200640 1 -- 172.21.15.46:0/859137418 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41bc0b0310 msgr2=0x7f41bc0ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.001+0000 7f41bb200640 1 --2- 172.21.15.46:0/859137418 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41bc0b0310 0x7f41bc0ce590 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009870 tx=0x7f41ac01c530 comp rx=0 tx=0).stop 2024-03-29T08:29:29.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.001+0000 7f41bb200640 1 -- 172.21.15.46:0/859137418 shutdown_connections 2024-03-29T08:29:29.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.001+0000 7f41bb200640 1 --2- 172.21.15.46:0/859137418 >> v2:172.21.15.73:3300/0 conn(0x7f41bc0b06f0 0x7f41bc069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.001+0000 7f41bb200640 1 --2- 172.21.15.46:0/859137418 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41bc0b0310 0x7f41bc0ce590 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.001+0000 7f41bb200640 1 -- 172.21.15.46:0/859137418 >> 172.21.15.46:0/859137418 conn(0x7f41bc098a40 msgr2=0x7f41bc0cf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:29.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.001+0000 7f41bb200640 1 -- 172.21.15.46:0/859137418 shutdown_connections 2024-03-29T08:29:29.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.001+0000 7f41bb200640 1 -- 172.21.15.46:0/859137418 wait complete. 2024-03-29T08:29:29.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.001+0000 7f41bb200640 1 Processor -- start 2024-03-29T08:29:29.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41bb200640 1 -- start start 2024-03-29T08:29:29.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41bb200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41bc0b0310 0x7f41bc15f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41bb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41bc0b06f0 0x7f41bc152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41bb200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41bc0cbe70 0x7f41bc152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41bb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc155130 con 0x7f41bc0b06f0 2024-03-29T08:29:29.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41bb200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc15f280 con 0x7f41bc0b0310 2024-03-29T08:29:29.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41bb200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc152b70 con 0x7f41bc0cbe70 2024-03-29T08:29:29.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41bc0b06f0 0x7f41bc152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41b3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41bc0b06f0 0x7f41bc152630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38732/0 (socket says 172.21.15.46:38732) 2024-03-29T08:29:29.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41b3e00640 1 -- 172.21.15.46:0/2204366833 learned_addr learned my addr 172.21.15.46:0/2204366833 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:29.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41ba800640 1 --2- 172.21.15.46:0/2204366833 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41bc0cbe70 0x7f41bc152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41b9e00640 1 --2- 172.21.15.46:0/2204366833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41bc0b0310 0x7f41bc15f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41b3e00640 1 -- 172.21.15.46:0/2204366833 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41bc0cbe70 msgr2=0x7f41bc152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41b3e00640 1 --2- 172.21.15.46:0/2204366833 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41bc0cbe70 0x7f41bc152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41b3e00640 1 -- 172.21.15.46:0/2204366833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41bc0b0310 msgr2=0x7f41bc15f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41b3e00640 1 --2- 172.21.15.46:0/2204366833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41bc0b0310 0x7f41bc15f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41b3e00640 1 -- 172.21.15.46:0/2204366833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a4009420 con 0x7f41bc0b06f0 2024-03-29T08:29:29.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41b3e00640 1 --2- 172.21.15.46:0/2204366833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41bc0b06f0 0x7f41bc152630 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f41a400e9b0 tx=0x7f41a400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:29.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41b2000640 1 -- 172.21.15.46:0/2204366833 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a400ca00 con 0x7f41bc0b06f0 2024-03-29T08:29:29.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41b2000640 1 -- 172.21.15.46:0/2204366833 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a400cb60 con 0x7f41bc0b06f0 2024-03-29T08:29:29.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41b2000640 1 -- 172.21.15.46:0/2204366833 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a4013470 con 0x7f41bc0b06f0 2024-03-29T08:29:29.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41bb200640 1 -- 172.21.15.46:0/2204366833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ac0094d0 con 0x7f41bc0b06f0 2024-03-29T08:29:29.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41bb200640 1 -- 172.21.15.46:0/2204366833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc0b2eb0 con 0x7f41bc0b06f0 2024-03-29T08:29:29.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.005+0000 7f41bb200640 1 -- 172.21.15.46:0/2204366833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41bc15f3c0 con 0x7f41bc0b06f0 2024-03-29T08:29:29.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.009+0000 7f41b2000640 1 -- 172.21.15.46:0/2204366833 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f41a40135d0 con 0x7f41bc0b06f0 2024-03-29T08:29:29.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.009+0000 7f41b2000640 1 --2- 172.21.15.46:0/2204366833 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4198066690 0x7f4198068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.009+0000 7f41b2000640 1 -- 172.21.15.46:0/2204366833 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41a4084500 con 0x7f41bc0b06f0 2024-03-29T08:29:29.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.009+0000 7f41b9e00640 1 --2- 172.21.15.46:0/2204366833 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4198066690 0x7f4198068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.009+0000 7f41b9e00640 1 --2- 172.21.15.46:0/2204366833 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4198066690 0x7f4198068b50 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f41ac0ae000 tx=0x7f41ac0b3040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:29.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.013+0000 7f41b2000640 1 -- 172.21.15.46:0/2204366833 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f41a40517d0 con 0x7f41bc0b06f0 2024-03-29T08:29:29.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.173+0000 7f41bb200640 1 -- 172.21.15.46:0/2204366833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41bc04e040 con 0x7f41bc0b06f0 2024-03-29T08:29:29.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.177+0000 7f41b2000640 1 -- 172.21.15.46:0/2204366833 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16067 (secure 0 0 0) 0x7f41a4054df0 con 0x7f41bc0b06f0 2024-03-29T08:29:29.180 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:29.180 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":121,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:29:15.473193+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:29:12.329121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-29T08:29:13.465939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:29:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.177+0000 7f41bb200640 1 -- 172.21.15.46:0/2204366833 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4198066690 msgr2=0x7f4198068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.177+0000 7f41bb200640 1 --2- 172.21.15.46:0/2204366833 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4198066690 0x7f4198068b50 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f41ac0ae000 tx=0x7f41ac0b3040 comp rx=0 tx=0).stop 2024-03-29T08:29:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.177+0000 7f41bb200640 1 -- 172.21.15.46:0/2204366833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41bc0b06f0 msgr2=0x7f41bc152630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.177+0000 7f41bb200640 1 --2- 172.21.15.46:0/2204366833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41bc0b06f0 0x7f41bc152630 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f41a400e9b0 tx=0x7f41a400ee70 comp rx=0 tx=0).stop 2024-03-29T08:29:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.177+0000 7f41bb200640 1 -- 172.21.15.46:0/2204366833 shutdown_connections 2024-03-29T08:29:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.177+0000 7f41bb200640 1 --2- 172.21.15.46:0/2204366833 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4198066690 0x7f4198068b50 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.177+0000 7f41bb200640 1 --2- 172.21.15.46:0/2204366833 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f41bc0cbe70 0x7f41bc152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.177+0000 7f41bb200640 1 --2- 172.21.15.46:0/2204366833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41bc0b06f0 0x7f41bc152630 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.177+0000 7f41bb200640 1 --2- 172.21.15.46:0/2204366833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41bc0b0310 0x7f41bc15f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.177+0000 7f41bb200640 1 -- 172.21.15.46:0/2204366833 >> 172.21.15.46:0/2204366833 conn(0x7f41bc098a40 msgr2=0x7f41bc06ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.177+0000 7f41bb200640 1 -- 172.21.15.46:0/2204366833 shutdown_connections 2024-03-29T08:29:29.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.177+0000 7f41bb200640 1 -- 172.21.15.46:0/2204366833 wait complete. 2024-03-29T08:29:29.201 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:29:12.329121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:29:13.465939+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:29:29.201 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:29:29.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.341+0000 7fe5a4a00640 1 Processor -- start 2024-03-29T08:29:29.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.345+0000 7fe5a4a00640 1 -- start start 2024-03-29T08:29:29.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.345+0000 7fe5a4a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe5a015cad0 0x7fe5a006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.345+0000 7fe5a4a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe5a006a4a0 0x7fe5a006d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.345+0000 7fe5a4a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5a015d080 con 0x7fe5a006eff0 2024-03-29T08:29:29.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.345+0000 7fe5a4a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a015dd00 con 0x7fe5a015cad0 2024-03-29T08:29:29.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.345+0000 7fe5a4a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a006f480 con 0x7fe5a006a4a0 2024-03-29T08:29:29.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.345+0000 7fe59e800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe5a006a4a0 0x7fe5a006d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.345+0000 7fe59de00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe5a015cad0 0x7fe5a006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.345+0000 7fe597e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe5a006eff0 0x7fe5a0069d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:48910/0 (socket says 172.21.15.46:48910) 2024-03-29T08:29:29.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.345+0000 7fe59e800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe5a006a4a0 0x7fe5a006d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:35982/0 (socket says 172.21.15.46:35982) 2024-03-29T08:29:29.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.345+0000 7fe597e00640 1 -- 172.21.15.46:0/1515384801 learned_addr learned my addr 172.21.15.46:0/1515384801 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:29.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.345+0000 7fe59de00640 1 -- 172.21.15.46:0/1515384801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe5a006a4a0 msgr2=0x7fe5a006d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:29.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.345+0000 7fe59de00640 1 --2- 172.21.15.46:0/1515384801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe5a006a4a0 0x7fe5a006d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.345+0000 7fe59de00640 1 -- 172.21.15.46:0/1515384801 >> v1:172.21.15.73:6789/0 conn(0x7fe5a006eff0 legacy=0x7fe5a0069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.345+0000 7fe59de00640 1 -- 172.21.15.46:0/1515384801 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a015d360 con 0x7fe5a015cad0 2024-03-29T08:29:29.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.349+0000 7fe59de00640 1 --2- 172.21.15.46:0/1515384801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5a015cad0 0x7fe5a006eab0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe5900097c0 tx=0x7fe59001c2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3433538eda990d2e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:29.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.349+0000 7fe59d400640 1 -- 172.21.15.46:0/1515384801 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5900ab600 con 0x7fe5a015cad0 2024-03-29T08:29:29.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.349+0000 7fe59d400640 1 -- 172.21.15.46:0/1515384801 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5900a7050 con 0x7fe5a015cad0 2024-03-29T08:29:29.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.349+0000 7fe59d400640 1 -- 172.21.15.46:0/1515384801 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5900ab980 con 0x7fe5a015cad0 2024-03-29T08:29:29.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.349+0000 7fe5a4a00640 1 -- 172.21.15.46:0/1515384801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5a015cad0 msgr2=0x7fe5a006eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.349+0000 7fe5a4a00640 1 --2- 172.21.15.46:0/1515384801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5a015cad0 0x7fe5a006eab0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe5900097c0 tx=0x7fe59001c2a0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.349+0000 7fe5a4a00640 1 -- 172.21.15.46:0/1515384801 shutdown_connections 2024-03-29T08:29:29.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.349+0000 7fe5a4a00640 1 --2- 172.21.15.46:0/1515384801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe5a006a4a0 0x7fe5a006d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.349+0000 7fe5a4a00640 1 --2- 172.21.15.46:0/1515384801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5a015cad0 0x7fe5a006eab0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.349+0000 7fe5a4a00640 1 -- 172.21.15.46:0/1515384801 >> 172.21.15.46:0/1515384801 conn(0x7fe5a0098a40 msgr2=0x7fe5a00b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:29.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.349+0000 7fe5a4a00640 1 -- 172.21.15.46:0/1515384801 shutdown_connections 2024-03-29T08:29:29.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.349+0000 7fe5a4a00640 1 -- 172.21.15.46:0/1515384801 wait complete. 2024-03-29T08:29:29.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.349+0000 7fe5a4a00640 1 Processor -- start 2024-03-29T08:29:29.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.349+0000 7fe5a4a00640 1 -- start start 2024-03-29T08:29:29.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe5a4a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe5a006a4a0 0x7fe5a0151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe5a4a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5a006eff0 0x7fe5a0151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe5a4a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a015cad0 0x7fe5a00ad640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe5a4a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a0152bb0 con 0x7fe5a015cad0 2024-03-29T08:29:29.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe5a4a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00adb80 con 0x7fe5a006eff0 2024-03-29T08:29:29.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe5a4a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a0151fa0 con 0x7fe5a006a4a0 2024-03-29T08:29:29.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe59e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a015cad0 0x7fe5a00ad640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe59e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a015cad0 0x7fe5a00ad640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38740/0 (socket says 172.21.15.46:38740) 2024-03-29T08:29:29.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe59e800640 1 -- 172.21.15.46:0/3263797316 learned_addr learned my addr 172.21.15.46:0/3263797316 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:29.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe59de00640 1 --2- 172.21.15.46:0/3263797316 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe5a006a4a0 0x7fe5a0151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe597e00640 1 --2- 172.21.15.46:0/3263797316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5a006eff0 0x7fe5a0151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe59e800640 1 -- 172.21.15.46:0/3263797316 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe5a006a4a0 msgr2=0x7fe5a0151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe59e800640 1 --2- 172.21.15.46:0/3263797316 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe5a006a4a0 0x7fe5a0151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe59e800640 1 -- 172.21.15.46:0/3263797316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5a006eff0 msgr2=0x7fe5a0151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe59e800640 1 --2- 172.21.15.46:0/3263797316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5a006eff0 0x7fe5a0151320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe59e800640 1 -- 172.21.15.46:0/3263797316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe590009420 con 0x7fe5a015cad0 2024-03-29T08:29:29.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe59e800640 1 --2- 172.21.15.46:0/3263797316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a015cad0 0x7fe5a00ad640 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fe598009dd0 tx=0x7fe59800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:29.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe596000640 1 -- 172.21.15.46:0/3263797316 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe598014480 con 0x7fe5a015cad0 2024-03-29T08:29:29.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe596000640 1 -- 172.21.15.46:0/3263797316 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59800f040 con 0x7fe5a015cad0 2024-03-29T08:29:29.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe5a4a00640 1 -- 172.21.15.46:0/3263797316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a0152280 con 0x7fe5a015cad0 2024-03-29T08:29:29.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe596000640 1 -- 172.21.15.46:0/3263797316 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe598014840 con 0x7fe5a015cad0 2024-03-29T08:29:29.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe5a4a00640 1 -- 172.21.15.46:0/3263797316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a0152840 con 0x7fe5a015cad0 2024-03-29T08:29:29.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.353+0000 7fe5a4a00640 1 -- 172.21.15.46:0/3263797316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a0152a00 con 0x7fe5a015cad0 2024-03-29T08:29:29.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.357+0000 7fe596000640 1 -- 172.21.15.46:0/3263797316 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fe5980149a0 con 0x7fe5a015cad0 2024-03-29T08:29:29.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.357+0000 7fe596000640 1 --2- 172.21.15.46:0/3263797316 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe57c0666c0 0x7fe57c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.357+0000 7fe596000640 1 -- 172.21.15.46:0/3263797316 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe598084ca0 con 0x7fe5a015cad0 2024-03-29T08:29:29.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.357+0000 7fe59de00640 1 --2- 172.21.15.46:0/3263797316 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe57c0666c0 0x7fe57c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.357+0000 7fe59de00640 1 --2- 172.21.15.46:0/3263797316 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe57c0666c0 0x7fe57c068b80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe5900ad000 tx=0x7fe5900a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:29.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.361+0000 7fe596000640 1 -- 172.21.15.46:0/3263797316 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fe598052020 con 0x7fe5a015cad0 2024-03-29T08:29:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.521+0000 7fe5a4a00640 1 -- 172.21.15.46:0/3263797316 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5a00cd8f0 con 0x7fe5a015cad0 2024-03-29T08:29:29.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.521+0000 7fe596000640 1 -- 172.21.15.46:0/3263797316 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe598055640 con 0x7fe5a015cad0 2024-03-29T08:29:29.527 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:29:29.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.525+0000 7fe5a4a00640 1 -- 172.21.15.46:0/3263797316 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe57c0666c0 msgr2=0x7fe57c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.525+0000 7fe5a4a00640 1 --2- 172.21.15.46:0/3263797316 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe57c0666c0 0x7fe57c068b80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe5900ad000 tx=0x7fe5900a6000 comp rx=0 tx=0).stop 2024-03-29T08:29:29.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.525+0000 7fe5a4a00640 1 -- 172.21.15.46:0/3263797316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a015cad0 msgr2=0x7fe5a00ad640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.525+0000 7fe5a4a00640 1 --2- 172.21.15.46:0/3263797316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a015cad0 0x7fe5a00ad640 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fe598009dd0 tx=0x7fe59800c490 comp rx=0 tx=0).stop 2024-03-29T08:29:29.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.525+0000 7fe5a4a00640 1 -- 172.21.15.46:0/3263797316 shutdown_connections 2024-03-29T08:29:29.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.525+0000 7fe5a4a00640 1 --2- 172.21.15.46:0/3263797316 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fe57c0666c0 0x7fe57c068b80 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.525+0000 7fe5a4a00640 1 --2- 172.21.15.46:0/3263797316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5a006eff0 0x7fe5a0151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.525+0000 7fe5a4a00640 1 --2- 172.21.15.46:0/3263797316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a015cad0 0x7fe5a00ad640 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.525+0000 7fe5a4a00640 1 --2- 172.21.15.46:0/3263797316 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fe5a006a4a0 0x7fe5a0151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.525+0000 7fe5a4a00640 1 -- 172.21.15.46:0/3263797316 >> 172.21.15.46:0/3263797316 conn(0x7fe5a0098a40 msgr2=0x7fe5a01603f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:29.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.525+0000 7fe5a4a00640 1 -- 172.21.15.46:0/3263797316 shutdown_connections 2024-03-29T08:29:29.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.525+0000 7fe5a4a00640 1 -- 172.21.15.46:0/3263797316 wait complete. 2024-03-29T08:29:29.549 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:29:29.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fdedaa00640 1 Processor -- start 2024-03-29T08:29:29.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fdedaa00640 1 -- start start 2024-03-29T08:29:29.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fdedaa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded40cbea0 0x7fded40ae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fdedaa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded415d260 con 0x7fded415df90 2024-03-29T08:29:29.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fdedaa00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded40aefd0 con 0x7fded4142a50 2024-03-29T08:29:29.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fdedaa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded415d4b0 con 0x7fded40cbea0 2024-03-29T08:29:29.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fded415df90 0x7fded40ae280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33752/0 (socket says 172.21.15.46:33752) 2024-03-29T08:29:29.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded3400640 1 -- 172.21.15.46:0/1956008645 learned_addr learned my addr 172.21.15.46:0/1956008645 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:29.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded8a00640 1 --2- 172.21.15.46:0/1956008645 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded40cbea0 0x7fded40ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded2a00640 1 -- 172.21.15.46:0/1956008645 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645283318 0 0) 0x7fded415d260 con 0x7fded415df90 2024-03-29T08:29:29.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded2a00640 1 -- 172.21.15.46:0/1956008645 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdeb8005740 con 0x7fded415df90 2024-03-29T08:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded2a00640 1 -- 172.21.15.46:0/1956008645 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1511108113 0 0) 0x7fded40aefd0 con 0x7fded4142a50 2024-03-29T08:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded2a00640 1 -- 172.21.15.46:0/1956008645 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fded415d260 con 0x7fded4142a50 2024-03-29T08:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded2a00640 1 -- 172.21.15.46:0/1956008645 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 874555114 0 0) 0x7fdeb8005740 con 0x7fded415df90 2024-03-29T08:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded2a00640 1 -- 172.21.15.46:0/1956008645 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded40cbea0 msgr2=0x7fded40ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded2a00640 1 --2- 172.21.15.46:0/1956008645 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded40cbea0 0x7fded40ae990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded2a00640 1 -- 172.21.15.46:0/1956008645 >> v1:172.21.15.73:6789/0 conn(0x7fded4142a50 legacy=0x7fded40cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded2a00640 1 -- 172.21.15.46:0/1956008645 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded415d790 con 0x7fded415df90 2024-03-29T08:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded2a00640 1 -- 172.21.15.46:0/1956008645 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fdec0001680 con 0x7fded415df90 2024-03-29T08:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded2a00640 1 -- 172.21.15.46:0/1956008645 >> v1:172.21.15.46:6789/0 conn(0x7fded415df90 legacy=0x7fded40ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded2a00640 1 --2- 172.21.15.46:0/1956008645 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded40cbea0 0x7fdeb8005d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded2a00640 1 --2- 172.21.15.46:0/1956008645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeb8006290 0x7fdeb80069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded2a00640 1 --2- 172.21.15.46:0/1956008645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeb8006f20 0x7fdeb80073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded2a00640 1 -- 172.21.15.46:0/1956008645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded415d4b0 con 0x7fdeb8006290 2024-03-29T08:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded2a00640 1 -- 172.21.15.46:0/1956008645 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb800ffa0 con 0x7fdeb8006f20 2024-03-29T08:29:29.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded2a00640 1 -- 172.21.15.46:0/1956008645 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeb80100e0 con 0x7fded40cbea0 2024-03-29T08:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded3400640 1 --2- 172.21.15.46:0/1956008645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeb8006290 0x7fdeb80069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded8a00640 1 --2- 172.21.15.46:0/1956008645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeb8006f20 0x7fdeb80073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded3e00640 1 --2- 172.21.15.46:0/1956008645 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded40cbea0 0x7fdeb8005d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded3400640 1 -- 172.21.15.46:0/1956008645 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded40cbea0 msgr2=0x7fdeb8005d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded3400640 1 --2- 172.21.15.46:0/1956008645 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fded40cbea0 0x7fdeb8005d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded3400640 1 -- 172.21.15.46:0/1956008645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeb8006f20 msgr2=0x7fdeb80073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded3400640 1 --2- 172.21.15.46:0/1956008645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeb8006f20 0x7fdeb80073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.689+0000 7fded3400640 1 -- 172.21.15.46:0/1956008645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeb8005740 con 0x7fdeb8006290 2024-03-29T08:29:29.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.693+0000 7fded3400640 1 --2- 172.21.15.46:0/1956008645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeb8006290 0x7fdeb80069e0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fdec000c190 tx=0x7fdec001cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa1685fdd11ffe8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.693+0000 7fded2a00640 1 -- 172.21.15.46:0/1956008645 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec00058e0 con 0x7fdeb8006290 2024-03-29T08:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.693+0000 7fded2a00640 1 -- 172.21.15.46:0/1956008645 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec0005a40 con 0x7fdeb8006290 2024-03-29T08:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.693+0000 7fded2a00640 1 -- 172.21.15.46:0/1956008645 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0005e50 con 0x7fdeb8006290 2024-03-29T08:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.693+0000 7fdedaa00640 1 -- 172.21.15.46:0/1956008645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeb8006290 msgr2=0x7fdeb80069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.693+0000 7fdedaa00640 1 --2- 172.21.15.46:0/1956008645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdeb8006290 0x7fdeb80069e0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fdec000c190 tx=0x7fdec001cbc0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.693+0000 7fded8a00640 1 -- 172.21.15.46:0/1956008645 reap_dead start 2024-03-29T08:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.693+0000 7fdedaa00640 1 -- 172.21.15.46:0/1956008645 shutdown_connections 2024-03-29T08:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.693+0000 7fdedaa00640 1 -- 172.21.15.46:0/1956008645 >> 172.21.15.46:0/1956008645 conn(0x7fded406a390 msgr2=0x7fded40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.693+0000 7fdedaa00640 1 -- 172.21.15.46:0/1956008645 shutdown_connections 2024-03-29T08:29:29.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.693+0000 7fdedaa00640 1 -- 172.21.15.46:0/1956008645 wait complete. 2024-03-29T08:29:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fdedaa00640 1 Processor -- start 2024-03-29T08:29:29.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fdedaa00640 1 -- start start 2024-03-29T08:29:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fdedaa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdeb8006290 0x7fded415ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fdedaa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded413bbc0 0x7fded4161a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fdedaa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fded413c040 0x7fded4161f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fdedaa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded4161610 con 0x7fded413bbc0 2024-03-29T08:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fdedaa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded4160a60 con 0x7fded413c040 2024-03-29T08:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fdedaa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded415f2a0 con 0x7fdeb8006290 2024-03-29T08:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fded3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded413bbc0 0x7fded4161a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fded3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded413bbc0 0x7fded4161a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38758/0 (socket says 172.21.15.46:38758) 2024-03-29T08:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fded3400640 1 -- 172.21.15.46:0/4102305222 learned_addr learned my addr 172.21.15.46:0/4102305222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fded8a00640 1 --2- 172.21.15.46:0/4102305222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fded413c040 0x7fded4161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fded3e00640 1 --2- 172.21.15.46:0/4102305222 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdeb8006290 0x7fded415ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fded3400640 1 -- 172.21.15.46:0/4102305222 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdeb8006290 msgr2=0x7fded415ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fded3400640 1 --2- 172.21.15.46:0/4102305222 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdeb8006290 0x7fded415ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fded3400640 1 -- 172.21.15.46:0/4102305222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fded413c040 msgr2=0x7fded4161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fded3400640 1 --2- 172.21.15.46:0/4102305222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fded413c040 0x7fded4161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fded3400640 1 -- 172.21.15.46:0/4102305222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded4160c70 con 0x7fded413bbc0 2024-03-29T08:29:29.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fded3400640 1 --2- 172.21.15.46:0/4102305222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded413bbc0 0x7fded4161a30 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fdec00ab040 tx=0x7fdeb8007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fdecbe00640 1 -- 172.21.15.46:0/4102305222 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec00049f0 con 0x7fded413bbc0 2024-03-29T08:29:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fdecbe00640 1 -- 172.21.15.46:0/4102305222 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec0004b50 con 0x7fded413bbc0 2024-03-29T08:29:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fdedaa00640 1 -- 172.21.15.46:0/4102305222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded4160f50 con 0x7fded413bbc0 2024-03-29T08:29:29.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fdecbe00640 1 -- 172.21.15.46:0/4102305222 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0001710 con 0x7fded413bbc0 2024-03-29T08:29:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.697+0000 7fdedaa00640 1 -- 172.21.15.46:0/4102305222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded415f540 con 0x7fded413bbc0 2024-03-29T08:29:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.701+0000 7fdedaa00640 1 -- 172.21.15.46:0/4102305222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded4161750 con 0x7fded413bbc0 2024-03-29T08:29:29.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.701+0000 7fdecbe00640 1 -- 172.21.15.46:0/4102305222 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdec0005040 con 0x7fded413bbc0 2024-03-29T08:29:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.701+0000 7fdecbe00640 1 --2- 172.21.15.46:0/4102305222 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeb40666c0 0x7fdeb4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.701+0000 7fdecbe00640 1 -- 172.21.15.46:0/4102305222 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdec011da80 con 0x7fded413bbc0 2024-03-29T08:29:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.701+0000 7fded3e00640 1 --2- 172.21.15.46:0/4102305222 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeb40666c0 0x7fdeb4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:29.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.701+0000 7fded3e00640 1 --2- 172.21.15.46:0/4102305222 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeb40666c0 0x7fdeb4068b80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fdec4007fd0 tx=0x7fdec4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:29.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.705+0000 7fdecbe00640 1 -- 172.21.15.46:0/4102305222 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fded4161750 con 0x7fded413bbc0 2024-03-29T08:29:29.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.865+0000 7fdedaa00640 1 -- 172.21.15.46:0/4102305222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fded4161750 con 0x7fded413bbc0 2024-03-29T08:29:29.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.865+0000 7fdecbe00640 1 -- 172.21.15.46:0/4102305222 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7fded4161750 con 0x7fded413bbc0 2024-03-29T08:29:29.869 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:29:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.865+0000 7fdedaa00640 1 -- 172.21.15.46:0/4102305222 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeb40666c0 msgr2=0x7fdeb4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.865+0000 7fdedaa00640 1 --2- 172.21.15.46:0/4102305222 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeb40666c0 0x7fdeb4068b80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fdec4007fd0 tx=0x7fdec4009040 comp rx=0 tx=0).stop 2024-03-29T08:29:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.865+0000 7fdedaa00640 1 -- 172.21.15.46:0/4102305222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded413bbc0 msgr2=0x7fded4161a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.865+0000 7fdedaa00640 1 --2- 172.21.15.46:0/4102305222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded413bbc0 0x7fded4161a30 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fdec00ab040 tx=0x7fdeb8007a10 comp rx=0 tx=0).stop 2024-03-29T08:29:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.865+0000 7fdedaa00640 1 -- 172.21.15.46:0/4102305222 shutdown_connections 2024-03-29T08:29:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.865+0000 7fdedaa00640 1 --2- 172.21.15.46:0/4102305222 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdeb40666c0 0x7fdeb4068b80 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.865+0000 7fdedaa00640 1 --2- 172.21.15.46:0/4102305222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fded413c040 0x7fded4161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.865+0000 7fdedaa00640 1 --2- 172.21.15.46:0/4102305222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fded413bbc0 0x7fded4161a30 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.865+0000 7fdedaa00640 1 --2- 172.21.15.46:0/4102305222 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdeb8006290 0x7fded415ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.865+0000 7fdedaa00640 1 -- 172.21.15.46:0/4102305222 >> 172.21.15.46:0/4102305222 conn(0x7fded406a390 msgr2=0x7fded40b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:29.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.865+0000 7fdedaa00640 1 -- 172.21.15.46:0/4102305222 shutdown_connections 2024-03-29T08:29:29.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:29.865+0000 7fdedaa00640 1 -- 172.21.15.46:0/4102305222 wait complete. 2024-03-29T08:29:29.890 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:29:30.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.029+0000 7fd35c400640 1 Processor -- start 2024-03-29T08:29:30.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.029+0000 7fd35c400640 1 -- start start 2024-03-29T08:29:30.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.029+0000 7fd35c400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3540b1cb0 0x7fd3540cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:30.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.029+0000 7fd35c400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35415cf40 con 0x7fd3540d0dc0 2024-03-29T08:29:30.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.029+0000 7fd35c400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3540b0250 con 0x7fd35415cb60 2024-03-29T08:29:30.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.029+0000 7fd35c400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3540b0810 con 0x7fd3540b1cb0 2024-03-29T08:29:30.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.029+0000 7fd35a400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd3540b1cb0 0x7fd3540cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:30.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.029+0000 7fd359000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd3540d0dc0 0x7fd3540cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33754/0 (socket says 172.21.15.46:33754) 2024-03-29T08:29:30.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.029+0000 7fd359000640 1 -- 172.21.15.46:0/1357429058 learned_addr learned my addr 172.21.15.46:0/1357429058 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:30.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.029+0000 7fd35a400640 1 -- 172.21.15.46:0/1357429058 >> v1:172.21.15.73:6789/0 conn(0x7fd35415cb60 legacy=0x7fd3540d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.029+0000 7fd35a400640 1 -- 172.21.15.46:0/1357429058 >> v1:172.21.15.46:6789/0 conn(0x7fd3540d0dc0 legacy=0x7fd3540cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.029+0000 7fd35a400640 1 -- 172.21.15.46:0/1357429058 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3540d0010 con 0x7fd3540b1cb0 2024-03-29T08:29:30.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.029+0000 7fd353e00640 1 -- 172.21.15.46:0/1357429058 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3814211292 0 0) 0x7fd35415cf40 con 0x7fd3540d0dc0 2024-03-29T08:29:30.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.033+0000 7fd35a400640 1 --2- 172.21.15.46:0/1357429058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3540b1cb0 0x7fd3540cf9f0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009d90 tx=0x7fd34c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7c64486d47df6b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:30.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.033+0000 7fd353e00640 1 -- 172.21.15.46:0/1357429058 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c0a5a00 con 0x7fd3540b1cb0 2024-03-29T08:29:30.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.033+0000 7fd353e00640 1 -- 172.21.15.46:0/1357429058 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c0a5b60 con 0x7fd3540b1cb0 2024-03-29T08:29:30.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.033+0000 7fd353e00640 1 -- 172.21.15.46:0/1357429058 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c0ad440 con 0x7fd3540b1cb0 2024-03-29T08:29:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.033+0000 7fd35c400640 1 -- 172.21.15.46:0/1357429058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3540b1cb0 msgr2=0x7fd3540cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.033+0000 7fd35c400640 1 --2- 172.21.15.46:0/1357429058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3540b1cb0 0x7fd3540cf9f0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009d90 tx=0x7fd34c01c940 comp rx=0 tx=0).stop 2024-03-29T08:29:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.033+0000 7fd35c400640 1 -- 172.21.15.46:0/1357429058 shutdown_connections 2024-03-29T08:29:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.033+0000 7fd35c400640 1 --2- 172.21.15.46:0/1357429058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3540b1cb0 0x7fd3540cf9f0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.033+0000 7fd35c400640 1 -- 172.21.15.46:0/1357429058 >> 172.21.15.46:0/1357429058 conn(0x7fd354099a10 msgr2=0x7fd354089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.033+0000 7fd35c400640 1 -- 172.21.15.46:0/1357429058 shutdown_connections 2024-03-29T08:29:30.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.033+0000 7fd35c400640 1 -- 172.21.15.46:0/1357429058 wait complete. 2024-03-29T08:29:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd35c400640 1 Processor -- start 2024-03-29T08:29:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd35c400640 1 -- start start 2024-03-29T08:29:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd35c400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3540b1cb0 0x7fd354153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd35c400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3540d0dc0 0x7fd354150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd35c400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd35415cb60 0x7fd3540c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd35c400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540b62e0 con 0x7fd3540b1cb0 2024-03-29T08:29:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd35c400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540b4520 con 0x7fd35415cb60 2024-03-29T08:29:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd35c400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3540b6000 con 0x7fd3540d0dc0 2024-03-29T08:29:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd359a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3540b1cb0 0x7fd354153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd359a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3540b1cb0 0x7fd354153c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38784/0 (socket says 172.21.15.46:38784) 2024-03-29T08:29:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd359a00640 1 -- 172.21.15.46:0/318935551 learned_addr learned my addr 172.21.15.46:0/318935551 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:30.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd35a400640 1 --2- 172.21.15.46:0/318935551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd35415cb60 0x7fd3540c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd359a00640 1 -- 172.21.15.46:0/318935551 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3540d0dc0 msgr2=0x7fd354150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd359a00640 1 --2- 172.21.15.46:0/318935551 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3540d0dc0 0x7fd354150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd359a00640 1 -- 172.21.15.46:0/318935551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd35415cb60 msgr2=0x7fd3540c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd359a00640 1 --2- 172.21.15.46:0/318935551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd35415cb60 0x7fd3540c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd359a00640 1 -- 172.21.15.46:0/318935551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c009a70 con 0x7fd3540b1cb0 2024-03-29T08:29:30.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd359a00640 1 --2- 172.21.15.46:0/318935551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3540b1cb0 0x7fd354153c50 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd34800c530 tx=0x7fd34800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:30.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd351600640 1 -- 172.21.15.46:0/318935551 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348005220 con 0x7fd3540b1cb0 2024-03-29T08:29:30.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd351600640 1 -- 172.21.15.46:0/318935551 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd348005380 con 0x7fd3540b1cb0 2024-03-29T08:29:30.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd351600640 1 -- 172.21.15.46:0/318935551 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348007270 con 0x7fd3540b1cb0 2024-03-29T08:29:30.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd35c400640 1 -- 172.21.15.46:0/318935551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3540b3ed0 con 0x7fd3540b1cb0 2024-03-29T08:29:30.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd35c400640 1 -- 172.21.15.46:0/318935551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3540b5c90 con 0x7fd3540b1cb0 2024-03-29T08:29:30.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.037+0000 7fd35c400640 1 -- 172.21.15.46:0/318935551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3540b3d30 con 0x7fd3540b1cb0 2024-03-29T08:29:30.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.041+0000 7fd351600640 1 -- 172.21.15.46:0/318935551 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd3480073d0 con 0x7fd3540b1cb0 2024-03-29T08:29:30.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.041+0000 7fd351600640 1 --2- 172.21.15.46:0/318935551 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3340666c0 0x7fd334068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:30.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.041+0000 7fd351600640 1 -- 172.21.15.46:0/318935551 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd348083ae0 con 0x7fd3540b1cb0 2024-03-29T08:29:30.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.041+0000 7fd359000640 1 --2- 172.21.15.46:0/318935551 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3340666c0 0x7fd334068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:30.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.041+0000 7fd359000640 1 --2- 172.21.15.46:0/318935551 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3340666c0 0x7fd334068b80 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd3400011f0 tx=0x7fd340007930 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:30.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.045+0000 7fd351600640 1 -- 172.21.15.46:0/318935551 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd3540b3d30 con 0x7fd3540b1cb0 2024-03-29T08:29:30.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.209+0000 7fd35c400640 1 -- 172.21.15.46:0/318935551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3540b3d30 con 0x7fd3540b1cb0 2024-03-29T08:29:30.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.209+0000 7fd351600640 1 -- 172.21.15.46:0/318935551 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd3540b3d30 con 0x7fd3540b1cb0 2024-03-29T08:29:30.215 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:29:30.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.213+0000 7fd35c400640 1 -- 172.21.15.46:0/318935551 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3340666c0 msgr2=0x7fd334068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.213+0000 7fd35c400640 1 --2- 172.21.15.46:0/318935551 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3340666c0 0x7fd334068b80 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd3400011f0 tx=0x7fd340007930 comp rx=0 tx=0).stop 2024-03-29T08:29:30.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.213+0000 7fd35c400640 1 -- 172.21.15.46:0/318935551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3540b1cb0 msgr2=0x7fd354153c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.213+0000 7fd35c400640 1 --2- 172.21.15.46:0/318935551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3540b1cb0 0x7fd354153c50 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd34800c530 tx=0x7fd34800c9f0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.213+0000 7fd35c400640 1 -- 172.21.15.46:0/318935551 shutdown_connections 2024-03-29T08:29:30.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.213+0000 7fd35c400640 1 --2- 172.21.15.46:0/318935551 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd3340666c0 0x7fd334068b80 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.213+0000 7fd35c400640 1 --2- 172.21.15.46:0/318935551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd35415cb60 0x7fd3540c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.213+0000 7fd35c400640 1 --2- 172.21.15.46:0/318935551 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd3540d0dc0 0x7fd354150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.213+0000 7fd35c400640 1 --2- 172.21.15.46:0/318935551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3540b1cb0 0x7fd354153c50 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.213+0000 7fd35c400640 1 -- 172.21.15.46:0/318935551 >> 172.21.15.46:0/318935551 conn(0x7fd354099a10 msgr2=0x7fd3540cd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:30.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.213+0000 7fd35c400640 1 -- 172.21.15.46:0/318935551 shutdown_connections 2024-03-29T08:29:30.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.213+0000 7fd35c400640 1 -- 172.21.15.46:0/318935551 wait complete. 2024-03-29T08:29:30.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:29:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.373+0000 7fb04c600640 1 Processor -- start 2024-03-29T08:29:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.373+0000 7fb04c600640 1 -- start start 2024-03-29T08:29:30.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.373+0000 7fb04c600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb04415db10 0x7fb044164410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:30.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.373+0000 7fb04c600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb044070c70 con 0x7fb04406dbb0 2024-03-29T08:29:30.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.373+0000 7fb04c600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb044070340 con 0x7fb044142ab0 2024-03-29T08:29:30.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.373+0000 7fb04c600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04415df40 con 0x7fb04415db10 2024-03-29T08:29:30.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.373+0000 7fb049200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb04406dbb0 0x7fb04415d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33768/0 (socket says 172.21.15.46:33768) 2024-03-29T08:29:30.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.373+0000 7fb049200640 1 -- 172.21.15.46:0/2973277146 learned_addr learned my addr 172.21.15.46:0/2973277146 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:30.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.373+0000 7fb04a600640 1 --2- 172.21.15.46:0/2973277146 >> v2:172.21.15.46:3300/0 conn(0x7fb04415db10 0x7fb044164410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:30.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.373+0000 7fb043e00640 1 -- 172.21.15.46:0/2973277146 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265258525 0 0) 0x7fb044070c70 con 0x7fb04406dbb0 2024-03-29T08:29:30.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.373+0000 7fb043e00640 1 -- 172.21.15.46:0/2973277146 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb02c005740 con 0x7fb04406dbb0 2024-03-29T08:29:30.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.373+0000 7fb04a600640 1 -- 172.21.15.46:0/2973277146 >> v1:172.21.15.73:6789/0 conn(0x7fb044142ab0 legacy=0x7fb04406d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.373+0000 7fb04a600640 1 -- 172.21.15.46:0/2973277146 >> v1:172.21.15.46:6789/0 conn(0x7fb04406dbb0 legacy=0x7fb04415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.373+0000 7fb04a600640 1 -- 172.21.15.46:0/2973277146 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb044070620 con 0x7fb04415db10 2024-03-29T08:29:30.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.377+0000 7fb04a600640 1 --2- 172.21.15.46:0/2973277146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04415db10 0x7fb044164410 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009d90 tx=0x7fb03c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daad42bc5f3f30c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:30.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.377+0000 7fb043e00640 1 -- 172.21.15.46:0/2973277146 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0a5730 con 0x7fb04415db10 2024-03-29T08:29:30.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.377+0000 7fb043e00640 1 -- 172.21.15.46:0/2973277146 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb03c0a5890 con 0x7fb04415db10 2024-03-29T08:29:30.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.377+0000 7fb043e00640 1 -- 172.21.15.46:0/2973277146 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c0a5c20 con 0x7fb04415db10 2024-03-29T08:29:30.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.377+0000 7fb04c600640 1 -- 172.21.15.46:0/2973277146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04415db10 msgr2=0x7fb044164410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.377+0000 7fb04c600640 1 --2- 172.21.15.46:0/2973277146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04415db10 0x7fb044164410 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009d90 tx=0x7fb03c01c970 comp rx=0 tx=0).stop 2024-03-29T08:29:30.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.377+0000 7fb04c600640 1 -- 172.21.15.46:0/2973277146 shutdown_connections 2024-03-29T08:29:30.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.377+0000 7fb04c600640 1 --2- 172.21.15.46:0/2973277146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04415db10 0x7fb044164410 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.377+0000 7fb04c600640 1 -- 172.21.15.46:0/2973277146 >> 172.21.15.46:0/2973277146 conn(0x7fb044066080 msgr2=0x7fb044069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.377+0000 7fb04c600640 1 -- 172.21.15.46:0/2973277146 shutdown_connections 2024-03-29T08:29:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.377+0000 7fb04c600640 1 -- 172.21.15.46:0/2973277146 wait complete. 2024-03-29T08:29:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb04c600640 1 Processor -- start 2024-03-29T08:29:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb04c600640 1 -- start start 2024-03-29T08:29:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb04c600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04406dbb0 0x7fb04413cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb04c600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb044142ab0 0x7fb04413f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb04c600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04415db10 0x7fb0440a88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb04c600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04413d510 con 0x7fb04406dbb0 2024-03-29T08:29:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb04c600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04413f930 con 0x7fb04415db10 2024-03-29T08:29:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb04c600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0440a7810 con 0x7fb044142ab0 2024-03-29T08:29:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb049c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04406dbb0 0x7fb04413cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb049c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04406dbb0 0x7fb04413cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38800/0 (socket says 172.21.15.46:38800) 2024-03-29T08:29:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb049c00640 1 -- 172.21.15.46:0/1313493097 learned_addr learned my addr 172.21.15.46:0/1313493097 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb04a600640 1 --2- 172.21.15.46:0/1313493097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04415db10 0x7fb0440a88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb049200640 1 --2- 172.21.15.46:0/1313493097 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb044142ab0 0x7fb04413f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb049c00640 1 -- 172.21.15.46:0/1313493097 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb044142ab0 msgr2=0x7fb04413f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb049c00640 1 --2- 172.21.15.46:0/1313493097 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb044142ab0 0x7fb04413f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb049c00640 1 -- 172.21.15.46:0/1313493097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04415db10 msgr2=0x7fb0440a88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb049c00640 1 --2- 172.21.15.46:0/1313493097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04415db10 0x7fb0440a88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb049c00640 1 -- 172.21.15.46:0/1313493097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c009a70 con 0x7fb04406dbb0 2024-03-29T08:29:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb049c00640 1 --2- 172.21.15.46:0/1313493097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04406dbb0 0x7fb04413cfd0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fb03800c500 tx=0x7fb03800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb041600640 1 -- 172.21.15.46:0/1313493097 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038005220 con 0x7fb04406dbb0 2024-03-29T08:29:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb041600640 1 -- 172.21.15.46:0/1313493097 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb038005380 con 0x7fb04406dbb0 2024-03-29T08:29:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb04c600640 1 -- 172.21.15.46:0/1313493097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0440a7af0 con 0x7fb04406dbb0 2024-03-29T08:29:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb041600640 1 -- 172.21.15.46:0/1313493097 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb038007270 con 0x7fb04406dbb0 2024-03-29T08:29:30.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.381+0000 7fb04c600640 1 -- 172.21.15.46:0/1313493097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0440a80b0 con 0x7fb04406dbb0 2024-03-29T08:29:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.385+0000 7fb04c600640 1 -- 172.21.15.46:0/1313493097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb044071a90 con 0x7fb04406dbb0 2024-03-29T08:29:30.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.385+0000 7fb041600640 1 -- 172.21.15.46:0/1313493097 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb0380073d0 con 0x7fb04406dbb0 2024-03-29T08:29:30.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.385+0000 7fb041600640 1 --2- 172.21.15.46:0/1313493097 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb024066690 0x7fb024068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:30.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.385+0000 7fb041600640 1 -- 172.21.15.46:0/1313493097 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0380839e0 con 0x7fb04406dbb0 2024-03-29T08:29:30.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.385+0000 7fb049200640 1 --2- 172.21.15.46:0/1313493097 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb024066690 0x7fb024068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:30.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.385+0000 7fb049200640 1 --2- 172.21.15.46:0/1313493097 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb024066690 0x7fb024068b50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb044140c30 tx=0x7fb030007d40 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:30.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.389+0000 7fb041600640 1 -- 172.21.15.46:0/1313493097 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb038050d70 con 0x7fb04406dbb0 2024-03-29T08:29:30.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.569+0000 7fb04c600640 1 -- 172.21.15.46:0/1313493097 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb04404e120 con 0x7fb04406dbb0 2024-03-29T08:29:30.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.569+0000 7fb041600640 1 -- 172.21.15.46:0/1313493097 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb038054390 con 0x7fb04406dbb0 2024-03-29T08:29:30.573 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:29:30.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.569+0000 7fb04c600640 1 -- 172.21.15.46:0/1313493097 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb024066690 msgr2=0x7fb024068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.569+0000 7fb04c600640 1 --2- 172.21.15.46:0/1313493097 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb024066690 0x7fb024068b50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb044140c30 tx=0x7fb030007d40 comp rx=0 tx=0).stop 2024-03-29T08:29:30.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.569+0000 7fb04c600640 1 -- 172.21.15.46:0/1313493097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04406dbb0 msgr2=0x7fb04413cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.569+0000 7fb04c600640 1 --2- 172.21.15.46:0/1313493097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04406dbb0 0x7fb04413cfd0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fb03800c500 tx=0x7fb03800c9c0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.569+0000 7fb04c600640 1 -- 172.21.15.46:0/1313493097 shutdown_connections 2024-03-29T08:29:30.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.569+0000 7fb04c600640 1 --2- 172.21.15.46:0/1313493097 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb024066690 0x7fb024068b50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.569+0000 7fb04c600640 1 --2- 172.21.15.46:0/1313493097 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb04415db10 0x7fb0440a88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.569+0000 7fb04c600640 1 --2- 172.21.15.46:0/1313493097 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb044142ab0 0x7fb04413f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.569+0000 7fb04c600640 1 --2- 172.21.15.46:0/1313493097 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb04406dbb0 0x7fb04413cfd0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.569+0000 7fb04c600640 1 -- 172.21.15.46:0/1313493097 >> 172.21.15.46:0/1313493097 conn(0x7fb044066080 msgr2=0x7fb044159cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:30.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.569+0000 7fb04c600640 1 -- 172.21.15.46:0/1313493097 shutdown_connections 2024-03-29T08:29:30.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.569+0000 7fb04c600640 1 -- 172.21.15.46:0/1313493097 wait complete. 2024-03-29T08:29:30.595 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:29:30.595 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:29:30.603 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:29:30.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.785+0000 7f94ad400640 1 Processor -- start 2024-03-29T08:29:30.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.785+0000 7f94ad400640 1 -- start start 2024-03-29T08:29:30.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.785+0000 7f94ad400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f94a80ae810 0x7f94a80caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.785+0000 7f94ad400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f94a80cb780 0x7f94a80aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.785+0000 7f94ad400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a80cbc10 con 0x7f94a80ce8a0 2024-03-29T08:29:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.785+0000 7f94ad400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a80c9bd0 con 0x7f94a80ae810 2024-03-29T08:29:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.785+0000 7f94ad400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a80c9eb0 con 0x7f94a80cb780 2024-03-29T08:29:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.785+0000 7f949fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f94a80ae810 0x7f94a80caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.785+0000 7f94a6800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f94a80ce8a0 0x7f94a80cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33774/0 (socket says 172.21.15.46:33774) 2024-03-29T08:29:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.785+0000 7f94a6800640 1 -- 172.21.15.46:0/3431373784 learned_addr learned my addr 172.21.15.46:0/3431373784 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.785+0000 7f94a7200640 1 --2- 172.21.15.46:0/3431373784 >> v2:172.21.15.73:3300/0 conn(0x7f94a80cb780 0x7f94a80aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.789+0000 7f949fe00640 1 -- 172.21.15.46:0/3431373784 >> v2:172.21.15.73:3300/0 conn(0x7f94a80cb780 msgr2=0x7f94a80aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.789+0000 7f949fe00640 1 --2- 172.21.15.46:0/3431373784 >> v2:172.21.15.73:3300/0 conn(0x7f94a80cb780 0x7f94a80aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.789+0000 7f949fe00640 1 -- 172.21.15.46:0/3431373784 >> v1:172.21.15.46:6789/0 conn(0x7f94a80ce8a0 legacy=0x7f94a80cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.789+0000 7f949fe00640 1 -- 172.21.15.46:0/3431373784 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a80ca190 con 0x7f94a80ae810 2024-03-29T08:29:30.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.789+0000 7f94a5e00640 1 -- 172.21.15.46:0/3431373784 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226173045 0 0) 0x7f94a80cbc10 con 0x7f94a80ce8a0 2024-03-29T08:29:30.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.789+0000 7f949fe00640 1 --2- 172.21.15.46:0/3431373784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a80ae810 0x7f94a80caf50 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f94a8070890 tx=0x7f949001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae15856955804cc1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:30.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.789+0000 7f94a5e00640 1 -- 172.21.15.46:0/3431373784 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94a80afa20 con 0x7f94a80ae810 2024-03-29T08:29:30.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.789+0000 7f94a5e00640 1 -- 172.21.15.46:0/3431373784 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94900ab650 con 0x7f94a80ae810 2024-03-29T08:29:30.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.789+0000 7f94a5e00640 1 -- 172.21.15.46:0/3431373784 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94900ab9e0 con 0x7f94a80ae810 2024-03-29T08:29:30.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.789+0000 7f94ad400640 1 -- 172.21.15.46:0/3431373784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a80ae810 msgr2=0x7f94a80caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.789+0000 7f94ad400640 1 --2- 172.21.15.46:0/3431373784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a80ae810 0x7f94a80caf50 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f94a8070890 tx=0x7f949001c450 comp rx=0 tx=0).stop 2024-03-29T08:29:30.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.789+0000 7f94ad400640 1 -- 172.21.15.46:0/3431373784 shutdown_connections 2024-03-29T08:29:30.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.789+0000 7f94ad400640 1 --2- 172.21.15.46:0/3431373784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a80ae810 0x7f94a80caf50 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.789+0000 7f94ad400640 1 --2- 172.21.15.46:0/3431373784 >> v2:172.21.15.73:3300/0 conn(0x7f94a80cb780 0x7f94a80aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.789+0000 7f94ad400640 1 -- 172.21.15.46:0/3431373784 >> 172.21.15.46:0/3431373784 conn(0x7f94a8098a40 msgr2=0x7f94a80cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:30.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f94ad400640 1 -- 172.21.15.46:0/3431373784 shutdown_connections 2024-03-29T08:29:30.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f94ad400640 1 -- 172.21.15.46:0/3431373784 wait complete. 2024-03-29T08:29:30.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f94ad400640 1 Processor -- start 2024-03-29T08:29:30.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f94ad400640 1 -- start start 2024-03-29T08:29:30.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f94ad400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f94a80ae810 0x7f94a8155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:30.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f94ad400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a80cb780 0x7f94a80cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:30.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f94ad400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a80ce8a0 0x7f94a80cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:30.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f94ad400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80acc40 con 0x7f94a80cb780 2024-03-29T08:29:30.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f94ad400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a8153d30 con 0x7f94a80ce8a0 2024-03-29T08:29:30.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f94ad400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a80ac310 con 0x7f94a80ae810 2024-03-29T08:29:30.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f949fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a80cb780 0x7f94a80cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:30.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f949fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a80cb780 0x7f94a80cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38818/0 (socket says 172.21.15.46:38818) 2024-03-29T08:29:30.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f949fe00640 1 -- 172.21.15.46:0/2929370397 learned_addr learned my addr 172.21.15.46:0/2929370397 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:30.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f94a7200640 1 --2- 172.21.15.46:0/2929370397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a80ce8a0 0x7f94a80cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:30.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f94a6800640 1 --2- 172.21.15.46:0/2929370397 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f94a80ae810 0x7f94a8155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:30.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f949fe00640 1 -- 172.21.15.46:0/2929370397 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f94a80ae810 msgr2=0x7f94a8155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f949fe00640 1 --2- 172.21.15.46:0/2929370397 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f94a80ae810 0x7f94a8155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f949fe00640 1 -- 172.21.15.46:0/2929370397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a80ce8a0 msgr2=0x7f94a80cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f949fe00640 1 --2- 172.21.15.46:0/2929370397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a80ce8a0 0x7f94a80cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f949fe00640 1 -- 172.21.15.46:0/2929370397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9490009420 con 0x7f94a80cb780 2024-03-29T08:29:30.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f94a7200640 1 --2- 172.21.15.46:0/2929370397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a80ce8a0 0x7f94a80cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:30.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f949fe00640 1 --2- 172.21.15.46:0/2929370397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a80cb780 0x7f94a80cbda0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f9490019060 tx=0x7f949001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:30.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f949ea00640 1 -- 172.21.15.46:0/2929370397 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94900b5810 con 0x7f94a80cb780 2024-03-29T08:29:30.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f949ea00640 1 -- 172.21.15.46:0/2929370397 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94900b5990 con 0x7f94a80cb780 2024-03-29T08:29:30.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f949ea00640 1 -- 172.21.15.46:0/2929370397 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94900b5d40 con 0x7f94a80cb780 2024-03-29T08:29:30.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.793+0000 7f94ad400640 1 -- 172.21.15.46:0/2929370397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a80ac5f0 con 0x7f94a80cb780 2024-03-29T08:29:30.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.797+0000 7f94ad400640 1 -- 172.21.15.46:0/2929370397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a81560b0 con 0x7f94a80cb780 2024-03-29T08:29:30.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.797+0000 7f94ad400640 1 -- 172.21.15.46:0/2929370397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a81562a0 con 0x7f94a80cb780 2024-03-29T08:29:30.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.797+0000 7f949ea00640 1 -- 172.21.15.46:0/2929370397 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f94900bf410 con 0x7f94a80cb780 2024-03-29T08:29:30.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.797+0000 7f949ea00640 1 --2- 172.21.15.46:0/2929370397 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9484066690 0x7f9484068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:30.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.797+0000 7f949ea00640 1 -- 172.21.15.46:0/2929370397 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f949011ee40 con 0x7f94a80cb780 2024-03-29T08:29:30.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.797+0000 7f94a6800640 1 --2- 172.21.15.46:0/2929370397 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9484066690 0x7f9484068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:30.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.801+0000 7f94a6800640 1 --2- 172.21.15.46:0/2929370397 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9484066690 0x7f9484068b50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9498000f70 tx=0x7f9498007c10 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:30.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.801+0000 7f949ea00640 1 -- 172.21.15.46:0/2929370397 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f94a81562a0 con 0x7f94a80cb780 2024-03-29T08:29:30.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.961+0000 7f94ad400640 1 -- 172.21.15.46:0/2929370397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94a81562a0 con 0x7f94a80cb780 2024-03-29T08:29:30.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.965+0000 7f949ea00640 1 -- 172.21.15.46:0/2929370397 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16067 (secure 0 0 0) 0x7f94a81562a0 con 0x7f94a80cb780 2024-03-29T08:29:30.968 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:30.968 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":121,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:29:15.473193+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:29:12.329121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-29T08:29:13.465939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:29:30.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.965+0000 7f94ad400640 1 -- 172.21.15.46:0/2929370397 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9484066690 msgr2=0x7f9484068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.965+0000 7f94ad400640 1 --2- 172.21.15.46:0/2929370397 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9484066690 0x7f9484068b50 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9498000f70 tx=0x7f9498007c10 comp rx=0 tx=0).stop 2024-03-29T08:29:30.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.965+0000 7f94ad400640 1 -- 172.21.15.46:0/2929370397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a80cb780 msgr2=0x7f94a80cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:30.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.965+0000 7f94ad400640 1 --2- 172.21.15.46:0/2929370397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a80cb780 0x7f94a80cbda0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f9490019060 tx=0x7f949001ac90 comp rx=0 tx=0).stop 2024-03-29T08:29:30.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.965+0000 7f94ad400640 1 -- 172.21.15.46:0/2929370397 shutdown_connections 2024-03-29T08:29:30.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.965+0000 7f94ad400640 1 --2- 172.21.15.46:0/2929370397 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9484066690 0x7f9484068b50 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.965+0000 7f94ad400640 1 --2- 172.21.15.46:0/2929370397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94a80ce8a0 0x7f94a80cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.965+0000 7f94ad400640 1 --2- 172.21.15.46:0/2929370397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f94a80cb780 0x7f94a80cbda0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.965+0000 7f94ad400640 1 --2- 172.21.15.46:0/2929370397 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f94a80ae810 0x7f94a8155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:30.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.965+0000 7f94ad400640 1 -- 172.21.15.46:0/2929370397 >> 172.21.15.46:0/2929370397 conn(0x7f94a8098a40 msgr2=0x7f94a806d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:30.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.965+0000 7f94ad400640 1 -- 172.21.15.46:0/2929370397 shutdown_connections 2024-03-29T08:29:30.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:30.965+0000 7f94ad400640 1 -- 172.21.15.46:0/2929370397 wait complete. 2024-03-29T08:29:30.990 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:29:12.329121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:29:13.465939+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:29:30.990 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:29:31.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.129+0000 7f481a200640 1 Processor -- start 2024-03-29T08:29:31.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.129+0000 7f481a200640 1 -- start start 2024-03-29T08:29:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.133+0000 7f481a200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f481415b9c0 0x7f481415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.133+0000 7f481a200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f481415e300 con 0x7f4814143d60 2024-03-29T08:29:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.133+0000 7f481a200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48140afcb0 con 0x7f4814144140 2024-03-29T08:29:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.133+0000 7f481a200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48140aff90 con 0x7f481415b9c0 2024-03-29T08:29:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.133+0000 7f480aa00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f481415b9c0 0x7f481415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.133+0000 7f4813400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4814143d60 0x7f48140af4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33786/0 (socket says 172.21.15.46:33786) 2024-03-29T08:29:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.133+0000 7f4813400640 1 -- 172.21.15.46:0/1878658460 learned_addr learned my addr 172.21.15.46:0/1878658460 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:31.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.133+0000 7f480aa00640 1 -- 172.21.15.46:0/1878658460 >> v1:172.21.15.73:6789/0 conn(0x7f4814144140 legacy=0x7f4814070280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:31.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.133+0000 7f480aa00640 1 -- 172.21.15.46:0/1878658460 >> v1:172.21.15.46:6789/0 conn(0x7f4814143d60 legacy=0x7f48140af4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:31.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.133+0000 7f480aa00640 1 -- 172.21.15.46:0/1878658460 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481415e5a0 con 0x7f481415b9c0 2024-03-29T08:29:31.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f480aa00640 1 --2- 172.21.15.46:0/1878658460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481415b9c0 0x7f481415eda0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0097c0 tx=0x7f47fc01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6165781693c9e627 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:31.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f4812a00640 1 -- 172.21.15.46:0/1878658460 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc0ac620 con 0x7f481415b9c0 2024-03-29T08:29:31.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f4812a00640 1 -- 172.21.15.46:0/1878658460 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47fc0ac780 con 0x7f481415b9c0 2024-03-29T08:29:31.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f4812a00640 1 -- 172.21.15.46:0/1878658460 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc0acb10 con 0x7f481415b9c0 2024-03-29T08:29:31.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f481a200640 1 -- 172.21.15.46:0/1878658460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481415b9c0 msgr2=0x7f481415eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:31.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f481a200640 1 --2- 172.21.15.46:0/1878658460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481415b9c0 0x7f481415eda0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0097c0 tx=0x7f47fc01c3a0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f481a200640 1 -- 172.21.15.46:0/1878658460 shutdown_connections 2024-03-29T08:29:31.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f481a200640 1 --2- 172.21.15.46:0/1878658460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f481415b9c0 0x7f481415eda0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f481a200640 1 -- 172.21.15.46:0/1878658460 >> 172.21.15.46:0/1878658460 conn(0x7f4814066080 msgr2=0x7f4814069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:31.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f481a200640 1 -- 172.21.15.46:0/1878658460 shutdown_connections 2024-03-29T08:29:31.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f481a200640 1 -- 172.21.15.46:0/1878658460 wait complete. 2024-03-29T08:29:31.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f481a200640 1 Processor -- start 2024-03-29T08:29:31.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f481a200640 1 -- start start 2024-03-29T08:29:31.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f481a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4814143d60 0x7f481413dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:31.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f481a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4814144140 0x7f4814164e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:31.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f481a200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f481415b9c0 0x7f48140ab670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:31.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f481a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4814165340 con 0x7f4814144140 2024-03-29T08:29:31.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f481a200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481413ef80 con 0x7f4814143d60 2024-03-29T08:29:31.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.137+0000 7f481a200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481413e370 con 0x7f481415b9c0 2024-03-29T08:29:31.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f480aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4814144140 0x7f4814164e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:31.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f480aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4814144140 0x7f4814164e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38842/0 (socket says 172.21.15.46:38842) 2024-03-29T08:29:31.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f480aa00640 1 -- 172.21.15.46:0/961058722 learned_addr learned my addr 172.21.15.46:0/961058722 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:31.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f4813400640 1 --2- 172.21.15.46:0/961058722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4814143d60 0x7f481413dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:31.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f4813e00640 1 --2- 172.21.15.46:0/961058722 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f481415b9c0 0x7f48140ab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:31.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f480aa00640 1 -- 172.21.15.46:0/961058722 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f481415b9c0 msgr2=0x7f48140ab670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:31.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f480aa00640 1 --2- 172.21.15.46:0/961058722 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f481415b9c0 0x7f48140ab670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f480aa00640 1 -- 172.21.15.46:0/961058722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4814143d60 msgr2=0x7f481413dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:31.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f480aa00640 1 --2- 172.21.15.46:0/961058722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4814143d60 0x7f481413dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f480aa00640 1 -- 172.21.15.46:0/961058722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc009420 con 0x7f4814144140 2024-03-29T08:29:31.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f480aa00640 1 --2- 172.21.15.46:0/961058722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4814144140 0x7f4814164e00 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f47fc018fb0 tx=0x7f47fc019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:31.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f480be00640 1 -- 172.21.15.46:0/961058722 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc0b5610 con 0x7f4814144140 2024-03-29T08:29:31.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f480be00640 1 -- 172.21.15.46:0/961058722 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47fc0b5790 con 0x7f4814144140 2024-03-29T08:29:31.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f480be00640 1 -- 172.21.15.46:0/961058722 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47fc0b5b40 con 0x7f4814144140 2024-03-29T08:29:31.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f481a200640 1 -- 172.21.15.46:0/961058722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481415cbf0 con 0x7f4814144140 2024-03-29T08:29:31.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f481a200640 1 -- 172.21.15.46:0/961058722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481413e930 con 0x7f4814144140 2024-03-29T08:29:31.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f481a200640 1 -- 172.21.15.46:0/961058722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48140707c0 con 0x7f4814144140 2024-03-29T08:29:31.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.141+0000 7f480be00640 1 -- 172.21.15.46:0/961058722 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f47fc0b3040 con 0x7f4814144140 2024-03-29T08:29:31.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.145+0000 7f480be00640 1 --2- 172.21.15.46:0/961058722 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f47f0066690 0x7f47f0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:31.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.145+0000 7f480be00640 1 -- 172.21.15.46:0/961058722 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47fc11df30 con 0x7f4814144140 2024-03-29T08:29:31.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.145+0000 7f4813400640 1 --2- 172.21.15.46:0/961058722 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f47f0066690 0x7f47f0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:31.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.145+0000 7f4813400640 1 --2- 172.21.15.46:0/961058722 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f47f0066690 0x7f47f0068b50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4804005890 tx=0x7f4804009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:31.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.145+0000 7f480be00640 1 -- 172.21.15.46:0/961058722 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f47fc0eb320 con 0x7f4814144140 2024-03-29T08:29:31.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.309+0000 7f481a200640 1 -- 172.21.15.46:0/961058722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4814071a70 con 0x7f4814144140 2024-03-29T08:29:31.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.309+0000 7f480be00640 1 -- 172.21.15.46:0/961058722 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f47fc0ac5e0 con 0x7f4814144140 2024-03-29T08:29:31.315 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:29:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.313+0000 7f481a200640 1 -- 172.21.15.46:0/961058722 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f47f0066690 msgr2=0x7f47f0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.313+0000 7f481a200640 1 --2- 172.21.15.46:0/961058722 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f47f0066690 0x7f47f0068b50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f4804005890 tx=0x7f4804009040 comp rx=0 tx=0).stop 2024-03-29T08:29:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.313+0000 7f481a200640 1 -- 172.21.15.46:0/961058722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4814144140 msgr2=0x7f4814164e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.313+0000 7f481a200640 1 --2- 172.21.15.46:0/961058722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4814144140 0x7f4814164e00 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f47fc018fb0 tx=0x7f47fc019140 comp rx=0 tx=0).stop 2024-03-29T08:29:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.313+0000 7f481a200640 1 -- 172.21.15.46:0/961058722 shutdown_connections 2024-03-29T08:29:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.313+0000 7f481a200640 1 --2- 172.21.15.46:0/961058722 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f47f0066690 0x7f47f0068b50 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.313+0000 7f481a200640 1 --2- 172.21.15.46:0/961058722 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f481415b9c0 0x7f48140ab670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.313+0000 7f481a200640 1 --2- 172.21.15.46:0/961058722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4814144140 0x7f4814164e00 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.313+0000 7f481a200640 1 --2- 172.21.15.46:0/961058722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4814143d60 0x7f481413dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.313+0000 7f481a200640 1 -- 172.21.15.46:0/961058722 >> 172.21.15.46:0/961058722 conn(0x7f4814066080 msgr2=0x7f48141407a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.313+0000 7f481a200640 1 -- 172.21.15.46:0/961058722 shutdown_connections 2024-03-29T08:29:31.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.313+0000 7f481a200640 1 -- 172.21.15.46:0/961058722 wait complete. 2024-03-29T08:29:31.336 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:29:31.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.473+0000 7f84ed400640 1 Processor -- start 2024-03-29T08:29:31.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.473+0000 7f84ed400640 1 -- start start 2024-03-29T08:29:31.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.477+0000 7f84ed400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84e815a750 0x7f84e80cb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:31.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.477+0000 7f84ed400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f84e80aec10 0x7f84e80cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:31.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.477+0000 7f84ed400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e80af140 con 0x7f84e80b0580 2024-03-29T08:29:31.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.477+0000 7f84ed400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e80af420 con 0x7f84e80aec10 2024-03-29T08:29:31.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.477+0000 7f84ed400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e80af700 con 0x7f84e815a750 2024-03-29T08:29:31.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.477+0000 7f84e7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f84e80b0580 0x7f84e815a040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33790/0 (socket says 172.21.15.46:33790) 2024-03-29T08:29:31.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.477+0000 7f84e7e00640 1 -- 172.21.15.46:0/1770588141 learned_addr learned my addr 172.21.15.46:0/1770588141 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:31.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.477+0000 7f84eca00640 1 --2- 172.21.15.46:0/1770588141 >> v2:172.21.15.73:3300/0 conn(0x7f84e80aec10 0x7f84e80cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:31.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.477+0000 7f84df400640 1 --2- 172.21.15.46:0/1770588141 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84e815a750 0x7f84e80cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:31.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.477+0000 7f84e7400640 1 -- 172.21.15.46:0/1770588141 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062178866 0 0) 0x7f84e80af140 con 0x7f84e80b0580 2024-03-29T08:29:31.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.477+0000 7f84e7400640 1 -- 172.21.15.46:0/1770588141 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84cc003530 con 0x7f84e80b0580 2024-03-29T08:29:31.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.477+0000 7f84eca00640 1 -- 172.21.15.46:0/1770588141 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84e815a750 msgr2=0x7f84e80cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:31.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.477+0000 7f84eca00640 1 --2- 172.21.15.46:0/1770588141 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84e815a750 0x7f84e80cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.477+0000 7f84eca00640 1 -- 172.21.15.46:0/1770588141 >> v1:172.21.15.46:6789/0 conn(0x7f84e80b0580 legacy=0x7f84e815a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:31.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.477+0000 7f84eca00640 1 -- 172.21.15.46:0/1770588141 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e80af9e0 con 0x7f84e80aec10 2024-03-29T08:29:31.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.477+0000 7f84df400640 1 --2- 172.21.15.46:0/1770588141 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84e815a750 0x7f84e80cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:29:31.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.481+0000 7f84eca00640 1 --2- 172.21.15.46:0/1770588141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84e80aec10 0x7f84e80cb5a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009d90 tx=0x7f84e001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44538942ff64430d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:31.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.481+0000 7f84e7400640 1 -- 172.21.15.46:0/1770588141 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e00a5a20 con 0x7f84e80aec10 2024-03-29T08:29:31.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.481+0000 7f84e7400640 1 -- 172.21.15.46:0/1770588141 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e00a5b80 con 0x7f84e80aec10 2024-03-29T08:29:31.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.481+0000 7f84e7400640 1 -- 172.21.15.46:0/1770588141 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e00ad440 con 0x7f84e80aec10 2024-03-29T08:29:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.481+0000 7f84ed400640 1 -- 172.21.15.46:0/1770588141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84e80aec10 msgr2=0x7f84e80cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.481+0000 7f84ed400640 1 --2- 172.21.15.46:0/1770588141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84e80aec10 0x7f84e80cb5a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f84e0009d90 tx=0x7f84e001c940 comp rx=0 tx=0).stop 2024-03-29T08:29:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.481+0000 7f84ed400640 1 -- 172.21.15.46:0/1770588141 shutdown_connections 2024-03-29T08:29:31.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.481+0000 7f84ed400640 1 --2- 172.21.15.46:0/1770588141 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84e815a750 0x7f84e80cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.481+0000 7f84ed400640 1 --2- 172.21.15.46:0/1770588141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84e80aec10 0x7f84e80cb5a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.481+0000 7f84ed400640 1 -- 172.21.15.46:0/1770588141 >> 172.21.15.46:0/1770588141 conn(0x7f84e8098a40 msgr2=0x7f84e80cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:31.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.481+0000 7f84ed400640 1 -- 172.21.15.46:0/1770588141 shutdown_connections 2024-03-29T08:29:31.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.481+0000 7f84ed400640 1 -- 172.21.15.46:0/1770588141 wait complete. 2024-03-29T08:29:31.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.481+0000 7f84ed400640 1 Processor -- start 2024-03-29T08:29:31.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.481+0000 7f84ed400640 1 -- start start 2024-03-29T08:29:31.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84ed400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84e80aec10 0x7f84e815ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:31.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84ed400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84e80b0580 0x7f84e8152830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:31.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84ed400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84e815a750 0x7f84e81f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:31.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84ed400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e81f7b50 con 0x7f84e80aec10 2024-03-29T08:29:31.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84ed400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e81f6f40 con 0x7f84e80b0580 2024-03-29T08:29:31.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84ed400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e81f7220 con 0x7f84e815a750 2024-03-29T08:29:31.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84e80aec10 0x7f84e815ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:31.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84e7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84e80aec10 0x7f84e815ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38852/0 (socket says 172.21.15.46:38852) 2024-03-29T08:29:31.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84e7e00640 1 -- 172.21.15.46:0/427256271 learned_addr learned my addr 172.21.15.46:0/427256271 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:31.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84df400640 1 --2- 172.21.15.46:0/427256271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84e80b0580 0x7f84e8152830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:31.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84eca00640 1 --2- 172.21.15.46:0/427256271 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84e815a750 0x7f84e81f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:31.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84e7e00640 1 -- 172.21.15.46:0/427256271 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84e815a750 msgr2=0x7f84e81f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:31.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84e7e00640 1 --2- 172.21.15.46:0/427256271 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84e815a750 0x7f84e81f63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84e7e00640 1 -- 172.21.15.46:0/427256271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84e80b0580 msgr2=0x7f84e8152830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:31.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84e7e00640 1 --2- 172.21.15.46:0/427256271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84e80b0580 0x7f84e8152830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84e7e00640 1 -- 172.21.15.46:0/427256271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d0009420 con 0x7f84e80aec10 2024-03-29T08:29:31.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84e7e00640 1 --2- 172.21.15.46:0/427256271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84e80aec10 0x7f84e815ab90 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f84d800c4d0 tx=0x7f84d800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:31.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84e4c00640 1 -- 172.21.15.46:0/427256271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8005cf0 con 0x7f84e80aec10 2024-03-29T08:29:31.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84e4c00640 1 -- 172.21.15.46:0/427256271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d8005e50 con 0x7f84e80aec10 2024-03-29T08:29:31.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84e4c00640 1 -- 172.21.15.46:0/427256271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d8007270 con 0x7f84e80aec10 2024-03-29T08:29:31.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84ed400640 1 -- 172.21.15.46:0/427256271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e0009a70 con 0x7f84e80aec10 2024-03-29T08:29:31.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84ed400640 1 -- 172.21.15.46:0/427256271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e81f77e0 con 0x7f84e80aec10 2024-03-29T08:29:31.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84ed400640 1 -- 172.21.15.46:0/427256271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e804e040 con 0x7f84e80aec10 2024-03-29T08:29:31.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.485+0000 7f84e4c00640 1 -- 172.21.15.46:0/427256271 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f84d80073d0 con 0x7f84e80aec10 2024-03-29T08:29:31.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.489+0000 7f84e4c00640 1 --2- 172.21.15.46:0/427256271 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f84c4066690 0x7f84c4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:31.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.489+0000 7f84e4c00640 1 -- 172.21.15.46:0/427256271 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84d8083690 con 0x7f84e80aec10 2024-03-29T08:29:31.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.489+0000 7f84df400640 1 --2- 172.21.15.46:0/427256271 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f84c4066690 0x7f84c4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:31.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.489+0000 7f84df400640 1 --2- 172.21.15.46:0/427256271 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f84c4066690 0x7f84c4068b50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f84e81575f0 tx=0x7f84d0009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:31.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.493+0000 7f84e4c00640 1 -- 172.21.15.46:0/427256271 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f84d8050ab0 con 0x7f84e80aec10 2024-03-29T08:29:31.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.653+0000 7f84ed400640 1 -- 172.21.15.46:0/427256271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f84e80ca360 con 0x7f84e80aec10 2024-03-29T08:29:31.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.653+0000 7f84e4c00640 1 -- 172.21.15.46:0/427256271 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7f84d80540d0 con 0x7f84e80aec10 2024-03-29T08:29:31.659 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:29:31.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.657+0000 7f84ed400640 1 -- 172.21.15.46:0/427256271 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f84c4066690 msgr2=0x7f84c4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.657+0000 7f84ed400640 1 --2- 172.21.15.46:0/427256271 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f84c4066690 0x7f84c4068b50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f84e81575f0 tx=0x7f84d0009190 comp rx=0 tx=0).stop 2024-03-29T08:29:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.657+0000 7f84ed400640 1 -- 172.21.15.46:0/427256271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84e80aec10 msgr2=0x7f84e815ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.657+0000 7f84ed400640 1 --2- 172.21.15.46:0/427256271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84e80aec10 0x7f84e815ab90 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f84d800c4d0 tx=0x7f84d800c990 comp rx=0 tx=0).stop 2024-03-29T08:29:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.657+0000 7f84ed400640 1 -- 172.21.15.46:0/427256271 shutdown_connections 2024-03-29T08:29:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.657+0000 7f84ed400640 1 --2- 172.21.15.46:0/427256271 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f84c4066690 0x7f84c4068b50 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.657+0000 7f84ed400640 1 --2- 172.21.15.46:0/427256271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84e80b0580 0x7f84e8152830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.657+0000 7f84ed400640 1 --2- 172.21.15.46:0/427256271 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f84e815a750 0x7f84e81f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.657+0000 7f84ed400640 1 --2- 172.21.15.46:0/427256271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84e80aec10 0x7f84e815ab90 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.657+0000 7f84ed400640 1 -- 172.21.15.46:0/427256271 >> 172.21.15.46:0/427256271 conn(0x7f84e8098a40 msgr2=0x7f84e80cb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.657+0000 7f84ed400640 1 -- 172.21.15.46:0/427256271 shutdown_connections 2024-03-29T08:29:31.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.657+0000 7f84ed400640 1 -- 172.21.15.46:0/427256271 wait complete. 2024-03-29T08:29:31.680 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:29:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.817+0000 7f314a600640 1 Processor -- start 2024-03-29T08:29:31.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.817+0000 7f314a600640 1 -- start start 2024-03-29T08:29:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.821+0000 7f314a600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3144142a50 0x7f31440cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.821+0000 7f314a600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f31440ae990 0x7f31440cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.821+0000 7f314a600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f314415d260 con 0x7f314415df90 2024-03-29T08:29:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.821+0000 7f314a600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31440aefd0 con 0x7f3144142a50 2024-03-29T08:29:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.821+0000 7f314a600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314415d4b0 con 0x7f31440ae990 2024-03-29T08:29:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.821+0000 7f313be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f314415df90 0x7f31440ae280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33794/0 (socket says 172.21.15.46:33794) 2024-03-29T08:29:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.821+0000 7f313be00640 1 -- 172.21.15.46:0/3079995271 learned_addr learned my addr 172.21.15.46:0/3079995271 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.821+0000 7f3143400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3144142a50 0x7f31440cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:31.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.821+0000 7f3143e00640 1 --2- 172.21.15.46:0/3079995271 >> v2:172.21.15.73:3300/0 conn(0x7f31440ae990 0x7f31440cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.821+0000 7f3142a00640 1 -- 172.21.15.46:0/3079995271 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915235899 0 0) 0x7f314415d260 con 0x7f314415df90 2024-03-29T08:29:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.821+0000 7f3143400640 1 -- 172.21.15.46:0/3079995271 >> v2:172.21.15.73:3300/0 conn(0x7f31440ae990 msgr2=0x7f31440cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.821+0000 7f3143400640 1 --2- 172.21.15.46:0/3079995271 >> v2:172.21.15.73:3300/0 conn(0x7f31440ae990 0x7f31440cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.821+0000 7f3143400640 1 -- 172.21.15.46:0/3079995271 >> v1:172.21.15.46:6789/0 conn(0x7f314415df90 legacy=0x7f31440ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:31.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.821+0000 7f3143400640 1 -- 172.21.15.46:0/3079995271 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314415d790 con 0x7f3144142a50 2024-03-29T08:29:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.825+0000 7f3143400640 1 --2- 172.21.15.46:0/3079995271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3144142a50 0x7f31440cb790 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f314406e4e0 tx=0x7f313401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3017c87a12aae2f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.825+0000 7f3142a00640 1 -- 172.21.15.46:0/3079995271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31340ab600 con 0x7f3144142a50 2024-03-29T08:29:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.825+0000 7f3142a00640 1 -- 172.21.15.46:0/3079995271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31340a7050 con 0x7f3144142a50 2024-03-29T08:29:31.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.825+0000 7f3142a00640 1 -- 172.21.15.46:0/3079995271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31340ab980 con 0x7f3144142a50 2024-03-29T08:29:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.825+0000 7f314a600640 1 -- 172.21.15.46:0/3079995271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3144142a50 msgr2=0x7f31440cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.825+0000 7f314a600640 1 --2- 172.21.15.46:0/3079995271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3144142a50 0x7f31440cb790 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f314406e4e0 tx=0x7f313401c3f0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.825+0000 7f314a600640 1 -- 172.21.15.46:0/3079995271 shutdown_connections 2024-03-29T08:29:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.825+0000 7f314a600640 1 --2- 172.21.15.46:0/3079995271 >> v2:172.21.15.73:3300/0 conn(0x7f31440ae990 0x7f31440cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.825+0000 7f314a600640 1 --2- 172.21.15.46:0/3079995271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3144142a50 0x7f31440cb790 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.825+0000 7f314a600640 1 -- 172.21.15.46:0/3079995271 >> 172.21.15.46:0/3079995271 conn(0x7f314406a390 msgr2=0x7f31440cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:31.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.825+0000 7f314a600640 1 -- 172.21.15.46:0/3079995271 shutdown_connections 2024-03-29T08:29:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.825+0000 7f314a600640 1 -- 172.21.15.46:0/3079995271 wait complete. 2024-03-29T08:29:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.825+0000 7f314a600640 1 Processor -- start 2024-03-29T08:29:31.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.825+0000 7f314a600640 1 -- start start 2024-03-29T08:29:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f314a600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31440ae990 0x7f314413bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f314a600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3144142a50 0x7f3144141210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f314a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314415df90 0x7f3144158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f314a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f314415a8d0 con 0x7f314415df90 2024-03-29T08:29:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f314a600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3144159ce0 con 0x7f31440ae990 2024-03-29T08:29:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f314a600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3144159f30 con 0x7f3144142a50 2024-03-29T08:29:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f3143e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314415df90 0x7f3144158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f3143e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314415df90 0x7f3144158590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38874/0 (socket says 172.21.15.46:38874) 2024-03-29T08:29:31.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f3143e00640 1 -- 172.21.15.46:0/77029842 learned_addr learned my addr 172.21.15.46:0/77029842 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f313be00640 1 --2- 172.21.15.46:0/77029842 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3144142a50 0x7f3144141210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f3143400640 1 --2- 172.21.15.46:0/77029842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31440ae990 0x7f314413bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f3143e00640 1 -- 172.21.15.46:0/77029842 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3144142a50 msgr2=0x7f3144141210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f3143e00640 1 --2- 172.21.15.46:0/77029842 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3144142a50 0x7f3144141210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f3143e00640 1 -- 172.21.15.46:0/77029842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31440ae990 msgr2=0x7f314413bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f3143e00640 1 --2- 172.21.15.46:0/77029842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31440ae990 0x7f314413bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f3143e00640 1 -- 172.21.15.46:0/77029842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3134009420 con 0x7f314415df90 2024-03-29T08:29:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f3143e00640 1 --2- 172.21.15.46:0/77029842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314415df90 0x7f3144158590 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f313c00efc0 tx=0x7f313c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f313b400640 1 -- 172.21.15.46:0/77029842 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c009ba0 con 0x7f314415df90 2024-03-29T08:29:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f313b400640 1 -- 172.21.15.46:0/77029842 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c00f040 con 0x7f314415df90 2024-03-29T08:29:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f313b400640 1 -- 172.21.15.46:0/77029842 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c0146c0 con 0x7f314415df90 2024-03-29T08:29:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f314a600640 1 -- 172.21.15.46:0/77029842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f314415a210 con 0x7f314415df90 2024-03-29T08:29:31.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f314a600640 1 -- 172.21.15.46:0/77029842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314413c410 con 0x7f314415df90 2024-03-29T08:29:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f314a600640 1 -- 172.21.15.46:0/77029842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31440b45a0 con 0x7f314415df90 2024-03-29T08:29:31.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.829+0000 7f313b400640 1 -- 172.21.15.46:0/77029842 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f313c014820 con 0x7f314415df90 2024-03-29T08:29:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.833+0000 7f313b400640 1 --2- 172.21.15.46:0/77029842 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3120066690 0x7f3120068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.833+0000 7f313b400640 1 -- 172.21.15.46:0/77029842 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f313c084ad0 con 0x7f314415df90 2024-03-29T08:29:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.833+0000 7f3143400640 1 --2- 172.21.15.46:0/77029842 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3120066690 0x7f3120068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:31.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.833+0000 7f3143400640 1 --2- 172.21.15.46:0/77029842 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3120066690 0x7f3120068b50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f31340ad000 tx=0x7f3134008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:31.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.837+0000 7f313b400640 1 -- 172.21.15.46:0/77029842 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f313c051f80 con 0x7f314415df90 2024-03-29T08:29:31.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.993+0000 7f314a600640 1 -- 172.21.15.46:0/77029842 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f314415aa10 con 0x7f314415df90 2024-03-29T08:29:31.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.993+0000 7f313b400640 1 -- 172.21.15.46:0/77029842 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7f313c0180a0 con 0x7f314415df90 2024-03-29T08:29:32.000 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:29:32.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.997+0000 7f314a600640 1 -- 172.21.15.46:0/77029842 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3120066690 msgr2=0x7f3120068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:32.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.997+0000 7f314a600640 1 --2- 172.21.15.46:0/77029842 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3120066690 0x7f3120068b50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f31340ad000 tx=0x7f3134008040 comp rx=0 tx=0).stop 2024-03-29T08:29:32.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.997+0000 7f314a600640 1 -- 172.21.15.46:0/77029842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314415df90 msgr2=0x7f3144158590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:32.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.997+0000 7f314a600640 1 --2- 172.21.15.46:0/77029842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314415df90 0x7f3144158590 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f313c00efc0 tx=0x7f313c00c490 comp rx=0 tx=0).stop 2024-03-29T08:29:32.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.997+0000 7f314a600640 1 -- 172.21.15.46:0/77029842 shutdown_connections 2024-03-29T08:29:32.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.997+0000 7f314a600640 1 --2- 172.21.15.46:0/77029842 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3120066690 0x7f3120068b50 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:32.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.997+0000 7f314a600640 1 --2- 172.21.15.46:0/77029842 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314415df90 0x7f3144158590 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:32.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.997+0000 7f314a600640 1 --2- 172.21.15.46:0/77029842 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3144142a50 0x7f3144141210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:32.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.997+0000 7f314a600640 1 --2- 172.21.15.46:0/77029842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31440ae990 0x7f314413bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:32.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.997+0000 7f314a600640 1 -- 172.21.15.46:0/77029842 >> 172.21.15.46:0/77029842 conn(0x7f314406a390 msgr2=0x7f314415bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:32.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.997+0000 7f314a600640 1 -- 172.21.15.46:0/77029842 shutdown_connections 2024-03-29T08:29:32.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:31.997+0000 7f314a600640 1 -- 172.21.15.46:0/77029842 wait complete. 2024-03-29T08:29:32.021 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:29:32.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.161+0000 7fc921a00640 1 Processor -- start 2024-03-29T08:29:32.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.161+0000 7fc921a00640 1 -- start start 2024-03-29T08:29:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.161+0000 7fc921a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc91c0b2640 0x7fc91c0b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.161+0000 7fc921a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc91c1604b0 con 0x7fc91c0709b0 2024-03-29T08:29:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.161+0000 7fc921a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc91c160700 con 0x7fc91c1461f0 2024-03-29T08:29:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.161+0000 7fc921a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc91c1465d0 con 0x7fc91c0b2640 2024-03-29T08:29:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.161+0000 7fc91b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc91c0b2640 0x7fc91c0b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:32.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.161+0000 7fc913e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc91c0709b0 0x7fc91c0b2d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:33800/0 (socket says 172.21.15.46:33800) 2024-03-29T08:29:32.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.161+0000 7fc913e00640 1 -- 172.21.15.46:0/1893227705 learned_addr learned my addr 172.21.15.46:0/1893227705 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:32.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.165+0000 7fc91b400640 1 -- 172.21.15.46:0/1893227705 >> v1:172.21.15.73:6789/0 conn(0x7fc91c1461f0 legacy=0x7fc91c0702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:32.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.165+0000 7fc91b400640 1 -- 172.21.15.46:0/1893227705 >> v1:172.21.15.46:6789/0 conn(0x7fc91c0709b0 legacy=0x7fc91c0b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:32.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.165+0000 7fc91b400640 1 -- 172.21.15.46:0/1893227705 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91c1609e0 con 0x7fc91c0b2640 2024-03-29T08:29:32.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.165+0000 7fc91b400640 1 --2- 172.21.15.46:0/1893227705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc91c0b2640 0x7fc91c0b3440 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fc914009d90 tx=0x7fc91401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad311c173194a27d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.165+0000 7fc91a000640 1 -- 172.21.15.46:0/1893227705 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9140a5970 con 0x7fc91c0b2640 2024-03-29T08:29:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.165+0000 7fc91a000640 1 -- 172.21.15.46:0/1893227705 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9140a5ad0 con 0x7fc91c0b2640 2024-03-29T08:29:32.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.165+0000 7fc91a000640 1 -- 172.21.15.46:0/1893227705 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9140a5e60 con 0x7fc91c0b2640 2024-03-29T08:29:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.165+0000 7fc921a00640 1 -- 172.21.15.46:0/1893227705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc91c0b2640 msgr2=0x7fc91c0b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.165+0000 7fc921a00640 1 --2- 172.21.15.46:0/1893227705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc91c0b2640 0x7fc91c0b3440 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fc914009d90 tx=0x7fc91401c8b0 comp rx=0 tx=0).stop 2024-03-29T08:29:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc921a00640 1 -- 172.21.15.46:0/1893227705 shutdown_connections 2024-03-29T08:29:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc921a00640 1 --2- 172.21.15.46:0/1893227705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc91c0b2640 0x7fc91c0b3440 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc921a00640 1 -- 172.21.15.46:0/1893227705 >> 172.21.15.46:0/1893227705 conn(0x7fc91c066080 msgr2=0x7fc91c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc921a00640 1 -- 172.21.15.46:0/1893227705 shutdown_connections 2024-03-29T08:29:32.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc921a00640 1 -- 172.21.15.46:0/1893227705 wait complete. 2024-03-29T08:29:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc921a00640 1 Processor -- start 2024-03-29T08:29:32.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc921a00640 1 -- start start 2024-03-29T08:29:32.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc921a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc91c0709b0 0x7fc91c139050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:32.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc921a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc91c0b2640 0x7fc91c15ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:32.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc921a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc91c1461f0 0x7fc91c144840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:32.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc921a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c13a240 con 0x7fc91c1461f0 2024-03-29T08:29:32.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc921a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c139630 con 0x7fc91c0709b0 2024-03-29T08:29:32.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc921a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c139910 con 0x7fc91c0b2640 2024-03-29T08:29:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc91b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc91c1461f0 0x7fc91c144840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc91aa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc91c0709b0 0x7fc91c139050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc913e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc91c0b2640 0x7fc91c15ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc91b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc91c1461f0 0x7fc91c144840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38900/0 (socket says 172.21.15.46:38900) 2024-03-29T08:29:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc91b400640 1 -- 172.21.15.46:0/959095863 learned_addr learned my addr 172.21.15.46:0/959095863 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc91b400640 1 -- 172.21.15.46:0/959095863 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc91c0b2640 msgr2=0x7fc91c15ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc91b400640 1 --2- 172.21.15.46:0/959095863 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc91c0b2640 0x7fc91c15ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc91b400640 1 -- 172.21.15.46:0/959095863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc91c0709b0 msgr2=0x7fc91c139050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc91b400640 1 --2- 172.21.15.46:0/959095863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc91c0709b0 0x7fc91c139050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc91b400640 1 -- 172.21.15.46:0/959095863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc914009a70 con 0x7fc91c1461f0 2024-03-29T08:29:32.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc91b400640 1 --2- 172.21.15.46:0/959095863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc91c1461f0 0x7fc91c144840 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fc9140b0040 tx=0x7fc914019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:32.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc912a00640 1 -- 172.21.15.46:0/959095863 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9140ade40 con 0x7fc91c1461f0 2024-03-29T08:29:32.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.169+0000 7fc912a00640 1 -- 172.21.15.46:0/959095863 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9140b9450 con 0x7fc91c1461f0 2024-03-29T08:29:32.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.173+0000 7fc921a00640 1 -- 172.21.15.46:0/959095863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91c139bf0 con 0x7fc91c1461f0 2024-03-29T08:29:32.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.173+0000 7fc912a00640 1 -- 172.21.15.46:0/959095863 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9140b9800 con 0x7fc91c1461f0 2024-03-29T08:29:32.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.173+0000 7fc921a00640 1 -- 172.21.15.46:0/959095863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc91c0b3680 con 0x7fc91c1461f0 2024-03-29T08:29:32.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.173+0000 7fc921a00640 1 -- 172.21.15.46:0/959095863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc91c13a3b0 con 0x7fc91c1461f0 2024-03-29T08:29:32.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.173+0000 7fc912a00640 1 -- 172.21.15.46:0/959095863 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc9140b9960 con 0x7fc91c1461f0 2024-03-29T08:29:32.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.173+0000 7fc912a00640 1 --2- 172.21.15.46:0/959095863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8f8066690 0x7fc8f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:32.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.173+0000 7fc912a00640 1 -- 172.21.15.46:0/959095863 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc91411fa10 con 0x7fc91c1461f0 2024-03-29T08:29:32.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.173+0000 7fc91aa00640 1 --2- 172.21.15.46:0/959095863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8f8066690 0x7fc8f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:32.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.173+0000 7fc91aa00640 1 --2- 172.21.15.46:0/959095863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8f8066690 0x7fc8f8068b50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc90c007fd0 tx=0x7fc90c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:32.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.177+0000 7fc912a00640 1 -- 172.21.15.46:0/959095863 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc9140ecf70 con 0x7fc91c1461f0 2024-03-29T08:29:32.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.349+0000 7fc921a00640 1 -- 172.21.15.46:0/959095863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc91c071a90 con 0x7fc91c1461f0 2024-03-29T08:29:32.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.353+0000 7fc912a00640 1 -- 172.21.15.46:0/959095863 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc9140f0590 con 0x7fc91c1461f0 2024-03-29T08:29:32.356 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:29:32.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.353+0000 7fc921a00640 1 -- 172.21.15.46:0/959095863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8f8066690 msgr2=0x7fc8f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:32.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.353+0000 7fc921a00640 1 --2- 172.21.15.46:0/959095863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8f8066690 0x7fc8f8068b50 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc90c007fd0 tx=0x7fc90c009040 comp rx=0 tx=0).stop 2024-03-29T08:29:32.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.353+0000 7fc921a00640 1 -- 172.21.15.46:0/959095863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc91c1461f0 msgr2=0x7fc91c144840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:32.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.353+0000 7fc921a00640 1 --2- 172.21.15.46:0/959095863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc91c1461f0 0x7fc91c144840 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fc9140b0040 tx=0x7fc914019650 comp rx=0 tx=0).stop 2024-03-29T08:29:32.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.353+0000 7fc921a00640 1 -- 172.21.15.46:0/959095863 shutdown_connections 2024-03-29T08:29:32.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.353+0000 7fc921a00640 1 --2- 172.21.15.46:0/959095863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8f8066690 0x7fc8f8068b50 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:32.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.353+0000 7fc921a00640 1 --2- 172.21.15.46:0/959095863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc91c1461f0 0x7fc91c144840 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:32.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.353+0000 7fc921a00640 1 --2- 172.21.15.46:0/959095863 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc91c0b2640 0x7fc91c15ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:32.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.353+0000 7fc921a00640 1 --2- 172.21.15.46:0/959095863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc91c0709b0 0x7fc91c139050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:32.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.353+0000 7fc921a00640 1 -- 172.21.15.46:0/959095863 >> 172.21.15.46:0/959095863 conn(0x7fc91c066080 msgr2=0x7fc91c15cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:32.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.353+0000 7fc921a00640 1 -- 172.21.15.46:0/959095863 shutdown_connections 2024-03-29T08:29:32.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:32.353+0000 7fc921a00640 1 -- 172.21.15.46:0/959095863 wait complete. 2024-03-29T08:29:32.377 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-03-29T08:29:32.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:32.378 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:32.396 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[50887]: fuse finished with error 0 and tester_r 0 2024-03-29T08:29:32.397 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:29:38.499 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:29:38.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:38.499 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:29:38.511 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-29T08:29:38.511 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-03-29T08:29:38.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.701+0000 7f0798200640 1 Processor -- start 2024-03-29T08:29:38.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.701+0000 7f0798200640 1 -- start start 2024-03-29T08:29:38.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.705+0000 7f0798200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f07900cbea0 0x7f07900b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.705+0000 7f0798200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f079015efe0 0x7f07900b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.705+0000 7f0798200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07900b20b0 con 0x7f0790143eb0 2024-03-29T08:29:38.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.705+0000 7f0798200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07900b2360 con 0x7f079015efe0 2024-03-29T08:29:38.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.705+0000 7f0798200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f079015e500 con 0x7f07900cbea0 2024-03-29T08:29:38.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.705+0000 7f0795800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0790143eb0 0x7f07900cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43344/0 (socket says 172.21.15.46:43344) 2024-03-29T08:29:38.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.705+0000 7f0795800640 1 -- 172.21.15.46:0/3056374747 learned_addr learned my addr 172.21.15.46:0/3056374747 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:38.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.705+0000 7f0796200640 1 --2- 172.21.15.46:0/3056374747 >> v2:172.21.15.46:3300/0 conn(0x7f079015efe0 0x7f07900b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.705+0000 7f0794e00640 1 --2- 172.21.15.46:0/3056374747 >> v2:172.21.15.73:3300/0 conn(0x7f07900cbea0 0x7f07900b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.705+0000 7f0796200640 1 -- 172.21.15.46:0/3056374747 >> v2:172.21.15.73:3300/0 conn(0x7f07900cbea0 msgr2=0x7f07900b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:38.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.705+0000 7f0796200640 1 --2- 172.21.15.46:0/3056374747 >> v2:172.21.15.73:3300/0 conn(0x7f07900cbea0 0x7f07900b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.705+0000 7f0796200640 1 -- 172.21.15.46:0/3056374747 >> v1:172.21.15.46:6789/0 conn(0x7f0790143eb0 legacy=0x7f07900cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:38.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.705+0000 7f0796200640 1 -- 172.21.15.46:0/3056374747 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079015e7e0 con 0x7f079015efe0 2024-03-29T08:29:38.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.705+0000 7f078fe00640 1 -- 172.21.15.46:0/3056374747 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3493714872 0 0) 0x7f07900b20b0 con 0x7f0790143eb0 2024-03-29T08:29:38.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.705+0000 7f0794e00640 1 --2- 172.21.15.46:0/3056374747 >> v2:172.21.15.73:3300/0 conn(0x7f07900cbea0 0x7f07900b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:38.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f0796200640 1 --2- 172.21.15.46:0/3056374747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f079015efe0 0x7f07900b1b70 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f079006ad40 tx=0x7f078801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b528fd4a577f040 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:38.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f078fe00640 1 -- 172.21.15.46:0/3056374747 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07880a5960 con 0x7f079015efe0 2024-03-29T08:29:38.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f078fe00640 1 -- 172.21.15.46:0/3056374747 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07880a5ac0 con 0x7f079015efe0 2024-03-29T08:29:38.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f078fe00640 1 -- 172.21.15.46:0/3056374747 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07880a5e50 con 0x7f079015efe0 2024-03-29T08:29:38.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f0798200640 1 -- 172.21.15.46:0/3056374747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f079015efe0 msgr2=0x7f07900b1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:38.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f0798200640 1 --2- 172.21.15.46:0/3056374747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f079015efe0 0x7f07900b1b70 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f079006ad40 tx=0x7f078801c900 comp rx=0 tx=0).stop 2024-03-29T08:29:38.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f0798200640 1 -- 172.21.15.46:0/3056374747 shutdown_connections 2024-03-29T08:29:38.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f0798200640 1 --2- 172.21.15.46:0/3056374747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f079015efe0 0x7f07900b1b70 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f0798200640 1 --2- 172.21.15.46:0/3056374747 >> v2:172.21.15.73:3300/0 conn(0x7f07900cbea0 0x7f07900b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f0798200640 1 -- 172.21.15.46:0/3056374747 >> 172.21.15.46:0/3056374747 conn(0x7f079006a390 msgr2=0x7f07900cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:38.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f0798200640 1 -- 172.21.15.46:0/3056374747 shutdown_connections 2024-03-29T08:29:38.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f0798200640 1 -- 172.21.15.46:0/3056374747 wait complete. 2024-03-29T08:29:38.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f0798200640 1 Processor -- start 2024-03-29T08:29:38.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f0798200640 1 -- start start 2024-03-29T08:29:38.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f0798200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07900cbea0 0x7f079015dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f0798200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0790143eb0 0x7f079013ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f0798200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f079015efe0 0x7f0790139a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f0798200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079015e020 con 0x7f079015efe0 2024-03-29T08:29:38.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f0798200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f079015cac0 con 0x7f07900cbea0 2024-03-29T08:29:38.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.709+0000 7f0798200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f079015cda0 con 0x7f0790143eb0 2024-03-29T08:29:38.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f0796200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f079015efe0 0x7f0790139a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f0796200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f079015efe0 0x7f0790139a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60010/0 (socket says 172.21.15.46:60010) 2024-03-29T08:29:38.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f0796200640 1 -- 172.21.15.46:0/261173384 learned_addr learned my addr 172.21.15.46:0/261173384 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:38.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f0795800640 1 --2- 172.21.15.46:0/261173384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07900cbea0 0x7f079015dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f0794e00640 1 --2- 172.21.15.46:0/261173384 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0790143eb0 0x7f079013ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f0796200640 1 -- 172.21.15.46:0/261173384 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0790143eb0 msgr2=0x7f079013ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:38.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f0796200640 1 --2- 172.21.15.46:0/261173384 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0790143eb0 0x7f079013ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f0796200640 1 -- 172.21.15.46:0/261173384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07900cbea0 msgr2=0x7f079015dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:38.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f0796200640 1 --2- 172.21.15.46:0/261173384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07900cbea0 0x7f079015dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f0796200640 1 -- 172.21.15.46:0/261173384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0788009a70 con 0x7f079015efe0 2024-03-29T08:29:38.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f0794e00640 1 --2- 172.21.15.46:0/261173384 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0790143eb0 0x7f079013ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:38.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f0796200640 1 --2- 172.21.15.46:0/261173384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f079015efe0 0x7f0790139a80 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f0788019510 tx=0x7f07880196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:38.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f078d600640 1 -- 172.21.15.46:0/261173384 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07880aedf0 con 0x7f079015efe0 2024-03-29T08:29:38.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f078d600640 1 -- 172.21.15.46:0/261173384 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0788009450 con 0x7f079015efe0 2024-03-29T08:29:38.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f078d600640 1 -- 172.21.15.46:0/261173384 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07880b7650 con 0x7f079015efe0 2024-03-29T08:29:38.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f0798200640 1 -- 172.21.15.46:0/261173384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f079015d080 con 0x7f079015efe0 2024-03-29T08:29:38.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f0798200640 1 -- 172.21.15.46:0/261173384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f079015d640 con 0x7f079015efe0 2024-03-29T08:29:38.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.713+0000 7f078d600640 1 -- 172.21.15.46:0/261173384 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f07880b4050 con 0x7f079015efe0 2024-03-29T08:29:38.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f078d600640 1 --2- 172.21.15.46:0/261173384 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0770066690 0x7f0770068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f078d600640 1 -- 172.21.15.46:0/261173384 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f078811f440 con 0x7f079015efe0 2024-03-29T08:29:38.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f0795800640 1 --2- 172.21.15.46:0/261173384 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0770066690 0x7f0770068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f079b6c3000 1 Processor -- start 2024-03-29T08:29:38.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f079b6c3000 1 -- start start 2024-03-29T08:29:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f0795800640 1 --2- 172.21.15.46:0/261173384 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0770066690 0x7f0770068b50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0784007fd0 tx=0x7f0784007d60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f079b6c3000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbb18cc0 0x55acdbb190a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f079b6c3000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbace8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f079b6c3000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbacee20 0x55acdbacf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f079b6c3000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdb993630 con 0x55acdbb195e0 2024-03-29T08:29:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f079b6c3000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdb920bd0 con 0x55acdbb18cc0 2024-03-29T08:29:38.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f079b6c3000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acdb91e1a0 con 0x55acdbacee20 2024-03-29T08:29:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f0794e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbace8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f0795800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbb18cc0 0x55acdbb190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f0794e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbace8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60020/0 (socket says 172.21.15.46:60020) 2024-03-29T08:29:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f0794e00640 1 -- 172.21.15.46:0/2778490075 learned_addr learned my addr 172.21.15.46:0/2778490075 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f0796200640 1 --2- 172.21.15.46:0/2778490075 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbacee20 0x55acdbacf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f0794e00640 1 -- 172.21.15.46:0/2778490075 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbacee20 msgr2=0x55acdbacf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f0794e00640 1 --2- 172.21.15.46:0/2778490075 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbacee20 0x55acdbacf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f0794e00640 1 -- 172.21.15.46:0/2778490075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbb18cc0 msgr2=0x55acdbb190a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f0794e00640 1 --2- 172.21.15.46:0/2778490075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbb18cc0 0x55acdbb190a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f0794e00640 1 -- 172.21.15.46:0/2778490075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acdb615e20 con 0x55acdbb195e0 2024-03-29T08:29:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f0795800640 1 --2- 172.21.15.46:0/2778490075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbb18cc0 0x55acdbb190a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:29:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f0794e00640 1 --2- 172.21.15.46:0/2778490075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbace8e0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f077c009b10 tx=0x7f077c0101f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf0f143151eb915 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:38.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f0782000640 1 -- 172.21.15.46:0/2778490075 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c010da0 con 0x55acdbb195e0 2024-03-29T08:29:38.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f0782000640 1 -- 172.21.15.46:0/2778490075 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077c00c050 con 0x55acdbb195e0 2024-03-29T08:29:38.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.717+0000 7f0782000640 1 -- 172.21.15.46:0/2778490075 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c015670 con 0x55acdbb195e0 2024-03-29T08:29:38.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 -- 172.21.15.46:0/2778490075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 msgr2=0x55acdbace8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:38.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 --2- 172.21.15.46:0/2778490075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbace8e0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f077c009b10 tx=0x7f077c0101f0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 -- 172.21.15.46:0/2778490075 shutdown_connections 2024-03-29T08:29:38.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 --2- 172.21.15.46:0/2778490075 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbacee20 0x55acdbacf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 --2- 172.21.15.46:0/2778490075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbace8e0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 --2- 172.21.15.46:0/2778490075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbb18cc0 0x55acdbb190a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 -- 172.21.15.46:0/2778490075 >> 172.21.15.46:0/2778490075 conn(0x55acdb8c9160 msgr2=0x55acdbad1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:38.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 -- 172.21.15.46:0/2778490075 shutdown_connections 2024-03-29T08:29:38.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 -- 172.21.15.46:0/2778490075 wait complete. 2024-03-29T08:29:38.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 -- start start 2024-03-29T08:29:38.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 Processor -- start 2024-03-29T08:29:38.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbacee20 0x55acdbada230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbb18cc0 0x55acdbada770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbadacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdb919a90 con 0x55acdbb195e0 2024-03-29T08:29:38.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdb8b8d40 con 0x55acdbb18cc0 2024-03-29T08:29:38.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acdb8d6f30 con 0x55acdbacee20 2024-03-29T08:29:38.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 10 client.0 _ll_register_callbacks cb 0x55acdb97b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:29:38.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0796200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbadacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0794e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbb18cc0 0x55acdbada770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0796200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbadacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60028/0 (socket says 172.21.15.46:60028) 2024-03-29T08:29:38.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0796200640 1 -- 172.21.15.46:0/1771387090 learned_addr learned my addr 172.21.15.46:0/1771387090 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0795800640 1 --2- 172.21.15.46:0/1771387090 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbacee20 0x55acdbada230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0796200640 1 -- 172.21.15.46:0/1771387090 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbacee20 msgr2=0x55acdbada230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0796200640 1 --2- 172.21.15.46:0/1771387090 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbacee20 0x55acdbada230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0796200640 1 -- 172.21.15.46:0/1771387090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbb18cc0 msgr2=0x55acdbada770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0796200640 1 --2- 172.21.15.46:0/1771387090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbb18cc0 0x55acdbada770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0796200640 1 -- 172.21.15.46:0/1771387090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0784004c80 con 0x55acdbb195e0 2024-03-29T08:29:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0794e00640 1 --2- 172.21.15.46:0/1771387090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbb18cc0 0x55acdbada770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:29:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0796200640 1 --2- 172.21.15.46:0/1771387090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbadacb0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f07880095c0 tx=0x7f07880f0d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbd4806149295f3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0776a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:29:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0776a00640 1 -- 172.21.15.46:0/1771387090 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07880ecca0 con 0x55acdbb195e0 2024-03-29T08:29:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0776a00640 1 -- 172.21.15.46:0/1771387090 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07880ece00 con 0x55acdbb195e0 2024-03-29T08:29:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 -- 172.21.15.46:0/1771387090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acdb9183e0 con 0x55acdbb195e0 2024-03-29T08:29:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0776a00640 1 -- 172.21.15.46:0/1771387090 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f078811f0b0 con 0x55acdbb195e0 2024-03-29T08:29:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 10 client.9345 fetch_fsmap learned FSMap version 94 2024-03-29T08:29:38.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 -- 172.21.15.46:0/1771387090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x7f077c009740 con 0x55acdbb195e0 2024-03-29T08:29:38.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0776a00640 1 -- 172.21.15.46:0/1771387090 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078811b080 con 0x55acdbb195e0 2024-03-29T08:29:38.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f0776a00640 1 -- 172.21.15.46:0/1771387090 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 94) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0788005d20 con 0x55acdbb195e0 2024-03-29T08:29:38.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 10 client.9345 fetch_fsmap finished waiting for FSMap version 94 2024-03-29T08:29:38.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 10 client.9345 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-03-29T08:29:38.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 20 client.9345 populate_metadata read hostname 'smithi046' 2024-03-29T08:29:38.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 --2- 172.21.15.46:0/1771387090 >> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] conn(0x55acdbadb6c0 0x55acdbb2ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 4 client.9345 mds_command: new command op to 9066 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:29:38.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.721+0000 7f079b6c3000 1 -- 172.21.15.46:0/1771387090 --> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acdb8e9d00 con 0x55acdbadb6c0 2024-03-29T08:29:38.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f0795800640 1 --2- 172.21.15.46:0/1771387090 >> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] conn(0x55acdbadb6c0 0x55acdbb2ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f0795800640 1 --2- 172.21.15.46:0/1771387090 >> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] conn(0x55acdbadb6c0 0x55acdbb2ad70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d613d5d5882b7c4b server_cookie=c658bd94bead265e in_seq=0 out_seq=0 2024-03-29T08:29:38.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f0776a00640 10 client.9345 ms_handle_connect on v2:172.21.15.46:6834/929698978 2024-03-29T08:29:38.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f0776a00640 1 -- 172.21.15.46:0/1771387090 <== mds.0 v2:172.21.15.46:6834/929698978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55acdb8e9d00 con 0x55acdbadb6c0 2024-03-29T08:29:38.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f0776a00640 10 client.9345 handle_command_reply: tid=0 2024-03-29T08:29:38.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f079b6c3000 1 client.9345 shutdown 2024-03-29T08:29:38.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f079b6c3000 1 -- 172.21.15.46:0/1771387090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 msgr2=0x55acdbadacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:38.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f079b6c3000 1 --2- 172.21.15.46:0/1771387090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbadacb0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f07880095c0 tx=0x7f07880f0d50 comp rx=0 tx=0).stop 2024-03-29T08:29:38.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f079b6c3000 1 -- 172.21.15.46:0/1771387090 shutdown_connections 2024-03-29T08:29:38.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f079b6c3000 1 --2- 172.21.15.46:0/1771387090 >> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] conn(0x55acdbadb6c0 0x55acdbb2ad70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f079b6c3000 1 --2- 172.21.15.46:0/1771387090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbb18cc0 0x55acdbada770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f079b6c3000 1 --2- 172.21.15.46:0/1771387090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbadacb0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f079b6c3000 1 --2- 172.21.15.46:0/1771387090 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbacee20 0x55acdbada230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f0776a00640 0 client.9345 ms_handle_reset on v2:172.21.15.46:6834/929698978 2024-03-29T08:29:38.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f079b6c3000 1 -- 172.21.15.46:0/1771387090 >> 172.21.15.46:0/1771387090 conn(0x55acdb8c9160 msgr2=0x55acdbacb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:38.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f079b6c3000 1 -- 172.21.15.46:0/1771387090 shutdown_connections 2024-03-29T08:29:38.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f079b6c3000 1 -- 172.21.15.46:0/1771387090 wait complete. 2024-03-29T08:29:38.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.725+0000 7f079b6c3000 20 client.9345 trim_cache size 0 max 16384 2024-03-29T08:29:38.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.737+0000 7f079b6c3000 1 Processor -- start 2024-03-29T08:29:38.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.737+0000 7f079b6c3000 1 -- start start 2024-03-29T08:29:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.737+0000 7f079b6c3000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbacee20 0x55acdbb16c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.737+0000 7f079b6c3000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbb18cc0 0x55acdbb171c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.737+0000 7f079b6c3000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbb28d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.737+0000 7f079b6c3000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdb5d6aa0 con 0x55acdbb195e0 2024-03-29T08:29:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.737+0000 7f079b6c3000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdb971920 con 0x55acdbb18cc0 2024-03-29T08:29:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.737+0000 7f079b6c3000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acdbac2930 con 0x55acdbacee20 2024-03-29T08:29:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.737+0000 7f0796200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbb28d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.737+0000 7f0796200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbb28d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60038/0 (socket says 172.21.15.46:60038) 2024-03-29T08:29:38.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.737+0000 7f0796200640 1 -- 172.21.15.46:0/3446188036 learned_addr learned my addr 172.21.15.46:0/3446188036 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:38.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0795800640 1 --2- 172.21.15.46:0/3446188036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbacee20 0x55acdbb16c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0794e00640 1 --2- 172.21.15.46:0/3446188036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbb18cc0 0x55acdbb171c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0796200640 1 -- 172.21.15.46:0/3446188036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbacee20 msgr2=0x55acdbb16c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:38.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0796200640 1 --2- 172.21.15.46:0/3446188036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbacee20 0x55acdbb16c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0796200640 1 -- 172.21.15.46:0/3446188036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbb18cc0 msgr2=0x55acdbb171c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:38.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0796200640 1 --2- 172.21.15.46:0/3446188036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbb18cc0 0x55acdbb171c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0796200640 1 -- 172.21.15.46:0/3446188036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acdb8deaa0 con 0x55acdbb195e0 2024-03-29T08:29:38.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0796200640 1 --2- 172.21.15.46:0/3446188036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbb28d60 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f07880f8450 tx=0x7f07880f2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29641645ae54ae58 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:38.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0782a00640 1 -- 172.21.15.46:0/3446188036 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07880ecca0 con 0x55acdbb195e0 2024-03-29T08:29:38.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0782a00640 1 -- 172.21.15.46:0/3446188036 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55acdb9183e0 con 0x55acdbb195e0 2024-03-29T08:29:38.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0782a00640 1 -- 172.21.15.46:0/3446188036 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078811f2b0 con 0x55acdbb195e0 2024-03-29T08:29:38.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 1 -- 172.21.15.46:0/3446188036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 msgr2=0x55acdbb28d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:38.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 1 --2- 172.21.15.46:0/3446188036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbb28d60 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f07880f8450 tx=0x7f07880f2000 comp rx=0 tx=0).stop 2024-03-29T08:29:38.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 1 -- 172.21.15.46:0/3446188036 shutdown_connections 2024-03-29T08:29:38.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 1 --2- 172.21.15.46:0/3446188036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbb18cc0 0x55acdbb171c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 1 --2- 172.21.15.46:0/3446188036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbb28d60 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 1 --2- 172.21.15.46:0/3446188036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbacee20 0x55acdbb16c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 1 -- 172.21.15.46:0/3446188036 >> 172.21.15.46:0/3446188036 conn(0x55acdb8c9160 msgr2=0x55acdbb262b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:38.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 1 -- 172.21.15.46:0/3446188036 shutdown_connections 2024-03-29T08:29:38.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 1 -- 172.21.15.46:0/3446188036 wait complete. 2024-03-29T08:29:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 1 -- start start 2024-03-29T08:29:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 1 Processor -- start 2024-03-29T08:29:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbacee20 0x55acdbb78570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbb18cc0 0x55acdbb78ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbb2b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdbad7a10 con 0x55acdbb195e0 2024-03-29T08:29:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdb8e90b0 con 0x55acdbacee20 2024-03-29T08:29:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acdb91b020 con 0x55acdbb18cc0 2024-03-29T08:29:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f079b6c3000 10 client.0 _ll_register_callbacks cb 0x7f077c0093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:29:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0796200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbb2b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0796200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbb2b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60052/0 (socket says 172.21.15.46:60052) 2024-03-29T08:29:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0796200640 1 -- 172.21.15.46:0/2666329000 learned_addr learned my addr 172.21.15.46:0/2666329000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0795800640 1 --2- 172.21.15.46:0/2666329000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbacee20 0x55acdbb78570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0794e00640 1 --2- 172.21.15.46:0/2666329000 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbb18cc0 0x55acdbb78ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0796200640 1 -- 172.21.15.46:0/2666329000 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbb18cc0 msgr2=0x55acdbb78ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0796200640 1 --2- 172.21.15.46:0/2666329000 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbb18cc0 0x55acdbb78ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0796200640 1 -- 172.21.15.46:0/2666329000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbacee20 msgr2=0x55acdbb78570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0796200640 1 --2- 172.21.15.46:0/2666329000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbacee20 0x55acdbb78570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0796200640 1 -- 172.21.15.46:0/2666329000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0784004c80 con 0x55acdbb195e0 2024-03-29T08:29:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.741+0000 7f0794e00640 1 --2- 172.21.15.46:0/2666329000 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbb18cc0 0x55acdbb78ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f0796200640 1 --2- 172.21.15.46:0/2666329000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbb2b580 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f07880ee8e0 tx=0x7f078811d840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e08c5ebabf95be server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f0776a00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:29:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f0776a00640 1 -- 172.21.15.46:0/2666329000 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07880f86f0 con 0x55acdbb195e0 2024-03-29T08:29:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f0776a00640 1 -- 172.21.15.46:0/2666329000 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f078811d8b0 con 0x55acdbb195e0 2024-03-29T08:29:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f079b6c3000 1 -- 172.21.15.46:0/2666329000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acdb8d6970 con 0x55acdbb195e0 2024-03-29T08:29:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f0776a00640 1 -- 172.21.15.46:0/2666329000 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) v2 ==== 24+0+0 (secure 0 0 0) 0x7f078811f2c0 con 0x55acdbb195e0 2024-03-29T08:29:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f079b6c3000 10 client.9351 fetch_fsmap learned FSMap version 94 2024-03-29T08:29:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f079b6c3000 1 -- 172.21.15.46:0/2666329000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) v3 -- 0x55acdb8c4df0 con 0x55acdbb195e0 2024-03-29T08:29:38.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f0776a00640 1 -- 172.21.15.46:0/2666329000 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55acdb8d6970 con 0x55acdbb195e0 2024-03-29T08:29:38.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f0776a00640 1 -- 172.21.15.46:0/2666329000 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 94) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0788005d20 con 0x55acdbb195e0 2024-03-29T08:29:38.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f079b6c3000 10 client.9351 fetch_fsmap finished waiting for FSMap version 94 2024-03-29T08:29:38.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f079b6c3000 10 client.9351 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-03-29T08:29:38.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f079b6c3000 20 client.9351 populate_metadata read hostname 'smithi046' 2024-03-29T08:29:38.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f079b6c3000 1 --2- 172.21.15.46:0/2666329000 >> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] conn(0x55acdbb2bac0 0x55acdbb2bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f079b6c3000 4 client.9351 mds_command: new command op to 9066 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-29T08:29:38.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f079b6c3000 1 -- 172.21.15.46:0/2666329000 --> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55acdb948a20 con 0x55acdbb2bac0 2024-03-29T08:29:38.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f0795800640 1 --2- 172.21.15.46:0/2666329000 >> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] conn(0x55acdbb2bac0 0x55acdbb2bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f0795800640 1 --2- 172.21.15.46:0/2666329000 >> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] conn(0x55acdbb2bac0 0x55acdbb2bea0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b72a6095a1ca21e server_cookie=2f358607090f06bf in_seq=0 out_seq=0 2024-03-29T08:29:38.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.745+0000 7f0776a00640 10 client.9351 ms_handle_connect on v2:172.21.15.46:6834/929698978 2024-03-29T08:29:38.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f0776a00640 1 -- 172.21.15.46:0/2666329000 <== mds.0 v2:172.21.15.46:6834/929698978 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55acdb948a20 con 0x55acdbb2bac0 2024-03-29T08:29:38.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f0776a00640 10 client.9351 handle_command_reply: tid=0 2024-03-29T08:29:38.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f079b6c3000 1 client.9351 shutdown 2024-03-29T08:29:38.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f079b6c3000 1 -- 172.21.15.46:0/2666329000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 msgr2=0x55acdbb2b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:38.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f079b6c3000 1 --2- 172.21.15.46:0/2666329000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbb2b580 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f07880ee8e0 tx=0x7f078811d840 comp rx=0 tx=0).stop 2024-03-29T08:29:38.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f079b6c3000 1 -- 172.21.15.46:0/2666329000 shutdown_connections 2024-03-29T08:29:38.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f079b6c3000 1 --2- 172.21.15.46:0/2666329000 >> [v2:172.21.15.46:6834/929698978,v1:172.21.15.46:6835/929698978] conn(0x55acdbb2bac0 0x55acdbb2bea0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f079b6c3000 1 --2- 172.21.15.46:0/2666329000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55acdbb195e0 0x55acdbb2b580 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f079b6c3000 1 --2- 172.21.15.46:0/2666329000 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55acdbb18cc0 0x55acdbb78ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f079b6c3000 1 --2- 172.21.15.46:0/2666329000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55acdbacee20 0x55acdbb78570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f0776a00640 0 client.9351 ms_handle_reset on v2:172.21.15.46:6834/929698978 2024-03-29T08:29:38.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f079b6c3000 1 -- 172.21.15.46:0/2666329000 >> 172.21.15.46:0/2666329000 conn(0x55acdb8c9160 msgr2=0x55acdbb15c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:38.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f079b6c3000 1 -- 172.21.15.46:0/2666329000 shutdown_connections 2024-03-29T08:29:38.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f079b6c3000 1 -- 172.21.15.46:0/2666329000 wait complete. 2024-03-29T08:29:38.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f079b6c3000 20 client.9351 trim_cache size 0 max 16384 2024-03-29T08:29:38.763 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:29:38.763 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "", 2024-03-29T08:29:38.763 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-03-29T08:29:38.763 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:29:38.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f0798200640 1 -- 172.21.15.46:0/261173384 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0770066690 msgr2=0x7f0770068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:38.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f0798200640 1 --2- 172.21.15.46:0/261173384 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0770066690 0x7f0770068b50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0784007fd0 tx=0x7f0784007d60 comp rx=0 tx=0).stop 2024-03-29T08:29:38.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f0798200640 1 -- 172.21.15.46:0/261173384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f079015efe0 msgr2=0x7f0790139a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:38.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f0798200640 1 --2- 172.21.15.46:0/261173384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f079015efe0 0x7f0790139a80 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f0788019510 tx=0x7f07880196a0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f0798200640 1 -- 172.21.15.46:0/261173384 shutdown_connections 2024-03-29T08:29:38.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f0798200640 1 --2- 172.21.15.46:0/261173384 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0770066690 0x7f0770068b50 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f0798200640 1 --2- 172.21.15.46:0/261173384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f079015efe0 0x7f0790139a80 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f0798200640 1 --2- 172.21.15.46:0/261173384 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0790143eb0 0x7f079013ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f0798200640 1 --2- 172.21.15.46:0/261173384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07900cbea0 0x7f079015dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f0798200640 1 -- 172.21.15.46:0/261173384 >> 172.21.15.46:0/261173384 conn(0x7f079006a390 msgr2=0x7f0790070180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:38.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f0798200640 1 -- 172.21.15.46:0/261173384 shutdown_connections 2024-03-29T08:29:38.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.757+0000 7f0798200640 1 -- 172.21.15.46:0/261173384 wait complete. 2024-03-29T08:29:38.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:29:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.917+0000 7fa9fd600640 1 Processor -- start 2024-03-29T08:29:38.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.917+0000 7fa9fd600640 1 -- start start 2024-03-29T08:29:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.917+0000 7fa9fd600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa9f815a900 0x7fa9f80b0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.917+0000 7fa9fd600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9f80d0030 0x7fa9f80b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.917+0000 7fa9fd600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f80b0ee0 con 0x7fa9f815e340 2024-03-29T08:29:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.917+0000 7fa9fd600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f815ace0 con 0x7fa9f80d0030 2024-03-29T08:29:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.917+0000 7fa9fd600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f80afcf0 con 0x7fa9f815a900 2024-03-29T08:29:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.921+0000 7fa9ee000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9f80d0030 0x7fa9f80b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.921+0000 7fa9ee000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa9f80d0030 0x7fa9f80b0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60056/0 (socket says 172.21.15.46:60056) 2024-03-29T08:29:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.921+0000 7fa9ee000640 1 -- 172.21.15.46:0/3050754770 learned_addr learned my addr 172.21.15.46:0/3050754770 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.921+0000 7fa9ee000640 1 -- 172.21.15.46:0/3050754770 >> v2:172.21.15.73:3300/0 conn(0x7fa9f815a900 msgr2=0x7fa9f80b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:38.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.921+0000 7fa9ee000640 1 --2- 172.21.15.46:0/3050754770 >> v2:172.21.15.73:3300/0 conn(0x7fa9f815a900 0x7fa9f80b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.921+0000 7fa9ee000640 1 -- 172.21.15.46:0/3050754770 >> v1:172.21.15.73:6789/0 conn(0x7fa9f815e340 legacy=0x7fa9f8069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:38.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.921+0000 7fa9ee000640 1 -- 172.21.15.46:0/3050754770 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f80aee00 con 0x7fa9f80d0030 2024-03-29T08:29:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.921+0000 7fa9ee000640 1 --2- 172.21.15.46:0/3050754770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f80d0030 0x7fa9f80b0740 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80cd0a0 tx=0x7fa9e001c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5451ac5c2ad043ae server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.921+0000 7fa9f6000640 1 -- 172.21.15.46:0/3050754770 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e00a57b0 con 0x7fa9f80d0030 2024-03-29T08:29:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.921+0000 7fa9f6000640 1 -- 172.21.15.46:0/3050754770 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e00a5910 con 0x7fa9f80d0030 2024-03-29T08:29:38.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.921+0000 7fa9f6000640 1 -- 172.21.15.46:0/3050754770 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e00a5ca0 con 0x7fa9f80d0030 2024-03-29T08:29:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9fd600640 1 -- 172.21.15.46:0/3050754770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f80d0030 msgr2=0x7fa9f80b0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9fd600640 1 --2- 172.21.15.46:0/3050754770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f80d0030 0x7fa9f80b0740 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fa9f80cd0a0 tx=0x7fa9e001c8f0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9fd600640 1 -- 172.21.15.46:0/3050754770 shutdown_connections 2024-03-29T08:29:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9fd600640 1 --2- 172.21.15.46:0/3050754770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f80d0030 0x7fa9f80b0740 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9fd600640 1 --2- 172.21.15.46:0/3050754770 >> v2:172.21.15.73:3300/0 conn(0x7fa9f815a900 0x7fa9f80b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9fd600640 1 -- 172.21.15.46:0/3050754770 >> 172.21.15.46:0/3050754770 conn(0x7fa9f8098a40 msgr2=0x7fa9f80cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9fd600640 1 -- 172.21.15.46:0/3050754770 shutdown_connections 2024-03-29T08:29:38.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9fd600640 1 -- 172.21.15.46:0/3050754770 wait complete. 2024-03-29T08:29:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9fd600640 1 Processor -- start 2024-03-29T08:29:38.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9fd600640 1 -- start start 2024-03-29T08:29:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9fd600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f80d0030 0x7fa9f81564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9fd600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9f815a900 0x7fa9f80c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9fd600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9f815e340 0x7fa9f80c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9fd600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f81f5160 con 0x7fa9f80d0030 2024-03-29T08:29:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9fd600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f80c3350 con 0x7fa9f815a900 2024-03-29T08:29:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9fd600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f8156a20 con 0x7fa9f815e340 2024-03-29T08:29:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9f6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f80d0030 0x7fa9f81564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9f6a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f80d0030 0x7fa9f81564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60064/0 (socket says 172.21.15.46:60064) 2024-03-29T08:29:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9f6a00640 1 -- 172.21.15.46:0/3558527555 learned_addr learned my addr 172.21.15.46:0/3558527555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9ee000640 1 --2- 172.21.15.46:0/3558527555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9f815a900 0x7fa9f80c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9f7400640 1 --2- 172.21.15.46:0/3558527555 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9f815e340 0x7fa9f80c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9f6a00640 1 -- 172.21.15.46:0/3558527555 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9f815e340 msgr2=0x7fa9f80c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9f6a00640 1 --2- 172.21.15.46:0/3558527555 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9f815e340 0x7fa9f80c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9f6a00640 1 -- 172.21.15.46:0/3558527555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9f815a900 msgr2=0x7fa9f80c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9f6a00640 1 --2- 172.21.15.46:0/3558527555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9f815a900 0x7fa9f80c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:38.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9f6a00640 1 -- 172.21.15.46:0/3558527555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e0009b30 con 0x7fa9f80d0030 2024-03-29T08:29:38.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9f6a00640 1 --2- 172.21.15.46:0/3558527555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f80d0030 0x7fa9f81564e0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800b410 tx=0x7fa9e800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:38.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.925+0000 7fa9ef400640 1 -- 172.21.15.46:0/3558527555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8011830 con 0x7fa9f80d0030 2024-03-29T08:29:38.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.929+0000 7fa9ef400640 1 -- 172.21.15.46:0/3558527555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e8011990 con 0x7fa9f80d0030 2024-03-29T08:29:38.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.929+0000 7fa9fd600640 1 -- 172.21.15.46:0/3558527555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f81f8c20 con 0x7fa9f80d0030 2024-03-29T08:29:38.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.929+0000 7fa9ef400640 1 -- 172.21.15.46:0/3558527555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8011d40 con 0x7fa9f80d0030 2024-03-29T08:29:38.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.929+0000 7fa9fd600640 1 -- 172.21.15.46:0/3558527555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f81f91e0 con 0x7fa9f80d0030 2024-03-29T08:29:38.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.929+0000 7fa9fd600640 1 -- 172.21.15.46:0/3558527555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f8066c90 con 0x7fa9f80d0030 2024-03-29T08:29:38.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.929+0000 7fa9ef400640 1 -- 172.21.15.46:0/3558527555 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa9e801a400 con 0x7fa9f80d0030 2024-03-29T08:29:38.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.929+0000 7fa9ef400640 1 --2- 172.21.15.46:0/3558527555 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa9d4066690 0x7fa9d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:38.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.929+0000 7fa9ef400640 1 -- 172.21.15.46:0/3558527555 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9e80843f0 con 0x7fa9f80d0030 2024-03-29T08:29:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.929+0000 7fa9ee000640 1 --2- 172.21.15.46:0/3558527555 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa9d4066690 0x7fa9d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:38.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.933+0000 7fa9ee000640 1 --2- 172.21.15.46:0/3558527555 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa9d4066690 0x7fa9d4068b50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00ae000 tx=0x7fa9e00a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:38.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:38.933+0000 7fa9ef400640 1 -- 172.21.15.46:0/3558527555 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa9e80516c0 con 0x7fa9f80d0030 2024-03-29T08:29:39.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:39.133+0000 7fa9fd600640 1 -- 172.21.15.46:0/3558527555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa9f806c240 con 0x7fa9f80d0030 2024-03-29T08:29:39.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:39.137+0000 7fa9ef400640 1 -- 172.21.15.46:0/3558527555 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v94) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa9e8015060 con 0x7fa9f80d0030 2024-03-29T08:29:39.141 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:29:39.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:39.137+0000 7fa9fd600640 1 -- 172.21.15.46:0/3558527555 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa9d4066690 msgr2=0x7fa9d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:39.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:39.137+0000 7fa9fd600640 1 --2- 172.21.15.46:0/3558527555 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa9d4066690 0x7fa9d4068b50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00ae000 tx=0x7fa9e00a7070 comp rx=0 tx=0).stop 2024-03-29T08:29:39.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:39.137+0000 7fa9fd600640 1 -- 172.21.15.46:0/3558527555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f80d0030 msgr2=0x7fa9f81564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:39.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:39.137+0000 7fa9fd600640 1 --2- 172.21.15.46:0/3558527555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f80d0030 0x7fa9f81564e0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800b410 tx=0x7fa9e800b8d0 comp rx=0 tx=0).stop 2024-03-29T08:29:39.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:39.137+0000 7fa9fd600640 1 -- 172.21.15.46:0/3558527555 shutdown_connections 2024-03-29T08:29:39.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:39.137+0000 7fa9fd600640 1 --2- 172.21.15.46:0/3558527555 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa9d4066690 0x7fa9d4068b50 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:39.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:39.137+0000 7fa9fd600640 1 --2- 172.21.15.46:0/3558527555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9f815a900 0x7fa9f80c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:39.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:39.137+0000 7fa9fd600640 1 --2- 172.21.15.46:0/3558527555 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa9f815e340 0x7fa9f80c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:39.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:39.137+0000 7fa9fd600640 1 --2- 172.21.15.46:0/3558527555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9f80d0030 0x7fa9f81564e0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:39.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:39.137+0000 7fa9fd600640 1 -- 172.21.15.46:0/3558527555 >> 172.21.15.46:0/3558527555 conn(0x7fa9f8098a40 msgr2=0x7fa9f8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:39.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:39.137+0000 7fa9fd600640 1 -- 172.21.15.46:0/3558527555 shutdown_connections 2024-03-29T08:29:39.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:39.137+0000 7fa9fd600640 1 -- 172.21.15.46:0/3558527555 wait complete. 2024-03-29T08:29:39.161 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:29:39.161 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:29:39.161 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-29T08:29:39.161 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T08:29:39.161 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:39.161 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:29:39.161 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:29:39.161 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:29:39.161 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:29:39.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:39.162 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-29T08:29:39.180 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:29:39.180 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:29:39.180 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:29:39.180 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:39.180 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:29:39.180 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:39.180 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:29:39.180 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:39.180 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-29T08:29:39.180 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:29:39.180 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:39.180 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-29T08:29:39.181 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:29:39.181 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:39.181 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-29T08:29:39.181 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T08:29:39.181 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41758sec preferred_lft 41758sec 2024-03-29T08:29:39.181 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-29T08:29:39.181 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:39.181 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:29:39.181 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:39.181 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-29T08:29:39.181 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:29:39.181 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:d9:58:09:4c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:39.181 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:29:39.181 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:39.181 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:29:39.182 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:39.182 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T08:29:39.182 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:39.182 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T08:29:39.182 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:39.182 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:29:39.182 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:29:39.182 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T08:29:39.182 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:39.182 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:29:39.182 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T08:29:39.182 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-29T08:29:39.182 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:39.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:39.183 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:29:39.242 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:29:39.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:39.244 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-29T08:29:39.298 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-29T08:29:39.298 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:29:39.298 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T08:29:39.298 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:29:39.298 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T08:29:39.299 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:29:39.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:39.299 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:29:39.299 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:29:39.299 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:29:39.299 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:29:39.299 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:29:39.299 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:29:39.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:39.389 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:29:39.398 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T08:29:39.399 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T08:29:39.399 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:29:39.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:39.399 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-03-29T08:29:39.462 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T08:29:39.462 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:29:39.463 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:39.511 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:29:39.512 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:29:39.512 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:39.563 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:29:39.623 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:29:39.624 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:29:39.685 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:29:39.687 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:29:39.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:39.688 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:29:39.739 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:29:39.739 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T08:29:39.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:29:39.798 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:29:39.803 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:29:39.803 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:29:39.845 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:29:39.863 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:29:39.865 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:29:39.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:39.865 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:29:39.877 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-03-29T08:29:39.873+0000 7fa5107fa580 -1 init, newargv = 0x558a2ffba250 newargc=15 2024-03-29T08:29:39.877 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[51330]: starting ceph client 2024-03-29T08:29:39.883 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[51330]: starting fuse 2024-03-29T08:29:39.886 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:29:39.886 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-29T08:29:39.887 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:39.887 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-03-29T08:29:39.887 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-03-29T08:29:39.887 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:39.887 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-03-29T08:29:39.887 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:39.887 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-03-29T08:29:39.887 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T08:29:39.887 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-03-29T08:29:39.887 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:29:39.887 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:39.887 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-03-29T08:29:39.887 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-03-29T08:29:39.888 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-03-29T08:29:39.888 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-03-29T08:29:39.888 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:39.888 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-03-29T08:29:39.888 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:29:39.888 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:29:39.888 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:29:39.888 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-03-29T08:29:39.888 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-03-29T08:29:39.888 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-03-29T08:29:39.888 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:29:39.888 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:39.888 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-03-29T08:29:39.888 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:40.017 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.51330.asok 2024-03-29T08:29:40.199 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51330.asok 2024-03-29T08:29:40.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:40.199 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51330.asok status 2024-03-29T08:29:40.376 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:29:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-03-29T08:29:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:29:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:29:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-03-29T08:29:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-03-29T08:29:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T08:29:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "51330", 2024-03-29T08:29:40.376 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-03-29T08:29:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-03-29T08:29:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-03-29T08:29:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 9363, 2024-03-29T08:29:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-03-29T08:29:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-03-29T08:29:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-03-29T08:29:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 9363 2024-03-29T08:29:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:29:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:29:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:29:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2254978976 2024-03-29T08:29:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:40.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:29:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:29:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:29:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2254978976 2024-03-29T08:29:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.9363 192.168.0.1:0/2254978976", 2024-03-29T08:29:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/2254978976", 2024-03-29T08:29:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-03-29T08:29:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 94, 2024-03-29T08:29:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 121, 2024-03-29T08:29:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:29:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-03-29T08:29:40.378 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-03-29T08:29:40.378 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:29:40.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:40.391 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-03-29T08:29:40.391 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:40.400 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-03-29T08:29:40.400 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:40.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:40.400 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:40.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:40.464 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-03-29T08:29:40.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:40.522 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-03-29T08:29:40.637 INFO:teuthology.orchestra.run.smithi046.stderr:8+0 records in 2024-03-29T08:29:40.637 INFO:teuthology.orchestra.run.smithi046.stderr:8+0 records out 2024-03-29T08:29:40.637 INFO:teuthology.orchestra.run.smithi046.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0635322 s, 132 MB/s 2024-03-29T08:29:40.638 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:40.638 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:40.638 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:29:40.638 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:40.638 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-03-29T08:29:40.639 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:40.706 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627779 2024-03-29T08:29:40.895 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:40.895 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:40.896 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:29:40.896 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:40.896 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-03-29T08:29:40.896 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:41.004 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627780 2024-03-29T08:29:41.194 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:29:41.194 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:29:41.244 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:29:41.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.421+0000 7f8cf6600640 1 Processor -- start 2024-03-29T08:29:41.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.421+0000 7f8cf6600640 1 -- start start 2024-03-29T08:29:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.425+0000 7f8cf6600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8cf0151880 0x7f8cf006b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.425+0000 7f8cf6600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8cf00ccc50 0x7f8cf00709f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.425+0000 7f8cf6600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cf006b760 con 0x7f8cf0070f30 2024-03-29T08:29:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.425+0000 7f8cf6600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cf00713c0 con 0x7f8cf00ccc50 2024-03-29T08:29:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.425+0000 7f8cf6600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf015bea0 con 0x7f8cf0151880 2024-03-29T08:29:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.425+0000 7f8cefe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8cf0070f30 0x7f8cf015e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43348/0 (socket says 172.21.15.46:43348) 2024-03-29T08:29:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.425+0000 7f8cefe00640 1 -- 172.21.15.46:0/1431352977 learned_addr learned my addr 172.21.15.46:0/1431352977 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.425+0000 7f8cef400640 1 --2- 172.21.15.46:0/1431352977 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8cf0151880 0x7f8cf006b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.425+0000 7f8ce7e00640 1 --2- 172.21.15.46:0/1431352977 >> v2:172.21.15.73:3300/0 conn(0x7f8cf00ccc50 0x7f8cf00709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.425+0000 7f8ceea00640 1 -- 172.21.15.46:0/1431352977 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24978985 0 0) 0x7f8ce80014a0 con 0x7f8cf0070f30 2024-03-29T08:29:41.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.425+0000 7f8ceea00640 1 -- 172.21.15.46:0/1431352977 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cd4003530 con 0x7f8cf0070f30 2024-03-29T08:29:41.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.425+0000 7f8cef400640 1 -- 172.21.15.46:0/1431352977 >> v2:172.21.15.73:3300/0 conn(0x7f8cf00ccc50 msgr2=0x7f8cf00709f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:41.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.425+0000 7f8cef400640 1 --2- 172.21.15.46:0/1431352977 >> v2:172.21.15.73:3300/0 conn(0x7f8cf00ccc50 0x7f8cf00709f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.425+0000 7f8cef400640 1 -- 172.21.15.46:0/1431352977 >> v1:172.21.15.46:6789/0 conn(0x7f8cf0070f30 legacy=0x7f8cf015e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:41.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.425+0000 7f8cef400640 1 -- 172.21.15.46:0/1431352977 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf015b290 con 0x7f8cf0151880 2024-03-29T08:29:41.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8cef400640 1 --2- 172.21.15.46:0/1431352977 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8cf0151880 0x7f8cf006b220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00097c0 tx=0x7f8ce001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f20377f7bb6aaef server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:41.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8ceea00640 1 -- 172.21.15.46:0/1431352977 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce00ab670 con 0x7f8cf0151880 2024-03-29T08:29:41.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8ceea00640 1 -- 172.21.15.46:0/1431352977 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce00ab7d0 con 0x7f8cf0151880 2024-03-29T08:29:41.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8ceea00640 1 -- 172.21.15.46:0/1431352977 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce00abb40 con 0x7f8cf0151880 2024-03-29T08:29:41.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8cf6600640 1 -- 172.21.15.46:0/1431352977 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8cf0151880 msgr2=0x7f8cf006b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:41.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8cf6600640 1 --2- 172.21.15.46:0/1431352977 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8cf0151880 0x7f8cf006b220 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00097c0 tx=0x7f8ce001c470 comp rx=0 tx=0).stop 2024-03-29T08:29:41.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8cf6600640 1 -- 172.21.15.46:0/1431352977 shutdown_connections 2024-03-29T08:29:41.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8cf6600640 1 --2- 172.21.15.46:0/1431352977 >> v2:172.21.15.73:3300/0 conn(0x7f8cf00ccc50 0x7f8cf00709f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8cf6600640 1 --2- 172.21.15.46:0/1431352977 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8cf0151880 0x7f8cf006b220 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8cf6600640 1 -- 172.21.15.46:0/1431352977 >> 172.21.15.46:0/1431352977 conn(0x7f8cf00ced50 msgr2=0x7f8cf00cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:41.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8cf6600640 1 -- 172.21.15.46:0/1431352977 shutdown_connections 2024-03-29T08:29:41.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8cf6600640 1 -- 172.21.15.46:0/1431352977 wait complete. 2024-03-29T08:29:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8cf6600640 1 Processor -- start 2024-03-29T08:29:41.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8cf6600640 1 -- start start 2024-03-29T08:29:41.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8cf6600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cf0070f30 0x7f8cf0159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:41.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8cf6600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf00ccc50 0x7f8cf015fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:41.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8cf6600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8cf0151880 0x7f8cf01646a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:41.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8cf6600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf00b2300 con 0x7f8cf00ccc50 2024-03-29T08:29:41.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8cf6600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf01596a0 con 0x7f8cf0070f30 2024-03-29T08:29:41.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.429+0000 7f8cf6600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf00b25a0 con 0x7f8cf0151880 2024-03-29T08:29:41.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8ce7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf00ccc50 0x7f8cf015fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:41.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8ce7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf00ccc50 0x7f8cf015fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60070/0 (socket says 172.21.15.46:60070) 2024-03-29T08:29:41.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8ce7e00640 1 -- 172.21.15.46:0/542110161 learned_addr learned my addr 172.21.15.46:0/542110161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:41.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8cef400640 1 --2- 172.21.15.46:0/542110161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cf0070f30 0x7f8cf0159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:41.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8cefe00640 1 --2- 172.21.15.46:0/542110161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8cf0151880 0x7f8cf01646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:41.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8ce7e00640 1 -- 172.21.15.46:0/542110161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8cf0151880 msgr2=0x7f8cf01646a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:41.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8ce7e00640 1 --2- 172.21.15.46:0/542110161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8cf0151880 0x7f8cf01646a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8ce7e00640 1 -- 172.21.15.46:0/542110161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cf0070f30 msgr2=0x7f8cf0159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:41.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8ce7e00640 1 --2- 172.21.15.46:0/542110161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cf0070f30 0x7f8cf0159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8ce7e00640 1 -- 172.21.15.46:0/542110161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce0009420 con 0x7f8cf00ccc50 2024-03-29T08:29:41.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8cefe00640 1 --2- 172.21.15.46:0/542110161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8cf0151880 0x7f8cf01646a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:41.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8ce7e00640 1 --2- 172.21.15.46:0/542110161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf00ccc50 0x7f8cf015fc90 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800e930 tx=0x7f8cd800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:41.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8ce7400640 1 -- 172.21.15.46:0/542110161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd8009690 con 0x7f8cf00ccc50 2024-03-29T08:29:41.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8ce7400640 1 -- 172.21.15.46:0/542110161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cd800ca00 con 0x7f8cf00ccc50 2024-03-29T08:29:41.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8cf6600640 1 -- 172.21.15.46:0/542110161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf00b2880 con 0x7f8cf00ccc50 2024-03-29T08:29:41.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8ce7400640 1 -- 172.21.15.46:0/542110161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cd800cd70 con 0x7f8cf00ccc50 2024-03-29T08:29:41.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8cf6600640 1 -- 172.21.15.46:0/542110161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf00b2e40 con 0x7f8cf00ccc50 2024-03-29T08:29:41.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8cf6600640 1 -- 172.21.15.46:0/542110161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf0066c90 con 0x7f8cf00ccc50 2024-03-29T08:29:41.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.433+0000 7f8ce7400640 1 -- 172.21.15.46:0/542110161 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8cd8013470 con 0x7f8cf00ccc50 2024-03-29T08:29:41.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.437+0000 7f8ce7400640 1 --2- 172.21.15.46:0/542110161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8ccc066690 0x7f8ccc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:41.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.437+0000 7f8ce7400640 1 -- 172.21.15.46:0/542110161 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8cd8084300 con 0x7f8cf00ccc50 2024-03-29T08:29:41.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.437+0000 7f8cef400640 1 --2- 172.21.15.46:0/542110161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8ccc066690 0x7f8ccc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:41.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.437+0000 7f8cef400640 1 --2- 172.21.15.46:0/542110161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8ccc066690 0x7f8ccc068b50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00af000 tx=0x7f8ce00a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:41.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.441+0000 7f8ce7400640 1 -- 172.21.15.46:0/542110161 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8cd8051870 con 0x7f8cf00ccc50 2024-03-29T08:29:41.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.601+0000 7f8cf6600640 1 -- 172.21.15.46:0/542110161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cf00cc5c0 con 0x7f8cf00ccc50 2024-03-29T08:29:41.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.601+0000 7f8ce7400640 1 -- 172.21.15.46:0/542110161 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16067 (secure 0 0 0) 0x7f8cd8054e90 con 0x7f8cf00ccc50 2024-03-29T08:29:41.608 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:41.608 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":121,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:29:15.473193+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:29:12.329121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-29T08:29:13.465939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:29:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.605+0000 7f8cf6600640 1 -- 172.21.15.46:0/542110161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8ccc066690 msgr2=0x7f8ccc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.605+0000 7f8cf6600640 1 --2- 172.21.15.46:0/542110161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8ccc066690 0x7f8ccc068b50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8ce00af000 tx=0x7f8ce00a7000 comp rx=0 tx=0).stop 2024-03-29T08:29:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.605+0000 7f8cf6600640 1 -- 172.21.15.46:0/542110161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf00ccc50 msgr2=0x7f8cf015fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.605+0000 7f8cf6600640 1 --2- 172.21.15.46:0/542110161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf00ccc50 0x7f8cf015fc90 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f8cd800e930 tx=0x7f8cd800edf0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.605+0000 7f8cf6600640 1 -- 172.21.15.46:0/542110161 shutdown_connections 2024-03-29T08:29:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.605+0000 7f8cf6600640 1 --2- 172.21.15.46:0/542110161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8ccc066690 0x7f8ccc068b50 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.605+0000 7f8cf6600640 1 --2- 172.21.15.46:0/542110161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8cf0151880 0x7f8cf01646a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.605+0000 7f8cf6600640 1 --2- 172.21.15.46:0/542110161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cf00ccc50 0x7f8cf015fc90 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.605+0000 7f8cf6600640 1 --2- 172.21.15.46:0/542110161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cf0070f30 0x7f8cf0159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.605+0000 7f8cf6600640 1 -- 172.21.15.46:0/542110161 >> 172.21.15.46:0/542110161 conn(0x7f8cf00ced50 msgr2=0x7f8cf015d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.605+0000 7f8cf6600640 1 -- 172.21.15.46:0/542110161 shutdown_connections 2024-03-29T08:29:41.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.605+0000 7f8cf6600640 1 -- 172.21.15.46:0/542110161 wait complete. 2024-03-29T08:29:41.628 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:29:12.329121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:29:13.465939+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:29:41.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:29:41.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61fe800640 1 Processor -- start 2024-03-29T08:29:41.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61fe800640 1 -- start start 2024-03-29T08:29:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61fe800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f61f8143eb0 0x7f61f80cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61fe800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61f815efe0 0x7f61f80cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61fe800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61f80b1630 0x7f61f80b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61fe800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f80b2360 con 0x7f61f80b1630 2024-03-29T08:29:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61fe800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f815e500 con 0x7f61f8143eb0 2024-03-29T08:29:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61fe800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f815e7e0 con 0x7f61f815efe0 2024-03-29T08:29:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61f7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61f80b1630 0x7f61f80b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61f7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f61f80b1630 0x7f61f80b1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60076/0 (socket says 172.21.15.46:60076) 2024-03-29T08:29:41.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61f7e00640 1 -- 172.21.15.46:0/3559891956 learned_addr learned my addr 172.21.15.46:0/3559891956 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61f7400640 1 --2- 172.21.15.46:0/3559891956 >> v2:172.21.15.73:3300/0 conn(0x7f61f8143eb0 0x7f61f80cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61eea00640 1 --2- 172.21.15.46:0/3559891956 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61f815efe0 0x7f61f80cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61f7e00640 1 -- 172.21.15.46:0/3559891956 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61f815efe0 msgr2=0x7f61f80cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61f7e00640 1 --2- 172.21.15.46:0/3559891956 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61f815efe0 0x7f61f80cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61f7e00640 1 -- 172.21.15.46:0/3559891956 >> v2:172.21.15.73:3300/0 conn(0x7f61f8143eb0 msgr2=0x7f61f80cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:41.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61f7e00640 1 --2- 172.21.15.46:0/3559891956 >> v2:172.21.15.73:3300/0 conn(0x7f61f8143eb0 0x7f61f80cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61f7e00640 1 -- 172.21.15.46:0/3559891956 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f815eac0 con 0x7f61f80b1630 2024-03-29T08:29:41.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.769+0000 7f61f7400640 1 --2- 172.21.15.46:0/3559891956 >> v2:172.21.15.73:3300/0 conn(0x7f61f8143eb0 0x7f61f80cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:41.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.773+0000 7f61f7e00640 1 --2- 172.21.15.46:0/3559891956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f80b1630 0x7f61f80b1a80 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f61f80b0b80 tx=0x7f61f001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06dfcc86828bfaf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:41.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.773+0000 7f61f6a00640 1 -- 172.21.15.46:0/3559891956 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f00a57b0 con 0x7f61f80b1630 2024-03-29T08:29:41.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.773+0000 7f61f6a00640 1 -- 172.21.15.46:0/3559891956 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f00a5910 con 0x7f61f80b1630 2024-03-29T08:29:41.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.773+0000 7f61f6a00640 1 -- 172.21.15.46:0/3559891956 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f00a5ca0 con 0x7f61f80b1630 2024-03-29T08:29:41.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.773+0000 7f61fe800640 1 -- 172.21.15.46:0/3559891956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f80b1630 msgr2=0x7f61f80b1a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:41.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.773+0000 7f61fe800640 1 --2- 172.21.15.46:0/3559891956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f80b1630 0x7f61f80b1a80 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f61f80b0b80 tx=0x7f61f001c9f0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.773+0000 7f61fe800640 1 -- 172.21.15.46:0/3559891956 shutdown_connections 2024-03-29T08:29:41.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.773+0000 7f61fe800640 1 --2- 172.21.15.46:0/3559891956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f80b1630 0x7f61f80b1a80 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.773+0000 7f61fe800640 1 --2- 172.21.15.46:0/3559891956 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61f815efe0 0x7f61f80cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.773+0000 7f61fe800640 1 --2- 172.21.15.46:0/3559891956 >> v2:172.21.15.73:3300/0 conn(0x7f61f8143eb0 0x7f61f80cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.773+0000 7f61fe800640 1 -- 172.21.15.46:0/3559891956 >> 172.21.15.46:0/3559891956 conn(0x7f61f806a390 msgr2=0x7f61f80cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.773+0000 7f61fe800640 1 -- 172.21.15.46:0/3559891956 shutdown_connections 2024-03-29T08:29:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.773+0000 7f61fe800640 1 -- 172.21.15.46:0/3559891956 wait complete. 2024-03-29T08:29:41.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61fe800640 1 Processor -- start 2024-03-29T08:29:41.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61fe800640 1 -- start start 2024-03-29T08:29:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61fe800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61f80b1630 0x7f61f8141d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:41.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61fe800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61f8143eb0 0x7f61f8139790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61fe800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f815efe0 0x7f61f813f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61fe800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f8142470 con 0x7f61f815efe0 2024-03-29T08:29:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61fe800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f8142750 con 0x7f61f80b1630 2024-03-29T08:29:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61fe800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f8142a30 con 0x7f61f8143eb0 2024-03-29T08:29:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f815efe0 0x7f61f813f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61f7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f815efe0 0x7f61f813f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60086/0 (socket says 172.21.15.46:60086) 2024-03-29T08:29:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61f7e00640 1 -- 172.21.15.46:0/2418219245 learned_addr learned my addr 172.21.15.46:0/2418219245 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61f7400640 1 --2- 172.21.15.46:0/2418219245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61f80b1630 0x7f61f8141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61eea00640 1 --2- 172.21.15.46:0/2418219245 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61f8143eb0 0x7f61f8139790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61f7e00640 1 -- 172.21.15.46:0/2418219245 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61f8143eb0 msgr2=0x7f61f8139790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61f7e00640 1 --2- 172.21.15.46:0/2418219245 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61f8143eb0 0x7f61f8139790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61f7e00640 1 -- 172.21.15.46:0/2418219245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61f80b1630 msgr2=0x7f61f8141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:41.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61f7e00640 1 --2- 172.21.15.46:0/2418219245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61f80b1630 0x7f61f8141d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61f7e00640 1 -- 172.21.15.46:0/2418219245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f0009b30 con 0x7f61f815efe0 2024-03-29T08:29:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61eea00640 1 --2- 172.21.15.46:0/2418219245 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61f8143eb0 0x7f61f8139790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61f7e00640 1 --2- 172.21.15.46:0/2418219245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f815efe0 0x7f61f813f140 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f61f00af040 tx=0x7f61f0019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61efe00640 1 -- 172.21.15.46:0/2418219245 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f00b8670 con 0x7f61f815efe0 2024-03-29T08:29:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61efe00640 1 -- 172.21.15.46:0/2418219245 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f00b87f0 con 0x7f61f815efe0 2024-03-29T08:29:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61efe00640 1 -- 172.21.15.46:0/2418219245 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61f00b8ba0 con 0x7f61f815efe0 2024-03-29T08:29:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61fe800640 1 -- 172.21.15.46:0/2418219245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f8142d10 con 0x7f61f815efe0 2024-03-29T08:29:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61fe800640 1 -- 172.21.15.46:0/2418219245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f813a590 con 0x7f61f815efe0 2024-03-29T08:29:41.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.777+0000 7f61fe800640 1 -- 172.21.15.46:0/2418219245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f8071a70 con 0x7f61f815efe0 2024-03-29T08:29:41.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.781+0000 7f61efe00640 1 -- 172.21.15.46:0/2418219245 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f61f00b8d00 con 0x7f61f815efe0 2024-03-29T08:29:41.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.781+0000 7f61efe00640 1 --2- 172.21.15.46:0/2418219245 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f61d4066690 0x7f61d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:41.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.781+0000 7f61efe00640 1 -- 172.21.15.46:0/2418219245 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61f011f6f0 con 0x7f61f815efe0 2024-03-29T08:29:41.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.781+0000 7f61f7400640 1 --2- 172.21.15.46:0/2418219245 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f61d4066690 0x7f61d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:41.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.781+0000 7f61f7400640 1 --2- 172.21.15.46:0/2418219245 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f61d4066690 0x7f61d4068b50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009950 tx=0x7f61e8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:41.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.785+0000 7f61efe00640 1 -- 172.21.15.46:0/2418219245 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f61f00ecb70 con 0x7f61f815efe0 2024-03-29T08:29:41.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.945+0000 7f61fe800640 1 -- 172.21.15.46:0/2418219245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61f813ce80 con 0x7f61f815efe0 2024-03-29T08:29:41.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.945+0000 7f61efe00640 1 -- 172.21.15.46:0/2418219245 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61f00f0190 con 0x7f61f815efe0 2024-03-29T08:29:41.950 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:29:41.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.949+0000 7f61fe800640 1 -- 172.21.15.46:0/2418219245 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f61d4066690 msgr2=0x7f61d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:41.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.949+0000 7f61fe800640 1 --2- 172.21.15.46:0/2418219245 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f61d4066690 0x7f61d4068b50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009950 tx=0x7f61e8008040 comp rx=0 tx=0).stop 2024-03-29T08:29:41.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.949+0000 7f61fe800640 1 -- 172.21.15.46:0/2418219245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f815efe0 msgr2=0x7f61f813f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:41.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.949+0000 7f61fe800640 1 --2- 172.21.15.46:0/2418219245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f815efe0 0x7f61f813f140 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f61f00af040 tx=0x7f61f0019790 comp rx=0 tx=0).stop 2024-03-29T08:29:41.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.949+0000 7f61fe800640 1 -- 172.21.15.46:0/2418219245 shutdown_connections 2024-03-29T08:29:41.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.949+0000 7f61fe800640 1 --2- 172.21.15.46:0/2418219245 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f61d4066690 0x7f61d4068b50 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.949+0000 7f61fe800640 1 --2- 172.21.15.46:0/2418219245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61f815efe0 0x7f61f813f140 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.949+0000 7f61fe800640 1 --2- 172.21.15.46:0/2418219245 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f61f8143eb0 0x7f61f8139790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.949+0000 7f61fe800640 1 --2- 172.21.15.46:0/2418219245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61f80b1630 0x7f61f8141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.949+0000 7f61fe800640 1 -- 172.21.15.46:0/2418219245 >> 172.21.15.46:0/2418219245 conn(0x7f61f806a390 msgr2=0x7f61f80708d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.949+0000 7f61fe800640 1 -- 172.21.15.46:0/2418219245 shutdown_connections 2024-03-29T08:29:41.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:41.949+0000 7f61fe800640 1 -- 172.21.15.46:0/2418219245 wait complete. 2024-03-29T08:29:41.971 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:29:42.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.109+0000 7f96f2200640 1 Processor -- start 2024-03-29T08:29:42.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.109+0000 7f96f2200640 1 -- start start 2024-03-29T08:29:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.113+0000 7f96f2200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96ec156f80 0x7f96ec15a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.113+0000 7f96f2200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f96ec0cc590 0x7f96ec0caa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.113+0000 7f96f2200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ec15dcb0 con 0x7f96ec06dc20 2024-03-29T08:29:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.113+0000 7f96f2200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ec15a9d0 con 0x7f96ec156f80 2024-03-29T08:29:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.113+0000 7f96f2200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ec15d0a0 con 0x7f96ec0cc590 2024-03-29T08:29:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.113+0000 7f96f0e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f96ec156f80 0x7f96ec15a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.113+0000 7f96e3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f96ec06dc20 0x7f96ec0cbe80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43360/0 (socket says 172.21.15.46:43360) 2024-03-29T08:29:42.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.113+0000 7f96e3e00640 1 -- 172.21.15.46:0/1556691238 learned_addr learned my addr 172.21.15.46:0/1556691238 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:42.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.113+0000 7f96f1800640 1 --2- 172.21.15.46:0/1556691238 >> v2:172.21.15.73:3300/0 conn(0x7f96ec0cc590 0x7f96ec0caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.113+0000 7f96f0e00640 1 -- 172.21.15.46:0/1556691238 >> v2:172.21.15.73:3300/0 conn(0x7f96ec0cc590 msgr2=0x7f96ec0caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:42.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.113+0000 7f96f0e00640 1 --2- 172.21.15.46:0/1556691238 >> v2:172.21.15.73:3300/0 conn(0x7f96ec0cc590 0x7f96ec0caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.113+0000 7f96f0e00640 1 -- 172.21.15.46:0/1556691238 >> v1:172.21.15.46:6789/0 conn(0x7f96ec06dc20 legacy=0x7f96ec0cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.113+0000 7f96f0e00640 1 -- 172.21.15.46:0/1556691238 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec15d380 con 0x7f96ec156f80 2024-03-29T08:29:42.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.113+0000 7f96f0e00640 1 --2- 172.21.15.46:0/1556691238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96ec156f80 0x7f96ec15a490 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f96ec06fec0 tx=0x7f96dc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c446b2fe6a0b113d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:42.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96ebe00640 1 -- 172.21.15.46:0/1556691238 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc0ab600 con 0x7f96ec156f80 2024-03-29T08:29:42.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96ebe00640 1 -- 172.21.15.46:0/1556691238 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96dc0a7050 con 0x7f96ec156f80 2024-03-29T08:29:42.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96ebe00640 1 -- 172.21.15.46:0/1556691238 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc0ab980 con 0x7f96ec156f80 2024-03-29T08:29:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f2200640 1 -- 172.21.15.46:0/1556691238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96ec156f80 msgr2=0x7f96ec15a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f2200640 1 --2- 172.21.15.46:0/1556691238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96ec156f80 0x7f96ec15a490 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f96ec06fec0 tx=0x7f96dc01c480 comp rx=0 tx=0).stop 2024-03-29T08:29:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f2200640 1 -- 172.21.15.46:0/1556691238 shutdown_connections 2024-03-29T08:29:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f2200640 1 --2- 172.21.15.46:0/1556691238 >> v2:172.21.15.73:3300/0 conn(0x7f96ec0cc590 0x7f96ec0caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f2200640 1 --2- 172.21.15.46:0/1556691238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96ec156f80 0x7f96ec15a490 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f2200640 1 -- 172.21.15.46:0/1556691238 >> 172.21.15.46:0/1556691238 conn(0x7f96ec0b0030 msgr2=0x7f96ec0b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f2200640 1 -- 172.21.15.46:0/1556691238 shutdown_connections 2024-03-29T08:29:42.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f2200640 1 -- 172.21.15.46:0/1556691238 wait complete. 2024-03-29T08:29:42.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f2200640 1 Processor -- start 2024-03-29T08:29:42.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f2200640 1 -- start start 2024-03-29T08:29:42.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f2200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96ec06dc20 0x7f96ec163f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f2200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96ec0cc590 0x7f96ec155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f2200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96ec156f80 0x7f96ec162450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f2200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec1563b0 con 0x7f96ec0cc590 2024-03-29T08:29:42.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f2200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec164540 con 0x7f96ec156f80 2024-03-29T08:29:42.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f2200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec164820 con 0x7f96ec06dc20 2024-03-29T08:29:42.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96ec0cc590 0x7f96ec155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96ec0cc590 0x7f96ec155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60114/0 (socket says 172.21.15.46:60114) 2024-03-29T08:29:42.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96e3e00640 1 -- 172.21.15.46:0/339169052 learned_addr learned my addr 172.21.15.46:0/339169052 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:42.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f0e00640 1 --2- 172.21.15.46:0/339169052 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96ec06dc20 0x7f96ec163f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.117+0000 7f96f1800640 1 --2- 172.21.15.46:0/339169052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96ec156f80 0x7f96ec162450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.121+0000 7f96e3e00640 1 -- 172.21.15.46:0/339169052 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96ec06dc20 msgr2=0x7f96ec163f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.121+0000 7f96e3e00640 1 --2- 172.21.15.46:0/339169052 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96ec06dc20 0x7f96ec163f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.121+0000 7f96e3e00640 1 -- 172.21.15.46:0/339169052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96ec156f80 msgr2=0x7f96ec162450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.121+0000 7f96e3e00640 1 --2- 172.21.15.46:0/339169052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96ec156f80 0x7f96ec162450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.121+0000 7f96e3e00640 1 -- 172.21.15.46:0/339169052 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc009420 con 0x7f96ec0cc590 2024-03-29T08:29:42.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.121+0000 7f96e3e00640 1 --2- 172.21.15.46:0/339169052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96ec0cc590 0x7f96ec155e70 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f96d400c500 tx=0x7f96d400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.121+0000 7f96e9600640 1 -- 172.21.15.46:0/339169052 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d4005cf0 con 0x7f96ec0cc590 2024-03-29T08:29:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.121+0000 7f96e9600640 1 -- 172.21.15.46:0/339169052 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d4005e50 con 0x7f96ec0cc590 2024-03-29T08:29:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.121+0000 7f96f2200640 1 -- 172.21.15.46:0/339169052 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ec164b00 con 0x7f96ec0cc590 2024-03-29T08:29:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.121+0000 7f96e9600640 1 -- 172.21.15.46:0/339169052 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d40072f0 con 0x7f96ec0cc590 2024-03-29T08:29:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.121+0000 7f96f2200640 1 -- 172.21.15.46:0/339169052 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ec1650c0 con 0x7f96ec0cc590 2024-03-29T08:29:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.121+0000 7f96f2200640 1 -- 172.21.15.46:0/339169052 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ec0cf1f0 con 0x7f96ec0cc590 2024-03-29T08:29:42.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.121+0000 7f96e9600640 1 -- 172.21.15.46:0/339169052 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f96d4007450 con 0x7f96ec0cc590 2024-03-29T08:29:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.121+0000 7f96e9600640 1 --2- 172.21.15.46:0/339169052 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f96c8066690 0x7f96c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.121+0000 7f96e9600640 1 -- 172.21.15.46:0/339169052 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96d40836a0 con 0x7f96ec0cc590 2024-03-29T08:29:42.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.121+0000 7f96f0e00640 1 --2- 172.21.15.46:0/339169052 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f96c8066690 0x7f96c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.125+0000 7f96f0e00640 1 --2- 172.21.15.46:0/339169052 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f96c8066690 0x7f96c8068b50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0ad000 tx=0x7f96dc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:42.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.125+0000 7f96e9600640 1 -- 172.21.15.46:0/339169052 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f96d4050ab0 con 0x7f96ec0cc590 2024-03-29T08:29:42.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.289+0000 7f96f2200640 1 -- 172.21.15.46:0/339169052 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96ec04e040 con 0x7f96ec0cc590 2024-03-29T08:29:42.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.289+0000 7f96e9600640 1 -- 172.21.15.46:0/339169052 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7f96d40540d0 con 0x7f96ec0cc590 2024-03-29T08:29:42.293 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:29:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.289+0000 7f96f2200640 1 -- 172.21.15.46:0/339169052 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f96c8066690 msgr2=0x7f96c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.289+0000 7f96f2200640 1 --2- 172.21.15.46:0/339169052 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f96c8066690 0x7f96c8068b50 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f96dc0ad000 tx=0x7f96dc008040 comp rx=0 tx=0).stop 2024-03-29T08:29:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.289+0000 7f96f2200640 1 -- 172.21.15.46:0/339169052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96ec0cc590 msgr2=0x7f96ec155e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.289+0000 7f96f2200640 1 --2- 172.21.15.46:0/339169052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96ec0cc590 0x7f96ec155e70 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f96d400c500 tx=0x7f96d400c9c0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.289+0000 7f96f2200640 1 -- 172.21.15.46:0/339169052 shutdown_connections 2024-03-29T08:29:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.289+0000 7f96f2200640 1 --2- 172.21.15.46:0/339169052 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f96c8066690 0x7f96c8068b50 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.289+0000 7f96f2200640 1 --2- 172.21.15.46:0/339169052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96ec156f80 0x7f96ec162450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.289+0000 7f96f2200640 1 --2- 172.21.15.46:0/339169052 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f96ec0cc590 0x7f96ec155e70 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.289+0000 7f96f2200640 1 --2- 172.21.15.46:0/339169052 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f96ec06dc20 0x7f96ec163f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.289+0000 7f96f2200640 1 -- 172.21.15.46:0/339169052 >> 172.21.15.46:0/339169052 conn(0x7f96ec0b0030 msgr2=0x7f96ec0b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:42.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.289+0000 7f96f2200640 1 -- 172.21.15.46:0/339169052 shutdown_connections 2024-03-29T08:29:42.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.289+0000 7f96f2200640 1 -- 172.21.15.46:0/339169052 wait complete. 2024-03-29T08:29:42.313 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:29:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.453+0000 7fdd31400640 1 Processor -- start 2024-03-29T08:29:42.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.453+0000 7fdd31400640 1 -- start start 2024-03-29T08:29:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.453+0000 7fdd31400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdd2c0cbea0 0x7fdd2c0b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.453+0000 7fdd31400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd2c15efe0 0x7fdd2c0b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.453+0000 7fdd31400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd2c0b20b0 con 0x7fdd2c143eb0 2024-03-29T08:29:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.453+0000 7fdd31400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd2c0b2360 con 0x7fdd2c15efe0 2024-03-29T08:29:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.453+0000 7fdd31400640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd2c15e500 con 0x7fdd2c0cbea0 2024-03-29T08:29:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.453+0000 7fdd2b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd2c15efe0 0x7fdd2c0b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.453+0000 7fdd2b200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdd2c15efe0 0x7fdd2c0b1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60118/0 (socket says 172.21.15.46:60118) 2024-03-29T08:29:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.453+0000 7fdd2b200640 1 -- 172.21.15.46:0/2250059021 learned_addr learned my addr 172.21.15.46:0/2250059021 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:42.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.453+0000 7fdd21e00640 1 --2- 172.21.15.46:0/2250059021 >> v2:172.21.15.73:3300/0 conn(0x7fdd2c0cbea0 0x7fdd2c0b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.453+0000 7fdd2b200640 1 -- 172.21.15.46:0/2250059021 >> v2:172.21.15.73:3300/0 conn(0x7fdd2c0cbea0 msgr2=0x7fdd2c0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:42.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.453+0000 7fdd2b200640 1 --2- 172.21.15.46:0/2250059021 >> v2:172.21.15.73:3300/0 conn(0x7fdd2c0cbea0 0x7fdd2c0b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.453+0000 7fdd2b200640 1 -- 172.21.15.46:0/2250059021 >> v1:172.21.15.73:6789/0 conn(0x7fdd2c143eb0 legacy=0x7fdd2c0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.453+0000 7fdd2b200640 1 -- 172.21.15.46:0/2250059021 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd2c15e7e0 con 0x7fdd2c15efe0 2024-03-29T08:29:42.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.457+0000 7fdd2b200640 1 --2- 172.21.15.46:0/2250059021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd2c15efe0 0x7fdd2c0b1b70 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c06ad40 tx=0x7fdd2401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fffa0598efd0c7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:42.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.457+0000 7fdd29e00640 1 -- 172.21.15.46:0/2250059021 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd240a5820 con 0x7fdd2c15efe0 2024-03-29T08:29:42.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.457+0000 7fdd29e00640 1 -- 172.21.15.46:0/2250059021 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd240a5980 con 0x7fdd2c15efe0 2024-03-29T08:29:42.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.457+0000 7fdd29e00640 1 -- 172.21.15.46:0/2250059021 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd240a5d10 con 0x7fdd2c15efe0 2024-03-29T08:29:42.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.457+0000 7fdd31400640 1 -- 172.21.15.46:0/2250059021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd2c15efe0 msgr2=0x7fdd2c0b1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.457+0000 7fdd31400640 1 --2- 172.21.15.46:0/2250059021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd2c15efe0 0x7fdd2c0b1b70 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c06ad40 tx=0x7fdd2401c9c0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.457+0000 7fdd31400640 1 -- 172.21.15.46:0/2250059021 shutdown_connections 2024-03-29T08:29:42.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.457+0000 7fdd31400640 1 --2- 172.21.15.46:0/2250059021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd2c15efe0 0x7fdd2c0b1b70 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.457+0000 7fdd31400640 1 --2- 172.21.15.46:0/2250059021 >> v2:172.21.15.73:3300/0 conn(0x7fdd2c0cbea0 0x7fdd2c0b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.457+0000 7fdd31400640 1 -- 172.21.15.46:0/2250059021 >> 172.21.15.46:0/2250059021 conn(0x7fdd2c06a390 msgr2=0x7fdd2c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:42.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.457+0000 7fdd31400640 1 -- 172.21.15.46:0/2250059021 shutdown_connections 2024-03-29T08:29:42.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.457+0000 7fdd31400640 1 -- 172.21.15.46:0/2250059021 wait complete. 2024-03-29T08:29:42.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd31400640 1 Processor -- start 2024-03-29T08:29:42.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd31400640 1 -- start start 2024-03-29T08:29:42.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd31400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd2c0cbea0 0x7fdd2c138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd31400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd2c143eb0 0x7fdd2c142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd31400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd2c15efe0 0x7fdd2c157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd31400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c13b290 con 0x7fdd2c15efe0 2024-03-29T08:29:42.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd31400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c141610 con 0x7fdd2c0cbea0 2024-03-29T08:29:42.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd31400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2c13bb60 con 0x7fdd2c143eb0 2024-03-29T08:29:42.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd2b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd2c15efe0 0x7fdd2c157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd21e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd2c143eb0 0x7fdd2c142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd2a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd2c0cbea0 0x7fdd2c138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd2b200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd2c15efe0 0x7fdd2c157b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60132/0 (socket says 172.21.15.46:60132) 2024-03-29T08:29:42.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd2b200640 1 -- 172.21.15.46:0/400433119 learned_addr learned my addr 172.21.15.46:0/400433119 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:42.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd21e00640 1 --2- 172.21.15.46:0/400433119 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd2c143eb0 0x7fdd2c142520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:38650/0 (socket says 172.21.15.46:38650) 2024-03-29T08:29:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd2b200640 1 -- 172.21.15.46:0/400433119 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd2c143eb0 msgr2=0x7fdd2c142520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd2b200640 1 --2- 172.21.15.46:0/400433119 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd2c143eb0 0x7fdd2c142520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd2b200640 1 -- 172.21.15.46:0/400433119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd2c0cbea0 msgr2=0x7fdd2c138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd2b200640 1 --2- 172.21.15.46:0/400433119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd2c0cbea0 0x7fdd2c138ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd2b200640 1 -- 172.21.15.46:0/400433119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd24009b30 con 0x7fdd2c15efe0 2024-03-29T08:29:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd2b200640 1 --2- 172.21.15.46:0/400433119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd2c15efe0 0x7fdd2c157b30 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fdd240af040 tx=0x7fdd24019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd23400640 1 -- 172.21.15.46:0/400433119 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd240b8670 con 0x7fdd2c15efe0 2024-03-29T08:29:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd23400640 1 -- 172.21.15.46:0/400433119 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd240b8810 con 0x7fdd2c15efe0 2024-03-29T08:29:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd23400640 1 -- 172.21.15.46:0/400433119 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd240b8bc0 con 0x7fdd2c15efe0 2024-03-29T08:29:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd31400640 1 -- 172.21.15.46:0/400433119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2c13ad90 con 0x7fdd2c15efe0 2024-03-29T08:29:42.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.461+0000 7fdd31400640 1 -- 172.21.15.46:0/400433119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2c13b780 con 0x7fdd2c15efe0 2024-03-29T08:29:42.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.465+0000 7fdd31400640 1 -- 172.21.15.46:0/400433119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd2c142240 con 0x7fdd2c15efe0 2024-03-29T08:29:42.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.465+0000 7fdd23400640 1 -- 172.21.15.46:0/400433119 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fdd240b8d20 con 0x7fdd2c15efe0 2024-03-29T08:29:42.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.465+0000 7fdd23400640 1 --2- 172.21.15.46:0/400433119 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd08066690 0x7fdd08068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.465+0000 7fdd23400640 1 -- 172.21.15.46:0/400433119 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd2411f490 con 0x7fdd2c15efe0 2024-03-29T08:29:42.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.465+0000 7fdd2a800640 1 --2- 172.21.15.46:0/400433119 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd08066690 0x7fdd08068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.465+0000 7fdd2a800640 1 --2- 172.21.15.46:0/400433119 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd08066690 0x7fdd08068b50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0cc540 tx=0x7fdd1c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:42.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.469+0000 7fdd23400640 1 -- 172.21.15.46:0/400433119 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fdd240ec940 con 0x7fdd2c15efe0 2024-03-29T08:29:42.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.629+0000 7fdd31400640 1 -- 172.21.15.46:0/400433119 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd2c0ccbc0 con 0x7fdd2c15efe0 2024-03-29T08:29:42.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.629+0000 7fdd23400640 1 -- 172.21.15.46:0/400433119 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd240eff60 con 0x7fdd2c15efe0 2024-03-29T08:29:42.635 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:29:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.633+0000 7fdd31400640 1 -- 172.21.15.46:0/400433119 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd08066690 msgr2=0x7fdd08068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.633+0000 7fdd31400640 1 --2- 172.21.15.46:0/400433119 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd08066690 0x7fdd08068b50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c0cc540 tx=0x7fdd1c009040 comp rx=0 tx=0).stop 2024-03-29T08:29:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.633+0000 7fdd31400640 1 -- 172.21.15.46:0/400433119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd2c15efe0 msgr2=0x7fdd2c157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.633+0000 7fdd31400640 1 --2- 172.21.15.46:0/400433119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd2c15efe0 0x7fdd2c157b30 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fdd240af040 tx=0x7fdd24019760 comp rx=0 tx=0).stop 2024-03-29T08:29:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.633+0000 7fdd31400640 1 -- 172.21.15.46:0/400433119 shutdown_connections 2024-03-29T08:29:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.633+0000 7fdd31400640 1 --2- 172.21.15.46:0/400433119 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fdd08066690 0x7fdd08068b50 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.633+0000 7fdd31400640 1 --2- 172.21.15.46:0/400433119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd2c15efe0 0x7fdd2c157b30 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.633+0000 7fdd31400640 1 --2- 172.21.15.46:0/400433119 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fdd2c143eb0 0x7fdd2c142520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.633+0000 7fdd31400640 1 --2- 172.21.15.46:0/400433119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd2c0cbea0 0x7fdd2c138ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.633+0000 7fdd31400640 1 -- 172.21.15.46:0/400433119 >> 172.21.15.46:0/400433119 conn(0x7fdd2c06a390 msgr2=0x7fdd2c06db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.633+0000 7fdd31400640 1 -- 172.21.15.46:0/400433119 shutdown_connections 2024-03-29T08:29:42.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.633+0000 7fdd31400640 1 -- 172.21.15.46:0/400433119 wait complete. 2024-03-29T08:29:42.656 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:29:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.793+0000 7f1006200640 1 Processor -- start 2024-03-29T08:29:42.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.793+0000 7f1006200640 1 -- start start 2024-03-29T08:29:42.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.793+0000 7f1006200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f100015a810 0x7f10000b0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.793+0000 7f1006200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f100015e2c0 0x7f10000b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.793+0000 7f1006200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10000afbf0 con 0x7f10000c6080 2024-03-29T08:29:42.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.793+0000 7f1006200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100015abf0 con 0x7f100015a810 2024-03-29T08:29:42.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.793+0000 7f1006200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10000aeb50 con 0x7f100015e2c0 2024-03-29T08:29:42.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.793+0000 7f1005800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f100015e2c0 0x7f10000b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.793+0000 7f1005800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f100015e2c0 0x7f10000b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:38660/0 (socket says 172.21.15.46:38660) 2024-03-29T08:29:42.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.793+0000 7f0ff7e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f10000c6080 0x7f1000069d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:56022/0 (socket says 172.21.15.46:56022) 2024-03-29T08:29:42.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.793+0000 7f1005800640 1 -- 172.21.15.46:0/2812842961 learned_addr learned my addr 172.21.15.46:0/2812842961 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:42.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.793+0000 7f1004e00640 1 --2- 172.21.15.46:0/2812842961 >> v2:172.21.15.73:3300/0 conn(0x7f100015a810 0x7f10000b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.793+0000 7f1005800640 1 -- 172.21.15.46:0/2812842961 >> v2:172.21.15.73:3300/0 conn(0x7f100015a810 msgr2=0x7f10000b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:42.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.793+0000 7f1005800640 1 --2- 172.21.15.46:0/2812842961 >> v2:172.21.15.73:3300/0 conn(0x7f100015a810 0x7f10000b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.793+0000 7f0fffe00640 1 -- 172.21.15.46:0/2812842961 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2634655970 0 0) 0x7f10000afbf0 con 0x7f10000c6080 2024-03-29T08:29:42.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.793+0000 7f1005800640 1 -- 172.21.15.46:0/2812842961 >> v1:172.21.15.73:6789/0 conn(0x7f10000c6080 legacy=0x7f1000069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.793+0000 7f1005800640 1 -- 172.21.15.46:0/2812842961 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10000aee30 con 0x7f100015e2c0 2024-03-29T08:29:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.797+0000 7f1005800640 1 --2- 172.21.15.46:0/2812842961 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f100015e2c0 0x7f10000b05c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009e50 tx=0x7f0ff801ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56519cd7b6db3d94 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.797+0000 7f0fffe00640 1 -- 172.21.15.46:0/2812842961 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff80a58d0 con 0x7f100015e2c0 2024-03-29T08:29:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.797+0000 7f0fffe00640 1 -- 172.21.15.46:0/2812842961 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff80a5a30 con 0x7f100015e2c0 2024-03-29T08:29:42.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.797+0000 7f0fffe00640 1 -- 172.21.15.46:0/2812842961 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff80a5dc0 con 0x7f100015e2c0 2024-03-29T08:29:42.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.797+0000 7f1006200640 1 -- 172.21.15.46:0/2812842961 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f100015e2c0 msgr2=0x7f10000b05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.797+0000 7f1006200640 1 --2- 172.21.15.46:0/2812842961 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f100015e2c0 0x7f10000b05c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009e50 tx=0x7f0ff801ca00 comp rx=0 tx=0).stop 2024-03-29T08:29:42.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.797+0000 7f1006200640 1 -- 172.21.15.46:0/2812842961 shutdown_connections 2024-03-29T08:29:42.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.797+0000 7f1006200640 1 --2- 172.21.15.46:0/2812842961 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f100015e2c0 0x7f10000b05c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.797+0000 7f1006200640 1 --2- 172.21.15.46:0/2812842961 >> v2:172.21.15.73:3300/0 conn(0x7f100015a810 0x7f10000b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.797+0000 7f1006200640 1 -- 172.21.15.46:0/2812842961 >> 172.21.15.46:0/2812842961 conn(0x7f1000098a40 msgr2=0x7f10000cf400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:42.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.797+0000 7f1006200640 1 -- 172.21.15.46:0/2812842961 shutdown_connections 2024-03-29T08:29:42.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.797+0000 7f1006200640 1 -- 172.21.15.46:0/2812842961 wait complete. 2024-03-29T08:29:42.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1006200640 1 Processor -- start 2024-03-29T08:29:42.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1006200640 1 -- start start 2024-03-29T08:29:42.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1006200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10000c6080 0x7f10001f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1006200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f100015a810 0x7f10000b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1006200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f100015e2c0 0x7f10000c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1006200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10000c6570 con 0x7f10000c6080 2024-03-29T08:29:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1006200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10000c6850 con 0x7f100015e2c0 2024-03-29T08:29:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1006200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10000c6b30 con 0x7f100015a810 2024-03-29T08:29:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1004e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10000c6080 0x7f10001f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1004e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10000c6080 0x7f10001f47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60148/0 (socket says 172.21.15.46:60148) 2024-03-29T08:29:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1004e00640 1 -- 172.21.15.46:0/685531839 learned_addr learned my addr 172.21.15.46:0/685531839 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1005800640 1 --2- 172.21.15.46:0/685531839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f100015e2c0 0x7f10000c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f0ff7e00640 1 --2- 172.21.15.46:0/685531839 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f100015a810 0x7f10000b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1004e00640 1 -- 172.21.15.46:0/685531839 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f100015a810 msgr2=0x7f10000b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1004e00640 1 --2- 172.21.15.46:0/685531839 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f100015a810 0x7f10000b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1004e00640 1 -- 172.21.15.46:0/685531839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f100015e2c0 msgr2=0x7f10000c3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1004e00640 1 --2- 172.21.15.46:0/685531839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f100015e2c0 0x7f10000c3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1004e00640 1 -- 172.21.15.46:0/685531839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff8009b30 con 0x7f10000c6080 2024-03-29T08:29:42.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1004e00640 1 --2- 172.21.15.46:0/685531839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10000c6080 0x7f10001f47c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000e930 tx=0x7f0ff000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:42.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f0ffd600640 1 -- 172.21.15.46:0/685531839 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0009690 con 0x7f10000c6080 2024-03-29T08:29:42.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f0ffd600640 1 -- 172.21.15.46:0/685531839 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff000ca00 con 0x7f10000c6080 2024-03-29T08:29:42.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f0ffd600640 1 -- 172.21.15.46:0/685531839 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff000cd70 con 0x7f10000c6080 2024-03-29T08:29:42.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1006200640 1 -- 172.21.15.46:0/685531839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10000c6e10 con 0x7f10000c6080 2024-03-29T08:29:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.801+0000 7f1006200640 1 -- 172.21.15.46:0/685531839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10000b3ef0 con 0x7f10000c6080 2024-03-29T08:29:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.805+0000 7f1006200640 1 -- 172.21.15.46:0/685531839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1000067170 con 0x7f10000c6080 2024-03-29T08:29:42.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.805+0000 7f0ffd600640 1 -- 172.21.15.46:0/685531839 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0ff0013470 con 0x7f10000c6080 2024-03-29T08:29:42.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.805+0000 7f0ffd600640 1 --2- 172.21.15.46:0/685531839 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0fdc066690 0x7f0fdc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:42.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.805+0000 7f0ffd600640 1 -- 172.21.15.46:0/685531839 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ff00843c0 con 0x7f10000c6080 2024-03-29T08:29:42.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.805+0000 7f0ff7e00640 1 --2- 172.21.15.46:0/685531839 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0fdc066690 0x7f0fdc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:42.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.805+0000 7f0ff7e00640 1 --2- 172.21.15.46:0/685531839 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0fdc066690 0x7f0fdc068b50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f10000ae300 tx=0x7f0fe8007d60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:42.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.809+0000 7f0ffd600640 1 -- 172.21.15.46:0/685531839 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0ff0051890 con 0x7f10000c6080 2024-03-29T08:29:42.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.981+0000 7f1006200640 1 -- 172.21.15.46:0/685531839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f100004e040 con 0x7f10000c6080 2024-03-29T08:29:42.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.981+0000 7f0ffd600640 1 -- 172.21.15.46:0/685531839 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0ff0054eb0 con 0x7f10000c6080 2024-03-29T08:29:42.987 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:29:42.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.985+0000 7f1006200640 1 -- 172.21.15.46:0/685531839 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0fdc066690 msgr2=0x7f0fdc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.985+0000 7f1006200640 1 --2- 172.21.15.46:0/685531839 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0fdc066690 0x7f0fdc068b50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f10000ae300 tx=0x7f0fe8007d60 comp rx=0 tx=0).stop 2024-03-29T08:29:42.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.985+0000 7f1006200640 1 -- 172.21.15.46:0/685531839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10000c6080 msgr2=0x7f10001f47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:42.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.985+0000 7f1006200640 1 --2- 172.21.15.46:0/685531839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10000c6080 0x7f10001f47c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000e930 tx=0x7f0ff000edf0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.985+0000 7f1006200640 1 -- 172.21.15.46:0/685531839 shutdown_connections 2024-03-29T08:29:42.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.985+0000 7f1006200640 1 --2- 172.21.15.46:0/685531839 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0fdc066690 0x7f0fdc068b50 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.985+0000 7f1006200640 1 --2- 172.21.15.46:0/685531839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f100015e2c0 0x7f10000c3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.985+0000 7f1006200640 1 --2- 172.21.15.46:0/685531839 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f100015a810 0x7f10000b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.985+0000 7f1006200640 1 --2- 172.21.15.46:0/685531839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10000c6080 0x7f10001f47c0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:42.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.985+0000 7f1006200640 1 -- 172.21.15.46:0/685531839 >> 172.21.15.46:0/685531839 conn(0x7f1000098a40 msgr2=0x7f10000b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:42.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.985+0000 7f1006200640 1 -- 172.21.15.46:0/685531839 shutdown_connections 2024-03-29T08:29:42.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:42.985+0000 7f1006200640 1 -- 172.21.15.46:0/685531839 wait complete. 2024-03-29T08:29:43.008 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:29:43.009 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:29:43.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:29:43.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.193+0000 7f46e0800640 1 Processor -- start 2024-03-29T08:29:43.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.193+0000 7f46e0800640 1 -- start start 2024-03-29T08:29:43.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.193+0000 7f46e0800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f46d8144210 0x7f46d80cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:43.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.193+0000 7f46e0800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f46d815ada0 0x7f46d80ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:43.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.193+0000 7f46e0800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46d80cb510 con 0x7f46d80ce950 2024-03-29T08:29:43.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.193+0000 7f46e0800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46d80cede0 con 0x7f46d815ada0 2024-03-29T08:29:43.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.193+0000 7f46e0800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d806d960 con 0x7f46d8144210 2024-03-29T08:29:43.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.197+0000 7f46dd400640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f46d815ada0 0x7f46d80ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:43.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.197+0000 7f46dde00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f46d8144210 0x7f46d80cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:43.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.197+0000 7f46de800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f46d80ce950 0x7f46d815e360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:56034/0 (socket says 172.21.15.46:56034) 2024-03-29T08:29:43.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.197+0000 7f46de800640 1 -- 172.21.15.46:0/98050653 learned_addr learned my addr 172.21.15.46:0/98050653 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:43.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.197+0000 7f46dd400640 1 -- 172.21.15.46:0/98050653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f46d8144210 msgr2=0x7f46d80cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:43.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.197+0000 7f46dd400640 1 --2- 172.21.15.46:0/98050653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f46d8144210 0x7f46d80cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.197+0000 7f46dd400640 1 -- 172.21.15.46:0/98050653 >> v1:172.21.15.73:6789/0 conn(0x7f46d80ce950 legacy=0x7f46d815e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:43.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.197+0000 7f46dd400640 1 -- 172.21.15.46:0/98050653 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d8070580 con 0x7f46d815ada0 2024-03-29T08:29:43.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.197+0000 7f46dde00640 1 --2- 172.21.15.46:0/98050653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f46d8144210 0x7f46d80cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:29:43.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.197+0000 7f46dd400640 1 --2- 172.21.15.46:0/98050653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46d815ada0 0x7f46d80ce410 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f46c80097c0 tx=0x7f46c801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=910b5f50bf1260bd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:43.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46dca00640 1 -- 172.21.15.46:0/98050653 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c80ab600 con 0x7f46d815ada0 2024-03-29T08:29:43.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46dca00640 1 -- 172.21.15.46:0/98050653 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c80a7050 con 0x7f46d815ada0 2024-03-29T08:29:43.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46dca00640 1 -- 172.21.15.46:0/98050653 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c80ab980 con 0x7f46d815ada0 2024-03-29T08:29:43.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46e0800640 1 -- 172.21.15.46:0/98050653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46d815ada0 msgr2=0x7f46d80ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:43.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46e0800640 1 --2- 172.21.15.46:0/98050653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46d815ada0 0x7f46d80ce410 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f46c80097c0 tx=0x7f46c801c4a0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46e0800640 1 -- 172.21.15.46:0/98050653 shutdown_connections 2024-03-29T08:29:43.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46e0800640 1 --2- 172.21.15.46:0/98050653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46d815ada0 0x7f46d80ce410 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46e0800640 1 --2- 172.21.15.46:0/98050653 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f46d8144210 0x7f46d80cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46e0800640 1 -- 172.21.15.46:0/98050653 >> 172.21.15.46:0/98050653 conn(0x7f46d8066110 msgr2=0x7f46d8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:43.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46e0800640 1 -- 172.21.15.46:0/98050653 shutdown_connections 2024-03-29T08:29:43.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46e0800640 1 -- 172.21.15.46:0/98050653 wait complete. 2024-03-29T08:29:43.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46e0800640 1 Processor -- start 2024-03-29T08:29:43.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46e0800640 1 -- start start 2024-03-29T08:29:43.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46e0800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46d80ce950 0x7f46d80ac940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46e0800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46d8144210 0x7f46d813bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46e0800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f46d815ada0 0x7f46d813c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46e0800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d813bac0 con 0x7f46d8144210 2024-03-29T08:29:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46e0800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d813aeb0 con 0x7f46d80ce950 2024-03-29T08:29:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46e0800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46d813b190 con 0x7f46d815ada0 2024-03-29T08:29:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46dd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46d8144210 0x7f46d813bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46dd400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46d8144210 0x7f46d813bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60164/0 (socket says 172.21.15.46:60164) 2024-03-29T08:29:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46dd400640 1 -- 172.21.15.46:0/3434460161 learned_addr learned my addr 172.21.15.46:0/3434460161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46dde00640 1 --2- 172.21.15.46:0/3434460161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46d80ce950 0x7f46d80ac940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46de800640 1 --2- 172.21.15.46:0/3434460161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f46d815ada0 0x7f46d813c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46dd400640 1 -- 172.21.15.46:0/3434460161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f46d815ada0 msgr2=0x7f46d813c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46dd400640 1 --2- 172.21.15.46:0/3434460161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f46d815ada0 0x7f46d813c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46dd400640 1 -- 172.21.15.46:0/3434460161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46d80ce950 msgr2=0x7f46d80ac940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46dd400640 1 --2- 172.21.15.46:0/3434460161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46d80ce950 0x7f46d80ac940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.201+0000 7f46dd400640 1 -- 172.21.15.46:0/3434460161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc009420 con 0x7f46d8144210 2024-03-29T08:29:43.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.205+0000 7f46dd400640 1 --2- 172.21.15.46:0/3434460161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46d8144210 0x7f46d813bf50 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f46c80ae040 tx=0x7f46c8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:43.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.205+0000 7f46d2000640 1 -- 172.21.15.46:0/3434460161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c80b4430 con 0x7f46d8144210 2024-03-29T08:29:43.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.205+0000 7f46d2000640 1 -- 172.21.15.46:0/3434460161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46c80b4590 con 0x7f46d8144210 2024-03-29T08:29:43.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.205+0000 7f46d2000640 1 -- 172.21.15.46:0/3434460161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46c80b4940 con 0x7f46d8144210 2024-03-29T08:29:43.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.205+0000 7f46e0800640 1 -- 172.21.15.46:0/3434460161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c8009420 con 0x7f46d8144210 2024-03-29T08:29:43.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.205+0000 7f46e0800640 1 -- 172.21.15.46:0/3434460161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d813b750 con 0x7f46d8144210 2024-03-29T08:29:43.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.205+0000 7f46e0800640 1 -- 172.21.15.46:0/3434460161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d80ce280 con 0x7f46d8144210 2024-03-29T08:29:43.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.205+0000 7f46d2000640 1 -- 172.21.15.46:0/3434460161 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f46c80b2040 con 0x7f46d8144210 2024-03-29T08:29:43.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.205+0000 7f46d2000640 1 --2- 172.21.15.46:0/3434460161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f46bc066690 0x7f46bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:43.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.205+0000 7f46d2000640 1 -- 172.21.15.46:0/3434460161 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46c811dfe0 con 0x7f46d8144210 2024-03-29T08:29:43.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.209+0000 7f46dde00640 1 --2- 172.21.15.46:0/3434460161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f46bc066690 0x7f46bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:43.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.209+0000 7f46dde00640 1 --2- 172.21.15.46:0/3434460161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f46bc066690 0x7f46bc068b50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009c80 tx=0x7f46cc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:43.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.213+0000 7f46d2000640 1 -- 172.21.15.46:0/3434460161 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f46c80eb2b0 con 0x7f46d8144210 2024-03-29T08:29:43.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.373+0000 7f46e0800640 1 -- 172.21.15.46:0/3434460161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46d8066c90 con 0x7f46d8144210 2024-03-29T08:29:43.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.373+0000 7f46d2000640 1 -- 172.21.15.46:0/3434460161 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16067 (secure 0 0 0) 0x7f46c80b0090 con 0x7f46d8144210 2024-03-29T08:29:43.379 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:43.379 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":121,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:29:15.473193+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:29:12.329121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-29T08:29:13.465939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:29:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.377+0000 7f46e0800640 1 -- 172.21.15.46:0/3434460161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f46bc066690 msgr2=0x7f46bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.377+0000 7f46e0800640 1 --2- 172.21.15.46:0/3434460161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f46bc066690 0x7f46bc068b50 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009c80 tx=0x7f46cc008040 comp rx=0 tx=0).stop 2024-03-29T08:29:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.377+0000 7f46e0800640 1 -- 172.21.15.46:0/3434460161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46d8144210 msgr2=0x7f46d813bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.377+0000 7f46e0800640 1 --2- 172.21.15.46:0/3434460161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46d8144210 0x7f46d813bf50 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f46c80ae040 tx=0x7f46c8019240 comp rx=0 tx=0).stop 2024-03-29T08:29:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.377+0000 7f46e0800640 1 -- 172.21.15.46:0/3434460161 shutdown_connections 2024-03-29T08:29:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.377+0000 7f46e0800640 1 --2- 172.21.15.46:0/3434460161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f46bc066690 0x7f46bc068b50 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.377+0000 7f46e0800640 1 --2- 172.21.15.46:0/3434460161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f46d815ada0 0x7f46d813c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.377+0000 7f46e0800640 1 --2- 172.21.15.46:0/3434460161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f46d8144210 0x7f46d813bf50 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.377+0000 7f46e0800640 1 --2- 172.21.15.46:0/3434460161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46d80ce950 0x7f46d80ac940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.377+0000 7f46e0800640 1 -- 172.21.15.46:0/3434460161 >> 172.21.15.46:0/3434460161 conn(0x7f46d8066110 msgr2=0x7f46d8140670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:43.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.377+0000 7f46e0800640 1 -- 172.21.15.46:0/3434460161 shutdown_connections 2024-03-29T08:29:43.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.377+0000 7f46e0800640 1 -- 172.21.15.46:0/3434460161 wait complete. 2024-03-29T08:29:43.400 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:29:12.329121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:29:13.465939+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:29:43.401 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:29:43.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.537+0000 7fd722400640 1 Processor -- start 2024-03-29T08:29:43.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.537+0000 7fd722400640 1 -- start start 2024-03-29T08:29:43.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.541+0000 7fd722400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd71c0cbea0 0x7fd71c0b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:43.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.541+0000 7fd722400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd71c0b2360 con 0x7fd71c15efe0 2024-03-29T08:29:43.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.541+0000 7fd722400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd71c15e500 con 0x7fd71c143eb0 2024-03-29T08:29:43.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.541+0000 7fd722400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c15e7e0 con 0x7fd71c0cbea0 2024-03-29T08:29:43.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.541+0000 7fd71be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd71c15efe0 0x7fd71c0b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43376/0 (socket says 172.21.15.46:43376) 2024-03-29T08:29:43.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.541+0000 7fd71be00640 1 -- 172.21.15.46:0/2615633444 learned_addr learned my addr 172.21.15.46:0/2615633444 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:43.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.541+0000 7fd712a00640 1 --2- 172.21.15.46:0/2615633444 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd71c0cbea0 0x7fd71c0b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:43.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.541+0000 7fd71aa00640 1 -- 172.21.15.46:0/2615633444 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156677766 0 0) 0x7fd71c0b2360 con 0x7fd71c15efe0 2024-03-29T08:29:43.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.541+0000 7fd71aa00640 1 -- 172.21.15.46:0/2615633444 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd700003530 con 0x7fd71c15efe0 2024-03-29T08:29:43.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.541+0000 7fd71aa00640 1 -- 172.21.15.46:0/2615633444 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 363252006 0 0) 0x7fd71c15e500 con 0x7fd71c143eb0 2024-03-29T08:29:43.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.541+0000 7fd71aa00640 1 -- 172.21.15.46:0/2615633444 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd71c0b2360 con 0x7fd71c143eb0 2024-03-29T08:29:43.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.541+0000 7fd712a00640 1 -- 172.21.15.46:0/2615633444 >> v1:172.21.15.73:6789/0 conn(0x7fd71c143eb0 legacy=0x7fd71c0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:43.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.541+0000 7fd712a00640 1 -- 172.21.15.46:0/2615633444 >> v1:172.21.15.46:6789/0 conn(0x7fd71c15efe0 legacy=0x7fd71c0b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:43.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.541+0000 7fd71aa00640 1 -- 172.21.15.46:0/2615633444 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4060919320 0 0) 0x7fd700003530 con 0x7fd71c15efe0 2024-03-29T08:29:43.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.541+0000 7fd712a00640 1 -- 172.21.15.46:0/2615633444 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c15eac0 con 0x7fd71c0cbea0 2024-03-29T08:29:43.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.545+0000 7fd712a00640 1 --2- 172.21.15.46:0/2615633444 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd71c0cbea0 0x7fd71c0b1630 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd7040097f0 tx=0x7fd70401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=191eaa92f426e2e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:43.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.545+0000 7fd71aa00640 1 -- 172.21.15.46:0/2615633444 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714001a00 con 0x7fd71c0cbea0 2024-03-29T08:29:43.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.545+0000 7fd71aa00640 1 -- 172.21.15.46:0/2615633444 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7040a8050 con 0x7fd71c0cbea0 2024-03-29T08:29:43.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.545+0000 7fd71aa00640 1 -- 172.21.15.46:0/2615633444 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7040ac620 con 0x7fd71c0cbea0 2024-03-29T08:29:43.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.545+0000 7fd722400640 1 -- 172.21.15.46:0/2615633444 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd71c0cbea0 msgr2=0x7fd71c0b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:43.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.545+0000 7fd722400640 1 --2- 172.21.15.46:0/2615633444 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd71c0cbea0 0x7fd71c0b1630 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd7040097f0 tx=0x7fd70401c400 comp rx=0 tx=0).stop 2024-03-29T08:29:43.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.545+0000 7fd722400640 1 -- 172.21.15.46:0/2615633444 shutdown_connections 2024-03-29T08:29:43.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.545+0000 7fd722400640 1 --2- 172.21.15.46:0/2615633444 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd71c0cbea0 0x7fd71c0b1630 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.545+0000 7fd722400640 1 -- 172.21.15.46:0/2615633444 >> 172.21.15.46:0/2615633444 conn(0x7fd71c06a390 msgr2=0x7fd71c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:43.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.545+0000 7fd722400640 1 -- 172.21.15.46:0/2615633444 shutdown_connections 2024-03-29T08:29:43.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.545+0000 7fd722400640 1 -- 172.21.15.46:0/2615633444 wait complete. 2024-03-29T08:29:43.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.545+0000 7fd722400640 1 Processor -- start 2024-03-29T08:29:43.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd722400640 1 -- start start 2024-03-29T08:29:43.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd722400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd71c0cbea0 0x7fd71c164f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:43.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd722400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd71c143eb0 0x7fd71c135ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:43.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd722400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c15efe0 0x7fd71c0afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:43.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd722400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c158760 con 0x7fd71c15efe0 2024-03-29T08:29:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd722400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c136410 con 0x7fd71c0cbea0 2024-03-29T08:29:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd722400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71c163e70 con 0x7fd71c143eb0 2024-03-29T08:29:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd71be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c15efe0 0x7fd71c0afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd71be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c15efe0 0x7fd71c0afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60180/0 (socket says 172.21.15.46:60180) 2024-03-29T08:29:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd71be00640 1 -- 172.21.15.46:0/4002146238 learned_addr learned my addr 172.21.15.46:0/4002146238 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd712a00640 1 --2- 172.21.15.46:0/4002146238 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd71c143eb0 0x7fd71c135ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd71b400640 1 --2- 172.21.15.46:0/4002146238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd71c0cbea0 0x7fd71c164f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd71be00640 1 -- 172.21.15.46:0/4002146238 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd71c143eb0 msgr2=0x7fd71c135ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd71be00640 1 --2- 172.21.15.46:0/4002146238 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd71c143eb0 0x7fd71c135ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd71be00640 1 -- 172.21.15.46:0/4002146238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd71c0cbea0 msgr2=0x7fd71c164f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd71be00640 1 --2- 172.21.15.46:0/4002146238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd71c0cbea0 0x7fd71c164f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd71be00640 1 -- 172.21.15.46:0/4002146238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd704009420 con 0x7fd71c15efe0 2024-03-29T08:29:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd71b400640 1 --2- 172.21.15.46:0/4002146238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd71c0cbea0 0x7fd71c164f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:43.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd71be00640 1 --2- 172.21.15.46:0/4002146238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c15efe0 0x7fd71c0afc30 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fd71400ccb0 tx=0x7fd71400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd713e00640 1 -- 172.21.15.46:0/4002146238 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714012400 con 0x7fd71c15efe0 2024-03-29T08:29:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd713e00640 1 -- 172.21.15.46:0/4002146238 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd714012560 con 0x7fd71c15efe0 2024-03-29T08:29:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd713e00640 1 -- 172.21.15.46:0/4002146238 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd714012910 con 0x7fd71c15efe0 2024-03-29T08:29:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd722400640 1 -- 172.21.15.46:0/4002146238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd71c164150 con 0x7fd71c15efe0 2024-03-29T08:29:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd722400640 1 -- 172.21.15.46:0/4002146238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd71c164710 con 0x7fd71c15efe0 2024-03-29T08:29:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.549+0000 7fd722400640 1 -- 172.21.15.46:0/4002146238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71c071a70 con 0x7fd71c15efe0 2024-03-29T08:29:43.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.553+0000 7fd713e00640 1 -- 172.21.15.46:0/4002146238 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd714012a70 con 0x7fd71c15efe0 2024-03-29T08:29:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.553+0000 7fd713e00640 1 --2- 172.21.15.46:0/4002146238 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd6f8066690 0x7fd6f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.553+0000 7fd713e00640 1 -- 172.21.15.46:0/4002146238 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd714084810 con 0x7fd71c15efe0 2024-03-29T08:29:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.553+0000 7fd71b400640 1 --2- 172.21.15.46:0/4002146238 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd6f8066690 0x7fd6f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:43.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.553+0000 7fd71b400640 1 --2- 172.21.15.46:0/4002146238 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd6f8066690 0x7fd6f8068b50 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0016e0 tx=0x7fd70c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:43.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.557+0000 7fd713e00640 1 -- 172.21.15.46:0/4002146238 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd714051bf0 con 0x7fd71c15efe0 2024-03-29T08:29:43.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.717+0000 7fd722400640 1 -- 172.21.15.46:0/4002146238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd71c0adea0 con 0x7fd71c15efe0 2024-03-29T08:29:43.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.717+0000 7fd713e00640 1 -- 172.21.15.46:0/4002146238 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd714055210 con 0x7fd71c15efe0 2024-03-29T08:29:43.723 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:29:43.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.721+0000 7fd722400640 1 -- 172.21.15.46:0/4002146238 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd6f8066690 msgr2=0x7fd6f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:43.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.721+0000 7fd722400640 1 --2- 172.21.15.46:0/4002146238 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd6f8066690 0x7fd6f8068b50 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0016e0 tx=0x7fd70c009040 comp rx=0 tx=0).stop 2024-03-29T08:29:43.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.721+0000 7fd722400640 1 -- 172.21.15.46:0/4002146238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c15efe0 msgr2=0x7fd71c0afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:43.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.721+0000 7fd722400640 1 --2- 172.21.15.46:0/4002146238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c15efe0 0x7fd71c0afc30 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fd71400ccb0 tx=0x7fd71400a430 comp rx=0 tx=0).stop 2024-03-29T08:29:43.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.721+0000 7fd722400640 1 -- 172.21.15.46:0/4002146238 shutdown_connections 2024-03-29T08:29:43.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.721+0000 7fd722400640 1 --2- 172.21.15.46:0/4002146238 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd6f8066690 0x7fd6f8068b50 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.721+0000 7fd722400640 1 --2- 172.21.15.46:0/4002146238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd71c15efe0 0x7fd71c0afc30 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.721+0000 7fd722400640 1 --2- 172.21.15.46:0/4002146238 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd71c143eb0 0x7fd71c135ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.721+0000 7fd722400640 1 --2- 172.21.15.46:0/4002146238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd71c0cbea0 0x7fd71c164f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.721+0000 7fd722400640 1 -- 172.21.15.46:0/4002146238 >> 172.21.15.46:0/4002146238 conn(0x7fd71c06a390 msgr2=0x7fd71c0b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:43.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.721+0000 7fd722400640 1 -- 172.21.15.46:0/4002146238 shutdown_connections 2024-03-29T08:29:43.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.721+0000 7fd722400640 1 -- 172.21.15.46:0/4002146238 wait complete. 2024-03-29T08:29:43.744 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:29:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.885+0000 7f2a39400640 1 Processor -- start 2024-03-29T08:29:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.885+0000 7f2a39400640 1 -- start start 2024-03-29T08:29:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.885+0000 7f2a39400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a34143eb0 0x7f2a340cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.885+0000 7f2a39400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2a3415efe0 0x7f2a340cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.885+0000 7f2a39400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a340b2360 con 0x7f2a340b1630 2024-03-29T08:29:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.885+0000 7f2a39400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3415e500 con 0x7f2a3415efe0 2024-03-29T08:29:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.885+0000 7f2a39400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3415e7e0 con 0x7f2a34143eb0 2024-03-29T08:29:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.885+0000 7f2a33200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2a340b1630 0x7f2a340b1a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43384/0 (socket says 172.21.15.46:43384) 2024-03-29T08:29:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.885+0000 7f2a33200640 1 -- 172.21.15.46:0/3585534786 learned_addr learned my addr 172.21.15.46:0/3585534786 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.885+0000 7f2a31e00640 1 --2- 172.21.15.46:0/3585534786 >> v2:172.21.15.46:3300/0 conn(0x7f2a3415efe0 0x7f2a340cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.885+0000 7f2a32800640 1 --2- 172.21.15.46:0/3585534786 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a34143eb0 0x7f2a340cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.885+0000 7f2a31e00640 1 -- 172.21.15.46:0/3585534786 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a34143eb0 msgr2=0x7f2a340cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.885+0000 7f2a31e00640 1 --2- 172.21.15.46:0/3585534786 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a34143eb0 0x7f2a340cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.885+0000 7f2a31e00640 1 -- 172.21.15.46:0/3585534786 >> v1:172.21.15.46:6789/0 conn(0x7f2a340b1630 legacy=0x7f2a340b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.885+0000 7f2a31e00640 1 -- 172.21.15.46:0/3585534786 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3415eac0 con 0x7f2a3415efe0 2024-03-29T08:29:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.885+0000 7f2a32800640 1 --2- 172.21.15.46:0/3585534786 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a34143eb0 0x7f2a340cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.885+0000 7f2a31400640 1 -- 172.21.15.46:0/3585534786 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 736596304 0 0) 0x7f2a340b2360 con 0x7f2a340b1630 2024-03-29T08:29:43.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.889+0000 7f2a31e00640 1 --2- 172.21.15.46:0/3585534786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3415efe0 0x7f2a340cbcd0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f2a240097c0 tx=0x7f2a2401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd287a93a4a98d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.889+0000 7f2a31400640 1 -- 172.21.15.46:0/3585534786 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a240ab600 con 0x7f2a3415efe0 2024-03-29T08:29:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.889+0000 7f2a31400640 1 -- 172.21.15.46:0/3585534786 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a240a7050 con 0x7f2a3415efe0 2024-03-29T08:29:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.889+0000 7f2a31400640 1 -- 172.21.15.46:0/3585534786 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a240ab980 con 0x7f2a3415efe0 2024-03-29T08:29:43.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.889+0000 7f2a39400640 1 -- 172.21.15.46:0/3585534786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3415efe0 msgr2=0x7f2a340cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.889+0000 7f2a39400640 1 --2- 172.21.15.46:0/3585534786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3415efe0 0x7f2a340cbcd0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f2a240097c0 tx=0x7f2a2401c4a0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.889+0000 7f2a39400640 1 -- 172.21.15.46:0/3585534786 shutdown_connections 2024-03-29T08:29:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.889+0000 7f2a39400640 1 --2- 172.21.15.46:0/3585534786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a3415efe0 0x7f2a340cbcd0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.889+0000 7f2a39400640 1 --2- 172.21.15.46:0/3585534786 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a34143eb0 0x7f2a340cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.889+0000 7f2a39400640 1 -- 172.21.15.46:0/3585534786 >> 172.21.15.46:0/3585534786 conn(0x7f2a3406a390 msgr2=0x7f2a340cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.889+0000 7f2a39400640 1 -- 172.21.15.46:0/3585534786 shutdown_connections 2024-03-29T08:29:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.889+0000 7f2a39400640 1 -- 172.21.15.46:0/3585534786 wait complete. 2024-03-29T08:29:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.889+0000 7f2a39400640 1 Processor -- start 2024-03-29T08:29:43.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.889+0000 7f2a39400640 1 -- start start 2024-03-29T08:29:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a39400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340b1630 0x7f2a34142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a39400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a34143eb0 0x7f2a340cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a39400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a3415efe0 0x7f2a340ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a39400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a340ae6d0 con 0x7f2a340b1630 2024-03-29T08:29:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a39400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a340ae9b0 con 0x7f2a34143eb0 2024-03-29T08:29:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a39400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a340aec90 con 0x7f2a3415efe0 2024-03-29T08:29:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a32800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340b1630 0x7f2a34142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a32800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340b1630 0x7f2a34142f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60190/0 (socket says 172.21.15.46:60190) 2024-03-29T08:29:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a32800640 1 -- 172.21.15.46:0/4139902350 learned_addr learned my addr 172.21.15.46:0/4139902350 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a33200640 1 --2- 172.21.15.46:0/4139902350 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a3415efe0 0x7f2a340ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a31e00640 1 --2- 172.21.15.46:0/4139902350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a34143eb0 0x7f2a340cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:43.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a32800640 1 -- 172.21.15.46:0/4139902350 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a3415efe0 msgr2=0x7f2a340ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a32800640 1 --2- 172.21.15.46:0/4139902350 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a3415efe0 0x7f2a340ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a32800640 1 -- 172.21.15.46:0/4139902350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a34143eb0 msgr2=0x7f2a340cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a32800640 1 --2- 172.21.15.46:0/4139902350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a34143eb0 0x7f2a340cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a32800640 1 -- 172.21.15.46:0/4139902350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a24009420 con 0x7f2a340b1630 2024-03-29T08:29:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a31e00640 1 --2- 172.21.15.46:0/4139902350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a34143eb0 0x7f2a340cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:43.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a32800640 1 --2- 172.21.15.46:0/4139902350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340b1630 0x7f2a34142f90 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00c7c0 tx=0x7f2a1c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a2aa00640 1 -- 172.21.15.46:0/4139902350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c007a30 con 0x7f2a340b1630 2024-03-29T08:29:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a2aa00640 1 -- 172.21.15.46:0/4139902350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c007b90 con 0x7f2a340b1630 2024-03-29T08:29:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a2aa00640 1 -- 172.21.15.46:0/4139902350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c00ce70 con 0x7f2a340b1630 2024-03-29T08:29:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a39400640 1 -- 172.21.15.46:0/4139902350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a340aef70 con 0x7f2a340b1630 2024-03-29T08:29:43.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a39400640 1 -- 172.21.15.46:0/4139902350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a341434d0 con 0x7f2a340b1630 2024-03-29T08:29:43.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a39400640 1 -- 172.21.15.46:0/4139902350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a340af490 con 0x7f2a340b1630 2024-03-29T08:29:43.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.893+0000 7f2a2aa00640 1 -- 172.21.15.46:0/4139902350 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2a1c0113f0 con 0x7f2a340b1630 2024-03-29T08:29:43.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.897+0000 7f2a2aa00640 1 --2- 172.21.15.46:0/4139902350 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2a10066690 0x7f2a10068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:43.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.897+0000 7f2a2aa00640 1 -- 172.21.15.46:0/4139902350 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a1c084580 con 0x7f2a340b1630 2024-03-29T08:29:43.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.897+0000 7f2a31e00640 1 --2- 172.21.15.46:0/4139902350 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2a10066690 0x7f2a10068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:43.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.897+0000 7f2a31e00640 1 --2- 172.21.15.46:0/4139902350 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2a10066690 0x7f2a10068b50 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2a240ad000 tx=0x7f2a240a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:43.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:43.901+0000 7f2a2aa00640 1 -- 172.21.15.46:0/4139902350 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2a340af490 con 0x7f2a340b1630 2024-03-29T08:29:44.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.061+0000 7f2a39400640 1 -- 172.21.15.46:0/4139902350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2a340af490 con 0x7f2a340b1630 2024-03-29T08:29:44.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.061+0000 7f2a2aa00640 1 -- 172.21.15.46:0/4139902350 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2a340af490 con 0x7f2a340b1630 2024-03-29T08:29:44.065 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:29:44.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.061+0000 7f2a39400640 1 -- 172.21.15.46:0/4139902350 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2a10066690 msgr2=0x7f2a10068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:44.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.061+0000 7f2a39400640 1 --2- 172.21.15.46:0/4139902350 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2a10066690 0x7f2a10068b50 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2a240ad000 tx=0x7f2a240a6000 comp rx=0 tx=0).stop 2024-03-29T08:29:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.061+0000 7f2a39400640 1 -- 172.21.15.46:0/4139902350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340b1630 msgr2=0x7f2a34142f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.061+0000 7f2a39400640 1 --2- 172.21.15.46:0/4139902350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340b1630 0x7f2a34142f90 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00c7c0 tx=0x7f2a1c00cc80 comp rx=0 tx=0).stop 2024-03-29T08:29:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.061+0000 7f2a39400640 1 -- 172.21.15.46:0/4139902350 shutdown_connections 2024-03-29T08:29:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.061+0000 7f2a39400640 1 --2- 172.21.15.46:0/4139902350 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2a10066690 0x7f2a10068b50 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.061+0000 7f2a39400640 1 --2- 172.21.15.46:0/4139902350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a34143eb0 0x7f2a340cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.061+0000 7f2a39400640 1 --2- 172.21.15.46:0/4139902350 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a3415efe0 0x7f2a340ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.061+0000 7f2a39400640 1 --2- 172.21.15.46:0/4139902350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340b1630 0x7f2a34142f90 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.061+0000 7f2a39400640 1 -- 172.21.15.46:0/4139902350 >> 172.21.15.46:0/4139902350 conn(0x7f2a3406a390 msgr2=0x7f2a3406cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.061+0000 7f2a39400640 1 -- 172.21.15.46:0/4139902350 shutdown_connections 2024-03-29T08:29:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.061+0000 7f2a39400640 1 -- 172.21.15.46:0/4139902350 wait complete. 2024-03-29T08:29:44.086 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:29:44.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.225+0000 7fc4d2000640 1 Processor -- start 2024-03-29T08:29:44.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.225+0000 7fc4d2000640 1 -- start start 2024-03-29T08:29:44.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.225+0000 7fc4d2000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4cc15de20 0x7fc4cc0b3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.225+0000 7fc4d2000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc4cc099450 0x7fc4cc15e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.225+0000 7fc4d2000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4cc099950 con 0x7fc4cc0b3ac0 2024-03-29T08:29:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.225+0000 7fc4d2000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4cc099c30 con 0x7fc4cc099450 2024-03-29T08:29:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.225+0000 7fc4d2000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc0cb590 con 0x7fc4cc15de20 2024-03-29T08:29:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.225+0000 7fc4cbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc4cc099450 0x7fc4cc15e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.225+0000 7fc4cbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc4cc099450 0x7fc4cc15e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60200/0 (socket says 172.21.15.46:60200) 2024-03-29T08:29:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.225+0000 7fc4cbe00640 1 -- 172.21.15.46:0/2668770785 learned_addr learned my addr 172.21.15.46:0/2668770785 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.225+0000 7fc4cb400640 1 --2- 172.21.15.46:0/2668770785 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4cc15de20 0x7fc4cc0b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.225+0000 7fc4cbe00640 1 -- 172.21.15.46:0/2668770785 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4cc15de20 msgr2=0x7fc4cc0b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.225+0000 7fc4cbe00640 1 --2- 172.21.15.46:0/2668770785 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4cc15de20 0x7fc4cc0b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.225+0000 7fc4cbe00640 1 -- 172.21.15.46:0/2668770785 >> v1:172.21.15.73:6789/0 conn(0x7fc4cc0b3ac0 legacy=0x7fc4cc0cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:44.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.225+0000 7fc4cbe00640 1 -- 172.21.15.46:0/2668770785 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc0cb870 con 0x7fc4cc099450 2024-03-29T08:29:44.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.229+0000 7fc4cbe00640 1 --2- 172.21.15.46:0/2668770785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4cc099450 0x7fc4cc15e740 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009e50 tx=0x7fc4c401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794df0e372cdd0ef server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:44.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.229+0000 7fc4caa00640 1 -- 172.21.15.46:0/2668770785 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c40a5750 con 0x7fc4cc099450 2024-03-29T08:29:44.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.229+0000 7fc4caa00640 1 -- 172.21.15.46:0/2668770785 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c40a58b0 con 0x7fc4cc099450 2024-03-29T08:29:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.229+0000 7fc4caa00640 1 -- 172.21.15.46:0/2668770785 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c40a5c40 con 0x7fc4cc099450 2024-03-29T08:29:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.229+0000 7fc4d2000640 1 -- 172.21.15.46:0/2668770785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4cc099450 msgr2=0x7fc4cc15e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.229+0000 7fc4d2000640 1 --2- 172.21.15.46:0/2668770785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4cc099450 0x7fc4cc15e740 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009e50 tx=0x7fc4c401ca30 comp rx=0 tx=0).stop 2024-03-29T08:29:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.229+0000 7fc4d2000640 1 -- 172.21.15.46:0/2668770785 shutdown_connections 2024-03-29T08:29:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.229+0000 7fc4d2000640 1 --2- 172.21.15.46:0/2668770785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4cc099450 0x7fc4cc15e740 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.229+0000 7fc4d2000640 1 --2- 172.21.15.46:0/2668770785 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4cc15de20 0x7fc4cc0b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.229+0000 7fc4d2000640 1 -- 172.21.15.46:0/2668770785 >> 172.21.15.46:0/2668770785 conn(0x7fc4cc0b2220 msgr2=0x7fc4cc0b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.229+0000 7fc4d2000640 1 -- 172.21.15.46:0/2668770785 shutdown_connections 2024-03-29T08:29:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.229+0000 7fc4d2000640 1 -- 172.21.15.46:0/2668770785 wait complete. 2024-03-29T08:29:44.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4d2000640 1 Processor -- start 2024-03-29T08:29:44.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4d2000640 1 -- start start 2024-03-29T08:29:44.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4d2000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4cc099450 0x7fc4cc0ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:44.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4d2000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4cc15de20 0x7fc4cc0c4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:44.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4d2000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4cc146980 0x7fc4cc0c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:44.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4d2000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc15c700 con 0x7fc4cc099450 2024-03-29T08:29:44.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4d2000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc0ca620 con 0x7fc4cc15de20 2024-03-29T08:29:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4d2000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc15baf0 con 0x7fc4cc146980 2024-03-29T08:29:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4cc099450 0x7fc4cc0ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4cc099450 0x7fc4cc0ca0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60210/0 (socket says 172.21.15.46:60210) 2024-03-29T08:29:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4cb400640 1 -- 172.21.15.46:0/1283767410 learned_addr learned my addr 172.21.15.46:0/1283767410 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4cbe00640 1 --2- 172.21.15.46:0/1283767410 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4cc146980 0x7fc4cc0c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4c2a00640 1 --2- 172.21.15.46:0/1283767410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4cc15de20 0x7fc4cc0c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4cb400640 1 -- 172.21.15.46:0/1283767410 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4cc146980 msgr2=0x7fc4cc0c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4cb400640 1 --2- 172.21.15.46:0/1283767410 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4cc146980 0x7fc4cc0c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4cb400640 1 -- 172.21.15.46:0/1283767410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4cc15de20 msgr2=0x7fc4cc0c4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4cb400640 1 --2- 172.21.15.46:0/1283767410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4cc15de20 0x7fc4cc0c4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4cb400640 1 -- 172.21.15.46:0/1283767410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c4009b30 con 0x7fc4cc099450 2024-03-29T08:29:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4cb400640 1 --2- 172.21.15.46:0/1283767410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4cc099450 0x7fc4cc0ca0e0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00e870 tx=0x7fc4bc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4c3e00640 1 -- 172.21.15.46:0/1283767410 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc009510 con 0x7fc4cc099450 2024-03-29T08:29:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4c3e00640 1 -- 172.21.15.46:0/1283767410 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc00eec0 con 0x7fc4cc099450 2024-03-29T08:29:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4d2000640 1 -- 172.21.15.46:0/1283767410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4cc15f6a0 con 0x7fc4cc099450 2024-03-29T08:29:44.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4c3e00640 1 -- 172.21.15.46:0/1283767410 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc00cb40 con 0x7fc4cc099450 2024-03-29T08:29:44.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4d2000640 1 -- 172.21.15.46:0/1283767410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4cc15c0b0 con 0x7fc4cc099450 2024-03-29T08:29:44.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.233+0000 7fc4d2000640 1 -- 172.21.15.46:0/1283767410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4cc0701d0 con 0x7fc4cc099450 2024-03-29T08:29:44.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.237+0000 7fc4c3e00640 1 -- 172.21.15.46:0/1283767410 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc4bc00ccc0 con 0x7fc4cc099450 2024-03-29T08:29:44.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.237+0000 7fc4c3e00640 1 --2- 172.21.15.46:0/1283767410 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc4a8066690 0x7fc4a8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:44.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.237+0000 7fc4c3e00640 1 -- 172.21.15.46:0/1283767410 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4bc0841b0 con 0x7fc4cc099450 2024-03-29T08:29:44.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.237+0000 7fc4c2a00640 1 --2- 172.21.15.46:0/1283767410 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc4a8066690 0x7fc4a8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:44.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.237+0000 7fc4c2a00640 1 --2- 172.21.15.46:0/1283767410 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc4a8066690 0x7fc4a8068b50 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc154590 tx=0x7fc4b4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:44.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.241+0000 7fc4c3e00640 1 -- 172.21.15.46:0/1283767410 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc4bc0515c0 con 0x7fc4cc099450 2024-03-29T08:29:44.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.401+0000 7fc4d2000640 1 -- 172.21.15.46:0/1283767410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc4cc04e120 con 0x7fc4cc099450 2024-03-29T08:29:44.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.401+0000 7fc4c3e00640 1 -- 172.21.15.46:0/1283767410 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc4bc054be0 con 0x7fc4cc099450 2024-03-29T08:29:44.406 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:29:44.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.405+0000 7fc4d2000640 1 -- 172.21.15.46:0/1283767410 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc4a8066690 msgr2=0x7fc4a8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:44.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.405+0000 7fc4d2000640 1 --2- 172.21.15.46:0/1283767410 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc4a8066690 0x7fc4a8068b50 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc154590 tx=0x7fc4b4009040 comp rx=0 tx=0).stop 2024-03-29T08:29:44.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.405+0000 7fc4d2000640 1 -- 172.21.15.46:0/1283767410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4cc099450 msgr2=0x7fc4cc0ca0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:44.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.405+0000 7fc4d2000640 1 --2- 172.21.15.46:0/1283767410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4cc099450 0x7fc4cc0ca0e0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00e870 tx=0x7fc4bc00ec70 comp rx=0 tx=0).stop 2024-03-29T08:29:44.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.405+0000 7fc4d2000640 1 -- 172.21.15.46:0/1283767410 shutdown_connections 2024-03-29T08:29:44.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.405+0000 7fc4d2000640 1 --2- 172.21.15.46:0/1283767410 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc4a8066690 0x7fc4a8068b50 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.405+0000 7fc4d2000640 1 --2- 172.21.15.46:0/1283767410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4cc15de20 0x7fc4cc0c4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.405+0000 7fc4d2000640 1 --2- 172.21.15.46:0/1283767410 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc4cc146980 0x7fc4cc0c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.405+0000 7fc4d2000640 1 --2- 172.21.15.46:0/1283767410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc4cc099450 0x7fc4cc0ca0e0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.405+0000 7fc4d2000640 1 -- 172.21.15.46:0/1283767410 >> 172.21.15.46:0/1283767410 conn(0x7fc4cc0b2220 msgr2=0x7fc4cc0b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:44.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.405+0000 7fc4d2000640 1 -- 172.21.15.46:0/1283767410 shutdown_connections 2024-03-29T08:29:44.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.405+0000 7fc4d2000640 1 -- 172.21.15.46:0/1283767410 wait complete. 2024-03-29T08:29:44.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:29:44.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.565+0000 7fa0f3c00640 1 Processor -- start 2024-03-29T08:29:44.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.565+0000 7fa0f3c00640 1 -- start start 2024-03-29T08:29:44.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.565+0000 7fa0f3c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa0ec06dbb0 0x7fa0ec15d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:44.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.565+0000 7fa0f3c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ec070c70 con 0x7fa0ec142ab0 2024-03-29T08:29:44.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.565+0000 7fa0f3c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ec070340 con 0x7fa0ec15d940 2024-03-29T08:29:44.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.565+0000 7fa0f3c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0ec070620 con 0x7fa0ec06dbb0 2024-03-29T08:29:44.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.565+0000 7fa0f1200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa0ec142ab0 0x7fa0ec06d4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43386/0 (socket says 172.21.15.46:43386) 2024-03-29T08:29:44.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.569+0000 7fa0f1200640 1 -- 172.21.15.46:0/3258753727 learned_addr learned my addr 172.21.15.46:0/3258753727 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:44.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.569+0000 7fa0e3e00640 1 --2- 172.21.15.46:0/3258753727 >> v2:172.21.15.46:3300/0 conn(0x7fa0ec06dbb0 0x7fa0ec15d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:44.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.569+0000 7fa0ebe00640 1 -- 172.21.15.46:0/3258753727 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300042810 0 0) 0x7fa0ec070c70 con 0x7fa0ec142ab0 2024-03-29T08:29:44.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.569+0000 7fa0ebe00640 1 -- 172.21.15.46:0/3258753727 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0d0003530 con 0x7fa0ec142ab0 2024-03-29T08:29:44.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.569+0000 7fa0e3e00640 1 -- 172.21.15.46:0/3258753727 >> v1:172.21.15.73:6789/0 conn(0x7fa0ec15d940 legacy=0x7fa0ec0b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:44.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.569+0000 7fa0e3e00640 1 -- 172.21.15.46:0/3258753727 >> v1:172.21.15.46:6789/0 conn(0x7fa0ec142ab0 legacy=0x7fa0ec06d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:44.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.569+0000 7fa0e3e00640 1 -- 172.21.15.46:0/3258753727 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec070900 con 0x7fa0ec06dbb0 2024-03-29T08:29:44.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.569+0000 7fa0e3e00640 1 --2- 172.21.15.46:0/3258753727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec06dbb0 0x7fa0ec15d400 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40097c0 tx=0x7fa0d401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b0770a87ee8879 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:44.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.569+0000 7fa0ebe00640 1 -- 172.21.15.46:0/3258753727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d40ac410 con 0x7fa0ec06dbb0 2024-03-29T08:29:44.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.569+0000 7fa0ebe00640 1 -- 172.21.15.46:0/3258753727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d40ac570 con 0x7fa0ec06dbb0 2024-03-29T08:29:44.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.569+0000 7fa0ebe00640 1 -- 172.21.15.46:0/3258753727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d40ac700 con 0x7fa0ec06dbb0 2024-03-29T08:29:44.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f3c00640 1 -- 172.21.15.46:0/3258753727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec06dbb0 msgr2=0x7fa0ec15d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:44.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f3c00640 1 --2- 172.21.15.46:0/3258753727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec06dbb0 0x7fa0ec15d400 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fa0d40097c0 tx=0x7fa0d401c4a0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f3c00640 1 -- 172.21.15.46:0/3258753727 shutdown_connections 2024-03-29T08:29:44.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f3c00640 1 --2- 172.21.15.46:0/3258753727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec06dbb0 0x7fa0ec15d400 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f3c00640 1 -- 172.21.15.46:0/3258753727 >> 172.21.15.46:0/3258753727 conn(0x7fa0ec066080 msgr2=0x7fa0ec069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:44.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f3c00640 1 -- 172.21.15.46:0/3258753727 shutdown_connections 2024-03-29T08:29:44.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f3c00640 1 -- 172.21.15.46:0/3258753727 wait complete. 2024-03-29T08:29:44.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f3c00640 1 Processor -- start 2024-03-29T08:29:44.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f3c00640 1 -- start start 2024-03-29T08:29:44.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f3c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0ec06dbb0 0x7fa0ec1f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:44.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f3c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa0ec142ab0 0x7fa0ec1415f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:44.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f3c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec15d940 0x7fa0ec15b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:44.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f3c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec0cbf50 con 0x7fa0ec15d940 2024-03-29T08:29:44.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f3c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec1f89d0 con 0x7fa0ec06dbb0 2024-03-29T08:29:44.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f3c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ec15ba80 con 0x7fa0ec142ab0 2024-03-29T08:29:44.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec15d940 0x7fa0ec15b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:44.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f1c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec15d940 0x7fa0ec15b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60236/0 (socket says 172.21.15.46:60236) 2024-03-29T08:29:44.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f1c00640 1 -- 172.21.15.46:0/2391866153 learned_addr learned my addr 172.21.15.46:0/2391866153 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:44.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0e3e00640 1 --2- 172.21.15.46:0/2391866153 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa0ec142ab0 0x7fa0ec1415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:44.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f1200640 1 --2- 172.21.15.46:0/2391866153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0ec06dbb0 0x7fa0ec1f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:44.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f1c00640 1 -- 172.21.15.46:0/2391866153 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa0ec142ab0 msgr2=0x7fa0ec1415f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:44.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f1c00640 1 --2- 172.21.15.46:0/2391866153 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa0ec142ab0 0x7fa0ec1415f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f1c00640 1 -- 172.21.15.46:0/2391866153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0ec06dbb0 msgr2=0x7fa0ec1f8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:44.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f1c00640 1 --2- 172.21.15.46:0/2391866153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0ec06dbb0 0x7fa0ec1f8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.573+0000 7fa0f1c00640 1 -- 172.21.15.46:0/2391866153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d4009420 con 0x7fa0ec15d940 2024-03-29T08:29:44.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.577+0000 7fa0f1c00640 1 --2- 172.21.15.46:0/2391866153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec15d940 0x7fa0ec15b0a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400c760 tx=0x7fa0e400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:44.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.577+0000 7fa0e9600640 1 -- 172.21.15.46:0/2391866153 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e4003b80 con 0x7fa0ec15d940 2024-03-29T08:29:44.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.577+0000 7fa0e9600640 1 -- 172.21.15.46:0/2391866153 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e4003ce0 con 0x7fa0ec15d940 2024-03-29T08:29:44.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.577+0000 7fa0e9600640 1 -- 172.21.15.46:0/2391866153 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400aa30 con 0x7fa0ec15d940 2024-03-29T08:29:44.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.577+0000 7fa0f3c00640 1 -- 172.21.15.46:0/2391866153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ec15b7a0 con 0x7fa0ec15d940 2024-03-29T08:29:44.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.577+0000 7fa0f3c00640 1 -- 172.21.15.46:0/2391866153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ec1f8660 con 0x7fa0ec15d940 2024-03-29T08:29:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.577+0000 7fa0f3c00640 1 -- 172.21.15.46:0/2391866153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ec1f8b60 con 0x7fa0ec15d940 2024-03-29T08:29:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.577+0000 7fa0e9600640 1 -- 172.21.15.46:0/2391866153 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa0e400ab90 con 0x7fa0ec15d940 2024-03-29T08:29:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.577+0000 7fa0e9600640 1 --2- 172.21.15.46:0/2391866153 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa0c8066690 0x7fa0c8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.577+0000 7fa0e9600640 1 -- 172.21.15.46:0/2391866153 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0e4083f30 con 0x7fa0ec15d940 2024-03-29T08:29:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.577+0000 7fa0f1200640 1 --2- 172.21.15.46:0/2391866153 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa0c8066690 0x7fa0c8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.581+0000 7fa0f1200640 1 --2- 172.21.15.46:0/2391866153 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa0c8066690 0x7fa0c8068b50 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc0011f0 tx=0x7fa0dc009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:44.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.581+0000 7fa0e9600640 1 -- 172.21.15.46:0/2391866153 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa0e4051290 con 0x7fa0ec15d940 2024-03-29T08:29:44.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.757+0000 7fa0f3c00640 1 -- 172.21.15.46:0/2391866153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa0ec071a90 con 0x7fa0ec15d940 2024-03-29T08:29:44.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.757+0000 7fa0e9600640 1 -- 172.21.15.46:0/2391866153 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa0e40548b0 con 0x7fa0ec15d940 2024-03-29T08:29:44.761 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2024-03-29T08:29:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.757+0000 7fa0f3c00640 1 -- 172.21.15.46:0/2391866153 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa0c8066690 msgr2=0x7fa0c8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.757+0000 7fa0f3c00640 1 --2- 172.21.15.46:0/2391866153 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa0c8066690 0x7fa0c8068b50 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc0011f0 tx=0x7fa0dc009040 comp rx=0 tx=0).stop 2024-03-29T08:29:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.757+0000 7fa0f3c00640 1 -- 172.21.15.46:0/2391866153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec15d940 msgr2=0x7fa0ec15b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.757+0000 7fa0f3c00640 1 --2- 172.21.15.46:0/2391866153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec15d940 0x7fa0ec15b0a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400c760 tx=0x7fa0e400cc20 comp rx=0 tx=0).stop 2024-03-29T08:29:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.757+0000 7fa0f3c00640 1 -- 172.21.15.46:0/2391866153 shutdown_connections 2024-03-29T08:29:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.757+0000 7fa0f3c00640 1 --2- 172.21.15.46:0/2391866153 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa0c8066690 0x7fa0c8068b50 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.757+0000 7fa0f3c00640 1 --2- 172.21.15.46:0/2391866153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0ec15d940 0x7fa0ec15b0a0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.757+0000 7fa0f3c00640 1 --2- 172.21.15.46:0/2391866153 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa0ec142ab0 0x7fa0ec1415f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.757+0000 7fa0f3c00640 1 --2- 172.21.15.46:0/2391866153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0ec06dbb0 0x7fa0ec1f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.757+0000 7fa0f3c00640 1 -- 172.21.15.46:0/2391866153 >> 172.21.15.46:0/2391866153 conn(0x7fa0ec066080 msgr2=0x7fa0ec15cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.757+0000 7fa0f3c00640 1 -- 172.21.15.46:0/2391866153 shutdown_connections 2024-03-29T08:29:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:44.757+0000 7fa0f3c00640 1 -- 172.21.15.46:0/2391866153 wait complete. 2024-03-29T08:29:44.783 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-03-29T08:29:44.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:44.783 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:44.801 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[51330]: fuse finished with error 0 and tester_r 0 2024-03-29T08:29:44.805 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:29:44.864 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:29:44.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:44.864 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:29:44.874 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:29:44.874 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:29:44.923 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:29:44.924 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi046.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:44.924 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:29:44.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:44.924 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:29:44.975 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-29T08:29:44.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:29:44.976 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T08:29:45.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.157+0000 7faece200640 1 Processor -- start 2024-03-29T08:29:45.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.157+0000 7faece200640 1 -- start start 2024-03-29T08:29:45.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.157+0000 7faece200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faec8142ab0 0x7faec806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.157+0000 7faece200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faec806d9e0 0x7faec815d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.157+0000 7faece200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec806de40 con 0x7faec815d940 2024-03-29T08:29:45.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.157+0000 7faece200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec8070c70 con 0x7faec8142ab0 2024-03-29T08:29:45.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.157+0000 7faece200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec8070340 con 0x7faec806d9e0 2024-03-29T08:29:45.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.157+0000 7faec7400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7faec8142ab0 0x7faec806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.157+0000 7faec7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7faec815d940 0x7faec80ae210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43396/0 (socket says 172.21.15.46:43396) 2024-03-29T08:29:45.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.157+0000 7faec7e00640 1 -- 172.21.15.46:0/877678687 learned_addr learned my addr 172.21.15.46:0/877678687 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:45.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.157+0000 7faebea00640 1 --2- 172.21.15.46:0/877678687 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faec806d9e0 0x7faec815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.157+0000 7faec7400640 1 -- 172.21.15.46:0/877678687 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faec806d9e0 msgr2=0x7faec815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:45.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.157+0000 7faec7400640 1 --2- 172.21.15.46:0/877678687 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faec806d9e0 0x7faec815d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.157+0000 7faec7400640 1 -- 172.21.15.46:0/877678687 >> v1:172.21.15.46:6789/0 conn(0x7faec815d940 legacy=0x7faec80ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:45.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.157+0000 7faec7400640 1 -- 172.21.15.46:0/877678687 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec8070620 con 0x7faec8142ab0 2024-03-29T08:29:45.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.157+0000 7faec6a00640 1 -- 172.21.15.46:0/877678687 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1838266161 0 0) 0x7faec806de40 con 0x7faec815d940 2024-03-29T08:29:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.161+0000 7faec7400640 1 --2- 172.21.15.46:0/877678687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec8142ab0 0x7faec806d4a0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7faeb80097c0 tx=0x7faeb801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0a2ad59adcf4c2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.161+0000 7faec6a00640 1 -- 172.21.15.46:0/877678687 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb80ab600 con 0x7faec8142ab0 2024-03-29T08:29:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.161+0000 7faec6a00640 1 -- 172.21.15.46:0/877678687 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb80a7050 con 0x7faec8142ab0 2024-03-29T08:29:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.161+0000 7faec6a00640 1 -- 172.21.15.46:0/877678687 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb80ab980 con 0x7faec8142ab0 2024-03-29T08:29:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.161+0000 7faece200640 1 -- 172.21.15.46:0/877678687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec8142ab0 msgr2=0x7faec806d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.161+0000 7faece200640 1 --2- 172.21.15.46:0/877678687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec8142ab0 0x7faec806d4a0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7faeb80097c0 tx=0x7faeb801c4a0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.161+0000 7faece200640 1 -- 172.21.15.46:0/877678687 shutdown_connections 2024-03-29T08:29:45.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.161+0000 7faece200640 1 --2- 172.21.15.46:0/877678687 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faec806d9e0 0x7faec815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.161+0000 7faece200640 1 --2- 172.21.15.46:0/877678687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec8142ab0 0x7faec806d4a0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.161+0000 7faece200640 1 -- 172.21.15.46:0/877678687 >> 172.21.15.46:0/877678687 conn(0x7faec8066080 msgr2=0x7faec8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:45.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.161+0000 7faece200640 1 -- 172.21.15.46:0/877678687 shutdown_connections 2024-03-29T08:29:45.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.161+0000 7faece200640 1 -- 172.21.15.46:0/877678687 wait complete. 2024-03-29T08:29:45.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faece200640 1 Processor -- start 2024-03-29T08:29:45.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faece200640 1 -- start start 2024-03-29T08:29:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faece200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec806d9e0 0x7faec8159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faece200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec8142ab0 0x7faec8139ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faece200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faec815d940 0x7faec81627d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faece200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec8159560 con 0x7faec806d9e0 2024-03-29T08:29:45.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faece200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec813dc60 con 0x7faec8142ab0 2024-03-29T08:29:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faece200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec815a5d0 con 0x7faec815d940 2024-03-29T08:29:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faec7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec806d9e0 0x7faec8159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faec7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec806d9e0 0x7faec8159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60248/0 (socket says 172.21.15.46:60248) 2024-03-29T08:29:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faec7400640 1 -- 172.21.15.46:0/3308664109 learned_addr learned my addr 172.21.15.46:0/3308664109 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faec7e00640 1 --2- 172.21.15.46:0/3308664109 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faec815d940 0x7faec81627d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faebea00640 1 --2- 172.21.15.46:0/3308664109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec8142ab0 0x7faec8139ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faec7400640 1 -- 172.21.15.46:0/3308664109 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faec815d940 msgr2=0x7faec81627d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faec7400640 1 --2- 172.21.15.46:0/3308664109 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faec815d940 0x7faec81627d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faec7400640 1 -- 172.21.15.46:0/3308664109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec8142ab0 msgr2=0x7faec8139ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faec7400640 1 --2- 172.21.15.46:0/3308664109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec8142ab0 0x7faec8139ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faec7400640 1 -- 172.21.15.46:0/3308664109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb8009420 con 0x7faec806d9e0 2024-03-29T08:29:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faec7400640 1 --2- 172.21.15.46:0/3308664109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec806d9e0 0x7faec8159020 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7faeb80ae040 tx=0x7faeb8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:45.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faebfe00640 1 -- 172.21.15.46:0/3308664109 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb80b4430 con 0x7faec806d9e0 2024-03-29T08:29:45.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faebfe00640 1 -- 172.21.15.46:0/3308664109 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb80b4590 con 0x7faec806d9e0 2024-03-29T08:29:45.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faebfe00640 1 -- 172.21.15.46:0/3308664109 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb80b4940 con 0x7faec806d9e0 2024-03-29T08:29:45.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faece200640 1 -- 172.21.15.46:0/3308664109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec8134820 con 0x7faec806d9e0 2024-03-29T08:29:45.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.165+0000 7faece200640 1 -- 172.21.15.46:0/3308664109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec8134b00 con 0x7faec806d9e0 2024-03-29T08:29:45.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.169+0000 7faece200640 1 -- 172.21.15.46:0/3308664109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec8071a90 con 0x7faec806d9e0 2024-03-29T08:29:45.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.169+0000 7faebfe00640 1 -- 172.21.15.46:0/3308664109 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7faeb80b2040 con 0x7faec806d9e0 2024-03-29T08:29:45.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.169+0000 7faebfe00640 1 --2- 172.21.15.46:0/3308664109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faea40666c0 0x7faea4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.169+0000 7faebfe00640 1 -- 172.21.15.46:0/3308664109 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faeb811eb50 con 0x7faec806d9e0 2024-03-29T08:29:45.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.169+0000 7faebea00640 1 --2- 172.21.15.46:0/3308664109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faea40666c0 0x7faea4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.169+0000 7faebea00640 1 --2- 172.21.15.46:0/3308664109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faea40666c0 0x7faea4068b80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7faec813d6b0 tx=0x7faeb000b040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:45.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.173+0000 7faebfe00640 1 -- 172.21.15.46:0/3308664109 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7faeb80ebf30 con 0x7faec806d9e0 2024-03-29T08:29:45.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.369+0000 7faece200640 1 -- 172.21.15.46:0/3308664109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7faec804e120 con 0x7faec806d9e0 2024-03-29T08:29:45.373 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-03-29T08:29:45.369+0000 7fc16ae00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:29:45.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.377+0000 7faebfe00640 1 -- 172.21.15.46:0/3308664109 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v95) v1 ==== 161+0+0 (secure 0 0 0) 0x7faeb811e090 con 0x7faec806d9e0 2024-03-29T08:29:45.381 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T08:29:45.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.377+0000 7faece200640 1 -- 172.21.15.46:0/3308664109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faea40666c0 msgr2=0x7faea4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:45.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.377+0000 7faece200640 1 --2- 172.21.15.46:0/3308664109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faea40666c0 0x7faea4068b80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7faec813d6b0 tx=0x7faeb000b040 comp rx=0 tx=0).stop 2024-03-29T08:29:45.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.377+0000 7faece200640 1 -- 172.21.15.46:0/3308664109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec806d9e0 msgr2=0x7faec8159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:45.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.377+0000 7faece200640 1 --2- 172.21.15.46:0/3308664109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec806d9e0 0x7faec8159020 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7faeb80ae040 tx=0x7faeb8019240 comp rx=0 tx=0).stop 2024-03-29T08:29:45.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.377+0000 7faece200640 1 -- 172.21.15.46:0/3308664109 shutdown_connections 2024-03-29T08:29:45.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.377+0000 7faece200640 1 --2- 172.21.15.46:0/3308664109 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7faea40666c0 0x7faea4068b80 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.377+0000 7faece200640 1 --2- 172.21.15.46:0/3308664109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec8142ab0 0x7faec8139ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.377+0000 7faece200640 1 --2- 172.21.15.46:0/3308664109 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7faec815d940 0x7faec81627d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.377+0000 7faece200640 1 --2- 172.21.15.46:0/3308664109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faec806d9e0 0x7faec8159020 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.377+0000 7faece200640 1 -- 172.21.15.46:0/3308664109 >> 172.21.15.46:0/3308664109 conn(0x7faec8066080 msgr2=0x7faec80ae210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:45.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.377+0000 7faece200640 1 -- 172.21.15.46:0/3308664109 shutdown_connections 2024-03-29T08:29:45.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.377+0000 7faece200640 1 -- 172.21.15.46:0/3308664109 wait complete. 2024-03-29T08:29:45.392 INFO:tasks.ceph.mds.a.smithi046.stderr: -7626> 2024-03-29T08:29:20.041+0000 7f7d71200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:29:45.404 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T08:29:45.407 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T08:29:45.444 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-03-29T08:29:45.452 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-03-29T08:29:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-03-29T08:29:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T08:29:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-03-29T08:29:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-03-29T08:29:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-03-29T08:29:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-03-29T08:29:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-03-29T08:29:45.452 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-03-29T08:29:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-03-29T08:29:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T08:29:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-03-29T08:29:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T08:29:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T08:29:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-03-29T08:29:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-03-29T08:29:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-03-29T08:29:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-03-29T08:29:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-03-29T08:29:45.453 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-03-29T08:29:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-03-29T08:29:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-03-29T08:29:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T08:29:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T08:29:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-03-29T08:29:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T08:29:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-03-29T08:29:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-03-29T08:29:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-03-29T08:29:45.454 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-03-29T08:29:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-03-29T08:29:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.73,[v2:172.21.15.73:3301,v1:172.21.15.73:6790]" 2024-03-29T08:29:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = e9b7960f-ec1f-4787-95c3-c806094d37f0 2024-03-29T08:29:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.455 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-03-29T08:29:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-03-29T08:29:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-03-29T08:29:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-03-29T08:29:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-03-29T08:29:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-03-29T08:29:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-03-29T08:29:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.455 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-03-29T08:29:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-03-29T08:29:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-03-29T08:29:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-03-29T08:29:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-03-29T08:29:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-03-29T08:29:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-03-29T08:29:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-03-29T08:29:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-03-29T08:29:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-03-29T08:29:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.456 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-03-29T08:29:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-03-29T08:29:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: bdev async discard = True 2024-03-29T08:29:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: bdev enable discard = True 2024-03-29T08:29:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore allocator = bitmap 2024-03-29T08:29:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-03-29T08:29:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-03-29T08:29:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 1/20 2024-03-29T08:29:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 1/20 2024-03-29T08:29:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:29:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-03-29T08:29:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 4/10 2024-03-29T08:29:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T08:29:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-03-29T08:29:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-03-29T08:29:45.457 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-03-29T08:29:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-03-29T08:29:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-03-29T08:29:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.458 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-03-29T08:29:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:29:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-03-29T08:29:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-03-29T08:29:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-03-29T08:29:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:29:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:29:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-03-29T08:29:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.458 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-03-29T08:29:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:29:45.458 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-03-29T08:29:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-03-29T08:29:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-03-29T08:29:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-03-29T08:29:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-03-29T08:29:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:29:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-03-29T08:29:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-03-29T08:29:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:29:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T08:29:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T08:29:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T08:29:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T08:29:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T08:29:45.459 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T08:29:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-03-29T08:29:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-03-29T08:29:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-03-29T08:29:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2024-03-29T08:29:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.460 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-03-29T08:29:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-03-29T08:29:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-03-29T08:29:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-03-29T08:29:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T08:29:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T08:29:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2024-03-29T08:29:45.460 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-03-29T08:29:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:29:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2024-03-29T08:29:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-03-29T08:29:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-03-29T08:29:45.461 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-03-29T08:29:45.461 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-03-29T08:29:45.461 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-03-29T08:29:45.461 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-03-29T08:29:45.461 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-03-29T08:29:45.461 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-03-29T08:29:45.461 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-03-29T08:29:45.461 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2024-03-29T08:29:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2024-03-29T08:29:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2024-03-29T08:29:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:29:45.461 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2024-03-29T08:29:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = True 2024-03-29T08:29:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2024-03-29T08:29:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = False 2024-03-29T08:29:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-03-29T08:29:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-03-29T08:29:45.462 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-03-29T08:29:45.462 INFO:teuthology.orchestra.run.smithi073.stdout:[global] 2024-03-29T08:29:45.462 INFO:teuthology.orchestra.run.smithi073.stdout: chdir = "" 2024-03-29T08:29:45.462 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T08:29:45.463 INFO:teuthology.orchestra.run.smithi073.stdout: auth supported = cephx 2024-03-29T08:29:45.463 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.463 INFO:teuthology.orchestra.run.smithi073.stdout: filestore xattr use omap = true 2024-03-29T08:29:45.463 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.463 INFO:teuthology.orchestra.run.smithi073.stdout: mon clock drift allowed = 1.000 2024-03-29T08:29:45.463 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.463 INFO:teuthology.orchestra.run.smithi073.stdout: osd crush chooseleaf type = 0 2024-03-29T08:29:45.463 INFO:teuthology.orchestra.run.smithi073.stdout: auth debug = true 2024-03-29T08:29:45.463 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.463 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on old message = true 2024-03-29T08:29:45.463 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on bug = true 2024-03-29T08:29:45.463 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.463 INFO:teuthology.orchestra.run.smithi073.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T08:29:45.463 INFO:teuthology.orchestra.run.smithi073.stdout: mon pg warn max object skew = 0 2024-03-29T08:29:45.463 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.464 INFO:teuthology.orchestra.run.smithi073.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T08:29:45.464 INFO:teuthology.orchestra.run.smithi073.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T08:29:45.464 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.464 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default size = 2 2024-03-29T08:29:45.464 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.464 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow primary affinity = true 2024-03-29T08:29:45.464 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow pg remap = true 2024-03-29T08:29:45.464 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on legacy crush tunables = false 2024-03-29T08:29:45.464 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on crush straw calc version zero = false 2024-03-29T08:29:45.464 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on no sortbitwise = false 2024-03-29T08:29:45.464 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on osd down out interval zero = false 2024-03-29T08:29:45.464 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on too few osds = false 2024-03-29T08:29:45.464 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T08:29:45.464 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T08:29:45.464 INFO:teuthology.orchestra.run.smithi073.stdout: mon_allow_pool_size_one = true 2024-03-29T08:29:45.465 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.465 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T08:29:45.465 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.465 INFO:teuthology.orchestra.run.smithi073.stdout: osd default data pool replay window = 5 2024-03-29T08:29:45.465 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.465 INFO:teuthology.orchestra.run.smithi073.stdout: mon allow pool delete = true 2024-03-29T08:29:45.465 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.465 INFO:teuthology.orchestra.run.smithi073.stdout: mon cluster log file level = debug 2024-03-29T08:29:45.465 INFO:teuthology.orchestra.run.smithi073.stdout: debug asserts on shutdown = true 2024-03-29T08:29:45.465 INFO:teuthology.orchestra.run.smithi073.stdout: mon health detail to clog = false 2024-03-29T08:29:45.465 INFO:teuthology.orchestra.run.smithi073.stdout: mon host = "172.21.15.46,172.21.15.73,[v2:172.21.15.73:3301,v1:172.21.15.73:6790]" 2024-03-29T08:29:45.465 INFO:teuthology.orchestra.run.smithi073.stdout: fsid = e9b7960f-ec1f-4787-95c3-c806094d37f0 2024-03-29T08:29:45.465 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.465 INFO:teuthology.orchestra.run.smithi073.stdout:[osd] 2024-03-29T08:29:45.465 INFO:teuthology.orchestra.run.smithi073.stdout: osd journal size = 100 2024-03-29T08:29:45.465 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.466 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub load threshold = 5.0 2024-03-29T08:29:45.466 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub max interval = 600 2024-03-29T08:29:45.466 INFO:teuthology.orchestra.run.smithi073.stdout: osd mclock profile = high_recovery_ops 2024-03-29T08:29:45.466 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.466 INFO:teuthology.orchestra.run.smithi073.stdout: osd recover clone overlap = true 2024-03-29T08:29:45.466 INFO:teuthology.orchestra.run.smithi073.stdout: osd recovery max chunk = 1048576 2024-03-29T08:29:45.466 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.466 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug shutdown = true 2024-03-29T08:29:45.466 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug op order = true 2024-03-29T08:29:45.466 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify stray on activate = true 2024-03-29T08:29:45.466 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.466 INFO:teuthology.orchestra.run.smithi073.stdout: osd open classes on start = true 2024-03-29T08:29:45.466 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug pg log writeout = true 2024-03-29T08:29:45.466 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.467 INFO:teuthology.orchestra.run.smithi073.stdout: osd deep scrub update digest min age = 30 2024-03-29T08:29:45.467 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.467 INFO:teuthology.orchestra.run.smithi073.stdout: osd map max advance = 10 2024-03-29T08:29:45.467 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.467 INFO:teuthology.orchestra.run.smithi073.stdout: journal zero on create = true 2024-03-29T08:29:45.467 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.467 INFO:teuthology.orchestra.run.smithi073.stdout: filestore ondisk finisher threads = 3 2024-03-29T08:29:45.467 INFO:teuthology.orchestra.run.smithi073.stdout: filestore apply finisher threads = 3 2024-03-29T08:29:45.467 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.467 INFO:teuthology.orchestra.run.smithi073.stdout: bdev debug aio = true 2024-03-29T08:29:45.467 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug misdirected ops = true 2024-03-29T08:29:45.467 INFO:teuthology.orchestra.run.smithi073.stdout: bdev async discard = True 2024-03-29T08:29:45.467 INFO:teuthology.orchestra.run.smithi073.stdout: bdev enable discard = True 2024-03-29T08:29:45.467 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore allocator = bitmap 2024-03-29T08:29:45.467 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore block size = 96636764160 2024-03-29T08:29:45.467 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore fsck on mount = True 2024-03-29T08:29:45.468 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluefs = 1/20 2024-03-29T08:29:45.468 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluestore = 1/20 2024-03-29T08:29:45.468 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:29:45.468 INFO:teuthology.orchestra.run.smithi073.stdout: debug osd = 20 2024-03-29T08:29:45.468 INFO:teuthology.orchestra.run.smithi073.stdout: debug rocksdb = 4/10 2024-03-29T08:29:45.468 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T08:29:45.468 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd full ratio = 0.9 2024-03-29T08:29:45.468 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd nearfull ratio = 0.8 2024-03-29T08:29:45.468 INFO:teuthology.orchestra.run.smithi073.stdout: osd failsafe full ratio = 0.95 2024-03-29T08:29:45.468 INFO:teuthology.orchestra.run.smithi073.stdout: osd objectstore = bluestore 2024-03-29T08:29:45.468 INFO:teuthology.orchestra.run.smithi073.stdout: osd op complaint time = 180 2024-03-29T08:29:45.468 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.468 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr] 2024-03-29T08:29:45.468 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:29:45.468 INFO:teuthology.orchestra.run.smithi073.stdout: debug mgr = 20 2024-03-29T08:29:45.468 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-03-29T08:29:45.469 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-03-29T08:29:45.469 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:29:45.469 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:29:45.469 INFO:teuthology.orchestra.run.smithi073.stdout: mgr/telemetry/nag = false 2024-03-29T08:29:45.469 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.469 INFO:teuthology.orchestra.run.smithi073.stdout:[mon] 2024-03-29T08:29:45.469 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:29:45.469 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-03-29T08:29:45.469 INFO:teuthology.orchestra.run.smithi073.stdout: debug paxos = 20 2024-03-29T08:29:45.469 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-03-29T08:29:45.469 INFO:teuthology.orchestra.run.smithi073.stdout: mon data avail warn = 5 2024-03-29T08:29:45.469 INFO:teuthology.orchestra.run.smithi073.stdout: mon mgr mkfs grace = 240 2024-03-29T08:29:45.469 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:29:45.469 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd reporter subtree level = osd 2024-03-29T08:29:45.469 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd prime pg temp = true 2024-03-29T08:29:45.469 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:29:45.470 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.470 INFO:teuthology.orchestra.run.smithi073.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T08:29:45.470 INFO:teuthology.orchestra.run.smithi073.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T08:29:45.470 INFO:teuthology.orchestra.run.smithi073.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T08:29:45.470 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.470 INFO:teuthology.orchestra.run.smithi073.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T08:29:45.470 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T08:29:45.470 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T08:29:45.470 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.470 INFO:teuthology.orchestra.run.smithi073.stdout: # 1m isn't quite enough 2024-03-29T08:29:45.470 INFO:teuthology.orchestra.run.smithi073.stdout: mon_down_mkfs_grace = 2m 2024-03-29T08:29:45.470 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.470 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_filestore_osds = false 2024-03-29T08:29:45.470 INFO:teuthology.orchestra.run.smithi073.stdout: mon op complaint time = 120 2024-03-29T08:29:45.470 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.471 INFO:teuthology.orchestra.run.smithi073.stdout:[client] 2024-03-29T08:29:45.471 INFO:teuthology.orchestra.run.smithi073.stdout: rgw cache enabled = true 2024-03-29T08:29:45.471 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable ops log = true 2024-03-29T08:29:45.471 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable usage log = true 2024-03-29T08:29:45.471 INFO:teuthology.orchestra.run.smithi073.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T08:29:45.471 INFO:teuthology.orchestra.run.smithi073.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T08:29:45.471 INFO:teuthology.orchestra.run.smithi073.stdout: client mount timeout = 600 2024-03-29T08:29:45.471 INFO:teuthology.orchestra.run.smithi073.stdout: debug client = 20 2024-03-29T08:29:45.471 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:29:45.471 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = "" 2024-03-29T08:29:45.471 INFO:teuthology.orchestra.run.smithi073.stdout: rados mon op timeout = 900 2024-03-29T08:29:45.471 INFO:teuthology.orchestra.run.smithi073.stdout: rados osd op timeout = 900 2024-03-29T08:29:45.471 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.a] 2024-03-29T08:29:45.471 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.b] 2024-03-29T08:29:45.471 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.c] 2024-03-29T08:29:45.471 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.a] 2024-03-29T08:29:45.471 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.c] 2024-03-29T08:29:45.472 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.b] 2024-03-29T08:29:45.472 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.d] 2024-03-29T08:29:45.472 INFO:teuthology.orchestra.run.smithi073.stdout:[mds] 2024-03-29T08:29:45.472 INFO:teuthology.orchestra.run.smithi073.stdout: debug mds = 20 2024-03-29T08:29:45.472 INFO:teuthology.orchestra.run.smithi073.stdout: debug mds balancer = 20 2024-03-29T08:29:45.472 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:29:45.472 INFO:teuthology.orchestra.run.smithi073.stdout: mds debug frag = True 2024-03-29T08:29:45.472 INFO:teuthology.orchestra.run.smithi073.stdout: mds debug scatterstat = True 2024-03-29T08:29:45.472 INFO:teuthology.orchestra.run.smithi073.stdout: mds op complaint time = 180 2024-03-29T08:29:45.472 INFO:teuthology.orchestra.run.smithi073.stdout: mds verify scatter = False 2024-03-29T08:29:45.472 INFO:teuthology.orchestra.run.smithi073.stdout: osd op complaint time = 180 2024-03-29T08:29:45.472 INFO:teuthology.orchestra.run.smithi073.stdout: rados mon op timeout = 900 2024-03-29T08:29:45.472 INFO:teuthology.orchestra.run.smithi073.stdout: rados osd op timeout = 900 2024-03-29T08:29:45.474 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T08:29:45.518 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T08:29:45.558 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-03-29T08:29:45.558 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-03-29T08:29:45.558 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T08:29:45.558 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-03-29T08:29:45.558 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.558 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-03-29T08:29:45.558 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.558 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-03-29T08:29:45.559 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.559 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-03-29T08:29:45.559 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-03-29T08:29:45.559 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.559 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-03-29T08:29:45.559 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-03-29T08:29:45.559 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.559 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T08:29:45.559 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-03-29T08:29:45.559 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.559 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T08:29:45.559 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T08:29:45.559 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.559 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-03-29T08:29:45.560 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.560 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-03-29T08:29:45.560 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-03-29T08:29:45.560 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-03-29T08:29:45.560 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-03-29T08:29:45.560 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-03-29T08:29:45.560 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-03-29T08:29:45.560 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-03-29T08:29:45.560 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T08:29:45.560 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T08:29:45.560 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-03-29T08:29:45.560 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.560 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T08:29:45.560 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.560 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-03-29T08:29:45.560 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.561 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-03-29T08:29:45.561 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.561 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-03-29T08:29:45.561 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-03-29T08:29:45.561 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-03-29T08:29:45.561 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.73,[v2:172.21.15.73:3301,v1:172.21.15.73:6790]" 2024-03-29T08:29:45.561 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = e9b7960f-ec1f-4787-95c3-c806094d37f0 2024-03-29T08:29:45.561 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.561 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-03-29T08:29:45.561 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-03-29T08:29:45.561 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.561 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-03-29T08:29:45.561 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-03-29T08:29:45.561 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-03-29T08:29:45.561 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.562 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-03-29T08:29:45.562 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-03-29T08:29:45.562 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.562 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-03-29T08:29:45.562 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-03-29T08:29:45.562 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-03-29T08:29:45.562 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.562 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-03-29T08:29:45.562 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-03-29T08:29:45.562 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.562 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-03-29T08:29:45.562 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.562 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-03-29T08:29:45.562 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.562 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-03-29T08:29:45.563 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.563 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-03-29T08:29:45.563 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-03-29T08:29:45.563 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.563 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-03-29T08:29:45.563 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-03-29T08:29:45.563 INFO:teuthology.orchestra.run.smithi046.stdout: bdev async discard = True 2024-03-29T08:29:45.563 INFO:teuthology.orchestra.run.smithi046.stdout: bdev enable discard = True 2024-03-29T08:29:45.563 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore allocator = bitmap 2024-03-29T08:29:45.563 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-03-29T08:29:45.563 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-03-29T08:29:45.563 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 1/20 2024-03-29T08:29:45.563 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 1/20 2024-03-29T08:29:45.563 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:29:45.563 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-03-29T08:29:45.563 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 4/10 2024-03-29T08:29:45.564 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T08:29:45.564 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-03-29T08:29:45.564 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-03-29T08:29:45.564 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-03-29T08:29:45.564 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-03-29T08:29:45.564 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-03-29T08:29:45.564 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.564 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-03-29T08:29:45.564 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:29:45.564 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-03-29T08:29:45.564 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-03-29T08:29:45.564 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-03-29T08:29:45.564 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:29:45.564 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:29:45.564 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-03-29T08:29:45.565 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.565 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-03-29T08:29:45.565 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:29:45.565 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-03-29T08:29:45.565 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-03-29T08:29:45.565 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-03-29T08:29:45.565 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-03-29T08:29:45.565 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-03-29T08:29:45.565 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:29:45.565 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-03-29T08:29:45.565 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-03-29T08:29:45.565 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:29:45.565 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.565 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T08:29:45.565 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T08:29:45.566 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T08:29:45.566 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.566 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T08:29:45.566 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T08:29:45.566 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T08:29:45.566 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.566 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-03-29T08:29:45.566 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-03-29T08:29:45.566 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.566 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-03-29T08:29:45.566 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2024-03-29T08:29:45.566 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:45.566 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-03-29T08:29:45.566 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-03-29T08:29:45.566 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-03-29T08:29:45.567 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-03-29T08:29:45.567 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T08:29:45.567 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T08:29:45.567 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2024-03-29T08:29:45.567 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-03-29T08:29:45.567 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:29:45.567 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2024-03-29T08:29:45.567 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-03-29T08:29:45.567 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-03-29T08:29:45.567 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-03-29T08:29:45.567 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-03-29T08:29:45.568 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-03-29T08:29:45.568 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-03-29T08:29:45.568 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-03-29T08:29:45.568 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-03-29T08:29:45.568 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-03-29T08:29:45.568 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2024-03-29T08:29:45.568 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2024-03-29T08:29:45.568 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2024-03-29T08:29:45.568 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:29:45.568 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2024-03-29T08:29:45.568 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = False 2024-03-29T08:29:45.568 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2024-03-29T08:29:45.568 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = False 2024-03-29T08:29:45.568 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-03-29T08:29:45.568 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-03-29T08:29:45.569 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-03-29T08:29:45.569 INFO:teuthology.orchestra.run.smithi073.stdout:[global] 2024-03-29T08:29:45.569 INFO:teuthology.orchestra.run.smithi073.stdout: chdir = "" 2024-03-29T08:29:45.569 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T08:29:45.569 INFO:teuthology.orchestra.run.smithi073.stdout: auth supported = cephx 2024-03-29T08:29:45.569 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.569 INFO:teuthology.orchestra.run.smithi073.stdout: filestore xattr use omap = true 2024-03-29T08:29:45.569 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.569 INFO:teuthology.orchestra.run.smithi073.stdout: mon clock drift allowed = 1.000 2024-03-29T08:29:45.570 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.570 INFO:teuthology.orchestra.run.smithi073.stdout: osd crush chooseleaf type = 0 2024-03-29T08:29:45.570 INFO:teuthology.orchestra.run.smithi073.stdout: auth debug = true 2024-03-29T08:29:45.570 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.570 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on old message = true 2024-03-29T08:29:45.570 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on bug = true 2024-03-29T08:29:45.570 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.570 INFO:teuthology.orchestra.run.smithi073.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T08:29:45.570 INFO:teuthology.orchestra.run.smithi073.stdout: mon pg warn max object skew = 0 2024-03-29T08:29:45.570 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.570 INFO:teuthology.orchestra.run.smithi073.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T08:29:45.570 INFO:teuthology.orchestra.run.smithi073.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T08:29:45.570 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.570 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default size = 2 2024-03-29T08:29:45.570 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.571 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow primary affinity = true 2024-03-29T08:29:45.571 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow pg remap = true 2024-03-29T08:29:45.571 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on legacy crush tunables = false 2024-03-29T08:29:45.571 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on crush straw calc version zero = false 2024-03-29T08:29:45.571 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on no sortbitwise = false 2024-03-29T08:29:45.571 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on osd down out interval zero = false 2024-03-29T08:29:45.571 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on too few osds = false 2024-03-29T08:29:45.571 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T08:29:45.571 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T08:29:45.571 INFO:teuthology.orchestra.run.smithi073.stdout: mon_allow_pool_size_one = true 2024-03-29T08:29:45.571 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.571 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T08:29:45.571 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.571 INFO:teuthology.orchestra.run.smithi073.stdout: osd default data pool replay window = 5 2024-03-29T08:29:45.571 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.572 INFO:teuthology.orchestra.run.smithi073.stdout: mon allow pool delete = true 2024-03-29T08:29:45.572 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.572 INFO:teuthology.orchestra.run.smithi073.stdout: mon cluster log file level = debug 2024-03-29T08:29:45.572 INFO:teuthology.orchestra.run.smithi073.stdout: debug asserts on shutdown = true 2024-03-29T08:29:45.572 INFO:teuthology.orchestra.run.smithi073.stdout: mon health detail to clog = false 2024-03-29T08:29:45.572 INFO:teuthology.orchestra.run.smithi073.stdout: mon host = "172.21.15.46,172.21.15.73,[v2:172.21.15.73:3301,v1:172.21.15.73:6790]" 2024-03-29T08:29:45.572 INFO:teuthology.orchestra.run.smithi073.stdout: fsid = e9b7960f-ec1f-4787-95c3-c806094d37f0 2024-03-29T08:29:45.572 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.572 INFO:teuthology.orchestra.run.smithi073.stdout:[osd] 2024-03-29T08:29:45.572 INFO:teuthology.orchestra.run.smithi073.stdout: osd journal size = 100 2024-03-29T08:29:45.572 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.572 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub load threshold = 5.0 2024-03-29T08:29:45.572 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub max interval = 600 2024-03-29T08:29:45.572 INFO:teuthology.orchestra.run.smithi073.stdout: osd mclock profile = high_recovery_ops 2024-03-29T08:29:45.572 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.572 INFO:teuthology.orchestra.run.smithi073.stdout: osd recover clone overlap = true 2024-03-29T08:29:45.573 INFO:teuthology.orchestra.run.smithi073.stdout: osd recovery max chunk = 1048576 2024-03-29T08:29:45.573 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.573 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug shutdown = true 2024-03-29T08:29:45.573 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug op order = true 2024-03-29T08:29:45.573 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify stray on activate = true 2024-03-29T08:29:45.573 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.573 INFO:teuthology.orchestra.run.smithi073.stdout: osd open classes on start = true 2024-03-29T08:29:45.573 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug pg log writeout = true 2024-03-29T08:29:45.573 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.573 INFO:teuthology.orchestra.run.smithi073.stdout: osd deep scrub update digest min age = 30 2024-03-29T08:29:45.573 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.573 INFO:teuthology.orchestra.run.smithi073.stdout: osd map max advance = 10 2024-03-29T08:29:45.573 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.573 INFO:teuthology.orchestra.run.smithi073.stdout: journal zero on create = true 2024-03-29T08:29:45.573 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.574 INFO:teuthology.orchestra.run.smithi073.stdout: filestore ondisk finisher threads = 3 2024-03-29T08:29:45.574 INFO:teuthology.orchestra.run.smithi073.stdout: filestore apply finisher threads = 3 2024-03-29T08:29:45.574 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.574 INFO:teuthology.orchestra.run.smithi073.stdout: bdev debug aio = true 2024-03-29T08:29:45.574 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug misdirected ops = true 2024-03-29T08:29:45.574 INFO:teuthology.orchestra.run.smithi073.stdout: bdev async discard = True 2024-03-29T08:29:45.574 INFO:teuthology.orchestra.run.smithi073.stdout: bdev enable discard = True 2024-03-29T08:29:45.574 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore allocator = bitmap 2024-03-29T08:29:45.574 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore block size = 96636764160 2024-03-29T08:29:45.574 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore fsck on mount = True 2024-03-29T08:29:45.574 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluefs = 1/20 2024-03-29T08:29:45.574 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluestore = 1/20 2024-03-29T08:29:45.574 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:29:45.574 INFO:teuthology.orchestra.run.smithi073.stdout: debug osd = 20 2024-03-29T08:29:45.574 INFO:teuthology.orchestra.run.smithi073.stdout: debug rocksdb = 4/10 2024-03-29T08:29:45.575 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T08:29:45.575 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd full ratio = 0.9 2024-03-29T08:29:45.575 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd nearfull ratio = 0.8 2024-03-29T08:29:45.575 INFO:teuthology.orchestra.run.smithi073.stdout: osd failsafe full ratio = 0.95 2024-03-29T08:29:45.575 INFO:teuthology.orchestra.run.smithi073.stdout: osd objectstore = bluestore 2024-03-29T08:29:45.575 INFO:teuthology.orchestra.run.smithi073.stdout: osd op complaint time = 180 2024-03-29T08:29:45.575 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.575 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr] 2024-03-29T08:29:45.575 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:29:45.575 INFO:teuthology.orchestra.run.smithi073.stdout: debug mgr = 20 2024-03-29T08:29:45.575 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-03-29T08:29:45.575 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-03-29T08:29:45.575 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:29:45.575 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:29:45.575 INFO:teuthology.orchestra.run.smithi073.stdout: mgr/telemetry/nag = false 2024-03-29T08:29:45.576 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.576 INFO:teuthology.orchestra.run.smithi073.stdout:[mon] 2024-03-29T08:29:45.576 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:29:45.576 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-03-29T08:29:45.576 INFO:teuthology.orchestra.run.smithi073.stdout: debug paxos = 20 2024-03-29T08:29:45.576 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-03-29T08:29:45.576 INFO:teuthology.orchestra.run.smithi073.stdout: mon data avail warn = 5 2024-03-29T08:29:45.576 INFO:teuthology.orchestra.run.smithi073.stdout: mon mgr mkfs grace = 240 2024-03-29T08:29:45.576 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:29:45.576 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd reporter subtree level = osd 2024-03-29T08:29:45.576 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd prime pg temp = true 2024-03-29T08:29:45.576 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:29:45.576 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.576 INFO:teuthology.orchestra.run.smithi073.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T08:29:45.576 INFO:teuthology.orchestra.run.smithi073.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T08:29:45.576 INFO:teuthology.orchestra.run.smithi073.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T08:29:45.577 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.577 INFO:teuthology.orchestra.run.smithi073.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T08:29:45.577 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T08:29:45.577 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T08:29:45.577 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.577 INFO:teuthology.orchestra.run.smithi073.stdout: # 1m isn't quite enough 2024-03-29T08:29:45.577 INFO:teuthology.orchestra.run.smithi073.stdout: mon_down_mkfs_grace = 2m 2024-03-29T08:29:45.577 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.577 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_filestore_osds = false 2024-03-29T08:29:45.577 INFO:teuthology.orchestra.run.smithi073.stdout: mon op complaint time = 120 2024-03-29T08:29:45.577 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:29:45.577 INFO:teuthology.orchestra.run.smithi073.stdout:[client] 2024-03-29T08:29:45.577 INFO:teuthology.orchestra.run.smithi073.stdout: rgw cache enabled = true 2024-03-29T08:29:45.578 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable ops log = true 2024-03-29T08:29:45.578 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable usage log = true 2024-03-29T08:29:45.578 INFO:teuthology.orchestra.run.smithi073.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T08:29:45.578 INFO:teuthology.orchestra.run.smithi073.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T08:29:45.578 INFO:teuthology.orchestra.run.smithi073.stdout: client mount timeout = 600 2024-03-29T08:29:45.578 INFO:teuthology.orchestra.run.smithi073.stdout: debug client = 20 2024-03-29T08:29:45.578 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:29:45.578 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = "" 2024-03-29T08:29:45.578 INFO:teuthology.orchestra.run.smithi073.stdout: rados mon op timeout = 900 2024-03-29T08:29:45.578 INFO:teuthology.orchestra.run.smithi073.stdout: rados osd op timeout = 900 2024-03-29T08:29:45.578 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.a] 2024-03-29T08:29:45.578 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.b] 2024-03-29T08:29:45.578 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.c] 2024-03-29T08:29:45.579 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.a] 2024-03-29T08:29:45.579 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.c] 2024-03-29T08:29:45.579 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.b] 2024-03-29T08:29:45.579 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.d] 2024-03-29T08:29:45.579 INFO:teuthology.orchestra.run.smithi073.stdout:[mds] 2024-03-29T08:29:45.579 INFO:teuthology.orchestra.run.smithi073.stdout: debug mds = 20 2024-03-29T08:29:45.579 INFO:teuthology.orchestra.run.smithi073.stdout: debug mds balancer = 20 2024-03-29T08:29:45.579 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:29:45.579 INFO:teuthology.orchestra.run.smithi073.stdout: mds debug frag = True 2024-03-29T08:29:45.579 INFO:teuthology.orchestra.run.smithi073.stdout: mds debug scatterstat = False 2024-03-29T08:29:45.579 INFO:teuthology.orchestra.run.smithi073.stdout: mds op complaint time = 180 2024-03-29T08:29:45.579 INFO:teuthology.orchestra.run.smithi073.stdout: mds verify scatter = False 2024-03-29T08:29:45.579 INFO:teuthology.orchestra.run.smithi073.stdout: osd op complaint time = 180 2024-03-29T08:29:45.579 INFO:teuthology.orchestra.run.smithi073.stdout: rados mon op timeout = 900 2024-03-29T08:29:45.579 INFO:teuthology.orchestra.run.smithi073.stdout: rados osd op timeout = 900 2024-03-29T08:29:45.581 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-03-29T08:29:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9610a18080 1 Processor -- start 2024-03-29T08:29:45.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9610a18080 1 -- start start 2024-03-29T08:29:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9610a18080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644872e10a0 0x5644873b1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9610a18080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5644872e1480 0x56448744d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9610a18080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564487314e10 0x56448744d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9610a18080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564487228960 con 0x564487314e10 2024-03-29T08:29:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9610a18080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564487228600 con 0x5644872e10a0 2024-03-29T08:29:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9610a18080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x56448721f690 con 0x5644872e1480 2024-03-29T08:29:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f960e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564487314e10 0x56448744d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f960e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564487314e10 0x56448744d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60288/0 (socket says 172.21.15.46:60288) 2024-03-29T08:29:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f960e600640 1 -- 172.21.15.46:0/75490502 learned_addr learned my addr 172.21.15.46:0/75490502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f960d200640 1 --2- 172.21.15.46:0/75490502 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5644872e1480 0x56448744d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f960dc00640 1 --2- 172.21.15.46:0/75490502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644872e10a0 0x5644873b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f960e600640 1 -- 172.21.15.46:0/75490502 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5644872e1480 msgr2=0x56448744d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:45.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f960e600640 1 --2- 172.21.15.46:0/75490502 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5644872e1480 0x56448744d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f960e600640 1 -- 172.21.15.46:0/75490502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644872e10a0 msgr2=0x5644873b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f960e600640 1 --2- 172.21.15.46:0/75490502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644872e10a0 0x5644873b1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f960e600640 1 -- 172.21.15.46:0/75490502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960800a1c0 con 0x564487314e10 2024-03-29T08:29:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f960d200640 1 --2- 172.21.15.46:0/75490502 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5644872e1480 0x56448744d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f960dc00640 1 --2- 172.21.15.46:0/75490502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644872e10a0 0x5644873b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f960e600640 1 --2- 172.21.15.46:0/75490502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564487314e10 0x56448744d880 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f960800a910 tx=0x7f960800add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0f03f63f5f5dc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9607400640 1 -- 172.21.15.46:0/75490502 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960800a400 con 0x564487314e10 2024-03-29T08:29:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9607400640 1 -- 172.21.15.46:0/75490502 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960800da80 con 0x564487314e10 2024-03-29T08:29:45.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9607400640 1 -- 172.21.15.46:0/75490502 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f960800de10 con 0x564487314e10 2024-03-29T08:29:45.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9610a18080 1 -- 172.21.15.46:0/75490502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564487314e10 msgr2=0x56448744d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:45.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9610a18080 1 --2- 172.21.15.46:0/75490502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564487314e10 0x56448744d880 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f960800a910 tx=0x7f960800add0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9610a18080 1 -- 172.21.15.46:0/75490502 shutdown_connections 2024-03-29T08:29:45.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9610a18080 1 --2- 172.21.15.46:0/75490502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564487314e10 0x56448744d880 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9610a18080 1 --2- 172.21.15.46:0/75490502 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5644872e1480 0x56448744d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9610a18080 1 --2- 172.21.15.46:0/75490502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644872e10a0 0x5644873b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9610a18080 1 -- 172.21.15.46:0/75490502 >> 172.21.15.46:0/75490502 conn(0x5644872e7550 msgr2=0x5644873b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.665+0000 7f9610a18080 1 -- 172.21.15.46:0/75490502 shutdown_connections 2024-03-29T08:29:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f9610a18080 1 -- 172.21.15.46:0/75490502 wait complete. 2024-03-29T08:29:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f9610a18080 1 Processor -- start 2024-03-29T08:29:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f9610a18080 1 -- start start 2024-03-29T08:29:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f9610a18080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644872e10a0 0x5644873295f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f9610a18080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644872e1480 0x564487327ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f9610a18080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x564487314e10 0x5644873ada10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f9610a18080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x564487328950 con 0x5644872e1480 2024-03-29T08:29:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f9610a18080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x564487328bf0 con 0x5644872e10a0 2024-03-29T08:29:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f9610a18080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x564487328e90 con 0x564487314e10 2024-03-29T08:29:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f960d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644872e1480 0x564487327ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f960d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644872e1480 0x564487327ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60298/0 (socket says 172.21.15.46:60298) 2024-03-29T08:29:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f960d200640 1 -- 172.21.15.46:0/3436703271 learned_addr learned my addr 172.21.15.46:0/3436703271 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:45.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f960e600640 1 --2- 172.21.15.46:0/3436703271 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x564487314e10 0x5644873ada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f960dc00640 1 --2- 172.21.15.46:0/3436703271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644872e10a0 0x5644873295f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f960d200640 1 -- 172.21.15.46:0/3436703271 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x564487314e10 msgr2=0x5644873ada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f960d200640 1 --2- 172.21.15.46:0/3436703271 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x564487314e10 0x5644873ada10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f960d200640 1 -- 172.21.15.46:0/3436703271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644872e10a0 msgr2=0x5644873295f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f960d200640 1 --2- 172.21.15.46:0/3436703271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644872e10a0 0x5644873295f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f960d200640 1 -- 172.21.15.46:0/3436703271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f960800a560 con 0x5644872e1480 2024-03-29T08:29:45.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f960d200640 1 --2- 172.21.15.46:0/3436703271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644872e1480 0x564487327ef0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0ae040 tx=0x5644873b92e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f9606000640 1 -- 172.21.15.46:0/3436703271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0b5410 con 0x5644872e1480 2024-03-29T08:29:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f9606000640 1 -- 172.21.15.46:0/3436703271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95fc0b5570 con 0x5644872e1480 2024-03-29T08:29:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f9606000640 1 -- 172.21.15.46:0/3436703271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0b58e0 con 0x5644872e1480 2024-03-29T08:29:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f9610a18080 1 -- 172.21.15.46:0/3436703271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc005d30 con 0x5644872e1480 2024-03-29T08:29:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.669+0000 7f9610a18080 1 -- 172.21.15.46:0/3436703271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644872378c0 con 0x5644872e1480 2024-03-29T08:29:45.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.673+0000 7f9606000640 1 -- 172.21.15.46:0/3436703271 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f95fc0b2050 con 0x5644872e1480 2024-03-29T08:29:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.673+0000 7f9606000640 1 --2- 172.21.15.46:0/3436703271 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f95f00650a0 0x7f95f0067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.673+0000 7f9606000640 1 -- 172.21.15.46:0/3436703271 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f95fc11dbc0 con 0x5644872e1480 2024-03-29T08:29:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.673+0000 7f9610a18080 1 --2- 172.21.15.46:0/3436703271 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x564487329b30 0x5644873269e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.673+0000 7f9610a18080 1 -- 172.21.15.46:0/3436703271 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v8 -- 0x564487328430 con 0x564487329b30 2024-03-29T08:29:45.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.673+0000 7f960dc00640 1 --2- 172.21.15.46:0/3436703271 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f95f00650a0 0x7f95f0067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.673+0000 7f960e600640 1 --2- 172.21.15.46:0/3436703271 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x564487329b30 0x5644873269e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.673+0000 7f960dc00640 1 --2- 172.21.15.46:0/3436703271 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f95f00650a0 0x7f95f0067560 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f960000f4f0 tx=0x7f9600001870 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:45.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.673+0000 7f960e600640 1 --2- 172.21.15.46:0/3436703271 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x564487329b30 0x5644873269e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.677+0000 7f960e600640 1 -- 172.21.15.46:0/3436703271 <== osd.4 v2:172.21.15.73:6800/27689 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v122'5 uv5 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9608018070 con 0x564487329b30 2024-03-29T08:29:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.677+0000 7f9610a18080 1 -- 172.21.15.46:0/3436703271 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x564487329b30 msgr2=0x5644873269e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:45.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.677+0000 7f9610a18080 1 --2- 172.21.15.46:0/3436703271 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x564487329b30 0x5644873269e0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.677+0000 7f9610a18080 1 -- 172.21.15.46:0/3436703271 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f95f00650a0 msgr2=0x7f95f0067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.677+0000 7f9610a18080 1 --2- 172.21.15.46:0/3436703271 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f95f00650a0 0x7f95f0067560 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f960000f4f0 tx=0x7f9600001870 comp rx=0 tx=0).stop 2024-03-29T08:29:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.677+0000 7f9610a18080 1 -- 172.21.15.46:0/3436703271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644872e1480 msgr2=0x564487327ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.677+0000 7f9610a18080 1 --2- 172.21.15.46:0/3436703271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644872e1480 0x564487327ef0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0ae040 tx=0x5644873b92e0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.677+0000 7f960e600640 1 -- 172.21.15.46:0/3436703271 reap_dead start 2024-03-29T08:29:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.677+0000 7f9610a18080 1 -- 172.21.15.46:0/3436703271 shutdown_connections 2024-03-29T08:29:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.677+0000 7f9610a18080 1 -- 172.21.15.46:0/3436703271 >> 172.21.15.46:0/3436703271 conn(0x5644872e7550 msgr2=0x564487312350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.677+0000 7f9610a18080 1 -- 172.21.15.46:0/3436703271 shutdown_connections 2024-03-29T08:29:45.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.677+0000 7f9610a18080 1 -- 172.21.15.46:0/3436703271 wait complete. 2024-03-29T08:29:45.685 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-29T08:29:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.825+0000 7f609a600640 1 Processor -- start 2024-03-29T08:29:45.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.825+0000 7f609a600640 1 -- start start 2024-03-29T08:29:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.825+0000 7f609a600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6094142a50 0x7f60940cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.825+0000 7f609a600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60940ae990 0x7f60940cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.825+0000 7f609a600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f609415d260 con 0x7f609415df90 2024-03-29T08:29:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.825+0000 7f609a600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60940aefd0 con 0x7f60940ae990 2024-03-29T08:29:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.825+0000 7f609a600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609415d4b0 con 0x7f6094142a50 2024-03-29T08:29:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.825+0000 7f6093e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60940ae990 0x7f60940cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.825+0000 7f6093e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60940ae990 0x7f60940cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60312/0 (socket says 172.21.15.46:60312) 2024-03-29T08:29:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.825+0000 7f6093e00640 1 -- 172.21.15.46:0/1151595906 learned_addr learned my addr 172.21.15.46:0/1151595906 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.825+0000 7f6093400640 1 --2- 172.21.15.46:0/1151595906 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6094142a50 0x7f60940cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.825+0000 7f6093e00640 1 -- 172.21.15.46:0/1151595906 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6094142a50 msgr2=0x7f60940cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.825+0000 7f6093e00640 1 --2- 172.21.15.46:0/1151595906 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6094142a50 0x7f60940cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.825+0000 7f6093e00640 1 -- 172.21.15.46:0/1151595906 >> v1:172.21.15.73:6789/0 conn(0x7f609415df90 legacy=0x7f60940ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:45.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.825+0000 7f6093e00640 1 -- 172.21.15.46:0/1151595906 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609415d790 con 0x7f60940ae990 2024-03-29T08:29:45.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.829+0000 7f6093e00640 1 --2- 172.21.15.46:0/1151595906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60940ae990 0x7f60940cbcd0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f608c009e50 tx=0x7f608c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69503e3a810ba0ae server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:45.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.829+0000 7f6092000640 1 -- 172.21.15.46:0/1151595906 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c0a5660 con 0x7f60940ae990 2024-03-29T08:29:45.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.829+0000 7f6092000640 1 -- 172.21.15.46:0/1151595906 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608c0a57c0 con 0x7f60940ae990 2024-03-29T08:29:45.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.829+0000 7f6092000640 1 -- 172.21.15.46:0/1151595906 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608c0a5b50 con 0x7f60940ae990 2024-03-29T08:29:45.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.829+0000 7f609a600640 1 -- 172.21.15.46:0/1151595906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60940ae990 msgr2=0x7f60940cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:45.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.829+0000 7f609a600640 1 --2- 172.21.15.46:0/1151595906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60940ae990 0x7f60940cbcd0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f608c009e50 tx=0x7f608c01c9a0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.829+0000 7f609a600640 1 -- 172.21.15.46:0/1151595906 shutdown_connections 2024-03-29T08:29:45.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.829+0000 7f609a600640 1 --2- 172.21.15.46:0/1151595906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60940ae990 0x7f60940cbcd0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.829+0000 7f609a600640 1 --2- 172.21.15.46:0/1151595906 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6094142a50 0x7f60940cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.829+0000 7f609a600640 1 -- 172.21.15.46:0/1151595906 >> 172.21.15.46:0/1151595906 conn(0x7f609406a390 msgr2=0x7f60940cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:45.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.829+0000 7f609a600640 1 -- 172.21.15.46:0/1151595906 shutdown_connections 2024-03-29T08:29:45.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.829+0000 7f609a600640 1 -- 172.21.15.46:0/1151595906 wait complete. 2024-03-29T08:29:45.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.829+0000 7f609a600640 1 Processor -- start 2024-03-29T08:29:45.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.829+0000 7f609a600640 1 -- start start 2024-03-29T08:29:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f609a600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60940ae990 0x7f609415a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f609a600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6094142a50 0x7f609413bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f609a600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f609415df90 0x7f60940b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f609a600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6094141cc0 con 0x7f60940ae990 2024-03-29T08:29:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f609a600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60941f5520 con 0x7f609415df90 2024-03-29T08:29:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f609a600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609415a840 con 0x7f6094142a50 2024-03-29T08:29:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f6093400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60940ae990 0x7f609415a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f6093400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60940ae990 0x7f609415a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60318/0 (socket says 172.21.15.46:60318) 2024-03-29T08:29:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f6093400640 1 -- 172.21.15.46:0/28210827 learned_addr learned my addr 172.21.15.46:0/28210827 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f6092a00640 1 --2- 172.21.15.46:0/28210827 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6094142a50 0x7f609413bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f6093e00640 1 --2- 172.21.15.46:0/28210827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f609415df90 0x7f60940b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f6093400640 1 -- 172.21.15.46:0/28210827 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6094142a50 msgr2=0x7f609413bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f6093400640 1 --2- 172.21.15.46:0/28210827 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6094142a50 0x7f609413bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f6093400640 1 -- 172.21.15.46:0/28210827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f609415df90 msgr2=0x7f60940b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f6093400640 1 --2- 172.21.15.46:0/28210827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f609415df90 0x7f60940b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f6093400640 1 -- 172.21.15.46:0/28210827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608c009b30 con 0x7f60940ae990 2024-03-29T08:29:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f6093400640 1 --2- 172.21.15.46:0/28210827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60940ae990 0x7f609415a1b0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f608400d800 tx=0x7f608400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f608b400640 1 -- 172.21.15.46:0/28210827 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608400b6c0 con 0x7f60940ae990 2024-03-29T08:29:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f608b400640 1 -- 172.21.15.46:0/28210827 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608400b820 con 0x7f60940ae990 2024-03-29T08:29:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f609a600640 1 -- 172.21.15.46:0/28210827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609415ab20 con 0x7f60940ae990 2024-03-29T08:29:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f608b400640 1 -- 172.21.15.46:0/28210827 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608400bb90 con 0x7f60940ae990 2024-03-29T08:29:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f609a600640 1 -- 172.21.15.46:0/28210827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609415b0e0 con 0x7f60940ae990 2024-03-29T08:29:45.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.833+0000 7f609a600640 1 -- 172.21.15.46:0/28210827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6094071a70 con 0x7f60940ae990 2024-03-29T08:29:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.837+0000 7f608b400640 1 -- 172.21.15.46:0/28210827 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f608400bcf0 con 0x7f60940ae990 2024-03-29T08:29:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.837+0000 7f608b400640 1 --2- 172.21.15.46:0/28210827 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6074066690 0x7f6074068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.837+0000 7f608b400640 1 -- 172.21.15.46:0/28210827 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6084084700 con 0x7f60940ae990 2024-03-29T08:29:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.837+0000 7f6092a00640 1 --2- 172.21.15.46:0/28210827 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6074066690 0x7f6074068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:45.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.837+0000 7f6092a00640 1 --2- 172.21.15.46:0/28210827 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6074066690 0x7f6074068b50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6094142240 tx=0x7f6080009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:45.841+0000 7f608b400640 1 -- 172.21.15.46:0/28210827 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f6084051a20 con 0x7f60940ae990 2024-03-29T08:29:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.037+0000 7f609a600640 1 -- 172.21.15.46:0/28210827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f6094159f90 con 0x7f60940ae990 2024-03-29T08:29:46.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.377+0000 7f608b400640 1 -- 172.21.15.46:0/28210827 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v96) v1 ==== 162+0+0 (secure 0 0 0) 0x7f6084055040 con 0x7f60940ae990 2024-03-29T08:29:46.384 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-29T08:29:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.381+0000 7f609a600640 1 -- 172.21.15.46:0/28210827 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6074066690 msgr2=0x7f6074068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.381+0000 7f609a600640 1 --2- 172.21.15.46:0/28210827 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6074066690 0x7f6074068b50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f6094142240 tx=0x7f6080009040 comp rx=0 tx=0).stop 2024-03-29T08:29:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.381+0000 7f609a600640 1 -- 172.21.15.46:0/28210827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60940ae990 msgr2=0x7f609415a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.381+0000 7f609a600640 1 --2- 172.21.15.46:0/28210827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60940ae990 0x7f609415a1b0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f608400d800 tx=0x7f608400dcc0 comp rx=0 tx=0).stop 2024-03-29T08:29:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.381+0000 7f609a600640 1 -- 172.21.15.46:0/28210827 shutdown_connections 2024-03-29T08:29:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.381+0000 7f609a600640 1 --2- 172.21.15.46:0/28210827 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6074066690 0x7f6074068b50 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.381+0000 7f609a600640 1 --2- 172.21.15.46:0/28210827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f609415df90 0x7f60940b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:46.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.381+0000 7f609a600640 1 --2- 172.21.15.46:0/28210827 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f6094142a50 0x7f609413bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.381+0000 7f609a600640 1 --2- 172.21.15.46:0/28210827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60940ae990 0x7f609415a1b0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.381+0000 7f609a600640 1 -- 172.21.15.46:0/28210827 >> 172.21.15.46:0/28210827 conn(0x7f609406a390 msgr2=0x7f60940b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.381+0000 7f609a600640 1 -- 172.21.15.46:0/28210827 shutdown_connections 2024-03-29T08:29:46.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.381+0000 7f609a600640 1 -- 172.21.15.46:0/28210827 wait complete. 2024-03-29T08:29:46.412 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:29:46.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.553+0000 7f38da800640 1 Processor -- start 2024-03-29T08:29:46.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.553+0000 7f38da800640 1 -- start start 2024-03-29T08:29:46.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.553+0000 7f38da800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f38d415cd90 0x7f38d406cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:46.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.553+0000 7f38da800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f38d415d940 0x7f38d41597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:46.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.553+0000 7f38da800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d406d130 con 0x7f38d41593d0 2024-03-29T08:29:46.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.553+0000 7f38da800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d415b8f0 con 0x7f38d415d940 2024-03-29T08:29:46.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.553+0000 7f38da800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d415ad50 con 0x7f38d415cd90 2024-03-29T08:29:46.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.553+0000 7f38d3400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f38d41593d0 0x7f38d415d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42136/0 (socket says 172.21.15.46:42136) 2024-03-29T08:29:46.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.553+0000 7f38d3400640 1 -- 172.21.15.46:0/2787224501 learned_addr learned my addr 172.21.15.46:0/2787224501 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:46.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.553+0000 7f38d3e00640 1 --2- 172.21.15.46:0/2787224501 >> v2:172.21.15.46:3300/0 conn(0x7f38d415d940 0x7f38d41597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:46.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.553+0000 7f38d2a00640 1 --2- 172.21.15.46:0/2787224501 >> v2:172.21.15.73:3300/0 conn(0x7f38d415cd90 0x7f38d406cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:46.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.553+0000 7f38d3e00640 1 -- 172.21.15.46:0/2787224501 >> v2:172.21.15.73:3300/0 conn(0x7f38d415cd90 msgr2=0x7f38d406cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:46.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.553+0000 7f38d3e00640 1 --2- 172.21.15.46:0/2787224501 >> v2:172.21.15.73:3300/0 conn(0x7f38d415cd90 0x7f38d406cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:46.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.553+0000 7f38d3e00640 1 -- 172.21.15.46:0/2787224501 >> v1:172.21.15.46:6789/0 conn(0x7f38d41593d0 legacy=0x7f38d415d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:46.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.553+0000 7f38d3e00640 1 -- 172.21.15.46:0/2787224501 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d415afc0 con 0x7f38d415d940 2024-03-29T08:29:46.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.553+0000 7f38d2000640 1 -- 172.21.15.46:0/2787224501 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3860484341 0 0) 0x7f38d406d130 con 0x7f38d41593d0 2024-03-29T08:29:46.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.557+0000 7f38d3e00640 1 --2- 172.21.15.46:0/2787224501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d415d940 0x7f38d41597b0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f38d40cf180 tx=0x7f38cc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16eb45a384a524ec server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:46.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.557+0000 7f38d2000640 1 -- 172.21.15.46:0/2787224501 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc0a59f0 con 0x7f38d415d940 2024-03-29T08:29:46.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.557+0000 7f38d2000640 1 -- 172.21.15.46:0/2787224501 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc0a5b50 con 0x7f38d415d940 2024-03-29T08:29:46.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.557+0000 7f38d2000640 1 -- 172.21.15.46:0/2787224501 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc0ad440 con 0x7f38d415d940 2024-03-29T08:29:46.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.557+0000 7f38da800640 1 -- 172.21.15.46:0/2787224501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d415d940 msgr2=0x7f38d41597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:46.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.557+0000 7f38da800640 1 --2- 172.21.15.46:0/2787224501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d415d940 0x7f38d41597b0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f38d40cf180 tx=0x7f38cc01c930 comp rx=0 tx=0).stop 2024-03-29T08:29:46.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.557+0000 7f38da800640 1 -- 172.21.15.46:0/2787224501 shutdown_connections 2024-03-29T08:29:46.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.557+0000 7f38da800640 1 --2- 172.21.15.46:0/2787224501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d415d940 0x7f38d41597b0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:46.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.557+0000 7f38da800640 1 --2- 172.21.15.46:0/2787224501 >> v2:172.21.15.73:3300/0 conn(0x7f38d415cd90 0x7f38d406cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:46.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.557+0000 7f38da800640 1 -- 172.21.15.46:0/2787224501 >> 172.21.15.46:0/2787224501 conn(0x7f38d4066110 msgr2=0x7f38d4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:46.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.557+0000 7f38da800640 1 -- 172.21.15.46:0/2787224501 shutdown_connections 2024-03-29T08:29:46.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.557+0000 7f38da800640 1 -- 172.21.15.46:0/2787224501 wait complete. 2024-03-29T08:29:46.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38da800640 1 Processor -- start 2024-03-29T08:29:46.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38da800640 1 -- start start 2024-03-29T08:29:46.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38da800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d41593d0 0x7f38d40ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:46.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38da800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38d415cd90 0x7f38d414ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:46.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38da800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d415d940 0x7f38d415edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:46.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38da800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d4163ee0 con 0x7f38d415d940 2024-03-29T08:29:46.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38da800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d41563f0 con 0x7f38d41593d0 2024-03-29T08:29:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38da800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d41566d0 con 0x7f38d415cd90 2024-03-29T08:29:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d415d940 0x7f38d415edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38d3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d415d940 0x7f38d415edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57172/0 (socket says 172.21.15.46:57172) 2024-03-29T08:29:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38d3e00640 1 -- 172.21.15.46:0/3380920483 learned_addr learned my addr 172.21.15.46:0/3380920483 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38d2a00640 1 --2- 172.21.15.46:0/3380920483 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38d415cd90 0x7f38d414ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38d3400640 1 --2- 172.21.15.46:0/3380920483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d41593d0 0x7f38d40ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38d3e00640 1 -- 172.21.15.46:0/3380920483 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38d415cd90 msgr2=0x7f38d414ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38d3e00640 1 --2- 172.21.15.46:0/3380920483 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38d415cd90 0x7f38d414ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38d3e00640 1 -- 172.21.15.46:0/3380920483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d41593d0 msgr2=0x7f38d40ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38d3e00640 1 --2- 172.21.15.46:0/3380920483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d41593d0 0x7f38d40ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38d3e00640 1 -- 172.21.15.46:0/3380920483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc009a70 con 0x7f38d415d940 2024-03-29T08:29:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38d3e00640 1 --2- 172.21.15.46:0/3380920483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d415d940 0x7f38d415edc0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f38cc019540 tx=0x7f38cc0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38cb400640 1 -- 172.21.15.46:0/3380920483 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc0ade00 con 0x7f38d415d940 2024-03-29T08:29:46.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38cb400640 1 -- 172.21.15.46:0/3380920483 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc009450 con 0x7f38d415d940 2024-03-29T08:29:46.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38da800640 1 -- 172.21.15.46:0/3380920483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d41569b0 con 0x7f38d415d940 2024-03-29T08:29:46.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38cb400640 1 -- 172.21.15.46:0/3380920483 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc0b7650 con 0x7f38d415d940 2024-03-29T08:29:46.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38da800640 1 -- 172.21.15.46:0/3380920483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d4156f70 con 0x7f38d415d940 2024-03-29T08:29:46.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.561+0000 7f38da800640 1 -- 172.21.15.46:0/3380920483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d4066c90 con 0x7f38d415d940 2024-03-29T08:29:46.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.565+0000 7f38cb400640 1 -- 172.21.15.46:0/3380920483 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f38cc0b4050 con 0x7f38d415d940 2024-03-29T08:29:46.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.565+0000 7f38cb400640 1 --2- 172.21.15.46:0/3380920483 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38b4066690 0x7f38b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:46.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.565+0000 7f38cb400640 1 -- 172.21.15.46:0/3380920483 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f38cc11f3d0 con 0x7f38d415d940 2024-03-29T08:29:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.565+0000 7f38d3400640 1 --2- 172.21.15.46:0/3380920483 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38b4066690 0x7f38b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:46.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.565+0000 7f38d3400640 1 --2- 172.21.15.46:0/3380920483 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38b4066690 0x7f38b4068b50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f38c4007fd0 tx=0x7f38c4007d60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:46.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.569+0000 7f38cb400640 1 -- 172.21.15.46:0/3380920483 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f38cc0ec780 con 0x7f38d415d940 2024-03-29T08:29:46.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.761+0000 7f38da800640 1 -- 172.21.15.46:0/3380920483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38d41643a0 con 0x7f38d415d940 2024-03-29T08:29:46.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.761+0000 7f38cb400640 1 -- 172.21.15.46:0/3380920483 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f38cc0efda0 con 0x7f38d415d940 2024-03-29T08:29:46.767 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:46.767 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":97,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2506132630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2506132630},{"type":"v1","addr":"172.21.15.73:6836","nonce":2506132630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2108115501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2108115501},{"type":"v1","addr":"172.21.15.73:6837","nonce":2108115501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":9045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3210062895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3210062895},{"type":"v1","addr":"172.21.15.46:6833","nonce":3210062895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:29:14.470858+0000","modified":"2024-03-29T08:29:46.385303+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9441},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9441":{"gid":9441,"name":"a","rank":0,"incarnation":97,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6835/276185207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":276185207},{"type":"v1","addr":"172.21.15.46:6835","nonce":276185207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-29T08:29:46.769 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 97 2024-03-29T08:29:46.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.765+0000 7f38da800640 1 -- 172.21.15.46:0/3380920483 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38b4066690 msgr2=0x7f38b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:46.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.765+0000 7f38da800640 1 --2- 172.21.15.46:0/3380920483 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38b4066690 0x7f38b4068b50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f38c4007fd0 tx=0x7f38c4007d60 comp rx=0 tx=0).stop 2024-03-29T08:29:46.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.765+0000 7f38da800640 1 -- 172.21.15.46:0/3380920483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d415d940 msgr2=0x7f38d415edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:46.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.765+0000 7f38da800640 1 --2- 172.21.15.46:0/3380920483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d415d940 0x7f38d415edc0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f38cc019540 tx=0x7f38cc0196d0 comp rx=0 tx=0).stop 2024-03-29T08:29:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.765+0000 7f38da800640 1 -- 172.21.15.46:0/3380920483 shutdown_connections 2024-03-29T08:29:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.765+0000 7f38da800640 1 --2- 172.21.15.46:0/3380920483 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f38b4066690 0x7f38b4068b50 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.765+0000 7f38da800640 1 --2- 172.21.15.46:0/3380920483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38d415d940 0x7f38d415edc0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.765+0000 7f38da800640 1 --2- 172.21.15.46:0/3380920483 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f38d415cd90 0x7f38d414ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.765+0000 7f38da800640 1 --2- 172.21.15.46:0/3380920483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d41593d0 0x7f38d40ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.765+0000 7f38da800640 1 -- 172.21.15.46:0/3380920483 >> 172.21.15.46:0/3380920483 conn(0x7f38d4066110 msgr2=0x7f38d414f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.765+0000 7f38da800640 1 -- 172.21.15.46:0/3380920483 shutdown_connections 2024-03-29T08:29:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:46.765+0000 7f38da800640 1 -- 172.21.15.46:0/3380920483 wait complete. 2024-03-29T08:29:47.189 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 97, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:29:14.470858+0000', 'modified': '2024-03-29T08:29:46.385303+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9441}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9441': {'gid': 9441, 'name': 'a', 'rank': 0, 'incarnation': 97, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.46:6835/276185207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 276185207}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 276185207}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:29:47.190 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9441:up:replay 2024-03-29T08:29:48.191 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:29:48.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.337+0000 7f69ce800640 1 Processor -- start 2024-03-29T08:29:48.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.337+0000 7f69ce800640 1 -- start start 2024-03-29T08:29:48.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.341+0000 7f69ce800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f69c80ccd10 0x7f69c8098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:48.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.341+0000 7f69ce800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f69c815cdb0 0x7f69c8098f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:48.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.341+0000 7f69ce800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c80d0030 con 0x7f69c815c9d0 2024-03-29T08:29:48.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.341+0000 7f69ce800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c80cf420 con 0x7f69c815cdb0 2024-03-29T08:29:48.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.341+0000 7f69ce800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c80cf700 con 0x7f69c80ccd10 2024-03-29T08:29:48.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.341+0000 7f69c7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f69c815cdb0 0x7f69c8098f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:48.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.341+0000 7f69c7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f69c815cdb0 0x7f69c8098f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57186/0 (socket says 172.21.15.46:57186) 2024-03-29T08:29:48.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.341+0000 7f69c7e00640 1 -- 172.21.15.46:0/1019889595 learned_addr learned my addr 172.21.15.46:0/1019889595 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:48.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.341+0000 7f69bfe00640 1 --2- 172.21.15.46:0/1019889595 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f69c80ccd10 0x7f69c8098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:48.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.341+0000 7f69c7e00640 1 -- 172.21.15.46:0/1019889595 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f69c80ccd10 msgr2=0x7f69c8098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:48.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.341+0000 7f69c7e00640 1 --2- 172.21.15.46:0/1019889595 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f69c80ccd10 0x7f69c8098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:48.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.341+0000 7f69c7e00640 1 -- 172.21.15.46:0/1019889595 >> v1:172.21.15.73:6789/0 conn(0x7f69c815c9d0 legacy=0x7f69c80cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:48.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.341+0000 7f69c7e00640 1 -- 172.21.15.46:0/1019889595 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c80cf9e0 con 0x7f69c815cdb0 2024-03-29T08:29:48.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.341+0000 7f69c7e00640 1 --2- 172.21.15.46:0/1019889595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c815cdb0 0x7f69c8098f80 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f69c0009e50 tx=0x7f69c001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaf01bb1c79798ee server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:48.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69c6a00640 1 -- 172.21.15.46:0/1019889595 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c00a58b0 con 0x7f69c815cdb0 2024-03-29T08:29:48.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69c6a00640 1 -- 172.21.15.46:0/1019889595 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c00a5a10 con 0x7f69c815cdb0 2024-03-29T08:29:48.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69c6a00640 1 -- 172.21.15.46:0/1019889595 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c00a5da0 con 0x7f69c815cdb0 2024-03-29T08:29:48.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69ce800640 1 -- 172.21.15.46:0/1019889595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c815cdb0 msgr2=0x7f69c8098f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:48.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69ce800640 1 --2- 172.21.15.46:0/1019889595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c815cdb0 0x7f69c8098f80 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f69c0009e50 tx=0x7f69c001ca00 comp rx=0 tx=0).stop 2024-03-29T08:29:48.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69ce800640 1 -- 172.21.15.46:0/1019889595 shutdown_connections 2024-03-29T08:29:48.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69ce800640 1 --2- 172.21.15.46:0/1019889595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c815cdb0 0x7f69c8098f80 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:48.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69ce800640 1 --2- 172.21.15.46:0/1019889595 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f69c80ccd10 0x7f69c8098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:48.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69ce800640 1 -- 172.21.15.46:0/1019889595 >> 172.21.15.46:0/1019889595 conn(0x7f69c80b21a0 msgr2=0x7f69c80b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:48.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69ce800640 1 -- 172.21.15.46:0/1019889595 shutdown_connections 2024-03-29T08:29:48.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69ce800640 1 -- 172.21.15.46:0/1019889595 wait complete. 2024-03-29T08:29:48.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69ce800640 1 Processor -- start 2024-03-29T08:29:48.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69ce800640 1 -- start start 2024-03-29T08:29:48.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69ce800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c80ccd10 0x7f69c80cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:48.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69ce800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f69c815c9d0 0x7f69c80c6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:48.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69ce800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69c815cdb0 0x7f69c814b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:48.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69ce800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c814a420 con 0x7f69c80ccd10 2024-03-29T08:29:48.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69ce800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c814a5e0 con 0x7f69c815cdb0 2024-03-29T08:29:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69ce800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c80ca420 con 0x7f69c815c9d0 2024-03-29T08:29:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c80ccd10 0x7f69c80cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69c7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c80ccd10 0x7f69c80cb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57202/0 (socket says 172.21.15.46:57202) 2024-03-29T08:29:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69c7400640 1 -- 172.21.15.46:0/3268741363 learned_addr learned my addr 172.21.15.46:0/3268741363 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69c7e00640 1 --2- 172.21.15.46:0/3268741363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69c815cdb0 0x7f69c814b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.345+0000 7f69bfe00640 1 --2- 172.21.15.46:0/3268741363 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f69c815c9d0 0x7f69c80c6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.349+0000 7f69c7400640 1 -- 172.21.15.46:0/3268741363 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f69c815c9d0 msgr2=0x7f69c80c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.349+0000 7f69c7400640 1 --2- 172.21.15.46:0/3268741363 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f69c815c9d0 0x7f69c80c6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.349+0000 7f69c7400640 1 -- 172.21.15.46:0/3268741363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69c815cdb0 msgr2=0x7f69c814b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.349+0000 7f69c7400640 1 --2- 172.21.15.46:0/3268741363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69c815cdb0 0x7f69c814b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.349+0000 7f69c7400640 1 -- 172.21.15.46:0/3268741363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c0009b30 con 0x7f69c80ccd10 2024-03-29T08:29:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.349+0000 7f69c7400640 1 --2- 172.21.15.46:0/3268741363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c80ccd10 0x7f69c80cb960 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f69b800c500 tx=0x7f69b800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.349+0000 7f69bf400640 1 -- 172.21.15.46:0/3268741363 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8005220 con 0x7f69c80ccd10 2024-03-29T08:29:48.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.349+0000 7f69bf400640 1 -- 172.21.15.46:0/3268741363 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b8005380 con 0x7f69c80ccd10 2024-03-29T08:29:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.349+0000 7f69bf400640 1 -- 172.21.15.46:0/3268741363 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8007270 con 0x7f69c80ccd10 2024-03-29T08:29:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.349+0000 7f69ce800640 1 -- 172.21.15.46:0/3268741363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c814a830 con 0x7f69c80ccd10 2024-03-29T08:29:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.349+0000 7f69ce800640 1 -- 172.21.15.46:0/3268741363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c814adf0 con 0x7f69c80ccd10 2024-03-29T08:29:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.349+0000 7f69ce800640 1 -- 172.21.15.46:0/3268741363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c8066c90 con 0x7f69c80ccd10 2024-03-29T08:29:48.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.349+0000 7f69bf400640 1 -- 172.21.15.46:0/3268741363 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f69b80073d0 con 0x7f69c80ccd10 2024-03-29T08:29:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.349+0000 7f69bf400640 1 --2- 172.21.15.46:0/3268741363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f69a4066690 0x7f69a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.349+0000 7f69bf400640 1 -- 172.21.15.46:0/3268741363 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f69b8083a40 con 0x7f69c80ccd10 2024-03-29T08:29:48.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.349+0000 7f69bfe00640 1 --2- 172.21.15.46:0/3268741363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f69a4066690 0x7f69a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:48.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.353+0000 7f69bfe00640 1 --2- 172.21.15.46:0/3268741363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f69a4066690 0x7f69a4068b50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f69c8072ee0 tx=0x7f69b0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:48.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.357+0000 7f69bf400640 1 -- 172.21.15.46:0/3268741363 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f69b8050d70 con 0x7f69c80ccd10 2024-03-29T08:29:48.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.549+0000 7f69ce800640 1 -- 172.21.15.46:0/3268741363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69c80b2ec0 con 0x7f69c80ccd10 2024-03-29T08:29:48.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.549+0000 7f69bf400640 1 -- 172.21.15.46:0/3268741363 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4735 (secure 0 0 0) 0x7f69b8054390 con 0x7f69c80ccd10 2024-03-29T08:29:48.555 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:48.555 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":99,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2506132630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2506132630},{"type":"v1","addr":"172.21.15.73:6836","nonce":2506132630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2108115501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2108115501},{"type":"v1","addr":"172.21.15.73:6837","nonce":2108115501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":9045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3210062895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3210062895},{"type":"v1","addr":"172.21.15.46:6833","nonce":3210062895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:29:14.470858+0000","modified":"2024-03-29T08:29:48.391515+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9441},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9441":{"gid":9441,"name":"a","rank":0,"incarnation":97,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.46:6835/276185207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":276185207},{"type":"v1","addr":"172.21.15.46:6835","nonce":276185207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-29T08:29:48.555 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 99 2024-03-29T08:29:48.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.553+0000 7f69ce800640 1 -- 172.21.15.46:0/3268741363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f69a4066690 msgr2=0x7f69a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:48.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.553+0000 7f69ce800640 1 --2- 172.21.15.46:0/3268741363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f69a4066690 0x7f69a4068b50 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f69c8072ee0 tx=0x7f69b0008040 comp rx=0 tx=0).stop 2024-03-29T08:29:48.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.553+0000 7f69ce800640 1 -- 172.21.15.46:0/3268741363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c80ccd10 msgr2=0x7f69c80cb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:48.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.553+0000 7f69ce800640 1 --2- 172.21.15.46:0/3268741363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c80ccd10 0x7f69c80cb960 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f69b800c500 tx=0x7f69b800c9c0 comp rx=0 tx=0).stop 2024-03-29T08:29:48.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.553+0000 7f69ce800640 1 -- 172.21.15.46:0/3268741363 shutdown_connections 2024-03-29T08:29:48.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.553+0000 7f69ce800640 1 --2- 172.21.15.46:0/3268741363 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f69a4066690 0x7f69a4068b50 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:48.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.553+0000 7f69ce800640 1 --2- 172.21.15.46:0/3268741363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69c815cdb0 0x7f69c814b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:48.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.553+0000 7f69ce800640 1 --2- 172.21.15.46:0/3268741363 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f69c815c9d0 0x7f69c80c6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:48.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.553+0000 7f69ce800640 1 --2- 172.21.15.46:0/3268741363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69c80ccd10 0x7f69c80cb960 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:48.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.553+0000 7f69ce800640 1 -- 172.21.15.46:0/3268741363 >> 172.21.15.46:0/3268741363 conn(0x7f69c80b21a0 msgr2=0x7f69c806b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:48.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.553+0000 7f69ce800640 1 -- 172.21.15.46:0/3268741363 shutdown_connections 2024-03-29T08:29:48.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:48.553+0000 7f69ce800640 1 -- 172.21.15.46:0/3268741363 wait complete. 2024-03-29T08:29:48.980 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:29:14.470858+0000', 'modified': '2024-03-29T08:29:48.391515+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9441}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9441': {'gid': 9441, 'name': 'a', 'rank': 0, 'incarnation': 97, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.46:6835/276185207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 276185207}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 276185207}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:29:48.980 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9441:up:rejoin 2024-03-29T08:29:49.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:29:50.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.129+0000 7ffaaf200640 1 Processor -- start 2024-03-29T08:29:50.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.129+0000 7ffaaf200640 1 -- start start 2024-03-29T08:29:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.129+0000 7ffaaf200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffaa815a7c0 0x7ffaa806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.129+0000 7ffaaf200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa806d130 con 0x7ffaa815cd90 2024-03-29T08:29:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.129+0000 7ffaaf200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa815b8f0 con 0x7ffaa81593d0 2024-03-29T08:29:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.129+0000 7ffaaf200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa815ad50 con 0x7ffaa815a7c0 2024-03-29T08:29:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.129+0000 7ffaad200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ffaa815a7c0 0x7ffaa806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.129+0000 7ffa9f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ffaa815cd90 0x7ffaa815d940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42142/0 (socket says 172.21.15.46:42142) 2024-03-29T08:29:50.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.129+0000 7ffa9f400640 1 -- 172.21.15.46:0/3651373652 learned_addr learned my addr 172.21.15.46:0/3651373652 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:50.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.129+0000 7ffaad200640 1 -- 172.21.15.46:0/3651373652 >> v1:172.21.15.73:6789/0 conn(0x7ffaa81593d0 legacy=0x7ffaa815d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.129+0000 7ffaad200640 1 -- 172.21.15.46:0/3651373652 >> v1:172.21.15.46:6789/0 conn(0x7ffaa815cd90 legacy=0x7ffaa815d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.129+0000 7ffaad200640 1 -- 172.21.15.46:0/3651373652 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa815e050 con 0x7ffaa815a7c0 2024-03-29T08:29:50.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.133+0000 7ffaad200640 1 --2- 172.21.15.46:0/3651373652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa815a7c0 0x7ffaa806cbf0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00097c0 tx=0x7ffaa001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb90fbeaa906c13 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:50.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.133+0000 7ffaa7400640 1 -- 172.21.15.46:0/3651373652 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa00a6040 con 0x7ffaa815a7c0 2024-03-29T08:29:50.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.133+0000 7ffaa7400640 1 -- 172.21.15.46:0/3651373652 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa00aa600 con 0x7ffaa815a7c0 2024-03-29T08:29:50.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.133+0000 7ffaa7400640 1 -- 172.21.15.46:0/3651373652 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa00aa990 con 0x7ffaa815a7c0 2024-03-29T08:29:50.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.133+0000 7ffaaf200640 1 -- 172.21.15.46:0/3651373652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa815a7c0 msgr2=0x7ffaa806cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.133+0000 7ffaaf200640 1 --2- 172.21.15.46:0/3651373652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa815a7c0 0x7ffaa806cbf0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00097c0 tx=0x7ffaa001c4a0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.133+0000 7ffaaf200640 1 -- 172.21.15.46:0/3651373652 shutdown_connections 2024-03-29T08:29:50.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.133+0000 7ffaaf200640 1 --2- 172.21.15.46:0/3651373652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa815a7c0 0x7ffaa806cbf0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.133+0000 7ffaaf200640 1 -- 172.21.15.46:0/3651373652 >> 172.21.15.46:0/3651373652 conn(0x7ffaa8066110 msgr2=0x7ffaa8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:50.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.133+0000 7ffaaf200640 1 -- 172.21.15.46:0/3651373652 shutdown_connections 2024-03-29T08:29:50.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.133+0000 7ffaaf200640 1 -- 172.21.15.46:0/3651373652 wait complete. 2024-03-29T08:29:50.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaaf200640 1 Processor -- start 2024-03-29T08:29:50.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaaf200640 1 -- start start 2024-03-29T08:29:50.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaaf200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffaa81593d0 0x7ffaa81f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaaf200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaa815a7c0 0x7ffaa81579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaaf200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa815cd90 0x7ffaa8155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaaf200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa81f39a0 con 0x7ffaa815cd90 2024-03-29T08:29:50.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaaf200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa81532b0 con 0x7ffaa815a7c0 2024-03-29T08:29:50.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaaf200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa8153590 con 0x7ffaa81593d0 2024-03-29T08:29:50.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaad200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa815cd90 0x7ffaa8155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaad200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa815cd90 0x7ffaa8155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57220/0 (socket says 172.21.15.46:57220) 2024-03-29T08:29:50.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaad200640 1 -- 172.21.15.46:0/1613820744 learned_addr learned my addr 172.21.15.46:0/1613820744 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:50.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffa9f400640 1 --2- 172.21.15.46:0/1613820744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaa815a7c0 0x7ffaa81579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaa7e00640 1 --2- 172.21.15.46:0/1613820744 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffaa81593d0 0x7ffaa81f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaad200640 1 -- 172.21.15.46:0/1613820744 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffaa81593d0 msgr2=0x7ffaa81f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaad200640 1 --2- 172.21.15.46:0/1613820744 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffaa81593d0 0x7ffaa81f3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaad200640 1 -- 172.21.15.46:0/1613820744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaa815a7c0 msgr2=0x7ffaa81579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaad200640 1 --2- 172.21.15.46:0/1613820744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaa815a7c0 0x7ffaa81579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaad200640 1 -- 172.21.15.46:0/1613820744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa0009420 con 0x7ffaa815cd90 2024-03-29T08:29:50.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaad200640 1 --2- 172.21.15.46:0/1613820744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa815cd90 0x7ffaa8155420 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00190b0 tx=0x7ffaa001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:50.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaa4c00640 1 -- 172.21.15.46:0/1613820744 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa00b3600 con 0x7ffaa815cd90 2024-03-29T08:29:50.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaa4c00640 1 -- 172.21.15.46:0/1613820744 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa00b3760 con 0x7ffaa815cd90 2024-03-29T08:29:50.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaaf200640 1 -- 172.21.15.46:0/1613820744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa8153870 con 0x7ffaa815cd90 2024-03-29T08:29:50.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaa4c00640 1 -- 172.21.15.46:0/1613820744 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa00b3af0 con 0x7ffaa815cd90 2024-03-29T08:29:50.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaaf200640 1 -- 172.21.15.46:0/1613820744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa8156190 con 0x7ffaa815cd90 2024-03-29T08:29:50.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.137+0000 7ffaaf200640 1 -- 172.21.15.46:0/1613820744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa8153d90 con 0x7ffaa815cd90 2024-03-29T08:29:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.141+0000 7ffaa4c00640 1 -- 172.21.15.46:0/1613820744 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ffaa00b3c50 con 0x7ffaa815cd90 2024-03-29T08:29:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.141+0000 7ffaa4c00640 1 --2- 172.21.15.46:0/1613820744 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffa84066690 0x7ffa84068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.141+0000 7ffaa4c00640 1 -- 172.21.15.46:0/1613820744 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ffaa011e060 con 0x7ffaa815cd90 2024-03-29T08:29:50.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.141+0000 7ffaa7e00640 1 --2- 172.21.15.46:0/1613820744 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffa84066690 0x7ffa84068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.141+0000 7ffaa7e00640 1 --2- 172.21.15.46:0/1613820744 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffa84066690 0x7ffa84068b50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ffa98007fd0 tx=0x7ffa98009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:50.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.145+0000 7ffaa4c00640 1 -- 172.21.15.46:0/1613820744 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ffaa8153d90 con 0x7ffaa815cd90 2024-03-29T08:29:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.345+0000 7ffaaf200640 1 -- 172.21.15.46:0/1613820744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaa8153d90 con 0x7ffaa815cd90 2024-03-29T08:29:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.345+0000 7ffaa4c00640 1 -- 172.21.15.46:0/1613820744 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4737 (secure 0 0 0) 0x7ffaa8153d90 con 0x7ffaa815cd90 2024-03-29T08:29:50.350 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:50.350 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2506132630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2506132630},{"type":"v1","addr":"172.21.15.73:6836","nonce":2506132630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2108115501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2108115501},{"type":"v1","addr":"172.21.15.73:6837","nonce":2108115501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":9045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3210062895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3210062895},{"type":"v1","addr":"172.21.15.46:6833","nonce":3210062895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:29:14.470858+0000","modified":"2024-03-29T08:29:49.396868+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9441},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9441":{"gid":9441,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/276185207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":276185207},{"type":"v1","addr":"172.21.15.46:6835","nonce":276185207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-29T08:29:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 100 2024-03-29T08:29:50.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.349+0000 7ffaaf200640 1 -- 172.21.15.46:0/1613820744 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffa84066690 msgr2=0x7ffa84068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.349+0000 7ffaaf200640 1 --2- 172.21.15.46:0/1613820744 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffa84066690 0x7ffa84068b50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ffa98007fd0 tx=0x7ffa98009040 comp rx=0 tx=0).stop 2024-03-29T08:29:50.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.349+0000 7ffaaf200640 1 -- 172.21.15.46:0/1613820744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa815cd90 msgr2=0x7ffaa8155420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.349+0000 7ffaaf200640 1 --2- 172.21.15.46:0/1613820744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa815cd90 0x7ffaa8155420 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00190b0 tx=0x7ffaa001ace0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.349+0000 7ffaaf200640 1 -- 172.21.15.46:0/1613820744 shutdown_connections 2024-03-29T08:29:50.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.349+0000 7ffaaf200640 1 --2- 172.21.15.46:0/1613820744 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ffa84066690 0x7ffa84068b50 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.349+0000 7ffaaf200640 1 --2- 172.21.15.46:0/1613820744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaa815a7c0 0x7ffaa81579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.349+0000 7ffaaf200640 1 --2- 172.21.15.46:0/1613820744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ffaa815cd90 0x7ffaa8155420 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.349+0000 7ffaaf200640 1 --2- 172.21.15.46:0/1613820744 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ffaa81593d0 0x7ffaa81f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.349+0000 7ffaaf200640 1 -- 172.21.15.46:0/1613820744 >> 172.21.15.46:0/1613820744 conn(0x7ffaa8066110 msgr2=0x7ffaa80ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:50.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.349+0000 7ffaaf200640 1 -- 172.21.15.46:0/1613820744 shutdown_connections 2024-03-29T08:29:50.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.349+0000 7ffaaf200640 1 -- 172.21.15.46:0/1613820744 wait complete. 2024-03-29T08:29:50.775 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:29:14.470858+0000', 'modified': '2024-03-29T08:29:49.396868+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9441}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9441': {'gid': 9441, 'name': 'a', 'rank': 0, 'incarnation': 97, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6835/276185207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 276185207}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 276185207}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:29:50.776 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T08:29:50.776 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-29T08:29:50.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.913+0000 7fcbbc200640 1 Processor -- start 2024-03-29T08:29:50.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.913+0000 7fcbbc200640 1 -- start start 2024-03-29T08:29:50.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.913+0000 7fcbbc200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcbb40ceaf0 0x7fcbb4098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.913+0000 7fcbbc200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb40cfe50 con 0x7fcbb4098f80 2024-03-29T08:29:50.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbbc200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb40993a0 con 0x7fcbb415c9c0 2024-03-29T08:29:50.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbbc200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb40cef80 con 0x7fcbb40ceaf0 2024-03-29T08:29:50.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbba200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcbb4098f80 0x7fcbb40cad80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42154/0 (socket says 172.21.15.46:42154) 2024-03-29T08:29:50.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbba200640 1 -- 172.21.15.46:0/3780342884 learned_addr learned my addr 172.21.15.46:0/3780342884 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:50.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb8e00640 1 --2- 172.21.15.46:0/3780342884 >> v2:172.21.15.73:3300/0 conn(0x7fcbb40ceaf0 0x7fcbb4098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3780342884 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460553932 0 0) 0x7fcbb40cfe50 con 0x7fcbb4098f80 2024-03-29T08:29:50.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3780342884 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb9c003530 con 0x7fcbb4098f80 2024-03-29T08:29:50.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3780342884 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583041007 0 0) 0x7fcbb40993a0 con 0x7fcbb415c9c0 2024-03-29T08:29:50.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3780342884 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbb40cfe50 con 0x7fcbb415c9c0 2024-03-29T08:29:50.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3780342884 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3767036975 0 0) 0x7fcb9c003530 con 0x7fcbb4098f80 2024-03-29T08:29:50.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3780342884 >> v2:172.21.15.73:3300/0 conn(0x7fcbb40ceaf0 msgr2=0x7fcbb4098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:50.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb3e00640 1 --2- 172.21.15.46:0/3780342884 >> v2:172.21.15.73:3300/0 conn(0x7fcbb40ceaf0 0x7fcbb4098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3780342884 >> v1:172.21.15.73:6789/0 conn(0x7fcbb415c9c0 legacy=0x7fcbb406be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3780342884 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb40cf240 con 0x7fcbb4098f80 2024-03-29T08:29:50.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3780342884 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fcbac001a00 con 0x7fcbb4098f80 2024-03-29T08:29:50.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3780342884 >> v1:172.21.15.46:6789/0 conn(0x7fcbb4098f80 legacy=0x7fcbb40cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb3e00640 1 --2- 172.21.15.46:0/3780342884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb9c003b10 0x7fcb9c003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb3e00640 1 --2- 172.21.15.46:0/3780342884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb9c0047d0 0x7fcb9c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb3e00640 1 --2- 172.21.15.46:0/3780342884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb9c005190 0x7fcb9c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3780342884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40cef80 con 0x7fcb9c0047d0 2024-03-29T08:29:50.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3780342884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c00c010 con 0x7fcb9c005190 2024-03-29T08:29:50.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3780342884 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c00c1d0 con 0x7fcb9c003b10 2024-03-29T08:29:50.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb8e00640 1 --2- 172.21.15.46:0/3780342884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb9c0047d0 0x7fcb9c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbba200640 1 --2- 172.21.15.46:0/3780342884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb9c005190 0x7fcb9c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb9800640 1 --2- 172.21.15.46:0/3780342884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb9c003b10 0x7fcb9c003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb8e00640 1 -- 172.21.15.46:0/3780342884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb9c003b10 msgr2=0x7fcb9c003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:50.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb8e00640 1 --2- 172.21.15.46:0/3780342884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb9c003b10 0x7fcb9c003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb8e00640 1 -- 172.21.15.46:0/3780342884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb9c005190 msgr2=0x7fcb9c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:50.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb8e00640 1 --2- 172.21.15.46:0/3780342884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb9c005190 0x7fcb9c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbba200640 1 -- 172.21.15.46:0/3780342884 reap_dead start 2024-03-29T08:29:50.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.917+0000 7fcbb8e00640 1 -- 172.21.15.46:0/3780342884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c003530 con 0x7fcb9c0047d0 2024-03-29T08:29:50.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.921+0000 7fcbb8e00640 1 --2- 172.21.15.46:0/3780342884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb9c0047d0 0x7fcb9c004c50 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009ac0 tx=0x7fcba001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cee8fd910bd661 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:50.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.921+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3780342884 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba00a5640 con 0x7fcb9c0047d0 2024-03-29T08:29:50.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.921+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3780342884 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba00a57a0 con 0x7fcb9c0047d0 2024-03-29T08:29:50.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.921+0000 7fcbb3e00640 1 -- 172.21.15.46:0/3780342884 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba00a5b30 con 0x7fcb9c0047d0 2024-03-29T08:29:50.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.921+0000 7fcbbc200640 1 -- 172.21.15.46:0/3780342884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb9c0047d0 msgr2=0x7fcb9c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.921+0000 7fcbbc200640 1 --2- 172.21.15.46:0/3780342884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb9c0047d0 0x7fcb9c004c50 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009ac0 tx=0x7fcba001ca50 comp rx=0 tx=0).stop 2024-03-29T08:29:50.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.921+0000 7fcbbc200640 1 -- 172.21.15.46:0/3780342884 shutdown_connections 2024-03-29T08:29:50.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.921+0000 7fcbbc200640 1 --2- 172.21.15.46:0/3780342884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb9c0047d0 0x7fcb9c004c50 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.921+0000 7fcbbc200640 1 -- 172.21.15.46:0/3780342884 >> 172.21.15.46:0/3780342884 conn(0x7fcbb40b2220 msgr2=0x7fcbb40b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:50.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.921+0000 7fcbbc200640 1 -- 172.21.15.46:0/3780342884 shutdown_connections 2024-03-29T08:29:50.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.921+0000 7fcbbc200640 1 -- 172.21.15.46:0/3780342884 wait complete. 2024-03-29T08:29:50.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.921+0000 7fcbbc200640 1 Processor -- start 2024-03-29T08:29:50.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.921+0000 7fcbbc200640 1 -- start start 2024-03-29T08:29:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbbc200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb9c0047d0 0x7fcbb414d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbbc200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb41e6fc0 0x7fcbb41e7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbbc200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbb40b52f0 0x7fcbb40b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbbc200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb414dfd0 con 0x7fcbb41e6fc0 2024-03-29T08:29:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbbc200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb41e7980 con 0x7fcbb40b52f0 2024-03-29T08:29:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbbc200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb40c67c0 con 0x7fcb9c0047d0 2024-03-29T08:29:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbb8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb41e6fc0 0x7fcbb41e7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbb8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb41e6fc0 0x7fcbb41e7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57232/0 (socket says 172.21.15.46:57232) 2024-03-29T08:29:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbb8e00640 1 -- 172.21.15.46:0/3973059141 learned_addr learned my addr 172.21.15.46:0/3973059141 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:50.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbb9800640 1 --2- 172.21.15.46:0/3973059141 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb9c0047d0 0x7fcbb414d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbba200640 1 --2- 172.21.15.46:0/3973059141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbb40b52f0 0x7fcbb40b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbb8e00640 1 -- 172.21.15.46:0/3973059141 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb9c0047d0 msgr2=0x7fcbb414d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbb8e00640 1 --2- 172.21.15.46:0/3973059141 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb9c0047d0 0x7fcbb414d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbb8e00640 1 -- 172.21.15.46:0/3973059141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbb40b52f0 msgr2=0x7fcbb40b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbb8e00640 1 --2- 172.21.15.46:0/3973059141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbb40b52f0 0x7fcbb40b5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbb8e00640 1 -- 172.21.15.46:0/3973059141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb40c4bf0 con 0x7fcbb41e6fc0 2024-03-29T08:29:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbb8e00640 1 --2- 172.21.15.46:0/3973059141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb41e6fc0 0x7fcbb41e7440 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009ac0 tx=0x7fcba00197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbb1600640 1 -- 172.21.15.46:0/3973059141 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba00b8640 con 0x7fcbb41e6fc0 2024-03-29T08:29:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbb1600640 1 -- 172.21.15.46:0/3973059141 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba00b87a0 con 0x7fcbb41e6fc0 2024-03-29T08:29:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbb1600640 1 -- 172.21.15.46:0/3973059141 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba00b8b50 con 0x7fcbb41e6fc0 2024-03-29T08:29:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbbc200640 1 -- 172.21.15.46:0/3973059141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb414da50 con 0x7fcbb41e6fc0 2024-03-29T08:29:50.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbbc200640 1 -- 172.21.15.46:0/3973059141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb40c4300 con 0x7fcbb41e6fc0 2024-03-29T08:29:50.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.925+0000 7fcbb1600640 1 -- 172.21.15.46:0/3973059141 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcba00b4040 con 0x7fcbb41e6fc0 2024-03-29T08:29:50.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbb1600640 1 --2- 172.21.15.46:0/3973059141 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcb94066690 0x7fcb94068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbb1600640 1 -- 172.21.15.46:0/3973059141 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcba011dfb0 con 0x7fcbb41e6fc0 2024-03-29T08:29:50.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbb9800640 1 --2- 172.21.15.46:0/3973059141 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcb94066690 0x7fcb94068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbbf6a4000 1 Processor -- start 2024-03-29T08:29:50.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbbf6a4000 1 -- start start 2024-03-29T08:29:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbb9800640 1 --2- 172.21.15.46:0/3973059141 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcb94066690 0x7fcb94068b50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcba8007860 tx=0x7fcba8007210 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbbf6a4000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb367db0 0x5623bb368190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbbf6a4000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb3686d0 0x5623bb31df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbbf6a4000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb31e4c0 0x5623bb31e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbbf6a4000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bb1e65a0 con 0x5623bb31e4c0 2024-03-29T08:29:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbbf6a4000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bb173af0 con 0x5623bb367db0 2024-03-29T08:29:50.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbbf6a4000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623bb1710c0 con 0x5623bb3686d0 2024-03-29T08:29:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbba200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb31e4c0 0x5623bb31e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbba200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb31e4c0 0x5623bb31e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57246/0 (socket says 172.21.15.46:57246) 2024-03-29T08:29:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbba200640 1 -- 172.21.15.46:0/2330196337 learned_addr learned my addr 172.21.15.46:0/2330196337 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbb8e00640 1 --2- 172.21.15.46:0/2330196337 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb3686d0 0x5623bb31df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbb9800640 1 --2- 172.21.15.46:0/2330196337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb367db0 0x5623bb368190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbba200640 1 -- 172.21.15.46:0/2330196337 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb3686d0 msgr2=0x5623bb31df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbba200640 1 --2- 172.21.15.46:0/2330196337 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb3686d0 0x5623bb31df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbba200640 1 -- 172.21.15.46:0/2330196337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb367db0 msgr2=0x5623bb368190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbba200640 1 --2- 172.21.15.46:0/2330196337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb367db0 0x5623bb368190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbba200640 1 -- 172.21.15.46:0/2330196337 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623bae68890 con 0x5623bb31e4c0 2024-03-29T08:29:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcbba200640 1 --2- 172.21.15.46:0/2330196337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb31e4c0 0x5623bb31e8a0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fcbac002240 tx=0x7fcbac007050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47951b81fe2ec99a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcba6000640 1 -- 172.21.15.46:0/2330196337 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac007a00 con 0x5623bb31e4c0 2024-03-29T08:29:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcba6000640 1 -- 172.21.15.46:0/2330196337 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbac015470 con 0x5623bb31e4c0 2024-03-29T08:29:50.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.929+0000 7fcba6000640 1 -- 172.21.15.46:0/2330196337 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0157e0 con 0x5623bb31e4c0 2024-03-29T08:29:50.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/2330196337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb31e4c0 msgr2=0x5623bb31e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/2330196337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb31e4c0 0x5623bb31e8a0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fcbac002240 tx=0x7fcbac007050 comp rx=0 tx=0).stop 2024-03-29T08:29:50.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/2330196337 shutdown_connections 2024-03-29T08:29:50.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/2330196337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb31e4c0 0x5623bb31e8a0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/2330196337 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb3686d0 0x5623bb31df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/2330196337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb367db0 0x5623bb368190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/2330196337 >> 172.21.15.46:0/2330196337 conn(0x5623bb324830 msgr2=0x5623bb31c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:50.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/2330196337 shutdown_connections 2024-03-29T08:29:50.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/2330196337 wait complete. 2024-03-29T08:29:50.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 -- start start 2024-03-29T08:29:50.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 Processor -- start 2024-03-29T08:29:50.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb31e4c0 0x5623bb323150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb367db0 0x5623bb368ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb3686d0 0x5623bb368ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bb16c9b0 con 0x5623bb367db0 2024-03-29T08:29:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bb10bc00 con 0x5623bb3686d0 2024-03-29T08:29:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623bb12a070 con 0x5623bb31e4c0 2024-03-29T08:29:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 10 client.0 _ll_register_callbacks cb 0x5623bb30a0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:29:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbb8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb367db0 0x5623bb368ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbb8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb367db0 0x5623bb368ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57248/0 (socket says 172.21.15.46:57248) 2024-03-29T08:29:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbb8e00640 1 -- 172.21.15.46:0/4239454710 learned_addr learned my addr 172.21.15.46:0/4239454710 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbba200640 1 --2- 172.21.15.46:0/4239454710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb3686d0 0x5623bb368ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbb9800640 1 --2- 172.21.15.46:0/4239454710 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb31e4c0 0x5623bb323150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbb8e00640 1 -- 172.21.15.46:0/4239454710 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb31e4c0 msgr2=0x5623bb323150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbb8e00640 1 --2- 172.21.15.46:0/4239454710 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb31e4c0 0x5623bb323150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbb8e00640 1 -- 172.21.15.46:0/4239454710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb3686d0 msgr2=0x5623bb368ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbb8e00640 1 --2- 172.21.15.46:0/4239454710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb3686d0 0x5623bb368ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbb8e00640 1 -- 172.21.15.46:0/4239454710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbac00f040 con 0x5623bb367db0 2024-03-29T08:29:50.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbb8e00640 1 --2- 172.21.15.46:0/4239454710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb367db0 0x5623bb368ab0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fcba001c880 tx=0x7fcba00eee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e9f45af5acf89a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcb9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:29:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcb9aa00640 1 -- 172.21.15.46:0/4239454710 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba00ef160 con 0x5623bb367db0 2024-03-29T08:29:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcb9aa00640 1 -- 172.21.15.46:0/4239454710 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcba00ef2c0 con 0x5623bb367db0 2024-03-29T08:29:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/4239454710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623bb1295f0 con 0x5623bb367db0 2024-03-29T08:29:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcb9aa00640 1 -- 172.21.15.46:0/4239454710 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcba00ee3f0 con 0x5623bb367db0 2024-03-29T08:29:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 10 client.9492 fetch_fsmap learned FSMap version 100 2024-03-29T08:29:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/4239454710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x5623bb131ab0 con 0x5623bb367db0 2024-03-29T08:29:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcb9aa00640 1 -- 172.21.15.46:0/4239454710 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5623bb1295f0 con 0x5623bb367db0 2024-03-29T08:29:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcb9aa00640 1 -- 172.21.15.46:0/4239454710 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 100) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcba0102d20 con 0x5623bb367db0 2024-03-29T08:29:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 10 client.9492 fetch_fsmap finished waiting for FSMap version 100 2024-03-29T08:29:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 10 client.9492 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T08:29:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 20 client.9492 populate_metadata read hostname 'smithi046' 2024-03-29T08:29:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/4239454710 >> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] conn(0x5623bb36e5d0 0x5623bb36e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 4 client.9492 mds_command: new command op to 9441 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:29:50.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.933+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/4239454710 --> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5623bb169790 con 0x5623bb36e5d0 2024-03-29T08:29:50.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcbb9800640 1 --2- 172.21.15.46:0/4239454710 >> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] conn(0x5623bb36e5d0 0x5623bb36e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcbb9800640 1 --2- 172.21.15.46:0/4239454710 >> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] conn(0x5623bb36e5d0 0x5623bb36e9b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb1d5649f9a692b2 server_cookie=c2f3b28b0ed215a4 in_seq=0 out_seq=0 2024-03-29T08:29:50.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcb9aa00640 10 client.9492 ms_handle_connect on v2:172.21.15.46:6834/276185207 2024-03-29T08:29:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcb9aa00640 1 -- 172.21.15.46:0/4239454710 <== mds.0 v2:172.21.15.46:6834/276185207 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5623bb169790 con 0x5623bb36e5d0 2024-03-29T08:29:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcb9aa00640 10 client.9492 handle_command_reply: tid=0 2024-03-29T08:29:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcbbf6a4000 1 client.9492 shutdown 2024-03-29T08:29:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/4239454710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb367db0 msgr2=0x5623bb368ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/4239454710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb367db0 0x5623bb368ab0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fcba001c880 tx=0x7fcba00eee00 comp rx=0 tx=0).stop 2024-03-29T08:29:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/4239454710 shutdown_connections 2024-03-29T08:29:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/4239454710 >> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] conn(0x5623bb36e5d0 0x5623bb36e9b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/4239454710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb3686d0 0x5623bb368ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/4239454710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb367db0 0x5623bb368ab0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/4239454710 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb31e4c0 0x5623bb323150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/4239454710 >> 172.21.15.46:0/4239454710 conn(0x5623bb324830 msgr2=0x5623bb31cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:50.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcb9aa00640 0 client.9492 ms_handle_reset on v2:172.21.15.46:6834/276185207 2024-03-29T08:29:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/4239454710 shutdown_connections 2024-03-29T08:29:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/4239454710 wait complete. 2024-03-29T08:29:50.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.937+0000 7fcbbf6a4000 20 client.9492 trim_cache size 0 max 16384 2024-03-29T08:29:50.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcbbf6a4000 1 Processor -- start 2024-03-29T08:29:50.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcbbf6a4000 1 -- start start 2024-03-29T08:29:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcbbf6a4000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb31e4c0 0x5623bb366eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcbbf6a4000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb367db0 0x5623bb3673f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcbbf6a4000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb3686d0 0x5623bb369890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcbbf6a4000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bae29940 con 0x5623bb367db0 2024-03-29T08:29:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcbbf6a4000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bb16a1c0 con 0x5623bb3686d0 2024-03-29T08:29:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcbbf6a4000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623bb16f510 con 0x5623bb31e4c0 2024-03-29T08:29:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcbb8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb367db0 0x5623bb3673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcbb8e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb367db0 0x5623bb3673f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57254/0 (socket says 172.21.15.46:57254) 2024-03-29T08:29:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcbb8e00640 1 -- 172.21.15.46:0/125369806 learned_addr learned my addr 172.21.15.46:0/125369806 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcbb8e00640 1 -- 172.21.15.46:0/125369806 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb31e4c0 msgr2=0x5623bb366eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:50.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcbb8e00640 1 --2- 172.21.15.46:0/125369806 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb31e4c0 0x5623bb366eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcbb8e00640 1 -- 172.21.15.46:0/125369806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb3686d0 msgr2=0x5623bb369890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:50.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcbb8e00640 1 --2- 172.21.15.46:0/125369806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb3686d0 0x5623bb369890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcbb8e00640 1 -- 172.21.15.46:0/125369806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623bb117c00 con 0x5623bb367db0 2024-03-29T08:29:50.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcbb8e00640 1 --2- 172.21.15.46:0/125369806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb367db0 0x5623bb3673f0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fcba00eb400 tx=0x7fcba0105000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831e94838cd88a8d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:50.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcba6a00640 1 -- 172.21.15.46:0/125369806 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba00ee3f0 con 0x5623bb367db0 2024-03-29T08:29:50.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcba6a00640 1 -- 172.21.15.46:0/125369806 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba00eea40 con 0x5623bb367db0 2024-03-29T08:29:50.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.953+0000 7fcba6a00640 1 -- 172.21.15.46:0/125369806 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba00a5640 con 0x5623bb367db0 2024-03-29T08:29:50.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/125369806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb367db0 msgr2=0x5623bb3673f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/125369806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb367db0 0x5623bb3673f0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fcba00eb400 tx=0x7fcba0105000 comp rx=0 tx=0).stop 2024-03-29T08:29:50.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/125369806 shutdown_connections 2024-03-29T08:29:50.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/125369806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb3686d0 0x5623bb369890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/125369806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb367db0 0x5623bb3673f0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/125369806 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb31e4c0 0x5623bb366eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/125369806 >> 172.21.15.46:0/125369806 conn(0x5623bb324830 msgr2=0x5623bb365790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:50.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/125369806 shutdown_connections 2024-03-29T08:29:50.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/125369806 wait complete. 2024-03-29T08:29:50.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 -- start start 2024-03-29T08:29:50.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 Processor -- start 2024-03-29T08:29:50.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb31e4c0 0x5623bb3c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb3686d0 0x5623bb36c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb36ca50 0x5623bb36ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bb135ef0 con 0x5623bb36ca50 2024-03-29T08:29:50.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bb169440 con 0x5623bb3686d0 2024-03-29T08:29:50.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5623bb16a980 con 0x5623bb31e4c0 2024-03-29T08:29:50.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 10 client.0 _ll_register_callbacks cb 0x5623bb131590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:29:50.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbba200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb36ca50 0x5623bb36ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbba200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb36ca50 0x5623bb36ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57258/0 (socket says 172.21.15.46:57258) 2024-03-29T08:29:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbba200640 1 -- 172.21.15.46:0/1427544825 learned_addr learned my addr 172.21.15.46:0/1427544825 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbb9800640 1 --2- 172.21.15.46:0/1427544825 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb31e4c0 0x5623bb3c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbb8e00640 1 --2- 172.21.15.46:0/1427544825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb3686d0 0x5623bb36c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbba200640 1 -- 172.21.15.46:0/1427544825 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb31e4c0 msgr2=0x5623bb3c1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbba200640 1 --2- 172.21.15.46:0/1427544825 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb31e4c0 0x5623bb3c1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbba200640 1 -- 172.21.15.46:0/1427544825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb3686d0 msgr2=0x5623bb36c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbba200640 1 --2- 172.21.15.46:0/1427544825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb3686d0 0x5623bb36c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbba200640 1 -- 172.21.15.46:0/1427544825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623bb131ab0 con 0x5623bb36ca50 2024-03-29T08:29:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbba200640 1 --2- 172.21.15.46:0/1427544825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb36ca50 0x5623bb36ce30 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fcbac001230 tx=0x7fcbac001d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffbb3df5bcd8a390 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcb9aa00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:29:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcb9aa00640 1 -- 172.21.15.46:0/1427544825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbac001df0 con 0x5623bb36ca50 2024-03-29T08:29:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcb9aa00640 1 -- 172.21.15.46:0/1427544825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcbac017070 con 0x5623bb36ca50 2024-03-29T08:29:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/1427544825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5623bb13c460 con 0x5623bb36ca50 2024-03-29T08:29:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcb9aa00640 1 -- 172.21.15.46:0/1427544825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbac017270 con 0x5623bb36ca50 2024-03-29T08:29:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 10 client.9498 fetch_fsmap learned FSMap version 100 2024-03-29T08:29:50.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/1427544825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x5623bae68cc0 con 0x5623bb36ca50 2024-03-29T08:29:50.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.957+0000 7fcb9aa00640 1 -- 172.21.15.46:0/1427544825 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5623bb13c460 con 0x5623bb36ca50 2024-03-29T08:29:50.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcb9aa00640 1 -- 172.21.15.46:0/1427544825 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 100) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcbac026020 con 0x5623bb36ca50 2024-03-29T08:29:50.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 10 client.9498 fetch_fsmap finished waiting for FSMap version 100 2024-03-29T08:29:50.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 10 client.9498 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T08:29:50.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 20 client.9498 populate_metadata read hostname 'smithi046' 2024-03-29T08:29:50.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/1427544825 >> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] conn(0x5623bb36d370 0x5623bb36d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:50.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 4 client.9498 mds_command: new command op to 9441 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T08:29:50.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/1427544825 --> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5623bb1359c0 con 0x5623bb36d370 2024-03-29T08:29:50.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbb9800640 1 --2- 172.21.15.46:0/1427544825 >> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] conn(0x5623bb36d370 0x5623bb36d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:50.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbb9800640 1 --2- 172.21.15.46:0/1427544825 >> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] conn(0x5623bb36d370 0x5623bb36d750 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c85cbf9f60e1609e server_cookie=b4c9a1a2f001e8c0 in_seq=0 out_seq=0 2024-03-29T08:29:50.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcb9aa00640 10 client.9498 ms_handle_connect on v2:172.21.15.46:6834/276185207 2024-03-29T08:29:50.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcb9aa00640 1 -- 172.21.15.46:0/1427544825 <== mds.0 v2:172.21.15.46:6834/276185207 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x5623bb1359c0 con 0x5623bb36d370 2024-03-29T08:29:50.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcb9aa00640 10 client.9498 handle_command_reply: tid=0 2024-03-29T08:29:50.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 1 client.9498 shutdown 2024-03-29T08:29:50.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/1427544825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb36ca50 msgr2=0x5623bb36ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/1427544825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb36ca50 0x5623bb36ce30 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fcbac001230 tx=0x7fcbac001d10 comp rx=0 tx=0).stop 2024-03-29T08:29:50.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/1427544825 shutdown_connections 2024-03-29T08:29:50.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/1427544825 >> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] conn(0x5623bb36d370 0x5623bb36d750 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/1427544825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623bb3686d0 0x5623bb36c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/1427544825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623bb36ca50 0x5623bb36ce30 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 1 --2- 172.21.15.46:0/1427544825 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5623bb31e4c0 0x5623bb3c1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcb9aa00640 0 client.9498 ms_handle_reset on v2:172.21.15.46:6834/276185207 2024-03-29T08:29:50.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/1427544825 >> 172.21.15.46:0/1427544825 conn(0x5623bb324830 msgr2=0x5623bb323cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:50.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/1427544825 shutdown_connections 2024-03-29T08:29:50.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 1 -- 172.21.15.46:0/1427544825 wait complete. 2024-03-29T08:29:50.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbf6a4000 20 client.9498 trim_cache size 0 max 16384 2024-03-29T08:29:50.969 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:29:50.970 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "e9b7960f-ec1f-4787-95c3-c806094d37f0", 2024-03-29T08:29:50.970 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-03-29T08:29:50.970 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 9441, 2024-03-29T08:29:50.970 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-29T08:29:50.970 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-29T08:29:50.970 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-29T08:29:50.970 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 4.5750377530000002, 2024-03-29T08:29:50.970 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 100, 2024-03-29T08:29:50.970 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 122, 2024-03-29T08:29:50.970 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 122, 2024-03-29T08:29:50.970 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 5.5159264539999997 2024-03-29T08:29:50.970 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:29:50.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbc200640 1 -- 172.21.15.46:0/3973059141 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcb94066690 msgr2=0x7fcb94068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbc200640 1 --2- 172.21.15.46:0/3973059141 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcb94066690 0x7fcb94068b50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcba8007860 tx=0x7fcba8007210 comp rx=0 tx=0).stop 2024-03-29T08:29:50.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbc200640 1 -- 172.21.15.46:0/3973059141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb41e6fc0 msgr2=0x7fcbb41e7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:50.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbc200640 1 --2- 172.21.15.46:0/3973059141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb41e6fc0 0x7fcbb41e7440 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009ac0 tx=0x7fcba00197f0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbc200640 1 -- 172.21.15.46:0/3973059141 shutdown_connections 2024-03-29T08:29:50.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbc200640 1 --2- 172.21.15.46:0/3973059141 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcb94066690 0x7fcb94068b50 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbc200640 1 --2- 172.21.15.46:0/3973059141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbb40b52f0 0x7fcbb40b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbc200640 1 --2- 172.21.15.46:0/3973059141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb41e6fc0 0x7fcbb41e7440 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbc200640 1 --2- 172.21.15.46:0/3973059141 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb9c0047d0 0x7fcbb414d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:50.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbc200640 1 -- 172.21.15.46:0/3973059141 >> 172.21.15.46:0/3973059141 conn(0x7fcbb40b2220 msgr2=0x7fcbb406f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:50.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbc200640 1 -- 172.21.15.46:0/3973059141 shutdown_connections 2024-03-29T08:29:50.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:50.961+0000 7fcbbc200640 1 -- 172.21.15.46:0/3973059141 wait complete. 2024-03-29T08:29:50.984 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:29:51.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.121+0000 7fa93e400640 1 Processor -- start 2024-03-29T08:29:51.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.121+0000 7fa93e400640 1 -- start start 2024-03-29T08:29:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.121+0000 7fa93e400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa93815c9d0 0x7fa9380cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.121+0000 7fa93e400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa938099150 0x7fa938166a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.121+0000 7fa93e400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9380d0030 con 0x7fa9380cef60 2024-03-29T08:29:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.121+0000 7fa93e400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9380cf420 con 0x7fa93815c9d0 2024-03-29T08:29:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.121+0000 7fa93e400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9380cf700 con 0x7fa938099150 2024-03-29T08:29:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.121+0000 7fa937400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa93815c9d0 0x7fa9380cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.121+0000 7fa92ea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa9380cef60 0x7fa938098a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42168/0 (socket says 172.21.15.46:42168) 2024-03-29T08:29:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.121+0000 7fa92ea00640 1 -- 172.21.15.46:0/4284304601 learned_addr learned my addr 172.21.15.46:0/4284304601 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.121+0000 7fa937e00640 1 --2- 172.21.15.46:0/4284304601 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa938099150 0x7fa938166a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.121+0000 7fa937400640 1 -- 172.21.15.46:0/4284304601 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa938099150 msgr2=0x7fa938166a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.121+0000 7fa937400640 1 --2- 172.21.15.46:0/4284304601 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa938099150 0x7fa938166a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:51.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.121+0000 7fa937400640 1 -- 172.21.15.46:0/4284304601 >> v1:172.21.15.46:6789/0 conn(0x7fa9380cef60 legacy=0x7fa938098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:51.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.121+0000 7fa937400640 1 -- 172.21.15.46:0/4284304601 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9380cf9e0 con 0x7fa93815c9d0 2024-03-29T08:29:51.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.125+0000 7fa937400640 1 --2- 172.21.15.46:0/4284304601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa93815c9d0 0x7fa9380cea20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fa9280097c0 tx=0x7fa92801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33aeaf695442694f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:51.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.125+0000 7fa936a00640 1 -- 172.21.15.46:0/4284304601 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9280aa5f0 con 0x7fa93815c9d0 2024-03-29T08:29:51.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.125+0000 7fa936a00640 1 -- 172.21.15.46:0/4284304601 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9280aa750 con 0x7fa93815c9d0 2024-03-29T08:29:51.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.125+0000 7fa936a00640 1 -- 172.21.15.46:0/4284304601 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9280aaae0 con 0x7fa93815c9d0 2024-03-29T08:29:51.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.125+0000 7fa93e400640 1 -- 172.21.15.46:0/4284304601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa93815c9d0 msgr2=0x7fa9380cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:51.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.125+0000 7fa93e400640 1 --2- 172.21.15.46:0/4284304601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa93815c9d0 0x7fa9380cea20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fa9280097c0 tx=0x7fa92801c470 comp rx=0 tx=0).stop 2024-03-29T08:29:51.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.125+0000 7fa93e400640 1 -- 172.21.15.46:0/4284304601 shutdown_connections 2024-03-29T08:29:51.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.125+0000 7fa93e400640 1 --2- 172.21.15.46:0/4284304601 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa938099150 0x7fa938166a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:51.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.125+0000 7fa93e400640 1 --2- 172.21.15.46:0/4284304601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa93815c9d0 0x7fa9380cea20 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:51.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.125+0000 7fa93e400640 1 -- 172.21.15.46:0/4284304601 >> 172.21.15.46:0/4284304601 conn(0x7fa9380b21a0 msgr2=0x7fa9380b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:51.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.125+0000 7fa93e400640 1 -- 172.21.15.46:0/4284304601 shutdown_connections 2024-03-29T08:29:51.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.125+0000 7fa93e400640 1 -- 172.21.15.46:0/4284304601 wait complete. 2024-03-29T08:29:51.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa93e400640 1 Processor -- start 2024-03-29T08:29:51.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa93e400640 1 -- start start 2024-03-29T08:29:51.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa93e400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938099150 0x7fa938152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa93e400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9380cef60 0x7fa9380c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa93e400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa93815c9d0 0x7fa9380cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa93e400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa938153230 con 0x7fa938099150 2024-03-29T08:29:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa93e400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9380c6830 con 0x7fa9380cef60 2024-03-29T08:29:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa93e400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9380ca1b0 con 0x7fa93815c9d0 2024-03-29T08:29:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa937400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938099150 0x7fa938152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa937400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938099150 0x7fa938152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57266/0 (socket says 172.21.15.46:57266) 2024-03-29T08:29:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa937400640 1 -- 172.21.15.46:0/807339984 learned_addr learned my addr 172.21.15.46:0/807339984 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa937e00640 1 --2- 172.21.15.46:0/807339984 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa93815c9d0 0x7fa9380cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa92ea00640 1 --2- 172.21.15.46:0/807339984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9380cef60 0x7fa9380c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa937400640 1 -- 172.21.15.46:0/807339984 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa93815c9d0 msgr2=0x7fa9380cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa937400640 1 --2- 172.21.15.46:0/807339984 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa93815c9d0 0x7fa9380cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa937400640 1 -- 172.21.15.46:0/807339984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9380cef60 msgr2=0x7fa9380c41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:51.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa937400640 1 --2- 172.21.15.46:0/807339984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9380cef60 0x7fa9380c41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa937400640 1 -- 172.21.15.46:0/807339984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa928009420 con 0x7fa938099150 2024-03-29T08:29:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa937400640 1 --2- 172.21.15.46:0/807339984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938099150 0x7fa938152cf0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fa9280ac040 tx=0x7fa92801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa92fe00640 1 -- 172.21.15.46:0/807339984 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9280be410 con 0x7fa938099150 2024-03-29T08:29:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa92fe00640 1 -- 172.21.15.46:0/807339984 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9280be570 con 0x7fa938099150 2024-03-29T08:29:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa92fe00640 1 -- 172.21.15.46:0/807339984 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9280be920 con 0x7fa938099150 2024-03-29T08:29:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa93e400640 1 -- 172.21.15.46:0/807339984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9380ca490 con 0x7fa938099150 2024-03-29T08:29:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.129+0000 7fa93e400640 1 -- 172.21.15.46:0/807339984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9380c6db0 con 0x7fa938099150 2024-03-29T08:29:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.133+0000 7fa93e400640 1 -- 172.21.15.46:0/807339984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa938066c90 con 0x7fa938099150 2024-03-29T08:29:51.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.133+0000 7fa92fe00640 1 -- 172.21.15.46:0/807339984 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa9280bea80 con 0x7fa938099150 2024-03-29T08:29:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.133+0000 7fa92fe00640 1 --2- 172.21.15.46:0/807339984 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa914066690 0x7fa914068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.133+0000 7fa92fe00640 1 -- 172.21.15.46:0/807339984 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fa9280a7070 con 0x7fa938099150 2024-03-29T08:29:51.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.133+0000 7fa92ea00640 1 --2- 172.21.15.46:0/807339984 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa914066690 0x7fa914068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:51.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.133+0000 7fa92ea00640 1 --2- 172.21.15.46:0/807339984 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa914066690 0x7fa914068b50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa9381554b0 tx=0x7fa920009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:51.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.137+0000 7fa92fe00640 1 -- 172.21.15.46:0/807339984 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa9280ebc70 con 0x7fa938099150 2024-03-29T08:29:51.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.333+0000 7fa93e400640 1 -- 172.21.15.46:0/807339984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa9380cace0 con 0x7fa938099150 2024-03-29T08:29:51.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.333+0000 7fa92fe00640 1 -- 172.21.15.46:0/807339984 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa9280ef290 con 0x7fa938099150 2024-03-29T08:29:51.338 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:29:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.333+0000 7fa93e400640 1 -- 172.21.15.46:0/807339984 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa914066690 msgr2=0x7fa914068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.333+0000 7fa93e400640 1 --2- 172.21.15.46:0/807339984 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa914066690 0x7fa914068b50 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa9381554b0 tx=0x7fa920009040 comp rx=0 tx=0).stop 2024-03-29T08:29:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.333+0000 7fa93e400640 1 -- 172.21.15.46:0/807339984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938099150 msgr2=0x7fa938152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.333+0000 7fa93e400640 1 --2- 172.21.15.46:0/807339984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938099150 0x7fa938152cf0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fa9280ac040 tx=0x7fa92801acb0 comp rx=0 tx=0).stop 2024-03-29T08:29:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.337+0000 7fa93e400640 1 -- 172.21.15.46:0/807339984 shutdown_connections 2024-03-29T08:29:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.337+0000 7fa93e400640 1 --2- 172.21.15.46:0/807339984 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa914066690 0x7fa914068b50 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.337+0000 7fa93e400640 1 --2- 172.21.15.46:0/807339984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9380cef60 0x7fa9380c41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.337+0000 7fa93e400640 1 --2- 172.21.15.46:0/807339984 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa93815c9d0 0x7fa9380cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.337+0000 7fa93e400640 1 --2- 172.21.15.46:0/807339984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa938099150 0x7fa938152cf0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.337+0000 7fa93e400640 1 -- 172.21.15.46:0/807339984 >> 172.21.15.46:0/807339984 conn(0x7fa9380b21a0 msgr2=0x7fa938155dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.337+0000 7fa93e400640 1 -- 172.21.15.46:0/807339984 shutdown_connections 2024-03-29T08:29:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:51.337+0000 7fa93e400640 1 -- 172.21.15.46:0/807339984 wait complete. 2024-03-29T08:29:51.359 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:29:51.359 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:29:51.359 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-29T08:29:51.359 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T08:29:51.359 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:51.359 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:29:51.359 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:29:51.359 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:29:51.359 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:29:51.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:51.360 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-29T08:29:51.369 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:29:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:29:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:29:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:29:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:51.370 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:29:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-29T08:29:51.370 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:29:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-29T08:29:51.370 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:29:51.370 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-29T08:29:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T08:29:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41746sec preferred_lft 41746sec 2024-03-29T08:29:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-29T08:29:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:51.371 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:29:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-29T08:29:51.371 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:29:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:d9:58:09:4c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:29:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:51.371 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:29:51.371 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T08:29:51.372 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T08:29:51.372 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:51.372 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T08:29:51.372 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:51.372 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:29:51.372 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:29:51.372 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T08:29:51.372 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:51.372 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:29:51.372 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T08:29:51.372 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-29T08:29:51.372 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:29:51.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:51.373 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:29:51.430 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:29:51.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:51.432 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-29T08:29:51.489 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-29T08:29:51.489 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:29:51.489 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T08:29:51.489 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:29:51.489 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T08:29:51.489 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:29:51.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:51.489 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:29:51.490 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:29:51.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:29:51.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:29:51.490 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:29:51.490 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:29:51.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:51.581 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:29:51.591 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T08:29:51.591 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T08:29:51.591 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:29:51.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:51.591 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-03-29T08:29:51.651 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T08:29:51.651 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:29:51.651 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:51.703 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:29:51.703 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:29:51.703 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:51.756 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:29:51.820 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:29:51.820 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:29:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:29:51.887 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:29:51.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:51.888 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:29:51.940 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:29:51.940 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T08:29:51.986 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:29:52.007 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:29:52.007 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:29:52.071 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:29:52.073 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:29:52.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:52.074 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:29:52.083 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-03-29T08:29:52.077+0000 7fd5eeca8580 -1 init, newargv = 0x55af5a59ae50 newargc=15 2024-03-29T08:29:52.083 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[51965]: starting ceph client 2024-03-29T08:29:52.090 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[51965]: starting fuse 2024-03-29T08:29:52.092 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:29:52.093 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-29T08:29:52.093 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:52.093 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-03-29T08:29:52.093 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-03-29T08:29:52.093 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:52.093 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-03-29T08:29:52.093 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:52.093 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-03-29T08:29:52.093 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T08:29:52.093 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-03-29T08:29:52.094 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:29:52.094 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:52.094 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-03-29T08:29:52.094 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-03-29T08:29:52.094 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-03-29T08:29:52.094 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-03-29T08:29:52.094 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:52.094 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-03-29T08:29:52.094 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:29:52.094 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:29:52.094 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:29:52.094 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-03-29T08:29:52.094 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-03-29T08:29:52.094 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-03-29T08:29:52.095 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:29:52.095 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:52.095 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-03-29T08:29:52.095 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:52.216 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.51965.asok 2024-03-29T08:29:52.390 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-03-29T08:29:52.389+0000 7f1773000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:29:52.400 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51965.asok 2024-03-29T08:29:52.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:52.400 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51965.asok status 2024-03-29T08:29:52.576 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:29:52.576 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-03-29T08:29:52.576 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:29:52.576 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:29:52.576 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-03-29T08:29:52.576 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-03-29T08:29:52.577 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T08:29:52.577 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "51965", 2024-03-29T08:29:52.577 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-03-29T08:29:52.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:52.577 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-03-29T08:29:52.577 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-03-29T08:29:52.577 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 9516, 2024-03-29T08:29:52.577 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-03-29T08:29:52.577 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-03-29T08:29:52.577 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-03-29T08:29:52.577 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 9516 2024-03-29T08:29:52.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:52.577 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:29:52.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:29:52.578 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:29:52.578 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3096377061 2024-03-29T08:29:52.578 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:29:52.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:52.578 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:29:52.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:29:52.578 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:29:52.578 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 3096377061 2024-03-29T08:29:52.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:29:52.578 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.9516 192.168.0.1:0/3096377061", 2024-03-29T08:29:52.578 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/3096377061", 2024-03-29T08:29:52.579 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-03-29T08:29:52.579 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 100, 2024-03-29T08:29:52.579 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 122, 2024-03-29T08:29:52.579 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:29:52.579 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-03-29T08:29:52.579 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-03-29T08:29:52.579 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:29:52.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:52.591 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-03-29T08:29:52.592 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:52.601 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-03-29T08:29:52.601 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:52.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:52.601 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:52.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:52.663 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-29T08:29:52.733 INFO:teuthology.orchestra.run.smithi046.stdout:./ 2024-03-29T08:29:52.734 INFO:teuthology.orchestra.run.smithi046.stdout:./parent 2024-03-29T08:29:52.734 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed 2024-03-29T08:29:52.734 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed/charlie 2024-03-29T08:29:52.734 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed/alpha 2024-03-29T08:29:52.734 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/unflushed 2024-03-29T08:29:52.735 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/unflushed/jfile 2024-03-29T08:29:52.736 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:52.736 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:52.737 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:29:52.737 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:52.737 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-29T08:29:52.737 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:52.843 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:29:53.032 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:53.033 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:53.033 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:29:53.033 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:53.033 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-03-29T08:29:53.033 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:53.144 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627776 2024-03-29T08:29:53.334 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:53.334 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:53.334 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:29:53.334 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:53.334 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-03-29T08:29:53.334 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:53.440 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627777 2024-03-29T08:29:53.629 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:53.630 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:53.630 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:29:53.630 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:53.630 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-03-29T08:29:53.630 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:53.741 INFO:teuthology.orchestra.run.smithi046.stdout:1099511628280 2024-03-29T08:29:53.930 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:53.930 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:53.930 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:29:53.930 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:53.930 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-03-29T08:29:53.930 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:54.040 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627778 2024-03-29T08:29:54.230 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:54.230 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:54.230 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:29:54.230 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:54.230 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-03-29T08:29:54.230 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:54.340 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627779 2024-03-29T08:29:54.529 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:29:54.529 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:29:54.529 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:29:54.529 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:29:54.529 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-03-29T08:29:54.529 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:29:54.638 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627780 2024-03-29T08:29:54.828 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:29:54.828 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:29:54.882 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:29:55.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.061+0000 7f12d5c00640 1 Processor -- start 2024-03-29T08:29:55.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.061+0000 7f12d5c00640 1 -- start start 2024-03-29T08:29:55.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.065+0000 7f12d5c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12d015cb00 0x7f12d0098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.065+0000 7f12d5c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12d0099810 0x7f12d0098f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.065+0000 7f12d5c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d00cb5d0 con 0x7f12d00b2b30 2024-03-29T08:29:55.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.065+0000 7f12d5c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d0099d10 con 0x7f12d0099810 2024-03-29T08:29:55.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.065+0000 7f12d5c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d00caca0 con 0x7f12d015cb00 2024-03-29T08:29:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.065+0000 7f12c7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12d0099810 0x7f12d0098f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.065+0000 7f12c7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12d0099810 0x7f12d0098f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57278/0 (socket says 172.21.15.46:57278) 2024-03-29T08:29:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.065+0000 7f12c7e00640 1 -- 172.21.15.46:0/4067473685 learned_addr learned my addr 172.21.15.46:0/4067473685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.065+0000 7f12cfe00640 1 --2- 172.21.15.46:0/4067473685 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12d015cb00 0x7f12d0098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.065+0000 7f12c7e00640 1 -- 172.21.15.46:0/4067473685 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12d015cb00 msgr2=0x7f12d0098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.065+0000 7f12c7e00640 1 --2- 172.21.15.46:0/4067473685 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12d015cb00 0x7f12d0098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.065+0000 7f12c7e00640 1 -- 172.21.15.46:0/4067473685 >> v1:172.21.15.73:6789/0 conn(0x7f12d00b2b30 legacy=0x7f12d006b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.065+0000 7f12c7e00640 1 -- 172.21.15.46:0/4067473685 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d00caf80 con 0x7f12d0099810 2024-03-29T08:29:55.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.065+0000 7f12c7e00640 1 --2- 172.21.15.46:0/4067473685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d0099810 0x7f12d0098f80 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009870 tx=0x7f12b801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7334c425cda1e367 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:55.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12cf400640 1 -- 172.21.15.46:0/4067473685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b80ab670 con 0x7f12d0099810 2024-03-29T08:29:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12cf400640 1 -- 172.21.15.46:0/4067473685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b80ab7d0 con 0x7f12d0099810 2024-03-29T08:29:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12cf400640 1 -- 172.21.15.46:0/4067473685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b80abb60 con 0x7f12d0099810 2024-03-29T08:29:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12d5c00640 1 -- 172.21.15.46:0/4067473685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d0099810 msgr2=0x7f12d0098f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12d5c00640 1 --2- 172.21.15.46:0/4067473685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d0099810 0x7f12d0098f80 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009870 tx=0x7f12b801c4d0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12d5c00640 1 -- 172.21.15.46:0/4067473685 shutdown_connections 2024-03-29T08:29:55.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12d5c00640 1 --2- 172.21.15.46:0/4067473685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d0099810 0x7f12d0098f80 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12d5c00640 1 --2- 172.21.15.46:0/4067473685 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12d015cb00 0x7f12d0098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12d5c00640 1 -- 172.21.15.46:0/4067473685 >> 172.21.15.46:0/4067473685 conn(0x7f12d00b2010 msgr2=0x7f12d00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:55.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12d5c00640 1 -- 172.21.15.46:0/4067473685 shutdown_connections 2024-03-29T08:29:55.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12d5c00640 1 -- 172.21.15.46:0/4067473685 wait complete. 2024-03-29T08:29:55.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12d5c00640 1 Processor -- start 2024-03-29T08:29:55.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12d5c00640 1 -- start start 2024-03-29T08:29:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12d5c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12d0099810 0x7f12d0153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12d5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d00b2b30 0x7f12d00ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12d5c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12d015cb00 0x7f12d00adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12d5c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d00ad530 con 0x7f12d00b2b30 2024-03-29T08:29:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12d5c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d015add0 con 0x7f12d0099810 2024-03-29T08:29:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12d5c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d015b0b0 con 0x7f12d015cb00 2024-03-29T08:29:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d00b2b30 0x7f12d00ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12c7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d00b2b30 0x7f12d00ad9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57282/0 (socket says 172.21.15.46:57282) 2024-03-29T08:29:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12c7e00640 1 -- 172.21.15.46:0/4262769718 learned_addr learned my addr 172.21.15.46:0/4262769718 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12cfe00640 1 --2- 172.21.15.46:0/4262769718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12d0099810 0x7f12d0153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12d5200640 1 --2- 172.21.15.46:0/4262769718 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12d015cb00 0x7f12d00adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12c7e00640 1 -- 172.21.15.46:0/4262769718 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12d015cb00 msgr2=0x7f12d00adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12c7e00640 1 --2- 172.21.15.46:0/4262769718 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12d015cb00 0x7f12d00adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12c7e00640 1 -- 172.21.15.46:0/4262769718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12d0099810 msgr2=0x7f12d0153ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.069+0000 7f12c7e00640 1 --2- 172.21.15.46:0/4262769718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12d0099810 0x7f12d0153ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.073+0000 7f12c7e00640 1 -- 172.21.15.46:0/4262769718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b80094d0 con 0x7f12d00b2b30 2024-03-29T08:29:55.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.073+0000 7f12c7e00640 1 --2- 172.21.15.46:0/4262769718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d00b2b30 0x7f12d00ad9c0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f12b80ad040 tx=0x7f12b801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.073+0000 7f12ccc00640 1 -- 172.21.15.46:0/4262769718 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b80b4800 con 0x7f12d00b2b30 2024-03-29T08:29:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.073+0000 7f12ccc00640 1 -- 172.21.15.46:0/4262769718 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b80b4980 con 0x7f12d00b2b30 2024-03-29T08:29:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.073+0000 7f12ccc00640 1 -- 172.21.15.46:0/4262769718 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b80b4d30 con 0x7f12d00b2b30 2024-03-29T08:29:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.073+0000 7f12d5c00640 1 -- 172.21.15.46:0/4262769718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d015b390 con 0x7f12d00b2b30 2024-03-29T08:29:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.073+0000 7f12d5c00640 1 -- 172.21.15.46:0/4262769718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d01526f0 con 0x7f12d00b2b30 2024-03-29T08:29:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.073+0000 7f12d5c00640 1 -- 172.21.15.46:0/4262769718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d0152880 con 0x7f12d00b2b30 2024-03-29T08:29:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.073+0000 7f12ccc00640 1 -- 172.21.15.46:0/4262769718 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f12b80c83a0 con 0x7f12d00b2b30 2024-03-29T08:29:55.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.073+0000 7f12ccc00640 1 --2- 172.21.15.46:0/4262769718 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12ac066690 0x7f12ac068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.073+0000 7f12ccc00640 1 -- 172.21.15.46:0/4262769718 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f12b811ec40 con 0x7f12d00b2b30 2024-03-29T08:29:55.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.073+0000 7f12cfe00640 1 --2- 172.21.15.46:0/4262769718 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12ac066690 0x7f12ac068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.077+0000 7f12cfe00640 1 --2- 172.21.15.46:0/4262769718 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12ac066690 0x7f12ac068b50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f12c0007a60 tx=0x7f12c0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:55.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.077+0000 7f12ccc00640 1 -- 172.21.15.46:0/4262769718 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f12b80ec080 con 0x7f12d00b2b30 2024-03-29T08:29:55.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.241+0000 7f12d5c00640 1 -- 172.21.15.46:0/4262769718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12d004e040 con 0x7f12d00b2b30 2024-03-29T08:29:55.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.241+0000 7f12ccc00640 1 -- 172.21.15.46:0/4262769718 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+16194 (secure 0 0 0) 0x7f12b80c4020 con 0x7f12d00b2b30 2024-03-29T08:29:55.247 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:55.247 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":122,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:29:45.374830+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:29:12.329121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-29T08:29:13.465939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/929698978":"2024-03-30T08:29:45.374806+0000","172.21.15.46:6834/929698978":"2024-03-30T08:29:45.374806+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:29:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.245+0000 7f12d5c00640 1 -- 172.21.15.46:0/4262769718 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12ac066690 msgr2=0x7f12ac068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.245+0000 7f12d5c00640 1 --2- 172.21.15.46:0/4262769718 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12ac066690 0x7f12ac068b50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f12c0007a60 tx=0x7f12c0008040 comp rx=0 tx=0).stop 2024-03-29T08:29:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.245+0000 7f12d5c00640 1 -- 172.21.15.46:0/4262769718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d00b2b30 msgr2=0x7f12d00ad9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.245+0000 7f12d5c00640 1 --2- 172.21.15.46:0/4262769718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d00b2b30 0x7f12d00ad9c0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f12b80ad040 tx=0x7f12b801ad10 comp rx=0 tx=0).stop 2024-03-29T08:29:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.245+0000 7f12d5c00640 1 -- 172.21.15.46:0/4262769718 shutdown_connections 2024-03-29T08:29:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.245+0000 7f12d5c00640 1 --2- 172.21.15.46:0/4262769718 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12ac066690 0x7f12ac068b50 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.245+0000 7f12d5c00640 1 --2- 172.21.15.46:0/4262769718 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12d015cb00 0x7f12d00adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.245+0000 7f12d5c00640 1 --2- 172.21.15.46:0/4262769718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12d00b2b30 0x7f12d00ad9c0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.245+0000 7f12d5c00640 1 --2- 172.21.15.46:0/4262769718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12d0099810 0x7f12d0153ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.245+0000 7f12d5c00640 1 -- 172.21.15.46:0/4262769718 >> 172.21.15.46:0/4262769718 conn(0x7f12d00b2010 msgr2=0x7f12d0155f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.245+0000 7f12d5c00640 1 -- 172.21.15.46:0/4262769718 shutdown_connections 2024-03-29T08:29:55.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.245+0000 7f12d5c00640 1 -- 172.21.15.46:0/4262769718 wait complete. 2024-03-29T08:29:55.268 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:29:12.329121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:29:13.465939+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:29:55.269 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:29:55.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.409+0000 7ff139000640 1 Processor -- start 2024-03-29T08:29:55.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.409+0000 7ff139000640 1 -- start start 2024-03-29T08:29:55.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.409+0000 7ff139000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff13415cb60 0x7ff1340d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.409+0000 7ff139000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff1340cf9f0 0x7ff1340b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.409+0000 7ff139000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13415cf40 con 0x7ff1340d0bf0 2024-03-29T08:29:55.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.409+0000 7ff139000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1340b0250 con 0x7ff1340cf9f0 2024-03-29T08:29:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.409+0000 7ff139000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340b0810 con 0x7ff13415cb60 2024-03-29T08:29:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.409+0000 7ff132800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff1340cf9f0 0x7ff1340b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.409+0000 7ff12be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff1340d0bf0 0x7ff1340cf2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42176/0 (socket says 172.21.15.46:42176) 2024-03-29T08:29:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.409+0000 7ff12be00640 1 -- 172.21.15.46:0/235059794 learned_addr learned my addr 172.21.15.46:0/235059794 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.409+0000 7ff131e00640 1 --2- 172.21.15.46:0/235059794 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff13415cb60 0x7ff1340d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.409+0000 7ff132800640 1 -- 172.21.15.46:0/235059794 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff13415cb60 msgr2=0x7ff1340d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.409+0000 7ff132800640 1 --2- 172.21.15.46:0/235059794 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff13415cb60 0x7ff1340d06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.409+0000 7ff132800640 1 -- 172.21.15.46:0/235059794 >> v1:172.21.15.46:6789/0 conn(0x7ff1340d0bf0 legacy=0x7ff1340cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.409+0000 7ff132800640 1 -- 172.21.15.46:0/235059794 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1340d1050 con 0x7ff1340cf9f0 2024-03-29T08:29:55.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.409+0000 7ff131e00640 1 --2- 172.21.15.46:0/235059794 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff13415cb60 0x7ff1340d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:55.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.413+0000 7ff132800640 1 --2- 172.21.15.46:0/235059794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1340cf9f0 0x7ff1340b0ca0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009d90 tx=0x7ff12c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd17d6ca7bd118e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:55.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.413+0000 7ff131400640 1 -- 172.21.15.46:0/235059794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0a5a30 con 0x7ff1340cf9f0 2024-03-29T08:29:55.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.413+0000 7ff131400640 1 -- 172.21.15.46:0/235059794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c0a5b90 con 0x7ff1340cf9f0 2024-03-29T08:29:55.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.413+0000 7ff131400640 1 -- 172.21.15.46:0/235059794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0ad440 con 0x7ff1340cf9f0 2024-03-29T08:29:55.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.413+0000 7ff139000640 1 -- 172.21.15.46:0/235059794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1340cf9f0 msgr2=0x7ff1340b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.413+0000 7ff139000640 1 --2- 172.21.15.46:0/235059794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1340cf9f0 0x7ff1340b0ca0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009d90 tx=0x7ff12c01c970 comp rx=0 tx=0).stop 2024-03-29T08:29:55.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.413+0000 7ff139000640 1 -- 172.21.15.46:0/235059794 shutdown_connections 2024-03-29T08:29:55.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.413+0000 7ff139000640 1 --2- 172.21.15.46:0/235059794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff1340cf9f0 0x7ff1340b0ca0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.413+0000 7ff139000640 1 --2- 172.21.15.46:0/235059794 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff13415cb60 0x7ff1340d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.413+0000 7ff139000640 1 -- 172.21.15.46:0/235059794 >> 172.21.15.46:0/235059794 conn(0x7ff134099a10 msgr2=0x7ff134089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:55.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.413+0000 7ff139000640 1 -- 172.21.15.46:0/235059794 shutdown_connections 2024-03-29T08:29:55.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.413+0000 7ff139000640 1 -- 172.21.15.46:0/235059794 wait complete. 2024-03-29T08:29:55.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.413+0000 7ff139000640 1 Processor -- start 2024-03-29T08:29:55.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.413+0000 7ff139000640 1 -- start start 2024-03-29T08:29:55.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff139000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff1340cf9f0 0x7ff1340b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff139000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1340d0bf0 0x7ff134157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff139000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13415cb60 0x7ff13415b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff139000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340b4ae0 con 0x7ff13415cb60 2024-03-29T08:29:55.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff139000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340b3ea0 con 0x7ff1340d0bf0 2024-03-29T08:29:55.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff139000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340b4140 con 0x7ff1340cf9f0 2024-03-29T08:29:55.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff132800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13415cb60 0x7ff13415b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff132800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13415cb60 0x7ff13415b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57290/0 (socket says 172.21.15.46:57290) 2024-03-29T08:29:55.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff132800640 1 -- 172.21.15.46:0/1866716264 learned_addr learned my addr 172.21.15.46:0/1866716264 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:55.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff12be00640 1 --2- 172.21.15.46:0/1866716264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1340d0bf0 0x7ff134157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff131e00640 1 --2- 172.21.15.46:0/1866716264 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff1340cf9f0 0x7ff1340b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff132800640 1 -- 172.21.15.46:0/1866716264 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff1340cf9f0 msgr2=0x7ff1340b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff132800640 1 --2- 172.21.15.46:0/1866716264 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff1340cf9f0 0x7ff1340b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff132800640 1 -- 172.21.15.46:0/1866716264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1340d0bf0 msgr2=0x7ff134157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff132800640 1 --2- 172.21.15.46:0/1866716264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1340d0bf0 0x7ff134157a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff132800640 1 -- 172.21.15.46:0/1866716264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c009a70 con 0x7ff13415cb60 2024-03-29T08:29:55.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff132800640 1 --2- 172.21.15.46:0/1866716264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13415cb60 0x7ff13415b0c0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7ff12c01b020 tx=0x7ff12c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:55.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff12a000640 1 -- 172.21.15.46:0/1866716264 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0addc0 con 0x7ff13415cb60 2024-03-29T08:29:55.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff12a000640 1 -- 172.21.15.46:0/1866716264 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c009450 con 0x7ff13415cb60 2024-03-29T08:29:55.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff12a000640 1 -- 172.21.15.46:0/1866716264 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0b9650 con 0x7ff13415cb60 2024-03-29T08:29:55.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff139000640 1 -- 172.21.15.46:0/1866716264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1340b4420 con 0x7ff13415cb60 2024-03-29T08:29:55.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff139000640 1 -- 172.21.15.46:0/1866716264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff134155f30 con 0x7ff13415cb60 2024-03-29T08:29:55.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.417+0000 7ff139000640 1 -- 172.21.15.46:0/1866716264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1340b4c20 con 0x7ff13415cb60 2024-03-29T08:29:55.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.421+0000 7ff12a000640 1 -- 172.21.15.46:0/1866716264 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff12c0b97b0 con 0x7ff13415cb60 2024-03-29T08:29:55.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.421+0000 7ff12a000640 1 --2- 172.21.15.46:0/1866716264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff1100666c0 0x7ff110068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.421+0000 7ff12a000640 1 -- 172.21.15.46:0/1866716264 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff12c11f430 con 0x7ff13415cb60 2024-03-29T08:29:55.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.421+0000 7ff131e00640 1 --2- 172.21.15.46:0/1866716264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff1100666c0 0x7ff110068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.421+0000 7ff131e00640 1 --2- 172.21.15.46:0/1866716264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff1100666c0 0x7ff110068b80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff1240098b0 tx=0x7ff124008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:55.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.425+0000 7ff12a000640 1 -- 172.21.15.46:0/1866716264 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff1340b4c20 con 0x7ff13415cb60 2024-03-29T08:29:55.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.585+0000 7ff139000640 1 -- 172.21.15.46:0/1866716264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff1340b4c20 con 0x7ff13415cb60 2024-03-29T08:29:55.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.589+0000 7ff12a000640 1 -- 172.21.15.46:0/1866716264 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff1340b4c20 con 0x7ff13415cb60 2024-03-29T08:29:55.592 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:29:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.589+0000 7ff139000640 1 -- 172.21.15.46:0/1866716264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff1100666c0 msgr2=0x7ff110068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.589+0000 7ff139000640 1 --2- 172.21.15.46:0/1866716264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff1100666c0 0x7ff110068b80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff1240098b0 tx=0x7ff124008040 comp rx=0 tx=0).stop 2024-03-29T08:29:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.589+0000 7ff139000640 1 -- 172.21.15.46:0/1866716264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13415cb60 msgr2=0x7ff13415b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.589+0000 7ff139000640 1 --2- 172.21.15.46:0/1866716264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13415cb60 0x7ff13415b0c0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7ff12c01b020 tx=0x7ff12c019710 comp rx=0 tx=0).stop 2024-03-29T08:29:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.589+0000 7ff139000640 1 -- 172.21.15.46:0/1866716264 shutdown_connections 2024-03-29T08:29:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.589+0000 7ff139000640 1 --2- 172.21.15.46:0/1866716264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff1100666c0 0x7ff110068b80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.589+0000 7ff139000640 1 --2- 172.21.15.46:0/1866716264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1340d0bf0 0x7ff134157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.589+0000 7ff139000640 1 --2- 172.21.15.46:0/1866716264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff13415cb60 0x7ff13415b0c0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.589+0000 7ff139000640 1 --2- 172.21.15.46:0/1866716264 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff1340cf9f0 0x7ff1340b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.589+0000 7ff139000640 1 -- 172.21.15.46:0/1866716264 >> 172.21.15.46:0/1866716264 conn(0x7ff134099a10 msgr2=0x7ff1340cb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.589+0000 7ff139000640 1 -- 172.21.15.46:0/1866716264 shutdown_connections 2024-03-29T08:29:55.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.589+0000 7ff139000640 1 -- 172.21.15.46:0/1866716264 wait complete. 2024-03-29T08:29:55.613 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:29:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.753+0000 7f1833000640 1 Processor -- start 2024-03-29T08:29:55.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.753+0000 7f1833000640 1 -- start start 2024-03-29T08:29:55.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.753+0000 7f1833000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f182c099780 0x7f182c0b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.753+0000 7f1833000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f182c15ee70 0x7f182c15f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.753+0000 7f1833000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182c06b150 con 0x7f182c098a40 2024-03-29T08:29:55.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.753+0000 7f1833000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182c06b6d0 con 0x7f182c15ee70 2024-03-29T08:29:55.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.753+0000 7f1833000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c15e190 con 0x7f182c099780 2024-03-29T08:29:55.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.753+0000 7f1832600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f182c15ee70 0x7f182c15f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.753+0000 7f1831c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f182c098a40 0x7f182c098e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42178/0 (socket says 172.21.15.46:42178) 2024-03-29T08:29:55.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.753+0000 7f1831c00640 1 -- 172.21.15.46:0/3419914657 learned_addr learned my addr 172.21.15.46:0/3419914657 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:55.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.753+0000 7f1831200640 1 --2- 172.21.15.46:0/3419914657 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f182c099780 0x7f182c0b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.753+0000 7f1832600640 1 -- 172.21.15.46:0/3419914657 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f182c099780 msgr2=0x7f182c0b1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:55.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.753+0000 7f1832600640 1 --2- 172.21.15.46:0/3419914657 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f182c099780 0x7f182c0b1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.753+0000 7f1832600640 1 -- 172.21.15.46:0/3419914657 >> v1:172.21.15.46:6789/0 conn(0x7f182c098a40 legacy=0x7f182c098e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.753+0000 7f1832600640 1 -- 172.21.15.46:0/3419914657 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c06b3f0 con 0x7f182c15ee70 2024-03-29T08:29:55.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.753+0000 7f1831200640 1 --2- 172.21.15.46:0/3419914657 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f182c099780 0x7f182c0b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:55.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.757+0000 7f1832600640 1 --2- 172.21.15.46:0/3419914657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c15ee70 0x7f182c15f300 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f1824009d90 tx=0x7f182401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22c582331ef2da9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:55.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.757+0000 7f182be00640 1 -- 172.21.15.46:0/3419914657 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240a5970 con 0x7f182c15ee70 2024-03-29T08:29:55.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.757+0000 7f182be00640 1 -- 172.21.15.46:0/3419914657 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18240a5ad0 con 0x7f182c15ee70 2024-03-29T08:29:55.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.757+0000 7f182be00640 1 -- 172.21.15.46:0/3419914657 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240a5e60 con 0x7f182c15ee70 2024-03-29T08:29:55.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.757+0000 7f1833000640 1 -- 172.21.15.46:0/3419914657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c15ee70 msgr2=0x7f182c15f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.757+0000 7f1833000640 1 --2- 172.21.15.46:0/3419914657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c15ee70 0x7f182c15f300 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f1824009d90 tx=0x7f182401c8b0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.757+0000 7f1833000640 1 -- 172.21.15.46:0/3419914657 shutdown_connections 2024-03-29T08:29:55.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.757+0000 7f1833000640 1 --2- 172.21.15.46:0/3419914657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c15ee70 0x7f182c15f300 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.757+0000 7f1833000640 1 --2- 172.21.15.46:0/3419914657 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f182c099780 0x7f182c0b1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.757+0000 7f1833000640 1 -- 172.21.15.46:0/3419914657 >> 172.21.15.46:0/3419914657 conn(0x7f182c0b1fb0 msgr2=0x7f182c0b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:55.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.757+0000 7f1833000640 1 -- 172.21.15.46:0/3419914657 shutdown_connections 2024-03-29T08:29:55.763 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.757+0000 7f1833000640 1 -- 172.21.15.46:0/3419914657 wait complete. 2024-03-29T08:29:55.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1833000640 1 Processor -- start 2024-03-29T08:29:55.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1833000640 1 -- start start 2024-03-29T08:29:55.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1833000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f182c098a40 0x7f182c158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1833000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c099780 0x7f182c14dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1833000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f182c15ee70 0x7f182c15ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1833000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0c6e70 con 0x7f182c099780 2024-03-29T08:29:55.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1833000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c14e1f0 con 0x7f182c15ee70 2024-03-29T08:29:55.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1833000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0c3ff0 con 0x7f182c098a40 2024-03-29T08:29:55.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1831200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c099780 0x7f182c14dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1831200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c099780 0x7f182c14dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57308/0 (socket says 172.21.15.46:57308) 2024-03-29T08:29:55.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1831200640 1 -- 172.21.15.46:0/461145657 learned_addr learned my addr 172.21.15.46:0/461145657 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:55.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1832600640 1 --2- 172.21.15.46:0/461145657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f182c15ee70 0x7f182c15ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1831c00640 1 --2- 172.21.15.46:0/461145657 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f182c098a40 0x7f182c158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1831200640 1 -- 172.21.15.46:0/461145657 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f182c098a40 msgr2=0x7f182c158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1831200640 1 --2- 172.21.15.46:0/461145657 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f182c098a40 0x7f182c158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1831200640 1 -- 172.21.15.46:0/461145657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f182c15ee70 msgr2=0x7f182c15ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1831200640 1 --2- 172.21.15.46:0/461145657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f182c15ee70 0x7f182c15ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1831200640 1 -- 172.21.15.46:0/461145657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1824009a70 con 0x7f182c099780 2024-03-29T08:29:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1831200640 1 --2- 172.21.15.46:0/461145657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c099780 0x7f182c14dcb0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f181800c7f0 tx=0x7f181800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1829600640 1 -- 172.21.15.46:0/461145657 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1818007820 con 0x7f182c099780 2024-03-29T08:29:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1829600640 1 -- 172.21.15.46:0/461145657 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1818007980 con 0x7f182c099780 2024-03-29T08:29:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1829600640 1 -- 172.21.15.46:0/461145657 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1818007d30 con 0x7f182c099780 2024-03-29T08:29:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1833000640 1 -- 172.21.15.46:0/461145657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182c0c66b0 con 0x7f182c099780 2024-03-29T08:29:55.768 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1833000640 1 -- 172.21.15.46:0/461145657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182c0c4290 con 0x7f182c099780 2024-03-29T08:29:55.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.761+0000 7f1833000640 1 -- 172.21.15.46:0/461145657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182c04e040 con 0x7f182c099780 2024-03-29T08:29:55.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.765+0000 7f1829600640 1 -- 172.21.15.46:0/461145657 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f18180113f0 con 0x7f182c099780 2024-03-29T08:29:55.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.765+0000 7f1829600640 1 --2- 172.21.15.46:0/461145657 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f180c066690 0x7f180c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:55.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.765+0000 7f1829600640 1 -- 172.21.15.46:0/461145657 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1818084460 con 0x7f182c099780 2024-03-29T08:29:55.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.765+0000 7f1831c00640 1 --2- 172.21.15.46:0/461145657 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f180c066690 0x7f180c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:55.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.765+0000 7f1831c00640 1 --2- 172.21.15.46:0/461145657 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f180c066690 0x7f180c068b50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1820005890 tx=0x7f1820009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:55.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.769+0000 7f1829600640 1 -- 172.21.15.46:0/461145657 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1818051810 con 0x7f182c099780 2024-03-29T08:29:55.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.929+0000 7f1833000640 1 -- 172.21.15.46:0/461145657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f182c0b2ae0 con 0x7f182c099780 2024-03-29T08:29:55.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.929+0000 7f1829600640 1 -- 172.21.15.46:0/461145657 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) v1 ==== 104+0+11 (secure 0 0 0) 0x7f181805b070 con 0x7f182c099780 2024-03-29T08:29:55.934 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:29:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.933+0000 7f1833000640 1 -- 172.21.15.46:0/461145657 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f180c066690 msgr2=0x7f180c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.933+0000 7f1833000640 1 --2- 172.21.15.46:0/461145657 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f180c066690 0x7f180c068b50 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f1820005890 tx=0x7f1820009040 comp rx=0 tx=0).stop 2024-03-29T08:29:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.933+0000 7f1833000640 1 -- 172.21.15.46:0/461145657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c099780 msgr2=0x7f182c14dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.933+0000 7f1833000640 1 --2- 172.21.15.46:0/461145657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c099780 0x7f182c14dcb0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f181800c7f0 tx=0x7f181800ccb0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.933+0000 7f1833000640 1 -- 172.21.15.46:0/461145657 shutdown_connections 2024-03-29T08:29:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.933+0000 7f1833000640 1 --2- 172.21.15.46:0/461145657 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f180c066690 0x7f180c068b50 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.933+0000 7f1833000640 1 --2- 172.21.15.46:0/461145657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f182c15ee70 0x7f182c15ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.933+0000 7f1833000640 1 --2- 172.21.15.46:0/461145657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f182c099780 0x7f182c14dcb0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.933+0000 7f1833000640 1 --2- 172.21.15.46:0/461145657 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f182c098a40 0x7f182c158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.933+0000 7f1833000640 1 -- 172.21.15.46:0/461145657 >> 172.21.15.46:0/461145657 conn(0x7f182c0b1fb0 msgr2=0x7f182c0cfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.933+0000 7f1833000640 1 -- 172.21.15.46:0/461145657 shutdown_connections 2024-03-29T08:29:55.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:55.933+0000 7f1833000640 1 -- 172.21.15.46:0/461145657 wait complete. 2024-03-29T08:29:55.955 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:29:56.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.093+0000 7f1b7ac00640 1 Processor -- start 2024-03-29T08:29:56.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b7ac00640 1 -- start start 2024-03-29T08:29:56.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b7ac00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1b74160db0 0x7f1b740cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b7ac00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b740cfb20 con 0x7f1b7415c950 2024-03-29T08:29:56.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b7ac00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b740cef10 con 0x7f1b740cea30 2024-03-29T08:29:56.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b7ac00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b740cf1f0 con 0x7f1b74160db0 2024-03-29T08:29:56.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1b7415c950 0x7f1b7406bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42194/0 (socket says 172.21.15.46:42194) 2024-03-29T08:29:56.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73e00640 1 -- 172.21.15.46:0/2696356499 learned_addr learned my addr 172.21.15.46:0/2696356499 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:56.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b78c00640 1 --2- 172.21.15.46:0/2696356499 >> v2:172.21.15.73:3300/0 conn(0x7f1b74160db0 0x7f1b740cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73400640 1 -- 172.21.15.46:0/2696356499 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753333629 0 0) 0x7f1b740cfb20 con 0x7f1b7415c950 2024-03-29T08:29:56.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73400640 1 -- 172.21.15.46:0/2696356499 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b58003530 con 0x7f1b7415c950 2024-03-29T08:29:56.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73400640 1 -- 172.21.15.46:0/2696356499 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 695939354 0 0) 0x7f1b58003530 con 0x7f1b7415c950 2024-03-29T08:29:56.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73400640 1 -- 172.21.15.46:0/2696356499 >> v2:172.21.15.73:3300/0 conn(0x7f1b74160db0 msgr2=0x7f1b740cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:56.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73400640 1 --2- 172.21.15.46:0/2696356499 >> v2:172.21.15.73:3300/0 conn(0x7f1b74160db0 0x7f1b740cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73400640 1 -- 172.21.15.46:0/2696356499 >> v1:172.21.15.73:6789/0 conn(0x7f1b740cea30 legacy=0x7f1b741606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73400640 1 -- 172.21.15.46:0/2696356499 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b740cf4d0 con 0x7f1b7415c950 2024-03-29T08:29:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73400640 1 -- 172.21.15.46:0/2696356499 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f1b64001680 con 0x7f1b7415c950 2024-03-29T08:29:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73400640 1 -- 172.21.15.46:0/2696356499 >> v1:172.21.15.46:6789/0 conn(0x7f1b7415c950 legacy=0x7f1b7406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73400640 1 --2- 172.21.15.46:0/2696356499 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b58003980 0x7f1b58003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73400640 1 --2- 172.21.15.46:0/2696356499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b580047d0 0x7f1b58004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73400640 1 --2- 172.21.15.46:0/2696356499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b58005190 0x7f1b5800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73400640 1 -- 172.21.15.46:0/2696356499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b740cf1f0 con 0x7f1b580047d0 2024-03-29T08:29:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73400640 1 -- 172.21.15.46:0/2696356499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5800bf50 con 0x7f1b58005190 2024-03-29T08:29:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73400640 1 -- 172.21.15.46:0/2696356499 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5800c110 con 0x7f1b58003980 2024-03-29T08:29:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b6b400640 1 --2- 172.21.15.46:0/2696356499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b580047d0 0x7f1b58004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b73e00640 1 --2- 172.21.15.46:0/2696356499 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b58003980 0x7f1b58003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b78c00640 1 --2- 172.21.15.46:0/2696356499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b58005190 0x7f1b5800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b6b400640 1 -- 172.21.15.46:0/2696356499 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b58003980 msgr2=0x7f1b58003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:56.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b6b400640 1 --2- 172.21.15.46:0/2696356499 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b58003980 0x7f1b58003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b6b400640 1 -- 172.21.15.46:0/2696356499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b58005190 msgr2=0x7f1b5800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b6b400640 1 --2- 172.21.15.46:0/2696356499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b58005190 0x7f1b5800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b6b400640 1 -- 172.21.15.46:0/2696356499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b58003530 con 0x7f1b580047d0 2024-03-29T08:29:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.097+0000 7f1b78c00640 1 -- 172.21.15.46:0/2696356499 reap_dead start 2024-03-29T08:29:56.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.101+0000 7f1b6b400640 1 --2- 172.21.15.46:0/2696356499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b580047d0 0x7f1b58004c50 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f1b740cee40 tx=0x7f1b5c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bf71e854a5198 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.101+0000 7f1b73400640 1 -- 172.21.15.46:0/2696356499 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c0a4ea0 con 0x7f1b580047d0 2024-03-29T08:29:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.101+0000 7f1b73400640 1 -- 172.21.15.46:0/2696356499 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5c0a7050 con 0x7f1b580047d0 2024-03-29T08:29:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.101+0000 7f1b73400640 1 -- 172.21.15.46:0/2696356499 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c0ab880 con 0x7f1b580047d0 2024-03-29T08:29:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.101+0000 7f1b7ac00640 1 -- 172.21.15.46:0/2696356499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b580047d0 msgr2=0x7f1b58004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.101+0000 7f1b7ac00640 1 --2- 172.21.15.46:0/2696356499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b580047d0 0x7f1b58004c50 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f1b740cee40 tx=0x7f1b5c01c3e0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.101+0000 7f1b7ac00640 1 -- 172.21.15.46:0/2696356499 shutdown_connections 2024-03-29T08:29:56.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.101+0000 7f1b7ac00640 1 --2- 172.21.15.46:0/2696356499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b580047d0 0x7f1b58004c50 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.101+0000 7f1b7ac00640 1 -- 172.21.15.46:0/2696356499 >> 172.21.15.46:0/2696356499 conn(0x7f1b740b21b0 msgr2=0x7f1b740b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:56.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.101+0000 7f1b7ac00640 1 -- 172.21.15.46:0/2696356499 shutdown_connections 2024-03-29T08:29:56.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.101+0000 7f1b7ac00640 1 -- 172.21.15.46:0/2696356499 wait complete. 2024-03-29T08:29:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b7ac00640 1 Processor -- start 2024-03-29T08:29:56.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b7ac00640 1 -- start start 2024-03-29T08:29:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b7ac00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b580047d0 0x7f1b7414ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b7ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74157c70 0x7f1b741e6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b7ac00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b740b4c90 0x7f1b740b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b7ac00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7414b0c0 con 0x7f1b74157c70 2024-03-29T08:29:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b7ac00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b740b5650 con 0x7f1b740b4c90 2024-03-29T08:29:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b7ac00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b741e7d10 con 0x7f1b580047d0 2024-03-29T08:29:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b6b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74157c70 0x7f1b741e6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b6b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74157c70 0x7f1b741e6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57324/0 (socket says 172.21.15.46:57324) 2024-03-29T08:29:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b6b400640 1 -- 172.21.15.46:0/1130226277 learned_addr learned my addr 172.21.15.46:0/1130226277 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b73e00640 1 --2- 172.21.15.46:0/1130226277 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b580047d0 0x7f1b7414ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b78c00640 1 --2- 172.21.15.46:0/1130226277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b740b4c90 0x7f1b740b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b6b400640 1 -- 172.21.15.46:0/1130226277 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b580047d0 msgr2=0x7f1b7414ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b6b400640 1 --2- 172.21.15.46:0/1130226277 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b580047d0 0x7f1b7414ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b6b400640 1 -- 172.21.15.46:0/1130226277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b740b4c90 msgr2=0x7f1b740b5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b6b400640 1 --2- 172.21.15.46:0/1130226277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b740b4c90 0x7f1b740b5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b6b400640 1 -- 172.21.15.46:0/1130226277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b741e7750 con 0x7f1b74157c70 2024-03-29T08:29:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b6b400640 1 --2- 172.21.15.46:0/1130226277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74157c70 0x7f1b741e6de0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c01c8c0 tx=0x7f1b58005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:56.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b70c00640 1 -- 172.21.15.46:0/1130226277 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c007450 con 0x7f1b74157c70 2024-03-29T08:29:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b70c00640 1 -- 172.21.15.46:0/1130226277 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5c0075b0 con 0x7f1b74157c70 2024-03-29T08:29:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b70c00640 1 -- 172.21.15.46:0/1130226277 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5c0b4640 con 0x7f1b74157c70 2024-03-29T08:29:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b7ac00640 1 -- 172.21.15.46:0/1130226277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b741e7a30 con 0x7f1b74157c70 2024-03-29T08:29:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b7ac00640 1 -- 172.21.15.46:0/1130226277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b741f3030 con 0x7f1b74157c70 2024-03-29T08:29:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.105+0000 7f1b7ac00640 1 -- 172.21.15.46:0/1130226277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b741f31f0 con 0x7f1b74157c70 2024-03-29T08:29:56.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.109+0000 7f1b70c00640 1 -- 172.21.15.46:0/1130226277 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1b5c0b2040 con 0x7f1b74157c70 2024-03-29T08:29:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.109+0000 7f1b70c00640 1 --2- 172.21.15.46:0/1130226277 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b50066690 0x7f1b50068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.109+0000 7f1b70c00640 1 -- 172.21.15.46:0/1130226277 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f1b5c11de60 con 0x7f1b74157c70 2024-03-29T08:29:56.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.109+0000 7f1b73e00640 1 --2- 172.21.15.46:0/1130226277 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b50066690 0x7f1b50068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.109+0000 7f1b73e00640 1 --2- 172.21.15.46:0/1130226277 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b50066690 0x7f1b50068b50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1b64000c10 tx=0x7f1b64001b50 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.113+0000 7f1b70c00640 1 -- 172.21.15.46:0/1130226277 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1b5c0eb210 con 0x7f1b74157c70 2024-03-29T08:29:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.273+0000 7f1b7ac00640 1 -- 172.21.15.46:0/1130226277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b7409e690 con 0x7f1b74157c70 2024-03-29T08:29:56.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.273+0000 7f1b70c00640 1 -- 172.21.15.46:0/1130226277 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b5c0ee830 con 0x7f1b74157c70 2024-03-29T08:29:56.278 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:29:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.277+0000 7f1b7ac00640 1 -- 172.21.15.46:0/1130226277 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b50066690 msgr2=0x7f1b50068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.277+0000 7f1b7ac00640 1 --2- 172.21.15.46:0/1130226277 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b50066690 0x7f1b50068b50 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1b64000c10 tx=0x7f1b64001b50 comp rx=0 tx=0).stop 2024-03-29T08:29:56.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.277+0000 7f1b7ac00640 1 -- 172.21.15.46:0/1130226277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74157c70 msgr2=0x7f1b741e6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.277+0000 7f1b7ac00640 1 --2- 172.21.15.46:0/1130226277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74157c70 0x7f1b741e6de0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c01c8c0 tx=0x7f1b58005800 comp rx=0 tx=0).stop 2024-03-29T08:29:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.277+0000 7f1b7ac00640 1 -- 172.21.15.46:0/1130226277 shutdown_connections 2024-03-29T08:29:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.277+0000 7f1b7ac00640 1 --2- 172.21.15.46:0/1130226277 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1b50066690 0x7f1b50068b50 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.277+0000 7f1b7ac00640 1 --2- 172.21.15.46:0/1130226277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b740b4c90 0x7f1b740b5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.277+0000 7f1b7ac00640 1 --2- 172.21.15.46:0/1130226277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74157c70 0x7f1b741e6de0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.277+0000 7f1b7ac00640 1 --2- 172.21.15.46:0/1130226277 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1b580047d0 0x7f1b7414ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.277+0000 7f1b7ac00640 1 -- 172.21.15.46:0/1130226277 >> 172.21.15.46:0/1130226277 conn(0x7f1b740b21b0 msgr2=0x7f1b7415d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.277+0000 7f1b7ac00640 1 -- 172.21.15.46:0/1130226277 shutdown_connections 2024-03-29T08:29:56.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.277+0000 7f1b7ac00640 1 -- 172.21.15.46:0/1130226277 wait complete. 2024-03-29T08:29:56.300 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:29:56.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.437+0000 7f9cae600640 1 Processor -- start 2024-03-29T08:29:56.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.437+0000 7f9cae600640 1 -- start start 2024-03-29T08:29:56.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.437+0000 7f9cae600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca806dbb0 0x7f9ca815d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.437+0000 7f9cae600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca8070c70 con 0x7f9ca815d940 2024-03-29T08:29:56.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.437+0000 7f9cae600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca8070340 con 0x7f9ca8142ab0 2024-03-29T08:29:56.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.437+0000 7f9cae600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca8070620 con 0x7f9ca806dbb0 2024-03-29T08:29:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.437+0000 7f9ca7e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9ca815d940 0x7f9ca80b1e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42198/0 (socket says 172.21.15.46:42198) 2024-03-29T08:29:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.437+0000 7f9ca7e00640 1 -- 172.21.15.46:0/2741954203 learned_addr learned my addr 172.21.15.46:0/2741954203 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.437+0000 7f9ca6a00640 1 --2- 172.21.15.46:0/2741954203 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca806dbb0 0x7f9ca815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.437+0000 7f9ca6000640 1 -- 172.21.15.46:0/2741954203 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154095119 0 0) 0x7f9ca8070c70 con 0x7f9ca815d940 2024-03-29T08:29:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.437+0000 7f9ca6000640 1 -- 172.21.15.46:0/2741954203 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c8c003530 con 0x7f9ca815d940 2024-03-29T08:29:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.437+0000 7f9ca6a00640 1 -- 172.21.15.46:0/2741954203 >> v1:172.21.15.73:6789/0 conn(0x7f9ca8142ab0 legacy=0x7f9ca806d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.437+0000 7f9ca6000640 1 -- 172.21.15.46:0/2741954203 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624224603 0 0) 0x7f9ca8070340 con 0x7f9ca8142ab0 2024-03-29T08:29:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.437+0000 7f9ca6a00640 1 -- 172.21.15.46:0/2741954203 >> v1:172.21.15.46:6789/0 conn(0x7f9ca815d940 legacy=0x7f9ca80b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.437+0000 7f9ca6a00640 1 -- 172.21.15.46:0/2741954203 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca8070900 con 0x7f9ca806dbb0 2024-03-29T08:29:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.441+0000 7f9ca6a00640 1 --2- 172.21.15.46:0/2741954203 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca806dbb0 0x7f9ca815d400 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9c940097f0 tx=0x7f9c9401c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88839c084ee589e2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.441+0000 7f9ca6000640 1 -- 172.21.15.46:0/2741954203 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c940ab3f0 con 0x7f9ca806dbb0 2024-03-29T08:29:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.441+0000 7f9ca6000640 1 -- 172.21.15.46:0/2741954203 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c940ab550 con 0x7f9ca806dbb0 2024-03-29T08:29:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.441+0000 7f9ca6000640 1 -- 172.21.15.46:0/2741954203 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c940ab8e0 con 0x7f9ca806dbb0 2024-03-29T08:29:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.441+0000 7f9cae600640 1 -- 172.21.15.46:0/2741954203 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca806dbb0 msgr2=0x7f9ca815d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.441+0000 7f9cae600640 1 --2- 172.21.15.46:0/2741954203 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca806dbb0 0x7f9ca815d400 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9c940097f0 tx=0x7f9c9401c4e0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.441+0000 7f9cae600640 1 -- 172.21.15.46:0/2741954203 shutdown_connections 2024-03-29T08:29:56.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.441+0000 7f9cae600640 1 --2- 172.21.15.46:0/2741954203 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca806dbb0 0x7f9ca815d400 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.441+0000 7f9cae600640 1 -- 172.21.15.46:0/2741954203 >> 172.21.15.46:0/2741954203 conn(0x7f9ca8066080 msgr2=0x7f9ca8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:56.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.441+0000 7f9cae600640 1 -- 172.21.15.46:0/2741954203 shutdown_connections 2024-03-29T08:29:56.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.441+0000 7f9cae600640 1 -- 172.21.15.46:0/2741954203 wait complete. 2024-03-29T08:29:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9cae600640 1 Processor -- start 2024-03-29T08:29:56.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9cae600640 1 -- start start 2024-03-29T08:29:56.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9cae600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca806dbb0 0x7f9ca8158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9cae600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ca8142ab0 0x7f9ca80aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9cae600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca815d940 0x7f9ca80abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9cae600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca81f5fa0 con 0x7f9ca815d940 2024-03-29T08:29:56.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9cae600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca81f62a0 con 0x7f9ca8142ab0 2024-03-29T08:29:56.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9cae600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca81f6580 con 0x7f9ca806dbb0 2024-03-29T08:29:56.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9ca7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca815d940 0x7f9ca80abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9ca7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca815d940 0x7f9ca80abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57328/0 (socket says 172.21.15.46:57328) 2024-03-29T08:29:56.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9ca7e00640 1 -- 172.21.15.46:0/3581488862 learned_addr learned my addr 172.21.15.46:0/3581488862 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:56.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9ca6a00640 1 --2- 172.21.15.46:0/3581488862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ca8142ab0 0x7f9ca80aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9ca7400640 1 --2- 172.21.15.46:0/3581488862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca806dbb0 0x7f9ca8158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9ca7e00640 1 -- 172.21.15.46:0/3581488862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca806dbb0 msgr2=0x7f9ca8158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9ca7e00640 1 --2- 172.21.15.46:0/3581488862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca806dbb0 0x7f9ca8158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9ca7e00640 1 -- 172.21.15.46:0/3581488862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ca8142ab0 msgr2=0x7f9ca80aba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9ca7e00640 1 --2- 172.21.15.46:0/3581488862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ca8142ab0 0x7f9ca80aba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9ca7e00640 1 -- 172.21.15.46:0/3581488862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c94009420 con 0x7f9ca815d940 2024-03-29T08:29:56.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9ca7400640 1 --2- 172.21.15.46:0/3581488862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca806dbb0 0x7f9ca8158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:56.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9ca7e00640 1 --2- 172.21.15.46:0/3581488862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca815d940 0x7f9ca80abf90 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000caa0 tx=0x7f9ca000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:56.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9c9f400640 1 -- 172.21.15.46:0/3581488862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca000ac00 con 0x7f9ca815d940 2024-03-29T08:29:56.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9c9f400640 1 -- 172.21.15.46:0/3581488862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ca000ad60 con 0x7f9ca815d940 2024-03-29T08:29:56.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9c9f400640 1 -- 172.21.15.46:0/3581488862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0012620 con 0x7f9ca815d940 2024-03-29T08:29:56.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9cae600640 1 -- 172.21.15.46:0/3581488862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca81f6860 con 0x7f9ca815d940 2024-03-29T08:29:56.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.445+0000 7f9cae600640 1 -- 172.21.15.46:0/3581488862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca81593d0 con 0x7f9ca815d940 2024-03-29T08:29:56.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.449+0000 7f9cae600640 1 -- 172.21.15.46:0/3581488862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca81f6d80 con 0x7f9ca815d940 2024-03-29T08:29:56.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.449+0000 7f9c9f400640 1 -- 172.21.15.46:0/3581488862 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9ca0007070 con 0x7f9ca815d940 2024-03-29T08:29:56.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.449+0000 7f9c9f400640 1 --2- 172.21.15.46:0/3581488862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c88066690 0x7f9c88068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.449+0000 7f9c9f400640 1 -- 172.21.15.46:0/3581488862 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f9ca00841c0 con 0x7f9ca815d940 2024-03-29T08:29:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.449+0000 7f9ca7400640 1 --2- 172.21.15.46:0/3581488862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c88066690 0x7f9c88068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.449+0000 7f9ca7400640 1 --2- 172.21.15.46:0/3581488862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c88066690 0x7f9c88068b50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9c98000f70 tx=0x7f9c98009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:56.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.453+0000 7f9c9f400640 1 -- 172.21.15.46:0/3581488862 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9ca81f6d80 con 0x7f9ca815d940 2024-03-29T08:29:56.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.629+0000 7f9cae600640 1 -- 172.21.15.46:0/3581488862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9ca81f6d80 con 0x7f9ca815d940 2024-03-29T08:29:56.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.629+0000 7f9c9f400640 1 -- 172.21.15.46:0/3581488862 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v122) v1 ==== 80+0+120 (secure 0 0 0) 0x7f9ca81f6d80 con 0x7f9ca815d940 2024-03-29T08:29:56.635 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6835/929698978 2024-03-30T08:29:45.374806+0000 2024-03-29T08:29:56.635 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6834/929698978 2024-03-30T08:29:45.374806+0000 2024-03-29T08:29:56.635 INFO:teuthology.orchestra.run.smithi046.stderr:listed 2 entries 2024-03-29T08:29:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.633+0000 7f9cae600640 1 -- 172.21.15.46:0/3581488862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c88066690 msgr2=0x7f9c88068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.633+0000 7f9cae600640 1 --2- 172.21.15.46:0/3581488862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c88066690 0x7f9c88068b50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9c98000f70 tx=0x7f9c98009040 comp rx=0 tx=0).stop 2024-03-29T08:29:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.633+0000 7f9cae600640 1 -- 172.21.15.46:0/3581488862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca815d940 msgr2=0x7f9ca80abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.633+0000 7f9cae600640 1 --2- 172.21.15.46:0/3581488862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca815d940 0x7f9ca80abf90 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000caa0 tx=0x7f9ca000cf60 comp rx=0 tx=0).stop 2024-03-29T08:29:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.633+0000 7f9cae600640 1 -- 172.21.15.46:0/3581488862 shutdown_connections 2024-03-29T08:29:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.633+0000 7f9cae600640 1 --2- 172.21.15.46:0/3581488862 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9c88066690 0x7f9c88068b50 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.633+0000 7f9cae600640 1 --2- 172.21.15.46:0/3581488862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ca8142ab0 0x7f9ca80aba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.633+0000 7f9cae600640 1 --2- 172.21.15.46:0/3581488862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ca815d940 0x7f9ca80abf90 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.633+0000 7f9cae600640 1 --2- 172.21.15.46:0/3581488862 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9ca806dbb0 0x7f9ca8158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.633+0000 7f9cae600640 1 -- 172.21.15.46:0/3581488862 >> 172.21.15.46:0/3581488862 conn(0x7f9ca8066080 msgr2=0x7f9ca80b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:56.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.633+0000 7f9cae600640 1 -- 172.21.15.46:0/3581488862 shutdown_connections 2024-03-29T08:29:56.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.633+0000 7f9cae600640 1 -- 172.21.15.46:0/3581488862 wait complete. 2024-03-29T08:29:56.656 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:29:56.656 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:29:56.665 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:29:56.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.845+0000 7fd80f200640 1 Processor -- start 2024-03-29T08:29:56.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.845+0000 7fd80f200640 1 -- start start 2024-03-29T08:29:56.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.845+0000 7fd80f200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8080ce8a0 0x7fd8080caf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.845+0000 7fd80f200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd8080cb780 0x7fd8080aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.845+0000 7fd80f200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8080cbc10 con 0x7fd8080ae810 2024-03-29T08:29:56.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.845+0000 7fd80f200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8080c9bd0 con 0x7fd8080cb780 2024-03-29T08:29:56.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.845+0000 7fd80f200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080c9eb0 con 0x7fd8080ce8a0 2024-03-29T08:29:56.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.845+0000 7fd7ff400640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd8080ae810 0x7fd8080cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:57926/0 (socket says 172.21.15.46:57926) 2024-03-29T08:29:56.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.845+0000 7fd7ff400640 1 -- 172.21.15.46:0/1524974432 learned_addr learned my addr 172.21.15.46:0/1524974432 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:56.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.845+0000 7fd807e00640 1 --2- 172.21.15.46:0/1524974432 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8080ce8a0 0x7fd8080caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.845+0000 7fd80d200640 1 --2- 172.21.15.46:0/1524974432 >> v2:172.21.15.73:3300/0 conn(0x7fd8080cb780 0x7fd8080aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.845+0000 7fd807e00640 1 -- 172.21.15.46:0/1524974432 >> v2:172.21.15.73:3300/0 conn(0x7fd8080cb780 msgr2=0x7fd8080aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:56.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.845+0000 7fd807400640 1 -- 172.21.15.46:0/1524974432 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1967914344 0 0) 0x7fd8080cbc10 con 0x7fd8080ae810 2024-03-29T08:29:56.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.845+0000 7fd807e00640 1 --2- 172.21.15.46:0/1524974432 >> v2:172.21.15.73:3300/0 conn(0x7fd8080cb780 0x7fd8080aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.845+0000 7fd807e00640 1 -- 172.21.15.46:0/1524974432 >> v1:172.21.15.73:6789/0 conn(0x7fd8080ae810 legacy=0x7fd8080cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.845+0000 7fd807e00640 1 -- 172.21.15.46:0/1524974432 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8080ca190 con 0x7fd8080ce8a0 2024-03-29T08:29:56.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.849+0000 7fd807e00640 1 --2- 172.21.15.46:0/1524974432 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8080ce8a0 0x7fd8080caf50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80097f0 tx=0x7fd7f801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=671610ee170c60c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:56.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.849+0000 7fd807400640 1 -- 172.21.15.46:0/1524974432 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8080afa20 con 0x7fd8080ce8a0 2024-03-29T08:29:56.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.849+0000 7fd807400640 1 -- 172.21.15.46:0/1524974432 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f80ab670 con 0x7fd8080ce8a0 2024-03-29T08:29:56.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.849+0000 7fd807400640 1 -- 172.21.15.46:0/1524974432 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80aba00 con 0x7fd8080ce8a0 2024-03-29T08:29:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.849+0000 7fd80f200640 1 -- 172.21.15.46:0/1524974432 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8080ce8a0 msgr2=0x7fd8080caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.849+0000 7fd80f200640 1 --2- 172.21.15.46:0/1524974432 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8080ce8a0 0x7fd8080caf50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80097f0 tx=0x7fd7f801c470 comp rx=0 tx=0).stop 2024-03-29T08:29:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.849+0000 7fd80f200640 1 -- 172.21.15.46:0/1524974432 shutdown_connections 2024-03-29T08:29:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.849+0000 7fd80f200640 1 --2- 172.21.15.46:0/1524974432 >> v2:172.21.15.73:3300/0 conn(0x7fd8080cb780 0x7fd8080aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.849+0000 7fd80f200640 1 --2- 172.21.15.46:0/1524974432 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8080ce8a0 0x7fd8080caf50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.849+0000 7fd80f200640 1 -- 172.21.15.46:0/1524974432 >> 172.21.15.46:0/1524974432 conn(0x7fd808098a40 msgr2=0x7fd8080cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:56.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.849+0000 7fd80f200640 1 -- 172.21.15.46:0/1524974432 shutdown_connections 2024-03-29T08:29:56.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.849+0000 7fd80f200640 1 -- 172.21.15.46:0/1524974432 wait complete. 2024-03-29T08:29:56.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.849+0000 7fd80f200640 1 Processor -- start 2024-03-29T08:29:56.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.849+0000 7fd80f200640 1 -- start start 2024-03-29T08:29:56.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd80f200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8080ae810 0x7fd8080cf740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd80f200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8080cb780 0x7fd8080c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd80f200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8080ce8a0 0x7fd8080c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd80f200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080c9a50 con 0x7fd8080ae810 2024-03-29T08:29:56.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd80f200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80809a3f0 con 0x7fd8080cb780 2024-03-29T08:29:56.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd80f200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd808099e00 con 0x7fd8080ce8a0 2024-03-29T08:29:56.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd807e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8080ae810 0x7fd8080cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd807e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8080ae810 0x7fd8080cf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44726/0 (socket says 172.21.15.46:44726) 2024-03-29T08:29:56.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd807e00640 1 -- 172.21.15.46:0/3787585257 learned_addr learned my addr 172.21.15.46:0/3787585257 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:56.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd7ff400640 1 --2- 172.21.15.46:0/3787585257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8080cb780 0x7fd8080c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd80d200640 1 --2- 172.21.15.46:0/3787585257 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8080ce8a0 0x7fd8080c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd807e00640 1 -- 172.21.15.46:0/3787585257 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8080ce8a0 msgr2=0x7fd8080c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd807e00640 1 --2- 172.21.15.46:0/3787585257 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8080ce8a0 0x7fd8080c7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd807e00640 1 -- 172.21.15.46:0/3787585257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8080cb780 msgr2=0x7fd8080c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:56.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd807e00640 1 --2- 172.21.15.46:0/3787585257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8080cb780 0x7fd8080c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:56.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd807e00640 1 -- 172.21.15.46:0/3787585257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f8009420 con 0x7fd8080ae810 2024-03-29T08:29:56.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd807e00640 1 --2- 172.21.15.46:0/3787585257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8080ae810 0x7fd8080cf740 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fd7f801ab20 tx=0x7fd7f8019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:56.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd804c00640 1 -- 172.21.15.46:0/3787585257 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80abea0 con 0x7fd8080ae810 2024-03-29T08:29:56.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd804c00640 1 -- 172.21.15.46:0/3787585257 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f80bf450 con 0x7fd8080ae810 2024-03-29T08:29:56.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd804c00640 1 -- 172.21.15.46:0/3787585257 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80bf7e0 con 0x7fd8080ae810 2024-03-29T08:29:56.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd80f200640 1 -- 172.21.15.46:0/3787585257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd80809d000 con 0x7fd8080ae810 2024-03-29T08:29:56.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd80f200640 1 -- 172.21.15.46:0/3787585257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80809a080 con 0x7fd8080ae810 2024-03-29T08:29:56.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd80f200640 1 -- 172.21.15.46:0/3787585257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd808066c90 con 0x7fd8080ae810 2024-03-29T08:29:56.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.853+0000 7fd804c00640 1 -- 172.21.15.46:0/3787585257 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd7f80b43f0 con 0x7fd8080ae810 2024-03-29T08:29:56.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.857+0000 7fd804c00640 1 --2- 172.21.15.46:0/3787585257 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7e4066690 0x7fd7e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:56.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.857+0000 7fd804c00640 1 -- 172.21.15.46:0/3787585257 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd7f811e650 con 0x7fd8080ae810 2024-03-29T08:29:56.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.857+0000 7fd7ff400640 1 --2- 172.21.15.46:0/3787585257 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7e4066690 0x7fd7e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:56.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.857+0000 7fd7ff400640 1 --2- 172.21.15.46:0/3787585257 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7e4066690 0x7fd7e4068b50 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0007a90 tx=0x7fd7f00079e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:56.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:56.861+0000 7fd804c00640 1 -- 172.21.15.46:0/3787585257 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd7f80eba10 con 0x7fd8080ae810 2024-03-29T08:29:57.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.021+0000 7fd80f200640 1 -- 172.21.15.46:0/3787585257 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8080d0160 con 0x7fd8080ae810 2024-03-29T08:29:57.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.021+0000 7fd804c00640 1 -- 172.21.15.46:0/3787585257 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+16194 (secure 0 0 0) 0x7fd7f80ef030 con 0x7fd8080ae810 2024-03-29T08:29:57.026 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:57.026 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":122,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:29:45.374830+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:29:12.329121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-29T08:29:13.465939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/929698978":"2024-03-30T08:29:45.374806+0000","172.21.15.46:6834/929698978":"2024-03-30T08:29:45.374806+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:29:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.029+0000 7fd80f200640 1 -- 172.21.15.46:0/3787585257 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7e4066690 msgr2=0x7fd7e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.029+0000 7fd80f200640 1 --2- 172.21.15.46:0/3787585257 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7e4066690 0x7fd7e4068b50 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0007a90 tx=0x7fd7f00079e0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.029+0000 7fd80f200640 1 -- 172.21.15.46:0/3787585257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8080ae810 msgr2=0x7fd8080cf740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.029+0000 7fd80f200640 1 --2- 172.21.15.46:0/3787585257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8080ae810 0x7fd8080cf740 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fd7f801ab20 tx=0x7fd7f8019210 comp rx=0 tx=0).stop 2024-03-29T08:29:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.029+0000 7fd80f200640 1 -- 172.21.15.46:0/3787585257 shutdown_connections 2024-03-29T08:29:57.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.029+0000 7fd80f200640 1 --2- 172.21.15.46:0/3787585257 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd7e4066690 0x7fd7e4068b50 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.029+0000 7fd80f200640 1 --2- 172.21.15.46:0/3787585257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8080cb780 0x7fd8080c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.029+0000 7fd80f200640 1 --2- 172.21.15.46:0/3787585257 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd8080ce8a0 0x7fd8080c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.029+0000 7fd80f200640 1 --2- 172.21.15.46:0/3787585257 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8080ae810 0x7fd8080cf740 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.029+0000 7fd80f200640 1 -- 172.21.15.46:0/3787585257 >> 172.21.15.46:0/3787585257 conn(0x7fd808098a40 msgr2=0x7fd8080ac310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.029+0000 7fd80f200640 1 -- 172.21.15.46:0/3787585257 shutdown_connections 2024-03-29T08:29:57.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.029+0000 7fd80f200640 1 -- 172.21.15.46:0/3787585257 wait complete. 2024-03-29T08:29:57.048 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:29:12.329121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:29:13.465939+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:29:57.049 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:29:57.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ea00640 1 Processor -- start 2024-03-29T08:29:57.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ea00640 1 -- start start 2024-03-29T08:29:57.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ea00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5b98159800 0x7f5b980ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ea00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5b980cbcd0 0x7f5b980cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ea00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b980cca90 con 0x7f5b980b17b0 2024-03-29T08:29:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ea00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b980cc610 con 0x7f5b980cbcd0 2024-03-29T08:29:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ea00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b980cd200 con 0x7f5b98159800 2024-03-29T08:29:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ca00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5b980cbcd0 0x7f5b980cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b8fe00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5b980b17b0 0x7f5b980cdfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46042/0 (socket says 172.21.15.46:46042) 2024-03-29T08:29:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b8fe00640 1 -- 172.21.15.46:0/3608066930 learned_addr learned my addr 172.21.15.46:0/3608066930 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:57.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b97e00640 1 --2- 172.21.15.46:0/3608066930 >> v2:172.21.15.73:3300/0 conn(0x7f5b98159800 0x7f5b980ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ca00640 1 -- 172.21.15.46:0/3608066930 >> v2:172.21.15.73:3300/0 conn(0x7f5b98159800 msgr2=0x7f5b980ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ca00640 1 --2- 172.21.15.46:0/3608066930 >> v2:172.21.15.73:3300/0 conn(0x7f5b98159800 0x7f5b980ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ca00640 1 -- 172.21.15.46:0/3608066930 >> v1:172.21.15.46:6789/0 conn(0x7f5b980b17b0 legacy=0x7f5b980cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b97400640 1 -- 172.21.15.46:0/3608066930 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591423046 0 0) 0x7f5b980cca90 con 0x7f5b980b17b0 2024-03-29T08:29:57.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ca00640 1 -- 172.21.15.46:0/3608066930 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b980cd4e0 con 0x7f5b980cbcd0 2024-03-29T08:29:57.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ca00640 1 --2- 172.21.15.46:0/3608066930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b980cbcd0 0x7f5b980cc0d0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f5b980d0690 tx=0x7f5b9001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c5096500c7315a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:57.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b97400640 1 -- 172.21.15.46:0/3608066930 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b900a5930 con 0x7f5b980cbcd0 2024-03-29T08:29:57.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b97400640 1 -- 172.21.15.46:0/3608066930 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b900a5a90 con 0x7f5b980cbcd0 2024-03-29T08:29:57.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b97400640 1 -- 172.21.15.46:0/3608066930 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b900a5e20 con 0x7f5b980cbcd0 2024-03-29T08:29:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ea00640 1 -- 172.21.15.46:0/3608066930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b980cbcd0 msgr2=0x7f5b980cc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ea00640 1 --2- 172.21.15.46:0/3608066930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b980cbcd0 0x7f5b980cc0d0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f5b980d0690 tx=0x7f5b9001c870 comp rx=0 tx=0).stop 2024-03-29T08:29:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ea00640 1 -- 172.21.15.46:0/3608066930 shutdown_connections 2024-03-29T08:29:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ea00640 1 --2- 172.21.15.46:0/3608066930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b980cbcd0 0x7f5b980cc0d0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ea00640 1 --2- 172.21.15.46:0/3608066930 >> v2:172.21.15.73:3300/0 conn(0x7f5b98159800 0x7f5b980ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ea00640 1 -- 172.21.15.46:0/3608066930 >> 172.21.15.46:0/3608066930 conn(0x7f5b98098a40 msgr2=0x7f5b980b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ea00640 1 -- 172.21.15.46:0/3608066930 shutdown_connections 2024-03-29T08:29:57.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ea00640 1 -- 172.21.15.46:0/3608066930 wait complete. 2024-03-29T08:29:57.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.193+0000 7f5b9ea00640 1 Processor -- start 2024-03-29T08:29:57.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ea00640 1 -- start start 2024-03-29T08:29:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ea00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b980b17b0 0x7f5b98147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ea00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5b980cbcd0 0x7f5b98155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b98159800 0x7f5b980af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ea00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b98147f50 con 0x7f5b98159800 2024-03-29T08:29:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ea00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b980aecb0 con 0x7f5b980b17b0 2024-03-29T08:29:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ea00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b98159d60 con 0x7f5b980cbcd0 2024-03-29T08:29:57.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b98159800 0x7f5b980af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ca00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b98159800 0x7f5b980af440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44756/0 (socket says 172.21.15.46:44756) 2024-03-29T08:29:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ca00640 1 -- 172.21.15.46:0/2408345171 learned_addr learned my addr 172.21.15.46:0/2408345171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b97e00640 1 --2- 172.21.15.46:0/2408345171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b980b17b0 0x7f5b98147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b8fe00640 1 --2- 172.21.15.46:0/2408345171 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5b980cbcd0 0x7f5b98155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ca00640 1 -- 172.21.15.46:0/2408345171 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5b980cbcd0 msgr2=0x7f5b98155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ca00640 1 --2- 172.21.15.46:0/2408345171 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5b980cbcd0 0x7f5b98155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ca00640 1 -- 172.21.15.46:0/2408345171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b980b17b0 msgr2=0x7f5b98147a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ca00640 1 --2- 172.21.15.46:0/2408345171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b980b17b0 0x7f5b98147a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ca00640 1 -- 172.21.15.46:0/2408345171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b90009a70 con 0x7f5b98159800 2024-03-29T08:29:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b97e00640 1 --2- 172.21.15.46:0/2408345171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b980b17b0 0x7f5b98147a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ca00640 1 --2- 172.21.15.46:0/2408345171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b98159800 0x7f5b980af440 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f5b900b0040 tx=0x7f5b90019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b94c00640 1 -- 172.21.15.46:0/2408345171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b900ade20 con 0x7f5b98159800 2024-03-29T08:29:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b94c00640 1 -- 172.21.15.46:0/2408345171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b90009450 con 0x7f5b98159800 2024-03-29T08:29:57.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b94c00640 1 -- 172.21.15.46:0/2408345171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b900b9690 con 0x7f5b98159800 2024-03-29T08:29:57.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ea00640 1 -- 172.21.15.46:0/2408345171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b98156150 con 0x7f5b98159800 2024-03-29T08:29:57.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ea00640 1 -- 172.21.15.46:0/2408345171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b980aef20 con 0x7f5b98159800 2024-03-29T08:29:57.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.197+0000 7f5b9ea00640 1 -- 172.21.15.46:0/2408345171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b980af140 con 0x7f5b98159800 2024-03-29T08:29:57.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.205+0000 7f5b94c00640 1 -- 172.21.15.46:0/2408345171 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f5b900b97f0 con 0x7f5b98159800 2024-03-29T08:29:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.205+0000 7f5b94c00640 1 --2- 172.21.15.46:0/2408345171 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5b74066690 0x7f5b74068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.205+0000 7f5b94c00640 1 -- 172.21.15.46:0/2408345171 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5b9011fb70 con 0x7f5b98159800 2024-03-29T08:29:57.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.205+0000 7f5b97e00640 1 --2- 172.21.15.46:0/2408345171 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5b74066690 0x7f5b74068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.205+0000 7f5b97e00640 1 --2- 172.21.15.46:0/2408345171 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5b74066690 0x7f5b74068b50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800b9f0 tx=0x7f5b88008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:57.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.209+0000 7f5b94c00640 1 -- 172.21.15.46:0/2408345171 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f5b900ecde0 con 0x7f5b98159800 2024-03-29T08:29:57.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.365+0000 7f5b9ea00640 1 -- 172.21.15.46:0/2408345171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b980b0b80 con 0x7f5b98159800 2024-03-29T08:29:57.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.365+0000 7f5b94c00640 1 -- 172.21.15.46:0/2408345171 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b900f0400 con 0x7f5b98159800 2024-03-29T08:29:57.368 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:29:57.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.365+0000 7f5b9ea00640 1 -- 172.21.15.46:0/2408345171 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5b74066690 msgr2=0x7f5b74068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:57.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.365+0000 7f5b9ea00640 1 --2- 172.21.15.46:0/2408345171 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5b74066690 0x7f5b74068b50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f5b8800b9f0 tx=0x7f5b88008040 comp rx=0 tx=0).stop 2024-03-29T08:29:57.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.365+0000 7f5b9ea00640 1 -- 172.21.15.46:0/2408345171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b98159800 msgr2=0x7f5b980af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.365+0000 7f5b9ea00640 1 --2- 172.21.15.46:0/2408345171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b98159800 0x7f5b980af440 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f5b900b0040 tx=0x7f5b90019610 comp rx=0 tx=0).stop 2024-03-29T08:29:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.365+0000 7f5b9ea00640 1 -- 172.21.15.46:0/2408345171 shutdown_connections 2024-03-29T08:29:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.365+0000 7f5b9ea00640 1 --2- 172.21.15.46:0/2408345171 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5b74066690 0x7f5b74068b50 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.365+0000 7f5b9ea00640 1 --2- 172.21.15.46:0/2408345171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b98159800 0x7f5b980af440 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.365+0000 7f5b9ea00640 1 --2- 172.21.15.46:0/2408345171 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5b980cbcd0 0x7f5b98155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.365+0000 7f5b9ea00640 1 --2- 172.21.15.46:0/2408345171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b980b17b0 0x7f5b98147a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.365+0000 7f5b9ea00640 1 -- 172.21.15.46:0/2408345171 >> 172.21.15.46:0/2408345171 conn(0x7f5b98098a40 msgr2=0x7f5b98069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.365+0000 7f5b9ea00640 1 -- 172.21.15.46:0/2408345171 shutdown_connections 2024-03-29T08:29:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.365+0000 7f5b9ea00640 1 -- 172.21.15.46:0/2408345171 wait complete. 2024-03-29T08:29:57.390 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:29:57.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f190fa00640 1 Processor -- start 2024-03-29T08:29:57.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f190fa00640 1 -- start start 2024-03-29T08:29:57.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f190fa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1908142a50 0x7f19080cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f190fa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815df90 0x7f19080cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f190fa00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190815d260 con 0x7f19080ae280 2024-03-29T08:29:57.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f190fa00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19080aefd0 con 0x7f1908142a50 2024-03-29T08:29:57.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f190fa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190815d4b0 con 0x7f190815df90 2024-03-29T08:29:57.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f1907e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815df90 0x7f19080cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f190d000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1908142a50 0x7f19080cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f1907e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815df90 0x7f19080cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:60036/0 (socket says 172.21.15.46:60036) 2024-03-29T08:29:57.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f1907e00640 1 -- 172.21.15.46:0/1515034733 learned_addr learned my addr 172.21.15.46:0/1515034733 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:57.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f190da00640 1 --1- 172.21.15.46:0/1515034733 >> v1:172.21.15.73:6789/0 conn(0x7f19080ae280 0x7f19080ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:57942/0 (socket says 172.21.15.46:57942) 2024-03-29T08:29:57.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f190d000640 1 -- 172.21.15.46:0/1515034733 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815df90 msgr2=0x7f19080cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:57.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f190d000640 1 --2- 172.21.15.46:0/1515034733 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815df90 0x7f19080cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f1907400640 1 -- 172.21.15.46:0/1515034733 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1384476867 0 0) 0x7f190815d260 con 0x7f19080ae280 2024-03-29T08:29:57.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f190d000640 1 -- 172.21.15.46:0/1515034733 >> v1:172.21.15.73:6789/0 conn(0x7f19080ae280 legacy=0x7f19080ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f190d000640 1 -- 172.21.15.46:0/1515034733 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190815d790 con 0x7f1908142a50 2024-03-29T08:29:57.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f1907e00640 1 --2- 172.21.15.46:0/1515034733 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815df90 0x7f19080cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:29:57.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f190d000640 1 --2- 172.21.15.46:0/1515034733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1908142a50 0x7f19080cb790 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0097c0 tx=0x7f18fc01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0dcba9ef350c61a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:57.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f1907400640 1 -- 172.21.15.46:0/1515034733 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc0ab600 con 0x7f1908142a50 2024-03-29T08:29:57.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f1907400640 1 -- 172.21.15.46:0/1515034733 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc0a7050 con 0x7f1908142a50 2024-03-29T08:29:57.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.533+0000 7f1907400640 1 -- 172.21.15.46:0/1515034733 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc0ab980 con 0x7f1908142a50 2024-03-29T08:29:57.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190fa00640 1 -- 172.21.15.46:0/1515034733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1908142a50 msgr2=0x7f19080cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:57.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190fa00640 1 --2- 172.21.15.46:0/1515034733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1908142a50 0x7f19080cb790 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0097c0 tx=0x7f18fc01c410 comp rx=0 tx=0).stop 2024-03-29T08:29:57.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190fa00640 1 -- 172.21.15.46:0/1515034733 shutdown_connections 2024-03-29T08:29:57.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190fa00640 1 --2- 172.21.15.46:0/1515034733 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815df90 0x7f19080cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190fa00640 1 --2- 172.21.15.46:0/1515034733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1908142a50 0x7f19080cb790 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190fa00640 1 -- 172.21.15.46:0/1515034733 >> 172.21.15.46:0/1515034733 conn(0x7f190806a390 msgr2=0x7f19080cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:57.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190fa00640 1 -- 172.21.15.46:0/1515034733 shutdown_connections 2024-03-29T08:29:57.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190fa00640 1 -- 172.21.15.46:0/1515034733 wait complete. 2024-03-29T08:29:57.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190fa00640 1 Processor -- start 2024-03-29T08:29:57.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190fa00640 1 -- start start 2024-03-29T08:29:57.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190fa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19080ae280 0x7f19081580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908142a50 0x7f19081349d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190fa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815df90 0x7f1908134f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190fa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1908159ca0 con 0x7f1908142a50 2024-03-29T08:29:57.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190fa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1908135450 con 0x7f19080ae280 2024-03-29T08:29:57.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190fa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190813b5e0 con 0x7f190815df90 2024-03-29T08:29:57.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f1907e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908142a50 0x7f19081349d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f1907e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908142a50 0x7f19081349d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44764/0 (socket says 172.21.15.46:44764) 2024-03-29T08:29:57.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f1907e00640 1 -- 172.21.15.46:0/2691811911 learned_addr learned my addr 172.21.15.46:0/2691811911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:57.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190d000640 1 --2- 172.21.15.46:0/2691811911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19080ae280 0x7f19081580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f190da00640 1 --2- 172.21.15.46:0/2691811911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815df90 0x7f1908134f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f1907e00640 1 -- 172.21.15.46:0/2691811911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815df90 msgr2=0x7f1908134f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f1907e00640 1 --2- 172.21.15.46:0/2691811911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815df90 0x7f1908134f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f1907e00640 1 -- 172.21.15.46:0/2691811911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19080ae280 msgr2=0x7f19081580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f1907e00640 1 --2- 172.21.15.46:0/2691811911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19080ae280 0x7f19081580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.537+0000 7f1907e00640 1 -- 172.21.15.46:0/2691811911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f40094d0 con 0x7f1908142a50 2024-03-29T08:29:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.541+0000 7f1907e00640 1 --2- 172.21.15.46:0/2691811911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908142a50 0x7f19081349d0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f18f400e960 tx=0x7f18f400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.541+0000 7f1904c00640 1 -- 172.21.15.46:0/2691811911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f400ca00 con 0x7f1908142a50 2024-03-29T08:29:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.541+0000 7f1904c00640 1 -- 172.21.15.46:0/2691811911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f400cb60 con 0x7f1908142a50 2024-03-29T08:29:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.541+0000 7f1904c00640 1 -- 172.21.15.46:0/2691811911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f400ced0 con 0x7f1908142a50 2024-03-29T08:29:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.541+0000 7f190fa00640 1 -- 172.21.15.46:0/2691811911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18fc009420 con 0x7f1908142a50 2024-03-29T08:29:57.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.541+0000 7f190fa00640 1 -- 172.21.15.46:0/2691811911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190813bba0 con 0x7f1908142a50 2024-03-29T08:29:57.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.541+0000 7f190fa00640 1 -- 172.21.15.46:0/2691811911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1908071a70 con 0x7f1908142a50 2024-03-29T08:29:57.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.541+0000 7f1904c00640 1 -- 172.21.15.46:0/2691811911 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f18f4013470 con 0x7f1908142a50 2024-03-29T08:29:57.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.541+0000 7f1904c00640 1 --2- 172.21.15.46:0/2691811911 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18e8066690 0x7f18e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.541+0000 7f1904c00640 1 -- 172.21.15.46:0/2691811911 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f18f4084360 con 0x7f1908142a50 2024-03-29T08:29:57.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.541+0000 7f190d000640 1 --2- 172.21.15.46:0/2691811911 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18e8066690 0x7f18e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.545+0000 7f190d000640 1 --2- 172.21.15.46:0/2691811911 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18e8066690 0x7f18e8068b50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0ad000 tx=0x7f18fc0a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:57.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.545+0000 7f1904c00640 1 -- 172.21.15.46:0/2691811911 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f18f40517b0 con 0x7f1908142a50 2024-03-29T08:29:57.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.705+0000 7f190fa00640 1 -- 172.21.15.46:0/2691811911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f190806ad00 con 0x7f1908142a50 2024-03-29T08:29:57.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.709+0000 7f1904c00640 1 -- 172.21.15.46:0/2691811911 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) v1 ==== 104+0+11 (secure 0 0 0) 0x7f18f4054dd0 con 0x7f1908142a50 2024-03-29T08:29:57.712 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:29:57.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.709+0000 7f190fa00640 1 -- 172.21.15.46:0/2691811911 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18e8066690 msgr2=0x7f18e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:57.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.709+0000 7f190fa00640 1 --2- 172.21.15.46:0/2691811911 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18e8066690 0x7f18e8068b50 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f18fc0ad000 tx=0x7f18fc0a6000 comp rx=0 tx=0).stop 2024-03-29T08:29:57.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.709+0000 7f190fa00640 1 -- 172.21.15.46:0/2691811911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908142a50 msgr2=0x7f19081349d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:57.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.709+0000 7f190fa00640 1 --2- 172.21.15.46:0/2691811911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908142a50 0x7f19081349d0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f18f400e960 tx=0x7f18f400ee20 comp rx=0 tx=0).stop 2024-03-29T08:29:57.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.709+0000 7f190fa00640 1 -- 172.21.15.46:0/2691811911 shutdown_connections 2024-03-29T08:29:57.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.709+0000 7f190fa00640 1 --2- 172.21.15.46:0/2691811911 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f18e8066690 0x7f18e8068b50 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.709+0000 7f190fa00640 1 --2- 172.21.15.46:0/2691811911 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f190815df90 0x7f1908134f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.709+0000 7f190fa00640 1 --2- 172.21.15.46:0/2691811911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1908142a50 0x7f19081349d0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.709+0000 7f190fa00640 1 --2- 172.21.15.46:0/2691811911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19080ae280 0x7f19081580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.709+0000 7f190fa00640 1 -- 172.21.15.46:0/2691811911 >> 172.21.15.46:0/2691811911 conn(0x7f190806a390 msgr2=0x7f19080ccf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:57.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.709+0000 7f190fa00640 1 -- 172.21.15.46:0/2691811911 shutdown_connections 2024-03-29T08:29:57.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.709+0000 7f190fa00640 1 -- 172.21.15.46:0/2691811911 wait complete. 2024-03-29T08:29:57.733 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:29:57.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.877+0000 7fc8cb400640 1 Processor -- start 2024-03-29T08:29:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.877+0000 7fc8cb400640 1 -- start start 2024-03-29T08:29:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.877+0000 7fc8cb400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc8c4143eb0 0x7fc8c40cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.877+0000 7fc8cb400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8c415efe0 0x7fc8c40cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.877+0000 7fc8cb400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8c40b2360 con 0x7fc8c40b1630 2024-03-29T08:29:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.877+0000 7fc8cb400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8c415e500 con 0x7fc8c4143eb0 2024-03-29T08:29:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.877+0000 7fc8cb400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c415e7e0 con 0x7fc8c415efe0 2024-03-29T08:29:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.877+0000 7fc8c8a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc8c4143eb0 0x7fc8c40cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.877+0000 7fc8c8a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc8c4143eb0 0x7fc8c40cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44778/0 (socket says 172.21.15.46:44778) 2024-03-29T08:29:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.877+0000 7fc8c8a00640 1 -- 172.21.15.46:0/163194127 learned_addr learned my addr 172.21.15.46:0/163194127 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:57.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.877+0000 7fc8c3e00640 1 --2- 172.21.15.46:0/163194127 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8c415efe0 0x7fc8c40cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.877+0000 7fc8c8a00640 1 -- 172.21.15.46:0/163194127 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8c415efe0 msgr2=0x7fc8c40cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:57.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.877+0000 7fc8c8a00640 1 --2- 172.21.15.46:0/163194127 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8c415efe0 0x7fc8c40cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.877+0000 7fc8c8a00640 1 -- 172.21.15.46:0/163194127 >> v1:172.21.15.73:6789/0 conn(0x7fc8c40b1630 legacy=0x7fc8c40b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:57.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.877+0000 7fc8c8a00640 1 -- 172.21.15.46:0/163194127 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c415eac0 con 0x7fc8c4143eb0 2024-03-29T08:29:57.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.877+0000 7fc8c3e00640 1 --2- 172.21.15.46:0/163194127 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8c415efe0 0x7fc8c40cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:57.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.881+0000 7fc8c8a00640 1 --2- 172.21.15.46:0/163194127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c4143eb0 0x7fc8c40cb790 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009870 tx=0x7fc8b801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c187643cb97145fd server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:57.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.881+0000 7fc8c3400640 1 -- 172.21.15.46:0/163194127 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b80ab600 con 0x7fc8c4143eb0 2024-03-29T08:29:57.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.881+0000 7fc8c3400640 1 -- 172.21.15.46:0/163194127 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b80a7050 con 0x7fc8c4143eb0 2024-03-29T08:29:57.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.881+0000 7fc8c3400640 1 -- 172.21.15.46:0/163194127 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b80ab980 con 0x7fc8c4143eb0 2024-03-29T08:29:57.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.881+0000 7fc8cb400640 1 -- 172.21.15.46:0/163194127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c4143eb0 msgr2=0x7fc8c40cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:57.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.881+0000 7fc8cb400640 1 --2- 172.21.15.46:0/163194127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c4143eb0 0x7fc8c40cb790 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fc8b8009870 tx=0x7fc8b801c500 comp rx=0 tx=0).stop 2024-03-29T08:29:57.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.881+0000 7fc8cb400640 1 -- 172.21.15.46:0/163194127 shutdown_connections 2024-03-29T08:29:57.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.881+0000 7fc8cb400640 1 --2- 172.21.15.46:0/163194127 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8c415efe0 0x7fc8c40cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.881+0000 7fc8cb400640 1 --2- 172.21.15.46:0/163194127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c4143eb0 0x7fc8c40cb790 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.881+0000 7fc8cb400640 1 -- 172.21.15.46:0/163194127 >> 172.21.15.46:0/163194127 conn(0x7fc8c406a390 msgr2=0x7fc8c40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:57.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.881+0000 7fc8cb400640 1 -- 172.21.15.46:0/163194127 shutdown_connections 2024-03-29T08:29:57.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.881+0000 7fc8cb400640 1 -- 172.21.15.46:0/163194127 wait complete. 2024-03-29T08:29:57.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.881+0000 7fc8cb400640 1 Processor -- start 2024-03-29T08:29:57.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8cb400640 1 -- start start 2024-03-29T08:29:57.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8cb400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8c40b1630 0x7fc8c4142870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8cb400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c4143eb0 0x7fc8c40c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8cb400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8c415efe0 0x7fc8c40c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8cb400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c40c3ae0 con 0x7fc8c4143eb0 2024-03-29T08:29:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8cb400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c40c3cd0 con 0x7fc8c40b1630 2024-03-29T08:29:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8cb400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c40c3fb0 con 0x7fc8c415efe0 2024-03-29T08:29:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c4143eb0 0x7fc8c40c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c4143eb0 0x7fc8c40c35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44780/0 (socket says 172.21.15.46:44780) 2024-03-29T08:29:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8c3e00640 1 -- 172.21.15.46:0/2387409703 learned_addr learned my addr 172.21.15.46:0/2387409703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8c9400640 1 --2- 172.21.15.46:0/2387409703 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8c415efe0 0x7fc8c40c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8c8a00640 1 --2- 172.21.15.46:0/2387409703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8c40b1630 0x7fc8c4142870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8c3e00640 1 -- 172.21.15.46:0/2387409703 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8c415efe0 msgr2=0x7fc8c40c4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:57.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8c3e00640 1 --2- 172.21.15.46:0/2387409703 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8c415efe0 0x7fc8c40c4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8c3e00640 1 -- 172.21.15.46:0/2387409703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8c40b1630 msgr2=0x7fc8c4142870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:57.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8c3e00640 1 --2- 172.21.15.46:0/2387409703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8c40b1630 0x7fc8c4142870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:57.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8c3e00640 1 -- 172.21.15.46:0/2387409703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b80094d0 con 0x7fc8c4143eb0 2024-03-29T08:29:57.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8c3e00640 1 --2- 172.21.15.46:0/2387409703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c4143eb0 0x7fc8c40c35a0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000c7c0 tx=0x7fc8b000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:57.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8c0c00640 1 -- 172.21.15.46:0/2387409703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b0007a30 con 0x7fc8c4143eb0 2024-03-29T08:29:57.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8c0c00640 1 -- 172.21.15.46:0/2387409703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8b0007b90 con 0x7fc8c4143eb0 2024-03-29T08:29:57.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8c0c00640 1 -- 172.21.15.46:0/2387409703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8b000ce70 con 0x7fc8c4143eb0 2024-03-29T08:29:57.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8cb400640 1 -- 172.21.15.46:0/2387409703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c40c4290 con 0x7fc8c4143eb0 2024-03-29T08:29:57.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8cb400640 1 -- 172.21.15.46:0/2387409703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c40c4850 con 0x7fc8c4143eb0 2024-03-29T08:29:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.885+0000 7fc8cb400640 1 -- 172.21.15.46:0/2387409703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c4071a70 con 0x7fc8c4143eb0 2024-03-29T08:29:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.889+0000 7fc8c0c00640 1 -- 172.21.15.46:0/2387409703 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc8b00113f0 con 0x7fc8c4143eb0 2024-03-29T08:29:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.889+0000 7fc8c0c00640 1 --2- 172.21.15.46:0/2387409703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8a4066690 0x7fc8a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.889+0000 7fc8c0c00640 1 -- 172.21.15.46:0/2387409703 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc8b00845e0 con 0x7fc8c4143eb0 2024-03-29T08:29:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.889+0000 7fc8c8a00640 1 --2- 172.21.15.46:0/2387409703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8a4066690 0x7fc8a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.889+0000 7fc8c8a00640 1 --2- 172.21.15.46:0/2387409703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8a4066690 0x7fc8a4068b50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80ad000 tx=0x7fc8b80a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:57.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:57.893+0000 7fc8c0c00640 1 -- 172.21.15.46:0/2387409703 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc8b0051990 con 0x7fc8c4143eb0 2024-03-29T08:29:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.053+0000 7fc8cb400640 1 -- 172.21.15.46:0/2387409703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8c40cadd0 con 0x7fc8c4143eb0 2024-03-29T08:29:58.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.053+0000 7fc8c0c00640 1 -- 172.21.15.46:0/2387409703 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc8b0054fb0 con 0x7fc8c4143eb0 2024-03-29T08:29:58.053 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:29:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.053+0000 7fc8cb400640 1 -- 172.21.15.46:0/2387409703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8a4066690 msgr2=0x7fc8a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.053+0000 7fc8cb400640 1 --2- 172.21.15.46:0/2387409703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8a4066690 0x7fc8a4068b50 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc8b80ad000 tx=0x7fc8b80a6000 comp rx=0 tx=0).stop 2024-03-29T08:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.053+0000 7fc8cb400640 1 -- 172.21.15.46:0/2387409703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c4143eb0 msgr2=0x7fc8c40c35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.053+0000 7fc8cb400640 1 --2- 172.21.15.46:0/2387409703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c4143eb0 0x7fc8c40c35a0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fc8b000c7c0 tx=0x7fc8b000cc80 comp rx=0 tx=0).stop 2024-03-29T08:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.053+0000 7fc8cb400640 1 -- 172.21.15.46:0/2387409703 shutdown_connections 2024-03-29T08:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.053+0000 7fc8cb400640 1 --2- 172.21.15.46:0/2387409703 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc8a4066690 0x7fc8a4068b50 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.053+0000 7fc8cb400640 1 --2- 172.21.15.46:0/2387409703 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc8c415efe0 0x7fc8c40c4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.053+0000 7fc8cb400640 1 --2- 172.21.15.46:0/2387409703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8c4143eb0 0x7fc8c40c35a0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.053+0000 7fc8cb400640 1 --2- 172.21.15.46:0/2387409703 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8c40b1630 0x7fc8c4142870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.053+0000 7fc8cb400640 1 -- 172.21.15.46:0/2387409703 >> 172.21.15.46:0/2387409703 conn(0x7fc8c406a390 msgr2=0x7fc8c40cc970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.053+0000 7fc8cb400640 1 -- 172.21.15.46:0/2387409703 shutdown_connections 2024-03-29T08:29:58.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.053+0000 7fc8cb400640 1 -- 172.21.15.46:0/2387409703 wait complete. 2024-03-29T08:29:58.078 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:29:58.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.217+0000 7fd384800640 1 Processor -- start 2024-03-29T08:29:58.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.217+0000 7fd384800640 1 -- start start 2024-03-29T08:29:58.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd384800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd37c160db0 0x7fd37c0cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd384800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37c0cfb20 con 0x7fd37c15c950 2024-03-29T08:29:58.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd384800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37c0cef10 con 0x7fd37c0cea30 2024-03-29T08:29:58.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd384800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd37c0cf1f0 con 0x7fd37c160db0 2024-03-29T08:29:58.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd381e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd37c15c950 0x7fd37c06bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46052/0 (socket says 172.21.15.46:46052) 2024-03-29T08:29:58.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd381e00640 1 -- 172.21.15.46:0/2774744036 learned_addr learned my addr 172.21.15.46:0/2774744036 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:58.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd382800640 1 --2- 172.21.15.46:0/2774744036 >> v2:172.21.15.73:3300/0 conn(0x7fd37c160db0 0x7fd37c0cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd380a00640 1 -- 172.21.15.46:0/2774744036 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4007455820 0 0) 0x7fd37c0cfb20 con 0x7fd37c15c950 2024-03-29T08:29:58.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd380a00640 1 -- 172.21.15.46:0/2774744036 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd364003530 con 0x7fd37c15c950 2024-03-29T08:29:58.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd380a00640 1 -- 172.21.15.46:0/2774744036 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245580381 0 0) 0x7fd37c0cef10 con 0x7fd37c0cea30 2024-03-29T08:29:58.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd380a00640 1 -- 172.21.15.46:0/2774744036 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd37c0cfb20 con 0x7fd37c0cea30 2024-03-29T08:29:58.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd380a00640 1 -- 172.21.15.46:0/2774744036 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 869276892 0 0) 0x7fd364003530 con 0x7fd37c15c950 2024-03-29T08:29:58.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd380a00640 1 -- 172.21.15.46:0/2774744036 >> v2:172.21.15.73:3300/0 conn(0x7fd37c160db0 msgr2=0x7fd37c0cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:58.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd380a00640 1 --2- 172.21.15.46:0/2774744036 >> v2:172.21.15.73:3300/0 conn(0x7fd37c160db0 0x7fd37c0cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd380a00640 1 -- 172.21.15.46:0/2774744036 >> v1:172.21.15.73:6789/0 conn(0x7fd37c0cea30 legacy=0x7fd37c1606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd380a00640 1 -- 172.21.15.46:0/2774744036 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c0cf4d0 con 0x7fd37c15c950 2024-03-29T08:29:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd380a00640 1 -- 172.21.15.46:0/2774744036 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fd370001680 con 0x7fd37c15c950 2024-03-29T08:29:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd380a00640 1 -- 172.21.15.46:0/2774744036 >> v1:172.21.15.46:6789/0 conn(0x7fd37c15c950 legacy=0x7fd37c06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd380a00640 1 --2- 172.21.15.46:0/2774744036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd364003b60 0x7fd364003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd380a00640 1 --2- 172.21.15.46:0/2774744036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3640047d0 0x7fd364004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd380a00640 1 --2- 172.21.15.46:0/2774744036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd364005190 0x7fd36400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd380a00640 1 -- 172.21.15.46:0/2774744036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c0cf1f0 con 0x7fd364003b60 2024-03-29T08:29:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd380a00640 1 -- 172.21.15.46:0/2774744036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36400bfb0 con 0x7fd3640047d0 2024-03-29T08:29:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd380a00640 1 -- 172.21.15.46:0/2774744036 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36400c170 con 0x7fd364005190 2024-03-29T08:29:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd381e00640 1 --2- 172.21.15.46:0/2774744036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd364003b60 0x7fd364003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd381400640 1 --2- 172.21.15.46:0/2774744036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3640047d0 0x7fd364004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd382800640 1 --2- 172.21.15.46:0/2774744036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd364005190 0x7fd36400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd381e00640 1 -- 172.21.15.46:0/2774744036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd364005190 msgr2=0x7fd36400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:58.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd381e00640 1 --2- 172.21.15.46:0/2774744036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd364005190 0x7fd36400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd381e00640 1 -- 172.21.15.46:0/2774744036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3640047d0 msgr2=0x7fd364004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:58.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd381e00640 1 --2- 172.21.15.46:0/2774744036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3640047d0 0x7fd364004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd381e00640 1 -- 172.21.15.46:0/2774744036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd364003530 con 0x7fd364003b60 2024-03-29T08:29:58.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd382800640 1 --2- 172.21.15.46:0/2774744036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd364005190 0x7fd36400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:58.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.221+0000 7fd382800640 1 -- 172.21.15.46:0/2774744036 reap_dead start 2024-03-29T08:29:58.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.225+0000 7fd381e00640 1 --2- 172.21.15.46:0/2774744036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd364003b60 0x7fd364003f60 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fd370007fb0 tx=0x7fd37001cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563392e24f13bbfe server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:58.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.225+0000 7fd380a00640 1 -- 172.21.15.46:0/2774744036 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370005ae0 con 0x7fd364003b60 2024-03-29T08:29:58.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.225+0000 7fd380a00640 1 -- 172.21.15.46:0/2774744036 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd370005c40 con 0x7fd364003b60 2024-03-29T08:29:58.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.225+0000 7fd380a00640 1 -- 172.21.15.46:0/2774744036 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3700a5a80 con 0x7fd364003b60 2024-03-29T08:29:58.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.225+0000 7fd384800640 1 -- 172.21.15.46:0/2774744036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd364003b60 msgr2=0x7fd364003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.225+0000 7fd384800640 1 --2- 172.21.15.46:0/2774744036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd364003b60 0x7fd364003f60 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fd370007fb0 tx=0x7fd37001cb40 comp rx=0 tx=0).stop 2024-03-29T08:29:58.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.225+0000 7fd384800640 1 -- 172.21.15.46:0/2774744036 shutdown_connections 2024-03-29T08:29:58.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.225+0000 7fd384800640 1 --2- 172.21.15.46:0/2774744036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd364003b60 0x7fd364003f60 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.225+0000 7fd384800640 1 -- 172.21.15.46:0/2774744036 >> 172.21.15.46:0/2774744036 conn(0x7fd37c0b21b0 msgr2=0x7fd37c0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:58.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.225+0000 7fd384800640 1 -- 172.21.15.46:0/2774744036 shutdown_connections 2024-03-29T08:29:58.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.225+0000 7fd384800640 1 -- 172.21.15.46:0/2774744036 wait complete. 2024-03-29T08:29:58.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd384800640 1 Processor -- start 2024-03-29T08:29:58.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd384800640 1 -- start start 2024-03-29T08:29:58.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd384800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd364003b60 0x7fd37c159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd384800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c1f66b0 0x7fd37c09c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd384800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd37c09c2e0 0x7fd37c09ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd384800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c159f60 con 0x7fd37c1f66b0 2024-03-29T08:29:58.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd384800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c09b810 con 0x7fd37c09c2e0 2024-03-29T08:29:58.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd384800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c09a770 con 0x7fd364003b60 2024-03-29T08:29:58.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd381400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c1f66b0 0x7fd37c09c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd381400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c1f66b0 0x7fd37c09c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44800/0 (socket says 172.21.15.46:44800) 2024-03-29T08:29:58.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd381400640 1 -- 172.21.15.46:0/3762503819 learned_addr learned my addr 172.21.15.46:0/3762503819 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd381e00640 1 --2- 172.21.15.46:0/3762503819 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd364003b60 0x7fd37c159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd382800640 1 --2- 172.21.15.46:0/3762503819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd37c09c2e0 0x7fd37c09ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd381400640 1 -- 172.21.15.46:0/3762503819 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd364003b60 msgr2=0x7fd37c159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd381400640 1 --2- 172.21.15.46:0/3762503819 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd364003b60 0x7fd37c159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd381400640 1 -- 172.21.15.46:0/3762503819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd37c09c2e0 msgr2=0x7fd37c09ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd381400640 1 --2- 172.21.15.46:0/3762503819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd37c09c2e0 0x7fd37c09ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd381400640 1 -- 172.21.15.46:0/3762503819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd37c09aa50 con 0x7fd37c1f66b0 2024-03-29T08:29:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd381400640 1 --2- 172.21.15.46:0/3762503819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c1f66b0 0x7fd37c09c780 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00f410 tx=0x7fd36c00f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd376000640 1 -- 172.21.15.46:0/3762503819 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c014630 con 0x7fd37c1f66b0 2024-03-29T08:29:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd376000640 1 -- 172.21.15.46:0/3762503819 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36c0147b0 con 0x7fd37c1f66b0 2024-03-29T08:29:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd384800640 1 -- 172.21.15.46:0/3762503819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37c09ad30 con 0x7fd37c1f66b0 2024-03-29T08:29:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd376000640 1 -- 172.21.15.46:0/3762503819 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36c014b60 con 0x7fd37c1f66b0 2024-03-29T08:29:58.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd384800640 1 -- 172.21.15.46:0/3762503819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37c09b2f0 con 0x7fd37c1f66b0 2024-03-29T08:29:58.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.229+0000 7fd384800640 1 -- 172.21.15.46:0/3762503819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37c09e690 con 0x7fd37c1f66b0 2024-03-29T08:29:58.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.233+0000 7fd376000640 1 -- 172.21.15.46:0/3762503819 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd36c014cc0 con 0x7fd37c1f66b0 2024-03-29T08:29:58.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.233+0000 7fd376000640 1 --2- 172.21.15.46:0/3762503819 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd360066690 0x7fd360068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.233+0000 7fd376000640 1 -- 172.21.15.46:0/3762503819 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd36c086e30 con 0x7fd37c1f66b0 2024-03-29T08:29:58.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.233+0000 7fd381e00640 1 --2- 172.21.15.46:0/3762503819 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd360066690 0x7fd360068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.233+0000 7fd381e00640 1 --2- 172.21.15.46:0/3762503819 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd360066690 0x7fd360068b50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd370007f80 tx=0x7fd3700b1040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:58.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.237+0000 7fd376000640 1 -- 172.21.15.46:0/3762503819 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd36c054110 con 0x7fd37c1f66b0 2024-03-29T08:29:58.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.409+0000 7fd384800640 1 -- 172.21.15.46:0/3762503819 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd37c06c600 con 0x7fd37c1f66b0 2024-03-29T08:29:58.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.413+0000 7fd376000640 1 -- 172.21.15.46:0/3762503819 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v123) v1 ==== 80+0+120 (secure 0 0 0) 0x7fd36c057730 con 0x7fd37c1f66b0 2024-03-29T08:29:58.412 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6835/929698978 2024-03-30T08:29:45.374806+0000 2024-03-29T08:29:58.412 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6834/929698978 2024-03-30T08:29:45.374806+0000 2024-03-29T08:29:58.412 INFO:teuthology.orchestra.run.smithi046.stderr:listed 2 entries 2024-03-29T08:29:58.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.413+0000 7fd384800640 1 -- 172.21.15.46:0/3762503819 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd360066690 msgr2=0x7fd360068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.413+0000 7fd384800640 1 --2- 172.21.15.46:0/3762503819 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd360066690 0x7fd360068b50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fd370007f80 tx=0x7fd3700b1040 comp rx=0 tx=0).stop 2024-03-29T08:29:58.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.413+0000 7fd384800640 1 -- 172.21.15.46:0/3762503819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c1f66b0 msgr2=0x7fd37c09c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.413+0000 7fd384800640 1 --2- 172.21.15.46:0/3762503819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c1f66b0 0x7fd37c09c780 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00f410 tx=0x7fd36c00f8d0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.413+0000 7fd384800640 1 -- 172.21.15.46:0/3762503819 shutdown_connections 2024-03-29T08:29:58.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.413+0000 7fd384800640 1 --2- 172.21.15.46:0/3762503819 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd360066690 0x7fd360068b50 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.413+0000 7fd384800640 1 --2- 172.21.15.46:0/3762503819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd37c09c2e0 0x7fd37c09ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.413+0000 7fd384800640 1 --2- 172.21.15.46:0/3762503819 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd37c1f66b0 0x7fd37c09c780 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.413+0000 7fd384800640 1 --2- 172.21.15.46:0/3762503819 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd364003b60 0x7fd37c159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.413+0000 7fd384800640 1 -- 172.21.15.46:0/3762503819 >> 172.21.15.46:0/3762503819 conn(0x7fd37c0b21b0 msgr2=0x7fd37c06b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:58.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.413+0000 7fd384800640 1 -- 172.21.15.46:0/3762503819 shutdown_connections 2024-03-29T08:29:58.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.413+0000 7fd384800640 1 -- 172.21.15.46:0/3762503819 wait complete. 2024-03-29T08:29:58.432 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-03-29T08:29:58.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:58.433 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T08:29:58.451 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[51965]: fuse finished with error 0 and tester_r 0 2024-03-29T08:29:58.453 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:29:58.470 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:29:58.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:29:58.470 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:29:58.520 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:29:58.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.705+0000 7f9b20000640 1 Processor -- start 2024-03-29T08:29:58.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.705+0000 7f9b20000640 1 -- start start 2024-03-29T08:29:58.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.705+0000 7f9b20000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b18144080 0x7f9b180ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.705+0000 7f9b20000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b18144460 con 0x7f9b180cdac0 2024-03-29T08:29:58.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.705+0000 7f9b20000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b1806c750 con 0x7f9b1806d510 2024-03-29T08:29:58.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.705+0000 7f9b20000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1806ca30 con 0x7f9b18144080 2024-03-29T08:29:58.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.709+0000 7f9b1d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b18144080 0x7f9b180ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.709+0000 7f9b1d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9b18144080 0x7f9b180ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44816/0 (socket says 172.21.15.46:44816) 2024-03-29T08:29:58.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.709+0000 7f9b1d600640 1 -- 172.21.15.46:0/1651830626 learned_addr learned my addr 172.21.15.46:0/1651830626 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:58.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.709+0000 7f9b1e000640 1 --1- 172.21.15.46:0/1651830626 >> v1:172.21.15.46:6789/0 conn(0x7f9b180cdac0 0x7f9b1815be90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46056/0 (socket says 172.21.15.46:46056) 2024-03-29T08:29:58.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.709+0000 7f9b1d600640 1 -- 172.21.15.46:0/1651830626 >> v1:172.21.15.73:6789/0 conn(0x7f9b1806d510 legacy=0x7f9b180cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.709+0000 7f9b1d600640 1 -- 172.21.15.46:0/1651830626 >> v1:172.21.15.46:6789/0 conn(0x7f9b180cdac0 legacy=0x7f9b1815be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.709+0000 7f9b1d600640 1 -- 172.21.15.46:0/1651830626 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1806cd10 con 0x7f9b18144080 2024-03-29T08:29:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.709+0000 7f9b1d600640 1 --2- 172.21.15.46:0/1651830626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b18144080 0x7f9b180ce410 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009870 tx=0x7f9b0c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740335922d58f172 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.709+0000 7f9b17e00640 1 -- 172.21.15.46:0/1651830626 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c0ab670 con 0x7f9b18144080 2024-03-29T08:29:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.709+0000 7f9b17e00640 1 -- 172.21.15.46:0/1651830626 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0c0ab7d0 con 0x7f9b18144080 2024-03-29T08:29:58.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.709+0000 7f9b17e00640 1 -- 172.21.15.46:0/1651830626 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c0abb60 con 0x7f9b18144080 2024-03-29T08:29:58.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b20000640 1 -- 172.21.15.46:0/1651830626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b18144080 msgr2=0x7f9b180ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b20000640 1 --2- 172.21.15.46:0/1651830626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b18144080 0x7f9b180ce410 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009870 tx=0x7f9b0c01c560 comp rx=0 tx=0).stop 2024-03-29T08:29:58.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b20000640 1 -- 172.21.15.46:0/1651830626 shutdown_connections 2024-03-29T08:29:58.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b20000640 1 --2- 172.21.15.46:0/1651830626 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b18144080 0x7f9b180ce410 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b20000640 1 -- 172.21.15.46:0/1651830626 >> 172.21.15.46:0/1651830626 conn(0x7f9b18065c80 msgr2=0x7f9b18069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:58.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b20000640 1 -- 172.21.15.46:0/1651830626 shutdown_connections 2024-03-29T08:29:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b20000640 1 -- 172.21.15.46:0/1651830626 wait complete. 2024-03-29T08:29:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b20000640 1 Processor -- start 2024-03-29T08:29:58.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b20000640 1 -- start start 2024-03-29T08:29:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b20000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b1806d510 0x7f9b1813af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b20000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b180cdac0 0x7f9b181402e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b20000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9b18144080 0x7f9b180b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b20000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1813efa0 con 0x7f9b180cdac0 2024-03-29T08:29:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b20000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1813f240 con 0x7f9b1806d510 2024-03-29T08:29:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b20000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1813f520 con 0x7f9b18144080 2024-03-29T08:29:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b1cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b180cdac0 0x7f9b181402e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b1cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b180cdac0 0x7f9b181402e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44832/0 (socket says 172.21.15.46:44832) 2024-03-29T08:29:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b1cc00640 1 -- 172.21.15.46:0/1525509080 learned_addr learned my addr 172.21.15.46:0/1525509080 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b1d600640 1 --2- 172.21.15.46:0/1525509080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b1806d510 0x7f9b1813af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b1e000640 1 --2- 172.21.15.46:0/1525509080 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9b18144080 0x7f9b180b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b1cc00640 1 -- 172.21.15.46:0/1525509080 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9b18144080 msgr2=0x7f9b180b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b1cc00640 1 --2- 172.21.15.46:0/1525509080 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9b18144080 0x7f9b180b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b1cc00640 1 -- 172.21.15.46:0/1525509080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b1806d510 msgr2=0x7f9b1813af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b1cc00640 1 --2- 172.21.15.46:0/1525509080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b1806d510 0x7f9b1813af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b1cc00640 1 -- 172.21.15.46:0/1525509080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c0094d0 con 0x7f9b180cdac0 2024-03-29T08:29:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.713+0000 7f9b1e000640 1 --2- 172.21.15.46:0/1525509080 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9b18144080 0x7f9b180b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.717+0000 7f9b1cc00640 1 --2- 172.21.15.46:0/1525509080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b180cdac0 0x7f9b181402e0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400c390 tx=0x7f9b0400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.717+0000 7f9b15600640 1 -- 172.21.15.46:0/1525509080 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b040052d0 con 0x7f9b180cdac0 2024-03-29T08:29:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.717+0000 7f9b15600640 1 -- 172.21.15.46:0/1525509080 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b04005430 con 0x7f9b180cdac0 2024-03-29T08:29:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.717+0000 7f9b20000640 1 -- 172.21.15.46:0/1525509080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1813f800 con 0x7f9b180cdac0 2024-03-29T08:29:58.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.717+0000 7f9b15600640 1 -- 172.21.15.46:0/1525509080 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b04003820 con 0x7f9b180cdac0 2024-03-29T08:29:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.717+0000 7f9b20000640 1 -- 172.21.15.46:0/1525509080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1813fdc0 con 0x7f9b180cdac0 2024-03-29T08:29:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.717+0000 7f9b20000640 1 -- 172.21.15.46:0/1525509080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b18066c90 con 0x7f9b180cdac0 2024-03-29T08:29:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.717+0000 7f9b15600640 1 -- 172.21.15.46:0/1525509080 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9b04003980 con 0x7f9b180cdac0 2024-03-29T08:29:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.717+0000 7f9b15600640 1 --2- 172.21.15.46:0/1525509080 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9af80666c0 0x7f9af8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:58.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.717+0000 7f9b15600640 1 -- 172.21.15.46:0/1525509080 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9b04083750 con 0x7f9b180cdac0 2024-03-29T08:29:58.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.717+0000 7f9b1d600640 1 --2- 172.21.15.46:0/1525509080 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9af80666c0 0x7f9af8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:58.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.721+0000 7f9b1d600640 1 --2- 172.21.15.46:0/1525509080 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9af80666c0 0x7f9af8068b80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0af000 tx=0x7f9b0c0b6040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:58.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.721+0000 7f9b15600640 1 -- 172.21.15.46:0/1525509080 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9b04050a40 con 0x7f9b180cdac0 2024-03-29T08:29:58.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.921+0000 7f9b20000640 1 -- 172.21.15.46:0/1525509080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b180b2f60 con 0x7f9b180cdac0 2024-03-29T08:29:58.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.921+0000 7f9b15600640 1 -- 172.21.15.46:0/1525509080 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4737 (secure 0 0 0) 0x7f9b04054060 con 0x7f9b180cdac0 2024-03-29T08:29:58.922 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:29:58.923 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2506132630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2506132630},{"type":"v1","addr":"172.21.15.73:6836","nonce":2506132630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2108115501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2108115501},{"type":"v1","addr":"172.21.15.73:6837","nonce":2108115501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":9045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3210062895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3210062895},{"type":"v1","addr":"172.21.15.46:6833","nonce":3210062895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:29:14.470858+0000","modified":"2024-03-29T08:29:49.396868+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9441},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9441":{"gid":9441,"name":"a","rank":0,"incarnation":97,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/276185207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":276185207},{"type":"v1","addr":"172.21.15.46:6835","nonce":276185207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-29T08:29:58.923 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 100 2024-03-29T08:29:58.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.925+0000 7f9b20000640 1 -- 172.21.15.46:0/1525509080 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9af80666c0 msgr2=0x7f9af8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.925+0000 7f9b20000640 1 --2- 172.21.15.46:0/1525509080 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9af80666c0 0x7f9af8068b80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0af000 tx=0x7f9b0c0b6040 comp rx=0 tx=0).stop 2024-03-29T08:29:58.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.925+0000 7f9b20000640 1 -- 172.21.15.46:0/1525509080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b180cdac0 msgr2=0x7f9b181402e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:58.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.925+0000 7f9b20000640 1 --2- 172.21.15.46:0/1525509080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b180cdac0 0x7f9b181402e0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f9b0400c390 tx=0x7f9b0400c850 comp rx=0 tx=0).stop 2024-03-29T08:29:58.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.925+0000 7f9b20000640 1 -- 172.21.15.46:0/1525509080 shutdown_connections 2024-03-29T08:29:58.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.925+0000 7f9b20000640 1 --2- 172.21.15.46:0/1525509080 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9af80666c0 0x7f9af8068b80 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.925+0000 7f9b20000640 1 --2- 172.21.15.46:0/1525509080 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9b18144080 0x7f9b180b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.925+0000 7f9b20000640 1 --2- 172.21.15.46:0/1525509080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b180cdac0 0x7f9b181402e0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.925+0000 7f9b20000640 1 --2- 172.21.15.46:0/1525509080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b1806d510 0x7f9b1813af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:58.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.925+0000 7f9b20000640 1 -- 172.21.15.46:0/1525509080 >> 172.21.15.46:0/1525509080 conn(0x7f9b18065c80 msgr2=0x7f9b18159050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:58.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.925+0000 7f9b20000640 1 -- 172.21.15.46:0/1525509080 shutdown_connections 2024-03-29T08:29:58.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:58.925+0000 7f9b20000640 1 -- 172.21.15.46:0/1525509080 wait complete. 2024-03-29T08:29:58.943 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parent mytag123 2024-03-29T08:29:59.111 INFO:teuthology.orchestra.run.smithi046.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-03-29T08:29:59.112 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-03-29T08:29:59.112 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-03-29T08:29:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.157+0000 7ff1f9d85080 1 Processor -- start 2024-03-29T08:29:59.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.157+0000 7ff1f9d85080 1 -- start start 2024-03-29T08:29:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.157+0000 7ff1f9d85080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560538951840 0x5605389dc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.157+0000 7ff1f9d85080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605389de610 0x5605389e3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.157+0000 7ff1f9d85080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605389de9f0 0x560538950f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.157+0000 7ff1f9d85080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560538850600 con 0x5605389de610 2024-03-29T08:29:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.157+0000 7ff1f9d85080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560538847690 con 0x5605389de9f0 2024-03-29T08:29:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.157+0000 7ff1f9d85080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5605389dad00 con 0x560538951840 2024-03-29T08:29:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.157+0000 7ff1f6600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605389de610 0x5605389e3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.157+0000 7ff1f6600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605389de610 0x5605389e3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44842/0 (socket says 172.21.15.46:44842) 2024-03-29T08:29:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.157+0000 7ff1f6600640 1 -- 172.21.15.46:0/3241401077 learned_addr learned my addr 172.21.15.46:0/3241401077 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.157+0000 7ff1f7000640 1 --2- 172.21.15.46:0/3241401077 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560538951840 0x5605389dc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.157+0000 7ff1f7a00640 1 --2- 172.21.15.46:0/3241401077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605389de9f0 0x560538950f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f6600640 1 -- 172.21.15.46:0/3241401077 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560538951840 msgr2=0x5605389dc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f6600640 1 --2- 172.21.15.46:0/3241401077 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560538951840 0x5605389dc020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f6600640 1 -- 172.21.15.46:0/3241401077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605389de9f0 msgr2=0x560538950f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f6600640 1 --2- 172.21.15.46:0/3241401077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605389de9f0 0x560538950f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f6600640 1 -- 172.21.15.46:0/3241401077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e4005df0 con 0x5605389de610 2024-03-29T08:29:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f6600640 1 --2- 172.21.15.46:0/3241401077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605389de610 0x5605389e3d30 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40af040 tx=0x7ff1e4003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d5b3b5c2bed837 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f5200640 1 -- 172.21.15.46:0/3241401077 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40a4a20 con 0x5605389de610 2024-03-29T08:29:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f5200640 1 -- 172.21.15.46:0/3241401077 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e40a4b80 con 0x5605389de610 2024-03-29T08:29:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f5200640 1 -- 172.21.15.46:0/3241401077 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40ac440 con 0x5605389de610 2024-03-29T08:29:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f9d85080 1 -- 172.21.15.46:0/3241401077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605389de610 msgr2=0x5605389e3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f9d85080 1 --2- 172.21.15.46:0/3241401077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605389de610 0x5605389e3d30 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7ff1e40af040 tx=0x7ff1e4003030 comp rx=0 tx=0).stop 2024-03-29T08:29:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f9d85080 1 -- 172.21.15.46:0/3241401077 shutdown_connections 2024-03-29T08:29:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f9d85080 1 --2- 172.21.15.46:0/3241401077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5605389de9f0 0x560538950f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f9d85080 1 --2- 172.21.15.46:0/3241401077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605389de610 0x5605389e3d30 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f9d85080 1 --2- 172.21.15.46:0/3241401077 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560538951840 0x5605389dc020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f9d85080 1 -- 172.21.15.46:0/3241401077 >> 172.21.15.46:0/3241401077 conn(0x560538909cf0 msgr2=0x56053890a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f9d85080 1 -- 172.21.15.46:0/3241401077 shutdown_connections 2024-03-29T08:29:59.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f9d85080 1 -- 172.21.15.46:0/3241401077 wait complete. 2024-03-29T08:29:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f9d85080 1 Processor -- start 2024-03-29T08:29:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f9d85080 1 -- start start 2024-03-29T08:29:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f9d85080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560538951840 0x5605389e6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f9d85080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605389de610 0x5605389d7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f9d85080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5605389de9f0 0x560538935410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f9d85080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605389e5a50 con 0x5605389de610 2024-03-29T08:29:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f9d85080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605389e5cf0 con 0x560538951840 2024-03-29T08:29:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f9d85080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5605389e5f90 con 0x5605389de9f0 2024-03-29T08:29:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f6600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605389de610 0x5605389d7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f6600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605389de610 0x5605389d7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44848/0 (socket says 172.21.15.46:44848) 2024-03-29T08:29:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f6600640 1 -- 172.21.15.46:0/3947620760 learned_addr learned my addr 172.21.15.46:0/3947620760 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f7000640 1 --2- 172.21.15.46:0/3947620760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560538951840 0x5605389e6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f7a00640 1 --2- 172.21.15.46:0/3947620760 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5605389de9f0 0x560538935410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f6600640 1 -- 172.21.15.46:0/3947620760 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5605389de9f0 msgr2=0x560538935410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f6600640 1 --2- 172.21.15.46:0/3947620760 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5605389de9f0 0x560538935410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f6600640 1 -- 172.21.15.46:0/3947620760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560538951840 msgr2=0x5605389e6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f6600640 1 --2- 172.21.15.46:0/3947620760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560538951840 0x5605389e6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f6600640 1 -- 172.21.15.46:0/3947620760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560538852970 con 0x5605389de610 2024-03-29T08:29:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.161+0000 7ff1f6600640 1 --2- 172.21.15.46:0/3947620760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605389de610 0x5605389d7e40 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4005410 tx=0x7ff1e4018560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.165+0000 7ff1ef400640 1 -- 172.21.15.46:0/3947620760 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40a4a20 con 0x5605389de610 2024-03-29T08:29:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.165+0000 7ff1ef400640 1 -- 172.21.15.46:0/3947620760 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e40c7440 con 0x5605389de610 2024-03-29T08:29:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.165+0000 7ff1ef400640 1 -- 172.21.15.46:0/3947620760 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40c77d0 con 0x5605389de610 2024-03-29T08:29:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.165+0000 7ff1f9d85080 1 -- 172.21.15.46:0/3947620760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560538879a60 con 0x5605389de610 2024-03-29T08:29:59.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.165+0000 7ff1f9d85080 1 -- 172.21.15.46:0/3947620760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56053885f8c0 con 0x5605389de610 2024-03-29T08:29:59.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.165+0000 7ff1ef400640 1 -- 172.21.15.46:0/3947620760 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff1e40b3070 con 0x5605389de610 2024-03-29T08:29:59.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.165+0000 7ff1ef400640 1 --2- 172.21.15.46:0/3947620760 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff1d8065070 0x7ff1d8067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.165+0000 7ff1ef400640 1 -- 172.21.15.46:0/3947620760 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff1e411f770 con 0x5605389de610 2024-03-29T08:29:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.165+0000 7ff1f9d85080 1 --2- 172.21.15.46:0/3947620760 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x560538935950 0x560538a768c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.165+0000 7ff1f7000640 1 --2- 172.21.15.46:0/3947620760 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff1d8065070 0x7ff1d8067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.165+0000 7ff1f9d85080 1 -- 172.21.15.46:0/3947620760 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:1 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) v8 -- 0x560538a76e00 con 0x560538935950 2024-03-29T08:29:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.165+0000 7ff1f7a00640 1 --2- 172.21.15.46:0/3947620760 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x560538935950 0x560538a768c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.165+0000 7ff1f7000640 1 --2- 172.21.15.46:0/3947620760 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff1d8065070 0x7ff1d8067530 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8005620 tx=0x7ff1e800c040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.169+0000 7ff1f7a00640 1 --2- 172.21.15.46:0/3947620760 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x560538935950 0x560538a768c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.169+0000 7ff1f7a00640 1 -- 172.21.15.46:0/3947620760 <== osd.7 v2:172.21.15.73:6816/27700 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff1f0002a50 con 0x560538935950 2024-03-29T08:29:59.169 INFO:teuthology.orchestra.run.smithi046.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-03-29T08:29:59.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.169+0000 7ff1f9d85080 1 -- 172.21.15.46:0/3947620760 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x560538935950 msgr2=0x560538a768c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.169+0000 7ff1f9d85080 1 --2- 172.21.15.46:0/3947620760 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x560538935950 0x560538a768c0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.169+0000 7ff1f9d85080 1 -- 172.21.15.46:0/3947620760 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff1d8065070 msgr2=0x7ff1d8067530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.169+0000 7ff1f9d85080 1 --2- 172.21.15.46:0/3947620760 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff1d8065070 0x7ff1d8067530 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff1e8005620 tx=0x7ff1e800c040 comp rx=0 tx=0).stop 2024-03-29T08:29:59.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.169+0000 7ff1f9d85080 1 -- 172.21.15.46:0/3947620760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605389de610 msgr2=0x5605389d7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.169+0000 7ff1f9d85080 1 --2- 172.21.15.46:0/3947620760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5605389de610 0x5605389d7e40 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4005410 tx=0x7ff1e4018560 comp rx=0 tx=0).stop 2024-03-29T08:29:59.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.169+0000 7ff1f7a00640 1 -- 172.21.15.46:0/3947620760 reap_dead start 2024-03-29T08:29:59.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.169+0000 7ff1f9d85080 1 -- 172.21.15.46:0/3947620760 shutdown_connections 2024-03-29T08:29:59.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.169+0000 7ff1f9d85080 1 -- 172.21.15.46:0/3947620760 >> 172.21.15.46:0/3947620760 conn(0x560538909cf0 msgr2=0x56053890a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.169+0000 7ff1f9d85080 1 -- 172.21.15.46:0/3947620760 shutdown_connections 2024-03-29T08:29:59.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.169+0000 7ff1f9d85080 1 -- 172.21.15.46:0/3947620760 wait complete. 2024-03-29T08:29:59.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:29:59.173 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-03-29T08:29:59.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27da49080 1 Processor -- start 2024-03-29T08:29:59.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27da49080 1 -- start start 2024-03-29T08:29:59.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27da49080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563b05e2be70 0x563b05ecfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27da49080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b05ed0610 0x563b05e429d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27da49080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b05ed09f0 0x563b05ec9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27da49080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b05da71e0 con 0x563b05ed09f0 2024-03-29T08:29:59.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27da49080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b05d42960 con 0x563b05ed0610 2024-03-29T08:29:59.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27da49080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b05d42600 con 0x563b05e2be70 2024-03-29T08:29:59.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b05ed09f0 0x563b05ec9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b05ed09f0 0x563b05ec9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44860/0 (socket says 172.21.15.46:44860) 2024-03-29T08:29:59.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27ac00640 1 -- 172.21.15.46:0/3150532036 learned_addr learned my addr 172.21.15.46:0/3150532036 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:59.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27a200640 1 --2- 172.21.15.46:0/3150532036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563b05e2be70 0x563b05ecfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff279800640 1 --2- 172.21.15.46:0/3150532036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b05ed0610 0x563b05e429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27ac00640 1 -- 172.21.15.46:0/3150532036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563b05e2be70 msgr2=0x563b05ecfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27ac00640 1 --2- 172.21.15.46:0/3150532036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563b05e2be70 0x563b05ecfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27ac00640 1 -- 172.21.15.46:0/3150532036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b05ed0610 msgr2=0x563b05e429d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27ac00640 1 --2- 172.21.15.46:0/3150532036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b05ed0610 0x563b05e429d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27ac00640 1 -- 172.21.15.46:0/3150532036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c005d30 con 0x563b05ed09f0 2024-03-29T08:29:59.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff279800640 1 --2- 172.21.15.46:0/3150532036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b05ed0610 0x563b05e429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:59.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff27ac00640 1 --2- 172.21.15.46:0/3150532036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b05ed09f0 0x563b05ec9580 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ff274006f60 tx=0x7ff274006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be72ce7bccfcc77 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff273e00640 1 -- 172.21.15.46:0/3150532036 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27400f710 con 0x563b05ed09f0 2024-03-29T08:29:59.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff273e00640 1 -- 172.21.15.46:0/3150532036 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27400f870 con 0x563b05ed09f0 2024-03-29T08:29:59.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.265+0000 7ff273e00640 1 -- 172.21.15.46:0/3150532036 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27400fc20 con 0x563b05ed09f0 2024-03-29T08:29:59.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 -- 172.21.15.46:0/3150532036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b05ed09f0 msgr2=0x563b05ec9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 --2- 172.21.15.46:0/3150532036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b05ed09f0 0x563b05ec9580 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7ff274006f60 tx=0x7ff274006f90 comp rx=0 tx=0).stop 2024-03-29T08:29:59.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 -- 172.21.15.46:0/3150532036 shutdown_connections 2024-03-29T08:29:59.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 --2- 172.21.15.46:0/3150532036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b05ed0610 0x563b05e429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 --2- 172.21.15.46:0/3150532036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b05ed09f0 0x563b05ec9580 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 --2- 172.21.15.46:0/3150532036 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563b05e2be70 0x563b05ecfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 -- 172.21.15.46:0/3150532036 >> 172.21.15.46:0/3150532036 conn(0x563b05dfbcf0 msgr2=0x563b05dfe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 -- 172.21.15.46:0/3150532036 shutdown_connections 2024-03-29T08:29:59.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 -- 172.21.15.46:0/3150532036 wait complete. 2024-03-29T08:29:59.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 Processor -- start 2024-03-29T08:29:59.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 -- start start 2024-03-29T08:29:59.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b05e2be70 0x563b05e25310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563b05ed0610 0x563b05eca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b05ed09f0 0x563b05ecabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b05d39690 con 0x563b05ed09f0 2024-03-29T08:29:59.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b05ecf2d0 con 0x563b05e2be70 2024-03-29T08:29:59.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b05ecf530 con 0x563b05ed0610 2024-03-29T08:29:59.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b05ed09f0 0x563b05ecabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27ac00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b05ed09f0 0x563b05ecabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44872/0 (socket says 172.21.15.46:44872) 2024-03-29T08:29:59.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27ac00640 1 -- 172.21.15.46:0/3990704884 learned_addr learned my addr 172.21.15.46:0/3990704884 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:59.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff279800640 1 --2- 172.21.15.46:0/3990704884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563b05ed0610 0x563b05eca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27a200640 1 --2- 172.21.15.46:0/3990704884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b05e2be70 0x563b05e25310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27ac00640 1 -- 172.21.15.46:0/3990704884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563b05ed0610 msgr2=0x563b05eca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27ac00640 1 --2- 172.21.15.46:0/3990704884 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x563b05ed0610 0x563b05eca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27ac00640 1 -- 172.21.15.46:0/3990704884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b05e2be70 msgr2=0x563b05e25310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27ac00640 1 --2- 172.21.15.46:0/3990704884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563b05e2be70 0x563b05e25310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27ac00640 1 -- 172.21.15.46:0/3990704884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b05d6ba60 con 0x563b05ed09f0 2024-03-29T08:29:59.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27ac00640 1 --2- 172.21.15.46:0/3990704884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b05ed09f0 0x563b05ecabf0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7ff274001dd0 tx=0x7ff274001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff272a00640 1 -- 172.21.15.46:0/3990704884 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27401e890 con 0x563b05ed09f0 2024-03-29T08:29:59.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff272a00640 1 -- 172.21.15.46:0/3990704884 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27401ea10 con 0x563b05ed09f0 2024-03-29T08:29:59.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff272a00640 1 -- 172.21.15.46:0/3990704884 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27401edc0 con 0x563b05ed09f0 2024-03-29T08:29:59.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 -- 172.21.15.46:0/3990704884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563b05d44d00 con 0x563b05ed09f0 2024-03-29T08:29:59.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.269+0000 7ff27da49080 1 -- 172.21.15.46:0/3990704884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b05d44760 con 0x563b05ed09f0 2024-03-29T08:29:59.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.273+0000 7ff272a00640 1 -- 172.21.15.46:0/3990704884 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff274012400 con 0x563b05ed09f0 2024-03-29T08:29:59.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.273+0000 7ff272a00640 1 --2- 172.21.15.46:0/3990704884 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff25c0650a0 0x7ff25c067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.273+0000 7ff272a00640 1 -- 172.21.15.46:0/3990704884 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff2740849c0 con 0x563b05ed09f0 2024-03-29T08:29:59.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.273+0000 7ff27da49080 1 --2- 172.21.15.46:0/3990704884 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x563b05e25850 0x563b05ecbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.273+0000 7ff27a200640 1 --2- 172.21.15.46:0/3990704884 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff25c0650a0 0x7ff25c067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.273+0000 7ff27da49080 1 -- 172.21.15.46:0/3990704884 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:1 17.12 17:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) v8 -- 0x563b05e2b210 con 0x563b05e25850 2024-03-29T08:29:59.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.273+0000 7ff279800640 1 --2- 172.21.15.46:0/3990704884 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x563b05e25850 0x563b05ecbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.273+0000 7ff27a200640 1 --2- 172.21.15.46:0/3990704884 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff25c0650a0 0x7ff25c067560 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0ae000 tx=0x7ff26c0a6070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.273+0000 7ff279800640 1 --2- 172.21.15.46:0/3990704884 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x563b05e25850 0x563b05ecbe70 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.273+0000 7ff279800640 1 -- 172.21.15.46:0/3990704884 <== osd.7 v2:172.21.15.73:6816/27700 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7ff26800e810 con 0x563b05e25850 2024-03-29T08:29:59.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.273+0000 7ff27da49080 1 -- 172.21.15.46:0/3990704884 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x563b05e25850 msgr2=0x563b05ecbe70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.273+0000 7ff27da49080 1 --2- 172.21.15.46:0/3990704884 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x563b05e25850 0x563b05ecbe70 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.277+0000 7ff27da49080 1 -- 172.21.15.46:0/3990704884 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff25c0650a0 msgr2=0x7ff25c067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.277+0000 7ff27da49080 1 --2- 172.21.15.46:0/3990704884 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff25c0650a0 0x7ff25c067560 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0ae000 tx=0x7ff26c0a6070 comp rx=0 tx=0).stop 2024-03-29T08:29:59.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.277+0000 7ff27da49080 1 -- 172.21.15.46:0/3990704884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b05ed09f0 msgr2=0x563b05ecabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.277+0000 7ff27da49080 1 --2- 172.21.15.46:0/3990704884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563b05ed09f0 0x563b05ecabf0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7ff274001dd0 tx=0x7ff274001260 comp rx=0 tx=0).stop 2024-03-29T08:29:59.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.277+0000 7ff27ac00640 1 -- 172.21.15.46:0/3990704884 reap_dead start 2024-03-29T08:29:59.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.277+0000 7ff27da49080 1 -- 172.21.15.46:0/3990704884 shutdown_connections 2024-03-29T08:29:59.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.277+0000 7ff27da49080 1 -- 172.21.15.46:0/3990704884 >> 172.21.15.46:0/3990704884 conn(0x563b05dfbcf0 msgr2=0x563b05dfe460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.277+0000 7ff27da49080 1 -- 172.21.15.46:0/3990704884 shutdown_connections 2024-03-29T08:29:59.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.277+0000 7ff27da49080 1 -- 172.21.15.46:0/3990704884 wait complete. 2024-03-29T08:29:59.280 INFO:teuthology.orchestra.run.smithi046.stdout:mytag123 2024-03-29T08:29:59.280 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-03-29T08:29:59.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9936f05080 1 Processor -- start 2024-03-29T08:29:59.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9936f05080 1 -- start start 2024-03-29T08:29:59.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9936f05080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654e25540b0 0x5654e25dbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9936f05080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654e25559a0 0x5654e25dc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9936f05080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5654e25df800 0x5654e26745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9936f05080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654e24b61e0 con 0x5654e25540b0 2024-03-29T08:29:59.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9936f05080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654e2451960 con 0x5654e25559a0 2024-03-29T08:29:59.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9936f05080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5654e2451600 con 0x5654e25df800 2024-03-29T08:29:59.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9934200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654e25540b0 0x5654e25dbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9934200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654e25540b0 0x5654e25dbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44884/0 (socket says 172.21.15.46:44884) 2024-03-29T08:29:59.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9934200640 1 -- 172.21.15.46:0/2148147982 learned_addr learned my addr 172.21.15.46:0/2148147982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:59.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9933800640 1 --2- 172.21.15.46:0/2148147982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654e25559a0 0x5654e25dc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9934c00640 1 --2- 172.21.15.46:0/2148147982 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5654e25df800 0x5654e26745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9934200640 1 -- 172.21.15.46:0/2148147982 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5654e25df800 msgr2=0x5654e26745b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9934200640 1 --2- 172.21.15.46:0/2148147982 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5654e25df800 0x5654e26745b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9934200640 1 -- 172.21.15.46:0/2148147982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654e25559a0 msgr2=0x5654e25dc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9934200640 1 --2- 172.21.15.46:0/2148147982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654e25559a0 0x5654e25dc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9934200640 1 -- 172.21.15.46:0/2148147982 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9924005df0 con 0x5654e25540b0 2024-03-29T08:29:59.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9934200640 1 --2- 172.21.15.46:0/2148147982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654e25540b0 0x5654e25dbd10 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f9924018380 tx=0x7f9924003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fbff9418a0b45b6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9932400640 1 -- 172.21.15.46:0/2148147982 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99240aa840 con 0x5654e25540b0 2024-03-29T08:29:59.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9932400640 1 -- 172.21.15.46:0/2148147982 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99240aa9a0 con 0x5654e25540b0 2024-03-29T08:29:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9932400640 1 -- 172.21.15.46:0/2148147982 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99240aad50 con 0x5654e25540b0 2024-03-29T08:29:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9936f05080 1 -- 172.21.15.46:0/2148147982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654e25540b0 msgr2=0x5654e25dbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9936f05080 1 --2- 172.21.15.46:0/2148147982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654e25540b0 0x5654e25dbd10 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f9924018380 tx=0x7f9924003030 comp rx=0 tx=0).stop 2024-03-29T08:29:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9936f05080 1 -- 172.21.15.46:0/2148147982 shutdown_connections 2024-03-29T08:29:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9936f05080 1 --2- 172.21.15.46:0/2148147982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654e25559a0 0x5654e25dc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9936f05080 1 --2- 172.21.15.46:0/2148147982 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5654e25df800 0x5654e26745b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9936f05080 1 --2- 172.21.15.46:0/2148147982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654e25540b0 0x5654e25dbd10 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9936f05080 1 -- 172.21.15.46:0/2148147982 >> 172.21.15.46:0/2148147982 conn(0x5654e2536fa0 msgr2=0x5654e253c030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9936f05080 1 -- 172.21.15.46:0/2148147982 shutdown_connections 2024-03-29T08:29:59.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.369+0000 7f9936f05080 1 -- 172.21.15.46:0/2148147982 wait complete. 2024-03-29T08:29:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9936f05080 1 Processor -- start 2024-03-29T08:29:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9936f05080 1 -- start start 2024-03-29T08:29:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9936f05080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5654e25540b0 0x5654e25c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9936f05080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654e25559a0 0x5654e25dbef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9936f05080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654e25df800 0x5654e25c2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9936f05080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654e2448690 con 0x5654e25df800 2024-03-29T08:29:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9936f05080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654e25c2fd0 con 0x5654e25559a0 2024-03-29T08:29:59.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9936f05080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5654e25c3270 con 0x5654e25540b0 2024-03-29T08:29:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9934c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654e25df800 0x5654e25c2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9934c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654e25df800 0x5654e25c2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44894/0 (socket says 172.21.15.46:44894) 2024-03-29T08:29:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9934c00640 1 -- 172.21.15.46:0/1277739551 learned_addr learned my addr 172.21.15.46:0/1277739551 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9934c00640 1 -- 172.21.15.46:0/1277739551 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5654e25540b0 msgr2=0x5654e25c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9934c00640 1 --2- 172.21.15.46:0/1277739551 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5654e25540b0 0x5654e25c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9934c00640 1 -- 172.21.15.46:0/1277739551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654e25559a0 msgr2=0x5654e25dbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9933800640 1 --2- 172.21.15.46:0/1277739551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654e25559a0 0x5654e25dbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9934c00640 1 --2- 172.21.15.46:0/1277739551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654e25559a0 0x5654e25dbef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9934c00640 1 -- 172.21.15.46:0/1277739551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654e24608c0 con 0x5654e25df800 2024-03-29T08:29:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9933800640 1 --2- 172.21.15.46:0/1277739551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5654e25559a0 0x5654e25dbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9934c00640 1 --2- 172.21.15.46:0/1277739551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654e25df800 0x5654e25c2570 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f992c004230 tx=0x7f992c004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9931000640 1 -- 172.21.15.46:0/1277739551 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c0030d0 con 0x5654e25df800 2024-03-29T08:29:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9931000640 1 -- 172.21.15.46:0/1277739551 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c003250 con 0x5654e25df800 2024-03-29T08:29:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9931000640 1 -- 172.21.15.46:0/1277739551 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c001790 con 0x5654e25df800 2024-03-29T08:29:59.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9936f05080 1 -- 172.21.15.46:0/1277739551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5654e2453d00 con 0x5654e25df800 2024-03-29T08:29:59.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.373+0000 7f9936f05080 1 -- 172.21.15.46:0/1277739551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654e2453760 con 0x5654e25df800 2024-03-29T08:29:59.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9931000640 1 -- 172.21.15.46:0/1277739551 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f992c0018f0 con 0x5654e25df800 2024-03-29T08:29:59.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9931000640 1 --2- 172.21.15.46:0/1277739551 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9914065070 0x7f9914067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9931000640 1 -- 172.21.15.46:0/1277739551 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f992c0822f0 con 0x5654e25df800 2024-03-29T08:29:59.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9936f05080 1 --2- 172.21.15.46:0/1277739551 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5654e25dc430 0x5654e267b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9936f05080 1 -- 172.21.15.46:0/1277739551 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:1 17.19 17:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) v8 -- 0x5654e25c2ab0 con 0x5654e25dc430 2024-03-29T08:29:59.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9933800640 1 --2- 172.21.15.46:0/1277739551 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5654e25dc430 0x5654e267b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9934200640 1 --2- 172.21.15.46:0/1277739551 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9914065070 0x7f9914067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9933800640 1 --2- 172.21.15.46:0/1277739551 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5654e25dc430 0x5654e267b690 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9934200640 1 --2- 172.21.15.46:0/1277739551 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9914065070 0x7f9914067530 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f99240b5000 tx=0x7f9924005ad0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9933800640 1 -- 172.21.15.46:0/1277739551 <== osd.3 v2:172.21.15.46:6808/27426 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f9920002020 con 0x5654e25dc430 2024-03-29T08:29:59.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9936f05080 1 -- 172.21.15.46:0/1277739551 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5654e25dc430 msgr2=0x5654e267b690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9936f05080 1 --2- 172.21.15.46:0/1277739551 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5654e25dc430 0x5654e267b690 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9936f05080 1 -- 172.21.15.46:0/1277739551 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9914065070 msgr2=0x7f9914067530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9936f05080 1 --2- 172.21.15.46:0/1277739551 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9914065070 0x7f9914067530 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f99240b5000 tx=0x7f9924005ad0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9936f05080 1 -- 172.21.15.46:0/1277739551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654e25df800 msgr2=0x5654e25c2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9936f05080 1 --2- 172.21.15.46:0/1277739551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5654e25df800 0x5654e25c2570 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f992c004230 tx=0x7f992c004260 comp rx=0 tx=0).stop 2024-03-29T08:29:59.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9934c00640 1 -- 172.21.15.46:0/1277739551 reap_dead start 2024-03-29T08:29:59.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9936f05080 1 -- 172.21.15.46:0/1277739551 shutdown_connections 2024-03-29T08:29:59.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9936f05080 1 -- 172.21.15.46:0/1277739551 >> 172.21.15.46:0/1277739551 conn(0x5654e2536fa0 msgr2=0x5654e253bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9936f05080 1 -- 172.21.15.46:0/1277739551 shutdown_connections 2024-03-29T08:29:59.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.377+0000 7f9936f05080 1 -- 172.21.15.46:0/1277739551 wait complete. 2024-03-29T08:29:59.383 INFO:teuthology.orchestra.run.smithi046.stdout:mytag123 2024-03-29T08:29:59.383 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-03-29T08:29:59.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1403db4080 1 Processor -- start 2024-03-29T08:29:59.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1403db4080 1 -- start start 2024-03-29T08:29:59.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1403db4080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55886135b440 0x558861496120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1403db4080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5588613780d0 0x558861403f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1403db4080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588613e9d10 0x558861402f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1403db4080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588612db1e0 con 0x5588613e9d10 2024-03-29T08:29:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1403db4080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558861276960 con 0x55886135b440 2024-03-29T08:29:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1403db4080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x558861276600 con 0x5588613780d0 2024-03-29T08:29:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1401a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588613e9d10 0x558861402f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1401a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588613e9d10 0x558861402f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44896/0 (socket says 172.21.15.46:44896) 2024-03-29T08:29:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1401a00640 1 -- 172.21.15.46:0/3060701217 learned_addr learned my addr 172.21.15.46:0/3060701217 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1401000640 1 --2- 172.21.15.46:0/3060701217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55886135b440 0x558861496120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1401a00640 1 -- 172.21.15.46:0/3060701217 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5588613780d0 msgr2=0x558861403f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1401a00640 1 --2- 172.21.15.46:0/3060701217 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5588613780d0 0x558861403f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1401a00640 1 -- 172.21.15.46:0/3060701217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55886135b440 msgr2=0x558861496120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1401a00640 1 --2- 172.21.15.46:0/3060701217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55886135b440 0x558861496120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1401a00640 1 -- 172.21.15.46:0/3060701217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc00a100 con 0x5588613e9d10 2024-03-29T08:29:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1401000640 1 --2- 172.21.15.46:0/3060701217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55886135b440 0x558861496120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f1401a00640 1 --2- 172.21.15.46:0/3060701217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588613e9d10 0x558861402f90 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00a3f0 tx=0x7f13fc0033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95c9a6ed7cef0f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f13faa00640 1 -- 172.21.15.46:0/3060701217 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0ab790 con 0x5588613e9d10 2024-03-29T08:29:59.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f13faa00640 1 -- 172.21.15.46:0/3060701217 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13fc0ab8f0 con 0x5588613e9d10 2024-03-29T08:29:59.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.473+0000 7f13faa00640 1 -- 172.21.15.46:0/3060701217 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0abc60 con 0x5588613e9d10 2024-03-29T08:29:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 -- 172.21.15.46:0/3060701217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588613e9d10 msgr2=0x558861402f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 --2- 172.21.15.46:0/3060701217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588613e9d10 0x558861402f90 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00a3f0 tx=0x7f13fc0033b0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 -- 172.21.15.46:0/3060701217 shutdown_connections 2024-03-29T08:29:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 --2- 172.21.15.46:0/3060701217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588613e9d10 0x558861402f90 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 --2- 172.21.15.46:0/3060701217 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5588613780d0 0x558861403f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 --2- 172.21.15.46:0/3060701217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55886135b440 0x558861496120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 -- 172.21.15.46:0/3060701217 >> 172.21.15.46:0/3060701217 conn(0x558861359910 msgr2=0x55886135cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 -- 172.21.15.46:0/3060701217 shutdown_connections 2024-03-29T08:29:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 -- 172.21.15.46:0/3060701217 wait complete. 2024-03-29T08:29:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 Processor -- start 2024-03-29T08:29:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 -- start start 2024-03-29T08:29:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55886135b440 0x55886149a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588613780d0 0x55886135e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588613e9d10 0x55886135eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55886126d690 con 0x5588613780d0 2024-03-29T08:29:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588614021f0 con 0x5588613e9d10 2024-03-29T08:29:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x558861402490 con 0x55886135b440 2024-03-29T08:29:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f13fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588613780d0 0x55886135e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1401000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55886135b440 0x55886149a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f13fbe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588613780d0 0x55886135e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44900/0 (socket says 172.21.15.46:44900) 2024-03-29T08:29:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f13fbe00640 1 -- 172.21.15.46:0/3430566246 learned_addr learned my addr 172.21.15.46:0/3430566246 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1401000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55886135b440 0x55886149a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:60212/0 (socket says 172.21.15.46:60212) 2024-03-29T08:29:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1401a00640 1 --2- 172.21.15.46:0/3430566246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588613e9d10 0x55886135eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f13fbe00640 1 -- 172.21.15.46:0/3430566246 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55886135b440 msgr2=0x55886149a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f13fbe00640 1 --2- 172.21.15.46:0/3430566246 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x55886135b440 0x55886149a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f13fbe00640 1 -- 172.21.15.46:0/3430566246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588613e9d10 msgr2=0x55886135eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f13fbe00640 1 --2- 172.21.15.46:0/3430566246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588613e9d10 0x55886135eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f13fbe00640 1 -- 172.21.15.46:0/3430566246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588612858c0 con 0x5588613780d0 2024-03-29T08:29:59.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f13fbe00640 1 --2- 172.21.15.46:0/3430566246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588613780d0 0x55886135e5f0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f13ec00bd60 tx=0x7f13ec009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f13f9600640 1 -- 172.21.15.46:0/3430566246 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec009e00 con 0x5588613780d0 2024-03-29T08:29:59.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f13f9600640 1 -- 172.21.15.46:0/3430566246 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ec00f410 con 0x5588613780d0 2024-03-29T08:29:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f13f9600640 1 -- 172.21.15.46:0/3430566246 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec00f7c0 con 0x5588613780d0 2024-03-29T08:29:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 -- 172.21.15.46:0/3430566246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558861278d00 con 0x5588613780d0 2024-03-29T08:29:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.477+0000 7f1403db4080 1 -- 172.21.15.46:0/3430566246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558861278760 con 0x5588613780d0 2024-03-29T08:29:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.481+0000 7f13f9600640 1 -- 172.21.15.46:0/3430566246 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f13ec00f920 con 0x5588613780d0 2024-03-29T08:29:59.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.481+0000 7f13f9600640 1 --2- 172.21.15.46:0/3430566246 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f13e00650a0 0x7f13e0067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.481+0000 7f13f9600640 1 -- 172.21.15.46:0/3430566246 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f13ec0828f0 con 0x5588613780d0 2024-03-29T08:29:59.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.481+0000 7f1403db4080 1 --2- 172.21.15.46:0/3430566246 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x55886149a990 0x5588614013f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.481+0000 7f1401000640 1 --2- 172.21.15.46:0/3430566246 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f13e00650a0 0x7f13e0067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.481+0000 7f1403db4080 1 -- 172.21.15.46:0/3430566246 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:1 17.5 17:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e124) v8 -- 0x5588614999b0 con 0x55886149a990 2024-03-29T08:29:59.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.481+0000 7f1401a00640 1 --2- 172.21.15.46:0/3430566246 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x55886149a990 0x5588614013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.481+0000 7f1401000640 1 --2- 172.21.15.46:0/3430566246 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f13e00650a0 0x7f13e0067560 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f13f4004180 tx=0x7f13f4001990 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.481+0000 7f1401a00640 1 --2- 172.21.15.46:0/3430566246 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x55886149a990 0x5588614013f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.481+0000 7f1401a00640 1 -- 172.21.15.46:0/3430566246 <== osd.0 v2:172.21.15.46:6800/27424 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f13fc0aba60 con 0x55886149a990 2024-03-29T08:29:59.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.481+0000 7f1403db4080 1 -- 172.21.15.46:0/3430566246 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x55886149a990 msgr2=0x5588614013f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.481+0000 7f1403db4080 1 --2- 172.21.15.46:0/3430566246 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x55886149a990 0x5588614013f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.485+0000 7f1403db4080 1 -- 172.21.15.46:0/3430566246 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f13e00650a0 msgr2=0x7f13e0067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.485+0000 7f1403db4080 1 --2- 172.21.15.46:0/3430566246 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f13e00650a0 0x7f13e0067560 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f13f4004180 tx=0x7f13f4001990 comp rx=0 tx=0).stop 2024-03-29T08:29:59.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.485+0000 7f1403db4080 1 -- 172.21.15.46:0/3430566246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588613780d0 msgr2=0x55886135e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.485+0000 7f1403db4080 1 --2- 172.21.15.46:0/3430566246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588613780d0 0x55886135e5f0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f13ec00bd60 tx=0x7f13ec009540 comp rx=0 tx=0).stop 2024-03-29T08:29:59.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.485+0000 7f1401a00640 1 -- 172.21.15.46:0/3430566246 reap_dead start 2024-03-29T08:29:59.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.485+0000 7f1403db4080 1 -- 172.21.15.46:0/3430566246 shutdown_connections 2024-03-29T08:29:59.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.485+0000 7f1403db4080 1 -- 172.21.15.46:0/3430566246 >> 172.21.15.46:0/3430566246 conn(0x558861359910 msgr2=0x558861405c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.485+0000 7f1403db4080 1 -- 172.21.15.46:0/3430566246 shutdown_connections 2024-03-29T08:29:59.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.485+0000 7f1403db4080 1 -- 172.21.15.46:0/3430566246 wait complete. 2024-03-29T08:29:59.488 INFO:teuthology.orchestra.run.smithi046.stdout:mytag123 2024-03-29T08:29:59.488 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-03-29T08:29:59.488 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-03-29T08:29:59.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.629+0000 7fcaf5600640 1 Processor -- start 2024-03-29T08:29:59.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.629+0000 7fcaf5600640 1 -- start start 2024-03-29T08:29:59.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.629+0000 7fcaf5600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcaf00cbea0 0x7fcaf00ae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.629+0000 7fcaf5600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf015d260 con 0x7fcaf0142a50 2024-03-29T08:29:59.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.629+0000 7fcaf5600640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf00aefd0 con 0x7fcaf015df90 2024-03-29T08:29:59.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.629+0000 7fcaf5600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf015d4b0 con 0x7fcaf00cbea0 2024-03-29T08:29:59.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.629+0000 7fcaeea00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcaf0142a50 0x7fcaf00cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46098/0 (socket says 172.21.15.46:46098) 2024-03-29T08:29:59.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.629+0000 7fcaeea00640 1 -- 172.21.15.46:0/2003944343 learned_addr learned my addr 172.21.15.46:0/2003944343 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:29:59.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaef400640 1 --2- 172.21.15.46:0/2003944343 >> v2:172.21.15.73:3300/0 conn(0x7fcaf00cbea0 0x7fcaf00ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaee000640 1 -- 172.21.15.46:0/2003944343 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620261543 0 0) 0x7fcaf015d260 con 0x7fcaf0142a50 2024-03-29T08:29:59.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaee000640 1 -- 172.21.15.46:0/2003944343 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcad4003530 con 0x7fcaf0142a50 2024-03-29T08:29:59.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaee000640 1 -- 172.21.15.46:0/2003944343 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4012059915 0 0) 0x7fcaf00aefd0 con 0x7fcaf015df90 2024-03-29T08:29:59.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaee000640 1 -- 172.21.15.46:0/2003944343 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaf015d260 con 0x7fcaf015df90 2024-03-29T08:29:59.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaee000640 1 -- 172.21.15.46:0/2003944343 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 489353242 0 0) 0x7fcad4003530 con 0x7fcaf0142a50 2024-03-29T08:29:59.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaee000640 1 -- 172.21.15.46:0/2003944343 >> v2:172.21.15.73:3300/0 conn(0x7fcaf00cbea0 msgr2=0x7fcaf00ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaee000640 1 --2- 172.21.15.46:0/2003944343 >> v2:172.21.15.73:3300/0 conn(0x7fcaf00cbea0 0x7fcaf00ae990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaee000640 1 -- 172.21.15.46:0/2003944343 >> v1:172.21.15.73:6789/0 conn(0x7fcaf015df90 legacy=0x7fcaf00ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaee000640 1 -- 172.21.15.46:0/2003944343 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf015d790 con 0x7fcaf0142a50 2024-03-29T08:29:59.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaee000640 1 -- 172.21.15.46:0/2003944343 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fcae0001680 con 0x7fcaf0142a50 2024-03-29T08:29:59.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaee000640 1 -- 172.21.15.46:0/2003944343 >> v1:172.21.15.46:6789/0 conn(0x7fcaf0142a50 legacy=0x7fcaf00cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaee000640 1 --2- 172.21.15.46:0/2003944343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4003b10 0x7fcad4003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaee000640 1 --2- 172.21.15.46:0/2003944343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcad40047d0 0x7fcad4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaee000640 1 --2- 172.21.15.46:0/2003944343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcad4005190 0x7fcad400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaee000640 1 -- 172.21.15.46:0/2003944343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf015d4b0 con 0x7fcad4003b10 2024-03-29T08:29:59.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaee000640 1 -- 172.21.15.46:0/2003944343 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad400c0c0 con 0x7fcad4005190 2024-03-29T08:29:59.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaee000640 1 -- 172.21.15.46:0/2003944343 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad400c280 con 0x7fcad40047d0 2024-03-29T08:29:59.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaeea00640 1 --2- 172.21.15.46:0/2003944343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4003b10 0x7fcad4003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcae6000640 1 --2- 172.21.15.46:0/2003944343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcad40047d0 0x7fcad4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaef400640 1 --2- 172.21.15.46:0/2003944343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcad4005190 0x7fcad400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaeea00640 1 -- 172.21.15.46:0/2003944343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcad40047d0 msgr2=0x7fcad4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaeea00640 1 --2- 172.21.15.46:0/2003944343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcad40047d0 0x7fcad4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaeea00640 1 -- 172.21.15.46:0/2003944343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcad4005190 msgr2=0x7fcad400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaeea00640 1 --2- 172.21.15.46:0/2003944343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcad4005190 0x7fcad400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaeea00640 1 -- 172.21.15.46:0/2003944343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad4003530 con 0x7fcad4003b10 2024-03-29T08:29:59.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcae6000640 1 --2- 172.21.15.46:0/2003944343 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcad40047d0 0x7fcad4004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:59.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.633+0000 7fcaef400640 1 -- 172.21.15.46:0/2003944343 reap_dead start 2024-03-29T08:29:59.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaeea00640 1 --2- 172.21.15.46:0/2003944343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4003b10 0x7fcad4003f10 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fcae000c160 tx=0x7fcae001cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d3060935180b51 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaee000640 1 -- 172.21.15.46:0/2003944343 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae00058e0 con 0x7fcad4003b10 2024-03-29T08:29:59.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaee000640 1 -- 172.21.15.46:0/2003944343 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae0005a60 con 0x7fcad4003b10 2024-03-29T08:29:59.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaee000640 1 -- 172.21.15.46:0/2003944343 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0005e50 con 0x7fcad4003b10 2024-03-29T08:29:59.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaf5600640 1 -- 172.21.15.46:0/2003944343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4003b10 msgr2=0x7fcad4003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaf5600640 1 --2- 172.21.15.46:0/2003944343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4003b10 0x7fcad4003f10 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fcae000c160 tx=0x7fcae001cbe0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaf5600640 1 -- 172.21.15.46:0/2003944343 shutdown_connections 2024-03-29T08:29:59.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaf5600640 1 --2- 172.21.15.46:0/2003944343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4003b10 0x7fcad4003f10 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaf5600640 1 -- 172.21.15.46:0/2003944343 >> 172.21.15.46:0/2003944343 conn(0x7fcaf006a390 msgr2=0x7fcaf00cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaf5600640 1 -- 172.21.15.46:0/2003944343 shutdown_connections 2024-03-29T08:29:59.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaf5600640 1 -- 172.21.15.46:0/2003944343 wait complete. 2024-03-29T08:29:59.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaf5600640 1 Processor -- start 2024-03-29T08:29:59.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaf5600640 1 -- start start 2024-03-29T08:29:59.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaf5600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4003b10 0x7fcaf01397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaf5600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcaf0159200 0x7fcaf013f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaf5600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcaf013bd20 0x7fcaf00ac900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaf5600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf013dff0 con 0x7fcad4003b10 2024-03-29T08:29:59.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaf5600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf013c5c0 con 0x7fcaf013bd20 2024-03-29T08:29:59.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaf5600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf015b9d0 con 0x7fcaf0159200 2024-03-29T08:29:59.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaeea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4003b10 0x7fcaf01397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaeea00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4003b10 0x7fcaf01397b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44920/0 (socket says 172.21.15.46:44920) 2024-03-29T08:29:59.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.637+0000 7fcaeea00640 1 -- 172.21.15.46:0/3342227397 learned_addr learned my addr 172.21.15.46:0/3342227397 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:59.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.641+0000 7fcae6000640 1 --2- 172.21.15.46:0/3342227397 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcaf0159200 0x7fcaf013f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.641+0000 7fcaef400640 1 --2- 172.21.15.46:0/3342227397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcaf013bd20 0x7fcaf00ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.641+0000 7fcaeea00640 1 -- 172.21.15.46:0/3342227397 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcaf0159200 msgr2=0x7fcaf013f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.641+0000 7fcaeea00640 1 --2- 172.21.15.46:0/3342227397 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcaf0159200 0x7fcaf013f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.641+0000 7fcaeea00640 1 -- 172.21.15.46:0/3342227397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcaf013bd20 msgr2=0x7fcaf00ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.641+0000 7fcaeea00640 1 --2- 172.21.15.46:0/3342227397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcaf013bd20 0x7fcaf00ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.641+0000 7fcaeea00640 1 -- 172.21.15.46:0/3342227397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf013f8f0 con 0x7fcad4003b10 2024-03-29T08:29:59.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.641+0000 7fcaeea00640 1 --2- 172.21.15.46:0/3342227397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4003b10 0x7fcaf01397b0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fcae001ca10 tx=0x7fcad4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.641+0000 7fcae7400640 1 -- 172.21.15.46:0/3342227397 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0001680 con 0x7fcad4003b10 2024-03-29T08:29:59.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.641+0000 7fcae7400640 1 -- 172.21.15.46:0/3342227397 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae00047f0 con 0x7fcad4003b10 2024-03-29T08:29:59.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.641+0000 7fcae7400640 1 -- 172.21.15.46:0/3342227397 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0004b60 con 0x7fcad4003b10 2024-03-29T08:29:59.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.641+0000 7fcaf5600640 1 -- 172.21.15.46:0/3342227397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf015b0a0 con 0x7fcad4003b10 2024-03-29T08:29:59.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.641+0000 7fcaf5600640 1 -- 172.21.15.46:0/3342227397 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf015b660 con 0x7fcad4003b10 2024-03-29T08:29:59.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.641+0000 7fcae7400640 1 -- 172.21.15.46:0/3342227397 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcae0004cc0 con 0x7fcad4003b10 2024-03-29T08:29:59.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcae7400640 1 --2- 172.21.15.46:0/3342227397 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcacc066690 0x7fcacc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcae7400640 1 -- 172.21.15.46:0/3342227397 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fcae011d6a0 con 0x7fcad4003b10 2024-03-29T08:29:59.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaf8991000 1 Processor -- start 2024-03-29T08:29:59.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcae6000640 1 --2- 172.21.15.46:0/3342227397 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcacc066690 0x7fcacc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaf8991000 1 -- start start 2024-03-29T08:29:59.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcae6000640 1 --2- 172.21.15.46:0/3342227397 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcacc066690 0x7fcacc068b50 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcaf015ae90 tx=0x7fcad8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaf8991000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269bdea70 0x560269bdee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaf8991000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269bdf390 0x560269b924c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaf8991000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269b92a00 0x560269b92de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaf8991000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x560269a5d630 con 0x560269b92a00 2024-03-29T08:29:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaf8991000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602699eabd0 con 0x560269bdf390 2024-03-29T08:29:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaf8991000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602699e81a0 con 0x560269bdea70 2024-03-29T08:29:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269b92a00 0x560269b92de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269b92a00 0x560269b92de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44926/0 (socket says 172.21.15.46:44926) 2024-03-29T08:29:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaef400640 1 -- 172.21.15.46:0/2359295642 learned_addr learned my addr 172.21.15.46:0/2359295642 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaeea00640 1 --2- 172.21.15.46:0/2359295642 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269bdea70 0x560269bdee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcae6000640 1 --2- 172.21.15.46:0/2359295642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269bdf390 0x560269b924c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaef400640 1 -- 172.21.15.46:0/2359295642 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269bdea70 msgr2=0x560269bdee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaef400640 1 --2- 172.21.15.46:0/2359295642 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269bdea70 0x560269bdee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaef400640 1 -- 172.21.15.46:0/2359295642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269bdf390 msgr2=0x560269b924c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaef400640 1 --2- 172.21.15.46:0/2359295642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269bdf390 0x560269b924c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaef400640 1 -- 172.21.15.46:0/2359295642 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602696dfe20 con 0x560269b92a00 2024-03-29T08:29:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaeea00640 1 --2- 172.21.15.46:0/2359295642 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269bdea70 0x560269bdee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaef400640 1 --2- 172.21.15.46:0/2359295642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269b92a00 0x560269b92de0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fcaf006e5a0 tx=0x7fcae8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e00fc13cd47cc45 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcadf400640 1 -- 172.21.15.46:0/2359295642 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae800e030 con 0x560269b92a00 2024-03-29T08:29:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcadf400640 1 -- 172.21.15.46:0/2359295642 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae800f040 con 0x560269b92a00 2024-03-29T08:29:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcadf400640 1 -- 172.21.15.46:0/2359295642 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8017670 con 0x560269b92a00 2024-03-29T08:29:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaf8991000 1 -- 172.21.15.46:0/2359295642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269b92a00 msgr2=0x560269b92de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaf8991000 1 --2- 172.21.15.46:0/2359295642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269b92a00 0x560269b92de0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fcaf006e5a0 tx=0x7fcae8009750 comp rx=0 tx=0).stop 2024-03-29T08:29:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaf8991000 1 -- 172.21.15.46:0/2359295642 shutdown_connections 2024-03-29T08:29:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaf8991000 1 --2- 172.21.15.46:0/2359295642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269bdf390 0x560269b924c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaf8991000 1 --2- 172.21.15.46:0/2359295642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269b92a00 0x560269b92de0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaf8991000 1 --2- 172.21.15.46:0/2359295642 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269bdea70 0x560269bdee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaf8991000 1 -- 172.21.15.46:0/2359295642 >> 172.21.15.46:0/2359295642 conn(0x560269993160 msgr2=0x560269bc1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaf8991000 1 -- 172.21.15.46:0/2359295642 shutdown_connections 2024-03-29T08:29:59.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.645+0000 7fcaf8991000 1 -- 172.21.15.46:0/2359295642 wait complete. 2024-03-29T08:29:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 1 -- start start 2024-03-29T08:29:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 1 Processor -- start 2024-03-29T08:29:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269b92a00 0x560269b9b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269bdea70 0x560269bc2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdf390 0x560269bc2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602699e3a90 con 0x560269bdf390 2024-03-29T08:29:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560269982d40 con 0x560269b92a00 2024-03-29T08:29:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602699a0f30 con 0x560269bdea70 2024-03-29T08:29:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 10 client.0 _ll_register_callbacks cb 0x560269a45810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:29:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdf390 0x560269bc2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdf390 0x560269bc2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44942/0 (socket says 172.21.15.46:44942) 2024-03-29T08:29:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaef400640 1 -- 172.21.15.46:0/2562196748 learned_addr learned my addr 172.21.15.46:0/2562196748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaeea00640 1 --2- 172.21.15.46:0/2562196748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269b92a00 0x560269b9b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcae6000640 1 --2- 172.21.15.46:0/2562196748 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269bdea70 0x560269bc2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaef400640 1 -- 172.21.15.46:0/2562196748 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269bdea70 msgr2=0x560269bc2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaef400640 1 --2- 172.21.15.46:0/2562196748 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269bdea70 0x560269bc2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaef400640 1 -- 172.21.15.46:0/2562196748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269b92a00 msgr2=0x560269b9b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaef400640 1 --2- 172.21.15.46:0/2562196748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269b92a00 0x560269b9b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaef400640 1 -- 172.21.15.46:0/2562196748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae8009e30 con 0x560269bdf390 2024-03-29T08:29:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaeea00640 1 --2- 172.21.15.46:0/2562196748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269b92a00 0x560269b9b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:29:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaef400640 1 --2- 172.21.15.46:0/2562196748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdf390 0x560269bc2f90 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fcae8000f90 tx=0x7fcae8004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19dad48df4067738 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcad3e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:29:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcad3e00640 1 -- 172.21.15.46:0/2562196748 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae800e070 con 0x560269bdf390 2024-03-29T08:29:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcad3e00640 1 -- 172.21.15.46:0/2562196748 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fcae8029660 con 0x560269bdf390 2024-03-29T08:29:59.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 1 -- 172.21.15.46:0/2562196748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602699e23e0 con 0x560269bdf390 2024-03-29T08:29:59.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcad3e00640 1 -- 172.21.15.46:0/2562196748 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcae8029820 con 0x560269bdf390 2024-03-29T08:29:59.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 10 client.9639 fetch_fsmap learned FSMap version 100 2024-03-29T08:29:59.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 1 -- 172.21.15.46:0/2562196748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x5602699a8aa0 con 0x560269bdf390 2024-03-29T08:29:59.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcad3e00640 1 -- 172.21.15.46:0/2562196748 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5602699e23e0 con 0x560269bdf390 2024-03-29T08:29:59.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcad3e00640 1 -- 172.21.15.46:0/2562196748 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 100) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcae8029980 con 0x560269bdf390 2024-03-29T08:29:59.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 10 client.9639 fetch_fsmap finished waiting for FSMap version 100 2024-03-29T08:29:59.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 10 client.9639 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-03-29T08:29:59.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 20 client.9639 populate_metadata read hostname 'smithi046' 2024-03-29T08:29:59.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 1 --2- 172.21.15.46:0/2562196748 >> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] conn(0x560269be5e00 0x560269be61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 4 client.9639 mds_command: new command op to 9441 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:29:59.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaf8991000 1 -- 172.21.15.46:0/2562196748 --> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602699b3d00 con 0x560269be5e00 2024-03-29T08:29:59.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.649+0000 7fcaeea00640 1 --2- 172.21.15.46:0/2562196748 >> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] conn(0x560269be5e00 0x560269be61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.653+0000 7fcaeea00640 1 --2- 172.21.15.46:0/2562196748 >> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] conn(0x560269be5e00 0x560269be61e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=118af0e8ab70f05a server_cookie=a6ae4a455e657f1b in_seq=0 out_seq=0 2024-03-29T08:29:59.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.653+0000 7fcad3e00640 10 client.9639 ms_handle_connect on v2:172.21.15.46:6834/276185207 2024-03-29T08:29:59.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.653+0000 7fcad3e00640 1 -- 172.21.15.46:0/2562196748 <== mds.0 v2:172.21.15.46:6834/276185207 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5602699b3d00 con 0x560269be5e00 2024-03-29T08:29:59.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.653+0000 7fcad3e00640 10 client.9639 handle_command_reply: tid=0 2024-03-29T08:29:59.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.653+0000 7fcaf8991000 1 client.9639 shutdown 2024-03-29T08:29:59.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.653+0000 7fcaf8991000 1 -- 172.21.15.46:0/2562196748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdf390 msgr2=0x560269bc2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.653+0000 7fcaf8991000 1 --2- 172.21.15.46:0/2562196748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdf390 0x560269bc2f90 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fcae8000f90 tx=0x7fcae8004a50 comp rx=0 tx=0).stop 2024-03-29T08:29:59.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.653+0000 7fcaf8991000 1 -- 172.21.15.46:0/2562196748 shutdown_connections 2024-03-29T08:29:59.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.653+0000 7fcaf8991000 1 --2- 172.21.15.46:0/2562196748 >> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] conn(0x560269be5e00 0x560269be61e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.653+0000 7fcaf8991000 1 --2- 172.21.15.46:0/2562196748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdf390 0x560269bc2f90 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.653+0000 7fcaf8991000 1 --2- 172.21.15.46:0/2562196748 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269bdea70 0x560269bc2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.653+0000 7fcad3e00640 0 client.9639 ms_handle_reset on v2:172.21.15.46:6834/276185207 2024-03-29T08:29:59.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.653+0000 7fcaf8991000 1 --2- 172.21.15.46:0/2562196748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269b92a00 0x560269b9b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.653+0000 7fcaf8991000 1 -- 172.21.15.46:0/2562196748 >> 172.21.15.46:0/2562196748 conn(0x560269993160 msgr2=0x560269b91bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.653+0000 7fcaf8991000 1 -- 172.21.15.46:0/2562196748 shutdown_connections 2024-03-29T08:29:59.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.653+0000 7fcaf8991000 1 -- 172.21.15.46:0/2562196748 wait complete. 2024-03-29T08:29:59.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.653+0000 7fcaf8991000 20 client.9639 trim_cache size 0 max 16384 2024-03-29T08:29:59.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcaf8991000 1 Processor -- start 2024-03-29T08:29:59.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcaf8991000 1 -- start start 2024-03-29T08:29:59.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcaf8991000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269b92a00 0x560269b91c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcaf8991000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269bdea70 0x560269b92170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcaf8991000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdf390 0x560269b92de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcaf8991000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602696a0aa0 con 0x560269bdf390 2024-03-29T08:29:59.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcaf8991000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560269aa0680 con 0x560269b92a00 2024-03-29T08:29:59.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcaf8991000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602699e65f0 con 0x560269bdea70 2024-03-29T08:29:59.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcaef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdf390 0x560269b92de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcaef400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdf390 0x560269b92de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44946/0 (socket says 172.21.15.46:44946) 2024-03-29T08:29:59.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcaef400640 1 -- 172.21.15.46:0/3792401210 learned_addr learned my addr 172.21.15.46:0/3792401210 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:59.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcaeea00640 1 --2- 172.21.15.46:0/3792401210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269b92a00 0x560269b91c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcae6000640 1 --2- 172.21.15.46:0/3792401210 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269bdea70 0x560269b92170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcaef400640 1 -- 172.21.15.46:0/3792401210 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269bdea70 msgr2=0x560269b92170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcaef400640 1 --2- 172.21.15.46:0/3792401210 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269bdea70 0x560269b92170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcaef400640 1 -- 172.21.15.46:0/3792401210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269b92a00 msgr2=0x560269b91c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcaef400640 1 --2- 172.21.15.46:0/3792401210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269b92a00 0x560269b91c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcaef400640 1 -- 172.21.15.46:0/3792401210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56026998edf0 con 0x560269bdf390 2024-03-29T08:29:59.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.665+0000 7fcae6000640 1 --2- 172.21.15.46:0/3792401210 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269bdea70 0x560269b92170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:29:59.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaef400640 1 --2- 172.21.15.46:0/3792401210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdf390 0x560269b92de0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fcae8003ab0 tx=0x7fcae8021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f7a9415735e424 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcadfe00640 1 -- 172.21.15.46:0/3792401210 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae80070c0 con 0x560269bdf390 2024-03-29T08:29:59.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcadfe00640 1 -- 172.21.15.46:0/3792401210 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae800f040 con 0x560269bdf390 2024-03-29T08:29:59.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcadfe00640 1 -- 172.21.15.46:0/3792401210 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae8020cc0 con 0x560269bdf390 2024-03-29T08:29:59.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 1 -- 172.21.15.46:0/3792401210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdf390 msgr2=0x560269b92de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 1 --2- 172.21.15.46:0/3792401210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdf390 0x560269b92de0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fcae8003ab0 tx=0x7fcae8021000 comp rx=0 tx=0).stop 2024-03-29T08:29:59.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 1 -- 172.21.15.46:0/3792401210 shutdown_connections 2024-03-29T08:29:59.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 1 --2- 172.21.15.46:0/3792401210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdf390 0x560269b92de0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 1 --2- 172.21.15.46:0/3792401210 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269bdea70 0x560269b92170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 1 --2- 172.21.15.46:0/3792401210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269b92a00 0x560269b91c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 1 -- 172.21.15.46:0/3792401210 >> 172.21.15.46:0/3792401210 conn(0x560269993160 msgr2=0x560269b9a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 1 -- 172.21.15.46:0/3792401210 shutdown_connections 2024-03-29T08:29:59.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 1 -- 172.21.15.46:0/3792401210 wait complete. 2024-03-29T08:29:59.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 1 -- start start 2024-03-29T08:29:59.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 1 Processor -- start 2024-03-29T08:29:59.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269b92a00 0x560269b95ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdea70 0x560269b960e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269bdf390 0x560269c3ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602699ebe60 con 0x560269bdea70 2024-03-29T08:29:59.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602699a4c60 con 0x560269bdf390 2024-03-29T08:29:59.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x560269b8c930 con 0x560269b92a00 2024-03-29T08:29:59.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaf8991000 10 client.0 _ll_register_callbacks cb 0x7fcae8009ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:29:59.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcae6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdea70 0x560269b960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcae6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdea70 0x560269b960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44948/0 (socket says 172.21.15.46:44948) 2024-03-29T08:29:59.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcae6000640 1 -- 172.21.15.46:0/1805612916 learned_addr learned my addr 172.21.15.46:0/1805612916 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:59.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaeea00640 1 --2- 172.21.15.46:0/1805612916 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269b92a00 0x560269b95ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcaef400640 1 --2- 172.21.15.46:0/1805612916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269bdf390 0x560269c3ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcae6000640 1 -- 172.21.15.46:0/1805612916 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269b92a00 msgr2=0x560269b95ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcae6000640 1 --2- 172.21.15.46:0/1805612916 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269b92a00 0x560269b95ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcae6000640 1 -- 172.21.15.46:0/1805612916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269bdf390 msgr2=0x560269c3ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcae6000640 1 --2- 172.21.15.46:0/1805612916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269bdf390 0x560269c3ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcae6000640 1 -- 172.21.15.46:0/1805612916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602699a8aa0 con 0x560269bdea70 2024-03-29T08:29:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcae6000640 1 --2- 172.21.15.46:0/1805612916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdea70 0x560269b960e0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fcad80156e0 tx=0x7fcad8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9a9d7541460d21 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.669+0000 7fcad3e00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:29:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.673+0000 7fcad3e00640 1 -- 172.21.15.46:0/1805612916 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad8023620 con 0x560269bdea70 2024-03-29T08:29:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.673+0000 7fcad3e00640 1 -- 172.21.15.46:0/1805612916 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fcad801a040 con 0x560269bdea70 2024-03-29T08:29:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.673+0000 7fcaf8991000 1 -- 172.21.15.46:0/1805612916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602699a0970 con 0x560269bdea70 2024-03-29T08:29:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.673+0000 7fcad3e00640 1 -- 172.21.15.46:0/1805612916 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcad8023780 con 0x560269bdea70 2024-03-29T08:29:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.673+0000 7fcaf8991000 10 client.9645 fetch_fsmap learned FSMap version 100 2024-03-29T08:29:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.673+0000 7fcaf8991000 1 -- 172.21.15.46:0/1805612916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x560269982960 con 0x560269bdea70 2024-03-29T08:29:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.673+0000 7fcad3e00640 1 -- 172.21.15.46:0/1805612916 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5602699a0970 con 0x560269bdea70 2024-03-29T08:29:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.673+0000 7fcad3e00640 1 -- 172.21.15.46:0/1805612916 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 100) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcad8020040 con 0x560269bdea70 2024-03-29T08:29:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.673+0000 7fcaf8991000 10 client.9645 fetch_fsmap finished waiting for FSMap version 100 2024-03-29T08:29:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.673+0000 7fcaf8991000 10 client.9645 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-03-29T08:29:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.673+0000 7fcaf8991000 20 client.9645 populate_metadata read hostname 'smithi046' 2024-03-29T08:29:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.673+0000 7fcaf8991000 1 --2- 172.21.15.46:0/1805612916 >> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] conn(0x560269c3b1d0 0x560269c3b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.673+0000 7fcaf8991000 4 client.9645 mds_command: new command op to 9441 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-03-29T08:29:59.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.673+0000 7fcaf8991000 1 -- 172.21.15.46:0/1805612916 --> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560269a12a20 con 0x560269c3b1d0 2024-03-29T08:29:59.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.673+0000 7fcaeea00640 1 --2- 172.21.15.46:0/1805612916 >> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] conn(0x560269c3b1d0 0x560269c3b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.673+0000 7fcaeea00640 1 --2- 172.21.15.46:0/1805612916 >> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] conn(0x560269c3b1d0 0x560269c3b5e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ca8f5fb3eb72c40 server_cookie=9bc6e9d6d12b087b in_seq=0 out_seq=0 2024-03-29T08:29:59.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.673+0000 7fcad3e00640 10 client.9645 ms_handle_connect on v2:172.21.15.46:6834/276185207 2024-03-29T08:29:59.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcad3e00640 1 -- 172.21.15.46:0/1805612916 <== mds.0 v2:172.21.15.46:6834/276185207 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560269a12a20 con 0x560269c3b1d0 2024-03-29T08:29:59.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcad3e00640 10 client.9645 handle_command_reply: tid=0 2024-03-29T08:29:59.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf8991000 1 client.9645 shutdown 2024-03-29T08:29:59.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf8991000 1 -- 172.21.15.46:0/1805612916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdea70 msgr2=0x560269b960e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf8991000 1 --2- 172.21.15.46:0/1805612916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdea70 0x560269b960e0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fcad80156e0 tx=0x7fcad8000bd0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf8991000 1 -- 172.21.15.46:0/1805612916 shutdown_connections 2024-03-29T08:29:59.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf8991000 1 --2- 172.21.15.46:0/1805612916 >> [v2:172.21.15.46:6834/276185207,v1:172.21.15.46:6835/276185207] conn(0x560269c3b1d0 0x560269c3b5e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf8991000 1 --2- 172.21.15.46:0/1805612916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560269bdf390 0x560269c3ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf8991000 1 --2- 172.21.15.46:0/1805612916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560269bdea70 0x560269b960e0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf8991000 1 --2- 172.21.15.46:0/1805612916 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x560269b92a00 0x560269b95ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcad3e00640 0 client.9645 ms_handle_reset on v2:172.21.15.46:6834/276185207 2024-03-29T08:29:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf8991000 1 -- 172.21.15.46:0/1805612916 >> 172.21.15.46:0/1805612916 conn(0x560269993160 msgr2=0x560269b90c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf8991000 1 -- 172.21.15.46:0/1805612916 shutdown_connections 2024-03-29T08:29:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf8991000 1 -- 172.21.15.46:0/1805612916 wait complete. 2024-03-29T08:29:59.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf8991000 20 client.9645 trim_cache size 0 max 16384 2024-03-29T08:29:59.691 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:29:59.691 INFO:teuthology.orchestra.run.smithi046.stdout: "message": "", 2024-03-29T08:29:59.691 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0 2024-03-29T08:29:59.691 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:29:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf5600640 1 -- 172.21.15.46:0/3342227397 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcacc066690 msgr2=0x7fcacc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf5600640 1 --2- 172.21.15.46:0/3342227397 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcacc066690 0x7fcacc068b50 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcaf015ae90 tx=0x7fcad8009040 comp rx=0 tx=0).stop 2024-03-29T08:29:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf5600640 1 -- 172.21.15.46:0/3342227397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4003b10 msgr2=0x7fcaf01397b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf5600640 1 --2- 172.21.15.46:0/3342227397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4003b10 0x7fcaf01397b0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fcae001ca10 tx=0x7fcad4005800 comp rx=0 tx=0).stop 2024-03-29T08:29:59.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf5600640 1 -- 172.21.15.46:0/3342227397 shutdown_connections 2024-03-29T08:29:59.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf5600640 1 --2- 172.21.15.46:0/3342227397 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcacc066690 0x7fcacc068b50 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf5600640 1 --2- 172.21.15.46:0/3342227397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcaf013bd20 0x7fcaf00ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf5600640 1 --2- 172.21.15.46:0/3342227397 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcaf0159200 0x7fcaf013f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf5600640 1 --2- 172.21.15.46:0/3342227397 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcad4003b10 0x7fcaf01397b0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf5600640 1 -- 172.21.15.46:0/3342227397 >> 172.21.15.46:0/3342227397 conn(0x7fcaf006a390 msgr2=0x7fcaf00708d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf5600640 1 -- 172.21.15.46:0/3342227397 shutdown_connections 2024-03-29T08:29:59.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.689+0000 7fcaf5600640 1 -- 172.21.15.46:0/3342227397 wait complete. 2024-03-29T08:29:59.709 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T08:29:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.849+0000 7f304b400640 1 Processor -- start 2024-03-29T08:29:59.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.849+0000 7f304b400640 1 -- start start 2024-03-29T08:29:59.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.849+0000 7f304b400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3044142ab0 0x7f304406d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.849+0000 7f304b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f304406d9e0 0x7f304415d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.849+0000 7f304b400640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304406de40 con 0x7f304415d940 2024-03-29T08:29:59.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.849+0000 7f304b400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3044070c70 con 0x7f304406d9e0 2024-03-29T08:29:59.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.849+0000 7f304b400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3044070340 con 0x7f3044142ab0 2024-03-29T08:29:59.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.853+0000 7f303be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f304406d9e0 0x7f304415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.853+0000 7f303be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f304406d9e0 0x7f304415d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44956/0 (socket says 172.21.15.46:44956) 2024-03-29T08:29:59.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.853+0000 7f303be00640 1 -- 172.21.15.46:0/1192135957 learned_addr learned my addr 172.21.15.46:0/1192135957 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:59.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.853+0000 7f3048a00640 1 --2- 172.21.15.46:0/1192135957 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3044142ab0 0x7f304406d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.853+0000 7f303be00640 1 -- 172.21.15.46:0/1192135957 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3044142ab0 msgr2=0x7f304406d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:29:59.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.853+0000 7f303be00640 1 --2- 172.21.15.46:0/1192135957 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3044142ab0 0x7f304406d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.853+0000 7f303be00640 1 -- 172.21.15.46:0/1192135957 >> v1:172.21.15.73:6789/0 conn(0x7f304415d940 legacy=0x7f30440ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.853+0000 7f303be00640 1 -- 172.21.15.46:0/1192135957 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3044070620 con 0x7f304406d9e0 2024-03-29T08:29:59.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.853+0000 7f303be00640 1 --2- 172.21.15.46:0/1192135957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f304406d9e0 0x7f304415d400 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f302c009870 tx=0x7f302c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19d2dd425e674d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.853+0000 7f3043e00640 1 -- 172.21.15.46:0/1192135957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c0ab600 con 0x7f304406d9e0 2024-03-29T08:29:59.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.853+0000 7f3043e00640 1 -- 172.21.15.46:0/1192135957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302c0a7050 con 0x7f304406d9e0 2024-03-29T08:29:59.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.853+0000 7f3043e00640 1 -- 172.21.15.46:0/1192135957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c0ab980 con 0x7f304406d9e0 2024-03-29T08:29:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f304b400640 1 -- 172.21.15.46:0/1192135957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f304406d9e0 msgr2=0x7f304415d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f304b400640 1 --2- 172.21.15.46:0/1192135957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f304406d9e0 0x7f304415d400 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f302c009870 tx=0x7f302c01c560 comp rx=0 tx=0).stop 2024-03-29T08:29:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f304b400640 1 -- 172.21.15.46:0/1192135957 shutdown_connections 2024-03-29T08:29:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f304b400640 1 --2- 172.21.15.46:0/1192135957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f304406d9e0 0x7f304415d400 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f304b400640 1 --2- 172.21.15.46:0/1192135957 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f3044142ab0 0x7f304406d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f304b400640 1 -- 172.21.15.46:0/1192135957 >> 172.21.15.46:0/1192135957 conn(0x7f3044066080 msgr2=0x7f3044069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:29:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f304b400640 1 -- 172.21.15.46:0/1192135957 shutdown_connections 2024-03-29T08:29:59.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f304b400640 1 -- 172.21.15.46:0/1192135957 wait complete. 2024-03-29T08:29:59.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f304b400640 1 Processor -- start 2024-03-29T08:29:59.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f304b400640 1 -- start start 2024-03-29T08:29:59.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f304b400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f304406d9e0 0x7f3044141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f304b400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3044142ab0 0x7f304413cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f304b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f304415d940 0x7f304413d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f304b400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30440ad310 con 0x7f304415d940 2024-03-29T08:29:59.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f304b400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3044157310 con 0x7f3044142ab0 2024-03-29T08:29:59.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f304b400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30440aca70 con 0x7f304406d9e0 2024-03-29T08:29:59.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f3049400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f304415d940 0x7f304413d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f3049400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f304415d940 0x7f304413d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44966/0 (socket says 172.21.15.46:44966) 2024-03-29T08:29:59.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f3049400640 1 -- 172.21.15.46:0/287041162 learned_addr learned my addr 172.21.15.46:0/287041162 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:29:59.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f3048a00640 1 --2- 172.21.15.46:0/287041162 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f304406d9e0 0x7f3044141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f303be00640 1 --2- 172.21.15.46:0/287041162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3044142ab0 0x7f304413cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f3049400640 1 -- 172.21.15.46:0/287041162 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f304406d9e0 msgr2=0x7f3044141d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f3049400640 1 --2- 172.21.15.46:0/287041162 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f304406d9e0 0x7f3044141d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f3049400640 1 -- 172.21.15.46:0/287041162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3044142ab0 msgr2=0x7f304413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:29:59.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f3049400640 1 --2- 172.21.15.46:0/287041162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3044142ab0 0x7f304413cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:29:59.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f3049400640 1 -- 172.21.15.46:0/287041162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c0094d0 con 0x7f304415d940 2024-03-29T08:29:59.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.857+0000 7f3049400640 1 --2- 172.21.15.46:0/287041162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f304415d940 0x7f304413d250 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f303c00c730 tx=0x7f303c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.861+0000 7f3041600640 1 -- 172.21.15.46:0/287041162 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c003b80 con 0x7f304415d940 2024-03-29T08:29:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.861+0000 7f3041600640 1 -- 172.21.15.46:0/287041162 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303c003ce0 con 0x7f304415d940 2024-03-29T08:29:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.861+0000 7f3041600640 1 -- 172.21.15.46:0/287041162 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c00aa30 con 0x7f304415d940 2024-03-29T08:29:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.861+0000 7f304b400640 1 -- 172.21.15.46:0/287041162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30440acd50 con 0x7f304415d940 2024-03-29T08:29:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.861+0000 7f304b400640 1 -- 172.21.15.46:0/287041162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30441422d0 con 0x7f304415d940 2024-03-29T08:29:59.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.861+0000 7f304b400640 1 -- 172.21.15.46:0/287041162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30441424c0 con 0x7f304415d940 2024-03-29T08:29:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.861+0000 7f3041600640 1 -- 172.21.15.46:0/287041162 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f303c00ab90 con 0x7f304415d940 2024-03-29T08:29:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.861+0000 7f3041600640 1 --2- 172.21.15.46:0/287041162 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3020066690 0x7f3020068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:29:59.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.861+0000 7f3041600640 1 -- 172.21.15.46:0/287041162 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f303c083ec0 con 0x7f304415d940 2024-03-29T08:29:59.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.861+0000 7f3048a00640 1 --2- 172.21.15.46:0/287041162 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3020066690 0x7f3020068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:29:59.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.865+0000 7f3048a00640 1 --2- 172.21.15.46:0/287041162 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3020066690 0x7f3020068b50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3034007a60 tx=0x7f3034008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:29:59.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:29:59.865+0000 7f3041600640 1 -- 172.21.15.46:0/287041162 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f303c051240 con 0x7f304415d940 2024-03-29T08:30:00.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.061+0000 7f304b400640 1 -- 172.21.15.46:0/287041162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f30441574d0 con 0x7f304415d940 2024-03-29T08:30:00.062 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-03-29T08:30:00.061+0000 7fc16ae00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:30:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.069+0000 7f3041600640 1 -- 172.21.15.46:0/287041162 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v101) v1 ==== 161+0+0 (secure 0 0 0) 0x7f303c054860 con 0x7f304415d940 2024-03-29T08:30:00.071 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T08:30:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.073+0000 7f304b400640 1 -- 172.21.15.46:0/287041162 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3020066690 msgr2=0x7f3020068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.073+0000 7f304b400640 1 --2- 172.21.15.46:0/287041162 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3020066690 0x7f3020068b50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f3034007a60 tx=0x7f3034008040 comp rx=0 tx=0).stop 2024-03-29T08:30:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.073+0000 7f304b400640 1 -- 172.21.15.46:0/287041162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f304415d940 msgr2=0x7f304413d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.073+0000 7f304b400640 1 --2- 172.21.15.46:0/287041162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f304415d940 0x7f304413d250 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f303c00c730 tx=0x7f303c00cbf0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.073+0000 7f304b400640 1 -- 172.21.15.46:0/287041162 shutdown_connections 2024-03-29T08:30:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.073+0000 7f304b400640 1 --2- 172.21.15.46:0/287041162 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3020066690 0x7f3020068b50 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.073+0000 7f304b400640 1 --2- 172.21.15.46:0/287041162 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3044142ab0 0x7f304413cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.073+0000 7f304b400640 1 --2- 172.21.15.46:0/287041162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f304415d940 0x7f304413d250 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.073+0000 7f304b400640 1 --2- 172.21.15.46:0/287041162 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f304406d9e0 0x7f3044141d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.073+0000 7f304b400640 1 -- 172.21.15.46:0/287041162 >> 172.21.15.46:0/287041162 conn(0x7f3044066080 msgr2=0x7f30440ae210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.073+0000 7f304b400640 1 -- 172.21.15.46:0/287041162 shutdown_connections 2024-03-29T08:30:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.073+0000 7f304b400640 1 -- 172.21.15.46:0/287041162 wait complete. 2024-03-29T08:30:00.083 INFO:tasks.ceph.mds.a.smithi046.stderr: -1996> 2024-03-29T08:29:52.389+0000 7f1773000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:30:00.099 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-03-29T08:30:00.132 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-03-29T08:30:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.145+0000 7f91a1a4a400 1 -- start start 2024-03-29T08:30:00.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.145+0000 7f91a1a4a400 1 Processor -- start 2024-03-29T08:30:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.145+0000 7f91a1a4a400 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a02125800 0x557a021b0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.145+0000 7f91a1a4a400 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a02125400 0x557a021b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.145+0000 7f91a1a4a400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a02125000 0x557a021b1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.145+0000 7f91a1a4a400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a0216be00 con 0x557a02125000 2024-03-29T08:30:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.145+0000 7f91a1a4a400 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a021f2a00 con 0x557a02125400 2024-03-29T08:30:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.145+0000 7f91a1a4a400 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a021f2640 con 0x557a02125800 2024-03-29T08:30:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.145+0000 7f919fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a02125000 0x557a021b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.145+0000 7f919fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a02125000 0x557a021b1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45002/0 (socket says 172.21.15.46:45002) 2024-03-29T08:30:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.145+0000 7f919e600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a02125400 0x557a021b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.145+0000 7f919f000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a02125800 0x557a021b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.145+0000 7f919fa00640 1 -- 172.21.15.46:0/3521132189 learned_addr learned my addr 172.21.15.46:0/3521132189 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:00.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919fa00640 1 -- 172.21.15.46:0/3521132189 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a02125800 msgr2=0x557a021b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919fa00640 1 --2- 172.21.15.46:0/3521132189 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a02125800 0x557a021b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919fa00640 1 -- 172.21.15.46:0/3521132189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a02125400 msgr2=0x557a021b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919fa00640 1 --2- 172.21.15.46:0/3521132189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a02125400 0x557a021b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919fa00640 1 -- 172.21.15.46:0/3521132189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a0215a9c0 con 0x557a02125000 2024-03-29T08:30:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919f000640 1 --2- 172.21.15.46:0/3521132189 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a02125800 0x557a021b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:30:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919e600640 1 --2- 172.21.15.46:0/3521132189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a02125400 0x557a021b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:30:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919fa00640 1 --2- 172.21.15.46:0/3521132189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a02125000 0x557a021b1080 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x557a02f94c90 tx=0x557a03038540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec630eef055255d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919be00640 1 -- 172.21.15.46:0/3521132189 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557a02179600 con 0x557a02125000 2024-03-29T08:30:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919be00640 1 -- 172.21.15.46:0/3521132189 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557a02182480 con 0x557a02125000 2024-03-29T08:30:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919be00640 1 -- 172.21.15.46:0/3521132189 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557a021791e0 con 0x557a02125000 2024-03-29T08:30:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3521132189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a0215ab60 con 0x557a02125000 2024-03-29T08:30:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3521132189 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x557a0215a340 con 0x557a02125000 2024-03-29T08:30:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f91a1a4a400 4 waiting for MDS map... 2024-03-29T08:30:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919be00640 1 -- 172.21.15.46:0/3521132189 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x557a02164c40 con 0x557a02125000 2024-03-29T08:30:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919be00640 1 -- 172.21.15.46:0/3521132189 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 101) v1 ==== 2573+0+0 (secure 0 0 0) 0x557a02172f00 con 0x557a02125000 2024-03-29T08:30:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f91a1a4a400 4 Got MDS map 101 2024-03-29T08:30:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f91a1a4a400 10 main: JournalTool::main 2024-03-29T08:30:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f91a1a4a400 4 main: JournalTool: connecting to RADOS... 2024-03-29T08:30:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f91a1a4a400 1 Processor -- start 2024-03-29T08:30:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f91a1a4a400 1 -- start start 2024-03-29T08:30:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f91a1a4a400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03052800 0x557a021b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f91a1a4a400 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a03052c00 0x557a021b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f91a1a4a400 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a03053000 0x557a021b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f91a1a4a400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a021f2500 con 0x557a03052800 2024-03-29T08:30:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f91a1a4a400 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a021f23c0 con 0x557a03052c00 2024-03-29T08:30:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f91a1a4a400 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a021f2a00 con 0x557a03053000 2024-03-29T08:30:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03052800 0x557a021b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03052800 0x557a021b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45010/0 (socket says 172.21.15.46:45010) 2024-03-29T08:30:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919e600640 1 -- 172.21.15.46:0/2444322467 learned_addr learned my addr 172.21.15.46:0/2444322467 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919e600640 1 -- 172.21.15.46:0/2444322467 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a03053000 msgr2=0x557a021b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919e600640 1 --2- 172.21.15.46:0/2444322467 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a03053000 0x557a021b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919e600640 1 -- 172.21.15.46:0/2444322467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a03052c00 msgr2=0x557a021b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919e600640 1 --2- 172.21.15.46:0/2444322467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a03052c00 0x557a021b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919e600640 1 -- 172.21.15.46:0/2444322467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a0215a000 con 0x557a03052800 2024-03-29T08:30:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.149+0000 7f919e600640 1 --2- 172.21.15.46:0/2444322467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03052800 0x557a021b1600 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x557a030aa120 tx=0x557a03060c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dda81aff186ed26 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f9199600640 1 -- 172.21.15.46:0/2444322467 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557a02179600 con 0x557a03052800 2024-03-29T08:30:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f9199600640 1 -- 172.21.15.46:0/2444322467 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557a02182480 con 0x557a03052800 2024-03-29T08:30:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f9199600640 1 -- 172.21.15.46:0/2444322467 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557a021791e0 con 0x557a03052800 2024-03-29T08:30:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 -- 172.21.15.46:0/2444322467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03052800 msgr2=0x557a021b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/2444322467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03052800 0x557a021b1600 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x557a030aa120 tx=0x557a03060c00 comp rx=0 tx=0).stop 2024-03-29T08:30:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 -- 172.21.15.46:0/2444322467 shutdown_connections 2024-03-29T08:30:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/2444322467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a03052c00 0x557a021b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/2444322467 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a03053000 0x557a021b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/2444322467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03052800 0x557a021b1600 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 -- 172.21.15.46:0/2444322467 >> 172.21.15.46:0/2444322467 conn(0x557a02124c00 msgr2=0x557a02151000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 -- 172.21.15.46:0/2444322467 shutdown_connections 2024-03-29T08:30:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 -- 172.21.15.46:0/2444322467 wait complete. 2024-03-29T08:30:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 Processor -- start 2024-03-29T08:30:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 -- start start 2024-03-29T08:30:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03052800 0x557a021b3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a03052c00 0x557a021b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a03053000 0x557a021b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a021f2280 con 0x557a03052800 2024-03-29T08:30:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a021f2140 con 0x557a03052c00 2024-03-29T08:30:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a021f2000 con 0x557a03053000 2024-03-29T08:30:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f919e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03052800 0x557a021b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f919e600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03052800 0x557a021b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45012/0 (socket says 172.21.15.46:45012) 2024-03-29T08:30:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f919e600640 1 -- 172.21.15.46:0/2536716988 learned_addr learned my addr 172.21.15.46:0/2536716988 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f919f000640 1 --2- 172.21.15.46:0/2536716988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a03052c00 0x557a021b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f919e600640 1 --2- 172.21.15.46:0/2536716988 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a03053000 0x557a021b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f919e600640 1 -- 172.21.15.46:0/2536716988 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a03053000 msgr2=0x557a021b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f919e600640 1 --2- 172.21.15.46:0/2536716988 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a03053000 0x557a021b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f919e600640 1 -- 172.21.15.46:0/2536716988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a03052c00 msgr2=0x557a021b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f919e600640 1 --2- 172.21.15.46:0/2536716988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a03052c00 0x557a021b2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f919e600640 1 -- 172.21.15.46:0/2536716988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a0215a9c0 con 0x557a03052800 2024-03-29T08:30:00.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f919f000640 1 --2- 172.21.15.46:0/2536716988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a03052c00 0x557a021b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f919e600640 1 --2- 172.21.15.46:0/2536716988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03052800 0x557a021b3180 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x557a030aade0 tx=0x557a03060c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f9197800640 1 -- 172.21.15.46:0/2536716988 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557a02178dc0 con 0x557a03052800 2024-03-29T08:30:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f9197800640 1 -- 172.21.15.46:0/2536716988 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557a02ef1500 con 0x557a03052800 2024-03-29T08:30:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 -- 172.21.15.46:0/2536716988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557a0215ab60 con 0x557a03052800 2024-03-29T08:30:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f9197800640 1 -- 172.21.15.46:0/2536716988 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557a02179600 con 0x557a03052800 2024-03-29T08:30:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 1 -- 172.21.15.46:0/2536716988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a0215a340 con 0x557a03052800 2024-03-29T08:30:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.153+0000 7f91a1a4a400 4 main: JournalTool: resolving pool 16 2024-03-29T08:30:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f9197800640 1 -- 172.21.15.46:0/2536716988 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x557a021f0840 con 0x557a03052800 2024-03-29T08:30:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f9197800640 1 --2- 172.21.15.46:0/2536716988 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x557a02409400 0x557a021b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f9197800640 1 -- 172.21.15.46:0/2536716988 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x557a02164a80 con 0x557a03052800 2024-03-29T08:30:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f91a1a4a400 4 main: JournalTool: creating IoCtx.. 2024-03-29T08:30:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f91a1a4a400 4 main: Executing for rank 0 2024-03-29T08:30:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f919f000640 1 --2- 172.21.15.46:0/2536716988 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x557a02409400 0x557a021b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f919f000640 1 --2- 172.21.15.46:0/2536716988 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x557a02409400 0x557a021b3700 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x557a0215d500 tx=0x557a030381e0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f91a1a4a400 1 -- start start 2024-03-29T08:30:00.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f91a1a4a400 1 Processor -- start 2024-03-29T08:30:00.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f91a1a4a400 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03053800 0x557a021b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f91a1a4a400 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a03053c00 0x557a021b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f91a1a4a400 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a02409800 0x557a021b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f91a1a4a400 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a021f3040 con 0x557a03053800 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f91a1a4a400 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a021f2000 con 0x557a02409800 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f91a1a4a400 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a021f2140 con 0x557a03053c00 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f919fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03053800 0x557a021b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f919fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03053800 0x557a021b2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45022/0 (socket says 172.21.15.46:45022) 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f919fa00640 1 -- 172.21.15.46:0/3749757678 learned_addr learned my addr 172.21.15.46:0/3749757678 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f919f000640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a03053c00 0x557a021b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.157+0000 7f919e600640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a02409800 0x557a021b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f919fa00640 1 -- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a03053c00 msgr2=0x557a021b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f919fa00640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a03053c00 0x557a021b1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f919fa00640 1 -- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a02409800 msgr2=0x557a021b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f919fa00640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a02409800 0x557a021b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f919fa00640 1 -- 172.21.15.46:0/3749757678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a024536c0 con 0x557a03053800 2024-03-29T08:30:00.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f919fa00640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03053800 0x557a021b2100 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x557a02f95080 tx=0x557a030383c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8060776d79019bbb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f9193c00640 1 -- 172.21.15.46:0/3749757678 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557a02178f20 con 0x557a03053800 2024-03-29T08:30:00.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f9193c00640 1 -- 172.21.15.46:0/3749757678 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557a02ef1c80 con 0x557a03053800 2024-03-29T08:30:00.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f9193c00640 1 -- 172.21.15.46:0/3749757678 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557a021786e0 con 0x557a03053800 2024-03-29T08:30:00.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a02453520 con 0x557a03053800 2024-03-29T08:30:00.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x557a02453380 con 0x557a03053800 2024-03-29T08:30:00.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f91a1a4a400 4 waiting for MDS map... 2024-03-29T08:30:00.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f9193c00640 1 -- 172.21.15.46:0/3749757678 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x557a021648c0 con 0x557a03053800 2024-03-29T08:30:00.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f9193c00640 1 -- 172.21.15.46:0/3749757678 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 101) v1 ==== 2573+0+0 (secure 0 0 0) 0x557a0255c900 con 0x557a03053800 2024-03-29T08:30:00.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f91a1a4a400 4 Got MDS map 101 2024-03-29T08:30:00.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x557a02409c00 0x557a0256e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:1 16.f 16:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x557a0255f000 con 0x557a02409c00 2024-03-29T08:30:00.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f919f000640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x557a02409c00 0x557a0256e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.161+0000 7f919f000640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x557a02409c00 0x557a0256e000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=c7e0443109bc26ad server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.165+0000 7f919f000640 1 -- 172.21.15.46:0/3749757678 <== osd.5 v2:172.21.15.73:6808/27690 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v125'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x557a0302ab40 con 0x557a02409c00 2024-03-29T08:30:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.165+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x557a0255f400 0x557a0256e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.165+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:2 16.14 16:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x557a0255f800 con 0x557a0255f400 2024-03-29T08:30:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.165+0000 7f919e600640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x557a0255f400 0x557a0256e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.165+0000 7f919e600640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x557a0255f400 0x557a0256e580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=e7d05fa52330d49d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f919e600640 1 -- 172.21.15.46:0/3749757678 <== osd.4 v2:172.21.15.73:6800/27689 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v125'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x557a0302ad80 con 0x557a0255f400 2024-03-29T08:30:00.169 INFO:teuthology.orchestra.run.smithi046.stdout:writing EResetJournal entry 2024-03-29T08:30:00.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:3 16.14 16:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x557a0255fc00 con 0x557a0255f400 2024-03-29T08:30:00.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x557a0257e000 0x557a0256eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:4 16.1b 16:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x557a0257e400 con 0x557a0257e000 2024-03-29T08:30:00.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x557a0257e800 0x557a0256f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:5 16.b 16:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x557a0257ec00 con 0x557a0257e800 2024-03-29T08:30:00.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x557a0257f000 0x557a0256f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:6 16.1f 16:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x557a0257f400 con 0x557a0257f000 2024-03-29T08:30:00.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x557a0257f800 0x557a0256fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:7 16.1a 16:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x557a0257fc00 con 0x557a0257f800 2024-03-29T08:30:00.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x557a02588000 0x557a02570100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:8 16.1e 16:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x557a02588400 con 0x557a02588000 2024-03-29T08:30:00.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f919fa00640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x557a0257e000 0x557a0256eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f919e600640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x557a0257f000 0x557a0256f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f919f000640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x557a0257e800 0x557a0256f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f919fa00640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x557a0257f800 0x557a0256fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f919f000640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x557a02588000 0x557a02570100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f919e600640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x557a0257f000 0x557a0256f600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=c1f212336f4cb32b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f919fa00640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x557a0257e000 0x557a0256eb00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=d470e5312dedbd90 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f919f000640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x557a02588000 0x557a02570100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=a178040c16c60249 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f919fa00640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x557a0257f800 0x557a0256fb80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=e7d216f0047b314d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f919f000640 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x557a0257e800 0x557a0256f080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=e2e7e836c2e0be23 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.169+0000 7f919e600640 1 -- 172.21.15.46:0/3749757678 <== osd.4 v2:172.21.15.73:6800/27689 2 ==== osd_op_reply(3 200.00000001 [delete] v125'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x557a0302ad80 con 0x557a0255f400 2024-03-29T08:30:00.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.173+0000 7f919f000640 1 -- 172.21.15.46:0/3749757678 <== osd.2 v2:172.21.15.46:6809/27428 1 ==== osd_op_reply(8 200.00000006 [delete] v125'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x557a0302ab40 con 0x557a02588000 2024-03-29T08:30:00.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.173+0000 7f919f000640 1 -- 172.21.15.46:0/3749757678 <== osd.7 v2:172.21.15.73:6816/27700 1 ==== osd_op_reply(5 200.00000003 [delete] v125'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x557a0302ab40 con 0x557a0257e800 2024-03-29T08:30:00.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.173+0000 7f919fa00640 1 -- 172.21.15.46:0/3749757678 <== osd.1 v2:172.21.15.46:6824/27427 1 ==== osd_op_reply(4 200.00000002 [delete] v125'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x557a0302b440 con 0x557a0257e000 2024-03-29T08:30:00.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.173+0000 7f9191e00640 1 -- 172.21.15.46:0/3749757678 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:9 16.14 16:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x557a02409000 con 0x557a0255f400 2024-03-29T08:30:00.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.173+0000 7f919e600640 1 -- 172.21.15.46:0/3749757678 <== osd.0 v2:172.21.15.46:6800/27424 1 ==== osd_op_reply(6 200.00000004 [delete] v125'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x557a0302ad80 con 0x557a0257f000 2024-03-29T08:30:00.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.173+0000 7f919fa00640 1 -- 172.21.15.46:0/3749757678 <== osd.6 v2:172.21.15.73:6824/27691 1 ==== osd_op_reply(7 200.00000005 [delete] v125'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x557a0302b440 con 0x557a0257f800 2024-03-29T08:30:00.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.173+0000 7f919e600640 1 -- 172.21.15.46:0/3749757678 <== osd.4 v2:172.21.15.73:6800/27689 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v125'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x557a0302ad80 con 0x557a0255f400 2024-03-29T08:30:00.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.173+0000 7f91a1a4a400 4 Successfully wrote new journal pointer and header for rank 8:0 2024-03-29T08:30:00.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x557a0257f000 msgr2=0x557a0256f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x557a0257f000 0x557a0256f600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x557a0257e000 msgr2=0x557a0256eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x557a0257e000 0x557a0256eb00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x557a02588000 msgr2=0x557a02570100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x557a02588000 0x557a02570100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x557a0255f400 msgr2=0x557a0256e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x557a0255f400 0x557a0256e580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x557a02409c00 msgr2=0x557a0256e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x557a02409c00 0x557a0256e000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x557a0257f800 msgr2=0x557a0256fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x557a0257f800 0x557a0256fb80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x557a0257e800 msgr2=0x557a0256f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x557a0257e800 0x557a0256f080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f919e600640 1 -- 172.21.15.46:0/3749757678 reap_dead start 2024-03-29T08:30:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f919e600640 1 -- 172.21.15.46:0/3749757678 reap_dead start 2024-03-29T08:30:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03053800 msgr2=0x557a021b2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3749757678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03053800 0x557a021b2100 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x557a02f95080 tx=0x557a030383c0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f919e600640 1 -- 172.21.15.46:0/3749757678 reap_dead start 2024-03-29T08:30:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 shutdown_connections 2024-03-29T08:30:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 >> 172.21.15.46:0/3749757678 conn(0x557a03053400 msgr2=0x557a02150800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 shutdown_connections 2024-03-29T08:30:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3749757678 wait complete. 2024-03-29T08:30:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/2536716988 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x557a02409400 msgr2=0x557a021b3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/2536716988 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x557a02409400 0x557a021b3700 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x557a0215d500 tx=0x557a030381e0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/2536716988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03052800 msgr2=0x557a021b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/2536716988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03052800 0x557a021b3180 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x557a030aade0 tx=0x557a03060c60 comp rx=0 tx=0).stop 2024-03-29T08:30:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/2536716988 shutdown_connections 2024-03-29T08:30:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/2536716988 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x557a02409400 0x557a021b3700 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/2536716988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a03052c00 0x557a021b2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/2536716988 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a03053000 0x557a021b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/2536716988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a03052800 0x557a021b3180 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/2536716988 >> 172.21.15.46:0/2536716988 conn(0x557a02124c00 msgr2=0x557a02151000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/2536716988 shutdown_connections 2024-03-29T08:30:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/2536716988 wait complete. 2024-03-29T08:30:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3521132189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a02125000 msgr2=0x557a021b1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3521132189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a02125000 0x557a021b1080 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x557a02f94c90 tx=0x557a03038540 comp rx=0 tx=0).stop 2024-03-29T08:30:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3521132189 shutdown_connections 2024-03-29T08:30:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3521132189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557a02125400 0x557a021b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3521132189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a02125000 0x557a021b1080 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 --2- 172.21.15.46:0/3521132189 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x557a02125800 0x557a021b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3521132189 >> 172.21.15.46:0/3521132189 conn(0x557a02125c00 msgr2=0x557a02151800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3521132189 shutdown_connections 2024-03-29T08:30:00.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.177+0000 7f91a1a4a400 1 -- 172.21.15.46:0/3521132189 wait complete. 2024-03-29T08:30:00.182 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.082471, result: writing EResetJournal entry 2024-03-29T08:30:00.182 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-03-29T08:30:00.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.265+0000 7f197cc918c0 1 -- start start 2024-03-29T08:30:00.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.265+0000 7f197cc918c0 1 Processor -- start 2024-03-29T08:30:00.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197cc918c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562353ed3800 0x562353f5e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197cc918c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562353ed3400 0x562353f5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197cc918c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562353ed3000 0x562353f5f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197cc918c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562353f19e00 con 0x562353ed3000 2024-03-29T08:30:00.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197cc918c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562353fa0a00 con 0x562353ed3400 2024-03-29T08:30:00.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197cc918c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x562353fa0640 con 0x562353ed3800 2024-03-29T08:30:00.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562353ed3000 0x562353f5f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197b800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562353ed3000 0x562353f5f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45030/0 (socket says 172.21.15.46:45030) 2024-03-29T08:30:00.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197b800640 1 -- 172.21.15.46:0/3161465611 learned_addr learned my addr 172.21.15.46:0/3161465611 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:00.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197a400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562353ed3400 0x562353f5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197ae00640 1 --2- 172.21.15.46:0/3161465611 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562353ed3800 0x562353f5e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197b800640 1 -- 172.21.15.46:0/3161465611 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562353ed3800 msgr2=0x562353f5e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197b800640 1 --2- 172.21.15.46:0/3161465611 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562353ed3800 0x562353f5e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197b800640 1 -- 172.21.15.46:0/3161465611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562353ed3400 msgr2=0x562353f5eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197b800640 1 --2- 172.21.15.46:0/3161465611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562353ed3400 0x562353f5eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197b800640 1 -- 172.21.15.46:0/3161465611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562353f06b60 con 0x562353ed3000 2024-03-29T08:30:00.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197b800640 1 --2- 172.21.15.46:0/3161465611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562353ed3000 0x562353f5f080 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x562354d3b440 tx=0x562354de2240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b54c306464d1111 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f1977c00640 1 -- 172.21.15.46:0/3161465611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562353f27600 con 0x562353ed3000 2024-03-29T08:30:00.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f1977c00640 1 -- 172.21.15.46:0/3161465611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562353f30900 con 0x562353ed3000 2024-03-29T08:30:00.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f1977c00640 1 -- 172.21.15.46:0/3161465611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562353f271e0 con 0x562353ed3000 2024-03-29T08:30:00.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197cc918c0 1 -- 172.21.15.46:0/3161465611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562353f06340 con 0x562353ed3000 2024-03-29T08:30:00.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197cc918c0 1 -- 172.21.15.46:0/3161465611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x562353f06000 con 0x562353ed3000 2024-03-29T08:30:00.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197cc918c0 4 waiting for MDS map... 2024-03-29T08:30:00.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f1977c00640 1 -- 172.21.15.46:0/3161465611 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x562353f12c40 con 0x562353ed3000 2024-03-29T08:30:00.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f1977c00640 1 -- 172.21.15.46:0/3161465611 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 101) v1 ==== 2573+0+0 (secure 0 0 0) 0x562353f20f00 con 0x562353ed3000 2024-03-29T08:30:00.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197cc918c0 4 Got MDS map 101 2024-03-29T08:30:00.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197cc918c0 4 datascan.main: Using metadata pool output 2024-03-29T08:30:00.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197cc918c0 4 datascan.main: connecting to RADOS... 2024-03-29T08:30:00.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197cc918c0 1 Processor -- start 2024-03-29T08:30:00.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197cc918c0 1 -- start start 2024-03-29T08:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197cc918c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562354df2c00 0x562353f5f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.269+0000 7f197cc918c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562354df3000 0x562353f5fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562354df3400 0x562353f60100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562353fa0500 con 0x562354df3000 2024-03-29T08:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562353fa03c0 con 0x562354df2c00 2024-03-29T08:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x562353fa0a00 con 0x562354df3400 2024-03-29T08:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562354df3000 0x562353f5fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562354df3000 0x562353f5fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45040/0 (socket says 172.21.15.46:45040) 2024-03-29T08:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197ae00640 1 -- 172.21.15.46:0/2464761283 learned_addr learned my addr 172.21.15.46:0/2464761283 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197ae00640 1 -- 172.21.15.46:0/2464761283 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562354df3400 msgr2=0x562353f60100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197ae00640 1 --2- 172.21.15.46:0/2464761283 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562354df3400 0x562353f60100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197ae00640 1 -- 172.21.15.46:0/2464761283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562354df2c00 msgr2=0x562353f5f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197ae00640 1 --2- 172.21.15.46:0/2464761283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562354df2c00 0x562353f5f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197ae00640 1 -- 172.21.15.46:0/2464761283 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562353f06ea0 con 0x562354df3000 2024-03-29T08:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197ae00640 1 --2- 172.21.15.46:0/2464761283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562354df3000 0x562353f5fb80 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x562354e02810 tx=0x562354e08c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48360e3488e64958 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f1975400640 1 -- 172.21.15.46:0/2464761283 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562353f27600 con 0x562354df3000 2024-03-29T08:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f1975400640 1 -- 172.21.15.46:0/2464761283 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562353f30900 con 0x562354df3000 2024-03-29T08:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f1975400640 1 -- 172.21.15.46:0/2464761283 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562353f271e0 con 0x562354df3000 2024-03-29T08:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 -- 172.21.15.46:0/2464761283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562354df3000 msgr2=0x562353f5fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 --2- 172.21.15.46:0/2464761283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562354df3000 0x562353f5fb80 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x562354e02810 tx=0x562354e08c00 comp rx=0 tx=0).stop 2024-03-29T08:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 -- 172.21.15.46:0/2464761283 shutdown_connections 2024-03-29T08:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 --2- 172.21.15.46:0/2464761283 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562354df3400 0x562353f60100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 --2- 172.21.15.46:0/2464761283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562354df3000 0x562353f5fb80 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 --2- 172.21.15.46:0/2464761283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562354df2c00 0x562353f5f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 -- 172.21.15.46:0/2464761283 >> 172.21.15.46:0/2464761283 conn(0x562353ed2c00 msgr2=0x562353efd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 -- 172.21.15.46:0/2464761283 shutdown_connections 2024-03-29T08:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 -- 172.21.15.46:0/2464761283 wait complete. 2024-03-29T08:30:00.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 Processor -- start 2024-03-29T08:30:00.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 -- start start 2024-03-29T08:30:00.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562354df2c00 0x562353f61180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562354df3000 0x562353f60c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562354df3400 0x562353f60680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x562353fa0280 con 0x562354df3000 2024-03-29T08:30:00.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562353fa0140 con 0x562354df3400 2024-03-29T08:30:00.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197cc918c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x562353fa0000 con 0x562354df2c00 2024-03-29T08:30:00.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562354df3000 0x562353f60c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197ae00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562354df3000 0x562353f60c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45042/0 (socket says 172.21.15.46:45042) 2024-03-29T08:30:00.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 learned_addr learned my addr 172.21.15.46:0/4002582748 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:00.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197a400640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562354df3400 0x562353f60680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197a400640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562354df2c00 0x562353f61180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562354df2c00 msgr2=0x562353f61180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197ae00640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562354df2c00 0x562353f61180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562354df3400 msgr2=0x562353f60680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197ae00640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562354df3400 0x562353f60680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562353f06b60 con 0x562354df3000 2024-03-29T08:30:00.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.273+0000 7f197a400640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562354df2c00 0x562353f61180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:00.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f197ae00640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562354df3000 0x562353f60c00 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x562354e034d0 tx=0x562354de21e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f1973600640 1 -- 172.21.15.46:0/4002582748 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562353f27600 con 0x562354df3000 2024-03-29T08:30:00.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f1973600640 1 -- 172.21.15.46:0/4002582748 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562354df9500 con 0x562354df3000 2024-03-29T08:30:00.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f1973600640 1 -- 172.21.15.46:0/4002582748 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562353f271e0 con 0x562354df3000 2024-03-29T08:30:00.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562353f06340 con 0x562354df3000 2024-03-29T08:30:00.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562353f06000 con 0x562354df3000 2024-03-29T08:30:00.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f197cc918c0 4 datascan.init: resolving metadata pool 16 2024-03-29T08:30:00.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f1973600640 1 -- 172.21.15.46:0/4002582748 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x562353f0a840 con 0x562354df3000 2024-03-29T08:30:00.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f1973600640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x562354e50c00 0x562353f61700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f1973600640 1 -- 172.21.15.46:0/4002582748 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x562353f12a80 con 0x562354df3000 2024-03-29T08:30:00.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f197cc918c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-29T08:30:00.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f197cc918c0 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-03-29T08:30:00.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f197cc918c0 4 datascan.main: opening data pool 'cephfs_data' 2024-03-29T08:30:00.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f197cc918c0 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x562354df3800 0x562353f60100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x562354df3c00 con 0x562354df3800 2024-03-29T08:30:00.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f197a400640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x562354e50c00 0x562353f61700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f197a400640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x562354df3800 0x562353f60100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f197a400640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x562354df3800 0x562353f60100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f197a400640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x562354e50c00 0x562353f61700 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x562354d3a900 tx=0x562354c54720 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.0 v2:172.21.15.46:6800/27424 1 ==== osd_op_reply(1 [pgnls start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x562354df3800 2024-03-29T08:30:00.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f1974a00640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x562354e51000 0x5623542b8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.277+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:2 17.20 17:04000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x562354e51400 con 0x562354e51000 2024-03-29T08:30:00.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.281+0000 7f197b800640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x562354e51000 0x5623542b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.281+0000 7f197b800640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x562354e51000 0x5623542b8000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.281+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.7 v2:172.21.15.73:6816/27700 1 ==== osd_op_reply(2 [pgnls start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x562354e51000 2024-03-29T08:30:00.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.281+0000 7f1975400640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x562354e51c00 0x5623542b8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.281+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:3 17.10 17:08000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c2000 con 0x562354e51c00 2024-03-29T08:30:00.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.281+0000 7f197ae00640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x562354e51c00 0x5623542b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.281+0000 7f197ae00640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x562354e51c00 0x5623542b8580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.281+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 <== osd.4 v2:172.21.15.73:6800/27689 1 ==== osd_op_reply(3 [pgnls start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4fc0 con 0x562354e51c00 2024-03-29T08:30:00.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.281+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:4 17.30 17:0c000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x562354e51800 con 0x562354e51000 2024-03-29T08:30:00.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.281+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.7 v2:172.21.15.73:6816/27700 2 ==== osd_op_reply(4 [pgnls start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x562354e51000 2024-03-29T08:30:00.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.281+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:5 17.8 17:10000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c2400 con 0x562354df3800 2024-03-29T08:30:00.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.281+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.0 v2:172.21.15.46:6800/27424 2 ==== osd_op_reply(5 [pgnls start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x562354df3800 2024-03-29T08:30:00.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.281+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:6 17.28 17:14000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c2800 con 0x562354e51c00 2024-03-29T08:30:00.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.281+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 <== osd.4 v2:172.21.15.73:6800/27689 2 ==== osd_op_reply(6 [pgnls start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4fc0 con 0x562354e51c00 2024-03-29T08:30:00.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.281+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:7 17.18 17:18000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c3400 con 0x562354e51000 2024-03-29T08:30:00.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.285+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.7 v2:172.21.15.73:6816/27700 3 ==== osd_op_reply(7 [pgnls start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x562354e51000 2024-03-29T08:30:00.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.285+0000 7f1974a00640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5623542c2c00 0x5623542b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.285+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:8 17.38 17:1c000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c3000 con 0x5623542c2c00 2024-03-29T08:30:00.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.285+0000 7f197a400640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5623542c2c00 0x5623542b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.285+0000 7f197a400640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5623542c2c00 0x5623542b8b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.285+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.3 v2:172.21.15.46:6808/27426 1 ==== osd_op_reply(8 [pgnls start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x5623542c2c00 2024-03-29T08:30:00.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.285+0000 7f1975400640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5623542c3800 0x5623542b9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.285+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:9 17.4 17:20000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c3c00 con 0x5623542c3800 2024-03-29T08:30:00.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.285+0000 7f197b800640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5623542c3800 0x5623542b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f197b800640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5623542c3800 0x5623542b9600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.1 v2:172.21.15.46:6824/27427 1 ==== osd_op_reply(9 [pgnls start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x5623542c3800 2024-03-29T08:30:00.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f1974a00640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x562354df3c00 0x5623542b9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:10 17.24 17:24000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x562354df2800 con 0x562354df3c00 2024-03-29T08:30:00.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f197ae00640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x562354df3c00 0x5623542b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f197ae00640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x562354df3c00 0x5623542b9080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 <== osd.5 v2:172.21.15.73:6808/27690 1 ==== osd_op_reply(10 [pgnls start_epoch 125 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4fc0 con 0x562354df3c00 2024-03-29T08:30:00.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:11 17.14 17:28000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c2400 con 0x562354df3c00 2024-03-29T08:30:00.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 <== osd.5 v2:172.21.15.73:6808/27690 2 ==== osd_op_reply(11 [pgnls start_epoch 125 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4fc0 con 0x562354df3c00 2024-03-29T08:30:00.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:12 17.34 17:2c000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x562354cad400 con 0x562354e51000 2024-03-29T08:30:00.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.7 v2:172.21.15.73:6816/27700 4 ==== osd_op_reply(12 [pgnls start_epoch 125 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x562354e51000 2024-03-29T08:30:00.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:13 17.c 17:30000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x562354e51400 con 0x5623542c2c00 2024-03-29T08:30:00.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.3 v2:172.21.15.46:6808/27426 2 ==== osd_op_reply(13 [pgnls start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x5623542c2c00 2024-03-29T08:30:00.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:14 17.2c 17:34000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x562354cad000 con 0x5623542c3800 2024-03-29T08:30:00.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.1 v2:172.21.15.46:6824/27427 2 ==== osd_op_reply(14 [pgnls start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x5623542c3800 2024-03-29T08:30:00.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:15 17.1c 17:38000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x562354e51800 con 0x562354df3c00 2024-03-29T08:30:00.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 <== osd.5 v2:172.21.15.73:6808/27690 3 ==== osd_op_reply(15 [pgnls start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4fc0 con 0x562354df3c00 2024-03-29T08:30:00.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:16 17.3c 17:3c000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c2000 con 0x562354e51c00 2024-03-29T08:30:00.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 <== osd.4 v2:172.21.15.73:6800/27689 3 ==== osd_op_reply(16 [pgnls start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4fc0 con 0x562354e51c00 2024-03-29T08:30:00.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.289+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:17 17.2 17:40000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x562354df2400 con 0x5623542c3800 2024-03-29T08:30:00.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.293+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.1 v2:172.21.15.46:6824/27427 3 ==== osd_op_reply(17 [pgnls start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x5623542c3800 2024-03-29T08:30:00.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.293+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:18 17.22 17:44000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c2800 con 0x5623542c2c00 2024-03-29T08:30:00.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.293+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.3 v2:172.21.15.46:6808/27426 3 ==== osd_op_reply(18 [pgnls start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x5623542c2c00 2024-03-29T08:30:00.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.293+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:19 17.12 17:48000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c3400 con 0x562354e51000 2024-03-29T08:30:00.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.293+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.7 v2:172.21.15.73:6816/27700 5 ==== osd_op_reply(19 [pgnls start_epoch 125 out=81b] v124'22 uv22 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562354dd4d80 con 0x562354e51000 2024-03-29T08:30:00.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.293+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:20 17.12 17:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d2000 con 0x562354e51000 2024-03-29T08:30:00.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.293+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.7 v2:172.21.15.73:6816/27700 6 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562354dd4d80 con 0x562354e51000 2024-03-29T08:30:00.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.293+0000 7f197cc918c0 10 datascan.operator(): handling object 1099511627778.0 2024-03-29T08:30:00.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.293+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:21 17.12 17:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d2400 con 0x562354e51000 2024-03-29T08:30:00.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.293+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.7 v2:172.21.15.73:6816/27700 7 ==== osd_op_reply(21 10000000002.00000000 [call] v125'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562354dd4d80 con 0x562354e51000 2024-03-29T08:30:00.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.293+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:22 17.32 17:4c000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d2800 con 0x562354df3800 2024-03-29T08:30:00.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.293+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.0 v2:172.21.15.46:6800/27424 3 ==== osd_op_reply(22 [pgnls start_epoch 125 out=49b] v121'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x562354df3800 2024-03-29T08:30:00.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.293+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:23 17.a 17:50000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d2000 con 0x5623542c3800 2024-03-29T08:30:00.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.293+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.1 v2:172.21.15.46:6824/27427 4 ==== osd_op_reply(23 [pgnls start_epoch 125 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x5623542c3800 2024-03-29T08:30:00.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.293+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:24 17.2a 17:54000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c2400 con 0x562354df3c00 2024-03-29T08:30:00.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.293+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 <== osd.5 v2:172.21.15.73:6808/27690 4 ==== osd_op_reply(24 [pgnls start_epoch 125 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4fc0 con 0x562354df3c00 2024-03-29T08:30:00.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.293+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:25 17.1a 17:58000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c3c00 con 0x562354e51c00 2024-03-29T08:30:00.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 <== osd.4 v2:172.21.15.73:6800/27689 4 ==== osd_op_reply(25 [pgnls start_epoch 125 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4fc0 con 0x562354e51c00 2024-03-29T08:30:00.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:26 17.3a 17:5c000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c3000 con 0x562354df3c00 2024-03-29T08:30:00.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 <== osd.5 v2:172.21.15.73:6808/27690 5 ==== osd_op_reply(26 [pgnls start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4fc0 con 0x562354df3c00 2024-03-29T08:30:00.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:27 17.6 17:60000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c3400 con 0x562354df3800 2024-03-29T08:30:00.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.0 v2:172.21.15.46:6800/27424 4 ==== osd_op_reply(27 [pgnls start_epoch 125 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x562354df3800 2024-03-29T08:30:00.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:28 17.26 17:64000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x562354df2800 con 0x562354df3c00 2024-03-29T08:30:00.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 <== osd.5 v2:172.21.15.73:6808/27690 6 ==== osd_op_reply(28 [pgnls start_epoch 125 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4fc0 con 0x562354df3c00 2024-03-29T08:30:00.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f1975400640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x562354df2400 0x5623542b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:29 17.16 17:68000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x562354cad400 con 0x562354df2400 2024-03-29T08:30:00.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f197a400640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x562354df2400 0x5623542b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f197a400640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x562354df2400 0x5623542b9b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.2 v2:172.21.15.46:6809/27428 1 ==== osd_op_reply(29 [pgnls start_epoch 125 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x562354df2400 2024-03-29T08:30:00.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:30 17.36 17:6c000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d3c00 con 0x562354e51000 2024-03-29T08:30:00.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.7 v2:172.21.15.73:6816/27700 8 ==== osd_op_reply(30 [pgnls start_epoch 125 out=81b] v121'33 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562354dd4d80 con 0x562354e51000 2024-03-29T08:30:00.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:31 17.36 17:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d2c00 con 0x562354e51000 2024-03-29T08:30:00.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.7 v2:172.21.15.73:6816/27700 9 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562354dd4d80 con 0x562354e51000 2024-03-29T08:30:00.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f197cc918c0 10 datascan.operator(): handling object 1099511628279.0 2024-03-29T08:30:00.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.297+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:32 17.36 17:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d3000 con 0x562354e51000 2024-03-29T08:30:00.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.301+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.7 v2:172.21.15.73:6816/27700 10 ==== osd_op_reply(32 100000001f7.00000000 [call] v125'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562354dd4d80 con 0x562354e51000 2024-03-29T08:30:00.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.301+0000 7f197cc918c0 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5623542d3400 0x562353f5fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.301+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:33 17.e 17:70000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d3800 con 0x5623542d3400 2024-03-29T08:30:00.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.301+0000 7f197b800640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5623542d3400 0x562353f5fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.301+0000 7f197b800640 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5623542d3400 0x562353f5fb80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.301+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.6 v2:172.21.15.73:6824/27691 1 ==== osd_op_reply(33 [pgnls start_epoch 125 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x5623542d3400 2024-03-29T08:30:00.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.301+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:34 17.2e 17:74000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c2000 con 0x5623542c2c00 2024-03-29T08:30:00.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.301+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.3 v2:172.21.15.46:6808/27426 4 ==== osd_op_reply(34 [pgnls start_epoch 125 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x5623542c2c00 2024-03-29T08:30:00.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.301+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:35 17.1e 17:78000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x562354cad000 con 0x5623542d3400 2024-03-29T08:30:00.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.301+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.6 v2:172.21.15.73:6824/27691 2 ==== osd_op_reply(35 [pgnls start_epoch 125 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x5623542d3400 2024-03-29T08:30:00.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.301+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:36 17.3e 17:7c000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c2400 con 0x562354df2400 2024-03-29T08:30:00.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.301+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.2 v2:172.21.15.46:6809/27428 2 ==== osd_op_reply(36 [pgnls start_epoch 125 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x562354df2400 2024-03-29T08:30:00.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.301+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:37 17.1 17:80000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d2400 con 0x562354df3c00 2024-03-29T08:30:00.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.301+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 <== osd.5 v2:172.21.15.73:6808/27690 7 ==== osd_op_reply(37 [pgnls start_epoch 125 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4fc0 con 0x562354df3c00 2024-03-29T08:30:00.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.301+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:38 17.21 17:84000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c3c00 con 0x5623542d3400 2024-03-29T08:30:00.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.301+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.6 v2:172.21.15.73:6824/27691 3 ==== osd_op_reply(38 [pgnls start_epoch 125 out=49b] v121'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x5623542d3400 2024-03-29T08:30:00.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.301+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:39 17.11 17:88000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d2000 con 0x562354e51c00 2024-03-29T08:30:00.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.305+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 <== osd.4 v2:172.21.15.73:6800/27689 5 ==== osd_op_reply(39 [pgnls start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4fc0 con 0x562354e51c00 2024-03-29T08:30:00.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.305+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:40 17.31 17:8c000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c3000 con 0x5623542d3400 2024-03-29T08:30:00.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.305+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.6 v2:172.21.15.73:6824/27691 4 ==== osd_op_reply(40 [pgnls start_epoch 125 out=81b] v121'15 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562354dd4d80 con 0x5623542d3400 2024-03-29T08:30:00.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.305+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:41 17.31 17:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c2800 con 0x5623542d3400 2024-03-29T08:30:00.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.305+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.6 v2:172.21.15.73:6824/27691 5 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562354dd4d80 con 0x5623542d3400 2024-03-29T08:30:00.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.305+0000 7f197cc918c0 10 datascan.operator(): handling object 1099511627780.1 2024-03-29T08:30:00.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.305+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:42 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c3400 con 0x562354df3800 2024-03-29T08:30:00.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.305+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.0 v2:172.21.15.46:6800/27424 5 ==== osd_op_reply(42 10000000004.00000000 [call] v125'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562354dd4b40 con 0x562354df3800 2024-03-29T08:30:00.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.305+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:43 17.9 17:90000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c2000 con 0x562354e51000 2024-03-29T08:30:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.305+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.7 v2:172.21.15.73:6816/27700 11 ==== osd_op_reply(43 [pgnls start_epoch 125 out=49b] v121'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x562354e51000 2024-03-29T08:30:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.305+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:44 17.29 17:94000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x562354e51800 con 0x562354df3800 2024-03-29T08:30:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.305+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.0 v2:172.21.15.46:6800/27424 6 ==== osd_op_reply(44 [pgnls start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x562354df3800 2024-03-29T08:30:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.305+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:45 17.19 17:98000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d3c00 con 0x5623542c2c00 2024-03-29T08:30:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.309+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.3 v2:172.21.15.46:6808/27426 5 ==== osd_op_reply(45 [pgnls start_epoch 125 out=81b] v124'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562354dd4b40 con 0x5623542c2c00 2024-03-29T08:30:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.309+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:46 17.19 17:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x562354cad000 con 0x5623542c2c00 2024-03-29T08:30:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.309+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.3 v2:172.21.15.46:6808/27426 6 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562354dd4b40 con 0x5623542c2c00 2024-03-29T08:30:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.309+0000 7f197cc918c0 10 datascan.operator(): handling object 1099511628280.0 2024-03-29T08:30:00.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.309+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:47 17.19 17:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x562354cad400 con 0x5623542c2c00 2024-03-29T08:30:00.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.309+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.3 v2:172.21.15.46:6808/27426 7 ==== osd_op_reply(47 100000001f8.00000000 [call] v125'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562354dd4b40 con 0x5623542c2c00 2024-03-29T08:30:00.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.309+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:48 17.39 17:9c000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d2800 con 0x5623542c2c00 2024-03-29T08:30:00.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.309+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.3 v2:172.21.15.46:6808/27426 8 ==== osd_op_reply(48 [pgnls start_epoch 125 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x5623542c2c00 2024-03-29T08:30:00.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.309+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:49 17.5 17:a0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d2c00 con 0x562354df3800 2024-03-29T08:30:00.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.309+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.0 v2:172.21.15.46:6800/27424 7 ==== osd_op_reply(49 [pgnls start_epoch 125 out=81b] v125'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562354dd4b40 con 0x562354df3800 2024-03-29T08:30:00.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.309+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:50 17.5 17:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d3c00 con 0x562354df3800 2024-03-29T08:30:00.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.309+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.0 v2:172.21.15.46:6800/27424 8 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562354dd4b40 con 0x562354df3800 2024-03-29T08:30:00.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.309+0000 7f197cc918c0 10 datascan.operator(): handling object 1099511627780.0 2024-03-29T08:30:00.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.309+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:51 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d2800 con 0x562354df3800 2024-03-29T08:30:00.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.0 v2:172.21.15.46:6800/27424 9 ==== osd_op_reply(51 10000000004.00000000 [call] v125'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562354dd4b40 con 0x562354df3800 2024-03-29T08:30:00.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:52 17.25 17:a4000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x562354cad000 con 0x562354e51c00 2024-03-29T08:30:00.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 <== osd.4 v2:172.21.15.73:6800/27689 6 ==== osd_op_reply(52 [pgnls start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4fc0 con 0x562354e51c00 2024-03-29T08:30:00.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:53 17.15 17:a8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d3000 con 0x5623542c2c00 2024-03-29T08:30:00.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.3 v2:172.21.15.46:6808/27426 9 ==== osd_op_reply(53 [pgnls start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x5623542c2c00 2024-03-29T08:30:00.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:54 17.35 17:ac000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542e4000 con 0x5623542c2c00 2024-03-29T08:30:00.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.3 v2:172.21.15.46:6808/27426 10 ==== osd_op_reply(54 [pgnls start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x5623542c2c00 2024-03-29T08:30:00.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:55 17.d 17:b0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d3800 con 0x562354df3800 2024-03-29T08:30:00.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.0 v2:172.21.15.46:6800/27424 10 ==== osd_op_reply(55 [pgnls start_epoch 125 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x562354df3800 2024-03-29T08:30:00.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:56 17.2d 17:b4000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542e4400 con 0x5623542c3800 2024-03-29T08:30:00.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.1 v2:172.21.15.46:6824/27427 5 ==== osd_op_reply(56 [pgnls start_epoch 125 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x5623542c3800 2024-03-29T08:30:00.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:57 17.1d 17:b8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x562354e51400 con 0x5623542c3800 2024-03-29T08:30:00.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.1 v2:172.21.15.46:6824/27427 6 ==== osd_op_reply(57 [pgnls start_epoch 125 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x5623542c3800 2024-03-29T08:30:00.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:58 17.3d 17:bc000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542e4800 con 0x5623542c3800 2024-03-29T08:30:00.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.1 v2:172.21.15.46:6824/27427 7 ==== osd_op_reply(58 [pgnls start_epoch 125 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x5623542c3800 2024-03-29T08:30:00.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:59 17.3 17:c0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542e5c00 con 0x562354df3800 2024-03-29T08:30:00.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.0 v2:172.21.15.46:6800/27424 11 ==== osd_op_reply(59 [pgnls start_epoch 125 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x562354df3800 2024-03-29T08:30:00.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:60 17.23 17:c4000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542e4c00 con 0x562354df3800 2024-03-29T08:30:00.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.0 v2:172.21.15.46:6800/27424 12 ==== osd_op_reply(60 [pgnls start_epoch 125 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x562354df3800 2024-03-29T08:30:00.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:61 17.13 17:c8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542e4000 con 0x562354e51c00 2024-03-29T08:30:00.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 <== osd.4 v2:172.21.15.73:6800/27689 7 ==== osd_op_reply(61 [pgnls start_epoch 125 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4fc0 con 0x562354e51c00 2024-03-29T08:30:00.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.313+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:62 17.33 17:cc000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542e5000 con 0x562354df2400 2024-03-29T08:30:00.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.2 v2:172.21.15.46:6809/27428 3 ==== osd_op_reply(62 [pgnls start_epoch 125 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x562354df2400 2024-03-29T08:30:00.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:63 17.b 17:d0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542e5c00 con 0x5623542d3400 2024-03-29T08:30:00.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.6 v2:172.21.15.73:6824/27691 6 ==== osd_op_reply(63 [pgnls start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x5623542d3400 2024-03-29T08:30:00.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:64 17.2b 17:d4000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542e5400 con 0x5623542c2c00 2024-03-29T08:30:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.3 v2:172.21.15.46:6808/27426 11 ==== osd_op_reply(64 [pgnls start_epoch 125 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x5623542c2c00 2024-03-29T08:30:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:65 17.1b 17:d8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542e4c00 con 0x5623542c2c00 2024-03-29T08:30:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.3 v2:172.21.15.46:6808/27426 12 ==== osd_op_reply(65 [pgnls start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x5623542c2c00 2024-03-29T08:30:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:66 17.3b 17:dc000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542e5800 con 0x562354e51000 2024-03-29T08:30:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.7 v2:172.21.15.73:6816/27700 12 ==== osd_op_reply(66 [pgnls start_epoch 125 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x562354e51000 2024-03-29T08:30:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:67 17.7 17:e0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d2c00 con 0x562354df3c00 2024-03-29T08:30:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 <== osd.5 v2:172.21.15.73:6808/27690 8 ==== osd_op_reply(67 [pgnls start_epoch 125 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4fc0 con 0x562354df3c00 2024-03-29T08:30:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:68 17.27 17:e4000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c3c00 con 0x5623542c2c00 2024-03-29T08:30:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.3 v2:172.21.15.46:6808/27426 13 ==== osd_op_reply(68 [pgnls start_epoch 125 out=49b] v121'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x5623542c2c00 2024-03-29T08:30:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:69 17.17 17:e8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d3c00 con 0x562354df2400 2024-03-29T08:30:00.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.2 v2:172.21.15.46:6809/27428 4 ==== osd_op_reply(69 [pgnls start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x562354df2400 2024-03-29T08:30:00.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:70 17.37 17:ec000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c3000 con 0x5623542d3400 2024-03-29T08:30:00.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.6 v2:172.21.15.73:6824/27691 7 ==== osd_op_reply(70 [pgnls start_epoch 125 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x5623542d3400 2024-03-29T08:30:00.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:71 17.f 17:f0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d2800 con 0x5623542d3400 2024-03-29T08:30:00.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.6 v2:172.21.15.73:6824/27691 8 ==== osd_op_reply(71 [pgnls start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x5623542d3400 2024-03-29T08:30:00.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:72 17.2f 17:f4000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542c2800 con 0x5623542c3800 2024-03-29T08:30:00.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.317+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.1 v2:172.21.15.46:6824/27427 8 ==== osd_op_reply(72 [pgnls start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x5623542c3800 2024-03-29T08:30:00.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f1975400640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:73 17.1f 17:f8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d3000 con 0x562354df3800 2024-03-29T08:30:00.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197a400640 1 -- 172.21.15.46:0/4002582748 <== osd.0 v2:172.21.15.46:6800/27424 13 ==== osd_op_reply(73 [pgnls start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4b40 con 0x562354df3800 2024-03-29T08:30:00.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f1974a00640 1 -- 172.21.15.46:0/4002582748 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:74 17.3f 17:fc000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5623542d2400 con 0x5623542d3400 2024-03-29T08:30:00.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197b800640 1 -- 172.21.15.46:0/4002582748 <== osd.6 v2:172.21.15.73:6824/27691 9 ==== osd_op_reply(74 [pgnls start_epoch 125 out=49b] v121'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562354dd4d80 con 0x5623542d3400 2024-03-29T08:30:00.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x562354df3800 msgr2=0x562353f60100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x562354df3800 0x562353f60100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5623542c3800 msgr2=0x5623542b9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5623542c3800 0x5623542b9600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x562354df2400 msgr2=0x5623542b9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x562354df2400 0x5623542b9b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5623542c2c00 msgr2=0x5623542b8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5623542c2c00 0x5623542b8b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x562354e51c00 msgr2=0x5623542b8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 reap_dead start 2024-03-29T08:30:00.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x562354e51c00 0x5623542b8580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x562354df3c00 msgr2=0x5623542b9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x562354df3c00 0x5623542b9080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5623542d3400 msgr2=0x562353f5fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5623542d3400 0x562353f5fb80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x562354e51000 msgr2=0x5623542b8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x562354e51000 0x5623542b8000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x562354e50c00 msgr2=0x562353f61700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x562354e50c00 0x562353f61700 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x562354d3a900 tx=0x562354c54720 comp rx=0 tx=0).stop 2024-03-29T08:30:00.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562354df3000 msgr2=0x562353f60c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562354df3000 0x562353f60c00 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x562354e034d0 tx=0x562354de21e0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197ae00640 1 -- 172.21.15.46:0/4002582748 reap_dead start 2024-03-29T08:30:00.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 shutdown_connections 2024-03-29T08:30:00.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 --2- 172.21.15.46:0/4002582748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562354df3000 0x562353f60c00 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 >> 172.21.15.46:0/4002582748 conn(0x562353ed2c00 msgr2=0x562353efd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 shutdown_connections 2024-03-29T08:30:00.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/4002582748 wait complete. 2024-03-29T08:30:00.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/3161465611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562353ed3000 msgr2=0x562353f5f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 --2- 172.21.15.46:0/3161465611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562353ed3000 0x562353f5f080 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x562354d3b440 tx=0x562354de2240 comp rx=0 tx=0).stop 2024-03-29T08:30:00.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/3161465611 shutdown_connections 2024-03-29T08:30:00.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 --2- 172.21.15.46:0/3161465611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562353ed3400 0x562353f5eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 --2- 172.21.15.46:0/3161465611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562353ed3000 0x562353f5f080 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 --2- 172.21.15.46:0/3161465611 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x562353ed3800 0x562353f5e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/3161465611 >> 172.21.15.46:0/3161465611 conn(0x562353ed3c00 msgr2=0x562353efd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/3161465611 shutdown_connections 2024-03-29T08:30:00.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.321+0000 7f197cc918c0 1 -- 172.21.15.46:0/3161465611 wait complete. 2024-03-29T08:30:00.327 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.144579, result: 2024-03-29T08:30:00.327 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-03-29T08:30:00.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732c9cd8c0 1 -- start start 2024-03-29T08:30:00.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732c9cd8c0 1 Processor -- start 2024-03-29T08:30:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732c9cd8c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643ff67d800 0x5643ff709b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732c9cd8c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643ff67d400 0x5643ff709600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732c9cd8c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5643ff67d000 0x5643ff709080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732c9cd8c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643ff74aa00 con 0x5643ff67d400 2024-03-29T08:30:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732c9cd8c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643ff74a640 con 0x5643ff67d800 2024-03-29T08:30:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732c9cd8c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643ff74a500 con 0x5643ff67d000 2024-03-29T08:30:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643ff67d400 0x5643ff709600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643ff67d400 0x5643ff709600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45074/0 (socket says 172.21.15.46:45074) 2024-03-29T08:30:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732a000640 1 -- 172.21.15.46:0/3703594287 learned_addr learned my addr 172.21.15.46:0/3703594287 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732aa00640 1 --2- 172.21.15.46:0/3703594287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643ff67d800 0x5643ff709b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732b400640 1 --2- 172.21.15.46:0/3703594287 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5643ff67d000 0x5643ff709080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732a000640 1 -- 172.21.15.46:0/3703594287 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5643ff67d000 msgr2=0x5643ff709080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732a000640 1 --2- 172.21.15.46:0/3703594287 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5643ff67d000 0x5643ff709080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732a000640 1 -- 172.21.15.46:0/3703594287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643ff67d800 msgr2=0x5643ff709b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732a000640 1 --2- 172.21.15.46:0/3703594287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643ff67d800 0x5643ff709b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732a000640 1 -- 172.21.15.46:0/3703594287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643ff6b09c0 con 0x5643ff67d400 2024-03-29T08:30:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732a000640 1 --2- 172.21.15.46:0/3703594287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643ff67d400 0x5643ff709600 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x5644004eb320 tx=0x5644004f8d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cadbdee2a69fb58b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f7327800640 1 -- 172.21.15.46:0/3703594287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5643ff6d1600 con 0x5643ff67d400 2024-03-29T08:30:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f7327800640 1 -- 172.21.15.46:0/3703594287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56440058e000 con 0x5643ff67d400 2024-03-29T08:30:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f7327800640 1 -- 172.21.15.46:0/3703594287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5643ff6d11e0 con 0x5643ff67d400 2024-03-29T08:30:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/3703594287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643ff6b0b60 con 0x5643ff67d400 2024-03-29T08:30:00.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/3703594287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5643ff6b0340 con 0x5643ff67d400 2024-03-29T08:30:00.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732c9cd8c0 4 waiting for MDS map... 2024-03-29T08:30:00.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f7327800640 1 -- 172.21.15.46:0/3703594287 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x5643ff6bcc40 con 0x5643ff67d400 2024-03-29T08:30:00.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f7327800640 1 -- 172.21.15.46:0/3703594287 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 101) v1 ==== 2573+0+0 (secure 0 0 0) 0x5643ff6caf00 con 0x5643ff67d400 2024-03-29T08:30:00.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732c9cd8c0 4 Got MDS map 101 2024-03-29T08:30:00.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732c9cd8c0 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-03-29T08:30:00.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732c9cd8c0 4 datascan.main: Using metadata pool output 2024-03-29T08:30:00.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.413+0000 7f732c9cd8c0 4 datascan.main: connecting to RADOS... 2024-03-29T08:30:00.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 Processor -- start 2024-03-29T08:30:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 -- start start 2024-03-29T08:30:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56440058cc00 0x5643ff708b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56440058d000 0x5643ff708580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56440058d400 0x5643ff70a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643ff6c3e00 con 0x56440058d400 2024-03-29T08:30:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643ff74a640 con 0x56440058cc00 2024-03-29T08:30:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643ff74a3c0 con 0x56440058d000 2024-03-29T08:30:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56440058d400 0x5643ff70a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56440058d400 0x5643ff70a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45086/0 (socket says 172.21.15.46:45086) 2024-03-29T08:30:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732a000640 1 -- 172.21.15.46:0/3542088833 learned_addr learned my addr 172.21.15.46:0/3542088833 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732b400640 1 --2- 172.21.15.46:0/3542088833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56440058cc00 0x5643ff708b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732aa00640 1 --2- 172.21.15.46:0/3542088833 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56440058d000 0x5643ff708580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732a000640 1 -- 172.21.15.46:0/3542088833 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56440058d000 msgr2=0x5643ff708580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732a000640 1 --2- 172.21.15.46:0/3542088833 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56440058d000 0x5643ff708580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732a000640 1 -- 172.21.15.46:0/3542088833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56440058cc00 msgr2=0x5643ff708b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732a000640 1 --2- 172.21.15.46:0/3542088833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56440058cc00 0x5643ff708b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732a000640 1 -- 172.21.15.46:0/3542088833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643ff6b0000 con 0x56440058d400 2024-03-29T08:30:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732aa00640 1 --2- 172.21.15.46:0/3542088833 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56440058d000 0x5643ff708580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732a000640 1 --2- 172.21.15.46:0/3542088833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56440058d400 0x5643ff70a100 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x5644005f26c0 tx=0x564400407140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6f3da7a2b53cdf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f7325000640 1 -- 172.21.15.46:0/3542088833 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5643ff6d0f20 con 0x56440058d400 2024-03-29T08:30:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f7325000640 1 -- 172.21.15.46:0/3542088833 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56440058e600 con 0x56440058d400 2024-03-29T08:30:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f7325000640 1 -- 172.21.15.46:0/3542088833 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5643ff6d1600 con 0x56440058d400 2024-03-29T08:30:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/3542088833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56440058d400 msgr2=0x5643ff70a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/3542088833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56440058d400 0x5643ff70a100 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x5644005f26c0 tx=0x564400407140 comp rx=0 tx=0).stop 2024-03-29T08:30:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/3542088833 shutdown_connections 2024-03-29T08:30:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/3542088833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56440058d400 0x5643ff70a100 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/3542088833 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56440058d000 0x5643ff708580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/3542088833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56440058cc00 0x5643ff708b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/3542088833 >> 172.21.15.46:0/3542088833 conn(0x5643ff67cc00 msgr2=0x5643ff6a9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/3542088833 shutdown_connections 2024-03-29T08:30:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/3542088833 wait complete. 2024-03-29T08:30:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.417+0000 7f732c9cd8c0 1 Processor -- start 2024-03-29T08:30:00.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732c9cd8c0 1 -- start start 2024-03-29T08:30:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732c9cd8c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56440058cc00 0x5643ff708b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732c9cd8c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56440058d000 0x5643ff708580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732c9cd8c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56440058d400 0x5643ff70a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732c9cd8c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643ff74a280 con 0x56440058cc00 2024-03-29T08:30:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732c9cd8c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643ff74a140 con 0x56440058d000 2024-03-29T08:30:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732c9cd8c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643ff74a000 con 0x56440058d400 2024-03-29T08:30:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56440058cc00 0x5643ff708b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56440058cc00 0x5643ff708b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45098/0 (socket says 172.21.15.46:45098) 2024-03-29T08:30:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 learned_addr learned my addr 172.21.15.46:0/2843095975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56440058d400 msgr2=0x5643ff70a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732b400640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x56440058d400 0x5643ff70a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56440058d000 msgr2=0x5643ff708580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732b400640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56440058d000 0x5643ff708580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643ff6b09c0 con 0x56440058cc00 2024-03-29T08:30:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732b400640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56440058cc00 0x5643ff708b00 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x5644005f33e0 tx=0x5644005a8e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f7323200640 1 -- 172.21.15.46:0/2843095975 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5643ff6d0f20 con 0x56440058cc00 2024-03-29T08:30:00.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f7323200640 1 -- 172.21.15.46:0/2843095975 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5643ff6da480 con 0x56440058cc00 2024-03-29T08:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f7323200640 1 -- 172.21.15.46:0/2843095975 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5643ff6d1600 con 0x56440058cc00 2024-03-29T08:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5643ff6b0b60 con 0x56440058cc00 2024-03-29T08:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643ff6b0ea0 con 0x56440058cc00 2024-03-29T08:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f732c9cd8c0 4 datascan.init: resolving metadata pool 16 2024-03-29T08:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f7323200640 1 -- 172.21.15.46:0/2843095975 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x5643ff6b4840 con 0x56440058cc00 2024-03-29T08:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f7323200640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5643ff95ec00 0x5643ffa42000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.421+0000 7f7323200640 1 -- 172.21.15.46:0/2843095975 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x5643ff6bca80 con 0x56440058cc00 2024-03-29T08:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732c9cd8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-29T08:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732c9cd8c0 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-03-29T08:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732c9cd8c0 4 datascan.main: opening data pool 'cephfs_data' 2024-03-29T08:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x56440058d800 0x5643ff70a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:1 16.14 16:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x56440058dc00 con 0x56440058d800 2024-03-29T08:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732aa00640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5643ff95ec00 0x5643ffa42000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732a000640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x56440058d800 0x5643ff70a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732aa00640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5643ff95ec00 0x5643ffa42000 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x5643ff749740 tx=0x5644004f80c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732a000640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x56440058d800 0x5643ff70a680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.4 v2:172.21.15.73:6800/27689 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv35 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5644003dcb40 con 0x56440058d800 2024-03-29T08:30:00.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5643ff95f000 0x5643ff70ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:2 16.1f 16:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ff95f400 con 0x5643ff95f000 2024-03-29T08:30:00.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732b400640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5643ff95f000 0x5643ff70ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732b400640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5643ff95f000 0x5643ff70ac00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.0 v2:172.21.15.46:6800/27424 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5644003dcd80 con 0x5643ff95f000 2024-03-29T08:30:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:3 17.0 17:00000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ff95f800 con 0x5643ff95f000 2024-03-29T08:30:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.0 v2:172.21.15.46:6800/27424 2 ==== osd_op_reply(3 [pgnls-filter start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ff95f000 2024-03-29T08:30:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f7324600640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5643ffa5a000 0x5643ffa42580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:4 17.20 17:04000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5a400 con 0x5643ffa5a000 2024-03-29T08:30:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.425+0000 7f732aa00640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5643ffa5a000 0x5643ffa42580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f732aa00640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5643ffa5a000 0x5643ffa42580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.7 v2:172.21.15.73:6816/27700 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x5643ffa5a000 2024-03-29T08:30:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:5 17.10 17:08000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5ac00 con 0x56440058d800 2024-03-29T08:30:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.4 v2:172.21.15.73:6800/27689 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x56440058d800 2024-03-29T08:30:00.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:6 17.30 17:0c000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5a800 con 0x5643ffa5a000 2024-03-29T08:30:00.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.7 v2:172.21.15.73:6816/27700 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x5643ffa5a000 2024-03-29T08:30:00.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:7 17.8 17:10000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5b000 con 0x5643ff95f000 2024-03-29T08:30:00.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.0 v2:172.21.15.46:6800/27424 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ff95f000 2024-03-29T08:30:00.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:8 17.28 17:14000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5b800 con 0x56440058d800 2024-03-29T08:30:00.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.4 v2:172.21.15.73:6800/27689 3 ==== osd_op_reply(8 [pgnls-filter start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x56440058d800 2024-03-29T08:30:00.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:9 17.18 17:18000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5b400 con 0x5643ffa5a000 2024-03-29T08:30:00.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.7 v2:172.21.15.73:6816/27700 3 ==== osd_op_reply(9 [pgnls-filter start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x5643ffa5a000 2024-03-29T08:30:00.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f7324600640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5643ffa5bc00 0x5643ffa42b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:10 17.38 17:1c000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x56440058c400 con 0x5643ffa5bc00 2024-03-29T08:30:00.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f732a000640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5643ffa5bc00 0x5643ffa42b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f732a000640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5643ffa5bc00 0x5643ffa42b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 1 ==== osd_op_reply(10 [pgnls-filter start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f7325000640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5643ffa5a400 0x5643ffa43600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.429+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:11 17.4 17:20000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5a800 con 0x5643ffa5a400 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f732b400640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5643ffa5a400 0x5643ffa43600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f732b400640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5643ffa5a400 0x5643ffa43600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.1 v2:172.21.15.46:6824/27427 1 ==== osd_op_reply(11 [pgnls-filter start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa5a400 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f7324600640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x56440058c800 0x5643ffa43080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:12 17.24 17:24000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x564400465400 con 0x56440058c800 2024-03-29T08:30:00.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f732aa00640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x56440058c800 0x5643ffa43080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f732aa00640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x56440058c800 0x5643ffa43080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.5 v2:172.21.15.73:6808/27690 1 ==== osd_op_reply(12 [pgnls-filter start_epoch 125 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x56440058c800 2024-03-29T08:30:00.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:13 17.14 17:28000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x56440058dc00 con 0x56440058c800 2024-03-29T08:30:00.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.5 v2:172.21.15.73:6808/27690 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 125 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x56440058c800 2024-03-29T08:30:00.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:14 17.34 17:2c000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x564400465000 con 0x5643ffa5a000 2024-03-29T08:30:00.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.7 v2:172.21.15.73:6816/27700 4 ==== osd_op_reply(14 [pgnls-filter start_epoch 125 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x5643ffa5a000 2024-03-29T08:30:00.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:15 17.c 17:30000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5b000 con 0x5643ffa5bc00 2024-03-29T08:30:00.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 2 ==== osd_op_reply(15 [pgnls-filter start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:16 17.2c 17:34000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5b400 con 0x5643ffa5a400 2024-03-29T08:30:00.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.1 v2:172.21.15.46:6824/27427 2 ==== osd_op_reply(16 [pgnls-filter start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa5a400 2024-03-29T08:30:00.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:17 17.1c 17:38000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5a800 con 0x56440058c800 2024-03-29T08:30:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.5 v2:172.21.15.73:6808/27690 3 ==== osd_op_reply(17 [pgnls-filter start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x56440058c800 2024-03-29T08:30:00.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.433+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:18 17.3c 17:3c000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5ac00 con 0x56440058d800 2024-03-29T08:30:00.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.4 v2:172.21.15.73:6800/27689 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x56440058d800 2024-03-29T08:30:00.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:19 17.2 17:40000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ff95f400 con 0x5643ffa5a400 2024-03-29T08:30:00.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.1 v2:172.21.15.46:6824/27427 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa5a400 2024-03-29T08:30:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:20 17.22 17:44000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5b800 con 0x5643ffa5bc00 2024-03-29T08:30:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:21 17.12 17:48000000::::head [pgnls-filter start_epoch 125 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ff95f800 con 0x5643ffa5a000 2024-03-29T08:30:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.7 v2:172.21.15.73:6816/27700 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 125 out=81b] v125'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5644003dcfc0 con 0x5643ffa5a000 2024-03-29T08:30:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:22 17.0 17:00000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ff95fc00 con 0x5643ff95f000 2024-03-29T08:30:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.0 v2:172.21.15.46:6800/27424 4 ==== osd_op_reply(22 [pgnls-filter start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ff95f000 2024-03-29T08:30:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:23 17.20 17:04000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7a000 con 0x5643ffa5a000 2024-03-29T08:30:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.7 v2:172.21.15.73:6816/27700 6 ==== osd_op_reply(23 [pgnls-filter start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x5643ffa5a000 2024-03-29T08:30:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:24 17.10 17:08000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7a000 con 0x56440058d800 2024-03-29T08:30:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.4 v2:172.21.15.73:6800/27689 5 ==== osd_op_reply(24 [pgnls-filter start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x56440058d800 2024-03-29T08:30:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:25 17.30 17:0c000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7a400 con 0x5643ffa5a000 2024-03-29T08:30:00.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.7 v2:172.21.15.73:6816/27700 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x5643ffa5a000 2024-03-29T08:30:00.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:26 17.8 17:10000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ff95f800 con 0x5643ff95f000 2024-03-29T08:30:00.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.0 v2:172.21.15.46:6800/27424 5 ==== osd_op_reply(26 [pgnls-filter start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ff95f000 2024-03-29T08:30:00.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:27 17.28 17:14000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7a800 con 0x56440058d800 2024-03-29T08:30:00.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.4 v2:172.21.15.73:6800/27689 6 ==== osd_op_reply(27 [pgnls-filter start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x56440058d800 2024-03-29T08:30:00.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:28 17.18 17:18000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5a800 con 0x5643ffa5a000 2024-03-29T08:30:00.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.7 v2:172.21.15.73:6816/27700 8 ==== osd_op_reply(28 [pgnls-filter start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x5643ffa5a000 2024-03-29T08:30:00.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:29 17.38 17:1c000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7ac00 con 0x5643ffa5bc00 2024-03-29T08:30:00.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.437+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:30 17.4 17:20000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x564400465400 con 0x5643ffa5a400 2024-03-29T08:30:00.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.1 v2:172.21.15.46:6824/27427 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa5a400 2024-03-29T08:30:00.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:31 17.24 17:24000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7b000 con 0x56440058c800 2024-03-29T08:30:00.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.5 v2:172.21.15.73:6808/27690 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 125 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x56440058c800 2024-03-29T08:30:00.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:32 17.14 17:28000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x564400465000 con 0x56440058c800 2024-03-29T08:30:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.5 v2:172.21.15.73:6808/27690 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 125 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x56440058c800 2024-03-29T08:30:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:33 17.34 17:2c000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5b400 con 0x5643ffa5a000 2024-03-29T08:30:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.7 v2:172.21.15.73:6816/27700 9 ==== osd_op_reply(33 [pgnls-filter start_epoch 125 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x5643ffa5a000 2024-03-29T08:30:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:34 17.c 17:30000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x56440058dc00 con 0x5643ffa5bc00 2024-03-29T08:30:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 5 ==== osd_op_reply(34 [pgnls-filter start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:35 17.2c 17:34000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x56440058c400 con 0x5643ffa5a400 2024-03-29T08:30:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.1 v2:172.21.15.46:6824/27427 5 ==== osd_op_reply(35 [pgnls-filter start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa5a400 2024-03-29T08:30:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:36 17.1c 17:38000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7a000 con 0x56440058c800 2024-03-29T08:30:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.5 v2:172.21.15.73:6808/27690 6 ==== osd_op_reply(36 [pgnls-filter start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x56440058c800 2024-03-29T08:30:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:37 17.3c 17:3c000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x564400465400 con 0x56440058d800 2024-03-29T08:30:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.4 v2:172.21.15.73:6800/27689 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x56440058d800 2024-03-29T08:30:00.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:38 17.2 17:40000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7a800 con 0x5643ffa5a400 2024-03-29T08:30:00.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.1 v2:172.21.15.46:6824/27427 6 ==== osd_op_reply(38 [pgnls-filter start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa5a400 2024-03-29T08:30:00.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:39 17.22 17:44000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ff95f400 con 0x5643ffa5bc00 2024-03-29T08:30:00.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 6 ==== osd_op_reply(39 [pgnls-filter start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:40 17.12 17:48000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7b400 con 0x5643ffa5a000 2024-03-29T08:30:00.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.7 v2:172.21.15.73:6816/27700 10 ==== osd_op_reply(40 [pgnls-filter start_epoch 125 out=49b] v125'23 uv23 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x5643ffa5a000 2024-03-29T08:30:00.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:41 17.32 17:4c000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ff95fc00 con 0x5643ff95f000 2024-03-29T08:30:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.0 v2:172.21.15.46:6800/27424 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 125 out=49b] v121'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ff95f000 2024-03-29T08:30:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.441+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:42 17.a 17:50000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7b800 con 0x5643ffa5a400 2024-03-29T08:30:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.1 v2:172.21.15.46:6824/27427 7 ==== osd_op_reply(42 [pgnls-filter start_epoch 125 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa5a400 2024-03-29T08:30:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:43 17.2a 17:54000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ff95f800 con 0x56440058c800 2024-03-29T08:30:00.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.5 v2:172.21.15.73:6808/27690 7 ==== osd_op_reply(43 [pgnls-filter start_epoch 125 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x56440058c800 2024-03-29T08:30:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:44 17.1a 17:58000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5b000 con 0x56440058d800 2024-03-29T08:30:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.4 v2:172.21.15.73:6800/27689 8 ==== osd_op_reply(44 [pgnls-filter start_epoch 125 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x56440058d800 2024-03-29T08:30:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:45 17.3a 17:5c000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7a400 con 0x56440058c800 2024-03-29T08:30:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.5 v2:172.21.15.73:6808/27690 8 ==== osd_op_reply(45 [pgnls-filter start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x56440058c800 2024-03-29T08:30:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:46 17.6 17:60000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5ac00 con 0x5643ff95f000 2024-03-29T08:30:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.0 v2:172.21.15.46:6800/27424 7 ==== osd_op_reply(46 [pgnls-filter start_epoch 125 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ff95f000 2024-03-29T08:30:00.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:47 17.26 17:64000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7b000 con 0x56440058c800 2024-03-29T08:30:00.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.5 v2:172.21.15.73:6808/27690 9 ==== osd_op_reply(47 [pgnls-filter start_epoch 125 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x56440058c800 2024-03-29T08:30:00.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f7324600640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5643ffa5b800 0x5643ffa43b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:48 17.16 17:68000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7bc00 con 0x5643ffa5b800 2024-03-29T08:30:00.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f732a000640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5643ffa5b800 0x5643ffa43b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f732a000640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5643ffa5b800 0x5643ffa43b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.2 v2:172.21.15.46:6809/27428 1 ==== osd_op_reply(48 [pgnls-filter start_epoch 125 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5b800 2024-03-29T08:30:00.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:49 17.36 17:6c000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7a000 con 0x5643ffa5a000 2024-03-29T08:30:00.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.445+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.7 v2:172.21.15.73:6816/27700 11 ==== osd_op_reply(49 [pgnls-filter start_epoch 125 out=81b] v125'34 uv34 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5644003dcfc0 con 0x5643ffa5a000 2024-03-29T08:30:00.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-03-29T08:30:00.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 10 datascan.operator(): handling object 100000001f7.0 2024-03-29T08:30:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:50 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa8e000 con 0x5643ffa5a000 2024-03-29T08:30:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.7 v2:172.21.15.73:6816/27700 12 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) v8 ==== 416+0+194 (crc 0 0 0) 0x5644003dcfc0 con 0x5643ffa5a000 2024-03-29T08:30:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-03-29T08:30:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-03-29T08:30:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-03-29T08:30:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:51 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa8e400 con 0x5643ffa5bc00 2024-03-29T08:30:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 7 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-03-29T08:30:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5643ffa8e800 0x5643ff70b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:52 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa8ec00 con 0x5643ffa8e800 2024-03-29T08:30:00.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732b400640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5643ffa8e800 0x5643ff70b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732b400640 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5643ffa8e800 0x5643ff70b700 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.6 v2:172.21.15.73:6824/27691 1 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+524 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa8e800 2024-03-29T08:30:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-03-29T08:30:00.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-03-29T08:30:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 20 datascan.get_frag_of: frag is * 2024-03-29T08:30:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:53 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa8f000 con 0x5643ffa5bc00 2024-03-29T08:30:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 8 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 10 datascan.find_or_create_dirfrag: read_version = 7 2024-03-29T08:30:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-03-29T08:30:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:54 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa8f400 con 0x5643ffa5bc00 2024-03-29T08:30:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 9 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-03-29T08:30:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 20 datascan.read_dentry: bravo_head not found in result 2024-03-29T08:30:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-03-29T08:30:00.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.449+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:55 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa8f800 con 0x5643ffa5bc00 2024-03-29T08:30:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 10 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v125'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f732c9cd8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-03-29T08:30:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:56 17.e 17:70000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa8fc00 con 0x5643ffa8e800 2024-03-29T08:30:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.6 v2:172.21.15.73:6824/27691 2 ==== osd_op_reply(56 [pgnls-filter start_epoch 125 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa8e800 2024-03-29T08:30:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:57 17.2e 17:74000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7b400 con 0x5643ffa5bc00 2024-03-29T08:30:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 11 ==== osd_op_reply(57 [pgnls-filter start_epoch 125 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:58 17.1e 17:78000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7a800 con 0x5643ffa8e800 2024-03-29T08:30:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.6 v2:172.21.15.73:6824/27691 3 ==== osd_op_reply(58 [pgnls-filter start_epoch 125 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa8e800 2024-03-29T08:30:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:59 17.3e 17:7c000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5a800 con 0x5643ffa5b800 2024-03-29T08:30:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.2 v2:172.21.15.46:6809/27428 2 ==== osd_op_reply(59 [pgnls-filter start_epoch 125 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5b800 2024-03-29T08:30:00.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:60 17.1 17:80000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7b800 con 0x56440058c800 2024-03-29T08:30:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.5 v2:172.21.15.73:6808/27690 10 ==== osd_op_reply(60 [pgnls-filter start_epoch 125 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x56440058c800 2024-03-29T08:30:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:61 17.21 17:84000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5b400 con 0x5643ffa8e800 2024-03-29T08:30:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.6 v2:172.21.15.73:6824/27691 4 ==== osd_op_reply(61 [pgnls-filter start_epoch 125 out=49b] v121'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa8e800 2024-03-29T08:30:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:62 17.11 17:88000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5ac00 con 0x56440058d800 2024-03-29T08:30:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.4 v2:172.21.15.73:6800/27689 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x56440058d800 2024-03-29T08:30:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:63 17.31 17:8c000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x564400465000 con 0x5643ffa8e800 2024-03-29T08:30:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.6 v2:172.21.15.73:6824/27691 5 ==== osd_op_reply(63 [pgnls-filter start_epoch 125 out=49b] v121'15 uv8 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa8e800 2024-03-29T08:30:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:64 17.9 17:90000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5b000 con 0x5643ffa5a000 2024-03-29T08:30:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.7 v2:172.21.15.73:6816/27700 13 ==== osd_op_reply(64 [pgnls-filter start_epoch 125 out=49b] v121'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x5643ffa5a000 2024-03-29T08:30:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:65 17.29 17:94000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa8ec00 con 0x5643ff95f000 2024-03-29T08:30:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.0 v2:172.21.15.46:6800/27424 8 ==== osd_op_reply(65 [pgnls-filter start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ff95f000 2024-03-29T08:30:00.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:66 17.19 17:98000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ff95fc00 con 0x5643ffa5bc00 2024-03-29T08:30:00.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 12 ==== osd_op_reply(66 [pgnls-filter start_epoch 125 out=49b] v125'24 uv24 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.453+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:67 17.39 17:9c000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa8e400 con 0x5643ffa5bc00 2024-03-29T08:30:00.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 13 ==== osd_op_reply(67 [pgnls-filter start_epoch 125 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ff95f400 con 0x5643ff95f000 2024-03-29T08:30:00.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.0 v2:172.21.15.46:6800/27424 9 ==== osd_op_reply(68 [pgnls-filter start_epoch 125 out=49b] v125'15 uv15 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ff95f000 2024-03-29T08:30:00.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:69 17.25 17:a4000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7ac00 con 0x56440058d800 2024-03-29T08:30:00.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.4 v2:172.21.15.73:6800/27689 10 ==== osd_op_reply(69 [pgnls-filter start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x56440058d800 2024-03-29T08:30:00.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:70 17.15 17:a8000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x564400465400 con 0x5643ffa5bc00 2024-03-29T08:30:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 14 ==== osd_op_reply(70 [pgnls-filter start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:71 17.35 17:ac000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7bc00 con 0x5643ffa5bc00 2024-03-29T08:30:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 15 ==== osd_op_reply(71 [pgnls-filter start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:72 17.d 17:b0000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x56440058c400 con 0x5643ff95f000 2024-03-29T08:30:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.0 v2:172.21.15.46:6800/27424 10 ==== osd_op_reply(72 [pgnls-filter start_epoch 125 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ff95f000 2024-03-29T08:30:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:73 17.2d 17:b4000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ff95f400 con 0x5643ffa5a400 2024-03-29T08:30:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.1 v2:172.21.15.46:6824/27427 8 ==== osd_op_reply(73 [pgnls-filter start_epoch 125 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa5a400 2024-03-29T08:30:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:74 17.1d 17:b8000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x56440058dc00 con 0x5643ffa5a400 2024-03-29T08:30:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.1 v2:172.21.15.46:6824/27427 9 ==== osd_op_reply(74 [pgnls-filter start_epoch 125 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa5a400 2024-03-29T08:30:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:75 17.3d 17:bc000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ff95fc00 con 0x5643ffa5a400 2024-03-29T08:30:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.1 v2:172.21.15.46:6824/27427 10 ==== osd_op_reply(75 [pgnls-filter start_epoch 125 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa5a400 2024-03-29T08:30:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:76 17.3 17:c0000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa8fc00 con 0x5643ff95f000 2024-03-29T08:30:00.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.0 v2:172.21.15.46:6800/27424 11 ==== osd_op_reply(76 [pgnls-filter start_epoch 125 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ff95f000 2024-03-29T08:30:00.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:77 17.23 17:c4000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5b400 con 0x5643ff95f000 2024-03-29T08:30:00.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.0 v2:172.21.15.46:6800/27424 12 ==== osd_op_reply(77 [pgnls-filter start_epoch 125 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ff95f000 2024-03-29T08:30:00.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.457+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:78 17.13 17:c8000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa8ec00 con 0x56440058d800 2024-03-29T08:30:00.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.4 v2:172.21.15.73:6800/27689 11 ==== osd_op_reply(78 [pgnls-filter start_epoch 125 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x56440058d800 2024-03-29T08:30:00.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:79 17.33 17:cc000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5a800 con 0x5643ffa5b800 2024-03-29T08:30:00.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.2 v2:172.21.15.46:6809/27428 3 ==== osd_op_reply(79 [pgnls-filter start_epoch 125 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5b800 2024-03-29T08:30:00.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:80 17.b 17:d0000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa8f000 con 0x5643ffa8e800 2024-03-29T08:30:00.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.6 v2:172.21.15.73:6824/27691 6 ==== osd_op_reply(80 [pgnls-filter start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa8e800 2024-03-29T08:30:00.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:81 17.2b 17:d4000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5ac00 con 0x5643ffa5bc00 2024-03-29T08:30:00.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 16 ==== osd_op_reply(81 [pgnls-filter start_epoch 125 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:82 17.1b 17:d8000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa8f400 con 0x5643ffa5bc00 2024-03-29T08:30:00.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 17 ==== osd_op_reply(82 [pgnls-filter start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:83 17.3b 17:dc000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7a800 con 0x5643ffa5a000 2024-03-29T08:30:00.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.7 v2:172.21.15.73:6816/27700 14 ==== osd_op_reply(83 [pgnls-filter start_epoch 125 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x5643ffa5a000 2024-03-29T08:30:00.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:84 17.7 17:e0000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa8f800 con 0x56440058c800 2024-03-29T08:30:00.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 <== osd.5 v2:172.21.15.73:6808/27690 11 ==== osd_op_reply(84 [pgnls-filter start_epoch 125 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcfc0 con 0x56440058c800 2024-03-29T08:30:00.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:85 17.27 17:e4000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa7b400 con 0x5643ffa5bc00 2024-03-29T08:30:00.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.3 v2:172.21.15.46:6808/27426 18 ==== osd_op_reply(85 [pgnls-filter start_epoch 125 out=49b] v121'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5bc00 2024-03-29T08:30:00.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:86 17.17 17:e8000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa8e400 con 0x5643ffa5b800 2024-03-29T08:30:00.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f732a000640 1 -- 172.21.15.46:0/2843095975 <== osd.2 v2:172.21.15.46:6809/27428 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcb40 con 0x5643ffa5b800 2024-03-29T08:30:00.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:87 17.37 17:ec000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x56440058c400 con 0x5643ffa8e800 2024-03-29T08:30:00.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.6 v2:172.21.15.73:6824/27691 7 ==== osd_op_reply(87 [pgnls-filter start_epoch 125 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa8e800 2024-03-29T08:30:00.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:88 17.f 17:f0000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5a800 con 0x5643ffa8e800 2024-03-29T08:30:00.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.6 v2:172.21.15.73:6824/27691 8 ==== osd_op_reply(88 [pgnls-filter start_epoch 125 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa8e800 2024-03-29T08:30:00.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.461+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:89 17.2f 17:f4000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x564400465000 con 0x5643ffa5a400 2024-03-29T08:30:00.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.1 v2:172.21.15.46:6824/27427 11 ==== osd_op_reply(89 [pgnls-filter start_epoch 125 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa5a400 2024-03-29T08:30:00.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f7324600640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:90 17.1f 17:f8000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5ac00 con 0x5643ff95f000 2024-03-29T08:30:00.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.0 v2:172.21.15.46:6800/27424 13 ==== osd_op_reply(90 [pgnls-filter start_epoch 125 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ff95f000 2024-03-29T08:30:00.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f7325000640 1 -- 172.21.15.46:0/2843095975 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:91 17.3f 17:fc000000::::head [pgnls-filter start_epoch 125 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5643ffa5b400 con 0x5643ffa8e800 2024-03-29T08:30:00.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732b400640 1 -- 172.21.15.46:0/2843095975 <== osd.6 v2:172.21.15.73:6824/27691 9 ==== osd_op_reply(91 [pgnls-filter start_epoch 125 out=49b] v121'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5644003dcd80 con 0x5643ffa8e800 2024-03-29T08:30:00.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5643ff95f000 msgr2=0x5643ff70ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5643ff95f000 0x5643ff70ac00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5643ffa5a400 msgr2=0x5643ffa43600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5643ffa5a400 0x5643ffa43600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5643ffa5b800 msgr2=0x5643ffa43b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5643ffa5b800 0x5643ffa43b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5643ffa5bc00 msgr2=0x5643ffa42b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5643ffa5bc00 0x5643ffa42b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x56440058d800 msgr2=0x5643ff70a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x56440058d800 0x5643ff70a680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 reap_dead start 2024-03-29T08:30:00.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x56440058c800 msgr2=0x5643ffa43080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x56440058c800 0x5643ffa43080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5643ffa8e800 msgr2=0x5643ff70b700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5643ffa8e800 0x5643ff70b700 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5643ffa5a000 msgr2=0x5643ffa42580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5643ffa5a000 0x5643ffa42580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5643ff95ec00 msgr2=0x5643ffa42000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5643ff95ec00 0x5643ffa42000 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x5643ff749740 tx=0x5644004f80c0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56440058cc00 msgr2=0x5643ff708b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56440058cc00 0x5643ff708b00 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x5644005f33e0 tx=0x5644005a8e40 comp rx=0 tx=0).stop 2024-03-29T08:30:00.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732aa00640 1 -- 172.21.15.46:0/2843095975 reap_dead start 2024-03-29T08:30:00.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 shutdown_connections 2024-03-29T08:30:00.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/2843095975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56440058cc00 0x5643ff708b00 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 >> 172.21.15.46:0/2843095975 conn(0x5643ff67cc00 msgr2=0x5643ff6a9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 shutdown_connections 2024-03-29T08:30:00.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/2843095975 wait complete. 2024-03-29T08:30:00.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/3703594287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643ff67d400 msgr2=0x5643ff709600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/3703594287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643ff67d400 0x5643ff709600 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x5644004eb320 tx=0x5644004f8d20 comp rx=0 tx=0).stop 2024-03-29T08:30:00.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/3703594287 shutdown_connections 2024-03-29T08:30:00.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/3703594287 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5643ff67d000 0x5643ff709080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/3703594287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5643ff67d400 0x5643ff709600 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 --2- 172.21.15.46:0/3703594287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5643ff67d800 0x5643ff709b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/3703594287 >> 172.21.15.46:0/3703594287 conn(0x5643ff67dc00 msgr2=0x5643ff6a9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/3703594287 shutdown_connections 2024-03-29T08:30:00.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.465+0000 7f732c9cd8c0 1 -- 172.21.15.46:0/3703594287 wait complete. 2024-03-29T08:30:00.471 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.143897, result: 2024-03-29T08:30:00.471 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-03-29T08:30:00.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9fda68c0 1 -- start start 2024-03-29T08:30:00.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9fda68c0 1 Processor -- start 2024-03-29T08:30:00.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9fda68c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606a673b800 0x5606a67c6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9fda68c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606a673b400 0x5606a67c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9fda68c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a673b000 0x5606a67c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9fda68c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a6808a00 con 0x5606a673b000 2024-03-29T08:30:00.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9fda68c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a6808780 con 0x5606a673b800 2024-03-29T08:30:00.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9fda68c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606a6808640 con 0x5606a673b400 2024-03-29T08:30:00.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a673b000 0x5606a67c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9e800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a673b000 0x5606a67c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45102/0 (socket says 172.21.15.46:45102) 2024-03-29T08:30:00.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9e800640 1 -- 172.21.15.46:0/1190287026 learned_addr learned my addr 172.21.15.46:0/1190287026 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:00.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9de00640 1 --2- 172.21.15.46:0/1190287026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606a673b800 0x5606a67c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9d400640 1 --2- 172.21.15.46:0/1190287026 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606a673b400 0x5606a67c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9e800640 1 -- 172.21.15.46:0/1190287026 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606a673b400 msgr2=0x5606a67c6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9e800640 1 --2- 172.21.15.46:0/1190287026 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606a673b400 0x5606a67c6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9e800640 1 -- 172.21.15.46:0/1190287026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606a673b800 msgr2=0x5606a67c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9e800640 1 --2- 172.21.15.46:0/1190287026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606a673b800 0x5606a67c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9e800640 1 -- 172.21.15.46:0/1190287026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606a676e9c0 con 0x5606a673b000 2024-03-29T08:30:00.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9e800640 1 --2- 172.21.15.46:0/1190287026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a673b000 0x5606a67c7080 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x5606a75a09f0 tx=0x5606a7644120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2ded4a11b4d87d server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9ac00640 1 -- 172.21.15.46:0/1190287026 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5606a678f080 con 0x5606a673b000 2024-03-29T08:30:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9ac00640 1 -- 172.21.15.46:0/1190287026 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5606a7510300 con 0x5606a673b000 2024-03-29T08:30:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9ac00640 1 -- 172.21.15.46:0/1190287026 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5606a678f600 con 0x5606a673b000 2024-03-29T08:30:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/1190287026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606a676e1a0 con 0x5606a673b000 2024-03-29T08:30:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/1190287026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5606a676e9c0 con 0x5606a673b000 2024-03-29T08:30:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9fda68c0 4 waiting for MDS map... 2024-03-29T08:30:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9ac00640 1 -- 172.21.15.46:0/1190287026 <== mon.0 v2:172.21.15.46:3300/0 4 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x5606a677ac40 con 0x5606a673b000 2024-03-29T08:30:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9ac00640 1 -- 172.21.15.46:0/1190287026 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 101) v1 ==== 2573+0+0 (secure 0 0 0) 0x5606a6788f00 con 0x5606a673b000 2024-03-29T08:30:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.513+0000 7f3b9fda68c0 4 Got MDS map 101 2024-03-29T08:30:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 4 datascan.main: Using metadata pool output 2024-03-29T08:30:00.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 4 datascan.main: connecting to RADOS... 2024-03-29T08:30:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 Processor -- start 2024-03-29T08:30:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 -- start start 2024-03-29T08:30:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a7513800 0x5606a67c7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606a7513c00 0x5606a67c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606a76b2000 0x5606a67c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a6781e00 con 0x5606a7513800 2024-03-29T08:30:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a6808780 con 0x5606a76b2000 2024-03-29T08:30:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606a6808a00 con 0x5606a7513c00 2024-03-29T08:30:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a7513800 0x5606a67c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a7513800 0x5606a67c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45116/0 (socket says 172.21.15.46:45116) 2024-03-29T08:30:00.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9d400640 1 -- 172.21.15.46:0/2736081540 learned_addr learned my addr 172.21.15.46:0/2736081540 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:00.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9de00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606a7513c00 0x5606a67c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9d400640 1 --2- 172.21.15.46:0/2736081540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606a76b2000 0x5606a67c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9d400640 1 -- 172.21.15.46:0/2736081540 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606a7513c00 msgr2=0x5606a67c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9d400640 1 --2- 172.21.15.46:0/2736081540 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606a7513c00 0x5606a67c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9d400640 1 -- 172.21.15.46:0/2736081540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606a76b2000 msgr2=0x5606a67c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9d400640 1 --2- 172.21.15.46:0/2736081540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606a76b2000 0x5606a67c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9d400640 1 -- 172.21.15.46:0/2736081540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606a676eb60 con 0x5606a7513800 2024-03-29T08:30:00.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9d400640 1 --2- 172.21.15.46:0/2736081540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a7513800 0x5606a67c7600 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x5606a7640930 tx=0x5606a7662c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d2c4dfb3ac4449 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b98400640 1 -- 172.21.15.46:0/2736081540 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5606a678f080 con 0x5606a7513800 2024-03-29T08:30:00.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b98400640 1 -- 172.21.15.46:0/2736081540 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5606a7510a80 con 0x5606a7513800 2024-03-29T08:30:00.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b98400640 1 -- 172.21.15.46:0/2736081540 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5606a678f600 con 0x5606a7513800 2024-03-29T08:30:00.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/2736081540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a7513800 msgr2=0x5606a67c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/2736081540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a7513800 0x5606a67c7600 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x5606a7640930 tx=0x5606a7662c60 comp rx=0 tx=0).stop 2024-03-29T08:30:00.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/2736081540 shutdown_connections 2024-03-29T08:30:00.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/2736081540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606a76b2000 0x5606a67c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/2736081540 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606a7513c00 0x5606a67c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/2736081540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a7513800 0x5606a67c7600 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/2736081540 >> 172.21.15.46:0/2736081540 conn(0x5606a673ac00 msgr2=0x5606a6767800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/2736081540 shutdown_connections 2024-03-29T08:30:00.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/2736081540 wait complete. 2024-03-29T08:30:00.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.517+0000 7f3b9fda68c0 1 Processor -- start 2024-03-29T08:30:00.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9fda68c0 1 -- start start 2024-03-29T08:30:00.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9fda68c0 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a7513c00 0x5606a67c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9fda68c0 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606a7513800 0x5606a67c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9fda68c0 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606a76b3800 0x5606a67c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9fda68c0 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a6808500 con 0x5606a7513c00 2024-03-29T08:30:00.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9fda68c0 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606a68083c0 con 0x5606a7513800 2024-03-29T08:30:00.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9fda68c0 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606a6808280 con 0x5606a76b3800 2024-03-29T08:30:00.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a7513c00 0x5606a67c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a7513c00 0x5606a67c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45126/0 (socket says 172.21.15.46:45126) 2024-03-29T08:30:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 learned_addr learned my addr 172.21.15.46:0/583952672 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9de00640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606a7513800 0x5606a67c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9d400640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606a76b3800 0x5606a67c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606a76b3800 msgr2=0x5606a67c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9d400640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606a76b3800 0x5606a67c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606a7513800 msgr2=0x5606a67c8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9d400640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606a7513800 0x5606a67c8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606a676e340 con 0x5606a7513c00 2024-03-29T08:30:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9de00640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606a7513800 0x5606a67c8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9d400640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a7513c00 0x5606a67c9180 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x5606a76415f0 tx=0x5606a7662cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b96600640 1 -- 172.21.15.46:0/583952672 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5606a678edc0 con 0x5606a7513c00 2024-03-29T08:30:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b96600640 1 -- 172.21.15.46:0/583952672 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5606a7511380 con 0x5606a7513c00 2024-03-29T08:30:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5606a676e000 con 0x5606a7513c00 2024-03-29T08:30:00.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b96600640 1 -- 172.21.15.46:0/583952672 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5606a678f080 con 0x5606a7513c00 2024-03-29T08:30:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606a676eea0 con 0x5606a7513c00 2024-03-29T08:30:00.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b9fda68c0 4 datascan.init: resolving metadata pool 16 2024-03-29T08:30:00.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.521+0000 7f3b96600640 1 -- 172.21.15.46:0/583952672 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x5606a6772840 con 0x5606a7513c00 2024-03-29T08:30:00.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b96600640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5606a6a22800 0x5606a67c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b96600640 1 -- 172.21.15.46:0/583952672 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x5606a677aa80 con 0x5606a7513c00 2024-03-29T08:30:00.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b9fda68c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-03-29T08:30:00.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b9fda68c0 4 datascan.main: resolving metadata pool 16 2024-03-29T08:30:00.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x5606a76b3400 0x5606a67c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:1 16.0 16:00000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a76b3000 con 0x5606a76b3400 2024-03-29T08:30:00.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b9de00640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5606a6a22800 0x5606a67c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b9d400640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x5606a76b3400 0x5606a67c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b9de00640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5606a6a22800 0x5606a67c9700 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x5606a684e6f0 tx=0x5606a7663da0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b9d400640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x5606a76b3400 0x5606a67c8100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.5 v2:172.21.15.73:6808/27690 1 ==== osd_op_reply(1 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ab40 con 0x5606a76b3400 2024-03-29T08:30:00.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b9d400640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5606a76b3000 0x5606a6b18000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:2 16.10 16:08000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23800 con 0x5606a76b3000 2024-03-29T08:30:00.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b9e800640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5606a76b3000 0x5606a6b18000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b9e800640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5606a76b3000 0x5606a6b18000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 1 ==== osd_op_reply(2 [pgnls start_epoch 125 out=177b] v124'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b9e800640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x5606a6a23800 0x5606a6b18580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:3 16.8 16:10000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a7513400 con 0x5606a6a23800 2024-03-29T08:30:00.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.525+0000 7f3b9de00640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x5606a6a23800 0x5606a6b18580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9de00640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x5606a6a23800 0x5606a6b18580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 1 ==== osd_op_reply(3 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9de00640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5606a7513400 0x5606a6b18b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:4 16.18 16:18000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23400 con 0x5606a7513400 2024-03-29T08:30:00.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9d400640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5606a7513400 0x5606a6b18b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9d400640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5606a7513400 0x5606a6b18b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 1 ==== osd_op_reply(4 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:5 16.4 16:20000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23400 con 0x5606a6a23800 2024-03-29T08:30:00.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 2 ==== osd_op_reply(5 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:6 16.14 16:28000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23400 con 0x5606a6a23800 2024-03-29T08:30:00.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 3 ==== osd_op_reply(6 [pgnls start_epoch 125 out=149b] v125'42 uv42 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9de00640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5606a6a23400 0x5606a6b19080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:7 16.c 16:30000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b28000 con 0x5606a6a23400 2024-03-29T08:30:00.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9e800640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5606a6a23400 0x5606a6b19080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9e800640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5606a6a23400 0x5606a6b19080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.0 v2:172.21.15.46:6800/27424 1 ==== osd_op_reply(7 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ad80 con 0x5606a6a23400 2024-03-29T08:30:00.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:8 16.1c 16:38000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b28000 con 0x5606a6a23800 2024-03-29T08:30:00.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 4 ==== osd_op_reply(8 [pgnls start_epoch 125 out=73b] v124'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.529+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:9 16.2 16:40000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b28000 con 0x5606a6a23400 2024-03-29T08:30:00.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.0 v2:172.21.15.46:6800/27424 2 ==== osd_op_reply(9 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ad80 con 0x5606a6a23400 2024-03-29T08:30:00.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9e800640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5606a6b28000 0x5606a6b19b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:10 16.12 16:48000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b28c00 con 0x5606a6b28000 2024-03-29T08:30:00.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9de00640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5606a6b28000 0x5606a6b19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9de00640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5606a6b28000 0x5606a6b19b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.1 v2:172.21.15.46:6824/27427 1 ==== osd_op_reply(10 [pgnls start_epoch 125 out=81b] v124'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6b28000 2024-03-29T08:30:00.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:11 16.a 16:50000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b28c00 con 0x5606a76b3000 2024-03-29T08:30:00.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 2 ==== osd_op_reply(11 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:12 16.1a 16:58000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b28c00 con 0x5606a76b3000 2024-03-29T08:30:00.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 3 ==== osd_op_reply(12 [pgnls start_epoch 125 out=49b] v125'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9e800640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5606a6b28c00 0x5606a6b1a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:13 16.6 16:60000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29000 con 0x5606a6b28c00 2024-03-29T08:30:00.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9d400640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5606a6b28c00 0x5606a6b1a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9d400640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5606a6b28c00 0x5606a6b1a100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.3 v2:172.21.15.46:6808/27426 1 ==== osd_op_reply(13 [pgnls start_epoch 125 out=77b] v124'8 uv8 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5606a763ab40 con 0x5606a6b28c00 2024-03-29T08:30:00.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:14 16.16 16:68000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29000 con 0x5606a7513400 2024-03-29T08:30:00.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 2 ==== osd_op_reply(14 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:15 16.e 16:70000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29000 con 0x5606a7513400 2024-03-29T08:30:00.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 3 ==== osd_op_reply(15 [pgnls start_epoch 125 out=73b] v124'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9d400640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5606a6b29000 0x5606a6b1a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:16 16.1e 16:78000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a6b29000 2024-03-29T08:30:00.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.533+0000 7f3b9e800640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5606a6b29000 0x5606a6b1a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9e800640 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5606a6b29000 0x5606a6b1a680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.2 v2:172.21.15.46:6809/27428 1 ==== osd_op_reply(16 [pgnls start_epoch 125 out=49b] v125'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ad80 con 0x5606a6b29000 2024-03-29T08:30:00.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:17 16.1 16:80000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a76b3000 2024-03-29T08:30:00.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 4 ==== osd_op_reply(17 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:18 16.11 16:88000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a6b29000 2024-03-29T08:30:00.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.2 v2:172.21.15.46:6809/27428 2 ==== osd_op_reply(18 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ad80 con 0x5606a6b29000 2024-03-29T08:30:00.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:19 16.9 16:90000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a6a23800 2024-03-29T08:30:00.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 5 ==== osd_op_reply(19 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:20 16.19 16:98000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a6b28c00 2024-03-29T08:30:00.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.3 v2:172.21.15.46:6808/27426 2 ==== osd_op_reply(20 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ab40 con 0x5606a6b28c00 2024-03-29T08:30:00.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:21 16.5 16:a0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a6a23800 2024-03-29T08:30:00.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 6 ==== osd_op_reply(21 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:22 16.15 16:a8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a7513400 2024-03-29T08:30:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 4 ==== osd_op_reply(22 [pgnls start_epoch 125 out=97b] v124'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:23 16.d 16:b0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a6a23400 2024-03-29T08:30:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.0 v2:172.21.15.46:6800/27424 3 ==== osd_op_reply(23 [pgnls start_epoch 125 out=74b] v120'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5606a763ad80 con 0x5606a6a23400 2024-03-29T08:30:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:24 16.1d 16:b8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a6a23800 2024-03-29T08:30:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 7 ==== osd_op_reply(24 [pgnls start_epoch 125 out=73b] v124'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.537+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:25 16.3 16:c0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a6b28c00 2024-03-29T08:30:00.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.3 v2:172.21.15.46:6808/27426 3 ==== osd_op_reply(25 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ab40 con 0x5606a6b28c00 2024-03-29T08:30:00.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:26 16.13 16:c8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a7513400 2024-03-29T08:30:00.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 5 ==== osd_op_reply(26 [pgnls start_epoch 125 out=98b] v124'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:27 16.b 16:d0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a7513400 2024-03-29T08:30:00.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 6 ==== osd_op_reply(27 [pgnls start_epoch 125 out=76b] v125'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:28 16.1b 16:d8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a6b28000 2024-03-29T08:30:00.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.1 v2:172.21.15.46:6824/27427 2 ==== osd_op_reply(28 [pgnls start_epoch 125 out=49b] v125'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6b28000 2024-03-29T08:30:00.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:29 16.7 16:e0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a6b28c00 2024-03-29T08:30:00.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.3 v2:172.21.15.46:6808/27426 4 ==== osd_op_reply(29 [pgnls start_epoch 125 out=105b] v125'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x5606a763ab40 con 0x5606a6b28c00 2024-03-29T08:30:00.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:30 16.17 16:e8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a6a23800 2024-03-29T08:30:00.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 8 ==== osd_op_reply(30 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:31 16.f 16:f0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a76b3400 2024-03-29T08:30:00.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.5 v2:172.21.15.73:6808/27690 2 ==== osd_op_reply(31 [pgnls start_epoch 125 out=73b] v125'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5606a763ab40 con 0x5606a76b3400 2024-03-29T08:30:00.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:32 16.1f 16:f8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a6a23400 2024-03-29T08:30:00.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.0 v2:172.21.15.46:6800/27424 4 ==== osd_op_reply(32 [pgnls start_epoch 125 out=101b] v125'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5606a763ad80 con 0x5606a6a23400 2024-03-29T08:30:00.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-03-29T08:30:00.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:33 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a76b2c00 con 0x5606a76b3000 2024-03-29T08:30:00.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-03-29T08:30:00.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:34 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a76b2800 con 0x5606a76b3000 2024-03-29T08:30:00.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-03-29T08:30:00.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:35 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a76b2400 con 0x5606a76b3000 2024-03-29T08:30:00.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-03-29T08:30:00.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.541+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:36 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a76b2000 con 0x5606a76b3000 2024-03-29T08:30:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-03-29T08:30:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:37 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29800 con 0x5606a76b3000 2024-03-29T08:30:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-03-29T08:30:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-29T08:30:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-03-29T08:30:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-29T08:30:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-03-29T08:30:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-29T08:30:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-03-29T08:30:00.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:38 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29400 con 0x5606a6a23800 2024-03-29T08:30:00.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 9 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv37 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-03-29T08:30:00.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:39 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6a23800 2024-03-29T08:30:00.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 10 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-03-29T08:30:00.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:40 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a76b2c00 con 0x5606a6b28000 2024-03-29T08:30:00.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.1 v2:172.21.15.46:6824/27427 3 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+523 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6b28000 2024-03-29T08:30:00.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-03-29T08:30:00.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-29T08:30:00.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-03-29T08:30:00.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:41 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a76b2800 con 0x5606a7513400 2024-03-29T08:30:00.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-03-29T08:30:00.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:42 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a76b2400 con 0x5606a7513400 2024-03-29T08:30:00.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 8 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-03-29T08:30:00.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-29T08:30:00.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-03-29T08:30:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-29T08:30:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-03-29T08:30:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:43 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6a23c00 con 0x5606a6a23800 2024-03-29T08:30:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 11 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-03-29T08:30:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-29T08:30:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-03-29T08:30:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:44 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a76b2c00 con 0x5606a7513400 2024-03-29T08:30:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 9 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-03-29T08:30:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-29T08:30:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-03-29T08:30:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.545+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:45 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b28800 con 0x5606a6b28c00 2024-03-29T08:30:00.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.3 v2:172.21.15.46:6808/27426 5 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763ab40 con 0x5606a6b28c00 2024-03-29T08:30:00.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-03-29T08:30:00.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:46 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b28400 con 0x5606a6b28c00 2024-03-29T08:30:00.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.3 v2:172.21.15.46:6808/27426 6 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1545 (crc 0 0 0) 0x5606a763ab40 con 0x5606a6b28c00 2024-03-29T08:30:00.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9fda68c0 20 datascan.scan_links: injected ino detected 2024-03-29T08:30:00.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-03-29T08:30:00.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-29T08:30:00.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-03-29T08:30:00.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-29T08:30:00.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9fda68c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-03-29T08:30:00.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:47 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29400 con 0x5606a6a23400 2024-03-29T08:30:00.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.0 v2:172.21.15.46:6800/27424 5 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x5606a763ad80 con 0x5606a6a23400 2024-03-29T08:30:00.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:48 16.0 16:00000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a76b3400 2024-03-29T08:30:00.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.5 v2:172.21.15.73:6808/27690 3 ==== osd_op_reply(48 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ab40 con 0x5606a76b3400 2024-03-29T08:30:00.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:49 16.10 16:08000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a76b3000 2024-03-29T08:30:00.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 10 ==== osd_op_reply(49 [pgnls start_epoch 125 out=177b] v124'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:50 16.8 16:10000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6a23800 2024-03-29T08:30:00.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 12 ==== osd_op_reply(50 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:51 16.18 16:18000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a7513400 2024-03-29T08:30:00.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 10 ==== osd_op_reply(51 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:52 16.4 16:20000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6a23800 2024-03-29T08:30:00.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 13 ==== osd_op_reply(52 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:53 16.14 16:28000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6a23800 2024-03-29T08:30:00.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 14 ==== osd_op_reply(53 [pgnls start_epoch 125 out=149b] v125'42 uv42 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:54 16.c 16:30000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6a23400 2024-03-29T08:30:00.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.0 v2:172.21.15.46:6800/27424 6 ==== osd_op_reply(54 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ad80 con 0x5606a6a23400 2024-03-29T08:30:00.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.549+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:55 16.1c 16:38000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6a23800 2024-03-29T08:30:00.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 15 ==== osd_op_reply(55 [pgnls start_epoch 125 out=73b] v124'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:56 16.2 16:40000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6a23400 2024-03-29T08:30:00.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.0 v2:172.21.15.46:6800/27424 7 ==== osd_op_reply(56 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ad80 con 0x5606a6a23400 2024-03-29T08:30:00.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:57 16.12 16:48000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6b28000 2024-03-29T08:30:00.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.1 v2:172.21.15.46:6824/27427 4 ==== osd_op_reply(57 [pgnls start_epoch 125 out=81b] v124'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6b28000 2024-03-29T08:30:00.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:58 16.a 16:50000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a76b3000 2024-03-29T08:30:00.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 11 ==== osd_op_reply(58 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:59 16.1a 16:58000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a76b3000 2024-03-29T08:30:00.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 12 ==== osd_op_reply(59 [pgnls start_epoch 125 out=49b] v125'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:60 16.6 16:60000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6b28c00 2024-03-29T08:30:00.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.3 v2:172.21.15.46:6808/27426 7 ==== osd_op_reply(60 [pgnls start_epoch 125 out=77b] v124'8 uv8 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5606a763ab40 con 0x5606a6b28c00 2024-03-29T08:30:00.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:61 16.16 16:68000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a7513400 2024-03-29T08:30:00.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 11 ==== osd_op_reply(61 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:62 16.e 16:70000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a7513400 2024-03-29T08:30:00.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 12 ==== osd_op_reply(62 [pgnls start_epoch 125 out=73b] v124'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:63 16.1e 16:78000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6b29000 2024-03-29T08:30:00.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.2 v2:172.21.15.46:6809/27428 3 ==== osd_op_reply(63 [pgnls start_epoch 125 out=49b] v125'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ad80 con 0x5606a6b29000 2024-03-29T08:30:00.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:64 16.1 16:80000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a76b3000 2024-03-29T08:30:00.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 13 ==== osd_op_reply(64 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- osd_op(unknown.0.0:65 16.11 16:88000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6b29000 2024-03-29T08:30:00.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.2 v2:172.21.15.46:6809/27428 4 ==== osd_op_reply(65 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ad80 con 0x5606a6b29000 2024-03-29T08:30:00.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:66 16.9 16:90000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6a23800 2024-03-29T08:30:00.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 16 ==== osd_op_reply(66 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.553+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:67 16.19 16:98000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6b28c00 2024-03-29T08:30:00.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.3 v2:172.21.15.46:6808/27426 8 ==== osd_op_reply(67 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ab40 con 0x5606a6b28c00 2024-03-29T08:30:00.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:68 16.5 16:a0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6a23800 2024-03-29T08:30:00.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 17 ==== osd_op_reply(68 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:69 16.15 16:a8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a7513400 2024-03-29T08:30:00.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 13 ==== osd_op_reply(69 [pgnls start_epoch 125 out=97b] v124'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:70 16.d 16:b0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6a23400 2024-03-29T08:30:00.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.0 v2:172.21.15.46:6800/27424 8 ==== osd_op_reply(70 [pgnls start_epoch 125 out=74b] v120'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5606a763ad80 con 0x5606a6a23400 2024-03-29T08:30:00.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:71 16.1d 16:b8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6a23800 2024-03-29T08:30:00.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 18 ==== osd_op_reply(71 [pgnls start_epoch 125 out=73b] v124'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:72 16.3 16:c0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6b28c00 2024-03-29T08:30:00.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.3 v2:172.21.15.46:6808/27426 9 ==== osd_op_reply(72 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763ab40 con 0x5606a6b28c00 2024-03-29T08:30:00.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:73 16.13 16:c8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a7513400 2024-03-29T08:30:00.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 14 ==== osd_op_reply(73 [pgnls start_epoch 125 out=98b] v124'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:74 16.b 16:d0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a7513400 2024-03-29T08:30:00.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 15 ==== osd_op_reply(74 [pgnls start_epoch 125 out=76b] v125'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:75 16.1b 16:d8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6b28000 2024-03-29T08:30:00.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.1 v2:172.21.15.46:6824/27427 5 ==== osd_op_reply(75 [pgnls start_epoch 125 out=49b] v125'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6b28000 2024-03-29T08:30:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:76 16.7 16:e0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6b28c00 2024-03-29T08:30:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.3 v2:172.21.15.46:6808/27426 10 ==== osd_op_reply(76 [pgnls start_epoch 125 out=105b] v125'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x5606a763ab40 con 0x5606a6b28c00 2024-03-29T08:30:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:77 16.17 16:e8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6a23800 2024-03-29T08:30:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 19 ==== osd_op_reply(77 [pgnls start_epoch 125 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.557+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- osd_op(unknown.0.0:78 16.f 16:f0000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a76b3400 2024-03-29T08:30:00.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.5 v2:172.21.15.73:6808/27690 4 ==== osd_op_reply(78 [pgnls start_epoch 125 out=73b] v125'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5606a763ab40 con 0x5606a76b3400 2024-03-29T08:30:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:79 16.1f 16:f8000000::::head [pgnls start_epoch 125 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a6a23400 2024-03-29T08:30:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.0 v2:172.21.15.46:6800/27424 9 ==== osd_op_reply(79 [pgnls start_epoch 125 out=101b] v125'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5606a763ad80 con 0x5606a6a23400 2024-03-29T08:30:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-03-29T08:30:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:80 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b28800 con 0x5606a76b3000 2024-03-29T08:30:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 14 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-03-29T08:30:00.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:81 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b28400 con 0x5606a76b3000 2024-03-29T08:30:00.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 15 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-03-29T08:30:00.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:82 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a76b2800 con 0x5606a76b3000 2024-03-29T08:30:00.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 16 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-03-29T08:30:00.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:83 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a76b2400 con 0x5606a76b3000 2024-03-29T08:30:00.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 17 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-03-29T08:30:00.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- osd_op(unknown.0.0:84 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a76b2c00 con 0x5606a76b3000 2024-03-29T08:30:00.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.6 v2:172.21.15.73:6824/27691 18 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763ad80 con 0x5606a76b3000 2024-03-29T08:30:00.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-03-29T08:30:00.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-03-29T08:30:00.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-03-29T08:30:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-03-29T08:30:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-03-29T08:30:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-03-29T08:30:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-03-29T08:30:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:85 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a76b2000 con 0x5606a6a23800 2024-03-29T08:30:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 20 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv37 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-03-29T08:30:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:86 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a76b2800 con 0x5606a6a23800 2024-03-29T08:30:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 21 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-03-29T08:30:00.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- osd_op(unknown.0.0:87 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29800 con 0x5606a6b28000 2024-03-29T08:30:00.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.1 v2:172.21.15.46:6824/27427 6 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+523 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6b28000 2024-03-29T08:30:00.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-03-29T08:30:00.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-03-29T08:30:00.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-03-29T08:30:00.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:88 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29400 con 0x5606a7513400 2024-03-29T08:30:00.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 16 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-03-29T08:30:00.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.561+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:89 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b29c00 con 0x5606a7513400 2024-03-29T08:30:00.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 17 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-03-29T08:30:00.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-03-29T08:30:00.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-03-29T08:30:00.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-03-29T08:30:00.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-03-29T08:30:00.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- osd_op(unknown.0.0:90 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b28800 con 0x5606a6a23800 2024-03-29T08:30:00.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 <== osd.4 v2:172.21.15.73:6800/27689 22 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763afc0 con 0x5606a6a23800 2024-03-29T08:30:00.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-03-29T08:30:00.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-03-29T08:30:00.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-03-29T08:30:00.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:91 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b28400 con 0x5606a7513400 2024-03-29T08:30:00.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 18 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-03-29T08:30:00.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-03-29T08:30:00.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-03-29T08:30:00.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:92 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b4e000 con 0x5606a6b28c00 2024-03-29T08:30:00.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.3 v2:172.21.15.46:6808/27426 11 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5606a763ab40 con 0x5606a6b28c00 2024-03-29T08:30:00.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-03-29T08:30:00.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:93 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b4e400 con 0x5606a6b28c00 2024-03-29T08:30:00.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.3 v2:172.21.15.46:6808/27426 12 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1545 (crc 0 0 0) 0x5606a763ab40 con 0x5606a6b28c00 2024-03-29T08:30:00.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 20 datascan.scan_links: injected ino detected 2024-03-29T08:30:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-03-29T08:30:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-03-29T08:30:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-03-29T08:30:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-03-29T08:30:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-03-29T08:30:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-03-29T08:30:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:94 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b4e800 con 0x5606a6a23400 2024-03-29T08:30:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.0 v2:172.21.15.46:6800/27424 10 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x5606a763ad80 con 0x5606a6a23400 2024-03-29T08:30:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-03-29T08:30:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-03-29T08:30:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-03-29T08:30:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: processing 1 injected_inos 2024-03-29T08:30:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-03-29T08:30:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-03-29T08:30:00.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:95 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b4ec00 con 0x5606a6b28c00 2024-03-29T08:30:00.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.3 v2:172.21.15.46:6808/27426 13 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x5606a763ab40 con 0x5606a6b28c00 2024-03-29T08:30:00.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-03-29T08:30:00.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 20 datascan.scan_links: first is now 2 2024-03-29T08:30:00.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- osd_op(unknown.0.0:96 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b4f000 con 0x5606a6b28c00 2024-03-29T08:30:00.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.3 v2:172.21.15.46:6808/27426 14 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v125'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5606a763ab40 con 0x5606a6b28c00 2024-03-29T08:30:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-03-29T08:30:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 datascan.scan_links: updating inotable 2024-03-29T08:30:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:97 16.13 16:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b4f400 con 0x5606a7513400 2024-03-29T08:30:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 19 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-03-29T08:30:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-03-29T08:30:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.565+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- osd_op(unknown.0.0:98 16.13 16:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b4f800 con 0x5606a7513400 2024-03-29T08:30:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9d400640 1 -- 172.21.15.46:0/583952672 <== osd.7 v2:172.21.15.73:6816/27700 20 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v125'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x5606a763ab40 con 0x5606a7513400 2024-03-29T08:30:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 10 datascan.scan_links: updating snaptable 2024-03-29T08:30:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- osd_op(unknown.0.0:99 16.d 16:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e125) v8 -- 0x5606a6b4fc00 con 0x5606a6a23400 2024-03-29T08:30:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9e800640 1 -- 172.21.15.46:0/583952672 <== osd.0 v2:172.21.15.46:6800/27424 11 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x5606a763ad80 con 0x5606a6a23400 2024-03-29T08:30:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5606a6a23400 msgr2=0x5606a6b19080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x5606a6a23400 0x5606a6b19080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5606a6b28000 msgr2=0x5606a6b19b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x5606a6b28000 0x5606a6b19b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5606a6b29000 msgr2=0x5606a6b1a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x5606a6b29000 0x5606a6b1a680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5606a6b28c00 msgr2=0x5606a6b1a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x5606a6b28c00 0x5606a6b1a100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x5606a6a23800 msgr2=0x5606a6b18580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x5606a6a23800 0x5606a6b18580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x5606a76b3400 msgr2=0x5606a67c8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x5606a76b3400 0x5606a67c8100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5606a76b3000 msgr2=0x5606a6b18000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x5606a76b3000 0x5606a6b18000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5606a7513400 msgr2=0x5606a6b18b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x5606a7513400 0x5606a6b18b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 reap_dead start 2024-03-29T08:30:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 reap_dead start 2024-03-29T08:30:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 reap_dead start 2024-03-29T08:30:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9de00640 1 -- 172.21.15.46:0/583952672 reap_dead start 2024-03-29T08:30:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5606a6a22800 msgr2=0x5606a67c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5606a6a22800 0x5606a67c9700 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x5606a684e6f0 tx=0x5606a7663da0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a7513c00 msgr2=0x5606a67c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a7513c00 0x5606a67c9180 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x5606a76415f0 tx=0x5606a7662cc0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 shutdown_connections 2024-03-29T08:30:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x5606a6a22800 0x5606a67c9700 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/583952672 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a7513c00 0x5606a67c9180 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 >> 172.21.15.46:0/583952672 conn(0x5606a673ac00 msgr2=0x5606a6767800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 shutdown_connections 2024-03-29T08:30:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/583952672 wait complete. 2024-03-29T08:30:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/1190287026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a673b000 msgr2=0x5606a67c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/1190287026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a673b000 0x5606a67c7080 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x5606a75a09f0 tx=0x5606a7644120 comp rx=0 tx=0).stop 2024-03-29T08:30:00.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/1190287026 shutdown_connections 2024-03-29T08:30:00.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/1190287026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606a673b000 0x5606a67c7080 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/1190287026 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606a673b400 0x5606a67c6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 --2- 172.21.15.46:0/1190287026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606a673b800 0x5606a67c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/1190287026 >> 172.21.15.46:0/1190287026 conn(0x5606a673bc00 msgr2=0x5606a6767000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/1190287026 shutdown_connections 2024-03-29T08:30:00.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.569+0000 7f3b9fda68c0 1 -- 172.21.15.46:0/1190287026 wait complete. 2024-03-29T08:30:00.576 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.105069, result: 2024-03-29T08:30:00.577 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T08:30:00.622 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T08:30:00.662 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-03-29T08:30:00.662 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-03-29T08:30:00.662 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T08:30:00.662 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-03-29T08:30:00.662 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.663 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-03-29T08:30:00.663 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.663 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-03-29T08:30:00.663 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.663 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-03-29T08:30:00.663 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-03-29T08:30:00.663 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.663 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-03-29T08:30:00.663 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-03-29T08:30:00.663 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.663 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T08:30:00.663 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-03-29T08:30:00.663 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.663 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T08:30:00.663 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T08:30:00.664 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.664 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-03-29T08:30:00.664 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.664 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-03-29T08:30:00.664 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-03-29T08:30:00.664 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-03-29T08:30:00.664 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-03-29T08:30:00.664 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-03-29T08:30:00.664 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-03-29T08:30:00.664 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-03-29T08:30:00.664 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T08:30:00.664 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T08:30:00.664 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-03-29T08:30:00.664 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.665 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T08:30:00.665 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.665 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-03-29T08:30:00.665 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.665 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-03-29T08:30:00.665 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.665 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-03-29T08:30:00.665 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-03-29T08:30:00.665 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-03-29T08:30:00.665 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.73,[v2:172.21.15.73:3301,v1:172.21.15.73:6790]" 2024-03-29T08:30:00.665 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = e9b7960f-ec1f-4787-95c3-c806094d37f0 2024-03-29T08:30:00.665 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.665 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-03-29T08:30:00.665 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-03-29T08:30:00.665 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.666 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-03-29T08:30:00.666 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-03-29T08:30:00.666 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-03-29T08:30:00.666 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.666 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-03-29T08:30:00.666 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-03-29T08:30:00.666 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.666 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-03-29T08:30:00.666 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-03-29T08:30:00.666 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-03-29T08:30:00.666 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.666 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-03-29T08:30:00.666 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-03-29T08:30:00.666 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.667 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-03-29T08:30:00.667 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.667 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-03-29T08:30:00.667 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.667 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-03-29T08:30:00.667 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.667 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-03-29T08:30:00.667 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-03-29T08:30:00.667 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.667 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-03-29T08:30:00.667 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-03-29T08:30:00.667 INFO:teuthology.orchestra.run.smithi046.stdout: bdev async discard = True 2024-03-29T08:30:00.667 INFO:teuthology.orchestra.run.smithi046.stdout: bdev enable discard = True 2024-03-29T08:30:00.667 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore allocator = bitmap 2024-03-29T08:30:00.667 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-03-29T08:30:00.668 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-03-29T08:30:00.668 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 1/20 2024-03-29T08:30:00.668 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 1/20 2024-03-29T08:30:00.668 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:30:00.668 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-03-29T08:30:00.668 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 4/10 2024-03-29T08:30:00.668 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T08:30:00.668 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-03-29T08:30:00.668 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-03-29T08:30:00.668 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-03-29T08:30:00.668 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-03-29T08:30:00.668 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-03-29T08:30:00.668 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.668 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-03-29T08:30:00.668 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:30:00.669 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-03-29T08:30:00.669 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-03-29T08:30:00.669 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-03-29T08:30:00.669 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:30:00.669 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:30:00.669 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-03-29T08:30:00.669 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.669 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-03-29T08:30:00.669 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:30:00.669 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-03-29T08:30:00.669 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-03-29T08:30:00.669 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-03-29T08:30:00.669 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-03-29T08:30:00.669 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-03-29T08:30:00.669 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:30:00.669 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-03-29T08:30:00.670 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-03-29T08:30:00.670 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:30:00.670 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.670 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T08:30:00.670 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T08:30:00.670 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T08:30:00.670 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.670 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T08:30:00.670 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T08:30:00.670 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T08:30:00.670 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.670 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-03-29T08:30:00.670 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-03-29T08:30:00.670 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.671 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-03-29T08:30:00.671 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2024-03-29T08:30:00.671 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.671 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-03-29T08:30:00.671 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-03-29T08:30:00.671 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-03-29T08:30:00.671 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-03-29T08:30:00.671 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T08:30:00.671 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T08:30:00.671 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2024-03-29T08:30:00.671 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-03-29T08:30:00.671 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:30:00.671 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2024-03-29T08:30:00.671 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-03-29T08:30:00.671 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-03-29T08:30:00.672 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-03-29T08:30:00.672 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-03-29T08:30:00.672 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-03-29T08:30:00.672 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-03-29T08:30:00.672 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-03-29T08:30:00.672 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-03-29T08:30:00.672 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-03-29T08:30:00.672 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2024-03-29T08:30:00.672 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2024-03-29T08:30:00.672 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2024-03-29T08:30:00.672 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:30:00.672 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2024-03-29T08:30:00.672 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = False 2024-03-29T08:30:00.672 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2024-03-29T08:30:00.672 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = True 2024-03-29T08:30:00.673 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-03-29T08:30:00.673 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-03-29T08:30:00.673 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-03-29T08:30:00.673 INFO:teuthology.orchestra.run.smithi073.stdout:[global] 2024-03-29T08:30:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: chdir = "" 2024-03-29T08:30:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T08:30:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: auth supported = cephx 2024-03-29T08:30:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: filestore xattr use omap = true 2024-03-29T08:30:00.673 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: mon clock drift allowed = 1.000 2024-03-29T08:30:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: osd crush chooseleaf type = 0 2024-03-29T08:30:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: auth debug = true 2024-03-29T08:30:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on old message = true 2024-03-29T08:30:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on bug = true 2024-03-29T08:30:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T08:30:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: mon pg warn max object skew = 0 2024-03-29T08:30:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T08:30:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T08:30:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.674 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default size = 2 2024-03-29T08:30:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow primary affinity = true 2024-03-29T08:30:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow pg remap = true 2024-03-29T08:30:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on legacy crush tunables = false 2024-03-29T08:30:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on crush straw calc version zero = false 2024-03-29T08:30:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on no sortbitwise = false 2024-03-29T08:30:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on osd down out interval zero = false 2024-03-29T08:30:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on too few osds = false 2024-03-29T08:30:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T08:30:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T08:30:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: mon_allow_pool_size_one = true 2024-03-29T08:30:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T08:30:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.675 INFO:teuthology.orchestra.run.smithi073.stdout: osd default data pool replay window = 5 2024-03-29T08:30:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: mon allow pool delete = true 2024-03-29T08:30:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: mon cluster log file level = debug 2024-03-29T08:30:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: debug asserts on shutdown = true 2024-03-29T08:30:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: mon health detail to clog = false 2024-03-29T08:30:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: mon host = "172.21.15.46,172.21.15.73,[v2:172.21.15.73:3301,v1:172.21.15.73:6790]" 2024-03-29T08:30:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: fsid = e9b7960f-ec1f-4787-95c3-c806094d37f0 2024-03-29T08:30:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.676 INFO:teuthology.orchestra.run.smithi073.stdout:[osd] 2024-03-29T08:30:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: osd journal size = 100 2024-03-29T08:30:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub load threshold = 5.0 2024-03-29T08:30:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub max interval = 600 2024-03-29T08:30:00.676 INFO:teuthology.orchestra.run.smithi073.stdout: osd mclock profile = high_recovery_ops 2024-03-29T08:30:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: osd recover clone overlap = true 2024-03-29T08:30:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: osd recovery max chunk = 1048576 2024-03-29T08:30:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug shutdown = true 2024-03-29T08:30:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug op order = true 2024-03-29T08:30:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify stray on activate = true 2024-03-29T08:30:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: osd open classes on start = true 2024-03-29T08:30:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug pg log writeout = true 2024-03-29T08:30:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: osd deep scrub update digest min age = 30 2024-03-29T08:30:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: osd map max advance = 10 2024-03-29T08:30:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.677 INFO:teuthology.orchestra.run.smithi073.stdout: journal zero on create = true 2024-03-29T08:30:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: filestore ondisk finisher threads = 3 2024-03-29T08:30:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: filestore apply finisher threads = 3 2024-03-29T08:30:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: bdev debug aio = true 2024-03-29T08:30:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug misdirected ops = true 2024-03-29T08:30:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: bdev async discard = True 2024-03-29T08:30:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: bdev enable discard = True 2024-03-29T08:30:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore allocator = bitmap 2024-03-29T08:30:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore block size = 96636764160 2024-03-29T08:30:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore fsck on mount = True 2024-03-29T08:30:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluefs = 1/20 2024-03-29T08:30:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluestore = 1/20 2024-03-29T08:30:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:30:00.678 INFO:teuthology.orchestra.run.smithi073.stdout: debug osd = 20 2024-03-29T08:30:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: debug rocksdb = 4/10 2024-03-29T08:30:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T08:30:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd full ratio = 0.9 2024-03-29T08:30:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd nearfull ratio = 0.8 2024-03-29T08:30:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: osd failsafe full ratio = 0.95 2024-03-29T08:30:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: osd objectstore = bluestore 2024-03-29T08:30:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: osd op complaint time = 180 2024-03-29T08:30:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.679 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr] 2024-03-29T08:30:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:30:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: debug mgr = 20 2024-03-29T08:30:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-03-29T08:30:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-03-29T08:30:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:30:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:30:00.679 INFO:teuthology.orchestra.run.smithi073.stdout: mgr/telemetry/nag = false 2024-03-29T08:30:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.680 INFO:teuthology.orchestra.run.smithi073.stdout:[mon] 2024-03-29T08:30:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:30:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-03-29T08:30:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: debug paxos = 20 2024-03-29T08:30:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-03-29T08:30:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: mon data avail warn = 5 2024-03-29T08:30:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: mon mgr mkfs grace = 240 2024-03-29T08:30:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:30:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd reporter subtree level = osd 2024-03-29T08:30:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd prime pg temp = true 2024-03-29T08:30:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:30:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T08:30:00.680 INFO:teuthology.orchestra.run.smithi073.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T08:30:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T08:30:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T08:30:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T08:30:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T08:30:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: # 1m isn't quite enough 2024-03-29T08:30:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: mon_down_mkfs_grace = 2m 2024-03-29T08:30:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_filestore_osds = false 2024-03-29T08:30:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: mon op complaint time = 120 2024-03-29T08:30:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.681 INFO:teuthology.orchestra.run.smithi073.stdout:[client] 2024-03-29T08:30:00.681 INFO:teuthology.orchestra.run.smithi073.stdout: rgw cache enabled = true 2024-03-29T08:30:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable ops log = true 2024-03-29T08:30:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable usage log = true 2024-03-29T08:30:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T08:30:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T08:30:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: client mount timeout = 600 2024-03-29T08:30:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: debug client = 20 2024-03-29T08:30:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:30:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = "" 2024-03-29T08:30:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: rados mon op timeout = 900 2024-03-29T08:30:00.682 INFO:teuthology.orchestra.run.smithi073.stdout: rados osd op timeout = 900 2024-03-29T08:30:00.682 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.a] 2024-03-29T08:30:00.682 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.b] 2024-03-29T08:30:00.682 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.c] 2024-03-29T08:30:00.682 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.a] 2024-03-29T08:30:00.683 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.c] 2024-03-29T08:30:00.683 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.b] 2024-03-29T08:30:00.683 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.d] 2024-03-29T08:30:00.683 INFO:teuthology.orchestra.run.smithi073.stdout:[mds] 2024-03-29T08:30:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: debug mds = 20 2024-03-29T08:30:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: debug mds balancer = 20 2024-03-29T08:30:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:30:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: mds debug frag = True 2024-03-29T08:30:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: mds debug scatterstat = False 2024-03-29T08:30:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: mds op complaint time = 180 2024-03-29T08:30:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: mds verify scatter = True 2024-03-29T08:30:00.683 INFO:teuthology.orchestra.run.smithi073.stdout: osd op complaint time = 180 2024-03-29T08:30:00.684 INFO:teuthology.orchestra.run.smithi073.stdout: rados mon op timeout = 900 2024-03-29T08:30:00.684 INFO:teuthology.orchestra.run.smithi073.stdout: rados osd op timeout = 900 2024-03-29T08:30:00.686 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T08:30:00.730 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-29T08:30:00.770 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-03-29T08:30:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-03-29T08:30:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T08:30:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-03-29T08:30:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-03-29T08:30:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-03-29T08:30:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-03-29T08:30:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-03-29T08:30:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-03-29T08:30:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-03-29T08:30:00.771 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T08:30:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-03-29T08:30:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T08:30:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T08:30:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-03-29T08:30:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-03-29T08:30:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-03-29T08:30:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-03-29T08:30:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-03-29T08:30:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-03-29T08:30:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-03-29T08:30:00.772 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-03-29T08:30:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T08:30:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T08:30:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-03-29T08:30:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T08:30:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-03-29T08:30:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-03-29T08:30:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-03-29T08:30:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-03-29T08:30:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-03-29T08:30:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.73,[v2:172.21.15.73:3301,v1:172.21.15.73:6790]" 2024-03-29T08:30:00.773 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = e9b7960f-ec1f-4787-95c3-c806094d37f0 2024-03-29T08:30:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.774 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-03-29T08:30:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-03-29T08:30:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-03-29T08:30:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-03-29T08:30:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-03-29T08:30:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-03-29T08:30:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-03-29T08:30:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-03-29T08:30:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-03-29T08:30:00.774 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-03-29T08:30:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-03-29T08:30:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-03-29T08:30:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-03-29T08:30:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-03-29T08:30:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-03-29T08:30:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-03-29T08:30:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-03-29T08:30:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-03-29T08:30:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-03-29T08:30:00.775 INFO:teuthology.orchestra.run.smithi046.stdout: bdev async discard = True 2024-03-29T08:30:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: bdev enable discard = True 2024-03-29T08:30:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore allocator = bitmap 2024-03-29T08:30:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-03-29T08:30:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-03-29T08:30:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 1/20 2024-03-29T08:30:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 1/20 2024-03-29T08:30:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:30:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-03-29T08:30:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 4/10 2024-03-29T08:30:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T08:30:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-03-29T08:30:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-03-29T08:30:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-03-29T08:30:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-03-29T08:30:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-03-29T08:30:00.776 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.777 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-03-29T08:30:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:30:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-03-29T08:30:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-03-29T08:30:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-03-29T08:30:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:30:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:30:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-03-29T08:30:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.777 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-03-29T08:30:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:30:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-03-29T08:30:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-03-29T08:30:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-03-29T08:30:00.777 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-03-29T08:30:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-03-29T08:30:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:30:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-03-29T08:30:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-03-29T08:30:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:30:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T08:30:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T08:30:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T08:30:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T08:30:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T08:30:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T08:30:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.778 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-03-29T08:30:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-03-29T08:30:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-03-29T08:30:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2024-03-29T08:30:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:00.779 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-03-29T08:30:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-03-29T08:30:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-03-29T08:30:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-03-29T08:30:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T08:30:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T08:30:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2024-03-29T08:30:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2024-03-29T08:30:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:30:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2024-03-29T08:30:00.779 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-03-29T08:30:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-03-29T08:30:00.780 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-03-29T08:30:00.780 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2024-03-29T08:30:00.780 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2024-03-29T08:30:00.780 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2024-03-29T08:30:00.780 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c] 2024-03-29T08:30:00.780 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2024-03-29T08:30:00.780 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d] 2024-03-29T08:30:00.780 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2024-03-29T08:30:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2024-03-29T08:30:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2024-03-29T08:30:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-03-29T08:30:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2024-03-29T08:30:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = True 2024-03-29T08:30:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2024-03-29T08:30:00.780 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = True 2024-03-29T08:30:00.781 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2024-03-29T08:30:00.781 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2024-03-29T08:30:00.781 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2024-03-29T08:30:00.781 INFO:teuthology.orchestra.run.smithi073.stdout:[global] 2024-03-29T08:30:00.781 INFO:teuthology.orchestra.run.smithi073.stdout: chdir = "" 2024-03-29T08:30:00.782 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-29T08:30:00.782 INFO:teuthology.orchestra.run.smithi073.stdout: auth supported = cephx 2024-03-29T08:30:00.782 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.782 INFO:teuthology.orchestra.run.smithi073.stdout: filestore xattr use omap = true 2024-03-29T08:30:00.782 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.782 INFO:teuthology.orchestra.run.smithi073.stdout: mon clock drift allowed = 1.000 2024-03-29T08:30:00.782 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.782 INFO:teuthology.orchestra.run.smithi073.stdout: osd crush chooseleaf type = 0 2024-03-29T08:30:00.782 INFO:teuthology.orchestra.run.smithi073.stdout: auth debug = true 2024-03-29T08:30:00.782 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.782 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on old message = true 2024-03-29T08:30:00.782 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on bug = true 2024-03-29T08:30:00.782 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.782 INFO:teuthology.orchestra.run.smithi073.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-29T08:30:00.783 INFO:teuthology.orchestra.run.smithi073.stdout: mon pg warn max object skew = 0 2024-03-29T08:30:00.783 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.783 INFO:teuthology.orchestra.run.smithi073.stdout: # disable pg_autoscaler by default for new pools 2024-03-29T08:30:00.783 INFO:teuthology.orchestra.run.smithi073.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-29T08:30:00.783 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.783 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default size = 2 2024-03-29T08:30:00.783 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.783 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow primary affinity = true 2024-03-29T08:30:00.783 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow pg remap = true 2024-03-29T08:30:00.783 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on legacy crush tunables = false 2024-03-29T08:30:00.783 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on crush straw calc version zero = false 2024-03-29T08:30:00.783 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on no sortbitwise = false 2024-03-29T08:30:00.783 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on osd down out interval zero = false 2024-03-29T08:30:00.783 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on too few osds = false 2024-03-29T08:30:00.783 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-29T08:30:00.783 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-29T08:30:00.784 INFO:teuthology.orchestra.run.smithi073.stdout: mon_allow_pool_size_one = true 2024-03-29T08:30:00.784 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.784 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-29T08:30:00.784 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.784 INFO:teuthology.orchestra.run.smithi073.stdout: osd default data pool replay window = 5 2024-03-29T08:30:00.784 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.784 INFO:teuthology.orchestra.run.smithi073.stdout: mon allow pool delete = true 2024-03-29T08:30:00.784 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.784 INFO:teuthology.orchestra.run.smithi073.stdout: mon cluster log file level = debug 2024-03-29T08:30:00.784 INFO:teuthology.orchestra.run.smithi073.stdout: debug asserts on shutdown = true 2024-03-29T08:30:00.784 INFO:teuthology.orchestra.run.smithi073.stdout: mon health detail to clog = false 2024-03-29T08:30:00.784 INFO:teuthology.orchestra.run.smithi073.stdout: mon host = "172.21.15.46,172.21.15.73,[v2:172.21.15.73:3301,v1:172.21.15.73:6790]" 2024-03-29T08:30:00.784 INFO:teuthology.orchestra.run.smithi073.stdout: fsid = e9b7960f-ec1f-4787-95c3-c806094d37f0 2024-03-29T08:30:00.784 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.784 INFO:teuthology.orchestra.run.smithi073.stdout:[osd] 2024-03-29T08:30:00.784 INFO:teuthology.orchestra.run.smithi073.stdout: osd journal size = 100 2024-03-29T08:30:00.785 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.785 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub load threshold = 5.0 2024-03-29T08:30:00.785 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub max interval = 600 2024-03-29T08:30:00.785 INFO:teuthology.orchestra.run.smithi073.stdout: osd mclock profile = high_recovery_ops 2024-03-29T08:30:00.785 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.785 INFO:teuthology.orchestra.run.smithi073.stdout: osd recover clone overlap = true 2024-03-29T08:30:00.785 INFO:teuthology.orchestra.run.smithi073.stdout: osd recovery max chunk = 1048576 2024-03-29T08:30:00.785 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.785 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug shutdown = true 2024-03-29T08:30:00.785 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug op order = true 2024-03-29T08:30:00.785 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify stray on activate = true 2024-03-29T08:30:00.785 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.785 INFO:teuthology.orchestra.run.smithi073.stdout: osd open classes on start = true 2024-03-29T08:30:00.785 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug pg log writeout = true 2024-03-29T08:30:00.785 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.785 INFO:teuthology.orchestra.run.smithi073.stdout: osd deep scrub update digest min age = 30 2024-03-29T08:30:00.786 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.786 INFO:teuthology.orchestra.run.smithi073.stdout: osd map max advance = 10 2024-03-29T08:30:00.786 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.786 INFO:teuthology.orchestra.run.smithi073.stdout: journal zero on create = true 2024-03-29T08:30:00.786 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.786 INFO:teuthology.orchestra.run.smithi073.stdout: filestore ondisk finisher threads = 3 2024-03-29T08:30:00.786 INFO:teuthology.orchestra.run.smithi073.stdout: filestore apply finisher threads = 3 2024-03-29T08:30:00.786 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.786 INFO:teuthology.orchestra.run.smithi073.stdout: bdev debug aio = true 2024-03-29T08:30:00.786 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug misdirected ops = true 2024-03-29T08:30:00.786 INFO:teuthology.orchestra.run.smithi073.stdout: bdev async discard = True 2024-03-29T08:30:00.786 INFO:teuthology.orchestra.run.smithi073.stdout: bdev enable discard = True 2024-03-29T08:30:00.786 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore allocator = bitmap 2024-03-29T08:30:00.786 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore block size = 96636764160 2024-03-29T08:30:00.786 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore fsck on mount = True 2024-03-29T08:30:00.786 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluefs = 1/20 2024-03-29T08:30:00.787 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluestore = 1/20 2024-03-29T08:30:00.787 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:30:00.787 INFO:teuthology.orchestra.run.smithi073.stdout: debug osd = 20 2024-03-29T08:30:00.787 INFO:teuthology.orchestra.run.smithi073.stdout: debug rocksdb = 4/10 2024-03-29T08:30:00.787 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd backfillfull_ratio = 0.85 2024-03-29T08:30:00.787 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd full ratio = 0.9 2024-03-29T08:30:00.787 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd nearfull ratio = 0.8 2024-03-29T08:30:00.787 INFO:teuthology.orchestra.run.smithi073.stdout: osd failsafe full ratio = 0.95 2024-03-29T08:30:00.787 INFO:teuthology.orchestra.run.smithi073.stdout: osd objectstore = bluestore 2024-03-29T08:30:00.787 INFO:teuthology.orchestra.run.smithi073.stdout: osd op complaint time = 180 2024-03-29T08:30:00.787 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.787 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr] 2024-03-29T08:30:00.787 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:30:00.787 INFO:teuthology.orchestra.run.smithi073.stdout: debug mgr = 20 2024-03-29T08:30:00.787 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-03-29T08:30:00.787 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-03-29T08:30:00.788 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:30:00.788 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:30:00.788 INFO:teuthology.orchestra.run.smithi073.stdout: mgr/telemetry/nag = false 2024-03-29T08:30:00.788 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.788 INFO:teuthology.orchestra.run.smithi073.stdout:[mon] 2024-03-29T08:30:00.788 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:30:00.788 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-03-29T08:30:00.788 INFO:teuthology.orchestra.run.smithi073.stdout: debug paxos = 20 2024-03-29T08:30:00.788 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-03-29T08:30:00.788 INFO:teuthology.orchestra.run.smithi073.stdout: mon data avail warn = 5 2024-03-29T08:30:00.788 INFO:teuthology.orchestra.run.smithi073.stdout: mon mgr mkfs grace = 240 2024-03-29T08:30:00.788 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-03-29T08:30:00.788 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd reporter subtree level = osd 2024-03-29T08:30:00.788 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd prime pg temp = true 2024-03-29T08:30:00.788 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-03-29T08:30:00.788 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.789 INFO:teuthology.orchestra.run.smithi073.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-29T08:30:00.789 INFO:teuthology.orchestra.run.smithi073.stdout: auth mon ticket ttl = 660 # 11m 2024-03-29T08:30:00.789 INFO:teuthology.orchestra.run.smithi073.stdout: auth service ticket ttl = 240 # 4m 2024-03-29T08:30:00.789 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.789 INFO:teuthology.orchestra.run.smithi073.stdout: # don't complain about insecure global_id in the test suite 2024-03-29T08:30:00.789 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-29T08:30:00.789 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-29T08:30:00.789 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.789 INFO:teuthology.orchestra.run.smithi073.stdout: # 1m isn't quite enough 2024-03-29T08:30:00.789 INFO:teuthology.orchestra.run.smithi073.stdout: mon_down_mkfs_grace = 2m 2024-03-29T08:30:00.789 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.789 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_filestore_osds = false 2024-03-29T08:30:00.789 INFO:teuthology.orchestra.run.smithi073.stdout: mon op complaint time = 120 2024-03-29T08:30:00.789 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-03-29T08:30:00.789 INFO:teuthology.orchestra.run.smithi073.stdout:[client] 2024-03-29T08:30:00.789 INFO:teuthology.orchestra.run.smithi073.stdout: rgw cache enabled = true 2024-03-29T08:30:00.790 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable ops log = true 2024-03-29T08:30:00.790 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable usage log = true 2024-03-29T08:30:00.790 INFO:teuthology.orchestra.run.smithi073.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-29T08:30:00.790 INFO:teuthology.orchestra.run.smithi073.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-29T08:30:00.790 INFO:teuthology.orchestra.run.smithi073.stdout: client mount timeout = 600 2024-03-29T08:30:00.790 INFO:teuthology.orchestra.run.smithi073.stdout: debug client = 20 2024-03-29T08:30:00.790 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:30:00.790 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = "" 2024-03-29T08:30:00.790 INFO:teuthology.orchestra.run.smithi073.stdout: rados mon op timeout = 900 2024-03-29T08:30:00.790 INFO:teuthology.orchestra.run.smithi073.stdout: rados osd op timeout = 900 2024-03-29T08:30:00.790 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.a] 2024-03-29T08:30:00.790 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.b] 2024-03-29T08:30:00.790 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.c] 2024-03-29T08:30:00.790 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.a] 2024-03-29T08:30:00.790 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.c] 2024-03-29T08:30:00.791 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.b] 2024-03-29T08:30:00.791 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.d] 2024-03-29T08:30:00.791 INFO:teuthology.orchestra.run.smithi073.stdout:[mds] 2024-03-29T08:30:00.791 INFO:teuthology.orchestra.run.smithi073.stdout: debug mds = 20 2024-03-29T08:30:00.791 INFO:teuthology.orchestra.run.smithi073.stdout: debug mds balancer = 20 2024-03-29T08:30:00.791 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-03-29T08:30:00.791 INFO:teuthology.orchestra.run.smithi073.stdout: mds debug frag = True 2024-03-29T08:30:00.791 INFO:teuthology.orchestra.run.smithi073.stdout: mds debug scatterstat = True 2024-03-29T08:30:00.791 INFO:teuthology.orchestra.run.smithi073.stdout: mds op complaint time = 180 2024-03-29T08:30:00.791 INFO:teuthology.orchestra.run.smithi073.stdout: mds verify scatter = True 2024-03-29T08:30:00.791 INFO:teuthology.orchestra.run.smithi073.stdout: osd op complaint time = 180 2024-03-29T08:30:00.791 INFO:teuthology.orchestra.run.smithi073.stdout: rados mon op timeout = 900 2024-03-29T08:30:00.791 INFO:teuthology.orchestra.run.smithi073.stdout: rados osd op timeout = 900 2024-03-29T08:30:00.793 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-03-29T08:30:00.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.981+0000 7ff50bc00640 1 Processor -- start 2024-03-29T08:30:00.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.981+0000 7ff50bc00640 1 -- start start 2024-03-29T08:30:00.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.985+0000 7ff50bc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff504142a50 0x7ff5040cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.985+0000 7ff50bc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff5040ae990 0x7ff5040cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.985+0000 7ff50bc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50415d260 con 0x7ff50415df90 2024-03-29T08:30:00.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.985+0000 7ff50bc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5040aefd0 con 0x7ff5040ae990 2024-03-29T08:30:00.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.985+0000 7ff50bc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50415d4b0 con 0x7ff504142a50 2024-03-29T08:30:00.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.985+0000 7ff509c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff5040ae990 0x7ff5040cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.985+0000 7ff509c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff5040ae990 0x7ff5040cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45134/0 (socket says 172.21.15.46:45134) 2024-03-29T08:30:00.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.985+0000 7ff509c00640 1 -- 172.21.15.46:0/4275936863 learned_addr learned my addr 172.21.15.46:0/4275936863 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:00.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.985+0000 7ff509200640 1 --2- 172.21.15.46:0/4275936863 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff504142a50 0x7ff5040cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.985+0000 7ff509c00640 1 -- 172.21.15.46:0/4275936863 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff504142a50 msgr2=0x7ff5040cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:00.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.985+0000 7ff509c00640 1 --2- 172.21.15.46:0/4275936863 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff504142a50 0x7ff5040cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.985+0000 7ff509c00640 1 -- 172.21.15.46:0/4275936863 >> v1:172.21.15.73:6789/0 conn(0x7ff50415df90 legacy=0x7ff5040ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.985+0000 7ff509c00640 1 -- 172.21.15.46:0/4275936863 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50415d790 con 0x7ff5040ae990 2024-03-29T08:30:00.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.989+0000 7ff509c00640 1 --2- 172.21.15.46:0/4275936863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5040ae990 0x7ff5040cbcd0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009e50 tx=0x7ff4fc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f56f2ddcd6b9909 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.989+0000 7ff503400640 1 -- 172.21.15.46:0/4275936863 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0a5660 con 0x7ff5040ae990 2024-03-29T08:30:00.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.989+0000 7ff503400640 1 -- 172.21.15.46:0/4275936863 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc0a57c0 con 0x7ff5040ae990 2024-03-29T08:30:00.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.989+0000 7ff503400640 1 -- 172.21.15.46:0/4275936863 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0a5b50 con 0x7ff5040ae990 2024-03-29T08:30:00.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.989+0000 7ff50bc00640 1 -- 172.21.15.46:0/4275936863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5040ae990 msgr2=0x7ff5040cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.989+0000 7ff50bc00640 1 --2- 172.21.15.46:0/4275936863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5040ae990 0x7ff5040cbcd0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009e50 tx=0x7ff4fc01c9a0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.989+0000 7ff50bc00640 1 -- 172.21.15.46:0/4275936863 shutdown_connections 2024-03-29T08:30:00.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.989+0000 7ff50bc00640 1 --2- 172.21.15.46:0/4275936863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5040ae990 0x7ff5040cbcd0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.989+0000 7ff50bc00640 1 --2- 172.21.15.46:0/4275936863 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff504142a50 0x7ff5040cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.989+0000 7ff50bc00640 1 -- 172.21.15.46:0/4275936863 >> 172.21.15.46:0/4275936863 conn(0x7ff50406a390 msgr2=0x7ff5040cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:00.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.989+0000 7ff50bc00640 1 -- 172.21.15.46:0/4275936863 shutdown_connections 2024-03-29T08:30:00.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.989+0000 7ff50bc00640 1 -- 172.21.15.46:0/4275936863 wait complete. 2024-03-29T08:30:00.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.989+0000 7ff50bc00640 1 Processor -- start 2024-03-29T08:30:00.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.989+0000 7ff50bc00640 1 -- start start 2024-03-29T08:30:00.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff50bc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5040ae990 0x7ff504159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff50bc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff504142a50 0x7ff50415e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff50bc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff50415df90 0x7ff50415af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff50bc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50415e8b0 con 0x7ff50415df90 2024-03-29T08:30:00.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff50bc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5040ad220 con 0x7ff504142a50 2024-03-29T08:30:00.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff50bc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50415eae0 con 0x7ff5040ae990 2024-03-29T08:30:00.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff509c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff50415df90 0x7ff50415af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff509c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff50415df90 0x7ff50415af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45146/0 (socket says 172.21.15.46:45146) 2024-03-29T08:30:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff509c00640 1 -- 172.21.15.46:0/3787081835 learned_addr learned my addr 172.21.15.46:0/3787081835 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff509200640 1 --2- 172.21.15.46:0/3787081835 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5040ae990 0x7ff504159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff503e00640 1 --2- 172.21.15.46:0/3787081835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff504142a50 0x7ff50415e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff509c00640 1 -- 172.21.15.46:0/3787081835 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5040ae990 msgr2=0x7ff504159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff509c00640 1 --2- 172.21.15.46:0/3787081835 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5040ae990 0x7ff504159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff509c00640 1 -- 172.21.15.46:0/3787081835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff504142a50 msgr2=0x7ff50415e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff509c00640 1 --2- 172.21.15.46:0/3787081835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff504142a50 0x7ff50415e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff509c00640 1 -- 172.21.15.46:0/3787081835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4fc009b30 con 0x7ff50415df90 2024-03-29T08:30:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff509c00640 1 --2- 172.21.15.46:0/3787081835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff50415df90 0x7ff50415af00 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0af040 tx=0x7ff4fc019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff500c00640 1 -- 172.21.15.46:0/3787081835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0a5660 con 0x7ff50415df90 2024-03-29T08:30:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff500c00640 1 -- 172.21.15.46:0/3787081835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4fc0b88a0 con 0x7ff50415df90 2024-03-29T08:30:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff500c00640 1 -- 172.21.15.46:0/3787081835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0b8c50 con 0x7ff50415df90 2024-03-29T08:30:00.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff50bc00640 1 -- 172.21.15.46:0/3787081835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5040ac460 con 0x7ff50415df90 2024-03-29T08:30:00.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff50bc00640 1 -- 172.21.15.46:0/3787081835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5040aca20 con 0x7ff50415df90 2024-03-29T08:30:00.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff50bc00640 1 -- 172.21.15.46:0/3787081835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff504071a70 con 0x7ff50415df90 2024-03-29T08:30:00.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.993+0000 7ff500c00640 1 -- 172.21.15.46:0/3787081835 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff4fc0b6440 con 0x7ff50415df90 2024-03-29T08:30:00.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.997+0000 7ff500c00640 1 --2- 172.21.15.46:0/3787081835 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff4e4066690 0x7ff4e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:00.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.997+0000 7ff500c00640 1 -- 172.21.15.46:0/3787081835 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7ff4fc11f840 con 0x7ff50415df90 2024-03-29T08:30:00.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.997+0000 7ff509200640 1 --2- 172.21.15.46:0/3787081835 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff4e4066690 0x7ff4e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:00.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:00.997+0000 7ff509200640 1 --2- 172.21.15.46:0/3787081835 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff4e4066690 0x7ff4e4068b50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff504159af0 tx=0x7ff4f8009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:01.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:01.001+0000 7ff500c00640 1 -- 172.21.15.46:0/3787081835 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff4fc0ecc00 con 0x7ff50415df90 2024-03-29T08:30:01.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:01.197+0000 7ff50bc00640 1 -- 172.21.15.46:0/3787081835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7ff5041643d0 con 0x7ff50415df90 2024-03-29T08:30:02.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.089+0000 7ff500c00640 1 -- 172.21.15.46:0/3787081835 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v103) v1 ==== 162+0+0 (secure 0 0 0) 0x7ff4fc009510 con 0x7ff50415df90 2024-03-29T08:30:02.088 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked joinable; MDS may join as newly active. 2024-03-29T08:30:02.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.089+0000 7ff50bc00640 1 -- 172.21.15.46:0/3787081835 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff4e4066690 msgr2=0x7ff4e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.089+0000 7ff50bc00640 1 --2- 172.21.15.46:0/3787081835 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff4e4066690 0x7ff4e4068b50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7ff504159af0 tx=0x7ff4f8009190 comp rx=0 tx=0).stop 2024-03-29T08:30:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.089+0000 7ff50bc00640 1 -- 172.21.15.46:0/3787081835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff50415df90 msgr2=0x7ff50415af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.089+0000 7ff50bc00640 1 --2- 172.21.15.46:0/3787081835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff50415df90 0x7ff50415af00 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0af040 tx=0x7ff4fc019740 comp rx=0 tx=0).stop 2024-03-29T08:30:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.089+0000 7ff50bc00640 1 -- 172.21.15.46:0/3787081835 shutdown_connections 2024-03-29T08:30:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.089+0000 7ff50bc00640 1 --2- 172.21.15.46:0/3787081835 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff4e4066690 0x7ff4e4068b50 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.089+0000 7ff50bc00640 1 --2- 172.21.15.46:0/3787081835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff504142a50 0x7ff50415e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.089+0000 7ff50bc00640 1 --2- 172.21.15.46:0/3787081835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff50415df90 0x7ff50415af00 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.089+0000 7ff50bc00640 1 --2- 172.21.15.46:0/3787081835 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff5040ae990 0x7ff504159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.089+0000 7ff50bc00640 1 -- 172.21.15.46:0/3787081835 >> 172.21.15.46:0/3787081835 conn(0x7ff50406a390 msgr2=0x7ff5040b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.093+0000 7ff50bc00640 1 -- 172.21.15.46:0/3787081835 shutdown_connections 2024-03-29T08:30:02.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.093+0000 7ff50bc00640 1 -- 172.21.15.46:0/3787081835 wait complete. 2024-03-29T08:30:02.118 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:30:02.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.261+0000 7fd87cc00640 1 Processor -- start 2024-03-29T08:30:02.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.261+0000 7fd87cc00640 1 -- start start 2024-03-29T08:30:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.265+0000 7fd87cc00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd8781593d0 0x7fd87806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.265+0000 7fd87cc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd87815d230 0x7fd87815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.265+0000 7fd87cc00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd87806d130 con 0x7fd87815dbd0 2024-03-29T08:30:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.265+0000 7fd87cc00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd87815b8f0 con 0x7fd8781593d0 2024-03-29T08:30:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.265+0000 7fd87cc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd87815ad50 con 0x7fd87815d230 2024-03-29T08:30:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.265+0000 7fd875e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd8781593d0 0x7fd87806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.265+0000 7fd875e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd8781593d0 0x7fd87806cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.46:55140/0 (socket says 172.21.15.46:55140) 2024-03-29T08:30:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.265+0000 7fd875e00640 1 -- 172.21.15.46:0/1058152500 learned_addr learned my addr 172.21.15.46:0/1058152500 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:02.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.265+0000 7fd876800640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd87815dbd0 0x7fd87815bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:58056/0 (socket says 172.21.15.46:58056) 2024-03-29T08:30:02.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.265+0000 7fd86d400640 1 --2- 172.21.15.46:0/1058152500 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd87815d230 0x7fd87815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:02.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.265+0000 7fd875e00640 1 -- 172.21.15.46:0/1058152500 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd87815d230 msgr2=0x7fd87815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:02.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.265+0000 7fd875e00640 1 --2- 172.21.15.46:0/1058152500 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd87815d230 0x7fd87815d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:02.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.265+0000 7fd875e00640 1 -- 172.21.15.46:0/1058152500 >> v1:172.21.15.73:6789/0 conn(0x7fd87815dbd0 legacy=0x7fd87815bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:02.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.265+0000 7fd875e00640 1 -- 172.21.15.46:0/1058152500 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87815e060 con 0x7fd8781593d0 2024-03-29T08:30:02.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd875e00640 1 --2- 172.21.15.46:0/1058152500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8781593d0 0x7fd87806cbf0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd868009870 tx=0x7fd86801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30cbd09466a6fdc7 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:02.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd875400640 1 -- 172.21.15.46:0/1058152500 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8680a9620 con 0x7fd8781593d0 2024-03-29T08:30:02.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd875400640 1 -- 172.21.15.46:0/1058152500 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8680a9780 con 0x7fd8781593d0 2024-03-29T08:30:02.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd875400640 1 -- 172.21.15.46:0/1058152500 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8680a9b10 con 0x7fd8781593d0 2024-03-29T08:30:02.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd87cc00640 1 -- 172.21.15.46:0/1058152500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8781593d0 msgr2=0x7fd87806cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:02.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd87cc00640 1 --2- 172.21.15.46:0/1058152500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8781593d0 0x7fd87806cbf0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd868009870 tx=0x7fd86801c560 comp rx=0 tx=0).stop 2024-03-29T08:30:02.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd87cc00640 1 -- 172.21.15.46:0/1058152500 shutdown_connections 2024-03-29T08:30:02.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd87cc00640 1 --2- 172.21.15.46:0/1058152500 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd87815d230 0x7fd87815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:02.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd87cc00640 1 --2- 172.21.15.46:0/1058152500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8781593d0 0x7fd87806cbf0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:02.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd87cc00640 1 -- 172.21.15.46:0/1058152500 >> 172.21.15.46:0/1058152500 conn(0x7fd878066110 msgr2=0x7fd878069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:02.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd87cc00640 1 -- 172.21.15.46:0/1058152500 shutdown_connections 2024-03-29T08:30:02.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd87cc00640 1 -- 172.21.15.46:0/1058152500 wait complete. 2024-03-29T08:30:02.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd87cc00640 1 Processor -- start 2024-03-29T08:30:02.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd87cc00640 1 -- start start 2024-03-29T08:30:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd87cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8781593d0 0x7fd8780a8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd87cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd87815d230 0x7fd8781520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd87cc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd87815dbd0 0x7fd8780cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd87cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8781572e0 con 0x7fd87815d230 2024-03-29T08:30:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd87cc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8781575c0 con 0x7fd8781593d0 2024-03-29T08:30:02.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.269+0000 7fd87cc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8781578a0 con 0x7fd87815dbd0 2024-03-29T08:30:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd86d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd87815d230 0x7fd8781520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd86d400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd87815d230 0x7fd8781520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45160/0 (socket says 172.21.15.46:45160) 2024-03-29T08:30:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd86d400640 1 -- 172.21.15.46:0/4162627709 learned_addr learned my addr 172.21.15.46:0/4162627709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd876800640 1 --2- 172.21.15.46:0/4162627709 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd87815dbd0 0x7fd8780cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd875e00640 1 --2- 172.21.15.46:0/4162627709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8781593d0 0x7fd8780a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd86d400640 1 -- 172.21.15.46:0/4162627709 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd87815dbd0 msgr2=0x7fd8780cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd86d400640 1 --2- 172.21.15.46:0/4162627709 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd87815dbd0 0x7fd8780cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd86d400640 1 -- 172.21.15.46:0/4162627709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8781593d0 msgr2=0x7fd8780a8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd86d400640 1 --2- 172.21.15.46:0/4162627709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8781593d0 0x7fd8780a8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd86d400640 1 -- 172.21.15.46:0/4162627709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8680094d0 con 0x7fd87815d230 2024-03-29T08:30:02.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd86d400640 1 --2- 172.21.15.46:0/4162627709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd87815d230 0x7fd8781520b0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fd86000efc0 tx=0x7fd86000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd86ea00640 1 -- 172.21.15.46:0/4162627709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860009b80 con 0x7fd87815d230 2024-03-29T08:30:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd86ea00640 1 -- 172.21.15.46:0/4162627709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86000f040 con 0x7fd87815d230 2024-03-29T08:30:02.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd86ea00640 1 -- 172.21.15.46:0/4162627709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8600146a0 con 0x7fd87815d230 2024-03-29T08:30:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd87cc00640 1 -- 172.21.15.46:0/4162627709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd878157b80 con 0x7fd87815d230 2024-03-29T08:30:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd87cc00640 1 -- 172.21.15.46:0/4162627709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8780a8990 con 0x7fd87815d230 2024-03-29T08:30:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd87cc00640 1 -- 172.21.15.46:0/4162627709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd878066c90 con 0x7fd87815d230 2024-03-29T08:30:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd86ea00640 1 -- 172.21.15.46:0/4162627709 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd860014800 con 0x7fd87815d230 2024-03-29T08:30:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.273+0000 7fd86ea00640 1 --2- 172.21.15.46:0/4162627709 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd854066690 0x7fd854068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:02.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.277+0000 7fd86ea00640 1 -- 172.21.15.46:0/4162627709 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fd860084cb0 con 0x7fd87815d230 2024-03-29T08:30:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.277+0000 7fd875e00640 1 --2- 172.21.15.46:0/4162627709 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd854066690 0x7fd854068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:02.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.277+0000 7fd875e00640 1 --2- 172.21.15.46:0/4162627709 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd854066690 0x7fd854068b50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd8680ae000 tx=0x7fd8680b3040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:02.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.277+0000 7fd86ea00640 1 -- 172.21.15.46:0/4162627709 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd860051fd0 con 0x7fd87815d230 2024-03-29T08:30:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.477+0000 7fd87cc00640 1 -- 172.21.15.46:0/4162627709 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd87806d3d0 con 0x7fd87815d230 2024-03-29T08:30:02.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.477+0000 7fd86ea00640 1 -- 172.21.15.46:0/4162627709 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4741 (secure 0 0 0) 0x7fd8600555f0 con 0x7fd87815d230 2024-03-29T08:30:02.479 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:02.480 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2506132630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2506132630},{"type":"v1","addr":"172.21.15.73:6836","nonce":2506132630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2108115501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2108115501},{"type":"v1","addr":"172.21.15.73:6837","nonce":2108115501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":9045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3210062895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3210062895},{"type":"v1","addr":"172.21.15.46:6833","nonce":3210062895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:29:14.470858+0000","modified":"2024-03-29T08:30:02.090028+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9660":{"gid":9660,"name":"a","rank":0,"incarnation":104,"state":"up:replay","state_seq":1,"addr":"172.21.15.46:6835/3867471367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3867471367},{"type":"v1","addr":"172.21.15.46:6835","nonce":3867471367}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-29T08:30:02.480 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 104 2024-03-29T08:30:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.481+0000 7fd87cc00640 1 -- 172.21.15.46:0/4162627709 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd854066690 msgr2=0x7fd854068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.481+0000 7fd87cc00640 1 --2- 172.21.15.46:0/4162627709 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd854066690 0x7fd854068b50 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd8680ae000 tx=0x7fd8680b3040 comp rx=0 tx=0).stop 2024-03-29T08:30:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.481+0000 7fd87cc00640 1 -- 172.21.15.46:0/4162627709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd87815d230 msgr2=0x7fd8781520b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.481+0000 7fd87cc00640 1 --2- 172.21.15.46:0/4162627709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd87815d230 0x7fd8781520b0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fd86000efc0 tx=0x7fd86000c490 comp rx=0 tx=0).stop 2024-03-29T08:30:02.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.481+0000 7fd87cc00640 1 -- 172.21.15.46:0/4162627709 shutdown_connections 2024-03-29T08:30:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.481+0000 7fd87cc00640 1 --2- 172.21.15.46:0/4162627709 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd854066690 0x7fd854068b50 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.481+0000 7fd87cc00640 1 --2- 172.21.15.46:0/4162627709 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd87815dbd0 0x7fd8780cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.481+0000 7fd87cc00640 1 --2- 172.21.15.46:0/4162627709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd87815d230 0x7fd8781520b0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.481+0000 7fd87cc00640 1 --2- 172.21.15.46:0/4162627709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8781593d0 0x7fd8780a8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.481+0000 7fd87cc00640 1 -- 172.21.15.46:0/4162627709 >> 172.21.15.46:0/4162627709 conn(0x7fd878066110 msgr2=0x7fd8780701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.481+0000 7fd87cc00640 1 -- 172.21.15.46:0/4162627709 shutdown_connections 2024-03-29T08:30:02.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:02.481+0000 7fd87cc00640 1 -- 172.21.15.46:0/4162627709 wait complete. 2024-03-29T08:30:02.906 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:29:14.470858+0000', 'modified': '2024-03-29T08:30:02.090028+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 125, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9660}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9660': {'gid': 9660, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.46:6835/3867471367', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 3867471367}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 3867471367}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:30:02.906 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9660:up:replay 2024-03-29T08:30:03.907 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:30:04.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.061+0000 7f87ea000640 1 Processor -- start 2024-03-29T08:30:04.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.061+0000 7f87ea000640 1 -- start start 2024-03-29T08:30:04.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.061+0000 7f87ea000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87e415a640 0x7f87e40ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:04.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.061+0000 7f87ea000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f87e415eac0 0x7f87e40cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.061+0000 7f87ea000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e40cee30 con 0x7f87e40cf2c0 2024-03-29T08:30:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.061+0000 7f87ea000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e415f120 con 0x7f87e415eac0 2024-03-29T08:30:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.061+0000 7f87ea000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40ce500 con 0x7f87e415a640 2024-03-29T08:30:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.061+0000 7f87e3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f87e40cf2c0 0x7f87e415e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46124/0 (socket says 172.21.15.46:46124) 2024-03-29T08:30:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.061+0000 7f87e3e00640 1 -- 172.21.15.46:0/3878479917 learned_addr learned my addr 172.21.15.46:0/3878479917 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.061+0000 7f87e8c00640 1 --2- 172.21.15.46:0/3878479917 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87e415a640 0x7f87e40ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.061+0000 7f87e9600640 1 --2- 172.21.15.46:0/3878479917 >> v2:172.21.15.73:3300/0 conn(0x7f87e415eac0 0x7f87e40cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.061+0000 7f87e3400640 1 -- 172.21.15.46:0/3878479917 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571860590 0 0) 0x7f87e40cee30 con 0x7f87e40cf2c0 2024-03-29T08:30:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.061+0000 7f87e3400640 1 -- 172.21.15.46:0/3878479917 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87cc003530 con 0x7f87e40cf2c0 2024-03-29T08:30:04.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.061+0000 7f87e8c00640 1 -- 172.21.15.46:0/3878479917 >> v2:172.21.15.73:3300/0 conn(0x7f87e415eac0 msgr2=0x7f87e40cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:04.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.061+0000 7f87e8c00640 1 --2- 172.21.15.46:0/3878479917 >> v2:172.21.15.73:3300/0 conn(0x7f87e415eac0 0x7f87e40cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:04.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.061+0000 7f87e8c00640 1 -- 172.21.15.46:0/3878479917 >> v1:172.21.15.46:6789/0 conn(0x7f87e40cf2c0 legacy=0x7f87e415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:04.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.061+0000 7f87e8c00640 1 -- 172.21.15.46:0/3878479917 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e40ce7e0 con 0x7f87e415a640 2024-03-29T08:30:04.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.065+0000 7f87e8c00640 1 --2- 172.21.15.46:0/3878479917 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87e415a640 0x7f87e40ccf50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f87d80097c0 tx=0x7f87d801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=405089fd4de5a2b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.065+0000 7f87e3400640 1 -- 172.21.15.46:0/3878479917 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d80ab600 con 0x7f87e415a640 2024-03-29T08:30:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.065+0000 7f87e3400640 1 -- 172.21.15.46:0/3878479917 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d80a7050 con 0x7f87e415a640 2024-03-29T08:30:04.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.065+0000 7f87e3400640 1 -- 172.21.15.46:0/3878479917 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d80ab980 con 0x7f87e415a640 2024-03-29T08:30:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.065+0000 7f87ea000640 1 -- 172.21.15.46:0/3878479917 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87e415a640 msgr2=0x7f87e40ccf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.065+0000 7f87ea000640 1 --2- 172.21.15.46:0/3878479917 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87e415a640 0x7f87e40ccf50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f87d80097c0 tx=0x7f87d801c4a0 comp rx=0 tx=0).stop 2024-03-29T08:30:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.065+0000 7f87ea000640 1 -- 172.21.15.46:0/3878479917 shutdown_connections 2024-03-29T08:30:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.065+0000 7f87ea000640 1 --2- 172.21.15.46:0/3878479917 >> v2:172.21.15.73:3300/0 conn(0x7f87e415eac0 0x7f87e40cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.065+0000 7f87ea000640 1 --2- 172.21.15.46:0/3878479917 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87e415a640 0x7f87e40ccf50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:04.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.065+0000 7f87ea000640 1 -- 172.21.15.46:0/3878479917 >> 172.21.15.46:0/3878479917 conn(0x7f87e40d0560 msgr2=0x7f87e40d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:04.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.065+0000 7f87ea000640 1 -- 172.21.15.46:0/3878479917 shutdown_connections 2024-03-29T08:30:04.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.065+0000 7f87ea000640 1 -- 172.21.15.46:0/3878479917 wait complete. 2024-03-29T08:30:04.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87ea000640 1 Processor -- start 2024-03-29T08:30:04.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87ea000640 1 -- start start 2024-03-29T08:30:04.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87ea000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87e40cf2c0 0x7f87e406ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:04.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87ea000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87e415a640 0x7f87e4152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:04.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87ea000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87e415eac0 0x7f87e406f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:04.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87ea000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40ca4e0 con 0x7f87e415a640 2024-03-29T08:30:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87ea000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e4159270 con 0x7f87e40cf2c0 2024-03-29T08:30:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87ea000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e4159510 con 0x7f87e415eac0 2024-03-29T08:30:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87e415a640 0x7f87e4152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87e3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87e415a640 0x7f87e4152730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45174/0 (socket says 172.21.15.46:45174) 2024-03-29T08:30:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87e3e00640 1 -- 172.21.15.46:0/4247015920 learned_addr learned my addr 172.21.15.46:0/4247015920 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87e9600640 1 --2- 172.21.15.46:0/4247015920 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87e415eac0 0x7f87e406f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87e8c00640 1 --2- 172.21.15.46:0/4247015920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87e40cf2c0 0x7f87e406ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87e3e00640 1 -- 172.21.15.46:0/4247015920 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87e415eac0 msgr2=0x7f87e406f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87e3e00640 1 --2- 172.21.15.46:0/4247015920 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87e415eac0 0x7f87e406f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87e3e00640 1 -- 172.21.15.46:0/4247015920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87e40cf2c0 msgr2=0x7f87e406ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87e3e00640 1 --2- 172.21.15.46:0/4247015920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87e40cf2c0 0x7f87e406ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87e3e00640 1 -- 172.21.15.46:0/4247015920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d8009420 con 0x7f87e415a640 2024-03-29T08:30:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87e3e00640 1 --2- 172.21.15.46:0/4247015920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87e415a640 0x7f87e4152730 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f87d000c4d0 tx=0x7f87d000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:04.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87e0c00640 1 -- 172.21.15.46:0/4247015920 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0005cf0 con 0x7f87e415a640 2024-03-29T08:30:04.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87e0c00640 1 -- 172.21.15.46:0/4247015920 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d0005e50 con 0x7f87e415a640 2024-03-29T08:30:04.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87ea000640 1 -- 172.21.15.46:0/4247015920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e40ca200 con 0x7f87e415a640 2024-03-29T08:30:04.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87e0c00640 1 -- 172.21.15.46:0/4247015920 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d0007270 con 0x7f87e415a640 2024-03-29T08:30:04.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87ea000640 1 -- 172.21.15.46:0/4247015920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e4159a90 con 0x7f87e415a640 2024-03-29T08:30:04.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.069+0000 7f87ea000640 1 -- 172.21.15.46:0/4247015920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e4159cb0 con 0x7f87e415a640 2024-03-29T08:30:04.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.073+0000 7f87e0c00640 1 -- 172.21.15.46:0/4247015920 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f87d00073d0 con 0x7f87e415a640 2024-03-29T08:30:04.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.073+0000 7f87e0c00640 1 --2- 172.21.15.46:0/4247015920 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f87c40666c0 0x7f87c4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:04.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.073+0000 7f87e0c00640 1 -- 172.21.15.46:0/4247015920 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f87d00837b0 con 0x7f87e415a640 2024-03-29T08:30:04.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.073+0000 7f87e8c00640 1 --2- 172.21.15.46:0/4247015920 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f87c40666c0 0x7f87c4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:04.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.073+0000 7f87e8c00640 1 --2- 172.21.15.46:0/4247015920 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f87c40666c0 0x7f87c4068b80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f87d80ad000 tx=0x7f87d80a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:04.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.077+0000 7f87e0c00640 1 -- 172.21.15.46:0/4247015920 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f87d0050a40 con 0x7f87e415a640 2024-03-29T08:30:04.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.273+0000 7f87ea000640 1 -- 172.21.15.46:0/4247015920 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87e404e040 con 0x7f87e415a640 2024-03-29T08:30:04.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.273+0000 7f87e0c00640 1 -- 172.21.15.46:0/4247015920 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+4741 (secure 0 0 0) 0x7f87d0054060 con 0x7f87e415a640 2024-03-29T08:30:04.274 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:04.275 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":106,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2506132630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2506132630},{"type":"v1","addr":"172.21.15.73:6836","nonce":2506132630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2108115501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2108115501},{"type":"v1","addr":"172.21.15.73:6837","nonce":2108115501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":9045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3210062895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3210062895},{"type":"v1","addr":"172.21.15.46:6833","nonce":3210062895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":106,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:29:14.470858+0000","modified":"2024-03-29T08:30:04.098015+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9660":{"gid":9660,"name":"a","rank":0,"incarnation":104,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.46:6835/3867471367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3867471367},{"type":"v1","addr":"172.21.15.46:6835","nonce":3867471367}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-29T08:30:04.275 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 106 2024-03-29T08:30:04.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.277+0000 7f87ea000640 1 -- 172.21.15.46:0/4247015920 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f87c40666c0 msgr2=0x7f87c4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:04.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.277+0000 7f87ea000640 1 --2- 172.21.15.46:0/4247015920 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f87c40666c0 0x7f87c4068b80 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f87d80ad000 tx=0x7f87d80a6000 comp rx=0 tx=0).stop 2024-03-29T08:30:04.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.277+0000 7f87ea000640 1 -- 172.21.15.46:0/4247015920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87e415a640 msgr2=0x7f87e4152730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:04.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.277+0000 7f87ea000640 1 --2- 172.21.15.46:0/4247015920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87e415a640 0x7f87e4152730 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f87d000c4d0 tx=0x7f87d000c990 comp rx=0 tx=0).stop 2024-03-29T08:30:04.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.277+0000 7f87ea000640 1 -- 172.21.15.46:0/4247015920 shutdown_connections 2024-03-29T08:30:04.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.277+0000 7f87ea000640 1 --2- 172.21.15.46:0/4247015920 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f87c40666c0 0x7f87c4068b80 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:04.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.277+0000 7f87ea000640 1 --2- 172.21.15.46:0/4247015920 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f87e415eac0 0x7f87e406f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:04.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.277+0000 7f87ea000640 1 --2- 172.21.15.46:0/4247015920 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87e415a640 0x7f87e4152730 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:04.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.277+0000 7f87ea000640 1 --2- 172.21.15.46:0/4247015920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f87e40cf2c0 0x7f87e406ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:04.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.277+0000 7f87ea000640 1 -- 172.21.15.46:0/4247015920 >> 172.21.15.46:0/4247015920 conn(0x7f87e40d0560 msgr2=0x7f87e40af3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:04.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.277+0000 7f87ea000640 1 -- 172.21.15.46:0/4247015920 shutdown_connections 2024-03-29T08:30:04.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:04.277+0000 7f87ea000640 1 -- 172.21.15.46:0/4247015920 wait complete. 2024-03-29T08:30:04.699 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 106, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:29:14.470858+0000', 'modified': '2024-03-29T08:30:04.098015+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 125, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9660}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9660': {'gid': 9660, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.46:6835/3867471367', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 3867471367}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 3867471367}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:30:04.699 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9660:up:rejoin 2024-03-29T08:30:05.700 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:30:05.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.853+0000 7f92d9200640 1 Processor -- start 2024-03-29T08:30:05.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.853+0000 7f92d9200640 1 -- start start 2024-03-29T08:30:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.853+0000 7f92d9200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92d40d0660 0x7f92d40d0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.853+0000 7f92d9200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f92d40cea40 0x7f92d40cee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.853+0000 7f92d9200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d40b14c0 con 0x7f92d415ccb0 2024-03-29T08:30:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.853+0000 7f92d9200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d40d0fc0 con 0x7f92d40cea40 2024-03-29T08:30:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.853+0000 7f92d9200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d415d090 con 0x7f92d40d0660 2024-03-29T08:30:05.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.853+0000 7f92d1400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92d40d0660 0x7f92d40d0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:05.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.853+0000 7f92d2800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f92d40cea40 0x7f92d40cee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:05.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.853+0000 7f92d1e00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f92d415ccb0 0x7f92d406bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:58060/0 (socket says 172.21.15.46:58060) 2024-03-29T08:30:05.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.853+0000 7f92d1e00640 1 -- 172.21.15.46:0/2275240697 learned_addr learned my addr 172.21.15.46:0/2275240697 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:05.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.853+0000 7f92d0a00640 1 -- 172.21.15.46:0/2275240697 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224125247 0 0) 0x7f92d40b14c0 con 0x7f92d415ccb0 2024-03-29T08:30:05.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.853+0000 7f92d1400640 1 -- 172.21.15.46:0/2275240697 >> v2:172.21.15.73:3300/0 conn(0x7f92d40cea40 msgr2=0x7f92d40cee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:05.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.853+0000 7f92d1400640 1 --2- 172.21.15.46:0/2275240697 >> v2:172.21.15.73:3300/0 conn(0x7f92d40cea40 0x7f92d40cee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:05.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.853+0000 7f92d1400640 1 -- 172.21.15.46:0/2275240697 >> v1:172.21.15.73:6789/0 conn(0x7f92d415ccb0 legacy=0x7f92d406bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:05.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.853+0000 7f92d1400640 1 -- 172.21.15.46:0/2275240697 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d40b08b0 con 0x7f92d40d0660 2024-03-29T08:30:05.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.857+0000 7f92d1400640 1 --2- 172.21.15.46:0/2275240697 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92d40d0660 0x7f92d40d0a80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f92c00097c0 tx=0x7f92c001ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f79329d5deffe6f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:05.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.857+0000 7f92d0a00640 1 -- 172.21.15.46:0/2275240697 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c00a8990 con 0x7f92d40d0660 2024-03-29T08:30:05.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.857+0000 7f92d0a00640 1 -- 172.21.15.46:0/2275240697 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c00a8af0 con 0x7f92d40d0660 2024-03-29T08:30:05.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.857+0000 7f92d0a00640 1 -- 172.21.15.46:0/2275240697 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c00a8e80 con 0x7f92d40d0660 2024-03-29T08:30:05.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.857+0000 7f92d9200640 1 -- 172.21.15.46:0/2275240697 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92d40d0660 msgr2=0x7f92d40d0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:05.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.857+0000 7f92d9200640 1 --2- 172.21.15.46:0/2275240697 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92d40d0660 0x7f92d40d0a80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f92c00097c0 tx=0x7f92c001ec90 comp rx=0 tx=0).stop 2024-03-29T08:30:05.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.857+0000 7f92d9200640 1 -- 172.21.15.46:0/2275240697 shutdown_connections 2024-03-29T08:30:05.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.857+0000 7f92d9200640 1 --2- 172.21.15.46:0/2275240697 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92d40d0660 0x7f92d40d0a80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:05.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.857+0000 7f92d9200640 1 --2- 172.21.15.46:0/2275240697 >> v2:172.21.15.73:3300/0 conn(0x7f92d40cea40 0x7f92d40cee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:05.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.857+0000 7f92d9200640 1 -- 172.21.15.46:0/2275240697 >> 172.21.15.46:0/2275240697 conn(0x7f92d4089fd0 msgr2=0x7f92d408a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:05.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.857+0000 7f92d9200640 1 -- 172.21.15.46:0/2275240697 shutdown_connections 2024-03-29T08:30:05.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.857+0000 7f92d9200640 1 -- 172.21.15.46:0/2275240697 wait complete. 2024-03-29T08:30:05.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d9200640 1 Processor -- start 2024-03-29T08:30:05.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d9200640 1 -- start start 2024-03-29T08:30:05.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d9200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92d40cea40 0x7f92d40c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:05.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d9200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92d40d0660 0x7f92d41f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:05.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d9200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92d415ccb0 0x7f92d4152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:05.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d9200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d415e2c0 con 0x7f92d415ccb0 2024-03-29T08:30:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d9200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d415e5a0 con 0x7f92d40cea40 2024-03-29T08:30:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d9200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d415e880 con 0x7f92d40d0660 2024-03-29T08:30:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92d415ccb0 0x7f92d4152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d2800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92d415ccb0 0x7f92d4152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45178/0 (socket says 172.21.15.46:45178) 2024-03-29T08:30:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d2800640 1 -- 172.21.15.46:0/121617591 learned_addr learned my addr 172.21.15.46:0/121617591 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d1e00640 1 --2- 172.21.15.46:0/121617591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92d40cea40 0x7f92d40c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d1400640 1 --2- 172.21.15.46:0/121617591 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92d40d0660 0x7f92d41f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d2800640 1 -- 172.21.15.46:0/121617591 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92d40d0660 msgr2=0x7f92d41f36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d2800640 1 --2- 172.21.15.46:0/121617591 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92d40d0660 0x7f92d41f36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d2800640 1 -- 172.21.15.46:0/121617591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92d40cea40 msgr2=0x7f92d40c6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d2800640 1 --2- 172.21.15.46:0/121617591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92d40cea40 0x7f92d40c6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d2800640 1 -- 172.21.15.46:0/121617591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c0009420 con 0x7f92d415ccb0 2024-03-29T08:30:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d2800640 1 --2- 172.21.15.46:0/121617591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92d415ccb0 0x7f92d4152d50 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00efc0 tx=0x7f92cc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92ca000640 1 -- 172.21.15.46:0/121617591 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc009ba0 con 0x7f92d415ccb0 2024-03-29T08:30:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92ca000640 1 -- 172.21.15.46:0/121617591 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92cc00f040 con 0x7f92d415ccb0 2024-03-29T08:30:05.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92ca000640 1 -- 172.21.15.46:0/121617591 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92cc0146c0 con 0x7f92d415ccb0 2024-03-29T08:30:05.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d9200640 1 -- 172.21.15.46:0/121617591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d415eb60 con 0x7f92d415ccb0 2024-03-29T08:30:05.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d9200640 1 -- 172.21.15.46:0/121617591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d415f120 con 0x7f92d415ccb0 2024-03-29T08:30:05.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.861+0000 7f92d9200640 1 -- 172.21.15.46:0/121617591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d415f310 con 0x7f92d415ccb0 2024-03-29T08:30:05.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.865+0000 7f92ca000640 1 -- 172.21.15.46:0/121617591 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f92cc014820 con 0x7f92d415ccb0 2024-03-29T08:30:05.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.865+0000 7f92ca000640 1 --2- 172.21.15.46:0/121617591 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f92b4066690 0x7f92b4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:05.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.865+0000 7f92ca000640 1 -- 172.21.15.46:0/121617591 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f92cc084c60 con 0x7f92d415ccb0 2024-03-29T08:30:05.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.865+0000 7f92d1e00640 1 --2- 172.21.15.46:0/121617591 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f92b4066690 0x7f92b4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:05.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.865+0000 7f92d1e00640 1 --2- 172.21.15.46:0/121617591 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f92b4066690 0x7f92b4068b50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f92c4007fd0 tx=0x7f92c4007d60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:05.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:05.869+0000 7f92ca000640 1 -- 172.21.15.46:0/121617591 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f92cc051f80 con 0x7f92d415ccb0 2024-03-29T08:30:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.065+0000 7f92d9200640 1 -- 172.21.15.46:0/121617591 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92d4066c90 con 0x7f92d415ccb0 2024-03-29T08:30:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.065+0000 7f92ca000640 1 -- 172.21.15.46:0/121617591 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4741 (secure 0 0 0) 0x7f92cc018020 con 0x7f92d415ccb0 2024-03-29T08:30:06.065 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:06.065 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":107,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2506132630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2506132630},{"type":"v1","addr":"172.21.15.73:6836","nonce":2506132630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2108115501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2108115501},{"type":"v1","addr":"172.21.15.73:6837","nonce":2108115501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":9045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3210062895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3210062895},{"type":"v1","addr":"172.21.15.46:6833","nonce":3210062895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:29:14.470858+0000","modified":"2024-03-29T08:30:05.103674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9660":{"gid":9660,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/3867471367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3867471367},{"type":"v1","addr":"172.21.15.46:6835","nonce":3867471367}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-29T08:30:06.065 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 107 2024-03-29T08:30:06.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.065+0000 7f92d9200640 1 -- 172.21.15.46:0/121617591 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f92b4066690 msgr2=0x7f92b4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:06.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.065+0000 7f92d9200640 1 --2- 172.21.15.46:0/121617591 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f92b4066690 0x7f92b4068b50 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f92c4007fd0 tx=0x7f92c4007d60 comp rx=0 tx=0).stop 2024-03-29T08:30:06.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.065+0000 7f92d9200640 1 -- 172.21.15.46:0/121617591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92d415ccb0 msgr2=0x7f92d4152d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:06.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.065+0000 7f92d9200640 1 --2- 172.21.15.46:0/121617591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92d415ccb0 0x7f92d4152d50 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f92cc00efc0 tx=0x7f92cc00c490 comp rx=0 tx=0).stop 2024-03-29T08:30:06.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.065+0000 7f92d9200640 1 -- 172.21.15.46:0/121617591 shutdown_connections 2024-03-29T08:30:06.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.065+0000 7f92d9200640 1 --2- 172.21.15.46:0/121617591 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f92b4066690 0x7f92b4068b50 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.065+0000 7f92d9200640 1 --2- 172.21.15.46:0/121617591 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92d415ccb0 0x7f92d4152d50 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.065+0000 7f92d9200640 1 --2- 172.21.15.46:0/121617591 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f92d40d0660 0x7f92d41f36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.065+0000 7f92d9200640 1 --2- 172.21.15.46:0/121617591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92d40cea40 0x7f92d40c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.065+0000 7f92d9200640 1 -- 172.21.15.46:0/121617591 >> 172.21.15.46:0/121617591 conn(0x7f92d4089fd0 msgr2=0x7f92d40ad640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:06.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.065+0000 7f92d9200640 1 -- 172.21.15.46:0/121617591 shutdown_connections 2024-03-29T08:30:06.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.065+0000 7f92d9200640 1 -- 172.21.15.46:0/121617591 wait complete. 2024-03-29T08:30:06.085 INFO:tasks.ceph.mds.a.smithi046.stderr:2024-03-29T08:30:06.085+0000 7fec71e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:30:06.489 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 107, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-29T08:29:14.470858+0000', 'modified': '2024-03-29T08:30:05.103674+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 125, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9660}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9660': {'gid': 9660, 'name': 'a', 'rank': 0, 'incarnation': 104, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.46:6835/3867471367', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6834', 'nonce': 3867471367}, {'type': 'v1', 'addr': '172.21.15.46:6835', 'nonce': 3867471367}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-03-29T08:30:06.489 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-29T08:30:06.489 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-29T08:30:06.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.633+0000 7f7232200640 1 Processor -- start 2024-03-29T08:30:06.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.633+0000 7f7232200640 1 -- start start 2024-03-29T08:30:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.633+0000 7f7232200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f722c158300 0x7f722c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.633+0000 7f7232200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f722c06c390 con 0x7f722c0709b0 2024-03-29T08:30:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.633+0000 7f7232200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f722c06d600 con 0x7f722c0af530 2024-03-29T08:30:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.633+0000 7f7232200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f722c0afff0 con 0x7f722c158300 2024-03-29T08:30:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.633+0000 7f722b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f722c158300 0x7f722c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.633+0000 7f722be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f722c0709b0 0x7f722c15bbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41986/0 (socket says 172.21.15.46:41986) 2024-03-29T08:30:06.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.633+0000 7f722b400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f722c158300 0x7f722c06d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48812/0 (socket says 172.21.15.46:48812) 2024-03-29T08:30:06.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.633+0000 7f722be00640 1 -- 172.21.15.46:0/3782393113 learned_addr learned my addr 172.21.15.46:0/3782393113 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:06.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.633+0000 7f722b400640 1 -- 172.21.15.46:0/3782393113 >> v1:172.21.15.73:6789/0 conn(0x7f722c0af530 legacy=0x7f722c0702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:06.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.633+0000 7f722b400640 1 -- 172.21.15.46:0/3782393113 >> v1:172.21.15.46:6789/0 conn(0x7f722c0709b0 legacy=0x7f722c15bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:06.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.633+0000 7f722b400640 1 -- 172.21.15.46:0/3782393113 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c06c5e0 con 0x7f722c158300 2024-03-29T08:30:06.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.633+0000 7f722aa00640 1 -- 172.21.15.46:0/3782393113 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2989113855 0 0) 0x7f722c06c390 con 0x7f722c0709b0 2024-03-29T08:30:06.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.637+0000 7f722b400640 1 --2- 172.21.15.46:0/3782393113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c158300 0x7f722c06d0c0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f721c0097c0 tx=0x7f721c01c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b1765028d7beba server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:06.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.637+0000 7f722aa00640 1 -- 172.21.15.46:0/3782393113 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c0ab600 con 0x7f722c158300 2024-03-29T08:30:06.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.637+0000 7f722aa00640 1 -- 172.21.15.46:0/3782393113 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c0a7050 con 0x7f722c158300 2024-03-29T08:30:06.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.637+0000 7f722aa00640 1 -- 172.21.15.46:0/3782393113 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c0ab980 con 0x7f722c158300 2024-03-29T08:30:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.637+0000 7f7232200640 1 -- 172.21.15.46:0/3782393113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c158300 msgr2=0x7f722c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.637+0000 7f7232200640 1 --2- 172.21.15.46:0/3782393113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c158300 0x7f722c06d0c0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f721c0097c0 tx=0x7f721c01c350 comp rx=0 tx=0).stop 2024-03-29T08:30:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.637+0000 7f7232200640 1 -- 172.21.15.46:0/3782393113 shutdown_connections 2024-03-29T08:30:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.637+0000 7f7232200640 1 --2- 172.21.15.46:0/3782393113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c158300 0x7f722c06d0c0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.637+0000 7f7232200640 1 -- 172.21.15.46:0/3782393113 >> 172.21.15.46:0/3782393113 conn(0x7f722c066080 msgr2=0x7f722c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.637+0000 7f7232200640 1 -- 172.21.15.46:0/3782393113 shutdown_connections 2024-03-29T08:30:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.637+0000 7f7232200640 1 -- 172.21.15.46:0/3782393113 wait complete. 2024-03-29T08:30:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7232200640 1 Processor -- start 2024-03-29T08:30:06.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7232200640 1 -- start start 2024-03-29T08:30:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7232200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f722c0709b0 0x7f722c13e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7232200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c0af530 0x7f722c0b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7232200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f722c158300 0x7f722c13b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7232200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0b18d0 con 0x7f722c0af530 2024-03-29T08:30:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7232200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0b1bb0 con 0x7f722c158300 2024-03-29T08:30:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7232200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0b1e90 con 0x7f722c0709b0 2024-03-29T08:30:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7223e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c0af530 0x7f722c0b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7223e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c0af530 0x7f722c0b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48816/0 (socket says 172.21.15.46:48816) 2024-03-29T08:30:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7223e00640 1 -- 172.21.15.46:0/1036289475 learned_addr learned my addr 172.21.15.46:0/1036289475 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f722b400640 1 --2- 172.21.15.46:0/1036289475 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f722c0709b0 0x7f722c13e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f722be00640 1 --2- 172.21.15.46:0/1036289475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f722c158300 0x7f722c13b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7223e00640 1 -- 172.21.15.46:0/1036289475 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f722c0709b0 msgr2=0x7f722c13e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7223e00640 1 --2- 172.21.15.46:0/1036289475 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f722c0709b0 0x7f722c13e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7223e00640 1 -- 172.21.15.46:0/1036289475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f722c158300 msgr2=0x7f722c13b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7223e00640 1 --2- 172.21.15.46:0/1036289475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f722c158300 0x7f722c13b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7223e00640 1 -- 172.21.15.46:0/1036289475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c009420 con 0x7f722c0af530 2024-03-29T08:30:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f722be00640 1 --2- 172.21.15.46:0/1036289475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f722c158300 0x7f722c13b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f722b400640 1 --2- 172.21.15.46:0/1036289475 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f722c0709b0 0x7f722c13e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7223e00640 1 --2- 172.21.15.46:0/1036289475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c0af530 0x7f722c0b2970 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f721400c500 tx=0x7f721400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7223400640 1 -- 172.21.15.46:0/1036289475 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7214005220 con 0x7f722c0af530 2024-03-29T08:30:06.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7223400640 1 -- 172.21.15.46:0/1036289475 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7214005380 con 0x7f722c0af530 2024-03-29T08:30:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7232200640 1 -- 172.21.15.46:0/1036289475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722c0b2170 con 0x7f722c0af530 2024-03-29T08:30:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7223400640 1 -- 172.21.15.46:0/1036289475 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7214007270 con 0x7f722c0af530 2024-03-29T08:30:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.641+0000 7f7232200640 1 -- 172.21.15.46:0/1036289475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722c13ed90 con 0x7f722c0af530 2024-03-29T08:30:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f7223400640 1 -- 172.21.15.46:0/1036289475 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f72140073d0 con 0x7f722c0af530 2024-03-29T08:30:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f7223400640 1 --2- 172.21.15.46:0/1036289475 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7208066690 0x7f7208068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f7223400640 1 -- 172.21.15.46:0/1036289475 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f7214083c00 con 0x7f722c0af530 2024-03-29T08:30:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f723561c000 1 Processor -- start 2024-03-29T08:30:06.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f722b400640 1 --2- 172.21.15.46:0/1036289475 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7208066690 0x7f7208068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f723561c000 1 -- start start 2024-03-29T08:30:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f722b400640 1 --2- 172.21.15.46:0/1036289475 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7208066690 0x7f7208068b50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f721c0ad000 tx=0x7f721c0a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f723561c000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7fe8db0 0x5606b7fe9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f723561c000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe96d0 0x5606b7f9ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f723561c000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7f9f4c0 0x5606b7f9f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f723561c000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b7e675a0 con 0x5606b7fe96d0 2024-03-29T08:30:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f723561c000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b7df4af0 con 0x5606b7f9f4c0 2024-03-29T08:30:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f723561c000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606b7df20c0 con 0x5606b7fe8db0 2024-03-29T08:30:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f7223e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe96d0 0x5606b7f9ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f7223e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe96d0 0x5606b7f9ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48826/0 (socket says 172.21.15.46:48826) 2024-03-29T08:30:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f7223e00640 1 -- 172.21.15.46:0/1671769401 learned_addr learned my addr 172.21.15.46:0/1671769401 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f722b400640 1 --2- 172.21.15.46:0/1671769401 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7fe8db0 0x5606b7fe9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f722be00640 1 --2- 172.21.15.46:0/1671769401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7f9f4c0 0x5606b7f9f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f7223e00640 1 -- 172.21.15.46:0/1671769401 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7fe8db0 msgr2=0x5606b7fe9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f7223e00640 1 --2- 172.21.15.46:0/1671769401 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7fe8db0 0x5606b7fe9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f7223e00640 1 -- 172.21.15.46:0/1671769401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7f9f4c0 msgr2=0x5606b7f9f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f7223e00640 1 --2- 172.21.15.46:0/1671769401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7f9f4c0 0x5606b7f9f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f7223e00640 1 -- 172.21.15.46:0/1671769401 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606b7ae9890 con 0x5606b7fe96d0 2024-03-29T08:30:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f722b400640 1 --2- 172.21.15.46:0/1671769401 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7fe8db0 0x5606b7fe9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f722be00640 1 --2- 172.21.15.46:0/1671769401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7f9f4c0 0x5606b7f9f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.645+0000 7f7223e00640 1 --2- 172.21.15.46:0/1671769401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe96d0 0x5606b7f9ef80 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f7214050fb0 tx=0x7f7214054720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7a7ea6283af34b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f721be00640 1 -- 172.21.15.46:0/1671769401 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72140588e0 con 0x5606b7fe96d0 2024-03-29T08:30:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f721be00640 1 -- 172.21.15.46:0/1671769401 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721405cea0 con 0x5606b7fe96d0 2024-03-29T08:30:06.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f721be00640 1 -- 172.21.15.46:0/1671769401 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72140511c0 con 0x5606b7fe96d0 2024-03-29T08:30:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 1 -- 172.21.15.46:0/1671769401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe96d0 msgr2=0x5606b7f9ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 1 --2- 172.21.15.46:0/1671769401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe96d0 0x5606b7f9ef80 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f7214050fb0 tx=0x7f7214054720 comp rx=0 tx=0).stop 2024-03-29T08:30:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 1 -- 172.21.15.46:0/1671769401 shutdown_connections 2024-03-29T08:30:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 1 --2- 172.21.15.46:0/1671769401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7f9f4c0 0x5606b7f9f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 1 --2- 172.21.15.46:0/1671769401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe96d0 0x5606b7f9ef80 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 1 --2- 172.21.15.46:0/1671769401 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7fe8db0 0x5606b7fe9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 1 -- 172.21.15.46:0/1671769401 >> 172.21.15.46:0/1671769401 conn(0x5606b7fa5830 msgr2=0x5606b7f9d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 1 -- 172.21.15.46:0/1671769401 shutdown_connections 2024-03-29T08:30:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 1 -- 172.21.15.46:0/1671769401 wait complete. 2024-03-29T08:30:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 1 -- start start 2024-03-29T08:30:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 1 Processor -- start 2024-03-29T08:30:06.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7f9f4c0 0x5606b7fa3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7fe8db0 0x5606b7fa3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7fe96d0 0x5606b7fa3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b7ded9b0 con 0x5606b7f9f4c0 2024-03-29T08:30:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b7d8cc00 con 0x5606b7fe8db0 2024-03-29T08:30:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606b7dab070 con 0x5606b7fe96d0 2024-03-29T08:30:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f723561c000 10 client.0 _ll_register_callbacks cb 0x5606b7f8b0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:30:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f722b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7f9f4c0 0x5606b7fa3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f722b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7f9f4c0 0x5606b7fa3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48838/0 (socket says 172.21.15.46:48838) 2024-03-29T08:30:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f722b400640 1 -- 172.21.15.46:0/793405815 learned_addr learned my addr 172.21.15.46:0/793405815 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f722be00640 1 --2- 172.21.15.46:0/793405815 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7fe96d0 0x5606b7fa3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f7223e00640 1 --2- 172.21.15.46:0/793405815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7fe8db0 0x5606b7fa3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f722b400640 1 -- 172.21.15.46:0/793405815 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7fe96d0 msgr2=0x5606b7fa3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:06.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f722b400640 1 --2- 172.21.15.46:0/793405815 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7fe96d0 0x5606b7fa3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f722b400640 1 -- 172.21.15.46:0/793405815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7fe8db0 msgr2=0x5606b7fa3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f722b400640 1 --2- 172.21.15.46:0/793405815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7fe8db0 0x5606b7fa3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f722b400640 1 -- 172.21.15.46:0/793405815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722c13ed90 con 0x5606b7f9f4c0 2024-03-29T08:30:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f722b400640 1 --2- 172.21.15.46:0/793405815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7f9f4c0 0x5606b7fa3230 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f721c0097c0 tx=0x7f721c01ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc6de2ecb30b1eb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.649+0000 7f7218c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:30:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f7218c00640 1 -- 172.21.15.46:0/793405815 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c0b7780 con 0x5606b7f9f4c0 2024-03-29T08:30:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f7218c00640 1 -- 172.21.15.46:0/793405815 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f721c0b78e0 con 0x5606b7f9f4c0 2024-03-29T08:30:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 1 -- 172.21.15.46:0/793405815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606b7daa5f0 con 0x5606b7f9f4c0 2024-03-29T08:30:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f7218c00640 1 -- 172.21.15.46:0/793405815 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f721c0b7aa0 con 0x5606b7f9f4c0 2024-03-29T08:30:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 10 client.9741 fetch_fsmap learned FSMap version 107 2024-03-29T08:30:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 1 -- 172.21.15.46:0/793405815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x5606b7db2ab0 con 0x5606b7f9f4c0 2024-03-29T08:30:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f7218c00640 1 -- 172.21.15.46:0/793405815 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5606b7daa5f0 con 0x5606b7f9f4c0 2024-03-29T08:30:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f7218c00640 1 -- 172.21.15.46:0/793405815 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 107) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f721c0b7c00 con 0x5606b7f9f4c0 2024-03-29T08:30:06.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 10 client.9741 fetch_fsmap finished waiting for FSMap version 107 2024-03-29T08:30:06.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 10 client.9741 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T08:30:06.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 20 client.9741 populate_metadata read hostname 'smithi046' 2024-03-29T08:30:06.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 1 --2- 172.21.15.46:0/793405815 >> [v2:172.21.15.46:6834/3867471367,v1:172.21.15.46:6835/3867471367] conn(0x5606b7fa46c0 0x5606b7ffa030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 4 client.9741 mds_command: new command op to 9660 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-29T08:30:06.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 1 -- 172.21.15.46:0/793405815 --> [v2:172.21.15.46:6834/3867471367,v1:172.21.15.46:6835/3867471367] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606b7dea790 con 0x5606b7fa46c0 2024-03-29T08:30:06.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f7223e00640 1 --2- 172.21.15.46:0/793405815 >> [v2:172.21.15.46:6834/3867471367,v1:172.21.15.46:6835/3867471367] conn(0x5606b7fa46c0 0x5606b7ffa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f7223e00640 1 --2- 172.21.15.46:0/793405815 >> [v2:172.21.15.46:6834/3867471367,v1:172.21.15.46:6835/3867471367] conn(0x5606b7fa46c0 0x5606b7ffa030 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1007bac096d0666 server_cookie=b29082301066f63a in_seq=0 out_seq=0 2024-03-29T08:30:06.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f7218c00640 10 client.9741 ms_handle_connect on v2:172.21.15.46:6834/3867471367 2024-03-29T08:30:06.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f7218c00640 1 -- 172.21.15.46:0/793405815 <== mds.0 v2:172.21.15.46:6834/3867471367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x5606b7dea790 con 0x5606b7fa46c0 2024-03-29T08:30:06.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f7218c00640 10 client.9741 handle_command_reply: tid=0 2024-03-29T08:30:06.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 1 client.9741 shutdown 2024-03-29T08:30:06.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 1 -- 172.21.15.46:0/793405815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7f9f4c0 msgr2=0x5606b7fa3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 1 --2- 172.21.15.46:0/793405815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7f9f4c0 0x5606b7fa3230 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f721c0097c0 tx=0x7f721c01ab90 comp rx=0 tx=0).stop 2024-03-29T08:30:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 1 -- 172.21.15.46:0/793405815 shutdown_connections 2024-03-29T08:30:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 1 --2- 172.21.15.46:0/793405815 >> [v2:172.21.15.46:6834/3867471367,v1:172.21.15.46:6835/3867471367] conn(0x5606b7fa46c0 0x5606b7ffa030 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 1 --2- 172.21.15.46:0/793405815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7fe8db0 0x5606b7fa3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 1 --2- 172.21.15.46:0/793405815 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7fe96d0 0x5606b7fa3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f7218c00640 0 client.9741 ms_handle_reset on v2:172.21.15.46:6834/3867471367 2024-03-29T08:30:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 1 --2- 172.21.15.46:0/793405815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7f9f4c0 0x5606b7fa3230 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 1 -- 172.21.15.46:0/793405815 >> 172.21.15.46:0/793405815 conn(0x5606b7fa5830 msgr2=0x5606b7f9df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 1 -- 172.21.15.46:0/793405815 shutdown_connections 2024-03-29T08:30:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 1 -- 172.21.15.46:0/793405815 wait complete. 2024-03-29T08:30:06.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.653+0000 7f723561c000 20 client.9741 trim_cache size 0 max 16384 2024-03-29T08:30:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f723561c000 1 Processor -- start 2024-03-29T08:30:06.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f723561c000 1 -- start start 2024-03-29T08:30:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f723561c000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7f9f4c0 0x5606b8042850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f723561c000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe8db0 0x5606b8042d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f723561c000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7fe96d0 0x5606b80432d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f723561c000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b7aaa940 con 0x5606b7fe8db0 2024-03-29T08:30:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f723561c000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b7ebe3e0 con 0x5606b7fe96d0 2024-03-29T08:30:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f723561c000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606b7db4c60 con 0x5606b7f9f4c0 2024-03-29T08:30:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f7223e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe8db0 0x5606b8042d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f7223e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe8db0 0x5606b8042d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48844/0 (socket says 172.21.15.46:48844) 2024-03-29T08:30:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f7223e00640 1 -- 172.21.15.46:0/1924174168 learned_addr learned my addr 172.21.15.46:0/1924174168 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f722be00640 1 --2- 172.21.15.46:0/1924174168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7fe96d0 0x5606b80432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f722b400640 1 --2- 172.21.15.46:0/1924174168 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7f9f4c0 0x5606b8042850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f7223e00640 1 -- 172.21.15.46:0/1924174168 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7f9f4c0 msgr2=0x5606b8042850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:06.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f7223e00640 1 --2- 172.21.15.46:0/1924174168 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7f9f4c0 0x5606b8042850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f7223e00640 1 -- 172.21.15.46:0/1924174168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7fe96d0 msgr2=0x5606b80432d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f7223e00640 1 --2- 172.21.15.46:0/1924174168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7fe96d0 0x5606b80432d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f7223e00640 1 -- 172.21.15.46:0/1924174168 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606b7d98c00 con 0x5606b7fe8db0 2024-03-29T08:30:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f722be00640 1 --2- 172.21.15.46:0/1924174168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7fe96d0 0x5606b80432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.669+0000 7f722b400640 1 --2- 172.21.15.46:0/1924174168 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7f9f4c0 0x5606b8042850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f7223e00640 1 --2- 172.21.15.46:0/1924174168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe8db0 0x5606b8042d90 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f721400cec0 tx=0x7f7214058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1338b6fb9407ec91 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f7220c00640 1 -- 172.21.15.46:0/1924174168 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7214071040 con 0x5606b7fe8db0 2024-03-29T08:30:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f7220c00640 1 -- 172.21.15.46:0/1924174168 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7214058960 con 0x5606b7fe8db0 2024-03-29T08:30:06.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f7220c00640 1 -- 172.21.15.46:0/1924174168 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721405cea0 con 0x5606b7fe8db0 2024-03-29T08:30:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 1 -- 172.21.15.46:0/1924174168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe8db0 msgr2=0x5606b8042d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 1 --2- 172.21.15.46:0/1924174168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe8db0 0x5606b8042d90 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f721400cec0 tx=0x7f7214058000 comp rx=0 tx=0).stop 2024-03-29T08:30:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 1 -- 172.21.15.46:0/1924174168 shutdown_connections 2024-03-29T08:30:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 1 --2- 172.21.15.46:0/1924174168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7fe96d0 0x5606b80432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 1 --2- 172.21.15.46:0/1924174168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe8db0 0x5606b8042d90 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 1 --2- 172.21.15.46:0/1924174168 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7f9f4c0 0x5606b8042850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 1 -- 172.21.15.46:0/1924174168 >> 172.21.15.46:0/1924174168 conn(0x5606b7fa5830 msgr2=0x5606b7fa27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 1 -- 172.21.15.46:0/1924174168 shutdown_connections 2024-03-29T08:30:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 1 -- 172.21.15.46:0/1924174168 wait complete. 2024-03-29T08:30:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 1 -- start start 2024-03-29T08:30:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 1 Processor -- start 2024-03-29T08:30:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7f9f4c0 0x5606b8043430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7fe8db0 0x5606b7fe6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe96d0 0x5606b7fe6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b7df0510 con 0x5606b7fe96d0 2024-03-29T08:30:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606b7ffe4b0 con 0x5606b7fe8db0 2024-03-29T08:30:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606b7da6f80 con 0x5606b7f9f4c0 2024-03-29T08:30:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f723561c000 10 client.0 _ll_register_callbacks cb 0x5606b7db2590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-29T08:30:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f722be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe96d0 0x5606b7fe6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f722be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe96d0 0x5606b7fe6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48860/0 (socket says 172.21.15.46:48860) 2024-03-29T08:30:06.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f722be00640 1 -- 172.21.15.46:0/3468071794 learned_addr learned my addr 172.21.15.46:0/3468071794 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f7223e00640 1 --2- 172.21.15.46:0/3468071794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7fe8db0 0x5606b7fe6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f722b400640 1 --2- 172.21.15.46:0/3468071794 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7f9f4c0 0x5606b8043430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f722be00640 1 -- 172.21.15.46:0/3468071794 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7f9f4c0 msgr2=0x5606b8043430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f722be00640 1 --2- 172.21.15.46:0/3468071794 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7f9f4c0 0x5606b8043430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f722be00640 1 -- 172.21.15.46:0/3468071794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7fe8db0 msgr2=0x5606b7fe6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f722be00640 1 --2- 172.21.15.46:0/3468071794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7fe8db0 0x5606b7fe6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f722be00640 1 -- 172.21.15.46:0/3468071794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5606b7ae9890 con 0x5606b7fe96d0 2024-03-29T08:30:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f722be00640 1 --2- 172.21.15.46:0/3468071794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe96d0 0x5606b7fe6940 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7224007810 tx=0x7f7224001a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce189a8c2a5b251 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f7218c00640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-03-29T08:30:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.673+0000 7f7218c00640 1 -- 172.21.15.46:0/3468071794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7224017660 con 0x5606b7fe96d0 2024-03-29T08:30:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7218c00640 1 -- 172.21.15.46:0/3468071794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f72240177c0 con 0x5606b7fe96d0 2024-03-29T08:30:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 1 -- 172.21.15.46:0/3468071794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606b7dbd460 con 0x5606b7fe96d0 2024-03-29T08:30:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7218c00640 1 -- 172.21.15.46:0/3468071794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7224017980 con 0x5606b7fe96d0 2024-03-29T08:30:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 10 client.9747 fetch_fsmap learned FSMap version 107 2024-03-29T08:30:06.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 1 -- 172.21.15.46:0/3468071794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x5606b7ae9cc0 con 0x5606b7fe96d0 2024-03-29T08:30:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7218c00640 1 -- 172.21.15.46:0/3468071794 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5606b7dbd460 con 0x5606b7fe96d0 2024-03-29T08:30:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7218c00640 1 -- 172.21.15.46:0/3468071794 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 107) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f7224017ae0 con 0x5606b7fe96d0 2024-03-29T08:30:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 10 client.9747 fetch_fsmap finished waiting for FSMap version 107 2024-03-29T08:30:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 10 client.9747 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-29T08:30:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 20 client.9747 populate_metadata read hostname 'smithi046' 2024-03-29T08:30:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 1 --2- 172.21.15.46:0/3468071794 >> [v2:172.21.15.46:6834/3867471367,v1:172.21.15.46:6835/3867471367] conn(0x5606b7fe6e80 0x5606b7fe7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 4 client.9747 mds_command: new command op to 9660 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-29T08:30:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 1 -- 172.21.15.46:0/3468071794 --> [v2:172.21.15.46:6834/3867471367,v1:172.21.15.46:6835/3867471367] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5606b7db69c0 con 0x5606b7fe6e80 2024-03-29T08:30:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f722b400640 1 --2- 172.21.15.46:0/3468071794 >> [v2:172.21.15.46:6834/3867471367,v1:172.21.15.46:6835/3867471367] conn(0x5606b7fe6e80 0x5606b7fe7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f722b400640 1 --2- 172.21.15.46:0/3468071794 >> [v2:172.21.15.46:6834/3867471367,v1:172.21.15.46:6835/3867471367] conn(0x5606b7fe6e80 0x5606b7fe7260 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29df70a2713a64b9 server_cookie=e11eb76bcac03d6 in_seq=0 out_seq=0 2024-03-29T08:30:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7218c00640 10 client.9747 ms_handle_connect on v2:172.21.15.46:6834/3867471367 2024-03-29T08:30:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7218c00640 1 -- 172.21.15.46:0/3468071794 <== mds.0 v2:172.21.15.46:6834/3867471367 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5606b7db69c0 con 0x5606b7fe6e80 2024-03-29T08:30:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7218c00640 10 client.9747 handle_command_reply: tid=0 2024-03-29T08:30:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 1 client.9747 shutdown 2024-03-29T08:30:06.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 1 -- 172.21.15.46:0/3468071794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe96d0 msgr2=0x5606b7fe6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 1 --2- 172.21.15.46:0/3468071794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe96d0 0x5606b7fe6940 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f7224007810 tx=0x7f7224001a10 comp rx=0 tx=0).stop 2024-03-29T08:30:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 1 -- 172.21.15.46:0/3468071794 shutdown_connections 2024-03-29T08:30:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 1 --2- 172.21.15.46:0/3468071794 >> [v2:172.21.15.46:6834/3867471367,v1:172.21.15.46:6835/3867471367] conn(0x5606b7fe6e80 0x5606b7fe7260 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 1 --2- 172.21.15.46:0/3468071794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5606b7fe8db0 0x5606b7fe6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 1 --2- 172.21.15.46:0/3468071794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606b7fe96d0 0x5606b7fe6940 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 1 --2- 172.21.15.46:0/3468071794 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x5606b7f9f4c0 0x5606b8043430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7218c00640 0 client.9747 ms_handle_reset on v2:172.21.15.46:6834/3867471367 2024-03-29T08:30:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 1 -- 172.21.15.46:0/3468071794 >> 172.21.15.46:0/3468071794 conn(0x5606b7fa5830 msgr2=0x5606b7fa2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 1 -- 172.21.15.46:0/3468071794 shutdown_connections 2024-03-29T08:30:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 1 -- 172.21.15.46:0/3468071794 wait complete. 2024-03-29T08:30:06.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f723561c000 20 client.9747 trim_cache size 0 max 16384 2024-03-29T08:30:06.685 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:30:06.686 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "e9b7960f-ec1f-4787-95c3-c806094d37f0", 2024-03-29T08:30:06.686 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-03-29T08:30:06.686 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 9660, 2024-03-29T08:30:06.686 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-03-29T08:30:06.686 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-03-29T08:30:06.686 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-03-29T08:30:06.686 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 4.586079689, 2024-03-29T08:30:06.686 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 107, 2024-03-29T08:30:06.686 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 125, 2024-03-29T08:30:06.686 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 125, 2024-03-29T08:30:06.686 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 6.5424046599999999 2024-03-29T08:30:06.686 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:30:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7232200640 1 -- 172.21.15.46:0/1036289475 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7208066690 msgr2=0x7f7208068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7232200640 1 --2- 172.21.15.46:0/1036289475 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7208066690 0x7f7208068b50 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f721c0ad000 tx=0x7f721c0a6000 comp rx=0 tx=0).stop 2024-03-29T08:30:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7232200640 1 -- 172.21.15.46:0/1036289475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c0af530 msgr2=0x7f722c0b2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7232200640 1 --2- 172.21.15.46:0/1036289475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c0af530 0x7f722c0b2970 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f721400c500 tx=0x7f721400c9c0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7232200640 1 -- 172.21.15.46:0/1036289475 shutdown_connections 2024-03-29T08:30:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7232200640 1 --2- 172.21.15.46:0/1036289475 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7208066690 0x7f7208068b50 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7232200640 1 --2- 172.21.15.46:0/1036289475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f722c158300 0x7f722c13b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7232200640 1 --2- 172.21.15.46:0/1036289475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f722c0af530 0x7f722c0b2970 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7232200640 1 --2- 172.21.15.46:0/1036289475 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f722c0709b0 0x7f722c13e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7232200640 1 -- 172.21.15.46:0/1036289475 >> 172.21.15.46:0/1036289475 conn(0x7f722c066080 msgr2=0x7f722c0702a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7232200640 1 -- 172.21.15.46:0/1036289475 shutdown_connections 2024-03-29T08:30:06.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.677+0000 7f7232200640 1 -- 172.21.15.46:0/1036289475 wait complete. 2024-03-29T08:30:06.697 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-29T08:30:06.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce83c00640 1 Processor -- start 2024-03-29T08:30:06.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce83c00640 1 -- start start 2024-03-29T08:30:06.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce83c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce7c1593d0 0x7fce7c06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce83c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fce7c15d230 0x7fce7c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce83c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fce7c15dbd0 0x7fce7c15bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce83c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7c06d130 con 0x7fce7c15dbd0 2024-03-29T08:30:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce83c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7c15b8f0 con 0x7fce7c15d230 2024-03-29T08:30:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce83c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c15ad50 con 0x7fce7c1593d0 2024-03-29T08:30:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce81c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fce7c15dbd0 0x7fce7c15bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce81c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fce7c15dbd0 0x7fce7c15bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48870/0 (socket says 172.21.15.46:48870) 2024-03-29T08:30:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce81c00640 1 -- 172.21.15.46:0/1684743922 learned_addr learned my addr 172.21.15.46:0/1684743922 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce73e00640 1 --2- 172.21.15.46:0/1684743922 >> v2:172.21.15.73:3300/0 conn(0x7fce7c15d230 0x7fce7c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce81200640 1 --2- 172.21.15.46:0/1684743922 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce7c1593d0 0x7fce7c06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce81c00640 1 -- 172.21.15.46:0/1684743922 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce7c1593d0 msgr2=0x7fce7c06cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce81c00640 1 --2- 172.21.15.46:0/1684743922 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce7c1593d0 0x7fce7c06cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce81c00640 1 -- 172.21.15.46:0/1684743922 >> v2:172.21.15.73:3300/0 conn(0x7fce7c15d230 msgr2=0x7fce7c15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:06.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce81c00640 1 --2- 172.21.15.46:0/1684743922 >> v2:172.21.15.73:3300/0 conn(0x7fce7c15d230 0x7fce7c15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.837+0000 7fce81c00640 1 -- 172.21.15.46:0/1684743922 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c15e060 con 0x7fce7c15dbd0 2024-03-29T08:30:06.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.841+0000 7fce81c00640 1 --2- 172.21.15.46:0/1684743922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c15dbd0 0x7fce7c15bd80 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fce7c071520 tx=0x7fce7401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3ba1eb431b6fe2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:06.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.841+0000 7fce7be00640 1 -- 172.21.15.46:0/1684743922 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce740a5870 con 0x7fce7c15dbd0 2024-03-29T08:30:06.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.841+0000 7fce7be00640 1 -- 172.21.15.46:0/1684743922 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce740a59d0 con 0x7fce7c15dbd0 2024-03-29T08:30:06.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.841+0000 7fce7be00640 1 -- 172.21.15.46:0/1684743922 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce740a5d60 con 0x7fce7c15dbd0 2024-03-29T08:30:06.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.841+0000 7fce83c00640 1 -- 172.21.15.46:0/1684743922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c15dbd0 msgr2=0x7fce7c15bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:06.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.841+0000 7fce83c00640 1 --2- 172.21.15.46:0/1684743922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c15dbd0 0x7fce7c15bd80 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fce7c071520 tx=0x7fce7401c9f0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.841+0000 7fce83c00640 1 -- 172.21.15.46:0/1684743922 shutdown_connections 2024-03-29T08:30:06.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.841+0000 7fce83c00640 1 --2- 172.21.15.46:0/1684743922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c15dbd0 0x7fce7c15bd80 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.841+0000 7fce83c00640 1 --2- 172.21.15.46:0/1684743922 >> v2:172.21.15.73:3300/0 conn(0x7fce7c15d230 0x7fce7c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.841+0000 7fce83c00640 1 --2- 172.21.15.46:0/1684743922 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce7c1593d0 0x7fce7c06cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.841+0000 7fce83c00640 1 -- 172.21.15.46:0/1684743922 >> 172.21.15.46:0/1684743922 conn(0x7fce7c066110 msgr2=0x7fce7c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:06.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.841+0000 7fce83c00640 1 -- 172.21.15.46:0/1684743922 shutdown_connections 2024-03-29T08:30:06.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.841+0000 7fce83c00640 1 -- 172.21.15.46:0/1684743922 wait complete. 2024-03-29T08:30:06.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce83c00640 1 Processor -- start 2024-03-29T08:30:06.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce83c00640 1 -- start start 2024-03-29T08:30:06.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce83c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce7c1593d0 0x7fce7c1f4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce83c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce7c15d230 0x7fce7c1578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce83c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c15dbd0 0x7fce7c151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce83c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c1475f0 con 0x7fce7c15dbd0 2024-03-29T08:30:06.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce83c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c150960 con 0x7fce7c15d230 2024-03-29T08:30:06.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce83c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c14fde0 con 0x7fce7c1593d0 2024-03-29T08:30:06.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce81c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c15dbd0 0x7fce7c151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce81c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c15dbd0 0x7fce7c151f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48880/0 (socket says 172.21.15.46:48880) 2024-03-29T08:30:06.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce81c00640 1 -- 172.21.15.46:0/1572211353 learned_addr learned my addr 172.21.15.46:0/1572211353 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:06.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce81200640 1 --2- 172.21.15.46:0/1572211353 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce7c1593d0 0x7fce7c1f4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce73e00640 1 --2- 172.21.15.46:0/1572211353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce7c15d230 0x7fce7c1578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce81c00640 1 -- 172.21.15.46:0/1572211353 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce7c1593d0 msgr2=0x7fce7c1f4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:06.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce81c00640 1 --2- 172.21.15.46:0/1572211353 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce7c1593d0 0x7fce7c1f4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce81c00640 1 -- 172.21.15.46:0/1572211353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce7c15d230 msgr2=0x7fce7c1578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:06.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce81c00640 1 --2- 172.21.15.46:0/1572211353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce7c15d230 0x7fce7c1578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:06.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce81c00640 1 -- 172.21.15.46:0/1572211353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce74009b30 con 0x7fce7c15dbd0 2024-03-29T08:30:06.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce81c00640 1 --2- 172.21.15.46:0/1572211353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c15dbd0 0x7fce7c151f60 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fce740af040 tx=0x7fce74019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:06.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce79600640 1 -- 172.21.15.46:0/1572211353 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce740b8640 con 0x7fce7c15dbd0 2024-03-29T08:30:06.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce79600640 1 -- 172.21.15.46:0/1572211353 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce740b87a0 con 0x7fce7c15dbd0 2024-03-29T08:30:06.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce83c00640 1 -- 172.21.15.46:0/1572211353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c1500c0 con 0x7fce7c15dbd0 2024-03-29T08:30:06.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce79600640 1 -- 172.21.15.46:0/1572211353 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce740b8b30 con 0x7fce7c15dbd0 2024-03-29T08:30:06.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.845+0000 7fce83c00640 1 -- 172.21.15.46:0/1572211353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c150680 con 0x7fce7c15dbd0 2024-03-29T08:30:06.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.849+0000 7fce83c00640 1 -- 172.21.15.46:0/1572211353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7c147760 con 0x7fce7c15dbd0 2024-03-29T08:30:06.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.849+0000 7fce79600640 1 -- 172.21.15.46:0/1572211353 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fce740b8c90 con 0x7fce7c15dbd0 2024-03-29T08:30:06.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.849+0000 7fce79600640 1 --2- 172.21.15.46:0/1572211353 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce580666c0 0x7fce58068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:06.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.849+0000 7fce79600640 1 -- 172.21.15.46:0/1572211353 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fce7411ed50 con 0x7fce7c15dbd0 2024-03-29T08:30:06.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.849+0000 7fce81200640 1 --2- 172.21.15.46:0/1572211353 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce580666c0 0x7fce58068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:06.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.849+0000 7fce81200640 1 --2- 172.21.15.46:0/1572211353 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce580666c0 0x7fce58068b80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009a40 tx=0x7fce6c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:06.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:06.853+0000 7fce79600640 1 -- 172.21.15.46:0/1572211353 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fce740ec070 con 0x7fce7c15dbd0 2024-03-29T08:30:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:07.049+0000 7fce83c00640 1 -- 172.21.15.46:0/1572211353 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fce7c158c50 con 0x7fce7c15dbd0 2024-03-29T08:30:07.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:07.049+0000 7fce79600640 1 -- 172.21.15.46:0/1572211353 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v107) v1 ==== 53+0+73 (secure 0 0 0) 0x7fce740ef690 con 0x7fce7c15dbd0 2024-03-29T08:30:07.050 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-29T08:30:07.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:07.053+0000 7fce83c00640 1 -- 172.21.15.46:0/1572211353 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce580666c0 msgr2=0x7fce58068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:07.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:07.053+0000 7fce83c00640 1 --2- 172.21.15.46:0/1572211353 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce580666c0 0x7fce58068b80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009a40 tx=0x7fce6c009190 comp rx=0 tx=0).stop 2024-03-29T08:30:07.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:07.053+0000 7fce83c00640 1 -- 172.21.15.46:0/1572211353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c15dbd0 msgr2=0x7fce7c151f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:07.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:07.053+0000 7fce83c00640 1 --2- 172.21.15.46:0/1572211353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c15dbd0 0x7fce7c151f60 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fce740af040 tx=0x7fce74019790 comp rx=0 tx=0).stop 2024-03-29T08:30:07.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:07.053+0000 7fce83c00640 1 -- 172.21.15.46:0/1572211353 shutdown_connections 2024-03-29T08:30:07.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:07.053+0000 7fce83c00640 1 --2- 172.21.15.46:0/1572211353 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fce580666c0 0x7fce58068b80 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:07.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:07.053+0000 7fce83c00640 1 --2- 172.21.15.46:0/1572211353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce7c15d230 0x7fce7c1578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:07.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:07.053+0000 7fce83c00640 1 --2- 172.21.15.46:0/1572211353 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce7c15dbd0 0x7fce7c151f60 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:07.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:07.053+0000 7fce83c00640 1 --2- 172.21.15.46:0/1572211353 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce7c1593d0 0x7fce7c1f4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:07.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:07.053+0000 7fce83c00640 1 -- 172.21.15.46:0/1572211353 >> 172.21.15.46:0/1572211353 conn(0x7fce7c066110 msgr2=0x7fce7c0b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:07.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:07.053+0000 7fce83c00640 1 -- 172.21.15.46:0/1572211353 shutdown_connections 2024-03-29T08:30:07.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:07.053+0000 7fce83c00640 1 -- 172.21.15.46:0/1572211353 wait complete. 2024-03-29T08:30:07.070 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-29T08:30:07.071 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-29T08:30:07.071 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-03-29T08:30:07.071 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-29T08:30:07.071 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-29T08:30:07.071 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-29T08:30:07.071 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-29T08:30:07.071 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-29T08:30:07.071 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:30:07.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:07.071 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-03-29T08:30:07.081 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-29T08:30:07.081 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-29T08:30:07.081 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-03-29T08:30:07.081 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:30:07.081 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-03-29T08:30:07.081 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:30:07.082 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:30:07.082 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:30:07.082 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-03-29T08:30:07.082 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:30:07.082 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:30:07.082 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-03-29T08:30:07.082 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-29T08:30:07.082 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:30:07.082 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-03-29T08:30:07.082 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-29T08:30:07.082 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 41730sec preferred_lft 41730sec 2024-03-29T08:30:07.082 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2024-03-29T08:30:07.082 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:30:07.083 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-29T08:30:07.083 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-03-29T08:30:07.083 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-03-29T08:30:07.083 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-29T08:30:07.083 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:d9:58:09:4c brd ff:ff:ff:ff:ff:ff 2024-03-29T08:30:07.083 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-29T08:30:07.083 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:30:07.083 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-29T08:30:07.083 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-29T08:30:07.083 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-29T08:30:07.083 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:30:07.085 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-29T08:30:07.086 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:30:07.086 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:30:07.086 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:30:07.086 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-29T08:30:07.086 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:30:07.086 INFO:teuthology.orchestra.run.smithi046.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-29T08:30:07.086 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T08:30:07.086 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-03-29T08:30:07.086 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-03-29T08:30:07.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:07.087 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-29T08:30:07.141 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:30:07.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:07.143 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-03-29T08:30:07.196 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-03-29T08:30:07.197 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-29T08:30:07.197 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-29T08:30:07.197 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-29T08:30:07.197 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-29T08:30:07.197 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-29T08:30:07.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:07.197 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2024-03-29T08:30:07.197 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-03-29T08:30:07.197 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-29T08:30:07.197 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-29T08:30:07.198 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-29T08:30:07.198 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-03-29T08:30:07.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:07.290 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:30:07.301 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T08:30:07.301 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T08:30:07.301 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:30:07.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:07.302 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-03-29T08:30:07.363 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-29T08:30:07.363 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:30:07.363 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-29T08:30:07.415 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-29T08:30:07.416 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-29T08:30:07.416 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:30:07.471 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:30:07.536 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:30:07.536 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:30:07.597 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:30:07.599 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:30:07.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:07.599 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:30:07.651 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-29T08:30:07.651 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-29T08:30:07.698 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2024-03-29T08:30:07.719 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-03-29T08:30:07.719 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-03-29T08:30:07.778 INFO:teuthology.orchestra.run.smithi046.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-03-29T08:30:07.779 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-03-29T08:30:07.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:07.780 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2024-03-29T08:30:07.794 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2024-03-29T08:30:07.793+0000 7fb54f3d9580 -1 init, newargv = 0x55d1b7667d40 newargc=15 2024-03-29T08:30:07.794 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[52871]: starting ceph client 2024-03-29T08:30:07.801 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[52871]: starting fuse 2024-03-29T08:30:07.802 INFO:teuthology.orchestra.run.smithi046.stdout:43 2024-03-29T08:30:07.802 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-03-29T08:30:07.802 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:30:07.802 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2024-03-29T08:30:07.802 DEBUG:teuthology.orchestra.run.smithi046:> import re 2024-03-29T08:30:07.802 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:30:07.802 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2024-03-29T08:30:07.802 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:30:07.802 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2024-03-29T08:30:07.802 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-29T08:30:07.803 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2024-03-29T08:30:07.803 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-29T08:30:07.803 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:30:07.803 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2024-03-29T08:30:07.803 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2024-03-29T08:30:07.803 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2024-03-29T08:30:07.803 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2024-03-29T08:30:07.803 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:30:07.803 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2024-03-29T08:30:07.803 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-29T08:30:07.803 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-29T08:30:07.803 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-29T08:30:07.804 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2024-03-29T08:30:07.804 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2024-03-29T08:30:07.804 DEBUG:teuthology.orchestra.run.smithi046:> return f 2024-03-29T08:30:07.804 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-29T08:30:07.804 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:30:07.804 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2024-03-29T08:30:07.804 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:30:07.928 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.52871.asok 2024-03-29T08:30:08.111 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52871.asok 2024-03-29T08:30:08.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:08.112 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52871.asok status 2024-03-29T08:30:08.292 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:30:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2024-03-29T08:30:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "d0e392d9aee635f2b65768c460c7a151ead7a5e2", 2024-03-29T08:30:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 18.2.2-1015-gd0e392d9 (d0e392d9aee635f2b65768c460c7a151ead7a5e2) reef (stable)", 2024-03-29T08:30:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2024-03-29T08:30:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2024-03-29T08:30:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-29T08:30:08.292 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "52871", 2024-03-29T08:30:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2024-03-29T08:30:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:30:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2024-03-29T08:30:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2024-03-29T08:30:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 9759, 2024-03-29T08:30:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2024-03-29T08:30:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2024-03-29T08:30:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2024-03-29T08:30:08.293 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 9759 2024-03-29T08:30:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:30:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:30:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:30:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:30:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1261977773 2024-03-29T08:30:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-03-29T08:30:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:30:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2024-03-29T08:30:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-03-29T08:30:08.294 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2024-03-29T08:30:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1261977773 2024-03-29T08:30:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-03-29T08:30:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.9759 192.168.0.1:0/1261977773", 2024-03-29T08:30:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/1261977773", 2024-03-29T08:30:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2024-03-29T08:30:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 107, 2024-03-29T08:30:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 125, 2024-03-29T08:30:08.295 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2024-03-29T08:30:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2024-03-29T08:30:08.296 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2024-03-29T08:30:08.296 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:30:08.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:08.306 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2024-03-29T08:30:08.306 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-29T08:30:08.318 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2024-03-29T08:30:08.318 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-29T08:30:08.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:08.319 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-29T08:30:08.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:08.379 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-03-29T08:30:08.441 INFO:teuthology.orchestra.run.smithi046.stdout:./ 2024-03-29T08:30:08.441 INFO:teuthology.orchestra.run.smithi046.stdout:./parent 2024-03-29T08:30:08.441 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed 2024-03-29T08:30:08.441 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed/charlie 2024-03-29T08:30:08.441 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed/alpha 2024-03-29T08:30:08.441 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/flushed/bravo 2024-03-29T08:30:08.441 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/unflushed 2024-03-29T08:30:08.441 INFO:teuthology.orchestra.run.smithi046.stdout:./parent/unflushed/jfile 2024-03-29T08:30:08.442 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:30:08.442 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:30:08.442 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:30:08.443 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:30:08.443 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-03-29T08:30:08.443 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:30:08.547 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-03-29T08:30:08.737 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:30:08.737 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:30:08.737 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:30:08.737 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:30:08.737 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-03-29T08:30:08.737 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:30:08.844 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627776 2024-03-29T08:30:09.033 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:30:09.034 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:30:09.034 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:30:09.034 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:30:09.034 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-03-29T08:30:09.034 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:30:09.140 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627777 2024-03-29T08:30:09.330 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:30:09.330 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:30:09.330 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:30:09.330 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:30:09.330 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-03-29T08:30:09.330 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:30:09.436 INFO:teuthology.orchestra.run.smithi046.stdout:1099511628280 2024-03-29T08:30:09.625 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:30:09.625 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:30:09.625 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:30:09.625 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:30:09.625 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-03-29T08:30:09.625 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:30:09.736 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627778 2024-03-29T08:30:09.969 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:30:09.971 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:30:09.971 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:30:09.971 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:30:09.971 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:30:09.971 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-03-29T08:30:09.971 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:30:10.023 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:30:10.091 INFO:teuthology.orchestra.run.smithi046.stdout:1099511628279 2024-03-29T08:30:10.279 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:30:10.279 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:30:10.280 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:30:10.280 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:30:10.280 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-03-29T08:30:10.280 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:30:10.389 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627779 2024-03-29T08:30:10.578 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits daemon-helper kill python3 -c ' 2024-03-29T08:30:10.578 DEBUG:teuthology.orchestra.run.smithi046:> import os 2024-03-29T08:30:10.578 DEBUG:teuthology.orchestra.run.smithi046:> import stat 2024-03-29T08:30:10.578 DEBUG:teuthology.orchestra.run.smithi046:> 2024-03-29T08:30:10.578 DEBUG:teuthology.orchestra.run.smithi046:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-03-29T08:30:10.578 DEBUG:teuthology.orchestra.run.smithi046:> ' 2024-03-29T08:30:10.685 INFO:teuthology.orchestra.run.smithi046.stdout:1099511627780 2024-03-29T08:30:10.874 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-29T08:30:10.875 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:30:10.982 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-29T08:30:11.038 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-29T08:30:11.038 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:30:11.038 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:30:11.047 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:30:11.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc07cc00640 1 Processor -- start 2024-03-29T08:30:11.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc07cc00640 1 -- start start 2024-03-29T08:30:11.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc07cc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc07806b8b0 0x7fc0780cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc07cc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc07815dc00 0x7fc0780cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc07cc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc07815cfb0 con 0x7fc0780d0660 2024-03-29T08:30:11.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc07cc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07806c100 con 0x7fc07806b8b0 2024-03-29T08:30:11.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc07cc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07806be20 con 0x7fc07815dc00 2024-03-29T08:30:11.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc075e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc0780d0660 0x7fc0780d0a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41990/0 (socket says 172.21.15.46:41990) 2024-03-29T08:30:11.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc075e00640 1 -- 172.21.15.46:0/2663138813 learned_addr learned my addr 172.21.15.46:0/2663138813 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc06fe00640 1 --2- 172.21.15.46:0/2663138813 >> v2:172.21.15.46:3300/0 conn(0x7fc07806b8b0 0x7fc0780cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc075400640 1 -- 172.21.15.46:0/2663138813 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453541080 0 0) 0x7fc07815cfb0 con 0x7fc0780d0660 2024-03-29T08:30:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc075400640 1 -- 172.21.15.46:0/2663138813 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc05c003530 con 0x7fc0780d0660 2024-03-29T08:30:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc076800640 1 --2- 172.21.15.46:0/2663138813 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc07815dc00 0x7fc0780cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc06fe00640 1 -- 172.21.15.46:0/2663138813 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc07815dc00 msgr2=0x7fc0780cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc06fe00640 1 --2- 172.21.15.46:0/2663138813 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc07815dc00 0x7fc0780cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc06fe00640 1 -- 172.21.15.46:0/2663138813 >> v1:172.21.15.46:6789/0 conn(0x7fc0780d0660 legacy=0x7fc0780d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc06fe00640 1 -- 172.21.15.46:0/2663138813 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0780cecf0 con 0x7fc07806b8b0 2024-03-29T08:30:11.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.233+0000 7fc076800640 1 --2- 172.21.15.46:0/2663138813 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc07815dc00 0x7fc0780cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:11.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.237+0000 7fc06fe00640 1 --2- 172.21.15.46:0/2663138813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07806b8b0 0x7fc0780cad20 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fc0600097c0 tx=0x7fc06001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cefed0963a829a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.237+0000 7fc075400640 1 -- 172.21.15.46:0/2663138813 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0600ab640 con 0x7fc07806b8b0 2024-03-29T08:30:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.237+0000 7fc075400640 1 -- 172.21.15.46:0/2663138813 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0600a7070 con 0x7fc07806b8b0 2024-03-29T08:30:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.237+0000 7fc075400640 1 -- 172.21.15.46:0/2663138813 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0600ab9c0 con 0x7fc07806b8b0 2024-03-29T08:30:11.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.237+0000 7fc07cc00640 1 -- 172.21.15.46:0/2663138813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07806b8b0 msgr2=0x7fc0780cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.237+0000 7fc07cc00640 1 --2- 172.21.15.46:0/2663138813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07806b8b0 0x7fc0780cad20 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fc0600097c0 tx=0x7fc06001c470 comp rx=0 tx=0).stop 2024-03-29T08:30:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.237+0000 7fc07cc00640 1 -- 172.21.15.46:0/2663138813 shutdown_connections 2024-03-29T08:30:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.237+0000 7fc07cc00640 1 --2- 172.21.15.46:0/2663138813 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc07815dc00 0x7fc0780cf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.237+0000 7fc07cc00640 1 --2- 172.21.15.46:0/2663138813 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07806b8b0 0x7fc0780cad20 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.237+0000 7fc07cc00640 1 -- 172.21.15.46:0/2663138813 >> 172.21.15.46:0/2663138813 conn(0x7fc078089fd0 msgr2=0x7fc07808a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.237+0000 7fc07cc00640 1 -- 172.21.15.46:0/2663138813 shutdown_connections 2024-03-29T08:30:11.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.237+0000 7fc07cc00640 1 -- 172.21.15.46:0/2663138813 wait complete. 2024-03-29T08:30:11.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc07cc00640 1 Processor -- start 2024-03-29T08:30:11.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc07cc00640 1 -- start start 2024-03-29T08:30:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc07cc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07806b8b0 0x7fc0780c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc07cc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0780d0660 0x7fc078154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc07cc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc07815dc00 0x7fc078155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc07cc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc078156560 con 0x7fc07806b8b0 2024-03-29T08:30:11.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc07cc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0780c8060 con 0x7fc07815dc00 2024-03-29T08:30:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc07cc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0781555a0 con 0x7fc0780d0660 2024-03-29T08:30:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc075e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07806b8b0 0x7fc0780c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc075e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07806b8b0 0x7fc0780c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48898/0 (socket says 172.21.15.46:48898) 2024-03-29T08:30:11.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc075e00640 1 -- 172.21.15.46:0/3699808815 learned_addr learned my addr 172.21.15.46:0/3699808815 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc075e00640 1 -- 172.21.15.46:0/3699808815 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0780d0660 msgr2=0x7fc078154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc06fe00640 1 --2- 172.21.15.46:0/3699808815 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0780d0660 0x7fc078154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc076800640 1 --2- 172.21.15.46:0/3699808815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc07815dc00 0x7fc078155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc075e00640 1 --2- 172.21.15.46:0/3699808815 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0780d0660 0x7fc078154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc075e00640 1 -- 172.21.15.46:0/3699808815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc07815dc00 msgr2=0x7fc078155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc075e00640 1 --2- 172.21.15.46:0/3699808815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc07815dc00 0x7fc078155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc075e00640 1 -- 172.21.15.46:0/3699808815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc060009420 con 0x7fc07806b8b0 2024-03-29T08:30:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc06fe00640 1 --2- 172.21.15.46:0/3699808815 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0780d0660 0x7fc078154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:11.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc076800640 1 --2- 172.21.15.46:0/3699808815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc07815dc00 0x7fc078155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc075e00640 1 --2- 172.21.15.46:0/3699808815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07806b8b0 0x7fc0780c6b00 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fc06800c4d0 tx=0x7fc06800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc06e000640 1 -- 172.21.15.46:0/3699808815 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068005cf0 con 0x7fc07806b8b0 2024-03-29T08:30:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc06e000640 1 -- 172.21.15.46:0/3699808815 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc068005e50 con 0x7fc07806b8b0 2024-03-29T08:30:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc06e000640 1 -- 172.21.15.46:0/3699808815 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068007270 con 0x7fc07806b8b0 2024-03-29T08:30:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc07cc00640 1 -- 172.21.15.46:0/3699808815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0780c4200 con 0x7fc07806b8b0 2024-03-29T08:30:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc07cc00640 1 -- 172.21.15.46:0/3699808815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0780c7aa0 con 0x7fc07806b8b0 2024-03-29T08:30:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.241+0000 7fc07cc00640 1 -- 172.21.15.46:0/3699808815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc078066c90 con 0x7fc07806b8b0 2024-03-29T08:30:11.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.245+0000 7fc06e000640 1 -- 172.21.15.46:0/3699808815 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc0680073d0 con 0x7fc07806b8b0 2024-03-29T08:30:11.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.245+0000 7fc06e000640 1 --2- 172.21.15.46:0/3699808815 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc054066690 0x7fc054068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.245+0000 7fc06e000640 1 -- 172.21.15.46:0/3699808815 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fc068083790 con 0x7fc07806b8b0 2024-03-29T08:30:11.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.245+0000 7fc06fe00640 1 --2- 172.21.15.46:0/3699808815 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc054066690 0x7fc054068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.245+0000 7fc06fe00640 1 --2- 172.21.15.46:0/3699808815 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc054066690 0x7fc054068b50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fc0600af000 tx=0x7fc060008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:11.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.249+0000 7fc06e000640 1 -- 172.21.15.46:0/3699808815 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc068050ab0 con 0x7fc07806b8b0 2024-03-29T08:30:11.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.413+0000 7fc07cc00640 1 -- 172.21.15.46:0/3699808815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0780b1820 con 0x7fc07806b8b0 2024-03-29T08:30:11.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.413+0000 7fc06e000640 1 -- 172.21.15.46:0/3699808815 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16406 (secure 0 0 0) 0x7fc0680540d0 con 0x7fc07806b8b0 2024-03-29T08:30:11.414 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:11.414 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":125,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:30:00.063547+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:29:12.329121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-29T08:29:13.465939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.7","mappings":[{"from":4,"to":3}]},{"pgid":"17.4","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6834/276185207":"2024-03-30T08:30:00.063526+0000","172.21.15.46:6835/929698978":"2024-03-30T08:29:45.374806+0000","172.21.15.46:6835/276185207":"2024-03-30T08:30:00.063526+0000","172.21.15.46:6834/929698978":"2024-03-30T08:29:45.374806+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:30:11.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.413+0000 7fc07cc00640 1 -- 172.21.15.46:0/3699808815 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc054066690 msgr2=0x7fc054068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.413+0000 7fc07cc00640 1 --2- 172.21.15.46:0/3699808815 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc054066690 0x7fc054068b50 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fc0600af000 tx=0x7fc060008040 comp rx=0 tx=0).stop 2024-03-29T08:30:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.413+0000 7fc07cc00640 1 -- 172.21.15.46:0/3699808815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07806b8b0 msgr2=0x7fc0780c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.413+0000 7fc07cc00640 1 --2- 172.21.15.46:0/3699808815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07806b8b0 0x7fc0780c6b00 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fc06800c4d0 tx=0x7fc06800c990 comp rx=0 tx=0).stop 2024-03-29T08:30:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.413+0000 7fc07cc00640 1 -- 172.21.15.46:0/3699808815 shutdown_connections 2024-03-29T08:30:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.413+0000 7fc07cc00640 1 --2- 172.21.15.46:0/3699808815 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc054066690 0x7fc054068b50 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.413+0000 7fc07cc00640 1 --2- 172.21.15.46:0/3699808815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc07815dc00 0x7fc078155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.413+0000 7fc07cc00640 1 --2- 172.21.15.46:0/3699808815 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc0780d0660 0x7fc078154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.413+0000 7fc07cc00640 1 --2- 172.21.15.46:0/3699808815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc07806b8b0 0x7fc0780c6b00 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.413+0000 7fc07cc00640 1 -- 172.21.15.46:0/3699808815 >> 172.21.15.46:0/3699808815 conn(0x7fc078089fd0 msgr2=0x7fc0780d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.413+0000 7fc07cc00640 1 -- 172.21.15.46:0/3699808815 shutdown_connections 2024-03-29T08:30:11.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.413+0000 7fc07cc00640 1 -- 172.21.15.46:0/3699808815 wait complete. 2024-03-29T08:30:11.436 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:29:12.329121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:29:13.465939+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:30:11.436 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:30:11.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f4220600640 1 Processor -- start 2024-03-29T08:30:11.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f4220600640 1 -- start start 2024-03-29T08:30:11.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f4220600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f421815cad0 0x7f421806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f4220600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f421806eff0 0x7f4218069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f4220600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421806f450 con 0x7f421806a2d0 2024-03-29T08:30:11.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f4220600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421815dd00 con 0x7f421806eff0 2024-03-29T08:30:11.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f4220600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421806a740 con 0x7f421815cad0 2024-03-29T08:30:11.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f421e600640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f421806a2d0 0x7f42181604d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41994/0 (socket says 172.21.15.46:41994) 2024-03-29T08:30:11.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f421e600640 1 -- 172.21.15.46:0/4274261070 learned_addr learned my addr 172.21.15.46:0/4274261070 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f421d200640 1 --2- 172.21.15.46:0/4274261070 >> v2:172.21.15.73:3300/0 conn(0x7f421806eff0 0x7f4218069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f421dc00640 1 --2- 172.21.15.46:0/4274261070 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f421815cad0 0x7f421806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f4217e00640 1 -- 172.21.15.46:0/4274261070 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967085167 0 0) 0x7f42100014a0 con 0x7f421806a2d0 2024-03-29T08:30:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f4217e00640 1 -- 172.21.15.46:0/4274261070 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4200003530 con 0x7f421806a2d0 2024-03-29T08:30:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f421d200640 1 -- 172.21.15.46:0/4274261070 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f421815cad0 msgr2=0x7f421806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f421d200640 1 --2- 172.21.15.46:0/4274261070 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f421815cad0 0x7f421806eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f4217e00640 1 -- 172.21.15.46:0/4274261070 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1207212844 0 0) 0x7f4200003530 con 0x7f421806a2d0 2024-03-29T08:30:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f421d200640 1 -- 172.21.15.46:0/4274261070 >> v1:172.21.15.46:6789/0 conn(0x7f421806a2d0 legacy=0x7f42181604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:11.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.581+0000 7f421d200640 1 -- 172.21.15.46:0/4274261070 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421815d080 con 0x7f421806eff0 2024-03-29T08:30:11.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.585+0000 7f421d200640 1 --2- 172.21.15.46:0/4274261070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f421806eff0 0x7f4218069d90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f42040097c0 tx=0x7f420401c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90d7558c8f08003a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:11.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.585+0000 7f4217e00640 1 -- 172.21.15.46:0/4274261070 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4210001b90 con 0x7f421806eff0 2024-03-29T08:30:11.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.585+0000 7f4217e00640 1 -- 172.21.15.46:0/4274261070 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42040a7050 con 0x7f421806eff0 2024-03-29T08:30:11.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.585+0000 7f4217e00640 1 -- 172.21.15.46:0/4274261070 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42040ab620 con 0x7f421806eff0 2024-03-29T08:30:11.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.585+0000 7f4220600640 1 -- 172.21.15.46:0/4274261070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f421806eff0 msgr2=0x7f4218069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:11.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.585+0000 7f4220600640 1 --2- 172.21.15.46:0/4274261070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f421806eff0 0x7f4218069d90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f42040097c0 tx=0x7f420401c330 comp rx=0 tx=0).stop 2024-03-29T08:30:11.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.585+0000 7f4220600640 1 -- 172.21.15.46:0/4274261070 shutdown_connections 2024-03-29T08:30:11.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.585+0000 7f4220600640 1 --2- 172.21.15.46:0/4274261070 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f421806eff0 0x7f4218069d90 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.585+0000 7f4220600640 1 --2- 172.21.15.46:0/4274261070 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f421815cad0 0x7f421806eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.585+0000 7f4220600640 1 -- 172.21.15.46:0/4274261070 >> 172.21.15.46:0/4274261070 conn(0x7f4218098a40 msgr2=0x7f42180b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:11.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.585+0000 7f4220600640 1 -- 172.21.15.46:0/4274261070 shutdown_connections 2024-03-29T08:30:11.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.585+0000 7f4220600640 1 -- 172.21.15.46:0/4274261070 wait complete. 2024-03-29T08:30:11.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f4220600640 1 Processor -- start 2024-03-29T08:30:11.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f4220600640 1 -- start start 2024-03-29T08:30:11.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f4220600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f421806a2d0 0x7f42180b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f4220600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f421806eff0 0x7f42180c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f4220600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f421815cad0 0x7f42180b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f4220600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42180b3aa0 con 0x7f421806eff0 2024-03-29T08:30:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f4220600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42180c4030 con 0x7f421815cad0 2024-03-29T08:30:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f4220600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42180c4310 con 0x7f421806a2d0 2024-03-29T08:30:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f421d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f421806eff0 0x7f42180c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f421d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f421806eff0 0x7f42180c5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48908/0 (socket says 172.21.15.46:48908) 2024-03-29T08:30:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f421d200640 1 -- 172.21.15.46:0/3980043792 learned_addr learned my addr 172.21.15.46:0/3980043792 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f421dc00640 1 --2- 172.21.15.46:0/3980043792 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f421806a2d0 0x7f42180b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f421e600640 1 --2- 172.21.15.46:0/3980043792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f421815cad0 0x7f42180b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f421d200640 1 -- 172.21.15.46:0/3980043792 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f421806a2d0 msgr2=0x7f42180b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:11.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f421d200640 1 --2- 172.21.15.46:0/3980043792 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f421806a2d0 0x7f42180b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f421d200640 1 -- 172.21.15.46:0/3980043792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f421815cad0 msgr2=0x7f42180b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:11.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f421d200640 1 --2- 172.21.15.46:0/3980043792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f421815cad0 0x7f42180b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f421d200640 1 -- 172.21.15.46:0/3980043792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4204009420 con 0x7f421806eff0 2024-03-29T08:30:11.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f421d200640 1 --2- 172.21.15.46:0/3980043792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f421806eff0 0x7f42180c5650 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f420401a9e0 tx=0x7f42040190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:11.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f4215600640 1 -- 172.21.15.46:0/3980043792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42040b4410 con 0x7f421806eff0 2024-03-29T08:30:11.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f4215600640 1 -- 172.21.15.46:0/3980043792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42040b4590 con 0x7f421806eff0 2024-03-29T08:30:11.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f4215600640 1 -- 172.21.15.46:0/3980043792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42040b4940 con 0x7f421806eff0 2024-03-29T08:30:11.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f4220600640 1 -- 172.21.15.46:0/3980043792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42180c45f0 con 0x7f421806eff0 2024-03-29T08:30:11.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.589+0000 7f4220600640 1 -- 172.21.15.46:0/3980043792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4218156540 con 0x7f421806eff0 2024-03-29T08:30:11.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.593+0000 7f4220600640 1 -- 172.21.15.46:0/3980043792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42180cd8f0 con 0x7f421806eff0 2024-03-29T08:30:11.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.593+0000 7f4215600640 1 -- 172.21.15.46:0/3980043792 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f42040b2040 con 0x7f421806eff0 2024-03-29T08:30:11.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.593+0000 7f4215600640 1 --2- 172.21.15.46:0/3980043792 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f41f8066690 0x7f41f8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.593+0000 7f4215600640 1 -- 172.21.15.46:0/3980043792 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f420411e1e0 con 0x7f421806eff0 2024-03-29T08:30:11.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.593+0000 7f421dc00640 1 --2- 172.21.15.46:0/3980043792 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f41f8066690 0x7f41f8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.593+0000 7f421dc00640 1 --2- 172.21.15.46:0/3980043792 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f41f8066690 0x7f41f8068b50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4218156d30 tx=0x7f420c009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:11.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.597+0000 7f4215600640 1 -- 172.21.15.46:0/3980043792 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f42040eb470 con 0x7f421806eff0 2024-03-29T08:30:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.757+0000 7f4220600640 1 -- 172.21.15.46:0/3980043792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f421809d700 con 0x7f421806eff0 2024-03-29T08:30:11.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.761+0000 7f4215600640 1 -- 172.21.15.46:0/3980043792 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f42040eea90 con 0x7f421806eff0 2024-03-29T08:30:11.760 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:30:11.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.761+0000 7f4220600640 1 -- 172.21.15.46:0/3980043792 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f41f8066690 msgr2=0x7f41f8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.761+0000 7f4220600640 1 --2- 172.21.15.46:0/3980043792 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f41f8066690 0x7f41f8068b50 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4218156d30 tx=0x7f420c009190 comp rx=0 tx=0).stop 2024-03-29T08:30:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.761+0000 7f4220600640 1 -- 172.21.15.46:0/3980043792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f421806eff0 msgr2=0x7f42180c5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.761+0000 7f4220600640 1 --2- 172.21.15.46:0/3980043792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f421806eff0 0x7f42180c5650 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f420401a9e0 tx=0x7f42040190d0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.761+0000 7f4220600640 1 -- 172.21.15.46:0/3980043792 shutdown_connections 2024-03-29T08:30:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.761+0000 7f4220600640 1 --2- 172.21.15.46:0/3980043792 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f41f8066690 0x7f41f8068b50 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.761+0000 7f4220600640 1 --2- 172.21.15.46:0/3980043792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f421815cad0 0x7f42180b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.761+0000 7f4220600640 1 --2- 172.21.15.46:0/3980043792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f421806eff0 0x7f42180c5650 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.761+0000 7f4220600640 1 --2- 172.21.15.46:0/3980043792 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f421806a2d0 0x7f42180b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.761+0000 7f4220600640 1 -- 172.21.15.46:0/3980043792 >> 172.21.15.46:0/3980043792 conn(0x7f4218098a40 msgr2=0x7f421815dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.761+0000 7f4220600640 1 -- 172.21.15.46:0/3980043792 shutdown_connections 2024-03-29T08:30:11.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.761+0000 7f4220600640 1 -- 172.21.15.46:0/3980043792 wait complete. 2024-03-29T08:30:11.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:30:11.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.921+0000 7f760fa00640 1 Processor -- start 2024-03-29T08:30:11.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.921+0000 7f760fa00640 1 -- start start 2024-03-29T08:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f760fa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7608070650 0x7f760806c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f760fa00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76080b2450 0x7f7608070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f760fa00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f760806c910 con 0x7f760806dbe0 2024-03-29T08:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f760fa00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76080b15b0 con 0x7f76080b2450 2024-03-29T08:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f760fa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7608070f70 con 0x7f7608070650 2024-03-29T08:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f760da00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f760806dbe0 0x7f76080abcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42006/0 (socket says 172.21.15.46:42006) 2024-03-29T08:30:11.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f760da00640 1 -- 172.21.15.46:0/4168046114 learned_addr learned my addr 172.21.15.46:0/4168046114 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f760d000640 1 --2- 172.21.15.46:0/4168046114 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7608070650 0x7f760806c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f75ffe00640 1 --2- 172.21.15.46:0/4168046114 >> v2:172.21.15.73:3300/0 conn(0x7f76080b2450 0x7f7608070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f7607e00640 1 -- 172.21.15.46:0/4168046114 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439304678 0 0) 0x7f760806c910 con 0x7f760806dbe0 2024-03-29T08:30:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f7607e00640 1 -- 172.21.15.46:0/4168046114 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75ec003530 con 0x7f760806dbe0 2024-03-29T08:30:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f7607e00640 1 -- 172.21.15.46:0/4168046114 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1878852721 0 0) 0x7f75ec003530 con 0x7f760806dbe0 2024-03-29T08:30:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f7607e00640 1 -- 172.21.15.46:0/4168046114 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7608070650 msgr2=0x7f760806c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f7607e00640 1 --2- 172.21.15.46:0/4168046114 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7608070650 0x7f760806c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f7607e00640 1 -- 172.21.15.46:0/4168046114 >> v2:172.21.15.73:3300/0 conn(0x7f76080b2450 msgr2=0x7f7608070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f7607e00640 1 --2- 172.21.15.46:0/4168046114 >> v2:172.21.15.73:3300/0 conn(0x7f76080b2450 0x7f7608070a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f7607e00640 1 -- 172.21.15.46:0/4168046114 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760806cb40 con 0x7f760806dbe0 2024-03-29T08:30:11.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f7607e00640 1 -- 172.21.15.46:0/4168046114 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f7600001a00 con 0x7f760806dbe0 2024-03-29T08:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f7607e00640 1 -- 172.21.15.46:0/4168046114 >> v1:172.21.15.46:6789/0 conn(0x7f760806dbe0 legacy=0x7f76080abcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f7607e00640 1 --2- 172.21.15.46:0/4168046114 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7608070650 0x7f75ec003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f7607e00640 1 --2- 172.21.15.46:0/4168046114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75ec003f50 0x7f75ec0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f7607e00640 1 --2- 172.21.15.46:0/4168046114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ec004d10 0x7f75ec005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f7607e00640 1 -- 172.21.15.46:0/4168046114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7608070f70 con 0x7f75ec004d10 2024-03-29T08:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f7607e00640 1 -- 172.21.15.46:0/4168046114 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec00b990 con 0x7f75ec003f50 2024-03-29T08:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f7607e00640 1 -- 172.21.15.46:0/4168046114 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec00bad0 con 0x7f7608070650 2024-03-29T08:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f75ffe00640 1 --2- 172.21.15.46:0/4168046114 >> v2:172.21.15.73:3300/0 conn(0x7f76080b2450 0x7f7608070a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f760da00640 1 --2- 172.21.15.46:0/4168046114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ec004d10 0x7f75ec005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f760d000640 1 --2- 172.21.15.46:0/4168046114 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7608070650 0x7f75ec003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f75ffe00640 1 --2- 172.21.15.46:0/4168046114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75ec003f50 0x7f75ec0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f760da00640 1 -- 172.21.15.46:0/4168046114 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7608070650 msgr2=0x7f75ec003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:11.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f760da00640 1 --2- 172.21.15.46:0/4168046114 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7608070650 0x7f75ec003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f760da00640 1 -- 172.21.15.46:0/4168046114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75ec003f50 msgr2=0x7f75ec0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f760da00640 1 --2- 172.21.15.46:0/4168046114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75ec003f50 0x7f75ec0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.925+0000 7f760da00640 1 -- 172.21.15.46:0/4168046114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec003530 con 0x7f75ec004d10 2024-03-29T08:30:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.929+0000 7f760da00640 1 --2- 172.21.15.46:0/4168046114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ec004d10 0x7f75ec005190 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f760000a7c0 tx=0x7f760001d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955037df35c9e9cf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.929+0000 7f7607e00640 1 -- 172.21.15.46:0/4168046114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76000ad630 con 0x7f75ec004d10 2024-03-29T08:30:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.929+0000 7f7607e00640 1 -- 172.21.15.46:0/4168046114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76000a8070 con 0x7f75ec004d10 2024-03-29T08:30:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.929+0000 7f7607e00640 1 -- 172.21.15.46:0/4168046114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76000ada30 con 0x7f75ec004d10 2024-03-29T08:30:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.929+0000 7f760fa00640 1 -- 172.21.15.46:0/4168046114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ec004d10 msgr2=0x7f75ec005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.929+0000 7f760fa00640 1 --2- 172.21.15.46:0/4168046114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75ec004d10 0x7f75ec005190 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f760000a7c0 tx=0x7f760001d2a0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.929+0000 7f760da00640 1 -- 172.21.15.46:0/4168046114 reap_dead start 2024-03-29T08:30:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.929+0000 7f760fa00640 1 -- 172.21.15.46:0/4168046114 shutdown_connections 2024-03-29T08:30:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.929+0000 7f760fa00640 1 -- 172.21.15.46:0/4168046114 >> 172.21.15.46:0/4168046114 conn(0x7f76080b0640 msgr2=0x7f76080b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:11.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.929+0000 7f760fa00640 1 -- 172.21.15.46:0/4168046114 shutdown_connections 2024-03-29T08:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.929+0000 7f760fa00640 1 -- 172.21.15.46:0/4168046114 wait complete. 2024-03-29T08:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760fa00640 1 Processor -- start 2024-03-29T08:30:11.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760fa00640 1 -- start start 2024-03-29T08:30:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760fa00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75ec004d10 0x7f76080ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760fa00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f76080ce0d0 0x7f76080c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760fa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76080c5f30 0x7f76081f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760fa00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76080c4a40 con 0x7f76080c5f30 2024-03-29T08:30:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760fa00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76080c63b0 con 0x7f75ec004d10 2024-03-29T08:30:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760fa00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76080c4e60 con 0x7f76080ce0d0 2024-03-29T08:30:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76080c5f30 0x7f76081f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76080c5f30 0x7f76081f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48926/0 (socket says 172.21.15.46:48926) 2024-03-29T08:30:11.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760da00640 1 -- 172.21.15.46:0/3051688863 learned_addr learned my addr 172.21.15.46:0/3051688863 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:11.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760d000640 1 --2- 172.21.15.46:0/3051688863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75ec004d10 0x7f76080ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f75ffe00640 1 --2- 172.21.15.46:0/3051688863 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f76080ce0d0 0x7f76080c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760da00640 1 -- 172.21.15.46:0/3051688863 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f76080ce0d0 msgr2=0x7f76080c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:11.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760da00640 1 --2- 172.21.15.46:0/3051688863 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f76080ce0d0 0x7f76080c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760da00640 1 -- 172.21.15.46:0/3051688863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75ec004d10 msgr2=0x7f76080ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:11.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760da00640 1 --2- 172.21.15.46:0/3051688863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75ec004d10 0x7f76080ca3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:11.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760da00640 1 -- 172.21.15.46:0/3051688863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76080c3b70 con 0x7f76080c5f30 2024-03-29T08:30:11.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760da00640 1 --2- 172.21.15.46:0/3051688863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76080c5f30 0x7f76081f5a50 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f760000a760 tx=0x7f75ec005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f7605600640 1 -- 172.21.15.46:0/3051688863 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7600001f10 con 0x7f76080c5f30 2024-03-29T08:30:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f7605600640 1 -- 172.21.15.46:0/3051688863 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76000017c0 con 0x7f76080c5f30 2024-03-29T08:30:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760fa00640 1 -- 172.21.15.46:0/3051688863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76080ca910 con 0x7f76080c5f30 2024-03-29T08:30:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f7605600640 1 -- 172.21.15.46:0/3051688863 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7600004b70 con 0x7f76080c5f30 2024-03-29T08:30:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760fa00640 1 -- 172.21.15.46:0/3051688863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76080c40a0 con 0x7f76080c5f30 2024-03-29T08:30:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.933+0000 7f760fa00640 1 -- 172.21.15.46:0/3051688863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76080c4b80 con 0x7f76080c5f30 2024-03-29T08:30:11.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.937+0000 7f7605600640 1 -- 172.21.15.46:0/3051688863 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f76000aa030 con 0x7f76080c5f30 2024-03-29T08:30:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.937+0000 7f7605600640 1 --2- 172.21.15.46:0/3051688863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f75e40666c0 0x7f75e4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.937+0000 7f7605600640 1 -- 172.21.15.46:0/3051688863 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f760011e280 con 0x7f76080c5f30 2024-03-29T08:30:11.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.937+0000 7f760d000640 1 --2- 172.21.15.46:0/3051688863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f75e40666c0 0x7f75e4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:11.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.937+0000 7f760d000640 1 --2- 172.21.15.46:0/3051688863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f75e40666c0 0x7f75e4068b80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f75f8009be0 tx=0x7f75f8009190 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:11.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:11.941+0000 7f7605600640 1 -- 172.21.15.46:0/3051688863 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f76080c4b80 con 0x7f76080c5f30 2024-03-29T08:30:12.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.101+0000 7f760fa00640 1 -- 172.21.15.46:0/3051688863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f76080c4b80 con 0x7f76080c5f30 2024-03-29T08:30:12.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.105+0000 7f7605600640 1 -- 172.21.15.46:0/3051688863 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7f76080c4b80 con 0x7f76080c5f30 2024-03-29T08:30:12.104 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:30:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.105+0000 7f760fa00640 1 -- 172.21.15.46:0/3051688863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f75e40666c0 msgr2=0x7f75e4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:12.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.105+0000 7f760fa00640 1 --2- 172.21.15.46:0/3051688863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f75e40666c0 0x7f75e4068b80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f75f8009be0 tx=0x7f75f8009190 comp rx=0 tx=0).stop 2024-03-29T08:30:12.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.105+0000 7f760fa00640 1 -- 172.21.15.46:0/3051688863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76080c5f30 msgr2=0x7f76081f5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:12.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.105+0000 7f760fa00640 1 --2- 172.21.15.46:0/3051688863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76080c5f30 0x7f76081f5a50 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f760000a760 tx=0x7f75ec005800 comp rx=0 tx=0).stop 2024-03-29T08:30:12.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.105+0000 7f760fa00640 1 -- 172.21.15.46:0/3051688863 shutdown_connections 2024-03-29T08:30:12.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.105+0000 7f760fa00640 1 --2- 172.21.15.46:0/3051688863 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f75e40666c0 0x7f75e4068b80 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.105+0000 7f760fa00640 1 --2- 172.21.15.46:0/3051688863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f76080c5f30 0x7f76081f5a50 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.105+0000 7f760fa00640 1 --2- 172.21.15.46:0/3051688863 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f76080ce0d0 0x7f76080c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.105+0000 7f760fa00640 1 --2- 172.21.15.46:0/3051688863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75ec004d10 0x7f76080ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.105+0000 7f760fa00640 1 -- 172.21.15.46:0/3051688863 >> 172.21.15.46:0/3051688863 conn(0x7f76080b0640 msgr2=0x7f76080ae5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:12.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.105+0000 7f760fa00640 1 -- 172.21.15.46:0/3051688863 shutdown_connections 2024-03-29T08:30:12.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.105+0000 7f760fa00640 1 -- 172.21.15.46:0/3051688863 wait complete. 2024-03-29T08:30:12.125 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:30:12.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.265+0000 7f9fb8c00640 1 Processor -- start 2024-03-29T08:30:12.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.265+0000 7f9fb8c00640 1 -- start start 2024-03-29T08:30:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.269+0000 7f9fb8c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9fb4143eb0 0x7f9fb40cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:12.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.269+0000 7f9fb8c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9fb415efe0 0x7f9fb40cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:12.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.269+0000 7f9fb8c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb40b2360 con 0x7f9fb40b1630 2024-03-29T08:30:12.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.269+0000 7f9fb8c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb415e500 con 0x7f9fb4143eb0 2024-03-29T08:30:12.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.269+0000 7f9fb8c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb415e7e0 con 0x7f9fb415efe0 2024-03-29T08:30:12.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.269+0000 7f9fb1e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9fb4143eb0 0x7f9fb40cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:12.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.269+0000 7f9fb2800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9fb40b1630 0x7f9fb40b1a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42016/0 (socket says 172.21.15.46:42016) 2024-03-29T08:30:12.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.269+0000 7f9fb2800640 1 -- 172.21.15.46:0/808747687 learned_addr learned my addr 172.21.15.46:0/808747687 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:12.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.269+0000 7f9fa9400640 1 --2- 172.21.15.46:0/808747687 >> v2:172.21.15.73:3300/0 conn(0x7f9fb415efe0 0x7f9fb40cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:12.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.269+0000 7f9fb1400640 1 -- 172.21.15.46:0/808747687 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316477546 0 0) 0x7f9fb40b2360 con 0x7f9fb40b1630 2024-03-29T08:30:12.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.269+0000 7f9fb1400640 1 -- 172.21.15.46:0/808747687 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f98003530 con 0x7f9fb40b1630 2024-03-29T08:30:12.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.269+0000 7f9fb1e00640 1 -- 172.21.15.46:0/808747687 >> v2:172.21.15.73:3300/0 conn(0x7f9fb415efe0 msgr2=0x7f9fb40cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:12.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.269+0000 7f9fb1e00640 1 --2- 172.21.15.46:0/808747687 >> v2:172.21.15.73:3300/0 conn(0x7f9fb415efe0 0x7f9fb40cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.269+0000 7f9fb1e00640 1 -- 172.21.15.46:0/808747687 >> v1:172.21.15.46:6789/0 conn(0x7f9fb40b1630 legacy=0x7f9fb40b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:12.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.269+0000 7f9fb1e00640 1 -- 172.21.15.46:0/808747687 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb415eac0 con 0x7f9fb4143eb0 2024-03-29T08:30:12.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.273+0000 7f9fb1e00640 1 --2- 172.21.15.46:0/808747687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb4143eb0 0x7f9fb40cb790 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4158930 tx=0x7f9fa401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190c6d081962f7a8 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.273+0000 7f9fb1400640 1 -- 172.21.15.46:0/808747687 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa40a8050 con 0x7f9fb4143eb0 2024-03-29T08:30:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.273+0000 7f9fb1400640 1 -- 172.21.15.46:0/808747687 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa40ac5f0 con 0x7f9fb4143eb0 2024-03-29T08:30:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.273+0000 7f9fb1400640 1 -- 172.21.15.46:0/808747687 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa40ac960 con 0x7f9fb4143eb0 2024-03-29T08:30:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.273+0000 7f9fb8c00640 1 -- 172.21.15.46:0/808747687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb4143eb0 msgr2=0x7f9fb40cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.273+0000 7f9fb8c00640 1 --2- 172.21.15.46:0/808747687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb4143eb0 0x7f9fb40cb790 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f9fb4158930 tx=0x7f9fa401c480 comp rx=0 tx=0).stop 2024-03-29T08:30:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.273+0000 7f9fb8c00640 1 -- 172.21.15.46:0/808747687 shutdown_connections 2024-03-29T08:30:12.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.273+0000 7f9fb8c00640 1 --2- 172.21.15.46:0/808747687 >> v2:172.21.15.73:3300/0 conn(0x7f9fb415efe0 0x7f9fb40cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.273+0000 7f9fb8c00640 1 --2- 172.21.15.46:0/808747687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb4143eb0 0x7f9fb40cb790 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.273+0000 7f9fb8c00640 1 -- 172.21.15.46:0/808747687 >> 172.21.15.46:0/808747687 conn(0x7f9fb406a390 msgr2=0x7f9fb40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:12.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.273+0000 7f9fb8c00640 1 -- 172.21.15.46:0/808747687 shutdown_connections 2024-03-29T08:30:12.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.273+0000 7f9fb8c00640 1 -- 172.21.15.46:0/808747687 wait complete. 2024-03-29T08:30:12.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.273+0000 7f9fb8c00640 1 Processor -- start 2024-03-29T08:30:12.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.273+0000 7f9fb8c00640 1 -- start start 2024-03-29T08:30:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb40b1630 0x7f9fb4159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb8c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9fb4143eb0 0x7f9fb415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fb415efe0 0x7f9fb413cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb8c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb415a070 con 0x7f9fb40b1630 2024-03-29T08:30:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb8c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb413ea70 con 0x7f9fb415efe0 2024-03-29T08:30:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb8c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb413ed50 con 0x7f9fb4143eb0 2024-03-29T08:30:12.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb40b1630 0x7f9fb4159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb40b1630 0x7f9fb4159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48942/0 (socket says 172.21.15.46:48942) 2024-03-29T08:30:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb1e00640 1 -- 172.21.15.46:0/3838098095 learned_addr learned my addr 172.21.15.46:0/3838098095 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb2800640 1 --2- 172.21.15.46:0/3838098095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fb415efe0 0x7f9fb413cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fa9400640 1 --2- 172.21.15.46:0/3838098095 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9fb4143eb0 0x7f9fb415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb1e00640 1 -- 172.21.15.46:0/3838098095 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9fb4143eb0 msgr2=0x7f9fb415fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb1e00640 1 --2- 172.21.15.46:0/3838098095 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9fb4143eb0 0x7f9fb415fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb1e00640 1 -- 172.21.15.46:0/3838098095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fb415efe0 msgr2=0x7f9fb413cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb1e00640 1 --2- 172.21.15.46:0/3838098095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fb415efe0 0x7f9fb413cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb1e00640 1 -- 172.21.15.46:0/3838098095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa4009420 con 0x7f9fb40b1630 2024-03-29T08:30:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fa9400640 1 --2- 172.21.15.46:0/3838098095 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9fb4143eb0 0x7f9fb415fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb1e00640 1 --2- 172.21.15.46:0/3838098095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb40b1630 0x7f9fb4159b30 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4000c10 tx=0x7f9fa4019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9faaa00640 1 -- 172.21.15.46:0/3838098095 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa40a8050 con 0x7f9fb40b1630 2024-03-29T08:30:12.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9faaa00640 1 -- 172.21.15.46:0/3838098095 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa40b5640 con 0x7f9fb40b1630 2024-03-29T08:30:12.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9faaa00640 1 -- 172.21.15.46:0/3838098095 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa40b59d0 con 0x7f9fb40b1630 2024-03-29T08:30:12.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb8c00640 1 -- 172.21.15.46:0/3838098095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb413f030 con 0x7f9fb40b1630 2024-03-29T08:30:12.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb8c00640 1 -- 172.21.15.46:0/3838098095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb413f5f0 con 0x7f9fb40b1630 2024-03-29T08:30:12.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9fb8c00640 1 -- 172.21.15.46:0/3838098095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb4071a70 con 0x7f9fb40b1630 2024-03-29T08:30:12.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.277+0000 7f9faaa00640 1 -- 172.21.15.46:0/3838098095 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9fa40b3040 con 0x7f9fb40b1630 2024-03-29T08:30:12.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.281+0000 7f9faaa00640 1 --2- 172.21.15.46:0/3838098095 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f900666c0 0x7f9f90068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:12.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.281+0000 7f9faaa00640 1 -- 172.21.15.46:0/3838098095 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f9fa411e090 con 0x7f9fb40b1630 2024-03-29T08:30:12.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.281+0000 7f9fa9400640 1 --2- 172.21.15.46:0/3838098095 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f900666c0 0x7f9f90068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:12.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.281+0000 7f9fa9400640 1 --2- 172.21.15.46:0/3838098095 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f900666c0 0x7f9f90068b80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40ce3e0 tx=0x7f9f9c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:12.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.285+0000 7f9faaa00640 1 -- 172.21.15.46:0/3838098095 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9fa40eb3b0 con 0x7f9fb40b1630 2024-03-29T08:30:12.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.441+0000 7f9fb8c00640 1 -- 172.21.15.46:0/3838098095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9fb40cadd0 con 0x7f9fb40b1630 2024-03-29T08:30:12.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.441+0000 7f9faaa00640 1 -- 172.21.15.46:0/3838098095 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9fa40ee9d0 con 0x7f9fb40b1630 2024-03-29T08:30:12.443 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:30:12.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.445+0000 7f9fb8c00640 1 -- 172.21.15.46:0/3838098095 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f900666c0 msgr2=0x7f9f90068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:12.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.445+0000 7f9fb8c00640 1 --2- 172.21.15.46:0/3838098095 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f900666c0 0x7f9f90068b80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40ce3e0 tx=0x7f9f9c008040 comp rx=0 tx=0).stop 2024-03-29T08:30:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.445+0000 7f9fb8c00640 1 -- 172.21.15.46:0/3838098095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb40b1630 msgr2=0x7f9fb4159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.445+0000 7f9fb8c00640 1 --2- 172.21.15.46:0/3838098095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb40b1630 0x7f9fb4159b30 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4000c10 tx=0x7f9fa4019220 comp rx=0 tx=0).stop 2024-03-29T08:30:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.445+0000 7f9fb8c00640 1 -- 172.21.15.46:0/3838098095 shutdown_connections 2024-03-29T08:30:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.445+0000 7f9fb8c00640 1 --2- 172.21.15.46:0/3838098095 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9f900666c0 0x7f9f90068b80 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.445+0000 7f9fb8c00640 1 --2- 172.21.15.46:0/3838098095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fb415efe0 0x7f9fb413cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.445+0000 7f9fb8c00640 1 --2- 172.21.15.46:0/3838098095 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9fb4143eb0 0x7f9fb415fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.445+0000 7f9fb8c00640 1 --2- 172.21.15.46:0/3838098095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9fb40b1630 0x7f9fb4159b30 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.445+0000 7f9fb8c00640 1 -- 172.21.15.46:0/3838098095 >> 172.21.15.46:0/3838098095 conn(0x7f9fb406a390 msgr2=0x7f9fb4140a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.445+0000 7f9fb8c00640 1 -- 172.21.15.46:0/3838098095 shutdown_connections 2024-03-29T08:30:12.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.445+0000 7f9fb8c00640 1 -- 172.21.15.46:0/3838098095 wait complete. 2024-03-29T08:30:12.465 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-29T08:30:12.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.605+0000 7f4371000640 1 Processor -- start 2024-03-29T08:30:12.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.605+0000 7f4371000640 1 -- start start 2024-03-29T08:30:12.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4371000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f436c0709b0 0x7f436c0af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:12.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4371000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f436c06c390 con 0x7f436c158300 2024-03-29T08:30:12.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4371000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f436c0afff0 con 0x7f436c0af530 2024-03-29T08:30:12.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4371000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c06c5e0 con 0x7f436c0709b0 2024-03-29T08:30:12.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f436c158300 0x7f436c06d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42020/0 (socket says 172.21.15.46:42020) 2024-03-29T08:30:12.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369e00640 1 -- 172.21.15.46:0/1392897661 learned_addr learned my addr 172.21.15.46:0/1392897661 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369400640 1 -- 172.21.15.46:0/1392897661 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179988626 0 0) 0x7f436c06c390 con 0x7f436c158300 2024-03-29T08:30:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f436a800640 1 --2- 172.21.15.46:0/1392897661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f436c0709b0 0x7f436c0af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369400640 1 -- 172.21.15.46:0/1392897661 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4350003530 con 0x7f436c158300 2024-03-29T08:30:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369400640 1 -- 172.21.15.46:0/1392897661 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3683501731 0 0) 0x7f4350003530 con 0x7f436c158300 2024-03-29T08:30:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369400640 1 -- 172.21.15.46:0/1392897661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f436c0709b0 msgr2=0x7f436c0af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369400640 1 --2- 172.21.15.46:0/1392897661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f436c0709b0 0x7f436c0af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369400640 1 -- 172.21.15.46:0/1392897661 >> v1:172.21.15.73:6789/0 conn(0x7f436c0af530 legacy=0x7f436c0702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369400640 1 -- 172.21.15.46:0/1392897661 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436c06c8c0 con 0x7f436c158300 2024-03-29T08:30:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369400640 1 -- 172.21.15.46:0/1392897661 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f435c001680 con 0x7f436c158300 2024-03-29T08:30:12.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369400640 1 -- 172.21.15.46:0/1392897661 >> v1:172.21.15.46:6789/0 conn(0x7f436c158300 legacy=0x7f436c06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369400640 1 --2- 172.21.15.46:0/1392897661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f436c0709b0 0x7f43500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369400640 1 --2- 172.21.15.46:0/1392897661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4350003f30 0x7f43500047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369400640 1 --2- 172.21.15.46:0/1392897661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4350004d10 0x7f4350005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369400640 1 -- 172.21.15.46:0/1392897661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c06c5e0 con 0x7f4350003f30 2024-03-29T08:30:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369400640 1 -- 172.21.15.46:0/1392897661 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f435000bbd0 con 0x7f4350004d10 2024-03-29T08:30:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369400640 1 -- 172.21.15.46:0/1392897661 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f435000bd90 con 0x7f436c0709b0 2024-03-29T08:30:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4361400640 1 --2- 172.21.15.46:0/1392897661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4350003f30 0x7f43500047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4369e00640 1 --2- 172.21.15.46:0/1392897661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f436c0709b0 0x7f43500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f436a800640 1 --2- 172.21.15.46:0/1392897661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4350004d10 0x7f4350005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4361400640 1 -- 172.21.15.46:0/1392897661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f436c0709b0 msgr2=0x7f43500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4361400640 1 --2- 172.21.15.46:0/1392897661 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f436c0709b0 0x7f43500039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4361400640 1 -- 172.21.15.46:0/1392897661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4350004d10 msgr2=0x7f4350005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4361400640 1 --2- 172.21.15.46:0/1392897661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4350004d10 0x7f4350005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.609+0000 7f4361400640 1 -- 172.21.15.46:0/1392897661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4350003530 con 0x7f4350003f30 2024-03-29T08:30:12.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.613+0000 7f4361400640 1 --2- 172.21.15.46:0/1392897661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4350003f30 0x7f43500047d0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f4354001780 tx=0x7f435401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f57b064cec5821 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.613+0000 7f4369400640 1 -- 172.21.15.46:0/1392897661 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43540052d0 con 0x7f4350003f30 2024-03-29T08:30:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.613+0000 7f4369400640 1 -- 172.21.15.46:0/1392897661 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43540a7050 con 0x7f4350003f30 2024-03-29T08:30:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.613+0000 7f4369400640 1 -- 172.21.15.46:0/1392897661 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354003600 con 0x7f4350003f30 2024-03-29T08:30:12.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.613+0000 7f4371000640 1 -- 172.21.15.46:0/1392897661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4350003f30 msgr2=0x7f43500047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.613+0000 7f4371000640 1 --2- 172.21.15.46:0/1392897661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4350003f30 0x7f43500047d0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f4354001780 tx=0x7f435401c3f0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.613+0000 7f436a800640 1 -- 172.21.15.46:0/1392897661 reap_dead start 2024-03-29T08:30:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.613+0000 7f4371000640 1 -- 172.21.15.46:0/1392897661 shutdown_connections 2024-03-29T08:30:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.613+0000 7f4371000640 1 -- 172.21.15.46:0/1392897661 >> 172.21.15.46:0/1392897661 conn(0x7f436c066080 msgr2=0x7f436c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.613+0000 7f4371000640 1 -- 172.21.15.46:0/1392897661 shutdown_connections 2024-03-29T08:30:12.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.613+0000 7f4371000640 1 -- 172.21.15.46:0/1392897661 wait complete. 2024-03-29T08:30:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.613+0000 7f4371000640 1 Processor -- start 2024-03-29T08:30:12.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.613+0000 7f4371000640 1 -- start start 2024-03-29T08:30:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4371000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4350003f30 0x7f436c14f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4371000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f436c14e0b0 0x7f436c0cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:12.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4371000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f436c0cccc0 0x7f436c150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4371000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c0cd140 con 0x7f436c14e0b0 2024-03-29T08:30:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4371000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436c14e530 con 0x7f4350003f30 2024-03-29T08:30:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4371000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f436c14e790 con 0x7f436c0cccc0 2024-03-29T08:30:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4361400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f436c14e0b0 0x7f436c0cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4361400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f436c14e0b0 0x7f436c0cc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48952/0 (socket says 172.21.15.46:48952) 2024-03-29T08:30:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4361400640 1 -- 172.21.15.46:0/839625818 learned_addr learned my addr 172.21.15.46:0/839625818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4369e00640 1 --2- 172.21.15.46:0/839625818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4350003f30 0x7f436c14f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f436a800640 1 --2- 172.21.15.46:0/839625818 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f436c0cccc0 0x7f436c150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4361400640 1 -- 172.21.15.46:0/839625818 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f436c0cccc0 msgr2=0x7f436c150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4361400640 1 --2- 172.21.15.46:0/839625818 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f436c0cccc0 0x7f436c150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4361400640 1 -- 172.21.15.46:0/839625818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4350003f30 msgr2=0x7f436c14f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4361400640 1 --2- 172.21.15.46:0/839625818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4350003f30 0x7f436c14f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4361400640 1 -- 172.21.15.46:0/839625818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f436c14ea70 con 0x7f436c14e0b0 2024-03-29T08:30:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4361400640 1 --2- 172.21.15.46:0/839625818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f436c14e0b0 0x7f436c0cc780 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f4354005040 tx=0x7f4350005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4362a00640 1 -- 172.21.15.46:0/839625818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354007180 con 0x7f436c14e0b0 2024-03-29T08:30:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4362a00640 1 -- 172.21.15.46:0/839625818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43540072e0 con 0x7f436c14e0b0 2024-03-29T08:30:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4362a00640 1 -- 172.21.15.46:0/839625818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4354007670 con 0x7f436c14e0b0 2024-03-29T08:30:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4371000640 1 -- 172.21.15.46:0/839625818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436c14ed50 con 0x7f436c14e0b0 2024-03-29T08:30:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4371000640 1 -- 172.21.15.46:0/839625818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436c14f310 con 0x7f436c14e0b0 2024-03-29T08:30:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.617+0000 7f4371000640 1 -- 172.21.15.46:0/839625818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436c071a90 con 0x7f436c14e0b0 2024-03-29T08:30:12.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.621+0000 7f4362a00640 1 -- 172.21.15.46:0/839625818 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f43540067e0 con 0x7f436c14e0b0 2024-03-29T08:30:12.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.621+0000 7f4362a00640 1 --2- 172.21.15.46:0/839625818 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4348066690 0x7f4348068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:12.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.621+0000 7f4362a00640 1 -- 172.21.15.46:0/839625818 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f435411e420 con 0x7f436c14e0b0 2024-03-29T08:30:12.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.621+0000 7f4369e00640 1 --2- 172.21.15.46:0/839625818 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4348066690 0x7f4348068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:12.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.621+0000 7f4369e00640 1 --2- 172.21.15.46:0/839625818 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4348066690 0x7f4348068b50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f435c001b50 tx=0x7f435c001090 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:12.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.625+0000 7f4362a00640 1 -- 172.21.15.46:0/839625818 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f43540eb900 con 0x7f436c14e0b0 2024-03-29T08:30:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.801+0000 7f4371000640 1 -- 172.21.15.46:0/839625818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f436c04e120 con 0x7f436c14e0b0 2024-03-29T08:30:12.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.801+0000 7f4362a00640 1 -- 172.21.15.46:0/839625818 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v125) v1 ==== 80+0+240 (secure 0 0 0) 0x7f43540eef20 con 0x7f436c14e0b0 2024-03-29T08:30:12.801 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6834/276185207 2024-03-30T08:30:00.063526+0000 2024-03-29T08:30:12.801 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6835/929698978 2024-03-30T08:29:45.374806+0000 2024-03-29T08:30:12.802 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6835/276185207 2024-03-30T08:30:00.063526+0000 2024-03-29T08:30:12.802 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6834/929698978 2024-03-30T08:29:45.374806+0000 2024-03-29T08:30:12.802 INFO:teuthology.orchestra.run.smithi046.stderr:listed 4 entries 2024-03-29T08:30:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.801+0000 7f4371000640 1 -- 172.21.15.46:0/839625818 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4348066690 msgr2=0x7f4348068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.801+0000 7f4371000640 1 --2- 172.21.15.46:0/839625818 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4348066690 0x7f4348068b50 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f435c001b50 tx=0x7f435c001090 comp rx=0 tx=0).stop 2024-03-29T08:30:12.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.801+0000 7f4371000640 1 -- 172.21.15.46:0/839625818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f436c14e0b0 msgr2=0x7f436c0cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.801+0000 7f4371000640 1 --2- 172.21.15.46:0/839625818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f436c14e0b0 0x7f436c0cc780 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f4354005040 tx=0x7f4350005800 comp rx=0 tx=0).stop 2024-03-29T08:30:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.801+0000 7f4371000640 1 -- 172.21.15.46:0/839625818 shutdown_connections 2024-03-29T08:30:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.801+0000 7f4371000640 1 --2- 172.21.15.46:0/839625818 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4348066690 0x7f4348068b50 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.801+0000 7f4371000640 1 --2- 172.21.15.46:0/839625818 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f436c0cccc0 0x7f436c150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.801+0000 7f4371000640 1 --2- 172.21.15.46:0/839625818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f436c14e0b0 0x7f436c0cc780 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.801+0000 7f4371000640 1 --2- 172.21.15.46:0/839625818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4350003f30 0x7f436c14f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.801+0000 7f4371000640 1 -- 172.21.15.46:0/839625818 >> 172.21.15.46:0/839625818 conn(0x7f436c066080 msgr2=0x7f436c156ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.801+0000 7f4371000640 1 -- 172.21.15.46:0/839625818 shutdown_connections 2024-03-29T08:30:12.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:12.801+0000 7f4371000640 1 -- 172.21.15.46:0/839625818 wait complete. 2024-03-29T08:30:12.826 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2024-03-29T08:30:12.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:12.826 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-29T08:30:12.844 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[52871]: fuse finished with error 0 and tester_r 0 2024-03-29T08:30:12.847 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:30:12.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:12.847 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-29T08:30:12.981 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:30:12.981 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:30:12.991 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:30:12.991 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:30:12.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:12.991 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-03-29T08:30:13.042 INFO:teuthology.orchestra.run.smithi073.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-03-29T08:30:13.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:30:13.043 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:30:13.044 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:30:13.053 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:30:13.053 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:30:13.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:13.053 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-03-29T08:30:13.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:30:13.105 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-03-29T08:30:13.105 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:30:13.105 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:30:13.114 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:30:13.114 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:30:13.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:13.115 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-03-29T08:30:13.166 INFO:teuthology.orchestra.run.smithi073.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-03-29T08:30:13.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:30:13.167 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:30:13.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f116a800640 1 Processor -- start 2024-03-29T08:30:13.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f116a800640 1 -- start start 2024-03-29T08:30:13.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f116a800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1164099150 0x7f11640cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:13.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f116a800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11640cb5d0 con 0x7f116415cb00 2024-03-29T08:30:13.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f116a800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11640caca0 con 0x7f1164099810 2024-03-29T08:30:13.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f116a800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1164099d10 con 0x7f1164099150 2024-03-29T08:30:13.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1169400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f116415cb00 0x7f1164098a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42024/0 (socket says 172.21.15.46:42024) 2024-03-29T08:30:13.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1169400640 1 -- 172.21.15.46:0/1416752507 learned_addr learned my addr 172.21.15.46:0/1416752507 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:13.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1168a00640 1 --2- 172.21.15.46:0/1416752507 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1164099150 0x7f11640cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:13.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1163e00640 1 -- 172.21.15.46:0/1416752507 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149241986 0 0) 0x7f11640cb5d0 con 0x7f116415cb00 2024-03-29T08:30:13.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1163e00640 1 -- 172.21.15.46:0/1416752507 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f114c003530 con 0x7f116415cb00 2024-03-29T08:30:13.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1163e00640 1 -- 172.21.15.46:0/1416752507 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777512171 0 0) 0x7f11640caca0 con 0x7f1164099810 2024-03-29T08:30:13.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1163e00640 1 -- 172.21.15.46:0/1416752507 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11640cb5d0 con 0x7f1164099810 2024-03-29T08:30:13.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1163e00640 1 -- 172.21.15.46:0/1416752507 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3952932255 0 0) 0x7f114c003530 con 0x7f116415cb00 2024-03-29T08:30:13.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1163e00640 1 -- 172.21.15.46:0/1416752507 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1164099150 msgr2=0x7f11640cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:13.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1163e00640 1 --2- 172.21.15.46:0/1416752507 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1164099150 0x7f11640cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1163e00640 1 -- 172.21.15.46:0/1416752507 >> v1:172.21.15.73:6789/0 conn(0x7f1164099810 legacy=0x7f11640cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:13.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1163e00640 1 -- 172.21.15.46:0/1416752507 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11640caf80 con 0x7f116415cb00 2024-03-29T08:30:13.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1163e00640 1 -- 172.21.15.46:0/1416752507 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7f1158001600 con 0x7f116415cb00 2024-03-29T08:30:13.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1163e00640 1 -- 172.21.15.46:0/1416752507 >> v1:172.21.15.46:6789/0 conn(0x7f116415cb00 legacy=0x7f1164098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:13.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1163e00640 1 --2- 172.21.15.46:0/1416752507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f114c003b40 0x7f114c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:13.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1163e00640 1 --2- 172.21.15.46:0/1416752507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f114c0047d0 0x7f114c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:13.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1163e00640 1 --2- 172.21.15.46:0/1416752507 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1164099150 0x7f114c0051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:13.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1163e00640 1 -- 172.21.15.46:0/1416752507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1164099d10 con 0x7f114c003b40 2024-03-29T08:30:13.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1163e00640 1 -- 172.21.15.46:0/1416752507 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c00bb20 con 0x7f114c0047d0 2024-03-29T08:30:13.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1163e00640 1 -- 172.21.15.46:0/1416752507 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114c00bcb0 con 0x7f1164099150 2024-03-29T08:30:13.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1169400640 1 --2- 172.21.15.46:0/1416752507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f114c003b40 0x7f114c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:13.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1168a00640 1 --2- 172.21.15.46:0/1416752507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f114c0047d0 0x7f114c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:13.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1169e00640 1 --2- 172.21.15.46:0/1416752507 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1164099150 0x7f114c0051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:13.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1169400640 1 -- 172.21.15.46:0/1416752507 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1164099150 msgr2=0x7f114c0051f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:13.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1169400640 1 --2- 172.21.15.46:0/1416752507 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1164099150 0x7f114c0051f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1169400640 1 -- 172.21.15.46:0/1416752507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f114c0047d0 msgr2=0x7f114c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:13.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1169400640 1 --2- 172.21.15.46:0/1416752507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f114c0047d0 0x7f114c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.309+0000 7f1169400640 1 -- 172.21.15.46:0/1416752507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c003530 con 0x7f114c003b40 2024-03-29T08:30:13.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.313+0000 7f1169400640 1 --2- 172.21.15.46:0/1416752507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f114c003b40 0x7f114c003f40 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f1158007f50 tx=0x7f115801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf063b97e84ed312 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:13.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.313+0000 7f1163e00640 1 -- 172.21.15.46:0/1416752507 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1158005ae0 con 0x7f114c003b40 2024-03-29T08:30:13.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.313+0000 7f1163e00640 1 -- 172.21.15.46:0/1416752507 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1158005c60 con 0x7f114c003b40 2024-03-29T08:30:13.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.313+0000 7f1163e00640 1 -- 172.21.15.46:0/1416752507 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1158007070 con 0x7f114c003b40 2024-03-29T08:30:13.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.313+0000 7f116a800640 1 -- 172.21.15.46:0/1416752507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f114c003b40 msgr2=0x7f114c003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:13.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.313+0000 7f116a800640 1 --2- 172.21.15.46:0/1416752507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f114c003b40 0x7f114c003f40 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f1158007f50 tx=0x7f115801ca70 comp rx=0 tx=0).stop 2024-03-29T08:30:13.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.313+0000 7f1169e00640 1 -- 172.21.15.46:0/1416752507 reap_dead start 2024-03-29T08:30:13.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.313+0000 7f116a800640 1 -- 172.21.15.46:0/1416752507 shutdown_connections 2024-03-29T08:30:13.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.313+0000 7f116a800640 1 -- 172.21.15.46:0/1416752507 >> 172.21.15.46:0/1416752507 conn(0x7f11640b2010 msgr2=0x7f11640b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:13.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.313+0000 7f116a800640 1 -- 172.21.15.46:0/1416752507 shutdown_connections 2024-03-29T08:30:13.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.313+0000 7f116a800640 1 -- 172.21.15.46:0/1416752507 wait complete. 2024-03-29T08:30:13.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f116a800640 1 Processor -- start 2024-03-29T08:30:13.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f116a800640 1 -- start start 2024-03-29T08:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f116a800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f114c003b40 0x7f116414c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f116a800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11640aef80 0x7f1164167230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f116a800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f116406f120 0x7f116406f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f116a800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116406fae0 con 0x7f114c003b40 2024-03-29T08:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f116a800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11640af400 con 0x7f11640aef80 2024-03-29T08:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f116a800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11640c89d0 con 0x7f116406f120 2024-03-29T08:30:13.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f1169400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f114c003b40 0x7f116414c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f1169400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f114c003b40 0x7f116414c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48978/0 (socket says 172.21.15.46:48978) 2024-03-29T08:30:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f1169400640 1 -- 172.21.15.46:0/548768193 learned_addr learned my addr 172.21.15.46:0/548768193 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f1168a00640 1 --2- 172.21.15.46:0/548768193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11640aef80 0x7f1164167230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f1169e00640 1 --2- 172.21.15.46:0/548768193 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f116406f120 0x7f116406f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f1169400640 1 -- 172.21.15.46:0/548768193 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f116406f120 msgr2=0x7f116406f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f1169400640 1 --2- 172.21.15.46:0/548768193 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f116406f120 0x7f116406f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f1169400640 1 -- 172.21.15.46:0/548768193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11640aef80 msgr2=0x7f1164167230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f1169400640 1 --2- 172.21.15.46:0/548768193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11640aef80 0x7f1164167230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f1169400640 1 -- 172.21.15.46:0/548768193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11640c9d10 con 0x7f114c003b40 2024-03-29T08:30:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f1168a00640 1 --2- 172.21.15.46:0/548768193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11640aef80 0x7f1164167230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:13.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f1169400640 1 --2- 172.21.15.46:0/548768193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f114c003b40 0x7f116414c790 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f115801c8a0 tx=0x7f114c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:13.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f1161600640 1 -- 172.21.15.46:0/548768193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11580b4070 con 0x7f114c003b40 2024-03-29T08:30:13.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f1161600640 1 -- 172.21.15.46:0/548768193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1158001400 con 0x7f114c003b40 2024-03-29T08:30:13.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f116a800640 1 -- 172.21.15.46:0/548768193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11640c8c70 con 0x7f114c003b40 2024-03-29T08:30:13.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f1161600640 1 -- 172.21.15.46:0/548768193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11580072f0 con 0x7f114c003b40 2024-03-29T08:30:13.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.317+0000 7f116a800640 1 -- 172.21.15.46:0/548768193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11640c9230 con 0x7f114c003b40 2024-03-29T08:30:13.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.321+0000 7f116a800640 1 -- 172.21.15.46:0/548768193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116404e040 con 0x7f114c003b40 2024-03-29T08:30:13.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.321+0000 7f1161600640 1 -- 172.21.15.46:0/548768193 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1158007450 con 0x7f114c003b40 2024-03-29T08:30:13.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.321+0000 7f1161600640 1 --2- 172.21.15.46:0/548768193 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11440666c0 0x7f1144068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:13.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.321+0000 7f1161600640 1 -- 172.21.15.46:0/548768193 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f115811dc50 con 0x7f114c003b40 2024-03-29T08:30:13.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.321+0000 7f1168a00640 1 --2- 172.21.15.46:0/548768193 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11440666c0 0x7f1144068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:13.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.321+0000 7f1168a00640 1 --2- 172.21.15.46:0/548768193 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11440666c0 0x7f1144068b80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f116414c740 tx=0x7f1150008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:13.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.325+0000 7f1161600640 1 -- 172.21.15.46:0/548768193 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f11580eae40 con 0x7f114c003b40 2024-03-29T08:30:13.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.517+0000 7f116a800640 1 -- 172.21.15.46:0/548768193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f116406df40 con 0x7f114c003b40 2024-03-29T08:30:13.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.517+0000 7f1161600640 1 -- 172.21.15.46:0/548768193 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4741 (secure 0 0 0) 0x7f11580ee460 con 0x7f114c003b40 2024-03-29T08:30:13.520 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:13.520 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":107,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2506132630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2506132630},{"type":"v1","addr":"172.21.15.73:6836","nonce":2506132630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2108115501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2108115501},{"type":"v1","addr":"172.21.15.73:6837","nonce":2108115501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":9045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3210062895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3210062895},{"type":"v1","addr":"172.21.15.46:6833","nonce":3210062895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:29:14.470858+0000","modified":"2024-03-29T08:30:05.103674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9660":{"gid":9660,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/3867471367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3867471367},{"type":"v1","addr":"172.21.15.46:6835","nonce":3867471367}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-29T08:30:13.520 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 107 2024-03-29T08:30:13.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.521+0000 7f116a800640 1 -- 172.21.15.46:0/548768193 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11440666c0 msgr2=0x7f1144068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:13.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.521+0000 7f116a800640 1 --2- 172.21.15.46:0/548768193 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11440666c0 0x7f1144068b80 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f116414c740 tx=0x7f1150008040 comp rx=0 tx=0).stop 2024-03-29T08:30:13.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.521+0000 7f116a800640 1 -- 172.21.15.46:0/548768193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f114c003b40 msgr2=0x7f116414c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.521+0000 7f116a800640 1 --2- 172.21.15.46:0/548768193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f114c003b40 0x7f116414c790 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f115801c8a0 tx=0x7f114c005800 comp rx=0 tx=0).stop 2024-03-29T08:30:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.521+0000 7f116a800640 1 -- 172.21.15.46:0/548768193 shutdown_connections 2024-03-29T08:30:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.521+0000 7f116a800640 1 --2- 172.21.15.46:0/548768193 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f11440666c0 0x7f1144068b80 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.521+0000 7f116a800640 1 --2- 172.21.15.46:0/548768193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11640aef80 0x7f1164167230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.521+0000 7f116a800640 1 --2- 172.21.15.46:0/548768193 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f116406f120 0x7f116406f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.521+0000 7f116a800640 1 --2- 172.21.15.46:0/548768193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f114c003b40 0x7f116414c790 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.521+0000 7f116a800640 1 -- 172.21.15.46:0/548768193 >> 172.21.15.46:0/548768193 conn(0x7f11640b2010 msgr2=0x7f1164098a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.521+0000 7f116a800640 1 -- 172.21.15.46:0/548768193 shutdown_connections 2024-03-29T08:30:13.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.521+0000 7f116a800640 1 -- 172.21.15.46:0/548768193 wait complete. 2024-03-29T08:30:13.541 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:30:13.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc959200640 1 Processor -- start 2024-03-29T08:30:13.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc959200640 1 -- start start 2024-03-29T08:30:13.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc959200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc95415dc00 0x7fc9540cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:13.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc959200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95415cfb0 con 0x7fc95406b8b0 2024-03-29T08:30:13.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc959200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95406be20 con 0x7fc9540d0660 2024-03-29T08:30:13.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc959200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95406c100 con 0x7fc95415dc00 2024-03-29T08:30:13.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc94be00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc95406b8b0 0x7fc9540cad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42040/0 (socket says 172.21.15.46:42040) 2024-03-29T08:30:13.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc94be00640 1 -- 172.21.15.46:0/3270398413 learned_addr learned my addr 172.21.15.46:0/3270398413 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:13.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc951400640 1 -- 172.21.15.46:0/3270398413 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4113738397 0 0) 0x7fc95415cfb0 con 0x7fc95406b8b0 2024-03-29T08:30:13.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc952800640 1 --2- 172.21.15.46:0/3270398413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc95415dc00 0x7fc9540cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:13.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc951400640 1 -- 172.21.15.46:0/3270398413 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc938003530 con 0x7fc95406b8b0 2024-03-29T08:30:13.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc951400640 1 -- 172.21.15.46:0/3270398413 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3747725576 0 0) 0x7fc938003530 con 0x7fc95406b8b0 2024-03-29T08:30:13.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc951400640 1 -- 172.21.15.46:0/3270398413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc95415dc00 msgr2=0x7fc9540cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:13.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc951400640 1 --2- 172.21.15.46:0/3270398413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc95415dc00 0x7fc9540cf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc951400640 1 -- 172.21.15.46:0/3270398413 >> v1:172.21.15.73:6789/0 conn(0x7fc9540d0660 legacy=0x7fc9540d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:13.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc951400640 1 -- 172.21.15.46:0/3270398413 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9540cecf0 con 0x7fc95406b8b0 2024-03-29T08:30:13.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc951400640 1 -- 172.21.15.46:0/3270398413 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fc93c001680 con 0x7fc95406b8b0 2024-03-29T08:30:13.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc951400640 1 -- 172.21.15.46:0/3270398413 >> v1:172.21.15.46:6789/0 conn(0x7fc95406b8b0 legacy=0x7fc9540cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:13.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc951400640 1 --2- 172.21.15.46:0/3270398413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc95415dc00 0x7fc9380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:13.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc951400640 1 --2- 172.21.15.46:0/3270398413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc938003f80 0x7fc9380047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:13.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc951400640 1 --2- 172.21.15.46:0/3270398413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc938004d10 0x7fc938005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:13.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc951400640 1 -- 172.21.15.46:0/3270398413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95406c100 con 0x7fc938003f80 2024-03-29T08:30:13.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc951400640 1 -- 172.21.15.46:0/3270398413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93800bad0 con 0x7fc938004d10 2024-03-29T08:30:13.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc951400640 1 -- 172.21.15.46:0/3270398413 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93800bc90 con 0x7fc95415dc00 2024-03-29T08:30:13.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc94be00640 1 --2- 172.21.15.46:0/3270398413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc938003f80 0x7fc9380047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:13.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc951e00640 1 --2- 172.21.15.46:0/3270398413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc95415dc00 0x7fc9380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:13.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.681+0000 7fc952800640 1 --2- 172.21.15.46:0/3270398413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc938004d10 0x7fc938005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:13.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.685+0000 7fc94be00640 1 -- 172.21.15.46:0/3270398413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc95415dc00 msgr2=0x7fc9380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:13.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.685+0000 7fc94be00640 1 --2- 172.21.15.46:0/3270398413 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc95415dc00 0x7fc9380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.685+0000 7fc94be00640 1 -- 172.21.15.46:0/3270398413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc938004d10 msgr2=0x7fc938005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:13.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.685+0000 7fc94be00640 1 --2- 172.21.15.46:0/3270398413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc938004d10 0x7fc938005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.685+0000 7fc94be00640 1 -- 172.21.15.46:0/3270398413 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc938003530 con 0x7fc938003f80 2024-03-29T08:30:13.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.685+0000 7fc94be00640 1 --2- 172.21.15.46:0/3270398413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc938003f80 0x7fc9380047d0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00c0e0 tx=0x7fc93c01f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8215dae8c98ce20 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:13.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.685+0000 7fc951400640 1 -- 172.21.15.46:0/3270398413 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0a8e50 con 0x7fc938003f80 2024-03-29T08:30:13.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.685+0000 7fc951400640 1 -- 172.21.15.46:0/3270398413 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c0058e0 con 0x7fc938003f80 2024-03-29T08:30:13.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.685+0000 7fc951400640 1 -- 172.21.15.46:0/3270398413 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c005d10 con 0x7fc938003f80 2024-03-29T08:30:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.685+0000 7fc959200640 1 -- 172.21.15.46:0/3270398413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc938003f80 msgr2=0x7fc9380047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.685+0000 7fc959200640 1 --2- 172.21.15.46:0/3270398413 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc938003f80 0x7fc9380047d0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fc93c00c0e0 tx=0x7fc93c01f230 comp rx=0 tx=0).stop 2024-03-29T08:30:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.685+0000 7fc952800640 1 -- 172.21.15.46:0/3270398413 reap_dead start 2024-03-29T08:30:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.685+0000 7fc959200640 1 -- 172.21.15.46:0/3270398413 shutdown_connections 2024-03-29T08:30:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.685+0000 7fc959200640 1 -- 172.21.15.46:0/3270398413 >> 172.21.15.46:0/3270398413 conn(0x7fc954089fd0 msgr2=0x7fc95408a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.685+0000 7fc959200640 1 -- 172.21.15.46:0/3270398413 shutdown_connections 2024-03-29T08:30:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.685+0000 7fc959200640 1 -- 172.21.15.46:0/3270398413 wait complete. 2024-03-29T08:30:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc959200640 1 Processor -- start 2024-03-29T08:30:13.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc959200640 1 -- start start 2024-03-29T08:30:13.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc959200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc938003f80 0x7fc954151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:13.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc959200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9541f21a0 0x7fc9540b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:13.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc959200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9540b0a60 0x7fc9540ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:13.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc959200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9540b4d70 con 0x7fc9541f21a0 2024-03-29T08:30:13.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc959200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9540b4460 con 0x7fc9540b0a60 2024-03-29T08:30:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc959200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9540b0fa0 con 0x7fc938003f80 2024-03-29T08:30:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc94be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9541f21a0 0x7fc9540b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc94be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9541f21a0 0x7fc9540b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48998/0 (socket says 172.21.15.46:48998) 2024-03-29T08:30:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc94be00640 1 -- 172.21.15.46:0/58751750 learned_addr learned my addr 172.21.15.46:0/58751750 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc951e00640 1 --2- 172.21.15.46:0/58751750 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc938003f80 0x7fc954151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc952800640 1 --2- 172.21.15.46:0/58751750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9540b0a60 0x7fc9540ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc94be00640 1 -- 172.21.15.46:0/58751750 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc938003f80 msgr2=0x7fc954151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc94be00640 1 --2- 172.21.15.46:0/58751750 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc938003f80 0x7fc954151a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc94be00640 1 -- 172.21.15.46:0/58751750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9540b0a60 msgr2=0x7fc9540ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc94be00640 1 --2- 172.21.15.46:0/58751750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9540b0a60 0x7fc9540ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc94be00640 1 -- 172.21.15.46:0/58751750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9540b46b0 con 0x7fc9541f21a0 2024-03-29T08:30:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc94be00640 1 --2- 172.21.15.46:0/58751750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9541f21a0 0x7fc9540b0520 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fc93c01f060 tx=0x7fc938005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc94a000640 1 -- 172.21.15.46:0/58751750 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c0b7070 con 0x7fc9541f21a0 2024-03-29T08:30:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc94a000640 1 -- 172.21.15.46:0/58751750 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc93c001600 con 0x7fc9541f21a0 2024-03-29T08:30:13.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc959200640 1 -- 172.21.15.46:0/58751750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9540b4990 con 0x7fc9541f21a0 2024-03-29T08:30:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc94a000640 1 -- 172.21.15.46:0/58751750 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93c005c50 con 0x7fc9541f21a0 2024-03-29T08:30:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.689+0000 7fc959200640 1 -- 172.21.15.46:0/58751750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc954151f80 con 0x7fc9541f21a0 2024-03-29T08:30:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.693+0000 7fc959200640 1 -- 172.21.15.46:0/58751750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9540b4eb0 con 0x7fc9541f21a0 2024-03-29T08:30:13.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.693+0000 7fc94a000640 1 -- 172.21.15.46:0/58751750 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fc93c007090 con 0x7fc9541f21a0 2024-03-29T08:30:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.693+0000 7fc94a000640 1 --2- 172.21.15.46:0/58751750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc930066690 0x7fc930068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.693+0000 7fc94a000640 1 -- 172.21.15.46:0/58751750 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fc93c120fe0 con 0x7fc9541f21a0 2024-03-29T08:30:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.693+0000 7fc951e00640 1 --2- 172.21.15.46:0/58751750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc930066690 0x7fc930068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:13.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.693+0000 7fc951e00640 1 --2- 172.21.15.46:0/58751750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc930066690 0x7fc930068b50 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc95406f1c0 tx=0x7fc944009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:13.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.697+0000 7fc94a000640 1 -- 172.21.15.46:0/58751750 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fc93c0ee290 con 0x7fc9541f21a0 2024-03-29T08:30:13.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.861+0000 7fc959200640 1 -- 172.21.15.46:0/58751750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc954066c90 con 0x7fc9541f21a0 2024-03-29T08:30:13.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.861+0000 7fc94a000640 1 -- 172.21.15.46:0/58751750 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16406 (secure 0 0 0) 0x7fc93c005db0 con 0x7fc9541f21a0 2024-03-29T08:30:13.862 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:13.862 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":125,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:30:00.063547+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:29:12.329121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-29T08:29:13.465939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.7","mappings":[{"from":4,"to":3}]},{"pgid":"17.4","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6834/276185207":"2024-03-30T08:30:00.063526+0000","172.21.15.46:6835/929698978":"2024-03-30T08:29:45.374806+0000","172.21.15.46:6835/276185207":"2024-03-30T08:30:00.063526+0000","172.21.15.46:6834/929698978":"2024-03-30T08:29:45.374806+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:30:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.861+0000 7fc959200640 1 -- 172.21.15.46:0/58751750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc930066690 msgr2=0x7fc930068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.861+0000 7fc959200640 1 --2- 172.21.15.46:0/58751750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc930066690 0x7fc930068b50 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc95406f1c0 tx=0x7fc944009040 comp rx=0 tx=0).stop 2024-03-29T08:30:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.861+0000 7fc959200640 1 -- 172.21.15.46:0/58751750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9541f21a0 msgr2=0x7fc9540b0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.861+0000 7fc959200640 1 --2- 172.21.15.46:0/58751750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9541f21a0 0x7fc9540b0520 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fc93c01f060 tx=0x7fc938005800 comp rx=0 tx=0).stop 2024-03-29T08:30:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.861+0000 7fc959200640 1 -- 172.21.15.46:0/58751750 shutdown_connections 2024-03-29T08:30:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.861+0000 7fc959200640 1 --2- 172.21.15.46:0/58751750 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fc930066690 0x7fc930068b50 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.861+0000 7fc959200640 1 --2- 172.21.15.46:0/58751750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc9540b0a60 0x7fc9540ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.861+0000 7fc959200640 1 --2- 172.21.15.46:0/58751750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc9541f21a0 0x7fc9540b0520 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.861+0000 7fc959200640 1 --2- 172.21.15.46:0/58751750 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fc938003f80 0x7fc954151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.861+0000 7fc959200640 1 -- 172.21.15.46:0/58751750 >> 172.21.15.46:0/58751750 conn(0x7fc954089fd0 msgr2=0x7fc95408a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.861+0000 7fc959200640 1 -- 172.21.15.46:0/58751750 shutdown_connections 2024-03-29T08:30:13.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:13.865+0000 7fc959200640 1 -- 172.21.15.46:0/58751750 wait complete. 2024-03-29T08:30:13.884 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-29T08:18:00.103209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-29T08:29:12.329121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-03-29T08:29:13.465939+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-29T08:30:13.884 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-29T08:30:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.025+0000 7f4d67800640 1 Processor -- start 2024-03-29T08:30:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.025+0000 7f4d67800640 1 -- start start 2024-03-29T08:30:14.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.029+0000 7f4d67800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4d60143180 0x7f4d600adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.029+0000 7f4d67800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4d600ae1e0 0x7f4d600cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.029+0000 7f4d67800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d600afe80 con 0x7f4d60142da0 2024-03-29T08:30:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.029+0000 7f4d67800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d6006cd80 con 0x7f4d600ae1e0 2024-03-29T08:30:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.029+0000 7f4d67800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d6006d020 con 0x7f4d60143180 2024-03-29T08:30:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.029+0000 7f4d65800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4d600ae1e0 0x7f4d600cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.029+0000 7f4d65800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4d600ae1e0 0x7f4d600cb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49008/0 (socket says 172.21.15.46:49008) 2024-03-29T08:30:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.029+0000 7f4d65800640 1 -- 172.21.15.46:0/2666258955 learned_addr learned my addr 172.21.15.46:0/2666258955 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.029+0000 7f4d64e00640 1 --1- 172.21.15.46:0/2666258955 >> v1:172.21.15.46:6789/0 conn(0x7f4d60142da0 0x7f4d600abac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42044/0 (socket says 172.21.15.46:42044) 2024-03-29T08:30:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.029+0000 7f4d57e00640 1 --2- 172.21.15.46:0/2666258955 >> v2:172.21.15.73:3300/0 conn(0x7f4d60143180 0x7f4d600adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.029+0000 7f4d65800640 1 -- 172.21.15.46:0/2666258955 >> v2:172.21.15.73:3300/0 conn(0x7f4d60143180 msgr2=0x7f4d600adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.029+0000 7f4d65800640 1 --2- 172.21.15.46:0/2666258955 >> v2:172.21.15.73:3300/0 conn(0x7f4d60143180 0x7f4d600adb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.029+0000 7f4d65800640 1 -- 172.21.15.46:0/2666258955 >> v1:172.21.15.46:6789/0 conn(0x7f4d60142da0 legacy=0x7f4d600abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.029+0000 7f4d65800640 1 -- 172.21.15.46:0/2666258955 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d600b00f0 con 0x7f4d600ae1e0 2024-03-29T08:30:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.029+0000 7f4d57e00640 1 --2- 172.21.15.46:0/2666258955 >> v2:172.21.15.73:3300/0 conn(0x7f4d60143180 0x7f4d600adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.033+0000 7f4d65800640 1 --2- 172.21.15.46:0/2666258955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d600ae1e0 0x7f4d600cb790 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f4d6006d7c0 tx=0x7f4d5801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b5cab5394ae29e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.033+0000 7f4d5fe00640 1 -- 172.21.15.46:0/2666258955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d580a57d0 con 0x7f4d600ae1e0 2024-03-29T08:30:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.033+0000 7f4d5fe00640 1 -- 172.21.15.46:0/2666258955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d580a5930 con 0x7f4d600ae1e0 2024-03-29T08:30:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.033+0000 7f4d5fe00640 1 -- 172.21.15.46:0/2666258955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d580a5cc0 con 0x7f4d600ae1e0 2024-03-29T08:30:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.033+0000 7f4d67800640 1 -- 172.21.15.46:0/2666258955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d600ae1e0 msgr2=0x7f4d600cb790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.033+0000 7f4d67800640 1 --2- 172.21.15.46:0/2666258955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d600ae1e0 0x7f4d600cb790 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f4d6006d7c0 tx=0x7f4d5801c9f0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.033+0000 7f4d67800640 1 -- 172.21.15.46:0/2666258955 shutdown_connections 2024-03-29T08:30:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.033+0000 7f4d67800640 1 --2- 172.21.15.46:0/2666258955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d600ae1e0 0x7f4d600cb790 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.033+0000 7f4d67800640 1 --2- 172.21.15.46:0/2666258955 >> v2:172.21.15.73:3300/0 conn(0x7f4d60143180 0x7f4d600adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.033+0000 7f4d67800640 1 -- 172.21.15.46:0/2666258955 >> 172.21.15.46:0/2666258955 conn(0x7f4d6006a390 msgr2=0x7f4d600cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.033+0000 7f4d67800640 1 -- 172.21.15.46:0/2666258955 shutdown_connections 2024-03-29T08:30:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.033+0000 7f4d67800640 1 -- 172.21.15.46:0/2666258955 wait complete. 2024-03-29T08:30:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.033+0000 7f4d67800640 1 Processor -- start 2024-03-29T08:30:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.033+0000 7f4d67800640 1 -- start start 2024-03-29T08:30:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.033+0000 7f4d67800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d600ae1e0 0x7f4d6015b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.033+0000 7f4d67800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d60142da0 0x7f4d60140480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.033+0000 7f4d67800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d60143180 0x7f4d601409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d67800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d60141f30 con 0x7f4d60143180 2024-03-29T08:30:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d67800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d60140f00 con 0x7f4d600ae1e0 2024-03-29T08:30:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d67800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d60141150 con 0x7f4d60142da0 2024-03-29T08:30:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d65800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d60143180 0x7f4d601409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d65800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d60143180 0x7f4d601409c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49018/0 (socket says 172.21.15.46:49018) 2024-03-29T08:30:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d65800640 1 -- 172.21.15.46:0/177179217 learned_addr learned my addr 172.21.15.46:0/177179217 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d57e00640 1 --2- 172.21.15.46:0/177179217 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d60142da0 0x7f4d60140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d64e00640 1 --2- 172.21.15.46:0/177179217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d600ae1e0 0x7f4d6015b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d65800640 1 -- 172.21.15.46:0/177179217 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d60142da0 msgr2=0x7f4d60140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d65800640 1 --2- 172.21.15.46:0/177179217 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d60142da0 0x7f4d60140480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d65800640 1 -- 172.21.15.46:0/177179217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d600ae1e0 msgr2=0x7f4d6015b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d65800640 1 --2- 172.21.15.46:0/177179217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d600ae1e0 0x7f4d6015b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d65800640 1 -- 172.21.15.46:0/177179217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d58009b30 con 0x7f4d60143180 2024-03-29T08:30:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d57e00640 1 --2- 172.21.15.46:0/177179217 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d60142da0 0x7f4d60140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d64e00640 1 --2- 172.21.15.46:0/177179217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d600ae1e0 0x7f4d6015b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d65800640 1 --2- 172.21.15.46:0/177179217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d60143180 0x7f4d601409c0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f4d58019600 tx=0x7f4d58019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:14.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d5d600640 1 -- 172.21.15.46:0/177179217 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d580b8670 con 0x7f4d60143180 2024-03-29T08:30:14.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d5d600640 1 -- 172.21.15.46:0/177179217 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d580b87d0 con 0x7f4d60143180 2024-03-29T08:30:14.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d67800640 1 -- 172.21.15.46:0/177179217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d60141430 con 0x7f4d60143180 2024-03-29T08:30:14.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d5d600640 1 -- 172.21.15.46:0/177179217 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d580b8bc0 con 0x7f4d60143180 2024-03-29T08:30:14.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d67800640 1 -- 172.21.15.46:0/177179217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d60142460 con 0x7f4d60143180 2024-03-29T08:30:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d67800640 1 -- 172.21.15.46:0/177179217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d60142650 con 0x7f4d60143180 2024-03-29T08:30:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d5d600640 1 -- 172.21.15.46:0/177179217 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f4d580b8d20 con 0x7f4d60143180 2024-03-29T08:30:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.037+0000 7f4d5d600640 1 --2- 172.21.15.46:0/177179217 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4d3c066690 0x7f4d3c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.041+0000 7f4d5d600640 1 -- 172.21.15.46:0/177179217 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f4d5811fa20 con 0x7f4d60143180 2024-03-29T08:30:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.041+0000 7f4d64e00640 1 --2- 172.21.15.46:0/177179217 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4d3c066690 0x7f4d3c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.041+0000 7f4d64e00640 1 --2- 172.21.15.46:0/177179217 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4d3c066690 0x7f4d3c068b50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4d50000f70 tx=0x7f4d50007d70 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:14.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.041+0000 7f4d5d600640 1 -- 172.21.15.46:0/177179217 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f4d580ecdb0 con 0x7f4d60143180 2024-03-29T08:30:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.205+0000 7f4d67800640 1 -- 172.21.15.46:0/177179217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4d60071a70 con 0x7f4d60143180 2024-03-29T08:30:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.205+0000 7f4d5d600640 1 -- 172.21.15.46:0/177179217 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4d580f03d0 con 0x7f4d60143180 2024-03-29T08:30:14.207 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-03-29T08:30:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.209+0000 7f4d67800640 1 -- 172.21.15.46:0/177179217 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4d3c066690 msgr2=0x7f4d3c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.209+0000 7f4d67800640 1 --2- 172.21.15.46:0/177179217 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4d3c066690 0x7f4d3c068b50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4d50000f70 tx=0x7f4d50007d70 comp rx=0 tx=0).stop 2024-03-29T08:30:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.209+0000 7f4d67800640 1 -- 172.21.15.46:0/177179217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d60143180 msgr2=0x7f4d601409c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.209+0000 7f4d67800640 1 --2- 172.21.15.46:0/177179217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d60143180 0x7f4d601409c0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f4d58019600 tx=0x7f4d58019790 comp rx=0 tx=0).stop 2024-03-29T08:30:14.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.209+0000 7f4d67800640 1 -- 172.21.15.46:0/177179217 shutdown_connections 2024-03-29T08:30:14.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.209+0000 7f4d67800640 1 --2- 172.21.15.46:0/177179217 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f4d3c066690 0x7f4d3c068b50 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.209+0000 7f4d67800640 1 --2- 172.21.15.46:0/177179217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d60143180 0x7f4d601409c0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.209+0000 7f4d67800640 1 --2- 172.21.15.46:0/177179217 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f4d60142da0 0x7f4d60140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.209+0000 7f4d67800640 1 --2- 172.21.15.46:0/177179217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d600ae1e0 0x7f4d6015b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.209+0000 7f4d67800640 1 -- 172.21.15.46:0/177179217 >> 172.21.15.46:0/177179217 conn(0x7f4d6006a390 msgr2=0x7f4d6013bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:14.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.209+0000 7f4d67800640 1 -- 172.21.15.46:0/177179217 shutdown_connections 2024-03-29T08:30:14.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.209+0000 7f4d67800640 1 -- 172.21.15.46:0/177179217 wait complete. 2024-03-29T08:30:14.228 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-29T08:30:14.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.369+0000 7fa677a00640 1 Processor -- start 2024-03-29T08:30:14.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa677a00640 1 -- start start 2024-03-29T08:30:14.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa677a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa67015a7f0 0x7fa6700cf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa677a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6700cbd00 con 0x7fa670098a40 2024-03-29T08:30:14.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa677a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6700cbfe0 con 0x7fa67015abd0 2024-03-29T08:30:14.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa677a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6700b4500 con 0x7fa67015a7f0 2024-03-29T08:30:14.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa675a00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa670098a40 0x7fa670098e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42058/0 (socket says 172.21.15.46:42058) 2024-03-29T08:30:14.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa675a00640 1 -- 172.21.15.46:0/743568719 learned_addr learned my addr 172.21.15.46:0/743568719 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:14.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa675000640 1 --2- 172.21.15.46:0/743568719 >> v2:172.21.15.73:3300/0 conn(0x7fa67015a7f0 0x7fa6700cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa66fe00640 1 -- 172.21.15.46:0/743568719 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662649881 0 0) 0x7fa6700cbd00 con 0x7fa670098a40 2024-03-29T08:30:14.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa66fe00640 1 -- 172.21.15.46:0/743568719 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa654003530 con 0x7fa670098a40 2024-03-29T08:30:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa66fe00640 1 -- 172.21.15.46:0/743568719 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1536978417 0 0) 0x7fa654003530 con 0x7fa670098a40 2024-03-29T08:30:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa66fe00640 1 -- 172.21.15.46:0/743568719 >> v2:172.21.15.73:3300/0 conn(0x7fa67015a7f0 msgr2=0x7fa6700cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa66fe00640 1 --2- 172.21.15.46:0/743568719 >> v2:172.21.15.73:3300/0 conn(0x7fa67015a7f0 0x7fa6700cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa66fe00640 1 -- 172.21.15.46:0/743568719 >> v1:172.21.15.73:6789/0 conn(0x7fa67015abd0 legacy=0x7fa6700cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa66fe00640 1 -- 172.21.15.46:0/743568719 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700cc2c0 con 0x7fa670098a40 2024-03-29T08:30:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa66fe00640 1 -- 172.21.15.46:0/743568719 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2095130264 0 0) 0x7fa668001a00 con 0x7fa670098a40 2024-03-29T08:30:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa66fe00640 1 -- 172.21.15.46:0/743568719 >> v1:172.21.15.46:6789/0 conn(0x7fa670098a40 legacy=0x7fa670098e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa66fe00640 1 --2- 172.21.15.46:0/743568719 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa654003980 0x7fa654003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa66fe00640 1 --2- 172.21.15.46:0/743568719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa654004380 0x7fa654004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa66fe00640 1 --2- 172.21.15.46:0/743568719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6540050f0 0x7fa65400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa66fe00640 1 -- 172.21.15.46:0/743568719 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700b4500 con 0x7fa6540050f0 2024-03-29T08:30:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa66fe00640 1 -- 172.21.15.46:0/743568719 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65400c030 con 0x7fa654004380 2024-03-29T08:30:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa66fe00640 1 -- 172.21.15.46:0/743568719 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65400c1f0 con 0x7fa654003980 2024-03-29T08:30:14.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa675a00640 1 --2- 172.21.15.46:0/743568719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6540050f0 0x7fa65400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa675000640 1 --2- 172.21.15.46:0/743568719 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa654003980 0x7fa654003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa667e00640 1 --2- 172.21.15.46:0/743568719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa654004380 0x7fa654004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa675a00640 1 -- 172.21.15.46:0/743568719 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa654003980 msgr2=0x7fa654003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:14.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa675a00640 1 --2- 172.21.15.46:0/743568719 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa654003980 0x7fa654003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa675a00640 1 -- 172.21.15.46:0/743568719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa654004380 msgr2=0x7fa654004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:14.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa675a00640 1 --2- 172.21.15.46:0/743568719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa654004380 0x7fa654004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa675a00640 1 -- 172.21.15.46:0/743568719 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa654003530 con 0x7fa6540050f0 2024-03-29T08:30:14.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.373+0000 7fa667e00640 1 --2- 172.21.15.46:0/743568719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa654004380 0x7fa654004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:14.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.377+0000 7fa675a00640 1 -- 172.21.15.46:0/743568719 reap_dead start 2024-03-29T08:30:14.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.377+0000 7fa675a00640 1 --2- 172.21.15.46:0/743568719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6540050f0 0x7fa65400b990 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fa66800a830 tx=0x7fa66801d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d1fe413bd06e5f server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:14.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.377+0000 7fa66fe00640 1 -- 172.21.15.46:0/743568719 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6680ad630 con 0x7fa6540050f0 2024-03-29T08:30:14.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.377+0000 7fa66fe00640 1 -- 172.21.15.46:0/743568719 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6680a8070 con 0x7fa6540050f0 2024-03-29T08:30:14.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.377+0000 7fa66fe00640 1 -- 172.21.15.46:0/743568719 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6680ad9f0 con 0x7fa6540050f0 2024-03-29T08:30:14.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.377+0000 7fa677a00640 1 -- 172.21.15.46:0/743568719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6540050f0 msgr2=0x7fa65400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.377+0000 7fa677a00640 1 --2- 172.21.15.46:0/743568719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6540050f0 0x7fa65400b990 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fa66800a830 tx=0x7fa66801d360 comp rx=0 tx=0).stop 2024-03-29T08:30:14.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.377+0000 7fa677a00640 1 -- 172.21.15.46:0/743568719 shutdown_connections 2024-03-29T08:30:14.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.377+0000 7fa677a00640 1 --2- 172.21.15.46:0/743568719 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6540050f0 0x7fa65400b990 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.377+0000 7fa677a00640 1 -- 172.21.15.46:0/743568719 >> 172.21.15.46:0/743568719 conn(0x7fa6700b2190 msgr2=0x7fa6700b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:14.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.377+0000 7fa677a00640 1 -- 172.21.15.46:0/743568719 shutdown_connections 2024-03-29T08:30:14.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.377+0000 7fa677a00640 1 -- 172.21.15.46:0/743568719 wait complete. 2024-03-29T08:30:14.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa677a00640 1 Processor -- start 2024-03-29T08:30:14.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa677a00640 1 -- start start 2024-03-29T08:30:14.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa677a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6540050f0 0x7fa670151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa677a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa670152250 0x7fa6700c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa677a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa670152780 0x7fa6700c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa677a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700ca620 con 0x7fa670152250 2024-03-29T08:30:14.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa677a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700c7910 con 0x7fa6540050f0 2024-03-29T08:30:14.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa677a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700ca060 con 0x7fa670152780 2024-03-29T08:30:14.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa667e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa670152250 0x7fa6700c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa667e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa670152250 0x7fa6700c3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49038/0 (socket says 172.21.15.46:49038) 2024-03-29T08:30:14.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa667e00640 1 -- 172.21.15.46:0/2539366310 learned_addr learned my addr 172.21.15.46:0/2539366310 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:14.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa667e00640 1 -- 172.21.15.46:0/2539366310 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa670152780 msgr2=0x7fa6700c41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa667e00640 1 --2- 172.21.15.46:0/2539366310 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa670152780 0x7fa6700c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa667e00640 1 -- 172.21.15.46:0/2539366310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6540050f0 msgr2=0x7fa670151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa667e00640 1 --2- 172.21.15.46:0/2539366310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6540050f0 0x7fa670151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa667e00640 1 -- 172.21.15.46:0/2539366310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700ca340 con 0x7fa670152250 2024-03-29T08:30:14.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa667e00640 1 --2- 172.21.15.46:0/2539366310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa670152250 0x7fa6700c3c90 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fa65800e5f0 tx=0x7fa65800eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:14.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa66d600640 1 -- 172.21.15.46:0/2539366310 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6580052e0 con 0x7fa670152250 2024-03-29T08:30:14.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa66d600640 1 -- 172.21.15.46:0/2539366310 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa658005440 con 0x7fa670152250 2024-03-29T08:30:14.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa677a00640 1 -- 172.21.15.46:0/2539366310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6700c7bb0 con 0x7fa670152250 2024-03-29T08:30:14.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa66d600640 1 -- 172.21.15.46:0/2539366310 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa658003820 con 0x7fa670152250 2024-03-29T08:30:14.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa677a00640 1 -- 172.21.15.46:0/2539366310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa670151f50 con 0x7fa670152250 2024-03-29T08:30:14.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.381+0000 7fa677a00640 1 -- 172.21.15.46:0/2539366310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6700c7750 con 0x7fa670152250 2024-03-29T08:30:14.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.385+0000 7fa66d600640 1 -- 172.21.15.46:0/2539366310 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa658003980 con 0x7fa670152250 2024-03-29T08:30:14.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.385+0000 7fa66d600640 1 --2- 172.21.15.46:0/2539366310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa64c066690 0x7fa64c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.385+0000 7fa66d600640 1 -- 172.21.15.46:0/2539366310 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fa658084130 con 0x7fa670152250 2024-03-29T08:30:14.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.385+0000 7fa675000640 1 --2- 172.21.15.46:0/2539366310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa64c066690 0x7fa64c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.385+0000 7fa675000640 1 --2- 172.21.15.46:0/2539366310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa64c066690 0x7fa64c068b50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa67009ddd0 tx=0x7fa660009340 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:14.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.389+0000 7fa66d600640 1 -- 172.21.15.46:0/2539366310 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fa6580513d0 con 0x7fa670152250 2024-03-29T08:30:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.549+0000 7fa677a00640 1 -- 172.21.15.46:0/2539366310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa67009e690 con 0x7fa670152250 2024-03-29T08:30:14.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.549+0000 7fa66d600640 1 -- 172.21.15.46:0/2539366310 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa6580549f0 con 0x7fa670152250 2024-03-29T08:30:14.549 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-03-29T08:30:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.549+0000 7fa677a00640 1 -- 172.21.15.46:0/2539366310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa64c066690 msgr2=0x7fa64c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.549+0000 7fa677a00640 1 --2- 172.21.15.46:0/2539366310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa64c066690 0x7fa64c068b50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa67009ddd0 tx=0x7fa660009340 comp rx=0 tx=0).stop 2024-03-29T08:30:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.549+0000 7fa677a00640 1 -- 172.21.15.46:0/2539366310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa670152250 msgr2=0x7fa6700c3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.549+0000 7fa677a00640 1 --2- 172.21.15.46:0/2539366310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa670152250 0x7fa6700c3c90 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fa65800e5f0 tx=0x7fa65800eab0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.549+0000 7fa677a00640 1 -- 172.21.15.46:0/2539366310 shutdown_connections 2024-03-29T08:30:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.549+0000 7fa677a00640 1 --2- 172.21.15.46:0/2539366310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa64c066690 0x7fa64c068b50 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.549+0000 7fa677a00640 1 --2- 172.21.15.46:0/2539366310 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa670152780 0x7fa6700c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.549+0000 7fa677a00640 1 --2- 172.21.15.46:0/2539366310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa670152250 0x7fa6700c3c90 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.549+0000 7fa677a00640 1 --2- 172.21.15.46:0/2539366310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6540050f0 0x7fa670151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.549+0000 7fa677a00640 1 -- 172.21.15.46:0/2539366310 >> 172.21.15.46:0/2539366310 conn(0x7fa6700b2190 msgr2=0x7fa6700b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.549+0000 7fa677a00640 1 -- 172.21.15.46:0/2539366310 shutdown_connections 2024-03-29T08:30:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.549+0000 7fa677a00640 1 -- 172.21.15.46:0/2539366310 wait complete. 2024-03-29T08:30:14.570 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-29T08:30:14.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a0c800640 1 Processor -- start 2024-03-29T08:30:14.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a0c800640 1 -- start start 2024-03-29T08:30:14.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a0c800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a04143eb0 0x7f2a040cb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a0c800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2a0415efe0 0x7f2a040cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a0c800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2a040b1630 0x7f2a040b1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a0c800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a040b2360 con 0x7f2a0415efe0 2024-03-29T08:30:14.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a0c800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a0415e500 con 0x7f2a040b1630 2024-03-29T08:30:14.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a0c800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0415e7e0 con 0x7f2a04143eb0 2024-03-29T08:30:14.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a09400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2a0415efe0 0x7f2a040cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a09400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2a0415efe0 0x7f2a040cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49042/0 (socket says 172.21.15.46:49042) 2024-03-29T08:30:14.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a09400640 1 -- 172.21.15.46:0/1843042701 learned_addr learned my addr 172.21.15.46:0/1843042701 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:14.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a09e00640 1 --2- 172.21.15.46:0/1843042701 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a04143eb0 0x7f2a040cb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a0a800640 1 --2- 172.21.15.46:0/1843042701 >> v2:172.21.15.73:3300/0 conn(0x7f2a040b1630 0x7f2a040b1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a09400640 1 -- 172.21.15.46:0/1843042701 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a04143eb0 msgr2=0x7f2a040cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a09400640 1 --2- 172.21.15.46:0/1843042701 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a04143eb0 0x7f2a040cb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a09400640 1 -- 172.21.15.46:0/1843042701 >> v2:172.21.15.73:3300/0 conn(0x7f2a040b1630 msgr2=0x7f2a040b1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a09400640 1 --2- 172.21.15.46:0/1843042701 >> v2:172.21.15.73:3300/0 conn(0x7f2a040b1630 0x7f2a040b1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.713+0000 7f2a09400640 1 -- 172.21.15.46:0/1843042701 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0415eac0 con 0x7f2a0415efe0 2024-03-29T08:30:14.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.717+0000 7f2a09400640 1 --2- 172.21.15.46:0/1843042701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0415efe0 0x7f2a040cbcd0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f2a0415f440 tx=0x7f29f401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f246896820c553 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:14.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.717+0000 7f2a08a00640 1 -- 172.21.15.46:0/1843042701 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40ab620 con 0x7f2a0415efe0 2024-03-29T08:30:14.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.717+0000 7f2a08a00640 1 -- 172.21.15.46:0/1843042701 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f40ab780 con 0x7f2a0415efe0 2024-03-29T08:30:14.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.717+0000 7f2a08a00640 1 -- 172.21.15.46:0/1843042701 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40abb10 con 0x7f2a0415efe0 2024-03-29T08:30:14.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.717+0000 7f2a0c800640 1 -- 172.21.15.46:0/1843042701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0415efe0 msgr2=0x7f2a040cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.717+0000 7f2a0c800640 1 --2- 172.21.15.46:0/1843042701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0415efe0 0x7f2a040cbcd0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f2a0415f440 tx=0x7f29f401c520 comp rx=0 tx=0).stop 2024-03-29T08:30:14.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.717+0000 7f2a0c800640 1 -- 172.21.15.46:0/1843042701 shutdown_connections 2024-03-29T08:30:14.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.717+0000 7f2a0c800640 1 --2- 172.21.15.46:0/1843042701 >> v2:172.21.15.73:3300/0 conn(0x7f2a040b1630 0x7f2a040b1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.717+0000 7f2a0c800640 1 --2- 172.21.15.46:0/1843042701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a0415efe0 0x7f2a040cbcd0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.717+0000 7f2a0c800640 1 --2- 172.21.15.46:0/1843042701 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a04143eb0 0x7f2a040cb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.717+0000 7f2a0c800640 1 -- 172.21.15.46:0/1843042701 >> 172.21.15.46:0/1843042701 conn(0x7f2a0406a390 msgr2=0x7f2a040cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:14.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.717+0000 7f2a0c800640 1 -- 172.21.15.46:0/1843042701 shutdown_connections 2024-03-29T08:30:14.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.717+0000 7f2a0c800640 1 -- 172.21.15.46:0/1843042701 wait complete. 2024-03-29T08:30:14.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.717+0000 7f2a0c800640 1 Processor -- start 2024-03-29T08:30:14.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.717+0000 7f2a0c800640 1 -- start start 2024-03-29T08:30:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a0c800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a040b1630 0x7f2a04137f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a0c800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a04143eb0 0x7f2a0413f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a0c800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a0415efe0 0x7f2a0413fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a0c800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0413b600 con 0x7f2a04143eb0 2024-03-29T08:30:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a0c800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0413bac0 con 0x7f2a0415efe0 2024-03-29T08:30:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a0c800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0413ff40 con 0x7f2a040b1630 2024-03-29T08:30:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a09400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a04143eb0 0x7f2a0413f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a09400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a04143eb0 0x7f2a0413f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49046/0 (socket says 172.21.15.46:49046) 2024-03-29T08:30:14.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a09400640 1 -- 172.21.15.46:0/852762857 learned_addr learned my addr 172.21.15.46:0/852762857 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a0a800640 1 --2- 172.21.15.46:0/852762857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a0415efe0 0x7f2a0413fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a09e00640 1 --2- 172.21.15.46:0/852762857 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a040b1630 0x7f2a04137f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a09400640 1 -- 172.21.15.46:0/852762857 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a040b1630 msgr2=0x7f2a04137f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a09400640 1 --2- 172.21.15.46:0/852762857 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a040b1630 0x7f2a04137f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a09400640 1 -- 172.21.15.46:0/852762857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a0415efe0 msgr2=0x7f2a0413fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a09400640 1 --2- 172.21.15.46:0/852762857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a0415efe0 0x7f2a0413fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a09400640 1 -- 172.21.15.46:0/852762857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f40094d0 con 0x7f2a04143eb0 2024-03-29T08:30:14.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a09400640 1 --2- 172.21.15.46:0/852762857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a04143eb0 0x7f2a0413f4c0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f29f40ad040 tx=0x7f29f40192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f29fe000640 1 -- 172.21.15.46:0/852762857 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40abdc0 con 0x7f2a04143eb0 2024-03-29T08:30:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f29fe000640 1 -- 172.21.15.46:0/852762857 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29f40bf410 con 0x7f2a04143eb0 2024-03-29T08:30:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f29fe000640 1 -- 172.21.15.46:0/852762857 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29f40bf7c0 con 0x7f2a04143eb0 2024-03-29T08:30:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a0c800640 1 -- 172.21.15.46:0/852762857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0415a140 con 0x7f2a04143eb0 2024-03-29T08:30:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a0c800640 1 -- 172.21.15.46:0/852762857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a04140190 con 0x7f2a04143eb0 2024-03-29T08:30:14.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f2a0c800640 1 -- 172.21.15.46:0/852762857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a04071a70 con 0x7f2a04143eb0 2024-03-29T08:30:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.721+0000 7f29fe000640 1 -- 172.21.15.46:0/852762857 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f29f40bf920 con 0x7f2a04143eb0 2024-03-29T08:30:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.725+0000 7f29fe000640 1 --2- 172.21.15.46:0/852762857 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f29e8066690 0x7f29e8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:14.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.725+0000 7f29fe000640 1 -- 172.21.15.46:0/852762857 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f29f411e7a0 con 0x7f2a04143eb0 2024-03-29T08:30:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.725+0000 7f2a09e00640 1 --2- 172.21.15.46:0/852762857 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f29e8066690 0x7f29e8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:14.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.725+0000 7f2a09e00640 1 --2- 172.21.15.46:0/852762857 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f29e8066690 0x7f29e8068b50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2a0413ed20 tx=0x7f29f8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:14.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.729+0000 7f29fe000640 1 -- 172.21.15.46:0/852762857 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f29f40eb980 con 0x7f2a04143eb0 2024-03-29T08:30:14.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.889+0000 7f2a0c800640 1 -- 172.21.15.46:0/852762857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2a0404e120 con 0x7f2a04143eb0 2024-03-29T08:30:14.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.889+0000 7f29fe000640 1 -- 172.21.15.46:0/852762857 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7f29f40eefa0 con 0x7f2a04143eb0 2024-03-29T08:30:14.891 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-03-29T08:30:14.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.889+0000 7f2a0c800640 1 -- 172.21.15.46:0/852762857 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f29e8066690 msgr2=0x7f29e8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.889+0000 7f2a0c800640 1 --2- 172.21.15.46:0/852762857 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f29e8066690 0x7f29e8068b50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2a0413ed20 tx=0x7f29f8008040 comp rx=0 tx=0).stop 2024-03-29T08:30:14.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.889+0000 7f2a0c800640 1 -- 172.21.15.46:0/852762857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a04143eb0 msgr2=0x7f2a0413f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:14.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.889+0000 7f2a0c800640 1 --2- 172.21.15.46:0/852762857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a04143eb0 0x7f2a0413f4c0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f29f40ad040 tx=0x7f29f40192c0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.889+0000 7f2a0c800640 1 -- 172.21.15.46:0/852762857 shutdown_connections 2024-03-29T08:30:14.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.889+0000 7f2a0c800640 1 --2- 172.21.15.46:0/852762857 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f29e8066690 0x7f29e8068b50 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.889+0000 7f2a0c800640 1 --2- 172.21.15.46:0/852762857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a0415efe0 0x7f2a0413fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.889+0000 7f2a0c800640 1 --2- 172.21.15.46:0/852762857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a04143eb0 0x7f2a0413f4c0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.889+0000 7f2a0c800640 1 --2- 172.21.15.46:0/852762857 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a040b1630 0x7f2a04137f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:14.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.889+0000 7f2a0c800640 1 -- 172.21.15.46:0/852762857 >> 172.21.15.46:0/852762857 conn(0x7f2a0406a390 msgr2=0x7f2a040cddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:14.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.889+0000 7f2a0c800640 1 -- 172.21.15.46:0/852762857 shutdown_connections 2024-03-29T08:30:14.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:14.889+0000 7f2a0c800640 1 -- 172.21.15.46:0/852762857 wait complete. 2024-03-29T08:30:14.914 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:30:15.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.057+0000 7f314f800640 1 Processor -- start 2024-03-29T08:30:15.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.057+0000 7f314f800640 1 -- start start 2024-03-29T08:30:15.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.057+0000 7f314f800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f314815feb0 0x7f3148071370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:15.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.057+0000 7f314f800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f31481605f0 0x7f3148069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:15.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.057+0000 7f314f800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3148161020 con 0x7f3148160a80 2024-03-29T08:30:15.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.057+0000 7f314f800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3148161300 con 0x7f314815feb0 2024-03-29T08:30:15.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.057+0000 7f314f800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3148070b70 con 0x7f31481605f0 2024-03-29T08:30:15.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.057+0000 7f314ce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f314815feb0 0x7f3148071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:15.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.061+0000 7f314ce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f314815feb0 0x7f3148071370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49056/0 (socket says 172.21.15.46:49056) 2024-03-29T08:30:15.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.061+0000 7f314ce00640 1 -- 172.21.15.46:0/1582886574 learned_addr learned my addr 172.21.15.46:0/1582886574 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:15.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.061+0000 7f3147e00640 1 --2- 172.21.15.46:0/1582886574 >> v2:172.21.15.73:3300/0 conn(0x7f31481605f0 0x7f3148069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:15.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.061+0000 7f314ce00640 1 -- 172.21.15.46:0/1582886574 >> v2:172.21.15.73:3300/0 conn(0x7f31481605f0 msgr2=0x7f3148069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:15.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.061+0000 7f314ce00640 1 --2- 172.21.15.46:0/1582886574 >> v2:172.21.15.73:3300/0 conn(0x7f31481605f0 0x7f3148069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.061+0000 7f314ce00640 1 -- 172.21.15.46:0/1582886574 >> v1:172.21.15.73:6789/0 conn(0x7f3148160a80 legacy=0x7f314806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:15.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.061+0000 7f314ce00640 1 -- 172.21.15.46:0/1582886574 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3148070e50 con 0x7f314815feb0 2024-03-29T08:30:15.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.061+0000 7f314ce00640 1 --2- 172.21.15.46:0/1582886574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314815feb0 0x7f3148071370 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f313c009870 tx=0x7f313c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7603c30b09e6c37a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:15.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.061+0000 7f3147400640 1 -- 172.21.15.46:0/1582886574 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c0ac600 con 0x7f314815feb0 2024-03-29T08:30:15.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.061+0000 7f3147400640 1 -- 172.21.15.46:0/1582886574 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c0a8050 con 0x7f314815feb0 2024-03-29T08:30:15.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.061+0000 7f3147400640 1 -- 172.21.15.46:0/1582886574 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c0ac980 con 0x7f314815feb0 2024-03-29T08:30:15.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.065+0000 7f314f800640 1 -- 172.21.15.46:0/1582886574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314815feb0 msgr2=0x7f3148071370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:15.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.065+0000 7f314f800640 1 --2- 172.21.15.46:0/1582886574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314815feb0 0x7f3148071370 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f313c009870 tx=0x7f313c01c490 comp rx=0 tx=0).stop 2024-03-29T08:30:15.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.065+0000 7f314f800640 1 -- 172.21.15.46:0/1582886574 shutdown_connections 2024-03-29T08:30:15.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.065+0000 7f314f800640 1 --2- 172.21.15.46:0/1582886574 >> v2:172.21.15.73:3300/0 conn(0x7f31481605f0 0x7f3148069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.065+0000 7f314f800640 1 --2- 172.21.15.46:0/1582886574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f314815feb0 0x7f3148071370 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.065+0000 7f314f800640 1 -- 172.21.15.46:0/1582886574 >> 172.21.15.46:0/1582886574 conn(0x7f31480ced50 msgr2=0x7f31480cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:15.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.065+0000 7f314f800640 1 -- 172.21.15.46:0/1582886574 shutdown_connections 2024-03-29T08:30:15.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.065+0000 7f314f800640 1 -- 172.21.15.46:0/1582886574 wait complete. 2024-03-29T08:30:15.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.065+0000 7f314f800640 1 Processor -- start 2024-03-29T08:30:15.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.065+0000 7f314f800640 1 -- start start 2024-03-29T08:30:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.065+0000 7f314f800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f314815feb0 0x7f3148145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.065+0000 7f314f800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31481605f0 0x7f31481574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.065+0000 7f314f800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3148160a80 0x7f31480afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.065+0000 7f314f800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31481588f0 con 0x7f3148160a80 2024-03-29T08:30:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.065+0000 7f314f800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3148157a10 con 0x7f314815feb0 2024-03-29T08:30:15.068 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.065+0000 7f314f800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3148157c70 con 0x7f31481605f0 2024-03-29T08:30:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.069+0000 7f314d800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3148160a80 0x7f31480afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.069+0000 7f3147e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31481605f0 0x7f31481574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.069+0000 7f3147e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31481605f0 0x7f31481574d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:36378/0 (socket says 172.21.15.46:36378) 2024-03-29T08:30:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.069+0000 7f3147e00640 1 -- 172.21.15.46:0/358603558 learned_addr learned my addr 172.21.15.46:0/358603558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.069+0000 7f314ce00640 1 --2- 172.21.15.46:0/358603558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f314815feb0 0x7f3148145f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.069+0000 7f3147e00640 1 -- 172.21.15.46:0/358603558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f314815feb0 msgr2=0x7f3148145f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.069+0000 7f3147e00640 1 --2- 172.21.15.46:0/358603558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f314815feb0 0x7f3148145f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.069+0000 7f3147e00640 1 -- 172.21.15.46:0/358603558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3148160a80 msgr2=0x7f31480afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:15.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.069+0000 7f3147e00640 1 --2- 172.21.15.46:0/358603558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3148160a80 0x7f31480afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.069+0000 7f3147e00640 1 -- 172.21.15.46:0/358603558 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313c0094d0 con 0x7f31481605f0 2024-03-29T08:30:15.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.069+0000 7f3147e00640 1 --2- 172.21.15.46:0/358603558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31481605f0 0x7f31481574d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f313400d800 tx=0x7f313400dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:15.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.073+0000 7f3144c00640 1 -- 172.21.15.46:0/358603558 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313400b6c0 con 0x7f31481605f0 2024-03-29T08:30:15.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.073+0000 7f3144c00640 1 -- 172.21.15.46:0/358603558 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313400b820 con 0x7f31481605f0 2024-03-29T08:30:15.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.073+0000 7f3144c00640 1 -- 172.21.15.46:0/358603558 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313400bb90 con 0x7f31481605f0 2024-03-29T08:30:15.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.073+0000 7f314f800640 1 -- 172.21.15.46:0/358603558 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3148157f50 con 0x7f31481605f0 2024-03-29T08:30:15.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.073+0000 7f314f800640 1 -- 172.21.15.46:0/358603558 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3148158510 con 0x7f31481605f0 2024-03-29T08:30:15.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.073+0000 7f314f800640 1 -- 172.21.15.46:0/358603558 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3148066c90 con 0x7f31481605f0 2024-03-29T08:30:15.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.073+0000 7f3144c00640 1 -- 172.21.15.46:0/358603558 <== mon.2 v2:172.21.15.73:3301/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f313400bcf0 con 0x7f31481605f0 2024-03-29T08:30:15.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.073+0000 7f3144c00640 1 --2- 172.21.15.46:0/358603558 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3128066690 0x7f3128068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.073+0000 7f3144c00640 1 -- 172.21.15.46:0/358603558 <== mon.2 v2:172.21.15.73:3301/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f31340846b0 con 0x7f31481605f0 2024-03-29T08:30:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.073+0000 7f314ce00640 1 --2- 172.21.15.46:0/358603558 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3128066690 0x7f3128068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:15.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.077+0000 7f314ce00640 1 --2- 172.21.15.46:0/358603558 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3128066690 0x7f3128068b50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f313c0ae000 tx=0x7f313c0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:15.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.077+0000 7f3144c00640 1 -- 172.21.15.46:0/358603558 <== mon.2 v2:172.21.15.73:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f31340519d0 con 0x7f31481605f0 2024-03-29T08:30:15.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.281+0000 7f314f800640 1 -- 172.21.15.46:0/358603558 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31480cb540 con 0x7f31481605f0 2024-03-29T08:30:15.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.281+0000 7f3144c00640 1 -- 172.21.15.46:0/358603558 <== mon.2 v2:172.21.15.73:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4741 (secure 0 0 0) 0x7f3134054ff0 con 0x7f31481605f0 2024-03-29T08:30:15.282 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:15.282 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":107,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2506132630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2506132630},{"type":"v1","addr":"172.21.15.73:6836","nonce":2506132630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2108115501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2108115501},{"type":"v1","addr":"172.21.15.73:6837","nonce":2108115501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":9045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3210062895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3210062895},{"type":"v1","addr":"172.21.15.46:6833","nonce":3210062895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:29:14.470858+0000","modified":"2024-03-29T08:30:05.103674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9660":{"gid":9660,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/3867471367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3867471367},{"type":"v1","addr":"172.21.15.46:6835","nonce":3867471367}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-29T08:30:15.283 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 107 2024-03-29T08:30:15.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.285+0000 7f314f800640 1 -- 172.21.15.46:0/358603558 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3128066690 msgr2=0x7f3128068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:15.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.285+0000 7f314f800640 1 --2- 172.21.15.46:0/358603558 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3128066690 0x7f3128068b50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f313c0ae000 tx=0x7f313c0a7000 comp rx=0 tx=0).stop 2024-03-29T08:30:15.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.285+0000 7f314f800640 1 -- 172.21.15.46:0/358603558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31481605f0 msgr2=0x7f31481574d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:15.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.285+0000 7f314f800640 1 --2- 172.21.15.46:0/358603558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31481605f0 0x7f31481574d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f313400d800 tx=0x7f313400dcc0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.285+0000 7f314f800640 1 -- 172.21.15.46:0/358603558 shutdown_connections 2024-03-29T08:30:15.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.285+0000 7f314f800640 1 --2- 172.21.15.46:0/358603558 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f3128066690 0x7f3128068b50 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.285+0000 7f314f800640 1 --2- 172.21.15.46:0/358603558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3148160a80 0x7f31480afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.285+0000 7f314f800640 1 --2- 172.21.15.46:0/358603558 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31481605f0 0x7f31481574d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.285+0000 7f314f800640 1 --2- 172.21.15.46:0/358603558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f314815feb0 0x7f3148145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.285+0000 7f314f800640 1 -- 172.21.15.46:0/358603558 >> 172.21.15.46:0/358603558 conn(0x7f31480ced50 msgr2=0x7f31480ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:15.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.285+0000 7f314f800640 1 -- 172.21.15.46:0/358603558 shutdown_connections 2024-03-29T08:30:15.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.285+0000 7f314f800640 1 -- 172.21.15.46:0/358603558 wait complete. 2024-03-29T08:30:15.303 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:30:15.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.441+0000 7f05e6000640 1 Processor -- start 2024-03-29T08:30:15.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.445+0000 7f05e6000640 1 -- start start 2024-03-29T08:30:15.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.445+0000 7f05e6000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05e0144210 0x7f05e00cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:15.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.445+0000 7f05e6000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f05e00ceb20 0x7f05e006c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:15.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.445+0000 7f05e6000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e00cb510 con 0x7f05e015ada0 2024-03-29T08:30:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.445+0000 7f05e6000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e0070580 con 0x7f05e00ceb20 2024-03-29T08:30:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.445+0000 7f05e6000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e0070860 con 0x7f05e0144210 2024-03-29T08:30:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.445+0000 7f05dfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f05e00ceb20 0x7f05e006c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.445+0000 7f05dfe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f05e00ceb20 0x7f05e006c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49072/0 (socket says 172.21.15.46:49072) 2024-03-29T08:30:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.445+0000 7f05dfe00640 1 -- 172.21.15.46:0/2919185020 learned_addr learned my addr 172.21.15.46:0/2919185020 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.445+0000 7f05df400640 1 --2- 172.21.15.46:0/2919185020 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05e0144210 0x7f05e00cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.445+0000 7f05dfe00640 1 -- 172.21.15.46:0/2919185020 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05e0144210 msgr2=0x7f05e00cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.445+0000 7f05dfe00640 1 --2- 172.21.15.46:0/2919185020 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05e0144210 0x7f05e00cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.445+0000 7f05dfe00640 1 -- 172.21.15.46:0/2919185020 >> v1:172.21.15.73:6789/0 conn(0x7f05e015ada0 legacy=0x7f05e00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:15.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.445+0000 7f05dfe00640 1 -- 172.21.15.46:0/2919185020 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e0070b40 con 0x7f05e00ceb20 2024-03-29T08:30:15.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.449+0000 7f05dfe00640 1 --2- 172.21.15.46:0/2919185020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e00ceb20 0x7f05e006c950 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009870 tx=0x7f05d801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0eafe530415535e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:15.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.449+0000 7f05dea00640 1 -- 172.21.15.46:0/2919185020 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d80ab670 con 0x7f05e00ceb20 2024-03-29T08:30:15.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.449+0000 7f05dea00640 1 -- 172.21.15.46:0/2919185020 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d80ab7d0 con 0x7f05e00ceb20 2024-03-29T08:30:15.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.449+0000 7f05dea00640 1 -- 172.21.15.46:0/2919185020 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d80abb60 con 0x7f05e00ceb20 2024-03-29T08:30:15.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.449+0000 7f05e6000640 1 -- 172.21.15.46:0/2919185020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e00ceb20 msgr2=0x7f05e006c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:15.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.449+0000 7f05e6000640 1 --2- 172.21.15.46:0/2919185020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e00ceb20 0x7f05e006c950 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009870 tx=0x7f05d801c560 comp rx=0 tx=0).stop 2024-03-29T08:30:15.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.449+0000 7f05e6000640 1 -- 172.21.15.46:0/2919185020 shutdown_connections 2024-03-29T08:30:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.449+0000 7f05e6000640 1 --2- 172.21.15.46:0/2919185020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e00ceb20 0x7f05e006c950 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.449+0000 7f05e6000640 1 --2- 172.21.15.46:0/2919185020 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05e0144210 0x7f05e00cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.449+0000 7f05e6000640 1 -- 172.21.15.46:0/2919185020 >> 172.21.15.46:0/2919185020 conn(0x7f05e0066110 msgr2=0x7f05e0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.449+0000 7f05e6000640 1 -- 172.21.15.46:0/2919185020 shutdown_connections 2024-03-29T08:30:15.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.449+0000 7f05e6000640 1 -- 172.21.15.46:0/2919185020 wait complete. 2024-03-29T08:30:15.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.449+0000 7f05e6000640 1 Processor -- start 2024-03-29T08:30:15.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.449+0000 7f05e6000640 1 -- start start 2024-03-29T08:30:15.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05e6000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e00ceb20 0x7f05e01f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:15.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05e6000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05e0144210 0x7f05e01f5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:15.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05e6000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e015ada0 0x7f05e01f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:15.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05e6000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e01f82a0 con 0x7f05e00ceb20 2024-03-29T08:30:15.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05e6000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e0160640 con 0x7f05e015ada0 2024-03-29T08:30:15.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05e6000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e015d4e0 con 0x7f05e0144210 2024-03-29T08:30:15.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05df400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e00ceb20 0x7f05e01f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:15.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05df400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e00ceb20 0x7f05e01f6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49076/0 (socket says 172.21.15.46:49076) 2024-03-29T08:30:15.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05df400640 1 -- 172.21.15.46:0/2684214293 learned_addr learned my addr 172.21.15.46:0/2684214293 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:15.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05d6a00640 1 --2- 172.21.15.46:0/2684214293 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05e0144210 0x7f05e01f5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:15.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05dfe00640 1 --2- 172.21.15.46:0/2684214293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e015ada0 0x7f05e01f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:15.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05df400640 1 -- 172.21.15.46:0/2684214293 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05e0144210 msgr2=0x7f05e01f5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:15.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05df400640 1 --2- 172.21.15.46:0/2684214293 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05e0144210 0x7f05e01f5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05df400640 1 -- 172.21.15.46:0/2684214293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e015ada0 msgr2=0x7f05e01f7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:15.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05df400640 1 --2- 172.21.15.46:0/2684214293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e015ada0 0x7f05e01f7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05df400640 1 -- 172.21.15.46:0/2684214293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d80094d0 con 0x7f05e00ceb20 2024-03-29T08:30:15.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05df400640 1 --2- 172.21.15.46:0/2684214293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e00ceb20 0x7f05e01f6080 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f05d0007ef0 tx=0x7f05d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:15.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05d7e00640 1 -- 172.21.15.46:0/2684214293 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0014620 con 0x7f05e00ceb20 2024-03-29T08:30:15.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05d7e00640 1 -- 172.21.15.46:0/2684214293 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d0014780 con 0x7f05e00ceb20 2024-03-29T08:30:15.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05d7e00640 1 -- 172.21.15.46:0/2684214293 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0014b30 con 0x7f05e00ceb20 2024-03-29T08:30:15.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05e6000640 1 -- 172.21.15.46:0/2684214293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e015d7c0 con 0x7f05e00ceb20 2024-03-29T08:30:15.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05e6000640 1 -- 172.21.15.46:0/2684214293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e0160920 con 0x7f05e00ceb20 2024-03-29T08:30:15.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05e6000640 1 -- 172.21.15.46:0/2684214293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e00ce280 con 0x7f05e00ceb20 2024-03-29T08:30:15.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.453+0000 7f05d7e00640 1 -- 172.21.15.46:0/2684214293 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f05d0014c90 con 0x7f05e00ceb20 2024-03-29T08:30:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.457+0000 7f05d7e00640 1 --2- 172.21.15.46:0/2684214293 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f05bc066690 0x7f05bc068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.457+0000 7f05d7e00640 1 -- 172.21.15.46:0/2684214293 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f05d0084b30 con 0x7f05e00ceb20 2024-03-29T08:30:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.457+0000 7f05d6a00640 1 --2- 172.21.15.46:0/2684214293 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f05bc066690 0x7f05bc068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:15.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.457+0000 7f05d6a00640 1 --2- 172.21.15.46:0/2684214293 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f05bc066690 0x7f05bc068b50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f05e015d3d0 tx=0x7f05c8009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:15.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.461+0000 7f05d7e00640 1 -- 172.21.15.46:0/2684214293 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f05d0051e50 con 0x7f05e00ceb20 2024-03-29T08:30:15.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.621+0000 7f05e6000640 1 -- 172.21.15.46:0/2684214293 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05e0066c90 con 0x7f05e00ceb20 2024-03-29T08:30:15.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.621+0000 7f05d7e00640 1 -- 172.21.15.46:0/2684214293 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16406 (secure 0 0 0) 0x7f05d007ce60 con 0x7f05e00ceb20 2024-03-29T08:30:15.623 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:15.623 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":125,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:30:00.063547+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:29:12.329121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-29T08:29:13.465939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.7","mappings":[{"from":4,"to":3}]},{"pgid":"17.4","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6834/276185207":"2024-03-30T08:30:00.063526+0000","172.21.15.46:6835/929698978":"2024-03-30T08:29:45.374806+0000","172.21.15.46:6835/276185207":"2024-03-30T08:30:00.063526+0000","172.21.15.46:6834/929698978":"2024-03-30T08:29:45.374806+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:30:15.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.625+0000 7f05e6000640 1 -- 172.21.15.46:0/2684214293 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f05bc066690 msgr2=0x7f05bc068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:15.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.625+0000 7f05e6000640 1 --2- 172.21.15.46:0/2684214293 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f05bc066690 0x7f05bc068b50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f05e015d3d0 tx=0x7f05c8009040 comp rx=0 tx=0).stop 2024-03-29T08:30:15.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.625+0000 7f05e6000640 1 -- 172.21.15.46:0/2684214293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e00ceb20 msgr2=0x7f05e01f6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:15.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.625+0000 7f05e6000640 1 --2- 172.21.15.46:0/2684214293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e00ceb20 0x7f05e01f6080 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f05d0007ef0 tx=0x7f05d000e4b0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.625+0000 7f05e6000640 1 -- 172.21.15.46:0/2684214293 shutdown_connections 2024-03-29T08:30:15.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.625+0000 7f05e6000640 1 --2- 172.21.15.46:0/2684214293 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f05bc066690 0x7f05bc068b50 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.625+0000 7f05e6000640 1 --2- 172.21.15.46:0/2684214293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e015ada0 0x7f05e01f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.625+0000 7f05e6000640 1 --2- 172.21.15.46:0/2684214293 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f05e0144210 0x7f05e01f5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.625+0000 7f05e6000640 1 --2- 172.21.15.46:0/2684214293 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05e00ceb20 0x7f05e01f6080 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.625+0000 7f05e6000640 1 -- 172.21.15.46:0/2684214293 >> 172.21.15.46:0/2684214293 conn(0x7f05e0066110 msgr2=0x7f05e00ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:15.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.625+0000 7f05e6000640 1 -- 172.21.15.46:0/2684214293 shutdown_connections 2024-03-29T08:30:15.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.625+0000 7f05e6000640 1 -- 172.21.15.46:0/2684214293 wait complete. 2024-03-29T08:30:15.644 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T08:30:15.644 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:30:15.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.785+0000 7f86c8800640 1 Processor -- start 2024-03-29T08:30:15.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.785+0000 7f86c8800640 1 -- start start 2024-03-29T08:30:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.785+0000 7f86c8800640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f86c0144210 0x7f86c00cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.785+0000 7f86c8800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c00cb510 con 0x7f86c015ada0 2024-03-29T08:30:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.785+0000 7f86c8800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c0070580 con 0x7f86c00ceb20 2024-03-29T08:30:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.785+0000 7f86c8800640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c0070860 con 0x7f86c0144210 2024-03-29T08:30:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.785+0000 7f86c5400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f86c015ada0 0x7f86c00ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42064/0 (socket says 172.21.15.46:42064) 2024-03-29T08:30:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.785+0000 7f86c5400640 1 -- 172.21.15.46:0/3778735698 learned_addr learned my addr 172.21.15.46:0/3778735698 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:15.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.785+0000 7f86c5e00640 1 --2- 172.21.15.46:0/3778735698 >> v2:172.21.15.73:3300/0 conn(0x7f86c0144210 0x7f86c00cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.785+0000 7f86c4a00640 1 -- 172.21.15.46:0/3778735698 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3867635707 0 0) 0x7f86c00cb510 con 0x7f86c015ada0 2024-03-29T08:30:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.785+0000 7f86c4a00640 1 -- 172.21.15.46:0/3778735698 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86a8003530 con 0x7f86c015ada0 2024-03-29T08:30:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.785+0000 7f86c4a00640 1 -- 172.21.15.46:0/3778735698 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206701320 0 0) 0x7f86c0070580 con 0x7f86c00ceb20 2024-03-29T08:30:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.785+0000 7f86c4a00640 1 -- 172.21.15.46:0/3778735698 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86c00cb510 con 0x7f86c00ceb20 2024-03-29T08:30:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.785+0000 7f86c5e00640 1 -- 172.21.15.46:0/3778735698 >> v1:172.21.15.73:6789/0 conn(0x7f86c00ceb20 legacy=0x7f86c006c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.785+0000 7f86c5e00640 1 -- 172.21.15.46:0/3778735698 >> v1:172.21.15.46:6789/0 conn(0x7f86c015ada0 legacy=0x7f86c00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:15.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.785+0000 7f86c5e00640 1 -- 172.21.15.46:0/3778735698 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c0070b40 con 0x7f86c0144210 2024-03-29T08:30:15.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.789+0000 7f86c5e00640 1 --2- 172.21.15.46:0/3778735698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86c0144210 0x7f86c00cb8b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f86b40097c0 tx=0x7f86b401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f130543af3165397 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:15.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.789+0000 7f86c4a00640 1 -- 172.21.15.46:0/3778735698 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b40ac410 con 0x7f86c0144210 2024-03-29T08:30:15.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.789+0000 7f86c4a00640 1 -- 172.21.15.46:0/3778735698 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b40ac570 con 0x7f86c0144210 2024-03-29T08:30:15.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.789+0000 7f86c4a00640 1 -- 172.21.15.46:0/3778735698 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b40ac900 con 0x7f86c0144210 2024-03-29T08:30:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.789+0000 7f86c8800640 1 -- 172.21.15.46:0/3778735698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86c0144210 msgr2=0x7f86c00cb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.789+0000 7f86c8800640 1 --2- 172.21.15.46:0/3778735698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86c0144210 0x7f86c00cb8b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f86b40097c0 tx=0x7f86b401c4a0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.789+0000 7f86c8800640 1 -- 172.21.15.46:0/3778735698 shutdown_connections 2024-03-29T08:30:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.789+0000 7f86c8800640 1 --2- 172.21.15.46:0/3778735698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86c0144210 0x7f86c00cb8b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.789+0000 7f86c8800640 1 -- 172.21.15.46:0/3778735698 >> 172.21.15.46:0/3778735698 conn(0x7f86c0066110 msgr2=0x7f86c0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.789+0000 7f86c8800640 1 -- 172.21.15.46:0/3778735698 shutdown_connections 2024-03-29T08:30:15.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.789+0000 7f86c8800640 1 -- 172.21.15.46:0/3778735698 wait complete. 2024-03-29T08:30:15.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c8800640 1 Processor -- start 2024-03-29T08:30:15.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c8800640 1 -- start start 2024-03-29T08:30:15.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c8800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86c00ceb20 0x7f86c01597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:15.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c8800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86c0144210 0x7f86c0159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:15.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c8800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c015ada0 0x7f86c00aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:15.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c8800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c0159640 con 0x7f86c015ada0 2024-03-29T08:30:15.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c8800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c00b3550 con 0x7f86c0144210 2024-03-29T08:30:15.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c8800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c0142de0 con 0x7f86c00ceb20 2024-03-29T08:30:15.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c015ada0 0x7f86c00aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:15.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c6800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c015ada0 0x7f86c00aebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49078/0 (socket says 172.21.15.46:49078) 2024-03-29T08:30:15.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c6800640 1 -- 172.21.15.46:0/1046852375 learned_addr learned my addr 172.21.15.46:0/1046852375 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:15.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c5400640 1 --2- 172.21.15.46:0/1046852375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86c0144210 0x7f86c0159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:15.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c5e00640 1 --2- 172.21.15.46:0/1046852375 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86c00ceb20 0x7f86c01597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:15.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c6800640 1 -- 172.21.15.46:0/1046852375 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86c00ceb20 msgr2=0x7f86c01597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:15.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c6800640 1 --2- 172.21.15.46:0/1046852375 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86c00ceb20 0x7f86c01597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c6800640 1 -- 172.21.15.46:0/1046852375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86c0144210 msgr2=0x7f86c0159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c6800640 1 --2- 172.21.15.46:0/1046852375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86c0144210 0x7f86c0159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c6800640 1 -- 172.21.15.46:0/1046852375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b4009420 con 0x7f86c015ada0 2024-03-29T08:30:15.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c6800640 1 --2- 172.21.15.46:0/1046852375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c015ada0 0x7f86c00aebb0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00c7a0 tx=0x7f86bc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86ba000640 1 -- 172.21.15.46:0/1046852375 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc003b80 con 0x7f86c015ada0 2024-03-29T08:30:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86ba000640 1 -- 172.21.15.46:0/1046852375 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86bc003ce0 con 0x7f86c015ada0 2024-03-29T08:30:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86ba000640 1 -- 172.21.15.46:0/1046852375 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc00aa90 con 0x7f86c015ada0 2024-03-29T08:30:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c8800640 1 -- 172.21.15.46:0/1046852375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c00b2f90 con 0x7f86c015ada0 2024-03-29T08:30:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c8800640 1 -- 172.21.15.46:0/1046852375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c0142a20 con 0x7f86c015ada0 2024-03-29T08:30:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.793+0000 7f86c8800640 1 -- 172.21.15.46:0/1046852375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c00ce280 con 0x7f86c015ada0 2024-03-29T08:30:15.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.797+0000 7f86ba000640 1 -- 172.21.15.46:0/1046852375 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f86bc00abf0 con 0x7f86c015ada0 2024-03-29T08:30:15.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.797+0000 7f86ba000640 1 --2- 172.21.15.46:0/1046852375 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f86a4066690 0x7f86a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:15.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.797+0000 7f86ba000640 1 -- 172.21.15.46:0/1046852375 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f86bc083ff0 con 0x7f86c015ada0 2024-03-29T08:30:15.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.797+0000 7f86c5e00640 1 --2- 172.21.15.46:0/1046852375 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f86a4066690 0x7f86a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:15.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.797+0000 7f86c5e00640 1 --2- 172.21.15.46:0/1046852375 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f86a4066690 0x7f86a4068b50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f86b40ae000 tx=0x7f86b4008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:15.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:15.801+0000 7f86ba000640 1 -- 172.21.15.46:0/1046852375 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f86bc051310 con 0x7f86c015ada0 2024-03-29T08:30:16.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.001+0000 7f86c8800640 1 -- 172.21.15.46:0/1046852375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86c0066c90 con 0x7f86c015ada0 2024-03-29T08:30:16.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.001+0000 7f86ba000640 1 -- 172.21.15.46:0/1046852375 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4741 (secure 0 0 0) 0x7f86bc054930 con 0x7f86c015ada0 2024-03-29T08:30:16.001 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:16.001 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":107,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2506132630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2506132630},{"type":"v1","addr":"172.21.15.73:6836","nonce":2506132630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2108115501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2108115501},{"type":"v1","addr":"172.21.15.73:6837","nonce":2108115501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":9045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3210062895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3210062895},{"type":"v1","addr":"172.21.15.46:6833","nonce":3210062895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:29:14.470858+0000","modified":"2024-03-29T08:30:05.103674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9660":{"gid":9660,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/3867471367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3867471367},{"type":"v1","addr":"172.21.15.46:6835","nonce":3867471367}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-29T08:30:16.001 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 107 2024-03-29T08:30:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.001+0000 7f86c8800640 1 -- 172.21.15.46:0/1046852375 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f86a4066690 msgr2=0x7f86a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.001+0000 7f86c8800640 1 --2- 172.21.15.46:0/1046852375 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f86a4066690 0x7f86a4068b50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f86b40ae000 tx=0x7f86b4008040 comp rx=0 tx=0).stop 2024-03-29T08:30:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.001+0000 7f86c8800640 1 -- 172.21.15.46:0/1046852375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c015ada0 msgr2=0x7f86c00aebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.001+0000 7f86c8800640 1 --2- 172.21.15.46:0/1046852375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c015ada0 0x7f86c00aebb0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00c7a0 tx=0x7f86bc00cc60 comp rx=0 tx=0).stop 2024-03-29T08:30:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.001+0000 7f86c8800640 1 -- 172.21.15.46:0/1046852375 shutdown_connections 2024-03-29T08:30:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.001+0000 7f86c8800640 1 --2- 172.21.15.46:0/1046852375 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f86a4066690 0x7f86a4068b50 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.001+0000 7f86c8800640 1 --2- 172.21.15.46:0/1046852375 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86c0144210 0x7f86c0159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.001+0000 7f86c8800640 1 --2- 172.21.15.46:0/1046852375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86c015ada0 0x7f86c00aebb0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.001+0000 7f86c8800640 1 --2- 172.21.15.46:0/1046852375 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f86c00ceb20 0x7f86c01597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.001+0000 7f86c8800640 1 -- 172.21.15.46:0/1046852375 >> 172.21.15.46:0/1046852375 conn(0x7f86c0066110 msgr2=0x7f86c013e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:16.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.001+0000 7f86c8800640 1 -- 172.21.15.46:0/1046852375 shutdown_connections 2024-03-29T08:30:16.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.001+0000 7f86c8800640 1 -- 172.21.15.46:0/1046852375 wait complete. 2024-03-29T08:30:16.022 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:30:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.161+0000 7f9dffc00640 1 Processor -- start 2024-03-29T08:30:16.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.161+0000 7f9dffc00640 1 -- start start 2024-03-29T08:30:16.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.165+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9df806d910 0x7f9df80701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.165+0000 7f9dffc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9df806dd70 0x7f9df8070700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.165+0000 7f9dffc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df815d7c0 con 0x7f9df81569a0 2024-03-29T08:30:16.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.165+0000 7f9dffc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df80b21d0 con 0x7f9df806dd70 2024-03-29T08:30:16.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.165+0000 7f9dffc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df815cc40 con 0x7f9df806d910 2024-03-29T08:30:16.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.165+0000 7f9dfd200640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9df81569a0 0x7f9df806c1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42070/0 (socket says 172.21.15.46:42070) 2024-03-29T08:30:16.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.165+0000 7f9dfd200640 1 -- 172.21.15.46:0/1077241746 learned_addr learned my addr 172.21.15.46:0/1077241746 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:16.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.165+0000 7f9dfdc00640 1 --2- 172.21.15.46:0/1077241746 >> v2:172.21.15.46:3300/0 conn(0x7f9df806dd70 0x7f9df8070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:16.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.165+0000 7f9defe00640 1 --2- 172.21.15.46:0/1077241746 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9df806d910 0x7f9df80701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:16.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.165+0000 7f9dfdc00640 1 -- 172.21.15.46:0/1077241746 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9df806d910 msgr2=0x7f9df80701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:16.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.165+0000 7f9dfdc00640 1 --2- 172.21.15.46:0/1077241746 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9df806d910 0x7f9df80701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.165+0000 7f9dfdc00640 1 -- 172.21.15.46:0/1077241746 >> v1:172.21.15.46:6789/0 conn(0x7f9df81569a0 legacy=0x7f9df806c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.165+0000 7f9dfdc00640 1 -- 172.21.15.46:0/1077241746 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df815cf20 con 0x7f9df806dd70 2024-03-29T08:30:16.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.165+0000 7f9defe00640 1 --2- 172.21.15.46:0/1077241746 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9df806d910 0x7f9df80701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:16.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.165+0000 7f9df7e00640 1 -- 172.21.15.46:0/1077241746 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3189830482 0 0) 0x7f9df815d7c0 con 0x7f9df81569a0 2024-03-29T08:30:16.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9dfdc00640 1 --2- 172.21.15.46:0/1077241746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df806dd70 0x7f9df8070700 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009d90 tx=0x7f9df001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f6e46cde43fd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:16.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9df7e00640 1 -- 172.21.15.46:0/1077241746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df00a57b0 con 0x7f9df806dd70 2024-03-29T08:30:16.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9df7e00640 1 -- 172.21.15.46:0/1077241746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df00a5910 con 0x7f9df806dd70 2024-03-29T08:30:16.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9df7e00640 1 -- 172.21.15.46:0/1077241746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df00a5ca0 con 0x7f9df806dd70 2024-03-29T08:30:16.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9dffc00640 1 -- 172.21.15.46:0/1077241746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df806dd70 msgr2=0x7f9df8070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9dffc00640 1 --2- 172.21.15.46:0/1077241746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df806dd70 0x7f9df8070700 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009d90 tx=0x7f9df001c970 comp rx=0 tx=0).stop 2024-03-29T08:30:16.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9dffc00640 1 -- 172.21.15.46:0/1077241746 shutdown_connections 2024-03-29T08:30:16.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9dffc00640 1 --2- 172.21.15.46:0/1077241746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df806dd70 0x7f9df8070700 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9dffc00640 1 --2- 172.21.15.46:0/1077241746 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9df806d910 0x7f9df80701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9dffc00640 1 -- 172.21.15.46:0/1077241746 >> 172.21.15.46:0/1077241746 conn(0x7f9df8066110 msgr2=0x7f9df8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:16.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9dffc00640 1 -- 172.21.15.46:0/1077241746 shutdown_connections 2024-03-29T08:30:16.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9dffc00640 1 -- 172.21.15.46:0/1077241746 wait complete. 2024-03-29T08:30:16.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9dffc00640 1 Processor -- start 2024-03-29T08:30:16.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9dffc00640 1 -- start start 2024-03-29T08:30:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9df806d910 0x7f9df815a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df806dd70 0x7f9df81511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9dffc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9df81569a0 0x7f9df813f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9dffc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df814fac0 con 0x7f9df806dd70 2024-03-29T08:30:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9dffc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df814eeb0 con 0x7f9df806d910 2024-03-29T08:30:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.169+0000 7f9dffc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df814f190 con 0x7f9df81569a0 2024-03-29T08:30:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9defe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df806dd70 0x7f9df81511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9defe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df806dd70 0x7f9df81511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49094/0 (socket says 172.21.15.46:49094) 2024-03-29T08:30:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9defe00640 1 -- 172.21.15.46:0/2794851575 learned_addr learned my addr 172.21.15.46:0/2794851575 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9dfdc00640 1 --2- 172.21.15.46:0/2794851575 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9df81569a0 0x7f9df813f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:16.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9dfd200640 1 --2- 172.21.15.46:0/2794851575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9df806d910 0x7f9df815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9defe00640 1 -- 172.21.15.46:0/2794851575 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9df81569a0 msgr2=0x7f9df813f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9defe00640 1 --2- 172.21.15.46:0/2794851575 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9df81569a0 0x7f9df813f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9defe00640 1 -- 172.21.15.46:0/2794851575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9df806d910 msgr2=0x7f9df815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9defe00640 1 --2- 172.21.15.46:0/2794851575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9df806d910 0x7f9df815a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9defe00640 1 -- 172.21.15.46:0/2794851575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df0009a70 con 0x7f9df806dd70 2024-03-29T08:30:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9dfd200640 1 --2- 172.21.15.46:0/2794851575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9df806d910 0x7f9df815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9defe00640 1 --2- 172.21.15.46:0/2794851575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df806dd70 0x7f9df81511c0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f9de000c7c0 tx=0x7f9de000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9df5600640 1 -- 172.21.15.46:0/2794851575 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de0007a30 con 0x7f9df806dd70 2024-03-29T08:30:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9df5600640 1 -- 172.21.15.46:0/2794851575 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de0007b90 con 0x7f9df806dd70 2024-03-29T08:30:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9dffc00640 1 -- 172.21.15.46:0/2794851575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df814f470 con 0x7f9df806dd70 2024-03-29T08:30:16.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9df5600640 1 -- 172.21.15.46:0/2794851575 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de000ce70 con 0x7f9df806dd70 2024-03-29T08:30:16.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9dffc00640 1 -- 172.21.15.46:0/2794851575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df814eb10 con 0x7f9df806dd70 2024-03-29T08:30:16.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9dffc00640 1 -- 172.21.15.46:0/2794851575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df8066c90 con 0x7f9df806dd70 2024-03-29T08:30:16.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.173+0000 7f9df5600640 1 -- 172.21.15.46:0/2794851575 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f9de00113f0 con 0x7f9df806dd70 2024-03-29T08:30:16.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.177+0000 7f9df5600640 1 --2- 172.21.15.46:0/2794851575 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9dd40666c0 0x7f9dd4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.177+0000 7f9df5600640 1 -- 172.21.15.46:0/2794851575 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f9de0084700 con 0x7f9df806dd70 2024-03-29T08:30:16.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.177+0000 7f9dfd200640 1 --2- 172.21.15.46:0/2794851575 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9dd40666c0 0x7f9dd4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:16.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.177+0000 7f9dfd200640 1 --2- 172.21.15.46:0/2794851575 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9dd40666c0 0x7f9dd4068b80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9de8000f70 tx=0x7f9de8007cb0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:16.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.177+0000 7f9df5600640 1 -- 172.21.15.46:0/2794851575 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f9de0051990 con 0x7f9df806dd70 2024-03-29T08:30:16.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.373+0000 7f9dffc00640 1 -- 172.21.15.46:0/2794851575 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9df80aeb20 con 0x7f9df806dd70 2024-03-29T08:30:16.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.377+0000 7f9df5600640 1 -- 172.21.15.46:0/2794851575 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4741 (secure 0 0 0) 0x7f9de0054fb0 con 0x7f9df806dd70 2024-03-29T08:30:16.376 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:16.376 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":107,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2506132630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2506132630},{"type":"v1","addr":"172.21.15.73:6836","nonce":2506132630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2108115501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2108115501},{"type":"v1","addr":"172.21.15.73:6837","nonce":2108115501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":9045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3210062895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3210062895},{"type":"v1","addr":"172.21.15.46:6833","nonce":3210062895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-29T08:29:14.470858+0000","modified":"2024-03-29T08:30:05.103674+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":125,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9660},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9660":{"gid":9660,"name":"a","rank":0,"incarnation":104,"state":"up:active","state_seq":4,"addr":"172.21.15.46:6835/3867471367","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3867471367},{"type":"v1","addr":"172.21.15.46:6835","nonce":3867471367}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-03-29T08:30:16.377 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 107 2024-03-29T08:30:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.377+0000 7f9dffc00640 1 -- 172.21.15.46:0/2794851575 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9dd40666c0 msgr2=0x7f9dd4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.377+0000 7f9dffc00640 1 --2- 172.21.15.46:0/2794851575 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9dd40666c0 0x7f9dd4068b80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9de8000f70 tx=0x7f9de8007cb0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.377+0000 7f9dffc00640 1 -- 172.21.15.46:0/2794851575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df806dd70 msgr2=0x7f9df81511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.377+0000 7f9dffc00640 1 --2- 172.21.15.46:0/2794851575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df806dd70 0x7f9df81511c0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f9de000c7c0 tx=0x7f9de000cc80 comp rx=0 tx=0).stop 2024-03-29T08:30:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.377+0000 7f9dffc00640 1 -- 172.21.15.46:0/2794851575 shutdown_connections 2024-03-29T08:30:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.377+0000 7f9dffc00640 1 --2- 172.21.15.46:0/2794851575 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9dd40666c0 0x7f9dd4068b80 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.377+0000 7f9dffc00640 1 --2- 172.21.15.46:0/2794851575 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f9df81569a0 0x7f9df813f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.377+0000 7f9dffc00640 1 --2- 172.21.15.46:0/2794851575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df806dd70 0x7f9df81511c0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.377+0000 7f9dffc00640 1 --2- 172.21.15.46:0/2794851575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9df806d910 0x7f9df815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.377+0000 7f9dffc00640 1 -- 172.21.15.46:0/2794851575 >> 172.21.15.46:0/2794851575 conn(0x7f9df8066110 msgr2=0x7f9df80ae010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.377+0000 7f9dffc00640 1 -- 172.21.15.46:0/2794851575 shutdown_connections 2024-03-29T08:30:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.377+0000 7f9dffc00640 1 -- 172.21.15.46:0/2794851575 wait complete. 2024-03-29T08:30:16.397 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:30:16.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.537+0000 7f12e8600640 1 Processor -- start 2024-03-29T08:30:16.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.537+0000 7f12e8600640 1 -- start start 2024-03-29T08:30:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.537+0000 7f12e8600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f12e015ada0 0x7f12e00ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.537+0000 7f12e8600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12e00ce950 0x7f12e006c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.537+0000 7f12e8600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e00cb510 con 0x7f12e0144210 2024-03-29T08:30:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.537+0000 7f12e8600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e00cede0 con 0x7f12e00ce950 2024-03-29T08:30:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.537+0000 7f12e8600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e0070580 con 0x7f12e015ada0 2024-03-29T08:30:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.537+0000 7f12e6600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12e00ce950 0x7f12e006c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:16.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.537+0000 7f12e5c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f12e0144210 0x7f12e00cb8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57778/0 (socket says 172.21.15.46:57778) 2024-03-29T08:30:16.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.537+0000 7f12e6600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12e00ce950 0x7f12e006c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38422/0 (socket says 172.21.15.46:38422) 2024-03-29T08:30:16.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.537+0000 7f12e5c00640 1 -- 172.21.15.46:0/4164420220 learned_addr learned my addr 172.21.15.46:0/4164420220 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:16.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.537+0000 7f12e5200640 1 --2- 172.21.15.46:0/4164420220 >> v2:172.21.15.73:3300/0 conn(0x7f12e015ada0 0x7f12e00ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:16.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.537+0000 7f12e6600640 1 -- 172.21.15.46:0/4164420220 >> v2:172.21.15.73:3300/0 conn(0x7f12e015ada0 msgr2=0x7f12e00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:16.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.537+0000 7f12e6600640 1 --2- 172.21.15.46:0/4164420220 >> v2:172.21.15.73:3300/0 conn(0x7f12e015ada0 0x7f12e00ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.537+0000 7f12e6600640 1 -- 172.21.15.46:0/4164420220 >> v1:172.21.15.46:6789/0 conn(0x7f12e0144210 legacy=0x7f12e00cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.537+0000 7f12e6600640 1 -- 172.21.15.46:0/4164420220 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e0070860 con 0x7f12e00ce950 2024-03-29T08:30:16.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.537+0000 7f12dfe00640 1 -- 172.21.15.46:0/4164420220 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4037424819 0 0) 0x7f12e00cb510 con 0x7f12e0144210 2024-03-29T08:30:16.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.541+0000 7f12e6600640 1 --2- 172.21.15.46:0/4164420220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e00ce950 0x7f12e006c950 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f12e006e410 tx=0x7f12d801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9526f13a0add102 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:16.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.541+0000 7f12dfe00640 1 -- 172.21.15.46:0/4164420220 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d80a58e0 con 0x7f12e00ce950 2024-03-29T08:30:16.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.541+0000 7f12dfe00640 1 -- 172.21.15.46:0/4164420220 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d80a5a40 con 0x7f12e00ce950 2024-03-29T08:30:16.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.541+0000 7f12dfe00640 1 -- 172.21.15.46:0/4164420220 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d80a5dd0 con 0x7f12e00ce950 2024-03-29T08:30:16.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.541+0000 7f12e8600640 1 -- 172.21.15.46:0/4164420220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e00ce950 msgr2=0x7f12e006c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.541+0000 7f12e8600640 1 --2- 172.21.15.46:0/4164420220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e00ce950 0x7f12e006c950 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f12e006e410 tx=0x7f12d801c930 comp rx=0 tx=0).stop 2024-03-29T08:30:16.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.541+0000 7f12e8600640 1 -- 172.21.15.46:0/4164420220 shutdown_connections 2024-03-29T08:30:16.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.541+0000 7f12e8600640 1 --2- 172.21.15.46:0/4164420220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e00ce950 0x7f12e006c950 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.541+0000 7f12e8600640 1 --2- 172.21.15.46:0/4164420220 >> v2:172.21.15.73:3300/0 conn(0x7f12e015ada0 0x7f12e00ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.541+0000 7f12e8600640 1 -- 172.21.15.46:0/4164420220 >> 172.21.15.46:0/4164420220 conn(0x7f12e0066110 msgr2=0x7f12e0069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:16.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.541+0000 7f12e8600640 1 -- 172.21.15.46:0/4164420220 shutdown_connections 2024-03-29T08:30:16.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.541+0000 7f12e8600640 1 -- 172.21.15.46:0/4164420220 wait complete. 2024-03-29T08:30:16.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e8600640 1 Processor -- start 2024-03-29T08:30:16.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e8600640 1 -- start start 2024-03-29T08:30:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e8600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e00ce950 0x7f12e01408d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e8600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12e0144210 0x7f12e013fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e8600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12e015ada0 0x7f12e0140020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e8600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e01608f0 con 0x7f12e00ce950 2024-03-29T08:30:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e8600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e015fd40 con 0x7f12e015ada0 2024-03-29T08:30:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e8600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e015ff50 con 0x7f12e0144210 2024-03-29T08:30:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e00ce950 0x7f12e01408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e5c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e00ce950 0x7f12e01408d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38438/0 (socket says 172.21.15.46:38438) 2024-03-29T08:30:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e5c00640 1 -- 172.21.15.46:0/93992314 learned_addr learned my addr 172.21.15.46:0/93992314 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e5c00640 1 -- 172.21.15.46:0/93992314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12e0144210 msgr2=0x7f12e013fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e5c00640 1 --2- 172.21.15.46:0/93992314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12e0144210 0x7f12e013fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e5c00640 1 -- 172.21.15.46:0/93992314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12e015ada0 msgr2=0x7f12e0140020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e5c00640 1 --2- 172.21.15.46:0/93992314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12e015ada0 0x7f12e0140020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e5c00640 1 -- 172.21.15.46:0/93992314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d8009a70 con 0x7f12e00ce950 2024-03-29T08:30:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e5c00640 1 --2- 172.21.15.46:0/93992314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e00ce950 0x7f12e01408d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f12d40011f0 tx=0x7f12d400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12dd600640 1 -- 172.21.15.46:0/93992314 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d40074e0 con 0x7f12e00ce950 2024-03-29T08:30:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12dd600640 1 -- 172.21.15.46:0/93992314 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d4007640 con 0x7f12e00ce950 2024-03-29T08:30:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e8600640 1 -- 172.21.15.46:0/93992314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e0160230 con 0x7f12e00ce950 2024-03-29T08:30:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12dd600640 1 -- 172.21.15.46:0/93992314 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d4004a10 con 0x7f12e00ce950 2024-03-29T08:30:16.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.545+0000 7f12e8600640 1 -- 172.21.15.46:0/93992314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e0160d10 con 0x7f12e00ce950 2024-03-29T08:30:16.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.549+0000 7f12e8600640 1 -- 172.21.15.46:0/93992314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e00ce280 con 0x7f12e00ce950 2024-03-29T08:30:16.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.549+0000 7f12dd600640 1 -- 172.21.15.46:0/93992314 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f12d4004b70 con 0x7f12e00ce950 2024-03-29T08:30:16.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.549+0000 7f12dd600640 1 --2- 172.21.15.46:0/93992314 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12c0066690 0x7f12c0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.549+0000 7f12dd600640 1 -- 172.21.15.46:0/93992314 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f12d40837f0 con 0x7f12e00ce950 2024-03-29T08:30:16.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.549+0000 7f12e5200640 1 --2- 172.21.15.46:0/93992314 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12c0066690 0x7f12c0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:16.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.549+0000 7f12e5200640 1 --2- 172.21.15.46:0/93992314 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12c0066690 0x7f12c0068b50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f12e00acd60 tx=0x7f12cc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:16.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.553+0000 7f12dd600640 1 -- 172.21.15.46:0/93992314 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f12d4050a60 con 0x7f12e00ce950 2024-03-29T08:30:16.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.713+0000 7f12e8600640 1 -- 172.21.15.46:0/93992314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12e0160a30 con 0x7f12e00ce950 2024-03-29T08:30:16.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.717+0000 7f12dd600640 1 -- 172.21.15.46:0/93992314 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16406 (secure 0 0 0) 0x7f12e0160a30 con 0x7f12e00ce950 2024-03-29T08:30:16.716 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:16.716 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":125,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:30:00.063547+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-03-29T08:29:12.329121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-03-29T08:29:13.465939+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.7","mappings":[{"from":4,"to":3}]},{"pgid":"17.4","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6834/276185207":"2024-03-30T08:30:00.063526+0000","172.21.15.46:6835/929698978":"2024-03-30T08:29:45.374806+0000","172.21.15.46:6835/276185207":"2024-03-30T08:30:00.063526+0000","172.21.15.46:6834/929698978":"2024-03-30T08:29:45.374806+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:30:16.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.717+0000 7f12e8600640 1 -- 172.21.15.46:0/93992314 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12c0066690 msgr2=0x7f12c0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.717+0000 7f12e8600640 1 --2- 172.21.15.46:0/93992314 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12c0066690 0x7f12c0068b50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f12e00acd60 tx=0x7f12cc008040 comp rx=0 tx=0).stop 2024-03-29T08:30:16.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.717+0000 7f12e8600640 1 -- 172.21.15.46:0/93992314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e00ce950 msgr2=0x7f12e01408d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.717+0000 7f12e8600640 1 --2- 172.21.15.46:0/93992314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e00ce950 0x7f12e01408d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f12d40011f0 tx=0x7f12d400c9b0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.717+0000 7f12e8600640 1 -- 172.21.15.46:0/93992314 shutdown_connections 2024-03-29T08:30:16.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.717+0000 7f12e8600640 1 --2- 172.21.15.46:0/93992314 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f12c0066690 0x7f12c0068b50 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.717+0000 7f12e8600640 1 --2- 172.21.15.46:0/93992314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12e015ada0 0x7f12e0140020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.717+0000 7f12e8600640 1 --2- 172.21.15.46:0/93992314 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f12e0144210 0x7f12e013fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.717+0000 7f12e8600640 1 --2- 172.21.15.46:0/93992314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12e00ce950 0x7f12e01408d0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.717+0000 7f12e8600640 1 -- 172.21.15.46:0/93992314 >> 172.21.15.46:0/93992314 conn(0x7f12e0066110 msgr2=0x7f12e01418f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:16.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.717+0000 7f12e8600640 1 -- 172.21.15.46:0/93992314 shutdown_connections 2024-03-29T08:30:16.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.717+0000 7f12e8600640 1 -- 172.21.15.46:0/93992314 wait complete. 2024-03-29T08:30:16.738 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-29T08:30:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.881+0000 7f24a6400640 1 Processor -- start 2024-03-29T08:30:16.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.881+0000 7f24a6400640 1 -- start start 2024-03-29T08:30:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.881+0000 7f24a6400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24a015a890 0x7f24a00cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.881+0000 7f24a6400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f24a00b02b0 0x7f24a00b0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.881+0000 7f24a6400640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a00aef30 con 0x7f24a0069d90 2024-03-29T08:30:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.881+0000 7f24a6400640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a00b0c80 con 0x7f24a00b02b0 2024-03-29T08:30:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.881+0000 7f24a6400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a015bc80 con 0x7f24a015a890 2024-03-29T08:30:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.881+0000 7f2497e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f24a0069d90 0x7f24a015e3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57788/0 (socket says 172.21.15.46:57788) 2024-03-29T08:30:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.881+0000 7f2497e00640 1 -- 172.21.15.46:0/1411588393 learned_addr learned my addr 172.21.15.46:0/1411588393 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:16.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.881+0000 7f24a5a00640 1 --2- 172.21.15.46:0/1411588393 >> v2:172.21.15.46:3300/0 conn(0x7f24a00b02b0 0x7f24a00b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.881+0000 7f24a5000640 1 --2- 172.21.15.46:0/1411588393 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24a015a890 0x7f24a00cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.881+0000 7f249fe00640 1 -- 172.21.15.46:0/1411588393 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631453144 0 0) 0x7f24a00aef30 con 0x7f24a0069d90 2024-03-29T08:30:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.881+0000 7f24a5a00640 1 -- 172.21.15.46:0/1411588393 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24a015a890 msgr2=0x7f24a00cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.881+0000 7f24a5a00640 1 --2- 172.21.15.46:0/1411588393 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24a015a890 0x7f24a00cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.881+0000 7f24a5a00640 1 -- 172.21.15.46:0/1411588393 >> v1:172.21.15.46:6789/0 conn(0x7f24a0069d90 legacy=0x7f24a015e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.881+0000 7f24a5a00640 1 -- 172.21.15.46:0/1411588393 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a00aec00 con 0x7f24a00b02b0 2024-03-29T08:30:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.885+0000 7f24a5a00640 1 --2- 172.21.15.46:0/1411588393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a00b02b0 0x7f24a00b0740 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f2498009d90 tx=0x7f249801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a866b63cbcf45cea server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.885+0000 7f249fe00640 1 -- 172.21.15.46:0/1411588393 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24980a5a30 con 0x7f24a00b02b0 2024-03-29T08:30:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.885+0000 7f249fe00640 1 -- 172.21.15.46:0/1411588393 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24980a5b90 con 0x7f24a00b02b0 2024-03-29T08:30:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.885+0000 7f249fe00640 1 -- 172.21.15.46:0/1411588393 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24980ad440 con 0x7f24a00b02b0 2024-03-29T08:30:16.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.885+0000 7f24a6400640 1 -- 172.21.15.46:0/1411588393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a00b02b0 msgr2=0x7f24a00b0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.885+0000 7f24a6400640 1 --2- 172.21.15.46:0/1411588393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a00b02b0 0x7f24a00b0740 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f2498009d90 tx=0x7f249801c970 comp rx=0 tx=0).stop 2024-03-29T08:30:16.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.885+0000 7f24a6400640 1 -- 172.21.15.46:0/1411588393 shutdown_connections 2024-03-29T08:30:16.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.885+0000 7f24a6400640 1 --2- 172.21.15.46:0/1411588393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a00b02b0 0x7f24a00b0740 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.885+0000 7f24a6400640 1 --2- 172.21.15.46:0/1411588393 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24a015a890 0x7f24a00cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.885+0000 7f24a6400640 1 -- 172.21.15.46:0/1411588393 >> 172.21.15.46:0/1411588393 conn(0x7f24a0098a40 msgr2=0x7f24a00cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.885+0000 7f24a6400640 1 -- 172.21.15.46:0/1411588393 shutdown_connections 2024-03-29T08:30:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.885+0000 7f24a6400640 1 -- 172.21.15.46:0/1411588393 wait complete. 2024-03-29T08:30:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.885+0000 7f24a6400640 1 Processor -- start 2024-03-29T08:30:16.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.885+0000 7f24a6400640 1 -- start start 2024-03-29T08:30:16.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a6400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a0069d90 0x7f24a0153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a6400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24a00b02b0 0x7f24a00acf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a6400640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24a015a890 0x7f24a00ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a6400640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a00ae5f0 con 0x7f24a0069d90 2024-03-29T08:30:16.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a6400640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a00acab0 con 0x7f24a00b02b0 2024-03-29T08:30:16.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a6400640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a0153bc0 con 0x7f24a015a890 2024-03-29T08:30:16.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a0069d90 0x7f24a0153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a5000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a0069d90 0x7f24a0153680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38446/0 (socket says 172.21.15.46:38446) 2024-03-29T08:30:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a5000640 1 -- 172.21.15.46:0/451609232 learned_addr learned my addr 172.21.15.46:0/451609232 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f2497e00640 1 --2- 172.21.15.46:0/451609232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24a00b02b0 0x7f24a00acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a5000640 1 -- 172.21.15.46:0/451609232 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24a015a890 msgr2=0x7f24a00ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a5000640 1 --2- 172.21.15.46:0/451609232 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24a015a890 0x7f24a00ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a5000640 1 -- 172.21.15.46:0/451609232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24a00b02b0 msgr2=0x7f24a00acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a5000640 1 --2- 172.21.15.46:0/451609232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24a00b02b0 0x7f24a00acf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a5000640 1 -- 172.21.15.46:0/451609232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2498009a70 con 0x7f24a0069d90 2024-03-29T08:30:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f2497e00640 1 --2- 172.21.15.46:0/451609232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24a00b02b0 0x7f24a00acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a5000640 1 --2- 172.21.15.46:0/451609232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a0069d90 0x7f24a0153680 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f249000d620 tx=0x7f249000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:16.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f249d600640 1 -- 172.21.15.46:0/451609232 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249000b620 con 0x7f24a0069d90 2024-03-29T08:30:16.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f249d600640 1 -- 172.21.15.46:0/451609232 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f249000b780 con 0x7f24a0069d90 2024-03-29T08:30:16.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a6400640 1 -- 172.21.15.46:0/451609232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a00ae310 con 0x7f24a0069d90 2024-03-29T08:30:16.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f249d600640 1 -- 172.21.15.46:0/451609232 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249000bb10 con 0x7f24a0069d90 2024-03-29T08:30:16.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a6400640 1 -- 172.21.15.46:0/451609232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a00ac740 con 0x7f24a0069d90 2024-03-29T08:30:16.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f24a6400640 1 -- 172.21.15.46:0/451609232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a015f170 con 0x7f24a0069d90 2024-03-29T08:30:16.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.889+0000 7f249d600640 1 -- 172.21.15.46:0/451609232 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f249000bc70 con 0x7f24a0069d90 2024-03-29T08:30:16.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.893+0000 7f249d600640 1 --2- 172.21.15.46:0/451609232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f247c0666c0 0x7f247c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:16.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.893+0000 7f249d600640 1 -- 172.21.15.46:0/451609232 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f24900847a0 con 0x7f24a0069d90 2024-03-29T08:30:16.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.893+0000 7f2497e00640 1 --2- 172.21.15.46:0/451609232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f247c0666c0 0x7f247c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:16.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.893+0000 7f2497e00640 1 --2- 172.21.15.46:0/451609232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f247c0666c0 0x7f247c068b80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f24a0164ac0 tx=0x7f2488009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:16.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:16.897+0000 7f249d600640 1 -- 172.21.15.46:0/451609232 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2490051990 con 0x7f24a0069d90 2024-03-29T08:30:17.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.093+0000 7f24a6400640 1 -- 172.21.15.46:0/451609232 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f24a004e040 con 0x7f24a0069d90 2024-03-29T08:30:17.094 INFO:tasks.ceph.mon.a.smithi046.stderr:2024-03-29T08:30:17.093+0000 7fc16ae00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-29T08:30:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.101+0000 7f249d600640 1 -- 172.21.15.46:0/451609232 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v108) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2490054fb0 con 0x7f24a0069d90 2024-03-29T08:30:17.102 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-29T08:30:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.105+0000 7f24a6400640 1 -- 172.21.15.46:0/451609232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f247c0666c0 msgr2=0x7f247c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.105+0000 7f24a6400640 1 --2- 172.21.15.46:0/451609232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f247c0666c0 0x7f247c068b80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f24a0164ac0 tx=0x7f2488009040 comp rx=0 tx=0).stop 2024-03-29T08:30:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.105+0000 7f24a6400640 1 -- 172.21.15.46:0/451609232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a0069d90 msgr2=0x7f24a0153680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.105+0000 7f24a6400640 1 --2- 172.21.15.46:0/451609232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a0069d90 0x7f24a0153680 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f249000d620 tx=0x7f249000dae0 comp rx=0 tx=0).stop 2024-03-29T08:30:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.105+0000 7f24a6400640 1 -- 172.21.15.46:0/451609232 shutdown_connections 2024-03-29T08:30:17.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.105+0000 7f24a6400640 1 --2- 172.21.15.46:0/451609232 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f247c0666c0 0x7f247c068b80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.105+0000 7f24a6400640 1 --2- 172.21.15.46:0/451609232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f24a00b02b0 0x7f24a00acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.105+0000 7f24a6400640 1 --2- 172.21.15.46:0/451609232 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f24a015a890 0x7f24a00ad9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.105+0000 7f24a6400640 1 --2- 172.21.15.46:0/451609232 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24a0069d90 0x7f24a0153680 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.105+0000 7f24a6400640 1 -- 172.21.15.46:0/451609232 >> 172.21.15.46:0/451609232 conn(0x7f24a0098a40 msgr2=0x7f24a0164fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.105+0000 7f24a6400640 1 -- 172.21.15.46:0/451609232 shutdown_connections 2024-03-29T08:30:17.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.105+0000 7f24a6400640 1 -- 172.21.15.46:0/451609232 wait complete. 2024-03-29T08:30:17.107 INFO:tasks.ceph.mds.a.smithi046.stderr: -1451> 2024-03-29T08:30:06.085+0000 7fec71e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-29T08:30:17.124 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-29T08:30:17.152 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2024-03-29T08:30:17.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.269+0000 7f5172000640 1 Processor -- start 2024-03-29T08:30:17.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.269+0000 7f5172000640 1 -- start start 2024-03-29T08:30:17.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.269+0000 7f5172000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f516c15aea0 0x7f516c0cff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:17.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.269+0000 7f5172000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f516c06a890 0x7f516c0cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:17.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.269+0000 7f5172000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516c0cb210 con 0x7f516c0b04f0 2024-03-29T08:30:17.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.269+0000 7f5172000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516c15bee0 con 0x7f516c15aea0 2024-03-29T08:30:17.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.269+0000 7f5172000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516c1553c0 con 0x7f516c06a890 2024-03-29T08:30:17.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.269+0000 7f516b400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f516c0b04f0 0x7f516c0b08d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57800/0 (socket says 172.21.15.46:57800) 2024-03-29T08:30:17.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.269+0000 7f516b400640 1 -- 172.21.15.46:0/3649865556 learned_addr learned my addr 172.21.15.46:0/3649865556 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:17.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.269+0000 7f5162a00640 1 --2- 172.21.15.46:0/3649865556 >> v2:172.21.15.46:3300/0 conn(0x7f516c15aea0 0x7f516c0cff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:17.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.269+0000 7f516be00640 1 --2- 172.21.15.46:0/3649865556 >> v2:172.21.15.73:3300/0 conn(0x7f516c06a890 0x7f516c0cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:17.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.269+0000 7f5162a00640 1 -- 172.21.15.46:0/3649865556 >> v2:172.21.15.73:3300/0 conn(0x7f516c06a890 msgr2=0x7f516c0cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:17.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.269+0000 7f5162a00640 1 --2- 172.21.15.46:0/3649865556 >> v2:172.21.15.73:3300/0 conn(0x7f516c06a890 0x7f516c0cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:17.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.269+0000 7f5162a00640 1 -- 172.21.15.46:0/3649865556 >> v1:172.21.15.46:6789/0 conn(0x7f516c0b04f0 legacy=0x7f516c0b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:17.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.269+0000 7f5162a00640 1 -- 172.21.15.46:0/3649865556 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516c15b360 con 0x7f516c15aea0 2024-03-29T08:30:17.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.269+0000 7f516aa00640 1 -- 172.21.15.46:0/3649865556 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 271719920 0 0) 0x7f516c0cb210 con 0x7f516c0b04f0 2024-03-29T08:30:17.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.273+0000 7f5162a00640 1 --2- 172.21.15.46:0/3649865556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f516c15aea0 0x7f516c0cff40 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f516c0cd060 tx=0x7f515401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443d8a22377aa06c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:17.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.273+0000 7f516aa00640 1 -- 172.21.15.46:0/3649865556 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51540ab600 con 0x7f516c15aea0 2024-03-29T08:30:17.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.273+0000 7f516aa00640 1 -- 172.21.15.46:0/3649865556 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51540a7050 con 0x7f516c15aea0 2024-03-29T08:30:17.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.273+0000 7f516aa00640 1 -- 172.21.15.46:0/3649865556 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51540ab980 con 0x7f516c15aea0 2024-03-29T08:30:17.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.273+0000 7f5172000640 1 -- 172.21.15.46:0/3649865556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f516c15aea0 msgr2=0x7f516c0cff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:17.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.273+0000 7f5172000640 1 --2- 172.21.15.46:0/3649865556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f516c15aea0 0x7f516c0cff40 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f516c0cd060 tx=0x7f515401c3f0 comp rx=0 tx=0).stop 2024-03-29T08:30:17.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.273+0000 7f5172000640 1 -- 172.21.15.46:0/3649865556 shutdown_connections 2024-03-29T08:30:17.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.273+0000 7f5172000640 1 --2- 172.21.15.46:0/3649865556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f516c15aea0 0x7f516c0cff40 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:17.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.273+0000 7f5172000640 1 --2- 172.21.15.46:0/3649865556 >> v2:172.21.15.73:3300/0 conn(0x7f516c06a890 0x7f516c0cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:17.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.273+0000 7f5172000640 1 -- 172.21.15.46:0/3649865556 >> 172.21.15.46:0/3649865556 conn(0x7f516c098a40 msgr2=0x7f516c0cf620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:17.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.273+0000 7f5172000640 1 -- 172.21.15.46:0/3649865556 shutdown_connections 2024-03-29T08:30:17.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.273+0000 7f5172000640 1 -- 172.21.15.46:0/3649865556 wait complete. 2024-03-29T08:30:17.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f5172000640 1 Processor -- start 2024-03-29T08:30:17.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f5172000640 1 -- start start 2024-03-29T08:30:17.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f5172000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f516c06a890 0x7f516c152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:17.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f5172000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f516c0b04f0 0x7f516c151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:17.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f5172000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f516c15aea0 0x7f516c1515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:17.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f5172000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c14ecf0 con 0x7f516c06a890 2024-03-29T08:30:17.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f5172000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c06db00 con 0x7f516c0b04f0 2024-03-29T08:30:17.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f5172000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516c06d820 con 0x7f516c15aea0 2024-03-29T08:30:17.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f516b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f516c06a890 0x7f516c152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f516b400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f516c06a890 0x7f516c152650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38476/0 (socket says 172.21.15.46:38476) 2024-03-29T08:30:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f516b400640 1 -- 172.21.15.46:0/1881844565 learned_addr learned my addr 172.21.15.46:0/1881844565 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f516be00640 1 --2- 172.21.15.46:0/1881844565 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f516c15aea0 0x7f516c1515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f5162a00640 1 --2- 172.21.15.46:0/1881844565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f516c0b04f0 0x7f516c151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f516b400640 1 -- 172.21.15.46:0/1881844565 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f516c15aea0 msgr2=0x7f516c1515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f516b400640 1 --2- 172.21.15.46:0/1881844565 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f516c15aea0 0x7f516c1515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f516b400640 1 -- 172.21.15.46:0/1881844565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f516c0b04f0 msgr2=0x7f516c151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f516b400640 1 --2- 172.21.15.46:0/1881844565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f516c0b04f0 0x7f516c151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f516b400640 1 -- 172.21.15.46:0/1881844565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5154009420 con 0x7f516c06a890 2024-03-29T08:30:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f516b400640 1 --2- 172.21.15.46:0/1881844565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f516c06a890 0x7f516c152650 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f515c0011f0 tx=0x7f515c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f5163e00640 1 -- 172.21.15.46:0/1881844565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c0074e0 con 0x7f516c06a890 2024-03-29T08:30:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f5163e00640 1 -- 172.21.15.46:0/1881844565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c007640 con 0x7f516c06a890 2024-03-29T08:30:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f5163e00640 1 -- 172.21.15.46:0/1881844565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c004a10 con 0x7f516c06a890 2024-03-29T08:30:17.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f5172000640 1 -- 172.21.15.46:0/1881844565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516c14ea10 con 0x7f516c06a890 2024-03-29T08:30:17.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.277+0000 7f5172000640 1 -- 172.21.15.46:0/1881844565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516c14b640 con 0x7f516c06a890 2024-03-29T08:30:17.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.281+0000 7f5172000640 1 -- 172.21.15.46:0/1881844565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516c1509d0 con 0x7f516c06a890 2024-03-29T08:30:17.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.281+0000 7f5163e00640 1 -- 172.21.15.46:0/1881844565 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f515c004b70 con 0x7f516c06a890 2024-03-29T08:30:17.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.281+0000 7f5163e00640 1 --2- 172.21.15.46:0/1881844565 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5148066690 0x7f5148068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:17.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.281+0000 7f5163e00640 1 -- 172.21.15.46:0/1881844565 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6025+0+0 (secure 0 0 0) 0x7f515c083470 con 0x7f516c06a890 2024-03-29T08:30:17.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.281+0000 7f5162a00640 1 --2- 172.21.15.46:0/1881844565 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5148066690 0x7f5148068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:17.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.281+0000 7f5162a00640 1 --2- 172.21.15.46:0/1881844565 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5148066690 0x7f5148068b50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f51540ad000 tx=0x7f5154008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:17.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.285+0000 7f5163e00640 1 -- 172.21.15.46:0/1881844565 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f516c1509d0 con 0x7f516c06a890 2024-03-29T08:30:17.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:17.481+0000 7f5172000640 1 -- 172.21.15.46:0/1881844565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f516c1509d0 con 0x7f516c06a890 2024-03-29T08:30:18.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.101+0000 7f5163e00640 1 -- 172.21.15.46:0/1881844565 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v109) v1 ==== 104+0+0 (secure 0 0 0) 0x7f516c1509d0 con 0x7f516c06a890 2024-03-29T08:30:18.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.105+0000 7f5172000640 1 -- 172.21.15.46:0/1881844565 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5148066690 msgr2=0x7f5148068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:18.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.105+0000 7f5172000640 1 --2- 172.21.15.46:0/1881844565 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5148066690 0x7f5148068b50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f51540ad000 tx=0x7f5154008040 comp rx=0 tx=0).stop 2024-03-29T08:30:18.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.105+0000 7f5172000640 1 -- 172.21.15.46:0/1881844565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f516c06a890 msgr2=0x7f516c152650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:18.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.105+0000 7f5172000640 1 --2- 172.21.15.46:0/1881844565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f516c06a890 0x7f516c152650 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f515c0011f0 tx=0x7f515c00c9b0 comp rx=0 tx=0).stop 2024-03-29T08:30:18.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.105+0000 7f5172000640 1 -- 172.21.15.46:0/1881844565 shutdown_connections 2024-03-29T08:30:18.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.105+0000 7f5172000640 1 --2- 172.21.15.46:0/1881844565 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5148066690 0x7f5148068b50 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:18.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.105+0000 7f5172000640 1 --2- 172.21.15.46:0/1881844565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f516c0b04f0 0x7f516c151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:18.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.105+0000 7f5172000640 1 --2- 172.21.15.46:0/1881844565 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f516c15aea0 0x7f516c1515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:18.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.105+0000 7f5172000640 1 --2- 172.21.15.46:0/1881844565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f516c06a890 0x7f516c152650 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:18.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.105+0000 7f5172000640 1 -- 172.21.15.46:0/1881844565 >> 172.21.15.46:0/1881844565 conn(0x7f516c098a40 msgr2=0x7f516c0b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:18.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.105+0000 7f5172000640 1 -- 172.21.15.46:0/1881844565 shutdown_connections 2024-03-29T08:30:18.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.105+0000 7f5172000640 1 -- 172.21.15.46:0/1881844565 wait complete. 2024-03-29T08:30:18.130 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-29T08:30:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.321+0000 7f67bda00640 1 Processor -- start 2024-03-29T08:30:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.321+0000 7f67bda00640 1 -- start start 2024-03-29T08:30:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.325+0000 7f67bda00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67b8160060 0x7f67b80cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.325+0000 7f67bda00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b80cea60 con 0x7f67b80cfc10 2024-03-29T08:30:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.325+0000 7f67bda00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b806bea0 con 0x7f67b80b1780 2024-03-29T08:30:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.325+0000 7f67bda00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b80ced00 con 0x7f67b8160060 2024-03-29T08:30:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.325+0000 7f67b6a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67b8160060 0x7f67b80cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:18.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.325+0000 7f67b7400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f67b80cfc10 0x7f67b8098a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57806/0 (socket says 172.21.15.46:57806) 2024-03-29T08:30:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.325+0000 7f67b7400640 1 -- 172.21.15.46:0/3316809791 learned_addr learned my addr 172.21.15.46:0/3316809791 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.325+0000 7f67b6a00640 1 -- 172.21.15.46:0/3316809791 >> v1:172.21.15.73:6789/0 conn(0x7f67b80b1780 legacy=0x7f67b80cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.325+0000 7f67b6a00640 1 -- 172.21.15.46:0/3316809791 >> v1:172.21.15.46:6789/0 conn(0x7f67b80cfc10 legacy=0x7f67b8098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:18.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.325+0000 7f67b6a00640 1 -- 172.21.15.46:0/3316809791 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b80cefe0 con 0x7f67b8160060 2024-03-29T08:30:18.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.329+0000 7f67b6a00640 1 --2- 172.21.15.46:0/3316809791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b8160060 0x7f67b80cad10 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f67a80097c0 tx=0x7f67a801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b620b0c25fe967bb server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:18.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.329+0000 7f67b6000640 1 -- 172.21.15.46:0/3316809791 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a80ab600 con 0x7f67b8160060 2024-03-29T08:30:18.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.329+0000 7f67b6000640 1 -- 172.21.15.46:0/3316809791 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a80a7050 con 0x7f67b8160060 2024-03-29T08:30:18.328 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.329+0000 7f67b6000640 1 -- 172.21.15.46:0/3316809791 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a80ab980 con 0x7f67b8160060 2024-03-29T08:30:18.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.329+0000 7f67bda00640 1 -- 172.21.15.46:0/3316809791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b8160060 msgr2=0x7f67b80cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:18.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.329+0000 7f67bda00640 1 --2- 172.21.15.46:0/3316809791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b8160060 0x7f67b80cad10 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f67a80097c0 tx=0x7f67a801c410 comp rx=0 tx=0).stop 2024-03-29T08:30:18.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.329+0000 7f67bda00640 1 -- 172.21.15.46:0/3316809791 shutdown_connections 2024-03-29T08:30:18.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.329+0000 7f67bda00640 1 --2- 172.21.15.46:0/3316809791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b8160060 0x7f67b80cad10 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:18.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.329+0000 7f67bda00640 1 -- 172.21.15.46:0/3316809791 >> 172.21.15.46:0/3316809791 conn(0x7f67b80b21b0 msgr2=0x7f67b80b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:18.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.329+0000 7f67bda00640 1 -- 172.21.15.46:0/3316809791 shutdown_connections 2024-03-29T08:30:18.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.329+0000 7f67bda00640 1 -- 172.21.15.46:0/3316809791 wait complete. 2024-03-29T08:30:18.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.329+0000 7f67bda00640 1 Processor -- start 2024-03-29T08:30:18.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.329+0000 7f67bda00640 1 -- start start 2024-03-29T08:30:18.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.329+0000 7f67bda00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67b80b1780 0x7f67b8151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:18.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.329+0000 7f67bda00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b80cfc10 0x7f67b80c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:18.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.329+0000 7f67bda00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f67b8160060 0x7f67b815ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:18.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67bda00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b80b6230 con 0x7f67b80cfc10 2024-03-29T08:30:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67bda00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b80b5c70 con 0x7f67b80b1780 2024-03-29T08:30:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67bda00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b80b5f50 con 0x7f67b8160060 2024-03-29T08:30:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b80cfc10 0x7f67b80c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67afe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b80cfc10 0x7f67b80c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38496/0 (socket says 172.21.15.46:38496) 2024-03-29T08:30:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67afe00640 1 -- 172.21.15.46:0/2244189132 learned_addr learned my addr 172.21.15.46:0/2244189132 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67b6a00640 1 --2- 172.21.15.46:0/2244189132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67b80b1780 0x7f67b8151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67b7400640 1 --2- 172.21.15.46:0/2244189132 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f67b8160060 0x7f67b815ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67afe00640 1 -- 172.21.15.46:0/2244189132 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f67b8160060 msgr2=0x7f67b815ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67afe00640 1 --2- 172.21.15.46:0/2244189132 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f67b8160060 0x7f67b815ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67afe00640 1 -- 172.21.15.46:0/2244189132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67b80b1780 msgr2=0x7f67b8151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67afe00640 1 --2- 172.21.15.46:0/2244189132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67b80b1780 0x7f67b8151030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67afe00640 1 -- 172.21.15.46:0/2244189132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a8009420 con 0x7f67b80cfc10 2024-03-29T08:30:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67b7400640 1 --2- 172.21.15.46:0/2244189132 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f67b8160060 0x7f67b815ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:18.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67afe00640 1 --2- 172.21.15.46:0/2244189132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b80cfc10 0x7f67b80c8680 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f67a000c500 tx=0x7f67a000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:18.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67aea00640 1 -- 172.21.15.46:0/2244189132 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0005220 con 0x7f67b80cfc10 2024-03-29T08:30:18.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67aea00640 1 -- 172.21.15.46:0/2244189132 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a0005380 con 0x7f67b80cfc10 2024-03-29T08:30:18.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67aea00640 1 -- 172.21.15.46:0/2244189132 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a0007270 con 0x7f67b80cfc10 2024-03-29T08:30:18.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67bda00640 1 -- 172.21.15.46:0/2244189132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67b815e780 con 0x7f67b80cfc10 2024-03-29T08:30:18.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67bda00640 1 -- 172.21.15.46:0/2244189132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b815f4c0 con 0x7f67b80cfc10 2024-03-29T08:30:18.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67bda00640 1 -- 172.21.15.46:0/2244189132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b815e5b0 con 0x7f67b80cfc10 2024-03-29T08:30:18.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.333+0000 7f67aea00640 1 -- 172.21.15.46:0/2244189132 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f67a00073d0 con 0x7f67b80cfc10 2024-03-29T08:30:18.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.337+0000 7f67aea00640 1 --2- 172.21.15.46:0/2244189132 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6794066690 0x7f6794068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:18.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.337+0000 7f67aea00640 1 -- 172.21.15.46:0/2244189132 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6025+0+0 (secure 0 0 0) 0x7f67a0083620 con 0x7f67b80cfc10 2024-03-29T08:30:18.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.337+0000 7f67b6a00640 1 --2- 172.21.15.46:0/2244189132 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6794066690 0x7f6794068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:18.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.337+0000 7f67b6a00640 1 --2- 172.21.15.46:0/2244189132 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6794066690 0x7f6794068b50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f67a80ad000 tx=0x7f67a80a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:18.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.341+0000 7f67aea00640 1 -- 172.21.15.46:0/2244189132 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f67a0050870 con 0x7f67b80cfc10 2024-03-29T08:30:18.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:18.505+0000 7f67bda00640 1 -- 172.21.15.46:0/2244189132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f67b8066c90 con 0x7f67b80cfc10 2024-03-29T08:30:19.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.109+0000 7f67aea00640 1 -- 172.21.15.46:0/2244189132 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v127) v1 ==== 181+0+0 (secure 0 0 0) 0x7f67a0053e90 con 0x7f67b80cfc10 2024-03-29T08:30:19.109 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-03-29T08:30:19.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.113+0000 7f67bda00640 1 -- 172.21.15.46:0/2244189132 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6794066690 msgr2=0x7f6794068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:19.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.113+0000 7f67bda00640 1 --2- 172.21.15.46:0/2244189132 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6794066690 0x7f6794068b50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f67a80ad000 tx=0x7f67a80a6000 comp rx=0 tx=0).stop 2024-03-29T08:30:19.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.113+0000 7f67bda00640 1 -- 172.21.15.46:0/2244189132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b80cfc10 msgr2=0x7f67b80c8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:19.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.113+0000 7f67bda00640 1 --2- 172.21.15.46:0/2244189132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b80cfc10 0x7f67b80c8680 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f67a000c500 tx=0x7f67a000c9c0 comp rx=0 tx=0).stop 2024-03-29T08:30:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.113+0000 7f67bda00640 1 -- 172.21.15.46:0/2244189132 shutdown_connections 2024-03-29T08:30:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.113+0000 7f67bda00640 1 --2- 172.21.15.46:0/2244189132 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f6794066690 0x7f6794068b50 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.113+0000 7f67bda00640 1 --2- 172.21.15.46:0/2244189132 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f67b8160060 0x7f67b815ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.113+0000 7f67bda00640 1 --2- 172.21.15.46:0/2244189132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67b80cfc10 0x7f67b80c8680 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.113+0000 7f67bda00640 1 --2- 172.21.15.46:0/2244189132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67b80b1780 0x7f67b8151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.113+0000 7f67bda00640 1 -- 172.21.15.46:0/2244189132 >> 172.21.15.46:0/2244189132 conn(0x7f67b80b21b0 msgr2=0x7f67b815c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.113+0000 7f67bda00640 1 -- 172.21.15.46:0/2244189132 shutdown_connections 2024-03-29T08:30:19.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.113+0000 7f67bda00640 1 -- 172.21.15.46:0/2244189132 wait complete. 2024-03-29T08:30:19.137 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-29T08:30:19.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.285+0000 7fde69200640 1 Processor -- start 2024-03-29T08:30:19.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.285+0000 7fde69200640 1 -- start start 2024-03-29T08:30:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.289+0000 7fde69200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde6415a7c0 0x7fde640cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.289+0000 7fde69200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fde6415eb60 0x7fde640b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:19.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.289+0000 7fde69200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde640cf230 con 0x7fde640cc270 2024-03-29T08:30:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.289+0000 7fde69200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde640cf6c0 con 0x7fde6415a7c0 2024-03-29T08:30:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.289+0000 7fde69200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde640ce620 con 0x7fde6415eb60 2024-03-29T08:30:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.289+0000 7fde61e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde6415a7c0 0x7fde640cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.289+0000 7fde61e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fde6415a7c0 0x7fde640cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38498/0 (socket says 172.21.15.46:38498) 2024-03-29T08:30:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.289+0000 7fde61e00640 1 -- 172.21.15.46:0/569411460 learned_addr learned my addr 172.21.15.46:0/569411460 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.289+0000 7fde62800640 1 --2- 172.21.15.46:0/569411460 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fde6415eb60 0x7fde640b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.289+0000 7fde61e00640 1 -- 172.21.15.46:0/569411460 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fde6415eb60 msgr2=0x7fde640b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.289+0000 7fde61e00640 1 --2- 172.21.15.46:0/569411460 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fde6415eb60 0x7fde640b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.289+0000 7fde61e00640 1 -- 172.21.15.46:0/569411460 >> v1:172.21.15.73:6789/0 conn(0x7fde640cc270 legacy=0x7fde6415e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:19.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.289+0000 7fde61e00640 1 -- 172.21.15.46:0/569411460 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde640ce900 con 0x7fde6415a7c0 2024-03-29T08:30:19.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.289+0000 7fde61e00640 1 --2- 172.21.15.46:0/569411460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde6415a7c0 0x7fde640cbd30 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fde54009e50 tx=0x7fde5401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0a8edc7829e7b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:19.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.289+0000 7fde61400640 1 -- 172.21.15.46:0/569411460 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540a57f0 con 0x7fde6415a7c0 2024-03-29T08:30:19.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde61400640 1 -- 172.21.15.46:0/569411460 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde540a5950 con 0x7fde6415a7c0 2024-03-29T08:30:19.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde61400640 1 -- 172.21.15.46:0/569411460 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540a5ce0 con 0x7fde6415a7c0 2024-03-29T08:30:19.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde69200640 1 -- 172.21.15.46:0/569411460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde6415a7c0 msgr2=0x7fde640cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:19.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde69200640 1 --2- 172.21.15.46:0/569411460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde6415a7c0 0x7fde640cbd30 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fde54009e50 tx=0x7fde5401ca30 comp rx=0 tx=0).stop 2024-03-29T08:30:19.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde69200640 1 -- 172.21.15.46:0/569411460 shutdown_connections 2024-03-29T08:30:19.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde69200640 1 --2- 172.21.15.46:0/569411460 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fde6415eb60 0x7fde640b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:19.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde69200640 1 --2- 172.21.15.46:0/569411460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde6415a7c0 0x7fde640cbd30 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:19.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde69200640 1 -- 172.21.15.46:0/569411460 >> 172.21.15.46:0/569411460 conn(0x7fde640b21d0 msgr2=0x7fde640b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:19.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde69200640 1 -- 172.21.15.46:0/569411460 shutdown_connections 2024-03-29T08:30:19.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde69200640 1 -- 172.21.15.46:0/569411460 wait complete. 2024-03-29T08:30:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde69200640 1 Processor -- start 2024-03-29T08:30:19.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde69200640 1 -- start start 2024-03-29T08:30:19.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde69200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde640cc270 0x7fde640b5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:19.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde69200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde6415a7c0 0x7fde64158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:19.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde69200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fde6415eb60 0x7fde640b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:19.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde69200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde640b5a60 con 0x7fde640cc270 2024-03-29T08:30:19.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde69200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde640b4480 con 0x7fde6415a7c0 2024-03-29T08:30:19.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde69200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde640b4760 con 0x7fde6415eb60 2024-03-29T08:30:19.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde61e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde640cc270 0x7fde640b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:19.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde61e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde640cc270 0x7fde640b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38508/0 (socket says 172.21.15.46:38508) 2024-03-29T08:30:19.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde61e00640 1 -- 172.21.15.46:0/3066667450 learned_addr learned my addr 172.21.15.46:0/3066667450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:19.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde62800640 1 --2- 172.21.15.46:0/3066667450 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fde6415eb60 0x7fde640b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:19.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde59400640 1 --2- 172.21.15.46:0/3066667450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde6415a7c0 0x7fde64158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:19.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde61e00640 1 -- 172.21.15.46:0/3066667450 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fde6415eb60 msgr2=0x7fde640b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:19.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde61e00640 1 --2- 172.21.15.46:0/3066667450 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fde6415eb60 0x7fde640b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:19.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde61e00640 1 -- 172.21.15.46:0/3066667450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde6415a7c0 msgr2=0x7fde64158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:19.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde61e00640 1 --2- 172.21.15.46:0/3066667450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde6415a7c0 0x7fde64158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:19.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.293+0000 7fde61e00640 1 -- 172.21.15.46:0/3066667450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde54009b30 con 0x7fde640cc270 2024-03-29T08:30:19.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.297+0000 7fde61e00640 1 --2- 172.21.15.46:0/3066667450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde640cc270 0x7fde640b5520 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fde5401a660 tx=0x7fde540197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:19.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.297+0000 7fde5aa00640 1 -- 172.21.15.46:0/3066667450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540bd070 con 0x7fde640cc270 2024-03-29T08:30:19.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.297+0000 7fde5aa00640 1 -- 172.21.15.46:0/3066667450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde540b8470 con 0x7fde640cc270 2024-03-29T08:30:19.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.297+0000 7fde69200640 1 -- 172.21.15.46:0/3066667450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde640b4a40 con 0x7fde640cc270 2024-03-29T08:30:19.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.297+0000 7fde5aa00640 1 -- 172.21.15.46:0/3066667450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde540b8840 con 0x7fde640cc270 2024-03-29T08:30:19.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.297+0000 7fde69200640 1 -- 172.21.15.46:0/3066667450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde640b5000 con 0x7fde640cc270 2024-03-29T08:30:19.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.297+0000 7fde69200640 1 -- 172.21.15.46:0/3066667450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde64066c90 con 0x7fde640cc270 2024-03-29T08:30:19.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.297+0000 7fde5aa00640 1 -- 172.21.15.46:0/3066667450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fde540b89a0 con 0x7fde640cc270 2024-03-29T08:30:19.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.297+0000 7fde5aa00640 1 --2- 172.21.15.46:0/3066667450 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fde40066690 0x7fde40068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:19.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.297+0000 7fde5aa00640 1 -- 172.21.15.46:0/3066667450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5558+0+0 (secure 0 0 0) 0x7fde5411ee20 con 0x7fde640cc270 2024-03-29T08:30:19.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.301+0000 7fde59400640 1 --2- 172.21.15.46:0/3066667450 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fde40066690 0x7fde40068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:19.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.301+0000 7fde59400640 1 --2- 172.21.15.46:0/3066667450 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fde40066690 0x7fde40068b50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fde641f8c10 tx=0x7fde4c009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:19.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.301+0000 7fde5aa00640 1 -- 172.21.15.46:0/3066667450 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fde540ec340 con 0x7fde640cc270 2024-03-29T08:30:19.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:19.469+0000 7fde69200640 1 -- 172.21.15.46:0/3066667450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fde640b2e30 con 0x7fde640cc270 2024-03-29T08:30:20.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.109+0000 7fde5aa00640 1 -- 172.21.15.46:0/3066667450 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v128) v1 ==== 169+0+0 (secure 0 0 0) 0x7fde540ef960 con 0x7fde640cc270 2024-03-29T08:30:20.111 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-03-29T08:30:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.117+0000 7fde69200640 1 -- 172.21.15.46:0/3066667450 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fde40066690 msgr2=0x7fde40068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.117+0000 7fde69200640 1 --2- 172.21.15.46:0/3066667450 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fde40066690 0x7fde40068b50 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fde641f8c10 tx=0x7fde4c009040 comp rx=0 tx=0).stop 2024-03-29T08:30:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.117+0000 7fde69200640 1 -- 172.21.15.46:0/3066667450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde640cc270 msgr2=0x7fde640b5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:20.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.117+0000 7fde69200640 1 --2- 172.21.15.46:0/3066667450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde640cc270 0x7fde640b5520 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fde5401a660 tx=0x7fde540197d0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.117+0000 7fde69200640 1 -- 172.21.15.46:0/3066667450 shutdown_connections 2024-03-29T08:30:20.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.117+0000 7fde69200640 1 --2- 172.21.15.46:0/3066667450 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fde40066690 0x7fde40068b50 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.117+0000 7fde69200640 1 --2- 172.21.15.46:0/3066667450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde6415a7c0 0x7fde64158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.117+0000 7fde69200640 1 --2- 172.21.15.46:0/3066667450 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fde6415eb60 0x7fde640b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.117+0000 7fde69200640 1 --2- 172.21.15.46:0/3066667450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde640cc270 0x7fde640b5520 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.117+0000 7fde69200640 1 -- 172.21.15.46:0/3066667450 >> 172.21.15.46:0/3066667450 conn(0x7fde640b21d0 msgr2=0x7fde64154670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:20.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.117+0000 7fde69200640 1 -- 172.21.15.46:0/3066667450 shutdown_connections 2024-03-29T08:30:20.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.117+0000 7fde69200640 1 -- 172.21.15.46:0/3066667450 wait complete. 2024-03-29T08:30:20.145 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-29T08:30:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.293+0000 7f8e91e00640 1 Processor -- start 2024-03-29T08:30:20.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.293+0000 7f8e91e00640 1 -- start start 2024-03-29T08:30:20.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.293+0000 7f8e91e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8e8c15df90 0x7f8e8c0ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:20.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.293+0000 7f8e91e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8e8c0cbea0 0x7f8e8c0ae7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:20.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.293+0000 7f8e91e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e8c15d260 con 0x7f8e8c142a50 2024-03-29T08:30:20.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.293+0000 7f8e91e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8c0aed00 con 0x7f8e8c15df90 2024-03-29T08:30:20.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.293+0000 7f8e91e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8c0aefd0 con 0x7f8e8c0cbea0 2024-03-29T08:30:20.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.293+0000 7f8e8a000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8e8c15df90 0x7f8e8c0ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:20.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.293+0000 7f8e8aa00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8e8c142a50 0x7f8e8c0cb790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57820/0 (socket says 172.21.15.46:57820) 2024-03-29T08:30:20.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.293+0000 7f8e8aa00640 1 -- 172.21.15.46:0/3598009266 learned_addr learned my addr 172.21.15.46:0/3598009266 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:20.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.293+0000 7f8e8b400640 1 --2- 172.21.15.46:0/3598009266 >> v2:172.21.15.73:3300/0 conn(0x7f8e8c0cbea0 0x7f8e8c0ae7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:20.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.293+0000 7f8e8a000640 1 -- 172.21.15.46:0/3598009266 >> v2:172.21.15.73:3300/0 conn(0x7f8e8c0cbea0 msgr2=0x7f8e8c0ae7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:20.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.293+0000 7f8e89600640 1 -- 172.21.15.46:0/3598009266 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569765427 0 0) 0x7f8e8c15d260 con 0x7f8e8c142a50 2024-03-29T08:30:20.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.293+0000 7f8e8a000640 1 --2- 172.21.15.46:0/3598009266 >> v2:172.21.15.73:3300/0 conn(0x7f8e8c0cbea0 0x7f8e8c0ae7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.293+0000 7f8e8a000640 1 -- 172.21.15.46:0/3598009266 >> v1:172.21.15.46:6789/0 conn(0x7f8e8c142a50 legacy=0x7f8e8c0cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:20.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.293+0000 7f8e8a000640 1 -- 172.21.15.46:0/3598009266 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8c15d4b0 con 0x7f8e8c15df90 2024-03-29T08:30:20.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.297+0000 7f8e8a000640 1 --2- 172.21.15.46:0/3598009266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c15df90 0x7f8e8c0ae280 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c06e4e0 tx=0x7f8e7801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74349e088415c766 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:20.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.297+0000 7f8e89600640 1 -- 172.21.15.46:0/3598009266 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e780ab600 con 0x7f8e8c15df90 2024-03-29T08:30:20.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.297+0000 7f8e89600640 1 -- 172.21.15.46:0/3598009266 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e780a7050 con 0x7f8e8c15df90 2024-03-29T08:30:20.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.297+0000 7f8e89600640 1 -- 172.21.15.46:0/3598009266 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e780ab980 con 0x7f8e8c15df90 2024-03-29T08:30:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.297+0000 7f8e91e00640 1 -- 172.21.15.46:0/3598009266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c15df90 msgr2=0x7f8e8c0ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.297+0000 7f8e91e00640 1 --2- 172.21.15.46:0/3598009266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c15df90 0x7f8e8c0ae280 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c06e4e0 tx=0x7f8e7801c3f0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.297+0000 7f8e91e00640 1 -- 172.21.15.46:0/3598009266 shutdown_connections 2024-03-29T08:30:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.297+0000 7f8e91e00640 1 --2- 172.21.15.46:0/3598009266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c15df90 0x7f8e8c0ae280 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.297+0000 7f8e91e00640 1 --2- 172.21.15.46:0/3598009266 >> v2:172.21.15.73:3300/0 conn(0x7f8e8c0cbea0 0x7f8e8c0ae7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.297+0000 7f8e91e00640 1 -- 172.21.15.46:0/3598009266 >> 172.21.15.46:0/3598009266 conn(0x7f8e8c06a390 msgr2=0x7f8e8c0cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.297+0000 7f8e91e00640 1 -- 172.21.15.46:0/3598009266 shutdown_connections 2024-03-29T08:30:20.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.297+0000 7f8e91e00640 1 -- 172.21.15.46:0/3598009266 wait complete. 2024-03-29T08:30:20.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.297+0000 7f8e91e00640 1 Processor -- start 2024-03-29T08:30:20.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e91e00640 1 -- start start 2024-03-29T08:30:20.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e91e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c0cbea0 0x7f8e8c13f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:20.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e91e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e8c142a50 0x7f8e8c1412e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:20.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e91e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e8c15df90 0x7f8e8c0a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:20.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e91e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c141820 con 0x7f8e8c0cbea0 2024-03-29T08:30:20.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e91e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c13fb00 con 0x7f8e8c15df90 2024-03-29T08:30:20.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e91e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8c13fde0 con 0x7f8e8c142a50 2024-03-29T08:30:20.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e8aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c0cbea0 0x7f8e8c13f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:20.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e8aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c0cbea0 0x7f8e8c13f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38516/0 (socket says 172.21.15.46:38516) 2024-03-29T08:30:20.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e8aa00640 1 -- 172.21.15.46:0/3499069369 learned_addr learned my addr 172.21.15.46:0/3499069369 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:20.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e8a000640 1 --2- 172.21.15.46:0/3499069369 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e8c142a50 0x7f8e8c1412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e8b400640 1 --2- 172.21.15.46:0/3499069369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e8c15df90 0x7f8e8c0a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e8aa00640 1 -- 172.21.15.46:0/3499069369 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e8c142a50 msgr2=0x7f8e8c1412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e8aa00640 1 --2- 172.21.15.46:0/3499069369 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e8c142a50 0x7f8e8c1412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e8aa00640 1 -- 172.21.15.46:0/3499069369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e8c15df90 msgr2=0x7f8e8c0a8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e8aa00640 1 --2- 172.21.15.46:0/3499069369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e8c15df90 0x7f8e8c0a8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e8aa00640 1 -- 172.21.15.46:0/3499069369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e78009420 con 0x7f8e8c0cbea0 2024-03-29T08:30:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e8aa00640 1 --2- 172.21.15.46:0/3499069369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c0cbea0 0x7f8e8c13f4e0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0011f0 tx=0x7f8e7c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e83400640 1 -- 172.21.15.46:0/3499069369 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0074e0 con 0x7f8e8c0cbea0 2024-03-29T08:30:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e83400640 1 -- 172.21.15.46:0/3499069369 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c007640 con 0x7f8e8c0cbea0 2024-03-29T08:30:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e83400640 1 -- 172.21.15.46:0/3499069369 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c004a10 con 0x7f8e8c0cbea0 2024-03-29T08:30:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e91e00640 1 -- 172.21.15.46:0/3499069369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8c1400c0 con 0x7f8e8c0cbea0 2024-03-29T08:30:20.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e91e00640 1 -- 172.21.15.46:0/3499069369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e8c0aac60 con 0x7f8e8c0cbea0 2024-03-29T08:30:20.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.301+0000 7f8e91e00640 1 -- 172.21.15.46:0/3499069369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e8c06ad60 con 0x7f8e8c0cbea0 2024-03-29T08:30:20.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.305+0000 7f8e83400640 1 -- 172.21.15.46:0/3499069369 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f8e7c004b70 con 0x7f8e8c0cbea0 2024-03-29T08:30:20.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.305+0000 7f8e83400640 1 --2- 172.21.15.46:0/3499069369 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8e6c066690 0x7f8e6c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:20.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.305+0000 7f8e83400640 1 -- 172.21.15.46:0/3499069369 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f8e7c083180 con 0x7f8e8c0cbea0 2024-03-29T08:30:20.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.305+0000 7f8e8a000640 1 --2- 172.21.15.46:0/3499069369 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8e6c066690 0x7f8e6c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:20.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.305+0000 7f8e8a000640 1 --2- 172.21.15.46:0/3499069369 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8e6c066690 0x7f8e6c068b50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8e780ad000 tx=0x7f8e780a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:20.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.309+0000 7f8e83400640 1 -- 172.21.15.46:0/3499069369 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f8e7c0507e0 con 0x7f8e8c0cbea0 2024-03-29T08:30:20.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.469+0000 7f8e91e00640 1 -- 172.21.15.46:0/3499069369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8e8c071a70 con 0x7f8e8c0cbea0 2024-03-29T08:30:20.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.473+0000 7f8e83400640 1 -- 172.21.15.46:0/3499069369 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8e7c053e00 con 0x7f8e8c0cbea0 2024-03-29T08:30:20.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.477+0000 7f8e91e00640 1 -- 172.21.15.46:0/3499069369 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8e6c066690 msgr2=0x7f8e6c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:20.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.477+0000 7f8e91e00640 1 --2- 172.21.15.46:0/3499069369 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8e6c066690 0x7f8e6c068b50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f8e780ad000 tx=0x7f8e780a6000 comp rx=0 tx=0).stop 2024-03-29T08:30:20.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.477+0000 7f8e91e00640 1 -- 172.21.15.46:0/3499069369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c0cbea0 msgr2=0x7f8e8c13f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:20.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.477+0000 7f8e91e00640 1 --2- 172.21.15.46:0/3499069369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c0cbea0 0x7f8e8c13f4e0 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c0011f0 tx=0x7f8e7c00c9b0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.477+0000 7f8e91e00640 1 -- 172.21.15.46:0/3499069369 shutdown_connections 2024-03-29T08:30:20.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.477+0000 7f8e91e00640 1 --2- 172.21.15.46:0/3499069369 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f8e6c066690 0x7f8e6c068b50 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.477+0000 7f8e91e00640 1 --2- 172.21.15.46:0/3499069369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e8c15df90 0x7f8e8c0a8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.477+0000 7f8e91e00640 1 --2- 172.21.15.46:0/3499069369 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f8e8c142a50 0x7f8e8c1412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.477+0000 7f8e91e00640 1 --2- 172.21.15.46:0/3499069369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e8c0cbea0 0x7f8e8c13f4e0 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.477+0000 7f8e91e00640 1 -- 172.21.15.46:0/3499069369 >> 172.21.15.46:0/3499069369 conn(0x7f8e8c06a390 msgr2=0x7f8e8c06c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.477+0000 7f8e91e00640 1 -- 172.21.15.46:0/3499069369 shutdown_connections 2024-03-29T08:30:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.477+0000 7f8e91e00640 1 -- 172.21.15.46:0/3499069369 wait complete. 2024-03-29T08:30:20.496 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-03-29T08:30:20.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf7e00640 1 Processor -- start 2024-03-29T08:30:20.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf7e00640 1 -- start start 2024-03-29T08:30:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf7e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdf015a940 0x7fcdf00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcdf015ad20 0x7fcdf00cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf7e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcdf00b0540 0x7fcdf00cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf7e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf00b00d0 con 0x7fcdf015ad20 2024-03-29T08:30:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf7e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf00b0ab0 con 0x7fcdf00b0540 2024-03-29T08:30:20.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf7e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00b0d90 con 0x7fcdf015a940 2024-03-29T08:30:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf4a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcdf015ad20 0x7fcdf00cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf4a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcdf015ad20 0x7fcdf00cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38530/0 (socket says 172.21.15.46:38530) 2024-03-29T08:30:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf4a00640 1 -- 172.21.15.46:0/287838757 learned_addr learned my addr 172.21.15.46:0/287838757 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf5e00640 1 --2- 172.21.15.46:0/287838757 >> v2:172.21.15.73:3300/0 conn(0x7fcdf00b0540 0x7fcdf00cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf5400640 1 --2- 172.21.15.46:0/287838757 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdf015a940 0x7fcdf00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf4a00640 1 -- 172.21.15.46:0/287838757 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdf015a940 msgr2=0x7fcdf00afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf4a00640 1 --2- 172.21.15.46:0/287838757 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdf015a940 0x7fcdf00afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf4a00640 1 -- 172.21.15.46:0/287838757 >> v2:172.21.15.73:3300/0 conn(0x7fcdf00b0540 msgr2=0x7fcdf00cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf4a00640 1 --2- 172.21.15.46:0/287838757 >> v2:172.21.15.73:3300/0 conn(0x7fcdf00b0540 0x7fcdf00cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.637+0000 7fcdf4a00640 1 -- 172.21.15.46:0/287838757 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf00b1070 con 0x7fcdf015ad20 2024-03-29T08:30:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.641+0000 7fcdf4a00640 1 --2- 172.21.15.46:0/287838757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf015ad20 0x7fcdf00cecb0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00c7720 tx=0x7fcddc01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb3787e828ae557 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.641+0000 7fcdefe00640 1 -- 172.21.15.46:0/287838757 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc0a7070 con 0x7fcdf015ad20 2024-03-29T08:30:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.641+0000 7fcdefe00640 1 -- 172.21.15.46:0/287838757 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcddc0ab430 con 0x7fcdf015ad20 2024-03-29T08:30:20.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.641+0000 7fcdefe00640 1 -- 172.21.15.46:0/287838757 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc0ab7c0 con 0x7fcdf015ad20 2024-03-29T08:30:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.641+0000 7fcdf7e00640 1 -- 172.21.15.46:0/287838757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf015ad20 msgr2=0x7fcdf00cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:20.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.641+0000 7fcdf7e00640 1 --2- 172.21.15.46:0/287838757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf015ad20 0x7fcdf00cecb0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00c7720 tx=0x7fcddc01c4c0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.641+0000 7fcdf7e00640 1 -- 172.21.15.46:0/287838757 shutdown_connections 2024-03-29T08:30:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.641+0000 7fcdf7e00640 1 --2- 172.21.15.46:0/287838757 >> v2:172.21.15.73:3300/0 conn(0x7fcdf00b0540 0x7fcdf00cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.641+0000 7fcdf7e00640 1 --2- 172.21.15.46:0/287838757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf015ad20 0x7fcdf00cecb0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.641+0000 7fcdf7e00640 1 --2- 172.21.15.46:0/287838757 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdf015a940 0x7fcdf00afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.641+0000 7fcdf7e00640 1 -- 172.21.15.46:0/287838757 >> 172.21.15.46:0/287838757 conn(0x7fcdf0089fd0 msgr2=0x7fcdf008a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.641+0000 7fcdf7e00640 1 -- 172.21.15.46:0/287838757 shutdown_connections 2024-03-29T08:30:20.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.641+0000 7fcdf7e00640 1 -- 172.21.15.46:0/287838757 wait complete. 2024-03-29T08:30:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.641+0000 7fcdf7e00640 1 Processor -- start 2024-03-29T08:30:20.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf7e00640 1 -- start start 2024-03-29T08:30:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf00b0540 0x7fcdf00c4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf7e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdf015a940 0x7fcdf0154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf7e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdf015ad20 0x7fcdf015b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf7e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00adc90 con 0x7fcdf00b0540 2024-03-29T08:30:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf7e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00c4740 con 0x7fcdf015ad20 2024-03-29T08:30:20.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf7e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0158fa0 con 0x7fcdf015a940 2024-03-29T08:30:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf00b0540 0x7fcdf00c4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf5400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf00b0540 0x7fcdf00c4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38536/0 (socket says 172.21.15.46:38536) 2024-03-29T08:30:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf5400640 1 -- 172.21.15.46:0/2554584149 learned_addr learned my addr 172.21.15.46:0/2554584149 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf4a00640 1 --2- 172.21.15.46:0/2554584149 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdf015a940 0x7fcdf0154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf5e00640 1 --2- 172.21.15.46:0/2554584149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdf015ad20 0x7fcdf015b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf5400640 1 -- 172.21.15.46:0/2554584149 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdf015a940 msgr2=0x7fcdf0154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf5400640 1 --2- 172.21.15.46:0/2554584149 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdf015a940 0x7fcdf0154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf5400640 1 -- 172.21.15.46:0/2554584149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdf015ad20 msgr2=0x7fcdf015b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf5400640 1 --2- 172.21.15.46:0/2554584149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdf015ad20 0x7fcdf015b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf5400640 1 -- 172.21.15.46:0/2554584149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc0094d0 con 0x7fcdf00b0540 2024-03-29T08:30:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf5e00640 1 --2- 172.21.15.46:0/2554584149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdf015ad20 0x7fcdf015b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf5400640 1 --2- 172.21.15.46:0/2554584149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf00b0540 0x7fcdf00c4200 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fcde400d830 tx=0x7fcde400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcded600640 1 -- 172.21.15.46:0/2554584149 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde400b6c0 con 0x7fcdf00b0540 2024-03-29T08:30:20.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcded600640 1 -- 172.21.15.46:0/2554584149 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde400b820 con 0x7fcdf00b0540 2024-03-29T08:30:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcded600640 1 -- 172.21.15.46:0/2554584149 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde400bb90 con 0x7fcdf00b0540 2024-03-29T08:30:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf7e00640 1 -- 172.21.15.46:0/2554584149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf0159280 con 0x7fcdf00b0540 2024-03-29T08:30:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf7e00640 1 -- 172.21.15.46:0/2554584149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf0155d90 con 0x7fcdf00b0540 2024-03-29T08:30:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.645+0000 7fcdf7e00640 1 -- 172.21.15.46:0/2554584149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf0066c90 con 0x7fcdf00b0540 2024-03-29T08:30:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.649+0000 7fcded600640 1 -- 172.21.15.46:0/2554584149 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcde401e460 con 0x7fcdf00b0540 2024-03-29T08:30:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.649+0000 7fcded600640 1 --2- 172.21.15.46:0/2554584149 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcdd0066690 0x7fcdd0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.649+0000 7fcded600640 1 -- 172.21.15.46:0/2554584149 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fcde40845b0 con 0x7fcdf00b0540 2024-03-29T08:30:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.649+0000 7fcdf4a00640 1 --2- 172.21.15.46:0/2554584149 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcdd0066690 0x7fcdd0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:20.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.649+0000 7fcdf4a00640 1 --2- 172.21.15.46:0/2554584149 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcdd0066690 0x7fcdd0068b50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0af000 tx=0x7fcddc0a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:20.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.653+0000 7fcded600640 1 -- 172.21.15.46:0/2554584149 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fcde4051c00 con 0x7fcdf00b0540 2024-03-29T08:30:20.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:20.869+0000 7fcdf7e00640 1 -- 172.21.15.46:0/2554584149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7fcdf00af860 con 0x7fcdf00b0540 2024-03-29T08:30:21.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:21.113+0000 7fcded600640 1 -- 172.21.15.46:0/2554584149 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v552) v1 ==== 144+0+0 (secure 0 0 0) 0x7fcde4055220 con 0x7fcdf00b0540 2024-03-29T08:30:21.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:21.117+0000 7fcdf7e00640 1 -- 172.21.15.46:0/2554584149 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcdd0066690 msgr2=0x7fcdd0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:21.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:21.117+0000 7fcdf7e00640 1 --2- 172.21.15.46:0/2554584149 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcdd0066690 0x7fcdd0068b50 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0af000 tx=0x7fcddc0a7000 comp rx=0 tx=0).stop 2024-03-29T08:30:21.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:21.117+0000 7fcdf7e00640 1 -- 172.21.15.46:0/2554584149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf00b0540 msgr2=0x7fcdf00c4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:21.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:21.117+0000 7fcdf7e00640 1 --2- 172.21.15.46:0/2554584149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf00b0540 0x7fcdf00c4200 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fcde400d830 tx=0x7fcde400dcf0 comp rx=0 tx=0).stop 2024-03-29T08:30:21.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:21.117+0000 7fcdf7e00640 1 -- 172.21.15.46:0/2554584149 shutdown_connections 2024-03-29T08:30:21.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:21.117+0000 7fcdf7e00640 1 --2- 172.21.15.46:0/2554584149 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcdd0066690 0x7fcdd0068b50 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:21.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:21.117+0000 7fcdf7e00640 1 --2- 172.21.15.46:0/2554584149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdf015ad20 0x7fcdf015b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:21.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:21.117+0000 7fcdf7e00640 1 --2- 172.21.15.46:0/2554584149 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcdf015a940 0x7fcdf0154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:21.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:21.117+0000 7fcdf7e00640 1 --2- 172.21.15.46:0/2554584149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcdf00b0540 0x7fcdf00c4200 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:21.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:21.117+0000 7fcdf7e00640 1 -- 172.21.15.46:0/2554584149 >> 172.21.15.46:0/2554584149 conn(0x7fcdf0089fd0 msgr2=0x7fcdf0159b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:21.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:21.117+0000 7fcdf7e00640 1 -- 172.21.15.46:0/2554584149 shutdown_connections 2024-03-29T08:30:21.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:21.117+0000 7fcdf7e00640 1 -- 172.21.15.46:0/2554584149 wait complete. 2024-03-29T08:30:21.137 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-03-29T08:30:21.138 INFO:tasks.cephfs_test_runner: 2024-03-29T08:30:21.138 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-29T08:30:21.138 INFO:tasks.cephfs_test_runner:Ran 7 tests in 696.761s 2024-03-29T08:30:21.138 INFO:tasks.cephfs_test_runner: 2024-03-29T08:30:21.138 INFO:tasks.cephfs_test_runner:OK 2024-03-29T08:30:21.138 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-29T08:30:21.148 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-29T08:30:21.155 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-29T08:30:21.156 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:30:21.156 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:30:21.165 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:30:21.165 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-03-29T08:30:21.165 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:30:21.216 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357712k,nr_inodes=4089428,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1723 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275992k,nr_inodes=818998,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275996k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-03-29T08:30:21.216 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:30:21.216 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:30:21.226 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:30:21.226 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-03-29T08:30:21.226 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-29T08:30:21.278 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354552k,nr_inodes=4088638,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=1010 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275360k,nr_inodes=818840,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275364k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-03-29T08:30:21.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:21.278 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-03-29T08:30:21.330 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-03-29T08:30:21.331 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-03-29T08:30:21.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:21.332 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-03-29T08:30:21.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:21.396 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-03-29T08:30:21.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:21.483 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link delete ceph-brx 2024-03-29T08:30:21.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:21.662 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-03-29T08:30:21.672 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-29T08:30:21.672 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-03-29T08:30:21.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:21.672 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-29T08:30:21.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:21.738 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-03-29T08:30:21.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-29T08:30:21.827 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-03-29T08:30:21.982 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-29T08:30:21.991 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-29T08:30:21.991 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:30:22.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.133+0000 7f57fe800640 1 Processor -- start 2024-03-29T08:30:22.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.133+0000 7f57fe800640 1 -- start start 2024-03-29T08:30:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.137+0000 7f57fe800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57f815cd90 0x7f57f806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.137+0000 7f57fe800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f57f815d940 0x7f57f81597b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.137+0000 7f57fe800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f806d130 con 0x7f57f81593d0 2024-03-29T08:30:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.137+0000 7f57fe800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f815b8f0 con 0x7f57f815d940 2024-03-29T08:30:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.137+0000 7f57fe800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f815ad50 con 0x7f57f815cd90 2024-03-29T08:30:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.137+0000 7f57f7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f57f815d940 0x7f57f81597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.137+0000 7f57f7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f57f815d940 0x7f57f81597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38538/0 (socket says 172.21.15.46:38538) 2024-03-29T08:30:22.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.137+0000 7f57f7e00640 1 -- 172.21.15.46:0/3351276349 learned_addr learned my addr 172.21.15.46:0/3351276349 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:22.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.137+0000 7f57eea00640 1 --2- 172.21.15.46:0/3351276349 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57f815cd90 0x7f57f806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.137+0000 7f57f7e00640 1 -- 172.21.15.46:0/3351276349 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57f815cd90 msgr2=0x7f57f806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:22.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.137+0000 7f57f7e00640 1 --2- 172.21.15.46:0/3351276349 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57f815cd90 0x7f57f806cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.137+0000 7f57f7e00640 1 -- 172.21.15.46:0/3351276349 >> v1:172.21.15.73:6789/0 conn(0x7f57f81593d0 legacy=0x7f57f815d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:22.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.137+0000 7f57f7e00640 1 -- 172.21.15.46:0/3351276349 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f815afc0 con 0x7f57f815d940 2024-03-29T08:30:22.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.137+0000 7f57eea00640 1 --2- 172.21.15.46:0/3351276349 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57f815cd90 0x7f57f806cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.141+0000 7f57f7e00640 1 --2- 172.21.15.46:0/3351276349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57f815d940 0x7f57f81597b0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009e50 tx=0x7f57f001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bb228c74ccb4c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.141+0000 7f57f6a00640 1 -- 172.21.15.46:0/3351276349 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f00a5900 con 0x7f57f815d940 2024-03-29T08:30:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.141+0000 7f57f6a00640 1 -- 172.21.15.46:0/3351276349 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f00a5a60 con 0x7f57f815d940 2024-03-29T08:30:22.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.141+0000 7f57f6a00640 1 -- 172.21.15.46:0/3351276349 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f00a5df0 con 0x7f57f815d940 2024-03-29T08:30:22.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.141+0000 7f57fe800640 1 -- 172.21.15.46:0/3351276349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57f815d940 msgr2=0x7f57f81597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:22.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.141+0000 7f57fe800640 1 --2- 172.21.15.46:0/3351276349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57f815d940 0x7f57f81597b0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009e50 tx=0x7f57f001ca30 comp rx=0 tx=0).stop 2024-03-29T08:30:22.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.141+0000 7f57fe800640 1 -- 172.21.15.46:0/3351276349 shutdown_connections 2024-03-29T08:30:22.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.141+0000 7f57fe800640 1 --2- 172.21.15.46:0/3351276349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57f815d940 0x7f57f81597b0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.141+0000 7f57fe800640 1 --2- 172.21.15.46:0/3351276349 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57f815cd90 0x7f57f806cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.141+0000 7f57fe800640 1 -- 172.21.15.46:0/3351276349 >> 172.21.15.46:0/3351276349 conn(0x7f57f8066110 msgr2=0x7f57f8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:22.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.141+0000 7f57fe800640 1 -- 172.21.15.46:0/3351276349 shutdown_connections 2024-03-29T08:30:22.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.141+0000 7f57fe800640 1 -- 172.21.15.46:0/3351276349 wait complete. 2024-03-29T08:30:22.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.141+0000 7f57fe800640 1 Processor -- start 2024-03-29T08:30:22.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.141+0000 7f57fe800640 1 -- start start 2024-03-29T08:30:22.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57fe800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57f81593d0 0x7f57f813be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57fe800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57f815cd90 0x7f57f8162d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57fe800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57f815d940 0x7f57f81632b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57fe800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f814b5f0 con 0x7f57f81593d0 2024-03-29T08:30:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57fe800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f813c390 con 0x7f57f815cd90 2024-03-29T08:30:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57fe800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f81655d0 con 0x7f57f815d940 2024-03-29T08:30:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57f7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57f81593d0 0x7f57f813be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57f7400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57f81593d0 0x7f57f813be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38542/0 (socket says 172.21.15.46:38542) 2024-03-29T08:30:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57f7400640 1 -- 172.21.15.46:0/1583113954 learned_addr learned my addr 172.21.15.46:0/1583113954 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57eea00640 1 --2- 172.21.15.46:0/1583113954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57f815cd90 0x7f57f8162d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57f7e00640 1 --2- 172.21.15.46:0/1583113954 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57f815d940 0x7f57f81632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57f7400640 1 -- 172.21.15.46:0/1583113954 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57f815d940 msgr2=0x7f57f81632b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57f7400640 1 --2- 172.21.15.46:0/1583113954 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57f815d940 0x7f57f81632b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57f7400640 1 -- 172.21.15.46:0/1583113954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57f815cd90 msgr2=0x7f57f8162d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57f7400640 1 --2- 172.21.15.46:0/1583113954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57f815cd90 0x7f57f8162d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57f7400640 1 -- 172.21.15.46:0/1583113954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f0009b30 con 0x7f57f81593d0 2024-03-29T08:30:22.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57f7400640 1 --2- 172.21.15.46:0/1583113954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57f81593d0 0x7f57f813be50 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f57e800c3c0 tx=0x7f57e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:22.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57efe00640 1 -- 172.21.15.46:0/1583113954 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e80052d0 con 0x7f57f81593d0 2024-03-29T08:30:22.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57efe00640 1 -- 172.21.15.46:0/1583113954 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e8005430 con 0x7f57f81593d0 2024-03-29T08:30:22.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57efe00640 1 -- 172.21.15.46:0/1583113954 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8003840 con 0x7f57f81593d0 2024-03-29T08:30:22.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57fe800640 1 -- 172.21.15.46:0/1583113954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f8150da0 con 0x7f57f81593d0 2024-03-29T08:30:22.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57fe800640 1 -- 172.21.15.46:0/1583113954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f8165890 con 0x7f57f81593d0 2024-03-29T08:30:22.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57fe800640 1 -- 172.21.15.46:0/1583113954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f8066c90 con 0x7f57f81593d0 2024-03-29T08:30:22.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.145+0000 7f57efe00640 1 -- 172.21.15.46:0/1583113954 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f57e80039a0 con 0x7f57f81593d0 2024-03-29T08:30:22.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.149+0000 7f57efe00640 1 --2- 172.21.15.46:0/1583113954 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f57d4066690 0x7f57d4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.149+0000 7f57efe00640 1 -- 172.21.15.46:0/1583113954 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f57e8083460 con 0x7f57f81593d0 2024-03-29T08:30:22.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.149+0000 7f57eea00640 1 --2- 172.21.15.46:0/1583113954 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f57d4066690 0x7f57d4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.149+0000 7f57eea00640 1 --2- 172.21.15.46:0/1583113954 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f57d4066690 0x7f57d4068b50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f57f814ff40 tx=0x7f57e0008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:22.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.153+0000 7f57efe00640 1 -- 172.21.15.46:0/1583113954 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f57e8050a30 con 0x7f57f81593d0 2024-03-29T08:30:22.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.313+0000 7f57fe800640 1 -- 172.21.15.46:0/1583113954 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f57f81643a0 con 0x7f57d4066690 2024-03-29T08:30:22.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.313+0000 7f57efe00640 1 -- 172.21.15.46:0/1583113954 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50731 (secure 0 0 0) 0x7f57f81643a0 con 0x7f57d4066690 2024-03-29T08:30:22.315 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:22.316 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:30:22.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.317+0000 7f57fe800640 1 -- 172.21.15.46:0/1583113954 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f57d4066690 msgr2=0x7f57d4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:22.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.317+0000 7f57fe800640 1 --2- 172.21.15.46:0/1583113954 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f57d4066690 0x7f57d4068b50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f57f814ff40 tx=0x7f57e0008040 comp rx=0 tx=0).stop 2024-03-29T08:30:22.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.317+0000 7f57fe800640 1 -- 172.21.15.46:0/1583113954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57f81593d0 msgr2=0x7f57f813be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:22.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.317+0000 7f57fe800640 1 --2- 172.21.15.46:0/1583113954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57f81593d0 0x7f57f813be50 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f57e800c3c0 tx=0x7f57e800c880 comp rx=0 tx=0).stop 2024-03-29T08:30:22.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.317+0000 7f57fe800640 1 -- 172.21.15.46:0/1583113954 shutdown_connections 2024-03-29T08:30:22.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.317+0000 7f57fe800640 1 --2- 172.21.15.46:0/1583113954 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f57d4066690 0x7f57d4068b50 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.317+0000 7f57fe800640 1 --2- 172.21.15.46:0/1583113954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57f815cd90 0x7f57f8162d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.317+0000 7f57fe800640 1 --2- 172.21.15.46:0/1583113954 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f57f815d940 0x7f57f81632b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.317+0000 7f57fe800640 1 --2- 172.21.15.46:0/1583113954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57f81593d0 0x7f57f813be50 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.317+0000 7f57fe800640 1 -- 172.21.15.46:0/1583113954 >> 172.21.15.46:0/1583113954 conn(0x7f57f8066110 msgr2=0x7f57f80701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:22.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.317+0000 7f57fe800640 1 -- 172.21.15.46:0/1583113954 shutdown_connections 2024-03-29T08:30:22.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.317+0000 7f57fe800640 1 -- 172.21.15.46:0/1583113954 wait complete. 2024-03-29T08:30:22.336 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-03-29T08:30:22.229432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":278796,"kb_used_data":29856,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754695924,"statfs":{"total":773094113280,"available":772808626176,"internally_reserved":0,"allocated":30572544,"data_stored":23458711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392378,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-177,"num_write_kb":-8295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":-2,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001630"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":259,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-03-29T08:30:19.389290+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:30:19.389290+0000","last_peered":"2024-03-29T08:30:19.389290+0000","last_clean":"2024-03-29T08:30:19.389290+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:30:19.389290+0000","last_undegraded":"2024-03-29T08:30:19.389290+0000","last_fullsized":"2024-03-29T08:30:19.389290+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509591,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36756,"kb_used_data":5640,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335084,"statfs":{"total":96636764160,"available":96599126016,"internally_reserved":0,"allocated":5775360,"data_stored":4881953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.72699999999999998,"15min":0.69799999999999995},"min":{"1min":0.53900000000000003,"5min":0.41899999999999998,"15min":0.34999999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.1639999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75900000000000001,"15min":0.72699999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.45800000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.2410000000000001},"last":0.82399999999999995}]},{"osd":1,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69399999999999995,"15min":0.68000000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.39600000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.202},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.505,"5min":0.48999999999999999,"15min":0.44},"max":{"1min":0.93700000000000006,"5min":0.99199999999999999,"15min":1.1339999999999999},"last":0.85899999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70299999999999996,"15min":0.67800000000000005},"min":{"1min":0.61199999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":1.085},"last":0.996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72599999999999998,"15min":0.70699999999999996},"min":{"1min":0.49099999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97599999999999998,"5min":1.006,"15min":1.288},"last":0.86899999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.56200000000000006,"5min":0.51100000000000001,"15min":0.46000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":1.018},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.73399999999999999,"15min":0.73699999999999999},"min":{"1min":0.59699999999999998,"5min":0.52300000000000002,"15min":0.48499999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.2609999999999999},"last":0.74099999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58999999999999997,"15min":0.60599999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.95299999999999996},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.65500000000000003},"min":{"1min":0.45000000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88400000000000001,"5min":0.89900000000000002,"15min":1.123},"last":1.089}]},{"osd":5,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60399999999999998,"15min":0.65300000000000002},"min":{"1min":0.40899999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85699999999999998,"5min":0.89600000000000002,"15min":1.056},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.627,"15min":0.66400000000000003},"min":{"1min":0.442,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.89800000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.93999999999999995}]},{"osd":6,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70199999999999996,"15min":0.71299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97699999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66700000000000004,"15min":0.70099999999999996},"min":{"1min":0.52600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.91400000000000003,"5min":0.93799999999999994,"15min":0.98299999999999998},"last":1.0620000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509591,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36748,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":5767168,"data_stored":4881943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72399999999999998,"15min":0.68400000000000005},"min":{"1min":0.58299999999999996,"5min":0.42699999999999999,"15min":0.38900000000000001},"max":{"1min":0.88500000000000001,"5min":0.93100000000000005,"15min":1.0189999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.748,"15min":0.72299999999999998},"min":{"1min":0.60699999999999998,"5min":0.51700000000000002,"15min":0.44500000000000001},"max":{"1min":0.90700000000000003,"5min":0.99299999999999999,"15min":1.079},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68600000000000005,"15min":0.65300000000000002},"min":{"1min":0.54100000000000004,"5min":0.45400000000000001,"15min":0.377},"max":{"1min":0.90300000000000002,"5min":0.96999999999999997,"15min":0.99299999999999999},"last":0.627},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68799999999999994,"15min":0.66500000000000004},"min":{"1min":0.53900000000000003,"5min":0.42999999999999999,"15min":0.42899999999999999},"max":{"1min":0.82799999999999996,"5min":0.95599999999999996,"15min":1.079},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76800000000000002,"15min":0.72899999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":0.92500000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.72599999999999998},"min":{"1min":0.61299999999999999,"5min":0.46400000000000002,"15min":0.435},"max":{"1min":0.93600000000000005,"5min":0.95099999999999996,"15min":0.95899999999999996},"last":0.71399999999999997}]},{"osd":3,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.50600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96899999999999997,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.55200000000000005,"5min":0.51000000000000001,"15min":0.47799999999999998},"max":{"1min":0.86599999999999999,"5min":0.94999999999999996,"15min":0.95299999999999996},"last":0.72699999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.629,"15min":0.64200000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.36499999999999999},"max":{"1min":0.95099999999999996,"5min":1.0840000000000001,"15min":1.556},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63600000000000001,"15min":0.67300000000000004},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.371},"max":{"1min":0.85099999999999998,"5min":0.996,"15min":2.0539999999999998},"last":0.74099999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.625,"15min":0.65600000000000003},"min":{"1min":0.41299999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85499999999999998,"5min":0.99399999999999999,"15min":1.0660000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.37},"max":{"1min":0.92800000000000005,"5min":1.042,"15min":1.1419999999999999},"last":0.52800000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70199999999999996,"15min":0.72099999999999997},"min":{"1min":0.49399999999999999,"5min":0.46800000000000003,"15min":0.38500000000000001},"max":{"1min":0.89200000000000002,"5min":1.03,"15min":1.03},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.68799999999999994,"15min":0.72199999999999998},"min":{"1min":0.47699999999999998,"5min":0.45400000000000001,"15min":0.44600000000000001},"max":{"1min":0.82699999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.77500000000000002}]}]},{"osd":4,"up_from":15,"seq":64424509590,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":5767168,"data_stored":4881943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.34499999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90000000000000002,"5min":1.026,"15min":2.1709999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68400000000000005,"15min":0.71299999999999997},"min":{"1min":0.40799999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84799999999999998,"5min":0.997,"15min":2.2869999999999999},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71499999999999997,"15min":0.74299999999999999},"min":{"1min":0.54800000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.871,"5min":1.0049999999999999,"15min":2.524},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.442,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":0.89100000000000001,"5min":1.3149999999999999,"15min":2.2679999999999998},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69399999999999995,"15min":0.71499999999999997},"min":{"1min":0.45000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93799999999999994,"5min":1.0720000000000001,"15min":2.363},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.745},"min":{"1min":0.49099999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.91300000000000003,"5min":0.97799999999999998,"15min":2.1880000000000002},"last":0.73399999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.751,"15min":0.75},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.46300000000000002},"max":{"1min":0.93999999999999995,"5min":1.3759999999999999,"15min":2.3029999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.752,"15min":0.78600000000000003},"min":{"1min":0.54600000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98099999999999998,"5min":1.016,"15min":2.2549999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61499999999999999,"15min":0.65400000000000003},"min":{"1min":0.48199999999999998,"5min":0.38300000000000001,"15min":0.36499999999999999},"max":{"1min":0.85799999999999998,"5min":0.998,"15min":1.0189999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.63600000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":1.0149999999999999},"last":0.46300000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.626,"15min":0.65100000000000002},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.39000000000000001},"max":{"1min":0.82299999999999995,"5min":1.091,"15min":1.091},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69399999999999995,"15min":0.71599999999999997},"min":{"1min":0.54300000000000004,"5min":0.499,"15min":0.499},"max":{"1min":0.81499999999999995,"5min":0.91500000000000004,"15min":1.1220000000000001},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69199999999999995,"15min":0.70799999999999996},"min":{"1min":0.498,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":1.1000000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71899999999999997,"15min":0.74299999999999999},"min":{"1min":0.51200000000000001,"5min":0.434,"15min":0.42899999999999999},"max":{"1min":0.85999999999999999,"5min":1.054,"15min":1.4430000000000001},"last":0.49399999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509593,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32672,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1589248,"data_stored":693393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.28000000000000003,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":0.59299999999999997,"5min":0.748,"15min":1.123},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51100000000000001,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":1.0409999999999999},"last":0.437}]},{"osd":1,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.42099999999999999,"15min":0.443},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.68100000000000005,"5min":0.76800000000000002,"15min":1.1599999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.5,"15min":0.51400000000000001},"min":{"1min":0.32600000000000001,"5min":0.32100000000000001,"15min":0.28599999999999998},"max":{"1min":0.72299999999999998,"5min":0.94499999999999995,"15min":1.4790000000000001},"last":0.41499999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.502,"15min":0.53700000000000003},"min":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.309},"max":{"1min":0.76400000000000001,"5min":0.80300000000000005,"15min":1.2529999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79000000000000004,"5min":0.83599999999999997,"15min":1.8580000000000001},"last":0.47799999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.75700000000000001,"15min":0.76900000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.42199999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.6419999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.52000000000000002,"5min":0.51500000000000001,"15min":0.45400000000000001},"max":{"1min":1.008,"5min":1.083,"15min":1.391},"last":0.82699999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0880000000000001,"5min":1.1850000000000001,"15min":1.671},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80900000000000005,"15min":0.82999999999999996},"min":{"1min":0.58699999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.073,"5min":1.1379999999999999,"15min":1.7110000000000001},"last":0.81100000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.77900000000000003,"15min":0.79300000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97899999999999998,"5min":1.232,"15min":1.669},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76900000000000002,"15min":0.79100000000000004},"min":{"1min":0.54600000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.0900000000000001,"5min":1.119,"15min":2.0150000000000001},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.81399999999999995},"min":{"1min":0.57399999999999995,"5min":0.50600000000000001,"15min":0.48799999999999999},"max":{"1min":1.026,"5min":1.1120000000000001,"15min":1.887},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.81100000000000005},"min":{"1min":0.57199999999999995,"5min":0.49399999999999999,"15min":0.47999999999999998},"max":{"1min":1.026,"5min":1.1539999999999999,"15min":2.044},"last":0.71999999999999997}]}]},{"osd":2,"up_from":15,"seq":64424509592,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1576960,"data_stored":687646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45600000000000002,"15min":0.46999999999999997},"min":{"1min":0.30599999999999999,"5min":0.25800000000000001,"15min":0.23899999999999999},"max":{"1min":0.58499999999999996,"5min":0.754,"15min":0.94299999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51800000000000002,"15min":0.52100000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32300000000000001},"max":{"1min":0.64000000000000001,"5min":0.749,"15min":2.0880000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.48299999999999998,"15min":0.49099999999999999},"min":{"1min":0.32600000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68400000000000005,"5min":0.82699999999999996,"15min":1.399},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.48599999999999999,"15min":0.498},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.29299999999999998},"max":{"1min":0.59899999999999998,"5min":0.83299999999999996,"15min":3.0459999999999998},"last":0.36399999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51200000000000001,"15min":0.51600000000000001},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.747,"5min":0.79900000000000004,"15min":2.1280000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58399999999999996,"15min":0.59299999999999997},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.35999999999999999},"max":{"1min":0.76300000000000001,"5min":0.88800000000000001,"15min":2.9950000000000001},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.78500000000000003,"15min":0.77900000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.49099999999999999},"max":{"1min":1.087,"5min":1.109,"15min":1.4239999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.78800000000000003,"15min":0.76700000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.435},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.254}]},{"osd":5,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.85099999999999998,"15min":0.83899999999999997},"min":{"1min":0.60999999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.179,"5min":1.179,"15min":1.4890000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.81499999999999995,"15min":0.82199999999999995},"min":{"1min":0.53700000000000003,"5min":0.52300000000000002,"15min":0.48799999999999999},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.526},"last":1.3540000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79100000000000004,"15min":0.78100000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.48599999999999999},"max":{"1min":1.0289999999999999,"5min":1.167,"15min":2.0470000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82099999999999995,"15min":0.79800000000000004},"min":{"1min":0.57299999999999995,"5min":0.54800000000000004,"15min":0.52500000000000002},"max":{"1min":0.97399999999999998,"5min":1.2609999999999999,"15min":1.524},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.79900000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.319},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.80100000000000005,"15min":0.79500000000000004},"min":{"1min":0.60599999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.96999999999999997,"5min":1.214,"15min":2},"last":0.629}]}]},{"osd":1,"up_from":15,"seq":64424509592,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2166784,"data_stored":1278007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.44,"15min":0.51600000000000001},"min":{"1min":0.249,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.67600000000000005,"5min":0.69699999999999995,"15min":4.2539999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.46999999999999997,"15min":0.54800000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.627,"5min":0.73399999999999999,"15min":4.1909999999999998},"last":0.318}]},{"osd":2,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.49399999999999999,"15min":0.498},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.66700000000000004,"5min":0.71099999999999997,"15min":0.82899999999999996},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.51900000000000002,"15min":0.52900000000000003},"min":{"1min":0.39900000000000002,"5min":0.35099999999999998,"15min":0.33600000000000002},"max":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.88200000000000001},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56699999999999995,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71999999999999997,"5min":0.88300000000000001,"15min":1.52},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.52300000000000002},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67100000000000004,"5min":0.94499999999999995,"15min":1.331},"last":0.42899999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77700000000000002,"15min":0.75600000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46200000000000002},"max":{"1min":0.89200000000000002,"5min":1.157,"15min":1.157},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.55400000000000005,"5min":0.502,"15min":0.47999999999999998},"max":{"1min":1.0069999999999999,"5min":1.212,"15min":1.23},"last":0.67600000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78700000000000003,"15min":0.77800000000000002},"min":{"1min":0.58199999999999996,"5min":0.503,"15min":0.503},"max":{"1min":0.91400000000000003,"5min":1.258,"15min":1.258},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77900000000000003,"15min":0.76900000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.46500000000000002},"max":{"1min":0.93899999999999995,"5min":1.1879999999999999,"15min":1.2509999999999999},"last":0.71199999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.77100000000000002,"15min":0.77200000000000002},"min":{"1min":0.56799999999999995,"5min":0.54900000000000004,"15min":0.5},"max":{"1min":0.96899999999999997,"5min":1.268,"15min":1.268},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.79500000000000004,"15min":0.81699999999999995},"min":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51300000000000001},"max":{"1min":1.0129999999999999,"5min":1.22,"15min":1.22},"last":0.77500000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.90600000000000003,"5min":1.2410000000000001,"15min":1.3089999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.59299999999999997,"5min":0.503,"15min":0.503},"max":{"1min":0.96299999999999997,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.84899999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509591,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36752,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335088,"statfs":{"total":96636764160,"available":96599130112,"internally_reserved":0,"allocated":5767168,"data_stored":4881943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.44,"15min":0.504},"min":{"1min":0.27800000000000002,"5min":0.27000000000000002,"15min":0.22800000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":3.9359999999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.49199999999999999,"15min":0.63700000000000001},"min":{"1min":0.33700000000000002,"5min":0.28499999999999998,"15min":0.253},"max":{"1min":0.66900000000000004,"5min":0.77000000000000002,"15min":8.0419999999999998},"last":0.5}]},{"osd":2,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.503,"15min":0.52300000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64600000000000002,"5min":0.78000000000000003,"15min":1.258},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53400000000000003,"15min":0.61499999999999999},"min":{"1min":0.38700000000000001,"5min":0.32800000000000001,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":0.81399999999999995,"15min":4.907},"last":0.439}]},{"osd":3,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49399999999999999,"15min":0.63900000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.28499999999999998},"max":{"1min":0.66600000000000004,"5min":0.83399999999999996,"15min":7.9569999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57599999999999996,"15min":0.60599999999999998},"min":{"1min":0.39600000000000002,"5min":0.36099999999999999,"15min":0.34799999999999998},"max":{"1min":0.72599999999999998,"5min":0.87,"15min":1.397},"last":0.41899999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.78900000000000003,"15min":0.86899999999999999},"min":{"1min":0.54200000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.282,"5min":1.282,"15min":4.4720000000000004},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78700000000000003,"15min":0.85099999999999998},"min":{"1min":0.499,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97999999999999998,"5min":1.153,"15min":4.3920000000000003},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78700000000000003,"15min":0.84399999999999997},"min":{"1min":0.52800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.24,"5min":1.24,"15min":4.3200000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78600000000000003,"15min":0.81699999999999995},"min":{"1min":0.58399999999999996,"5min":0.56000000000000005,"15min":0.54600000000000004},"max":{"1min":1.0680000000000001,"5min":1.107,"15min":1.6759999999999999},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79200000000000004,"15min":0.81999999999999995},"min":{"1min":0.57299999999999995,"5min":0.55800000000000005,"15min":0.54400000000000004},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.5920000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80900000000000005,"15min":0.879},"min":{"1min":0.63,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":4.4100000000000001},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81299999999999994,"15min":0.88900000000000001},"min":{"1min":0.57199999999999995,"5min":0.54100000000000004,"15min":0.496},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":4.5209999999999999},"last":0.62},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.79400000000000004,"15min":0.873},"min":{"1min":0.59099999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":4.4420000000000002},"last":0.58599999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542296,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33228,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2162688,"data_stored":1271883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68899999999999995,"15min":0.66600000000000004},"min":{"1min":0.51800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.97199999999999998,"15min":0.98499999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71899999999999997,"15min":0.68600000000000005},"min":{"1min":0.44700000000000001,"5min":0.42999999999999999,"15min":0.39800000000000002},"max":{"1min":0.81499999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.60599999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.38200000000000001},"max":{"1min":0.80700000000000005,"5min":1.008,"15min":2.5129999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73999999999999999,"15min":0.70599999999999996},"min":{"1min":0.63700000000000001,"5min":0.51300000000000001,"15min":0.40500000000000003},"max":{"1min":0.878,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70499999999999996,"15min":0.68600000000000005},"min":{"1min":0.52200000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88800000000000001,"5min":0.997,"15min":0.997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.74099999999999999,"15min":0.71799999999999997},"min":{"1min":0.56399999999999995,"5min":0.47499999999999998,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":1.03,"15min":1.03},"last":0.70099999999999996}]},{"osd":3,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.74099999999999999,"15min":0.72499999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.47099999999999997},"max":{"1min":0.92600000000000005,"5min":1.038,"15min":1.038},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.435},"max":{"1min":0.84499999999999997,"5min":1.0209999999999999,"15min":1.954},"last":0.80800000000000005}]},{"osd":4,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.628,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.93500000000000005},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.64200000000000002},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.373},"max":{"1min":0.82699999999999996,"5min":0.89200000000000002,"15min":1.0820000000000001},"last":0.88200000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.64700000000000002},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.071},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.68500000000000005},"min":{"1min":0.52000000000000002,"5min":0.45300000000000001,"15min":0.432},"max":{"1min":0.93799999999999994,"5min":0.94299999999999995,"15min":1.0609999999999999},"last":0.90000000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67300000000000004,"15min":0.69099999999999995},"min":{"1min":0.48899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88200000000000001,"5min":0.92200000000000004,"15min":1.1200000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69699999999999995,"15min":0.72499999999999998},"min":{"1min":0.53300000000000003,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.04},"last":0.77700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:30:22.338 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:30:22.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.477+0000 7f7c03000640 1 Processor -- start 2024-03-29T08:30:22.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.477+0000 7f7c03000640 1 -- start start 2024-03-29T08:30:22.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.481+0000 7f7c03000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7bfc144210 0x7f7bfc0cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.481+0000 7f7c03000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7bfc0ceb20 0x7f7bfc06c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.481+0000 7f7c03000640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bfc0cb510 con 0x7f7bfc15ada0 2024-03-29T08:30:22.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.481+0000 7f7c03000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bfc070580 con 0x7f7bfc0ceb20 2024-03-29T08:30:22.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.481+0000 7f7c03000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc070860 con 0x7f7bfc144210 2024-03-29T08:30:22.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.481+0000 7f7c01000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7bfc0ceb20 0x7f7bfc06c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.481+0000 7f7c01000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7bfc0ceb20 0x7f7bfc06c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38548/0 (socket says 172.21.15.46:38548) 2024-03-29T08:30:22.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.481+0000 7f7c01000640 1 -- 172.21.15.46:0/3189660148 learned_addr learned my addr 172.21.15.46:0/3189660148 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:22.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.481+0000 7f7bfbe00640 1 --2- 172.21.15.46:0/3189660148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7bfc144210 0x7f7bfc0cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.481+0000 7f7c01000640 1 -- 172.21.15.46:0/3189660148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7bfc144210 msgr2=0x7f7bfc0cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:22.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.481+0000 7f7c01000640 1 --2- 172.21.15.46:0/3189660148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7bfc144210 0x7f7bfc0cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.481+0000 7f7c01000640 1 -- 172.21.15.46:0/3189660148 >> v1:172.21.15.73:6789/0 conn(0x7f7bfc15ada0 legacy=0x7f7bfc0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:22.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.481+0000 7f7c01000640 1 -- 172.21.15.46:0/3189660148 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc070b40 con 0x7f7bfc0ceb20 2024-03-29T08:30:22.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.481+0000 7f7c01000640 1 --2- 172.21.15.46:0/3189660148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bfc0ceb20 0x7f7bfc06c950 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009870 tx=0x7f7bf401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8041eb00ce517ebf server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:22.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7bfb400640 1 -- 172.21.15.46:0/3189660148 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40ab670 con 0x7f7bfc0ceb20 2024-03-29T08:30:22.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7bfb400640 1 -- 172.21.15.46:0/3189660148 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf40ab7d0 con 0x7f7bfc0ceb20 2024-03-29T08:30:22.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7bfb400640 1 -- 172.21.15.46:0/3189660148 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf40abb60 con 0x7f7bfc0ceb20 2024-03-29T08:30:22.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7c03000640 1 -- 172.21.15.46:0/3189660148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bfc0ceb20 msgr2=0x7f7bfc06c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7c03000640 1 --2- 172.21.15.46:0/3189660148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bfc0ceb20 0x7f7bfc06c950 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f7bf4009870 tx=0x7f7bf401c560 comp rx=0 tx=0).stop 2024-03-29T08:30:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7c03000640 1 -- 172.21.15.46:0/3189660148 shutdown_connections 2024-03-29T08:30:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7c03000640 1 --2- 172.21.15.46:0/3189660148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bfc0ceb20 0x7f7bfc06c950 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7c03000640 1 --2- 172.21.15.46:0/3189660148 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7bfc144210 0x7f7bfc0cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7c03000640 1 -- 172.21.15.46:0/3189660148 >> 172.21.15.46:0/3189660148 conn(0x7f7bfc066110 msgr2=0x7f7bfc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7c03000640 1 -- 172.21.15.46:0/3189660148 shutdown_connections 2024-03-29T08:30:22.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7c03000640 1 -- 172.21.15.46:0/3189660148 wait complete. 2024-03-29T08:30:22.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7c03000640 1 Processor -- start 2024-03-29T08:30:22.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7c03000640 1 -- start start 2024-03-29T08:30:22.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7c03000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bfc0ceb20 0x7f7bfc141b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7c03000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bfc144210 0x7f7bfc1f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7c03000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7bfc15ada0 0x7f7bfc156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7c03000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc1f7160 con 0x7f7bfc144210 2024-03-29T08:30:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7c03000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc153c70 con 0x7f7bfc0ceb20 2024-03-29T08:30:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7c03000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc156550 con 0x7f7bfc15ada0 2024-03-29T08:30:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7bf3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bfc144210 0x7f7bfc1f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7bf3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bfc144210 0x7f7bfc1f6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38564/0 (socket says 172.21.15.46:38564) 2024-03-29T08:30:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7bf3400640 1 -- 172.21.15.46:0/2441790687 learned_addr learned my addr 172.21.15.46:0/2441790687 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7bfbe00640 1 --2- 172.21.15.46:0/2441790687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bfc0ceb20 0x7f7bfc141b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.485+0000 7f7c01000640 1 --2- 172.21.15.46:0/2441790687 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7bfc15ada0 0x7f7bfc156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.489+0000 7f7bf3400640 1 -- 172.21.15.46:0/2441790687 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7bfc15ada0 msgr2=0x7f7bfc156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.489+0000 7f7bf3400640 1 --2- 172.21.15.46:0/2441790687 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7bfc15ada0 0x7f7bfc156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.489+0000 7f7bf3400640 1 -- 172.21.15.46:0/2441790687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bfc0ceb20 msgr2=0x7f7bfc141b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.489+0000 7f7bf3400640 1 --2- 172.21.15.46:0/2441790687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bfc0ceb20 0x7f7bfc141b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.489+0000 7f7bf3400640 1 -- 172.21.15.46:0/2441790687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf40094d0 con 0x7f7bfc144210 2024-03-29T08:30:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.489+0000 7f7bf3400640 1 --2- 172.21.15.46:0/2441790687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bfc144210 0x7f7bfc1f6c20 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f7be400c3c0 tx=0x7f7be400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.489+0000 7f7bf8c00640 1 -- 172.21.15.46:0/2441790687 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be40052d0 con 0x7f7bfc144210 2024-03-29T08:30:22.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.489+0000 7f7bf8c00640 1 -- 172.21.15.46:0/2441790687 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be4005430 con 0x7f7bfc144210 2024-03-29T08:30:22.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.489+0000 7f7bf8c00640 1 -- 172.21.15.46:0/2441790687 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be4003840 con 0x7f7bfc144210 2024-03-29T08:30:22.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.489+0000 7f7c03000640 1 -- 172.21.15.46:0/2441790687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bfc142170 con 0x7f7bfc144210 2024-03-29T08:30:22.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.489+0000 7f7c03000640 1 -- 172.21.15.46:0/2441790687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bfc142730 con 0x7f7bfc144210 2024-03-29T08:30:22.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.489+0000 7f7c03000640 1 -- 172.21.15.46:0/2441790687 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bfc0ce280 con 0x7f7bfc144210 2024-03-29T08:30:22.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.489+0000 7f7bf8c00640 1 -- 172.21.15.46:0/2441790687 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f7be40039a0 con 0x7f7bfc144210 2024-03-29T08:30:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.489+0000 7f7bf8c00640 1 --2- 172.21.15.46:0/2441790687 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7bd8066690 0x7f7bd8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.489+0000 7f7bf8c00640 1 -- 172.21.15.46:0/2441790687 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f7be40833c0 con 0x7f7bfc144210 2024-03-29T08:30:22.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.493+0000 7f7bfbe00640 1 --2- 172.21.15.46:0/2441790687 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7bd8066690 0x7f7bd8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.493+0000 7f7bfbe00640 1 --2- 172.21.15.46:0/2441790687 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7bd8066690 0x7f7bd8068b50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7bec003b90 tx=0x7f7bec0098d0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:22.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.497+0000 7f7bf8c00640 1 -- 172.21.15.46:0/2441790687 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f7be4050a30 con 0x7f7bfc144210 2024-03-29T08:30:22.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.657+0000 7f7c03000640 1 -- 172.21.15.46:0/2441790687 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7bfc06d6a0 con 0x7f7bd8066690 2024-03-29T08:30:22.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.657+0000 7f7bf8c00640 1 -- 172.21.15.46:0/2441790687 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50731 (secure 0 0 0) 0x7f7bec004740 con 0x7f7bd8066690 2024-03-29T08:30:22.659 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:22.660 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:30:22.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.661+0000 7f7c03000640 1 -- 172.21.15.46:0/2441790687 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7bd8066690 msgr2=0x7f7bd8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:22.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.661+0000 7f7c03000640 1 --2- 172.21.15.46:0/2441790687 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7bd8066690 0x7f7bd8068b50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7bec003b90 tx=0x7f7bec0098d0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.661+0000 7f7c03000640 1 -- 172.21.15.46:0/2441790687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bfc144210 msgr2=0x7f7bfc1f6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:22.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.661+0000 7f7c03000640 1 --2- 172.21.15.46:0/2441790687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bfc144210 0x7f7bfc1f6c20 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f7be400c3c0 tx=0x7f7be400c880 comp rx=0 tx=0).stop 2024-03-29T08:30:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.661+0000 7f7c03000640 1 -- 172.21.15.46:0/2441790687 shutdown_connections 2024-03-29T08:30:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.661+0000 7f7c03000640 1 --2- 172.21.15.46:0/2441790687 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7bd8066690 0x7f7bd8068b50 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.661+0000 7f7c03000640 1 --2- 172.21.15.46:0/2441790687 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f7bfc15ada0 0x7f7bfc156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.661+0000 7f7c03000640 1 --2- 172.21.15.46:0/2441790687 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bfc144210 0x7f7bfc1f6c20 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.661+0000 7f7c03000640 1 --2- 172.21.15.46:0/2441790687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bfc0ceb20 0x7f7bfc141b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.661+0000 7f7c03000640 1 -- 172.21.15.46:0/2441790687 >> 172.21.15.46:0/2441790687 conn(0x7f7bfc066110 msgr2=0x7f7bfc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.661+0000 7f7c03000640 1 -- 172.21.15.46:0/2441790687 shutdown_connections 2024-03-29T08:30:22.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.661+0000 7f7c03000640 1 -- 172.21.15.46:0/2441790687 wait complete. 2024-03-29T08:30:22.681 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-03-29T08:30:22.229432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":278796,"kb_used_data":29856,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754695924,"statfs":{"total":773094113280,"available":772808626176,"internally_reserved":0,"allocated":30572544,"data_stored":23458711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392378,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-177,"num_write_kb":-8295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":-2,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001630"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":259,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-03-29T08:30:19.389290+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:30:19.389290+0000","last_peered":"2024-03-29T08:30:19.389290+0000","last_clean":"2024-03-29T08:30:19.389290+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:30:19.389290+0000","last_undegraded":"2024-03-29T08:30:19.389290+0000","last_fullsized":"2024-03-29T08:30:19.389290+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509591,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36756,"kb_used_data":5640,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335084,"statfs":{"total":96636764160,"available":96599126016,"internally_reserved":0,"allocated":5775360,"data_stored":4881953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.72699999999999998,"15min":0.69799999999999995},"min":{"1min":0.53900000000000003,"5min":0.41899999999999998,"15min":0.34999999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.1639999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75900000000000001,"15min":0.72699999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.45800000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.2410000000000001},"last":0.82399999999999995}]},{"osd":1,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69399999999999995,"15min":0.68000000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.39600000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.202},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.505,"5min":0.48999999999999999,"15min":0.44},"max":{"1min":0.93700000000000006,"5min":0.99199999999999999,"15min":1.1339999999999999},"last":0.85899999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70299999999999996,"15min":0.67800000000000005},"min":{"1min":0.61199999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":1.085},"last":0.996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72599999999999998,"15min":0.70699999999999996},"min":{"1min":0.49099999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97599999999999998,"5min":1.006,"15min":1.288},"last":0.86899999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.56200000000000006,"5min":0.51100000000000001,"15min":0.46000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":1.018},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.73399999999999999,"15min":0.73699999999999999},"min":{"1min":0.59699999999999998,"5min":0.52300000000000002,"15min":0.48499999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.2609999999999999},"last":0.74099999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58999999999999997,"15min":0.60599999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.95299999999999996},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.65500000000000003},"min":{"1min":0.45000000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88400000000000001,"5min":0.89900000000000002,"15min":1.123},"last":1.089}]},{"osd":5,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60399999999999998,"15min":0.65300000000000002},"min":{"1min":0.40899999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85699999999999998,"5min":0.89600000000000002,"15min":1.056},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.627,"15min":0.66400000000000003},"min":{"1min":0.442,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.89800000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.93999999999999995}]},{"osd":6,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70199999999999996,"15min":0.71299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97699999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66700000000000004,"15min":0.70099999999999996},"min":{"1min":0.52600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.91400000000000003,"5min":0.93799999999999994,"15min":0.98299999999999998},"last":1.0620000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509591,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36748,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":5767168,"data_stored":4881943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72399999999999998,"15min":0.68400000000000005},"min":{"1min":0.58299999999999996,"5min":0.42699999999999999,"15min":0.38900000000000001},"max":{"1min":0.88500000000000001,"5min":0.93100000000000005,"15min":1.0189999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.748,"15min":0.72299999999999998},"min":{"1min":0.60699999999999998,"5min":0.51700000000000002,"15min":0.44500000000000001},"max":{"1min":0.90700000000000003,"5min":0.99299999999999999,"15min":1.079},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68600000000000005,"15min":0.65300000000000002},"min":{"1min":0.54100000000000004,"5min":0.45400000000000001,"15min":0.377},"max":{"1min":0.90300000000000002,"5min":0.96999999999999997,"15min":0.99299999999999999},"last":0.627},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68799999999999994,"15min":0.66500000000000004},"min":{"1min":0.53900000000000003,"5min":0.42999999999999999,"15min":0.42899999999999999},"max":{"1min":0.82799999999999996,"5min":0.95599999999999996,"15min":1.079},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76800000000000002,"15min":0.72899999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":0.92500000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.72599999999999998},"min":{"1min":0.61299999999999999,"5min":0.46400000000000002,"15min":0.435},"max":{"1min":0.93600000000000005,"5min":0.95099999999999996,"15min":0.95899999999999996},"last":0.71399999999999997}]},{"osd":3,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.50600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96899999999999997,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.55200000000000005,"5min":0.51000000000000001,"15min":0.47799999999999998},"max":{"1min":0.86599999999999999,"5min":0.94999999999999996,"15min":0.95299999999999996},"last":0.72699999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.629,"15min":0.64200000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.36499999999999999},"max":{"1min":0.95099999999999996,"5min":1.0840000000000001,"15min":1.556},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63600000000000001,"15min":0.67300000000000004},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.371},"max":{"1min":0.85099999999999998,"5min":0.996,"15min":2.0539999999999998},"last":0.74099999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.625,"15min":0.65600000000000003},"min":{"1min":0.41299999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85499999999999998,"5min":0.99399999999999999,"15min":1.0660000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.37},"max":{"1min":0.92800000000000005,"5min":1.042,"15min":1.1419999999999999},"last":0.52800000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70199999999999996,"15min":0.72099999999999997},"min":{"1min":0.49399999999999999,"5min":0.46800000000000003,"15min":0.38500000000000001},"max":{"1min":0.89200000000000002,"5min":1.03,"15min":1.03},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.68799999999999994,"15min":0.72199999999999998},"min":{"1min":0.47699999999999998,"5min":0.45400000000000001,"15min":0.44600000000000001},"max":{"1min":0.82699999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.77500000000000002}]}]},{"osd":4,"up_from":15,"seq":64424509590,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":5767168,"data_stored":4881943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.34499999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90000000000000002,"5min":1.026,"15min":2.1709999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68400000000000005,"15min":0.71299999999999997},"min":{"1min":0.40799999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84799999999999998,"5min":0.997,"15min":2.2869999999999999},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71499999999999997,"15min":0.74299999999999999},"min":{"1min":0.54800000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.871,"5min":1.0049999999999999,"15min":2.524},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.442,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":0.89100000000000001,"5min":1.3149999999999999,"15min":2.2679999999999998},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69399999999999995,"15min":0.71499999999999997},"min":{"1min":0.45000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93799999999999994,"5min":1.0720000000000001,"15min":2.363},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.745},"min":{"1min":0.49099999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.91300000000000003,"5min":0.97799999999999998,"15min":2.1880000000000002},"last":0.73399999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.751,"15min":0.75},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.46300000000000002},"max":{"1min":0.93999999999999995,"5min":1.3759999999999999,"15min":2.3029999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.752,"15min":0.78600000000000003},"min":{"1min":0.54600000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98099999999999998,"5min":1.016,"15min":2.2549999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61499999999999999,"15min":0.65400000000000003},"min":{"1min":0.48199999999999998,"5min":0.38300000000000001,"15min":0.36499999999999999},"max":{"1min":0.85799999999999998,"5min":0.998,"15min":1.0189999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.63600000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":1.0149999999999999},"last":0.46300000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.626,"15min":0.65100000000000002},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.39000000000000001},"max":{"1min":0.82299999999999995,"5min":1.091,"15min":1.091},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69399999999999995,"15min":0.71599999999999997},"min":{"1min":0.54300000000000004,"5min":0.499,"15min":0.499},"max":{"1min":0.81499999999999995,"5min":0.91500000000000004,"15min":1.1220000000000001},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69199999999999995,"15min":0.70799999999999996},"min":{"1min":0.498,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":1.1000000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71899999999999997,"15min":0.74299999999999999},"min":{"1min":0.51200000000000001,"5min":0.434,"15min":0.42899999999999999},"max":{"1min":0.85999999999999999,"5min":1.054,"15min":1.4430000000000001},"last":0.49399999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509593,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32672,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1589248,"data_stored":693393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.28000000000000003,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":0.59299999999999997,"5min":0.748,"15min":1.123},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51100000000000001,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":1.0409999999999999},"last":0.437}]},{"osd":1,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.42099999999999999,"15min":0.443},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.68100000000000005,"5min":0.76800000000000002,"15min":1.1599999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.5,"15min":0.51400000000000001},"min":{"1min":0.32600000000000001,"5min":0.32100000000000001,"15min":0.28599999999999998},"max":{"1min":0.72299999999999998,"5min":0.94499999999999995,"15min":1.4790000000000001},"last":0.41499999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.502,"15min":0.53700000000000003},"min":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.309},"max":{"1min":0.76400000000000001,"5min":0.80300000000000005,"15min":1.2529999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79000000000000004,"5min":0.83599999999999997,"15min":1.8580000000000001},"last":0.47799999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.75700000000000001,"15min":0.76900000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.42199999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.6419999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.52000000000000002,"5min":0.51500000000000001,"15min":0.45400000000000001},"max":{"1min":1.008,"5min":1.083,"15min":1.391},"last":0.82699999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0880000000000001,"5min":1.1850000000000001,"15min":1.671},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80900000000000005,"15min":0.82999999999999996},"min":{"1min":0.58699999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.073,"5min":1.1379999999999999,"15min":1.7110000000000001},"last":0.81100000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.77900000000000003,"15min":0.79300000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97899999999999998,"5min":1.232,"15min":1.669},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76900000000000002,"15min":0.79100000000000004},"min":{"1min":0.54600000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.0900000000000001,"5min":1.119,"15min":2.0150000000000001},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.81399999999999995},"min":{"1min":0.57399999999999995,"5min":0.50600000000000001,"15min":0.48799999999999999},"max":{"1min":1.026,"5min":1.1120000000000001,"15min":1.887},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.81100000000000005},"min":{"1min":0.57199999999999995,"5min":0.49399999999999999,"15min":0.47999999999999998},"max":{"1min":1.026,"5min":1.1539999999999999,"15min":2.044},"last":0.71999999999999997}]}]},{"osd":2,"up_from":15,"seq":64424509592,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1576960,"data_stored":687646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45600000000000002,"15min":0.46999999999999997},"min":{"1min":0.30599999999999999,"5min":0.25800000000000001,"15min":0.23899999999999999},"max":{"1min":0.58499999999999996,"5min":0.754,"15min":0.94299999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51800000000000002,"15min":0.52100000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32300000000000001},"max":{"1min":0.64000000000000001,"5min":0.749,"15min":2.0880000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.48299999999999998,"15min":0.49099999999999999},"min":{"1min":0.32600000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68400000000000005,"5min":0.82699999999999996,"15min":1.399},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.48599999999999999,"15min":0.498},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.29299999999999998},"max":{"1min":0.59899999999999998,"5min":0.83299999999999996,"15min":3.0459999999999998},"last":0.36399999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51200000000000001,"15min":0.51600000000000001},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.747,"5min":0.79900000000000004,"15min":2.1280000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58399999999999996,"15min":0.59299999999999997},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.35999999999999999},"max":{"1min":0.76300000000000001,"5min":0.88800000000000001,"15min":2.9950000000000001},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.78500000000000003,"15min":0.77900000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.49099999999999999},"max":{"1min":1.087,"5min":1.109,"15min":1.4239999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.78800000000000003,"15min":0.76700000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.435},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.254}]},{"osd":5,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.85099999999999998,"15min":0.83899999999999997},"min":{"1min":0.60999999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.179,"5min":1.179,"15min":1.4890000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.81499999999999995,"15min":0.82199999999999995},"min":{"1min":0.53700000000000003,"5min":0.52300000000000002,"15min":0.48799999999999999},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.526},"last":1.3540000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79100000000000004,"15min":0.78100000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.48599999999999999},"max":{"1min":1.0289999999999999,"5min":1.167,"15min":2.0470000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82099999999999995,"15min":0.79800000000000004},"min":{"1min":0.57299999999999995,"5min":0.54800000000000004,"15min":0.52500000000000002},"max":{"1min":0.97399999999999998,"5min":1.2609999999999999,"15min":1.524},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.79900000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.319},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.80100000000000005,"15min":0.79500000000000004},"min":{"1min":0.60599999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.96999999999999997,"5min":1.214,"15min":2},"last":0.629}]}]},{"osd":1,"up_from":15,"seq":64424509592,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2166784,"data_stored":1278007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.44,"15min":0.51600000000000001},"min":{"1min":0.249,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.67600000000000005,"5min":0.69699999999999995,"15min":4.2539999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.46999999999999997,"15min":0.54800000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.627,"5min":0.73399999999999999,"15min":4.1909999999999998},"last":0.318}]},{"osd":2,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.49399999999999999,"15min":0.498},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.66700000000000004,"5min":0.71099999999999997,"15min":0.82899999999999996},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.51900000000000002,"15min":0.52900000000000003},"min":{"1min":0.39900000000000002,"5min":0.35099999999999998,"15min":0.33600000000000002},"max":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.88200000000000001},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56699999999999995,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71999999999999997,"5min":0.88300000000000001,"15min":1.52},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.52300000000000002},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67100000000000004,"5min":0.94499999999999995,"15min":1.331},"last":0.42899999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77700000000000002,"15min":0.75600000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46200000000000002},"max":{"1min":0.89200000000000002,"5min":1.157,"15min":1.157},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.55400000000000005,"5min":0.502,"15min":0.47999999999999998},"max":{"1min":1.0069999999999999,"5min":1.212,"15min":1.23},"last":0.67600000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78700000000000003,"15min":0.77800000000000002},"min":{"1min":0.58199999999999996,"5min":0.503,"15min":0.503},"max":{"1min":0.91400000000000003,"5min":1.258,"15min":1.258},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77900000000000003,"15min":0.76900000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.46500000000000002},"max":{"1min":0.93899999999999995,"5min":1.1879999999999999,"15min":1.2509999999999999},"last":0.71199999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.77100000000000002,"15min":0.77200000000000002},"min":{"1min":0.56799999999999995,"5min":0.54900000000000004,"15min":0.5},"max":{"1min":0.96899999999999997,"5min":1.268,"15min":1.268},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.79500000000000004,"15min":0.81699999999999995},"min":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51300000000000001},"max":{"1min":1.0129999999999999,"5min":1.22,"15min":1.22},"last":0.77500000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.90600000000000003,"5min":1.2410000000000001,"15min":1.3089999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.59299999999999997,"5min":0.503,"15min":0.503},"max":{"1min":0.96299999999999997,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.84899999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509591,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36752,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335088,"statfs":{"total":96636764160,"available":96599130112,"internally_reserved":0,"allocated":5767168,"data_stored":4881943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.44,"15min":0.504},"min":{"1min":0.27800000000000002,"5min":0.27000000000000002,"15min":0.22800000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":3.9359999999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.49199999999999999,"15min":0.63700000000000001},"min":{"1min":0.33700000000000002,"5min":0.28499999999999998,"15min":0.253},"max":{"1min":0.66900000000000004,"5min":0.77000000000000002,"15min":8.0419999999999998},"last":0.5}]},{"osd":2,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.503,"15min":0.52300000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64600000000000002,"5min":0.78000000000000003,"15min":1.258},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53400000000000003,"15min":0.61499999999999999},"min":{"1min":0.38700000000000001,"5min":0.32800000000000001,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":0.81399999999999995,"15min":4.907},"last":0.439}]},{"osd":3,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49399999999999999,"15min":0.63900000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.28499999999999998},"max":{"1min":0.66600000000000004,"5min":0.83399999999999996,"15min":7.9569999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57599999999999996,"15min":0.60599999999999998},"min":{"1min":0.39600000000000002,"5min":0.36099999999999999,"15min":0.34799999999999998},"max":{"1min":0.72599999999999998,"5min":0.87,"15min":1.397},"last":0.41899999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.78900000000000003,"15min":0.86899999999999999},"min":{"1min":0.54200000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.282,"5min":1.282,"15min":4.4720000000000004},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78700000000000003,"15min":0.85099999999999998},"min":{"1min":0.499,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97999999999999998,"5min":1.153,"15min":4.3920000000000003},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78700000000000003,"15min":0.84399999999999997},"min":{"1min":0.52800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.24,"5min":1.24,"15min":4.3200000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78600000000000003,"15min":0.81699999999999995},"min":{"1min":0.58399999999999996,"5min":0.56000000000000005,"15min":0.54600000000000004},"max":{"1min":1.0680000000000001,"5min":1.107,"15min":1.6759999999999999},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79200000000000004,"15min":0.81999999999999995},"min":{"1min":0.57299999999999995,"5min":0.55800000000000005,"15min":0.54400000000000004},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.5920000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80900000000000005,"15min":0.879},"min":{"1min":0.63,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":4.4100000000000001},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81299999999999994,"15min":0.88900000000000001},"min":{"1min":0.57199999999999995,"5min":0.54100000000000004,"15min":0.496},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":4.5209999999999999},"last":0.62},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.79400000000000004,"15min":0.873},"min":{"1min":0.59099999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":4.4420000000000002},"last":0.58599999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542296,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33228,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2162688,"data_stored":1271883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68899999999999995,"15min":0.66600000000000004},"min":{"1min":0.51800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.97199999999999998,"15min":0.98499999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71899999999999997,"15min":0.68600000000000005},"min":{"1min":0.44700000000000001,"5min":0.42999999999999999,"15min":0.39800000000000002},"max":{"1min":0.81499999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.60599999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.38200000000000001},"max":{"1min":0.80700000000000005,"5min":1.008,"15min":2.5129999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73999999999999999,"15min":0.70599999999999996},"min":{"1min":0.63700000000000001,"5min":0.51300000000000001,"15min":0.40500000000000003},"max":{"1min":0.878,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70499999999999996,"15min":0.68600000000000005},"min":{"1min":0.52200000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88800000000000001,"5min":0.997,"15min":0.997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.74099999999999999,"15min":0.71799999999999997},"min":{"1min":0.56399999999999995,"5min":0.47499999999999998,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":1.03,"15min":1.03},"last":0.70099999999999996}]},{"osd":3,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.74099999999999999,"15min":0.72499999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.47099999999999997},"max":{"1min":0.92600000000000005,"5min":1.038,"15min":1.038},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.435},"max":{"1min":0.84499999999999997,"5min":1.0209999999999999,"15min":1.954},"last":0.80800000000000005}]},{"osd":4,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.628,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.93500000000000005},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.64200000000000002},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.373},"max":{"1min":0.82699999999999996,"5min":0.89200000000000002,"15min":1.0820000000000001},"last":0.88200000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.64700000000000002},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.071},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.68500000000000005},"min":{"1min":0.52000000000000002,"5min":0.45300000000000001,"15min":0.432},"max":{"1min":0.93799999999999994,"5min":0.94299999999999995,"15min":1.0609999999999999},"last":0.90000000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67300000000000004,"15min":0.69099999999999995},"min":{"1min":0.48899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88200000000000001,"5min":0.92200000000000004,"15min":1.1200000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69699999999999995,"15min":0.72499999999999998},"min":{"1min":0.53300000000000003,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.04},"last":0.77700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:30:22.682 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-29T08:30:22.682 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:30:22.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.821+0000 7f0ebb200640 1 Processor -- start 2024-03-29T08:30:22.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.821+0000 7f0ebb200640 1 -- start start 2024-03-29T08:30:22.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.825+0000 7f0ebb200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0eb415a890 0x7f0eb4098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.825+0000 7f0ebb200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0eb40b2b30 0x7f0eb4098f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.825+0000 7f0ebb200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb40cc580 con 0x7f0eb40994c0 2024-03-29T08:30:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.825+0000 7f0ebb200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb40cce20 con 0x7f0eb415a890 2024-03-29T08:30:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.825+0000 7f0ebb200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb40cc860 con 0x7f0eb40b2b30 2024-03-29T08:30:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.825+0000 7f0eb9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0eb415a890 0x7f0eb4098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.825+0000 7f0eb9e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0eb415a890 0x7f0eb4098a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38574/0 (socket says 172.21.15.46:38574) 2024-03-29T08:30:22.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.825+0000 7f0eb9e00640 1 -- 172.21.15.46:0/3260047873 learned_addr learned my addr 172.21.15.46:0/3260047873 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.825+0000 7f0eb9400640 1 --2- 172.21.15.46:0/3260047873 >> v2:172.21.15.73:3300/0 conn(0x7f0eb40b2b30 0x7f0eb4098f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.825+0000 7f0eb9e00640 1 -- 172.21.15.46:0/3260047873 >> v2:172.21.15.73:3300/0 conn(0x7f0eb40b2b30 msgr2=0x7f0eb4098f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.825+0000 7f0eb9e00640 1 --2- 172.21.15.46:0/3260047873 >> v2:172.21.15.73:3300/0 conn(0x7f0eb40b2b30 0x7f0eb4098f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.825+0000 7f0eb9e00640 1 -- 172.21.15.46:0/3260047873 >> v1:172.21.15.73:6789/0 conn(0x7f0eb40994c0 legacy=0x7f0eb40cb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.825+0000 7f0eb9e00640 1 -- 172.21.15.46:0/3260047873 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb40ccb40 con 0x7f0eb415a890 2024-03-29T08:30:22.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.825+0000 7f0eb9400640 1 --2- 172.21.15.46:0/3260047873 >> v2:172.21.15.73:3300/0 conn(0x7f0eb40b2b30 0x7f0eb4098f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.829+0000 7f0eb9e00640 1 --2- 172.21.15.46:0/3260047873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb415a890 0x7f0eb4098a40 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40716a0 tx=0x7f0ea801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7033b7e2e6dc7b server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.829+0000 7f0eb8a00640 1 -- 172.21.15.46:0/3260047873 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea80a7070 con 0x7f0eb415a890 2024-03-29T08:30:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.829+0000 7f0eb8a00640 1 -- 172.21.15.46:0/3260047873 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea80ab430 con 0x7f0eb415a890 2024-03-29T08:30:22.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.829+0000 7f0eb8a00640 1 -- 172.21.15.46:0/3260047873 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea80ab7c0 con 0x7f0eb415a890 2024-03-29T08:30:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.829+0000 7f0ebb200640 1 -- 172.21.15.46:0/3260047873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb415a890 msgr2=0x7f0eb4098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.829+0000 7f0ebb200640 1 --2- 172.21.15.46:0/3260047873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb415a890 0x7f0eb4098a40 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f0eb40716a0 tx=0x7f0ea801c490 comp rx=0 tx=0).stop 2024-03-29T08:30:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.829+0000 7f0ebb200640 1 -- 172.21.15.46:0/3260047873 shutdown_connections 2024-03-29T08:30:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.829+0000 7f0ebb200640 1 --2- 172.21.15.46:0/3260047873 >> v2:172.21.15.73:3300/0 conn(0x7f0eb40b2b30 0x7f0eb4098f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.829+0000 7f0ebb200640 1 --2- 172.21.15.46:0/3260047873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb415a890 0x7f0eb4098a40 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.829+0000 7f0ebb200640 1 -- 172.21.15.46:0/3260047873 >> 172.21.15.46:0/3260047873 conn(0x7f0eb40b2010 msgr2=0x7f0eb40b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:22.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.829+0000 7f0ebb200640 1 -- 172.21.15.46:0/3260047873 shutdown_connections 2024-03-29T08:30:22.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.829+0000 7f0ebb200640 1 -- 172.21.15.46:0/3260047873 wait complete. 2024-03-29T08:30:22.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.829+0000 7f0ebb200640 1 Processor -- start 2024-03-29T08:30:22.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.829+0000 7f0ebb200640 1 -- start start 2024-03-29T08:30:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0ebb200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40994c0 0x7f0eb406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0ebb200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0eb40b2b30 0x7f0eb4151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0ebb200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb415a890 0x7f0eb406c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0ebb200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40c7fe0 con 0x7f0eb415a890 2024-03-29T08:30:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0ebb200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb4158140 con 0x7f0eb40994c0 2024-03-29T08:30:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0ebb200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40b3010 con 0x7f0eb40b2b30 2024-03-29T08:30:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0eba800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb415a890 0x7f0eb406c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0eba800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb415a890 0x7f0eb406c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38578/0 (socket says 172.21.15.46:38578) 2024-03-29T08:30:22.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0eba800640 1 -- 172.21.15.46:0/392091429 learned_addr learned my addr 172.21.15.46:0/392091429 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0eb9e00640 1 --2- 172.21.15.46:0/392091429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40994c0 0x7f0eb406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0eb9400640 1 --2- 172.21.15.46:0/392091429 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0eb40b2b30 0x7f0eb4151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0eba800640 1 -- 172.21.15.46:0/392091429 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0eb40b2b30 msgr2=0x7f0eb4151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0eba800640 1 --2- 172.21.15.46:0/392091429 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0eb40b2b30 0x7f0eb4151f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0eba800640 1 -- 172.21.15.46:0/392091429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40994c0 msgr2=0x7f0eb406f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0eba800640 1 --2- 172.21.15.46:0/392091429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40994c0 0x7f0eb406f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0eba800640 1 -- 172.21.15.46:0/392091429 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea80094d0 con 0x7f0eb415a890 2024-03-29T08:30:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0eba800640 1 --2- 172.21.15.46:0/392091429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb415a890 0x7f0eb406c190 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000c760 tx=0x7f0eb000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0eae000640 1 -- 172.21.15.46:0/392091429 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb0003b80 con 0x7f0eb415a890 2024-03-29T08:30:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0eae000640 1 -- 172.21.15.46:0/392091429 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb0003ce0 con 0x7f0eb415a890 2024-03-29T08:30:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0eae000640 1 -- 172.21.15.46:0/392091429 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb000aa30 con 0x7f0eb415a890 2024-03-29T08:30:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0ebb200640 1 -- 172.21.15.46:0/392091429 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb406ff10 con 0x7f0eb415a890 2024-03-29T08:30:22.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0ebb200640 1 -- 172.21.15.46:0/392091429 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb40c8230 con 0x7f0eb415a890 2024-03-29T08:30:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0ebb200640 1 -- 172.21.15.46:0/392091429 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb404e040 con 0x7f0eb415a890 2024-03-29T08:30:22.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.833+0000 7f0eae000640 1 -- 172.21.15.46:0/392091429 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0eb000ab90 con 0x7f0eb415a890 2024-03-29T08:30:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.837+0000 7f0eae000640 1 --2- 172.21.15.46:0/392091429 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e980666c0 0x7f0e98068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.837+0000 7f0eae000640 1 -- 172.21.15.46:0/392091429 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f0eb0083d30 con 0x7f0eb415a890 2024-03-29T08:30:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.837+0000 7f0eb9e00640 1 --2- 172.21.15.46:0/392091429 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e980666c0 0x7f0e98068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:22.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.837+0000 7f0eb9e00640 1 --2- 172.21.15.46:0/392091429 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e980666c0 0x7f0e98068b80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80af000 tx=0x7f0ea80a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:22.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.841+0000 7f0eae000640 1 -- 172.21.15.46:0/392091429 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0eb00512f0 con 0x7f0eb415a890 2024-03-29T08:30:22.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:22.997+0000 7f0ebb200640 1 -- 172.21.15.46:0/392091429 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0eb4158280 con 0x7f0e980666c0 2024-03-29T08:30:23.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.001+0000 7f0eae000640 1 -- 172.21.15.46:0/392091429 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50731 (secure 0 0 0) 0x7f0eb4158280 con 0x7f0e980666c0 2024-03-29T08:30:23.001 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:23.002 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:30:23.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.001+0000 7f0ebb200640 1 -- 172.21.15.46:0/392091429 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e980666c0 msgr2=0x7f0e98068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:23.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.001+0000 7f0ebb200640 1 --2- 172.21.15.46:0/392091429 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e980666c0 0x7f0e98068b80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80af000 tx=0x7f0ea80a7000 comp rx=0 tx=0).stop 2024-03-29T08:30:23.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.001+0000 7f0ebb200640 1 -- 172.21.15.46:0/392091429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb415a890 msgr2=0x7f0eb406c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:23.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.001+0000 7f0ebb200640 1 --2- 172.21.15.46:0/392091429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb415a890 0x7f0eb406c190 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f0eb000c760 tx=0x7f0eb000cc20 comp rx=0 tx=0).stop 2024-03-29T08:30:23.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.001+0000 7f0ebb200640 1 -- 172.21.15.46:0/392091429 shutdown_connections 2024-03-29T08:30:23.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.001+0000 7f0ebb200640 1 --2- 172.21.15.46:0/392091429 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0e980666c0 0x7f0e98068b80 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:23.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.001+0000 7f0ebb200640 1 --2- 172.21.15.46:0/392091429 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0eb415a890 0x7f0eb406c190 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:23.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.001+0000 7f0ebb200640 1 --2- 172.21.15.46:0/392091429 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0eb40b2b30 0x7f0eb4151f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:23.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.001+0000 7f0ebb200640 1 --2- 172.21.15.46:0/392091429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40994c0 0x7f0eb406f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:23.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.001+0000 7f0ebb200640 1 -- 172.21.15.46:0/392091429 >> 172.21.15.46:0/392091429 conn(0x7f0eb40b2010 msgr2=0x7f0eb4155910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:23.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.001+0000 7f0ebb200640 1 -- 172.21.15.46:0/392091429 shutdown_connections 2024-03-29T08:30:23.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.001+0000 7f0ebb200640 1 -- 172.21.15.46:0/392091429 wait complete. 2024-03-29T08:30:23.022 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-03-29T08:30:22.229432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":278796,"kb_used_data":29856,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754695924,"statfs":{"total":773094113280,"available":772808626176,"internally_reserved":0,"allocated":30572544,"data_stored":23458711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392378,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-177,"num_write_kb":-8295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":-2,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001630"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":259,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-03-29T08:30:19.389290+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:30:19.389290+0000","last_peered":"2024-03-29T08:30:19.389290+0000","last_clean":"2024-03-29T08:30:19.389290+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:30:19.389290+0000","last_undegraded":"2024-03-29T08:30:19.389290+0000","last_fullsized":"2024-03-29T08:30:19.389290+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509591,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36756,"kb_used_data":5640,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335084,"statfs":{"total":96636764160,"available":96599126016,"internally_reserved":0,"allocated":5775360,"data_stored":4881953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.72699999999999998,"15min":0.69799999999999995},"min":{"1min":0.53900000000000003,"5min":0.41899999999999998,"15min":0.34999999999999998},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.1639999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75900000000000001,"15min":0.72699999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.45800000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.2410000000000001},"last":0.82399999999999995}]},{"osd":1,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.69399999999999995,"15min":0.68000000000000005},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.39600000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.202},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.70799999999999996,"15min":0.68200000000000005},"min":{"1min":0.505,"5min":0.48999999999999999,"15min":0.44},"max":{"1min":0.93700000000000006,"5min":0.99199999999999999,"15min":1.1339999999999999},"last":0.85899999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70299999999999996,"15min":0.67800000000000005},"min":{"1min":0.61199999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":1.085},"last":0.996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72599999999999998,"15min":0.70699999999999996},"min":{"1min":0.49099999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.97599999999999998,"5min":1.006,"15min":1.288},"last":0.86899999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.71799999999999997,"15min":0.71299999999999997},"min":{"1min":0.56200000000000006,"5min":0.51100000000000001,"15min":0.46000000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":1.018},"last":1.1120000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.73399999999999999,"15min":0.73699999999999999},"min":{"1min":0.59699999999999998,"5min":0.52300000000000002,"15min":0.48499999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.2609999999999999},"last":0.74099999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58999999999999997,"15min":0.60599999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.95299999999999996},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.65500000000000003},"min":{"1min":0.45000000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88400000000000001,"5min":0.89900000000000002,"15min":1.123},"last":1.089}]},{"osd":5,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.60399999999999998,"15min":0.65300000000000002},"min":{"1min":0.40899999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.85699999999999998,"5min":0.89600000000000002,"15min":1.056},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.627,"15min":0.66400000000000003},"min":{"1min":0.442,"5min":0.39800000000000002,"15min":0.39500000000000002},"max":{"1min":0.89800000000000002,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.93999999999999995}]},{"osd":6,"last update":"Fri Mar 29 08:29:23 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70199999999999996,"15min":0.71299999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.97699999999999998,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.66700000000000004,"15min":0.70099999999999996},"min":{"1min":0.52600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.91400000000000003,"5min":0.93799999999999994,"15min":0.98299999999999998},"last":1.0620000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509591,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36748,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":5767168,"data_stored":4881943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72399999999999998,"15min":0.68400000000000005},"min":{"1min":0.58299999999999996,"5min":0.42699999999999999,"15min":0.38900000000000001},"max":{"1min":0.88500000000000001,"5min":0.93100000000000005,"15min":1.0189999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.748,"15min":0.72299999999999998},"min":{"1min":0.60699999999999998,"5min":0.51700000000000002,"15min":0.44500000000000001},"max":{"1min":0.90700000000000003,"5min":0.99299999999999999,"15min":1.079},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68600000000000005,"15min":0.65300000000000002},"min":{"1min":0.54100000000000004,"5min":0.45400000000000001,"15min":0.377},"max":{"1min":0.90300000000000002,"5min":0.96999999999999997,"15min":0.99299999999999999},"last":0.627},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.68799999999999994,"15min":0.66500000000000004},"min":{"1min":0.53900000000000003,"5min":0.42999999999999999,"15min":0.42899999999999999},"max":{"1min":0.82799999999999996,"5min":0.95599999999999996,"15min":1.079},"last":0.65700000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76800000000000002,"15min":0.72899999999999998},"min":{"1min":0.501,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":0.92500000000000004,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.74199999999999999,"15min":0.72599999999999998},"min":{"1min":0.61299999999999999,"5min":0.46400000000000002,"15min":0.435},"max":{"1min":0.93600000000000005,"5min":0.95099999999999996,"15min":0.95899999999999996},"last":0.71399999999999997}]},{"osd":3,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70799999999999996,"15min":0.70599999999999996},"min":{"1min":0.50600000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96899999999999997,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.55200000000000005,"5min":0.51000000000000001,"15min":0.47799999999999998},"max":{"1min":0.86599999999999999,"5min":0.94999999999999996,"15min":0.95299999999999996},"last":0.72699999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.629,"15min":0.64200000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.36499999999999999},"max":{"1min":0.95099999999999996,"5min":1.0840000000000001,"15min":1.556},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63600000000000001,"15min":0.67300000000000004},"min":{"1min":0.42899999999999999,"5min":0.38200000000000001,"15min":0.371},"max":{"1min":0.85099999999999998,"5min":0.996,"15min":2.0539999999999998},"last":0.74099999999999999}]},{"osd":5,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.625,"15min":0.65600000000000003},"min":{"1min":0.41299999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.85499999999999998,"5min":0.99399999999999999,"15min":1.0660000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.37},"max":{"1min":0.92800000000000005,"5min":1.042,"15min":1.1419999999999999},"last":0.52800000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.70199999999999996,"15min":0.72099999999999997},"min":{"1min":0.49399999999999999,"5min":0.46800000000000003,"15min":0.38500000000000001},"max":{"1min":0.89200000000000002,"5min":1.03,"15min":1.03},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.68799999999999994,"15min":0.72199999999999998},"min":{"1min":0.47699999999999998,"5min":0.45400000000000001,"15min":0.44600000000000001},"max":{"1min":0.82699999999999996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.77500000000000002}]}]},{"osd":4,"up_from":15,"seq":64424509590,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36748,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335092,"statfs":{"total":96636764160,"available":96599134208,"internally_reserved":0,"allocated":5767168,"data_stored":4881943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.69799999999999995,"15min":0.70099999999999996},"min":{"1min":0.34499999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.90000000000000002,"5min":1.026,"15min":2.1709999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68400000000000005,"15min":0.71299999999999997},"min":{"1min":0.40799999999999997,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.84799999999999998,"5min":0.997,"15min":2.2869999999999999},"last":0.52800000000000002}]},{"osd":1,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71499999999999997,"15min":0.74299999999999999},"min":{"1min":0.54800000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.871,"5min":1.0049999999999999,"15min":2.524},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.442,"5min":0.438,"15min":0.40699999999999997},"max":{"1min":0.89100000000000001,"5min":1.3149999999999999,"15min":2.2679999999999998},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.69399999999999995,"15min":0.71499999999999997},"min":{"1min":0.45000000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93799999999999994,"5min":1.0720000000000001,"15min":2.363},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.72099999999999997,"15min":0.745},"min":{"1min":0.49099999999999999,"5min":0.434,"15min":0.434},"max":{"1min":0.91300000000000003,"5min":0.97799999999999998,"15min":2.1880000000000002},"last":0.73399999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.751,"15min":0.75},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.46300000000000002},"max":{"1min":0.93999999999999995,"5min":1.3759999999999999,"15min":2.3029999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.752,"15min":0.78600000000000003},"min":{"1min":0.54600000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98099999999999998,"5min":1.016,"15min":2.2549999999999999},"last":0.71399999999999997}]},{"osd":5,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.61499999999999999,"15min":0.65400000000000003},"min":{"1min":0.48199999999999998,"5min":0.38300000000000001,"15min":0.36499999999999999},"max":{"1min":0.85799999999999998,"5min":0.998,"15min":1.0189999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61899999999999999,"15min":0.63600000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":1.0149999999999999},"last":0.46300000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.626,"15min":0.65100000000000002},"min":{"1min":0.46300000000000002,"5min":0.42599999999999999,"15min":0.39000000000000001},"max":{"1min":0.82299999999999995,"5min":1.091,"15min":1.091},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69399999999999995,"15min":0.71599999999999997},"min":{"1min":0.54300000000000004,"5min":0.499,"15min":0.499},"max":{"1min":0.81499999999999995,"5min":0.91500000000000004,"15min":1.1220000000000001},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Mar 29 08:29:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69199999999999995,"15min":0.70799999999999996},"min":{"1min":0.498,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":1.1000000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71899999999999997,"15min":0.74299999999999999},"min":{"1min":0.51200000000000001,"5min":0.434,"15min":0.42899999999999999},"max":{"1min":0.85999999999999999,"5min":1.054,"15min":1.4430000000000001},"last":0.49399999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509593,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32672,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1589248,"data_stored":693393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.28000000000000003,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":0.59299999999999997,"5min":0.748,"15min":1.123},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51100000000000001,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":1.0409999999999999},"last":0.437}]},{"osd":1,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.42099999999999999,"15min":0.443},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.68100000000000005,"5min":0.76800000000000002,"15min":1.1599999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.5,"15min":0.51400000000000001},"min":{"1min":0.32600000000000001,"5min":0.32100000000000001,"15min":0.28599999999999998},"max":{"1min":0.72299999999999998,"5min":0.94499999999999995,"15min":1.4790000000000001},"last":0.41499999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.502,"15min":0.53700000000000003},"min":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.309},"max":{"1min":0.76400000000000001,"5min":0.80300000000000005,"15min":1.2529999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79000000000000004,"5min":0.83599999999999997,"15min":1.8580000000000001},"last":0.47799999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.75700000000000001,"15min":0.76900000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.42199999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.6419999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.52000000000000002,"5min":0.51500000000000001,"15min":0.45400000000000001},"max":{"1min":1.008,"5min":1.083,"15min":1.391},"last":0.82699999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0880000000000001,"5min":1.1850000000000001,"15min":1.671},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80900000000000005,"15min":0.82999999999999996},"min":{"1min":0.58699999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.073,"5min":1.1379999999999999,"15min":1.7110000000000001},"last":0.81100000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.77900000000000003,"15min":0.79300000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97899999999999998,"5min":1.232,"15min":1.669},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76900000000000002,"15min":0.79100000000000004},"min":{"1min":0.54600000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.0900000000000001,"5min":1.119,"15min":2.0150000000000001},"last":0.77100000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.81399999999999995},"min":{"1min":0.57399999999999995,"5min":0.50600000000000001,"15min":0.48799999999999999},"max":{"1min":1.026,"5min":1.1120000000000001,"15min":1.887},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.81100000000000005},"min":{"1min":0.57199999999999995,"5min":0.49399999999999999,"15min":0.47999999999999998},"max":{"1min":1.026,"5min":1.1539999999999999,"15min":2.044},"last":0.71999999999999997}]}]},{"osd":2,"up_from":15,"seq":64424509592,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1576960,"data_stored":687646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45600000000000002,"15min":0.46999999999999997},"min":{"1min":0.30599999999999999,"5min":0.25800000000000001,"15min":0.23899999999999999},"max":{"1min":0.58499999999999996,"5min":0.754,"15min":0.94299999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51800000000000002,"15min":0.52100000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32300000000000001},"max":{"1min":0.64000000000000001,"5min":0.749,"15min":2.0880000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.48299999999999998,"15min":0.49099999999999999},"min":{"1min":0.32600000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68400000000000005,"5min":0.82699999999999996,"15min":1.399},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.48599999999999999,"15min":0.498},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.29299999999999998},"max":{"1min":0.59899999999999998,"5min":0.83299999999999996,"15min":3.0459999999999998},"last":0.36399999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51200000000000001,"15min":0.51600000000000001},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.747,"5min":0.79900000000000004,"15min":2.1280000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58399999999999996,"15min":0.59299999999999997},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.35999999999999999},"max":{"1min":0.76300000000000001,"5min":0.88800000000000001,"15min":2.9950000000000001},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.78500000000000003,"15min":0.77900000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.49099999999999999},"max":{"1min":1.087,"5min":1.109,"15min":1.4239999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.78800000000000003,"15min":0.76700000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.435},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.254}]},{"osd":5,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.85099999999999998,"15min":0.83899999999999997},"min":{"1min":0.60999999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.179,"5min":1.179,"15min":1.4890000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.81499999999999995,"15min":0.82199999999999995},"min":{"1min":0.53700000000000003,"5min":0.52300000000000002,"15min":0.48799999999999999},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.526},"last":1.3540000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79100000000000004,"15min":0.78100000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.48599999999999999},"max":{"1min":1.0289999999999999,"5min":1.167,"15min":2.0470000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82099999999999995,"15min":0.79800000000000004},"min":{"1min":0.57299999999999995,"5min":0.54800000000000004,"15min":0.52500000000000002},"max":{"1min":0.97399999999999998,"5min":1.2609999999999999,"15min":1.524},"last":0.71699999999999997}]},{"osd":7,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.79900000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.319},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.80100000000000005,"15min":0.79500000000000004},"min":{"1min":0.60599999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.96999999999999997,"5min":1.214,"15min":2},"last":0.629}]}]},{"osd":1,"up_from":15,"seq":64424509592,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2166784,"data_stored":1278007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.44,"15min":0.51600000000000001},"min":{"1min":0.249,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.67600000000000005,"5min":0.69699999999999995,"15min":4.2539999999999996},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.46999999999999997,"15min":0.54800000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.627,"5min":0.73399999999999999,"15min":4.1909999999999998},"last":0.318}]},{"osd":2,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.49399999999999999,"15min":0.498},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.66700000000000004,"5min":0.71099999999999997,"15min":0.82899999999999996},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.51900000000000002,"15min":0.52900000000000003},"min":{"1min":0.39900000000000002,"5min":0.35099999999999998,"15min":0.33600000000000002},"max":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.88200000000000001},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56699999999999995,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71999999999999997,"5min":0.88300000000000001,"15min":1.52},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.52300000000000002},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67100000000000004,"5min":0.94499999999999995,"15min":1.331},"last":0.42899999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77700000000000002,"15min":0.75600000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46200000000000002},"max":{"1min":0.89200000000000002,"5min":1.157,"15min":1.157},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.55400000000000005,"5min":0.502,"15min":0.47999999999999998},"max":{"1min":1.0069999999999999,"5min":1.212,"15min":1.23},"last":0.67600000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78700000000000003,"15min":0.77800000000000002},"min":{"1min":0.58199999999999996,"5min":0.503,"15min":0.503},"max":{"1min":0.91400000000000003,"5min":1.258,"15min":1.258},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77900000000000003,"15min":0.76900000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.46500000000000002},"max":{"1min":0.93899999999999995,"5min":1.1879999999999999,"15min":1.2509999999999999},"last":0.71199999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.77100000000000002,"15min":0.77200000000000002},"min":{"1min":0.56799999999999995,"5min":0.54900000000000004,"15min":0.5},"max":{"1min":0.96899999999999997,"5min":1.268,"15min":1.268},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.79500000000000004,"15min":0.81699999999999995},"min":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51300000000000001},"max":{"1min":1.0129999999999999,"5min":1.22,"15min":1.22},"last":0.77500000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.90600000000000003,"5min":1.2410000000000001,"15min":1.3089999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.59299999999999997,"5min":0.503,"15min":0.503},"max":{"1min":0.96299999999999997,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.84899999999999998}]}]},{"osd":0,"up_from":15,"seq":64424509591,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36752,"kb_used_data":5632,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335088,"statfs":{"total":96636764160,"available":96599130112,"internally_reserved":0,"allocated":5767168,"data_stored":4881943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.44,"15min":0.504},"min":{"1min":0.27800000000000002,"5min":0.27000000000000002,"15min":0.22800000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":3.9359999999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.49199999999999999,"15min":0.63700000000000001},"min":{"1min":0.33700000000000002,"5min":0.28499999999999998,"15min":0.253},"max":{"1min":0.66900000000000004,"5min":0.77000000000000002,"15min":8.0419999999999998},"last":0.5}]},{"osd":2,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.503,"15min":0.52300000000000002},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.64600000000000002,"5min":0.78000000000000003,"15min":1.258},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.53400000000000003,"15min":0.61499999999999999},"min":{"1min":0.38700000000000001,"5min":0.32800000000000001,"15min":0.312},"max":{"1min":0.69199999999999995,"5min":0.81399999999999995,"15min":4.907},"last":0.439}]},{"osd":3,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.49399999999999999,"15min":0.63900000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.28499999999999998},"max":{"1min":0.66600000000000004,"5min":0.83399999999999996,"15min":7.9569999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.57599999999999996,"15min":0.60599999999999998},"min":{"1min":0.39600000000000002,"5min":0.36099999999999999,"15min":0.34799999999999998},"max":{"1min":0.72599999999999998,"5min":0.87,"15min":1.397},"last":0.41899999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.78900000000000003,"15min":0.86899999999999999},"min":{"1min":0.54200000000000004,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.282,"5min":1.282,"15min":4.4720000000000004},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.78700000000000003,"15min":0.85099999999999998},"min":{"1min":0.499,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.97999999999999998,"5min":1.153,"15min":4.3920000000000003},"last":0.52100000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.78700000000000003,"15min":0.84399999999999997},"min":{"1min":0.52800000000000002,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.24,"5min":1.24,"15min":4.3200000000000003},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.78600000000000003,"15min":0.81699999999999995},"min":{"1min":0.58399999999999996,"5min":0.56000000000000005,"15min":0.54600000000000004},"max":{"1min":1.0680000000000001,"5min":1.107,"15min":1.6759999999999999},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.79200000000000004,"15min":0.81999999999999995},"min":{"1min":0.57299999999999995,"5min":0.55800000000000005,"15min":0.54400000000000004},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.5920000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80900000000000005,"15min":0.879},"min":{"1min":0.63,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":4.4100000000000001},"last":0.56399999999999995}]},{"osd":7,"last update":"Fri Mar 29 08:29:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81299999999999994,"15min":0.88900000000000001},"min":{"1min":0.57199999999999995,"5min":0.54100000000000004,"15min":0.496},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":4.5209999999999999},"last":0.62},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.79400000000000004,"15min":0.873},"min":{"1min":0.59099999999999997,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":4.4420000000000002},"last":0.58599999999999997}]}]},{"osd":5,"up_from":14,"seq":60129542296,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33228,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2162688,"data_stored":1271883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68899999999999995,"15min":0.66600000000000004},"min":{"1min":0.51800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.97199999999999998,"15min":0.98499999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71899999999999997,"15min":0.68600000000000005},"min":{"1min":0.44700000000000001,"5min":0.42999999999999999,"15min":0.39800000000000002},"max":{"1min":0.81499999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.60599999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.38200000000000001},"max":{"1min":0.80700000000000005,"5min":1.008,"15min":2.5129999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73999999999999999,"15min":0.70599999999999996},"min":{"1min":0.63700000000000001,"5min":0.51300000000000001,"15min":0.40500000000000003},"max":{"1min":0.878,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70499999999999996,"15min":0.68600000000000005},"min":{"1min":0.52200000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88800000000000001,"5min":0.997,"15min":0.997},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.74099999999999999,"15min":0.71799999999999997},"min":{"1min":0.56399999999999995,"5min":0.47499999999999998,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":1.03,"15min":1.03},"last":0.70099999999999996}]},{"osd":3,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.74099999999999999,"15min":0.72499999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.47099999999999997},"max":{"1min":0.92600000000000005,"5min":1.038,"15min":1.038},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.435},"max":{"1min":0.84499999999999997,"5min":1.0209999999999999,"15min":1.954},"last":0.80800000000000005}]},{"osd":4,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.628,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.93500000000000005},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.64200000000000002},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.373},"max":{"1min":0.82699999999999996,"5min":0.89200000000000002,"15min":1.0820000000000001},"last":0.88200000000000001}]},{"osd":6,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.64700000000000002},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.071},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.68500000000000005},"min":{"1min":0.52000000000000002,"5min":0.45300000000000001,"15min":0.432},"max":{"1min":0.93799999999999994,"5min":0.94299999999999995,"15min":1.0609999999999999},"last":0.90000000000000002}]},{"osd":7,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67300000000000004,"15min":0.69099999999999995},"min":{"1min":0.48899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88200000000000001,"5min":0.92200000000000004,"15min":1.1200000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69699999999999995,"15min":0.72499999999999998},"min":{"1min":0.53300000000000003,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.04},"last":0.77700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:30:23.024 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-29T08:30:23.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.165+0000 7fefaa800640 1 Processor -- start 2024-03-29T08:30:23.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.165+0000 7fefaa800640 1 -- start start 2024-03-29T08:30:23.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.165+0000 7fefaa800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefa4144210 0x7fefa40cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:23.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.165+0000 7fefaa800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fefa415ada0 0x7fefa40ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:23.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.165+0000 7fefaa800640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefa40cb510 con 0x7fefa40ce950 2024-03-29T08:30:23.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.165+0000 7fefaa800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa40cede0 con 0x7fefa415ada0 2024-03-29T08:30:23.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.165+0000 7fefaa800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa406d960 con 0x7fefa4144210 2024-03-29T08:30:23.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.165+0000 7fef9be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fefa415ada0 0x7fefa40ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:23.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.165+0000 7fef9be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fefa415ada0 0x7fefa40ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38590/0 (socket says 172.21.15.46:38590) 2024-03-29T08:30:23.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.165+0000 7fef9be00640 1 -- 172.21.15.46:0/1701601650 learned_addr learned my addr 172.21.15.46:0/1701601650 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:23.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.165+0000 7fefa3e00640 1 --1- 172.21.15.46:0/1701601650 >> v1:172.21.15.46:6789/0 conn(0x7fefa40ce950 0x7fefa415e360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57834/0 (socket says 172.21.15.46:57834) 2024-03-29T08:30:23.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.165+0000 7fefa3400640 1 --2- 172.21.15.46:0/1701601650 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefa4144210 0x7fefa40cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:23.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.165+0000 7fef9be00640 1 -- 172.21.15.46:0/1701601650 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefa4144210 msgr2=0x7fefa40cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:23.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.165+0000 7fef9be00640 1 --2- 172.21.15.46:0/1701601650 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefa4144210 0x7fefa40cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:23.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.165+0000 7fef9be00640 1 -- 172.21.15.46:0/1701601650 >> v1:172.21.15.46:6789/0 conn(0x7fefa40ce950 legacy=0x7fefa415e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:23.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.165+0000 7fef9be00640 1 -- 172.21.15.46:0/1701601650 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa4070580 con 0x7fefa415ada0 2024-03-29T08:30:23.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.165+0000 7fefa2a00640 1 -- 172.21.15.46:0/1701601650 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222954537 0 0) 0x7fefa40cb510 con 0x7fefa40ce950 2024-03-29T08:30:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.169+0000 7fef9be00640 1 --2- 172.21.15.46:0/1701601650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa415ada0 0x7fefa40ce410 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009870 tx=0x7fef8c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85000086f2478ffe server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.169+0000 7fefa2a00640 1 -- 172.21.15.46:0/1701601650 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0ab600 con 0x7fefa415ada0 2024-03-29T08:30:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.169+0000 7fefa2a00640 1 -- 172.21.15.46:0/1701601650 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c0a7050 con 0x7fefa415ada0 2024-03-29T08:30:23.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.169+0000 7fefa2a00640 1 -- 172.21.15.46:0/1701601650 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0ab980 con 0x7fefa415ada0 2024-03-29T08:30:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.169+0000 7fefaa800640 1 -- 172.21.15.46:0/1701601650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa415ada0 msgr2=0x7fefa40ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.169+0000 7fefaa800640 1 --2- 172.21.15.46:0/1701601650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa415ada0 0x7fefa40ce410 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fef8c009870 tx=0x7fef8c01c560 comp rx=0 tx=0).stop 2024-03-29T08:30:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.169+0000 7fefaa800640 1 -- 172.21.15.46:0/1701601650 shutdown_connections 2024-03-29T08:30:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.169+0000 7fefaa800640 1 --2- 172.21.15.46:0/1701601650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa415ada0 0x7fefa40ce410 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:23.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.169+0000 7fefaa800640 1 --2- 172.21.15.46:0/1701601650 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefa4144210 0x7fefa40cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:23.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.169+0000 7fefaa800640 1 -- 172.21.15.46:0/1701601650 >> 172.21.15.46:0/1701601650 conn(0x7fefa4066110 msgr2=0x7fefa4069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:23.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.169+0000 7fefaa800640 1 -- 172.21.15.46:0/1701601650 shutdown_connections 2024-03-29T08:30:23.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.169+0000 7fefaa800640 1 -- 172.21.15.46:0/1701601650 wait complete. 2024-03-29T08:30:23.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.169+0000 7fefaa800640 1 Processor -- start 2024-03-29T08:30:23.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefaa800640 1 -- start start 2024-03-29T08:30:23.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefaa800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefa40ce950 0x7fefa40b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:23.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefaa800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefa4144210 0x7fefa41596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:23.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefaa800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa415ada0 0x7fefa4159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:23.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefaa800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa413cb70 con 0x7fefa415ada0 2024-03-29T08:30:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefaa800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa413bad0 con 0x7fefa40ce950 2024-03-29T08:30:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefaa800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa413bdb0 con 0x7fefa4144210 2024-03-29T08:30:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefa3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa415ada0 0x7fefa4159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefa3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa415ada0 0x7fefa4159be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38596/0 (socket says 172.21.15.46:38596) 2024-03-29T08:30:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefa3e00640 1 -- 172.21.15.46:0/1469290310 learned_addr learned my addr 172.21.15.46:0/1469290310 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefa3400640 1 --2- 172.21.15.46:0/1469290310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefa40ce950 0x7fefa40b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fef9be00640 1 --2- 172.21.15.46:0/1469290310 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefa4144210 0x7fefa41596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefa3e00640 1 -- 172.21.15.46:0/1469290310 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefa4144210 msgr2=0x7fefa41596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefa3e00640 1 --2- 172.21.15.46:0/1469290310 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefa4144210 0x7fefa41596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefa3e00640 1 -- 172.21.15.46:0/1469290310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefa40ce950 msgr2=0x7fefa40b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefa3e00640 1 --2- 172.21.15.46:0/1469290310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefa40ce950 0x7fefa40b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefa3e00640 1 -- 172.21.15.46:0/1469290310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c0094d0 con 0x7fefa415ada0 2024-03-29T08:30:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefa3400640 1 --2- 172.21.15.46:0/1469290310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefa40ce950 0x7fefa40b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fef9be00640 1 --2- 172.21.15.46:0/1469290310 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefa4144210 0x7fefa41596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:23.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefa3e00640 1 --2- 172.21.15.46:0/1469290310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa415ada0 0x7fefa4159be0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00c7a0 tx=0x7fef9c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fef9b400640 1 -- 172.21.15.46:0/1469290310 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00a640 con 0x7fefa415ada0 2024-03-29T08:30:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fef9b400640 1 -- 172.21.15.46:0/1469290310 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9c00a7a0 con 0x7fefa415ada0 2024-03-29T08:30:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefaa800640 1 -- 172.21.15.46:0/1469290310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa413c090 con 0x7fefa415ada0 2024-03-29T08:30:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fef9b400640 1 -- 172.21.15.46:0/1469290310 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00ab90 con 0x7fefa415ada0 2024-03-29T08:30:23.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefaa800640 1 -- 172.21.15.46:0/1469290310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa413c650 con 0x7fefa415ada0 2024-03-29T08:30:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.173+0000 7fefaa800640 1 -- 172.21.15.46:0/1469290310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa40ce280 con 0x7fefa415ada0 2024-03-29T08:30:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.177+0000 7fef9b400640 1 -- 172.21.15.46:0/1469290310 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fef9c00acf0 con 0x7fefa415ada0 2024-03-29T08:30:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.177+0000 7fef9b400640 1 --2- 172.21.15.46:0/1469290310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fef80066690 0x7fef80068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.177+0000 7fef9b400640 1 -- 172.21.15.46:0/1469290310 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fef9c083550 con 0x7fefa415ada0 2024-03-29T08:30:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.177+0000 7fefa3400640 1 --2- 172.21.15.46:0/1469290310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fef80066690 0x7fef80068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:23.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.177+0000 7fefa3400640 1 --2- 172.21.15.46:0/1469290310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fef80066690 0x7fef80068b50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fef94009840 tx=0x7fef94008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:23.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.181+0000 7fef9b400640 1 -- 172.21.15.46:0/1469290310 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fef9c050a70 con 0x7fefa415ada0 2024-03-29T08:30:23.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.341+0000 7fefaa800640 1 -- 172.21.15.46:0/1469290310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefa413c840 con 0x7fefa415ada0 2024-03-29T08:30:23.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.345+0000 7fef9b400640 1 -- 172.21.15.46:0/1469290310 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+12736 (secure 0 0 0) 0x7fef9c018020 con 0x7fefa415ada0 2024-03-29T08:30:23.344 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:23.344 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":128,"fsid":"e9b7960f-ec1f-4787-95c3-c806094d37f0","created":"2024-03-29T08:17:48.112932+0000","modified":"2024-03-29T08:30:20.108356+0000","last_up_change":"2024-03-29T08:17:59.745620+0000","last_in_change":"2024-03-29T08:17:50.635668+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-29T08:18:00.103209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"240e3cf2-f51c-4b8d-bb05-9f765ac1b07b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":27424},{"type":"v1","addr":"172.21.15.46:6801","nonce":27424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":27424},{"type":"v1","addr":"172.21.15.46:6803","nonce":27424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":27424},{"type":"v1","addr":"172.21.15.46:6807","nonce":27424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":27424},{"type":"v1","addr":"172.21.15.46:6805","nonce":27424}]},"public_addr":"172.21.15.46:6801/27424","cluster_addr":"172.21.15.46:6803/27424","heartbeat_back_addr":"172.21.15.46:6807/27424","heartbeat_front_addr":"172.21.15.46:6805/27424","state":["exists","up"]},{"osd":1,"uuid":"caa4904f-5189-4846-a8c7-c769e3db5a87","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":27427},{"type":"v1","addr":"172.21.15.46:6825","nonce":27427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":27427},{"type":"v1","addr":"172.21.15.46:6827","nonce":27427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":27427},{"type":"v1","addr":"172.21.15.46:6831","nonce":27427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":27427},{"type":"v1","addr":"172.21.15.46:6829","nonce":27427}]},"public_addr":"172.21.15.46:6825/27427","cluster_addr":"172.21.15.46:6827/27427","heartbeat_back_addr":"172.21.15.46:6831/27427","heartbeat_front_addr":"172.21.15.46:6829/27427","state":["exists","up"]},{"osd":2,"uuid":"4455b5b9-82b8-44b2-a73e-511adaf55986","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":27428},{"type":"v1","addr":"172.21.15.46:6811","nonce":27428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":27428},{"type":"v1","addr":"172.21.15.46:6815","nonce":27428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":27428},{"type":"v1","addr":"172.21.15.46:6823","nonce":27428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":27428},{"type":"v1","addr":"172.21.15.46:6819","nonce":27428}]},"public_addr":"172.21.15.46:6811/27428","cluster_addr":"172.21.15.46:6815/27428","heartbeat_back_addr":"172.21.15.46:6823/27428","heartbeat_front_addr":"172.21.15.46:6819/27428","state":["exists","up"]},{"osd":3,"uuid":"eec611d5-4630-40c0-90cd-ab9dc2e96a94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":27426},{"type":"v1","addr":"172.21.15.46:6810","nonce":27426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":27426},{"type":"v1","addr":"172.21.15.46:6814","nonce":27426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":27426},{"type":"v1","addr":"172.21.15.46:6822","nonce":27426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":27426},{"type":"v1","addr":"172.21.15.46:6818","nonce":27426}]},"public_addr":"172.21.15.46:6810/27426","cluster_addr":"172.21.15.46:6814/27426","heartbeat_back_addr":"172.21.15.46:6822/27426","heartbeat_front_addr":"172.21.15.46:6818/27426","state":["exists","up"]},{"osd":4,"uuid":"55692b87-e81a-48db-bb4a-37b2b44062e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":27689},{"type":"v1","addr":"172.21.15.73:6801","nonce":27689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":27689},{"type":"v1","addr":"172.21.15.73:6803","nonce":27689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":27689},{"type":"v1","addr":"172.21.15.73:6807","nonce":27689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":27689},{"type":"v1","addr":"172.21.15.73:6805","nonce":27689}]},"public_addr":"172.21.15.73:6801/27689","cluster_addr":"172.21.15.73:6803/27689","heartbeat_back_addr":"172.21.15.73:6807/27689","heartbeat_front_addr":"172.21.15.73:6805/27689","state":["exists","up"]},{"osd":5,"uuid":"812dd47d-6a75-4797-be49-c324af4ff32b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":27690},{"type":"v1","addr":"172.21.15.73:6809","nonce":27690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":27690},{"type":"v1","addr":"172.21.15.73:6811","nonce":27690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":27690},{"type":"v1","addr":"172.21.15.73:6815","nonce":27690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":27690},{"type":"v1","addr":"172.21.15.73:6813","nonce":27690}]},"public_addr":"172.21.15.73:6809/27690","cluster_addr":"172.21.15.73:6811/27690","heartbeat_back_addr":"172.21.15.73:6815/27690","heartbeat_front_addr":"172.21.15.73:6813/27690","state":["exists","up"]},{"osd":6,"uuid":"649cf36a-83c0-489b-9ac7-986efc89f190","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":27691},{"type":"v1","addr":"172.21.15.73:6825","nonce":27691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":27691},{"type":"v1","addr":"172.21.15.73:6827","nonce":27691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":27691},{"type":"v1","addr":"172.21.15.73:6831","nonce":27691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":27691},{"type":"v1","addr":"172.21.15.73:6829","nonce":27691}]},"public_addr":"172.21.15.73:6825/27691","cluster_addr":"172.21.15.73:6827/27691","heartbeat_back_addr":"172.21.15.73:6831/27691","heartbeat_front_addr":"172.21.15.73:6829/27691","state":["exists","up"]},{"osd":7,"uuid":"d4175336-2aac-4a9d-8de1-e54d0040cce7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":27700},{"type":"v1","addr":"172.21.15.73:6817","nonce":27700}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":27700},{"type":"v1","addr":"172.21.15.73:6819","nonce":27700}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":27700},{"type":"v1","addr":"172.21.15.73:6823","nonce":27700}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":27700},{"type":"v1","addr":"172.21.15.73:6821","nonce":27700}]},"public_addr":"172.21.15.73:6817/27700","cluster_addr":"172.21.15.73:6819/27700","heartbeat_back_addr":"172.21.15.73:6823/27700","heartbeat_front_addr":"172.21.15.73:6821/27700","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.351428+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.110767+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.410368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.267216+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.174167+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.035794+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.229166+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-03-29T08:17:56.314526+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6835/3867471367":"2024-03-30T08:30:17.095529+0000","172.21.15.46:6834/276185207":"2024-03-30T08:30:00.063526+0000","172.21.15.46:6834/3867471367":"2024-03-30T08:30:17.095529+0000","172.21.15.46:6835/929698978":"2024-03-30T08:29:45.374806+0000","172.21.15.46:6835/276185207":"2024-03-30T08:30:00.063526+0000","172.21.15.46:6834/929698978":"2024-03-30T08:29:45.374806+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-29T08:30:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.345+0000 7fefaa800640 1 -- 172.21.15.46:0/1469290310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fef80066690 msgr2=0x7fef80068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.345+0000 7fefaa800640 1 --2- 172.21.15.46:0/1469290310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fef80066690 0x7fef80068b50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fef94009840 tx=0x7fef94008040 comp rx=0 tx=0).stop 2024-03-29T08:30:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.345+0000 7fefaa800640 1 -- 172.21.15.46:0/1469290310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa415ada0 msgr2=0x7fefa4159be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.345+0000 7fefaa800640 1 --2- 172.21.15.46:0/1469290310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa415ada0 0x7fefa4159be0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00c7a0 tx=0x7fef9c00cc60 comp rx=0 tx=0).stop 2024-03-29T08:30:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.345+0000 7fefaa800640 1 -- 172.21.15.46:0/1469290310 shutdown_connections 2024-03-29T08:30:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.345+0000 7fefaa800640 1 --2- 172.21.15.46:0/1469290310 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fef80066690 0x7fef80068b50 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.345+0000 7fefaa800640 1 --2- 172.21.15.46:0/1469290310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefa415ada0 0x7fefa4159be0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.345+0000 7fefaa800640 1 --2- 172.21.15.46:0/1469290310 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fefa4144210 0x7fefa41596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.345+0000 7fefaa800640 1 --2- 172.21.15.46:0/1469290310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefa40ce950 0x7fefa40b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.345+0000 7fefaa800640 1 -- 172.21.15.46:0/1469290310 >> 172.21.15.46:0/1469290310 conn(0x7fefa4066110 msgr2=0x7fefa4140f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.345+0000 7fefaa800640 1 -- 172.21.15.46:0/1469290310 shutdown_connections 2024-03-29T08:30:23.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:23.345+0000 7fefaa800640 1 -- 172.21.15.46:0/1469290310 wait complete. 2024-03-29T08:30:24.365 INFO:tasks.ceph:Scrubbing osd.0 2024-03-29T08:30:24.366 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-29T08:30:24.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.521+0000 7fa5cee00640 1 Processor -- start 2024-03-29T08:30:24.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.521+0000 7fa5cee00640 1 -- start start 2024-03-29T08:30:24.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.525+0000 7fa5cee00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5c81598a0 0x7fa5c80cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:24.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.525+0000 7fa5cee00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa5c815dc30 0x7fa5c806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:24.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.525+0000 7fa5cee00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c80b1bc0 con 0x7fa5c80b1470 2024-03-29T08:30:24.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.525+0000 7fa5cee00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c80b1ea0 con 0x7fa5c815dc30 2024-03-29T08:30:24.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.525+0000 7fa5cee00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80b2180 con 0x7fa5c81598a0 2024-03-29T08:30:24.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.525+0000 7fa5cce00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa5c815dc30 0x7fa5c806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:24.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.525+0000 7fa5bf400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa5c80b1470 0x7fa5c815d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57838/0 (socket says 172.21.15.46:57838) 2024-03-29T08:30:24.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.525+0000 7fa5bf400640 1 -- 172.21.15.46:0/373862524 learned_addr learned my addr 172.21.15.46:0/373862524 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:24.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.525+0000 7fa5c7e00640 1 --2- 172.21.15.46:0/373862524 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5c81598a0 0x7fa5c80cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:24.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.525+0000 7fa5cce00640 1 -- 172.21.15.46:0/373862524 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5c81598a0 msgr2=0x7fa5c80cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:24.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.525+0000 7fa5cce00640 1 --2- 172.21.15.46:0/373862524 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5c81598a0 0x7fa5c80cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:24.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.525+0000 7fa5cce00640 1 -- 172.21.15.46:0/373862524 >> v1:172.21.15.46:6789/0 conn(0x7fa5c80b1470 legacy=0x7fa5c815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:24.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.525+0000 7fa5cce00640 1 -- 172.21.15.46:0/373862524 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c80b2460 con 0x7fa5c815dc30 2024-03-29T08:30:24.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.525+0000 7fa5c7400640 1 -- 172.21.15.46:0/373862524 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3086105151 0 0) 0x7fa5c80b1bc0 con 0x7fa5c80b1470 2024-03-29T08:30:24.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.525+0000 7fa5c7e00640 1 --2- 172.21.15.46:0/373862524 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5c81598a0 0x7fa5c80cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:24.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.525+0000 7fa5cce00640 1 --2- 172.21.15.46:0/373862524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5c815dc30 0x7fa5c806c6c0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009d90 tx=0x7fa5c001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ec3550b5913b26 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:24.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5c7400640 1 -- 172.21.15.46:0/373862524 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c00a5a20 con 0x7fa5c815dc30 2024-03-29T08:30:24.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5c7400640 1 -- 172.21.15.46:0/373862524 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c00a5b80 con 0x7fa5c815dc30 2024-03-29T08:30:24.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5c7400640 1 -- 172.21.15.46:0/373862524 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c00ad440 con 0x7fa5c815dc30 2024-03-29T08:30:24.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5cee00640 1 -- 172.21.15.46:0/373862524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5c815dc30 msgr2=0x7fa5c806c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:24.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5cee00640 1 --2- 172.21.15.46:0/373862524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5c815dc30 0x7fa5c806c6c0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fa5c0009d90 tx=0x7fa5c001c940 comp rx=0 tx=0).stop 2024-03-29T08:30:24.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5cee00640 1 -- 172.21.15.46:0/373862524 shutdown_connections 2024-03-29T08:30:24.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5cee00640 1 --2- 172.21.15.46:0/373862524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5c815dc30 0x7fa5c806c6c0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:24.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5cee00640 1 --2- 172.21.15.46:0/373862524 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5c81598a0 0x7fa5c80cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:24.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5cee00640 1 -- 172.21.15.46:0/373862524 >> 172.21.15.46:0/373862524 conn(0x7fa5c80b42d0 msgr2=0x7fa5c8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:24.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5cee00640 1 -- 172.21.15.46:0/373862524 shutdown_connections 2024-03-29T08:30:24.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5cee00640 1 -- 172.21.15.46:0/373862524 wait complete. 2024-03-29T08:30:24.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5cee00640 1 Processor -- start 2024-03-29T08:30:24.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5cee00640 1 -- start start 2024-03-29T08:30:24.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5cee00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5c80b1470 0x7fa5c81640f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:24.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5cee00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5c81598a0 0x7fa5c815ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:24.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5cee00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5c815dc30 0x7fa5c80cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:24.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5cee00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8152db0 con 0x7fa5c81598a0 2024-03-29T08:30:24.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5cee00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c81521a0 con 0x7fa5c815dc30 2024-03-29T08:30:24.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5cee00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c8152480 con 0x7fa5c80b1470 2024-03-29T08:30:24.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5c81598a0 0x7fa5c815ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:24.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5bf400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5c81598a0 0x7fa5c815ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38620/0 (socket says 172.21.15.46:38620) 2024-03-29T08:30:24.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5bf400640 1 -- 172.21.15.46:0/173514864 learned_addr learned my addr 172.21.15.46:0/173514864 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:24.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5c7e00640 1 --2- 172.21.15.46:0/173514864 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5c80b1470 0x7fa5c81640f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:24.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.529+0000 7fa5cce00640 1 --2- 172.21.15.46:0/173514864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5c815dc30 0x7fa5c80cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:24.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5bf400640 1 -- 172.21.15.46:0/173514864 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5c80b1470 msgr2=0x7fa5c81640f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:24.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5bf400640 1 --2- 172.21.15.46:0/173514864 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa5c80b1470 0x7fa5c81640f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:24.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5bf400640 1 -- 172.21.15.46:0/173514864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5c815dc30 msgr2=0x7fa5c80cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:24.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5bf400640 1 --2- 172.21.15.46:0/173514864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5c815dc30 0x7fa5c80cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:24.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5bf400640 1 -- 172.21.15.46:0/173514864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c0009a70 con 0x7fa5c81598a0 2024-03-29T08:30:24.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5bf400640 1 --2- 172.21.15.46:0/173514864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5c81598a0 0x7fa5c815ca40 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000c550 tx=0x7fa5b000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:24.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5c4c00640 1 -- 172.21.15.46:0/173514864 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0005cf0 con 0x7fa5c81598a0 2024-03-29T08:30:24.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5c4c00640 1 -- 172.21.15.46:0/173514864 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b0005e70 con 0x7fa5c81598a0 2024-03-29T08:30:24.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5c4c00640 1 -- 172.21.15.46:0/173514864 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b0007310 con 0x7fa5c81598a0 2024-03-29T08:30:24.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5cee00640 1 -- 172.21.15.46:0/173514864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c8152760 con 0x7fa5c81598a0 2024-03-29T08:30:24.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5cee00640 1 -- 172.21.15.46:0/173514864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c815cf80 con 0x7fa5c81598a0 2024-03-29T08:30:24.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5cee00640 1 -- 172.21.15.46:0/173514864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa5c8066c90 con 0x7fa5c81598a0 2024-03-29T08:30:24.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5c4c00640 1 -- 172.21.15.46:0/173514864 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa5b0007470 con 0x7fa5c81598a0 2024-03-29T08:30:24.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5c4c00640 1 --2- 172.21.15.46:0/173514864 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa5a4066690 0x7fa5a4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:24.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5c4c00640 1 -- 172.21.15.46:0/173514864 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fa5b00835e0 con 0x7fa5c81598a0 2024-03-29T08:30:24.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5c4c00640 1 --2- 172.21.15.46:0/173514864 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x7fa5a406c630 0x7fa5a406ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:24.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5c4c00640 1 -- 172.21.15.46:0/173514864 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5a406f100 con 0x7fa5a406c630 2024-03-29T08:30:24.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5c4c00640 1 -- 172.21.15.46:0/173514864 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5b0085bc0 con 0x7fa5c81598a0 2024-03-29T08:30:24.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.533+0000 7fa5c7e00640 1 --2- 172.21.15.46:0/173514864 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa5a4066690 0x7fa5a4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.537+0000 7fa5cce00640 1 --2- 172.21.15.46:0/173514864 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x7fa5a406c630 0x7fa5a406ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:24.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.537+0000 7fa5c7e00640 1 --2- 172.21.15.46:0/173514864 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa5a4066690 0x7fa5a4068b50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800c900 tx=0x7fa5b8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:24.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.537+0000 7fa5cce00640 1 --2- 172.21.15.46:0/173514864 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x7fa5a406c630 0x7fa5a406ea50 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:24.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.537+0000 7fa5c4c00640 1 -- 172.21.15.46:0/173514864 <== osd.0 v2:172.21.15.46:6800/27424 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fa5a406f100 con 0x7fa5a406c630 2024-03-29T08:30:24.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.549+0000 7fa5cee00640 1 -- 172.21.15.46:0/173514864 --> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa5c80cc910 con 0x7fa5a406c630 2024-03-29T08:30:24.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.549+0000 7fa5c4c00640 1 -- 172.21.15.46:0/173514864 <== osd.0 v2:172.21.15.46:6800/27424 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa5c80cc910 con 0x7fa5a406c630 2024-03-29T08:30:24.550 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:30:24.550 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T08:30:24.550 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:30:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.549+0000 7fa5cee00640 1 -- 172.21.15.46:0/173514864 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x7fa5a406c630 msgr2=0x7fa5a406ea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.549+0000 7fa5cee00640 1 --2- 172.21.15.46:0/173514864 >> [v2:172.21.15.46:6800/27424,v1:172.21.15.46:6801/27424] conn(0x7fa5a406c630 0x7fa5a406ea50 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:24.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.549+0000 7fa5cee00640 1 -- 172.21.15.46:0/173514864 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa5a4066690 msgr2=0x7fa5a4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.549+0000 7fa5cee00640 1 --2- 172.21.15.46:0/173514864 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa5a4066690 0x7fa5a4068b50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800c900 tx=0x7fa5b8008040 comp rx=0 tx=0).stop 2024-03-29T08:30:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.549+0000 7fa5cee00640 1 -- 172.21.15.46:0/173514864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5c81598a0 msgr2=0x7fa5c815ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.549+0000 7fa5cee00640 1 --2- 172.21.15.46:0/173514864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa5c81598a0 0x7fa5c815ca40 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000c550 tx=0x7fa5b000ca10 comp rx=0 tx=0).stop 2024-03-29T08:30:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.549+0000 7fa5cce00640 1 -- 172.21.15.46:0/173514864 reap_dead start 2024-03-29T08:30:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.549+0000 7fa5cee00640 1 -- 172.21.15.46:0/173514864 shutdown_connections 2024-03-29T08:30:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.549+0000 7fa5cee00640 1 -- 172.21.15.46:0/173514864 >> 172.21.15.46:0/173514864 conn(0x7fa5c80b42d0 msgr2=0x7fa5c8157d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.549+0000 7fa5cee00640 1 -- 172.21.15.46:0/173514864 shutdown_connections 2024-03-29T08:30:24.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.549+0000 7fa5cee00640 1 -- 172.21.15.46:0/173514864 wait complete. 2024-03-29T08:30:24.569 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-29T08:30:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.709+0000 7f2aa2200640 1 Processor -- start 2024-03-29T08:30:24.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.709+0000 7f2aa2200640 1 -- start start 2024-03-29T08:30:24.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.709+0000 7f2aa2200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2a9c15e080 0x7f2a9c0ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:24.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.709+0000 7f2aa2200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2a9c099950 0x7f2a9c098a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:24.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.709+0000 7f2aa2200640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a9c0993d0 con 0x7f2a9c098f80 2024-03-29T08:30:24.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.709+0000 7f2aa2200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9c0cb500 con 0x7f2a9c099950 2024-03-29T08:30:24.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.709+0000 7f2aa2200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9c0cb7e0 con 0x7f2a9c15e080 2024-03-29T08:30:24.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.709+0000 7f2a93e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2a9c099950 0x7f2a9c098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:24.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.709+0000 7f2aa1800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2a9c098f80 0x7f2a9c15e500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57848/0 (socket says 172.21.15.46:57848) 2024-03-29T08:30:24.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.709+0000 7f2aa1800640 1 -- 172.21.15.46:0/1082375289 learned_addr learned my addr 172.21.15.46:0/1082375289 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.709+0000 7f2aa0e00640 1 --2- 172.21.15.46:0/1082375289 >> v2:172.21.15.73:3300/0 conn(0x7f2a9c15e080 0x7f2a9c0ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.709+0000 7f2a93e00640 1 -- 172.21.15.46:0/1082375289 >> v2:172.21.15.73:3300/0 conn(0x7f2a9c15e080 msgr2=0x7f2a9c0ce2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.709+0000 7f2a93e00640 1 --2- 172.21.15.46:0/1082375289 >> v2:172.21.15.73:3300/0 conn(0x7f2a9c15e080 0x7f2a9c0ce2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.709+0000 7f2a93e00640 1 -- 172.21.15.46:0/1082375289 >> v1:172.21.15.46:6789/0 conn(0x7f2a9c098f80 legacy=0x7f2a9c15e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.713+0000 7f2a93e00640 1 -- 172.21.15.46:0/1082375289 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c0cbac0 con 0x7f2a9c099950 2024-03-29T08:30:24.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.713+0000 7f2aa0e00640 1 --2- 172.21.15.46:0/1082375289 >> v2:172.21.15.73:3300/0 conn(0x7f2a9c15e080 0x7f2a9c0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:24.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.709+0000 7f2a9be00640 1 -- 172.21.15.46:0/1082375289 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782545686 0 0) 0x7f2a940014a0 con 0x7f2a9c098f80 2024-03-29T08:30:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.713+0000 7f2a93e00640 1 --2- 172.21.15.46:0/1082375289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c099950 0x7f2a9c098a40 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c071b80 tx=0x7f2a8401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cff6dd9d70e177a server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.713+0000 7f2a9be00640 1 -- 172.21.15.46:0/1082375289 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a840ab600 con 0x7f2a9c099950 2024-03-29T08:30:24.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.713+0000 7f2a9be00640 1 -- 172.21.15.46:0/1082375289 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a840a7050 con 0x7f2a9c099950 2024-03-29T08:30:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.713+0000 7f2a9be00640 1 -- 172.21.15.46:0/1082375289 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a840ab980 con 0x7f2a9c099950 2024-03-29T08:30:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.713+0000 7f2aa2200640 1 -- 172.21.15.46:0/1082375289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c099950 msgr2=0x7f2a9c098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:24.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.713+0000 7f2aa2200640 1 --2- 172.21.15.46:0/1082375289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c099950 0x7f2a9c098a40 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c071b80 tx=0x7f2a8401c450 comp rx=0 tx=0).stop 2024-03-29T08:30:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.713+0000 7f2aa2200640 1 -- 172.21.15.46:0/1082375289 shutdown_connections 2024-03-29T08:30:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.713+0000 7f2aa2200640 1 --2- 172.21.15.46:0/1082375289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c099950 0x7f2a9c098a40 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.713+0000 7f2aa2200640 1 --2- 172.21.15.46:0/1082375289 >> v2:172.21.15.73:3300/0 conn(0x7f2a9c15e080 0x7f2a9c0ce2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.713+0000 7f2aa2200640 1 -- 172.21.15.46:0/1082375289 >> 172.21.15.46:0/1082375289 conn(0x7f2a9c0b1fe0 msgr2=0x7f2a9c0b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.713+0000 7f2aa2200640 1 -- 172.21.15.46:0/1082375289 shutdown_connections 2024-03-29T08:30:24.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.713+0000 7f2aa2200640 1 -- 172.21.15.46:0/1082375289 wait complete. 2024-03-29T08:30:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2aa2200640 1 Processor -- start 2024-03-29T08:30:24.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2aa2200640 1 -- start start 2024-03-29T08:30:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2aa2200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a9c098f80 0x7f2a9c154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:24.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2aa2200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c099950 0x7f2a9c1469e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2aa2200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a9c15e080 0x7f2a9c159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2aa2200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c156330 con 0x7f2a9c099950 2024-03-29T08:30:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2aa2200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c154390 con 0x7f2a9c098f80 2024-03-29T08:30:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2aa2200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a9c156750 con 0x7f2a9c15e080 2024-03-29T08:30:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2a93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c099950 0x7f2a9c1469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2a93e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c099950 0x7f2a9c1469e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38632/0 (socket says 172.21.15.46:38632) 2024-03-29T08:30:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2a93e00640 1 -- 172.21.15.46:0/3536277166 learned_addr learned my addr 172.21.15.46:0/3536277166 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2aa0e00640 1 --2- 172.21.15.46:0/3536277166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a9c098f80 0x7f2a9c154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2aa1800640 1 --2- 172.21.15.46:0/3536277166 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a9c15e080 0x7f2a9c159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2a93e00640 1 -- 172.21.15.46:0/3536277166 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a9c15e080 msgr2=0x7f2a9c159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2a93e00640 1 --2- 172.21.15.46:0/3536277166 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a9c15e080 0x7f2a9c159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2a93e00640 1 -- 172.21.15.46:0/3536277166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a9c098f80 msgr2=0x7f2a9c154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2a93e00640 1 --2- 172.21.15.46:0/3536277166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a9c098f80 0x7f2a9c154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:24.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2a93e00640 1 -- 172.21.15.46:0/3536277166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a84009420 con 0x7f2a9c099950 2024-03-29T08:30:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2aa1800640 1 --2- 172.21.15.46:0/3536277166 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a9c15e080 0x7f2a9c159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2aa0e00640 1 --2- 172.21.15.46:0/3536277166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a9c098f80 0x7f2a9c154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2a93e00640 1 --2- 172.21.15.46:0/3536277166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c099950 0x7f2a9c1469e0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f2a840ae040 tx=0x7f2a840191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2a99600640 1 -- 172.21.15.46:0/3536277166 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a840b4840 con 0x7f2a9c099950 2024-03-29T08:30:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2a99600640 1 -- 172.21.15.46:0/3536277166 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a840b49c0 con 0x7f2a9c099950 2024-03-29T08:30:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2a99600640 1 -- 172.21.15.46:0/3536277166 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a840b4d70 con 0x7f2a9c099950 2024-03-29T08:30:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2aa2200640 1 -- 172.21.15.46:0/3536277166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a9c1553f0 con 0x7f2a9c099950 2024-03-29T08:30:24.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.717+0000 7f2aa2200640 1 -- 172.21.15.46:0/3536277166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a9c155990 con 0x7f2a9c099950 2024-03-29T08:30:24.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.721+0000 7f2aa2200640 1 -- 172.21.15.46:0/3536277166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9c04e040 con 0x7f2a9c099950 2024-03-29T08:30:24.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.721+0000 7f2a99600640 1 -- 172.21.15.46:0/3536277166 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2a840b2040 con 0x7f2a9c099950 2024-03-29T08:30:24.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.721+0000 7f2a99600640 1 --2- 172.21.15.46:0/3536277166 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2a78066690 0x7f2a78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:24.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.721+0000 7f2a99600640 1 -- 172.21.15.46:0/3536277166 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f2a8411e3f0 con 0x7f2a9c099950 2024-03-29T08:30:24.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.721+0000 7f2aa0e00640 1 --2- 172.21.15.46:0/3536277166 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2a78066690 0x7f2a78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:24.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.721+0000 7f2aa0e00640 1 --2- 172.21.15.46:0/3536277166 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2a78066690 0x7f2a78068b50 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c008890 tx=0x7f2a8c008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:24.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.725+0000 7f2a99600640 1 -- 172.21.15.46:0/3536277166 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2a840eb910 con 0x7f2a9c099950 2024-03-29T08:30:24.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.885+0000 7f2aa2200640 1 -- 172.21.15.46:0/3536277166 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a9c15d380 con 0x7f2a78066690 2024-03-29T08:30:24.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.885+0000 7f2a99600640 1 -- 172.21.15.46:0/3536277166 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2a8c00c040 con 0x7f2a78066690 2024-03-29T08:30:24.886 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 0 to deep-scrub 2024-03-29T08:30:24.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.889+0000 7f2aa2200640 1 -- 172.21.15.46:0/3536277166 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2a78066690 msgr2=0x7f2a78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:24.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.889+0000 7f2aa2200640 1 --2- 172.21.15.46:0/3536277166 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2a78066690 0x7f2a78068b50 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c008890 tx=0x7f2a8c008040 comp rx=0 tx=0).stop 2024-03-29T08:30:24.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.889+0000 7f2aa2200640 1 -- 172.21.15.46:0/3536277166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c099950 msgr2=0x7f2a9c1469e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:24.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.889+0000 7f2aa2200640 1 --2- 172.21.15.46:0/3536277166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c099950 0x7f2a9c1469e0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f2a840ae040 tx=0x7f2a840191f0 comp rx=0 tx=0).stop 2024-03-29T08:30:24.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.889+0000 7f2aa2200640 1 -- 172.21.15.46:0/3536277166 shutdown_connections 2024-03-29T08:30:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.889+0000 7f2aa2200640 1 --2- 172.21.15.46:0/3536277166 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2a78066690 0x7f2a78068b50 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.889+0000 7f2aa2200640 1 --2- 172.21.15.46:0/3536277166 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2a9c15e080 0x7f2a9c159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.889+0000 7f2aa2200640 1 --2- 172.21.15.46:0/3536277166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a9c099950 0x7f2a9c1469e0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.889+0000 7f2aa2200640 1 --2- 172.21.15.46:0/3536277166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a9c098f80 0x7f2a9c154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.889+0000 7f2aa2200640 1 -- 172.21.15.46:0/3536277166 >> 172.21.15.46:0/3536277166 conn(0x7f2a9c0b1fe0 msgr2=0x7f2a9c0b3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.889+0000 7f2aa2200640 1 -- 172.21.15.46:0/3536277166 shutdown_connections 2024-03-29T08:30:24.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:24.889+0000 7f2aa2200640 1 -- 172.21.15.46:0/3536277166 wait complete. 2024-03-29T08:30:24.907 INFO:tasks.ceph:Scrubbing osd.1 2024-03-29T08:30:24.907 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-29T08:30:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.049+0000 7f9581800640 1 Processor -- start 2024-03-29T08:30:25.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.049+0000 7f9581800640 1 -- start start 2024-03-29T08:30:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.053+0000 7f9581800640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f957c15dc30 0x7f957c0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.053+0000 7f9581800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f957c0b1470 0x7f957c06c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.053+0000 7f9581800640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957c0b1bc0 con 0x7f957c1598a0 2024-03-29T08:30:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.053+0000 7f9581800640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957c0b1ea0 con 0x7f957c15dc30 2024-03-29T08:30:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.053+0000 7f9581800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957c0b2180 con 0x7f957c0b1470 2024-03-29T08:30:25.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.053+0000 7f9572000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f957c15dc30 0x7f957c0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.053+0000 7f957aa00640 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f957c1598a0 0x7f957c15d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.46:43848/0 (socket says 172.21.15.46:43848) 2024-03-29T08:30:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.053+0000 7f9572000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f957c15dc30 0x7f957c0cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38642/0 (socket says 172.21.15.46:38642) 2024-03-29T08:30:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.053+0000 7f957aa00640 1 -- 172.21.15.46:0/2452865772 learned_addr learned my addr 172.21.15.46:0/2452865772 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.053+0000 7f957b400640 1 --2- 172.21.15.46:0/2452865772 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f957c0b1470 0x7f957c06c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.053+0000 7f9572000640 1 -- 172.21.15.46:0/2452865772 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f957c0b1470 msgr2=0x7f957c06c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.053+0000 7f9572000640 1 --2- 172.21.15.46:0/2452865772 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f957c0b1470 0x7f957c06c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.053+0000 7f9572000640 1 -- 172.21.15.46:0/2452865772 >> v1:172.21.15.73:6789/0 conn(0x7f957c1598a0 legacy=0x7f957c15d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.053+0000 7f9572000640 1 -- 172.21.15.46:0/2452865772 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957c0b2460 con 0x7f957c15dc30 2024-03-29T08:30:25.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.057+0000 7f9572000640 1 --2- 172.21.15.46:0/2452865772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c15dc30 0x7f957c0cadb0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f956400ba60 tx=0x7f956401eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31be9b5c6a99f490 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:25.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.057+0000 7f957a000640 1 -- 172.21.15.46:0/2452865772 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95640a7ba0 con 0x7f957c15dc30 2024-03-29T08:30:25.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.057+0000 7f957a000640 1 -- 172.21.15.46:0/2452865772 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95640a7d00 con 0x7f957c15dc30 2024-03-29T08:30:25.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.057+0000 7f957a000640 1 -- 172.21.15.46:0/2452865772 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95640af660 con 0x7f957c15dc30 2024-03-29T08:30:25.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.057+0000 7f9581800640 1 -- 172.21.15.46:0/2452865772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c15dc30 msgr2=0x7f957c0cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.057+0000 7f9581800640 1 --2- 172.21.15.46:0/2452865772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c15dc30 0x7f957c0cadb0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f956400ba60 tx=0x7f956401eb40 comp rx=0 tx=0).stop 2024-03-29T08:30:25.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.057+0000 7f9581800640 1 -- 172.21.15.46:0/2452865772 shutdown_connections 2024-03-29T08:30:25.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.057+0000 7f9581800640 1 --2- 172.21.15.46:0/2452865772 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f957c0b1470 0x7f957c06c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.057+0000 7f9581800640 1 --2- 172.21.15.46:0/2452865772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c15dc30 0x7f957c0cadb0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.057+0000 7f9581800640 1 -- 172.21.15.46:0/2452865772 >> 172.21.15.46:0/2452865772 conn(0x7f957c0b42d0 msgr2=0x7f957c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:25.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.057+0000 7f9581800640 1 -- 172.21.15.46:0/2452865772 shutdown_connections 2024-03-29T08:30:25.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.057+0000 7f9581800640 1 -- 172.21.15.46:0/2452865772 wait complete. 2024-03-29T08:30:25.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.057+0000 7f9581800640 1 Processor -- start 2024-03-29T08:30:25.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.057+0000 7f9581800640 1 -- start start 2024-03-29T08:30:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f9581800640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c0b1470 0x7f957c1f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f9581800640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f957c1598a0 0x7f957c14d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f9581800640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f957c15dc30 0x7f957c1543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f9581800640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c157b80 con 0x7f957c0b1470 2024-03-29T08:30:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f9581800640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c156f40 con 0x7f957c15dc30 2024-03-29T08:30:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f9581800640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957c1571e0 con 0x7f957c1598a0 2024-03-29T08:30:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f957aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c0b1470 0x7f957c1f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f957aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c0b1470 0x7f957c1f4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38658/0 (socket says 172.21.15.46:38658) 2024-03-29T08:30:25.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f957aa00640 1 -- 172.21.15.46:0/271190615 learned_addr learned my addr 172.21.15.46:0/271190615 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f9572000640 1 --2- 172.21.15.46:0/271190615 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f957c1598a0 0x7f957c14d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f957b400640 1 --2- 172.21.15.46:0/271190615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f957c15dc30 0x7f957c1543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f957aa00640 1 -- 172.21.15.46:0/271190615 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f957c1598a0 msgr2=0x7f957c14d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f957aa00640 1 --2- 172.21.15.46:0/271190615 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f957c1598a0 0x7f957c14d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f957aa00640 1 -- 172.21.15.46:0/271190615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f957c15dc30 msgr2=0x7f957c1543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f957aa00640 1 --2- 172.21.15.46:0/271190615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f957c15dc30 0x7f957c1543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f957aa00640 1 -- 172.21.15.46:0/271190615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956400b6c0 con 0x7f957c0b1470 2024-03-29T08:30:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f957b400640 1 --2- 172.21.15.46:0/271190615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f957c15dc30 0x7f957c1543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f957aa00640 1 --2- 172.21.15.46:0/271190615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c0b1470 0x7f957c1f4d20 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f956c005890 tx=0x7f956c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f9573400640 1 -- 172.21.15.46:0/271190615 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c005cf0 con 0x7f957c0b1470 2024-03-29T08:30:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f9573400640 1 -- 172.21.15.46:0/271190615 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956c005e50 con 0x7f957c0b1470 2024-03-29T08:30:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f9581800640 1 -- 172.21.15.46:0/271190615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957c1574c0 con 0x7f957c0b1470 2024-03-29T08:30:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f9573400640 1 -- 172.21.15.46:0/271190615 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c0072b0 con 0x7f957c0b1470 2024-03-29T08:30:25.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f9581800640 1 -- 172.21.15.46:0/271190615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957c1f5260 con 0x7f957c0b1470 2024-03-29T08:30:25.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f9581800640 1 -- 172.21.15.46:0/271190615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f957c066c90 con 0x7f957c0b1470 2024-03-29T08:30:25.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.061+0000 7f9573400640 1 -- 172.21.15.46:0/271190615 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f956c007410 con 0x7f957c0b1470 2024-03-29T08:30:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.065+0000 7f9573400640 1 --2- 172.21.15.46:0/271190615 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9558066690 0x7f9558068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.065+0000 7f9573400640 1 -- 172.21.15.46:0/271190615 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f956c083400 con 0x7f957c0b1470 2024-03-29T08:30:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.065+0000 7f9573400640 1 --2- 172.21.15.46:0/271190615 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x7f955806c630 0x7f955806ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.065+0000 7f9573400640 1 -- 172.21.15.46:0/271190615 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f955806f100 con 0x7f955806c630 2024-03-29T08:30:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.065+0000 7f9573400640 1 -- 172.21.15.46:0/271190615 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7f956c0859e0 con 0x7f957c0b1470 2024-03-29T08:30:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.065+0000 7f9572000640 1 --2- 172.21.15.46:0/271190615 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9558066690 0x7f9558068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.065+0000 7f957b400640 1 --2- 172.21.15.46:0/271190615 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x7f955806c630 0x7f955806ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.065+0000 7f957b400640 1 --2- 172.21.15.46:0/271190615 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x7f955806c630 0x7f955806ea50 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.065+0000 7f9572000640 1 --2- 172.21.15.46:0/271190615 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9558066690 0x7f9558068b50 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f95640b1000 tx=0x7f95640a9070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:25.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.065+0000 7f9573400640 1 -- 172.21.15.46:0/271190615 <== osd.1 v2:172.21.15.46:6824/27427 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f955806f100 con 0x7f955806c630 2024-03-29T08:30:25.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.077+0000 7f9581800640 1 -- 172.21.15.46:0/271190615 --> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f957c157cc0 con 0x7f955806c630 2024-03-29T08:30:25.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.077+0000 7f9573400640 1 -- 172.21.15.46:0/271190615 <== osd.1 v2:172.21.15.46:6824/27427 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f957c157cc0 con 0x7f955806c630 2024-03-29T08:30:25.079 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:30:25.079 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T08:30:25.079 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:30:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.077+0000 7f9581800640 1 -- 172.21.15.46:0/271190615 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x7f955806c630 msgr2=0x7f955806ea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.077+0000 7f9581800640 1 --2- 172.21.15.46:0/271190615 >> [v2:172.21.15.46:6824/27427,v1:172.21.15.46:6825/27427] conn(0x7f955806c630 0x7f955806ea50 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.077+0000 7f9581800640 1 -- 172.21.15.46:0/271190615 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9558066690 msgr2=0x7f9558068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.077+0000 7f9581800640 1 --2- 172.21.15.46:0/271190615 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f9558066690 0x7f9558068b50 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f95640b1000 tx=0x7f95640a9070 comp rx=0 tx=0).stop 2024-03-29T08:30:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.077+0000 7f9581800640 1 -- 172.21.15.46:0/271190615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c0b1470 msgr2=0x7f957c1f4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.077+0000 7f9581800640 1 --2- 172.21.15.46:0/271190615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f957c0b1470 0x7f957c1f4d20 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f956c005890 tx=0x7f956c00c960 comp rx=0 tx=0).stop 2024-03-29T08:30:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.077+0000 7f957b400640 1 -- 172.21.15.46:0/271190615 reap_dead start 2024-03-29T08:30:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.077+0000 7f9581800640 1 -- 172.21.15.46:0/271190615 shutdown_connections 2024-03-29T08:30:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.077+0000 7f9581800640 1 -- 172.21.15.46:0/271190615 >> 172.21.15.46:0/271190615 conn(0x7f957c0b42d0 msgr2=0x7f957c155d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:25.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.077+0000 7f9581800640 1 -- 172.21.15.46:0/271190615 shutdown_connections 2024-03-29T08:30:25.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.077+0000 7f9581800640 1 -- 172.21.15.46:0/271190615 wait complete. 2024-03-29T08:30:25.097 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-29T08:30:25.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.237+0000 7ff335c00640 1 Processor -- start 2024-03-29T08:30:25.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.237+0000 7ff335c00640 1 -- start start 2024-03-29T08:30:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.241+0000 7ff335c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff33015ca10 0x7ff33006c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.241+0000 7ff335c00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff33006ada0 0x7ff33006b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.241+0000 7ff335c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33006da20 con 0x7ff3300715d0 2024-03-29T08:30:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.241+0000 7ff335c00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff33006b770 con 0x7ff33006ada0 2024-03-29T08:30:25.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.241+0000 7ff335c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3300ac490 con 0x7ff33015ca10 2024-03-29T08:30:25.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.241+0000 7ff32f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff33006ada0 0x7ff33006b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.241+0000 7ff326000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff3300715d0 0x7ff3300abd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57854/0 (socket says 172.21.15.46:57854) 2024-03-29T08:30:25.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.241+0000 7ff326000640 1 -- 172.21.15.46:0/4082257022 learned_addr learned my addr 172.21.15.46:0/4082257022 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:25.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.241+0000 7ff32f400640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff33006ada0 0x7ff33006b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38662/0 (socket says 172.21.15.46:38662) 2024-03-29T08:30:25.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.241+0000 7ff32ea00640 1 --2- 172.21.15.46:0/4082257022 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff33015ca10 0x7ff33006c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.241+0000 7ff32e000640 1 -- 172.21.15.46:0/4082257022 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2426422889 0 0) 0x7ff33006da20 con 0x7ff3300715d0 2024-03-29T08:30:25.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.241+0000 7ff32e000640 1 -- 172.21.15.46:0/4082257022 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff314003530 con 0x7ff3300715d0 2024-03-29T08:30:25.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.241+0000 7ff32f400640 1 -- 172.21.15.46:0/4082257022 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff33015ca10 msgr2=0x7ff33006c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:25.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.241+0000 7ff32f400640 1 --2- 172.21.15.46:0/4082257022 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff33015ca10 0x7ff33006c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.241+0000 7ff32f400640 1 -- 172.21.15.46:0/4082257022 >> v1:172.21.15.46:6789/0 conn(0x7ff3300715d0 legacy=0x7ff3300abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.241+0000 7ff32f400640 1 -- 172.21.15.46:0/4082257022 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff330066060 con 0x7ff33006ada0 2024-03-29T08:30:25.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff32f400640 1 --2- 172.21.15.46:0/4082257022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff33006ada0 0x7ff33006b230 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7ff3280097c0 tx=0x7ff32801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569305464233f948 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:25.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff32e000640 1 -- 172.21.15.46:0/4082257022 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3280ad640 con 0x7ff33006ada0 2024-03-29T08:30:25.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff32e000640 1 -- 172.21.15.46:0/4082257022 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3280ad7a0 con 0x7ff33006ada0 2024-03-29T08:30:25.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff32e000640 1 -- 172.21.15.46:0/4082257022 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3280adb10 con 0x7ff33006ada0 2024-03-29T08:30:25.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff335c00640 1 -- 172.21.15.46:0/4082257022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff33006ada0 msgr2=0x7ff33006b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff335c00640 1 --2- 172.21.15.46:0/4082257022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff33006ada0 0x7ff33006b230 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7ff3280097c0 tx=0x7ff32801c4a0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff335c00640 1 -- 172.21.15.46:0/4082257022 shutdown_connections 2024-03-29T08:30:25.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff335c00640 1 --2- 172.21.15.46:0/4082257022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff33006ada0 0x7ff33006b230 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff335c00640 1 --2- 172.21.15.46:0/4082257022 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff33015ca10 0x7ff33006c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff335c00640 1 -- 172.21.15.46:0/4082257022 >> 172.21.15.46:0/4082257022 conn(0x7ff330070ab0 msgr2=0x7ff330070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:25.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff335c00640 1 -- 172.21.15.46:0/4082257022 shutdown_connections 2024-03-29T08:30:25.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff335c00640 1 -- 172.21.15.46:0/4082257022 wait complete. 2024-03-29T08:30:25.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff335c00640 1 Processor -- start 2024-03-29T08:30:25.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff335c00640 1 -- start start 2024-03-29T08:30:25.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff335c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff33006ada0 0x7ff33014ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff335c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3300715d0 0x7ff33015da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff335c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff33015ca10 0x7ff33015eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff335c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33015d630 con 0x7ff3300715d0 2024-03-29T08:30:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff335c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33015df90 con 0x7ff33015ca10 2024-03-29T08:30:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff335c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33015f050 con 0x7ff33006ada0 2024-03-29T08:30:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff326000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3300715d0 0x7ff33015da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff326000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3300715d0 0x7ff33015da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38674/0 (socket says 172.21.15.46:38674) 2024-03-29T08:30:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.245+0000 7ff326000640 1 -- 172.21.15.46:0/2414696930 learned_addr learned my addr 172.21.15.46:0/2414696930 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff32ea00640 1 --2- 172.21.15.46:0/2414696930 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff33006ada0 0x7ff33014ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff32f400640 1 --2- 172.21.15.46:0/2414696930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff33015ca10 0x7ff33015eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff326000640 1 -- 172.21.15.46:0/2414696930 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff33006ada0 msgr2=0x7ff33014ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff326000640 1 --2- 172.21.15.46:0/2414696930 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff33006ada0 0x7ff33014ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff326000640 1 -- 172.21.15.46:0/2414696930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff33015ca10 msgr2=0x7ff33015eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff326000640 1 --2- 172.21.15.46:0/2414696930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff33015ca10 0x7ff33015eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff326000640 1 -- 172.21.15.46:0/2414696930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff328009420 con 0x7ff3300715d0 2024-03-29T08:30:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff32f400640 1 --2- 172.21.15.46:0/2414696930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff33015ca10 0x7ff33015eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff326000640 1 --2- 172.21.15.46:0/2414696930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3300715d0 0x7ff33015da50 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7ff31800cb40 tx=0x7ff31800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:25.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff327400640 1 -- 172.21.15.46:0/2414696930 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31800aca0 con 0x7ff3300715d0 2024-03-29T08:30:25.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff327400640 1 -- 172.21.15.46:0/2414696930 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31800ae00 con 0x7ff3300715d0 2024-03-29T08:30:25.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff327400640 1 -- 172.21.15.46:0/2414696930 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318012620 con 0x7ff3300715d0 2024-03-29T08:30:25.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff335c00640 1 -- 172.21.15.46:0/2414696930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33015f2f0 con 0x7ff3300715d0 2024-03-29T08:30:25.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff335c00640 1 -- 172.21.15.46:0/2414696930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33015cf70 con 0x7ff3300715d0 2024-03-29T08:30:25.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff335c00640 1 -- 172.21.15.46:0/2414696930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33015f810 con 0x7ff3300715d0 2024-03-29T08:30:25.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff327400640 1 -- 172.21.15.46:0/2414696930 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7ff318007070 con 0x7ff3300715d0 2024-03-29T08:30:25.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff327400640 1 --2- 172.21.15.46:0/2414696930 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff30c0666c0 0x7ff30c068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff327400640 1 -- 172.21.15.46:0/2414696930 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7ff318084080 con 0x7ff3300715d0 2024-03-29T08:30:25.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.249+0000 7ff32ea00640 1 --2- 172.21.15.46:0/2414696930 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff30c0666c0 0x7ff30c068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.253+0000 7ff32ea00640 1 --2- 172.21.15.46:0/2414696930 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff30c0666c0 0x7ff30c068b80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff320009a50 tx=0x7ff320008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:25.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.253+0000 7ff327400640 1 -- 172.21.15.46:0/2414696930 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7ff3180b5980 con 0x7ff3300715d0 2024-03-29T08:30:25.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.413+0000 7ff335c00640 1 -- 172.21.15.46:0/2414696930 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7ff33015cdf0 con 0x7ff30c0666c0 2024-03-29T08:30:25.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.413+0000 7ff327400640 1 -- 172.21.15.46:0/2414696930 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff3200119a0 con 0x7ff30c0666c0 2024-03-29T08:30:25.416 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 1 to deep-scrub 2024-03-29T08:30:25.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.417+0000 7ff335c00640 1 -- 172.21.15.46:0/2414696930 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff30c0666c0 msgr2=0x7ff30c068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.417+0000 7ff335c00640 1 --2- 172.21.15.46:0/2414696930 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff30c0666c0 0x7ff30c068b80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff320009a50 tx=0x7ff320008040 comp rx=0 tx=0).stop 2024-03-29T08:30:25.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.417+0000 7ff335c00640 1 -- 172.21.15.46:0/2414696930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3300715d0 msgr2=0x7ff33015da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.417+0000 7ff335c00640 1 --2- 172.21.15.46:0/2414696930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3300715d0 0x7ff33015da50 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7ff31800cb40 tx=0x7ff31800a060 comp rx=0 tx=0).stop 2024-03-29T08:30:25.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.417+0000 7ff335c00640 1 -- 172.21.15.46:0/2414696930 shutdown_connections 2024-03-29T08:30:25.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.417+0000 7ff335c00640 1 --2- 172.21.15.46:0/2414696930 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7ff30c0666c0 0x7ff30c068b80 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.417+0000 7ff335c00640 1 --2- 172.21.15.46:0/2414696930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff33015ca10 0x7ff33015eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.417+0000 7ff335c00640 1 --2- 172.21.15.46:0/2414696930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3300715d0 0x7ff33015da50 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.417+0000 7ff335c00640 1 --2- 172.21.15.46:0/2414696930 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7ff33006ada0 0x7ff33014ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.417+0000 7ff335c00640 1 -- 172.21.15.46:0/2414696930 >> 172.21.15.46:0/2414696930 conn(0x7ff330070ab0 msgr2=0x7ff3300acf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:25.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.417+0000 7ff335c00640 1 -- 172.21.15.46:0/2414696930 shutdown_connections 2024-03-29T08:30:25.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.417+0000 7ff335c00640 1 -- 172.21.15.46:0/2414696930 wait complete. 2024-03-29T08:30:25.438 INFO:tasks.ceph:Scrubbing osd.2 2024-03-29T08:30:25.438 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-29T08:30:25.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.585+0000 7fcb2da00640 1 Processor -- start 2024-03-29T08:30:25.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.585+0000 7fcb2da00640 1 -- start start 2024-03-29T08:30:25.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.585+0000 7fcb2da00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb2815cc50 0x7fcb28161100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.585+0000 7fcb2da00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcb2806b470 0x7fcb280ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.585+0000 7fcb2da00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb281606b0 con 0x7fcb280709f0 2024-03-29T08:30:25.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.585+0000 7fcb2da00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb2815d030 con 0x7fcb2806b470 2024-03-29T08:30:25.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.585+0000 7fcb2da00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb28160c70 con 0x7fcb2815cc50 2024-03-29T08:30:25.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.585+0000 7fcb26a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb2815cc50 0x7fcb28161100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.585+0000 7fcb26a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb2815cc50 0x7fcb28161100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3301/0 says I am v2:172.21.15.46:56694/0 (socket says 172.21.15.46:56694) 2024-03-29T08:30:25.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.585+0000 7fcb26a00640 1 -- 172.21.15.46:0/2570858359 learned_addr learned my addr 172.21.15.46:0/2570858359 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:25.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.585+0000 7fcb1e000640 1 --2- 172.21.15.46:0/2570858359 >> v2:172.21.15.73:3300/0 conn(0x7fcb2806b470 0x7fcb280ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.585+0000 7fcb26a00640 1 -- 172.21.15.46:0/2570858359 >> v2:172.21.15.73:3300/0 conn(0x7fcb2806b470 msgr2=0x7fcb280ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:25.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.585+0000 7fcb26a00640 1 --2- 172.21.15.46:0/2570858359 >> v2:172.21.15.73:3300/0 conn(0x7fcb2806b470 0x7fcb280ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.585+0000 7fcb26a00640 1 -- 172.21.15.46:0/2570858359 >> v1:172.21.15.73:6789/0 conn(0x7fcb280709f0 legacy=0x7fcb2806d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.585+0000 7fcb26a00640 1 -- 172.21.15.46:0/2570858359 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb280cd190 con 0x7fcb2815cc50 2024-03-29T08:30:25.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.589+0000 7fcb26a00640 1 --2- 172.21.15.46:0/2570858359 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb2815cc50 0x7fcb28161100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009870 tx=0x7fcb2806e7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43798678b0f1d424 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:25.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.589+0000 7fcb26000640 1 -- 172.21.15.46:0/2570858359 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb180ab650 con 0x7fcb2815cc50 2024-03-29T08:30:25.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.589+0000 7fcb26000640 1 -- 172.21.15.46:0/2570858359 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb180ab7b0 con 0x7fcb2815cc50 2024-03-29T08:30:25.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.589+0000 7fcb26000640 1 -- 172.21.15.46:0/2570858359 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb180abb40 con 0x7fcb2815cc50 2024-03-29T08:30:25.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.589+0000 7fcb2da00640 1 -- 172.21.15.46:0/2570858359 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb2815cc50 msgr2=0x7fcb28161100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.589+0000 7fcb2da00640 1 --2- 172.21.15.46:0/2570858359 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb2815cc50 0x7fcb28161100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcb18009870 tx=0x7fcb2806e7e0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.589+0000 7fcb2da00640 1 -- 172.21.15.46:0/2570858359 shutdown_connections 2024-03-29T08:30:25.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.589+0000 7fcb2da00640 1 --2- 172.21.15.46:0/2570858359 >> v2:172.21.15.73:3300/0 conn(0x7fcb2806b470 0x7fcb280ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.589+0000 7fcb2da00640 1 --2- 172.21.15.46:0/2570858359 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb2815cc50 0x7fcb28161100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.589+0000 7fcb2da00640 1 -- 172.21.15.46:0/2570858359 >> 172.21.15.46:0/2570858359 conn(0x7fcb280ced50 msgr2=0x7fcb280cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:25.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.589+0000 7fcb2da00640 1 -- 172.21.15.46:0/2570858359 shutdown_connections 2024-03-29T08:30:25.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.589+0000 7fcb2da00640 1 -- 172.21.15.46:0/2570858359 wait complete. 2024-03-29T08:30:25.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb2da00640 1 Processor -- start 2024-03-29T08:30:25.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb2da00640 1 -- start start 2024-03-29T08:30:25.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb2da00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb2806b470 0x7fcb2815c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb2da00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb280709f0 0x7fcb28149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb2da00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb2815cc50 0x7fcb280b3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb2da00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2814a3f0 con 0x7fcb280709f0 2024-03-29T08:30:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb2da00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2814a6d0 con 0x7fcb2806b470 2024-03-29T08:30:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb2da00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2814a9b0 con 0x7fcb2815cc50 2024-03-29T08:30:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb1e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb280709f0 0x7fcb28149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb1e000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb280709f0 0x7fcb28149bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38680/0 (socket says 172.21.15.46:38680) 2024-03-29T08:30:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb1e000640 1 -- 172.21.15.46:0/223576801 learned_addr learned my addr 172.21.15.46:0/223576801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb26a00640 1 --2- 172.21.15.46:0/223576801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb2806b470 0x7fcb2815c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb27400640 1 --2- 172.21.15.46:0/223576801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb2815cc50 0x7fcb280b3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb1e000640 1 -- 172.21.15.46:0/223576801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb2815cc50 msgr2=0x7fcb280b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb1e000640 1 --2- 172.21.15.46:0/223576801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb2815cc50 0x7fcb280b3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb1e000640 1 -- 172.21.15.46:0/223576801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb2806b470 msgr2=0x7fcb2815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb1e000640 1 --2- 172.21.15.46:0/223576801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb2806b470 0x7fcb2815c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb1e000640 1 -- 172.21.15.46:0/223576801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb180094d0 con 0x7fcb280709f0 2024-03-29T08:30:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb27400640 1 --2- 172.21.15.46:0/223576801 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fcb2815cc50 0x7fcb280b3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb26a00640 1 --2- 172.21.15.46:0/223576801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb2806b470 0x7fcb2815c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb1e000640 1 --2- 172.21.15.46:0/223576801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb280709f0 0x7fcb28149bc0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000d7d0 tx=0x7fcb1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb1f400640 1 -- 172.21.15.46:0/223576801 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10018070 con 0x7fcb280709f0 2024-03-29T08:30:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb1f400640 1 -- 172.21.15.46:0/223576801 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb10004020 con 0x7fcb280709f0 2024-03-29T08:30:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb1f400640 1 -- 172.21.15.46:0/223576801 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb100041d0 con 0x7fcb280709f0 2024-03-29T08:30:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb2da00640 1 -- 172.21.15.46:0/223576801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2814ac90 con 0x7fcb280709f0 2024-03-29T08:30:25.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb2da00640 1 -- 172.21.15.46:0/223576801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2814a100 con 0x7fcb280709f0 2024-03-29T08:30:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.593+0000 7fcb2da00640 1 -- 172.21.15.46:0/223576801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcb28071230 con 0x7fcb280709f0 2024-03-29T08:30:25.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.597+0000 7fcb1f400640 1 -- 172.21.15.46:0/223576801 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcb10004330 con 0x7fcb280709f0 2024-03-29T08:30:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.597+0000 7fcb1f400640 1 --2- 172.21.15.46:0/223576801 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcb04066690 0x7fcb04068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.597+0000 7fcb1f400640 1 -- 172.21.15.46:0/223576801 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fcb10083ea0 con 0x7fcb280709f0 2024-03-29T08:30:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.597+0000 7fcb1f400640 1 --2- 172.21.15.46:0/223576801 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x7fcb0406c630 0x7fcb0406ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.597+0000 7fcb1f400640 1 -- 172.21.15.46:0/223576801 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb0406f100 con 0x7fcb0406c630 2024-03-29T08:30:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.597+0000 7fcb1f400640 1 -- 172.21.15.46:0/223576801 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb10086480 con 0x7fcb280709f0 2024-03-29T08:30:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.597+0000 7fcb27400640 1 --2- 172.21.15.46:0/223576801 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x7fcb0406c630 0x7fcb0406ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.597+0000 7fcb26a00640 1 --2- 172.21.15.46:0/223576801 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcb04066690 0x7fcb04068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.597+0000 7fcb27400640 1 --2- 172.21.15.46:0/223576801 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x7fcb0406c630 0x7fcb0406ea50 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.597+0000 7fcb26a00640 1 --2- 172.21.15.46:0/223576801 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcb04066690 0x7fcb04068b50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcb180af000 tx=0x7fcb18004740 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:25.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.597+0000 7fcb1f400640 1 -- 172.21.15.46:0/223576801 <== osd.2 v2:172.21.15.46:6809/27428 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fcb0406f100 con 0x7fcb0406c630 2024-03-29T08:30:25.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.613+0000 7fcb2da00640 1 -- 172.21.15.46:0/223576801 --> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcb28160900 con 0x7fcb0406c630 2024-03-29T08:30:25.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.613+0000 7fcb1f400640 1 -- 172.21.15.46:0/223576801 <== osd.2 v2:172.21.15.46:6809/27428 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fcb20001ef0 con 0x7fcb0406c630 2024-03-29T08:30:25.612 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:30:25.612 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T08:30:25.612 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:30:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.613+0000 7fcb2da00640 1 -- 172.21.15.46:0/223576801 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x7fcb0406c630 msgr2=0x7fcb0406ea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.613+0000 7fcb2da00640 1 --2- 172.21.15.46:0/223576801 >> [v2:172.21.15.46:6809/27428,v1:172.21.15.46:6811/27428] conn(0x7fcb0406c630 0x7fcb0406ea50 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.613+0000 7fcb2da00640 1 -- 172.21.15.46:0/223576801 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcb04066690 msgr2=0x7fcb04068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.613+0000 7fcb2da00640 1 --2- 172.21.15.46:0/223576801 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcb04066690 0x7fcb04068b50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcb180af000 tx=0x7fcb18004740 comp rx=0 tx=0).stop 2024-03-29T08:30:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.613+0000 7fcb2da00640 1 -- 172.21.15.46:0/223576801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb280709f0 msgr2=0x7fcb28149bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.613+0000 7fcb2da00640 1 --2- 172.21.15.46:0/223576801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb280709f0 0x7fcb28149bc0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000d7d0 tx=0x7fcb1000dc90 comp rx=0 tx=0).stop 2024-03-29T08:30:25.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.613+0000 7fcb27400640 1 -- 172.21.15.46:0/223576801 reap_dead start 2024-03-29T08:30:25.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.613+0000 7fcb2da00640 1 -- 172.21.15.46:0/223576801 shutdown_connections 2024-03-29T08:30:25.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.613+0000 7fcb2da00640 1 -- 172.21.15.46:0/223576801 >> 172.21.15.46:0/223576801 conn(0x7fcb280ced50 msgr2=0x7fcb28152090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:25.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.613+0000 7fcb2da00640 1 -- 172.21.15.46:0/223576801 shutdown_connections 2024-03-29T08:30:25.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.613+0000 7fcb2da00640 1 -- 172.21.15.46:0/223576801 wait complete. 2024-03-29T08:30:25.631 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-29T08:30:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.773+0000 7fbb5be00640 1 Processor -- start 2024-03-29T08:30:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.773+0000 7fbb5be00640 1 -- start start 2024-03-29T08:30:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.773+0000 7fbb5be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb5415a5d0 0x7fbb540b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.773+0000 7fbb5be00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbb540b1640 0x7fbb540ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.773+0000 7fbb5be00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb540ce040 con 0x7fbb540cd500 2024-03-29T08:30:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.773+0000 7fbb5be00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb540b0ce0 con 0x7fbb540b1640 2024-03-29T08:30:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.773+0000 7fbb5be00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540cda80 con 0x7fbb5415a5d0 2024-03-29T08:30:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.773+0000 7fbb58a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbb540b1640 0x7fbb540ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.773+0000 7fbb58a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbb540b1640 0x7fbb540ccfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38696/0 (socket says 172.21.15.46:38696) 2024-03-29T08:30:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.773+0000 7fbb58a00640 1 -- 172.21.15.46:0/3432524621 learned_addr learned my addr 172.21.15.46:0/3432524621 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.773+0000 7fbb59400640 1 --2- 172.21.15.46:0/3432524621 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb5415a5d0 0x7fbb540b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.773+0000 7fbb58a00640 1 -- 172.21.15.46:0/3432524621 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb5415a5d0 msgr2=0x7fbb540b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.773+0000 7fbb58a00640 1 --2- 172.21.15.46:0/3432524621 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb5415a5d0 0x7fbb540b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.773+0000 7fbb58a00640 1 -- 172.21.15.46:0/3432524621 >> v1:172.21.15.73:6789/0 conn(0x7fbb540cd500 legacy=0x7fbb5406ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.773+0000 7fbb58a00640 1 -- 172.21.15.46:0/3432524621 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb540cdd60 con 0x7fbb540b1640 2024-03-29T08:30:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.777+0000 7fbb58a00640 1 --2- 172.21.15.46:0/3432524621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb540b1640 0x7fbb540ccfc0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009870 tx=0x7fbb4001cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940735acfa8f6e64 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.777+0000 7fbb53e00640 1 -- 172.21.15.46:0/3432524621 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb400aa620 con 0x7fbb540b1640 2024-03-29T08:30:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.777+0000 7fbb53e00640 1 -- 172.21.15.46:0/3432524621 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb400aa780 con 0x7fbb540b1640 2024-03-29T08:30:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.777+0000 7fbb53e00640 1 -- 172.21.15.46:0/3432524621 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb400aab10 con 0x7fbb540b1640 2024-03-29T08:30:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.777+0000 7fbb5be00640 1 -- 172.21.15.46:0/3432524621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb540b1640 msgr2=0x7fbb540ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.777+0000 7fbb5be00640 1 --2- 172.21.15.46:0/3432524621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb540b1640 0x7fbb540ccfc0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009870 tx=0x7fbb4001cf00 comp rx=0 tx=0).stop 2024-03-29T08:30:25.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.777+0000 7fbb5be00640 1 -- 172.21.15.46:0/3432524621 shutdown_connections 2024-03-29T08:30:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.777+0000 7fbb5be00640 1 --2- 172.21.15.46:0/3432524621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb540b1640 0x7fbb540ccfc0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.777+0000 7fbb5be00640 1 --2- 172.21.15.46:0/3432524621 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb5415a5d0 0x7fbb540b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.777+0000 7fbb5be00640 1 -- 172.21.15.46:0/3432524621 >> 172.21.15.46:0/3432524621 conn(0x7fbb54099a10 msgr2=0x7fbb54089fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.777+0000 7fbb5be00640 1 -- 172.21.15.46:0/3432524621 shutdown_connections 2024-03-29T08:30:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.777+0000 7fbb5be00640 1 -- 172.21.15.46:0/3432524621 wait complete. 2024-03-29T08:30:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb5be00640 1 Processor -- start 2024-03-29T08:30:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb5be00640 1 -- start start 2024-03-29T08:30:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb5be00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb540b1640 0x7fbb5415d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb5be00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb540cd500 0x7fbb5415fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb5be00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb5415a5d0 0x7fbb5406e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb5be00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb541600c0 con 0x7fbb5415a5d0 2024-03-29T08:30:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb5be00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540aed70 con 0x7fbb540b1640 2024-03-29T08:30:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb5be00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb54153930 con 0x7fbb540cd500 2024-03-29T08:30:25.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb59e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb5415a5d0 0x7fbb5406e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb59400640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb540b1640 0x7fbb5415d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb59e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb5415a5d0 0x7fbb5406e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38708/0 (socket says 172.21.15.46:38708) 2024-03-29T08:30:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb59e00640 1 -- 172.21.15.46:0/318357449 learned_addr learned my addr 172.21.15.46:0/318357449 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb58a00640 1 --2- 172.21.15.46:0/318357449 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb540cd500 0x7fbb5415fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb59e00640 1 -- 172.21.15.46:0/318357449 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb540cd500 msgr2=0x7fbb5415fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb59e00640 1 --2- 172.21.15.46:0/318357449 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb540cd500 0x7fbb5415fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb59e00640 1 -- 172.21.15.46:0/318357449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb540b1640 msgr2=0x7fbb5415d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb59e00640 1 --2- 172.21.15.46:0/318357449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb540b1640 0x7fbb5415d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb59e00640 1 -- 172.21.15.46:0/318357449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb400094d0 con 0x7fbb5415a5d0 2024-03-29T08:30:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb59e00640 1 --2- 172.21.15.46:0/318357449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb5415a5d0 0x7fbb5406e990 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00c770 tx=0x7fbb4c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:25.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb51600640 1 -- 172.21.15.46:0/318357449 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c003b80 con 0x7fbb5415a5d0 2024-03-29T08:30:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb51600640 1 -- 172.21.15.46:0/318357449 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4c003ce0 con 0x7fbb5415a5d0 2024-03-29T08:30:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb5be00640 1 -- 172.21.15.46:0/318357449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb54153370 con 0x7fbb5415a5d0 2024-03-29T08:30:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb51600640 1 -- 172.21.15.46:0/318357449 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c00aa30 con 0x7fbb5415a5d0 2024-03-29T08:30:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb5be00640 1 -- 172.21.15.46:0/318357449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb540aefa0 con 0x7fbb5415a5d0 2024-03-29T08:30:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.781+0000 7fbb5be00640 1 -- 172.21.15.46:0/318357449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb5409e7d0 con 0x7fbb5415a5d0 2024-03-29T08:30:25.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.785+0000 7fbb51600640 1 -- 172.21.15.46:0/318357449 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fbb4c00ab90 con 0x7fbb5415a5d0 2024-03-29T08:30:25.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.785+0000 7fbb51600640 1 --2- 172.21.15.46:0/318357449 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbb340666c0 0x7fbb34068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:25.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.785+0000 7fbb51600640 1 -- 172.21.15.46:0/318357449 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fbb4c083b90 con 0x7fbb5415a5d0 2024-03-29T08:30:25.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.785+0000 7fbb59400640 1 --2- 172.21.15.46:0/318357449 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbb340666c0 0x7fbb34068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:25.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.785+0000 7fbb59400640 1 --2- 172.21.15.46:0/318357449 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbb340666c0 0x7fbb34068b80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fbb480079c0 tx=0x7fbb48008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:25.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.789+0000 7fbb51600640 1 -- 172.21.15.46:0/318357449 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fbb4c0511f0 con 0x7fbb5415a5d0 2024-03-29T08:30:25.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.949+0000 7fbb5be00640 1 -- 172.21.15.46:0/318357449 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb54169770 con 0x7fbb340666c0 2024-03-29T08:30:25.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.949+0000 7fbb51600640 1 -- 172.21.15.46:0/318357449 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbb480108d0 con 0x7fbb340666c0 2024-03-29T08:30:25.951 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 2 to deep-scrub 2024-03-29T08:30:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.953+0000 7fbb5be00640 1 -- 172.21.15.46:0/318357449 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbb340666c0 msgr2=0x7fbb34068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.953+0000 7fbb5be00640 1 --2- 172.21.15.46:0/318357449 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbb340666c0 0x7fbb34068b80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fbb480079c0 tx=0x7fbb48008040 comp rx=0 tx=0).stop 2024-03-29T08:30:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.953+0000 7fbb5be00640 1 -- 172.21.15.46:0/318357449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb5415a5d0 msgr2=0x7fbb5406e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.953+0000 7fbb5be00640 1 --2- 172.21.15.46:0/318357449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb5415a5d0 0x7fbb5406e990 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00c770 tx=0x7fbb4c00cc30 comp rx=0 tx=0).stop 2024-03-29T08:30:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.953+0000 7fbb5be00640 1 -- 172.21.15.46:0/318357449 shutdown_connections 2024-03-29T08:30:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.953+0000 7fbb5be00640 1 --2- 172.21.15.46:0/318357449 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fbb340666c0 0x7fbb34068b80 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.953+0000 7fbb5be00640 1 --2- 172.21.15.46:0/318357449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb5415a5d0 0x7fbb5406e990 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.953+0000 7fbb5be00640 1 --2- 172.21.15.46:0/318357449 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fbb540cd500 0x7fbb5415fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.953+0000 7fbb5be00640 1 --2- 172.21.15.46:0/318357449 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb540b1640 0x7fbb5415d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.953+0000 7fbb5be00640 1 -- 172.21.15.46:0/318357449 >> 172.21.15.46:0/318357449 conn(0x7fbb54099a10 msgr2=0x7fbb5408a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.953+0000 7fbb5be00640 1 -- 172.21.15.46:0/318357449 shutdown_connections 2024-03-29T08:30:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:25.953+0000 7fbb5be00640 1 -- 172.21.15.46:0/318357449 wait complete. 2024-03-29T08:30:25.973 INFO:tasks.ceph:Scrubbing osd.3 2024-03-29T08:30:25.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-29T08:30:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.117+0000 7fa73a200640 1 Processor -- start 2024-03-29T08:30:26.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.117+0000 7fa73a200640 1 -- start start 2024-03-29T08:30:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.117+0000 7fa73a200640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa73415cbf0 0x7fa734069d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.117+0000 7fa73a200640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7340cff00 0x7fa7340caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.117+0000 7fa73a200640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7340ca640 con 0x7fa73406a2d0 2024-03-29T08:30:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.117+0000 7fa73a200640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7340d03e0 con 0x7fa7340cff00 2024-03-29T08:30:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.117+0000 7fa73a200640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa73406a740 con 0x7fa73415cbf0 2024-03-29T08:30:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.117+0000 7fa733e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7340cff00 0x7fa7340caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.117+0000 7fa733e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa7340cff00 0x7fa7340caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38720/0 (socket says 172.21.15.46:38720) 2024-03-29T08:30:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.117+0000 7fa733e00640 1 -- 172.21.15.46:0/405246470 learned_addr learned my addr 172.21.15.46:0/405246470 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.121+0000 7fa733400640 1 --2- 172.21.15.46:0/405246470 >> v2:172.21.15.73:3300/0 conn(0x7fa73415cbf0 0x7fa734069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.121+0000 7fa733e00640 1 -- 172.21.15.46:0/405246470 >> v2:172.21.15.73:3300/0 conn(0x7fa73415cbf0 msgr2=0x7fa734069d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.121+0000 7fa733e00640 1 --2- 172.21.15.46:0/405246470 >> v2:172.21.15.73:3300/0 conn(0x7fa73415cbf0 0x7fa734069d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.121+0000 7fa733e00640 1 -- 172.21.15.46:0/405246470 >> v1:172.21.15.73:6789/0 conn(0x7fa73406a2d0 legacy=0x7fa7340cf7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.121+0000 7fa733e00640 1 -- 172.21.15.46:0/405246470 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7340cec20 con 0x7fa7340cff00 2024-03-29T08:30:26.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.121+0000 7fa733e00640 1 --2- 172.21.15.46:0/405246470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7340cff00 0x7fa7340caad0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fa7340d0690 tx=0x7fa72c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc5782b33957ad0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:26.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.121+0000 7fa732a00640 1 -- 172.21.15.46:0/405246470 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c0a57f0 con 0x7fa7340cff00 2024-03-29T08:30:26.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.121+0000 7fa732a00640 1 -- 172.21.15.46:0/405246470 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c0a5950 con 0x7fa7340cff00 2024-03-29T08:30:26.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.121+0000 7fa732a00640 1 -- 172.21.15.46:0/405246470 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c0a5ce0 con 0x7fa7340cff00 2024-03-29T08:30:26.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.121+0000 7fa73a200640 1 -- 172.21.15.46:0/405246470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7340cff00 msgr2=0x7fa7340caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.121+0000 7fa73a200640 1 --2- 172.21.15.46:0/405246470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7340cff00 0x7fa7340caad0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fa7340d0690 tx=0x7fa72c01c930 comp rx=0 tx=0).stop 2024-03-29T08:30:26.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa73a200640 1 -- 172.21.15.46:0/405246470 shutdown_connections 2024-03-29T08:30:26.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa73a200640 1 --2- 172.21.15.46:0/405246470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7340cff00 0x7fa7340caad0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa73a200640 1 --2- 172.21.15.46:0/405246470 >> v2:172.21.15.73:3300/0 conn(0x7fa73415cbf0 0x7fa734069d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa73a200640 1 -- 172.21.15.46:0/405246470 >> 172.21.15.46:0/405246470 conn(0x7fa734071810 msgr2=0x7fa734098a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:26.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa73a200640 1 -- 172.21.15.46:0/405246470 shutdown_connections 2024-03-29T08:30:26.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa73a200640 1 -- 172.21.15.46:0/405246470 wait complete. 2024-03-29T08:30:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa73a200640 1 Processor -- start 2024-03-29T08:30:26.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa73a200640 1 -- start start 2024-03-29T08:30:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa73a200640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa73406a2d0 0x7fa734149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa73a200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7340cff00 0x7fa7340c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa73a200640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa73415cbf0 0x7fa7340c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa73a200640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73415b290 con 0x7fa7340cff00 2024-03-29T08:30:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa73a200640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa734156570 con 0x7fa73406a2d0 2024-03-29T08:30:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa73a200640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73415acd0 con 0x7fa73415cbf0 2024-03-29T08:30:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa72aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7340cff00 0x7fa7340c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa72aa00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7340cff00 0x7fa7340c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38728/0 (socket says 172.21.15.46:38728) 2024-03-29T08:30:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa72aa00640 1 -- 172.21.15.46:0/4220366163 learned_addr learned my addr 172.21.15.46:0/4220366163 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa733400640 1 --2- 172.21.15.46:0/4220366163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa73406a2d0 0x7fa734149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa733e00640 1 --2- 172.21.15.46:0/4220366163 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa73415cbf0 0x7fa7340c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa72aa00640 1 -- 172.21.15.46:0/4220366163 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa73415cbf0 msgr2=0x7fa7340c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa72aa00640 1 --2- 172.21.15.46:0/4220366163 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fa73415cbf0 0x7fa7340c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa72aa00640 1 -- 172.21.15.46:0/4220366163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa73406a2d0 msgr2=0x7fa734149790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa72aa00640 1 --2- 172.21.15.46:0/4220366163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa73406a2d0 0x7fa734149790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa72aa00640 1 -- 172.21.15.46:0/4220366163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c009b30 con 0x7fa7340cff00 2024-03-29T08:30:26.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa72aa00640 1 --2- 172.21.15.46:0/4220366163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7340cff00 0x7fa7340c8130 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00c530 tx=0x7fa71c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa72be00640 1 -- 172.21.15.46:0/4220366163 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c005220 con 0x7fa7340cff00 2024-03-29T08:30:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa72be00640 1 -- 172.21.15.46:0/4220366163 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c005380 con 0x7fa7340cff00 2024-03-29T08:30:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa72be00640 1 -- 172.21.15.46:0/4220366163 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c007270 con 0x7fa7340cff00 2024-03-29T08:30:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.125+0000 7fa73a200640 1 -- 172.21.15.46:0/4220366163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73415afb0 con 0x7fa7340cff00 2024-03-29T08:30:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.129+0000 7fa73a200640 1 -- 172.21.15.46:0/4220366163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7340c6bd0 con 0x7fa7340cff00 2024-03-29T08:30:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.129+0000 7fa73a200640 1 -- 172.21.15.46:0/4220366163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa734066c90 con 0x7fa7340cff00 2024-03-29T08:30:26.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.129+0000 7fa72be00640 1 -- 172.21.15.46:0/4220366163 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fa71c0073d0 con 0x7fa7340cff00 2024-03-29T08:30:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.129+0000 7fa72be00640 1 --2- 172.21.15.46:0/4220366163 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa7100666c0 0x7fa710068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.129+0000 7fa72be00640 1 -- 172.21.15.46:0/4220366163 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fa71c083750 con 0x7fa7340cff00 2024-03-29T08:30:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.129+0000 7fa72be00640 1 --2- 172.21.15.46:0/4220366163 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x7fa71006c660 0x7fa71006ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.129+0000 7fa72be00640 1 -- 172.21.15.46:0/4220366163 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa71006f130 con 0x7fa71006c660 2024-03-29T08:30:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.129+0000 7fa72be00640 1 -- 172.21.15.46:0/4220366163 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa71c085d30 con 0x7fa7340cff00 2024-03-29T08:30:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.129+0000 7fa733400640 1 --2- 172.21.15.46:0/4220366163 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa7100666c0 0x7fa710068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.129+0000 7fa733e00640 1 --2- 172.21.15.46:0/4220366163 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x7fa71006c660 0x7fa71006ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.129+0000 7fa733400640 1 --2- 172.21.15.46:0/4220366163 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa7100666c0 0x7fa710068b80 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa7240079c0 tx=0x7fa724008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:26.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.129+0000 7fa733e00640 1 --2- 172.21.15.46:0/4220366163 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x7fa71006c660 0x7fa71006ea80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:26.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.133+0000 7fa72be00640 1 -- 172.21.15.46:0/4220366163 <== osd.3 v2:172.21.15.46:6808/27426 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fa71006f130 con 0x7fa71006c660 2024-03-29T08:30:26.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.145+0000 7fa73a200640 1 -- 172.21.15.46:0/4220366163 --> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa73415d100 con 0x7fa71006c660 2024-03-29T08:30:26.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.145+0000 7fa72be00640 1 -- 172.21.15.46:0/4220366163 <== osd.3 v2:172.21.15.46:6808/27426 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa73415d100 con 0x7fa71006c660 2024-03-29T08:30:26.145 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:30:26.145 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T08:30:26.145 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:30:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.145+0000 7fa73a200640 1 -- 172.21.15.46:0/4220366163 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x7fa71006c660 msgr2=0x7fa71006ea80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.145+0000 7fa73a200640 1 --2- 172.21.15.46:0/4220366163 >> [v2:172.21.15.46:6808/27426,v1:172.21.15.46:6810/27426] conn(0x7fa71006c660 0x7fa71006ea80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.145+0000 7fa73a200640 1 -- 172.21.15.46:0/4220366163 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa7100666c0 msgr2=0x7fa710068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.145+0000 7fa73a200640 1 --2- 172.21.15.46:0/4220366163 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fa7100666c0 0x7fa710068b80 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa7240079c0 tx=0x7fa724008040 comp rx=0 tx=0).stop 2024-03-29T08:30:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.145+0000 7fa73a200640 1 -- 172.21.15.46:0/4220366163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7340cff00 msgr2=0x7fa7340c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.145+0000 7fa73a200640 1 --2- 172.21.15.46:0/4220366163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa7340cff00 0x7fa7340c8130 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fa71c00c530 tx=0x7fa71c00c9f0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.145+0000 7fa733e00640 1 -- 172.21.15.46:0/4220366163 reap_dead start 2024-03-29T08:30:26.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.145+0000 7fa73a200640 1 -- 172.21.15.46:0/4220366163 shutdown_connections 2024-03-29T08:30:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.145+0000 7fa73a200640 1 -- 172.21.15.46:0/4220366163 >> 172.21.15.46:0/4220366163 conn(0x7fa734071810 msgr2=0x7fa73415e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.145+0000 7fa73a200640 1 -- 172.21.15.46:0/4220366163 shutdown_connections 2024-03-29T08:30:26.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.145+0000 7fa73a200640 1 -- 172.21.15.46:0/4220366163 wait complete. 2024-03-29T08:30:26.163 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-29T08:30:26.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb54d000640 1 Processor -- start 2024-03-29T08:30:26.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb54d000640 1 -- start start 2024-03-29T08:30:26.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb54d000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb5480af270 0x7fb5480cb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb54d000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5480d0ff0 0x7fb5480cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb54d000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5480cf3a0 con 0x7fb5480ccd50 2024-03-29T08:30:26.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb54d000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5480cabc0 con 0x7fb5480af270 2024-03-29T08:30:26.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb54d000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480cf5f0 con 0x7fb5480d0ff0 2024-03-29T08:30:26.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb547400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb5480ccd50 0x7fb5480cd130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:57860/0 (socket says 172.21.15.46:57860) 2024-03-29T08:30:26.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb547400640 1 -- 172.21.15.46:0/2497737306 learned_addr learned my addr 172.21.15.46:0/2497737306 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:26.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb53fe00640 1 --2- 172.21.15.46:0/2497737306 >> v2:172.21.15.73:3300/0 conn(0x7fb5480af270 0x7fb5480cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb547e00640 1 --2- 172.21.15.46:0/2497737306 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5480d0ff0 0x7fb5480cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb546a00640 1 -- 172.21.15.46:0/2497737306 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938290022 0 0) 0x7fb5480cf3a0 con 0x7fb5480ccd50 2024-03-29T08:30:26.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb546a00640 1 -- 172.21.15.46:0/2497737306 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb52c003530 con 0x7fb5480ccd50 2024-03-29T08:30:26.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb53fe00640 1 -- 172.21.15.46:0/2497737306 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5480d0ff0 msgr2=0x7fb5480cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:26.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb53fe00640 1 --2- 172.21.15.46:0/2497737306 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5480d0ff0 0x7fb5480cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb546a00640 1 -- 172.21.15.46:0/2497737306 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3354402800 0 0) 0x7fb52c003530 con 0x7fb5480ccd50 2024-03-29T08:30:26.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb53fe00640 1 -- 172.21.15.46:0/2497737306 >> v1:172.21.15.46:6789/0 conn(0x7fb5480ccd50 legacy=0x7fb5480cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.305+0000 7fb53fe00640 1 -- 172.21.15.46:0/2497737306 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5480cf8d0 con 0x7fb5480af270 2024-03-29T08:30:26.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.309+0000 7fb53fe00640 1 --2- 172.21.15.46:0/2497737306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5480af270 0x7fb5480cb070 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb5300097c0 tx=0x7fb53001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=738927c715021347 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:26.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.309+0000 7fb546a00640 1 -- 172.21.15.46:0/2497737306 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538001680 con 0x7fb5480af270 2024-03-29T08:30:26.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.309+0000 7fb546a00640 1 -- 172.21.15.46:0/2497737306 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5300a7050 con 0x7fb5480af270 2024-03-29T08:30:26.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.309+0000 7fb546a00640 1 -- 172.21.15.46:0/2497737306 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5300ab620 con 0x7fb5480af270 2024-03-29T08:30:26.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.309+0000 7fb54d000640 1 -- 172.21.15.46:0/2497737306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5480af270 msgr2=0x7fb5480cb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.309+0000 7fb54d000640 1 --2- 172.21.15.46:0/2497737306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5480af270 0x7fb5480cb070 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb5300097c0 tx=0x7fb53001c3c0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.309+0000 7fb54d000640 1 -- 172.21.15.46:0/2497737306 shutdown_connections 2024-03-29T08:30:26.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.309+0000 7fb54d000640 1 --2- 172.21.15.46:0/2497737306 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5480d0ff0 0x7fb5480cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.309+0000 7fb54d000640 1 --2- 172.21.15.46:0/2497737306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5480af270 0x7fb5480cb070 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.309+0000 7fb54d000640 1 -- 172.21.15.46:0/2497737306 >> 172.21.15.46:0/2497737306 conn(0x7fb5480d04d0 msgr2=0x7fb5480d08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:26.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.309+0000 7fb54d000640 1 -- 172.21.15.46:0/2497737306 shutdown_connections 2024-03-29T08:30:26.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.309+0000 7fb54d000640 1 -- 172.21.15.46:0/2497737306 wait complete. 2024-03-29T08:30:26.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb54d000640 1 Processor -- start 2024-03-29T08:30:26.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb54d000640 1 -- start start 2024-03-29T08:30:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb54d000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5480af270 0x7fb54814d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb54d000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5480ccd50 0x7fb548158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb54d000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5480d0ff0 0x7fb54815b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb54d000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480b0b70 con 0x7fb5480af270 2024-03-29T08:30:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb54d000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54814d590 con 0x7fb5480ccd50 2024-03-29T08:30:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb54d000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480aff60 con 0x7fb5480d0ff0 2024-03-29T08:30:26.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb547400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5480af270 0x7fb54814d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb547400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5480af270 0x7fb54814d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38734/0 (socket says 172.21.15.46:38734) 2024-03-29T08:30:26.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb547400640 1 -- 172.21.15.46:0/3418511223 learned_addr learned my addr 172.21.15.46:0/3418511223 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:26.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb53fe00640 1 --2- 172.21.15.46:0/3418511223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5480ccd50 0x7fb548158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb547e00640 1 --2- 172.21.15.46:0/3418511223 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5480d0ff0 0x7fb54815b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb547400640 1 -- 172.21.15.46:0/3418511223 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5480d0ff0 msgr2=0x7fb54815b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb547400640 1 --2- 172.21.15.46:0/3418511223 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5480d0ff0 0x7fb54815b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb547400640 1 -- 172.21.15.46:0/3418511223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5480ccd50 msgr2=0x7fb548158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb547400640 1 --2- 172.21.15.46:0/3418511223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5480ccd50 0x7fb548158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb547400640 1 -- 172.21.15.46:0/3418511223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb530009420 con 0x7fb5480af270 2024-03-29T08:30:26.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb547400640 1 --2- 172.21.15.46:0/3418511223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5480af270 0x7fb54814d050 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fb53800c780 tx=0x7fb53800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:26.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb53f400640 1 -- 172.21.15.46:0/3418511223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538005cf0 con 0x7fb5480af270 2024-03-29T08:30:26.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb53f400640 1 -- 172.21.15.46:0/3418511223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb538005e50 con 0x7fb5480af270 2024-03-29T08:30:26.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb53f400640 1 -- 172.21.15.46:0/3418511223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5380072b0 con 0x7fb5480af270 2024-03-29T08:30:26.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb54d000640 1 -- 172.21.15.46:0/3418511223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5480b0240 con 0x7fb5480af270 2024-03-29T08:30:26.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.313+0000 7fb54d000640 1 -- 172.21.15.46:0/3418511223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5480b0800 con 0x7fb5480af270 2024-03-29T08:30:26.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.317+0000 7fb54d000640 1 -- 172.21.15.46:0/3418511223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb548067170 con 0x7fb5480af270 2024-03-29T08:30:26.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.317+0000 7fb53f400640 1 -- 172.21.15.46:0/3418511223 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fb538007410 con 0x7fb5480af270 2024-03-29T08:30:26.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.317+0000 7fb53f400640 1 --2- 172.21.15.46:0/3418511223 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb524066690 0x7fb524068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.317+0000 7fb53f400640 1 -- 172.21.15.46:0/3418511223 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fb538083c30 con 0x7fb5480af270 2024-03-29T08:30:26.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.317+0000 7fb53fe00640 1 --2- 172.21.15.46:0/3418511223 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb524066690 0x7fb524068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.317+0000 7fb53fe00640 1 --2- 172.21.15.46:0/3418511223 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb524066690 0x7fb524068b50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb5300ad000 tx=0x7fb5300a6000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:26.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.321+0000 7fb53f400640 1 -- 172.21.15.46:0/3418511223 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fb538051290 con 0x7fb5480af270 2024-03-29T08:30:26.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.481+0000 7fb54d000640 1 -- 172.21.15.46:0/3418511223 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5480b0cb0 con 0x7fb524066690 2024-03-29T08:30:26.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.481+0000 7fb53f400640 1 -- 172.21.15.46:0/3418511223 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb5300ab580 con 0x7fb524066690 2024-03-29T08:30:26.483 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 3 to deep-scrub 2024-03-29T08:30:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.485+0000 7fb54d000640 1 -- 172.21.15.46:0/3418511223 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb524066690 msgr2=0x7fb524068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.485+0000 7fb54d000640 1 --2- 172.21.15.46:0/3418511223 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb524066690 0x7fb524068b50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb5300ad000 tx=0x7fb5300a6000 comp rx=0 tx=0).stop 2024-03-29T08:30:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.485+0000 7fb54d000640 1 -- 172.21.15.46:0/3418511223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5480af270 msgr2=0x7fb54814d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.485+0000 7fb54d000640 1 --2- 172.21.15.46:0/3418511223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5480af270 0x7fb54814d050 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fb53800c780 tx=0x7fb53800cc40 comp rx=0 tx=0).stop 2024-03-29T08:30:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.485+0000 7fb54d000640 1 -- 172.21.15.46:0/3418511223 shutdown_connections 2024-03-29T08:30:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.485+0000 7fb54d000640 1 --2- 172.21.15.46:0/3418511223 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fb524066690 0x7fb524068b50 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.485+0000 7fb54d000640 1 --2- 172.21.15.46:0/3418511223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5480ccd50 0x7fb548158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.485+0000 7fb54d000640 1 --2- 172.21.15.46:0/3418511223 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fb5480d0ff0 0x7fb54815b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.485+0000 7fb54d000640 1 --2- 172.21.15.46:0/3418511223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb5480af270 0x7fb54814d050 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.485+0000 7fb54d000640 1 -- 172.21.15.46:0/3418511223 >> 172.21.15.46:0/3418511223 conn(0x7fb5480d04d0 msgr2=0x7fb54806fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.485+0000 7fb54d000640 1 -- 172.21.15.46:0/3418511223 shutdown_connections 2024-03-29T08:30:26.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.485+0000 7fb54d000640 1 -- 172.21.15.46:0/3418511223 wait complete. 2024-03-29T08:30:26.505 INFO:tasks.ceph:Scrubbing osd.4 2024-03-29T08:30:26.505 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-29T08:30:26.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.649+0000 7f2546a00640 1 Processor -- start 2024-03-29T08:30:26.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.649+0000 7f2546a00640 1 -- start start 2024-03-29T08:30:26.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.649+0000 7f2546a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f254015c870 0x7f25400ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.649+0000 7f2546a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f254006b8b0 0x7f254006bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.649+0000 7f2546a00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f254006c260 con 0x7f254015cc50 2024-03-29T08:30:26.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.649+0000 7f2546a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25400cef50 con 0x7f254015c870 2024-03-29T08:30:26.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.649+0000 7f2546a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25400cf230 con 0x7f254006b8b0 2024-03-29T08:30:26.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.649+0000 7f253fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f254015c870 0x7f25400ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.649+0000 7f253fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f254015c870 0x7f25400ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52742/0 (socket says 172.21.15.46:52742) 2024-03-29T08:30:26.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.649+0000 7f253fe00640 1 -- 172.21.15.46:0/2166164354 learned_addr learned my addr 172.21.15.46:0/2166164354 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:26.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.649+0000 7f2544a00640 1 --2- 172.21.15.46:0/2166164354 >> v2:172.21.15.73:3300/0 conn(0x7f254006b8b0 0x7f254006bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.649+0000 7f253fe00640 1 -- 172.21.15.46:0/2166164354 >> v2:172.21.15.73:3300/0 conn(0x7f254006b8b0 msgr2=0x7f254006bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:26.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.649+0000 7f253fe00640 1 --2- 172.21.15.46:0/2166164354 >> v2:172.21.15.73:3300/0 conn(0x7f254006b8b0 0x7f254006bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.649+0000 7f253fe00640 1 -- 172.21.15.46:0/2166164354 >> v1:172.21.15.73:6789/0 conn(0x7f254015cc50 legacy=0x7f25400997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:26.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.649+0000 7f253fe00640 1 -- 172.21.15.46:0/2166164354 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25400cf510 con 0x7f254015c870 2024-03-29T08:30:26.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.653+0000 7f253fe00640 1 --2- 172.21.15.46:0/2166164354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f254015c870 0x7f25400ce8e0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f25400cacc0 tx=0x7f253001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137c7652d28702a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:26.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.653+0000 7f253f400640 1 -- 172.21.15.46:0/2166164354 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25300ab680 con 0x7f254015c870 2024-03-29T08:30:26.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.653+0000 7f253f400640 1 -- 172.21.15.46:0/2166164354 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25300ab7e0 con 0x7f254015c870 2024-03-29T08:30:26.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.653+0000 7f253f400640 1 -- 172.21.15.46:0/2166164354 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25300abb70 con 0x7f254015c870 2024-03-29T08:30:26.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.653+0000 7f2546a00640 1 -- 172.21.15.46:0/2166164354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f254015c870 msgr2=0x7f25400ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.653+0000 7f2546a00640 1 --2- 172.21.15.46:0/2166164354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f254015c870 0x7f25400ce8e0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f25400cacc0 tx=0x7f253001c460 comp rx=0 tx=0).stop 2024-03-29T08:30:26.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.653+0000 7f2546a00640 1 -- 172.21.15.46:0/2166164354 shutdown_connections 2024-03-29T08:30:26.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.653+0000 7f2546a00640 1 --2- 172.21.15.46:0/2166164354 >> v2:172.21.15.73:3300/0 conn(0x7f254006b8b0 0x7f254006bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.653+0000 7f2546a00640 1 --2- 172.21.15.46:0/2166164354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f254015c870 0x7f25400ce8e0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.653+0000 7f2546a00640 1 -- 172.21.15.46:0/2166164354 >> 172.21.15.46:0/2166164354 conn(0x7f25400b21d0 msgr2=0x7f25400b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:26.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.653+0000 7f2546a00640 1 -- 172.21.15.46:0/2166164354 shutdown_connections 2024-03-29T08:30:26.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.653+0000 7f2546a00640 1 -- 172.21.15.46:0/2166164354 wait complete. 2024-03-29T08:30:26.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f2546a00640 1 Processor -- start 2024-03-29T08:30:26.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f2546a00640 1 -- start start 2024-03-29T08:30:26.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f2546a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f254006b8b0 0x7f2540155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f2546a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f254015c870 0x7f25400b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f2546a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f254015cc50 0x7f25400b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f2546a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25401f9730 con 0x7f254006b8b0 2024-03-29T08:30:26.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f2546a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25401f8ab0 con 0x7f254015c870 2024-03-29T08:30:26.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f2546a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25401f8d90 con 0x7f254015cc50 2024-03-29T08:30:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f253fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f254006b8b0 0x7f2540155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f253fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f254006b8b0 0x7f2540155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52750/0 (socket says 172.21.15.46:52750) 2024-03-29T08:30:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f253fe00640 1 -- 172.21.15.46:0/1398712759 learned_addr learned my addr 172.21.15.46:0/1398712759 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f253fe00640 1 -- 172.21.15.46:0/1398712759 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f254015cc50 msgr2=0x7f25400b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f253fe00640 1 --2- 172.21.15.46:0/1398712759 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f254015cc50 0x7f25400b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f253fe00640 1 -- 172.21.15.46:0/1398712759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f254015c870 msgr2=0x7f25400b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f253fe00640 1 --2- 172.21.15.46:0/1398712759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f254015c870 0x7f25400b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f253fe00640 1 -- 172.21.15.46:0/1398712759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25300094d0 con 0x7f254006b8b0 2024-03-29T08:30:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f253fe00640 1 --2- 172.21.15.46:0/1398712759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f254006b8b0 0x7f2540155200 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f2530019070 tx=0x7f253001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f253cc00640 1 -- 172.21.15.46:0/1398712759 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25300b4800 con 0x7f254006b8b0 2024-03-29T08:30:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f253cc00640 1 -- 172.21.15.46:0/1398712759 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25300b4960 con 0x7f254006b8b0 2024-03-29T08:30:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f253cc00640 1 -- 172.21.15.46:0/1398712759 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25300b4d10 con 0x7f254006b8b0 2024-03-29T08:30:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f2546a00640 1 -- 172.21.15.46:0/1398712759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25401f9070 con 0x7f254006b8b0 2024-03-29T08:30:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.657+0000 7f2546a00640 1 -- 172.21.15.46:0/1398712759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2540155740 con 0x7f254006b8b0 2024-03-29T08:30:26.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.661+0000 7f2546a00640 1 -- 172.21.15.46:0/1398712759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2540066c90 con 0x7f254006b8b0 2024-03-29T08:30:26.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.661+0000 7f253cc00640 1 -- 172.21.15.46:0/1398712759 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f25300b3380 con 0x7f254006b8b0 2024-03-29T08:30:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.661+0000 7f253cc00640 1 --2- 172.21.15.46:0/1398712759 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f251c066690 0x7f251c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.661+0000 7f253cc00640 1 -- 172.21.15.46:0/1398712759 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f253011e9b0 con 0x7f254006b8b0 2024-03-29T08:30:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.661+0000 7f253cc00640 1 --2- 172.21.15.46:0/1398712759 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x7f251c06c630 0x7f251c06ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.661+0000 7f253cc00640 1 -- 172.21.15.46:0/1398712759 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f251c06f100 con 0x7f251c06c630 2024-03-29T08:30:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.661+0000 7f253cc00640 1 -- 172.21.15.46:0/1398712759 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25301232a0 con 0x7f254006b8b0 2024-03-29T08:30:26.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.661+0000 7f2537e00640 1 --2- 172.21.15.46:0/1398712759 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f251c066690 0x7f251c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.661+0000 7f2544a00640 1 --2- 172.21.15.46:0/1398712759 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x7f251c06c630 0x7f251c06ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.661+0000 7f2537e00640 1 --2- 172.21.15.46:0/1398712759 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f251c066690 0x7f251c068b50 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2528000f70 tx=0x7f25280079c0 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:26.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.661+0000 7f2544a00640 1 --2- 172.21.15.46:0/1398712759 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x7f251c06c630 0x7f251c06ea50 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:26.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.665+0000 7f253cc00640 1 -- 172.21.15.46:0/1398712759 <== osd.4 v2:172.21.15.73:6800/27689 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f251c06f100 con 0x7f251c06c630 2024-03-29T08:30:26.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.677+0000 7f2546a00640 1 -- 172.21.15.46:0/1398712759 --> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f25401f9520 con 0x7f251c06c630 2024-03-29T08:30:26.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.677+0000 7f253cc00640 1 -- 172.21.15.46:0/1398712759 <== osd.4 v2:172.21.15.73:6800/27689 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2538009ad0 con 0x7f251c06c630 2024-03-29T08:30:26.677 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:30:26.677 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T08:30:26.677 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:30:26.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.677+0000 7f2546a00640 1 -- 172.21.15.46:0/1398712759 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x7f251c06c630 msgr2=0x7f251c06ea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.677+0000 7f2546a00640 1 --2- 172.21.15.46:0/1398712759 >> [v2:172.21.15.73:6800/27689,v1:172.21.15.73:6801/27689] conn(0x7f251c06c630 0x7f251c06ea50 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.677+0000 7f2546a00640 1 -- 172.21.15.46:0/1398712759 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f251c066690 msgr2=0x7f251c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.677+0000 7f2546a00640 1 --2- 172.21.15.46:0/1398712759 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f251c066690 0x7f251c068b50 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2528000f70 tx=0x7f25280079c0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.677+0000 7f2546a00640 1 -- 172.21.15.46:0/1398712759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f254006b8b0 msgr2=0x7f2540155200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.677+0000 7f2546a00640 1 --2- 172.21.15.46:0/1398712759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f254006b8b0 0x7f2540155200 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f2530019070 tx=0x7f253001aca0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.677+0000 7f2544a00640 1 -- 172.21.15.46:0/1398712759 reap_dead start 2024-03-29T08:30:26.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.677+0000 7f2546a00640 1 -- 172.21.15.46:0/1398712759 shutdown_connections 2024-03-29T08:30:26.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.677+0000 7f2546a00640 1 -- 172.21.15.46:0/1398712759 >> 172.21.15.46:0/1398712759 conn(0x7f25400b21d0 msgr2=0x7f25400cae30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:26.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.677+0000 7f2546a00640 1 -- 172.21.15.46:0/1398712759 shutdown_connections 2024-03-29T08:30:26.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.677+0000 7f2546a00640 1 -- 172.21.15.46:0/1398712759 wait complete. 2024-03-29T08:30:26.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-29T08:30:26.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cc0a00640 1 Processor -- start 2024-03-29T08:30:26.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cc0a00640 1 -- start start 2024-03-29T08:30:26.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cc0a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2cbc0ae250 0x7f2cbc071230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cc0a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2cbc0cd9e0 0x7f2cbc0cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cc0a00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cbc15bef0 con 0x7f2cbc06ada0 2024-03-29T08:30:26.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cc0a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc071770 con 0x7f2cbc0cd9e0 2024-03-29T08:30:26.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cc0a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc15ac30 con 0x7f2cbc0ae250 2024-03-29T08:30:26.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cba800640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2cbc06ada0 0x7f2cbc06b180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53228/0 (socket says 172.21.15.46:53228) 2024-03-29T08:30:26.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cba800640 1 -- 172.21.15.46:0/2749367622 learned_addr learned my addr 172.21.15.46:0/2749367622 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:26.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cb1e00640 1 --2- 172.21.15.46:0/2749367622 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2cbc0ae250 0x7f2cbc071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cbb200640 1 --2- 172.21.15.46:0/2749367622 >> v2:172.21.15.73:3300/0 conn(0x7f2cbc0cd9e0 0x7f2cbc0cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cb9e00640 1 -- 172.21.15.46:0/2749367622 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486549900 0 0) 0x7f2cbc15bef0 con 0x7f2cbc06ada0 2024-03-29T08:30:26.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cb9e00640 1 -- 172.21.15.46:0/2749367622 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ca0003530 con 0x7f2cbc06ada0 2024-03-29T08:30:26.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cbb200640 1 -- 172.21.15.46:0/2749367622 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2cbc0ae250 msgr2=0x7f2cbc071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:26.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cbb200640 1 --2- 172.21.15.46:0/2749367622 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2cbc0ae250 0x7f2cbc071230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cbb200640 1 -- 172.21.15.46:0/2749367622 >> v1:172.21.15.46:6789/0 conn(0x7f2cbc06ada0 legacy=0x7f2cbc06b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cb9e00640 1 -- 172.21.15.46:0/2749367622 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1549407432 0 0) 0x7f2ca0003530 con 0x7f2cbc06ada0 2024-03-29T08:30:26.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.837+0000 7f2cbb200640 1 -- 172.21.15.46:0/2749367622 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc15b9d0 con 0x7f2cbc0cd9e0 2024-03-29T08:30:26.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.841+0000 7f2cbb200640 1 --2- 172.21.15.46:0/2749367622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cbc0cd9e0 0x7f2cbc0cde70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009d90 tx=0x7f2cb401c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7edd41ed64b36187 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:26.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.841+0000 7f2cb9e00640 1 -- 172.21.15.46:0/2749367622 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac001680 con 0x7f2cbc0cd9e0 2024-03-29T08:30:26.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.841+0000 7f2cb9e00640 1 -- 172.21.15.46:0/2749367622 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb40a5660 con 0x7f2cbc0cd9e0 2024-03-29T08:30:26.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.841+0000 7f2cb9e00640 1 -- 172.21.15.46:0/2749367622 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40a59f0 con 0x7f2cbc0cd9e0 2024-03-29T08:30:26.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.841+0000 7f2cc0a00640 1 -- 172.21.15.46:0/2749367622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cbc0cd9e0 msgr2=0x7f2cbc0cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.841+0000 7f2cc0a00640 1 --2- 172.21.15.46:0/2749367622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cbc0cd9e0 0x7f2cbc0cde70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009d90 tx=0x7f2cb401c860 comp rx=0 tx=0).stop 2024-03-29T08:30:26.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.841+0000 7f2cc0a00640 1 -- 172.21.15.46:0/2749367622 shutdown_connections 2024-03-29T08:30:26.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.841+0000 7f2cc0a00640 1 --2- 172.21.15.46:0/2749367622 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2cbc0ae250 0x7f2cbc071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.841+0000 7f2cc0a00640 1 --2- 172.21.15.46:0/2749367622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cbc0cd9e0 0x7f2cbc0cde70 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.841+0000 7f2cc0a00640 1 -- 172.21.15.46:0/2749367622 >> 172.21.15.46:0/2749367622 conn(0x7f2cbc070710 msgr2=0x7f2cbc070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:26.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.841+0000 7f2cc0a00640 1 -- 172.21.15.46:0/2749367622 shutdown_connections 2024-03-29T08:30:26.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.841+0000 7f2cc0a00640 1 -- 172.21.15.46:0/2749367622 wait complete. 2024-03-29T08:30:26.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cc0a00640 1 Processor -- start 2024-03-29T08:30:26.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cc0a00640 1 -- start start 2024-03-29T08:30:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cc0a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cbc06ada0 0x7f2cbc147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cc0a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cbc0ae250 0x7f2cbc152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cc0a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2cbc0cd9e0 0x7f2cbc145650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cc0a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc153820 con 0x7f2cbc0ae250 2024-03-29T08:30:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cc0a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc153b00 con 0x7f2cbc06ada0 2024-03-29T08:30:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cc0a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc14c580 con 0x7f2cbc0cd9e0 2024-03-29T08:30:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cb1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cbc0ae250 0x7f2cbc152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cb1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cbc0ae250 0x7f2cbc152ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52752/0 (socket says 172.21.15.46:52752) 2024-03-29T08:30:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cb1e00640 1 -- 172.21.15.46:0/2883408431 learned_addr learned my addr 172.21.15.46:0/2883408431 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cbb200640 1 --2- 172.21.15.46:0/2883408431 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2cbc0cd9e0 0x7f2cbc145650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cba800640 1 --2- 172.21.15.46:0/2883408431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cbc06ada0 0x7f2cbc147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cb1e00640 1 -- 172.21.15.46:0/2883408431 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2cbc0cd9e0 msgr2=0x7f2cbc145650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cb1e00640 1 --2- 172.21.15.46:0/2883408431 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2cbc0cd9e0 0x7f2cbc145650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cb1e00640 1 -- 172.21.15.46:0/2883408431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cbc06ada0 msgr2=0x7f2cbc147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cb1e00640 1 --2- 172.21.15.46:0/2883408431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cbc06ada0 0x7f2cbc147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cb1e00640 1 -- 172.21.15.46:0/2883408431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb4009a70 con 0x7f2cbc0ae250 2024-03-29T08:30:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cb1e00640 1 --2- 172.21.15.46:0/2883408431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cbc0ae250 0x7f2cbc152ff0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400e930 tx=0x7f2ca400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cb3400640 1 -- 172.21.15.46:0/2883408431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4009690 con 0x7f2cbc0ae250 2024-03-29T08:30:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cb3400640 1 -- 172.21.15.46:0/2883408431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca400ca00 con 0x7f2cbc0ae250 2024-03-29T08:30:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cc0a00640 1 -- 172.21.15.46:0/2883408431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc14c860 con 0x7f2cbc0ae250 2024-03-29T08:30:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cb3400640 1 -- 172.21.15.46:0/2883408431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca400cd70 con 0x7f2cbc0ae250 2024-03-29T08:30:26.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.845+0000 7f2cc0a00640 1 -- 172.21.15.46:0/2883408431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc145b90 con 0x7f2cbc0ae250 2024-03-29T08:30:26.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.849+0000 7f2cc0a00640 1 -- 172.21.15.46:0/2883408431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc04e040 con 0x7f2cbc0ae250 2024-03-29T08:30:26.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.849+0000 7f2cb3400640 1 -- 172.21.15.46:0/2883408431 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f2ca4013470 con 0x7f2cbc0ae250 2024-03-29T08:30:26.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.849+0000 7f2cb3400640 1 --2- 172.21.15.46:0/2883408431 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2c98066690 0x7f2c98068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:26.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.849+0000 7f2cb3400640 1 -- 172.21.15.46:0/2883408431 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f2ca4084140 con 0x7f2cbc0ae250 2024-03-29T08:30:26.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.849+0000 7f2cba800640 1 --2- 172.21.15.46:0/2883408431 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2c98066690 0x7f2c98068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:26.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.849+0000 7f2cba800640 1 --2- 172.21.15.46:0/2883408431 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2c98066690 0x7f2c98068b50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2cac001ec0 tx=0x7f2cac001a60 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:26.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:26.853+0000 7f2cb3400640 1 -- 172.21.15.46:0/2883408431 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f2ca4051870 con 0x7f2cbc0ae250 2024-03-29T08:30:27.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.013+0000 7f2cc0a00640 1 -- 172.21.15.46:0/2883408431 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cbc0ae630 con 0x7f2c98066690 2024-03-29T08:30:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.013+0000 7f2cb3400640 1 -- 172.21.15.46:0/2883408431 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2cac006e50 con 0x7f2c98066690 2024-03-29T08:30:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 4 to deep-scrub 2024-03-29T08:30:27.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.017+0000 7f2cc0a00640 1 -- 172.21.15.46:0/2883408431 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2c98066690 msgr2=0x7f2c98068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.017+0000 7f2cc0a00640 1 --2- 172.21.15.46:0/2883408431 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2c98066690 0x7f2c98068b50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2cac001ec0 tx=0x7f2cac001a60 comp rx=0 tx=0).stop 2024-03-29T08:30:27.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.017+0000 7f2cc0a00640 1 -- 172.21.15.46:0/2883408431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cbc0ae250 msgr2=0x7f2cbc152ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.017+0000 7f2cc0a00640 1 --2- 172.21.15.46:0/2883408431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cbc0ae250 0x7f2cbc152ff0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f2ca400e930 tx=0x7f2ca400edf0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.017+0000 7f2cc0a00640 1 -- 172.21.15.46:0/2883408431 shutdown_connections 2024-03-29T08:30:27.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.017+0000 7f2cc0a00640 1 --2- 172.21.15.46:0/2883408431 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f2c98066690 0x7f2c98068b50 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.017+0000 7f2cc0a00640 1 --2- 172.21.15.46:0/2883408431 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f2cbc0cd9e0 0x7f2cbc145650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.017+0000 7f2cc0a00640 1 --2- 172.21.15.46:0/2883408431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cbc0ae250 0x7f2cbc152ff0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.017+0000 7f2cc0a00640 1 --2- 172.21.15.46:0/2883408431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cbc06ada0 0x7f2cbc147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.017+0000 7f2cc0a00640 1 -- 172.21.15.46:0/2883408431 >> 172.21.15.46:0/2883408431 conn(0x7f2cbc070710 msgr2=0x7f2cbc14f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:27.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.017+0000 7f2cc0a00640 1 -- 172.21.15.46:0/2883408431 shutdown_connections 2024-03-29T08:30:27.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.017+0000 7f2cc0a00640 1 -- 172.21.15.46:0/2883408431 wait complete. 2024-03-29T08:30:27.036 INFO:tasks.ceph:Scrubbing osd.5 2024-03-29T08:30:27.036 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-29T08:30:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.181+0000 7fce0fe00640 1 Processor -- start 2024-03-29T08:30:27.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.181+0000 7fce0fe00640 1 -- start start 2024-03-29T08:30:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.181+0000 7fce0fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fce100cafe0 0x7fce100cb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.181+0000 7fce0fe00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce100b1520 con 0x7fce100cb960 2024-03-29T08:30:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.181+0000 7fce0fe00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce100b1800 con 0x7fce1015ddc0 2024-03-29T08:30:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.181+0000 7fce0fe00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce100b1ae0 con 0x7fce100cafe0 2024-03-29T08:30:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.181+0000 7fce07e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fce100cafe0 0x7fce100cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.181+0000 7fce0f400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fce100cb960 0x7fce100c76a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53232/0 (socket says 172.21.15.46:53232) 2024-03-29T08:30:27.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.181+0000 7fce0f400640 1 -- 172.21.15.46:0/2315384246 learned_addr learned my addr 172.21.15.46:0/2315384246 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.181+0000 7fce07e00640 1 -- 172.21.15.46:0/2315384246 >> v1:172.21.15.73:6789/0 conn(0x7fce1015ddc0 legacy=0x7fce100ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.181+0000 7fce07e00640 1 -- 172.21.15.46:0/2315384246 >> v1:172.21.15.46:6789/0 conn(0x7fce100cb960 legacy=0x7fce100c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.181+0000 7fce07e00640 1 -- 172.21.15.46:0/2315384246 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce100cbe60 con 0x7fce100cafe0 2024-03-29T08:30:27.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.181+0000 7fce0e000640 1 -- 172.21.15.46:0/2315384246 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016643154 0 0) 0x7fce100b1520 con 0x7fce100cb960 2024-03-29T08:30:27.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.185+0000 7fce07e00640 1 --2- 172.21.15.46:0/2315384246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce100cafe0 0x7fce100cb420 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80097c0 tx=0x7fcdf801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ee695014d0b920 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:27.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.185+0000 7fce0e000640 1 -- 172.21.15.46:0/2315384246 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf80a8990 con 0x7fce100cafe0 2024-03-29T08:30:27.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.185+0000 7fce0e000640 1 -- 172.21.15.46:0/2315384246 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf80a8af0 con 0x7fce100cafe0 2024-03-29T08:30:27.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.185+0000 7fce0e000640 1 -- 172.21.15.46:0/2315384246 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf80a8e80 con 0x7fce100cafe0 2024-03-29T08:30:27.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.185+0000 7fce0fe00640 1 -- 172.21.15.46:0/2315384246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce100cafe0 msgr2=0x7fce100cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.185+0000 7fce0fe00640 1 --2- 172.21.15.46:0/2315384246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce100cafe0 0x7fce100cb420 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80097c0 tx=0x7fcdf801ec90 comp rx=0 tx=0).stop 2024-03-29T08:30:27.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.185+0000 7fce0fe00640 1 -- 172.21.15.46:0/2315384246 shutdown_connections 2024-03-29T08:30:27.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.185+0000 7fce0fe00640 1 --2- 172.21.15.46:0/2315384246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce100cafe0 0x7fce100cb420 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.185+0000 7fce0fe00640 1 -- 172.21.15.46:0/2315384246 >> 172.21.15.46:0/2315384246 conn(0x7fce100b1fc0 msgr2=0x7fce100b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:27.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.185+0000 7fce0fe00640 1 -- 172.21.15.46:0/2315384246 shutdown_connections 2024-03-29T08:30:27.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.185+0000 7fce0fe00640 1 -- 172.21.15.46:0/2315384246 wait complete. 2024-03-29T08:30:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce0fe00640 1 Processor -- start 2024-03-29T08:30:27.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce0fe00640 1 -- start start 2024-03-29T08:30:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce0fe00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce100cafe0 0x7fce100b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce0fe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce100cb960 0x7fce100c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce0fe00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce1015ddc0 0x7fce10155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce0fe00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce100c8bf0 con 0x7fce100cb960 2024-03-29T08:30:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce0fe00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce100c8e90 con 0x7fce100cafe0 2024-03-29T08:30:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce0fe00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce100c9170 con 0x7fce1015ddc0 2024-03-29T08:30:27.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce100cb960 0x7fce100c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce07e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce100cb960 0x7fce100c9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52774/0 (socket says 172.21.15.46:52774) 2024-03-29T08:30:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce07e00640 1 -- 172.21.15.46:0/4160358957 learned_addr learned my addr 172.21.15.46:0/4160358957 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce0ea00640 1 --2- 172.21.15.46:0/4160358957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce100cafe0 0x7fce100b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce0f400640 1 --2- 172.21.15.46:0/4160358957 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce1015ddc0 0x7fce10155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce07e00640 1 -- 172.21.15.46:0/4160358957 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce1015ddc0 msgr2=0x7fce10155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce07e00640 1 --2- 172.21.15.46:0/4160358957 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fce1015ddc0 0x7fce10155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce07e00640 1 -- 172.21.15.46:0/4160358957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce100cafe0 msgr2=0x7fce100b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce07e00640 1 --2- 172.21.15.46:0/4160358957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce100cafe0 0x7fce100b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce07e00640 1 -- 172.21.15.46:0/4160358957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf8009420 con 0x7fce100cb960 2024-03-29T08:30:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce07e00640 1 --2- 172.21.15.46:0/4160358957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce100cb960 0x7fce100c9f30 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fcdf801c8c0 tx=0x7fcdf801ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce06a00640 1 -- 172.21.15.46:0/4160358957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf80b0e20 con 0x7fce100cb960 2024-03-29T08:30:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce06a00640 1 -- 172.21.15.46:0/4160358957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf8004030 con 0x7fce100cb960 2024-03-29T08:30:27.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce06a00640 1 -- 172.21.15.46:0/4160358957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf80043a0 con 0x7fce100cb960 2024-03-29T08:30:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce0fe00640 1 -- 172.21.15.46:0/4160358957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce100c9450 con 0x7fce100cb960 2024-03-29T08:30:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce0fe00640 1 -- 172.21.15.46:0/4160358957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce100c9a10 con 0x7fce100cb960 2024-03-29T08:30:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.189+0000 7fce0fe00640 1 -- 172.21.15.46:0/4160358957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fce1015e1a0 con 0x7fce100cb960 2024-03-29T08:30:27.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.193+0000 7fce06a00640 1 -- 172.21.15.46:0/4160358957 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fcdf80b7040 con 0x7fce100cb960 2024-03-29T08:30:27.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.193+0000 7fce06a00640 1 --2- 172.21.15.46:0/4160358957 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcdec066690 0x7fcdec068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.193+0000 7fce06a00640 1 -- 172.21.15.46:0/4160358957 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fcdf81216a0 con 0x7fce100cb960 2024-03-29T08:30:27.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.193+0000 7fce06a00640 1 --2- 172.21.15.46:0/4160358957 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x7fcdec06c630 0x7fcdec06ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.193+0000 7fce06a00640 1 -- 172.21.15.46:0/4160358957 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcdec06f100 con 0x7fcdec06c630 2024-03-29T08:30:27.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.193+0000 7fce06a00640 1 -- 172.21.15.46:0/4160358957 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdf8121a10 con 0x7fce100cb960 2024-03-29T08:30:27.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.193+0000 7fce0ea00640 1 --2- 172.21.15.46:0/4160358957 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcdec066690 0x7fcdec068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.193+0000 7fce0f400640 1 --2- 172.21.15.46:0/4160358957 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x7fcdec06c630 0x7fcdec06ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.193+0000 7fce0ea00640 1 --2- 172.21.15.46:0/4160358957 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcdec066690 0x7fcdec068b50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fce1006fd30 tx=0x7fce00009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:27.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.193+0000 7fce0f400640 1 --2- 172.21.15.46:0/4160358957 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x7fcdec06c630 0x7fcdec06ea50 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:27.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.193+0000 7fce06a00640 1 -- 172.21.15.46:0/4160358957 <== osd.5 v2:172.21.15.73:6808/27690 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7fcdec06f100 con 0x7fcdec06c630 2024-03-29T08:30:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.209+0000 7fce0fe00640 1 -- 172.21.15.46:0/4160358957 --> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fce100c9c00 con 0x7fcdec06c630 2024-03-29T08:30:27.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.209+0000 7fce06a00640 1 -- 172.21.15.46:0/4160358957 <== osd.5 v2:172.21.15.73:6808/27690 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fce100c9c00 con 0x7fcdec06c630 2024-03-29T08:30:27.209 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:30:27.209 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T08:30:27.209 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:30:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.209+0000 7fce0fe00640 1 -- 172.21.15.46:0/4160358957 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x7fcdec06c630 msgr2=0x7fcdec06ea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.209+0000 7fce0fe00640 1 --2- 172.21.15.46:0/4160358957 >> [v2:172.21.15.73:6808/27690,v1:172.21.15.73:6809/27690] conn(0x7fcdec06c630 0x7fcdec06ea50 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.209+0000 7fce0fe00640 1 -- 172.21.15.46:0/4160358957 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcdec066690 msgr2=0x7fcdec068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.209+0000 7fce0fe00640 1 --2- 172.21.15.46:0/4160358957 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fcdec066690 0x7fcdec068b50 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fce1006fd30 tx=0x7fce00009040 comp rx=0 tx=0).stop 2024-03-29T08:30:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.209+0000 7fce0fe00640 1 -- 172.21.15.46:0/4160358957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce100cb960 msgr2=0x7fce100c9f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.209+0000 7fce0fe00640 1 --2- 172.21.15.46:0/4160358957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce100cb960 0x7fce100c9f30 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fcdf801c8c0 tx=0x7fcdf801ba30 comp rx=0 tx=0).stop 2024-03-29T08:30:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.209+0000 7fce0f400640 1 -- 172.21.15.46:0/4160358957 reap_dead start 2024-03-29T08:30:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.209+0000 7fce0fe00640 1 -- 172.21.15.46:0/4160358957 shutdown_connections 2024-03-29T08:30:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.209+0000 7fce0fe00640 1 -- 172.21.15.46:0/4160358957 >> 172.21.15.46:0/4160358957 conn(0x7fce100b1fc0 msgr2=0x7fce1015b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.209+0000 7fce0fe00640 1 -- 172.21.15.46:0/4160358957 shutdown_connections 2024-03-29T08:30:27.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.209+0000 7fce0fe00640 1 -- 172.21.15.46:0/4160358957 wait complete. 2024-03-29T08:30:27.227 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-29T08:30:27.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.369+0000 7f15d1e00640 1 Processor -- start 2024-03-29T08:30:27.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.369+0000 7f15d1e00640 1 -- start start 2024-03-29T08:30:27.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.369+0000 7f15d1e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15cc142ab0 0x7f15cc06d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.369+0000 7f15d1e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f15cc15db10 0x7f15cc0b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.369+0000 7f15d1e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15cc06de40 con 0x7f15cc06d9e0 2024-03-29T08:30:27.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.369+0000 7f15d1e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15cc070c70 con 0x7f15cc15db10 2024-03-29T08:30:27.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.369+0000 7f15d1e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc070340 con 0x7f15cc142ab0 2024-03-29T08:30:27.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.369+0000 7f15c3e00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f15cc06d9e0 0x7f15cc15d400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53240/0 (socket says 172.21.15.46:53240) 2024-03-29T08:30:27.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.369+0000 7f15c3e00640 1 -- 172.21.15.46:0/3066768002 learned_addr learned my addr 172.21.15.46:0/3066768002 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:27.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.369+0000 7f15caa00640 1 --2- 172.21.15.46:0/3066768002 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15cc142ab0 0x7f15cc06d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.369+0000 7f15cb400640 1 --2- 172.21.15.46:0/3066768002 >> v2:172.21.15.73:3300/0 conn(0x7f15cc15db10 0x7f15cc0b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.369+0000 7f15ca000640 1 -- 172.21.15.46:0/3066768002 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400458384 0 0) 0x7f15cc06de40 con 0x7f15cc06d9e0 2024-03-29T08:30:27.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.369+0000 7f15ca000640 1 -- 172.21.15.46:0/3066768002 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15b0003530 con 0x7f15cc06d9e0 2024-03-29T08:30:27.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.369+0000 7f15caa00640 1 -- 172.21.15.46:0/3066768002 >> v2:172.21.15.73:3300/0 conn(0x7f15cc15db10 msgr2=0x7f15cc0b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:27.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.369+0000 7f15caa00640 1 --2- 172.21.15.46:0/3066768002 >> v2:172.21.15.73:3300/0 conn(0x7f15cc15db10 0x7f15cc0b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.369+0000 7f15caa00640 1 -- 172.21.15.46:0/3066768002 >> v1:172.21.15.46:6789/0 conn(0x7f15cc06d9e0 legacy=0x7f15cc15d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.369+0000 7f15caa00640 1 -- 172.21.15.46:0/3066768002 --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc070620 con 0x7f15cc142ab0 2024-03-29T08:30:27.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.373+0000 7f15caa00640 1 --2- 172.21.15.46:0/3066768002 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15cc142ab0 0x7f15cc06d4a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0097c0 tx=0x7f15bc01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17db06207972c104 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:27.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.373+0000 7f15ca000640 1 -- 172.21.15.46:0/3066768002 <== mon.2 v2:172.21.15.73:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc0a4e80 con 0x7f15cc142ab0 2024-03-29T08:30:27.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.373+0000 7f15ca000640 1 -- 172.21.15.46:0/3066768002 <== mon.2 v2:172.21.15.73:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15bc0ac440 con 0x7f15cc142ab0 2024-03-29T08:30:27.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.373+0000 7f15ca000640 1 -- 172.21.15.46:0/3066768002 <== mon.2 v2:172.21.15.73:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc0ac7b0 con 0x7f15cc142ab0 2024-03-29T08:30:27.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.373+0000 7f15d1e00640 1 -- 172.21.15.46:0/3066768002 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15cc142ab0 msgr2=0x7f15cc06d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.373+0000 7f15d1e00640 1 --2- 172.21.15.46:0/3066768002 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15cc142ab0 0x7f15cc06d4a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0097c0 tx=0x7f15bc01c4a0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.373+0000 7f15d1e00640 1 -- 172.21.15.46:0/3066768002 shutdown_connections 2024-03-29T08:30:27.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.373+0000 7f15d1e00640 1 --2- 172.21.15.46:0/3066768002 >> v2:172.21.15.73:3300/0 conn(0x7f15cc15db10 0x7f15cc0b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.373+0000 7f15d1e00640 1 --2- 172.21.15.46:0/3066768002 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15cc142ab0 0x7f15cc06d4a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.373+0000 7f15d1e00640 1 -- 172.21.15.46:0/3066768002 >> 172.21.15.46:0/3066768002 conn(0x7f15cc066080 msgr2=0x7f15cc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:27.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.373+0000 7f15d1e00640 1 -- 172.21.15.46:0/3066768002 shutdown_connections 2024-03-29T08:30:27.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.373+0000 7f15d1e00640 1 -- 172.21.15.46:0/3066768002 wait complete. 2024-03-29T08:30:27.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15d1e00640 1 Processor -- start 2024-03-29T08:30:27.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15d1e00640 1 -- start start 2024-03-29T08:30:27.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15d1e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15cc06d9e0 0x7f15cc138520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15d1e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15cc142ab0 0x7f15cc156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15d1e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15cc15db10 0x7f15cc156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15d1e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc13cf20 con 0x7f15cc142ab0 2024-03-29T08:30:27.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15d1e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc138a60 con 0x7f15cc06d9e0 2024-03-29T08:30:27.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15d1e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc139630 con 0x7f15cc15db10 2024-03-29T08:30:27.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15cc142ab0 0x7f15cc156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15c3e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15cc142ab0 0x7f15cc156810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52784/0 (socket says 172.21.15.46:52784) 2024-03-29T08:30:27.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15c3e00640 1 -- 172.21.15.46:0/2012127241 learned_addr learned my addr 172.21.15.46:0/2012127241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:27.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15caa00640 1 --2- 172.21.15.46:0/2012127241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15cc06d9e0 0x7f15cc138520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15cb400640 1 --2- 172.21.15.46:0/2012127241 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15cc15db10 0x7f15cc156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15c3e00640 1 -- 172.21.15.46:0/2012127241 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15cc15db10 msgr2=0x7f15cc156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15c3e00640 1 --2- 172.21.15.46:0/2012127241 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15cc15db10 0x7f15cc156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15c3e00640 1 -- 172.21.15.46:0/2012127241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15cc06d9e0 msgr2=0x7f15cc138520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15c3e00640 1 --2- 172.21.15.46:0/2012127241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15cc06d9e0 0x7f15cc138520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15c3e00640 1 -- 172.21.15.46:0/2012127241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc009420 con 0x7f15cc142ab0 2024-03-29T08:30:27.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15c3e00640 1 --2- 172.21.15.46:0/2012127241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15cc142ab0 0x7f15cc156810 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f15b400c4d0 tx=0x7f15b400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:27.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15c3400640 1 -- 172.21.15.46:0/2012127241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4005cf0 con 0x7f15cc142ab0 2024-03-29T08:30:27.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15c3400640 1 -- 172.21.15.46:0/2012127241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b4005e50 con 0x7f15cc142ab0 2024-03-29T08:30:27.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15c3400640 1 -- 172.21.15.46:0/2012127241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4007270 con 0x7f15cc142ab0 2024-03-29T08:30:27.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15d1e00640 1 -- 172.21.15.46:0/2012127241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15cc139910 con 0x7f15cc142ab0 2024-03-29T08:30:27.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15d1e00640 1 -- 172.21.15.46:0/2012127241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15cc13cbb0 con 0x7f15cc142ab0 2024-03-29T08:30:27.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.377+0000 7f15d1e00640 1 -- 172.21.15.46:0/2012127241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15cc071a90 con 0x7f15cc142ab0 2024-03-29T08:30:27.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.381+0000 7f15c3400640 1 -- 172.21.15.46:0/2012127241 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f15b40073d0 con 0x7f15cc142ab0 2024-03-29T08:30:27.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.381+0000 7f15c3400640 1 --2- 172.21.15.46:0/2012127241 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15a8066690 0x7f15a8068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.381+0000 7f15c3400640 1 -- 172.21.15.46:0/2012127241 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f15b4083570 con 0x7f15cc142ab0 2024-03-29T08:30:27.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.381+0000 7f15caa00640 1 --2- 172.21.15.46:0/2012127241 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15a8066690 0x7f15a8068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.381+0000 7f15caa00640 1 --2- 172.21.15.46:0/2012127241 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15a8066690 0x7f15a8068b50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0ae000 tx=0x7f15bc008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:27.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.385+0000 7f15c3400640 1 -- 172.21.15.46:0/2012127241 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f15b4050ab0 con 0x7f15cc142ab0 2024-03-29T08:30:27.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.549+0000 7f15d1e00640 1 -- 172.21.15.46:0/2012127241 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f15cc071310 con 0x7f15a8066690 2024-03-29T08:30:27.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.549+0000 7f15c3400640 1 -- 172.21.15.46:0/2012127241 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f15bc0ac7f0 con 0x7f15a8066690 2024-03-29T08:30:27.549 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 5 to deep-scrub 2024-03-29T08:30:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.549+0000 7f15d1e00640 1 -- 172.21.15.46:0/2012127241 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15a8066690 msgr2=0x7f15a8068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.549+0000 7f15d1e00640 1 --2- 172.21.15.46:0/2012127241 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15a8066690 0x7f15a8068b50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f15bc0ae000 tx=0x7f15bc008040 comp rx=0 tx=0).stop 2024-03-29T08:30:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.549+0000 7f15d1e00640 1 -- 172.21.15.46:0/2012127241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15cc142ab0 msgr2=0x7f15cc156810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.549+0000 7f15d1e00640 1 --2- 172.21.15.46:0/2012127241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15cc142ab0 0x7f15cc156810 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f15b400c4d0 tx=0x7f15b400c990 comp rx=0 tx=0).stop 2024-03-29T08:30:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.549+0000 7f15d1e00640 1 -- 172.21.15.46:0/2012127241 shutdown_connections 2024-03-29T08:30:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.549+0000 7f15d1e00640 1 --2- 172.21.15.46:0/2012127241 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f15a8066690 0x7f15a8068b50 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.549+0000 7f15d1e00640 1 --2- 172.21.15.46:0/2012127241 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f15cc15db10 0x7f15cc156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.549+0000 7f15d1e00640 1 --2- 172.21.15.46:0/2012127241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15cc142ab0 0x7f15cc156810 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.549+0000 7f15d1e00640 1 --2- 172.21.15.46:0/2012127241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15cc06d9e0 0x7f15cc138520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.549+0000 7f15d1e00640 1 -- 172.21.15.46:0/2012127241 >> 172.21.15.46:0/2012127241 conn(0x7f15cc066080 msgr2=0x7f15cc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:27.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.549+0000 7f15d1e00640 1 -- 172.21.15.46:0/2012127241 shutdown_connections 2024-03-29T08:30:27.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.549+0000 7f15d1e00640 1 -- 172.21.15.46:0/2012127241 wait complete. 2024-03-29T08:30:27.570 INFO:tasks.ceph:Scrubbing osd.6 2024-03-29T08:30:27.571 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-29T08:30:27.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.713+0000 7f18bf000640 1 Processor -- start 2024-03-29T08:30:27.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.713+0000 7f18bf000640 1 -- start start 2024-03-29T08:30:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.717+0000 7f18bf000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f18b81598a0 0x7f18b80cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.717+0000 7f18bf000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f18b80b1470 0x7f18b815d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.717+0000 7f18bf000640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b815de80 con 0x7f18b815da60 2024-03-29T08:30:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.717+0000 7f18bf000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b80b1bc0 con 0x7f18b81598a0 2024-03-29T08:30:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.717+0000 7f18bf000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b80b1ea0 con 0x7f18b80b1470 2024-03-29T08:30:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.717+0000 7f18b7e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f18b81598a0 0x7f18b80cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.717+0000 7f18bd000640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f18b815da60 0x7f18b8159c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53252/0 (socket says 172.21.15.46:53252) 2024-03-29T08:30:27.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.717+0000 7f18bd000640 1 -- 172.21.15.46:0/509038471 learned_addr learned my addr 172.21.15.46:0/509038471 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:27.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.717+0000 7f18af400640 1 --2- 172.21.15.46:0/509038471 >> v2:172.21.15.73:3300/0 conn(0x7f18b80b1470 0x7f18b815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.717+0000 7f18b7400640 1 -- 172.21.15.46:0/509038471 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944005055 0 0) 0x7f18b815de80 con 0x7f18b815da60 2024-03-29T08:30:27.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.717+0000 7f18b7400640 1 -- 172.21.15.46:0/509038471 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f189c003530 con 0x7f18b815da60 2024-03-29T08:30:27.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.717+0000 7f18b7e00640 1 -- 172.21.15.46:0/509038471 >> v2:172.21.15.73:3300/0 conn(0x7f18b80b1470 msgr2=0x7f18b815d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:27.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.717+0000 7f18b7e00640 1 --2- 172.21.15.46:0/509038471 >> v2:172.21.15.73:3300/0 conn(0x7f18b80b1470 0x7f18b815d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.717+0000 7f18b7e00640 1 -- 172.21.15.46:0/509038471 >> v1:172.21.15.46:6789/0 conn(0x7f18b815da60 legacy=0x7f18b8159c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.717+0000 7f18b7e00640 1 -- 172.21.15.46:0/509038471 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b80b2180 con 0x7f18b81598a0 2024-03-29T08:30:27.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.717+0000 7f18b7e00640 1 --2- 172.21.15.46:0/509038471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18b81598a0 0x7f18b80cadb0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f18b80b28a0 tx=0x7f18a801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d6d450e0bf2764 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:27.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18b7400640 1 -- 172.21.15.46:0/509038471 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a80ab650 con 0x7f18b81598a0 2024-03-29T08:30:27.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18b7400640 1 -- 172.21.15.46:0/509038471 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a80ab7b0 con 0x7f18b81598a0 2024-03-29T08:30:27.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18b7400640 1 -- 172.21.15.46:0/509038471 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a80abb40 con 0x7f18b81598a0 2024-03-29T08:30:27.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18bf000640 1 -- 172.21.15.46:0/509038471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18b81598a0 msgr2=0x7f18b80cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18bf000640 1 --2- 172.21.15.46:0/509038471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18b81598a0 0x7f18b80cadb0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f18b80b28a0 tx=0x7f18a801c450 comp rx=0 tx=0).stop 2024-03-29T08:30:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18bf000640 1 -- 172.21.15.46:0/509038471 shutdown_connections 2024-03-29T08:30:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18bf000640 1 --2- 172.21.15.46:0/509038471 >> v2:172.21.15.73:3300/0 conn(0x7f18b80b1470 0x7f18b815d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18bf000640 1 --2- 172.21.15.46:0/509038471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18b81598a0 0x7f18b80cadb0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18bf000640 1 -- 172.21.15.46:0/509038471 >> 172.21.15.46:0/509038471 conn(0x7f18b80b42d0 msgr2=0x7f18b8069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18bf000640 1 -- 172.21.15.46:0/509038471 shutdown_connections 2024-03-29T08:30:27.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18bf000640 1 -- 172.21.15.46:0/509038471 wait complete. 2024-03-29T08:30:27.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18bf000640 1 Processor -- start 2024-03-29T08:30:27.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18bf000640 1 -- start start 2024-03-29T08:30:27.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18bf000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18b80b1470 0x7f18b8152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18bf000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18b81598a0 0x7f18b8166460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18bf000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f18b815da60 0x7f18b815b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18bf000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b8166ee0 con 0x7f18b80b1470 2024-03-29T08:30:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18bf000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b81671c0 con 0x7f18b81598a0 2024-03-29T08:30:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18bf000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b81674a0 con 0x7f18b815da60 2024-03-29T08:30:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18b80b1470 0x7f18b8152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18b7e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18b80b1470 0x7f18b8152130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52800/0 (socket says 172.21.15.46:52800) 2024-03-29T08:30:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18b7e00640 1 -- 172.21.15.46:0/1014564256 learned_addr learned my addr 172.21.15.46:0/1014564256 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18b7e00640 1 -- 172.21.15.46:0/1014564256 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f18b815da60 msgr2=0x7f18b815b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18b7e00640 1 --2- 172.21.15.46:0/1014564256 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f18b815da60 0x7f18b815b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18b7e00640 1 -- 172.21.15.46:0/1014564256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18b81598a0 msgr2=0x7f18b8166460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18b7e00640 1 --2- 172.21.15.46:0/1014564256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18b81598a0 0x7f18b8166460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.721+0000 7f18b7e00640 1 -- 172.21.15.46:0/1014564256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a8009420 con 0x7f18b80b1470 2024-03-29T08:30:27.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.725+0000 7f18b7e00640 1 --2- 172.21.15.46:0/1014564256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18b80b1470 0x7f18b8152130 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f18a80ad040 tx=0x7f18a801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.725+0000 7f18b4c00640 1 -- 172.21.15.46:0/1014564256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a80bf610 con 0x7f18b80b1470 2024-03-29T08:30:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.725+0000 7f18b4c00640 1 -- 172.21.15.46:0/1014564256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a80bf7b0 con 0x7f18b80b1470 2024-03-29T08:30:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.725+0000 7f18b4c00640 1 -- 172.21.15.46:0/1014564256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a80bfb60 con 0x7f18b80b1470 2024-03-29T08:30:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.725+0000 7f18bf000640 1 -- 172.21.15.46:0/1014564256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b8167780 con 0x7f18b80b1470 2024-03-29T08:30:27.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.725+0000 7f18bf000640 1 -- 172.21.15.46:0/1014564256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b81535d0 con 0x7f18b80b1470 2024-03-29T08:30:27.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.725+0000 7f18bf000640 1 -- 172.21.15.46:0/1014564256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f18b81565a0 con 0x7f18b80b1470 2024-03-29T08:30:27.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.725+0000 7f18b4c00640 1 -- 172.21.15.46:0/1014564256 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f18a80b5400 con 0x7f18b80b1470 2024-03-29T08:30:27.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.725+0000 7f18b4c00640 1 --2- 172.21.15.46:0/1014564256 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1894066690 0x7f1894068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.725+0000 7f18b4c00640 1 -- 172.21.15.46:0/1014564256 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f18a811ec80 con 0x7f18b80b1470 2024-03-29T08:30:27.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.725+0000 7f18b4c00640 1 --2- 172.21.15.46:0/1014564256 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x7f189406c630 0x7f189406ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.725+0000 7f18b4c00640 1 -- 172.21.15.46:0/1014564256 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f189406f100 con 0x7f189406c630 2024-03-29T08:30:27.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.725+0000 7f18b4c00640 1 -- 172.21.15.46:0/1014564256 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18a81232a0 con 0x7f18b80b1470 2024-03-29T08:30:27.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.729+0000 7f18af400640 1 --2- 172.21.15.46:0/1014564256 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1894066690 0x7f1894068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.729+0000 7f18bd000640 1 --2- 172.21.15.46:0/1014564256 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x7f189406c630 0x7f189406ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.729+0000 7f18af400640 1 --2- 172.21.15.46:0/1014564256 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1894066690 0x7f1894068b50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f18a0006aa0 tx=0x7f18a0006a30 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:27.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.729+0000 7f18bd000640 1 --2- 172.21.15.46:0/1014564256 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x7f189406c630 0x7f189406ea50 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:27.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.729+0000 7f18b4c00640 1 -- 172.21.15.46:0/1014564256 <== osd.6 v2:172.21.15.73:6824/27691 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f189406f100 con 0x7f189406c630 2024-03-29T08:30:27.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.741+0000 7f18bf000640 1 -- 172.21.15.46:0/1014564256 --> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f18b8156720 con 0x7f189406c630 2024-03-29T08:30:27.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.741+0000 7f18b4c00640 1 -- 172.21.15.46:0/1014564256 <== osd.6 v2:172.21.15.73:6824/27691 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f18b8156720 con 0x7f189406c630 2024-03-29T08:30:27.742 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:30:27.742 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T08:30:27.742 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:30:27.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.741+0000 7f18bf000640 1 -- 172.21.15.46:0/1014564256 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x7f189406c630 msgr2=0x7f189406ea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.741+0000 7f18bf000640 1 --2- 172.21.15.46:0/1014564256 >> [v2:172.21.15.73:6824/27691,v1:172.21.15.73:6825/27691] conn(0x7f189406c630 0x7f189406ea50 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.741+0000 7f18bf000640 1 -- 172.21.15.46:0/1014564256 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1894066690 msgr2=0x7f1894068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.741+0000 7f18bf000640 1 --2- 172.21.15.46:0/1014564256 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1894066690 0x7f1894068b50 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f18a0006aa0 tx=0x7f18a0006a30 comp rx=0 tx=0).stop 2024-03-29T08:30:27.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.741+0000 7f18bf000640 1 -- 172.21.15.46:0/1014564256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18b80b1470 msgr2=0x7f18b8152130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.741+0000 7f18bf000640 1 --2- 172.21.15.46:0/1014564256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f18b80b1470 0x7f18b8152130 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f18a80ad040 tx=0x7f18a801ac90 comp rx=0 tx=0).stop 2024-03-29T08:30:27.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.741+0000 7f18bd000640 1 -- 172.21.15.46:0/1014564256 reap_dead start 2024-03-29T08:30:27.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.741+0000 7f18bf000640 1 -- 172.21.15.46:0/1014564256 shutdown_connections 2024-03-29T08:30:27.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.741+0000 7f18bf000640 1 -- 172.21.15.46:0/1014564256 >> 172.21.15.46:0/1014564256 conn(0x7f18b80b42d0 msgr2=0x7f18b80af950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:27.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.741+0000 7f18bf000640 1 -- 172.21.15.46:0/1014564256 shutdown_connections 2024-03-29T08:30:27.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.741+0000 7f18bf000640 1 -- 172.21.15.46:0/1014564256 wait complete. 2024-03-29T08:30:27.760 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-29T08:30:27.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.901+0000 7f1fa3a00640 1 Processor -- start 2024-03-29T08:30:27.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.901+0000 7f1fa3a00640 1 -- start start 2024-03-29T08:30:27.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.901+0000 7f1fa3a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f9c1461f0 0x7f1f9c0702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.901+0000 7f1fa3a00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1f9c0707e0 0x7f1f9c0b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.901+0000 7f1fa3a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1f9c0b3270 0x7f1f9c06c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.901+0000 7f1fa3a00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9c1465d0 con 0x7f1f9c0b3270 2024-03-29T08:30:27.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.901+0000 7f1fa3a00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9c1604b0 con 0x7f1f9c0707e0 2024-03-29T08:30:27.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.901+0000 7f1fa3a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c070c20 con 0x7f1f9c1461f0 2024-03-29T08:30:27.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.901+0000 7f1fa1a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1f9c0b3270 0x7f1f9c06c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.901+0000 7f1fa1a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1f9c0b3270 0x7f1f9c06c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52802/0 (socket says 172.21.15.46:52802) 2024-03-29T08:30:27.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.901+0000 7f1fa1a00640 1 -- 172.21.15.46:0/3127056006 learned_addr learned my addr 172.21.15.46:0/3127056006 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:27.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.901+0000 7f1fa1000640 1 --2- 172.21.15.46:0/3127056006 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f9c1461f0 0x7f1f9c0702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.901+0000 7f1f93e00640 1 --2- 172.21.15.46:0/3127056006 >> v2:172.21.15.73:3300/0 conn(0x7f1f9c0707e0 0x7f1f9c0b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1fa1a00640 1 -- 172.21.15.46:0/3127056006 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f9c1461f0 msgr2=0x7f1f9c0702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:27.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1fa1a00640 1 --2- 172.21.15.46:0/3127056006 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f9c1461f0 0x7f1f9c0702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1fa1a00640 1 -- 172.21.15.46:0/3127056006 >> v2:172.21.15.73:3300/0 conn(0x7f1f9c0707e0 msgr2=0x7f1f9c0b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:27.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1fa1a00640 1 --2- 172.21.15.46:0/3127056006 >> v2:172.21.15.73:3300/0 conn(0x7f1f9c0707e0 0x7f1f9c0b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1fa1a00640 1 -- 172.21.15.46:0/3127056006 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c160700 con 0x7f1f9c0b3270 2024-03-29T08:30:27.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1fa1a00640 1 --2- 172.21.15.46:0/3127056006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c0b3270 0x7f1f9c06c390 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c071740 tx=0x7f1f9401c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ce018bcc587722 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:27.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1f9be00640 1 -- 172.21.15.46:0/3127056006 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f940a6500 con 0x7f1f9c0b3270 2024-03-29T08:30:27.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1f9be00640 1 -- 172.21.15.46:0/3127056006 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f940a6660 con 0x7f1f9c0b3270 2024-03-29T08:30:27.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1f9be00640 1 -- 172.21.15.46:0/3127056006 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f940a69f0 con 0x7f1f9c0b3270 2024-03-29T08:30:27.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1fa3a00640 1 -- 172.21.15.46:0/3127056006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c0b3270 msgr2=0x7f1f9c06c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1fa3a00640 1 --2- 172.21.15.46:0/3127056006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c0b3270 0x7f1f9c06c390 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c071740 tx=0x7f1f9401c670 comp rx=0 tx=0).stop 2024-03-29T08:30:27.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1fa3a00640 1 -- 172.21.15.46:0/3127056006 shutdown_connections 2024-03-29T08:30:27.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1fa3a00640 1 --2- 172.21.15.46:0/3127056006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c0b3270 0x7f1f9c06c390 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1fa3a00640 1 --2- 172.21.15.46:0/3127056006 >> v2:172.21.15.73:3300/0 conn(0x7f1f9c0707e0 0x7f1f9c0b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1fa3a00640 1 --2- 172.21.15.46:0/3127056006 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f9c1461f0 0x7f1f9c0702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1fa3a00640 1 -- 172.21.15.46:0/3127056006 >> 172.21.15.46:0/3127056006 conn(0x7f1f9c066080 msgr2=0x7f1f9c069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:27.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1fa3a00640 1 -- 172.21.15.46:0/3127056006 shutdown_connections 2024-03-29T08:30:27.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.905+0000 7f1fa3a00640 1 -- 172.21.15.46:0/3127056006 wait complete. 2024-03-29T08:30:27.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa3a00640 1 Processor -- start 2024-03-29T08:30:27.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa3a00640 1 -- start start 2024-03-29T08:30:27.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa3a00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c0707e0 0x7f1f9c0ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa3a00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f9c0b3270 0x7f1f9c1414d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa3a00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f9c1461f0 0x7f1f9c1f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa3a00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c141c10 con 0x7f1f9c0707e0 2024-03-29T08:30:27.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa3a00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c0adf40 con 0x7f1f9c1461f0 2024-03-29T08:30:27.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa3a00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c141a10 con 0x7f1f9c0b3270 2024-03-29T08:30:27.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c0707e0 0x7f1f9c0ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa1000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c0707e0 0x7f1f9c0ada00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52818/0 (socket says 172.21.15.46:52818) 2024-03-29T08:30:27.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa1000640 1 -- 172.21.15.46:0/2962433374 learned_addr learned my addr 172.21.15.46:0/2962433374 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:27.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1f93e00640 1 --2- 172.21.15.46:0/2962433374 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f9c0b3270 0x7f1f9c1414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa1a00640 1 --2- 172.21.15.46:0/2962433374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f9c1461f0 0x7f1f9c1f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa1000640 1 -- 172.21.15.46:0/2962433374 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f9c0b3270 msgr2=0x7f1f9c1414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa1000640 1 --2- 172.21.15.46:0/2962433374 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f9c0b3270 0x7f1f9c1414d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa1000640 1 -- 172.21.15.46:0/2962433374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f9c1461f0 msgr2=0x7f1f9c1f58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:27.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa1000640 1 --2- 172.21.15.46:0/2962433374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f9c1461f0 0x7f1f9c1f58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:27.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa1000640 1 -- 172.21.15.46:0/2962433374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f94009b30 con 0x7f1f9c0707e0 2024-03-29T08:30:27.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1f93e00640 1 --2- 172.21.15.46:0/2962433374 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f9c0b3270 0x7f1f9c1414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:27.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa1000640 1 --2- 172.21.15.46:0/2962433374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c0707e0 0x7f1f9c0ada00 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00d620 tx=0x7f1f8c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:27.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1f99600640 1 -- 172.21.15.46:0/2962433374 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c00b620 con 0x7f1f9c0707e0 2024-03-29T08:30:27.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1f99600640 1 -- 172.21.15.46:0/2962433374 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8c00b780 con 0x7f1f9c0707e0 2024-03-29T08:30:27.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa3a00640 1 -- 172.21.15.46:0/2962433374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9c1f62c0 con 0x7f1f9c0707e0 2024-03-29T08:30:27.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1f99600640 1 -- 172.21.15.46:0/2962433374 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c00bb10 con 0x7f1f9c0707e0 2024-03-29T08:30:27.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.909+0000 7f1fa3a00640 1 -- 172.21.15.46:0/2962433374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9c1f6880 con 0x7f1f9c0707e0 2024-03-29T08:30:27.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.913+0000 7f1fa3a00640 1 -- 172.21.15.46:0/2962433374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f9c1f6a70 con 0x7f1f9c0707e0 2024-03-29T08:30:27.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.913+0000 7f1f99600640 1 -- 172.21.15.46:0/2962433374 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1f8c00bc70 con 0x7f1f9c0707e0 2024-03-29T08:30:27.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.913+0000 7f1f99600640 1 --2- 172.21.15.46:0/2962433374 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f78066690 0x7f1f78068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:27.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.913+0000 7f1f99600640 1 -- 172.21.15.46:0/2962433374 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f1f8c084410 con 0x7f1f9c0707e0 2024-03-29T08:30:27.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.913+0000 7f1f93e00640 1 --2- 172.21.15.46:0/2962433374 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f78066690 0x7f1f78068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:27.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.913+0000 7f1f93e00640 1 --2- 172.21.15.46:0/2962433374 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f78066690 0x7f1f78068b50 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1f84004660 tx=0x7f1f84008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:27.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:27.917+0000 7f1f99600640 1 -- 172.21.15.46:0/2962433374 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f1f8c051930 con 0x7f1f9c0707e0 2024-03-29T08:30:28.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.081+0000 7f1fa3a00640 1 -- 172.21.15.46:0/2962433374 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f9c0b36e0 con 0x7f1f78066690 2024-03-29T08:30:28.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.081+0000 7f1f99600640 1 -- 172.21.15.46:0/2962433374 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1f8400f830 con 0x7f1f78066690 2024-03-29T08:30:28.081 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 6 to deep-scrub 2024-03-29T08:30:28.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.081+0000 7f1fa3a00640 1 -- 172.21.15.46:0/2962433374 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f78066690 msgr2=0x7f1f78068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.081+0000 7f1fa3a00640 1 --2- 172.21.15.46:0/2962433374 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f78066690 0x7f1f78068b50 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1f84004660 tx=0x7f1f84008040 comp rx=0 tx=0).stop 2024-03-29T08:30:28.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.081+0000 7f1fa3a00640 1 -- 172.21.15.46:0/2962433374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c0707e0 msgr2=0x7f1f9c0ada00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.081+0000 7f1fa3a00640 1 --2- 172.21.15.46:0/2962433374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c0707e0 0x7f1f9c0ada00 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00d620 tx=0x7f1f8c00dae0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.081+0000 7f1fa3a00640 1 -- 172.21.15.46:0/2962433374 shutdown_connections 2024-03-29T08:30:28.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.081+0000 7f1fa3a00640 1 --2- 172.21.15.46:0/2962433374 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1f78066690 0x7f1f78068b50 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.081+0000 7f1fa3a00640 1 --2- 172.21.15.46:0/2962433374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f9c1461f0 0x7f1f9c1f58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.081+0000 7f1fa3a00640 1 --2- 172.21.15.46:0/2962433374 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1f9c0b3270 0x7f1f9c1414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.081+0000 7f1fa3a00640 1 --2- 172.21.15.46:0/2962433374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1f9c0707e0 0x7f1f9c0ada00 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.081+0000 7f1fa3a00640 1 -- 172.21.15.46:0/2962433374 >> 172.21.15.46:0/2962433374 conn(0x7f1f9c066080 msgr2=0x7f1f9c0cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:28.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.081+0000 7f1fa3a00640 1 -- 172.21.15.46:0/2962433374 shutdown_connections 2024-03-29T08:30:28.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.081+0000 7f1fa3a00640 1 -- 172.21.15.46:0/2962433374 wait complete. 2024-03-29T08:30:28.103 INFO:tasks.ceph:Scrubbing osd.7 2024-03-29T08:30:28.103 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-29T08:30:28.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.245+0000 7f1ec4600640 1 Processor -- start 2024-03-29T08:30:28.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.245+0000 7f1ec4600640 1 -- start start 2024-03-29T08:30:28.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.249+0000 7f1ec4600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1ebc0cec80 0x7f1ebc141770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.249+0000 7f1ec4600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1ebc141cb0 0x7f1ebc06db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.249+0000 7f1ec4600640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ebc06cd80 con 0x7f1ebc140f40 2024-03-29T08:30:28.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.249+0000 7f1ec4600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ebc0cbff0 con 0x7f1ebc0cec80 2024-03-29T08:30:28.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.249+0000 7f1ec4600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc06d060 con 0x7f1ebc141cb0 2024-03-29T08:30:28.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.249+0000 7f1ec1c00640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1ebc140f40 0x7f1ebc0cb8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53266/0 (socket says 172.21.15.46:53266) 2024-03-29T08:30:28.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.249+0000 7f1ec1c00640 1 -- 172.21.15.46:0/3540150415 learned_addr learned my addr 172.21.15.46:0/3540150415 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:28.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.249+0000 7f1ec1200640 1 --2- 172.21.15.46:0/3540150415 >> v2:172.21.15.46:3300/0 conn(0x7f1ebc0cec80 0x7f1ebc141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.249+0000 7f1ebbe00640 1 -- 172.21.15.46:0/3540150415 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102960418 0 0) 0x7f1ebc06cd80 con 0x7f1ebc140f40 2024-03-29T08:30:28.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.249+0000 7f1ec2600640 1 --2- 172.21.15.46:0/3540150415 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1ebc141cb0 0x7f1ebc06db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.249+0000 7f1ebbe00640 1 -- 172.21.15.46:0/3540150415 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ea4003530 con 0x7f1ebc140f40 2024-03-29T08:30:28.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.249+0000 7f1ec1200640 1 -- 172.21.15.46:0/3540150415 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1ebc141cb0 msgr2=0x7f1ebc06db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:28.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.249+0000 7f1ec1200640 1 --2- 172.21.15.46:0/3540150415 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1ebc141cb0 0x7f1ebc06db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.249+0000 7f1ec1200640 1 -- 172.21.15.46:0/3540150415 >> v1:172.21.15.46:6789/0 conn(0x7f1ebc140f40 legacy=0x7f1ebc0cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.249+0000 7f1ec1200640 1 -- 172.21.15.46:0/3540150415 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc06d340 con 0x7f1ebc0cec80 2024-03-29T08:30:28.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.249+0000 7f1ec2600640 1 --2- 172.21.15.46:0/3540150415 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1ebc141cb0 0x7f1ebc06db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:28.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec1200640 1 --2- 172.21.15.46:0/3540150415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ebc0cec80 0x7f1ebc141770 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80097c0 tx=0x7f1ea801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7992743515ce7b44 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:28.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ebbe00640 1 -- 172.21.15.46:0/3540150415 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea80ad680 con 0x7f1ebc0cec80 2024-03-29T08:30:28.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ebbe00640 1 -- 172.21.15.46:0/3540150415 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea80ad7e0 con 0x7f1ebc0cec80 2024-03-29T08:30:28.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ebbe00640 1 -- 172.21.15.46:0/3540150415 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea80adb70 con 0x7f1ebc0cec80 2024-03-29T08:30:28.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec4600640 1 -- 172.21.15.46:0/3540150415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ebc0cec80 msgr2=0x7f1ebc141770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec4600640 1 --2- 172.21.15.46:0/3540150415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ebc0cec80 0x7f1ebc141770 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80097c0 tx=0x7f1ea801c470 comp rx=0 tx=0).stop 2024-03-29T08:30:28.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec4600640 1 -- 172.21.15.46:0/3540150415 shutdown_connections 2024-03-29T08:30:28.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec4600640 1 --2- 172.21.15.46:0/3540150415 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1ebc141cb0 0x7f1ebc06db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec4600640 1 --2- 172.21.15.46:0/3540150415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ebc0cec80 0x7f1ebc141770 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec4600640 1 -- 172.21.15.46:0/3540150415 >> 172.21.15.46:0/3540150415 conn(0x7f1ebc0b42d0 msgr2=0x7f1ebc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:28.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec4600640 1 -- 172.21.15.46:0/3540150415 shutdown_connections 2024-03-29T08:30:28.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec4600640 1 -- 172.21.15.46:0/3540150415 wait complete. 2024-03-29T08:30:28.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec4600640 1 Processor -- start 2024-03-29T08:30:28.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec4600640 1 -- start start 2024-03-29T08:30:28.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec4600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1ebc0cec80 0x7f1ebc15dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec4600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ebc140f40 0x7f1ebc13fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec4600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ebc141cb0 0x7f1ebc13d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec4600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc140540 con 0x7f1ebc140f40 2024-03-29T08:30:28.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec4600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc139b50 con 0x7f1ebc141cb0 2024-03-29T08:30:28.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec4600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc139e30 con 0x7f1ebc0cec80 2024-03-29T08:30:28.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ebc140f40 0x7f1ebc13fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec1200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ebc140f40 0x7f1ebc13fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52828/0 (socket says 172.21.15.46:52828) 2024-03-29T08:30:28.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.253+0000 7f1ec1200640 1 -- 172.21.15.46:0/3078943827 learned_addr learned my addr 172.21.15.46:0/3078943827 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:28.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1ec2600640 1 --2- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ebc141cb0 0x7f1ebc13d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1ec1c00640 1 --2- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1ebc0cec80 0x7f1ebc15dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1ec1200640 1 -- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1ebc0cec80 msgr2=0x7f1ebc15dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1ec1200640 1 --2- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1ebc0cec80 0x7f1ebc15dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1ec1200640 1 -- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ebc141cb0 msgr2=0x7f1ebc13d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1ec1200640 1 --2- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ebc141cb0 0x7f1ebc13d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1ec1200640 1 -- 172.21.15.46:0/3078943827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea8009420 con 0x7f1ebc140f40 2024-03-29T08:30:28.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1ec2600640 1 --2- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ebc141cb0 0x7f1ebc13d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:28.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1ec1c00640 1 --2- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f1ebc0cec80 0x7f1ebc15dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-29T08:30:28.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1ec1200640 1 --2- 172.21.15.46:0/3078943827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ebc140f40 0x7f1ebc13fc40 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f1ea801ab20 tx=0x7f1ea801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:28.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1eb9600640 1 -- 172.21.15.46:0/3078943827 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea80add60 con 0x7f1ebc140f40 2024-03-29T08:30:28.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1eb9600640 1 -- 172.21.15.46:0/3078943827 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea80b8450 con 0x7f1ebc140f40 2024-03-29T08:30:28.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1ec4600640 1 -- 172.21.15.46:0/3078943827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc13a110 con 0x7f1ebc140f40 2024-03-29T08:30:28.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1eb9600640 1 -- 172.21.15.46:0/3078943827 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea80b8800 con 0x7f1ebc140f40 2024-03-29T08:30:28.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1ec4600640 1 -- 172.21.15.46:0/3078943827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc140180 con 0x7f1ebc140f40 2024-03-29T08:30:28.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1ec4600640 1 -- 172.21.15.46:0/3078943827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1ebc167650 con 0x7f1ebc140f40 2024-03-29T08:30:28.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1eb9600640 1 -- 172.21.15.46:0/3078943827 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f1ea80b4070 con 0x7f1ebc140f40 2024-03-29T08:30:28.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1eb9600640 1 --2- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1e9c066690 0x7f1e9c068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1eb9600640 1 -- 172.21.15.46:0/3078943827 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f1ea811ea50 con 0x7f1ebc140f40 2024-03-29T08:30:28.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1eb9600640 1 --2- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x7f1e9c06c630 0x7f1e9c06ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1eb9600640 1 -- 172.21.15.46:0/3078943827 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e9c06f100 con 0x7f1e9c06c630 2024-03-29T08:30:28.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1eb9600640 1 -- 172.21.15.46:0/3078943827 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ea811edc0 con 0x7f1ebc140f40 2024-03-29T08:30:28.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.257+0000 7f1ec1c00640 1 --2- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1e9c066690 0x7f1e9c068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.261+0000 7f1ec2600640 1 --2- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x7f1e9c06c630 0x7f1e9c06ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.261+0000 7f1ec2600640 1 --2- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x7f1e9c06c630 0x7f1e9c06ea50 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:28.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.261+0000 7f1ec1c00640 1 --2- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1e9c066690 0x7f1e9c068b50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00011f0 tx=0x7f1eb0009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:28.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.261+0000 7f1eb9600640 1 -- 172.21.15.46:0/3078943827 <== osd.7 v2:172.21.15.73:6816/27700 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25254 (crc 0 0 0) 0x7f1e9c06f100 con 0x7f1e9c06c630 2024-03-29T08:30:28.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.273+0000 7f1ec4600640 1 -- 172.21.15.46:0/3078943827 --> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1ebc066c90 con 0x7f1e9c06c630 2024-03-29T08:30:28.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.273+0000 7f1eb9600640 1 -- 172.21.15.46:0/3078943827 <== osd.7 v2:172.21.15.73:6816/27700 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1ebc066c90 con 0x7f1e9c06c630 2024-03-29T08:30:28.274 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-03-29T08:30:28.275 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-29T08:30:28.275 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-03-29T08:30:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.273+0000 7f1ec4600640 1 -- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x7f1e9c06c630 msgr2=0x7f1e9c06ea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.273+0000 7f1ec4600640 1 --2- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:6816/27700,v1:172.21.15.73:6817/27700] conn(0x7f1e9c06c630 0x7f1e9c06ea50 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.273+0000 7f1ec4600640 1 -- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1e9c066690 msgr2=0x7f1e9c068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.273+0000 7f1ec4600640 1 --2- 172.21.15.46:0/3078943827 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f1e9c066690 0x7f1e9c068b50 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f1eb00011f0 tx=0x7f1eb0009040 comp rx=0 tx=0).stop 2024-03-29T08:30:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.273+0000 7f1ec4600640 1 -- 172.21.15.46:0/3078943827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ebc140f40 msgr2=0x7f1ebc13fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.273+0000 7f1ec4600640 1 --2- 172.21.15.46:0/3078943827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1ebc140f40 0x7f1ebc13fc40 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f1ea801ab20 tx=0x7f1ea801acb0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.273+0000 7f1ec2600640 1 -- 172.21.15.46:0/3078943827 reap_dead start 2024-03-29T08:30:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.273+0000 7f1ec4600640 1 -- 172.21.15.46:0/3078943827 shutdown_connections 2024-03-29T08:30:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.273+0000 7f1ec4600640 1 -- 172.21.15.46:0/3078943827 >> 172.21.15.46:0/3078943827 conn(0x7f1ebc0b42d0 msgr2=0x7f1ebc0af860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.273+0000 7f1ec4600640 1 -- 172.21.15.46:0/3078943827 shutdown_connections 2024-03-29T08:30:28.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.273+0000 7f1ec4600640 1 -- 172.21.15.46:0/3078943827 wait complete. 2024-03-29T08:30:28.293 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-29T08:30:28.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.433+0000 7fd20f600640 1 Processor -- start 2024-03-29T08:30:28.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.433+0000 7fd20f600640 1 -- start start 2024-03-29T08:30:28.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.437+0000 7fd20f600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd208142ab0 0x7fd20806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.437+0000 7fd20f600640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd20806d9e0 0x7fd20815d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.437+0000 7fd20f600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd20815d940 0x7fd2080ae210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.437+0000 7fd20f600640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd20806de40 con 0x7fd20815d940 2024-03-29T08:30:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.437+0000 7fd20f600640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd208070c70 con 0x7fd20806d9e0 2024-03-29T08:30:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.437+0000 7fd20f600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd208070340 con 0x7fd208142ab0 2024-03-29T08:30:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.437+0000 7fd20d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd20815d940 0x7fd2080ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.437+0000 7fd20d600640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd20815d940 0x7fd2080ae210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52836/0 (socket says 172.21.15.46:52836) 2024-03-29T08:30:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.437+0000 7fd20d600640 1 -- 172.21.15.46:0/2491012530 learned_addr learned my addr 172.21.15.46:0/2491012530 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.437+0000 7fd20cc00640 1 --2- 172.21.15.46:0/2491012530 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd208142ab0 0x7fd20806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.437+0000 7fd1ffe00640 1 --2- 172.21.15.46:0/2491012530 >> v2:172.21.15.73:3300/0 conn(0x7fd20806d9e0 0x7fd20815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.437+0000 7fd20d600640 1 -- 172.21.15.46:0/2491012530 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd208142ab0 msgr2=0x7fd20806d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:28.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.437+0000 7fd20d600640 1 --2- 172.21.15.46:0/2491012530 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd208142ab0 0x7fd20806d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.437+0000 7fd20d600640 1 -- 172.21.15.46:0/2491012530 >> v2:172.21.15.73:3300/0 conn(0x7fd20806d9e0 msgr2=0x7fd20815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:28.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.437+0000 7fd20d600640 1 --2- 172.21.15.46:0/2491012530 >> v2:172.21.15.73:3300/0 conn(0x7fd20806d9e0 0x7fd20815d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.437+0000 7fd20d600640 1 -- 172.21.15.46:0/2491012530 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd208070620 con 0x7fd20815d940 2024-03-29T08:30:28.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20d600640 1 --2- 172.21.15.46:0/2491012530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd20815d940 0x7fd2080ae210 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fd20806ad70 tx=0x7fd20001e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6f9b1f34a508f6 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:28.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd207e00640 1 -- 172.21.15.46:0/2491012530 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20000b2d0 con 0x7fd20815d940 2024-03-29T08:30:28.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd207e00640 1 -- 172.21.15.46:0/2491012530 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20000b430 con 0x7fd20815d940 2024-03-29T08:30:28.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd207e00640 1 -- 172.21.15.46:0/2491012530 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2000a7870 con 0x7fd20815d940 2024-03-29T08:30:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20f600640 1 -- 172.21.15.46:0/2491012530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd20815d940 msgr2=0x7fd2080ae210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20f600640 1 --2- 172.21.15.46:0/2491012530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd20815d940 0x7fd2080ae210 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fd20806ad70 tx=0x7fd20001e930 comp rx=0 tx=0).stop 2024-03-29T08:30:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20f600640 1 -- 172.21.15.46:0/2491012530 shutdown_connections 2024-03-29T08:30:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20f600640 1 --2- 172.21.15.46:0/2491012530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd20815d940 0x7fd2080ae210 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20f600640 1 --2- 172.21.15.46:0/2491012530 >> v2:172.21.15.73:3300/0 conn(0x7fd20806d9e0 0x7fd20815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20f600640 1 --2- 172.21.15.46:0/2491012530 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd208142ab0 0x7fd20806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20f600640 1 -- 172.21.15.46:0/2491012530 >> 172.21.15.46:0/2491012530 conn(0x7fd208066080 msgr2=0x7fd208069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20f600640 1 -- 172.21.15.46:0/2491012530 shutdown_connections 2024-03-29T08:30:28.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20f600640 1 -- 172.21.15.46:0/2491012530 wait complete. 2024-03-29T08:30:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20f600640 1 Processor -- start 2024-03-29T08:30:28.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20f600640 1 -- start start 2024-03-29T08:30:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20f600640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd20806d9e0 0x7fd2080ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20f600640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd208142ab0 0x7fd2080ab300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20f600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd20815d940 0x7fd2080cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20f600640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2080ab970 con 0x7fd208142ab0 2024-03-29T08:30:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20f600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2080abc50 con 0x7fd20815d940 2024-03-29T08:30:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20f600640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2080abf30 con 0x7fd20806d9e0 2024-03-29T08:30:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd1ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd208142ab0 0x7fd2080ab300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd1ffe00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd208142ab0 0x7fd2080ab300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52850/0 (socket says 172.21.15.46:52850) 2024-03-29T08:30:28.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd1ffe00640 1 -- 172.21.15.46:0/3745298161 learned_addr learned my addr 172.21.15.46:0/3745298161 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20cc00640 1 --2- 172.21.15.46:0/3745298161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd20806d9e0 0x7fd2080ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.441+0000 7fd20d600640 1 --2- 172.21.15.46:0/3745298161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd20815d940 0x7fd2080cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.445+0000 7fd1ffe00640 1 -- 172.21.15.46:0/3745298161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd20806d9e0 msgr2=0x7fd2080ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.445+0000 7fd1ffe00640 1 --2- 172.21.15.46:0/3745298161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd20806d9e0 0x7fd2080ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.445+0000 7fd1ffe00640 1 -- 172.21.15.46:0/3745298161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd20815d940 msgr2=0x7fd2080cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.445+0000 7fd1ffe00640 1 --2- 172.21.15.46:0/3745298161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd20815d940 0x7fd2080cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.445+0000 7fd1ffe00640 1 -- 172.21.15.46:0/3745298161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20000bd40 con 0x7fd208142ab0 2024-03-29T08:30:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.445+0000 7fd20d600640 1 --2- 172.21.15.46:0/3745298161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd20815d940 0x7fd2080cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.445+0000 7fd1ffe00640 1 --2- 172.21.15.46:0/3745298161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd208142ab0 0x7fd2080ab300 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000d800 tx=0x7fd1f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.445+0000 7fd205600640 1 -- 172.21.15.46:0/3745298161 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f000b6c0 con 0x7fd208142ab0 2024-03-29T08:30:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.445+0000 7fd205600640 1 -- 172.21.15.46:0/3745298161 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f000b820 con 0x7fd208142ab0 2024-03-29T08:30:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.445+0000 7fd20f600640 1 -- 172.21.15.46:0/3745298161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2080ac210 con 0x7fd208142ab0 2024-03-29T08:30:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.445+0000 7fd205600640 1 -- 172.21.15.46:0/3745298161 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f000bb90 con 0x7fd208142ab0 2024-03-29T08:30:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.445+0000 7fd20f600640 1 -- 172.21.15.46:0/3745298161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd208141eb0 con 0x7fd208142ab0 2024-03-29T08:30:28.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.445+0000 7fd20f600640 1 -- 172.21.15.46:0/3745298161 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd208071a90 con 0x7fd208142ab0 2024-03-29T08:30:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.445+0000 7fd205600640 1 -- 172.21.15.46:0/3745298161 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7fd1f000bcf0 con 0x7fd208142ab0 2024-03-29T08:30:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.445+0000 7fd205600640 1 --2- 172.21.15.46:0/3745298161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd1e4066690 0x7fd1e4068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.445+0000 7fd205600640 1 -- 172.21.15.46:0/3745298161 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fd1f0084470 con 0x7fd208142ab0 2024-03-29T08:30:28.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.445+0000 7fd20cc00640 1 --2- 172.21.15.46:0/3745298161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd1e4066690 0x7fd1e4068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.449+0000 7fd20cc00640 1 --2- 172.21.15.46:0/3745298161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd1e4066690 0x7fd1e4068b50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80079e0 tx=0x7fd1f8008040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:28.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.449+0000 7fd205600640 1 -- 172.21.15.46:0/3745298161 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7fd1f0051a20 con 0x7fd208142ab0 2024-03-29T08:30:28.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.609+0000 7fd20f600640 1 -- 172.21.15.46:0/3745298161 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fd20806c390 con 0x7fd1e4066690 2024-03-29T08:30:28.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.609+0000 7fd205600640 1 -- 172.21.15.46:0/3745298161 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd1f80108d0 con 0x7fd1e4066690 2024-03-29T08:30:28.611 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 7 to deep-scrub 2024-03-29T08:30:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.613+0000 7fd20f600640 1 -- 172.21.15.46:0/3745298161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd1e4066690 msgr2=0x7fd1e4068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.613+0000 7fd20f600640 1 --2- 172.21.15.46:0/3745298161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd1e4066690 0x7fd1e4068b50 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80079e0 tx=0x7fd1f8008040 comp rx=0 tx=0).stop 2024-03-29T08:30:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.613+0000 7fd20f600640 1 -- 172.21.15.46:0/3745298161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd208142ab0 msgr2=0x7fd2080ab300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.613+0000 7fd20f600640 1 --2- 172.21.15.46:0/3745298161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd208142ab0 0x7fd2080ab300 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000d800 tx=0x7fd1f000dcc0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.613+0000 7fd20f600640 1 -- 172.21.15.46:0/3745298161 shutdown_connections 2024-03-29T08:30:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.613+0000 7fd20f600640 1 --2- 172.21.15.46:0/3745298161 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7fd1e4066690 0x7fd1e4068b50 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.613+0000 7fd20f600640 1 --2- 172.21.15.46:0/3745298161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd20815d940 0x7fd2080cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.613+0000 7fd20f600640 1 --2- 172.21.15.46:0/3745298161 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd208142ab0 0x7fd2080ab300 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.613+0000 7fd20f600640 1 --2- 172.21.15.46:0/3745298161 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7fd20806d9e0 0x7fd2080ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.613+0000 7fd20f600640 1 -- 172.21.15.46:0/3745298161 >> 172.21.15.46:0/3745298161 conn(0x7fd208066080 msgr2=0x7fd208069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.613+0000 7fd20f600640 1 -- 172.21.15.46:0/3745298161 shutdown_connections 2024-03-29T08:30:28.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.613+0000 7fd20f600640 1 -- 172.21.15.46:0/3745298161 wait complete. 2024-03-29T08:30:28.633 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:30:28.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.773+0000 7f576fc00640 1 Processor -- start 2024-03-29T08:30:28.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.773+0000 7f576fc00640 1 -- start start 2024-03-29T08:30:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.773+0000 7f576fc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5768098a40 0x7f576815ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.773+0000 7f576fc00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f57680ce490 0x7f5768098e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.773+0000 7f576fc00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5768099360 con 0x7f5768099790 2024-03-29T08:30:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.773+0000 7f576fc00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576815b370 con 0x7f57680ce490 2024-03-29T08:30:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.773+0000 7f576fc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576815b650 con 0x7f5768098a40 2024-03-29T08:30:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.773+0000 7f575fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f57680ce490 0x7f5768098e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.773+0000 7f575fe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f57680ce490 0x7f5768098e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52854/0 (socket says 172.21.15.46:52854) 2024-03-29T08:30:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.773+0000 7f575fe00640 1 -- 172.21.15.46:0/3521015957 learned_addr learned my addr 172.21.15.46:0/3521015957 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:28.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.773+0000 7f576d200640 1 --2- 172.21.15.46:0/3521015957 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5768098a40 0x7f576815ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.773+0000 7f575fe00640 1 -- 172.21.15.46:0/3521015957 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5768098a40 msgr2=0x7f576815ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:28.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.773+0000 7f575fe00640 1 --2- 172.21.15.46:0/3521015957 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5768098a40 0x7f576815ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.773+0000 7f575fe00640 1 -- 172.21.15.46:0/3521015957 >> v1:172.21.15.46:6789/0 conn(0x7f5768099790 legacy=0x7f576815e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.773+0000 7f575fe00640 1 -- 172.21.15.46:0/3521015957 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576815b930 con 0x7f57680ce490 2024-03-29T08:30:28.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.773+0000 7f5767e00640 1 -- 172.21.15.46:0/3521015957 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18832181 0 0) 0x7f5768099360 con 0x7f5768099790 2024-03-29T08:30:28.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.777+0000 7f575fe00640 1 --2- 172.21.15.46:0/3521015957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57680ce490 0x7f5768098e20 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f5750009870 tx=0x7f575001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4372b7ff3351f87e server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.777+0000 7f5767e00640 1 -- 172.21.15.46:0/3521015957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57500ab640 con 0x7f57680ce490 2024-03-29T08:30:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.777+0000 7f5767e00640 1 -- 172.21.15.46:0/3521015957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57500a7070 con 0x7f57680ce490 2024-03-29T08:30:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.777+0000 7f5767e00640 1 -- 172.21.15.46:0/3521015957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57500ab9c0 con 0x7f57680ce490 2024-03-29T08:30:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.777+0000 7f576fc00640 1 -- 172.21.15.46:0/3521015957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57680ce490 msgr2=0x7f5768098e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.777+0000 7f576fc00640 1 --2- 172.21.15.46:0/3521015957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57680ce490 0x7f5768098e20 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f5750009870 tx=0x7f575001c530 comp rx=0 tx=0).stop 2024-03-29T08:30:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.777+0000 7f576fc00640 1 -- 172.21.15.46:0/3521015957 shutdown_connections 2024-03-29T08:30:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.777+0000 7f576fc00640 1 --2- 172.21.15.46:0/3521015957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57680ce490 0x7f5768098e20 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.777+0000 7f576fc00640 1 --2- 172.21.15.46:0/3521015957 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5768098a40 0x7f576815ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.777+0000 7f576fc00640 1 -- 172.21.15.46:0/3521015957 >> 172.21.15.46:0/3521015957 conn(0x7f57680b21a0 msgr2=0x7f57680b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.777+0000 7f576fc00640 1 -- 172.21.15.46:0/3521015957 shutdown_connections 2024-03-29T08:30:28.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.777+0000 7f576fc00640 1 -- 172.21.15.46:0/3521015957 wait complete. 2024-03-29T08:30:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576fc00640 1 Processor -- start 2024-03-29T08:30:28.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576fc00640 1 -- start start 2024-03-29T08:30:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576fc00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5768098a40 0x7f576815c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576fc00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5768099790 0x7f57680c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576fc00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57680ce490 0x7f57680c9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576fc00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57681597d0 con 0x7f5768098a40 2024-03-29T08:30:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576fc00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57680c4310 con 0x7f57680ce490 2024-03-29T08:30:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576fc00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57680c45f0 con 0x7f5768099790 2024-03-29T08:30:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5768098a40 0x7f576815c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576d200640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5768098a40 0x7f576815c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52868/0 (socket says 172.21.15.46:52868) 2024-03-29T08:30:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576d200640 1 -- 172.21.15.46:0/2954985385 learned_addr learned my addr 172.21.15.46:0/2954985385 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f575fe00640 1 --2- 172.21.15.46:0/2954985385 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5768099790 0x7f57680c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576dc00640 1 --2- 172.21.15.46:0/2954985385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57680ce490 0x7f57680c9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576d200640 1 -- 172.21.15.46:0/2954985385 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5768099790 msgr2=0x7f57680c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576d200640 1 --2- 172.21.15.46:0/2954985385 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5768099790 0x7f57680c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576d200640 1 -- 172.21.15.46:0/2954985385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57680ce490 msgr2=0x7f57680c9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576d200640 1 --2- 172.21.15.46:0/2954985385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57680ce490 0x7f57680c9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576d200640 1 -- 172.21.15.46:0/2954985385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57500094d0 con 0x7f5768098a40 2024-03-29T08:30:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f575fe00640 1 --2- 172.21.15.46:0/2954985385 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5768099790 0x7f57680c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576d200640 1 --2- 172.21.15.46:0/2954985385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5768098a40 0x7f576815c8c0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f575800e870 tx=0x7f575800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f5765600640 1 -- 172.21.15.46:0/2954985385 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5758009510 con 0x7f5768098a40 2024-03-29T08:30:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f5765600640 1 -- 172.21.15.46:0/2954985385 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575800eec0 con 0x7f5768098a40 2024-03-29T08:30:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576fc00640 1 -- 172.21.15.46:0/2954985385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57680c48d0 con 0x7f5768098a40 2024-03-29T08:30:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f5765600640 1 -- 172.21.15.46:0/2954985385 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575800cb40 con 0x7f5768098a40 2024-03-29T08:30:28.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.781+0000 7f576fc00640 1 -- 172.21.15.46:0/2954985385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57680c4e90 con 0x7f5768098a40 2024-03-29T08:30:28.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.785+0000 7f576fc00640 1 -- 172.21.15.46:0/2954985385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5768066c90 con 0x7f5768098a40 2024-03-29T08:30:28.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.785+0000 7f5765600640 1 -- 172.21.15.46:0/2954985385 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f575800cca0 con 0x7f5768098a40 2024-03-29T08:30:28.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.785+0000 7f5765600640 1 --2- 172.21.15.46:0/2954985385 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5744066690 0x7f5744068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:28.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.785+0000 7f5765600640 1 -- 172.21.15.46:0/2954985385 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f5758084050 con 0x7f5768098a40 2024-03-29T08:30:28.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.785+0000 7f575fe00640 1 --2- 172.21.15.46:0/2954985385 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5744066690 0x7f5744068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:28.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.785+0000 7f575fe00640 1 --2- 172.21.15.46:0/2954985385 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5744066690 0x7f5744068b50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f57500af000 tx=0x7f57500a7000 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:28.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.789+0000 7f5765600640 1 -- 172.21.15.46:0/2954985385 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f57580b5980 con 0x7f5768098a40 2024-03-29T08:30:28.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.945+0000 7f576fc00640 1 -- 172.21.15.46:0/2954985385 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f57680c3ef0 con 0x7f5744066690 2024-03-29T08:30:28.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.949+0000 7f5765600640 1 -- 172.21.15.46:0/2954985385 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50494 (secure 0 0 0) 0x7f57680c3ef0 con 0x7f5744066690 2024-03-29T08:30:28.949 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:28.949 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:30:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.949+0000 7f576fc00640 1 -- 172.21.15.46:0/2954985385 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5744066690 msgr2=0x7f5744068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.949+0000 7f576fc00640 1 --2- 172.21.15.46:0/2954985385 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5744066690 0x7f5744068b50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f57500af000 tx=0x7f57500a7000 comp rx=0 tx=0).stop 2024-03-29T08:30:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.949+0000 7f576fc00640 1 -- 172.21.15.46:0/2954985385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5768098a40 msgr2=0x7f576815c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.949+0000 7f576fc00640 1 --2- 172.21.15.46:0/2954985385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5768098a40 0x7f576815c8c0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f575800e870 tx=0x7f575800ec70 comp rx=0 tx=0).stop 2024-03-29T08:30:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.949+0000 7f576fc00640 1 -- 172.21.15.46:0/2954985385 shutdown_connections 2024-03-29T08:30:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.949+0000 7f576fc00640 1 --2- 172.21.15.46:0/2954985385 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f5744066690 0x7f5744068b50 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.949+0000 7f576fc00640 1 --2- 172.21.15.46:0/2954985385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57680ce490 0x7f57680c9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.949+0000 7f576fc00640 1 --2- 172.21.15.46:0/2954985385 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f5768099790 0x7f57680c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.949+0000 7f576fc00640 1 --2- 172.21.15.46:0/2954985385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5768098a40 0x7f576815c8c0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.949+0000 7f576fc00640 1 -- 172.21.15.46:0/2954985385 >> 172.21.15.46:0/2954985385 conn(0x7f57680b21a0 msgr2=0x7f576806fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.949+0000 7f576fc00640 1 -- 172.21.15.46:0/2954985385 shutdown_connections 2024-03-29T08:30:28.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:28.949+0000 7f576fc00640 1 -- 172.21.15.46:0/2954985385 wait complete. 2024-03-29T08:30:28.970 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-03-29T08:30:28.230668+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262536,"kb_used_data":13576,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754712184,"statfs":{"total":773094113280,"available":772825276416,"internally_reserved":0,"allocated":13901824,"data_stored":6727880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.115269"},"pg_stats":[{"pgid":"1.0","version":"18'117","reported_seq":259,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-03-29T08:30:19.389290+0000","last_change":"2024-03-29T08:18:02.057112+0000","last_active":"2024-03-29T08:30:19.389290+0000","last_peered":"2024-03-29T08:30:19.389290+0000","last_clean":"2024-03-29T08:30:19.389290+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:30:19.389290+0000","last_undegraded":"2024-03-29T08:30:19.389290+0000","last_fullsized":"2024-03-29T08:30:19.389290+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-29T08:18:00.748441+0000","last_clean_scrub_stamp":"2024-03-29T08:18:00.748441+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T11:45:18.561812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32672,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1589248,"data_stored":693393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.71999999999999997,"15min":0.69899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.34999999999999998},"max":{"1min":1.028,"5min":1.0289999999999999,"15min":1.1639999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.75800000000000001,"15min":0.72999999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.45800000000000002},"max":{"1min":0.95299999999999996,"5min":1.01,"15min":1.2410000000000001},"last":0.65600000000000003}]},{"osd":1,"last update":"Fri Mar 29 08:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.39600000000000002},"max":{"1min":0.85499999999999998,"5min":1.0740000000000001,"15min":1.202},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.71299999999999997,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.505,"15min":0.44},"max":{"1min":0.92800000000000005,"5min":0.99199999999999999,"15min":1.1339999999999999},"last":0.68899999999999995}]},{"osd":2,"last update":"Fri Mar 29 08:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71999999999999997,"15min":0.68500000000000005},"min":{"1min":0.52100000000000002,"5min":0.502,"15min":0.46000000000000002},"max":{"1min":0.996,"5min":0.996,"15min":1.085},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71499999999999997,"15min":0.70899999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88600000000000001,"5min":1.006,"15min":1.288},"last":0.58299999999999996}]},{"osd":3,"last update":"Fri Mar 29 08:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.46000000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73499999999999999,"15min":0.73699999999999999},"min":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.48499999999999999},"max":{"1min":0.93500000000000005,"5min":1.1200000000000001,"15min":1.2609999999999999},"last":0.751}]},{"osd":4,"last update":"Fri Mar 29 08:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59999999999999998,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61099999999999999,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.123},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri Mar 29 08:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61299999999999999,"15min":0.65000000000000002},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.056},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64400000000000002,"15min":0.66400000000000003},"min":{"1min":0.51800000000000002,"5min":0.442,"15min":0.39500000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.95599999999999996},"last":0.56200000000000006}]},{"osd":6,"last update":"Fri Mar 29 08:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.45900000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.96699999999999997,"5min":0.97699999999999998,"15min":1.0169999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66600000000000004,"15min":0.69599999999999995},"min":{"1min":0.47699999999999998,"5min":0.437,"15min":0.437},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.627}]}]},{"osd":6,"up_from":15,"seq":64424509592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32672,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1589248,"data_stored":693393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71399999999999997,"15min":0.68600000000000005},"min":{"1min":0.46100000000000002,"5min":0.42699999999999999,"15min":0.38900000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.74199999999999999,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.44500000000000001},"max":{"1min":0.95299999999999996,"5min":0.99299999999999999,"15min":1.079},"last":0.92300000000000004}]},{"osd":1,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68600000000000005,"15min":0.65700000000000003},"min":{"1min":0.45600000000000002,"5min":0.45400000000000001,"15min":0.377},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.67900000000000005,"15min":0.66600000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.079},"last":0.78000000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.753,"15min":0.73099999999999998},"min":{"1min":0.52200000000000002,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.73599999999999999,"15min":0.72599999999999998},"min":{"1min":0.47399999999999998,"5min":0.46400000000000002,"15min":0.435},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.83399999999999996}]},{"osd":3,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0309999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.879},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70999999999999996,"15min":0.70399999999999996},"min":{"1min":0.51200000000000001,"5min":0.51000000000000001,"15min":0.47799999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.79700000000000004}]},{"osd":4,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.621,"15min":0.63800000000000001},"min":{"1min":0.39700000000000002,"5min":0.375,"15min":0.36499999999999999},"max":{"1min":0.84499999999999997,"5min":1.0840000000000001,"15min":1.556},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.622,"15min":0.66500000000000004},"min":{"1min":0.432,"5min":0.38200000000000001,"15min":0.371},"max":{"1min":0.83399999999999996,"5min":0.96299999999999997,"15min":2.0539999999999998},"last":0.68400000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.625,"15min":0.64900000000000002},"min":{"1min":0.41999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82199999999999995,"5min":0.99399999999999999,"15min":1.0660000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67200000000000004,"15min":0.69199999999999995},"min":{"1min":0.47399999999999998,"5min":0.44400000000000001,"15min":0.37},"max":{"1min":0.91600000000000004,"5min":1.042,"15min":1.1419999999999999},"last":0.89000000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69499999999999995,"15min":0.71599999999999997},"min":{"1min":0.53600000000000003,"5min":0.46800000000000003,"15min":0.38500000000000001},"max":{"1min":0.88,"5min":1.03,"15min":1.03},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69199999999999995,"15min":0.71799999999999997},"min":{"1min":0.45800000000000002,"5min":0.45400000000000001,"15min":0.44600000000000001},"max":{"1min":0.996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.59899999999999998}]}]},{"osd":4,"up_from":15,"seq":64424509591,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32672,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1589248,"data_stored":693393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:30:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.48099999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.92700000000000005,"5min":1.026,"15min":2.1709999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.67300000000000004,"15min":0.70199999999999996},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.997,"15min":2.2869999999999999},"last":0.48999999999999999}]},{"osd":1,"last update":"Fri Mar 29 08:30:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.69499999999999995,"15min":0.73199999999999998},"min":{"1min":0.45500000000000002,"5min":0.436,"15min":0.41899999999999998},"max":{"1min":0.90000000000000002,"5min":1.0049999999999999,"15min":2.524},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.48999999999999999,"5min":0.441,"15min":0.40699999999999997},"max":{"1min":0.78700000000000003,"5min":1.3149999999999999,"15min":2.2679999999999998},"last":0.63100000000000001}]},{"osd":2,"last update":"Fri Mar 29 08:30:22 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.68799999999999994,"15min":0.70699999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42699999999999999},"max":{"1min":0.79400000000000004,"5min":1.0720000000000001,"15min":2.363},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.71599999999999997,"15min":0.73899999999999999},"min":{"1min":0.54200000000000004,"5min":0.45600000000000002,"15min":0.434},"max":{"1min":0.81499999999999995,"5min":0.97799999999999998,"15min":2.1880000000000002},"last":0.60099999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:30:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.871,"5min":1.3759999999999999,"15min":2.3029999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.748,"15min":0.77900000000000003},"min":{"1min":0.53900000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.83699999999999997,"5min":1.016,"15min":2.2549999999999999},"last":0.82099999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:30:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61399999999999999,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.38300000000000001,"15min":0.36499999999999999},"max":{"1min":0.76700000000000002,"5min":0.998,"15min":1.0189999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61599999999999999,"15min":0.63400000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91700000000000004,"5min":0.92000000000000004,"15min":1.0149999999999999},"last":0.79700000000000004}]},{"osd":6,"last update":"Fri Mar 29 08:30:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.622,"15min":0.64800000000000002},"min":{"1min":0.432,"5min":0.42599999999999999,"15min":0.39000000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":1.091},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70099999999999996,"15min":0.71199999999999997},"min":{"1min":0.51000000000000001,"5min":0.505,"15min":0.499},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":1.1220000000000001},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Mar 29 08:30:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68999999999999995,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89700000000000002,"5min":0.92800000000000005,"15min":1.1000000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71699999999999997,"15min":0.73799999999999999},"min":{"1min":0.48399999999999999,"5min":0.434,"15min":0.42899999999999999},"max":{"1min":0.96999999999999997,"5min":1.036,"15min":1.4430000000000001},"last":0.78300000000000003}]}]},{"osd":3,"up_from":15,"seq":64424509594,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32672,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1589248,"data_stored":693393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.28000000000000003,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":0.59299999999999997,"5min":0.748,"15min":1.123},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.51100000000000001,"15min":0.52000000000000002},"min":{"1min":0.34899999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":1.0409999999999999},"last":0.39700000000000002}]},{"osd":1,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.42099999999999999,"15min":0.443},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.26500000000000001},"max":{"1min":0.68100000000000005,"5min":0.76800000000000002,"15min":1.1599999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.5,"15min":0.51400000000000001},"min":{"1min":0.32600000000000001,"5min":0.32100000000000001,"15min":0.28599999999999998},"max":{"1min":0.72299999999999998,"5min":0.94499999999999995,"15min":1.4790000000000001},"last":0.38200000000000001}]},{"osd":2,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.502,"15min":0.53700000000000003},"min":{"1min":0.33500000000000002,"5min":0.32900000000000001,"15min":0.309},"max":{"1min":0.76400000000000001,"5min":0.80300000000000005,"15min":1.2529999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57999999999999996,"15min":0.61699999999999999},"min":{"1min":0.443,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79000000000000004,"5min":0.83599999999999997,"15min":1.8580000000000001},"last":0.499}]},{"osd":4,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.75700000000000001,"15min":0.76900000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.42199999999999999},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.6419999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.52000000000000002,"5min":0.51500000000000001,"15min":0.45400000000000001},"max":{"1min":1.008,"5min":1.083,"15min":1.391},"last":0.83399999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.0880000000000001,"5min":1.1850000000000001,"15min":1.671},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.80900000000000005,"15min":0.82999999999999996},"min":{"1min":0.58699999999999997,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.073,"5min":1.1379999999999999,"15min":1.7110000000000001},"last":0.84599999999999997}]},{"osd":6,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.77900000000000003,"15min":0.79300000000000004},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.97899999999999998,"5min":1.232,"15min":1.669},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76900000000000002,"15min":0.79100000000000004},"min":{"1min":0.54600000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.0900000000000001,"5min":1.119,"15min":2.0150000000000001},"last":0.81799999999999995}]},{"osd":7,"last update":"Fri Mar 29 08:29:25 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.78000000000000003,"15min":0.81399999999999995},"min":{"1min":0.57399999999999995,"5min":0.50600000000000001,"15min":0.48799999999999999},"max":{"1min":1.026,"5min":1.1120000000000001,"15min":1.887},"last":0.748},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.81100000000000005},"min":{"1min":0.57199999999999995,"5min":0.49399999999999999,"15min":0.47999999999999998},"max":{"1min":1.026,"5min":1.1539999999999999,"15min":2.044},"last":0.79900000000000004}]}]},{"osd":2,"up_from":15,"seq":64424509593,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32668,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1589248,"data_stored":693393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45600000000000002,"15min":0.46999999999999997},"min":{"1min":0.30599999999999999,"5min":0.25800000000000001,"15min":0.23899999999999999},"max":{"1min":0.58499999999999996,"5min":0.754,"15min":0.94299999999999995},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51800000000000002,"15min":0.52100000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32300000000000001},"max":{"1min":0.64000000000000001,"5min":0.749,"15min":2.0880000000000001},"last":0.51100000000000001}]},{"osd":1,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.48299999999999998,"15min":0.49099999999999999},"min":{"1min":0.32600000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68400000000000005,"5min":0.82699999999999996,"15min":1.399},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.48599999999999999,"15min":0.498},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.29299999999999998},"max":{"1min":0.59899999999999998,"5min":0.83299999999999996,"15min":3.0459999999999998},"last":0.38900000000000001}]},{"osd":3,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51200000000000001,"15min":0.51600000000000001},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.747,"5min":0.79900000000000004,"15min":2.1280000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58399999999999996,"15min":0.59299999999999997},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.35999999999999999},"max":{"1min":0.76300000000000001,"5min":0.88800000000000001,"15min":2.9950000000000001},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.78500000000000003,"15min":0.77900000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.49099999999999999},"max":{"1min":1.087,"5min":1.109,"15min":1.4239999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.78800000000000003,"15min":0.76700000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.435},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.59399999999999997}]},{"osd":5,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.85099999999999998,"15min":0.83899999999999997},"min":{"1min":0.60999999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.179,"5min":1.179,"15min":1.4890000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.81499999999999995,"15min":0.82199999999999995},"min":{"1min":0.53700000000000003,"5min":0.52300000000000002,"15min":0.48799999999999999},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.526},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79100000000000004,"15min":0.78100000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.48599999999999999},"max":{"1min":1.0289999999999999,"5min":1.167,"15min":2.0470000000000002},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82099999999999995,"15min":0.79800000000000004},"min":{"1min":0.57299999999999995,"5min":0.54800000000000004,"15min":0.52500000000000002},"max":{"1min":0.97399999999999998,"5min":1.2609999999999999,"15min":1.524},"last":0.66000000000000003}]},{"osd":7,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.79900000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.80100000000000005,"15min":0.79500000000000004},"min":{"1min":0.60599999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.96999999999999997,"5min":1.214,"15min":2},"last":0.58399999999999996}]}]},{"osd":1,"up_from":15,"seq":64424509593,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33256,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338584,"statfs":{"total":96636764160,"available":96602710016,"internally_reserved":0,"allocated":2183168,"data_stored":1283761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.44,"15min":0.51600000000000001},"min":{"1min":0.249,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.67600000000000005,"5min":0.69699999999999995,"15min":4.2539999999999996},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.46999999999999997,"15min":0.54800000000000004},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.627,"5min":0.73399999999999999,"15min":4.1909999999999998},"last":0.35099999999999998}]},{"osd":2,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.49399999999999999,"15min":0.498},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.66700000000000004,"5min":0.71099999999999997,"15min":0.82899999999999996},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.51900000000000002,"15min":0.52900000000000003},"min":{"1min":0.39900000000000002,"5min":0.35099999999999998,"15min":0.33600000000000002},"max":{"1min":0.68500000000000005,"5min":0.72499999999999998,"15min":0.88200000000000001},"last":0.40999999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56699999999999995,"15min":0.58099999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71999999999999997,"5min":0.88300000000000001,"15min":1.52},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50900000000000001,"15min":0.52300000000000002},"min":{"1min":0.379,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.67100000000000004,"5min":0.94499999999999995,"15min":1.331},"last":0.44400000000000001}]},{"osd":4,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77700000000000002,"15min":0.75600000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46200000000000002},"max":{"1min":0.89200000000000002,"5min":1.157,"15min":1.157},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.55400000000000005,"5min":0.502,"15min":0.47999999999999998},"max":{"1min":1.0069999999999999,"5min":1.212,"15min":1.23},"last":0.622}]},{"osd":5,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78700000000000003,"15min":0.77800000000000002},"min":{"1min":0.58199999999999996,"5min":0.503,"15min":0.503},"max":{"1min":0.91400000000000003,"5min":1.258,"15min":1.258},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77900000000000003,"15min":0.76900000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.46500000000000002},"max":{"1min":0.93899999999999995,"5min":1.1879999999999999,"15min":1.2509999999999999},"last":0.60799999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.77100000000000002,"15min":0.77200000000000002},"min":{"1min":0.56799999999999995,"5min":0.54900000000000004,"15min":0.5},"max":{"1min":0.96899999999999997,"5min":1.268,"15min":1.268},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.79500000000000004,"15min":0.81699999999999995},"min":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.51300000000000001},"max":{"1min":1.0129999999999999,"5min":1.22,"15min":1.22},"last":0.55700000000000005}]},{"osd":7,"last update":"Fri Mar 29 08:29:24 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.90600000000000003,"5min":1.2410000000000001,"15min":1.3089999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.78800000000000003,"15min":0.78700000000000003},"min":{"1min":0.59299999999999997,"5min":0.503,"15min":0.503},"max":{"1min":0.96299999999999997,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.73899999999999999}]}]},{"osd":0,"up_from":15,"seq":64424509592,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32668,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1589248,"data_stored":693393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.438,"15min":0.48899999999999999},"min":{"1min":0.29099999999999998,"5min":0.27000000000000002,"15min":0.22800000000000001},"max":{"1min":0.55400000000000005,"5min":0.76100000000000001,"15min":3.9359999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.48399999999999999,"15min":0.60399999999999998},"min":{"1min":0.28699999999999998,"5min":0.28499999999999998,"15min":0.253},"max":{"1min":0.61699999999999999,"5min":0.77000000000000002,"15min":8.0419999999999998},"last":0.317}]},{"osd":2,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.502,"15min":0.51600000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.59299999999999997,"5min":0.78000000000000003,"15min":1.258},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52900000000000003,"15min":0.59499999999999997},"min":{"1min":0.33000000000000002,"5min":0.32800000000000001,"15min":0.312},"max":{"1min":0.65900000000000003,"5min":0.81399999999999995,"15min":4.907},"last":0.35899999999999999}]},{"osd":3,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.49399999999999999,"15min":0.60799999999999998},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.28499999999999998},"max":{"1min":0.69499999999999995,"5min":0.83399999999999996,"15min":7.9569999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.56799999999999995,"15min":0.59299999999999997},"min":{"1min":0.40899999999999997,"5min":0.36099999999999999,"15min":0.34799999999999998},"max":{"1min":0.56699999999999995,"5min":0.87,"15min":1.397},"last":0.441}]},{"osd":4,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78700000000000003,"15min":0.84999999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.99099999999999999,"5min":1.282,"15min":4.4720000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.77600000000000002,"15min":0.83299999999999996},"min":{"1min":0.52100000000000002,"5min":0.496,"15min":0.47699999999999998},"max":{"1min":1.04,"5min":1.153,"15min":4.3920000000000003},"last":0.81599999999999995}]},{"osd":5,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79500000000000004,"15min":0.83199999999999996},"min":{"1min":0.56899999999999995,"5min":0.52800000000000002,"15min":0.48699999999999999},"max":{"1min":1.117,"5min":1.24,"15min":4.3200000000000003},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.77300000000000002,"15min":0.80600000000000005},"min":{"1min":0.59999999999999998,"5min":0.56000000000000005,"15min":0.54600000000000004},"max":{"1min":0.95899999999999996,"5min":1.081,"15min":1.6759999999999999},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.79000000000000004,"15min":0.81200000000000006},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.54400000000000004},"max":{"1min":1.0209999999999999,"5min":1.1220000000000001,"15min":1.5920000000000001},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.80500000000000005,"15min":0.86099999999999999},"min":{"1min":0.55300000000000005,"5min":0.54200000000000004,"15min":0.51500000000000001},"max":{"1min":1.137,"5min":1.2150000000000001,"15min":4.4100000000000001},"last":0.72699999999999998}]},{"osd":7,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.81200000000000006,"15min":0.871},"min":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.496},"max":{"1min":1.0609999999999999,"5min":1.1459999999999999,"15min":4.5209999999999999},"last":0.745},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79200000000000004,"15min":0.85399999999999998},"min":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.50900000000000001},"max":{"1min":1.044,"5min":1.3129999999999999,"15min":4.4420000000000002},"last":0.67900000000000005}]}]},{"osd":5,"up_from":14,"seq":60129542297,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33256,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338584,"statfs":{"total":96636764160,"available":96602710016,"internally_reserved":0,"allocated":2183168,"data_stored":1283761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.68899999999999995,"15min":0.66600000000000004},"min":{"1min":0.51800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.84799999999999998,"5min":0.97199999999999998,"15min":0.98499999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71899999999999997,"15min":0.68600000000000005},"min":{"1min":0.44700000000000001,"5min":0.42999999999999999,"15min":0.39800000000000002},"max":{"1min":0.81499999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.69699999999999995}]},{"osd":1,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.69699999999999995,"15min":0.70599999999999996},"min":{"1min":0.46500000000000002,"5min":0.42499999999999999,"15min":0.38200000000000001},"max":{"1min":0.80700000000000005,"5min":1.008,"15min":2.5129999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73999999999999999,"15min":0.70599999999999996},"min":{"1min":0.63700000000000001,"5min":0.51300000000000001,"15min":0.40500000000000003},"max":{"1min":0.878,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.70499999999999996,"15min":0.68600000000000005},"min":{"1min":0.52200000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88800000000000001,"5min":0.997,"15min":0.997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.74099999999999999,"15min":0.71799999999999997},"min":{"1min":0.56399999999999995,"5min":0.47499999999999998,"15min":0.39200000000000002},"max":{"1min":0.83099999999999996,"5min":1.03,"15min":1.03},"last":0.79300000000000004}]},{"osd":3,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.74099999999999999,"15min":0.72499999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.47099999999999997},"max":{"1min":0.92600000000000005,"5min":1.038,"15min":1.038},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.435},"max":{"1min":0.84499999999999997,"5min":1.0209999999999999,"15min":1.954},"last":0.67100000000000004}]},{"osd":4,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.628,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.93500000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.628,"15min":0.64200000000000002},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.373},"max":{"1min":0.82699999999999996,"5min":0.89200000000000002,"15min":1.0820000000000001},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.623,"15min":0.64700000000000002},"min":{"1min":0.46100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":1.071},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.67700000000000005,"15min":0.68500000000000005},"min":{"1min":0.52000000000000002,"5min":0.45300000000000001,"15min":0.432},"max":{"1min":0.93799999999999994,"5min":0.94299999999999995,"15min":1.0609999999999999},"last":0.622}]},{"osd":7,"last update":"Fri Mar 29 08:29:22 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.67300000000000004,"15min":0.69099999999999995},"min":{"1min":0.48899999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.88200000000000001,"5min":0.92200000000000004,"15min":1.1200000000000001},"last":0.749},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.69699999999999995,"15min":0.72499999999999998},"min":{"1min":0.53300000000000003,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.04},"last":0.81399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:30:28.973 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-29T08:18:00.748441+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=29, tm_hour=8, tm_min=18, tm_sec=0, tm_wday=4, tm_yday=89, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=29, tm_hour=8, tm_min=30, tm_sec=23, tm_wday=4, tm_yday=89, tm_isdst=0) 2024-03-29T08:30:28.974 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-29T08:30:40.078 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:30:40.082 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:30:48.975 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-29T08:30:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e19e00640 1 Processor -- start 2024-03-29T08:30:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e19e00640 1 -- start start 2024-03-29T08:30:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e19e00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0e140d0660 0x7f0e140d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e19e00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e1406b8b0 0x7f0e140cad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e19e00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1415cfb0 con 0x7f0e1415dc00 2024-03-29T08:30:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e19e00640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e1406c100 con 0x7f0e140d0660 2024-03-29T08:30:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e19e00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e140d0f80 con 0x7f0e1406b8b0 2024-03-29T08:30:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e12a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0e140d0660 0x7f0e140d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e13400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0e1415dc00 0x7f0e140cf7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55246/0 (socket says 172.21.15.46:55246) 2024-03-29T08:30:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e12a00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0e140d0660 0x7f0e140d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36708/0 (socket says 172.21.15.46:36708) 2024-03-29T08:30:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e13400640 1 -- 172.21.15.46:0/2827665798 learned_addr learned my addr 172.21.15.46:0/2827665798 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:49.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e0a000640 1 --2- 172.21.15.46:0/2827665798 >> v2:172.21.15.73:3300/0 conn(0x7f0e1406b8b0 0x7f0e140cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e12a00640 1 -- 172.21.15.46:0/2827665798 >> v2:172.21.15.73:3300/0 conn(0x7f0e1406b8b0 msgr2=0x7f0e140cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:49.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e12a00640 1 --2- 172.21.15.46:0/2827665798 >> v2:172.21.15.73:3300/0 conn(0x7f0e1406b8b0 0x7f0e140cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e12a00640 1 -- 172.21.15.46:0/2827665798 >> v1:172.21.15.46:6789/0 conn(0x7f0e1415dc00 legacy=0x7f0e140cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:49.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e12a00640 1 -- 172.21.15.46:0/2827665798 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1406be20 con 0x7f0e140d0660 2024-03-29T08:30:49.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e0a000640 1 --2- 172.21.15.46:0/2827665798 >> v2:172.21.15.73:3300/0 conn(0x7f0e1406b8b0 0x7f0e140cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:49.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.129+0000 7f0e12000640 1 -- 172.21.15.46:0/2827665798 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2530671566 0 0) 0x7f0e1415cfb0 con 0x7f0e1415dc00 2024-03-29T08:30:49.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.133+0000 7f0e12a00640 1 --2- 172.21.15.46:0/2827665798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e140d0660 0x7f0e140d0a40 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f0e1406acb0 tx=0x7f0e0401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5423237dcd531953 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:49.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.133+0000 7f0e12000640 1 -- 172.21.15.46:0/2827665798 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e040a59c0 con 0x7f0e140d0660 2024-03-29T08:30:49.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.133+0000 7f0e12000640 1 -- 172.21.15.46:0/2827665798 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e040a5b20 con 0x7f0e140d0660 2024-03-29T08:30:49.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.133+0000 7f0e12000640 1 -- 172.21.15.46:0/2827665798 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e040ad440 con 0x7f0e140d0660 2024-03-29T08:30:49.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.133+0000 7f0e19e00640 1 -- 172.21.15.46:0/2827665798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e140d0660 msgr2=0x7f0e140d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:49.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.133+0000 7f0e19e00640 1 --2- 172.21.15.46:0/2827665798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e140d0660 0x7f0e140d0a40 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f0e1406acb0 tx=0x7f0e0401c900 comp rx=0 tx=0).stop 2024-03-29T08:30:49.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.133+0000 7f0e19e00640 1 -- 172.21.15.46:0/2827665798 shutdown_connections 2024-03-29T08:30:49.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.133+0000 7f0e19e00640 1 --2- 172.21.15.46:0/2827665798 >> v2:172.21.15.73:3300/0 conn(0x7f0e1406b8b0 0x7f0e140cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.133+0000 7f0e19e00640 1 --2- 172.21.15.46:0/2827665798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e140d0660 0x7f0e140d0a40 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.133+0000 7f0e19e00640 1 -- 172.21.15.46:0/2827665798 >> 172.21.15.46:0/2827665798 conn(0x7f0e14089fd0 msgr2=0x7f0e1408a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:49.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.133+0000 7f0e19e00640 1 -- 172.21.15.46:0/2827665798 shutdown_connections 2024-03-29T08:30:49.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.133+0000 7f0e19e00640 1 -- 172.21.15.46:0/2827665798 wait complete. 2024-03-29T08:30:49.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e19e00640 1 Processor -- start 2024-03-29T08:30:49.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e19e00640 1 -- start start 2024-03-29T08:30:49.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e19e00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e1406b8b0 0x7f0e14155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e19e00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e140d0660 0x7f0e1414fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e19e00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1415dc00 0x7f0e1414a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e19e00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e140c8680 con 0x7f0e140d0660 2024-03-29T08:30:49.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e19e00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e14146740 con 0x7f0e1415dc00 2024-03-29T08:30:49.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e19e00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e140c82e0 con 0x7f0e1406b8b0 2024-03-29T08:30:49.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e0a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e140d0660 0x7f0e1414fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e0a000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e140d0660 0x7f0e1414fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36722/0 (socket says 172.21.15.46:36722) 2024-03-29T08:30:49.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e0a000640 1 -- 172.21.15.46:0/987418264 learned_addr learned my addr 172.21.15.46:0/987418264 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:49.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e13400640 1 --2- 172.21.15.46:0/987418264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1415dc00 0x7f0e1414a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e12a00640 1 --2- 172.21.15.46:0/987418264 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e1406b8b0 0x7f0e14155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e0a000640 1 -- 172.21.15.46:0/987418264 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e1406b8b0 msgr2=0x7f0e14155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:49.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e0a000640 1 --2- 172.21.15.46:0/987418264 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e1406b8b0 0x7f0e14155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e0a000640 1 -- 172.21.15.46:0/987418264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1415dc00 msgr2=0x7f0e1414a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:49.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e0a000640 1 --2- 172.21.15.46:0/987418264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1415dc00 0x7f0e1414a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e0a000640 1 -- 172.21.15.46:0/987418264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e04009a70 con 0x7f0e140d0660 2024-03-29T08:30:49.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e0a000640 1 --2- 172.21.15.46:0/987418264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e140d0660 0x7f0e1414fa00 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc00d800 tx=0x7f0dfc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e0be00640 1 -- 172.21.15.46:0/987418264 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc00b6c0 con 0x7f0e140d0660 2024-03-29T08:30:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e0be00640 1 -- 172.21.15.46:0/987418264 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dfc00b820 con 0x7f0e140d0660 2024-03-29T08:30:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e0be00640 1 -- 172.21.15.46:0/987418264 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc00bbd0 con 0x7f0e140d0660 2024-03-29T08:30:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e19e00640 1 -- 172.21.15.46:0/987418264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e141469e0 con 0x7f0e140d0660 2024-03-29T08:30:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.137+0000 7f0e19e00640 1 -- 172.21.15.46:0/987418264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1414f090 con 0x7f0e140d0660 2024-03-29T08:30:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.141+0000 7f0e19e00640 1 -- 172.21.15.46:0/987418264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e14066c90 con 0x7f0e140d0660 2024-03-29T08:30:49.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.141+0000 7f0e0be00640 1 -- 172.21.15.46:0/987418264 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f0dfc00bd30 con 0x7f0e140d0660 2024-03-29T08:30:49.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.141+0000 7f0e0be00640 1 --2- 172.21.15.46:0/987418264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0df0066690 0x7f0df0068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.141+0000 7f0e0be00640 1 -- 172.21.15.46:0/987418264 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f0dfc084320 con 0x7f0e140d0660 2024-03-29T08:30:49.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.141+0000 7f0e12a00640 1 --2- 172.21.15.46:0/987418264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0df0066690 0x7f0df0068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.141+0000 7f0e12a00640 1 --2- 172.21.15.46:0/987418264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0df0066690 0x7f0df0068b50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0e040af000 tx=0x7f0e040a7070 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.145+0000 7f0e0be00640 1 -- 172.21.15.46:0/987418264 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f0dfc051980 con 0x7f0e140d0660 2024-03-29T08:30:49.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.305+0000 7f0e19e00640 1 -- 172.21.15.46:0/987418264 --> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e140d02f0 con 0x7f0df0066690 2024-03-29T08:30:49.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.309+0000 7f0e0be00640 1 -- 172.21.15.46:0/987418264 <== mgr.4114 v2:172.21.15.73:6832/27375 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50196 (secure 0 0 0) 0x7f0e040bc070 con 0x7f0df0066690 2024-03-29T08:30:49.310 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:49.311 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-03-29T08:30:49.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.309+0000 7f0e19e00640 1 -- 172.21.15.46:0/987418264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0df0066690 msgr2=0x7f0df0068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:49.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.309+0000 7f0e19e00640 1 --2- 172.21.15.46:0/987418264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0df0066690 0x7f0df0068b50 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f0e040af000 tx=0x7f0e040a7070 comp rx=0 tx=0).stop 2024-03-29T08:30:49.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.309+0000 7f0e19e00640 1 -- 172.21.15.46:0/987418264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e140d0660 msgr2=0x7f0e1414fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:49.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.309+0000 7f0e19e00640 1 --2- 172.21.15.46:0/987418264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e140d0660 0x7f0e1414fa00 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc00d800 tx=0x7f0dfc00dcc0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.309+0000 7f0e19e00640 1 -- 172.21.15.46:0/987418264 shutdown_connections 2024-03-29T08:30:49.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.309+0000 7f0e19e00640 1 --2- 172.21.15.46:0/987418264 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f0df0066690 0x7f0df0068b50 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.309+0000 7f0e19e00640 1 --2- 172.21.15.46:0/987418264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e1415dc00 0x7f0e1414a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.309+0000 7f0e19e00640 1 --2- 172.21.15.46:0/987418264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e140d0660 0x7f0e1414fa00 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.309+0000 7f0e19e00640 1 --2- 172.21.15.46:0/987418264 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f0e1406b8b0 0x7f0e14155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.309+0000 7f0e19e00640 1 -- 172.21.15.46:0/987418264 >> 172.21.15.46:0/987418264 conn(0x7f0e14089fd0 msgr2=0x7f0e14156ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:49.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.309+0000 7f0e19e00640 1 -- 172.21.15.46:0/987418264 shutdown_connections 2024-03-29T08:30:49.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.313+0000 7f0e19e00640 1 -- 172.21.15.46:0/987418264 wait complete. 2024-03-29T08:30:49.331 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":506,"stamp":"2024-03-29T08:30:48.235193+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262512,"kb_used_data":13576,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754712208,"statfs":{"total":773094113280,"available":772825300992,"internally_reserved":0,"allocated":13901824,"data_stored":6727880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002637"},"pg_stats":[{"pgid":"1.0","version":"128'119","reported_seq":273,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-03-29T08:30:25.643120+0000","last_change":"2024-03-29T08:30:25.643199+0000","last_active":"2024-03-29T08:30:25.643120+0000","last_peered":"2024-03-29T08:30:25.643120+0000","last_clean":"2024-03-29T08:30:25.643120+0000","last_became_active":"2024-03-29T08:18:02.056136+0000","last_became_peered":"2024-03-29T08:18:02.056136+0000","last_unstale":"2024-03-29T08:30:25.643120+0000","last_undegraded":"2024-03-29T08:30:25.643120+0000","last_fullsized":"2024-03-29T08:30:25.643120+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'119","last_scrub_stamp":"2024-03-29T08:30:25.643102+0000","last_deep_scrub":"128'119","last_deep_scrub_stamp":"2024-03-29T08:30:25.643102+0000","last_clean_scrub_stamp":"2024-03-29T08:30:25.643102+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-30T14:22:01.175198+0000","scrub_duration":0.0057048189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32672,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1589248,"data_stored":693393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.71999999999999997,"15min":0.69899999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.34999999999999998},"max":{"1min":1.028,"5min":1.0289999999999999,"15min":1.1639999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.75800000000000001,"15min":0.72999999999999998},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.45800000000000002},"max":{"1min":0.95299999999999996,"5min":1.01,"15min":1.2410000000000001},"last":0.72799999999999998}]},{"osd":1,"last update":"Fri Mar 29 08:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.39600000000000002},"max":{"1min":0.85499999999999998,"5min":1.0740000000000001,"15min":1.202},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.71299999999999997,"15min":0.68700000000000006},"min":{"1min":0.51800000000000002,"5min":0.505,"15min":0.44},"max":{"1min":0.92800000000000005,"5min":0.99199999999999999,"15min":1.1339999999999999},"last":0.65200000000000002}]},{"osd":2,"last update":"Fri Mar 29 08:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71999999999999997,"15min":0.68500000000000005},"min":{"1min":0.52100000000000002,"5min":0.502,"15min":0.46000000000000002},"max":{"1min":0.996,"5min":0.996,"15min":1.085},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71499999999999997,"15min":0.70899999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.88600000000000001,"5min":1.006,"15min":1.288},"last":0.625}]},{"osd":3,"last update":"Fri Mar 29 08:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72399999999999998,"15min":0.71399999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.46000000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.73499999999999999,"15min":0.73699999999999999},"min":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.48499999999999999},"max":{"1min":0.93500000000000005,"5min":1.1200000000000001,"15min":1.2609999999999999},"last":0.66100000000000003}]},{"osd":4,"last update":"Fri Mar 29 08:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59999999999999998,"15min":0.60499999999999998},"min":{"1min":0.40999999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.61099999999999999,"15min":0.64800000000000002},"min":{"1min":0.43099999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.089,"5min":1.089,"15min":1.123},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Mar 29 08:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.61299999999999999,"15min":0.65000000000000002},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.056},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64400000000000002,"15min":0.66400000000000003},"min":{"1min":0.51800000000000002,"5min":0.442,"15min":0.39500000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.95599999999999996},"last":0.59699999999999998}]},{"osd":6,"last update":"Fri Mar 29 08:30:23 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.70899999999999996,"15min":0.71299999999999997},"min":{"1min":0.45900000000000002,"5min":0.436,"15min":0.436},"max":{"1min":0.96699999999999997,"5min":0.97699999999999998,"15min":1.0169999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.66600000000000004,"15min":0.69599999999999995},"min":{"1min":0.47699999999999998,"5min":0.437,"15min":0.437},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.57399999999999995}]}]},{"osd":6,"up_from":15,"seq":64424509596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32672,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1589248,"data_stored":693393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.71399999999999997,"15min":0.68600000000000005},"min":{"1min":0.46100000000000002,"5min":0.42699999999999999,"15min":0.38900000000000001},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.74199999999999999,"15min":0.72399999999999998},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.44500000000000001},"max":{"1min":0.95299999999999996,"5min":0.99299999999999999,"15min":1.079},"last":0.78000000000000003}]},{"osd":1,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.68600000000000005,"15min":0.65700000000000003},"min":{"1min":0.45600000000000002,"5min":0.45400000000000001,"15min":0.377},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.67900000000000005,"15min":0.66600000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.079},"last":0.80000000000000004}]},{"osd":2,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.753,"15min":0.73099999999999998},"min":{"1min":0.52200000000000002,"5min":0.501,"15min":0.44800000000000001},"max":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.73599999999999999,"15min":0.72599999999999998},"min":{"1min":0.47399999999999998,"5min":0.46400000000000002,"15min":0.435},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.84099999999999997}]},{"osd":3,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.439,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0309999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70999999999999996,"15min":0.70399999999999996},"min":{"1min":0.51200000000000001,"5min":0.51000000000000001,"15min":0.47799999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.82199999999999995}]},{"osd":4,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.621,"15min":0.63800000000000001},"min":{"1min":0.39700000000000002,"5min":0.375,"15min":0.36499999999999999},"max":{"1min":0.84499999999999997,"5min":1.0840000000000001,"15min":1.556},"last":0.626},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.622,"15min":0.66500000000000004},"min":{"1min":0.432,"5min":0.38200000000000001,"15min":0.371},"max":{"1min":0.83399999999999996,"5min":0.96299999999999997,"15min":2.0539999999999998},"last":0.63400000000000001}]},{"osd":5,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.625,"15min":0.64900000000000002},"min":{"1min":0.41999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.82199999999999995,"5min":0.99399999999999999,"15min":1.0660000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.67200000000000004,"15min":0.69199999999999995},"min":{"1min":0.47399999999999998,"5min":0.44400000000000001,"15min":0.37},"max":{"1min":0.91600000000000004,"5min":1.042,"15min":1.1419999999999999},"last":0.59199999999999997}]},{"osd":7,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69499999999999995,"15min":0.71599999999999997},"min":{"1min":0.53600000000000003,"5min":0.46800000000000003,"15min":0.38500000000000001},"max":{"1min":0.88,"5min":1.03,"15min":1.03},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.69199999999999995,"15min":0.71799999999999997},"min":{"1min":0.45800000000000002,"5min":0.45400000000000001,"15min":0.44600000000000001},"max":{"1min":0.996,"5min":1.1160000000000001,"15min":1.1160000000000001},"last":0.66700000000000004}]}]},{"osd":4,"up_from":15,"seq":64424509595,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32672,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1589248,"data_stored":693393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:30:22 2024","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.69399999999999995,"15min":0.69799999999999995},"min":{"1min":0.48099999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.92700000000000005,"5min":1.026,"15min":2.1709999999999998},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.67300000000000004,"15min":0.70199999999999996},"min":{"1min":0.41699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85599999999999998,"5min":0.997,"15min":2.2869999999999999},"last":0.76800000000000002}]},{"osd":1,"last update":"Fri Mar 29 08:30:22 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.69499999999999995,"15min":0.73199999999999998},"min":{"1min":0.45500000000000002,"5min":0.436,"15min":0.41899999999999998},"max":{"1min":0.90000000000000002,"5min":1.0049999999999999,"15min":2.524},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.48999999999999999,"5min":0.441,"15min":0.40699999999999997},"max":{"1min":0.78700000000000003,"5min":1.3149999999999999,"15min":2.2679999999999998},"last":0.78300000000000003}]},{"osd":2,"last update":"Fri Mar 29 08:30:22 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.68799999999999994,"15min":0.70699999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42699999999999999},"max":{"1min":0.79400000000000004,"5min":1.0720000000000001,"15min":2.363},"last":1.044},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.71599999999999997,"15min":0.73899999999999999},"min":{"1min":0.54200000000000004,"5min":0.45600000000000002,"15min":0.434},"max":{"1min":0.81499999999999995,"5min":0.97799999999999998,"15min":2.1880000000000002},"last":0.90700000000000003}]},{"osd":3,"last update":"Fri Mar 29 08:30:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.73999999999999999,"15min":0.74399999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.871,"5min":1.3759999999999999,"15min":2.3029999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.748,"15min":0.77900000000000003},"min":{"1min":0.53900000000000003,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.83699999999999997,"5min":1.016,"15min":2.2549999999999999},"last":0.93500000000000005}]},{"osd":5,"last update":"Fri Mar 29 08:30:22 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61399999999999999,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.38300000000000001,"15min":0.36499999999999999},"max":{"1min":0.76700000000000002,"5min":0.998,"15min":1.0189999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61599999999999999,"15min":0.63400000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.91700000000000004,"5min":0.92000000000000004,"15min":1.0149999999999999},"last":0.61399999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:30:22 2024","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.622,"15min":0.64800000000000002},"min":{"1min":0.432,"5min":0.42599999999999999,"15min":0.39000000000000001},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":1.091},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70099999999999996,"15min":0.71199999999999997},"min":{"1min":0.51000000000000001,"5min":0.505,"15min":0.499},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":1.1220000000000001},"last":0.63400000000000001}]},{"osd":7,"last update":"Fri Mar 29 08:30:22 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.68999999999999995,"15min":0.70399999999999996},"min":{"1min":0.498,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.89700000000000002,"5min":0.92800000000000005,"15min":1.1000000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71699999999999997,"15min":0.73799999999999999},"min":{"1min":0.48399999999999999,"5min":0.434,"15min":0.42899999999999999},"max":{"1min":0.96999999999999997,"5min":1.036,"15min":1.4430000000000001},"last":0.65400000000000003}]}]},{"osd":3,"up_from":15,"seq":64424509598,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32672,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1589248,"data_stored":693393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:30:28 2024","interfaces":[{"interface":"back","average":{"1min":3.5110000000000001,"5min":1.0549999999999999,"15min":0.65400000000000003},"min":{"1min":0.28199999999999997,"5min":0.28000000000000003,"15min":0.26200000000000001},"max":{"1min":49.948,"5min":49.948,"15min":49.948},"last":0.51800000000000002},{"interface":"front","average":{"1min":1.1319999999999999,"5min":0.63500000000000001,"15min":0.56100000000000005},"min":{"1min":0.36299999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":11.085000000000001,"5min":11.085000000000001,"15min":11.085000000000001},"last":0.54100000000000004}]},{"osd":1,"last update":"Fri Mar 29 08:30:28 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.436,"15min":0.44400000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.26500000000000001},"max":{"1min":1.9910000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.2889999999999999,"5min":0.66100000000000003,"15min":0.56399999999999995},"min":{"1min":0.33700000000000002,"5min":0.32100000000000001,"15min":0.28599999999999998},"max":{"1min":13.964,"5min":13.964,"15min":13.964},"last":0.44800000000000001}]},{"osd":2,"last update":"Fri Mar 29 08:30:28 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":0.623,"15min":0.57299999999999995},"min":{"1min":0.33000000000000002,"5min":0.32900000000000001,"15min":0.309},"max":{"1min":10.999000000000001,"5min":10.999000000000001,"15min":10.999000000000001},"last":0.501},{"interface":"front","average":{"1min":1.3620000000000001,"5min":0.73999999999999999,"15min":0.66400000000000003},"min":{"1min":0.42399999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":13.913,"5min":13.913,"15min":13.913},"last":0.61599999999999999}]},{"osd":4,"last update":"Fri Mar 29 08:30:28 2024","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":0.86099999999999999,"15min":0.80300000000000005},"min":{"1min":0.438,"5min":0.438,"15min":0.42199999999999999},"max":{"1min":10.727,"5min":10.727,"15min":10.727},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.755,"15min":0.76300000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.45400000000000001},"max":{"1min":1.8999999999999999,"5min":1.8999999999999999,"15min":1.8999999999999999},"last":0.84999999999999998}]},{"osd":5,"last update":"Fri Mar 29 08:30:28 2024","interfaces":[{"interface":"back","average":{"1min":1.5129999999999999,"5min":0.93200000000000005,"15min":0.84499999999999997},"min":{"1min":0.55600000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":13.597,"5min":13.597,"15min":13.597},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.421,"5min":0.92600000000000005,"15min":0.86599999999999999},"min":{"1min":0.51900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":10.923,"5min":10.923,"15min":10.923},"last":0.77400000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:30:28 2024","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.78600000000000003,"15min":0.79300000000000004},"min":{"1min":0.55800000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.508,"5min":0.91300000000000003,"15min":0.83499999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":13.544,"5min":13.544,"15min":13.544},"last":0.80500000000000005}]},{"osd":7,"last update":"Fri Mar 29 08:30:28 2024","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":0.89200000000000002,"15min":0.84699999999999998},"min":{"1min":0.54400000000000004,"5min":0.50600000000000001,"15min":0.48799999999999999},"max":{"1min":10.786,"5min":10.786,"15min":10.786},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.79300000000000004,"15min":0.80700000000000005},"min":{"1min":0.60099999999999998,"5min":0.49399999999999999,"15min":0.47999999999999998},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":2.044},"last":0.92700000000000005}]}]},{"osd":2,"up_from":15,"seq":64424509597,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32668,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1589248,"data_stored":693393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.45600000000000002,"15min":0.46999999999999997},"min":{"1min":0.30599999999999999,"5min":0.25800000000000001,"15min":0.23899999999999999},"max":{"1min":0.58499999999999996,"5min":0.754,"15min":0.94299999999999995},"last":0.627},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51800000000000002,"15min":0.52100000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.32300000000000001},"max":{"1min":0.64000000000000001,"5min":0.749,"15min":2.0880000000000001},"last":0.69799999999999995}]},{"osd":1,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.48299999999999998,"15min":0.49099999999999999},"min":{"1min":0.32600000000000001,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68400000000000005,"5min":0.82699999999999996,"15min":1.399},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.48599999999999999,"15min":0.498},"min":{"1min":0.32900000000000001,"5min":0.312,"15min":0.29299999999999998},"max":{"1min":0.59899999999999998,"5min":0.83299999999999996,"15min":3.0459999999999998},"last":0.73299999999999998}]},{"osd":3,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.51200000000000001,"15min":0.51600000000000001},"min":{"1min":0.35299999999999998,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.747,"5min":0.79900000000000004,"15min":2.1280000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.58399999999999996,"15min":0.59299999999999997},"min":{"1min":0.42899999999999999,"5min":0.39200000000000002,"15min":0.35999999999999999},"max":{"1min":0.76300000000000001,"5min":0.88800000000000001,"15min":2.9950000000000001},"last":0.77200000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.78500000000000003,"15min":0.77900000000000003},"min":{"1min":0.504,"5min":0.504,"15min":0.49099999999999999},"max":{"1min":1.087,"5min":1.109,"15min":1.4239999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.78800000000000003,"15min":0.76700000000000002},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.435},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":1.1200000000000001}]},{"osd":5,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.85099999999999998,"15min":0.83899999999999997},"min":{"1min":0.60999999999999999,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.179,"5min":1.179,"15min":1.4890000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.81499999999999995,"15min":0.82199999999999995},"min":{"1min":0.53700000000000003,"5min":0.52300000000000002,"15min":0.48799999999999999},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.526},"last":0.99199999999999999}]},{"osd":6,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.79100000000000004,"15min":0.78100000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.48599999999999999},"max":{"1min":1.0289999999999999,"5min":1.167,"15min":2.0470000000000002},"last":1.1000000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.82099999999999995,"15min":0.79800000000000004},"min":{"1min":0.57299999999999995,"5min":0.54800000000000004,"15min":0.52500000000000002},"max":{"1min":0.97399999999999998,"5min":1.2609999999999999,"15min":1.524},"last":1.1839999999999999}]},{"osd":7,"last update":"Fri Mar 29 08:30:19 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.79900000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.505},"max":{"1min":1.319,"5min":1.319,"15min":1.319},"last":1.163},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.80100000000000005,"15min":0.79500000000000004},"min":{"1min":0.60599999999999998,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.96999999999999997,"5min":1.214,"15min":2},"last":0.96899999999999997}]}]},{"osd":1,"up_from":15,"seq":64424509597,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2183168,"data_stored":1283761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:30:25 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.44900000000000001,"15min":0.502},"min":{"1min":0.29399999999999998,"5min":0.249,"15min":0.23799999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":4.2539999999999996},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.47399999999999998,"15min":0.53300000000000003},"min":{"1min":0.307,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":4.1909999999999998},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:30:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.49299999999999999,"15min":0.49399999999999999},"min":{"1min":0.32900000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.82899999999999996},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.52600000000000002,"15min":0.52800000000000002},"min":{"1min":0.35199999999999998,"5min":0.35099999999999998,"15min":0.33600000000000002},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.88200000000000001},"last":0.46000000000000002}]},{"osd":3,"last update":"Fri Mar 29 08:30:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.39200000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77400000000000002,"5min":0.88300000000000001,"15min":1.52},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.51100000000000001,"15min":0.52100000000000002},"min":{"1min":0.376,"5min":0.35499999999999998,"15min":0.34100000000000003},"max":{"1min":0.84799999999999998,"5min":0.94499999999999995,"15min":1.331},"last":0.52500000000000002}]},{"osd":4,"last update":"Fri Mar 29 08:30:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.76900000000000002,"15min":0.755},"min":{"1min":0.54200000000000004,"5min":0.46400000000000002,"15min":0.46200000000000002},"max":{"1min":0.92200000000000004,"5min":1.157,"15min":1.157},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.76600000000000001,"15min":0.76800000000000002},"min":{"1min":0.54900000000000004,"5min":0.502,"15min":0.47999999999999998},"max":{"1min":1.0109999999999999,"5min":1.212,"15min":1.23},"last":0.71899999999999997}]},{"osd":5,"last update":"Fri Mar 29 08:30:25 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.77700000000000002,"15min":0.77300000000000002},"min":{"1min":0.58999999999999997,"5min":0.503,"15min":0.503},"max":{"1min":0.90800000000000003,"5min":1.258,"15min":1.258},"last":0.623},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.77500000000000002,"15min":0.76600000000000001},"min":{"1min":0.56799999999999995,"5min":0.49199999999999999,"15min":0.46500000000000002},"max":{"1min":1.0089999999999999,"5min":1.1879999999999999,"15min":1.2509999999999999},"last":0.69599999999999995}]},{"osd":6,"last update":"Fri Mar 29 08:30:25 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77100000000000002,"15min":0.76900000000000002},"min":{"1min":0.55200000000000005,"5min":0.54900000000000004,"15min":0.5},"max":{"1min":0.97299999999999998,"5min":1.268,"15min":1.268},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.79300000000000004,"15min":0.80900000000000005},"min":{"1min":0.55700000000000005,"5min":0.51800000000000002,"15min":0.51300000000000001},"max":{"1min":1.1040000000000001,"5min":1.22,"15min":1.22},"last":0.67600000000000005}]},{"osd":7,"last update":"Fri Mar 29 08:30:25 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.78600000000000003,"15min":0.79200000000000004},"min":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0900000000000001,"5min":1.2410000000000001,"15min":1.3089999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.79300000000000004,"15min":0.78400000000000003},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.503},"max":{"1min":1.0660000000000001,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.66000000000000003}]}]},{"osd":0,"up_from":15,"seq":64424509596,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32668,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339172,"statfs":{"total":96636764160,"available":96603312128,"internally_reserved":0,"allocated":1589248,"data_stored":693393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.438,"15min":0.48899999999999999},"min":{"1min":0.29099999999999998,"5min":0.27000000000000002,"15min":0.22800000000000001},"max":{"1min":0.55400000000000005,"5min":0.76100000000000001,"15min":3.9359999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.436,"5min":0.48399999999999999,"15min":0.60399999999999998},"min":{"1min":0.28699999999999998,"5min":0.28499999999999998,"15min":0.253},"max":{"1min":0.61699999999999999,"5min":0.77000000000000002,"15min":8.0419999999999998},"last":0.35899999999999999}]},{"osd":2,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.502,"15min":0.51600000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.59299999999999997,"5min":0.78000000000000003,"15min":1.258},"last":0.373},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.52900000000000003,"15min":0.59499999999999997},"min":{"1min":0.33000000000000002,"5min":0.32800000000000001,"15min":0.312},"max":{"1min":0.65900000000000003,"5min":0.81399999999999995,"15min":4.907},"last":0.55800000000000005}]},{"osd":3,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.49399999999999999,"15min":0.60799999999999998},"min":{"1min":0.33200000000000002,"5min":0.30399999999999999,"15min":0.28499999999999998},"max":{"1min":0.69499999999999995,"5min":0.83399999999999996,"15min":7.9569999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.56799999999999995,"15min":0.59299999999999997},"min":{"1min":0.40899999999999997,"5min":0.36099999999999999,"15min":0.34799999999999998},"max":{"1min":0.56699999999999995,"5min":0.87,"15min":1.397},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78700000000000003,"15min":0.84999999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":0.99099999999999999,"5min":1.282,"15min":4.4720000000000004},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.77600000000000002,"15min":0.83299999999999996},"min":{"1min":0.52100000000000002,"5min":0.496,"15min":0.47699999999999998},"max":{"1min":1.04,"5min":1.153,"15min":4.3920000000000003},"last":0.69999999999999996}]},{"osd":5,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79500000000000004,"15min":0.83199999999999996},"min":{"1min":0.56899999999999995,"5min":0.52800000000000002,"15min":0.48699999999999999},"max":{"1min":1.117,"5min":1.24,"15min":4.3200000000000003},"last":0.755},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.77300000000000002,"15min":0.80600000000000005},"min":{"1min":0.59999999999999998,"5min":0.56000000000000005,"15min":0.54600000000000004},"max":{"1min":0.95899999999999996,"5min":1.081,"15min":1.6759999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.79000000000000004,"15min":0.81200000000000006},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.54400000000000004},"max":{"1min":1.0209999999999999,"5min":1.1220000000000001,"15min":1.5920000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.80500000000000005,"15min":0.86099999999999999},"min":{"1min":0.55300000000000005,"5min":0.54200000000000004,"15min":0.51500000000000001},"max":{"1min":1.137,"5min":1.2150000000000001,"15min":4.4100000000000001},"last":0.72299999999999998}]},{"osd":7,"last update":"Fri Mar 29 08:30:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.81200000000000006,"15min":0.871},"min":{"1min":0.59399999999999997,"5min":0.57199999999999995,"15min":0.496},"max":{"1min":1.0609999999999999,"5min":1.1459999999999999,"15min":4.5209999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.79200000000000004,"15min":0.85399999999999998},"min":{"1min":0.58599999999999997,"5min":0.55400000000000005,"15min":0.50900000000000001},"max":{"1min":1.044,"5min":1.3129999999999999,"15min":4.4420000000000002},"last":0.78400000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542301,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33244,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338596,"statfs":{"total":96636764160,"available":96602722304,"internally_reserved":0,"allocated":2183168,"data_stored":1283761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Mar 29 08:30:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68899999999999995,"15min":0.66900000000000004},"min":{"1min":0.48899999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.874,"5min":0.97199999999999998,"15min":0.98499999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.70299999999999996,"15min":0.68500000000000005},"min":{"1min":0.432,"5min":0.42999999999999999,"15min":0.39800000000000002},"max":{"1min":0.93799999999999994,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66900000000000004}]},{"osd":1,"last update":"Fri Mar 29 08:30:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.45400000000000001,"5min":0.42499999999999999,"15min":0.38200000000000001},"max":{"1min":0.90400000000000003,"5min":1.008,"15min":2.5129999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.72399999999999998,"15min":0.70399999999999996},"min":{"1min":0.501,"5min":0.501,"15min":0.40500000000000003},"max":{"1min":0.89100000000000001,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.745}]},{"osd":2,"last update":"Fri Mar 29 08:30:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70799999999999996,"15min":0.68600000000000005},"min":{"1min":0.53000000000000003,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95199999999999996,"5min":0.997,"15min":0.997},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.73299999999999998,"15min":0.71699999999999997},"min":{"1min":0.53600000000000003,"5min":0.47499999999999998,"15min":0.39200000000000002},"max":{"1min":0.89000000000000001,"5min":1.03,"15min":1.03},"last":0.71599999999999997}]},{"osd":3,"last update":"Fri Mar 29 08:30:26 2024","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.73199999999999998,"15min":0.72199999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.038},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.74199999999999999,"15min":0.73399999999999999},"min":{"1min":0.58399999999999996,"5min":0.46700000000000003,"15min":0.435},"max":{"1min":0.91900000000000004,"5min":0.995,"15min":1.954},"last":0.72299999999999998}]},{"osd":4,"last update":"Fri Mar 29 08:30:26 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.47999999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.872,"5min":0.91600000000000004,"15min":0.93500000000000005},"last":0.503},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.48099999999999998,"5min":0.38500000000000001,"15min":0.373},"max":{"1min":0.88200000000000001,"5min":0.89200000000000002,"15min":1.0820000000000001},"last":0.441}]},{"osd":6,"last update":"Fri Mar 29 08:30:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.622,"15min":0.64300000000000002},"min":{"1min":0.44800000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79200000000000004,"5min":0.95999999999999996,"15min":1.071},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.432},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0609999999999999},"last":0.63}]},{"osd":7,"last update":"Fri Mar 29 08:30:26 2024","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.67900000000000005,"15min":0.68999999999999995},"min":{"1min":0.52300000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":1.1200000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.69899999999999995,"15min":0.72299999999999998},"min":{"1min":0.503,"5min":0.42999999999999999,"15min":0.40000000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":1.04},"last":0.60499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-29T08:30:49.333 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-29T08:30:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.472+0000 7f31d8c00640 1 Processor -- start 2024-03-29T08:30:49.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.472+0000 7f31d8c00640 1 -- start start 2024-03-29T08:30:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.472+0000 7f31d8c00640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f31d40cbea0 0x7f31d40ae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.472+0000 7f31d8c00640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d415d260 con 0x7f31d415df90 2024-03-29T08:30:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.472+0000 7f31d8c00640 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31d40aefd0 con 0x7f31d4142a50 2024-03-29T08:30:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.472+0000 7f31d8c00640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31d415d4b0 con 0x7f31d40cbea0 2024-03-29T08:30:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.472+0000 7f31d1400640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f31d415df90 0x7f31d40ae280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55254/0 (socket says 172.21.15.46:55254) 2024-03-29T08:30:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.472+0000 7f31d1400640 1 -- 172.21.15.46:0/1321858931 learned_addr learned my addr 172.21.15.46:0/1321858931 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-03-29T08:30:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.472+0000 7f31d2800640 1 --2- 172.21.15.46:0/1321858931 >> v2:172.21.15.73:3300/0 conn(0x7f31d40cbea0 0x7f31d40ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.472+0000 7f31d0a00640 1 -- 172.21.15.46:0/1321858931 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409537409 0 0) 0x7f31d415d260 con 0x7f31d415df90 2024-03-29T08:30:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.472+0000 7f31d0a00640 1 -- 172.21.15.46:0/1321858931 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31b8003530 con 0x7f31d415df90 2024-03-29T08:30:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.472+0000 7f31d0a00640 1 -- 172.21.15.46:0/1321858931 <== mon.1 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170238203 0 0) 0x7f31d40aefd0 con 0x7f31d4142a50 2024-03-29T08:30:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.472+0000 7f31d0a00640 1 -- 172.21.15.46:0/1321858931 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d415d260 con 0x7f31d4142a50 2024-03-29T08:30:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.472+0000 7f31d0a00640 1 -- 172.21.15.46:0/1321858931 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 568695462 0 0) 0x7f31b8003530 con 0x7f31d415df90 2024-03-29T08:30:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.472+0000 7f31d2800640 1 -- 172.21.15.46:0/1321858931 >> v1:172.21.15.73:6789/0 conn(0x7f31d4142a50 legacy=0x7f31d40cb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.472+0000 7f31d2800640 1 -- 172.21.15.46:0/1321858931 >> v1:172.21.15.46:6789/0 conn(0x7f31d415df90 legacy=0x7f31d40ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:49.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.472+0000 7f31d2800640 1 -- 172.21.15.46:0/1321858931 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d415d790 con 0x7f31d40cbea0 2024-03-29T08:30:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.476+0000 7f31d2800640 1 --2- 172.21.15.46:0/1321858931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d40cbea0 0x7f31d40ae990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009d90 tx=0x7f31cc01c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34f72dd3cbe774e9 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.476+0000 7f31d0a00640 1 -- 172.21.15.46:0/1321858931 <== mon.1 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0001680 con 0x7f31d40cbea0 2024-03-29T08:30:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.476+0000 7f31d0a00640 1 -- 172.21.15.46:0/1321858931 <== mon.1 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31cc0a6450 con 0x7f31d40cbea0 2024-03-29T08:30:49.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.476+0000 7f31d0a00640 1 -- 172.21.15.46:0/1321858931 <== mon.1 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31cc0a67c0 con 0x7f31d40cbea0 2024-03-29T08:30:49.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.476+0000 7f31d8c00640 1 -- 172.21.15.46:0/1321858931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d40cbea0 msgr2=0x7f31d40ae990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.476+0000 7f31d8c00640 1 --2- 172.21.15.46:0/1321858931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d40cbea0 0x7f31d40ae990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f31cc009d90 tx=0x7f31cc01c800 comp rx=0 tx=0).stop 2024-03-29T08:30:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.476+0000 7f31d8c00640 1 -- 172.21.15.46:0/1321858931 shutdown_connections 2024-03-29T08:30:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.476+0000 7f31d8c00640 1 --2- 172.21.15.46:0/1321858931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d40cbea0 0x7f31d40ae990 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.476+0000 7f31d8c00640 1 -- 172.21.15.46:0/1321858931 >> 172.21.15.46:0/1321858931 conn(0x7f31d406a390 msgr2=0x7f31d40cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.476+0000 7f31d8c00640 1 -- 172.21.15.46:0/1321858931 shutdown_connections 2024-03-29T08:30:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.476+0000 7f31d8c00640 1 -- 172.21.15.46:0/1321858931 wait complete. 2024-03-29T08:30:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d8c00640 1 Processor -- start 2024-03-29T08:30:49.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d8c00640 1 -- start start 2024-03-29T08:30:49.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d8c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d40cbea0 0x7f31d413d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d8c00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31d4142a50 0x7f31d4159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d8c00640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31d415df90 0x7f31d413a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d8c00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d415a410 con 0x7f31d4142a50 2024-03-29T08:30:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d8c00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d415a1e0 con 0x7f31d40cbea0 2024-03-29T08:30:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d8c00640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d413a830 con 0x7f31d415df90 2024-03-29T08:30:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31d4142a50 0x7f31d4159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d1400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31d4142a50 0x7f31d4159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36724/0 (socket says 172.21.15.46:36724) 2024-03-29T08:30:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d1400640 1 -- 172.21.15.46:0/4125327394 learned_addr learned my addr 172.21.15.46:0/4125327394 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d1e00640 1 --2- 172.21.15.46:0/4125327394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d40cbea0 0x7f31d413d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d2800640 1 --2- 172.21.15.46:0/4125327394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31d415df90 0x7f31d413a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d1400640 1 -- 172.21.15.46:0/4125327394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31d415df90 msgr2=0x7f31d413a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d1400640 1 --2- 172.21.15.46:0/4125327394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31d415df90 0x7f31d413a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d1400640 1 -- 172.21.15.46:0/4125327394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d40cbea0 msgr2=0x7f31d413d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d1400640 1 --2- 172.21.15.46:0/4125327394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d40cbea0 0x7f31d413d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d1400640 1 -- 172.21.15.46:0/4125327394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31cc009a70 con 0x7f31d4142a50 2024-03-29T08:30:49.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d1400640 1 --2- 172.21.15.46:0/4125327394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31d4142a50 0x7f31d4159ca0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f31c000c730 tx=0x7f31c000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:49.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31ca000640 1 -- 172.21.15.46:0/4125327394 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0005ae0 con 0x7f31d4142a50 2024-03-29T08:30:49.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31ca000640 1 -- 172.21.15.46:0/4125327394 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31c0005c40 con 0x7f31d4142a50 2024-03-29T08:30:49.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31ca000640 1 -- 172.21.15.46:0/4125327394 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31c0007070 con 0x7f31d4142a50 2024-03-29T08:30:49.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d8c00640 1 -- 172.21.15.46:0/4125327394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d415bb40 con 0x7f31d4142a50 2024-03-29T08:30:49.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d8c00640 1 -- 172.21.15.46:0/4125327394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d415c100 con 0x7f31d4142a50 2024-03-29T08:30:49.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.480+0000 7f31d8c00640 1 -- 172.21.15.46:0/4125327394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d406ad60 con 0x7f31d4142a50 2024-03-29T08:30:49.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.484+0000 7f31ca000640 1 -- 172.21.15.46:0/4125327394 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f31c00071d0 con 0x7f31d4142a50 2024-03-29T08:30:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.484+0000 7f31ca000640 1 --2- 172.21.15.46:0/4125327394 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f31b40666c0 0x7f31b4068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.484+0000 7f31ca000640 1 -- 172.21.15.46:0/4125327394 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f31c0083560 con 0x7f31d4142a50 2024-03-29T08:30:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.484+0000 7f31d1e00640 1 --2- 172.21.15.46:0/4125327394 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f31b40666c0 0x7f31b4068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.484+0000 7f31d1e00640 1 --2- 172.21.15.46:0/4125327394 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f31b40666c0 0x7f31b4068b80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f31c40011f0 tx=0x7f31c4009040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:49.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.488+0000 7f31ca000640 1 -- 172.21.15.46:0/4125327394 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f31c0050b30 con 0x7f31d4142a50 2024-03-29T08:30:49.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.648+0000 7f31d8c00640 1 -- 172.21.15.46:0/4125327394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f31d4071a70 con 0x7f31d4142a50 2024-03-29T08:30:49.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.652+0000 7f31ca000640 1 -- 172.21.15.46:0/4125327394 <== mon.0 v2:172.21.15.46:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f31c0001970 con 0x7f31d4142a50 2024-03-29T08:30:49.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.652+0000 7f31ca000640 1 -- 172.21.15.46:0/4125327394 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) v1 ==== 123+0+0 (secure 0 0 0) 0x7f31c00170c0 con 0x7f31d4142a50 2024-03-29T08:30:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.656+0000 7f31d8c00640 1 -- 172.21.15.46:0/4125327394 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f31b40666c0 msgr2=0x7f31b4068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.656+0000 7f31d8c00640 1 --2- 172.21.15.46:0/4125327394 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f31b40666c0 0x7f31b4068b80 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f31c40011f0 tx=0x7f31c4009040 comp rx=0 tx=0).stop 2024-03-29T08:30:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.656+0000 7f31d8c00640 1 -- 172.21.15.46:0/4125327394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31d4142a50 msgr2=0x7f31d4159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.656+0000 7f31d8c00640 1 --2- 172.21.15.46:0/4125327394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31d4142a50 0x7f31d4159ca0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f31c000c730 tx=0x7f31c000cbf0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.656+0000 7f31d8c00640 1 -- 172.21.15.46:0/4125327394 shutdown_connections 2024-03-29T08:30:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.656+0000 7f31d8c00640 1 --2- 172.21.15.46:0/4125327394 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f31b40666c0 0x7f31b4068b80 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.656+0000 7f31d8c00640 1 --2- 172.21.15.46:0/4125327394 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f31d415df90 0x7f31d413a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.656+0000 7f31d8c00640 1 --2- 172.21.15.46:0/4125327394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31d4142a50 0x7f31d4159ca0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.656+0000 7f31d8c00640 1 --2- 172.21.15.46:0/4125327394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31d40cbea0 0x7f31d413d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.656+0000 7f31d8c00640 1 -- 172.21.15.46:0/4125327394 >> 172.21.15.46:0/4125327394 conn(0x7f31d406a390 msgr2=0x7f31d40ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:49.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.656+0000 7f31d8c00640 1 -- 172.21.15.46:0/4125327394 shutdown_connections 2024-03-29T08:30:49.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.656+0000 7f31d8c00640 1 -- 172.21.15.46:0/4125327394 wait complete. 2024-03-29T08:30:49.677 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-29T08:30:49.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-29T08:30:49.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75c3000640 1 Processor -- start 2024-03-29T08:30:49.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75c3000640 1 -- start start 2024-03-29T08:30:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75c3000640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f75bc153970 0x7f75bc1611e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75c3000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75bc153d50 0x7f75bc15cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75c3000640 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f75bc0ad3e0 0x7f75bc0ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75c3000640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75bc0addb0 con 0x7f75bc153970 2024-03-29T08:30:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75c3000640 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75bc15d870 con 0x7f75bc0ad3e0 2024-03-29T08:30:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75c3000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc15db50 con 0x7f75bc153d50 2024-03-29T08:30:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75bbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f75bc153970 0x7f75bc1611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75bbe00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f75bc153970 0x7f75bc1611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36732/0 (socket says 172.21.15.46:36732) 2024-03-29T08:30:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75bbe00640 1 -- 172.21.15.46:0/3851601785 learned_addr learned my addr 172.21.15.46:0/3851601785 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75c1000640 1 --2- 172.21.15.46:0/3851601785 >> v2:172.21.15.73:3300/0 conn(0x7f75bc0ad3e0 0x7f75bc0ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75b3400640 1 --2- 172.21.15.46:0/3851601785 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75bc153d50 0x7f75bc15cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75bbe00640 1 -- 172.21.15.46:0/3851601785 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75bc153d50 msgr2=0x7f75bc15cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75bbe00640 1 --2- 172.21.15.46:0/3851601785 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75bc153d50 0x7f75bc15cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75bbe00640 1 -- 172.21.15.46:0/3851601785 >> v2:172.21.15.73:3300/0 conn(0x7f75bc0ad3e0 msgr2=0x7f75bc0ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-29T08:30:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75bbe00640 1 --2- 172.21.15.46:0/3851601785 >> v2:172.21.15.73:3300/0 conn(0x7f75bc0ad3e0 0x7f75bc0ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75bbe00640 1 -- 172.21.15.46:0/3851601785 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc15de30 con 0x7f75bc153970 2024-03-29T08:30:49.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.816+0000 7f75c1000640 1 --2- 172.21.15.46:0/3851601785 >> v2:172.21.15.73:3300/0 conn(0x7f75bc0ad3e0 0x7f75bc0ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-29T08:30:49.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.820+0000 7f75bbe00640 1 --2- 172.21.15.46:0/3851601785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75bc153970 0x7f75bc1611e0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0cf430 tx=0x7f75ac01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de5b16e0e31ea4c server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.820+0000 7f75bb400640 1 -- 172.21.15.46:0/3851601785 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac0ab650 con 0x7f75bc153970 2024-03-29T08:30:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.820+0000 7f75bb400640 1 -- 172.21.15.46:0/3851601785 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f75ac0ab7b0 con 0x7f75bc153970 2024-03-29T08:30:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.820+0000 7f75bb400640 1 -- 172.21.15.46:0/3851601785 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac0abb60 con 0x7f75bc153970 2024-03-29T08:30:49.822 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.820+0000 7f75c3000640 1 -- 172.21.15.46:0/3851601785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75bc153970 msgr2=0x7f75bc1611e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.820+0000 7f75c3000640 1 --2- 172.21.15.46:0/3851601785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75bc153970 0x7f75bc1611e0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0cf430 tx=0x7f75ac01c4f0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.820+0000 7f75c3000640 1 -- 172.21.15.46:0/3851601785 shutdown_connections 2024-03-29T08:30:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.820+0000 7f75c3000640 1 --2- 172.21.15.46:0/3851601785 >> v2:172.21.15.73:3300/0 conn(0x7f75bc0ad3e0 0x7f75bc0ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.820+0000 7f75c3000640 1 --2- 172.21.15.46:0/3851601785 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75bc153d50 0x7f75bc15cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.820+0000 7f75c3000640 1 --2- 172.21.15.46:0/3851601785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75bc153970 0x7f75bc1611e0 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.820+0000 7f75c3000640 1 -- 172.21.15.46:0/3851601785 >> 172.21.15.46:0/3851601785 conn(0x7f75bc066110 msgr2=0x7f75bc069d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.820+0000 7f75c3000640 1 -- 172.21.15.46:0/3851601785 shutdown_connections 2024-03-29T08:30:49.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.820+0000 7f75c3000640 1 -- 172.21.15.46:0/3851601785 wait complete. 2024-03-29T08:30:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75c3000640 1 Processor -- start 2024-03-29T08:30:49.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75c3000640 1 -- start start 2024-03-29T08:30:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75c3000640 1 --2- >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75bc0ad3e0 0x7f75bc151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75c3000640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75bc153970 0x7f75bc0b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75c3000640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75bc1501e0 0x7f75bc148860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75c3000640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc153270 con 0x7f75bc153970 2024-03-29T08:30:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75c3000640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc14b0f0 con 0x7f75bc1501e0 2024-03-29T08:30:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75c3000640 1 -- --> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc152660 con 0x7f75bc0ad3e0 2024-03-29T08:30:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75bc153970 0x7f75bc0b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75b3400640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75bc153970 0x7f75bc0b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36736/0 (socket says 172.21.15.46:36736) 2024-03-29T08:30:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75b3400640 1 -- 172.21.15.46:0/1502097831 learned_addr learned my addr 172.21.15.46:0/1502097831 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-03-29T08:30:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75c1000640 1 --2- 172.21.15.46:0/1502097831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75bc1501e0 0x7f75bc148860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75bbe00640 1 --2- 172.21.15.46:0/1502097831 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75bc0ad3e0 0x7f75bc151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75b3400640 1 -- 172.21.15.46:0/1502097831 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75bc0ad3e0 msgr2=0x7f75bc151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75b3400640 1 --2- 172.21.15.46:0/1502097831 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75bc0ad3e0 0x7f75bc151fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75b3400640 1 -- 172.21.15.46:0/1502097831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75bc1501e0 msgr2=0x7f75bc148860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75b3400640 1 --2- 172.21.15.46:0/1502097831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75bc1501e0 0x7f75bc148860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75b3400640 1 -- 172.21.15.46:0/1502097831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ac0094d0 con 0x7f75bc153970 2024-03-29T08:30:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75c1000640 1 --2- 172.21.15.46:0/1502097831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75bc1501e0 0x7f75bc148860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-29T08:30:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75b3400640 1 --2- 172.21.15.46:0/1502097831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75bc153970 0x7f75bc0b0860 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f75a400d800 tx=0x7f75a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75b8c00640 1 -- 172.21.15.46:0/1502097831 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a400b6c0 con 0x7f75bc153970 2024-03-29T08:30:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75b8c00640 1 -- 172.21.15.46:0/1502097831 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f75a400b820 con 0x7f75bc153970 2024-03-29T08:30:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75b8c00640 1 -- 172.21.15.46:0/1502097831 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a400bbb0 con 0x7f75bc153970 2024-03-29T08:30:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75c3000640 1 -- 172.21.15.46:0/1502097831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75bc152940 con 0x7f75bc153970 2024-03-29T08:30:49.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.824+0000 7f75c3000640 1 -- 172.21.15.46:0/1502097831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75bc152f00 con 0x7f75bc153970 2024-03-29T08:30:49.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.828+0000 7f75c3000640 1 -- 172.21.15.46:0/1502097831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75bc153420 con 0x7f75bc153970 2024-03-29T08:30:49.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.828+0000 7f75b8c00640 1 -- 172.21.15.46:0/1502097831 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 86084+0+0 (secure 0 0 0) 0x7f75a400bd10 con 0x7f75bc153970 2024-03-29T08:30:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.828+0000 7f75b8c00640 1 --2- 172.21.15.46:0/1502097831 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7598066690 0x7f7598068b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-29T08:30:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.828+0000 7f75b8c00640 1 -- 172.21.15.46:0/1502097831 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f75a40844d0 con 0x7f75bc153970 2024-03-29T08:30:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.828+0000 7f75bbe00640 1 --2- 172.21.15.46:0/1502097831 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7598066690 0x7f7598068b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-29T08:30:49.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.828+0000 7f75bbe00640 1 --2- 172.21.15.46:0/1502097831 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7598066690 0x7f7598068b50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f75ac0af000 tx=0x7f75ac0b6040 comp rx=0 tx=0).ready entity=mgr.4114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-29T08:30:49.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:49.832+0000 7f75b8c00640 1 -- 172.21.15.46:0/1502097831 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+174438 (secure 0 0 0) 0x7f75bc153420 con 0x7f75bc153970 2024-03-29T08:30:50.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:50.024+0000 7f75c3000640 1 -- 172.21.15.46:0/1502097831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75bc153420 con 0x7f75bc153970 2024-03-29T08:30:50.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:50.024+0000 7f75b8c00640 1 -- 172.21.15.46:0/1502097831 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+3531 (secure 0 0 0) 0x7f75bc153420 con 0x7f75bc153970 2024-03-29T08:30:50.027 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-03-29T08:30:50.027 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":109,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7631,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2506132630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2506132630},{"type":"v1","addr":"172.21.15.73:6836","nonce":2506132630}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":8566,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2108115501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2108115501},{"type":"v1","addr":"172.21.15.73:6837","nonce":2108115501}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":9045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/3210062895","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3210062895},{"type":"v1","addr":"172.21.15.46:6833","nonce":3210062895}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":88},{"gid":9879,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/4210760407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":4210760407},{"type":"v1","addr":"172.21.15.46:6835","nonce":4210760407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":109}],"filesystems":[]} 2024-03-29T08:30:50.028 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 109 2024-03-29T08:30:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:50.028+0000 7f75c3000640 1 -- 172.21.15.46:0/1502097831 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7598066690 msgr2=0x7f7598068b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:50.028+0000 7f75c3000640 1 --2- 172.21.15.46:0/1502097831 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7598066690 0x7f7598068b50 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f75ac0af000 tx=0x7f75ac0b6040 comp rx=0 tx=0).stop 2024-03-29T08:30:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:50.028+0000 7f75c3000640 1 -- 172.21.15.46:0/1502097831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75bc153970 msgr2=0x7f75bc0b0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-29T08:30:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:50.028+0000 7f75c3000640 1 --2- 172.21.15.46:0/1502097831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75bc153970 0x7f75bc0b0860 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f75a400d800 tx=0x7f75a400dcc0 comp rx=0 tx=0).stop 2024-03-29T08:30:50.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:50.028+0000 7f75c3000640 1 -- 172.21.15.46:0/1502097831 shutdown_connections 2024-03-29T08:30:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:50.028+0000 7f75c3000640 1 --2- 172.21.15.46:0/1502097831 >> [v2:172.21.15.73:6832/27375,v1:172.21.15.73:6833/27375] conn(0x7f7598066690 0x7f7598068b50 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:50.028+0000 7f75c3000640 1 --2- 172.21.15.46:0/1502097831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75bc1501e0 0x7f75bc148860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:50.028+0000 7f75c3000640 1 --2- 172.21.15.46:0/1502097831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75bc153970 0x7f75bc0b0860 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:50.028+0000 7f75c3000640 1 --2- 172.21.15.46:0/1502097831 >> [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] conn(0x7f75bc0ad3e0 0x7f75bc151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-29T08:30:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:50.028+0000 7f75c3000640 1 -- 172.21.15.46:0/1502097831 >> 172.21.15.46:0/1502097831 conn(0x7f75bc066110 msgr2=0x7f75bc15f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-29T08:30:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:50.028+0000 7f75c3000640 1 -- 172.21.15.46:0/1502097831 shutdown_connections 2024-03-29T08:30:50.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-03-29T08:30:50.028+0000 7f75c3000640 1 -- 172.21.15.46:0/1502097831 wait complete. 2024-03-29T08:30:50.049 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-29T08:30:50.049 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-29T08:30:50.049 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-29T08:30:50.049 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:30:50.060 INFO:tasks.ceph.mds.a:Stopped 2024-03-29T08:30:50.061 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-03-29T08:30:50.061 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:30:50.120 INFO:tasks.ceph.mds.c:Stopped 2024-03-29T08:30:50.120 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-29T08:30:50.120 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:30:50.132 INFO:tasks.ceph.mds.b:Stopped 2024-03-29T08:30:50.132 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-03-29T08:30:50.132 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:30:50.187 INFO:tasks.ceph.mds.d:Stopped 2024-03-29T08:30:50.187 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-29T08:30:50.188 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-29T08:30:50.188 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:30:50.255 INFO:tasks.ceph.osd.0:Stopped 2024-03-29T08:30:50.256 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-29T08:30:50.256 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:30:50.337 INFO:tasks.ceph.osd.1:Stopped 2024-03-29T08:30:50.337 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-29T08:30:50.337 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:30:50.427 INFO:tasks.ceph.osd.2:Stopped 2024-03-29T08:30:50.428 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-29T08:30:50.428 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:30:56.529 INFO:tasks.ceph.osd.3:Stopped 2024-03-29T08:30:56.530 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-29T08:30:56.530 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:30:56.604 INFO:tasks.ceph.osd.4:Stopped 2024-03-29T08:30:56.605 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-29T08:30:56.605 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:30:56.657 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-03-29T08:30:56.697 INFO:tasks.ceph.osd.5:Stopped 2024-03-29T08:30:56.698 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-29T08:30:56.698 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:30:56.772 INFO:tasks.ceph.osd.6:Stopped 2024-03-29T08:30:56.772 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-29T08:30:56.772 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:30:56.839 INFO:tasks.ceph.osd.7:Stopped 2024-03-29T08:30:56.839 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-29T08:30:56.839 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-29T08:30:56.839 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:30:56.895 INFO:tasks.ceph.mgr.y:Stopped 2024-03-29T08:30:56.896 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-29T08:30:56.896 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:30:56.952 INFO:tasks.ceph.mgr.x:Stopped 2024-03-29T08:30:56.953 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-29T08:30:56.953 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-29T08:30:56.953 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:30:56.974 INFO:tasks.ceph.mon.a:Stopped 2024-03-29T08:30:56.974 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-29T08:30:56.974 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:30:56.989 INFO:tasks.ceph.mon.b:Stopped 2024-03-29T08:30:56.990 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-29T08:30:56.990 INFO:teuthology.orchestra.run:waiting for 300 2024-03-29T08:30:57.046 INFO:tasks.ceph.mon.c:Stopped 2024-03-29T08:30:57.046 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-29T08:30:57.047 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-03-29T08:30:57.080 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:23:09.266194+0000 mds.c (mds.0) 1 : cluster [WRN] Corrupt dentry 'file_to_be_damaged' in dir frag 0x10000000000: Invalid tag char 'd' pos 0: Malformed input [buffer:3](/subdir) 2024-03-29T08:30:57.081 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2024-03-29T08:30:57.081 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-03-29T08:30:57.157 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-03-29T08:30:57.235 INFO:teuthology.orchestra.run.smithi046.stdout:2024-03-29T08:25:09.366379+0000 mds.c (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-03-29T08:30:57.236 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:30:57.236 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-29T08:30:57.560 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:30:57.561 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-29T08:30:57.768 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:30:57.768 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-29T08:30:58.044 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:30:58.044 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-29T08:30:58.202 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:30:58.203 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-29T08:30:58.455 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:30:58.456 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-29T08:30:58.653 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:30:58.654 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-29T08:30:58.817 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:30:58.817 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-29T08:30:58.959 INFO:tasks.ceph:Archiving mon data... 2024-03-29T08:30:58.960 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628113/data/mon.a.tgz 2024-03-29T08:30:58.960 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-03-29T08:30:58.970 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.CMfyzzpt8p 2024-03-29T08:30:58.970 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.CMfyzzpt8p 2024-03-29T08:30:59.116 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.CMfyzzpt8p 2024-03-29T08:30:59.192 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.CMfyzzpt8p is 409KB 2024-03-29T08:30:59.262 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.CMfyzzpt8p 2024-03-29T08:30:59.271 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628113/data/mon.b.tgz 2024-03-29T08:30:59.271 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2024-03-29T08:30:59.280 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.6QjGoKR88K 2024-03-29T08:30:59.280 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.6QjGoKR88K 2024-03-29T08:30:59.426 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.6QjGoKR88K 2024-03-29T08:30:59.502 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.6QjGoKR88K is 421KB 2024-03-29T08:30:59.565 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.6QjGoKR88K 2024-03-29T08:30:59.573 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628113/data/mon.c.tgz 2024-03-29T08:30:59.573 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2024-03-29T08:30:59.621 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.Pm6D7Z12Y8 2024-03-29T08:30:59.622 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Pm6D7Z12Y8 2024-03-29T08:30:59.760 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.Pm6D7Z12Y8 2024-03-29T08:30:59.838 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.Pm6D7Z12Y8 is 421KB 2024-03-29T08:30:59.902 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.Pm6D7Z12Y8 2024-03-29T08:30:59.910 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-29T08:30:59.910 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-29T08:30:59.914 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-29T08:30:59.973 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-03-29T08:30:59.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-03-29T08:30:59.978 DEBUG:teuthology.orchestra.run.smithi073:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-03-29T08:31:00.006 INFO:teuthology.orchestra.run.smithi046.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-03-29T08:31:00.054 INFO:teuthology.orchestra.run.smithi073.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-03-29T08:31:00.060 INFO:tasks.ceph:Archiving crash dumps... 2024-03-29T08:31:00.062 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628113/remote/smithi046/crash 2024-03-29T08:31:00.063 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-29T08:31:00.085 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628113/remote/smithi073/crash 2024-03-29T08:31:00.086 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-29T08:31:00.123 INFO:tasks.ceph:Shutting down logrotate 2024-03-29T08:31:00.124 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:31:00.130 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-29T08:31:00.224 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-29T08:31:00.245 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-29T08:31:00.263 INFO:tasks.ceph:Compressing logs... 2024-03-29T08:31:00.264 DEBUG:teuthology.orchestra.run.smithi046:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T08:31:00.290 DEBUG:teuthology.orchestra.run.smithi073:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T08:31:01.659 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-03-29T08:31:04.504 INFO:tasks.ceph:Archiving logs... 2024-03-29T08:31:04.504 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628113/remote/smithi046/log 2024-03-29T08:31:04.505 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-29T08:31:05.481 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628113/remote/smithi073/log 2024-03-29T08:31:05.482 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-29T08:31:05.846 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-29T08:31:05.856 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-29T08:31:05.856 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-29T08:31:05.859 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-29T08:31:05.949 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-29T08:31:05.958 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-29T08:31:05.959 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T08:31:05.962 DEBUG:teuthology.orchestra.run.smithi073:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-29T08:31:05.975 INFO:teuthology.orchestra.run.smithi046.stdout: remote refid st t when poll reach delay offset jitter 2024-03-29T08:31:05.975 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================== 2024-03-29T08:31:05.975 INFO:teuthology.orchestra.run.smithi046.stdout:*hv01.front.sepi 44.4.53.2 2 u 29 64 377 0.090 -1.527 0.277 2024-03-29T08:31:05.975 INFO:teuthology.orchestra.run.smithi046.stdout:+hv02.front.sepi 63.231.80.2 3 u 35 64 377 0.062 -0.520 0.273 2024-03-29T08:31:05.975 INFO:teuthology.orchestra.run.smithi046.stdout:+hv03.front.sepi 74.6.168.72 3 u 37 64 377 0.097 -1.804 0.299 2024-03-29T08:31:05.975 INFO:teuthology.orchestra.run.smithi046.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-03-29T08:31:05.978 INFO:teuthology.orchestra.run.smithi073.stdout: remote refid st t when poll reach delay offset jitter 2024-03-29T08:31:05.978 INFO:teuthology.orchestra.run.smithi073.stdout:============================================================================== 2024-03-29T08:31:05.978 INFO:teuthology.orchestra.run.smithi073.stdout:*hv01.front.sepi 44.4.53.2 2 u 34 64 377 0.075 -6.567 2.646 2024-03-29T08:31:05.978 INFO:teuthology.orchestra.run.smithi073.stdout:+hv02.front.sepi 63.231.80.2 3 u 36 64 377 0.091 -3.719 1.781 2024-03-29T08:31:05.978 INFO:teuthology.orchestra.run.smithi073.stdout:+hv03.front.sepi 74.6.168.72 3 u 32 64 377 0.081 -6.349 2.269 2024-03-29T08:31:05.978 INFO:teuthology.orchestra.run.smithi073.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-03-29T08:31:05.979 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-29T08:31:05.989 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-29T08:31:05.989 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-29T08:31:05.999 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-29T08:31:06.009 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-29T08:31:06.017 INFO:teuthology.task.internal:Duration was 1489.085825 seconds 2024-03-29T08:31:06.018 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-29T08:31:06.027 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-29T08:31:06.027 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-29T08:31:06.031 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-29T08:31:06.117 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-29T08:31:06.117 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-03-29T08:31:06.118 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-29T08:31:06.138 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi073.front.sepia.ceph.com 2024-03-29T08:31:06.139 DEBUG:teuthology.orchestra.run.smithi073:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-29T08:31:06.160 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-29T08:31:06.160 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T08:31:06.186 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-29T08:31:06.241 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-29T08:31:06.241 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-29T08:31:06.274 DEBUG:teuthology.orchestra.run.smithi073:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-29T08:31:06.364 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-29T08:31:06.374 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-29T08:31:06.374 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-29T08:31:06.394 DEBUG:teuthology.orchestra.run.smithi073:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-29T08:31:06.414 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-29T08:31:06.424 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-29T08:31:06.438 DEBUG:teuthology.orchestra.run.smithi073:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-29T08:31:06.452 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-03-29T08:31:06.475 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = core 2024-03-29T08:31:06.493 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-29T08:31:06.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:31:06.518 DEBUG:teuthology.orchestra.run.smithi073:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-29T08:31:06.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-29T08:31:06.545 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-29T08:31:06.556 INFO:teuthology.task.internal:Transferring archived files... 2024-03-29T08:31:06.556 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628113/remote/smithi046 2024-03-29T08:31:06.557 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-29T08:31:06.586 DEBUG:teuthology.misc:Transferring archived files from smithi073:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628113/remote/smithi073 2024-03-29T08:31:06.586 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-29T08:31:06.611 INFO:teuthology.task.internal:Removing archive directory... 2024-03-29T08:31:06.611 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-29T08:31:06.626 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-29T08:31:06.662 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-29T08:31:06.673 INFO:teuthology.task.internal:Not uploading archives. 2024-03-29T08:31:06.673 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-29T08:31:06.715 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-29T08:31:06.715 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-29T08:31:06.718 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-29T08:31:06.723 INFO:teuthology.orchestra.run.smithi046.stdout: 658116 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 29 08:31 /home/ubuntu/cephtest 2024-03-29T08:31:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: 658116 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 29 08:31 /home/ubuntu/cephtest 2024-03-29T08:31:06.728 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-29T08:31:06.739 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-29T08:31:06.759 INFO:teuthology.nuke:Checking targets against current locks 2024-03-29T08:31:06.787 DEBUG:teuthology.nuke:shortname: smithi046 2024-03-29T08:31:06.787 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T08:31:06.809 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628113', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 07:55:22.108927', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyO35H5HPJGFuhHNFtjtIjlv5ApnHunOmuTYxW+bNF+3/oX9wx9k2O2ciaYaKvoLd7IlwtsfjApdTCDY5sDlsM='} 2024-03-29T08:31:06.812 DEBUG:teuthology.nuke:shortname: smithi073 2024-03-29T08:31:06.813 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-29T08:31:06.839 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-28_11:21:59-fs-wip-vshankar-testing-20240326.105515-reef-testing-default-smithi/7628113', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-29 07:55:22.109895', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIfgdX6RoDeC9CUYoUy1+4Ab439iXukfjJUOSMT3kXSQs1Ev1THOAgjRZ7iAskt1D6wDonDlIfV1FXULpPxnirY='} 2024-03-29T08:31:06.875 INFO:teuthology.orchestra.console.smithi046:Power off 2024-03-29T08:31:06.875 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-29T08:31:06.898 DEBUG:teuthology.orchestra.console.smithi046:power off output: Chassis Power Control: Down/Off 2024-03-29T08:31:06.898 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T08:31:06.915 INFO:teuthology.orchestra.console.smithi073:Power off 2024-03-29T08:31:06.915 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-29T08:31:06.924 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2024-03-29T08:31:06.937 DEBUG:teuthology.orchestra.console.smithi073:power off output: Chassis Power Control: Down/Off 2024-03-29T08:31:06.937 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T08:31:06.957 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is on 2024-03-29T08:31:10.926 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T08:31:10.958 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T08:31:11.039 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2024-03-29T08:31:11.068 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is on 2024-03-29T08:31:15.045 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T08:31:15.069 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-29T08:31:15.156 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off 2024-03-29T08:31:15.179 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is off 2024-03-29T08:31:15.257 INFO:teuthology.orchestra.console.smithi046:Power off completed 2024-03-29T08:31:15.280 INFO:teuthology.orchestra.console.smithi073:Power off completed 2024-03-29T08:31:15.385 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/forward-scrub} duration: 1489.085824728012 failure_reason: '"2024-03-29T08:25:09.366379+0000 mds.c (mds.0) 1 : cluster [ERR] dir 0x10000000000 object missing on disk; some files may be lost (/dir)" in cluster log' flavor: default mon.a-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 mon.b-kernel-sha1: 1df0c7e8a2354c42d738e5454ba83ea7332deac2 owner: scheduled_vshankar@teuthology success: false 2024-03-29T08:31:15.385 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-29T08:31:15.476 INFO:teuthology.run:FAIL